2024-03-21T09:41:20.557 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-21T09:41:20.557 DEBUG:teuthology.run:Teuthology command: teuthology --name vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi --archive /home/teuthworker/archive/vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi/7612201 --verbose --description fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/reef/{v18.2.1} 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} --owner scheduled_vshankar@teuthology -- /home/teuthworker/archive/vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi/7612201/orig.config.yaml 2024-03-21T09:41:20.611 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-21T09:41:20.696 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi/7612201 branch: wip-vshankar-testing-20240320.053002 description: fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/reef/{v18.2.1} 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} email: vshankar@redhat.com first_in_suite: false job_id: '7612201' kernel: flavor: default kdb: true sha1: 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 last_in_suite: false machine_type: smithi meta: - desc: 'setup ceph/v18.2.1 ' name: vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi no_nested_subset: false nuke-on-error: true os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-vshankar-testing-20240320.053002 ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: mon pg warn min per osd: 0 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - OSD_DOWN - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN sha1: 2edfee68ad7243bed422a049b6f5f58619300f11 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 2edfee68ad7243bed422a049b6f5f58619300f11 kclient: syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-vshankar-testing-20240320.053002 sha1: 2edfee68ad7243bed422a049b6f5f58619300f11 owner: scheduled_vshankar@teuthology priority: 100 repo: https://git.ceph.com/ceph-ci.git roles: - - host.a - client.0 - osd.0 - osd.1 - osd.2 - - host.b - client.1 - osd.3 - osd.4 - osd.5 seed: 8073 sha1: 2edfee68ad7243bed422a049b6f5f58619300f11 sleep_before_teardown: 0 subset: 203/1024 suite: fs suite_branch: wip-vshankar-testing-20240320.053002 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_2edfee68ad7243bed422a049b6f5f58619300f11/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 2edfee68ad7243bed422a049b6f5f58619300f11 targets: smithi153.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEK+pZRh8C5g97Z5EKqiwVZN1TgOhSP+anuzhpn57soIsbFFu2yJZ61lZNaSxEA7pqBweuQJEQNVej1INKNuN0M= smithi174.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBK9SuABKKZw4eM/oCdrw27HqA0dR2KfwELS4U22iliOPzYjSRZHyVzTcbup4BjtRM4QJfydoBH6WOH7s5E9o2xo= tasks: - install: exclude_packages: - ceph-volume tag: v18.2.1 - print: '**** done install task...' - cephadm: compiled_cephadm_branch: reef conf: osd: osd_class_default_list: '*' osd_class_load_list: '*' image: quay.io/ceph/ceph:v18.2.1 roleless: true - print: '**** done end installing v18.2.1 cephadm ...' - cephadm.shell: host.a: - ceph config set mgr mgr/cephadm/use_repo_digest true --force - print: '**** done cephadm.shell ceph config set mgr...' - cephadm.shell: host.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: host.a: - ceph fs volume create cephfs --placement=4 - ceph fs dump - cephadm.shell: host.a: - ceph fs set cephfs max_mds 2 - cephadm.shell: host.a: - ceph fs set cephfs allow_standby_replay true - cephadm.shell: host.a: - ceph fs set cephfs inline_data true --yes-i-really-really-mean-it - cephadm.shell: host.a: - ceph fs dump - ceph --format=json fs dump | jq -e ".filesystems | length == 1" - while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done - fs.pre_upgrade_save: null - ceph-fuse: null - print: '**** done client' - parallel: - upgrade-tasks - workload-tasks - cephadm.shell: host.a: - ceph fs dump - fs.post_upgrade_checks: null teuthology: fragments_dropped: [] meta: {} postmerge: [] variables: fail_fs: true teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-03-20_11:21:18 tube: smithi upgrade-tasks: sequential: - cephadm.shell: env: - sha1 host.a: - ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force - ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force - ceph config set global log_to_journald false --force - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 --daemon-types mgr - while ceph orch upgrade status | jq '.in_progress' | grep true && ! ceph orch upgrade status | jq '.message' | grep Error ; do ceph orch ps ; ceph versions ; ceph orch upgrade status ; sleep 30 ; done - ceph versions | jq -e '.mgr | length == 1' - ceph versions | jq -e '.mgr | keys' | grep $sha1 - ceph versions | jq -e '.overall | length == 2' - ceph orch upgrade check quay.ceph.io/ceph-ci/ceph:$sha1 | jq -e '.up_to_date | length == 2' - ceph orch ps - cephadm.shell: env: - sha1 host.a: - ceph config set mgr mgr/orchestrator/fail_fs true - cephadm.shell: env: - sha1 host.a: - ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force - ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force - ceph config set global log_to_journald false --force - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 - cephadm.shell: env: - sha1 host.a: - while ceph orch upgrade status | jq '.in_progress' | grep true && ! ceph orch upgrade status | jq '.message' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done - ceph orch ps - ceph orch upgrade status - ceph health detail - ceph versions - echo "wait for servicemap items w/ changing names to refresh" - sleep 60 - ceph orch ps - ceph versions - ceph versions | jq -e '.overall | length == 1' - ceph versions | jq -e '.overall | keys' | grep $sha1 user: vshankar verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 workload-tasks: sequential: - workunit: clients: all: - suites/fsstress.sh 2024-03-21T09:41:20.697 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_2edfee68ad7243bed422a049b6f5f58619300f11/qa; will attempt to use it 2024-03-21T09:41:20.697 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_2edfee68ad7243bed422a049b6f5f58619300f11/qa/tasks 2024-03-21T09:41:20.698 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-03-21T09:41:20.699 INFO:teuthology.task.internal:Checking packages... 2024-03-21T09:41:20.724 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '2edfee68ad7243bed422a049b6f5f58619300f11' 2024-03-21T09:41:20.724 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-21T09:41:20.724 INFO:teuthology.packaging:ref: None 2024-03-21T09:41:20.724 INFO:teuthology.packaging:tag: None 2024-03-21T09:41:20.724 INFO:teuthology.packaging:branch: wip-vshankar-testing-20240320.053002 2024-03-21T09:41:20.724 INFO:teuthology.packaging:sha1: 2edfee68ad7243bed422a049b6f5f58619300f11 2024-03-21T09:41:20.724 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-vshankar-testing-20240320.053002 2024-03-21T09:41:20.864 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2231.g2edfee68 2024-03-21T09:41:20.865 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-03-21T09:41:20.871 INFO:teuthology.task.internal:no buildpackages task found 2024-03-21T09:41:20.871 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-03-21T09:41:20.891 INFO:teuthology.task.internal:Saving configuration 2024-03-21T09:41:20.914 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-03-21T09:41:20.921 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-21T09:41:20.946 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi153.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi/7612201', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-21 09:35:50.188678', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEK+pZRh8C5g97Z5EKqiwVZN1TgOhSP+anuzhpn57soIsbFFu2yJZ61lZNaSxEA7pqBweuQJEQNVej1INKNuN0M='} 2024-03-21T09:41:20.967 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi174.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi/7612201', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-21 09:35:50.190512', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBK9SuABKKZw4eM/oCdrw27HqA0dR2KfwELS4U22iliOPzYjSRZHyVzTcbup4BjtRM4QJfydoBH6WOH7s5E9o2xo='} 2024-03-21T09:41:20.967 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-03-21T09:41:20.972 INFO:teuthology.task.internal:roles: ubuntu@smithi153.front.sepia.ceph.com - ['host.a', 'client.0', 'osd.0', 'osd.1', 'osd.2'] 2024-03-21T09:41:20.972 INFO:teuthology.task.internal:roles: ubuntu@smithi174.front.sepia.ceph.com - ['host.b', 'client.1', 'osd.3', 'osd.4', 'osd.5'] 2024-03-21T09:41:20.972 INFO:teuthology.run_tasks:Running task console_log... 2024-03-21T09:41:21.057 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fda3b316160>, signals=[15]) 2024-03-21T09:41:21.058 INFO:teuthology.run_tasks:Running task internal.connect... 2024-03-21T09:41:21.063 INFO:teuthology.task.internal:Opening connections... 2024-03-21T09:41:21.063 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi153.front.sepia.ceph.com 2024-03-21T09:41:21.066 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi153.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T09:41:21.156 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi174.front.sepia.ceph.com 2024-03-21T09:41:21.157 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T09:41:21.247 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-03-21T09:41:21.254 DEBUG:teuthology.orchestra.run.smithi153:> uname -m 2024-03-21T09:41:21.271 INFO:teuthology.orchestra.run.smithi153.stdout:x86_64 2024-03-21T09:41:21.271 DEBUG:teuthology.orchestra.run.smithi153:> cat /etc/os-release 2024-03-21T09:41:21.327 INFO:teuthology.orchestra.run.smithi153.stdout:NAME="CentOS Stream" 2024-03-21T09:41:21.327 INFO:teuthology.orchestra.run.smithi153.stdout:VERSION="9" 2024-03-21T09:41:21.327 INFO:teuthology.orchestra.run.smithi153.stdout:ID="centos" 2024-03-21T09:41:21.327 INFO:teuthology.orchestra.run.smithi153.stdout:ID_LIKE="rhel fedora" 2024-03-21T09:41:21.327 INFO:teuthology.orchestra.run.smithi153.stdout:VERSION_ID="9" 2024-03-21T09:41:21.327 INFO:teuthology.orchestra.run.smithi153.stdout:PLATFORM_ID="platform:el9" 2024-03-21T09:41:21.328 INFO:teuthology.orchestra.run.smithi153.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-21T09:41:21.328 INFO:teuthology.orchestra.run.smithi153.stdout:ANSI_COLOR="0;31" 2024-03-21T09:41:21.328 INFO:teuthology.orchestra.run.smithi153.stdout:LOGO="fedora-logo-icon" 2024-03-21T09:41:21.328 INFO:teuthology.orchestra.run.smithi153.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-21T09:41:21.328 INFO:teuthology.orchestra.run.smithi153.stdout:HOME_URL="https://centos.org/" 2024-03-21T09:41:21.328 INFO:teuthology.orchestra.run.smithi153.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-21T09:41:21.328 INFO:teuthology.orchestra.run.smithi153.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-21T09:41:21.328 INFO:teuthology.orchestra.run.smithi153.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-21T09:41:21.329 INFO:teuthology.lock.ops:Updating smithi153.front.sepia.ceph.com on lock server 2024-03-21T09:41:21.351 DEBUG:teuthology.orchestra.run.smithi174:> uname -m 2024-03-21T09:41:21.374 INFO:teuthology.orchestra.run.smithi174.stdout:x86_64 2024-03-21T09:41:21.374 DEBUG:teuthology.orchestra.run.smithi174:> cat /etc/os-release 2024-03-21T09:41:21.431 INFO:teuthology.orchestra.run.smithi174.stdout:NAME="CentOS Stream" 2024-03-21T09:41:21.431 INFO:teuthology.orchestra.run.smithi174.stdout:VERSION="9" 2024-03-21T09:41:21.432 INFO:teuthology.orchestra.run.smithi174.stdout:ID="centos" 2024-03-21T09:41:21.432 INFO:teuthology.orchestra.run.smithi174.stdout:ID_LIKE="rhel fedora" 2024-03-21T09:41:21.432 INFO:teuthology.orchestra.run.smithi174.stdout:VERSION_ID="9" 2024-03-21T09:41:21.432 INFO:teuthology.orchestra.run.smithi174.stdout:PLATFORM_ID="platform:el9" 2024-03-21T09:41:21.432 INFO:teuthology.orchestra.run.smithi174.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-21T09:41:21.432 INFO:teuthology.orchestra.run.smithi174.stdout:ANSI_COLOR="0;31" 2024-03-21T09:41:21.432 INFO:teuthology.orchestra.run.smithi174.stdout:LOGO="fedora-logo-icon" 2024-03-21T09:41:21.432 INFO:teuthology.orchestra.run.smithi174.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-21T09:41:21.432 INFO:teuthology.orchestra.run.smithi174.stdout:HOME_URL="https://centos.org/" 2024-03-21T09:41:21.432 INFO:teuthology.orchestra.run.smithi174.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-21T09:41:21.432 INFO:teuthology.orchestra.run.smithi174.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-21T09:41:21.432 INFO:teuthology.orchestra.run.smithi174.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-21T09:41:21.433 INFO:teuthology.lock.ops:Updating smithi174.front.sepia.ceph.com on lock server 2024-03-21T09:41:21.457 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-03-21T09:41:21.482 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-03-21T09:41:21.514 INFO:teuthology.task.internal:Checking for old test directory... 2024-03-21T09:41:21.514 DEBUG:teuthology.orchestra.run.smithi153:> test '!' -e /home/ubuntu/cephtest 2024-03-21T09:41:21.517 DEBUG:teuthology.orchestra.run.smithi174:> test '!' -e /home/ubuntu/cephtest 2024-03-21T09:41:21.533 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-03-21T09:41:21.543 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-03-21T09:41:21.543 DEBUG:teuthology.orchestra.run.smithi153:> test -z $(ls -A /var/lib/ceph) 2024-03-21T09:41:21.575 DEBUG:teuthology.orchestra.run.smithi174:> test -z $(ls -A /var/lib/ceph) 2024-03-21T09:41:21.681 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-03-21T09:41:21.741 INFO:teuthology.run_tasks:Running task kernel... 2024-03-21T09:41:21.762 INFO:teuthology.task.kernel:normalize config orig: {'flavor': 'default', 'kdb': True, 'sha1': '5ec3e6e88b2e2783e09498588bb1f433ae9c8986'} 2024-03-21T09:41:21.762 INFO:teuthology.task.kernel:config {'host.a': {'flavor': 'default', 'kdb': True, 'sha1': '5ec3e6e88b2e2783e09498588bb1f433ae9c8986'}, 'host.b': {'flavor': 'default', 'kdb': True, 'sha1': '5ec3e6e88b2e2783e09498588bb1f433ae9c8986'}}, timeout 300 2024-03-21T09:41:21.763 DEBUG:teuthology.orchestra.run.smithi153:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-21T09:41:21.763 DEBUG:teuthology.orchestra.run.smithi174:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-21T09:41:21.779 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-21T09:41:21.779 DEBUG:teuthology.task.kernel:sha1 for host.a is 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 2024-03-21T09:41:21.780 INFO:teuthology.task.kernel:Checking kernel version of host.a, want "5ec3e6e88b2e2783e09498588bb1f433ae9c8986"... 2024-03-21T09:41:21.780 DEBUG:teuthology.orchestra.run.smithi153:> uname -r 2024-03-21T09:41:21.780 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-21T09:41:21.781 DEBUG:teuthology.task.kernel:sha1 for host.b is 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 2024-03-21T09:41:21.781 INFO:teuthology.task.kernel:Checking kernel version of host.b, want "5ec3e6e88b2e2783e09498588bb1f433ae9c8986"... 2024-03-21T09:41:21.781 DEBUG:teuthology.orchestra.run.smithi174:> uname -r 2024-03-21T09:41:21.837 INFO:teuthology.orchestra.run.smithi153.stdout:5.14.0-361.el9.x86_64 2024-03-21T09:41:21.837 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 2024-03-21T09:41:21.838 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-03-21T09:41:21.838 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5ec3e6e88b2e2783e09498588bb1f433ae9c8986 2024-03-21T09:41:21.842 INFO:teuthology.orchestra.run.smithi174.stdout:5.14.0-361.el9.x86_64 2024-03-21T09:41:21.842 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 2024-03-21T09:41:21.843 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-03-21T09:41:21.843 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5ec3e6e88b2e2783e09498588bb1f433ae9c8986 2024-03-21T09:41:21.982 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum upgrade -y linux-firmware 2024-03-21T09:41:22.011 DEBUG:teuthology.orchestra.run.smithi153:> sudo yum upgrade -y linux-firmware 2024-03-21T09:41:24.225 INFO:teuthology.orchestra.run.smithi174.stdout:CentOS-9 - AppStream 11 MB/s | 6.7 MB 00:00 2024-03-21T09:41:24.393 INFO:teuthology.orchestra.run.smithi153.stdout:CentOS-9 - AppStream 10 MB/s | 6.7 MB 00:00 2024-03-21T09:41:26.194 INFO:teuthology.orchestra.run.smithi153.stdout:CentOS-9 - BaseOS 3.0 MB/s | 2.0 MB 00:00 2024-03-21T09:41:26.291 INFO:teuthology.orchestra.run.smithi174.stdout:CentOS-9 - BaseOS 2.1 MB/s | 2.0 MB 00:00 2024-03-21T09:41:30.845 INFO:teuthology.orchestra.run.smithi153.stdout:CentOS Stream 9 - BaseOS 1.9 MB/s | 8.0 MB 00:04 2024-03-21T09:41:33.605 INFO:teuthology.orchestra.run.smithi174.stdout:CentOS Stream 9 - BaseOS 1.2 MB/s | 8.0 MB 00:06 2024-03-21T09:41:38.875 INFO:teuthology.orchestra.run.smithi153.stdout:CentOS Stream 9 - AppStream 2.7 MB/s | 19 MB 00:07 2024-03-21T09:41:42.647 INFO:teuthology.orchestra.run.smithi174.stdout:CentOS Stream 9 - AppStream 2.4 MB/s | 19 MB 00:07 2024-03-21T09:41:43.161 INFO:teuthology.orchestra.run.smithi153.stdout:CentOS Stream 9 - CRB 7.1 MB/s | 6.1 MB 00:00 2024-03-21T09:41:44.710 INFO:teuthology.orchestra.run.smithi153.stdout:CentOS Stream 9 - Extras packages 38 kB/s | 16 kB 00:00 2024-03-21T09:41:46.841 INFO:teuthology.orchestra.run.smithi174.stdout:CentOS Stream 9 - CRB 8.0 MB/s | 6.1 MB 00:00 2024-03-21T09:41:48.365 INFO:teuthology.orchestra.run.smithi174.stdout:CentOS Stream 9 - Extras packages 41 kB/s | 16 kB 00:00 2024-03-21T09:41:49.130 INFO:teuthology.orchestra.run.smithi153.stdout:Extra Packages for Enterprise Linux 4.8 MB/s | 21 MB 00:04 2024-03-21T09:41:50.391 INFO:teuthology.orchestra.run.smithi174.stdout:Extra Packages for Enterprise Linux 11 MB/s | 21 MB 00:01 2024-03-21T09:41:54.052 INFO:teuthology.orchestra.run.smithi153.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-03-21T09:41:55.391 INFO:teuthology.orchestra.run.smithi174.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-03-21T09:41:56.241 INFO:teuthology.orchestra.run.smithi153.stdout:Dependencies resolved. 2024-03-21T09:41:56.251 INFO:teuthology.orchestra.run.smithi153.stdout:================================================================================ 2024-03-21T09:41:56.251 INFO:teuthology.orchestra.run.smithi153.stdout: Package Arch Version Repository Size 2024-03-21T09:41:56.252 INFO:teuthology.orchestra.run.smithi153.stdout:================================================================================ 2024-03-21T09:41:56.252 INFO:teuthology.orchestra.run.smithi153.stdout:Upgrading: 2024-03-21T09:41:56.252 INFO:teuthology.orchestra.run.smithi153.stdout: linux-firmware noarch 20240219-143.el9 CentOS-BaseOS 375 M 2024-03-21T09:41:56.252 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T09:41:56.252 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction Summary 2024-03-21T09:41:56.252 INFO:teuthology.orchestra.run.smithi153.stdout:================================================================================ 2024-03-21T09:41:56.252 INFO:teuthology.orchestra.run.smithi153.stdout:Upgrade 1 Package 2024-03-21T09:41:56.252 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T09:41:56.253 INFO:teuthology.orchestra.run.smithi153.stdout:Total download size: 375 M 2024-03-21T09:41:56.253 INFO:teuthology.orchestra.run.smithi153.stdout:Downloading Packages: 2024-03-21T09:41:57.681 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2024-03-21T09:41:57.695 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2024-03-21T09:41:57.695 INFO:teuthology.orchestra.run.smithi174.stdout: Package Arch Version Repository Size 2024-03-21T09:41:57.695 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2024-03-21T09:41:57.695 INFO:teuthology.orchestra.run.smithi174.stdout:Upgrading: 2024-03-21T09:41:57.695 INFO:teuthology.orchestra.run.smithi174.stdout: linux-firmware noarch 20240219-143.el9 CentOS-BaseOS 375 M 2024-03-21T09:41:57.695 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-21T09:41:57.695 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction Summary 2024-03-21T09:41:57.695 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2024-03-21T09:41:57.696 INFO:teuthology.orchestra.run.smithi174.stdout:Upgrade 1 Package 2024-03-21T09:41:57.696 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-21T09:41:57.696 INFO:teuthology.orchestra.run.smithi174.stdout:Total download size: 375 M 2024-03-21T09:41:57.696 INFO:teuthology.orchestra.run.smithi174.stdout:Downloading Packages: 2024-03-21T09:42:07.416 INFO:teuthology.orchestra.run.smithi174.stdout:linux-firmware-20240219-143.el9.noarch.rpm 39 MB/s | 375 MB 00:09 2024-03-21T09:42:07.417 INFO:teuthology.orchestra.run.smithi174.stdout:-------------------------------------------------------------------------------- 2024-03-21T09:42:07.418 INFO:teuthology.orchestra.run.smithi174.stdout:Total 39 MB/s | 375 MB 00:09 2024-03-21T09:42:07.418 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction check 2024-03-21T09:42:07.508 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction check succeeded. 2024-03-21T09:42:07.508 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction test 2024-03-21T09:42:09.972 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction test succeeded. 2024-03-21T09:42:09.972 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction 2024-03-21T09:42:13.260 INFO:teuthology.orchestra.run.smithi153.stdout:linux-firmware-20240219-143.el9.noarch.rpm 22 MB/s | 375 MB 00:17 2024-03-21T09:42:13.261 INFO:teuthology.orchestra.run.smithi153.stdout:-------------------------------------------------------------------------------- 2024-03-21T09:42:13.261 INFO:teuthology.orchestra.run.smithi153.stdout:Total 22 MB/s | 375 MB 00:17 2024-03-21T09:42:13.262 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction check 2024-03-21T09:42:13.357 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction check succeeded. 2024-03-21T09:42:13.357 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction test 2024-03-21T09:42:15.740 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction test succeeded. 2024-03-21T09:42:15.740 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction 2024-03-21T09:42:16.008 INFO:teuthology.orchestra.run.smithi174.stdout: Preparing : 1/1 2024-03-21T09:42:16.301 INFO:teuthology.orchestra.run.smithi174.stdout: Upgrading : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-21T09:42:16.381 INFO:teuthology.orchestra.run.smithi174.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-21T09:42:17.756 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2024-03-21T09:42:17.756 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-21T09:42:18.127 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-21T09:42:18.127 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-21T09:42:18.127 INFO:teuthology.orchestra.run.smithi174.stdout:Upgraded: 2024-03-21T09:42:18.127 INFO:teuthology.orchestra.run.smithi174.stdout: linux-firmware-20240219-143.el9.noarch 2024-03-21T09:42:18.127 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-21T09:42:18.127 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2024-03-21T09:42:18.428 INFO:teuthology.task.kernel:Downloading kernel 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 on host.b... 2024-03-21T09:42:18.429 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5ec3e6e88b2e2783e09498588bb1f433ae9c8986 2024-03-21T09:42:18.648 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/5ec3e6e88b2e2783e09498588bb1f433ae9c8986/centos/9/flavors/default/x86_64/ 2024-03-21T09:42:18.649 DEBUG:teuthology.orchestra.run.smithi174:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.8.0_g5ec3e6e88b2e-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/5ec3e6e88b2e2783e09498588bb1f433ae9c8986/centos/9/flavors/default/x86_64/ --input-file=- 2024-03-21T09:42:18.651 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi174.front.sepia.ceph.com to complete... 2024-03-21T09:42:22.082 INFO:teuthology.orchestra.run.smithi153.stdout: Preparing : 1/1 2024-03-21T09:42:22.332 INFO:teuthology.orchestra.run.smithi153.stdout: Upgrading : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-21T09:42:22.426 INFO:teuthology.orchestra.run.smithi153.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-21T09:42:23.612 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2024-03-21T09:42:23.612 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-21T09:42:23.954 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-21T09:42:23.954 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T09:42:23.955 INFO:teuthology.orchestra.run.smithi153.stdout:Upgraded: 2024-03-21T09:42:23.955 INFO:teuthology.orchestra.run.smithi153.stdout: linux-firmware-20240219-143.el9.noarch 2024-03-21T09:42:23.955 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T09:42:23.955 INFO:teuthology.orchestra.run.smithi153.stdout:Complete! 2024-03-21T09:42:24.226 INFO:teuthology.task.kernel:Downloading kernel 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 on host.a... 2024-03-21T09:42:24.226 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5ec3e6e88b2e2783e09498588bb1f433ae9c8986 2024-03-21T09:42:24.460 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/5ec3e6e88b2e2783e09498588bb1f433ae9c8986/centos/9/flavors/default/x86_64/ 2024-03-21T09:42:24.460 DEBUG:teuthology.orchestra.run.smithi153:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.8.0_g5ec3e6e88b2e-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/5ec3e6e88b2e2783e09498588bb1f433ae9c8986/centos/9/flavors/default/x86_64/ --input-file=- 2024-03-21T09:42:24.463 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi153.front.sepia.ceph.com to complete... 2024-03-21T09:42:55.635 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21 09:42:55 URL:https://4.chacra.ceph.com/r/kernel/testing/5ec3e6e88b2e2783e09498588bb1f433ae9c8986/centos/9/flavors/default/x86_64/kernel-6.8.0_g5ec3e6e88b2e-1.x86_64.rpm [336309687/336309687] -> "/tmp/kernel.x86_64.rpm" [1] 2024-03-21T09:42:55.635 INFO:teuthology.orchestra.run.smithi174.stderr:FINISHED --2024-03-21 09:42:55-- 2024-03-21T09:42:55.636 INFO:teuthology.orchestra.run.smithi174.stderr:Total wall clock time: 37s 2024-03-21T09:42:55.636 INFO:teuthology.orchestra.run.smithi174.stderr:Downloaded: 1 files, 321M in 36s (8.79 MB/s) 2024-03-21T09:42:55.639 INFO:teuthology.task.kernel:Installing kernel 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 on host.b... 2024-03-21T09:42:55.640 DEBUG:teuthology.orchestra.run.smithi174:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-03-21T09:42:57.579 INFO:teuthology.orchestra.run.smithi174.stdout:Verifying... ######################################## 2024-03-21T09:42:57.712 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing... ######################################## 2024-03-21T09:42:57.792 INFO:teuthology.orchestra.run.smithi174.stdout:Updating / installing... 2024-03-21T09:43:05.349 INFO:teuthology.orchestra.run.smithi174.stdout:kernel-6.8.0_g5ec3e6e88b2e-1 ######################################## 2024-03-21T09:43:15.477 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21 09:43:15 URL:https://4.chacra.ceph.com/r/kernel/testing/5ec3e6e88b2e2783e09498588bb1f433ae9c8986/centos/9/flavors/default/x86_64/kernel-6.8.0_g5ec3e6e88b2e-1.x86_64.rpm [336309687/336309687] -> "/tmp/kernel.x86_64.rpm" [1] 2024-03-21T09:43:15.477 INFO:teuthology.orchestra.run.smithi153.stderr:FINISHED --2024-03-21 09:43:15-- 2024-03-21T09:43:15.477 INFO:teuthology.orchestra.run.smithi153.stderr:Total wall clock time: 51s 2024-03-21T09:43:15.477 INFO:teuthology.orchestra.run.smithi153.stderr:Downloaded: 1 files, 321M in 50s (6.36 MB/s) 2024-03-21T09:43:15.480 INFO:teuthology.task.kernel:Installing kernel 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 on host.a... 2024-03-21T09:43:15.481 DEBUG:teuthology.orchestra.run.smithi153:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-03-21T09:43:17.416 INFO:teuthology.orchestra.run.smithi153.stdout:Verifying... ######################################## 2024-03-21T09:43:17.466 INFO:teuthology.orchestra.run.smithi153.stdout:Preparing... ######################################## 2024-03-21T09:43:17.546 INFO:teuthology.orchestra.run.smithi153.stdout:Updating / installing... 2024-03-21T09:43:19.206 INFO:teuthology.orchestra.run.smithi174.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-21T09:43:25.047 INFO:teuthology.orchestra.run.smithi153.stdout:kernel-6.8.0_g5ec3e6e88b2e-1 ######################################## 2024-03-21T09:43:38.962 INFO:teuthology.orchestra.run.smithi153.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-21T09:43:41.776 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi174.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-03-21T09:43:41.777 DEBUG:teuthology.orchestra.run.smithi174:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-03-21T09:43:41.853 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e 2024-03-21T09:43:41.853 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/System.map 2024-03-21T09:43:41.853 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/config 2024-03-21T09:43:41.853 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel 2024-03-21T09:43:41.853 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch 2024-03-21T09:43:41.853 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86 2024-03-21T09:43:41.853 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto 2024-03-21T09:43:41.853 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-03-21T09:43:41.853 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-03-21T09:43:41.853 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-03-21T09:43:41.853 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-03-21T09:43:41.854 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-03-21T09:43:41.854 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-03-21T09:43:41.854 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-03-21T09:43:41.854 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/crc32c-intel.ko 2024-03-21T09:43:41.854 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-03-21T09:43:41.854 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-03-21T09:43:41.854 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-03-21T09:43:41.854 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-03-21T09:43:41.854 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/serpent-avx2.ko 2024-03-21T09:43:41.854 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-03-21T09:43:41.854 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-03-21T09:43:41.854 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-03-21T09:43:41.854 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-03-21T09:43:41.855 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-03-21T09:43:41.855 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-03-21T09:43:41.855 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-03-21T09:43:41.855 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kernel 2024-03-21T09:43:41.855 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kernel/cpu 2024-03-21T09:43:41.855 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kernel/cpu/mce 2024-03-21T09:43:41.855 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-03-21T09:43:41.855 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kvm 2024-03-21T09:43:41.855 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kvm/kvm-amd.ko 2024-03-21T09:43:41.855 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kvm/kvm-intel.ko 2024-03-21T09:43:41.855 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kvm/kvm.ko 2024-03-21T09:43:41.855 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto 2024-03-21T09:43:41.855 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/ansi_cprng.ko 2024-03-21T09:43:41.855 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/anubis.ko 2024-03-21T09:43:41.855 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/arc4.ko 2024-03-21T09:43:41.856 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx 2024-03-21T09:43:41.856 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/async_memcpy.ko 2024-03-21T09:43:41.856 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/async_pq.ko 2024-03-21T09:43:41.856 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/async_raid6_recov.ko 2024-03-21T09:43:41.856 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/async_tx.ko 2024-03-21T09:43:41.856 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/async_xor.ko 2024-03-21T09:43:41.856 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/raid6test.ko 2024-03-21T09:43:41.856 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/authenc.ko 2024-03-21T09:43:41.856 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/authencesn.ko 2024-03-21T09:43:41.856 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/blake2b_generic.ko 2024-03-21T09:43:41.856 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/blowfish_common.ko 2024-03-21T09:43:41.856 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/blowfish_generic.ko 2024-03-21T09:43:41.856 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/camellia_generic.ko 2024-03-21T09:43:41.856 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/cast5_generic.ko 2024-03-21T09:43:41.857 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/cast6_generic.ko 2024-03-21T09:43:41.857 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/cast_common.ko 2024-03-21T09:43:41.857 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/ccm.ko 2024-03-21T09:43:41.857 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/cmac.ko 2024-03-21T09:43:41.857 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/crc32_generic.ko 2024-03-21T09:43:41.857 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/crypto_engine.ko 2024-03-21T09:43:41.857 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/crypto_user.ko 2024-03-21T09:43:41.857 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/cts.ko 2024-03-21T09:43:41.857 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/des_generic.ko 2024-03-21T09:43:41.857 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/dh_generic.ko 2024-03-21T09:43:41.857 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/echainiv.ko 2024-03-21T09:43:41.857 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/essiv.ko 2024-03-21T09:43:41.857 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/fcrypt.ko 2024-03-21T09:43:41.857 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/khazad.ko 2024-03-21T09:43:41.858 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/lz4.ko 2024-03-21T09:43:41.858 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/lz4hc.ko 2024-03-21T09:43:41.858 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/md4.ko 2024-03-21T09:43:41.858 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/michael_mic.ko 2024-03-21T09:43:41.858 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/pcbc.ko 2024-03-21T09:43:41.858 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/pcrypt.ko 2024-03-21T09:43:41.858 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/rmd160.ko 2024-03-21T09:43:41.858 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/seed.ko 2024-03-21T09:43:41.858 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/serpent_generic.ko 2024-03-21T09:43:41.858 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/tcrypt.ko 2024-03-21T09:43:41.858 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/tea.ko 2024-03-21T09:43:41.858 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/twofish_common.ko 2024-03-21T09:43:41.858 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/twofish_generic.ko 2024-03-21T09:43:41.859 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/vmac.ko 2024-03-21T09:43:41.859 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/wp512.ko 2024-03-21T09:43:41.859 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/xcbc.ko 2024-03-21T09:43:41.859 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/xor.ko 2024-03-21T09:43:41.859 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/xxhash_generic.ko 2024-03-21T09:43:41.859 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers 2024-03-21T09:43:41.859 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi 2024-03-21T09:43:41.859 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/acpi_ipmi.ko 2024-03-21T09:43:41.859 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/acpi_pad.ko 2024-03-21T09:43:41.859 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/custom_method.ko 2024-03-21T09:43:41.859 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/ec_sys.ko 2024-03-21T09:43:41.859 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/platform_profile.ko 2024-03-21T09:43:41.859 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/sbs.ko 2024-03-21T09:43:41.860 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/sbshc.ko 2024-03-21T09:43:41.860 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/video.ko 2024-03-21T09:43:41.860 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata 2024-03-21T09:43:41.860 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/acard-ahci.ko 2024-03-21T09:43:41.860 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/ahci_platform.ko 2024-03-21T09:43:41.860 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/ata_generic.ko 2024-03-21T09:43:41.860 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/libahci_platform.ko 2024-03-21T09:43:41.860 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_acpi.ko 2024-03-21T09:43:41.860 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_ali.ko 2024-03-21T09:43:41.860 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_amd.ko 2024-03-21T09:43:41.860 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_artop.ko 2024-03-21T09:43:41.860 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_atiixp.ko 2024-03-21T09:43:41.860 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_atp867x.ko 2024-03-21T09:43:41.860 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_cmd640.ko 2024-03-21T09:43:41.861 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_cmd64x.ko 2024-03-21T09:43:41.861 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_cypress.ko 2024-03-21T09:43:41.861 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_efar.ko 2024-03-21T09:43:41.861 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_hpt366.ko 2024-03-21T09:43:41.861 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_hpt37x.ko 2024-03-21T09:43:41.861 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_hpt3x2n.ko 2024-03-21T09:43:41.861 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_hpt3x3.ko 2024-03-21T09:43:41.861 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_it8213.ko 2024-03-21T09:43:41.861 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_it821x.ko 2024-03-21T09:43:41.861 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_jmicron.ko 2024-03-21T09:43:41.861 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_marvell.ko 2024-03-21T09:43:41.861 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_mpiix.ko 2024-03-21T09:43:41.861 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_netcell.ko 2024-03-21T09:43:41.861 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_ninja32.ko 2024-03-21T09:43:41.862 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_ns87410.ko 2024-03-21T09:43:41.862 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_ns87415.ko 2024-03-21T09:43:41.862 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_oldpiix.ko 2024-03-21T09:43:41.862 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_opti.ko 2024-03-21T09:43:41.862 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_optidma.ko 2024-03-21T09:43:41.862 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_pcmcia.ko 2024-03-21T09:43:41.862 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_pdc2027x.ko 2024-03-21T09:43:41.862 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-03-21T09:43:41.862 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_piccolo.ko 2024-03-21T09:43:41.862 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_rdc.ko 2024-03-21T09:43:41.862 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_sch.ko 2024-03-21T09:43:41.862 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_serverworks.ko 2024-03-21T09:43:41.862 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_sil680.ko 2024-03-21T09:43:41.863 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_sis.ko 2024-03-21T09:43:41.863 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_sl82c105.ko 2024-03-21T09:43:41.863 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_triflex.ko 2024-03-21T09:43:41.863 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_via.ko 2024-03-21T09:43:41.863 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pdc_adma.ko 2024-03-21T09:43:41.863 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_inic162x.ko 2024-03-21T09:43:41.863 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_mv.ko 2024-03-21T09:43:41.863 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_nv.ko 2024-03-21T09:43:41.863 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_promise.ko 2024-03-21T09:43:41.863 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_qstor.ko 2024-03-21T09:43:41.863 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_sil.ko 2024-03-21T09:43:41.863 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_sil24.ko 2024-03-21T09:43:41.863 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_sis.ko 2024-03-21T09:43:41.863 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_svw.ko 2024-03-21T09:43:41.864 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_sx4.ko 2024-03-21T09:43:41.864 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_uli.ko 2024-03-21T09:43:41.864 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_via.ko 2024-03-21T09:43:41.864 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_vsc.ko 2024-03-21T09:43:41.864 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/auxdisplay 2024-03-21T09:43:41.864 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/auxdisplay/ks0108.ko 2024-03-21T09:43:41.864 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/base 2024-03-21T09:43:41.864 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/base/regmap 2024-03-21T09:43:41.864 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/base/regmap/regmap-i2c.ko 2024-03-21T09:43:41.864 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/base/regmap/regmap-mmio.ko 2024-03-21T09:43:41.864 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/bcma 2024-03-21T09:43:41.864 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/bcma/bcma.ko 2024-03-21T09:43:41.864 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block 2024-03-21T09:43:41.864 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/aoe 2024-03-21T09:43:41.864 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/aoe/aoe.ko 2024-03-21T09:43:41.865 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/brd.ko 2024-03-21T09:43:41.865 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/drbd 2024-03-21T09:43:41.865 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/drbd/drbd.ko 2024-03-21T09:43:41.865 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/floppy.ko 2024-03-21T09:43:41.865 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/loop.ko 2024-03-21T09:43:41.865 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/mtip32xx 2024-03-21T09:43:41.865 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-03-21T09:43:41.865 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/nbd.ko 2024-03-21T09:43:41.865 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/null_blk 2024-03-21T09:43:41.865 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/null_blk/null_blk.ko 2024-03-21T09:43:41.865 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/pktcdvd.ko 2024-03-21T09:43:41.865 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/rbd.ko 2024-03-21T09:43:41.865 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/virtio_blk.ko 2024-03-21T09:43:41.865 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/xen-blkback 2024-03-21T09:43:41.866 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-03-21T09:43:41.866 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/xen-blkfront.ko 2024-03-21T09:43:41.866 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/zram 2024-03-21T09:43:41.866 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/zram/zram.ko 2024-03-21T09:43:41.866 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char 2024-03-21T09:43:41.866 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hangcheck-timer.ko 2024-03-21T09:43:41.866 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random 2024-03-21T09:43:41.866 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random/amd-rng.ko 2024-03-21T09:43:41.866 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random/intel-rng.ko 2024-03-21T09:43:41.866 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-03-21T09:43:41.866 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random/via-rng.ko 2024-03-21T09:43:41.866 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random/virtio-rng.ko 2024-03-21T09:43:41.866 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi 2024-03-21T09:43:41.867 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-03-21T09:43:41.867 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-03-21T09:43:41.867 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-03-21T09:43:41.867 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_si.ko 2024-03-21T09:43:41.867 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-03-21T09:43:41.867 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-03-21T09:43:41.867 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/lp.ko 2024-03-21T09:43:41.867 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/mwave 2024-03-21T09:43:41.867 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/mwave/mwave.ko 2024-03-21T09:43:41.867 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ppdev.ko 2024-03-21T09:43:41.867 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tlclk.ko 2024-03-21T09:43:41.867 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm 2024-03-21T09:43:41.867 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm.ko 2024-03-21T09:43:41.867 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm_atmel.ko 2024-03-21T09:43:41.868 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm_infineon.ko 2024-03-21T09:43:41.868 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm_nsc.ko 2024-03-21T09:43:41.868 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm_tis.ko 2024-03-21T09:43:41.868 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-03-21T09:43:41.868 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/uv_mmtimer.ko 2024-03-21T09:43:41.868 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/virtio_console.ko 2024-03-21T09:43:41.868 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq 2024-03-21T09:43:41.868 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-03-21T09:43:41.868 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-03-21T09:43:41.868 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/p4-clockmod.ko 2024-03-21T09:43:41.868 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-03-21T09:43:41.868 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/powernow-k8.ko 2024-03-21T09:43:41.868 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/speedstep-lib.ko 2024-03-21T09:43:41.868 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto 2024-03-21T09:43:41.869 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/ccp 2024-03-21T09:43:41.869 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-03-21T09:43:41.869 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/ccp/ccp.ko 2024-03-21T09:43:41.869 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel 2024-03-21T09:43:41.869 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel/qat 2024-03-21T09:43:41.869 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel/qat/qat_common 2024-03-21T09:43:41.869 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-03-21T09:43:41.869 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-03-21T09:43:41.869 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-03-21T09:43:41.869 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/padlock-aes.ko 2024-03-21T09:43:41.869 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/padlock-sha.ko 2024-03-21T09:43:41.869 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/virtio 2024-03-21T09:43:41.869 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-03-21T09:43:41.869 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dca 2024-03-21T09:43:41.870 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dca/dca.ko 2024-03-21T09:43:41.870 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/devfreq 2024-03-21T09:43:41.870 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-03-21T09:43:41.870 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dma 2024-03-21T09:43:41.870 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dma/dw 2024-03-21T09:43:41.870 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dma/dw/dw_dmac.ko 2024-03-21T09:43:41.870 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dma/ioat 2024-03-21T09:43:41.870 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dma/ioat/ioatdma.ko 2024-03-21T09:43:41.870 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac 2024-03-21T09:43:41.870 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/amd64_edac.ko 2024-03-21T09:43:41.870 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/e752x_edac.ko 2024-03-21T09:43:41.870 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/edac_mce_amd.ko 2024-03-21T09:43:41.870 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i3000_edac.ko 2024-03-21T09:43:41.871 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i3200_edac.ko 2024-03-21T09:43:41.871 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i5100_edac.ko 2024-03-21T09:43:41.871 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i5400_edac.ko 2024-03-21T09:43:41.871 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i7300_edac.ko 2024-03-21T09:43:41.871 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i7core_edac.ko 2024-03-21T09:43:41.871 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i82975x_edac.ko 2024-03-21T09:43:41.871 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/ie31200_edac.ko 2024-03-21T09:43:41.871 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/sb_edac.ko 2024-03-21T09:43:41.871 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/x38_edac.ko 2024-03-21T09:43:41.871 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/firmware 2024-03-21T09:43:41.871 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/firmware/edd.ko 2024-03-21T09:43:41.871 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/firmware/iscsi_ibft.ko 2024-03-21T09:43:41.871 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio 2024-03-21T09:43:41.871 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio/gpio-ich.ko 2024-03-21T09:43:41.872 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio/gpio-viperboard.ko 2024-03-21T09:43:41.872 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid 2024-03-21T09:43:41.872 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-appleir.ko 2024-03-21T09:43:41.872 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-aureal.ko 2024-03-21T09:43:41.872 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-axff.ko 2024-03-21T09:43:41.872 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-dr.ko 2024-03-21T09:43:41.872 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-elecom.ko 2024-03-21T09:43:41.872 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-elo.ko 2024-03-21T09:43:41.872 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-emsff.ko 2024-03-21T09:43:41.872 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-gaff.ko 2024-03-21T09:43:41.872 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-gt683r.ko 2024-03-21T09:43:41.872 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-gyration.ko 2024-03-21T09:43:41.872 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-holtek-kbd.ko 2024-03-21T09:43:41.873 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-holtek-mouse.ko 2024-03-21T09:43:41.873 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-holtekff.ko 2024-03-21T09:43:41.873 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-hyperv.ko 2024-03-21T09:43:41.873 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-icade.ko 2024-03-21T09:43:41.873 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-keytouch.ko 2024-03-21T09:43:41.873 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-kye.ko 2024-03-21T09:43:41.873 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-lcpower.ko 2024-03-21T09:43:41.873 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-led.ko 2024-03-21T09:43:41.873 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-lenovo.ko 2024-03-21T09:43:41.873 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-logitech-dj.ko 2024-03-21T09:43:41.873 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-03-21T09:43:41.873 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-multitouch.ko 2024-03-21T09:43:41.873 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-ortek.ko 2024-03-21T09:43:41.873 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-penmount.ko 2024-03-21T09:43:41.874 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-petalynx.ko 2024-03-21T09:43:41.874 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-picolcd.ko 2024-03-21T09:43:41.874 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-pl.ko 2024-03-21T09:43:41.874 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-plantronics.ko 2024-03-21T09:43:41.874 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-primax.ko 2024-03-21T09:43:41.874 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-rmi.ko 2024-03-21T09:43:41.874 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-arvo.ko 2024-03-21T09:43:41.874 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-common.ko 2024-03-21T09:43:41.874 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-isku.ko 2024-03-21T09:43:41.874 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-kone.ko 2024-03-21T09:43:41.874 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-03-21T09:43:41.874 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-konepure.ko 2024-03-21T09:43:41.874 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-03-21T09:43:41.874 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-lua.ko 2024-03-21T09:43:41.875 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-pyra.ko 2024-03-21T09:43:41.875 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-ryos.ko 2024-03-21T09:43:41.875 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-savu.ko 2024-03-21T09:43:41.875 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat.ko 2024-03-21T09:43:41.875 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-saitek.ko 2024-03-21T09:43:41.875 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-samsung.ko 2024-03-21T09:43:41.875 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-sensor-hub.ko 2024-03-21T09:43:41.875 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-sjoy.ko 2024-03-21T09:43:41.875 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-sony.ko 2024-03-21T09:43:41.875 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-speedlink.ko 2024-03-21T09:43:41.875 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-steelseries.ko 2024-03-21T09:43:41.875 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-sunplus.ko 2024-03-21T09:43:41.875 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-thrustmaster.ko 2024-03-21T09:43:41.875 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-tivo.ko 2024-03-21T09:43:41.876 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-tmff.ko 2024-03-21T09:43:41.876 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-topseed.ko 2024-03-21T09:43:41.876 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-twinhan.ko 2024-03-21T09:43:41.876 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-uclogic.ko 2024-03-21T09:43:41.876 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-waltop.ko 2024-03-21T09:43:41.876 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-wiimote.ko 2024-03-21T09:43:41.876 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-xinmo.ko 2024-03-21T09:43:41.876 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-zpff.ko 2024-03-21T09:43:41.876 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-zydacron.ko 2024-03-21T09:43:41.876 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/uhid.ko 2024-03-21T09:43:41.876 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/wacom.ko 2024-03-21T09:43:41.876 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hv 2024-03-21T09:43:41.876 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hv/hv_balloon.ko 2024-03-21T09:43:41.876 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hv/hv_utils.ko 2024-03-21T09:43:41.877 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hv/hv_vmbus.ko 2024-03-21T09:43:41.877 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon 2024-03-21T09:43:41.877 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/abituguru.ko 2024-03-21T09:43:41.877 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/abituguru3.ko 2024-03-21T09:43:41.877 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/acpi_power_meter.ko 2024-03-21T09:43:41.877 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ad7414.ko 2024-03-21T09:43:41.877 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ad7418.ko 2024-03-21T09:43:41.877 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adc128d818.ko 2024-03-21T09:43:41.877 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adm1025.ko 2024-03-21T09:43:41.877 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adm1026.ko 2024-03-21T09:43:41.877 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adm1029.ko 2024-03-21T09:43:41.877 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adm1031.ko 2024-03-21T09:43:41.877 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adm9240.ko 2024-03-21T09:43:41.877 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ads7828.ko 2024-03-21T09:43:41.878 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7410.ko 2024-03-21T09:43:41.878 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7411.ko 2024-03-21T09:43:41.878 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7462.ko 2024-03-21T09:43:41.878 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7470.ko 2024-03-21T09:43:41.878 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7475.ko 2024-03-21T09:43:41.878 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7x10.ko 2024-03-21T09:43:41.878 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/amc6821.ko 2024-03-21T09:43:41.878 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/applesmc.ko 2024-03-21T09:43:41.878 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/asb100.ko 2024-03-21T09:43:41.878 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/asc7621.ko 2024-03-21T09:43:41.878 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/asus_atk0110.ko 2024-03-21T09:43:41.878 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/atxp1.ko 2024-03-21T09:43:41.878 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/coretemp.ko 2024-03-21T09:43:41.878 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-03-21T09:43:41.879 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/dme1737.ko 2024-03-21T09:43:41.879 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ds1621.ko 2024-03-21T09:43:41.879 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ds620.ko 2024-03-21T09:43:41.879 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/emc1403.ko 2024-03-21T09:43:41.879 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/emc6w201.ko 2024-03-21T09:43:41.879 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/f71805f.ko 2024-03-21T09:43:41.879 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/f71882fg.ko 2024-03-21T09:43:41.879 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/f75375s.ko 2024-03-21T09:43:41.879 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/fam15h_power.ko 2024-03-21T09:43:41.879 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/fschmd.ko 2024-03-21T09:43:41.879 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/g760a.ko 2024-03-21T09:43:41.879 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/g762.ko 2024-03-21T09:43:41.879 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/gl518sm.ko 2024-03-21T09:43:41.879 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/gl520sm.ko 2024-03-21T09:43:41.880 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/hwmon-vid.ko 2024-03-21T09:43:41.880 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/i5500_temp.ko 2024-03-21T09:43:41.880 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/i5k_amb.ko 2024-03-21T09:43:41.880 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ibmaem.ko 2024-03-21T09:43:41.880 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ibmpex.ko 2024-03-21T09:43:41.880 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ina209.ko 2024-03-21T09:43:41.880 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ina2xx.ko 2024-03-21T09:43:41.880 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/it87.ko 2024-03-21T09:43:41.880 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/k10temp.ko 2024-03-21T09:43:41.881 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/k8temp.ko 2024-03-21T09:43:41.881 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lineage-pem.ko 2024-03-21T09:43:41.881 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm63.ko 2024-03-21T09:43:41.881 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm73.ko 2024-03-21T09:43:41.881 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm75.ko 2024-03-21T09:43:41.881 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm77.ko 2024-03-21T09:43:41.881 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm78.ko 2024-03-21T09:43:41.881 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm80.ko 2024-03-21T09:43:41.881 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm83.ko 2024-03-21T09:43:41.881 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm85.ko 2024-03-21T09:43:41.881 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm87.ko 2024-03-21T09:43:41.881 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm90.ko 2024-03-21T09:43:41.881 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm92.ko 2024-03-21T09:43:41.881 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm93.ko 2024-03-21T09:43:41.882 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm95234.ko 2024-03-21T09:43:41.882 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm95241.ko 2024-03-21T09:43:41.882 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm95245.ko 2024-03-21T09:43:41.882 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc2945.ko 2024-03-21T09:43:41.882 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4151.ko 2024-03-21T09:43:41.882 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4215.ko 2024-03-21T09:43:41.882 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4222.ko 2024-03-21T09:43:41.882 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4245.ko 2024-03-21T09:43:41.882 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4260.ko 2024-03-21T09:43:41.882 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4261.ko 2024-03-21T09:43:41.882 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max16065.ko 2024-03-21T09:43:41.882 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max1619.ko 2024-03-21T09:43:41.882 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max1668.ko 2024-03-21T09:43:41.883 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max197.ko 2024-03-21T09:43:41.883 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max6639.ko 2024-03-21T09:43:41.883 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max6650.ko 2024-03-21T09:43:41.883 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max6697.ko 2024-03-21T09:43:41.883 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/mcp3021.ko 2024-03-21T09:43:41.883 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/nct6683.ko 2024-03-21T09:43:41.883 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/nct6775-core.ko 2024-03-21T09:43:41.883 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/nct6775.ko 2024-03-21T09:43:41.883 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/nct7802.ko 2024-03-21T09:43:41.883 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pc87360.ko 2024-03-21T09:43:41.883 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pc87427.ko 2024-03-21T09:43:41.883 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pcf8591.ko 2024-03-21T09:43:41.883 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus 2024-03-21T09:43:41.883 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-03-21T09:43:41.884 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-03-21T09:43:41.884 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-03-21T09:43:41.884 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/max16064.ko 2024-03-21T09:43:41.884 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/max34440.ko 2024-03-21T09:43:41.884 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/max8688.ko 2024-03-21T09:43:41.884 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-03-21T09:43:41.884 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-03-21T09:43:41.884 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-03-21T09:43:41.884 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-03-21T09:43:41.884 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-03-21T09:43:41.884 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-03-21T09:43:41.884 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/powr1220.ko 2024-03-21T09:43:41.884 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sch5627.ko 2024-03-21T09:43:41.884 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sch5636.ko 2024-03-21T09:43:41.885 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sch56xx-common.ko 2024-03-21T09:43:41.885 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sht15.ko 2024-03-21T09:43:41.885 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sht21.ko 2024-03-21T09:43:41.885 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/shtc1.ko 2024-03-21T09:43:41.885 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sis5595.ko 2024-03-21T09:43:41.885 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/smsc47b397.ko 2024-03-21T09:43:41.885 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/smsc47m1.ko 2024-03-21T09:43:41.885 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/smsc47m192.ko 2024-03-21T09:43:41.885 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/thmc50.ko 2024-03-21T09:43:41.885 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/tmp102.ko 2024-03-21T09:43:41.885 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/tmp103.ko 2024-03-21T09:43:41.885 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/tmp401.ko 2024-03-21T09:43:41.886 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/tmp421.ko 2024-03-21T09:43:41.886 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/via-cputemp.ko 2024-03-21T09:43:41.886 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/via686a.ko 2024-03-21T09:43:41.886 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/vt1211.ko 2024-03-21T09:43:41.886 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/vt8231.ko 2024-03-21T09:43:41.886 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83627ehf.ko 2024-03-21T09:43:41.886 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83627hf.ko 2024-03-21T09:43:41.886 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83781d.ko 2024-03-21T09:43:41.886 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83791d.ko 2024-03-21T09:43:41.886 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83792d.ko 2024-03-21T09:43:41.886 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83793.ko 2024-03-21T09:43:41.886 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83795.ko 2024-03-21T09:43:41.886 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83l785ts.ko 2024-03-21T09:43:41.887 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83l786ng.ko 2024-03-21T09:43:41.887 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c 2024-03-21T09:43:41.887 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/algos 2024-03-21T09:43:41.887 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-03-21T09:43:41.887 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-03-21T09:43:41.887 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses 2024-03-21T09:43:41.887 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-03-21T09:43:41.887 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-03-21T09:43:41.887 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-03-21T09:43:41.887 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-03-21T09:43:41.887 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-03-21T09:43:41.887 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-03-21T09:43:41.887 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-03-21T09:43:41.887 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-03-21T09:43:41.888 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-i801.ko 2024-03-21T09:43:41.888 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-isch.ko 2024-03-21T09:43:41.888 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-03-21T09:43:41.888 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-03-21T09:43:41.888 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-03-21T09:43:41.888 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-parport.ko 2024-03-21T09:43:41.888 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-03-21T09:43:41.888 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-03-21T09:43:41.888 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-03-21T09:43:41.888 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-03-21T09:43:41.888 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-03-21T09:43:41.888 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-03-21T09:43:41.888 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-via.ko 2024-03-21T09:43:41.888 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-03-21T09:43:41.889 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-03-21T09:43:41.889 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/i2c-dev.ko 2024-03-21T09:43:41.889 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/i2c-mux.ko 2024-03-21T09:43:41.889 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-03-21T09:43:41.889 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/i2c-smbus.ko 2024-03-21T09:43:41.889 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/i2c-stub.ko 2024-03-21T09:43:41.889 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband 2024-03-21T09:43:41.889 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core 2024-03-21T09:43:41.889 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/ib_cm.ko 2024-03-21T09:43:41.889 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/ib_core.ko 2024-03-21T09:43:41.889 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/ib_umad.ko 2024-03-21T09:43:41.889 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-03-21T09:43:41.889 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/iw_cm.ko 2024-03-21T09:43:41.889 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/rdma_cm.ko 2024-03-21T09:43:41.889 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-03-21T09:43:41.890 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw 2024-03-21T09:43:41.890 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/cxgb4 2024-03-21T09:43:41.890 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-03-21T09:43:41.890 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mlx4 2024-03-21T09:43:41.890 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-03-21T09:43:41.890 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mlx5 2024-03-21T09:43:41.890 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-03-21T09:43:41.890 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mthca 2024-03-21T09:43:41.890 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-03-21T09:43:41.890 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp 2024-03-21T09:43:41.890 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/ipoib 2024-03-21T09:43:41.890 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-03-21T09:43:41.890 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/iser 2024-03-21T09:43:41.891 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-03-21T09:43:41.891 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/isert 2024-03-21T09:43:41.891 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-03-21T09:43:41.891 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/srp 2024-03-21T09:43:41.891 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-03-21T09:43:41.891 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/srpt 2024-03-21T09:43:41.891 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-03-21T09:43:41.891 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input 2024-03-21T09:43:41.891 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/gameport 2024-03-21T09:43:41.891 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-03-21T09:43:41.891 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/gameport/fm801-gp.ko 2024-03-21T09:43:41.891 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/gameport/gameport.ko 2024-03-21T09:43:41.891 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/joydev.ko 2024-03-21T09:43:41.891 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/keyboard 2024-03-21T09:43:41.891 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/keyboard/gpio_keys.ko 2024-03-21T09:43:41.892 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse 2024-03-21T09:43:41.892 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/appletouch.ko 2024-03-21T09:43:41.892 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/bcm5974.ko 2024-03-21T09:43:41.892 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/cyapatp.ko 2024-03-21T09:43:41.892 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/elan_i2c.ko 2024-03-21T09:43:41.892 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/sermouse.ko 2024-03-21T09:43:41.892 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-03-21T09:43:41.892 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/synaptics_usb.ko 2024-03-21T09:43:41.892 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/vsxxxaa.ko 2024-03-21T09:43:41.892 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/rmi4 2024-03-21T09:43:41.892 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/rmi4/rmi_core.ko 2024-03-21T09:43:41.892 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio 2024-03-21T09:43:41.892 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio/altera_ps2.ko 2024-03-21T09:43:41.892 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio/arc_ps2.ko 2024-03-21T09:43:41.892 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-03-21T09:43:41.893 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio/serio_raw.ko 2024-03-21T09:43:41.893 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/sparse-keymap.ko 2024-03-21T09:43:41.893 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds 2024-03-21T09:43:41.893 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-blinkm.ko 2024-03-21T09:43:41.893 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-lm3530.ko 2024-03-21T09:43:41.893 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-lp3944.ko 2024-03-21T09:43:41.893 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-ss4200.ko 2024-03-21T09:43:41.893 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger 2024-03-21T09:43:41.893 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-03-21T09:43:41.893 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-03-21T09:43:41.893 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-03-21T09:43:41.893 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-03-21T09:43:41.893 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-03-21T09:43:41.893 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-03-21T09:43:41.893 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-03-21T09:43:41.894 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-03-21T09:43:41.894 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-03-21T09:43:41.894 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md 2024-03-21T09:43:41.894 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/bcache 2024-03-21T09:43:41.894 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/bcache/bcache.ko 2024-03-21T09:43:41.894 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-bio-prison.ko 2024-03-21T09:43:41.894 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-cache-smq.ko 2024-03-21T09:43:41.894 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-cache.ko 2024-03-21T09:43:41.894 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-crypt.ko 2024-03-21T09:43:41.894 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-delay.ko 2024-03-21T09:43:41.894 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-flakey.ko 2024-03-21T09:43:41.894 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-log-userspace.ko 2024-03-21T09:43:41.894 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-multipath.ko 2024-03-21T09:43:41.894 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-queue-length.ko 2024-03-21T09:43:41.895 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-raid.ko 2024-03-21T09:43:41.895 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-round-robin.ko 2024-03-21T09:43:41.895 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-service-time.ko 2024-03-21T09:43:41.895 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-switch.ko 2024-03-21T09:43:41.895 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-thin-pool.ko 2024-03-21T09:43:41.895 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-verity.ko 2024-03-21T09:43:41.895 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/persistent-data 2024-03-21T09:43:41.895 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-03-21T09:43:41.895 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/raid0.ko 2024-03-21T09:43:41.895 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/raid1.ko 2024-03-21T09:43:41.895 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/raid10.ko 2024-03-21T09:43:41.895 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/raid456.ko 2024-03-21T09:43:41.895 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message 2024-03-21T09:43:41.895 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion 2024-03-21T09:43:41.895 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptbase.ko 2024-03-21T09:43:41.895 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptctl.ko 2024-03-21T09:43:41.896 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptfc.ko 2024-03-21T09:43:41.896 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptlan.ko 2024-03-21T09:43:41.896 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptsas.ko 2024-03-21T09:43:41.896 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptscsih.ko 2024-03-21T09:43:41.896 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptspi.ko 2024-03-21T09:43:41.896 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd 2024-03-21T09:43:41.896 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/lpc_ich.ko 2024-03-21T09:43:41.896 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/lpc_sch.ko 2024-03-21T09:43:41.896 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/mfd-core.ko 2024-03-21T09:43:41.896 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/sm501.ko 2024-03-21T09:43:41.896 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/viperboard.ko 2024-03-21T09:43:41.896 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/vx855.ko 2024-03-21T09:43:41.896 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/wl1273-core.ko 2024-03-21T09:43:41.896 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc 2024-03-21T09:43:41.896 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/altera-stapl 2024-03-21T09:43:41.896 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-03-21T09:43:41.897 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/apds9802als.ko 2024-03-21T09:43:41.897 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/apds990x.ko 2024-03-21T09:43:41.897 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/bh1770glc.ko 2024-03-21T09:43:41.897 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/cb710 2024-03-21T09:43:41.897 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/cb710/cb710.ko 2024-03-21T09:43:41.897 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/eeprom 2024-03-21T09:43:41.897 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/eeprom/at24.ko 2024-03-21T09:43:41.897 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-03-21T09:43:41.897 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/eeprom/max6875.ko 2024-03-21T09:43:41.897 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/enclosure.ko 2024-03-21T09:43:41.897 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/hpilo.ko 2024-03-21T09:43:41.897 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/isl29003.ko 2024-03-21T09:43:41.897 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/isl29020.ko 2024-03-21T09:43:41.897 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/lis3lv02d 2024-03-21T09:43:41.897 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-03-21T09:43:41.898 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-03-21T09:43:41.898 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/mei 2024-03-21T09:43:41.898 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/mei/mei-me.ko 2024-03-21T09:43:41.898 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/mei/mei-txe.ko 2024-03-21T09:43:41.898 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/mei/mei.ko 2024-03-21T09:43:41.898 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-gru 2024-03-21T09:43:41.898 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-gru/gru.ko 2024-03-21T09:43:41.898 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-xp 2024-03-21T09:43:41.898 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-xp/xp.ko 2024-03-21T09:43:41.898 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-xp/xpc.ko 2024-03-21T09:43:41.898 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-03-21T09:43:41.898 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/tifm_7xx1.ko 2024-03-21T09:43:41.898 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/tifm_core.ko 2024-03-21T09:43:41.898 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/tsl2550.ko 2024-03-21T09:43:41.899 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/vmw_balloon.ko 2024-03-21T09:43:41.899 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/vmw_vmci 2024-03-21T09:43:41.899 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-03-21T09:43:41.899 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net 2024-03-21T09:43:41.899 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/bonding 2024-03-21T09:43:41.899 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/bonding/bonding.ko 2024-03-21T09:43:41.899 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa 2024-03-21T09:43:41.899 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/b53 2024-03-21T09:43:41.899 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/b53/b53_common.ko 2024-03-21T09:43:41.899 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/bcm-sf2.ko 2024-03-21T09:43:41.899 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/mv88e6060.ko 2024-03-21T09:43:41.899 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/mv88e6xxx 2024-03-21T09:43:41.899 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-03-21T09:43:41.900 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dummy.ko 2024-03-21T09:43:41.900 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/eql.ko 2024-03-21T09:43:41.900 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet 2024-03-21T09:43:41.900 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/3com 2024-03-21T09:43:41.900 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-03-21T09:43:41.900 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-03-21T09:43:41.900 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-03-21T09:43:41.900 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-03-21T09:43:41.900 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/8390 2024-03-21T09:43:41.900 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/8390/8390.ko 2024-03-21T09:43:41.900 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-03-21T09:43:41.901 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-03-21T09:43:41.901 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-03-21T09:43:41.901 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/adaptec 2024-03-21T09:43:41.901 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-03-21T09:43:41.901 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/agere 2024-03-21T09:43:41.901 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/agere/et131x.ko 2024-03-21T09:43:41.901 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/alteon 2024-03-21T09:43:41.901 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-03-21T09:43:41.901 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/altera 2024-03-21T09:43:41.901 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-03-21T09:43:41.901 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/amd 2024-03-21T09:43:41.901 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-03-21T09:43:41.901 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-03-21T09:43:41.901 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-03-21T09:43:41.901 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros 2024-03-21T09:43:41.902 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/alx 2024-03-21T09:43:41.902 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-03-21T09:43:41.902 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atl1c 2024-03-21T09:43:41.902 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-03-21T09:43:41.902 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atl1e 2024-03-21T09:43:41.902 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-03-21T09:43:41.902 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atlx 2024-03-21T09:43:41.902 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-03-21T09:43:41.902 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-03-21T09:43:41.902 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom 2024-03-21T09:43:41.902 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-03-21T09:43:41.902 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-03-21T09:43:41.902 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-03-21T09:43:41.903 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-03-21T09:43:41.903 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-03-21T09:43:41.903 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/genet 2024-03-21T09:43:41.903 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-03-21T09:43:41.903 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-03-21T09:43:41.903 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/brocade 2024-03-21T09:43:41.903 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/brocade/bna 2024-03-21T09:43:41.903 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-03-21T09:43:41.903 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio 2024-03-21T09:43:41.903 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb 2024-03-21T09:43:41.903 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-03-21T09:43:41.903 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-03-21T09:43:41.903 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-03-21T09:43:41.903 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-03-21T09:43:41.904 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-03-21T09:43:41.904 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-03-21T09:43:41.904 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-03-21T09:43:41.904 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-03-21T09:43:41.904 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-03-21T09:43:41.904 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/cisco 2024-03-21T09:43:41.904 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/cisco/enic 2024-03-21T09:43:41.904 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-03-21T09:43:41.904 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec 2024-03-21T09:43:41.904 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip 2024-03-21T09:43:41.904 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-03-21T09:43:41.904 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-03-21T09:43:41.904 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-03-21T09:43:41.904 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-03-21T09:43:41.905 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-03-21T09:43:41.905 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-03-21T09:43:41.905 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dlink 2024-03-21T09:43:41.905 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-03-21T09:43:41.905 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-03-21T09:43:41.905 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dnet.ko 2024-03-21T09:43:41.905 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/emulex 2024-03-21T09:43:41.905 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/emulex/benet 2024-03-21T09:43:41.905 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-03-21T09:43:41.905 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/ethoc.ko 2024-03-21T09:43:41.905 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/fealnx.ko 2024-03-21T09:43:41.905 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel 2024-03-21T09:43:41.905 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/e100.ko 2024-03-21T09:43:41.905 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/e1000 2024-03-21T09:43:41.905 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-03-21T09:43:41.906 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/e1000e 2024-03-21T09:43:41.906 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-03-21T09:43:41.906 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/fm10k 2024-03-21T09:43:41.906 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-03-21T09:43:41.906 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/i40e 2024-03-21T09:43:41.906 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-03-21T09:43:41.906 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/iavf 2024-03-21T09:43:41.906 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-03-21T09:43:41.906 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/igb 2024-03-21T09:43:41.906 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-03-21T09:43:41.906 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/igbvf 2024-03-21T09:43:41.906 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-03-21T09:43:41.906 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/ixgbe 2024-03-21T09:43:41.907 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-03-21T09:43:41.907 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/ixgbevf 2024-03-21T09:43:41.907 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-03-21T09:43:41.907 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/jme.ko 2024-03-21T09:43:41.907 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/marvell 2024-03-21T09:43:41.907 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-03-21T09:43:41.907 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/marvell/skge.ko 2024-03-21T09:43:41.907 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-03-21T09:43:41.907 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox 2024-03-21T09:43:41.907 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx4 2024-03-21T09:43:41.907 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-03-21T09:43:41.907 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-03-21T09:43:41.907 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx5 2024-03-21T09:43:41.908 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-03-21T09:43:41.908 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-03-21T09:43:41.908 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/micrel 2024-03-21T09:43:41.908 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-03-21T09:43:41.908 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/myricom 2024-03-21T09:43:41.908 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/myricom/myri10ge 2024-03-21T09:43:41.908 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-03-21T09:43:41.908 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/natsemi 2024-03-21T09:43:41.908 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-03-21T09:43:41.908 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-03-21T09:43:41.908 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/neterion 2024-03-21T09:43:41.908 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-03-21T09:43:41.908 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/nvidia 2024-03-21T09:43:41.908 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-03-21T09:43:41.909 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/packetengines 2024-03-21T09:43:41.909 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-03-21T09:43:41.909 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-03-21T09:43:41.909 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic 2024-03-21T09:43:41.909 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic/netxen 2024-03-21T09:43:41.909 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-03-21T09:43:41.909 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-03-21T09:43:41.909 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-03-21T09:43:41.909 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-03-21T09:43:41.909 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/rdc 2024-03-21T09:43:41.909 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-03-21T09:43:41.909 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/realtek 2024-03-21T09:43:41.909 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-03-21T09:43:41.909 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-03-21T09:43:41.910 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/realtek/atp.ko 2024-03-21T09:43:41.910 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-03-21T09:43:41.910 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/rocker 2024-03-21T09:43:41.910 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-03-21T09:43:41.910 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sfc 2024-03-21T09:43:41.910 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-03-21T09:43:41.910 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/silan 2024-03-21T09:43:41.910 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-03-21T09:43:41.910 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sis 2024-03-21T09:43:41.910 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sis/sis190.ko 2024-03-21T09:43:41.910 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sis/sis900.ko 2024-03-21T09:43:41.910 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/smsc 2024-03-21T09:43:41.911 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-03-21T09:43:41.911 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-03-21T09:43:41.911 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-03-21T09:43:41.911 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-03-21T09:43:41.911 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/stmicro 2024-03-21T09:43:41.911 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/stmicro/stmmac 2024-03-21T09:43:41.911 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-03-21T09:43:41.911 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-03-21T09:43:41.911 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sun 2024-03-21T09:43:41.911 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sun/cassini.ko 2024-03-21T09:43:41.911 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sun/niu.ko 2024-03-21T09:43:41.911 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sun/sungem.ko 2024-03-21T09:43:41.911 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-03-21T09:43:41.911 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/tehuti 2024-03-21T09:43:41.912 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-03-21T09:43:41.912 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/ti 2024-03-21T09:43:41.912 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/ti/tlan.ko 2024-03-21T09:43:41.912 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/via 2024-03-21T09:43:41.912 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-03-21T09:43:41.912 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-03-21T09:43:41.912 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/wiznet 2024-03-21T09:43:41.912 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-03-21T09:43:41.912 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-03-21T09:43:41.912 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/xircom 2024-03-21T09:43:41.912 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-03-21T09:43:41.912 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/geneve.ko 2024-03-21T09:43:41.912 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/hyperv 2024-03-21T09:43:41.912 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-03-21T09:43:41.913 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ieee802154 2024-03-21T09:43:41.913 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ieee802154/fakelb.ko 2024-03-21T09:43:41.913 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ifb.ko 2024-03-21T09:43:41.913 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ipvlan 2024-03-21T09:43:41.913 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ipvlan/ipvlan.ko 2024-03-21T09:43:41.913 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/macvlan.ko 2024-03-21T09:43:41.913 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/macvtap.ko 2024-03-21T09:43:41.913 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mdio 2024-03-21T09:43:41.913 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mdio.ko 2024-03-21T09:43:41.913 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-03-21T09:43:41.913 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-03-21T09:43:41.913 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mdio/mdio-regmap.ko 2024-03-21T09:43:41.913 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mii.ko 2024-03-21T09:43:41.913 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/net_failover.ko 2024-03-21T09:43:41.914 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/netconsole.ko 2024-03-21T09:43:41.914 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/nlmon.ko 2024-03-21T09:43:41.914 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/pcs 2024-03-21T09:43:41.914 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/pcs/pcs-lynx.ko 2024-03-21T09:43:41.914 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-03-21T09:43:41.914 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy 2024-03-21T09:43:41.914 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/amd.ko 2024-03-21T09:43:41.914 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-03-21T09:43:41.914 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-03-21T09:43:41.914 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/bcm7xxx.ko 2024-03-21T09:43:41.914 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/bcm87xx.ko 2024-03-21T09:43:41.914 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/broadcom.ko 2024-03-21T09:43:41.914 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/cicada.ko 2024-03-21T09:43:41.915 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/davicom.ko 2024-03-21T09:43:41.915 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/dp83640.ko 2024-03-21T09:43:41.915 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/et1011c.ko 2024-03-21T09:43:41.915 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/icplus.ko 2024-03-21T09:43:41.915 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/lxt.ko 2024-03-21T09:43:41.915 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/marvell.ko 2024-03-21T09:43:41.915 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/micrel.ko 2024-03-21T09:43:41.915 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/national.ko 2024-03-21T09:43:41.915 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/phylink.ko 2024-03-21T09:43:41.915 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/qsemi.ko 2024-03-21T09:43:41.915 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/realtek.ko 2024-03-21T09:43:41.915 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/smsc.ko 2024-03-21T09:43:41.915 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/ste10Xp.ko 2024-03-21T09:43:41.915 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/vitesse.ko 2024-03-21T09:43:41.915 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp 2024-03-21T09:43:41.916 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/bsd_comp.ko 2024-03-21T09:43:41.916 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/ppp_async.ko 2024-03-21T09:43:41.916 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/ppp_deflate.ko 2024-03-21T09:43:41.916 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/ppp_generic.ko 2024-03-21T09:43:41.916 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/ppp_mppe.ko 2024-03-21T09:43:41.916 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/ppp_synctty.ko 2024-03-21T09:43:41.916 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/pppoe.ko 2024-03-21T09:43:41.916 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/pppox.ko 2024-03-21T09:43:41.916 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/pptp.ko 2024-03-21T09:43:41.916 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/slip 2024-03-21T09:43:41.916 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/slip/slhc.ko 2024-03-21T09:43:41.916 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/slip/slip.ko 2024-03-21T09:43:41.916 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/sungem_phy.ko 2024-03-21T09:43:41.916 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/tap.ko 2024-03-21T09:43:41.916 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team 2024-03-21T09:43:41.917 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team.ko 2024-03-21T09:43:41.917 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team_mode_activebackup.ko 2024-03-21T09:43:41.917 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team_mode_broadcast.ko 2024-03-21T09:43:41.917 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-03-21T09:43:41.917 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team_mode_random.ko 2024-03-21T09:43:41.917 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-03-21T09:43:41.917 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/tun.ko 2024-03-21T09:43:41.917 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/veth.ko 2024-03-21T09:43:41.917 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/virtio_net.ko 2024-03-21T09:43:41.917 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/vmxnet3 2024-03-21T09:43:41.917 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-03-21T09:43:41.917 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/vxlan 2024-03-21T09:43:41.917 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/vxlan/vxlan.ko 2024-03-21T09:43:41.917 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/xen-netback 2024-03-21T09:43:41.918 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/xen-netback/xen-netback.ko 2024-03-21T09:43:41.918 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/xen-netfront.ko 2024-03-21T09:43:41.918 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ntb 2024-03-21T09:43:41.918 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ntb/ntb.ko 2024-03-21T09:43:41.918 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/nvme 2024-03-21T09:43:41.918 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/nvme/host 2024-03-21T09:43:41.918 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/nvme/host/nvme-core.ko 2024-03-21T09:43:41.918 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/nvme/host/nvme.ko 2024-03-21T09:43:41.918 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/parport 2024-03-21T09:43:41.918 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/parport/parport.ko 2024-03-21T09:43:41.918 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/parport/parport_cs.ko 2024-03-21T09:43:41.918 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/parport/parport_pc.ko 2024-03-21T09:43:41.919 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/parport/parport_serial.ko 2024-03-21T09:43:41.919 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci 2024-03-21T09:43:41.919 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci/hotplug 2024-03-21T09:43:41.919 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-03-21T09:43:41.919 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci/pcie 2024-03-21T09:43:41.919 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci/pcie/aer_inject.ko 2024-03-21T09:43:41.919 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci/xen-pcifront.ko 2024-03-21T09:43:41.919 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pcmcia 2024-03-21T09:43:41.919 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pcmcia/i82092.ko 2024-03-21T09:43:41.919 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pcmcia/pd6729.ko 2024-03-21T09:43:41.920 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pcmcia/yenta_socket.ko 2024-03-21T09:43:41.920 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pinctrl 2024-03-21T09:43:41.920 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pinctrl/intel 2024-03-21T09:43:41.920 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-03-21T09:43:41.920 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform 2024-03-21T09:43:41.920 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86 2024-03-21T09:43:41.920 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/acer-wmi.ko 2024-03-21T09:43:41.920 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/acerhdf.ko 2024-03-21T09:43:41.920 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/apple-gmux.ko 2024-03-21T09:43:41.920 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/asus-laptop.ko 2024-03-21T09:43:41.920 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-03-21T09:43:41.920 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/asus-wmi.ko 2024-03-21T09:43:41.920 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/classmate-laptop.ko 2024-03-21T09:43:41.920 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-03-21T09:43:41.921 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-03-21T09:43:41.921 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-03-21T09:43:41.921 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-03-21T09:43:41.921 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/hdaps.ko 2024-03-21T09:43:41.921 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/intel 2024-03-21T09:43:41.921 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-03-21T09:43:41.921 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/intel_ips.ko 2024-03-21T09:43:41.921 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/msi-wmi.ko 2024-03-21T09:43:41.921 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/mxm-wmi.ko 2024-03-21T09:43:41.921 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-03-21T09:43:41.921 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/samsung-laptop.ko 2024-03-21T09:43:41.921 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/samsung-q10.ko 2024-03-21T09:43:41.921 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/topstar-laptop.ko 2024-03-21T09:43:41.921 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-03-21T09:43:41.922 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/toshiba_haps.ko 2024-03-21T09:43:41.922 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/wmi-bmof.ko 2024-03-21T09:43:41.922 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/wmi.ko 2024-03-21T09:43:41.922 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/powercap 2024-03-21T09:43:41.922 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/powercap/intel_rapl_common.ko 2024-03-21T09:43:41.922 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/powercap/intel_rapl_msr.ko 2024-03-21T09:43:41.922 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps 2024-03-21T09:43:41.922 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps/clients 2024-03-21T09:43:41.922 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps/clients/pps-gpio.ko 2024-03-21T09:43:41.922 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps/clients/pps-ldisc.ko 2024-03-21T09:43:41.922 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps/clients/pps_parport.ko 2024-03-21T09:43:41.922 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps/pps_core.ko 2024-03-21T09:43:41.922 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ptp 2024-03-21T09:43:41.923 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ptp/ptp.ko 2024-03-21T09:43:41.923 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ptp/ptp_kvm.ko 2024-03-21T09:43:41.923 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc 2024-03-21T09:43:41.923 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-bq32k.ko 2024-03-21T09:43:41.923 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1286.ko 2024-03-21T09:43:41.923 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1307.ko 2024-03-21T09:43:41.923 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1374.ko 2024-03-21T09:43:41.923 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1511.ko 2024-03-21T09:43:41.923 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1553.ko 2024-03-21T09:43:41.923 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1672.ko 2024-03-21T09:43:41.923 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1742.ko 2024-03-21T09:43:41.923 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds2404.ko 2024-03-21T09:43:41.923 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds3232.ko 2024-03-21T09:43:41.923 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-em3027.ko 2024-03-21T09:43:41.923 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-fm3130.ko 2024-03-21T09:43:41.924 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-isl12022.ko 2024-03-21T09:43:41.924 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-isl1208.ko 2024-03-21T09:43:41.924 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-m41t80.ko 2024-03-21T09:43:41.924 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-m48t35.ko 2024-03-21T09:43:41.924 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-m48t59.ko 2024-03-21T09:43:41.924 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-max6900.ko 2024-03-21T09:43:41.924 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-msm6242.ko 2024-03-21T09:43:41.924 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-pcf2127.ko 2024-03-21T09:43:41.924 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-pcf85063.ko 2024-03-21T09:43:41.924 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-pcf8523.ko 2024-03-21T09:43:41.924 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-pcf8563.ko 2024-03-21T09:43:41.924 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-pcf8583.ko 2024-03-21T09:43:41.924 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-rp5c01.ko 2024-03-21T09:43:41.924 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-rs5c372.ko 2024-03-21T09:43:41.924 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-03-21T09:43:41.925 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-rx8025.ko 2024-03-21T09:43:41.925 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-rx8581.ko 2024-03-21T09:43:41.925 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-03-21T09:43:41.925 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-x1205.ko 2024-03-21T09:43:41.925 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi 2024-03-21T09:43:41.925 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/3w-9xxx.ko 2024-03-21T09:43:41.925 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/3w-sas.ko 2024-03-21T09:43:41.925 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/3w-xxxx.ko 2024-03-21T09:43:41.925 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/BusLogic.ko 2024-03-21T09:43:41.925 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/a100u2w.ko 2024-03-21T09:43:41.925 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/aacraid 2024-03-21T09:43:41.925 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/aacraid/aacraid.ko 2024-03-21T09:43:41.925 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/advansys.ko 2024-03-21T09:43:41.925 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/aic7xxx 2024-03-21T09:43:41.926 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-03-21T09:43:41.926 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-03-21T09:43:41.926 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/am53c974.ko 2024-03-21T09:43:41.926 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/arcmsr 2024-03-21T09:43:41.926 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-03-21T09:43:41.926 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/atp870u.ko 2024-03-21T09:43:41.926 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/be2iscsi 2024-03-21T09:43:41.926 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-03-21T09:43:41.926 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bfa 2024-03-21T09:43:41.926 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bfa/bfa.ko 2024-03-21T09:43:41.926 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bnx2fc 2024-03-21T09:43:41.926 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-03-21T09:43:41.926 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bnx2i 2024-03-21T09:43:41.927 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-03-21T09:43:41.927 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/ch.ko 2024-03-21T09:43:41.927 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/csiostor 2024-03-21T09:43:41.927 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/csiostor/csiostor.ko 2024-03-21T09:43:41.927 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi 2024-03-21T09:43:41.927 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi/cxgb3i 2024-03-21T09:43:41.927 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-03-21T09:43:41.927 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi/cxgb4i 2024-03-21T09:43:41.927 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-03-21T09:43:41.927 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-03-21T09:43:41.927 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/dc395x.ko 2024-03-21T09:43:41.927 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/device_handler 2024-03-21T09:43:41.927 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-03-21T09:43:41.927 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-03-21T09:43:41.928 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-03-21T09:43:41.928 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-03-21T09:43:41.928 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/dmx3191d.ko 2024-03-21T09:43:41.928 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/esas2r 2024-03-21T09:43:41.928 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/esas2r/esas2r.ko 2024-03-21T09:43:41.928 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/esp_scsi.ko 2024-03-21T09:43:41.928 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/fcoe 2024-03-21T09:43:41.928 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/fcoe/fcoe.ko 2024-03-21T09:43:41.928 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-03-21T09:43:41.928 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/fnic 2024-03-21T09:43:41.928 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/fnic/fnic.ko 2024-03-21T09:43:41.928 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/hpsa.ko 2024-03-21T09:43:41.928 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/hptiop.ko 2024-03-21T09:43:41.928 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/hv_storvsc.ko 2024-03-21T09:43:41.929 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/initio.ko 2024-03-21T09:43:41.929 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/ipr.ko 2024-03-21T09:43:41.929 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/ips.ko 2024-03-21T09:43:41.929 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/isci 2024-03-21T09:43:41.929 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/isci/isci.ko 2024-03-21T09:43:41.929 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-03-21T09:43:41.929 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/iscsi_tcp.ko 2024-03-21T09:43:41.929 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libfc 2024-03-21T09:43:41.929 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libfc/libfc.ko 2024-03-21T09:43:41.929 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libiscsi.ko 2024-03-21T09:43:41.929 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libiscsi_tcp.ko 2024-03-21T09:43:41.929 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libsas 2024-03-21T09:43:41.929 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libsas/libsas.ko 2024-03-21T09:43:41.929 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/lpfc 2024-03-21T09:43:41.929 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/lpfc/lpfc.ko 2024-03-21T09:43:41.930 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/megaraid 2024-03-21T09:43:41.930 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/megaraid.ko 2024-03-21T09:43:41.930 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-03-21T09:43:41.930 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-03-21T09:43:41.930 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-03-21T09:43:41.930 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/mpt3sas 2024-03-21T09:43:41.930 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-03-21T09:43:41.930 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/mvsas 2024-03-21T09:43:41.930 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/mvsas/mvsas.ko 2024-03-21T09:43:41.930 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/mvumi.ko 2024-03-21T09:43:41.930 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/pm8001 2024-03-21T09:43:41.930 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-03-21T09:43:41.930 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/pmcraid.ko 2024-03-21T09:43:41.930 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla1280.ko 2024-03-21T09:43:41.931 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla2xxx 2024-03-21T09:43:41.931 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-03-21T09:43:41.931 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-03-21T09:43:41.931 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla4xxx 2024-03-21T09:43:41.931 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-03-21T09:43:41.931 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/raid_class.ko 2024-03-21T09:43:41.931 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_debug.ko 2024-03-21T09:43:41.931 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_transport_fc.ko 2024-03-21T09:43:41.931 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-03-21T09:43:41.931 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_transport_sas.ko 2024-03-21T09:43:41.931 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_transport_spi.ko 2024-03-21T09:43:41.931 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_transport_srp.ko 2024-03-21T09:43:41.931 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/ses.ko 2024-03-21T09:43:41.931 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/st.ko 2024-03-21T09:43:41.932 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/stex.ko 2024-03-21T09:43:41.932 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/sym53c8xx_2 2024-03-21T09:43:41.932 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-03-21T09:43:41.932 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/virtio_scsi.ko 2024-03-21T09:43:41.932 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/vmw_pvscsi.ko 2024-03-21T09:43:41.932 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/wd719x.ko 2024-03-21T09:43:41.932 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ssb 2024-03-21T09:43:41.932 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ssb/ssb.ko 2024-03-21T09:43:41.932 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target 2024-03-21T09:43:41.932 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/iscsi 2024-03-21T09:43:41.932 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-03-21T09:43:41.932 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/loopback 2024-03-21T09:43:41.932 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/loopback/tcm_loop.ko 2024-03-21T09:43:41.933 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/target_core_file.ko 2024-03-21T09:43:41.933 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/target_core_iblock.ko 2024-03-21T09:43:41.933 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/target_core_mod.ko 2024-03-21T09:43:41.933 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/target_core_pscsi.ko 2024-03-21T09:43:41.933 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/target_core_user.ko 2024-03-21T09:43:41.933 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/tcm_fc 2024-03-21T09:43:41.933 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-03-21T09:43:41.933 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal 2024-03-21T09:43:41.933 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel 2024-03-21T09:43:41.933 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal 2024-03-21T09:43:41.933 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-03-21T09:43:41.933 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-03-21T09:43:41.933 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-03-21T09:43:41.933 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-03-21T09:43:41.934 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-03-21T09:43:41.934 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-03-21T09:43:41.934 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-03-21T09:43:41.934 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-03-21T09:43:41.934 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-03-21T09:43:41.934 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-03-21T09:43:41.934 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-03-21T09:43:41.934 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-03-21T09:43:41.934 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-03-21T09:43:41.934 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-03-21T09:43:41.934 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-03-21T09:43:41.934 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-03-21T09:43:41.934 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-03-21T09:43:41.934 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-03-21T09:43:41.935 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty 2024-03-21T09:43:41.935 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/ipwireless 2024-03-21T09:43:41.935 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-03-21T09:43:41.935 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/n_gsm.ko 2024-03-21T09:43:41.935 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/n_hdlc.ko 2024-03-21T09:43:41.935 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/nozomi.ko 2024-03-21T09:43:41.935 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial 2024-03-21T09:43:41.935 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/8250 2024-03-21T09:43:41.935 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-03-21T09:43:41.935 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/arc_uart.ko 2024-03-21T09:43:41.935 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/jsm 2024-03-21T09:43:41.935 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/jsm/jsm.ko 2024-03-21T09:43:41.935 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ufs 2024-03-21T09:43:41.936 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ufs/core 2024-03-21T09:43:41.936 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ufs/core/ufshcd-core.ko 2024-03-21T09:43:41.936 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ufs/host 2024-03-21T09:43:41.936 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-03-21T09:43:41.936 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio 2024-03-21T09:43:41.936 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/uio.ko 2024-03-21T09:43:41.936 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/uio_aec.ko 2024-03-21T09:43:41.936 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/uio_cif.ko 2024-03-21T09:43:41.936 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/uio_pci_generic.ko 2024-03-21T09:43:41.936 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/uio_sercos3.ko 2024-03-21T09:43:41.936 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb 2024-03-21T09:43:41.936 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/class 2024-03-21T09:43:41.936 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/class/cdc-acm.ko 2024-03-21T09:43:41.936 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/class/cdc-wdm.ko 2024-03-21T09:43:41.936 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/class/usblp.ko 2024-03-21T09:43:41.936 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/class/usbtmc.ko 2024-03-21T09:43:41.937 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/host 2024-03-21T09:43:41.937 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/host/sl811-hcd.ko 2024-03-21T09:43:41.937 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/image 2024-03-21T09:43:41.937 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/image/mdc800.ko 2024-03-21T09:43:41.937 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/image/microtek.ko 2024-03-21T09:43:41.937 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc 2024-03-21T09:43:41.937 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/adutux.ko 2024-03-21T09:43:41.937 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/appledisplay.ko 2024-03-21T09:43:41.937 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/emi26.ko 2024-03-21T09:43:41.937 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/emi62.ko 2024-03-21T09:43:41.937 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/ezusb.ko 2024-03-21T09:43:41.937 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/idmouse.ko 2024-03-21T09:43:41.937 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/iowarrior.ko 2024-03-21T09:43:41.937 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/isight_firmware.ko 2024-03-21T09:43:41.937 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/ldusb.ko 2024-03-21T09:43:41.938 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/legousbtower.ko 2024-03-21T09:43:41.938 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/sisusbvga 2024-03-21T09:43:41.938 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-03-21T09:43:41.938 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/trancevibrator.ko 2024-03-21T09:43:41.938 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/usb3503.ko 2024-03-21T09:43:41.938 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/usblcd.ko 2024-03-21T09:43:41.938 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/usbsevseg.ko 2024-03-21T09:43:41.938 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/uss720.ko 2024-03-21T09:43:41.938 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/yurex.ko 2024-03-21T09:43:41.938 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/phy 2024-03-21T09:43:41.938 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/phy/phy-generic.ko 2024-03-21T09:43:41.938 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial 2024-03-21T09:43:41.938 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/aircable.ko 2024-03-21T09:43:41.939 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ark3116.ko 2024-03-21T09:43:41.939 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/belkin_sa.ko 2024-03-21T09:43:41.939 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ch341.ko 2024-03-21T09:43:41.939 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/cp210x.ko 2024-03-21T09:43:41.939 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/cyberjack.ko 2024-03-21T09:43:41.939 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/cypress_m8.ko 2024-03-21T09:43:41.939 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/digi_acceleport.ko 2024-03-21T09:43:41.939 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/empeg.ko 2024-03-21T09:43:41.939 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ftdi_sio.ko 2024-03-21T09:43:41.939 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/garmin_gps.ko 2024-03-21T09:43:41.939 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/io_edgeport.ko 2024-03-21T09:43:41.939 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/io_ti.ko 2024-03-21T09:43:41.939 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ipaq.ko 2024-03-21T09:43:41.939 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ipw.ko 2024-03-21T09:43:41.939 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ir-usb.ko 2024-03-21T09:43:41.940 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-03-21T09:43:41.940 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/keyspan.ko 2024-03-21T09:43:41.940 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/keyspan_pda.ko 2024-03-21T09:43:41.940 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/kl5kusb105.ko 2024-03-21T09:43:41.940 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/kobil_sct.ko 2024-03-21T09:43:41.940 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/mct_u232.ko 2024-03-21T09:43:41.940 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/mos7720.ko 2024-03-21T09:43:41.940 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/mos7840.ko 2024-03-21T09:43:41.940 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/navman.ko 2024-03-21T09:43:41.940 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/omninet.ko 2024-03-21T09:43:41.940 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/opticon.ko 2024-03-21T09:43:41.940 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/option.ko 2024-03-21T09:43:41.940 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/oti6858.ko 2024-03-21T09:43:41.940 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/pl2303.ko 2024-03-21T09:43:41.941 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/qcaux.ko 2024-03-21T09:43:41.941 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/qcserial.ko 2024-03-21T09:43:41.941 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/quatech2.ko 2024-03-21T09:43:41.941 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/safe_serial.ko 2024-03-21T09:43:41.941 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/sierra.ko 2024-03-21T09:43:41.941 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/spcp8x5.ko 2024-03-21T09:43:41.941 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ssu100.ko 2024-03-21T09:43:41.941 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/symbolserial.ko 2024-03-21T09:43:41.941 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-03-21T09:43:41.941 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-03-21T09:43:41.941 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/usb_debug.ko 2024-03-21T09:43:41.941 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/usb_wwan.ko 2024-03-21T09:43:41.941 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/visor.ko 2024-03-21T09:43:41.941 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/whiteheat.ko 2024-03-21T09:43:41.942 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/xsens_mt.ko 2024-03-21T09:43:41.942 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage 2024-03-21T09:43:41.942 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/uas.ko 2024-03-21T09:43:41.942 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-alauda.ko 2024-03-21T09:43:41.942 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-cypress.ko 2024-03-21T09:43:41.942 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-datafab.ko 2024-03-21T09:43:41.942 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-03-21T09:43:41.942 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-freecom.ko 2024-03-21T09:43:41.942 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-isd200.ko 2024-03-21T09:43:41.942 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-03-21T09:43:41.942 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-karma.ko 2024-03-21T09:43:41.942 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-onetouch.ko 2024-03-21T09:43:41.942 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-realtek.ko 2024-03-21T09:43:41.942 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-sddr09.ko 2024-03-21T09:43:41.943 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-sddr55.ko 2024-03-21T09:43:41.943 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-usbat.ko 2024-03-21T09:43:41.943 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/usb-storage.ko 2024-03-21T09:43:41.943 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/usbip 2024-03-21T09:43:41.943 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/usbip/usbip-core.ko 2024-03-21T09:43:41.943 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/usbip/usbip-host.ko 2024-03-21T09:43:41.943 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-03-21T09:43:41.943 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio 2024-03-21T09:43:41.943 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio/pci 2024-03-21T09:43:41.943 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-03-21T09:43:41.943 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio/pci/vfio-pci.ko 2024-03-21T09:43:41.943 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio/vfio.ko 2024-03-21T09:43:41.943 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-03-21T09:43:41.943 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vhost 2024-03-21T09:43:41.944 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vhost/vhost.ko 2024-03-21T09:43:41.944 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vhost/vhost_iotlb.ko 2024-03-21T09:43:41.944 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vhost/vhost_net.ko 2024-03-21T09:43:41.944 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vhost/vhost_scsi.ko 2024-03-21T09:43:41.944 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video 2024-03-21T09:43:41.944 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight 2024-03-21T09:43:41.944 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight/apple_bl.ko 2024-03-21T09:43:41.944 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight/lcd.ko 2024-03-21T09:43:41.944 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight/platform_lcd.ko 2024-03-21T09:43:41.944 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio 2024-03-21T09:43:41.944 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio.ko 2024-03-21T09:43:41.944 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_balloon.ko 2024-03-21T09:43:41.944 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_mmio.ko 2024-03-21T09:43:41.944 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_pci.ko 2024-03-21T09:43:41.945 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-03-21T09:43:41.945 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-03-21T09:43:41.945 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_ring.ko 2024-03-21T09:43:41.945 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1 2024-03-21T09:43:41.945 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/masters 2024-03-21T09:43:41.945 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/masters/ds2482.ko 2024-03-21T09:43:41.945 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/masters/ds2490.ko 2024-03-21T09:43:41.945 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves 2024-03-21T09:43:41.945 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-03-21T09:43:41.945 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-03-21T09:43:41.945 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-03-21T09:43:41.945 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-03-21T09:43:41.945 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-03-21T09:43:41.945 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-03-21T09:43:41.945 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-03-21T09:43:41.946 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-03-21T09:43:41.946 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-03-21T09:43:41.946 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_smem.ko 2024-03-21T09:43:41.946 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_therm.ko 2024-03-21T09:43:41.946 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/wire.ko 2024-03-21T09:43:41.946 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog 2024-03-21T09:43:41.946 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/alim1535_wdt.ko 2024-03-21T09:43:41.946 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/alim7101_wdt.ko 2024-03-21T09:43:41.946 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/f71808e_wdt.ko 2024-03-21T09:43:41.946 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/hpwdt.ko 2024-03-21T09:43:41.946 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/i6300esb.ko 2024-03-21T09:43:41.946 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-03-21T09:43:41.946 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/iTCO_wdt.ko 2024-03-21T09:43:41.946 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/ib700wdt.ko 2024-03-21T09:43:41.947 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/ibmasr.ko 2024-03-21T09:43:41.947 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-03-21T09:43:41.947 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/it8712f_wdt.ko 2024-03-21T09:43:41.947 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/it87_wdt.ko 2024-03-21T09:43:41.947 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/machzwd.ko 2024-03-21T09:43:41.947 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/nv_tco.ko 2024-03-21T09:43:41.947 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/pcwd_pci.ko 2024-03-21T09:43:41.947 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/pcwd_usb.ko 2024-03-21T09:43:41.947 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-03-21T09:43:41.947 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/sch311x_wdt.ko 2024-03-21T09:43:41.947 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/softdog.ko 2024-03-21T09:43:41.947 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/sp5100_tco.ko 2024-03-21T09:43:41.947 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/via_wdt.ko 2024-03-21T09:43:41.947 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-03-21T09:43:41.948 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/w83877f_wdt.ko 2024-03-21T09:43:41.948 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/w83977f_wdt.ko 2024-03-21T09:43:41.948 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/wdt_pci.ko 2024-03-21T09:43:41.948 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/xen_wdt.ko 2024-03-21T09:43:41.948 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen 2024-03-21T09:43:41.948 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-acpi-processor.ko 2024-03-21T09:43:41.948 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-evtchn.ko 2024-03-21T09:43:41.948 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-gntalloc.ko 2024-03-21T09:43:41.948 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-gntdev.ko 2024-03-21T09:43:41.948 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-pciback 2024-03-21T09:43:41.948 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-03-21T09:43:41.948 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-privcmd.ko 2024-03-21T09:43:41.948 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xenfs 2024-03-21T09:43:41.948 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xenfs/xenfs.ko 2024-03-21T09:43:41.949 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs 2024-03-21T09:43:41.949 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/9p 2024-03-21T09:43:41.949 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/9p/9p.ko 2024-03-21T09:43:41.949 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/affs 2024-03-21T09:43:41.949 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/affs/affs.ko 2024-03-21T09:43:41.949 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/befs 2024-03-21T09:43:41.949 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/befs/befs.ko 2024-03-21T09:43:41.949 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/binfmt_misc.ko 2024-03-21T09:43:41.949 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/btrfs 2024-03-21T09:43:41.949 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/btrfs/btrfs.ko 2024-03-21T09:43:41.949 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/cachefiles 2024-03-21T09:43:41.949 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/cachefiles/cachefiles.ko 2024-03-21T09:43:41.949 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ceph 2024-03-21T09:43:41.949 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ceph/ceph.ko 2024-03-21T09:43:41.950 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/coda 2024-03-21T09:43:41.950 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/coda/coda.ko 2024-03-21T09:43:41.950 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/cramfs 2024-03-21T09:43:41.950 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/cramfs/cramfs.ko 2024-03-21T09:43:41.950 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/dlm 2024-03-21T09:43:41.950 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/dlm/dlm.ko 2024-03-21T09:43:41.950 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ecryptfs 2024-03-21T09:43:41.950 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ecryptfs/ecryptfs.ko 2024-03-21T09:43:41.950 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/f2fs 2024-03-21T09:43:41.950 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/f2fs/f2fs.ko 2024-03-21T09:43:41.950 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fat 2024-03-21T09:43:41.950 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fat/fat.ko 2024-03-21T09:43:41.950 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fat/msdos.ko 2024-03-21T09:43:41.951 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fat/vfat.ko 2024-03-21T09:43:41.951 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fuse 2024-03-21T09:43:41.951 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fuse/cuse.ko 2024-03-21T09:43:41.951 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fuse/fuse.ko 2024-03-21T09:43:41.951 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/gfs2 2024-03-21T09:43:41.951 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/gfs2/gfs2.ko 2024-03-21T09:43:41.951 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/hfs 2024-03-21T09:43:41.951 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/hfs/hfs.ko 2024-03-21T09:43:41.951 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/hfsplus 2024-03-21T09:43:41.951 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/hfsplus/hfsplus.ko 2024-03-21T09:43:41.951 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/isofs 2024-03-21T09:43:41.951 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/isofs/isofs.ko 2024-03-21T09:43:41.951 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/jfs 2024-03-21T09:43:41.951 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/jfs/jfs.ko 2024-03-21T09:43:41.952 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/lockd 2024-03-21T09:43:41.952 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/lockd/lockd.ko 2024-03-21T09:43:41.952 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/minix 2024-03-21T09:43:41.952 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/minix/minix.ko 2024-03-21T09:43:41.952 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/netfs 2024-03-21T09:43:41.952 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/netfs/netfs.ko 2024-03-21T09:43:41.952 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs 2024-03-21T09:43:41.952 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/blocklayout 2024-03-21T09:43:41.952 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-03-21T09:43:41.952 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/filelayout 2024-03-21T09:43:41.952 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-03-21T09:43:41.952 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/flexfilelayout 2024-03-21T09:43:41.952 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-03-21T09:43:41.952 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/nfs.ko 2024-03-21T09:43:41.953 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/nfsv3.ko 2024-03-21T09:43:41.953 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/nfsv4.ko 2024-03-21T09:43:41.953 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs_common 2024-03-21T09:43:41.953 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs_common/grace.ko 2024-03-21T09:43:41.953 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs_common/nfs_acl.ko 2024-03-21T09:43:41.953 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfsd 2024-03-21T09:43:41.953 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfsd/nfsd.ko 2024-03-21T09:43:41.953 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nilfs2 2024-03-21T09:43:41.953 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nilfs2/nilfs2.ko 2024-03-21T09:43:41.953 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls 2024-03-21T09:43:41.953 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-celtic.ko 2024-03-21T09:43:41.953 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-centeuro.ko 2024-03-21T09:43:41.953 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-croatian.ko 2024-03-21T09:43:41.953 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-cyrillic.ko 2024-03-21T09:43:41.954 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-gaelic.ko 2024-03-21T09:43:41.954 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-greek.ko 2024-03-21T09:43:41.954 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-iceland.ko 2024-03-21T09:43:41.954 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-inuit.ko 2024-03-21T09:43:41.954 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-roman.ko 2024-03-21T09:43:41.954 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-romanian.ko 2024-03-21T09:43:41.954 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-turkish.ko 2024-03-21T09:43:41.954 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp1250.ko 2024-03-21T09:43:41.954 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp1251.ko 2024-03-21T09:43:41.954 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp1255.ko 2024-03-21T09:43:41.954 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp737.ko 2024-03-21T09:43:41.954 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp775.ko 2024-03-21T09:43:41.954 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp850.ko 2024-03-21T09:43:41.954 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp852.ko 2024-03-21T09:43:41.955 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp855.ko 2024-03-21T09:43:41.955 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp857.ko 2024-03-21T09:43:41.955 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp860.ko 2024-03-21T09:43:41.955 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp861.ko 2024-03-21T09:43:41.955 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp862.ko 2024-03-21T09:43:41.955 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp863.ko 2024-03-21T09:43:41.955 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp864.ko 2024-03-21T09:43:41.955 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp865.ko 2024-03-21T09:43:41.955 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp866.ko 2024-03-21T09:43:41.955 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp869.ko 2024-03-21T09:43:41.955 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp874.ko 2024-03-21T09:43:41.955 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp932.ko 2024-03-21T09:43:41.955 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp936.ko 2024-03-21T09:43:41.955 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp949.ko 2024-03-21T09:43:41.956 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp950.ko 2024-03-21T09:43:41.956 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_euc-jp.ko 2024-03-21T09:43:41.956 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-1.ko 2024-03-21T09:43:41.956 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-13.ko 2024-03-21T09:43:41.956 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-14.ko 2024-03-21T09:43:41.956 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-15.ko 2024-03-21T09:43:41.956 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-2.ko 2024-03-21T09:43:41.956 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-3.ko 2024-03-21T09:43:41.956 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-4.ko 2024-03-21T09:43:41.956 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-5.ko 2024-03-21T09:43:41.956 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-6.ko 2024-03-21T09:43:41.956 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-7.ko 2024-03-21T09:43:41.956 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-9.ko 2024-03-21T09:43:41.956 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_koi8-r.ko 2024-03-21T09:43:41.957 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_koi8-ru.ko 2024-03-21T09:43:41.957 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_koi8-u.ko 2024-03-21T09:43:41.957 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_ucs2_utils.ko 2024-03-21T09:43:41.957 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_utf8.ko 2024-03-21T09:43:41.957 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2 2024-03-21T09:43:41.957 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/cluster 2024-03-21T09:43:41.957 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-03-21T09:43:41.957 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/dlm 2024-03-21T09:43:41.957 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-03-21T09:43:41.957 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/dlmfs 2024-03-21T09:43:41.957 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-03-21T09:43:41.957 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/ocfs2.ko 2024-03-21T09:43:41.957 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-03-21T09:43:41.957 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-03-21T09:43:41.958 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-03-21T09:43:41.958 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/overlayfs 2024-03-21T09:43:41.958 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/overlayfs/overlay.ko 2024-03-21T09:43:41.958 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/pstore 2024-03-21T09:43:41.958 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/pstore/ramoops.ko 2024-03-21T09:43:41.958 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/reiserfs 2024-03-21T09:43:41.958 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/reiserfs/reiserfs.ko 2024-03-21T09:43:41.958 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/romfs 2024-03-21T09:43:41.958 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/romfs/romfs.ko 2024-03-21T09:43:41.958 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb 2024-03-21T09:43:41.958 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb/client 2024-03-21T09:43:41.958 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb/client/cifs.ko 2024-03-21T09:43:41.958 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb/common 2024-03-21T09:43:41.959 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb/common/cifs_arc4.ko 2024-03-21T09:43:41.959 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb/common/cifs_md4.ko 2024-03-21T09:43:41.959 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/squashfs 2024-03-21T09:43:41.959 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/squashfs/squashfs.ko 2024-03-21T09:43:41.959 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/sysv 2024-03-21T09:43:41.959 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/sysv/sysv.ko 2024-03-21T09:43:41.959 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/udf 2024-03-21T09:43:41.959 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/udf/udf.ko 2024-03-21T09:43:41.959 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ufs 2024-03-21T09:43:41.959 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ufs/ufs.ko 2024-03-21T09:43:41.959 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/xfs 2024-03-21T09:43:41.959 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/xfs/xfs.ko 2024-03-21T09:43:41.959 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel 2024-03-21T09:43:41.959 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel/rcu 2024-03-21T09:43:41.959 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel/rcu/rcutorture.ko 2024-03-21T09:43:41.960 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel/torture.ko 2024-03-21T09:43:41.960 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel/trace 2024-03-21T09:43:41.960 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-03-21T09:43:41.960 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib 2024-03-21T09:43:41.960 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/asn1_encoder.ko 2024-03-21T09:43:41.960 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/crc-itu-t.ko 2024-03-21T09:43:41.960 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/crc8.ko 2024-03-21T09:43:41.960 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/crypto 2024-03-21T09:43:41.960 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/crypto/libarc4.ko 2024-03-21T09:43:41.960 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/crypto/libdes.ko 2024-03-21T09:43:41.960 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/libcrc32c.ko 2024-03-21T09:43:41.960 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/lru_cache.ko 2024-03-21T09:43:41.960 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/lz4 2024-03-21T09:43:41.960 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/lz4/lz4_compress.ko 2024-03-21T09:43:41.961 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/lz4/lz4hc_compress.ko 2024-03-21T09:43:41.961 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/math 2024-03-21T09:43:41.961 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/math/cordic.ko 2024-03-21T09:43:41.961 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/raid6 2024-03-21T09:43:41.961 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/raid6/raid6_pq.ko 2024-03-21T09:43:41.961 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/reed_solomon 2024-03-21T09:43:41.961 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/reed_solomon/reed_solomon.ko 2024-03-21T09:43:41.961 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/ts_bm.ko 2024-03-21T09:43:41.961 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/ts_fsm.ko 2024-03-21T09:43:41.961 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/ts_kmp.ko 2024-03-21T09:43:41.961 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/zstd 2024-03-21T09:43:41.961 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/zstd/zstd_compress.ko 2024-03-21T09:43:41.961 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/mm 2024-03-21T09:43:41.961 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/mm/hwpoison-inject.ko 2024-03-21T09:43:41.962 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net 2024-03-21T09:43:41.962 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan 2024-03-21T09:43:41.962 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/6lowpan.ko 2024-03-21T09:43:41.962 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_dest.ko 2024-03-21T09:43:41.962 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_fragment.ko 2024-03-21T09:43:41.962 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_hop.ko 2024-03-21T09:43:41.962 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_ipv6.ko 2024-03-21T09:43:41.962 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_mobility.ko 2024-03-21T09:43:41.962 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_routing.ko 2024-03-21T09:43:41.962 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_udp.ko 2024-03-21T09:43:41.962 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802 2024-03-21T09:43:41.962 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802/garp.ko 2024-03-21T09:43:41.962 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802/mrp.ko 2024-03-21T09:43:41.962 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802/p8022.ko 2024-03-21T09:43:41.963 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802/psnap.ko 2024-03-21T09:43:41.963 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802/stp.ko 2024-03-21T09:43:41.963 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/8021q 2024-03-21T09:43:41.963 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/8021q/8021q.ko 2024-03-21T09:43:41.963 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/9p 2024-03-21T09:43:41.963 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/9p/9pnet.ko 2024-03-21T09:43:41.963 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/9p/9pnet_fd.ko 2024-03-21T09:43:41.963 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/9p/9pnet_rdma.ko 2024-03-21T09:43:41.963 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/9p/9pnet_virtio.ko 2024-03-21T09:43:41.963 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/batman-adv 2024-03-21T09:43:41.963 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/batman-adv/batman-adv.ko 2024-03-21T09:43:41.963 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge 2024-03-21T09:43:41.963 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/br_netfilter.ko 2024-03-21T09:43:41.963 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/bridge.ko 2024-03-21T09:43:41.964 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter 2024-03-21T09:43:41.964 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-03-21T09:43:41.964 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_among.ko 2024-03-21T09:43:41.964 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_arp.ko 2024-03-21T09:43:41.964 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-03-21T09:43:41.964 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-03-21T09:43:41.964 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_ip.ko 2024-03-21T09:43:41.964 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-03-21T09:43:41.964 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_limit.ko 2024-03-21T09:43:41.964 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_log.ko 2024-03-21T09:43:41.964 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_mark.ko 2024-03-21T09:43:41.964 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-03-21T09:43:41.964 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-03-21T09:43:41.964 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-03-21T09:43:41.965 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-03-21T09:43:41.965 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_snat.ko 2024-03-21T09:43:41.965 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_stp.ko 2024-03-21T09:43:41.965 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-03-21T09:43:41.965 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-03-21T09:43:41.965 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-03-21T09:43:41.965 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-03-21T09:43:41.965 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebtables.ko 2024-03-21T09:43:41.965 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-03-21T09:43:41.965 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-03-21T09:43:41.965 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-03-21T09:43:41.965 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ceph 2024-03-21T09:43:41.965 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ceph/libceph.ko 2024-03-21T09:43:41.965 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/core 2024-03-21T09:43:41.965 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/core/failover.ko 2024-03-21T09:43:41.966 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/core/pktgen.ko 2024-03-21T09:43:41.966 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dccp 2024-03-21T09:43:41.966 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dccp/dccp.ko 2024-03-21T09:43:41.966 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dccp/dccp_diag.ko 2024-03-21T09:43:41.966 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dccp/dccp_ipv4.ko 2024-03-21T09:43:41.966 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dccp/dccp_ipv6.ko 2024-03-21T09:43:41.966 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dns_resolver 2024-03-21T09:43:41.966 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dns_resolver/dns_resolver.ko 2024-03-21T09:43:41.966 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa 2024-03-21T09:43:41.966 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa/dsa_core.ko 2024-03-21T09:43:41.966 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa/tag_brcm.ko 2024-03-21T09:43:41.966 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa/tag_dsa.ko 2024-03-21T09:43:41.966 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa/tag_none.ko 2024-03-21T09:43:41.967 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa/tag_trailer.ko 2024-03-21T09:43:41.967 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ieee802154 2024-03-21T09:43:41.967 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ieee802154/6lowpan 2024-03-21T09:43:41.967 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-03-21T09:43:41.967 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ieee802154/ieee802154.ko 2024-03-21T09:43:41.967 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ieee802154/ieee802154_socket.ko 2024-03-21T09:43:41.967 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4 2024-03-21T09:43:41.967 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ah4.ko 2024-03-21T09:43:41.967 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/esp4.ko 2024-03-21T09:43:41.967 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/fou.ko 2024-03-21T09:43:41.967 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/gre.ko 2024-03-21T09:43:41.967 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/inet_diag.ko 2024-03-21T09:43:41.967 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ip_gre.ko 2024-03-21T09:43:41.967 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ip_tunnel.ko 2024-03-21T09:43:41.968 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ip_vti.ko 2024-03-21T09:43:41.968 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ipcomp.ko 2024-03-21T09:43:41.968 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ipip.ko 2024-03-21T09:43:41.968 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter 2024-03-21T09:43:41.968 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/arp_tables.ko 2024-03-21T09:43:41.968 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-03-21T09:43:41.968 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-03-21T09:43:41.968 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-03-21T09:43:41.968 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-03-21T09:43:41.968 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-03-21T09:43:41.968 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-03-21T09:43:41.968 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-03-21T09:43:41.968 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-03-21T09:43:41.968 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-03-21T09:43:41.969 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/iptable_security.ko 2024-03-21T09:43:41.969 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-03-21T09:43:41.969 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-03-21T09:43:41.969 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-03-21T09:43:41.969 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-03-21T09:43:41.969 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-03-21T09:43:41.969 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-03-21T09:43:41.969 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-03-21T09:43:41.969 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-03-21T09:43:41.969 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-03-21T09:43:41.969 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-03-21T09:43:41.969 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_bic.ko 2024-03-21T09:43:41.969 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_dctcp.ko 2024-03-21T09:43:41.969 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_diag.ko 2024-03-21T09:43:41.970 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_highspeed.ko 2024-03-21T09:43:41.970 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_htcp.ko 2024-03-21T09:43:41.970 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_hybla.ko 2024-03-21T09:43:41.970 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_illinois.ko 2024-03-21T09:43:41.970 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_lp.ko 2024-03-21T09:43:41.970 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_scalable.ko 2024-03-21T09:43:41.970 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_vegas.ko 2024-03-21T09:43:41.970 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_veno.ko 2024-03-21T09:43:41.970 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_westwood.ko 2024-03-21T09:43:41.970 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_yeah.ko 2024-03-21T09:43:41.970 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tunnel4.ko 2024-03-21T09:43:41.970 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/udp_diag.ko 2024-03-21T09:43:41.970 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/udp_tunnel.ko 2024-03-21T09:43:41.971 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/xfrm4_tunnel.ko 2024-03-21T09:43:41.971 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6 2024-03-21T09:43:41.971 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/ah6.ko 2024-03-21T09:43:41.971 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/esp6.ko 2024-03-21T09:43:41.971 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/fou6.ko 2024-03-21T09:43:41.971 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/ip6_tunnel.ko 2024-03-21T09:43:41.971 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-03-21T09:43:41.971 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/ip6_vti.ko 2024-03-21T09:43:41.971 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/ipcomp6.ko 2024-03-21T09:43:41.971 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter 2024-03-21T09:43:41.971 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-03-21T09:43:41.971 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-03-21T09:43:41.971 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-03-21T09:43:41.972 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-03-21T09:43:41.972 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-03-21T09:43:41.972 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-03-21T09:43:41.972 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-03-21T09:43:41.972 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-03-21T09:43:41.972 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-03-21T09:43:41.972 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-03-21T09:43:41.972 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-03-21T09:43:41.972 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-03-21T09:43:41.972 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-03-21T09:43:41.972 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-03-21T09:43:41.972 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-03-21T09:43:41.972 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-03-21T09:43:41.972 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-03-21T09:43:41.973 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-03-21T09:43:41.973 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-03-21T09:43:41.973 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-03-21T09:43:41.973 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-03-21T09:43:41.973 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-03-21T09:43:41.973 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-03-21T09:43:41.973 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-03-21T09:43:41.973 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/sit.ko 2024-03-21T09:43:41.973 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/tunnel6.ko 2024-03-21T09:43:41.973 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/xfrm6_tunnel.ko 2024-03-21T09:43:41.973 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/key 2024-03-21T09:43:41.973 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/key/af_key.ko 2024-03-21T09:43:41.973 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp 2024-03-21T09:43:41.973 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_core.ko 2024-03-21T09:43:41.973 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_debugfs.ko 2024-03-21T09:43:41.974 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_eth.ko 2024-03-21T09:43:41.974 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_ip.ko 2024-03-21T09:43:41.974 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_ip6.ko 2024-03-21T09:43:41.974 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_netlink.ko 2024-03-21T09:43:41.974 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_ppp.ko 2024-03-21T09:43:41.974 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/llc 2024-03-21T09:43:41.974 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/llc/llc.ko 2024-03-21T09:43:41.974 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/mac802154 2024-03-21T09:43:41.974 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/mac802154/mac802154.ko 2024-03-21T09:43:41.974 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/mpls 2024-03-21T09:43:41.974 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/mpls/mpls_gso.ko 2024-03-21T09:43:41.974 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter 2024-03-21T09:43:41.974 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset 2024-03-21T09:43:41.975 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set.ko 2024-03-21T09:43:41.975 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-03-21T09:43:41.975 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-03-21T09:43:41.975 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-03-21T09:43:41.975 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-03-21T09:43:41.975 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-03-21T09:43:41.975 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-03-21T09:43:41.975 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-03-21T09:43:41.975 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-03-21T09:43:41.975 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-03-21T09:43:41.975 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-03-21T09:43:41.975 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-03-21T09:43:41.975 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-03-21T09:43:41.976 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-03-21T09:43:41.976 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-03-21T09:43:41.976 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-03-21T09:43:41.976 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs 2024-03-21T09:43:41.976 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs.ko 2024-03-21T09:43:41.976 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-03-21T09:43:41.976 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-03-21T09:43:41.976 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-03-21T09:43:41.976 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-03-21T09:43:41.976 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-03-21T09:43:41.976 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-03-21T09:43:41.976 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-03-21T09:43:41.976 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-03-21T09:43:41.976 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-03-21T09:43:41.976 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-03-21T09:43:41.976 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-03-21T09:43:41.977 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-03-21T09:43:41.977 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-03-21T09:43:41.977 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conncount.ko 2024-03-21T09:43:41.977 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack.ko 2024-03-21T09:43:41.977 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-03-21T09:43:41.977 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-03-21T09:43:41.977 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-03-21T09:43:41.977 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_h323.ko 2024-03-21T09:43:41.977 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_irc.ko 2024-03-21T09:43:41.977 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-03-21T09:43:41.977 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-03-21T09:43:41.977 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-03-21T09:43:41.977 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_sane.ko 2024-03-21T09:43:41.977 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_sip.ko 2024-03-21T09:43:41.978 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-03-21T09:43:41.978 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-03-21T09:43:41.978 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_dup_netdev.ko 2024-03-21T09:43:41.978 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_flow_table.ko 2024-03-21T09:43:41.978 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_flow_table_inet.ko 2024-03-21T09:43:41.978 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_log_syslog.ko 2024-03-21T09:43:41.978 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat.ko 2024-03-21T09:43:41.978 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat_amanda.ko 2024-03-21T09:43:41.978 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat_ftp.ko 2024-03-21T09:43:41.978 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat_irc.ko 2024-03-21T09:43:41.978 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat_sip.ko 2024-03-21T09:43:41.978 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat_tftp.ko 2024-03-21T09:43:41.978 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_synproxy_core.ko 2024-03-21T09:43:41.979 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_tables.ko 2024-03-21T09:43:41.979 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nfnetlink.ko 2024-03-21T09:43:41.979 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nfnetlink_acct.ko 2024-03-21T09:43:41.979 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nfnetlink_log.ko 2024-03-21T09:43:41.979 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nfnetlink_osf.ko 2024-03-21T09:43:41.979 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nfnetlink_queue.ko 2024-03-21T09:43:41.979 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_chain_nat.ko 2024-03-21T09:43:41.979 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_compat.ko 2024-03-21T09:43:41.979 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_ct.ko 2024-03-21T09:43:41.979 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_dup_netdev.ko 2024-03-21T09:43:41.979 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_fib.ko 2024-03-21T09:43:41.979 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_fwd_netdev.ko 2024-03-21T09:43:41.979 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_hash.ko 2024-03-21T09:43:41.979 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_limit.ko 2024-03-21T09:43:41.979 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_log.ko 2024-03-21T09:43:41.979 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_masq.ko 2024-03-21T09:43:41.980 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_nat.ko 2024-03-21T09:43:41.980 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_numgen.ko 2024-03-21T09:43:41.980 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_queue.ko 2024-03-21T09:43:41.980 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_quota.ko 2024-03-21T09:43:41.980 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_redir.ko 2024-03-21T09:43:41.980 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_reject.ko 2024-03-21T09:43:41.980 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_reject_inet.ko 2024-03-21T09:43:41.980 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_socket.ko 2024-03-21T09:43:41.980 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_tunnel.ko 2024-03-21T09:43:41.980 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_AUDIT.ko 2024-03-21T09:43:41.980 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_CHECKSUM.ko 2024-03-21T09:43:41.980 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_CLASSIFY.ko 2024-03-21T09:43:41.981 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-03-21T09:43:41.981 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_CT.ko 2024-03-21T09:43:41.981 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_DSCP.ko 2024-03-21T09:43:41.981 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_HL.ko 2024-03-21T09:43:41.981 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_HMARK.ko 2024-03-21T09:43:41.981 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_IDLETIMER.ko 2024-03-21T09:43:41.981 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_LED.ko 2024-03-21T09:43:41.981 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_LOG.ko 2024-03-21T09:43:41.981 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_MASQUERADE.ko 2024-03-21T09:43:41.981 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_NETMAP.ko 2024-03-21T09:43:41.981 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_NFLOG.ko 2024-03-21T09:43:41.981 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_NFQUEUE.ko 2024-03-21T09:43:41.981 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_RATEEST.ko 2024-03-21T09:43:41.981 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_REDIRECT.ko 2024-03-21T09:43:41.981 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_SECMARK.ko 2024-03-21T09:43:41.982 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_TCPMSS.ko 2024-03-21T09:43:41.982 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-03-21T09:43:41.982 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_TEE.ko 2024-03-21T09:43:41.982 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_TPROXY.ko 2024-03-21T09:43:41.982 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_TRACE.ko 2024-03-21T09:43:41.982 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_addrtype.ko 2024-03-21T09:43:41.982 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_bpf.ko 2024-03-21T09:43:41.982 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_cgroup.ko 2024-03-21T09:43:41.982 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_cluster.ko 2024-03-21T09:43:41.982 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_comment.ko 2024-03-21T09:43:41.982 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_connbytes.ko 2024-03-21T09:43:41.982 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_connlabel.ko 2024-03-21T09:43:41.982 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_connlimit.ko 2024-03-21T09:43:41.982 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_connmark.ko 2024-03-21T09:43:41.983 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_conntrack.ko 2024-03-21T09:43:41.983 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_cpu.ko 2024-03-21T09:43:41.983 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_dccp.ko 2024-03-21T09:43:41.983 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_devgroup.ko 2024-03-21T09:43:41.983 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_dscp.ko 2024-03-21T09:43:41.983 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_ecn.ko 2024-03-21T09:43:41.983 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_esp.ko 2024-03-21T09:43:41.983 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_hashlimit.ko 2024-03-21T09:43:41.983 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_helper.ko 2024-03-21T09:43:41.983 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_hl.ko 2024-03-21T09:43:41.983 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_ipcomp.ko 2024-03-21T09:43:41.983 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_iprange.ko 2024-03-21T09:43:41.983 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_ipvs.ko 2024-03-21T09:43:41.984 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_l2tp.ko 2024-03-21T09:43:41.984 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_length.ko 2024-03-21T09:43:41.984 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_limit.ko 2024-03-21T09:43:41.984 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_mac.ko 2024-03-21T09:43:41.984 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_mark.ko 2024-03-21T09:43:41.984 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_multiport.ko 2024-03-21T09:43:41.984 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_nat.ko 2024-03-21T09:43:41.984 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_nfacct.ko 2024-03-21T09:43:41.984 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_osf.ko 2024-03-21T09:43:41.984 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_owner.ko 2024-03-21T09:43:41.984 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_physdev.ko 2024-03-21T09:43:41.984 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_pkttype.ko 2024-03-21T09:43:41.984 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_policy.ko 2024-03-21T09:43:41.985 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_quota.ko 2024-03-21T09:43:41.985 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_rateest.ko 2024-03-21T09:43:41.985 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_realm.ko 2024-03-21T09:43:41.985 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_recent.ko 2024-03-21T09:43:41.985 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_sctp.ko 2024-03-21T09:43:41.985 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_set.ko 2024-03-21T09:43:41.985 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_socket.ko 2024-03-21T09:43:41.985 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_state.ko 2024-03-21T09:43:41.985 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_statistic.ko 2024-03-21T09:43:41.985 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_string.ko 2024-03-21T09:43:41.985 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_tcpmss.ko 2024-03-21T09:43:41.985 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_time.ko 2024-03-21T09:43:41.985 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_u32.ko 2024-03-21T09:43:41.985 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netlink 2024-03-21T09:43:41.985 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netlink/netlink_diag.ko 2024-03-21T09:43:41.985 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/nsh 2024-03-21T09:43:41.986 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/nsh/nsh.ko 2024-03-21T09:43:41.986 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/openvswitch 2024-03-21T09:43:41.986 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/openvswitch/openvswitch.ko 2024-03-21T09:43:41.986 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/openvswitch/vport-geneve.ko 2024-03-21T09:43:41.986 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/openvswitch/vport-gre.ko 2024-03-21T09:43:41.986 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/openvswitch/vport-vxlan.ko 2024-03-21T09:43:41.986 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/packet 2024-03-21T09:43:41.986 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/packet/af_packet_diag.ko 2024-03-21T09:43:41.986 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/rds 2024-03-21T09:43:41.986 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/rds/rds.ko 2024-03-21T09:43:41.986 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/rds/rds_rdma.ko 2024-03-21T09:43:41.986 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/rds/rds_tcp.ko 2024-03-21T09:43:41.986 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched 2024-03-21T09:43:41.986 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_csum.ko 2024-03-21T09:43:41.987 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_gact.ko 2024-03-21T09:43:41.987 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_mirred.ko 2024-03-21T09:43:41.987 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_nat.ko 2024-03-21T09:43:41.987 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_pedit.ko 2024-03-21T09:43:41.987 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_police.ko 2024-03-21T09:43:41.987 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_simple.ko 2024-03-21T09:43:41.987 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_skbedit.ko 2024-03-21T09:43:41.987 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_vlan.ko 2024-03-21T09:43:41.987 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_basic.ko 2024-03-21T09:43:41.987 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_bpf.ko 2024-03-21T09:43:41.987 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_flow.ko 2024-03-21T09:43:41.987 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_fw.ko 2024-03-21T09:43:41.987 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_route.ko 2024-03-21T09:43:41.988 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_u32.ko 2024-03-21T09:43:41.988 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_cmp.ko 2024-03-21T09:43:41.988 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_ipset.ko 2024-03-21T09:43:41.988 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_meta.ko 2024-03-21T09:43:41.988 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_nbyte.ko 2024-03-21T09:43:41.988 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_text.ko 2024-03-21T09:43:41.988 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_u32.ko 2024-03-21T09:43:41.988 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_choke.ko 2024-03-21T09:43:41.988 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_codel.ko 2024-03-21T09:43:41.988 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_drr.ko 2024-03-21T09:43:41.988 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_fq.ko 2024-03-21T09:43:41.988 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_gred.ko 2024-03-21T09:43:41.988 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_hfsc.ko 2024-03-21T09:43:41.989 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_hhf.ko 2024-03-21T09:43:41.989 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_htb.ko 2024-03-21T09:43:41.989 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_ingress.ko 2024-03-21T09:43:41.989 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_mqprio.ko 2024-03-21T09:43:41.989 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_mqprio_lib.ko 2024-03-21T09:43:41.989 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_multiq.ko 2024-03-21T09:43:41.989 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_netem.ko 2024-03-21T09:43:41.989 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_pie.ko 2024-03-21T09:43:41.989 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_plug.ko 2024-03-21T09:43:41.989 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_prio.ko 2024-03-21T09:43:41.989 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_qfq.ko 2024-03-21T09:43:41.989 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_red.ko 2024-03-21T09:43:41.989 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_sfb.ko 2024-03-21T09:43:41.989 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_sfq.ko 2024-03-21T09:43:41.990 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_tbf.ko 2024-03-21T09:43:41.990 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_teql.ko 2024-03-21T09:43:41.990 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sctp 2024-03-21T09:43:41.990 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sctp/sctp.ko 2024-03-21T09:43:41.990 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sctp/sctp_diag.ko 2024-03-21T09:43:41.990 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc 2024-03-21T09:43:41.990 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/auth_gss 2024-03-21T09:43:41.990 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-03-21T09:43:41.990 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-03-21T09:43:41.990 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/sunrpc.ko 2024-03-21T09:43:41.990 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/xprtrdma 2024-03-21T09:43:41.990 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-03-21T09:43:41.990 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/tipc 2024-03-21T09:43:41.991 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/tipc/diag.ko 2024-03-21T09:43:41.991 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/tipc/tipc.ko 2024-03-21T09:43:41.991 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/unix 2024-03-21T09:43:41.991 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/unix/unix_diag.ko 2024-03-21T09:43:41.991 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock 2024-03-21T09:43:41.991 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-03-21T09:43:41.991 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-03-21T09:43:41.991 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock/vsock.ko 2024-03-21T09:43:41.991 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock/vsock_diag.ko 2024-03-21T09:43:41.991 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock/vsock_loopback.ko 2024-03-21T09:43:41.991 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/xfrm 2024-03-21T09:43:41.991 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/xfrm/xfrm_ipcomp.ko 2024-03-21T09:43:41.991 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security 2024-03-21T09:43:41.992 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security/keys 2024-03-21T09:43:41.992 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security/keys/encrypted-keys 2024-03-21T09:43:41.992 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-03-21T09:43:41.992 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security/keys/trusted-keys 2024-03-21T09:43:41.992 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security/keys/trusted-keys/trusted.ko 2024-03-21T09:43:41.992 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/virt 2024-03-21T09:43:41.992 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/virt/lib 2024-03-21T09:43:41.992 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/virt/lib/irqbypass.ko 2024-03-21T09:43:41.992 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.alias 2024-03-21T09:43:41.992 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.alias.bin 2024-03-21T09:43:41.992 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.builtin 2024-03-21T09:43:41.992 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.builtin.alias.bin 2024-03-21T09:43:41.992 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.builtin.bin 2024-03-21T09:43:41.992 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.builtin.modinfo 2024-03-21T09:43:41.993 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.dep 2024-03-21T09:43:41.993 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.dep.bin 2024-03-21T09:43:41.993 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.devname 2024-03-21T09:43:41.993 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.order 2024-03-21T09:43:41.993 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.softdep 2024-03-21T09:43:41.993 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.symbols 2024-03-21T09:43:41.993 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.symbols.bin 2024-03-21T09:43:41.993 INFO:teuthology.orchestra.run.smithi174.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/vmlinuz 2024-03-21T09:43:41.997 DEBUG:teuthology.task.kernel:get_image_version: 6.8.0-g5ec3e6e88b2e 2024-03-21T09:43:41.997 DEBUG:teuthology.orchestra.run.smithi174:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-03-21T09:43:42.019 INFO:teuthology.orchestra.run.smithi174.stdout:postinstall scriptlet (using /bin/sh): 2024-03-21T09:43:42.019 INFO:teuthology.orchestra.run.smithi174.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-03-21T09:43:42.020 INFO:teuthology.orchestra.run.smithi174.stdout: /usr/bin/kernel-install add 6.8.0-g5ec3e6e88b2e /lib/modules/6.8.0-g5ec3e6e88b2e/vmlinuz 2024-03-21T09:43:42.020 INFO:teuthology.orchestra.run.smithi174.stdout:fi 2024-03-21T09:43:42.020 INFO:teuthology.orchestra.run.smithi174.stdout:for file in vmlinuz System.map config; do 2024-03-21T09:43:42.020 INFO:teuthology.orchestra.run.smithi174.stdout: if ! cmp --silent "/lib/modules/6.8.0-g5ec3e6e88b2e/${file}" "/boot/${file}-6.8.0-g5ec3e6e88b2e"; then 2024-03-21T09:43:42.020 INFO:teuthology.orchestra.run.smithi174.stdout: cp "/lib/modules/6.8.0-g5ec3e6e88b2e/${file}" "/boot/${file}-6.8.0-g5ec3e6e88b2e" 2024-03-21T09:43:42.020 INFO:teuthology.orchestra.run.smithi174.stdout: fi 2024-03-21T09:43:42.020 INFO:teuthology.orchestra.run.smithi174.stdout:done 2024-03-21T09:43:42.020 INFO:teuthology.orchestra.run.smithi174.stdout:preuninstall scriptlet (using /bin/sh): 2024-03-21T09:43:42.020 INFO:teuthology.orchestra.run.smithi174.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2024-03-21T09:43:42.020 INFO:teuthology.orchestra.run.smithi174.stdout:new-kernel-pkg --remove 6.8.0-g5ec3e6e88b2e --rminitrd --initrdfile=/boot/initramfs-6.8.0-g5ec3e6e88b2e.img 2024-03-21T09:43:42.020 INFO:teuthology.orchestra.run.smithi174.stdout:elif [ -x /usr/bin/kernel-install ]; then 2024-03-21T09:43:42.020 INFO:teuthology.orchestra.run.smithi174.stdout:kernel-install remove 6.8.0-g5ec3e6e88b2e 2024-03-21T09:43:42.020 INFO:teuthology.orchestra.run.smithi174.stdout:fi 2024-03-21T09:43:42.021 INFO:teuthology.orchestra.run.smithi174.stdout:postuninstall scriptlet (using /bin/sh): 2024-03-21T09:43:42.021 INFO:teuthology.orchestra.run.smithi174.stdout:if [ -x /sbin/update-bootloader ]; then 2024-03-21T09:43:42.021 INFO:teuthology.orchestra.run.smithi174.stdout:/sbin/update-bootloader --remove 6.8.0-g5ec3e6e88b2e 2024-03-21T09:43:42.021 INFO:teuthology.orchestra.run.smithi174.stdout:fi 2024-03-21T09:43:42.022 DEBUG:teuthology.orchestra.run.smithi174:> sudo rpm -qi grub2-tools 2024-03-21T09:43:42.097 INFO:teuthology.orchestra.run.smithi174.stdout:Name : grub2-tools 2024-03-21T09:43:42.097 INFO:teuthology.orchestra.run.smithi174.stdout:Epoch : 1 2024-03-21T09:43:42.097 INFO:teuthology.orchestra.run.smithi174.stdout:Version : 2.06 2024-03-21T09:43:42.098 INFO:teuthology.orchestra.run.smithi174.stdout:Release : 68.el9 2024-03-21T09:43:42.098 INFO:teuthology.orchestra.run.smithi174.stdout:Architecture: x86_64 2024-03-21T09:43:42.098 INFO:teuthology.orchestra.run.smithi174.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-21T09:43:42.098 INFO:teuthology.orchestra.run.smithi174.stdout:Group : Unspecified 2024-03-21T09:43:42.098 INFO:teuthology.orchestra.run.smithi174.stdout:Size : 8264254 2024-03-21T09:43:42.098 INFO:teuthology.orchestra.run.smithi174.stdout:License : GPLv3+ 2024-03-21T09:43:42.098 INFO:teuthology.orchestra.run.smithi174.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-21T09:43:42.098 INFO:teuthology.orchestra.run.smithi174.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-21T09:43:42.098 INFO:teuthology.orchestra.run.smithi174.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-21T09:43:42.098 INFO:teuthology.orchestra.run.smithi174.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-21T09:43:42.098 INFO:teuthology.orchestra.run.smithi174.stdout:Packager : builder@centos.org 2024-03-21T09:43:42.098 INFO:teuthology.orchestra.run.smithi174.stdout:Vendor : CentOS 2024-03-21T09:43:42.098 INFO:teuthology.orchestra.run.smithi174.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-21T09:43:42.098 INFO:teuthology.orchestra.run.smithi174.stdout:Summary : Support tools for GRUB. 2024-03-21T09:43:42.098 INFO:teuthology.orchestra.run.smithi174.stdout:Description : 2024-03-21T09:43:42.099 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-21T09:43:42.099 INFO:teuthology.orchestra.run.smithi174.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-21T09:43:42.099 INFO:teuthology.orchestra.run.smithi174.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-21T09:43:42.099 INFO:teuthology.orchestra.run.smithi174.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-21T09:43:42.099 INFO:teuthology.orchestra.run.smithi174.stdout:hardware devices. 2024-03-21T09:43:42.099 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-21T09:43:42.099 INFO:teuthology.orchestra.run.smithi174.stdout:This subpackage provides tools for support of all platforms. 2024-03-21T09:43:42.100 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-21T09:43:42.100 INFO:teuthology.task.kernel:Updating grub on smithi174 to boot 6.8.0-g5ec3e6e88b2e 2024-03-21T09:43:42.100 DEBUG:teuthology.orchestra.run.smithi174:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-21T09:43:42.709 INFO:teuthology.orchestra.run.smithi174.stderr:Generating grub configuration file ... 2024-03-21T09:43:44.045 INFO:teuthology.orchestra.run.smithi174.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-21T09:43:44.076 INFO:teuthology.orchestra.run.smithi174.stderr:done 2024-03-21T09:43:44.078 DEBUG:teuthology.orchestra.run.smithi174:> mktemp 2024-03-21T09:43:44.093 INFO:teuthology.orchestra.run.smithi174.stdout:/tmp/tmp.vqDYbkpOr2 2024-03-21T09:43:44.093 DEBUG:teuthology.orchestra.run.smithi174:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.vqDYbkpOr2 2024-03-21T09:43:44.164 DEBUG:teuthology.orchestra.run.smithi174:> sudo chmod 0666 /tmp/tmp.vqDYbkpOr2 2024-03-21T09:43:44.300 DEBUG:teuthology.orchestra.remote:smithi174:/tmp/tmp.vqDYbkpOr2 is 6KB 2024-03-21T09:43:44.349 DEBUG:teuthology.orchestra.run.smithi174:> rm -fr /tmp/tmp.vqDYbkpOr2 2024-03-21T09:43:44.363 DEBUG:teuthology.orchestra.run.smithi174:> sudo /bin/ls /boot/loader/entries || true 2024-03-21T09:43:44.428 INFO:teuthology.orchestra.run.smithi174.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-21T09:43:44.428 INFO:teuthology.orchestra.run.smithi174.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-21T09:43:44.429 INFO:teuthology.orchestra.run.smithi174.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-21T09:43:44.429 INFO:teuthology.orchestra.run.smithi174.stdout:15949e3c88704300882482eddfd7baae-6.8.0-g5ec3e6e88b2e.conf 2024-03-21T09:43:44.430 DEBUG:teuthology.orchestra.run.smithi174:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.8.0-g5ec3e6e88b2e 2024-03-21T09:43:44.578 DEBUG:teuthology.orchestra.run.smithi174:> sudo shutdown -r now 2024-03-21T09:44:01.322 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi153.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-03-21T09:44:01.322 DEBUG:teuthology.orchestra.run.smithi153:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-03-21T09:44:01.397 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e 2024-03-21T09:44:01.397 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/System.map 2024-03-21T09:44:01.397 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/config 2024-03-21T09:44:01.397 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel 2024-03-21T09:44:01.398 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch 2024-03-21T09:44:01.398 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86 2024-03-21T09:44:01.398 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto 2024-03-21T09:44:01.398 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-03-21T09:44:01.398 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-03-21T09:44:01.398 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-03-21T09:44:01.398 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-03-21T09:44:01.398 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-03-21T09:44:01.398 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-03-21T09:44:01.398 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-03-21T09:44:01.398 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/crc32c-intel.ko 2024-03-21T09:44:01.398 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-03-21T09:44:01.399 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-03-21T09:44:01.399 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-03-21T09:44:01.399 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-03-21T09:44:01.399 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/serpent-avx2.ko 2024-03-21T09:44:01.399 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-03-21T09:44:01.399 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-03-21T09:44:01.399 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-03-21T09:44:01.399 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-03-21T09:44:01.399 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-03-21T09:44:01.400 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-03-21T09:44:01.400 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-03-21T09:44:01.400 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kernel 2024-03-21T09:44:01.400 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kernel/cpu 2024-03-21T09:44:01.400 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kernel/cpu/mce 2024-03-21T09:44:01.400 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-03-21T09:44:01.400 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kvm 2024-03-21T09:44:01.400 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kvm/kvm-amd.ko 2024-03-21T09:44:01.400 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kvm/kvm-intel.ko 2024-03-21T09:44:01.400 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/arch/x86/kvm/kvm.ko 2024-03-21T09:44:01.400 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto 2024-03-21T09:44:01.400 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/ansi_cprng.ko 2024-03-21T09:44:01.401 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/anubis.ko 2024-03-21T09:44:01.401 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/arc4.ko 2024-03-21T09:44:01.401 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx 2024-03-21T09:44:01.401 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/async_memcpy.ko 2024-03-21T09:44:01.401 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/async_pq.ko 2024-03-21T09:44:01.401 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/async_raid6_recov.ko 2024-03-21T09:44:01.401 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/async_tx.ko 2024-03-21T09:44:01.401 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/async_xor.ko 2024-03-21T09:44:01.401 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/async_tx/raid6test.ko 2024-03-21T09:44:01.401 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/authenc.ko 2024-03-21T09:44:01.401 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/authencesn.ko 2024-03-21T09:44:01.402 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/blake2b_generic.ko 2024-03-21T09:44:01.402 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/blowfish_common.ko 2024-03-21T09:44:01.402 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/blowfish_generic.ko 2024-03-21T09:44:01.402 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/camellia_generic.ko 2024-03-21T09:44:01.402 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/cast5_generic.ko 2024-03-21T09:44:01.402 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/cast6_generic.ko 2024-03-21T09:44:01.402 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/cast_common.ko 2024-03-21T09:44:01.402 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/ccm.ko 2024-03-21T09:44:01.402 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/cmac.ko 2024-03-21T09:44:01.402 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/crc32_generic.ko 2024-03-21T09:44:01.402 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/crypto_engine.ko 2024-03-21T09:44:01.402 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/crypto_user.ko 2024-03-21T09:44:01.402 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/cts.ko 2024-03-21T09:44:01.403 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/des_generic.ko 2024-03-21T09:44:01.403 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/dh_generic.ko 2024-03-21T09:44:01.403 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/echainiv.ko 2024-03-21T09:44:01.403 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/essiv.ko 2024-03-21T09:44:01.403 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/fcrypt.ko 2024-03-21T09:44:01.403 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/khazad.ko 2024-03-21T09:44:01.403 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/lz4.ko 2024-03-21T09:44:01.403 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/lz4hc.ko 2024-03-21T09:44:01.403 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/md4.ko 2024-03-21T09:44:01.403 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/michael_mic.ko 2024-03-21T09:44:01.403 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/pcbc.ko 2024-03-21T09:44:01.403 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/pcrypt.ko 2024-03-21T09:44:01.404 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/rmd160.ko 2024-03-21T09:44:01.404 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/seed.ko 2024-03-21T09:44:01.404 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/serpent_generic.ko 2024-03-21T09:44:01.404 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/tcrypt.ko 2024-03-21T09:44:01.404 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/tea.ko 2024-03-21T09:44:01.404 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/twofish_common.ko 2024-03-21T09:44:01.404 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/twofish_generic.ko 2024-03-21T09:44:01.404 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/vmac.ko 2024-03-21T09:44:01.404 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/wp512.ko 2024-03-21T09:44:01.404 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/xcbc.ko 2024-03-21T09:44:01.404 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/xor.ko 2024-03-21T09:44:01.405 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/crypto/xxhash_generic.ko 2024-03-21T09:44:01.405 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers 2024-03-21T09:44:01.405 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi 2024-03-21T09:44:01.405 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/acpi_ipmi.ko 2024-03-21T09:44:01.405 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/acpi_pad.ko 2024-03-21T09:44:01.405 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/custom_method.ko 2024-03-21T09:44:01.405 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/ec_sys.ko 2024-03-21T09:44:01.405 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/platform_profile.ko 2024-03-21T09:44:01.405 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/sbs.ko 2024-03-21T09:44:01.405 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/sbshc.ko 2024-03-21T09:44:01.405 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/acpi/video.ko 2024-03-21T09:44:01.405 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata 2024-03-21T09:44:01.406 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/acard-ahci.ko 2024-03-21T09:44:01.406 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/ahci_platform.ko 2024-03-21T09:44:01.406 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/ata_generic.ko 2024-03-21T09:44:01.406 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/libahci_platform.ko 2024-03-21T09:44:01.406 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_acpi.ko 2024-03-21T09:44:01.406 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_ali.ko 2024-03-21T09:44:01.406 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_amd.ko 2024-03-21T09:44:01.406 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_artop.ko 2024-03-21T09:44:01.406 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_atiixp.ko 2024-03-21T09:44:01.406 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_atp867x.ko 2024-03-21T09:44:01.406 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_cmd640.ko 2024-03-21T09:44:01.407 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_cmd64x.ko 2024-03-21T09:44:01.407 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_cypress.ko 2024-03-21T09:44:01.407 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_efar.ko 2024-03-21T09:44:01.407 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_hpt366.ko 2024-03-21T09:44:01.407 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_hpt37x.ko 2024-03-21T09:44:01.407 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_hpt3x2n.ko 2024-03-21T09:44:01.407 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_hpt3x3.ko 2024-03-21T09:44:01.407 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_it8213.ko 2024-03-21T09:44:01.407 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_it821x.ko 2024-03-21T09:44:01.407 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_jmicron.ko 2024-03-21T09:44:01.407 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_marvell.ko 2024-03-21T09:44:01.407 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_mpiix.ko 2024-03-21T09:44:01.407 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_netcell.ko 2024-03-21T09:44:01.407 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_ninja32.ko 2024-03-21T09:44:01.408 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_ns87410.ko 2024-03-21T09:44:01.408 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_ns87415.ko 2024-03-21T09:44:01.408 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_oldpiix.ko 2024-03-21T09:44:01.408 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_opti.ko 2024-03-21T09:44:01.408 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_optidma.ko 2024-03-21T09:44:01.408 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_pcmcia.ko 2024-03-21T09:44:01.408 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_pdc2027x.ko 2024-03-21T09:44:01.408 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-03-21T09:44:01.408 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_piccolo.ko 2024-03-21T09:44:01.408 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_rdc.ko 2024-03-21T09:44:01.408 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_sch.ko 2024-03-21T09:44:01.408 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_serverworks.ko 2024-03-21T09:44:01.408 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_sil680.ko 2024-03-21T09:44:01.408 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_sis.ko 2024-03-21T09:44:01.409 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_sl82c105.ko 2024-03-21T09:44:01.409 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_triflex.ko 2024-03-21T09:44:01.409 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pata_via.ko 2024-03-21T09:44:01.409 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/pdc_adma.ko 2024-03-21T09:44:01.409 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_inic162x.ko 2024-03-21T09:44:01.409 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_mv.ko 2024-03-21T09:44:01.409 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_nv.ko 2024-03-21T09:44:01.409 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_promise.ko 2024-03-21T09:44:01.409 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_qstor.ko 2024-03-21T09:44:01.409 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_sil.ko 2024-03-21T09:44:01.409 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_sil24.ko 2024-03-21T09:44:01.410 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_sis.ko 2024-03-21T09:44:01.410 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_svw.ko 2024-03-21T09:44:01.410 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_sx4.ko 2024-03-21T09:44:01.410 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_uli.ko 2024-03-21T09:44:01.410 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_via.ko 2024-03-21T09:44:01.410 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ata/sata_vsc.ko 2024-03-21T09:44:01.410 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/auxdisplay 2024-03-21T09:44:01.410 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/auxdisplay/ks0108.ko 2024-03-21T09:44:01.410 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/base 2024-03-21T09:44:01.410 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/base/regmap 2024-03-21T09:44:01.410 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/base/regmap/regmap-i2c.ko 2024-03-21T09:44:01.410 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/base/regmap/regmap-mmio.ko 2024-03-21T09:44:01.410 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/bcma 2024-03-21T09:44:01.411 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/bcma/bcma.ko 2024-03-21T09:44:01.411 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block 2024-03-21T09:44:01.411 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/aoe 2024-03-21T09:44:01.411 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/aoe/aoe.ko 2024-03-21T09:44:01.411 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/brd.ko 2024-03-21T09:44:01.411 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/drbd 2024-03-21T09:44:01.411 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/drbd/drbd.ko 2024-03-21T09:44:01.411 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/floppy.ko 2024-03-21T09:44:01.411 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/loop.ko 2024-03-21T09:44:01.411 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/mtip32xx 2024-03-21T09:44:01.411 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-03-21T09:44:01.411 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/nbd.ko 2024-03-21T09:44:01.411 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/null_blk 2024-03-21T09:44:01.411 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/null_blk/null_blk.ko 2024-03-21T09:44:01.412 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/pktcdvd.ko 2024-03-21T09:44:01.412 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/rbd.ko 2024-03-21T09:44:01.412 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/virtio_blk.ko 2024-03-21T09:44:01.412 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/xen-blkback 2024-03-21T09:44:01.412 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-03-21T09:44:01.412 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/xen-blkfront.ko 2024-03-21T09:44:01.412 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/zram 2024-03-21T09:44:01.412 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/block/zram/zram.ko 2024-03-21T09:44:01.412 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char 2024-03-21T09:44:01.412 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hangcheck-timer.ko 2024-03-21T09:44:01.412 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random 2024-03-21T09:44:01.412 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random/amd-rng.ko 2024-03-21T09:44:01.412 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random/intel-rng.ko 2024-03-21T09:44:01.412 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-03-21T09:44:01.413 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random/via-rng.ko 2024-03-21T09:44:01.413 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/hw_random/virtio-rng.ko 2024-03-21T09:44:01.413 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi 2024-03-21T09:44:01.413 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-03-21T09:44:01.413 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-03-21T09:44:01.413 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-03-21T09:44:01.413 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_si.ko 2024-03-21T09:44:01.413 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-03-21T09:44:01.413 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-03-21T09:44:01.413 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/lp.ko 2024-03-21T09:44:01.413 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/mwave 2024-03-21T09:44:01.413 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/mwave/mwave.ko 2024-03-21T09:44:01.413 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/ppdev.ko 2024-03-21T09:44:01.413 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tlclk.ko 2024-03-21T09:44:01.413 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm 2024-03-21T09:44:01.414 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm.ko 2024-03-21T09:44:01.414 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm_atmel.ko 2024-03-21T09:44:01.414 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm_infineon.ko 2024-03-21T09:44:01.414 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm_nsc.ko 2024-03-21T09:44:01.414 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm_tis.ko 2024-03-21T09:44:01.414 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-03-21T09:44:01.414 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/uv_mmtimer.ko 2024-03-21T09:44:01.414 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/char/virtio_console.ko 2024-03-21T09:44:01.414 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq 2024-03-21T09:44:01.414 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-03-21T09:44:01.414 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-03-21T09:44:01.414 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/p4-clockmod.ko 2024-03-21T09:44:01.414 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-03-21T09:44:01.414 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/powernow-k8.ko 2024-03-21T09:44:01.415 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/cpufreq/speedstep-lib.ko 2024-03-21T09:44:01.415 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto 2024-03-21T09:44:01.415 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/ccp 2024-03-21T09:44:01.415 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-03-21T09:44:01.415 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/ccp/ccp.ko 2024-03-21T09:44:01.415 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel 2024-03-21T09:44:01.415 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel/qat 2024-03-21T09:44:01.415 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel/qat/qat_common 2024-03-21T09:44:01.415 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-03-21T09:44:01.415 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-03-21T09:44:01.415 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-03-21T09:44:01.415 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/padlock-aes.ko 2024-03-21T09:44:01.415 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/padlock-sha.ko 2024-03-21T09:44:01.415 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/virtio 2024-03-21T09:44:01.415 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-03-21T09:44:01.416 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dca 2024-03-21T09:44:01.416 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dca/dca.ko 2024-03-21T09:44:01.416 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/devfreq 2024-03-21T09:44:01.416 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-03-21T09:44:01.416 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dma 2024-03-21T09:44:01.416 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dma/dw 2024-03-21T09:44:01.416 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dma/dw/dw_dmac.ko 2024-03-21T09:44:01.416 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dma/ioat 2024-03-21T09:44:01.416 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/dma/ioat/ioatdma.ko 2024-03-21T09:44:01.416 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac 2024-03-21T09:44:01.416 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/amd64_edac.ko 2024-03-21T09:44:01.416 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/e752x_edac.ko 2024-03-21T09:44:01.416 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/edac_mce_amd.ko 2024-03-21T09:44:01.417 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i3000_edac.ko 2024-03-21T09:44:01.417 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i3200_edac.ko 2024-03-21T09:44:01.417 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i5100_edac.ko 2024-03-21T09:44:01.417 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i5400_edac.ko 2024-03-21T09:44:01.417 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i7300_edac.ko 2024-03-21T09:44:01.417 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i7core_edac.ko 2024-03-21T09:44:01.417 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/i82975x_edac.ko 2024-03-21T09:44:01.417 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/ie31200_edac.ko 2024-03-21T09:44:01.417 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/sb_edac.ko 2024-03-21T09:44:01.417 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/edac/x38_edac.ko 2024-03-21T09:44:01.417 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/firmware 2024-03-21T09:44:01.417 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/firmware/edd.ko 2024-03-21T09:44:01.417 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/firmware/iscsi_ibft.ko 2024-03-21T09:44:01.418 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio 2024-03-21T09:44:01.418 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio/gpio-ich.ko 2024-03-21T09:44:01.418 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/gpio/gpio-viperboard.ko 2024-03-21T09:44:01.418 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid 2024-03-21T09:44:01.418 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-appleir.ko 2024-03-21T09:44:01.418 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-aureal.ko 2024-03-21T09:44:01.418 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-axff.ko 2024-03-21T09:44:01.418 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-dr.ko 2024-03-21T09:44:01.418 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-elecom.ko 2024-03-21T09:44:01.418 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-elo.ko 2024-03-21T09:44:01.418 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-emsff.ko 2024-03-21T09:44:01.418 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-gaff.ko 2024-03-21T09:44:01.418 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-gt683r.ko 2024-03-21T09:44:01.418 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-gyration.ko 2024-03-21T09:44:01.419 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-holtek-kbd.ko 2024-03-21T09:44:01.419 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-holtek-mouse.ko 2024-03-21T09:44:01.419 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-holtekff.ko 2024-03-21T09:44:01.419 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-hyperv.ko 2024-03-21T09:44:01.419 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-icade.ko 2024-03-21T09:44:01.419 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-keytouch.ko 2024-03-21T09:44:01.419 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-kye.ko 2024-03-21T09:44:01.419 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-lcpower.ko 2024-03-21T09:44:01.419 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-led.ko 2024-03-21T09:44:01.419 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-lenovo.ko 2024-03-21T09:44:01.419 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-logitech-dj.ko 2024-03-21T09:44:01.419 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-03-21T09:44:01.419 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-multitouch.ko 2024-03-21T09:44:01.419 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-ortek.ko 2024-03-21T09:44:01.419 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-penmount.ko 2024-03-21T09:44:01.420 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-petalynx.ko 2024-03-21T09:44:01.420 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-picolcd.ko 2024-03-21T09:44:01.420 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-pl.ko 2024-03-21T09:44:01.420 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-plantronics.ko 2024-03-21T09:44:01.420 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-primax.ko 2024-03-21T09:44:01.420 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-rmi.ko 2024-03-21T09:44:01.420 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-arvo.ko 2024-03-21T09:44:01.420 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-common.ko 2024-03-21T09:44:01.420 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-isku.ko 2024-03-21T09:44:01.420 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-kone.ko 2024-03-21T09:44:01.420 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-03-21T09:44:01.420 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-konepure.ko 2024-03-21T09:44:01.421 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-03-21T09:44:01.421 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-lua.ko 2024-03-21T09:44:01.421 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-pyra.ko 2024-03-21T09:44:01.421 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-ryos.ko 2024-03-21T09:44:01.421 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat-savu.ko 2024-03-21T09:44:01.421 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-roccat.ko 2024-03-21T09:44:01.421 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-saitek.ko 2024-03-21T09:44:01.421 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-samsung.ko 2024-03-21T09:44:01.421 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-sensor-hub.ko 2024-03-21T09:44:01.421 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-sjoy.ko 2024-03-21T09:44:01.421 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-sony.ko 2024-03-21T09:44:01.421 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-speedlink.ko 2024-03-21T09:44:01.421 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-steelseries.ko 2024-03-21T09:44:01.421 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-sunplus.ko 2024-03-21T09:44:01.422 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-thrustmaster.ko 2024-03-21T09:44:01.422 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-tivo.ko 2024-03-21T09:44:01.422 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-tmff.ko 2024-03-21T09:44:01.422 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-topseed.ko 2024-03-21T09:44:01.422 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-twinhan.ko 2024-03-21T09:44:01.422 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-uclogic.ko 2024-03-21T09:44:01.422 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-waltop.ko 2024-03-21T09:44:01.422 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-wiimote.ko 2024-03-21T09:44:01.422 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-xinmo.ko 2024-03-21T09:44:01.422 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-zpff.ko 2024-03-21T09:44:01.422 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/hid-zydacron.ko 2024-03-21T09:44:01.422 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/uhid.ko 2024-03-21T09:44:01.422 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hid/wacom.ko 2024-03-21T09:44:01.423 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hv 2024-03-21T09:44:01.423 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hv/hv_balloon.ko 2024-03-21T09:44:01.423 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hv/hv_utils.ko 2024-03-21T09:44:01.423 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hv/hv_vmbus.ko 2024-03-21T09:44:01.423 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon 2024-03-21T09:44:01.423 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/abituguru.ko 2024-03-21T09:44:01.423 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/abituguru3.ko 2024-03-21T09:44:01.423 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/acpi_power_meter.ko 2024-03-21T09:44:01.423 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ad7414.ko 2024-03-21T09:44:01.423 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ad7418.ko 2024-03-21T09:44:01.423 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adc128d818.ko 2024-03-21T09:44:01.423 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adm1025.ko 2024-03-21T09:44:01.423 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adm1026.ko 2024-03-21T09:44:01.424 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adm1029.ko 2024-03-21T09:44:01.424 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adm1031.ko 2024-03-21T09:44:01.424 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adm9240.ko 2024-03-21T09:44:01.424 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ads7828.ko 2024-03-21T09:44:01.424 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7410.ko 2024-03-21T09:44:01.424 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7411.ko 2024-03-21T09:44:01.424 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7462.ko 2024-03-21T09:44:01.424 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7470.ko 2024-03-21T09:44:01.424 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7475.ko 2024-03-21T09:44:01.424 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/adt7x10.ko 2024-03-21T09:44:01.424 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/amc6821.ko 2024-03-21T09:44:01.424 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/applesmc.ko 2024-03-21T09:44:01.424 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/asb100.ko 2024-03-21T09:44:01.425 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/asc7621.ko 2024-03-21T09:44:01.425 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/asus_atk0110.ko 2024-03-21T09:44:01.425 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/atxp1.ko 2024-03-21T09:44:01.425 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/coretemp.ko 2024-03-21T09:44:01.425 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-03-21T09:44:01.425 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/dme1737.ko 2024-03-21T09:44:01.425 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ds1621.ko 2024-03-21T09:44:01.425 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ds620.ko 2024-03-21T09:44:01.425 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/emc1403.ko 2024-03-21T09:44:01.425 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/emc6w201.ko 2024-03-21T09:44:01.425 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/f71805f.ko 2024-03-21T09:44:01.425 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/f71882fg.ko 2024-03-21T09:44:01.425 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/f75375s.ko 2024-03-21T09:44:01.425 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/fam15h_power.ko 2024-03-21T09:44:01.426 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/fschmd.ko 2024-03-21T09:44:01.426 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/g760a.ko 2024-03-21T09:44:01.426 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/g762.ko 2024-03-21T09:44:01.426 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/gl518sm.ko 2024-03-21T09:44:01.426 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/gl520sm.ko 2024-03-21T09:44:01.426 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/hwmon-vid.ko 2024-03-21T09:44:01.426 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/i5500_temp.ko 2024-03-21T09:44:01.426 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/i5k_amb.ko 2024-03-21T09:44:01.426 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ibmaem.ko 2024-03-21T09:44:01.426 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ibmpex.ko 2024-03-21T09:44:01.426 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ina209.ko 2024-03-21T09:44:01.426 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ina2xx.ko 2024-03-21T09:44:01.427 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/it87.ko 2024-03-21T09:44:01.427 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/k10temp.ko 2024-03-21T09:44:01.427 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/k8temp.ko 2024-03-21T09:44:01.427 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lineage-pem.ko 2024-03-21T09:44:01.427 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm63.ko 2024-03-21T09:44:01.427 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm73.ko 2024-03-21T09:44:01.427 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm75.ko 2024-03-21T09:44:01.427 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm77.ko 2024-03-21T09:44:01.427 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm78.ko 2024-03-21T09:44:01.427 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm80.ko 2024-03-21T09:44:01.428 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm83.ko 2024-03-21T09:44:01.428 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm85.ko 2024-03-21T09:44:01.428 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm87.ko 2024-03-21T09:44:01.428 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm90.ko 2024-03-21T09:44:01.428 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm92.ko 2024-03-21T09:44:01.428 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm93.ko 2024-03-21T09:44:01.428 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm95234.ko 2024-03-21T09:44:01.428 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm95241.ko 2024-03-21T09:44:01.428 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/lm95245.ko 2024-03-21T09:44:01.428 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc2945.ko 2024-03-21T09:44:01.428 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4151.ko 2024-03-21T09:44:01.428 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4215.ko 2024-03-21T09:44:01.429 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4222.ko 2024-03-21T09:44:01.429 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4245.ko 2024-03-21T09:44:01.429 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4260.ko 2024-03-21T09:44:01.429 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/ltc4261.ko 2024-03-21T09:44:01.429 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max16065.ko 2024-03-21T09:44:01.429 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max1619.ko 2024-03-21T09:44:01.429 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max1668.ko 2024-03-21T09:44:01.429 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max197.ko 2024-03-21T09:44:01.430 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max6639.ko 2024-03-21T09:44:01.430 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max6650.ko 2024-03-21T09:44:01.430 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/max6697.ko 2024-03-21T09:44:01.430 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/mcp3021.ko 2024-03-21T09:44:01.430 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/nct6683.ko 2024-03-21T09:44:01.430 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/nct6775-core.ko 2024-03-21T09:44:01.430 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/nct6775.ko 2024-03-21T09:44:01.430 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/nct7802.ko 2024-03-21T09:44:01.430 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pc87360.ko 2024-03-21T09:44:01.431 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pc87427.ko 2024-03-21T09:44:01.431 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pcf8591.ko 2024-03-21T09:44:01.431 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus 2024-03-21T09:44:01.431 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-03-21T09:44:01.431 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-03-21T09:44:01.431 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-03-21T09:44:01.431 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/max16064.ko 2024-03-21T09:44:01.431 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/max34440.ko 2024-03-21T09:44:01.431 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/max8688.ko 2024-03-21T09:44:01.431 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-03-21T09:44:01.431 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-03-21T09:44:01.431 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-03-21T09:44:01.432 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-03-21T09:44:01.432 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-03-21T09:44:01.432 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-03-21T09:44:01.432 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/powr1220.ko 2024-03-21T09:44:01.432 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sch5627.ko 2024-03-21T09:44:01.432 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sch5636.ko 2024-03-21T09:44:01.432 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sch56xx-common.ko 2024-03-21T09:44:01.432 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sht15.ko 2024-03-21T09:44:01.432 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sht21.ko 2024-03-21T09:44:01.432 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/shtc1.ko 2024-03-21T09:44:01.432 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/sis5595.ko 2024-03-21T09:44:01.432 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/smsc47b397.ko 2024-03-21T09:44:01.432 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/smsc47m1.ko 2024-03-21T09:44:01.433 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/smsc47m192.ko 2024-03-21T09:44:01.433 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/thmc50.ko 2024-03-21T09:44:01.433 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/tmp102.ko 2024-03-21T09:44:01.433 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/tmp103.ko 2024-03-21T09:44:01.433 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/tmp401.ko 2024-03-21T09:44:01.433 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/tmp421.ko 2024-03-21T09:44:01.433 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/via-cputemp.ko 2024-03-21T09:44:01.433 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/via686a.ko 2024-03-21T09:44:01.433 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/vt1211.ko 2024-03-21T09:44:01.433 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/vt8231.ko 2024-03-21T09:44:01.433 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83627ehf.ko 2024-03-21T09:44:01.433 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83627hf.ko 2024-03-21T09:44:01.433 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83781d.ko 2024-03-21T09:44:01.433 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83791d.ko 2024-03-21T09:44:01.434 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83792d.ko 2024-03-21T09:44:01.434 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83793.ko 2024-03-21T09:44:01.434 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83795.ko 2024-03-21T09:44:01.434 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83l785ts.ko 2024-03-21T09:44:01.434 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/hwmon/w83l786ng.ko 2024-03-21T09:44:01.434 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c 2024-03-21T09:44:01.434 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/algos 2024-03-21T09:44:01.434 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-03-21T09:44:01.434 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-03-21T09:44:01.434 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses 2024-03-21T09:44:01.434 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-03-21T09:44:01.434 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-03-21T09:44:01.434 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-03-21T09:44:01.435 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-03-21T09:44:01.435 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-03-21T09:44:01.435 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-03-21T09:44:01.435 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-03-21T09:44:01.435 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-03-21T09:44:01.435 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-i801.ko 2024-03-21T09:44:01.435 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-isch.ko 2024-03-21T09:44:01.435 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-03-21T09:44:01.435 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-03-21T09:44:01.435 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-03-21T09:44:01.435 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-parport.ko 2024-03-21T09:44:01.435 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-03-21T09:44:01.435 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-03-21T09:44:01.436 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-03-21T09:44:01.436 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-03-21T09:44:01.436 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-03-21T09:44:01.436 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-03-21T09:44:01.436 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-via.ko 2024-03-21T09:44:01.436 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-03-21T09:44:01.436 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-03-21T09:44:01.436 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/i2c-dev.ko 2024-03-21T09:44:01.436 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/i2c-mux.ko 2024-03-21T09:44:01.436 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-03-21T09:44:01.436 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/i2c-smbus.ko 2024-03-21T09:44:01.436 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/i2c/i2c-stub.ko 2024-03-21T09:44:01.436 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband 2024-03-21T09:44:01.437 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core 2024-03-21T09:44:01.437 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/ib_cm.ko 2024-03-21T09:44:01.437 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/ib_core.ko 2024-03-21T09:44:01.437 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/ib_umad.ko 2024-03-21T09:44:01.437 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-03-21T09:44:01.437 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/iw_cm.ko 2024-03-21T09:44:01.437 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/rdma_cm.ko 2024-03-21T09:44:01.437 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-03-21T09:44:01.437 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw 2024-03-21T09:44:01.437 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/cxgb4 2024-03-21T09:44:01.438 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-03-21T09:44:01.438 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mlx4 2024-03-21T09:44:01.438 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-03-21T09:44:01.438 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mlx5 2024-03-21T09:44:01.438 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-03-21T09:44:01.438 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mthca 2024-03-21T09:44:01.438 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-03-21T09:44:01.438 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp 2024-03-21T09:44:01.438 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/ipoib 2024-03-21T09:44:01.438 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-03-21T09:44:01.439 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/iser 2024-03-21T09:44:01.439 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-03-21T09:44:01.439 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/isert 2024-03-21T09:44:01.439 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-03-21T09:44:01.439 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/srp 2024-03-21T09:44:01.439 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-03-21T09:44:01.439 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/srpt 2024-03-21T09:44:01.439 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-03-21T09:44:01.439 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input 2024-03-21T09:44:01.439 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/gameport 2024-03-21T09:44:01.439 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-03-21T09:44:01.439 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/gameport/fm801-gp.ko 2024-03-21T09:44:01.439 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/gameport/gameport.ko 2024-03-21T09:44:01.439 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/joydev.ko 2024-03-21T09:44:01.439 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/keyboard 2024-03-21T09:44:01.440 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/keyboard/gpio_keys.ko 2024-03-21T09:44:01.440 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse 2024-03-21T09:44:01.440 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/appletouch.ko 2024-03-21T09:44:01.440 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/bcm5974.ko 2024-03-21T09:44:01.440 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/cyapatp.ko 2024-03-21T09:44:01.440 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/elan_i2c.ko 2024-03-21T09:44:01.440 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/sermouse.ko 2024-03-21T09:44:01.440 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-03-21T09:44:01.440 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/synaptics_usb.ko 2024-03-21T09:44:01.440 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/mouse/vsxxxaa.ko 2024-03-21T09:44:01.440 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/rmi4 2024-03-21T09:44:01.440 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/rmi4/rmi_core.ko 2024-03-21T09:44:01.440 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio 2024-03-21T09:44:01.441 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio/altera_ps2.ko 2024-03-21T09:44:01.441 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio/arc_ps2.ko 2024-03-21T09:44:01.441 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-03-21T09:44:01.441 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/serio/serio_raw.ko 2024-03-21T09:44:01.441 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/input/sparse-keymap.ko 2024-03-21T09:44:01.441 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds 2024-03-21T09:44:01.441 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-blinkm.ko 2024-03-21T09:44:01.441 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-lm3530.ko 2024-03-21T09:44:01.441 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-lp3944.ko 2024-03-21T09:44:01.441 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/leds-ss4200.ko 2024-03-21T09:44:01.441 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger 2024-03-21T09:44:01.441 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-03-21T09:44:01.441 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-03-21T09:44:01.441 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-03-21T09:44:01.442 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-03-21T09:44:01.442 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-03-21T09:44:01.442 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-03-21T09:44:01.442 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-03-21T09:44:01.442 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-03-21T09:44:01.442 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-03-21T09:44:01.442 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md 2024-03-21T09:44:01.442 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/bcache 2024-03-21T09:44:01.442 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/bcache/bcache.ko 2024-03-21T09:44:01.442 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-bio-prison.ko 2024-03-21T09:44:01.442 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-cache-smq.ko 2024-03-21T09:44:01.442 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-cache.ko 2024-03-21T09:44:01.443 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-crypt.ko 2024-03-21T09:44:01.443 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-delay.ko 2024-03-21T09:44:01.443 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-flakey.ko 2024-03-21T09:44:01.443 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-log-userspace.ko 2024-03-21T09:44:01.443 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-multipath.ko 2024-03-21T09:44:01.443 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-queue-length.ko 2024-03-21T09:44:01.443 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-raid.ko 2024-03-21T09:44:01.443 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-round-robin.ko 2024-03-21T09:44:01.443 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-service-time.ko 2024-03-21T09:44:01.443 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-switch.ko 2024-03-21T09:44:01.443 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-thin-pool.ko 2024-03-21T09:44:01.443 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/dm-verity.ko 2024-03-21T09:44:01.443 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/persistent-data 2024-03-21T09:44:01.444 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-03-21T09:44:01.444 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/raid0.ko 2024-03-21T09:44:01.444 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/raid1.ko 2024-03-21T09:44:01.444 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/raid10.ko 2024-03-21T09:44:01.444 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/md/raid456.ko 2024-03-21T09:44:01.444 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message 2024-03-21T09:44:01.444 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion 2024-03-21T09:44:01.444 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptbase.ko 2024-03-21T09:44:01.444 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptctl.ko 2024-03-21T09:44:01.444 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptfc.ko 2024-03-21T09:44:01.444 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptlan.ko 2024-03-21T09:44:01.444 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptsas.ko 2024-03-21T09:44:01.444 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptscsih.ko 2024-03-21T09:44:01.444 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/message/fusion/mptspi.ko 2024-03-21T09:44:01.444 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd 2024-03-21T09:44:01.445 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/lpc_ich.ko 2024-03-21T09:44:01.445 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/lpc_sch.ko 2024-03-21T09:44:01.445 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/mfd-core.ko 2024-03-21T09:44:01.445 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/sm501.ko 2024-03-21T09:44:01.445 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/viperboard.ko 2024-03-21T09:44:01.445 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/vx855.ko 2024-03-21T09:44:01.445 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/mfd/wl1273-core.ko 2024-03-21T09:44:01.445 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc 2024-03-21T09:44:01.445 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/altera-stapl 2024-03-21T09:44:01.445 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-03-21T09:44:01.445 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/apds9802als.ko 2024-03-21T09:44:01.445 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/apds990x.ko 2024-03-21T09:44:01.445 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/bh1770glc.ko 2024-03-21T09:44:01.445 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/cb710 2024-03-21T09:44:01.446 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/cb710/cb710.ko 2024-03-21T09:44:01.446 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/eeprom 2024-03-21T09:44:01.446 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/eeprom/at24.ko 2024-03-21T09:44:01.446 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-03-21T09:44:01.446 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/eeprom/max6875.ko 2024-03-21T09:44:01.446 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/enclosure.ko 2024-03-21T09:44:01.446 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/hpilo.ko 2024-03-21T09:44:01.446 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/isl29003.ko 2024-03-21T09:44:01.446 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/isl29020.ko 2024-03-21T09:44:01.446 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/lis3lv02d 2024-03-21T09:44:01.446 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-03-21T09:44:01.446 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-03-21T09:44:01.447 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/mei 2024-03-21T09:44:01.447 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/mei/mei-me.ko 2024-03-21T09:44:01.447 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/mei/mei-txe.ko 2024-03-21T09:44:01.447 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/mei/mei.ko 2024-03-21T09:44:01.447 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-gru 2024-03-21T09:44:01.447 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-gru/gru.ko 2024-03-21T09:44:01.447 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-xp 2024-03-21T09:44:01.447 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-xp/xp.ko 2024-03-21T09:44:01.447 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-xp/xpc.ko 2024-03-21T09:44:01.447 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-03-21T09:44:01.447 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/tifm_7xx1.ko 2024-03-21T09:44:01.447 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/tifm_core.ko 2024-03-21T09:44:01.447 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/tsl2550.ko 2024-03-21T09:44:01.447 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/vmw_balloon.ko 2024-03-21T09:44:01.448 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/vmw_vmci 2024-03-21T09:44:01.448 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-03-21T09:44:01.448 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net 2024-03-21T09:44:01.448 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/bonding 2024-03-21T09:44:01.448 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/bonding/bonding.ko 2024-03-21T09:44:01.448 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa 2024-03-21T09:44:01.448 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/b53 2024-03-21T09:44:01.448 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/b53/b53_common.ko 2024-03-21T09:44:01.448 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/bcm-sf2.ko 2024-03-21T09:44:01.448 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/mv88e6060.ko 2024-03-21T09:44:01.448 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/mv88e6xxx 2024-03-21T09:44:01.448 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-03-21T09:44:01.448 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/dummy.ko 2024-03-21T09:44:01.449 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/eql.ko 2024-03-21T09:44:01.449 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet 2024-03-21T09:44:01.449 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/3com 2024-03-21T09:44:01.449 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-03-21T09:44:01.449 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-03-21T09:44:01.449 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-03-21T09:44:01.449 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-03-21T09:44:01.449 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/8390 2024-03-21T09:44:01.449 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/8390/8390.ko 2024-03-21T09:44:01.449 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-03-21T09:44:01.449 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-03-21T09:44:01.449 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-03-21T09:44:01.450 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/adaptec 2024-03-21T09:44:01.450 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-03-21T09:44:01.450 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/agere 2024-03-21T09:44:01.450 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/agere/et131x.ko 2024-03-21T09:44:01.450 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/alteon 2024-03-21T09:44:01.450 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-03-21T09:44:01.450 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/altera 2024-03-21T09:44:01.450 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-03-21T09:44:01.450 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/amd 2024-03-21T09:44:01.450 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-03-21T09:44:01.450 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-03-21T09:44:01.450 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-03-21T09:44:01.451 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros 2024-03-21T09:44:01.451 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/alx 2024-03-21T09:44:01.451 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-03-21T09:44:01.451 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atl1c 2024-03-21T09:44:01.451 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-03-21T09:44:01.451 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atl1e 2024-03-21T09:44:01.451 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-03-21T09:44:01.451 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atlx 2024-03-21T09:44:01.451 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-03-21T09:44:01.451 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-03-21T09:44:01.451 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom 2024-03-21T09:44:01.451 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-03-21T09:44:01.451 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-03-21T09:44:01.452 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-03-21T09:44:01.452 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-03-21T09:44:01.452 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-03-21T09:44:01.452 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/genet 2024-03-21T09:44:01.452 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-03-21T09:44:01.452 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-03-21T09:44:01.452 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/brocade 2024-03-21T09:44:01.452 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/brocade/bna 2024-03-21T09:44:01.452 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-03-21T09:44:01.452 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio 2024-03-21T09:44:01.452 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb 2024-03-21T09:44:01.452 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-03-21T09:44:01.452 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-03-21T09:44:01.452 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-03-21T09:44:01.452 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-03-21T09:44:01.453 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-03-21T09:44:01.453 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-03-21T09:44:01.453 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-03-21T09:44:01.453 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-03-21T09:44:01.453 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-03-21T09:44:01.453 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/cisco 2024-03-21T09:44:01.453 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/cisco/enic 2024-03-21T09:44:01.453 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-03-21T09:44:01.453 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec 2024-03-21T09:44:01.453 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip 2024-03-21T09:44:01.453 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-03-21T09:44:01.453 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-03-21T09:44:01.453 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-03-21T09:44:01.453 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-03-21T09:44:01.453 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-03-21T09:44:01.454 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-03-21T09:44:01.454 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dlink 2024-03-21T09:44:01.454 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-03-21T09:44:01.454 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-03-21T09:44:01.454 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/dnet.ko 2024-03-21T09:44:01.454 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/emulex 2024-03-21T09:44:01.454 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/emulex/benet 2024-03-21T09:44:01.454 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-03-21T09:44:01.454 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/ethoc.ko 2024-03-21T09:44:01.454 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/fealnx.ko 2024-03-21T09:44:01.454 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel 2024-03-21T09:44:01.454 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/e100.ko 2024-03-21T09:44:01.454 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/e1000 2024-03-21T09:44:01.455 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-03-21T09:44:01.455 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/e1000e 2024-03-21T09:44:01.455 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-03-21T09:44:01.455 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/fm10k 2024-03-21T09:44:01.455 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-03-21T09:44:01.455 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/i40e 2024-03-21T09:44:01.455 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-03-21T09:44:01.455 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/iavf 2024-03-21T09:44:01.455 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-03-21T09:44:01.455 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/igb 2024-03-21T09:44:01.455 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-03-21T09:44:01.455 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/igbvf 2024-03-21T09:44:01.456 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-03-21T09:44:01.456 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/ixgbe 2024-03-21T09:44:01.456 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-03-21T09:44:01.456 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/ixgbevf 2024-03-21T09:44:01.456 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-03-21T09:44:01.456 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/jme.ko 2024-03-21T09:44:01.456 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/marvell 2024-03-21T09:44:01.456 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-03-21T09:44:01.456 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/marvell/skge.ko 2024-03-21T09:44:01.456 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-03-21T09:44:01.456 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox 2024-03-21T09:44:01.456 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx4 2024-03-21T09:44:01.456 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-03-21T09:44:01.456 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-03-21T09:44:01.457 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx5 2024-03-21T09:44:01.457 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-03-21T09:44:01.457 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-03-21T09:44:01.457 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/micrel 2024-03-21T09:44:01.457 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-03-21T09:44:01.457 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/myricom 2024-03-21T09:44:01.457 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/myricom/myri10ge 2024-03-21T09:44:01.457 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-03-21T09:44:01.457 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/natsemi 2024-03-21T09:44:01.457 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-03-21T09:44:01.457 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-03-21T09:44:01.457 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/neterion 2024-03-21T09:44:01.457 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-03-21T09:44:01.458 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/nvidia 2024-03-21T09:44:01.458 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-03-21T09:44:01.458 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/packetengines 2024-03-21T09:44:01.458 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-03-21T09:44:01.458 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-03-21T09:44:01.458 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic 2024-03-21T09:44:01.458 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic/netxen 2024-03-21T09:44:01.458 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-03-21T09:44:01.458 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-03-21T09:44:01.458 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-03-21T09:44:01.458 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-03-21T09:44:01.458 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/rdc 2024-03-21T09:44:01.458 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-03-21T09:44:01.458 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/realtek 2024-03-21T09:44:01.459 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-03-21T09:44:01.459 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-03-21T09:44:01.459 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/realtek/atp.ko 2024-03-21T09:44:01.459 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-03-21T09:44:01.459 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/rocker 2024-03-21T09:44:01.459 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-03-21T09:44:01.459 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sfc 2024-03-21T09:44:01.459 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-03-21T09:44:01.459 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/silan 2024-03-21T09:44:01.459 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-03-21T09:44:01.459 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sis 2024-03-21T09:44:01.459 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sis/sis190.ko 2024-03-21T09:44:01.459 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sis/sis900.ko 2024-03-21T09:44:01.459 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/smsc 2024-03-21T09:44:01.459 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-03-21T09:44:01.460 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-03-21T09:44:01.460 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-03-21T09:44:01.460 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-03-21T09:44:01.460 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/stmicro 2024-03-21T09:44:01.460 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/stmicro/stmmac 2024-03-21T09:44:01.460 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-03-21T09:44:01.460 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-03-21T09:44:01.460 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sun 2024-03-21T09:44:01.460 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sun/cassini.ko 2024-03-21T09:44:01.460 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sun/niu.ko 2024-03-21T09:44:01.460 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sun/sungem.ko 2024-03-21T09:44:01.460 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-03-21T09:44:01.461 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/tehuti 2024-03-21T09:44:01.461 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-03-21T09:44:01.461 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/ti 2024-03-21T09:44:01.461 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/ti/tlan.ko 2024-03-21T09:44:01.461 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/via 2024-03-21T09:44:01.461 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-03-21T09:44:01.461 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-03-21T09:44:01.461 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/wiznet 2024-03-21T09:44:01.461 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-03-21T09:44:01.461 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-03-21T09:44:01.461 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/xircom 2024-03-21T09:44:01.461 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-03-21T09:44:01.462 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/geneve.ko 2024-03-21T09:44:01.462 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/hyperv 2024-03-21T09:44:01.462 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-03-21T09:44:01.462 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ieee802154 2024-03-21T09:44:01.462 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ieee802154/fakelb.ko 2024-03-21T09:44:01.462 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ifb.ko 2024-03-21T09:44:01.462 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ipvlan 2024-03-21T09:44:01.462 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ipvlan/ipvlan.ko 2024-03-21T09:44:01.462 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/macvlan.ko 2024-03-21T09:44:01.463 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/macvtap.ko 2024-03-21T09:44:01.463 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mdio 2024-03-21T09:44:01.463 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mdio.ko 2024-03-21T09:44:01.463 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-03-21T09:44:01.463 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-03-21T09:44:01.463 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mdio/mdio-regmap.ko 2024-03-21T09:44:01.463 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/mii.ko 2024-03-21T09:44:01.463 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/net_failover.ko 2024-03-21T09:44:01.463 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/netconsole.ko 2024-03-21T09:44:01.463 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/nlmon.ko 2024-03-21T09:44:01.463 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/pcs 2024-03-21T09:44:01.464 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/pcs/pcs-lynx.ko 2024-03-21T09:44:01.464 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-03-21T09:44:01.464 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy 2024-03-21T09:44:01.464 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/amd.ko 2024-03-21T09:44:01.464 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-03-21T09:44:01.464 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-03-21T09:44:01.464 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/bcm7xxx.ko 2024-03-21T09:44:01.464 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/bcm87xx.ko 2024-03-21T09:44:01.464 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/broadcom.ko 2024-03-21T09:44:01.464 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/cicada.ko 2024-03-21T09:44:01.464 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/davicom.ko 2024-03-21T09:44:01.464 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/dp83640.ko 2024-03-21T09:44:01.464 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/et1011c.ko 2024-03-21T09:44:01.464 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/icplus.ko 2024-03-21T09:44:01.465 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/lxt.ko 2024-03-21T09:44:01.465 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/marvell.ko 2024-03-21T09:44:01.465 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/micrel.ko 2024-03-21T09:44:01.465 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/national.ko 2024-03-21T09:44:01.465 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/phylink.ko 2024-03-21T09:44:01.465 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/qsemi.ko 2024-03-21T09:44:01.465 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/realtek.ko 2024-03-21T09:44:01.465 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/smsc.ko 2024-03-21T09:44:01.465 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/ste10Xp.ko 2024-03-21T09:44:01.465 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/phy/vitesse.ko 2024-03-21T09:44:01.465 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp 2024-03-21T09:44:01.465 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/bsd_comp.ko 2024-03-21T09:44:01.466 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/ppp_async.ko 2024-03-21T09:44:01.466 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/ppp_deflate.ko 2024-03-21T09:44:01.466 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/ppp_generic.ko 2024-03-21T09:44:01.466 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/ppp_mppe.ko 2024-03-21T09:44:01.466 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/ppp_synctty.ko 2024-03-21T09:44:01.466 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/pppoe.ko 2024-03-21T09:44:01.466 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/pppox.ko 2024-03-21T09:44:01.466 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/ppp/pptp.ko 2024-03-21T09:44:01.466 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/slip 2024-03-21T09:44:01.466 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/slip/slhc.ko 2024-03-21T09:44:01.466 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/slip/slip.ko 2024-03-21T09:44:01.467 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/sungem_phy.ko 2024-03-21T09:44:01.467 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/tap.ko 2024-03-21T09:44:01.467 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team 2024-03-21T09:44:01.467 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team.ko 2024-03-21T09:44:01.467 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team_mode_activebackup.ko 2024-03-21T09:44:01.467 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team_mode_broadcast.ko 2024-03-21T09:44:01.467 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-03-21T09:44:01.467 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team_mode_random.ko 2024-03-21T09:44:01.467 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-03-21T09:44:01.468 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/tun.ko 2024-03-21T09:44:01.468 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/veth.ko 2024-03-21T09:44:01.468 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/virtio_net.ko 2024-03-21T09:44:01.468 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/vmxnet3 2024-03-21T09:44:01.468 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-03-21T09:44:01.468 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/vxlan 2024-03-21T09:44:01.468 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/vxlan/vxlan.ko 2024-03-21T09:44:01.468 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/xen-netback 2024-03-21T09:44:01.468 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/xen-netback/xen-netback.ko 2024-03-21T09:44:01.468 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/net/xen-netfront.ko 2024-03-21T09:44:01.468 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ntb 2024-03-21T09:44:01.468 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ntb/ntb.ko 2024-03-21T09:44:01.469 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/nvme 2024-03-21T09:44:01.469 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/nvme/host 2024-03-21T09:44:01.469 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/nvme/host/nvme-core.ko 2024-03-21T09:44:01.469 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/nvme/host/nvme.ko 2024-03-21T09:44:01.469 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/parport 2024-03-21T09:44:01.469 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/parport/parport.ko 2024-03-21T09:44:01.469 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/parport/parport_cs.ko 2024-03-21T09:44:01.469 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/parport/parport_pc.ko 2024-03-21T09:44:01.469 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/parport/parport_serial.ko 2024-03-21T09:44:01.469 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci 2024-03-21T09:44:01.469 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci/hotplug 2024-03-21T09:44:01.469 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-03-21T09:44:01.469 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci/pcie 2024-03-21T09:44:01.470 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci/pcie/aer_inject.ko 2024-03-21T09:44:01.470 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pci/xen-pcifront.ko 2024-03-21T09:44:01.470 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pcmcia 2024-03-21T09:44:01.470 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pcmcia/i82092.ko 2024-03-21T09:44:01.470 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pcmcia/pd6729.ko 2024-03-21T09:44:01.470 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pcmcia/yenta_socket.ko 2024-03-21T09:44:01.471 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pinctrl 2024-03-21T09:44:01.471 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pinctrl/intel 2024-03-21T09:44:01.471 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-03-21T09:44:01.471 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform 2024-03-21T09:44:01.471 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86 2024-03-21T09:44:01.471 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/acer-wmi.ko 2024-03-21T09:44:01.471 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/acerhdf.ko 2024-03-21T09:44:01.471 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/apple-gmux.ko 2024-03-21T09:44:01.471 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/asus-laptop.ko 2024-03-21T09:44:01.471 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-03-21T09:44:01.471 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/asus-wmi.ko 2024-03-21T09:44:01.472 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/classmate-laptop.ko 2024-03-21T09:44:01.472 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-03-21T09:44:01.472 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-03-21T09:44:01.472 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-03-21T09:44:01.472 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-03-21T09:44:01.472 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/hdaps.ko 2024-03-21T09:44:01.472 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/intel 2024-03-21T09:44:01.472 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-03-21T09:44:01.472 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/intel_ips.ko 2024-03-21T09:44:01.472 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/msi-wmi.ko 2024-03-21T09:44:01.473 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/mxm-wmi.ko 2024-03-21T09:44:01.473 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-03-21T09:44:01.473 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/samsung-laptop.ko 2024-03-21T09:44:01.473 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/samsung-q10.ko 2024-03-21T09:44:01.473 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/topstar-laptop.ko 2024-03-21T09:44:01.473 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-03-21T09:44:01.473 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/toshiba_haps.ko 2024-03-21T09:44:01.473 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/wmi-bmof.ko 2024-03-21T09:44:01.473 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/platform/x86/wmi.ko 2024-03-21T09:44:01.473 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/powercap 2024-03-21T09:44:01.473 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/powercap/intel_rapl_common.ko 2024-03-21T09:44:01.473 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/powercap/intel_rapl_msr.ko 2024-03-21T09:44:01.474 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps 2024-03-21T09:44:01.474 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps/clients 2024-03-21T09:44:01.474 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps/clients/pps-gpio.ko 2024-03-21T09:44:01.474 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps/clients/pps-ldisc.ko 2024-03-21T09:44:01.474 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps/clients/pps_parport.ko 2024-03-21T09:44:01.474 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/pps/pps_core.ko 2024-03-21T09:44:01.474 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ptp 2024-03-21T09:44:01.474 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ptp/ptp.ko 2024-03-21T09:44:01.474 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ptp/ptp_kvm.ko 2024-03-21T09:44:01.474 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc 2024-03-21T09:44:01.474 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-bq32k.ko 2024-03-21T09:44:01.474 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1286.ko 2024-03-21T09:44:01.475 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1307.ko 2024-03-21T09:44:01.475 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1374.ko 2024-03-21T09:44:01.475 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1511.ko 2024-03-21T09:44:01.475 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1553.ko 2024-03-21T09:44:01.475 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1672.ko 2024-03-21T09:44:01.475 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds1742.ko 2024-03-21T09:44:01.475 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds2404.ko 2024-03-21T09:44:01.475 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-ds3232.ko 2024-03-21T09:44:01.475 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-em3027.ko 2024-03-21T09:44:01.475 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-fm3130.ko 2024-03-21T09:44:01.475 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-isl12022.ko 2024-03-21T09:44:01.475 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-isl1208.ko 2024-03-21T09:44:01.476 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-m41t80.ko 2024-03-21T09:44:01.476 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-m48t35.ko 2024-03-21T09:44:01.476 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-m48t59.ko 2024-03-21T09:44:01.476 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-max6900.ko 2024-03-21T09:44:01.476 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-msm6242.ko 2024-03-21T09:44:01.476 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-pcf2127.ko 2024-03-21T09:44:01.476 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-pcf85063.ko 2024-03-21T09:44:01.476 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-pcf8523.ko 2024-03-21T09:44:01.476 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-pcf8563.ko 2024-03-21T09:44:01.476 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-pcf8583.ko 2024-03-21T09:44:01.476 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-rp5c01.ko 2024-03-21T09:44:01.476 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-rs5c372.ko 2024-03-21T09:44:01.476 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-03-21T09:44:01.477 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-rx8025.ko 2024-03-21T09:44:01.477 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-rx8581.ko 2024-03-21T09:44:01.477 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-03-21T09:44:01.477 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/rtc/rtc-x1205.ko 2024-03-21T09:44:01.477 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi 2024-03-21T09:44:01.477 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/3w-9xxx.ko 2024-03-21T09:44:01.477 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/3w-sas.ko 2024-03-21T09:44:01.477 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/3w-xxxx.ko 2024-03-21T09:44:01.477 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/BusLogic.ko 2024-03-21T09:44:01.477 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/a100u2w.ko 2024-03-21T09:44:01.477 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/aacraid 2024-03-21T09:44:01.477 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/aacraid/aacraid.ko 2024-03-21T09:44:01.477 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/advansys.ko 2024-03-21T09:44:01.477 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/aic7xxx 2024-03-21T09:44:01.478 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-03-21T09:44:01.478 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-03-21T09:44:01.478 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/am53c974.ko 2024-03-21T09:44:01.478 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/arcmsr 2024-03-21T09:44:01.478 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-03-21T09:44:01.478 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/atp870u.ko 2024-03-21T09:44:01.478 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/be2iscsi 2024-03-21T09:44:01.478 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-03-21T09:44:01.478 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bfa 2024-03-21T09:44:01.478 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bfa/bfa.ko 2024-03-21T09:44:01.478 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bnx2fc 2024-03-21T09:44:01.478 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-03-21T09:44:01.478 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bnx2i 2024-03-21T09:44:01.478 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-03-21T09:44:01.479 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/ch.ko 2024-03-21T09:44:01.479 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/csiostor 2024-03-21T09:44:01.479 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/csiostor/csiostor.ko 2024-03-21T09:44:01.479 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi 2024-03-21T09:44:01.479 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi/cxgb3i 2024-03-21T09:44:01.479 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-03-21T09:44:01.479 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi/cxgb4i 2024-03-21T09:44:01.479 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-03-21T09:44:01.479 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-03-21T09:44:01.479 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/dc395x.ko 2024-03-21T09:44:01.479 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/device_handler 2024-03-21T09:44:01.479 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-03-21T09:44:01.479 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-03-21T09:44:01.479 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-03-21T09:44:01.481 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-03-21T09:44:01.481 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/dmx3191d.ko 2024-03-21T09:44:01.481 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/esas2r 2024-03-21T09:44:01.481 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/esas2r/esas2r.ko 2024-03-21T09:44:01.481 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/esp_scsi.ko 2024-03-21T09:44:01.481 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/fcoe 2024-03-21T09:44:01.481 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/fcoe/fcoe.ko 2024-03-21T09:44:01.481 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-03-21T09:44:01.481 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/fnic 2024-03-21T09:44:01.481 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/fnic/fnic.ko 2024-03-21T09:44:01.481 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/hpsa.ko 2024-03-21T09:44:01.481 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/hptiop.ko 2024-03-21T09:44:01.481 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/hv_storvsc.ko 2024-03-21T09:44:01.482 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/initio.ko 2024-03-21T09:44:01.482 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/ipr.ko 2024-03-21T09:44:01.482 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/ips.ko 2024-03-21T09:44:01.482 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/isci 2024-03-21T09:44:01.482 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/isci/isci.ko 2024-03-21T09:44:01.482 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-03-21T09:44:01.482 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/iscsi_tcp.ko 2024-03-21T09:44:01.482 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libfc 2024-03-21T09:44:01.482 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libfc/libfc.ko 2024-03-21T09:44:01.482 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libiscsi.ko 2024-03-21T09:44:01.482 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libiscsi_tcp.ko 2024-03-21T09:44:01.482 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libsas 2024-03-21T09:44:01.482 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/libsas/libsas.ko 2024-03-21T09:44:01.483 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/lpfc 2024-03-21T09:44:01.483 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/lpfc/lpfc.ko 2024-03-21T09:44:01.483 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/megaraid 2024-03-21T09:44:01.483 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/megaraid.ko 2024-03-21T09:44:01.483 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-03-21T09:44:01.483 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-03-21T09:44:01.483 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-03-21T09:44:01.483 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/mpt3sas 2024-03-21T09:44:01.483 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-03-21T09:44:01.483 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/mvsas 2024-03-21T09:44:01.483 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/mvsas/mvsas.ko 2024-03-21T09:44:01.483 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/mvumi.ko 2024-03-21T09:44:01.483 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/pm8001 2024-03-21T09:44:01.484 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-03-21T09:44:01.484 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/pmcraid.ko 2024-03-21T09:44:01.484 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla1280.ko 2024-03-21T09:44:01.484 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla2xxx 2024-03-21T09:44:01.484 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-03-21T09:44:01.484 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-03-21T09:44:01.484 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla4xxx 2024-03-21T09:44:01.484 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-03-21T09:44:01.484 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/raid_class.ko 2024-03-21T09:44:01.484 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_debug.ko 2024-03-21T09:44:01.484 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_transport_fc.ko 2024-03-21T09:44:01.484 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-03-21T09:44:01.484 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_transport_sas.ko 2024-03-21T09:44:01.484 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_transport_spi.ko 2024-03-21T09:44:01.485 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/scsi_transport_srp.ko 2024-03-21T09:44:01.485 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/ses.ko 2024-03-21T09:44:01.485 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/st.ko 2024-03-21T09:44:01.485 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/stex.ko 2024-03-21T09:44:01.485 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/sym53c8xx_2 2024-03-21T09:44:01.485 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-03-21T09:44:01.485 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/virtio_scsi.ko 2024-03-21T09:44:01.485 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/vmw_pvscsi.ko 2024-03-21T09:44:01.485 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/scsi/wd719x.ko 2024-03-21T09:44:01.485 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ssb 2024-03-21T09:44:01.485 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ssb/ssb.ko 2024-03-21T09:44:01.485 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target 2024-03-21T09:44:01.485 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/iscsi 2024-03-21T09:44:01.485 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-03-21T09:44:01.486 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/loopback 2024-03-21T09:44:01.486 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/loopback/tcm_loop.ko 2024-03-21T09:44:01.486 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/target_core_file.ko 2024-03-21T09:44:01.486 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/target_core_iblock.ko 2024-03-21T09:44:01.486 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/target_core_mod.ko 2024-03-21T09:44:01.486 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/target_core_pscsi.ko 2024-03-21T09:44:01.486 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/target_core_user.ko 2024-03-21T09:44:01.486 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/tcm_fc 2024-03-21T09:44:01.486 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-03-21T09:44:01.486 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal 2024-03-21T09:44:01.486 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel 2024-03-21T09:44:01.486 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal 2024-03-21T09:44:01.486 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-03-21T09:44:01.487 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-03-21T09:44:01.487 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-03-21T09:44:01.487 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-03-21T09:44:01.487 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-03-21T09:44:01.487 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-03-21T09:44:01.487 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-03-21T09:44:01.487 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-03-21T09:44:01.487 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-03-21T09:44:01.487 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-03-21T09:44:01.487 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-03-21T09:44:01.487 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-03-21T09:44:01.487 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-03-21T09:44:01.487 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-03-21T09:44:01.488 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-03-21T09:44:01.488 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-03-21T09:44:01.488 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-03-21T09:44:01.488 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-03-21T09:44:01.488 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty 2024-03-21T09:44:01.488 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/ipwireless 2024-03-21T09:44:01.488 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-03-21T09:44:01.488 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/n_gsm.ko 2024-03-21T09:44:01.488 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/n_hdlc.ko 2024-03-21T09:44:01.488 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/nozomi.ko 2024-03-21T09:44:01.488 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial 2024-03-21T09:44:01.488 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/8250 2024-03-21T09:44:01.488 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-03-21T09:44:01.488 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/arc_uart.ko 2024-03-21T09:44:01.489 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/jsm 2024-03-21T09:44:01.489 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/tty/serial/jsm/jsm.ko 2024-03-21T09:44:01.489 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ufs 2024-03-21T09:44:01.489 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ufs/core 2024-03-21T09:44:01.489 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ufs/core/ufshcd-core.ko 2024-03-21T09:44:01.489 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ufs/host 2024-03-21T09:44:01.489 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-03-21T09:44:01.489 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio 2024-03-21T09:44:01.489 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/uio.ko 2024-03-21T09:44:01.489 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/uio_aec.ko 2024-03-21T09:44:01.489 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/uio_cif.ko 2024-03-21T09:44:01.489 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/uio_pci_generic.ko 2024-03-21T09:44:01.489 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/uio/uio_sercos3.ko 2024-03-21T09:44:01.489 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb 2024-03-21T09:44:01.490 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/class 2024-03-21T09:44:01.490 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/class/cdc-acm.ko 2024-03-21T09:44:01.490 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/class/cdc-wdm.ko 2024-03-21T09:44:01.490 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/class/usblp.ko 2024-03-21T09:44:01.490 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/class/usbtmc.ko 2024-03-21T09:44:01.490 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/host 2024-03-21T09:44:01.490 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/host/sl811-hcd.ko 2024-03-21T09:44:01.490 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/image 2024-03-21T09:44:01.490 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/image/mdc800.ko 2024-03-21T09:44:01.490 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/image/microtek.ko 2024-03-21T09:44:01.490 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc 2024-03-21T09:44:01.490 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/adutux.ko 2024-03-21T09:44:01.490 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/appledisplay.ko 2024-03-21T09:44:01.491 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/emi26.ko 2024-03-21T09:44:01.491 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/emi62.ko 2024-03-21T09:44:01.491 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/ezusb.ko 2024-03-21T09:44:01.491 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/idmouse.ko 2024-03-21T09:44:01.491 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/iowarrior.ko 2024-03-21T09:44:01.491 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/isight_firmware.ko 2024-03-21T09:44:01.491 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/ldusb.ko 2024-03-21T09:44:01.491 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/legousbtower.ko 2024-03-21T09:44:01.491 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/sisusbvga 2024-03-21T09:44:01.491 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-03-21T09:44:01.491 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/trancevibrator.ko 2024-03-21T09:44:01.491 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/usb3503.ko 2024-03-21T09:44:01.491 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/usblcd.ko 2024-03-21T09:44:01.491 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/usbsevseg.ko 2024-03-21T09:44:01.491 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/uss720.ko 2024-03-21T09:44:01.492 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/misc/yurex.ko 2024-03-21T09:44:01.492 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/phy 2024-03-21T09:44:01.492 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/phy/phy-generic.ko 2024-03-21T09:44:01.492 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial 2024-03-21T09:44:01.492 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/aircable.ko 2024-03-21T09:44:01.492 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ark3116.ko 2024-03-21T09:44:01.492 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/belkin_sa.ko 2024-03-21T09:44:01.492 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ch341.ko 2024-03-21T09:44:01.492 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/cp210x.ko 2024-03-21T09:44:01.492 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/cyberjack.ko 2024-03-21T09:44:01.492 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/cypress_m8.ko 2024-03-21T09:44:01.492 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/digi_acceleport.ko 2024-03-21T09:44:01.492 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/empeg.ko 2024-03-21T09:44:01.492 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ftdi_sio.ko 2024-03-21T09:44:01.492 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/garmin_gps.ko 2024-03-21T09:44:01.493 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/io_edgeport.ko 2024-03-21T09:44:01.493 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/io_ti.ko 2024-03-21T09:44:01.493 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ipaq.ko 2024-03-21T09:44:01.493 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ipw.ko 2024-03-21T09:44:01.493 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ir-usb.ko 2024-03-21T09:44:01.493 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-03-21T09:44:01.493 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/keyspan.ko 2024-03-21T09:44:01.493 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/keyspan_pda.ko 2024-03-21T09:44:01.493 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/kl5kusb105.ko 2024-03-21T09:44:01.493 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/kobil_sct.ko 2024-03-21T09:44:01.493 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/mct_u232.ko 2024-03-21T09:44:01.493 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/mos7720.ko 2024-03-21T09:44:01.493 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/mos7840.ko 2024-03-21T09:44:01.493 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/navman.ko 2024-03-21T09:44:01.494 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/omninet.ko 2024-03-21T09:44:01.494 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/opticon.ko 2024-03-21T09:44:01.494 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/option.ko 2024-03-21T09:44:01.494 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/oti6858.ko 2024-03-21T09:44:01.494 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/pl2303.ko 2024-03-21T09:44:01.494 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/qcaux.ko 2024-03-21T09:44:01.494 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/qcserial.ko 2024-03-21T09:44:01.494 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/quatech2.ko 2024-03-21T09:44:01.494 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/safe_serial.ko 2024-03-21T09:44:01.494 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/sierra.ko 2024-03-21T09:44:01.494 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/spcp8x5.ko 2024-03-21T09:44:01.494 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ssu100.ko 2024-03-21T09:44:01.494 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/symbolserial.ko 2024-03-21T09:44:01.495 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-03-21T09:44:01.495 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-03-21T09:44:01.495 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/usb_debug.ko 2024-03-21T09:44:01.495 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/usb_wwan.ko 2024-03-21T09:44:01.495 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/visor.ko 2024-03-21T09:44:01.495 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/whiteheat.ko 2024-03-21T09:44:01.495 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/serial/xsens_mt.ko 2024-03-21T09:44:01.495 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage 2024-03-21T09:44:01.495 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/uas.ko 2024-03-21T09:44:01.495 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-alauda.ko 2024-03-21T09:44:01.495 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-cypress.ko 2024-03-21T09:44:01.495 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-datafab.ko 2024-03-21T09:44:01.495 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-03-21T09:44:01.495 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-freecom.ko 2024-03-21T09:44:01.495 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-isd200.ko 2024-03-21T09:44:01.496 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-03-21T09:44:01.496 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-karma.ko 2024-03-21T09:44:01.496 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-onetouch.ko 2024-03-21T09:44:01.496 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-realtek.ko 2024-03-21T09:44:01.496 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-sddr09.ko 2024-03-21T09:44:01.496 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-sddr55.ko 2024-03-21T09:44:01.496 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/ums-usbat.ko 2024-03-21T09:44:01.496 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/storage/usb-storage.ko 2024-03-21T09:44:01.496 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/usbip 2024-03-21T09:44:01.496 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/usbip/usbip-core.ko 2024-03-21T09:44:01.496 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/usbip/usbip-host.ko 2024-03-21T09:44:01.496 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-03-21T09:44:01.496 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio 2024-03-21T09:44:01.497 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio/pci 2024-03-21T09:44:01.497 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-03-21T09:44:01.497 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio/pci/vfio-pci.ko 2024-03-21T09:44:01.497 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio/vfio.ko 2024-03-21T09:44:01.497 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-03-21T09:44:01.497 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vhost 2024-03-21T09:44:01.497 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vhost/vhost.ko 2024-03-21T09:44:01.497 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vhost/vhost_iotlb.ko 2024-03-21T09:44:01.497 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vhost/vhost_net.ko 2024-03-21T09:44:01.497 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/vhost/vhost_scsi.ko 2024-03-21T09:44:01.497 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video 2024-03-21T09:44:01.497 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight 2024-03-21T09:44:01.497 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight/apple_bl.ko 2024-03-21T09:44:01.497 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight/lcd.ko 2024-03-21T09:44:01.498 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/video/backlight/platform_lcd.ko 2024-03-21T09:44:01.498 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio 2024-03-21T09:44:01.498 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio.ko 2024-03-21T09:44:01.498 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_balloon.ko 2024-03-21T09:44:01.498 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_mmio.ko 2024-03-21T09:44:01.498 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_pci.ko 2024-03-21T09:44:01.498 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-03-21T09:44:01.498 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-03-21T09:44:01.498 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/virtio/virtio_ring.ko 2024-03-21T09:44:01.498 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1 2024-03-21T09:44:01.498 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/masters 2024-03-21T09:44:01.498 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/masters/ds2482.ko 2024-03-21T09:44:01.498 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/masters/ds2490.ko 2024-03-21T09:44:01.499 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves 2024-03-21T09:44:01.499 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-03-21T09:44:01.499 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-03-21T09:44:01.499 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-03-21T09:44:01.499 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-03-21T09:44:01.499 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-03-21T09:44:01.499 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-03-21T09:44:01.499 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-03-21T09:44:01.499 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-03-21T09:44:01.499 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-03-21T09:44:01.499 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_smem.ko 2024-03-21T09:44:01.499 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/slaves/w1_therm.ko 2024-03-21T09:44:01.499 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/w1/wire.ko 2024-03-21T09:44:01.499 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog 2024-03-21T09:44:01.499 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/alim1535_wdt.ko 2024-03-21T09:44:01.500 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/alim7101_wdt.ko 2024-03-21T09:44:01.500 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/f71808e_wdt.ko 2024-03-21T09:44:01.500 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/hpwdt.ko 2024-03-21T09:44:01.500 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/i6300esb.ko 2024-03-21T09:44:01.500 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-03-21T09:44:01.500 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/iTCO_wdt.ko 2024-03-21T09:44:01.500 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/ib700wdt.ko 2024-03-21T09:44:01.500 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/ibmasr.ko 2024-03-21T09:44:01.500 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-03-21T09:44:01.500 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/it8712f_wdt.ko 2024-03-21T09:44:01.500 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/it87_wdt.ko 2024-03-21T09:44:01.500 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/machzwd.ko 2024-03-21T09:44:01.500 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/nv_tco.ko 2024-03-21T09:44:01.500 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/pcwd_pci.ko 2024-03-21T09:44:01.501 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/pcwd_usb.ko 2024-03-21T09:44:01.501 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-03-21T09:44:01.501 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/sch311x_wdt.ko 2024-03-21T09:44:01.501 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/softdog.ko 2024-03-21T09:44:01.501 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/sp5100_tco.ko 2024-03-21T09:44:01.501 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/via_wdt.ko 2024-03-21T09:44:01.501 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-03-21T09:44:01.501 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/w83877f_wdt.ko 2024-03-21T09:44:01.501 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/w83977f_wdt.ko 2024-03-21T09:44:01.501 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/wdt_pci.ko 2024-03-21T09:44:01.501 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/watchdog/xen_wdt.ko 2024-03-21T09:44:01.501 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen 2024-03-21T09:44:01.501 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-acpi-processor.ko 2024-03-21T09:44:01.502 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-evtchn.ko 2024-03-21T09:44:01.502 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-gntalloc.ko 2024-03-21T09:44:01.502 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-gntdev.ko 2024-03-21T09:44:01.502 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-pciback 2024-03-21T09:44:01.502 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-03-21T09:44:01.502 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xen-privcmd.ko 2024-03-21T09:44:01.502 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xenfs 2024-03-21T09:44:01.502 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/drivers/xen/xenfs/xenfs.ko 2024-03-21T09:44:01.502 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs 2024-03-21T09:44:01.502 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/9p 2024-03-21T09:44:01.502 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/9p/9p.ko 2024-03-21T09:44:01.502 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/affs 2024-03-21T09:44:01.502 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/affs/affs.ko 2024-03-21T09:44:01.503 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/befs 2024-03-21T09:44:01.503 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/befs/befs.ko 2024-03-21T09:44:01.503 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/binfmt_misc.ko 2024-03-21T09:44:01.503 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/btrfs 2024-03-21T09:44:01.503 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/btrfs/btrfs.ko 2024-03-21T09:44:01.503 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/cachefiles 2024-03-21T09:44:01.503 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/cachefiles/cachefiles.ko 2024-03-21T09:44:01.503 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ceph 2024-03-21T09:44:01.503 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ceph/ceph.ko 2024-03-21T09:44:01.503 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/coda 2024-03-21T09:44:01.503 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/coda/coda.ko 2024-03-21T09:44:01.503 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/cramfs 2024-03-21T09:44:01.503 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/cramfs/cramfs.ko 2024-03-21T09:44:01.504 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/dlm 2024-03-21T09:44:01.504 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/dlm/dlm.ko 2024-03-21T09:44:01.504 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ecryptfs 2024-03-21T09:44:01.504 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ecryptfs/ecryptfs.ko 2024-03-21T09:44:01.504 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/f2fs 2024-03-21T09:44:01.504 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/f2fs/f2fs.ko 2024-03-21T09:44:01.504 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fat 2024-03-21T09:44:01.504 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fat/fat.ko 2024-03-21T09:44:01.504 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fat/msdos.ko 2024-03-21T09:44:01.504 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fat/vfat.ko 2024-03-21T09:44:01.504 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fuse 2024-03-21T09:44:01.504 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fuse/cuse.ko 2024-03-21T09:44:01.504 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/fuse/fuse.ko 2024-03-21T09:44:01.504 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/gfs2 2024-03-21T09:44:01.504 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/gfs2/gfs2.ko 2024-03-21T09:44:01.505 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/hfs 2024-03-21T09:44:01.505 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/hfs/hfs.ko 2024-03-21T09:44:01.505 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/hfsplus 2024-03-21T09:44:01.505 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/hfsplus/hfsplus.ko 2024-03-21T09:44:01.505 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/isofs 2024-03-21T09:44:01.505 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/isofs/isofs.ko 2024-03-21T09:44:01.505 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/jfs 2024-03-21T09:44:01.505 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/jfs/jfs.ko 2024-03-21T09:44:01.505 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/lockd 2024-03-21T09:44:01.505 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/lockd/lockd.ko 2024-03-21T09:44:01.505 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/minix 2024-03-21T09:44:01.505 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/minix/minix.ko 2024-03-21T09:44:01.505 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/netfs 2024-03-21T09:44:01.505 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/netfs/netfs.ko 2024-03-21T09:44:01.506 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs 2024-03-21T09:44:01.506 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/blocklayout 2024-03-21T09:44:01.506 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-03-21T09:44:01.506 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/filelayout 2024-03-21T09:44:01.506 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-03-21T09:44:01.506 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/flexfilelayout 2024-03-21T09:44:01.506 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-03-21T09:44:01.506 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/nfs.ko 2024-03-21T09:44:01.506 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/nfsv3.ko 2024-03-21T09:44:01.506 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs/nfsv4.ko 2024-03-21T09:44:01.506 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs_common 2024-03-21T09:44:01.506 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs_common/grace.ko 2024-03-21T09:44:01.506 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfs_common/nfs_acl.ko 2024-03-21T09:44:01.507 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfsd 2024-03-21T09:44:01.507 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nfsd/nfsd.ko 2024-03-21T09:44:01.507 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nilfs2 2024-03-21T09:44:01.507 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nilfs2/nilfs2.ko 2024-03-21T09:44:01.507 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls 2024-03-21T09:44:01.507 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-celtic.ko 2024-03-21T09:44:01.507 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-centeuro.ko 2024-03-21T09:44:01.507 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-croatian.ko 2024-03-21T09:44:01.507 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-cyrillic.ko 2024-03-21T09:44:01.507 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-gaelic.ko 2024-03-21T09:44:01.507 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-greek.ko 2024-03-21T09:44:01.507 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-iceland.ko 2024-03-21T09:44:01.508 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-inuit.ko 2024-03-21T09:44:01.508 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-roman.ko 2024-03-21T09:44:01.508 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-romanian.ko 2024-03-21T09:44:01.508 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/mac-turkish.ko 2024-03-21T09:44:01.508 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp1250.ko 2024-03-21T09:44:01.508 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp1251.ko 2024-03-21T09:44:01.508 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp1255.ko 2024-03-21T09:44:01.508 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp737.ko 2024-03-21T09:44:01.508 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp775.ko 2024-03-21T09:44:01.508 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp850.ko 2024-03-21T09:44:01.508 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp852.ko 2024-03-21T09:44:01.508 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp855.ko 2024-03-21T09:44:01.508 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp857.ko 2024-03-21T09:44:01.508 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp860.ko 2024-03-21T09:44:01.509 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp861.ko 2024-03-21T09:44:01.509 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp862.ko 2024-03-21T09:44:01.509 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp863.ko 2024-03-21T09:44:01.509 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp864.ko 2024-03-21T09:44:01.509 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp865.ko 2024-03-21T09:44:01.509 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp866.ko 2024-03-21T09:44:01.509 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp869.ko 2024-03-21T09:44:01.509 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp874.ko 2024-03-21T09:44:01.509 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp932.ko 2024-03-21T09:44:01.509 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp936.ko 2024-03-21T09:44:01.509 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp949.ko 2024-03-21T09:44:01.509 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_cp950.ko 2024-03-21T09:44:01.509 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_euc-jp.ko 2024-03-21T09:44:01.509 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-1.ko 2024-03-21T09:44:01.510 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-13.ko 2024-03-21T09:44:01.510 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-14.ko 2024-03-21T09:44:01.510 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-15.ko 2024-03-21T09:44:01.510 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-2.ko 2024-03-21T09:44:01.510 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-3.ko 2024-03-21T09:44:01.510 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-4.ko 2024-03-21T09:44:01.510 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-5.ko 2024-03-21T09:44:01.510 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-6.ko 2024-03-21T09:44:01.510 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-7.ko 2024-03-21T09:44:01.510 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_iso8859-9.ko 2024-03-21T09:44:01.510 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_koi8-r.ko 2024-03-21T09:44:01.510 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_koi8-ru.ko 2024-03-21T09:44:01.510 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_koi8-u.ko 2024-03-21T09:44:01.511 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_ucs2_utils.ko 2024-03-21T09:44:01.511 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/nls/nls_utf8.ko 2024-03-21T09:44:01.511 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2 2024-03-21T09:44:01.511 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/cluster 2024-03-21T09:44:01.511 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-03-21T09:44:01.511 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/dlm 2024-03-21T09:44:01.511 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-03-21T09:44:01.511 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/dlmfs 2024-03-21T09:44:01.511 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-03-21T09:44:01.511 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/ocfs2.ko 2024-03-21T09:44:01.511 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-03-21T09:44:01.511 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-03-21T09:44:01.511 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-03-21T09:44:01.511 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/overlayfs 2024-03-21T09:44:01.512 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/overlayfs/overlay.ko 2024-03-21T09:44:01.512 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/pstore 2024-03-21T09:44:01.512 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/pstore/ramoops.ko 2024-03-21T09:44:01.512 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/reiserfs 2024-03-21T09:44:01.512 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/reiserfs/reiserfs.ko 2024-03-21T09:44:01.512 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/romfs 2024-03-21T09:44:01.512 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/romfs/romfs.ko 2024-03-21T09:44:01.512 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb 2024-03-21T09:44:01.512 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb/client 2024-03-21T09:44:01.512 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb/client/cifs.ko 2024-03-21T09:44:01.512 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb/common 2024-03-21T09:44:01.512 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb/common/cifs_arc4.ko 2024-03-21T09:44:01.512 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/smb/common/cifs_md4.ko 2024-03-21T09:44:01.513 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/squashfs 2024-03-21T09:44:01.513 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/squashfs/squashfs.ko 2024-03-21T09:44:01.513 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/sysv 2024-03-21T09:44:01.513 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/sysv/sysv.ko 2024-03-21T09:44:01.513 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/udf 2024-03-21T09:44:01.513 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/udf/udf.ko 2024-03-21T09:44:01.513 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ufs 2024-03-21T09:44:01.513 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/ufs/ufs.ko 2024-03-21T09:44:01.513 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/xfs 2024-03-21T09:44:01.513 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/fs/xfs/xfs.ko 2024-03-21T09:44:01.513 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel 2024-03-21T09:44:01.513 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel/rcu 2024-03-21T09:44:01.513 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel/rcu/rcutorture.ko 2024-03-21T09:44:01.514 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel/torture.ko 2024-03-21T09:44:01.514 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel/trace 2024-03-21T09:44:01.514 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-03-21T09:44:01.514 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib 2024-03-21T09:44:01.514 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/asn1_encoder.ko 2024-03-21T09:44:01.514 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/crc-itu-t.ko 2024-03-21T09:44:01.514 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/crc8.ko 2024-03-21T09:44:01.514 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/crypto 2024-03-21T09:44:01.514 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/crypto/libarc4.ko 2024-03-21T09:44:01.514 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/crypto/libdes.ko 2024-03-21T09:44:01.514 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/libcrc32c.ko 2024-03-21T09:44:01.514 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/lru_cache.ko 2024-03-21T09:44:01.514 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/lz4 2024-03-21T09:44:01.514 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/lz4/lz4_compress.ko 2024-03-21T09:44:01.515 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/lz4/lz4hc_compress.ko 2024-03-21T09:44:01.515 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/math 2024-03-21T09:44:01.515 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/math/cordic.ko 2024-03-21T09:44:01.515 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/raid6 2024-03-21T09:44:01.515 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/raid6/raid6_pq.ko 2024-03-21T09:44:01.515 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/reed_solomon 2024-03-21T09:44:01.515 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/reed_solomon/reed_solomon.ko 2024-03-21T09:44:01.515 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/ts_bm.ko 2024-03-21T09:44:01.515 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/ts_fsm.ko 2024-03-21T09:44:01.515 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/ts_kmp.ko 2024-03-21T09:44:01.515 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/zstd 2024-03-21T09:44:01.515 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/lib/zstd/zstd_compress.ko 2024-03-21T09:44:01.515 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/mm 2024-03-21T09:44:01.516 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/mm/hwpoison-inject.ko 2024-03-21T09:44:01.516 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net 2024-03-21T09:44:01.516 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan 2024-03-21T09:44:01.516 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/6lowpan.ko 2024-03-21T09:44:01.516 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_dest.ko 2024-03-21T09:44:01.516 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_fragment.ko 2024-03-21T09:44:01.516 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_hop.ko 2024-03-21T09:44:01.516 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_ipv6.ko 2024-03-21T09:44:01.516 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_mobility.ko 2024-03-21T09:44:01.516 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_routing.ko 2024-03-21T09:44:01.516 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/6lowpan/nhc_udp.ko 2024-03-21T09:44:01.516 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802 2024-03-21T09:44:01.516 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802/garp.ko 2024-03-21T09:44:01.516 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802/mrp.ko 2024-03-21T09:44:01.517 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802/p8022.ko 2024-03-21T09:44:01.517 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802/psnap.ko 2024-03-21T09:44:01.517 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/802/stp.ko 2024-03-21T09:44:01.517 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/8021q 2024-03-21T09:44:01.517 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/8021q/8021q.ko 2024-03-21T09:44:01.517 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/9p 2024-03-21T09:44:01.517 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/9p/9pnet.ko 2024-03-21T09:44:01.517 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/9p/9pnet_fd.ko 2024-03-21T09:44:01.517 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/9p/9pnet_rdma.ko 2024-03-21T09:44:01.517 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/9p/9pnet_virtio.ko 2024-03-21T09:44:01.517 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/batman-adv 2024-03-21T09:44:01.517 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/batman-adv/batman-adv.ko 2024-03-21T09:44:01.517 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge 2024-03-21T09:44:01.518 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/br_netfilter.ko 2024-03-21T09:44:01.518 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/bridge.ko 2024-03-21T09:44:01.518 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter 2024-03-21T09:44:01.518 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-03-21T09:44:01.518 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_among.ko 2024-03-21T09:44:01.518 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_arp.ko 2024-03-21T09:44:01.518 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-03-21T09:44:01.518 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-03-21T09:44:01.518 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_ip.ko 2024-03-21T09:44:01.518 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-03-21T09:44:01.518 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_limit.ko 2024-03-21T09:44:01.518 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_log.ko 2024-03-21T09:44:01.518 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_mark.ko 2024-03-21T09:44:01.519 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-03-21T09:44:01.519 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-03-21T09:44:01.519 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-03-21T09:44:01.519 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-03-21T09:44:01.519 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_snat.ko 2024-03-21T09:44:01.519 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_stp.ko 2024-03-21T09:44:01.519 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-03-21T09:44:01.519 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-03-21T09:44:01.519 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-03-21T09:44:01.519 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-03-21T09:44:01.519 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/ebtables.ko 2024-03-21T09:44:01.519 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-03-21T09:44:01.519 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-03-21T09:44:01.519 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-03-21T09:44:01.519 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ceph 2024-03-21T09:44:01.520 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ceph/libceph.ko 2024-03-21T09:44:01.520 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/core 2024-03-21T09:44:01.520 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/core/failover.ko 2024-03-21T09:44:01.520 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/core/pktgen.ko 2024-03-21T09:44:01.520 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dccp 2024-03-21T09:44:01.520 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dccp/dccp.ko 2024-03-21T09:44:01.520 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dccp/dccp_diag.ko 2024-03-21T09:44:01.520 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dccp/dccp_ipv4.ko 2024-03-21T09:44:01.520 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dccp/dccp_ipv6.ko 2024-03-21T09:44:01.520 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dns_resolver 2024-03-21T09:44:01.520 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dns_resolver/dns_resolver.ko 2024-03-21T09:44:01.520 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa 2024-03-21T09:44:01.520 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa/dsa_core.ko 2024-03-21T09:44:01.521 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa/tag_brcm.ko 2024-03-21T09:44:01.521 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa/tag_dsa.ko 2024-03-21T09:44:01.521 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa/tag_none.ko 2024-03-21T09:44:01.521 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/dsa/tag_trailer.ko 2024-03-21T09:44:01.521 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ieee802154 2024-03-21T09:44:01.521 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ieee802154/6lowpan 2024-03-21T09:44:01.521 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-03-21T09:44:01.521 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ieee802154/ieee802154.ko 2024-03-21T09:44:01.521 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ieee802154/ieee802154_socket.ko 2024-03-21T09:44:01.521 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4 2024-03-21T09:44:01.521 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ah4.ko 2024-03-21T09:44:01.521 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/esp4.ko 2024-03-21T09:44:01.521 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/fou.ko 2024-03-21T09:44:01.521 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/gre.ko 2024-03-21T09:44:01.522 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/inet_diag.ko 2024-03-21T09:44:01.522 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ip_gre.ko 2024-03-21T09:44:01.522 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ip_tunnel.ko 2024-03-21T09:44:01.522 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ip_vti.ko 2024-03-21T09:44:01.522 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ipcomp.ko 2024-03-21T09:44:01.522 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/ipip.ko 2024-03-21T09:44:01.522 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter 2024-03-21T09:44:01.522 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/arp_tables.ko 2024-03-21T09:44:01.522 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-03-21T09:44:01.522 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-03-21T09:44:01.522 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-03-21T09:44:01.522 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-03-21T09:44:01.522 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-03-21T09:44:01.522 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-03-21T09:44:01.523 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-03-21T09:44:01.523 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-03-21T09:44:01.523 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-03-21T09:44:01.523 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/iptable_security.ko 2024-03-21T09:44:01.523 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-03-21T09:44:01.523 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-03-21T09:44:01.523 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-03-21T09:44:01.523 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-03-21T09:44:01.523 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-03-21T09:44:01.523 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-03-21T09:44:01.523 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-03-21T09:44:01.523 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-03-21T09:44:01.523 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-03-21T09:44:01.523 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-03-21T09:44:01.523 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_bic.ko 2024-03-21T09:44:01.524 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_dctcp.ko 2024-03-21T09:44:01.524 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_diag.ko 2024-03-21T09:44:01.524 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_highspeed.ko 2024-03-21T09:44:01.524 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_htcp.ko 2024-03-21T09:44:01.524 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_hybla.ko 2024-03-21T09:44:01.524 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_illinois.ko 2024-03-21T09:44:01.524 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_lp.ko 2024-03-21T09:44:01.524 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_scalable.ko 2024-03-21T09:44:01.524 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_vegas.ko 2024-03-21T09:44:01.524 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_veno.ko 2024-03-21T09:44:01.524 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_westwood.ko 2024-03-21T09:44:01.524 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tcp_yeah.ko 2024-03-21T09:44:01.524 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/tunnel4.ko 2024-03-21T09:44:01.524 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/udp_diag.ko 2024-03-21T09:44:01.525 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/udp_tunnel.ko 2024-03-21T09:44:01.525 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv4/xfrm4_tunnel.ko 2024-03-21T09:44:01.525 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6 2024-03-21T09:44:01.525 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/ah6.ko 2024-03-21T09:44:01.525 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/esp6.ko 2024-03-21T09:44:01.525 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/fou6.ko 2024-03-21T09:44:01.525 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/ip6_tunnel.ko 2024-03-21T09:44:01.525 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-03-21T09:44:01.525 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/ip6_vti.ko 2024-03-21T09:44:01.525 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/ipcomp6.ko 2024-03-21T09:44:01.525 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter 2024-03-21T09:44:01.525 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-03-21T09:44:01.525 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-03-21T09:44:01.525 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-03-21T09:44:01.525 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-03-21T09:44:01.526 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-03-21T09:44:01.526 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-03-21T09:44:01.526 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-03-21T09:44:01.526 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-03-21T09:44:01.526 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-03-21T09:44:01.526 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-03-21T09:44:01.526 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-03-21T09:44:01.526 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-03-21T09:44:01.526 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-03-21T09:44:01.526 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-03-21T09:44:01.526 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-03-21T09:44:01.526 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-03-21T09:44:01.526 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-03-21T09:44:01.526 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-03-21T09:44:01.526 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-03-21T09:44:01.527 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-03-21T09:44:01.527 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-03-21T09:44:01.527 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-03-21T09:44:01.527 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-03-21T09:44:01.527 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-03-21T09:44:01.527 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/sit.ko 2024-03-21T09:44:01.527 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/tunnel6.ko 2024-03-21T09:44:01.527 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/ipv6/xfrm6_tunnel.ko 2024-03-21T09:44:01.527 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/key 2024-03-21T09:44:01.527 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/key/af_key.ko 2024-03-21T09:44:01.527 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp 2024-03-21T09:44:01.527 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_core.ko 2024-03-21T09:44:01.527 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_debugfs.ko 2024-03-21T09:44:01.527 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_eth.ko 2024-03-21T09:44:01.528 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_ip.ko 2024-03-21T09:44:01.528 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_ip6.ko 2024-03-21T09:44:01.528 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_netlink.ko 2024-03-21T09:44:01.528 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/l2tp/l2tp_ppp.ko 2024-03-21T09:44:01.528 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/llc 2024-03-21T09:44:01.528 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/llc/llc.ko 2024-03-21T09:44:01.528 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/mac802154 2024-03-21T09:44:01.528 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/mac802154/mac802154.ko 2024-03-21T09:44:01.528 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/mpls 2024-03-21T09:44:01.528 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/mpls/mpls_gso.ko 2024-03-21T09:44:01.528 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter 2024-03-21T09:44:01.528 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset 2024-03-21T09:44:01.528 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set.ko 2024-03-21T09:44:01.528 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-03-21T09:44:01.529 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-03-21T09:44:01.529 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-03-21T09:44:01.529 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-03-21T09:44:01.529 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-03-21T09:44:01.529 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-03-21T09:44:01.529 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-03-21T09:44:01.529 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-03-21T09:44:01.529 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-03-21T09:44:01.529 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-03-21T09:44:01.529 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-03-21T09:44:01.529 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-03-21T09:44:01.529 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-03-21T09:44:01.529 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-03-21T09:44:01.529 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-03-21T09:44:01.530 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs 2024-03-21T09:44:01.530 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs.ko 2024-03-21T09:44:01.530 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-03-21T09:44:01.530 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-03-21T09:44:01.530 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-03-21T09:44:01.530 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-03-21T09:44:01.530 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-03-21T09:44:01.530 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-03-21T09:44:01.530 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-03-21T09:44:01.530 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-03-21T09:44:01.530 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-03-21T09:44:01.530 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-03-21T09:44:01.530 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-03-21T09:44:01.531 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-03-21T09:44:01.531 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-03-21T09:44:01.531 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conncount.ko 2024-03-21T09:44:01.531 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack.ko 2024-03-21T09:44:01.531 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-03-21T09:44:01.531 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-03-21T09:44:01.531 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-03-21T09:44:01.531 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_h323.ko 2024-03-21T09:44:01.531 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_irc.ko 2024-03-21T09:44:01.531 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-03-21T09:44:01.531 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-03-21T09:44:01.531 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-03-21T09:44:01.531 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_sane.ko 2024-03-21T09:44:01.532 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_sip.ko 2024-03-21T09:44:01.532 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-03-21T09:44:01.532 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-03-21T09:44:01.532 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_dup_netdev.ko 2024-03-21T09:44:01.532 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_flow_table.ko 2024-03-21T09:44:01.532 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_flow_table_inet.ko 2024-03-21T09:44:01.532 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_log_syslog.ko 2024-03-21T09:44:01.532 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat.ko 2024-03-21T09:44:01.532 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat_amanda.ko 2024-03-21T09:44:01.532 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat_ftp.ko 2024-03-21T09:44:01.532 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat_irc.ko 2024-03-21T09:44:01.532 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat_sip.ko 2024-03-21T09:44:01.532 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_nat_tftp.ko 2024-03-21T09:44:01.533 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_synproxy_core.ko 2024-03-21T09:44:01.533 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nf_tables.ko 2024-03-21T09:44:01.533 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nfnetlink.ko 2024-03-21T09:44:01.533 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nfnetlink_acct.ko 2024-03-21T09:44:01.533 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nfnetlink_log.ko 2024-03-21T09:44:01.533 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nfnetlink_osf.ko 2024-03-21T09:44:01.533 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nfnetlink_queue.ko 2024-03-21T09:44:01.533 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_chain_nat.ko 2024-03-21T09:44:01.533 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_compat.ko 2024-03-21T09:44:01.533 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_ct.ko 2024-03-21T09:44:01.533 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_dup_netdev.ko 2024-03-21T09:44:01.533 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_fib.ko 2024-03-21T09:44:01.533 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_fwd_netdev.ko 2024-03-21T09:44:01.533 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_hash.ko 2024-03-21T09:44:01.534 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_limit.ko 2024-03-21T09:44:01.534 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_log.ko 2024-03-21T09:44:01.534 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_masq.ko 2024-03-21T09:44:01.534 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_nat.ko 2024-03-21T09:44:01.534 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_numgen.ko 2024-03-21T09:44:01.534 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_queue.ko 2024-03-21T09:44:01.534 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_quota.ko 2024-03-21T09:44:01.534 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_redir.ko 2024-03-21T09:44:01.534 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_reject.ko 2024-03-21T09:44:01.534 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_reject_inet.ko 2024-03-21T09:44:01.534 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_socket.ko 2024-03-21T09:44:01.534 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/nft_tunnel.ko 2024-03-21T09:44:01.534 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_AUDIT.ko 2024-03-21T09:44:01.534 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_CHECKSUM.ko 2024-03-21T09:44:01.535 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_CLASSIFY.ko 2024-03-21T09:44:01.535 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-03-21T09:44:01.535 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_CT.ko 2024-03-21T09:44:01.535 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_DSCP.ko 2024-03-21T09:44:01.535 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_HL.ko 2024-03-21T09:44:01.535 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_HMARK.ko 2024-03-21T09:44:01.535 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_IDLETIMER.ko 2024-03-21T09:44:01.535 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_LED.ko 2024-03-21T09:44:01.535 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_LOG.ko 2024-03-21T09:44:01.535 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_MASQUERADE.ko 2024-03-21T09:44:01.535 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_NETMAP.ko 2024-03-21T09:44:01.535 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_NFLOG.ko 2024-03-21T09:44:01.535 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_NFQUEUE.ko 2024-03-21T09:44:01.536 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_RATEEST.ko 2024-03-21T09:44:01.536 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_REDIRECT.ko 2024-03-21T09:44:01.536 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_SECMARK.ko 2024-03-21T09:44:01.536 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_TCPMSS.ko 2024-03-21T09:44:01.536 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-03-21T09:44:01.536 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_TEE.ko 2024-03-21T09:44:01.536 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_TPROXY.ko 2024-03-21T09:44:01.536 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_TRACE.ko 2024-03-21T09:44:01.536 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_addrtype.ko 2024-03-21T09:44:01.536 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_bpf.ko 2024-03-21T09:44:01.536 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_cgroup.ko 2024-03-21T09:44:01.536 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_cluster.ko 2024-03-21T09:44:01.536 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_comment.ko 2024-03-21T09:44:01.536 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_connbytes.ko 2024-03-21T09:44:01.537 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_connlabel.ko 2024-03-21T09:44:01.537 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_connlimit.ko 2024-03-21T09:44:01.537 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_connmark.ko 2024-03-21T09:44:01.537 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_conntrack.ko 2024-03-21T09:44:01.537 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_cpu.ko 2024-03-21T09:44:01.537 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_dccp.ko 2024-03-21T09:44:01.537 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_devgroup.ko 2024-03-21T09:44:01.537 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_dscp.ko 2024-03-21T09:44:01.537 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_ecn.ko 2024-03-21T09:44:01.537 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_esp.ko 2024-03-21T09:44:01.537 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_hashlimit.ko 2024-03-21T09:44:01.537 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_helper.ko 2024-03-21T09:44:01.537 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_hl.ko 2024-03-21T09:44:01.538 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_ipcomp.ko 2024-03-21T09:44:01.538 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_iprange.ko 2024-03-21T09:44:01.538 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_ipvs.ko 2024-03-21T09:44:01.538 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_l2tp.ko 2024-03-21T09:44:01.538 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_length.ko 2024-03-21T09:44:01.538 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_limit.ko 2024-03-21T09:44:01.538 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_mac.ko 2024-03-21T09:44:01.538 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_mark.ko 2024-03-21T09:44:01.538 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_multiport.ko 2024-03-21T09:44:01.538 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_nat.ko 2024-03-21T09:44:01.538 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_nfacct.ko 2024-03-21T09:44:01.538 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_osf.ko 2024-03-21T09:44:01.538 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_owner.ko 2024-03-21T09:44:01.538 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_physdev.ko 2024-03-21T09:44:01.539 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_pkttype.ko 2024-03-21T09:44:01.539 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_policy.ko 2024-03-21T09:44:01.539 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_quota.ko 2024-03-21T09:44:01.539 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_rateest.ko 2024-03-21T09:44:01.539 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_realm.ko 2024-03-21T09:44:01.539 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_recent.ko 2024-03-21T09:44:01.539 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_sctp.ko 2024-03-21T09:44:01.539 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_set.ko 2024-03-21T09:44:01.539 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_socket.ko 2024-03-21T09:44:01.539 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_state.ko 2024-03-21T09:44:01.539 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_statistic.ko 2024-03-21T09:44:01.539 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_string.ko 2024-03-21T09:44:01.540 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_tcpmss.ko 2024-03-21T09:44:01.540 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_time.ko 2024-03-21T09:44:01.540 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netfilter/xt_u32.ko 2024-03-21T09:44:01.540 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netlink 2024-03-21T09:44:01.540 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/netlink/netlink_diag.ko 2024-03-21T09:44:01.540 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/nsh 2024-03-21T09:44:01.540 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/nsh/nsh.ko 2024-03-21T09:44:01.540 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/openvswitch 2024-03-21T09:44:01.540 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/openvswitch/openvswitch.ko 2024-03-21T09:44:01.540 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/openvswitch/vport-geneve.ko 2024-03-21T09:44:01.540 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/openvswitch/vport-gre.ko 2024-03-21T09:44:01.540 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/openvswitch/vport-vxlan.ko 2024-03-21T09:44:01.540 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/packet 2024-03-21T09:44:01.541 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/packet/af_packet_diag.ko 2024-03-21T09:44:01.541 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/rds 2024-03-21T09:44:01.541 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/rds/rds.ko 2024-03-21T09:44:01.541 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/rds/rds_rdma.ko 2024-03-21T09:44:01.541 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/rds/rds_tcp.ko 2024-03-21T09:44:01.541 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched 2024-03-21T09:44:01.541 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_csum.ko 2024-03-21T09:44:01.541 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_gact.ko 2024-03-21T09:44:01.541 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_mirred.ko 2024-03-21T09:44:01.541 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_nat.ko 2024-03-21T09:44:01.541 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_pedit.ko 2024-03-21T09:44:01.541 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_police.ko 2024-03-21T09:44:01.541 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_simple.ko 2024-03-21T09:44:01.541 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_skbedit.ko 2024-03-21T09:44:01.542 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/act_vlan.ko 2024-03-21T09:44:01.542 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_basic.ko 2024-03-21T09:44:01.542 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_bpf.ko 2024-03-21T09:44:01.542 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_flow.ko 2024-03-21T09:44:01.542 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_fw.ko 2024-03-21T09:44:01.542 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_route.ko 2024-03-21T09:44:01.542 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/cls_u32.ko 2024-03-21T09:44:01.542 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_cmp.ko 2024-03-21T09:44:01.542 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_ipset.ko 2024-03-21T09:44:01.542 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_meta.ko 2024-03-21T09:44:01.542 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_nbyte.ko 2024-03-21T09:44:01.542 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_text.ko 2024-03-21T09:44:01.542 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/em_u32.ko 2024-03-21T09:44:01.542 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_choke.ko 2024-03-21T09:44:01.543 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_codel.ko 2024-03-21T09:44:01.543 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_drr.ko 2024-03-21T09:44:01.543 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_fq.ko 2024-03-21T09:44:01.543 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_gred.ko 2024-03-21T09:44:01.543 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_hfsc.ko 2024-03-21T09:44:01.543 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_hhf.ko 2024-03-21T09:44:01.543 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_htb.ko 2024-03-21T09:44:01.543 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_ingress.ko 2024-03-21T09:44:01.543 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_mqprio.ko 2024-03-21T09:44:01.543 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_mqprio_lib.ko 2024-03-21T09:44:01.543 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_multiq.ko 2024-03-21T09:44:01.543 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_netem.ko 2024-03-21T09:44:01.543 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_pie.ko 2024-03-21T09:44:01.543 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_plug.ko 2024-03-21T09:44:01.543 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_prio.ko 2024-03-21T09:44:01.543 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_qfq.ko 2024-03-21T09:44:01.544 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_red.ko 2024-03-21T09:44:01.544 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_sfb.ko 2024-03-21T09:44:01.544 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_sfq.ko 2024-03-21T09:44:01.544 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_tbf.ko 2024-03-21T09:44:01.544 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sched/sch_teql.ko 2024-03-21T09:44:01.544 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sctp 2024-03-21T09:44:01.544 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sctp/sctp.ko 2024-03-21T09:44:01.544 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sctp/sctp_diag.ko 2024-03-21T09:44:01.544 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc 2024-03-21T09:44:01.544 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/auth_gss 2024-03-21T09:44:01.544 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-03-21T09:44:01.544 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-03-21T09:44:01.544 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/sunrpc.ko 2024-03-21T09:44:01.544 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/xprtrdma 2024-03-21T09:44:01.544 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-03-21T09:44:01.544 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/tipc 2024-03-21T09:44:01.545 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/tipc/diag.ko 2024-03-21T09:44:01.545 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/tipc/tipc.ko 2024-03-21T09:44:01.545 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/unix 2024-03-21T09:44:01.545 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/unix/unix_diag.ko 2024-03-21T09:44:01.545 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock 2024-03-21T09:44:01.545 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-03-21T09:44:01.545 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-03-21T09:44:01.545 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock/vsock.ko 2024-03-21T09:44:01.545 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock/vsock_diag.ko 2024-03-21T09:44:01.545 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/vmw_vsock/vsock_loopback.ko 2024-03-21T09:44:01.545 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/xfrm 2024-03-21T09:44:01.545 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/net/xfrm/xfrm_ipcomp.ko 2024-03-21T09:44:01.545 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security 2024-03-21T09:44:01.545 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security/keys 2024-03-21T09:44:01.546 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security/keys/encrypted-keys 2024-03-21T09:44:01.546 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-03-21T09:44:01.546 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security/keys/trusted-keys 2024-03-21T09:44:01.546 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/security/keys/trusted-keys/trusted.ko 2024-03-21T09:44:01.546 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/virt 2024-03-21T09:44:01.546 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/virt/lib 2024-03-21T09:44:01.546 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/kernel/virt/lib/irqbypass.ko 2024-03-21T09:44:01.546 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.alias 2024-03-21T09:44:01.546 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.alias.bin 2024-03-21T09:44:01.546 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.builtin 2024-03-21T09:44:01.546 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.builtin.alias.bin 2024-03-21T09:44:01.546 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.builtin.bin 2024-03-21T09:44:01.546 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.builtin.modinfo 2024-03-21T09:44:01.546 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.dep 2024-03-21T09:44:01.546 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.dep.bin 2024-03-21T09:44:01.547 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.devname 2024-03-21T09:44:01.547 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.order 2024-03-21T09:44:01.547 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.softdep 2024-03-21T09:44:01.547 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.symbols 2024-03-21T09:44:01.547 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/modules.symbols.bin 2024-03-21T09:44:01.547 INFO:teuthology.orchestra.run.smithi153.stdout:/lib/modules/6.8.0-g5ec3e6e88b2e/vmlinuz 2024-03-21T09:44:01.550 DEBUG:teuthology.task.kernel:get_image_version: 6.8.0-g5ec3e6e88b2e 2024-03-21T09:44:01.550 DEBUG:teuthology.orchestra.run.smithi153:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-03-21T09:44:01.572 INFO:teuthology.orchestra.run.smithi153.stdout:postinstall scriptlet (using /bin/sh): 2024-03-21T09:44:01.573 INFO:teuthology.orchestra.run.smithi153.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-03-21T09:44:01.573 INFO:teuthology.orchestra.run.smithi153.stdout: /usr/bin/kernel-install add 6.8.0-g5ec3e6e88b2e /lib/modules/6.8.0-g5ec3e6e88b2e/vmlinuz 2024-03-21T09:44:01.573 INFO:teuthology.orchestra.run.smithi153.stdout:fi 2024-03-21T09:44:01.573 INFO:teuthology.orchestra.run.smithi153.stdout:for file in vmlinuz System.map config; do 2024-03-21T09:44:01.573 INFO:teuthology.orchestra.run.smithi153.stdout: if ! cmp --silent "/lib/modules/6.8.0-g5ec3e6e88b2e/${file}" "/boot/${file}-6.8.0-g5ec3e6e88b2e"; then 2024-03-21T09:44:01.573 INFO:teuthology.orchestra.run.smithi153.stdout: cp "/lib/modules/6.8.0-g5ec3e6e88b2e/${file}" "/boot/${file}-6.8.0-g5ec3e6e88b2e" 2024-03-21T09:44:01.573 INFO:teuthology.orchestra.run.smithi153.stdout: fi 2024-03-21T09:44:01.573 INFO:teuthology.orchestra.run.smithi153.stdout:done 2024-03-21T09:44:01.573 INFO:teuthology.orchestra.run.smithi153.stdout:preuninstall scriptlet (using /bin/sh): 2024-03-21T09:44:01.573 INFO:teuthology.orchestra.run.smithi153.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2024-03-21T09:44:01.573 INFO:teuthology.orchestra.run.smithi153.stdout:new-kernel-pkg --remove 6.8.0-g5ec3e6e88b2e --rminitrd --initrdfile=/boot/initramfs-6.8.0-g5ec3e6e88b2e.img 2024-03-21T09:44:01.573 INFO:teuthology.orchestra.run.smithi153.stdout:elif [ -x /usr/bin/kernel-install ]; then 2024-03-21T09:44:01.573 INFO:teuthology.orchestra.run.smithi153.stdout:kernel-install remove 6.8.0-g5ec3e6e88b2e 2024-03-21T09:44:01.574 INFO:teuthology.orchestra.run.smithi153.stdout:fi 2024-03-21T09:44:01.574 INFO:teuthology.orchestra.run.smithi153.stdout:postuninstall scriptlet (using /bin/sh): 2024-03-21T09:44:01.574 INFO:teuthology.orchestra.run.smithi153.stdout:if [ -x /sbin/update-bootloader ]; then 2024-03-21T09:44:01.574 INFO:teuthology.orchestra.run.smithi153.stdout:/sbin/update-bootloader --remove 6.8.0-g5ec3e6e88b2e 2024-03-21T09:44:01.574 INFO:teuthology.orchestra.run.smithi153.stdout:fi 2024-03-21T09:44:01.575 DEBUG:teuthology.orchestra.run.smithi153:> sudo rpm -qi grub2-tools 2024-03-21T09:44:01.649 INFO:teuthology.orchestra.run.smithi153.stdout:Name : grub2-tools 2024-03-21T09:44:01.649 INFO:teuthology.orchestra.run.smithi153.stdout:Epoch : 1 2024-03-21T09:44:01.649 INFO:teuthology.orchestra.run.smithi153.stdout:Version : 2.06 2024-03-21T09:44:01.649 INFO:teuthology.orchestra.run.smithi153.stdout:Release : 68.el9 2024-03-21T09:44:01.649 INFO:teuthology.orchestra.run.smithi153.stdout:Architecture: x86_64 2024-03-21T09:44:01.649 INFO:teuthology.orchestra.run.smithi153.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-21T09:44:01.649 INFO:teuthology.orchestra.run.smithi153.stdout:Group : Unspecified 2024-03-21T09:44:01.649 INFO:teuthology.orchestra.run.smithi153.stdout:Size : 8264254 2024-03-21T09:44:01.649 INFO:teuthology.orchestra.run.smithi153.stdout:License : GPLv3+ 2024-03-21T09:44:01.649 INFO:teuthology.orchestra.run.smithi153.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-21T09:44:01.650 INFO:teuthology.orchestra.run.smithi153.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-21T09:44:01.650 INFO:teuthology.orchestra.run.smithi153.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-21T09:44:01.650 INFO:teuthology.orchestra.run.smithi153.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-21T09:44:01.650 INFO:teuthology.orchestra.run.smithi153.stdout:Packager : builder@centos.org 2024-03-21T09:44:01.650 INFO:teuthology.orchestra.run.smithi153.stdout:Vendor : CentOS 2024-03-21T09:44:01.650 INFO:teuthology.orchestra.run.smithi153.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-21T09:44:01.650 INFO:teuthology.orchestra.run.smithi153.stdout:Summary : Support tools for GRUB. 2024-03-21T09:44:01.650 INFO:teuthology.orchestra.run.smithi153.stdout:Description : 2024-03-21T09:44:01.650 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T09:44:01.650 INFO:teuthology.orchestra.run.smithi153.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-21T09:44:01.650 INFO:teuthology.orchestra.run.smithi153.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-21T09:44:01.650 INFO:teuthology.orchestra.run.smithi153.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-21T09:44:01.650 INFO:teuthology.orchestra.run.smithi153.stdout:hardware devices. 2024-03-21T09:44:01.650 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T09:44:01.651 INFO:teuthology.orchestra.run.smithi153.stdout:This subpackage provides tools for support of all platforms. 2024-03-21T09:44:01.651 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-21T09:44:01.652 INFO:teuthology.task.kernel:Updating grub on smithi153 to boot 6.8.0-g5ec3e6e88b2e 2024-03-21T09:44:01.652 DEBUG:teuthology.orchestra.run.smithi153:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-21T09:44:02.233 INFO:teuthology.orchestra.run.smithi153.stderr:Generating grub configuration file ... 2024-03-21T09:44:03.667 INFO:teuthology.orchestra.run.smithi153.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-21T09:44:03.691 INFO:teuthology.orchestra.run.smithi153.stderr:done 2024-03-21T09:44:03.693 DEBUG:teuthology.orchestra.run.smithi153:> mktemp 2024-03-21T09:44:03.708 INFO:teuthology.orchestra.run.smithi153.stdout:/tmp/tmp.68rd8tqXcX 2024-03-21T09:44:03.709 DEBUG:teuthology.orchestra.run.smithi153:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.68rd8tqXcX 2024-03-21T09:44:03.781 DEBUG:teuthology.orchestra.run.smithi153:> sudo chmod 0666 /tmp/tmp.68rd8tqXcX 2024-03-21T09:44:03.924 DEBUG:teuthology.orchestra.remote:smithi153:/tmp/tmp.68rd8tqXcX is 6KB 2024-03-21T09:44:03.974 DEBUG:teuthology.orchestra.run.smithi153:> rm -fr /tmp/tmp.68rd8tqXcX 2024-03-21T09:44:03.988 DEBUG:teuthology.orchestra.run.smithi153:> sudo /bin/ls /boot/loader/entries || true 2024-03-21T09:44:04.053 INFO:teuthology.orchestra.run.smithi153.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-21T09:44:04.053 INFO:teuthology.orchestra.run.smithi153.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-21T09:44:04.053 INFO:teuthology.orchestra.run.smithi153.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-21T09:44:04.053 INFO:teuthology.orchestra.run.smithi153.stdout:15949e3c88704300882482eddfd7baae-6.8.0-g5ec3e6e88b2e.conf 2024-03-21T09:44:04.054 DEBUG:teuthology.orchestra.run.smithi153:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.8.0-g5ec3e6e88b2e 2024-03-21T09:44:04.186 DEBUG:teuthology.orchestra.run.smithi153:> sudo shutdown -r now 2024-03-21T09:44:14.590 INFO:teuthology.task.kernel:Checking client host.b for new kernel version... 2024-03-21T09:44:14.591 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi174.front.sepia.ceph.com' 2024-03-21T09:44:14.591 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T09:44:33.058 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.174 2024-03-21T09:44:34.190 INFO:teuthology.task.kernel:Checking client host.a for new kernel version... 2024-03-21T09:44:34.191 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi153.front.sepia.ceph.com' 2024-03-21T09:44:34.191 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi153.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T09:44:42.066 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi174.front.sepia.ceph.com' 2024-03-21T09:44:42.067 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T09:44:45.118 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.174 2024-03-21T09:44:52.510 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.153 2024-03-21T09:44:57.123 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi174.front.sepia.ceph.com' 2024-03-21T09:44:57.124 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T09:45:01.514 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi153.front.sepia.ceph.com' 2024-03-21T09:45:01.515 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi153.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T09:45:04.574 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.153 2024-03-21T09:45:16.586 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi153.front.sepia.ceph.com' 2024-03-21T09:45:16.587 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi153.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T09:45:29.097 DEBUG:teuthology.orchestra.run.smithi174:> true 2024-03-21T09:45:29.669 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi174.front.sepia.ceph.com' 2024-03-21T09:45:29.669 INFO:teuthology.task.kernel:Checking kernel version of host.b, want "6.8.0-g5ec3e6e88b2e"... 2024-03-21T09:45:29.670 DEBUG:teuthology.orchestra.run.smithi174:> uname -r 2024-03-21T09:45:29.711 INFO:teuthology.orchestra.run.smithi174.stdout:6.8.0-g5ec3e6e88b2e 2024-03-21T09:45:29.711 DEBUG:teuthology.task.kernel:current kernel version is 6.8.0-g5ec3e6e88b2e vs 6.8.0-g5ec3e6e88b2e 2024-03-21T09:45:29.711 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-21T09:45:29.712 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-21T09:45:30.712 INFO:teuthology.task.kernel:Enabling kdb on host.b... 2024-03-21T09:45:30.712 DEBUG:teuthology.orchestra.run.smithi174:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-21T09:45:30.838 INFO:teuthology.orchestra.run.smithi174.stdout:ttyS1 2024-03-21T09:45:30.887 DEBUG:teuthology.parallel:result is None 2024-03-21T09:45:35.006 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.153 2024-03-21T09:45:50.007 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi153.front.sepia.ceph.com' 2024-03-21T09:45:50.008 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi153.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T09:45:50.516 DEBUG:teuthology.orchestra.run.smithi153:> true 2024-03-21T09:45:51.180 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi153.front.sepia.ceph.com' 2024-03-21T09:45:51.181 INFO:teuthology.task.kernel:Checking kernel version of host.a, want "6.8.0-g5ec3e6e88b2e"... 2024-03-21T09:45:51.181 DEBUG:teuthology.orchestra.run.smithi153:> uname -r 2024-03-21T09:45:51.223 INFO:teuthology.orchestra.run.smithi153.stdout:6.8.0-g5ec3e6e88b2e 2024-03-21T09:45:51.223 DEBUG:teuthology.task.kernel:current kernel version is 6.8.0-g5ec3e6e88b2e vs 6.8.0-g5ec3e6e88b2e 2024-03-21T09:45:51.223 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-21T09:45:51.223 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-21T09:45:52.224 INFO:teuthology.task.kernel:Enabling kdb on host.a... 2024-03-21T09:45:52.224 DEBUG:teuthology.orchestra.run.smithi153:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-21T09:45:52.326 INFO:teuthology.orchestra.run.smithi153.stdout:ttyS1 2024-03-21T09:45:52.383 DEBUG:teuthology.parallel:result is None 2024-03-21T09:45:52.384 INFO:teuthology.run_tasks:Running task internal.base... 2024-03-21T09:45:52.397 INFO:teuthology.task.internal:Creating test directory... 2024-03-21T09:45:52.397 DEBUG:teuthology.orchestra.run.smithi153:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-21T09:45:52.403 DEBUG:teuthology.orchestra.run.smithi174:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-21T09:45:52.463 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-03-21T09:45:52.470 INFO:teuthology.run_tasks:Running task internal.archive... 2024-03-21T09:45:52.492 INFO:teuthology.task.internal:Creating archive directory... 2024-03-21T09:45:52.492 DEBUG:teuthology.orchestra.run.smithi153:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-21T09:45:52.499 DEBUG:teuthology.orchestra.run.smithi174:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-21T09:45:52.565 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-03-21T09:45:52.572 INFO:teuthology.task.internal:Enabling coredump saving... 2024-03-21T09:45:52.572 DEBUG:teuthology.orchestra.run.smithi153:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-21T09:45:52.586 DEBUG:teuthology.orchestra.run.smithi174:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-21T09:45:52.661 INFO:teuthology.orchestra.run.smithi153.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-21T09:45:52.683 INFO:teuthology.orchestra.run.smithi174.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-21T09:45:52.695 INFO:teuthology.orchestra.run.smithi153.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-21T09:45:52.725 INFO:teuthology.orchestra.run.smithi174.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-21T09:45:52.729 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-03-21T09:45:52.737 INFO:teuthology.task.internal:Configuring sudo... 2024-03-21T09:45:52.737 DEBUG:teuthology.orchestra.run.smithi153:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-21T09:45:52.741 DEBUG:teuthology.orchestra.run.smithi174:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-21T09:45:52.840 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-03-21T09:45:52.850 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-03-21T09:45:52.850 DEBUG:teuthology.orchestra.run.smithi153:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-21T09:45:52.857 DEBUG:teuthology.orchestra.run.smithi174:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-21T09:45:52.911 DEBUG:teuthology.orchestra.run.smithi153:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-21T09:45:53.003 DEBUG:teuthology.orchestra.run.smithi153:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-21T09:45:53.101 DEBUG:teuthology.orchestra.run.smithi153:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-21T09:45:53.168 DEBUG:teuthology.orchestra.run.smithi153:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-21T09:45:53.252 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-03-21T09:45:53.253 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-21T09:45:53.324 DEBUG:teuthology.orchestra.run.smithi174:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-21T09:45:53.410 DEBUG:teuthology.orchestra.run.smithi174:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-21T09:45:53.514 DEBUG:teuthology.orchestra.run.smithi174:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-21T09:45:53.600 DEBUG:teuthology.orchestra.run.smithi174:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-21T09:45:53.698 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-03-21T09:45:53.698 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-21T09:45:53.786 DEBUG:teuthology.orchestra.run.smithi153:> sudo service rsyslog restart 2024-03-21T09:45:53.791 DEBUG:teuthology.orchestra.run.smithi174:> sudo service rsyslog restart 2024-03-21T09:45:53.889 INFO:teuthology.orchestra.run.smithi153.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-21T09:45:53.964 INFO:teuthology.orchestra.run.smithi174.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-21T09:45:54.284 INFO:teuthology.run_tasks:Running task internal.timer... 2024-03-21T09:45:54.292 INFO:teuthology.task.internal:Starting timer... 2024-03-21T09:45:54.292 INFO:teuthology.run_tasks:Running task pcp... 2024-03-21T09:45:54.321 INFO:teuthology.run_tasks:Running task selinux... 2024-03-21T09:45:54.352 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-03-21T09:45:54.353 DEBUG:teuthology.orchestra.run.smithi153:> sudo service auditd rotate 2024-03-21T09:45:54.480 INFO:teuthology.orchestra.run.smithi153.stdout:Rotating logs: 2024-03-21T09:45:54.484 DEBUG:teuthology.orchestra.run.smithi174:> sudo service auditd rotate 2024-03-21T09:45:54.609 INFO:teuthology.orchestra.run.smithi174.stdout:Rotating logs: 2024-03-21T09:45:54.613 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-03-21T09:45:54.613 DEBUG:teuthology.orchestra.run.smithi153:> /usr/sbin/getenforce 2024-03-21T09:45:54.660 INFO:teuthology.orchestra.run.smithi153.stdout:Permissive 2024-03-21T09:45:54.661 DEBUG:teuthology.orchestra.run.smithi174:> /usr/sbin/getenforce 2024-03-21T09:45:54.717 INFO:teuthology.orchestra.run.smithi174.stdout:Permissive 2024-03-21T09:45:54.717 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi153.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi174.front.sepia.ceph.com': 'permissive'} 2024-03-21T09:45:54.718 DEBUG:teuthology.orchestra.run.smithi153:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-21T09:45:54.794 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-21T09:45:54.794 DEBUG:teuthology.orchestra.run.smithi174:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-21T09:45:54.880 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-21T09:45:54.881 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-03-21T09:45:54.881 DEBUG:teuthology.orchestra.run.smithi153:> sudo /usr/sbin/setenforce permissive 2024-03-21T09:45:54.973 DEBUG:teuthology.orchestra.run.smithi174:> sudo /usr/sbin/setenforce permissive 2024-03-21T09:45:55.057 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-03-21T09:45:55.067 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-03-21T09:45:55.084 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-03-21T09:45:55.086 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi153.front.sepia.ceph.com,smithi174.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-03-21T09:52:02.932 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi153.front.sepia.ceph.com'), Remote(name='ubuntu@smithi174.front.sepia.ceph.com')] 2024-03-21T09:52:02.933 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi153.front.sepia.ceph.com' 2024-03-21T09:52:02.935 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi153.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T09:52:03.075 DEBUG:teuthology.orchestra.run.smithi153:> true 2024-03-21T09:52:03.229 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi153.front.sepia.ceph.com' 2024-03-21T09:52:03.230 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi174.front.sepia.ceph.com' 2024-03-21T09:52:03.230 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T09:52:03.366 DEBUG:teuthology.orchestra.run.smithi174:> true 2024-03-21T09:52:03.522 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi174.front.sepia.ceph.com' 2024-03-21T09:52:03.523 INFO:teuthology.run_tasks:Running task clock... 2024-03-21T09:52:03.537 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-03-21T09:52:03.537 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-21T09:52:03.538 DEBUG:teuthology.orchestra.run.smithi153:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-21T09:52:03.546 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-21T09:52:03.546 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-21T09:52:03.641 INFO:teuthology.orchestra.run.smithi153.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-21T09:52:03.649 INFO:teuthology.orchestra.run.smithi174.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-21T09:52:03.691 INFO:teuthology.orchestra.run.smithi153.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-21T09:52:03.701 INFO:teuthology.orchestra.run.smithi174.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-21T09:52:03.786 INFO:teuthology.orchestra.run.smithi153.stderr:sudo: ntpd: command not found 2024-03-21T09:52:03.798 INFO:teuthology.orchestra.run.smithi174.stderr:sudo: ntpd: command not found 2024-03-21T09:52:03.827 INFO:teuthology.orchestra.run.smithi153.stdout:506 Cannot talk to daemon 2024-03-21T09:52:03.841 INFO:teuthology.orchestra.run.smithi174.stdout:506 Cannot talk to daemon 2024-03-21T09:52:03.880 INFO:teuthology.orchestra.run.smithi153.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-21T09:52:03.893 INFO:teuthology.orchestra.run.smithi174.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-21T09:52:03.935 INFO:teuthology.orchestra.run.smithi153.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-21T09:52:03.949 INFO:teuthology.orchestra.run.smithi174.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-21T09:52:04.060 INFO:teuthology.orchestra.run.smithi153.stderr:bash: line 1: ntpq: command not found 2024-03-21T09:52:04.068 INFO:teuthology.orchestra.run.smithi153.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-21T09:52:04.068 INFO:teuthology.orchestra.run.smithi153.stdout:=============================================================================== 2024-03-21T09:52:04.068 INFO:teuthology.orchestra.run.smithi153.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T09:52:04.068 INFO:teuthology.orchestra.run.smithi153.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T09:52:04.069 INFO:teuthology.orchestra.run.smithi153.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T09:52:04.069 INFO:teuthology.orchestra.run.smithi153.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T09:52:04.077 INFO:teuthology.orchestra.run.smithi174.stderr:bash: line 1: ntpq: command not found 2024-03-21T09:52:04.083 INFO:teuthology.orchestra.run.smithi174.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-21T09:52:04.084 INFO:teuthology.orchestra.run.smithi174.stdout:=============================================================================== 2024-03-21T09:52:04.084 INFO:teuthology.orchestra.run.smithi174.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T09:52:04.084 INFO:teuthology.orchestra.run.smithi174.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T09:52:04.084 INFO:teuthology.orchestra.run.smithi174.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T09:52:04.084 INFO:teuthology.orchestra.run.smithi174.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T09:52:04.085 INFO:teuthology.run_tasks:Running task install... 2024-03-21T09:52:04.093 DEBUG:teuthology.task.install:project ceph 2024-03-21T09:52:04.093 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '2edfee68ad7243bed422a049b6f5f58619300f11'}} 2024-03-21T09:52:04.093 DEBUG:teuthology.task.install:config {'exclude_packages': ['ceph-volume'], 'tag': 'v18.2.1', 'flavor': 'default', 'sha1': '2edfee68ad7243bed422a049b6f5f58619300f11'} 2024-03-21T09:52:04.093 INFO:teuthology.task.install:Using flavor: default 2024-03-21T09:52:04.099 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-03-21T09:52:04.099 INFO:teuthology.task.install:extra packages: [] 2024-03-21T09:52:04.099 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2edfee68ad7243bed422a049b6f5f58619300f11', 'tag': 'v18.2.1', 'wait_for_package': False} 2024-03-21T09:52:04.100 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-03-21T09:52:04.100 INFO:teuthology.packaging:ref: None 2024-03-21T09:52:04.100 INFO:teuthology.packaging:tag: v18.2.1 2024-03-21T09:52:04.100 INFO:teuthology.packaging:branch: None 2024-03-21T09:52:04.100 INFO:teuthology.packaging:sha1: 2edfee68ad7243bed422a049b6f5f58619300f11 2024-03-21T09:52:04.497 DEBUG:teuthology.repo_utils:git ls-remote https://git.ceph.com/ceph-ci v18.2.1^{} -> None 2024-03-21T09:52:04.497 INFO:teuthology.packaging:Tag 'v18.2.1' not found in https://git.ceph.com/ceph-ci; will also look in https://git.ceph.com/ceph 2024-03-21T09:52:06.728 DEBUG:teuthology.repo_utils:git ls-remote https://git.ceph.com/ceph v18.2.1^{} -> 7fe91d5d5842e04be3b4f514d6dd990c54b29c76 2024-03-21T09:52:06.728 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7fe91d5d5842e04be3b4f514d6dd990c54b29c76 2024-03-21T09:52:06.731 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2edfee68ad7243bed422a049b6f5f58619300f11', 'tag': 'v18.2.1', 'wait_for_package': False} 2024-03-21T09:52:06.732 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-03-21T09:52:06.732 INFO:teuthology.packaging:ref: None 2024-03-21T09:52:06.732 INFO:teuthology.packaging:tag: v18.2.1 2024-03-21T09:52:06.732 INFO:teuthology.packaging:branch: None 2024-03-21T09:52:06.732 INFO:teuthology.packaging:sha1: 2edfee68ad7243bed422a049b6f5f58619300f11 2024-03-21T09:52:06.732 INFO:teuthology.packaging:Tag 'v18.2.1' not found in https://git.ceph.com/ceph-ci; will also look in https://git.ceph.com/ceph 2024-03-21T09:52:06.732 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7fe91d5d5842e04be3b4f514d6dd990c54b29c76 2024-03-21T09:52:06.886 INFO:teuthology.task.install.rpm:Pulling from https://chacra.ceph.com/r/ceph/reef/7fe91d5d5842e04be3b4f514d6dd990c54b29c76/centos/9/flavors/default/ 2024-03-21T09:52:06.886 INFO:teuthology.task.install.rpm:Package version is 18.2.1-0 2024-03-21T09:52:06.899 INFO:teuthology.task.install.rpm:Pulling from https://chacra.ceph.com/r/ceph/reef/7fe91d5d5842e04be3b4f514d6dd990c54b29c76/centos/9/flavors/default/ 2024-03-21T09:52:06.899 INFO:teuthology.task.install.rpm:Package version is 18.2.1-0 2024-03-21T09:52:06.928 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://chacra.ceph.com/r/ceph/reef/7fe91d5d5842e04be3b4f514d6dd990c54b29c76/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://chacra.ceph.com/r/ceph/reef/7fe91d5d5842e04be3b4f514d6dd990c54b29c76/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://chacra.ceph.com/r/ceph/reef/7fe91d5d5842e04be3b4f514d6dd990c54b29c76/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-21T09:52:06.928 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-03-21T09:52:06.928 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-21T09:52:06.936 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://chacra.ceph.com/r/ceph/reef/7fe91d5d5842e04be3b4f514d6dd990c54b29c76/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://chacra.ceph.com/r/ceph/reef/7fe91d5d5842e04be3b4f514d6dd990c54b29c76/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://chacra.ceph.com/r/ceph/reef/7fe91d5d5842e04be3b4f514d6dd990c54b29c76/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-21T09:52:06.937 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-03-21T09:52:06.937 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-21T09:52:07.030 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-03-21T09:52:07.030 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-03-21T09:52:07.030 INFO:teuthology.packaging:ref: None 2024-03-21T09:52:07.030 INFO:teuthology.packaging:tag: v18.2.1 2024-03-21T09:52:07.030 INFO:teuthology.packaging:branch: None 2024-03-21T09:52:07.030 INFO:teuthology.packaging:sha1: 2edfee68ad7243bed422a049b6f5f58619300f11 2024-03-21T09:52:07.030 DEBUG:teuthology.orchestra.run.smithi174:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/v18.2.1/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-21T09:52:07.036 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-03-21T09:52:07.037 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-03-21T09:52:07.037 INFO:teuthology.packaging:ref: None 2024-03-21T09:52:07.037 INFO:teuthology.packaging:tag: v18.2.1 2024-03-21T09:52:07.037 INFO:teuthology.packaging:branch: None 2024-03-21T09:52:07.037 INFO:teuthology.packaging:sha1: 2edfee68ad7243bed422a049b6f5f58619300f11 2024-03-21T09:52:07.037 DEBUG:teuthology.orchestra.run.smithi153:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/v18.2.1/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-21T09:52:07.168 DEBUG:teuthology.orchestra.run.smithi174:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-21T09:52:07.176 DEBUG:teuthology.orchestra.run.smithi153:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-21T09:52:07.294 DEBUG:teuthology.orchestra.run.smithi174:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-21T09:52:07.305 DEBUG:teuthology.orchestra.run.smithi153:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-21T09:52:07.381 INFO:teuthology.orchestra.run.smithi174.stdout:check_obsoletes = 1 2024-03-21T09:52:07.386 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum clean all 2024-03-21T09:52:07.390 INFO:teuthology.orchestra.run.smithi153.stdout:check_obsoletes = 1 2024-03-21T09:52:07.394 DEBUG:teuthology.orchestra.run.smithi153:> sudo yum clean all 2024-03-21T09:52:07.733 INFO:teuthology.orchestra.run.smithi174.stdout:74 files removed 2024-03-21T09:52:07.757 INFO:teuthology.orchestra.run.smithi153.stdout:74 files removed 2024-03-21T09:52:07.774 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum -y install ceph-radosgw 2024-03-21T09:52:07.796 DEBUG:teuthology.orchestra.run.smithi153:> sudo yum -y install ceph-radosgw 2024-03-21T09:52:08.489 INFO:teuthology.orchestra.run.smithi174.stdout:ceph packages for x86_64 364 kB/s | 76 kB 00:00 2024-03-21T09:52:08.503 INFO:teuthology.orchestra.run.smithi153.stdout:ceph packages for x86_64 380 kB/s | 76 kB 00:00 2024-03-21T09:52:08.754 INFO:teuthology.orchestra.run.smithi153.stdout:ceph noarch packages 51 kB/s | 9.4 kB 00:00 2024-03-21T09:52:08.773 INFO:teuthology.orchestra.run.smithi174.stdout:ceph noarch packages 42 kB/s | 9.4 kB 00:00 2024-03-21T09:52:09.023 INFO:teuthology.orchestra.run.smithi153.stdout:ceph source packages 9.5 kB/s | 2.2 kB 00:00 2024-03-21T09:52:09.040 INFO:teuthology.orchestra.run.smithi174.stdout:ceph source packages 10 kB/s | 2.2 kB 00:00 2024-03-21T09:52:09.567 INFO:teuthology.orchestra.run.smithi153.stdout:CentOS-9 - AppStream 14 MB/s | 6.7 MB 00:00 2024-03-21T09:52:09.837 INFO:teuthology.orchestra.run.smithi174.stdout:CentOS-9 - AppStream 8.9 MB/s | 6.7 MB 00:00 2024-03-21T09:52:11.159 INFO:teuthology.orchestra.run.smithi153.stdout:CentOS-9 - BaseOS 6.3 MB/s | 2.0 MB 00:00 2024-03-21T09:52:11.596 INFO:teuthology.orchestra.run.smithi174.stdout:CentOS-9 - BaseOS 4.2 MB/s | 2.0 MB 00:00 2024-03-21T09:52:13.164 INFO:teuthology.orchestra.run.smithi153.stdout:CentOS Stream 9 - BaseOS 4.9 MB/s | 8.0 MB 00:01 2024-03-21T09:52:13.641 INFO:teuthology.orchestra.run.smithi174.stdout:CentOS Stream 9 - BaseOS 4.9 MB/s | 8.0 MB 00:01 2024-03-21T09:52:16.157 INFO:teuthology.orchestra.run.smithi153.stdout:CentOS Stream 9 - AppStream 10 MB/s | 19 MB 00:01 2024-03-21T09:52:20.542 INFO:teuthology.orchestra.run.smithi153.stdout:CentOS Stream 9 - CRB 10 MB/s | 6.1 MB 00:00 2024-03-21T09:52:22.175 INFO:teuthology.orchestra.run.smithi153.stdout:CentOS Stream 9 - Extras packages 53 kB/s | 16 kB 00:00 2024-03-21T09:52:24.044 INFO:teuthology.orchestra.run.smithi153.stdout:Extra Packages for Enterprise Linux 12 MB/s | 21 MB 00:01 2024-03-21T09:52:25.157 INFO:teuthology.orchestra.run.smithi174.stdout:CentOS Stream 9 - AppStream 1.8 MB/s | 19 MB 00:10 2024-03-21T09:52:29.860 INFO:teuthology.orchestra.run.smithi153.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-03-21T09:52:29.997 INFO:teuthology.orchestra.run.smithi174.stdout:CentOS Stream 9 - CRB 6.0 MB/s | 6.1 MB 00:01 2024-03-21T09:52:31.700 INFO:teuthology.orchestra.run.smithi174.stdout:CentOS Stream 9 - Extras packages 39 kB/s | 16 kB 00:00 2024-03-21T09:52:32.456 INFO:teuthology.orchestra.run.smithi153.stdout:Dependencies resolved. 2024-03-21T09:52:32.472 INFO:teuthology.orchestra.run.smithi153.stdout:================================================================================ 2024-03-21T09:52:32.472 INFO:teuthology.orchestra.run.smithi153.stdout: Package Arch Version Repository Size 2024-03-21T09:52:32.472 INFO:teuthology.orchestra.run.smithi153.stdout:================================================================================ 2024-03-21T09:52:32.472 INFO:teuthology.orchestra.run.smithi153.stdout:Installing: 2024-03-21T09:52:32.473 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-radosgw x86_64 2:18.2.1-0.el9 ceph 7.7 M 2024-03-21T09:52:32.473 INFO:teuthology.orchestra.run.smithi153.stdout:Upgrading: 2024-03-21T09:52:32.473 INFO:teuthology.orchestra.run.smithi153.stdout: librados2 x86_64 2:18.2.1-0.el9 ceph 3.3 M 2024-03-21T09:52:32.473 INFO:teuthology.orchestra.run.smithi153.stdout: librbd1 x86_64 2:18.2.1-0.el9 ceph 3.0 M 2024-03-21T09:52:32.473 INFO:teuthology.orchestra.run.smithi153.stdout:Installing dependencies: 2024-03-21T09:52:32.473 INFO:teuthology.orchestra.run.smithi153.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-21T09:52:32.473 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-base x86_64 2:18.2.1-0.el9 ceph 5.2 M 2024-03-21T09:52:32.473 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-common x86_64 2:18.2.1-0.el9 ceph 18 M 2024-03-21T09:52:32.473 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-selinux x86_64 2:18.2.1-0.el9 ceph 24 k 2024-03-21T09:52:32.473 INFO:teuthology.orchestra.run.smithi153.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2024-03-21T09:52:32.473 INFO:teuthology.orchestra.run.smithi153.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-21T09:52:32.473 INFO:teuthology.orchestra.run.smithi153.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-21T09:52:32.473 INFO:teuthology.orchestra.run.smithi153.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-21T09:52:32.473 INFO:teuthology.orchestra.run.smithi153.stdout: libcephfs2 x86_64 2:18.2.1-0.el9 ceph 658 k 2024-03-21T09:52:32.474 INFO:teuthology.orchestra.run.smithi153.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-21T09:52:32.474 INFO:teuthology.orchestra.run.smithi153.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-21T09:52:32.474 INFO:teuthology.orchestra.run.smithi153.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-21T09:52:32.474 INFO:teuthology.orchestra.run.smithi153.stdout: libradosstriper1 x86_64 2:18.2.1-0.el9 ceph 474 k 2024-03-21T09:52:32.474 INFO:teuthology.orchestra.run.smithi153.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-21T09:52:32.474 INFO:teuthology.orchestra.run.smithi153.stdout: librgw2 x86_64 2:18.2.1-0.el9 ceph 4.5 M 2024-03-21T09:52:32.474 INFO:teuthology.orchestra.run.smithi153.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-21T09:52:32.474 INFO:teuthology.orchestra.run.smithi153.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-21T09:52:32.474 INFO:teuthology.orchestra.run.smithi153.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-21T09:52:32.474 INFO:teuthology.orchestra.run.smithi153.stdout: python3-ceph-argparse x86_64 2:18.2.1-0.el9 ceph 45 k 2024-03-21T09:52:32.474 INFO:teuthology.orchestra.run.smithi153.stdout: python3-ceph-common x86_64 2:18.2.1-0.el9 ceph 124 k 2024-03-21T09:52:32.475 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cephfs x86_64 2:18.2.1-0.el9 ceph 161 k 2024-03-21T09:52:32.475 INFO:teuthology.orchestra.run.smithi153.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-21T09:52:32.475 INFO:teuthology.orchestra.run.smithi153.stdout: python3-rados x86_64 2:18.2.1-0.el9 ceph 321 k 2024-03-21T09:52:32.475 INFO:teuthology.orchestra.run.smithi153.stdout: python3-rbd x86_64 2:18.2.1-0.el9 ceph 297 k 2024-03-21T09:52:32.475 INFO:teuthology.orchestra.run.smithi153.stdout: python3-rgw x86_64 2:18.2.1-0.el9 ceph 99 k 2024-03-21T09:52:32.475 INFO:teuthology.orchestra.run.smithi153.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-21T09:52:32.475 INFO:teuthology.orchestra.run.smithi153.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-21T09:52:32.475 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T09:52:32.475 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction Summary 2024-03-21T09:52:32.475 INFO:teuthology.orchestra.run.smithi153.stdout:================================================================================ 2024-03-21T09:52:32.475 INFO:teuthology.orchestra.run.smithi153.stdout:Install 28 Packages 2024-03-21T09:52:32.475 INFO:teuthology.orchestra.run.smithi153.stdout:Upgrade 2 Packages 2024-03-21T09:52:32.475 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T09:52:32.476 INFO:teuthology.orchestra.run.smithi153.stdout:Total download size: 53 M 2024-03-21T09:52:32.476 INFO:teuthology.orchestra.run.smithi153.stdout:Downloading Packages: 2024-03-21T09:52:32.927 INFO:teuthology.orchestra.run.smithi174.stdout:Extra Packages for Enterprise Linux 20 MB/s | 21 MB 00:01 2024-03-21T09:52:33.020 INFO:teuthology.orchestra.run.smithi153.stdout:(1/30): ceph-base-18.2.1-0.el9.x86_64.rpm 11 MB/s | 5.2 MB 00:00 2024-03-21T09:52:33.220 INFO:teuthology.orchestra.run.smithi153.stdout:(2/30): ceph-selinux-18.2.1-0.el9.x86_64.rpm 121 kB/s | 24 kB 00:00 2024-03-21T09:52:33.362 INFO:teuthology.orchestra.run.smithi153.stdout:(3/30): ceph-radosgw-18.2.1-0.el9.x86_64.rpm 9.3 MB/s | 7.7 MB 00:00 2024-03-21T09:52:33.404 INFO:teuthology.orchestra.run.smithi153.stdout:(4/30): libcephfs2-18.2.1-0.el9.x86_64.rpm 3.5 MB/s | 658 kB 00:00 2024-03-21T09:52:33.446 INFO:teuthology.orchestra.run.smithi153.stdout:(5/30): libradosstriper1-18.2.1-0.el9.x86_64.rp 5.6 MB/s | 474 kB 00:00 2024-03-21T09:52:33.713 INFO:teuthology.orchestra.run.smithi153.stdout:(6/30): ceph-common-18.2.1-0.el9.x86_64.rpm 16 MB/s | 18 MB 00:01 2024-03-21T09:52:33.738 INFO:teuthology.orchestra.run.smithi153.stdout:(7/30): python3-ceph-argparse-18.2.1-0.el9.x86_ 154 kB/s | 45 kB 00:00 2024-03-21T09:52:33.772 INFO:teuthology.orchestra.run.smithi153.stdout:(8/30): python3-ceph-common-18.2.1-0.el9.x86_64 2.1 MB/s | 124 kB 00:00 2024-03-21T09:52:33.831 INFO:teuthology.orchestra.run.smithi153.stdout:(9/30): python3-cephfs-18.2.1-0.el9.x86_64.rpm 1.7 MB/s | 161 kB 00:00 2024-03-21T09:52:33.873 INFO:teuthology.orchestra.run.smithi153.stdout:(10/30): python3-rados-18.2.1-0.el9.x86_64.rpm 3.2 MB/s | 321 kB 00:00 2024-03-21T09:52:33.906 INFO:teuthology.orchestra.run.smithi153.stdout:(11/30): python3-rbd-18.2.1-0.el9.x86_64.rpm 3.9 MB/s | 297 kB 00:00 2024-03-21T09:52:33.940 INFO:teuthology.orchestra.run.smithi153.stdout:(12/30): python3-rgw-18.2.1-0.el9.x86_64.rpm 1.5 MB/s | 99 kB 00:00 2024-03-21T09:52:34.099 INFO:teuthology.orchestra.run.smithi153.stdout:(13/30): librgw2-18.2.1-0.el9.x86_64.rpm 6.4 MB/s | 4.5 MB 00:00 2024-03-21T09:52:34.249 INFO:teuthology.orchestra.run.smithi153.stdout:(14/30): boost-program-options-1.75.0-8.el9.x86 311 kB/s | 106 kB 00:00 2024-03-21T09:52:34.266 INFO:teuthology.orchestra.run.smithi153.stdout:(15/30): librabbitmq-0.11.0-7.el9.x86_64.rpm 419 kB/s | 44 kB 00:00 2024-03-21T09:52:34.366 INFO:teuthology.orchestra.run.smithi153.stdout:(16/30): libpmemobj-1.12.1-1.el9.x86_64.rpm 445 kB/s | 159 kB 00:00 2024-03-21T09:52:34.408 INFO:teuthology.orchestra.run.smithi153.stdout:(17/30): librdkafka-1.6.1-102.el9.x86_64.rpm 4.1 MB/s | 661 kB 00:00 2024-03-21T09:52:34.441 INFO:teuthology.orchestra.run.smithi153.stdout:(18/30): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.6 MB/s | 282 kB 00:00 2024-03-21T09:52:34.467 INFO:teuthology.orchestra.run.smithi153.stdout:(19/30): python3-prettytable-0.7.2-27.el9.noarc 432 kB/s | 41 kB 00:00 2024-03-21T09:52:35.126 INFO:teuthology.orchestra.run.smithi153.stdout:(20/30): fmt-8.1.1-5.el9.x86_64.rpm 155 kB/s | 111 kB 00:00 2024-03-21T09:52:35.267 INFO:teuthology.orchestra.run.smithi153.stdout:(21/30): libarrow-doc-9.0.0-10.el9.noarch.rpm 184 kB/s | 26 kB 00:00 2024-03-21T09:52:35.342 INFO:teuthology.orchestra.run.smithi153.stdout:(22/30): gperftools-libs-2.9.1-2.el9.x86_64.rpm 353 kB/s | 309 kB 00:00 2024-03-21T09:52:35.418 INFO:teuthology.orchestra.run.smithi153.stdout:(23/30): liboath-2.6.7-2.el9.x86_64.rpm 325 kB/s | 49 kB 00:00 2024-03-21T09:52:35.485 INFO:teuthology.orchestra.run.smithi153.stdout:(24/30): libunwind-1.6.2-1.el9.x86_64.rpm 476 kB/s | 67 kB 00:00 2024-03-21T09:52:35.635 INFO:teuthology.orchestra.run.smithi153.stdout:(25/30): re2-20211101-3.el9.x86_64.rpm 1.3 MB/s | 192 kB 00:00 2024-03-21T09:52:35.819 INFO:teuthology.orchestra.run.smithi153.stdout:(26/30): libarrow-9.0.0-10.el9.x86_64.rpm 3.3 MB/s | 4.4 MB 00:01 2024-03-21T09:52:35.869 INFO:teuthology.orchestra.run.smithi153.stdout:(27/30): parquet-libs-9.0.0-10.el9.x86_64.rpm 1.8 MB/s | 838 kB 00:00 2024-03-21T09:52:36.011 INFO:teuthology.orchestra.run.smithi153.stdout:(28/30): librados2-18.2.1-0.el9.x86_64.rpm 17 MB/s | 3.3 MB 00:00 2024-03-21T09:52:36.069 INFO:teuthology.orchestra.run.smithi153.stdout:(29/30): thrift-0.15.0-2.el9.x86_64.rpm 3.6 MB/s | 1.6 MB 00:00 2024-03-21T09:52:36.145 INFO:teuthology.orchestra.run.smithi153.stdout:(30/30): librbd1-18.2.1-0.el9.x86_64.rpm 11 MB/s | 3.0 MB 00:00 2024-03-21T09:52:36.160 INFO:teuthology.orchestra.run.smithi153.stdout:-------------------------------------------------------------------------------- 2024-03-21T09:52:36.160 INFO:teuthology.orchestra.run.smithi153.stdout:Total 14 MB/s | 53 MB 00:03 2024-03-21T09:52:36.161 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction check 2024-03-21T09:52:36.233 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction check succeeded. 2024-03-21T09:52:36.233 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction test 2024-03-21T09:52:36.745 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction test succeeded. 2024-03-21T09:52:36.746 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction 2024-03-21T09:52:38.214 INFO:teuthology.orchestra.run.smithi153.stdout: Preparing : 1/1 2024-03-21T09:52:38.321 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/32 2024-03-21T09:52:38.373 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/32 2024-03-21T09:52:38.469 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/32 2024-03-21T09:52:38.844 INFO:teuthology.orchestra.run.smithi174.stdout:lab-extras 13 kB/s | 1.7 kB 00:00 2024-03-21T09:52:38.868 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/32 2024-03-21T09:52:38.907 INFO:teuthology.orchestra.run.smithi153.stdout: Upgrading : librados2-2:18.2.1-0.el9.x86_64 5/32 2024-03-21T09:52:39.021 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: librados2-2:18.2.1-0.el9.x86_64 5/32 2024-03-21T09:52:39.119 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-rados-2:18.2.1-0.el9.x86_64 6/32 2024-03-21T09:52:39.142 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libcephfs2-2:18.2.1-0.el9.x86_64 7/32 2024-03-21T09:52:39.225 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: libcephfs2-2:18.2.1-0.el9.x86_64 7/32 2024-03-21T09:52:39.267 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : liboath-2.6.7-2.el9.x86_64 8/32 2024-03-21T09:52:39.335 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-ceph-argparse-2:18.2.1-0.el9.x86_64 9/32 2024-03-21T09:52:39.433 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-cephfs-2:18.2.1-0.el9.x86_64 10/32 2024-03-21T09:52:39.461 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libradosstriper1-2:18.2.1-0.el9.x86_64 11/32 2024-03-21T09:52:39.553 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: libradosstriper1-2:18.2.1-0.el9.x86_64 11/32 2024-03-21T09:52:39.734 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : re2-1:20211101-3.el9.x86_64 12/32 2024-03-21T09:52:39.772 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 13/32 2024-03-21T09:52:39.832 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/32 2024-03-21T09:52:39.870 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/32 2024-03-21T09:52:39.911 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 16/32 2024-03-21T09:52:39.960 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : fmt-8.1.1-5.el9.x86_64 17/32 2024-03-21T09:52:40.009 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 18/32 2024-03-21T09:52:40.404 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 19/32 2024-03-21T09:52:40.450 INFO:teuthology.orchestra.run.smithi153.stdout: Upgrading : librbd1-2:18.2.1-0.el9.x86_64 20/32 2024-03-21T09:52:40.589 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: librbd1-2:18.2.1-0.el9.x86_64 20/32 2024-03-21T09:52:40.624 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-rbd-2:18.2.1-0.el9.x86_64 21/32 2024-03-21T09:52:40.686 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 22/32 2024-03-21T09:52:41.175 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 23/32 2024-03-21T09:52:41.203 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : librgw2-2:18.2.1-0.el9.x86_64 24/32 2024-03-21T09:52:41.295 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: librgw2-2:18.2.1-0.el9.x86_64 24/32 2024-03-21T09:52:41.394 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-rgw-2:18.2.1-0.el9.x86_64 25/32 2024-03-21T09:52:41.422 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-ceph-common-2:18.2.1-0.el9.x86_64 26/32 2024-03-21T09:52:41.484 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2024-03-21T09:52:41.495 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2024-03-21T09:52:41.496 INFO:teuthology.orchestra.run.smithi174.stdout: Package Arch Version Repository Size 2024-03-21T09:52:41.496 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2024-03-21T09:52:41.496 INFO:teuthology.orchestra.run.smithi174.stdout:Installing: 2024-03-21T09:52:41.496 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-radosgw x86_64 2:18.2.1-0.el9 ceph 7.7 M 2024-03-21T09:52:41.496 INFO:teuthology.orchestra.run.smithi174.stdout:Upgrading: 2024-03-21T09:52:41.496 INFO:teuthology.orchestra.run.smithi174.stdout: librados2 x86_64 2:18.2.1-0.el9 ceph 3.3 M 2024-03-21T09:52:41.496 INFO:teuthology.orchestra.run.smithi174.stdout: librbd1 x86_64 2:18.2.1-0.el9 ceph 3.0 M 2024-03-21T09:52:41.496 INFO:teuthology.orchestra.run.smithi174.stdout:Installing dependencies: 2024-03-21T09:52:41.496 INFO:teuthology.orchestra.run.smithi174.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-21T09:52:41.496 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-base x86_64 2:18.2.1-0.el9 ceph 5.2 M 2024-03-21T09:52:41.496 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-common x86_64 2:18.2.1-0.el9 ceph 18 M 2024-03-21T09:52:41.497 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-selinux x86_64 2:18.2.1-0.el9 ceph 24 k 2024-03-21T09:52:41.497 INFO:teuthology.orchestra.run.smithi174.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2024-03-21T09:52:41.497 INFO:teuthology.orchestra.run.smithi174.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-21T09:52:41.497 INFO:teuthology.orchestra.run.smithi174.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-21T09:52:41.497 INFO:teuthology.orchestra.run.smithi174.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-21T09:52:41.497 INFO:teuthology.orchestra.run.smithi174.stdout: libcephfs2 x86_64 2:18.2.1-0.el9 ceph 658 k 2024-03-21T09:52:41.497 INFO:teuthology.orchestra.run.smithi174.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-21T09:52:41.497 INFO:teuthology.orchestra.run.smithi174.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-21T09:52:41.497 INFO:teuthology.orchestra.run.smithi174.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-21T09:52:41.497 INFO:teuthology.orchestra.run.smithi174.stdout: libradosstriper1 x86_64 2:18.2.1-0.el9 ceph 474 k 2024-03-21T09:52:41.497 INFO:teuthology.orchestra.run.smithi174.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-21T09:52:41.498 INFO:teuthology.orchestra.run.smithi174.stdout: librgw2 x86_64 2:18.2.1-0.el9 ceph 4.5 M 2024-03-21T09:52:41.498 INFO:teuthology.orchestra.run.smithi174.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-21T09:52:41.498 INFO:teuthology.orchestra.run.smithi174.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-21T09:52:41.498 INFO:teuthology.orchestra.run.smithi174.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-21T09:52:41.498 INFO:teuthology.orchestra.run.smithi174.stdout: python3-ceph-argparse x86_64 2:18.2.1-0.el9 ceph 45 k 2024-03-21T09:52:41.498 INFO:teuthology.orchestra.run.smithi174.stdout: python3-ceph-common x86_64 2:18.2.1-0.el9 ceph 124 k 2024-03-21T09:52:41.498 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cephfs x86_64 2:18.2.1-0.el9 ceph 161 k 2024-03-21T09:52:41.498 INFO:teuthology.orchestra.run.smithi174.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-21T09:52:41.498 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rados x86_64 2:18.2.1-0.el9 ceph 321 k 2024-03-21T09:52:41.498 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rbd x86_64 2:18.2.1-0.el9 ceph 297 k 2024-03-21T09:52:41.498 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rgw x86_64 2:18.2.1-0.el9 ceph 99 k 2024-03-21T09:52:41.498 INFO:teuthology.orchestra.run.smithi174.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-21T09:52:41.498 INFO:teuthology.orchestra.run.smithi174.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-21T09:52:41.499 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-21T09:52:41.499 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction Summary 2024-03-21T09:52:41.499 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2024-03-21T09:52:41.499 INFO:teuthology.orchestra.run.smithi174.stdout:Install 28 Packages 2024-03-21T09:52:41.499 INFO:teuthology.orchestra.run.smithi174.stdout:Upgrade 2 Packages 2024-03-21T09:52:41.499 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-21T09:52:41.499 INFO:teuthology.orchestra.run.smithi174.stdout:Total download size: 53 M 2024-03-21T09:52:41.499 INFO:teuthology.orchestra.run.smithi174.stdout:Downloading Packages: 2024-03-21T09:52:41.870 INFO:teuthology.orchestra.run.smithi174.stdout:(1/30): ceph-base-18.2.1-0.el9.x86_64.rpm 19 MB/s | 5.2 MB 00:00 2024-03-21T09:52:41.895 INFO:teuthology.orchestra.run.smithi174.stdout:(2/30): ceph-selinux-18.2.1-0.el9.x86_64.rpm 972 kB/s | 24 kB 00:00 2024-03-21T09:52:41.945 INFO:teuthology.orchestra.run.smithi174.stdout:(3/30): libcephfs2-18.2.1-0.el9.x86_64.rpm 13 MB/s | 658 kB 00:00 2024-03-21T09:52:41.987 INFO:teuthology.orchestra.run.smithi174.stdout:(4/30): libradosstriper1-18.2.1-0.el9.x86_64.rp 11 MB/s | 474 kB 00:00 2024-03-21T09:52:42.146 INFO:teuthology.orchestra.run.smithi174.stdout:(5/30): ceph-radosgw-18.2.1-0.el9.x86_64.rpm 14 MB/s | 7.7 MB 00:00 2024-03-21T09:52:42.196 INFO:teuthology.orchestra.run.smithi174.stdout:(6/30): python3-ceph-argparse-18.2.1-0.el9.x86_ 920 kB/s | 45 kB 00:00 2024-03-21T09:52:42.230 INFO:teuthology.orchestra.run.smithi174.stdout:(7/30): python3-ceph-common-18.2.1-0.el9.x86_64 3.8 MB/s | 124 kB 00:00 2024-03-21T09:52:42.263 INFO:teuthology.orchestra.run.smithi174.stdout:(8/30): python3-cephfs-18.2.1-0.el9.x86_64.rpm 4.9 MB/s | 161 kB 00:00 2024-03-21T09:52:42.305 INFO:teuthology.orchestra.run.smithi174.stdout:(9/30): python3-rados-18.2.1-0.el9.x86_64.rpm 7.7 MB/s | 321 kB 00:00 2024-03-21T09:52:42.347 INFO:teuthology.orchestra.run.smithi174.stdout:(10/30): python3-rbd-18.2.1-0.el9.x86_64.rpm 7.1 MB/s | 297 kB 00:00 2024-03-21T09:52:42.381 INFO:teuthology.orchestra.run.smithi174.stdout:(11/30): python3-rgw-18.2.1-0.el9.x86_64.rpm 3.0 MB/s | 99 kB 00:00 2024-03-21T09:52:42.531 INFO:teuthology.orchestra.run.smithi174.stdout:(12/30): librgw2-18.2.1-0.el9.x86_64.rpm 8.2 MB/s | 4.5 MB 00:00 2024-03-21T09:52:42.590 INFO:teuthology.orchestra.run.smithi174.stdout:(13/30): boost-program-options-1.75.0-8.el9.x86 510 kB/s | 106 kB 00:00 2024-03-21T09:52:42.615 INFO:teuthology.orchestra.run.smithi174.stdout:(14/30): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.9 MB/s | 159 kB 00:00 2024-03-21T09:52:42.640 INFO:teuthology.orchestra.run.smithi174.stdout:(15/30): librabbitmq-0.11.0-7.el9.x86_64.rpm 911 kB/s | 44 kB 00:00 2024-03-21T09:52:42.682 INFO:teuthology.orchestra.run.smithi174.stdout:(16/30): librdkafka-1.6.1-102.el9.x86_64.rpm 9.8 MB/s | 661 kB 00:00 2024-03-21T09:52:42.716 INFO:teuthology.orchestra.run.smithi174.stdout:(17/30): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.9 MB/s | 282 kB 00:00 2024-03-21T09:52:42.741 INFO:teuthology.orchestra.run.smithi174.stdout:(18/30): python3-prettytable-0.7.2-27.el9.noarc 739 kB/s | 41 kB 00:00 2024-03-21T09:52:43.084 INFO:teuthology.orchestra.run.smithi174.stdout:(19/30): ceph-common-18.2.1-0.el9.x86_64.rpm 12 MB/s | 18 MB 00:01 2024-03-21T09:52:43.134 INFO:teuthology.orchestra.run.smithi174.stdout:(20/30): fmt-8.1.1-5.el9.x86_64.rpm 266 kB/s | 111 kB 00:00 2024-03-21T09:52:43.217 INFO:teuthology.orchestra.run.smithi174.stdout:(21/30): libarrow-doc-9.0.0-10.el9.noarch.rpm 314 kB/s | 26 kB 00:00 2024-03-21T09:52:43.251 INFO:teuthology.orchestra.run.smithi174.stdout:(22/30): gperftools-libs-2.9.1-2.el9.x86_64.rpm 608 kB/s | 309 kB 00:00 2024-03-21T09:52:43.276 INFO:teuthology.orchestra.run.smithi174.stdout:(23/30): liboath-2.6.7-2.el9.x86_64.rpm 840 kB/s | 49 kB 00:00 2024-03-21T09:52:43.301 INFO:teuthology.orchestra.run.smithi174.stdout:(24/30): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-03-21T09:52:43.435 INFO:teuthology.orchestra.run.smithi174.stdout:(25/30): re2-20211101-3.el9.x86_64.rpm 1.4 MB/s | 192 kB 00:00 2024-03-21T09:52:43.685 INFO:teuthology.orchestra.run.smithi174.stdout:(26/30): libarrow-9.0.0-10.el9.x86_64.rpm 7.4 MB/s | 4.4 MB 00:00 2024-03-21T09:52:43.727 INFO:teuthology.orchestra.run.smithi174.stdout:(27/30): parquet-libs-9.0.0-10.el9.x86_64.rpm 1.8 MB/s | 838 kB 00:00 2024-03-21T09:52:43.852 INFO:teuthology.orchestra.run.smithi174.stdout:(28/30): librbd1-18.2.1-0.el9.x86_64.rpm 24 MB/s | 3.0 MB 00:00 2024-03-21T09:52:43.884 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-common-2:18.2.1-0.el9.x86_64 27/32 2024-03-21T09:52:43.919 INFO:teuthology.orchestra.run.smithi174.stdout:(29/30): librados2-18.2.1-0.el9.x86_64.rpm 14 MB/s | 3.3 MB 00:00 2024-03-21T09:52:43.944 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-common-2:18.2.1-0.el9.x86_64 27/32 2024-03-21T09:52:44.012 INFO:teuthology.orchestra.run.smithi174.stdout:(30/30): thrift-0.15.0-2.el9.x86_64.rpm 2.7 MB/s | 1.6 MB 00:00 2024-03-21T09:52:44.018 INFO:teuthology.orchestra.run.smithi174.stdout:-------------------------------------------------------------------------------- 2024-03-21T09:52:44.019 INFO:teuthology.orchestra.run.smithi174.stdout:Total 21 MB/s | 53 MB 00:02 2024-03-21T09:52:44.020 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction check 2024-03-21T09:52:44.095 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction check succeeded. 2024-03-21T09:52:44.096 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction test 2024-03-21T09:52:44.570 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-common-2:18.2.1-0.el9.x86_64 27/32 2024-03-21T09:52:44.612 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction test succeeded. 2024-03-21T09:52:44.612 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction 2024-03-21T09:52:44.614 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-base-2:18.2.1-0.el9.x86_64 28/32 2024-03-21T09:52:44.763 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-base-2:18.2.1-0.el9.x86_64 28/32 2024-03-21T09:52:44.763 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-21T09:52:44.763 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-21T09:52:44.763 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T09:52:44.813 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-selinux-2:18.2.1-0.el9.x86_64 29/32 2024-03-21T09:52:46.098 INFO:teuthology.orchestra.run.smithi174.stdout: Preparing : 1/1 2024-03-21T09:52:46.199 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/32 2024-03-21T09:52:46.235 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/32 2024-03-21T09:52:46.369 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/32 2024-03-21T09:52:46.768 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/32 2024-03-21T09:52:46.840 INFO:teuthology.orchestra.run.smithi174.stdout: Upgrading : librados2-2:18.2.1-0.el9.x86_64 5/32 2024-03-21T09:52:46.958 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: librados2-2:18.2.1-0.el9.x86_64 5/32 2024-03-21T09:52:47.062 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-rados-2:18.2.1-0.el9.x86_64 6/32 2024-03-21T09:52:47.091 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libcephfs2-2:18.2.1-0.el9.x86_64 7/32 2024-03-21T09:52:47.175 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: libcephfs2-2:18.2.1-0.el9.x86_64 7/32 2024-03-21T09:52:47.217 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : liboath-2.6.7-2.el9.x86_64 8/32 2024-03-21T09:52:47.286 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-ceph-argparse-2:18.2.1-0.el9.x86_64 9/32 2024-03-21T09:52:47.404 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-cephfs-2:18.2.1-0.el9.x86_64 10/32 2024-03-21T09:52:47.435 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libradosstriper1-2:18.2.1-0.el9.x86_64 11/32 2024-03-21T09:52:47.528 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: libradosstriper1-2:18.2.1-0.el9.x86_64 11/32 2024-03-21T09:52:47.712 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : re2-1:20211101-3.el9.x86_64 12/32 2024-03-21T09:52:47.745 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 13/32 2024-03-21T09:52:47.792 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/32 2024-03-21T09:52:47.827 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/32 2024-03-21T09:52:47.867 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 16/32 2024-03-21T09:52:47.916 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : fmt-8.1.1-5.el9.x86_64 17/32 2024-03-21T09:52:47.956 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 18/32 2024-03-21T09:52:48.339 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 19/32 2024-03-21T09:52:48.374 INFO:teuthology.orchestra.run.smithi174.stdout: Upgrading : librbd1-2:18.2.1-0.el9.x86_64 20/32 2024-03-21T09:52:48.488 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: librbd1-2:18.2.1-0.el9.x86_64 20/32 2024-03-21T09:52:48.523 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-rbd-2:18.2.1-0.el9.x86_64 21/32 2024-03-21T09:52:48.588 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 22/32 2024-03-21T09:52:49.076 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 23/32 2024-03-21T09:52:49.110 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : librgw2-2:18.2.1-0.el9.x86_64 24/32 2024-03-21T09:52:49.202 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: librgw2-2:18.2.1-0.el9.x86_64 24/32 2024-03-21T09:52:49.303 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-rgw-2:18.2.1-0.el9.x86_64 25/32 2024-03-21T09:52:49.345 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-ceph-common-2:18.2.1-0.el9.x86_64 26/32 2024-03-21T09:52:51.794 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-common-2:18.2.1-0.el9.x86_64 27/32 2024-03-21T09:52:51.834 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-common-2:18.2.1-0.el9.x86_64 27/32 2024-03-21T09:52:52.466 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-common-2:18.2.1-0.el9.x86_64 27/32 2024-03-21T09:52:52.520 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-base-2:18.2.1-0.el9.x86_64 28/32 2024-03-21T09:52:52.698 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-base-2:18.2.1-0.el9.x86_64 28/32 2024-03-21T09:52:52.698 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-21T09:52:52.698 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-21T09:52:52.698 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-21T09:52:52.729 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-selinux-2:18.2.1-0.el9.x86_64 29/32 2024-03-21T09:52:55.603 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-selinux-2:18.2.1-0.el9.x86_64 29/32 2024-03-21T09:52:55.603 INFO:teuthology.orchestra.run.smithi153.stdout:skipping the directory /sys 2024-03-21T09:52:55.604 INFO:teuthology.orchestra.run.smithi153.stdout:skipping the directory /proc 2024-03-21T09:52:55.604 INFO:teuthology.orchestra.run.smithi153.stdout:skipping the directory /mnt 2024-03-21T09:52:55.604 INFO:teuthology.orchestra.run.smithi153.stdout:skipping the directory /var/tmp 2024-03-21T09:52:55.604 INFO:teuthology.orchestra.run.smithi153.stdout:skipping the directory /home 2024-03-21T09:52:55.604 INFO:teuthology.orchestra.run.smithi153.stdout:skipping the directory /root 2024-03-21T09:52:55.604 INFO:teuthology.orchestra.run.smithi153.stdout:skipping the directory /tmp 2024-03-21T09:52:55.604 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T09:52:56.444 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-radosgw-2:18.2.1-0.el9.x86_64 30/32 2024-03-21T09:52:56.540 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-radosgw-2:18.2.1-0.el9.x86_64 30/32 2024-03-21T09:52:56.540 INFO:teuthology.orchestra.run.smithi153.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T09:52:56.541 INFO:teuthology.orchestra.run.smithi153.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-21T09:52:56.541 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-21T09:52:56.541 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-21T09:52:56.541 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T09:52:56.545 INFO:teuthology.orchestra.run.smithi153.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-03-21T09:52:56.665 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-03-21T09:52:56.665 INFO:teuthology.orchestra.run.smithi153.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 32/32 2024-03-21T09:52:58.124 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 32/32 2024-03-21T09:52:58.124 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-base-2:18.2.1-0.el9.x86_64 1/32 2024-03-21T09:52:58.124 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-common-2:18.2.1-0.el9.x86_64 2/32 2024-03-21T09:52:58.124 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-radosgw-2:18.2.1-0.el9.x86_64 3/32 2024-03-21T09:52:58.125 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-selinux-2:18.2.1-0.el9.x86_64 4/32 2024-03-21T09:52:58.125 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libcephfs2-2:18.2.1-0.el9.x86_64 5/32 2024-03-21T09:52:58.125 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libradosstriper1-2:18.2.1-0.el9.x86_64 6/32 2024-03-21T09:52:58.125 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : librgw2-2:18.2.1-0.el9.x86_64 7/32 2024-03-21T09:52:58.125 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-ceph-argparse-2:18.2.1-0.el9.x86_64 8/32 2024-03-21T09:52:58.125 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-ceph-common-2:18.2.1-0.el9.x86_64 9/32 2024-03-21T09:52:58.125 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-cephfs-2:18.2.1-0.el9.x86_64 10/32 2024-03-21T09:52:58.125 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-rados-2:18.2.1-0.el9.x86_64 11/32 2024-03-21T09:52:58.125 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-rbd-2:18.2.1-0.el9.x86_64 12/32 2024-03-21T09:52:58.125 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-rgw-2:18.2.1-0.el9.x86_64 13/32 2024-03-21T09:52:58.125 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/32 2024-03-21T09:52:58.125 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/32 2024-03-21T09:52:58.125 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/32 2024-03-21T09:52:58.125 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/32 2024-03-21T09:52:58.126 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/32 2024-03-21T09:52:58.126 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/32 2024-03-21T09:52:58.126 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 20/32 2024-03-21T09:52:58.126 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 21/32 2024-03-21T09:52:58.126 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 22/32 2024-03-21T09:52:58.126 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 23/32 2024-03-21T09:52:58.126 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 24/32 2024-03-21T09:52:58.126 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 25/32 2024-03-21T09:52:58.126 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 26/32 2024-03-21T09:52:58.126 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : re2-1:20211101-3.el9.x86_64 27/32 2024-03-21T09:52:58.127 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 28/32 2024-03-21T09:52:58.127 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : librados2-2:18.2.1-0.el9.x86_64 29/32 2024-03-21T09:52:58.127 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 30/32 2024-03-21T09:52:58.127 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : librbd1-2:18.2.1-0.el9.x86_64 31/32 2024-03-21T09:52:59.044 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 32/32 2024-03-21T09:52:59.045 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T09:52:59.045 INFO:teuthology.orchestra.run.smithi153.stdout:Upgraded: 2024-03-21T09:52:59.045 INFO:teuthology.orchestra.run.smithi153.stdout: librados2-2:18.2.1-0.el9.x86_64 librbd1-2:18.2.1-0.el9.x86_64 2024-03-21T09:52:59.045 INFO:teuthology.orchestra.run.smithi153.stdout:Installed: 2024-03-21T09:52:59.045 INFO:teuthology.orchestra.run.smithi153.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-21T09:52:59.046 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-base-2:18.2.1-0.el9.x86_64 2024-03-21T09:52:59.046 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-common-2:18.2.1-0.el9.x86_64 2024-03-21T09:52:59.046 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-radosgw-2:18.2.1-0.el9.x86_64 2024-03-21T09:52:59.046 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-selinux-2:18.2.1-0.el9.x86_64 2024-03-21T09:52:59.046 INFO:teuthology.orchestra.run.smithi153.stdout: fmt-8.1.1-5.el9.x86_64 2024-03-21T09:52:59.046 INFO:teuthology.orchestra.run.smithi153.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-21T09:52:59.047 INFO:teuthology.orchestra.run.smithi153.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-21T09:52:59.047 INFO:teuthology.orchestra.run.smithi153.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-21T09:52:59.047 INFO:teuthology.orchestra.run.smithi153.stdout: libcephfs2-2:18.2.1-0.el9.x86_64 2024-03-21T09:52:59.047 INFO:teuthology.orchestra.run.smithi153.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-21T09:52:59.047 INFO:teuthology.orchestra.run.smithi153.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-21T09:52:59.047 INFO:teuthology.orchestra.run.smithi153.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-21T09:52:59.047 INFO:teuthology.orchestra.run.smithi153.stdout: libradosstriper1-2:18.2.1-0.el9.x86_64 2024-03-21T09:52:59.047 INFO:teuthology.orchestra.run.smithi153.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-21T09:52:59.048 INFO:teuthology.orchestra.run.smithi153.stdout: librgw2-2:18.2.1-0.el9.x86_64 2024-03-21T09:52:59.048 INFO:teuthology.orchestra.run.smithi153.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-21T09:52:59.048 INFO:teuthology.orchestra.run.smithi153.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-21T09:52:59.048 INFO:teuthology.orchestra.run.smithi153.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-21T09:52:59.048 INFO:teuthology.orchestra.run.smithi153.stdout: python3-ceph-argparse-2:18.2.1-0.el9.x86_64 2024-03-21T09:52:59.048 INFO:teuthology.orchestra.run.smithi153.stdout: python3-ceph-common-2:18.2.1-0.el9.x86_64 2024-03-21T09:52:59.048 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cephfs-2:18.2.1-0.el9.x86_64 2024-03-21T09:52:59.048 INFO:teuthology.orchestra.run.smithi153.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-21T09:52:59.049 INFO:teuthology.orchestra.run.smithi153.stdout: python3-rados-2:18.2.1-0.el9.x86_64 2024-03-21T09:52:59.049 INFO:teuthology.orchestra.run.smithi153.stdout: python3-rbd-2:18.2.1-0.el9.x86_64 2024-03-21T09:52:59.049 INFO:teuthology.orchestra.run.smithi153.stdout: python3-rgw-2:18.2.1-0.el9.x86_64 2024-03-21T09:52:59.049 INFO:teuthology.orchestra.run.smithi153.stdout: re2-1:20211101-3.el9.x86_64 2024-03-21T09:52:59.049 INFO:teuthology.orchestra.run.smithi153.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-21T09:52:59.049 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T09:52:59.049 INFO:teuthology.orchestra.run.smithi153.stdout:Complete! 2024-03-21T09:52:59.495 DEBUG:teuthology.orchestra.run.smithi153:> sudo yum -y install ceph-test 2024-03-21T09:53:00.347 INFO:teuthology.orchestra.run.smithi153.stdout:Last metadata expiration check: 0:00:31 ago on Thu 21 Mar 2024 09:52:29 AM UTC. 2024-03-21T09:53:00.507 INFO:teuthology.orchestra.run.smithi153.stdout:Dependencies resolved. 2024-03-21T09:53:00.507 INFO:teuthology.orchestra.run.smithi153.stdout:================================================================================ 2024-03-21T09:53:00.508 INFO:teuthology.orchestra.run.smithi153.stdout: Package Arch Version Repository Size 2024-03-21T09:53:00.508 INFO:teuthology.orchestra.run.smithi153.stdout:================================================================================ 2024-03-21T09:53:00.508 INFO:teuthology.orchestra.run.smithi153.stdout:Installing: 2024-03-21T09:53:00.508 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-test x86_64 2:18.2.1-0.el9 ceph 40 M 2024-03-21T09:53:00.508 INFO:teuthology.orchestra.run.smithi153.stdout:Installing dependencies: 2024-03-21T09:53:00.508 INFO:teuthology.orchestra.run.smithi153.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-21T09:53:00.508 INFO:teuthology.orchestra.run.smithi153.stdout: libcephsqlite x86_64 2:18.2.1-0.el9 ceph 165 k 2024-03-21T09:53:00.508 INFO:teuthology.orchestra.run.smithi153.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-21T09:53:00.508 INFO:teuthology.orchestra.run.smithi153.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-21T09:53:00.508 INFO:teuthology.orchestra.run.smithi153.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-21T09:53:00.508 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T09:53:00.508 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction Summary 2024-03-21T09:53:00.508 INFO:teuthology.orchestra.run.smithi153.stdout:================================================================================ 2024-03-21T09:53:00.509 INFO:teuthology.orchestra.run.smithi153.stdout:Install 6 Packages 2024-03-21T09:53:00.509 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T09:53:00.509 INFO:teuthology.orchestra.run.smithi153.stdout:Total download size: 41 M 2024-03-21T09:53:00.509 INFO:teuthology.orchestra.run.smithi153.stdout:Installed size: 167 M 2024-03-21T09:53:00.509 INFO:teuthology.orchestra.run.smithi153.stdout:Downloading Packages: 2024-03-21T09:53:00.808 INFO:teuthology.orchestra.run.smithi153.stdout:(1/6): libcephsqlite-18.2.1-0.el9.x86_64.rpm 1.0 MB/s | 165 kB 00:00 2024-03-21T09:53:00.900 INFO:teuthology.orchestra.run.smithi153.stdout:(2/6): oniguruma-6.9.6-1.el9.5.x86_64.rpm 846 kB/s | 217 kB 00:00 2024-03-21T09:53:00.925 INFO:teuthology.orchestra.run.smithi153.stdout:(3/6): socat-1.7.4.1-5.el9.x86_64.rpm 2.5 MB/s | 300 kB 00:00 2024-03-21T09:53:00.950 INFO:teuthology.orchestra.run.smithi153.stdout:(4/6): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.3 MB/s | 63 kB 00:00 2024-03-21T09:53:02.117 INFO:teuthology.orchestra.run.smithi153.stdout:(5/6): ceph-test-18.2.1-0.el9.x86_64.rpm 27 MB/s | 40 MB 00:01 2024-03-21T09:53:02.142 INFO:teuthology.orchestra.run.smithi153.stdout:(6/6): jq-1.6-16.el9.x86_64.rpm 154 kB/s | 187 kB 00:01 2024-03-21T09:53:02.149 INFO:teuthology.orchestra.run.smithi153.stdout:-------------------------------------------------------------------------------- 2024-03-21T09:53:02.149 INFO:teuthology.orchestra.run.smithi153.stdout:Total 25 MB/s | 41 MB 00:01 2024-03-21T09:53:02.190 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction check 2024-03-21T09:53:02.244 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction check succeeded. 2024-03-21T09:53:02.244 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction test 2024-03-21T09:53:02.665 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction test succeeded. 2024-03-21T09:53:02.665 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction 2024-03-21T09:53:03.340 INFO:teuthology.orchestra.run.smithi153.stdout: Preparing : 1/1 2024-03-21T09:53:03.433 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/6 2024-03-21T09:53:03.481 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/6 2024-03-21T09:53:03.522 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 3/6 2024-03-21T09:53:03.593 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : jq-1.6-16.el9.x86_64 4/6 2024-03-21T09:53:03.621 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libcephsqlite-2:18.2.1-0.el9.x86_64 5/6 2024-03-21T09:53:03.669 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-selinux-2:18.2.1-0.el9.x86_64 29/32 2024-03-21T09:53:03.669 INFO:teuthology.orchestra.run.smithi174.stdout:skipping the directory /sys 2024-03-21T09:53:03.669 INFO:teuthology.orchestra.run.smithi174.stdout:skipping the directory /proc 2024-03-21T09:53:03.669 INFO:teuthology.orchestra.run.smithi174.stdout:skipping the directory /mnt 2024-03-21T09:53:03.669 INFO:teuthology.orchestra.run.smithi174.stdout:skipping the directory /var/tmp 2024-03-21T09:53:03.669 INFO:teuthology.orchestra.run.smithi174.stdout:skipping the directory /home 2024-03-21T09:53:03.669 INFO:teuthology.orchestra.run.smithi174.stdout:skipping the directory /root 2024-03-21T09:53:03.669 INFO:teuthology.orchestra.run.smithi174.stdout:skipping the directory /tmp 2024-03-21T09:53:03.670 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-21T09:53:04.512 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-radosgw-2:18.2.1-0.el9.x86_64 30/32 2024-03-21T09:53:04.605 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-radosgw-2:18.2.1-0.el9.x86_64 30/32 2024-03-21T09:53:04.606 INFO:teuthology.orchestra.run.smithi174.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T09:53:04.606 INFO:teuthology.orchestra.run.smithi174.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-21T09:53:04.606 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-21T09:53:04.606 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-21T09:53:04.606 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-21T09:53:04.610 INFO:teuthology.orchestra.run.smithi174.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-03-21T09:53:04.732 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-03-21T09:53:04.733 INFO:teuthology.orchestra.run.smithi174.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 32/32 2024-03-21T09:53:06.235 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 32/32 2024-03-21T09:53:06.236 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-base-2:18.2.1-0.el9.x86_64 1/32 2024-03-21T09:53:06.236 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-common-2:18.2.1-0.el9.x86_64 2/32 2024-03-21T09:53:06.236 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-radosgw-2:18.2.1-0.el9.x86_64 3/32 2024-03-21T09:53:06.236 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-selinux-2:18.2.1-0.el9.x86_64 4/32 2024-03-21T09:53:06.236 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libcephfs2-2:18.2.1-0.el9.x86_64 5/32 2024-03-21T09:53:06.236 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libradosstriper1-2:18.2.1-0.el9.x86_64 6/32 2024-03-21T09:53:06.236 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librgw2-2:18.2.1-0.el9.x86_64 7/32 2024-03-21T09:53:06.236 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-ceph-argparse-2:18.2.1-0.el9.x86_64 8/32 2024-03-21T09:53:06.236 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-ceph-common-2:18.2.1-0.el9.x86_64 9/32 2024-03-21T09:53:06.236 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-cephfs-2:18.2.1-0.el9.x86_64 10/32 2024-03-21T09:53:06.236 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-rados-2:18.2.1-0.el9.x86_64 11/32 2024-03-21T09:53:06.236 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-rbd-2:18.2.1-0.el9.x86_64 12/32 2024-03-21T09:53:06.236 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-rgw-2:18.2.1-0.el9.x86_64 13/32 2024-03-21T09:53:06.237 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/32 2024-03-21T09:53:06.237 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/32 2024-03-21T09:53:06.237 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/32 2024-03-21T09:53:06.237 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/32 2024-03-21T09:53:06.237 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/32 2024-03-21T09:53:06.237 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/32 2024-03-21T09:53:06.237 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 20/32 2024-03-21T09:53:06.237 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 21/32 2024-03-21T09:53:06.237 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 22/32 2024-03-21T09:53:06.237 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 23/32 2024-03-21T09:53:06.237 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 24/32 2024-03-21T09:53:06.237 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 25/32 2024-03-21T09:53:06.237 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 26/32 2024-03-21T09:53:06.237 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : re2-1:20211101-3.el9.x86_64 27/32 2024-03-21T09:53:06.238 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 28/32 2024-03-21T09:53:06.238 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librados2-2:18.2.1-0.el9.x86_64 29/32 2024-03-21T09:53:06.238 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 30/32 2024-03-21T09:53:06.238 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librbd1-2:18.2.1-0.el9.x86_64 31/32 2024-03-21T09:53:07.228 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 32/32 2024-03-21T09:53:07.228 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-21T09:53:07.228 INFO:teuthology.orchestra.run.smithi174.stdout:Upgraded: 2024-03-21T09:53:07.228 INFO:teuthology.orchestra.run.smithi174.stdout: librados2-2:18.2.1-0.el9.x86_64 librbd1-2:18.2.1-0.el9.x86_64 2024-03-21T09:53:07.228 INFO:teuthology.orchestra.run.smithi174.stdout:Installed: 2024-03-21T09:53:07.228 INFO:teuthology.orchestra.run.smithi174.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-21T09:53:07.229 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-base-2:18.2.1-0.el9.x86_64 2024-03-21T09:53:07.229 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-common-2:18.2.1-0.el9.x86_64 2024-03-21T09:53:07.229 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-radosgw-2:18.2.1-0.el9.x86_64 2024-03-21T09:53:07.229 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-selinux-2:18.2.1-0.el9.x86_64 2024-03-21T09:53:07.229 INFO:teuthology.orchestra.run.smithi174.stdout: fmt-8.1.1-5.el9.x86_64 2024-03-21T09:53:07.229 INFO:teuthology.orchestra.run.smithi174.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-21T09:53:07.229 INFO:teuthology.orchestra.run.smithi174.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-21T09:53:07.229 INFO:teuthology.orchestra.run.smithi174.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-21T09:53:07.229 INFO:teuthology.orchestra.run.smithi174.stdout: libcephfs2-2:18.2.1-0.el9.x86_64 2024-03-21T09:53:07.229 INFO:teuthology.orchestra.run.smithi174.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-21T09:53:07.229 INFO:teuthology.orchestra.run.smithi174.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-21T09:53:07.229 INFO:teuthology.orchestra.run.smithi174.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-21T09:53:07.229 INFO:teuthology.orchestra.run.smithi174.stdout: libradosstriper1-2:18.2.1-0.el9.x86_64 2024-03-21T09:53:07.229 INFO:teuthology.orchestra.run.smithi174.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-21T09:53:07.230 INFO:teuthology.orchestra.run.smithi174.stdout: librgw2-2:18.2.1-0.el9.x86_64 2024-03-21T09:53:07.230 INFO:teuthology.orchestra.run.smithi174.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-21T09:53:07.230 INFO:teuthology.orchestra.run.smithi174.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-21T09:53:07.230 INFO:teuthology.orchestra.run.smithi174.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-21T09:53:07.230 INFO:teuthology.orchestra.run.smithi174.stdout: python3-ceph-argparse-2:18.2.1-0.el9.x86_64 2024-03-21T09:53:07.230 INFO:teuthology.orchestra.run.smithi174.stdout: python3-ceph-common-2:18.2.1-0.el9.x86_64 2024-03-21T09:53:07.230 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cephfs-2:18.2.1-0.el9.x86_64 2024-03-21T09:53:07.230 INFO:teuthology.orchestra.run.smithi174.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-21T09:53:07.230 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rados-2:18.2.1-0.el9.x86_64 2024-03-21T09:53:07.230 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rbd-2:18.2.1-0.el9.x86_64 2024-03-21T09:53:07.230 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rgw-2:18.2.1-0.el9.x86_64 2024-03-21T09:53:07.230 INFO:teuthology.orchestra.run.smithi174.stdout: re2-1:20211101-3.el9.x86_64 2024-03-21T09:53:07.230 INFO:teuthology.orchestra.run.smithi174.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-21T09:53:07.230 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-21T09:53:07.231 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2024-03-21T09:53:07.681 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum -y install ceph-test 2024-03-21T09:53:07.779 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: libcephsqlite-2:18.2.1-0.el9.x86_64 5/6 2024-03-21T09:53:07.882 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-test-2:18.2.1-0.el9.x86_64 6/6 2024-03-21T09:53:08.513 INFO:teuthology.orchestra.run.smithi174.stdout:Last metadata expiration check: 0:00:30 ago on Thu 21 Mar 2024 09:52:38 AM UTC. 2024-03-21T09:53:08.675 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2024-03-21T09:53:08.676 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2024-03-21T09:53:08.676 INFO:teuthology.orchestra.run.smithi174.stdout: Package Arch Version Repository Size 2024-03-21T09:53:08.676 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2024-03-21T09:53:08.676 INFO:teuthology.orchestra.run.smithi174.stdout:Installing: 2024-03-21T09:53:08.677 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-test x86_64 2:18.2.1-0.el9 ceph 40 M 2024-03-21T09:53:08.677 INFO:teuthology.orchestra.run.smithi174.stdout:Installing dependencies: 2024-03-21T09:53:08.677 INFO:teuthology.orchestra.run.smithi174.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-21T09:53:08.677 INFO:teuthology.orchestra.run.smithi174.stdout: libcephsqlite x86_64 2:18.2.1-0.el9 ceph 165 k 2024-03-21T09:53:08.677 INFO:teuthology.orchestra.run.smithi174.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-21T09:53:08.677 INFO:teuthology.orchestra.run.smithi174.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-21T09:53:08.677 INFO:teuthology.orchestra.run.smithi174.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-21T09:53:08.677 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-21T09:53:08.677 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction Summary 2024-03-21T09:53:08.677 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2024-03-21T09:53:08.677 INFO:teuthology.orchestra.run.smithi174.stdout:Install 6 Packages 2024-03-21T09:53:08.678 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-21T09:53:08.678 INFO:teuthology.orchestra.run.smithi174.stdout:Total download size: 41 M 2024-03-21T09:53:08.678 INFO:teuthology.orchestra.run.smithi174.stdout:Installed size: 167 M 2024-03-21T09:53:08.678 INFO:teuthology.orchestra.run.smithi174.stdout:Downloading Packages: 2024-03-21T09:53:08.874 INFO:teuthology.orchestra.run.smithi174.stdout:(1/6): libcephsqlite-18.2.1-0.el9.x86_64.rpm 1.2 MB/s | 165 kB 00:00 2024-03-21T09:53:09.041 INFO:teuthology.orchestra.run.smithi174.stdout:(2/6): oniguruma-6.9.6-1.el9.5.x86_64.rpm 718 kB/s | 217 kB 00:00 2024-03-21T09:53:09.067 INFO:teuthology.orchestra.run.smithi174.stdout:(3/6): socat-1.7.4.1-5.el9.x86_64.rpm 1.5 MB/s | 300 kB 00:00 2024-03-21T09:53:09.083 INFO:teuthology.orchestra.run.smithi174.stdout:(4/6): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 63 kB 00:00 2024-03-21T09:53:09.551 INFO:teuthology.orchestra.run.smithi174.stdout:(5/6): ceph-test-18.2.1-0.el9.x86_64.rpm 49 MB/s | 40 MB 00:00 2024-03-21T09:53:09.576 INFO:teuthology.orchestra.run.smithi174.stdout:(6/6): jq-1.6-16.el9.x86_64.rpm 368 kB/s | 187 kB 00:00 2024-03-21T09:53:09.582 INFO:teuthology.orchestra.run.smithi174.stdout:-------------------------------------------------------------------------------- 2024-03-21T09:53:09.582 INFO:teuthology.orchestra.run.smithi174.stdout:Total 45 MB/s | 41 MB 00:00 2024-03-21T09:53:09.622 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction check 2024-03-21T09:53:09.675 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction check succeeded. 2024-03-21T09:53:09.675 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction test 2024-03-21T09:53:10.021 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-test-2:18.2.1-0.el9.x86_64 6/6 2024-03-21T09:53:10.022 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-test-2:18.2.1-0.el9.x86_64 1/6 2024-03-21T09:53:10.022 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libcephsqlite-2:18.2.1-0.el9.x86_64 2/6 2024-03-21T09:53:10.022 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 3/6 2024-03-21T09:53:10.022 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 4/6 2024-03-21T09:53:10.022 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 5/6 2024-03-21T09:53:10.062 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction test succeeded. 2024-03-21T09:53:10.062 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction 2024-03-21T09:53:10.384 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : jq-1.6-16.el9.x86_64 6/6 2024-03-21T09:53:10.384 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T09:53:10.384 INFO:teuthology.orchestra.run.smithi153.stdout:Installed: 2024-03-21T09:53:10.384 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-test-2:18.2.1-0.el9.x86_64 jq-1.6-16.el9.x86_64 2024-03-21T09:53:10.384 INFO:teuthology.orchestra.run.smithi153.stdout: libcephsqlite-2:18.2.1-0.el9.x86_64 oniguruma-6.9.6-1.el9.5.x86_64 2024-03-21T09:53:10.384 INFO:teuthology.orchestra.run.smithi153.stdout: socat-1.7.4.1-5.el9.x86_64 xmlstarlet-1.6.1-20.el9.x86_64 2024-03-21T09:53:10.385 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T09:53:10.385 INFO:teuthology.orchestra.run.smithi153.stdout:Complete! 2024-03-21T09:53:10.667 INFO:teuthology.orchestra.run.smithi174.stdout: Preparing : 1/1 2024-03-21T09:53:10.758 DEBUG:teuthology.orchestra.run.smithi153:> sudo yum -y install ceph 2024-03-21T09:53:10.759 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/6 2024-03-21T09:53:10.807 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/6 2024-03-21T09:53:10.846 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 3/6 2024-03-21T09:53:10.896 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : jq-1.6-16.el9.x86_64 4/6 2024-03-21T09:53:10.921 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libcephsqlite-2:18.2.1-0.el9.x86_64 5/6 2024-03-21T09:53:11.651 INFO:teuthology.orchestra.run.smithi153.stdout:Last metadata expiration check: 0:00:42 ago on Thu 21 Mar 2024 09:52:29 AM UTC. 2024-03-21T09:53:11.824 INFO:teuthology.orchestra.run.smithi153.stdout:Dependencies resolved. 2024-03-21T09:53:11.827 INFO:teuthology.orchestra.run.smithi153.stdout:================================================================================ 2024-03-21T09:53:11.827 INFO:teuthology.orchestra.run.smithi153.stdout: Package Arch Version Repository Size 2024-03-21T09:53:11.827 INFO:teuthology.orchestra.run.smithi153.stdout:================================================================================ 2024-03-21T09:53:11.827 INFO:teuthology.orchestra.run.smithi153.stdout:Installing: 2024-03-21T09:53:11.827 INFO:teuthology.orchestra.run.smithi153.stdout: ceph x86_64 2:18.2.1-0.el9 ceph 6.4 k 2024-03-21T09:53:11.827 INFO:teuthology.orchestra.run.smithi153.stdout:Installing dependencies: 2024-03-21T09:53:11.828 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mds x86_64 2:18.2.1-0.el9 ceph 2.1 M 2024-03-21T09:53:11.828 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr x86_64 2:18.2.1-0.el9 ceph 1.4 M 2024-03-21T09:53:11.828 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-modules-core noarch 2:18.2.1-0.el9 ceph-noarch 242 k 2024-03-21T09:53:11.828 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mon x86_64 2:18.2.1-0.el9 ceph 4.4 M 2024-03-21T09:53:11.828 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-osd x86_64 2:18.2.1-0.el9 ceph 18 M 2024-03-21T09:53:11.828 INFO:teuthology.orchestra.run.smithi153.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-21T09:53:11.828 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-21T09:53:11.828 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-21T09:53:11.828 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-21T09:53:11.828 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-21T09:53:11.828 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-21T09:53:11.828 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-21T09:53:11.828 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-21T09:53:11.829 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-21T09:53:11.829 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-21T09:53:11.829 INFO:teuthology.orchestra.run.smithi153.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-21T09:53:11.829 INFO:teuthology.orchestra.run.smithi153.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-21T09:53:11.829 INFO:teuthology.orchestra.run.smithi153.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-21T09:53:11.829 INFO:teuthology.orchestra.run.smithi153.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-21T09:53:11.830 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-21T09:53:11.830 INFO:teuthology.orchestra.run.smithi153.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-21T09:53:11.830 INFO:teuthology.orchestra.run.smithi153.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-21T09:53:11.830 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-21T09:53:11.830 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-21T09:53:11.830 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-21T09:53:11.830 INFO:teuthology.orchestra.run.smithi153.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-21T09:53:11.830 INFO:teuthology.orchestra.run.smithi153.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-21T09:53:11.830 INFO:teuthology.orchestra.run.smithi153.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-21T09:53:11.830 INFO:teuthology.orchestra.run.smithi153.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-21T09:53:11.830 INFO:teuthology.orchestra.run.smithi153.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-21T09:53:11.830 INFO:teuthology.orchestra.run.smithi153.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-21T09:53:11.830 INFO:teuthology.orchestra.run.smithi153.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-21T09:53:11.830 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T09:53:11.831 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction Summary 2024-03-21T09:53:11.831 INFO:teuthology.orchestra.run.smithi153.stdout:================================================================================ 2024-03-21T09:53:11.831 INFO:teuthology.orchestra.run.smithi153.stdout:Install 33 Packages 2024-03-21T09:53:11.831 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T09:53:11.831 INFO:teuthology.orchestra.run.smithi153.stdout:Total download size: 30 M 2024-03-21T09:53:11.831 INFO:teuthology.orchestra.run.smithi153.stdout:Installed size: 110 M 2024-03-21T09:53:11.831 INFO:teuthology.orchestra.run.smithi153.stdout:Downloading Packages: 2024-03-21T09:53:12.495 INFO:teuthology.orchestra.run.smithi153.stdout:(1/33): ceph-mds-18.2.1-0.el9.x86_64.rpm 9.8 MB/s | 2.1 MB 00:00 2024-03-21T09:53:12.520 INFO:teuthology.orchestra.run.smithi153.stdout:(2/33): ceph-18.2.1-0.el9.x86_64.rpm 27 kB/s | 6.4 kB 00:00 2024-03-21T09:53:12.595 INFO:teuthology.orchestra.run.smithi153.stdout:(3/33): ceph-mgr-18.2.1-0.el9.x86_64.rpm 4.6 MB/s | 1.4 MB 00:00 2024-03-21T09:53:12.737 INFO:teuthology.orchestra.run.smithi153.stdout:(4/33): ceph-mgr-modules-core-18.2.1-0.el9.noar 1.7 MB/s | 242 kB 00:00 2024-03-21T09:53:12.963 INFO:teuthology.orchestra.run.smithi153.stdout:(5/33): ceph-mon-18.2.1-0.el9.x86_64.rpm 9.4 MB/s | 4.4 MB 00:00 2024-03-21T09:53:12.988 INFO:teuthology.orchestra.run.smithi153.stdout:(6/33): python3-mako-1.1.4-6.el9.noarch.rpm 579 kB/s | 144 kB 00:00 2024-03-21T09:53:13.005 INFO:teuthology.orchestra.run.smithi153.stdout:(7/33): python3-markupsafe-1.1.1-12.el9.x86_64. 779 kB/s | 32 kB 00:00 2024-03-21T09:53:13.030 INFO:teuthology.orchestra.run.smithi153.stdout:(8/33): python3-pytz-2021.1-5.el9.noarch.rpm 1.1 MB/s | 47 kB 00:00 2024-03-21T09:53:13.047 INFO:teuthology.orchestra.run.smithi153.stdout:(9/33): python3-toml-0.10.2-6.el9.noarch.rpm 949 kB/s | 39 kB 00:00 2024-03-21T09:53:13.097 INFO:teuthology.orchestra.run.smithi153.stdout:(10/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.6 MB/s | 241 kB 00:00 2024-03-21T09:53:13.164 INFO:teuthology.orchestra.run.smithi153.stdout:(11/33): python3-cryptography-36.0.1-4.el9.x86_ 9.8 MB/s | 1.1 MB 00:00 2024-03-21T09:53:13.189 INFO:teuthology.orchestra.run.smithi153.stdout:(12/33): python3-ply-3.11-14.el9.noarch.rpm 1.2 MB/s | 103 kB 00:00 2024-03-21T09:53:13.231 INFO:teuthology.orchestra.run.smithi153.stdout:(13/33): python3-pycparser-2.20-6.el9.noarch.rp 1.9 MB/s | 124 kB 00:00 2024-03-21T09:53:13.265 INFO:teuthology.orchestra.run.smithi153.stdout:(14/33): python3-urllib3-1.26.5-5.el9.noarch.rp 2.4 MB/s | 187 kB 00:00 2024-03-21T09:53:13.374 INFO:teuthology.orchestra.run.smithi153.stdout:(15/33): python3-requests-2.25.1-8.el9.noarch.r 892 kB/s | 125 kB 00:00 2024-03-21T09:53:13.600 INFO:teuthology.orchestra.run.smithi153.stdout:(16/33): ceph-osd-18.2.1-0.el9.x86_64.rpm 16 MB/s | 18 MB 00:01 2024-03-21T09:53:13.701 INFO:teuthology.orchestra.run.smithi153.stdout:(17/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 100 kB/s | 43 kB 00:00 2024-03-21T09:53:13.808 INFO:teuthology.orchestra.run.smithi153.stdout:(18/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 99 kB/s | 11 kB 00:00 2024-03-21T09:53:13.908 INFO:teuthology.orchestra.run.smithi153.stdout:(19/33): python3-jaraco-classes-3.2.1-5.el9.noa 177 kB/s | 18 kB 00:00 2024-03-21T09:53:14.000 INFO:teuthology.orchestra.run.smithi153.stdout:(20/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 275 kB/s | 172 kB 00:00 2024-03-21T09:53:14.026 INFO:teuthology.orchestra.run.smithi153.stdout:(21/33): python3-jaraco-collections-3.0.0-8.el9 199 kB/s | 23 kB 00:00 2024-03-21T09:53:14.102 INFO:teuthology.orchestra.run.smithi153.stdout:(22/33): python3-jaraco-functools-3.5.0-2.el9.n 194 kB/s | 19 kB 00:00 2024-03-21T09:53:14.126 INFO:teuthology.orchestra.run.smithi153.stdout:(23/33): python3-jaraco-text-3.2.0-6.el9.noarch 198 kB/s | 20 kB 00:00 2024-03-21T09:53:14.185 INFO:teuthology.orchestra.run.smithi153.stdout:(24/33): python3-cherrypy-18.6.1-2.el9.noarch.r 614 kB/s | 358 kB 00:00 2024-03-21T09:53:14.210 INFO:teuthology.orchestra.run.smithi153.stdout:(25/33): python3-logutils-0.3.5-21.el9.noarch.r 428 kB/s | 46 kB 00:00 2024-03-21T09:53:14.294 INFO:teuthology.orchestra.run.smithi153.stdout:(26/33): python3-pecan-1.4.2-3.el9.noarch.rpm 2.5 MB/s | 272 kB 00:00 2024-03-21T09:53:14.311 INFO:teuthology.orchestra.run.smithi153.stdout:(27/33): python3-more-itertools-8.12.0-2.el9.no 430 kB/s | 79 kB 00:00 2024-03-21T09:53:14.336 INFO:teuthology.orchestra.run.smithi153.stdout:(28/33): python3-portend-3.1.0-2.el9.noarch.rpm 131 kB/s | 16 kB 00:00 2024-03-21T09:53:14.394 INFO:teuthology.orchestra.run.smithi153.stdout:(29/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 900 kB/s | 90 kB 00:00 2024-03-21T09:53:14.440 INFO:teuthology.orchestra.run.smithi153.stdout:(30/33): python3-tempora-5.0.0-2.el9.noarch.rpm 360 kB/s | 36 kB 00:00 2024-03-21T09:53:14.504 INFO:teuthology.orchestra.run.smithi153.stdout:(31/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 3.9 MB/s | 427 kB 00:00 2024-03-21T09:53:14.529 INFO:teuthology.orchestra.run.smithi153.stdout:(32/33): python3-webob-1.8.7-6.el9.noarch.rpm 1.2 MB/s | 230 kB 00:00 2024-03-21T09:53:14.553 INFO:teuthology.orchestra.run.smithi153.stdout:(33/33): python3-zc-lockfile-2.0-10.el9.noarch. 172 kB/s | 20 kB 00:00 2024-03-21T09:53:14.560 INFO:teuthology.orchestra.run.smithi153.stdout:-------------------------------------------------------------------------------- 2024-03-21T09:53:14.560 INFO:teuthology.orchestra.run.smithi153.stdout:Total 11 MB/s | 30 MB 00:02 2024-03-21T09:53:14.601 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction check 2024-03-21T09:53:14.686 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction check succeeded. 2024-03-21T09:53:14.686 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction test 2024-03-21T09:53:15.095 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: libcephsqlite-2:18.2.1-0.el9.x86_64 5/6 2024-03-21T09:53:15.138 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction test succeeded. 2024-03-21T09:53:15.139 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction 2024-03-21T09:53:15.210 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-test-2:18.2.1-0.el9.x86_64 6/6 2024-03-21T09:53:16.489 INFO:teuthology.orchestra.run.smithi153.stdout: Preparing : 1/1 2024-03-21T09:53:16.582 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-03-21T09:53:16.628 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-03-21T09:53:16.675 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-03-21T09:53:16.825 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-03-21T09:53:16.918 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-03-21T09:53:16.980 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 6/33 2024-03-21T09:53:17.073 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-03-21T09:53:17.133 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 8/33 2024-03-21T09:53:17.185 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 9/33 2024-03-21T09:53:17.251 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-ply-3.11-14.el9.noarch 10/33 2024-03-21T09:53:17.289 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-test-2:18.2.1-0.el9.x86_64 6/6 2024-03-21T09:53:17.290 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-test-2:18.2.1-0.el9.x86_64 1/6 2024-03-21T09:53:17.290 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libcephsqlite-2:18.2.1-0.el9.x86_64 2/6 2024-03-21T09:53:17.290 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 3/6 2024-03-21T09:53:17.290 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 4/6 2024-03-21T09:53:17.290 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 5/6 2024-03-21T09:53:17.334 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 11/33 2024-03-21T09:53:17.590 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 12/33 2024-03-21T09:53:17.613 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : jq-1.6-16.el9.x86_64 6/6 2024-03-21T09:53:17.613 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-21T09:53:17.613 INFO:teuthology.orchestra.run.smithi174.stdout:Installed: 2024-03-21T09:53:17.613 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-test-2:18.2.1-0.el9.x86_64 jq-1.6-16.el9.x86_64 2024-03-21T09:53:17.613 INFO:teuthology.orchestra.run.smithi174.stdout: libcephsqlite-2:18.2.1-0.el9.x86_64 oniguruma-6.9.6-1.el9.5.x86_64 2024-03-21T09:53:17.614 INFO:teuthology.orchestra.run.smithi174.stdout: socat-1.7.4.1-5.el9.x86_64 xmlstarlet-1.6.1-20.el9.x86_64 2024-03-21T09:53:17.614 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-21T09:53:17.614 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2024-03-21T09:53:17.656 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 13/33 2024-03-21T09:53:17.698 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 14/33 2024-03-21T09:53:17.746 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 15/33 2024-03-21T09:53:17.784 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 16/33 2024-03-21T09:53:17.902 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 17/33 2024-03-21T09:53:17.942 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-03-21T09:53:17.977 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 19/33 2024-03-21T09:53:18.014 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum -y install ceph 2024-03-21T09:53:18.027 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 20/33 2024-03-21T09:53:18.092 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/33 2024-03-21T09:53:18.132 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 22/33 2024-03-21T09:53:18.274 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 23/33 2024-03-21T09:53:18.317 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 24/33 2024-03-21T09:53:18.435 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/33 2024-03-21T09:53:18.637 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 26/33 2024-03-21T09:53:18.802 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-03-21T09:53:18.872 INFO:teuthology.orchestra.run.smithi174.stdout:Last metadata expiration check: 0:00:40 ago on Thu 21 Mar 2024 09:52:38 AM UTC. 2024-03-21T09:53:18.987 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-mgr-modules-core-2:18.2.1-0.el9.noarch 28/33 2024-03-21T09:53:19.026 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-mgr-2:18.2.1-0.el9.x86_64 29/33 2024-03-21T09:53:19.027 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2024-03-21T09:53:19.030 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2024-03-21T09:53:19.030 INFO:teuthology.orchestra.run.smithi174.stdout: Package Arch Version Repository Size 2024-03-21T09:53:19.030 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2024-03-21T09:53:19.030 INFO:teuthology.orchestra.run.smithi174.stdout:Installing: 2024-03-21T09:53:19.031 INFO:teuthology.orchestra.run.smithi174.stdout: ceph x86_64 2:18.2.1-0.el9 ceph 6.4 k 2024-03-21T09:53:19.031 INFO:teuthology.orchestra.run.smithi174.stdout:Installing dependencies: 2024-03-21T09:53:19.031 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mds x86_64 2:18.2.1-0.el9 ceph 2.1 M 2024-03-21T09:53:19.031 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr x86_64 2:18.2.1-0.el9 ceph 1.4 M 2024-03-21T09:53:19.031 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-modules-core noarch 2:18.2.1-0.el9 ceph-noarch 242 k 2024-03-21T09:53:19.031 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mon x86_64 2:18.2.1-0.el9 ceph 4.4 M 2024-03-21T09:53:19.031 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-osd x86_64 2:18.2.1-0.el9 ceph 18 M 2024-03-21T09:53:19.031 INFO:teuthology.orchestra.run.smithi174.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-21T09:53:19.031 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-21T09:53:19.031 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-21T09:53:19.031 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-21T09:53:19.031 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-21T09:53:19.031 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-21T09:53:19.032 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-21T09:53:19.032 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-21T09:53:19.032 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-21T09:53:19.032 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-21T09:53:19.032 INFO:teuthology.orchestra.run.smithi174.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-21T09:53:19.032 INFO:teuthology.orchestra.run.smithi174.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-21T09:53:19.032 INFO:teuthology.orchestra.run.smithi174.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-21T09:53:19.032 INFO:teuthology.orchestra.run.smithi174.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-21T09:53:19.032 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-21T09:53:19.032 INFO:teuthology.orchestra.run.smithi174.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-21T09:53:19.033 INFO:teuthology.orchestra.run.smithi174.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-21T09:53:19.033 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-21T09:53:19.033 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-21T09:53:19.033 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-21T09:53:19.033 INFO:teuthology.orchestra.run.smithi174.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-21T09:53:19.033 INFO:teuthology.orchestra.run.smithi174.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-21T09:53:19.033 INFO:teuthology.orchestra.run.smithi174.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-21T09:53:19.033 INFO:teuthology.orchestra.run.smithi174.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-21T09:53:19.033 INFO:teuthology.orchestra.run.smithi174.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-21T09:53:19.033 INFO:teuthology.orchestra.run.smithi174.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-21T09:53:19.033 INFO:teuthology.orchestra.run.smithi174.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-21T09:53:19.033 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-21T09:53:19.033 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction Summary 2024-03-21T09:53:19.033 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2024-03-21T09:53:19.034 INFO:teuthology.orchestra.run.smithi174.stdout:Install 33 Packages 2024-03-21T09:53:19.034 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-21T09:53:19.034 INFO:teuthology.orchestra.run.smithi174.stdout:Total download size: 30 M 2024-03-21T09:53:19.034 INFO:teuthology.orchestra.run.smithi174.stdout:Installed size: 110 M 2024-03-21T09:53:19.035 INFO:teuthology.orchestra.run.smithi174.stdout:Downloading Packages: 2024-03-21T09:53:19.098 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-mgr-2:18.2.1-0.el9.x86_64 29/33 2024-03-21T09:53:19.098 INFO:teuthology.orchestra.run.smithi153.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T09:53:19.098 INFO:teuthology.orchestra.run.smithi153.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-21T09:53:19.098 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-21T09:53:19.098 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-21T09:53:19.098 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T09:53:19.499 INFO:teuthology.orchestra.run.smithi174.stdout:(1/33): ceph-18.2.1-0.el9.x86_64.rpm 34 kB/s | 6.4 kB 00:00 2024-03-21T09:53:19.602 INFO:teuthology.orchestra.run.smithi174.stdout:(2/33): ceph-mgr-18.2.1-0.el9.x86_64.rpm 5.0 MB/s | 1.4 MB 00:00 2024-03-21T09:53:19.653 INFO:teuthology.orchestra.run.smithi174.stdout:(3/33): ceph-mds-18.2.1-0.el9.x86_64.rpm 6.2 MB/s | 2.1 MB 00:00 2024-03-21T09:53:19.695 INFO:teuthology.orchestra.run.smithi174.stdout:(4/33): ceph-mgr-modules-core-18.2.1-0.el9.noar 5.9 MB/s | 242 kB 00:00 2024-03-21T09:53:19.854 INFO:teuthology.orchestra.run.smithi174.stdout:(5/33): ceph-mon-18.2.1-0.el9.x86_64.rpm 12 MB/s | 4.4 MB 00:00 2024-03-21T09:53:19.920 INFO:teuthology.orchestra.run.smithi174.stdout:(6/33): python3-mako-1.1.4-6.el9.noarch.rpm 643 kB/s | 144 kB 00:00 2024-03-21T09:53:19.946 INFO:teuthology.orchestra.run.smithi174.stdout:(7/33): python3-markupsafe-1.1.1-12.el9.x86_64. 345 kB/s | 32 kB 00:00 2024-03-21T09:53:19.988 INFO:teuthology.orchestra.run.smithi174.stdout:(8/33): python3-pytz-2021.1-5.el9.noarch.rpm 713 kB/s | 47 kB 00:00 2024-03-21T09:53:20.013 INFO:teuthology.orchestra.run.smithi174.stdout:(9/33): python3-toml-0.10.2-6.el9.noarch.rpm 581 kB/s | 39 kB 00:00 2024-03-21T09:53:20.105 INFO:teuthology.orchestra.run.smithi174.stdout:(10/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.0 MB/s | 241 kB 00:00 2024-03-21T09:53:20.356 INFO:teuthology.orchestra.run.smithi174.stdout:(11/33): ceph-osd-18.2.1-0.el9.x86_64.rpm 23 MB/s | 18 MB 00:00 2024-03-21T09:53:20.414 INFO:teuthology.orchestra.run.smithi174.stdout:(12/33): python3-cryptography-36.0.1-4.el9.x86_ 2.9 MB/s | 1.1 MB 00:00 2024-03-21T09:53:20.430 INFO:teuthology.orchestra.run.smithi174.stdout:(13/33): python3-ply-3.11-14.el9.noarch.rpm 317 kB/s | 103 kB 00:00 2024-03-21T09:53:20.456 INFO:teuthology.orchestra.run.smithi174.stdout:(14/33): python3-pycparser-2.20-6.el9.noarch.rp 1.3 MB/s | 124 kB 00:00 2024-03-21T09:53:20.518 INFO:teuthology.orchestra.run.smithi174.stdout:(15/33): python3-urllib3-1.26.5-5.el9.noarch.rp 1.8 MB/s | 187 kB 00:00 2024-03-21T09:53:20.718 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-osd-2:18.2.1-0.el9.x86_64 30/33 2024-03-21T09:53:20.731 INFO:teuthology.orchestra.run.smithi174.stdout:(16/33): python3-requests-2.25.1-8.el9.noarch.r 458 kB/s | 125 kB 00:00 2024-03-21T09:53:20.794 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-osd-2:18.2.1-0.el9.x86_64 30/33 2024-03-21T09:53:20.794 INFO:teuthology.orchestra.run.smithi153.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T09:53:20.794 INFO:teuthology.orchestra.run.smithi153.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-21T09:53:20.795 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-21T09:53:20.795 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-21T09:53:20.795 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T09:53:21.173 INFO:teuthology.orchestra.run.smithi174.stdout:(17/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 61 kB/s | 43 kB 00:00 2024-03-21T09:53:21.277 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-mon-2:18.2.1-0.el9.x86_64 31/33 2024-03-21T09:53:21.282 INFO:teuthology.orchestra.run.smithi174.stdout:(18/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 99 kB/s | 11 kB 00:00 2024-03-21T09:53:21.351 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-mon-2:18.2.1-0.el9.x86_64 31/33 2024-03-21T09:53:21.351 INFO:teuthology.orchestra.run.smithi153.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T09:53:21.351 INFO:teuthology.orchestra.run.smithi153.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-21T09:53:21.351 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-21T09:53:21.351 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-21T09:53:21.351 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T09:53:21.382 INFO:teuthology.orchestra.run.smithi174.stdout:(19/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 199 kB/s | 172 kB 00:00 2024-03-21T09:53:21.407 INFO:teuthology.orchestra.run.smithi174.stdout:(20/33): python3-jaraco-classes-3.2.1-5.el9.noa 142 kB/s | 18 kB 00:00 2024-03-21T09:53:21.491 INFO:teuthology.orchestra.run.smithi174.stdout:(21/33): python3-jaraco-collections-3.0.0-8.el9 214 kB/s | 23 kB 00:00 2024-03-21T09:53:21.544 INFO:teuthology.orchestra.run.smithi174.stdout:(22/33): python3-jaraco-functools-3.5.0-2.el9.n 142 kB/s | 19 kB 00:00 2024-03-21T09:53:21.569 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-mds-2:18.2.1-0.el9.x86_64 32/33 2024-03-21T09:53:21.601 INFO:teuthology.orchestra.run.smithi174.stdout:(23/33): python3-jaraco-text-3.2.0-6.el9.noarch 182 kB/s | 20 kB 00:00 2024-03-21T09:53:21.638 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-mds-2:18.2.1-0.el9.x86_64 32/33 2024-03-21T09:53:21.638 INFO:teuthology.orchestra.run.smithi153.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T09:53:21.638 INFO:teuthology.orchestra.run.smithi153.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-21T09:53:21.638 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-21T09:53:21.639 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-21T09:53:21.639 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T09:53:21.650 INFO:teuthology.orchestra.run.smithi174.stdout:(24/33): python3-logutils-0.3.5-21.el9.noarch.r 443 kB/s | 46 kB 00:00 2024-03-21T09:53:21.704 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-2:18.2.1-0.el9.x86_64 33/33 2024-03-21T09:53:21.717 INFO:teuthology.orchestra.run.smithi174.stdout:(25/33): python3-more-itertools-8.12.0-2.el9.no 677 kB/s | 79 kB 00:00 2024-03-21T09:53:21.750 INFO:teuthology.orchestra.run.smithi174.stdout:(26/33): python3-cherrypy-18.6.1-2.el9.noarch.r 353 kB/s | 358 kB 00:01 2024-03-21T09:53:21.824 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-2:18.2.1-0.el9.x86_64 33/33 2024-03-21T09:53:21.824 INFO:teuthology.orchestra.run.smithi153.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-03-21T09:53:21.824 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T09:53:21.834 INFO:teuthology.orchestra.run.smithi174.stdout:(27/33): python3-portend-3.1.0-2.el9.noarch.rpm 141 kB/s | 16 kB 00:00 2024-03-21T09:53:21.951 INFO:teuthology.orchestra.run.smithi174.stdout:(28/33): python3-tempora-5.0.0-2.el9.noarch.rpm 308 kB/s | 36 kB 00:00 2024-03-21T09:53:21.976 INFO:teuthology.orchestra.run.smithi174.stdout:(29/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 400 kB/s | 90 kB 00:00 2024-03-21T09:53:22.135 INFO:teuthology.orchestra.run.smithi174.stdout:(30/33): python3-pecan-1.4.2-3.el9.noarch.rpm 563 kB/s | 272 kB 00:00 2024-03-21T09:53:22.252 INFO:teuthology.orchestra.run.smithi174.stdout:(31/33): python3-webob-1.8.7-6.el9.noarch.rpm 767 kB/s | 230 kB 00:00 2024-03-21T09:53:22.277 INFO:teuthology.orchestra.run.smithi174.stdout:(32/33): python3-zc-lockfile-2.0-10.el9.noarch. 142 kB/s | 20 kB 00:00 2024-03-21T09:53:22.635 INFO:teuthology.orchestra.run.smithi174.stdout:(33/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 649 kB/s | 427 kB 00:00 2024-03-21T09:53:22.648 INFO:teuthology.orchestra.run.smithi174.stdout:-------------------------------------------------------------------------------- 2024-03-21T09:53:22.648 INFO:teuthology.orchestra.run.smithi174.stdout:Total 8.2 MB/s | 30 MB 00:03 2024-03-21T09:53:22.690 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction check 2024-03-21T09:53:22.775 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction check succeeded. 2024-03-21T09:53:22.776 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction test 2024-03-21T09:53:23.233 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction test succeeded. 2024-03-21T09:53:23.233 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction 2024-03-21T09:53:23.697 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-2:18.2.1-0.el9.x86_64 1/33 2024-03-21T09:53:23.697 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-mds-2:18.2.1-0.el9.x86_64 2/33 2024-03-21T09:53:23.697 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-mgr-2:18.2.1-0.el9.x86_64 3/33 2024-03-21T09:53:23.697 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-mon-2:18.2.1-0.el9.x86_64 4/33 2024-03-21T09:53:23.697 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-osd-2:18.2.1-0.el9.x86_64 5/33 2024-03-21T09:53:23.697 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-mgr-modules-core-2:18.2.1-0.el9.noarch 6/33 2024-03-21T09:53:23.697 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 7/33 2024-03-21T09:53:23.697 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 8/33 2024-03-21T09:53:23.698 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 9/33 2024-03-21T09:53:23.698 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 10/33 2024-03-21T09:53:23.698 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 11/33 2024-03-21T09:53:23.698 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 12/33 2024-03-21T09:53:23.698 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-ply-3.11-14.el9.noarch 13/33 2024-03-21T09:53:23.698 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 14/33 2024-03-21T09:53:23.698 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 15/33 2024-03-21T09:53:23.698 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 16/33 2024-03-21T09:53:23.698 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-03-21T09:53:23.698 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-03-21T09:53:23.698 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-03-21T09:53:23.698 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-03-21T09:53:23.698 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-03-21T09:53:23.699 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-03-21T09:53:23.699 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-03-21T09:53:23.699 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-03-21T09:53:23.699 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-03-21T09:53:23.699 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-03-21T09:53:23.699 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-03-21T09:53:23.699 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-03-21T09:53:23.700 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-03-21T09:53:23.700 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-03-21T09:53:23.700 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 31/33 2024-03-21T09:53:23.700 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-03-21T09:53:24.619 INFO:teuthology.orchestra.run.smithi174.stdout: Preparing : 1/1 2024-03-21T09:53:24.680 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-03-21T09:53:24.680 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T09:53:24.680 INFO:teuthology.orchestra.run.smithi153.stdout:Installed: 2024-03-21T09:53:24.680 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-2:18.2.1-0.el9.x86_64 2024-03-21T09:53:24.680 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mds-2:18.2.1-0.el9.x86_64 2024-03-21T09:53:24.680 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-2:18.2.1-0.el9.x86_64 2024-03-21T09:53:24.680 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-modules-core-2:18.2.1-0.el9.noarch 2024-03-21T09:53:24.680 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mon-2:18.2.1-0.el9.x86_64 2024-03-21T09:53:24.680 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-osd-2:18.2.1-0.el9.x86_64 2024-03-21T09:53:24.680 INFO:teuthology.orchestra.run.smithi153.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-21T09:53:24.681 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-21T09:53:24.681 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-21T09:53:24.681 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-21T09:53:24.681 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-21T09:53:24.681 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-21T09:53:24.681 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-21T09:53:24.681 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-21T09:53:24.681 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-21T09:53:24.681 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-21T09:53:24.681 INFO:teuthology.orchestra.run.smithi153.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-21T09:53:24.681 INFO:teuthology.orchestra.run.smithi153.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-21T09:53:24.681 INFO:teuthology.orchestra.run.smithi153.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-21T09:53:24.681 INFO:teuthology.orchestra.run.smithi153.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-21T09:53:24.681 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-21T09:53:24.682 INFO:teuthology.orchestra.run.smithi153.stdout: python3-ply-3.11-14.el9.noarch 2024-03-21T09:53:24.682 INFO:teuthology.orchestra.run.smithi153.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-21T09:53:24.682 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-21T09:53:24.682 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-21T09:53:24.682 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-21T09:53:24.682 INFO:teuthology.orchestra.run.smithi153.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-21T09:53:24.682 INFO:teuthology.orchestra.run.smithi153.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-21T09:53:24.682 INFO:teuthology.orchestra.run.smithi153.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-21T09:53:24.682 INFO:teuthology.orchestra.run.smithi153.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-21T09:53:24.682 INFO:teuthology.orchestra.run.smithi153.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-21T09:53:24.682 INFO:teuthology.orchestra.run.smithi153.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-21T09:53:24.682 INFO:teuthology.orchestra.run.smithi153.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-21T09:53:24.682 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T09:53:24.682 INFO:teuthology.orchestra.run.smithi153.stdout:Complete! 2024-03-21T09:53:24.689 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-03-21T09:53:24.723 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-03-21T09:53:24.769 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-03-21T09:53:24.909 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-03-21T09:53:25.016 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-03-21T09:53:25.047 DEBUG:teuthology.orchestra.run.smithi153:> sudo yum -y install ceph-base 2024-03-21T09:53:25.069 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 6/33 2024-03-21T09:53:25.172 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-03-21T09:53:25.240 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 8/33 2024-03-21T09:53:25.282 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 9/33 2024-03-21T09:53:25.347 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-ply-3.11-14.el9.noarch 10/33 2024-03-21T09:53:25.424 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 11/33 2024-03-21T09:53:25.677 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 12/33 2024-03-21T09:53:25.753 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 13/33 2024-03-21T09:53:25.799 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 14/33 2024-03-21T09:53:25.846 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 15/33 2024-03-21T09:53:25.882 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 16/33 2024-03-21T09:53:25.885 INFO:teuthology.orchestra.run.smithi153.stdout:Last metadata expiration check: 0:00:56 ago on Thu 21 Mar 2024 09:52:29 AM UTC. 2024-03-21T09:53:25.982 INFO:teuthology.orchestra.run.smithi153.stdout:Package ceph-base-2:18.2.1-0.el9.x86_64 is already installed. 2024-03-21T09:53:25.984 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 17/33 2024-03-21T09:53:26.023 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-03-21T09:53:26.036 INFO:teuthology.orchestra.run.smithi153.stdout:Dependencies resolved. 2024-03-21T09:53:26.037 INFO:teuthology.orchestra.run.smithi153.stdout:Nothing to do. 2024-03-21T09:53:26.037 INFO:teuthology.orchestra.run.smithi153.stdout:Complete! 2024-03-21T09:53:26.056 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 19/33 2024-03-21T09:53:26.104 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 20/33 2024-03-21T09:53:26.117 DEBUG:teuthology.orchestra.run.smithi153:> sudo yum -y install cephadm 2024-03-21T09:53:26.157 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/33 2024-03-21T09:53:26.194 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 22/33 2024-03-21T09:53:26.323 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 23/33 2024-03-21T09:53:26.373 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 24/33 2024-03-21T09:53:26.490 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/33 2024-03-21T09:53:26.668 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 26/33 2024-03-21T09:53:26.823 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-03-21T09:53:26.937 INFO:teuthology.orchestra.run.smithi153.stdout:Last metadata expiration check: 0:00:57 ago on Thu 21 Mar 2024 09:52:29 AM UTC. 2024-03-21T09:53:27.009 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mgr-modules-core-2:18.2.1-0.el9.noarch 28/33 2024-03-21T09:53:27.040 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mgr-2:18.2.1-0.el9.x86_64 29/33 2024-03-21T09:53:27.088 INFO:teuthology.orchestra.run.smithi153.stdout:Dependencies resolved. 2024-03-21T09:53:27.088 INFO:teuthology.orchestra.run.smithi153.stdout:================================================================================ 2024-03-21T09:53:27.088 INFO:teuthology.orchestra.run.smithi153.stdout: Package Architecture Version Repository Size 2024-03-21T09:53:27.088 INFO:teuthology.orchestra.run.smithi153.stdout:================================================================================ 2024-03-21T09:53:27.088 INFO:teuthology.orchestra.run.smithi153.stdout:Installing: 2024-03-21T09:53:27.088 INFO:teuthology.orchestra.run.smithi153.stdout: cephadm noarch 2:18.2.1-0.el9 ceph-noarch 221 k 2024-03-21T09:53:27.088 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T09:53:27.088 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction Summary 2024-03-21T09:53:27.088 INFO:teuthology.orchestra.run.smithi153.stdout:================================================================================ 2024-03-21T09:53:27.088 INFO:teuthology.orchestra.run.smithi153.stdout:Install 1 Package 2024-03-21T09:53:27.089 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T09:53:27.089 INFO:teuthology.orchestra.run.smithi153.stdout:Total download size: 221 k 2024-03-21T09:53:27.089 INFO:teuthology.orchestra.run.smithi153.stdout:Installed size: 213 k 2024-03-21T09:53:27.089 INFO:teuthology.orchestra.run.smithi153.stdout:Downloading Packages: 2024-03-21T09:53:27.112 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-mgr-2:18.2.1-0.el9.x86_64 29/33 2024-03-21T09:53:27.113 INFO:teuthology.orchestra.run.smithi174.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T09:53:27.113 INFO:teuthology.orchestra.run.smithi174.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-21T09:53:27.113 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-21T09:53:27.113 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-21T09:53:27.113 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-21T09:53:27.192 INFO:teuthology.orchestra.run.smithi153.stdout:cephadm-18.2.1-0.el9.noarch.rpm 2.1 MB/s | 221 kB 00:00 2024-03-21T09:53:27.194 INFO:teuthology.orchestra.run.smithi153.stdout:-------------------------------------------------------------------------------- 2024-03-21T09:53:27.194 INFO:teuthology.orchestra.run.smithi153.stdout:Total 2.1 MB/s | 221 kB 00:00 2024-03-21T09:53:27.194 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction check 2024-03-21T09:53:27.202 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction check succeeded. 2024-03-21T09:53:27.202 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction test 2024-03-21T09:53:27.206 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction test succeeded. 2024-03-21T09:53:27.206 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction 2024-03-21T09:53:27.306 INFO:teuthology.orchestra.run.smithi153.stdout: Preparing : 1/1 2024-03-21T09:53:27.963 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: cephadm-2:18.2.1-0.el9.noarch 1/1 2024-03-21T09:53:28.097 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : cephadm-2:18.2.1-0.el9.noarch 1/1 2024-03-21T09:53:28.593 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: cephadm-2:18.2.1-0.el9.noarch 1/1 2024-03-21T09:53:28.750 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-osd-2:18.2.1-0.el9.x86_64 30/33 2024-03-21T09:53:28.829 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-osd-2:18.2.1-0.el9.x86_64 30/33 2024-03-21T09:53:28.829 INFO:teuthology.orchestra.run.smithi174.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T09:53:28.830 INFO:teuthology.orchestra.run.smithi174.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-21T09:53:28.830 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-21T09:53:28.830 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-21T09:53:28.830 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-21T09:53:28.914 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : cephadm-2:18.2.1-0.el9.noarch 1/1 2024-03-21T09:53:28.914 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T09:53:28.914 INFO:teuthology.orchestra.run.smithi153.stdout:Installed: 2024-03-21T09:53:28.914 INFO:teuthology.orchestra.run.smithi153.stdout: cephadm-2:18.2.1-0.el9.noarch 2024-03-21T09:53:28.914 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T09:53:28.914 INFO:teuthology.orchestra.run.smithi153.stdout:Complete! 2024-03-21T09:53:29.160 DEBUG:teuthology.orchestra.run.smithi153:> sudo yum -y install ceph-immutable-object-cache 2024-03-21T09:53:29.325 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mon-2:18.2.1-0.el9.x86_64 31/33 2024-03-21T09:53:29.394 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-mon-2:18.2.1-0.el9.x86_64 31/33 2024-03-21T09:53:29.394 INFO:teuthology.orchestra.run.smithi174.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T09:53:29.394 INFO:teuthology.orchestra.run.smithi174.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-21T09:53:29.394 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-21T09:53:29.394 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-21T09:53:29.394 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-21T09:53:29.617 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mds-2:18.2.1-0.el9.x86_64 32/33 2024-03-21T09:53:29.685 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-mds-2:18.2.1-0.el9.x86_64 32/33 2024-03-21T09:53:29.685 INFO:teuthology.orchestra.run.smithi174.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T09:53:29.685 INFO:teuthology.orchestra.run.smithi174.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-21T09:53:29.685 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-21T09:53:29.685 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-21T09:53:29.685 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-21T09:53:29.813 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-2:18.2.1-0.el9.x86_64 33/33 2024-03-21T09:53:29.936 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-2:18.2.1-0.el9.x86_64 33/33 2024-03-21T09:53:29.936 INFO:teuthology.orchestra.run.smithi174.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-03-21T09:53:29.936 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-21T09:53:30.020 INFO:teuthology.orchestra.run.smithi153.stdout:Last metadata expiration check: 0:01:01 ago on Thu 21 Mar 2024 09:52:29 AM UTC. 2024-03-21T09:53:30.177 INFO:teuthology.orchestra.run.smithi153.stdout:Dependencies resolved. 2024-03-21T09:53:30.178 INFO:teuthology.orchestra.run.smithi153.stdout:================================================================================ 2024-03-21T09:53:30.178 INFO:teuthology.orchestra.run.smithi153.stdout: Package Arch Version Repo Size 2024-03-21T09:53:30.178 INFO:teuthology.orchestra.run.smithi153.stdout:================================================================================ 2024-03-21T09:53:30.178 INFO:teuthology.orchestra.run.smithi153.stdout:Installing: 2024-03-21T09:53:30.178 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-immutable-object-cache x86_64 2:18.2.1-0.el9 ceph 142 k 2024-03-21T09:53:30.178 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T09:53:30.179 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction Summary 2024-03-21T09:53:30.179 INFO:teuthology.orchestra.run.smithi153.stdout:================================================================================ 2024-03-21T09:53:30.179 INFO:teuthology.orchestra.run.smithi153.stdout:Install 1 Package 2024-03-21T09:53:30.179 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T09:53:30.179 INFO:teuthology.orchestra.run.smithi153.stdout:Total download size: 142 k 2024-03-21T09:53:30.179 INFO:teuthology.orchestra.run.smithi153.stdout:Installed size: 395 k 2024-03-21T09:53:30.180 INFO:teuthology.orchestra.run.smithi153.stdout:Downloading Packages: 2024-03-21T09:53:30.282 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-immutable-object-cache-18.2.1-0.el9.x86_64 1.4 MB/s | 142 kB 00:00 2024-03-21T09:53:30.282 INFO:teuthology.orchestra.run.smithi153.stdout:-------------------------------------------------------------------------------- 2024-03-21T09:53:30.283 INFO:teuthology.orchestra.run.smithi153.stdout:Total 1.3 MB/s | 142 kB 00:00 2024-03-21T09:53:30.283 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction check 2024-03-21T09:53:30.295 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction check succeeded. 2024-03-21T09:53:30.295 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction test 2024-03-21T09:53:30.365 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction test succeeded. 2024-03-21T09:53:30.366 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction 2024-03-21T09:53:30.548 INFO:teuthology.orchestra.run.smithi153.stdout: Preparing : 1/1 2024-03-21T09:53:30.625 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-immutable-object-cache-2:18.2.1-0.el9.x86_64 1/1 2024-03-21T09:53:30.698 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.1-0.el9.x86_64 1/1 2024-03-21T09:53:30.698 INFO:teuthology.orchestra.run.smithi153.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T09:53:30.698 INFO:teuthology.orchestra.run.smithi153.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-21T09:53:30.698 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T09:53:31.750 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-2:18.2.1-0.el9.x86_64 1/33 2024-03-21T09:53:31.750 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mds-2:18.2.1-0.el9.x86_64 2/33 2024-03-21T09:53:31.750 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mgr-2:18.2.1-0.el9.x86_64 3/33 2024-03-21T09:53:31.750 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mon-2:18.2.1-0.el9.x86_64 4/33 2024-03-21T09:53:31.751 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-osd-2:18.2.1-0.el9.x86_64 5/33 2024-03-21T09:53:31.751 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mgr-modules-core-2:18.2.1-0.el9.noarch 6/33 2024-03-21T09:53:31.751 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 7/33 2024-03-21T09:53:31.751 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 8/33 2024-03-21T09:53:31.751 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 9/33 2024-03-21T09:53:31.751 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 10/33 2024-03-21T09:53:31.751 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 11/33 2024-03-21T09:53:31.751 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 12/33 2024-03-21T09:53:31.751 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-ply-3.11-14.el9.noarch 13/33 2024-03-21T09:53:31.751 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 14/33 2024-03-21T09:53:31.752 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 15/33 2024-03-21T09:53:31.752 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 16/33 2024-03-21T09:53:31.752 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-03-21T09:53:31.752 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-03-21T09:53:31.752 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-03-21T09:53:31.752 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-03-21T09:53:31.752 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-03-21T09:53:31.752 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-03-21T09:53:31.753 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-03-21T09:53:31.753 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-03-21T09:53:31.753 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-03-21T09:53:31.753 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-03-21T09:53:31.753 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-03-21T09:53:31.753 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-03-21T09:53:31.753 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-03-21T09:53:31.753 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-03-21T09:53:31.754 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 31/33 2024-03-21T09:53:31.754 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-03-21T09:53:31.998 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-immutable-object-cache-2:18.2.1-0.el9.x86_64 1/1 2024-03-21T09:53:31.999 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T09:53:31.999 INFO:teuthology.orchestra.run.smithi153.stdout:Installed: 2024-03-21T09:53:31.999 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-immutable-object-cache-2:18.2.1-0.el9.x86_64 2024-03-21T09:53:31.999 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T09:53:31.999 INFO:teuthology.orchestra.run.smithi153.stdout:Complete! 2024-03-21T09:53:32.283 DEBUG:teuthology.orchestra.run.smithi153:> sudo yum -y install ceph-mgr 2024-03-21T09:53:32.741 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-03-21T09:53:32.741 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-21T09:53:32.742 INFO:teuthology.orchestra.run.smithi174.stdout:Installed: 2024-03-21T09:53:32.742 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-2:18.2.1-0.el9.x86_64 2024-03-21T09:53:32.742 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mds-2:18.2.1-0.el9.x86_64 2024-03-21T09:53:32.742 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-2:18.2.1-0.el9.x86_64 2024-03-21T09:53:32.742 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-modules-core-2:18.2.1-0.el9.noarch 2024-03-21T09:53:32.742 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mon-2:18.2.1-0.el9.x86_64 2024-03-21T09:53:32.742 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-osd-2:18.2.1-0.el9.x86_64 2024-03-21T09:53:32.742 INFO:teuthology.orchestra.run.smithi174.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-21T09:53:32.742 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-21T09:53:32.742 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-21T09:53:32.742 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-21T09:53:32.742 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-21T09:53:32.742 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-21T09:53:32.743 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-21T09:53:32.743 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-21T09:53:32.743 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-21T09:53:32.743 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-21T09:53:32.743 INFO:teuthology.orchestra.run.smithi174.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-21T09:53:32.743 INFO:teuthology.orchestra.run.smithi174.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-21T09:53:32.743 INFO:teuthology.orchestra.run.smithi174.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-21T09:53:32.743 INFO:teuthology.orchestra.run.smithi174.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-21T09:53:32.743 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-21T09:53:32.743 INFO:teuthology.orchestra.run.smithi174.stdout: python3-ply-3.11-14.el9.noarch 2024-03-21T09:53:32.743 INFO:teuthology.orchestra.run.smithi174.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-21T09:53:32.743 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-21T09:53:32.743 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-21T09:53:32.743 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-21T09:53:32.744 INFO:teuthology.orchestra.run.smithi174.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-21T09:53:32.744 INFO:teuthology.orchestra.run.smithi174.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-21T09:53:32.744 INFO:teuthology.orchestra.run.smithi174.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-21T09:53:32.744 INFO:teuthology.orchestra.run.smithi174.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-21T09:53:32.744 INFO:teuthology.orchestra.run.smithi174.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-21T09:53:32.744 INFO:teuthology.orchestra.run.smithi174.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-21T09:53:32.744 INFO:teuthology.orchestra.run.smithi174.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-21T09:53:32.744 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-21T09:53:32.744 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2024-03-21T09:53:33.122 INFO:teuthology.orchestra.run.smithi153.stdout:Last metadata expiration check: 0:01:04 ago on Thu 21 Mar 2024 09:52:29 AM UTC. 2024-03-21T09:53:33.125 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum -y install ceph-base 2024-03-21T09:53:33.217 INFO:teuthology.orchestra.run.smithi153.stdout:Package ceph-mgr-2:18.2.1-0.el9.x86_64 is already installed. 2024-03-21T09:53:33.270 INFO:teuthology.orchestra.run.smithi153.stdout:Dependencies resolved. 2024-03-21T09:53:33.271 INFO:teuthology.orchestra.run.smithi153.stdout:Nothing to do. 2024-03-21T09:53:33.272 INFO:teuthology.orchestra.run.smithi153.stdout:Complete! 2024-03-21T09:53:33.339 DEBUG:teuthology.orchestra.run.smithi153:> sudo yum -y install ceph-mgr-dashboard 2024-03-21T09:53:33.907 INFO:teuthology.orchestra.run.smithi174.stdout:Last metadata expiration check: 0:00:55 ago on Thu 21 Mar 2024 09:52:38 AM UTC. 2024-03-21T09:53:34.003 INFO:teuthology.orchestra.run.smithi174.stdout:Package ceph-base-2:18.2.1-0.el9.x86_64 is already installed. 2024-03-21T09:53:34.057 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2024-03-21T09:53:34.058 INFO:teuthology.orchestra.run.smithi174.stdout:Nothing to do. 2024-03-21T09:53:34.058 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2024-03-21T09:53:34.124 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum -y install cephadm 2024-03-21T09:53:34.133 INFO:teuthology.orchestra.run.smithi153.stdout:Last metadata expiration check: 0:01:05 ago on Thu 21 Mar 2024 09:52:29 AM UTC. 2024-03-21T09:53:34.290 INFO:teuthology.orchestra.run.smithi153.stdout:Dependencies resolved. 2024-03-21T09:53:34.291 INFO:teuthology.orchestra.run.smithi153.stdout:================================================================================ 2024-03-21T09:53:34.291 INFO:teuthology.orchestra.run.smithi153.stdout: Package Arch Version Repository Size 2024-03-21T09:53:34.291 INFO:teuthology.orchestra.run.smithi153.stdout:================================================================================ 2024-03-21T09:53:34.291 INFO:teuthology.orchestra.run.smithi153.stdout:Installing: 2024-03-21T09:53:34.291 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-dashboard noarch 2:18.2.1-0.el9 ceph-noarch 1.8 M 2024-03-21T09:53:34.291 INFO:teuthology.orchestra.run.smithi153.stdout:Installing dependencies: 2024-03-21T09:53:34.291 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-grafana-dashboards noarch 2:18.2.1-0.el9 ceph-noarch 23 k 2024-03-21T09:53:34.291 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-prometheus-alerts noarch 2:18.2.1-0.el9 ceph-noarch 15 k 2024-03-21T09:53:34.291 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-03-21T09:53:34.291 INFO:teuthology.orchestra.run.smithi153.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-21T09:53:34.291 INFO:teuthology.orchestra.run.smithi153.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-21T09:53:34.292 INFO:teuthology.orchestra.run.smithi153.stdout:Installing weak dependencies: 2024-03-21T09:53:34.292 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-03-21T09:53:34.292 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T09:53:34.292 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction Summary 2024-03-21T09:53:34.292 INFO:teuthology.orchestra.run.smithi153.stdout:================================================================================ 2024-03-21T09:53:34.292 INFO:teuthology.orchestra.run.smithi153.stdout:Install 7 Packages 2024-03-21T09:53:34.292 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T09:53:34.293 INFO:teuthology.orchestra.run.smithi153.stdout:Total download size: 2.1 M 2024-03-21T09:53:34.293 INFO:teuthology.orchestra.run.smithi153.stdout:Installed size: 8.5 M 2024-03-21T09:53:34.293 INFO:teuthology.orchestra.run.smithi153.stdout:Downloading Packages: 2024-03-21T09:53:34.702 INFO:teuthology.orchestra.run.smithi153.stdout:(1/7): ceph-grafana-dashboards-18.2.1-0.el9.noa 111 kB/s | 23 kB 00:00 2024-03-21T09:53:34.735 INFO:teuthology.orchestra.run.smithi153.stdout:(2/7): ceph-prometheus-alerts-18.2.1-0.el9.noar 60 kB/s | 15 kB 00:00 2024-03-21T09:53:34.802 INFO:teuthology.orchestra.run.smithi153.stdout:(3/7): ceph-mgr-dashboard-18.2.1-0.el9.noarch.r 5.7 MB/s | 1.8 MB 00:00 2024-03-21T09:53:34.903 INFO:teuthology.orchestra.run.smithi153.stdout:(4/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 45 kB/s | 9.0 kB 00:00 2024-03-21T09:53:34.951 INFO:teuthology.orchestra.run.smithi174.stdout:Last metadata expiration check: 0:00:56 ago on Thu 21 Mar 2024 09:52:38 AM UTC. 2024-03-21T09:53:35.044 INFO:teuthology.orchestra.run.smithi153.stdout:(5/7): python3-jwt-2.4.0-1.el9.noarch.rpm 133 kB/s | 41 kB 00:00 2024-03-21T09:53:35.062 INFO:teuthology.orchestra.run.smithi153.stdout:(6/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 119 kB/s | 31 kB 00:00 2024-03-21T09:53:35.101 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2024-03-21T09:53:35.102 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2024-03-21T09:53:35.102 INFO:teuthology.orchestra.run.smithi174.stdout: Package Architecture Version Repository Size 2024-03-21T09:53:35.102 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2024-03-21T09:53:35.102 INFO:teuthology.orchestra.run.smithi174.stdout:Installing: 2024-03-21T09:53:35.102 INFO:teuthology.orchestra.run.smithi174.stdout: cephadm noarch 2:18.2.1-0.el9 ceph-noarch 221 k 2024-03-21T09:53:35.102 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-21T09:53:35.102 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction Summary 2024-03-21T09:53:35.102 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2024-03-21T09:53:35.102 INFO:teuthology.orchestra.run.smithi174.stdout:Install 1 Package 2024-03-21T09:53:35.102 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-21T09:53:35.102 INFO:teuthology.orchestra.run.smithi174.stdout:Total download size: 221 k 2024-03-21T09:53:35.102 INFO:teuthology.orchestra.run.smithi174.stdout:Installed size: 213 k 2024-03-21T09:53:35.102 INFO:teuthology.orchestra.run.smithi174.stdout:Downloading Packages: 2024-03-21T09:53:35.188 INFO:teuthology.orchestra.run.smithi174.stdout:cephadm-18.2.1-0.el9.noarch.rpm 2.5 MB/s | 221 kB 00:00 2024-03-21T09:53:35.188 INFO:teuthology.orchestra.run.smithi174.stdout:-------------------------------------------------------------------------------- 2024-03-21T09:53:35.189 INFO:teuthology.orchestra.run.smithi174.stdout:Total 2.5 MB/s | 221 kB 00:00 2024-03-21T09:53:35.189 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction check 2024-03-21T09:53:35.195 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction check succeeded. 2024-03-21T09:53:35.195 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction test 2024-03-21T09:53:35.198 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction test succeeded. 2024-03-21T09:53:35.198 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction 2024-03-21T09:53:35.261 INFO:teuthology.orchestra.run.smithi153.stdout:(7/7): python3-routes-2.5.1-5.el9.noarch.rpm 527 kB/s | 188 kB 00:00 2024-03-21T09:53:35.266 INFO:teuthology.orchestra.run.smithi153.stdout:-------------------------------------------------------------------------------- 2024-03-21T09:53:35.267 INFO:teuthology.orchestra.run.smithi153.stdout:Total 2.1 MB/s | 2.1 MB 00:00 2024-03-21T09:53:35.267 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction check 2024-03-21T09:53:35.288 INFO:teuthology.orchestra.run.smithi174.stdout: Preparing : 1/1 2024-03-21T09:53:35.293 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction check succeeded. 2024-03-21T09:53:35.293 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction test 2024-03-21T09:53:35.431 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction test succeeded. 2024-03-21T09:53:35.431 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction 2024-03-21T09:53:35.812 INFO:teuthology.orchestra.run.smithi153.stdout: Preparing : 1/1 2024-03-21T09:53:35.879 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-03-21T09:53:35.927 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-03-21T09:53:35.942 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: cephadm-2:18.2.1-0.el9.noarch 1/1 2024-03-21T09:53:36.011 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-03-21T09:53:36.053 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-03-21T09:53:36.093 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : cephadm-2:18.2.1-0.el9.noarch 1/1 2024-03-21T09:53:36.100 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-prometheus-alerts-2:18.2.1-0.el9.noarch 5/7 2024-03-21T09:53:36.423 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-grafana-dashboards-2:18.2.1-0.el9.noarch 6/7 2024-03-21T09:53:36.457 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-mgr-dashboard-2:18.2.1-0.el9.noarch 7/7 2024-03-21T09:53:36.590 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: cephadm-2:18.2.1-0.el9.noarch 1/1 2024-03-21T09:53:36.932 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : cephadm-2:18.2.1-0.el9.noarch 1/1 2024-03-21T09:53:36.932 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-21T09:53:36.932 INFO:teuthology.orchestra.run.smithi174.stdout:Installed: 2024-03-21T09:53:36.932 INFO:teuthology.orchestra.run.smithi174.stdout: cephadm-2:18.2.1-0.el9.noarch 2024-03-21T09:53:36.932 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-21T09:53:36.932 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2024-03-21T09:53:37.100 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.1-0.el9.noarch 7/7 2024-03-21T09:53:37.100 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-grafana-dashboards-2:18.2.1-0.el9.noarch 1/7 2024-03-21T09:53:37.101 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-mgr-dashboard-2:18.2.1-0.el9.noarch 2/7 2024-03-21T09:53:37.101 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-prometheus-alerts-2:18.2.1-0.el9.noarch 3/7 2024-03-21T09:53:37.101 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-03-21T09:53:37.101 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-03-21T09:53:37.101 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-03-21T09:53:37.198 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum -y install ceph-immutable-object-cache 2024-03-21T09:53:37.522 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-03-21T09:53:37.522 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T09:53:37.522 INFO:teuthology.orchestra.run.smithi153.stdout:Installed: 2024-03-21T09:53:37.522 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-grafana-dashboards-2:18.2.1-0.el9.noarch 2024-03-21T09:53:37.523 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-dashboard-2:18.2.1-0.el9.noarch 2024-03-21T09:53:37.523 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-prometheus-alerts-2:18.2.1-0.el9.noarch 2024-03-21T09:53:37.523 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-03-21T09:53:37.523 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-03-21T09:53:37.523 INFO:teuthology.orchestra.run.smithi153.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-21T09:53:37.523 INFO:teuthology.orchestra.run.smithi153.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-21T09:53:37.523 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T09:53:37.523 INFO:teuthology.orchestra.run.smithi153.stdout:Complete! 2024-03-21T09:53:37.784 DEBUG:teuthology.orchestra.run.smithi153:> sudo yum -y install ceph-mgr-diskprediction-local 2024-03-21T09:53:38.045 INFO:teuthology.orchestra.run.smithi174.stdout:Last metadata expiration check: 0:01:00 ago on Thu 21 Mar 2024 09:52:38 AM UTC. 2024-03-21T09:53:38.202 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2024-03-21T09:53:38.202 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2024-03-21T09:53:38.202 INFO:teuthology.orchestra.run.smithi174.stdout: Package Arch Version Repo Size 2024-03-21T09:53:38.202 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2024-03-21T09:53:38.202 INFO:teuthology.orchestra.run.smithi174.stdout:Installing: 2024-03-21T09:53:38.202 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-immutable-object-cache x86_64 2:18.2.1-0.el9 ceph 142 k 2024-03-21T09:53:38.202 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-21T09:53:38.202 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction Summary 2024-03-21T09:53:38.203 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2024-03-21T09:53:38.203 INFO:teuthology.orchestra.run.smithi174.stdout:Install 1 Package 2024-03-21T09:53:38.203 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-21T09:53:38.203 INFO:teuthology.orchestra.run.smithi174.stdout:Total download size: 142 k 2024-03-21T09:53:38.203 INFO:teuthology.orchestra.run.smithi174.stdout:Installed size: 395 k 2024-03-21T09:53:38.203 INFO:teuthology.orchestra.run.smithi174.stdout:Downloading Packages: 2024-03-21T09:53:38.277 INFO:teuthology.orchestra.run.smithi174.stdout:ceph-immutable-object-cache-18.2.1-0.el9.x86_64 1.9 MB/s | 142 kB 00:00 2024-03-21T09:53:38.277 INFO:teuthology.orchestra.run.smithi174.stdout:-------------------------------------------------------------------------------- 2024-03-21T09:53:38.277 INFO:teuthology.orchestra.run.smithi174.stdout:Total 1.9 MB/s | 142 kB 00:00 2024-03-21T09:53:38.278 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction check 2024-03-21T09:53:38.286 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction check succeeded. 2024-03-21T09:53:38.287 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction test 2024-03-21T09:53:38.344 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction test succeeded. 2024-03-21T09:53:38.345 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction 2024-03-21T09:53:38.514 INFO:teuthology.orchestra.run.smithi174.stdout: Preparing : 1/1 2024-03-21T09:53:38.595 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-immutable-object-cache-2:18.2.1-0.el9.x86_64 1/1 2024-03-21T09:53:38.615 INFO:teuthology.orchestra.run.smithi153.stdout:Last metadata expiration check: 0:01:09 ago on Thu 21 Mar 2024 09:52:29 AM UTC. 2024-03-21T09:53:38.670 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.1-0.el9.x86_64 1/1 2024-03-21T09:53:38.670 INFO:teuthology.orchestra.run.smithi174.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T09:53:38.670 INFO:teuthology.orchestra.run.smithi174.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-21T09:53:38.670 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-21T09:53:38.775 INFO:teuthology.orchestra.run.smithi153.stdout:Dependencies resolved. 2024-03-21T09:53:38.776 INFO:teuthology.orchestra.run.smithi153.stdout:================================================================================ 2024-03-21T09:53:38.776 INFO:teuthology.orchestra.run.smithi153.stdout: Package Arch Version Repository Size 2024-03-21T09:53:38.776 INFO:teuthology.orchestra.run.smithi153.stdout:================================================================================ 2024-03-21T09:53:38.776 INFO:teuthology.orchestra.run.smithi153.stdout:Installing: 2024-03-21T09:53:38.776 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.1-0.el9 ceph-noarch 7.4 M 2024-03-21T09:53:38.776 INFO:teuthology.orchestra.run.smithi153.stdout:Installing dependencies: 2024-03-21T09:53:38.776 INFO:teuthology.orchestra.run.smithi153.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-21T09:53:38.776 INFO:teuthology.orchestra.run.smithi153.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-21T09:53:38.777 INFO:teuthology.orchestra.run.smithi153.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-21T09:53:38.777 INFO:teuthology.orchestra.run.smithi153.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-21T09:53:38.777 INFO:teuthology.orchestra.run.smithi153.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-21T09:53:38.777 INFO:teuthology.orchestra.run.smithi153.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-21T09:53:38.777 INFO:teuthology.orchestra.run.smithi153.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-21T09:53:38.777 INFO:teuthology.orchestra.run.smithi153.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-21T09:53:38.777 INFO:teuthology.orchestra.run.smithi153.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-21T09:53:38.777 INFO:teuthology.orchestra.run.smithi153.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-21T09:53:38.777 INFO:teuthology.orchestra.run.smithi153.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-21T09:53:38.777 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T09:53:38.777 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction Summary 2024-03-21T09:53:38.777 INFO:teuthology.orchestra.run.smithi153.stdout:================================================================================ 2024-03-21T09:53:38.777 INFO:teuthology.orchestra.run.smithi153.stdout:Install 12 Packages 2024-03-21T09:53:38.777 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T09:53:38.778 INFO:teuthology.orchestra.run.smithi153.stdout:Total download size: 37 M 2024-03-21T09:53:38.778 INFO:teuthology.orchestra.run.smithi153.stdout:Installed size: 209 M 2024-03-21T09:53:38.778 INFO:teuthology.orchestra.run.smithi153.stdout:Downloading Packages: 2024-03-21T09:53:39.185 INFO:teuthology.orchestra.run.smithi153.stdout:(1/12): flexiblas-3.0.4-8.el9.x86_64.rpm 171 kB/s | 32 kB 00:00 2024-03-21T09:53:39.227 INFO:teuthology.orchestra.run.smithi153.stdout:(2/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 420 kB/s | 17 kB 00:00 2024-03-21T09:53:39.302 INFO:teuthology.orchestra.run.smithi153.stdout:(3/12): openblas-0.3.21-2.el9.x86_64.rpm 463 kB/s | 34 kB 00:00 2024-03-21T09:53:39.561 INFO:teuthology.orchestra.run.smithi153.stdout:(4/12): ceph-mgr-diskprediction-local-18.2.1-0. 13 MB/s | 7.4 MB 00:00 2024-03-21T09:53:39.967 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-immutable-object-cache-2:18.2.1-0.el9.x86_64 1/1 2024-03-21T09:53:39.967 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-21T09:53:39.967 INFO:teuthology.orchestra.run.smithi174.stdout:Installed: 2024-03-21T09:53:39.967 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-immutable-object-cache-2:18.2.1-0.el9.x86_64 2024-03-21T09:53:39.967 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-21T09:53:39.967 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2024-03-21T09:53:40.011 INFO:teuthology.orchestra.run.smithi153.stdout:(5/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 3.0 MB/s | 3.0 MB 00:01 2024-03-21T09:53:40.278 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum -y install ceph-mgr 2024-03-21T09:53:40.378 INFO:teuthology.orchestra.run.smithi153.stdout:(6/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 598 kB/s | 217 kB 00:00 2024-03-21T09:53:40.695 INFO:teuthology.orchestra.run.smithi153.stdout:(7/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 3.4 MB/s | 4.7 MB 00:01 2024-03-21T09:53:40.878 INFO:teuthology.orchestra.run.smithi153.stdout:(8/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 3.7 MB/s | 4.8 MB 00:01 2024-03-21T09:53:41.078 INFO:teuthology.orchestra.run.smithi153.stdout:(9/12): libquadmath-11.4.1-3.el9.x86_64.rpm 1.0 MB/s | 195 kB 00:00 2024-03-21T09:53:41.113 INFO:teuthology.orchestra.run.smithi174.stdout:Last metadata expiration check: 0:01:03 ago on Thu 21 Mar 2024 09:52:38 AM UTC. 2024-03-21T09:53:41.210 INFO:teuthology.orchestra.run.smithi174.stdout:Package ceph-mgr-2:18.2.1-0.el9.x86_64 is already installed. 2024-03-21T09:53:41.265 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2024-03-21T09:53:41.266 INFO:teuthology.orchestra.run.smithi174.stdout:Nothing to do. 2024-03-21T09:53:41.266 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2024-03-21T09:53:41.295 INFO:teuthology.orchestra.run.smithi153.stdout:(10/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 17 MB/s | 16 MB 00:00 2024-03-21T09:53:41.329 INFO:teuthology.orchestra.run.smithi153.stdout:(11/12): libgfortran-11.4.1-3.el9.x86_64.rpm 1.3 MB/s | 803 kB 00:00 2024-03-21T09:53:41.338 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum -y install ceph-mgr-dashboard 2024-03-21T09:53:41.529 INFO:teuthology.orchestra.run.smithi153.stdout:(12/12): python3-devel-3.9.17-2.el9.x86_64.rpm 548 kB/s | 245 kB 00:00 2024-03-21T09:53:41.534 INFO:teuthology.orchestra.run.smithi153.stdout:-------------------------------------------------------------------------------- 2024-03-21T09:53:41.534 INFO:teuthology.orchestra.run.smithi153.stdout:Total 13 MB/s | 37 MB 00:02 2024-03-21T09:53:41.648 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction check 2024-03-21T09:53:41.711 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction check succeeded. 2024-03-21T09:53:41.711 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction test 2024-03-21T09:53:42.185 INFO:teuthology.orchestra.run.smithi174.stdout:Last metadata expiration check: 0:01:04 ago on Thu 21 Mar 2024 09:52:38 AM UTC. 2024-03-21T09:53:42.226 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction test succeeded. 2024-03-21T09:53:42.226 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction 2024-03-21T09:53:42.342 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2024-03-21T09:53:42.342 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2024-03-21T09:53:42.342 INFO:teuthology.orchestra.run.smithi174.stdout: Package Arch Version Repository Size 2024-03-21T09:53:42.342 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2024-03-21T09:53:42.343 INFO:teuthology.orchestra.run.smithi174.stdout:Installing: 2024-03-21T09:53:42.343 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-dashboard noarch 2:18.2.1-0.el9 ceph-noarch 1.8 M 2024-03-21T09:53:42.343 INFO:teuthology.orchestra.run.smithi174.stdout:Installing dependencies: 2024-03-21T09:53:42.343 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-grafana-dashboards noarch 2:18.2.1-0.el9 ceph-noarch 23 k 2024-03-21T09:53:42.343 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-prometheus-alerts noarch 2:18.2.1-0.el9 ceph-noarch 15 k 2024-03-21T09:53:42.343 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-03-21T09:53:42.343 INFO:teuthology.orchestra.run.smithi174.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-21T09:53:42.343 INFO:teuthology.orchestra.run.smithi174.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-21T09:53:42.343 INFO:teuthology.orchestra.run.smithi174.stdout:Installing weak dependencies: 2024-03-21T09:53:42.343 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-03-21T09:53:42.343 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-21T09:53:42.343 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction Summary 2024-03-21T09:53:42.343 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2024-03-21T09:53:42.343 INFO:teuthology.orchestra.run.smithi174.stdout:Install 7 Packages 2024-03-21T09:53:42.344 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-21T09:53:42.344 INFO:teuthology.orchestra.run.smithi174.stdout:Total download size: 2.1 M 2024-03-21T09:53:42.344 INFO:teuthology.orchestra.run.smithi174.stdout:Installed size: 8.5 M 2024-03-21T09:53:42.344 INFO:teuthology.orchestra.run.smithi174.stdout:Downloading Packages: 2024-03-21T09:53:42.706 INFO:teuthology.orchestra.run.smithi174.stdout:(1/7): ceph-grafana-dashboards-18.2.1-0.el9.noa 127 kB/s | 23 kB 00:00 2024-03-21T09:53:42.730 INFO:teuthology.orchestra.run.smithi174.stdout:(2/7): ceph-prometheus-alerts-18.2.1-0.el9.noar 70 kB/s | 15 kB 00:00 2024-03-21T09:53:42.798 INFO:teuthology.orchestra.run.smithi174.stdout:(3/7): ceph-mgr-dashboard-18.2.1-0.el9.noarch.r 6.4 MB/s | 1.8 MB 00:00 2024-03-21T09:53:42.906 INFO:teuthology.orchestra.run.smithi174.stdout:(4/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 45 kB/s | 9.0 kB 00:00 2024-03-21T09:53:43.031 INFO:teuthology.orchestra.run.smithi174.stdout:(5/7): python3-jwt-2.4.0-1.el9.noarch.rpm 136 kB/s | 41 kB 00:00 2024-03-21T09:53:43.057 INFO:teuthology.orchestra.run.smithi174.stdout:(6/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 119 kB/s | 31 kB 00:00 2024-03-21T09:53:43.066 INFO:teuthology.orchestra.run.smithi153.stdout: Preparing : 1/1 2024-03-21T09:53:43.142 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 1/12 2024-03-21T09:53:43.177 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-03-21T09:53:43.257 INFO:teuthology.orchestra.run.smithi174.stdout:(7/7): python3-routes-2.5.1-5.el9.noarch.rpm 539 kB/s | 188 kB 00:00 2024-03-21T09:53:43.261 INFO:teuthology.orchestra.run.smithi174.stdout:-------------------------------------------------------------------------------- 2024-03-21T09:53:43.262 INFO:teuthology.orchestra.run.smithi174.stdout:Total 2.2 MB/s | 2.1 MB 00:00 2024-03-21T09:53:43.263 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction check 2024-03-21T09:53:43.288 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction check succeeded. 2024-03-21T09:53:43.289 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction test 2024-03-21T09:53:43.322 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 3/12 2024-03-21T09:53:43.372 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 4/12 2024-03-21T09:53:43.425 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction test succeeded. 2024-03-21T09:53:43.425 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction 2024-03-21T09:53:43.623 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-03-21T09:53:43.661 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-03-21T09:53:43.772 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-03-21T09:53:43.854 INFO:teuthology.orchestra.run.smithi174.stdout: Preparing : 1/1 2024-03-21T09:53:43.917 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-03-21T09:53:43.963 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-03-21T09:53:44.045 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-03-21T09:53:44.082 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-03-21T09:53:44.125 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-prometheus-alerts-2:18.2.1-0.el9.noarch 5/7 2024-03-21T09:53:44.437 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-grafana-dashboards-2:18.2.1-0.el9.noarch 6/7 2024-03-21T09:53:44.468 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mgr-dashboard-2:18.2.1-0.el9.noarch 7/7 2024-03-21T09:53:44.603 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-03-21T09:53:44.744 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-03-21T09:53:45.065 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.1-0.el9.noarch 7/7 2024-03-21T09:53:45.065 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-grafana-dashboards-2:18.2.1-0.el9.noarch 1/7 2024-03-21T09:53:45.065 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mgr-dashboard-2:18.2.1-0.el9.noarch 2/7 2024-03-21T09:53:45.065 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-prometheus-alerts-2:18.2.1-0.el9.noarch 3/7 2024-03-21T09:53:45.065 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-03-21T09:53:45.065 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-03-21T09:53:45.065 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-03-21T09:53:45.433 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-03-21T09:53:45.433 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-21T09:53:45.433 INFO:teuthology.orchestra.run.smithi174.stdout:Installed: 2024-03-21T09:53:45.433 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-grafana-dashboards-2:18.2.1-0.el9.noarch 2024-03-21T09:53:45.433 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-dashboard-2:18.2.1-0.el9.noarch 2024-03-21T09:53:45.433 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-prometheus-alerts-2:18.2.1-0.el9.noarch 2024-03-21T09:53:45.433 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-03-21T09:53:45.434 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-03-21T09:53:45.434 INFO:teuthology.orchestra.run.smithi174.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-21T09:53:45.434 INFO:teuthology.orchestra.run.smithi174.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-21T09:53:45.434 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-21T09:53:45.434 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2024-03-21T09:53:45.735 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum -y install ceph-mgr-diskprediction-local 2024-03-21T09:53:46.311 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-03-21T09:53:46.565 INFO:teuthology.orchestra.run.smithi174.stdout:Last metadata expiration check: 0:01:08 ago on Thu 21 Mar 2024 09:52:38 AM UTC. 2024-03-21T09:53:46.724 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2024-03-21T09:53:46.726 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2024-03-21T09:53:46.726 INFO:teuthology.orchestra.run.smithi174.stdout: Package Arch Version Repository Size 2024-03-21T09:53:46.726 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2024-03-21T09:53:46.726 INFO:teuthology.orchestra.run.smithi174.stdout:Installing: 2024-03-21T09:53:46.726 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.1-0.el9 ceph-noarch 7.4 M 2024-03-21T09:53:46.726 INFO:teuthology.orchestra.run.smithi174.stdout:Installing dependencies: 2024-03-21T09:53:46.726 INFO:teuthology.orchestra.run.smithi174.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-21T09:53:46.726 INFO:teuthology.orchestra.run.smithi174.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-21T09:53:46.726 INFO:teuthology.orchestra.run.smithi174.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-21T09:53:46.727 INFO:teuthology.orchestra.run.smithi174.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-21T09:53:46.727 INFO:teuthology.orchestra.run.smithi174.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-21T09:53:46.727 INFO:teuthology.orchestra.run.smithi174.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-21T09:53:46.727 INFO:teuthology.orchestra.run.smithi174.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-21T09:53:46.727 INFO:teuthology.orchestra.run.smithi174.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-21T09:53:46.727 INFO:teuthology.orchestra.run.smithi174.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-21T09:53:46.727 INFO:teuthology.orchestra.run.smithi174.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-21T09:53:46.727 INFO:teuthology.orchestra.run.smithi174.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-21T09:53:46.727 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-21T09:53:46.727 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction Summary 2024-03-21T09:53:46.727 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2024-03-21T09:53:46.727 INFO:teuthology.orchestra.run.smithi174.stdout:Install 12 Packages 2024-03-21T09:53:46.727 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-21T09:53:46.728 INFO:teuthology.orchestra.run.smithi174.stdout:Total download size: 37 M 2024-03-21T09:53:46.728 INFO:teuthology.orchestra.run.smithi174.stdout:Installed size: 209 M 2024-03-21T09:53:46.728 INFO:teuthology.orchestra.run.smithi174.stdout:Downloading Packages: 2024-03-21T09:53:47.022 INFO:teuthology.orchestra.run.smithi174.stdout:(1/12): flexiblas-3.0.4-8.el9.x86_64.rpm 188 kB/s | 32 kB 00:00 2024-03-21T09:53:47.055 INFO:teuthology.orchestra.run.smithi174.stdout:(2/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 526 kB/s | 17 kB 00:00 2024-03-21T09:53:47.156 INFO:teuthology.orchestra.run.smithi174.stdout:(3/12): ceph-mgr-diskprediction-local-18.2.1-0. 25 MB/s | 7.4 MB 00:00 2024-03-21T09:53:47.181 INFO:teuthology.orchestra.run.smithi174.stdout:(4/12): openblas-0.3.21-2.el9.x86_64.rpm 273 kB/s | 34 kB 00:00 2024-03-21T09:53:47.381 INFO:teuthology.orchestra.run.smithi174.stdout:(5/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 21 MB/s | 4.7 MB 00:00 2024-03-21T09:53:47.445 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-03-21T09:53:47.476 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.1-0.el9.noarc 12/12 2024-03-21T09:53:47.481 INFO:teuthology.orchestra.run.smithi174.stdout:(6/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 16 MB/s | 4.8 MB 00:00 2024-03-21T09:53:47.565 INFO:teuthology.orchestra.run.smithi174.stdout:(7/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 4.2 MB/s | 3.0 MB 00:00 2024-03-21T09:53:47.590 INFO:teuthology.orchestra.run.smithi174.stdout:(8/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 1.1 MB/s | 217 kB 00:00 2024-03-21T09:53:47.915 INFO:teuthology.orchestra.run.smithi174.stdout:(9/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 37 MB/s | 16 MB 00:00 2024-03-21T09:53:47.949 INFO:teuthology.orchestra.run.smithi174.stdout:(10/12): libquadmath-11.4.1-3.el9.x86_64.rpm 567 kB/s | 195 kB 00:00 2024-03-21T09:53:48.066 INFO:teuthology.orchestra.run.smithi174.stdout:(11/12): libgfortran-11.4.1-3.el9.x86_64.rpm 1.7 MB/s | 803 kB 00:00 2024-03-21T09:53:48.708 INFO:teuthology.orchestra.run.smithi174.stdout:(12/12): python3-devel-3.9.17-2.el9.x86_64.rpm 310 kB/s | 245 kB 00:00 2024-03-21T09:53:48.712 INFO:teuthology.orchestra.run.smithi174.stdout:-------------------------------------------------------------------------------- 2024-03-21T09:53:48.713 INFO:teuthology.orchestra.run.smithi174.stdout:Total 19 MB/s | 37 MB 00:01 2024-03-21T09:53:48.831 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction check 2024-03-21T09:53:48.895 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction check succeeded. 2024-03-21T09:53:48.895 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction test 2024-03-21T09:53:49.422 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction test succeeded. 2024-03-21T09:53:49.422 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction 2024-03-21T09:53:50.077 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.1-0.el9.noarc 12/12 2024-03-21T09:53:50.077 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.1-0.el9.noarc 1/12 2024-03-21T09:53:50.077 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 2/12 2024-03-21T09:53:50.077 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 3/12 2024-03-21T09:53:50.078 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 4/12 2024-03-21T09:53:50.078 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 5/12 2024-03-21T09:53:50.078 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-03-21T09:53:50.078 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 7/12 2024-03-21T09:53:50.078 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 8/12 2024-03-21T09:53:50.078 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 9/12 2024-03-21T09:53:50.078 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 10/12 2024-03-21T09:53:50.078 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 11/12 2024-03-21T09:53:50.291 INFO:teuthology.orchestra.run.smithi174.stdout: Preparing : 1/1 2024-03-21T09:53:50.372 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 1/12 2024-03-21T09:53:50.406 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-03-21T09:53:50.550 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 3/12 2024-03-21T09:53:50.571 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 12/12 2024-03-21T09:53:50.572 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T09:53:50.572 INFO:teuthology.orchestra.run.smithi153.stdout:Installed: 2024-03-21T09:53:50.572 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-diskprediction-local-2:18.2.1-0.el9.noarch 2024-03-21T09:53:50.572 INFO:teuthology.orchestra.run.smithi153.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-21T09:53:50.572 INFO:teuthology.orchestra.run.smithi153.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-21T09:53:50.572 INFO:teuthology.orchestra.run.smithi153.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-21T09:53:50.572 INFO:teuthology.orchestra.run.smithi153.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-21T09:53:50.572 INFO:teuthology.orchestra.run.smithi153.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-21T09:53:50.572 INFO:teuthology.orchestra.run.smithi153.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-21T09:53:50.572 INFO:teuthology.orchestra.run.smithi153.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-21T09:53:50.572 INFO:teuthology.orchestra.run.smithi153.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-21T09:53:50.573 INFO:teuthology.orchestra.run.smithi153.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-21T09:53:50.573 INFO:teuthology.orchestra.run.smithi153.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-21T09:53:50.573 INFO:teuthology.orchestra.run.smithi153.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-21T09:53:50.573 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T09:53:50.573 INFO:teuthology.orchestra.run.smithi153.stdout:Complete! 2024-03-21T09:53:50.591 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 4/12 2024-03-21T09:53:50.839 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-03-21T09:53:50.890 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-03-21T09:53:50.997 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-03-21T09:53:51.017 DEBUG:teuthology.orchestra.run.smithi153:> sudo yum -y install ceph-mgr-rook 2024-03-21T09:53:51.827 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-03-21T09:53:51.889 INFO:teuthology.orchestra.run.smithi153.stdout:Last metadata expiration check: 0:01:22 ago on Thu 21 Mar 2024 09:52:29 AM UTC. 2024-03-21T09:53:51.956 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-03-21T09:53:52.048 INFO:teuthology.orchestra.run.smithi153.stdout:Dependencies resolved. 2024-03-21T09:53:52.049 INFO:teuthology.orchestra.run.smithi153.stdout:================================================================================ 2024-03-21T09:53:52.049 INFO:teuthology.orchestra.run.smithi153.stdout: Package Arch Version Repository Size 2024-03-21T09:53:52.049 INFO:teuthology.orchestra.run.smithi153.stdout:================================================================================ 2024-03-21T09:53:52.050 INFO:teuthology.orchestra.run.smithi153.stdout:Installing: 2024-03-21T09:53:52.050 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-rook noarch 2:18.2.1-0.el9 ceph-noarch 50 k 2024-03-21T09:53:52.050 INFO:teuthology.orchestra.run.smithi153.stdout:Installing dependencies: 2024-03-21T09:53:52.050 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-21T09:53:52.050 INFO:teuthology.orchestra.run.smithi153.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-21T09:53:52.050 INFO:teuthology.orchestra.run.smithi153.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-03-21T09:53:52.050 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-21T09:53:52.050 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-21T09:53:52.050 INFO:teuthology.orchestra.run.smithi153.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-21T09:53:52.050 INFO:teuthology.orchestra.run.smithi153.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-21T09:53:52.050 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-21T09:53:52.050 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-21T09:53:52.051 INFO:teuthology.orchestra.run.smithi153.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-21T09:53:52.051 INFO:teuthology.orchestra.run.smithi153.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-21T09:53:52.051 INFO:teuthology.orchestra.run.smithi153.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-21T09:53:52.051 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T09:53:52.051 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction Summary 2024-03-21T09:53:52.051 INFO:teuthology.orchestra.run.smithi153.stdout:================================================================================ 2024-03-21T09:53:52.051 INFO:teuthology.orchestra.run.smithi153.stdout:Install 13 Packages 2024-03-21T09:53:52.051 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T09:53:52.052 INFO:teuthology.orchestra.run.smithi153.stdout:Total download size: 2.0 M 2024-03-21T09:53:52.052 INFO:teuthology.orchestra.run.smithi153.stdout:Installed size: 26 M 2024-03-21T09:53:52.052 INFO:teuthology.orchestra.run.smithi153.stdout:Downloading Packages: 2024-03-21T09:53:52.236 INFO:teuthology.orchestra.run.smithi153.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 198 kB/s | 24 kB 00:00 2024-03-21T09:53:52.261 INFO:teuthology.orchestra.run.smithi153.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 122 kB/s | 18 kB 00:00 2024-03-21T09:53:52.286 INFO:teuthology.orchestra.run.smithi153.stdout:(3/13): ceph-mgr-rook-18.2.1-0.el9.noarch.rpm 288 kB/s | 50 kB 00:00 2024-03-21T09:53:52.312 INFO:teuthology.orchestra.run.smithi153.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.3 MB/s | 174 kB 00:00 2024-03-21T09:53:52.337 INFO:teuthology.orchestra.run.smithi153.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 132 kB 00:00 2024-03-21T09:53:52.362 INFO:teuthology.orchestra.run.smithi153.stdout:(6/13): python3-pyasn1-modules-0.4.8-6.el9.noar 2.8 MB/s | 211 kB 00:00 2024-03-21T09:53:52.387 INFO:teuthology.orchestra.run.smithi153.stdout:(7/13): python3-requests-oauthlib-1.3.0-12.el9. 583 kB/s | 43 kB 00:00 2024-03-21T09:53:52.504 INFO:teuthology.orchestra.run.smithi153.stdout:(8/13): python3-certifi-2021.10.8-2.el9.noarch. 107 kB/s | 15 kB 00:00 2024-03-21T09:53:52.530 INFO:teuthology.orchestra.run.smithi153.stdout:(9/13): python3-cachetools-4.2.4-1.el9.noarch.r 194 kB/s | 32 kB 00:00 2024-03-21T09:53:52.649 INFO:teuthology.orchestra.run.smithi153.stdout:(10/13): python3-google-auth-2.28.2-1.el9.noarc 786 kB/s | 203 kB 00:00 2024-03-21T09:53:52.672 INFO:teuthology.orchestra.run.smithi153.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 418 kB/s | 59 kB 00:00 2024-03-21T09:53:52.747 INFO:teuthology.orchestra.run.smithi153.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 900 kB/s | 90 kB 00:00 2024-03-21T09:53:52.806 INFO:teuthology.orchestra.run.smithi153.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 3.4 MB/s | 1.0 MB 00:00 2024-03-21T09:53:52.810 INFO:teuthology.orchestra.run.smithi153.stdout:-------------------------------------------------------------------------------- 2024-03-21T09:53:52.811 INFO:teuthology.orchestra.run.smithi153.stdout:Total 2.7 MB/s | 2.0 MB 00:00 2024-03-21T09:53:52.812 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction check 2024-03-21T09:53:52.853 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction check succeeded. 2024-03-21T09:53:52.853 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction test 2024-03-21T09:53:53.005 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction test succeeded. 2024-03-21T09:53:53.006 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction 2024-03-21T09:53:53.482 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-03-21T09:53:53.663 INFO:teuthology.orchestra.run.smithi153.stdout: Preparing : 1/1 2024-03-21T09:53:53.877 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-03-21T09:53:53.941 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-03-21T09:53:54.014 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-03-21T09:53:54.052 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-03-21T09:53:54.099 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-03-21T09:53:54.225 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-03-21T09:53:54.365 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 7/13 2024-03-21T09:53:54.442 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-03-21T09:53:54.612 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-03-21T09:53:54.645 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.1-0.el9.noarc 12/12 2024-03-21T09:53:55.375 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-03-21T09:53:55.433 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-03-21T09:53:55.466 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-03-21T09:53:55.522 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-03-21T09:53:55.551 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-mgr-rook-2:18.2.1-0.el9.noarch 13/13 2024-03-21T09:53:56.520 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.1-0.el9.noarch 13/13 2024-03-21T09:53:56.521 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-mgr-rook-2:18.2.1-0.el9.noarch 1/13 2024-03-21T09:53:56.521 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-03-21T09:53:56.521 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-03-21T09:53:56.521 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-03-21T09:53:56.521 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-03-21T09:53:56.521 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-03-21T09:53:56.521 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-03-21T09:53:56.521 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-03-21T09:53:56.521 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-03-21T09:53:56.521 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 10/13 2024-03-21T09:53:56.521 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-03-21T09:53:56.521 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-03-21T09:53:57.007 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-03-21T09:53:57.007 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T09:53:57.007 INFO:teuthology.orchestra.run.smithi153.stdout:Installed: 2024-03-21T09:53:57.007 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-rook-2:18.2.1-0.el9.noarch 2024-03-21T09:53:57.007 INFO:teuthology.orchestra.run.smithi153.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-21T09:53:57.007 INFO:teuthology.orchestra.run.smithi153.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-21T09:53:57.007 INFO:teuthology.orchestra.run.smithi153.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-03-21T09:53:57.007 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-21T09:53:57.007 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-21T09:53:57.007 INFO:teuthology.orchestra.run.smithi153.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-21T09:53:57.007 INFO:teuthology.orchestra.run.smithi153.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-21T09:53:57.007 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-21T09:53:57.008 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-21T09:53:57.008 INFO:teuthology.orchestra.run.smithi153.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-21T09:53:57.008 INFO:teuthology.orchestra.run.smithi153.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-21T09:53:57.008 INFO:teuthology.orchestra.run.smithi153.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-21T09:53:57.008 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T09:53:57.008 INFO:teuthology.orchestra.run.smithi153.stdout:Complete! 2024-03-21T09:53:57.063 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.1-0.el9.noarc 12/12 2024-03-21T09:53:57.063 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.1-0.el9.noarc 1/12 2024-03-21T09:53:57.063 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 2/12 2024-03-21T09:53:57.063 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 3/12 2024-03-21T09:53:57.063 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 4/12 2024-03-21T09:53:57.063 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 5/12 2024-03-21T09:53:57.063 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-03-21T09:53:57.063 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 7/12 2024-03-21T09:53:57.063 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 8/12 2024-03-21T09:53:57.063 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 9/12 2024-03-21T09:53:57.063 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 10/12 2024-03-21T09:53:57.064 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 11/12 2024-03-21T09:53:57.353 DEBUG:teuthology.orchestra.run.smithi153:> sudo yum -y install ceph-mgr-cephadm 2024-03-21T09:53:57.488 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 12/12 2024-03-21T09:53:57.488 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-21T09:53:57.488 INFO:teuthology.orchestra.run.smithi174.stdout:Installed: 2024-03-21T09:53:57.488 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-diskprediction-local-2:18.2.1-0.el9.noarch 2024-03-21T09:53:57.488 INFO:teuthology.orchestra.run.smithi174.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-21T09:53:57.488 INFO:teuthology.orchestra.run.smithi174.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-21T09:53:57.488 INFO:teuthology.orchestra.run.smithi174.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-21T09:53:57.489 INFO:teuthology.orchestra.run.smithi174.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-21T09:53:57.489 INFO:teuthology.orchestra.run.smithi174.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-21T09:53:57.489 INFO:teuthology.orchestra.run.smithi174.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-21T09:53:57.489 INFO:teuthology.orchestra.run.smithi174.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-21T09:53:57.489 INFO:teuthology.orchestra.run.smithi174.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-21T09:53:57.489 INFO:teuthology.orchestra.run.smithi174.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-21T09:53:57.489 INFO:teuthology.orchestra.run.smithi174.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-21T09:53:57.489 INFO:teuthology.orchestra.run.smithi174.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-21T09:53:57.489 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-21T09:53:57.489 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2024-03-21T09:53:57.900 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum -y install ceph-mgr-rook 2024-03-21T09:53:58.193 INFO:teuthology.orchestra.run.smithi153.stdout:Last metadata expiration check: 0:01:29 ago on Thu 21 Mar 2024 09:52:29 AM UTC. 2024-03-21T09:53:58.354 INFO:teuthology.orchestra.run.smithi153.stdout:Dependencies resolved. 2024-03-21T09:53:58.355 INFO:teuthology.orchestra.run.smithi153.stdout:================================================================================ 2024-03-21T09:53:58.355 INFO:teuthology.orchestra.run.smithi153.stdout: Package Arch Version Repository Size 2024-03-21T09:53:58.355 INFO:teuthology.orchestra.run.smithi153.stdout:================================================================================ 2024-03-21T09:53:58.355 INFO:teuthology.orchestra.run.smithi153.stdout:Installing: 2024-03-21T09:53:58.355 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-cephadm noarch 2:18.2.1-0.el9 ceph-noarch 132 k 2024-03-21T09:53:58.355 INFO:teuthology.orchestra.run.smithi153.stdout:Installing dependencies: 2024-03-21T09:53:58.355 INFO:teuthology.orchestra.run.smithi153.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-03-21T09:53:58.355 INFO:teuthology.orchestra.run.smithi153.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-21T09:53:58.356 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-21T09:53:58.356 INFO:teuthology.orchestra.run.smithi153.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-21T09:53:58.356 INFO:teuthology.orchestra.run.smithi153.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-21T09:53:58.356 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T09:53:58.356 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction Summary 2024-03-21T09:53:58.356 INFO:teuthology.orchestra.run.smithi153.stdout:================================================================================ 2024-03-21T09:53:58.356 INFO:teuthology.orchestra.run.smithi153.stdout:Install 6 Packages 2024-03-21T09:53:58.356 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T09:53:58.357 INFO:teuthology.orchestra.run.smithi153.stdout:Total download size: 6.8 M 2024-03-21T09:53:58.357 INFO:teuthology.orchestra.run.smithi153.stdout:Installed size: 33 M 2024-03-21T09:53:58.357 INFO:teuthology.orchestra.run.smithi153.stdout:Downloading Packages: 2024-03-21T09:53:58.748 INFO:teuthology.orchestra.run.smithi174.stdout:Last metadata expiration check: 0:01:20 ago on Thu 21 Mar 2024 09:52:38 AM UTC. 2024-03-21T09:53:58.859 INFO:teuthology.orchestra.run.smithi153.stdout:(1/6): ceph-mgr-cephadm-18.2.1-0.el9.noarch.rpm 665 kB/s | 132 kB 00:00 2024-03-21T09:53:58.905 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2024-03-21T09:53:58.907 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2024-03-21T09:53:58.907 INFO:teuthology.orchestra.run.smithi174.stdout: Package Arch Version Repository Size 2024-03-21T09:53:58.907 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2024-03-21T09:53:58.907 INFO:teuthology.orchestra.run.smithi174.stdout:Installing: 2024-03-21T09:53:58.907 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-rook noarch 2:18.2.1-0.el9 ceph-noarch 50 k 2024-03-21T09:53:58.907 INFO:teuthology.orchestra.run.smithi174.stdout:Installing dependencies: 2024-03-21T09:53:58.908 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-21T09:53:58.908 INFO:teuthology.orchestra.run.smithi174.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-21T09:53:58.908 INFO:teuthology.orchestra.run.smithi174.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-03-21T09:53:58.908 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-21T09:53:58.908 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-21T09:53:58.909 INFO:teuthology.orchestra.run.smithi174.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-21T09:53:58.909 INFO:teuthology.orchestra.run.smithi174.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-21T09:53:58.909 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-21T09:53:58.909 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-21T09:53:58.909 INFO:teuthology.orchestra.run.smithi174.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-21T09:53:58.909 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-21T09:53:58.910 INFO:teuthology.orchestra.run.smithi174.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-21T09:53:58.910 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-21T09:53:58.910 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction Summary 2024-03-21T09:53:58.910 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2024-03-21T09:53:58.910 INFO:teuthology.orchestra.run.smithi174.stdout:Install 13 Packages 2024-03-21T09:53:58.911 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-21T09:53:58.912 INFO:teuthology.orchestra.run.smithi174.stdout:Total download size: 2.0 M 2024-03-21T09:53:58.912 INFO:teuthology.orchestra.run.smithi174.stdout:Installed size: 26 M 2024-03-21T09:53:58.912 INFO:teuthology.orchestra.run.smithi174.stdout:Downloading Packages: 2024-03-21T09:53:59.068 INFO:teuthology.orchestra.run.smithi153.stdout:(2/6): python3-babel-2.9.1-2.el9.noarch.rpm 14 MB/s | 5.8 MB 00:00 2024-03-21T09:53:59.154 INFO:teuthology.orchestra.run.smithi174.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 156 kB/s | 24 kB 00:00 2024-03-21T09:53:59.179 INFO:teuthology.orchestra.run.smithi174.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 99 kB/s | 18 kB 00:00 2024-03-21T09:53:59.205 INFO:teuthology.orchestra.run.smithi174.stdout:(3/13): ceph-mgr-rook-18.2.1-0.el9.noarch.rpm 242 kB/s | 50 kB 00:00 2024-03-21T09:53:59.255 INFO:teuthology.orchestra.run.smithi174.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.7 MB/s | 174 kB 00:00 2024-03-21T09:53:59.314 INFO:teuthology.orchestra.run.smithi174.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 991 kB/s | 132 kB 00:00 2024-03-21T09:53:59.318 INFO:teuthology.orchestra.run.smithi153.stdout:(3/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 378 kB/s | 248 kB 00:00 2024-03-21T09:53:59.339 INFO:teuthology.orchestra.run.smithi174.stdout:(6/13): python3-pyasn1-modules-0.4.8-6.el9.noar 1.5 MB/s | 211 kB 00:00 2024-03-21T09:53:59.364 INFO:teuthology.orchestra.run.smithi174.stdout:(7/13): python3-requests-oauthlib-1.3.0-12.el9. 406 kB/s | 43 kB 00:00 2024-03-21T09:53:59.411 INFO:teuthology.orchestra.run.smithi153.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 169 kB/s | 58 kB 00:00 2024-03-21T09:53:59.465 INFO:teuthology.orchestra.run.smithi174.stdout:(8/13): python3-certifi-2021.10.8-2.el9.noarch. 152 kB/s | 15 kB 00:00 2024-03-21T09:53:59.469 INFO:teuthology.orchestra.run.smithi153.stdout:(5/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 906 kB/s | 548 kB 00:00 2024-03-21T09:53:59.481 INFO:teuthology.orchestra.run.smithi174.stdout:(9/13): python3-cachetools-4.2.4-1.el9.noarch.r 194 kB/s | 32 kB 00:00 2024-03-21T09:53:59.582 INFO:teuthology.orchestra.run.smithi174.stdout:(10/13): python3-google-auth-2.28.2-1.el9.noarc 940 kB/s | 203 kB 00:00 2024-03-21T09:53:59.599 INFO:teuthology.orchestra.run.smithi174.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 508 kB/s | 59 kB 00:00 2024-03-21T09:53:59.665 INFO:teuthology.orchestra.run.smithi174.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.1 MB/s | 90 kB 00:00 2024-03-21T09:53:59.677 INFO:teuthology.orchestra.run.smithi153.stdout:(6/6): python3-typing-extensions-4.4.0-2.el9.no 142 kB/s | 51 kB 00:00 2024-03-21T09:53:59.680 INFO:teuthology.orchestra.run.smithi153.stdout:-------------------------------------------------------------------------------- 2024-03-21T09:53:59.681 INFO:teuthology.orchestra.run.smithi153.stdout:Total 5.1 MB/s | 6.8 MB 00:01 2024-03-21T09:53:59.721 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction check 2024-03-21T09:53:59.724 INFO:teuthology.orchestra.run.smithi174.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 3.9 MB/s | 1.0 MB 00:00 2024-03-21T09:53:59.728 INFO:teuthology.orchestra.run.smithi174.stdout:-------------------------------------------------------------------------------- 2024-03-21T09:53:59.730 INFO:teuthology.orchestra.run.smithi174.stdout:Total 2.5 MB/s | 2.0 MB 00:00 2024-03-21T09:53:59.731 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction check 2024-03-21T09:53:59.757 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction check succeeded. 2024-03-21T09:53:59.757 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction test 2024-03-21T09:53:59.771 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction check succeeded. 2024-03-21T09:53:59.771 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction test 2024-03-21T09:53:59.901 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction test succeeded. 2024-03-21T09:53:59.901 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction 2024-03-21T09:53:59.921 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction test succeeded. 2024-03-21T09:53:59.922 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction 2024-03-21T09:54:00.291 INFO:teuthology.orchestra.run.smithi153.stdout: Preparing : 1/1 2024-03-21T09:54:00.509 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-03-21T09:54:00.564 INFO:teuthology.orchestra.run.smithi174.stdout: Preparing : 1/1 2024-03-21T09:54:00.576 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-03-21T09:54:00.780 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-03-21T09:54:00.842 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-03-21T09:54:00.907 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-03-21T09:54:00.955 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-03-21T09:54:01.000 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-03-21T09:54:01.128 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-03-21T09:54:01.274 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 7/13 2024-03-21T09:54:01.280 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-03-21T09:54:01.343 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-03-21T09:54:01.379 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-03-21T09:54:01.491 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-03-21T09:54:01.524 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-mgr-cephadm-2:18.2.1-0.el9.noarch 6/6 2024-03-21T09:54:02.258 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-03-21T09:54:02.317 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-03-21T09:54:02.350 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-03-21T09:54:02.356 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.1-0.el9.noarch 6/6 2024-03-21T09:54:02.356 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-mgr-cephadm-2:18.2.1-0.el9.noarch 1/6 2024-03-21T09:54:02.356 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-03-21T09:54:02.356 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-03-21T09:54:02.356 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-03-21T09:54:02.356 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-03-21T09:54:02.431 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-03-21T09:54:02.463 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mgr-rook-2:18.2.1-0.el9.noarch 13/13 2024-03-21T09:54:02.745 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-03-21T09:54:02.746 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T09:54:02.746 INFO:teuthology.orchestra.run.smithi153.stdout:Installed: 2024-03-21T09:54:02.746 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-cephadm-2:18.2.1-0.el9.noarch 2024-03-21T09:54:02.746 INFO:teuthology.orchestra.run.smithi153.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-03-21T09:54:02.746 INFO:teuthology.orchestra.run.smithi153.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-21T09:54:02.746 INFO:teuthology.orchestra.run.smithi153.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-21T09:54:02.746 INFO:teuthology.orchestra.run.smithi153.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-21T09:54:02.746 INFO:teuthology.orchestra.run.smithi153.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-21T09:54:02.746 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T09:54:02.746 INFO:teuthology.orchestra.run.smithi153.stdout:Complete! 2024-03-21T09:54:03.046 DEBUG:teuthology.orchestra.run.smithi153:> sudo yum -y install ceph-fuse 2024-03-21T09:54:03.377 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.1-0.el9.noarch 13/13 2024-03-21T09:54:03.377 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mgr-rook-2:18.2.1-0.el9.noarch 1/13 2024-03-21T09:54:03.377 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-03-21T09:54:03.378 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-03-21T09:54:03.378 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-03-21T09:54:03.378 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-03-21T09:54:03.378 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-03-21T09:54:03.378 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-03-21T09:54:03.378 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-03-21T09:54:03.378 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-03-21T09:54:03.378 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 10/13 2024-03-21T09:54:03.378 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-03-21T09:54:03.378 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-03-21T09:54:03.830 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-03-21T09:54:03.830 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-21T09:54:03.831 INFO:teuthology.orchestra.run.smithi174.stdout:Installed: 2024-03-21T09:54:03.831 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-rook-2:18.2.1-0.el9.noarch 2024-03-21T09:54:03.831 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-21T09:54:03.831 INFO:teuthology.orchestra.run.smithi174.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-21T09:54:03.831 INFO:teuthology.orchestra.run.smithi174.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-03-21T09:54:03.831 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-21T09:54:03.831 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-21T09:54:03.831 INFO:teuthology.orchestra.run.smithi174.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-21T09:54:03.831 INFO:teuthology.orchestra.run.smithi174.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-21T09:54:03.831 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-21T09:54:03.831 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-21T09:54:03.831 INFO:teuthology.orchestra.run.smithi174.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-21T09:54:03.831 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-21T09:54:03.832 INFO:teuthology.orchestra.run.smithi174.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-21T09:54:03.832 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-21T09:54:03.832 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2024-03-21T09:54:03.850 INFO:teuthology.orchestra.run.smithi153.stdout:Last metadata expiration check: 0:01:34 ago on Thu 21 Mar 2024 09:52:29 AM UTC. 2024-03-21T09:54:04.009 INFO:teuthology.orchestra.run.smithi153.stdout:Dependencies resolved. 2024-03-21T09:54:04.010 INFO:teuthology.orchestra.run.smithi153.stdout:================================================================================ 2024-03-21T09:54:04.010 INFO:teuthology.orchestra.run.smithi153.stdout: Package Architecture Version Repository Size 2024-03-21T09:54:04.010 INFO:teuthology.orchestra.run.smithi153.stdout:================================================================================ 2024-03-21T09:54:04.010 INFO:teuthology.orchestra.run.smithi153.stdout:Installing: 2024-03-21T09:54:04.010 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-fuse x86_64 2:18.2.1-0.el9 ceph 839 k 2024-03-21T09:54:04.010 INFO:teuthology.orchestra.run.smithi153.stdout:Installing dependencies: 2024-03-21T09:54:04.010 INFO:teuthology.orchestra.run.smithi153.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-21T09:54:04.010 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T09:54:04.010 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction Summary 2024-03-21T09:54:04.010 INFO:teuthology.orchestra.run.smithi153.stdout:================================================================================ 2024-03-21T09:54:04.010 INFO:teuthology.orchestra.run.smithi153.stdout:Install 2 Packages 2024-03-21T09:54:04.010 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T09:54:04.011 INFO:teuthology.orchestra.run.smithi153.stdout:Total download size: 918 k 2024-03-21T09:54:04.011 INFO:teuthology.orchestra.run.smithi153.stdout:Installed size: 2.7 M 2024-03-21T09:54:04.011 INFO:teuthology.orchestra.run.smithi153.stdout:Downloading Packages: 2024-03-21T09:54:04.160 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum -y install ceph-mgr-cephadm 2024-03-21T09:54:04.229 INFO:teuthology.orchestra.run.smithi153.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 362 kB/s | 79 kB 00:00 2024-03-21T09:54:04.271 INFO:teuthology.orchestra.run.smithi153.stdout:(2/2): ceph-fuse-18.2.1-0.el9.x86_64.rpm 3.2 MB/s | 839 kB 00:00 2024-03-21T09:54:04.274 INFO:teuthology.orchestra.run.smithi153.stdout:-------------------------------------------------------------------------------- 2024-03-21T09:54:04.275 INFO:teuthology.orchestra.run.smithi153.stdout:Total 3.4 MB/s | 918 kB 00:00 2024-03-21T09:54:04.275 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction check 2024-03-21T09:54:04.344 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction check succeeded. 2024-03-21T09:54:04.344 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction test 2024-03-21T09:54:04.436 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction test succeeded. 2024-03-21T09:54:04.437 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction 2024-03-21T09:54:04.688 INFO:teuthology.orchestra.run.smithi153.stdout: Preparing : 1/1 2024-03-21T09:54:04.853 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-03-21T09:54:04.923 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : ceph-fuse-2:18.2.1-0.el9.x86_64 2/2 2024-03-21T09:54:05.044 INFO:teuthology.orchestra.run.smithi174.stdout:Last metadata expiration check: 0:01:27 ago on Thu 21 Mar 2024 09:52:38 AM UTC. 2024-03-21T09:54:05.206 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2024-03-21T09:54:05.206 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2024-03-21T09:54:05.207 INFO:teuthology.orchestra.run.smithi174.stdout: Package Arch Version Repository Size 2024-03-21T09:54:05.207 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2024-03-21T09:54:05.207 INFO:teuthology.orchestra.run.smithi174.stdout:Installing: 2024-03-21T09:54:05.207 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-cephadm noarch 2:18.2.1-0.el9 ceph-noarch 132 k 2024-03-21T09:54:05.207 INFO:teuthology.orchestra.run.smithi174.stdout:Installing dependencies: 2024-03-21T09:54:05.207 INFO:teuthology.orchestra.run.smithi174.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-03-21T09:54:05.207 INFO:teuthology.orchestra.run.smithi174.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-21T09:54:05.207 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-21T09:54:05.207 INFO:teuthology.orchestra.run.smithi174.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-21T09:54:05.207 INFO:teuthology.orchestra.run.smithi174.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-21T09:54:05.207 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-21T09:54:05.207 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction Summary 2024-03-21T09:54:05.207 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2024-03-21T09:54:05.207 INFO:teuthology.orchestra.run.smithi174.stdout:Install 6 Packages 2024-03-21T09:54:05.208 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-21T09:54:05.208 INFO:teuthology.orchestra.run.smithi174.stdout:Total download size: 6.8 M 2024-03-21T09:54:05.208 INFO:teuthology.orchestra.run.smithi174.stdout:Installed size: 33 M 2024-03-21T09:54:05.208 INFO:teuthology.orchestra.run.smithi174.stdout:Downloading Packages: 2024-03-21T09:54:05.710 INFO:teuthology.orchestra.run.smithi174.stdout:(1/6): ceph-mgr-cephadm-18.2.1-0.el9.noarch.rpm 652 kB/s | 132 kB 00:00 2024-03-21T09:54:05.897 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: ceph-fuse-2:18.2.1-0.el9.x86_64 2/2 2024-03-21T09:54:05.897 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : ceph-fuse-2:18.2.1-0.el9.x86_64 1/2 2024-03-21T09:54:05.960 INFO:teuthology.orchestra.run.smithi174.stdout:(2/6): python3-babel-2.9.1-2.el9.noarch.rpm 13 MB/s | 5.8 MB 00:00 2024-03-21T09:54:06.223 INFO:teuthology.orchestra.run.smithi174.stdout:(3/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 347 kB/s | 248 kB 00:00 2024-03-21T09:54:06.280 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-03-21T09:54:06.280 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T09:54:06.280 INFO:teuthology.orchestra.run.smithi153.stdout:Installed: 2024-03-21T09:54:06.281 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-fuse-2:18.2.1-0.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-03-21T09:54:06.281 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T09:54:06.281 INFO:teuthology.orchestra.run.smithi153.stdout:Complete! 2024-03-21T09:54:06.611 INFO:teuthology.orchestra.run.smithi174.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 89 kB/s | 58 kB 00:00 2024-03-21T09:54:06.652 DEBUG:teuthology.orchestra.run.smithi153:> sudo yum -y install librados-devel 2024-03-21T09:54:06.853 INFO:teuthology.orchestra.run.smithi174.stdout:(5/6): python3-typing-extensions-4.4.0-2.el9.no 82 kB/s | 51 kB 00:00 2024-03-21T09:54:07.111 INFO:teuthology.orchestra.run.smithi174.stdout:(6/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 392 kB/s | 548 kB 00:01 2024-03-21T09:54:07.119 INFO:teuthology.orchestra.run.smithi174.stdout:-------------------------------------------------------------------------------- 2024-03-21T09:54:07.120 INFO:teuthology.orchestra.run.smithi174.stdout:Total 3.6 MB/s | 6.8 MB 00:01 2024-03-21T09:54:07.160 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction check 2024-03-21T09:54:07.196 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction check succeeded. 2024-03-21T09:54:07.197 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction test 2024-03-21T09:54:07.339 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction test succeeded. 2024-03-21T09:54:07.339 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction 2024-03-21T09:54:07.524 INFO:teuthology.orchestra.run.smithi153.stdout:Last metadata expiration check: 0:01:38 ago on Thu 21 Mar 2024 09:52:29 AM UTC. 2024-03-21T09:54:07.681 INFO:teuthology.orchestra.run.smithi153.stdout:Dependencies resolved. 2024-03-21T09:54:07.682 INFO:teuthology.orchestra.run.smithi153.stdout:================================================================================ 2024-03-21T09:54:07.682 INFO:teuthology.orchestra.run.smithi153.stdout: Package Architecture Version Repository Size 2024-03-21T09:54:07.682 INFO:teuthology.orchestra.run.smithi153.stdout:================================================================================ 2024-03-21T09:54:07.683 INFO:teuthology.orchestra.run.smithi153.stdout:Installing: 2024-03-21T09:54:07.683 INFO:teuthology.orchestra.run.smithi153.stdout: librados-devel x86_64 2:18.2.1-0.el9 ceph 127 k 2024-03-21T09:54:07.683 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T09:54:07.683 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction Summary 2024-03-21T09:54:07.683 INFO:teuthology.orchestra.run.smithi153.stdout:================================================================================ 2024-03-21T09:54:07.683 INFO:teuthology.orchestra.run.smithi153.stdout:Install 1 Package 2024-03-21T09:54:07.683 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T09:54:07.683 INFO:teuthology.orchestra.run.smithi153.stdout:Total download size: 127 k 2024-03-21T09:54:07.683 INFO:teuthology.orchestra.run.smithi153.stdout:Installed size: 456 k 2024-03-21T09:54:07.683 INFO:teuthology.orchestra.run.smithi153.stdout:Downloading Packages: 2024-03-21T09:54:07.738 INFO:teuthology.orchestra.run.smithi174.stdout: Preparing : 1/1 2024-03-21T09:54:07.767 INFO:teuthology.orchestra.run.smithi153.stdout:librados-devel-18.2.1-0.el9.x86_64.rpm 1.5 MB/s | 127 kB 00:00 2024-03-21T09:54:07.768 INFO:teuthology.orchestra.run.smithi153.stdout:-------------------------------------------------------------------------------- 2024-03-21T09:54:07.768 INFO:teuthology.orchestra.run.smithi153.stdout:Total 1.4 MB/s | 127 kB 00:00 2024-03-21T09:54:07.768 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction check 2024-03-21T09:54:07.775 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction check succeeded. 2024-03-21T09:54:07.776 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction test 2024-03-21T09:54:07.842 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction test succeeded. 2024-03-21T09:54:07.842 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction 2024-03-21T09:54:07.941 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-03-21T09:54:08.002 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-03-21T09:54:08.002 INFO:teuthology.orchestra.run.smithi153.stdout: Preparing : 1/1 2024-03-21T09:54:08.132 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : librados-devel-2:18.2.1-0.el9.x86_64 1/1 2024-03-21T09:54:08.679 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-03-21T09:54:08.692 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: librados-devel-2:18.2.1-0.el9.x86_64 1/1 2024-03-21T09:54:08.767 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-03-21T09:54:08.840 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-03-21T09:54:08.866 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mgr-cephadm-2:18.2.1-0.el9.noarch 6/6 2024-03-21T09:54:08.974 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : librados-devel-2:18.2.1-0.el9.x86_64 1/1 2024-03-21T09:54:08.974 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T09:54:08.974 INFO:teuthology.orchestra.run.smithi153.stdout:Installed: 2024-03-21T09:54:08.974 INFO:teuthology.orchestra.run.smithi153.stdout: librados-devel-2:18.2.1-0.el9.x86_64 2024-03-21T09:54:08.974 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T09:54:08.974 INFO:teuthology.orchestra.run.smithi153.stdout:Complete! 2024-03-21T09:54:09.336 DEBUG:teuthology.orchestra.run.smithi153:> sudo yum -y install libcephfs2 2024-03-21T09:54:09.709 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.1-0.el9.noarch 6/6 2024-03-21T09:54:09.709 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mgr-cephadm-2:18.2.1-0.el9.noarch 1/6 2024-03-21T09:54:09.709 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-03-21T09:54:09.709 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-03-21T09:54:09.710 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-03-21T09:54:09.710 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-03-21T09:54:10.079 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-03-21T09:54:10.079 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-21T09:54:10.079 INFO:teuthology.orchestra.run.smithi174.stdout:Installed: 2024-03-21T09:54:10.079 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-cephadm-2:18.2.1-0.el9.noarch 2024-03-21T09:54:10.079 INFO:teuthology.orchestra.run.smithi174.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-03-21T09:54:10.079 INFO:teuthology.orchestra.run.smithi174.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-21T09:54:10.079 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-21T09:54:10.080 INFO:teuthology.orchestra.run.smithi174.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-21T09:54:10.080 INFO:teuthology.orchestra.run.smithi174.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-21T09:54:10.080 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-21T09:54:10.080 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2024-03-21T09:54:10.210 INFO:teuthology.orchestra.run.smithi153.stdout:Last metadata expiration check: 0:01:41 ago on Thu 21 Mar 2024 09:52:29 AM UTC. 2024-03-21T09:54:10.308 INFO:teuthology.orchestra.run.smithi153.stdout:Package libcephfs2-2:18.2.1-0.el9.x86_64 is already installed. 2024-03-21T09:54:10.366 INFO:teuthology.orchestra.run.smithi153.stdout:Dependencies resolved. 2024-03-21T09:54:10.367 INFO:teuthology.orchestra.run.smithi153.stdout:Nothing to do. 2024-03-21T09:54:10.367 INFO:teuthology.orchestra.run.smithi153.stdout:Complete! 2024-03-21T09:54:10.398 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum -y install ceph-fuse 2024-03-21T09:54:10.430 DEBUG:teuthology.orchestra.run.smithi153:> sudo yum -y install libcephfs-devel 2024-03-21T09:54:11.239 INFO:teuthology.orchestra.run.smithi174.stdout:Last metadata expiration check: 0:01:33 ago on Thu 21 Mar 2024 09:52:38 AM UTC. 2024-03-21T09:54:11.248 INFO:teuthology.orchestra.run.smithi153.stdout:Last metadata expiration check: 0:01:42 ago on Thu 21 Mar 2024 09:52:29 AM UTC. 2024-03-21T09:54:11.399 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2024-03-21T09:54:11.400 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2024-03-21T09:54:11.400 INFO:teuthology.orchestra.run.smithi174.stdout: Package Architecture Version Repository Size 2024-03-21T09:54:11.400 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2024-03-21T09:54:11.400 INFO:teuthology.orchestra.run.smithi174.stdout:Installing: 2024-03-21T09:54:11.400 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-fuse x86_64 2:18.2.1-0.el9 ceph 839 k 2024-03-21T09:54:11.400 INFO:teuthology.orchestra.run.smithi174.stdout:Installing dependencies: 2024-03-21T09:54:11.400 INFO:teuthology.orchestra.run.smithi174.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-21T09:54:11.400 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-21T09:54:11.400 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction Summary 2024-03-21T09:54:11.400 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2024-03-21T09:54:11.400 INFO:teuthology.orchestra.run.smithi174.stdout:Install 2 Packages 2024-03-21T09:54:11.401 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-21T09:54:11.401 INFO:teuthology.orchestra.run.smithi174.stdout:Total download size: 918 k 2024-03-21T09:54:11.401 INFO:teuthology.orchestra.run.smithi174.stdout:Installed size: 2.7 M 2024-03-21T09:54:11.401 INFO:teuthology.orchestra.run.smithi174.stdout:Downloading Packages: 2024-03-21T09:54:11.409 INFO:teuthology.orchestra.run.smithi153.stdout:Dependencies resolved. 2024-03-21T09:54:11.410 INFO:teuthology.orchestra.run.smithi153.stdout:================================================================================ 2024-03-21T09:54:11.410 INFO:teuthology.orchestra.run.smithi153.stdout: Package Architecture Version Repository Size 2024-03-21T09:54:11.410 INFO:teuthology.orchestra.run.smithi153.stdout:================================================================================ 2024-03-21T09:54:11.410 INFO:teuthology.orchestra.run.smithi153.stdout:Installing: 2024-03-21T09:54:11.410 INFO:teuthology.orchestra.run.smithi153.stdout: libcephfs-devel x86_64 2:18.2.1-0.el9 ceph 31 k 2024-03-21T09:54:11.410 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T09:54:11.410 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction Summary 2024-03-21T09:54:11.410 INFO:teuthology.orchestra.run.smithi153.stdout:================================================================================ 2024-03-21T09:54:11.410 INFO:teuthology.orchestra.run.smithi153.stdout:Install 1 Package 2024-03-21T09:54:11.410 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T09:54:11.411 INFO:teuthology.orchestra.run.smithi153.stdout:Total download size: 31 k 2024-03-21T09:54:11.411 INFO:teuthology.orchestra.run.smithi153.stdout:Installed size: 139 k 2024-03-21T09:54:11.411 INFO:teuthology.orchestra.run.smithi153.stdout:Downloading Packages: 2024-03-21T09:54:11.512 INFO:teuthology.orchestra.run.smithi153.stdout:libcephfs-devel-18.2.1-0.el9.x86_64.rpm 308 kB/s | 31 kB 00:00 2024-03-21T09:54:11.513 INFO:teuthology.orchestra.run.smithi153.stdout:-------------------------------------------------------------------------------- 2024-03-21T09:54:11.513 INFO:teuthology.orchestra.run.smithi153.stdout:Total 302 kB/s | 31 kB 00:00 2024-03-21T09:54:11.513 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction check 2024-03-21T09:54:11.517 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction check succeeded. 2024-03-21T09:54:11.517 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction test 2024-03-21T09:54:11.521 INFO:teuthology.orchestra.run.smithi174.stdout:(1/2): ceph-fuse-18.2.1-0.el9.x86_64.rpm 6.9 MB/s | 839 kB 00:00 2024-03-21T09:54:11.554 INFO:teuthology.orchestra.run.smithi174.stdout:(2/2): fuse-2.9.9-15.el9.x86_64.rpm 517 kB/s | 79 kB 00:00 2024-03-21T09:54:11.556 INFO:teuthology.orchestra.run.smithi174.stdout:-------------------------------------------------------------------------------- 2024-03-21T09:54:11.557 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction test succeeded. 2024-03-21T09:54:11.557 INFO:teuthology.orchestra.run.smithi174.stdout:Total 5.8 MB/s | 918 kB 00:00 2024-03-21T09:54:11.557 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction check 2024-03-21T09:54:11.557 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction 2024-03-21T09:54:11.599 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction check succeeded. 2024-03-21T09:54:11.599 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction test 2024-03-21T09:54:11.693 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction test succeeded. 2024-03-21T09:54:11.693 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction 2024-03-21T09:54:11.697 INFO:teuthology.orchestra.run.smithi153.stdout: Preparing : 1/1 2024-03-21T09:54:11.841 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libcephfs-devel-2:18.2.1-0.el9.x86_64 1/1 2024-03-21T09:54:11.932 INFO:teuthology.orchestra.run.smithi174.stdout: Preparing : 1/1 2024-03-21T09:54:12.102 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-03-21T09:54:12.174 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-fuse-2:18.2.1-0.el9.x86_64 2/2 2024-03-21T09:54:12.329 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: libcephfs-devel-2:18.2.1-0.el9.x86_64 1/1 2024-03-21T09:54:12.609 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libcephfs-devel-2:18.2.1-0.el9.x86_64 1/1 2024-03-21T09:54:12.609 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T09:54:12.609 INFO:teuthology.orchestra.run.smithi153.stdout:Installed: 2024-03-21T09:54:12.609 INFO:teuthology.orchestra.run.smithi153.stdout: libcephfs-devel-2:18.2.1-0.el9.x86_64 2024-03-21T09:54:12.609 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T09:54:12.609 INFO:teuthology.orchestra.run.smithi153.stdout:Complete! 2024-03-21T09:54:12.837 DEBUG:teuthology.orchestra.run.smithi153:> sudo yum -y install librados2 2024-03-21T09:54:13.068 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-fuse-2:18.2.1-0.el9.x86_64 2/2 2024-03-21T09:54:13.068 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-fuse-2:18.2.1-0.el9.x86_64 1/2 2024-03-21T09:54:13.406 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-03-21T09:54:13.407 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-21T09:54:13.407 INFO:teuthology.orchestra.run.smithi174.stdout:Installed: 2024-03-21T09:54:13.407 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-fuse-2:18.2.1-0.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-03-21T09:54:13.407 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-21T09:54:13.407 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2024-03-21T09:54:13.677 INFO:teuthology.orchestra.run.smithi153.stdout:Last metadata expiration check: 0:01:44 ago on Thu 21 Mar 2024 09:52:29 AM UTC. 2024-03-21T09:54:13.760 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum -y install librados-devel 2024-03-21T09:54:13.779 INFO:teuthology.orchestra.run.smithi153.stdout:Package librados2-2:18.2.1-0.el9.x86_64 is already installed. 2024-03-21T09:54:13.836 INFO:teuthology.orchestra.run.smithi153.stdout:Dependencies resolved. 2024-03-21T09:54:13.837 INFO:teuthology.orchestra.run.smithi153.stdout:Nothing to do. 2024-03-21T09:54:13.837 INFO:teuthology.orchestra.run.smithi153.stdout:Complete! 2024-03-21T09:54:13.902 DEBUG:teuthology.orchestra.run.smithi153:> sudo yum -y install librbd1 2024-03-21T09:54:14.616 INFO:teuthology.orchestra.run.smithi174.stdout:Last metadata expiration check: 0:01:36 ago on Thu 21 Mar 2024 09:52:38 AM UTC. 2024-03-21T09:54:14.732 INFO:teuthology.orchestra.run.smithi153.stdout:Last metadata expiration check: 0:01:45 ago on Thu 21 Mar 2024 09:52:29 AM UTC. 2024-03-21T09:54:14.776 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2024-03-21T09:54:14.778 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2024-03-21T09:54:14.778 INFO:teuthology.orchestra.run.smithi174.stdout: Package Architecture Version Repository Size 2024-03-21T09:54:14.778 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2024-03-21T09:54:14.778 INFO:teuthology.orchestra.run.smithi174.stdout:Installing: 2024-03-21T09:54:14.778 INFO:teuthology.orchestra.run.smithi174.stdout: librados-devel x86_64 2:18.2.1-0.el9 ceph 127 k 2024-03-21T09:54:14.778 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-21T09:54:14.778 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction Summary 2024-03-21T09:54:14.778 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2024-03-21T09:54:14.778 INFO:teuthology.orchestra.run.smithi174.stdout:Install 1 Package 2024-03-21T09:54:14.778 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-21T09:54:14.778 INFO:teuthology.orchestra.run.smithi174.stdout:Total download size: 127 k 2024-03-21T09:54:14.779 INFO:teuthology.orchestra.run.smithi174.stdout:Installed size: 456 k 2024-03-21T09:54:14.779 INFO:teuthology.orchestra.run.smithi174.stdout:Downloading Packages: 2024-03-21T09:54:14.835 INFO:teuthology.orchestra.run.smithi153.stdout:Package librbd1-2:18.2.1-0.el9.x86_64 is already installed. 2024-03-21T09:54:14.859 INFO:teuthology.orchestra.run.smithi174.stdout:librados-devel-18.2.1-0.el9.x86_64.rpm 1.6 MB/s | 127 kB 00:00 2024-03-21T09:54:14.859 INFO:teuthology.orchestra.run.smithi174.stdout:-------------------------------------------------------------------------------- 2024-03-21T09:54:14.860 INFO:teuthology.orchestra.run.smithi174.stdout:Total 1.5 MB/s | 127 kB 00:00 2024-03-21T09:54:14.860 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction check 2024-03-21T09:54:14.866 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction check succeeded. 2024-03-21T09:54:14.866 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction test 2024-03-21T09:54:14.892 INFO:teuthology.orchestra.run.smithi153.stdout:Dependencies resolved. 2024-03-21T09:54:14.893 INFO:teuthology.orchestra.run.smithi153.stdout:Nothing to do. 2024-03-21T09:54:14.893 INFO:teuthology.orchestra.run.smithi153.stdout:Complete! 2024-03-21T09:54:14.926 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction test succeeded. 2024-03-21T09:54:14.926 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction 2024-03-21T09:54:14.955 DEBUG:teuthology.orchestra.run.smithi153:> sudo yum -y install python3-rados 2024-03-21T09:54:15.084 INFO:teuthology.orchestra.run.smithi174.stdout: Preparing : 1/1 2024-03-21T09:54:15.192 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : librados-devel-2:18.2.1-0.el9.x86_64 1/1 2024-03-21T09:54:15.724 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: librados-devel-2:18.2.1-0.el9.x86_64 1/1 2024-03-21T09:54:15.783 INFO:teuthology.orchestra.run.smithi153.stdout:Last metadata expiration check: 0:01:46 ago on Thu 21 Mar 2024 09:52:29 AM UTC. 2024-03-21T09:54:15.880 INFO:teuthology.orchestra.run.smithi153.stdout:Package python3-rados-2:18.2.1-0.el9.x86_64 is already installed. 2024-03-21T09:54:15.937 INFO:teuthology.orchestra.run.smithi153.stdout:Dependencies resolved. 2024-03-21T09:54:15.939 INFO:teuthology.orchestra.run.smithi153.stdout:Nothing to do. 2024-03-21T09:54:15.939 INFO:teuthology.orchestra.run.smithi153.stdout:Complete! 2024-03-21T09:54:16.003 DEBUG:teuthology.orchestra.run.smithi153:> sudo yum -y install python3-rgw 2024-03-21T09:54:16.027 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librados-devel-2:18.2.1-0.el9.x86_64 1/1 2024-03-21T09:54:16.028 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-21T09:54:16.028 INFO:teuthology.orchestra.run.smithi174.stdout:Installed: 2024-03-21T09:54:16.028 INFO:teuthology.orchestra.run.smithi174.stdout: librados-devel-2:18.2.1-0.el9.x86_64 2024-03-21T09:54:16.028 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-21T09:54:16.028 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2024-03-21T09:54:16.350 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum -y install libcephfs2 2024-03-21T09:54:16.805 INFO:teuthology.orchestra.run.smithi153.stdout:Last metadata expiration check: 0:01:47 ago on Thu 21 Mar 2024 09:52:29 AM UTC. 2024-03-21T09:54:16.901 INFO:teuthology.orchestra.run.smithi153.stdout:Package python3-rgw-2:18.2.1-0.el9.x86_64 is already installed. 2024-03-21T09:54:16.959 INFO:teuthology.orchestra.run.smithi153.stdout:Dependencies resolved. 2024-03-21T09:54:16.960 INFO:teuthology.orchestra.run.smithi153.stdout:Nothing to do. 2024-03-21T09:54:16.960 INFO:teuthology.orchestra.run.smithi153.stdout:Complete! 2024-03-21T09:54:17.037 DEBUG:teuthology.orchestra.run.smithi153:> sudo yum -y install python3-cephfs 2024-03-21T09:54:17.186 INFO:teuthology.orchestra.run.smithi174.stdout:Last metadata expiration check: 0:01:39 ago on Thu 21 Mar 2024 09:52:38 AM UTC. 2024-03-21T09:54:17.283 INFO:teuthology.orchestra.run.smithi174.stdout:Package libcephfs2-2:18.2.1-0.el9.x86_64 is already installed. 2024-03-21T09:54:17.340 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2024-03-21T09:54:17.341 INFO:teuthology.orchestra.run.smithi174.stdout:Nothing to do. 2024-03-21T09:54:17.341 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2024-03-21T09:54:17.413 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum -y install libcephfs-devel 2024-03-21T09:54:17.873 INFO:teuthology.orchestra.run.smithi153.stdout:Last metadata expiration check: 0:01:48 ago on Thu 21 Mar 2024 09:52:29 AM UTC. 2024-03-21T09:54:17.982 INFO:teuthology.orchestra.run.smithi153.stdout:Package python3-cephfs-2:18.2.1-0.el9.x86_64 is already installed. 2024-03-21T09:54:18.040 INFO:teuthology.orchestra.run.smithi153.stdout:Dependencies resolved. 2024-03-21T09:54:18.041 INFO:teuthology.orchestra.run.smithi153.stdout:Nothing to do. 2024-03-21T09:54:18.041 INFO:teuthology.orchestra.run.smithi153.stdout:Complete! 2024-03-21T09:54:18.106 DEBUG:teuthology.orchestra.run.smithi153:> sudo yum -y install python3-rbd 2024-03-21T09:54:18.212 INFO:teuthology.orchestra.run.smithi174.stdout:Last metadata expiration check: 0:01:40 ago on Thu 21 Mar 2024 09:52:38 AM UTC. 2024-03-21T09:54:18.371 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2024-03-21T09:54:18.372 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2024-03-21T09:54:18.372 INFO:teuthology.orchestra.run.smithi174.stdout: Package Architecture Version Repository Size 2024-03-21T09:54:18.372 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2024-03-21T09:54:18.372 INFO:teuthology.orchestra.run.smithi174.stdout:Installing: 2024-03-21T09:54:18.372 INFO:teuthology.orchestra.run.smithi174.stdout: libcephfs-devel x86_64 2:18.2.1-0.el9 ceph 31 k 2024-03-21T09:54:18.372 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-21T09:54:18.372 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction Summary 2024-03-21T09:54:18.372 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2024-03-21T09:54:18.372 INFO:teuthology.orchestra.run.smithi174.stdout:Install 1 Package 2024-03-21T09:54:18.372 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-21T09:54:18.373 INFO:teuthology.orchestra.run.smithi174.stdout:Total download size: 31 k 2024-03-21T09:54:18.373 INFO:teuthology.orchestra.run.smithi174.stdout:Installed size: 139 k 2024-03-21T09:54:18.373 INFO:teuthology.orchestra.run.smithi174.stdout:Downloading Packages: 2024-03-21T09:54:18.462 INFO:teuthology.orchestra.run.smithi174.stdout:libcephfs-devel-18.2.1-0.el9.x86_64.rpm 348 kB/s | 31 kB 00:00 2024-03-21T09:54:18.463 INFO:teuthology.orchestra.run.smithi174.stdout:-------------------------------------------------------------------------------- 2024-03-21T09:54:18.463 INFO:teuthology.orchestra.run.smithi174.stdout:Total 342 kB/s | 31 kB 00:00 2024-03-21T09:54:18.463 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction check 2024-03-21T09:54:18.467 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction check succeeded. 2024-03-21T09:54:18.467 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction test 2024-03-21T09:54:18.497 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction test succeeded. 2024-03-21T09:54:18.498 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction 2024-03-21T09:54:18.627 INFO:teuthology.orchestra.run.smithi174.stdout: Preparing : 1/1 2024-03-21T09:54:18.789 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libcephfs-devel-2:18.2.1-0.el9.x86_64 1/1 2024-03-21T09:54:18.934 INFO:teuthology.orchestra.run.smithi153.stdout:Last metadata expiration check: 0:01:49 ago on Thu 21 Mar 2024 09:52:29 AM UTC. 2024-03-21T09:54:19.031 INFO:teuthology.orchestra.run.smithi153.stdout:Package python3-rbd-2:18.2.1-0.el9.x86_64 is already installed. 2024-03-21T09:54:19.088 INFO:teuthology.orchestra.run.smithi153.stdout:Dependencies resolved. 2024-03-21T09:54:19.089 INFO:teuthology.orchestra.run.smithi153.stdout:Nothing to do. 2024-03-21T09:54:19.089 INFO:teuthology.orchestra.run.smithi153.stdout:Complete! 2024-03-21T09:54:19.156 DEBUG:teuthology.orchestra.run.smithi153:> sudo yum -y install rbd-fuse 2024-03-21T09:54:19.245 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: libcephfs-devel-2:18.2.1-0.el9.x86_64 1/1 2024-03-21T09:54:19.515 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libcephfs-devel-2:18.2.1-0.el9.x86_64 1/1 2024-03-21T09:54:19.515 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-21T09:54:19.515 INFO:teuthology.orchestra.run.smithi174.stdout:Installed: 2024-03-21T09:54:19.516 INFO:teuthology.orchestra.run.smithi174.stdout: libcephfs-devel-2:18.2.1-0.el9.x86_64 2024-03-21T09:54:19.516 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-21T09:54:19.516 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2024-03-21T09:54:19.714 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum -y install librados2 2024-03-21T09:54:19.966 INFO:teuthology.orchestra.run.smithi153.stdout:Last metadata expiration check: 0:01:50 ago on Thu 21 Mar 2024 09:52:29 AM UTC. 2024-03-21T09:54:20.126 INFO:teuthology.orchestra.run.smithi153.stdout:Dependencies resolved. 2024-03-21T09:54:20.127 INFO:teuthology.orchestra.run.smithi153.stdout:================================================================================ 2024-03-21T09:54:20.127 INFO:teuthology.orchestra.run.smithi153.stdout: Package Architecture Version Repository Size 2024-03-21T09:54:20.127 INFO:teuthology.orchestra.run.smithi153.stdout:================================================================================ 2024-03-21T09:54:20.127 INFO:teuthology.orchestra.run.smithi153.stdout:Installing: 2024-03-21T09:54:20.127 INFO:teuthology.orchestra.run.smithi153.stdout: rbd-fuse x86_64 2:18.2.1-0.el9 ceph 86 k 2024-03-21T09:54:20.127 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T09:54:20.127 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction Summary 2024-03-21T09:54:20.127 INFO:teuthology.orchestra.run.smithi153.stdout:================================================================================ 2024-03-21T09:54:20.127 INFO:teuthology.orchestra.run.smithi153.stdout:Install 1 Package 2024-03-21T09:54:20.128 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T09:54:20.128 INFO:teuthology.orchestra.run.smithi153.stdout:Total download size: 86 k 2024-03-21T09:54:20.128 INFO:teuthology.orchestra.run.smithi153.stdout:Installed size: 226 k 2024-03-21T09:54:20.128 INFO:teuthology.orchestra.run.smithi153.stdout:Downloading Packages: 2024-03-21T09:54:20.215 INFO:teuthology.orchestra.run.smithi153.stdout:rbd-fuse-18.2.1-0.el9.x86_64.rpm 989 kB/s | 86 kB 00:00 2024-03-21T09:54:20.216 INFO:teuthology.orchestra.run.smithi153.stdout:-------------------------------------------------------------------------------- 2024-03-21T09:54:20.216 INFO:teuthology.orchestra.run.smithi153.stdout:Total 968 kB/s | 86 kB 00:00 2024-03-21T09:54:20.216 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction check 2024-03-21T09:54:20.225 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction check succeeded. 2024-03-21T09:54:20.225 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction test 2024-03-21T09:54:20.285 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction test succeeded. 2024-03-21T09:54:20.286 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction 2024-03-21T09:54:20.430 INFO:teuthology.orchestra.run.smithi153.stdout: Preparing : 1/1 2024-03-21T09:54:20.538 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : rbd-fuse-2:18.2.1-0.el9.x86_64 1/1 2024-03-21T09:54:20.575 INFO:teuthology.orchestra.run.smithi174.stdout:Last metadata expiration check: 0:01:42 ago on Thu 21 Mar 2024 09:52:38 AM UTC. 2024-03-21T09:54:20.679 INFO:teuthology.orchestra.run.smithi174.stdout:Package librados2-2:18.2.1-0.el9.x86_64 is already installed. 2024-03-21T09:54:20.736 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2024-03-21T09:54:20.737 INFO:teuthology.orchestra.run.smithi174.stdout:Nothing to do. 2024-03-21T09:54:20.737 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2024-03-21T09:54:20.811 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum -y install librbd1 2024-03-21T09:54:21.194 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: rbd-fuse-2:18.2.1-0.el9.x86_64 1/1 2024-03-21T09:54:21.521 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : rbd-fuse-2:18.2.1-0.el9.x86_64 1/1 2024-03-21T09:54:21.521 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T09:54:21.521 INFO:teuthology.orchestra.run.smithi153.stdout:Installed: 2024-03-21T09:54:21.521 INFO:teuthology.orchestra.run.smithi153.stdout: rbd-fuse-2:18.2.1-0.el9.x86_64 2024-03-21T09:54:21.521 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T09:54:21.521 INFO:teuthology.orchestra.run.smithi153.stdout:Complete! 2024-03-21T09:54:21.617 INFO:teuthology.orchestra.run.smithi174.stdout:Last metadata expiration check: 0:01:43 ago on Thu 21 Mar 2024 09:52:38 AM UTC. 2024-03-21T09:54:21.722 INFO:teuthology.orchestra.run.smithi174.stdout:Package librbd1-2:18.2.1-0.el9.x86_64 is already installed. 2024-03-21T09:54:21.779 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2024-03-21T09:54:21.781 INFO:teuthology.orchestra.run.smithi174.stdout:Nothing to do. 2024-03-21T09:54:21.781 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2024-03-21T09:54:21.802 DEBUG:teuthology.orchestra.run.smithi153:> sudo yum -y install rbd-mirror 2024-03-21T09:54:21.846 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum -y install python3-rados 2024-03-21T09:54:22.659 INFO:teuthology.orchestra.run.smithi153.stdout:Last metadata expiration check: 0:01:53 ago on Thu 21 Mar 2024 09:52:29 AM UTC. 2024-03-21T09:54:22.675 INFO:teuthology.orchestra.run.smithi174.stdout:Last metadata expiration check: 0:01:44 ago on Thu 21 Mar 2024 09:52:38 AM UTC. 2024-03-21T09:54:22.772 INFO:teuthology.orchestra.run.smithi174.stdout:Package python3-rados-2:18.2.1-0.el9.x86_64 is already installed. 2024-03-21T09:54:22.813 INFO:teuthology.orchestra.run.smithi153.stdout:Dependencies resolved. 2024-03-21T09:54:22.813 INFO:teuthology.orchestra.run.smithi153.stdout:================================================================================ 2024-03-21T09:54:22.813 INFO:teuthology.orchestra.run.smithi153.stdout: Package Architecture Version Repository Size 2024-03-21T09:54:22.813 INFO:teuthology.orchestra.run.smithi153.stdout:================================================================================ 2024-03-21T09:54:22.813 INFO:teuthology.orchestra.run.smithi153.stdout:Installing: 2024-03-21T09:54:22.813 INFO:teuthology.orchestra.run.smithi153.stdout: rbd-mirror x86_64 2:18.2.1-0.el9 ceph 3.0 M 2024-03-21T09:54:22.814 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T09:54:22.814 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction Summary 2024-03-21T09:54:22.814 INFO:teuthology.orchestra.run.smithi153.stdout:================================================================================ 2024-03-21T09:54:22.814 INFO:teuthology.orchestra.run.smithi153.stdout:Install 1 Package 2024-03-21T09:54:22.814 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T09:54:22.814 INFO:teuthology.orchestra.run.smithi153.stdout:Total download size: 3.0 M 2024-03-21T09:54:22.815 INFO:teuthology.orchestra.run.smithi153.stdout:Installed size: 12 M 2024-03-21T09:54:22.815 INFO:teuthology.orchestra.run.smithi153.stdout:Downloading Packages: 2024-03-21T09:54:22.830 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2024-03-21T09:54:22.831 INFO:teuthology.orchestra.run.smithi174.stdout:Nothing to do. 2024-03-21T09:54:22.831 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2024-03-21T09:54:22.918 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum -y install python3-rgw 2024-03-21T09:54:22.960 INFO:teuthology.orchestra.run.smithi153.stdout:rbd-mirror-18.2.1-0.el9.x86_64.rpm 21 MB/s | 3.0 MB 00:00 2024-03-21T09:54:22.961 INFO:teuthology.orchestra.run.smithi153.stdout:-------------------------------------------------------------------------------- 2024-03-21T09:54:22.962 INFO:teuthology.orchestra.run.smithi153.stdout:Total 20 MB/s | 3.0 MB 00:00 2024-03-21T09:54:22.962 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction check 2024-03-21T09:54:22.977 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction check succeeded. 2024-03-21T09:54:22.977 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction test 2024-03-21T09:54:23.067 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction test succeeded. 2024-03-21T09:54:23.068 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction 2024-03-21T09:54:23.520 INFO:teuthology.orchestra.run.smithi153.stdout: Preparing : 1/1 2024-03-21T09:54:23.586 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : rbd-mirror-2:18.2.1-0.el9.x86_64 1/1 2024-03-21T09:54:23.659 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: rbd-mirror-2:18.2.1-0.el9.x86_64 1/1 2024-03-21T09:54:23.659 INFO:teuthology.orchestra.run.smithi153.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T09:54:23.659 INFO:teuthology.orchestra.run.smithi153.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-21T09:54:23.659 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-21T09:54:23.660 INFO:teuthology.orchestra.run.smithi153.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-21T09:54:23.660 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T09:54:23.712 INFO:teuthology.orchestra.run.smithi174.stdout:Last metadata expiration check: 0:01:45 ago on Thu 21 Mar 2024 09:52:38 AM UTC. 2024-03-21T09:54:23.808 INFO:teuthology.orchestra.run.smithi174.stdout:Package python3-rgw-2:18.2.1-0.el9.x86_64 is already installed. 2024-03-21T09:54:23.865 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2024-03-21T09:54:23.866 INFO:teuthology.orchestra.run.smithi174.stdout:Nothing to do. 2024-03-21T09:54:23.866 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2024-03-21T09:54:23.944 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum -y install python3-cephfs 2024-03-21T09:54:24.741 INFO:teuthology.orchestra.run.smithi174.stdout:Last metadata expiration check: 0:01:46 ago on Thu 21 Mar 2024 09:52:38 AM UTC. 2024-03-21T09:54:24.844 INFO:teuthology.orchestra.run.smithi174.stdout:Package python3-cephfs-2:18.2.1-0.el9.x86_64 is already installed. 2024-03-21T09:54:24.899 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2024-03-21T09:54:24.900 INFO:teuthology.orchestra.run.smithi174.stdout:Nothing to do. 2024-03-21T09:54:24.900 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2024-03-21T09:54:24.974 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum -y install python3-rbd 2024-03-21T09:54:25.210 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : rbd-mirror-2:18.2.1-0.el9.x86_64 1/1 2024-03-21T09:54:25.210 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T09:54:25.210 INFO:teuthology.orchestra.run.smithi153.stdout:Installed: 2024-03-21T09:54:25.210 INFO:teuthology.orchestra.run.smithi153.stdout: rbd-mirror-2:18.2.1-0.el9.x86_64 2024-03-21T09:54:25.210 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T09:54:25.210 INFO:teuthology.orchestra.run.smithi153.stdout:Complete! 2024-03-21T09:54:25.531 DEBUG:teuthology.orchestra.run.smithi153:> sudo yum -y install rbd-nbd 2024-03-21T09:54:25.822 INFO:teuthology.orchestra.run.smithi174.stdout:Last metadata expiration check: 0:01:47 ago on Thu 21 Mar 2024 09:52:38 AM UTC. 2024-03-21T09:54:25.923 INFO:teuthology.orchestra.run.smithi174.stdout:Package python3-rbd-2:18.2.1-0.el9.x86_64 is already installed. 2024-03-21T09:54:25.983 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2024-03-21T09:54:25.984 INFO:teuthology.orchestra.run.smithi174.stdout:Nothing to do. 2024-03-21T09:54:25.984 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2024-03-21T09:54:26.053 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum -y install rbd-fuse 2024-03-21T09:54:26.415 INFO:teuthology.orchestra.run.smithi153.stdout:Last metadata expiration check: 0:01:57 ago on Thu 21 Mar 2024 09:52:29 AM UTC. 2024-03-21T09:54:26.575 INFO:teuthology.orchestra.run.smithi153.stdout:Dependencies resolved. 2024-03-21T09:54:26.576 INFO:teuthology.orchestra.run.smithi153.stdout:================================================================================ 2024-03-21T09:54:26.576 INFO:teuthology.orchestra.run.smithi153.stdout: Package Architecture Version Repository Size 2024-03-21T09:54:26.576 INFO:teuthology.orchestra.run.smithi153.stdout:================================================================================ 2024-03-21T09:54:26.576 INFO:teuthology.orchestra.run.smithi153.stdout:Installing: 2024-03-21T09:54:26.576 INFO:teuthology.orchestra.run.smithi153.stdout: rbd-nbd x86_64 2:18.2.1-0.el9 ceph 171 k 2024-03-21T09:54:26.576 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T09:54:26.576 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction Summary 2024-03-21T09:54:26.576 INFO:teuthology.orchestra.run.smithi153.stdout:================================================================================ 2024-03-21T09:54:26.576 INFO:teuthology.orchestra.run.smithi153.stdout:Install 1 Package 2024-03-21T09:54:26.576 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T09:54:26.576 INFO:teuthology.orchestra.run.smithi153.stdout:Total download size: 171 k 2024-03-21T09:54:26.576 INFO:teuthology.orchestra.run.smithi153.stdout:Installed size: 494 k 2024-03-21T09:54:26.576 INFO:teuthology.orchestra.run.smithi153.stdout:Downloading Packages: 2024-03-21T09:54:26.664 INFO:teuthology.orchestra.run.smithi153.stdout:rbd-nbd-18.2.1-0.el9.x86_64.rpm 1.9 MB/s | 171 kB 00:00 2024-03-21T09:54:26.665 INFO:teuthology.orchestra.run.smithi153.stdout:-------------------------------------------------------------------------------- 2024-03-21T09:54:26.666 INFO:teuthology.orchestra.run.smithi153.stdout:Total 1.9 MB/s | 171 kB 00:00 2024-03-21T09:54:26.666 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction check 2024-03-21T09:54:26.676 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction check succeeded. 2024-03-21T09:54:26.676 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction test 2024-03-21T09:54:26.744 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction test succeeded. 2024-03-21T09:54:26.745 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction 2024-03-21T09:54:26.856 INFO:teuthology.orchestra.run.smithi174.stdout:Last metadata expiration check: 0:01:48 ago on Thu 21 Mar 2024 09:52:38 AM UTC. 2024-03-21T09:54:26.910 INFO:teuthology.orchestra.run.smithi153.stdout: Preparing : 1/1 2024-03-21T09:54:27.016 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2024-03-21T09:54:27.017 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2024-03-21T09:54:27.017 INFO:teuthology.orchestra.run.smithi174.stdout: Package Architecture Version Repository Size 2024-03-21T09:54:27.017 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2024-03-21T09:54:27.018 INFO:teuthology.orchestra.run.smithi174.stdout:Installing: 2024-03-21T09:54:27.018 INFO:teuthology.orchestra.run.smithi174.stdout: rbd-fuse x86_64 2:18.2.1-0.el9 ceph 86 k 2024-03-21T09:54:27.018 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-21T09:54:27.018 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction Summary 2024-03-21T09:54:27.018 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2024-03-21T09:54:27.018 INFO:teuthology.orchestra.run.smithi174.stdout:Install 1 Package 2024-03-21T09:54:27.018 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-21T09:54:27.019 INFO:teuthology.orchestra.run.smithi174.stdout:Total download size: 86 k 2024-03-21T09:54:27.019 INFO:teuthology.orchestra.run.smithi174.stdout:Installed size: 226 k 2024-03-21T09:54:27.019 INFO:teuthology.orchestra.run.smithi174.stdout:Downloading Packages: 2024-03-21T09:54:27.031 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : rbd-nbd-2:18.2.1-0.el9.x86_64 1/1 2024-03-21T09:54:27.107 INFO:teuthology.orchestra.run.smithi174.stdout:rbd-fuse-18.2.1-0.el9.x86_64.rpm 977 kB/s | 86 kB 00:00 2024-03-21T09:54:27.108 INFO:teuthology.orchestra.run.smithi174.stdout:-------------------------------------------------------------------------------- 2024-03-21T09:54:27.108 INFO:teuthology.orchestra.run.smithi174.stdout:Total 956 kB/s | 86 kB 00:00 2024-03-21T09:54:27.108 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction check 2024-03-21T09:54:27.116 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction check succeeded. 2024-03-21T09:54:27.116 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction test 2024-03-21T09:54:27.178 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction test succeeded. 2024-03-21T09:54:27.178 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction 2024-03-21T09:54:27.342 INFO:teuthology.orchestra.run.smithi174.stdout: Preparing : 1/1 2024-03-21T09:54:27.467 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : rbd-fuse-2:18.2.1-0.el9.x86_64 1/1 2024-03-21T09:54:27.588 INFO:teuthology.orchestra.run.smithi153.stdout: Running scriptlet: rbd-nbd-2:18.2.1-0.el9.x86_64 1/1 2024-03-21T09:54:27.897 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : rbd-nbd-2:18.2.1-0.el9.x86_64 1/1 2024-03-21T09:54:27.897 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T09:54:27.897 INFO:teuthology.orchestra.run.smithi153.stdout:Installed: 2024-03-21T09:54:27.898 INFO:teuthology.orchestra.run.smithi153.stdout: rbd-nbd-2:18.2.1-0.el9.x86_64 2024-03-21T09:54:27.898 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T09:54:27.898 INFO:teuthology.orchestra.run.smithi153.stdout:Complete! 2024-03-21T09:54:27.972 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: rbd-fuse-2:18.2.1-0.el9.x86_64 1/1 2024-03-21T09:54:28.160 DEBUG:teuthology.parallel:result is None 2024-03-21T09:54:28.282 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : rbd-fuse-2:18.2.1-0.el9.x86_64 1/1 2024-03-21T09:54:28.283 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-21T09:54:28.283 INFO:teuthology.orchestra.run.smithi174.stdout:Installed: 2024-03-21T09:54:28.283 INFO:teuthology.orchestra.run.smithi174.stdout: rbd-fuse-2:18.2.1-0.el9.x86_64 2024-03-21T09:54:28.283 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-21T09:54:28.283 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2024-03-21T09:54:28.593 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum -y install rbd-mirror 2024-03-21T09:54:29.443 INFO:teuthology.orchestra.run.smithi174.stdout:Last metadata expiration check: 0:01:51 ago on Thu 21 Mar 2024 09:52:38 AM UTC. 2024-03-21T09:54:29.597 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2024-03-21T09:54:29.597 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2024-03-21T09:54:29.598 INFO:teuthology.orchestra.run.smithi174.stdout: Package Architecture Version Repository Size 2024-03-21T09:54:29.598 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2024-03-21T09:54:29.598 INFO:teuthology.orchestra.run.smithi174.stdout:Installing: 2024-03-21T09:54:29.598 INFO:teuthology.orchestra.run.smithi174.stdout: rbd-mirror x86_64 2:18.2.1-0.el9 ceph 3.0 M 2024-03-21T09:54:29.598 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-21T09:54:29.598 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction Summary 2024-03-21T09:54:29.598 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2024-03-21T09:54:29.598 INFO:teuthology.orchestra.run.smithi174.stdout:Install 1 Package 2024-03-21T09:54:29.598 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-21T09:54:29.599 INFO:teuthology.orchestra.run.smithi174.stdout:Total download size: 3.0 M 2024-03-21T09:54:29.599 INFO:teuthology.orchestra.run.smithi174.stdout:Installed size: 12 M 2024-03-21T09:54:29.599 INFO:teuthology.orchestra.run.smithi174.stdout:Downloading Packages: 2024-03-21T09:54:29.709 INFO:teuthology.orchestra.run.smithi174.stdout:rbd-mirror-18.2.1-0.el9.x86_64.rpm 27 MB/s | 3.0 MB 00:00 2024-03-21T09:54:29.710 INFO:teuthology.orchestra.run.smithi174.stdout:-------------------------------------------------------------------------------- 2024-03-21T09:54:29.710 INFO:teuthology.orchestra.run.smithi174.stdout:Total 27 MB/s | 3.0 MB 00:00 2024-03-21T09:54:29.710 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction check 2024-03-21T09:54:29.721 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction check succeeded. 2024-03-21T09:54:29.721 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction test 2024-03-21T09:54:29.798 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction test succeeded. 2024-03-21T09:54:29.799 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction 2024-03-21T09:54:30.253 INFO:teuthology.orchestra.run.smithi174.stdout: Preparing : 1/1 2024-03-21T09:54:30.318 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : rbd-mirror-2:18.2.1-0.el9.x86_64 1/1 2024-03-21T09:54:30.389 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: rbd-mirror-2:18.2.1-0.el9.x86_64 1/1 2024-03-21T09:54:30.389 INFO:teuthology.orchestra.run.smithi174.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T09:54:30.389 INFO:teuthology.orchestra.run.smithi174.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-21T09:54:30.389 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-21T09:54:30.389 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-21T09:54:30.389 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-21T09:54:31.758 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : rbd-mirror-2:18.2.1-0.el9.x86_64 1/1 2024-03-21T09:54:31.758 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-21T09:54:31.758 INFO:teuthology.orchestra.run.smithi174.stdout:Installed: 2024-03-21T09:54:31.758 INFO:teuthology.orchestra.run.smithi174.stdout: rbd-mirror-2:18.2.1-0.el9.x86_64 2024-03-21T09:54:31.759 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-21T09:54:31.759 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2024-03-21T09:54:32.038 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum -y install rbd-nbd 2024-03-21T09:54:32.888 INFO:teuthology.orchestra.run.smithi174.stdout:Last metadata expiration check: 0:01:54 ago on Thu 21 Mar 2024 09:52:38 AM UTC. 2024-03-21T09:54:33.049 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2024-03-21T09:54:33.050 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2024-03-21T09:54:33.050 INFO:teuthology.orchestra.run.smithi174.stdout: Package Architecture Version Repository Size 2024-03-21T09:54:33.050 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2024-03-21T09:54:33.050 INFO:teuthology.orchestra.run.smithi174.stdout:Installing: 2024-03-21T09:54:33.050 INFO:teuthology.orchestra.run.smithi174.stdout: rbd-nbd x86_64 2:18.2.1-0.el9 ceph 171 k 2024-03-21T09:54:33.050 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-21T09:54:33.050 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction Summary 2024-03-21T09:54:33.050 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2024-03-21T09:54:33.050 INFO:teuthology.orchestra.run.smithi174.stdout:Install 1 Package 2024-03-21T09:54:33.050 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-21T09:54:33.050 INFO:teuthology.orchestra.run.smithi174.stdout:Total download size: 171 k 2024-03-21T09:54:33.051 INFO:teuthology.orchestra.run.smithi174.stdout:Installed size: 494 k 2024-03-21T09:54:33.051 INFO:teuthology.orchestra.run.smithi174.stdout:Downloading Packages: 2024-03-21T09:54:33.121 INFO:teuthology.orchestra.run.smithi174.stdout:rbd-nbd-18.2.1-0.el9.x86_64.rpm 2.4 MB/s | 171 kB 00:00 2024-03-21T09:54:33.121 INFO:teuthology.orchestra.run.smithi174.stdout:-------------------------------------------------------------------------------- 2024-03-21T09:54:33.121 INFO:teuthology.orchestra.run.smithi174.stdout:Total 2.3 MB/s | 171 kB 00:00 2024-03-21T09:54:33.122 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction check 2024-03-21T09:54:33.131 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction check succeeded. 2024-03-21T09:54:33.131 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction test 2024-03-21T09:54:33.193 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction test succeeded. 2024-03-21T09:54:33.193 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction 2024-03-21T09:54:33.392 INFO:teuthology.orchestra.run.smithi174.stdout: Preparing : 1/1 2024-03-21T09:54:33.511 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : rbd-nbd-2:18.2.1-0.el9.x86_64 1/1 2024-03-21T09:54:34.041 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: rbd-nbd-2:18.2.1-0.el9.x86_64 1/1 2024-03-21T09:54:34.388 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : rbd-nbd-2:18.2.1-0.el9.x86_64 1/1 2024-03-21T09:54:34.388 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-21T09:54:34.388 INFO:teuthology.orchestra.run.smithi174.stdout:Installed: 2024-03-21T09:54:34.388 INFO:teuthology.orchestra.run.smithi174.stdout: rbd-nbd-2:18.2.1-0.el9.x86_64 2024-03-21T09:54:34.388 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-21T09:54:34.388 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2024-03-21T09:54:34.707 DEBUG:teuthology.parallel:result is None 2024-03-21T09:54:34.707 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-03-21T09:54:34.708 INFO:teuthology.packaging:ref: None 2024-03-21T09:54:34.708 INFO:teuthology.packaging:tag: v18.2.1 2024-03-21T09:54:34.708 INFO:teuthology.packaging:branch: None 2024-03-21T09:54:34.708 INFO:teuthology.packaging:sha1: 2edfee68ad7243bed422a049b6f5f58619300f11 2024-03-21T09:54:34.708 INFO:teuthology.packaging:Tag 'v18.2.1' not found in https://git.ceph.com/ceph-ci; will also look in https://git.ceph.com/ceph 2024-03-21T09:54:34.708 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7fe91d5d5842e04be3b4f514d6dd990c54b29c76 2024-03-21T09:54:34.855 DEBUG:teuthology.orchestra.run.smithi153:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-21T09:54:34.925 INFO:teuthology.orchestra.run.smithi153.stdout:18.2.1-0.el9 2024-03-21T09:54:34.926 INFO:teuthology.packaging:The installed version of ceph is 18.2.1-0.el9 2024-03-21T09:54:34.926 INFO:teuthology.task.install:The correct ceph version 18.2.1-0 is installed. 2024-03-21T09:54:34.928 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-03-21T09:54:34.928 INFO:teuthology.packaging:ref: None 2024-03-21T09:54:34.928 INFO:teuthology.packaging:tag: v18.2.1 2024-03-21T09:54:34.928 INFO:teuthology.packaging:branch: None 2024-03-21T09:54:34.928 INFO:teuthology.packaging:sha1: 2edfee68ad7243bed422a049b6f5f58619300f11 2024-03-21T09:54:34.928 INFO:teuthology.packaging:Tag 'v18.2.1' not found in https://git.ceph.com/ceph-ci; will also look in https://git.ceph.com/ceph 2024-03-21T09:54:34.928 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7fe91d5d5842e04be3b4f514d6dd990c54b29c76 2024-03-21T09:54:35.083 DEBUG:teuthology.orchestra.run.smithi174:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-21T09:54:35.150 INFO:teuthology.orchestra.run.smithi174.stdout:18.2.1-0.el9 2024-03-21T09:54:35.150 INFO:teuthology.packaging:The installed version of ceph is 18.2.1-0.el9 2024-03-21T09:54:35.150 INFO:teuthology.task.install:The correct ceph version 18.2.1-0 is installed. 2024-03-21T09:54:35.152 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-03-21T09:54:35.152 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-03-21T09:54:35.152 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-21T09:54:35.256 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-03-21T09:54:35.256 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-21T09:54:35.347 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-03-21T09:54:35.348 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-03-21T09:54:35.348 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd of=/usr/bin/daemon-helper 2024-03-21T09:54:35.445 DEBUG:teuthology.orchestra.run.smithi153:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-21T09:54:35.587 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-03-21T09:54:35.587 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/usr/bin/daemon-helper 2024-03-21T09:54:35.689 DEBUG:teuthology.orchestra.run.smithi174:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-21T09:54:35.829 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-03-21T09:54:35.829 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-03-21T09:54:35.829 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-21T09:54:35.929 DEBUG:teuthology.orchestra.run.smithi153:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-21T09:54:36.068 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-03-21T09:54:36.068 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-21T09:54:36.163 DEBUG:teuthology.orchestra.run.smithi174:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-21T09:54:36.295 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-03-21T09:54:36.295 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-03-21T09:54:36.295 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd of=/usr/bin/stdin-killer 2024-03-21T09:54:36.395 DEBUG:teuthology.orchestra.run.smithi153:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-21T09:54:36.491 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-03-21T09:54:36.491 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/usr/bin/stdin-killer 2024-03-21T09:54:36.583 DEBUG:teuthology.orchestra.run.smithi174:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-21T09:54:36.671 INFO:teuthology.run_tasks:Running task print... 2024-03-21T09:54:36.681 INFO:teuthology.task.print:**** done install task... 2024-03-21T09:54:36.681 INFO:teuthology.run_tasks:Running task cephadm... 2024-03-21T09:54:36.939 INFO:tasks.cephadm:Config: {'compiled_cephadm_branch': 'reef', 'conf': {'osd': {'osd_class_default_list': '*', 'osd_class_load_list': '*', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'mon pg warn min per osd': 0}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}}, 'image': 'quay.io/ceph/ceph:v18.2.1', 'roleless': True, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'OSD_DOWN', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN'], 'sha1': '2edfee68ad7243bed422a049b6f5f58619300f11'} 2024-03-21T09:54:37.105 INFO:tasks.cephadm:Cluster image is quay.io/ceph/ceph:v18.2.1 2024-03-21T09:54:37.106 INFO:tasks.cephadm:Cluster fsid is 070ed3e4-e769-11ee-95cd-87774f69a715 2024-03-21T09:54:37.106 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-03-21T09:54:37.106 INFO:tasks.cephadm:No mon roles; fabricating mons 2024-03-21T09:54:37.106 INFO:tasks.cephadm:Monitor IPs: {'mon.smithi153': '172.21.15.153', 'mon.smithi174': '172.21.15.174'} 2024-03-21T09:54:37.106 INFO:tasks.cephadm:Normalizing hostnames... 2024-03-21T09:54:37.106 DEBUG:teuthology.orchestra.run.smithi153:> sudo hostname $(hostname -s) 2024-03-21T09:54:37.204 DEBUG:teuthology.orchestra.run.smithi174:> sudo hostname $(hostname -s) 2024-03-21T09:54:37.302 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra for reef 2024-03-21T09:54:37.302 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2edfee68ad7243bed422a049b6f5f58619300f11 2024-03-21T09:54:37.443 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '2edfee68ad7243bed422a049b6f5f58619300f11', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/77791/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-2231-g2edfee68', 'node_name': '172.21.5.34+adami04', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-2231.g2edfee68'}, 'url': 'https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240320.053002/2edfee68ad7243bed422a049b6f5f58619300f11/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-03-20 07:25:08.579979', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-vshankar-testing-20240320.053002', 'chacra_url': 'https://2.chacra.ceph.com/repos/ceph/wip-vshankar-testing-20240320.053002/2edfee68ad7243bed422a049b6f5f58619300f11/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-03-21T09:54:37.696 INFO:tasks.util.chacra:got chacra host 4.chacra.ceph.com, ref reef, sha1 916c5be8b550efc2c6d48e3bfe4cf657e2993f8f from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&ref=reef 2024-03-21T09:54:37.697 INFO:tasks.cephadm:Discovered cachra url: https://4.chacra.ceph.com/binaries/ceph/reef/916c5be8b550efc2c6d48e3bfe4cf657e2993f8f/centos/9/x86_64/flavors/default/cephadm 2024-03-21T09:54:37.698 DEBUG:teuthology.orchestra.run.smithi153:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/reef/916c5be8b550efc2c6d48e3bfe4cf657e2993f8f/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-21T09:54:38.218 INFO:teuthology.orchestra.run.smithi153.stdout:-rw-r--r--. 1 ubuntu ubuntu 215105 Mar 21 09:54 /home/ubuntu/cephtest/cephadm 2024-03-21T09:54:38.218 DEBUG:teuthology.orchestra.run.smithi174:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/reef/916c5be8b550efc2c6d48e3bfe4cf657e2993f8f/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-21T09:54:38.851 INFO:teuthology.orchestra.run.smithi174.stdout:-rw-r--r--. 1 ubuntu ubuntu 215105 Mar 21 09:54 /home/ubuntu/cephtest/cephadm 2024-03-21T09:54:38.852 DEBUG:teuthology.orchestra.run.smithi153:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-21T09:54:38.912 DEBUG:teuthology.orchestra.run.smithi174:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-21T09:54:38.987 INFO:tasks.cephadm:Pulling image quay.io/ceph/ceph:v18.2.1 on all hosts... 2024-03-21T09:54:38.987 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 pull 2024-03-21T09:54:38.994 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 pull 2024-03-21T09:54:40.048 INFO:teuthology.orchestra.run.smithi153.stderr:Pulling container image quay.io/ceph/ceph:v18.2.1... 2024-03-21T09:54:40.107 INFO:teuthology.orchestra.run.smithi174.stderr:Pulling container image quay.io/ceph/ceph:v18.2.1... 2024-03-21T09:55:04.564 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T09:55:04.564 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_version": "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)", 2024-03-21T09:55:04.564 INFO:teuthology.orchestra.run.smithi153.stdout: "image_id": "5be31c24972a920012b90a9769e8313e2490c82aee752aefd49af9cf4c0f3fcf", 2024-03-21T09:55:04.564 INFO:teuthology.orchestra.run.smithi153.stdout: "repo_digests": [ 2024-03-21T09:55:04.564 INFO:teuthology.orchestra.run.smithi153.stdout: "quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3", 2024-03-21T09:55:04.564 INFO:teuthology.orchestra.run.smithi153.stdout: "quay.io/ceph/ceph@sha256:e8e55db8b4fd270dbec25bc764437a2a3abb707971c4dba5f559fb83018049dc" 2024-03-21T09:55:04.564 INFO:teuthology.orchestra.run.smithi153.stdout: ] 2024-03-21T09:55:04.564 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T09:55:07.359 INFO:teuthology.orchestra.run.smithi174.stdout:{ 2024-03-21T09:55:07.359 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_version": "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)", 2024-03-21T09:55:07.359 INFO:teuthology.orchestra.run.smithi174.stdout: "image_id": "5be31c24972a920012b90a9769e8313e2490c82aee752aefd49af9cf4c0f3fcf", 2024-03-21T09:55:07.359 INFO:teuthology.orchestra.run.smithi174.stdout: "repo_digests": [ 2024-03-21T09:55:07.359 INFO:teuthology.orchestra.run.smithi174.stdout: "quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3", 2024-03-21T09:55:07.359 INFO:teuthology.orchestra.run.smithi174.stdout: "quay.io/ceph/ceph@sha256:e8e55db8b4fd270dbec25bc764437a2a3abb707971c4dba5f559fb83018049dc" 2024-03-21T09:55:07.360 INFO:teuthology.orchestra.run.smithi174.stdout: ] 2024-03-21T09:55:07.360 INFO:teuthology.orchestra.run.smithi174.stdout:} 2024-03-21T09:55:07.391 DEBUG:teuthology.orchestra.run.smithi153:> sudo mkdir -p /etc/ceph 2024-03-21T09:55:07.487 DEBUG:teuthology.orchestra.run.smithi174:> sudo mkdir -p /etc/ceph 2024-03-21T09:55:07.596 DEBUG:teuthology.orchestra.run.smithi153:> sudo chmod 777 /etc/ceph 2024-03-21T09:55:07.684 DEBUG:teuthology.orchestra.run.smithi174:> sudo chmod 777 /etc/ceph 2024-03-21T09:55:07.812 INFO:tasks.cephadm:Writing seed config... 2024-03-21T09:55:07.813 INFO:tasks.cephadm: override: [osd] osd_class_default_list = * 2024-03-21T09:55:07.813 INFO:tasks.cephadm: override: [osd] osd_class_load_list = * 2024-03-21T09:55:07.813 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-03-21T09:55:07.813 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-03-21T09:55:07.813 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-03-21T09:55:07.813 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-03-21T09:55:07.813 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-03-21T09:55:07.813 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-03-21T09:55:07.813 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-03-21T09:55:07.813 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-03-21T09:55:07.813 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-03-21T09:55:07.814 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-03-21T09:55:07.814 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-03-21T09:55:07.814 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-03-21T09:55:07.814 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-03-21T09:55:07.814 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-03-21T09:55:07.814 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-03-21T09:55:07.814 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-03-21T09:55:07.814 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-03-21T09:55:07.814 INFO:tasks.cephadm: override: [client] debug client = 20 2024-03-21T09:55:07.814 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-03-21T09:55:07.814 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-03-21T09:55:07.814 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-03-21T09:55:07.814 INFO:tasks.cephadm: override: [global] mon pg warn min per osd = 0 2024-03-21T09:55:07.814 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-03-21T09:55:07.815 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-03-21T09:55:07.815 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-03-21T09:55:07.815 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-03-21T09:55:07.815 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-03-21T09:55:07.815 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-03-21T09:55:07.815 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-03-21T09:55:07.815 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-03-21T09:55:07.815 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-03-21T09:55:07.815 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-03-21T09:55:07.815 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-03-21T09:55:07.815 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-03-21T09:55:07.815 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-03-21T09:55:07.815 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-03-21T09:55:07.815 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-03-21T09:55:07.815 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-03-21T09:55:07.816 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-03-21T09:55:07.816 DEBUG:teuthology.orchestra.run.smithi153:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-03-21T09:55:07.868 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 070ed3e4-e769-11ee-95cd-87774f69a715 mon pg warn min per osd = 0 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = true bdev debug aio = true osd sloppy crc = true osd_class_default_list = * osd_class_load_list = * bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-03-21T09:55:07.869 DEBUG:teuthology.orchestra.run.smithi153:mon.smithi153> sudo journalctl -f -n 0 -u ceph-070ed3e4-e769-11ee-95cd-87774f69a715@mon.smithi153.service 2024-03-21T09:55:07.916 INFO:tasks.cephadm:Bootstrapping... 2024-03-21T09:55:07.916 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 -v bootstrap --fsid 070ed3e4-e769-11ee-95cd-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-ip 172.21.15.153 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-03-21T09:55:08.168 INFO:teuthology.orchestra.run.smithi153.stdout:-------------------------------------------------------------------------------- 2024-03-21T09:55:08.168 INFO:teuthology.orchestra.run.smithi153.stdout:cephadm ['--image', 'quay.io/ceph/ceph:v18.2.1', '-v', 'bootstrap', '--fsid', '070ed3e4-e769-11ee-95cd-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-ip', '172.21.15.153', '--skip-admin-label'] 2024-03-21T09:55:08.223 INFO:teuthology.orchestra.run.smithi153.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-21T09:55:08.223 INFO:teuthology.orchestra.run.smithi153.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-03-21T09:55:08.224 INFO:teuthology.orchestra.run.smithi153.stdout:Verifying podman|docker is present... 2024-03-21T09:55:08.278 INFO:teuthology.orchestra.run.smithi153.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-21T09:55:08.278 INFO:teuthology.orchestra.run.smithi153.stdout:Verifying lvm2 is present... 2024-03-21T09:55:08.278 INFO:teuthology.orchestra.run.smithi153.stdout:Verifying time synchronization is in place... 2024-03-21T09:55:08.293 INFO:teuthology.orchestra.run.smithi153.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-21T09:55:08.293 INFO:teuthology.orchestra.run.smithi153.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-21T09:55:08.308 INFO:teuthology.orchestra.run.smithi153.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-21T09:55:08.308 INFO:teuthology.orchestra.run.smithi153.stdout:systemctl: stdout inactive 2024-03-21T09:55:08.328 INFO:teuthology.orchestra.run.smithi153.stdout:systemctl: stdout enabled 2024-03-21T09:55:08.349 INFO:teuthology.orchestra.run.smithi153.stdout:systemctl: stdout active 2024-03-21T09:55:08.349 INFO:teuthology.orchestra.run.smithi153.stdout:Unit chronyd.service is enabled and running 2024-03-21T09:55:08.350 INFO:teuthology.orchestra.run.smithi153.stdout:Repeating the final host check... 2024-03-21T09:55:08.404 INFO:teuthology.orchestra.run.smithi153.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-21T09:55:08.405 INFO:teuthology.orchestra.run.smithi153.stdout:podman (/bin/podman) version 4.9.3 is present 2024-03-21T09:55:08.405 INFO:teuthology.orchestra.run.smithi153.stdout:systemctl is present 2024-03-21T09:55:08.405 INFO:teuthology.orchestra.run.smithi153.stdout:lvcreate is present 2024-03-21T09:55:08.418 INFO:teuthology.orchestra.run.smithi153.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-21T09:55:08.418 INFO:teuthology.orchestra.run.smithi153.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-21T09:55:08.435 INFO:teuthology.orchestra.run.smithi153.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-21T09:55:08.435 INFO:teuthology.orchestra.run.smithi153.stdout:systemctl: stdout inactive 2024-03-21T09:55:08.456 INFO:teuthology.orchestra.run.smithi153.stdout:systemctl: stdout enabled 2024-03-21T09:55:08.476 INFO:teuthology.orchestra.run.smithi153.stdout:systemctl: stdout active 2024-03-21T09:55:08.476 INFO:teuthology.orchestra.run.smithi153.stdout:Unit chronyd.service is enabled and running 2024-03-21T09:55:08.476 INFO:teuthology.orchestra.run.smithi153.stdout:Host looks OK 2024-03-21T09:55:08.477 INFO:teuthology.orchestra.run.smithi153.stdout:Cluster fsid: 070ed3e4-e769-11ee-95cd-87774f69a715 2024-03-21T09:55:08.477 INFO:teuthology.orchestra.run.smithi153.stdout:Acquiring lock 139695909276352 on /run/cephadm/070ed3e4-e769-11ee-95cd-87774f69a715.lock 2024-03-21T09:55:08.478 INFO:teuthology.orchestra.run.smithi153.stdout:Lock 139695909276352 acquired on /run/cephadm/070ed3e4-e769-11ee-95cd-87774f69a715.lock 2024-03-21T09:55:08.479 INFO:teuthology.orchestra.run.smithi153.stdout:Verifying IP 172.21.15.153 port 3300 ... 2024-03-21T09:55:08.479 INFO:teuthology.orchestra.run.smithi153.stdout:Verifying IP 172.21.15.153 port 6789 ... 2024-03-21T09:55:08.480 INFO:teuthology.orchestra.run.smithi153.stdout:Base mon IP(s) is [172.21.15.153:3300, 172.21.15.153:6789], mon addrv is [v2:172.21.15.153:3300,v1:172.21.15.153:6789] 2024-03-21T09:55:08.494 INFO:teuthology.orchestra.run.smithi153.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.153 metric 100 2024-03-21T09:55:08.494 INFO:teuthology.orchestra.run.smithi153.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.153 metric 100 2024-03-21T09:55:08.506 INFO:teuthology.orchestra.run.smithi153.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-03-21T09:55:08.506 INFO:teuthology.orchestra.run.smithi153.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-03-21T09:55:08.506 INFO:teuthology.orchestra.run.smithi153.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1798sec hoplimit 64 pref medium 2024-03-21T09:55:08.517 INFO:teuthology.orchestra.run.smithi153.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-03-21T09:55:08.518 INFO:teuthology.orchestra.run.smithi153.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-03-21T09:55:08.518 INFO:teuthology.orchestra.run.smithi153.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-21T09:55:08.518 INFO:teuthology.orchestra.run.smithi153.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-03-21T09:55:08.518 INFO:teuthology.orchestra.run.smithi153.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe88:6deb/64 scope link 2024-03-21T09:55:08.518 INFO:teuthology.orchestra.run.smithi153.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-21T09:55:08.520 INFO:teuthology.orchestra.run.smithi153.stdout:Mon IP `172.21.15.153` is in CIDR network `172.21.0.0/20` 2024-03-21T09:55:08.520 INFO:teuthology.orchestra.run.smithi153.stdout:Mon IP `172.21.15.153` is in CIDR network `172.21.0.0/20` 2024-03-21T09:55:08.521 INFO:teuthology.orchestra.run.smithi153.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-03-21T09:55:08.523 INFO:teuthology.orchestra.run.smithi153.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-03-21T09:55:08.526 INFO:teuthology.orchestra.run.smithi153.stdout:Pulling container image quay.io/ceph/ceph:v18.2.1... 2024-03-21T09:55:09.510 INFO:teuthology.orchestra.run.smithi153.stdout:/bin/podman: stdout 5be31c24972a920012b90a9769e8313e2490c82aee752aefd49af9cf4c0f3fcf 2024-03-21T09:55:09.510 INFO:teuthology.orchestra.run.smithi153.stdout:/bin/podman: stderr Trying to pull quay.io/ceph/ceph:v18.2.1... 2024-03-21T09:55:09.510 INFO:teuthology.orchestra.run.smithi153.stdout:/bin/podman: stderr Getting image source signatures 2024-03-21T09:55:09.511 INFO:teuthology.orchestra.run.smithi153.stdout:/bin/podman: stderr Copying blob sha256:a733d3c618b71f19c168ebecd1953429dce2c1631835ca182e9551c36dce5989 2024-03-21T09:55:09.511 INFO:teuthology.orchestra.run.smithi153.stdout:/bin/podman: stderr Copying blob sha256:7feca07754707458c3945cf0062cf4dabc512f6d90fe1a9a1370b362b6011124 2024-03-21T09:55:09.511 INFO:teuthology.orchestra.run.smithi153.stdout:/bin/podman: stderr Copying config sha256:5be31c24972a920012b90a9769e8313e2490c82aee752aefd49af9cf4c0f3fcf 2024-03-21T09:55:09.511 INFO:teuthology.orchestra.run.smithi153.stdout:/bin/podman: stderr Writing manifest to image destination 2024-03-21T09:55:10.712 INFO:teuthology.orchestra.run.smithi153.stdout:ceph: stdout ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable) 2024-03-21T09:55:10.712 INFO:teuthology.orchestra.run.smithi153.stdout:Ceph version: ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable) 2024-03-21T09:55:10.712 INFO:teuthology.orchestra.run.smithi153.stdout:Extracting ceph user uid/gid from container image... 2024-03-21T09:55:11.800 INFO:teuthology.orchestra.run.smithi153.stdout:stat: stdout 167 167 2024-03-21T09:55:11.800 INFO:teuthology.orchestra.run.smithi153.stdout:Creating initial keys... 2024-03-21T09:55:12.780 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-authtool: stdout AQAABPxl84ZNDBAAwK62t6/2VAo/bb71HG6qVQ== 2024-03-21T09:55:13.851 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-authtool: stdout AQABBPxlgsD8CxAAqq6ZdJ0lT5dype1kdJba6g== 2024-03-21T09:55:14.894 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph-authtool: stdout AQACBPxll9qCDxAAOAya04QXcQ87Y5wNfiZkvQ== 2024-03-21T09:55:14.896 INFO:teuthology.orchestra.run.smithi153.stdout:Creating initial monmap... 2024-03-21T09:55:16.045 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-21T09:55:16.045 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2024-03-21T09:55:16.045 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 070ed3e4-e769-11ee-95cd-87774f69a715 2024-03-21T09:55:16.045 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-21T09:55:16.045 INFO:teuthology.orchestra.run.smithi153.stdout:monmaptool for smithi153 [v2:172.21.15.153:3300,v1:172.21.15.153:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-21T09:55:16.045 INFO:teuthology.orchestra.run.smithi153.stdout:setting min_mon_release = pacific 2024-03-21T09:55:16.045 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/monmaptool: set fsid to 070ed3e4-e769-11ee-95cd-87774f69a715 2024-03-21T09:55:16.045 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-21T09:55:16.045 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T09:55:16.045 INFO:teuthology.orchestra.run.smithi153.stdout:Creating mon... 2024-03-21T09:55:17.135 INFO:teuthology.orchestra.run.smithi153.stdout:create mon.smithi153 on 2024-03-21T09:55:17.476 INFO:teuthology.orchestra.run.smithi153.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-03-21T09:55:17.708 INFO:teuthology.orchestra.run.smithi153.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-03-21T09:55:18.001 INFO:teuthology.orchestra.run.smithi153.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-070ed3e4-e769-11ee-95cd-87774f69a715.target → /etc/systemd/system/ceph-070ed3e4-e769-11ee-95cd-87774f69a715.target. 2024-03-21T09:55:18.001 INFO:teuthology.orchestra.run.smithi153.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-070ed3e4-e769-11ee-95cd-87774f69a715.target → /etc/systemd/system/ceph-070ed3e4-e769-11ee-95cd-87774f69a715.target. 2024-03-21T09:55:18.329 INFO:teuthology.orchestra.run.smithi153.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-070ed3e4-e769-11ee-95cd-87774f69a715@mon.smithi153 2024-03-21T09:55:18.330 INFO:teuthology.orchestra.run.smithi153.stdout:systemctl: stderr Failed to reset failed state of unit ceph-070ed3e4-e769-11ee-95cd-87774f69a715@mon.smithi153.service: Unit ceph-070ed3e4-e769-11ee-95cd-87774f69a715@mon.smithi153.service not loaded. 2024-03-21T09:55:18.607 INFO:teuthology.orchestra.run.smithi153.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-070ed3e4-e769-11ee-95cd-87774f69a715.target.wants/ceph-070ed3e4-e769-11ee-95cd-87774f69a715@mon.smithi153.service → /etc/systemd/system/ceph-070ed3e4-e769-11ee-95cd-87774f69a715@.service. 2024-03-21T09:55:19.277 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:18 smithi153 podman[28731]: 2024-03-21 09:55:18.993559295 +0000 UTC m=+0.140836454 container create 22cb43f4f5d6053bca480f13ee8e68c620becde0dc5391da10d42c215de5325e (image=quay.io/ceph/ceph:v18.2.1, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-mon-smithi153, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240222, ceph=True, CEPH_POINT_RELEASE=-18.2.1, GIT_BRANCH=HEAD, RELEASE=HEAD, GIT_CLEAN=True, io.buildah.version=1.29.1, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=https://github.com/ceph/ceph-container.git) 2024-03-21T09:55:19.277 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:19 smithi153 podman[28731]: 2024-03-21 09:55:19.274654685 +0000 UTC m=+0.421931853 container init 22cb43f4f5d6053bca480f13ee8e68c620becde0dc5391da10d42c215de5325e (image=quay.io/ceph/ceph:v18.2.1, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-mon-smithi153, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.29.1, maintainer=Guillaume Abrioux , RELEASE=HEAD, ceph=True, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.build-date=20240222, CEPH_POINT_RELEASE=-18.2.1, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-03-21T09:55:19.403 INFO:teuthology.orchestra.run.smithi153.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-21T09:55:19.403 INFO:teuthology.orchestra.run.smithi153.stdout:systemctl: stdout disabled 2024-03-21T09:55:19.417 INFO:teuthology.orchestra.run.smithi153.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-21T09:55:19.417 INFO:teuthology.orchestra.run.smithi153.stdout:systemctl: stdout inactive 2024-03-21T09:55:19.417 INFO:teuthology.orchestra.run.smithi153.stdout:firewalld.service is not enabled 2024-03-21T09:55:19.417 INFO:teuthology.orchestra.run.smithi153.stdout:Not possible to enable service . firewalld.service is not available 2024-03-21T09:55:19.417 INFO:teuthology.orchestra.run.smithi153.stdout:Waiting for mon to start... 2024-03-21T09:55:19.418 INFO:teuthology.orchestra.run.smithi153.stdout:Waiting for mon... 2024-03-21T09:55:19.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:19 smithi153 podman[28731]: 2024-03-21 09:55:19.283452278 +0000 UTC m=+0.430729442 container start 22cb43f4f5d6053bca480f13ee8e68c620becde0dc5391da10d42c215de5325e (image=quay.io/ceph/ceph:v18.2.1, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-mon-smithi153, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=-18.2.1, org.label-schema.schema-version=1.0, RELEASE=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=https://github.com/ceph/ceph-container.git, io.buildah.version=1.29.1, org.label-schema.license=GPLv2, ceph=True) 2024-03-21T09:55:19.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:19 smithi153 ceph-mon[28750]: mkfs 070ed3e4-e769-11ee-95cd-87774f69a715 2024-03-21T09:55:19.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:19 smithi153 ceph-mon[28750]: mon.smithi153 is new leader, mons smithi153 in quorum (ranks 0) 2024-03-21T09:55:19.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:19 smithi153 bash[28731]: 22cb43f4f5d6053bca480f13ee8e68c620becde0dc5391da10d42c215de5325e 2024-03-21T09:55:19.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:19 smithi153 systemd[1]: Started Ceph mon.smithi153 for 070ed3e4-e769-11ee-95cd-87774f69a715. 2024-03-21T09:55:20.704 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:20 smithi153 ceph-mon[28750]: mon.smithi153 is new leader, mons smithi153 in quorum (ranks 0) 2024-03-21T09:55:20.704 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:20 smithi153 ceph-mon[28750]: monmap e1: 1 mons at {smithi153=[v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-21T09:55:20.704 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:20 smithi153 ceph-mon[28750]: fsmap 2024-03-21T09:55:20.704 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:20 smithi153 ceph-mon[28750]: osdmap e1: 0 total, 0 up, 0 in 2024-03-21T09:55:20.704 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:20 smithi153 ceph-mon[28750]: mgrmap e1: no daemons active 2024-03-21T09:55:20.704 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:20 smithi153 ceph-mon[28750]: from='client.? 172.21.15.153:0/426381440' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-03-21T09:55:24.605 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout cluster: 2024-03-21T09:55:24.605 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout id: 070ed3e4-e769-11ee-95cd-87774f69a715 2024-03-21T09:55:24.605 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-03-21T09:55:24.605 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout 2024-03-21T09:55:24.605 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout services: 2024-03-21T09:55:24.605 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum smithi153 (age 0.710202s) 2024-03-21T09:55:24.611 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-03-21T09:55:24.611 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-03-21T09:55:24.611 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout 2024-03-21T09:55:24.611 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout data: 2024-03-21T09:55:24.611 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-03-21T09:55:24.611 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-03-21T09:55:24.611 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-03-21T09:55:24.611 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout pgs: 2024-03-21T09:55:24.611 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout 2024-03-21T09:55:24.611 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:19.978+0000 7fa6f7c00700 1 Processor -- start 2024-03-21T09:55:24.612 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:19.978+0000 7fa6f7c00700 1 -- start start 2024-03-21T09:55:24.612 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:19.978+0000 7fa6f7c00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa6f0108680 0x7fa6f0108aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:55:24.612 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:19.978+0000 7fa6f7c00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f0109070 con 0x7fa6f0108680 2024-03-21T09:55:24.612 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:19.979+0000 7fa6f5200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa6f0108680 0x7fa6f0108aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:55:24.612 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:19.979+0000 7fa6f5200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa6f0108680 0x7fa6f0108aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:47140/0 (socket says 172.21.15.153:47140) 2024-03-21T09:55:24.612 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:19.979+0000 7fa6f5200700 1 -- 172.21.15.153:0/2460568412 learned_addr learned my addr 172.21.15.153:0/2460568412 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T09:55:24.612 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:19.980+0000 7fa6f5200700 1 -- 172.21.15.153:0/2460568412 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6f0109830 con 0x7fa6f0108680 2024-03-21T09:55:24.612 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:19.980+0000 7fa6f5200700 1 --2- 172.21.15.153:0/2460568412 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa6f0108680 0x7fa6f0108aa0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fa6e40098a0 tx=0x7fa6e4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bc8424de1d36f79 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:55:24.612 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:19.980+0000 7fa6ef400700 1 -- 172.21.15.153:0/2460568412 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa6e4004180 con 0x7fa6f0108680 2024-03-21T09:55:24.612 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:19.981+0000 7fa6ef400700 1 -- 172.21.15.153:0/2460568412 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6e40042e0 con 0x7fa6f0108680 2024-03-21T09:55:24.612 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:19.981+0000 7fa6ef400700 1 -- 172.21.15.153:0/2460568412 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa6e40036a0 con 0x7fa6f0108680 2024-03-21T09:55:24.612 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:19.981+0000 7fa6f7c00700 1 -- 172.21.15.153:0/2460568412 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa6f0108680 msgr2=0x7fa6f0108aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:55:24.612 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:19.981+0000 7fa6f7c00700 1 --2- 172.21.15.153:0/2460568412 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa6f0108680 0x7fa6f0108aa0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fa6e40098a0 tx=0x7fa6e4009bb0 comp rx=0 tx=0).stop 2024-03-21T09:55:24.613 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:19.981+0000 7fa6f7c00700 1 -- 172.21.15.153:0/2460568412 shutdown_connections 2024-03-21T09:55:24.613 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:19.981+0000 7fa6f7c00700 1 --2- 172.21.15.153:0/2460568412 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa6f0108680 0x7fa6f0108aa0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:55:24.613 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:19.981+0000 7fa6f7c00700 1 -- 172.21.15.153:0/2460568412 >> 172.21.15.153:0/2460568412 conn(0x7fa6f007bb20 msgr2=0x7fa6f01064e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:55:24.613 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:19.981+0000 7fa6f7c00700 1 -- 172.21.15.153:0/2460568412 shutdown_connections 2024-03-21T09:55:24.613 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:19.982+0000 7fa6f7c00700 1 -- 172.21.15.153:0/2460568412 wait complete. 2024-03-21T09:55:24.613 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:19.983+0000 7fa6f7c00700 1 Processor -- start 2024-03-21T09:55:24.613 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:19.983+0000 7fa6f7c00700 1 -- start start 2024-03-21T09:55:24.613 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:19.983+0000 7fa6f7c00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa6f0108680 0x7fa6f019bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:55:24.613 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:19.983+0000 7fa6f7c00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f019c000 con 0x7fa6f0108680 2024-03-21T09:55:24.613 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:19.983+0000 7fa6f5200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa6f0108680 0x7fa6f019bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:55:24.613 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:19.983+0000 7fa6f5200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa6f0108680 0x7fa6f019bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:47148/0 (socket says 172.21.15.153:47148) 2024-03-21T09:55:24.613 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:19.983+0000 7fa6f5200700 1 -- 172.21.15.153:0/426381440 learned_addr learned my addr 172.21.15.153:0/426381440 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T09:55:24.613 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:19.984+0000 7fa6f5200700 1 -- 172.21.15.153:0/426381440 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6e4009550 con 0x7fa6f0108680 2024-03-21T09:55:24.613 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:19.984+0000 7fa6f5200700 1 --2- 172.21.15.153:0/426381440 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa6f0108680 0x7fa6f019bac0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fa6e4004000 tx=0x7fa6e4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:55:24.614 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:19.984+0000 7fa6ed600700 1 -- 172.21.15.153:0/426381440 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa6e4003ce0 con 0x7fa6f0108680 2024-03-21T09:55:24.614 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:19.984+0000 7fa6ed600700 1 -- 172.21.15.153:0/426381440 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6e4020070 con 0x7fa6f0108680 2024-03-21T09:55:24.614 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:19.984+0000 7fa6ed600700 1 -- 172.21.15.153:0/426381440 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa6e4011630 con 0x7fa6f0108680 2024-03-21T09:55:24.614 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:19.985+0000 7fa6f7c00700 1 -- 172.21.15.153:0/426381440 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6f019c200 con 0x7fa6f0108680 2024-03-21T09:55:24.614 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:19.985+0000 7fa6f7c00700 1 -- 172.21.15.153:0/426381440 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6f019c6a0 con 0x7fa6f0108680 2024-03-21T09:55:24.614 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:19.985+0000 7fa6ed600700 1 -- 172.21.15.153:0/426381440 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fa6e4022030 con 0x7fa6f0108680 2024-03-21T09:55:24.614 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:19.985+0000 7fa6ed600700 1 -- 172.21.15.153:0/426381440 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fa6e4025d40 con 0x7fa6f0108680 2024-03-21T09:55:24.614 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:19.986+0000 7fa6f7c00700 1 -- 172.21.15.153:0/426381440 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6f0195370 con 0x7fa6f0108680 2024-03-21T09:55:24.614 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:19.989+0000 7fa6ed600700 1 -- 172.21.15.153:0/426381440 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7fa6e4025480 con 0x7fa6f0108680 2024-03-21T09:55:24.614 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:20.067+0000 7fa6f7c00700 1 -- 172.21.15.153:0/426381440 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7fa6f002cc70 con 0x7fa6f0108680 2024-03-21T09:55:24.614 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:20.068+0000 7fa6ed600700 1 -- 172.21.15.153:0/426381440 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+325 (secure 0 0 0) 0x7fa6e4036070 con 0x7fa6f0108680 2024-03-21T09:55:24.614 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:20.070+0000 7fa6f7c00700 1 -- 172.21.15.153:0/426381440 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa6f0108680 msgr2=0x7fa6f019bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:55:24.615 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:20.070+0000 7fa6f7c00700 1 --2- 172.21.15.153:0/426381440 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa6f0108680 0x7fa6f019bac0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fa6e4004000 tx=0x7fa6e4004750 comp rx=0 tx=0).stop 2024-03-21T09:55:24.616 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:20.071+0000 7fa6f7c00700 1 -- 172.21.15.153:0/426381440 shutdown_connections 2024-03-21T09:55:24.616 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:20.071+0000 7fa6f7c00700 1 --2- 172.21.15.153:0/426381440 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa6f0108680 0x7fa6f019bac0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:55:24.616 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:20.071+0000 7fa6f7c00700 1 -- 172.21.15.153:0/426381440 >> 172.21.15.153:0/426381440 conn(0x7fa6f007bb20 msgr2=0x7fa6f018fff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:55:24.616 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:20.071+0000 7fa6f7c00700 1 -- 172.21.15.153:0/426381440 shutdown_connections 2024-03-21T09:55:24.616 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:20.071+0000 7fa6f7c00700 1 -- 172.21.15.153:0/426381440 wait complete. 2024-03-21T09:55:24.616 INFO:teuthology.orchestra.run.smithi153.stdout:mon is available 2024-03-21T09:55:24.616 INFO:teuthology.orchestra.run.smithi153.stdout:Assimilating anything we can from ceph.conf... 2024-03-21T09:55:30.128 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout 2024-03-21T09:55:30.128 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout [global] 2024-03-21T09:55:30.128 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout fsid = 070ed3e4-e769-11ee-95cd-87774f69a715 2024-03-21T09:55:30.128 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.153:3300,v1:172.21.15.153:6789] 2024-03-21T09:55:30.128 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-21T09:55:30.128 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-21T09:55:30.129 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-21T09:55:30.129 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-21T09:55:30.129 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout 2024-03-21T09:55:30.129 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-21T09:55:30.129 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-21T09:55:30.129 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout 2024-03-21T09:55:30.129 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout [osd] 2024-03-21T09:55:30.129 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-21T09:55:30.129 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-21T09:55:30.129 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-21T09:55:30.129 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-21T09:55:30.129 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-21T09:55:30.129 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:29.423+0000 7fb296000700 1 Processor -- start 2024-03-21T09:55:30.130 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:29.423+0000 7fb296000700 1 -- start start 2024-03-21T09:55:30.130 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:29.424+0000 7fb296000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb290108630 0x7fb290108a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:55:30.130 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:29.424+0000 7fb296000700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb290109020 con 0x7fb290108630 2024-03-21T09:55:30.130 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:29.425+0000 7fb28f400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb290108630 0x7fb290108a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:55:30.130 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:29.425+0000 7fb28f400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb290108630 0x7fb290108a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:33844/0 (socket says 172.21.15.153:33844) 2024-03-21T09:55:30.130 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:29.425+0000 7fb28f400700 1 -- 172.21.15.153:0/1932257061 learned_addr learned my addr 172.21.15.153:0/1932257061 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T09:55:30.130 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:29.425+0000 7fb28f400700 1 -- 172.21.15.153:0/1932257061 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb290109830 con 0x7fb290108630 2024-03-21T09:55:30.130 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:29.426+0000 7fb28f400700 1 --2- 172.21.15.153:0/1932257061 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb290108630 0x7fb290108a50 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fb2800098a0 tx=0x7fb280009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd5a746205b699bf server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:55:30.130 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:29.426+0000 7fb28ea00700 1 -- 172.21.15.153:0/1932257061 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb280004180 con 0x7fb290108630 2024-03-21T09:55:30.130 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:29.426+0000 7fb28ea00700 1 -- 172.21.15.153:0/1932257061 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2800042e0 con 0x7fb290108630 2024-03-21T09:55:30.131 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:29.426+0000 7fb28ea00700 1 -- 172.21.15.153:0/1932257061 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb2800036a0 con 0x7fb290108630 2024-03-21T09:55:30.131 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:29.426+0000 7fb296000700 1 -- 172.21.15.153:0/1932257061 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb290108630 msgr2=0x7fb290108a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:55:30.131 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:29.426+0000 7fb296000700 1 --2- 172.21.15.153:0/1932257061 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb290108630 0x7fb290108a50 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fb2800098a0 tx=0x7fb280009bb0 comp rx=0 tx=0).stop 2024-03-21T09:55:30.131 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:29.427+0000 7fb296000700 1 -- 172.21.15.153:0/1932257061 shutdown_connections 2024-03-21T09:55:30.131 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:29.427+0000 7fb296000700 1 --2- 172.21.15.153:0/1932257061 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb290108630 0x7fb290108a50 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:55:30.131 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:29.427+0000 7fb296000700 1 -- 172.21.15.153:0/1932257061 >> 172.21.15.153:0/1932257061 conn(0x7fb290103c10 msgr2=0x7fb290106050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:55:30.131 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:29.427+0000 7fb296000700 1 -- 172.21.15.153:0/1932257061 shutdown_connections 2024-03-21T09:55:30.131 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:29.427+0000 7fb296000700 1 -- 172.21.15.153:0/1932257061 wait complete. 2024-03-21T09:55:30.131 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:29.428+0000 7fb296000700 1 Processor -- start 2024-03-21T09:55:30.131 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:29.428+0000 7fb296000700 1 -- start start 2024-03-21T09:55:30.131 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:29.428+0000 7fb296000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb290108630 0x7fb29019ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:55:30.131 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:29.428+0000 7fb296000700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb29019bf70 con 0x7fb290108630 2024-03-21T09:55:30.132 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:29.428+0000 7fb28f400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb290108630 0x7fb29019ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:55:30.132 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:29.428+0000 7fb28f400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb290108630 0x7fb29019ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:33850/0 (socket says 172.21.15.153:33850) 2024-03-21T09:55:30.132 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:29.428+0000 7fb28f400700 1 -- 172.21.15.153:0/3402644365 learned_addr learned my addr 172.21.15.153:0/3402644365 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T09:55:30.132 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:29.429+0000 7fb28f400700 1 -- 172.21.15.153:0/3402644365 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb280009550 con 0x7fb290108630 2024-03-21T09:55:30.132 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:29.429+0000 7fb28f400700 1 --2- 172.21.15.153:0/3402644365 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb290108630 0x7fb29019ba30 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fb280004000 tx=0x7fb280004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:55:30.132 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:29.429+0000 7fb28cc00700 1 -- 172.21.15.153:0/3402644365 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb280003ce0 con 0x7fb290108630 2024-03-21T09:55:30.132 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:29.429+0000 7fb28cc00700 1 -- 172.21.15.153:0/3402644365 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb280020070 con 0x7fb290108630 2024-03-21T09:55:30.132 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:29.429+0000 7fb296000700 1 -- 172.21.15.153:0/3402644365 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb29019c170 con 0x7fb290108630 2024-03-21T09:55:30.132 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:29.429+0000 7fb28cc00700 1 -- 172.21.15.153:0/3402644365 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb280011630 con 0x7fb290108630 2024-03-21T09:55:30.132 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:29.430+0000 7fb296000700 1 -- 172.21.15.153:0/3402644365 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb29019c610 con 0x7fb290108630 2024-03-21T09:55:30.132 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:29.430+0000 7fb28cc00700 1 -- 172.21.15.153:0/3402644365 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fb280022030 con 0x7fb290108630 2024-03-21T09:55:30.133 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:29.430+0000 7fb28cc00700 1 -- 172.21.15.153:0/3402644365 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fb280025cb0 con 0x7fb290108630 2024-03-21T09:55:30.133 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:29.431+0000 7fb296000700 1 -- 172.21.15.153:0/3402644365 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb290195370 con 0x7fb290108630 2024-03-21T09:55:30.133 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:29.434+0000 7fb28cc00700 1 -- 172.21.15.153:0/3402644365 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7fb280025480 con 0x7fb290108630 2024-03-21T09:55:30.133 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:29.508+0000 7fb296000700 1 -- 172.21.15.153:0/3402644365 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fb2900623c0 con 0x7fb290108630 2024-03-21T09:55:30.133 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:29.518+0000 7fb28cc00700 1 -- 172.21.15.153:0/3402644365 <== mon.0 v2:172.21.15.153:3300/0 7 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb280011bd0 con 0x7fb290108630 2024-03-21T09:55:30.133 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:29.518+0000 7fb28cc00700 1 -- 172.21.15.153:0/3402644365 <== mon.0 v2:172.21.15.153:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+431 (secure 0 0 0) 0x7fb28001c500 con 0x7fb290108630 2024-03-21T09:55:30.133 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:29.519+0000 7fb296000700 1 -- 172.21.15.153:0/3402644365 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb290108630 msgr2=0x7fb29019ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:55:30.133 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:29.519+0000 7fb296000700 1 --2- 172.21.15.153:0/3402644365 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb290108630 0x7fb29019ba30 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fb280004000 tx=0x7fb280004750 comp rx=0 tx=0).stop 2024-03-21T09:55:30.133 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:29.519+0000 7fb296000700 1 -- 172.21.15.153:0/3402644365 shutdown_connections 2024-03-21T09:55:30.133 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:29.519+0000 7fb296000700 1 --2- 172.21.15.153:0/3402644365 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb290108630 0x7fb29019ba30 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:55:30.133 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:29.519+0000 7fb296000700 1 -- 172.21.15.153:0/3402644365 >> 172.21.15.153:0/3402644365 conn(0x7fb290103c10 msgr2=0x7fb29018fff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:55:30.133 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:29.519+0000 7fb296000700 1 -- 172.21.15.153:0/3402644365 shutdown_connections 2024-03-21T09:55:30.134 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:29.520+0000 7fb296000700 1 -- 172.21.15.153:0/3402644365 wait complete. 2024-03-21T09:55:30.134 INFO:teuthology.orchestra.run.smithi153.stdout:Generating new minimal ceph.conf... 2024-03-21T09:55:30.425 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:30 smithi153 ceph-mon[28750]: from='client.? 172.21.15.153:0/3402644365' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-21T09:55:30.425 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:30 smithi153 ceph-mon[28750]: from='client.? 172.21.15.153:0/3402644365' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-03-21T09:55:31.413 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:30.705+0000 7f8335e00700 1 Processor -- start 2024-03-21T09:55:31.413 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:30.705+0000 7f8335e00700 1 -- start start 2024-03-21T09:55:31.413 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:30.706+0000 7f8335e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f833010e030 0x7f833010e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:55:31.413 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:30.706+0000 7f8335e00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f833010ea20 con 0x7f833010e030 2024-03-21T09:55:31.413 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:30.706+0000 7f832ea00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f833010e030 0x7f833010e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:55:31.413 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:30.706+0000 7f832ea00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f833010e030 0x7f833010e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:33852/0 (socket says 172.21.15.153:33852) 2024-03-21T09:55:31.413 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:30.706+0000 7f832ea00700 1 -- 172.21.15.153:0/683896223 learned_addr learned my addr 172.21.15.153:0/683896223 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T09:55:31.413 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:30.707+0000 7f832ea00700 1 -- 172.21.15.153:0/683896223 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f833010eb60 con 0x7f833010e030 2024-03-21T09:55:31.413 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:30.707+0000 7f832ea00700 1 --2- 172.21.15.153:0/683896223 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f833010e030 0x7f833010e450 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f83200098a0 tx=0x7f8320009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=611b3c4d01d562 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:55:31.414 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:30.707+0000 7f832e000700 1 -- 172.21.15.153:0/683896223 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8320004180 con 0x7f833010e030 2024-03-21T09:55:31.414 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:30.708+0000 7f832e000700 1 -- 172.21.15.153:0/683896223 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f83200042e0 con 0x7f833010e030 2024-03-21T09:55:31.414 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:30.708+0000 7f832e000700 1 -- 172.21.15.153:0/683896223 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8320011a50 con 0x7f833010e030 2024-03-21T09:55:31.414 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:30.708+0000 7f8335e00700 1 -- 172.21.15.153:0/683896223 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f833010e030 msgr2=0x7f833010e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:55:31.414 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:30.708+0000 7f8335e00700 1 --2- 172.21.15.153:0/683896223 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f833010e030 0x7f833010e450 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f83200098a0 tx=0x7f8320009bb0 comp rx=0 tx=0).stop 2024-03-21T09:55:31.414 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:30.708+0000 7f8335e00700 1 -- 172.21.15.153:0/683896223 shutdown_connections 2024-03-21T09:55:31.414 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:30.708+0000 7f8335e00700 1 --2- 172.21.15.153:0/683896223 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f833010e030 0x7f833010e450 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:55:31.414 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:30.708+0000 7f8335e00700 1 -- 172.21.15.153:0/683896223 >> 172.21.15.153:0/683896223 conn(0x7f833007bb20 msgr2=0x7f833007df80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:55:31.415 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:30.709+0000 7f8335e00700 1 -- 172.21.15.153:0/683896223 shutdown_connections 2024-03-21T09:55:31.415 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:30.709+0000 7f8335e00700 1 -- 172.21.15.153:0/683896223 wait complete. 2024-03-21T09:55:31.415 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:30.709+0000 7f8335e00700 1 Processor -- start 2024-03-21T09:55:31.415 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:30.710+0000 7f8335e00700 1 -- start start 2024-03-21T09:55:31.415 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:30.710+0000 7f8335e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f833010e030 0x7f83301a0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:55:31.415 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:30.710+0000 7f8335e00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83301a0c80 con 0x7f833010e030 2024-03-21T09:55:31.415 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:30.710+0000 7f832ea00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f833010e030 0x7f83301a0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:55:31.415 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:30.710+0000 7f832ea00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f833010e030 0x7f83301a0740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:33866/0 (socket says 172.21.15.153:33866) 2024-03-21T09:55:31.415 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:30.710+0000 7f832ea00700 1 -- 172.21.15.153:0/2994127715 learned_addr learned my addr 172.21.15.153:0/2994127715 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T09:55:31.415 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:30.711+0000 7f832ea00700 1 -- 172.21.15.153:0/2994127715 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8320009550 con 0x7f833010e030 2024-03-21T09:55:31.415 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:30.711+0000 7f832ea00700 1 --2- 172.21.15.153:0/2994127715 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f833010e030 0x7f83301a0740 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f8320003de0 tx=0x7f8320011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:55:31.415 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:30.711+0000 7f8334a00700 1 -- 172.21.15.153:0/2994127715 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8320003930 con 0x7f833010e030 2024-03-21T09:55:31.415 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:30.711+0000 7f8334a00700 1 -- 172.21.15.153:0/2994127715 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8320003a90 con 0x7f833010e030 2024-03-21T09:55:31.416 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:30.711+0000 7f8334a00700 1 -- 172.21.15.153:0/2994127715 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f83200196c0 con 0x7f833010e030 2024-03-21T09:55:31.416 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:30.711+0000 7f8335e00700 1 -- 172.21.15.153:0/2994127715 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83301a0e80 con 0x7f833010e030 2024-03-21T09:55:31.416 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:30.711+0000 7f8335e00700 1 -- 172.21.15.153:0/2994127715 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83301a1320 con 0x7f833010e030 2024-03-21T09:55:31.416 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:30.712+0000 7f8334a00700 1 -- 172.21.15.153:0/2994127715 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f8320019c30 con 0x7f833010e030 2024-03-21T09:55:31.416 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:30.712+0000 7f8334a00700 1 -- 172.21.15.153:0/2994127715 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f832002c3a0 con 0x7f833010e030 2024-03-21T09:55:31.416 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:30.712+0000 7f8335e00700 1 -- 172.21.15.153:0/2994127715 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83300623c0 con 0x7f833010e030 2024-03-21T09:55:31.416 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:30.715+0000 7f8334a00700 1 -- 172.21.15.153:0/2994127715 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7f833010eb60 con 0x7f833010e030 2024-03-21T09:55:31.416 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:30.791+0000 7f8335e00700 1 -- 172.21.15.153:0/2994127715 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f833010d930 con 0x7f833010e030 2024-03-21T09:55:31.416 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:30.792+0000 7f8334a00700 1 -- 172.21.15.153:0/2994127715 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+177 (secure 0 0 0) 0x7f832001abc0 con 0x7f833010e030 2024-03-21T09:55:31.416 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:30.793+0000 7f8335e00700 1 -- 172.21.15.153:0/2994127715 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f833010e030 msgr2=0x7f83301a0740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:55:31.416 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:30.793+0000 7f8335e00700 1 --2- 172.21.15.153:0/2994127715 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f833010e030 0x7f83301a0740 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f8320003de0 tx=0x7f8320011ab0 comp rx=0 tx=0).stop 2024-03-21T09:55:31.416 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:30.793+0000 7f8335e00700 1 -- 172.21.15.153:0/2994127715 shutdown_connections 2024-03-21T09:55:31.416 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:30.793+0000 7f8335e00700 1 --2- 172.21.15.153:0/2994127715 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f833010e030 0x7f83301a0740 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:55:31.416 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:30.793+0000 7f8335e00700 1 -- 172.21.15.153:0/2994127715 >> 172.21.15.153:0/2994127715 conn(0x7f833007bb20 msgr2=0x7f833007e2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:55:31.416 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:30.793+0000 7f8335e00700 1 -- 172.21.15.153:0/2994127715 shutdown_connections 2024-03-21T09:55:31.417 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:30.794+0000 7f8335e00700 1 -- 172.21.15.153:0/2994127715 wait complete. 2024-03-21T09:55:31.417 INFO:teuthology.orchestra.run.smithi153.stdout:Restarting the monitor... 2024-03-21T09:55:31.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:31 smithi153 ceph-mon[28750]: from='client.? 172.21.15.153:0/2994127715' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:55:31.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:31 smithi153 systemd[1]: Stopping Ceph mon.smithi153 for 070ed3e4-e769-11ee-95cd-87774f69a715... 2024-03-21T09:55:32.299 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:31 smithi153 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-mon-smithi153[28746]: 2024-03-21T09:55:31.842+0000 7f5aa1a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi153 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T09:55:32.299 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:31 smithi153 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-mon-smithi153[28746]: 2024-03-21T09:55:31.842+0000 7f5aa1a00700 -1 mon.smithi153@0(leader) e1 *** Got Signal Terminated *** 2024-03-21T09:55:32.299 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:32 smithi153 podman[28992]: 2024-03-21 09:55:32.028023226 +0000 UTC m=+0.317715511 container died 22cb43f4f5d6053bca480f13ee8e68c620becde0dc5391da10d42c215de5325e (image=quay.io/ceph/ceph:v18.2.1, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-mon-smithi153, GIT_CLEAN=True, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.29.1, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=-18.2.1, RELEASE=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240222) 2024-03-21T09:55:32.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:32 smithi153 podman[28992]: 2024-03-21 09:55:32.401749856 +0000 UTC m=+0.691442154 container cleanup 22cb43f4f5d6053bca480f13ee8e68c620becde0dc5391da10d42c215de5325e (image=quay.io/ceph/ceph:v18.2.1, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-mon-smithi153, org.label-schema.schema-version=1.0, io.buildah.version=1.29.1, CEPH_POINT_RELEASE=-18.2.1, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240222, RELEASE=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-03-21T09:55:32.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:32 smithi153 bash[28992]: ceph-070ed3e4-e769-11ee-95cd-87774f69a715-mon-smithi153 2024-03-21T09:55:33.006 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:32 smithi153 podman[29005]: 2024-03-21 09:55:32.577515743 +0000 UTC m=+0.536027715 container remove 22cb43f4f5d6053bca480f13ee8e68c620becde0dc5391da10d42c215de5325e (image=quay.io/ceph/ceph:v18.2.1, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-mon-smithi153, RELEASE=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.29.1, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=-18.2.1, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD) 2024-03-21T09:55:33.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 systemd[1]: ceph-070ed3e4-e769-11ee-95cd-87774f69a715@mon.smithi153.service: Deactivated successfully. 2024-03-21T09:55:33.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 systemd[1]: Stopped Ceph mon.smithi153 for 070ed3e4-e769-11ee-95cd-87774f69a715. 2024-03-21T09:55:33.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 systemd[1]: ceph-070ed3e4-e769-11ee-95cd-87774f69a715@mon.smithi153.service: Consumed 1.812s CPU time. 2024-03-21T09:55:33.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 systemd[1]: Starting Ceph mon.smithi153 for 070ed3e4-e769-11ee-95cd-87774f69a715... 2024-03-21T09:55:33.660 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 podman[29098]: 2024-03-21 09:55:33.40298014 +0000 UTC m=+0.128536599 container create 813963a95d695aa24cfc165fa64d61eefb808093dabd0ae5739a25830dddc77d (image=quay.io/ceph/ceph:v18.2.1, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-mon-smithi153, io.buildah.version=1.29.1, CEPH_POINT_RELEASE=-18.2.1, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, ceph=True, RELEASE=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240222, GIT_REPO=https://github.com/ceph/ceph-container.git) 2024-03-21T09:55:33.660 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 podman[29098]: 2024-03-21 09:55:33.309013865 +0000 UTC m=+0.034570313 image pull 5be31c24972a920012b90a9769e8313e2490c82aee752aefd49af9cf4c0f3fcf quay.io/ceph/ceph:v18.2.1 2024-03-21T09:55:33.660 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 podman[29098]: 2024-03-21 09:55:33.658257583 +0000 UTC m=+0.383837051 container init 813963a95d695aa24cfc165fa64d61eefb808093dabd0ae5739a25830dddc77d (image=quay.io/ceph/ceph:v18.2.1, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-mon-smithi153, GIT_REPO=https://github.com/ceph/ceph-container.git, io.buildah.version=1.29.1, CEPH_POINT_RELEASE=-18.2.1, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-03-21T09:55:33.759 INFO:teuthology.orchestra.run.smithi153.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-03-21T09:55:34.058 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 podman[29098]: 2024-03-21 09:55:33.669021023 +0000 UTC m=+0.394577514 container start 813963a95d695aa24cfc165fa64d61eefb808093dabd0ae5739a25830dddc77d (image=quay.io/ceph/ceph:v18.2.1, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-mon-smithi153, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=HEAD, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240222, io.buildah.version=1.29.1, CEPH_POINT_RELEASE=-18.2.1, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, GIT_REPO=https://github.com/ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-03-21T09:55:34.059 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: set uid:gid to 167:167 (ceph:ceph) 2024-03-21T09:55:34.059 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable), process ceph-mon, pid 2 2024-03-21T09:55:34.059 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: pidfile_write: ignore empty --pid-file 2024-03-21T09:55:34.059 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: load: jerasure load: lrc 2024-03-21T09:55:34.059 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: RocksDB version: 7.9.2 2024-03-21T09:55:34.059 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Git sha 0 2024-03-21T09:55:34.059 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Compile date 2023-12-11 22:07:34 2024-03-21T09:55:34.059 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: DB SUMMARY 2024-03-21T09:55:34.059 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: DB Session ID: 2XX617NL5LSQE0MZR2Q3 2024-03-21T09:55:34.059 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: CURRENT file: CURRENT 2024-03-21T09:55:34.059 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: IDENTITY file: IDENTITY 2024-03-21T09:55:34.059 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-03-21T09:55:34.060 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi153/store.db dir, Total Num: 1, files: 000008.sst 2024-03-21T09:55:34.060 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi153/store.db: 000009.log size: 98517 ; 2024-03-21T09:55:34.060 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.error_if_exists: 0 2024-03-21T09:55:34.060 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.create_if_missing: 0 2024-03-21T09:55:34.060 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.paranoid_checks: 1 2024-03-21T09:55:34.060 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-21T09:55:34.060 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-21T09:55:34.060 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-21T09:55:34.060 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.env: 0x55facda33720 2024-03-21T09:55:34.060 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.fs: PosixFileSystem 2024-03-21T09:55:34.060 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.info_log: 0x55fad0b852e0 2024-03-21T09:55:34.060 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.max_file_opening_threads: 16 2024-03-21T09:55:34.060 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.statistics: (nil) 2024-03-21T09:55:34.060 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.use_fsync: 0 2024-03-21T09:55:34.061 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.max_log_file_size: 0 2024-03-21T09:55:34.061 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-21T09:55:34.061 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-21T09:55:34.061 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.keep_log_file_num: 1000 2024-03-21T09:55:34.061 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.recycle_log_file_num: 0 2024-03-21T09:55:34.061 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.allow_fallocate: 1 2024-03-21T09:55:34.061 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.allow_mmap_reads: 0 2024-03-21T09:55:34.062 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.allow_mmap_writes: 0 2024-03-21T09:55:34.062 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.use_direct_reads: 0 2024-03-21T09:55:34.062 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-21T09:55:34.062 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.create_missing_column_families: 0 2024-03-21T09:55:34.062 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.db_log_dir: 2024-03-21T09:55:34.062 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.wal_dir: 2024-03-21T09:55:34.062 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-21T09:55:34.062 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-21T09:55:34.062 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-21T09:55:34.062 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-21T09:55:34.062 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-21T09:55:34.063 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-21T09:55:34.063 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.advise_random_on_open: 1 2024-03-21T09:55:34.063 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.db_write_buffer_size: 0 2024-03-21T09:55:34.063 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.write_buffer_manager: 0x55facfdf6140 2024-03-21T09:55:34.063 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-21T09:55:34.063 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-21T09:55:34.063 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-21T09:55:34.063 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.rate_limiter: (nil) 2024-03-21T09:55:34.063 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-21T09:55:34.063 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.wal_recovery_mode: 2 2024-03-21T09:55:34.063 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.enable_thread_tracking: 0 2024-03-21T09:55:34.063 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.enable_pipelined_write: 0 2024-03-21T09:55:34.063 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.unordered_write: 0 2024-03-21T09:55:34.064 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-21T09:55:34.064 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-21T09:55:34.064 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-21T09:55:34.064 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-21T09:55:34.064 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.row_cache: None 2024-03-21T09:55:34.064 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.wal_filter: None 2024-03-21T09:55:34.064 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-21T09:55:34.064 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.allow_ingest_behind: 0 2024-03-21T09:55:34.064 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.two_write_queues: 0 2024-03-21T09:55:34.064 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.manual_wal_flush: 0 2024-03-21T09:55:34.064 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.wal_compression: 0 2024-03-21T09:55:34.064 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.atomic_flush: 0 2024-03-21T09:55:34.064 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-21T09:55:34.064 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-21T09:55:34.065 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-21T09:55:34.065 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.log_readahead_size: 0 2024-03-21T09:55:34.065 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-21T09:55:34.065 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.best_efforts_recovery: 0 2024-03-21T09:55:34.065 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-21T09:55:34.065 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-21T09:55:34.065 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.allow_data_in_errors: 0 2024-03-21T09:55:34.065 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.db_host_id: __hostname__ 2024-03-21T09:55:34.065 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-21T09:55:34.065 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.max_background_jobs: 2 2024-03-21T09:55:34.065 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.max_background_compactions: -1 2024-03-21T09:55:34.065 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.max_subcompactions: 1 2024-03-21T09:55:34.065 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-21T09:55:34.065 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-21T09:55:34.065 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-21T09:55:34.066 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.max_total_wal_size: 0 2024-03-21T09:55:34.066 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-21T09:55:34.066 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-21T09:55:34.066 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-21T09:55:34.066 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-21T09:55:34.066 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.max_open_files: -1 2024-03-21T09:55:34.066 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.bytes_per_sync: 0 2024-03-21T09:55:34.066 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-21T09:55:34.066 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-21T09:55:34.066 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.compaction_readahead_size: 0 2024-03-21T09:55:34.066 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.max_background_flushes: -1 2024-03-21T09:55:34.066 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Compression algorithms supported: 2024-03-21T09:55:34.066 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-21T09:55:34.066 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: kZSTD supported: 0 2024-03-21T09:55:34.067 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: kXpressCompression supported: 0 2024-03-21T09:55:34.067 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: kLZ4HCCompression supported: 1 2024-03-21T09:55:34.067 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: kZlibCompression supported: 1 2024-03-21T09:55:34.067 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: kSnappyCompression supported: 1 2024-03-21T09:55:34.067 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: kLZ4Compression supported: 1 2024-03-21T09:55:34.067 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: kBZip2Compression supported: 0 2024-03-21T09:55:34.067 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-21T09:55:34.067 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-21T09:55:34.067 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi153/store.db/MANIFEST-000010 2024-03-21T09:55:34.067 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-21T09:55:34.067 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-21T09:55:34.067 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.merge_operator: 2024-03-21T09:55:34.067 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.compaction_filter: None 2024-03-21T09:55:34.067 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.compaction_filter_factory: None 2024-03-21T09:55:34.067 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.sst_partitioner_factory: None 2024-03-21T09:55:34.068 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-21T09:55:34.068 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-21T09:55:34.068 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55fad0b853e0) 2024-03-21T09:55:34.068 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: cache_index_and_filter_blocks: 1 2024-03-21T09:55:34.068 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-21T09:55:34.068 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-21T09:55:34.068 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: pin_top_level_index_and_filter: 1 2024-03-21T09:55:34.068 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: index_type: 0 2024-03-21T09:55:34.068 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: data_block_index_type: 0 2024-03-21T09:55:34.068 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: index_shortening: 1 2024-03-21T09:55:34.068 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-21T09:55:34.068 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: checksum: 4 2024-03-21T09:55:34.068 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: no_block_cache: 0 2024-03-21T09:55:34.068 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: block_cache: 0x55facfe97350 2024-03-21T09:55:34.069 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: block_cache_name: BinnedLRUCache 2024-03-21T09:55:34.069 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: block_cache_options: 2024-03-21T09:55:34.069 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: capacity : 536870912 2024-03-21T09:55:34.069 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: num_shard_bits : 4 2024-03-21T09:55:34.069 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: strict_capacity_limit : 0 2024-03-21T09:55:34.069 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: high_pri_pool_ratio: 0.000 2024-03-21T09:55:34.069 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: block_cache_compressed: (nil) 2024-03-21T09:55:34.069 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: persistent_cache: (nil) 2024-03-21T09:55:34.069 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: block_size: 4096 2024-03-21T09:55:34.069 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: block_size_deviation: 10 2024-03-21T09:55:34.069 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: block_restart_interval: 16 2024-03-21T09:55:34.069 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: index_block_restart_interval: 1 2024-03-21T09:55:34.069 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: metadata_block_size: 4096 2024-03-21T09:55:34.069 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: partition_filters: 0 2024-03-21T09:55:34.070 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: use_delta_encoding: 1 2024-03-21T09:55:34.070 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: filter_policy: bloomfilter 2024-03-21T09:55:34.070 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: whole_key_filtering: 1 2024-03-21T09:55:34.070 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: verify_compression: 0 2024-03-21T09:55:34.070 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: read_amp_bytes_per_bit: 0 2024-03-21T09:55:34.070 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: format_version: 5 2024-03-21T09:55:34.070 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: enable_index_compression: 1 2024-03-21T09:55:34.070 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: block_align: 0 2024-03-21T09:55:34.070 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: max_auto_readahead_size: 262144 2024-03-21T09:55:34.070 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: prepopulate_block_cache: 0 2024-03-21T09:55:34.070 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: initial_auto_readahead_size: 8192 2024-03-21T09:55:34.070 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: num_file_reads_for_auto_readahead: 2 2024-03-21T09:55:34.070 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.write_buffer_size: 33554432 2024-03-21T09:55:34.070 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.max_write_buffer_number: 2 2024-03-21T09:55:34.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.compression: NoCompression 2024-03-21T09:55:34.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.bottommost_compression: Disabled 2024-03-21T09:55:34.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.prefix_extractor: nullptr 2024-03-21T09:55:34.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-21T09:55:34.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.num_levels: 7 2024-03-21T09:55:34.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-21T09:55:34.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-21T09:55:34.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-21T09:55:34.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-21T09:55:34.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-21T09:55:34.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-21T09:55:34.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-21T09:55:34.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-21T09:55:34.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-21T09:55:34.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-21T09:55:34.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-21T09:55:34.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-21T09:55:34.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-21T09:55:34.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.compression_opts.level: 32767 2024-03-21T09:55:34.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.compression_opts.strategy: 0 2024-03-21T09:55:34.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-21T09:55:34.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-21T09:55:34.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-21T09:55:34.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-21T09:55:34.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.compression_opts.enabled: false 2024-03-21T09:55:34.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-21T09:55:34.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-21T09:55:34.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-21T09:55:34.073 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-21T09:55:34.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.target_file_size_base: 67108864 2024-03-21T09:55:34.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-21T09:55:34.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-21T09:55:34.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-21T09:55:34.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-21T09:55:34.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-21T09:55:34.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-21T09:55:34.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-21T09:55:34.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-21T09:55:34.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-21T09:55:34.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-21T09:55:34.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-21T09:55:34.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-21T09:55:34.074 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-21T09:55:34.075 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-21T09:55:34.075 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.arena_block_size: 1048576 2024-03-21T09:55:34.075 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-21T09:55:34.075 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-21T09:55:34.075 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.disable_auto_compactions: 0 2024-03-21T09:55:34.075 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-21T09:55:34.075 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-21T09:55:34.075 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-21T09:55:34.075 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-21T09:55:34.075 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-21T09:55:34.075 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-21T09:55:34.075 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-21T09:55:34.075 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-21T09:55:34.075 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-21T09:55:34.075 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-21T09:55:34.076 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.table_properties_collectors: 2024-03-21T09:55:34.076 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.inplace_update_support: 0 2024-03-21T09:55:34.076 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-21T09:55:34.076 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-21T09:55:34.076 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-21T09:55:34.076 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-21T09:55:34.076 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.bloom_locality: 0 2024-03-21T09:55:34.076 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.max_successive_merges: 0 2024-03-21T09:55:34.076 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-21T09:55:34.076 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.paranoid_file_checks: 0 2024-03-21T09:55:34.076 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.force_consistency_checks: 1 2024-03-21T09:55:34.076 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.report_bg_io_stats: 0 2024-03-21T09:55:34.076 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.ttl: 2592000 2024-03-21T09:55:34.076 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-21T09:55:34.076 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-21T09:55:34.077 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-21T09:55:34.077 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.enable_blob_files: false 2024-03-21T09:55:34.077 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.min_blob_size: 0 2024-03-21T09:55:34.077 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.blob_file_size: 268435456 2024-03-21T09:55:34.077 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-21T09:55:34.077 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-21T09:55:34.077 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-21T09:55:34.077 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-21T09:55:34.077 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-21T09:55:34.077 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.blob_file_starting_level: 0 2024-03-21T09:55:34.077 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-21T09:55:34.077 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi153/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-03-21T09:55:34.077 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-03-21T09:55:34.077 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: cbb17578-3ee9-4747-bb14-d021fbdfda98 2024-03-21T09:55:34.077 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711014933723674, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-03-21T09:55:34.078 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-03-21T09:55:34.078 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711014933725677, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 93373, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 328, "table_properties": {"data_size": 91390, "index_size": 265, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13388, "raw_average_key_size": 49, "raw_value_size": 84078, "raw_average_value_size": 310, "num_data_blocks": 11, "num_entries": 271, "num_filter_entries": 271, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1711014933, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "cbb17578-3ee9-4747-bb14-d021fbdfda98", "db_session_id": "2XX617NL5LSQE0MZR2Q3", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-03-21T09:55:34.078 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711014933725822, "job": 1, "event": "recovery_finished"} 2024-03-21T09:55:34.078 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-03-21T09:55:34.078 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi153/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-21T09:55:34.078 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55facff34000 2024-03-21T09:55:34.078 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: DB pointer 0x55facff20000 2024-03-21T09:55:34.078 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-21T09:55:34.078 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-21T09:55:34.078 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: ** DB Stats ** 2024-03-21T09:55:34.079 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-21T09:55:34.079 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-21T09:55:34.079 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-21T09:55:34.079 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-21T09:55:34.079 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-21T09:55:34.079 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-21T09:55:34.079 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-21T09:55:34.079 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: 2024-03-21T09:55:34.079 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: ** Compaction Stats [default] ** 2024-03-21T09:55:34.079 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-21T09:55:34.079 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-21T09:55:34.079 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: L0 2/0 93.06 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 74.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-21T09:55:34.079 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: Sum 2/0 93.06 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 74.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-21T09:55:34.079 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 74.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-21T09:55:34.080 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: 2024-03-21T09:55:34.080 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: ** Compaction Stats [default] ** 2024-03-21T09:55:34.080 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-21T09:55:34.080 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-21T09:55:34.080 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 74.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-21T09:55:34.080 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: 2024-03-21T09:55:34.080 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-21T09:55:34.080 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: 2024-03-21T09:55:34.080 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-21T09:55:34.080 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-21T09:55:34.080 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-21T09:55:34.080 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-21T09:55:34.080 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-21T09:55:34.080 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-21T09:55:34.081 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: Cumulative compaction: 0.00 GB write, 14.11 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-21T09:55:34.081 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: Interval compaction: 0.00 GB write, 14.11 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-21T09:55:34.081 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-21T09:55:34.081 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: Block cache BinnedLRUCache@0x55facfe97350#2 capacity: 512.00 MB usage: 1.36 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-03-21T09:55:34.081 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2024-03-21T09:55:34.081 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: 2024-03-21T09:55:34.081 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-21T09:55:34.081 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: starting mon.smithi153 rank 0 at public addrs [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] at bind addrs [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi153 fsid 070ed3e4-e769-11ee-95cd-87774f69a715 2024-03-21T09:55:34.081 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: mon.smithi153@-1(???) e1 preinit fsid 070ed3e4-e769-11ee-95cd-87774f69a715 2024-03-21T09:55:34.081 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: mon.smithi153@-1(???).mds e1 new map 2024-03-21T09:55:34.081 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: mon.smithi153@-1(???).mds e1 print_map 2024-03-21T09:55:34.081 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: e1 2024-03-21T09:55:34.081 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-21T09:55:34.081 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T09:55:34.082 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: legacy client fscid: -1 2024-03-21T09:55:34.082 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: 2024-03-21T09:55:34.082 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: No filesystems configured 2024-03-21T09:55:34.082 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: mon.smithi153@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-21T09:55:34.082 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: mon.smithi153@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-21T09:55:34.082 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: mon.smithi153@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-21T09:55:34.082 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: mon.smithi153@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-21T09:55:34.082 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: mon.smithi153@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-03-21T09:55:34.082 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 bash[29098]: 813963a95d695aa24cfc165fa64d61eefb808093dabd0ae5739a25830dddc77d 2024-03-21T09:55:34.082 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 systemd[1]: Started Ceph mon.smithi153 for 070ed3e4-e769-11ee-95cd-87774f69a715. 2024-03-21T09:55:34.082 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: mon.smithi153 is new leader, mons smithi153 in quorum (ranks 0) 2024-03-21T09:55:34.082 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: monmap e1: 1 mons at {smithi153=[v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-21T09:55:34.082 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: fsmap 2024-03-21T09:55:34.082 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: osdmap e1: 0 total, 0 up, 0 in 2024-03-21T09:55:34.083 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:33 smithi153 ceph-mon[29117]: mgrmap e1: no daemons active 2024-03-21T09:55:36.236 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:35.520+0000 7f1f8e400700 1 Processor -- start 2024-03-21T09:55:36.238 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:35.521+0000 7f1f8e400700 1 -- start start 2024-03-21T09:55:36.238 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:35.521+0000 7f1f8e400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f8810e030 0x7f1f8810e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:55:36.238 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:35.521+0000 7f1f8e400700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8810ea20 con 0x7f1f8810e030 2024-03-21T09:55:36.238 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:35.522+0000 7f1f87400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f8810e030 0x7f1f8810e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:55:36.238 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:35.522+0000 7f1f87400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f8810e030 0x7f1f8810e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:49646/0 (socket says 172.21.15.153:49646) 2024-03-21T09:55:36.238 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:35.522+0000 7f1f87400700 1 -- 172.21.15.153:0/3549440923 learned_addr learned my addr 172.21.15.153:0/3549440923 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T09:55:36.238 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:35.522+0000 7f1f87400700 1 -- 172.21.15.153:0/3549440923 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f8810eb60 con 0x7f1f8810e030 2024-03-21T09:55:36.238 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:35.522+0000 7f1f87400700 1 --2- 172.21.15.153:0/3549440923 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f8810e030 0x7f1f8810e450 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f1f780098a0 tx=0x7f1f78009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b88e0ced89b00fa6 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:55:36.239 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:35.523+0000 7f1f86000700 1 -- 172.21.15.153:0/3549440923 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1f78004180 con 0x7f1f8810e030 2024-03-21T09:55:36.239 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:35.523+0000 7f1f86000700 1 -- 172.21.15.153:0/3549440923 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1f780042e0 con 0x7f1f8810e030 2024-03-21T09:55:36.239 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:35.523+0000 7f1f86000700 1 -- 172.21.15.153:0/3549440923 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1f78011a50 con 0x7f1f8810e030 2024-03-21T09:55:36.239 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:35.524+0000 7f1f8e400700 1 -- 172.21.15.153:0/3549440923 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f8810e030 msgr2=0x7f1f8810e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:55:36.239 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:35.524+0000 7f1f8e400700 1 --2- 172.21.15.153:0/3549440923 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f8810e030 0x7f1f8810e450 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f1f780098a0 tx=0x7f1f78009bb0 comp rx=0 tx=0).stop 2024-03-21T09:55:36.239 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:35.524+0000 7f1f8e400700 1 -- 172.21.15.153:0/3549440923 shutdown_connections 2024-03-21T09:55:36.239 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:35.524+0000 7f1f8e400700 1 --2- 172.21.15.153:0/3549440923 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f8810e030 0x7f1f8810e450 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:55:36.239 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:35.524+0000 7f1f8e400700 1 -- 172.21.15.153:0/3549440923 >> 172.21.15.153:0/3549440923 conn(0x7f1f8807bb20 msgr2=0x7f1f8807df80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:55:36.243 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:35.524+0000 7f1f8e400700 1 -- 172.21.15.153:0/3549440923 shutdown_connections 2024-03-21T09:55:36.243 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:35.524+0000 7f1f8e400700 1 -- 172.21.15.153:0/3549440923 wait complete. 2024-03-21T09:55:36.243 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:35.525+0000 7f1f8e400700 1 Processor -- start 2024-03-21T09:55:36.243 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:35.525+0000 7f1f8e400700 1 -- start start 2024-03-21T09:55:36.243 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:35.526+0000 7f1f8e400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f8810e030 0x7f1f881a0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:55:36.243 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:35.526+0000 7f1f8e400700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f881a0cb0 con 0x7f1f8810e030 2024-03-21T09:55:36.243 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:35.526+0000 7f1f87400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f8810e030 0x7f1f881a0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:55:36.243 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:35.526+0000 7f1f87400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f8810e030 0x7f1f881a0770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:49654/0 (socket says 172.21.15.153:49654) 2024-03-21T09:55:36.243 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:35.526+0000 7f1f87400700 1 -- 172.21.15.153:0/1936109619 learned_addr learned my addr 172.21.15.153:0/1936109619 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T09:55:36.243 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:35.526+0000 7f1f87400700 1 -- 172.21.15.153:0/1936109619 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f78009550 con 0x7f1f8810e030 2024-03-21T09:55:36.243 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:35.527+0000 7f1f87400700 1 --2- 172.21.15.153:0/1936109619 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f8810e030 0x7f1f881a0770 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f1f78003de0 tx=0x7f1f78011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:55:36.243 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:35.527+0000 7f1f7fe00700 1 -- 172.21.15.153:0/1936109619 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1f78003930 con 0x7f1f8810e030 2024-03-21T09:55:36.243 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:35.527+0000 7f1f7fe00700 1 -- 172.21.15.153:0/1936109619 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1f78003a90 con 0x7f1f8810e030 2024-03-21T09:55:36.244 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:35.527+0000 7f1f8e400700 1 -- 172.21.15.153:0/1936109619 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f881a0eb0 con 0x7f1f8810e030 2024-03-21T09:55:36.244 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:35.527+0000 7f1f7fe00700 1 -- 172.21.15.153:0/1936109619 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1f780196c0 con 0x7f1f8810e030 2024-03-21T09:55:36.244 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:35.527+0000 7f1f8e400700 1 -- 172.21.15.153:0/1936109619 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f881a1350 con 0x7f1f8810e030 2024-03-21T09:55:36.244 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:35.527+0000 7f1f7fe00700 1 -- 172.21.15.153:0/1936109619 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f1f78019ca0 con 0x7f1f8810e030 2024-03-21T09:55:36.244 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:35.527+0000 7f1f7fe00700 1 -- 172.21.15.153:0/1936109619 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f1f7802c3a0 con 0x7f1f8810e030 2024-03-21T09:55:36.244 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:35.528+0000 7f1f8e400700 1 -- 172.21.15.153:0/1936109619 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f8819a840 con 0x7f1f8810e030 2024-03-21T09:55:36.244 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:35.532+0000 7f1f7fe00700 1 -- 172.21.15.153:0/1936109619 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7f1f8810eb60 con 0x7f1f8810e030 2024-03-21T09:55:36.244 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:35.608+0000 7f1f8e400700 1 -- 172.21.15.153:0/1936109619 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f1f880619a0 con 0x7f1f8810e030 2024-03-21T09:55:36.244 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:35.614+0000 7f1f7fe00700 1 -- 172.21.15.153:0/1936109619 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f1f7801abc0 con 0x7f1f8810e030 2024-03-21T09:55:36.245 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:35.615+0000 7f1f8e400700 1 -- 172.21.15.153:0/1936109619 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f8810e030 msgr2=0x7f1f881a0770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:55:36.245 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:35.615+0000 7f1f8e400700 1 --2- 172.21.15.153:0/1936109619 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f8810e030 0x7f1f881a0770 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f1f78003de0 tx=0x7f1f78011ab0 comp rx=0 tx=0).stop 2024-03-21T09:55:36.245 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:35.615+0000 7f1f8e400700 1 -- 172.21.15.153:0/1936109619 shutdown_connections 2024-03-21T09:55:36.245 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:35.615+0000 7f1f8e400700 1 --2- 172.21.15.153:0/1936109619 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f8810e030 0x7f1f881a0770 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:55:36.245 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:35.615+0000 7f1f8e400700 1 -- 172.21.15.153:0/1936109619 >> 172.21.15.153:0/1936109619 conn(0x7f1f8807bb20 msgr2=0x7f1f8807e360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:55:36.245 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:35.616+0000 7f1f8e400700 1 -- 172.21.15.153:0/1936109619 shutdown_connections 2024-03-21T09:55:36.245 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:35.616+0000 7f1f8e400700 1 -- 172.21.15.153:0/1936109619 wait complete. 2024-03-21T09:55:36.245 INFO:teuthology.orchestra.run.smithi153.stdout:Wrote config to /etc/ceph/ceph.conf 2024-03-21T09:55:36.274 INFO:teuthology.orchestra.run.smithi153.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-03-21T09:55:36.274 INFO:teuthology.orchestra.run.smithi153.stdout:Creating mgr... 2024-03-21T09:55:36.275 INFO:teuthology.orchestra.run.smithi153.stdout:Verifying port 0.0.0.0:9283 ... 2024-03-21T09:55:36.276 INFO:teuthology.orchestra.run.smithi153.stdout:Verifying port 0.0.0.0:8765 ... 2024-03-21T09:55:36.277 INFO:teuthology.orchestra.run.smithi153.stdout:Verifying port 0.0.0.0:8443 ... 2024-03-21T09:55:36.697 INFO:teuthology.orchestra.run.smithi153.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-070ed3e4-e769-11ee-95cd-87774f69a715@mgr.smithi153.tluxni 2024-03-21T09:55:36.697 INFO:teuthology.orchestra.run.smithi153.stdout:systemctl: stderr Failed to reset failed state of unit ceph-070ed3e4-e769-11ee-95cd-87774f69a715@mgr.smithi153.tluxni.service: Unit ceph-070ed3e4-e769-11ee-95cd-87774f69a715@mgr.smithi153.tluxni.service not loaded. 2024-03-21T09:55:36.951 INFO:teuthology.orchestra.run.smithi153.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-070ed3e4-e769-11ee-95cd-87774f69a715.target.wants/ceph-070ed3e4-e769-11ee-95cd-87774f69a715@mgr.smithi153.tluxni.service → /etc/systemd/system/ceph-070ed3e4-e769-11ee-95cd-87774f69a715@.service. 2024-03-21T09:55:36.979 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:36 smithi153 ceph-mon[29117]: from='client.? 172.21.15.153:0/1936109619' entity='client.admin' 2024-03-21T09:55:37.685 INFO:teuthology.orchestra.run.smithi153.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-21T09:55:37.685 INFO:teuthology.orchestra.run.smithi153.stdout:systemctl: stdout disabled 2024-03-21T09:55:37.702 INFO:teuthology.orchestra.run.smithi153.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-21T09:55:37.703 INFO:teuthology.orchestra.run.smithi153.stdout:systemctl: stdout inactive 2024-03-21T09:55:37.703 INFO:teuthology.orchestra.run.smithi153.stdout:firewalld.service is not enabled 2024-03-21T09:55:37.703 INFO:teuthology.orchestra.run.smithi153.stdout:Not possible to enable service . firewalld.service is not available 2024-03-21T09:55:37.733 INFO:teuthology.orchestra.run.smithi153.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-21T09:55:37.734 INFO:teuthology.orchestra.run.smithi153.stdout:systemctl: stdout disabled 2024-03-21T09:55:37.752 INFO:teuthology.orchestra.run.smithi153.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-21T09:55:37.753 INFO:teuthology.orchestra.run.smithi153.stdout:systemctl: stdout inactive 2024-03-21T09:55:37.753 INFO:teuthology.orchestra.run.smithi153.stdout:firewalld.service is not enabled 2024-03-21T09:55:37.753 INFO:teuthology.orchestra.run.smithi153.stdout:Not possible to open ports <[9283, 8765, 8443]>. firewalld.service is not available 2024-03-21T09:55:37.753 INFO:teuthology.orchestra.run.smithi153.stdout:Waiting for mgr to start... 2024-03-21T09:55:37.754 INFO:teuthology.orchestra.run.smithi153.stdout:Waiting for mgr... 2024-03-21T09:55:38.796 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:38 smithi153 ceph-mon[29117]: from='client.? 172.21.15.153:0/1482124503' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-21T09:55:39.173 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout 2024-03-21T09:55:39.174 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout { 2024-03-21T09:55:39.175 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "fsid": "070ed3e4-e769-11ee-95cd-87774f69a715", 2024-03-21T09:55:39.175 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "health": { 2024-03-21T09:55:39.175 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-21T09:55:39.175 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-21T09:55:39.175 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-21T09:55:39.175 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout }, 2024-03-21T09:55:39.175 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-21T09:55:39.177 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-21T09:55:39.177 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout 0 2024-03-21T09:55:39.177 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout ], 2024-03-21T09:55:39.177 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-21T09:55:39.177 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "smithi153" 2024-03-21T09:55:39.177 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout ], 2024-03-21T09:55:39.177 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "quorum_age": 4, 2024-03-21T09:55:39.177 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-21T09:55:39.177 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T09:55:39.178 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-03-21T09:55:39.178 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-21T09:55:39.178 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout }, 2024-03-21T09:55:39.178 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-21T09:55:39.178 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T09:55:39.178 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-21T09:55:39.178 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-21T09:55:39.178 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-21T09:55:39.186 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-21T09:55:39.186 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-21T09:55:39.186 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-21T09:55:39.186 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout }, 2024-03-21T09:55:39.186 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-21T09:55:39.186 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-21T09:55:39.186 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-21T09:55:39.186 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-21T09:55:39.186 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-21T09:55:39.186 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-21T09:55:39.186 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-21T09:55:39.186 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-21T09:55:39.187 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-21T09:55:39.187 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout }, 2024-03-21T09:55:39.187 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-21T09:55:39.187 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T09:55:39.187 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-21T09:55:39.187 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-21T09:55:39.187 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout }, 2024-03-21T09:55:39.187 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-21T09:55:39.187 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-21T09:55:39.187 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-21T09:55:39.187 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-21T09:55:39.187 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-21T09:55:39.187 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-21T09:55:39.187 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "restful" 2024-03-21T09:55:39.188 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout ], 2024-03-21T09:55:39.188 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-21T09:55:39.188 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout }, 2024-03-21T09:55:39.188 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-21T09:55:39.188 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T09:55:39.188 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "modified": "2024-03-21T09:55:19.360779+0000", 2024-03-21T09:55:39.188 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-21T09:55:39.188 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout }, 2024-03-21T09:55:39.188 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-21T09:55:39.188 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout } 2024-03-21T09:55:39.188 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:38.342+0000 7f6568c00700 1 Processor -- start 2024-03-21T09:55:39.188 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:38.342+0000 7f6568c00700 1 -- start start 2024-03-21T09:55:39.188 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:38.343+0000 7f6568c00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f656407ad60 0x7f65640791c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:55:39.188 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:38.343+0000 7f6568c00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6564079790 con 0x7f656407ad60 2024-03-21T09:55:39.189 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:38.343+0000 7f6561e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f656407ad60 0x7f65640791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:55:39.189 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:38.343+0000 7f6561e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f656407ad60 0x7f65640791c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:49686/0 (socket says 172.21.15.153:49686) 2024-03-21T09:55:39.189 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:38.343+0000 7f6561e00700 1 -- 172.21.15.153:0/2796515076 learned_addr learned my addr 172.21.15.153:0/2796515076 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T09:55:39.189 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:38.344+0000 7f6561e00700 1 -- 172.21.15.153:0/2796515076 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65640798d0 con 0x7f656407ad60 2024-03-21T09:55:39.189 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:38.344+0000 7f6561e00700 1 --2- 172.21.15.153:0/2796515076 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f656407ad60 0x7f65640791c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f65540098a0 tx=0x7f6554009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b49a59bf308db64d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:55:39.189 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:38.344+0000 7f6560a00700 1 -- 172.21.15.153:0/2796515076 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6554004180 con 0x7f656407ad60 2024-03-21T09:55:39.189 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:38.345+0000 7f6560a00700 1 -- 172.21.15.153:0/2796515076 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f65540042e0 con 0x7f656407ad60 2024-03-21T09:55:39.189 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:38.345+0000 7f6560a00700 1 -- 172.21.15.153:0/2796515076 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6554011a50 con 0x7f656407ad60 2024-03-21T09:55:39.189 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:38.345+0000 7f6568c00700 1 -- 172.21.15.153:0/2796515076 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f656407ad60 msgr2=0x7f65640791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:55:39.189 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:38.345+0000 7f6568c00700 1 --2- 172.21.15.153:0/2796515076 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f656407ad60 0x7f65640791c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f65540098a0 tx=0x7f6554009bb0 comp rx=0 tx=0).stop 2024-03-21T09:55:39.189 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:38.345+0000 7f6568c00700 1 -- 172.21.15.153:0/2796515076 shutdown_connections 2024-03-21T09:55:39.189 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:38.345+0000 7f6568c00700 1 --2- 172.21.15.153:0/2796515076 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f656407ad60 0x7f65640791c0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:55:39.189 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:38.345+0000 7f6568c00700 1 -- 172.21.15.153:0/2796515076 >> 172.21.15.153:0/2796515076 conn(0x7f65641019b0 msgr2=0x7f6564103e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:55:39.189 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:38.345+0000 7f6568c00700 1 -- 172.21.15.153:0/2796515076 shutdown_connections 2024-03-21T09:55:39.190 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:38.346+0000 7f6568c00700 1 -- 172.21.15.153:0/2796515076 wait complete. 2024-03-21T09:55:39.190 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:38.346+0000 7f6568c00700 1 Processor -- start 2024-03-21T09:55:39.190 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:38.347+0000 7f6568c00700 1 -- start start 2024-03-21T09:55:39.190 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:38.347+0000 7f6568c00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f656407ad60 0x7f656419c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:55:39.190 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:38.347+0000 7f6568c00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f656419c830 con 0x7f656407ad60 2024-03-21T09:55:39.190 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:38.347+0000 7f6561e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f656407ad60 0x7f656419c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:55:39.190 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:38.347+0000 7f6561e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f656407ad60 0x7f656419c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:49700/0 (socket says 172.21.15.153:49700) 2024-03-21T09:55:39.190 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:38.347+0000 7f6561e00700 1 -- 172.21.15.153:0/1482124503 learned_addr learned my addr 172.21.15.153:0/1482124503 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T09:55:39.190 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:38.348+0000 7f6561e00700 1 -- 172.21.15.153:0/1482124503 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6554009550 con 0x7f656407ad60 2024-03-21T09:55:39.190 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:38.348+0000 7f6561e00700 1 --2- 172.21.15.153:0/1482124503 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f656407ad60 0x7f656419c2f0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6554003de0 tx=0x7f6554011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:55:39.190 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:38.348+0000 7f655aa00700 1 -- 172.21.15.153:0/1482124503 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6554003930 con 0x7f656407ad60 2024-03-21T09:55:39.190 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:38.348+0000 7f655aa00700 1 -- 172.21.15.153:0/1482124503 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6554003a90 con 0x7f656407ad60 2024-03-21T09:55:39.190 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:38.348+0000 7f6568c00700 1 -- 172.21.15.153:0/1482124503 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f656419ca30 con 0x7f656407ad60 2024-03-21T09:55:39.191 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:38.348+0000 7f655aa00700 1 -- 172.21.15.153:0/1482124503 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6554019730 con 0x7f656407ad60 2024-03-21T09:55:39.191 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:38.348+0000 7f6568c00700 1 -- 172.21.15.153:0/1482124503 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f656419ced0 con 0x7f656407ad60 2024-03-21T09:55:39.191 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:38.349+0000 7f655aa00700 1 -- 172.21.15.153:0/1482124503 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f6554019c80 con 0x7f656407ad60 2024-03-21T09:55:39.191 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:38.349+0000 7f655aa00700 1 -- 172.21.15.153:0/1482124503 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f655402c3a0 con 0x7f656407ad60 2024-03-21T09:55:39.191 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:38.350+0000 7f6568c00700 1 -- 172.21.15.153:0/1482124503 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65640623c0 con 0x7f656407ad60 2024-03-21T09:55:39.191 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:38.352+0000 7f655aa00700 1 -- 172.21.15.153:0/1482124503 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7f655401abc0 con 0x7f656407ad60 2024-03-21T09:55:39.191 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:38.430+0000 7f6568c00700 1 -- 172.21.15.153:0/1482124503 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f656419f890 con 0x7f656407ad60 2024-03-21T09:55:39.191 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:38.431+0000 7f655aa00700 1 -- 172.21.15.153:0/1482124503 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1246 (secure 0 0 0) 0x7f655401a430 con 0x7f656407ad60 2024-03-21T09:55:39.191 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:38.432+0000 7f6568c00700 1 -- 172.21.15.153:0/1482124503 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f656407ad60 msgr2=0x7f656419c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:55:39.191 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:38.432+0000 7f6568c00700 1 --2- 172.21.15.153:0/1482124503 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f656407ad60 0x7f656419c2f0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6554003de0 tx=0x7f6554011ab0 comp rx=0 tx=0).stop 2024-03-21T09:55:39.191 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:38.432+0000 7f6568c00700 1 -- 172.21.15.153:0/1482124503 shutdown_connections 2024-03-21T09:55:39.191 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:38.432+0000 7f6568c00700 1 --2- 172.21.15.153:0/1482124503 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f656407ad60 0x7f656419c2f0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:55:39.191 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:38.432+0000 7f6568c00700 1 -- 172.21.15.153:0/1482124503 >> 172.21.15.153:0/1482124503 conn(0x7f65641019b0 msgr2=0x7f65641030c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:55:39.191 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:38.432+0000 7f6568c00700 1 -- 172.21.15.153:0/1482124503 shutdown_connections 2024-03-21T09:55:39.192 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:38.433+0000 7f6568c00700 1 -- 172.21.15.153:0/1482124503 wait complete. 2024-03-21T09:55:39.192 INFO:teuthology.orchestra.run.smithi153.stdout:mgr not available, waiting (1/15)... 2024-03-21T09:55:42.221 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:41 smithi153 ceph-mon[29117]: from='client.? 172.21.15.153:0/785836858' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-21T09:55:42.624 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout 2024-03-21T09:55:42.624 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout { 2024-03-21T09:55:42.624 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "fsid": "070ed3e4-e769-11ee-95cd-87774f69a715", 2024-03-21T09:55:42.625 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "health": { 2024-03-21T09:55:42.625 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-21T09:55:42.625 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-21T09:55:42.625 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-21T09:55:42.625 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout }, 2024-03-21T09:55:42.625 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-21T09:55:42.626 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-21T09:55:42.626 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout 0 2024-03-21T09:55:42.627 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout ], 2024-03-21T09:55:42.627 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-21T09:55:42.627 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "smithi153" 2024-03-21T09:55:42.627 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout ], 2024-03-21T09:55:42.627 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-03-21T09:55:42.627 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-21T09:55:42.628 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T09:55:42.628 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-03-21T09:55:42.628 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-21T09:55:42.628 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout }, 2024-03-21T09:55:42.628 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-21T09:55:42.628 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T09:55:42.628 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-21T09:55:42.628 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-21T09:55:42.638 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-21T09:55:42.638 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-21T09:55:42.638 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-21T09:55:42.638 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-21T09:55:42.639 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout }, 2024-03-21T09:55:42.639 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-21T09:55:42.639 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-21T09:55:42.639 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-21T09:55:42.639 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-21T09:55:42.639 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-21T09:55:42.639 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-21T09:55:42.639 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-21T09:55:42.639 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-21T09:55:42.639 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-21T09:55:42.639 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout }, 2024-03-21T09:55:42.639 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-21T09:55:42.639 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T09:55:42.639 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-21T09:55:42.639 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-21T09:55:42.639 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout }, 2024-03-21T09:55:42.640 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-21T09:55:42.640 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-21T09:55:42.640 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-21T09:55:42.640 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-21T09:55:42.640 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-21T09:55:42.640 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-21T09:55:42.640 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "restful" 2024-03-21T09:55:42.640 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout ], 2024-03-21T09:55:42.640 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-21T09:55:42.640 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout }, 2024-03-21T09:55:42.640 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-21T09:55:42.640 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T09:55:42.640 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "modified": "2024-03-21T09:55:19.360779+0000", 2024-03-21T09:55:42.640 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-21T09:55:42.640 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout }, 2024-03-21T09:55:42.641 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-21T09:55:42.641 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout } 2024-03-21T09:55:42.641 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:41.773+0000 7f85a8400700 1 Processor -- start 2024-03-21T09:55:42.641 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:41.773+0000 7f85a8400700 1 -- start start 2024-03-21T09:55:42.641 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:41.774+0000 7f85a8400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f85a007ad80 0x7f85a00791e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:55:42.641 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:41.774+0000 7f85a8400700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85a00797b0 con 0x7f85a007ad80 2024-03-21T09:55:42.641 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:41.774+0000 7f85a5a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f85a007ad80 0x7f85a00791e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:55:42.641 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:41.774+0000 7f85a5a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f85a007ad80 0x7f85a00791e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:49704/0 (socket says 172.21.15.153:49704) 2024-03-21T09:55:42.641 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:41.774+0000 7f85a5a00700 1 -- 172.21.15.153:0/4053181708 learned_addr learned my addr 172.21.15.153:0/4053181708 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T09:55:42.641 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:41.774+0000 7f85a5a00700 1 -- 172.21.15.153:0/4053181708 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85a00798f0 con 0x7f85a007ad80 2024-03-21T09:55:42.641 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:41.775+0000 7f85a5a00700 1 --2- 172.21.15.153:0/4053181708 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f85a007ad80 0x7f85a00791e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f85940098a0 tx=0x7f8594009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9753d961a67c6319 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:55:42.641 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:41.775+0000 7f859fe00700 1 -- 172.21.15.153:0/4053181708 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8594004180 con 0x7f85a007ad80 2024-03-21T09:55:42.642 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:41.775+0000 7f859fe00700 1 -- 172.21.15.153:0/4053181708 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f85940042e0 con 0x7f85a007ad80 2024-03-21T09:55:42.642 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:41.775+0000 7f859fe00700 1 -- 172.21.15.153:0/4053181708 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8594011a50 con 0x7f85a007ad80 2024-03-21T09:55:42.642 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:41.776+0000 7f85a8400700 1 -- 172.21.15.153:0/4053181708 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f85a007ad80 msgr2=0x7f85a00791e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:55:42.642 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:41.776+0000 7f85a8400700 1 --2- 172.21.15.153:0/4053181708 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f85a007ad80 0x7f85a00791e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f85940098a0 tx=0x7f8594009bb0 comp rx=0 tx=0).stop 2024-03-21T09:55:42.642 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:41.776+0000 7f85a8400700 1 -- 172.21.15.153:0/4053181708 shutdown_connections 2024-03-21T09:55:42.642 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:41.776+0000 7f85a8400700 1 --2- 172.21.15.153:0/4053181708 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f85a007ad80 0x7f85a00791e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:55:42.642 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:41.776+0000 7f85a8400700 1 -- 172.21.15.153:0/4053181708 >> 172.21.15.153:0/4053181708 conn(0x7f85a01019b0 msgr2=0x7f85a0103e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:55:42.642 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:41.776+0000 7f85a8400700 1 -- 172.21.15.153:0/4053181708 shutdown_connections 2024-03-21T09:55:42.642 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:41.776+0000 7f85a8400700 1 -- 172.21.15.153:0/4053181708 wait complete. 2024-03-21T09:55:42.642 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:41.777+0000 7f85a8400700 1 Processor -- start 2024-03-21T09:55:42.642 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:41.777+0000 7f85a8400700 1 -- start start 2024-03-21T09:55:42.642 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:41.778+0000 7f85a8400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f85a007ad80 0x7f85a019c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:55:42.642 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:41.778+0000 7f85a8400700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85a019c820 con 0x7f85a007ad80 2024-03-21T09:55:42.643 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:41.778+0000 7f85a5a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f85a007ad80 0x7f85a019c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:55:42.643 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:41.778+0000 7f85a5a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f85a007ad80 0x7f85a019c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:49720/0 (socket says 172.21.15.153:49720) 2024-03-21T09:55:42.643 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:41.778+0000 7f85a5a00700 1 -- 172.21.15.153:0/785836858 learned_addr learned my addr 172.21.15.153:0/785836858 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T09:55:42.643 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:41.778+0000 7f85a5a00700 1 -- 172.21.15.153:0/785836858 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8594009550 con 0x7f85a007ad80 2024-03-21T09:55:42.643 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:41.779+0000 7f85a5a00700 1 --2- 172.21.15.153:0/785836858 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f85a007ad80 0x7f85a019c2e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8594003de0 tx=0x7f8594011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:55:42.643 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:41.779+0000 7f859e000700 1 -- 172.21.15.153:0/785836858 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8594003930 con 0x7f85a007ad80 2024-03-21T09:55:42.643 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:41.779+0000 7f859e000700 1 -- 172.21.15.153:0/785836858 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8594003a90 con 0x7f85a007ad80 2024-03-21T09:55:42.643 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:41.779+0000 7f859e000700 1 -- 172.21.15.153:0/785836858 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8594019730 con 0x7f85a007ad80 2024-03-21T09:55:42.643 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:41.779+0000 7f85a8400700 1 -- 172.21.15.153:0/785836858 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85a019ca20 con 0x7f85a007ad80 2024-03-21T09:55:42.643 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:41.779+0000 7f85a8400700 1 -- 172.21.15.153:0/785836858 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85a019cec0 con 0x7f85a007ad80 2024-03-21T09:55:42.643 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:41.780+0000 7f859e000700 1 -- 172.21.15.153:0/785836858 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f8594019cb0 con 0x7f85a007ad80 2024-03-21T09:55:42.643 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:41.780+0000 7f859e000700 1 -- 172.21.15.153:0/785836858 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f859402c3a0 con 0x7f85a007ad80 2024-03-21T09:55:42.643 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:41.780+0000 7f85a8400700 1 -- 172.21.15.153:0/785836858 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85a00623c0 con 0x7f85a007ad80 2024-03-21T09:55:42.643 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:41.785+0000 7f859e000700 1 -- 172.21.15.153:0/785836858 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7f859401abc0 con 0x7f85a007ad80 2024-03-21T09:55:42.644 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:41.863+0000 7f85a8400700 1 -- 172.21.15.153:0/785836858 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f85a019f8e0 con 0x7f85a007ad80 2024-03-21T09:55:42.644 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:41.863+0000 7f859e000700 1 -- 172.21.15.153:0/785836858 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1246 (secure 0 0 0) 0x7f8594005a20 con 0x7f85a007ad80 2024-03-21T09:55:42.644 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:41.865+0000 7f85a8400700 1 -- 172.21.15.153:0/785836858 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f85a007ad80 msgr2=0x7f85a019c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:55:42.644 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:41.865+0000 7f85a8400700 1 --2- 172.21.15.153:0/785836858 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f85a007ad80 0x7f85a019c2e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8594003de0 tx=0x7f8594011ab0 comp rx=0 tx=0).stop 2024-03-21T09:55:42.644 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:41.865+0000 7f85a8400700 1 -- 172.21.15.153:0/785836858 shutdown_connections 2024-03-21T09:55:42.644 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:41.865+0000 7f85a8400700 1 --2- 172.21.15.153:0/785836858 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f85a007ad80 0x7f85a019c2e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:55:42.644 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:41.865+0000 7f85a8400700 1 -- 172.21.15.153:0/785836858 >> 172.21.15.153:0/785836858 conn(0x7f85a01019b0 msgr2=0x7f85a01030c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:55:42.644 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:41.865+0000 7f85a8400700 1 -- 172.21.15.153:0/785836858 shutdown_connections 2024-03-21T09:55:42.644 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:41.865+0000 7f85a8400700 1 -- 172.21.15.153:0/785836858 wait complete. 2024-03-21T09:55:42.644 INFO:teuthology.orchestra.run.smithi153.stdout:mgr not available, waiting (2/15)... 2024-03-21T09:55:45.758 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:45 smithi153 ceph-mon[29117]: from='client.? 172.21.15.153:0/1682958157' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-21T09:55:46.070 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout 2024-03-21T09:55:46.082 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout { 2024-03-21T09:55:46.082 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "fsid": "070ed3e4-e769-11ee-95cd-87774f69a715", 2024-03-21T09:55:46.082 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "health": { 2024-03-21T09:55:46.082 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-21T09:55:46.082 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-21T09:55:46.082 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-21T09:55:46.082 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout }, 2024-03-21T09:55:46.082 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-21T09:55:46.082 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-21T09:55:46.083 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout 0 2024-03-21T09:55:46.083 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout ], 2024-03-21T09:55:46.083 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-21T09:55:46.083 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "smithi153" 2024-03-21T09:55:46.083 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout ], 2024-03-21T09:55:46.083 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "quorum_age": 11, 2024-03-21T09:55:46.083 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-21T09:55:46.083 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T09:55:46.083 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-03-21T09:55:46.083 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-21T09:55:46.083 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout }, 2024-03-21T09:55:46.083 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-21T09:55:46.083 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T09:55:46.083 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-21T09:55:46.083 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-21T09:55:46.084 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-21T09:55:46.084 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-21T09:55:46.084 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-21T09:55:46.084 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-21T09:55:46.084 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout }, 2024-03-21T09:55:46.084 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-21T09:55:46.084 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-21T09:55:46.084 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-21T09:55:46.084 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-21T09:55:46.084 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-21T09:55:46.084 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-21T09:55:46.084 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-21T09:55:46.084 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-21T09:55:46.084 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-21T09:55:46.085 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout }, 2024-03-21T09:55:46.085 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-21T09:55:46.085 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T09:55:46.085 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-21T09:55:46.085 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-21T09:55:46.085 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout }, 2024-03-21T09:55:46.085 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-21T09:55:46.085 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-21T09:55:46.085 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-21T09:55:46.085 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-21T09:55:46.085 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-21T09:55:46.085 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-21T09:55:46.085 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "restful" 2024-03-21T09:55:46.085 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout ], 2024-03-21T09:55:46.085 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-21T09:55:46.086 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout }, 2024-03-21T09:55:46.086 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-21T09:55:46.086 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T09:55:46.086 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "modified": "2024-03-21T09:55:19.360779+0000", 2024-03-21T09:55:46.086 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-21T09:55:46.086 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout }, 2024-03-21T09:55:46.086 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-21T09:55:46.086 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout } 2024-03-21T09:55:46.086 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:45.270+0000 7f0163400700 1 Processor -- start 2024-03-21T09:55:46.086 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:45.270+0000 7f0163400700 1 -- start start 2024-03-21T09:55:46.086 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:45.271+0000 7f0163400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f015c07ad60 0x7f015c0791c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:55:46.086 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:45.271+0000 7f0163400700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f015c079790 con 0x7f015c07ad60 2024-03-21T09:55:46.086 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:45.271+0000 7f0160a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f015c07ad60 0x7f015c0791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:55:46.087 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:45.271+0000 7f0160a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f015c07ad60 0x7f015c0791c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41198/0 (socket says 172.21.15.153:41198) 2024-03-21T09:55:46.087 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:45.272+0000 7f0160a00700 1 -- 172.21.15.153:0/4187080775 learned_addr learned my addr 172.21.15.153:0/4187080775 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T09:55:46.087 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:45.272+0000 7f0160a00700 1 -- 172.21.15.153:0/4187080775 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f015c0798d0 con 0x7f015c07ad60 2024-03-21T09:55:46.087 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:45.272+0000 7f0160a00700 1 --2- 172.21.15.153:0/4187080775 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f015c07ad60 0x7f015c0791c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f01500098a0 tx=0x7f0150009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a817147c663a830 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:55:46.087 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:45.273+0000 7f015b400700 1 -- 172.21.15.153:0/4187080775 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0150004180 con 0x7f015c07ad60 2024-03-21T09:55:46.087 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:45.273+0000 7f015b400700 1 -- 172.21.15.153:0/4187080775 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f01500042e0 con 0x7f015c07ad60 2024-03-21T09:55:46.087 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:45.273+0000 7f015b400700 1 -- 172.21.15.153:0/4187080775 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0150011a50 con 0x7f015c07ad60 2024-03-21T09:55:46.087 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:45.273+0000 7f0163400700 1 -- 172.21.15.153:0/4187080775 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f015c07ad60 msgr2=0x7f015c0791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:55:46.087 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:45.273+0000 7f0163400700 1 --2- 172.21.15.153:0/4187080775 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f015c07ad60 0x7f015c0791c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f01500098a0 tx=0x7f0150009bb0 comp rx=0 tx=0).stop 2024-03-21T09:55:46.087 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:45.274+0000 7f0163400700 1 -- 172.21.15.153:0/4187080775 shutdown_connections 2024-03-21T09:55:46.087 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:45.274+0000 7f0163400700 1 --2- 172.21.15.153:0/4187080775 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f015c07ad60 0x7f015c0791c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:55:46.087 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:45.274+0000 7f0163400700 1 -- 172.21.15.153:0/4187080775 >> 172.21.15.153:0/4187080775 conn(0x7f015c1019b0 msgr2=0x7f015c103e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:55:46.087 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:45.274+0000 7f0163400700 1 -- 172.21.15.153:0/4187080775 shutdown_connections 2024-03-21T09:55:46.088 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:45.274+0000 7f0163400700 1 -- 172.21.15.153:0/4187080775 wait complete. 2024-03-21T09:55:46.088 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:45.275+0000 7f0163400700 1 Processor -- start 2024-03-21T09:55:46.088 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:45.275+0000 7f0163400700 1 -- start start 2024-03-21T09:55:46.088 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:45.275+0000 7f0163400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f015c07ad60 0x7f015c19c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:55:46.088 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:45.275+0000 7f0163400700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f015c19c830 con 0x7f015c07ad60 2024-03-21T09:55:46.088 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:45.275+0000 7f0160a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f015c07ad60 0x7f015c19c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:55:46.088 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:45.275+0000 7f0160a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f015c07ad60 0x7f015c19c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41206/0 (socket says 172.21.15.153:41206) 2024-03-21T09:55:46.088 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:45.275+0000 7f0160a00700 1 -- 172.21.15.153:0/1682958157 learned_addr learned my addr 172.21.15.153:0/1682958157 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T09:55:46.088 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:45.276+0000 7f0160a00700 1 -- 172.21.15.153:0/1682958157 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0150009550 con 0x7f015c07ad60 2024-03-21T09:55:46.088 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:45.276+0000 7f0160a00700 1 --2- 172.21.15.153:0/1682958157 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f015c07ad60 0x7f015c19c2f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0150003de0 tx=0x7f0150011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:55:46.088 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:45.276+0000 7f0159600700 1 -- 172.21.15.153:0/1682958157 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0150003930 con 0x7f015c07ad60 2024-03-21T09:55:46.088 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:45.276+0000 7f0159600700 1 -- 172.21.15.153:0/1682958157 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0150003a90 con 0x7f015c07ad60 2024-03-21T09:55:46.088 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:45.276+0000 7f0163400700 1 -- 172.21.15.153:0/1682958157 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f015c19ca30 con 0x7f015c07ad60 2024-03-21T09:55:46.089 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:45.276+0000 7f0159600700 1 -- 172.21.15.153:0/1682958157 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0150019730 con 0x7f015c07ad60 2024-03-21T09:55:46.089 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:45.276+0000 7f0163400700 1 -- 172.21.15.153:0/1682958157 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f015c19ced0 con 0x7f015c07ad60 2024-03-21T09:55:46.089 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:45.277+0000 7f0159600700 1 -- 172.21.15.153:0/1682958157 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f0150019c80 con 0x7f015c07ad60 2024-03-21T09:55:46.089 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:45.277+0000 7f0159600700 1 -- 172.21.15.153:0/1682958157 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f015002c3a0 con 0x7f015c07ad60 2024-03-21T09:55:46.089 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:45.277+0000 7f0163400700 1 -- 172.21.15.153:0/1682958157 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f015c0623c0 con 0x7f015c07ad60 2024-03-21T09:55:46.089 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:45.282+0000 7f0159600700 1 -- 172.21.15.153:0/1682958157 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7f015001abc0 con 0x7f015c07ad60 2024-03-21T09:55:46.089 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:45.360+0000 7f0163400700 1 -- 172.21.15.153:0/1682958157 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f015c19f890 con 0x7f015c07ad60 2024-03-21T09:55:46.089 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:45.360+0000 7f0159600700 1 -- 172.21.15.153:0/1682958157 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1247 (secure 0 0 0) 0x7f015001a430 con 0x7f015c07ad60 2024-03-21T09:55:46.089 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:45.361+0000 7f0163400700 1 -- 172.21.15.153:0/1682958157 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f015c07ad60 msgr2=0x7f015c19c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:55:46.089 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:45.361+0000 7f0163400700 1 --2- 172.21.15.153:0/1682958157 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f015c07ad60 0x7f015c19c2f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0150003de0 tx=0x7f0150011ab0 comp rx=0 tx=0).stop 2024-03-21T09:55:46.089 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:45.361+0000 7f0163400700 1 -- 172.21.15.153:0/1682958157 shutdown_connections 2024-03-21T09:55:46.089 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:45.361+0000 7f0163400700 1 --2- 172.21.15.153:0/1682958157 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f015c07ad60 0x7f015c19c2f0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:55:46.089 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:45.361+0000 7f0163400700 1 -- 172.21.15.153:0/1682958157 >> 172.21.15.153:0/1682958157 conn(0x7f015c1019b0 msgr2=0x7f015c1030c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:55:46.089 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:45.362+0000 7f0163400700 1 -- 172.21.15.153:0/1682958157 shutdown_connections 2024-03-21T09:55:46.090 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:45.362+0000 7f0163400700 1 -- 172.21.15.153:0/1682958157 wait complete. 2024-03-21T09:55:46.090 INFO:teuthology.orchestra.run.smithi153.stdout:mgr not available, waiting (3/15)... 2024-03-21T09:55:46.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:46 smithi153 ceph-mon[29117]: Activating manager daemon smithi153.tluxni 2024-03-21T09:55:46.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:46 smithi153 ceph-mon[29117]: mgrmap e2: smithi153.tluxni(active, starting, since 0.0097677s) 2024-03-21T09:55:46.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:46 smithi153 ceph-mon[29117]: from='mgr.14100 172.21.15.153:0/658568256' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-21T09:55:46.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:46 smithi153 ceph-mon[29117]: from='mgr.14100 172.21.15.153:0/658568256' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-21T09:55:46.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:46 smithi153 ceph-mon[29117]: from='mgr.14100 172.21.15.153:0/658568256' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-21T09:55:46.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:46 smithi153 ceph-mon[29117]: from='mgr.14100 172.21.15.153:0/658568256' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mon metadata", "id": "smithi153"}]: dispatch 2024-03-21T09:55:46.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:46 smithi153 ceph-mon[29117]: from='mgr.14100 172.21.15.153:0/658568256' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mgr metadata", "who": "smithi153.tluxni", "id": "smithi153.tluxni"}]: dispatch 2024-03-21T09:55:46.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:46 smithi153 ceph-mon[29117]: Manager daemon smithi153.tluxni is now available 2024-03-21T09:55:46.808 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:46 smithi153 ceph-mon[29117]: from='mgr.14100 172.21.15.153:0/658568256' entity='mgr.smithi153.tluxni' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi153.tluxni/mirror_snapshot_schedule"}]: dispatch 2024-03-21T09:55:47.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:47 smithi153 ceph-mon[29117]: from='mgr.14100 172.21.15.153:0/658568256' entity='mgr.smithi153.tluxni' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi153.tluxni/trash_purge_schedule"}]: dispatch 2024-03-21T09:55:47.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:47 smithi153 ceph-mon[29117]: from='mgr.14100 172.21.15.153:0/658568256' entity='mgr.smithi153.tluxni' 2024-03-21T09:55:47.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:47 smithi153 ceph-mon[29117]: from='mgr.14100 172.21.15.153:0/658568256' entity='mgr.smithi153.tluxni' 2024-03-21T09:55:47.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:47 smithi153 ceph-mon[29117]: from='mgr.14100 172.21.15.153:0/658568256' entity='mgr.smithi153.tluxni' 2024-03-21T09:55:47.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:47 smithi153 ceph-mon[29117]: mgrmap e3: smithi153.tluxni(active, since 1.02169s) 2024-03-21T09:55:49.671 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:49 smithi153 ceph-mon[29117]: mgrmap e4: smithi153.tluxni(active, since 2s) 2024-03-21T09:55:49.672 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:49 smithi153 ceph-mon[29117]: from='client.? 172.21.15.153:0/2830242873' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-21T09:55:49.686 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout 2024-03-21T09:55:49.687 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout { 2024-03-21T09:55:49.687 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "fsid": "070ed3e4-e769-11ee-95cd-87774f69a715", 2024-03-21T09:55:49.687 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "health": { 2024-03-21T09:55:49.687 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-21T09:55:49.687 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-21T09:55:49.687 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-21T09:55:49.687 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout }, 2024-03-21T09:55:49.687 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-21T09:55:49.688 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-21T09:55:49.688 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout 0 2024-03-21T09:55:49.688 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout ], 2024-03-21T09:55:49.688 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-21T09:55:49.688 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "smithi153" 2024-03-21T09:55:49.688 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout ], 2024-03-21T09:55:49.688 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "quorum_age": 15, 2024-03-21T09:55:49.689 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-21T09:55:49.689 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T09:55:49.689 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-03-21T09:55:49.689 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-21T09:55:49.689 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout }, 2024-03-21T09:55:49.689 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-21T09:55:49.689 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T09:55:49.689 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-21T09:55:49.689 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-21T09:55:49.690 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-21T09:55:49.706 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-21T09:55:49.706 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-21T09:55:49.706 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-21T09:55:49.706 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout }, 2024-03-21T09:55:49.707 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-21T09:55:49.707 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-21T09:55:49.707 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-21T09:55:49.707 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-21T09:55:49.707 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-21T09:55:49.707 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-21T09:55:49.707 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-21T09:55:49.707 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-21T09:55:49.707 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-21T09:55:49.707 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout }, 2024-03-21T09:55:49.707 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-21T09:55:49.708 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T09:55:49.708 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-21T09:55:49.708 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-21T09:55:49.708 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout }, 2024-03-21T09:55:49.708 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-21T09:55:49.708 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-21T09:55:49.708 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-21T09:55:49.708 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-21T09:55:49.709 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-21T09:55:49.709 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-21T09:55:49.709 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "restful" 2024-03-21T09:55:49.709 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout ], 2024-03-21T09:55:49.709 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-21T09:55:49.709 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout }, 2024-03-21T09:55:49.710 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-21T09:55:49.710 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T09:55:49.710 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "modified": "2024-03-21T09:55:19.360779+0000", 2024-03-21T09:55:49.710 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-21T09:55:49.710 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout }, 2024-03-21T09:55:49.710 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-21T09:55:49.710 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout } 2024-03-21T09:55:49.710 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:48.677+0000 7f2524400700 1 Processor -- start 2024-03-21T09:55:49.711 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:48.678+0000 7f2524400700 1 -- start start 2024-03-21T09:55:49.711 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:48.678+0000 7f2524400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f251c07ad80 0x7f251c0791e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:55:49.711 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:48.678+0000 7f2524400700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f251c0797b0 con 0x7f251c07ad80 2024-03-21T09:55:49.711 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:48.678+0000 7f2521a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f251c07ad80 0x7f251c0791e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:55:49.711 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:48.678+0000 7f2521a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f251c07ad80 0x7f251c0791e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41274/0 (socket says 172.21.15.153:41274) 2024-03-21T09:55:49.711 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:48.678+0000 7f2521a00700 1 -- 172.21.15.153:0/4066162297 learned_addr learned my addr 172.21.15.153:0/4066162297 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T09:55:49.711 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:48.679+0000 7f2521a00700 1 -- 172.21.15.153:0/4066162297 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f251c0798f0 con 0x7f251c07ad80 2024-03-21T09:55:49.711 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:48.679+0000 7f2521a00700 1 --2- 172.21.15.153:0/4066162297 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f251c07ad80 0x7f251c0791e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2510009e80 tx=0x7f251000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef15edf2d51de89f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:55:49.711 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:48.679+0000 7f251be00700 1 -- 172.21.15.153:0/4066162297 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f251000e070 con 0x7f251c07ad80 2024-03-21T09:55:49.712 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:48.679+0000 7f251be00700 1 -- 172.21.15.153:0/4066162297 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2510003e90 con 0x7f251c07ad80 2024-03-21T09:55:49.712 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:48.680+0000 7f251be00700 1 -- 172.21.15.153:0/4066162297 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2510004380 con 0x7f251c07ad80 2024-03-21T09:55:49.712 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:48.680+0000 7f2524400700 1 -- 172.21.15.153:0/4066162297 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f251c07ad80 msgr2=0x7f251c0791e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:55:49.712 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:48.680+0000 7f2524400700 1 --2- 172.21.15.153:0/4066162297 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f251c07ad80 0x7f251c0791e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2510009e80 tx=0x7f251000b330 comp rx=0 tx=0).stop 2024-03-21T09:55:49.712 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:48.680+0000 7f2524400700 1 -- 172.21.15.153:0/4066162297 shutdown_connections 2024-03-21T09:55:49.712 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:48.680+0000 7f2524400700 1 --2- 172.21.15.153:0/4066162297 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f251c07ad80 0x7f251c0791e0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:55:49.712 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:48.680+0000 7f2524400700 1 -- 172.21.15.153:0/4066162297 >> 172.21.15.153:0/4066162297 conn(0x7f251c1019b0 msgr2=0x7f251c103e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:55:49.712 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:48.680+0000 7f2524400700 1 -- 172.21.15.153:0/4066162297 shutdown_connections 2024-03-21T09:55:49.712 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:48.680+0000 7f2524400700 1 -- 172.21.15.153:0/4066162297 wait complete. 2024-03-21T09:55:49.712 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:48.681+0000 7f2524400700 1 Processor -- start 2024-03-21T09:55:49.712 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:48.681+0000 7f2524400700 1 -- start start 2024-03-21T09:55:49.712 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:48.682+0000 7f2524400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f251c07ad80 0x7f251c19c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:55:49.712 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:48.682+0000 7f2524400700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f251c19c7e0 con 0x7f251c07ad80 2024-03-21T09:55:49.712 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:48.682+0000 7f2521a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f251c07ad80 0x7f251c19c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:55:49.713 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:48.682+0000 7f2521a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f251c07ad80 0x7f251c19c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41280/0 (socket says 172.21.15.153:41280) 2024-03-21T09:55:49.713 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:48.682+0000 7f2521a00700 1 -- 172.21.15.153:0/2830242873 learned_addr learned my addr 172.21.15.153:0/2830242873 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T09:55:49.713 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:48.682+0000 7f2521a00700 1 -- 172.21.15.153:0/2830242873 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2510009b30 con 0x7f251c07ad80 2024-03-21T09:55:49.713 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:48.683+0000 7f2521a00700 1 --2- 172.21.15.153:0/2830242873 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f251c07ad80 0x7f251c19c2a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f251000b7b0 tx=0x7f2510004920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:55:49.713 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:48.683+0000 7f251a000700 1 -- 172.21.15.153:0/2830242873 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f251000e040 con 0x7f251c07ad80 2024-03-21T09:55:49.713 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:48.683+0000 7f251a000700 1 -- 172.21.15.153:0/2830242873 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2510003c30 con 0x7f251c07ad80 2024-03-21T09:55:49.713 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:48.683+0000 7f2524400700 1 -- 172.21.15.153:0/2830242873 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f251c19c9e0 con 0x7f251c07ad80 2024-03-21T09:55:49.713 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:48.683+0000 7f251a000700 1 -- 172.21.15.153:0/2830242873 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f251001b740 con 0x7f251c07ad80 2024-03-21T09:55:49.713 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:48.683+0000 7f2524400700 1 -- 172.21.15.153:0/2830242873 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f251c19cee0 con 0x7f251c07ad80 2024-03-21T09:55:49.713 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:48.684+0000 7f251a000700 1 -- 172.21.15.153:0/2830242873 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 4) v1 ==== 45272+0+0 (secure 0 0 0) 0x7f2510019040 con 0x7f251c07ad80 2024-03-21T09:55:49.713 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:48.684+0000 7f2524400700 1 -- 172.21.15.153:0/2830242873 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f251c062430 con 0x7f251c07ad80 2024-03-21T09:55:49.713 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:48.685+0000 7f251a000700 1 --2- 172.21.15.153:0/2830242873 >> [v2:172.21.15.153:6800/270837948,v1:172.21.15.153:6801/270837948] conn(0x7f2504038320 0x7f250403a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:55:49.713 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:48.685+0000 7f251a000700 1 -- 172.21.15.153:0/2830242873 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f251004bcb0 con 0x7f251c07ad80 2024-03-21T09:55:49.714 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:48.685+0000 7f2521000700 1 --2- 172.21.15.153:0/2830242873 >> [v2:172.21.15.153:6800/270837948,v1:172.21.15.153:6801/270837948] conn(0x7f2504038320 0x7f250403a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:55:49.714 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:48.686+0000 7f2521000700 1 --2- 172.21.15.153:0/2830242873 >> [v2:172.21.15.153:6800/270837948,v1:172.21.15.153:6801/270837948] conn(0x7f2504038320 0x7f250403a7e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f2508006fd0 tx=0x7f2508006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:55:49.714 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:48.691+0000 7f251a000700 1 -- 172.21.15.153:0/2830242873 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f251c062430 con 0x7f251c07ad80 2024-03-21T09:55:49.714 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:48.985+0000 7f2524400700 1 -- 172.21.15.153:0/2830242873 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f251c062430 con 0x7f251c07ad80 2024-03-21T09:55:49.714 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:48.985+0000 7f251a000700 1 -- 172.21.15.153:0/2830242873 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1246 (secure 0 0 0) 0x7f251c062430 con 0x7f251c07ad80 2024-03-21T09:55:49.714 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:48.990+0000 7f2524400700 1 -- 172.21.15.153:0/2830242873 >> [v2:172.21.15.153:6800/270837948,v1:172.21.15.153:6801/270837948] conn(0x7f2504038320 msgr2=0x7f250403a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:55:49.714 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:48.990+0000 7f2524400700 1 --2- 172.21.15.153:0/2830242873 >> [v2:172.21.15.153:6800/270837948,v1:172.21.15.153:6801/270837948] conn(0x7f2504038320 0x7f250403a7e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f2508006fd0 tx=0x7f2508006e40 comp rx=0 tx=0).stop 2024-03-21T09:55:49.714 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:48.990+0000 7f2524400700 1 -- 172.21.15.153:0/2830242873 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f251c07ad80 msgr2=0x7f251c19c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:55:49.714 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:48.990+0000 7f2524400700 1 --2- 172.21.15.153:0/2830242873 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f251c07ad80 0x7f251c19c2a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f251000b7b0 tx=0x7f2510004920 comp rx=0 tx=0).stop 2024-03-21T09:55:49.714 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:48.990+0000 7f2524400700 1 -- 172.21.15.153:0/2830242873 shutdown_connections 2024-03-21T09:55:49.714 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:48.990+0000 7f2524400700 1 --2- 172.21.15.153:0/2830242873 >> [v2:172.21.15.153:6800/270837948,v1:172.21.15.153:6801/270837948] conn(0x7f2504038320 0x7f250403a7e0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:55:49.714 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:48.990+0000 7f2524400700 1 --2- 172.21.15.153:0/2830242873 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f251c07ad80 0x7f251c19c2a0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:55:49.714 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:48.990+0000 7f2524400700 1 -- 172.21.15.153:0/2830242873 >> 172.21.15.153:0/2830242873 conn(0x7f251c1019b0 msgr2=0x7f251c103080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:55:49.715 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:48.990+0000 7f2524400700 1 -- 172.21.15.153:0/2830242873 shutdown_connections 2024-03-21T09:55:49.715 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:48.990+0000 7f2524400700 1 -- 172.21.15.153:0/2830242873 wait complete. 2024-03-21T09:55:49.715 INFO:teuthology.orchestra.run.smithi153.stdout:mgr is available 2024-03-21T09:55:50.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:50 smithi153 ceph-mon[29117]: from='client.? 172.21.15.153:0/2406199658' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-21T09:55:51.226 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout 2024-03-21T09:55:51.226 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout [global] 2024-03-21T09:55:51.226 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout fsid = 070ed3e4-e769-11ee-95cd-87774f69a715 2024-03-21T09:55:51.226 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-21T09:55:51.226 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-21T09:55:51.226 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-21T09:55:51.226 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-21T09:55:51.226 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout 2024-03-21T09:55:51.226 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-21T09:55:51.227 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-21T09:55:51.227 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout 2024-03-21T09:55:51.227 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout [osd] 2024-03-21T09:55:51.227 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-21T09:55:51.227 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-21T09:55:51.227 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-21T09:55:51.227 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-21T09:55:51.227 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-21T09:55:51.227 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:50.256+0000 7ff36ba00700 1 Processor -- start 2024-03-21T09:55:51.227 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:50.256+0000 7ff36ba00700 1 -- start start 2024-03-21T09:55:51.227 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:50.257+0000 7ff36ba00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff364108620 0x7ff364108a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:55:51.227 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:50.257+0000 7ff36ba00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff364109010 con 0x7ff364108620 2024-03-21T09:55:51.227 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:50.257+0000 7ff369000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff364108620 0x7ff364108a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:55:51.228 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:50.257+0000 7ff369000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff364108620 0x7ff364108a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41282/0 (socket says 172.21.15.153:41282) 2024-03-21T09:55:51.228 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:50.257+0000 7ff369000700 1 -- 172.21.15.153:0/704051488 learned_addr learned my addr 172.21.15.153:0/704051488 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T09:55:51.228 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:50.258+0000 7ff369000700 1 -- 172.21.15.153:0/704051488 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff364109820 con 0x7ff364108620 2024-03-21T09:55:51.228 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:50.258+0000 7ff369000700 1 --2- 172.21.15.153:0/704051488 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff364108620 0x7ff364108a40 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff3580098a0 tx=0x7ff358009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e6333aa2c1ea69d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:55:51.228 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:50.258+0000 7ff363400700 1 -- 172.21.15.153:0/704051488 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff358004180 con 0x7ff364108620 2024-03-21T09:55:51.228 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:50.259+0000 7ff363400700 1 -- 172.21.15.153:0/704051488 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff3580042e0 con 0x7ff364108620 2024-03-21T09:55:51.228 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:50.259+0000 7ff363400700 1 -- 172.21.15.153:0/704051488 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff358011a50 con 0x7ff364108620 2024-03-21T09:55:51.228 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:50.259+0000 7ff36ba00700 1 -- 172.21.15.153:0/704051488 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff364108620 msgr2=0x7ff364108a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:55:51.228 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:50.259+0000 7ff36ba00700 1 --2- 172.21.15.153:0/704051488 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff364108620 0x7ff364108a40 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff3580098a0 tx=0x7ff358009bb0 comp rx=0 tx=0).stop 2024-03-21T09:55:51.228 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:50.259+0000 7ff36ba00700 1 -- 172.21.15.153:0/704051488 shutdown_connections 2024-03-21T09:55:51.228 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:50.260+0000 7ff36ba00700 1 --2- 172.21.15.153:0/704051488 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff364108620 0x7ff364108a40 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:55:51.228 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:50.260+0000 7ff36ba00700 1 -- 172.21.15.153:0/704051488 >> 172.21.15.153:0/704051488 conn(0x7ff36407bb20 msgr2=0x7ff3641064e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:55:51.228 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:50.260+0000 7ff36ba00700 1 -- 172.21.15.153:0/704051488 shutdown_connections 2024-03-21T09:55:51.229 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:50.260+0000 7ff36ba00700 1 -- 172.21.15.153:0/704051488 wait complete. 2024-03-21T09:55:51.229 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:50.261+0000 7ff36ba00700 1 Processor -- start 2024-03-21T09:55:51.229 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:50.261+0000 7ff36ba00700 1 -- start start 2024-03-21T09:55:51.229 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:50.261+0000 7ff36ba00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff364108620 0x7ff36419c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:55:51.229 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:50.261+0000 7ff36ba00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff36419c8a0 con 0x7ff364108620 2024-03-21T09:55:51.229 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:50.262+0000 7ff369000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff364108620 0x7ff36419c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:55:51.229 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:50.262+0000 7ff369000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff364108620 0x7ff36419c360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41286/0 (socket says 172.21.15.153:41286) 2024-03-21T09:55:51.229 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:50.262+0000 7ff369000700 1 -- 172.21.15.153:0/2406199658 learned_addr learned my addr 172.21.15.153:0/2406199658 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T09:55:51.229 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:50.262+0000 7ff369000700 1 -- 172.21.15.153:0/2406199658 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff358009550 con 0x7ff364108620 2024-03-21T09:55:51.229 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:50.262+0000 7ff369000700 1 --2- 172.21.15.153:0/2406199658 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff364108620 0x7ff36419c360 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff358003de0 tx=0x7ff358011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:55:51.229 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:50.262+0000 7ff361600700 1 -- 172.21.15.153:0/2406199658 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff358003930 con 0x7ff364108620 2024-03-21T09:55:51.229 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:50.263+0000 7ff361600700 1 -- 172.21.15.153:0/2406199658 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff358003a90 con 0x7ff364108620 2024-03-21T09:55:51.229 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:50.263+0000 7ff36ba00700 1 -- 172.21.15.153:0/2406199658 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff36419caa0 con 0x7ff364108620 2024-03-21T09:55:51.230 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:50.263+0000 7ff361600700 1 -- 172.21.15.153:0/2406199658 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff3580196c0 con 0x7ff364108620 2024-03-21T09:55:51.230 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:50.263+0000 7ff36ba00700 1 -- 172.21.15.153:0/2406199658 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff36419cf40 con 0x7ff364108620 2024-03-21T09:55:51.230 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:50.264+0000 7ff361600700 1 -- 172.21.15.153:0/2406199658 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 4) v1 ==== 45272+0+0 (secure 0 0 0) 0x7ff35801a430 con 0x7ff364108620 2024-03-21T09:55:51.230 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:50.264+0000 7ff36ba00700 1 -- 172.21.15.153:0/2406199658 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3640623c0 con 0x7ff364108620 2024-03-21T09:55:51.230 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:50.264+0000 7ff361600700 1 --2- 172.21.15.153:0/2406199658 >> [v2:172.21.15.153:6800/270837948,v1:172.21.15.153:6801/270837948] conn(0x7ff34c0382d0 0x7ff34c03a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:55:51.230 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:50.264+0000 7ff361600700 1 -- 172.21.15.153:0/2406199658 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7ff35804c070 con 0x7ff364108620 2024-03-21T09:55:51.230 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:50.265+0000 7ff363e00700 1 --2- 172.21.15.153:0/2406199658 >> [v2:172.21.15.153:6800/270837948,v1:172.21.15.153:6801/270837948] conn(0x7ff34c0382d0 0x7ff34c03a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:55:51.230 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:50.266+0000 7ff363e00700 1 --2- 172.21.15.153:0/2406199658 >> [v2:172.21.15.153:6800/270837948,v1:172.21.15.153:6801/270837948] conn(0x7ff34c0382d0 0x7ff34c03a790 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff350006fd0 tx=0x7ff350006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:55:51.230 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:50.269+0000 7ff361600700 1 -- 172.21.15.153:0/2406199658 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7ff35807f0e0 con 0x7ff364108620 2024-03-21T09:55:51.230 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:50.481+0000 7ff36ba00700 1 -- 172.21.15.153:0/2406199658 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7ff36410ca50 con 0x7ff364108620 2024-03-21T09:55:51.230 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:50.484+0000 7ff361600700 1 -- 172.21.15.153:0/2406199658 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7ff35801a6e0 con 0x7ff364108620 2024-03-21T09:55:51.230 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:50.486+0000 7ff36ba00700 1 -- 172.21.15.153:0/2406199658 >> [v2:172.21.15.153:6800/270837948,v1:172.21.15.153:6801/270837948] conn(0x7ff34c0382d0 msgr2=0x7ff34c03a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:55:51.231 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:50.486+0000 7ff36ba00700 1 --2- 172.21.15.153:0/2406199658 >> [v2:172.21.15.153:6800/270837948,v1:172.21.15.153:6801/270837948] conn(0x7ff34c0382d0 0x7ff34c03a790 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff350006fd0 tx=0x7ff350006e40 comp rx=0 tx=0).stop 2024-03-21T09:55:51.231 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:50.486+0000 7ff36ba00700 1 -- 172.21.15.153:0/2406199658 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff364108620 msgr2=0x7ff36419c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:55:51.231 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:50.486+0000 7ff36ba00700 1 --2- 172.21.15.153:0/2406199658 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff364108620 0x7ff36419c360 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff358003de0 tx=0x7ff358011ab0 comp rx=0 tx=0).stop 2024-03-21T09:55:51.231 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:50.487+0000 7ff36ba00700 1 -- 172.21.15.153:0/2406199658 shutdown_connections 2024-03-21T09:55:51.231 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:50.487+0000 7ff36ba00700 1 --2- 172.21.15.153:0/2406199658 >> [v2:172.21.15.153:6800/270837948,v1:172.21.15.153:6801/270837948] conn(0x7ff34c0382d0 0x7ff34c03a790 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:55:51.231 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:50.487+0000 7ff36ba00700 1 --2- 172.21.15.153:0/2406199658 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff364108620 0x7ff36419c360 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:55:51.231 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:50.487+0000 7ff36ba00700 1 -- 172.21.15.153:0/2406199658 >> 172.21.15.153:0/2406199658 conn(0x7ff36407bb20 msgr2=0x7ff364106780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:55:51.231 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:50.487+0000 7ff36ba00700 1 -- 172.21.15.153:0/2406199658 shutdown_connections 2024-03-21T09:55:51.231 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:50.487+0000 7ff36ba00700 1 -- 172.21.15.153:0/2406199658 wait complete. 2024-03-21T09:55:51.231 INFO:teuthology.orchestra.run.smithi153.stdout:Enabling cephadm module... 2024-03-21T09:55:52.539 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:52 smithi153 ceph-mon[29117]: from='client.? 172.21.15.153:0/3482742839' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-03-21T09:55:52.906 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:51.840+0000 7f6c81400700 1 Processor -- start 2024-03-21T09:55:52.906 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:51.840+0000 7f6c81400700 1 -- start start 2024-03-21T09:55:52.906 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:51.841+0000 7f6c81400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6c7c108630 0x7f6c7c108a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:55:52.907 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:51.841+0000 7f6c81400700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c7c109020 con 0x7f6c7c108630 2024-03-21T09:55:52.907 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:51.841+0000 7f6c7a800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6c7c108630 0x7f6c7c108a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:55:52.907 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:51.841+0000 7f6c7a800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6c7c108630 0x7f6c7c108a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41296/0 (socket says 172.21.15.153:41296) 2024-03-21T09:55:52.907 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:51.841+0000 7f6c7a800700 1 -- 172.21.15.153:0/183669228 learned_addr learned my addr 172.21.15.153:0/183669228 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T09:55:52.907 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:51.842+0000 7f6c7a800700 1 -- 172.21.15.153:0/183669228 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c7c109830 con 0x7f6c7c108630 2024-03-21T09:55:52.907 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:51.842+0000 7f6c7a800700 1 --2- 172.21.15.153:0/183669228 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6c7c108630 0x7f6c7c108a50 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c0098a0 tx=0x7f6c6c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99016010ee90e067 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:55:52.907 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:51.842+0000 7f6c79e00700 1 -- 172.21.15.153:0/183669228 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6c6c004180 con 0x7f6c7c108630 2024-03-21T09:55:52.907 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:51.843+0000 7f6c79e00700 1 -- 172.21.15.153:0/183669228 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6c6c0042e0 con 0x7f6c7c108630 2024-03-21T09:55:52.907 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:51.843+0000 7f6c79e00700 1 -- 172.21.15.153:0/183669228 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6c6c011a50 con 0x7f6c7c108630 2024-03-21T09:55:52.907 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:51.843+0000 7f6c81400700 1 -- 172.21.15.153:0/183669228 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6c7c108630 msgr2=0x7f6c7c108a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:55:52.908 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:51.843+0000 7f6c81400700 1 --2- 172.21.15.153:0/183669228 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6c7c108630 0x7f6c7c108a50 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c0098a0 tx=0x7f6c6c009bb0 comp rx=0 tx=0).stop 2024-03-21T09:55:52.908 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:51.843+0000 7f6c81400700 1 -- 172.21.15.153:0/183669228 shutdown_connections 2024-03-21T09:55:52.908 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:51.843+0000 7f6c81400700 1 --2- 172.21.15.153:0/183669228 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6c7c108630 0x7f6c7c108a50 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:55:52.908 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:51.843+0000 7f6c81400700 1 -- 172.21.15.153:0/183669228 >> 172.21.15.153:0/183669228 conn(0x7f6c7c103c10 msgr2=0x7f6c7c106050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:55:52.908 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:51.843+0000 7f6c81400700 1 -- 172.21.15.153:0/183669228 shutdown_connections 2024-03-21T09:55:52.908 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:51.843+0000 7f6c81400700 1 -- 172.21.15.153:0/183669228 wait complete. 2024-03-21T09:55:52.908 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:51.844+0000 7f6c81400700 1 Processor -- start 2024-03-21T09:55:52.908 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:51.844+0000 7f6c81400700 1 -- start start 2024-03-21T09:55:52.908 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:51.845+0000 7f6c81400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6c7c108630 0x7f6c7c19c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:55:52.908 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:51.845+0000 7f6c81400700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c7c19c8b0 con 0x7f6c7c108630 2024-03-21T09:55:52.908 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:51.845+0000 7f6c7a800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6c7c108630 0x7f6c7c19c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:55:52.908 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:51.845+0000 7f6c7a800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6c7c108630 0x7f6c7c19c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41308/0 (socket says 172.21.15.153:41308) 2024-03-21T09:55:52.908 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:51.845+0000 7f6c7a800700 1 -- 172.21.15.153:0/3482742839 learned_addr learned my addr 172.21.15.153:0/3482742839 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T09:55:52.908 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:51.845+0000 7f6c7a800700 1 -- 172.21.15.153:0/3482742839 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c6c009550 con 0x7f6c7c108630 2024-03-21T09:55:52.909 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:51.846+0000 7f6c7a800700 1 --2- 172.21.15.153:0/3482742839 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6c7c108630 0x7f6c7c19c370 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c003de0 tx=0x7f6c6c011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:55:52.909 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:51.846+0000 7f6c73400700 1 -- 172.21.15.153:0/3482742839 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6c6c003930 con 0x7f6c7c108630 2024-03-21T09:55:52.909 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:51.846+0000 7f6c73400700 1 -- 172.21.15.153:0/3482742839 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6c6c003a90 con 0x7f6c7c108630 2024-03-21T09:55:52.909 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:51.846+0000 7f6c73400700 1 -- 172.21.15.153:0/3482742839 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6c6c0196c0 con 0x7f6c7c108630 2024-03-21T09:55:52.909 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:51.846+0000 7f6c81400700 1 -- 172.21.15.153:0/3482742839 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c7c19cab0 con 0x7f6c7c108630 2024-03-21T09:55:52.909 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:51.846+0000 7f6c81400700 1 -- 172.21.15.153:0/3482742839 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c7c19cf50 con 0x7f6c7c108630 2024-03-21T09:55:52.909 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:51.847+0000 7f6c81400700 1 -- 172.21.15.153:0/3482742839 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c7c0623c0 con 0x7f6c7c108630 2024-03-21T09:55:52.909 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:51.848+0000 7f6c73400700 1 -- 172.21.15.153:0/3482742839 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 4) v1 ==== 45272+0+0 (secure 0 0 0) 0x7f6c6c01abc0 con 0x7f6c7c108630 2024-03-21T09:55:52.909 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:51.848+0000 7f6c73400700 1 --2- 172.21.15.153:0/3482742839 >> [v2:172.21.15.153:6800/270837948,v1:172.21.15.153:6801/270837948] conn(0x7f6c60038320 0x7f6c6003a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:55:52.909 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:51.848+0000 7f6c73400700 1 -- 172.21.15.153:0/3482742839 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f6c6c04c010 con 0x7f6c7c108630 2024-03-21T09:55:52.909 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:51.848+0000 7f6c73e00700 1 --2- 172.21.15.153:0/3482742839 >> [v2:172.21.15.153:6800/270837948,v1:172.21.15.153:6801/270837948] conn(0x7f6c60038320 0x7f6c6003a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:55:52.909 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:51.849+0000 7f6c73e00700 1 --2- 172.21.15.153:0/3482742839 >> [v2:172.21.15.153:6800/270837948,v1:172.21.15.153:6801/270837948] conn(0x7f6c60038320 0x7f6c6003a7e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6c64006fd0 tx=0x7f6c64006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:55:52.910 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:51.853+0000 7f6c73400700 1 -- 172.21.15.153:0/3482742839 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f6c6c010790 con 0x7f6c7c108630 2024-03-21T09:55:52.910 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:52.124+0000 7f6c81400700 1 -- 172.21.15.153:0/3482742839 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f6c7c19fd90 con 0x7f6c7c108630 2024-03-21T09:55:52.910 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:52.181+0000 7f6c73400700 1 -- 172.21.15.153:0/3482742839 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mgrmap(e 5) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f6c6c01a7b0 con 0x7f6c7c108630 2024-03-21T09:55:52.910 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:52.183+0000 7f6c73400700 1 -- 172.21.15.153:0/3482742839 <== mon.0 v2:172.21.15.153:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7f6c6c04bd70 con 0x7f6c7c108630 2024-03-21T09:55:52.910 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:52.187+0000 7f6c81400700 1 -- 172.21.15.153:0/3482742839 >> [v2:172.21.15.153:6800/270837948,v1:172.21.15.153:6801/270837948] conn(0x7f6c60038320 msgr2=0x7f6c6003a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:55:52.910 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:52.187+0000 7f6c81400700 1 --2- 172.21.15.153:0/3482742839 >> [v2:172.21.15.153:6800/270837948,v1:172.21.15.153:6801/270837948] conn(0x7f6c60038320 0x7f6c6003a7e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6c64006fd0 tx=0x7f6c64006e40 comp rx=0 tx=0).stop 2024-03-21T09:55:52.910 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:52.187+0000 7f6c81400700 1 -- 172.21.15.153:0/3482742839 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6c7c108630 msgr2=0x7f6c7c19c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:55:52.910 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:52.187+0000 7f6c81400700 1 --2- 172.21.15.153:0/3482742839 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6c7c108630 0x7f6c7c19c370 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c003de0 tx=0x7f6c6c011ab0 comp rx=0 tx=0).stop 2024-03-21T09:55:52.910 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:52.187+0000 7f6c81400700 1 -- 172.21.15.153:0/3482742839 shutdown_connections 2024-03-21T09:55:52.910 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:52.187+0000 7f6c81400700 1 --2- 172.21.15.153:0/3482742839 >> [v2:172.21.15.153:6800/270837948,v1:172.21.15.153:6801/270837948] conn(0x7f6c60038320 0x7f6c6003a7e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:55:52.910 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:52.187+0000 7f6c81400700 1 --2- 172.21.15.153:0/3482742839 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6c7c108630 0x7f6c7c19c370 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:55:52.910 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:52.187+0000 7f6c81400700 1 -- 172.21.15.153:0/3482742839 >> 172.21.15.153:0/3482742839 conn(0x7f6c7c103c10 msgr2=0x7f6c7c105320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:55:52.911 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:52.188+0000 7f6c81400700 1 -- 172.21.15.153:0/3482742839 shutdown_connections 2024-03-21T09:55:52.911 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:52.188+0000 7f6c81400700 1 -- 172.21.15.153:0/3482742839 wait complete. 2024-03-21T09:55:53.461 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:53 smithi153 ceph-mon[29117]: from='client.? 172.21.15.153:0/3482742839' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-03-21T09:55:53.461 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:53 smithi153 ceph-mon[29117]: mgrmap e5: smithi153.tluxni(active, since 5s) 2024-03-21T09:55:54.722 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout { 2024-03-21T09:55:54.722 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-03-21T09:55:54.722 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-21T09:55:54.722 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "active_name": "smithi153.tluxni", 2024-03-21T09:55:54.722 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-21T09:55:54.722 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout } 2024-03-21T09:55:54.722 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:53.459+0000 7fd35e000700 1 Processor -- start 2024-03-21T09:55:54.722 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:53.459+0000 7fd35e000700 1 -- start start 2024-03-21T09:55:54.722 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:53.460+0000 7fd35e000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd358106430 0x7fd358106850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:55:54.723 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:53.460+0000 7fd35e000700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd358106e20 con 0x7fd358106430 2024-03-21T09:55:54.723 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:53.460+0000 7fd357400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd358106430 0x7fd358106850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:55:54.723 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:53.460+0000 7fd357400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd358106430 0x7fd358106850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:57692/0 (socket says 172.21.15.153:57692) 2024-03-21T09:55:54.723 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:53.460+0000 7fd357400700 1 -- 172.21.15.153:0/2769809779 learned_addr learned my addr 172.21.15.153:0/2769809779 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T09:55:54.723 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:53.461+0000 7fd357400700 1 -- 172.21.15.153:0/2769809779 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd358107630 con 0x7fd358106430 2024-03-21T09:55:54.723 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:53.461+0000 7fd357400700 1 --2- 172.21.15.153:0/2769809779 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd358106430 0x7fd358106850 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd3480098a0 tx=0x7fd348009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f6a9c6a82dd0e58 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:55:54.723 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:53.461+0000 7fd356000700 1 -- 172.21.15.153:0/2769809779 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd348004180 con 0x7fd358106430 2024-03-21T09:55:54.723 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:53.462+0000 7fd356000700 1 -- 172.21.15.153:0/2769809779 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd3480042e0 con 0x7fd358106430 2024-03-21T09:55:54.723 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:53.462+0000 7fd356000700 1 -- 172.21.15.153:0/2769809779 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd348011a50 con 0x7fd358106430 2024-03-21T09:55:54.723 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:53.462+0000 7fd35e000700 1 -- 172.21.15.153:0/2769809779 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd358106430 msgr2=0x7fd358106850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:55:54.723 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:53.462+0000 7fd35e000700 1 --2- 172.21.15.153:0/2769809779 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd358106430 0x7fd358106850 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd3480098a0 tx=0x7fd348009bb0 comp rx=0 tx=0).stop 2024-03-21T09:55:54.724 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:53.462+0000 7fd35e000700 1 -- 172.21.15.153:0/2769809779 shutdown_connections 2024-03-21T09:55:54.724 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:53.462+0000 7fd35e000700 1 --2- 172.21.15.153:0/2769809779 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd358106430 0x7fd358106850 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:55:54.724 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:53.462+0000 7fd35e000700 1 -- 172.21.15.153:0/2769809779 >> 172.21.15.153:0/2769809779 conn(0x7fd3581019b0 msgr2=0x7fd358103e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:55:54.724 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:53.462+0000 7fd35e000700 1 -- 172.21.15.153:0/2769809779 shutdown_connections 2024-03-21T09:55:54.724 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:53.463+0000 7fd35e000700 1 -- 172.21.15.153:0/2769809779 wait complete. 2024-03-21T09:55:54.724 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:53.463+0000 7fd35e000700 1 Processor -- start 2024-03-21T09:55:54.724 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:53.464+0000 7fd35e000700 1 -- start start 2024-03-21T09:55:54.724 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:53.464+0000 7fd35e000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd358106430 0x7fd35819c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:55:54.724 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:53.464+0000 7fd35e000700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd35819c880 con 0x7fd358106430 2024-03-21T09:55:54.724 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:53.464+0000 7fd357400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd358106430 0x7fd35819c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:55:54.725 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:53.464+0000 7fd357400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd358106430 0x7fd35819c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:57706/0 (socket says 172.21.15.153:57706) 2024-03-21T09:55:54.725 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:53.464+0000 7fd357400700 1 -- 172.21.15.153:0/3775100031 learned_addr learned my addr 172.21.15.153:0/3775100031 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T09:55:54.725 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:53.465+0000 7fd357400700 1 -- 172.21.15.153:0/3775100031 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd348009550 con 0x7fd358106430 2024-03-21T09:55:54.725 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:53.465+0000 7fd357400700 1 --2- 172.21.15.153:0/3775100031 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd358106430 0x7fd35819c340 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd348003de0 tx=0x7fd348011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:55:54.725 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:53.465+0000 7fd34fe00700 1 -- 172.21.15.153:0/3775100031 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd348003930 con 0x7fd358106430 2024-03-21T09:55:54.725 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:53.465+0000 7fd34fe00700 1 -- 172.21.15.153:0/3775100031 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd348003a90 con 0x7fd358106430 2024-03-21T09:55:54.725 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:53.465+0000 7fd34fe00700 1 -- 172.21.15.153:0/3775100031 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd3480196c0 con 0x7fd358106430 2024-03-21T09:55:54.725 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:53.465+0000 7fd35e000700 1 -- 172.21.15.153:0/3775100031 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd35819ca80 con 0x7fd358106430 2024-03-21T09:55:54.725 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:53.465+0000 7fd35e000700 1 -- 172.21.15.153:0/3775100031 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd35819cf20 con 0x7fd358106430 2024-03-21T09:55:54.725 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:53.466+0000 7fd35e000700 1 -- 172.21.15.153:0/3775100031 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3580623c0 con 0x7fd358106430 2024-03-21T09:55:54.725 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:53.467+0000 7fd34fe00700 1 -- 172.21.15.153:0/3775100031 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 45283+0+0 (secure 0 0 0) 0x7fd34801abc0 con 0x7fd358106430 2024-03-21T09:55:54.725 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:53.467+0000 7fd34fe00700 1 --2- 172.21.15.153:0/3775100031 >> [v2:172.21.15.153:6800/270837948,v1:172.21.15.153:6801/270837948] conn(0x7fd33c038370 0x7fd33c03a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:55:54.726 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:53.468+0000 7fd34fe00700 1 -- 172.21.15.153:0/3775100031 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fd34804c090 con 0x7fd358106430 2024-03-21T09:55:54.726 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:53.468+0000 7fd356a00700 1 -- 172.21.15.153:0/3775100031 >> [v2:172.21.15.153:6800/270837948,v1:172.21.15.153:6801/270837948] conn(0x7fd33c038370 msgr2=0x7fd33c03a830 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.153:6800/270837948 2024-03-21T09:55:54.726 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:53.468+0000 7fd356a00700 1 --2- 172.21.15.153:0/3775100031 >> [v2:172.21.15.153:6800/270837948,v1:172.21.15.153:6801/270837948] conn(0x7fd33c038370 0x7fd33c03a830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-21T09:55:54.726 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:53.473+0000 7fd34fe00700 1 -- 172.21.15.153:0/3775100031 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fd348010770 con 0x7fd358106430 2024-03-21T09:55:54.726 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:53.669+0000 7fd356a00700 1 -- 172.21.15.153:0/3775100031 >> [v2:172.21.15.153:6800/270837948,v1:172.21.15.153:6801/270837948] conn(0x7fd33c038370 msgr2=0x7fd33c03a830 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.153:6800/270837948 2024-03-21T09:55:54.726 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:53.669+0000 7fd356a00700 1 --2- 172.21.15.153:0/3775100031 >> [v2:172.21.15.153:6800/270837948,v1:172.21.15.153:6801/270837948] conn(0x7fd33c038370 0x7fd33c03a830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-21T09:55:54.726 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:53.743+0000 7fd35e000700 1 -- 172.21.15.153:0/3775100031 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7fd35819fd00 con 0x7fd358106430 2024-03-21T09:55:54.726 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:53.744+0000 7fd34fe00700 1 -- 172.21.15.153:0/3775100031 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+103 (secure 0 0 0) 0x7fd348010380 con 0x7fd358106430 2024-03-21T09:55:54.726 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:53.747+0000 7fd35e000700 1 -- 172.21.15.153:0/3775100031 >> [v2:172.21.15.153:6800/270837948,v1:172.21.15.153:6801/270837948] conn(0x7fd33c038370 msgr2=0x7fd33c03a830 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-21T09:55:54.726 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:53.747+0000 7fd35e000700 1 --2- 172.21.15.153:0/3775100031 >> [v2:172.21.15.153:6800/270837948,v1:172.21.15.153:6801/270837948] conn(0x7fd33c038370 0x7fd33c03a830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:55:54.727 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:53.747+0000 7fd35e000700 1 -- 172.21.15.153:0/3775100031 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd358106430 msgr2=0x7fd35819c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:55:54.727 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:53.747+0000 7fd35e000700 1 --2- 172.21.15.153:0/3775100031 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd358106430 0x7fd35819c340 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd348003de0 tx=0x7fd348011ab0 comp rx=0 tx=0).stop 2024-03-21T09:55:54.727 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:53.747+0000 7fd35e000700 1 -- 172.21.15.153:0/3775100031 shutdown_connections 2024-03-21T09:55:54.727 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:53.748+0000 7fd35e000700 1 --2- 172.21.15.153:0/3775100031 >> [v2:172.21.15.153:6800/270837948,v1:172.21.15.153:6801/270837948] conn(0x7fd33c038370 0x7fd33c03a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:55:54.727 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:53.748+0000 7fd35e000700 1 --2- 172.21.15.153:0/3775100031 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd358106430 0x7fd35819c340 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:55:54.727 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:53.748+0000 7fd35e000700 1 -- 172.21.15.153:0/3775100031 >> 172.21.15.153:0/3775100031 conn(0x7fd3581019b0 msgr2=0x7fd358103060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:55:54.727 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:53.748+0000 7fd35e000700 1 -- 172.21.15.153:0/3775100031 shutdown_connections 2024-03-21T09:55:54.727 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:53.748+0000 7fd35e000700 1 -- 172.21.15.153:0/3775100031 wait complete. 2024-03-21T09:55:54.727 INFO:teuthology.orchestra.run.smithi153.stdout:Waiting for the mgr to restart... 2024-03-21T09:55:54.727 INFO:teuthology.orchestra.run.smithi153.stdout:Waiting for mgr epoch 5... 2024-03-21T09:55:55.041 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:55:54 smithi153 ceph-mon[29117]: from='client.? 172.21.15.153:0/3775100031' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-21T09:56:01.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:00 smithi153 ceph-mon[29117]: Active manager daemon smithi153.tluxni restarted 2024-03-21T09:56:01.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:00 smithi153 ceph-mon[29117]: Activating manager daemon smithi153.tluxni 2024-03-21T09:56:01.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:00 smithi153 ceph-mon[29117]: osdmap e2: 0 total, 0 up, 0 in 2024-03-21T09:56:01.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:00 smithi153 ceph-mon[29117]: mgrmap e6: smithi153.tluxni(active, starting, since 0.00970439s) 2024-03-21T09:56:01.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:00 smithi153 ceph-mon[29117]: from='mgr.14120 172.21.15.153:0/1521951433' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mon metadata", "id": "smithi153"}]: dispatch 2024-03-21T09:56:01.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:00 smithi153 ceph-mon[29117]: from='mgr.14120 172.21.15.153:0/1521951433' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mgr metadata", "who": "smithi153.tluxni", "id": "smithi153.tluxni"}]: dispatch 2024-03-21T09:56:01.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:00 smithi153 ceph-mon[29117]: from='mgr.14120 172.21.15.153:0/1521951433' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-21T09:56:01.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:00 smithi153 ceph-mon[29117]: from='mgr.14120 172.21.15.153:0/1521951433' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-21T09:56:01.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:00 smithi153 ceph-mon[29117]: from='mgr.14120 172.21.15.153:0/1521951433' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-21T09:56:01.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:00 smithi153 ceph-mon[29117]: Manager daemon smithi153.tluxni is now available 2024-03-21T09:56:01.308 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:00 smithi153 ceph-mon[29117]: from='mgr.14120 172.21.15.153:0/1521951433' entity='mgr.smithi153.tluxni' 2024-03-21T09:56:01.308 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:00 smithi153 ceph-mon[29117]: from='mgr.14120 172.21.15.153:0/1521951433' entity='mgr.smithi153.tluxni' 2024-03-21T09:56:02.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:01 smithi153 ceph-mon[29117]: Found migration_current of "None". Setting to last migration. 2024-03-21T09:56:02.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:01 smithi153 ceph-mon[29117]: from='mgr.14120 172.21.15.153:0/1521951433' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:56:02.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:01 smithi153 ceph-mon[29117]: from='mgr.14120 172.21.15.153:0/1521951433' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:56:02.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:01 smithi153 ceph-mon[29117]: from='mgr.14120 172.21.15.153:0/1521951433' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:56:02.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:01 smithi153 ceph-mon[29117]: from='mgr.14120 172.21.15.153:0/1521951433' entity='mgr.smithi153.tluxni' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi153.tluxni/mirror_snapshot_schedule"}]: dispatch 2024-03-21T09:56:02.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:01 smithi153 ceph-mon[29117]: from='mgr.14120 172.21.15.153:0/1521951433' entity='mgr.smithi153.tluxni' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi153.tluxni/trash_purge_schedule"}]: dispatch 2024-03-21T09:56:02.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:01 smithi153 ceph-mon[29117]: mgrmap e7: smithi153.tluxni(active, since 1.01291s) 2024-03-21T09:56:02.616 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout { 2024-03-21T09:56:02.616 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-03-21T09:56:02.617 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-21T09:56:02.617 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout } 2024-03-21T09:56:02.617 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:55.339+0000 7f64cc400700 1 Processor -- start 2024-03-21T09:56:02.619 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:55.339+0000 7f64cc400700 1 -- start start 2024-03-21T09:56:02.619 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:55.340+0000 7f64cc400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f64c4106430 0x7f64c4106850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:56:02.619 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:55.340+0000 7f64cc400700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64c4106e20 con 0x7f64c4106430 2024-03-21T09:56:02.619 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:55.340+0000 7f64c9a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f64c4106430 0x7f64c4106850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:56:02.620 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:55.340+0000 7f64c9a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f64c4106430 0x7f64c4106850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:57718/0 (socket says 172.21.15.153:57718) 2024-03-21T09:56:02.620 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:55.340+0000 7f64c9a00700 1 -- 172.21.15.153:0/1286958238 learned_addr learned my addr 172.21.15.153:0/1286958238 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T09:56:02.620 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:55.341+0000 7f64c9a00700 1 -- 172.21.15.153:0/1286958238 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64c4107630 con 0x7f64c4106430 2024-03-21T09:56:02.620 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:55.341+0000 7f64c9a00700 1 --2- 172.21.15.153:0/1286958238 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f64c4106430 0x7f64c4106850 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f64b80098a0 tx=0x7f64b8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31147e192c73f3de server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:56:02.620 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:55.341+0000 7f64c3e00700 1 -- 172.21.15.153:0/1286958238 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f64b8004180 con 0x7f64c4106430 2024-03-21T09:56:02.620 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:55.341+0000 7f64c3e00700 1 -- 172.21.15.153:0/1286958238 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f64b80042e0 con 0x7f64c4106430 2024-03-21T09:56:02.620 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:55.341+0000 7f64c3e00700 1 -- 172.21.15.153:0/1286958238 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f64b8011a50 con 0x7f64c4106430 2024-03-21T09:56:02.620 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:55.342+0000 7f64cc400700 1 -- 172.21.15.153:0/1286958238 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f64c4106430 msgr2=0x7f64c4106850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:56:02.620 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:55.342+0000 7f64cc400700 1 --2- 172.21.15.153:0/1286958238 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f64c4106430 0x7f64c4106850 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f64b80098a0 tx=0x7f64b8009bb0 comp rx=0 tx=0).stop 2024-03-21T09:56:02.620 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:55.342+0000 7f64cc400700 1 -- 172.21.15.153:0/1286958238 shutdown_connections 2024-03-21T09:56:02.620 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:55.342+0000 7f64cc400700 1 --2- 172.21.15.153:0/1286958238 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f64c4106430 0x7f64c4106850 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:56:02.620 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:55.342+0000 7f64cc400700 1 -- 172.21.15.153:0/1286958238 >> 172.21.15.153:0/1286958238 conn(0x7f64c41019b0 msgr2=0x7f64c4103e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:56:02.620 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:55.342+0000 7f64cc400700 1 -- 172.21.15.153:0/1286958238 shutdown_connections 2024-03-21T09:56:02.621 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:55.342+0000 7f64cc400700 1 -- 172.21.15.153:0/1286958238 wait complete. 2024-03-21T09:56:02.621 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:55.343+0000 7f64cc400700 1 Processor -- start 2024-03-21T09:56:02.621 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:55.343+0000 7f64cc400700 1 -- start start 2024-03-21T09:56:02.621 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:55.344+0000 7f64cc400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f64c4106430 0x7f64c407ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:56:02.625 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:55.344+0000 7f64cc400700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64c407b090 con 0x7f64c4106430 2024-03-21T09:56:02.625 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:55.344+0000 7f64c9a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f64c4106430 0x7f64c407ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:56:02.625 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:55.344+0000 7f64c9a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f64c4106430 0x7f64c407ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:57728/0 (socket says 172.21.15.153:57728) 2024-03-21T09:56:02.625 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:55.344+0000 7f64c9a00700 1 -- 172.21.15.153:0/2273237093 learned_addr learned my addr 172.21.15.153:0/2273237093 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T09:56:02.625 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:55.344+0000 7f64c9a00700 1 -- 172.21.15.153:0/2273237093 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64b8009550 con 0x7f64c4106430 2024-03-21T09:56:02.625 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:55.345+0000 7f64c9a00700 1 --2- 172.21.15.153:0/2273237093 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f64c4106430 0x7f64c407ab50 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f64b8003de0 tx=0x7f64b8011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:56:02.625 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:55.345+0000 7f64c2000700 1 -- 172.21.15.153:0/2273237093 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f64b8003930 con 0x7f64c4106430 2024-03-21T09:56:02.625 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:55.345+0000 7f64c2000700 1 -- 172.21.15.153:0/2273237093 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f64b8003a90 con 0x7f64c4106430 2024-03-21T09:56:02.625 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:55.345+0000 7f64c2000700 1 -- 172.21.15.153:0/2273237093 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f64b80196c0 con 0x7f64c4106430 2024-03-21T09:56:02.625 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:55.345+0000 7f64cc400700 1 -- 172.21.15.153:0/2273237093 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64c40791a0 con 0x7f64c4106430 2024-03-21T09:56:02.626 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:55.345+0000 7f64cc400700 1 -- 172.21.15.153:0/2273237093 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64c4079640 con 0x7f64c4106430 2024-03-21T09:56:02.626 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:55.347+0000 7f64c2000700 1 -- 172.21.15.153:0/2273237093 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 5) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f64b8019820 con 0x7f64c4106430 2024-03-21T09:56:02.626 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:55.347+0000 7f64c2000700 1 --2- 172.21.15.153:0/2273237093 >> [v2:172.21.15.153:6800/270837948,v1:172.21.15.153:6801/270837948] conn(0x7f64ac038370 0x7f64ac03a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:56:02.626 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:55.347+0000 7f64c2000700 1 -- 172.21.15.153:0/2273237093 --> [v2:172.21.15.153:6800/270837948,v1:172.21.15.153:6801/270837948] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f64ac03af40 con 0x7f64ac038370 2024-03-21T09:56:02.626 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:55.347+0000 7f64c2000700 1 -- 172.21.15.153:0/2273237093 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f64b804c270 con 0x7f64c4106430 2024-03-21T09:56:02.626 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:55.347+0000 7f64c9000700 1 -- 172.21.15.153:0/2273237093 >> [v2:172.21.15.153:6800/270837948,v1:172.21.15.153:6801/270837948] conn(0x7f64ac038370 msgr2=0x7f64ac03a830 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.153:6800/270837948 2024-03-21T09:56:02.626 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:55.347+0000 7f64c9000700 1 --2- 172.21.15.153:0/2273237093 >> [v2:172.21.15.153:6800/270837948,v1:172.21.15.153:6801/270837948] conn(0x7f64ac038370 0x7f64ac03a830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-21T09:56:02.626 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:55.548+0000 7f64c9000700 1 -- 172.21.15.153:0/2273237093 >> [v2:172.21.15.153:6800/270837948,v1:172.21.15.153:6801/270837948] conn(0x7f64ac038370 msgr2=0x7f64ac03a830 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.153:6800/270837948 2024-03-21T09:56:02.626 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:55.548+0000 7f64c9000700 1 --2- 172.21.15.153:0/2273237093 >> [v2:172.21.15.153:6800/270837948,v1:172.21.15.153:6801/270837948] conn(0x7f64ac038370 0x7f64ac03a830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-21T09:56:02.626 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:55.949+0000 7f64c9000700 1 -- 172.21.15.153:0/2273237093 >> [v2:172.21.15.153:6800/270837948,v1:172.21.15.153:6801/270837948] conn(0x7f64ac038370 msgr2=0x7f64ac03a830 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.153:6800/270837948 2024-03-21T09:56:02.626 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:55.949+0000 7f64c9000700 1 --2- 172.21.15.153:0/2273237093 >> [v2:172.21.15.153:6800/270837948,v1:172.21.15.153:6801/270837948] conn(0x7f64ac038370 0x7f64ac03a830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-21T09:56:02.627 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:56.750+0000 7f64c9000700 1 -- 172.21.15.153:0/2273237093 >> [v2:172.21.15.153:6800/270837948,v1:172.21.15.153:6801/270837948] conn(0x7f64ac038370 msgr2=0x7f64ac03a830 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.153:6800/270837948 2024-03-21T09:56:02.627 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:56.750+0000 7f64c9000700 1 --2- 172.21.15.153:0/2273237093 >> [v2:172.21.15.153:6800/270837948,v1:172.21.15.153:6801/270837948] conn(0x7f64ac038370 0x7f64ac03a830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-21T09:56:02.627 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:58.352+0000 7f64c9000700 1 -- 172.21.15.153:0/2273237093 >> [v2:172.21.15.153:6800/270837948,v1:172.21.15.153:6801/270837948] conn(0x7f64ac038370 msgr2=0x7f64ac03a830 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.153:6800/270837948 2024-03-21T09:56:02.627 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:55:58.352+0000 7f64c9000700 1 --2- 172.21.15.153:0/2273237093 >> [v2:172.21.15.153:6800/270837948,v1:172.21.15.153:6801/270837948] conn(0x7f64ac038370 0x7f64ac03a830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-21T09:56:02.627 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:00.855+0000 7f64c2000700 1 -- 172.21.15.153:0/2273237093 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mgrmap(e 6) v1 ==== 45050+0+0 (secure 0 0 0) 0x7f64b801a430 con 0x7f64c4106430 2024-03-21T09:56:02.627 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:00.855+0000 7f64c2000700 1 -- 172.21.15.153:0/2273237093 >> [v2:172.21.15.153:6800/270837948,v1:172.21.15.153:6801/270837948] conn(0x7f64ac038370 msgr2=0x7f64ac03a830 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-21T09:56:02.627 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:00.855+0000 7f64c2000700 1 --2- 172.21.15.153:0/2273237093 >> [v2:172.21.15.153:6800/270837948,v1:172.21.15.153:6801/270837948] conn(0x7f64ac038370 0x7f64ac03a830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:56:02.627 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:01.855+0000 7f64c2000700 1 -- 172.21.15.153:0/2273237093 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mgrmap(e 7) v1 ==== 45177+0+0 (secure 0 0 0) 0x7f64b801abc0 con 0x7f64c4106430 2024-03-21T09:56:02.627 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:01.855+0000 7f64c2000700 1 --2- 172.21.15.153:0/2273237093 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7f64ac03bc50 0x7f64ac03e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:56:02.627 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:01.855+0000 7f64c2000700 1 -- 172.21.15.153:0/2273237093 --> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f64ac03af40 con 0x7f64ac03bc50 2024-03-21T09:56:02.627 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:01.858+0000 7f64c9000700 1 --2- 172.21.15.153:0/2273237093 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7f64ac03bc50 0x7f64ac03e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:56:02.627 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:01.861+0000 7f64c9000700 1 --2- 172.21.15.153:0/2273237093 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7f64ac03bc50 0x7f64ac03e040 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f64b0003a10 tx=0x7f64b00092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:56:02.627 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:01.862+0000 7f64c2000700 1 -- 172.21.15.153:0/2273237093 <== mgr.14120 v2:172.21.15.153:6800/3178125243 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7f64ac03af40 con 0x7f64ac03bc50 2024-03-21T09:56:02.628 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:01.869+0000 7f64cc400700 1 -- 172.21.15.153:0/2273237093 --> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f64c407a6f0 con 0x7f64ac03bc50 2024-03-21T09:56:02.628 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:01.870+0000 7f64c2000700 1 -- 172.21.15.153:0/2273237093 <== mgr.14120 v2:172.21.15.153:6800/3178125243 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f64c407a6f0 con 0x7f64ac03bc50 2024-03-21T09:56:02.628 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:01.870+0000 7f64cc400700 1 -- 172.21.15.153:0/2273237093 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7f64ac03bc50 msgr2=0x7f64ac03e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:56:02.628 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:01.870+0000 7f64cc400700 1 --2- 172.21.15.153:0/2273237093 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7f64ac03bc50 0x7f64ac03e040 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f64b0003a10 tx=0x7f64b00092b0 comp rx=0 tx=0).stop 2024-03-21T09:56:02.628 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:01.870+0000 7f64cc400700 1 -- 172.21.15.153:0/2273237093 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f64c4106430 msgr2=0x7f64c407ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:56:02.628 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:01.870+0000 7f64cc400700 1 --2- 172.21.15.153:0/2273237093 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f64c4106430 0x7f64c407ab50 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f64b8003de0 tx=0x7f64b8011ab0 comp rx=0 tx=0).stop 2024-03-21T09:56:02.628 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:01.871+0000 7f64cc400700 1 -- 172.21.15.153:0/2273237093 shutdown_connections 2024-03-21T09:56:02.628 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:01.871+0000 7f64cc400700 1 --2- 172.21.15.153:0/2273237093 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7f64ac03bc50 0x7f64ac03e040 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:56:02.628 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:01.871+0000 7f64cc400700 1 --2- 172.21.15.153:0/2273237093 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f64c4106430 0x7f64c407ab50 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:56:02.628 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:01.871+0000 7f64cc400700 1 --2- 172.21.15.153:0/2273237093 >> [v2:172.21.15.153:6800/270837948,v1:172.21.15.153:6801/270837948] conn(0x7f64ac038370 0x7f64ac03a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:56:02.629 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:01.871+0000 7f64cc400700 1 -- 172.21.15.153:0/2273237093 >> 172.21.15.153:0/2273237093 conn(0x7f64c41019b0 msgr2=0x7f64c41030a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:56:02.629 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:01.871+0000 7f64cc400700 1 -- 172.21.15.153:0/2273237093 shutdown_connections 2024-03-21T09:56:02.629 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:01.871+0000 7f64cc400700 1 -- 172.21.15.153:0/2273237093 wait complete. 2024-03-21T09:56:02.629 INFO:teuthology.orchestra.run.smithi153.stdout:mgr epoch 5 is available 2024-03-21T09:56:02.629 INFO:teuthology.orchestra.run.smithi153.stdout:Setting orchestrator backend to cephadm... 2024-03-21T09:56:03.206 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:02 smithi153 ceph-mon[29117]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-21T09:56:03.206 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:02 smithi153 ceph-mon[29117]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-21T09:56:03.207 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:02 smithi153 ceph-mon[29117]: from='mgr.14120 172.21.15.153:0/1521951433' entity='mgr.smithi153.tluxni' 2024-03-21T09:56:03.207 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:02 smithi153 ceph-mon[29117]: from='mgr.14120 172.21.15.153:0/1521951433' entity='mgr.smithi153.tluxni' 2024-03-21T09:56:03.207 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:02 smithi153 ceph-mon[29117]: from='mgr.14120 172.21.15.153:0/1521951433' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:56:04.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:03 smithi153 ceph-mon[29117]: [21/Mar/2024:09:56:02] ENGINE Bus STARTING 2024-03-21T09:56:04.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:04 smithi153 ceph-mon[29117]: [21/Mar/2024:09:56:02] ENGINE Serving on https://172.21.15.153:7150 2024-03-21T09:56:04.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:04 smithi153 ceph-mon[29117]: [21/Mar/2024:09:56:02] ENGINE Serving on http://172.21.15.153:8765 2024-03-21T09:56:04.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:04 smithi153 ceph-mon[29117]: [21/Mar/2024:09:56:02] ENGINE Bus STARTED 2024-03-21T09:56:04.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:04 smithi153 ceph-mon[29117]: mgrmap e8: smithi153.tluxni(active, since 2s) 2024-03-21T09:56:04.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:04 smithi153 ceph-mon[29117]: from='mgr.14120 172.21.15.153:0/1521951433' entity='mgr.smithi153.tluxni' 2024-03-21T09:56:04.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:04 smithi153 ceph-mon[29117]: from='mgr.14120 172.21.15.153:0/1521951433' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:56:04.144 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:03.204+0000 7f212f400700 1 Processor -- start 2024-03-21T09:56:04.144 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:03.204+0000 7f212f400700 1 -- start start 2024-03-21T09:56:04.145 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:03.205+0000 7f212f400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2128106420 0x7f2128106840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:56:04.145 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:03.205+0000 7f212f400700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2128106e10 con 0x7f2128106420 2024-03-21T09:56:04.145 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:03.205+0000 7f212ca00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2128106420 0x7f2128106840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:56:04.145 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:03.205+0000 7f212ca00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2128106420 0x7f2128106840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:49676/0 (socket says 172.21.15.153:49676) 2024-03-21T09:56:04.145 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:03.205+0000 7f212ca00700 1 -- 172.21.15.153:0/353550896 learned_addr learned my addr 172.21.15.153:0/353550896 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T09:56:04.145 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:03.206+0000 7f212ca00700 1 -- 172.21.15.153:0/353550896 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2128107620 con 0x7f2128106420 2024-03-21T09:56:04.145 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:03.206+0000 7f212ca00700 1 --2- 172.21.15.153:0/353550896 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2128106420 0x7f2128106840 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f211c0098a0 tx=0x7f211c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f492378884b2b3f6 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:56:04.145 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:03.206+0000 7f2127400700 1 -- 172.21.15.153:0/353550896 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f211c004180 con 0x7f2128106420 2024-03-21T09:56:04.145 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:03.206+0000 7f2127400700 1 -- 172.21.15.153:0/353550896 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f211c0042e0 con 0x7f2128106420 2024-03-21T09:56:04.145 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:03.206+0000 7f2127400700 1 -- 172.21.15.153:0/353550896 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f211c011a50 con 0x7f2128106420 2024-03-21T09:56:04.145 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:03.207+0000 7f212f400700 1 -- 172.21.15.153:0/353550896 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2128106420 msgr2=0x7f2128106840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:56:04.145 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:03.207+0000 7f212f400700 1 --2- 172.21.15.153:0/353550896 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2128106420 0x7f2128106840 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f211c0098a0 tx=0x7f211c009bb0 comp rx=0 tx=0).stop 2024-03-21T09:56:04.145 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:03.208+0000 7f212f400700 1 -- 172.21.15.153:0/353550896 shutdown_connections 2024-03-21T09:56:04.145 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:03.208+0000 7f212f400700 1 --2- 172.21.15.153:0/353550896 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2128106420 0x7f2128106840 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:56:04.146 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:03.208+0000 7f212f400700 1 -- 172.21.15.153:0/353550896 >> 172.21.15.153:0/353550896 conn(0x7f2128101a00 msgr2=0x7f2128103e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:56:04.146 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:03.208+0000 7f212f400700 1 -- 172.21.15.153:0/353550896 shutdown_connections 2024-03-21T09:56:04.146 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:03.208+0000 7f212f400700 1 -- 172.21.15.153:0/353550896 wait complete. 2024-03-21T09:56:04.146 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:03.209+0000 7f212f400700 1 Processor -- start 2024-03-21T09:56:04.146 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:03.209+0000 7f212f400700 1 -- start start 2024-03-21T09:56:04.146 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:03.210+0000 7f212f400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2128106420 0x7f212819c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:56:04.146 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:03.210+0000 7f212f400700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f212819c7f0 con 0x7f2128106420 2024-03-21T09:56:04.146 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:03.210+0000 7f212ca00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2128106420 0x7f212819c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:56:04.146 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:03.210+0000 7f212ca00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2128106420 0x7f212819c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:49688/0 (socket says 172.21.15.153:49688) 2024-03-21T09:56:04.146 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:03.210+0000 7f212ca00700 1 -- 172.21.15.153:0/1850172058 learned_addr learned my addr 172.21.15.153:0/1850172058 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T09:56:04.146 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:03.210+0000 7f212ca00700 1 -- 172.21.15.153:0/1850172058 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f211c009550 con 0x7f2128106420 2024-03-21T09:56:04.146 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:03.211+0000 7f212ca00700 1 --2- 172.21.15.153:0/1850172058 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2128106420 0x7f212819c2b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f211c000c00 tx=0x7f211c011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:56:04.146 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:03.211+0000 7f2125600700 1 -- 172.21.15.153:0/1850172058 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f211c003b90 con 0x7f2128106420 2024-03-21T09:56:04.146 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:03.211+0000 7f2125600700 1 -- 172.21.15.153:0/1850172058 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f211c0248b0 con 0x7f2128106420 2024-03-21T09:56:04.147 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:03.211+0000 7f212f400700 1 -- 172.21.15.153:0/1850172058 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f212819c9f0 con 0x7f2128106420 2024-03-21T09:56:04.147 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:03.211+0000 7f2125600700 1 -- 172.21.15.153:0/1850172058 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f211c01a8e0 con 0x7f2128106420 2024-03-21T09:56:04.147 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:03.211+0000 7f212f400700 1 -- 172.21.15.153:0/1850172058 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f212819ce90 con 0x7f2128106420 2024-03-21T09:56:04.147 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:03.212+0000 7f2125600700 1 -- 172.21.15.153:0/1850172058 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f211c024a20 con 0x7f2128106420 2024-03-21T09:56:04.147 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:03.213+0000 7f212f400700 1 -- 172.21.15.153:0/1850172058 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2128196120 con 0x7f2128106420 2024-03-21T09:56:04.147 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:03.213+0000 7f2125600700 1 --2- 172.21.15.153:0/1850172058 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7f2110038370 0x7f211003a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:56:04.147 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:03.213+0000 7f2125600700 1 -- 172.21.15.153:0/1850172058 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f211c04c410 con 0x7f2128106420 2024-03-21T09:56:04.147 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:03.213+0000 7f2127e00700 1 --2- 172.21.15.153:0/1850172058 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7f2110038370 0x7f211003a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:56:04.147 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:03.214+0000 7f2127e00700 1 --2- 172.21.15.153:0/1850172058 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7f2110038370 0x7f211003a830 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2114006fd0 tx=0x7f2114006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:56:04.147 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:03.219+0000 7f2125600700 1 -- 172.21.15.153:0/1850172058 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f211c0189a0 con 0x7f2128106420 2024-03-21T09:56:04.147 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:03.449+0000 7f212f400700 1 -- 172.21.15.153:0/1850172058 --> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f21280611d0 con 0x7f2110038370 2024-03-21T09:56:04.147 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:03.466+0000 7f2125600700 1 -- 172.21.15.153:0/1850172058 <== mgr.14120 v2:172.21.15.153:6800/3178125243 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f21280611d0 con 0x7f2110038370 2024-03-21T09:56:04.147 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:03.469+0000 7f212f400700 1 -- 172.21.15.153:0/1850172058 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7f2110038370 msgr2=0x7f211003a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:56:04.147 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:03.469+0000 7f212f400700 1 --2- 172.21.15.153:0/1850172058 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7f2110038370 0x7f211003a830 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2114006fd0 tx=0x7f2114006e40 comp rx=0 tx=0).stop 2024-03-21T09:56:04.148 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:03.469+0000 7f212f400700 1 -- 172.21.15.153:0/1850172058 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2128106420 msgr2=0x7f212819c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:56:04.148 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:03.469+0000 7f212f400700 1 --2- 172.21.15.153:0/1850172058 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2128106420 0x7f212819c2b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f211c000c00 tx=0x7f211c011ab0 comp rx=0 tx=0).stop 2024-03-21T09:56:04.148 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:03.469+0000 7f212f400700 1 -- 172.21.15.153:0/1850172058 shutdown_connections 2024-03-21T09:56:04.148 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:03.469+0000 7f212f400700 1 --2- 172.21.15.153:0/1850172058 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7f2110038370 0x7f211003a830 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:56:04.148 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:03.469+0000 7f212f400700 1 --2- 172.21.15.153:0/1850172058 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2128106420 0x7f212819c2b0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:56:04.148 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:03.469+0000 7f212f400700 1 -- 172.21.15.153:0/1850172058 >> 172.21.15.153:0/1850172058 conn(0x7f2128101a00 msgr2=0x7f2128109dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:56:04.148 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:03.469+0000 7f212f400700 1 -- 172.21.15.153:0/1850172058 shutdown_connections 2024-03-21T09:56:04.148 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:03.469+0000 7f212f400700 1 -- 172.21.15.153:0/1850172058 wait complete. 2024-03-21T09:56:05.286 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:05 smithi153 ceph-mon[29117]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:56:05.597 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout value unchanged 2024-03-21T09:56:05.597 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:04.725+0000 7ffb13a00700 1 Processor -- start 2024-03-21T09:56:05.597 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:04.725+0000 7ffb13a00700 1 -- start start 2024-03-21T09:56:05.598 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:04.726+0000 7ffb13a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb0c07ad80 0x7ffb0c0791e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:56:05.598 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:04.726+0000 7ffb13a00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb0c0797b0 con 0x7ffb0c07ad80 2024-03-21T09:56:05.598 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:04.727+0000 7ffb11000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb0c07ad80 0x7ffb0c0791e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:56:05.598 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:04.727+0000 7ffb11000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb0c07ad80 0x7ffb0c0791e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:49702/0 (socket says 172.21.15.153:49702) 2024-03-21T09:56:05.598 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:04.727+0000 7ffb11000700 1 -- 172.21.15.153:0/2198344198 learned_addr learned my addr 172.21.15.153:0/2198344198 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T09:56:05.598 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:04.728+0000 7ffb11000700 1 -- 172.21.15.153:0/2198344198 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb0c0798f0 con 0x7ffb0c07ad80 2024-03-21T09:56:05.598 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:04.728+0000 7ffb11000700 1 --2- 172.21.15.153:0/2198344198 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb0c07ad80 0x7ffb0c0791e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ffaf40098a0 tx=0x7ffaf4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec010b6a41284edb server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:56:05.598 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:04.728+0000 7ffb0be00700 1 -- 172.21.15.153:0/2198344198 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ffaf4004180 con 0x7ffb0c07ad80 2024-03-21T09:56:05.598 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:04.729+0000 7ffb0be00700 1 -- 172.21.15.153:0/2198344198 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7ffaf40042e0 con 0x7ffb0c07ad80 2024-03-21T09:56:05.598 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:04.729+0000 7ffb0be00700 1 -- 172.21.15.153:0/2198344198 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ffaf4011a50 con 0x7ffb0c07ad80 2024-03-21T09:56:05.598 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:04.729+0000 7ffb13a00700 1 -- 172.21.15.153:0/2198344198 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb0c07ad80 msgr2=0x7ffb0c0791e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:56:05.598 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:04.729+0000 7ffb13a00700 1 --2- 172.21.15.153:0/2198344198 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb0c07ad80 0x7ffb0c0791e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ffaf40098a0 tx=0x7ffaf4009bb0 comp rx=0 tx=0).stop 2024-03-21T09:56:05.598 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:04.729+0000 7ffb13a00700 1 -- 172.21.15.153:0/2198344198 shutdown_connections 2024-03-21T09:56:05.599 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:04.729+0000 7ffb13a00700 1 --2- 172.21.15.153:0/2198344198 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb0c07ad80 0x7ffb0c0791e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:56:05.599 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:04.729+0000 7ffb13a00700 1 -- 172.21.15.153:0/2198344198 >> 172.21.15.153:0/2198344198 conn(0x7ffb0c1019b0 msgr2=0x7ffb0c103e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:56:05.599 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:04.730+0000 7ffb13a00700 1 -- 172.21.15.153:0/2198344198 shutdown_connections 2024-03-21T09:56:05.599 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:04.730+0000 7ffb13a00700 1 -- 172.21.15.153:0/2198344198 wait complete. 2024-03-21T09:56:05.599 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:04.731+0000 7ffb13a00700 1 Processor -- start 2024-03-21T09:56:05.599 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:04.731+0000 7ffb13a00700 1 -- start start 2024-03-21T09:56:05.599 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:04.731+0000 7ffb13a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb0c07ad80 0x7ffb0c19c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:56:05.599 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:04.731+0000 7ffb13a00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb0c19c820 con 0x7ffb0c07ad80 2024-03-21T09:56:05.599 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:04.732+0000 7ffb11000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb0c07ad80 0x7ffb0c19c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:56:05.599 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:04.732+0000 7ffb11000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb0c07ad80 0x7ffb0c19c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:49714/0 (socket says 172.21.15.153:49714) 2024-03-21T09:56:05.599 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:04.732+0000 7ffb11000700 1 -- 172.21.15.153:0/1532687816 learned_addr learned my addr 172.21.15.153:0/1532687816 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T09:56:05.599 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:04.732+0000 7ffb11000700 1 -- 172.21.15.153:0/1532687816 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaf4009550 con 0x7ffb0c07ad80 2024-03-21T09:56:05.599 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:04.732+0000 7ffb11000700 1 --2- 172.21.15.153:0/1532687816 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb0c07ad80 0x7ffb0c19c2e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ffaf4003de0 tx=0x7ffaf4011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:56:05.600 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:04.733+0000 7ffb0a000700 1 -- 172.21.15.153:0/1532687816 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ffaf4003930 con 0x7ffb0c07ad80 2024-03-21T09:56:05.600 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:04.733+0000 7ffb0a000700 1 -- 172.21.15.153:0/1532687816 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7ffaf4003a90 con 0x7ffb0c07ad80 2024-03-21T09:56:05.600 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:04.733+0000 7ffb13a00700 1 -- 172.21.15.153:0/1532687816 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb0c19ca20 con 0x7ffb0c07ad80 2024-03-21T09:56:05.600 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:04.733+0000 7ffb0a000700 1 -- 172.21.15.153:0/1532687816 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ffaf4019730 con 0x7ffb0c07ad80 2024-03-21T09:56:05.600 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:04.733+0000 7ffb13a00700 1 -- 172.21.15.153:0/1532687816 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb0c19cec0 con 0x7ffb0c07ad80 2024-03-21T09:56:05.600 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:04.734+0000 7ffb0a000700 1 -- 172.21.15.153:0/1532687816 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7ffaf40199b0 con 0x7ffb0c07ad80 2024-03-21T09:56:05.600 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:04.734+0000 7ffb13a00700 1 -- 172.21.15.153:0/1532687816 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb0c10c7b0 con 0x7ffb0c07ad80 2024-03-21T09:56:05.600 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:04.734+0000 7ffb0a000700 1 --2- 172.21.15.153:0/1532687816 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7ffaec038320 0x7ffaec03a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:56:05.600 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:04.734+0000 7ffb0a000700 1 -- 172.21.15.153:0/1532687816 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7ffaf404c420 con 0x7ffb0c07ad80 2024-03-21T09:56:05.600 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:04.735+0000 7ffb03e00700 1 --2- 172.21.15.153:0/1532687816 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7ffaec038320 0x7ffaec03a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:56:05.600 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:04.735+0000 7ffb03e00700 1 --2- 172.21.15.153:0/1532687816 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7ffaec038320 0x7ffaec03a7e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ffafc006fd0 tx=0x7ffafc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:56:05.600 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:04.740+0000 7ffb0a000700 1 -- 172.21.15.153:0/1532687816 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7ffaf4023370 con 0x7ffb0c07ad80 2024-03-21T09:56:05.600 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:04.959+0000 7ffb13a00700 1 -- 172.21.15.153:0/1532687816 --> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7ffb0c103ce0 con 0x7ffaec038320 2024-03-21T09:56:05.601 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:04.961+0000 7ffb0a000700 1 -- 172.21.15.153:0/1532687816 <== mgr.14120 v2:172.21.15.153:6800/3178125243 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7ffb0c103ce0 con 0x7ffaec038320 2024-03-21T09:56:05.601 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:04.964+0000 7ffb13a00700 1 -- 172.21.15.153:0/1532687816 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7ffaec038320 msgr2=0x7ffaec03a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:56:05.601 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:04.964+0000 7ffb13a00700 1 --2- 172.21.15.153:0/1532687816 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7ffaec038320 0x7ffaec03a7e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ffafc006fd0 tx=0x7ffafc006e40 comp rx=0 tx=0).stop 2024-03-21T09:56:05.601 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:04.964+0000 7ffb13a00700 1 -- 172.21.15.153:0/1532687816 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb0c07ad80 msgr2=0x7ffb0c19c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:56:05.601 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:04.964+0000 7ffb13a00700 1 --2- 172.21.15.153:0/1532687816 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb0c07ad80 0x7ffb0c19c2e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ffaf4003de0 tx=0x7ffaf4011ab0 comp rx=0 tx=0).stop 2024-03-21T09:56:05.601 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:04.964+0000 7ffb13a00700 1 -- 172.21.15.153:0/1532687816 shutdown_connections 2024-03-21T09:56:05.601 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:04.964+0000 7ffb13a00700 1 --2- 172.21.15.153:0/1532687816 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7ffaec038320 0x7ffaec03a7e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:56:05.601 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:04.964+0000 7ffb13a00700 1 --2- 172.21.15.153:0/1532687816 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffb0c07ad80 0x7ffb0c19c2e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:56:05.601 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:04.964+0000 7ffb13a00700 1 -- 172.21.15.153:0/1532687816 >> 172.21.15.153:0/1532687816 conn(0x7ffb0c1019b0 msgr2=0x7ffb0c1030c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:56:05.601 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:04.964+0000 7ffb13a00700 1 -- 172.21.15.153:0/1532687816 shutdown_connections 2024-03-21T09:56:05.601 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:04.964+0000 7ffb13a00700 1 -- 172.21.15.153:0/1532687816 wait complete. 2024-03-21T09:56:05.602 INFO:teuthology.orchestra.run.smithi153.stdout:Generating ssh key... 2024-03-21T09:56:06.164 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:06 smithi153 ceph-mon[29117]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:56:07.933 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:06.162+0000 7f408b800700 1 Processor -- start 2024-03-21T09:56:07.933 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:06.162+0000 7f408b800700 1 -- start start 2024-03-21T09:56:07.933 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:06.163+0000 7f408b800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4084106420 0x7f4084106840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:56:07.933 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:06.163+0000 7f408b800700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4084106e10 con 0x7f4084106420 2024-03-21T09:56:07.933 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:06.163+0000 7f4088e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4084106420 0x7f4084106840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:56:07.933 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:06.163+0000 7f4088e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4084106420 0x7f4084106840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:49718/0 (socket says 172.21.15.153:49718) 2024-03-21T09:56:07.937 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:06.163+0000 7f4088e00700 1 -- 172.21.15.153:0/2066648784 learned_addr learned my addr 172.21.15.153:0/2066648784 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T09:56:07.937 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:06.164+0000 7f4088e00700 1 -- 172.21.15.153:0/2066648784 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4084107620 con 0x7f4084106420 2024-03-21T09:56:07.937 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:06.164+0000 7f4088e00700 1 --2- 172.21.15.153:0/2066648784 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4084106420 0x7f4084106840 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f40780098a0 tx=0x7f4078009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9296d6838faf21 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:56:07.937 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:06.164+0000 7f4083400700 1 -- 172.21.15.153:0/2066648784 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4078004180 con 0x7f4084106420 2024-03-21T09:56:07.937 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:06.164+0000 7f4083400700 1 -- 172.21.15.153:0/2066648784 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f40780042e0 con 0x7f4084106420 2024-03-21T09:56:07.937 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:06.164+0000 7f4083400700 1 -- 172.21.15.153:0/2066648784 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4078011a50 con 0x7f4084106420 2024-03-21T09:56:07.937 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:06.165+0000 7f408b800700 1 -- 172.21.15.153:0/2066648784 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4084106420 msgr2=0x7f4084106840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:56:07.937 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:06.165+0000 7f408b800700 1 --2- 172.21.15.153:0/2066648784 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4084106420 0x7f4084106840 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f40780098a0 tx=0x7f4078009bb0 comp rx=0 tx=0).stop 2024-03-21T09:56:07.937 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:06.165+0000 7f408b800700 1 -- 172.21.15.153:0/2066648784 shutdown_connections 2024-03-21T09:56:07.937 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:06.165+0000 7f408b800700 1 --2- 172.21.15.153:0/2066648784 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4084106420 0x7f4084106840 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:56:07.937 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:06.165+0000 7f408b800700 1 -- 172.21.15.153:0/2066648784 >> 172.21.15.153:0/2066648784 conn(0x7f4084101a00 msgr2=0x7f4084103e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:56:07.937 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:06.165+0000 7f408b800700 1 -- 172.21.15.153:0/2066648784 shutdown_connections 2024-03-21T09:56:07.938 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:06.165+0000 7f408b800700 1 -- 172.21.15.153:0/2066648784 wait complete. 2024-03-21T09:56:07.938 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:06.166+0000 7f408b800700 1 Processor -- start 2024-03-21T09:56:07.938 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:06.166+0000 7f408b800700 1 -- start start 2024-03-21T09:56:07.938 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:06.167+0000 7f408b800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4084106420 0x7f408419c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:56:07.938 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:06.167+0000 7f408b800700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f408419c7e0 con 0x7f4084106420 2024-03-21T09:56:07.938 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:06.167+0000 7f4088e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4084106420 0x7f408419c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:56:07.938 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:06.167+0000 7f4088e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4084106420 0x7f408419c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:49732/0 (socket says 172.21.15.153:49732) 2024-03-21T09:56:07.938 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:06.167+0000 7f4088e00700 1 -- 172.21.15.153:0/2201657052 learned_addr learned my addr 172.21.15.153:0/2201657052 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T09:56:07.938 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:06.167+0000 7f4088e00700 1 -- 172.21.15.153:0/2201657052 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4078009550 con 0x7f4084106420 2024-03-21T09:56:07.938 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:06.168+0000 7f4088e00700 1 --2- 172.21.15.153:0/2201657052 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4084106420 0x7f408419c2a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4078000c00 tx=0x7f4078011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:56:07.938 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:06.168+0000 7f4081600700 1 -- 172.21.15.153:0/2201657052 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4078003b90 con 0x7f4084106420 2024-03-21T09:56:07.938 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:06.168+0000 7f4081600700 1 -- 172.21.15.153:0/2201657052 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f40780248b0 con 0x7f4084106420 2024-03-21T09:56:07.938 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:06.168+0000 7f408b800700 1 -- 172.21.15.153:0/2201657052 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f408419c9e0 con 0x7f4084106420 2024-03-21T09:56:07.939 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:06.168+0000 7f4081600700 1 -- 172.21.15.153:0/2201657052 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f407801a8e0 con 0x7f4084106420 2024-03-21T09:56:07.939 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:06.168+0000 7f408b800700 1 -- 172.21.15.153:0/2201657052 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f408419ce80 con 0x7f4084106420 2024-03-21T09:56:07.939 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:06.169+0000 7f4081600700 1 -- 172.21.15.153:0/2201657052 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f4078024a20 con 0x7f4084106420 2024-03-21T09:56:07.941 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:06.169+0000 7f408b800700 1 -- 172.21.15.153:0/2201657052 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40840623c0 con 0x7f4084106420 2024-03-21T09:56:07.941 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:06.170+0000 7f4081600700 1 --2- 172.21.15.153:0/2201657052 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7f406c038320 0x7f406c03a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:56:07.941 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:06.170+0000 7f4081600700 1 -- 172.21.15.153:0/2201657052 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f407804c3d0 con 0x7f4084106420 2024-03-21T09:56:07.941 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:06.170+0000 7f4083e00700 1 --2- 172.21.15.153:0/2201657052 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7f406c038320 0x7f406c03a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:56:07.941 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:06.171+0000 7f4083e00700 1 --2- 172.21.15.153:0/2201657052 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7f406c038320 0x7f406c03a7e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f4070006fd0 tx=0x7f4070006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:56:07.941 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:06.175+0000 7f4081600700 1 -- 172.21.15.153:0/2201657052 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f4078024440 con 0x7f4084106420 2024-03-21T09:56:07.941 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:06.387+0000 7f408b800700 1 -- 172.21.15.153:0/2201657052 --> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f408410a820 con 0x7f406c038320 2024-03-21T09:56:07.941 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:07.241+0000 7f4081600700 1 -- 172.21.15.153:0/2201657052 <== mgr.14120 v2:172.21.15.153:6800/3178125243 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f408410a820 con 0x7f406c038320 2024-03-21T09:56:07.941 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:07.244+0000 7f408b800700 1 -- 172.21.15.153:0/2201657052 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7f406c038320 msgr2=0x7f406c03a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:56:07.941 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:07.244+0000 7f408b800700 1 --2- 172.21.15.153:0/2201657052 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7f406c038320 0x7f406c03a7e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f4070006fd0 tx=0x7f4070006e40 comp rx=0 tx=0).stop 2024-03-21T09:56:07.941 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:07.244+0000 7f408b800700 1 -- 172.21.15.153:0/2201657052 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4084106420 msgr2=0x7f408419c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:56:07.942 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:07.244+0000 7f408b800700 1 --2- 172.21.15.153:0/2201657052 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4084106420 0x7f408419c2a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4078000c00 tx=0x7f4078011ab0 comp rx=0 tx=0).stop 2024-03-21T09:56:07.942 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:07.244+0000 7f408b800700 1 -- 172.21.15.153:0/2201657052 shutdown_connections 2024-03-21T09:56:07.942 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:07.244+0000 7f408b800700 1 --2- 172.21.15.153:0/2201657052 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7f406c038320 0x7f406c03a7e0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:56:07.942 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:07.244+0000 7f408b800700 1 --2- 172.21.15.153:0/2201657052 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4084106420 0x7f408419c2a0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:56:07.942 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:07.244+0000 7f408b800700 1 -- 172.21.15.153:0/2201657052 >> 172.21.15.153:0/2201657052 conn(0x7f4084101a00 msgr2=0x7f40841027b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:56:07.942 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:07.245+0000 7f408b800700 1 -- 172.21.15.153:0/2201657052 shutdown_connections 2024-03-21T09:56:07.942 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:07.245+0000 7f408b800700 1 -- 172.21.15.153:0/2201657052 wait complete. 2024-03-21T09:56:08.513 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:08 smithi153 ceph-mon[29117]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:56:08.513 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:08 smithi153 ceph-mon[29117]: Generating ssh key... 2024-03-21T09:56:08.514 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:08 smithi153 ceph-mon[29117]: from='mgr.14120 172.21.15.153:0/1521951433' entity='mgr.smithi153.tluxni' 2024-03-21T09:56:08.514 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:08 smithi153 ceph-mon[29117]: from='mgr.14120 172.21.15.153:0/1521951433' entity='mgr.smithi153.tluxni' 2024-03-21T09:56:09.408 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCvtI91Qo4EqNMgPkrfe1M6dO3/I+ER4cqFIDuaPn+ypn3Nn4aj1tvUVyoOriALOdtZObs6uBRhA7mdf6c9wcr2SP7AXdsDmGdIeA8YDnaW8kiC+rK2S5Of/6ZxhfqDt+mNlVRHjfHZ4iCAxgn4L3R9bdsoAJwQLdBm1WfjHaVgcB9zAOLcTMQl6Q9s4y1PLAavsv5XjBWoYhLjZDtMq24+vMBQfMxkQ9zvuyoR7w/5VejMLOyAFU8kmUnNwEjQq75LPz2huBabWlLaQRjtAL7bZMe0w7uthIVMZDvwtAXef0XfYgoRW7CdHO/Z63pD+eY2pHfLnO25yp1j/I6FIzub5G7+hbucaUpKdRigkBdJmEOi9YUvIUXnaeLKob+yyWDQpdQ6FrIT7t9nAzgiBiaE200m2wkD+TyQpQ9eFgEpShVbHNZEiHXailMO8sn5dZGiWB8SgV6DvxSxz4C+5Up3sCugg7h+29qDOktYjY7rVPja/UlNjnw2rOOPrHwNswU= ceph-070ed3e4-e769-11ee-95cd-87774f69a715 2024-03-21T09:56:09.408 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:08.511+0000 7f951fa00700 1 Processor -- start 2024-03-21T09:56:09.408 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:08.511+0000 7f951fa00700 1 -- start start 2024-03-21T09:56:09.408 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:08.512+0000 7f951fa00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f951807ad80 0x7f95180791e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:56:09.408 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:08.512+0000 7f951fa00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95180797b0 con 0x7f951807ad80 2024-03-21T09:56:09.408 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:08.512+0000 7f951d000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f951807ad80 0x7f95180791e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:56:09.408 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:08.512+0000 7f951d000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f951807ad80 0x7f95180791e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:49744/0 (socket says 172.21.15.153:49744) 2024-03-21T09:56:09.408 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:08.512+0000 7f951d000700 1 -- 172.21.15.153:0/1589016510 learned_addr learned my addr 172.21.15.153:0/1589016510 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T09:56:09.409 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:08.513+0000 7f951d000700 1 -- 172.21.15.153:0/1589016510 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95180798f0 con 0x7f951807ad80 2024-03-21T09:56:09.409 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:08.513+0000 7f951d000700 1 --2- 172.21.15.153:0/1589016510 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f951807ad80 0x7f95180791e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f950c009e80 tx=0x7f950c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a746c22b8fe8e80 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:56:09.409 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:08.513+0000 7f9517400700 1 -- 172.21.15.153:0/1589016510 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f950c00e070 con 0x7f951807ad80 2024-03-21T09:56:09.409 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:08.513+0000 7f9517400700 1 -- 172.21.15.153:0/1589016510 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f950c003e90 con 0x7f951807ad80 2024-03-21T09:56:09.409 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:08.513+0000 7f9517400700 1 -- 172.21.15.153:0/1589016510 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f950c004380 con 0x7f951807ad80 2024-03-21T09:56:09.409 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:08.514+0000 7f951fa00700 1 -- 172.21.15.153:0/1589016510 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f951807ad80 msgr2=0x7f95180791e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:56:09.409 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:08.514+0000 7f951fa00700 1 --2- 172.21.15.153:0/1589016510 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f951807ad80 0x7f95180791e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f950c009e80 tx=0x7f950c00b330 comp rx=0 tx=0).stop 2024-03-21T09:56:09.409 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:08.514+0000 7f951fa00700 1 -- 172.21.15.153:0/1589016510 shutdown_connections 2024-03-21T09:56:09.409 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:08.514+0000 7f951fa00700 1 --2- 172.21.15.153:0/1589016510 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f951807ad80 0x7f95180791e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:56:09.409 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:08.514+0000 7f951fa00700 1 -- 172.21.15.153:0/1589016510 >> 172.21.15.153:0/1589016510 conn(0x7f95181019b0 msgr2=0x7f9518103e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:56:09.409 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:08.514+0000 7f951fa00700 1 -- 172.21.15.153:0/1589016510 shutdown_connections 2024-03-21T09:56:09.409 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:08.514+0000 7f951fa00700 1 -- 172.21.15.153:0/1589016510 wait complete. 2024-03-21T09:56:09.409 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:08.515+0000 7f951fa00700 1 Processor -- start 2024-03-21T09:56:09.409 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:08.516+0000 7f951fa00700 1 -- start start 2024-03-21T09:56:09.410 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:08.516+0000 7f951fa00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f951807ad80 0x7f951819c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:56:09.410 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:08.516+0000 7f951fa00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f951819c7e0 con 0x7f951807ad80 2024-03-21T09:56:09.410 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:08.516+0000 7f951d000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f951807ad80 0x7f951819c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:56:09.410 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:08.516+0000 7f951d000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f951807ad80 0x7f951819c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:49752/0 (socket says 172.21.15.153:49752) 2024-03-21T09:56:09.410 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:08.516+0000 7f951d000700 1 -- 172.21.15.153:0/1599792158 learned_addr learned my addr 172.21.15.153:0/1599792158 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T09:56:09.410 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:08.516+0000 7f951d000700 1 -- 172.21.15.153:0/1599792158 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f950c009b30 con 0x7f951807ad80 2024-03-21T09:56:09.410 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:08.517+0000 7f951d000700 1 --2- 172.21.15.153:0/1599792158 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f951807ad80 0x7f951819c2a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f950c00b7b0 tx=0x7f950c004920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:56:09.410 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:08.517+0000 7f9515600700 1 -- 172.21.15.153:0/1599792158 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f950c00e040 con 0x7f951807ad80 2024-03-21T09:56:09.410 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:08.517+0000 7f9515600700 1 -- 172.21.15.153:0/1599792158 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f950c003c30 con 0x7f951807ad80 2024-03-21T09:56:09.410 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:08.517+0000 7f9515600700 1 -- 172.21.15.153:0/1599792158 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f950c01b740 con 0x7f951807ad80 2024-03-21T09:56:09.410 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:08.517+0000 7f951fa00700 1 -- 172.21.15.153:0/1599792158 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f951819c9e0 con 0x7f951807ad80 2024-03-21T09:56:09.410 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:08.517+0000 7f951fa00700 1 -- 172.21.15.153:0/1599792158 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f951819cee0 con 0x7f951807ad80 2024-03-21T09:56:09.410 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:08.518+0000 7f9515600700 1 -- 172.21.15.153:0/1599792158 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f950c019040 con 0x7f951807ad80 2024-03-21T09:56:09.411 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:08.518+0000 7f951fa00700 1 -- 172.21.15.153:0/1599792158 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f951810c7b0 con 0x7f951807ad80 2024-03-21T09:56:09.411 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:08.519+0000 7f9515600700 1 --2- 172.21.15.153:0/1599792158 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7f9500038370 0x7f950003a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:56:09.411 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:08.519+0000 7f9515600700 1 -- 172.21.15.153:0/1599792158 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f950c04bc90 con 0x7f951807ad80 2024-03-21T09:56:09.411 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:08.519+0000 7f9517e00700 1 --2- 172.21.15.153:0/1599792158 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7f9500038370 0x7f950003a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:56:09.411 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:08.520+0000 7f9517e00700 1 --2- 172.21.15.153:0/1599792158 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7f9500038370 0x7f950003a830 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9504006fd0 tx=0x7f9504006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:56:09.411 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:08.524+0000 7f9515600700 1 -- 172.21.15.153:0/1599792158 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f950c023be0 con 0x7f951807ad80 2024-03-21T09:56:09.411 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:08.738+0000 7f951fa00700 1 -- 172.21.15.153:0/1599792158 --> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f9518103ca0 con 0x7f9500038370 2024-03-21T09:56:09.411 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:08.739+0000 7f9515600700 1 -- 172.21.15.153:0/1599792158 <== mgr.14120 v2:172.21.15.153:6800/3178125243 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7f9518103ca0 con 0x7f9500038370 2024-03-21T09:56:09.411 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:08.741+0000 7f951fa00700 1 -- 172.21.15.153:0/1599792158 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7f9500038370 msgr2=0x7f950003a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:56:09.412 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:08.741+0000 7f951fa00700 1 --2- 172.21.15.153:0/1599792158 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7f9500038370 0x7f950003a830 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9504006fd0 tx=0x7f9504006e40 comp rx=0 tx=0).stop 2024-03-21T09:56:09.412 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:08.741+0000 7f951fa00700 1 -- 172.21.15.153:0/1599792158 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f951807ad80 msgr2=0x7f951819c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:56:09.412 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:08.741+0000 7f951fa00700 1 --2- 172.21.15.153:0/1599792158 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f951807ad80 0x7f951819c2a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f950c00b7b0 tx=0x7f950c004920 comp rx=0 tx=0).stop 2024-03-21T09:56:09.412 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:08.741+0000 7f951fa00700 1 -- 172.21.15.153:0/1599792158 shutdown_connections 2024-03-21T09:56:09.412 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:08.741+0000 7f951fa00700 1 --2- 172.21.15.153:0/1599792158 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7f9500038370 0x7f950003a830 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:56:09.412 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:08.742+0000 7f951fa00700 1 --2- 172.21.15.153:0/1599792158 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f951807ad80 0x7f951819c2a0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:56:09.412 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:08.742+0000 7f951fa00700 1 -- 172.21.15.153:0/1599792158 >> 172.21.15.153:0/1599792158 conn(0x7f95181019b0 msgr2=0x7f9518103080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:56:09.412 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:08.742+0000 7f951fa00700 1 -- 172.21.15.153:0/1599792158 shutdown_connections 2024-03-21T09:56:09.412 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:08.742+0000 7f951fa00700 1 -- 172.21.15.153:0/1599792158 wait complete. 2024-03-21T09:56:09.413 INFO:teuthology.orchestra.run.smithi153.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-03-21T09:56:09.413 INFO:teuthology.orchestra.run.smithi153.stdout:Adding key to root@localhost authorized_keys... 2024-03-21T09:56:09.413 INFO:teuthology.orchestra.run.smithi153.stdout:Adding host smithi153... 2024-03-21T09:56:10.015 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:09 smithi153 ceph-mon[29117]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:56:10.947 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:10 smithi153 ceph-mon[29117]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi153", "addr": "172.21.15.153", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:56:12.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:12 smithi153 ceph-mon[29117]: Deploying cephadm binary to smithi153 2024-03-21T09:56:15.535 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout Added host 'smithi153' with addr '172.21.15.153' 2024-03-21T09:56:15.536 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:10.013+0000 7fec15800700 1 Processor -- start 2024-03-21T09:56:15.536 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:10.013+0000 7fec15800700 1 -- start start 2024-03-21T09:56:15.536 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:10.014+0000 7fec15800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec1010e030 0x7fec1010e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:56:15.536 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:10.014+0000 7fec15800700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec1010ea20 con 0x7fec1010e030 2024-03-21T09:56:15.536 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:10.014+0000 7fec0ea00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec1010e030 0x7fec1010e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:56:15.536 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:10.014+0000 7fec0ea00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec1010e030 0x7fec1010e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:49766/0 (socket says 172.21.15.153:49766) 2024-03-21T09:56:15.536 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:10.014+0000 7fec0ea00700 1 -- 172.21.15.153:0/2346607574 learned_addr learned my addr 172.21.15.153:0/2346607574 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T09:56:15.536 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:10.015+0000 7fec0ea00700 1 -- 172.21.15.153:0/2346607574 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec1010eb60 con 0x7fec1010e030 2024-03-21T09:56:15.536 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:10.015+0000 7fec0ea00700 1 --2- 172.21.15.153:0/2346607574 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec1010e030 0x7fec1010e450 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fec000098a0 tx=0x7fec00009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b25c7cca629a762e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:56:15.536 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:10.016+0000 7fec0d600700 1 -- 172.21.15.153:0/2346607574 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fec00004180 con 0x7fec1010e030 2024-03-21T09:56:15.536 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:10.016+0000 7fec0d600700 1 -- 172.21.15.153:0/2346607574 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fec000042e0 con 0x7fec1010e030 2024-03-21T09:56:15.536 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:10.016+0000 7fec0d600700 1 -- 172.21.15.153:0/2346607574 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fec00011a50 con 0x7fec1010e030 2024-03-21T09:56:15.536 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:10.016+0000 7fec15800700 1 -- 172.21.15.153:0/2346607574 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec1010e030 msgr2=0x7fec1010e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:56:15.537 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:10.017+0000 7fec15800700 1 --2- 172.21.15.153:0/2346607574 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec1010e030 0x7fec1010e450 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fec000098a0 tx=0x7fec00009bb0 comp rx=0 tx=0).stop 2024-03-21T09:56:15.537 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:10.017+0000 7fec15800700 1 -- 172.21.15.153:0/2346607574 shutdown_connections 2024-03-21T09:56:15.537 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:10.017+0000 7fec15800700 1 --2- 172.21.15.153:0/2346607574 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec1010e030 0x7fec1010e450 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:56:15.537 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:10.017+0000 7fec15800700 1 -- 172.21.15.153:0/2346607574 >> 172.21.15.153:0/2346607574 conn(0x7fec1007bb20 msgr2=0x7fec1007df80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:56:15.537 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:10.017+0000 7fec15800700 1 -- 172.21.15.153:0/2346607574 shutdown_connections 2024-03-21T09:56:15.537 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:10.017+0000 7fec15800700 1 -- 172.21.15.153:0/2346607574 wait complete. 2024-03-21T09:56:15.537 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:10.018+0000 7fec15800700 1 Processor -- start 2024-03-21T09:56:15.537 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:10.018+0000 7fec15800700 1 -- start start 2024-03-21T09:56:15.537 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:10.018+0000 7fec15800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec1010e030 0x7fec101a0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:56:15.537 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:10.018+0000 7fec15800700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec101a0cb0 con 0x7fec1010e030 2024-03-21T09:56:15.537 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:10.018+0000 7fec0ea00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec1010e030 0x7fec101a0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:56:15.537 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:10.019+0000 7fec0ea00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec1010e030 0x7fec101a0770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:49782/0 (socket says 172.21.15.153:49782) 2024-03-21T09:56:15.537 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:10.019+0000 7fec0ea00700 1 -- 172.21.15.153:0/3615278941 learned_addr learned my addr 172.21.15.153:0/3615278941 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T09:56:15.538 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:10.019+0000 7fec0ea00700 1 -- 172.21.15.153:0/3615278941 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec00009550 con 0x7fec1010e030 2024-03-21T09:56:15.538 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:10.019+0000 7fec0ea00700 1 --2- 172.21.15.153:0/3615278941 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec1010e030 0x7fec101a0770 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fec00003de0 tx=0x7fec00011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:56:15.538 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:10.019+0000 7fec07400700 1 -- 172.21.15.153:0/3615278941 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fec00003930 con 0x7fec1010e030 2024-03-21T09:56:15.538 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:10.019+0000 7fec07400700 1 -- 172.21.15.153:0/3615278941 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fec00003a90 con 0x7fec1010e030 2024-03-21T09:56:15.538 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:10.019+0000 7fec07400700 1 -- 172.21.15.153:0/3615278941 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fec000196c0 con 0x7fec1010e030 2024-03-21T09:56:15.538 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:10.019+0000 7fec15800700 1 -- 172.21.15.153:0/3615278941 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec101a0eb0 con 0x7fec1010e030 2024-03-21T09:56:15.538 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:10.020+0000 7fec15800700 1 -- 172.21.15.153:0/3615278941 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec101a1350 con 0x7fec1010e030 2024-03-21T09:56:15.538 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:10.021+0000 7fec07400700 1 -- 172.21.15.153:0/3615278941 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7fec0001abc0 con 0x7fec1010e030 2024-03-21T09:56:15.538 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:10.021+0000 7fec15800700 1 -- 172.21.15.153:0/3615278941 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec1019a840 con 0x7fec1010e030 2024-03-21T09:56:15.538 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:10.021+0000 7fec07400700 1 --2- 172.21.15.153:0/3615278941 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7febf4038370 0x7febf403a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:56:15.538 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:10.021+0000 7fec07400700 1 -- 172.21.15.153:0/3615278941 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fec0004c2a0 con 0x7fec1010e030 2024-03-21T09:56:15.538 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:10.021+0000 7fec0e000700 1 --2- 172.21.15.153:0/3615278941 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7febf4038370 0x7febf403a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:56:15.538 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:10.022+0000 7fec0e000700 1 --2- 172.21.15.153:0/3615278941 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7febf4038370 0x7febf403a830 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7febfc006fd0 tx=0x7febfc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:56:15.539 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:10.027+0000 7fec07400700 1 -- 172.21.15.153:0/3615278941 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fec000189a0 con 0x7fec1010e030 2024-03-21T09:56:15.539 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:10.244+0000 7fec15800700 1 -- 172.21.15.153:0/3615278941 --> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi153", "addr": "172.21.15.153", "target": ["mon-mgr", ""]}) v1 -- 0x7fec1010eb60 con 0x7febf4038370 2024-03-21T09:56:15.539 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:14.780+0000 7fec07400700 1 -- 172.21.15.153:0/3615278941 <== mgr.14120 v2:172.21.15.153:6800/3178125243 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fec1010eb60 con 0x7febf4038370 2024-03-21T09:56:15.539 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:14.785+0000 7fec15800700 1 -- 172.21.15.153:0/3615278941 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7febf4038370 msgr2=0x7febf403a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:56:15.539 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:14.785+0000 7fec15800700 1 --2- 172.21.15.153:0/3615278941 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7febf4038370 0x7febf403a830 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7febfc006fd0 tx=0x7febfc006e40 comp rx=0 tx=0).stop 2024-03-21T09:56:15.539 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:14.785+0000 7fec15800700 1 -- 172.21.15.153:0/3615278941 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec1010e030 msgr2=0x7fec101a0770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:56:15.539 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:14.785+0000 7fec15800700 1 --2- 172.21.15.153:0/3615278941 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec1010e030 0x7fec101a0770 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fec00003de0 tx=0x7fec00011ab0 comp rx=0 tx=0).stop 2024-03-21T09:56:15.539 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:14.785+0000 7fec15800700 1 -- 172.21.15.153:0/3615278941 shutdown_connections 2024-03-21T09:56:15.539 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:14.785+0000 7fec15800700 1 --2- 172.21.15.153:0/3615278941 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7febf4038370 0x7febf403a830 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:56:15.539 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:14.785+0000 7fec15800700 1 --2- 172.21.15.153:0/3615278941 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec1010e030 0x7fec101a0770 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:56:15.539 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:14.785+0000 7fec15800700 1 -- 172.21.15.153:0/3615278941 >> 172.21.15.153:0/3615278941 conn(0x7fec1007bb20 msgr2=0x7fec1007e360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:56:15.539 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:14.785+0000 7fec15800700 1 -- 172.21.15.153:0/3615278941 shutdown_connections 2024-03-21T09:56:15.539 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:14.785+0000 7fec15800700 1 -- 172.21.15.153:0/3615278941 wait complete. 2024-03-21T09:56:15.540 INFO:teuthology.orchestra.run.smithi153.stdout:Deploying mon service with default placement... 2024-03-21T09:56:16.028 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:15 smithi153 ceph-mon[29117]: from='mgr.14120 172.21.15.153:0/1521951433' entity='mgr.smithi153.tluxni' 2024-03-21T09:56:16.029 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:15 smithi153 ceph-mon[29117]: Added host smithi153 2024-03-21T09:56:16.029 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:15 smithi153 ceph-mon[29117]: from='mgr.14120 172.21.15.153:0/1521951433' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:56:17.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:17 smithi153 ceph-mon[29117]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:56:17.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:17 smithi153 ceph-mon[29117]: Saving service mon spec with placement count:5 2024-03-21T09:56:17.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:17 smithi153 ceph-mon[29117]: from='mgr.14120 172.21.15.153:0/1521951433' entity='mgr.smithi153.tluxni' 2024-03-21T09:56:18.087 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-03-21T09:56:18.088 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:16.175+0000 7f2848200700 1 Processor -- start 2024-03-21T09:56:18.088 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:16.175+0000 7f2848200700 1 -- start start 2024-03-21T09:56:18.088 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:16.175+0000 7f2848200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f284007adc0 0x7f2840079220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:56:18.088 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:16.176+0000 7f2848200700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28400797f0 con 0x7f284007adc0 2024-03-21T09:56:18.088 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:16.176+0000 7f2845800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f284007adc0 0x7f2840079220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:56:18.088 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:16.176+0000 7f2845800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f284007adc0 0x7f2840079220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:46434/0 (socket says 172.21.15.153:46434) 2024-03-21T09:56:18.088 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:16.176+0000 7f2845800700 1 -- 172.21.15.153:0/2531495612 learned_addr learned my addr 172.21.15.153:0/2531495612 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T09:56:18.088 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:16.177+0000 7f2845800700 1 -- 172.21.15.153:0/2531495612 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2840079930 con 0x7f284007adc0 2024-03-21T09:56:18.088 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:16.177+0000 7f2845800700 1 --2- 172.21.15.153:0/2531495612 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f284007adc0 0x7f2840079220 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f28340098a0 tx=0x7f2834009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aae5ea2a0d4beaa1 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:56:18.088 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:16.177+0000 7f283fe00700 1 -- 172.21.15.153:0/2531495612 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2834004180 con 0x7f284007adc0 2024-03-21T09:56:18.093 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:16.177+0000 7f283fe00700 1 -- 172.21.15.153:0/2531495612 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f28340042e0 con 0x7f284007adc0 2024-03-21T09:56:18.093 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:16.177+0000 7f283fe00700 1 -- 172.21.15.153:0/2531495612 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2834011a50 con 0x7f284007adc0 2024-03-21T09:56:18.093 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:16.178+0000 7f2848200700 1 -- 172.21.15.153:0/2531495612 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f284007adc0 msgr2=0x7f2840079220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:56:18.093 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:16.178+0000 7f2848200700 1 --2- 172.21.15.153:0/2531495612 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f284007adc0 0x7f2840079220 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f28340098a0 tx=0x7f2834009bb0 comp rx=0 tx=0).stop 2024-03-21T09:56:18.093 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:16.178+0000 7f2848200700 1 -- 172.21.15.153:0/2531495612 shutdown_connections 2024-03-21T09:56:18.093 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:16.178+0000 7f2848200700 1 --2- 172.21.15.153:0/2531495612 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f284007adc0 0x7f2840079220 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:56:18.093 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:16.178+0000 7f2848200700 1 -- 172.21.15.153:0/2531495612 >> 172.21.15.153:0/2531495612 conn(0x7f28401019b0 msgr2=0x7f2840103e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:56:18.093 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:16.178+0000 7f2848200700 1 -- 172.21.15.153:0/2531495612 shutdown_connections 2024-03-21T09:56:18.093 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:16.178+0000 7f2848200700 1 -- 172.21.15.153:0/2531495612 wait complete. 2024-03-21T09:56:18.093 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:16.179+0000 7f2848200700 1 Processor -- start 2024-03-21T09:56:18.094 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:16.179+0000 7f2848200700 1 -- start start 2024-03-21T09:56:18.094 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:16.179+0000 7f2848200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f284007adc0 0x7f284019c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:56:18.094 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:16.179+0000 7f2848200700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f284019c850 con 0x7f284007adc0 2024-03-21T09:56:18.094 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:16.180+0000 7f2845800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f284007adc0 0x7f284019c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:56:18.094 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:16.180+0000 7f2845800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f284007adc0 0x7f284019c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:46440/0 (socket says 172.21.15.153:46440) 2024-03-21T09:56:18.094 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:16.180+0000 7f2845800700 1 -- 172.21.15.153:0/1064572610 learned_addr learned my addr 172.21.15.153:0/1064572610 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T09:56:18.094 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:16.180+0000 7f2845800700 1 -- 172.21.15.153:0/1064572610 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2834009550 con 0x7f284007adc0 2024-03-21T09:56:18.094 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:16.180+0000 7f2845800700 1 --2- 172.21.15.153:0/1064572610 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f284007adc0 0x7f284019c310 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2834003de0 tx=0x7f2834011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:56:18.094 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:16.180+0000 7f283e000700 1 -- 172.21.15.153:0/1064572610 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2834003930 con 0x7f284007adc0 2024-03-21T09:56:18.094 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:16.181+0000 7f283e000700 1 -- 172.21.15.153:0/1064572610 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f2834003a90 con 0x7f284007adc0 2024-03-21T09:56:18.094 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:16.181+0000 7f283e000700 1 -- 172.21.15.153:0/1064572610 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f28340196c0 con 0x7f284007adc0 2024-03-21T09:56:18.094 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:16.181+0000 7f2848200700 1 -- 172.21.15.153:0/1064572610 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f284019ca50 con 0x7f284007adc0 2024-03-21T09:56:18.095 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:16.181+0000 7f2848200700 1 -- 172.21.15.153:0/1064572610 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f284019cef0 con 0x7f284007adc0 2024-03-21T09:56:18.095 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:16.182+0000 7f283e000700 1 -- 172.21.15.153:0/1064572610 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f283401abc0 con 0x7f284007adc0 2024-03-21T09:56:18.095 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:16.182+0000 7f2848200700 1 -- 172.21.15.153:0/1064572610 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f284010c770 con 0x7f284007adc0 2024-03-21T09:56:18.095 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:16.182+0000 7f283e000700 1 --2- 172.21.15.153:0/1064572610 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7f2828038370 0x7f282803a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:56:18.095 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:16.182+0000 7f283e000700 1 -- 172.21.15.153:0/1064572610 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f283404c4a0 con 0x7f284007adc0 2024-03-21T09:56:18.095 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:16.183+0000 7f2844e00700 1 --2- 172.21.15.153:0/1064572610 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7f2828038370 0x7f282803a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:56:18.095 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:16.183+0000 7f2844e00700 1 --2- 172.21.15.153:0/1064572610 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7f2828038370 0x7f282803a830 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f282c006fd0 tx=0x7f282c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:56:18.095 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:16.187+0000 7f283e000700 1 -- 172.21.15.153:0/1064572610 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f2834010770 con 0x7f284007adc0 2024-03-21T09:56:18.095 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:16.418+0000 7f2848200700 1 -- 172.21.15.153:0/1064572610 --> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}) v1 -- 0x7f2840103cc0 con 0x7f2828038370 2024-03-21T09:56:18.095 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:16.424+0000 7f283e000700 1 -- 172.21.15.153:0/1064572610 <== mgr.14120 v2:172.21.15.153:6800/3178125243 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f2840103cc0 con 0x7f2828038370 2024-03-21T09:56:18.095 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:16.427+0000 7f2848200700 1 -- 172.21.15.153:0/1064572610 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7f2828038370 msgr2=0x7f282803a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:56:18.095 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:16.427+0000 7f2848200700 1 --2- 172.21.15.153:0/1064572610 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7f2828038370 0x7f282803a830 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f282c006fd0 tx=0x7f282c006e40 comp rx=0 tx=0).stop 2024-03-21T09:56:18.095 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:16.427+0000 7f2848200700 1 -- 172.21.15.153:0/1064572610 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f284007adc0 msgr2=0x7f284019c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:56:18.096 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:16.427+0000 7f2848200700 1 --2- 172.21.15.153:0/1064572610 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f284007adc0 0x7f284019c310 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2834003de0 tx=0x7f2834011ab0 comp rx=0 tx=0).stop 2024-03-21T09:56:18.096 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:16.427+0000 7f2848200700 1 -- 172.21.15.153:0/1064572610 shutdown_connections 2024-03-21T09:56:18.096 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:16.427+0000 7f2848200700 1 --2- 172.21.15.153:0/1064572610 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7f2828038370 0x7f282803a830 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:56:18.096 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:16.427+0000 7f2848200700 1 --2- 172.21.15.153:0/1064572610 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f284007adc0 0x7f284019c310 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:56:18.096 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:16.427+0000 7f2848200700 1 -- 172.21.15.153:0/1064572610 >> 172.21.15.153:0/1064572610 conn(0x7f28401019b0 msgr2=0x7f28401030a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:56:18.096 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:16.427+0000 7f2848200700 1 -- 172.21.15.153:0/1064572610 shutdown_connections 2024-03-21T09:56:18.096 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:16.427+0000 7f2848200700 1 -- 172.21.15.153:0/1064572610 wait complete. 2024-03-21T09:56:18.096 INFO:teuthology.orchestra.run.smithi153.stdout:Deploying mgr service with default placement... 2024-03-21T09:56:18.938 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:18 smithi153 ceph-mon[29117]: from='mgr.14120 172.21.15.153:0/1521951433' entity='mgr.smithi153.tluxni' 2024-03-21T09:56:18.938 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:18 smithi153 ceph-mon[29117]: from='mgr.14120 172.21.15.153:0/1521951433' entity='mgr.smithi153.tluxni' 2024-03-21T09:56:19.623 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-03-21T09:56:19.623 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:18.676+0000 7f45aa200700 1 Processor -- start 2024-03-21T09:56:19.623 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:18.677+0000 7f45aa200700 1 -- start start 2024-03-21T09:56:19.623 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:18.677+0000 7f45aa200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f45a4106420 0x7f45a4106840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:56:19.623 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:18.677+0000 7f45aa200700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45a4106e10 con 0x7f45a4106420 2024-03-21T09:56:19.624 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:18.677+0000 7f45a3400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f45a4106420 0x7f45a4106840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:56:19.624 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:18.678+0000 7f45a3400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f45a4106420 0x7f45a4106840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:46456/0 (socket says 172.21.15.153:46456) 2024-03-21T09:56:19.624 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:18.678+0000 7f45a3400700 1 -- 172.21.15.153:0/2344885662 learned_addr learned my addr 172.21.15.153:0/2344885662 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T09:56:19.624 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:18.678+0000 7f45a3400700 1 -- 172.21.15.153:0/2344885662 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45a4107620 con 0x7f45a4106420 2024-03-21T09:56:19.624 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:18.678+0000 7f45a3400700 1 --2- 172.21.15.153:0/2344885662 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f45a4106420 0x7f45a4106840 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f45940098a0 tx=0x7f4594009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c76c325a830e3ab5 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:56:19.627 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:18.679+0000 7f45a2000700 1 -- 172.21.15.153:0/2344885662 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4594004180 con 0x7f45a4106420 2024-03-21T09:56:19.627 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:18.679+0000 7f45a2000700 1 -- 172.21.15.153:0/2344885662 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f45940042e0 con 0x7f45a4106420 2024-03-21T09:56:19.627 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:18.679+0000 7f45a2000700 1 -- 172.21.15.153:0/2344885662 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4594011af0 con 0x7f45a4106420 2024-03-21T09:56:19.627 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:18.679+0000 7f45aa200700 1 -- 172.21.15.153:0/2344885662 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f45a4106420 msgr2=0x7f45a4106840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:56:19.627 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:18.679+0000 7f45aa200700 1 --2- 172.21.15.153:0/2344885662 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f45a4106420 0x7f45a4106840 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f45940098a0 tx=0x7f4594009bb0 comp rx=0 tx=0).stop 2024-03-21T09:56:19.627 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:18.680+0000 7f45aa200700 1 -- 172.21.15.153:0/2344885662 shutdown_connections 2024-03-21T09:56:19.627 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:18.680+0000 7f45aa200700 1 --2- 172.21.15.153:0/2344885662 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f45a4106420 0x7f45a4106840 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:56:19.627 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:18.680+0000 7f45aa200700 1 -- 172.21.15.153:0/2344885662 >> 172.21.15.153:0/2344885662 conn(0x7f45a4101a00 msgr2=0x7f45a4103e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:56:19.627 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:18.680+0000 7f45aa200700 1 -- 172.21.15.153:0/2344885662 shutdown_connections 2024-03-21T09:56:19.627 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:18.680+0000 7f45aa200700 1 -- 172.21.15.153:0/2344885662 wait complete. 2024-03-21T09:56:19.627 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:18.681+0000 7f45aa200700 1 Processor -- start 2024-03-21T09:56:19.628 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:18.681+0000 7f45aa200700 1 -- start start 2024-03-21T09:56:19.628 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:18.681+0000 7f45aa200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f45a4106420 0x7f45a419c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:56:19.628 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:18.681+0000 7f45aa200700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45a419c860 con 0x7f45a4106420 2024-03-21T09:56:19.628 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:18.681+0000 7f45a3400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f45a4106420 0x7f45a419c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:56:19.628 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:18.681+0000 7f45a3400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f45a4106420 0x7f45a419c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:46462/0 (socket says 172.21.15.153:46462) 2024-03-21T09:56:19.628 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:18.681+0000 7f45a3400700 1 -- 172.21.15.153:0/3939269358 learned_addr learned my addr 172.21.15.153:0/3939269358 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T09:56:19.628 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:18.682+0000 7f45a3400700 1 -- 172.21.15.153:0/3939269358 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4594009550 con 0x7f45a4106420 2024-03-21T09:56:19.628 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:18.682+0000 7f45a3400700 1 --2- 172.21.15.153:0/3939269358 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f45a4106420 0x7f45a419c320 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4594000c00 tx=0x7f4594011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:56:19.628 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:18.682+0000 7f459be00700 1 -- 172.21.15.153:0/3939269358 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4594003a70 con 0x7f45a4106420 2024-03-21T09:56:19.628 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:18.682+0000 7f459be00700 1 -- 172.21.15.153:0/3939269358 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f4594003bd0 con 0x7f45a4106420 2024-03-21T09:56:19.628 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:18.682+0000 7f45aa200700 1 -- 172.21.15.153:0/3939269358 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45a419ca60 con 0x7f45a4106420 2024-03-21T09:56:19.628 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:18.682+0000 7f459be00700 1 -- 172.21.15.153:0/3939269358 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f45940238c0 con 0x7f45a4106420 2024-03-21T09:56:19.628 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:18.682+0000 7f45aa200700 1 -- 172.21.15.153:0/3939269358 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45a419cf00 con 0x7f45a4106420 2024-03-21T09:56:19.631 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:18.683+0000 7f45aa200700 1 -- 172.21.15.153:0/3939269358 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45a40623c0 con 0x7f45a4106420 2024-03-21T09:56:19.631 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:18.684+0000 7f459be00700 1 -- 172.21.15.153:0/3939269358 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f4594023b30 con 0x7f45a4106420 2024-03-21T09:56:19.631 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:18.684+0000 7f459be00700 1 --2- 172.21.15.153:0/3939269358 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7f4588038370 0x7f458803a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:56:19.631 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:18.684+0000 7f459be00700 1 -- 172.21.15.153:0/3939269358 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f459404c240 con 0x7f45a4106420 2024-03-21T09:56:19.631 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:18.685+0000 7f45a2a00700 1 --2- 172.21.15.153:0/3939269358 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7f4588038370 0x7f458803a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:56:19.631 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:18.685+0000 7f45a2a00700 1 --2- 172.21.15.153:0/3939269358 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7f4588038370 0x7f458803a830 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f4590006fd0 tx=0x7f4590006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:56:19.631 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:18.689+0000 7f459be00700 1 -- 172.21.15.153:0/3939269358 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f4594010380 con 0x7f45a4106420 2024-03-21T09:56:19.632 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:18.909+0000 7f45aa200700 1 -- 172.21.15.153:0/3939269358 --> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}) v1 -- 0x7f45a410a820 con 0x7f4588038370 2024-03-21T09:56:19.632 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:18.917+0000 7f459be00700 1 -- 172.21.15.153:0/3939269358 <== mgr.14120 v2:172.21.15.153:6800/3178125243 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f45a410a820 con 0x7f4588038370 2024-03-21T09:56:19.632 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:18.920+0000 7f45aa200700 1 -- 172.21.15.153:0/3939269358 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7f4588038370 msgr2=0x7f458803a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:56:19.632 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:18.920+0000 7f45aa200700 1 --2- 172.21.15.153:0/3939269358 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7f4588038370 0x7f458803a830 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f4590006fd0 tx=0x7f4590006e40 comp rx=0 tx=0).stop 2024-03-21T09:56:19.632 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:18.920+0000 7f45aa200700 1 -- 172.21.15.153:0/3939269358 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f45a4106420 msgr2=0x7f45a419c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:56:19.632 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:18.920+0000 7f45aa200700 1 --2- 172.21.15.153:0/3939269358 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f45a4106420 0x7f45a419c320 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4594000c00 tx=0x7f4594011b50 comp rx=0 tx=0).stop 2024-03-21T09:56:19.632 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:18.920+0000 7f45aa200700 1 -- 172.21.15.153:0/3939269358 shutdown_connections 2024-03-21T09:56:19.632 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:18.920+0000 7f45aa200700 1 --2- 172.21.15.153:0/3939269358 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7f4588038370 0x7f458803a830 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:56:19.632 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:18.920+0000 7f45aa200700 1 --2- 172.21.15.153:0/3939269358 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f45a4106420 0x7f45a419c320 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:56:19.632 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:18.920+0000 7f45aa200700 1 -- 172.21.15.153:0/3939269358 >> 172.21.15.153:0/3939269358 conn(0x7f45a4101a00 msgr2=0x7f45a41030a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:56:19.632 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:18.920+0000 7f45aa200700 1 -- 172.21.15.153:0/3939269358 shutdown_connections 2024-03-21T09:56:19.632 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:18.920+0000 7f45aa200700 1 -- 172.21.15.153:0/3939269358 wait complete. 2024-03-21T09:56:19.633 INFO:teuthology.orchestra.run.smithi153.stdout:Deploying crash service with default placement... 2024-03-21T09:56:20.211 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:19 smithi153 ceph-mon[29117]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:56:20.211 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:19 smithi153 ceph-mon[29117]: Saving service mgr spec with placement count:2 2024-03-21T09:56:20.212 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:19 smithi153 ceph-mon[29117]: from='mgr.14120 172.21.15.153:0/1521951433' entity='mgr.smithi153.tluxni' 2024-03-21T09:56:21.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:21 smithi153 ceph-mon[29117]: from='client.14146 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:56:21.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:21 smithi153 ceph-mon[29117]: Saving service crash spec with placement * 2024-03-21T09:56:21.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:21 smithi153 ceph-mon[29117]: from='mgr.14120 172.21.15.153:0/1521951433' entity='mgr.smithi153.tluxni' 2024-03-21T09:56:21.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:21 smithi153 ceph-mon[29117]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:56:21.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:21 smithi153 ceph-mon[29117]: from='mgr.14120 172.21.15.153:0/1521951433' entity='mgr.smithi153.tluxni' 2024-03-21T09:56:21.602 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout Scheduled crash update... 2024-03-21T09:56:21.602 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:20.209+0000 7f8392600700 1 Processor -- start 2024-03-21T09:56:21.602 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:20.209+0000 7f8392600700 1 -- start start 2024-03-21T09:56:21.602 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:20.210+0000 7f8392600700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f838c108630 0x7f838c108a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:56:21.602 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:20.210+0000 7f8392600700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f838c109020 con 0x7f838c108630 2024-03-21T09:56:21.602 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:20.210+0000 7f838b400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f838c108630 0x7f838c108a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:56:21.602 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:20.211+0000 7f838b400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f838c108630 0x7f838c108a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:46474/0 (socket says 172.21.15.153:46474) 2024-03-21T09:56:21.602 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:20.211+0000 7f838b400700 1 -- 172.21.15.153:0/1475887646 learned_addr learned my addr 172.21.15.153:0/1475887646 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T09:56:21.602 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:20.211+0000 7f838b400700 1 -- 172.21.15.153:0/1475887646 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f838c109830 con 0x7f838c108630 2024-03-21T09:56:21.602 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:20.211+0000 7f838b400700 1 --2- 172.21.15.153:0/1475887646 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f838c108630 0x7f838c108a50 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f837c0098a0 tx=0x7f837c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f9327300d9797f9 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:56:21.603 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:20.212+0000 7f838a000700 1 -- 172.21.15.153:0/1475887646 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f837c004180 con 0x7f838c108630 2024-03-21T09:56:21.603 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:20.212+0000 7f838a000700 1 -- 172.21.15.153:0/1475887646 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f837c0042e0 con 0x7f838c108630 2024-03-21T09:56:21.603 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:20.212+0000 7f838a000700 1 -- 172.21.15.153:0/1475887646 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f837c011af0 con 0x7f838c108630 2024-03-21T09:56:21.603 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:20.212+0000 7f8392600700 1 -- 172.21.15.153:0/1475887646 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f838c108630 msgr2=0x7f838c108a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:56:21.603 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:20.212+0000 7f8392600700 1 --2- 172.21.15.153:0/1475887646 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f838c108630 0x7f838c108a50 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f837c0098a0 tx=0x7f837c009bb0 comp rx=0 tx=0).stop 2024-03-21T09:56:21.603 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:20.213+0000 7f8392600700 1 -- 172.21.15.153:0/1475887646 shutdown_connections 2024-03-21T09:56:21.603 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:20.213+0000 7f8392600700 1 --2- 172.21.15.153:0/1475887646 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f838c108630 0x7f838c108a50 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:56:21.603 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:20.213+0000 7f8392600700 1 -- 172.21.15.153:0/1475887646 >> 172.21.15.153:0/1475887646 conn(0x7f838c103c10 msgr2=0x7f838c106050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:56:21.603 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:20.213+0000 7f8392600700 1 -- 172.21.15.153:0/1475887646 shutdown_connections 2024-03-21T09:56:21.603 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:20.213+0000 7f8392600700 1 -- 172.21.15.153:0/1475887646 wait complete. 2024-03-21T09:56:21.603 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:20.214+0000 7f8392600700 1 Processor -- start 2024-03-21T09:56:21.603 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:20.214+0000 7f8392600700 1 -- start start 2024-03-21T09:56:21.603 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:20.214+0000 7f8392600700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f838c108630 0x7f838c19c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:56:21.604 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:20.214+0000 7f8392600700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f838c19c900 con 0x7f838c108630 2024-03-21T09:56:21.604 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:20.215+0000 7f838b400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f838c108630 0x7f838c19c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:56:21.604 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:20.215+0000 7f838b400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f838c108630 0x7f838c19c3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:46478/0 (socket says 172.21.15.153:46478) 2024-03-21T09:56:21.604 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:20.215+0000 7f838b400700 1 -- 172.21.15.153:0/4274054863 learned_addr learned my addr 172.21.15.153:0/4274054863 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T09:56:21.604 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:20.215+0000 7f838b400700 1 -- 172.21.15.153:0/4274054863 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f837c009550 con 0x7f838c108630 2024-03-21T09:56:21.604 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:20.215+0000 7f838b400700 1 --2- 172.21.15.153:0/4274054863 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f838c108630 0x7f838c19c3c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f837c011670 tx=0x7f837c011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:56:21.604 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:20.216+0000 7f8383e00700 1 -- 172.21.15.153:0/4274054863 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f837c0039f0 con 0x7f838c108630 2024-03-21T09:56:21.604 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:20.216+0000 7f8383e00700 1 -- 172.21.15.153:0/4274054863 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f837c003b50 con 0x7f838c108630 2024-03-21T09:56:21.604 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:20.216+0000 7f8383e00700 1 -- 172.21.15.153:0/4274054863 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f837c0247c0 con 0x7f838c108630 2024-03-21T09:56:21.604 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:20.216+0000 7f8392600700 1 -- 172.21.15.153:0/4274054863 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f838c19cb00 con 0x7f838c108630 2024-03-21T09:56:21.604 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:20.216+0000 7f8392600700 1 -- 172.21.15.153:0/4274054863 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f838c19cfa0 con 0x7f838c108630 2024-03-21T09:56:21.604 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:20.217+0000 7f8383e00700 1 -- 172.21.15.153:0/4274054863 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f837c01a430 con 0x7f838c108630 2024-03-21T09:56:21.604 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:20.217+0000 7f8383e00700 1 --2- 172.21.15.153:0/4274054863 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7f8370038370 0x7f837003a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:56:21.605 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:20.217+0000 7f8383e00700 1 -- 172.21.15.153:0/4274054863 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f837c04c370 con 0x7f838c108630 2024-03-21T09:56:21.605 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:20.217+0000 7f8392600700 1 -- 172.21.15.153:0/4274054863 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f838c0623c0 con 0x7f838c108630 2024-03-21T09:56:21.605 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:20.218+0000 7f838aa00700 1 --2- 172.21.15.153:0/4274054863 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7f8370038370 0x7f837003a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:56:21.605 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:20.219+0000 7f838aa00700 1 --2- 172.21.15.153:0/4274054863 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7f8370038370 0x7f837003a830 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f8378006fd0 tx=0x7f8378006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:56:21.605 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:20.224+0000 7f8383e00700 1 -- 172.21.15.153:0/4274054863 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f837c01a730 con 0x7f838c108630 2024-03-21T09:56:21.605 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:20.455+0000 7f8392600700 1 -- 172.21.15.153:0/4274054863 --> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}) v1 -- 0x7f838c105f60 con 0x7f8370038370 2024-03-21T09:56:21.605 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:20.460+0000 7f8383e00700 1 -- 172.21.15.153:0/4274054863 <== mgr.14120 v2:172.21.15.153:6800/3178125243 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7f838c105f60 con 0x7f8370038370 2024-03-21T09:56:21.605 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:20.465+0000 7f8392600700 1 -- 172.21.15.153:0/4274054863 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7f8370038370 msgr2=0x7f837003a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:56:21.605 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:20.465+0000 7f8392600700 1 --2- 172.21.15.153:0/4274054863 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7f8370038370 0x7f837003a830 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f8378006fd0 tx=0x7f8378006e40 comp rx=0 tx=0).stop 2024-03-21T09:56:21.605 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:20.465+0000 7f8392600700 1 -- 172.21.15.153:0/4274054863 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f838c108630 msgr2=0x7f838c19c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:56:21.605 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:20.465+0000 7f8392600700 1 --2- 172.21.15.153:0/4274054863 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f838c108630 0x7f838c19c3c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f837c011670 tx=0x7f837c011b50 comp rx=0 tx=0).stop 2024-03-21T09:56:21.606 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:20.465+0000 7f8392600700 1 -- 172.21.15.153:0/4274054863 shutdown_connections 2024-03-21T09:56:21.606 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:20.465+0000 7f8392600700 1 --2- 172.21.15.153:0/4274054863 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7f8370038370 0x7f837003a830 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:56:21.606 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:20.465+0000 7f8392600700 1 --2- 172.21.15.153:0/4274054863 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f838c108630 0x7f838c19c3c0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:56:21.606 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:20.465+0000 7f8392600700 1 -- 172.21.15.153:0/4274054863 >> 172.21.15.153:0/4274054863 conn(0x7f838c103c10 msgr2=0x7f838c105340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:56:21.606 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:20.465+0000 7f8392600700 1 -- 172.21.15.153:0/4274054863 shutdown_connections 2024-03-21T09:56:21.606 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:20.465+0000 7f8392600700 1 -- 172.21.15.153:0/4274054863 wait complete. 2024-03-21T09:56:21.606 INFO:teuthology.orchestra.run.smithi153.stdout:Deploying ceph-exporter service with default placement... 2024-03-21T09:56:23.187 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout Scheduled ceph-exporter update... 2024-03-21T09:56:23.188 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:22.168+0000 7f7644e00700 1 Processor -- start 2024-03-21T09:56:23.188 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:22.168+0000 7f7644e00700 1 -- start start 2024-03-21T09:56:23.188 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:22.168+0000 7f7644e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7640106950 0x7f7640106d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:56:23.188 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:22.168+0000 7f7644e00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7640107340 con 0x7f7640106950 2024-03-21T09:56:23.188 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:22.169+0000 7f763de00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7640106950 0x7f7640106d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:56:23.188 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:22.169+0000 7f763de00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7640106950 0x7f7640106d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:46490/0 (socket says 172.21.15.153:46490) 2024-03-21T09:56:23.188 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:22.169+0000 7f763de00700 1 -- 172.21.15.153:0/1072679664 learned_addr learned my addr 172.21.15.153:0/1072679664 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T09:56:23.188 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:22.169+0000 7f763de00700 1 -- 172.21.15.153:0/1072679664 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7640107480 con 0x7f7640106950 2024-03-21T09:56:23.188 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:22.170+0000 7f763de00700 1 --2- 172.21.15.153:0/1072679664 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7640106950 0x7f7640106d70 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f76300098a0 tx=0x7f7630009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d8b2632bec583b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:56:23.188 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:22.170+0000 7f763ca00700 1 -- 172.21.15.153:0/1072679664 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7630004180 con 0x7f7640106950 2024-03-21T09:56:23.188 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:22.170+0000 7f763ca00700 1 -- 172.21.15.153:0/1072679664 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f76300042e0 con 0x7f7640106950 2024-03-21T09:56:23.188 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:22.170+0000 7f763ca00700 1 -- 172.21.15.153:0/1072679664 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7630011af0 con 0x7f7640106950 2024-03-21T09:56:23.188 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:22.171+0000 7f7644e00700 1 -- 172.21.15.153:0/1072679664 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7640106950 msgr2=0x7f7640106d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:56:23.189 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:22.171+0000 7f7644e00700 1 --2- 172.21.15.153:0/1072679664 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7640106950 0x7f7640106d70 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f76300098a0 tx=0x7f7630009bb0 comp rx=0 tx=0).stop 2024-03-21T09:56:23.194 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:22.171+0000 7f7644e00700 1 -- 172.21.15.153:0/1072679664 shutdown_connections 2024-03-21T09:56:23.194 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:22.171+0000 7f7644e00700 1 --2- 172.21.15.153:0/1072679664 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7640106950 0x7f7640106d70 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:56:23.194 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:22.171+0000 7f7644e00700 1 -- 172.21.15.153:0/1072679664 >> 172.21.15.153:0/1072679664 conn(0x7f7640100040 msgr2=0x7f76401024a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:56:23.194 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:22.171+0000 7f7644e00700 1 -- 172.21.15.153:0/1072679664 shutdown_connections 2024-03-21T09:56:23.194 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:22.171+0000 7f7644e00700 1 -- 172.21.15.153:0/1072679664 wait complete. 2024-03-21T09:56:23.195 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:22.172+0000 7f7644e00700 1 Processor -- start 2024-03-21T09:56:23.195 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:22.172+0000 7f7644e00700 1 -- start start 2024-03-21T09:56:23.195 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:22.172+0000 7f7644e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7640106950 0x7f764019c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:56:23.195 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:22.172+0000 7f7644e00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f764019c8e0 con 0x7f7640106950 2024-03-21T09:56:23.195 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:22.172+0000 7f763de00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7640106950 0x7f764019c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:56:23.195 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:22.173+0000 7f763de00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7640106950 0x7f764019c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:46498/0 (socket says 172.21.15.153:46498) 2024-03-21T09:56:23.195 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:22.173+0000 7f763de00700 1 -- 172.21.15.153:0/3950387940 learned_addr learned my addr 172.21.15.153:0/3950387940 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T09:56:23.195 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:22.173+0000 7f763de00700 1 -- 172.21.15.153:0/3950387940 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7630009550 con 0x7f7640106950 2024-03-21T09:56:23.195 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:22.173+0000 7f763de00700 1 --2- 172.21.15.153:0/3950387940 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7640106950 0x7f764019c3a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7630011670 tx=0x7f7630011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:56:23.195 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:22.173+0000 7f7636a00700 1 -- 172.21.15.153:0/3950387940 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f76300039f0 con 0x7f7640106950 2024-03-21T09:56:23.195 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:22.173+0000 7f7636a00700 1 -- 172.21.15.153:0/3950387940 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f7630003b50 con 0x7f7640106950 2024-03-21T09:56:23.195 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:22.173+0000 7f7636a00700 1 -- 172.21.15.153:0/3950387940 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f76300247c0 con 0x7f7640106950 2024-03-21T09:56:23.195 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:22.173+0000 7f7644e00700 1 -- 172.21.15.153:0/3950387940 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f764019cae0 con 0x7f7640106950 2024-03-21T09:56:23.196 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:22.174+0000 7f7644e00700 1 -- 172.21.15.153:0/3950387940 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f764019cf80 con 0x7f7640106950 2024-03-21T09:56:23.196 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:22.175+0000 7f7636a00700 1 -- 172.21.15.153:0/3950387940 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f763001ac00 con 0x7f7640106950 2024-03-21T09:56:23.196 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:22.175+0000 7f7644e00700 1 -- 172.21.15.153:0/3950387940 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7640196040 con 0x7f7640106950 2024-03-21T09:56:23.196 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:22.175+0000 7f7636a00700 1 --2- 172.21.15.153:0/3950387940 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7f7624038370 0x7f762403a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:56:23.196 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:22.176+0000 7f7636a00700 1 -- 172.21.15.153:0/3950387940 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f763004c450 con 0x7f7640106950 2024-03-21T09:56:23.196 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:22.176+0000 7f763d400700 1 --2- 172.21.15.153:0/3950387940 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7f7624038370 0x7f762403a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:56:23.196 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:22.176+0000 7f763d400700 1 --2- 172.21.15.153:0/3950387940 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7f7624038370 0x7f762403a830 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f762c006fd0 tx=0x7f762c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:56:23.196 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:22.181+0000 7f7636a00700 1 -- 172.21.15.153:0/3950387940 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f763001a430 con 0x7f7640106950 2024-03-21T09:56:23.196 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:22.426+0000 7f7644e00700 1 -- 172.21.15.153:0/3950387940 --> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "ceph-exporter", "target": ["mon-mgr", ""]}) v1 -- 0x7f76400611d0 con 0x7f7624038370 2024-03-21T09:56:23.196 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:22.433+0000 7f7636a00700 1 -- 172.21.15.153:0/3950387940 <== mgr.14120 v2:172.21.15.153:6800/3178125243 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+34 (secure 0 0 0) 0x7f76400611d0 con 0x7f7624038370 2024-03-21T09:56:23.196 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:22.436+0000 7f7644e00700 1 -- 172.21.15.153:0/3950387940 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7f7624038370 msgr2=0x7f762403a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:56:23.196 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:22.436+0000 7f7644e00700 1 --2- 172.21.15.153:0/3950387940 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7f7624038370 0x7f762403a830 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f762c006fd0 tx=0x7f762c006e40 comp rx=0 tx=0).stop 2024-03-21T09:56:23.196 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:22.436+0000 7f7644e00700 1 -- 172.21.15.153:0/3950387940 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7640106950 msgr2=0x7f764019c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:56:23.196 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:22.436+0000 7f7644e00700 1 --2- 172.21.15.153:0/3950387940 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7640106950 0x7f764019c3a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7630011670 tx=0x7f7630011b50 comp rx=0 tx=0).stop 2024-03-21T09:56:23.197 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:22.436+0000 7f7644e00700 1 -- 172.21.15.153:0/3950387940 shutdown_connections 2024-03-21T09:56:23.197 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:22.436+0000 7f7644e00700 1 --2- 172.21.15.153:0/3950387940 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7f7624038370 0x7f762403a830 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:56:23.197 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:22.436+0000 7f7644e00700 1 --2- 172.21.15.153:0/3950387940 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7640106950 0x7f764019c3a0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:56:23.197 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:22.436+0000 7f7644e00700 1 -- 172.21.15.153:0/3950387940 >> 172.21.15.153:0/3950387940 conn(0x7f7640100040 msgr2=0x7f7640193180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:56:23.197 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:22.436+0000 7f7644e00700 1 -- 172.21.15.153:0/3950387940 shutdown_connections 2024-03-21T09:56:23.197 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:22.436+0000 7f7644e00700 1 -- 172.21.15.153:0/3950387940 wait complete. 2024-03-21T09:56:23.197 INFO:teuthology.orchestra.run.smithi153.stdout:Deploying prometheus service with default placement... 2024-03-21T09:56:23.716 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:23 smithi153 ceph-mon[29117]: from='client.14148 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "ceph-exporter", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:56:23.717 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:23 smithi153 ceph-mon[29117]: Saving service ceph-exporter spec with placement * 2024-03-21T09:56:23.717 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:23 smithi153 ceph-mon[29117]: from='mgr.14120 172.21.15.153:0/1521951433' entity='mgr.smithi153.tluxni' 2024-03-21T09:56:23.717 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:23 smithi153 ceph-mon[29117]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:56:23.717 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:23 smithi153 ceph-mon[29117]: from='mgr.14120 172.21.15.153:0/1521951433' entity='mgr.smithi153.tluxni' 2024-03-21T09:56:25.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:25 smithi153 ceph-mon[29117]: from='client.14150 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:56:25.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:25 smithi153 ceph-mon[29117]: Saving service prometheus spec with placement count:1 2024-03-21T09:56:25.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:25 smithi153 ceph-mon[29117]: from='mgr.14120 172.21.15.153:0/1521951433' entity='mgr.smithi153.tluxni' 2024-03-21T09:56:25.354 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout Scheduled prometheus update... 2024-03-21T09:56:25.354 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:23.780+0000 7f722a000700 1 Processor -- start 2024-03-21T09:56:25.355 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:23.780+0000 7f722a000700 1 -- start start 2024-03-21T09:56:25.355 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:23.780+0000 7f722a000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f722407ad80 0x7f72240791e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:56:25.355 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:23.780+0000 7f722a000700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72240797b0 con 0x7f722407ad80 2024-03-21T09:56:25.355 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:23.781+0000 7f7223400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f722407ad80 0x7f72240791e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:56:25.355 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:23.781+0000 7f7223400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f722407ad80 0x7f72240791e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:50338/0 (socket says 172.21.15.153:50338) 2024-03-21T09:56:25.355 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:23.781+0000 7f7223400700 1 -- 172.21.15.153:0/57082836 learned_addr learned my addr 172.21.15.153:0/57082836 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T09:56:25.355 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:23.781+0000 7f7223400700 1 -- 172.21.15.153:0/57082836 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72240798f0 con 0x7f722407ad80 2024-03-21T09:56:25.355 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:23.781+0000 7f7223400700 1 --2- 172.21.15.153:0/57082836 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f722407ad80 0x7f72240791e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f72140098a0 tx=0x7f7214009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76eabef4ff94540b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:56:25.355 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:23.782+0000 7f7222a00700 1 -- 172.21.15.153:0/57082836 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7214004180 con 0x7f722407ad80 2024-03-21T09:56:25.355 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:23.782+0000 7f7222a00700 1 -- 172.21.15.153:0/57082836 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f72140042e0 con 0x7f722407ad80 2024-03-21T09:56:25.355 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:23.782+0000 7f7222a00700 1 -- 172.21.15.153:0/57082836 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7214011af0 con 0x7f722407ad80 2024-03-21T09:56:25.355 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:23.782+0000 7f722a000700 1 -- 172.21.15.153:0/57082836 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f722407ad80 msgr2=0x7f72240791e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:56:25.355 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:23.782+0000 7f722a000700 1 --2- 172.21.15.153:0/57082836 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f722407ad80 0x7f72240791e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f72140098a0 tx=0x7f7214009bb0 comp rx=0 tx=0).stop 2024-03-21T09:56:25.356 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:23.783+0000 7f722a000700 1 -- 172.21.15.153:0/57082836 shutdown_connections 2024-03-21T09:56:25.356 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:23.783+0000 7f722a000700 1 --2- 172.21.15.153:0/57082836 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f722407ad80 0x7f72240791e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:56:25.356 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:23.783+0000 7f722a000700 1 -- 172.21.15.153:0/57082836 >> 172.21.15.153:0/57082836 conn(0x7f72241019b0 msgr2=0x7f7224103e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:56:25.356 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:23.783+0000 7f722a000700 1 -- 172.21.15.153:0/57082836 shutdown_connections 2024-03-21T09:56:25.356 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:23.783+0000 7f722a000700 1 -- 172.21.15.153:0/57082836 wait complete. 2024-03-21T09:56:25.356 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:23.784+0000 7f722a000700 1 Processor -- start 2024-03-21T09:56:25.356 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:23.784+0000 7f722a000700 1 -- start start 2024-03-21T09:56:25.356 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:23.784+0000 7f722a000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f722407ad80 0x7f722419c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:56:25.356 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:23.784+0000 7f722a000700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f722419c860 con 0x7f722407ad80 2024-03-21T09:56:25.356 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:23.785+0000 7f7223400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f722407ad80 0x7f722419c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:56:25.356 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:23.785+0000 7f7223400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f722407ad80 0x7f722419c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:50354/0 (socket says 172.21.15.153:50354) 2024-03-21T09:56:25.356 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:23.785+0000 7f7223400700 1 -- 172.21.15.153:0/2478098010 learned_addr learned my addr 172.21.15.153:0/2478098010 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T09:56:25.356 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:23.785+0000 7f7223400700 1 -- 172.21.15.153:0/2478098010 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7214009550 con 0x7f722407ad80 2024-03-21T09:56:25.356 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:23.785+0000 7f7223400700 1 --2- 172.21.15.153:0/2478098010 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f722407ad80 0x7f722419c320 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7214011670 tx=0x7f7214011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:56:25.356 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:23.785+0000 7f7220c00700 1 -- 172.21.15.153:0/2478098010 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7214003a40 con 0x7f722407ad80 2024-03-21T09:56:25.357 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:23.786+0000 7f7220c00700 1 -- 172.21.15.153:0/2478098010 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f7214003ba0 con 0x7f722407ad80 2024-03-21T09:56:25.357 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:23.786+0000 7f722a000700 1 -- 172.21.15.153:0/2478098010 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f722419ca60 con 0x7f722407ad80 2024-03-21T09:56:25.357 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:23.786+0000 7f7220c00700 1 -- 172.21.15.153:0/2478098010 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7214024790 con 0x7f722407ad80 2024-03-21T09:56:25.357 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:23.786+0000 7f722a000700 1 -- 172.21.15.153:0/2478098010 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f722419cf00 con 0x7f722407ad80 2024-03-21T09:56:25.357 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:23.787+0000 7f7220c00700 1 -- 172.21.15.153:0/2478098010 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f721401a430 con 0x7f722407ad80 2024-03-21T09:56:25.357 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:23.787+0000 7f722a000700 1 -- 172.21.15.153:0/2478098010 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f722410c810 con 0x7f722407ad80 2024-03-21T09:56:25.357 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:23.787+0000 7f7220c00700 1 --2- 172.21.15.153:0/2478098010 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7f7204038320 0x7f720403a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:56:25.357 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:23.788+0000 7f7220c00700 1 -- 172.21.15.153:0/2478098010 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f721404c5e0 con 0x7f722407ad80 2024-03-21T09:56:25.357 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:23.788+0000 7f721be00700 1 --2- 172.21.15.153:0/2478098010 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7f7204038320 0x7f720403a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:56:25.357 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:23.789+0000 7f721be00700 1 --2- 172.21.15.153:0/2478098010 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7f7204038320 0x7f720403a7e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f720c006fd0 tx=0x7f720c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:56:25.357 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:23.793+0000 7f7220c00700 1 -- 172.21.15.153:0/2478098010 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f7214010380 con 0x7f722407ad80 2024-03-21T09:56:25.357 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:24.049+0000 7f722a000700 1 -- 172.21.15.153:0/2478098010 --> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}) v1 -- 0x7f722419fd40 con 0x7f7204038320 2024-03-21T09:56:25.357 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:24.057+0000 7f7220c00700 1 -- 172.21.15.153:0/2478098010 <== mgr.14120 v2:172.21.15.153:6800/3178125243 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+31 (secure 0 0 0) 0x7f722419fd40 con 0x7f7204038320 2024-03-21T09:56:25.357 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:24.060+0000 7f722a000700 1 -- 172.21.15.153:0/2478098010 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7f7204038320 msgr2=0x7f720403a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:56:25.358 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:24.060+0000 7f722a000700 1 --2- 172.21.15.153:0/2478098010 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7f7204038320 0x7f720403a7e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f720c006fd0 tx=0x7f720c006e40 comp rx=0 tx=0).stop 2024-03-21T09:56:25.358 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:24.060+0000 7f722a000700 1 -- 172.21.15.153:0/2478098010 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f722407ad80 msgr2=0x7f722419c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:56:25.358 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:24.060+0000 7f722a000700 1 --2- 172.21.15.153:0/2478098010 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f722407ad80 0x7f722419c320 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7214011670 tx=0x7f7214011b50 comp rx=0 tx=0).stop 2024-03-21T09:56:25.358 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:24.060+0000 7f722a000700 1 -- 172.21.15.153:0/2478098010 shutdown_connections 2024-03-21T09:56:25.358 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:24.060+0000 7f722a000700 1 --2- 172.21.15.153:0/2478098010 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7f7204038320 0x7f720403a7e0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:56:25.358 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:24.060+0000 7f722a000700 1 --2- 172.21.15.153:0/2478098010 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f722407ad80 0x7f722419c320 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:56:25.358 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:24.060+0000 7f722a000700 1 -- 172.21.15.153:0/2478098010 >> 172.21.15.153:0/2478098010 conn(0x7f72241019b0 msgr2=0x7f72241030c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:56:25.358 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:24.061+0000 7f722a000700 1 -- 172.21.15.153:0/2478098010 shutdown_connections 2024-03-21T09:56:25.358 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:24.061+0000 7f722a000700 1 -- 172.21.15.153:0/2478098010 wait complete. 2024-03-21T09:56:25.358 INFO:teuthology.orchestra.run.smithi153.stdout:Deploying grafana service with default placement... 2024-03-21T09:56:26.251 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:26 smithi153 ceph-mon[29117]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:56:27.165 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout Scheduled grafana update... 2024-03-21T09:56:27.166 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:26.249+0000 7fce23e00700 1 Processor -- start 2024-03-21T09:56:27.166 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:26.249+0000 7fce23e00700 1 -- start start 2024-03-21T09:56:27.166 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:26.250+0000 7fce23e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fce24106260 0x7fce24106680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:56:27.166 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:26.250+0000 7fce23e00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce24106c50 con 0x7fce24106260 2024-03-21T09:56:27.166 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:26.250+0000 7fce22a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fce24106260 0x7fce24106680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:56:27.166 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:26.251+0000 7fce22a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fce24106260 0x7fce24106680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:50370/0 (socket says 172.21.15.153:50370) 2024-03-21T09:56:27.166 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:26.251+0000 7fce22a00700 1 -- 172.21.15.153:0/3767149846 learned_addr learned my addr 172.21.15.153:0/3767149846 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T09:56:27.166 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:26.251+0000 7fce22a00700 1 -- 172.21.15.153:0/3767149846 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce24107460 con 0x7fce24106260 2024-03-21T09:56:27.166 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:26.251+0000 7fce22a00700 1 --2- 172.21.15.153:0/3767149846 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fce24106260 0x7fce24106680 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fce140098a0 tx=0x7fce14009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6af68d9f15f9bc7 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:56:27.166 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:26.252+0000 7fce21600700 1 -- 172.21.15.153:0/3767149846 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fce14004180 con 0x7fce24106260 2024-03-21T09:56:27.166 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:26.252+0000 7fce21600700 1 -- 172.21.15.153:0/3767149846 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fce140042e0 con 0x7fce24106260 2024-03-21T09:56:27.166 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:26.252+0000 7fce21600700 1 -- 172.21.15.153:0/3767149846 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fce14011af0 con 0x7fce24106260 2024-03-21T09:56:27.167 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:26.252+0000 7fce23e00700 1 -- 172.21.15.153:0/3767149846 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fce24106260 msgr2=0x7fce24106680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:56:27.167 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:26.252+0000 7fce23e00700 1 --2- 172.21.15.153:0/3767149846 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fce24106260 0x7fce24106680 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fce140098a0 tx=0x7fce14009bb0 comp rx=0 tx=0).stop 2024-03-21T09:56:27.167 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:26.253+0000 7fce23e00700 1 -- 172.21.15.153:0/3767149846 shutdown_connections 2024-03-21T09:56:27.167 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:26.253+0000 7fce23e00700 1 --2- 172.21.15.153:0/3767149846 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fce24106260 0x7fce24106680 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:56:27.167 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:26.253+0000 7fce23e00700 1 -- 172.21.15.153:0/3767149846 >> 172.21.15.153:0/3767149846 conn(0x7fce24101820 msgr2=0x7fce24103c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:56:27.167 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:26.253+0000 7fce23e00700 1 -- 172.21.15.153:0/3767149846 shutdown_connections 2024-03-21T09:56:27.167 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:26.253+0000 7fce23e00700 1 -- 172.21.15.153:0/3767149846 wait complete. 2024-03-21T09:56:27.167 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:26.254+0000 7fce23e00700 1 Processor -- start 2024-03-21T09:56:27.167 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:26.254+0000 7fce23e00700 1 -- start start 2024-03-21T09:56:27.167 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:26.254+0000 7fce23e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fce24106260 0x7fce24199f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:56:27.167 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:26.254+0000 7fce23e00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce2419a480 con 0x7fce24106260 2024-03-21T09:56:27.167 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:26.254+0000 7fce22a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fce24106260 0x7fce24199f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:56:27.167 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:26.254+0000 7fce22a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fce24106260 0x7fce24199f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:50382/0 (socket says 172.21.15.153:50382) 2024-03-21T09:56:27.167 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:26.254+0000 7fce22a00700 1 -- 172.21.15.153:0/2263693985 learned_addr learned my addr 172.21.15.153:0/2263693985 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T09:56:27.168 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:26.255+0000 7fce22a00700 1 -- 172.21.15.153:0/2263693985 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce14009550 con 0x7fce24106260 2024-03-21T09:56:27.168 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:26.255+0000 7fce22a00700 1 --2- 172.21.15.153:0/2263693985 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fce24106260 0x7fce24199f40 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fce14000c00 tx=0x7fce14011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:56:27.168 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:26.255+0000 7fce28a00700 1 -- 172.21.15.153:0/2263693985 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fce14003a70 con 0x7fce24106260 2024-03-21T09:56:27.168 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:26.255+0000 7fce28a00700 1 -- 172.21.15.153:0/2263693985 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fce14003bd0 con 0x7fce24106260 2024-03-21T09:56:27.168 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:26.255+0000 7fce28a00700 1 -- 172.21.15.153:0/2263693985 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fce140238c0 con 0x7fce24106260 2024-03-21T09:56:27.168 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:26.255+0000 7fce23e00700 1 -- 172.21.15.153:0/2263693985 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce2419a680 con 0x7fce24106260 2024-03-21T09:56:27.168 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:26.256+0000 7fce23e00700 1 -- 172.21.15.153:0/2263693985 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce2419ab20 con 0x7fce24106260 2024-03-21T09:56:27.168 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:26.257+0000 7fce28a00700 1 -- 172.21.15.153:0/2263693985 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7fce14023b30 con 0x7fce24106260 2024-03-21T09:56:27.168 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:26.257+0000 7fce23e00700 1 -- 172.21.15.153:0/2263693985 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce240623c0 con 0x7fce24106260 2024-03-21T09:56:27.168 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:26.257+0000 7fce28a00700 1 --2- 172.21.15.153:0/2263693985 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7fce08038370 0x7fce0803a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:56:27.168 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:26.257+0000 7fce28a00700 1 -- 172.21.15.153:0/2263693985 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fce1404c240 con 0x7fce24106260 2024-03-21T09:56:27.168 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:26.258+0000 7fce22000700 1 --2- 172.21.15.153:0/2263693985 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7fce08038370 0x7fce0803a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:56:27.168 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:26.258+0000 7fce22000700 1 --2- 172.21.15.153:0/2263693985 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7fce08038370 0x7fce0803a830 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fce10006fd0 tx=0x7fce10006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:56:27.169 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:26.262+0000 7fce28a00700 1 -- 172.21.15.153:0/2263693985 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fce14010380 con 0x7fce24106260 2024-03-21T09:56:27.169 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:26.489+0000 7fce23e00700 1 -- 172.21.15.153:0/2263693985 --> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}) v1 -- 0x7fce2410a660 con 0x7fce08038370 2024-03-21T09:56:27.169 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:26.501+0000 7fce28a00700 1 -- 172.21.15.153:0/2263693985 <== mgr.14120 v2:172.21.15.153:6800/3178125243 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7fce2410a660 con 0x7fce08038370 2024-03-21T09:56:27.169 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:26.504+0000 7fce23e00700 1 -- 172.21.15.153:0/2263693985 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7fce08038370 msgr2=0x7fce0803a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:56:27.169 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:26.504+0000 7fce23e00700 1 --2- 172.21.15.153:0/2263693985 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7fce08038370 0x7fce0803a830 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fce10006fd0 tx=0x7fce10006e40 comp rx=0 tx=0).stop 2024-03-21T09:56:27.169 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:26.504+0000 7fce23e00700 1 -- 172.21.15.153:0/2263693985 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fce24106260 msgr2=0x7fce24199f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:56:27.169 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:26.504+0000 7fce23e00700 1 --2- 172.21.15.153:0/2263693985 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fce24106260 0x7fce24199f40 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fce14000c00 tx=0x7fce14011b50 comp rx=0 tx=0).stop 2024-03-21T09:56:27.169 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:26.504+0000 7fce23e00700 1 -- 172.21.15.153:0/2263693985 shutdown_connections 2024-03-21T09:56:27.169 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:26.504+0000 7fce23e00700 1 --2- 172.21.15.153:0/2263693985 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7fce08038370 0x7fce0803a830 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:56:27.169 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:26.504+0000 7fce23e00700 1 --2- 172.21.15.153:0/2263693985 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fce24106260 0x7fce24199f40 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:56:27.169 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:26.504+0000 7fce23e00700 1 -- 172.21.15.153:0/2263693985 >> 172.21.15.153:0/2263693985 conn(0x7fce24101820 msgr2=0x7fce24102610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:56:27.169 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:26.504+0000 7fce23e00700 1 -- 172.21.15.153:0/2263693985 shutdown_connections 2024-03-21T09:56:27.169 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:26.504+0000 7fce23e00700 1 -- 172.21.15.153:0/2263693985 wait complete. 2024-03-21T09:56:27.170 INFO:teuthology.orchestra.run.smithi153.stdout:Deploying node-exporter service with default placement... 2024-03-21T09:56:27.763 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:27 smithi153 ceph-mon[29117]: from='client.14152 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:56:27.763 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:27 smithi153 ceph-mon[29117]: Saving service grafana spec with placement count:1 2024-03-21T09:56:27.763 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:27 smithi153 ceph-mon[29117]: from='mgr.14120 172.21.15.153:0/1521951433' entity='mgr.smithi153.tluxni' 2024-03-21T09:56:27.763 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:27 smithi153 ceph-mon[29117]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:56:28.684 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout Scheduled node-exporter update... 2024-03-21T09:56:28.684 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:27.762+0000 7fd55fe00700 1 Processor -- start 2024-03-21T09:56:28.684 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:27.762+0000 7fd55fe00700 1 -- start start 2024-03-21T09:56:28.684 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:27.762+0000 7fd55fe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd558106420 0x7fd558106840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:56:28.684 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:27.762+0000 7fd55fe00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd558106e10 con 0x7fd558106420 2024-03-21T09:56:28.684 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:27.762+0000 7fd55d400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd558106420 0x7fd558106840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:56:28.685 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:27.762+0000 7fd55d400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd558106420 0x7fd558106840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:50390/0 (socket says 172.21.15.153:50390) 2024-03-21T09:56:28.685 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:27.762+0000 7fd55d400700 1 -- 172.21.15.153:0/4206502675 learned_addr learned my addr 172.21.15.153:0/4206502675 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T09:56:28.685 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:27.762+0000 7fd55d400700 1 -- 172.21.15.153:0/4206502675 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd558107620 con 0x7fd558106420 2024-03-21T09:56:28.685 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:27.763+0000 7fd55d400700 1 --2- 172.21.15.153:0/4206502675 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd558106420 0x7fd558106840 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd54c009e80 tx=0x7fd54c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4c8315eb324a917 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:56:28.685 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:27.763+0000 7fd557e00700 1 -- 172.21.15.153:0/4206502675 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd54c00e070 con 0x7fd558106420 2024-03-21T09:56:28.685 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:27.763+0000 7fd557e00700 1 -- 172.21.15.153:0/4206502675 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fd54c00b8c0 con 0x7fd558106420 2024-03-21T09:56:28.685 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:27.763+0000 7fd557e00700 1 -- 172.21.15.153:0/4206502675 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd54c0044d0 con 0x7fd558106420 2024-03-21T09:56:28.685 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:27.764+0000 7fd55fe00700 1 -- 172.21.15.153:0/4206502675 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd558106420 msgr2=0x7fd558106840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:56:28.685 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:27.764+0000 7fd55fe00700 1 --2- 172.21.15.153:0/4206502675 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd558106420 0x7fd558106840 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd54c009e80 tx=0x7fd54c00b330 comp rx=0 tx=0).stop 2024-03-21T09:56:28.685 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:27.764+0000 7fd55fe00700 1 -- 172.21.15.153:0/4206502675 shutdown_connections 2024-03-21T09:56:28.685 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:27.764+0000 7fd55fe00700 1 --2- 172.21.15.153:0/4206502675 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd558106420 0x7fd558106840 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:56:28.685 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:27.764+0000 7fd55fe00700 1 -- 172.21.15.153:0/4206502675 >> 172.21.15.153:0/4206502675 conn(0x7fd558101a00 msgr2=0x7fd558103e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:56:28.685 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:27.764+0000 7fd55fe00700 1 -- 172.21.15.153:0/4206502675 shutdown_connections 2024-03-21T09:56:28.685 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:27.764+0000 7fd55fe00700 1 -- 172.21.15.153:0/4206502675 wait complete. 2024-03-21T09:56:28.686 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:27.765+0000 7fd55fe00700 1 Processor -- start 2024-03-21T09:56:28.686 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:27.765+0000 7fd55fe00700 1 -- start start 2024-03-21T09:56:28.686 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:27.765+0000 7fd55fe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd558106420 0x7fd55807bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:56:28.686 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:27.765+0000 7fd55fe00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd55807c1a0 con 0x7fd558106420 2024-03-21T09:56:28.686 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:27.766+0000 7fd55d400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd558106420 0x7fd55807bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:56:28.686 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:27.766+0000 7fd55d400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd558106420 0x7fd55807bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:50392/0 (socket says 172.21.15.153:50392) 2024-03-21T09:56:28.686 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:27.766+0000 7fd55d400700 1 -- 172.21.15.153:0/424923465 learned_addr learned my addr 172.21.15.153:0/424923465 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T09:56:28.686 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:27.766+0000 7fd55d400700 1 -- 172.21.15.153:0/424923465 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd54c009b30 con 0x7fd558106420 2024-03-21T09:56:28.686 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:27.766+0000 7fd55d400700 1 --2- 172.21.15.153:0/424923465 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd558106420 0x7fd55807bc60 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd54c004940 tx=0x7fd54c004a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:56:28.686 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:27.767+0000 7fd556000700 1 -- 172.21.15.153:0/424923465 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd54c00e040 con 0x7fd558106420 2024-03-21T09:56:28.686 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:27.767+0000 7fd556000700 1 -- 172.21.15.153:0/424923465 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fd54c00bc40 con 0x7fd558106420 2024-03-21T09:56:28.686 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:27.767+0000 7fd556000700 1 -- 172.21.15.153:0/424923465 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd54c0126e0 con 0x7fd558106420 2024-03-21T09:56:28.686 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:27.767+0000 7fd55fe00700 1 -- 172.21.15.153:0/424923465 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd55807a2b0 con 0x7fd558106420 2024-03-21T09:56:28.686 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:27.767+0000 7fd55fe00700 1 -- 172.21.15.153:0/424923465 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd55807a7b0 con 0x7fd558106420 2024-03-21T09:56:28.687 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:27.768+0000 7fd55fe00700 1 -- 172.21.15.153:0/424923465 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5580623c0 con 0x7fd558106420 2024-03-21T09:56:28.687 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:27.768+0000 7fd556000700 1 -- 172.21.15.153:0/424923465 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7fd54c019040 con 0x7fd558106420 2024-03-21T09:56:28.687 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:27.769+0000 7fd556000700 1 --2- 172.21.15.153:0/424923465 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7fd540038370 0x7fd54003a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:56:28.687 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:27.769+0000 7fd556000700 1 -- 172.21.15.153:0/424923465 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fd54c04bcd0 con 0x7fd558106420 2024-03-21T09:56:28.687 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:27.769+0000 7fd55ca00700 1 --2- 172.21.15.153:0/424923465 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7fd540038370 0x7fd54003a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:56:28.687 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:27.770+0000 7fd55ca00700 1 --2- 172.21.15.153:0/424923465 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7fd540038370 0x7fd54003a830 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd544006fd0 tx=0x7fd544006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:56:28.687 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:27.774+0000 7fd556000700 1 -- 172.21.15.153:0/424923465 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fd54c012cd0 con 0x7fd558106420 2024-03-21T09:56:28.687 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:27.999+0000 7fd55fe00700 1 -- 172.21.15.153:0/424923465 --> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}) v1 -- 0x7fd55810a820 con 0x7fd540038370 2024-03-21T09:56:28.687 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:28.012+0000 7fd556000700 1 -- 172.21.15.153:0/424923465 <== mgr.14120 v2:172.21.15.153:6800/3178125243 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+34 (secure 0 0 0) 0x7fd55810a820 con 0x7fd540038370 2024-03-21T09:56:28.687 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:28.015+0000 7fd55fe00700 1 -- 172.21.15.153:0/424923465 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7fd540038370 msgr2=0x7fd54003a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:56:28.687 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:28.015+0000 7fd55fe00700 1 --2- 172.21.15.153:0/424923465 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7fd540038370 0x7fd54003a830 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd544006fd0 tx=0x7fd544006e40 comp rx=0 tx=0).stop 2024-03-21T09:56:28.687 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:28.015+0000 7fd55fe00700 1 -- 172.21.15.153:0/424923465 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd558106420 msgr2=0x7fd55807bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:56:28.687 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:28.015+0000 7fd55fe00700 1 --2- 172.21.15.153:0/424923465 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd558106420 0x7fd55807bc60 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd54c004940 tx=0x7fd54c004a20 comp rx=0 tx=0).stop 2024-03-21T09:56:28.688 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:28.015+0000 7fd55fe00700 1 -- 172.21.15.153:0/424923465 shutdown_connections 2024-03-21T09:56:28.688 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:28.015+0000 7fd55fe00700 1 --2- 172.21.15.153:0/424923465 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7fd540038370 0x7fd54003a830 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:56:28.688 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:28.015+0000 7fd55fe00700 1 --2- 172.21.15.153:0/424923465 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd558106420 0x7fd55807bc60 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:56:28.688 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:28.015+0000 7fd55fe00700 1 -- 172.21.15.153:0/424923465 >> 172.21.15.153:0/424923465 conn(0x7fd558101a00 msgr2=0x7fd5581027b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:56:28.688 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:28.015+0000 7fd55fe00700 1 -- 172.21.15.153:0/424923465 shutdown_connections 2024-03-21T09:56:28.688 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:28.015+0000 7fd55fe00700 1 -- 172.21.15.153:0/424923465 wait complete. 2024-03-21T09:56:28.688 INFO:teuthology.orchestra.run.smithi153.stdout:Deploying alertmanager service with default placement... 2024-03-21T09:56:29.270 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:29 smithi153 ceph-mon[29117]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:56:29.270 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:29 smithi153 ceph-mon[29117]: Saving service node-exporter spec with placement * 2024-03-21T09:56:29.270 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:29 smithi153 ceph-mon[29117]: from='mgr.14120 172.21.15.153:0/1521951433' entity='mgr.smithi153.tluxni' 2024-03-21T09:56:30.230 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:30 smithi153 ceph-mon[29117]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:56:30.230 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:30 smithi153 ceph-mon[29117]: from='mgr.14120 172.21.15.153:0/1521951433' entity='mgr.smithi153.tluxni' 2024-03-21T09:56:30.249 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout Scheduled alertmanager update... 2024-03-21T09:56:30.249 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:29.269+0000 7fcbd3e00700 1 Processor -- start 2024-03-21T09:56:30.250 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:29.269+0000 7fcbd3e00700 1 -- start start 2024-03-21T09:56:30.250 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:29.270+0000 7fcbd3e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcbcc108680 0x7fcbcc108aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:56:30.250 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:29.270+0000 7fcbd3e00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbcc109070 con 0x7fcbcc108680 2024-03-21T09:56:30.250 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:29.270+0000 7fcbd1400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcbcc108680 0x7fcbcc108aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:56:30.250 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:29.270+0000 7fcbd1400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcbcc108680 0x7fcbcc108aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:50400/0 (socket says 172.21.15.153:50400) 2024-03-21T09:56:30.250 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:29.270+0000 7fcbd1400700 1 -- 172.21.15.153:0/2308442603 learned_addr learned my addr 172.21.15.153:0/2308442603 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T09:56:30.250 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:29.270+0000 7fcbd1400700 1 -- 172.21.15.153:0/2308442603 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbcc109830 con 0x7fcbcc108680 2024-03-21T09:56:30.250 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:29.271+0000 7fcbd1400700 1 --2- 172.21.15.153:0/2308442603 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcbcc108680 0x7fcbcc108aa0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fcbc00098a0 tx=0x7fcbc0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec30a04128141f3 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:56:30.250 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:29.271+0000 7fcbcbe00700 1 -- 172.21.15.153:0/2308442603 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcbc0004180 con 0x7fcbcc108680 2024-03-21T09:56:30.250 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:29.271+0000 7fcbcbe00700 1 -- 172.21.15.153:0/2308442603 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fcbc00042e0 con 0x7fcbcc108680 2024-03-21T09:56:30.250 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:29.271+0000 7fcbcbe00700 1 -- 172.21.15.153:0/2308442603 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcbc0011af0 con 0x7fcbcc108680 2024-03-21T09:56:30.250 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:29.272+0000 7fcbd3e00700 1 -- 172.21.15.153:0/2308442603 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcbcc108680 msgr2=0x7fcbcc108aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:56:30.250 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:29.272+0000 7fcbd3e00700 1 --2- 172.21.15.153:0/2308442603 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcbcc108680 0x7fcbcc108aa0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fcbc00098a0 tx=0x7fcbc0009bb0 comp rx=0 tx=0).stop 2024-03-21T09:56:30.251 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:29.272+0000 7fcbd3e00700 1 -- 172.21.15.153:0/2308442603 shutdown_connections 2024-03-21T09:56:30.251 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:29.272+0000 7fcbd3e00700 1 --2- 172.21.15.153:0/2308442603 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcbcc108680 0x7fcbcc108aa0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:56:30.251 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:29.272+0000 7fcbd3e00700 1 -- 172.21.15.153:0/2308442603 >> 172.21.15.153:0/2308442603 conn(0x7fcbcc07bb20 msgr2=0x7fcbcc1064e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:56:30.251 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:29.272+0000 7fcbd3e00700 1 -- 172.21.15.153:0/2308442603 shutdown_connections 2024-03-21T09:56:30.251 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:29.272+0000 7fcbd3e00700 1 -- 172.21.15.153:0/2308442603 wait complete. 2024-03-21T09:56:30.251 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:29.273+0000 7fcbd3e00700 1 Processor -- start 2024-03-21T09:56:30.251 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:29.273+0000 7fcbd3e00700 1 -- start start 2024-03-21T09:56:30.251 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:29.274+0000 7fcbd3e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcbcc108680 0x7fcbcc19c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:56:30.251 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:29.274+0000 7fcbd3e00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbcc19c8e0 con 0x7fcbcc108680 2024-03-21T09:56:30.251 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:29.274+0000 7fcbd1400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcbcc108680 0x7fcbcc19c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:56:30.251 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:29.274+0000 7fcbd1400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcbcc108680 0x7fcbcc19c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:50402/0 (socket says 172.21.15.153:50402) 2024-03-21T09:56:30.251 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:29.274+0000 7fcbd1400700 1 -- 172.21.15.153:0/1115228901 learned_addr learned my addr 172.21.15.153:0/1115228901 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T09:56:30.251 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:29.274+0000 7fcbd1400700 1 -- 172.21.15.153:0/1115228901 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbc0009550 con 0x7fcbcc108680 2024-03-21T09:56:30.251 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:29.275+0000 7fcbd1400700 1 --2- 172.21.15.153:0/1115228901 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcbcc108680 0x7fcbcc19c3a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fcbc0011670 tx=0x7fcbc0011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:56:30.252 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:29.275+0000 7fcbca000700 1 -- 172.21.15.153:0/1115228901 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcbc00039f0 con 0x7fcbcc108680 2024-03-21T09:56:30.252 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:29.275+0000 7fcbca000700 1 -- 172.21.15.153:0/1115228901 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fcbc0003b50 con 0x7fcbcc108680 2024-03-21T09:56:30.252 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:29.275+0000 7fcbd3e00700 1 -- 172.21.15.153:0/1115228901 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbcc19cae0 con 0x7fcbcc108680 2024-03-21T09:56:30.252 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:29.275+0000 7fcbca000700 1 -- 172.21.15.153:0/1115228901 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcbc00247c0 con 0x7fcbcc108680 2024-03-21T09:56:30.252 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:29.275+0000 7fcbd3e00700 1 -- 172.21.15.153:0/1115228901 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbcc19cf80 con 0x7fcbcc108680 2024-03-21T09:56:30.252 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:29.276+0000 7fcbd3e00700 1 -- 172.21.15.153:0/1115228901 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbcc196470 con 0x7fcbcc108680 2024-03-21T09:56:30.252 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:29.276+0000 7fcbca000700 1 -- 172.21.15.153:0/1115228901 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7fcbc001ac00 con 0x7fcbcc108680 2024-03-21T09:56:30.252 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:29.277+0000 7fcbca000700 1 --2- 172.21.15.153:0/1115228901 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7fcbb4038320 0x7fcbb403a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:56:30.252 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:29.277+0000 7fcbca000700 1 -- 172.21.15.153:0/1115228901 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fcbc004c4b0 con 0x7fcbcc108680 2024-03-21T09:56:30.252 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:29.277+0000 7fcbd0a00700 1 --2- 172.21.15.153:0/1115228901 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7fcbb4038320 0x7fcbb403a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:56:30.252 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:29.278+0000 7fcbd0a00700 1 --2- 172.21.15.153:0/1115228901 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7fcbb4038320 0x7fcbb403a7e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fcbb8006fd0 tx=0x7fcbb8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:56:30.252 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:29.282+0000 7fcbca000700 1 -- 172.21.15.153:0/1115228901 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fcbc0010380 con 0x7fcbcc108680 2024-03-21T09:56:30.252 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:29.506+0000 7fcbd3e00700 1 -- 172.21.15.153:0/1115228901 --> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}) v1 -- 0x7fcbcc0611d0 con 0x7fcbb4038320 2024-03-21T09:56:30.253 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:29.516+0000 7fcbca000700 1 -- 172.21.15.153:0/1115228901 <== mgr.14120 v2:172.21.15.153:6800/3178125243 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+33 (secure 0 0 0) 0x7fcbcc0611d0 con 0x7fcbb4038320 2024-03-21T09:56:30.253 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:29.519+0000 7fcbd3e00700 1 -- 172.21.15.153:0/1115228901 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7fcbb4038320 msgr2=0x7fcbb403a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:56:30.253 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:29.519+0000 7fcbd3e00700 1 --2- 172.21.15.153:0/1115228901 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7fcbb4038320 0x7fcbb403a7e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fcbb8006fd0 tx=0x7fcbb8006e40 comp rx=0 tx=0).stop 2024-03-21T09:56:30.253 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:29.519+0000 7fcbd3e00700 1 -- 172.21.15.153:0/1115228901 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcbcc108680 msgr2=0x7fcbcc19c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:56:30.253 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:29.519+0000 7fcbd3e00700 1 --2- 172.21.15.153:0/1115228901 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcbcc108680 0x7fcbcc19c3a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fcbc0011670 tx=0x7fcbc0011b50 comp rx=0 tx=0).stop 2024-03-21T09:56:30.253 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:29.520+0000 7fcbd3e00700 1 -- 172.21.15.153:0/1115228901 shutdown_connections 2024-03-21T09:56:30.253 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:29.520+0000 7fcbd3e00700 1 --2- 172.21.15.153:0/1115228901 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7fcbb4038320 0x7fcbb403a7e0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:56:30.253 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:29.520+0000 7fcbd3e00700 1 --2- 172.21.15.153:0/1115228901 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcbcc108680 0x7fcbcc19c3a0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:56:30.253 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:29.520+0000 7fcbd3e00700 1 -- 172.21.15.153:0/1115228901 >> 172.21.15.153:0/1115228901 conn(0x7fcbcc07bb20 msgr2=0x7fcbcc106840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:56:30.253 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:29.520+0000 7fcbd3e00700 1 -- 172.21.15.153:0/1115228901 shutdown_connections 2024-03-21T09:56:30.253 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:29.520+0000 7fcbd3e00700 1 -- 172.21.15.153:0/1115228901 wait complete. 2024-03-21T09:56:31.130 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:31 smithi153 ceph-mon[29117]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:56:31.130 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:31 smithi153 ceph-mon[29117]: Saving service alertmanager spec with placement count:1 2024-03-21T09:56:31.731 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:30.873+0000 7fcc3fe00700 1 Processor -- start 2024-03-21T09:56:31.731 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:30.873+0000 7fcc3fe00700 1 -- start start 2024-03-21T09:56:31.732 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:30.874+0000 7fcc3fe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcc38106430 0x7fcc38106850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:56:31.732 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:30.874+0000 7fcc3fe00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc38106e20 con 0x7fcc38106430 2024-03-21T09:56:31.733 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:30.874+0000 7fcc3d400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcc38106430 0x7fcc38106850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:56:31.733 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:30.874+0000 7fcc3d400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcc38106430 0x7fcc38106850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:50416/0 (socket says 172.21.15.153:50416) 2024-03-21T09:56:31.733 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:30.874+0000 7fcc3d400700 1 -- 172.21.15.153:0/1211294086 learned_addr learned my addr 172.21.15.153:0/1211294086 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T09:56:31.733 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:30.874+0000 7fcc3d400700 1 -- 172.21.15.153:0/1211294086 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc38107630 con 0x7fcc38106430 2024-03-21T09:56:31.733 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:30.875+0000 7fcc3d400700 1 --2- 172.21.15.153:0/1211294086 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcc38106430 0x7fcc38106850 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c0098a0 tx=0x7fcc2c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a15f13dc4d0350 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:56:31.734 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:30.875+0000 7fcc37e00700 1 -- 172.21.15.153:0/1211294086 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcc2c004180 con 0x7fcc38106430 2024-03-21T09:56:31.734 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:30.875+0000 7fcc37e00700 1 -- 172.21.15.153:0/1211294086 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fcc2c0042e0 con 0x7fcc38106430 2024-03-21T09:56:31.734 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:30.875+0000 7fcc37e00700 1 -- 172.21.15.153:0/1211294086 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcc2c011af0 con 0x7fcc38106430 2024-03-21T09:56:31.734 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:30.876+0000 7fcc3fe00700 1 -- 172.21.15.153:0/1211294086 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcc38106430 msgr2=0x7fcc38106850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:56:31.734 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:30.876+0000 7fcc3fe00700 1 --2- 172.21.15.153:0/1211294086 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcc38106430 0x7fcc38106850 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c0098a0 tx=0x7fcc2c009bb0 comp rx=0 tx=0).stop 2024-03-21T09:56:31.734 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:30.876+0000 7fcc3fe00700 1 -- 172.21.15.153:0/1211294086 shutdown_connections 2024-03-21T09:56:31.734 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:30.876+0000 7fcc3fe00700 1 --2- 172.21.15.153:0/1211294086 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcc38106430 0x7fcc38106850 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:56:31.740 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:30.876+0000 7fcc3fe00700 1 -- 172.21.15.153:0/1211294086 >> 172.21.15.153:0/1211294086 conn(0x7fcc381019d0 msgr2=0x7fcc38103e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:56:31.741 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:30.876+0000 7fcc3fe00700 1 -- 172.21.15.153:0/1211294086 shutdown_connections 2024-03-21T09:56:31.741 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:30.876+0000 7fcc3fe00700 1 -- 172.21.15.153:0/1211294086 wait complete. 2024-03-21T09:56:31.741 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:30.877+0000 7fcc3fe00700 1 Processor -- start 2024-03-21T09:56:31.741 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:30.877+0000 7fcc3fe00700 1 -- start start 2024-03-21T09:56:31.741 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:30.878+0000 7fcc3fe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcc38106430 0x7fcc3807bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:56:31.741 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:30.878+0000 7fcc3fe00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc3807c1a0 con 0x7fcc38106430 2024-03-21T09:56:31.741 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:30.878+0000 7fcc3d400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcc38106430 0x7fcc3807bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:56:31.741 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:30.878+0000 7fcc3d400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcc38106430 0x7fcc3807bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:50426/0 (socket says 172.21.15.153:50426) 2024-03-21T09:56:31.741 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:30.878+0000 7fcc3d400700 1 -- 172.21.15.153:0/2355818306 learned_addr learned my addr 172.21.15.153:0/2355818306 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T09:56:31.741 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:30.878+0000 7fcc3d400700 1 -- 172.21.15.153:0/2355818306 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc2c009550 con 0x7fcc38106430 2024-03-21T09:56:31.741 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:30.879+0000 7fcc3d400700 1 --2- 172.21.15.153:0/2355818306 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcc38106430 0x7fcc3807bc60 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c011670 tx=0x7fcc2c011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:56:31.741 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:30.879+0000 7fcc36000700 1 -- 172.21.15.153:0/2355818306 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcc2c0039f0 con 0x7fcc38106430 2024-03-21T09:56:31.741 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:30.879+0000 7fcc36000700 1 -- 172.21.15.153:0/2355818306 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fcc2c003b50 con 0x7fcc38106430 2024-03-21T09:56:31.742 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:30.879+0000 7fcc36000700 1 -- 172.21.15.153:0/2355818306 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcc2c0247c0 con 0x7fcc38106430 2024-03-21T09:56:31.742 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:30.879+0000 7fcc3fe00700 1 -- 172.21.15.153:0/2355818306 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc3807a2b0 con 0x7fcc38106430 2024-03-21T09:56:31.742 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:30.879+0000 7fcc3fe00700 1 -- 172.21.15.153:0/2355818306 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc3807a750 con 0x7fcc38106430 2024-03-21T09:56:31.742 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:30.880+0000 7fcc3fe00700 1 -- 172.21.15.153:0/2355818306 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc380623c0 con 0x7fcc38106430 2024-03-21T09:56:31.742 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:30.880+0000 7fcc36000700 1 -- 172.21.15.153:0/2355818306 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7fcc2c01ac00 con 0x7fcc38106430 2024-03-21T09:56:31.742 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:30.881+0000 7fcc36000700 1 --2- 172.21.15.153:0/2355818306 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7fcc20038370 0x7fcc2003a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:56:31.742 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:30.881+0000 7fcc36000700 1 -- 172.21.15.153:0/2355818306 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fcc2c04c460 con 0x7fcc38106430 2024-03-21T09:56:31.742 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:30.881+0000 7fcc3ca00700 1 --2- 172.21.15.153:0/2355818306 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7fcc20038370 0x7fcc2003a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:56:31.742 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:30.882+0000 7fcc3ca00700 1 --2- 172.21.15.153:0/2355818306 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7fcc20038370 0x7fcc2003a830 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fcc24006fd0 tx=0x7fcc24006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:56:31.742 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:30.886+0000 7fcc36000700 1 -- 172.21.15.153:0/2355818306 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fcc2c01a430 con 0x7fcc38106430 2024-03-21T09:56:31.742 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:31.098+0000 7fcc3fe00700 1 -- 172.21.15.153:0/2355818306 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7fcc3810a860 con 0x7fcc38106430 2024-03-21T09:56:31.742 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:31.106+0000 7fcc36000700 1 -- 172.21.15.153:0/2355818306 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7fcc2c018b40 con 0x7fcc38106430 2024-03-21T09:56:31.742 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:31.109+0000 7fcc3fe00700 1 -- 172.21.15.153:0/2355818306 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7fcc20038370 msgr2=0x7fcc2003a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:56:31.743 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:31.109+0000 7fcc3fe00700 1 --2- 172.21.15.153:0/2355818306 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7fcc20038370 0x7fcc2003a830 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fcc24006fd0 tx=0x7fcc24006e40 comp rx=0 tx=0).stop 2024-03-21T09:56:31.743 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:31.109+0000 7fcc3fe00700 1 -- 172.21.15.153:0/2355818306 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcc38106430 msgr2=0x7fcc3807bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:56:31.743 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:31.109+0000 7fcc3fe00700 1 --2- 172.21.15.153:0/2355818306 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcc38106430 0x7fcc3807bc60 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c011670 tx=0x7fcc2c011b50 comp rx=0 tx=0).stop 2024-03-21T09:56:31.743 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:31.110+0000 7fcc3fe00700 1 -- 172.21.15.153:0/2355818306 shutdown_connections 2024-03-21T09:56:31.743 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:31.110+0000 7fcc3fe00700 1 --2- 172.21.15.153:0/2355818306 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7fcc20038370 0x7fcc2003a830 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:56:31.743 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:31.110+0000 7fcc3fe00700 1 --2- 172.21.15.153:0/2355818306 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcc38106430 0x7fcc3807bc60 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:56:31.743 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:31.110+0000 7fcc3fe00700 1 -- 172.21.15.153:0/2355818306 >> 172.21.15.153:0/2355818306 conn(0x7fcc381019d0 msgr2=0x7fcc381030d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:56:31.743 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:31.110+0000 7fcc3fe00700 1 -- 172.21.15.153:0/2355818306 shutdown_connections 2024-03-21T09:56:31.743 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:31.110+0000 7fcc3fe00700 1 -- 172.21.15.153:0/2355818306 wait complete. 2024-03-21T09:56:32.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:32 smithi153 ceph-mon[29117]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:56:32.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:32 smithi153 ceph-mon[29117]: from='client.? 172.21.15.153:0/2355818306' entity='client.admin' 2024-03-21T09:56:33.264 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:32.364+0000 7f1e92e00700 1 Processor -- start 2024-03-21T09:56:33.264 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:32.364+0000 7f1e92e00700 1 -- start start 2024-03-21T09:56:33.264 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:32.365+0000 7f1e92e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1e8c104d10 0x7f1e8c105130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:56:33.264 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:32.365+0000 7f1e92e00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e8c079e60 con 0x7f1e8c104d10 2024-03-21T09:56:33.264 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:32.365+0000 7f1e8be00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1e8c104d10 0x7f1e8c105130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:56:33.264 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:32.365+0000 7f1e8be00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1e8c104d10 0x7f1e8c105130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:50440/0 (socket says 172.21.15.153:50440) 2024-03-21T09:56:33.265 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:32.365+0000 7f1e8be00700 1 -- 172.21.15.153:0/1280152771 learned_addr learned my addr 172.21.15.153:0/1280152771 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T09:56:33.265 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:32.365+0000 7f1e8be00700 1 -- 172.21.15.153:0/1280152771 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e8c105670 con 0x7f1e8c104d10 2024-03-21T09:56:33.265 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:32.366+0000 7f1e8be00700 1 --2- 172.21.15.153:0/1280152771 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1e8c104d10 0x7f1e8c105130 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c0098a0 tx=0x7f1e7c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeab6a8fb516fc56 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:56:33.265 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:32.366+0000 7f1e8aa00700 1 -- 172.21.15.153:0/1280152771 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1e7c004180 con 0x7f1e8c104d10 2024-03-21T09:56:33.265 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:32.366+0000 7f1e8aa00700 1 -- 172.21.15.153:0/1280152771 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f1e7c0042e0 con 0x7f1e8c104d10 2024-03-21T09:56:33.265 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:32.366+0000 7f1e8aa00700 1 -- 172.21.15.153:0/1280152771 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1e7c011af0 con 0x7f1e8c104d10 2024-03-21T09:56:33.265 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:32.367+0000 7f1e92e00700 1 -- 172.21.15.153:0/1280152771 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1e8c104d10 msgr2=0x7f1e8c105130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:56:33.265 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:32.367+0000 7f1e92e00700 1 --2- 172.21.15.153:0/1280152771 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1e8c104d10 0x7f1e8c105130 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c0098a0 tx=0x7f1e7c009bb0 comp rx=0 tx=0).stop 2024-03-21T09:56:33.265 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:32.367+0000 7f1e92e00700 1 -- 172.21.15.153:0/1280152771 shutdown_connections 2024-03-21T09:56:33.265 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:32.367+0000 7f1e92e00700 1 --2- 172.21.15.153:0/1280152771 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1e8c104d10 0x7f1e8c105130 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:56:33.265 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:32.367+0000 7f1e92e00700 1 -- 172.21.15.153:0/1280152771 >> 172.21.15.153:0/1280152771 conn(0x7f1e8c1008d0 msgr2=0x7f1e8c102d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:56:33.266 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:32.367+0000 7f1e92e00700 1 -- 172.21.15.153:0/1280152771 shutdown_connections 2024-03-21T09:56:33.266 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:32.367+0000 7f1e92e00700 1 -- 172.21.15.153:0/1280152771 wait complete. 2024-03-21T09:56:33.266 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:32.368+0000 7f1e92e00700 1 Processor -- start 2024-03-21T09:56:33.266 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:32.368+0000 7f1e92e00700 1 -- start start 2024-03-21T09:56:33.266 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:32.368+0000 7f1e92e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1e8c104d10 0x7f1e8c19c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:56:33.266 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:32.368+0000 7f1e92e00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e8c19c880 con 0x7f1e8c104d10 2024-03-21T09:56:33.266 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:32.369+0000 7f1e8be00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1e8c104d10 0x7f1e8c19c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:56:33.266 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:32.369+0000 7f1e8be00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1e8c104d10 0x7f1e8c19c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:50446/0 (socket says 172.21.15.153:50446) 2024-03-21T09:56:33.267 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:32.369+0000 7f1e8be00700 1 -- 172.21.15.153:0/3176362283 learned_addr learned my addr 172.21.15.153:0/3176362283 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T09:56:33.267 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:32.369+0000 7f1e8be00700 1 -- 172.21.15.153:0/3176362283 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e7c009550 con 0x7f1e8c104d10 2024-03-21T09:56:33.267 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:32.369+0000 7f1e8be00700 1 --2- 172.21.15.153:0/3176362283 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1e8c104d10 0x7f1e8c19c340 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c011670 tx=0x7f1e7c011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:56:33.267 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:32.370+0000 7f1e88c00700 1 -- 172.21.15.153:0/3176362283 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1e7c003a40 con 0x7f1e8c104d10 2024-03-21T09:56:33.267 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:32.370+0000 7f1e88c00700 1 -- 172.21.15.153:0/3176362283 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f1e7c003ba0 con 0x7f1e8c104d10 2024-03-21T09:56:33.267 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:32.370+0000 7f1e88c00700 1 -- 172.21.15.153:0/3176362283 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1e7c024790 con 0x7f1e8c104d10 2024-03-21T09:56:33.267 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:32.370+0000 7f1e92e00700 1 -- 172.21.15.153:0/3176362283 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e8c19ca80 con 0x7f1e8c104d10 2024-03-21T09:56:33.267 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:32.370+0000 7f1e92e00700 1 -- 172.21.15.153:0/3176362283 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e8c19cf20 con 0x7f1e8c104d10 2024-03-21T09:56:33.267 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:32.371+0000 7f1e88c00700 1 -- 172.21.15.153:0/3176362283 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f1e7c01a430 con 0x7f1e8c104d10 2024-03-21T09:56:33.267 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:32.371+0000 7f1e92e00700 1 -- 172.21.15.153:0/3176362283 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e8c195c20 con 0x7f1e8c104d10 2024-03-21T09:56:33.267 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:32.371+0000 7f1e88c00700 1 --2- 172.21.15.153:0/3176362283 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7f1e70038370 0x7f1e7003a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:56:33.268 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:32.371+0000 7f1e88c00700 1 -- 172.21.15.153:0/3176362283 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f1e7c04c4f0 con 0x7f1e8c104d10 2024-03-21T09:56:33.268 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:32.372+0000 7f1e8b400700 1 --2- 172.21.15.153:0/3176362283 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7f1e70038370 0x7f1e7003a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:56:33.268 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:32.372+0000 7f1e8b400700 1 --2- 172.21.15.153:0/3176362283 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7f1e70038370 0x7f1e7003a830 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f1e78006fd0 tx=0x7f1e78006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:56:33.268 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:32.376+0000 7f1e88c00700 1 -- 172.21.15.153:0/3176362283 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f1e7c01ac50 con 0x7f1e8c104d10 2024-03-21T09:56:33.268 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:32.587+0000 7f1e92e00700 1 -- 172.21.15.153:0/3176362283 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f1e8c02cc70 con 0x7f1e8c104d10 2024-03-21T09:56:33.268 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:32.593+0000 7f1e88c00700 1 -- 172.21.15.153:0/3176362283 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f1e7c018b40 con 0x7f1e8c104d10 2024-03-21T09:56:33.268 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:32.596+0000 7f1e92e00700 1 -- 172.21.15.153:0/3176362283 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7f1e70038370 msgr2=0x7f1e7003a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:56:33.268 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:32.596+0000 7f1e92e00700 1 --2- 172.21.15.153:0/3176362283 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7f1e70038370 0x7f1e7003a830 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f1e78006fd0 tx=0x7f1e78006e40 comp rx=0 tx=0).stop 2024-03-21T09:56:33.268 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:32.596+0000 7f1e92e00700 1 -- 172.21.15.153:0/3176362283 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1e8c104d10 msgr2=0x7f1e8c19c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:56:33.268 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:32.596+0000 7f1e92e00700 1 --2- 172.21.15.153:0/3176362283 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1e8c104d10 0x7f1e8c19c340 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c011670 tx=0x7f1e7c011b50 comp rx=0 tx=0).stop 2024-03-21T09:56:33.268 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:32.596+0000 7f1e92e00700 1 -- 172.21.15.153:0/3176362283 shutdown_connections 2024-03-21T09:56:33.269 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:32.596+0000 7f1e92e00700 1 --2- 172.21.15.153:0/3176362283 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7f1e70038370 0x7f1e7003a830 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:56:33.269 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:32.596+0000 7f1e92e00700 1 --2- 172.21.15.153:0/3176362283 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1e8c104d10 0x7f1e8c19c340 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:56:33.269 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:32.596+0000 7f1e92e00700 1 -- 172.21.15.153:0/3176362283 >> 172.21.15.153:0/3176362283 conn(0x7f1e8c1008d0 msgr2=0x7f1e8c1908d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:56:33.269 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:32.596+0000 7f1e92e00700 1 -- 172.21.15.153:0/3176362283 shutdown_connections 2024-03-21T09:56:33.269 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:32.596+0000 7f1e92e00700 1 -- 172.21.15.153:0/3176362283 wait complete. 2024-03-21T09:56:33.269 INFO:teuthology.orchestra.run.smithi153.stdout:Enabling the dashboard module... 2024-03-21T09:56:33.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:33 smithi153 ceph-mon[29117]: from='client.? 172.21.15.153:0/3176362283' entity='client.admin' 2024-03-21T09:56:33.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:33 smithi153 ceph-mon[29117]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:56:35.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:34 smithi153 ceph-mon[29117]: from='client.? 172.21.15.153:0/6826067' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-03-21T09:56:35.491 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:33.849+0000 7f9137000700 1 Processor -- start 2024-03-21T09:56:35.492 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:33.849+0000 7f9137000700 1 -- start start 2024-03-21T09:56:35.492 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:33.849+0000 7f9137000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f913010e030 0x7f913010e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:56:35.492 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:33.849+0000 7f9137000700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f913010ea20 con 0x7f913010e030 2024-03-21T09:56:35.492 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:33.850+0000 7f912fe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f913010e030 0x7f913010e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:56:35.492 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:33.850+0000 7f912fe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f913010e030 0x7f913010e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:46220/0 (socket says 172.21.15.153:46220) 2024-03-21T09:56:35.492 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:33.850+0000 7f912fe00700 1 -- 172.21.15.153:0/1937551917 learned_addr learned my addr 172.21.15.153:0/1937551917 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T09:56:35.492 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:33.850+0000 7f912fe00700 1 -- 172.21.15.153:0/1937551917 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f913010eb60 con 0x7f913010e030 2024-03-21T09:56:35.492 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:33.851+0000 7f912fe00700 1 --2- 172.21.15.153:0/1937551917 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f913010e030 0x7f913010e450 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f91200098a0 tx=0x7f9120009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69a024857afc93ae server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:56:35.492 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:33.851+0000 7f912ea00700 1 -- 172.21.15.153:0/1937551917 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9120004180 con 0x7f913010e030 2024-03-21T09:56:35.492 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:33.851+0000 7f912ea00700 1 -- 172.21.15.153:0/1937551917 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f91200042e0 con 0x7f913010e030 2024-03-21T09:56:35.493 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:33.851+0000 7f912ea00700 1 -- 172.21.15.153:0/1937551917 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9120011af0 con 0x7f913010e030 2024-03-21T09:56:35.493 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:33.852+0000 7f9137000700 1 -- 172.21.15.153:0/1937551917 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f913010e030 msgr2=0x7f913010e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:56:35.493 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:33.852+0000 7f9137000700 1 --2- 172.21.15.153:0/1937551917 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f913010e030 0x7f913010e450 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f91200098a0 tx=0x7f9120009bb0 comp rx=0 tx=0).stop 2024-03-21T09:56:35.493 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:33.852+0000 7f9137000700 1 -- 172.21.15.153:0/1937551917 shutdown_connections 2024-03-21T09:56:35.493 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:33.852+0000 7f9137000700 1 --2- 172.21.15.153:0/1937551917 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f913010e030 0x7f913010e450 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:56:35.493 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:33.852+0000 7f9137000700 1 -- 172.21.15.153:0/1937551917 >> 172.21.15.153:0/1937551917 conn(0x7f913007bb20 msgr2=0x7f913007df80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:56:35.493 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:33.852+0000 7f9137000700 1 -- 172.21.15.153:0/1937551917 shutdown_connections 2024-03-21T09:56:35.493 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:33.852+0000 7f9137000700 1 -- 172.21.15.153:0/1937551917 wait complete. 2024-03-21T09:56:35.493 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:33.853+0000 7f9137000700 1 Processor -- start 2024-03-21T09:56:35.493 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:33.853+0000 7f9137000700 1 -- start start 2024-03-21T09:56:35.493 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:33.853+0000 7f9137000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f913010e030 0x7f91301a07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:56:35.493 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:33.853+0000 7f9137000700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91301a0d20 con 0x7f913010e030 2024-03-21T09:56:35.493 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:33.854+0000 7f912fe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f913010e030 0x7f91301a07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:56:35.494 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:33.854+0000 7f912fe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f913010e030 0x7f91301a07e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:46226/0 (socket says 172.21.15.153:46226) 2024-03-21T09:56:35.494 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:33.854+0000 7f912fe00700 1 -- 172.21.15.153:0/6826067 learned_addr learned my addr 172.21.15.153:0/6826067 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T09:56:35.494 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:33.854+0000 7f912fe00700 1 -- 172.21.15.153:0/6826067 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9120009550 con 0x7f913010e030 2024-03-21T09:56:35.494 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:33.854+0000 7f912fe00700 1 --2- 172.21.15.153:0/6826067 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f913010e030 0x7f91301a07e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9120011670 tx=0x7f9120011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:56:35.494 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:33.854+0000 7f912cc00700 1 -- 172.21.15.153:0/6826067 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f91200039f0 con 0x7f913010e030 2024-03-21T09:56:35.494 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:33.855+0000 7f912cc00700 1 -- 172.21.15.153:0/6826067 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f9120003b50 con 0x7f913010e030 2024-03-21T09:56:35.494 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:33.855+0000 7f9137000700 1 -- 172.21.15.153:0/6826067 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91301a0f20 con 0x7f913010e030 2024-03-21T09:56:35.494 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:33.855+0000 7f912cc00700 1 -- 172.21.15.153:0/6826067 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9120024690 con 0x7f913010e030 2024-03-21T09:56:35.494 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:33.855+0000 7f9137000700 1 -- 172.21.15.153:0/6826067 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91301a13c0 con 0x7f913010e030 2024-03-21T09:56:35.494 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:33.856+0000 7f912cc00700 1 -- 172.21.15.153:0/6826067 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f912001a430 con 0x7f913010e030 2024-03-21T09:56:35.494 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:33.856+0000 7f9137000700 1 -- 172.21.15.153:0/6826067 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f913019a8b0 con 0x7f913010e030 2024-03-21T09:56:35.494 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:33.856+0000 7f912cc00700 1 --2- 172.21.15.153:0/6826067 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7f9114038370 0x7f911403a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:56:35.494 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:33.856+0000 7f912cc00700 1 -- 172.21.15.153:0/6826067 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f912004c0c0 con 0x7f913010e030 2024-03-21T09:56:35.495 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:33.857+0000 7f912f400700 1 --2- 172.21.15.153:0/6826067 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7f9114038370 0x7f911403a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:56:35.495 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:33.857+0000 7f912f400700 1 --2- 172.21.15.153:0/6826067 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7f9114038370 0x7f911403a830 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f911c006fd0 tx=0x7f911c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:56:35.495 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:33.863+0000 7f912cc00700 1 -- 172.21.15.153:0/6826067 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f912007f0e0 con 0x7f913010e030 2024-03-21T09:56:35.495 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:34.134+0000 7f9137000700 1 -- 172.21.15.153:0/6826067 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7f913010eb60 con 0x7f913010e030 2024-03-21T09:56:35.495 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:34.763+0000 7f912cc00700 1 -- 172.21.15.153:0/6826067 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mgrmap(e 9) v1 ==== 45296+0+0 (secure 0 0 0) 0x7f912002d3f0 con 0x7f913010e030 2024-03-21T09:56:35.495 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:34.766+0000 7f912cc00700 1 -- 172.21.15.153:0/6826067 <== mon.0 v2:172.21.15.153:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7f912002a870 con 0x7f913010e030 2024-03-21T09:56:35.495 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:34.770+0000 7f9137000700 1 -- 172.21.15.153:0/6826067 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7f9114038370 msgr2=0x7f911403a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:56:35.495 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:34.770+0000 7f9137000700 1 --2- 172.21.15.153:0/6826067 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7f9114038370 0x7f911403a830 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f911c006fd0 tx=0x7f911c006e40 comp rx=0 tx=0).stop 2024-03-21T09:56:35.495 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:34.770+0000 7f9137000700 1 -- 172.21.15.153:0/6826067 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f913010e030 msgr2=0x7f91301a07e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:56:35.495 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:34.770+0000 7f9137000700 1 --2- 172.21.15.153:0/6826067 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f913010e030 0x7f91301a07e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9120011670 tx=0x7f9120011b50 comp rx=0 tx=0).stop 2024-03-21T09:56:35.495 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:34.770+0000 7f9137000700 1 -- 172.21.15.153:0/6826067 shutdown_connections 2024-03-21T09:56:35.495 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:34.770+0000 7f9137000700 1 --2- 172.21.15.153:0/6826067 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7f9114038370 0x7f911403a830 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:56:35.495 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:34.770+0000 7f9137000700 1 --2- 172.21.15.153:0/6826067 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f913010e030 0x7f91301a07e0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:56:35.496 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:34.770+0000 7f9137000700 1 -- 172.21.15.153:0/6826067 >> 172.21.15.153:0/6826067 conn(0x7f913007bb20 msgr2=0x7f913007e360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:56:35.496 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:34.770+0000 7f9137000700 1 -- 172.21.15.153:0/6826067 shutdown_connections 2024-03-21T09:56:35.496 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:34.770+0000 7f9137000700 1 -- 172.21.15.153:0/6826067 wait complete. 2024-03-21T09:56:36.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:35 smithi153 ceph-mon[29117]: from='client.? 172.21.15.153:0/6826067' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-03-21T09:56:36.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:35 smithi153 ceph-mon[29117]: mgrmap e9: smithi153.tluxni(active, since 33s) 2024-03-21T09:56:36.926 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:36 smithi153 ceph-mon[29117]: from='client.? 172.21.15.153:0/4033597308' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-21T09:56:37.139 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout { 2024-03-21T09:56:37.140 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-03-21T09:56:37.140 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-21T09:56:37.140 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "active_name": "smithi153.tluxni", 2024-03-21T09:56:37.140 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-21T09:56:37.140 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout } 2024-03-21T09:56:37.140 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:36.078+0000 7f3f28400700 1 Processor -- start 2024-03-21T09:56:37.140 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:36.078+0000 7f3f28400700 1 -- start start 2024-03-21T09:56:37.140 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:36.078+0000 7f3f28400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3f2007ad80 0x7f3f200791e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:56:37.140 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:36.078+0000 7f3f28400700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f200797b0 con 0x7f3f2007ad80 2024-03-21T09:56:37.140 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:36.078+0000 7f3f25a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3f2007ad80 0x7f3f200791e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:56:37.140 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:36.079+0000 7f3f25a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3f2007ad80 0x7f3f200791e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:46240/0 (socket says 172.21.15.153:46240) 2024-03-21T09:56:37.140 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:36.079+0000 7f3f25a00700 1 -- 172.21.15.153:0/2902340512 learned_addr learned my addr 172.21.15.153:0/2902340512 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T09:56:37.141 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:36.079+0000 7f3f25a00700 1 -- 172.21.15.153:0/2902340512 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f200798f0 con 0x7f3f2007ad80 2024-03-21T09:56:37.141 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:36.079+0000 7f3f25a00700 1 --2- 172.21.15.153:0/2902340512 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3f2007ad80 0x7f3f200791e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3f140098a0 tx=0x7f3f14009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d80421620e4a3a65 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:56:37.141 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:36.079+0000 7f3f1fe00700 1 -- 172.21.15.153:0/2902340512 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3f14004180 con 0x7f3f2007ad80 2024-03-21T09:56:37.141 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:36.080+0000 7f3f1fe00700 1 -- 172.21.15.153:0/2902340512 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f3f140042e0 con 0x7f3f2007ad80 2024-03-21T09:56:37.141 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:36.080+0000 7f3f1fe00700 1 -- 172.21.15.153:0/2902340512 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3f14011af0 con 0x7f3f2007ad80 2024-03-21T09:56:37.141 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:36.080+0000 7f3f28400700 1 -- 172.21.15.153:0/2902340512 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3f2007ad80 msgr2=0x7f3f200791e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:56:37.141 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:36.080+0000 7f3f28400700 1 --2- 172.21.15.153:0/2902340512 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3f2007ad80 0x7f3f200791e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3f140098a0 tx=0x7f3f14009bb0 comp rx=0 tx=0).stop 2024-03-21T09:56:37.141 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:36.080+0000 7f3f28400700 1 -- 172.21.15.153:0/2902340512 shutdown_connections 2024-03-21T09:56:37.141 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:36.080+0000 7f3f28400700 1 --2- 172.21.15.153:0/2902340512 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3f2007ad80 0x7f3f200791e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:56:37.141 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:36.080+0000 7f3f28400700 1 -- 172.21.15.153:0/2902340512 >> 172.21.15.153:0/2902340512 conn(0x7f3f201019b0 msgr2=0x7f3f20103e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:56:37.141 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:36.080+0000 7f3f28400700 1 -- 172.21.15.153:0/2902340512 shutdown_connections 2024-03-21T09:56:37.141 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:36.081+0000 7f3f28400700 1 -- 172.21.15.153:0/2902340512 wait complete. 2024-03-21T09:56:37.141 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:36.081+0000 7f3f28400700 1 Processor -- start 2024-03-21T09:56:37.142 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:36.082+0000 7f3f28400700 1 -- start start 2024-03-21T09:56:37.142 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:36.082+0000 7f3f28400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3f2007ad80 0x7f3f2019c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:56:37.142 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:36.082+0000 7f3f28400700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f2019c860 con 0x7f3f2007ad80 2024-03-21T09:56:37.142 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:36.082+0000 7f3f25a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3f2007ad80 0x7f3f2019c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:56:37.142 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:36.082+0000 7f3f25a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3f2007ad80 0x7f3f2019c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:46244/0 (socket says 172.21.15.153:46244) 2024-03-21T09:56:37.142 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:36.082+0000 7f3f25a00700 1 -- 172.21.15.153:0/4033597308 learned_addr learned my addr 172.21.15.153:0/4033597308 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T09:56:37.142 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:36.083+0000 7f3f25a00700 1 -- 172.21.15.153:0/4033597308 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f14009550 con 0x7f3f2007ad80 2024-03-21T09:56:37.142 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:36.083+0000 7f3f25a00700 1 --2- 172.21.15.153:0/4033597308 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3f2007ad80 0x7f3f2019c320 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3f14011670 tx=0x7f3f14011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:56:37.142 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:36.083+0000 7f3f1e000700 1 -- 172.21.15.153:0/4033597308 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3f14003a40 con 0x7f3f2007ad80 2024-03-21T09:56:37.142 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:36.083+0000 7f3f1e000700 1 -- 172.21.15.153:0/4033597308 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f3f14003ba0 con 0x7f3f2007ad80 2024-03-21T09:56:37.142 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:36.083+0000 7f3f28400700 1 -- 172.21.15.153:0/4033597308 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f2019ca60 con 0x7f3f2007ad80 2024-03-21T09:56:37.142 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:36.083+0000 7f3f1e000700 1 -- 172.21.15.153:0/4033597308 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3f14024790 con 0x7f3f2007ad80 2024-03-21T09:56:37.142 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:36.083+0000 7f3f28400700 1 -- 172.21.15.153:0/4033597308 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f2019cf00 con 0x7f3f2007ad80 2024-03-21T09:56:37.143 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:36.084+0000 7f3f1e000700 1 -- 172.21.15.153:0/4033597308 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 9) v1 ==== 45296+0+0 (secure 0 0 0) 0x7f3f1401a430 con 0x7f3f2007ad80 2024-03-21T09:56:37.143 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:36.084+0000 7f3f28400700 1 -- 172.21.15.153:0/4033597308 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f2010c810 con 0x7f3f2007ad80 2024-03-21T09:56:37.143 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:36.085+0000 7f3f1e000700 1 --2- 172.21.15.153:0/4033597308 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7f3f080383c0 0x7f3f0803a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:56:37.143 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:36.085+0000 7f3f1e000700 1 -- 172.21.15.153:0/4033597308 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f3f1404c540 con 0x7f3f2007ad80 2024-03-21T09:56:37.143 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:36.085+0000 7f3f25000700 1 -- 172.21.15.153:0/4033597308 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7f3f080383c0 msgr2=0x7f3f0803a880 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.153:6800/3178125243 2024-03-21T09:56:37.143 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:36.085+0000 7f3f25000700 1 --2- 172.21.15.153:0/4033597308 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7f3f080383c0 0x7f3f0803a880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-21T09:56:37.143 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:36.090+0000 7f3f1e000700 1 -- 172.21.15.153:0/4033597308 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f3f14010380 con 0x7f3f2007ad80 2024-03-21T09:56:37.143 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:36.286+0000 7f3f25000700 1 -- 172.21.15.153:0/4033597308 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7f3f080383c0 msgr2=0x7f3f0803a880 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.153:6800/3178125243 2024-03-21T09:56:37.143 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:36.286+0000 7f3f25000700 1 --2- 172.21.15.153:0/4033597308 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7f3f080383c0 0x7f3f0803a880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-21T09:56:37.143 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:36.356+0000 7f3f28400700 1 -- 172.21.15.153:0/4033597308 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f3f2019fce0 con 0x7f3f2007ad80 2024-03-21T09:56:37.143 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:36.358+0000 7f3f1e000700 1 -- 172.21.15.153:0/4033597308 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+103 (secure 0 0 0) 0x7f3f14018b40 con 0x7f3f2007ad80 2024-03-21T09:56:37.143 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:36.361+0000 7f3f28400700 1 -- 172.21.15.153:0/4033597308 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7f3f080383c0 msgr2=0x7f3f0803a880 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-21T09:56:37.143 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:36.361+0000 7f3f28400700 1 --2- 172.21.15.153:0/4033597308 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7f3f080383c0 0x7f3f0803a880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:56:37.143 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:36.361+0000 7f3f28400700 1 -- 172.21.15.153:0/4033597308 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3f2007ad80 msgr2=0x7f3f2019c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:56:37.144 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:36.361+0000 7f3f28400700 1 --2- 172.21.15.153:0/4033597308 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3f2007ad80 0x7f3f2019c320 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3f14011670 tx=0x7f3f14011b50 comp rx=0 tx=0).stop 2024-03-21T09:56:37.144 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:36.361+0000 7f3f28400700 1 -- 172.21.15.153:0/4033597308 shutdown_connections 2024-03-21T09:56:37.144 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:36.361+0000 7f3f28400700 1 --2- 172.21.15.153:0/4033597308 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7f3f080383c0 0x7f3f0803a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:56:37.144 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:36.361+0000 7f3f28400700 1 --2- 172.21.15.153:0/4033597308 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3f2007ad80 0x7f3f2019c320 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:56:37.144 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:36.361+0000 7f3f28400700 1 -- 172.21.15.153:0/4033597308 >> 172.21.15.153:0/4033597308 conn(0x7f3f201019b0 msgr2=0x7f3f201030c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:56:37.144 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:36.362+0000 7f3f28400700 1 -- 172.21.15.153:0/4033597308 shutdown_connections 2024-03-21T09:56:37.144 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:36.362+0000 7f3f28400700 1 -- 172.21.15.153:0/4033597308 wait complete. 2024-03-21T09:56:37.144 INFO:teuthology.orchestra.run.smithi153.stdout:Waiting for the mgr to restart... 2024-03-21T09:56:37.144 INFO:teuthology.orchestra.run.smithi153.stdout:Waiting for mgr epoch 9... 2024-03-21T09:56:43.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:43 smithi153 ceph-mon[29117]: Active manager daemon smithi153.tluxni restarted 2024-03-21T09:56:43.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:43 smithi153 ceph-mon[29117]: Activating manager daemon smithi153.tluxni 2024-03-21T09:56:43.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:43 smithi153 ceph-mon[29117]: osdmap e3: 0 total, 0 up, 0 in 2024-03-21T09:56:43.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:43 smithi153 ceph-mon[29117]: mgrmap e10: smithi153.tluxni(active, starting, since 0.00619761s) 2024-03-21T09:56:43.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:43 smithi153 ceph-mon[29117]: from='mgr.14164 172.21.15.153:0/3445176977' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mon metadata", "id": "smithi153"}]: dispatch 2024-03-21T09:56:43.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:43 smithi153 ceph-mon[29117]: from='mgr.14164 172.21.15.153:0/3445176977' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mgr metadata", "who": "smithi153.tluxni", "id": "smithi153.tluxni"}]: dispatch 2024-03-21T09:56:43.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:43 smithi153 ceph-mon[29117]: from='mgr.14164 172.21.15.153:0/3445176977' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-21T09:56:43.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:43 smithi153 ceph-mon[29117]: from='mgr.14164 172.21.15.153:0/3445176977' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-21T09:56:43.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:43 smithi153 ceph-mon[29117]: from='mgr.14164 172.21.15.153:0/3445176977' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-21T09:56:43.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:43 smithi153 ceph-mon[29117]: Manager daemon smithi153.tluxni is now available 2024-03-21T09:56:43.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:43 smithi153 ceph-mon[29117]: from='mgr.14164 172.21.15.153:0/3445176977' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:56:43.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:43 smithi153 ceph-mon[29117]: from='mgr.14164 172.21.15.153:0/3445176977' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:56:44.785 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:44 smithi153 ceph-mon[29117]: from='mgr.14164 172.21.15.153:0/3445176977' entity='mgr.smithi153.tluxni' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi153.tluxni/mirror_snapshot_schedule"}]: dispatch 2024-03-21T09:56:44.785 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:44 smithi153 ceph-mon[29117]: from='mgr.14164 172.21.15.153:0/3445176977' entity='mgr.smithi153.tluxni' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi153.tluxni/trash_purge_schedule"}]: dispatch 2024-03-21T09:56:44.785 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:44 smithi153 ceph-mon[29117]: [21/Mar/2024:09:56:43] ENGINE Bus STARTING 2024-03-21T09:56:44.785 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:44 smithi153 ceph-mon[29117]: mgrmap e11: smithi153.tluxni(active, since 1.01193s) 2024-03-21T09:56:45.416 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout { 2024-03-21T09:56:45.416 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-03-21T09:56:45.417 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-21T09:56:45.417 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout } 2024-03-21T09:56:45.417 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:37.717+0000 7f0797000700 1 Processor -- start 2024-03-21T09:56:45.424 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:37.717+0000 7f0797000700 1 -- start start 2024-03-21T09:56:45.424 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:37.718+0000 7f0797000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0790106430 0x7f0790106850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:56:45.424 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:37.718+0000 7f0797000700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0790106e20 con 0x7f0790106430 2024-03-21T09:56:45.424 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:37.718+0000 7f078fe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0790106430 0x7f0790106850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:56:45.424 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:37.718+0000 7f078fe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0790106430 0x7f0790106850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:46252/0 (socket says 172.21.15.153:46252) 2024-03-21T09:56:45.425 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:37.718+0000 7f078fe00700 1 -- 172.21.15.153:0/1080674634 learned_addr learned my addr 172.21.15.153:0/1080674634 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T09:56:45.425 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:37.718+0000 7f078fe00700 1 -- 172.21.15.153:0/1080674634 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0790107630 con 0x7f0790106430 2024-03-21T09:56:45.425 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:37.719+0000 7f078fe00700 1 --2- 172.21.15.153:0/1080674634 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0790106430 0x7f0790106850 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f07800098a0 tx=0x7f0780009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9169bd11248cf0d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:56:45.425 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:37.719+0000 7f078ea00700 1 -- 172.21.15.153:0/1080674634 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0780004180 con 0x7f0790106430 2024-03-21T09:56:45.425 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:37.719+0000 7f078ea00700 1 -- 172.21.15.153:0/1080674634 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f07800042e0 con 0x7f0790106430 2024-03-21T09:56:45.425 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:37.719+0000 7f078ea00700 1 -- 172.21.15.153:0/1080674634 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0780011af0 con 0x7f0790106430 2024-03-21T09:56:45.425 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:37.720+0000 7f0797000700 1 -- 172.21.15.153:0/1080674634 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0790106430 msgr2=0x7f0790106850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:56:45.425 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:37.720+0000 7f0797000700 1 --2- 172.21.15.153:0/1080674634 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0790106430 0x7f0790106850 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f07800098a0 tx=0x7f0780009bb0 comp rx=0 tx=0).stop 2024-03-21T09:56:45.425 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:37.720+0000 7f0797000700 1 -- 172.21.15.153:0/1080674634 shutdown_connections 2024-03-21T09:56:45.425 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:37.720+0000 7f0797000700 1 --2- 172.21.15.153:0/1080674634 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0790106430 0x7f0790106850 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:56:45.425 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:37.720+0000 7f0797000700 1 -- 172.21.15.153:0/1080674634 >> 172.21.15.153:0/1080674634 conn(0x7f07901019b0 msgr2=0x7f0790103e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:56:45.425 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:37.720+0000 7f0797000700 1 -- 172.21.15.153:0/1080674634 shutdown_connections 2024-03-21T09:56:45.425 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:37.720+0000 7f0797000700 1 -- 172.21.15.153:0/1080674634 wait complete. 2024-03-21T09:56:45.425 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:37.721+0000 7f0797000700 1 Processor -- start 2024-03-21T09:56:45.426 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:37.721+0000 7f0797000700 1 -- start start 2024-03-21T09:56:45.426 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:37.721+0000 7f0797000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0790106430 0x7f079019c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:56:45.426 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:37.721+0000 7f0797000700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f079019c8b0 con 0x7f0790106430 2024-03-21T09:56:45.426 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:37.722+0000 7f078fe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0790106430 0x7f079019c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:56:45.426 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:37.722+0000 7f078fe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0790106430 0x7f079019c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:46258/0 (socket says 172.21.15.153:46258) 2024-03-21T09:56:45.426 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:37.722+0000 7f078fe00700 1 -- 172.21.15.153:0/978214006 learned_addr learned my addr 172.21.15.153:0/978214006 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T09:56:45.426 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:37.722+0000 7f078fe00700 1 -- 172.21.15.153:0/978214006 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0780009550 con 0x7f0790106430 2024-03-21T09:56:45.426 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:37.722+0000 7f078fe00700 1 --2- 172.21.15.153:0/978214006 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0790106430 0x7f079019c370 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0780011670 tx=0x7f0780011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:56:45.426 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:37.723+0000 7f078cc00700 1 -- 172.21.15.153:0/978214006 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f07800039f0 con 0x7f0790106430 2024-03-21T09:56:45.426 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:37.723+0000 7f078cc00700 1 -- 172.21.15.153:0/978214006 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0780003b50 con 0x7f0790106430 2024-03-21T09:56:45.426 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:37.723+0000 7f078cc00700 1 -- 172.21.15.153:0/978214006 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f07800247c0 con 0x7f0790106430 2024-03-21T09:56:45.426 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:37.723+0000 7f0797000700 1 -- 172.21.15.153:0/978214006 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f079019cab0 con 0x7f0790106430 2024-03-21T09:56:45.426 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:37.723+0000 7f0797000700 1 -- 172.21.15.153:0/978214006 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f079019cf50 con 0x7f0790106430 2024-03-21T09:56:45.427 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:37.724+0000 7f078cc00700 1 -- 172.21.15.153:0/978214006 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 9) v1 ==== 45296+0+0 (secure 0 0 0) 0x7f078001ac00 con 0x7f0790106430 2024-03-21T09:56:45.427 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:37.724+0000 7f078cc00700 1 --2- 172.21.15.153:0/978214006 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7f0774038410 0x7f077403a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:56:45.427 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:37.724+0000 7f0797000700 1 -- 172.21.15.153:0/978214006 --> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f07900623c0 con 0x7f0774038410 2024-03-21T09:56:45.427 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:37.724+0000 7f078cc00700 1 -- 172.21.15.153:0/978214006 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f078004c450 con 0x7f0790106430 2024-03-21T09:56:45.427 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:37.724+0000 7f078f400700 1 -- 172.21.15.153:0/978214006 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7f0774038410 msgr2=0x7f077403a8d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.153:6800/3178125243 2024-03-21T09:56:45.427 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:37.725+0000 7f078f400700 1 --2- 172.21.15.153:0/978214006 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7f0774038410 0x7f077403a8d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-21T09:56:45.427 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:37.926+0000 7f078f400700 1 -- 172.21.15.153:0/978214006 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7f0774038410 msgr2=0x7f077403a8d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.153:6800/3178125243 2024-03-21T09:56:45.427 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:37.926+0000 7f078f400700 1 --2- 172.21.15.153:0/978214006 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7f0774038410 0x7f077403a8d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-21T09:56:45.427 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:38.327+0000 7f078f400700 1 -- 172.21.15.153:0/978214006 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7f0774038410 msgr2=0x7f077403a8d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.153:6800/3178125243 2024-03-21T09:56:45.427 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:38.327+0000 7f078f400700 1 --2- 172.21.15.153:0/978214006 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7f0774038410 0x7f077403a8d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-21T09:56:45.427 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:39.128+0000 7f078f400700 1 -- 172.21.15.153:0/978214006 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7f0774038410 msgr2=0x7f077403a8d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.153:6800/3178125243 2024-03-21T09:56:45.427 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:39.128+0000 7f078f400700 1 --2- 172.21.15.153:0/978214006 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7f0774038410 0x7f077403a8d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-21T09:56:45.427 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:40.730+0000 7f078f400700 1 -- 172.21.15.153:0/978214006 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7f0774038410 msgr2=0x7f077403a8d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.153:6800/3178125243 2024-03-21T09:56:45.428 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:40.730+0000 7f078f400700 1 --2- 172.21.15.153:0/978214006 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7f0774038410 0x7f077403a8d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-21T09:56:45.428 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:43.484+0000 7f078cc00700 1 -- 172.21.15.153:0/978214006 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mgrmap(e 10) v1 ==== 45063+0+0 (secure 0 0 0) 0x7f078002d770 con 0x7f0790106430 2024-03-21T09:56:45.428 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:43.485+0000 7f078cc00700 1 -- 172.21.15.153:0/978214006 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7f0774038410 msgr2=0x7f077403a8d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-21T09:56:45.428 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:43.485+0000 7f078cc00700 1 --2- 172.21.15.153:0/978214006 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7f0774038410 0x7f077403a8d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:56:45.428 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:44.488+0000 7f078cc00700 1 -- 172.21.15.153:0/978214006 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mgrmap(e 11) v1 ==== 45190+0+0 (secure 0 0 0) 0x7f0780021d40 con 0x7f0790106430 2024-03-21T09:56:45.428 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:44.488+0000 7f078cc00700 1 --2- 172.21.15.153:0/978214006 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7f077403bcd0 0x7f077403e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:56:45.428 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:44.488+0000 7f078cc00700 1 -- 172.21.15.153:0/978214006 --> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f07900623c0 con 0x7f077403bcd0 2024-03-21T09:56:45.428 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:44.490+0000 7f078f400700 1 --2- 172.21.15.153:0/978214006 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7f077403bcd0 0x7f077403e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:56:45.428 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:44.495+0000 7f078f400700 1 --2- 172.21.15.153:0/978214006 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7f077403bcd0 0x7f077403e0c0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f077c003a10 tx=0x7f077c0092b0 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:56:45.428 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:44.497+0000 7f078cc00700 1 -- 172.21.15.153:0/978214006 <== mgr.14164 v2:172.21.15.153:6800/1143522828 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7f07900623c0 con 0x7f077403bcd0 2024-03-21T09:56:45.428 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:44.508+0000 7f0797000700 1 -- 172.21.15.153:0/978214006 --> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f079019d6b0 con 0x7f077403bcd0 2024-03-21T09:56:45.428 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:44.509+0000 7f078cc00700 1 -- 172.21.15.153:0/978214006 <== mgr.14164 v2:172.21.15.153:6800/1143522828 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7f079019d6b0 con 0x7f077403bcd0 2024-03-21T09:56:45.428 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:44.509+0000 7f0797000700 1 -- 172.21.15.153:0/978214006 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7f077403bcd0 msgr2=0x7f077403e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:56:45.429 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:44.509+0000 7f0797000700 1 --2- 172.21.15.153:0/978214006 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7f077403bcd0 0x7f077403e0c0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f077c003a10 tx=0x7f077c0092b0 comp rx=0 tx=0).stop 2024-03-21T09:56:45.429 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:44.509+0000 7f0797000700 1 -- 172.21.15.153:0/978214006 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0790106430 msgr2=0x7f079019c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:56:45.429 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:44.509+0000 7f0797000700 1 --2- 172.21.15.153:0/978214006 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0790106430 0x7f079019c370 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0780011670 tx=0x7f0780011b50 comp rx=0 tx=0).stop 2024-03-21T09:56:45.429 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:44.510+0000 7f0797000700 1 -- 172.21.15.153:0/978214006 shutdown_connections 2024-03-21T09:56:45.429 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:44.510+0000 7f0797000700 1 --2- 172.21.15.153:0/978214006 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7f077403bcd0 0x7f077403e0c0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:56:45.429 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:44.510+0000 7f0797000700 1 --2- 172.21.15.153:0/978214006 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0790106430 0x7f079019c370 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:56:45.429 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:44.510+0000 7f0797000700 1 --2- 172.21.15.153:0/978214006 >> [v2:172.21.15.153:6800/3178125243,v1:172.21.15.153:6801/3178125243] conn(0x7f0774038410 0x7f077403a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:56:45.429 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:44.510+0000 7f0797000700 1 -- 172.21.15.153:0/978214006 >> 172.21.15.153:0/978214006 conn(0x7f07901019b0 msgr2=0x7f0790103080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:56:45.429 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:44.510+0000 7f0797000700 1 -- 172.21.15.153:0/978214006 shutdown_connections 2024-03-21T09:56:45.429 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:44.510+0000 7f0797000700 1 -- 172.21.15.153:0/978214006 wait complete. 2024-03-21T09:56:45.429 INFO:teuthology.orchestra.run.smithi153.stdout:mgr epoch 9 is available 2024-03-21T09:56:45.429 INFO:teuthology.orchestra.run.smithi153.stdout:Generating a dashboard self-signed certificate... 2024-03-21T09:56:45.810 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:45 smithi153 ceph-mon[29117]: [21/Mar/2024:09:56:44] ENGINE Serving on https://172.21.15.153:7150 2024-03-21T09:56:45.810 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:45 smithi153 ceph-mon[29117]: [21/Mar/2024:09:56:44] ENGINE Serving on http://172.21.15.153:8765 2024-03-21T09:56:45.810 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:45 smithi153 ceph-mon[29117]: [21/Mar/2024:09:56:44] ENGINE Bus STARTED 2024-03-21T09:56:45.810 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:45 smithi153 ceph-mon[29117]: from='client.14168 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-21T09:56:45.810 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:45 smithi153 ceph-mon[29117]: from='client.14168 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-21T09:56:46.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:46 smithi153 ceph-mon[29117]: mgrmap e12: smithi153.tluxni(active, since 2s) 2024-03-21T09:56:46.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:46 smithi153 ceph-mon[29117]: from='mgr.14164 172.21.15.153:0/3445176977' entity='mgr.smithi153.tluxni' 2024-03-21T09:56:47.565 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-03-21T09:56:47.565 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:46.440+0000 7fb261c00700 1 Processor -- start 2024-03-21T09:56:47.566 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:46.440+0000 7fb261c00700 1 -- start start 2024-03-21T09:56:47.566 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:46.440+0000 7fb261c00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb25c106eb0 0x7fb25c1092a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:56:47.566 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:46.440+0000 7fb261c00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb25c074680 con 0x7fb25c106eb0 2024-03-21T09:56:47.566 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:46.441+0000 7fb25aa00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb25c106eb0 0x7fb25c1092a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:56:47.566 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:46.441+0000 7fb25aa00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb25c106eb0 0x7fb25c1092a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:47560/0 (socket says 172.21.15.153:47560) 2024-03-21T09:56:47.566 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:46.441+0000 7fb25aa00700 1 -- 172.21.15.153:0/441509345 learned_addr learned my addr 172.21.15.153:0/441509345 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T09:56:47.566 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:46.441+0000 7fb25aa00700 1 -- 172.21.15.153:0/441509345 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb25c1097e0 con 0x7fb25c106eb0 2024-03-21T09:56:47.566 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:46.442+0000 7fb25aa00700 1 --2- 172.21.15.153:0/441509345 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb25c106eb0 0x7fb25c1092a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb24c0098a0 tx=0x7fb24c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=585099fcc61a5b52 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:56:47.566 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:46.442+0000 7fb259600700 1 -- 172.21.15.153:0/441509345 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb24c004180 con 0x7fb25c106eb0 2024-03-21T09:56:47.566 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:46.442+0000 7fb259600700 1 -- 172.21.15.153:0/441509345 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fb24c0042e0 con 0x7fb25c106eb0 2024-03-21T09:56:47.566 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:46.442+0000 7fb259600700 1 -- 172.21.15.153:0/441509345 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb24c011af0 con 0x7fb25c106eb0 2024-03-21T09:56:47.566 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:46.443+0000 7fb261c00700 1 -- 172.21.15.153:0/441509345 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb25c106eb0 msgr2=0x7fb25c1092a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:56:47.567 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:46.443+0000 7fb261c00700 1 --2- 172.21.15.153:0/441509345 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb25c106eb0 0x7fb25c1092a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb24c0098a0 tx=0x7fb24c009bb0 comp rx=0 tx=0).stop 2024-03-21T09:56:47.567 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:46.443+0000 7fb261c00700 1 -- 172.21.15.153:0/441509345 shutdown_connections 2024-03-21T09:56:47.567 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:46.443+0000 7fb261c00700 1 --2- 172.21.15.153:0/441509345 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb25c106eb0 0x7fb25c1092a0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:56:47.567 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:46.443+0000 7fb261c00700 1 -- 172.21.15.153:0/441509345 >> 172.21.15.153:0/441509345 conn(0x7fb25c1008c0 msgr2=0x7fb25c102d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:56:47.567 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:46.443+0000 7fb261c00700 1 -- 172.21.15.153:0/441509345 shutdown_connections 2024-03-21T09:56:47.567 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:46.443+0000 7fb261c00700 1 -- 172.21.15.153:0/441509345 wait complete. 2024-03-21T09:56:47.567 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:46.444+0000 7fb261c00700 1 Processor -- start 2024-03-21T09:56:47.567 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:46.445+0000 7fb261c00700 1 -- start start 2024-03-21T09:56:47.567 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:46.445+0000 7fb261c00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb25c106eb0 0x7fb25c19c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:56:47.568 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:46.445+0000 7fb261c00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb25c19c880 con 0x7fb25c106eb0 2024-03-21T09:56:47.568 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:46.445+0000 7fb25aa00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb25c106eb0 0x7fb25c19c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:56:47.568 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:46.445+0000 7fb25aa00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb25c106eb0 0x7fb25c19c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:47572/0 (socket says 172.21.15.153:47572) 2024-03-21T09:56:47.568 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:46.445+0000 7fb25aa00700 1 -- 172.21.15.153:0/2941263266 learned_addr learned my addr 172.21.15.153:0/2941263266 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T09:56:47.568 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:46.446+0000 7fb25aa00700 1 -- 172.21.15.153:0/2941263266 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb24c009550 con 0x7fb25c106eb0 2024-03-21T09:56:47.568 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:46.446+0000 7fb25aa00700 1 --2- 172.21.15.153:0/2941263266 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb25c106eb0 0x7fb25c19c340 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb24c011670 tx=0x7fb24c011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:56:47.568 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:46.446+0000 7fb253400700 1 -- 172.21.15.153:0/2941263266 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb24c003a40 con 0x7fb25c106eb0 2024-03-21T09:56:47.568 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:46.446+0000 7fb253400700 1 -- 172.21.15.153:0/2941263266 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fb24c003ba0 con 0x7fb25c106eb0 2024-03-21T09:56:47.568 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:46.446+0000 7fb261c00700 1 -- 172.21.15.153:0/2941263266 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb25c19ca80 con 0x7fb25c106eb0 2024-03-21T09:56:47.568 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:46.446+0000 7fb253400700 1 -- 172.21.15.153:0/2941263266 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb24c024790 con 0x7fb25c106eb0 2024-03-21T09:56:47.568 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:46.447+0000 7fb261c00700 1 -- 172.21.15.153:0/2941263266 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb25c19cf20 con 0x7fb25c106eb0 2024-03-21T09:56:47.568 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:46.448+0000 7fb253400700 1 -- 172.21.15.153:0/2941263266 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 12) v1 ==== 45296+0+0 (secure 0 0 0) 0x7fb24c02d3f0 con 0x7fb25c106eb0 2024-03-21T09:56:47.568 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:46.448+0000 7fb261c00700 1 -- 172.21.15.153:0/2941263266 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb25c10a5d0 con 0x7fb25c106eb0 2024-03-21T09:56:47.568 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:46.448+0000 7fb253400700 1 --2- 172.21.15.153:0/2941263266 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7fb2400383c0 0x7fb24003a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:56:47.569 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:46.448+0000 7fb253400700 1 -- 172.21.15.153:0/2941263266 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fb24c04c7a0 con 0x7fb25c106eb0 2024-03-21T09:56:47.569 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:46.449+0000 7fb25a000700 1 --2- 172.21.15.153:0/2941263266 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7fb2400383c0 0x7fb24003a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:56:47.569 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:46.449+0000 7fb25a000700 1 --2- 172.21.15.153:0/2941263266 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7fb2400383c0 0x7fb24003a880 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb248006fd0 tx=0x7fb248006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:56:47.569 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:46.454+0000 7fb253400700 1 -- 172.21.15.153:0/2941263266 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fb24c010c00 con 0x7fb25c106eb0 2024-03-21T09:56:47.569 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:46.764+0000 7fb261c00700 1 -- 172.21.15.153:0/2941263266 --> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7fb25c102bc0 con 0x7fb2400383c0 2024-03-21T09:56:47.569 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:46.859+0000 7fb253400700 1 -- 172.21.15.153:0/2941263266 <== mgr.14164 v2:172.21.15.153:6800/1143522828 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7fb25c102bc0 con 0x7fb2400383c0 2024-03-21T09:56:47.569 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:46.862+0000 7fb261c00700 1 -- 172.21.15.153:0/2941263266 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7fb2400383c0 msgr2=0x7fb24003a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:56:47.569 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:46.862+0000 7fb261c00700 1 --2- 172.21.15.153:0/2941263266 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7fb2400383c0 0x7fb24003a880 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb248006fd0 tx=0x7fb248006e40 comp rx=0 tx=0).stop 2024-03-21T09:56:47.569 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:46.862+0000 7fb261c00700 1 -- 172.21.15.153:0/2941263266 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb25c106eb0 msgr2=0x7fb25c19c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:56:47.569 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:46.862+0000 7fb261c00700 1 --2- 172.21.15.153:0/2941263266 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb25c106eb0 0x7fb25c19c340 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb24c011670 tx=0x7fb24c011b50 comp rx=0 tx=0).stop 2024-03-21T09:56:47.569 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:46.862+0000 7fb261c00700 1 -- 172.21.15.153:0/2941263266 shutdown_connections 2024-03-21T09:56:47.569 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:46.862+0000 7fb261c00700 1 --2- 172.21.15.153:0/2941263266 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7fb2400383c0 0x7fb24003a880 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:56:47.569 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:46.862+0000 7fb261c00700 1 --2- 172.21.15.153:0/2941263266 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb25c106eb0 0x7fb25c19c340 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:56:47.569 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:46.862+0000 7fb261c00700 1 -- 172.21.15.153:0/2941263266 >> 172.21.15.153:0/2941263266 conn(0x7fb25c1008c0 msgr2=0x7fb25c101f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:56:47.569 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:46.862+0000 7fb261c00700 1 -- 172.21.15.153:0/2941263266 shutdown_connections 2024-03-21T09:56:47.570 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:46.862+0000 7fb261c00700 1 -- 172.21.15.153:0/2941263266 wait complete. 2024-03-21T09:56:47.570 INFO:teuthology.orchestra.run.smithi153.stdout:Creating initial admin user... 2024-03-21T09:56:48.155 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:47 smithi153 ceph-mon[29117]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:56:48.155 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:47 smithi153 ceph-mon[29117]: from='mgr.14164 172.21.15.153:0/3445176977' entity='mgr.smithi153.tluxni' 2024-03-21T09:56:48.155 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:47 smithi153 ceph-mon[29117]: from='mgr.14164 172.21.15.153:0/3445176977' entity='mgr.smithi153.tluxni' 2024-03-21T09:56:48.155 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:47 smithi153 ceph-mon[29117]: from='mgr.14164 172.21.15.153:0/3445176977' entity='mgr.smithi153.tluxni' 2024-03-21T09:56:49.759 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:49 smithi153 ceph-mon[29117]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:56:49.759 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:49 smithi153 ceph-mon[29117]: from='mgr.14164 172.21.15.153:0/3445176977' entity='mgr.smithi153.tluxni' 2024-03-21T09:56:50.076 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$xrteGr8hK.lBlXJIVwquoOB29Zd5tEuD0CFcaGH4U31qRwdPXIClG", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1711015008, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-03-21T09:56:50.076 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:48.153+0000 7f737be00700 1 Processor -- start 2024-03-21T09:56:50.076 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:48.153+0000 7f737be00700 1 -- start start 2024-03-21T09:56:50.076 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:48.154+0000 7f737be00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f737c07ada0 0x7f737c079200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:56:50.076 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:48.154+0000 7f737be00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f737c0797d0 con 0x7f737c07ada0 2024-03-21T09:56:50.076 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:48.154+0000 7f737aa00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f737c07ada0 0x7f737c079200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:56:50.076 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:48.154+0000 7f737aa00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f737c07ada0 0x7f737c079200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:47588/0 (socket says 172.21.15.153:47588) 2024-03-21T09:56:50.076 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:48.154+0000 7f737aa00700 1 -- 172.21.15.153:0/2941879470 learned_addr learned my addr 172.21.15.153:0/2941879470 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T09:56:50.076 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:48.155+0000 7f737aa00700 1 -- 172.21.15.153:0/2941879470 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f737c079910 con 0x7f737c07ada0 2024-03-21T09:56:50.076 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:48.155+0000 7f737aa00700 1 --2- 172.21.15.153:0/2941879470 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f737c07ada0 0x7f737c079200 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f736c0098a0 tx=0x7f736c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b88c5af32b3ba31 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:56:50.076 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:48.155+0000 7f737a000700 1 -- 172.21.15.153:0/2941879470 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f736c004180 con 0x7f737c07ada0 2024-03-21T09:56:50.081 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:48.155+0000 7f737a000700 1 -- 172.21.15.153:0/2941879470 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f736c0042e0 con 0x7f737c07ada0 2024-03-21T09:56:50.081 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:48.155+0000 7f737a000700 1 -- 172.21.15.153:0/2941879470 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f736c011af0 con 0x7f737c07ada0 2024-03-21T09:56:50.081 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:48.156+0000 7f737be00700 1 -- 172.21.15.153:0/2941879470 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f737c07ada0 msgr2=0x7f737c079200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:56:50.081 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:48.156+0000 7f737be00700 1 --2- 172.21.15.153:0/2941879470 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f737c07ada0 0x7f737c079200 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f736c0098a0 tx=0x7f736c009bb0 comp rx=0 tx=0).stop 2024-03-21T09:56:50.081 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:48.157+0000 7f737be00700 1 -- 172.21.15.153:0/2941879470 shutdown_connections 2024-03-21T09:56:50.081 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:48.157+0000 7f737be00700 1 --2- 172.21.15.153:0/2941879470 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f737c07ada0 0x7f737c079200 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:56:50.081 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:48.157+0000 7f737be00700 1 -- 172.21.15.153:0/2941879470 >> 172.21.15.153:0/2941879470 conn(0x7f737c101800 msgr2=0x7f737c103c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:56:50.082 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:48.157+0000 7f737be00700 1 -- 172.21.15.153:0/2941879470 shutdown_connections 2024-03-21T09:56:50.082 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:48.157+0000 7f737be00700 1 -- 172.21.15.153:0/2941879470 wait complete. 2024-03-21T09:56:50.082 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:48.158+0000 7f737be00700 1 Processor -- start 2024-03-21T09:56:50.082 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:48.158+0000 7f737be00700 1 -- start start 2024-03-21T09:56:50.082 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:48.158+0000 7f737be00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f737c07ada0 0x7f737c1a05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:56:50.082 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:48.158+0000 7f737be00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f737c1a0b00 con 0x7f737c07ada0 2024-03-21T09:56:50.082 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:48.158+0000 7f737aa00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f737c07ada0 0x7f737c1a05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:56:50.082 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:48.158+0000 7f737aa00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f737c07ada0 0x7f737c1a05c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:47604/0 (socket says 172.21.15.153:47604) 2024-03-21T09:56:50.082 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:48.158+0000 7f737aa00700 1 -- 172.21.15.153:0/1071736791 learned_addr learned my addr 172.21.15.153:0/1071736791 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T09:56:50.082 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:48.159+0000 7f737aa00700 1 -- 172.21.15.153:0/1071736791 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f736c009550 con 0x7f737c07ada0 2024-03-21T09:56:50.082 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:48.159+0000 7f737aa00700 1 --2- 172.21.15.153:0/1071736791 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f737c07ada0 0x7f737c1a05c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f736c011670 tx=0x7f736c011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:56:50.082 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:48.159+0000 7f7373400700 1 -- 172.21.15.153:0/1071736791 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f736c0039f0 con 0x7f737c07ada0 2024-03-21T09:56:50.082 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:48.159+0000 7f7373400700 1 -- 172.21.15.153:0/1071736791 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f736c003b50 con 0x7f737c07ada0 2024-03-21T09:56:50.083 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:48.159+0000 7f737be00700 1 -- 172.21.15.153:0/1071736791 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f737c1a0d00 con 0x7f737c07ada0 2024-03-21T09:56:50.083 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:48.159+0000 7f7373400700 1 -- 172.21.15.153:0/1071736791 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f736c0247c0 con 0x7f737c07ada0 2024-03-21T09:56:50.083 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:48.160+0000 7f737be00700 1 -- 172.21.15.153:0/1071736791 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f737c1a11a0 con 0x7f737c07ada0 2024-03-21T09:56:50.083 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:48.161+0000 7f737be00700 1 -- 172.21.15.153:0/1071736791 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f737c108210 con 0x7f737c07ada0 2024-03-21T09:56:50.083 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:48.161+0000 7f7373400700 1 -- 172.21.15.153:0/1071736791 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 12) v1 ==== 45296+0+0 (secure 0 0 0) 0x7f736c01ac00 con 0x7f737c07ada0 2024-03-21T09:56:50.083 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:48.162+0000 7f7373400700 1 --2- 172.21.15.153:0/1071736791 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7f7360038370 0x7f736003a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:56:50.083 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:48.162+0000 7f7373400700 1 -- 172.21.15.153:0/1071736791 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f736c04c5d0 con 0x7f737c07ada0 2024-03-21T09:56:50.083 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:48.162+0000 7f7373e00700 1 --2- 172.21.15.153:0/1071736791 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7f7360038370 0x7f736003a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:56:50.083 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:48.163+0000 7f7373e00700 1 --2- 172.21.15.153:0/1071736791 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7f7360038370 0x7f736003a830 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7364006fd0 tx=0x7f7364006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:56:50.083 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:48.171+0000 7f7373400700 1 -- 172.21.15.153:0/1071736791 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f736c01a430 con 0x7f737c07ada0 2024-03-21T09:56:50.083 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:48.434+0000 7f737be00700 1 -- 172.21.15.153:0/1071736791 --> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f737c0621e0 con 0x7f7360038370 2024-03-21T09:56:50.083 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:48.723+0000 7f7373400700 1 -- 172.21.15.153:0/1071736791 <== mgr.14164 v2:172.21.15.153:6800/1143522828 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f737c0621e0 con 0x7f7360038370 2024-03-21T09:56:50.083 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:48.727+0000 7f737be00700 1 -- 172.21.15.153:0/1071736791 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7f7360038370 msgr2=0x7f736003a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:56:50.083 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:48.727+0000 7f737be00700 1 --2- 172.21.15.153:0/1071736791 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7f7360038370 0x7f736003a830 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7364006fd0 tx=0x7f7364006e40 comp rx=0 tx=0).stop 2024-03-21T09:56:50.084 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:48.727+0000 7f737be00700 1 -- 172.21.15.153:0/1071736791 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f737c07ada0 msgr2=0x7f737c1a05c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:56:50.084 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:48.727+0000 7f737be00700 1 --2- 172.21.15.153:0/1071736791 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f737c07ada0 0x7f737c1a05c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f736c011670 tx=0x7f736c011b50 comp rx=0 tx=0).stop 2024-03-21T09:56:50.084 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:48.727+0000 7f737be00700 1 -- 172.21.15.153:0/1071736791 shutdown_connections 2024-03-21T09:56:50.084 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:48.727+0000 7f737be00700 1 --2- 172.21.15.153:0/1071736791 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7f7360038370 0x7f736003a830 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:56:50.084 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:48.727+0000 7f737be00700 1 --2- 172.21.15.153:0/1071736791 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f737c07ada0 0x7f737c1a05c0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:56:50.084 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:48.727+0000 7f737be00700 1 -- 172.21.15.153:0/1071736791 >> 172.21.15.153:0/1071736791 conn(0x7f737c101800 msgr2=0x7f737c107500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:56:50.084 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:48.727+0000 7f737be00700 1 -- 172.21.15.153:0/1071736791 shutdown_connections 2024-03-21T09:56:50.084 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:48.727+0000 7f737be00700 1 -- 172.21.15.153:0/1071736791 wait complete. 2024-03-21T09:56:50.084 INFO:teuthology.orchestra.run.smithi153.stdout:Fetching dashboard port number... 2024-03-21T09:56:50.960 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:50 smithi153 ceph-mon[29117]: mgrmap e13: smithi153.tluxni(active, since 6s) 2024-03-21T09:56:51.609 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stdout 8443 2024-03-21T09:56:51.610 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:50.661+0000 7f54f1800700 1 Processor -- start 2024-03-21T09:56:51.610 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:50.662+0000 7f54f1800700 1 -- start start 2024-03-21T09:56:51.610 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:50.662+0000 7f54f1800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f54ec106f40 0x7f54ec109330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:56:51.610 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:50.662+0000 7f54f1800700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54ec079e40 con 0x7f54ec106f40 2024-03-21T09:56:51.611 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:50.662+0000 7f54eaa00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f54ec106f40 0x7f54ec109330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:56:51.611 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:50.663+0000 7f54eaa00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f54ec106f40 0x7f54ec109330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:47614/0 (socket says 172.21.15.153:47614) 2024-03-21T09:56:51.612 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:50.663+0000 7f54eaa00700 1 -- 172.21.15.153:0/1687226468 learned_addr learned my addr 172.21.15.153:0/1687226468 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T09:56:51.612 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:50.663+0000 7f54eaa00700 1 -- 172.21.15.153:0/1687226468 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54ec109870 con 0x7f54ec106f40 2024-03-21T09:56:51.612 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:50.663+0000 7f54eaa00700 1 --2- 172.21.15.153:0/1687226468 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f54ec106f40 0x7f54ec109330 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f54dc0098a0 tx=0x7f54dc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf06fced4b75764a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:56:51.612 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:50.664+0000 7f54ea000700 1 -- 172.21.15.153:0/1687226468 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f54dc004180 con 0x7f54ec106f40 2024-03-21T09:56:51.612 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:50.664+0000 7f54ea000700 1 -- 172.21.15.153:0/1687226468 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f54dc0042e0 con 0x7f54ec106f40 2024-03-21T09:56:51.612 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:50.664+0000 7f54ea000700 1 -- 172.21.15.153:0/1687226468 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f54dc011af0 con 0x7f54ec106f40 2024-03-21T09:56:51.612 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:50.665+0000 7f54f1800700 1 -- 172.21.15.153:0/1687226468 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f54ec106f40 msgr2=0x7f54ec109330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:56:51.612 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:50.665+0000 7f54f1800700 1 --2- 172.21.15.153:0/1687226468 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f54ec106f40 0x7f54ec109330 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f54dc0098a0 tx=0x7f54dc009bb0 comp rx=0 tx=0).stop 2024-03-21T09:56:51.612 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:50.665+0000 7f54f1800700 1 -- 172.21.15.153:0/1687226468 shutdown_connections 2024-03-21T09:56:51.616 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:50.665+0000 7f54f1800700 1 --2- 172.21.15.153:0/1687226468 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f54ec106f40 0x7f54ec109330 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:56:51.616 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:50.665+0000 7f54f1800700 1 -- 172.21.15.153:0/1687226468 >> 172.21.15.153:0/1687226468 conn(0x7f54ec1008d0 msgr2=0x7f54ec102d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:56:51.616 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:50.665+0000 7f54f1800700 1 -- 172.21.15.153:0/1687226468 shutdown_connections 2024-03-21T09:56:51.616 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:50.665+0000 7f54f1800700 1 -- 172.21.15.153:0/1687226468 wait complete. 2024-03-21T09:56:51.616 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:50.666+0000 7f54f1800700 1 Processor -- start 2024-03-21T09:56:51.616 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:50.666+0000 7f54f1800700 1 -- start start 2024-03-21T09:56:51.616 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:50.667+0000 7f54f1800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f54ec106f40 0x7f54ec1068a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:56:51.617 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:50.667+0000 7f54f1800700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54ec106de0 con 0x7f54ec106f40 2024-03-21T09:56:51.617 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:50.667+0000 7f54eaa00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f54ec106f40 0x7f54ec1068a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:56:51.617 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:50.667+0000 7f54eaa00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f54ec106f40 0x7f54ec1068a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:47630/0 (socket says 172.21.15.153:47630) 2024-03-21T09:56:51.617 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:50.667+0000 7f54eaa00700 1 -- 172.21.15.153:0/2832697686 learned_addr learned my addr 172.21.15.153:0/2832697686 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T09:56:51.617 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:50.667+0000 7f54eaa00700 1 -- 172.21.15.153:0/2832697686 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54dc009550 con 0x7f54ec106f40 2024-03-21T09:56:51.617 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:50.668+0000 7f54eaa00700 1 --2- 172.21.15.153:0/2832697686 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f54ec106f40 0x7f54ec1068a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f54dc011670 tx=0x7f54dc011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:56:51.617 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:50.668+0000 7f54e3e00700 1 -- 172.21.15.153:0/2832697686 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f54dc003a40 con 0x7f54ec106f40 2024-03-21T09:56:51.617 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:50.668+0000 7f54e3e00700 1 -- 172.21.15.153:0/2832697686 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f54dc003ba0 con 0x7f54ec106f40 2024-03-21T09:56:51.617 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:50.668+0000 7f54f1800700 1 -- 172.21.15.153:0/2832697686 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54ec104ef0 con 0x7f54ec106f40 2024-03-21T09:56:51.617 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:50.668+0000 7f54e3e00700 1 -- 172.21.15.153:0/2832697686 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f54dc024790 con 0x7f54ec106f40 2024-03-21T09:56:51.617 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:50.668+0000 7f54f1800700 1 -- 172.21.15.153:0/2832697686 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54ec105390 con 0x7f54ec106f40 2024-03-21T09:56:51.617 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:50.669+0000 7f54f1800700 1 -- 172.21.15.153:0/2832697686 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54ec10a5b0 con 0x7f54ec106f40 2024-03-21T09:56:51.617 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:50.669+0000 7f54e3e00700 1 -- 172.21.15.153:0/2832697686 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 13) v1 ==== 45340+0+0 (secure 0 0 0) 0x7f54dc01a430 con 0x7f54ec106f40 2024-03-21T09:56:51.617 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:50.670+0000 7f54e3e00700 1 --2- 172.21.15.153:0/2832697686 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7f54d0038460 0x7f54d003a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:56:51.618 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:50.670+0000 7f54e3e00700 1 -- 172.21.15.153:0/2832697686 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f54dc04c700 con 0x7f54ec106f40 2024-03-21T09:56:51.618 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:50.670+0000 7f54e2000700 1 --2- 172.21.15.153:0/2832697686 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7f54d0038460 0x7f54d003a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:56:51.618 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:50.671+0000 7f54e2000700 1 --2- 172.21.15.153:0/2832697686 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7f54d0038460 0x7f54d003a920 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f54d4006fd0 tx=0x7f54d4006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:56:51.618 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:50.675+0000 7f54e3e00700 1 -- 172.21.15.153:0/2832697686 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f54dc018b40 con 0x7f54ec106f40 2024-03-21T09:56:51.618 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:50.888+0000 7f54f1800700 1 -- 172.21.15.153:0/2832697686 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7f54ec10a7d0 con 0x7f54ec106f40 2024-03-21T09:56:51.618 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:50.888+0000 7f54e3e00700 1 -- 172.21.15.153:0/2832697686 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7f54dc018b40 con 0x7f54ec106f40 2024-03-21T09:56:51.618 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:50.891+0000 7f54f1800700 1 -- 172.21.15.153:0/2832697686 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7f54d0038460 msgr2=0x7f54d003a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:56:51.618 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:50.891+0000 7f54f1800700 1 --2- 172.21.15.153:0/2832697686 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7f54d0038460 0x7f54d003a920 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f54d4006fd0 tx=0x7f54d4006e40 comp rx=0 tx=0).stop 2024-03-21T09:56:51.618 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:50.891+0000 7f54f1800700 1 -- 172.21.15.153:0/2832697686 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f54ec106f40 msgr2=0x7f54ec1068a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:56:51.618 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:50.891+0000 7f54f1800700 1 --2- 172.21.15.153:0/2832697686 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f54ec106f40 0x7f54ec1068a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f54dc011670 tx=0x7f54dc011b50 comp rx=0 tx=0).stop 2024-03-21T09:56:51.618 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:50.891+0000 7f54f1800700 1 -- 172.21.15.153:0/2832697686 shutdown_connections 2024-03-21T09:56:51.618 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:50.891+0000 7f54f1800700 1 --2- 172.21.15.153:0/2832697686 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7f54d0038460 0x7f54d003a920 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:56:51.618 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:50.891+0000 7f54f1800700 1 --2- 172.21.15.153:0/2832697686 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f54ec106f40 0x7f54ec1068a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:56:51.619 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:50.891+0000 7f54f1800700 1 -- 172.21.15.153:0/2832697686 >> 172.21.15.153:0/2832697686 conn(0x7f54ec1008d0 msgr2=0x7f54ec1016b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:56:51.619 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:50.891+0000 7f54f1800700 1 -- 172.21.15.153:0/2832697686 shutdown_connections 2024-03-21T09:56:51.619 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:50.891+0000 7f54f1800700 1 -- 172.21.15.153:0/2832697686 wait complete. 2024-03-21T09:56:51.652 INFO:teuthology.orchestra.run.smithi153.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-21T09:56:51.652 INFO:teuthology.orchestra.run.smithi153.stdout:systemctl: stdout disabled 2024-03-21T09:56:51.677 INFO:teuthology.orchestra.run.smithi153.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-21T09:56:51.677 INFO:teuthology.orchestra.run.smithi153.stdout:systemctl: stdout inactive 2024-03-21T09:56:51.677 INFO:teuthology.orchestra.run.smithi153.stdout:firewalld.service is not enabled 2024-03-21T09:56:51.678 INFO:teuthology.orchestra.run.smithi153.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-03-21T09:56:51.685 INFO:teuthology.orchestra.run.smithi153.stdout:Ceph Dashboard is now available at: 2024-03-21T09:56:51.685 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T09:56:51.685 INFO:teuthology.orchestra.run.smithi153.stdout: URL: https://smithi153.front.sepia.ceph.com:8443/ 2024-03-21T09:56:51.685 INFO:teuthology.orchestra.run.smithi153.stdout: User: admin 2024-03-21T09:56:51.685 INFO:teuthology.orchestra.run.smithi153.stdout: Password: jwx9bw6tfy 2024-03-21T09:56:51.685 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T09:56:51.686 INFO:teuthology.orchestra.run.smithi153.stdout:Saving cluster configuration to /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/config directory 2024-03-21T09:56:51.688 INFO:teuthology.orchestra.run.smithi153.stdout:Enabling autotune for osd_memory_target 2024-03-21T09:56:51.845 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:51 smithi153 ceph-mon[29117]: from='client.? 172.21.15.153:0/2832697686' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-03-21T09:56:53.223 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:52.337+0000 7fa9c4c00700 1 Processor -- start 2024-03-21T09:56:53.223 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:52.337+0000 7fa9c4c00700 1 -- start start 2024-03-21T09:56:53.223 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:52.337+0000 7fa9c4c00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa9c0104d00 0x7fa9c0105120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:56:53.223 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:52.337+0000 7fa9c4c00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9c0074620 con 0x7fa9c0104d00 2024-03-21T09:56:53.223 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:52.338+0000 7fa9bde00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa9c0104d00 0x7fa9c0105120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:56:53.223 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:52.338+0000 7fa9bde00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa9c0104d00 0x7fa9c0105120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:47640/0 (socket says 172.21.15.153:47640) 2024-03-21T09:56:53.223 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:52.338+0000 7fa9bde00700 1 -- 172.21.15.153:0/1545434063 learned_addr learned my addr 172.21.15.153:0/1545434063 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T09:56:53.223 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:52.338+0000 7fa9bde00700 1 -- 172.21.15.153:0/1545434063 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9c0105660 con 0x7fa9c0104d00 2024-03-21T09:56:53.224 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:52.339+0000 7fa9bde00700 1 --2- 172.21.15.153:0/1545434063 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa9c0104d00 0x7fa9c0105120 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa9b00098a0 tx=0x7fa9b0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6c179931526ca5 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:56:53.224 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:52.339+0000 7fa9bca00700 1 -- 172.21.15.153:0/1545434063 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa9b0004180 con 0x7fa9c0104d00 2024-03-21T09:56:53.224 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:52.339+0000 7fa9bca00700 1 -- 172.21.15.153:0/1545434063 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa9b00042e0 con 0x7fa9c0104d00 2024-03-21T09:56:53.224 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:52.339+0000 7fa9bca00700 1 -- 172.21.15.153:0/1545434063 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa9b0011af0 con 0x7fa9c0104d00 2024-03-21T09:56:53.224 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:52.340+0000 7fa9c4c00700 1 -- 172.21.15.153:0/1545434063 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa9c0104d00 msgr2=0x7fa9c0105120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:56:53.224 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:52.340+0000 7fa9c4c00700 1 --2- 172.21.15.153:0/1545434063 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa9c0104d00 0x7fa9c0105120 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa9b00098a0 tx=0x7fa9b0009bb0 comp rx=0 tx=0).stop 2024-03-21T09:56:53.224 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:52.340+0000 7fa9c4c00700 1 -- 172.21.15.153:0/1545434063 shutdown_connections 2024-03-21T09:56:53.224 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:52.340+0000 7fa9c4c00700 1 --2- 172.21.15.153:0/1545434063 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa9c0104d00 0x7fa9c0105120 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:56:53.224 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:52.340+0000 7fa9c4c00700 1 -- 172.21.15.153:0/1545434063 >> 172.21.15.153:0/1545434063 conn(0x7fa9c01008c0 msgr2=0x7fa9c0102d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:56:53.224 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:52.340+0000 7fa9c4c00700 1 -- 172.21.15.153:0/1545434063 shutdown_connections 2024-03-21T09:56:53.224 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:52.340+0000 7fa9c4c00700 1 -- 172.21.15.153:0/1545434063 wait complete. 2024-03-21T09:56:53.224 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:52.341+0000 7fa9c4c00700 1 Processor -- start 2024-03-21T09:56:53.224 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:52.341+0000 7fa9c4c00700 1 -- start start 2024-03-21T09:56:53.225 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:52.342+0000 7fa9c4c00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa9c0104d00 0x7fa9c019c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:56:53.225 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:52.342+0000 7fa9c4c00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9c019c8a0 con 0x7fa9c0104d00 2024-03-21T09:56:53.225 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:52.342+0000 7fa9bde00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa9c0104d00 0x7fa9c019c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:56:53.225 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:52.342+0000 7fa9bde00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa9c0104d00 0x7fa9c019c360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:47644/0 (socket says 172.21.15.153:47644) 2024-03-21T09:56:53.225 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:52.342+0000 7fa9bde00700 1 -- 172.21.15.153:0/1467457394 learned_addr learned my addr 172.21.15.153:0/1467457394 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T09:56:53.225 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:52.342+0000 7fa9bde00700 1 -- 172.21.15.153:0/1467457394 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9b0009550 con 0x7fa9c0104d00 2024-03-21T09:56:53.225 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:52.343+0000 7fa9bde00700 1 --2- 172.21.15.153:0/1467457394 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa9c0104d00 0x7fa9c019c360 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa9b0011670 tx=0x7fa9b0011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:56:53.225 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:52.343+0000 7fa9b6a00700 1 -- 172.21.15.153:0/1467457394 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa9b0003a40 con 0x7fa9c0104d00 2024-03-21T09:56:53.225 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:52.343+0000 7fa9b6a00700 1 -- 172.21.15.153:0/1467457394 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa9b0003ba0 con 0x7fa9c0104d00 2024-03-21T09:56:53.225 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:52.343+0000 7fa9c4c00700 1 -- 172.21.15.153:0/1467457394 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9c019caa0 con 0x7fa9c0104d00 2024-03-21T09:56:53.225 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:52.343+0000 7fa9b6a00700 1 -- 172.21.15.153:0/1467457394 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa9b0024790 con 0x7fa9c0104d00 2024-03-21T09:56:53.225 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:52.343+0000 7fa9c4c00700 1 -- 172.21.15.153:0/1467457394 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9c019cf40 con 0x7fa9c0104d00 2024-03-21T09:56:53.225 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:52.344+0000 7fa9b6a00700 1 -- 172.21.15.153:0/1467457394 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 13) v1 ==== 45340+0+0 (secure 0 0 0) 0x7fa9b001a430 con 0x7fa9c0104d00 2024-03-21T09:56:53.226 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:52.344+0000 7fa9c4c00700 1 -- 172.21.15.153:0/1467457394 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9c0195c10 con 0x7fa9c0104d00 2024-03-21T09:56:53.226 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:52.344+0000 7fa9b6a00700 1 --2- 172.21.15.153:0/1467457394 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7fa9a4038410 0x7fa9a403a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:56:53.226 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:52.345+0000 7fa9b6a00700 1 -- 172.21.15.153:0/1467457394 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa9b004c800 con 0x7fa9c0104d00 2024-03-21T09:56:53.226 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:52.345+0000 7fa9bd400700 1 --2- 172.21.15.153:0/1467457394 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7fa9a4038410 0x7fa9a403a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:56:53.226 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:52.345+0000 7fa9bd400700 1 --2- 172.21.15.153:0/1467457394 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7fa9a4038410 0x7fa9a403a8d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac006fd0 tx=0x7fa9ac006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:56:53.226 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:52.350+0000 7fa9b6a00700 1 -- 172.21.15.153:0/1467457394 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fa9b0010380 con 0x7fa9c0104d00 2024-03-21T09:56:53.226 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:52.563+0000 7fa9c4c00700 1 -- 172.21.15.153:0/1467457394 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7fa9c002cc70 con 0x7fa9c0104d00 2024-03-21T09:56:53.226 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:52.564+0000 7fa9b6a00700 1 -- 172.21.15.153:0/1467457394 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7fa9b0018b40 con 0x7fa9c0104d00 2024-03-21T09:56:53.227 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:52.566+0000 7fa9c4c00700 1 -- 172.21.15.153:0/1467457394 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7fa9a4038410 msgr2=0x7fa9a403a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:56:53.227 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:52.566+0000 7fa9c4c00700 1 --2- 172.21.15.153:0/1467457394 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7fa9a4038410 0x7fa9a403a8d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac006fd0 tx=0x7fa9ac006e40 comp rx=0 tx=0).stop 2024-03-21T09:56:53.227 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:52.566+0000 7fa9c4c00700 1 -- 172.21.15.153:0/1467457394 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa9c0104d00 msgr2=0x7fa9c019c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:56:53.227 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:52.566+0000 7fa9c4c00700 1 --2- 172.21.15.153:0/1467457394 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa9c0104d00 0x7fa9c019c360 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa9b0011670 tx=0x7fa9b0011b50 comp rx=0 tx=0).stop 2024-03-21T09:56:53.228 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:52.566+0000 7fa9c4c00700 1 -- 172.21.15.153:0/1467457394 shutdown_connections 2024-03-21T09:56:53.228 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:52.566+0000 7fa9c4c00700 1 --2- 172.21.15.153:0/1467457394 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7fa9a4038410 0x7fa9a403a8d0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:56:53.229 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:52.566+0000 7fa9c4c00700 1 --2- 172.21.15.153:0/1467457394 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa9c0104d00 0x7fa9c019c360 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:56:53.229 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:52.566+0000 7fa9c4c00700 1 -- 172.21.15.153:0/1467457394 >> 172.21.15.153:0/1467457394 conn(0x7fa9c01008c0 msgr2=0x7fa9c01908c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:56:53.229 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:52.567+0000 7fa9c4c00700 1 -- 172.21.15.153:0/1467457394 shutdown_connections 2024-03-21T09:56:53.229 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:52.567+0000 7fa9c4c00700 1 -- 172.21.15.153:0/1467457394 wait complete. 2024-03-21T09:56:54.841 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:53.832+0000 7fa307e00700 1 Processor -- start 2024-03-21T09:56:54.841 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:53.832+0000 7fa307e00700 1 -- start start 2024-03-21T09:56:54.841 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:53.833+0000 7fa307e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa30007adc0 0x7fa300079220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:56:54.841 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:53.833+0000 7fa307e00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3000797f0 con 0x7fa30007adc0 2024-03-21T09:56:54.842 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:53.833+0000 7fa305400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa30007adc0 0x7fa300079220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:56:54.842 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:53.833+0000 7fa305400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa30007adc0 0x7fa300079220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:60812/0 (socket says 172.21.15.153:60812) 2024-03-21T09:56:54.842 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:53.833+0000 7fa305400700 1 -- 172.21.15.153:0/3242720948 learned_addr learned my addr 172.21.15.153:0/3242720948 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T09:56:54.842 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:53.834+0000 7fa305400700 1 -- 172.21.15.153:0/3242720948 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa300079930 con 0x7fa30007adc0 2024-03-21T09:56:54.842 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:53.834+0000 7fa305400700 1 --2- 172.21.15.153:0/3242720948 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa30007adc0 0x7fa300079220 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa2f40098a0 tx=0x7fa2f4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f158fb484c528628 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:56:54.842 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:53.834+0000 7fa2ffe00700 1 -- 172.21.15.153:0/3242720948 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa2f4004180 con 0x7fa30007adc0 2024-03-21T09:56:54.842 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:53.835+0000 7fa2ffe00700 1 -- 172.21.15.153:0/3242720948 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa2f40042e0 con 0x7fa30007adc0 2024-03-21T09:56:54.842 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:53.835+0000 7fa2ffe00700 1 -- 172.21.15.153:0/3242720948 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa2f4011af0 con 0x7fa30007adc0 2024-03-21T09:56:54.842 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:53.835+0000 7fa307e00700 1 -- 172.21.15.153:0/3242720948 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa30007adc0 msgr2=0x7fa300079220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:56:54.842 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:53.835+0000 7fa307e00700 1 --2- 172.21.15.153:0/3242720948 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa30007adc0 0x7fa300079220 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa2f40098a0 tx=0x7fa2f4009bb0 comp rx=0 tx=0).stop 2024-03-21T09:56:54.842 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:53.835+0000 7fa307e00700 1 -- 172.21.15.153:0/3242720948 shutdown_connections 2024-03-21T09:56:54.842 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:53.835+0000 7fa307e00700 1 --2- 172.21.15.153:0/3242720948 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa30007adc0 0x7fa300079220 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:56:54.842 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:53.835+0000 7fa307e00700 1 -- 172.21.15.153:0/3242720948 >> 172.21.15.153:0/3242720948 conn(0x7fa3001019b0 msgr2=0x7fa300103e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:56:54.843 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:53.835+0000 7fa307e00700 1 -- 172.21.15.153:0/3242720948 shutdown_connections 2024-03-21T09:56:54.843 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:53.835+0000 7fa307e00700 1 -- 172.21.15.153:0/3242720948 wait complete. 2024-03-21T09:56:54.843 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:53.836+0000 7fa307e00700 1 Processor -- start 2024-03-21T09:56:54.843 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:53.837+0000 7fa307e00700 1 -- start start 2024-03-21T09:56:54.843 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:53.837+0000 7fa307e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa30007adc0 0x7fa30019c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:56:54.843 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:53.837+0000 7fa307e00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa30019c890 con 0x7fa30007adc0 2024-03-21T09:56:54.843 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:53.837+0000 7fa305400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa30007adc0 0x7fa30019c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:56:54.843 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:53.837+0000 7fa305400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa30007adc0 0x7fa30019c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:60816/0 (socket says 172.21.15.153:60816) 2024-03-21T09:56:54.843 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:53.837+0000 7fa305400700 1 -- 172.21.15.153:0/790367946 learned_addr learned my addr 172.21.15.153:0/790367946 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T09:56:54.843 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:53.838+0000 7fa305400700 1 -- 172.21.15.153:0/790367946 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2f4009550 con 0x7fa30007adc0 2024-03-21T09:56:54.843 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:53.838+0000 7fa305400700 1 --2- 172.21.15.153:0/790367946 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa30007adc0 0x7fa30019c350 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa2f4011670 tx=0x7fa2f4011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:56:54.843 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:53.838+0000 7fa2fe000700 1 -- 172.21.15.153:0/790367946 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa2f40039f0 con 0x7fa30007adc0 2024-03-21T09:56:54.843 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:53.838+0000 7fa2fe000700 1 -- 172.21.15.153:0/790367946 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa2f4003b50 con 0x7fa30007adc0 2024-03-21T09:56:54.843 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:53.838+0000 7fa307e00700 1 -- 172.21.15.153:0/790367946 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa30019ca90 con 0x7fa30007adc0 2024-03-21T09:56:54.844 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:53.838+0000 7fa2fe000700 1 -- 172.21.15.153:0/790367946 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa2f40247c0 con 0x7fa30007adc0 2024-03-21T09:56:54.844 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:53.838+0000 7fa307e00700 1 -- 172.21.15.153:0/790367946 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa30019cf30 con 0x7fa30007adc0 2024-03-21T09:56:54.844 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:53.839+0000 7fa2fe000700 1 -- 172.21.15.153:0/790367946 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 13) v1 ==== 45340+0+0 (secure 0 0 0) 0x7fa2f401a430 con 0x7fa30007adc0 2024-03-21T09:56:54.844 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:53.840+0000 7fa307e00700 1 -- 172.21.15.153:0/790367946 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa30010c7d0 con 0x7fa30007adc0 2024-03-21T09:56:54.844 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:53.840+0000 7fa2fe000700 1 --2- 172.21.15.153:0/790367946 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7fa2e8038410 0x7fa2e803a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:56:54.844 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:53.840+0000 7fa2fe000700 1 -- 172.21.15.153:0/790367946 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa2f404c5a0 con 0x7fa30007adc0 2024-03-21T09:56:54.844 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:53.840+0000 7fa304a00700 1 --2- 172.21.15.153:0/790367946 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7fa2e8038410 0x7fa2e803a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:56:54.844 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:53.841+0000 7fa304a00700 1 --2- 172.21.15.153:0/790367946 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7fa2e8038410 0x7fa2e803a8d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec006fd0 tx=0x7fa2ec006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:56:54.844 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:53.845+0000 7fa2fe000700 1 -- 172.21.15.153:0/790367946 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fa2f4018b40 con 0x7fa30007adc0 2024-03-21T09:56:54.844 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:54.179+0000 7fa307e00700 1 -- 172.21.15.153:0/790367946 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7fa30010c9f0 con 0x7fa30007adc0 2024-03-21T09:56:54.844 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:54.187+0000 7fa2fe000700 1 -- 172.21.15.153:0/790367946 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v34)=0 set mgr/dashboard/cluster/status v34) v1 ==== 153+0+0 (secure 0 0 0) 0x7fa2f4018b40 con 0x7fa30007adc0 2024-03-21T09:56:54.844 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:54.189+0000 7fa307e00700 1 -- 172.21.15.153:0/790367946 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7fa2e8038410 msgr2=0x7fa2e803a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:56:54.844 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:54.189+0000 7fa307e00700 1 --2- 172.21.15.153:0/790367946 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7fa2e8038410 0x7fa2e803a8d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec006fd0 tx=0x7fa2ec006e40 comp rx=0 tx=0).stop 2024-03-21T09:56:54.845 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:54.189+0000 7fa307e00700 1 -- 172.21.15.153:0/790367946 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa30007adc0 msgr2=0x7fa30019c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:56:54.845 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:54.190+0000 7fa307e00700 1 --2- 172.21.15.153:0/790367946 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa30007adc0 0x7fa30019c350 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa2f4011670 tx=0x7fa2f4011b50 comp rx=0 tx=0).stop 2024-03-21T09:56:54.845 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:54.190+0000 7fa307e00700 1 -- 172.21.15.153:0/790367946 shutdown_connections 2024-03-21T09:56:54.845 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:54.190+0000 7fa307e00700 1 --2- 172.21.15.153:0/790367946 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7fa2e8038410 0x7fa2e803a8d0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:56:54.845 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:54.190+0000 7fa307e00700 1 --2- 172.21.15.153:0/790367946 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa30007adc0 0x7fa30019c350 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:56:54.845 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:54.190+0000 7fa307e00700 1 -- 172.21.15.153:0/790367946 >> 172.21.15.153:0/790367946 conn(0x7fa3001019b0 msgr2=0x7fa300103060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:56:54.845 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:54.190+0000 7fa307e00700 1 -- 172.21.15.153:0/790367946 shutdown_connections 2024-03-21T09:56:54.845 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr 2024-03-21T09:56:54.190+0000 7fa307e00700 1 -- 172.21.15.153:0/790367946 wait complete. 2024-03-21T09:56:54.845 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-03-21T09:56:54.845 INFO:teuthology.orchestra.run.smithi153.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-03-21T09:56:54.845 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T09:56:54.845 INFO:teuthology.orchestra.run.smithi153.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 070ed3e4-e769-11ee-95cd-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-03-21T09:56:54.845 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T09:56:54.845 INFO:teuthology.orchestra.run.smithi153.stdout:Or, if you are only running a single cluster on this host: 2024-03-21T09:56:54.845 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T09:56:54.846 INFO:teuthology.orchestra.run.smithi153.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-03-21T09:56:54.846 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T09:56:54.846 INFO:teuthology.orchestra.run.smithi153.stdout:Please consider enabling telemetry to help improve Ceph: 2024-03-21T09:56:54.846 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T09:56:54.846 INFO:teuthology.orchestra.run.smithi153.stdout: ceph telemetry on 2024-03-21T09:56:54.846 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T09:56:54.846 INFO:teuthology.orchestra.run.smithi153.stdout:For more information see: 2024-03-21T09:56:54.846 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T09:56:54.846 INFO:teuthology.orchestra.run.smithi153.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-03-21T09:56:54.846 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T09:56:54.846 INFO:teuthology.orchestra.run.smithi153.stdout:Bootstrap complete. 2024-03-21T09:56:54.918 INFO:tasks.cephadm:Fetching config... 2024-03-21T09:56:54.919 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-03-21T09:56:54.919 DEBUG:teuthology.orchestra.run.smithi153:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-03-21T09:56:55.008 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-03-21T09:56:55.008 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-03-21T09:56:55.009 DEBUG:teuthology.orchestra.run.smithi153:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-03-21T09:56:55.088 INFO:tasks.cephadm:Fetching mon keyring... 2024-03-21T09:56:55.089 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-03-21T09:56:55.089 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi153/keyring of=/dev/stdout 2024-03-21T09:56:55.215 INFO:tasks.cephadm:Fetching pub ssh key... 2024-03-21T09:56:55.216 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-03-21T09:56:55.216 DEBUG:teuthology.orchestra.run.smithi153:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-03-21T09:56:55.310 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-03-21T09:56:55.310 DEBUG:teuthology.orchestra.run.smithi153:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCvtI91Qo4EqNMgPkrfe1M6dO3/I+ER4cqFIDuaPn+ypn3Nn4aj1tvUVyoOriALOdtZObs6uBRhA7mdf6c9wcr2SP7AXdsDmGdIeA8YDnaW8kiC+rK2S5Of/6ZxhfqDt+mNlVRHjfHZ4iCAxgn4L3R9bdsoAJwQLdBm1WfjHaVgcB9zAOLcTMQl6Q9s4y1PLAavsv5XjBWoYhLjZDtMq24+vMBQfMxkQ9zvuyoR7w/5VejMLOyAFU8kmUnNwEjQq75LPz2huBabWlLaQRjtAL7bZMe0w7uthIVMZDvwtAXef0XfYgoRW7CdHO/Z63pD+eY2pHfLnO25yp1j/I6FIzub5G7+hbucaUpKdRigkBdJmEOi9YUvIUXnaeLKob+yyWDQpdQ6FrIT7t9nAzgiBiaE200m2wkD+TyQpQ9eFgEpShVbHNZEiHXailMO8sn5dZGiWB8SgV6DvxSxz4C+5Up3sCugg7h+29qDOktYjY7rVPja/UlNjnw2rOOPrHwNswU= ceph-070ed3e4-e769-11ee-95cd-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-21T09:56:55.423 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:55 smithi153 ceph-mon[29117]: from='client.? 172.21.15.153:0/790367946' entity='client.admin' 2024-03-21T09:56:55.441 INFO:teuthology.orchestra.run.smithi153.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCvtI91Qo4EqNMgPkrfe1M6dO3/I+ER4cqFIDuaPn+ypn3Nn4aj1tvUVyoOriALOdtZObs6uBRhA7mdf6c9wcr2SP7AXdsDmGdIeA8YDnaW8kiC+rK2S5Of/6ZxhfqDt+mNlVRHjfHZ4iCAxgn4L3R9bdsoAJwQLdBm1WfjHaVgcB9zAOLcTMQl6Q9s4y1PLAavsv5XjBWoYhLjZDtMq24+vMBQfMxkQ9zvuyoR7w/5VejMLOyAFU8kmUnNwEjQq75LPz2huBabWlLaQRjtAL7bZMe0w7uthIVMZDvwtAXef0XfYgoRW7CdHO/Z63pD+eY2pHfLnO25yp1j/I6FIzub5G7+hbucaUpKdRigkBdJmEOi9YUvIUXnaeLKob+yyWDQpdQ6FrIT7t9nAzgiBiaE200m2wkD+TyQpQ9eFgEpShVbHNZEiHXailMO8sn5dZGiWB8SgV6DvxSxz4C+5Up3sCugg7h+29qDOktYjY7rVPja/UlNjnw2rOOPrHwNswU= ceph-070ed3e4-e769-11ee-95cd-87774f69a715 2024-03-21T09:56:55.479 DEBUG:teuthology.orchestra.run.smithi174:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCvtI91Qo4EqNMgPkrfe1M6dO3/I+ER4cqFIDuaPn+ypn3Nn4aj1tvUVyoOriALOdtZObs6uBRhA7mdf6c9wcr2SP7AXdsDmGdIeA8YDnaW8kiC+rK2S5Of/6ZxhfqDt+mNlVRHjfHZ4iCAxgn4L3R9bdsoAJwQLdBm1WfjHaVgcB9zAOLcTMQl6Q9s4y1PLAavsv5XjBWoYhLjZDtMq24+vMBQfMxkQ9zvuyoR7w/5VejMLOyAFU8kmUnNwEjQq75LPz2huBabWlLaQRjtAL7bZMe0w7uthIVMZDvwtAXef0XfYgoRW7CdHO/Z63pD+eY2pHfLnO25yp1j/I6FIzub5G7+hbucaUpKdRigkBdJmEOi9YUvIUXnaeLKob+yyWDQpdQ6FrIT7t9nAzgiBiaE200m2wkD+TyQpQ9eFgEpShVbHNZEiHXailMO8sn5dZGiWB8SgV6DvxSxz4C+5Up3sCugg7h+29qDOktYjY7rVPja/UlNjnw2rOOPrHwNswU= ceph-070ed3e4-e769-11ee-95cd-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-21T09:56:55.613 INFO:teuthology.orchestra.run.smithi174.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCvtI91Qo4EqNMgPkrfe1M6dO3/I+ER4cqFIDuaPn+ypn3Nn4aj1tvUVyoOriALOdtZObs6uBRhA7mdf6c9wcr2SP7AXdsDmGdIeA8YDnaW8kiC+rK2S5Of/6ZxhfqDt+mNlVRHjfHZ4iCAxgn4L3R9bdsoAJwQLdBm1WfjHaVgcB9zAOLcTMQl6Q9s4y1PLAavsv5XjBWoYhLjZDtMq24+vMBQfMxkQ9zvuyoR7w/5VejMLOyAFU8kmUnNwEjQq75LPz2huBabWlLaQRjtAL7bZMe0w7uthIVMZDvwtAXef0XfYgoRW7CdHO/Z63pD+eY2pHfLnO25yp1j/I6FIzub5G7+hbucaUpKdRigkBdJmEOi9YUvIUXnaeLKob+yyWDQpdQ6FrIT7t9nAzgiBiaE200m2wkD+TyQpQ9eFgEpShVbHNZEiHXailMO8sn5dZGiWB8SgV6DvxSxz4C+5Up3sCugg7h+29qDOktYjY7rVPja/UlNjnw2rOOPrHwNswU= ceph-070ed3e4-e769-11ee-95cd-87774f69a715 2024-03-21T09:56:55.683 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 070ed3e4-e769-11ee-95cd-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-03-21T09:56:56.058 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi153/config 2024-03-21T09:56:58.688 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:56:58.686+0000 7f8cbae00700 1 -- 172.21.15.153:0/4152034020 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8cb4102a90 msgr2=0x7f8cb4102eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:56:58.688 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:56:58.686+0000 7f8cbae00700 1 --2- 172.21.15.153:0/4152034020 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8cb4102a90 0x7f8cb4102eb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8ca4009910 tx=0x7f8ca4009c20 comp rx=0 tx=0).stop 2024-03-21T09:56:58.689 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:56:58.686+0000 7f8cbae00700 1 -- 172.21.15.153:0/4152034020 shutdown_connections 2024-03-21T09:56:58.689 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:56:58.686+0000 7f8cbae00700 1 --2- 172.21.15.153:0/4152034020 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8cb4102a90 0x7f8cb4102eb0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:56:58.689 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:56:58.686+0000 7f8cbae00700 1 -- 172.21.15.153:0/4152034020 >> 172.21.15.153:0/4152034020 conn(0x7f8cb40fe010 msgr2=0x7f8cb4100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:56:58.689 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:56:58.687+0000 7f8cbae00700 1 -- 172.21.15.153:0/4152034020 shutdown_connections 2024-03-21T09:56:58.689 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:56:58.687+0000 7f8cbae00700 1 -- 172.21.15.153:0/4152034020 wait complete. 2024-03-21T09:56:58.690 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:56:58.688+0000 7f8cbae00700 1 Processor -- start 2024-03-21T09:56:58.690 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:56:58.688+0000 7f8cbae00700 1 -- start start 2024-03-21T09:56:58.690 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:56:58.688+0000 7f8cbae00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8cb4102a90 0x7f8cb4197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:56:58.690 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:56:58.688+0000 7f8cbae00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cb4198100 con 0x7f8cb4102a90 2024-03-21T09:56:58.690 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:56:58.688+0000 7f8cb3e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8cb4102a90 0x7f8cb4197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:56:58.690 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:56:58.688+0000 7f8cb3e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8cb4102a90 0x7f8cb4197bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:60828/0 (socket says 172.21.15.153:60828) 2024-03-21T09:56:58.690 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:56:58.688+0000 7f8cb3e00700 1 -- 172.21.15.153:0/386596607 learned_addr learned my addr 172.21.15.153:0/386596607 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T09:56:58.691 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:56:58.689+0000 7f8cb3e00700 1 -- 172.21.15.153:0/386596607 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ca40095f0 con 0x7f8cb4102a90 2024-03-21T09:56:58.691 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:56:58.689+0000 7f8cb3e00700 1 --2- 172.21.15.153:0/386596607 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8cb4102a90 0x7f8cb4197bc0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8ca4005b40 tx=0x7f8ca4004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:56:58.691 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:56:58.689+0000 7f8cb0c00700 1 -- 172.21.15.153:0/386596607 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8ca401d070 con 0x7f8cb4102a90 2024-03-21T09:56:58.691 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:56:58.689+0000 7f8cb0c00700 1 -- 172.21.15.153:0/386596607 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f8ca400be10 con 0x7f8cb4102a90 2024-03-21T09:56:58.691 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:56:58.689+0000 7f8cbae00700 1 -- 172.21.15.153:0/386596607 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cb4198300 con 0x7f8cb4102a90 2024-03-21T09:56:58.692 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:56:58.689+0000 7f8cb0c00700 1 -- 172.21.15.153:0/386596607 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8ca400f460 con 0x7f8cb4102a90 2024-03-21T09:56:58.692 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:56:58.690+0000 7f8cbae00700 1 -- 172.21.15.153:0/386596607 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cb41987a0 con 0x7f8cb4102a90 2024-03-21T09:56:58.693 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:56:58.691+0000 7f8cbae00700 1 -- 172.21.15.153:0/386596607 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cb41916b0 con 0x7f8cb4102a90 2024-03-21T09:56:58.693 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:56:58.691+0000 7f8cb0c00700 1 -- 172.21.15.153:0/386596607 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 13) v1 ==== 45340+0+0 (secure 0 0 0) 0x7f8ca40227e0 con 0x7f8cb4102a90 2024-03-21T09:56:58.693 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:56:58.691+0000 7f8cb0c00700 1 --2- 172.21.15.153:0/386596607 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7f8c98038410 0x7f8c9803a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:56:58.693 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:56:58.691+0000 7f8cb0c00700 1 -- 172.21.15.153:0/386596607 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f8ca404c360 con 0x7f8cb4102a90 2024-03-21T09:56:58.694 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:56:58.691+0000 7f8cb3400700 1 --2- 172.21.15.153:0/386596607 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7f8c98038410 0x7f8c9803a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:56:58.694 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:56:58.692+0000 7f8cb3400700 1 --2- 172.21.15.153:0/386596607 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7f8c98038410 0x7f8c9803a8d0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8ca0006fd0 tx=0x7f8ca0006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:56:58.698 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:56:58.696+0000 7f8cb0c00700 1 -- 172.21.15.153:0/386596607 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f8ca402ba00 con 0x7f8cb4102a90 2024-03-21T09:56:58.919 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:56:58.916+0000 7f8cbae00700 1 -- 172.21.15.153:0/386596607 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f8cb402cc70 con 0x7f8cb4102a90 2024-03-21T09:56:58.926 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:56:58.924+0000 7f8cb0c00700 1 -- 172.21.15.153:0/386596607 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7f8ca4027030 con 0x7f8cb4102a90 2024-03-21T09:56:58.929 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:56:58.927+0000 7f8cbae00700 1 -- 172.21.15.153:0/386596607 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7f8c98038410 msgr2=0x7f8c9803a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:56:58.929 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:56:58.927+0000 7f8cbae00700 1 --2- 172.21.15.153:0/386596607 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7f8c98038410 0x7f8c9803a8d0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8ca0006fd0 tx=0x7f8ca0006e40 comp rx=0 tx=0).stop 2024-03-21T09:56:58.929 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:56:58.927+0000 7f8cbae00700 1 -- 172.21.15.153:0/386596607 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8cb4102a90 msgr2=0x7f8cb4197bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:56:58.929 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:56:58.927+0000 7f8cbae00700 1 --2- 172.21.15.153:0/386596607 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8cb4102a90 0x7f8cb4197bc0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8ca4005b40 tx=0x7f8ca4004740 comp rx=0 tx=0).stop 2024-03-21T09:56:58.929 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:56:58.927+0000 7f8cbae00700 1 -- 172.21.15.153:0/386596607 shutdown_connections 2024-03-21T09:56:58.929 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:56:58.927+0000 7f8cbae00700 1 --2- 172.21.15.153:0/386596607 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7f8c98038410 0x7f8c9803a8d0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:56:58.929 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:56:58.927+0000 7f8cbae00700 1 --2- 172.21.15.153:0/386596607 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8cb4102a90 0x7f8cb4197bc0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:56:58.930 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:56:58.927+0000 7f8cbae00700 1 -- 172.21.15.153:0/386596607 >> 172.21.15.153:0/386596607 conn(0x7f8cb40fe010 msgr2=0x7f8cb40fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:56:58.930 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:56:58.928+0000 7f8cbae00700 1 -- 172.21.15.153:0/386596607 shutdown_connections 2024-03-21T09:56:58.930 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:56:58.928+0000 7f8cbae00700 1 -- 172.21.15.153:0/386596607 wait complete. 2024-03-21T09:56:59.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:58 smithi153 ceph-mon[29117]: from='mgr.14164 172.21.15.153:0/3445176977' entity='mgr.smithi153.tluxni' 2024-03-21T09:56:59.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:58 smithi153 ceph-mon[29117]: from='mgr.14164 172.21.15.153:0/3445176977' entity='mgr.smithi153.tluxni' 2024-03-21T09:56:59.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:58 smithi153 ceph-mon[29117]: from='mgr.14164 172.21.15.153:0/3445176977' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "who": "osd/host:smithi153", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:56:59.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:58 smithi153 ceph-mon[29117]: from='mgr.14164 172.21.15.153:0/3445176977' entity='mgr.smithi153.tluxni' 2024-03-21T09:56:59.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:58 smithi153 ceph-mon[29117]: from='mgr.14164 172.21.15.153:0/3445176977' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi153", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-21T09:56:59.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:58 smithi153 ceph-mon[29117]: from='mgr.14164 172.21.15.153:0/3445176977' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi153", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-03-21T09:56:59.308 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:58 smithi153 ceph-mon[29117]: from='mgr.14164 172.21.15.153:0/3445176977' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:56:59.308 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:58 smithi153 ceph-mon[29117]: Deploying daemon ceph-exporter.smithi153 on smithi153 2024-03-21T09:56:59.308 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:56:58 smithi153 ceph-mon[29117]: from='client.? 172.21.15.153:0/386596607' entity='client.admin' 2024-03-21T09:56:59.716 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-03-21T09:56:59.716 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 070ed3e4-e769-11ee-95cd-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-03-21T09:57:00.087 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi153/config 2024-03-21T09:57:03.185 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:03.183+0000 7f6335a00700 1 -- 172.21.15.153:0/2986370539 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6330074dc0 msgr2=0x7f6330073220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:57:03.185 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:03.183+0000 7f6335a00700 1 --2- 172.21.15.153:0/2986370539 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6330074dc0 0x7f6330073220 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6320009910 tx=0x7f6320009c20 comp rx=0 tx=0).stop 2024-03-21T09:57:03.185 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:03.183+0000 7f6335a00700 1 -- 172.21.15.153:0/2986370539 shutdown_connections 2024-03-21T09:57:03.185 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:03.183+0000 7f6335a00700 1 --2- 172.21.15.153:0/2986370539 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6330074dc0 0x7f6330073220 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:57:03.186 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:03.183+0000 7f6335a00700 1 -- 172.21.15.153:0/2986370539 >> 172.21.15.153:0/2986370539 conn(0x7f63300fbe30 msgr2=0x7f63300fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:57:03.186 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:03.183+0000 7f6335a00700 1 -- 172.21.15.153:0/2986370539 shutdown_connections 2024-03-21T09:57:03.186 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:03.184+0000 7f6335a00700 1 -- 172.21.15.153:0/2986370539 wait complete. 2024-03-21T09:57:03.186 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:03.185+0000 7f6335a00700 1 Processor -- start 2024-03-21T09:57:03.186 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:03.185+0000 7f6335a00700 1 -- start start 2024-03-21T09:57:03.187 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:03.185+0000 7f6335a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6330074dc0 0x7f633019c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:57:03.187 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:03.185+0000 7f6335a00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f633019c690 con 0x7f6330074dc0 2024-03-21T09:57:03.187 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:03.186+0000 7f632ea00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6330074dc0 0x7f633019c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:57:03.188 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:03.186+0000 7f632ea00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6330074dc0 0x7f633019c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:50326/0 (socket says 172.21.15.153:50326) 2024-03-21T09:57:03.188 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:03.186+0000 7f632ea00700 1 -- 172.21.15.153:0/55003049 learned_addr learned my addr 172.21.15.153:0/55003049 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T09:57:03.188 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:03.186+0000 7f632ea00700 1 -- 172.21.15.153:0/55003049 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63200095f0 con 0x7f6330074dc0 2024-03-21T09:57:03.188 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:03.186+0000 7f632ea00700 1 --2- 172.21.15.153:0/55003049 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6330074dc0 0x7f633019c150 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6320005950 tx=0x7f6320009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:57:03.189 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:03.187+0000 7f6327400700 1 -- 172.21.15.153:0/55003049 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f632001d070 con 0x7f6330074dc0 2024-03-21T09:57:03.189 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:03.187+0000 7f6327400700 1 -- 172.21.15.153:0/55003049 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f632000be10 con 0x7f6330074dc0 2024-03-21T09:57:03.189 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:03.187+0000 7f6335a00700 1 -- 172.21.15.153:0/55003049 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63301044f0 con 0x7f6330074dc0 2024-03-21T09:57:03.189 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:03.187+0000 7f6327400700 1 -- 172.21.15.153:0/55003049 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f632000f460 con 0x7f6330074dc0 2024-03-21T09:57:03.189 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:03.187+0000 7f6335a00700 1 -- 172.21.15.153:0/55003049 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f633019cba0 con 0x7f6330074dc0 2024-03-21T09:57:03.189 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:03.188+0000 7f6327400700 1 -- 172.21.15.153:0/55003049 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 13) v1 ==== 45340+0+0 (secure 0 0 0) 0x7f632000b960 con 0x7f6330074dc0 2024-03-21T09:57:03.190 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:03.188+0000 7f6335a00700 1 -- 172.21.15.153:0/55003049 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6330195c70 con 0x7f6330074dc0 2024-03-21T09:57:03.190 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:03.188+0000 7f6327400700 1 --2- 172.21.15.153:0/55003049 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7f63140383c0 0x7f631403a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:57:03.190 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:03.188+0000 7f6327400700 1 -- 172.21.15.153:0/55003049 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f632004c480 con 0x7f6330074dc0 2024-03-21T09:57:03.190 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:03.189+0000 7f632e000700 1 --2- 172.21.15.153:0/55003049 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7f63140383c0 0x7f631403a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:57:03.191 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:03.189+0000 7f632e000700 1 --2- 172.21.15.153:0/55003049 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7f63140383c0 0x7f631403a880 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f631c006fd0 tx=0x7f631c006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:57:03.195 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:03.194+0000 7f6327400700 1 -- 172.21.15.153:0/55003049 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f632002bba0 con 0x7f6330074dc0 2024-03-21T09:57:03.418 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:03.416+0000 7f6335a00700 1 -- 172.21.15.153:0/55003049 --> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7f63300611d0 con 0x7f63140383c0 2024-03-21T09:57:03.424 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:03.423+0000 7f6327400700 1 -- 172.21.15.153:0/55003049 <== mgr.14164 v2:172.21.15.153:6800/1143522828 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f63300611d0 con 0x7f63140383c0 2024-03-21T09:57:03.430 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:03.425+0000 7f6335a00700 1 -- 172.21.15.153:0/55003049 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7f63140383c0 msgr2=0x7f631403a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:57:03.430 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:03.425+0000 7f6335a00700 1 --2- 172.21.15.153:0/55003049 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7f63140383c0 0x7f631403a880 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f631c006fd0 tx=0x7f631c006e40 comp rx=0 tx=0).stop 2024-03-21T09:57:03.430 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:03.425+0000 7f6335a00700 1 -- 172.21.15.153:0/55003049 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6330074dc0 msgr2=0x7f633019c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:57:03.430 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:03.425+0000 7f6335a00700 1 --2- 172.21.15.153:0/55003049 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6330074dc0 0x7f633019c150 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6320005950 tx=0x7f6320009e60 comp rx=0 tx=0).stop 2024-03-21T09:57:03.430 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:03.426+0000 7f6335a00700 1 -- 172.21.15.153:0/55003049 shutdown_connections 2024-03-21T09:57:03.430 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:03.426+0000 7f6335a00700 1 --2- 172.21.15.153:0/55003049 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7f63140383c0 0x7f631403a880 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:57:03.431 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:03.426+0000 7f6335a00700 1 --2- 172.21.15.153:0/55003049 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6330074dc0 0x7f633019c150 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:57:03.431 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:03.426+0000 7f6335a00700 1 -- 172.21.15.153:0/55003049 >> 172.21.15.153:0/55003049 conn(0x7f63300fbe30 msgr2=0x7f63300fcb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:57:03.431 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:03.426+0000 7f6335a00700 1 -- 172.21.15.153:0/55003049 shutdown_connections 2024-03-21T09:57:03.431 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:03.426+0000 7f6335a00700 1 -- 172.21.15.153:0/55003049 wait complete. 2024-03-21T09:57:04.354 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi174 2024-03-21T09:57:04.354 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-03-21T09:57:04.354 DEBUG:teuthology.orchestra.run.smithi174:> dd of=/etc/ceph/ceph.conf 2024-03-21T09:57:04.408 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-03-21T09:57:04.408 DEBUG:teuthology.orchestra.run.smithi174:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-21T09:57:04.502 INFO:tasks.cephadm:Adding host smithi174 to orchestrator... 2024-03-21T09:57:04.502 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 070ed3e4-e769-11ee-95cd-87774f69a715 -- ceph orch host add smithi174 2024-03-21T09:57:04.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:57:04 smithi153 ceph-mon[29117]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:57:04.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:57:04 smithi153 ceph-mon[29117]: from='mgr.14164 172.21.15.153:0/3445176977' entity='mgr.smithi153.tluxni' 2024-03-21T09:57:04.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:57:04 smithi153 ceph-mon[29117]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:57:05.032 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi153/config 2024-03-21T09:57:06.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:57:06 smithi153 ceph-mon[29117]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:57:06.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:57:06 smithi153 ceph-mon[29117]: from='mgr.14164 172.21.15.153:0/3445176977' entity='mgr.smithi153.tluxni' 2024-03-21T09:57:06.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:57:06 smithi153 ceph-mon[29117]: from='mgr.14164 172.21.15.153:0/3445176977' entity='mgr.smithi153.tluxni' 2024-03-21T09:57:06.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:57:06 smithi153 ceph-mon[29117]: from='mgr.14164 172.21.15.153:0/3445176977' entity='mgr.smithi153.tluxni' 2024-03-21T09:57:06.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:57:06 smithi153 ceph-mon[29117]: from='mgr.14164 172.21.15.153:0/3445176977' entity='mgr.smithi153.tluxni' 2024-03-21T09:57:06.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:57:06 smithi153 ceph-mon[29117]: from='mgr.14164 172.21.15.153:0/3445176977' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi153", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-21T09:57:06.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:57:06 smithi153 ceph-mon[29117]: from='mgr.14164 172.21.15.153:0/3445176977' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi153", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-03-21T09:57:06.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:57:06 smithi153 ceph-mon[29117]: from='mgr.14164 172.21.15.153:0/3445176977' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:57:07.339 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:07.336+0000 7f80f4800700 1 -- 172.21.15.153:0/1965407706 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f80ec101330 msgr2=0x7f80ec103720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:57:07.339 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:07.336+0000 7f80f4800700 1 --2- 172.21.15.153:0/1965407706 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f80ec101330 0x7f80ec103720 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f80e0009910 tx=0x7f80e0009c20 comp rx=0 tx=0).stop 2024-03-21T09:57:07.339 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:07.337+0000 7f80f4800700 1 -- 172.21.15.153:0/1965407706 shutdown_connections 2024-03-21T09:57:07.339 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:07.337+0000 7f80f4800700 1 --2- 172.21.15.153:0/1965407706 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f80ec101330 0x7f80ec103720 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:57:07.339 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:07.337+0000 7f80f4800700 1 -- 172.21.15.153:0/1965407706 >> 172.21.15.153:0/1965407706 conn(0x7f80ec0fad20 msgr2=0x7f80ec0fd180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:57:07.339 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:07.337+0000 7f80f4800700 1 -- 172.21.15.153:0/1965407706 shutdown_connections 2024-03-21T09:57:07.339 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:07.337+0000 7f80f4800700 1 -- 172.21.15.153:0/1965407706 wait complete. 2024-03-21T09:57:07.340 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:07.338+0000 7f80f4800700 1 Processor -- start 2024-03-21T09:57:07.340 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:07.338+0000 7f80f4800700 1 -- start start 2024-03-21T09:57:07.340 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:07.339+0000 7f80f4800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f80ec101330 0x7f80ec197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:57:07.340 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:07.339+0000 7f80f4800700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80ec1980e0 con 0x7f80ec101330 2024-03-21T09:57:07.341 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:07.339+0000 7f80f1e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f80ec101330 0x7f80ec197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:57:07.341 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:07.339+0000 7f80f1e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f80ec101330 0x7f80ec197ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:50348/0 (socket says 172.21.15.153:50348) 2024-03-21T09:57:07.341 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:07.339+0000 7f80f1e00700 1 -- 172.21.15.153:0/3489816832 learned_addr learned my addr 172.21.15.153:0/3489816832 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T09:57:07.341 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:07.340+0000 7f80f1e00700 1 -- 172.21.15.153:0/3489816832 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80e00095f0 con 0x7f80ec101330 2024-03-21T09:57:07.342 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:07.340+0000 7f80f1e00700 1 --2- 172.21.15.153:0/3489816832 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f80ec101330 0x7f80ec197ba0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f80e00092e0 tx=0x7f80e0009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:57:07.342 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:07.340+0000 7f80e6a00700 1 -- 172.21.15.153:0/3489816832 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f80e001d070 con 0x7f80ec101330 2024-03-21T09:57:07.342 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:07.340+0000 7f80e6a00700 1 -- 172.21.15.153:0/3489816832 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f80e000be10 con 0x7f80ec101330 2024-03-21T09:57:07.342 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:07.340+0000 7f80e6a00700 1 -- 172.21.15.153:0/3489816832 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f80e000f460 con 0x7f80ec101330 2024-03-21T09:57:07.342 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:07.340+0000 7f80f4800700 1 -- 172.21.15.153:0/3489816832 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80ec1982e0 con 0x7f80ec101330 2024-03-21T09:57:07.342 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:07.340+0000 7f80f4800700 1 -- 172.21.15.153:0/3489816832 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80ec198780 con 0x7f80ec101330 2024-03-21T09:57:07.343 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:07.342+0000 7f80f4800700 1 -- 172.21.15.153:0/3489816832 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80ec1918c0 con 0x7f80ec101330 2024-03-21T09:57:07.344 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:07.342+0000 7f80e6a00700 1 -- 172.21.15.153:0/3489816832 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 13) v1 ==== 45340+0+0 (secure 0 0 0) 0x7f80e0022470 con 0x7f80ec101330 2024-03-21T09:57:07.344 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:07.343+0000 7f80e6a00700 1 --2- 172.21.15.153:0/3489816832 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7f80d4038410 0x7f80d403a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:57:07.344 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:07.343+0000 7f80e6a00700 1 -- 172.21.15.153:0/3489816832 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f80e004c400 con 0x7f80ec101330 2024-03-21T09:57:07.344 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:07.343+0000 7f80f1400700 1 --2- 172.21.15.153:0/3489816832 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7f80d4038410 0x7f80d403a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:57:07.345 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:07.344+0000 7f80f1400700 1 --2- 172.21.15.153:0/3489816832 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7f80d4038410 0x7f80d403a8d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f80dc006fd0 tx=0x7f80dc006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:57:07.350 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:07.349+0000 7f80e6a00700 1 -- 172.21.15.153:0/3489816832 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f80e000b960 con 0x7f80ec101330 2024-03-21T09:57:07.573 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:07.572+0000 7f80f4800700 1 -- 172.21.15.153:0/3489816832 --> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi174", "target": ["mon-mgr", ""]}) v1 -- 0x7f80ec0611d0 con 0x7f80d4038410 2024-03-21T09:57:07.800 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:57:07 smithi153 ceph-mon[29117]: Deploying daemon crash.smithi153 on smithi153 2024-03-21T09:57:09.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:57:08 smithi153 ceph-mon[29117]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:57:09.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:57:08 smithi153 ceph-mon[29117]: from='client.14191 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi174", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:57:09.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:57:08 smithi153 ceph-mon[29117]: from='mgr.14164 172.21.15.153:0/3445176977' entity='mgr.smithi153.tluxni' 2024-03-21T09:57:11.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:57:10 smithi153 ceph-mon[29117]: Deploying cephadm binary to smithi174 2024-03-21T09:57:11.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:57:10 smithi153 ceph-mon[29117]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:57:11.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:57:10 smithi153 ceph-mon[29117]: from='mgr.14164 172.21.15.153:0/3445176977' entity='mgr.smithi153.tluxni' 2024-03-21T09:57:11.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:57:10 smithi153 ceph-mon[29117]: from='mgr.14164 172.21.15.153:0/3445176977' entity='mgr.smithi153.tluxni' 2024-03-21T09:57:11.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:57:10 smithi153 ceph-mon[29117]: from='mgr.14164 172.21.15.153:0/3445176977' entity='mgr.smithi153.tluxni' 2024-03-21T09:57:11.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:57:10 smithi153 ceph-mon[29117]: from='mgr.14164 172.21.15.153:0/3445176977' entity='mgr.smithi153.tluxni' 2024-03-21T09:57:12.123 INFO:teuthology.orchestra.run.smithi153.stdout:Added host 'smithi174' with addr '172.21.15.174' 2024-03-21T09:57:12.123 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:12.119+0000 7f80e6a00700 1 -- 172.21.15.153:0/3489816832 <== mgr.14164 v2:172.21.15.153:6800/1143522828 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f80ec0611d0 con 0x7f80d4038410 2024-03-21T09:57:12.124 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:12.123+0000 7f80f4800700 1 -- 172.21.15.153:0/3489816832 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7f80d4038410 msgr2=0x7f80d403a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:57:12.125 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:12.123+0000 7f80f4800700 1 --2- 172.21.15.153:0/3489816832 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7f80d4038410 0x7f80d403a8d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f80dc006fd0 tx=0x7f80dc006e40 comp rx=0 tx=0).stop 2024-03-21T09:57:12.125 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:12.123+0000 7f80f4800700 1 -- 172.21.15.153:0/3489816832 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f80ec101330 msgr2=0x7f80ec197ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:57:12.125 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:12.123+0000 7f80f4800700 1 --2- 172.21.15.153:0/3489816832 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f80ec101330 0x7f80ec197ba0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f80e00092e0 tx=0x7f80e0009e60 comp rx=0 tx=0).stop 2024-03-21T09:57:12.125 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:12.124+0000 7f80f4800700 1 -- 172.21.15.153:0/3489816832 shutdown_connections 2024-03-21T09:57:12.126 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:12.124+0000 7f80f4800700 1 --2- 172.21.15.153:0/3489816832 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7f80d4038410 0x7f80d403a8d0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:57:12.126 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:12.124+0000 7f80f4800700 1 --2- 172.21.15.153:0/3489816832 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f80ec101330 0x7f80ec197ba0 secure :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f80e00092e0 tx=0x7f80e0009e60 comp rx=0 tx=0).stop 2024-03-21T09:57:12.126 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:12.124+0000 7f80f4800700 1 -- 172.21.15.153:0/3489816832 >> 172.21.15.153:0/3489816832 conn(0x7f80ec0fad20 msgr2=0x7f80ec0fba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:57:12.126 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:12.124+0000 7f80f4800700 1 -- 172.21.15.153:0/3489816832 shutdown_connections 2024-03-21T09:57:12.126 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:12.125+0000 7f80f4800700 1 -- 172.21.15.153:0/3489816832 wait complete. 2024-03-21T09:57:12.953 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 070ed3e4-e769-11ee-95cd-87774f69a715 -- ceph orch host ls --format=json 2024-03-21T09:57:13.022 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:57:12 smithi153 ceph-mon[29117]: Deploying daemon node-exporter.smithi153 on smithi153 2024-03-21T09:57:13.022 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:57:12 smithi153 ceph-mon[29117]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:57:13.022 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:57:12 smithi153 ceph-mon[29117]: from='mgr.14164 172.21.15.153:0/3445176977' entity='mgr.smithi153.tluxni' 2024-03-21T09:57:13.316 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi153/config 2024-03-21T09:57:14.031 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:57:13 smithi153 ceph-mon[29117]: Added host smithi174 2024-03-21T09:57:14.031 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:57:13 smithi153 ceph-mon[29117]: from='mgr.14164 172.21.15.153:0/3445176977' entity='mgr.smithi153.tluxni' 2024-03-21T09:57:15.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:57:14 smithi153 ceph-mon[29117]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:57:16.572 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:16.570+0000 7f5498200700 1 -- 172.21.15.153:0/3234642642 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5490100890 msgr2=0x7f5490100cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:57:16.573 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:16.570+0000 7f5498200700 1 --2- 172.21.15.153:0/3234642642 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5490100890 0x7f5490100cb0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f5484009910 tx=0x7f5484009c20 comp rx=0 tx=0).stop 2024-03-21T09:57:16.573 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:16.570+0000 7f5498200700 1 -- 172.21.15.153:0/3234642642 shutdown_connections 2024-03-21T09:57:16.573 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:16.570+0000 7f5498200700 1 --2- 172.21.15.153:0/3234642642 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5490100890 0x7f5490100cb0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:57:16.573 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:16.570+0000 7f5498200700 1 -- 172.21.15.153:0/3234642642 >> 172.21.15.153:0/3234642642 conn(0x7f54900fbe30 msgr2=0x7f54900fe270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:57:16.573 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:16.570+0000 7f5498200700 1 -- 172.21.15.153:0/3234642642 shutdown_connections 2024-03-21T09:57:16.573 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:16.571+0000 7f5498200700 1 -- 172.21.15.153:0/3234642642 wait complete. 2024-03-21T09:57:16.574 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:16.571+0000 7f5498200700 1 Processor -- start 2024-03-21T09:57:16.574 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:16.572+0000 7f5498200700 1 -- start start 2024-03-21T09:57:16.574 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:16.572+0000 7f5498200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5490100890 0x7f5490197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:57:16.574 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:16.572+0000 7f5498200700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54901980f0 con 0x7f5490100890 2024-03-21T09:57:16.574 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:16.572+0000 7f5495800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5490100890 0x7f5490197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:57:16.574 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:16.572+0000 7f5495800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5490100890 0x7f5490197bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:54350/0 (socket says 172.21.15.153:54350) 2024-03-21T09:57:16.575 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:16.572+0000 7f5495800700 1 -- 172.21.15.153:0/2540457864 learned_addr learned my addr 172.21.15.153:0/2540457864 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T09:57:16.575 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:16.573+0000 7f5495800700 1 -- 172.21.15.153:0/2540457864 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54840095f0 con 0x7f5490100890 2024-03-21T09:57:16.575 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:16.573+0000 7f5495800700 1 --2- 172.21.15.153:0/2540457864 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5490100890 0x7f5490197bb0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5484005950 tx=0x7f5484009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:57:16.575 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:16.573+0000 7f548e000700 1 -- 172.21.15.153:0/2540457864 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f548401d070 con 0x7f5490100890 2024-03-21T09:57:16.575 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:16.573+0000 7f548e000700 1 -- 172.21.15.153:0/2540457864 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f548400be10 con 0x7f5490100890 2024-03-21T09:57:16.575 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:16.573+0000 7f5498200700 1 -- 172.21.15.153:0/2540457864 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54901982f0 con 0x7f5490100890 2024-03-21T09:57:16.575 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:16.573+0000 7f548e000700 1 -- 172.21.15.153:0/2540457864 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f548400f460 con 0x7f5490100890 2024-03-21T09:57:16.576 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:16.573+0000 7f5498200700 1 -- 172.21.15.153:0/2540457864 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5490198790 con 0x7f5490100890 2024-03-21T09:57:16.576 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:16.574+0000 7f548e000700 1 -- 172.21.15.153:0/2540457864 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 13) v1 ==== 45340+0+0 (secure 0 0 0) 0x7f54840227c0 con 0x7f5490100890 2024-03-21T09:57:16.576 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:16.574+0000 7f5498200700 1 -- 172.21.15.153:0/2540457864 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54901918b0 con 0x7f5490100890 2024-03-21T09:57:16.577 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:16.575+0000 7f548e000700 1 --2- 172.21.15.153:0/2540457864 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7f54780383c0 0x7f547803a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:57:16.577 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:16.575+0000 7f548e000700 1 -- 172.21.15.153:0/2540457864 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f548404c4e0 con 0x7f5490100890 2024-03-21T09:57:16.577 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:16.575+0000 7f5494e00700 1 --2- 172.21.15.153:0/2540457864 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7f54780383c0 0x7f547803a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:57:16.577 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:16.576+0000 7f5494e00700 1 --2- 172.21.15.153:0/2540457864 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7f54780383c0 0x7f547803a880 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f547c006fd0 tx=0x7f547c006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:57:16.582 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:16.580+0000 7f548e000700 1 -- 172.21.15.153:0/2540457864 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f548402bba0 con 0x7f5490100890 2024-03-21T09:57:16.805 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:16.804+0000 7f5498200700 1 -- 172.21.15.153:0/2540457864 --> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f54900611d0 con 0x7f54780383c0 2024-03-21T09:57:16.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:57:16 smithi153 ceph-mon[29117]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:57:16.807 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:16.806+0000 7f548e000700 1 -- 172.21.15.153:0/2540457864 <== mgr.14164 v2:172.21.15.153:6800/1143522828 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+161 (secure 0 0 0) 0x7f54900611d0 con 0x7f54780383c0 2024-03-21T09:57:16.807 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T09:57:16.808 INFO:teuthology.orchestra.run.smithi153.stdout:[{"addr": "172.21.15.153", "hostname": "smithi153", "labels": [], "status": ""}, {"addr": "172.21.15.174", "hostname": "smithi174", "labels": [], "status": ""}] 2024-03-21T09:57:16.811 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:16.809+0000 7f5498200700 1 -- 172.21.15.153:0/2540457864 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7f54780383c0 msgr2=0x7f547803a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:57:16.811 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:16.809+0000 7f5498200700 1 --2- 172.21.15.153:0/2540457864 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7f54780383c0 0x7f547803a880 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f547c006fd0 tx=0x7f547c006e40 comp rx=0 tx=0).stop 2024-03-21T09:57:16.811 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:16.809+0000 7f5498200700 1 -- 172.21.15.153:0/2540457864 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5490100890 msgr2=0x7f5490197bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:57:16.811 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:16.809+0000 7f5498200700 1 --2- 172.21.15.153:0/2540457864 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5490100890 0x7f5490197bb0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5484005950 tx=0x7f5484009e60 comp rx=0 tx=0).stop 2024-03-21T09:57:16.811 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:16.809+0000 7f5498200700 1 -- 172.21.15.153:0/2540457864 shutdown_connections 2024-03-21T09:57:16.811 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:16.809+0000 7f5498200700 1 --2- 172.21.15.153:0/2540457864 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7f54780383c0 0x7f547803a880 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:57:16.811 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:16.809+0000 7f5498200700 1 --2- 172.21.15.153:0/2540457864 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5490100890 0x7f5490197bb0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:57:16.811 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:16.809+0000 7f5498200700 1 -- 172.21.15.153:0/2540457864 >> 172.21.15.153:0/2540457864 conn(0x7f54900fbe30 msgr2=0x7f54900fe270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:57:16.811 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:16.809+0000 7f5498200700 1 -- 172.21.15.153:0/2540457864 shutdown_connections 2024-03-21T09:57:16.811 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:16.809+0000 7f5498200700 1 -- 172.21.15.153:0/2540457864 wait complete. 2024-03-21T09:57:17.764 INFO:tasks.cephadm:Setting crush tunables to default 2024-03-21T09:57:17.764 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 070ed3e4-e769-11ee-95cd-87774f69a715 -- ceph osd crush tunables default 2024-03-21T09:57:17.844 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:57:17 smithi153 ceph-mon[29117]: from='client.14193 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T09:57:17.845 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:57:17 smithi153 ceph-mon[29117]: from='mgr.14164 172.21.15.153:0/3445176977' entity='mgr.smithi153.tluxni' 2024-03-21T09:57:17.845 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:57:17 smithi153 ceph-mon[29117]: from='mgr.14164 172.21.15.153:0/3445176977' entity='mgr.smithi153.tluxni' 2024-03-21T09:57:17.845 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:57:17 smithi153 ceph-mon[29117]: from='mgr.14164 172.21.15.153:0/3445176977' entity='mgr.smithi153.tluxni' 2024-03-21T09:57:17.845 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:57:17 smithi153 ceph-mon[29117]: from='mgr.14164 172.21.15.153:0/3445176977' entity='mgr.smithi153.tluxni' 2024-03-21T09:57:18.143 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi153/config 2024-03-21T09:57:18.994 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:57:18 smithi153 ceph-mon[29117]: Deploying daemon alertmanager.smithi153 on smithi153 2024-03-21T09:57:18.994 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:57:18 smithi153 ceph-mon[29117]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:57:18.994 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:57:18 smithi153 ceph-mon[29117]: from='mgr.14164 172.21.15.153:0/3445176977' entity='mgr.smithi153.tluxni' 2024-03-21T09:57:20.946 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:57:20 smithi153 ceph-mon[29117]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:57:20.947 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:20.943+0000 7fdf8ea00700 1 -- 172.21.15.153:0/2121065499 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdf88102a90 msgr2=0x7fdf88102eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:57:20.947 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:20.943+0000 7fdf8ea00700 1 --2- 172.21.15.153:0/2121065499 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdf88102a90 0x7fdf88102eb0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fdf78009910 tx=0x7fdf78009c20 comp rx=0 tx=0).stop 2024-03-21T09:57:20.947 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:20.943+0000 7fdf8ea00700 1 -- 172.21.15.153:0/2121065499 shutdown_connections 2024-03-21T09:57:20.947 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:20.943+0000 7fdf8ea00700 1 --2- 172.21.15.153:0/2121065499 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdf88102a90 0x7fdf88102eb0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:57:20.947 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:20.943+0000 7fdf8ea00700 1 -- 172.21.15.153:0/2121065499 >> 172.21.15.153:0/2121065499 conn(0x7fdf880fe010 msgr2=0x7fdf88100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:57:20.947 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:20.943+0000 7fdf8ea00700 1 -- 172.21.15.153:0/2121065499 shutdown_connections 2024-03-21T09:57:20.948 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:20.943+0000 7fdf8ea00700 1 -- 172.21.15.153:0/2121065499 wait complete. 2024-03-21T09:57:20.948 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:20.944+0000 7fdf8ea00700 1 Processor -- start 2024-03-21T09:57:20.948 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:20.944+0000 7fdf8ea00700 1 -- start start 2024-03-21T09:57:20.948 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:20.945+0000 7fdf8ea00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdf88102a90 0x7fdf8819bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:57:20.948 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:20.945+0000 7fdf8ea00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf8819c520 con 0x7fdf88102a90 2024-03-21T09:57:20.948 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:20.945+0000 7fdf87e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdf88102a90 0x7fdf8819bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:57:20.948 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:20.945+0000 7fdf87e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdf88102a90 0x7fdf8819bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:54368/0 (socket says 172.21.15.153:54368) 2024-03-21T09:57:20.948 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:20.945+0000 7fdf87e00700 1 -- 172.21.15.153:0/456578016 learned_addr learned my addr 172.21.15.153:0/456578016 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T09:57:20.949 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:20.945+0000 7fdf87e00700 1 -- 172.21.15.153:0/456578016 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf780095f0 con 0x7fdf88102a90 2024-03-21T09:57:20.949 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:20.947+0000 7fdf87e00700 1 --2- 172.21.15.153:0/456578016 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdf88102a90 0x7fdf8819bfe0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fdf78005950 tx=0x7fdf78009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:57:20.949 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:20.948+0000 7fdf84c00700 1 -- 172.21.15.153:0/456578016 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdf7801d070 con 0x7fdf88102a90 2024-03-21T09:57:20.949 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:20.948+0000 7fdf8ea00700 1 -- 172.21.15.153:0/456578016 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf8819c720 con 0x7fdf88102a90 2024-03-21T09:57:20.949 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:20.948+0000 7fdf8ea00700 1 -- 172.21.15.153:0/456578016 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf8819cbc0 con 0x7fdf88102a90 2024-03-21T09:57:20.950 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:20.949+0000 7fdf84c00700 1 -- 172.21.15.153:0/456578016 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fdf78005c00 con 0x7fdf88102a90 2024-03-21T09:57:20.950 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:20.949+0000 7fdf84c00700 1 -- 172.21.15.153:0/456578016 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdf7800fbe0 con 0x7fdf88102a90 2024-03-21T09:57:20.951 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:20.950+0000 7fdf84c00700 1 -- 172.21.15.153:0/456578016 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 13) v1 ==== 45340+0+0 (secure 0 0 0) 0x7fdf7800fd40 con 0x7fdf88102a90 2024-03-21T09:57:20.952 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:20.950+0000 7fdf8ea00700 1 -- 172.21.15.153:0/456578016 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf88078980 con 0x7fdf88102a90 2024-03-21T09:57:20.952 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:20.951+0000 7fdf84c00700 1 --2- 172.21.15.153:0/456578016 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7fdf6c038460 0x7fdf6c03a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:57:20.952 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:20.951+0000 7fdf84c00700 1 -- 172.21.15.153:0/456578016 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fdf7804d3d0 con 0x7fdf88102a90 2024-03-21T09:57:20.952 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:20.951+0000 7fdf87400700 1 --2- 172.21.15.153:0/456578016 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7fdf6c038460 0x7fdf6c03a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:57:20.953 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:20.952+0000 7fdf87400700 1 --2- 172.21.15.153:0/456578016 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7fdf6c038460 0x7fdf6c03a920 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fdf74006fd0 tx=0x7fdf74006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:57:20.962 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:20.961+0000 7fdf84c00700 1 -- 172.21.15.153:0/456578016 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fdf7800e3e0 con 0x7fdf88102a90 2024-03-21T09:57:21.211 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:21.209+0000 7fdf8ea00700 1 -- 172.21.15.153:0/456578016 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fdf8802cc70 con 0x7fdf88102a90 2024-03-21T09:57:21.795 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:21.793+0000 7fdf84c00700 1 -- 172.21.15.153:0/456578016 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7fdf78027020 con 0x7fdf88102a90 2024-03-21T09:57:21.799 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:21.797+0000 7fdf8ea00700 1 -- 172.21.15.153:0/456578016 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7fdf6c038460 msgr2=0x7fdf6c03a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:57:21.799 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:21.797+0000 7fdf8ea00700 1 --2- 172.21.15.153:0/456578016 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7fdf6c038460 0x7fdf6c03a920 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fdf74006fd0 tx=0x7fdf74006e40 comp rx=0 tx=0).stop 2024-03-21T09:57:21.799 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:21.797+0000 7fdf8ea00700 1 -- 172.21.15.153:0/456578016 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdf88102a90 msgr2=0x7fdf8819bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:57:21.799 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:21.797+0000 7fdf8ea00700 1 --2- 172.21.15.153:0/456578016 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdf88102a90 0x7fdf8819bfe0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fdf78005950 tx=0x7fdf78009e60 comp rx=0 tx=0).stop 2024-03-21T09:57:21.799 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:21.797+0000 7fdf8ea00700 1 -- 172.21.15.153:0/456578016 shutdown_connections 2024-03-21T09:57:21.799 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:21.797+0000 7fdf8ea00700 1 --2- 172.21.15.153:0/456578016 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7fdf6c038460 0x7fdf6c03a920 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:57:21.799 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:21.797+0000 7fdf8ea00700 1 --2- 172.21.15.153:0/456578016 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdf88102a90 0x7fdf8819bfe0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:57:21.799 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:21.797+0000 7fdf8ea00700 1 -- 172.21.15.153:0/456578016 >> 172.21.15.153:0/456578016 conn(0x7fdf880fe010 msgr2=0x7fdf880fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:57:21.800 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:21.798+0000 7fdf8ea00700 1 -- 172.21.15.153:0/456578016 shutdown_connections 2024-03-21T09:57:21.800 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:57:21.798+0000 7fdf8ea00700 1 -- 172.21.15.153:0/456578016 wait complete. 2024-03-21T09:57:21.800 INFO:teuthology.orchestra.run.smithi153.stderr:adjusted tunables profile to default 2024-03-21T09:57:22.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:57:21 smithi153 ceph-mon[29117]: from='client.? 172.21.15.153:0/456578016' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-21T09:57:23.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:57:22 smithi153 ceph-mon[29117]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:57:23.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:57:22 smithi153 ceph-mon[29117]: from='client.? 172.21.15.153:0/456578016' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-21T09:57:23.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:57:22 smithi153 ceph-mon[29117]: osdmap e4: 0 total, 0 up, 0 in 2024-03-21T09:57:23.595 INFO:tasks.cephadm:Adding mon.smithi153 on smithi153 2024-03-21T09:57:23.595 INFO:tasks.cephadm:Adding mon.smithi174 on smithi174 2024-03-21T09:57:23.595 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 070ed3e4-e769-11ee-95cd-87774f69a715 -- ceph orch apply mon '2;smithi153:172.21.15.153=smithi153;smithi174:172.21.15.174=smithi174' 2024-03-21T09:57:23.973 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-21T09:57:24.068 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-21T09:57:25.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:57:24 smithi153 ceph-mon[29117]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:57:26.085 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:26.083+0000 7f9bcce00700 1 -- 172.21.15.174:0/1923329607 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9bc81008b0 msgr2=0x7f9bc8100cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:57:26.085 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:26.083+0000 7f9bcce00700 1 --2- 172.21.15.174:0/1923329607 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9bc81008b0 0x7f9bc8100cd0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f9bb8009910 tx=0x7f9bb8009c20 comp rx=0 tx=0).stop 2024-03-21T09:57:26.085 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:26.083+0000 7f9bcce00700 1 -- 172.21.15.174:0/1923329607 shutdown_connections 2024-03-21T09:57:26.086 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:26.083+0000 7f9bcce00700 1 --2- 172.21.15.174:0/1923329607 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9bc81008b0 0x7f9bc8100cd0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:57:26.086 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:26.083+0000 7f9bcce00700 1 -- 172.21.15.174:0/1923329607 >> 172.21.15.174:0/1923329607 conn(0x7f9bc80fbe30 msgr2=0x7f9bc80fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:57:26.086 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:26.083+0000 7f9bcce00700 1 -- 172.21.15.174:0/1923329607 shutdown_connections 2024-03-21T09:57:26.086 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:26.083+0000 7f9bcce00700 1 -- 172.21.15.174:0/1923329607 wait complete. 2024-03-21T09:57:26.086 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:26.084+0000 7f9bcce00700 1 Processor -- start 2024-03-21T09:57:26.087 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:26.084+0000 7f9bcce00700 1 -- start start 2024-03-21T09:57:26.087 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:26.085+0000 7f9bcce00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9bc81008b0 0x7f9bc8197c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:57:26.087 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:26.085+0000 7f9bcce00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bc8198170 con 0x7f9bc81008b0 2024-03-21T09:57:26.087 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:26.085+0000 7f9bc5e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9bc81008b0 0x7f9bc8197c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:57:26.087 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:26.085+0000 7f9bc5e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9bc81008b0 0x7f9bc8197c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.174:42602/0 (socket says 172.21.15.174:42602) 2024-03-21T09:57:26.087 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:26.085+0000 7f9bc5e00700 1 -- 172.21.15.174:0/1875335180 learned_addr learned my addr 172.21.15.174:0/1875335180 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-03-21T09:57:26.088 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:26.086+0000 7f9bc5e00700 1 -- 172.21.15.174:0/1875335180 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bb80095f0 con 0x7f9bc81008b0 2024-03-21T09:57:26.088 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:26.086+0000 7f9bc5e00700 1 --2- 172.21.15.174:0/1875335180 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9bc81008b0 0x7f9bc8197c30 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f9bb8005950 tx=0x7f9bb8009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:57:26.088 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:26.087+0000 7f9bbea00700 1 -- 172.21.15.174:0/1875335180 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9bb801d070 con 0x7f9bc81008b0 2024-03-21T09:57:26.088 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:26.087+0000 7f9bcce00700 1 -- 172.21.15.174:0/1875335180 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bc8198370 con 0x7f9bc81008b0 2024-03-21T09:57:26.089 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:26.087+0000 7f9bbea00700 1 -- 172.21.15.174:0/1875335180 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f9bb800be10 con 0x7f9bc81008b0 2024-03-21T09:57:26.089 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:26.087+0000 7f9bbea00700 1 -- 172.21.15.174:0/1875335180 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9bb800f460 con 0x7f9bc81008b0 2024-03-21T09:57:26.089 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:26.087+0000 7f9bcce00700 1 -- 172.21.15.174:0/1875335180 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bc8198810 con 0x7f9bc81008b0 2024-03-21T09:57:26.090 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:26.088+0000 7f9bcce00700 1 -- 172.21.15.174:0/1875335180 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bc81918d0 con 0x7f9bc81008b0 2024-03-21T09:57:26.090 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:26.089+0000 7f9bbea00700 1 -- 172.21.15.174:0/1875335180 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 13) v1 ==== 45340+0+0 (secure 0 0 0) 0x7f9bb80227c0 con 0x7f9bc81008b0 2024-03-21T09:57:26.091 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:26.089+0000 7f9bbea00700 1 --2- 172.21.15.174:0/1875335180 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7f9bac0383c0 0x7f9bac03a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:57:26.091 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:26.089+0000 7f9bbea00700 1 -- 172.21.15.174:0/1875335180 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f9bb804c4e0 con 0x7f9bc81008b0 2024-03-21T09:57:26.091 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:26.090+0000 7f9bc5400700 1 --2- 172.21.15.174:0/1875335180 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7f9bac0383c0 0x7f9bac03a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:57:26.092 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:26.091+0000 7f9bc5400700 1 --2- 172.21.15.174:0/1875335180 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7f9bac0383c0 0x7f9bac03a880 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f9bb4006fd0 tx=0x7f9bb4006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:57:26.108 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:26.106+0000 7f9bbea00700 1 -- 172.21.15.174:0/1875335180 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f9bb802bba0 con 0x7f9bc81008b0 2024-03-21T09:57:26.336 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:26.333+0000 7f9bcce00700 1 -- 172.21.15.174:0/1875335180 --> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi153:172.21.15.153=smithi153;smithi174:172.21.15.174=smithi174", "target": ["mon-mgr", ""]}) v1 -- 0x7f9bc8061340 con 0x7f9bac0383c0 2024-03-21T09:57:26.351 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:26.350+0000 7f9bbea00700 1 -- 172.21.15.174:0/1875335180 <== mgr.14164 v2:172.21.15.153:6800/1143522828 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f9bc8061340 con 0x7f9bac0383c0 2024-03-21T09:57:26.352 INFO:teuthology.orchestra.run.smithi174.stdout:Scheduled mon update... 2024-03-21T09:57:26.355 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:26.352+0000 7f9bcce00700 1 -- 172.21.15.174:0/1875335180 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7f9bac0383c0 msgr2=0x7f9bac03a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:57:26.355 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:26.353+0000 7f9bcce00700 1 --2- 172.21.15.174:0/1875335180 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7f9bac0383c0 0x7f9bac03a880 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f9bb4006fd0 tx=0x7f9bb4006e40 comp rx=0 tx=0).stop 2024-03-21T09:57:26.355 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:26.353+0000 7f9bcce00700 1 -- 172.21.15.174:0/1875335180 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9bc81008b0 msgr2=0x7f9bc8197c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:57:26.355 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:26.353+0000 7f9bcce00700 1 --2- 172.21.15.174:0/1875335180 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9bc81008b0 0x7f9bc8197c30 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f9bb8005950 tx=0x7f9bb8009e60 comp rx=0 tx=0).stop 2024-03-21T09:57:26.355 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:26.353+0000 7f9bcce00700 1 -- 172.21.15.174:0/1875335180 shutdown_connections 2024-03-21T09:57:26.355 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:26.353+0000 7f9bcce00700 1 --2- 172.21.15.174:0/1875335180 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7f9bac0383c0 0x7f9bac03a880 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:57:26.355 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:26.353+0000 7f9bcce00700 1 --2- 172.21.15.174:0/1875335180 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9bc81008b0 0x7f9bc8197c30 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:57:26.355 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:26.353+0000 7f9bcce00700 1 -- 172.21.15.174:0/1875335180 >> 172.21.15.174:0/1875335180 conn(0x7f9bc80fbe30 msgr2=0x7f9bc80fcb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:57:26.355 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:26.353+0000 7f9bcce00700 1 -- 172.21.15.174:0/1875335180 shutdown_connections 2024-03-21T09:57:26.356 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:26.353+0000 7f9bcce00700 1 -- 172.21.15.174:0/1875335180 wait complete. 2024-03-21T09:57:27.056 DEBUG:teuthology.orchestra.run.smithi174:mon.smithi174> sudo journalctl -f -n 0 -u ceph-070ed3e4-e769-11ee-95cd-87774f69a715@mon.smithi174.service 2024-03-21T09:57:27.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:57:26 smithi153 ceph-mon[29117]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:57:27.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:57:26 smithi153 ceph-mon[29117]: from='mgr.14164 172.21.15.153:0/3445176977' entity='mgr.smithi153.tluxni' 2024-03-21T09:57:27.062 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-21T09:57:27.062 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 070ed3e4-e769-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-21T09:57:27.434 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-21T09:57:27.529 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-21T09:57:28.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:57:27 smithi153 ceph-mon[29117]: from='client.14197 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi153:172.21.15.153=smithi153;smithi174:172.21.15.174=smithi174", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:57:28.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:57:27 smithi153 ceph-mon[29117]: Saving service mon spec with placement smithi153:172.21.15.153=smithi153;smithi174:172.21.15.174=smithi174;count:2 2024-03-21T09:57:29.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:57:28 smithi153 ceph-mon[29117]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:57:29.447 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:29.444+0000 7f69dd200700 1 -- 172.21.15.174:0/282455196 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f69d8074dc0 msgr2=0x7f69d8073220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:57:29.447 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:29.444+0000 7f69dd200700 1 --2- 172.21.15.174:0/282455196 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f69d8074dc0 0x7f69d8073220 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f69c8009910 tx=0x7f69c8009c20 comp rx=0 tx=0).stop 2024-03-21T09:57:29.447 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:29.444+0000 7f69dd200700 1 -- 172.21.15.174:0/282455196 shutdown_connections 2024-03-21T09:57:29.447 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:29.444+0000 7f69dd200700 1 --2- 172.21.15.174:0/282455196 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f69d8074dc0 0x7f69d8073220 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:57:29.447 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:29.444+0000 7f69dd200700 1 -- 172.21.15.174:0/282455196 >> 172.21.15.174:0/282455196 conn(0x7f69d80fbe30 msgr2=0x7f69d80fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:57:29.447 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:29.445+0000 7f69dd200700 1 -- 172.21.15.174:0/282455196 shutdown_connections 2024-03-21T09:57:29.447 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:29.445+0000 7f69dd200700 1 -- 172.21.15.174:0/282455196 wait complete. 2024-03-21T09:57:29.448 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:29.446+0000 7f69dd200700 1 Processor -- start 2024-03-21T09:57:29.448 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:29.446+0000 7f69dd200700 1 -- start start 2024-03-21T09:57:29.448 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:29.446+0000 7f69dd200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f69d8074dc0 0x7f69d8197be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:57:29.448 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:29.446+0000 7f69dd200700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69d8198120 con 0x7f69d8074dc0 2024-03-21T09:57:29.449 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:29.447+0000 7f69d5e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f69d8074dc0 0x7f69d8197be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:57:29.449 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:29.447+0000 7f69d5e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f69d8074dc0 0x7f69d8197be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.174:57306/0 (socket says 172.21.15.174:57306) 2024-03-21T09:57:29.449 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:29.447+0000 7f69d5e00700 1 -- 172.21.15.174:0/3438582286 learned_addr learned my addr 172.21.15.174:0/3438582286 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-03-21T09:57:29.449 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:29.447+0000 7f69d5e00700 1 -- 172.21.15.174:0/3438582286 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69c80095f0 con 0x7f69d8074dc0 2024-03-21T09:57:29.450 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:29.447+0000 7f69d5e00700 1 --2- 172.21.15.174:0/3438582286 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f69d8074dc0 0x7f69d8197be0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f69c8005b40 tx=0x7f69c8004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:57:29.450 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:29.448+0000 7f69cea00700 1 -- 172.21.15.174:0/3438582286 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f69c801d070 con 0x7f69d8074dc0 2024-03-21T09:57:29.450 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:29.448+0000 7f69cea00700 1 -- 172.21.15.174:0/3438582286 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f69c800be10 con 0x7f69d8074dc0 2024-03-21T09:57:29.450 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:29.448+0000 7f69dd200700 1 -- 172.21.15.174:0/3438582286 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69d8198320 con 0x7f69d8074dc0 2024-03-21T09:57:29.450 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:29.448+0000 7f69cea00700 1 -- 172.21.15.174:0/3438582286 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f69c800f460 con 0x7f69d8074dc0 2024-03-21T09:57:29.450 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:29.448+0000 7f69dd200700 1 -- 172.21.15.174:0/3438582286 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69d81987c0 con 0x7f69d8074dc0 2024-03-21T09:57:29.451 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:29.449+0000 7f69dd200700 1 -- 172.21.15.174:0/3438582286 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69d81916e0 con 0x7f69d8074dc0 2024-03-21T09:57:29.451 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:29.449+0000 7f69cea00700 1 -- 172.21.15.174:0/3438582286 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 13) v1 ==== 45340+0+0 (secure 0 0 0) 0x7f69c80227e0 con 0x7f69d8074dc0 2024-03-21T09:57:29.452 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:29.450+0000 7f69cea00700 1 --2- 172.21.15.174:0/3438582286 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7f69bc038410 0x7f69bc03a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:57:29.452 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:29.450+0000 7f69cea00700 1 -- 172.21.15.174:0/3438582286 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f69c804c420 con 0x7f69d8074dc0 2024-03-21T09:57:29.452 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:29.450+0000 7f69d5400700 1 --2- 172.21.15.174:0/3438582286 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7f69bc038410 0x7f69bc03a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:57:29.453 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:29.452+0000 7f69d5400700 1 --2- 172.21.15.174:0/3438582286 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7f69bc038410 0x7f69bc03a8d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f69c4006fd0 tx=0x7f69c4006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:57:29.467 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:29.465+0000 7f69cea00700 1 -- 172.21.15.174:0/3438582286 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f69c802ba00 con 0x7f69d8074dc0 2024-03-21T09:57:29.741 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:29.738+0000 7f69dd200700 1 -- 172.21.15.174:0/3438582286 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f69d802cc70 con 0x7f69d8074dc0 2024-03-21T09:57:29.741 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:29.740+0000 7f69cea00700 1 -- 172.21.15.174:0/3438582286 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7f69c8027020 con 0x7f69d8074dc0 2024-03-21T09:57:29.742 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-21T09:57:29.742 INFO:teuthology.orchestra.run.smithi174.stdout:{"epoch":1,"fsid":"070ed3e4-e769-11ee-95cd-87774f69a715","modified":"2024-03-21T09:55:15.397487Z","created":"2024-03-21T09:55:15.397487Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi153","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:3300","nonce":0},{"type":"v1","addr":"172.21.15.153:6789","nonce":0}]},"addr":"172.21.15.153:6789/0","public_addr":"172.21.15.153:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-21T09:57:29.744 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:29.742+0000 7f69dd200700 1 -- 172.21.15.174:0/3438582286 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7f69bc038410 msgr2=0x7f69bc03a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:57:29.744 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:29.742+0000 7f69dd200700 1 --2- 172.21.15.174:0/3438582286 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7f69bc038410 0x7f69bc03a8d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f69c4006fd0 tx=0x7f69c4006e40 comp rx=0 tx=0).stop 2024-03-21T09:57:29.744 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:29.742+0000 7f69dd200700 1 -- 172.21.15.174:0/3438582286 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f69d8074dc0 msgr2=0x7f69d8197be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:57:29.744 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:29.742+0000 7f69dd200700 1 --2- 172.21.15.174:0/3438582286 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f69d8074dc0 0x7f69d8197be0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f69c8005b40 tx=0x7f69c8004740 comp rx=0 tx=0).stop 2024-03-21T09:57:29.744 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:29.742+0000 7f69dd200700 1 -- 172.21.15.174:0/3438582286 shutdown_connections 2024-03-21T09:57:29.744 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:29.742+0000 7f69dd200700 1 --2- 172.21.15.174:0/3438582286 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7f69bc038410 0x7f69bc03a8d0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:57:29.745 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:29.742+0000 7f69dd200700 1 --2- 172.21.15.174:0/3438582286 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f69d8074dc0 0x7f69d8197be0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:57:29.745 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:29.742+0000 7f69dd200700 1 -- 172.21.15.174:0/3438582286 >> 172.21.15.174:0/3438582286 conn(0x7f69d80fbe30 msgr2=0x7f69d80fcb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:57:29.745 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:29.742+0000 7f69dd200700 1 -- 172.21.15.174:0/3438582286 shutdown_connections 2024-03-21T09:57:29.745 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:29.742+0000 7f69dd200700 1 -- 172.21.15.174:0/3438582286 wait complete. 2024-03-21T09:57:29.745 INFO:teuthology.orchestra.run.smithi174.stderr:dumped monmap epoch 1 2024-03-21T09:57:30.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:57:29 smithi153 ceph-mon[29117]: from='mgr.14164 172.21.15.153:0/3445176977' entity='mgr.smithi153.tluxni' 2024-03-21T09:57:30.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:57:29 smithi153 ceph-mon[29117]: from='mgr.14164 172.21.15.153:0/3445176977' entity='mgr.smithi153.tluxni' 2024-03-21T09:57:30.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:57:29 smithi153 ceph-mon[29117]: from='mgr.14164 172.21.15.153:0/3445176977' entity='mgr.smithi153.tluxni' 2024-03-21T09:57:30.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:57:29 smithi153 ceph-mon[29117]: from='mgr.14164 172.21.15.153:0/3445176977' entity='mgr.smithi153.tluxni' 2024-03-21T09:57:30.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:57:29 smithi153 ceph-mon[29117]: from='mgr.14164 172.21.15.153:0/3445176977' entity='mgr.smithi153.tluxni' 2024-03-21T09:57:30.308 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:57:29 smithi153 ceph-mon[29117]: from='mgr.14164 172.21.15.153:0/3445176977' entity='mgr.smithi153.tluxni' 2024-03-21T09:57:30.308 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:57:29 smithi153 ceph-mon[29117]: Regenerating cephadm self-signed grafana TLS certificates 2024-03-21T09:57:30.308 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:57:29 smithi153 ceph-mon[29117]: from='mgr.14164 172.21.15.153:0/3445176977' entity='mgr.smithi153.tluxni' 2024-03-21T09:57:30.308 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:57:29 smithi153 ceph-mon[29117]: from='mgr.14164 172.21.15.153:0/3445176977' entity='mgr.smithi153.tluxni' 2024-03-21T09:57:30.308 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:57:29 smithi153 ceph-mon[29117]: from='mgr.14164 172.21.15.153:0/3445176977' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-03-21T09:57:30.308 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:57:29 smithi153 ceph-mon[29117]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-03-21T09:57:30.308 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:57:29 smithi153 ceph-mon[29117]: from='mgr.14164 172.21.15.153:0/3445176977' entity='mgr.smithi153.tluxni' 2024-03-21T09:57:30.308 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:57:29 smithi153 ceph-mon[29117]: Deploying daemon grafana.smithi153 on smithi153 2024-03-21T09:57:30.308 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:57:29 smithi153 ceph-mon[29117]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:57:30.308 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:57:29 smithi153 ceph-mon[29117]: from='client.? 172.21.15.174:0/3438582286' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-21T09:57:31.451 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-21T09:57:31.452 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 070ed3e4-e769-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-21T09:57:31.832 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-21T09:57:31.927 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-21T09:57:32.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:57:32 smithi153 ceph-mon[29117]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:57:33.792 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:33.788+0000 7f1c1ae00700 1 -- 172.21.15.174:0/1000269848 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c14106fa0 msgr2=0x7f1c141073c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:57:33.792 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:33.788+0000 7f1c1ae00700 1 --2- 172.21.15.174:0/1000269848 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c14106fa0 0x7f1c141073c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1c04009910 tx=0x7f1c04009c20 comp rx=0 tx=0).stop 2024-03-21T09:57:33.792 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:33.788+0000 7f1c1ae00700 1 -- 172.21.15.174:0/1000269848 shutdown_connections 2024-03-21T09:57:33.792 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:33.788+0000 7f1c1ae00700 1 --2- 172.21.15.174:0/1000269848 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c14106fa0 0x7f1c141073c0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:57:33.792 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:33.788+0000 7f1c1ae00700 1 -- 172.21.15.174:0/1000269848 >> 172.21.15.174:0/1000269848 conn(0x7f1c14075ff0 msgr2=0x7f1c14078450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:57:33.793 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:33.788+0000 7f1c1ae00700 1 -- 172.21.15.174:0/1000269848 shutdown_connections 2024-03-21T09:57:33.793 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:33.788+0000 7f1c1ae00700 1 -- 172.21.15.174:0/1000269848 wait complete. 2024-03-21T09:57:33.793 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:33.789+0000 7f1c1ae00700 1 Processor -- start 2024-03-21T09:57:33.793 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:33.790+0000 7f1c1ae00700 1 -- start start 2024-03-21T09:57:33.793 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:33.790+0000 7f1c1ae00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c14106fa0 0x7f1c1419bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:57:33.793 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:33.790+0000 7f1c1ae00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c1419c470 con 0x7f1c14106fa0 2024-03-21T09:57:33.796 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:33.790+0000 7f1c13e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c14106fa0 0x7f1c1419bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:57:33.797 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:33.791+0000 7f1c13e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c14106fa0 0x7f1c1419bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.174:57330/0 (socket says 172.21.15.174:57330) 2024-03-21T09:57:33.797 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:33.791+0000 7f1c13e00700 1 -- 172.21.15.174:0/529107719 learned_addr learned my addr 172.21.15.174:0/529107719 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-03-21T09:57:33.797 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:33.791+0000 7f1c13e00700 1 -- 172.21.15.174:0/529107719 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c040095f0 con 0x7f1c14106fa0 2024-03-21T09:57:33.797 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:33.791+0000 7f1c13e00700 1 --2- 172.21.15.174:0/529107719 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c14106fa0 0x7f1c1419bf30 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1c04005b40 tx=0x7f1c04004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:57:33.797 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:33.792+0000 7f1c11600700 1 -- 172.21.15.174:0/529107719 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1c0401d070 con 0x7f1c14106fa0 2024-03-21T09:57:33.798 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:33.792+0000 7f1c1ae00700 1 -- 172.21.15.174:0/529107719 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c1419c670 con 0x7f1c14106fa0 2024-03-21T09:57:33.798 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:33.792+0000 7f1c11600700 1 -- 172.21.15.174:0/529107719 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f1c0400be10 con 0x7f1c14106fa0 2024-03-21T09:57:33.798 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:33.792+0000 7f1c1ae00700 1 -- 172.21.15.174:0/529107719 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c1419cb10 con 0x7f1c14106fa0 2024-03-21T09:57:33.798 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:33.792+0000 7f1c11600700 1 -- 172.21.15.174:0/529107719 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1c0400f460 con 0x7f1c14106fa0 2024-03-21T09:57:33.798 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:33.793+0000 7f1c1ae00700 1 -- 172.21.15.174:0/529107719 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c14195a20 con 0x7f1c14106fa0 2024-03-21T09:57:33.798 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:33.794+0000 7f1c11600700 1 -- 172.21.15.174:0/529107719 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 13) v1 ==== 45340+0+0 (secure 0 0 0) 0x7f1c040227c0 con 0x7f1c14106fa0 2024-03-21T09:57:33.799 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:33.794+0000 7f1c11600700 1 --2- 172.21.15.174:0/529107719 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7f1bf8038410 0x7f1bf803a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:57:33.799 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:33.794+0000 7f1c11600700 1 -- 172.21.15.174:0/529107719 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f1c0404c790 con 0x7f1c14106fa0 2024-03-21T09:57:33.799 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:33.795+0000 7f1c0be00700 1 --2- 172.21.15.174:0/529107719 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7f1bf8038410 0x7f1bf803a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:57:33.800 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:33.796+0000 7f1c0be00700 1 --2- 172.21.15.174:0/529107719 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7f1bf8038410 0x7f1bf803a8d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc006fd0 tx=0x7f1bfc006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:57:33.810 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:33.808+0000 7f1c11600700 1 -- 172.21.15.174:0/529107719 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f1c0400b960 con 0x7f1c14106fa0 2024-03-21T09:57:34.105 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:34.103+0000 7f1c1ae00700 1 -- 172.21.15.174:0/529107719 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f1c1402cc70 con 0x7f1c14106fa0 2024-03-21T09:57:34.106 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:34.104+0000 7f1c11600700 1 -- 172.21.15.174:0/529107719 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7f1c04027020 con 0x7f1c14106fa0 2024-03-21T09:57:34.106 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-21T09:57:34.106 INFO:teuthology.orchestra.run.smithi174.stdout:{"epoch":1,"fsid":"070ed3e4-e769-11ee-95cd-87774f69a715","modified":"2024-03-21T09:55:15.397487Z","created":"2024-03-21T09:55:15.397487Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi153","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:3300","nonce":0},{"type":"v1","addr":"172.21.15.153:6789","nonce":0}]},"addr":"172.21.15.153:6789/0","public_addr":"172.21.15.153:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-21T09:57:34.109 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:34.107+0000 7f1c1ae00700 1 -- 172.21.15.174:0/529107719 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7f1bf8038410 msgr2=0x7f1bf803a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:57:34.110 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:34.107+0000 7f1c1ae00700 1 --2- 172.21.15.174:0/529107719 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7f1bf8038410 0x7f1bf803a8d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc006fd0 tx=0x7f1bfc006e40 comp rx=0 tx=0).stop 2024-03-21T09:57:34.110 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:34.107+0000 7f1c1ae00700 1 -- 172.21.15.174:0/529107719 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c14106fa0 msgr2=0x7f1c1419bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:57:34.110 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:34.107+0000 7f1c1ae00700 1 --2- 172.21.15.174:0/529107719 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c14106fa0 0x7f1c1419bf30 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1c04005b40 tx=0x7f1c04004740 comp rx=0 tx=0).stop 2024-03-21T09:57:34.110 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:34.107+0000 7f1c1ae00700 1 -- 172.21.15.174:0/529107719 shutdown_connections 2024-03-21T09:57:34.110 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:34.107+0000 7f1c1ae00700 1 --2- 172.21.15.174:0/529107719 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7f1bf8038410 0x7f1bf803a8d0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:57:34.110 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:34.107+0000 7f1c1ae00700 1 --2- 172.21.15.174:0/529107719 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c14106fa0 0x7f1c1419bf30 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:57:34.110 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:34.107+0000 7f1c1ae00700 1 -- 172.21.15.174:0/529107719 >> 172.21.15.174:0/529107719 conn(0x7f1c14075ff0 msgr2=0x7f1c14076cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:57:34.110 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:34.108+0000 7f1c1ae00700 1 -- 172.21.15.174:0/529107719 shutdown_connections 2024-03-21T09:57:34.110 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:34.108+0000 7f1c1ae00700 1 -- 172.21.15.174:0/529107719 wait complete. 2024-03-21T09:57:34.111 INFO:teuthology.orchestra.run.smithi174.stderr:dumped monmap epoch 1 2024-03-21T09:57:35.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:57:34 smithi153 ceph-mon[29117]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:57:35.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:57:34 smithi153 ceph-mon[29117]: from='mgr.14164 172.21.15.153:0/3445176977' entity='mgr.smithi153.tluxni' 2024-03-21T09:57:35.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:57:34 smithi153 ceph-mon[29117]: from='client.? 172.21.15.174:0/529107719' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-21T09:57:35.752 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-21T09:57:35.752 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 070ed3e4-e769-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-21T09:57:36.126 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-21T09:57:36.223 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-21T09:57:37.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:57:36 smithi153 ceph-mon[29117]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:57:38.153 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:38.151+0000 7f5423200700 1 -- 172.21.15.174:0/3555380376 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f54241028d0 msgr2=0x7f5424102cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:57:38.153 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:38.151+0000 7f5423200700 1 --2- 172.21.15.174:0/3555380376 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f54241028d0 0x7f5424102cf0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f5414009910 tx=0x7f5414009c20 comp rx=0 tx=0).stop 2024-03-21T09:57:38.154 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:38.151+0000 7f5423200700 1 -- 172.21.15.174:0/3555380376 shutdown_connections 2024-03-21T09:57:38.154 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:38.151+0000 7f5423200700 1 --2- 172.21.15.174:0/3555380376 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f54241028d0 0x7f5424102cf0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:57:38.154 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:38.151+0000 7f5423200700 1 -- 172.21.15.174:0/3555380376 >> 172.21.15.174:0/3555380376 conn(0x7f54240fde90 msgr2=0x7f54241002b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:57:38.154 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:38.151+0000 7f5423200700 1 -- 172.21.15.174:0/3555380376 shutdown_connections 2024-03-21T09:57:38.154 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:38.151+0000 7f5423200700 1 -- 172.21.15.174:0/3555380376 wait complete. 2024-03-21T09:57:38.154 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:38.153+0000 7f5423200700 1 Processor -- start 2024-03-21T09:57:38.154 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:38.153+0000 7f5423200700 1 -- start start 2024-03-21T09:57:38.155 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:38.153+0000 7f5423200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f54241028d0 0x7f54241979a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:57:38.155 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:38.153+0000 7f5423200700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5424197ee0 con 0x7f54241028d0 2024-03-21T09:57:38.155 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:38.154+0000 7f5421e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f54241028d0 0x7f54241979a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:57:38.155 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:38.154+0000 7f5421e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f54241028d0 0x7f54241979a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.174:57352/0 (socket says 172.21.15.174:57352) 2024-03-21T09:57:38.156 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:38.154+0000 7f5421e00700 1 -- 172.21.15.174:0/572438432 learned_addr learned my addr 172.21.15.174:0/572438432 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-03-21T09:57:38.156 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:38.154+0000 7f5421e00700 1 -- 172.21.15.174:0/572438432 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54140095f0 con 0x7f54241028d0 2024-03-21T09:57:38.156 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:38.155+0000 7f5421e00700 1 --2- 172.21.15.174:0/572438432 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f54241028d0 0x7f54241979a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5414005b40 tx=0x7f5414004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:57:38.157 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:38.155+0000 7f541aa00700 1 -- 172.21.15.174:0/572438432 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f541401d070 con 0x7f54241028d0 2024-03-21T09:57:38.157 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:38.155+0000 7f5423200700 1 -- 172.21.15.174:0/572438432 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54241980e0 con 0x7f54241028d0 2024-03-21T09:57:38.157 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:38.155+0000 7f5423200700 1 -- 172.21.15.174:0/572438432 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5424198580 con 0x7f54241028d0 2024-03-21T09:57:38.157 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:38.155+0000 7f541aa00700 1 -- 172.21.15.174:0/572438432 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f541400be10 con 0x7f54241028d0 2024-03-21T09:57:38.157 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:38.156+0000 7f541aa00700 1 -- 172.21.15.174:0/572438432 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f541400f460 con 0x7f54241028d0 2024-03-21T09:57:38.158 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:38.157+0000 7f5423200700 1 -- 172.21.15.174:0/572438432 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54241914f0 con 0x7f54241028d0 2024-03-21T09:57:38.159 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:38.157+0000 7f541aa00700 1 -- 172.21.15.174:0/572438432 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 13) v1 ==== 45340+0+0 (secure 0 0 0) 0x7f54140227c0 con 0x7f54241028d0 2024-03-21T09:57:38.159 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:38.157+0000 7f541aa00700 1 --2- 172.21.15.174:0/572438432 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7f5408038410 0x7f540803a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:57:38.159 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:38.157+0000 7f541aa00700 1 -- 172.21.15.174:0/572438432 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f541404c2d0 con 0x7f54241028d0 2024-03-21T09:57:38.159 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:38.158+0000 7f5421400700 1 --2- 172.21.15.174:0/572438432 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7f5408038410 0x7f540803a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:57:38.160 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:38.159+0000 7f5421400700 1 --2- 172.21.15.174:0/572438432 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7f5408038410 0x7f540803a8d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f5410006fd0 tx=0x7f5410006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:57:38.176 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:38.175+0000 7f541aa00700 1 -- 172.21.15.174:0/572438432 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f541400b960 con 0x7f54241028d0 2024-03-21T09:57:38.478 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:38.476+0000 7f5423200700 1 -- 172.21.15.174:0/572438432 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f542402cc70 con 0x7f54241028d0 2024-03-21T09:57:38.480 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:38.478+0000 7f541aa00700 1 -- 172.21.15.174:0/572438432 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7f5414027030 con 0x7f54241028d0 2024-03-21T09:57:38.480 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-21T09:57:38.480 INFO:teuthology.orchestra.run.smithi174.stdout:{"epoch":1,"fsid":"070ed3e4-e769-11ee-95cd-87774f69a715","modified":"2024-03-21T09:55:15.397487Z","created":"2024-03-21T09:55:15.397487Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi153","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:3300","nonce":0},{"type":"v1","addr":"172.21.15.153:6789","nonce":0}]},"addr":"172.21.15.153:6789/0","public_addr":"172.21.15.153:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-21T09:57:38.483 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:38.481+0000 7f5423200700 1 -- 172.21.15.174:0/572438432 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7f5408038410 msgr2=0x7f540803a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:57:38.483 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:38.481+0000 7f5423200700 1 --2- 172.21.15.174:0/572438432 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7f5408038410 0x7f540803a8d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f5410006fd0 tx=0x7f5410006e40 comp rx=0 tx=0).stop 2024-03-21T09:57:38.484 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:38.481+0000 7f5423200700 1 -- 172.21.15.174:0/572438432 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f54241028d0 msgr2=0x7f54241979a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:57:38.484 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:38.481+0000 7f5423200700 1 --2- 172.21.15.174:0/572438432 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f54241028d0 0x7f54241979a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5414005b40 tx=0x7f5414004740 comp rx=0 tx=0).stop 2024-03-21T09:57:38.484 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:38.481+0000 7f5423200700 1 -- 172.21.15.174:0/572438432 shutdown_connections 2024-03-21T09:57:38.484 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:38.481+0000 7f5423200700 1 --2- 172.21.15.174:0/572438432 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7f5408038410 0x7f540803a8d0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:57:38.484 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:38.481+0000 7f5423200700 1 --2- 172.21.15.174:0/572438432 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f54241028d0 0x7f54241979a0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:57:38.484 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:38.481+0000 7f5423200700 1 -- 172.21.15.174:0/572438432 >> 172.21.15.174:0/572438432 conn(0x7f54240fde90 msgr2=0x7f54240feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:57:38.484 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:38.481+0000 7f5423200700 1 -- 172.21.15.174:0/572438432 shutdown_connections 2024-03-21T09:57:38.484 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:38.481+0000 7f5423200700 1 -- 172.21.15.174:0/572438432 wait complete. 2024-03-21T09:57:38.485 INFO:teuthology.orchestra.run.smithi174.stderr:dumped monmap epoch 1 2024-03-21T09:57:39.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:57:38 smithi153 ceph-mon[29117]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:57:39.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:57:38 smithi153 ceph-mon[29117]: from='client.? 172.21.15.174:0/572438432' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-21T09:57:40.196 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-21T09:57:40.197 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 070ed3e4-e769-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-21T09:57:40.581 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-21T09:57:40.677 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-21T09:57:41.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:57:40 smithi153 ceph-mon[29117]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:57:42.569 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:42.566+0000 7fd68da00700 1 -- 172.21.15.174:0/2612045819 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd6881008b0 msgr2=0x7fd688100cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:57:42.569 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:42.566+0000 7fd68da00700 1 --2- 172.21.15.174:0/2612045819 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd6881008b0 0x7fd688100cd0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd67800b0a0 tx=0x7fd67800b3b0 comp rx=0 tx=0).stop 2024-03-21T09:57:42.569 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:42.567+0000 7fd68da00700 1 -- 172.21.15.174:0/2612045819 shutdown_connections 2024-03-21T09:57:42.569 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:42.567+0000 7fd68da00700 1 --2- 172.21.15.174:0/2612045819 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd6881008b0 0x7fd688100cd0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:57:42.569 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:42.567+0000 7fd68da00700 1 -- 172.21.15.174:0/2612045819 >> 172.21.15.174:0/2612045819 conn(0x7fd6880fbe30 msgr2=0x7fd6880fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:57:42.569 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:42.567+0000 7fd68da00700 1 -- 172.21.15.174:0/2612045819 shutdown_connections 2024-03-21T09:57:42.569 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:42.567+0000 7fd68da00700 1 -- 172.21.15.174:0/2612045819 wait complete. 2024-03-21T09:57:42.570 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:42.568+0000 7fd68da00700 1 Processor -- start 2024-03-21T09:57:42.570 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:42.568+0000 7fd68da00700 1 -- start start 2024-03-21T09:57:42.570 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:42.569+0000 7fd68da00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd6881008b0 0x7fd688197bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:57:42.570 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:42.569+0000 7fd68da00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd688198110 con 0x7fd6881008b0 2024-03-21T09:57:42.571 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:42.569+0000 7fd686a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd6881008b0 0x7fd688197bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:57:42.571 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:42.569+0000 7fd686a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd6881008b0 0x7fd688197bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.174:55202/0 (socket says 172.21.15.174:55202) 2024-03-21T09:57:42.571 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:42.569+0000 7fd686a00700 1 -- 172.21.15.174:0/2405294466 learned_addr learned my addr 172.21.15.174:0/2405294466 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-03-21T09:57:42.571 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:42.570+0000 7fd686a00700 1 -- 172.21.15.174:0/2405294466 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd678009d00 con 0x7fd6881008b0 2024-03-21T09:57:42.572 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:42.570+0000 7fd686a00700 1 --2- 172.21.15.174:0/2405294466 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd6881008b0 0x7fd688197bd0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd67800b830 tx=0x7fd67800bd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:57:42.572 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:42.571+0000 7fd67fe00700 1 -- 172.21.15.174:0/2405294466 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd67800e040 con 0x7fd6881008b0 2024-03-21T09:57:42.572 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:42.571+0000 7fd68da00700 1 -- 172.21.15.174:0/2405294466 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd688198310 con 0x7fd6881008b0 2024-03-21T09:57:42.573 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:42.571+0000 7fd68da00700 1 -- 172.21.15.174:0/2405294466 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd688198730 con 0x7fd6881008b0 2024-03-21T09:57:42.573 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:42.571+0000 7fd67fe00700 1 -- 172.21.15.174:0/2405294466 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fd678012af0 con 0x7fd6881008b0 2024-03-21T09:57:42.573 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:42.571+0000 7fd67fe00700 1 -- 172.21.15.174:0/2405294466 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd67801d9a0 con 0x7fd6881008b0 2024-03-21T09:57:42.573 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:42.572+0000 7fd68da00700 1 -- 172.21.15.174:0/2405294466 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6881918d0 con 0x7fd6881008b0 2024-03-21T09:57:42.574 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:42.573+0000 7fd67fe00700 1 -- 172.21.15.174:0/2405294466 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 13) v1 ==== 45340+0+0 (secure 0 0 0) 0x7fd678012c60 con 0x7fd6881008b0 2024-03-21T09:57:42.575 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:42.573+0000 7fd67fe00700 1 --2- 172.21.15.174:0/2405294466 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7fd66c038410 0x7fd66c03a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:57:42.575 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:42.573+0000 7fd67fe00700 1 -- 172.21.15.174:0/2405294466 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd67804d4b0 con 0x7fd6881008b0 2024-03-21T09:57:42.575 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:42.574+0000 7fd67e000700 1 --2- 172.21.15.174:0/2405294466 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7fd66c038410 0x7fd66c03a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:57:42.577 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:42.575+0000 7fd67e000700 1 --2- 172.21.15.174:0/2405294466 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7fd66c038410 0x7fd66c03a8d0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd670006fd0 tx=0x7fd670006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:57:42.591 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:42.589+0000 7fd67fe00700 1 -- 172.21.15.174:0/2405294466 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fd67802fd50 con 0x7fd6881008b0 2024-03-21T09:57:42.891 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:42.889+0000 7fd68da00700 1 -- 172.21.15.174:0/2405294466 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fd68802cee0 con 0x7fd6881008b0 2024-03-21T09:57:42.892 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:42.890+0000 7fd67fe00700 1 -- 172.21.15.174:0/2405294466 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7fd67801ced0 con 0x7fd6881008b0 2024-03-21T09:57:42.892 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-21T09:57:42.892 INFO:teuthology.orchestra.run.smithi174.stdout:{"epoch":1,"fsid":"070ed3e4-e769-11ee-95cd-87774f69a715","modified":"2024-03-21T09:55:15.397487Z","created":"2024-03-21T09:55:15.397487Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi153","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:3300","nonce":0},{"type":"v1","addr":"172.21.15.153:6789","nonce":0}]},"addr":"172.21.15.153:6789/0","public_addr":"172.21.15.153:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-21T09:57:42.894 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:42.893+0000 7fd68da00700 1 -- 172.21.15.174:0/2405294466 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7fd66c038410 msgr2=0x7fd66c03a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:57:42.894 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:42.893+0000 7fd68da00700 1 --2- 172.21.15.174:0/2405294466 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7fd66c038410 0x7fd66c03a8d0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd670006fd0 tx=0x7fd670006e40 comp rx=0 tx=0).stop 2024-03-21T09:57:42.895 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:42.893+0000 7fd68da00700 1 -- 172.21.15.174:0/2405294466 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd6881008b0 msgr2=0x7fd688197bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:57:42.895 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:42.893+0000 7fd68da00700 1 --2- 172.21.15.174:0/2405294466 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd6881008b0 0x7fd688197bd0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd67800b830 tx=0x7fd67800bd40 comp rx=0 tx=0).stop 2024-03-21T09:57:42.895 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:42.893+0000 7fd68da00700 1 -- 172.21.15.174:0/2405294466 shutdown_connections 2024-03-21T09:57:42.895 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:42.893+0000 7fd68da00700 1 --2- 172.21.15.174:0/2405294466 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7fd66c038410 0x7fd66c03a8d0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:57:42.895 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:42.893+0000 7fd68da00700 1 --2- 172.21.15.174:0/2405294466 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd6881008b0 0x7fd688197bd0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:57:42.895 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:42.893+0000 7fd68da00700 1 -- 172.21.15.174:0/2405294466 >> 172.21.15.174:0/2405294466 conn(0x7fd6880fbe30 msgr2=0x7fd6880fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:57:42.895 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:42.893+0000 7fd68da00700 1 -- 172.21.15.174:0/2405294466 shutdown_connections 2024-03-21T09:57:42.896 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:42.893+0000 7fd68da00700 1 -- 172.21.15.174:0/2405294466 wait complete. 2024-03-21T09:57:42.896 INFO:teuthology.orchestra.run.smithi174.stderr:dumped monmap epoch 1 2024-03-21T09:57:43.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:57:42 smithi153 ceph-mon[29117]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:57:44.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:57:43 smithi153 ceph-mon[29117]: from='client.? 172.21.15.174:0/2405294466' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-21T09:57:44.527 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-21T09:57:44.527 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 070ed3e4-e769-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-21T09:57:44.902 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-21T09:57:45.003 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-21T09:57:45.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:57:44 smithi153 ceph-mon[29117]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:57:46.816 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:46.813+0000 7f3cdf200700 1 -- 172.21.15.174:0/906315274 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3cd8102a90 msgr2=0x7f3cd8102eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:57:46.816 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:46.813+0000 7f3cdf200700 1 --2- 172.21.15.174:0/906315274 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3cd8102a90 0x7f3cd8102eb0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f3cc8009910 tx=0x7f3cc8009c20 comp rx=0 tx=0).stop 2024-03-21T09:57:46.816 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:46.813+0000 7f3cdf200700 1 -- 172.21.15.174:0/906315274 shutdown_connections 2024-03-21T09:57:46.816 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:46.813+0000 7f3cdf200700 1 --2- 172.21.15.174:0/906315274 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3cd8102a90 0x7f3cd8102eb0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:57:46.816 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:46.813+0000 7f3cdf200700 1 -- 172.21.15.174:0/906315274 >> 172.21.15.174:0/906315274 conn(0x7f3cd80fe010 msgr2=0x7f3cd8100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:57:46.816 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:46.813+0000 7f3cdf200700 1 -- 172.21.15.174:0/906315274 shutdown_connections 2024-03-21T09:57:46.816 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:46.813+0000 7f3cdf200700 1 -- 172.21.15.174:0/906315274 wait complete. 2024-03-21T09:57:46.817 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:46.815+0000 7f3cdf200700 1 Processor -- start 2024-03-21T09:57:46.817 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:46.815+0000 7f3cdf200700 1 -- start start 2024-03-21T09:57:46.817 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:46.816+0000 7f3cdf200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3cd8102a90 0x7f3cd8197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:57:46.817 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:46.816+0000 7f3cdf200700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd81980f0 con 0x7f3cd8102a90 2024-03-21T09:57:46.818 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:46.816+0000 7f3cd7e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3cd8102a90 0x7f3cd8197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:57:46.818 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:46.816+0000 7f3cd7e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3cd8102a90 0x7f3cd8197bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.174:55228/0 (socket says 172.21.15.174:55228) 2024-03-21T09:57:46.818 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:46.816+0000 7f3cd7e00700 1 -- 172.21.15.174:0/2058423343 learned_addr learned my addr 172.21.15.174:0/2058423343 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-03-21T09:57:46.819 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:46.817+0000 7f3cd7e00700 1 -- 172.21.15.174:0/2058423343 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cc80095f0 con 0x7f3cd8102a90 2024-03-21T09:57:46.819 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:46.817+0000 7f3cd7e00700 1 --2- 172.21.15.174:0/2058423343 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3cd8102a90 0x7f3cd8197bb0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3cc8005950 tx=0x7f3cc8009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:57:46.819 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:46.818+0000 7f3cd4c00700 1 -- 172.21.15.174:0/2058423343 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3cc801d070 con 0x7f3cd8102a90 2024-03-21T09:57:46.820 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:46.818+0000 7f3cd4c00700 1 -- 172.21.15.174:0/2058423343 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f3cc800be10 con 0x7f3cd8102a90 2024-03-21T09:57:46.820 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:46.818+0000 7f3cdf200700 1 -- 172.21.15.174:0/2058423343 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cd81982f0 con 0x7f3cd8102a90 2024-03-21T09:57:46.820 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:46.818+0000 7f3cd4c00700 1 -- 172.21.15.174:0/2058423343 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3cc800f460 con 0x7f3cd8102a90 2024-03-21T09:57:46.820 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:46.818+0000 7f3cdf200700 1 -- 172.21.15.174:0/2058423343 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cd8198790 con 0x7f3cd8102a90 2024-03-21T09:57:46.821 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:46.819+0000 7f3cdf200700 1 -- 172.21.15.174:0/2058423343 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cd81916b0 con 0x7f3cd8102a90 2024-03-21T09:57:46.821 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:46.819+0000 7f3cd4c00700 1 -- 172.21.15.174:0/2058423343 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 13) v1 ==== 45340+0+0 (secure 0 0 0) 0x7f3cc80227c0 con 0x7f3cd8102a90 2024-03-21T09:57:46.821 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:46.820+0000 7f3cd4c00700 1 --2- 172.21.15.174:0/2058423343 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7f3cbc038410 0x7f3cbc03a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:57:46.822 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:46.820+0000 7f3cd4c00700 1 -- 172.21.15.174:0/2058423343 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f3cc804c560 con 0x7f3cd8102a90 2024-03-21T09:57:46.822 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:46.820+0000 7f3cd7400700 1 --2- 172.21.15.174:0/2058423343 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7f3cbc038410 0x7f3cbc03a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:57:46.823 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:46.821+0000 7f3cd7400700 1 --2- 172.21.15.174:0/2058423343 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7f3cbc038410 0x7f3cbc03a8d0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f3cc4006fd0 tx=0x7f3cc4006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:57:46.833 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:46.831+0000 7f3cd4c00700 1 -- 172.21.15.174:0/2058423343 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f3cc802ba00 con 0x7f3cd8102a90 2024-03-21T09:57:47.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:57:46 smithi153 ceph-mon[29117]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:57:47.129 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:47.127+0000 7f3cdf200700 1 -- 172.21.15.174:0/2058423343 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f3cd802cc70 con 0x7f3cd8102a90 2024-03-21T09:57:47.130 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:47.128+0000 7f3cd4c00700 1 -- 172.21.15.174:0/2058423343 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7f3cc8027030 con 0x7f3cd8102a90 2024-03-21T09:57:47.130 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-21T09:57:47.130 INFO:teuthology.orchestra.run.smithi174.stdout:{"epoch":1,"fsid":"070ed3e4-e769-11ee-95cd-87774f69a715","modified":"2024-03-21T09:55:15.397487Z","created":"2024-03-21T09:55:15.397487Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi153","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:3300","nonce":0},{"type":"v1","addr":"172.21.15.153:6789","nonce":0}]},"addr":"172.21.15.153:6789/0","public_addr":"172.21.15.153:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-21T09:57:47.133 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:47.131+0000 7f3cdf200700 1 -- 172.21.15.174:0/2058423343 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7f3cbc038410 msgr2=0x7f3cbc03a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:57:47.134 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:47.131+0000 7f3cdf200700 1 --2- 172.21.15.174:0/2058423343 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7f3cbc038410 0x7f3cbc03a8d0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f3cc4006fd0 tx=0x7f3cc4006e40 comp rx=0 tx=0).stop 2024-03-21T09:57:47.134 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:47.131+0000 7f3cdf200700 1 -- 172.21.15.174:0/2058423343 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3cd8102a90 msgr2=0x7f3cd8197bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:57:47.134 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:47.131+0000 7f3cdf200700 1 --2- 172.21.15.174:0/2058423343 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3cd8102a90 0x7f3cd8197bb0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3cc8005950 tx=0x7f3cc8009e60 comp rx=0 tx=0).stop 2024-03-21T09:57:47.134 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:47.131+0000 7f3cdf200700 1 -- 172.21.15.174:0/2058423343 shutdown_connections 2024-03-21T09:57:47.134 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:47.131+0000 7f3cdf200700 1 --2- 172.21.15.174:0/2058423343 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7f3cbc038410 0x7f3cbc03a8d0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:57:47.134 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:47.131+0000 7f3cdf200700 1 --2- 172.21.15.174:0/2058423343 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3cd8102a90 0x7f3cd8197bb0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:57:47.134 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:47.131+0000 7f3cdf200700 1 -- 172.21.15.174:0/2058423343 >> 172.21.15.174:0/2058423343 conn(0x7f3cd80fe010 msgr2=0x7f3cd80fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:57:47.134 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:47.131+0000 7f3cdf200700 1 -- 172.21.15.174:0/2058423343 shutdown_connections 2024-03-21T09:57:47.135 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:47.131+0000 7f3cdf200700 1 -- 172.21.15.174:0/2058423343 wait complete. 2024-03-21T09:57:47.135 INFO:teuthology.orchestra.run.smithi174.stderr:dumped monmap epoch 1 2024-03-21T09:57:48.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:57:47 smithi153 ceph-mon[29117]: from='client.? 172.21.15.174:0/2058423343' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-21T09:57:48.822 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-21T09:57:48.823 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 070ed3e4-e769-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-21T09:57:49.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:57:48 smithi153 ceph-mon[29117]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:57:49.201 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-21T09:57:49.301 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-21T09:57:51.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:57:50 smithi153 ceph-mon[29117]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:57:51.194 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:51.191+0000 7fef98600700 1 -- 172.21.15.174:0/3748719183 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fef90106fa0 msgr2=0x7fef901073c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:57:51.194 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:51.191+0000 7fef98600700 1 --2- 172.21.15.174:0/3748719183 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fef90106fa0 0x7fef901073c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fef84009910 tx=0x7fef84009c20 comp rx=0 tx=0).stop 2024-03-21T09:57:51.194 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:51.192+0000 7fef98600700 1 -- 172.21.15.174:0/3748719183 shutdown_connections 2024-03-21T09:57:51.194 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:51.192+0000 7fef98600700 1 --2- 172.21.15.174:0/3748719183 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fef90106fa0 0x7fef901073c0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:57:51.194 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:51.192+0000 7fef98600700 1 -- 172.21.15.174:0/3748719183 >> 172.21.15.174:0/3748719183 conn(0x7fef90075ff0 msgr2=0x7fef90078450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:57:51.194 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:51.192+0000 7fef98600700 1 -- 172.21.15.174:0/3748719183 shutdown_connections 2024-03-21T09:57:51.194 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:51.192+0000 7fef98600700 1 -- 172.21.15.174:0/3748719183 wait complete. 2024-03-21T09:57:51.195 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:51.193+0000 7fef98600700 1 Processor -- start 2024-03-21T09:57:51.195 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:51.193+0000 7fef98600700 1 -- start start 2024-03-21T09:57:51.195 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:51.194+0000 7fef98600700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fef90106fa0 0x7fef9019bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:57:51.195 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:51.194+0000 7fef98600700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef9019c4e0 con 0x7fef90106fa0 2024-03-21T09:57:51.196 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:51.194+0000 7fef95c00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fef90106fa0 0x7fef9019bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:57:51.196 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:51.194+0000 7fef95c00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fef90106fa0 0x7fef9019bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.174:42432/0 (socket says 172.21.15.174:42432) 2024-03-21T09:57:51.196 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:51.194+0000 7fef95c00700 1 -- 172.21.15.174:0/95722269 learned_addr learned my addr 172.21.15.174:0/95722269 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-03-21T09:57:51.196 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:51.195+0000 7fef95c00700 1 -- 172.21.15.174:0/95722269 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef840095f0 con 0x7fef90106fa0 2024-03-21T09:57:51.197 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:51.195+0000 7fef95c00700 1 --2- 172.21.15.174:0/95722269 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fef90106fa0 0x7fef9019bfa0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fef840092e0 tx=0x7fef84009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:57:51.197 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:51.196+0000 7fef8e000700 1 -- 172.21.15.174:0/95722269 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fef8401d070 con 0x7fef90106fa0 2024-03-21T09:57:51.198 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:51.196+0000 7fef98600700 1 -- 172.21.15.174:0/95722269 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef9019c6e0 con 0x7fef90106fa0 2024-03-21T09:57:51.198 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:51.196+0000 7fef8e000700 1 -- 172.21.15.174:0/95722269 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fef8400be10 con 0x7fef90106fa0 2024-03-21T09:57:51.198 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:51.196+0000 7fef8e000700 1 -- 172.21.15.174:0/95722269 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fef8400f460 con 0x7fef90106fa0 2024-03-21T09:57:51.198 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:51.196+0000 7fef98600700 1 -- 172.21.15.174:0/95722269 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef9019cb80 con 0x7fef90106fa0 2024-03-21T09:57:51.198 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:51.197+0000 7fef98600700 1 -- 172.21.15.174:0/95722269 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef90195a20 con 0x7fef90106fa0 2024-03-21T09:57:51.199 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:51.197+0000 7fef8e000700 1 -- 172.21.15.174:0/95722269 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 13) v1 ==== 45340+0+0 (secure 0 0 0) 0x7fef84022470 con 0x7fef90106fa0 2024-03-21T09:57:51.199 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:51.198+0000 7fef8e000700 1 --2- 172.21.15.174:0/95722269 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7fef780383c0 0x7fef7803a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:57:51.199 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:51.198+0000 7fef8e000700 1 -- 172.21.15.174:0/95722269 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fef8404c620 con 0x7fef90106fa0 2024-03-21T09:57:51.200 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:51.198+0000 7fef95200700 1 --2- 172.21.15.174:0/95722269 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7fef780383c0 0x7fef7803a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:57:51.200 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:51.199+0000 7fef95200700 1 --2- 172.21.15.174:0/95722269 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7fef780383c0 0x7fef7803a880 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fef7c006fd0 tx=0x7fef7c006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:57:51.216 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:51.215+0000 7fef8e000700 1 -- 172.21.15.174:0/95722269 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fef8400b960 con 0x7fef90106fa0 2024-03-21T09:57:51.512 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:51.510+0000 7fef98600700 1 -- 172.21.15.174:0/95722269 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fef9002cc70 con 0x7fef90106fa0 2024-03-21T09:57:51.513 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:51.511+0000 7fef8e000700 1 -- 172.21.15.174:0/95722269 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7fef84027020 con 0x7fef90106fa0 2024-03-21T09:57:51.513 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-21T09:57:51.513 INFO:teuthology.orchestra.run.smithi174.stdout:{"epoch":1,"fsid":"070ed3e4-e769-11ee-95cd-87774f69a715","modified":"2024-03-21T09:55:15.397487Z","created":"2024-03-21T09:55:15.397487Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi153","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:3300","nonce":0},{"type":"v1","addr":"172.21.15.153:6789","nonce":0}]},"addr":"172.21.15.153:6789/0","public_addr":"172.21.15.153:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-21T09:57:51.516 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:51.514+0000 7fef98600700 1 -- 172.21.15.174:0/95722269 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7fef780383c0 msgr2=0x7fef7803a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:57:51.516 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:51.514+0000 7fef98600700 1 --2- 172.21.15.174:0/95722269 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7fef780383c0 0x7fef7803a880 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fef7c006fd0 tx=0x7fef7c006e40 comp rx=0 tx=0).stop 2024-03-21T09:57:51.516 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:51.514+0000 7fef98600700 1 -- 172.21.15.174:0/95722269 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fef90106fa0 msgr2=0x7fef9019bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:57:51.516 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:51.514+0000 7fef98600700 1 --2- 172.21.15.174:0/95722269 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fef90106fa0 0x7fef9019bfa0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fef840092e0 tx=0x7fef84009e60 comp rx=0 tx=0).stop 2024-03-21T09:57:51.516 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:51.514+0000 7fef98600700 1 -- 172.21.15.174:0/95722269 shutdown_connections 2024-03-21T09:57:51.516 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:51.514+0000 7fef98600700 1 --2- 172.21.15.174:0/95722269 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7fef780383c0 0x7fef7803a880 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:57:51.517 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:51.514+0000 7fef98600700 1 --2- 172.21.15.174:0/95722269 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fef90106fa0 0x7fef9019bfa0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:57:51.517 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:51.514+0000 7fef98600700 1 -- 172.21.15.174:0/95722269 >> 172.21.15.174:0/95722269 conn(0x7fef90075ff0 msgr2=0x7fef90076cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:57:51.517 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:51.514+0000 7fef98600700 1 -- 172.21.15.174:0/95722269 shutdown_connections 2024-03-21T09:57:51.517 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:51.514+0000 7fef98600700 1 -- 172.21.15.174:0/95722269 wait complete. 2024-03-21T09:57:51.517 INFO:teuthology.orchestra.run.smithi174.stderr:dumped monmap epoch 1 2024-03-21T09:57:52.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:57:51 smithi153 ceph-mon[29117]: from='client.? 172.21.15.174:0/95722269' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-21T09:57:53.061 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:57:52 smithi153 ceph-mon[29117]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:57:53.243 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-21T09:57:53.244 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 070ed3e4-e769-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-21T09:57:53.620 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-21T09:57:53.719 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-21T09:57:55.258 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:57:54 smithi153 ceph-mon[29117]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:57:55.707 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:55.706+0000 7f8830e00700 1 -- 172.21.15.174:0/1534444232 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f882c0ff180 msgr2=0x7f882c0ff5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:57:55.707 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:55.706+0000 7f8830e00700 1 --2- 172.21.15.174:0/1534444232 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f882c0ff180 0x7f882c0ff5a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f881c009910 tx=0x7f881c009c20 comp rx=0 tx=0).stop 2024-03-21T09:57:55.707 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:55.706+0000 7f8830e00700 1 -- 172.21.15.174:0/1534444232 shutdown_connections 2024-03-21T09:57:55.707 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:55.706+0000 7f8830e00700 1 --2- 172.21.15.174:0/1534444232 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f882c0ff180 0x7f882c0ff5a0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:57:55.708 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:55.706+0000 7f8830e00700 1 -- 172.21.15.174:0/1534444232 >> 172.21.15.174:0/1534444232 conn(0x7f882c0fad20 msgr2=0x7f882c0fd180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:57:55.708 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:55.706+0000 7f8830e00700 1 -- 172.21.15.174:0/1534444232 shutdown_connections 2024-03-21T09:57:55.708 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:55.706+0000 7f8830e00700 1 -- 172.21.15.174:0/1534444232 wait complete. 2024-03-21T09:57:55.708 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:55.707+0000 7f8830e00700 1 Processor -- start 2024-03-21T09:57:55.708 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:55.707+0000 7f8830e00700 1 -- start start 2024-03-21T09:57:55.709 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:55.707+0000 7f8830e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f882c0ff180 0x7f882c197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:57:55.709 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:55.707+0000 7f8830e00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f882c1980a0 con 0x7f882c0ff180 2024-03-21T09:57:55.709 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:55.708+0000 7f8829e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f882c0ff180 0x7f882c197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:57:55.709 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:55.708+0000 7f8829e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f882c0ff180 0x7f882c197b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.174:42446/0 (socket says 172.21.15.174:42446) 2024-03-21T09:57:55.709 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:55.708+0000 7f8829e00700 1 -- 172.21.15.174:0/1047544954 learned_addr learned my addr 172.21.15.174:0/1047544954 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-03-21T09:57:55.710 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:55.708+0000 7f8829e00700 1 -- 172.21.15.174:0/1047544954 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f881c0095f0 con 0x7f882c0ff180 2024-03-21T09:57:55.710 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:55.708+0000 7f8829e00700 1 --2- 172.21.15.174:0/1047544954 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f882c0ff180 0x7f882c197b60 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f881c005b40 tx=0x7f881c004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:57:55.710 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:55.709+0000 7f8822a00700 1 -- 172.21.15.174:0/1047544954 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f881c01d070 con 0x7f882c0ff180 2024-03-21T09:57:55.710 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:55.709+0000 7f8822a00700 1 -- 172.21.15.174:0/1047544954 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f881c00be10 con 0x7f882c0ff180 2024-03-21T09:57:55.710 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:55.709+0000 7f8830e00700 1 -- 172.21.15.174:0/1047544954 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f882c1982a0 con 0x7f882c0ff180 2024-03-21T09:57:55.710 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:55.709+0000 7f8822a00700 1 -- 172.21.15.174:0/1047544954 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f881c00f460 con 0x7f882c0ff180 2024-03-21T09:57:55.710 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:55.709+0000 7f8830e00700 1 -- 172.21.15.174:0/1047544954 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f882c198740 con 0x7f882c0ff180 2024-03-21T09:57:55.711 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:55.710+0000 7f8822a00700 1 -- 172.21.15.174:0/1047544954 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 13) v1 ==== 45340+0+0 (secure 0 0 0) 0x7f881c0227e0 con 0x7f882c0ff180 2024-03-21T09:57:55.711 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:55.710+0000 7f8830e00700 1 -- 172.21.15.174:0/1047544954 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f882c1916b0 con 0x7f882c0ff180 2024-03-21T09:57:55.711 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:55.710+0000 7f8822a00700 1 --2- 172.21.15.174:0/1047544954 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7f88100383c0 0x7f881003a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:57:55.711 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:55.711+0000 7f8822a00700 1 -- 172.21.15.174:0/1047544954 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f881c04c580 con 0x7f882c0ff180 2024-03-21T09:57:55.712 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:55.711+0000 7f8829400700 1 --2- 172.21.15.174:0/1047544954 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7f88100383c0 0x7f881003a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:57:55.712 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:55.712+0000 7f8829400700 1 --2- 172.21.15.174:0/1047544954 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7f88100383c0 0x7f881003a880 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f8818006fd0 tx=0x7f8818006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:57:55.718 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:55.717+0000 7f8822a00700 1 -- 172.21.15.174:0/1047544954 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f881c02bba0 con 0x7f882c0ff180 2024-03-21T09:57:56.015 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:56.014+0000 7f8830e00700 1 -- 172.21.15.174:0/1047544954 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f882c02cc70 con 0x7f882c0ff180 2024-03-21T09:57:56.016 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:56.015+0000 7f8822a00700 1 -- 172.21.15.174:0/1047544954 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7f881c027020 con 0x7f882c0ff180 2024-03-21T09:57:56.016 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-21T09:57:56.016 INFO:teuthology.orchestra.run.smithi174.stdout:{"epoch":1,"fsid":"070ed3e4-e769-11ee-95cd-87774f69a715","modified":"2024-03-21T09:55:15.397487Z","created":"2024-03-21T09:55:15.397487Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi153","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:3300","nonce":0},{"type":"v1","addr":"172.21.15.153:6789","nonce":0}]},"addr":"172.21.15.153:6789/0","public_addr":"172.21.15.153:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-21T09:57:56.019 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:56.018+0000 7f8830e00700 1 -- 172.21.15.174:0/1047544954 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7f88100383c0 msgr2=0x7f881003a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:57:56.019 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:56.018+0000 7f8830e00700 1 --2- 172.21.15.174:0/1047544954 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7f88100383c0 0x7f881003a880 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f8818006fd0 tx=0x7f8818006e40 comp rx=0 tx=0).stop 2024-03-21T09:57:56.019 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:56.018+0000 7f8830e00700 1 -- 172.21.15.174:0/1047544954 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f882c0ff180 msgr2=0x7f882c197b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:57:56.019 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:56.018+0000 7f8830e00700 1 --2- 172.21.15.174:0/1047544954 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f882c0ff180 0x7f882c197b60 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f881c005b40 tx=0x7f881c004740 comp rx=0 tx=0).stop 2024-03-21T09:57:56.019 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:56.018+0000 7f8830e00700 1 -- 172.21.15.174:0/1047544954 shutdown_connections 2024-03-21T09:57:56.020 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:56.018+0000 7f8830e00700 1 --2- 172.21.15.174:0/1047544954 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7f88100383c0 0x7f881003a880 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:57:56.020 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:56.018+0000 7f8830e00700 1 --2- 172.21.15.174:0/1047544954 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f882c0ff180 0x7f882c197b60 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:57:56.020 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:56.018+0000 7f8830e00700 1 -- 172.21.15.174:0/1047544954 >> 172.21.15.174:0/1047544954 conn(0x7f882c0fad20 msgr2=0x7f882c0fba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:57:56.020 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:56.018+0000 7f8830e00700 1 -- 172.21.15.174:0/1047544954 shutdown_connections 2024-03-21T09:57:56.020 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:57:56.019+0000 7f8830e00700 1 -- 172.21.15.174:0/1047544954 wait complete. 2024-03-21T09:57:56.021 INFO:teuthology.orchestra.run.smithi174.stderr:dumped monmap epoch 1 2024-03-21T09:57:56.925 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:57:56 smithi153 ceph-mon[29117]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:57:56.926 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:57:56 smithi153 ceph-mon[29117]: from='client.? 172.21.15.174:0/1047544954' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-21T09:57:57.821 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-21T09:57:57.822 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 070ed3e4-e769-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-21T09:57:58.183 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-21T09:57:58.278 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-21T09:57:59.044 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:57:58 smithi153 ceph-mon[29117]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:58:00.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:00 smithi153 ceph-mon[29117]: from='mgr.14164 172.21.15.153:0/3445176977' entity='mgr.smithi153.tluxni' 2024-03-21T09:58:00.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:00 smithi153 ceph-mon[29117]: from='mgr.14164 172.21.15.153:0/3445176977' entity='mgr.smithi153.tluxni' 2024-03-21T09:58:00.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:00 smithi153 ceph-mon[29117]: from='mgr.14164 172.21.15.153:0/3445176977' entity='mgr.smithi153.tluxni' 2024-03-21T09:58:00.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:00 smithi153 ceph-mon[29117]: from='mgr.14164 172.21.15.153:0/3445176977' entity='mgr.smithi153.tluxni' 2024-03-21T09:58:00.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:00 smithi153 ceph-mon[29117]: from='mgr.14164 172.21.15.153:0/3445176977' entity='mgr.smithi153.tluxni' 2024-03-21T09:58:00.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:00 smithi153 ceph-mon[29117]: from='mgr.14164 172.21.15.153:0/3445176977' entity='mgr.smithi153.tluxni' 2024-03-21T09:58:00.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:00 smithi153 ceph-mon[29117]: from='mgr.14164 172.21.15.153:0/3445176977' entity='mgr.smithi153.tluxni' 2024-03-21T09:58:00.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:00 smithi153 ceph-mon[29117]: from='mgr.14164 172.21.15.153:0/3445176977' entity='mgr.smithi153.tluxni' 2024-03-21T09:58:00.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:00 smithi153 ceph-mon[29117]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:58:00.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:00 smithi153 ceph-mon[29117]: Deploying daemon prometheus.smithi153 on smithi153 2024-03-21T09:58:00.422 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:00.421+0000 7f5647400700 1 -- 172.21.15.174:0/2333229930 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5640101330 msgr2=0x7f5640103720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:58:00.422 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:00.421+0000 7f5647400700 1 --2- 172.21.15.174:0/2333229930 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5640101330 0x7f5640103720 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f5634009910 tx=0x7f5634009c20 comp rx=0 tx=0).stop 2024-03-21T09:58:00.422 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:00.421+0000 7f5647400700 1 -- 172.21.15.174:0/2333229930 shutdown_connections 2024-03-21T09:58:00.422 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:00.421+0000 7f5647400700 1 --2- 172.21.15.174:0/2333229930 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5640101330 0x7f5640103720 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:58:00.422 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:00.421+0000 7f5647400700 1 -- 172.21.15.174:0/2333229930 >> 172.21.15.174:0/2333229930 conn(0x7f56400fad20 msgr2=0x7f56400fd180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:58:00.423 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:00.421+0000 7f5647400700 1 -- 172.21.15.174:0/2333229930 shutdown_connections 2024-03-21T09:58:00.423 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:00.421+0000 7f5647400700 1 -- 172.21.15.174:0/2333229930 wait complete. 2024-03-21T09:58:00.423 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:00.422+0000 7f5647400700 1 Processor -- start 2024-03-21T09:58:00.423 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:00.422+0000 7f5647400700 1 -- start start 2024-03-21T09:58:00.423 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:00.422+0000 7f5647400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5640101330 0x7f5640197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:58:00.423 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:00.422+0000 7f5647400700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56401980e0 con 0x7f5640101330 2024-03-21T09:58:00.424 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:00.423+0000 7f5644a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5640101330 0x7f5640197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:58:00.424 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:00.423+0000 7f5644a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5640101330 0x7f5640197ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.174:42272/0 (socket says 172.21.15.174:42272) 2024-03-21T09:58:00.424 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:00.423+0000 7f5644a00700 1 -- 172.21.15.174:0/170749756 learned_addr learned my addr 172.21.15.174:0/170749756 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-03-21T09:58:00.424 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:00.424+0000 7f5644a00700 1 -- 172.21.15.174:0/170749756 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56340095f0 con 0x7f5640101330 2024-03-21T09:58:00.425 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:00.424+0000 7f5644a00700 1 --2- 172.21.15.174:0/170749756 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5640101330 0x7f5640197ba0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f56340092e0 tx=0x7f5634009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:58:00.425 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:00.424+0000 7f563d600700 1 -- 172.21.15.174:0/170749756 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f563401d070 con 0x7f5640101330 2024-03-21T09:58:00.425 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:00.424+0000 7f5647400700 1 -- 172.21.15.174:0/170749756 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56401982e0 con 0x7f5640101330 2024-03-21T09:58:00.425 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:00.424+0000 7f5647400700 1 -- 172.21.15.174:0/170749756 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5640198780 con 0x7f5640101330 2024-03-21T09:58:00.426 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:00.424+0000 7f563d600700 1 -- 172.21.15.174:0/170749756 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f563400be10 con 0x7f5640101330 2024-03-21T09:58:00.426 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:00.424+0000 7f563d600700 1 -- 172.21.15.174:0/170749756 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f563400f460 con 0x7f5640101330 2024-03-21T09:58:00.426 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:00.426+0000 7f5647400700 1 -- 172.21.15.174:0/170749756 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56401918c0 con 0x7f5640101330 2024-03-21T09:58:00.426 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:00.426+0000 7f563d600700 1 -- 172.21.15.174:0/170749756 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 13) v1 ==== 45340+0+0 (secure 0 0 0) 0x7f5634022470 con 0x7f5640101330 2024-03-21T09:58:00.427 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:00.426+0000 7f563d600700 1 --2- 172.21.15.174:0/170749756 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7f56280383c0 0x7f562803a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:58:00.427 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:00.426+0000 7f563d600700 1 -- 172.21.15.174:0/170749756 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f563404c400 con 0x7f5640101330 2024-03-21T09:58:00.427 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:00.427+0000 7f563fe00700 1 --2- 172.21.15.174:0/170749756 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7f56280383c0 0x7f562803a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:58:00.428 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:00.427+0000 7f563fe00700 1 --2- 172.21.15.174:0/170749756 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7f56280383c0 0x7f562803a880 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f562c006fd0 tx=0x7f562c006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:58:00.436 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:00.435+0000 7f563d600700 1 -- 172.21.15.174:0/170749756 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f563400b960 con 0x7f5640101330 2024-03-21T09:58:00.733 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:00.732+0000 7f5647400700 1 -- 172.21.15.174:0/170749756 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f564002cc70 con 0x7f5640101330 2024-03-21T09:58:00.734 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:00.733+0000 7f563d600700 1 -- 172.21.15.174:0/170749756 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7f5634027090 con 0x7f5640101330 2024-03-21T09:58:00.734 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-21T09:58:00.735 INFO:teuthology.orchestra.run.smithi174.stdout:{"epoch":1,"fsid":"070ed3e4-e769-11ee-95cd-87774f69a715","modified":"2024-03-21T09:55:15.397487Z","created":"2024-03-21T09:55:15.397487Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi153","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:3300","nonce":0},{"type":"v1","addr":"172.21.15.153:6789","nonce":0}]},"addr":"172.21.15.153:6789/0","public_addr":"172.21.15.153:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-21T09:58:00.737 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:00.736+0000 7f5647400700 1 -- 172.21.15.174:0/170749756 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7f56280383c0 msgr2=0x7f562803a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:58:00.737 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:00.736+0000 7f5647400700 1 --2- 172.21.15.174:0/170749756 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7f56280383c0 0x7f562803a880 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f562c006fd0 tx=0x7f562c006e40 comp rx=0 tx=0).stop 2024-03-21T09:58:00.737 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:00.736+0000 7f5647400700 1 -- 172.21.15.174:0/170749756 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5640101330 msgr2=0x7f5640197ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:58:00.737 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:00.736+0000 7f5647400700 1 --2- 172.21.15.174:0/170749756 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5640101330 0x7f5640197ba0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f56340092e0 tx=0x7f5634009e60 comp rx=0 tx=0).stop 2024-03-21T09:58:00.737 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:00.736+0000 7f5647400700 1 -- 172.21.15.174:0/170749756 shutdown_connections 2024-03-21T09:58:00.737 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:00.736+0000 7f5647400700 1 --2- 172.21.15.174:0/170749756 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7f56280383c0 0x7f562803a880 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:58:00.738 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:00.736+0000 7f5647400700 1 --2- 172.21.15.174:0/170749756 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5640101330 0x7f5640197ba0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:58:00.738 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:00.736+0000 7f5647400700 1 -- 172.21.15.174:0/170749756 >> 172.21.15.174:0/170749756 conn(0x7f56400fad20 msgr2=0x7f56400fba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:58:00.738 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:00.736+0000 7f5647400700 1 -- 172.21.15.174:0/170749756 shutdown_connections 2024-03-21T09:58:00.738 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:00.736+0000 7f5647400700 1 -- 172.21.15.174:0/170749756 wait complete. 2024-03-21T09:58:00.739 INFO:teuthology.orchestra.run.smithi174.stderr:dumped monmap epoch 1 2024-03-21T09:58:01.635 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:01 smithi153 ceph-mon[29117]: from='client.? 172.21.15.174:0/170749756' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-21T09:58:02.701 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-21T09:58:02.702 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 070ed3e4-e769-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-21T09:58:02.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:02 smithi153 ceph-mon[29117]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:58:03.071 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-21T09:58:03.169 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-21T09:58:05.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:04 smithi153 ceph-mon[29117]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:58:05.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:04 smithi153 ceph-mon[29117]: from='mgr.14164 172.21.15.153:0/3445176977' entity='mgr.smithi153.tluxni' 2024-03-21T09:58:05.440 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:05.439+0000 7fa563a00700 1 -- 172.21.15.174:0/3389257734 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa55c074dc0 msgr2=0x7fa55c073220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:58:05.440 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:05.439+0000 7fa563a00700 1 --2- 172.21.15.174:0/3389257734 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa55c074dc0 0x7fa55c073220 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa550009910 tx=0x7fa550009c20 comp rx=0 tx=0).stop 2024-03-21T09:58:05.440 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:05.439+0000 7fa563a00700 1 -- 172.21.15.174:0/3389257734 shutdown_connections 2024-03-21T09:58:05.440 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:05.439+0000 7fa563a00700 1 --2- 172.21.15.174:0/3389257734 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa55c074dc0 0x7fa55c073220 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:58:05.440 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:05.439+0000 7fa563a00700 1 -- 172.21.15.174:0/3389257734 >> 172.21.15.174:0/3389257734 conn(0x7fa55c0fbe30 msgr2=0x7fa55c0fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:58:05.440 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:05.439+0000 7fa563a00700 1 -- 172.21.15.174:0/3389257734 shutdown_connections 2024-03-21T09:58:05.440 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:05.439+0000 7fa563a00700 1 -- 172.21.15.174:0/3389257734 wait complete. 2024-03-21T09:58:05.441 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:05.440+0000 7fa563a00700 1 Processor -- start 2024-03-21T09:58:05.441 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:05.440+0000 7fa563a00700 1 -- start start 2024-03-21T09:58:05.441 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:05.441+0000 7fa563a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa55c074dc0 0x7fa55c19c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:58:05.441 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:05.441+0000 7fa563a00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa55c19c690 con 0x7fa55c074dc0 2024-03-21T09:58:05.442 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:05.442+0000 7fa561000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa55c074dc0 0x7fa55c19c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:58:05.442 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:05.442+0000 7fa561000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa55c074dc0 0x7fa55c19c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.174:42296/0 (socket says 172.21.15.174:42296) 2024-03-21T09:58:05.442 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:05.442+0000 7fa561000700 1 -- 172.21.15.174:0/2344274788 learned_addr learned my addr 172.21.15.174:0/2344274788 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-03-21T09:58:05.443 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:05.442+0000 7fa561000700 1 -- 172.21.15.174:0/2344274788 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5500095f0 con 0x7fa55c074dc0 2024-03-21T09:58:05.444 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:05.443+0000 7fa561000700 1 --2- 172.21.15.174:0/2344274788 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa55c074dc0 0x7fa55c19c150 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa550005950 tx=0x7fa550009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:58:05.444 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:05.443+0000 7fa559600700 1 -- 172.21.15.174:0/2344274788 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa55001d070 con 0x7fa55c074dc0 2024-03-21T09:58:05.444 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:05.443+0000 7fa559600700 1 -- 172.21.15.174:0/2344274788 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa55000be10 con 0x7fa55c074dc0 2024-03-21T09:58:05.444 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:05.443+0000 7fa563a00700 1 -- 172.21.15.174:0/2344274788 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa55c1044f0 con 0x7fa55c074dc0 2024-03-21T09:58:05.444 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:05.443+0000 7fa559600700 1 -- 172.21.15.174:0/2344274788 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa55000f460 con 0x7fa55c074dc0 2024-03-21T09:58:05.445 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:05.443+0000 7fa563a00700 1 -- 172.21.15.174:0/2344274788 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa55c19cba0 con 0x7fa55c074dc0 2024-03-21T09:58:05.445 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:05.444+0000 7fa563a00700 1 -- 172.21.15.174:0/2344274788 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa55c195c70 con 0x7fa55c074dc0 2024-03-21T09:58:05.445 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:05.445+0000 7fa559600700 1 -- 172.21.15.174:0/2344274788 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 13) v1 ==== 45340+0+0 (secure 0 0 0) 0x7fa5500227c0 con 0x7fa55c074dc0 2024-03-21T09:58:05.446 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:05.445+0000 7fa559600700 1 --2- 172.21.15.174:0/2344274788 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7fa5440383c0 0x7fa54403a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:58:05.446 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:05.445+0000 7fa559600700 1 -- 172.21.15.174:0/2344274788 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa55004c2f0 con 0x7fa55c074dc0 2024-03-21T09:58:05.447 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:05.446+0000 7fa55be00700 1 --2- 172.21.15.174:0/2344274788 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7fa5440383c0 0x7fa54403a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:58:05.448 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:05.447+0000 7fa55be00700 1 --2- 172.21.15.174:0/2344274788 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7fa5440383c0 0x7fa54403a880 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa548006fd0 tx=0x7fa548006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:58:05.462 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:05.461+0000 7fa559600700 1 -- 172.21.15.174:0/2344274788 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fa55000b960 con 0x7fa55c074dc0 2024-03-21T09:58:05.763 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:05.762+0000 7fa563a00700 1 -- 172.21.15.174:0/2344274788 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fa55c0623c0 con 0x7fa55c074dc0 2024-03-21T09:58:05.764 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:05.763+0000 7fa559600700 1 -- 172.21.15.174:0/2344274788 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7fa550027020 con 0x7fa55c074dc0 2024-03-21T09:58:05.765 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-21T09:58:05.765 INFO:teuthology.orchestra.run.smithi174.stdout:{"epoch":1,"fsid":"070ed3e4-e769-11ee-95cd-87774f69a715","modified":"2024-03-21T09:55:15.397487Z","created":"2024-03-21T09:55:15.397487Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi153","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:3300","nonce":0},{"type":"v1","addr":"172.21.15.153:6789","nonce":0}]},"addr":"172.21.15.153:6789/0","public_addr":"172.21.15.153:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-21T09:58:05.767 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:05.766+0000 7fa563a00700 1 -- 172.21.15.174:0/2344274788 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7fa5440383c0 msgr2=0x7fa54403a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:58:05.768 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:05.766+0000 7fa563a00700 1 --2- 172.21.15.174:0/2344274788 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7fa5440383c0 0x7fa54403a880 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa548006fd0 tx=0x7fa548006e40 comp rx=0 tx=0).stop 2024-03-21T09:58:05.768 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:05.766+0000 7fa563a00700 1 -- 172.21.15.174:0/2344274788 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa55c074dc0 msgr2=0x7fa55c19c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:58:05.768 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:05.766+0000 7fa563a00700 1 --2- 172.21.15.174:0/2344274788 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa55c074dc0 0x7fa55c19c150 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa550005950 tx=0x7fa550009e60 comp rx=0 tx=0).stop 2024-03-21T09:58:05.768 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:05.766+0000 7fa563a00700 1 -- 172.21.15.174:0/2344274788 shutdown_connections 2024-03-21T09:58:05.768 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:05.766+0000 7fa563a00700 1 --2- 172.21.15.174:0/2344274788 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7fa5440383c0 0x7fa54403a880 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:58:05.768 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:05.766+0000 7fa563a00700 1 --2- 172.21.15.174:0/2344274788 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa55c074dc0 0x7fa55c19c150 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:58:05.768 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:05.766+0000 7fa563a00700 1 -- 172.21.15.174:0/2344274788 >> 172.21.15.174:0/2344274788 conn(0x7fa55c0fbe30 msgr2=0x7fa55c0fcb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:58:05.768 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:05.767+0000 7fa563a00700 1 -- 172.21.15.174:0/2344274788 shutdown_connections 2024-03-21T09:58:05.768 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:05.767+0000 7fa563a00700 1 -- 172.21.15.174:0/2344274788 wait complete. 2024-03-21T09:58:05.769 INFO:teuthology.orchestra.run.smithi174.stderr:dumped monmap epoch 1 2024-03-21T09:58:07.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:06 smithi153 ceph-mon[29117]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:58:07.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:06 smithi153 ceph-mon[29117]: from='client.? 172.21.15.174:0/2344274788' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-21T09:58:07.430 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-21T09:58:07.431 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 070ed3e4-e769-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-21T09:58:07.812 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-21T09:58:07.925 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-21T09:58:09.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:08 smithi153 ceph-mon[29117]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:58:09.831 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:09.829+0000 7fee6a200700 1 -- 172.21.15.174:0/2155998777 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee64102a90 msgr2=0x7fee64102eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:58:09.831 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:09.829+0000 7fee6a200700 1 --2- 172.21.15.174:0/2155998777 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee64102a90 0x7fee64102eb0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fee54009910 tx=0x7fee54009c20 comp rx=0 tx=0).stop 2024-03-21T09:58:09.831 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:09.829+0000 7fee6a200700 1 -- 172.21.15.174:0/2155998777 shutdown_connections 2024-03-21T09:58:09.831 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:09.829+0000 7fee6a200700 1 --2- 172.21.15.174:0/2155998777 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee64102a90 0x7fee64102eb0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:58:09.831 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:09.829+0000 7fee6a200700 1 -- 172.21.15.174:0/2155998777 >> 172.21.15.174:0/2155998777 conn(0x7fee640fe010 msgr2=0x7fee64100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:58:09.831 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:09.829+0000 7fee6a200700 1 -- 172.21.15.174:0/2155998777 shutdown_connections 2024-03-21T09:58:09.831 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:09.830+0000 7fee6a200700 1 -- 172.21.15.174:0/2155998777 wait complete. 2024-03-21T09:58:09.832 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:09.831+0000 7fee6a200700 1 Processor -- start 2024-03-21T09:58:09.832 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:09.831+0000 7fee6a200700 1 -- start start 2024-03-21T09:58:09.832 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:09.831+0000 7fee6a200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee64102a90 0x7fee64197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:58:09.832 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:09.831+0000 7fee6a200700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee641980f0 con 0x7fee64102a90 2024-03-21T09:58:09.833 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:09.832+0000 7fee63400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee64102a90 0x7fee64197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:58:09.833 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:09.832+0000 7fee63400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee64102a90 0x7fee64197bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.174:35268/0 (socket says 172.21.15.174:35268) 2024-03-21T09:58:09.833 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:09.832+0000 7fee63400700 1 -- 172.21.15.174:0/602082756 learned_addr learned my addr 172.21.15.174:0/602082756 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-03-21T09:58:09.833 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:09.833+0000 7fee63400700 1 -- 172.21.15.174:0/602082756 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee540095f0 con 0x7fee64102a90 2024-03-21T09:58:09.834 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:09.833+0000 7fee63400700 1 --2- 172.21.15.174:0/602082756 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee64102a90 0x7fee64197bb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fee54005950 tx=0x7fee54009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:58:09.834 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:09.834+0000 7fee60c00700 1 -- 172.21.15.174:0/602082756 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fee5401d070 con 0x7fee64102a90 2024-03-21T09:58:09.834 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:09.834+0000 7fee6a200700 1 -- 172.21.15.174:0/602082756 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee641982f0 con 0x7fee64102a90 2024-03-21T09:58:09.834 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:09.834+0000 7fee6a200700 1 -- 172.21.15.174:0/602082756 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee64198790 con 0x7fee64102a90 2024-03-21T09:58:09.835 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:09.834+0000 7fee60c00700 1 -- 172.21.15.174:0/602082756 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fee5400be10 con 0x7fee64102a90 2024-03-21T09:58:09.835 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:09.834+0000 7fee60c00700 1 -- 172.21.15.174:0/602082756 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fee5400f460 con 0x7fee64102a90 2024-03-21T09:58:09.836 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:09.835+0000 7fee6a200700 1 -- 172.21.15.174:0/602082756 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee641916b0 con 0x7fee64102a90 2024-03-21T09:58:09.836 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:09.835+0000 7fee60c00700 1 -- 172.21.15.174:0/602082756 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 13) v1 ==== 45340+0+0 (secure 0 0 0) 0x7fee540227c0 con 0x7fee64102a90 2024-03-21T09:58:09.836 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:09.836+0000 7fee60c00700 1 --2- 172.21.15.174:0/602082756 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7fee480383c0 0x7fee4803a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:58:09.837 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:09.836+0000 7fee60c00700 1 -- 172.21.15.174:0/602082756 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fee5404c310 con 0x7fee64102a90 2024-03-21T09:58:09.837 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:09.836+0000 7fee5be00700 1 --2- 172.21.15.174:0/602082756 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7fee480383c0 0x7fee4803a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:58:09.838 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:09.837+0000 7fee5be00700 1 --2- 172.21.15.174:0/602082756 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7fee480383c0 0x7fee4803a880 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fee4c006fd0 tx=0x7fee4c006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:58:09.852 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:09.851+0000 7fee60c00700 1 -- 172.21.15.174:0/602082756 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fee5400b960 con 0x7fee64102a90 2024-03-21T09:58:10.148 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:10.147+0000 7fee6a200700 1 -- 172.21.15.174:0/602082756 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fee6402cc70 con 0x7fee64102a90 2024-03-21T09:58:10.149 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:10.149+0000 7fee60c00700 1 -- 172.21.15.174:0/602082756 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7fee54027030 con 0x7fee64102a90 2024-03-21T09:58:10.150 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-21T09:58:10.150 INFO:teuthology.orchestra.run.smithi174.stdout:{"epoch":1,"fsid":"070ed3e4-e769-11ee-95cd-87774f69a715","modified":"2024-03-21T09:55:15.397487Z","created":"2024-03-21T09:55:15.397487Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi153","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:3300","nonce":0},{"type":"v1","addr":"172.21.15.153:6789","nonce":0}]},"addr":"172.21.15.153:6789/0","public_addr":"172.21.15.153:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-21T09:58:10.153 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:10.152+0000 7fee6a200700 1 -- 172.21.15.174:0/602082756 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7fee480383c0 msgr2=0x7fee4803a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:58:10.153 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:10.152+0000 7fee6a200700 1 --2- 172.21.15.174:0/602082756 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7fee480383c0 0x7fee4803a880 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fee4c006fd0 tx=0x7fee4c006e40 comp rx=0 tx=0).stop 2024-03-21T09:58:10.154 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:10.152+0000 7fee6a200700 1 -- 172.21.15.174:0/602082756 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee64102a90 msgr2=0x7fee64197bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:58:10.154 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:10.152+0000 7fee6a200700 1 --2- 172.21.15.174:0/602082756 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee64102a90 0x7fee64197bb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fee54005950 tx=0x7fee54009e60 comp rx=0 tx=0).stop 2024-03-21T09:58:10.154 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:10.152+0000 7fee6a200700 1 -- 172.21.15.174:0/602082756 shutdown_connections 2024-03-21T09:58:10.154 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:10.152+0000 7fee6a200700 1 --2- 172.21.15.174:0/602082756 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7fee480383c0 0x7fee4803a880 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:58:10.154 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:10.152+0000 7fee6a200700 1 --2- 172.21.15.174:0/602082756 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee64102a90 0x7fee64197bb0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:58:10.154 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:10.152+0000 7fee6a200700 1 -- 172.21.15.174:0/602082756 >> 172.21.15.174:0/602082756 conn(0x7fee640fe010 msgr2=0x7fee640fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:58:10.154 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:10.153+0000 7fee6a200700 1 -- 172.21.15.174:0/602082756 shutdown_connections 2024-03-21T09:58:10.154 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:10.153+0000 7fee6a200700 1 -- 172.21.15.174:0/602082756 wait complete. 2024-03-21T09:58:10.154 INFO:teuthology.orchestra.run.smithi174.stderr:dumped monmap epoch 1 2024-03-21T09:58:11.007 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:10 smithi153 ceph-mon[29117]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:58:11.007 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:10 smithi153 ceph-mon[29117]: from='client.? 172.21.15.174:0/602082756' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-21T09:58:11.906 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-21T09:58:11.906 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 070ed3e4-e769-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-21T09:58:12.280 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-21T09:58:12.377 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-21T09:58:13.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:12 smithi153 ceph-mon[29117]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:58:14.263 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:14.261+0000 7f7f94c00700 1 -- 172.21.15.174:0/3969634248 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7f90102aa0 msgr2=0x7f7f90102ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:58:14.263 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:14.261+0000 7f7f94c00700 1 --2- 172.21.15.174:0/3969634248 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7f90102aa0 0x7f7f90102ec0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7f80009910 tx=0x7f7f80009c20 comp rx=0 tx=0).stop 2024-03-21T09:58:14.263 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:14.262+0000 7f7f94c00700 1 -- 172.21.15.174:0/3969634248 shutdown_connections 2024-03-21T09:58:14.263 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:14.262+0000 7f7f94c00700 1 --2- 172.21.15.174:0/3969634248 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7f90102aa0 0x7f7f90102ec0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:58:14.263 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:14.262+0000 7f7f94c00700 1 -- 172.21.15.174:0/3969634248 >> 172.21.15.174:0/3969634248 conn(0x7f7f900fe020 msgr2=0x7f7f90100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:58:14.263 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:14.262+0000 7f7f94c00700 1 -- 172.21.15.174:0/3969634248 shutdown_connections 2024-03-21T09:58:14.263 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:14.262+0000 7f7f94c00700 1 -- 172.21.15.174:0/3969634248 wait complete. 2024-03-21T09:58:14.264 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:14.263+0000 7f7f94c00700 1 Processor -- start 2024-03-21T09:58:14.264 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:14.264+0000 7f7f94c00700 1 -- start start 2024-03-21T09:58:14.265 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:14.264+0000 7f7f94c00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7f90102aa0 0x7f7f90197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:58:14.265 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:14.264+0000 7f7f94c00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f90198100 con 0x7f7f90102aa0 2024-03-21T09:58:14.265 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:14.264+0000 7f7f8de00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7f90102aa0 0x7f7f90197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:58:14.266 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:14.264+0000 7f7f8de00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7f90102aa0 0x7f7f90197bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.174:35288/0 (socket says 172.21.15.174:35288) 2024-03-21T09:58:14.266 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:14.264+0000 7f7f8de00700 1 -- 172.21.15.174:0/1384740171 learned_addr learned my addr 172.21.15.174:0/1384740171 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-03-21T09:58:14.266 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:14.265+0000 7f7f8de00700 1 -- 172.21.15.174:0/1384740171 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f800095f0 con 0x7f7f90102aa0 2024-03-21T09:58:14.266 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:14.265+0000 7f7f8de00700 1 --2- 172.21.15.174:0/1384740171 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7f90102aa0 0x7f7f90197bc0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7f80005b40 tx=0x7f7f80004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:58:14.267 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:14.266+0000 7f7f86a00700 1 -- 172.21.15.174:0/1384740171 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7f8001d070 con 0x7f7f90102aa0 2024-03-21T09:58:14.267 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:14.266+0000 7f7f86a00700 1 -- 172.21.15.174:0/1384740171 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f7f8000be10 con 0x7f7f90102aa0 2024-03-21T09:58:14.267 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:14.266+0000 7f7f94c00700 1 -- 172.21.15.174:0/1384740171 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f90198300 con 0x7f7f90102aa0 2024-03-21T09:58:14.267 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:14.266+0000 7f7f86a00700 1 -- 172.21.15.174:0/1384740171 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7f8000f460 con 0x7f7f90102aa0 2024-03-21T09:58:14.267 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:14.266+0000 7f7f94c00700 1 -- 172.21.15.174:0/1384740171 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f901987a0 con 0x7f7f90102aa0 2024-03-21T09:58:14.268 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:14.267+0000 7f7f86a00700 1 -- 172.21.15.174:0/1384740171 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 13) v1 ==== 45340+0+0 (secure 0 0 0) 0x7f7f800227e0 con 0x7f7f90102aa0 2024-03-21T09:58:14.268 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:14.267+0000 7f7f94c00700 1 -- 172.21.15.174:0/1384740171 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f901916c0 con 0x7f7f90102aa0 2024-03-21T09:58:14.268 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:14.267+0000 7f7f86a00700 1 --2- 172.21.15.174:0/1384740171 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7f7f740383c0 0x7f7f7403a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:58:14.268 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:14.267+0000 7f7f86a00700 1 -- 172.21.15.174:0/1384740171 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f7f8004c470 con 0x7f7f90102aa0 2024-03-21T09:58:14.269 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:14.268+0000 7f7f8d400700 1 --2- 172.21.15.174:0/1384740171 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7f7f740383c0 0x7f7f7403a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:58:14.269 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:14.269+0000 7f7f8d400700 1 --2- 172.21.15.174:0/1384740171 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7f7f740383c0 0x7f7f7403a880 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c006fd0 tx=0x7f7f7c006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:58:14.276 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:14.275+0000 7f7f86a00700 1 -- 172.21.15.174:0/1384740171 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f7f8002bba0 con 0x7f7f90102aa0 2024-03-21T09:58:14.572 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:14.571+0000 7f7f94c00700 1 -- 172.21.15.174:0/1384740171 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f7f9002cc70 con 0x7f7f90102aa0 2024-03-21T09:58:14.573 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:14.572+0000 7f7f86a00700 1 -- 172.21.15.174:0/1384740171 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7f7f80027020 con 0x7f7f90102aa0 2024-03-21T09:58:14.573 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-21T09:58:14.573 INFO:teuthology.orchestra.run.smithi174.stdout:{"epoch":1,"fsid":"070ed3e4-e769-11ee-95cd-87774f69a715","modified":"2024-03-21T09:55:15.397487Z","created":"2024-03-21T09:55:15.397487Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi153","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:3300","nonce":0},{"type":"v1","addr":"172.21.15.153:6789","nonce":0}]},"addr":"172.21.15.153:6789/0","public_addr":"172.21.15.153:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-21T09:58:14.576 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:14.574+0000 7f7f94c00700 1 -- 172.21.15.174:0/1384740171 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7f7f740383c0 msgr2=0x7f7f7403a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:58:14.576 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:14.574+0000 7f7f94c00700 1 --2- 172.21.15.174:0/1384740171 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7f7f740383c0 0x7f7f7403a880 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c006fd0 tx=0x7f7f7c006e40 comp rx=0 tx=0).stop 2024-03-21T09:58:14.576 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:14.574+0000 7f7f94c00700 1 -- 172.21.15.174:0/1384740171 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7f90102aa0 msgr2=0x7f7f90197bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:58:14.576 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:14.574+0000 7f7f94c00700 1 --2- 172.21.15.174:0/1384740171 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7f90102aa0 0x7f7f90197bc0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7f80005b40 tx=0x7f7f80004740 comp rx=0 tx=0).stop 2024-03-21T09:58:14.576 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:14.575+0000 7f7f94c00700 1 -- 172.21.15.174:0/1384740171 shutdown_connections 2024-03-21T09:58:14.576 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:14.575+0000 7f7f94c00700 1 --2- 172.21.15.174:0/1384740171 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7f7f740383c0 0x7f7f7403a880 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:58:14.576 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:14.575+0000 7f7f94c00700 1 --2- 172.21.15.174:0/1384740171 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7f90102aa0 0x7f7f90197bc0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:58:14.576 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:14.575+0000 7f7f94c00700 1 -- 172.21.15.174:0/1384740171 >> 172.21.15.174:0/1384740171 conn(0x7f7f900fe020 msgr2=0x7f7f900fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:58:14.576 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:14.575+0000 7f7f94c00700 1 -- 172.21.15.174:0/1384740171 shutdown_connections 2024-03-21T09:58:14.576 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:14.575+0000 7f7f94c00700 1 -- 172.21.15.174:0/1384740171 wait complete. 2024-03-21T09:58:14.577 INFO:teuthology.orchestra.run.smithi174.stderr:dumped monmap epoch 1 2024-03-21T09:58:15.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:14 smithi153 ceph-mon[29117]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:58:15.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:14 smithi153 ceph-mon[29117]: from='mgr.14164 172.21.15.153:0/3445176977' entity='mgr.smithi153.tluxni' 2024-03-21T09:58:15.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:14 smithi153 ceph-mon[29117]: from='mgr.14164 172.21.15.153:0/3445176977' entity='mgr.smithi153.tluxni' 2024-03-21T09:58:15.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:14 smithi153 ceph-mon[29117]: from='mgr.14164 172.21.15.153:0/3445176977' entity='mgr.smithi153.tluxni' 2024-03-21T09:58:15.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:14 smithi153 ceph-mon[29117]: from='mgr.14164 172.21.15.153:0/3445176977' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-03-21T09:58:15.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:14 smithi153 ceph-mon[29117]: from='client.? 172.21.15.174:0/1384740171' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-21T09:58:16.250 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-21T09:58:16.251 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 070ed3e4-e769-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-21T09:58:16.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:16 smithi153 ceph-mon[29117]: from='mgr.14164 172.21.15.153:0/3445176977' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-03-21T09:58:16.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:16 smithi153 ceph-mon[29117]: mgrmap e14: smithi153.tluxni(active, since 91s) 2024-03-21T09:58:16.620 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-21T09:58:16.719 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-21T09:58:18.545 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:18.544+0000 7f393dc00700 1 -- 172.21.15.174:0/2800256756 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f39381008b0 msgr2=0x7f3938100cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:58:18.545 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:18.544+0000 7f393dc00700 1 --2- 172.21.15.174:0/2800256756 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f39381008b0 0x7f3938100cd0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3928009910 tx=0x7f3928009c20 comp rx=0 tx=0).stop 2024-03-21T09:58:18.546 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:18.544+0000 7f393dc00700 1 -- 172.21.15.174:0/2800256756 shutdown_connections 2024-03-21T09:58:18.546 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:18.544+0000 7f393dc00700 1 --2- 172.21.15.174:0/2800256756 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f39381008b0 0x7f3938100cd0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:58:18.546 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:18.544+0000 7f393dc00700 1 -- 172.21.15.174:0/2800256756 >> 172.21.15.174:0/2800256756 conn(0x7f39380fbe30 msgr2=0x7f39380fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:58:18.546 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:18.544+0000 7f393dc00700 1 -- 172.21.15.174:0/2800256756 shutdown_connections 2024-03-21T09:58:18.546 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:18.544+0000 7f393dc00700 1 -- 172.21.15.174:0/2800256756 wait complete. 2024-03-21T09:58:18.547 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:18.545+0000 7f393dc00700 1 Processor -- start 2024-03-21T09:58:18.547 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:18.545+0000 7f393dc00700 1 -- start start 2024-03-21T09:58:18.547 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:18.546+0000 7f393dc00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f39381008b0 0x7f3938197c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:58:18.547 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:18.546+0000 7f393dc00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3938198150 con 0x7f39381008b0 2024-03-21T09:58:18.547 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:18.546+0000 7f3936a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f39381008b0 0x7f3938197c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:58:18.548 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:18.546+0000 7f3936a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f39381008b0 0x7f3938197c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.174:35304/0 (socket says 172.21.15.174:35304) 2024-03-21T09:58:18.548 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:18.546+0000 7f3936a00700 1 -- 172.21.15.174:0/3892457159 learned_addr learned my addr 172.21.15.174:0/3892457159 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-03-21T09:58:18.548 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:18.547+0000 7f3936a00700 1 -- 172.21.15.174:0/3892457159 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39280095f0 con 0x7f39381008b0 2024-03-21T09:58:18.548 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:18.547+0000 7f3936a00700 1 --2- 172.21.15.174:0/3892457159 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f39381008b0 0x7f3938197c10 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3928005950 tx=0x7f3928009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:58:18.549 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:18.548+0000 7f392f400700 1 -- 172.21.15.174:0/3892457159 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f392801d070 con 0x7f39381008b0 2024-03-21T09:58:18.549 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:18.548+0000 7f392f400700 1 -- 172.21.15.174:0/3892457159 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f392800be10 con 0x7f39381008b0 2024-03-21T09:58:18.549 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:18.548+0000 7f392f400700 1 -- 172.21.15.174:0/3892457159 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f392800f460 con 0x7f39381008b0 2024-03-21T09:58:18.549 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:18.548+0000 7f393dc00700 1 -- 172.21.15.174:0/3892457159 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3938198350 con 0x7f39381008b0 2024-03-21T09:58:18.549 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:18.548+0000 7f393dc00700 1 -- 172.21.15.174:0/3892457159 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39381987f0 con 0x7f39381008b0 2024-03-21T09:58:18.549 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:18.549+0000 7f393dc00700 1 -- 172.21.15.174:0/3892457159 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39381918d0 con 0x7f39381008b0 2024-03-21T09:58:18.550 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:18.549+0000 7f392f400700 1 -- 172.21.15.174:0/3892457159 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 14) v1 ==== 45354+0+0 (secure 0 0 0) 0x7f39280227c0 con 0x7f39381008b0 2024-03-21T09:58:18.551 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:18.550+0000 7f392f400700 1 --2- 172.21.15.174:0/3892457159 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7f391c038410 0x7f391c03a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:58:18.551 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:18.550+0000 7f392f400700 1 -- 172.21.15.174:0/3892457159 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f392804c550 con 0x7f39381008b0 2024-03-21T09:58:18.551 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:18.550+0000 7f3936000700 1 -- 172.21.15.174:0/3892457159 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7f391c038410 msgr2=0x7f391c03a8d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.153:6800/1143522828 2024-03-21T09:58:18.551 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:18.550+0000 7f3936000700 1 --2- 172.21.15.174:0/3892457159 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7f391c038410 0x7f391c03a8d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-21T09:58:18.567 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:18.566+0000 7f392f400700 1 -- 172.21.15.174:0/3892457159 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f392802bba0 con 0x7f39381008b0 2024-03-21T09:58:18.753 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:18.751+0000 7f3936000700 1 -- 172.21.15.174:0/3892457159 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7f391c038410 msgr2=0x7f391c03a8d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.153:6800/1143522828 2024-03-21T09:58:18.753 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:18.751+0000 7f3936000700 1 --2- 172.21.15.174:0/3892457159 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7f391c038410 0x7f391c03a8d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-21T09:58:18.864 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:18.859+0000 7f393dc00700 1 -- 172.21.15.174:0/3892457159 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f3938061a40 con 0x7f39381008b0 2024-03-21T09:58:18.864 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:18.860+0000 7f392f400700 1 -- 172.21.15.174:0/3892457159 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7f3928027030 con 0x7f39381008b0 2024-03-21T09:58:18.865 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-21T09:58:18.865 INFO:teuthology.orchestra.run.smithi174.stdout:{"epoch":1,"fsid":"070ed3e4-e769-11ee-95cd-87774f69a715","modified":"2024-03-21T09:55:15.397487Z","created":"2024-03-21T09:55:15.397487Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi153","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:3300","nonce":0},{"type":"v1","addr":"172.21.15.153:6789","nonce":0}]},"addr":"172.21.15.153:6789/0","public_addr":"172.21.15.153:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-21T09:58:18.866 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:18.863+0000 7f393dc00700 1 -- 172.21.15.174:0/3892457159 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7f391c038410 msgr2=0x7f391c03a8d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-21T09:58:18.866 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:18.863+0000 7f393dc00700 1 --2- 172.21.15.174:0/3892457159 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7f391c038410 0x7f391c03a8d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:58:18.866 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:18.863+0000 7f393dc00700 1 -- 172.21.15.174:0/3892457159 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f39381008b0 msgr2=0x7f3938197c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:58:18.866 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:18.863+0000 7f393dc00700 1 --2- 172.21.15.174:0/3892457159 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f39381008b0 0x7f3938197c10 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3928005950 tx=0x7f3928009e60 comp rx=0 tx=0).stop 2024-03-21T09:58:18.866 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:18.863+0000 7f393dc00700 1 -- 172.21.15.174:0/3892457159 shutdown_connections 2024-03-21T09:58:18.866 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:18.863+0000 7f393dc00700 1 --2- 172.21.15.174:0/3892457159 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7f391c038410 0x7f391c03a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:58:18.866 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:18.863+0000 7f393dc00700 1 --2- 172.21.15.174:0/3892457159 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f39381008b0 0x7f3938197c10 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:58:18.866 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:18.863+0000 7f393dc00700 1 -- 172.21.15.174:0/3892457159 >> 172.21.15.174:0/3892457159 conn(0x7f39380fbe30 msgr2=0x7f39380fcb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:58:18.866 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:18.863+0000 7f393dc00700 1 -- 172.21.15.174:0/3892457159 shutdown_connections 2024-03-21T09:58:18.867 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:18.863+0000 7f393dc00700 1 -- 172.21.15.174:0/3892457159 wait complete. 2024-03-21T09:58:18.867 INFO:teuthology.orchestra.run.smithi174.stderr:dumped monmap epoch 1 2024-03-21T09:58:20.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:19 smithi153 ceph-mon[29117]: from='client.? 172.21.15.174:0/3892457159' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-21T09:58:20.624 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-21T09:58:20.624 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 070ed3e4-e769-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-21T09:58:21.002 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-21T09:58:21.100 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-21T09:58:22.961 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:22.960+0000 7fe937c00700 1 -- 172.21.15.174:0/2983140025 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe930106fa0 msgr2=0x7fe9301073c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:58:22.961 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:22.960+0000 7fe937c00700 1 --2- 172.21.15.174:0/2983140025 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe930106fa0 0x7fe9301073c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe924009910 tx=0x7fe924009c20 comp rx=0 tx=0).stop 2024-03-21T09:58:22.962 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:22.960+0000 7fe937c00700 1 -- 172.21.15.174:0/2983140025 shutdown_connections 2024-03-21T09:58:22.962 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:22.960+0000 7fe937c00700 1 --2- 172.21.15.174:0/2983140025 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe930106fa0 0x7fe9301073c0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:58:22.962 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:22.960+0000 7fe937c00700 1 -- 172.21.15.174:0/2983140025 >> 172.21.15.174:0/2983140025 conn(0x7fe930075ff0 msgr2=0x7fe930078450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:58:22.962 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:22.960+0000 7fe937c00700 1 -- 172.21.15.174:0/2983140025 shutdown_connections 2024-03-21T09:58:22.962 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:22.960+0000 7fe937c00700 1 -- 172.21.15.174:0/2983140025 wait complete. 2024-03-21T09:58:22.962 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:22.961+0000 7fe937c00700 1 Processor -- start 2024-03-21T09:58:22.963 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:22.962+0000 7fe937c00700 1 -- start start 2024-03-21T09:58:22.963 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:22.962+0000 7fe937c00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe930106fa0 0x7fe93019bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:58:22.963 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:22.962+0000 7fe937c00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe93019c4e0 con 0x7fe930106fa0 2024-03-21T09:58:22.963 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:22.963+0000 7fe935200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe930106fa0 0x7fe93019bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:58:22.963 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:22.963+0000 7fe935200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe930106fa0 0x7fe93019bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.174:42268/0 (socket says 172.21.15.174:42268) 2024-03-21T09:58:22.963 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:22.963+0000 7fe935200700 1 -- 172.21.15.174:0/1499334225 learned_addr learned my addr 172.21.15.174:0/1499334225 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-03-21T09:58:22.964 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:22.963+0000 7fe935200700 1 -- 172.21.15.174:0/1499334225 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9240095f0 con 0x7fe930106fa0 2024-03-21T09:58:22.964 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:22.964+0000 7fe935200700 1 --2- 172.21.15.174:0/1499334225 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe930106fa0 0x7fe93019bfa0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe9240092e0 tx=0x7fe924009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:58:22.965 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:22.964+0000 7fe92d600700 1 -- 172.21.15.174:0/1499334225 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe92401d070 con 0x7fe930106fa0 2024-03-21T09:58:22.965 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:22.964+0000 7fe937c00700 1 -- 172.21.15.174:0/1499334225 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe93019c6e0 con 0x7fe930106fa0 2024-03-21T09:58:22.965 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:22.964+0000 7fe92d600700 1 -- 172.21.15.174:0/1499334225 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fe92400be10 con 0x7fe930106fa0 2024-03-21T09:58:22.965 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:22.964+0000 7fe92d600700 1 -- 172.21.15.174:0/1499334225 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe92400f460 con 0x7fe930106fa0 2024-03-21T09:58:22.965 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:22.964+0000 7fe937c00700 1 -- 172.21.15.174:0/1499334225 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe93019cb80 con 0x7fe930106fa0 2024-03-21T09:58:22.966 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:22.965+0000 7fe937c00700 1 -- 172.21.15.174:0/1499334225 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe930195a20 con 0x7fe930106fa0 2024-03-21T09:58:22.966 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:22.966+0000 7fe92d600700 1 -- 172.21.15.174:0/1499334225 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 14) v1 ==== 45354+0+0 (secure 0 0 0) 0x7fe924022470 con 0x7fe930106fa0 2024-03-21T09:58:22.967 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:22.966+0000 7fe92d600700 1 --2- 172.21.15.174:0/1499334225 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7fe918038460 0x7fe91803a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:58:22.967 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:22.966+0000 7fe92d600700 1 -- 172.21.15.174:0/1499334225 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fe92404cbc0 con 0x7fe930106fa0 2024-03-21T09:58:22.967 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:22.966+0000 7fe92fe00700 1 -- 172.21.15.174:0/1499334225 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7fe918038460 msgr2=0x7fe91803a920 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.153:6800/1143522828 2024-03-21T09:58:22.967 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:22.967+0000 7fe92fe00700 1 --2- 172.21.15.174:0/1499334225 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7fe918038460 0x7fe91803a920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-21T09:58:22.985 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:22.984+0000 7fe92d600700 1 -- 172.21.15.174:0/1499334225 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fe92400b960 con 0x7fe930106fa0 2024-03-21T09:58:23.169 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:23.168+0000 7fe92fe00700 1 -- 172.21.15.174:0/1499334225 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7fe918038460 msgr2=0x7fe91803a920 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.153:6800/1143522828 2024-03-21T09:58:23.169 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:23.168+0000 7fe92fe00700 1 --2- 172.21.15.174:0/1499334225 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7fe918038460 0x7fe91803a920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-21T09:58:23.274 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:23.273+0000 7fe937c00700 1 -- 172.21.15.174:0/1499334225 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fe93002cc70 con 0x7fe930106fa0 2024-03-21T09:58:23.276 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:23.275+0000 7fe92d600700 1 -- 172.21.15.174:0/1499334225 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7fe924027030 con 0x7fe930106fa0 2024-03-21T09:58:23.276 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-21T09:58:23.276 INFO:teuthology.orchestra.run.smithi174.stdout:{"epoch":1,"fsid":"070ed3e4-e769-11ee-95cd-87774f69a715","modified":"2024-03-21T09:55:15.397487Z","created":"2024-03-21T09:55:15.397487Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi153","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:3300","nonce":0},{"type":"v1","addr":"172.21.15.153:6789","nonce":0}]},"addr":"172.21.15.153:6789/0","public_addr":"172.21.15.153:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-21T09:58:23.278 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:23.277+0000 7fe937c00700 1 -- 172.21.15.174:0/1499334225 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7fe918038460 msgr2=0x7fe91803a920 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-21T09:58:23.278 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:23.277+0000 7fe937c00700 1 --2- 172.21.15.174:0/1499334225 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7fe918038460 0x7fe91803a920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:58:23.278 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:23.277+0000 7fe937c00700 1 -- 172.21.15.174:0/1499334225 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe930106fa0 msgr2=0x7fe93019bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:58:23.278 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:23.277+0000 7fe937c00700 1 --2- 172.21.15.174:0/1499334225 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe930106fa0 0x7fe93019bfa0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe9240092e0 tx=0x7fe924009e60 comp rx=0 tx=0).stop 2024-03-21T09:58:23.278 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:23.277+0000 7fe937c00700 1 -- 172.21.15.174:0/1499334225 shutdown_connections 2024-03-21T09:58:23.278 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:23.277+0000 7fe937c00700 1 --2- 172.21.15.174:0/1499334225 >> [v2:172.21.15.153:6800/1143522828,v1:172.21.15.153:6801/1143522828] conn(0x7fe918038460 0x7fe91803a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:58:23.278 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:23.277+0000 7fe937c00700 1 --2- 172.21.15.174:0/1499334225 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe930106fa0 0x7fe93019bfa0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:58:23.279 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:23.277+0000 7fe937c00700 1 -- 172.21.15.174:0/1499334225 >> 172.21.15.174:0/1499334225 conn(0x7fe930075ff0 msgr2=0x7fe930076cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:58:23.279 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:23.277+0000 7fe937c00700 1 -- 172.21.15.174:0/1499334225 shutdown_connections 2024-03-21T09:58:23.279 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:23.277+0000 7fe937c00700 1 -- 172.21.15.174:0/1499334225 wait complete. 2024-03-21T09:58:23.279 INFO:teuthology.orchestra.run.smithi174.stderr:dumped monmap epoch 1 2024-03-21T09:58:23.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:23 smithi153 ceph-mon[29117]: from='client.? 172.21.15.174:0/1499334225' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-21T09:58:24.969 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-21T09:58:24.970 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 070ed3e4-e769-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-21T09:58:25.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:24 smithi153 ceph-mon[29117]: Active manager daemon smithi153.tluxni restarted 2024-03-21T09:58:25.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:24 smithi153 ceph-mon[29117]: Activating manager daemon smithi153.tluxni 2024-03-21T09:58:25.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:24 smithi153 ceph-mon[29117]: osdmap e5: 0 total, 0 up, 0 in 2024-03-21T09:58:25.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:24 smithi153 ceph-mon[29117]: mgrmap e15: smithi153.tluxni(active, starting, since 0.0103498s) 2024-03-21T09:58:25.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:24 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mon metadata", "id": "smithi153"}]: dispatch 2024-03-21T09:58:25.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:24 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mgr metadata", "who": "smithi153.tluxni", "id": "smithi153.tluxni"}]: dispatch 2024-03-21T09:58:25.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:24 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-21T09:58:25.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:24 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-21T09:58:25.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:24 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-21T09:58:25.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:24 smithi153 ceph-mon[29117]: Manager daemon smithi153.tluxni is now available 2024-03-21T09:58:25.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:24 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:58:25.058 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:24 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:58:25.058 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:24 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:58:25.058 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:24 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T09:58:25.058 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:24 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi153.tluxni/mirror_snapshot_schedule"}]: dispatch 2024-03-21T09:58:25.058 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:24 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi153.tluxni/trash_purge_schedule"}]: dispatch 2024-03-21T09:58:25.350 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-21T09:58:25.445 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-21T09:58:26.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:26 smithi153 ceph-mon[29117]: [21/Mar/2024:09:58:24] ENGINE Bus STARTING 2024-03-21T09:58:26.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:26 smithi153 ceph-mon[29117]: [21/Mar/2024:09:58:24] ENGINE Serving on http://172.21.15.153:8765 2024-03-21T09:58:26.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:26 smithi153 ceph-mon[29117]: [21/Mar/2024:09:58:24] ENGINE Serving on https://172.21.15.153:7150 2024-03-21T09:58:26.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:26 smithi153 ceph-mon[29117]: [21/Mar/2024:09:58:24] ENGINE Bus STARTED 2024-03-21T09:58:26.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:26 smithi153 ceph-mon[29117]: mgrmap e16: smithi153.tluxni(active, since 1.0153s) 2024-03-21T09:58:26.558 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:26 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:58:27.392 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:27.391+0000 7fdf51800700 1 -- 172.21.15.174:0/1331265036 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdf4c101130 msgr2=0x7fdf4c103520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:58:27.392 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:27.391+0000 7fdf51800700 1 --2- 172.21.15.174:0/1331265036 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdf4c101130 0x7fdf4c103520 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fdf34009910 tx=0x7fdf34009c20 comp rx=0 tx=0).stop 2024-03-21T09:58:27.392 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:27.391+0000 7fdf51800700 1 -- 172.21.15.174:0/1331265036 shutdown_connections 2024-03-21T09:58:27.392 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:27.391+0000 7fdf51800700 1 --2- 172.21.15.174:0/1331265036 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdf4c101130 0x7fdf4c103520 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:58:27.392 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:27.391+0000 7fdf51800700 1 -- 172.21.15.174:0/1331265036 >> 172.21.15.174:0/1331265036 conn(0x7fdf4c0fab20 msgr2=0x7fdf4c0fcf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:58:27.393 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:27.391+0000 7fdf51800700 1 -- 172.21.15.174:0/1331265036 shutdown_connections 2024-03-21T09:58:27.393 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:27.391+0000 7fdf51800700 1 -- 172.21.15.174:0/1331265036 wait complete. 2024-03-21T09:58:27.393 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:27.392+0000 7fdf51800700 1 Processor -- start 2024-03-21T09:58:27.394 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:27.392+0000 7fdf51800700 1 -- start start 2024-03-21T09:58:27.394 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:27.392+0000 7fdf51800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdf4c101130 0x7fdf4c100a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:58:27.394 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:27.392+0000 7fdf51800700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf4c100fd0 con 0x7fdf4c101130 2024-03-21T09:58:27.394 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:27.393+0000 7fdf4be00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdf4c101130 0x7fdf4c100a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:58:27.394 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:27.393+0000 7fdf4be00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdf4c101130 0x7fdf4c100a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.174:42284/0 (socket says 172.21.15.174:42284) 2024-03-21T09:58:27.394 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:27.393+0000 7fdf4be00700 1 -- 172.21.15.174:0/4075299662 learned_addr learned my addr 172.21.15.174:0/4075299662 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-03-21T09:58:27.394 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:27.394+0000 7fdf4be00700 1 -- 172.21.15.174:0/4075299662 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf340095f0 con 0x7fdf4c101130 2024-03-21T09:58:27.395 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:27.394+0000 7fdf4be00700 1 --2- 172.21.15.174:0/4075299662 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdf4c101130 0x7fdf4c100a90 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fdf34005b40 tx=0x7fdf34004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:58:27.395 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:27.395+0000 7fdf49600700 1 -- 172.21.15.174:0/4075299662 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdf3401d070 con 0x7fdf4c101130 2024-03-21T09:58:27.395 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:27.395+0000 7fdf51800700 1 -- 172.21.15.174:0/4075299662 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf4c0ff0e0 con 0x7fdf4c101130 2024-03-21T09:58:27.395 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:27.395+0000 7fdf49600700 1 -- 172.21.15.174:0/4075299662 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fdf3400be10 con 0x7fdf4c101130 2024-03-21T09:58:27.396 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:27.395+0000 7fdf49600700 1 -- 172.21.15.174:0/4075299662 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdf3400f460 con 0x7fdf4c101130 2024-03-21T09:58:27.396 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:27.395+0000 7fdf51800700 1 -- 172.21.15.174:0/4075299662 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf4c0ff580 con 0x7fdf4c101130 2024-03-21T09:58:27.396 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:27.396+0000 7fdf51800700 1 -- 172.21.15.174:0/4075299662 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf4c0623c0 con 0x7fdf4c101130 2024-03-21T09:58:27.397 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:27.396+0000 7fdf49600700 1 -- 172.21.15.174:0/4075299662 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 17) v1 ==== 45398+0+0 (secure 0 0 0) 0x7fdf340227e0 con 0x7fdf4c101130 2024-03-21T09:58:27.397 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:27.397+0000 7fdf49600700 1 --2- 172.21.15.174:0/4075299662 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fdf30038500 0x7fdf3003a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:58:27.397 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:27.397+0000 7fdf49600700 1 -- 172.21.15.174:0/4075299662 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7fdf3404cd80 con 0x7fdf4c101130 2024-03-21T09:58:27.398 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:27.397+0000 7fdf43e00700 1 --2- 172.21.15.174:0/4075299662 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fdf30038500 0x7fdf3003a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:58:27.398 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:27.398+0000 7fdf43e00700 1 --2- 172.21.15.174:0/4075299662 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fdf30038500 0x7fdf3003a9c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c006fd0 tx=0x7fdf3c006e40 comp rx=0 tx=0).ready entity=mgr.14221 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:58:27.409 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:27.408+0000 7fdf49600700 1 -- 172.21.15.174:0/4075299662 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fdf3402bdf0 con 0x7fdf4c101130 2024-03-21T09:58:27.694 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:27.693+0000 7fdf51800700 1 -- 172.21.15.174:0/4075299662 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fdf4c18eca0 con 0x7fdf4c101130 2024-03-21T09:58:27.695 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:27.694+0000 7fdf49600700 1 -- 172.21.15.174:0/4075299662 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7fdf34027030 con 0x7fdf4c101130 2024-03-21T09:58:27.695 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-21T09:58:27.696 INFO:teuthology.orchestra.run.smithi174.stdout:{"epoch":1,"fsid":"070ed3e4-e769-11ee-95cd-87774f69a715","modified":"2024-03-21T09:55:15.397487Z","created":"2024-03-21T09:55:15.397487Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi153","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:3300","nonce":0},{"type":"v1","addr":"172.21.15.153:6789","nonce":0}]},"addr":"172.21.15.153:6789/0","public_addr":"172.21.15.153:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-21T09:58:27.698 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:27.697+0000 7fdf51800700 1 -- 172.21.15.174:0/4075299662 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fdf30038500 msgr2=0x7fdf3003a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:58:27.698 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:27.697+0000 7fdf51800700 1 --2- 172.21.15.174:0/4075299662 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fdf30038500 0x7fdf3003a9c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c006fd0 tx=0x7fdf3c006e40 comp rx=0 tx=0).stop 2024-03-21T09:58:27.698 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:27.697+0000 7fdf51800700 1 -- 172.21.15.174:0/4075299662 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdf4c101130 msgr2=0x7fdf4c100a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:58:27.698 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:27.697+0000 7fdf51800700 1 --2- 172.21.15.174:0/4075299662 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdf4c101130 0x7fdf4c100a90 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fdf34005b40 tx=0x7fdf34004740 comp rx=0 tx=0).stop 2024-03-21T09:58:27.698 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:27.697+0000 7fdf51800700 1 -- 172.21.15.174:0/4075299662 shutdown_connections 2024-03-21T09:58:27.698 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:27.697+0000 7fdf51800700 1 --2- 172.21.15.174:0/4075299662 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fdf30038500 0x7fdf3003a9c0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:58:27.698 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:27.697+0000 7fdf51800700 1 --2- 172.21.15.174:0/4075299662 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdf4c101130 0x7fdf4c100a90 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:58:27.699 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:27.697+0000 7fdf51800700 1 -- 172.21.15.174:0/4075299662 >> 172.21.15.174:0/4075299662 conn(0x7fdf4c0fab20 msgr2=0x7fdf4c0fb800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:58:27.699 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:27.697+0000 7fdf51800700 1 -- 172.21.15.174:0/4075299662 shutdown_connections 2024-03-21T09:58:27.699 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:27.697+0000 7fdf51800700 1 -- 172.21.15.174:0/4075299662 wait complete. 2024-03-21T09:58:27.700 INFO:teuthology.orchestra.run.smithi174.stderr:dumped monmap epoch 1 2024-03-21T09:58:27.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:27 smithi153 ceph-mon[29117]: mgrmap e17: smithi153.tluxni(active, since 2s) 2024-03-21T09:58:27.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:27 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:58:28.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:28 smithi153 ceph-mon[29117]: from='client.? 172.21.15.174:0/4075299662' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-21T09:58:29.333 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-21T09:58:29.333 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 070ed3e4-e769-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-21T09:58:29.810 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:29 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:58:30.362 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-21T09:58:30.461 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-21T09:58:32.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:31 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:58:32.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:31 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:58:32.963 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:32.951+0000 7ff55b000700 1 -- 172.21.15.174:0/3354294035 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff5541008b0 msgr2=0x7ff554100cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:58:32.963 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:32.951+0000 7ff55b000700 1 --2- 172.21.15.174:0/3354294035 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff5541008b0 0x7ff554100cd0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff544009910 tx=0x7ff544009c20 comp rx=0 tx=0).stop 2024-03-21T09:58:32.963 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:32.951+0000 7ff55b000700 1 -- 172.21.15.174:0/3354294035 shutdown_connections 2024-03-21T09:58:32.963 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:32.951+0000 7ff55b000700 1 --2- 172.21.15.174:0/3354294035 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff5541008b0 0x7ff554100cd0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:58:32.963 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:32.951+0000 7ff55b000700 1 -- 172.21.15.174:0/3354294035 >> 172.21.15.174:0/3354294035 conn(0x7ff5540fbe30 msgr2=0x7ff5540fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:58:32.963 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:32.951+0000 7ff55b000700 1 -- 172.21.15.174:0/3354294035 shutdown_connections 2024-03-21T09:58:32.963 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:32.952+0000 7ff55b000700 1 -- 172.21.15.174:0/3354294035 wait complete. 2024-03-21T09:58:32.963 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:32.952+0000 7ff55b000700 1 Processor -- start 2024-03-21T09:58:32.963 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:32.953+0000 7ff55b000700 1 -- start start 2024-03-21T09:58:32.963 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:32.953+0000 7ff55b000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff5541008b0 0x7ff554197be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:58:32.964 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:32.953+0000 7ff55b000700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff554198120 con 0x7ff5541008b0 2024-03-21T09:58:32.964 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:32.954+0000 7ff553e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff5541008b0 0x7ff554197be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:58:32.964 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:32.954+0000 7ff553e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff5541008b0 0x7ff554197be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.174:33512/0 (socket says 172.21.15.174:33512) 2024-03-21T09:58:32.964 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:32.954+0000 7ff553e00700 1 -- 172.21.15.174:0/3835914961 learned_addr learned my addr 172.21.15.174:0/3835914961 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-03-21T09:58:32.964 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:32.954+0000 7ff553e00700 1 -- 172.21.15.174:0/3835914961 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5440095f0 con 0x7ff5541008b0 2024-03-21T09:58:32.964 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:32.955+0000 7ff553e00700 1 --2- 172.21.15.174:0/3835914961 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff5541008b0 0x7ff554197be0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff544005950 tx=0x7ff544009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:58:32.964 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:32.955+0000 7ff551600700 1 -- 172.21.15.174:0/3835914961 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff54401d070 con 0x7ff5541008b0 2024-03-21T09:58:32.964 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:32.955+0000 7ff55b000700 1 -- 172.21.15.174:0/3835914961 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff554198320 con 0x7ff5541008b0 2024-03-21T09:58:32.964 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:32.955+0000 7ff551600700 1 -- 172.21.15.174:0/3835914961 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff54400be10 con 0x7ff5541008b0 2024-03-21T09:58:32.964 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:32.955+0000 7ff551600700 1 -- 172.21.15.174:0/3835914961 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff54400f460 con 0x7ff5541008b0 2024-03-21T09:58:32.964 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:32.955+0000 7ff55b000700 1 -- 172.21.15.174:0/3835914961 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5541987c0 con 0x7ff5541008b0 2024-03-21T09:58:32.964 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:32.956+0000 7ff55b000700 1 -- 172.21.15.174:0/3835914961 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5541918d0 con 0x7ff5541008b0 2024-03-21T09:58:32.964 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:32.957+0000 7ff551600700 1 -- 172.21.15.174:0/3835914961 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 17) v1 ==== 45398+0+0 (secure 0 0 0) 0x7ff5440227c0 con 0x7ff5541008b0 2024-03-21T09:58:32.965 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:32.957+0000 7ff551600700 1 --2- 172.21.15.174:0/3835914961 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7ff5340384b0 0x7ff53403a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:58:32.965 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:32.957+0000 7ff551600700 1 -- 172.21.15.174:0/3835914961 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7ff54404c630 con 0x7ff5541008b0 2024-03-21T09:58:32.965 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:32.958+0000 7ff54be00700 1 --2- 172.21.15.174:0/3835914961 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7ff5340384b0 0x7ff53403a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:58:32.965 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:32.960+0000 7ff54be00700 1 --2- 172.21.15.174:0/3835914961 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7ff5340384b0 0x7ff53403a970 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff53c006fd0 tx=0x7ff53c006e40 comp rx=0 tx=0).ready entity=mgr.14221 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:58:32.974 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:32.973+0000 7ff551600700 1 -- 172.21.15.174:0/3835914961 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7ff54402bba0 con 0x7ff5541008b0 2024-03-21T09:58:33.289 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:33.287+0000 7ff55b000700 1 -- 172.21.15.174:0/3835914961 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7ff55402cc70 con 0x7ff5541008b0 2024-03-21T09:58:33.290 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:33.289+0000 7ff551600700 1 -- 172.21.15.174:0/3835914961 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7ff544027070 con 0x7ff5541008b0 2024-03-21T09:58:33.290 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-21T09:58:33.290 INFO:teuthology.orchestra.run.smithi174.stdout:{"epoch":1,"fsid":"070ed3e4-e769-11ee-95cd-87774f69a715","modified":"2024-03-21T09:55:15.397487Z","created":"2024-03-21T09:55:15.397487Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi153","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:3300","nonce":0},{"type":"v1","addr":"172.21.15.153:6789","nonce":0}]},"addr":"172.21.15.153:6789/0","public_addr":"172.21.15.153:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-21T09:58:33.292 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:33.292+0000 7ff55b000700 1 -- 172.21.15.174:0/3835914961 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7ff5340384b0 msgr2=0x7ff53403a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:58:33.293 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:33.292+0000 7ff55b000700 1 --2- 172.21.15.174:0/3835914961 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7ff5340384b0 0x7ff53403a970 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff53c006fd0 tx=0x7ff53c006e40 comp rx=0 tx=0).stop 2024-03-21T09:58:33.293 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:33.292+0000 7ff55b000700 1 -- 172.21.15.174:0/3835914961 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff5541008b0 msgr2=0x7ff554197be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:58:33.293 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:33.292+0000 7ff55b000700 1 --2- 172.21.15.174:0/3835914961 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff5541008b0 0x7ff554197be0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff544005950 tx=0x7ff544009e60 comp rx=0 tx=0).stop 2024-03-21T09:58:33.293 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:33.292+0000 7ff55b000700 1 -- 172.21.15.174:0/3835914961 shutdown_connections 2024-03-21T09:58:33.293 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:33.292+0000 7ff55b000700 1 --2- 172.21.15.174:0/3835914961 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7ff5340384b0 0x7ff53403a970 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:58:33.293 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:33.292+0000 7ff55b000700 1 --2- 172.21.15.174:0/3835914961 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff5541008b0 0x7ff554197be0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:58:33.294 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:33.292+0000 7ff55b000700 1 -- 172.21.15.174:0/3835914961 >> 172.21.15.174:0/3835914961 conn(0x7ff5540fbe30 msgr2=0x7ff5540fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:58:33.294 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:33.292+0000 7ff55b000700 1 -- 172.21.15.174:0/3835914961 shutdown_connections 2024-03-21T09:58:33.294 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:33.292+0000 7ff55b000700 1 -- 172.21.15.174:0/3835914961 wait complete. 2024-03-21T09:58:33.294 INFO:teuthology.orchestra.run.smithi174.stderr:dumped monmap epoch 1 2024-03-21T09:58:34.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:33 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:58:34.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:33 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:58:34.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:33 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "who": "osd/host:smithi153", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:58:34.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:33 smithi153 ceph-mon[29117]: from='client.? 172.21.15.174:0/3835914961' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-21T09:58:34.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:33 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:58:34.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:33 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:58:34.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:33 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:58:34.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:33 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:58:34.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:33 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "who": "osd/host:smithi174", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:58:34.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:33 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:58:34.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:33 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:58:34.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:33 smithi153 ceph-mon[29117]: Updating smithi153:/etc/ceph/ceph.conf 2024-03-21T09:58:34.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:33 smithi153 ceph-mon[29117]: Updating smithi174:/etc/ceph/ceph.conf 2024-03-21T09:58:34.901 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-21T09:58:34.902 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 070ed3e4-e769-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-21T09:58:35.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:34 smithi153 ceph-mon[29117]: Updating smithi174:/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T09:58:35.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:34 smithi153 ceph-mon[29117]: Updating smithi153:/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T09:58:35.257 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T09:58:36.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:35 smithi153 ceph-mon[29117]: Updating smithi153:/etc/ceph/ceph.client.admin.keyring 2024-03-21T09:58:36.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:35 smithi153 ceph-mon[29117]: Updating smithi174:/etc/ceph/ceph.client.admin.keyring 2024-03-21T09:58:36.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:35 smithi153 ceph-mon[29117]: Updating smithi174:/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-21T09:58:36.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:35 smithi153 ceph-mon[29117]: Updating smithi153:/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-21T09:58:37.321 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:37.318+0000 7f6f9a400700 1 -- 172.21.15.174:0/1103163425 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6f94107090 msgr2=0x7f6f94107470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:58:37.321 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:37.318+0000 7f6f9a400700 1 --2- 172.21.15.174:0/1103163425 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6f94107090 0x7f6f94107470 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f6f84009910 tx=0x7f6f84009c20 comp rx=0 tx=0).stop 2024-03-21T09:58:37.321 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:37.318+0000 7f6f9a400700 1 -- 172.21.15.174:0/1103163425 shutdown_connections 2024-03-21T09:58:37.321 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:37.318+0000 7f6f9a400700 1 --2- 172.21.15.174:0/1103163425 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6f94107090 0x7f6f94107470 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:58:37.321 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:37.318+0000 7f6f9a400700 1 -- 172.21.15.174:0/1103163425 >> 172.21.15.174:0/1103163425 conn(0x7f6f94076660 msgr2=0x7f6f94078a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:58:37.321 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:37.318+0000 7f6f9a400700 1 -- 172.21.15.174:0/1103163425 shutdown_connections 2024-03-21T09:58:37.321 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:37.318+0000 7f6f9a400700 1 -- 172.21.15.174:0/1103163425 wait complete. 2024-03-21T09:58:37.322 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:37.319+0000 7f6f9a400700 1 Processor -- start 2024-03-21T09:58:37.322 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:37.320+0000 7f6f9a400700 1 -- start start 2024-03-21T09:58:37.322 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:37.320+0000 7f6f9a400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6f94107090 0x7f6f9419c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:58:37.322 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:37.320+0000 7f6f9a400700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9419c860 con 0x7f6f94107090 2024-03-21T09:58:37.323 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:37.321+0000 7f6f93400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6f94107090 0x7f6f9419c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:58:37.323 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:37.321+0000 7f6f93400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6f94107090 0x7f6f9419c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.174:33538/0 (socket says 172.21.15.174:33538) 2024-03-21T09:58:37.323 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:37.321+0000 7f6f93400700 1 -- 172.21.15.174:0/817017622 learned_addr learned my addr 172.21.15.174:0/817017622 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-03-21T09:58:37.323 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:37.322+0000 7f6f93400700 1 -- 172.21.15.174:0/817017622 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f840095f0 con 0x7f6f94107090 2024-03-21T09:58:37.324 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:37.322+0000 7f6f93400700 1 --2- 172.21.15.174:0/817017622 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6f94107090 0x7f6f9419c320 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6f84005b40 tx=0x7f6f84004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:58:37.324 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:37.323+0000 7f6f8be00700 1 -- 172.21.15.174:0/817017622 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6f8401d070 con 0x7f6f94107090 2024-03-21T09:58:37.325 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:37.323+0000 7f6f9a400700 1 -- 172.21.15.174:0/817017622 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f9419ca60 con 0x7f6f94107090 2024-03-21T09:58:37.325 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:37.323+0000 7f6f8be00700 1 -- 172.21.15.174:0/817017622 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f6f840044d0 con 0x7f6f94107090 2024-03-21T09:58:37.325 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:37.323+0000 7f6f8be00700 1 -- 172.21.15.174:0/817017622 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6f84003930 con 0x7f6f94107090 2024-03-21T09:58:37.325 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:37.323+0000 7f6f9a400700 1 -- 172.21.15.174:0/817017622 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f9419ce20 con 0x7f6f94107090 2024-03-21T09:58:37.325 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:37.324+0000 7f6f9a400700 1 -- 172.21.15.174:0/817017622 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f940623c0 con 0x7f6f94107090 2024-03-21T09:58:37.326 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:37.324+0000 7f6f8be00700 1 -- 172.21.15.174:0/817017622 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 17) v1 ==== 45398+0+0 (secure 0 0 0) 0x7f6f8400b960 con 0x7f6f94107090 2024-03-21T09:58:37.326 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:37.325+0000 7f6f8be00700 1 --2- 172.21.15.174:0/817017622 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f6f78038500 0x7f6f7803a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:58:37.326 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:37.325+0000 7f6f8be00700 1 -- 172.21.15.174:0/817017622 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f6f8404c480 con 0x7f6f94107090 2024-03-21T09:58:37.327 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:37.325+0000 7f6f92a00700 1 --2- 172.21.15.174:0/817017622 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f6f78038500 0x7f6f7803a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:58:37.328 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:37.327+0000 7f6f92a00700 1 --2- 172.21.15.174:0/817017622 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f6f78038500 0x7f6f7803a9c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6f80006fd0 tx=0x7f6f80006e40 comp rx=0 tx=0).ready entity=mgr.14221 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:58:37.344 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:37.342+0000 7f6f8be00700 1 -- 172.21.15.174:0/817017622 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f6f8402a540 con 0x7f6f94107090 2024-03-21T09:58:37.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:37 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:58:37.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:37 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:58:37.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:37 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:58:37.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:37 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:58:37.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:37 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:58:37.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:37 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi174", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-21T09:58:37.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:37 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi174", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-03-21T09:58:37.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:37 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:58:37.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:37 smithi153 ceph-mon[29117]: Deploying daemon ceph-exporter.smithi174 on smithi174 2024-03-21T09:58:37.645 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:37.643+0000 7f6f9a400700 1 -- 172.21.15.174:0/817017622 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f6f9410eeb0 con 0x7f6f94107090 2024-03-21T09:58:37.646 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:37.644+0000 7f6f8be00700 1 -- 172.21.15.174:0/817017622 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7f6f8402a360 con 0x7f6f94107090 2024-03-21T09:58:37.647 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-21T09:58:37.647 INFO:teuthology.orchestra.run.smithi174.stdout:{"epoch":1,"fsid":"070ed3e4-e769-11ee-95cd-87774f69a715","modified":"2024-03-21T09:55:15.397487Z","created":"2024-03-21T09:55:15.397487Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi153","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:3300","nonce":0},{"type":"v1","addr":"172.21.15.153:6789","nonce":0}]},"addr":"172.21.15.153:6789/0","public_addr":"172.21.15.153:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-21T09:58:37.649 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:37.647+0000 7f6f9a400700 1 -- 172.21.15.174:0/817017622 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f6f78038500 msgr2=0x7f6f7803a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:58:37.649 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:37.647+0000 7f6f9a400700 1 --2- 172.21.15.174:0/817017622 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f6f78038500 0x7f6f7803a9c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6f80006fd0 tx=0x7f6f80006e40 comp rx=0 tx=0).stop 2024-03-21T09:58:37.649 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:37.647+0000 7f6f9a400700 1 -- 172.21.15.174:0/817017622 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6f94107090 msgr2=0x7f6f9419c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:58:37.649 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:37.647+0000 7f6f9a400700 1 --2- 172.21.15.174:0/817017622 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6f94107090 0x7f6f9419c320 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6f84005b40 tx=0x7f6f84004c80 comp rx=0 tx=0).stop 2024-03-21T09:58:37.649 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:37.647+0000 7f6f9a400700 1 -- 172.21.15.174:0/817017622 shutdown_connections 2024-03-21T09:58:37.649 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:37.647+0000 7f6f9a400700 1 --2- 172.21.15.174:0/817017622 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f6f78038500 0x7f6f7803a9c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:58:37.650 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:37.647+0000 7f6f9a400700 1 --2- 172.21.15.174:0/817017622 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6f94107090 0x7f6f9419c320 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:58:37.650 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:37.647+0000 7f6f9a400700 1 -- 172.21.15.174:0/817017622 >> 172.21.15.174:0/817017622 conn(0x7f6f94076660 msgr2=0x7f6f9410b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:58:37.650 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:37.647+0000 7f6f9a400700 1 -- 172.21.15.174:0/817017622 shutdown_connections 2024-03-21T09:58:37.650 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:37.647+0000 7f6f9a400700 1 -- 172.21.15.174:0/817017622 wait complete. 2024-03-21T09:58:37.650 INFO:teuthology.orchestra.run.smithi174.stderr:dumped monmap epoch 1 2024-03-21T09:58:38.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:38 smithi153 ceph-mon[29117]: from='client.? 172.21.15.174:0/817017622' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-21T09:58:39.264 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-21T09:58:39.265 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 070ed3e4-e769-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-21T09:58:39.824 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T09:58:40.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:39 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T09:58:42.365 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:42.362+0000 7f6558a00700 1 -- 172.21.15.174:0/1904202256 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f65540731c0 msgr2=0x7f65540735a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:58:42.365 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:42.362+0000 7f6558a00700 1 --2- 172.21.15.174:0/1904202256 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f65540731c0 0x7f65540735a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6544009910 tx=0x7f6544009c20 comp rx=0 tx=0).stop 2024-03-21T09:58:42.366 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:42.362+0000 7f6558a00700 1 -- 172.21.15.174:0/1904202256 shutdown_connections 2024-03-21T09:58:42.366 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:42.362+0000 7f6558a00700 1 --2- 172.21.15.174:0/1904202256 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f65540731c0 0x7f65540735a0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:58:42.366 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:42.362+0000 7f6558a00700 1 -- 172.21.15.174:0/1904202256 >> 172.21.15.174:0/1904202256 conn(0x7f65540fc380 msgr2=0x7f65540fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:58:42.366 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:42.362+0000 7f6558a00700 1 -- 172.21.15.174:0/1904202256 shutdown_connections 2024-03-21T09:58:42.366 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:42.362+0000 7f6558a00700 1 -- 172.21.15.174:0/1904202256 wait complete. 2024-03-21T09:58:42.366 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:42.363+0000 7f6558a00700 1 Processor -- start 2024-03-21T09:58:42.366 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:42.363+0000 7f6558a00700 1 -- start start 2024-03-21T09:58:42.367 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:42.364+0000 7f6558a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f65540731c0 0x7f6554197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:58:42.367 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:42.364+0000 7f6558a00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6554198510 con 0x7f65540731c0 2024-03-21T09:58:42.367 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:42.365+0000 7f6551e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f65540731c0 0x7f6554197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:58:42.367 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:42.365+0000 7f6551e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f65540731c0 0x7f6554197fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.174:52928/0 (socket says 172.21.15.174:52928) 2024-03-21T09:58:42.367 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:42.365+0000 7f6551e00700 1 -- 172.21.15.174:0/1306174926 learned_addr learned my addr 172.21.15.174:0/1306174926 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-03-21T09:58:42.367 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:42.365+0000 7f6551e00700 1 -- 172.21.15.174:0/1306174926 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65440095f0 con 0x7f65540731c0 2024-03-21T09:58:42.368 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:42.366+0000 7f6551e00700 1 --2- 172.21.15.174:0/1306174926 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f65540731c0 0x7f6554197fd0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f65440092e0 tx=0x7f6544004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:58:42.368 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:42.366+0000 7f654aa00700 1 -- 172.21.15.174:0/1306174926 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f654401d070 con 0x7f65540731c0 2024-03-21T09:58:42.368 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:42.366+0000 7f6558a00700 1 -- 172.21.15.174:0/1306174926 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6554198710 con 0x7f65540731c0 2024-03-21T09:58:42.369 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:42.366+0000 7f654aa00700 1 -- 172.21.15.174:0/1306174926 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f654400be10 con 0x7f65540731c0 2024-03-21T09:58:42.369 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:42.366+0000 7f654aa00700 1 -- 172.21.15.174:0/1306174926 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6544022e10 con 0x7f65540731c0 2024-03-21T09:58:42.369 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:42.366+0000 7f6558a00700 1 -- 172.21.15.174:0/1306174926 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6554198b30 con 0x7f65540731c0 2024-03-21T09:58:42.369 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:42.367+0000 7f6558a00700 1 -- 172.21.15.174:0/1306174926 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65540623c0 con 0x7f65540731c0 2024-03-21T09:58:42.370 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:42.368+0000 7f654aa00700 1 -- 172.21.15.174:0/1306174926 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 17) v1 ==== 45398+0+0 (secure 0 0 0) 0x7f6544022470 con 0x7f65540731c0 2024-03-21T09:58:42.370 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:42.368+0000 7f654aa00700 1 --2- 172.21.15.174:0/1306174926 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f6538038460 0x7f653803a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:58:42.370 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:42.368+0000 7f654aa00700 1 -- 172.21.15.174:0/1306174926 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f654404c560 con 0x7f65540731c0 2024-03-21T09:58:42.370 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:42.369+0000 7f6551400700 1 --2- 172.21.15.174:0/1306174926 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f6538038460 0x7f653803a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:58:42.371 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:42.370+0000 7f6551400700 1 --2- 172.21.15.174:0/1306174926 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f6538038460 0x7f653803a920 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6540006fd0 tx=0x7f6540006e40 comp rx=0 tx=0).ready entity=mgr.14221 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:58:42.385 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:42.383+0000 7f654aa00700 1 -- 172.21.15.174:0/1306174926 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f654402a540 con 0x7f65540731c0 2024-03-21T09:58:42.696 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:42.694+0000 7f6558a00700 1 -- 172.21.15.174:0/1306174926 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f6554109d10 con 0x7f65540731c0 2024-03-21T09:58:42.697 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:42.695+0000 7f654aa00700 1 -- 172.21.15.174:0/1306174926 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7f6544027030 con 0x7f65540731c0 2024-03-21T09:58:42.697 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-21T09:58:42.697 INFO:teuthology.orchestra.run.smithi174.stdout:{"epoch":1,"fsid":"070ed3e4-e769-11ee-95cd-87774f69a715","modified":"2024-03-21T09:55:15.397487Z","created":"2024-03-21T09:55:15.397487Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi153","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:3300","nonce":0},{"type":"v1","addr":"172.21.15.153:6789","nonce":0}]},"addr":"172.21.15.153:6789/0","public_addr":"172.21.15.153:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-21T09:58:42.701 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:42.699+0000 7f6558a00700 1 -- 172.21.15.174:0/1306174926 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f6538038460 msgr2=0x7f653803a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:58:42.702 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:42.699+0000 7f6558a00700 1 --2- 172.21.15.174:0/1306174926 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f6538038460 0x7f653803a920 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6540006fd0 tx=0x7f6540006e40 comp rx=0 tx=0).stop 2024-03-21T09:58:42.702 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:42.699+0000 7f6558a00700 1 -- 172.21.15.174:0/1306174926 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f65540731c0 msgr2=0x7f6554197fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:58:42.702 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:42.699+0000 7f6558a00700 1 --2- 172.21.15.174:0/1306174926 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f65540731c0 0x7f6554197fd0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f65440092e0 tx=0x7f6544004c80 comp rx=0 tx=0).stop 2024-03-21T09:58:42.702 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:42.699+0000 7f6558a00700 1 -- 172.21.15.174:0/1306174926 shutdown_connections 2024-03-21T09:58:42.702 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:42.699+0000 7f6558a00700 1 --2- 172.21.15.174:0/1306174926 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f6538038460 0x7f653803a920 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:58:42.702 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:42.699+0000 7f6558a00700 1 --2- 172.21.15.174:0/1306174926 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f65540731c0 0x7f6554197fd0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:58:42.702 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:42.699+0000 7f6558a00700 1 -- 172.21.15.174:0/1306174926 >> 172.21.15.174:0/1306174926 conn(0x7f65540fc380 msgr2=0x7f6554107370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:58:42.702 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:42.699+0000 7f6558a00700 1 -- 172.21.15.174:0/1306174926 shutdown_connections 2024-03-21T09:58:42.702 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:42.699+0000 7f6558a00700 1 -- 172.21.15.174:0/1306174926 wait complete. 2024-03-21T09:58:42.703 INFO:teuthology.orchestra.run.smithi174.stderr:dumped monmap epoch 1 2024-03-21T09:58:43.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:42 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:58:43.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:42 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:58:43.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:42 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:58:43.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:42 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:58:43.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:42 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi174", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-21T09:58:43.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:42 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi174", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-03-21T09:58:43.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:42 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:58:43.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:42 smithi153 ceph-mon[29117]: Deploying daemon crash.smithi174 on smithi174 2024-03-21T09:58:43.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:42 smithi153 ceph-mon[29117]: from='client.? 172.21.15.174:0/1306174926' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-21T09:58:44.722 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-21T09:58:44.723 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 070ed3e4-e769-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-21T09:58:45.314 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T09:58:45.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:45 smithi153 ceph-mon[29117]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:58:45.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:45 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:58:47.431 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:47.428+0000 7f5c6ce00700 1 -- 172.21.15.174:0/4126359164 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5c680731c0 msgr2=0x7f5c680735a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:58:47.431 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:47.428+0000 7f5c6ce00700 1 --2- 172.21.15.174:0/4126359164 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5c680731c0 0x7f5c680735a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5c58009910 tx=0x7f5c58009c20 comp rx=0 tx=0).stop 2024-03-21T09:58:47.431 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:47.429+0000 7f5c6ce00700 1 -- 172.21.15.174:0/4126359164 shutdown_connections 2024-03-21T09:58:47.431 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:47.429+0000 7f5c6ce00700 1 --2- 172.21.15.174:0/4126359164 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5c680731c0 0x7f5c680735a0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:58:47.431 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:47.429+0000 7f5c6ce00700 1 -- 172.21.15.174:0/4126359164 >> 172.21.15.174:0/4126359164 conn(0x7f5c680fc380 msgr2=0x7f5c680fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:58:47.431 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:47.429+0000 7f5c6ce00700 1 -- 172.21.15.174:0/4126359164 shutdown_connections 2024-03-21T09:58:47.432 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:47.429+0000 7f5c6ce00700 1 -- 172.21.15.174:0/4126359164 wait complete. 2024-03-21T09:58:47.432 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:47.430+0000 7f5c6ce00700 1 Processor -- start 2024-03-21T09:58:47.432 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:47.430+0000 7f5c6ce00700 1 -- start start 2024-03-21T09:58:47.432 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:47.430+0000 7f5c6ce00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5c680731c0 0x7f5c68197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:58:47.432 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:47.431+0000 7f5c6ce00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c68198510 con 0x7f5c680731c0 2024-03-21T09:58:47.433 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:47.431+0000 7f5c65e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5c680731c0 0x7f5c68197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:58:47.433 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:47.431+0000 7f5c65e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5c680731c0 0x7f5c68197fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.174:52956/0 (socket says 172.21.15.174:52956) 2024-03-21T09:58:47.433 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:47.431+0000 7f5c65e00700 1 -- 172.21.15.174:0/681634284 learned_addr learned my addr 172.21.15.174:0/681634284 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-03-21T09:58:47.433 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:47.432+0000 7f5c65e00700 1 -- 172.21.15.174:0/681634284 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c580095f0 con 0x7f5c680731c0 2024-03-21T09:58:47.434 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:47.432+0000 7f5c65e00700 1 --2- 172.21.15.174:0/681634284 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5c680731c0 0x7f5c68197fd0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5c580092e0 tx=0x7f5c58004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:58:47.434 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:47.433+0000 7f5c5ea00700 1 -- 172.21.15.174:0/681634284 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5c5801d070 con 0x7f5c680731c0 2024-03-21T09:58:47.434 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:47.433+0000 7f5c6ce00700 1 -- 172.21.15.174:0/681634284 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c68198710 con 0x7f5c680731c0 2024-03-21T09:58:47.434 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:47.433+0000 7f5c5ea00700 1 -- 172.21.15.174:0/681634284 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f5c5800be10 con 0x7f5c680731c0 2024-03-21T09:58:47.435 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:47.433+0000 7f5c5ea00700 1 -- 172.21.15.174:0/681634284 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5c58022e10 con 0x7f5c680731c0 2024-03-21T09:58:47.435 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:47.433+0000 7f5c6ce00700 1 -- 172.21.15.174:0/681634284 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c68198b30 con 0x7f5c680731c0 2024-03-21T09:58:47.436 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:47.434+0000 7f5c6ce00700 1 -- 172.21.15.174:0/681634284 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c680623c0 con 0x7f5c680731c0 2024-03-21T09:58:47.436 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:47.434+0000 7f5c5ea00700 1 -- 172.21.15.174:0/681634284 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 17) v1 ==== 45398+0+0 (secure 0 0 0) 0x7f5c58022470 con 0x7f5c680731c0 2024-03-21T09:58:47.437 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:47.435+0000 7f5c5ea00700 1 --2- 172.21.15.174:0/681634284 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f5c4c038460 0x7f5c4c03a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:58:47.437 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:47.435+0000 7f5c5ea00700 1 -- 172.21.15.174:0/681634284 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f5c5804c560 con 0x7f5c680731c0 2024-03-21T09:58:47.437 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:47.436+0000 7f5c65400700 1 --2- 172.21.15.174:0/681634284 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f5c4c038460 0x7f5c4c03a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:58:47.438 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:47.437+0000 7f5c65400700 1 --2- 172.21.15.174:0/681634284 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f5c4c038460 0x7f5c4c03a920 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5c54006fd0 tx=0x7f5c54006e40 comp rx=0 tx=0).ready entity=mgr.14221 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:58:47.451 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:47.449+0000 7f5c5ea00700 1 -- 172.21.15.174:0/681634284 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f5c5802a540 con 0x7f5c680731c0 2024-03-21T09:58:47.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:47 smithi153 ceph-mon[29117]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:58:47.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:47 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:58:47.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:47 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:58:47.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:47 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:58:47.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:47 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:58:47.815 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:47.812+0000 7f5c6ce00700 1 -- 172.21.15.174:0/681634284 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f5c68109d10 con 0x7f5c680731c0 2024-03-21T09:58:47.817 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:47.813+0000 7f5c5ea00700 1 -- 172.21.15.174:0/681634284 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7f5c58027030 con 0x7f5c680731c0 2024-03-21T09:58:47.818 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-21T09:58:47.818 INFO:teuthology.orchestra.run.smithi174.stdout:{"epoch":1,"fsid":"070ed3e4-e769-11ee-95cd-87774f69a715","modified":"2024-03-21T09:55:15.397487Z","created":"2024-03-21T09:55:15.397487Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi153","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:3300","nonce":0},{"type":"v1","addr":"172.21.15.153:6789","nonce":0}]},"addr":"172.21.15.153:6789/0","public_addr":"172.21.15.153:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-21T09:58:47.818 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:47.816+0000 7f5c6ce00700 1 -- 172.21.15.174:0/681634284 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f5c4c038460 msgr2=0x7f5c4c03a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:58:47.819 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:47.816+0000 7f5c6ce00700 1 --2- 172.21.15.174:0/681634284 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f5c4c038460 0x7f5c4c03a920 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5c54006fd0 tx=0x7f5c54006e40 comp rx=0 tx=0).stop 2024-03-21T09:58:47.819 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:47.816+0000 7f5c6ce00700 1 -- 172.21.15.174:0/681634284 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5c680731c0 msgr2=0x7f5c68197fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:58:47.819 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:47.816+0000 7f5c6ce00700 1 --2- 172.21.15.174:0/681634284 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5c680731c0 0x7f5c68197fd0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5c580092e0 tx=0x7f5c58004c80 comp rx=0 tx=0).stop 2024-03-21T09:58:47.819 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:47.816+0000 7f5c6ce00700 1 -- 172.21.15.174:0/681634284 shutdown_connections 2024-03-21T09:58:47.819 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:47.816+0000 7f5c6ce00700 1 --2- 172.21.15.174:0/681634284 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f5c4c038460 0x7f5c4c03a920 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:58:47.819 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:47.816+0000 7f5c6ce00700 1 --2- 172.21.15.174:0/681634284 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5c680731c0 0x7f5c68197fd0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:58:47.819 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:47.816+0000 7f5c6ce00700 1 -- 172.21.15.174:0/681634284 >> 172.21.15.174:0/681634284 conn(0x7f5c680fc380 msgr2=0x7f5c68107370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:58:47.819 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:47.816+0000 7f5c6ce00700 1 -- 172.21.15.174:0/681634284 shutdown_connections 2024-03-21T09:58:47.819 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:47.816+0000 7f5c6ce00700 1 -- 172.21.15.174:0/681634284 wait complete. 2024-03-21T09:58:47.819 INFO:teuthology.orchestra.run.smithi174.stderr:dumped monmap epoch 1 2024-03-21T09:58:48.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:48 smithi153 ceph-mon[29117]: Deploying daemon node-exporter.smithi174 on smithi174 2024-03-21T09:58:48.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:48 smithi153 ceph-mon[29117]: from='client.? 172.21.15.174:0/681634284' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-21T09:58:49.666 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-21T09:58:49.667 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 070ed3e4-e769-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-21T09:58:49.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:49 smithi153 ceph-mon[29117]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:58:50.070 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T09:58:50.708 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:50 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:58:51.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:51 smithi153 ceph-mon[29117]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:58:53.167 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:53.165+0000 7f29c2a00700 1 -- 172.21.15.174:0/1373036501 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f29bc108a00 msgr2=0x7f29bc108de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:58:53.168 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:53.165+0000 7f29c2a00700 1 --2- 172.21.15.174:0/1373036501 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f29bc108a00 0x7f29bc108de0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f29ac009910 tx=0x7f29ac009c20 comp rx=0 tx=0).stop 2024-03-21T09:58:53.168 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:53.165+0000 7f29c2a00700 1 -- 172.21.15.174:0/1373036501 shutdown_connections 2024-03-21T09:58:53.168 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:53.165+0000 7f29c2a00700 1 --2- 172.21.15.174:0/1373036501 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f29bc108a00 0x7f29bc108de0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:58:53.168 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:53.165+0000 7f29c2a00700 1 -- 172.21.15.174:0/1373036501 >> 172.21.15.174:0/1373036501 conn(0x7f29bc0fe4b0 msgr2=0x7f29bc1008d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:58:53.168 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:53.165+0000 7f29c2a00700 1 -- 172.21.15.174:0/1373036501 shutdown_connections 2024-03-21T09:58:53.168 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:53.165+0000 7f29c2a00700 1 -- 172.21.15.174:0/1373036501 wait complete. 2024-03-21T09:58:53.169 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:53.167+0000 7f29c2a00700 1 Processor -- start 2024-03-21T09:58:53.169 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:53.167+0000 7f29c2a00700 1 -- start start 2024-03-21T09:58:53.169 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:53.167+0000 7f29c2a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f29bc108a00 0x7f29bc197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:58:53.169 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:53.167+0000 7f29c2a00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29bc198460 con 0x7f29bc108a00 2024-03-21T09:58:53.170 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:53.168+0000 7f29bbe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f29bc108a00 0x7f29bc197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:58:53.170 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:53.168+0000 7f29bbe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f29bc108a00 0x7f29bc197f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.174:57808/0 (socket says 172.21.15.174:57808) 2024-03-21T09:58:53.170 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:53.168+0000 7f29bbe00700 1 -- 172.21.15.174:0/1982552966 learned_addr learned my addr 172.21.15.174:0/1982552966 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-03-21T09:58:53.170 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:53.169+0000 7f29bbe00700 1 -- 172.21.15.174:0/1982552966 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29ac0095f0 con 0x7f29bc108a00 2024-03-21T09:58:53.171 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:53.169+0000 7f29bbe00700 1 --2- 172.21.15.174:0/1982552966 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f29bc108a00 0x7f29bc197f20 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f29ac005b40 tx=0x7f29ac004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:58:53.172 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:53.169+0000 7f29b8c00700 1 -- 172.21.15.174:0/1982552966 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f29ac01d070 con 0x7f29bc108a00 2024-03-21T09:58:53.172 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:53.170+0000 7f29b8c00700 1 -- 172.21.15.174:0/1982552966 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f29ac0044d0 con 0x7f29bc108a00 2024-03-21T09:58:53.172 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:53.170+0000 7f29b8c00700 1 -- 172.21.15.174:0/1982552966 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f29ac003910 con 0x7f29bc108a00 2024-03-21T09:58:53.172 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:53.170+0000 7f29c2a00700 1 -- 172.21.15.174:0/1982552966 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29bc198660 con 0x7f29bc108a00 2024-03-21T09:58:53.172 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:53.170+0000 7f29c2a00700 1 -- 172.21.15.174:0/1982552966 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29bc198ae0 con 0x7f29bc108a00 2024-03-21T09:58:53.173 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:53.171+0000 7f29c2a00700 1 -- 172.21.15.174:0/1982552966 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29bc0623c0 con 0x7f29bc108a00 2024-03-21T09:58:53.173 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:53.171+0000 7f29b8c00700 1 -- 172.21.15.174:0/1982552966 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 17) v1 ==== 45398+0+0 (secure 0 0 0) 0x7f29ac00b960 con 0x7f29bc108a00 2024-03-21T09:58:53.174 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:53.172+0000 7f29b8c00700 1 --2- 172.21.15.174:0/1982552966 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f29a0038500 0x7f29a003a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:58:53.174 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:53.172+0000 7f29b8c00700 1 -- 172.21.15.174:0/1982552966 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f29ac04c470 con 0x7f29bc108a00 2024-03-21T09:58:53.174 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:53.173+0000 7f29bb400700 1 --2- 172.21.15.174:0/1982552966 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f29a0038500 0x7f29a003a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:58:53.175 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:53.174+0000 7f29bb400700 1 --2- 172.21.15.174:0/1982552966 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f29a0038500 0x7f29a003a9c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f29a8006fd0 tx=0x7f29a8006e40 comp rx=0 tx=0).ready entity=mgr.14221 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:58:53.189 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:53.188+0000 7f29b8c00700 1 -- 172.21.15.174:0/1982552966 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f29ac02a540 con 0x7f29bc108a00 2024-03-21T09:58:53.494 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:53.492+0000 7f29c2a00700 1 -- 172.21.15.174:0/1982552966 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f29bc1067a0 con 0x7f29bc108a00 2024-03-21T09:58:53.495 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:53.493+0000 7f29b8c00700 1 -- 172.21.15.174:0/1982552966 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7f29ac02a360 con 0x7f29bc108a00 2024-03-21T09:58:53.495 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-21T09:58:53.495 INFO:teuthology.orchestra.run.smithi174.stdout:{"epoch":1,"fsid":"070ed3e4-e769-11ee-95cd-87774f69a715","modified":"2024-03-21T09:55:15.397487Z","created":"2024-03-21T09:55:15.397487Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi153","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:3300","nonce":0},{"type":"v1","addr":"172.21.15.153:6789","nonce":0}]},"addr":"172.21.15.153:6789/0","public_addr":"172.21.15.153:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-21T09:58:53.498 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:53.496+0000 7f29c2a00700 1 -- 172.21.15.174:0/1982552966 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f29a0038500 msgr2=0x7f29a003a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:58:53.498 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:53.496+0000 7f29c2a00700 1 --2- 172.21.15.174:0/1982552966 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f29a0038500 0x7f29a003a9c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f29a8006fd0 tx=0x7f29a8006e40 comp rx=0 tx=0).stop 2024-03-21T09:58:53.499 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:53.496+0000 7f29c2a00700 1 -- 172.21.15.174:0/1982552966 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f29bc108a00 msgr2=0x7f29bc197f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:58:53.499 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:53.496+0000 7f29c2a00700 1 --2- 172.21.15.174:0/1982552966 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f29bc108a00 0x7f29bc197f20 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f29ac005b40 tx=0x7f29ac004c80 comp rx=0 tx=0).stop 2024-03-21T09:58:53.499 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:53.496+0000 7f29c2a00700 1 -- 172.21.15.174:0/1982552966 shutdown_connections 2024-03-21T09:58:53.499 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:53.496+0000 7f29c2a00700 1 --2- 172.21.15.174:0/1982552966 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f29a0038500 0x7f29a003a9c0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:58:53.499 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:53.496+0000 7f29c2a00700 1 --2- 172.21.15.174:0/1982552966 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f29bc108a00 0x7f29bc197f20 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:58:53.499 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:53.496+0000 7f29c2a00700 1 -- 172.21.15.174:0/1982552966 >> 172.21.15.174:0/1982552966 conn(0x7f29bc0fe4b0 msgr2=0x7f29bc10adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:58:53.499 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:53.497+0000 7f29c2a00700 1 -- 172.21.15.174:0/1982552966 shutdown_connections 2024-03-21T09:58:53.499 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:53.497+0000 7f29c2a00700 1 -- 172.21.15.174:0/1982552966 wait complete. 2024-03-21T09:58:53.499 INFO:teuthology.orchestra.run.smithi174.stderr:dumped monmap epoch 1 2024-03-21T09:58:53.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:53 smithi153 ceph-mon[29117]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:58:53.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:53 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:58:53.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:53 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:58:53.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:53 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:58:53.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:53 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:58:53.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:53 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi174.qpcwat", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-21T09:58:53.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:53 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.smithi174.qpcwat", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-21T09:58:53.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:53 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-21T09:58:53.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:53 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:58:54.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:54 smithi153 ceph-mon[29117]: Deploying daemon mgr.smithi174.qpcwat on smithi174 2024-03-21T09:58:54.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:54 smithi153 ceph-mon[29117]: from='client.? 172.21.15.174:0/1982552966' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-21T09:58:54.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:54 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T09:58:54.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:54 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:58:55.191 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-21T09:58:55.191 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 070ed3e4-e769-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-21T09:58:55.609 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T09:58:55.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:55 smithi153 ceph-mon[29117]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:58:57.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:57 smithi153 ceph-mon[29117]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:58:57.985 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:57.982+0000 7f55fce00700 1 -- 172.21.15.174:0/1619679509 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f55f8106820 msgr2=0x7f55f8106c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:58:57.985 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:57.982+0000 7f55fce00700 1 --2- 172.21.15.174:0/1619679509 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f55f8106820 0x7f55f8106c00 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f55e8009910 tx=0x7f55e8009c20 comp rx=0 tx=0).stop 2024-03-21T09:58:57.985 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:57.982+0000 7f55fce00700 1 -- 172.21.15.174:0/1619679509 shutdown_connections 2024-03-21T09:58:57.985 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:57.982+0000 7f55fce00700 1 --2- 172.21.15.174:0/1619679509 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f55f8106820 0x7f55f8106c00 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:58:57.985 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:57.982+0000 7f55fce00700 1 -- 172.21.15.174:0/1619679509 >> 172.21.15.174:0/1619679509 conn(0x7f55f80752b0 msgr2=0x7f55f80756c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:58:57.985 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:57.983+0000 7f55fce00700 1 -- 172.21.15.174:0/1619679509 shutdown_connections 2024-03-21T09:58:57.986 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:57.983+0000 7f55fce00700 1 -- 172.21.15.174:0/1619679509 wait complete. 2024-03-21T09:58:57.986 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:57.984+0000 7f55fce00700 1 Processor -- start 2024-03-21T09:58:57.987 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:57.984+0000 7f55fce00700 1 -- start start 2024-03-21T09:58:57.987 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:57.984+0000 7f55fce00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f55f8106820 0x7f55f8195cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:58:57.987 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:57.984+0000 7f55fce00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55f8196210 con 0x7f55f8106820 2024-03-21T09:58:57.987 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:57.985+0000 7f55f5e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f55f8106820 0x7f55f8195cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:58:57.987 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:57.985+0000 7f55f5e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f55f8106820 0x7f55f8195cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.174:57836/0 (socket says 172.21.15.174:57836) 2024-03-21T09:58:57.987 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:57.985+0000 7f55f5e00700 1 -- 172.21.15.174:0/4163927159 learned_addr learned my addr 172.21.15.174:0/4163927159 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-03-21T09:58:57.988 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:57.986+0000 7f55f5e00700 1 -- 172.21.15.174:0/4163927159 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55e80095f0 con 0x7f55f8106820 2024-03-21T09:58:57.988 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:57.986+0000 7f55f5e00700 1 --2- 172.21.15.174:0/4163927159 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f55f8106820 0x7f55f8195cd0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f55e8005b40 tx=0x7f55e8004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:58:57.988 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:57.987+0000 7f55eea00700 1 -- 172.21.15.174:0/4163927159 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f55e801d070 con 0x7f55f8106820 2024-03-21T09:58:57.988 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:57.987+0000 7f55fce00700 1 -- 172.21.15.174:0/4163927159 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55f8196410 con 0x7f55f8106820 2024-03-21T09:58:57.989 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:57.987+0000 7f55fce00700 1 -- 172.21.15.174:0/4163927159 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55f8196890 con 0x7f55f8106820 2024-03-21T09:58:57.989 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:57.987+0000 7f55eea00700 1 -- 172.21.15.174:0/4163927159 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f55e80044d0 con 0x7f55f8106820 2024-03-21T09:58:57.989 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:57.987+0000 7f55eea00700 1 -- 172.21.15.174:0/4163927159 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f55e8003830 con 0x7f55f8106820 2024-03-21T09:58:57.989 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:57.988+0000 7f55fce00700 1 -- 172.21.15.174:0/4163927159 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55d8005320 con 0x7f55f8106820 2024-03-21T09:58:57.991 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:57.989+0000 7f55eea00700 1 -- 172.21.15.174:0/4163927159 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 17) v1 ==== 45398+0+0 (secure 0 0 0) 0x7f55e800b960 con 0x7f55f8106820 2024-03-21T09:58:57.991 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:57.990+0000 7f55eea00700 1 --2- 172.21.15.174:0/4163927159 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f55dc038500 0x7f55dc03a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:58:57.992 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:57.990+0000 7f55eea00700 1 -- 172.21.15.174:0/4163927159 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f55e804c660 con 0x7f55f8106820 2024-03-21T09:58:57.992 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:57.991+0000 7f55f5400700 1 --2- 172.21.15.174:0/4163927159 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f55dc038500 0x7f55dc03a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:58:57.993 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:57.992+0000 7f55f5400700 1 --2- 172.21.15.174:0/4163927159 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f55dc038500 0x7f55dc03a9c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f55e4006fd0 tx=0x7f55e4006e40 comp rx=0 tx=0).ready entity=mgr.14221 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:58:58.019 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:58.017+0000 7f55eea00700 1 -- 172.21.15.174:0/4163927159 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f55e8004300 con 0x7f55f8106820 2024-03-21T09:58:58.328 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:58.326+0000 7f55fce00700 1 -- 172.21.15.174:0/4163927159 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f55d8005190 con 0x7f55f8106820 2024-03-21T09:58:58.330 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:58.328+0000 7f55eea00700 1 -- 172.21.15.174:0/4163927159 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7f55e8030020 con 0x7f55f8106820 2024-03-21T09:58:58.330 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-21T09:58:58.330 INFO:teuthology.orchestra.run.smithi174.stdout:{"epoch":1,"fsid":"070ed3e4-e769-11ee-95cd-87774f69a715","modified":"2024-03-21T09:55:15.397487Z","created":"2024-03-21T09:55:15.397487Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi153","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:3300","nonce":0},{"type":"v1","addr":"172.21.15.153:6789","nonce":0}]},"addr":"172.21.15.153:6789/0","public_addr":"172.21.15.153:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-21T09:58:58.333 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:58.330+0000 7f55fce00700 1 -- 172.21.15.174:0/4163927159 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f55dc038500 msgr2=0x7f55dc03a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:58:58.333 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:58.330+0000 7f55fce00700 1 --2- 172.21.15.174:0/4163927159 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f55dc038500 0x7f55dc03a9c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f55e4006fd0 tx=0x7f55e4006e40 comp rx=0 tx=0).stop 2024-03-21T09:58:58.333 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:58.331+0000 7f55fce00700 1 -- 172.21.15.174:0/4163927159 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f55f8106820 msgr2=0x7f55f8195cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:58:58.333 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:58.331+0000 7f55fce00700 1 --2- 172.21.15.174:0/4163927159 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f55f8106820 0x7f55f8195cd0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f55e8005b40 tx=0x7f55e8004c80 comp rx=0 tx=0).stop 2024-03-21T09:58:58.333 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:58.331+0000 7f55fce00700 1 -- 172.21.15.174:0/4163927159 shutdown_connections 2024-03-21T09:58:58.333 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:58.331+0000 7f55fce00700 1 --2- 172.21.15.174:0/4163927159 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f55dc038500 0x7f55dc03a9c0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:58:58.333 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:58.331+0000 7f55fce00700 1 --2- 172.21.15.174:0/4163927159 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f55f8106820 0x7f55f8195cd0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:58:58.333 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:58.331+0000 7f55fce00700 1 -- 172.21.15.174:0/4163927159 >> 172.21.15.174:0/4163927159 conn(0x7f55f80752b0 msgr2=0x7f55f80fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:58:58.333 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:58.331+0000 7f55fce00700 1 -- 172.21.15.174:0/4163927159 shutdown_connections 2024-03-21T09:58:58.333 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:58:58.331+0000 7f55fce00700 1 -- 172.21.15.174:0/4163927159 wait complete. 2024-03-21T09:58:58.334 INFO:teuthology.orchestra.run.smithi174.stderr:dumped monmap epoch 1 2024-03-21T09:58:59.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:58 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:58:59.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:58 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:58:59.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:58 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:58:59.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:58 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:58:59.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:58 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T09:58:59.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:58 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:58:59.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:58 smithi153 ceph-mon[29117]: Deploying daemon mon.smithi174 on smithi174 2024-03-21T09:58:59.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:58 smithi153 ceph-mon[29117]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:58:59.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:58:58 smithi153 ceph-mon[29117]: from='client.? 172.21.15.174:0/4163927159' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-21T09:59:00.019 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-21T09:59:00.019 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 070ed3e4-e769-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-21T09:59:00.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:00 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:59:01.122 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T09:59:01.730 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:01 smithi153 ceph-mon[29117]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:59:02.953 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:02 smithi174 podman[37441]: 2024-03-21 09:59:02.734474524 +0000 UTC m=+0.126707163 image pull 5be31c24972a920012b90a9769e8313e2490c82aee752aefd49af9cf4c0f3fcf quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3 2024-03-21T09:59:03.305 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 podman[37441]: 2024-03-21 09:59:03.16045683 +0000 UTC m=+0.552689468 container create 99204923eec612ae776aff9c666751603c4144d04fd61bf839168efafed8d00d (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-mon-smithi174, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.29.1, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=-18.2.1, org.label-schema.build-date=20240222, RELEASE=HEAD) 2024-03-21T09:59:03.653 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 podman[37441]: 2024-03-21 09:59:03.650464975 +0000 UTC m=+1.042697651 container init 99204923eec612ae776aff9c666751603c4144d04fd61bf839168efafed8d00d (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-mon-smithi174, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.29.1, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=HEAD, CEPH_POINT_RELEASE=-18.2.1, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.build-date=20240222, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T09:59:03.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:03 smithi153 ceph-mon[29117]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:59:03.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 podman[37441]: 2024-03-21 09:59:03.662284494 +0000 UTC m=+1.054517131 container start 99204923eec612ae776aff9c666751603c4144d04fd61bf839168efafed8d00d (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-mon-smithi174, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=-18.2.1, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.29.1, GIT_REPO=https://github.com/ceph/ceph-container.git, RELEASE=HEAD, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-03-21T09:59:03.953 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: set uid:gid to 167:167 (ceph:ceph) 2024-03-21T09:59:03.953 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable), process ceph-mon, pid 2 2024-03-21T09:59:03.953 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: pidfile_write: ignore empty --pid-file 2024-03-21T09:59:03.953 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: load: jerasure load: lrc 2024-03-21T09:59:03.953 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: RocksDB version: 7.9.2 2024-03-21T09:59:03.953 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Git sha 0 2024-03-21T09:59:03.953 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Compile date 2023-12-11 22:07:34 2024-03-21T09:59:03.953 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: DB SUMMARY 2024-03-21T09:59:03.953 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: DB Session ID: N6EVAMUH34AXI2KEKXAG 2024-03-21T09:59:03.953 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: CURRENT file: CURRENT 2024-03-21T09:59:03.953 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: IDENTITY file: IDENTITY 2024-03-21T09:59:03.953 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-03-21T09:59:03.953 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi174/store.db dir, Total Num: 0, files: 2024-03-21T09:59:03.954 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi174/store.db: 000004.log size: 511 ; 2024-03-21T09:59:03.954 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.error_if_exists: 0 2024-03-21T09:59:03.954 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.create_if_missing: 0 2024-03-21T09:59:03.954 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.paranoid_checks: 1 2024-03-21T09:59:03.954 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-21T09:59:03.954 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-21T09:59:03.954 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-21T09:59:03.954 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.env: 0x55f2af9c0720 2024-03-21T09:59:03.954 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.fs: PosixFileSystem 2024-03-21T09:59:03.954 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.info_log: 0x55f2b2b7b2e0 2024-03-21T09:59:03.954 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.max_file_opening_threads: 16 2024-03-21T09:59:03.954 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.statistics: (nil) 2024-03-21T09:59:03.954 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.use_fsync: 0 2024-03-21T09:59:03.954 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.max_log_file_size: 0 2024-03-21T09:59:03.955 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-21T09:59:03.955 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-21T09:59:03.955 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.keep_log_file_num: 1000 2024-03-21T09:59:03.955 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.recycle_log_file_num: 0 2024-03-21T09:59:03.955 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.allow_fallocate: 1 2024-03-21T09:59:03.955 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.allow_mmap_reads: 0 2024-03-21T09:59:03.956 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.allow_mmap_writes: 0 2024-03-21T09:59:03.956 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.use_direct_reads: 0 2024-03-21T09:59:03.956 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-21T09:59:03.956 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.create_missing_column_families: 0 2024-03-21T09:59:03.956 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.db_log_dir: 2024-03-21T09:59:03.956 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.wal_dir: 2024-03-21T09:59:03.956 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-21T09:59:03.956 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-21T09:59:03.956 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-21T09:59:03.956 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-21T09:59:03.956 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-21T09:59:03.956 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-21T09:59:03.956 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.advise_random_on_open: 1 2024-03-21T09:59:03.957 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.db_write_buffer_size: 0 2024-03-21T09:59:03.957 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.write_buffer_manager: 0x55f2b1dec140 2024-03-21T09:59:03.957 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-21T09:59:03.957 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-21T09:59:03.957 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-21T09:59:03.957 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.rate_limiter: (nil) 2024-03-21T09:59:03.957 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-21T09:59:03.957 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.wal_recovery_mode: 2 2024-03-21T09:59:03.957 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.enable_thread_tracking: 0 2024-03-21T09:59:03.957 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.enable_pipelined_write: 0 2024-03-21T09:59:03.957 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.unordered_write: 0 2024-03-21T09:59:03.957 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-21T09:59:03.958 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-21T09:59:03.958 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-21T09:59:03.958 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-21T09:59:03.958 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.row_cache: None 2024-03-21T09:59:03.958 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.wal_filter: None 2024-03-21T09:59:03.958 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-21T09:59:03.958 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.allow_ingest_behind: 0 2024-03-21T09:59:03.958 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.two_write_queues: 0 2024-03-21T09:59:03.958 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.manual_wal_flush: 0 2024-03-21T09:59:03.958 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.wal_compression: 0 2024-03-21T09:59:03.958 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.atomic_flush: 0 2024-03-21T09:59:03.958 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-21T09:59:03.958 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-21T09:59:03.958 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-21T09:59:03.959 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.log_readahead_size: 0 2024-03-21T09:59:03.959 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-21T09:59:03.959 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.best_efforts_recovery: 0 2024-03-21T09:59:03.959 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-21T09:59:03.959 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-21T09:59:03.959 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.allow_data_in_errors: 0 2024-03-21T09:59:03.959 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.db_host_id: __hostname__ 2024-03-21T09:59:03.959 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-21T09:59:03.959 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.max_background_jobs: 2 2024-03-21T09:59:03.959 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.max_background_compactions: -1 2024-03-21T09:59:03.959 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.max_subcompactions: 1 2024-03-21T09:59:03.959 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-21T09:59:03.959 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-21T09:59:03.960 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-21T09:59:03.960 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.max_total_wal_size: 0 2024-03-21T09:59:03.960 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-21T09:59:03.960 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-21T09:59:03.960 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-21T09:59:03.960 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-21T09:59:03.960 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.max_open_files: -1 2024-03-21T09:59:03.960 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.bytes_per_sync: 0 2024-03-21T09:59:03.960 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-21T09:59:03.960 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-21T09:59:03.960 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.compaction_readahead_size: 0 2024-03-21T09:59:03.960 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.max_background_flushes: -1 2024-03-21T09:59:03.960 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Compression algorithms supported: 2024-03-21T09:59:03.960 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-21T09:59:03.960 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: kZSTD supported: 0 2024-03-21T09:59:03.961 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: kXpressCompression supported: 0 2024-03-21T09:59:03.961 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: kLZ4HCCompression supported: 1 2024-03-21T09:59:03.961 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: kZlibCompression supported: 1 2024-03-21T09:59:03.961 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: kSnappyCompression supported: 1 2024-03-21T09:59:03.961 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: kLZ4Compression supported: 1 2024-03-21T09:59:03.961 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: kBZip2Compression supported: 0 2024-03-21T09:59:03.961 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-21T09:59:03.961 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-21T09:59:03.961 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi174/store.db/MANIFEST-000005 2024-03-21T09:59:03.961 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-21T09:59:03.961 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-21T09:59:03.961 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.merge_operator: 2024-03-21T09:59:03.961 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.compaction_filter: None 2024-03-21T09:59:03.962 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.compaction_filter_factory: None 2024-03-21T09:59:03.962 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.sst_partitioner_factory: None 2024-03-21T09:59:03.962 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-21T09:59:03.962 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-21T09:59:03.962 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55f2b2b7b3e0) 2024-03-21T09:59:03.962 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: cache_index_and_filter_blocks: 1 2024-03-21T09:59:03.962 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-21T09:59:03.962 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-21T09:59:03.962 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: pin_top_level_index_and_filter: 1 2024-03-21T09:59:03.962 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: index_type: 0 2024-03-21T09:59:03.962 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: data_block_index_type: 0 2024-03-21T09:59:03.962 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: index_shortening: 1 2024-03-21T09:59:03.962 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-21T09:59:03.963 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: checksum: 4 2024-03-21T09:59:03.963 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: no_block_cache: 0 2024-03-21T09:59:03.963 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: block_cache: 0x55f2b1e8d350 2024-03-21T09:59:03.963 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: block_cache_name: BinnedLRUCache 2024-03-21T09:59:03.963 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: block_cache_options: 2024-03-21T09:59:03.963 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: capacity : 536870912 2024-03-21T09:59:03.963 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: num_shard_bits : 4 2024-03-21T09:59:03.963 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: strict_capacity_limit : 0 2024-03-21T09:59:03.963 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: high_pri_pool_ratio: 0.000 2024-03-21T09:59:03.963 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: block_cache_compressed: (nil) 2024-03-21T09:59:03.963 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: persistent_cache: (nil) 2024-03-21T09:59:03.963 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: block_size: 4096 2024-03-21T09:59:03.963 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: block_size_deviation: 10 2024-03-21T09:59:03.964 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: block_restart_interval: 16 2024-03-21T09:59:03.964 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: index_block_restart_interval: 1 2024-03-21T09:59:03.964 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: metadata_block_size: 4096 2024-03-21T09:59:03.964 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: partition_filters: 0 2024-03-21T09:59:03.964 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: use_delta_encoding: 1 2024-03-21T09:59:03.964 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: filter_policy: bloomfilter 2024-03-21T09:59:03.965 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: whole_key_filtering: 1 2024-03-21T09:59:03.965 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: verify_compression: 0 2024-03-21T09:59:03.965 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: read_amp_bytes_per_bit: 0 2024-03-21T09:59:03.965 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: format_version: 5 2024-03-21T09:59:03.965 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: enable_index_compression: 1 2024-03-21T09:59:03.965 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: block_align: 0 2024-03-21T09:59:03.965 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: max_auto_readahead_size: 262144 2024-03-21T09:59:03.965 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: prepopulate_block_cache: 0 2024-03-21T09:59:03.965 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: initial_auto_readahead_size: 8192 2024-03-21T09:59:03.965 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: num_file_reads_for_auto_readahead: 2 2024-03-21T09:59:03.966 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.write_buffer_size: 33554432 2024-03-21T09:59:03.966 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.max_write_buffer_number: 2 2024-03-21T09:59:03.966 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.compression: NoCompression 2024-03-21T09:59:03.966 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.bottommost_compression: Disabled 2024-03-21T09:59:03.966 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.prefix_extractor: nullptr 2024-03-21T09:59:03.966 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-21T09:59:03.966 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.num_levels: 7 2024-03-21T09:59:03.966 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-21T09:59:03.968 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-21T09:59:03.968 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-21T09:59:03.968 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-21T09:59:03.968 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-21T09:59:03.968 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-21T09:59:03.968 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-21T09:59:03.968 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-21T09:59:03.969 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-21T09:59:03.969 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-21T09:59:03.969 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-21T09:59:03.969 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-21T09:59:03.969 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-21T09:59:03.969 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.compression_opts.level: 32767 2024-03-21T09:59:03.969 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.compression_opts.strategy: 0 2024-03-21T09:59:03.969 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-21T09:59:03.969 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-21T09:59:03.969 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-21T09:59:03.969 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-21T09:59:03.969 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.compression_opts.enabled: false 2024-03-21T09:59:03.969 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-21T09:59:03.969 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-21T09:59:03.970 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-21T09:59:03.970 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-21T09:59:03.970 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.target_file_size_base: 67108864 2024-03-21T09:59:03.970 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-21T09:59:03.970 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-21T09:59:03.970 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-21T09:59:03.970 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-21T09:59:03.970 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-21T09:59:03.970 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-21T09:59:03.970 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-21T09:59:03.971 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-21T09:59:03.971 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-21T09:59:03.971 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-21T09:59:03.971 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-21T09:59:03.971 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-21T09:59:03.971 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-21T09:59:03.971 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-21T09:59:03.971 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.arena_block_size: 1048576 2024-03-21T09:59:03.971 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-21T09:59:03.971 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-21T09:59:03.971 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.disable_auto_compactions: 0 2024-03-21T09:59:03.971 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-21T09:59:03.971 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-21T09:59:03.971 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-21T09:59:03.972 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-21T09:59:03.972 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-21T09:59:03.972 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-21T09:59:03.972 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-21T09:59:03.972 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-21T09:59:03.972 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-21T09:59:03.972 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-21T09:59:03.972 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.table_properties_collectors: 2024-03-21T09:59:03.972 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.inplace_update_support: 0 2024-03-21T09:59:03.972 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-21T09:59:03.972 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-21T09:59:03.972 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-21T09:59:03.972 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-21T09:59:03.972 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.bloom_locality: 0 2024-03-21T09:59:03.972 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.max_successive_merges: 0 2024-03-21T09:59:03.973 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-21T09:59:03.973 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.paranoid_file_checks: 0 2024-03-21T09:59:03.973 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.force_consistency_checks: 1 2024-03-21T09:59:03.973 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.report_bg_io_stats: 0 2024-03-21T09:59:03.973 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.ttl: 2592000 2024-03-21T09:59:03.973 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-21T09:59:03.973 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-21T09:59:03.973 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-21T09:59:03.973 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.enable_blob_files: false 2024-03-21T09:59:03.973 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.min_blob_size: 0 2024-03-21T09:59:03.973 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.blob_file_size: 268435456 2024-03-21T09:59:03.973 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-21T09:59:03.973 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-21T09:59:03.973 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-21T09:59:03.973 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-21T09:59:03.974 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-21T09:59:03.974 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.blob_file_starting_level: 0 2024-03-21T09:59:03.974 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-21T09:59:03.974 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi174/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-03-21T09:59:03.974 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-03-21T09:59:03.974 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 6b684d55-665e-4251-a5ca-36c284b8aed1 2024-03-21T09:59:03.974 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711015143712022, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-03-21T09:59:03.974 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-03-21T09:59:03.974 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711015143712924, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1711015143, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "6b684d55-665e-4251-a5ca-36c284b8aed1", "db_session_id": "N6EVAMUH34AXI2KEKXAG", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-03-21T09:59:03.974 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711015143713134, "job": 1, "event": "recovery_finished"} 2024-03-21T09:59:03.974 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-03-21T09:59:03.974 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi174/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-21T09:59:03.974 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55f2b1f2a000 2024-03-21T09:59:03.975 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: DB pointer 0x55f2b1f16000 2024-03-21T09:59:03.975 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-21T09:59:03.975 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-21T09:59:03.975 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: ** DB Stats ** 2024-03-21T09:59:03.975 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-21T09:59:03.975 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-21T09:59:03.975 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-21T09:59:03.975 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-21T09:59:03.975 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-21T09:59:03.975 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-21T09:59:03.975 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-21T09:59:03.975 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: 2024-03-21T09:59:03.975 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: ** Compaction Stats [default] ** 2024-03-21T09:59:03.975 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-21T09:59:03.976 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-21T09:59:03.976 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-21T09:59:03.976 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-21T09:59:03.976 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-21T09:59:03.976 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: 2024-03-21T09:59:03.976 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: ** Compaction Stats [default] ** 2024-03-21T09:59:03.976 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-21T09:59:03.976 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-21T09:59:03.976 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-21T09:59:03.976 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: 2024-03-21T09:59:03.976 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-21T09:59:03.976 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: 2024-03-21T09:59:03.976 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-21T09:59:03.976 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-21T09:59:03.977 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-21T09:59:03.977 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-21T09:59:03.977 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-21T09:59:03.977 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-21T09:59:03.977 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: Cumulative compaction: 0.00 GB write, 0.28 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-21T09:59:03.977 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: Interval compaction: 0.00 GB write, 0.28 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-21T09:59:03.977 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-21T09:59:03.977 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: Block cache BinnedLRUCache@0x55f2b1e8d350#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.2e-05 secs_since: 0 2024-03-21T09:59:03.977 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-03-21T09:59:03.977 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: 2024-03-21T09:59:03.977 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-21T09:59:03.977 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: mon.smithi174 does not exist in monmap, will attempt to join an existing cluster 2024-03-21T09:59:03.977 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: using public_addr v2:172.21.15.174:0/0 -> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] 2024-03-21T09:59:03.977 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: starting mon.smithi174 rank -1 at public addrs [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] at bind addrs [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi174 fsid 070ed3e4-e769-11ee-95cd-87774f69a715 2024-03-21T09:59:03.977 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: mon.smithi174@-1(???) e0 preinit fsid 070ed3e4-e769-11ee-95cd-87774f69a715 2024-03-21T09:59:03.978 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: mon.smithi174@-1(synchronizing).mds e1 new map 2024-03-21T09:59:03.978 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: mon.smithi174@-1(synchronizing).mds e1 print_map 2024-03-21T09:59:03.978 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: e1 2024-03-21T09:59:03.978 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-21T09:59:03.978 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T09:59:03.978 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: legacy client fscid: -1 2024-03-21T09:59:03.978 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: 2024-03-21T09:59:03.978 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: No filesystems configured 2024-03-21T09:59:03.978 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: mon.smithi174@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-03-21T09:59:03.978 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: mon.smithi174@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-03-21T09:59:03.978 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: mon.smithi174@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-03-21T09:59:03.978 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: mon.smithi174@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-03-21T09:59:03.978 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: mon.smithi174@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-03-21T09:59:03.979 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: mon.smithi174@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-03-21T09:59:03.979 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: mon.smithi174@-1(synchronizing).osd e5 e5: 0 total, 0 up, 0 in 2024-03-21T09:59:03.979 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: mon.smithi174@-1(synchronizing).osd e5 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-21T09:59:03.979 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: mon.smithi174@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2024-03-21T09:59:03.979 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: mon.smithi174@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2024-03-21T09:59:03.979 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: mon.smithi174@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2024-03-21T09:59:03.979 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: Active manager daemon smithi153.tluxni restarted 2024-03-21T09:59:03.979 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: Activating manager daemon smithi153.tluxni 2024-03-21T09:59:03.979 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: osdmap e5: 0 total, 0 up, 0 in 2024-03-21T09:59:03.979 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: mgrmap e15: smithi153.tluxni(active, starting, since 0.0103498s) 2024-03-21T09:59:03.979 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mon metadata", "id": "smithi153"}]: dispatch 2024-03-21T09:59:03.979 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mgr metadata", "who": "smithi153.tluxni", "id": "smithi153.tluxni"}]: dispatch 2024-03-21T09:59:03.979 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-21T09:59:03.979 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-21T09:59:03.980 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-21T09:59:03.980 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: Manager daemon smithi153.tluxni is now available 2024-03-21T09:59:03.980 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:59:03.980 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:59:03.980 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:59:03.980 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T09:59:03.980 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi153.tluxni/mirror_snapshot_schedule"}]: dispatch 2024-03-21T09:59:03.980 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi153.tluxni/trash_purge_schedule"}]: dispatch 2024-03-21T09:59:03.980 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: [21/Mar/2024:09:58:24] ENGINE Bus STARTING 2024-03-21T09:59:03.980 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: [21/Mar/2024:09:58:24] ENGINE Serving on http://172.21.15.153:8765 2024-03-21T09:59:03.980 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: [21/Mar/2024:09:58:24] ENGINE Serving on https://172.21.15.153:7150 2024-03-21T09:59:03.980 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: [21/Mar/2024:09:58:24] ENGINE Bus STARTED 2024-03-21T09:59:03.980 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: mgrmap e16: smithi153.tluxni(active, since 1.0153s) 2024-03-21T09:59:03.980 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:59:03.980 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: mgrmap e17: smithi153.tluxni(active, since 2s) 2024-03-21T09:59:03.981 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:59:03.981 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: from='client.? 172.21.15.174:0/4075299662' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-21T09:59:03.981 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:59:03.981 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:59:03.981 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:59:03.981 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:59:03.981 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:59:03.981 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "who": "osd/host:smithi153", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:59:03.981 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: from='client.? 172.21.15.174:0/3835914961' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-21T09:59:03.981 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:59:03.981 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:59:03.981 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:59:03.981 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:59:03.982 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "who": "osd/host:smithi174", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:59:03.982 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:59:03.982 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:59:03.982 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: Updating smithi153:/etc/ceph/ceph.conf 2024-03-21T09:59:03.982 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: Updating smithi174:/etc/ceph/ceph.conf 2024-03-21T09:59:03.982 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: Updating smithi174:/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T09:59:03.982 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: Updating smithi153:/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T09:59:03.982 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: Updating smithi153:/etc/ceph/ceph.client.admin.keyring 2024-03-21T09:59:03.982 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: Updating smithi174:/etc/ceph/ceph.client.admin.keyring 2024-03-21T09:59:03.982 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: Updating smithi174:/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-21T09:59:03.982 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: Updating smithi153:/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-21T09:59:03.982 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:59:03.982 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:59:03.983 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:59:03.983 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:59:03.983 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:59:03.983 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi174", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-21T09:59:03.983 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi174", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-03-21T09:59:03.983 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:59:03.983 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: Deploying daemon ceph-exporter.smithi174 on smithi174 2024-03-21T09:59:03.983 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: from='client.? 172.21.15.174:0/817017622' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-21T09:59:03.983 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T09:59:03.983 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:59:03.983 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:59:03.983 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:59:03.983 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:59:03.983 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi174", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-21T09:59:03.984 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi174", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-03-21T09:59:03.984 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:59:03.984 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: Deploying daemon crash.smithi174 on smithi174 2024-03-21T09:59:03.984 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: from='client.? 172.21.15.174:0/1306174926' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-21T09:59:03.984 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:59:03.984 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:59:03.984 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:59:03.984 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:59:03.984 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:59:03.984 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:59:03.984 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:59:03.984 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: Deploying daemon node-exporter.smithi174 on smithi174 2024-03-21T09:59:03.984 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: from='client.? 172.21.15.174:0/681634284' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-21T09:59:03.984 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:59:03.984 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:59:03.985 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:59:03.985 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:59:03.985 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:59:03.985 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:59:03.985 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:59:03.985 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:59:03.985 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi174.qpcwat", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-21T09:59:03.985 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.smithi174.qpcwat", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-21T09:59:03.985 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-21T09:59:03.985 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:59:03.985 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: Deploying daemon mgr.smithi174.qpcwat on smithi174 2024-03-21T09:59:03.985 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: from='client.? 172.21.15.174:0/1982552966' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-21T09:59:03.985 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T09:59:03.985 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:59:03.985 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:59:03.986 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:59:03.986 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:59:03.986 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:59:03.986 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:59:03.986 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:59:03.986 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T09:59:03.986 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:59:03.986 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: Deploying daemon mon.smithi174 on smithi174 2024-03-21T09:59:03.986 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:59:03.986 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: from='client.? 172.21.15.174:0/4163927159' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-21T09:59:03.986 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:59:03.986 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:59:03.986 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:59:03.987 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: mon.smithi174@-1(synchronizing).paxosservice(auth 1..8) refresh upgraded, format 0 -> 3 2024-03-21T09:59:03.987 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: expand_channel_meta expand map: {default=false} 2024-03-21T09:59:03.987 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: expand_channel_meta from 'false' to 'false' 2024-03-21T09:59:03.987 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: expand_channel_meta expanded map: {default=false} 2024-03-21T09:59:03.987 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: expand_channel_meta expand map: {default=info} 2024-03-21T09:59:03.987 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: expand_channel_meta from 'info' to 'info' 2024-03-21T09:59:03.987 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: expand_channel_meta expanded map: {default=info} 2024-03-21T09:59:03.987 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: expand_channel_meta expand map: {default=daemon} 2024-03-21T09:59:03.987 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: expand_channel_meta from 'daemon' to 'daemon' 2024-03-21T09:59:03.987 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: expand_channel_meta expanded map: {default=daemon} 2024-03-21T09:59:03.987 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: expand_channel_meta expand map: {default=debug} 2024-03-21T09:59:03.987 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: expand_channel_meta from 'debug' to 'debug' 2024-03-21T09:59:03.987 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: expand_channel_meta expanded map: {default=debug} 2024-03-21T09:59:03.987 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 ceph-mon[37496]: mon.smithi174@-1(synchronizing) e1 handle_conf_change mon_allow_pool_delete,mon_cluster_log_to_file 2024-03-21T09:59:03.987 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 bash[37441]: 99204923eec612ae776aff9c666751603c4144d04fd61bf839168efafed8d00d 2024-03-21T09:59:03.988 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:03 smithi174 systemd[1]: Started Ceph mon.smithi174 for 070ed3e4-e769-11ee-95cd-87774f69a715. 2024-03-21T09:59:09.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:08 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mon metadata", "id": "smithi153"}]: dispatch 2024-03-21T09:59:09.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:08 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mon metadata", "id": "smithi174"}]: dispatch 2024-03-21T09:59:09.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:08 smithi174 ceph-mon[37496]: mon.smithi153 calling monitor election 2024-03-21T09:59:09.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:08 smithi174 ceph-mon[37496]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:59:09.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:08 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mon metadata", "id": "smithi174"}]: dispatch 2024-03-21T09:59:09.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:08 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mon metadata", "id": "smithi174"}]: dispatch 2024-03-21T09:59:09.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:08 smithi174 ceph-mon[37496]: mon.smithi174 calling monitor election 2024-03-21T09:59:09.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:08 smithi174 ceph-mon[37496]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:59:09.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:08 smithi174 ceph-mon[37496]: from='mgr.? 172.21.15.174:0/735165249' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi174.qpcwat/crt"}]: dispatch 2024-03-21T09:59:09.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:08 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mon metadata", "id": "smithi174"}]: dispatch 2024-03-21T09:59:09.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:08 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mon metadata", "id": "smithi174"}]: dispatch 2024-03-21T09:59:09.203 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:08 smithi174 ceph-mon[37496]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:59:09.203 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:08 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mon metadata", "id": "smithi174"}]: dispatch 2024-03-21T09:59:09.203 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:08 smithi174 ceph-mon[37496]: mon.smithi153 is new leader, mons smithi153,smithi174 in quorum (ranks 0,1) 2024-03-21T09:59:09.203 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:08 smithi174 ceph-mon[37496]: monmap e2: 2 mons at {smithi153=[v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0],smithi174=[v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-21T09:59:09.203 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:08 smithi174 ceph-mon[37496]: fsmap 2024-03-21T09:59:09.203 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:08 smithi174 ceph-mon[37496]: osdmap e5: 0 total, 0 up, 0 in 2024-03-21T09:59:09.203 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:08 smithi174 ceph-mon[37496]: mgrmap e17: smithi153.tluxni(active, since 44s) 2024-03-21T09:59:09.203 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:08 smithi174 ceph-mon[37496]: Standby manager daemon smithi174.qpcwat started 2024-03-21T09:59:09.203 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:08 smithi174 ceph-mon[37496]: from='mgr.? 172.21.15.174:0/735165249' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-21T09:59:09.203 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:08 smithi174 ceph-mon[37496]: overall HEALTH_OK 2024-03-21T09:59:09.203 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:08 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:59:09.203 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:08 smithi174 ceph-mon[37496]: from='mgr.? 172.21.15.174:0/735165249' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi174.qpcwat/key"}]: dispatch 2024-03-21T09:59:09.203 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:08 smithi174 ceph-mon[37496]: from='mgr.? 172.21.15.174:0/735165249' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-21T09:59:09.203 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:08 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:59:09.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:08 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mon metadata", "id": "smithi153"}]: dispatch 2024-03-21T09:59:09.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:08 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mon metadata", "id": "smithi174"}]: dispatch 2024-03-21T09:59:09.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:08 smithi153 ceph-mon[29117]: mon.smithi153 calling monitor election 2024-03-21T09:59:09.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:08 smithi153 ceph-mon[29117]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:59:09.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:08 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mon metadata", "id": "smithi174"}]: dispatch 2024-03-21T09:59:09.308 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:08 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mon metadata", "id": "smithi174"}]: dispatch 2024-03-21T09:59:09.308 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:08 smithi153 ceph-mon[29117]: mon.smithi174 calling monitor election 2024-03-21T09:59:09.308 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:08 smithi153 ceph-mon[29117]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:59:09.308 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:08 smithi153 ceph-mon[29117]: from='mgr.? 172.21.15.174:0/735165249' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi174.qpcwat/crt"}]: dispatch 2024-03-21T09:59:09.308 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:08 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mon metadata", "id": "smithi174"}]: dispatch 2024-03-21T09:59:09.308 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:08 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mon metadata", "id": "smithi174"}]: dispatch 2024-03-21T09:59:09.308 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:08 smithi153 ceph-mon[29117]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:59:09.308 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:08 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mon metadata", "id": "smithi174"}]: dispatch 2024-03-21T09:59:09.308 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:08 smithi153 ceph-mon[29117]: mon.smithi153 is new leader, mons smithi153,smithi174 in quorum (ranks 0,1) 2024-03-21T09:59:09.308 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:08 smithi153 ceph-mon[29117]: monmap e2: 2 mons at {smithi153=[v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0],smithi174=[v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-21T09:59:09.308 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:08 smithi153 ceph-mon[29117]: fsmap 2024-03-21T09:59:09.308 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:08 smithi153 ceph-mon[29117]: osdmap e5: 0 total, 0 up, 0 in 2024-03-21T09:59:09.308 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:08 smithi153 ceph-mon[29117]: mgrmap e17: smithi153.tluxni(active, since 44s) 2024-03-21T09:59:09.308 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:08 smithi153 ceph-mon[29117]: Standby manager daemon smithi174.qpcwat started 2024-03-21T09:59:09.309 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:08 smithi153 ceph-mon[29117]: from='mgr.? 172.21.15.174:0/735165249' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-21T09:59:09.309 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:08 smithi153 ceph-mon[29117]: overall HEALTH_OK 2024-03-21T09:59:09.309 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:08 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:59:09.309 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:08 smithi153 ceph-mon[29117]: from='mgr.? 172.21.15.174:0/735165249' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi174.qpcwat/key"}]: dispatch 2024-03-21T09:59:09.309 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:08 smithi153 ceph-mon[29117]: from='mgr.? 172.21.15.174:0/735165249' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-21T09:59:09.309 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:08 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:59:09.916 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:59:09.913+0000 7f7f48000700 1 -- 172.21.15.174:0/423397285 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7f28005cc0 msgr2=0x7f7f28008150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:59:09.916 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:59:09.913+0000 7f7f48000700 1 --2- 172.21.15.174:0/423397285 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7f28005cc0 0x7f7f28008150 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7f400745c0 tx=0x7f7f3400ff20 comp rx=0 tx=0).stop 2024-03-21T09:59:09.916 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:59:09.913+0000 7f7f48000700 1 -- 172.21.15.174:0/423397285 shutdown_connections 2024-03-21T09:59:09.916 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:59:09.913+0000 7f7f48000700 1 --2- 172.21.15.174:0/423397285 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7f28005cc0 0x7f7f28008150 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:59:09.916 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:59:09.913+0000 7f7f48000700 1 -- 172.21.15.174:0/423397285 >> 172.21.15.174:0/423397285 conn(0x7f7f400fc320 msgr2=0x7f7f400fe740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:59:09.916 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:59:09.914+0000 7f7f48000700 1 -- 172.21.15.174:0/423397285 shutdown_connections 2024-03-21T09:59:09.916 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:59:09.914+0000 7f7f48000700 1 -- 172.21.15.174:0/423397285 wait complete. 2024-03-21T09:59:09.916 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:59:09.915+0000 7f7f48000700 1 Processor -- start 2024-03-21T09:59:09.916 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:59:09.915+0000 7f7f48000700 1 -- start start 2024-03-21T09:59:09.917 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:59:09.916+0000 7f7f48000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7f28005cc0 0x7f7f401980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:59:09.917 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:59:09.916+0000 7f7f48000700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7f401985f0 0x7f7f4019ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:59:09.917 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:59:09.916+0000 7f7f48000700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f40198c10 con 0x7f7f28005cc0 2024-03-21T09:59:09.918 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:59:09.916+0000 7f7f48000700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f40198d80 con 0x7f7f401985f0 2024-03-21T09:59:09.918 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:59:09.916+0000 7f7f45600700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7f28005cc0 0x7f7f401980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:59:09.918 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:59:09.916+0000 7f7f45600700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7f28005cc0 0x7f7f401980b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.174:56728/0 (socket says 172.21.15.174:56728) 2024-03-21T09:59:09.918 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:59:09.916+0000 7f7f45600700 1 -- 172.21.15.174:0/1418755370 learned_addr learned my addr 172.21.15.174:0/1418755370 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-03-21T09:59:09.918 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:59:09.917+0000 7f7f44c00700 1 --2- 172.21.15.174:0/1418755370 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7f401985f0 0x7f7f4019ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:59:09.918 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:59:09.917+0000 7f7f44c00700 1 -- 172.21.15.174:0/1418755370 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7f401985f0 msgr2=0x7f7f4019ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-03-21T09:59:09.919 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:59:09.917+0000 7f7f44c00700 1 -- 172.21.15.174:0/1418755370 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7f401985f0 msgr2=0x7f7f4019ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-21T09:59:09.919 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:59:09.917+0000 7f7f44c00700 1 --2- 172.21.15.174:0/1418755370 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7f401985f0 0x7f7f4019ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-21T09:59:09.919 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:59:09.917+0000 7f7f44c00700 1 --2- 172.21.15.174:0/1418755370 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7f401985f0 0x7f7f4019ca60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-21T09:59:09.919 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:59:09.917+0000 7f7f45600700 1 -- 172.21.15.174:0/1418755370 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7f401985f0 msgr2=0x7f7f4019ca60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-21T09:59:09.919 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:59:09.917+0000 7f7f45600700 1 --2- 172.21.15.174:0/1418755370 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7f401985f0 0x7f7f4019ca60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:59:09.919 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:59:09.917+0000 7f7f45600700 1 -- 172.21.15.174:0/1418755370 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f3400f970 con 0x7f7f28005cc0 2024-03-21T09:59:09.919 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:59:09.918+0000 7f7f45600700 1 --2- 172.21.15.174:0/1418755370 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7f28005cc0 0x7f7f401980b0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7f34000c00 tx=0x7f7f34004710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:59:09.920 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:59:09.918+0000 7f7f3e000700 1 -- 172.21.15.174:0/1418755370 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f34021040 con 0x7f7f28005cc0 2024-03-21T09:59:09.920 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:59:09.918+0000 7f7f3e000700 1 -- 172.21.15.174:0/1418755370 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f7f340044d0 con 0x7f7f28005cc0 2024-03-21T09:59:09.920 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:59:09.918+0000 7f7f48000700 1 -- 172.21.15.174:0/1418755370 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f4019d000 con 0x7f7f28005cc0 2024-03-21T09:59:09.920 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:59:09.918+0000 7f7f3e000700 1 -- 172.21.15.174:0/1418755370 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f340142a0 con 0x7f7f28005cc0 2024-03-21T09:59:09.920 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:59:09.918+0000 7f7f48000700 1 -- 172.21.15.174:0/1418755370 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f4019d550 con 0x7f7f28005cc0 2024-03-21T09:59:09.920 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:59:09.920+0000 7f7f48000700 1 -- 172.21.15.174:0/1418755370 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f400623c0 con 0x7f7f28005cc0 2024-03-21T09:59:09.922 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:59:09.921+0000 7f7f3e000700 1 -- 172.21.15.174:0/1418755370 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 18) v1 ==== 90258+0+0 (secure 0 0 0) 0x7f7f34014510 con 0x7f7f28005cc0 2024-03-21T09:59:09.922 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:59:09.922+0000 7f7f3e000700 1 --2- 172.21.15.174:0/1418755370 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f7f2006c5c0 0x7f7f2006ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:59:09.923 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:59:09.922+0000 7f7f3e000700 1 -- 172.21.15.174:0/1418755370 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f7f3408da70 con 0x7f7f28005cc0 2024-03-21T09:59:09.923 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:59:09.922+0000 7f7f44c00700 1 --2- 172.21.15.174:0/1418755370 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f7f2006c5c0 0x7f7f2006ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:59:09.924 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:59:09.923+0000 7f7f44c00700 1 --2- 172.21.15.174:0/1418755370 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f7f2006c5c0 0x7f7f2006ea80 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f7f2c009850 tx=0x7f7f2c008040 comp rx=0 tx=0).ready entity=mgr.14221 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:59:09.936 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:59:09.935+0000 7f7f3e000700 1 -- 172.21.15.174:0/1418755370 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f7f340594d0 con 0x7f7f28005cc0 2024-03-21T09:59:10.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:09 smithi174 ceph-mon[37496]: mgrmap e18: smithi153.tluxni(active, since 44s), standbys: smithi174.qpcwat 2024-03-21T09:59:10.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:09 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:59:10.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:09 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mgr metadata", "who": "smithi174.qpcwat", "id": "smithi174.qpcwat"}]: dispatch 2024-03-21T09:59:10.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:09 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:59:10.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:09 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:59:10.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:09 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:59:10.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:09 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T09:59:10.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:09 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:59:10.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:09 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mon metadata", "id": "smithi174"}]: dispatch 2024-03-21T09:59:10.273 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:59:10.273+0000 7f7f48000700 1 -- 172.21.15.174:0/1418755370 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f7f40199410 con 0x7f7f28005cc0 2024-03-21T09:59:10.275 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:59:10.274+0000 7f7f3e000700 1 -- 172.21.15.174:0/1418755370 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1026 (secure 0 0 0) 0x7f7f3402b420 con 0x7f7f28005cc0 2024-03-21T09:59:10.275 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-21T09:59:10.275 INFO:teuthology.orchestra.run.smithi174.stdout:{"epoch":2,"fsid":"070ed3e4-e769-11ee-95cd-87774f69a715","modified":"2024-03-21T09:59:03.783609Z","created":"2024-03-21T09:55:15.397487Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi153","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:3300","nonce":0},{"type":"v1","addr":"172.21.15.153:6789","nonce":0}]},"addr":"172.21.15.153:6789/0","public_addr":"172.21.15.153:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"smithi174","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:3300","nonce":0},{"type":"v1","addr":"172.21.15.174:6789","nonce":0}]},"addr":"172.21.15.174:6789/0","public_addr":"172.21.15.174:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-21T09:59:10.277 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:59:10.276+0000 7f7f48000700 1 -- 172.21.15.174:0/1418755370 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f7f2006c5c0 msgr2=0x7f7f2006ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:59:10.278 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:59:10.276+0000 7f7f48000700 1 --2- 172.21.15.174:0/1418755370 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f7f2006c5c0 0x7f7f2006ea80 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f7f2c009850 tx=0x7f7f2c008040 comp rx=0 tx=0).stop 2024-03-21T09:59:10.278 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:59:10.276+0000 7f7f48000700 1 -- 172.21.15.174:0/1418755370 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7f28005cc0 msgr2=0x7f7f401980b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:59:10.278 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:59:10.276+0000 7f7f48000700 1 --2- 172.21.15.174:0/1418755370 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7f28005cc0 0x7f7f401980b0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7f34000c00 tx=0x7f7f34004710 comp rx=0 tx=0).stop 2024-03-21T09:59:10.278 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:59:10.277+0000 7f7f48000700 1 -- 172.21.15.174:0/1418755370 shutdown_connections 2024-03-21T09:59:10.278 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:59:10.277+0000 7f7f48000700 1 --2- 172.21.15.174:0/1418755370 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f7f2006c5c0 0x7f7f2006ea80 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:59:10.278 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:59:10.277+0000 7f7f48000700 1 --2- 172.21.15.174:0/1418755370 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7f28005cc0 0x7f7f401980b0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:59:10.278 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:59:10.277+0000 7f7f48000700 1 --2- 172.21.15.174:0/1418755370 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7f401985f0 0x7f7f4019ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:59:10.278 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:59:10.277+0000 7f7f48000700 1 -- 172.21.15.174:0/1418755370 >> 172.21.15.174:0/1418755370 conn(0x7f7f400fc320 msgr2=0x7f7f40104ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:59:10.278 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:59:10.277+0000 7f7f48000700 1 -- 172.21.15.174:0/1418755370 shutdown_connections 2024-03-21T09:59:10.278 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T09:59:10.277+0000 7f7f48000700 1 -- 172.21.15.174:0/1418755370 wait complete. 2024-03-21T09:59:10.279 INFO:teuthology.orchestra.run.smithi174.stderr:dumped monmap epoch 2 2024-03-21T09:59:10.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:09 smithi153 ceph-mon[29117]: mgrmap e18: smithi153.tluxni(active, since 44s), standbys: smithi174.qpcwat 2024-03-21T09:59:10.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:09 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:59:10.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:09 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mgr metadata", "who": "smithi174.qpcwat", "id": "smithi174.qpcwat"}]: dispatch 2024-03-21T09:59:10.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:09 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:59:10.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:09 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:59:10.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:09 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:59:10.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:09 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T09:59:10.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:09 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:59:10.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:09 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mon metadata", "id": "smithi174"}]: dispatch 2024-03-21T09:59:11.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:10 smithi153 ceph-mon[29117]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:59:11.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:10 smithi153 ceph-mon[29117]: from='client.? 172.21.15.174:0/1418755370' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-21T09:59:11.123 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:10 smithi174 ceph-mon[37496]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:59:11.123 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:10 smithi174 ceph-mon[37496]: from='client.? 172.21.15.174:0/1418755370' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-21T09:59:11.169 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-03-21T09:59:11.169 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 070ed3e4-e769-11ee-95cd-87774f69a715 -- ceph config generate-minimal-conf 2024-03-21T09:59:11.584 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi153/config 2024-03-21T09:59:13.540 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:59:13.538+0000 7f7f3b000700 1 -- 172.21.15.153:0/4180501024 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7f34106830 msgr2=0x7f7f34106c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:59:13.541 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:59:13.538+0000 7f7f3b000700 1 --2- 172.21.15.153:0/4180501024 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7f34106830 0x7f7f34106c10 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7f24009910 tx=0x7f7f24009c20 comp rx=0 tx=0).stop 2024-03-21T09:59:13.541 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:59:13.538+0000 7f7f3b000700 1 -- 172.21.15.153:0/4180501024 shutdown_connections 2024-03-21T09:59:13.541 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:59:13.538+0000 7f7f3b000700 1 --2- 172.21.15.153:0/4180501024 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7f34106830 0x7f7f34106c10 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:59:13.541 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:59:13.538+0000 7f7f3b000700 1 -- 172.21.15.153:0/4180501024 >> 172.21.15.153:0/4180501024 conn(0x7f7f340fc380 msgr2=0x7f7f340fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:59:13.541 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:59:13.538+0000 7f7f3b000700 1 -- 172.21.15.153:0/4180501024 shutdown_connections 2024-03-21T09:59:13.541 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:59:13.538+0000 7f7f3b000700 1 -- 172.21.15.153:0/4180501024 wait complete. 2024-03-21T09:59:13.542 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:59:13.540+0000 7f7f3b000700 1 Processor -- start 2024-03-21T09:59:13.542 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:59:13.540+0000 7f7f3b000700 1 -- start start 2024-03-21T09:59:13.542 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:59:13.540+0000 7f7f3b000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7f34106830 0x7f7f341980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:59:13.542 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:59:13.540+0000 7f7f3b000700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7f34198600 0x7f7f3419ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:59:13.542 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:59:13.540+0000 7f7f3b000700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f34198c20 con 0x7f7f34106830 2024-03-21T09:59:13.542 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:59:13.540+0000 7f7f3b000700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f34198d90 con 0x7f7f34198600 2024-03-21T09:59:13.542 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:59:13.540+0000 7f7f33e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7f34106830 0x7f7f341980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:59:13.542 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:59:13.540+0000 7f7f33e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7f34106830 0x7f7f341980c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:57762/0 (socket says 172.21.15.153:57762) 2024-03-21T09:59:13.542 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:59:13.540+0000 7f7f33e00700 1 -- 172.21.15.153:0/1904708644 learned_addr learned my addr 172.21.15.153:0/1904708644 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T09:59:13.543 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:59:13.541+0000 7f7f33400700 1 --2- 172.21.15.153:0/1904708644 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7f34198600 0x7f7f3419ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:59:13.543 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:59:13.541+0000 7f7f33e00700 1 -- 172.21.15.153:0/1904708644 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7f34198600 msgr2=0x7f7f3419ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:59:13.543 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:59:13.541+0000 7f7f33e00700 1 --2- 172.21.15.153:0/1904708644 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7f34198600 0x7f7f3419ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:59:13.543 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:59:13.541+0000 7f7f33e00700 1 -- 172.21.15.153:0/1904708644 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f240095f0 con 0x7f7f34106830 2024-03-21T09:59:13.543 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:59:13.541+0000 7f7f33e00700 1 --2- 172.21.15.153:0/1904708644 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7f34106830 0x7f7f341980c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7f24006010 tx=0x7f7f2400bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:59:13.543 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:59:13.541+0000 7f7f30c00700 1 -- 172.21.15.153:0/1904708644 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f2401d070 con 0x7f7f34106830 2024-03-21T09:59:13.543 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:59:13.542+0000 7f7f30c00700 1 -- 172.21.15.153:0/1904708644 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f7f24005140 con 0x7f7f34106830 2024-03-21T09:59:13.544 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:59:13.542+0000 7f7f3b000700 1 -- 172.21.15.153:0/1904708644 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f3419d070 con 0x7f7f34106830 2024-03-21T09:59:13.544 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:59:13.542+0000 7f7f30c00700 1 -- 172.21.15.153:0/1904708644 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f24017440 con 0x7f7f34106830 2024-03-21T09:59:13.544 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:59:13.542+0000 7f7f3b000700 1 -- 172.21.15.153:0/1904708644 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f3419d5c0 con 0x7f7f34106830 2024-03-21T09:59:13.544 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:59:13.543+0000 7f7f3b000700 1 -- 172.21.15.153:0/1904708644 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f3419d8a0 con 0x7f7f34106830 2024-03-21T09:59:13.545 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:59:13.544+0000 7f7f30c00700 1 -- 172.21.15.153:0/1904708644 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 18) v1 ==== 90258+0+0 (secure 0 0 0) 0x7f7f2400f460 con 0x7f7f34106830 2024-03-21T09:59:13.547 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:59:13.545+0000 7f7f30c00700 1 --2- 172.21.15.153:0/1904708644 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f7f1806c5c0 0x7f7f1806ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:59:13.547 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:59:13.545+0000 7f7f30c00700 1 -- 172.21.15.153:0/1904708644 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f7f2408ba60 con 0x7f7f34106830 2024-03-21T09:59:13.547 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:59:13.546+0000 7f7f33400700 1 --2- 172.21.15.153:0/1904708644 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f7f1806c5c0 0x7f7f1806ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:59:13.548 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:59:13.546+0000 7f7f33400700 1 --2- 172.21.15.153:0/1904708644 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f7f1806c5c0 0x7f7f1806ea80 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7f20009850 tx=0x7f7f20008040 comp rx=0 tx=0).ready entity=mgr.14221 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:59:13.551 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:59:13.550+0000 7f7f30c00700 1 -- 172.21.15.153:0/1904708644 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f7f24057440 con 0x7f7f34106830 2024-03-21T09:59:13.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:13 smithi174 ceph-mon[37496]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:59:13.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:13 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:59:13.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:13 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:59:13.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:13 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:59:13.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:13 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:59:13.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:13 smithi153 ceph-mon[29117]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:59:13.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:13 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:59:13.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:13 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:59:13.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:13 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:59:13.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:13 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:59:13.813 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:59:13.812+0000 7f7f3b000700 1 -- 172.21.15.153:0/1904708644 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f7f340623c0 con 0x7f7f34106830 2024-03-21T09:59:13.813 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:59:13.812+0000 7f7f30c00700 1 -- 172.21.15.153:0/1904708644 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v10) v1 ==== 76+0+227 (secure 0 0 0) 0x7f7f24026100 con 0x7f7f34106830 2024-03-21T09:59:13.814 INFO:teuthology.orchestra.run.smithi153.stdout:# minimal ceph.conf for 070ed3e4-e769-11ee-95cd-87774f69a715 2024-03-21T09:59:13.814 INFO:teuthology.orchestra.run.smithi153.stdout:[global] 2024-03-21T09:59:13.814 INFO:teuthology.orchestra.run.smithi153.stdout: fsid = 070ed3e4-e769-11ee-95cd-87774f69a715 2024-03-21T09:59:13.814 INFO:teuthology.orchestra.run.smithi153.stdout: mon_host = [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] 2024-03-21T09:59:13.816 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:59:13.815+0000 7f7f3b000700 1 -- 172.21.15.153:0/1904708644 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f7f1806c5c0 msgr2=0x7f7f1806ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:59:13.816 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:59:13.815+0000 7f7f3b000700 1 --2- 172.21.15.153:0/1904708644 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f7f1806c5c0 0x7f7f1806ea80 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7f20009850 tx=0x7f7f20008040 comp rx=0 tx=0).stop 2024-03-21T09:59:13.816 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:59:13.815+0000 7f7f3b000700 1 -- 172.21.15.153:0/1904708644 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7f34106830 msgr2=0x7f7f341980c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:59:13.816 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:59:13.815+0000 7f7f3b000700 1 --2- 172.21.15.153:0/1904708644 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7f34106830 0x7f7f341980c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7f24006010 tx=0x7f7f2400bda0 comp rx=0 tx=0).stop 2024-03-21T09:59:13.817 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:59:13.815+0000 7f7f3b000700 1 -- 172.21.15.153:0/1904708644 shutdown_connections 2024-03-21T09:59:13.817 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:59:13.815+0000 7f7f3b000700 1 --2- 172.21.15.153:0/1904708644 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f7f1806c5c0 0x7f7f1806ea80 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:59:13.817 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:59:13.815+0000 7f7f3b000700 1 --2- 172.21.15.153:0/1904708644 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7f34106830 0x7f7f341980c0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:59:13.817 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:59:13.815+0000 7f7f3b000700 1 --2- 172.21.15.153:0/1904708644 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7f34198600 0x7f7f3419ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:59:13.817 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:59:13.815+0000 7f7f3b000700 1 -- 172.21.15.153:0/1904708644 >> 172.21.15.153:0/1904708644 conn(0x7f7f340fc380 msgr2=0x7f7f34108c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:59:13.817 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:59:13.815+0000 7f7f3b000700 1 -- 172.21.15.153:0/1904708644 shutdown_connections 2024-03-21T09:59:13.817 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:59:13.815+0000 7f7f3b000700 1 -- 172.21.15.153:0/1904708644 wait complete. 2024-03-21T09:59:14.341 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:14 smithi153 ceph-mon[29117]: Updating smithi153:/etc/ceph/ceph.conf 2024-03-21T09:59:14.342 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:14 smithi153 ceph-mon[29117]: Updating smithi174:/etc/ceph/ceph.conf 2024-03-21T09:59:14.342 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:14 smithi153 ceph-mon[29117]: Updating smithi153:/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T09:59:14.342 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:14 smithi153 ceph-mon[29117]: Updating smithi174:/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T09:59:14.342 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:14 smithi153 ceph-mon[29117]: from='client.? 172.21.15.153:0/1904708644' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:59:14.342 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:14 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:59:14.342 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:14 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:59:14.342 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:14 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:59:14.342 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:14 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:59:14.342 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:14 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:59:14.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:14 smithi174 ceph-mon[37496]: Updating smithi153:/etc/ceph/ceph.conf 2024-03-21T09:59:14.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:14 smithi174 ceph-mon[37496]: Updating smithi174:/etc/ceph/ceph.conf 2024-03-21T09:59:14.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:14 smithi174 ceph-mon[37496]: Updating smithi153:/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T09:59:14.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:14 smithi174 ceph-mon[37496]: Updating smithi174:/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T09:59:14.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:14 smithi174 ceph-mon[37496]: from='client.? 172.21.15.153:0/1904708644' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:59:14.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:14 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:59:14.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:14 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:59:14.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:14 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:59:14.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:14 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:59:14.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:14 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:59:14.748 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-03-21T09:59:14.748 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-03-21T09:59:14.748 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd of=/etc/ceph/ceph.conf 2024-03-21T09:59:14.825 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-03-21T09:59:14.825 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-21T09:59:14.948 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-03-21T09:59:14.948 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/etc/ceph/ceph.conf 2024-03-21T09:59:15.054 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-03-21T09:59:15.054 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-21T09:59:15.192 INFO:tasks.cephadm:Deploying OSDs... 2024-03-21T09:59:15.193 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-03-21T09:59:15.193 DEBUG:teuthology.orchestra.run.smithi153:> dd if=/scratch_devs of=/dev/stdout 2024-03-21T09:59:15.234 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-21T09:59:15.234 DEBUG:teuthology.orchestra.run.smithi153:> stat /dev/vg_nvme/lv_1 2024-03-21T09:59:15.325 INFO:teuthology.orchestra.run.smithi153.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-21T09:59:15.325 INFO:teuthology.orchestra.run.smithi153.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-21T09:59:15.325 INFO:teuthology.orchestra.run.smithi153.stdout:Device: 5h/5d Inode: 638 Links: 1 2024-03-21T09:59:15.325 INFO:teuthology.orchestra.run.smithi153.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-21T09:59:15.325 INFO:teuthology.orchestra.run.smithi153.stdout:Context: system_u:object_r:device_t:s0 2024-03-21T09:59:15.325 INFO:teuthology.orchestra.run.smithi153.stdout:Access: 2024-03-21 09:58:13.349368732 +0000 2024-03-21T09:59:15.325 INFO:teuthology.orchestra.run.smithi153.stdout:Modify: 2024-03-21 09:51:30.721802575 +0000 2024-03-21T09:59:15.325 INFO:teuthology.orchestra.run.smithi153.stdout:Change: 2024-03-21 09:51:30.721802575 +0000 2024-03-21T09:59:15.325 INFO:teuthology.orchestra.run.smithi153.stdout: Birth: 2024-03-21 09:51:30.721802575 +0000 2024-03-21T09:59:15.325 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-21T09:59:15.412 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records in 2024-03-21T09:59:15.412 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records out 2024-03-21T09:59:15.413 INFO:teuthology.orchestra.run.smithi153.stderr:512 bytes copied, 0.000372847 s, 1.4 MB/s 2024-03-21T09:59:15.418 DEBUG:teuthology.orchestra.run.smithi153:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-21T09:59:15.460 DEBUG:teuthology.orchestra.run.smithi153:> stat /dev/vg_nvme/lv_2 2024-03-21T09:59:15.550 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:15 smithi153 ceph-mon[29117]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:59:15.550 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:15 smithi153 ceph-mon[29117]: Reconfiguring mon.smithi153 (unknown last config time)... 2024-03-21T09:59:15.550 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:15 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T09:59:15.550 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:15 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-21T09:59:15.550 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:15 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:59:15.550 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:15 smithi153 ceph-mon[29117]: Reconfiguring daemon mon.smithi153 on smithi153 2024-03-21T09:59:15.551 INFO:teuthology.orchestra.run.smithi153.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-21T09:59:15.551 INFO:teuthology.orchestra.run.smithi153.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-21T09:59:15.551 INFO:teuthology.orchestra.run.smithi153.stdout:Device: 5h/5d Inode: 659 Links: 1 2024-03-21T09:59:15.551 INFO:teuthology.orchestra.run.smithi153.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-21T09:59:15.551 INFO:teuthology.orchestra.run.smithi153.stdout:Context: system_u:object_r:device_t:s0 2024-03-21T09:59:15.552 INFO:teuthology.orchestra.run.smithi153.stdout:Access: 2024-03-21 09:58:13.350368713 +0000 2024-03-21T09:59:15.552 INFO:teuthology.orchestra.run.smithi153.stdout:Modify: 2024-03-21 09:51:31.290792455 +0000 2024-03-21T09:59:15.552 INFO:teuthology.orchestra.run.smithi153.stdout:Change: 2024-03-21 09:51:31.290792455 +0000 2024-03-21T09:59:15.552 INFO:teuthology.orchestra.run.smithi153.stdout: Birth: 2024-03-21 09:51:31.290792455 +0000 2024-03-21T09:59:15.552 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-21T09:59:15.620 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records in 2024-03-21T09:59:15.620 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records out 2024-03-21T09:59:15.620 INFO:teuthology.orchestra.run.smithi153.stderr:512 bytes copied, 0.000313699 s, 1.6 MB/s 2024-03-21T09:59:15.623 DEBUG:teuthology.orchestra.run.smithi153:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-21T09:59:15.661 DEBUG:teuthology.orchestra.run.smithi153:> stat /dev/vg_nvme/lv_3 2024-03-21T09:59:15.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:15 smithi174 ceph-mon[37496]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:59:15.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:15 smithi174 ceph-mon[37496]: Reconfiguring mon.smithi153 (unknown last config time)... 2024-03-21T09:59:15.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:15 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T09:59:15.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:15 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-21T09:59:15.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:15 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:59:15.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:15 smithi174 ceph-mon[37496]: Reconfiguring daemon mon.smithi153 on smithi153 2024-03-21T09:59:15.744 INFO:teuthology.orchestra.run.smithi153.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-21T09:59:15.744 INFO:teuthology.orchestra.run.smithi153.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-21T09:59:15.744 INFO:teuthology.orchestra.run.smithi153.stdout:Device: 5h/5d Inode: 678 Links: 1 2024-03-21T09:59:15.744 INFO:teuthology.orchestra.run.smithi153.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-21T09:59:15.744 INFO:teuthology.orchestra.run.smithi153.stdout:Context: system_u:object_r:device_t:s0 2024-03-21T09:59:15.744 INFO:teuthology.orchestra.run.smithi153.stdout:Access: 2024-03-21 09:58:13.350368713 +0000 2024-03-21T09:59:15.744 INFO:teuthology.orchestra.run.smithi153.stdout:Modify: 2024-03-21 09:51:31.853782441 +0000 2024-03-21T09:59:15.744 INFO:teuthology.orchestra.run.smithi153.stdout:Change: 2024-03-21 09:51:31.853782441 +0000 2024-03-21T09:59:15.744 INFO:teuthology.orchestra.run.smithi153.stdout: Birth: 2024-03-21 09:51:31.853782441 +0000 2024-03-21T09:59:15.744 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-21T09:59:15.868 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records in 2024-03-21T09:59:15.868 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records out 2024-03-21T09:59:15.868 INFO:teuthology.orchestra.run.smithi153.stderr:512 bytes copied, 0.00034613 s, 1.5 MB/s 2024-03-21T09:59:15.872 DEBUG:teuthology.orchestra.run.smithi153:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-21T09:59:15.928 DEBUG:teuthology.orchestra.run.smithi153:> stat /dev/vg_nvme/lv_4 2024-03-21T09:59:15.974 INFO:teuthology.orchestra.run.smithi153.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-21T09:59:15.974 INFO:teuthology.orchestra.run.smithi153.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-21T09:59:15.974 INFO:teuthology.orchestra.run.smithi153.stdout:Device: 5h/5d Inode: 696 Links: 1 2024-03-21T09:59:15.974 INFO:teuthology.orchestra.run.smithi153.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-21T09:59:15.974 INFO:teuthology.orchestra.run.smithi153.stdout:Context: system_u:object_r:device_t:s0 2024-03-21T09:59:15.974 INFO:teuthology.orchestra.run.smithi153.stdout:Access: 2024-03-21 09:58:13.351368696 +0000 2024-03-21T09:59:15.974 INFO:teuthology.orchestra.run.smithi153.stdout:Modify: 2024-03-21 09:51:32.414772464 +0000 2024-03-21T09:59:15.974 INFO:teuthology.orchestra.run.smithi153.stdout:Change: 2024-03-21 09:51:32.414772464 +0000 2024-03-21T09:59:15.974 INFO:teuthology.orchestra.run.smithi153.stdout: Birth: 2024-03-21 09:51:32.414772464 +0000 2024-03-21T09:59:15.974 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-21T09:59:16.103 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records in 2024-03-21T09:59:16.103 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records out 2024-03-21T09:59:16.103 INFO:teuthology.orchestra.run.smithi153.stderr:512 bytes copied, 0.000321741 s, 1.6 MB/s 2024-03-21T09:59:16.106 DEBUG:teuthology.orchestra.run.smithi153:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-21T09:59:16.153 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-03-21T09:59:16.153 DEBUG:teuthology.orchestra.run.smithi174:> dd if=/scratch_devs of=/dev/stdout 2024-03-21T09:59:16.210 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-21T09:59:16.211 DEBUG:teuthology.orchestra.run.smithi174:> stat /dev/vg_nvme/lv_1 2024-03-21T09:59:16.262 INFO:teuthology.orchestra.run.smithi174.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-21T09:59:16.262 INFO:teuthology.orchestra.run.smithi174.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-21T09:59:16.262 INFO:teuthology.orchestra.run.smithi174.stdout:Device: 5h/5d Inode: 634 Links: 1 2024-03-21T09:59:16.262 INFO:teuthology.orchestra.run.smithi174.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-21T09:59:16.262 INFO:teuthology.orchestra.run.smithi174.stdout:Context: system_u:object_r:device_t:s0 2024-03-21T09:59:16.262 INFO:teuthology.orchestra.run.smithi174.stdout:Access: 2024-03-21 09:59:02.176634194 +0000 2024-03-21T09:59:16.263 INFO:teuthology.orchestra.run.smithi174.stdout:Modify: 2024-03-21 09:51:30.588717976 +0000 2024-03-21T09:59:16.263 INFO:teuthology.orchestra.run.smithi174.stdout:Change: 2024-03-21 09:51:30.588717976 +0000 2024-03-21T09:59:16.263 INFO:teuthology.orchestra.run.smithi174.stdout: Birth: 2024-03-21 09:51:30.588717976 +0000 2024-03-21T09:59:16.263 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-21T09:59:16.355 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records in 2024-03-21T09:59:16.355 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records out 2024-03-21T09:59:16.355 INFO:teuthology.orchestra.run.smithi174.stderr:512 bytes copied, 0.00033662 s, 1.5 MB/s 2024-03-21T09:59:16.359 DEBUG:teuthology.orchestra.run.smithi174:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-21T09:59:16.416 DEBUG:teuthology.orchestra.run.smithi174:> stat /dev/vg_nvme/lv_2 2024-03-21T09:59:16.466 INFO:teuthology.orchestra.run.smithi174.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-21T09:59:16.467 INFO:teuthology.orchestra.run.smithi174.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-21T09:59:16.467 INFO:teuthology.orchestra.run.smithi174.stdout:Device: 5h/5d Inode: 652 Links: 1 2024-03-21T09:59:16.467 INFO:teuthology.orchestra.run.smithi174.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-21T09:59:16.467 INFO:teuthology.orchestra.run.smithi174.stdout:Context: system_u:object_r:device_t:s0 2024-03-21T09:59:16.467 INFO:teuthology.orchestra.run.smithi174.stdout:Access: 2024-03-21 09:59:02.176634194 +0000 2024-03-21T09:59:16.467 INFO:teuthology.orchestra.run.smithi174.stdout:Modify: 2024-03-21 09:51:31.134710730 +0000 2024-03-21T09:59:16.467 INFO:teuthology.orchestra.run.smithi174.stdout:Change: 2024-03-21 09:51:31.134710730 +0000 2024-03-21T09:59:16.467 INFO:teuthology.orchestra.run.smithi174.stdout: Birth: 2024-03-21 09:51:31.134710730 +0000 2024-03-21T09:59:16.467 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-21T09:59:16.559 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records in 2024-03-21T09:59:16.559 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records out 2024-03-21T09:59:16.559 INFO:teuthology.orchestra.run.smithi174.stderr:512 bytes copied, 0.000343287 s, 1.5 MB/s 2024-03-21T09:59:16.564 DEBUG:teuthology.orchestra.run.smithi174:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-21T09:59:16.619 DEBUG:teuthology.orchestra.run.smithi174:> stat /dev/vg_nvme/lv_3 2024-03-21T09:59:16.667 INFO:teuthology.orchestra.run.smithi174.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-21T09:59:16.667 INFO:teuthology.orchestra.run.smithi174.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-21T09:59:16.667 INFO:teuthology.orchestra.run.smithi174.stdout:Device: 5h/5d Inode: 669 Links: 1 2024-03-21T09:59:16.667 INFO:teuthology.orchestra.run.smithi174.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-21T09:59:16.667 INFO:teuthology.orchestra.run.smithi174.stdout:Context: system_u:object_r:device_t:s0 2024-03-21T09:59:16.667 INFO:teuthology.orchestra.run.smithi174.stdout:Access: 2024-03-21 09:59:02.177634180 +0000 2024-03-21T09:59:16.667 INFO:teuthology.orchestra.run.smithi174.stdout:Modify: 2024-03-21 09:51:31.697703258 +0000 2024-03-21T09:59:16.667 INFO:teuthology.orchestra.run.smithi174.stdout:Change: 2024-03-21 09:51:31.697703258 +0000 2024-03-21T09:59:16.667 INFO:teuthology.orchestra.run.smithi174.stdout: Birth: 2024-03-21 09:51:31.697703258 +0000 2024-03-21T09:59:16.668 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-21T09:59:16.755 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records in 2024-03-21T09:59:16.755 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records out 2024-03-21T09:59:16.755 INFO:teuthology.orchestra.run.smithi174.stderr:512 bytes copied, 0.000384542 s, 1.3 MB/s 2024-03-21T09:59:16.760 DEBUG:teuthology.orchestra.run.smithi174:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-21T09:59:16.814 DEBUG:teuthology.orchestra.run.smithi174:> stat /dev/vg_nvme/lv_4 2024-03-21T09:59:16.860 INFO:teuthology.orchestra.run.smithi174.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-21T09:59:16.860 INFO:teuthology.orchestra.run.smithi174.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-21T09:59:16.860 INFO:teuthology.orchestra.run.smithi174.stdout:Device: 5h/5d Inode: 689 Links: 1 2024-03-21T09:59:16.860 INFO:teuthology.orchestra.run.smithi174.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-21T09:59:16.860 INFO:teuthology.orchestra.run.smithi174.stdout:Context: system_u:object_r:device_t:s0 2024-03-21T09:59:16.861 INFO:teuthology.orchestra.run.smithi174.stdout:Access: 2024-03-21 09:59:02.178634167 +0000 2024-03-21T09:59:16.861 INFO:teuthology.orchestra.run.smithi174.stdout:Modify: 2024-03-21 09:51:32.292695362 +0000 2024-03-21T09:59:16.861 INFO:teuthology.orchestra.run.smithi174.stdout:Change: 2024-03-21 09:51:32.292695362 +0000 2024-03-21T09:59:16.861 INFO:teuthology.orchestra.run.smithi174.stdout: Birth: 2024-03-21 09:51:32.292695362 +0000 2024-03-21T09:59:16.861 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-21T09:59:16.995 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records in 2024-03-21T09:59:16.995 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records out 2024-03-21T09:59:16.995 INFO:teuthology.orchestra.run.smithi174.stderr:512 bytes copied, 0.000362881 s, 1.4 MB/s 2024-03-21T09:59:16.999 DEBUG:teuthology.orchestra.run.smithi174:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-21T09:59:17.055 INFO:tasks.cephadm:Deploying osd.0 on smithi153 with /dev/vg_nvme/lv_4... 2024-03-21T09:59:17.056 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 070ed3e4-e769-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-21T09:59:17.764 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:17 smithi153 ceph-mon[29117]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:59:17.764 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:17 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:59:17.765 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:17 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:59:17.765 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:17 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi153.tluxni", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-21T09:59:17.765 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:17 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-21T09:59:17.765 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:17 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:59:17.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:17 smithi174 ceph-mon[37496]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:59:17.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:17 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:59:17.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:17 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:59:17.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:17 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi153.tluxni", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-21T09:59:17.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:17 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-21T09:59:17.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:17 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:59:18.437 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi153/config 2024-03-21T09:59:18.618 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:18 smithi153 ceph-mon[29117]: Reconfiguring mgr.smithi153.tluxni (unknown last config time)... 2024-03-21T09:59:18.618 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:18 smithi153 ceph-mon[29117]: Reconfiguring daemon mgr.smithi153.tluxni on smithi153 2024-03-21T09:59:18.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:18 smithi174 ceph-mon[37496]: Reconfiguring mgr.smithi153.tluxni (unknown last config time)... 2024-03-21T09:59:18.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:18 smithi174 ceph-mon[37496]: Reconfiguring daemon mgr.smithi153.tluxni on smithi153 2024-03-21T09:59:19.749 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:19 smithi153 ceph-mon[29117]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:59:19.749 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:19 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:59:19.749 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:19 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:59:19.749 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:19 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi153", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-21T09:59:19.749 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:19 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:59:19.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:19 smithi174 ceph-mon[37496]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:59:19.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:19 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:59:19.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:19 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:59:19.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:19 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi153", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-21T09:59:19.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:19 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:59:20.705 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:20 smithi153 ceph-mon[29117]: Reconfiguring ceph-exporter.smithi153 (monmap changed)... 2024-03-21T09:59:20.705 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:20 smithi153 ceph-mon[29117]: Reconfiguring daemon ceph-exporter.smithi153 on smithi153 2024-03-21T09:59:20.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:20 smithi174 ceph-mon[37496]: Reconfiguring ceph-exporter.smithi153 (monmap changed)... 2024-03-21T09:59:20.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:20 smithi174 ceph-mon[37496]: Reconfiguring daemon ceph-exporter.smithi153 on smithi153 2024-03-21T09:59:21.280 INFO:teuthology.orchestra.run.smithi153.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-21T09:59:21.280 INFO:teuthology.orchestra.run.smithi153.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-21T09:59:21.280 INFO:teuthology.orchestra.run.smithi153.stderr: stderr: 10+0 records in 2024-03-21T09:59:21.280 INFO:teuthology.orchestra.run.smithi153.stderr:10+0 records out 2024-03-21T09:59:21.280 INFO:teuthology.orchestra.run.smithi153.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.047231 s, 222 MB/s 2024-03-21T09:59:21.280 INFO:teuthology.orchestra.run.smithi153.stderr:--> Zapping successful for: 2024-03-21T09:59:21.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:21 smithi153 ceph-mon[29117]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:59:21.827 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:21 smithi174 ceph-mon[37496]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:59:22.306 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 070ed3e4-e769-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi153:vg_nvme/lv_4 2024-03-21T09:59:22.960 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:22 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:59:22.960 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:22 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:59:22.960 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:22 smithi153 ceph-mon[29117]: Reconfiguring crash.smithi153 (monmap changed)... 2024-03-21T09:59:22.960 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:22 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi153", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-21T09:59:22.960 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:22 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:59:22.960 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:22 smithi153 ceph-mon[29117]: Reconfiguring daemon crash.smithi153 on smithi153 2024-03-21T09:59:22.960 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:22 smithi153 ceph-mon[29117]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:59:23.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:22 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:59:23.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:22 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:59:23.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:22 smithi174 ceph-mon[37496]: Reconfiguring crash.smithi153 (monmap changed)... 2024-03-21T09:59:23.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:22 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi153", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-21T09:59:23.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:22 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:59:23.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:22 smithi174 ceph-mon[37496]: Reconfiguring daemon crash.smithi153 on smithi153 2024-03-21T09:59:23.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:22 smithi174 ceph-mon[37496]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:59:23.431 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi153/config 2024-03-21T09:59:24.700 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:24 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:59:24.700 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:24 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:59:24.700 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:24 smithi153 ceph-mon[29117]: Reconfiguring alertmanager.smithi153 (dependencies changed)... 2024-03-21T09:59:24.700 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:24 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T09:59:24.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:24 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:59:24.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:24 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:59:24.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:24 smithi174 ceph-mon[37496]: Reconfiguring alertmanager.smithi153 (dependencies changed)... 2024-03-21T09:59:24.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:24 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T09:59:25.555 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:25 smithi153 ceph-mon[29117]: Reconfiguring daemon alertmanager.smithi153 on smithi153 2024-03-21T09:59:25.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:25 smithi153 ceph-mon[29117]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:59:25.853 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:25 smithi174 ceph-mon[37496]: Reconfiguring daemon alertmanager.smithi153 on smithi153 2024-03-21T09:59:25.854 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:25 smithi174 ceph-mon[37496]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:59:26.636 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:59:26.635+0000 7fefa2600700 1 -- 172.21.15.153:0/1274034154 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fef9c0737f0 msgr2=0x7fef9c073c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:59:26.636 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:59:26.635+0000 7fefa2600700 1 --2- 172.21.15.153:0/1274034154 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fef9c0737f0 0x7fef9c073c70 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fef84009910 tx=0x7fef84009c20 comp rx=0 tx=0).stop 2024-03-21T09:59:26.637 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:59:26.635+0000 7fefa2600700 1 -- 172.21.15.153:0/1274034154 shutdown_connections 2024-03-21T09:59:26.637 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:59:26.635+0000 7fefa2600700 1 --2- 172.21.15.153:0/1274034154 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fef9c0737f0 0x7fef9c073c70 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:59:26.637 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:59:26.635+0000 7fefa2600700 1 --2- 172.21.15.153:0/1274034154 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fef9c074dc0 0x7fef9c073220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:59:26.637 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:59:26.635+0000 7fefa2600700 1 -- 172.21.15.153:0/1274034154 >> 172.21.15.153:0/1274034154 conn(0x7fef9c0fbf60 msgr2=0x7fef9c0fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:59:26.637 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:59:26.635+0000 7fefa2600700 1 -- 172.21.15.153:0/1274034154 shutdown_connections 2024-03-21T09:59:26.637 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:59:26.636+0000 7fefa2600700 1 -- 172.21.15.153:0/1274034154 wait complete. 2024-03-21T09:59:26.639 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:59:26.637+0000 7fefa2600700 1 Processor -- start 2024-03-21T09:59:26.639 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:59:26.637+0000 7fefa2600700 1 -- start start 2024-03-21T09:59:26.639 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:59:26.637+0000 7fefa2600700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fef9c0737f0 0x7fef9c19c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:59:26.639 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:59:26.637+0000 7fefa2600700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fef9c074dc0 0x7fef9c19cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:59:26.639 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:59:26.637+0000 7fefa2600700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef9c19d340 con 0x7fef9c0737f0 2024-03-21T09:59:26.639 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:59:26.637+0000 7fefa2600700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef9c19d480 con 0x7fef9c074dc0 2024-03-21T09:59:26.639 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:59:26.637+0000 7fefa1200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fef9c0737f0 0x7fef9c19c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:59:26.640 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:59:26.638+0000 7fefa1200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fef9c0737f0 0x7fef9c19c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:36636/0 (socket says 172.21.15.153:36636) 2024-03-21T09:59:26.640 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:59:26.638+0000 7fefa1200700 1 -- 172.21.15.153:0/2477832378 learned_addr learned my addr 172.21.15.153:0/2477832378 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T09:59:26.640 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:59:26.638+0000 7fef93e00700 1 --2- 172.21.15.153:0/2477832378 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fef9c074dc0 0x7fef9c19cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:59:26.640 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:59:26.638+0000 7fefa1200700 1 -- 172.21.15.153:0/2477832378 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fef9c074dc0 msgr2=0x7fef9c19cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:59:26.640 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:59:26.638+0000 7fefa1200700 1 --2- 172.21.15.153:0/2477832378 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fef9c074dc0 0x7fef9c19cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:59:26.640 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:59:26.638+0000 7fefa1200700 1 -- 172.21.15.153:0/2477832378 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef840095f0 con 0x7fef9c0737f0 2024-03-21T09:59:26.640 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:59:26.638+0000 7fefa1200700 1 --2- 172.21.15.153:0/2477832378 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fef9c0737f0 0x7fef9c19c7e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fef8c00d900 tx=0x7fef8c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:59:26.640 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:59:26.638+0000 7fef9a000700 1 -- 172.21.15.153:0/2477832378 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fef8c009900 con 0x7fef9c0737f0 2024-03-21T09:59:26.640 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:59:26.639+0000 7fef9a000700 1 -- 172.21.15.153:0/2477832378 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fef8c010460 con 0x7fef9c0737f0 2024-03-21T09:59:26.640 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:59:26.639+0000 7fefa2600700 1 -- 172.21.15.153:0/2477832378 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef9c1a1f30 con 0x7fef9c0737f0 2024-03-21T09:59:26.641 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:59:26.639+0000 7fef9a000700 1 -- 172.21.15.153:0/2477832378 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fef8c00f5f0 con 0x7fef9c0737f0 2024-03-21T09:59:26.641 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:59:26.639+0000 7fefa2600700 1 -- 172.21.15.153:0/2477832378 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef9c1a23f0 con 0x7fef9c0737f0 2024-03-21T09:59:26.641 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:59:26.640+0000 7fefa2600700 1 -- 172.21.15.153:0/2477832378 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef9c10fb90 con 0x7fef9c0737f0 2024-03-21T09:59:26.642 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:59:26.641+0000 7fef9a000700 1 -- 172.21.15.153:0/2477832378 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 18) v1 ==== 90258+0+0 (secure 0 0 0) 0x7fef8c0105d0 con 0x7fef9c0737f0 2024-03-21T09:59:26.643 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:59:26.642+0000 7fef9a000700 1 --2- 172.21.15.153:0/2477832378 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fef7c06c470 0x7fef7c06e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:59:26.643 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:59:26.642+0000 7fef9a000700 1 -- 172.21.15.153:0/2477832378 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7fef8c08a330 con 0x7fef9c0737f0 2024-03-21T09:59:26.643 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:59:26.643+0000 7fef93e00700 1 --2- 172.21.15.153:0/2477832378 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fef7c06c470 0x7fef7c06e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:59:26.643 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:59:26.643+0000 7fef93e00700 1 --2- 172.21.15.153:0/2477832378 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fef7c06c470 0x7fef7c06e930 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fef84006010 tx=0x7fef8400b560 comp rx=0 tx=0).ready entity=mgr.14221 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:59:26.648 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:59:26.648+0000 7fef9a000700 1 -- 172.21.15.153:0/2477832378 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fef8c0595e0 con 0x7fef9c0737f0 2024-03-21T09:59:26.874 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T09:59:26.873+0000 7fefa2600700 1 -- 172.21.15.153:0/2477832378 --> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fef9c1039a0 con 0x7fef7c06c470 2024-03-21T09:59:27.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:27 smithi174 ceph-mon[37496]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:59:27.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:27 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T09:59:27.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:27 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T09:59:27.703 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:27 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:59:27.785 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:27 smithi153 ceph-mon[29117]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:59:27.785 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:27 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T09:59:27.785 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:27 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T09:59:27.786 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:27 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:59:28.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:28 smithi153 ceph-mon[29117]: from='client.14260 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:59:28.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:28 smithi174 ceph-mon[37496]: from='client.14260 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T09:59:29.779 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:29 smithi153 ceph-mon[29117]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:59:29.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:29 smithi174 ceph-mon[37496]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:59:31.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:31 smithi153 ceph-mon[29117]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:59:31.860 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:31 smithi174 ceph-mon[37496]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:59:33.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:32 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:59:33.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:32 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:59:33.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:32 smithi153 ceph-mon[29117]: Reconfiguring grafana.smithi153 (dependencies changed)... 2024-03-21T09:59:33.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:32 smithi153 ceph-mon[29117]: Reconfiguring daemon grafana.smithi153 on smithi153 2024-03-21T09:59:33.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:32 smithi153 ceph-mon[29117]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:59:33.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:32 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:59:33.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:32 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:59:33.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:32 smithi174 ceph-mon[37496]: Reconfiguring grafana.smithi153 (dependencies changed)... 2024-03-21T09:59:33.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:32 smithi174 ceph-mon[37496]: Reconfiguring daemon grafana.smithi153 on smithi153 2024-03-21T09:59:33.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:32 smithi174 ceph-mon[37496]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:59:34.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:33 smithi153 ceph-mon[29117]: from='client.? 172.21.15.153:0/1064050466' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0f2e1224-6f45-4f23-b97a-5b89319a90f1"}]: dispatch 2024-03-21T09:59:34.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:33 smithi153 ceph-mon[29117]: from='client.? 172.21.15.153:0/1064050466' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0f2e1224-6f45-4f23-b97a-5b89319a90f1"}]': finished 2024-03-21T09:59:34.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:33 smithi153 ceph-mon[29117]: osdmap e6: 1 total, 0 up, 1 in 2024-03-21T09:59:34.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:33 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T09:59:34.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:33 smithi174 ceph-mon[37496]: from='client.? 172.21.15.153:0/1064050466' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0f2e1224-6f45-4f23-b97a-5b89319a90f1"}]: dispatch 2024-03-21T09:59:34.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:33 smithi174 ceph-mon[37496]: from='client.? 172.21.15.153:0/1064050466' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0f2e1224-6f45-4f23-b97a-5b89319a90f1"}]': finished 2024-03-21T09:59:34.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:33 smithi174 ceph-mon[37496]: osdmap e6: 1 total, 0 up, 1 in 2024-03-21T09:59:34.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:33 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T09:59:35.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:34 smithi174 ceph-mon[37496]: from='client.? 172.21.15.153:0/393733752' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T09:59:35.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:34 smithi174 ceph-mon[37496]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:59:35.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:34 smithi153 ceph-mon[29117]: from='client.? 172.21.15.153:0/393733752' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T09:59:35.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:34 smithi153 ceph-mon[29117]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:59:37.648 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:37 smithi153 ceph-mon[29117]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:59:37.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:37 smithi174 ceph-mon[37496]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:59:39.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:39 smithi153 ceph-mon[29117]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:59:39.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:39 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T09:59:39.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:39 smithi174 ceph-mon[37496]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:59:39.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:39 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T09:59:41.638 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:41 smithi153 ceph-mon[29117]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:59:41.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:41 smithi174 ceph-mon[37496]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:59:43.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:43 smithi153 ceph-mon[29117]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:59:43.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:43 smithi174 ceph-mon[37496]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:59:44.962 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:44 smithi153 ceph-mon[29117]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:59:44.963 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:44 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:59:44.963 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:44 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:59:45.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:44 smithi174 ceph-mon[37496]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:59:45.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:44 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:59:45.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:44 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:59:46.142 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:45 smithi153 ceph-mon[29117]: Reconfiguring prometheus.smithi153 (dependencies changed)... 2024-03-21T09:59:46.142 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:45 smithi153 ceph-mon[29117]: Reconfiguring daemon prometheus.smithi153 on smithi153 2024-03-21T09:59:46.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:45 smithi174 ceph-mon[37496]: Reconfiguring prometheus.smithi153 (dependencies changed)... 2024-03-21T09:59:46.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:45 smithi174 ceph-mon[37496]: Reconfiguring daemon prometheus.smithi153 on smithi153 2024-03-21T09:59:47.017 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:46 smithi153 ceph-mon[29117]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:59:47.017 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:46 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-21T09:59:47.017 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:46 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:59:47.017 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:46 smithi153 ceph-mon[29117]: Deploying daemon osd.0 on smithi153 2024-03-21T09:59:47.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:46 smithi174 ceph-mon[37496]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:59:47.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:46 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-21T09:59:47.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:46 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:59:47.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:46 smithi174 ceph-mon[37496]: Deploying daemon osd.0 on smithi153 2024-03-21T09:59:49.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:49 smithi153 ceph-mon[29117]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:59:49.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:49 smithi174 ceph-mon[37496]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:59:51.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:51 smithi174 ceph-mon[37496]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:59:51.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:51 smithi153 ceph-mon[29117]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:59:53.388 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:53 smithi174 ceph-mon[37496]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:59:53.388 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:53 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:59:53.388 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:53 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:59:53.388 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:53 smithi174 ceph-mon[37496]: Reconfiguring ceph-exporter.smithi174 (monmap changed)... 2024-03-21T09:59:53.388 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:53 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi174", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-21T09:59:53.388 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:53 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:59:53.389 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:53 smithi174 ceph-mon[37496]: Reconfiguring daemon ceph-exporter.smithi174 on smithi174 2024-03-21T09:59:53.599 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:53 smithi153 ceph-mon[29117]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:59:53.599 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:53 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:59:53.599 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:53 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:59:53.599 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:53 smithi153 ceph-mon[29117]: Reconfiguring ceph-exporter.smithi174 (monmap changed)... 2024-03-21T09:59:53.599 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:53 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi174", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-21T09:59:53.600 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:53 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:59:53.600 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:53 smithi153 ceph-mon[29117]: Reconfiguring daemon ceph-exporter.smithi174 on smithi174 2024-03-21T09:59:55.356 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:55 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:59:55.356 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:55 smithi174 ceph-mon[37496]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:59:55.357 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:55 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:59:55.357 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:55 smithi174 ceph-mon[37496]: Reconfiguring crash.smithi174 (monmap changed)... 2024-03-21T09:59:55.357 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:55 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi174", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-21T09:59:55.357 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:55 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:59:55.357 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:55 smithi174 ceph-mon[37496]: Reconfiguring daemon crash.smithi174 on smithi174 2024-03-21T09:59:55.357 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:55 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T09:59:55.373 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:55 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:59:55.373 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:55 smithi153 ceph-mon[29117]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:59:55.373 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:55 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:59:55.373 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:55 smithi153 ceph-mon[29117]: Reconfiguring crash.smithi174 (monmap changed)... 2024-03-21T09:59:55.373 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:55 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi174", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-21T09:59:55.374 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:55 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:59:55.374 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:55 smithi153 ceph-mon[29117]: Reconfiguring daemon crash.smithi174 on smithi174 2024-03-21T09:59:55.374 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:55 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T09:59:57.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:57 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:59:57.453 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:57 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:59:57.453 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:57 smithi174 ceph-mon[37496]: Reconfiguring mgr.smithi174.qpcwat (monmap changed)... 2024-03-21T09:59:57.453 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:57 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi174.qpcwat", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-21T09:59:57.453 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:57 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-21T09:59:57.453 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:57 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:59:57.453 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:57 smithi174 ceph-mon[37496]: Reconfiguring daemon mgr.smithi174.qpcwat on smithi174 2024-03-21T09:59:57.453 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:57 smithi174 ceph-mon[37496]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:59:57.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:57 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:59:57.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:57 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:59:57.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:57 smithi153 ceph-mon[29117]: Reconfiguring mgr.smithi174.qpcwat (monmap changed)... 2024-03-21T09:59:57.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:57 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi174.qpcwat", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-21T09:59:57.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:57 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-21T09:59:57.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:57 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:59:57.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:57 smithi153 ceph-mon[29117]: Reconfiguring daemon mgr.smithi174.qpcwat on smithi174 2024-03-21T09:59:57.558 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:57 smithi153 ceph-mon[29117]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:59:59.166 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:59 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:59:59.166 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:59 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:59:59.166 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:59 smithi174 ceph-mon[37496]: Reconfiguring mon.smithi174 (monmap changed)... 2024-03-21T09:59:59.166 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:59 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T09:59:59.167 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:59 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-21T09:59:59.167 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:59 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:59:59.167 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:59 smithi174 ceph-mon[37496]: Reconfiguring daemon mon.smithi174 on smithi174 2024-03-21T09:59:59.167 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 09:59:59 smithi174 ceph-mon[37496]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T09:59:59.220 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:59 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:59:59.220 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:59 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T09:59:59.221 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:59 smithi153 ceph-mon[29117]: Reconfiguring mon.smithi174 (monmap changed)... 2024-03-21T09:59:59.221 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:59 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T09:59:59.221 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:59 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-21T09:59:59.221 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:59 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:59:59.221 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:59 smithi153 ceph-mon[29117]: Reconfiguring daemon mon.smithi174 on smithi174 2024-03-21T09:59:59.221 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 09:59:59 smithi153 ceph-mon[29117]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:00:00.608 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:00 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:00:00.608 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:00 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:00:00.608 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:00 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:00:00.608 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:00 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:00:00.608 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:00 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-21T10:00:00.608 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:00 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi153.front.sepia.ceph.com:9093"}]: dispatch 2024-03-21T10:00:00.608 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:00 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:00:00.608 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:00 smithi153 ceph-mon[29117]: overall HEALTH_OK 2024-03-21T10:00:00.608 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:00 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-21T10:00:00.609 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:00 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi153.front.sepia.ceph.com:3000"}]: dispatch 2024-03-21T10:00:00.609 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:00 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:00:00.609 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:00 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-21T10:00:00.609 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:00 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi153.front.sepia.ceph.com:9095"}]: dispatch 2024-03-21T10:00:00.609 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:00 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:00:00.609 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:00 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:00:00.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:00 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:00:00.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:00 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:00:00.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:00 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:00:00.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:00 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:00:00.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:00 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-21T10:00:00.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:00 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi153.front.sepia.ceph.com:9093"}]: dispatch 2024-03-21T10:00:00.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:00 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:00:00.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:00 smithi174 ceph-mon[37496]: overall HEALTH_OK 2024-03-21T10:00:00.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:00 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-21T10:00:00.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:00 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi153.front.sepia.ceph.com:3000"}]: dispatch 2024-03-21T10:00:00.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:00 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:00:00.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:00 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-21T10:00:00.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:00 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi153.front.sepia.ceph.com:9095"}]: dispatch 2024-03-21T10:00:00.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:00 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:00:00.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:00 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:00:01.679 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:01 smithi153 ceph-mon[29117]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-21T10:00:01.680 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:01 smithi153 ceph-mon[29117]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi153.front.sepia.ceph.com:9093"}]: dispatch 2024-03-21T10:00:01.680 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:01 smithi153 ceph-mon[29117]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-21T10:00:01.680 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:01 smithi153 ceph-mon[29117]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi153.front.sepia.ceph.com:3000"}]: dispatch 2024-03-21T10:00:01.680 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:01 smithi153 ceph-mon[29117]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-21T10:00:01.680 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:01 smithi153 ceph-mon[29117]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi153.front.sepia.ceph.com:9095"}]: dispatch 2024-03-21T10:00:01.680 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:01 smithi153 ceph-mon[29117]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:00:01.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:01 smithi174 ceph-mon[37496]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-21T10:00:01.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:01 smithi174 ceph-mon[37496]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi153.front.sepia.ceph.com:9093"}]: dispatch 2024-03-21T10:00:01.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:01 smithi174 ceph-mon[37496]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-21T10:00:01.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:01 smithi174 ceph-mon[37496]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi153.front.sepia.ceph.com:3000"}]: dispatch 2024-03-21T10:00:01.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:01 smithi174 ceph-mon[37496]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-21T10:00:01.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:01 smithi174 ceph-mon[37496]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi153.front.sepia.ceph.com:9095"}]: dispatch 2024-03-21T10:00:01.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:01 smithi174 ceph-mon[37496]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:00:03.658 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:03 smithi153 ceph-mon[29117]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:00:03.658 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:03 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:00:03.659 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:03 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:00:03.659 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:03 smithi153 ceph-mon[29117]: from='osd.0 [v2:172.21.15.153:6802/1050381948,v1:172.21.15.153:6803/1050381948]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-21T10:00:03.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:03 smithi174 ceph-mon[37496]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:00:03.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:03 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:00:03.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:03 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:00:03.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:03 smithi174 ceph-mon[37496]: from='osd.0 [v2:172.21.15.153:6802/1050381948,v1:172.21.15.153:6803/1050381948]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-21T10:00:03.804 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:00:03.803+0000 7fef9a000700 1 -- 172.21.15.153:0/2477832378 <== mgr.14221 v2:172.21.15.153:6800/1639929315 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fef9c1039a0 con 0x7fef7c06c470 2024-03-21T10:00:03.805 INFO:teuthology.orchestra.run.smithi153.stdout:Created osd(s) 0 on host 'smithi153' 2024-03-21T10:00:03.808 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:00:03.806+0000 7fefa2600700 1 -- 172.21.15.153:0/2477832378 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fef7c06c470 msgr2=0x7fef7c06e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:00:03.808 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:00:03.806+0000 7fefa2600700 1 --2- 172.21.15.153:0/2477832378 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fef7c06c470 0x7fef7c06e930 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fef84006010 tx=0x7fef8400b560 comp rx=0 tx=0).stop 2024-03-21T10:00:03.808 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:00:03.806+0000 7fefa2600700 1 -- 172.21.15.153:0/2477832378 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fef9c0737f0 msgr2=0x7fef9c19c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:00:03.808 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:00:03.806+0000 7fefa2600700 1 --2- 172.21.15.153:0/2477832378 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fef9c0737f0 0x7fef9c19c7e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fef8c00d900 tx=0x7fef8c00dc10 comp rx=0 tx=0).stop 2024-03-21T10:00:03.808 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:00:03.806+0000 7fefa2600700 1 -- 172.21.15.153:0/2477832378 shutdown_connections 2024-03-21T10:00:03.808 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:00:03.806+0000 7fefa2600700 1 --2- 172.21.15.153:0/2477832378 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fef7c06c470 0x7fef7c06e930 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:00:03.808 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:00:03.806+0000 7fefa2600700 1 --2- 172.21.15.153:0/2477832378 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fef9c0737f0 0x7fef9c19c7e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:00:03.808 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:00:03.806+0000 7fefa2600700 1 --2- 172.21.15.153:0/2477832378 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fef9c074dc0 0x7fef9c19cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:00:03.808 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:00:03.806+0000 7fefa2600700 1 -- 172.21.15.153:0/2477832378 >> 172.21.15.153:0/2477832378 conn(0x7fef9c0fbf60 msgr2=0x7fef9c102280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:00:03.808 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:00:03.806+0000 7fefa2600700 1 -- 172.21.15.153:0/2477832378 shutdown_connections 2024-03-21T10:00:03.808 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:00:03.806+0000 7fefa2600700 1 -- 172.21.15.153:0/2477832378 wait complete. 2024-03-21T10:00:04.517 DEBUG:teuthology.orchestra.run.smithi153:osd.0> sudo journalctl -f -n 0 -u ceph-070ed3e4-e769-11ee-95cd-87774f69a715@osd.0.service 2024-03-21T10:00:04.524 INFO:tasks.cephadm:Deploying osd.1 on smithi153 with /dev/vg_nvme/lv_3... 2024-03-21T10:00:04.525 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 070ed3e4-e769-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-21T10:00:04.973 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:04 smithi153 ceph-mon[29117]: from='osd.0 [v2:172.21.15.153:6802/1050381948,v1:172.21.15.153:6803/1050381948]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-21T10:00:04.974 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:04 smithi153 ceph-mon[29117]: osdmap e7: 1 total, 0 up, 1 in 2024-03-21T10:00:04.974 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:04 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T10:00:04.974 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:04 smithi153 ceph-mon[29117]: from='osd.0 [v2:172.21.15.153:6802/1050381948,v1:172.21.15.153:6803/1050381948]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-03-21T10:00:04.974 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:04 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:00:04.974 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:04 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:00:04.974 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:04 smithi153 ceph-mon[29117]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:00:04.974 INFO:journalctl@ceph.osd.0.smithi153.stdout:Mar 21 10:00:04 smithi153 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-0[48092]: 2024-03-21T10:00:04.712+0000 7f061ac00700 -1 osd.0 0 waiting for initial osdmap 2024-03-21T10:00:04.974 INFO:journalctl@ceph.osd.0.smithi153.stdout:Mar 21 10:00:04 smithi153 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-0[48092]: 2024-03-21T10:00:04.720+0000 7f0616400700 -1 osd.0 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-21T10:00:05.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:04 smithi174 ceph-mon[37496]: from='osd.0 [v2:172.21.15.153:6802/1050381948,v1:172.21.15.153:6803/1050381948]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-21T10:00:05.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:04 smithi174 ceph-mon[37496]: osdmap e7: 1 total, 0 up, 1 in 2024-03-21T10:00:05.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:04 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T10:00:05.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:04 smithi174 ceph-mon[37496]: from='osd.0 [v2:172.21.15.153:6802/1050381948,v1:172.21.15.153:6803/1050381948]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-03-21T10:00:05.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:04 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:00:05.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:04 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:00:05.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:04 smithi174 ceph-mon[37496]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:00:05.333 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi153/config 2024-03-21T10:00:05.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:05 smithi153 ceph-mon[29117]: from='osd.0 [v2:172.21.15.153:6802/1050381948,v1:172.21.15.153:6803/1050381948]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]': finished 2024-03-21T10:00:05.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:05 smithi153 ceph-mon[29117]: osdmap e8: 1 total, 0 up, 1 in 2024-03-21T10:00:05.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:05 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T10:00:05.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:05 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T10:00:06.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:05 smithi174 ceph-mon[37496]: from='osd.0 [v2:172.21.15.153:6802/1050381948,v1:172.21.15.153:6803/1050381948]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]': finished 2024-03-21T10:00:06.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:05 smithi174 ceph-mon[37496]: osdmap e8: 1 total, 0 up, 1 in 2024-03-21T10:00:06.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:05 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T10:00:06.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:05 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T10:00:06.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:06 smithi153 ceph-mon[29117]: purged_snaps scrub starts 2024-03-21T10:00:06.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:06 smithi153 ceph-mon[29117]: purged_snaps scrub ok 2024-03-21T10:00:06.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:06 smithi153 ceph-mon[29117]: osd.0 [v2:172.21.15.153:6802/1050381948,v1:172.21.15.153:6803/1050381948] boot 2024-03-21T10:00:06.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:06 smithi153 ceph-mon[29117]: osdmap e9: 1 total, 1 up, 1 in 2024-03-21T10:00:06.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:06 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T10:00:06.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:06 smithi153 ceph-mon[29117]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:00:06.977 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:06 smithi174 ceph-mon[37496]: purged_snaps scrub starts 2024-03-21T10:00:06.977 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:06 smithi174 ceph-mon[37496]: purged_snaps scrub ok 2024-03-21T10:00:06.977 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:06 smithi174 ceph-mon[37496]: osd.0 [v2:172.21.15.153:6802/1050381948,v1:172.21.15.153:6803/1050381948] boot 2024-03-21T10:00:06.977 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:06 smithi174 ceph-mon[37496]: osdmap e9: 1 total, 1 up, 1 in 2024-03-21T10:00:06.977 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:06 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T10:00:06.977 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:06 smithi174 ceph-mon[37496]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T10:00:07.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:07 smithi153 ceph-mon[29117]: osdmap e10: 1 total, 1 up, 1 in 2024-03-21T10:00:08.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:07 smithi174 ceph-mon[37496]: osdmap e10: 1 total, 1 up, 1 in 2024-03-21T10:00:09.031 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:08 smithi153 ceph-mon[29117]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T10:00:09.031 INFO:teuthology.orchestra.run.smithi153.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-21T10:00:09.031 INFO:teuthology.orchestra.run.smithi153.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-21T10:00:09.031 INFO:teuthology.orchestra.run.smithi153.stderr: stderr: 10+0 records in 2024-03-21T10:00:09.032 INFO:teuthology.orchestra.run.smithi153.stderr:10+0 records out 2024-03-21T10:00:09.032 INFO:teuthology.orchestra.run.smithi153.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0457361 s, 229 MB/s 2024-03-21T10:00:09.032 INFO:teuthology.orchestra.run.smithi153.stderr:--> Zapping successful for: 2024-03-21T10:00:09.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:08 smithi174 ceph-mon[37496]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T10:00:10.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:09 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:00:10.165 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 070ed3e4-e769-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi153:vg_nvme/lv_3 2024-03-21T10:00:10.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:09 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:00:10.556 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi153/config 2024-03-21T10:00:11.015 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:10 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:00:11.015 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:10 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:00:11.015 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:10 smithi153 ceph-mon[29117]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T10:00:11.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:10 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:00:11.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:10 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:00:11.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:10 smithi174 ceph-mon[37496]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T10:00:13.204 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:00:12.733+0000 7ff27f600700 1 -- 172.21.15.153:0/3572313336 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff2780ffd90 msgr2=0x7ff278100210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:00:13.205 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:00:12.733+0000 7ff27f600700 1 --2- 172.21.15.153:0/3572313336 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff2780ffd90 0x7ff278100210 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff264009910 tx=0x7ff264009c20 comp rx=0 tx=0).stop 2024-03-21T10:00:13.205 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:00:12.733+0000 7ff27f600700 1 -- 172.21.15.153:0/3572313336 shutdown_connections 2024-03-21T10:00:13.205 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:00:12.733+0000 7ff27f600700 1 --2- 172.21.15.153:0/3572313336 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff2780ffd90 0x7ff278100210 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:00:13.205 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:00:12.733+0000 7ff27f600700 1 --2- 172.21.15.153:0/3572313336 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff2780ff430 0x7ff2780ff850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:00:13.205 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:00:12.733+0000 7ff27f600700 1 -- 172.21.15.153:0/3572313336 >> 172.21.15.153:0/3572313336 conn(0x7ff2780fb030 msgr2=0x7ff2780fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:00:13.205 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:00:12.733+0000 7ff27f600700 1 -- 172.21.15.153:0/3572313336 shutdown_connections 2024-03-21T10:00:13.205 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:00:12.733+0000 7ff27f600700 1 -- 172.21.15.153:0/3572313336 wait complete. 2024-03-21T10:00:13.205 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:00:12.734+0000 7ff27f600700 1 Processor -- start 2024-03-21T10:00:13.205 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:00:12.735+0000 7ff27f600700 1 -- start start 2024-03-21T10:00:13.206 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:00:12.735+0000 7ff27f600700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff2780ff430 0x7ff278198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:00:13.206 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:00:12.735+0000 7ff27f600700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff2780ffd90 0x7ff278198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:00:13.206 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:00:12.735+0000 7ff27f600700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2781990e0 con 0x7ff2780ffd90 2024-03-21T10:00:13.206 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:00:12.735+0000 7ff27f600700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff278199220 con 0x7ff2780ff430 2024-03-21T10:00:13.206 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:00:12.735+0000 7ff277e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff2780ffd90 0x7ff278198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:00:13.206 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:00:12.735+0000 7ff277e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff2780ffd90 0x7ff278198ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:45648/0 (socket says 172.21.15.153:45648) 2024-03-21T10:00:13.206 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:00:12.735+0000 7ff277e00700 1 -- 172.21.15.153:0/3546396331 learned_addr learned my addr 172.21.15.153:0/3546396331 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:00:13.206 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:00:12.735+0000 7ff27cc00700 1 --2- 172.21.15.153:0/3546396331 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff2780ff430 0x7ff278198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:00:13.206 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:00:12.736+0000 7ff277e00700 1 -- 172.21.15.153:0/3546396331 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff2780ff430 msgr2=0x7ff278198580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:00:13.206 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:00:12.736+0000 7ff277e00700 1 --2- 172.21.15.153:0/3546396331 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff2780ff430 0x7ff278198580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:00:13.207 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:00:12.736+0000 7ff277e00700 1 -- 172.21.15.153:0/3546396331 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2640095f0 con 0x7ff2780ffd90 2024-03-21T10:00:13.207 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:00:12.736+0000 7ff277e00700 1 --2- 172.21.15.153:0/3546396331 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff2780ffd90 0x7ff278198ac0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff264005b40 tx=0x7ff26400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:00:13.207 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:00:12.736+0000 7ff275600700 1 -- 172.21.15.153:0/3546396331 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff26401d070 con 0x7ff2780ffd90 2024-03-21T10:00:13.207 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:00:12.736+0000 7ff275600700 1 -- 172.21.15.153:0/3546396331 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff264022470 con 0x7ff2780ffd90 2024-03-21T10:00:13.207 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:00:12.736+0000 7ff27f600700 1 -- 172.21.15.153:0/3546396331 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff27819dc70 con 0x7ff2780ffd90 2024-03-21T10:00:13.207 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:00:12.736+0000 7ff275600700 1 -- 172.21.15.153:0/3546396331 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff26400f690 con 0x7ff2780ffd90 2024-03-21T10:00:13.207 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:00:12.736+0000 7ff27f600700 1 -- 172.21.15.153:0/3546396331 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff27819e1e0 con 0x7ff2780ffd90 2024-03-21T10:00:13.207 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:00:12.737+0000 7ff27f600700 1 -- 172.21.15.153:0/3546396331 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff27810b980 con 0x7ff2780ffd90 2024-03-21T10:00:13.207 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:00:12.738+0000 7ff275600700 1 -- 172.21.15.153:0/3546396331 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 18) v1 ==== 90258+0+0 (secure 0 0 0) 0x7ff2640225e0 con 0x7ff2780ffd90 2024-03-21T10:00:13.207 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:00:12.739+0000 7ff275600700 1 --2- 172.21.15.153:0/3546396331 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7ff26006c470 0x7ff26006e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:00:13.207 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:00:12.739+0000 7ff275600700 1 -- 172.21.15.153:0/3546396331 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1982+0+0 (secure 0 0 0) 0x7ff26408c280 con 0x7ff2780ffd90 2024-03-21T10:00:13.207 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:00:12.739+0000 7ff27cc00700 1 --2- 172.21.15.153:0/3546396331 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7ff26006c470 0x7ff26006e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:00:13.207 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:00:12.740+0000 7ff27cc00700 1 --2- 172.21.15.153:0/3546396331 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7ff26006c470 0x7ff26006e930 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ff26c006fd0 tx=0x7ff26c008040 comp rx=0 tx=0).ready entity=mgr.14221 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:00:13.207 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:00:12.744+0000 7ff275600700 1 -- 172.21.15.153:0/3546396331 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7ff26405b470 con 0x7ff2780ffd90 2024-03-21T10:00:13.208 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:00:12.971+0000 7ff27f600700 1 -- 172.21.15.153:0/3546396331 --> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7ff278109180 con 0x7ff26006c470 2024-03-21T10:00:13.555 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:13 smithi153 ceph-mon[29117]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T10:00:13.555 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:13 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T10:00:13.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:13 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T10:00:13.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:13 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:00:13.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:13 smithi174 ceph-mon[37496]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T10:00:13.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:13 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T10:00:13.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:13 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T10:00:13.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:13 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:00:14.638 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:14 smithi153 ceph-mon[29117]: from='client.14278 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:00:14.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:14 smithi174 ceph-mon[37496]: from='client.14278 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:00:15.624 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:15 smithi153 ceph-mon[29117]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T10:00:15.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:15 smithi174 ceph-mon[37496]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T10:00:17.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:17 smithi153 ceph-mon[29117]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T10:00:17.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:17 smithi153 ceph-mon[29117]: Detected new or changed devices on smithi153 2024-03-21T10:00:17.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:17 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:00:17.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:17 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:00:17.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:17 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:00:17.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:17 smithi153 ceph-mon[29117]: Adjusting osd_memory_target on smithi153 to 11843M 2024-03-21T10:00:17.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:17 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:00:17.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:17 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:00:17.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:17 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:00:17.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:17 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:00:17.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:17 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:00:17.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:17 smithi174 ceph-mon[37496]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T10:00:17.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:17 smithi174 ceph-mon[37496]: Detected new or changed devices on smithi153 2024-03-21T10:00:17.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:17 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:00:17.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:17 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:00:17.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:17 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:00:17.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:17 smithi174 ceph-mon[37496]: Adjusting osd_memory_target on smithi153 to 11843M 2024-03-21T10:00:17.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:17 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:00:17.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:17 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:00:17.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:17 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:00:17.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:17 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:00:17.953 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:17 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:00:18.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:18 smithi153 ceph-mon[29117]: from='client.? 172.21.15.153:0/1444845793' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "76195edb-aa7a-4627-a211-09a578de65c5"}]: dispatch 2024-03-21T10:00:18.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:18 smithi153 ceph-mon[29117]: from='client.? 172.21.15.153:0/1444845793' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "76195edb-aa7a-4627-a211-09a578de65c5"}]': finished 2024-03-21T10:00:18.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:18 smithi153 ceph-mon[29117]: osdmap e11: 2 total, 1 up, 2 in 2024-03-21T10:00:18.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:18 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T10:00:18.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:18 smithi174 ceph-mon[37496]: from='client.? 172.21.15.153:0/1444845793' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "76195edb-aa7a-4627-a211-09a578de65c5"}]: dispatch 2024-03-21T10:00:18.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:18 smithi174 ceph-mon[37496]: from='client.? 172.21.15.153:0/1444845793' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "76195edb-aa7a-4627-a211-09a578de65c5"}]': finished 2024-03-21T10:00:18.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:18 smithi174 ceph-mon[37496]: osdmap e11: 2 total, 1 up, 2 in 2024-03-21T10:00:18.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:18 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T10:00:19.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:19 smithi153 ceph-mon[29117]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T10:00:19.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:19 smithi153 ceph-mon[29117]: from='client.? 172.21.15.153:0/2677671072' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T10:00:19.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:19 smithi174 ceph-mon[37496]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T10:00:19.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:19 smithi174 ceph-mon[37496]: from='client.? 172.21.15.153:0/2677671072' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T10:00:21.305 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:20 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:00:21.305 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:20 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:00:21.305 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:20 smithi153 ceph-mon[29117]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T10:00:21.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:20 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:00:21.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:20 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:00:21.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:20 smithi174 ceph-mon[37496]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T10:00:23.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:23 smithi174 ceph-mon[37496]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T10:00:23.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:23 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:00:23.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:23 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:00:23.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:23 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:00:23.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:23 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:00:23.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:23 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:00:23.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:23 smithi153 ceph-mon[29117]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T10:00:23.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:23 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:00:23.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:23 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:00:23.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:23 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:00:23.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:23 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:00:23.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:23 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:00:25.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:24 smithi174 ceph-mon[37496]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T10:00:25.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:24 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:00:25.251 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:24 smithi153 ceph-mon[29117]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T10:00:25.251 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:24 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:00:27.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:27 smithi153 ceph-mon[29117]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T10:00:27.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:27 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-21T10:00:27.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:27 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:00:27.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:27 smithi174 ceph-mon[37496]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T10:00:27.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:27 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-21T10:00:27.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:27 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:00:28.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:28 smithi153 ceph-mon[29117]: Deploying daemon osd.1 on smithi153 2024-03-21T10:00:28.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:28 smithi174 ceph-mon[37496]: Deploying daemon osd.1 on smithi153 2024-03-21T10:00:29.619 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:29 smithi153 ceph-mon[29117]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T10:00:29.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:29 smithi174 ceph-mon[37496]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T10:00:31.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:31 smithi153 ceph-mon[29117]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T10:00:31.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:31 smithi174 ceph-mon[37496]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T10:00:33.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:33 smithi174 ceph-mon[37496]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T10:00:33.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:33 smithi153 ceph-mon[29117]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T10:00:35.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:34 smithi174 ceph-mon[37496]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T10:00:35.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:34 smithi153 ceph-mon[29117]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T10:00:36.156 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:36 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:00:36.156 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:36 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:00:36.156 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:36 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:00:36.386 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:36 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:00:36.387 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:36 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:00:36.387 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:36 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:00:37.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:37 smithi153 ceph-mon[29117]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T10:00:37.354 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:37 smithi174 ceph-mon[37496]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T10:00:39.154 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:38 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:00:39.154 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:38 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:00:39.154 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:38 smithi153 ceph-mon[29117]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T10:00:39.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:38 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:00:39.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:38 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:00:39.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:38 smithi174 ceph-mon[37496]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T10:00:40.131 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:00:40.130+0000 7ff275600700 1 -- 172.21.15.153:0/3546396331 <== mgr.14221 v2:172.21.15.153:6800/1639929315 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7ff278109180 con 0x7ff26006c470 2024-03-21T10:00:40.132 INFO:teuthology.orchestra.run.smithi153.stdout:Created osd(s) 1 on host 'smithi153' 2024-03-21T10:00:40.132 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:39 smithi153 ceph-mon[29117]: from='osd.1 [v2:172.21.15.153:6810/3504985760,v1:172.21.15.153:6811/3504985760]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-21T10:00:40.132 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:39 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:00:40.135 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:00:40.133+0000 7ff27f600700 1 -- 172.21.15.153:0/3546396331 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7ff26006c470 msgr2=0x7ff26006e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:00:40.136 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:00:40.133+0000 7ff27f600700 1 --2- 172.21.15.153:0/3546396331 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7ff26006c470 0x7ff26006e930 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ff26c006fd0 tx=0x7ff26c008040 comp rx=0 tx=0).stop 2024-03-21T10:00:40.136 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:00:40.133+0000 7ff27f600700 1 -- 172.21.15.153:0/3546396331 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff2780ffd90 msgr2=0x7ff278198ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:00:40.136 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:00:40.133+0000 7ff27f600700 1 --2- 172.21.15.153:0/3546396331 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff2780ffd90 0x7ff278198ac0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff264005b40 tx=0x7ff26400be20 comp rx=0 tx=0).stop 2024-03-21T10:00:40.136 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:00:40.134+0000 7ff27f600700 1 -- 172.21.15.153:0/3546396331 shutdown_connections 2024-03-21T10:00:40.136 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:00:40.134+0000 7ff27f600700 1 --2- 172.21.15.153:0/3546396331 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7ff26006c470 0x7ff26006e930 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:00:40.136 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:00:40.134+0000 7ff27f600700 1 --2- 172.21.15.153:0/3546396331 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff2780ff430 0x7ff278198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:00:40.136 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:00:40.134+0000 7ff27f600700 1 --2- 172.21.15.153:0/3546396331 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff2780ffd90 0x7ff278198ac0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:00:40.136 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:00:40.134+0000 7ff27f600700 1 -- 172.21.15.153:0/3546396331 >> 172.21.15.153:0/3546396331 conn(0x7ff2780fb030 msgr2=0x7ff278107a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:00:40.136 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:00:40.134+0000 7ff27f600700 1 -- 172.21.15.153:0/3546396331 shutdown_connections 2024-03-21T10:00:40.136 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:00:40.134+0000 7ff27f600700 1 -- 172.21.15.153:0/3546396331 wait complete. 2024-03-21T10:00:40.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:39 smithi174 ceph-mon[37496]: from='osd.1 [v2:172.21.15.153:6810/3504985760,v1:172.21.15.153:6811/3504985760]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-21T10:00:40.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:39 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:00:41.001 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:40 smithi153 ceph-mon[29117]: from='osd.1 [v2:172.21.15.153:6810/3504985760,v1:172.21.15.153:6811/3504985760]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-21T10:00:41.001 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:40 smithi153 ceph-mon[29117]: osdmap e12: 2 total, 1 up, 2 in 2024-03-21T10:00:41.001 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:40 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T10:00:41.001 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:40 smithi153 ceph-mon[29117]: from='osd.1 [v2:172.21.15.153:6810/3504985760,v1:172.21.15.153:6811/3504985760]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-03-21T10:00:41.001 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:40 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:00:41.001 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:40 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:00:41.001 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:40 smithi153 ceph-mon[29117]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T10:00:41.052 DEBUG:teuthology.orchestra.run.smithi153:osd.1> sudo journalctl -f -n 0 -u ceph-070ed3e4-e769-11ee-95cd-87774f69a715@osd.1.service 2024-03-21T10:00:41.059 INFO:tasks.cephadm:Deploying osd.2 on smithi153 with /dev/vg_nvme/lv_2... 2024-03-21T10:00:41.059 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 070ed3e4-e769-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-21T10:00:41.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:40 smithi174 ceph-mon[37496]: from='osd.1 [v2:172.21.15.153:6810/3504985760,v1:172.21.15.153:6811/3504985760]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-21T10:00:41.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:40 smithi174 ceph-mon[37496]: osdmap e12: 2 total, 1 up, 2 in 2024-03-21T10:00:41.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:40 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T10:00:41.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:40 smithi174 ceph-mon[37496]: from='osd.1 [v2:172.21.15.153:6810/3504985760,v1:172.21.15.153:6811/3504985760]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-03-21T10:00:41.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:40 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:00:41.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:40 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:00:41.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:40 smithi174 ceph-mon[37496]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T10:00:41.821 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi153/config 2024-03-21T10:00:42.002 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:41 smithi153 ceph-mon[29117]: from='osd.1 [v2:172.21.15.153:6810/3504985760,v1:172.21.15.153:6811/3504985760]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]': finished 2024-03-21T10:00:42.003 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:41 smithi153 ceph-mon[29117]: osdmap e13: 2 total, 1 up, 2 in 2024-03-21T10:00:42.003 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:41 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T10:00:42.003 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:41 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T10:00:42.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:41 smithi174 ceph-mon[37496]: from='osd.1 [v2:172.21.15.153:6810/3504985760,v1:172.21.15.153:6811/3504985760]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]': finished 2024-03-21T10:00:42.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:41 smithi174 ceph-mon[37496]: osdmap e13: 2 total, 1 up, 2 in 2024-03-21T10:00:42.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:41 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T10:00:42.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:41 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T10:00:43.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:42 smithi174 ceph-mon[37496]: purged_snaps scrub starts 2024-03-21T10:00:43.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:42 smithi174 ceph-mon[37496]: purged_snaps scrub ok 2024-03-21T10:00:43.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:42 smithi174 ceph-mon[37496]: osd.1 [v2:172.21.15.153:6810/3504985760,v1:172.21.15.153:6811/3504985760] boot 2024-03-21T10:00:43.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:42 smithi174 ceph-mon[37496]: osdmap e14: 2 total, 2 up, 2 in 2024-03-21T10:00:43.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:42 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T10:00:43.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:42 smithi174 ceph-mon[37496]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T10:00:43.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:42 smithi153 ceph-mon[29117]: purged_snaps scrub starts 2024-03-21T10:00:43.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:42 smithi153 ceph-mon[29117]: purged_snaps scrub ok 2024-03-21T10:00:43.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:42 smithi153 ceph-mon[29117]: osd.1 [v2:172.21.15.153:6810/3504985760,v1:172.21.15.153:6811/3504985760] boot 2024-03-21T10:00:43.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:42 smithi153 ceph-mon[29117]: osdmap e14: 2 total, 2 up, 2 in 2024-03-21T10:00:43.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:42 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T10:00:43.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:42 smithi153 ceph-mon[29117]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T10:00:43.944 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:43 smithi153 ceph-mon[29117]: osdmap e15: 2 total, 2 up, 2 in 2024-03-21T10:00:44.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:43 smithi174 ceph-mon[37496]: osdmap e15: 2 total, 2 up, 2 in 2024-03-21T10:00:45.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:44 smithi153 ceph-mon[29117]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T10:00:45.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:44 smithi174 ceph-mon[37496]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T10:00:45.405 INFO:teuthology.orchestra.run.smithi153.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-21T10:00:45.406 INFO:teuthology.orchestra.run.smithi153.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-21T10:00:45.406 INFO:teuthology.orchestra.run.smithi153.stderr: stderr: 10+0 records in 2024-03-21T10:00:45.406 INFO:teuthology.orchestra.run.smithi153.stderr:10+0 records out 2024-03-21T10:00:45.406 INFO:teuthology.orchestra.run.smithi153.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0320244 s, 327 MB/s 2024-03-21T10:00:45.406 INFO:teuthology.orchestra.run.smithi153.stderr:--> Zapping successful for: 2024-03-21T10:00:46.196 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 070ed3e4-e769-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi153:vg_nvme/lv_2 2024-03-21T10:00:46.421 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:46 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:00:46.421 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:46 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:00:46.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:46 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:00:46.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:46 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:00:47.380 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:47 smithi153 ceph-mon[29117]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T10:00:47.399 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi153/config 2024-03-21T10:00:47.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:47 smithi174 ceph-mon[37496]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T10:00:49.521 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:00:49.519+0000 7f0761200700 1 -- 172.21.15.153:0/2543559517 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f075c100bb0 msgr2=0x7f075c100fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:00:49.521 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:00:49.519+0000 7f0761200700 1 --2- 172.21.15.153:0/2543559517 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f075c100bb0 0x7f075c100fd0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f074c009910 tx=0x7f074c009c20 comp rx=0 tx=0).stop 2024-03-21T10:00:49.521 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:00:49.519+0000 7f0761200700 1 -- 172.21.15.153:0/2543559517 shutdown_connections 2024-03-21T10:00:49.522 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:00:49.519+0000 7f0761200700 1 --2- 172.21.15.153:0/2543559517 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f075c101db0 0x7f075c102210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:00:49.522 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:00:49.519+0000 7f0761200700 1 --2- 172.21.15.153:0/2543559517 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f075c100bb0 0x7f075c100fd0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:00:49.522 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:00:49.519+0000 7f0761200700 1 -- 172.21.15.153:0/2543559517 >> 172.21.15.153:0/2543559517 conn(0x7f075c0fc110 msgr2=0x7f075c0fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:00:49.522 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:00:49.519+0000 7f0761200700 1 -- 172.21.15.153:0/2543559517 shutdown_connections 2024-03-21T10:00:49.522 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:00:49.519+0000 7f0761200700 1 -- 172.21.15.153:0/2543559517 wait complete. 2024-03-21T10:00:49.524 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:00:49.520+0000 7f0761200700 1 Processor -- start 2024-03-21T10:00:49.524 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:00:49.520+0000 7f0761200700 1 -- start start 2024-03-21T10:00:49.524 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:00:49.521+0000 7f0761200700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f075c100bb0 0x7f075c198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:00:49.524 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:00:49.521+0000 7f0761200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f075c101db0 0x7f075c198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:00:49.524 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:00:49.521+0000 7f0761200700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f075c1990e0 con 0x7f075c101db0 2024-03-21T10:00:49.524 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:00:49.521+0000 7f0761200700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f075c199220 con 0x7f075c100bb0 2024-03-21T10:00:49.524 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:00:49.521+0000 7f0759400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f075c101db0 0x7f075c198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:00:49.524 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:00:49.521+0000 7f0759400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f075c101db0 0x7f075c198ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:43090/0 (socket says 172.21.15.153:43090) 2024-03-21T10:00:49.525 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:00:49.521+0000 7f0759400700 1 -- 172.21.15.153:0/1737260316 learned_addr learned my addr 172.21.15.153:0/1737260316 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:00:49.525 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:00:49.521+0000 7f0759e00700 1 --2- 172.21.15.153:0/1737260316 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f075c100bb0 0x7f075c198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:00:49.525 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:00:49.522+0000 7f0759400700 1 -- 172.21.15.153:0/1737260316 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f075c100bb0 msgr2=0x7f075c198580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:00:49.525 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:00:49.522+0000 7f0759400700 1 --2- 172.21.15.153:0/1737260316 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f075c100bb0 0x7f075c198580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:00:49.525 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:00:49.522+0000 7f0759400700 1 -- 172.21.15.153:0/1737260316 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f074c0095f0 con 0x7f075c101db0 2024-03-21T10:00:49.525 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:00:49.522+0000 7f0759e00700 1 --2- 172.21.15.153:0/1737260316 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f075c100bb0 0x7f075c198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:00:49.525 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:00:49.522+0000 7f0759400700 1 --2- 172.21.15.153:0/1737260316 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f075c101db0 0x7f075c198ac0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f074800d8d0 tx=0x7f074800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:00:49.525 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:00:49.522+0000 7f0752a00700 1 -- 172.21.15.153:0/1737260316 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0748009940 con 0x7f075c101db0 2024-03-21T10:00:49.525 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:00:49.522+0000 7f0752a00700 1 -- 172.21.15.153:0/1737260316 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0748010460 con 0x7f075c101db0 2024-03-21T10:00:49.525 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:00:49.522+0000 7f0761200700 1 -- 172.21.15.153:0/1737260316 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f075c19dcd0 con 0x7f075c101db0 2024-03-21T10:00:49.525 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:00:49.522+0000 7f0752a00700 1 -- 172.21.15.153:0/1737260316 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f074800f5f0 con 0x7f075c101db0 2024-03-21T10:00:49.525 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:00:49.522+0000 7f0761200700 1 -- 172.21.15.153:0/1737260316 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f075c19e220 con 0x7f075c101db0 2024-03-21T10:00:49.526 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:00:49.524+0000 7f0761200700 1 -- 172.21.15.153:0/1737260316 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f075c105e90 con 0x7f075c101db0 2024-03-21T10:00:49.526 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:00:49.525+0000 7f0752a00700 1 -- 172.21.15.153:0/1737260316 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 18) v1 ==== 90258+0+0 (secure 0 0 0) 0x7f07480105d0 con 0x7f075c101db0 2024-03-21T10:00:49.527 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:00:49.526+0000 7f0752a00700 1 --2- 172.21.15.153:0/1737260316 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f074006c470 0x7f074006e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:00:49.528 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:00:49.526+0000 7f0752a00700 1 -- 172.21.15.153:0/1737260316 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 2414+0+0 (secure 0 0 0) 0x7f074808aad0 con 0x7f075c101db0 2024-03-21T10:00:49.528 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:00:49.526+0000 7f0759e00700 1 --2- 172.21.15.153:0/1737260316 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f074006c470 0x7f074006e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:00:49.528 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:00:49.527+0000 7f0759e00700 1 --2- 172.21.15.153:0/1737260316 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f074006c470 0x7f074006e930 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f074c006010 tx=0x7f074c00b560 comp rx=0 tx=0).ready entity=mgr.14221 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:00:49.533 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:00:49.532+0000 7f0752a00700 1 -- 172.21.15.153:0/1737260316 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f07480599f0 con 0x7f075c101db0 2024-03-21T10:00:49.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:49 smithi174 ceph-mon[37496]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T10:00:49.768 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:00:49.766+0000 7f0761200700 1 -- 172.21.15.153:0/1737260316 --> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f075c0611d0 con 0x7f074006c470 2024-03-21T10:00:49.768 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:49 smithi153 ceph-mon[29117]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T10:00:50.656 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:50 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T10:00:50.656 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:50 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T10:00:50.656 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:50 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:00:50.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:50 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T10:00:50.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:50 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T10:00:50.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:50 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:00:51.670 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:51 smithi153 ceph-mon[29117]: from='client.14296 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:00:51.670 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:51 smithi153 ceph-mon[29117]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T10:00:51.670 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:51 smithi153 ceph-mon[29117]: Detected new or changed devices on smithi153 2024-03-21T10:00:51.670 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:51 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:00:51.670 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:51 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:00:51.670 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:51 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:00:51.670 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:51 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:00:51.671 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:51 smithi153 ceph-mon[29117]: Adjusting osd_memory_target on smithi153 to 5921M 2024-03-21T10:00:51.671 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:51 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:00:51.671 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:51 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:00:51.671 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:51 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:00:51.671 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:51 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:00:51.671 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:51 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:00:51.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:51 smithi174 ceph-mon[37496]: from='client.14296 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi153:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:00:51.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:51 smithi174 ceph-mon[37496]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T10:00:51.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:51 smithi174 ceph-mon[37496]: Detected new or changed devices on smithi153 2024-03-21T10:00:51.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:51 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:00:51.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:51 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:00:51.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:51 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:00:51.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:51 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:00:51.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:51 smithi174 ceph-mon[37496]: Adjusting osd_memory_target on smithi153 to 5921M 2024-03-21T10:00:51.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:51 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:00:51.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:51 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:00:51.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:51 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:00:51.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:51 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:00:51.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:51 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:00:53.684 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:53 smithi153 ceph-mon[29117]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T10:00:53.684 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:53 smithi153 ceph-mon[29117]: from='client.? 172.21.15.153:0/3055356011' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3ec17cf8-f0ea-45c4-b084-b304024d6e92"}]: dispatch 2024-03-21T10:00:53.684 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:53 smithi153 ceph-mon[29117]: from='client.? 172.21.15.153:0/3055356011' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3ec17cf8-f0ea-45c4-b084-b304024d6e92"}]': finished 2024-03-21T10:00:53.684 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:53 smithi153 ceph-mon[29117]: osdmap e16: 3 total, 2 up, 3 in 2024-03-21T10:00:53.684 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:53 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T10:00:53.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:53 smithi174 ceph-mon[37496]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T10:00:53.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:53 smithi174 ceph-mon[37496]: from='client.? 172.21.15.153:0/3055356011' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3ec17cf8-f0ea-45c4-b084-b304024d6e92"}]: dispatch 2024-03-21T10:00:53.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:53 smithi174 ceph-mon[37496]: from='client.? 172.21.15.153:0/3055356011' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3ec17cf8-f0ea-45c4-b084-b304024d6e92"}]': finished 2024-03-21T10:00:53.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:53 smithi174 ceph-mon[37496]: osdmap e16: 3 total, 2 up, 3 in 2024-03-21T10:00:53.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:53 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T10:00:54.804 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:54 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:00:54.804 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:54 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:00:54.804 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:54 smithi153 ceph-mon[29117]: from='client.? 172.21.15.153:0/3210854310' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T10:00:54.804 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:54 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:00:54.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:54 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:00:54.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:54 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:00:54.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:54 smithi174 ceph-mon[37496]: from='client.? 172.21.15.153:0/3210854310' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T10:00:54.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:54 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:00:55.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:55 smithi153 ceph-mon[29117]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T10:00:55.853 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:55 smithi174 ceph-mon[37496]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T10:00:57.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:57 smithi153 ceph-mon[29117]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T10:00:57.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:57 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:00:57.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:57 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:00:57.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:57 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:00:57.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:57 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:00:57.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:57 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:00:57.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:57 smithi174 ceph-mon[37496]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T10:00:57.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:57 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:00:57.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:57 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:00:57.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:57 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:00:57.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:57 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:00:57.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:57 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:00:59.808 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:00:59 smithi153 ceph-mon[29117]: pgmap v81: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T10:00:59.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:00:59 smithi174 ceph-mon[37496]: pgmap v81: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T10:01:01.757 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:01 smithi153 ceph-mon[29117]: pgmap v82: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T10:01:01.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:01 smithi174 ceph-mon[37496]: pgmap v82: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T10:01:02.885 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:02 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-21T10:01:02.885 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:02 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:01:02.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:02 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-21T10:01:02.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:02 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:01:03.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:03 smithi153 ceph-mon[29117]: Deploying daemon osd.2 on smithi153 2024-03-21T10:01:03.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:03 smithi153 ceph-mon[29117]: pgmap v83: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T10:01:03.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:03 smithi174 ceph-mon[37496]: Deploying daemon osd.2 on smithi153 2024-03-21T10:01:03.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:03 smithi174 ceph-mon[37496]: pgmap v83: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T10:01:05.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:04 smithi174 ceph-mon[37496]: pgmap v84: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T10:01:05.274 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:04 smithi153 ceph-mon[29117]: pgmap v84: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T10:01:07.392 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:07 smithi153 ceph-mon[29117]: pgmap v85: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T10:01:07.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:07 smithi174 ceph-mon[37496]: pgmap v85: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T10:01:09.472 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:09 smithi153 ceph-mon[29117]: pgmap v86: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T10:01:09.473 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:09 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:01:09.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:09 smithi174 ceph-mon[37496]: pgmap v86: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T10:01:09.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:09 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:01:11.186 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:10 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:01:11.186 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:10 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:01:11.186 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:10 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:01:11.186 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:10 smithi153 ceph-mon[29117]: pgmap v87: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T10:01:11.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:10 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:01:11.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:10 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:01:11.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:10 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:01:11.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:10 smithi174 ceph-mon[37496]: pgmap v87: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T10:01:13.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:13 smithi174 ceph-mon[37496]: pgmap v88: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T10:01:13.786 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:13 smithi153 ceph-mon[29117]: pgmap v88: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T10:01:14.755 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:01:14.753+0000 7f0752a00700 1 -- 172.21.15.153:0/1737260316 <== mgr.14221 v2:172.21.15.153:6800/1639929315 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f075c0611d0 con 0x7f074006c470 2024-03-21T10:01:14.755 INFO:teuthology.orchestra.run.smithi153.stdout:Created osd(s) 2 on host 'smithi153' 2024-03-21T10:01:14.759 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:01:14.758+0000 7f0761200700 1 -- 172.21.15.153:0/1737260316 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f074006c470 msgr2=0x7f074006e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:01:14.759 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:01:14.758+0000 7f0761200700 1 --2- 172.21.15.153:0/1737260316 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f074006c470 0x7f074006e930 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f074c006010 tx=0x7f074c00b560 comp rx=0 tx=0).stop 2024-03-21T10:01:14.759 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:01:14.758+0000 7f0761200700 1 -- 172.21.15.153:0/1737260316 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f075c101db0 msgr2=0x7f075c198ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:01:14.759 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:01:14.758+0000 7f0761200700 1 --2- 172.21.15.153:0/1737260316 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f075c101db0 0x7f075c198ac0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f074800d8d0 tx=0x7f074800dc90 comp rx=0 tx=0).stop 2024-03-21T10:01:14.759 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:01:14.758+0000 7f0761200700 1 -- 172.21.15.153:0/1737260316 shutdown_connections 2024-03-21T10:01:14.760 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:01:14.758+0000 7f0761200700 1 --2- 172.21.15.153:0/1737260316 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f074006c470 0x7f074006e930 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:01:14.760 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:01:14.758+0000 7f0761200700 1 --2- 172.21.15.153:0/1737260316 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f075c100bb0 0x7f075c198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:01:14.760 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:01:14.758+0000 7f0761200700 1 --2- 172.21.15.153:0/1737260316 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f075c101db0 0x7f075c198ac0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:01:14.760 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:01:14.758+0000 7f0761200700 1 -- 172.21.15.153:0/1737260316 >> 172.21.15.153:0/1737260316 conn(0x7f075c0fc110 msgr2=0x7f075c102fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:01:14.760 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:01:14.759+0000 7f0761200700 1 -- 172.21.15.153:0/1737260316 shutdown_connections 2024-03-21T10:01:14.760 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:01:14.759+0000 7f0761200700 1 -- 172.21.15.153:0/1737260316 wait complete. 2024-03-21T10:01:15.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:14 smithi174 ceph-mon[37496]: from='osd.2 [v2:172.21.15.153:6818/2844395219,v1:172.21.15.153:6819/2844395219]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-21T10:01:15.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:14 smithi174 ceph-mon[37496]: pgmap v89: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T10:01:15.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:14 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:01:15.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:14 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:01:15.269 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:14 smithi153 ceph-mon[29117]: from='osd.2 [v2:172.21.15.153:6818/2844395219,v1:172.21.15.153:6819/2844395219]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-21T10:01:15.269 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:14 smithi153 ceph-mon[29117]: pgmap v89: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T10:01:15.269 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:14 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:01:15.269 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:14 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:01:15.808 DEBUG:teuthology.orchestra.run.smithi153:osd.2> sudo journalctl -f -n 0 -u ceph-070ed3e4-e769-11ee-95cd-87774f69a715@osd.2.service 2024-03-21T10:01:15.813 INFO:tasks.cephadm:Deploying osd.3 on smithi174 with /dev/vg_nvme/lv_4... 2024-03-21T10:01:15.813 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 070ed3e4-e769-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-21T10:01:16.035 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:15 smithi153 ceph-mon[29117]: from='osd.2 [v2:172.21.15.153:6818/2844395219,v1:172.21.15.153:6819/2844395219]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-21T10:01:16.035 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:15 smithi153 ceph-mon[29117]: osdmap e17: 3 total, 2 up, 3 in 2024-03-21T10:01:16.035 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:15 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T10:01:16.036 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:15 smithi153 ceph-mon[29117]: from='osd.2 [v2:172.21.15.153:6818/2844395219,v1:172.21.15.153:6819/2844395219]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-03-21T10:01:16.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:15 smithi174 ceph-mon[37496]: from='osd.2 [v2:172.21.15.153:6818/2844395219,v1:172.21.15.153:6819/2844395219]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-21T10:01:16.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:15 smithi174 ceph-mon[37496]: osdmap e17: 3 total, 2 up, 3 in 2024-03-21T10:01:16.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:15 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T10:01:16.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:15 smithi174 ceph-mon[37496]: from='osd.2 [v2:172.21.15.153:6818/2844395219,v1:172.21.15.153:6819/2844395219]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-03-21T10:01:16.221 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi174/config 2024-03-21T10:01:17.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:16 smithi174 ceph-mon[37496]: from='osd.2 [v2:172.21.15.153:6818/2844395219,v1:172.21.15.153:6819/2844395219]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]': finished 2024-03-21T10:01:17.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:16 smithi174 ceph-mon[37496]: osdmap e18: 3 total, 2 up, 3 in 2024-03-21T10:01:17.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:16 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T10:01:17.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:16 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T10:01:17.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:16 smithi174 ceph-mon[37496]: pgmap v92: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T10:01:17.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:16 smithi153 ceph-mon[29117]: from='osd.2 [v2:172.21.15.153:6818/2844395219,v1:172.21.15.153:6819/2844395219]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]': finished 2024-03-21T10:01:17.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:16 smithi153 ceph-mon[29117]: osdmap e18: 3 total, 2 up, 3 in 2024-03-21T10:01:17.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:16 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T10:01:17.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:16 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T10:01:17.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:16 smithi153 ceph-mon[29117]: pgmap v92: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T10:01:18.041 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:17 smithi174 ceph-mon[37496]: purged_snaps scrub starts 2024-03-21T10:01:18.041 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:17 smithi174 ceph-mon[37496]: purged_snaps scrub ok 2024-03-21T10:01:18.041 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:17 smithi174 ceph-mon[37496]: osd.2 [v2:172.21.15.153:6818/2844395219,v1:172.21.15.153:6819/2844395219] boot 2024-03-21T10:01:18.041 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:17 smithi174 ceph-mon[37496]: osdmap e19: 3 total, 3 up, 3 in 2024-03-21T10:01:18.041 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:17 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T10:01:18.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:17 smithi153 ceph-mon[29117]: purged_snaps scrub starts 2024-03-21T10:01:18.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:17 smithi153 ceph-mon[29117]: purged_snaps scrub ok 2024-03-21T10:01:18.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:17 smithi153 ceph-mon[29117]: osd.2 [v2:172.21.15.153:6818/2844395219,v1:172.21.15.153:6819/2844395219] boot 2024-03-21T10:01:18.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:17 smithi153 ceph-mon[29117]: osdmap e19: 3 total, 3 up, 3 in 2024-03-21T10:01:18.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:17 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T10:01:18.728 INFO:teuthology.orchestra.run.smithi174.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-21T10:01:18.728 INFO:teuthology.orchestra.run.smithi174.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-21T10:01:18.728 INFO:teuthology.orchestra.run.smithi174.stderr: stderr: 10+0 records in 2024-03-21T10:01:18.729 INFO:teuthology.orchestra.run.smithi174.stderr:10+0 records out 2024-03-21T10:01:18.729 INFO:teuthology.orchestra.run.smithi174.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0343649 s, 305 MB/s 2024-03-21T10:01:18.729 INFO:teuthology.orchestra.run.smithi174.stderr:--> Zapping successful for: 2024-03-21T10:01:19.081 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:18 smithi174 ceph-mon[37496]: osdmap e20: 3 total, 3 up, 3 in 2024-03-21T10:01:19.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:18 smithi153 ceph-mon[29117]: osdmap e20: 3 total, 3 up, 3 in 2024-03-21T10:01:19.437 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 070ed3e4-e769-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi174:vg_nvme/lv_4 2024-03-21T10:01:19.793 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi174/config 2024-03-21T10:01:20.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:19 smithi174 ceph-mon[37496]: pgmap v95: 0 pgs: ; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2024-03-21T10:01:20.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:19 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-21T10:01:20.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:19 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:01:20.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:19 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:01:20.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:19 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-21T10:01:20.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:19 smithi174 ceph-mon[37496]: osdmap e21: 3 total, 3 up, 3 in 2024-03-21T10:01:20.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:19 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-21T10:01:20.220 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:19 smithi153 ceph-mon[29117]: pgmap v95: 0 pgs: ; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2024-03-21T10:01:20.220 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:19 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-21T10:01:20.220 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:19 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:01:20.220 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:19 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:01:20.220 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:19 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-21T10:01:20.220 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:19 smithi153 ceph-mon[29117]: osdmap e21: 3 total, 3 up, 3 in 2024-03-21T10:01:20.220 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:19 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-21T10:01:21.212 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:21 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-21T10:01:21.212 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:21 smithi153 ceph-mon[29117]: osdmap e22: 3 total, 3 up, 3 in 2024-03-21T10:01:21.212 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:21 smithi153 ceph-mon[29117]: pgmap v98: 1 pgs: 1 unknown; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2024-03-21T10:01:21.279 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:21 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-21T10:01:21.279 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:21 smithi174 ceph-mon[37496]: osdmap e22: 3 total, 3 up, 3 in 2024-03-21T10:01:21.280 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:21 smithi174 ceph-mon[37496]: pgmap v98: 1 pgs: 1 unknown; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2024-03-21T10:01:21.745 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:01:21.742+0000 7f94d6600700 1 -- 172.21.15.174:0/2311433990 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f94d0107200 msgr2=0x7f94d0107620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:01:21.745 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:01:21.742+0000 7f94d6600700 1 --2- 172.21.15.174:0/2311433990 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f94d0107200 0x7f94d0107620 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f94c0009910 tx=0x7f94c0009c20 comp rx=0 tx=0).stop 2024-03-21T10:01:21.745 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:01:21.743+0000 7f94d6600700 1 -- 172.21.15.174:0/2311433990 shutdown_connections 2024-03-21T10:01:21.745 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:01:21.743+0000 7f94d6600700 1 --2- 172.21.15.174:0/2311433990 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f94d0108400 0x7f94d0108860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:01:21.745 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:01:21.743+0000 7f94d6600700 1 --2- 172.21.15.174:0/2311433990 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f94d0107200 0x7f94d0107620 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:01:21.746 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:01:21.743+0000 7f94d6600700 1 -- 172.21.15.174:0/2311433990 >> 172.21.15.174:0/2311433990 conn(0x7f94d00761b0 msgr2=0x7f94d0078610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:01:21.746 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:01:21.743+0000 7f94d6600700 1 -- 172.21.15.174:0/2311433990 shutdown_connections 2024-03-21T10:01:21.746 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:01:21.743+0000 7f94d6600700 1 -- 172.21.15.174:0/2311433990 wait complete. 2024-03-21T10:01:21.747 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:01:21.744+0000 7f94d6600700 1 Processor -- start 2024-03-21T10:01:21.747 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:01:21.744+0000 7f94d6600700 1 -- start start 2024-03-21T10:01:21.747 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:01:21.744+0000 7f94d6600700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f94d0107200 0x7f94d019c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:01:21.747 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:01:21.744+0000 7f94d6600700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f94d0108400 0x7f94d019ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:01:21.747 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:01:21.744+0000 7f94d6600700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94d019d4f0 con 0x7f94d0107200 2024-03-21T10:01:21.747 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:01:21.744+0000 7f94d6600700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94d019d630 con 0x7f94d0108400 2024-03-21T10:01:21.747 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:01:21.745+0000 7f94cea00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f94d0108400 0x7f94d019ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:01:21.747 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:01:21.745+0000 7f94cea00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f94d0108400 0x7f94d019ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:44134/0 (socket says 172.21.15.174:44134) 2024-03-21T10:01:21.748 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:01:21.745+0000 7f94cea00700 1 -- 172.21.15.174:0/2115733537 learned_addr learned my addr 172.21.15.174:0/2115733537 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-03-21T10:01:21.748 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:01:21.745+0000 7f94cf400700 1 --2- 172.21.15.174:0/2115733537 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f94d0107200 0x7f94d019c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:01:21.748 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:01:21.745+0000 7f94cea00700 1 -- 172.21.15.174:0/2115733537 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f94d0107200 msgr2=0x7f94d019c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:01:21.748 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:01:21.745+0000 7f94cea00700 1 --2- 172.21.15.174:0/2115733537 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f94d0107200 0x7f94d019c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:01:21.748 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:01:21.745+0000 7f94cea00700 1 -- 172.21.15.174:0/2115733537 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94c00095f0 con 0x7f94d0108400 2024-03-21T10:01:21.748 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:01:21.745+0000 7f94cea00700 1 --2- 172.21.15.174:0/2115733537 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f94d0108400 0x7f94d019ced0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f94bc00d8d0 tx=0x7f94bc00dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:01:21.749 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:01:21.746+0000 7f94c7e00700 1 -- 172.21.15.174:0/2115733537 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f94bc009940 con 0x7f94d0108400 2024-03-21T10:01:21.749 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:01:21.746+0000 7f94c7e00700 1 -- 172.21.15.174:0/2115733537 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f94bc010460 con 0x7f94d0108400 2024-03-21T10:01:21.749 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:01:21.746+0000 7f94c7e00700 1 -- 172.21.15.174:0/2115733537 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f94bc00f5f0 con 0x7f94d0108400 2024-03-21T10:01:21.749 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:01:21.746+0000 7f94d6600700 1 -- 172.21.15.174:0/2115733537 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94d01a20e0 con 0x7f94d0108400 2024-03-21T10:01:21.749 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:01:21.746+0000 7f94d6600700 1 -- 172.21.15.174:0/2115733537 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94d01a2630 con 0x7f94d0108400 2024-03-21T10:01:21.749 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:01:21.747+0000 7f94d6600700 1 -- 172.21.15.174:0/2115733537 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94d010fd60 con 0x7f94d0108400 2024-03-21T10:01:21.750 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:01:21.748+0000 7f94c7e00700 1 -- 172.21.15.174:0/2115733537 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) v1 ==== 90258+0+0 (secure 0 0 0) 0x7f94bc0105d0 con 0x7f94d0108400 2024-03-21T10:01:21.751 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:01:21.749+0000 7f94c7e00700 1 --2- 172.21.15.174:0/2115733537 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f94b406c460 0x7f94b406e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:01:21.751 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:01:21.749+0000 7f94c7e00700 1 -- 172.21.15.174:0/2115733537 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 3232+0+0 (secure 0 0 0) 0x7f94bc08ade0 con 0x7f94d0108400 2024-03-21T10:01:21.751 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:01:21.750+0000 7f94cf400700 1 --2- 172.21.15.174:0/2115733537 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f94b406c460 0x7f94b406e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:01:21.752 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:01:21.750+0000 7f94cf400700 1 --2- 172.21.15.174:0/2115733537 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f94b406c460 0x7f94b406e920 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f94c0006010 tx=0x7f94c0005a90 comp rx=0 tx=0).ready entity=mgr.14221 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:01:21.756 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:01:21.755+0000 7f94c7e00700 1 -- 172.21.15.174:0/2115733537 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f94bc0599f0 con 0x7f94d0108400 2024-03-21T10:01:21.923 INFO:journalctl@ceph.osd.0.smithi153.stdout:Mar 21 10:01:21 smithi153 sudo[65075]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-21T10:01:21.923 INFO:journalctl@ceph.osd.0.smithi153.stdout:Mar 21 10:01:21 smithi153 sudo[65075]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-21T10:01:21.924 INFO:journalctl@ceph.osd.0.smithi153.stdout:Mar 21 10:01:21 smithi153 sudo[65075]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-21T10:01:21.924 INFO:journalctl@ceph.osd.0.smithi153.stdout:Mar 21 10:01:21 smithi153 sudo[65075]: pam_unix(sudo:session): session closed for user root 2024-03-21T10:01:21.977 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:01:21.975+0000 7f94d6600700 1 -- 172.21.15.174:0/2115733537 --> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f94d010cd50 con 0x7f94b406c460 2024-03-21T10:01:22.228 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:22 smithi153 ceph-mon[29117]: osdmap e23: 3 total, 3 up, 3 in 2024-03-21T10:01:22.228 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:22 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T10:01:22.228 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:22 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T10:01:22.228 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:22 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:01:22.252 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:22 smithi174 ceph-mon[37496]: osdmap e23: 3 total, 3 up, 3 in 2024-03-21T10:01:22.252 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:22 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T10:01:22.252 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:22 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T10:01:22.252 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:22 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:01:22.556 INFO:journalctl@ceph.osd.0.smithi153.stdout:Mar 21 10:01:22 smithi153 sudo[65099]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-21T10:01:22.556 INFO:journalctl@ceph.osd.0.smithi153.stdout:Mar 21 10:01:22 smithi153 sudo[65099]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-21T10:01:22.556 INFO:journalctl@ceph.osd.0.smithi153.stdout:Mar 21 10:01:22 smithi153 sudo[65099]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-21T10:01:22.556 INFO:journalctl@ceph.osd.0.smithi153.stdout:Mar 21 10:01:22 smithi153 sudo[65099]: pam_unix(sudo:session): session closed for user root 2024-03-21T10:01:23.299 INFO:journalctl@ceph.osd.1.smithi153.stdout:Mar 21 10:01:22 smithi153 sudo[65111]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-21T10:01:23.299 INFO:journalctl@ceph.osd.1.smithi153.stdout:Mar 21 10:01:22 smithi153 sudo[65111]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-21T10:01:23.299 INFO:journalctl@ceph.osd.1.smithi153.stdout:Mar 21 10:01:22 smithi153 sudo[65111]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-21T10:01:23.299 INFO:journalctl@ceph.osd.1.smithi153.stdout:Mar 21 10:01:22 smithi153 sudo[65111]: pam_unix(sudo:session): session closed for user root 2024-03-21T10:01:23.308 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:01:23.306+0000 7f94c7e00700 1 -- 172.21.15.174:0/2115733537 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f94bc056330 con 0x7f94d0108400 2024-03-21T10:01:23.556 INFO:journalctl@ceph.osd.1.smithi153.stdout:Mar 21 10:01:23 smithi153 sudo[65324]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-21T10:01:23.556 INFO:journalctl@ceph.osd.1.smithi153.stdout:Mar 21 10:01:23 smithi153 sudo[65324]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-21T10:01:23.556 INFO:journalctl@ceph.osd.1.smithi153.stdout:Mar 21 10:01:23 smithi153 sudo[65324]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-21T10:01:23.556 INFO:journalctl@ceph.osd.1.smithi153.stdout:Mar 21 10:01:23 smithi153 sudo[65324]: pam_unix(sudo:session): session closed for user root 2024-03-21T10:01:23.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:23 smithi153 ceph-mon[29117]: from='client.24135 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:01:23.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:23 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:01:23.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:23 smithi153 ceph-mon[29117]: pgmap v100: 1 pgs: 1 active+clean; 449 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-21T10:01:23.665 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:23 smithi174 ceph-mon[37496]: from='client.24135 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:01:23.665 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:23 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:01:23.665 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:23 smithi174 ceph-mon[37496]: pgmap v100: 1 pgs: 1 active+clean; 449 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-21T10:01:24.227 INFO:journalctl@ceph.osd.2.smithi153.stdout:Mar 21 10:01:24 smithi153 sudo[67853]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-21T10:01:24.227 INFO:journalctl@ceph.osd.2.smithi153.stdout:Mar 21 10:01:24 smithi153 sudo[67853]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-21T10:01:24.227 INFO:journalctl@ceph.osd.2.smithi153.stdout:Mar 21 10:01:24 smithi153 sudo[67853]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-21T10:01:24.227 INFO:journalctl@ceph.osd.2.smithi153.stdout:Mar 21 10:01:24 smithi153 sudo[67853]: pam_unix(sudo:session): session closed for user root 2024-03-21T10:01:24.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:24 smithi153 ceph-mon[29117]: mgrmap e19: smithi153.tluxni(active, since 2m), standbys: smithi174.qpcwat 2024-03-21T10:01:24.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:24 smithi174 ceph-mon[37496]: mgrmap e19: smithi153.tluxni(active, since 2m), standbys: smithi174.qpcwat 2024-03-21T10:01:25.056 INFO:journalctl@ceph.osd.2.smithi153.stdout:Mar 21 10:01:24 smithi153 sudo[67859]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-21T10:01:25.057 INFO:journalctl@ceph.osd.2.smithi153.stdout:Mar 21 10:01:24 smithi153 sudo[67859]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-21T10:01:25.057 INFO:journalctl@ceph.osd.2.smithi153.stdout:Mar 21 10:01:24 smithi153 sudo[67859]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-21T10:01:25.057 INFO:journalctl@ceph.osd.2.smithi153.stdout:Mar 21 10:01:24 smithi153 sudo[67859]: pam_unix(sudo:session): session closed for user root 2024-03-21T10:01:25.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:25 smithi153 sudo[67862]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-21T10:01:25.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:25 smithi153 sudo[67862]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-21T10:01:25.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:25 smithi153 sudo[67862]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-21T10:01:25.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:25 smithi153 sudo[67862]: pam_unix(sudo:session): session closed for user root 2024-03-21T10:01:26.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:25 smithi153 sudo[67865]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-21T10:01:26.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:25 smithi153 sudo[67865]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-21T10:01:26.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:25 smithi153 sudo[67865]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-21T10:01:26.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:25 smithi153 sudo[67865]: pam_unix(sudo:session): session closed for user root 2024-03-21T10:01:26.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:25 smithi153 ceph-mon[29117]: pgmap v101: 1 pgs: 1 active+clean; 449 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-21T10:01:26.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:25 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:01:26.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:25 smithi153 ceph-mon[29117]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-21T10:01:26.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:25 smithi174 ceph-mon[37496]: pgmap v101: 1 pgs: 1 active+clean; 449 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-21T10:01:26.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:25 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:01:26.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:25 smithi174 ceph-mon[37496]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-21T10:01:26.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:26 smithi174 sudo[41337]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-21T10:01:26.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:26 smithi174 sudo[41337]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-21T10:01:26.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:26 smithi174 sudo[41337]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-21T10:01:26.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:26 smithi174 sudo[41337]: pam_unix(sudo:session): session closed for user root 2024-03-21T10:01:27.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:26 smithi174 sudo[41340]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-21T10:01:27.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:26 smithi174 sudo[41340]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-21T10:01:27.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:26 smithi174 sudo[41340]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-21T10:01:27.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:26 smithi174 sudo[41340]: pam_unix(sudo:session): session closed for user root 2024-03-21T10:01:27.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:26 smithi174 ceph-mon[37496]: Detected new or changed devices on smithi153 2024-03-21T10:01:27.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:26 smithi174 ceph-mon[37496]: from='client.? 172.21.15.174:0/4014999747' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "285146f6-6209-47ee-9e5b-2e075c949231"}]: dispatch 2024-03-21T10:01:27.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:26 smithi174 ceph-mon[37496]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-21T10:01:27.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:26 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mon metadata", "id": "smithi153"}]: dispatch 2024-03-21T10:01:27.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:26 smithi174 ceph-mon[37496]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "285146f6-6209-47ee-9e5b-2e075c949231"}]: dispatch 2024-03-21T10:01:27.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:26 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mon metadata", "id": "smithi174"}]: dispatch 2024-03-21T10:01:27.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:26 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mon metadata", "id": "smithi153"}]: dispatch 2024-03-21T10:01:27.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:26 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mon metadata", "id": "smithi174"}]: dispatch 2024-03-21T10:01:27.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:26 smithi174 ceph-mon[37496]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "285146f6-6209-47ee-9e5b-2e075c949231"}]': finished 2024-03-21T10:01:27.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:26 smithi174 ceph-mon[37496]: osdmap e24: 4 total, 3 up, 4 in 2024-03-21T10:01:27.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:26 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:01:27.203 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:26 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T10:01:27.203 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:26 smithi174 ceph-mon[37496]: pgmap v103: 1 pgs: 1 active+clean; 449 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-21T10:01:27.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:26 smithi153 ceph-mon[29117]: Detected new or changed devices on smithi153 2024-03-21T10:01:27.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:26 smithi153 ceph-mon[29117]: from='client.? 172.21.15.174:0/4014999747' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "285146f6-6209-47ee-9e5b-2e075c949231"}]: dispatch 2024-03-21T10:01:27.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:26 smithi153 ceph-mon[29117]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-21T10:01:27.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:26 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mon metadata", "id": "smithi153"}]: dispatch 2024-03-21T10:01:27.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:26 smithi153 ceph-mon[29117]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "285146f6-6209-47ee-9e5b-2e075c949231"}]: dispatch 2024-03-21T10:01:27.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:26 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mon metadata", "id": "smithi174"}]: dispatch 2024-03-21T10:01:27.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:26 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mon metadata", "id": "smithi153"}]: dispatch 2024-03-21T10:01:27.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:26 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mon metadata", "id": "smithi174"}]: dispatch 2024-03-21T10:01:27.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:26 smithi153 ceph-mon[29117]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "285146f6-6209-47ee-9e5b-2e075c949231"}]': finished 2024-03-21T10:01:27.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:26 smithi153 ceph-mon[29117]: osdmap e24: 4 total, 3 up, 4 in 2024-03-21T10:01:27.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:26 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:01:27.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:26 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T10:01:27.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:26 smithi153 ceph-mon[29117]: pgmap v103: 1 pgs: 1 active+clean; 449 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-21T10:01:28.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:27 smithi153 ceph-mon[29117]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-21T10:01:28.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:27 smithi153 ceph-mon[29117]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-21T10:01:28.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:27 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:01:28.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:27 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:01:28.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:27 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:01:28.308 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:27 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:01:28.308 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:27 smithi153 ceph-mon[29117]: Adjusting osd_memory_target on smithi153 to 3947M 2024-03-21T10:01:28.308 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:27 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:01:28.308 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:27 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:01:28.308 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:27 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:01:28.308 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:27 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:01:28.308 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:27 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:01:28.308 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:27 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:01:28.308 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:27 smithi153 ceph-mon[29117]: from='client.? 172.21.15.174:0/2498582398' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T10:01:28.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:27 smithi174 ceph-mon[37496]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-21T10:01:28.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:27 smithi174 ceph-mon[37496]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-21T10:01:28.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:27 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:01:28.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:27 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:01:28.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:27 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:01:28.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:27 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:01:28.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:27 smithi174 ceph-mon[37496]: Adjusting osd_memory_target on smithi153 to 3947M 2024-03-21T10:01:28.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:27 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:01:28.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:27 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:01:28.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:27 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:01:28.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:27 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:01:28.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:27 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:01:28.453 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:27 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:01:28.453 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:27 smithi174 ceph-mon[37496]: from='client.? 172.21.15.174:0/2498582398' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T10:01:29.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:28 smithi174 ceph-mon[37496]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 87 KiB/s wr, 5 op/s 2024-03-21T10:01:29.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:28 smithi153 ceph-mon[29117]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 87 KiB/s wr, 5 op/s 2024-03-21T10:01:31.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:31 smithi153 ceph-mon[29117]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-03-21T10:01:31.662 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:31 smithi174 ceph-mon[37496]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-03-21T10:01:33.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:33 smithi174 ceph-mon[37496]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.7 KiB/s rd, 131 KiB/s wr, 8 op/s 2024-03-21T10:01:33.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:33 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:01:33.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:33 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:01:33.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:33 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:01:33.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:33 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:01:33.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:33 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:01:33.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:33 smithi153 ceph-mon[29117]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.7 KiB/s rd, 131 KiB/s wr, 8 op/s 2024-03-21T10:01:33.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:33 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:01:33.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:33 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:01:33.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:33 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:01:33.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:33 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:01:33.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:33 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:01:35.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:34 smithi153 ceph-mon[29117]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.7 KiB/s rd, 131 KiB/s wr, 8 op/s 2024-03-21T10:01:35.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:34 smithi174 ceph-mon[37496]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.7 KiB/s rd, 131 KiB/s wr, 8 op/s 2024-03-21T10:01:36.274 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:35 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-21T10:01:36.275 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:35 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:01:36.275 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:35 smithi174 ceph-mon[37496]: Deploying daemon osd.3 on smithi174 2024-03-21T10:01:36.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:35 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-21T10:01:36.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:35 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:01:36.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:35 smithi153 ceph-mon[29117]: Deploying daemon osd.3 on smithi174 2024-03-21T10:01:37.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:36 smithi174 ceph-mon[37496]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.6 KiB/s rd, 125 KiB/s wr, 8 op/s 2024-03-21T10:01:37.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:36 smithi153 ceph-mon[29117]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.6 KiB/s rd, 125 KiB/s wr, 8 op/s 2024-03-21T10:01:39.625 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:39 smithi174 ceph-mon[37496]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 109 KiB/s wr, 7 op/s 2024-03-21T10:01:39.625 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:39 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:01:39.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:39 smithi153 ceph-mon[29117]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 109 KiB/s wr, 7 op/s 2024-03-21T10:01:39.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:39 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:01:41.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:41 smithi174 ceph-mon[37496]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-03-21T10:01:41.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:41 smithi153 ceph-mon[29117]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-03-21T10:01:43.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:43 smithi174 ceph-mon[37496]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-03-21T10:01:43.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:43 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:01:43.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:43 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:01:43.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:43 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:01:43.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:43 smithi153 ceph-mon[29117]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-03-21T10:01:43.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:43 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:01:43.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:43 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:01:43.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:43 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:01:45.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:44 smithi174 ceph-mon[37496]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-21T10:01:45.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:44 smithi153 ceph-mon[29117]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-21T10:01:47.661 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:47 smithi174 ceph-mon[37496]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-21T10:01:47.661 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:47 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:01:47.661 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:47 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:01:47.661 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:47 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:01:47.661 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:47 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:01:47.661 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:47 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:01:47.661 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:47 smithi174 ceph-mon[37496]: from='osd.3 [v2:172.21.15.174:6800/393592876,v1:172.21.15.174:6801/393592876]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-21T10:01:47.661 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:47 smithi174 ceph-mon[37496]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-21T10:01:47.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:47 smithi153 ceph-mon[29117]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-21T10:01:47.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:47 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:01:47.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:47 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:01:47.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:47 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:01:47.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:47 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:01:47.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:47 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:01:47.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:47 smithi153 ceph-mon[29117]: from='osd.3 [v2:172.21.15.174:6800/393592876,v1:172.21.15.174:6801/393592876]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-21T10:01:47.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:47 smithi153 ceph-mon[29117]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-21T10:01:48.078 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:01:48.076+0000 7f94c7e00700 1 -- 172.21.15.174:0/2115733537 <== mgr.14221 v2:172.21.15.153:6800/1639929315 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f94d010cd50 con 0x7f94b406c460 2024-03-21T10:01:48.078 INFO:teuthology.orchestra.run.smithi174.stdout:Created osd(s) 3 on host 'smithi174' 2024-03-21T10:01:48.080 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:01:48.079+0000 7f94d6600700 1 -- 172.21.15.174:0/2115733537 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f94b406c460 msgr2=0x7f94b406e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:01:48.080 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:01:48.079+0000 7f94d6600700 1 --2- 172.21.15.174:0/2115733537 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f94b406c460 0x7f94b406e920 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f94c0006010 tx=0x7f94c0005a90 comp rx=0 tx=0).stop 2024-03-21T10:01:48.080 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:01:48.079+0000 7f94d6600700 1 -- 172.21.15.174:0/2115733537 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f94d0108400 msgr2=0x7f94d019ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:01:48.081 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:01:48.079+0000 7f94d6600700 1 --2- 172.21.15.174:0/2115733537 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f94d0108400 0x7f94d019ced0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f94bc00d8d0 tx=0x7f94bc00dc90 comp rx=0 tx=0).stop 2024-03-21T10:01:48.081 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:01:48.080+0000 7f94d6600700 1 -- 172.21.15.174:0/2115733537 shutdown_connections 2024-03-21T10:01:48.081 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:01:48.080+0000 7f94d6600700 1 --2- 172.21.15.174:0/2115733537 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f94b406c460 0x7f94b406e920 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:01:48.081 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:01:48.080+0000 7f94d6600700 1 --2- 172.21.15.174:0/2115733537 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f94d0107200 0x7f94d019c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:01:48.081 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:01:48.080+0000 7f94d6600700 1 --2- 172.21.15.174:0/2115733537 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f94d0108400 0x7f94d019ced0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:01:48.081 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:01:48.080+0000 7f94d6600700 1 -- 172.21.15.174:0/2115733537 >> 172.21.15.174:0/2115733537 conn(0x7f94d00761b0 msgr2=0x7f94d010b630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:01:48.082 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:01:48.080+0000 7f94d6600700 1 -- 172.21.15.174:0/2115733537 shutdown_connections 2024-03-21T10:01:48.082 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:01:48.080+0000 7f94d6600700 1 -- 172.21.15.174:0/2115733537 wait complete. 2024-03-21T10:01:49.021 DEBUG:teuthology.orchestra.run.smithi174:osd.3> sudo journalctl -f -n 0 -u ceph-070ed3e4-e769-11ee-95cd-87774f69a715@osd.3.service 2024-03-21T10:01:49.028 INFO:tasks.cephadm:Deploying osd.4 on smithi174 with /dev/vg_nvme/lv_3... 2024-03-21T10:01:49.028 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 070ed3e4-e769-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-21T10:01:49.275 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:49 smithi174 ceph-mon[37496]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-21T10:01:49.275 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:49 smithi174 ceph-mon[37496]: osdmap e25: 4 total, 3 up, 4 in 2024-03-21T10:01:49.275 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:49 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T10:01:49.275 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:49 smithi174 ceph-mon[37496]: from='osd.3 [v2:172.21.15.174:6800/393592876,v1:172.21.15.174:6801/393592876]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-03-21T10:01:49.275 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:49 smithi174 ceph-mon[37496]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-03-21T10:01:49.275 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:49 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:01:49.275 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:49 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:01:49.275 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:49 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:01:49.275 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:49 smithi174 ceph-mon[37496]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-21T10:01:49.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:49 smithi153 ceph-mon[29117]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-21T10:01:49.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:49 smithi153 ceph-mon[29117]: osdmap e25: 4 total, 3 up, 4 in 2024-03-21T10:01:49.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:49 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T10:01:49.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:49 smithi153 ceph-mon[29117]: from='osd.3 [v2:172.21.15.174:6800/393592876,v1:172.21.15.174:6801/393592876]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-03-21T10:01:49.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:49 smithi153 ceph-mon[29117]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-03-21T10:01:49.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:49 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:01:49.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:49 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:01:49.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:49 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:01:49.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:49 smithi153 ceph-mon[29117]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-21T10:01:50.183 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:50 smithi174 ceph-mon[37496]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-03-21T10:01:50.184 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:50 smithi174 ceph-mon[37496]: osdmap e26: 4 total, 3 up, 4 in 2024-03-21T10:01:50.184 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:50 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T10:01:50.184 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:50 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T10:01:50.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:50 smithi153 ceph-mon[29117]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-03-21T10:01:50.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:50 smithi153 ceph-mon[29117]: osdmap e26: 4 total, 3 up, 4 in 2024-03-21T10:01:50.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:50 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T10:01:50.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:50 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T10:01:50.354 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi174/config 2024-03-21T10:01:51.297 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:51 smithi174 ceph-mon[37496]: purged_snaps scrub starts 2024-03-21T10:01:51.298 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:51 smithi174 ceph-mon[37496]: purged_snaps scrub ok 2024-03-21T10:01:51.298 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:51 smithi174 ceph-mon[37496]: osd.3 [v2:172.21.15.174:6800/393592876,v1:172.21.15.174:6801/393592876] boot 2024-03-21T10:01:51.298 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:51 smithi174 ceph-mon[37496]: osdmap e27: 4 total, 4 up, 4 in 2024-03-21T10:01:51.298 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:51 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T10:01:51.298 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:51 smithi174 ceph-mon[37496]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-21T10:01:51.298 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:51 smithi174 ceph-mon[37496]: osdmap e28: 4 total, 4 up, 4 in 2024-03-21T10:01:51.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:51 smithi153 ceph-mon[29117]: purged_snaps scrub starts 2024-03-21T10:01:51.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:51 smithi153 ceph-mon[29117]: purged_snaps scrub ok 2024-03-21T10:01:51.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:51 smithi153 ceph-mon[29117]: osd.3 [v2:172.21.15.174:6800/393592876,v1:172.21.15.174:6801/393592876] boot 2024-03-21T10:01:51.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:51 smithi153 ceph-mon[29117]: osdmap e27: 4 total, 4 up, 4 in 2024-03-21T10:01:51.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:51 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T10:01:51.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:51 smithi153 ceph-mon[29117]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-21T10:01:51.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:51 smithi153 ceph-mon[29117]: osdmap e28: 4 total, 4 up, 4 in 2024-03-21T10:01:53.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:53 smithi153 ceph-mon[29117]: osdmap e29: 4 total, 4 up, 4 in 2024-03-21T10:01:53.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:53 smithi153 ceph-mon[29117]: pgmap v121: 1 pgs: 1 remapped+peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-21T10:01:53.319 INFO:teuthology.orchestra.run.smithi174.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-21T10:01:53.319 INFO:teuthology.orchestra.run.smithi174.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-21T10:01:53.319 INFO:teuthology.orchestra.run.smithi174.stderr: stderr: 10+0 records in 2024-03-21T10:01:53.319 INFO:teuthology.orchestra.run.smithi174.stderr:10+0 records out 2024-03-21T10:01:53.319 INFO:teuthology.orchestra.run.smithi174.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0375026 s, 280 MB/s 2024-03-21T10:01:53.319 INFO:teuthology.orchestra.run.smithi174.stderr:--> Zapping successful for: 2024-03-21T10:01:53.320 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:53 smithi174 ceph-mon[37496]: osdmap e29: 4 total, 4 up, 4 in 2024-03-21T10:01:53.320 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:53 smithi174 ceph-mon[37496]: pgmap v121: 1 pgs: 1 remapped+peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-21T10:01:54.064 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 070ed3e4-e769-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi174:vg_nvme/lv_3 2024-03-21T10:01:54.437 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi174/config 2024-03-21T10:01:54.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:54 smithi174 ceph-mon[37496]: osdmap e30: 4 total, 4 up, 4 in 2024-03-21T10:01:54.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:54 smithi174 ceph-mon[37496]: Detected new or changed devices on smithi174 2024-03-21T10:01:54.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:54 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:01:54.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:54 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:01:54.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:54 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:01:54.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:54 smithi174 ceph-mon[37496]: Adjusting osd_memory_target on smithi174 to 14915M 2024-03-21T10:01:54.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:54 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:01:54.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:54 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:01:54.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:54 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:01:54.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:54 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:01:54.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:54 smithi153 ceph-mon[29117]: osdmap e30: 4 total, 4 up, 4 in 2024-03-21T10:01:54.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:54 smithi153 ceph-mon[29117]: Detected new or changed devices on smithi174 2024-03-21T10:01:54.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:54 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:01:54.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:54 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:01:54.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:54 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:01:54.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:54 smithi153 ceph-mon[29117]: Adjusting osd_memory_target on smithi174 to 14915M 2024-03-21T10:01:54.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:54 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:01:54.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:54 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:01:54.558 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:54 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:01:54.558 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:54 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:01:55.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:55 smithi174 ceph-mon[37496]: osdmap e31: 4 total, 4 up, 4 in 2024-03-21T10:01:55.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:55 smithi174 ceph-mon[37496]: pgmap v124: 1 pgs: 1 remapped+peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-21T10:01:55.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:55 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:01:55.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:55 smithi153 ceph-mon[29117]: osdmap e31: 4 total, 4 up, 4 in 2024-03-21T10:01:55.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:55 smithi153 ceph-mon[29117]: pgmap v124: 1 pgs: 1 remapped+peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-21T10:01:55.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:55 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:01:56.275 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:01:56.274+0000 7f675ac00700 1 -- 172.21.15.174:0/889051203 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6754103f70 msgr2=0x7f67541043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:01:56.275 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:01:56.274+0000 7f675ac00700 1 --2- 172.21.15.174:0/889051203 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6754103f70 0x7f67541043f0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f673c009910 tx=0x7f673c009c20 comp rx=0 tx=0).stop 2024-03-21T10:01:56.275 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:01:56.274+0000 7f675ac00700 1 -- 172.21.15.174:0/889051203 shutdown_connections 2024-03-21T10:01:56.275 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:01:56.274+0000 7f675ac00700 1 --2- 172.21.15.174:0/889051203 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6754103f70 0x7f67541043f0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:01:56.275 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:01:56.274+0000 7f675ac00700 1 --2- 172.21.15.174:0/889051203 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6754102e10 0x7f6754103230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:01:56.275 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:01:56.274+0000 7f675ac00700 1 -- 172.21.15.174:0/889051203 >> 172.21.15.174:0/889051203 conn(0x7f67540fe3d0 msgr2=0x7f67541007f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:01:56.275 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:01:56.274+0000 7f675ac00700 1 -- 172.21.15.174:0/889051203 shutdown_connections 2024-03-21T10:01:56.276 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:01:56.274+0000 7f675ac00700 1 -- 172.21.15.174:0/889051203 wait complete. 2024-03-21T10:01:56.277 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:01:56.275+0000 7f675ac00700 1 Processor -- start 2024-03-21T10:01:56.277 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:01:56.275+0000 7f675ac00700 1 -- start start 2024-03-21T10:01:56.277 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:01:56.275+0000 7f675ac00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6754102e10 0x7f6754198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:01:56.277 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:01:56.275+0000 7f675ac00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6754103f70 0x7f6754198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:01:56.277 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:01:56.275+0000 7f675ac00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67541990a0 con 0x7f6754103f70 2024-03-21T10:01:56.277 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:01:56.275+0000 7f675ac00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67541991e0 con 0x7f6754102e10 2024-03-21T10:01:56.277 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:01:56.276+0000 7f6753e00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6754102e10 0x7f6754198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:01:56.278 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:01:56.276+0000 7f6753e00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6754102e10 0x7f6754198540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:49760/0 (socket says 172.21.15.174:49760) 2024-03-21T10:01:56.278 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:01:56.276+0000 7f6753e00700 1 -- 172.21.15.174:0/289145083 learned_addr learned my addr 172.21.15.174:0/289145083 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-03-21T10:01:56.278 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:01:56.276+0000 7f6753400700 1 --2- 172.21.15.174:0/289145083 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6754103f70 0x7f6754198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:01:56.278 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:01:56.276+0000 7f6753e00700 1 -- 172.21.15.174:0/289145083 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6754103f70 msgr2=0x7f6754198a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:01:56.278 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:01:56.276+0000 7f6753e00700 1 --2- 172.21.15.174:0/289145083 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6754103f70 0x7f6754198a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:01:56.278 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:01:56.276+0000 7f6753e00700 1 -- 172.21.15.174:0/289145083 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f673c0095f0 con 0x7f6754102e10 2024-03-21T10:01:56.278 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:01:56.276+0000 7f6753400700 1 --2- 172.21.15.174:0/289145083 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6754103f70 0x7f6754198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:01:56.278 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:01:56.276+0000 7f6753e00700 1 --2- 172.21.15.174:0/289145083 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6754102e10 0x7f6754198540 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f674400d8d0 tx=0x7f674400dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:01:56.279 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:01:56.277+0000 7f6750c00700 1 -- 172.21.15.174:0/289145083 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6744009940 con 0x7f6754102e10 2024-03-21T10:01:56.279 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:01:56.277+0000 7f675ac00700 1 -- 172.21.15.174:0/289145083 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f675419dc90 con 0x7f6754102e10 2024-03-21T10:01:56.279 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:01:56.277+0000 7f6750c00700 1 -- 172.21.15.174:0/289145083 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f6744010460 con 0x7f6754102e10 2024-03-21T10:01:56.279 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:01:56.277+0000 7f6750c00700 1 -- 172.21.15.174:0/289145083 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f674400f5f0 con 0x7f6754102e10 2024-03-21T10:01:56.279 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:01:56.277+0000 7f675ac00700 1 -- 172.21.15.174:0/289145083 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f675419e190 con 0x7f6754102e10 2024-03-21T10:01:56.279 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:01:56.278+0000 7f675ac00700 1 -- 172.21.15.174:0/289145083 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f675410b8f0 con 0x7f6754102e10 2024-03-21T10:01:56.279 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:01:56.279+0000 7f6750c00700 1 -- 172.21.15.174:0/289145083 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f67440105d0 con 0x7f6754102e10 2024-03-21T10:01:56.280 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:01:56.280+0000 7f6750c00700 1 --2- 172.21.15.174:0/289145083 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f673806c4e0 0x7f673806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:01:56.280 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:01:56.280+0000 7f6750c00700 1 -- 172.21.15.174:0/289145083 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 3774+0+0 (secure 0 0 0) 0x7f674408b210 con 0x7f6754102e10 2024-03-21T10:01:56.281 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:01:56.280+0000 7f6753400700 1 --2- 172.21.15.174:0/289145083 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f673806c4e0 0x7f673806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:01:56.282 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:01:56.282+0000 7f6753400700 1 --2- 172.21.15.174:0/289145083 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f673806c4e0 0x7f673806e9a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f673c006010 tx=0x7f673c00b560 comp rx=0 tx=0).ready entity=mgr.14221 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:01:56.286 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:01:56.286+0000 7f6750c00700 1 -- 172.21.15.174:0/289145083 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f6744059a80 con 0x7f6754102e10 2024-03-21T10:01:56.510 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:01:56.508+0000 7f675ac00700 1 -- 172.21.15.174:0/289145083 --> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f6754108950 con 0x7f673806c4e0 2024-03-21T10:01:57.639 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:57 smithi174 ceph-mon[37496]: pgmap v125: 1 pgs: 1 remapped+peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-21T10:01:57.639 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:57 smithi174 ceph-mon[37496]: from='client.24155 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:01:57.639 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:57 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T10:01:57.639 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:57 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T10:01:57.639 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:57 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:01:57.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:57 smithi153 ceph-mon[29117]: pgmap v125: 1 pgs: 1 remapped+peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-21T10:01:57.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:57 smithi153 ceph-mon[29117]: from='client.24155 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:01:57.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:57 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T10:01:57.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:57 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T10:01:57.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:57 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:01:59.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:01:59 smithi174 ceph-mon[37496]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-03-21T10:01:59.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:01:59 smithi153 ceph-mon[29117]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-03-21T10:02:00.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:00 smithi174 ceph-mon[37496]: from='client.? 172.21.15.174:0/405046546' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f05cec2d-c649-489d-98a3-f24a75fe1f95"}]: dispatch 2024-03-21T10:02:00.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:00 smithi174 ceph-mon[37496]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f05cec2d-c649-489d-98a3-f24a75fe1f95"}]: dispatch 2024-03-21T10:02:00.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:00 smithi174 ceph-mon[37496]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f05cec2d-c649-489d-98a3-f24a75fe1f95"}]': finished 2024-03-21T10:02:00.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:00 smithi174 ceph-mon[37496]: osdmap e32: 5 total, 4 up, 5 in 2024-03-21T10:02:00.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:00 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T10:02:00.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:00 smithi153 ceph-mon[29117]: from='client.? 172.21.15.174:0/405046546' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f05cec2d-c649-489d-98a3-f24a75fe1f95"}]: dispatch 2024-03-21T10:02:00.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:00 smithi153 ceph-mon[29117]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f05cec2d-c649-489d-98a3-f24a75fe1f95"}]: dispatch 2024-03-21T10:02:00.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:00 smithi153 ceph-mon[29117]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f05cec2d-c649-489d-98a3-f24a75fe1f95"}]': finished 2024-03-21T10:02:00.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:00 smithi153 ceph-mon[29117]: osdmap e32: 5 total, 4 up, 5 in 2024-03-21T10:02:00.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:00 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T10:02:01.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:01 smithi153 ceph-mon[29117]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-03-21T10:02:01.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:01 smithi153 ceph-mon[29117]: from='client.? 172.21.15.174:0/1882941625' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T10:02:01.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:01 smithi174 ceph-mon[37496]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-03-21T10:02:01.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:01 smithi174 ceph-mon[37496]: from='client.? 172.21.15.174:0/1882941625' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T10:02:03.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:03 smithi174 ceph-mon[37496]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-03-21T10:02:03.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:03 smithi153 ceph-mon[29117]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-03-21T10:02:05.107 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:04 smithi174 ceph-mon[37496]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-21T10:02:05.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:04 smithi153 ceph-mon[29117]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-21T10:02:07.627 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:07 smithi174 ceph-mon[37496]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-21T10:02:07.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:07 smithi153 ceph-mon[29117]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-21T10:02:09.680 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:09 smithi174 ceph-mon[37496]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-21T10:02:09.680 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:09 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-21T10:02:09.680 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:09 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:02:09.680 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:09 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:02:09.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:09 smithi153 ceph-mon[29117]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-21T10:02:09.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:09 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-21T10:02:09.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:09 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:02:09.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:09 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:02:10.657 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:10 smithi174 ceph-mon[37496]: Deploying daemon osd.4 on smithi174 2024-03-21T10:02:10.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:10 smithi153 ceph-mon[29117]: Deploying daemon osd.4 on smithi174 2024-03-21T10:02:11.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:11 smithi174 ceph-mon[37496]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-21T10:02:11.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:11 smithi153 ceph-mon[29117]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-21T10:02:13.680 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:13 smithi174 ceph-mon[37496]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-21T10:02:13.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:13 smithi153 ceph-mon[29117]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-21T10:02:15.192 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:14 smithi174 ceph-mon[37496]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-21T10:02:15.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:14 smithi153 ceph-mon[29117]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-21T10:02:16.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:16 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:02:16.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:16 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:02:16.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:16 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:02:16.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:16 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:02:16.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:16 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:02:16.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:16 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:02:17.329 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:17 smithi174 ceph-mon[37496]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-21T10:02:17.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:17 smithi153 ceph-mon[29117]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-21T10:02:19.553 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:19 smithi174 ceph-mon[37496]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-21T10:02:19.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:19 smithi153 ceph-mon[29117]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-21T10:02:20.599 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:20 smithi174 ceph-mon[37496]: from='osd.4 [v2:172.21.15.174:6808/116016676,v1:172.21.15.174:6809/116016676]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-21T10:02:20.599 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:20 smithi174 ceph-mon[37496]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-21T10:02:20.708 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:20 smithi153 ceph-mon[29117]: from='osd.4 [v2:172.21.15.174:6808/116016676,v1:172.21.15.174:6809/116016676]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-21T10:02:20.708 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:20 smithi153 ceph-mon[29117]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-21T10:02:20.709 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:02:20.707+0000 7f6750c00700 1 -- 172.21.15.174:0/289145083 <== mgr.14221 v2:172.21.15.153:6800/1639929315 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f6754108950 con 0x7f673806c4e0 2024-03-21T10:02:20.709 INFO:teuthology.orchestra.run.smithi174.stdout:Created osd(s) 4 on host 'smithi174' 2024-03-21T10:02:20.711 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:02:20.709+0000 7f675ac00700 1 -- 172.21.15.174:0/289145083 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f673806c4e0 msgr2=0x7f673806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:02:20.711 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:02:20.709+0000 7f675ac00700 1 --2- 172.21.15.174:0/289145083 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f673806c4e0 0x7f673806e9a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f673c006010 tx=0x7f673c00b560 comp rx=0 tx=0).stop 2024-03-21T10:02:20.712 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:02:20.710+0000 7f675ac00700 1 -- 172.21.15.174:0/289145083 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6754102e10 msgr2=0x7f6754198540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:02:20.712 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:02:20.710+0000 7f675ac00700 1 --2- 172.21.15.174:0/289145083 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6754102e10 0x7f6754198540 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f674400d8d0 tx=0x7f674400dc90 comp rx=0 tx=0).stop 2024-03-21T10:02:20.712 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:02:20.710+0000 7f675ac00700 1 -- 172.21.15.174:0/289145083 shutdown_connections 2024-03-21T10:02:20.712 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:02:20.710+0000 7f675ac00700 1 --2- 172.21.15.174:0/289145083 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f673806c4e0 0x7f673806e9a0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:02:20.712 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:02:20.710+0000 7f675ac00700 1 --2- 172.21.15.174:0/289145083 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6754102e10 0x7f6754198540 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:02:20.712 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:02:20.710+0000 7f675ac00700 1 --2- 172.21.15.174:0/289145083 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6754103f70 0x7f6754198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:02:20.712 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:02:20.710+0000 7f675ac00700 1 -- 172.21.15.174:0/289145083 >> 172.21.15.174:0/289145083 conn(0x7f67540fe3d0 msgr2=0x7f6754107230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:02:20.712 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:02:20.710+0000 7f675ac00700 1 -- 172.21.15.174:0/289145083 shutdown_connections 2024-03-21T10:02:20.712 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:02:20.710+0000 7f675ac00700 1 -- 172.21.15.174:0/289145083 wait complete. 2024-03-21T10:02:21.460 DEBUG:teuthology.orchestra.run.smithi174:osd.4> sudo journalctl -f -n 0 -u ceph-070ed3e4-e769-11ee-95cd-87774f69a715@osd.4.service 2024-03-21T10:02:21.465 INFO:tasks.cephadm:Deploying osd.5 on smithi174 with /dev/vg_nvme/lv_2... 2024-03-21T10:02:21.465 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 070ed3e4-e769-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-21T10:02:21.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:21 smithi174 ceph-mon[37496]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-21T10:02:21.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:21 smithi174 ceph-mon[37496]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-21T10:02:21.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:21 smithi174 ceph-mon[37496]: osdmap e33: 5 total, 4 up, 5 in 2024-03-21T10:02:21.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:21 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T10:02:21.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:21 smithi174 ceph-mon[37496]: from='osd.4 [v2:172.21.15.174:6808/116016676,v1:172.21.15.174:6809/116016676]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-03-21T10:02:21.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:21 smithi174 ceph-mon[37496]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-03-21T10:02:21.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:21 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:02:21.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:21 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:02:21.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:21 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:02:21.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:21 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:02:21.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:21 smithi153 ceph-mon[29117]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-21T10:02:21.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:21 smithi153 ceph-mon[29117]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-21T10:02:21.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:21 smithi153 ceph-mon[29117]: osdmap e33: 5 total, 4 up, 5 in 2024-03-21T10:02:21.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:21 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T10:02:21.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:21 smithi153 ceph-mon[29117]: from='osd.4 [v2:172.21.15.174:6808/116016676,v1:172.21.15.174:6809/116016676]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-03-21T10:02:21.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:21 smithi153 ceph-mon[29117]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-03-21T10:02:21.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:21 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:02:21.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:21 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:02:21.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:21 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:02:21.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:21 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:02:21.875 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi174/config 2024-03-21T10:02:22.649 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:22 smithi174 ceph-mon[37496]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-03-21T10:02:22.649 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:22 smithi174 ceph-mon[37496]: osdmap e34: 5 total, 4 up, 5 in 2024-03-21T10:02:22.650 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:22 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T10:02:22.650 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:22 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T10:02:22.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:22 smithi153 ceph-mon[29117]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-03-21T10:02:22.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:22 smithi153 ceph-mon[29117]: osdmap e34: 5 total, 4 up, 5 in 2024-03-21T10:02:22.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:22 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T10:02:22.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:22 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T10:02:23.663 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:23 smithi174 ceph-mon[37496]: purged_snaps scrub starts 2024-03-21T10:02:23.663 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:23 smithi174 ceph-mon[37496]: purged_snaps scrub ok 2024-03-21T10:02:23.663 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:23 smithi174 ceph-mon[37496]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-21T10:02:23.663 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:23 smithi174 ceph-mon[37496]: osd.4 [v2:172.21.15.174:6808/116016676,v1:172.21.15.174:6809/116016676] boot 2024-03-21T10:02:23.663 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:23 smithi174 ceph-mon[37496]: osdmap e35: 5 total, 5 up, 5 in 2024-03-21T10:02:23.664 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:23 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T10:02:23.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:23 smithi153 ceph-mon[29117]: purged_snaps scrub starts 2024-03-21T10:02:23.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:23 smithi153 ceph-mon[29117]: purged_snaps scrub ok 2024-03-21T10:02:23.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:23 smithi153 ceph-mon[29117]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-21T10:02:23.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:23 smithi153 ceph-mon[29117]: osd.4 [v2:172.21.15.174:6808/116016676,v1:172.21.15.174:6809/116016676] boot 2024-03-21T10:02:23.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:23 smithi153 ceph-mon[29117]: osdmap e35: 5 total, 5 up, 5 in 2024-03-21T10:02:23.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:23 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T10:02:24.506 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:24 smithi174 ceph-mon[37496]: osdmap e36: 5 total, 5 up, 5 in 2024-03-21T10:02:24.506 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:24 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:02:24.670 INFO:teuthology.orchestra.run.smithi174.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-21T10:02:24.670 INFO:teuthology.orchestra.run.smithi174.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-21T10:02:24.670 INFO:teuthology.orchestra.run.smithi174.stderr: stderr: 10+0 records in 2024-03-21T10:02:24.670 INFO:teuthology.orchestra.run.smithi174.stderr:10+0 records out 2024-03-21T10:02:24.670 INFO:teuthology.orchestra.run.smithi174.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0365533 s, 287 MB/s 2024-03-21T10:02:24.670 INFO:teuthology.orchestra.run.smithi174.stderr:--> Zapping successful for: 2024-03-21T10:02:24.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:24 smithi153 ceph-mon[29117]: osdmap e36: 5 total, 5 up, 5 in 2024-03-21T10:02:24.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:24 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:02:25.498 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:25 smithi174 ceph-mon[37496]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-21T10:02:25.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:25 smithi153 ceph-mon[29117]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-21T10:02:26.023 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 070ed3e4-e769-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi174:vg_nvme/lv_2 2024-03-21T10:02:26.402 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi174/config 2024-03-21T10:02:27.674 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:27 smithi174 ceph-mon[37496]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-21T10:02:27.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:27 smithi153 ceph-mon[29117]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-21T10:02:28.507 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:02:28.504+0000 7f676ac00700 1 -- 172.21.15.174:0/3933265444 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6764103fd0 msgr2=0x7f6764104430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:02:28.507 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:02:28.504+0000 7f676ac00700 1 --2- 172.21.15.174:0/3933265444 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6764103fd0 0x7f6764104430 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f674c009910 tx=0x7f674c009c20 comp rx=0 tx=0).stop 2024-03-21T10:02:28.508 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:02:28.504+0000 7f676ac00700 1 -- 172.21.15.174:0/3933265444 shutdown_connections 2024-03-21T10:02:28.508 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:02:28.504+0000 7f676ac00700 1 --2- 172.21.15.174:0/3933265444 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6764103fd0 0x7f6764104430 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:02:28.508 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:02:28.504+0000 7f676ac00700 1 --2- 172.21.15.174:0/3933265444 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6764102dd0 0x7f67641031f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:02:28.508 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:02:28.504+0000 7f676ac00700 1 -- 172.21.15.174:0/3933265444 >> 172.21.15.174:0/3933265444 conn(0x7f67640fe370 msgr2=0x7f67641007b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:02:28.508 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:02:28.504+0000 7f676ac00700 1 -- 172.21.15.174:0/3933265444 shutdown_connections 2024-03-21T10:02:28.508 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:02:28.504+0000 7f676ac00700 1 -- 172.21.15.174:0/3933265444 wait complete. 2024-03-21T10:02:28.509 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:02:28.505+0000 7f676ac00700 1 Processor -- start 2024-03-21T10:02:28.509 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:02:28.506+0000 7f676ac00700 1 -- start start 2024-03-21T10:02:28.514 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:02:28.506+0000 7f676ac00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6764102dd0 0x7f6764198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:02:28.514 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:02:28.506+0000 7f676ac00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6764103fd0 0x7f6764198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:02:28.514 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:02:28.506+0000 7f676ac00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67641990c0 con 0x7f6764102dd0 2024-03-21T10:02:28.514 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:02:28.506+0000 7f676ac00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6764199200 con 0x7f6764103fd0 2024-03-21T10:02:28.514 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:02:28.506+0000 7f675be00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6764103fd0 0x7f6764198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:02:28.514 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:02:28.506+0000 7f675be00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6764103fd0 0x7f6764198aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:38450/0 (socket says 172.21.15.174:38450) 2024-03-21T10:02:28.515 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:02:28.506+0000 7f675be00700 1 -- 172.21.15.174:0/5422972 learned_addr learned my addr 172.21.15.174:0/5422972 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-03-21T10:02:28.515 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:02:28.506+0000 7f6763e00700 1 --2- 172.21.15.174:0/5422972 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6764102dd0 0x7f6764198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:02:28.515 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:02:28.507+0000 7f675be00700 1 -- 172.21.15.174:0/5422972 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6764102dd0 msgr2=0x7f6764198560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:02:28.515 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:02:28.507+0000 7f675be00700 1 --2- 172.21.15.174:0/5422972 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6764102dd0 0x7f6764198560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:02:28.515 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:02:28.507+0000 7f675be00700 1 -- 172.21.15.174:0/5422972 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f674c0095f0 con 0x7f6764103fd0 2024-03-21T10:02:28.516 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:02:28.507+0000 7f6763e00700 1 --2- 172.21.15.174:0/5422972 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6764102dd0 0x7f6764198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:02:28.516 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:02:28.507+0000 7f675be00700 1 --2- 172.21.15.174:0/5422972 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6764103fd0 0x7f6764198aa0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f674c00bd70 tx=0x7f674c00be50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:02:28.516 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:02:28.507+0000 7f6761600700 1 -- 172.21.15.174:0/5422972 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f674c01d070 con 0x7f6764103fd0 2024-03-21T10:02:28.516 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:02:28.507+0000 7f6761600700 1 -- 172.21.15.174:0/5422972 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f674c022470 con 0x7f6764103fd0 2024-03-21T10:02:28.516 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:02:28.507+0000 7f676ac00700 1 -- 172.21.15.174:0/5422972 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f676419dc50 con 0x7f6764103fd0 2024-03-21T10:02:28.516 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:02:28.507+0000 7f6761600700 1 -- 172.21.15.174:0/5422972 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f674c00f670 con 0x7f6764103fd0 2024-03-21T10:02:28.517 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:02:28.507+0000 7f676ac00700 1 -- 172.21.15.174:0/5422972 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f676419e140 con 0x7f6764103fd0 2024-03-21T10:02:28.517 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:02:28.509+0000 7f676ac00700 1 -- 172.21.15.174:0/5422972 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f676410b930 con 0x7f6764103fd0 2024-03-21T10:02:28.517 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:02:28.509+0000 7f6761600700 1 -- 172.21.15.174:0/5422972 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f674c022a50 con 0x7f6764103fd0 2024-03-21T10:02:28.517 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:02:28.510+0000 7f6761600700 1 --2- 172.21.15.174:0/5422972 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f674806c4e0 0x7f674806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:02:28.517 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:02:28.510+0000 7f6761600700 1 -- 172.21.15.174:0/5422972 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4206+0+0 (secure 0 0 0) 0x7f674c08c900 con 0x7f6764103fd0 2024-03-21T10:02:28.517 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:02:28.511+0000 7f6763e00700 1 --2- 172.21.15.174:0/5422972 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f674806c4e0 0x7f674806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:02:28.518 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:02:28.512+0000 7f6763e00700 1 --2- 172.21.15.174:0/5422972 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f674806c4e0 0x7f674806e9a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6754005fd0 tx=0x7f6754005dc0 comp rx=0 tx=0).ready entity=mgr.14221 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:02:28.519 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:02:28.516+0000 7f6761600700 1 -- 172.21.15.174:0/5422972 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f674c05b190 con 0x7f6764103fd0 2024-03-21T10:02:28.748 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:02:28.745+0000 7f676ac00700 1 -- 172.21.15.174:0/5422972 --> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f6764108920 con 0x7f674806c4e0 2024-03-21T10:02:29.691 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:29 smithi174 ceph-mon[37496]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-21T10:02:29.691 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:29 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T10:02:29.691 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:29 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T10:02:29.691 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:29 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:02:29.691 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:29 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:02:29.691 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:29 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:02:29.691 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:29 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:02:29.691 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:29 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:02:29.691 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:29 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:02:29.692 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:29 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:02:29.692 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:29 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:02:29.692 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:29 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:02:29.692 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:29 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:02:29.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:29 smithi153 ceph-mon[29117]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-21T10:02:29.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:29 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T10:02:29.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:29 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T10:02:29.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:29 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:02:29.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:29 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:02:29.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:29 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:02:29.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:29 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:02:29.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:29 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:02:29.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:29 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:02:29.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:29 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:02:29.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:29 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:02:29.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:29 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:02:29.808 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:29 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:02:30.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:30 smithi174 ceph-mon[37496]: from='client.24173 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:02:30.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:30 smithi174 ceph-mon[37496]: Detected new or changed devices on smithi174 2024-03-21T10:02:30.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:30 smithi174 ceph-mon[37496]: Adjusting osd_memory_target on smithi174 to 7457M 2024-03-21T10:02:30.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:30 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:02:30.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:30 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:02:30.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:30 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:02:30.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:30 smithi153 ceph-mon[29117]: from='client.24173 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:02:30.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:30 smithi153 ceph-mon[29117]: Detected new or changed devices on smithi174 2024-03-21T10:02:30.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:30 smithi153 ceph-mon[29117]: Adjusting osd_memory_target on smithi174 to 7457M 2024-03-21T10:02:30.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:30 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:02:30.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:30 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:02:30.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:30 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:02:31.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:31 smithi174 ceph-mon[37496]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-21T10:02:31.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:31 smithi153 ceph-mon[29117]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-21T10:02:32.764 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:32 smithi174 ceph-mon[37496]: from='client.? 172.21.15.174:0/4092482280' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f430a3b2-e1b5-4698-9089-812fcf3a4959"}]: dispatch 2024-03-21T10:02:32.764 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:32 smithi174 ceph-mon[37496]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f430a3b2-e1b5-4698-9089-812fcf3a4959"}]: dispatch 2024-03-21T10:02:32.764 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:32 smithi174 ceph-mon[37496]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f430a3b2-e1b5-4698-9089-812fcf3a4959"}]': finished 2024-03-21T10:02:32.764 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:32 smithi174 ceph-mon[37496]: osdmap e37: 6 total, 5 up, 6 in 2024-03-21T10:02:32.764 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:32 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T10:02:32.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:32 smithi153 ceph-mon[29117]: from='client.? 172.21.15.174:0/4092482280' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f430a3b2-e1b5-4698-9089-812fcf3a4959"}]: dispatch 2024-03-21T10:02:32.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:32 smithi153 ceph-mon[29117]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f430a3b2-e1b5-4698-9089-812fcf3a4959"}]: dispatch 2024-03-21T10:02:32.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:32 smithi153 ceph-mon[29117]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f430a3b2-e1b5-4698-9089-812fcf3a4959"}]': finished 2024-03-21T10:02:32.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:32 smithi153 ceph-mon[29117]: osdmap e37: 6 total, 5 up, 6 in 2024-03-21T10:02:32.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:32 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T10:02:33.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:33 smithi153 ceph-mon[29117]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-21T10:02:33.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:33 smithi153 ceph-mon[29117]: from='client.? 172.21.15.174:0/1304711300' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T10:02:33.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:33 smithi174 ceph-mon[37496]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-21T10:02:33.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:33 smithi174 ceph-mon[37496]: from='client.? 172.21.15.174:0/1304711300' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T10:02:35.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:34 smithi153 ceph-mon[29117]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-21T10:02:35.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:34 smithi174 ceph-mon[37496]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-21T10:02:37.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:37 smithi174 ceph-mon[37496]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-21T10:02:37.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:37 smithi153 ceph-mon[29117]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-21T10:02:39.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:39 smithi174 ceph-mon[37496]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-21T10:02:39.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:39 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:02:39.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:39 smithi153 ceph-mon[29117]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-21T10:02:39.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:39 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:02:41.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:41 smithi153 ceph-mon[29117]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-21T10:02:41.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:41 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-21T10:02:41.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:41 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:02:41.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:41 smithi174 ceph-mon[37496]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-21T10:02:41.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:41 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-21T10:02:41.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:41 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:02:42.552 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:42 smithi174 ceph-mon[37496]: Deploying daemon osd.5 on smithi174 2024-03-21T10:02:42.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:42 smithi153 ceph-mon[29117]: Deploying daemon osd.5 on smithi174 2024-03-21T10:02:43.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:43 smithi174 ceph-mon[37496]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-21T10:02:43.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:43 smithi153 ceph-mon[29117]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-21T10:02:45.060 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:44 smithi174 ceph-mon[37496]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-21T10:02:45.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:44 smithi153 ceph-mon[29117]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-21T10:02:47.652 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:47 smithi174 ceph-mon[37496]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-21T10:02:47.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:47 smithi153 ceph-mon[29117]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-21T10:02:49.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:49 smithi174 ceph-mon[37496]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-21T10:02:49.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:49 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:02:49.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:49 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:02:49.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:49 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:02:49.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:49 smithi153 ceph-mon[29117]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-21T10:02:49.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:49 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:02:49.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:49 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:02:49.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:49 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:02:51.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:51 smithi174 ceph-mon[37496]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-21T10:02:51.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:51 smithi153 ceph-mon[29117]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-21T10:02:53.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:53 smithi174 ceph-mon[37496]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-21T10:02:53.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:53 smithi174 ceph-mon[37496]: from='osd.5 [v2:172.21.15.174:6816/184680961,v1:172.21.15.174:6817/184680961]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-21T10:02:53.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:53 smithi174 ceph-mon[37496]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-21T10:02:53.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:53 smithi153 ceph-mon[29117]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-21T10:02:53.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:53 smithi153 ceph-mon[29117]: from='osd.5 [v2:172.21.15.174:6816/184680961,v1:172.21.15.174:6817/184680961]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-21T10:02:53.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:53 smithi153 ceph-mon[29117]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-21T10:02:54.051 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:02:54.050+0000 7f6761600700 1 -- 172.21.15.174:0/5422972 <== mgr.14221 v2:172.21.15.153:6800/1639929315 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f6764108920 con 0x7f674806c4e0 2024-03-21T10:02:54.052 INFO:teuthology.orchestra.run.smithi174.stdout:Created osd(s) 5 on host 'smithi174' 2024-03-21T10:02:54.054 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:02:54.053+0000 7f676ac00700 1 -- 172.21.15.174:0/5422972 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f674806c4e0 msgr2=0x7f674806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:02:54.054 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:02:54.053+0000 7f676ac00700 1 --2- 172.21.15.174:0/5422972 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f674806c4e0 0x7f674806e9a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6754005fd0 tx=0x7f6754005dc0 comp rx=0 tx=0).stop 2024-03-21T10:02:54.054 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:02:54.053+0000 7f676ac00700 1 -- 172.21.15.174:0/5422972 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6764103fd0 msgr2=0x7f6764198aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:02:54.054 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:02:54.053+0000 7f676ac00700 1 --2- 172.21.15.174:0/5422972 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6764103fd0 0x7f6764198aa0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f674c00bd70 tx=0x7f674c00be50 comp rx=0 tx=0).stop 2024-03-21T10:02:54.055 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:02:54.053+0000 7f676ac00700 1 -- 172.21.15.174:0/5422972 shutdown_connections 2024-03-21T10:02:54.055 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:02:54.053+0000 7f676ac00700 1 --2- 172.21.15.174:0/5422972 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f674806c4e0 0x7f674806e9a0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:02:54.055 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:02:54.053+0000 7f676ac00700 1 --2- 172.21.15.174:0/5422972 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6764102dd0 0x7f6764198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:02:54.055 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:02:54.053+0000 7f676ac00700 1 --2- 172.21.15.174:0/5422972 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6764103fd0 0x7f6764198aa0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:02:54.055 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:02:54.053+0000 7f676ac00700 1 -- 172.21.15.174:0/5422972 >> 172.21.15.174:0/5422972 conn(0x7f67640fe370 msgr2=0x7f6764107200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:02:54.055 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:02:54.053+0000 7f676ac00700 1 -- 172.21.15.174:0/5422972 shutdown_connections 2024-03-21T10:02:54.055 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:02:54.053+0000 7f676ac00700 1 -- 172.21.15.174:0/5422972 wait complete. 2024-03-21T10:02:54.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:54 smithi174 ceph-mon[37496]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-21T10:02:54.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:54 smithi174 ceph-mon[37496]: osdmap e38: 6 total, 5 up, 6 in 2024-03-21T10:02:54.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:54 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T10:02:54.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:54 smithi174 ceph-mon[37496]: from='osd.5 [v2:172.21.15.174:6816/184680961,v1:172.21.15.174:6817/184680961]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-03-21T10:02:54.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:54 smithi174 ceph-mon[37496]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-03-21T10:02:54.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:54 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:02:54.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:54 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:02:54.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:54 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:02:54.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:54 smithi153 ceph-mon[29117]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-21T10:02:54.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:54 smithi153 ceph-mon[29117]: osdmap e38: 6 total, 5 up, 6 in 2024-03-21T10:02:54.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:54 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T10:02:54.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:54 smithi153 ceph-mon[29117]: from='osd.5 [v2:172.21.15.174:6816/184680961,v1:172.21.15.174:6817/184680961]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-03-21T10:02:54.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:54 smithi153 ceph-mon[29117]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-03-21T10:02:54.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:54 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:02:54.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:54 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:02:54.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:54 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:02:54.913 DEBUG:teuthology.orchestra.run.smithi174:osd.5> sudo journalctl -f -n 0 -u ceph-070ed3e4-e769-11ee-95cd-87774f69a715@osd.5.service 2024-03-21T10:02:54.919 INFO:tasks.cephadm:Waiting for 6 OSDs to come up... 2024-03-21T10:02:54.919 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 070ed3e4-e769-11ee-95cd-87774f69a715 -- ceph osd stat -f json 2024-03-21T10:02:55.337 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi153/config 2024-03-21T10:02:55.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:55 smithi174 ceph-mon[37496]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-21T10:02:55.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:55 smithi174 ceph-mon[37496]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-03-21T10:02:55.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:55 smithi174 ceph-mon[37496]: osdmap e39: 6 total, 5 up, 6 in 2024-03-21T10:02:55.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:55 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T10:02:55.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:55 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T10:02:55.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:55 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:02:55.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:55 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:02:55.760 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:55 smithi153 ceph-mon[29117]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-21T10:02:55.760 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:55 smithi153 ceph-mon[29117]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-03-21T10:02:55.760 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:55 smithi153 ceph-mon[29117]: osdmap e39: 6 total, 5 up, 6 in 2024-03-21T10:02:55.761 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:55 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T10:02:55.761 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:55 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T10:02:55.761 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:55 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:02:55.761 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:55 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:02:56.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:56 smithi153 ceph-mon[29117]: purged_snaps scrub starts 2024-03-21T10:02:56.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:56 smithi153 ceph-mon[29117]: purged_snaps scrub ok 2024-03-21T10:02:56.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:56 smithi153 ceph-mon[29117]: osd.5 [v2:172.21.15.174:6816/184680961,v1:172.21.15.174:6817/184680961] boot 2024-03-21T10:02:56.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:56 smithi153 ceph-mon[29117]: osdmap e40: 6 total, 6 up, 6 in 2024-03-21T10:02:56.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:56 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T10:02:56.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:56 smithi174 ceph-mon[37496]: purged_snaps scrub starts 2024-03-21T10:02:56.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:56 smithi174 ceph-mon[37496]: purged_snaps scrub ok 2024-03-21T10:02:56.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:56 smithi174 ceph-mon[37496]: osd.5 [v2:172.21.15.174:6816/184680961,v1:172.21.15.174:6817/184680961] boot 2024-03-21T10:02:56.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:56 smithi174 ceph-mon[37496]: osdmap e40: 6 total, 6 up, 6 in 2024-03-21T10:02:56.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:56 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T10:02:57.529 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:02:57.526+0000 7f5b31400700 1 -- 172.21.15.153:0/1506743626 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5b2c0737f0 msgr2=0x7f5b2c073c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:02:57.529 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:02:57.526+0000 7f5b31400700 1 --2- 172.21.15.153:0/1506743626 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5b2c0737f0 0x7f5b2c073c70 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5b18009910 tx=0x7f5b18009c20 comp rx=0 tx=0).stop 2024-03-21T10:02:57.529 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:02:57.527+0000 7f5b31400700 1 -- 172.21.15.153:0/1506743626 shutdown_connections 2024-03-21T10:02:57.529 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:02:57.527+0000 7f5b31400700 1 --2- 172.21.15.153:0/1506743626 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5b2c0737f0 0x7f5b2c073c70 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:02:57.530 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:02:57.527+0000 7f5b31400700 1 --2- 172.21.15.153:0/1506743626 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b2c074dc0 0x7f5b2c073220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:02:57.530 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:02:57.527+0000 7f5b31400700 1 -- 172.21.15.153:0/1506743626 >> 172.21.15.153:0/1506743626 conn(0x7f5b2c0fbf30 msgr2=0x7f5b2c0fe370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:02:57.530 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:02:57.527+0000 7f5b31400700 1 -- 172.21.15.153:0/1506743626 shutdown_connections 2024-03-21T10:02:57.530 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:02:57.527+0000 7f5b31400700 1 -- 172.21.15.153:0/1506743626 wait complete. 2024-03-21T10:02:57.530 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:02:57.528+0000 7f5b31400700 1 Processor -- start 2024-03-21T10:02:57.530 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:02:57.528+0000 7f5b31400700 1 -- start start 2024-03-21T10:02:57.532 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:02:57.528+0000 7f5b31400700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b2c0737f0 0x7f5b2c19c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:02:57.532 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:02:57.528+0000 7f5b31400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5b2c074dc0 0x7f5b2c19ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:02:57.532 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:02:57.528+0000 7f5b31400700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b2c19d2d0 con 0x7f5b2c074dc0 2024-03-21T10:02:57.532 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:02:57.528+0000 7f5b31400700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b2c19d410 con 0x7f5b2c0737f0 2024-03-21T10:02:57.532 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:02:57.529+0000 7f5b2b400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5b2c074dc0 0x7f5b2c19ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:02:57.533 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:02:57.529+0000 7f5b2b400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5b2c074dc0 0x7f5b2c19ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:44402/0 (socket says 172.21.15.153:44402) 2024-03-21T10:02:57.533 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:02:57.529+0000 7f5b2b400700 1 -- 172.21.15.153:0/3650304287 learned_addr learned my addr 172.21.15.153:0/3650304287 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:02:57.533 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:02:57.529+0000 7f5b2be00700 1 --2- 172.21.15.153:0/3650304287 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b2c0737f0 0x7f5b2c19c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:02:57.533 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:02:57.529+0000 7f5b2b400700 1 -- 172.21.15.153:0/3650304287 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b2c0737f0 msgr2=0x7f5b2c19c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:02:57.533 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:02:57.529+0000 7f5b2b400700 1 --2- 172.21.15.153:0/3650304287 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b2c0737f0 0x7f5b2c19c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:02:57.533 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:02:57.529+0000 7f5b2b400700 1 -- 172.21.15.153:0/3650304287 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b180095f0 con 0x7f5b2c074dc0 2024-03-21T10:02:57.533 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:02:57.529+0000 7f5b2be00700 1 --2- 172.21.15.153:0/3650304287 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b2c0737f0 0x7f5b2c19c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:02:57.533 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:02:57.530+0000 7f5b2b400700 1 --2- 172.21.15.153:0/3650304287 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5b2c074dc0 0x7f5b2c19ccb0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5b1800bd70 tx=0x7f5b1800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:02:57.533 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:02:57.530+0000 7f5b28c00700 1 -- 172.21.15.153:0/3650304287 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5b1801d070 con 0x7f5b2c074dc0 2024-03-21T10:02:57.533 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:02:57.530+0000 7f5b28c00700 1 -- 172.21.15.153:0/3650304287 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f5b1800f460 con 0x7f5b2c074dc0 2024-03-21T10:02:57.533 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:02:57.530+0000 7f5b28c00700 1 -- 172.21.15.153:0/3650304287 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5b18021620 con 0x7f5b2c074dc0 2024-03-21T10:02:57.533 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:02:57.530+0000 7f5b31400700 1 -- 172.21.15.153:0/3650304287 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b2c1a1e60 con 0x7f5b2c074dc0 2024-03-21T10:02:57.534 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:02:57.530+0000 7f5b31400700 1 -- 172.21.15.153:0/3650304287 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b2c1a22d0 con 0x7f5b2c074dc0 2024-03-21T10:02:57.534 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:02:57.531+0000 7f5b31400700 1 -- 172.21.15.153:0/3650304287 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b2c10fb20 con 0x7f5b2c074dc0 2024-03-21T10:02:57.534 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:02:57.532+0000 7f5b28c00700 1 -- 172.21.15.153:0/3650304287 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f5b1800fa40 con 0x7f5b2c074dc0 2024-03-21T10:02:57.535 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:57 smithi174 ceph-mon[37496]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:02:57.535 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:57 smithi174 ceph-mon[37496]: osdmap e41: 6 total, 6 up, 6 in 2024-03-21T10:02:57.536 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:02:57.533+0000 7f5b28c00700 1 --2- 172.21.15.153:0/3650304287 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f5b1006c4e0 0x7f5b1006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:02:57.536 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:02:57.534+0000 7f5b28c00700 1 -- 172.21.15.153:0/3650304287 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f5b1808c7f0 con 0x7f5b2c074dc0 2024-03-21T10:02:57.536 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:02:57.534+0000 7f5b2be00700 1 --2- 172.21.15.153:0/3650304287 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f5b1006c4e0 0x7f5b1006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:02:57.536 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:02:57.534+0000 7f5b2be00700 1 --2- 172.21.15.153:0/3650304287 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f5b1006c4e0 0x7f5b1006e9a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5b1c005950 tx=0x7f5b1c0058e0 comp rx=0 tx=0).ready entity=mgr.14221 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:02:57.540 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:02:57.539+0000 7f5b28c00700 1 -- 172.21.15.153:0/3650304287 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f5b1805ad80 con 0x7f5b2c074dc0 2024-03-21T10:02:57.630 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:57 smithi153 ceph-mon[29117]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:02:57.630 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:57 smithi153 ceph-mon[29117]: osdmap e41: 6 total, 6 up, 6 in 2024-03-21T10:02:57.760 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:02:57.759+0000 7f5b31400700 1 -- 172.21.15.153:0/3650304287 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f5b2c1a29d0 con 0x7f5b2c074dc0 2024-03-21T10:02:57.761 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:02:57.760+0000 7f5b28c00700 1 -- 172.21.15.153:0/3650304287 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v41) v1 ==== 74+0+130 (secure 0 0 0) 0x7f5b1805a910 con 0x7f5b2c074dc0 2024-03-21T10:02:57.761 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:02:57.764 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:02:57.762+0000 7f5b31400700 1 -- 172.21.15.153:0/3650304287 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f5b1006c4e0 msgr2=0x7f5b1006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:02:57.764 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:02:57.762+0000 7f5b31400700 1 --2- 172.21.15.153:0/3650304287 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f5b1006c4e0 0x7f5b1006e9a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5b1c005950 tx=0x7f5b1c0058e0 comp rx=0 tx=0).stop 2024-03-21T10:02:57.764 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:02:57.762+0000 7f5b31400700 1 -- 172.21.15.153:0/3650304287 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5b2c074dc0 msgr2=0x7f5b2c19ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:02:57.764 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:02:57.762+0000 7f5b31400700 1 --2- 172.21.15.153:0/3650304287 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5b2c074dc0 0x7f5b2c19ccb0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5b1800bd70 tx=0x7f5b1800be50 comp rx=0 tx=0).stop 2024-03-21T10:02:57.765 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:02:57.763+0000 7f5b31400700 1 -- 172.21.15.153:0/3650304287 shutdown_connections 2024-03-21T10:02:57.765 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:02:57.763+0000 7f5b31400700 1 --2- 172.21.15.153:0/3650304287 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f5b1006c4e0 0x7f5b1006e9a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:02:57.765 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:02:57.763+0000 7f5b31400700 1 --2- 172.21.15.153:0/3650304287 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b2c0737f0 0x7f5b2c19c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:02:57.765 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:02:57.763+0000 7f5b31400700 1 --2- 172.21.15.153:0/3650304287 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5b2c074dc0 0x7f5b2c19ccb0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:02:57.765 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:02:57.763+0000 7f5b31400700 1 -- 172.21.15.153:0/3650304287 >> 172.21.15.153:0/3650304287 conn(0x7f5b2c0fbf30 msgr2=0x7f5b2c102210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:02:57.765 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:02:57.763+0000 7f5b31400700 1 -- 172.21.15.153:0/3650304287 shutdown_connections 2024-03-21T10:02:57.765 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:02:57.763+0000 7f5b31400700 1 -- 172.21.15.153:0/3650304287 wait complete. 2024-03-21T10:02:58.524 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":41,"num_osds":6,"num_up_osds":6,"osd_up_since":1711015375,"num_in_osds":6,"osd_in_since":1711015352,"num_remapped_pgs":0} 2024-03-21T10:02:58.525 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 070ed3e4-e769-11ee-95cd-87774f69a715 -- ceph osd dump --format=json 2024-03-21T10:02:58.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:58 smithi153 ceph-mon[29117]: from='client.? 172.21.15.153:0/3650304287' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-21T10:02:58.903 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi153/config 2024-03-21T10:02:58.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:58 smithi174 ceph-mon[37496]: from='client.? 172.21.15.153:0/3650304287' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-21T10:02:59.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:02:59 smithi174 ceph-mon[37496]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:02:59.747 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:02:59 smithi153 ceph-mon[29117]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:03:00.854 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:00.853+0000 7f26acc00700 1 -- 172.21.15.153:0/1567278196 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26a81071d0 msgr2=0x7f26a8107650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:03:00.854 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:00.853+0000 7f26acc00700 1 --2- 172.21.15.153:0/1567278196 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26a81071d0 0x7f26a8107650 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2694009910 tx=0x7f2694009c20 comp rx=0 tx=0).stop 2024-03-21T10:03:00.855 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:00.853+0000 7f26acc00700 1 -- 172.21.15.153:0/1567278196 shutdown_connections 2024-03-21T10:03:00.855 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:00.853+0000 7f26acc00700 1 --2- 172.21.15.153:0/1567278196 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26a81071d0 0x7f26a8107650 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:00.855 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:00.853+0000 7f26acc00700 1 --2- 172.21.15.153:0/1567278196 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26a810d280 0x7f26a810d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:00.855 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:00.854+0000 7f26acc00700 1 -- 172.21.15.153:0/1567278196 >> 172.21.15.153:0/1567278196 conn(0x7f26a80769e0 msgr2=0x7f26a8078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:03:00.855 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:00.854+0000 7f26acc00700 1 -- 172.21.15.153:0/1567278196 shutdown_connections 2024-03-21T10:03:00.855 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:00.854+0000 7f26acc00700 1 -- 172.21.15.153:0/1567278196 wait complete. 2024-03-21T10:03:00.856 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:00.855+0000 7f26acc00700 1 Processor -- start 2024-03-21T10:03:00.856 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:00.855+0000 7f26acc00700 1 -- start start 2024-03-21T10:03:00.856 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:00.855+0000 7f26acc00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26a81071d0 0x7f26a819ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:03:00.856 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:00.855+0000 7f26acc00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26a810d280 0x7f26a819d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:03:00.856 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:00.855+0000 7f26acc00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26a819d880 con 0x7f26a81071d0 2024-03-21T10:03:00.857 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:00.855+0000 7f26acc00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26a81a05b0 con 0x7f26a810d280 2024-03-21T10:03:00.857 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:00.855+0000 7f26a5e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26a81071d0 0x7f26a819ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:03:00.857 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:00.855+0000 7f26a5e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26a81071d0 0x7f26a819ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:44410/0 (socket says 172.21.15.153:44410) 2024-03-21T10:03:00.857 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:00.855+0000 7f26a5e00700 1 -- 172.21.15.153:0/1458905751 learned_addr learned my addr 172.21.15.153:0/1458905751 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:03:00.857 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:00.856+0000 7f26a5400700 1 --2- 172.21.15.153:0/1458905751 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26a810d280 0x7f26a819d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:03:00.857 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:00.856+0000 7f26a5e00700 1 -- 172.21.15.153:0/1458905751 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26a810d280 msgr2=0x7f26a819d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:03:00.857 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:00.856+0000 7f26a5e00700 1 --2- 172.21.15.153:0/1458905751 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26a810d280 0x7f26a819d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:00.857 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:00.856+0000 7f26a5e00700 1 -- 172.21.15.153:0/1458905751 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26940095f0 con 0x7f26a81071d0 2024-03-21T10:03:00.857 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:00.856+0000 7f26a5e00700 1 --2- 172.21.15.153:0/1458905751 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26a81071d0 0x7f26a819ccb0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f269800d8d0 tx=0x7f269800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:03:00.858 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:00.856+0000 7f269ea00700 1 -- 172.21.15.153:0/1458905751 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2698009940 con 0x7f26a81071d0 2024-03-21T10:03:00.858 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:00.856+0000 7f269ea00700 1 -- 172.21.15.153:0/1458905751 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f2698010460 con 0x7f26a81071d0 2024-03-21T10:03:00.858 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:00.856+0000 7f269ea00700 1 -- 172.21.15.153:0/1458905751 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f269800f5f0 con 0x7f26a81071d0 2024-03-21T10:03:00.858 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:00.856+0000 7f26acc00700 1 -- 172.21.15.153:0/1458905751 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26a81a0890 con 0x7f26a81071d0 2024-03-21T10:03:00.858 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:00.857+0000 7f26acc00700 1 -- 172.21.15.153:0/1458905751 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26a81a0de0 con 0x7f26a81071d0 2024-03-21T10:03:00.858 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:00.858+0000 7f26acc00700 1 -- 172.21.15.153:0/1458905751 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26a810f6c0 con 0x7f26a81071d0 2024-03-21T10:03:00.859 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:00.859+0000 7f269ea00700 1 -- 172.21.15.153:0/1458905751 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f26980105d0 con 0x7f26a81071d0 2024-03-21T10:03:00.860 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:00.859+0000 7f269ea00700 1 --2- 172.21.15.153:0/1458905751 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f268c06c4e0 0x7f268c06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:03:00.860 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:00.860+0000 7f269ea00700 1 -- 172.21.15.153:0/1458905751 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f269808b490 con 0x7f26a81071d0 2024-03-21T10:03:00.860 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:00.860+0000 7f26a5400700 1 --2- 172.21.15.153:0/1458905751 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f268c06c4e0 0x7f268c06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:03:00.860 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:00.860+0000 7f26a5400700 1 --2- 172.21.15.153:0/1458905751 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f268c06c4e0 0x7f268c06e9a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f26a819e290 tx=0x7f269401a040 comp rx=0 tx=0).ready entity=mgr.14221 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:03:00.865 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:00.865+0000 7f269ea00700 1 -- 172.21.15.153:0/1458905751 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f2698055f90 con 0x7f26a81071d0 2024-03-21T10:03:01.082 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:01.082+0000 7f26acc00700 1 -- 172.21.15.153:0/1458905751 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f26a804ea90 con 0x7f26a81071d0 2024-03-21T10:03:01.084 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:01.084+0000 7f269ea00700 1 -- 172.21.15.153:0/1458905751 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+11360 (secure 0 0 0) 0x7f269808b070 con 0x7f26a81071d0 2024-03-21T10:03:01.085 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:03:01.085 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":41,"fsid":"070ed3e4-e769-11ee-95cd-87774f69a715","created":"2024-03-21T09:55:19.360240+0000","modified":"2024-03-21T10:02:56.435311+0000","last_up_change":"2024-03-21T10:02:55.420099+0000","last_in_change":"2024-03-21T10:02:32.306573+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T10:01:18.943639+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0f2e1224-6f45-4f23-b97a-5b89319a90f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":1050381948},{"type":"v1","addr":"172.21.15.153:6803","nonce":1050381948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":1050381948},{"type":"v1","addr":"172.21.15.153:6805","nonce":1050381948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":1050381948},{"type":"v1","addr":"172.21.15.153:6809","nonce":1050381948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":1050381948},{"type":"v1","addr":"172.21.15.153:6807","nonce":1050381948}]},"public_addr":"172.21.15.153:6803/1050381948","cluster_addr":"172.21.15.153:6805/1050381948","heartbeat_back_addr":"172.21.15.153:6809/1050381948","heartbeat_front_addr":"172.21.15.153:6807/1050381948","state":["exists","up"]},{"osd":1,"uuid":"76195edb-aa7a-4627-a211-09a578de65c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":3504985760},{"type":"v1","addr":"172.21.15.153:6811","nonce":3504985760}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":3504985760},{"type":"v1","addr":"172.21.15.153:6813","nonce":3504985760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":3504985760},{"type":"v1","addr":"172.21.15.153:6817","nonce":3504985760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":3504985760},{"type":"v1","addr":"172.21.15.153:6815","nonce":3504985760}]},"public_addr":"172.21.15.153:6811/3504985760","cluster_addr":"172.21.15.153:6813/3504985760","heartbeat_back_addr":"172.21.15.153:6817/3504985760","heartbeat_front_addr":"172.21.15.153:6815/3504985760","state":["exists","up"]},{"osd":2,"uuid":"3ec17cf8-f0ea-45c4-b084-b304024d6e92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":2844395219},{"type":"v1","addr":"172.21.15.153:6819","nonce":2844395219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":2844395219},{"type":"v1","addr":"172.21.15.153:6821","nonce":2844395219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":2844395219},{"type":"v1","addr":"172.21.15.153:6825","nonce":2844395219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":2844395219},{"type":"v1","addr":"172.21.15.153:6823","nonce":2844395219}]},"public_addr":"172.21.15.153:6819/2844395219","cluster_addr":"172.21.15.153:6821/2844395219","heartbeat_back_addr":"172.21.15.153:6825/2844395219","heartbeat_front_addr":"172.21.15.153:6823/2844395219","state":["exists","up"]},{"osd":3,"uuid":"285146f6-6209-47ee-9e5b-2e075c949231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":393592876},{"type":"v1","addr":"172.21.15.174:6801","nonce":393592876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":393592876},{"type":"v1","addr":"172.21.15.174:6803","nonce":393592876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":393592876},{"type":"v1","addr":"172.21.15.174:6807","nonce":393592876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":393592876},{"type":"v1","addr":"172.21.15.174:6805","nonce":393592876}]},"public_addr":"172.21.15.174:6801/393592876","cluster_addr":"172.21.15.174:6803/393592876","heartbeat_back_addr":"172.21.15.174:6807/393592876","heartbeat_front_addr":"172.21.15.174:6805/393592876","state":["exists","up"]},{"osd":4,"uuid":"f05cec2d-c649-489d-98a3-f24a75fe1f95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":116016676},{"type":"v1","addr":"172.21.15.174:6809","nonce":116016676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":116016676},{"type":"v1","addr":"172.21.15.174:6811","nonce":116016676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":116016676},{"type":"v1","addr":"172.21.15.174:6815","nonce":116016676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":116016676},{"type":"v1","addr":"172.21.15.174:6813","nonce":116016676}]},"public_addr":"172.21.15.174:6809/116016676","cluster_addr":"172.21.15.174:6811/116016676","heartbeat_back_addr":"172.21.15.174:6815/116016676","heartbeat_front_addr":"172.21.15.174:6813/116016676","state":["exists","up"]},{"osd":5,"uuid":"f430a3b2-e1b5-4698-9089-812fcf3a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":184680961},{"type":"v1","addr":"172.21.15.174:6817","nonce":184680961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":184680961},{"type":"v1","addr":"172.21.15.174:6819","nonce":184680961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":184680961},{"type":"v1","addr":"172.21.15.174:6823","nonce":184680961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":184680961},{"type":"v1","addr":"172.21.15.174:6821","nonce":184680961}]},"public_addr":"172.21.15.174:6817/184680961","cluster_addr":"172.21.15.174:6819/184680961","heartbeat_back_addr":"172.21.15.174:6823/184680961","heartbeat_front_addr":"172.21.15.174:6821/184680961","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:00:04.356688+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:00:39.988638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:01:14.882381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:01:48.072468+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:02:21.211701+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:02:54.161138+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.153:0/3278686288":"2024-03-22T09:58:24.250138+0000","172.21.15.153:0/652104290":"2024-03-22T09:58:24.250138+0000","172.21.15.153:0/405164318":"2024-03-22T09:58:24.250138+0000","172.21.15.153:6800/1143522828":"2024-03-22T09:58:24.250138+0000","172.21.15.153:0/3456116624":"2024-03-22T09:56:43.480866+0000","172.21.15.153:6801/3178125243":"2024-03-22T09:56:43.480866+0000","172.21.15.153:0/2407536999":"2024-03-22T09:56:00.849533+0000","172.21.15.153:6800/3178125243":"2024-03-22T09:56:43.480866+0000","172.21.15.153:0/1163205431":"2024-03-22T09:56:43.480866+0000","172.21.15.153:0/623594592":"2024-03-22T09:56:00.849533+0000","172.21.15.153:6801/1143522828":"2024-03-22T09:58:24.250138+0000","172.21.15.153:6800/270837948":"2024-03-22T09:56:00.849533+0000","172.21.15.153:6801/270837948":"2024-03-22T09:56:00.849533+0000","172.21.15.153:0/816489762":"2024-03-22T09:56:43.480866+0000","172.21.15.153:0/3435678189":"2024-03-22T09:56:00.849533+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T10:03:01.088 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:01.087+0000 7f26acc00700 1 -- 172.21.15.153:0/1458905751 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f268c06c4e0 msgr2=0x7f268c06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:03:01.088 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:01.087+0000 7f26acc00700 1 --2- 172.21.15.153:0/1458905751 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f268c06c4e0 0x7f268c06e9a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f26a819e290 tx=0x7f269401a040 comp rx=0 tx=0).stop 2024-03-21T10:03:01.088 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:01.087+0000 7f26acc00700 1 -- 172.21.15.153:0/1458905751 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26a81071d0 msgr2=0x7f26a819ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:03:01.088 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:01.087+0000 7f26acc00700 1 --2- 172.21.15.153:0/1458905751 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26a81071d0 0x7f26a819ccb0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f269800d8d0 tx=0x7f269800dc90 comp rx=0 tx=0).stop 2024-03-21T10:03:01.088 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:01.087+0000 7f26acc00700 1 -- 172.21.15.153:0/1458905751 shutdown_connections 2024-03-21T10:03:01.088 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:01.087+0000 7f26acc00700 1 --2- 172.21.15.153:0/1458905751 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f268c06c4e0 0x7f268c06e9a0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:01.088 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:01.087+0000 7f26acc00700 1 --2- 172.21.15.153:0/1458905751 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26a81071d0 0x7f26a819ccb0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:01.088 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:01.087+0000 7f26acc00700 1 --2- 172.21.15.153:0/1458905751 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26a810d280 0x7f26a819d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:01.088 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:01.087+0000 7f26acc00700 1 -- 172.21.15.153:0/1458905751 >> 172.21.15.153:0/1458905751 conn(0x7f26a80769e0 msgr2=0x7f26a8078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:03:01.088 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:01.087+0000 7f26acc00700 1 -- 172.21.15.153:0/1458905751 shutdown_connections 2024-03-21T10:03:01.089 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:01.087+0000 7f26acc00700 1 -- 172.21.15.153:0/1458905751 wait complete. 2024-03-21T10:03:01.691 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:03:01 smithi153 ceph-mon[29117]: Detected new or changed devices on smithi174 2024-03-21T10:03:01.692 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:03:01 smithi153 ceph-mon[29117]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:03:01.692 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:03:01 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:03:01.692 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:03:01 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:03:01.692 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:03:01 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:03:01.692 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:03:01 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:03:01.692 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:03:01 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:03:01.692 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:03:01 smithi153 ceph-mon[29117]: Adjusting osd_memory_target on smithi174 to 4971M 2024-03-21T10:03:01.692 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:03:01 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:03:01.692 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:03:01 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:03:01.692 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:03:01 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:03:01.692 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:03:01 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:03:01.692 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:03:01 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:03:01.692 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:03:01 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:03:01.692 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:03:01 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:03:01.693 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:03:01 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:03:01.693 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:03:01 smithi153 ceph-mon[29117]: from='client.? 172.21.15.153:0/1458905751' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T10:03:01.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:03:01 smithi174 ceph-mon[37496]: Detected new or changed devices on smithi174 2024-03-21T10:03:01.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:03:01 smithi174 ceph-mon[37496]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:03:01.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:03:01 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:03:01.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:03:01 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:03:01.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:03:01 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:03:01.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:03:01 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:03:01.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:03:01 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:03:01.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:03:01 smithi174 ceph-mon[37496]: Adjusting osd_memory_target on smithi174 to 4971M 2024-03-21T10:03:01.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:03:01 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:03:01.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:03:01 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:03:01.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:03:01 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:03:01.703 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:03:01 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:03:01.703 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:03:01 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:03:01.703 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:03:01 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:03:01.703 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:03:01 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:03:01.703 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:03:01 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:03:01.703 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:03:01 smithi174 ceph-mon[37496]: from='client.? 172.21.15.153:0/1458905751' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T10:03:01.746 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-21T10:01:18.943639+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-21T10:03:01.746 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 070ed3e4-e769-11ee-95cd-87774f69a715 -- ceph osd pool get .mgr pg_num 2024-03-21T10:03:02.157 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi153/config 2024-03-21T10:03:03.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:03:03 smithi153 ceph-mon[29117]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:03:03.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:03:03 smithi174 ceph-mon[37496]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:03:04.075 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:04.074+0000 7fb164600700 1 -- 172.21.15.153:0/1441997540 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb15c1039b0 msgr2=0x7fb15c107a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:03:04.075 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:04.074+0000 7fb164600700 1 --2- 172.21.15.153:0/1441997540 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb15c1039b0 0x7fb15c107a00 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb148009910 tx=0x7fb148009c20 comp rx=0 tx=0).stop 2024-03-21T10:03:04.075 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:04.074+0000 7fb164600700 1 -- 172.21.15.153:0/1441997540 shutdown_connections 2024-03-21T10:03:04.075 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:04.074+0000 7fb164600700 1 --2- 172.21.15.153:0/1441997540 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb15c1039b0 0x7fb15c107a00 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:04.075 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:04.074+0000 7fb164600700 1 --2- 172.21.15.153:0/1441997540 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb15c103000 0x7fb15c1033e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:04.075 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:04.074+0000 7fb164600700 1 -- 172.21.15.153:0/1441997540 >> 172.21.15.153:0/1441997540 conn(0x7fb15c0fe8d0 msgr2=0x7fb15c100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:03:04.075 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:04.074+0000 7fb164600700 1 -- 172.21.15.153:0/1441997540 shutdown_connections 2024-03-21T10:03:04.075 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:04.074+0000 7fb164600700 1 -- 172.21.15.153:0/1441997540 wait complete. 2024-03-21T10:03:04.076 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:04.076+0000 7fb164600700 1 Processor -- start 2024-03-21T10:03:04.076 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:04.076+0000 7fb164600700 1 -- start start 2024-03-21T10:03:04.078 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:04.077+0000 7fb164600700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb15c103000 0x7fb15c19e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:03:04.078 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:04.077+0000 7fb164600700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb15c1039b0 0x7fb15c19ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:03:04.078 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:04.077+0000 7fb164600700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb15c19f2e0 con 0x7fb15c1039b0 2024-03-21T10:03:04.078 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:04.077+0000 7fb164600700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb15c198790 con 0x7fb15c103000 2024-03-21T10:03:04.078 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:04.077+0000 7fb161200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb15c1039b0 0x7fb15c19ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:03:04.078 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:04.077+0000 7fb161200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb15c1039b0 0x7fb15c19ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:55276/0 (socket says 172.21.15.153:55276) 2024-03-21T10:03:04.078 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:04.077+0000 7fb161200700 1 -- 172.21.15.153:0/1035227746 learned_addr learned my addr 172.21.15.153:0/1035227746 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:03:04.078 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:04.077+0000 7fb161c00700 1 --2- 172.21.15.153:0/1035227746 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb15c103000 0x7fb15c19e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:03:04.079 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:04.077+0000 7fb161200700 1 -- 172.21.15.153:0/1035227746 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb15c103000 msgr2=0x7fb15c19e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:03:04.079 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:04.077+0000 7fb161200700 1 --2- 172.21.15.153:0/1035227746 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb15c103000 0x7fb15c19e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:04.079 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:04.077+0000 7fb161200700 1 -- 172.21.15.153:0/1035227746 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1480095f0 con 0x7fb15c1039b0 2024-03-21T10:03:04.079 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:04.078+0000 7fb161200700 1 --2- 172.21.15.153:0/1035227746 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb15c1039b0 0x7fb15c19ec50 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb14800bda0 tx=0x7fb14800be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:03:04.080 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:04.078+0000 7fb15a000700 1 -- 172.21.15.153:0/1035227746 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb14801d070 con 0x7fb15c1039b0 2024-03-21T10:03:04.080 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:04.078+0000 7fb15a000700 1 -- 172.21.15.153:0/1035227746 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fb14800f460 con 0x7fb15c1039b0 2024-03-21T10:03:04.080 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:04.078+0000 7fb15a000700 1 -- 172.21.15.153:0/1035227746 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb148017630 con 0x7fb15c1039b0 2024-03-21T10:03:04.080 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:04.078+0000 7fb164600700 1 -- 172.21.15.153:0/1035227746 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb15c198a10 con 0x7fb15c1039b0 2024-03-21T10:03:04.080 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:04.078+0000 7fb164600700 1 -- 172.21.15.153:0/1035227746 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb15c198f00 con 0x7fb15c1039b0 2024-03-21T10:03:04.080 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:04.079+0000 7fb164600700 1 -- 172.21.15.153:0/1035227746 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb15c10b310 con 0x7fb15c1039b0 2024-03-21T10:03:04.081 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:04.081+0000 7fb15a000700 1 -- 172.21.15.153:0/1035227746 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fb14800fad0 con 0x7fb15c1039b0 2024-03-21T10:03:04.082 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:04.081+0000 7fb15a000700 1 --2- 172.21.15.153:0/1035227746 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fb14406c4e0 0x7fb14406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:03:04.082 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:04.082+0000 7fb15a000700 1 -- 172.21.15.153:0/1035227746 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fb14808c920 con 0x7fb15c1039b0 2024-03-21T10:03:04.082 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:04.082+0000 7fb161c00700 1 --2- 172.21.15.153:0/1035227746 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fb14406c4e0 0x7fb14406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:03:04.083 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:04.082+0000 7fb161c00700 1 --2- 172.21.15.153:0/1035227746 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fb14406c4e0 0x7fb14406e9a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb150005950 tx=0x7fb1500058e0 comp rx=0 tx=0).ready entity=mgr.14221 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:03:04.087 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:04.086+0000 7fb15a000700 1 -- 172.21.15.153:0/1035227746 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fb14805aeb0 con 0x7fb15c1039b0 2024-03-21T10:03:04.300 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:04.299+0000 7fb164600700 1 -- 172.21.15.153:0/1035227746 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb15c199230 con 0x7fb15c1039b0 2024-03-21T10:03:04.301 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:04.300+0000 7fb15a000700 1 -- 172.21.15.153:0/1035227746 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v41) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb14805aa40 con 0x7fb15c1039b0 2024-03-21T10:03:04.301 INFO:teuthology.orchestra.run.smithi153.stdout:pg_num: 1 2024-03-21T10:03:04.304 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:04.303+0000 7fb164600700 1 -- 172.21.15.153:0/1035227746 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fb14406c4e0 msgr2=0x7fb14406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:03:04.304 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:04.303+0000 7fb164600700 1 --2- 172.21.15.153:0/1035227746 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fb14406c4e0 0x7fb14406e9a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb150005950 tx=0x7fb1500058e0 comp rx=0 tx=0).stop 2024-03-21T10:03:04.304 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:04.303+0000 7fb164600700 1 -- 172.21.15.153:0/1035227746 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb15c1039b0 msgr2=0x7fb15c19ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:03:04.304 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:04.303+0000 7fb164600700 1 --2- 172.21.15.153:0/1035227746 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb15c1039b0 0x7fb15c19ec50 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb14800bda0 tx=0x7fb14800be80 comp rx=0 tx=0).stop 2024-03-21T10:03:04.304 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:04.304+0000 7fb164600700 1 -- 172.21.15.153:0/1035227746 shutdown_connections 2024-03-21T10:03:04.305 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:04.304+0000 7fb164600700 1 --2- 172.21.15.153:0/1035227746 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fb14406c4e0 0x7fb14406e9a0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:04.305 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:04.304+0000 7fb164600700 1 --2- 172.21.15.153:0/1035227746 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb15c103000 0x7fb15c19e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:04.305 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:04.304+0000 7fb164600700 1 --2- 172.21.15.153:0/1035227746 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb15c1039b0 0x7fb15c19ec50 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:04.305 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:04.304+0000 7fb164600700 1 -- 172.21.15.153:0/1035227746 >> 172.21.15.153:0/1035227746 conn(0x7fb15c0fe8d0 msgr2=0x7fb15c107270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:03:04.305 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:04.304+0000 7fb164600700 1 -- 172.21.15.153:0/1035227746 shutdown_connections 2024-03-21T10:03:04.305 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:04.304+0000 7fb164600700 1 -- 172.21.15.153:0/1035227746 wait complete. 2024-03-21T10:03:04.969 INFO:tasks.cephadm:Setting up client nodes... 2024-03-21T10:03:04.969 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 070ed3e4-e769-11ee-95cd-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-03-21T10:03:05.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:03:04 smithi153 ceph-mon[29117]: from='client.? 172.21.15.153:0/1035227746' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T10:03:05.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:03:04 smithi153 ceph-mon[29117]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:03:05.370 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi153/config 2024-03-21T10:03:05.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:03:04 smithi174 ceph-mon[37496]: from='client.? 172.21.15.153:0/1035227746' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T10:03:05.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:03:04 smithi174 ceph-mon[37496]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:03:07.427 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:07.425+0000 7fca3d600700 1 -- 172.21.15.153:0/1986593047 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca38103ff0 msgr2=0x7fca38104450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:03:07.427 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:07.425+0000 7fca3d600700 1 --2- 172.21.15.153:0/1986593047 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca38103ff0 0x7fca38104450 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fca24009910 tx=0x7fca24009c20 comp rx=0 tx=0).stop 2024-03-21T10:03:07.428 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:07.425+0000 7fca3d600700 1 -- 172.21.15.153:0/1986593047 shutdown_connections 2024-03-21T10:03:07.428 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:07.425+0000 7fca3d600700 1 --2- 172.21.15.153:0/1986593047 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca38103ff0 0x7fca38104450 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:07.428 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:07.425+0000 7fca3d600700 1 --2- 172.21.15.153:0/1986593047 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fca38102df0 0x7fca38103210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:07.428 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:07.425+0000 7fca3d600700 1 -- 172.21.15.153:0/1986593047 >> 172.21.15.153:0/1986593047 conn(0x7fca380fe390 msgr2=0x7fca381007d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:03:07.429 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:07.425+0000 7fca3d600700 1 -- 172.21.15.153:0/1986593047 shutdown_connections 2024-03-21T10:03:07.429 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:07.425+0000 7fca3d600700 1 -- 172.21.15.153:0/1986593047 wait complete. 2024-03-21T10:03:07.429 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:07.427+0000 7fca3d600700 1 Processor -- start 2024-03-21T10:03:07.429 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:07.427+0000 7fca3d600700 1 -- start start 2024-03-21T10:03:07.429 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:07.427+0000 7fca3d600700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca38102df0 0x7fca381985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:03:07.429 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:07.427+0000 7fca3d600700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fca38103ff0 0x7fca38198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:03:07.429 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:07.427+0000 7fca3d600700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca38199100 con 0x7fca38102df0 2024-03-21T10:03:07.429 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:07.427+0000 7fca3d600700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca38199240 con 0x7fca38103ff0 2024-03-21T10:03:07.429 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:07.427+0000 7fca36a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca38102df0 0x7fca381985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:03:07.429 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:07.427+0000 7fca36a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca38102df0 0x7fca381985a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:55290/0 (socket says 172.21.15.153:55290) 2024-03-21T10:03:07.429 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:07.427+0000 7fca36a00700 1 -- 172.21.15.153:0/678197290 learned_addr learned my addr 172.21.15.153:0/678197290 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:03:07.429 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:07.428+0000 7fca36000700 1 --2- 172.21.15.153:0/678197290 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fca38103ff0 0x7fca38198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:03:07.429 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:07.428+0000 7fca36a00700 1 -- 172.21.15.153:0/678197290 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fca38103ff0 msgr2=0x7fca38198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:03:07.429 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:07.428+0000 7fca36a00700 1 --2- 172.21.15.153:0/678197290 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fca38103ff0 0x7fca38198ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:07.430 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:07.428+0000 7fca36a00700 1 -- 172.21.15.153:0/678197290 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca240095f0 con 0x7fca38102df0 2024-03-21T10:03:07.430 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:07.428+0000 7fca36a00700 1 --2- 172.21.15.153:0/678197290 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca38102df0 0x7fca381985a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fca2800c8f0 tx=0x7fca2800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:03:07.430 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:07.428+0000 7fca2f400700 1 -- 172.21.15.153:0/678197290 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fca28007a70 con 0x7fca38102df0 2024-03-21T10:03:07.431 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:07.428+0000 7fca2f400700 1 -- 172.21.15.153:0/678197290 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fca28007bd0 con 0x7fca38102df0 2024-03-21T10:03:07.431 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:07.428+0000 7fca2f400700 1 -- 172.21.15.153:0/678197290 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fca28018630 con 0x7fca38102df0 2024-03-21T10:03:07.431 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:07.428+0000 7fca3d600700 1 -- 172.21.15.153:0/678197290 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca3819dcf0 con 0x7fca38102df0 2024-03-21T10:03:07.431 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:07.429+0000 7fca3d600700 1 -- 172.21.15.153:0/678197290 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca3819e1e0 con 0x7fca38102df0 2024-03-21T10:03:07.431 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:07.430+0000 7fca3d600700 1 -- 172.21.15.153:0/678197290 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca3810b9c0 con 0x7fca38102df0 2024-03-21T10:03:07.431 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:07.431+0000 7fca2f400700 1 -- 172.21.15.153:0/678197290 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fca2800f450 con 0x7fca38102df0 2024-03-21T10:03:07.432 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:07.432+0000 7fca2f400700 1 --2- 172.21.15.153:0/678197290 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fca1c06c4e0 0x7fca1c06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:03:07.432 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:07.432+0000 7fca2f400700 1 -- 172.21.15.153:0/678197290 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fca2808b490 con 0x7fca38102df0 2024-03-21T10:03:07.432 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:07.432+0000 7fca36000700 1 --2- 172.21.15.153:0/678197290 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fca1c06c4e0 0x7fca1c06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:03:07.433 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:07.433+0000 7fca36000700 1 --2- 172.21.15.153:0/678197290 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fca1c06c4e0 0x7fca1c06e9a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fca24000c00 tx=0x7fca24005fb0 comp rx=0 tx=0).ready entity=mgr.14221 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:03:07.437 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:07.437+0000 7fca2f400700 1 -- 172.21.15.153:0/678197290 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fca28059ad0 con 0x7fca38102df0 2024-03-21T10:03:07.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:03:07 smithi153 ceph-mon[29117]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:03:07.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:03:07 smithi174 ceph-mon[37496]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:03:07.735 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:07.734+0000 7fca3d600700 1 -- 172.21.15.153:0/678197290 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7fca3819e8a0 con 0x7fca38102df0 2024-03-21T10:03:07.744 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:07.743+0000 7fca2f400700 1 -- 172.21.15.153:0/678197290 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v16) v1 ==== 170+0+59 (secure 0 0 0) 0x7fca2808b0f0 con 0x7fca38102df0 2024-03-21T10:03:07.744 INFO:teuthology.orchestra.run.smithi153.stdout:[client.0] 2024-03-21T10:03:07.744 INFO:teuthology.orchestra.run.smithi153.stdout: key = AQDbBfxldMXXKxAAOWnuLhGONEXig5EqyFO6kw== 2024-03-21T10:03:07.747 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:07.746+0000 7fca3d600700 1 -- 172.21.15.153:0/678197290 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fca1c06c4e0 msgr2=0x7fca1c06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:03:07.747 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:07.746+0000 7fca3d600700 1 --2- 172.21.15.153:0/678197290 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fca1c06c4e0 0x7fca1c06e9a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fca24000c00 tx=0x7fca24005fb0 comp rx=0 tx=0).stop 2024-03-21T10:03:07.747 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:07.746+0000 7fca3d600700 1 -- 172.21.15.153:0/678197290 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca38102df0 msgr2=0x7fca381985a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:03:07.747 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:07.746+0000 7fca3d600700 1 --2- 172.21.15.153:0/678197290 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca38102df0 0x7fca381985a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fca2800c8f0 tx=0x7fca2800ccb0 comp rx=0 tx=0).stop 2024-03-21T10:03:07.747 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:07.746+0000 7fca3d600700 1 -- 172.21.15.153:0/678197290 shutdown_connections 2024-03-21T10:03:07.747 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:07.746+0000 7fca3d600700 1 --2- 172.21.15.153:0/678197290 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fca1c06c4e0 0x7fca1c06e9a0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:07.747 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:07.746+0000 7fca3d600700 1 --2- 172.21.15.153:0/678197290 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca38102df0 0x7fca381985a0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:07.748 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:07.746+0000 7fca3d600700 1 --2- 172.21.15.153:0/678197290 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fca38103ff0 0x7fca38198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:07.748 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:07.746+0000 7fca3d600700 1 -- 172.21.15.153:0/678197290 >> 172.21.15.153:0/678197290 conn(0x7fca380fe390 msgr2=0x7fca38107220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:03:07.748 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:07.747+0000 7fca3d600700 1 -- 172.21.15.153:0/678197290 shutdown_connections 2024-03-21T10:03:07.748 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:07.747+0000 7fca3d600700 1 -- 172.21.15.153:0/678197290 wait complete. 2024-03-21T10:03:08.423 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-03-21T10:03:08.423 DEBUG:teuthology.orchestra.run.smithi153:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-03-21T10:03:08.423 DEBUG:teuthology.orchestra.run.smithi153:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-03-21T10:03:08.575 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 070ed3e4-e769-11ee-95cd-87774f69a715 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-03-21T10:03:08.664 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:03:08 smithi174 ceph-mon[37496]: from='client.? 172.21.15.153:0/678197290' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-21T10:03:08.664 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:03:08 smithi174 ceph-mon[37496]: from='client.? 172.21.15.153:0/678197290' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-21T10:03:08.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:03:08 smithi153 ceph-mon[29117]: from='client.? 172.21.15.153:0/678197290' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-21T10:03:08.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:03:08 smithi153 ceph-mon[29117]: from='client.? 172.21.15.153:0/678197290' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-21T10:03:08.981 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi174/config 2024-03-21T10:03:09.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:03:09 smithi174 ceph-mon[37496]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:03:09.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:03:09 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:03:09.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:03:09 smithi153 ceph-mon[29117]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:03:09.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:03:09 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:03:11.200 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:03:11.198+0000 7f1d19600700 1 -- 172.21.15.174:0/2746024373 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1d14103e20 msgr2=0x7f1d14104280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:03:11.200 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:03:11.198+0000 7f1d19600700 1 --2- 172.21.15.174:0/2746024373 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1d14103e20 0x7f1d14104280 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1d00009910 tx=0x7f1d00009c20 comp rx=0 tx=0).stop 2024-03-21T10:03:11.200 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:03:11.198+0000 7f1d19600700 1 -- 172.21.15.174:0/2746024373 shutdown_connections 2024-03-21T10:03:11.200 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:03:11.198+0000 7f1d19600700 1 --2- 172.21.15.174:0/2746024373 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1d14103e20 0x7f1d14104280 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:11.200 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:03:11.198+0000 7f1d19600700 1 --2- 172.21.15.174:0/2746024373 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1d14102c20 0x7f1d14103040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:11.200 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:03:11.198+0000 7f1d19600700 1 -- 172.21.15.174:0/2746024373 >> 172.21.15.174:0/2746024373 conn(0x7f1d140fe1a0 msgr2=0x7f1d14100600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:03:11.200 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:03:11.198+0000 7f1d19600700 1 -- 172.21.15.174:0/2746024373 shutdown_connections 2024-03-21T10:03:11.200 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:03:11.198+0000 7f1d19600700 1 -- 172.21.15.174:0/2746024373 wait complete. 2024-03-21T10:03:11.202 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:03:11.199+0000 7f1d19600700 1 Processor -- start 2024-03-21T10:03:11.202 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:03:11.200+0000 7f1d19600700 1 -- start start 2024-03-21T10:03:11.203 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:03:11.200+0000 7f1d19600700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1d14102c20 0x7f1d141983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:03:11.203 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:03:11.200+0000 7f1d19600700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1d14103e20 0x7f1d141988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:03:11.203 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:03:11.200+0000 7f1d19600700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d14198f00 con 0x7f1d14102c20 2024-03-21T10:03:11.203 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:03:11.200+0000 7f1d19600700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d14199040 con 0x7f1d14103e20 2024-03-21T10:03:11.203 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:03:11.200+0000 7f1d13400700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1d14103e20 0x7f1d141988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:03:11.203 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:03:11.200+0000 7f1d13400700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1d14103e20 0x7f1d141988e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:34060/0 (socket says 172.21.15.174:34060) 2024-03-21T10:03:11.203 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:03:11.200+0000 7f1d13400700 1 -- 172.21.15.174:0/2348612279 learned_addr learned my addr 172.21.15.174:0/2348612279 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-03-21T10:03:11.203 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:03:11.200+0000 7f1d13e00700 1 --2- 172.21.15.174:0/2348612279 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1d14102c20 0x7f1d141983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:03:11.203 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:03:11.201+0000 7f1d13400700 1 -- 172.21.15.174:0/2348612279 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1d14102c20 msgr2=0x7f1d141983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:03:11.203 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:03:11.201+0000 7f1d13400700 1 --2- 172.21.15.174:0/2348612279 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1d14102c20 0x7f1d141983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:11.204 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:03:11.201+0000 7f1d13400700 1 -- 172.21.15.174:0/2348612279 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d000095f0 con 0x7f1d14103e20 2024-03-21T10:03:11.204 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:03:11.201+0000 7f1d13400700 1 --2- 172.21.15.174:0/2348612279 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1d14103e20 0x7f1d141988e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f1d0000b5c0 tx=0x7f1d0000be20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:03:11.204 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:03:11.201+0000 7f1d10c00700 1 -- 172.21.15.174:0/2348612279 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1d0001d070 con 0x7f1d14103e20 2024-03-21T10:03:11.204 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:03:11.201+0000 7f1d10c00700 1 -- 172.21.15.174:0/2348612279 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f1d00022470 con 0x7f1d14103e20 2024-03-21T10:03:11.204 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:03:11.201+0000 7f1d19600700 1 -- 172.21.15.174:0/2348612279 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d1419da90 con 0x7f1d14103e20 2024-03-21T10:03:11.204 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:03:11.201+0000 7f1d10c00700 1 -- 172.21.15.174:0/2348612279 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1d0000f690 con 0x7f1d14103e20 2024-03-21T10:03:11.204 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:03:11.202+0000 7f1d19600700 1 -- 172.21.15.174:0/2348612279 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d1419df80 con 0x7f1d14103e20 2024-03-21T10:03:11.205 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:03:11.203+0000 7f1d19600700 1 -- 172.21.15.174:0/2348612279 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d1410b7a0 con 0x7f1d14103e20 2024-03-21T10:03:11.205 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:03:11.203+0000 7f1d10c00700 1 -- 172.21.15.174:0/2348612279 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f1d00004b70 con 0x7f1d14103e20 2024-03-21T10:03:11.205 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:03:11.204+0000 7f1d10c00700 1 --2- 172.21.15.174:0/2348612279 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f1cf806c4e0 0x7f1cf806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:03:11.205 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:03:11.204+0000 7f1d10c00700 1 -- 172.21.15.174:0/2348612279 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f1d0008cce0 con 0x7f1d14103e20 2024-03-21T10:03:11.205 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:03:11.205+0000 7f1d13e00700 1 --2- 172.21.15.174:0/2348612279 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f1cf806c4e0 0x7f1cf806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:03:11.206 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:03:11.206+0000 7f1d13e00700 1 --2- 172.21.15.174:0/2348612279 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f1cf806c4e0 0x7f1cf806e9a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1d04005fd0 tx=0x7f1d0400a560 comp rx=0 tx=0).ready entity=mgr.14221 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:03:11.210 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:03:11.209+0000 7f1d10c00700 1 -- 172.21.15.174:0/2348612279 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f1d0005b2a0 con 0x7f1d14103e20 2024-03-21T10:03:11.504 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:03:11.503+0000 7f1d19600700 1 -- 172.21.15.174:0/2348612279 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f1d1419e690 con 0x7f1d14103e20 2024-03-21T10:03:11.515 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:03:11.514+0000 7f1d10c00700 1 -- 172.21.15.174:0/2348612279 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v17) v1 ==== 170+0+59 (secure 0 0 0) 0x7f1d00027050 con 0x7f1d14103e20 2024-03-21T10:03:11.515 INFO:teuthology.orchestra.run.smithi174.stdout:[client.1] 2024-03-21T10:03:11.515 INFO:teuthology.orchestra.run.smithi174.stdout: key = AQDfBfxl+1kmHhAAmZQPz7JLhkcGy/J8pkl+eQ== 2024-03-21T10:03:11.518 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:03:11.517+0000 7f1d19600700 1 -- 172.21.15.174:0/2348612279 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f1cf806c4e0 msgr2=0x7f1cf806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:03:11.518 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:03:11.517+0000 7f1d19600700 1 --2- 172.21.15.174:0/2348612279 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f1cf806c4e0 0x7f1cf806e9a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1d04005fd0 tx=0x7f1d0400a560 comp rx=0 tx=0).stop 2024-03-21T10:03:11.518 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:03:11.517+0000 7f1d19600700 1 -- 172.21.15.174:0/2348612279 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1d14103e20 msgr2=0x7f1d141988e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:03:11.518 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:03:11.517+0000 7f1d19600700 1 --2- 172.21.15.174:0/2348612279 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1d14103e20 0x7f1d141988e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f1d0000b5c0 tx=0x7f1d0000be20 comp rx=0 tx=0).stop 2024-03-21T10:03:11.519 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:03:11.517+0000 7f1d19600700 1 -- 172.21.15.174:0/2348612279 shutdown_connections 2024-03-21T10:03:11.519 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:03:11.517+0000 7f1d19600700 1 --2- 172.21.15.174:0/2348612279 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f1cf806c4e0 0x7f1cf806e9a0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:11.519 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:03:11.517+0000 7f1d19600700 1 --2- 172.21.15.174:0/2348612279 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1d14102c20 0x7f1d141983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:11.519 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:03:11.517+0000 7f1d19600700 1 --2- 172.21.15.174:0/2348612279 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1d14103e20 0x7f1d141988e0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:11.519 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:03:11.517+0000 7f1d19600700 1 -- 172.21.15.174:0/2348612279 >> 172.21.15.174:0/2348612279 conn(0x7f1d140fe1a0 msgr2=0x7f1d14107050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:03:11.519 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:03:11.517+0000 7f1d19600700 1 -- 172.21.15.174:0/2348612279 shutdown_connections 2024-03-21T10:03:11.519 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:03:11.518+0000 7f1d19600700 1 -- 172.21.15.174:0/2348612279 wait complete. 2024-03-21T10:03:11.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:03:11 smithi174 ceph-mon[37496]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:03:11.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:03:11 smithi153 ceph-mon[29117]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:03:12.333 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-03-21T10:03:12.334 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-03-21T10:03:12.334 DEBUG:teuthology.orchestra.run.smithi174:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-03-21T10:03:12.489 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-03-21T10:03:12.489 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-03-21T10:03:12.489 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 070ed3e4-e769-11ee-95cd-87774f69a715 -- ceph mgr dump --format=json 2024-03-21T10:03:12.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:03:12 smithi174 ceph-mon[37496]: from='client.? 172.21.15.174:0/2348612279' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-21T10:03:12.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:03:12 smithi174 ceph-mon[37496]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-21T10:03:12.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:03:12 smithi174 ceph-mon[37496]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-21T10:03:12.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:03:12 smithi153 ceph-mon[29117]: from='client.? 172.21.15.174:0/2348612279' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-21T10:03:12.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:03:12 smithi153 ceph-mon[29117]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-21T10:03:12.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:03:12 smithi153 ceph-mon[29117]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-21T10:03:12.893 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi153/config 2024-03-21T10:03:13.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:03:13 smithi153 ceph-mon[29117]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:03:13.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:03:13 smithi174 ceph-mon[37496]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:03:14.936 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:14.934+0000 7f9afc400700 1 -- 172.21.15.153:0/7883455 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9af4100da0 msgr2=0x7f9af4101180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:03:14.937 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:14.934+0000 7f9afc400700 1 --2- 172.21.15.153:0/7883455 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9af4100da0 0x7f9af4101180 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f9ae8009910 tx=0x7f9ae8009c20 comp rx=0 tx=0).stop 2024-03-21T10:03:14.937 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:14.934+0000 7f9afc400700 1 -- 172.21.15.153:0/7883455 shutdown_connections 2024-03-21T10:03:14.937 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:14.934+0000 7f9afc400700 1 --2- 172.21.15.153:0/7883455 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9af4101750 0x7f9af41057a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:14.937 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:14.934+0000 7f9afc400700 1 --2- 172.21.15.153:0/7883455 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9af4100da0 0x7f9af4101180 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:14.937 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:14.934+0000 7f9afc400700 1 -- 172.21.15.153:0/7883455 >> 172.21.15.153:0/7883455 conn(0x7f9af40fc630 msgr2=0x7f9af40fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:03:14.937 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:14.935+0000 7f9afc400700 1 -- 172.21.15.153:0/7883455 shutdown_connections 2024-03-21T10:03:14.937 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:14.935+0000 7f9afc400700 1 -- 172.21.15.153:0/7883455 wait complete. 2024-03-21T10:03:14.938 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:14.936+0000 7f9afc400700 1 Processor -- start 2024-03-21T10:03:14.938 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:14.936+0000 7f9afc400700 1 -- start start 2024-03-21T10:03:14.938 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:14.936+0000 7f9afc400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9af4100da0 0x7f9af419c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:03:14.938 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:14.936+0000 7f9afc400700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9af4101750 0x7f9af419cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:03:14.939 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:14.936+0000 7f9afc400700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9af419d170 con 0x7f9af4100da0 2024-03-21T10:03:14.939 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:14.936+0000 7f9afc400700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9af4196620 con 0x7f9af4101750 2024-03-21T10:03:14.939 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:14.936+0000 7f9af9a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9af4100da0 0x7f9af419c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:03:14.939 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:14.936+0000 7f9af9a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9af4100da0 0x7f9af419c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:56736/0 (socket says 172.21.15.153:56736) 2024-03-21T10:03:14.939 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:14.936+0000 7f9af9a00700 1 -- 172.21.15.153:0/3768484081 learned_addr learned my addr 172.21.15.153:0/3768484081 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:03:14.939 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:14.937+0000 7f9af9000700 1 --2- 172.21.15.153:0/3768484081 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9af4101750 0x7f9af419cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:03:14.939 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:14.937+0000 7f9af9a00700 1 -- 172.21.15.153:0/3768484081 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9af4101750 msgr2=0x7f9af419cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:03:14.939 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:14.937+0000 7f9af9a00700 1 --2- 172.21.15.153:0/3768484081 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9af4101750 0x7f9af419cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:14.939 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:14.937+0000 7f9af9a00700 1 -- 172.21.15.153:0/3768484081 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ae80095f0 con 0x7f9af4100da0 2024-03-21T10:03:14.939 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:14.937+0000 7f9af9a00700 1 --2- 172.21.15.153:0/3768484081 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9af4100da0 0x7f9af419c5a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f9ae8006010 tx=0x7f9ae800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:03:14.940 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:14.937+0000 7f9af2000700 1 -- 172.21.15.153:0/3768484081 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9ae801d070 con 0x7f9af4100da0 2024-03-21T10:03:14.940 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:14.937+0000 7f9afc400700 1 -- 172.21.15.153:0/3768484081 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9af41968a0 con 0x7f9af4100da0 2024-03-21T10:03:14.940 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:14.937+0000 7f9af2000700 1 -- 172.21.15.153:0/3768484081 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f9ae8004f50 con 0x7f9af4100da0 2024-03-21T10:03:14.940 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:14.938+0000 7f9afc400700 1 -- 172.21.15.153:0/3768484081 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9af4196d90 con 0x7f9af4100da0 2024-03-21T10:03:14.940 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:14.938+0000 7f9af2000700 1 -- 172.21.15.153:0/3768484081 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9ae802b440 con 0x7f9af4100da0 2024-03-21T10:03:14.940 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:14.939+0000 7f9afc400700 1 -- 172.21.15.153:0/3768484081 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9af4109140 con 0x7f9af4100da0 2024-03-21T10:03:14.941 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:14.940+0000 7f9af2000700 1 -- 172.21.15.153:0/3768484081 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f9ae800f460 con 0x7f9af4100da0 2024-03-21T10:03:14.942 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:14.940+0000 7f9af2000700 1 --2- 172.21.15.153:0/3768484081 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f9adc06c4e0 0x7f9adc06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:03:14.942 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:14.941+0000 7f9af2000700 1 -- 172.21.15.153:0/3768484081 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f9ae808c6a0 con 0x7f9af4100da0 2024-03-21T10:03:14.942 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:14.941+0000 7f9af9000700 1 --2- 172.21.15.153:0/3768484081 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f9adc06c4e0 0x7f9adc06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:03:14.946 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:14.941+0000 7f9af9000700 1 --2- 172.21.15.153:0/3768484081 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f9adc06c4e0 0x7f9adc06e9a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9ae0005950 tx=0x7f9ae00058e0 comp rx=0 tx=0).ready entity=mgr.14221 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:03:14.955 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:14.954+0000 7f9af2000700 1 -- 172.21.15.153:0/3768484081 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f9ae805ad80 con 0x7f9af4100da0 2024-03-21T10:03:15.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:03:15 smithi153 ceph-mon[29117]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:03:15.232 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:15.230+0000 7f9afc400700 1 -- 172.21.15.153:0/3768484081 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f9af404ea90 con 0x7f9af4100da0 2024-03-21T10:03:15.238 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:15.236+0000 7f9af2000700 1 -- 172.21.15.153:0/3768484081 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v19) v1 ==== 74+0+173048 (secure 0 0 0) 0x7f9ae8026090 con 0x7f9af4100da0 2024-03-21T10:03:15.242 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:03:15.244 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:15.242+0000 7f9afc400700 1 -- 172.21.15.153:0/3768484081 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f9adc06c4e0 msgr2=0x7f9adc06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:03:15.244 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:15.242+0000 7f9afc400700 1 --2- 172.21.15.153:0/3768484081 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f9adc06c4e0 0x7f9adc06e9a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9ae0005950 tx=0x7f9ae00058e0 comp rx=0 tx=0).stop 2024-03-21T10:03:15.244 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:15.242+0000 7f9afc400700 1 -- 172.21.15.153:0/3768484081 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9af4100da0 msgr2=0x7f9af419c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:03:15.244 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:15.242+0000 7f9afc400700 1 --2- 172.21.15.153:0/3768484081 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9af4100da0 0x7f9af419c5a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f9ae8006010 tx=0x7f9ae800be20 comp rx=0 tx=0).stop 2024-03-21T10:03:15.244 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:15.242+0000 7f9afc400700 1 -- 172.21.15.153:0/3768484081 shutdown_connections 2024-03-21T10:03:15.244 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:15.242+0000 7f9afc400700 1 --2- 172.21.15.153:0/3768484081 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f9adc06c4e0 0x7f9adc06e9a0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:15.244 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:15.242+0000 7f9afc400700 1 --2- 172.21.15.153:0/3768484081 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9af4100da0 0x7f9af419c5a0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:15.244 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:15.242+0000 7f9afc400700 1 --2- 172.21.15.153:0/3768484081 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9af4101750 0x7f9af419cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:15.245 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:15.242+0000 7f9afc400700 1 -- 172.21.15.153:0/3768484081 >> 172.21.15.153:0/3768484081 conn(0x7f9af40fc630 msgr2=0x7f9af40fdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:03:15.245 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:15.242+0000 7f9afc400700 1 -- 172.21.15.153:0/3768484081 shutdown_connections 2024-03-21T10:03:15.245 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:15.242+0000 7f9afc400700 1 -- 172.21.15.153:0/3768484081 wait complete. 2024-03-21T10:03:15.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:03:15 smithi174 ceph-mon[37496]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:03:15.987 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":19,"active_gid":14221,"active_name":"smithi153.tluxni","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":1639929315},{"type":"v1","addr":"172.21.15.153:6801","nonce":1639929315}]},"active_addr":"172.21.15.153:6801/1639929315","active_change":"2024-03-21T09:58:24.250494+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":14246,"name":"smithi174.qpcwat","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.2","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.2","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.153:8443/","prometheus":"http://172.21.15.153:9283/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":5,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.153:0","nonce":2801644471}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.153:0","nonce":2969012027}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.153:0","nonce":4237751158}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.153:0","nonce":2431412876}]}]} 2024-03-21T10:03:15.992 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-03-21T10:03:15.992 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-03-21T10:03:15.992 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 070ed3e4-e769-11ee-95cd-87774f69a715 -- ceph osd dump --format=json 2024-03-21T10:03:16.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:03:16 smithi153 ceph-mon[29117]: from='client.? 172.21.15.153:0/3768484081' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-21T10:03:16.406 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi153/config 2024-03-21T10:03:16.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:03:16 smithi174 ceph-mon[37496]: from='client.? 172.21.15.153:0/3768484081' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-21T10:03:17.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:03:17 smithi153 ceph-mon[29117]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:03:17.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:03:17 smithi174 ceph-mon[37496]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:03:18.263 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:18.259+0000 7f352ce00700 1 -- 172.21.15.153:0/635126394 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f352810d280 msgr2=0x7f352810d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:03:18.263 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:18.259+0000 7f352ce00700 1 --2- 172.21.15.153:0/635126394 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f352810d280 0x7f352810d660 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3518009910 tx=0x7f3518009c20 comp rx=0 tx=0).stop 2024-03-21T10:03:18.263 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:18.259+0000 7f352ce00700 1 -- 172.21.15.153:0/635126394 shutdown_connections 2024-03-21T10:03:18.263 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:18.259+0000 7f352ce00700 1 --2- 172.21.15.153:0/635126394 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f35281071d0 0x7f3528107650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:18.263 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:18.259+0000 7f352ce00700 1 --2- 172.21.15.153:0/635126394 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f352810d280 0x7f352810d660 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:18.263 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:18.259+0000 7f352ce00700 1 -- 172.21.15.153:0/635126394 >> 172.21.15.153:0/635126394 conn(0x7f35280769e0 msgr2=0x7f3528078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:03:18.263 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:18.259+0000 7f352ce00700 1 -- 172.21.15.153:0/635126394 shutdown_connections 2024-03-21T10:03:18.263 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:18.259+0000 7f352ce00700 1 -- 172.21.15.153:0/635126394 wait complete. 2024-03-21T10:03:18.263 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:18.260+0000 7f352ce00700 1 Processor -- start 2024-03-21T10:03:18.263 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:18.260+0000 7f352ce00700 1 -- start start 2024-03-21T10:03:18.263 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:18.260+0000 7f352ce00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f35281071d0 0x7f352819ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:03:18.264 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:18.260+0000 7f352ce00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f352810d280 0x7f352819d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:03:18.264 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:18.260+0000 7f352ce00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f352819d810 con 0x7f35281071d0 2024-03-21T10:03:18.264 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:18.260+0000 7f352ce00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f352819d980 con 0x7f352810d280 2024-03-21T10:03:18.264 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:18.261+0000 7f3525e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f35281071d0 0x7f352819ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:03:18.264 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:18.261+0000 7f3525e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f35281071d0 0x7f352819ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:56756/0 (socket says 172.21.15.153:56756) 2024-03-21T10:03:18.264 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:18.261+0000 7f3525e00700 1 -- 172.21.15.153:0/2280045623 learned_addr learned my addr 172.21.15.153:0/2280045623 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:03:18.264 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:18.261+0000 7f351d400700 1 --2- 172.21.15.153:0/2280045623 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f352810d280 0x7f352819d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:03:18.264 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:18.261+0000 7f3525e00700 1 -- 172.21.15.153:0/2280045623 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f352810d280 msgr2=0x7f352819d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:03:18.264 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:18.261+0000 7f3525e00700 1 --2- 172.21.15.153:0/2280045623 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f352810d280 0x7f352819d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:18.264 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:18.261+0000 7f3525e00700 1 -- 172.21.15.153:0/2280045623 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35180095f0 con 0x7f35281071d0 2024-03-21T10:03:18.264 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:18.261+0000 7f3525e00700 1 --2- 172.21.15.153:0/2280045623 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f35281071d0 0x7f352819ccd0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f351800b5c0 tx=0x7f351800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:03:18.265 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:18.262+0000 7f351f400700 1 -- 172.21.15.153:0/2280045623 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f351801d070 con 0x7f35281071d0 2024-03-21T10:03:18.265 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:18.262+0000 7f351f400700 1 -- 172.21.15.153:0/2280045623 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f351800f460 con 0x7f35281071d0 2024-03-21T10:03:18.265 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:18.262+0000 7f352ce00700 1 -- 172.21.15.153:0/2280045623 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35281a06c0 con 0x7f35281071d0 2024-03-21T10:03:18.265 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:18.262+0000 7f351f400700 1 -- 172.21.15.153:0/2280045623 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3518021640 con 0x7f35281071d0 2024-03-21T10:03:18.265 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:18.262+0000 7f352ce00700 1 -- 172.21.15.153:0/2280045623 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35281a0bb0 con 0x7f35281071d0 2024-03-21T10:03:18.265 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:18.263+0000 7f352ce00700 1 -- 172.21.15.153:0/2280045623 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f352804ea90 con 0x7f35281071d0 2024-03-21T10:03:18.266 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:18.264+0000 7f351f400700 1 -- 172.21.15.153:0/2280045623 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f351800f5d0 con 0x7f35281071d0 2024-03-21T10:03:18.266 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:18.265+0000 7f351f400700 1 --2- 172.21.15.153:0/2280045623 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f350806c4e0 0x7f350806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:03:18.266 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:18.265+0000 7f351f400700 1 -- 172.21.15.153:0/2280045623 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f351808c8f0 con 0x7f35281071d0 2024-03-21T10:03:18.266 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:18.265+0000 7f351d400700 1 --2- 172.21.15.153:0/2280045623 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f350806c4e0 0x7f350806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:03:18.267 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:18.266+0000 7f351d400700 1 --2- 172.21.15.153:0/2280045623 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f350806c4e0 0x7f350806e9a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f352819e2b0 tx=0x7f3510005d50 comp rx=0 tx=0).ready entity=mgr.14221 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:03:18.271 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:18.270+0000 7f351f400700 1 -- 172.21.15.153:0/2280045623 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f3518057410 con 0x7f35281071d0 2024-03-21T10:03:18.486 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:18.484+0000 7f352ce00700 1 -- 172.21.15.153:0/2280045623 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f35281a1000 con 0x7f35281071d0 2024-03-21T10:03:18.487 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:18.486+0000 7f351f400700 1 -- 172.21.15.153:0/2280045623 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+11360 (secure 0 0 0) 0x7f351805aa30 con 0x7f35281071d0 2024-03-21T10:03:18.487 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:03:18.488 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":41,"fsid":"070ed3e4-e769-11ee-95cd-87774f69a715","created":"2024-03-21T09:55:19.360240+0000","modified":"2024-03-21T10:02:56.435311+0000","last_up_change":"2024-03-21T10:02:55.420099+0000","last_in_change":"2024-03-21T10:02:32.306573+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T10:01:18.943639+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0f2e1224-6f45-4f23-b97a-5b89319a90f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":1050381948},{"type":"v1","addr":"172.21.15.153:6803","nonce":1050381948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":1050381948},{"type":"v1","addr":"172.21.15.153:6805","nonce":1050381948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":1050381948},{"type":"v1","addr":"172.21.15.153:6809","nonce":1050381948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":1050381948},{"type":"v1","addr":"172.21.15.153:6807","nonce":1050381948}]},"public_addr":"172.21.15.153:6803/1050381948","cluster_addr":"172.21.15.153:6805/1050381948","heartbeat_back_addr":"172.21.15.153:6809/1050381948","heartbeat_front_addr":"172.21.15.153:6807/1050381948","state":["exists","up"]},{"osd":1,"uuid":"76195edb-aa7a-4627-a211-09a578de65c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":3504985760},{"type":"v1","addr":"172.21.15.153:6811","nonce":3504985760}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":3504985760},{"type":"v1","addr":"172.21.15.153:6813","nonce":3504985760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":3504985760},{"type":"v1","addr":"172.21.15.153:6817","nonce":3504985760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":3504985760},{"type":"v1","addr":"172.21.15.153:6815","nonce":3504985760}]},"public_addr":"172.21.15.153:6811/3504985760","cluster_addr":"172.21.15.153:6813/3504985760","heartbeat_back_addr":"172.21.15.153:6817/3504985760","heartbeat_front_addr":"172.21.15.153:6815/3504985760","state":["exists","up"]},{"osd":2,"uuid":"3ec17cf8-f0ea-45c4-b084-b304024d6e92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":2844395219},{"type":"v1","addr":"172.21.15.153:6819","nonce":2844395219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":2844395219},{"type":"v1","addr":"172.21.15.153:6821","nonce":2844395219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":2844395219},{"type":"v1","addr":"172.21.15.153:6825","nonce":2844395219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":2844395219},{"type":"v1","addr":"172.21.15.153:6823","nonce":2844395219}]},"public_addr":"172.21.15.153:6819/2844395219","cluster_addr":"172.21.15.153:6821/2844395219","heartbeat_back_addr":"172.21.15.153:6825/2844395219","heartbeat_front_addr":"172.21.15.153:6823/2844395219","state":["exists","up"]},{"osd":3,"uuid":"285146f6-6209-47ee-9e5b-2e075c949231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":393592876},{"type":"v1","addr":"172.21.15.174:6801","nonce":393592876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":393592876},{"type":"v1","addr":"172.21.15.174:6803","nonce":393592876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":393592876},{"type":"v1","addr":"172.21.15.174:6807","nonce":393592876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":393592876},{"type":"v1","addr":"172.21.15.174:6805","nonce":393592876}]},"public_addr":"172.21.15.174:6801/393592876","cluster_addr":"172.21.15.174:6803/393592876","heartbeat_back_addr":"172.21.15.174:6807/393592876","heartbeat_front_addr":"172.21.15.174:6805/393592876","state":["exists","up"]},{"osd":4,"uuid":"f05cec2d-c649-489d-98a3-f24a75fe1f95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":116016676},{"type":"v1","addr":"172.21.15.174:6809","nonce":116016676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":116016676},{"type":"v1","addr":"172.21.15.174:6811","nonce":116016676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":116016676},{"type":"v1","addr":"172.21.15.174:6815","nonce":116016676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":116016676},{"type":"v1","addr":"172.21.15.174:6813","nonce":116016676}]},"public_addr":"172.21.15.174:6809/116016676","cluster_addr":"172.21.15.174:6811/116016676","heartbeat_back_addr":"172.21.15.174:6815/116016676","heartbeat_front_addr":"172.21.15.174:6813/116016676","state":["exists","up"]},{"osd":5,"uuid":"f430a3b2-e1b5-4698-9089-812fcf3a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":184680961},{"type":"v1","addr":"172.21.15.174:6817","nonce":184680961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":184680961},{"type":"v1","addr":"172.21.15.174:6819","nonce":184680961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":184680961},{"type":"v1","addr":"172.21.15.174:6823","nonce":184680961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":184680961},{"type":"v1","addr":"172.21.15.174:6821","nonce":184680961}]},"public_addr":"172.21.15.174:6817/184680961","cluster_addr":"172.21.15.174:6819/184680961","heartbeat_back_addr":"172.21.15.174:6823/184680961","heartbeat_front_addr":"172.21.15.174:6821/184680961","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:00:04.356688+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:00:39.988638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:01:14.882381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:01:48.072468+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:02:21.211701+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:02:54.161138+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.153:0/3278686288":"2024-03-22T09:58:24.250138+0000","172.21.15.153:0/652104290":"2024-03-22T09:58:24.250138+0000","172.21.15.153:0/405164318":"2024-03-22T09:58:24.250138+0000","172.21.15.153:6800/1143522828":"2024-03-22T09:58:24.250138+0000","172.21.15.153:0/3456116624":"2024-03-22T09:56:43.480866+0000","172.21.15.153:6801/3178125243":"2024-03-22T09:56:43.480866+0000","172.21.15.153:0/2407536999":"2024-03-22T09:56:00.849533+0000","172.21.15.153:6800/3178125243":"2024-03-22T09:56:43.480866+0000","172.21.15.153:0/1163205431":"2024-03-22T09:56:43.480866+0000","172.21.15.153:0/623594592":"2024-03-22T09:56:00.849533+0000","172.21.15.153:6801/1143522828":"2024-03-22T09:58:24.250138+0000","172.21.15.153:6800/270837948":"2024-03-22T09:56:00.849533+0000","172.21.15.153:6801/270837948":"2024-03-22T09:56:00.849533+0000","172.21.15.153:0/816489762":"2024-03-22T09:56:43.480866+0000","172.21.15.153:0/3435678189":"2024-03-22T09:56:00.849533+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T10:03:18.490 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:18.488+0000 7f352ce00700 1 -- 172.21.15.153:0/2280045623 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f350806c4e0 msgr2=0x7f350806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:03:18.490 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:18.488+0000 7f352ce00700 1 --2- 172.21.15.153:0/2280045623 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f350806c4e0 0x7f350806e9a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f352819e2b0 tx=0x7f3510005d50 comp rx=0 tx=0).stop 2024-03-21T10:03:18.490 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:18.488+0000 7f352ce00700 1 -- 172.21.15.153:0/2280045623 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f35281071d0 msgr2=0x7f352819ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:03:18.490 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:18.488+0000 7f352ce00700 1 --2- 172.21.15.153:0/2280045623 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f35281071d0 0x7f352819ccd0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f351800b5c0 tx=0x7f351800be20 comp rx=0 tx=0).stop 2024-03-21T10:03:18.491 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:18.489+0000 7f352ce00700 1 -- 172.21.15.153:0/2280045623 shutdown_connections 2024-03-21T10:03:18.491 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:18.489+0000 7f352ce00700 1 --2- 172.21.15.153:0/2280045623 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f350806c4e0 0x7f350806e9a0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:18.491 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:18.489+0000 7f352ce00700 1 --2- 172.21.15.153:0/2280045623 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f35281071d0 0x7f352819ccd0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:18.491 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:18.489+0000 7f352ce00700 1 --2- 172.21.15.153:0/2280045623 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f352810d280 0x7f352819d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:18.491 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:18.489+0000 7f352ce00700 1 -- 172.21.15.153:0/2280045623 >> 172.21.15.153:0/2280045623 conn(0x7f35280769e0 msgr2=0x7f3528078320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:03:18.491 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:18.489+0000 7f352ce00700 1 -- 172.21.15.153:0/2280045623 shutdown_connections 2024-03-21T10:03:18.491 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:18.489+0000 7f352ce00700 1 -- 172.21.15.153:0/2280045623 wait complete. 2024-03-21T10:03:19.252 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-03-21T10:03:19.253 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 070ed3e4-e769-11ee-95cd-87774f69a715 -- ceph osd dump --format=json 2024-03-21T10:03:19.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:03:19 smithi153 ceph-mon[29117]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:03:19.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:03:19 smithi153 ceph-mon[29117]: from='client.? 172.21.15.153:0/2280045623' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T10:03:19.658 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi153/config 2024-03-21T10:03:19.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:03:19 smithi174 ceph-mon[37496]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:03:19.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:03:19 smithi174 ceph-mon[37496]: from='client.? 172.21.15.153:0/2280045623' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T10:03:21.501 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:03:21 smithi153 ceph-mon[29117]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:03:21.664 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:21.662+0000 7f2310400700 1 -- 172.21.15.153:0/3168912838 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2308105b50 msgr2=0x7f2308068410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:03:21.665 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:21.662+0000 7f2310400700 1 --2- 172.21.15.153:0/3168912838 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2308105b50 0x7f2308068410 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f22fc009910 tx=0x7f22fc009c20 comp rx=0 tx=0).stop 2024-03-21T10:03:21.665 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:21.662+0000 7f2310400700 1 -- 172.21.15.153:0/3168912838 shutdown_connections 2024-03-21T10:03:21.665 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:21.662+0000 7f2310400700 1 --2- 172.21.15.153:0/3168912838 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2308068950 0x7f2308068dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:21.665 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:21.662+0000 7f2310400700 1 --2- 172.21.15.153:0/3168912838 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2308105b50 0x7f2308068410 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:21.665 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:21.662+0000 7f2310400700 1 -- 172.21.15.153:0/3168912838 >> 172.21.15.153:0/3168912838 conn(0x7f23080755a0 msgr2=0x7f23080759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:03:21.665 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:21.663+0000 7f2310400700 1 -- 172.21.15.153:0/3168912838 shutdown_connections 2024-03-21T10:03:21.665 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:21.663+0000 7f2310400700 1 -- 172.21.15.153:0/3168912838 wait complete. 2024-03-21T10:03:21.667 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:21.664+0000 7f2310400700 1 Processor -- start 2024-03-21T10:03:21.667 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:21.664+0000 7f2310400700 1 -- start start 2024-03-21T10:03:21.668 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:21.664+0000 7f2310400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2308068950 0x7f23081a2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:03:21.668 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:21.664+0000 7f2310400700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2308105b50 0x7f23081a2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:03:21.668 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:21.664+0000 7f2310400700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23081a3680 con 0x7f2308068950 2024-03-21T10:03:21.668 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:21.664+0000 7f2310400700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f230819cb30 con 0x7f2308105b50 2024-03-21T10:03:21.668 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:21.664+0000 7f230da00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2308068950 0x7f23081a2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:03:21.668 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:21.664+0000 7f230da00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2308068950 0x7f23081a2ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:56770/0 (socket says 172.21.15.153:56770) 2024-03-21T10:03:21.668 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:21.665+0000 7f230da00700 1 -- 172.21.15.153:0/3550453166 learned_addr learned my addr 172.21.15.153:0/3550453166 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:03:21.668 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:21.665+0000 7f230d000700 1 --2- 172.21.15.153:0/3550453166 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2308105b50 0x7f23081a2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:03:21.668 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:21.665+0000 7f230da00700 1 -- 172.21.15.153:0/3550453166 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2308105b50 msgr2=0x7f23081a2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:03:21.668 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:21.665+0000 7f230da00700 1 --2- 172.21.15.153:0/3550453166 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2308105b50 0x7f23081a2ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:21.669 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:21.665+0000 7f230da00700 1 -- 172.21.15.153:0/3550453166 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22fc0095f0 con 0x7f2308068950 2024-03-21T10:03:21.669 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:21.665+0000 7f230da00700 1 --2- 172.21.15.153:0/3550453166 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2308068950 0x7f23081a2ab0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f22fc00bda0 tx=0x7f22fc00be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:03:21.669 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:21.665+0000 7f2306000700 1 -- 172.21.15.153:0/3550453166 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f22fc01d070 con 0x7f2308068950 2024-03-21T10:03:21.669 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:21.665+0000 7f2306000700 1 -- 172.21.15.153:0/3550453166 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f22fc00f460 con 0x7f2308068950 2024-03-21T10:03:21.669 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:21.665+0000 7f2306000700 1 -- 172.21.15.153:0/3550453166 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f22fc017630 con 0x7f2308068950 2024-03-21T10:03:21.669 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:21.666+0000 7f2310400700 1 -- 172.21.15.153:0/3550453166 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f230819cdb0 con 0x7f2308068950 2024-03-21T10:03:21.669 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:21.666+0000 7f2310400700 1 -- 172.21.15.153:0/3550453166 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f230819d2a0 con 0x7f2308068950 2024-03-21T10:03:21.670 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:21.667+0000 7f2310400700 1 -- 172.21.15.153:0/3550453166 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f230804ea90 con 0x7f2308068950 2024-03-21T10:03:21.670 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:21.668+0000 7f2306000700 1 -- 172.21.15.153:0/3550453166 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f22fc00fab0 con 0x7f2308068950 2024-03-21T10:03:21.670 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:21.669+0000 7f2306000700 1 --2- 172.21.15.153:0/3550453166 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f22f006c4e0 0x7f22f006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:03:21.670 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:21.669+0000 7f2306000700 1 -- 172.21.15.153:0/3550453166 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f22fc08c990 con 0x7f2308068950 2024-03-21T10:03:21.671 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:21.669+0000 7f230d000700 1 --2- 172.21.15.153:0/3550453166 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f22f006c4e0 0x7f22f006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:03:21.671 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:21.670+0000 7f230d000700 1 --2- 172.21.15.153:0/3550453166 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f22f006c4e0 0x7f22f006e9a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f22f4005950 tx=0x7f22f40058e0 comp rx=0 tx=0).ready entity=mgr.14221 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:03:21.675 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:21.674+0000 7f2306000700 1 -- 172.21.15.153:0/3550453166 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f22fc05af20 con 0x7f2308068950 2024-03-21T10:03:21.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:03:21 smithi174 ceph-mon[37496]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:03:21.889 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:21.887+0000 7f2310400700 1 -- 172.21.15.153:0/3550453166 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f230819dfd0 con 0x7f2308068950 2024-03-21T10:03:21.889 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:21.888+0000 7f2306000700 1 -- 172.21.15.153:0/3550453166 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+11360 (secure 0 0 0) 0x7f22fc05aab0 con 0x7f2308068950 2024-03-21T10:03:21.890 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:03:21.890 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":41,"fsid":"070ed3e4-e769-11ee-95cd-87774f69a715","created":"2024-03-21T09:55:19.360240+0000","modified":"2024-03-21T10:02:56.435311+0000","last_up_change":"2024-03-21T10:02:55.420099+0000","last_in_change":"2024-03-21T10:02:32.306573+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T10:01:18.943639+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0f2e1224-6f45-4f23-b97a-5b89319a90f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":1050381948},{"type":"v1","addr":"172.21.15.153:6803","nonce":1050381948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":1050381948},{"type":"v1","addr":"172.21.15.153:6805","nonce":1050381948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":1050381948},{"type":"v1","addr":"172.21.15.153:6809","nonce":1050381948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":1050381948},{"type":"v1","addr":"172.21.15.153:6807","nonce":1050381948}]},"public_addr":"172.21.15.153:6803/1050381948","cluster_addr":"172.21.15.153:6805/1050381948","heartbeat_back_addr":"172.21.15.153:6809/1050381948","heartbeat_front_addr":"172.21.15.153:6807/1050381948","state":["exists","up"]},{"osd":1,"uuid":"76195edb-aa7a-4627-a211-09a578de65c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":3504985760},{"type":"v1","addr":"172.21.15.153:6811","nonce":3504985760}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":3504985760},{"type":"v1","addr":"172.21.15.153:6813","nonce":3504985760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":3504985760},{"type":"v1","addr":"172.21.15.153:6817","nonce":3504985760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":3504985760},{"type":"v1","addr":"172.21.15.153:6815","nonce":3504985760}]},"public_addr":"172.21.15.153:6811/3504985760","cluster_addr":"172.21.15.153:6813/3504985760","heartbeat_back_addr":"172.21.15.153:6817/3504985760","heartbeat_front_addr":"172.21.15.153:6815/3504985760","state":["exists","up"]},{"osd":2,"uuid":"3ec17cf8-f0ea-45c4-b084-b304024d6e92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":2844395219},{"type":"v1","addr":"172.21.15.153:6819","nonce":2844395219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":2844395219},{"type":"v1","addr":"172.21.15.153:6821","nonce":2844395219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":2844395219},{"type":"v1","addr":"172.21.15.153:6825","nonce":2844395219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":2844395219},{"type":"v1","addr":"172.21.15.153:6823","nonce":2844395219}]},"public_addr":"172.21.15.153:6819/2844395219","cluster_addr":"172.21.15.153:6821/2844395219","heartbeat_back_addr":"172.21.15.153:6825/2844395219","heartbeat_front_addr":"172.21.15.153:6823/2844395219","state":["exists","up"]},{"osd":3,"uuid":"285146f6-6209-47ee-9e5b-2e075c949231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":393592876},{"type":"v1","addr":"172.21.15.174:6801","nonce":393592876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":393592876},{"type":"v1","addr":"172.21.15.174:6803","nonce":393592876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":393592876},{"type":"v1","addr":"172.21.15.174:6807","nonce":393592876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":393592876},{"type":"v1","addr":"172.21.15.174:6805","nonce":393592876}]},"public_addr":"172.21.15.174:6801/393592876","cluster_addr":"172.21.15.174:6803/393592876","heartbeat_back_addr":"172.21.15.174:6807/393592876","heartbeat_front_addr":"172.21.15.174:6805/393592876","state":["exists","up"]},{"osd":4,"uuid":"f05cec2d-c649-489d-98a3-f24a75fe1f95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":116016676},{"type":"v1","addr":"172.21.15.174:6809","nonce":116016676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":116016676},{"type":"v1","addr":"172.21.15.174:6811","nonce":116016676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":116016676},{"type":"v1","addr":"172.21.15.174:6815","nonce":116016676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":116016676},{"type":"v1","addr":"172.21.15.174:6813","nonce":116016676}]},"public_addr":"172.21.15.174:6809/116016676","cluster_addr":"172.21.15.174:6811/116016676","heartbeat_back_addr":"172.21.15.174:6815/116016676","heartbeat_front_addr":"172.21.15.174:6813/116016676","state":["exists","up"]},{"osd":5,"uuid":"f430a3b2-e1b5-4698-9089-812fcf3a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":184680961},{"type":"v1","addr":"172.21.15.174:6817","nonce":184680961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":184680961},{"type":"v1","addr":"172.21.15.174:6819","nonce":184680961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":184680961},{"type":"v1","addr":"172.21.15.174:6823","nonce":184680961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":184680961},{"type":"v1","addr":"172.21.15.174:6821","nonce":184680961}]},"public_addr":"172.21.15.174:6817/184680961","cluster_addr":"172.21.15.174:6819/184680961","heartbeat_back_addr":"172.21.15.174:6823/184680961","heartbeat_front_addr":"172.21.15.174:6821/184680961","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:00:04.356688+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:00:39.988638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:01:14.882381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:01:48.072468+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:02:21.211701+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:02:54.161138+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.153:0/3278686288":"2024-03-22T09:58:24.250138+0000","172.21.15.153:0/652104290":"2024-03-22T09:58:24.250138+0000","172.21.15.153:0/405164318":"2024-03-22T09:58:24.250138+0000","172.21.15.153:6800/1143522828":"2024-03-22T09:58:24.250138+0000","172.21.15.153:0/3456116624":"2024-03-22T09:56:43.480866+0000","172.21.15.153:6801/3178125243":"2024-03-22T09:56:43.480866+0000","172.21.15.153:0/2407536999":"2024-03-22T09:56:00.849533+0000","172.21.15.153:6800/3178125243":"2024-03-22T09:56:43.480866+0000","172.21.15.153:0/1163205431":"2024-03-22T09:56:43.480866+0000","172.21.15.153:0/623594592":"2024-03-22T09:56:00.849533+0000","172.21.15.153:6801/1143522828":"2024-03-22T09:58:24.250138+0000","172.21.15.153:6800/270837948":"2024-03-22T09:56:00.849533+0000","172.21.15.153:6801/270837948":"2024-03-22T09:56:00.849533+0000","172.21.15.153:0/816489762":"2024-03-22T09:56:43.480866+0000","172.21.15.153:0/3435678189":"2024-03-22T09:56:00.849533+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T10:03:21.894 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:21.892+0000 7f2310400700 1 -- 172.21.15.153:0/3550453166 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f22f006c4e0 msgr2=0x7f22f006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:03:21.895 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:21.892+0000 7f2310400700 1 --2- 172.21.15.153:0/3550453166 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f22f006c4e0 0x7f22f006e9a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f22f4005950 tx=0x7f22f40058e0 comp rx=0 tx=0).stop 2024-03-21T10:03:21.895 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:21.892+0000 7f2310400700 1 -- 172.21.15.153:0/3550453166 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2308068950 msgr2=0x7f23081a2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:03:21.895 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:21.892+0000 7f2310400700 1 --2- 172.21.15.153:0/3550453166 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2308068950 0x7f23081a2ab0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f22fc00bda0 tx=0x7f22fc00be80 comp rx=0 tx=0).stop 2024-03-21T10:03:21.895 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:21.893+0000 7f2310400700 1 -- 172.21.15.153:0/3550453166 shutdown_connections 2024-03-21T10:03:21.895 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:21.893+0000 7f2310400700 1 --2- 172.21.15.153:0/3550453166 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f22f006c4e0 0x7f22f006e9a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:21.895 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:21.893+0000 7f2310400700 1 --2- 172.21.15.153:0/3550453166 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2308068950 0x7f23081a2ab0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:21.895 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:21.893+0000 7f2310400700 1 --2- 172.21.15.153:0/3550453166 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2308105b50 0x7f23081a2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:21.895 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:21.893+0000 7f2310400700 1 -- 172.21.15.153:0/3550453166 >> 172.21.15.153:0/3550453166 conn(0x7f23080755a0 msgr2=0x7f23081014b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:03:21.895 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:21.893+0000 7f2310400700 1 -- 172.21.15.153:0/3550453166 shutdown_connections 2024-03-21T10:03:21.895 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:21.893+0000 7f2310400700 1 -- 172.21.15.153:0/3550453166 wait complete. 2024-03-21T10:03:22.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:03:22 smithi153 ceph-mon[29117]: from='client.? 172.21.15.153:0/3550453166' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T10:03:22.659 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 070ed3e4-e769-11ee-95cd-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2024-03-21T10:03:22.660 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 070ed3e4-e769-11ee-95cd-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2024-03-21T10:03:22.660 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 070ed3e4-e769-11ee-95cd-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2024-03-21T10:03:22.661 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 070ed3e4-e769-11ee-95cd-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2024-03-21T10:03:22.661 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 070ed3e4-e769-11ee-95cd-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2024-03-21T10:03:22.661 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 070ed3e4-e769-11ee-95cd-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2024-03-21T10:03:22.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:03:22 smithi174 ceph-mon[37496]: from='client.? 172.21.15.153:0/3550453166' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T10:03:23.311 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi153/config 2024-03-21T10:03:23.311 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi153/config 2024-03-21T10:03:23.322 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi153/config 2024-03-21T10:03:23.322 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi153/config 2024-03-21T10:03:23.324 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi153/config 2024-03-21T10:03:23.326 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi153/config 2024-03-21T10:03:23.695 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:03:23 smithi174 ceph-mon[37496]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:03:23.731 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:03:23 smithi153 ceph-mon[29117]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:03:25.265 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:03:25 smithi153 ceph-mon[29117]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:03:25.265 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:03:25 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:03:25.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:03:25 smithi174 ceph-mon[37496]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:03:25.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:03:25 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:03:27.487 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:03:27 smithi153 ceph-mon[29117]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:03:27.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:03:27 smithi174 ceph-mon[37496]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:03:29.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:03:29 smithi174 ceph-mon[37496]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:03:29.718 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:03:29 smithi153 ceph-mon[29117]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:03:29.882 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:29.880+0000 7fbe76600700 1 -- 172.21.15.153:0/4051836663 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbe70073160 msgr2=0x7fbe70073540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:03:29.882 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:29.880+0000 7fbe76600700 1 --2- 172.21.15.153:0/4051836663 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbe70073160 0x7fbe70073540 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fbe60009910 tx=0x7fbe60009c20 comp rx=0 tx=0).stop 2024-03-21T10:03:29.882 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:29.880+0000 7fbe76600700 1 -- 172.21.15.153:0/4051836663 shutdown_connections 2024-03-21T10:03:29.883 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:29.880+0000 7fbe76600700 1 --2- 172.21.15.153:0/4051836663 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbe70073a80 0x7fbe7010cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:29.883 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:29.880+0000 7fbe76600700 1 --2- 172.21.15.153:0/4051836663 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbe70073160 0x7fbe70073540 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:29.883 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:29.880+0000 7fbe76600700 1 -- 172.21.15.153:0/4051836663 >> 172.21.15.153:0/4051836663 conn(0x7fbe700fc630 msgr2=0x7fbe700fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:03:29.883 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:29.880+0000 7fbe76600700 1 -- 172.21.15.153:0/4051836663 shutdown_connections 2024-03-21T10:03:29.883 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:29.880+0000 7fbe76600700 1 -- 172.21.15.153:0/4051836663 wait complete. 2024-03-21T10:03:29.884 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:29.881+0000 7fbe76600700 1 Processor -- start 2024-03-21T10:03:29.884 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:29.882+0000 7fbe76600700 1 -- start start 2024-03-21T10:03:29.884 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:29.882+0000 7fbe76600700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbe70073160 0x7fbe70198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:03:29.884 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:29.882+0000 7fbe76600700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbe70073a80 0x7fbe70198ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:03:29.884 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:29.882+0000 7fbe76600700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe70199550 con 0x7fbe70073160 2024-03-21T10:03:29.884 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:29.882+0000 7fbe76600700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe7019d360 con 0x7fbe70073a80 2024-03-21T10:03:29.885 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:29.882+0000 7fbe6f400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbe70073160 0x7fbe70198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:03:29.885 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:29.882+0000 7fbe6f400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbe70073160 0x7fbe70198980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41880/0 (socket says 172.21.15.153:41880) 2024-03-21T10:03:29.885 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:29.882+0000 7fbe6f400700 1 -- 172.21.15.153:0/560028391 learned_addr learned my addr 172.21.15.153:0/560028391 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:03:29.885 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:29.883+0000 7fbe6ea00700 1 --2- 172.21.15.153:0/560028391 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbe70073a80 0x7fbe70198ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:03:29.885 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:29.883+0000 7fbe6f400700 1 -- 172.21.15.153:0/560028391 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbe70073a80 msgr2=0x7fbe70198ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:03:29.885 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:29.883+0000 7fbe6f400700 1 --2- 172.21.15.153:0/560028391 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbe70073a80 0x7fbe70198ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:29.885 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:29.883+0000 7fbe6f400700 1 -- 172.21.15.153:0/560028391 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe600095f0 con 0x7fbe70073160 2024-03-21T10:03:29.885 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:29.883+0000 7fbe6f400700 1 --2- 172.21.15.153:0/560028391 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbe70073160 0x7fbe70198980 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fbe60006010 tx=0x7fbe6000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:03:29.886 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:29.883+0000 7fbe67e00700 1 -- 172.21.15.153:0/560028391 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe6001d070 con 0x7fbe70073160 2024-03-21T10:03:29.886 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:29.883+0000 7fbe67e00700 1 -- 172.21.15.153:0/560028391 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fbe6000f460 con 0x7fbe70073160 2024-03-21T10:03:29.886 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:29.883+0000 7fbe67e00700 1 -- 172.21.15.153:0/560028391 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe60017630 con 0x7fbe70073160 2024-03-21T10:03:29.886 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:29.883+0000 7fbe76600700 1 -- 172.21.15.153:0/560028391 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe7019d5e0 con 0x7fbe70073160 2024-03-21T10:03:29.886 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:29.884+0000 7fbe76600700 1 -- 172.21.15.153:0/560028391 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe7019dad0 con 0x7fbe70073160 2024-03-21T10:03:29.886 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:29.885+0000 7fbe76600700 1 -- 172.21.15.153:0/560028391 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbe70110250 con 0x7fbe70073160 2024-03-21T10:03:29.887 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:29.886+0000 7fbe67e00700 1 -- 172.21.15.153:0/560028391 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fbe6000f5d0 con 0x7fbe70073160 2024-03-21T10:03:29.888 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:29.887+0000 7fbe67e00700 1 --2- 172.21.15.153:0/560028391 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fbe5406c4e0 0x7fbe5406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:03:29.889 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:29.887+0000 7fbe67e00700 1 -- 172.21.15.153:0/560028391 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fbe6008cad0 con 0x7fbe70073160 2024-03-21T10:03:29.889 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:29.887+0000 7fbe6ea00700 1 --2- 172.21.15.153:0/560028391 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fbe5406c4e0 0x7fbe5406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:03:29.889 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:29.887+0000 7fbe67e00700 1 --2- 172.21.15.153:0/560028391 >> [v2:172.21.15.174:6800/393592876,v1:172.21.15.174:6801/393592876] conn(0x7fbe540721c0 0x7fbe540745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:03:29.889 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:29.887+0000 7fbe67e00700 1 -- 172.21.15.153:0/560028391 --> [v2:172.21.15.174:6800/393592876,v1:172.21.15.174:6801/393592876] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbe54074c90 con 0x7fbe540721c0 2024-03-21T10:03:29.889 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:29.887+0000 7fbe67e00700 1 -- 172.21.15.153:0/560028391 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbe600912a0 con 0x7fbe70073160 2024-03-21T10:03:29.890 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:29.888+0000 7fbe6fe00700 1 --2- 172.21.15.153:0/560028391 >> [v2:172.21.15.174:6800/393592876,v1:172.21.15.174:6801/393592876] conn(0x7fbe540721c0 0x7fbe540745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:03:29.890 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:29.888+0000 7fbe6ea00700 1 --2- 172.21.15.153:0/560028391 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fbe5406c4e0 0x7fbe5406e9a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c005fd0 tx=0x7fbe5c00a560 comp rx=0 tx=0).ready entity=mgr.14221 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:03:29.890 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:29.889+0000 7fbe6fe00700 1 --2- 172.21.15.153:0/560028391 >> [v2:172.21.15.174:6800/393592876,v1:172.21.15.174:6801/393592876] conn(0x7fbe540721c0 0x7fbe540745e0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:03:29.892 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:29.891+0000 7fbe67e00700 1 -- 172.21.15.153:0/560028391 <== osd.3 v2:172.21.15.174:6800/393592876 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fbe54074c90 con 0x7fbe540721c0 2024-03-21T10:03:29.911 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:29.909+0000 7fbe76600700 1 -- 172.21.15.153:0/560028391 --> [v2:172.21.15.174:6800/393592876,v1:172.21.15.174:6801/393592876] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fbe7004ea90 con 0x7fbe540721c0 2024-03-21T10:03:29.912 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:29.910+0000 7fbe67e00700 1 -- 172.21.15.153:0/560028391 <== osd.3 v2:172.21.15.174:6800/393592876 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fbe7004ea90 con 0x7fbe540721c0 2024-03-21T10:03:29.913 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:29.911+0000 7fbe76600700 1 -- 172.21.15.153:0/560028391 >> [v2:172.21.15.174:6800/393592876,v1:172.21.15.174:6801/393592876] conn(0x7fbe540721c0 msgr2=0x7fbe540745e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:03:29.913 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:29.911+0000 7fbe76600700 1 --2- 172.21.15.153:0/560028391 >> [v2:172.21.15.174:6800/393592876,v1:172.21.15.174:6801/393592876] conn(0x7fbe540721c0 0x7fbe540745e0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:29.913 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:29.911+0000 7fbe76600700 1 -- 172.21.15.153:0/560028391 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fbe5406c4e0 msgr2=0x7fbe5406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:03:29.913 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:29.911+0000 7fbe76600700 1 --2- 172.21.15.153:0/560028391 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fbe5406c4e0 0x7fbe5406e9a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c005fd0 tx=0x7fbe5c00a560 comp rx=0 tx=0).stop 2024-03-21T10:03:29.913 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:29.911+0000 7fbe76600700 1 -- 172.21.15.153:0/560028391 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbe70073160 msgr2=0x7fbe70198980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:03:29.913 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:29.911+0000 7fbe76600700 1 --2- 172.21.15.153:0/560028391 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbe70073160 0x7fbe70198980 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fbe60006010 tx=0x7fbe6000be20 comp rx=0 tx=0).stop 2024-03-21T10:03:29.913 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:29.911+0000 7fbe76600700 1 -- 172.21.15.153:0/560028391 shutdown_connections 2024-03-21T10:03:29.913 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:29.911+0000 7fbe76600700 1 --2- 172.21.15.153:0/560028391 >> [v2:172.21.15.174:6800/393592876,v1:172.21.15.174:6801/393592876] conn(0x7fbe540721c0 0x7fbe540745e0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:29.913 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:29.911+0000 7fbe76600700 1 --2- 172.21.15.153:0/560028391 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fbe5406c4e0 0x7fbe5406e9a0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:29.913 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:29.911+0000 7fbe76600700 1 --2- 172.21.15.153:0/560028391 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbe70073160 0x7fbe70198980 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:29.913 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:29.911+0000 7fbe76600700 1 --2- 172.21.15.153:0/560028391 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbe70073a80 0x7fbe70198ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:29.913 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:29.911+0000 7fbe76600700 1 -- 172.21.15.153:0/560028391 >> 172.21.15.153:0/560028391 conn(0x7fbe700fc630 msgr2=0x7fbe701076e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:03:29.914 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:29.912+0000 7fbe76600700 1 -- 172.21.15.153:0/560028391 shutdown_connections 2024-03-21T10:03:29.914 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:29.912+0000 7fbe76600700 1 -- 172.21.15.153:0/560028391 wait complete. 2024-03-21T10:03:31.482 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:03:31 smithi153 ceph-mon[29117]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:03:31.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:03:31 smithi174 ceph-mon[37496]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:03:31.974 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:31.888+0000 7f136f200700 1 -- 172.21.15.153:0/518405650 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1368069020 msgr2=0x7f1368105790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:03:31.974 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:31.888+0000 7f136f200700 1 --2- 172.21.15.153:0/518405650 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1368069020 0x7f1368105790 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f1354009910 tx=0x7f1354009c20 comp rx=0 tx=0).stop 2024-03-21T10:03:31.974 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:31.888+0000 7f136f200700 1 -- 172.21.15.153:0/518405650 shutdown_connections 2024-03-21T10:03:31.975 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:31.888+0000 7f136f200700 1 --2- 172.21.15.153:0/518405650 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1368069020 0x7f1368105790 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:31.975 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:31.888+0000 7f136f200700 1 --2- 172.21.15.153:0/518405650 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1368068670 0x7f1368068a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:31.975 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:31.888+0000 7f136f200700 1 -- 172.21.15.153:0/518405650 >> 172.21.15.153:0/518405650 conn(0x7f13680755a0 msgr2=0x7f13680759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:03:31.975 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:31.889+0000 7f136f200700 1 -- 172.21.15.153:0/518405650 shutdown_connections 2024-03-21T10:03:31.975 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:31.890+0000 7f136f200700 1 -- 172.21.15.153:0/518405650 wait complete. 2024-03-21T10:03:31.975 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:31.891+0000 7f136f200700 1 Processor -- start 2024-03-21T10:03:31.975 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:31.891+0000 7f136f200700 1 -- start start 2024-03-21T10:03:31.975 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:31.892+0000 7f136f200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1368068670 0x7f13680ffca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:03:31.975 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:31.892+0000 7f136f200700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1368069020 0x7f13681001e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:03:31.975 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:31.892+0000 7f136f200700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f136819a590 con 0x7f1368068670 2024-03-21T10:03:31.975 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:31.892+0000 7f136f200700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f136819a6d0 con 0x7f1368069020 2024-03-21T10:03:31.975 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:31.892+0000 7f1367e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1368068670 0x7f13680ffca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:03:31.976 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:31.892+0000 7f1367e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1368068670 0x7f13680ffca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41912/0 (socket says 172.21.15.153:41912) 2024-03-21T10:03:31.976 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:31.892+0000 7f1367e00700 1 -- 172.21.15.153:0/3473557200 learned_addr learned my addr 172.21.15.153:0/3473557200 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:03:31.976 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:31.893+0000 7f1367400700 1 --2- 172.21.15.153:0/3473557200 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1368069020 0x7f13681001e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:03:31.976 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:31.893+0000 7f1367e00700 1 -- 172.21.15.153:0/3473557200 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1368069020 msgr2=0x7f13681001e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:03:31.976 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:31.893+0000 7f1367e00700 1 --2- 172.21.15.153:0/3473557200 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1368069020 0x7f13681001e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:31.976 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:31.893+0000 7f1367e00700 1 -- 172.21.15.153:0/3473557200 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13540095f0 con 0x7f1368068670 2024-03-21T10:03:31.976 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:31.893+0000 7f1367e00700 1 --2- 172.21.15.153:0/3473557200 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1368068670 0x7f13680ffca0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f135800b4b0 tx=0x7f135800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:03:31.976 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:31.893+0000 7f1364c00700 1 -- 172.21.15.153:0/3473557200 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1358011800 con 0x7f1368068670 2024-03-21T10:03:31.976 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:31.893+0000 7f136f200700 1 -- 172.21.15.153:0/3473557200 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1368100780 con 0x7f1368068670 2024-03-21T10:03:31.976 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:31.893+0000 7f136f200700 1 -- 172.21.15.153:0/3473557200 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13681a71d0 con 0x7f1368068670 2024-03-21T10:03:31.976 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:31.894+0000 7f1364c00700 1 -- 172.21.15.153:0/3473557200 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f1358011e10 con 0x7f1368068670 2024-03-21T10:03:31.976 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:31.894+0000 7f1364c00700 1 -- 172.21.15.153:0/3473557200 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f135800f2e0 con 0x7f1368068670 2024-03-21T10:03:31.976 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:31.894+0000 7f136f200700 1 -- 172.21.15.153:0/3473557200 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1348000ff0 con 0x7f1368068670 2024-03-21T10:03:31.977 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:31.897+0000 7f1364c00700 1 -- 172.21.15.153:0/3473557200 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f1358011960 con 0x7f1368068670 2024-03-21T10:03:31.977 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:31.897+0000 7f1364c00700 1 --2- 172.21.15.153:0/3473557200 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f134c06c4e0 0x7f134c06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:03:31.977 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:31.898+0000 7f1364c00700 1 -- 172.21.15.153:0/3473557200 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f135808b5d0 con 0x7f1368068670 2024-03-21T10:03:31.977 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:31.898+0000 7f1367400700 1 --2- 172.21.15.153:0/3473557200 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f134c06c4e0 0x7f134c06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:03:31.977 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:31.898+0000 7f1364c00700 1 --2- 172.21.15.153:0/3473557200 >> [v2:172.21.15.174:6808/116016676,v1:172.21.15.174:6809/116016676] conn(0x7f134c0721c0 0x7f134c0745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:03:31.977 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:31.898+0000 7f1364c00700 1 -- 172.21.15.153:0/3473557200 --> [v2:172.21.15.174:6808/116016676,v1:172.21.15.174:6809/116016676] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f134c074c90 con 0x7f134c0721c0 2024-03-21T10:03:31.977 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:31.898+0000 7f1364c00700 1 -- 172.21.15.153:0/3473557200 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f135808ba30 con 0x7f1368068670 2024-03-21T10:03:31.977 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:31.898+0000 7f1367400700 1 --2- 172.21.15.153:0/3473557200 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f134c06c4e0 0x7f134c06e9a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1368101490 tx=0x7f135401a040 comp rx=0 tx=0).ready entity=mgr.14221 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:03:31.977 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:31.898+0000 7f136d200700 1 --2- 172.21.15.153:0/3473557200 >> [v2:172.21.15.174:6808/116016676,v1:172.21.15.174:6809/116016676] conn(0x7f134c0721c0 0x7f134c0745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:03:31.977 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:31.899+0000 7f136d200700 1 --2- 172.21.15.153:0/3473557200 >> [v2:172.21.15.174:6808/116016676,v1:172.21.15.174:6809/116016676] conn(0x7f134c0721c0 0x7f134c0745e0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:03:31.977 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:31.902+0000 7f1364c00700 1 -- 172.21.15.153:0/3473557200 <== osd.4 v2:172.21.15.174:6808/116016676 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f134c074c90 con 0x7f134c0721c0 2024-03-21T10:03:31.977 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:31.919+0000 7f136f200700 1 -- 172.21.15.153:0/3473557200 --> [v2:172.21.15.174:6808/116016676,v1:172.21.15.174:6809/116016676] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1348002ce0 con 0x7f134c0721c0 2024-03-21T10:03:31.977 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:31.919+0000 7f1364c00700 1 -- 172.21.15.153:0/3473557200 <== osd.4 v2:172.21.15.174:6808/116016676 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f1348002ce0 con 0x7f134c0721c0 2024-03-21T10:03:31.977 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:31.920+0000 7f136f200700 1 -- 172.21.15.153:0/3473557200 >> [v2:172.21.15.174:6808/116016676,v1:172.21.15.174:6809/116016676] conn(0x7f134c0721c0 msgr2=0x7f134c0745e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:03:31.978 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:31.920+0000 7f136f200700 1 --2- 172.21.15.153:0/3473557200 >> [v2:172.21.15.174:6808/116016676,v1:172.21.15.174:6809/116016676] conn(0x7f134c0721c0 0x7f134c0745e0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:31.978 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:31.920+0000 7f136f200700 1 -- 172.21.15.153:0/3473557200 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f134c06c4e0 msgr2=0x7f134c06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:03:31.978 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:31.920+0000 7f136f200700 1 --2- 172.21.15.153:0/3473557200 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f134c06c4e0 0x7f134c06e9a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1368101490 tx=0x7f135401a040 comp rx=0 tx=0).stop 2024-03-21T10:03:31.978 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:31.920+0000 7f136f200700 1 -- 172.21.15.153:0/3473557200 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1368068670 msgr2=0x7f13680ffca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:03:31.978 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:31.920+0000 7f136f200700 1 --2- 172.21.15.153:0/3473557200 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1368068670 0x7f13680ffca0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f135800b4b0 tx=0x7f135800b870 comp rx=0 tx=0).stop 2024-03-21T10:03:31.978 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:31.920+0000 7f136f200700 1 -- 172.21.15.153:0/3473557200 shutdown_connections 2024-03-21T10:03:31.978 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:31.920+0000 7f136f200700 1 --2- 172.21.15.153:0/3473557200 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f134c06c4e0 0x7f134c06e9a0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:31.979 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:31.920+0000 7f136f200700 1 --2- 172.21.15.153:0/3473557200 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1368068670 0x7f13680ffca0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:31.979 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:31.920+0000 7f136f200700 1 --2- 172.21.15.153:0/3473557200 >> [v2:172.21.15.174:6808/116016676,v1:172.21.15.174:6809/116016676] conn(0x7f134c0721c0 0x7f134c0745e0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:31.979 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:31.920+0000 7f136f200700 1 --2- 172.21.15.153:0/3473557200 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1368069020 0x7f13681001e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:31.979 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:31.920+0000 7f136f200700 1 -- 172.21.15.153:0/3473557200 >> 172.21.15.153:0/3473557200 conn(0x7f13680755a0 msgr2=0x7f13680fe7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:03:31.979 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:31.920+0000 7f136f200700 1 -- 172.21.15.153:0/3473557200 shutdown_connections 2024-03-21T10:03:31.979 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:31.920+0000 7f136f200700 1 -- 172.21.15.153:0/3473557200 wait complete. 2024-03-21T10:03:32.225 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:30.965+0000 7f9e74c00700 1 -- 172.21.15.153:0/3301664263 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9e70107220 msgr2=0x7f9e70107600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:03:32.226 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:30.965+0000 7f9e74c00700 1 --2- 172.21.15.153:0/3301664263 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9e70107220 0x7f9e70107600 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9e60009910 tx=0x7f9e60009c20 comp rx=0 tx=0).stop 2024-03-21T10:03:32.226 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:30.965+0000 7f9e74c00700 1 -- 172.21.15.153:0/3301664263 shutdown_connections 2024-03-21T10:03:32.226 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:30.965+0000 7f9e74c00700 1 --2- 172.21.15.153:0/3301664263 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9e70107bd0 0x7f9e7010bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:32.226 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:30.965+0000 7f9e74c00700 1 --2- 172.21.15.153:0/3301664263 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9e70107220 0x7f9e70107600 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:32.226 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:30.965+0000 7f9e74c00700 1 -- 172.21.15.153:0/3301664263 >> 172.21.15.153:0/3301664263 conn(0x7f9e700767e0 msgr2=0x7f9e70078c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:03:32.226 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:30.965+0000 7f9e74c00700 1 -- 172.21.15.153:0/3301664263 shutdown_connections 2024-03-21T10:03:32.226 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:30.966+0000 7f9e74c00700 1 -- 172.21.15.153:0/3301664263 wait complete. 2024-03-21T10:03:32.226 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:30.967+0000 7f9e74c00700 1 Processor -- start 2024-03-21T10:03:32.226 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:30.967+0000 7f9e74c00700 1 -- start start 2024-03-21T10:03:32.226 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:30.967+0000 7f9e74c00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9e70107220 0x7f9e701a2a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:03:32.227 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:30.967+0000 7f9e74c00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9e70107bd0 0x7f9e701a2f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:03:32.227 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:30.967+0000 7f9e74c00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e701a35f0 con 0x7f9e70107bd0 2024-03-21T10:03:32.227 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:30.967+0000 7f9e74c00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e7019caa0 con 0x7f9e70107220 2024-03-21T10:03:32.227 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:30.968+0000 7f9e6ea00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9e70107bd0 0x7f9e701a2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:03:32.227 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:30.968+0000 7f9e6ea00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9e70107bd0 0x7f9e701a2f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41898/0 (socket says 172.21.15.153:41898) 2024-03-21T10:03:32.227 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:30.968+0000 7f9e6ea00700 1 -- 172.21.15.153:0/3581763721 learned_addr learned my addr 172.21.15.153:0/3581763721 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:03:32.227 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:30.968+0000 7f9e6f400700 1 --2- 172.21.15.153:0/3581763721 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9e70107220 0x7f9e701a2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:03:32.227 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:30.968+0000 7f9e6ea00700 1 -- 172.21.15.153:0/3581763721 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9e70107220 msgr2=0x7f9e701a2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:03:32.227 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:30.968+0000 7f9e6ea00700 1 --2- 172.21.15.153:0/3581763721 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9e70107220 0x7f9e701a2a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:32.227 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:30.968+0000 7f9e6ea00700 1 -- 172.21.15.153:0/3581763721 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e600095f0 con 0x7f9e70107bd0 2024-03-21T10:03:32.227 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:30.968+0000 7f9e6ea00700 1 --2- 172.21.15.153:0/3581763721 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9e70107bd0 0x7f9e701a2f60 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c00d900 tx=0x7f9e5c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:03:32.227 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:30.969+0000 7f9e67e00700 1 -- 172.21.15.153:0/3581763721 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9e5c009900 con 0x7f9e70107bd0 2024-03-21T10:03:32.227 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:30.969+0000 7f9e67e00700 1 -- 172.21.15.153:0/3581763721 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f9e5c010460 con 0x7f9e70107bd0 2024-03-21T10:03:32.228 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:30.969+0000 7f9e67e00700 1 -- 172.21.15.153:0/3581763721 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9e5c00f5f0 con 0x7f9e70107bd0 2024-03-21T10:03:32.228 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:30.969+0000 7f9e74c00700 1 -- 172.21.15.153:0/3581763721 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e7019cd80 con 0x7f9e70107bd0 2024-03-21T10:03:32.228 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:30.969+0000 7f9e74c00700 1 -- 172.21.15.153:0/3581763721 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e7019d2d0 con 0x7f9e70107bd0 2024-03-21T10:03:32.228 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:30.970+0000 7f9e74c00700 1 -- 172.21.15.153:0/3581763721 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9e70114420 con 0x7f9e70107bd0 2024-03-21T10:03:32.228 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:30.971+0000 7f9e67e00700 1 -- 172.21.15.153:0/3581763721 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f9e5c0105d0 con 0x7f9e70107bd0 2024-03-21T10:03:32.228 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:30.972+0000 7f9e67e00700 1 --2- 172.21.15.153:0/3581763721 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f9e5406c4e0 0x7f9e5406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:03:32.228 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:30.972+0000 7f9e67e00700 1 -- 172.21.15.153:0/3581763721 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f9e5c08b3d0 con 0x7f9e70107bd0 2024-03-21T10:03:32.228 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:30.972+0000 7f9e6f400700 1 --2- 172.21.15.153:0/3581763721 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f9e5406c4e0 0x7f9e5406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:03:32.228 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:30.972+0000 7f9e67e00700 1 --2- 172.21.15.153:0/3581763721 >> [v2:172.21.15.153:6818/2844395219,v1:172.21.15.153:6819/2844395219] conn(0x7f9e540721c0 0x7f9e540745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:03:32.228 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:30.972+0000 7f9e67e00700 1 -- 172.21.15.153:0/3581763721 --> [v2:172.21.15.153:6818/2844395219,v1:172.21.15.153:6819/2844395219] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9e54074c90 con 0x7f9e540721c0 2024-03-21T10:03:32.228 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:30.972+0000 7f9e67e00700 1 -- 172.21.15.153:0/3581763721 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9e5c08da00 con 0x7f9e70107bd0 2024-03-21T10:03:32.228 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:30.973+0000 7f9e6fe00700 1 --2- 172.21.15.153:0/3581763721 >> [v2:172.21.15.153:6818/2844395219,v1:172.21.15.153:6819/2844395219] conn(0x7f9e540721c0 0x7f9e540745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:03:32.228 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:30.973+0000 7f9e6f400700 1 --2- 172.21.15.153:0/3581763721 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f9e5406c4e0 0x7f9e5406e9a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9e60006010 tx=0x7f9e6001a040 comp rx=0 tx=0).ready entity=mgr.14221 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:03:32.228 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:30.973+0000 7f9e6fe00700 1 --2- 172.21.15.153:0/3581763721 >> [v2:172.21.15.153:6818/2844395219,v1:172.21.15.153:6819/2844395219] conn(0x7f9e540721c0 0x7f9e540745e0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:03:32.229 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:30.974+0000 7f9e67e00700 1 -- 172.21.15.153:0/3581763721 <== osd.2 v2:172.21.15.153:6818/2844395219 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f9e54074c90 con 0x7f9e540721c0 2024-03-21T10:03:32.229 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:30.991+0000 7f9e74c00700 1 -- 172.21.15.153:0/3581763721 --> [v2:172.21.15.153:6818/2844395219,v1:172.21.15.153:6819/2844395219] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9e7019d730 con 0x7f9e540721c0 2024-03-21T10:03:32.229 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:30.991+0000 7f9e67e00700 1 -- 172.21.15.153:0/3581763721 <== osd.2 v2:172.21.15.153:6818/2844395219 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f9e7019d730 con 0x7f9e540721c0 2024-03-21T10:03:32.229 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:30.992+0000 7f9e74c00700 1 -- 172.21.15.153:0/3581763721 >> [v2:172.21.15.153:6818/2844395219,v1:172.21.15.153:6819/2844395219] conn(0x7f9e540721c0 msgr2=0x7f9e540745e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:03:32.229 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:30.992+0000 7f9e74c00700 1 --2- 172.21.15.153:0/3581763721 >> [v2:172.21.15.153:6818/2844395219,v1:172.21.15.153:6819/2844395219] conn(0x7f9e540721c0 0x7f9e540745e0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:32.230 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:30.992+0000 7f9e74c00700 1 -- 172.21.15.153:0/3581763721 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f9e5406c4e0 msgr2=0x7f9e5406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:03:32.230 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:30.992+0000 7f9e74c00700 1 --2- 172.21.15.153:0/3581763721 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f9e5406c4e0 0x7f9e5406e9a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9e60006010 tx=0x7f9e6001a040 comp rx=0 tx=0).stop 2024-03-21T10:03:32.230 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:30.992+0000 7f9e74c00700 1 -- 172.21.15.153:0/3581763721 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9e70107bd0 msgr2=0x7f9e701a2f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:03:32.230 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:30.992+0000 7f9e74c00700 1 --2- 172.21.15.153:0/3581763721 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9e70107bd0 0x7f9e701a2f60 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c00d900 tx=0x7f9e5c00dcc0 comp rx=0 tx=0).stop 2024-03-21T10:03:32.230 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:30.992+0000 7f9e74c00700 1 -- 172.21.15.153:0/3581763721 shutdown_connections 2024-03-21T10:03:32.230 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:30.992+0000 7f9e74c00700 1 --2- 172.21.15.153:0/3581763721 >> [v2:172.21.15.153:6818/2844395219,v1:172.21.15.153:6819/2844395219] conn(0x7f9e540721c0 0x7f9e540745e0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:32.230 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:30.992+0000 7f9e74c00700 1 --2- 172.21.15.153:0/3581763721 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f9e5406c4e0 0x7f9e5406e9a0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:32.230 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:30.992+0000 7f9e74c00700 1 --2- 172.21.15.153:0/3581763721 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9e70107220 0x7f9e701a2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:32.230 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:30.992+0000 7f9e74c00700 1 --2- 172.21.15.153:0/3581763721 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9e70107bd0 0x7f9e701a2f60 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:32.230 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:30.992+0000 7f9e74c00700 1 -- 172.21.15.153:0/3581763721 >> 172.21.15.153:0/3581763721 conn(0x7f9e700767e0 msgr2=0x7f9e70077e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:03:32.230 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:30.992+0000 7f9e74c00700 1 -- 172.21.15.153:0/3581763721 shutdown_connections 2024-03-21T10:03:32.230 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:30.992+0000 7f9e74c00700 1 -- 172.21.15.153:0/3581763721 wait complete. 2024-03-21T10:03:32.735 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:32.733+0000 7fe541400700 1 -- 172.21.15.153:0/4224675153 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe53c1039b0 msgr2=0x7fe53c107a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:03:32.735 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:32.733+0000 7fe541400700 1 --2- 172.21.15.153:0/4224675153 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe53c1039b0 0x7fe53c107a00 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe528009910 tx=0x7fe528009c20 comp rx=0 tx=0).stop 2024-03-21T10:03:32.735 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:32.733+0000 7fe541400700 1 -- 172.21.15.153:0/4224675153 shutdown_connections 2024-03-21T10:03:32.735 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:32.733+0000 7fe541400700 1 --2- 172.21.15.153:0/4224675153 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe53c1039b0 0x7fe53c107a00 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:32.735 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:32.733+0000 7fe541400700 1 --2- 172.21.15.153:0/4224675153 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe53c103000 0x7fe53c1033e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:32.735 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:32.733+0000 7fe541400700 1 -- 172.21.15.153:0/4224675153 >> 172.21.15.153:0/4224675153 conn(0x7fe53c0fe8d0 msgr2=0x7fe53c100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:03:32.735 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:32.733+0000 7fe541400700 1 -- 172.21.15.153:0/4224675153 shutdown_connections 2024-03-21T10:03:32.736 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:32.733+0000 7fe541400700 1 -- 172.21.15.153:0/4224675153 wait complete. 2024-03-21T10:03:32.736 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:32.735+0000 7fe541400700 1 Processor -- start 2024-03-21T10:03:32.736 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:32.735+0000 7fe541400700 1 -- start start 2024-03-21T10:03:32.737 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:32.735+0000 7fe541400700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe53c103000 0x7fe53c0752a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:03:32.737 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:32.735+0000 7fe541400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe53c1039b0 0x7fe53c0757e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:03:32.737 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:32.735+0000 7fe541400700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe53c079430 con 0x7fe53c1039b0 2024-03-21T10:03:32.737 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:32.735+0000 7fe541400700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe53c075d20 con 0x7fe53c103000 2024-03-21T10:03:32.738 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:32.735+0000 7fe539e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe53c1039b0 0x7fe53c0757e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:03:32.738 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:32.735+0000 7fe539e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe53c1039b0 0x7fe53c0757e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41928/0 (socket says 172.21.15.153:41928) 2024-03-21T10:03:32.739 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:32.735+0000 7fe539e00700 1 -- 172.21.15.153:0/3903774078 learned_addr learned my addr 172.21.15.153:0/3903774078 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:03:32.739 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:32.736+0000 7fe53a800700 1 --2- 172.21.15.153:0/3903774078 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe53c103000 0x7fe53c0752a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:03:32.739 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:32.736+0000 7fe539e00700 1 -- 172.21.15.153:0/3903774078 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe53c103000 msgr2=0x7fe53c0752a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:03:32.739 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:32.736+0000 7fe539e00700 1 --2- 172.21.15.153:0/3903774078 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe53c103000 0x7fe53c0752a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:32.739 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:32.736+0000 7fe539e00700 1 -- 172.21.15.153:0/3903774078 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5280095f0 con 0x7fe53c1039b0 2024-03-21T10:03:32.739 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:32.736+0000 7fe539e00700 1 --2- 172.21.15.153:0/3903774078 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe53c1039b0 0x7fe53c0757e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe52800bda0 tx=0x7fe52800be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:03:32.739 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:32.737+0000 7fe533400700 1 -- 172.21.15.153:0/3903774078 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe52801d070 con 0x7fe53c1039b0 2024-03-21T10:03:32.739 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:32.737+0000 7fe533400700 1 -- 172.21.15.153:0/3903774078 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fe52800f460 con 0x7fe53c1039b0 2024-03-21T10:03:32.740 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:32.737+0000 7fe533400700 1 -- 172.21.15.153:0/3903774078 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe528017630 con 0x7fe53c1039b0 2024-03-21T10:03:32.740 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:32.737+0000 7fe541400700 1 -- 172.21.15.153:0/3903774078 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe53c075fa0 con 0x7fe53c1039b0 2024-03-21T10:03:32.740 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:32.737+0000 7fe541400700 1 -- 172.21.15.153:0/3903774078 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe53c1a7170 con 0x7fe53c1039b0 2024-03-21T10:03:32.742 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:32.738+0000 7fe541400700 1 -- 172.21.15.153:0/3903774078 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe53c1101e0 con 0x7fe53c1039b0 2024-03-21T10:03:32.742 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:32.739+0000 7fe533400700 1 -- 172.21.15.153:0/3903774078 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fe52800fab0 con 0x7fe53c1039b0 2024-03-21T10:03:32.742 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:32.740+0000 7fe533400700 1 --2- 172.21.15.153:0/3903774078 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fe52006c4e0 0x7fe52006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:03:32.742 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:32.740+0000 7fe533400700 1 -- 172.21.15.153:0/3903774078 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fe52808c930 con 0x7fe53c1039b0 2024-03-21T10:03:32.742 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:32.740+0000 7fe53a800700 1 --2- 172.21.15.153:0/3903774078 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fe52006c4e0 0x7fe52006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:03:32.742 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:32.740+0000 7fe533400700 1 --2- 172.21.15.153:0/3903774078 >> [v2:172.21.15.153:6810/3504985760,v1:172.21.15.153:6811/3504985760] conn(0x7fe5200721c0 0x7fe5200745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:03:32.743 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:32.740+0000 7fe533400700 1 -- 172.21.15.153:0/3903774078 --> [v2:172.21.15.153:6810/3504985760,v1:172.21.15.153:6811/3504985760] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe520074c90 con 0x7fe5200721c0 2024-03-21T10:03:32.743 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:32.740+0000 7fe533400700 1 -- 172.21.15.153:0/3903774078 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe5280912a0 con 0x7fe53c1039b0 2024-03-21T10:03:32.743 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:32.741+0000 7fe53b200700 1 --2- 172.21.15.153:0/3903774078 >> [v2:172.21.15.153:6810/3504985760,v1:172.21.15.153:6811/3504985760] conn(0x7fe5200721c0 0x7fe5200745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:03:32.743 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:32.741+0000 7fe53a800700 1 --2- 172.21.15.153:0/3903774078 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fe52006c4e0 0x7fe52006e9a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe52c005950 tx=0x7fe52c0058e0 comp rx=0 tx=0).ready entity=mgr.14221 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:03:32.743 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:32.741+0000 7fe53b200700 1 --2- 172.21.15.153:0/3903774078 >> [v2:172.21.15.153:6810/3504985760,v1:172.21.15.153:6811/3504985760] conn(0x7fe5200721c0 0x7fe5200745e0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:03:32.743 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:32.742+0000 7fe533400700 1 -- 172.21.15.153:0/3903774078 <== osd.1 v2:172.21.15.153:6810/3504985760 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fe520074c90 con 0x7fe5200721c0 2024-03-21T10:03:32.761 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:32.759+0000 7fe541400700 1 -- 172.21.15.153:0/3903774078 --> [v2:172.21.15.153:6810/3504985760,v1:172.21.15.153:6811/3504985760] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe53c04ea90 con 0x7fe5200721c0 2024-03-21T10:03:32.762 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:32.759+0000 7fe533400700 1 -- 172.21.15.153:0/3903774078 <== osd.1 v2:172.21.15.153:6810/3504985760 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fe53c04ea90 con 0x7fe5200721c0 2024-03-21T10:03:32.763 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:32.759+0000 7fe541400700 1 -- 172.21.15.153:0/3903774078 >> [v2:172.21.15.153:6810/3504985760,v1:172.21.15.153:6811/3504985760] conn(0x7fe5200721c0 msgr2=0x7fe5200745e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:03:32.763 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:32.759+0000 7fe541400700 1 --2- 172.21.15.153:0/3903774078 >> [v2:172.21.15.153:6810/3504985760,v1:172.21.15.153:6811/3504985760] conn(0x7fe5200721c0 0x7fe5200745e0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:32.763 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:32.759+0000 7fe541400700 1 -- 172.21.15.153:0/3903774078 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fe52006c4e0 msgr2=0x7fe52006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:03:32.763 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:32.759+0000 7fe541400700 1 --2- 172.21.15.153:0/3903774078 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fe52006c4e0 0x7fe52006e9a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe52c005950 tx=0x7fe52c0058e0 comp rx=0 tx=0).stop 2024-03-21T10:03:32.763 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:32.759+0000 7fe541400700 1 -- 172.21.15.153:0/3903774078 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe53c1039b0 msgr2=0x7fe53c0757e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:03:32.763 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:32.759+0000 7fe541400700 1 --2- 172.21.15.153:0/3903774078 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe53c1039b0 0x7fe53c0757e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe52800bda0 tx=0x7fe52800be80 comp rx=0 tx=0).stop 2024-03-21T10:03:32.763 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:32.759+0000 7fe541400700 1 -- 172.21.15.153:0/3903774078 shutdown_connections 2024-03-21T10:03:32.763 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:32.759+0000 7fe541400700 1 --2- 172.21.15.153:0/3903774078 >> [v2:172.21.15.153:6810/3504985760,v1:172.21.15.153:6811/3504985760] conn(0x7fe5200721c0 0x7fe5200745e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:32.764 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:32.759+0000 7fe541400700 1 --2- 172.21.15.153:0/3903774078 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fe52006c4e0 0x7fe52006e9a0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:32.764 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:32.759+0000 7fe541400700 1 --2- 172.21.15.153:0/3903774078 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe53c103000 0x7fe53c0752a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:32.764 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:32.759+0000 7fe541400700 1 --2- 172.21.15.153:0/3903774078 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe53c1039b0 0x7fe53c0757e0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:32.764 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:32.759+0000 7fe541400700 1 -- 172.21.15.153:0/3903774078 >> 172.21.15.153:0/3903774078 conn(0x7fe53c0fe8d0 msgr2=0x7fe53c100cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:03:32.764 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:32.759+0000 7fe541400700 1 -- 172.21.15.153:0/3903774078 shutdown_connections 2024-03-21T10:03:32.764 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:32.759+0000 7fe541400700 1 -- 172.21.15.153:0/3903774078 wait complete. 2024-03-21T10:03:33.032 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:33.030+0000 7f3b4e400700 1 -- 172.21.15.153:0/2647147769 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3b481071d0 msgr2=0x7f3b48107650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:03:33.032 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:33.030+0000 7f3b4e400700 1 --2- 172.21.15.153:0/2647147769 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3b481071d0 0x7f3b48107650 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f3b30009910 tx=0x7f3b30009c20 comp rx=0 tx=0).stop 2024-03-21T10:03:33.032 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:33.030+0000 7f3b4e400700 1 -- 172.21.15.153:0/2647147769 shutdown_connections 2024-03-21T10:03:33.032 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:33.030+0000 7f3b4e400700 1 --2- 172.21.15.153:0/2647147769 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3b481071d0 0x7f3b48107650 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:33.033 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:33.030+0000 7f3b4e400700 1 --2- 172.21.15.153:0/2647147769 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3b4810d280 0x7f3b4810d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:33.033 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:33.030+0000 7f3b4e400700 1 -- 172.21.15.153:0/2647147769 >> 172.21.15.153:0/2647147769 conn(0x7f3b480769e0 msgr2=0x7f3b48078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:03:33.033 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:33.030+0000 7f3b4e400700 1 -- 172.21.15.153:0/2647147769 shutdown_connections 2024-03-21T10:03:33.033 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:33.030+0000 7f3b4e400700 1 -- 172.21.15.153:0/2647147769 wait complete. 2024-03-21T10:03:33.033 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:33.032+0000 7f3b4e400700 1 Processor -- start 2024-03-21T10:03:33.033 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:33.032+0000 7f3b4e400700 1 -- start start 2024-03-21T10:03:33.034 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:33.032+0000 7f3b4e400700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3b481071d0 0x7f3b4819cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:03:33.034 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:33.032+0000 7f3b4e400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3b4810d280 0x7f3b4819d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:03:33.035 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:33.032+0000 7f3b4e400700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b4819d9b0 con 0x7f3b4810d280 2024-03-21T10:03:33.035 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:33.032+0000 7f3b4e400700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b481a1740 con 0x7f3b481071d0 2024-03-21T10:03:33.035 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:33.033+0000 7f3b3fe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3b4810d280 0x7f3b4819d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:03:33.035 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:33.033+0000 7f3b3fe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3b4810d280 0x7f3b4819d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:54342/0 (socket says 172.21.15.153:54342) 2024-03-21T10:03:33.035 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:33.033+0000 7f3b3fe00700 1 -- 172.21.15.153:0/723820468 learned_addr learned my addr 172.21.15.153:0/723820468 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:03:33.035 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:33.033+0000 7f3b47400700 1 --2- 172.21.15.153:0/723820468 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3b481071d0 0x7f3b4819cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:03:33.035 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:33.033+0000 7f3b3fe00700 1 -- 172.21.15.153:0/723820468 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3b481071d0 msgr2=0x7f3b4819cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:03:33.035 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:33.033+0000 7f3b3fe00700 1 --2- 172.21.15.153:0/723820468 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3b481071d0 0x7f3b4819cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:33.035 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:33.033+0000 7f3b3fe00700 1 -- 172.21.15.153:0/723820468 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b300095f0 con 0x7f3b4810d280 2024-03-21T10:03:33.036 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:33.034+0000 7f3b3fe00700 1 --2- 172.21.15.153:0/723820468 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3b4810d280 0x7f3b4819d2d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3b3000b5c0 tx=0x7f3b3000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:03:33.036 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:33.034+0000 7f3b44c00700 1 -- 172.21.15.153:0/723820468 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3b3001d070 con 0x7f3b4810d280 2024-03-21T10:03:33.036 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:33.034+0000 7f3b44c00700 1 -- 172.21.15.153:0/723820468 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f3b30004fa0 con 0x7f3b4810d280 2024-03-21T10:03:33.036 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:33.034+0000 7f3b44c00700 1 -- 172.21.15.153:0/723820468 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3b3000f460 con 0x7f3b4810d280 2024-03-21T10:03:33.036 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:33.034+0000 7f3b4e400700 1 -- 172.21.15.153:0/723820468 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b481a19c0 con 0x7f3b4810d280 2024-03-21T10:03:33.036 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:33.034+0000 7f3b4e400700 1 -- 172.21.15.153:0/723820468 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b481a1eb0 con 0x7f3b4810d280 2024-03-21T10:03:33.037 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:33.035+0000 7f3b4e400700 1 -- 172.21.15.153:0/723820468 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3b481146a0 con 0x7f3b4810d280 2024-03-21T10:03:33.037 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:33.036+0000 7f3b44c00700 1 -- 172.21.15.153:0/723820468 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f3b30022810 con 0x7f3b4810d280 2024-03-21T10:03:33.038 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:33.037+0000 7f3b44c00700 1 --2- 172.21.15.153:0/723820468 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f3b2c06c4e0 0x7f3b2c06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:03:33.039 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:33.037+0000 7f3b44c00700 1 -- 172.21.15.153:0/723820468 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f3b3008d160 con 0x7f3b4810d280 2024-03-21T10:03:33.039 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:33.038+0000 7f3b47400700 1 --2- 172.21.15.153:0/723820468 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f3b2c06c4e0 0x7f3b2c06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:03:33.039 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:33.038+0000 7f3b44c00700 1 --2- 172.21.15.153:0/723820468 >> [v2:172.21.15.153:6802/1050381948,v1:172.21.15.153:6803/1050381948] conn(0x7f3b2c0721c0 0x7f3b2c0745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:03:33.039 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:33.038+0000 7f3b44c00700 1 -- 172.21.15.153:0/723820468 --> [v2:172.21.15.153:6802/1050381948,v1:172.21.15.153:6803/1050381948] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3b2c074c90 con 0x7f3b2c0721c0 2024-03-21T10:03:33.039 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:33.038+0000 7f3b44c00700 1 -- 172.21.15.153:0/723820468 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3b3008d5c0 con 0x7f3b4810d280 2024-03-21T10:03:33.040 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:33.038+0000 7f3b47e00700 1 --2- 172.21.15.153:0/723820468 >> [v2:172.21.15.153:6802/1050381948,v1:172.21.15.153:6803/1050381948] conn(0x7f3b2c0721c0 0x7f3b2c0745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:03:33.040 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:33.038+0000 7f3b47400700 1 --2- 172.21.15.153:0/723820468 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f3b2c06c4e0 0x7f3b2c06e9a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3b38005950 tx=0x7f3b380058e0 comp rx=0 tx=0).ready entity=mgr.14221 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:03:33.040 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:33.039+0000 7f3b47e00700 1 --2- 172.21.15.153:0/723820468 >> [v2:172.21.15.153:6802/1050381948,v1:172.21.15.153:6803/1050381948] conn(0x7f3b2c0721c0 0x7f3b2c0745e0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:03:33.041 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:33.040+0000 7f3b44c00700 1 -- 172.21.15.153:0/723820468 <== osd.0 v2:172.21.15.153:6802/1050381948 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f3b2c074c90 con 0x7f3b2c0721c0 2024-03-21T10:03:33.059 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:33.057+0000 7f3b4e400700 1 -- 172.21.15.153:0/723820468 --> [v2:172.21.15.153:6802/1050381948,v1:172.21.15.153:6803/1050381948] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3b4804ea90 con 0x7f3b2c0721c0 2024-03-21T10:03:33.059 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:33.058+0000 7f3b44c00700 1 -- 172.21.15.153:0/723820468 <== osd.0 v2:172.21.15.153:6802/1050381948 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f3b4804ea90 con 0x7f3b2c0721c0 2024-03-21T10:03:33.060 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:33.058+0000 7f3b4e400700 1 -- 172.21.15.153:0/723820468 >> [v2:172.21.15.153:6802/1050381948,v1:172.21.15.153:6803/1050381948] conn(0x7f3b2c0721c0 msgr2=0x7f3b2c0745e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:03:33.060 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:33.058+0000 7f3b4e400700 1 --2- 172.21.15.153:0/723820468 >> [v2:172.21.15.153:6802/1050381948,v1:172.21.15.153:6803/1050381948] conn(0x7f3b2c0721c0 0x7f3b2c0745e0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:33.060 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:33.059+0000 7f3b4e400700 1 -- 172.21.15.153:0/723820468 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f3b2c06c4e0 msgr2=0x7f3b2c06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:03:33.061 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:33.059+0000 7f3b4e400700 1 --2- 172.21.15.153:0/723820468 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f3b2c06c4e0 0x7f3b2c06e9a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3b38005950 tx=0x7f3b380058e0 comp rx=0 tx=0).stop 2024-03-21T10:03:33.061 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:33.059+0000 7f3b4e400700 1 -- 172.21.15.153:0/723820468 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3b4810d280 msgr2=0x7f3b4819d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:03:33.061 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:33.059+0000 7f3b4e400700 1 --2- 172.21.15.153:0/723820468 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3b4810d280 0x7f3b4819d2d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3b3000b5c0 tx=0x7f3b3000be20 comp rx=0 tx=0).stop 2024-03-21T10:03:33.061 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:33.059+0000 7f3b4e400700 1 -- 172.21.15.153:0/723820468 shutdown_connections 2024-03-21T10:03:33.061 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:33.059+0000 7f3b4e400700 1 --2- 172.21.15.153:0/723820468 >> [v2:172.21.15.153:6802/1050381948,v1:172.21.15.153:6803/1050381948] conn(0x7f3b2c0721c0 0x7f3b2c0745e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:33.061 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:33.059+0000 7f3b4e400700 1 --2- 172.21.15.153:0/723820468 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f3b2c06c4e0 0x7f3b2c06e9a0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:33.061 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:33.059+0000 7f3b4e400700 1 --2- 172.21.15.153:0/723820468 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3b481071d0 0x7f3b4819cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:33.061 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:33.059+0000 7f3b4e400700 1 --2- 172.21.15.153:0/723820468 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3b4810d280 0x7f3b4819d2d0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:33.061 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:33.059+0000 7f3b4e400700 1 -- 172.21.15.153:0/723820468 >> 172.21.15.153:0/723820468 conn(0x7f3b480769e0 msgr2=0x7f3b48078480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:03:33.061 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:33.059+0000 7f3b4e400700 1 -- 172.21.15.153:0/723820468 shutdown_connections 2024-03-21T10:03:33.062 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:33.060+0000 7f3b4e400700 1 -- 172.21.15.153:0/723820468 wait complete. 2024-03-21T10:03:33.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:03:33 smithi174 ceph-mon[37496]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:03:33.768 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:03:33 smithi153 ceph-mon[29117]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:03:33.824 INFO:teuthology.orchestra.run.smithi153.stdout:38654705708 2024-03-21T10:03:33.825 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 070ed3e4-e769-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.0 2024-03-21T10:03:34.205 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:34.203+0000 7f2527800700 1 -- 172.21.15.153:0/317053954 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f25200731c0 msgr2=0x7f25200735a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:03:34.205 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:34.203+0000 7f2527800700 1 --2- 172.21.15.153:0/317053954 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f25200731c0 0x7f25200735a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f2514009910 tx=0x7f2514009c20 comp rx=0 tx=0).stop 2024-03-21T10:03:34.205 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:34.203+0000 7f2527800700 1 -- 172.21.15.153:0/317053954 shutdown_connections 2024-03-21T10:03:34.205 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:34.203+0000 7f2527800700 1 --2- 172.21.15.153:0/317053954 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2520073ae0 0x7f252010ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:34.205 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:34.203+0000 7f2527800700 1 --2- 172.21.15.153:0/317053954 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f25200731c0 0x7f25200735a0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:34.205 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:34.203+0000 7f2527800700 1 -- 172.21.15.153:0/317053954 >> 172.21.15.153:0/317053954 conn(0x7f25200fc630 msgr2=0x7f25200fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:03:34.205 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:34.203+0000 7f2527800700 1 -- 172.21.15.153:0/317053954 shutdown_connections 2024-03-21T10:03:34.205 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:34.203+0000 7f2527800700 1 -- 172.21.15.153:0/317053954 wait complete. 2024-03-21T10:03:34.207 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:34.204+0000 7f2527800700 1 Processor -- start 2024-03-21T10:03:34.208 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:34.204+0000 7f2527800700 1 -- start start 2024-03-21T10:03:34.208 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:34.205+0000 7f2527800700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f25200731c0 0x7f252019e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:03:34.208 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:34.205+0000 7f2527800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2520073ae0 0x7f252019ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:03:34.208 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:34.205+0000 7f2527800700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f252019f360 con 0x7f2520073ae0 2024-03-21T10:03:34.208 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:34.205+0000 7f2527800700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2520198860 con 0x7f25200731c0 2024-03-21T10:03:34.208 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:34.205+0000 7f251fe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2520073ae0 0x7f252019ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:03:34.208 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:34.205+0000 7f251fe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2520073ae0 0x7f252019ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:54366/0 (socket says 172.21.15.153:54366) 2024-03-21T10:03:34.208 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:34.205+0000 7f251fe00700 1 -- 172.21.15.153:0/1396761474 learned_addr learned my addr 172.21.15.153:0/1396761474 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:03:34.208 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:34.205+0000 7f2524e00700 1 --2- 172.21.15.153:0/1396761474 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f25200731c0 0x7f252019e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:03:34.209 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:34.205+0000 7f251fe00700 1 -- 172.21.15.153:0/1396761474 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f25200731c0 msgr2=0x7f252019e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:03:34.209 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:34.206+0000 7f251fe00700 1 --2- 172.21.15.153:0/1396761474 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f25200731c0 0x7f252019e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:34.209 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:34.206+0000 7f251fe00700 1 -- 172.21.15.153:0/1396761474 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25140095f0 con 0x7f2520073ae0 2024-03-21T10:03:34.209 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:34.206+0000 7f251fe00700 1 --2- 172.21.15.153:0/1396761474 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2520073ae0 0x7f252019ecd0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f250c00b4b0 tx=0x7f250c00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:03:34.209 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:34.206+0000 7f251d600700 1 -- 172.21.15.153:0/1396761474 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f250c011800 con 0x7f2520073ae0 2024-03-21T10:03:34.209 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:34.206+0000 7f251d600700 1 -- 172.21.15.153:0/1396761474 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f250c011e10 con 0x7f2520073ae0 2024-03-21T10:03:34.209 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:34.206+0000 7f2527800700 1 -- 172.21.15.153:0/1396761474 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2520198b40 con 0x7f2520073ae0 2024-03-21T10:03:34.210 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:34.206+0000 7f251d600700 1 -- 172.21.15.153:0/1396761474 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f250c00f2e0 con 0x7f2520073ae0 2024-03-21T10:03:34.210 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:34.206+0000 7f2527800700 1 -- 172.21.15.153:0/1396761474 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2520199090 con 0x7f2520073ae0 2024-03-21T10:03:34.211 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:34.207+0000 7f2527800700 1 -- 172.21.15.153:0/1396761474 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f25201102e0 con 0x7f2520073ae0 2024-03-21T10:03:34.211 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:34.209+0000 7f251d600700 1 -- 172.21.15.153:0/1396761474 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f250c00f440 con 0x7f2520073ae0 2024-03-21T10:03:34.212 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:34.209+0000 7f251d600700 1 --2- 172.21.15.153:0/1396761474 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f250806c4e0 0x7f250806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:03:34.212 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:34.210+0000 7f251d600700 1 -- 172.21.15.153:0/1396761474 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f250c08b070 con 0x7f2520073ae0 2024-03-21T10:03:34.212 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:34.210+0000 7f2524e00700 1 --2- 172.21.15.153:0/1396761474 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f250806c4e0 0x7f250806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:03:34.212 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:34.210+0000 7f251d600700 1 --2- 172.21.15.153:0/1396761474 >> [v2:172.21.15.174:6816/184680961,v1:172.21.15.174:6817/184680961] conn(0x7f25080721c0 0x7f25080745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:03:34.212 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:34.210+0000 7f251d600700 1 -- 172.21.15.153:0/1396761474 --> [v2:172.21.15.174:6816/184680961,v1:172.21.15.174:6817/184680961] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2508074c90 con 0x7f25080721c0 2024-03-21T10:03:34.212 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:34.210+0000 7f251d600700 1 -- 172.21.15.153:0/1396761474 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f250c08d740 con 0x7f2520073ae0 2024-03-21T10:03:34.212 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:34.211+0000 7f2525800700 1 --2- 172.21.15.153:0/1396761474 >> [v2:172.21.15.174:6816/184680961,v1:172.21.15.174:6817/184680961] conn(0x7f25080721c0 0x7f25080745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:03:34.213 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:34.211+0000 7f2524e00700 1 --2- 172.21.15.153:0/1396761474 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f250806c4e0 0x7f250806e9a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2514006010 tx=0x7f251401a040 comp rx=0 tx=0).ready entity=mgr.14221 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:03:34.213 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:34.211+0000 7f2525800700 1 --2- 172.21.15.153:0/1396761474 >> [v2:172.21.15.174:6816/184680961,v1:172.21.15.174:6817/184680961] conn(0x7f25080721c0 0x7f25080745e0 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:03:34.215 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:34.214+0000 7f251d600700 1 -- 172.21.15.153:0/1396761474 <== osd.5 v2:172.21.15.174:6816/184680961 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f2508074c90 con 0x7f25080721c0 2024-03-21T10:03:34.233 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:34.231+0000 7f2527800700 1 -- 172.21.15.153:0/1396761474 --> [v2:172.21.15.174:6816/184680961,v1:172.21.15.174:6817/184680961] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f252004ea90 con 0x7f25080721c0 2024-03-21T10:03:34.234 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:34.232+0000 7f251d600700 1 -- 172.21.15.153:0/1396761474 <== osd.5 v2:172.21.15.174:6816/184680961 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f252004ea90 con 0x7f25080721c0 2024-03-21T10:03:34.235 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:34.233+0000 7f2527800700 1 -- 172.21.15.153:0/1396761474 >> [v2:172.21.15.174:6816/184680961,v1:172.21.15.174:6817/184680961] conn(0x7f25080721c0 msgr2=0x7f25080745e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:03:34.235 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:34.233+0000 7f2527800700 1 --2- 172.21.15.153:0/1396761474 >> [v2:172.21.15.174:6816/184680961,v1:172.21.15.174:6817/184680961] conn(0x7f25080721c0 0x7f25080745e0 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:34.235 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:34.233+0000 7f2527800700 1 -- 172.21.15.153:0/1396761474 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f250806c4e0 msgr2=0x7f250806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:03:34.235 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:34.233+0000 7f2527800700 1 --2- 172.21.15.153:0/1396761474 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f250806c4e0 0x7f250806e9a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2514006010 tx=0x7f251401a040 comp rx=0 tx=0).stop 2024-03-21T10:03:34.235 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:34.233+0000 7f2527800700 1 -- 172.21.15.153:0/1396761474 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2520073ae0 msgr2=0x7f252019ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:03:34.235 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:34.233+0000 7f2527800700 1 --2- 172.21.15.153:0/1396761474 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2520073ae0 0x7f252019ecd0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f250c00b4b0 tx=0x7f250c00b870 comp rx=0 tx=0).stop 2024-03-21T10:03:34.235 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:34.233+0000 7f2527800700 1 -- 172.21.15.153:0/1396761474 shutdown_connections 2024-03-21T10:03:34.235 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:34.233+0000 7f2527800700 1 --2- 172.21.15.153:0/1396761474 >> [v2:172.21.15.174:6816/184680961,v1:172.21.15.174:6817/184680961] conn(0x7f25080721c0 0x7f25080745e0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:34.235 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:34.233+0000 7f2527800700 1 --2- 172.21.15.153:0/1396761474 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f250806c4e0 0x7f250806e9a0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:34.236 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:34.233+0000 7f2527800700 1 --2- 172.21.15.153:0/1396761474 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f25200731c0 0x7f252019e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:34.236 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:34.233+0000 7f2527800700 1 --2- 172.21.15.153:0/1396761474 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2520073ae0 0x7f252019ecd0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:34.236 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:34.233+0000 7f2527800700 1 -- 172.21.15.153:0/1396761474 >> 172.21.15.153:0/1396761474 conn(0x7f25200fc630 msgr2=0x7f2520107680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:03:34.236 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:34.233+0000 7f2527800700 1 -- 172.21.15.153:0/1396761474 shutdown_connections 2024-03-21T10:03:34.236 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:34.233+0000 7f2527800700 1 -- 172.21.15.153:0/1396761474 wait complete. 2024-03-21T10:03:34.550 INFO:teuthology.orchestra.run.smithi153.stdout:115964117014 2024-03-21T10:03:34.550 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 070ed3e4-e769-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.3 2024-03-21T10:03:34.870 INFO:teuthology.orchestra.run.smithi153.stdout:81604378652 2024-03-21T10:03:34.871 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 070ed3e4-e769-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.2 2024-03-21T10:03:35.182 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:03:35 smithi153 ceph-mon[29117]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:03:35.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:03:35 smithi174 ceph-mon[37496]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:03:35.738 INFO:teuthology.orchestra.run.smithi153.stdout:60129542180 2024-03-21T10:03:35.738 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 070ed3e4-e769-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.1 2024-03-21T10:03:36.291 INFO:teuthology.orchestra.run.smithi153.stdout:150323855376 2024-03-21T10:03:36.292 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 070ed3e4-e769-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.4 2024-03-21T10:03:36.585 INFO:teuthology.orchestra.run.smithi153.stdout:171798691849 2024-03-21T10:03:36.585 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 070ed3e4-e769-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.5 2024-03-21T10:03:36.925 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi153/config 2024-03-21T10:03:36.929 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi153/config 2024-03-21T10:03:36.934 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi153/config 2024-03-21T10:03:36.935 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi153/config 2024-03-21T10:03:36.937 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi153/config 2024-03-21T10:03:37.650 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:03:37 smithi153 ceph-mon[29117]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:03:37.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:03:37 smithi174 ceph-mon[37496]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:03:39.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:03:39 smithi153 ceph-mon[29117]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:03:39.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:03:39 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:03:39.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:03:39 smithi174 ceph-mon[37496]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:03:39.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:03:39 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:03:39.820 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi153/config 2024-03-21T10:03:41.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:03:41 smithi153 ceph-mon[29117]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:03:41.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:03:41 smithi174 ceph-mon[37496]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:03:43.482 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:03:43 smithi153 ceph-mon[29117]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:03:43.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:03:43 smithi174 ceph-mon[37496]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:03:43.846 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:43.379+0000 7fcceee00700 1 -- 172.21.15.153:0/2391059313 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcce8100de0 msgr2=0x7fcce81011c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:03:43.846 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:43.379+0000 7fcceee00700 1 --2- 172.21.15.153:0/2391059313 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcce8100de0 0x7fcce81011c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fccd8009910 tx=0x7fccd8009c20 comp rx=0 tx=0).stop 2024-03-21T10:03:43.846 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:43.379+0000 7fcceee00700 1 -- 172.21.15.153:0/2391059313 shutdown_connections 2024-03-21T10:03:43.846 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:43.379+0000 7fcceee00700 1 --2- 172.21.15.153:0/2391059313 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcce8101790 0x7fcce81057e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:43.847 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:43.379+0000 7fcceee00700 1 --2- 172.21.15.153:0/2391059313 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcce8100de0 0x7fcce81011c0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:43.847 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:43.379+0000 7fcceee00700 1 -- 172.21.15.153:0/2391059313 >> 172.21.15.153:0/2391059313 conn(0x7fcce80fc670 msgr2=0x7fcce80fea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:03:43.847 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:43.380+0000 7fcceee00700 1 -- 172.21.15.153:0/2391059313 shutdown_connections 2024-03-21T10:03:43.847 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:43.380+0000 7fcceee00700 1 -- 172.21.15.153:0/2391059313 wait complete. 2024-03-21T10:03:43.847 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:43.381+0000 7fcceee00700 1 Processor -- start 2024-03-21T10:03:43.847 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:43.381+0000 7fcceee00700 1 -- start start 2024-03-21T10:03:43.847 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:43.381+0000 7fcceee00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcce8100de0 0x7fcce819c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:03:43.847 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:43.381+0000 7fcceee00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcce8101790 0x7fcce819ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:03:43.847 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:43.381+0000 7fcceee00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcce819d120 con 0x7fcce8101790 2024-03-21T10:03:43.847 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:43.381+0000 7fcceee00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcce81965d0 con 0x7fcce8100de0 2024-03-21T10:03:43.847 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:43.382+0000 7fcce7400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcce8101790 0x7fcce819ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:03:43.847 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:43.382+0000 7fcce7400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcce8101790 0x7fcce819ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:40834/0 (socket says 172.21.15.153:40834) 2024-03-21T10:03:43.847 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:43.382+0000 7fcce7400700 1 -- 172.21.15.153:0/114177135 learned_addr learned my addr 172.21.15.153:0/114177135 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:03:43.848 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:43.382+0000 7fcce7e00700 1 --2- 172.21.15.153:0/114177135 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcce8100de0 0x7fcce819c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:03:43.848 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:43.382+0000 7fcce7400700 1 -- 172.21.15.153:0/114177135 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcce8100de0 msgr2=0x7fcce819c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:03:43.848 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:43.382+0000 7fcce7400700 1 --2- 172.21.15.153:0/114177135 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcce8100de0 0x7fcce819c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:43.848 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:43.382+0000 7fcce7400700 1 -- 172.21.15.153:0/114177135 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccd80095f0 con 0x7fcce8101790 2024-03-21T10:03:43.848 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:43.382+0000 7fcce7400700 1 --2- 172.21.15.153:0/114177135 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcce8101790 0x7fcce819ca90 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fccd400d8d0 tx=0x7fccd400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:03:43.848 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:43.383+0000 7fcce4c00700 1 -- 172.21.15.153:0/114177135 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fccd4004a00 con 0x7fcce8101790 2024-03-21T10:03:43.848 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:43.383+0000 7fcce4c00700 1 -- 172.21.15.153:0/114177135 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fccd40054d0 con 0x7fcce8101790 2024-03-21T10:03:43.848 INFO:teuthology.orchestra.run.smithi153.stdout:150323855378 2024-03-21T10:03:43.849 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:43.383+0000 7fcceee00700 1 -- 172.21.15.153:0/114177135 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcce81968b0 con 0x7fcce8101790 2024-03-21T10:03:43.849 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:43.383+0000 7fcce4c00700 1 -- 172.21.15.153:0/114177135 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fccd4009d40 con 0x7fcce8101790 2024-03-21T10:03:43.849 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:43.384+0000 7fcceee00700 1 -- 172.21.15.153:0/114177135 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcce8196e00 con 0x7fcce8101790 2024-03-21T10:03:43.849 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:43.385+0000 7fcceee00700 1 -- 172.21.15.153:0/114177135 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcce8109180 con 0x7fcce8101790 2024-03-21T10:03:43.849 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:43.385+0000 7fcce4c00700 1 -- 172.21.15.153:0/114177135 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fccd4005020 con 0x7fcce8101790 2024-03-21T10:03:43.849 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:43.386+0000 7fcce4c00700 1 --2- 172.21.15.153:0/114177135 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fcccc06c4e0 0x7fcccc06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:03:43.849 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:43.386+0000 7fcce4c00700 1 -- 172.21.15.153:0/114177135 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fccd408ae00 con 0x7fcce8101790 2024-03-21T10:03:43.849 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:43.387+0000 7fcce7e00700 1 --2- 172.21.15.153:0/114177135 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fcccc06c4e0 0x7fcccc06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:03:43.849 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:43.387+0000 7fcce7e00700 1 --2- 172.21.15.153:0/114177135 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fcccc06c4e0 0x7fcccc06e9a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fccd8006010 tx=0x7fccd801a040 comp rx=0 tx=0).ready entity=mgr.14221 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:03:43.850 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:43.392+0000 7fcce4c00700 1 -- 172.21.15.153:0/114177135 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fccd4059390 con 0x7fcce8101790 2024-03-21T10:03:43.850 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:43.621+0000 7fcceee00700 1 -- 172.21.15.153:0/114177135 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fcce81970f0 con 0x7fcce8101790 2024-03-21T10:03:43.850 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:43.622+0000 7fcce4c00700 1 -- 172.21.15.153:0/114177135 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fccd4017090 con 0x7fcce8101790 2024-03-21T10:03:43.850 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:43.625+0000 7fcceee00700 1 -- 172.21.15.153:0/114177135 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fcccc06c4e0 msgr2=0x7fcccc06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:03:43.850 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:43.625+0000 7fcceee00700 1 --2- 172.21.15.153:0/114177135 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fcccc06c4e0 0x7fcccc06e9a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fccd8006010 tx=0x7fccd801a040 comp rx=0 tx=0).stop 2024-03-21T10:03:43.850 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:43.625+0000 7fcceee00700 1 -- 172.21.15.153:0/114177135 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcce8101790 msgr2=0x7fcce819ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:03:43.850 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:43.625+0000 7fcceee00700 1 --2- 172.21.15.153:0/114177135 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcce8101790 0x7fcce819ca90 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fccd400d8d0 tx=0x7fccd400dc90 comp rx=0 tx=0).stop 2024-03-21T10:03:43.850 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:43.625+0000 7fcceee00700 1 -- 172.21.15.153:0/114177135 shutdown_connections 2024-03-21T10:03:43.850 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:43.625+0000 7fcceee00700 1 --2- 172.21.15.153:0/114177135 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fcccc06c4e0 0x7fcccc06e9a0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:43.850 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:43.625+0000 7fcceee00700 1 --2- 172.21.15.153:0/114177135 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcce8100de0 0x7fcce819c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:43.850 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:43.625+0000 7fcceee00700 1 --2- 172.21.15.153:0/114177135 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcce8101790 0x7fcce819ca90 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:43.850 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:43.625+0000 7fcceee00700 1 -- 172.21.15.153:0/114177135 >> 172.21.15.153:0/114177135 conn(0x7fcce80fc670 msgr2=0x7fcce80fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:03:43.850 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:43.625+0000 7fcceee00700 1 -- 172.21.15.153:0/114177135 shutdown_connections 2024-03-21T10:03:43.850 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:43.625+0000 7fcceee00700 1 -- 172.21.15.153:0/114177135 wait complete. 2024-03-21T10:03:44.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:03:44 smithi153 ceph-mon[29117]: from='client.? 172.21.15.153:0/114177135' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-21T10:03:44.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:03:44 smithi174 ceph-mon[37496]: from='client.? 172.21.15.153:0/114177135' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-21T10:03:44.836 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:44.834+0000 7f4d58a00700 1 -- 172.21.15.153:0/526139686 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d54102e10 msgr2=0x7f4d541031f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:03:44.836 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:44.834+0000 7f4d58a00700 1 --2- 172.21.15.153:0/526139686 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d54102e10 0x7f4d541031f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4d44009910 tx=0x7f4d44009c20 comp rx=0 tx=0).stop 2024-03-21T10:03:44.836 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:44.834+0000 7f4d58a00700 1 -- 172.21.15.153:0/526139686 shutdown_connections 2024-03-21T10:03:44.836 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:44.834+0000 7f4d58a00700 1 --2- 172.21.15.153:0/526139686 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4d541037c0 0x7f4d54107810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:44.836 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:44.834+0000 7f4d58a00700 1 --2- 172.21.15.153:0/526139686 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d54102e10 0x7f4d541031f0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:44.836 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:44.834+0000 7f4d58a00700 1 -- 172.21.15.153:0/526139686 >> 172.21.15.153:0/526139686 conn(0x7f4d540fe6c0 msgr2=0x7f4d54100ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:03:44.836 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:44.834+0000 7f4d58a00700 1 -- 172.21.15.153:0/526139686 shutdown_connections 2024-03-21T10:03:44.836 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:44.834+0000 7f4d58a00700 1 -- 172.21.15.153:0/526139686 wait complete. 2024-03-21T10:03:44.837 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:44.835+0000 7f4d58a00700 1 Processor -- start 2024-03-21T10:03:44.837 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:44.836+0000 7f4d58a00700 1 -- start start 2024-03-21T10:03:44.838 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:44.836+0000 7f4d58a00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4d54102e10 0x7f4d5419e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:03:44.838 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:44.836+0000 7f4d58a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d541037c0 0x7f4d5419eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:03:44.839 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:44.836+0000 7f4d58a00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d5419f1b0 con 0x7f4d541037c0 2024-03-21T10:03:44.839 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:44.836+0000 7f4d58a00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d54198660 con 0x7f4d54102e10 2024-03-21T10:03:44.839 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:44.836+0000 7f4d51e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d541037c0 0x7f4d5419eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:03:44.839 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:44.836+0000 7f4d51e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d541037c0 0x7f4d5419eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:40866/0 (socket says 172.21.15.153:40866) 2024-03-21T10:03:44.839 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:44.836+0000 7f4d51e00700 1 -- 172.21.15.153:0/447151258 learned_addr learned my addr 172.21.15.153:0/447151258 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:03:44.839 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:44.837+0000 7f4d52800700 1 --2- 172.21.15.153:0/447151258 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4d54102e10 0x7f4d5419e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:03:44.839 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:44.837+0000 7f4d51e00700 1 -- 172.21.15.153:0/447151258 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4d54102e10 msgr2=0x7f4d5419e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:03:44.839 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:44.837+0000 7f4d51e00700 1 --2- 172.21.15.153:0/447151258 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4d54102e10 0x7f4d5419e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:44.839 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:44.837+0000 7f4d51e00700 1 -- 172.21.15.153:0/447151258 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d440095f0 con 0x7f4d541037c0 2024-03-21T10:03:44.839 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:44.837+0000 7f4d51e00700 1 --2- 172.21.15.153:0/447151258 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d541037c0 0x7f4d5419eb20 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4d4000c8f0 tx=0x7f4d4000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:03:44.840 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:44.837+0000 7f4d4b400700 1 -- 172.21.15.153:0/447151258 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4d40007a50 con 0x7f4d541037c0 2024-03-21T10:03:44.840 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:44.837+0000 7f4d4b400700 1 -- 172.21.15.153:0/447151258 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f4d40007bb0 con 0x7f4d541037c0 2024-03-21T10:03:44.840 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:44.837+0000 7f4d58a00700 1 -- 172.21.15.153:0/447151258 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d54198940 con 0x7f4d541037c0 2024-03-21T10:03:44.840 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:44.837+0000 7f4d4b400700 1 -- 172.21.15.153:0/447151258 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4d40018630 con 0x7f4d541037c0 2024-03-21T10:03:44.840 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:44.838+0000 7f4d58a00700 1 -- 172.21.15.153:0/447151258 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d54198e90 con 0x7f4d541037c0 2024-03-21T10:03:44.841 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:44.839+0000 7f4d58a00700 1 -- 172.21.15.153:0/447151258 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d5410b1b0 con 0x7f4d541037c0 2024-03-21T10:03:44.841 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:44.840+0000 7f4d4b400700 1 -- 172.21.15.153:0/447151258 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f4d40007d20 con 0x7f4d541037c0 2024-03-21T10:03:44.842 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:44.840+0000 7f4d4b400700 1 --2- 172.21.15.153:0/447151258 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f4d3806c4e0 0x7f4d3806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:03:44.842 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:44.841+0000 7f4d4b400700 1 -- 172.21.15.153:0/447151258 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f4d4008b550 con 0x7f4d541037c0 2024-03-21T10:03:44.842 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:44.841+0000 7f4d52800700 1 --2- 172.21.15.153:0/447151258 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f4d3806c4e0 0x7f4d3806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:03:44.843 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:44.841+0000 7f4d52800700 1 --2- 172.21.15.153:0/447151258 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f4d3806c4e0 0x7f4d3806e9a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4d4400b5c0 tx=0x7f4d4401a040 comp rx=0 tx=0).ready entity=mgr.14221 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:03:44.847 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:44.846+0000 7f4d4b400700 1 -- 172.21.15.153:0/447151258 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f4d40059a60 con 0x7f4d541037c0 2024-03-21T10:03:45.076 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:45.074+0000 7f4d58a00700 1 -- 172.21.15.153:0/447151258 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f4d54199110 con 0x7f4d541037c0 2024-03-21T10:03:45.076 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:45.075+0000 7f4d4b400700 1 -- 172.21.15.153:0/447151258 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4d400595f0 con 0x7f4d541037c0 2024-03-21T10:03:45.077 INFO:teuthology.orchestra.run.smithi153.stdout:81604378655 2024-03-21T10:03:45.079 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:45.077+0000 7f4d58a00700 1 -- 172.21.15.153:0/447151258 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f4d3806c4e0 msgr2=0x7f4d3806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:03:45.080 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:45.077+0000 7f4d58a00700 1 --2- 172.21.15.153:0/447151258 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f4d3806c4e0 0x7f4d3806e9a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4d4400b5c0 tx=0x7f4d4401a040 comp rx=0 tx=0).stop 2024-03-21T10:03:45.080 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:45.078+0000 7f4d58a00700 1 -- 172.21.15.153:0/447151258 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d541037c0 msgr2=0x7f4d5419eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:03:45.080 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:45.078+0000 7f4d58a00700 1 --2- 172.21.15.153:0/447151258 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d541037c0 0x7f4d5419eb20 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4d4000c8f0 tx=0x7f4d4000ccb0 comp rx=0 tx=0).stop 2024-03-21T10:03:45.080 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:45.078+0000 7f4d58a00700 1 -- 172.21.15.153:0/447151258 shutdown_connections 2024-03-21T10:03:45.080 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:45.078+0000 7f4d58a00700 1 --2- 172.21.15.153:0/447151258 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f4d3806c4e0 0x7f4d3806e9a0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:45.080 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:45.078+0000 7f4d58a00700 1 --2- 172.21.15.153:0/447151258 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4d54102e10 0x7f4d5419e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:45.080 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:45.078+0000 7f4d58a00700 1 --2- 172.21.15.153:0/447151258 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4d541037c0 0x7f4d5419eb20 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:45.080 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:45.078+0000 7f4d58a00700 1 -- 172.21.15.153:0/447151258 >> 172.21.15.153:0/447151258 conn(0x7f4d540fe6c0 msgr2=0x7f4d54100ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:03:45.080 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:45.078+0000 7f4d58a00700 1 -- 172.21.15.153:0/447151258 shutdown_connections 2024-03-21T10:03:45.080 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:45.078+0000 7f4d58a00700 1 -- 172.21.15.153:0/447151258 wait complete. 2024-03-21T10:03:45.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:03:45 smithi174 ceph-mon[37496]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:03:45.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:03:45 smithi174 ceph-mon[37496]: from='client.? 172.21.15.153:0/447151258' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-21T10:03:45.761 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:03:45 smithi153 ceph-mon[29117]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:03:45.762 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:03:45 smithi153 ceph-mon[29117]: from='client.? 172.21.15.153:0/447151258' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-21T10:03:46.659 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:03:46 smithi153 ceph-mon[29117]: from='client.? 172.21.15.153:0/1800066326' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-21T10:03:46.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:03:46 smithi174 ceph-mon[37496]: from='client.? 172.21.15.153:0/1800066326' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-21T10:03:46.724 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.663+0000 7f694ce00700 1 -- 172.21.15.153:0/2723156084 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6948102ff0 msgr2=0x7f69481033d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:03:46.724 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.663+0000 7f694ce00700 1 --2- 172.21.15.153:0/2723156084 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6948102ff0 0x7f69481033d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6938009910 tx=0x7f6938009c20 comp rx=0 tx=0).stop 2024-03-21T10:03:46.724 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.663+0000 7f694ce00700 1 -- 172.21.15.153:0/2723156084 shutdown_connections 2024-03-21T10:03:46.724 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.663+0000 7f694ce00700 1 --2- 172.21.15.153:0/2723156084 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f69481039a0 0x7f69481079f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:46.725 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.663+0000 7f694ce00700 1 --2- 172.21.15.153:0/2723156084 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6948102ff0 0x7f69481033d0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:46.725 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.663+0000 7f694ce00700 1 -- 172.21.15.153:0/2723156084 >> 172.21.15.153:0/2723156084 conn(0x7f69480fe880 msgr2=0x7f6948100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:03:46.725 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.663+0000 7f694ce00700 1 -- 172.21.15.153:0/2723156084 shutdown_connections 2024-03-21T10:03:46.725 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.663+0000 7f694ce00700 1 -- 172.21.15.153:0/2723156084 wait complete. 2024-03-21T10:03:46.725 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.664+0000 7f694ce00700 1 Processor -- start 2024-03-21T10:03:46.725 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.664+0000 7f694ce00700 1 -- start start 2024-03-21T10:03:46.725 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.665+0000 7f694ce00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6948102ff0 0x7f6948198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:03:46.725 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.665+0000 7f694ce00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f69481039a0 0x7f6948198eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:03:46.725 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.665+0000 7f694ce00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69481994b0 con 0x7f6948102ff0 2024-03-21T10:03:46.725 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.665+0000 7f694ce00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6948199620 con 0x7f69481039a0 2024-03-21T10:03:46.725 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.665+0000 7f6945e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6948102ff0 0x7f6948198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:03:46.725 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.665+0000 7f6945e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6948102ff0 0x7f6948198970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:40904/0 (socket says 172.21.15.153:40904) 2024-03-21T10:03:46.725 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.665+0000 7f6945e00700 1 -- 172.21.15.153:0/1941550998 learned_addr learned my addr 172.21.15.153:0/1941550998 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:03:46.726 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.665+0000 7f6945400700 1 --2- 172.21.15.153:0/1941550998 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f69481039a0 0x7f6948198eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:03:46.726 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.665+0000 7f6945e00700 1 -- 172.21.15.153:0/1941550998 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f69481039a0 msgr2=0x7f6948198eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:03:46.726 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.665+0000 7f6945e00700 1 --2- 172.21.15.153:0/1941550998 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f69481039a0 0x7f6948198eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:46.726 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.665+0000 7f6945e00700 1 -- 172.21.15.153:0/1941550998 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69380095f0 con 0x7f6948102ff0 2024-03-21T10:03:46.726 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.666+0000 7f6945e00700 1 --2- 172.21.15.153:0/1941550998 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6948102ff0 0x7f6948198970 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f6938006010 tx=0x7f693800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:03:46.726 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.666+0000 7f693ea00700 1 -- 172.21.15.153:0/1941550998 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f693801d070 con 0x7f6948102ff0 2024-03-21T10:03:46.726 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.666+0000 7f693ea00700 1 -- 172.21.15.153:0/1941550998 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f693800f460 con 0x7f6948102ff0 2024-03-21T10:03:46.726 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.666+0000 7f694ce00700 1 -- 172.21.15.153:0/1941550998 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f694819d440 con 0x7f6948102ff0 2024-03-21T10:03:46.726 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.666+0000 7f694ce00700 1 -- 172.21.15.153:0/1941550998 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f694819d930 con 0x7f6948102ff0 2024-03-21T10:03:46.726 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.666+0000 7f693ea00700 1 -- 172.21.15.153:0/1941550998 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6938017630 con 0x7f6948102ff0 2024-03-21T10:03:46.726 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.668+0000 7f694ce00700 1 -- 172.21.15.153:0/1941550998 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f694810b390 con 0x7f6948102ff0 2024-03-21T10:03:46.726 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.669+0000 7f693ea00700 1 -- 172.21.15.153:0/1941550998 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f693800f5d0 con 0x7f6948102ff0 2024-03-21T10:03:46.726 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.670+0000 7f693ea00700 1 --2- 172.21.15.153:0/1941550998 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f692c06c4e0 0x7f692c06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:03:46.727 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.671+0000 7f693ea00700 1 -- 172.21.15.153:0/1941550998 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f693808ca90 con 0x7f6948102ff0 2024-03-21T10:03:46.727 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.671+0000 7f6945400700 1 --2- 172.21.15.153:0/1941550998 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f692c06c4e0 0x7f692c06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:03:46.727 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.671+0000 7f6945400700 1 --2- 172.21.15.153:0/1941550998 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f692c06c4e0 0x7f692c06e9a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6934005ea0 tx=0x7f6934005e30 comp rx=0 tx=0).ready entity=mgr.14221 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:03:46.727 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.675+0000 7f693ea00700 1 -- 172.21.15.153:0/1941550998 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f693805b020 con 0x7f6948102ff0 2024-03-21T10:03:46.833 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.794+0000 7efe57000700 1 -- 172.21.15.153:0/3964451690 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efe50073ae0 msgr2=0x7efe5010ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:03:46.833 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.794+0000 7efe57000700 1 --2- 172.21.15.153:0/3964451690 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efe50073ae0 0x7efe5010ce40 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7efe3c009910 tx=0x7efe3c009c20 comp rx=0 tx=0).stop 2024-03-21T10:03:46.833 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.794+0000 7efe57000700 1 -- 172.21.15.153:0/3964451690 shutdown_connections 2024-03-21T10:03:46.833 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.794+0000 7efe57000700 1 --2- 172.21.15.153:0/3964451690 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efe50073ae0 0x7efe5010ce40 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:46.833 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.794+0000 7efe57000700 1 --2- 172.21.15.153:0/3964451690 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efe500731c0 0x7efe500735a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:46.833 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.794+0000 7efe57000700 1 -- 172.21.15.153:0/3964451690 >> 172.21.15.153:0/3964451690 conn(0x7efe500fc630 msgr2=0x7efe500fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:03:46.833 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.794+0000 7efe57000700 1 -- 172.21.15.153:0/3964451690 shutdown_connections 2024-03-21T10:03:46.833 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.794+0000 7efe57000700 1 -- 172.21.15.153:0/3964451690 wait complete. 2024-03-21T10:03:46.833 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.795+0000 7efe57000700 1 Processor -- start 2024-03-21T10:03:46.834 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.795+0000 7efe57000700 1 -- start start 2024-03-21T10:03:46.834 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.795+0000 7efe57000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efe500731c0 0x7efe5019e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:03:46.834 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.796+0000 7efe57000700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efe50073ae0 0x7efe5019ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:03:46.834 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.796+0000 7efe57000700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe5019f360 con 0x7efe500731c0 2024-03-21T10:03:46.834 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.796+0000 7efe57000700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe50198810 con 0x7efe50073ae0 2024-03-21T10:03:46.834 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.796+0000 7efe4fe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efe500731c0 0x7efe5019e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:03:46.834 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.796+0000 7efe4fe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efe500731c0 0x7efe5019e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:40930/0 (socket says 172.21.15.153:40930) 2024-03-21T10:03:46.834 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.796+0000 7efe4fe00700 1 -- 172.21.15.153:0/1082871666 learned_addr learned my addr 172.21.15.153:0/1082871666 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:03:46.834 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.796+0000 7efe4f400700 1 --2- 172.21.15.153:0/1082871666 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efe50073ae0 0x7efe5019ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:03:46.834 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.796+0000 7efe4fe00700 1 -- 172.21.15.153:0/1082871666 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efe50073ae0 msgr2=0x7efe5019ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:03:46.834 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.796+0000 7efe4fe00700 1 --2- 172.21.15.153:0/1082871666 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efe50073ae0 0x7efe5019ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:46.834 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.796+0000 7efe4fe00700 1 -- 172.21.15.153:0/1082871666 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe3c0095f0 con 0x7efe500731c0 2024-03-21T10:03:46.835 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.797+0000 7efe4fe00700 1 --2- 172.21.15.153:0/1082871666 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efe500731c0 0x7efe5019e790 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7efe4000b4b0 tx=0x7efe4000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:03:46.835 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.797+0000 7efe4cc00700 1 -- 172.21.15.153:0/1082871666 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe40011800 con 0x7efe500731c0 2024-03-21T10:03:46.835 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.797+0000 7efe57000700 1 -- 172.21.15.153:0/1082871666 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe50198af0 con 0x7efe500731c0 2024-03-21T10:03:46.835 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.797+0000 7efe4cc00700 1 -- 172.21.15.153:0/1082871666 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7efe40011e10 con 0x7efe500731c0 2024-03-21T10:03:46.835 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.797+0000 7efe4cc00700 1 -- 172.21.15.153:0/1082871666 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe4000f2e0 con 0x7efe500731c0 2024-03-21T10:03:46.835 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.797+0000 7efe57000700 1 -- 172.21.15.153:0/1082871666 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe50199040 con 0x7efe500731c0 2024-03-21T10:03:46.835 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.798+0000 7efe57000700 1 -- 172.21.15.153:0/1082871666 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe5010a540 con 0x7efe500731c0 2024-03-21T10:03:46.835 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.800+0000 7efe4cc00700 1 -- 172.21.15.153:0/1082871666 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7efe40010380 con 0x7efe500731c0 2024-03-21T10:03:46.835 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.800+0000 7efe4cc00700 1 --2- 172.21.15.153:0/1082871666 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7efe3406c4e0 0x7efe3406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:03:46.835 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.801+0000 7efe4cc00700 1 -- 172.21.15.153:0/1082871666 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7efe4008b720 con 0x7efe500731c0 2024-03-21T10:03:46.835 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.801+0000 7efe4f400700 1 --2- 172.21.15.153:0/1082871666 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7efe3406c4e0 0x7efe3406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:03:46.835 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.801+0000 7efe4f400700 1 --2- 172.21.15.153:0/1082871666 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7efe3406c4e0 0x7efe3406e9a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7efe500fdd10 tx=0x7efe3c01a040 comp rx=0 tx=0).ready entity=mgr.14221 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:03:46.835 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.808+0000 7efe4cc00700 1 -- 172.21.15.153:0/1082871666 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7efe40059cb0 con 0x7efe500731c0 2024-03-21T10:03:46.925 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.924+0000 7f694ce00700 1 -- 172.21.15.153:0/1941550998 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f694810ad70 con 0x7f6948102ff0 2024-03-21T10:03:46.926 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.924+0000 7f693ea00700 1 -- 172.21.15.153:0/1941550998 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6938055e60 con 0x7f6948102ff0 2024-03-21T10:03:46.926 INFO:teuthology.orchestra.run.smithi153.stdout:60129542182 2024-03-21T10:03:46.929 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.927+0000 7f694ce00700 1 -- 172.21.15.153:0/1941550998 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f692c06c4e0 msgr2=0x7f692c06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:03:46.929 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.927+0000 7f694ce00700 1 --2- 172.21.15.153:0/1941550998 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f692c06c4e0 0x7f692c06e9a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6934005ea0 tx=0x7f6934005e30 comp rx=0 tx=0).stop 2024-03-21T10:03:46.930 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.927+0000 7f694ce00700 1 -- 172.21.15.153:0/1941550998 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6948102ff0 msgr2=0x7f6948198970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:03:46.930 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.927+0000 7f694ce00700 1 --2- 172.21.15.153:0/1941550998 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6948102ff0 0x7f6948198970 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f6938006010 tx=0x7f693800be20 comp rx=0 tx=0).stop 2024-03-21T10:03:46.930 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.927+0000 7f694ce00700 1 -- 172.21.15.153:0/1941550998 shutdown_connections 2024-03-21T10:03:46.930 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.927+0000 7f694ce00700 1 --2- 172.21.15.153:0/1941550998 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f692c06c4e0 0x7f692c06e9a0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:46.930 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.927+0000 7f694ce00700 1 --2- 172.21.15.153:0/1941550998 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6948102ff0 0x7f6948198970 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:46.930 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.927+0000 7f694ce00700 1 --2- 172.21.15.153:0/1941550998 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f69481039a0 0x7f6948198eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:46.930 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.927+0000 7f694ce00700 1 -- 172.21.15.153:0/1941550998 >> 172.21.15.153:0/1941550998 conn(0x7f69480fe880 msgr2=0x7f69480fff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:03:46.930 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.927+0000 7f694ce00700 1 -- 172.21.15.153:0/1941550998 shutdown_connections 2024-03-21T10:03:46.930 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.927+0000 7f694ce00700 1 -- 172.21.15.153:0/1941550998 wait complete. 2024-03-21T10:03:47.026 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.419+0000 7f0a3ce00700 1 -- 172.21.15.153:0/3396727257 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0a381039a0 msgr2=0x7f0a381079f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:03:47.042 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.419+0000 7f0a3ce00700 1 --2- 172.21.15.153:0/3396727257 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0a381039a0 0x7f0a381079f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0a24009910 tx=0x7f0a24009c20 comp rx=0 tx=0).stop 2024-03-21T10:03:47.042 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.420+0000 7f0a3ce00700 1 -- 172.21.15.153:0/3396727257 shutdown_connections 2024-03-21T10:03:47.042 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.420+0000 7f0a3ce00700 1 --2- 172.21.15.153:0/3396727257 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0a381039a0 0x7f0a381079f0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:47.042 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.420+0000 7f0a3ce00700 1 --2- 172.21.15.153:0/3396727257 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0a38102ff0 0x7f0a381033d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:47.042 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.420+0000 7f0a3ce00700 1 -- 172.21.15.153:0/3396727257 >> 172.21.15.153:0/3396727257 conn(0x7f0a380fe880 msgr2=0x7f0a38100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:03:47.042 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.420+0000 7f0a3ce00700 1 -- 172.21.15.153:0/3396727257 shutdown_connections 2024-03-21T10:03:47.042 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.420+0000 7f0a3ce00700 1 -- 172.21.15.153:0/3396727257 wait complete. 2024-03-21T10:03:47.042 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.421+0000 7f0a3ce00700 1 Processor -- start 2024-03-21T10:03:47.042 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.421+0000 7f0a3ce00700 1 -- start start 2024-03-21T10:03:47.042 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.421+0000 7f0a3ce00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0a38102ff0 0x7f0a381989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:03:47.042 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.421+0000 7f0a3ce00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0a381039a0 0x7f0a38198ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:03:47.043 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.421+0000 7f0a3ce00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a38199570 con 0x7f0a38102ff0 2024-03-21T10:03:47.043 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.421+0000 7f0a3ce00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a3819d380 con 0x7f0a381039a0 2024-03-21T10:03:47.043 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.422+0000 7f0a35e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0a38102ff0 0x7f0a381989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:03:47.043 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.422+0000 7f0a35e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0a38102ff0 0x7f0a381989a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:40892/0 (socket says 172.21.15.153:40892) 2024-03-21T10:03:47.043 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.422+0000 7f0a35e00700 1 -- 172.21.15.153:0/3031635190 learned_addr learned my addr 172.21.15.153:0/3031635190 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:03:47.043 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.422+0000 7f0a35400700 1 --2- 172.21.15.153:0/3031635190 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0a381039a0 0x7f0a38198ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:03:47.043 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.422+0000 7f0a35e00700 1 -- 172.21.15.153:0/3031635190 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0a381039a0 msgr2=0x7f0a38198ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:03:47.043 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.422+0000 7f0a35e00700 1 --2- 172.21.15.153:0/3031635190 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0a381039a0 0x7f0a38198ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:47.043 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.422+0000 7f0a35e00700 1 -- 172.21.15.153:0/3031635190 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a240095f0 con 0x7f0a38102ff0 2024-03-21T10:03:47.043 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.422+0000 7f0a35400700 1 --2- 172.21.15.153:0/3031635190 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0a381039a0 0x7f0a38198ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:03:47.043 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.423+0000 7f0a35e00700 1 --2- 172.21.15.153:0/3031635190 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0a38102ff0 0x7f0a381989a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0a2800d8d0 tx=0x7f0a2800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:03:47.043 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.425+0000 7f0a2ea00700 1 -- 172.21.15.153:0/3031635190 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0a28009940 con 0x7f0a38102ff0 2024-03-21T10:03:47.043 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.425+0000 7f0a2ea00700 1 -- 172.21.15.153:0/3031635190 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0a28010460 con 0x7f0a38102ff0 2024-03-21T10:03:47.043 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.425+0000 7f0a3ce00700 1 -- 172.21.15.153:0/3031635190 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a3819d660 con 0x7f0a38102ff0 2024-03-21T10:03:47.044 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.425+0000 7f0a2ea00700 1 -- 172.21.15.153:0/3031635190 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0a2800f5f0 con 0x7f0a38102ff0 2024-03-21T10:03:47.044 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.425+0000 7f0a3ce00700 1 -- 172.21.15.153:0/3031635190 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a3819db80 con 0x7f0a38102ff0 2024-03-21T10:03:47.044 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.426+0000 7f0a3ce00700 1 -- 172.21.15.153:0/3031635190 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a3810b3f0 con 0x7f0a38102ff0 2024-03-21T10:03:47.044 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.427+0000 7f0a2ea00700 1 -- 172.21.15.153:0/3031635190 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f0a280105d0 con 0x7f0a38102ff0 2024-03-21T10:03:47.044 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.428+0000 7f0a2ea00700 1 --2- 172.21.15.153:0/3031635190 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f0a1c06c4e0 0x7f0a1c06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:03:47.044 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.428+0000 7f0a2ea00700 1 -- 172.21.15.153:0/3031635190 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f0a2808b3f0 con 0x7f0a38102ff0 2024-03-21T10:03:47.044 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.429+0000 7f0a35400700 1 --2- 172.21.15.153:0/3031635190 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f0a1c06c4e0 0x7f0a1c06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:03:47.044 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.429+0000 7f0a35400700 1 --2- 172.21.15.153:0/3031635190 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f0a1c06c4e0 0x7f0a1c06e9a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0a38199910 tx=0x7f0a2401a040 comp rx=0 tx=0).ready entity=mgr.14221 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:03:47.044 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.433+0000 7f0a2ea00700 1 -- 172.21.15.153:0/3031635190 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f0a28059a20 con 0x7f0a38102ff0 2024-03-21T10:03:47.044 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.657+0000 7f0a3ce00700 1 -- 172.21.15.153:0/3031635190 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f0a3819ddf0 con 0x7f0a38102ff0 2024-03-21T10:03:47.044 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.657+0000 7f0a2ea00700 1 -- 172.21.15.153:0/3031635190 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0a2808b070 con 0x7f0a38102ff0 2024-03-21T10:03:47.045 INFO:teuthology.orchestra.run.smithi153.stdout:115964117018 2024-03-21T10:03:47.045 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.660+0000 7f0a3ce00700 1 -- 172.21.15.153:0/3031635190 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f0a1c06c4e0 msgr2=0x7f0a1c06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:03:47.045 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.660+0000 7f0a3ce00700 1 --2- 172.21.15.153:0/3031635190 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f0a1c06c4e0 0x7f0a1c06e9a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0a38199910 tx=0x7f0a2401a040 comp rx=0 tx=0).stop 2024-03-21T10:03:47.045 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.660+0000 7f0a3ce00700 1 -- 172.21.15.153:0/3031635190 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0a38102ff0 msgr2=0x7f0a381989a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:03:47.045 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.660+0000 7f0a3ce00700 1 --2- 172.21.15.153:0/3031635190 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0a38102ff0 0x7f0a381989a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0a2800d8d0 tx=0x7f0a2800dc90 comp rx=0 tx=0).stop 2024-03-21T10:03:47.045 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.661+0000 7f0a3ce00700 1 -- 172.21.15.153:0/3031635190 shutdown_connections 2024-03-21T10:03:47.045 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.661+0000 7f0a3ce00700 1 --2- 172.21.15.153:0/3031635190 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f0a1c06c4e0 0x7f0a1c06e9a0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:47.045 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.661+0000 7f0a3ce00700 1 --2- 172.21.15.153:0/3031635190 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0a38102ff0 0x7f0a381989a0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:47.046 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.661+0000 7f0a3ce00700 1 --2- 172.21.15.153:0/3031635190 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0a381039a0 0x7f0a38198ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:47.046 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.661+0000 7f0a3ce00700 1 -- 172.21.15.153:0/3031635190 >> 172.21.15.153:0/3031635190 conn(0x7f0a380fe880 msgr2=0x7f0a38100c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:03:47.046 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.661+0000 7f0a3ce00700 1 -- 172.21.15.153:0/3031635190 shutdown_connections 2024-03-21T10:03:47.046 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.661+0000 7f0a3ce00700 1 -- 172.21.15.153:0/3031635190 wait complete. 2024-03-21T10:03:47.046 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:47.039+0000 7efe57000700 1 -- 172.21.15.153:0/1082871666 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7efe5004ea90 con 0x7efe500731c0 2024-03-21T10:03:47.046 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:47.040+0000 7efe4cc00700 1 -- 172.21.15.153:0/1082871666 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7efe40059840 con 0x7efe500731c0 2024-03-21T10:03:47.046 INFO:teuthology.orchestra.run.smithi153.stdout:171798691852 2024-03-21T10:03:47.047 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:47.042+0000 7efe57000700 1 -- 172.21.15.153:0/1082871666 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7efe3406c4e0 msgr2=0x7efe3406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:03:47.047 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:47.042+0000 7efe57000700 1 --2- 172.21.15.153:0/1082871666 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7efe3406c4e0 0x7efe3406e9a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7efe500fdd10 tx=0x7efe3c01a040 comp rx=0 tx=0).stop 2024-03-21T10:03:47.047 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:47.042+0000 7efe57000700 1 -- 172.21.15.153:0/1082871666 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efe500731c0 msgr2=0x7efe5019e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:03:47.047 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:47.042+0000 7efe57000700 1 --2- 172.21.15.153:0/1082871666 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efe500731c0 0x7efe5019e790 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7efe4000b4b0 tx=0x7efe4000b870 comp rx=0 tx=0).stop 2024-03-21T10:03:47.047 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:47.042+0000 7efe57000700 1 -- 172.21.15.153:0/1082871666 shutdown_connections 2024-03-21T10:03:47.047 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:47.042+0000 7efe57000700 1 --2- 172.21.15.153:0/1082871666 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7efe3406c4e0 0x7efe3406e9a0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:47.048 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:47.042+0000 7efe57000700 1 --2- 172.21.15.153:0/1082871666 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efe500731c0 0x7efe5019e790 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:47.048 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:47.042+0000 7efe57000700 1 --2- 172.21.15.153:0/1082871666 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efe50073ae0 0x7efe5019ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:47.048 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:47.042+0000 7efe57000700 1 -- 172.21.15.153:0/1082871666 >> 172.21.15.153:0/1082871666 conn(0x7efe500fc630 msgr2=0x7efe50107680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:03:47.048 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:47.043+0000 7efe57000700 1 -- 172.21.15.153:0/1082871666 shutdown_connections 2024-03-21T10:03:47.048 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:47.043+0000 7efe57000700 1 -- 172.21.15.153:0/1082871666 wait complete. 2024-03-21T10:03:47.444 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:45.929+0000 7f4e0a200700 1 -- 172.21.15.153:0/1987913891 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e04106980 msgr2=0x7f4e04106d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:03:47.445 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:45.929+0000 7f4e0a200700 1 --2- 172.21.15.153:0/1987913891 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e04106980 0x7f4e04106d60 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4df8009910 tx=0x7f4df8009c20 comp rx=0 tx=0).stop 2024-03-21T10:03:47.445 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:45.930+0000 7f4e0a200700 1 -- 172.21.15.153:0/1987913891 shutdown_connections 2024-03-21T10:03:47.445 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:45.930+0000 7f4e0a200700 1 --2- 172.21.15.153:0/1987913891 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4e04100980 0x7f4e04100e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:47.445 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:45.930+0000 7f4e0a200700 1 --2- 172.21.15.153:0/1987913891 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e04106980 0x7f4e04106d60 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:47.445 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:45.930+0000 7f4e0a200700 1 -- 172.21.15.153:0/1987913891 >> 172.21.15.153:0/1987913891 conn(0x7f4e040fc470 msgr2=0x7f4e040fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:03:47.445 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:45.930+0000 7f4e0a200700 1 -- 172.21.15.153:0/1987913891 shutdown_connections 2024-03-21T10:03:47.445 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:45.930+0000 7f4e0a200700 1 -- 172.21.15.153:0/1987913891 wait complete. 2024-03-21T10:03:47.446 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:45.931+0000 7f4e0a200700 1 Processor -- start 2024-03-21T10:03:47.446 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:45.931+0000 7f4e0a200700 1 -- start start 2024-03-21T10:03:47.446 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:45.931+0000 7f4e0a200700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4e04100980 0x7f4e0419e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:03:47.446 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:45.931+0000 7f4e0a200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e04106980 0x7f4e0419ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:03:47.446 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:45.931+0000 7f4e0a200700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e0419f100 con 0x7f4e04106980 2024-03-21T10:03:47.446 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:45.931+0000 7f4e0a200700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e041985b0 con 0x7f4e04100980 2024-03-21T10:03:47.446 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:45.932+0000 7f4e03e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e04106980 0x7f4e0419ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:03:47.447 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:45.932+0000 7f4e03e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e04106980 0x7f4e0419ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:40884/0 (socket says 172.21.15.153:40884) 2024-03-21T10:03:47.447 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:45.932+0000 7f4e03e00700 1 -- 172.21.15.153:0/1800066326 learned_addr learned my addr 172.21.15.153:0/1800066326 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:03:47.447 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:45.932+0000 7f4e08e00700 1 --2- 172.21.15.153:0/1800066326 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4e04100980 0x7f4e0419e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:03:47.447 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:45.932+0000 7f4e03e00700 1 -- 172.21.15.153:0/1800066326 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4e04100980 msgr2=0x7f4e0419e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:03:47.447 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:45.932+0000 7f4e03e00700 1 --2- 172.21.15.153:0/1800066326 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4e04100980 0x7f4e0419e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:47.447 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:45.932+0000 7f4e03e00700 1 -- 172.21.15.153:0/1800066326 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4df80095f0 con 0x7f4e04106980 2024-03-21T10:03:47.447 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:45.933+0000 7f4e03e00700 1 --2- 172.21.15.153:0/1800066326 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e04106980 0x7f4e0419ea70 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f4df000d8d0 tx=0x7f4df000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:03:47.448 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:45.933+0000 7f4e01600700 1 -- 172.21.15.153:0/1800066326 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4df0009940 con 0x7f4e04106980 2024-03-21T10:03:47.448 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:45.933+0000 7f4e01600700 1 -- 172.21.15.153:0/1800066326 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f4df0010460 con 0x7f4e04106980 2024-03-21T10:03:47.448 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:45.933+0000 7f4e0a200700 1 -- 172.21.15.153:0/1800066326 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e04198890 con 0x7f4e04106980 2024-03-21T10:03:47.448 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:45.933+0000 7f4e01600700 1 -- 172.21.15.153:0/1800066326 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4df000f5f0 con 0x7f4e04106980 2024-03-21T10:03:47.448 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:45.933+0000 7f4e0a200700 1 -- 172.21.15.153:0/1800066326 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e04198de0 con 0x7f4e04106980 2024-03-21T10:03:47.448 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:45.934+0000 7f4e0a200700 1 -- 172.21.15.153:0/1800066326 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e0404f2e0 con 0x7f4e04106980 2024-03-21T10:03:47.448 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:45.935+0000 7f4e01600700 1 -- 172.21.15.153:0/1800066326 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f4df00105d0 con 0x7f4e04106980 2024-03-21T10:03:47.448 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:45.936+0000 7f4e01600700 1 --2- 172.21.15.153:0/1800066326 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f4dec06c4e0 0x7f4dec06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:03:47.449 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:45.936+0000 7f4e01600700 1 -- 172.21.15.153:0/1800066326 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f4df008b490 con 0x7f4e04106980 2024-03-21T10:03:47.449 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:45.936+0000 7f4e08e00700 1 --2- 172.21.15.153:0/1800066326 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f4dec06c4e0 0x7f4dec06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:03:47.449 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:45.937+0000 7f4e08e00700 1 --2- 172.21.15.153:0/1800066326 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f4dec06c4e0 0x7f4dec06e9a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4df8006010 tx=0x7f4df801a040 comp rx=0 tx=0).ready entity=mgr.14221 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:03:47.449 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:45.941+0000 7f4e01600700 1 -- 172.21.15.153:0/1800066326 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f4df0059a20 con 0x7f4e04106980 2024-03-21T10:03:47.449 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.174+0000 7f4e0a200700 1 -- 172.21.15.153:0/1800066326 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f4e04199230 con 0x7f4e04106980 2024-03-21T10:03:47.449 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.175+0000 7f4e01600700 1 -- 172.21.15.153:0/1800066326 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4df008b070 con 0x7f4e04106980 2024-03-21T10:03:47.450 INFO:teuthology.orchestra.run.smithi153.stdout:38654705710 2024-03-21T10:03:47.450 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.178+0000 7f4e0a200700 1 -- 172.21.15.153:0/1800066326 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f4dec06c4e0 msgr2=0x7f4dec06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:03:47.451 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.178+0000 7f4e0a200700 1 --2- 172.21.15.153:0/1800066326 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f4dec06c4e0 0x7f4dec06e9a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4df8006010 tx=0x7f4df801a040 comp rx=0 tx=0).stop 2024-03-21T10:03:47.451 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.178+0000 7f4e0a200700 1 -- 172.21.15.153:0/1800066326 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e04106980 msgr2=0x7f4e0419ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:03:47.451 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.178+0000 7f4e0a200700 1 --2- 172.21.15.153:0/1800066326 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e04106980 0x7f4e0419ea70 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f4df000d8d0 tx=0x7f4df000dc90 comp rx=0 tx=0).stop 2024-03-21T10:03:47.451 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.178+0000 7f4e0a200700 1 -- 172.21.15.153:0/1800066326 shutdown_connections 2024-03-21T10:03:47.451 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.178+0000 7f4e0a200700 1 --2- 172.21.15.153:0/1800066326 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f4dec06c4e0 0x7f4dec06e9a0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:47.451 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.178+0000 7f4e0a200700 1 --2- 172.21.15.153:0/1800066326 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4e04100980 0x7f4e0419e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:47.452 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.178+0000 7f4e0a200700 1 --2- 172.21.15.153:0/1800066326 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e04106980 0x7f4e0419ea70 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:47.452 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.178+0000 7f4e0a200700 1 -- 172.21.15.153:0/1800066326 >> 172.21.15.153:0/1800066326 conn(0x7f4e040fc470 msgr2=0x7f4e04073000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:03:47.452 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.178+0000 7f4e0a200700 1 -- 172.21.15.153:0/1800066326 shutdown_connections 2024-03-21T10:03:47.452 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:46.178+0000 7f4e0a200700 1 -- 172.21.15.153:0/1800066326 wait complete. 2024-03-21T10:03:47.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:03:47 smithi174 ceph-mon[37496]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:03:47.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:03:47 smithi174 ceph-mon[37496]: from='client.? 172.21.15.153:0/3031635190' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-21T10:03:47.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:03:47 smithi174 ceph-mon[37496]: from='client.? 172.21.15.153:0/1941550998' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-21T10:03:47.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:03:47 smithi174 ceph-mon[37496]: from='client.? 172.21.15.153:0/1082871666' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-21T10:03:47.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:03:47 smithi153 ceph-mon[29117]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:03:47.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:03:47 smithi153 ceph-mon[29117]: from='client.? 172.21.15.153:0/3031635190' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-21T10:03:47.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:03:47 smithi153 ceph-mon[29117]: from='client.? 172.21.15.153:0/1941550998' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-21T10:03:47.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:03:47 smithi153 ceph-mon[29117]: from='client.? 172.21.15.153:0/1082871666' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-21T10:03:48.216 INFO:tasks.cephadm.ceph_manager.ceph:need seq 150323855376 got 150323855378 for osd.4 2024-03-21T10:03:48.217 DEBUG:teuthology.parallel:result is None 2024-03-21T10:03:49.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:03:49 smithi174 ceph-mon[37496]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:03:49.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:03:49 smithi153 ceph-mon[29117]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:03:50.051 INFO:tasks.cephadm.ceph_manager.ceph:need seq 81604378652 got 81604378655 for osd.2 2024-03-21T10:03:50.052 DEBUG:teuthology.parallel:result is None 2024-03-21T10:03:50.184 INFO:tasks.cephadm.ceph_manager.ceph:need seq 38654705708 got 38654705710 for osd.0 2024-03-21T10:03:50.184 DEBUG:teuthology.parallel:result is None 2024-03-21T10:03:50.502 INFO:tasks.cephadm.ceph_manager.ceph:need seq 60129542180 got 60129542182 for osd.1 2024-03-21T10:03:50.503 DEBUG:teuthology.parallel:result is None 2024-03-21T10:03:50.652 INFO:tasks.cephadm.ceph_manager.ceph:need seq 115964117014 got 115964117018 for osd.3 2024-03-21T10:03:50.652 DEBUG:teuthology.parallel:result is None 2024-03-21T10:03:50.793 INFO:tasks.cephadm.ceph_manager.ceph:need seq 171798691849 got 171798691852 for osd.5 2024-03-21T10:03:50.794 DEBUG:teuthology.parallel:result is None 2024-03-21T10:03:50.794 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-03-21T10:03:50.794 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 070ed3e4-e769-11ee-95cd-87774f69a715 -- ceph pg dump --format=json 2024-03-21T10:03:51.214 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi153/config 2024-03-21T10:03:51.688 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:03:51 smithi153 ceph-mon[29117]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:03:51.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:03:51 smithi174 ceph-mon[37496]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:03:53.341 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:53.338+0000 7f51cfe00700 1 -- 172.21.15.153:0/3074131112 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f51c8103990 msgr2=0x7f51c81079e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:03:53.341 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:53.338+0000 7f51cfe00700 1 --2- 172.21.15.153:0/3074131112 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f51c8103990 0x7f51c81079e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f51b4009910 tx=0x7f51b4009c20 comp rx=0 tx=0).stop 2024-03-21T10:03:53.341 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:53.338+0000 7f51cfe00700 1 -- 172.21.15.153:0/3074131112 shutdown_connections 2024-03-21T10:03:53.341 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:53.338+0000 7f51cfe00700 1 --2- 172.21.15.153:0/3074131112 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f51c8103990 0x7f51c81079e0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:53.342 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:53.339+0000 7f51cfe00700 1 --2- 172.21.15.153:0/3074131112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f51c8102fe0 0x7f51c81033c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:53.342 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:53.339+0000 7f51cfe00700 1 -- 172.21.15.153:0/3074131112 >> 172.21.15.153:0/3074131112 conn(0x7f51c80fe8d0 msgr2=0x7f51c8100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:03:53.342 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:53.339+0000 7f51cfe00700 1 -- 172.21.15.153:0/3074131112 shutdown_connections 2024-03-21T10:03:53.342 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:53.339+0000 7f51cfe00700 1 -- 172.21.15.153:0/3074131112 wait complete. 2024-03-21T10:03:53.342 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:53.340+0000 7f51cfe00700 1 Processor -- start 2024-03-21T10:03:53.342 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:53.340+0000 7f51cfe00700 1 -- start start 2024-03-21T10:03:53.344 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:53.340+0000 7f51cfe00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f51c8102fe0 0x7f51c819e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:03:53.344 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:53.340+0000 7f51cfe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f51c8103990 0x7f51c819ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:03:53.344 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:53.340+0000 7f51cfe00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51c819f2d0 con 0x7f51c8103990 2024-03-21T10:03:53.344 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:53.340+0000 7f51cfe00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51c8198780 con 0x7f51c8102fe0 2024-03-21T10:03:53.344 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:53.340+0000 7f51cca00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f51c8103990 0x7f51c819ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:03:53.344 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:53.340+0000 7f51cca00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f51c8103990 0x7f51c819ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:47436/0 (socket says 172.21.15.153:47436) 2024-03-21T10:03:53.344 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:53.340+0000 7f51cca00700 1 -- 172.21.15.153:0/3906066268 learned_addr learned my addr 172.21.15.153:0/3906066268 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:03:53.344 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:53.341+0000 7f51cd400700 1 --2- 172.21.15.153:0/3906066268 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f51c8102fe0 0x7f51c819e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:03:53.344 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:53.341+0000 7f51cca00700 1 -- 172.21.15.153:0/3906066268 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f51c8102fe0 msgr2=0x7f51c819e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:03:53.345 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:53.341+0000 7f51cca00700 1 --2- 172.21.15.153:0/3906066268 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f51c8102fe0 0x7f51c819e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:53.345 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:53.341+0000 7f51cca00700 1 -- 172.21.15.153:0/3906066268 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51b40095f0 con 0x7f51c8103990 2024-03-21T10:03:53.345 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:53.341+0000 7f51cca00700 1 --2- 172.21.15.153:0/3906066268 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f51c8103990 0x7f51c819ec40 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f51b400bda0 tx=0x7f51b400be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:03:53.345 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:53.341+0000 7f51c6000700 1 -- 172.21.15.153:0/3906066268 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f51b401d070 con 0x7f51c8103990 2024-03-21T10:03:53.345 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:53.341+0000 7f51c6000700 1 -- 172.21.15.153:0/3906066268 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f51b4022470 con 0x7f51c8103990 2024-03-21T10:03:53.345 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:53.342+0000 7f51cfe00700 1 -- 172.21.15.153:0/3906066268 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51c8198a00 con 0x7f51c8103990 2024-03-21T10:03:53.345 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:53.342+0000 7f51c6000700 1 -- 172.21.15.153:0/3906066268 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f51b400f690 con 0x7f51c8103990 2024-03-21T10:03:53.345 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:53.342+0000 7f51cfe00700 1 -- 172.21.15.153:0/3906066268 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51c8198ef0 con 0x7f51c8103990 2024-03-21T10:03:53.346 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:53.343+0000 7f51cfe00700 1 -- 172.21.15.153:0/3906066268 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51c810b3e0 con 0x7f51c8103990 2024-03-21T10:03:53.346 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:53.344+0000 7f51c6000700 1 -- 172.21.15.153:0/3906066268 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f51b4022ae0 con 0x7f51c8103990 2024-03-21T10:03:53.347 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:53.345+0000 7f51c6000700 1 --2- 172.21.15.153:0/3906066268 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f51b006c4e0 0x7f51b006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:03:53.347 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:53.346+0000 7f51c6000700 1 -- 172.21.15.153:0/3906066268 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f51b408ca90 con 0x7f51c8103990 2024-03-21T10:03:53.347 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:53.346+0000 7f51cd400700 1 --2- 172.21.15.153:0/3906066268 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f51b006c4e0 0x7f51b006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:03:53.347 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:53.346+0000 7f51cd400700 1 --2- 172.21.15.153:0/3906066268 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f51b006c4e0 0x7f51b006e9a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f51bc005fd0 tx=0x7f51bc00a560 comp rx=0 tx=0).ready entity=mgr.14221 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:03:53.352 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:53.351+0000 7f51c6000700 1 -- 172.21.15.153:0/3906066268 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f51b405b0d0 con 0x7f51c8103990 2024-03-21T10:03:53.563 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:53.561+0000 7f51cfe00700 1 -- 172.21.15.153:0/3906066268 --> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f51c802cd50 con 0x7f51b006c4e0 2024-03-21T10:03:53.566 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:53.565+0000 7f51c6000700 1 -- 172.21.15.153:0/3906066268 <== mgr.14221 v2:172.21.15.153:6800/1639929315 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+27102 (secure 0 0 0) 0x7f51c802cd50 con 0x7f51b006c4e0 2024-03-21T10:03:53.567 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:03:53.570 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:53.568+0000 7f51cfe00700 1 -- 172.21.15.153:0/3906066268 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f51b006c4e0 msgr2=0x7f51b006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:03:53.570 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:53.568+0000 7f51cfe00700 1 --2- 172.21.15.153:0/3906066268 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f51b006c4e0 0x7f51b006e9a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f51bc005fd0 tx=0x7f51bc00a560 comp rx=0 tx=0).stop 2024-03-21T10:03:53.570 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:53.568+0000 7f51cfe00700 1 -- 172.21.15.153:0/3906066268 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f51c8103990 msgr2=0x7f51c819ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:03:53.570 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:53.568+0000 7f51cfe00700 1 --2- 172.21.15.153:0/3906066268 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f51c8103990 0x7f51c819ec40 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f51b400bda0 tx=0x7f51b400be80 comp rx=0 tx=0).stop 2024-03-21T10:03:53.570 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:53.569+0000 7f51cfe00700 1 -- 172.21.15.153:0/3906066268 shutdown_connections 2024-03-21T10:03:53.570 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:53.569+0000 7f51cfe00700 1 --2- 172.21.15.153:0/3906066268 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f51b006c4e0 0x7f51b006e9a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:53.571 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:53.569+0000 7f51cfe00700 1 --2- 172.21.15.153:0/3906066268 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f51c8102fe0 0x7f51c819e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:53.571 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:53.569+0000 7f51cfe00700 1 --2- 172.21.15.153:0/3906066268 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f51c8103990 0x7f51c819ec40 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:53.571 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:53.569+0000 7f51cfe00700 1 -- 172.21.15.153:0/3906066268 >> 172.21.15.153:0/3906066268 conn(0x7f51c80fe8d0 msgr2=0x7f51c8100c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:03:53.571 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:53.569+0000 7f51cfe00700 1 -- 172.21.15.153:0/3906066268 shutdown_connections 2024-03-21T10:03:53.571 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:53.569+0000 7f51cfe00700 1 -- 172.21.15.153:0/3906066268 wait complete. 2024-03-21T10:03:53.572 INFO:teuthology.orchestra.run.smithi153.stderr:dumped all 2024-03-21T10:03:53.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:03:53 smithi174 ceph-mon[37496]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:03:53.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:03:53 smithi153 ceph-mon[29117]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:03:54.325 INFO:teuthology.orchestra.run.smithi153.stdout:{"pg_ready":true,"pg_map":{"version":193,"stamp":"2024-03-21T10:03:52.342967+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":165636,"kb_used_data":3924,"kb_used_omap":0,"kb_used_meta":161664,"kb_avail":562305276,"statfs":{"total":575970213888,"available":575800602624,"internally_reserved":0,"allocated":4018176,"data_stored":2627442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":165543936},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003132"},"pg_stats":[{"pgid":"1.0","version":"24'117","reported_seq":186,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-21T10:02:55.429993+0000","last_change":"2024-03-21T10:02:22.422341+0000","last_active":"2024-03-21T10:02:55.429993+0000","last_peered":"2024-03-21T10:02:55.429993+0000","last_clean":"2024-03-21T10:02:55.429993+0000","last_became_active":"2024-03-21T10:02:22.421975+0000","last_became_peered":"2024-03-21T10:02:22.421975+0000","last_unstale":"2024-03-21T10:02:55.429993+0000","last_undegraded":"2024-03-21T10:02:55.429993+0000","last_fullsized":"2024-03-21T10:02:55.429993+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:01:18.993851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:01:18.993851+0000","last_clean_scrub_stamp":"2024-03-21T10:01:18.993851+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:35:11.952506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":40,"seq":171798691853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27316,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717836,"statfs":{"total":95995035648,"available":95967064064,"internally_reserved":0,"allocated":372736,"data_stored":142723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.974}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.893}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93000000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0580000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1000000000000001}]}]},{"osd":4,"up_from":35,"seq":150323855380,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27316,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717836,"statfs":{"total":95995035648,"available":95967064064,"internally_reserved":0,"allocated":372736,"data_stored":142723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 21 10:03:24 2024","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":1.018,"15min":1.018},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.448,"5min":1.448,"15min":1.448},"last":0.621},{"interface":"front","average":{"1min":1.04,"5min":1.04,"15min":1.04},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.681,"5min":1.681,"15min":1.681},"last":0.88400000000000001}]},{"osd":1,"last update":"Thu Mar 21 10:03:24 2024","interfaces":[{"interface":"back","average":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.94399999999999995},{"interface":"front","average":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.427,"5min":1.427,"15min":1.427},"last":1.008}]},{"osd":2,"last update":"Thu Mar 21 10:03:24 2024","interfaces":[{"interface":"back","average":{"1min":1.123,"5min":1.123,"15min":1.123},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.4630000000000001,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":1.173},{"interface":"front","average":{"1min":1.157,"5min":1.157,"15min":1.157},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":2.1030000000000002,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":1.111}]},{"osd":3,"last update":"Thu Mar 21 10:03:24 2024","interfaces":[{"interface":"back","average":{"1min":1.083,"5min":1.083,"15min":1.083},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.401,"5min":1.401,"15min":1.401},"last":1.1970000000000001},{"interface":"front","average":{"1min":1.151,"5min":1.151,"15min":1.151},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.5189999999999999,"5min":1.5189999999999999,"15min":1.5189999999999999},"last":1.151}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2490000000000001}]}]},{"osd":3,"up_from":27,"seq":115964117019,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":966656,"data_stored":733091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 21 10:02:52 2024","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":2.0609999999999999,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":1.008},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.7490000000000001,"5min":1.7490000000000001,"15min":1.7490000000000001},"last":0.95699999999999996}]},{"osd":1,"last update":"Thu Mar 21 10:02:52 2024","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":2.165,"5min":2.165,"15min":2.165},"last":0.79100000000000004},{"interface":"front","average":{"1min":1.016,"5min":1.016,"15min":1.016},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":2.3740000000000001,"5min":2.3740000000000001,"15min":2.3740000000000001},"last":1.0680000000000001}]},{"osd":2,"last update":"Thu Mar 21 10:02:52 2024","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":1.018,"15min":1.018},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.6870000000000001,"5min":1.6870000000000001,"15min":1.6870000000000001},"last":1.131},{"interface":"front","average":{"1min":1.05,"5min":1.05,"15min":1.05},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":2.226,"5min":2.226,"15min":2.226},"last":1.1799999999999999}]},{"osd":4,"last update":"Thu Mar 21 10:03:24 2024","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.5620000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":1.331},{"interface":"front","average":{"1min":1.02,"5min":1.02,"15min":1.02},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":2.3199999999999998,"5min":2.3199999999999998,"15min":2.3199999999999998},"last":1.296}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.377}]}]},{"osd":2,"up_from":19,"seq":81604378657,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27316,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717836,"statfs":{"total":95995035648,"available":95967064064,"internally_reserved":0,"allocated":372736,"data_stored":142723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 21 10:03:18 2024","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.79700000000000004,"15min":0.79500000000000004},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":1.1140000000000001},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.82799999999999996,"15min":0.81499999999999995},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.4359999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":0.73099999999999998}]},{"osd":1,"last update":"Thu Mar 21 10:03:18 2024","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.82699999999999996,"15min":0.81699999999999995},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.133,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.94699999999999995},{"interface":"front","average":{"1min":1.03,"5min":0.95499999999999996,"15min":0.94299999999999995},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.49,"5min":1.49,"15min":1.49},"last":1.002}]},{"osd":3,"last update":"Thu Mar 21 10:03:50 2024","interfaces":[{"interface":"back","average":{"1min":1.206,"5min":1.05,"15min":1.024},"min":{"1min":0.83299999999999996,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.5920000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":1.171},{"interface":"front","average":{"1min":1.24,"5min":1.135,"15min":1.117},"min":{"1min":1.02,"5min":0.96199999999999997,"15min":0.96199999999999997},"max":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":1.3819999999999999}]},{"osd":4,"last update":"Thu Mar 21 10:03:24 2024","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"min":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"max":{"1min":1.6319999999999999,"5min":1.6319999999999999,"15min":1.6319999999999999},"last":1.262},{"interface":"front","average":{"1min":1.149,"5min":1.149,"15min":1.149},"min":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":1.5700000000000001,"5min":1.5700000000000001,"15min":1.5700000000000001},"last":1.2330000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5109999999999999}]}]},{"osd":0,"up_from":9,"seq":38654705712,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":966656,"data_stored":733091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Mar 21 10:03:49 2024","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.79600000000000004,"15min":0.76600000000000001},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.9810000000000001,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.96199999999999997,"5min":0.85199999999999998,"15min":0.83099999999999996},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.5580000000000001,"5min":1.5580000000000001,"15min":1.5580000000000001},"last":1.0349999999999999}]},{"osd":2,"last update":"Thu Mar 21 10:03:23 2024","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.86299999999999999,"15min":0.85299999999999998},"min":{"1min":0.52400000000000002,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.2390000000000001,"5min":1.911,"15min":1.911},"last":1.1619999999999999},{"interface":"front","average":{"1min":1.0189999999999999,"5min":0.95099999999999996,"15min":0.93999999999999995},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.5289999999999999,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":0.94099999999999995}]},{"osd":3,"last update":"Thu Mar 21 10:02:54 2024","interfaces":[{"interface":"back","average":{"1min":1.048,"5min":1.048,"15min":1.048},"min":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":1.5289999999999999},{"interface":"front","average":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"min":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"max":{"1min":1.466,"5min":1.466,"15min":1.466},"last":1.393}]},{"osd":4,"last update":"Thu Mar 21 10:03:23 2024","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.232,"15min":1.232},"min":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":1.7969999999999999,"5min":1.7969999999999999,"15min":1.7969999999999999},"last":1.353},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.6779999999999999,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":1.101}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2849999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542184,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":966656,"data_stored":733091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 21 10:03:50 2024","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.71199999999999997,"15min":0.67000000000000004},"min":{"1min":0.48999999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.77700000000000002,"15min":0.71899999999999997},"min":{"1min":0.49099999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.7669999999999999,"5min":1.7669999999999999,"15min":1.7669999999999999},"last":1.413}]},{"osd":2,"last update":"Thu Mar 21 10:03:20 2024","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":1.1180000000000001,"15min":1.1619999999999999},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.0649999999999999,"5min":8.8320000000000007,"15min":8.8320000000000007},"last":1.304},{"interface":"front","average":{"1min":1.01,"5min":1.222,"15min":1.2569999999999999},"min":{"1min":0.747,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.226,"5min":8.7400000000000002,"15min":8.7400000000000002},"last":0.90400000000000003}]},{"osd":3,"last update":"Thu Mar 21 10:02:54 2024","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":1.069,"15min":1.069},"min":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":1.488},{"interface":"front","average":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"min":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"max":{"1min":1.6950000000000001,"5min":1.6950000000000001,"15min":1.6950000000000001},"last":1.004}]},{"osd":4,"last update":"Thu Mar 21 10:03:23 2024","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":1.542},{"interface":"front","average":{"1min":1.242,"5min":1.242,"15min":1.242},"min":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":1.585,"5min":1.585,"15min":1.585},"last":1.5720000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6399999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-21T10:03:54.327 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 070ed3e4-e769-11ee-95cd-87774f69a715 -- ceph pg dump --format=json 2024-03-21T10:03:54.559 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:03:54 smithi153 ceph-mon[29117]: from='client.14430 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T10:03:54.559 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:03:54 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:03:54.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:03:54 smithi174 ceph-mon[37496]: from='client.14430 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T10:03:54.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:03:54 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:03:54.759 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi153/config 2024-03-21T10:03:55.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:03:55 smithi153 ceph-mon[29117]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:03:55.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:03:55 smithi174 ceph-mon[37496]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:03:56.908 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:56.903+0000 7f96bb800700 1 -- 172.21.15.153:0/3047961293 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f96b4100b50 msgr2=0x7f96b4100fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:03:56.908 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:56.903+0000 7f96bb800700 1 --2- 172.21.15.153:0/3047961293 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f96b4100b50 0x7f96b4100fd0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f969c009910 tx=0x7f969c009c20 comp rx=0 tx=0).stop 2024-03-21T10:03:56.908 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:56.904+0000 7f96bb800700 1 -- 172.21.15.153:0/3047961293 shutdown_connections 2024-03-21T10:03:56.908 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:56.904+0000 7f96bb800700 1 --2- 172.21.15.153:0/3047961293 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f96b4100b50 0x7f96b4100fd0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:56.908 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:56.904+0000 7f96bb800700 1 --2- 172.21.15.153:0/3047961293 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f96b4106c00 0x7f96b4106fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:56.908 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:56.904+0000 7f96bb800700 1 -- 172.21.15.153:0/3047961293 >> 172.21.15.153:0/3047961293 conn(0x7f96b40fc640 msgr2=0x7f96b40fea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:03:56.908 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:56.904+0000 7f96bb800700 1 -- 172.21.15.153:0/3047961293 shutdown_connections 2024-03-21T10:03:56.908 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:56.904+0000 7f96bb800700 1 -- 172.21.15.153:0/3047961293 wait complete. 2024-03-21T10:03:56.908 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:56.905+0000 7f96bb800700 1 Processor -- start 2024-03-21T10:03:56.908 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:56.905+0000 7f96bb800700 1 -- start start 2024-03-21T10:03:56.909 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:56.905+0000 7f96bb800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f96b4100b50 0x7f96b419e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:03:56.909 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:56.905+0000 7f96bb800700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f96b4106c00 0x7f96b419eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:03:56.909 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:56.905+0000 7f96bb800700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96b419f230 con 0x7f96b4100b50 2024-03-21T10:03:56.909 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:56.905+0000 7f96bb800700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96b41986e0 con 0x7f96b4106c00 2024-03-21T10:03:56.909 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:56.906+0000 7f96b8e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f96b4100b50 0x7f96b419e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:03:56.909 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:56.906+0000 7f96b8e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f96b4100b50 0x7f96b419e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:47466/0 (socket says 172.21.15.153:47466) 2024-03-21T10:03:56.909 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:56.906+0000 7f96b8e00700 1 -- 172.21.15.153:0/1220713688 learned_addr learned my addr 172.21.15.153:0/1220713688 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:03:56.909 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:56.906+0000 7f96abe00700 1 --2- 172.21.15.153:0/1220713688 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f96b4106c00 0x7f96b419eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:03:56.909 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:56.906+0000 7f96b8e00700 1 -- 172.21.15.153:0/1220713688 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f96b4106c00 msgr2=0x7f96b419eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:03:56.909 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:56.906+0000 7f96b8e00700 1 --2- 172.21.15.153:0/1220713688 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f96b4106c00 0x7f96b419eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:56.909 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:56.906+0000 7f96b8e00700 1 -- 172.21.15.153:0/1220713688 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f969c0095f0 con 0x7f96b4100b50 2024-03-21T10:03:56.909 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:56.906+0000 7f96abe00700 1 --2- 172.21.15.153:0/1220713688 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f96b4106c00 0x7f96b419eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:03:56.909 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:56.907+0000 7f96b8e00700 1 --2- 172.21.15.153:0/1220713688 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f96b4100b50 0x7f96b419e660 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f96a400d8d0 tx=0x7f96a400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:03:56.909 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:56.907+0000 7f96b2000700 1 -- 172.21.15.153:0/1220713688 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f96a4009940 con 0x7f96b4100b50 2024-03-21T10:03:56.910 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:56.907+0000 7f96b2000700 1 -- 172.21.15.153:0/1220713688 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f96a4010460 con 0x7f96b4100b50 2024-03-21T10:03:56.910 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:56.907+0000 7f96b2000700 1 -- 172.21.15.153:0/1220713688 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f96a400f5f0 con 0x7f96b4100b50 2024-03-21T10:03:56.910 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:56.907+0000 7f96bb800700 1 -- 172.21.15.153:0/1220713688 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96b41989c0 con 0x7f96b4100b50 2024-03-21T10:03:56.910 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:56.907+0000 7f96bb800700 1 -- 172.21.15.153:0/1220713688 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96b4198ec0 con 0x7f96b4100b50 2024-03-21T10:03:56.910 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:56.908+0000 7f96bb800700 1 -- 172.21.15.153:0/1220713688 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96b404ea90 con 0x7f96b4100b50 2024-03-21T10:03:56.910 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:56.909+0000 7f96b2000700 1 -- 172.21.15.153:0/1220713688 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f96a40105d0 con 0x7f96b4100b50 2024-03-21T10:03:56.911 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:56.910+0000 7f96b2000700 1 --2- 172.21.15.153:0/1220713688 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f969806c4e0 0x7f969806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:03:56.911 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:56.910+0000 7f96b2000700 1 -- 172.21.15.153:0/1220713688 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f96a408b4f0 con 0x7f96b4100b50 2024-03-21T10:03:56.911 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:56.911+0000 7f96abe00700 1 --2- 172.21.15.153:0/1220713688 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f969806c4e0 0x7f969806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:03:56.911 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:56.911+0000 7f96abe00700 1 --2- 172.21.15.153:0/1220713688 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f969806c4e0 0x7f969806e9a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f969c006010 tx=0x7f969c00b540 comp rx=0 tx=0).ready entity=mgr.14221 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:03:56.916 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:56.915+0000 7f96b2000700 1 -- 172.21.15.153:0/1220713688 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f96a4059aa0 con 0x7f96b4100b50 2024-03-21T10:03:57.126 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:57.126+0000 7f96bb800700 1 -- 172.21.15.153:0/1220713688 --> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f96b41993a0 con 0x7f969806c4e0 2024-03-21T10:03:57.128 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:57.128+0000 7f96b2000700 1 -- 172.21.15.153:0/1220713688 <== mgr.14221 v2:172.21.15.153:6800/1639929315 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+27210 (secure 0 0 0) 0x7f96b41993a0 con 0x7f969806c4e0 2024-03-21T10:03:57.129 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:03:57.132 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:57.131+0000 7f96bb800700 1 -- 172.21.15.153:0/1220713688 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f969806c4e0 msgr2=0x7f969806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:03:57.132 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:57.131+0000 7f96bb800700 1 --2- 172.21.15.153:0/1220713688 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f969806c4e0 0x7f969806e9a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f969c006010 tx=0x7f969c00b540 comp rx=0 tx=0).stop 2024-03-21T10:03:57.132 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:57.131+0000 7f96bb800700 1 -- 172.21.15.153:0/1220713688 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f96b4100b50 msgr2=0x7f96b419e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:03:57.132 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:57.131+0000 7f96bb800700 1 --2- 172.21.15.153:0/1220713688 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f96b4100b50 0x7f96b419e660 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f96a400d8d0 tx=0x7f96a400dc90 comp rx=0 tx=0).stop 2024-03-21T10:03:57.132 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:57.131+0000 7f96bb800700 1 -- 172.21.15.153:0/1220713688 shutdown_connections 2024-03-21T10:03:57.132 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:57.131+0000 7f96bb800700 1 --2- 172.21.15.153:0/1220713688 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f969806c4e0 0x7f969806e9a0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:57.132 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:57.131+0000 7f96bb800700 1 --2- 172.21.15.153:0/1220713688 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f96b4100b50 0x7f96b419e660 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:57.132 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:57.131+0000 7f96bb800700 1 --2- 172.21.15.153:0/1220713688 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f96b4106c00 0x7f96b419eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:03:57.133 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:57.131+0000 7f96bb800700 1 -- 172.21.15.153:0/1220713688 >> 172.21.15.153:0/1220713688 conn(0x7f96b40fc640 msgr2=0x7f96b40fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:03:57.133 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:57.131+0000 7f96bb800700 1 -- 172.21.15.153:0/1220713688 shutdown_connections 2024-03-21T10:03:57.133 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:03:57.131+0000 7f96bb800700 1 -- 172.21.15.153:0/1220713688 wait complete. 2024-03-21T10:03:57.133 INFO:teuthology.orchestra.run.smithi153.stderr:dumped all 2024-03-21T10:03:57.749 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:03:57 smithi153 ceph-mon[29117]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:03:57.754 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:03:57 smithi174 ceph-mon[37496]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:03:57.800 INFO:teuthology.orchestra.run.smithi153.stdout:{"pg_ready":true,"pg_map":{"version":195,"stamp":"2024-03-21T10:03:56.344178+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":165636,"kb_used_data":3924,"kb_used_omap":0,"kb_used_meta":161664,"kb_avail":562305276,"statfs":{"total":575970213888,"available":575800602624,"internally_reserved":0,"allocated":4018176,"data_stored":2627442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":165543936},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003220"},"pg_stats":[{"pgid":"1.0","version":"24'117","reported_seq":186,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-21T10:02:55.429993+0000","last_change":"2024-03-21T10:02:22.422341+0000","last_active":"2024-03-21T10:02:55.429993+0000","last_peered":"2024-03-21T10:02:55.429993+0000","last_clean":"2024-03-21T10:02:55.429993+0000","last_became_active":"2024-03-21T10:02:22.421975+0000","last_became_peered":"2024-03-21T10:02:22.421975+0000","last_unstale":"2024-03-21T10:02:55.429993+0000","last_undegraded":"2024-03-21T10:02:55.429993+0000","last_fullsized":"2024-03-21T10:02:55.429993+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T10:01:18.993851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T10:01:18.993851+0000","last_clean_scrub_stamp":"2024-03-21T10:01:18.993851+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:35:11.952506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":40,"seq":171798691854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27316,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717836,"statfs":{"total":95995035648,"available":95967064064,"internally_reserved":0,"allocated":372736,"data_stored":142723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98599999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82099999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1830000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92500000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0329999999999999}]}]},{"osd":4,"up_from":35,"seq":150323855381,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27316,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717836,"statfs":{"total":95995035648,"available":95967064064,"internally_reserved":0,"allocated":372736,"data_stored":142723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 21 10:03:24 2024","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":1.018,"15min":1.018},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.448,"5min":1.448,"15min":1.448},"last":0.68799999999999994},{"interface":"front","average":{"1min":1.04,"5min":1.04,"15min":1.04},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.681,"5min":1.681,"15min":1.681},"last":1.103}]},{"osd":1,"last update":"Thu Mar 21 10:03:24 2024","interfaces":[{"interface":"back","average":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":1.1439999999999999},{"interface":"front","average":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.427,"5min":1.427,"15min":1.427},"last":0.90700000000000003}]},{"osd":2,"last update":"Thu Mar 21 10:03:24 2024","interfaces":[{"interface":"back","average":{"1min":1.123,"5min":1.123,"15min":1.123},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.4630000000000001,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":0.81599999999999995},{"interface":"front","average":{"1min":1.157,"5min":1.157,"15min":1.157},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":2.1030000000000002,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":0.96999999999999997}]},{"osd":3,"last update":"Thu Mar 21 10:03:24 2024","interfaces":[{"interface":"back","average":{"1min":1.083,"5min":1.083,"15min":1.083},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.401,"5min":1.401,"15min":1.401},"last":1.2090000000000001},{"interface":"front","average":{"1min":1.151,"5min":1.151,"15min":1.151},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.5189999999999999,"5min":1.5189999999999999,"15min":1.5189999999999999},"last":1.0620000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2569999999999999}]}]},{"osd":3,"up_from":27,"seq":115964117020,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":966656,"data_stored":733091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 21 10:03:54 2024","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.999,"15min":1.004},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.3100000000000001,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":1.24},{"interface":"front","average":{"1min":1.0189999999999999,"5min":0.97999999999999998,"15min":0.97399999999999998},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.3500000000000001,"5min":1.7490000000000001,"15min":1.7490000000000001},"last":1.077}]},{"osd":1,"last update":"Thu Mar 21 10:03:54 2024","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.94799999999999995,"15min":0.94099999999999995},"min":{"1min":0.67600000000000005,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.248,"5min":2.165,"15min":2.165},"last":1.177},{"interface":"front","average":{"1min":0.96699999999999997,"5min":1.006,"15min":1.012},"min":{"1min":0.76500000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.1299999999999999,"5min":2.3740000000000001,"15min":2.3740000000000001},"last":0.97599999999999998}]},{"osd":2,"last update":"Thu Mar 21 10:03:54 2024","interfaces":[{"interface":"back","average":{"1min":1.02,"5min":1.018,"15min":1.018},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.29,"5min":1.6870000000000001,"15min":1.6870000000000001},"last":0.94699999999999995},{"interface":"front","average":{"1min":1.0629999999999999,"5min":1.052,"15min":1.05},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":1.3500000000000001,"5min":2.226,"15min":2.226},"last":1.0209999999999999}]},{"osd":4,"last update":"Thu Mar 21 10:03:24 2024","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.5620000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":1.1459999999999999},{"interface":"front","average":{"1min":1.02,"5min":1.02,"15min":1.02},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":2.3199999999999998,"5min":2.3199999999999998,"15min":2.3199999999999998},"last":1.1240000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2949999999999999}]}]},{"osd":2,"up_from":19,"seq":81604378657,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27316,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717836,"statfs":{"total":95995035648,"available":95967064064,"internally_reserved":0,"allocated":372736,"data_stored":142723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 21 10:03:18 2024","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.79700000000000004,"15min":0.79500000000000004},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":1.1140000000000001},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.82799999999999996,"15min":0.81499999999999995},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.4359999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":0.73099999999999998}]},{"osd":1,"last update":"Thu Mar 21 10:03:18 2024","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.82699999999999996,"15min":0.81699999999999995},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.133,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.94699999999999995},{"interface":"front","average":{"1min":1.03,"5min":0.95499999999999996,"15min":0.94299999999999995},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.49,"5min":1.49,"15min":1.49},"last":1.002}]},{"osd":3,"last update":"Thu Mar 21 10:03:50 2024","interfaces":[{"interface":"back","average":{"1min":1.206,"5min":1.05,"15min":1.024},"min":{"1min":0.83299999999999996,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.5920000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":1.171},{"interface":"front","average":{"1min":1.24,"5min":1.135,"15min":1.117},"min":{"1min":1.02,"5min":0.96199999999999997,"15min":0.96199999999999997},"max":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":1.3819999999999999}]},{"osd":4,"last update":"Thu Mar 21 10:03:24 2024","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"min":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"max":{"1min":1.6319999999999999,"5min":1.6319999999999999,"15min":1.6319999999999999},"last":1.262},{"interface":"front","average":{"1min":1.149,"5min":1.149,"15min":1.149},"min":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":1.5700000000000001,"5min":1.5700000000000001,"15min":1.5700000000000001},"last":1.2330000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5109999999999999}]}]},{"osd":0,"up_from":9,"seq":38654705712,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":966656,"data_stored":733091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Mar 21 10:03:49 2024","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.79600000000000004,"15min":0.76600000000000001},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.9810000000000001,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.96199999999999997,"5min":0.85199999999999998,"15min":0.83099999999999996},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.5580000000000001,"5min":1.5580000000000001,"15min":1.5580000000000001},"last":1.0349999999999999}]},{"osd":2,"last update":"Thu Mar 21 10:03:23 2024","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.86299999999999999,"15min":0.85299999999999998},"min":{"1min":0.52400000000000002,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.2390000000000001,"5min":1.911,"15min":1.911},"last":1.1619999999999999},{"interface":"front","average":{"1min":1.0189999999999999,"5min":0.95099999999999996,"15min":0.93999999999999995},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.5289999999999999,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":0.94099999999999995}]},{"osd":3,"last update":"Thu Mar 21 10:02:54 2024","interfaces":[{"interface":"back","average":{"1min":1.048,"5min":1.048,"15min":1.048},"min":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":1.5289999999999999},{"interface":"front","average":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"min":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"max":{"1min":1.466,"5min":1.466,"15min":1.466},"last":1.393}]},{"osd":4,"last update":"Thu Mar 21 10:03:23 2024","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.232,"15min":1.232},"min":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":1.7969999999999999,"5min":1.7969999999999999,"15min":1.7969999999999999},"last":1.353},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.6779999999999999,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":1.101}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2849999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542184,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":966656,"data_stored":733091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Mar 21 10:03:50 2024","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.71199999999999997,"15min":0.67000000000000004},"min":{"1min":0.48999999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.77700000000000002,"15min":0.71899999999999997},"min":{"1min":0.49099999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.7669999999999999,"5min":1.7669999999999999,"15min":1.7669999999999999},"last":1.413}]},{"osd":2,"last update":"Thu Mar 21 10:03:20 2024","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":1.1180000000000001,"15min":1.1619999999999999},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.0649999999999999,"5min":8.8320000000000007,"15min":8.8320000000000007},"last":1.304},{"interface":"front","average":{"1min":1.01,"5min":1.222,"15min":1.2569999999999999},"min":{"1min":0.747,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.226,"5min":8.7400000000000002,"15min":8.7400000000000002},"last":0.90400000000000003}]},{"osd":3,"last update":"Thu Mar 21 10:02:54 2024","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":1.069,"15min":1.069},"min":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":1.488},{"interface":"front","average":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"min":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"max":{"1min":1.6950000000000001,"5min":1.6950000000000001,"15min":1.6950000000000001},"last":1.004}]},{"osd":4,"last update":"Thu Mar 21 10:03:23 2024","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":1.542},{"interface":"front","average":{"1min":1.242,"5min":1.242,"15min":1.242},"min":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":1.585,"5min":1.585,"15min":1.585},"last":1.5720000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6399999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-21T10:03:57.801 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-03-21T10:03:57.801 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-03-21T10:03:57.801 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-03-21T10:03:57.801 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 070ed3e4-e769-11ee-95cd-87774f69a715 -- ceph health --format=json 2024-03-21T10:03:58.211 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi153/config 2024-03-21T10:03:58.674 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:03:58 smithi153 ceph-mon[29117]: from='client.14434 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T10:03:58.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:03:58 smithi174 ceph-mon[37496]: from='client.14434 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T10:03:59.680 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:03:59 smithi153 ceph-mon[29117]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:03:59.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:03:59 smithi174 ceph-mon[37496]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:04:00.152 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:00.151+0000 7f81c2000700 1 -- 172.21.15.153:0/3899667758 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f81bc1071d0 msgr2=0x7f81bc107650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:04:00.152 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:00.151+0000 7f81c2000700 1 --2- 172.21.15.153:0/3899667758 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f81bc1071d0 0x7f81bc107650 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f81a8009910 tx=0x7f81a8009c20 comp rx=0 tx=0).stop 2024-03-21T10:04:00.152 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:00.151+0000 7f81c2000700 1 -- 172.21.15.153:0/3899667758 shutdown_connections 2024-03-21T10:04:00.152 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:00.151+0000 7f81c2000700 1 --2- 172.21.15.153:0/3899667758 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f81bc1071d0 0x7f81bc107650 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:04:00.152 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:00.151+0000 7f81c2000700 1 --2- 172.21.15.153:0/3899667758 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f81bc10d280 0x7f81bc10d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:04:00.153 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:00.151+0000 7f81c2000700 1 -- 172.21.15.153:0/3899667758 >> 172.21.15.153:0/3899667758 conn(0x7f81bc0769e0 msgr2=0x7f81bc078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:04:00.153 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:00.151+0000 7f81c2000700 1 -- 172.21.15.153:0/3899667758 shutdown_connections 2024-03-21T10:04:00.153 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:00.151+0000 7f81c2000700 1 -- 172.21.15.153:0/3899667758 wait complete. 2024-03-21T10:04:00.155 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:00.152+0000 7f81c2000700 1 Processor -- start 2024-03-21T10:04:00.155 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:00.152+0000 7f81c2000700 1 -- start start 2024-03-21T10:04:00.155 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:00.153+0000 7f81c2000700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f81bc1071d0 0x7f81bc19ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:04:00.155 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:00.153+0000 7f81c2000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f81bc10d280 0x7f81bc19d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:04:00.155 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:00.153+0000 7f81c2000700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81bc19d880 con 0x7f81bc10d280 2024-03-21T10:04:00.155 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:00.153+0000 7f81c2000700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81bc1a05b0 con 0x7f81bc1071d0 2024-03-21T10:04:00.155 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:00.153+0000 7f81baa00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f81bc10d280 0x7f81bc19d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:04:00.155 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:00.153+0000 7f81baa00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f81bc10d280 0x7f81bc19d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:47480/0 (socket says 172.21.15.153:47480) 2024-03-21T10:04:00.155 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:00.153+0000 7f81baa00700 1 -- 172.21.15.153:0/3061268701 learned_addr learned my addr 172.21.15.153:0/3061268701 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:04:00.156 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:00.153+0000 7f81bb400700 1 --2- 172.21.15.153:0/3061268701 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f81bc1071d0 0x7f81bc19ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:04:00.156 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:00.153+0000 7f81baa00700 1 -- 172.21.15.153:0/3061268701 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f81bc1071d0 msgr2=0x7f81bc19ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:04:00.156 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:00.153+0000 7f81baa00700 1 --2- 172.21.15.153:0/3061268701 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f81bc1071d0 0x7f81bc19ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:04:00.156 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:00.153+0000 7f81baa00700 1 -- 172.21.15.153:0/3061268701 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81a80095f0 con 0x7f81bc10d280 2024-03-21T10:04:00.156 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:00.154+0000 7f81baa00700 1 --2- 172.21.15.153:0/3061268701 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f81bc10d280 0x7f81bc19d1f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f81a8006010 tx=0x7f81a800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:04:00.156 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:00.154+0000 7f81b3e00700 1 -- 172.21.15.153:0/3061268701 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f81a801d070 con 0x7f81bc10d280 2024-03-21T10:04:00.156 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:00.154+0000 7f81b3e00700 1 -- 172.21.15.153:0/3061268701 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f81a8004f50 con 0x7f81bc10d280 2024-03-21T10:04:00.156 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:00.154+0000 7f81b3e00700 1 -- 172.21.15.153:0/3061268701 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f81a8021640 con 0x7f81bc10d280 2024-03-21T10:04:00.157 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:00.154+0000 7f81c2000700 1 -- 172.21.15.153:0/3061268701 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81bc1a0830 con 0x7f81bc10d280 2024-03-21T10:04:00.157 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:00.154+0000 7f81c2000700 1 -- 172.21.15.153:0/3061268701 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81bc1a0d20 con 0x7f81bc10d280 2024-03-21T10:04:00.158 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:00.155+0000 7f81c2000700 1 -- 172.21.15.153:0/3061268701 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81bc10f6c0 con 0x7f81bc10d280 2024-03-21T10:04:00.158 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:00.156+0000 7f81b3e00700 1 -- 172.21.15.153:0/3061268701 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f81a800f460 con 0x7f81bc10d280 2024-03-21T10:04:00.158 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:00.157+0000 7f81b3e00700 1 --2- 172.21.15.153:0/3061268701 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f81a006c4e0 0x7f81a006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:04:00.158 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:00.157+0000 7f81b3e00700 1 -- 172.21.15.153:0/3061268701 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f81a808c760 con 0x7f81bc10d280 2024-03-21T10:04:00.158 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:00.157+0000 7f81bb400700 1 --2- 172.21.15.153:0/3061268701 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f81a006c4e0 0x7f81a006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:04:00.159 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:00.158+0000 7f81bb400700 1 --2- 172.21.15.153:0/3061268701 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f81a006c4e0 0x7f81a006e9a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f81ac005950 tx=0x7f81ac0058e0 comp rx=0 tx=0).ready entity=mgr.14221 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:04:00.163 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:00.162+0000 7f81b3e00700 1 -- 172.21.15.153:0/3061268701 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f81a8057260 con 0x7f81bc10d280 2024-03-21T10:04:00.423 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:00.423+0000 7f81c2000700 1 -- 172.21.15.153:0/3061268701 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f81bc04ea90 con 0x7f81bc10d280 2024-03-21T10:04:00.424 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:00.423+0000 7f81b3e00700 1 -- 172.21.15.153:0/3061268701 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f81a800fe60 con 0x7f81bc10d280 2024-03-21T10:04:00.424 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:04:00.424 INFO:teuthology.orchestra.run.smithi153.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-21T10:04:00.427 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:00.426+0000 7f81c2000700 1 -- 172.21.15.153:0/3061268701 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f81a006c4e0 msgr2=0x7f81a006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:04:00.427 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:00.426+0000 7f81c2000700 1 --2- 172.21.15.153:0/3061268701 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f81a006c4e0 0x7f81a006e9a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f81ac005950 tx=0x7f81ac0058e0 comp rx=0 tx=0).stop 2024-03-21T10:04:00.427 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:00.426+0000 7f81c2000700 1 -- 172.21.15.153:0/3061268701 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f81bc10d280 msgr2=0x7f81bc19d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:04:00.427 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:00.426+0000 7f81c2000700 1 --2- 172.21.15.153:0/3061268701 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f81bc10d280 0x7f81bc19d1f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f81a8006010 tx=0x7f81a800be20 comp rx=0 tx=0).stop 2024-03-21T10:04:00.427 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:00.426+0000 7f81c2000700 1 -- 172.21.15.153:0/3061268701 shutdown_connections 2024-03-21T10:04:00.427 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:00.426+0000 7f81c2000700 1 --2- 172.21.15.153:0/3061268701 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f81a006c4e0 0x7f81a006e9a0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:04:00.427 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:00.426+0000 7f81c2000700 1 --2- 172.21.15.153:0/3061268701 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f81bc1071d0 0x7f81bc19ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:04:00.427 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:00.426+0000 7f81c2000700 1 --2- 172.21.15.153:0/3061268701 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f81bc10d280 0x7f81bc19d1f0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:04:00.427 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:00.426+0000 7f81c2000700 1 -- 172.21.15.153:0/3061268701 >> 172.21.15.153:0/3061268701 conn(0x7f81bc0769e0 msgr2=0x7f81bc078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:04:00.427 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:00.426+0000 7f81c2000700 1 -- 172.21.15.153:0/3061268701 shutdown_connections 2024-03-21T10:04:00.427 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:00.426+0000 7f81c2000700 1 -- 172.21.15.153:0/3061268701 wait complete. 2024-03-21T10:04:00.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:00 smithi153 ceph-mon[29117]: from='client.? 172.21.15.153:0/3061268701' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-21T10:04:00.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:00 smithi174 ceph-mon[37496]: from='client.? 172.21.15.153:0/3061268701' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-21T10:04:01.181 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-03-21T10:04:01.181 INFO:tasks.cephadm:Setup complete, yielding 2024-03-21T10:04:01.181 INFO:teuthology.run_tasks:Running task print... 2024-03-21T10:04:01.193 INFO:teuthology.task.print:**** done end installing v18.2.1 cephadm ... 2024-03-21T10:04:01.194 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-21T10:04:01.221 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi153.front.sepia.ceph.com 2024-03-21T10:04:01.221 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 070ed3e4-e769-11ee-95cd-87774f69a715 -- bash -c 'ceph config set mgr mgr/cephadm/use_repo_digest true --force' 2024-03-21T10:04:01.638 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi153/config 2024-03-21T10:04:01.786 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:01 smithi153 ceph-mon[29117]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:04:01.786 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:01 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:04:01.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:01 smithi174 ceph-mon[37496]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:04:01.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:01 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:04:02.786 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:02 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:04:02.786 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:02 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:04:02.786 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:02 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:04:02.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:02 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:04:02.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:02 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:04:02.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:02 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:04:03.595 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:03.594+0000 7ff5a2400700 1 -- 172.21.15.153:0/1073340912 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff59c103f70 msgr2=0x7ff59c106360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:04:03.595 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:03.594+0000 7ff5a2400700 1 --2- 172.21.15.153:0/1073340912 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff59c103f70 0x7ff59c106360 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff588009910 tx=0x7ff588009c20 comp rx=0 tx=0).stop 2024-03-21T10:04:03.595 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:03.594+0000 7ff5a2400700 1 -- 172.21.15.153:0/1073340912 shutdown_connections 2024-03-21T10:04:03.596 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:03.594+0000 7ff5a2400700 1 --2- 172.21.15.153:0/1073340912 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff59c103f70 0x7ff59c106360 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:04:03.596 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:03.594+0000 7ff5a2400700 1 --2- 172.21.15.153:0/1073340912 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff59c101640 0x7ff59c103a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:04:03.596 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:03.594+0000 7ff5a2400700 1 -- 172.21.15.153:0/1073340912 >> 172.21.15.153:0/1073340912 conn(0x7ff59c0fb050 msgr2=0x7ff59c0fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:04:03.596 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:03.594+0000 7ff5a2400700 1 -- 172.21.15.153:0/1073340912 shutdown_connections 2024-03-21T10:04:03.596 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:03.595+0000 7ff5a2400700 1 -- 172.21.15.153:0/1073340912 wait complete. 2024-03-21T10:04:03.597 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:03.596+0000 7ff5a2400700 1 Processor -- start 2024-03-21T10:04:03.597 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:03.596+0000 7ff5a2400700 1 -- start start 2024-03-21T10:04:03.597 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:03.596+0000 7ff5a2400700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff59c101640 0x7ff59c198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:04:03.597 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:03.596+0000 7ff5a2400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff59c103f70 0x7ff59c198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:04:03.598 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:03.596+0000 7ff5a2400700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff59c199090 con 0x7ff59c103f70 2024-03-21T10:04:03.598 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:03.596+0000 7ff5a2400700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff59c1991d0 con 0x7ff59c101640 2024-03-21T10:04:03.598 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:03.596+0000 7ff59aa00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff59c103f70 0x7ff59c198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:04:03.598 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:03.597+0000 7ff59aa00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff59c103f70 0x7ff59c198a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:48020/0 (socket says 172.21.15.153:48020) 2024-03-21T10:04:03.598 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:03.597+0000 7ff59aa00700 1 -- 172.21.15.153:0/221703796 learned_addr learned my addr 172.21.15.153:0/221703796 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:04:03.598 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:03.597+0000 7ff59b400700 1 --2- 172.21.15.153:0/221703796 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff59c101640 0x7ff59c198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:04:03.598 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:03.597+0000 7ff59aa00700 1 -- 172.21.15.153:0/221703796 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff59c101640 msgr2=0x7ff59c198530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:04:03.598 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:03.597+0000 7ff59aa00700 1 --2- 172.21.15.153:0/221703796 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff59c101640 0x7ff59c198530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:04:03.598 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:03.597+0000 7ff59aa00700 1 -- 172.21.15.153:0/221703796 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5880095f0 con 0x7ff59c103f70 2024-03-21T10:04:03.598 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:03.597+0000 7ff59b400700 1 --2- 172.21.15.153:0/221703796 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff59c101640 0x7ff59c198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:04:03.599 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:03.597+0000 7ff59aa00700 1 --2- 172.21.15.153:0/221703796 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff59c103f70 0x7ff59c198a70 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff588005b40 tx=0x7ff58800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:04:03.599 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:03.598+0000 7ff593e00700 1 -- 172.21.15.153:0/221703796 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff58801d070 con 0x7ff59c103f70 2024-03-21T10:04:03.599 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:03.598+0000 7ff593e00700 1 -- 172.21.15.153:0/221703796 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff58800f460 con 0x7ff59c103f70 2024-03-21T10:04:03.599 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:03.598+0000 7ff5a2400700 1 -- 172.21.15.153:0/221703796 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff59c19dc20 con 0x7ff59c103f70 2024-03-21T10:04:03.599 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:03.598+0000 7ff593e00700 1 -- 172.21.15.153:0/221703796 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff588021640 con 0x7ff59c103f70 2024-03-21T10:04:03.599 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:03.598+0000 7ff5a2400700 1 -- 172.21.15.153:0/221703796 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff59c19e110 con 0x7ff59c103f70 2024-03-21T10:04:03.600 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:03.599+0000 7ff5a2400700 1 -- 172.21.15.153:0/221703796 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff59c0fcc20 con 0x7ff59c103f70 2024-03-21T10:04:03.600 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:03.600+0000 7ff593e00700 1 -- 172.21.15.153:0/221703796 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7ff58800fa90 con 0x7ff59c103f70 2024-03-21T10:04:03.602 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:03.601+0000 7ff593e00700 1 --2- 172.21.15.153:0/221703796 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7ff58006c4e0 0x7ff58006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:04:03.602 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:03.601+0000 7ff593e00700 1 -- 172.21.15.153:0/221703796 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7ff58808c910 con 0x7ff59c103f70 2024-03-21T10:04:03.602 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:03.602+0000 7ff59b400700 1 --2- 172.21.15.153:0/221703796 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7ff58006c4e0 0x7ff58006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:04:03.602 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:03.602+0000 7ff59b400700 1 --2- 172.21.15.153:0/221703796 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7ff58006c4e0 0x7ff58006e9a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff58c005950 tx=0x7ff58c0058e0 comp rx=0 tx=0).ready entity=mgr.14221 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:04:03.606 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:03.606+0000 7ff593e00700 1 -- 172.21.15.153:0/221703796 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7ff58805ae20 con 0x7ff59c103f70 2024-03-21T10:04:03.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:03 smithi153 ceph-mon[29117]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:04:03.820 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:03.819+0000 7ff5a2400700 1 -- 172.21.15.153:0/221703796 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/use_repo_digest}] v 0) v1 -- 0x7ff59c02cc70 con 0x7ff59c103f70 2024-03-21T10:04:03.831 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:03.830+0000 7ff593e00700 1 -- 172.21.15.153:0/221703796 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/use_repo_digest}]=0 v20)=0 v20) v1 ==== 143+0+0 (secure 0 0 0) 0x7ff58805a9b0 con 0x7ff59c103f70 2024-03-21T10:04:03.834 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:03.834+0000 7ff5a2400700 1 -- 172.21.15.153:0/221703796 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7ff58006c4e0 msgr2=0x7ff58006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:04:03.834 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:03.834+0000 7ff5a2400700 1 --2- 172.21.15.153:0/221703796 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7ff58006c4e0 0x7ff58006e9a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff58c005950 tx=0x7ff58c0058e0 comp rx=0 tx=0).stop 2024-03-21T10:04:03.835 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:03.834+0000 7ff5a2400700 1 -- 172.21.15.153:0/221703796 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff59c103f70 msgr2=0x7ff59c198a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:04:03.835 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:03.834+0000 7ff5a2400700 1 --2- 172.21.15.153:0/221703796 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff59c103f70 0x7ff59c198a70 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff588005b40 tx=0x7ff58800be20 comp rx=0 tx=0).stop 2024-03-21T10:04:03.835 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:03.834+0000 7ff5a2400700 1 -- 172.21.15.153:0/221703796 shutdown_connections 2024-03-21T10:04:03.835 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:03.834+0000 7ff5a2400700 1 --2- 172.21.15.153:0/221703796 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7ff58006c4e0 0x7ff58006e9a0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:04:03.835 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:03.834+0000 7ff5a2400700 1 --2- 172.21.15.153:0/221703796 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff59c101640 0x7ff59c198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:04:03.835 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:03.834+0000 7ff5a2400700 1 --2- 172.21.15.153:0/221703796 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff59c103f70 0x7ff59c198a70 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:04:03.835 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:03.834+0000 7ff5a2400700 1 -- 172.21.15.153:0/221703796 >> 172.21.15.153:0/221703796 conn(0x7ff59c0fb050 msgr2=0x7ff59c0ffcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:04:03.835 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:03.834+0000 7ff5a2400700 1 -- 172.21.15.153:0/221703796 shutdown_connections 2024-03-21T10:04:03.835 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:03.834+0000 7ff5a2400700 1 -- 172.21.15.153:0/221703796 wait complete. 2024-03-21T10:04:03.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:03 smithi174 ceph-mon[37496]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:04:04.550 INFO:teuthology.run_tasks:Running task print... 2024-03-21T10:04:04.559 INFO:teuthology.task.print:**** done cephadm.shell ceph config set mgr... 2024-03-21T10:04:04.560 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-21T10:04:04.595 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi153.front.sepia.ceph.com 2024-03-21T10:04:04.595 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 070ed3e4-e769-11ee-95cd-87774f69a715 -- bash -c 'ceph orch status' 2024-03-21T10:04:04.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:04 smithi174 ceph-mon[37496]: from='client.? 172.21.15.153:0/221703796' entity='client.admin' 2024-03-21T10:04:04.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:04 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:04:04.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:04 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:04:04.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:04 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:04:04.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:04 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:04:04.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:04 smithi174 ceph-mon[37496]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:04:05.003 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi153/config 2024-03-21T10:04:05.156 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:04 smithi153 ceph-mon[29117]: from='client.? 172.21.15.153:0/221703796' entity='client.admin' 2024-03-21T10:04:05.156 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:04 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:04:05.156 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:04 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:04:05.156 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:04 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:04:05.156 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:04 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:04:05.156 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:04 smithi153 ceph-mon[29117]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:04:07.032 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:07.030+0000 7f2dce400700 1 -- 172.21.15.153:0/4050605672 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2dc8107200 msgr2=0x7f2dc8107620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:04:07.032 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:07.030+0000 7f2dce400700 1 --2- 172.21.15.153:0/4050605672 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2dc8107200 0x7f2dc8107620 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f2db8009910 tx=0x7f2db8009c20 comp rx=0 tx=0).stop 2024-03-21T10:04:07.032 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:07.031+0000 7f2dce400700 1 -- 172.21.15.153:0/4050605672 shutdown_connections 2024-03-21T10:04:07.032 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:07.031+0000 7f2dce400700 1 --2- 172.21.15.153:0/4050605672 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2dc8108400 0x7f2dc8108860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:04:07.032 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:07.031+0000 7f2dce400700 1 --2- 172.21.15.153:0/4050605672 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2dc8107200 0x7f2dc8107620 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:04:07.032 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:07.031+0000 7f2dce400700 1 -- 172.21.15.153:0/4050605672 >> 172.21.15.153:0/4050605672 conn(0x7f2dc80761b0 msgr2=0x7f2dc8078610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:04:07.032 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:07.031+0000 7f2dce400700 1 -- 172.21.15.153:0/4050605672 shutdown_connections 2024-03-21T10:04:07.032 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:07.031+0000 7f2dce400700 1 -- 172.21.15.153:0/4050605672 wait complete. 2024-03-21T10:04:07.033 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:07.032+0000 7f2dce400700 1 Processor -- start 2024-03-21T10:04:07.034 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:07.032+0000 7f2dce400700 1 -- start start 2024-03-21T10:04:07.034 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:07.032+0000 7f2dce400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2dc8107200 0x7f2dc819c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:04:07.034 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:07.032+0000 7f2dce400700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2dc8108400 0x7f2dc819cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:04:07.034 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:07.032+0000 7f2dce400700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dc819d510 con 0x7f2dc8107200 2024-03-21T10:04:07.034 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:07.032+0000 7f2dce400700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dc819d650 con 0x7f2dc8108400 2024-03-21T10:04:07.034 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:07.032+0000 7f2dc7400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2dc8107200 0x7f2dc819c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:04:07.034 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:07.033+0000 7f2dc7400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2dc8107200 0x7f2dc819c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:48042/0 (socket says 172.21.15.153:48042) 2024-03-21T10:04:07.034 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:07.033+0000 7f2dc7400700 1 -- 172.21.15.153:0/3408812090 learned_addr learned my addr 172.21.15.153:0/3408812090 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:04:07.034 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:07.033+0000 7f2dc6a00700 1 --2- 172.21.15.153:0/3408812090 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2dc8108400 0x7f2dc819cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:04:07.034 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:07.033+0000 7f2dc7400700 1 -- 172.21.15.153:0/3408812090 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2dc8108400 msgr2=0x7f2dc819cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:04:07.034 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:07.033+0000 7f2dc7400700 1 --2- 172.21.15.153:0/3408812090 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2dc8108400 0x7f2dc819cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:04:07.034 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:07.033+0000 7f2dc7400700 1 -- 172.21.15.153:0/3408812090 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2db80095f0 con 0x7f2dc8107200 2024-03-21T10:04:07.034 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:07.033+0000 7f2dc7400700 1 --2- 172.21.15.153:0/3408812090 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2dc8107200 0x7f2dc819c9b0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f2db800bd70 tx=0x7f2db800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:04:07.035 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:07.034+0000 7f2dbfe00700 1 -- 172.21.15.153:0/3408812090 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2db801d070 con 0x7f2dc8107200 2024-03-21T10:04:07.035 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:07.034+0000 7f2dbfe00700 1 -- 172.21.15.153:0/3408812090 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f2db800f460 con 0x7f2dc8107200 2024-03-21T10:04:07.035 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:07.034+0000 7f2dbfe00700 1 -- 172.21.15.153:0/3408812090 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2db8021640 con 0x7f2dc8107200 2024-03-21T10:04:07.035 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:07.034+0000 7f2dce400700 1 -- 172.21.15.153:0/3408812090 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2dc81a20a0 con 0x7f2dc8107200 2024-03-21T10:04:07.035 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:07.034+0000 7f2dce400700 1 -- 172.21.15.153:0/3408812090 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dc81a2540 con 0x7f2dc8107200 2024-03-21T10:04:07.036 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:07.035+0000 7f2dce400700 1 -- 172.21.15.153:0/3408812090 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2dc810fd60 con 0x7f2dc8107200 2024-03-21T10:04:07.036 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:07.036+0000 7f2dbfe00700 1 -- 172.21.15.153:0/3408812090 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f2db800fab0 con 0x7f2dc8107200 2024-03-21T10:04:07.037 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:07.037+0000 7f2dbfe00700 1 --2- 172.21.15.153:0/3408812090 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f2dac06c4e0 0x7f2dac06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:04:07.037 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:07.037+0000 7f2dbfe00700 1 -- 172.21.15.153:0/3408812090 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f2db808ca10 con 0x7f2dc8107200 2024-03-21T10:04:07.037 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:07.037+0000 7f2dc6a00700 1 --2- 172.21.15.153:0/3408812090 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f2dac06c4e0 0x7f2dac06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:04:07.038 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:07.038+0000 7f2dc6a00700 1 --2- 172.21.15.153:0/3408812090 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f2dac06c4e0 0x7f2dac06e9a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2db4005fd0 tx=0x7f2db400a560 comp rx=0 tx=0).ready entity=mgr.14221 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:04:07.042 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:07.042+0000 7f2dbfe00700 1 -- 172.21.15.153:0/3408812090 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f2db805af20 con 0x7f2dc8107200 2024-03-21T10:04:07.279 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:07.279+0000 7f2dce400700 1 -- 172.21.15.153:0/3408812090 --> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2dc810cd50 con 0x7f2dac06c4e0 2024-03-21T10:04:07.281 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:07.280+0000 7f2dbfe00700 1 -- 172.21.15.153:0/3408812090 <== mgr.14221 v2:172.21.15.153:6800/1639929315 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f2dc810cd50 con 0x7f2dac06c4e0 2024-03-21T10:04:07.281 INFO:teuthology.orchestra.run.smithi153.stdout:Backend: cephadm 2024-03-21T10:04:07.281 INFO:teuthology.orchestra.run.smithi153.stdout:Available: Yes 2024-03-21T10:04:07.281 INFO:teuthology.orchestra.run.smithi153.stdout:Paused: No 2024-03-21T10:04:07.284 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:07.284+0000 7f2dce400700 1 -- 172.21.15.153:0/3408812090 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f2dac06c4e0 msgr2=0x7f2dac06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:04:07.284 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:07.284+0000 7f2dce400700 1 --2- 172.21.15.153:0/3408812090 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f2dac06c4e0 0x7f2dac06e9a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2db4005fd0 tx=0x7f2db400a560 comp rx=0 tx=0).stop 2024-03-21T10:04:07.285 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:07.284+0000 7f2dce400700 1 -- 172.21.15.153:0/3408812090 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2dc8107200 msgr2=0x7f2dc819c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:04:07.285 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:07.284+0000 7f2dce400700 1 --2- 172.21.15.153:0/3408812090 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2dc8107200 0x7f2dc819c9b0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f2db800bd70 tx=0x7f2db800be50 comp rx=0 tx=0).stop 2024-03-21T10:04:07.285 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:07.284+0000 7f2dce400700 1 -- 172.21.15.153:0/3408812090 shutdown_connections 2024-03-21T10:04:07.285 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:07.284+0000 7f2dce400700 1 --2- 172.21.15.153:0/3408812090 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f2dac06c4e0 0x7f2dac06e9a0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:04:07.285 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:07.284+0000 7f2dce400700 1 --2- 172.21.15.153:0/3408812090 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2dc8107200 0x7f2dc819c9b0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:04:07.285 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:07.284+0000 7f2dce400700 1 --2- 172.21.15.153:0/3408812090 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2dc8108400 0x7f2dc819cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:04:07.285 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:07.284+0000 7f2dce400700 1 -- 172.21.15.153:0/3408812090 >> 172.21.15.153:0/3408812090 conn(0x7f2dc80761b0 msgr2=0x7f2dc810b630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:04:07.285 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:07.284+0000 7f2dce400700 1 -- 172.21.15.153:0/3408812090 shutdown_connections 2024-03-21T10:04:07.285 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:07.284+0000 7f2dce400700 1 -- 172.21.15.153:0/3408812090 wait complete. 2024-03-21T10:04:07.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:07 smithi153 ceph-mon[29117]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:04:07.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:07 smithi174 ceph-mon[37496]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:04:08.017 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 070ed3e4-e769-11ee-95cd-87774f69a715 -- bash -c 'ceph orch ps' 2024-03-21T10:04:08.416 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi153/config 2024-03-21T10:04:08.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:08 smithi174 ceph-mon[37496]: from='client.14446 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:04:08.722 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:08 smithi153 ceph-mon[29117]: from='client.14446 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:04:09.698 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:09 smithi174 ceph-mon[37496]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:04:09.699 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:09 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:04:09.722 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:09 smithi153 ceph-mon[29117]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:04:09.722 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:09 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:04:10.487 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:10.486+0000 7f0c88e00700 1 -- 172.21.15.153:0/1671737223 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0c84074dc0 msgr2=0x7f0c84073220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:04:10.488 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:10.486+0000 7f0c88e00700 1 --2- 172.21.15.153:0/1671737223 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0c84074dc0 0x7f0c84073220 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f0c74009910 tx=0x7f0c74009c20 comp rx=0 tx=0).stop 2024-03-21T10:04:10.488 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:10.487+0000 7f0c88e00700 1 -- 172.21.15.153:0/1671737223 shutdown_connections 2024-03-21T10:04:10.488 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:10.487+0000 7f0c88e00700 1 --2- 172.21.15.153:0/1671737223 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0c840737f0 0x7f0c84073c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:04:10.488 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:10.487+0000 7f0c88e00700 1 --2- 172.21.15.153:0/1671737223 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0c84074dc0 0x7f0c84073220 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:04:10.488 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:10.487+0000 7f0c88e00700 1 -- 172.21.15.153:0/1671737223 >> 172.21.15.153:0/1671737223 conn(0x7f0c840fc110 msgr2=0x7f0c840fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:04:10.488 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:10.487+0000 7f0c88e00700 1 -- 172.21.15.153:0/1671737223 shutdown_connections 2024-03-21T10:04:10.488 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:10.487+0000 7f0c88e00700 1 -- 172.21.15.153:0/1671737223 wait complete. 2024-03-21T10:04:10.489 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:10.489+0000 7f0c88e00700 1 Processor -- start 2024-03-21T10:04:10.489 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:10.489+0000 7f0c88e00700 1 -- start start 2024-03-21T10:04:10.490 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:10.489+0000 7f0c88e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0c840737f0 0x7f0c84198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:04:10.490 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:10.489+0000 7f0c88e00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0c84074dc0 0x7f0c84198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:04:10.490 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:10.489+0000 7f0c88e00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c841990d0 con 0x7f0c840737f0 2024-03-21T10:04:10.490 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:10.489+0000 7f0c88e00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c84199210 con 0x7f0c84074dc0 2024-03-21T10:04:10.490 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:10.490+0000 7f0c81e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0c840737f0 0x7f0c84198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:04:10.490 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:10.490+0000 7f0c81e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0c840737f0 0x7f0c84198570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:48050/0 (socket says 172.21.15.153:48050) 2024-03-21T10:04:10.490 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:10.490+0000 7f0c81e00700 1 -- 172.21.15.153:0/3171217406 learned_addr learned my addr 172.21.15.153:0/3171217406 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:04:10.491 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:10.490+0000 7f0c81400700 1 --2- 172.21.15.153:0/3171217406 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0c84074dc0 0x7f0c84198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:04:10.491 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:10.490+0000 7f0c81e00700 1 -- 172.21.15.153:0/3171217406 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0c84074dc0 msgr2=0x7f0c84198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:04:10.492 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:10.490+0000 7f0c81e00700 1 --2- 172.21.15.153:0/3171217406 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0c84074dc0 0x7f0c84198ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:04:10.492 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:10.490+0000 7f0c81e00700 1 -- 172.21.15.153:0/3171217406 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c740095f0 con 0x7f0c840737f0 2024-03-21T10:04:10.492 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:10.490+0000 7f0c81e00700 1 --2- 172.21.15.153:0/3171217406 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0c840737f0 0x7f0c84198570 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f0c7400bda0 tx=0x7f0c7400be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:04:10.492 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:10.491+0000 7f0c7aa00700 1 -- 172.21.15.153:0/3171217406 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c7401d070 con 0x7f0c840737f0 2024-03-21T10:04:10.492 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:10.491+0000 7f0c7aa00700 1 -- 172.21.15.153:0/3171217406 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0c7400f460 con 0x7f0c840737f0 2024-03-21T10:04:10.492 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:10.491+0000 7f0c88e00700 1 -- 172.21.15.153:0/3171217406 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c8419dc60 con 0x7f0c840737f0 2024-03-21T10:04:10.492 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:10.491+0000 7f0c7aa00700 1 -- 172.21.15.153:0/3171217406 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c74021620 con 0x7f0c840737f0 2024-03-21T10:04:10.492 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:10.491+0000 7f0c88e00700 1 -- 172.21.15.153:0/3171217406 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c8419e150 con 0x7f0c840737f0 2024-03-21T10:04:10.492 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:10.492+0000 7f0c88e00700 1 -- 172.21.15.153:0/3171217406 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c8410b920 con 0x7f0c840737f0 2024-03-21T10:04:10.493 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:10.493+0000 7f0c7aa00700 1 -- 172.21.15.153:0/3171217406 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f0c7400f5d0 con 0x7f0c840737f0 2024-03-21T10:04:10.495 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:10.494+0000 7f0c7aa00700 1 --2- 172.21.15.153:0/3171217406 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f0c6806c4e0 0x7f0c6806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:04:10.495 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:10.494+0000 7f0c7aa00700 1 -- 172.21.15.153:0/3171217406 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f0c7408c7d0 con 0x7f0c840737f0 2024-03-21T10:04:10.495 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:10.494+0000 7f0c81400700 1 --2- 172.21.15.153:0/3171217406 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f0c6806c4e0 0x7f0c6806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:04:10.495 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:10.495+0000 7f0c81400700 1 --2- 172.21.15.153:0/3171217406 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f0c6806c4e0 0x7f0c6806e9a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0c70005950 tx=0x7f0c700058e0 comp rx=0 tx=0).ready entity=mgr.14221 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:04:10.499 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:10.499+0000 7f0c7aa00700 1 -- 172.21.15.153:0/3171217406 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f0c7405ad80 con 0x7f0c840737f0 2024-03-21T10:04:10.724 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:10.722+0000 7f0c88e00700 1 -- 172.21.15.153:0/3171217406 --> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f0c841080b0 con 0x7f0c6806c4e0 2024-03-21T10:04:10.735 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:10.734+0000 7f0c7aa00700 1 -- 172.21.15.153:0/3171217406 <== mgr.14221 v2:172.21.15.153:6800/1639929315 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+2860 (secure 0 0 0) 0x7f0c841080b0 con 0x7f0c6806c4e0 2024-03-21T10:04:10.736 INFO:teuthology.orchestra.run.smithi153.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-21T10:04:10.736 INFO:teuthology.orchestra.run.smithi153.stdout:alertmanager.smithi153 smithi153 *:9093,9094 running (4m) 2m ago 6m 13.6M - 0.25.0 c8568f914cd2 2cfcc225c7e6 2024-03-21T10:04:10.737 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi153 smithi153 running (7m) 2m ago 7m 7117k - 18.2.1 5be31c24972a 5b61994396a6 2024-03-21T10:04:10.737 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi174 smithi174 running (5m) 75s ago 5m 6962k - 18.2.1 5be31c24972a 4f7a5fc3ac28 2024-03-21T10:04:10.737 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi153 smithi153 running (7m) 2m ago 7m 6665k - 18.2.1 5be31c24972a 90a8b8ea03d1 2024-03-21T10:04:10.737 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi174 smithi174 running (5m) 75s ago 5m 6665k - 18.2.1 5be31c24972a a4c929e4dc77 2024-03-21T10:04:10.737 INFO:teuthology.orchestra.run.smithi153.stdout:grafana.smithi153 smithi153 *:3000 running (4m) 2m ago 6m 73.3M - 9.4.7 954c08fa6188 3af5800ecfef 2024-03-21T10:04:10.737 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi153.tluxni smithi153 *:9283,8765,8443 running (8m) 2m ago 8m 483M - 18.2.1 5be31c24972a 4978543154ff 2024-03-21T10:04:10.737 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi174.qpcwat smithi174 *:8443,9283,8765 running (5m) 75s ago 5m 440M - 18.2.1 5be31c24972a 6d0c64be5134 2024-03-21T10:04:10.737 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi153 smithi153 running (8m) 2m ago 8m 43.0M 2048M 18.2.1 5be31c24972a 813963a95d69 2024-03-21T10:04:10.737 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi174 smithi174 running (5m) 75s ago 5m 33.3M 2048M 18.2.1 5be31c24972a 99204923eec6 2024-03-21T10:04:10.737 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi153 smithi153 *:9100 running (6m) 2m ago 6m 7830k - 1.5.0 0da6a335fe13 babbc29a0eaf 2024-03-21T10:04:10.737 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi174 smithi174 *:9100 running (5m) 75s ago 5m 7834k - 1.5.0 0da6a335fe13 86841435042a 2024-03-21T10:04:10.737 INFO:teuthology.orchestra.run.smithi153.stdout:osd.0 smithi153 running (4m) 2m ago 4m 39.8M 3947M 18.2.1 5be31c24972a 28912a2319b4 2024-03-21T10:04:10.737 INFO:teuthology.orchestra.run.smithi153.stdout:osd.1 smithi153 running (3m) 2m ago 3m 39.9M 3947M 18.2.1 5be31c24972a ca60681e2314 2024-03-21T10:04:10.737 INFO:teuthology.orchestra.run.smithi153.stdout:osd.2 smithi153 running (3m) 2m ago 3m 39.9M 3947M 18.2.1 5be31c24972a 8db799f8962a 2024-03-21T10:04:10.738 INFO:teuthology.orchestra.run.smithi153.stdout:osd.3 smithi174 running (2m) 75s ago 2m 40.4M 4971M 18.2.1 5be31c24972a dafc89e0658d 2024-03-21T10:04:10.738 INFO:teuthology.orchestra.run.smithi153.stdout:osd.4 smithi174 running (114s) 75s ago 114s 37.9M 4971M 18.2.1 5be31c24972a c4cd15701f05 2024-03-21T10:04:10.738 INFO:teuthology.orchestra.run.smithi153.stdout:osd.5 smithi174 running (82s) 75s ago 81s 11.0M 4971M 18.2.1 5be31c24972a 3b69886ce22e 2024-03-21T10:04:10.738 INFO:teuthology.orchestra.run.smithi153.stdout:prometheus.smithi153 smithi153 *:9095 running (4m) 2m ago 5m 43.3M - 2.43.0 a07b618ecd1d c30cdeb81cc9 2024-03-21T10:04:10.738 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:10.736+0000 7f0c88e00700 1 -- 172.21.15.153:0/3171217406 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f0c6806c4e0 msgr2=0x7f0c6806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:04:10.738 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:10.736+0000 7f0c88e00700 1 --2- 172.21.15.153:0/3171217406 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f0c6806c4e0 0x7f0c6806e9a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0c70005950 tx=0x7f0c700058e0 comp rx=0 tx=0).stop 2024-03-21T10:04:10.739 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:10.736+0000 7f0c88e00700 1 -- 172.21.15.153:0/3171217406 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0c840737f0 msgr2=0x7f0c84198570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:04:10.739 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:10.736+0000 7f0c88e00700 1 --2- 172.21.15.153:0/3171217406 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0c840737f0 0x7f0c84198570 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f0c7400bda0 tx=0x7f0c7400be80 comp rx=0 tx=0).stop 2024-03-21T10:04:10.739 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:10.737+0000 7f0c88e00700 1 -- 172.21.15.153:0/3171217406 shutdown_connections 2024-03-21T10:04:10.739 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:10.737+0000 7f0c88e00700 1 --2- 172.21.15.153:0/3171217406 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f0c6806c4e0 0x7f0c6806e9a0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:04:10.739 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:10.737+0000 7f0c88e00700 1 --2- 172.21.15.153:0/3171217406 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0c840737f0 0x7f0c84198570 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:04:10.739 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:10.737+0000 7f0c88e00700 1 --2- 172.21.15.153:0/3171217406 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0c84074dc0 0x7f0c84198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:04:10.739 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:10.737+0000 7f0c88e00700 1 -- 172.21.15.153:0/3171217406 >> 172.21.15.153:0/3171217406 conn(0x7f0c840fc110 msgr2=0x7f0c84106990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:04:10.739 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:10.737+0000 7f0c88e00700 1 -- 172.21.15.153:0/3171217406 shutdown_connections 2024-03-21T10:04:10.739 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:10.737+0000 7f0c88e00700 1 -- 172.21.15.153:0/3171217406 wait complete. 2024-03-21T10:04:11.473 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 070ed3e4-e769-11ee-95cd-87774f69a715 -- bash -c 'ceph orch ls' 2024-03-21T10:04:11.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:11 smithi174 ceph-mon[37496]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:04:11.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:11 smithi153 ceph-mon[29117]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:04:11.878 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi153/config 2024-03-21T10:04:12.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:12 smithi174 ceph-mon[37496]: from='client.14450 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:04:12.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:12 smithi153 ceph-mon[29117]: from='client.14450 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:04:13.553 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:13 smithi153 ceph-mon[29117]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:04:13.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:13 smithi174 ceph-mon[37496]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:04:14.014 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:14.012+0000 7f7300400700 1 -- 172.21.15.153:0/2357764895 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72f8103f70 msgr2=0x7f72f8106360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:04:14.015 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:14.012+0000 7f7300400700 1 --2- 172.21.15.153:0/2357764895 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72f8103f70 0x7f72f8106360 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f72e4009910 tx=0x7f72e4009c20 comp rx=0 tx=0).stop 2024-03-21T10:04:14.015 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:14.012+0000 7f7300400700 1 -- 172.21.15.153:0/2357764895 shutdown_connections 2024-03-21T10:04:14.015 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:14.012+0000 7f7300400700 1 --2- 172.21.15.153:0/2357764895 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72f8103f70 0x7f72f8106360 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:04:14.015 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:14.012+0000 7f7300400700 1 --2- 172.21.15.153:0/2357764895 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f72f8101640 0x7f72f8103a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:04:14.015 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:14.012+0000 7f7300400700 1 -- 172.21.15.153:0/2357764895 >> 172.21.15.153:0/2357764895 conn(0x7f72f80fb030 msgr2=0x7f72f80fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:04:14.015 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:14.012+0000 7f7300400700 1 -- 172.21.15.153:0/2357764895 shutdown_connections 2024-03-21T10:04:14.015 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:14.012+0000 7f7300400700 1 -- 172.21.15.153:0/2357764895 wait complete. 2024-03-21T10:04:14.019 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:14.013+0000 7f7300400700 1 Processor -- start 2024-03-21T10:04:14.020 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:14.014+0000 7f7300400700 1 -- start start 2024-03-21T10:04:14.020 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:14.014+0000 7f7300400700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f72f8101640 0x7f72f8198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:04:14.020 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:14.014+0000 7f7300400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72f8103f70 0x7f72f8198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:04:14.020 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:14.014+0000 7f7300400700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72f81990c0 con 0x7f72f8103f70 2024-03-21T10:04:14.020 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:14.014+0000 7f7300400700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72f8199200 con 0x7f72f8101640 2024-03-21T10:04:14.020 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:14.014+0000 7f72fd000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72f8103f70 0x7f72f8198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:04:14.020 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:14.014+0000 7f72fd000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72f8103f70 0x7f72f8198aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:59688/0 (socket says 172.21.15.153:59688) 2024-03-21T10:04:14.020 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:14.014+0000 7f72fd000700 1 -- 172.21.15.153:0/1793052520 learned_addr learned my addr 172.21.15.153:0/1793052520 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:04:14.020 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:14.014+0000 7f72fda00700 1 --2- 172.21.15.153:0/1793052520 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f72f8101640 0x7f72f8198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:04:14.020 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:14.015+0000 7f72fd000700 1 -- 172.21.15.153:0/1793052520 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f72f8101640 msgr2=0x7f72f8198560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:04:14.020 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:14.015+0000 7f72fd000700 1 --2- 172.21.15.153:0/1793052520 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f72f8101640 0x7f72f8198560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:04:14.020 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:14.015+0000 7f72fd000700 1 -- 172.21.15.153:0/1793052520 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72e40095f0 con 0x7f72f8103f70 2024-03-21T10:04:14.020 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:14.015+0000 7f72fd000700 1 --2- 172.21.15.153:0/1793052520 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72f8103f70 0x7f72f8198aa0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f72e4005b40 tx=0x7f72e400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:04:14.020 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:14.015+0000 7f72f6000700 1 -- 172.21.15.153:0/1793052520 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f72e401d070 con 0x7f72f8103f70 2024-03-21T10:04:14.021 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:14.015+0000 7f72f6000700 1 -- 172.21.15.153:0/1793052520 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f72e400f460 con 0x7f72f8103f70 2024-03-21T10:04:14.021 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:14.015+0000 7f7300400700 1 -- 172.21.15.153:0/1793052520 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72f819dc50 con 0x7f72f8103f70 2024-03-21T10:04:14.022 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:14.015+0000 7f72f6000700 1 -- 172.21.15.153:0/1793052520 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f72e4021640 con 0x7f72f8103f70 2024-03-21T10:04:14.022 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:14.015+0000 7f7300400700 1 -- 172.21.15.153:0/1793052520 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72f819e140 con 0x7f72f8103f70 2024-03-21T10:04:14.022 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:14.017+0000 7f7300400700 1 -- 172.21.15.153:0/1793052520 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72f80fcc20 con 0x7f72f8103f70 2024-03-21T10:04:14.022 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:14.017+0000 7f72f6000700 1 -- 172.21.15.153:0/1793052520 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f72e400fa90 con 0x7f72f8103f70 2024-03-21T10:04:14.022 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:14.018+0000 7f72f6000700 1 --2- 172.21.15.153:0/1793052520 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f72e006c4e0 0x7f72e006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:04:14.022 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:14.018+0000 7f72f6000700 1 -- 172.21.15.153:0/1793052520 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f72e408c870 con 0x7f72f8103f70 2024-03-21T10:04:14.022 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:14.019+0000 7f72fda00700 1 --2- 172.21.15.153:0/1793052520 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f72e006c4e0 0x7f72e006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:04:14.022 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:14.019+0000 7f72fda00700 1 --2- 172.21.15.153:0/1793052520 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f72e006c4e0 0x7f72e006e9a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f72ec005950 tx=0x7f72ec0058e0 comp rx=0 tx=0).ready entity=mgr.14221 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:04:14.028 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:14.023+0000 7f72f6000700 1 -- 172.21.15.153:0/1793052520 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f72e405ae00 con 0x7f72f8103f70 2024-03-21T10:04:14.251 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:14.245+0000 7f7300400700 1 -- 172.21.15.153:0/1793052520 --> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f72f80611d0 con 0x7f72e006c4e0 2024-03-21T10:04:14.259 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:14.258+0000 7f72f6000700 1 -- 172.21.15.153:0/1793052520 <== mgr.14221 v2:172.21.15.153:6800/1639929315 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+1310 (secure 0 0 0) 0x7f72f80611d0 con 0x7f72e006c4e0 2024-03-21T10:04:14.260 INFO:teuthology.orchestra.run.smithi153.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-03-21T10:04:14.260 INFO:teuthology.orchestra.run.smithi153.stdout:alertmanager ?:9093,9094 1/1 2m ago 7m count:1 2024-03-21T10:04:14.260 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter 2/2 2m ago 7m * 2024-03-21T10:04:14.260 INFO:teuthology.orchestra.run.smithi153.stdout:crash 2/2 2m ago 7m * 2024-03-21T10:04:14.260 INFO:teuthology.orchestra.run.smithi153.stdout:grafana ?:3000 1/1 2m ago 7m count:1 2024-03-21T10:04:14.260 INFO:teuthology.orchestra.run.smithi153.stdout:mgr 2/2 2m ago 7m count:2 2024-03-21T10:04:14.260 INFO:teuthology.orchestra.run.smithi153.stdout:mon 2/2 2m ago 6m smithi153:172.21.15.153=smithi153;smithi174:172.21.15.174=smithi174;count:2 2024-03-21T10:04:14.260 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter ?:9100 2/2 2m ago 7m * 2024-03-21T10:04:14.261 INFO:teuthology.orchestra.run.smithi153.stdout:osd 6 2m ago - 2024-03-21T10:04:14.261 INFO:teuthology.orchestra.run.smithi153.stdout:prometheus ?:9095 1/1 2m ago 7m count:1 2024-03-21T10:04:14.280 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:14.261+0000 7f7300400700 1 -- 172.21.15.153:0/1793052520 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f72e006c4e0 msgr2=0x7f72e006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:04:14.280 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:14.261+0000 7f7300400700 1 --2- 172.21.15.153:0/1793052520 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f72e006c4e0 0x7f72e006e9a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f72ec005950 tx=0x7f72ec0058e0 comp rx=0 tx=0).stop 2024-03-21T10:04:14.280 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:14.261+0000 7f7300400700 1 -- 172.21.15.153:0/1793052520 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72f8103f70 msgr2=0x7f72f8198aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:04:14.280 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:14.261+0000 7f7300400700 1 --2- 172.21.15.153:0/1793052520 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72f8103f70 0x7f72f8198aa0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f72e4005b40 tx=0x7f72e400be20 comp rx=0 tx=0).stop 2024-03-21T10:04:14.280 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:14.261+0000 7f7300400700 1 -- 172.21.15.153:0/1793052520 shutdown_connections 2024-03-21T10:04:14.280 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:14.261+0000 7f7300400700 1 --2- 172.21.15.153:0/1793052520 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f72e006c4e0 0x7f72e006e9a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:04:14.280 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:14.261+0000 7f7300400700 1 --2- 172.21.15.153:0/1793052520 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f72f8101640 0x7f72f8198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:04:14.280 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:14.261+0000 7f7300400700 1 --2- 172.21.15.153:0/1793052520 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72f8103f70 0x7f72f8198aa0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:04:14.280 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:14.261+0000 7f7300400700 1 -- 172.21.15.153:0/1793052520 >> 172.21.15.153:0/1793052520 conn(0x7f72f80fb030 msgr2=0x7f72f80ffcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:04:14.280 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:14.261+0000 7f7300400700 1 -- 172.21.15.153:0/1793052520 shutdown_connections 2024-03-21T10:04:14.280 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:14.262+0000 7f7300400700 1 -- 172.21.15.153:0/1793052520 wait complete. 2024-03-21T10:04:15.134 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 070ed3e4-e769-11ee-95cd-87774f69a715 -- bash -c 'ceph orch host ls' 2024-03-21T10:04:15.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:15 smithi174 ceph-mon[37496]: from='client.14454 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:04:15.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:15 smithi174 ceph-mon[37496]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:04:15.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:15 smithi153 ceph-mon[29117]: from='client.14454 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:04:15.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:15 smithi153 ceph-mon[29117]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:04:15.556 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi153/config 2024-03-21T10:04:17.614 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:17.612+0000 7f1051400700 1 -- 172.21.15.153:0/1699001222 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f104c102df0 msgr2=0x7f104c103210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:04:17.614 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:17.612+0000 7f1051400700 1 --2- 172.21.15.153:0/1699001222 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f104c102df0 0x7f104c103210 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f103c009910 tx=0x7f103c009c20 comp rx=0 tx=0).stop 2024-03-21T10:04:17.614 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:17.612+0000 7f1051400700 1 -- 172.21.15.153:0/1699001222 shutdown_connections 2024-03-21T10:04:17.615 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:17.612+0000 7f1051400700 1 --2- 172.21.15.153:0/1699001222 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f104c103ff0 0x7f104c104450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:04:17.615 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:17.612+0000 7f1051400700 1 --2- 172.21.15.153:0/1699001222 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f104c102df0 0x7f104c103210 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:04:17.615 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:17.612+0000 7f1051400700 1 -- 172.21.15.153:0/1699001222 >> 172.21.15.153:0/1699001222 conn(0x7f104c0fe390 msgr2=0x7f104c1007d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:04:17.615 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:17.612+0000 7f1051400700 1 -- 172.21.15.153:0/1699001222 shutdown_connections 2024-03-21T10:04:17.615 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:17.612+0000 7f1051400700 1 -- 172.21.15.153:0/1699001222 wait complete. 2024-03-21T10:04:17.616 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:17.614+0000 7f1051400700 1 Processor -- start 2024-03-21T10:04:17.616 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:17.614+0000 7f1051400700 1 -- start start 2024-03-21T10:04:17.616 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:17.614+0000 7f1051400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f104c102df0 0x7f104c198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:04:17.616 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:17.614+0000 7f1051400700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f104c103ff0 0x7f104c198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:04:17.616 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:17.614+0000 7f1051400700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f104c1990e0 con 0x7f104c102df0 2024-03-21T10:04:17.616 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:17.614+0000 7f1051400700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f104c199220 con 0x7f104c103ff0 2024-03-21T10:04:17.616 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:17.614+0000 7f104a800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f104c102df0 0x7f104c198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:04:17.617 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:17.614+0000 7f104a800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f104c102df0 0x7f104c198580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:59702/0 (socket says 172.21.15.153:59702) 2024-03-21T10:04:17.617 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:17.614+0000 7f104a800700 1 -- 172.21.15.153:0/3204580789 learned_addr learned my addr 172.21.15.153:0/3204580789 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:04:17.617 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:17.615+0000 7f1049e00700 1 --2- 172.21.15.153:0/3204580789 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f104c103ff0 0x7f104c198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:04:17.617 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:17.615+0000 7f104a800700 1 -- 172.21.15.153:0/3204580789 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f104c103ff0 msgr2=0x7f104c198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:04:17.617 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:17.615+0000 7f104a800700 1 --2- 172.21.15.153:0/3204580789 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f104c103ff0 0x7f104c198ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:04:17.617 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:17.615+0000 7f104a800700 1 -- 172.21.15.153:0/3204580789 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f103c0095f0 con 0x7f104c102df0 2024-03-21T10:04:17.618 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:17.615+0000 7f104a800700 1 --2- 172.21.15.153:0/3204580789 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f104c102df0 0x7f104c198580 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f103c00bd70 tx=0x7f103c00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:04:17.618 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:17.615+0000 7f1043400700 1 -- 172.21.15.153:0/3204580789 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f103c01d070 con 0x7f104c102df0 2024-03-21T10:04:17.618 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:17.615+0000 7f1043400700 1 -- 172.21.15.153:0/3204580789 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f103c00f460 con 0x7f104c102df0 2024-03-21T10:04:17.618 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:17.616+0000 7f1051400700 1 -- 172.21.15.153:0/3204580789 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f104c19dc70 con 0x7f104c102df0 2024-03-21T10:04:17.618 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:17.616+0000 7f1043400700 1 -- 172.21.15.153:0/3204580789 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f103c021640 con 0x7f104c102df0 2024-03-21T10:04:17.618 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:17.616+0000 7f1051400700 1 -- 172.21.15.153:0/3204580789 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f104c19e160 con 0x7f104c102df0 2024-03-21T10:04:17.618 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:17.617+0000 7f1051400700 1 -- 172.21.15.153:0/3204580789 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f104c10b950 con 0x7f104c102df0 2024-03-21T10:04:17.619 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:17.618+0000 7f1043400700 1 -- 172.21.15.153:0/3204580789 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f103c00fab0 con 0x7f104c102df0 2024-03-21T10:04:17.620 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:17.619+0000 7f1043400700 1 --2- 172.21.15.153:0/3204580789 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f103006c4e0 0x7f103006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:04:17.620 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:17.619+0000 7f1043400700 1 -- 172.21.15.153:0/3204580789 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f103c08ca10 con 0x7f104c102df0 2024-03-21T10:04:17.621 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:17.619+0000 7f1049e00700 1 --2- 172.21.15.153:0/3204580789 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f103006c4e0 0x7f103006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:04:17.621 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:17.620+0000 7f1049e00700 1 --2- 172.21.15.153:0/3204580789 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f103006c4e0 0x7f103006e9a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1038005950 tx=0x7f1038005cf0 comp rx=0 tx=0).ready entity=mgr.14221 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:04:17.624 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:17.623+0000 7f1043400700 1 -- 172.21.15.153:0/3204580789 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f103c05af20 con 0x7f104c102df0 2024-03-21T10:04:17.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:17 smithi174 ceph-mon[37496]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:04:17.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:17 smithi153 ceph-mon[29117]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:04:17.844 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:17.843+0000 7f1051400700 1 -- 172.21.15.153:0/3204580789 --> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f104c108940 con 0x7f103006c4e0 2024-03-21T10:04:17.848 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:17.846+0000 7f1043400700 1 -- 172.21.15.153:0/3204580789 <== mgr.14221 v2:172.21.15.153:6800/1639929315 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+148 (secure 0 0 0) 0x7f104c108940 con 0x7f103006c4e0 2024-03-21T10:04:17.848 INFO:teuthology.orchestra.run.smithi153.stdout:HOST ADDR LABELS STATUS 2024-03-21T10:04:17.848 INFO:teuthology.orchestra.run.smithi153.stdout:smithi153 172.21.15.153 2024-03-21T10:04:17.848 INFO:teuthology.orchestra.run.smithi153.stdout:smithi174 172.21.15.174 2024-03-21T10:04:17.848 INFO:teuthology.orchestra.run.smithi153.stdout:2 hosts in cluster 2024-03-21T10:04:17.851 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:17.849+0000 7f1051400700 1 -- 172.21.15.153:0/3204580789 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f103006c4e0 msgr2=0x7f103006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:04:17.851 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:17.849+0000 7f1051400700 1 --2- 172.21.15.153:0/3204580789 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f103006c4e0 0x7f103006e9a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1038005950 tx=0x7f1038005cf0 comp rx=0 tx=0).stop 2024-03-21T10:04:17.851 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:17.849+0000 7f1051400700 1 -- 172.21.15.153:0/3204580789 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f104c102df0 msgr2=0x7f104c198580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:04:17.851 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:17.849+0000 7f1051400700 1 --2- 172.21.15.153:0/3204580789 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f104c102df0 0x7f104c198580 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f103c00bd70 tx=0x7f103c00be50 comp rx=0 tx=0).stop 2024-03-21T10:04:17.851 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:17.849+0000 7f1051400700 1 -- 172.21.15.153:0/3204580789 shutdown_connections 2024-03-21T10:04:17.851 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:17.849+0000 7f1051400700 1 --2- 172.21.15.153:0/3204580789 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f103006c4e0 0x7f103006e9a0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:04:17.851 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:17.849+0000 7f1051400700 1 --2- 172.21.15.153:0/3204580789 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f104c102df0 0x7f104c198580 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:04:17.851 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:17.849+0000 7f1051400700 1 --2- 172.21.15.153:0/3204580789 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f104c103ff0 0x7f104c198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:04:17.851 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:17.849+0000 7f1051400700 1 -- 172.21.15.153:0/3204580789 >> 172.21.15.153:0/3204580789 conn(0x7f104c0fe390 msgr2=0x7f104c107220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:04:17.851 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:17.849+0000 7f1051400700 1 -- 172.21.15.153:0/3204580789 shutdown_connections 2024-03-21T10:04:17.851 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:17.849+0000 7f1051400700 1 -- 172.21.15.153:0/3204580789 wait complete. 2024-03-21T10:04:18.663 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 070ed3e4-e769-11ee-95cd-87774f69a715 -- bash -c 'ceph orch device ls' 2024-03-21T10:04:19.082 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi153/config 2024-03-21T10:04:19.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:19 smithi153 ceph-mon[29117]: from='client.14458 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:04:19.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:19 smithi153 ceph-mon[29117]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:04:19.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:19 smithi174 ceph-mon[37496]: from='client.14458 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:04:19.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:19 smithi174 ceph-mon[37496]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:04:21.228 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:21.225+0000 7ff231000700 1 -- 172.21.15.153:0/269584214 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff22c107200 msgr2=0x7ff22c107620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:04:21.228 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:21.225+0000 7ff231000700 1 --2- 172.21.15.153:0/269584214 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff22c107200 0x7ff22c107620 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff21c009910 tx=0x7ff21c009c20 comp rx=0 tx=0).stop 2024-03-21T10:04:21.228 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:21.226+0000 7ff231000700 1 -- 172.21.15.153:0/269584214 shutdown_connections 2024-03-21T10:04:21.228 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:21.226+0000 7ff231000700 1 --2- 172.21.15.153:0/269584214 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff22c108400 0x7ff22c108860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:04:21.228 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:21.226+0000 7ff231000700 1 --2- 172.21.15.153:0/269584214 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff22c107200 0x7ff22c107620 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:04:21.228 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:21.226+0000 7ff231000700 1 -- 172.21.15.153:0/269584214 >> 172.21.15.153:0/269584214 conn(0x7ff22c0761b0 msgr2=0x7ff22c078610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:04:21.228 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:21.226+0000 7ff231000700 1 -- 172.21.15.153:0/269584214 shutdown_connections 2024-03-21T10:04:21.228 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:21.226+0000 7ff231000700 1 -- 172.21.15.153:0/269584214 wait complete. 2024-03-21T10:04:21.230 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:21.227+0000 7ff231000700 1 Processor -- start 2024-03-21T10:04:21.230 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:21.227+0000 7ff231000700 1 -- start start 2024-03-21T10:04:21.230 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:21.227+0000 7ff231000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff22c107200 0x7ff22c19c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:04:21.230 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:21.227+0000 7ff231000700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff22c108400 0x7ff22c19cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:04:21.230 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:21.227+0000 7ff231000700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff22c19d510 con 0x7ff22c107200 2024-03-21T10:04:21.230 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:21.227+0000 7ff231000700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff22c19d650 con 0x7ff22c108400 2024-03-21T10:04:21.230 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:21.227+0000 7ff229e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff22c107200 0x7ff22c19c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:04:21.230 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:21.227+0000 7ff229e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff22c107200 0x7ff22c19c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:59720/0 (socket says 172.21.15.153:59720) 2024-03-21T10:04:21.231 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:21.227+0000 7ff229e00700 1 -- 172.21.15.153:0/643579978 learned_addr learned my addr 172.21.15.153:0/643579978 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:04:21.231 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:21.228+0000 7ff223e00700 1 --2- 172.21.15.153:0/643579978 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff22c108400 0x7ff22c19cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:04:21.231 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:21.228+0000 7ff229e00700 1 -- 172.21.15.153:0/643579978 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff22c108400 msgr2=0x7ff22c19cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:04:21.231 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:21.228+0000 7ff229e00700 1 --2- 172.21.15.153:0/643579978 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff22c108400 0x7ff22c19cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:04:21.231 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:21.228+0000 7ff229e00700 1 -- 172.21.15.153:0/643579978 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff21c0095f0 con 0x7ff22c107200 2024-03-21T10:04:21.231 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:21.228+0000 7ff223e00700 1 --2- 172.21.15.153:0/643579978 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff22c108400 0x7ff22c19cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:04:21.231 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:21.228+0000 7ff229e00700 1 --2- 172.21.15.153:0/643579978 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff22c107200 0x7ff22c19c9b0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7ff21c00bd70 tx=0x7ff21c00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:04:21.231 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:21.228+0000 7ff222a00700 1 -- 172.21.15.153:0/643579978 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff21c01d070 con 0x7ff22c107200 2024-03-21T10:04:21.231 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:21.228+0000 7ff222a00700 1 -- 172.21.15.153:0/643579978 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff21c00f460 con 0x7ff22c107200 2024-03-21T10:04:21.231 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:21.228+0000 7ff231000700 1 -- 172.21.15.153:0/643579978 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff22c1a20a0 con 0x7ff22c107200 2024-03-21T10:04:21.231 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:21.228+0000 7ff222a00700 1 -- 172.21.15.153:0/643579978 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff21c021620 con 0x7ff22c107200 2024-03-21T10:04:21.232 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:21.229+0000 7ff231000700 1 -- 172.21.15.153:0/643579978 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff22c1a2590 con 0x7ff22c107200 2024-03-21T10:04:21.232 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:21.230+0000 7ff231000700 1 -- 172.21.15.153:0/643579978 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff22c10fd60 con 0x7ff22c107200 2024-03-21T10:04:21.232 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:21.231+0000 7ff222a00700 1 -- 172.21.15.153:0/643579978 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7ff21c00fa40 con 0x7ff22c107200 2024-03-21T10:04:21.234 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:21.232+0000 7ff222a00700 1 --2- 172.21.15.153:0/643579978 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7ff21006c4e0 0x7ff21006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:04:21.234 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:21.232+0000 7ff222a00700 1 -- 172.21.15.153:0/643579978 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7ff21c08c820 con 0x7ff22c107200 2024-03-21T10:04:21.234 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:21.232+0000 7ff223e00700 1 --2- 172.21.15.153:0/643579978 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7ff21006c4e0 0x7ff21006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:04:21.234 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:21.233+0000 7ff223e00700 1 --2- 172.21.15.153:0/643579978 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7ff21006c4e0 0x7ff21006e9a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff214005fd0 tx=0x7ff214005ee0 comp rx=0 tx=0).ready entity=mgr.14221 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:04:21.238 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:21.237+0000 7ff222a00700 1 -- 172.21.15.153:0/643579978 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7ff21c05adb0 con 0x7ff22c107200 2024-03-21T10:04:21.459 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:21.458+0000 7ff231000700 1 -- 172.21.15.153:0/643579978 --> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7ff22c10cd50 con 0x7ff21006c4e0 2024-03-21T10:04:21.461 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:21.460+0000 7ff222a00700 1 -- 172.21.15.153:0/643579978 <== mgr.14221 v2:172.21.15.153:6800/1639929315 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+399 (secure 0 0 0) 0x7ff22c10cd50 con 0x7ff21006c4e0 2024-03-21T10:04:21.461 INFO:teuthology.orchestra.run.smithi153.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-03-21T10:04:21.461 INFO:teuthology.orchestra.run.smithi153.stdout:smithi153 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300MV400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-03-21T10:04:21.461 INFO:teuthology.orchestra.run.smithi153.stdout:smithi174 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT62050074400BGN 372G No 81s ago Has a FileSystem, LVM detected 2024-03-21T10:04:21.464 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:21.462+0000 7ff231000700 1 -- 172.21.15.153:0/643579978 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7ff21006c4e0 msgr2=0x7ff21006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:04:21.464 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:21.462+0000 7ff231000700 1 --2- 172.21.15.153:0/643579978 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7ff21006c4e0 0x7ff21006e9a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff214005fd0 tx=0x7ff214005ee0 comp rx=0 tx=0).stop 2024-03-21T10:04:21.464 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:21.462+0000 7ff231000700 1 -- 172.21.15.153:0/643579978 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff22c107200 msgr2=0x7ff22c19c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:04:21.464 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:21.462+0000 7ff231000700 1 --2- 172.21.15.153:0/643579978 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff22c107200 0x7ff22c19c9b0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7ff21c00bd70 tx=0x7ff21c00be50 comp rx=0 tx=0).stop 2024-03-21T10:04:21.465 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:21.463+0000 7ff231000700 1 -- 172.21.15.153:0/643579978 shutdown_connections 2024-03-21T10:04:21.465 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:21.463+0000 7ff231000700 1 --2- 172.21.15.153:0/643579978 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7ff21006c4e0 0x7ff21006e9a0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:04:21.465 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:21.463+0000 7ff231000700 1 --2- 172.21.15.153:0/643579978 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff22c107200 0x7ff22c19c9b0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:04:21.465 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:21.463+0000 7ff231000700 1 --2- 172.21.15.153:0/643579978 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff22c108400 0x7ff22c19cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:04:21.465 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:21.463+0000 7ff231000700 1 -- 172.21.15.153:0/643579978 >> 172.21.15.153:0/643579978 conn(0x7ff22c0761b0 msgr2=0x7ff22c10b630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:04:21.465 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:21.463+0000 7ff231000700 1 -- 172.21.15.153:0/643579978 shutdown_connections 2024-03-21T10:04:21.465 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:21.463+0000 7ff231000700 1 -- 172.21.15.153:0/643579978 wait complete. 2024-03-21T10:04:21.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:21 smithi174 ceph-mon[37496]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:04:21.760 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:21 smithi153 ceph-mon[29117]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:04:22.348 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-21T10:04:22.358 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi153.front.sepia.ceph.com 2024-03-21T10:04:22.359 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 070ed3e4-e769-11ee-95cd-87774f69a715 -- bash -c 'ceph fs volume create cephfs --placement=4' 2024-03-21T10:04:22.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:22 smithi174 ceph-mon[37496]: from='client.14462 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:04:22.752 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi153/config 2024-03-21T10:04:22.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:22 smithi153 ceph-mon[29117]: from='client.14462 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:04:23.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:23 smithi174 ceph-mon[37496]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:04:23.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:23 smithi153 ceph-mon[29117]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:04:24.830 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:24.828+0000 7faea8000700 1 -- 172.21.15.153:0/629023874 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faea0074d40 msgr2=0x7faea00731a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:04:24.830 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:24.828+0000 7faea8000700 1 --2- 172.21.15.153:0/629023874 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faea0074d40 0x7faea00731a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fae94009910 tx=0x7fae94009c20 comp rx=0 tx=0).stop 2024-03-21T10:04:24.830 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:24.828+0000 7faea8000700 1 -- 172.21.15.153:0/629023874 shutdown_connections 2024-03-21T10:04:24.830 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:24.828+0000 7faea8000700 1 --2- 172.21.15.153:0/629023874 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faea0073770 0x7faea0073bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:04:24.830 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:24.828+0000 7faea8000700 1 --2- 172.21.15.153:0/629023874 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faea0074d40 0x7faea00731a0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:04:24.830 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:24.828+0000 7faea8000700 1 -- 172.21.15.153:0/629023874 >> 172.21.15.153:0/629023874 conn(0x7faea00fc110 msgr2=0x7faea00fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:04:24.830 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:24.828+0000 7faea8000700 1 -- 172.21.15.153:0/629023874 shutdown_connections 2024-03-21T10:04:24.831 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:24.828+0000 7faea8000700 1 -- 172.21.15.153:0/629023874 wait complete. 2024-03-21T10:04:24.832 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:24.829+0000 7faea8000700 1 Processor -- start 2024-03-21T10:04:24.832 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:24.829+0000 7faea8000700 1 -- start start 2024-03-21T10:04:24.832 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:24.830+0000 7faea8000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faea0073770 0x7faea0198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:04:24.832 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:24.830+0000 7faea8000700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faea0074d40 0x7faea0198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:04:24.832 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:24.830+0000 7faea8000700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faea01990d0 con 0x7faea0073770 2024-03-21T10:04:24.832 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:24.830+0000 7faea8000700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faea0199210 con 0x7faea0074d40 2024-03-21T10:04:24.832 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:24.830+0000 7faea5600700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faea0073770 0x7faea0198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:04:24.832 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:24.830+0000 7faea5600700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faea0073770 0x7faea0198570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:35872/0 (socket says 172.21.15.153:35872) 2024-03-21T10:04:24.833 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:24.830+0000 7faea5600700 1 -- 172.21.15.153:0/3332349052 learned_addr learned my addr 172.21.15.153:0/3332349052 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:04:24.833 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:24.830+0000 7faea4c00700 1 --2- 172.21.15.153:0/3332349052 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faea0074d40 0x7faea0198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:04:24.833 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:24.830+0000 7faea5600700 1 -- 172.21.15.153:0/3332349052 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faea0074d40 msgr2=0x7faea0198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:04:24.833 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:24.830+0000 7faea5600700 1 --2- 172.21.15.153:0/3332349052 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faea0074d40 0x7faea0198ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:04:24.833 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:24.830+0000 7faea5600700 1 -- 172.21.15.153:0/3332349052 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae940095f0 con 0x7faea0073770 2024-03-21T10:04:24.833 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:24.831+0000 7faea4c00700 1 --2- 172.21.15.153:0/3332349052 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faea0074d40 0x7faea0198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:04:24.833 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:24.831+0000 7faea5600700 1 --2- 172.21.15.153:0/3332349052 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faea0073770 0x7faea0198570 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fae94005b40 tx=0x7fae9400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:04:24.834 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:24.831+0000 7fae9e000700 1 -- 172.21.15.153:0/3332349052 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fae9401d070 con 0x7faea0073770 2024-03-21T10:04:24.834 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:24.831+0000 7fae9e000700 1 -- 172.21.15.153:0/3332349052 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fae94022470 con 0x7faea0073770 2024-03-21T10:04:24.834 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:24.831+0000 7fae9e000700 1 -- 172.21.15.153:0/3332349052 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fae9400f690 con 0x7faea0073770 2024-03-21T10:04:24.834 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:24.831+0000 7faea8000700 1 -- 172.21.15.153:0/3332349052 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faea019dc60 con 0x7faea0073770 2024-03-21T10:04:24.834 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:24.831+0000 7faea8000700 1 -- 172.21.15.153:0/3332349052 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faea019e150 con 0x7faea0073770 2024-03-21T10:04:24.834 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:24.832+0000 7faea8000700 1 -- 172.21.15.153:0/3332349052 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faea010b920 con 0x7faea0073770 2024-03-21T10:04:24.834 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:24.833+0000 7fae9e000700 1 -- 172.21.15.153:0/3332349052 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fae940225e0 con 0x7faea0073770 2024-03-21T10:04:24.836 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:24.835+0000 7fae9e000700 1 --2- 172.21.15.153:0/3332349052 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fae8806c4e0 0x7fae8806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:04:24.836 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:24.835+0000 7fae9e000700 1 -- 172.21.15.153:0/3332349052 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fae9408cd40 con 0x7faea0073770 2024-03-21T10:04:24.836 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:24.835+0000 7faea4c00700 1 --2- 172.21.15.153:0/3332349052 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fae8806c4e0 0x7fae8806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:04:24.837 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:24.836+0000 7faea4c00700 1 --2- 172.21.15.153:0/3332349052 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fae8806c4e0 0x7fae8806e9a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fae8c005fd0 tx=0x7fae8c005dc0 comp rx=0 tx=0).ready entity=mgr.14221 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:04:24.841 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:24.840+0000 7fae9e000700 1 -- 172.21.15.153:0/3332349052 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fae9405b350 con 0x7faea0073770 2024-03-21T10:04:25.087 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:25.085+0000 7faea8000700 1 -- 172.21.15.153:0/3332349052 --> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] -- mgr_command(tid 0: {"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7faea0108130 con 0x7fae8806c4e0 2024-03-21T10:04:25.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:25 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:04:25.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:25 smithi153 ceph-mon[29117]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:04:25.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:25 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:04:25.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:25 smithi174 ceph-mon[37496]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:04:26.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:26 smithi153 ceph-mon[29117]: from='client.14466 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:04:26.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:26 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-03-21T10:04:26.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:26 smithi174 ceph-mon[37496]: from='client.14466 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:04:26.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:26 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-03-21T10:04:27.121 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:27.119+0000 7fae9e000700 1 -- 172.21.15.153:0/3332349052 <== mgr.14221 v2:172.21.15.153:6800/1639929315 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7faea0108130 con 0x7fae8806c4e0 2024-03-21T10:04:27.124 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:27.122+0000 7faea8000700 1 -- 172.21.15.153:0/3332349052 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fae8806c4e0 msgr2=0x7fae8806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:04:27.125 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:27.122+0000 7faea8000700 1 --2- 172.21.15.153:0/3332349052 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fae8806c4e0 0x7fae8806e9a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fae8c005fd0 tx=0x7fae8c005dc0 comp rx=0 tx=0).stop 2024-03-21T10:04:27.125 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:27.123+0000 7faea8000700 1 -- 172.21.15.153:0/3332349052 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faea0073770 msgr2=0x7faea0198570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:04:27.125 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:27.123+0000 7faea8000700 1 --2- 172.21.15.153:0/3332349052 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faea0073770 0x7faea0198570 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fae94005b40 tx=0x7fae9400be20 comp rx=0 tx=0).stop 2024-03-21T10:04:27.125 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:27.123+0000 7faea8000700 1 -- 172.21.15.153:0/3332349052 shutdown_connections 2024-03-21T10:04:27.125 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:27.123+0000 7faea8000700 1 --2- 172.21.15.153:0/3332349052 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fae8806c4e0 0x7fae8806e9a0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:04:27.125 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:27.123+0000 7faea8000700 1 --2- 172.21.15.153:0/3332349052 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faea0073770 0x7faea0198570 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:04:27.125 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:27.123+0000 7faea8000700 1 --2- 172.21.15.153:0/3332349052 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faea0074d40 0x7faea0198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:04:27.125 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:27.123+0000 7faea8000700 1 -- 172.21.15.153:0/3332349052 >> 172.21.15.153:0/3332349052 conn(0x7faea00fc110 msgr2=0x7faea0106a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:04:27.125 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:27.123+0000 7faea8000700 1 -- 172.21.15.153:0/3332349052 shutdown_connections 2024-03-21T10:04:27.125 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:27.123+0000 7faea8000700 1 -- 172.21.15.153:0/3332349052 wait complete. 2024-03-21T10:04:27.353 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:27 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-03-21T10:04:27.354 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:27 smithi174 ceph-mon[37496]: osdmap e42: 6 total, 6 up, 6 in 2024-03-21T10:04:27.354 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:27 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-03-21T10:04:27.354 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:27 smithi174 ceph-mon[37496]: pgmap v211: 33 pgs: 1 active+clean, 32 unknown; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:04:27.525 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:27 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-03-21T10:04:27.526 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:27 smithi153 ceph-mon[29117]: osdmap e42: 6 total, 6 up, 6 in 2024-03-21T10:04:27.526 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:27 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-03-21T10:04:27.526 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:27 smithi153 ceph-mon[29117]: pgmap v211: 33 pgs: 1 active+clean, 32 unknown; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:04:27.526 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:27 smithi153 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-mon-smithi153[29113]: 2024-03-21T10:04:27.085+0000 7fc705200700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-21T10:04:28.007 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 070ed3e4-e769-11ee-95cd-87774f69a715 -- bash -c 'ceph fs dump' 2024-03-21T10:04:28.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:28 smithi153 ceph-mon[29117]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T10:04:28.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:28 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd='[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-03-21T10:04:28.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:28 smithi153 ceph-mon[29117]: osdmap e43: 6 total, 6 up, 6 in 2024-03-21T10:04:28.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:28 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-03-21T10:04:28.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:28 smithi153 ceph-mon[29117]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-21T10:04:28.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:28 smithi153 ceph-mon[29117]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-21T10:04:28.071 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:28 smithi153 ceph-mon[29117]: osdmap e44: 6 total, 6 up, 6 in 2024-03-21T10:04:28.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:28 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-03-21T10:04:28.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:28 smithi153 ceph-mon[29117]: fsmap cephfs:0 2024-03-21T10:04:28.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:28 smithi153 ceph-mon[29117]: Saving service mds.cephfs spec with placement count:4 2024-03-21T10:04:28.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:28 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:04:28.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:28 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:04:28.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:28 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:04:28.072 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:28 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:04:28.328 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:28 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:04:28.328 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:28 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi153.zipckr", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T10:04:28.328 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:28 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi153.zipckr", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T10:04:28.328 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:28 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:04:28.329 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:28 smithi153 ceph-mon[29117]: Deploying daemon mds.cephfs.smithi153.zipckr on smithi153 2024-03-21T10:04:28.383 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:28 smithi174 ceph-mon[37496]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T10:04:28.385 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:28 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd='[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-03-21T10:04:28.385 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:28 smithi174 ceph-mon[37496]: osdmap e43: 6 total, 6 up, 6 in 2024-03-21T10:04:28.385 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:28 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-03-21T10:04:28.385 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:28 smithi174 ceph-mon[37496]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-21T10:04:28.385 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:28 smithi174 ceph-mon[37496]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-21T10:04:28.385 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:28 smithi174 ceph-mon[37496]: osdmap e44: 6 total, 6 up, 6 in 2024-03-21T10:04:28.385 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:28 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-03-21T10:04:28.385 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:28 smithi174 ceph-mon[37496]: fsmap cephfs:0 2024-03-21T10:04:28.385 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:28 smithi174 ceph-mon[37496]: Saving service mds.cephfs spec with placement count:4 2024-03-21T10:04:28.385 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:28 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:04:28.386 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:28 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:04:28.386 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:28 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:04:28.386 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:28 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:04:28.386 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:28 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:04:28.386 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:28 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi153.zipckr", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T10:04:28.386 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:28 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi153.zipckr", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T10:04:28.386 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:28 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:04:28.386 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:28 smithi174 ceph-mon[37496]: Deploying daemon mds.cephfs.smithi153.zipckr on smithi153 2024-03-21T10:04:29.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:29 smithi174 ceph-mon[37496]: osdmap e45: 6 total, 6 up, 6 in 2024-03-21T10:04:29.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:29 smithi174 ceph-mon[37496]: pgmap v215: 65 pgs: 15 creating+peering, 49 unknown, 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:04:29.496 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:29 smithi153 ceph-mon[29117]: osdmap e45: 6 total, 6 up, 6 in 2024-03-21T10:04:29.496 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:29 smithi153 ceph-mon[29117]: pgmap v215: 65 pgs: 15 creating+peering, 49 unknown, 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:04:29.630 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi153/config 2024-03-21T10:04:30.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:30 smithi174 ceph-mon[37496]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-21T10:04:30.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:30 smithi174 ceph-mon[37496]: osdmap e46: 6 total, 6 up, 6 in 2024-03-21T10:04:30.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:30 smithi153 ceph-mon[29117]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-21T10:04:30.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:30 smithi153 ceph-mon[29117]: osdmap e46: 6 total, 6 up, 6 in 2024-03-21T10:04:31.407 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:31 smithi153 ceph-mon[29117]: pgmap v217: 65 pgs: 15 creating+peering, 43 unknown, 7 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:04:31.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:31 smithi174 ceph-mon[37496]: pgmap v217: 65 pgs: 15 creating+peering, 43 unknown, 7 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:04:32.232 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:32.230+0000 7f74b1400700 1 -- 172.21.15.153:0/3230198528 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f74ac0ff430 msgr2=0x7f74ac0ff850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:04:32.232 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:32.230+0000 7f74b1400700 1 --2- 172.21.15.153:0/3230198528 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f74ac0ff430 0x7f74ac0ff850 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f749c009910 tx=0x7f749c009c20 comp rx=0 tx=0).stop 2024-03-21T10:04:32.232 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:32.230+0000 7f74b1400700 1 -- 172.21.15.153:0/3230198528 shutdown_connections 2024-03-21T10:04:32.232 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:32.230+0000 7f74b1400700 1 --2- 172.21.15.153:0/3230198528 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f74ac0ffd90 0x7f74ac100210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:04:32.233 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:32.230+0000 7f74b1400700 1 --2- 172.21.15.153:0/3230198528 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f74ac0ff430 0x7f74ac0ff850 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:04:32.233 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:32.230+0000 7f74b1400700 1 -- 172.21.15.153:0/3230198528 >> 172.21.15.153:0/3230198528 conn(0x7f74ac0fb030 msgr2=0x7f74ac0fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:04:32.233 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:32.230+0000 7f74b1400700 1 -- 172.21.15.153:0/3230198528 shutdown_connections 2024-03-21T10:04:32.233 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:32.230+0000 7f74b1400700 1 -- 172.21.15.153:0/3230198528 wait complete. 2024-03-21T10:04:32.233 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:32.232+0000 7f74b1400700 1 Processor -- start 2024-03-21T10:04:32.233 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:32.232+0000 7f74b1400700 1 -- start start 2024-03-21T10:04:32.234 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:32.232+0000 7f74b1400700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f74ac0ff430 0x7f74ac1985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:04:32.234 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:32.232+0000 7f74b1400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f74ac0ffd90 0x7f74ac198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:04:32.234 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:32.232+0000 7f74b1400700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74ac199110 con 0x7f74ac0ffd90 2024-03-21T10:04:32.234 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:32.232+0000 7f74b1400700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74ac199250 con 0x7f74ac0ff430 2024-03-21T10:04:32.234 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:32.233+0000 7f74a3e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f74ac0ffd90 0x7f74ac198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:04:32.234 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:32.233+0000 7f74a3e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f74ac0ffd90 0x7f74ac198af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:35912/0 (socket says 172.21.15.153:35912) 2024-03-21T10:04:32.235 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:32.233+0000 7f74a3e00700 1 -- 172.21.15.153:0/3129128135 learned_addr learned my addr 172.21.15.153:0/3129128135 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:04:32.235 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:32.233+0000 7f74aa800700 1 --2- 172.21.15.153:0/3129128135 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f74ac0ff430 0x7f74ac1985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:04:32.235 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:32.233+0000 7f74a3e00700 1 -- 172.21.15.153:0/3129128135 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f74ac0ff430 msgr2=0x7f74ac1985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:04:32.236 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:32.233+0000 7f74a3e00700 1 --2- 172.21.15.153:0/3129128135 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f74ac0ff430 0x7f74ac1985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:04:32.236 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:32.233+0000 7f74a3e00700 1 -- 172.21.15.153:0/3129128135 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f749c0095f0 con 0x7f74ac0ffd90 2024-03-21T10:04:32.236 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:32.234+0000 7f74a3e00700 1 --2- 172.21.15.153:0/3129128135 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f74ac0ffd90 0x7f74ac198af0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f749400b4b0 tx=0x7f749400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:04:32.236 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:32.234+0000 7f74a3400700 1 -- 172.21.15.153:0/3129128135 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7494011800 con 0x7f74ac0ffd90 2024-03-21T10:04:32.236 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:32.234+0000 7f74a3400700 1 -- 172.21.15.153:0/3129128135 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f7494011e10 con 0x7f74ac0ffd90 2024-03-21T10:04:32.236 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:32.234+0000 7f74b1400700 1 -- 172.21.15.153:0/3129128135 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74ac19dd00 con 0x7f74ac0ffd90 2024-03-21T10:04:32.236 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:32.234+0000 7f74a3400700 1 -- 172.21.15.153:0/3129128135 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f749400f2e0 con 0x7f74ac0ffd90 2024-03-21T10:04:32.236 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:32.234+0000 7f74b1400700 1 -- 172.21.15.153:0/3129128135 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74ac19e2d0 con 0x7f74ac0ffd90 2024-03-21T10:04:32.236 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:32.235+0000 7f74b1400700 1 -- 172.21.15.153:0/3129128135 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74ac10b980 con 0x7f74ac0ffd90 2024-03-21T10:04:32.237 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:32.236+0000 7f74a3400700 1 -- 172.21.15.153:0/3129128135 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f7494010380 con 0x7f74ac0ffd90 2024-03-21T10:04:32.238 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:32.237+0000 7f74a3400700 1 --2- 172.21.15.153:0/3129128135 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f749006c4e0 0x7f749006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:04:32.239 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:32.237+0000 7f74a3400700 1 -- 172.21.15.153:0/3129128135 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f749408b440 con 0x7f74ac0ffd90 2024-03-21T10:04:32.239 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:32.237+0000 7f74aa800700 1 --2- 172.21.15.153:0/3129128135 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f749006c4e0 0x7f749006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:04:32.239 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:32.238+0000 7f74aa800700 1 --2- 172.21.15.153:0/3129128135 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f749006c4e0 0x7f749006e9a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f749c0059d0 tx=0x7f749c005920 comp rx=0 tx=0).ready entity=mgr.14221 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:04:32.244 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:32.242+0000 7f74a3400700 1 -- 172.21.15.153:0/3129128135 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f7494059690 con 0x7f74ac0ffd90 2024-03-21T10:04:32.505 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:32.503+0000 7f74b1400700 1 -- 172.21.15.153:0/3129128135 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f74ac10bb70 con 0x7f74ac0ffd90 2024-03-21T10:04:32.505 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:32.504+0000 7f74a3400700 1 -- 172.21.15.153:0/3129128135 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 4 v4) v1 ==== 75+0+1266 (secure 0 0 0) 0x7f7494059220 con 0x7f74ac0ffd90 2024-03-21T10:04:32.506 INFO:teuthology.orchestra.run.smithi153.stdout:e4 2024-03-21T10:04:32.506 INFO:teuthology.orchestra.run.smithi153.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-21T10:04:32.506 INFO:teuthology.orchestra.run.smithi153.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:04:32.506 INFO:teuthology.orchestra.run.smithi153.stdout:legacy client fscid: 1 2024-03-21T10:04:32.507 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:04:32.507 INFO:teuthology.orchestra.run.smithi153.stdout:Filesystem 'cephfs' (1) 2024-03-21T10:04:32.507 INFO:teuthology.orchestra.run.smithi153.stdout:fs_name cephfs 2024-03-21T10:04:32.507 INFO:teuthology.orchestra.run.smithi153.stdout:epoch 4 2024-03-21T10:04:32.507 INFO:teuthology.orchestra.run.smithi153.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-21T10:04:32.507 INFO:teuthology.orchestra.run.smithi153.stdout:created 2024-03-21T10:04:27.086576+0000 2024-03-21T10:04:32.507 INFO:teuthology.orchestra.run.smithi153.stdout:modified 2024-03-21T10:04:32.131404+0000 2024-03-21T10:04:32.507 INFO:teuthology.orchestra.run.smithi153.stdout:tableserver 0 2024-03-21T10:04:32.507 INFO:teuthology.orchestra.run.smithi153.stdout:root 0 2024-03-21T10:04:32.507 INFO:teuthology.orchestra.run.smithi153.stdout:session_timeout 60 2024-03-21T10:04:32.507 INFO:teuthology.orchestra.run.smithi153.stdout:session_autoclose 300 2024-03-21T10:04:32.507 INFO:teuthology.orchestra.run.smithi153.stdout:max_file_size 1099511627776 2024-03-21T10:04:32.507 INFO:teuthology.orchestra.run.smithi153.stdout:required_client_features {} 2024-03-21T10:04:32.507 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure 0 2024-03-21T10:04:32.508 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure_osd_epoch 0 2024-03-21T10:04:32.508 INFO:teuthology.orchestra.run.smithi153.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:04:32.508 INFO:teuthology.orchestra.run.smithi153.stdout:max_mds 1 2024-03-21T10:04:32.508 INFO:teuthology.orchestra.run.smithi153.stdout:in 0 2024-03-21T10:04:32.508 INFO:teuthology.orchestra.run.smithi153.stdout:up {0=14470} 2024-03-21T10:04:32.508 INFO:teuthology.orchestra.run.smithi153.stdout:failed 2024-03-21T10:04:32.508 INFO:teuthology.orchestra.run.smithi153.stdout:damaged 2024-03-21T10:04:32.508 INFO:teuthology.orchestra.run.smithi153.stdout:stopped 2024-03-21T10:04:32.508 INFO:teuthology.orchestra.run.smithi153.stdout:data_pools [3] 2024-03-21T10:04:32.508 INFO:teuthology.orchestra.run.smithi153.stdout:metadata_pool 2 2024-03-21T10:04:32.508 INFO:teuthology.orchestra.run.smithi153.stdout:inline_data disabled 2024-03-21T10:04:32.508 INFO:teuthology.orchestra.run.smithi153.stdout:balancer 2024-03-21T10:04:32.508 INFO:teuthology.orchestra.run.smithi153.stdout:bal_rank_mask -1 2024-03-21T10:04:32.508 INFO:teuthology.orchestra.run.smithi153.stdout:standby_count_wanted 0 2024-03-21T10:04:32.509 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.zipckr{0:14470} state up:creating seq 1 addr [v2:172.21.15.153:6826/1353876997,v1:172.21.15.153:6827/1353876997] compat {c=[1],r=[1],i=[7ff]}] 2024-03-21T10:04:32.509 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:04:32.509 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:04:32.510 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:32.507+0000 7f74b1400700 1 -- 172.21.15.153:0/3129128135 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f749006c4e0 msgr2=0x7f749006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:04:32.510 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:32.507+0000 7f74b1400700 1 --2- 172.21.15.153:0/3129128135 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f749006c4e0 0x7f749006e9a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f749c0059d0 tx=0x7f749c005920 comp rx=0 tx=0).stop 2024-03-21T10:04:32.510 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:32.507+0000 7f74b1400700 1 -- 172.21.15.153:0/3129128135 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f74ac0ffd90 msgr2=0x7f74ac198af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:04:32.510 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:32.507+0000 7f74b1400700 1 --2- 172.21.15.153:0/3129128135 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f74ac0ffd90 0x7f74ac198af0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f749400b4b0 tx=0x7f749400b870 comp rx=0 tx=0).stop 2024-03-21T10:04:32.510 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:32.507+0000 7f74b1400700 1 -- 172.21.15.153:0/3129128135 shutdown_connections 2024-03-21T10:04:32.510 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:32.507+0000 7f74b1400700 1 --2- 172.21.15.153:0/3129128135 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f749006c4e0 0x7f749006e9a0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:04:32.510 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:32.507+0000 7f74b1400700 1 --2- 172.21.15.153:0/3129128135 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f74ac0ff430 0x7f74ac1985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:04:32.510 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:32.507+0000 7f74b1400700 1 --2- 172.21.15.153:0/3129128135 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f74ac0ffd90 0x7f74ac198af0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:04:32.510 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:32.507+0000 7f74b1400700 1 -- 172.21.15.153:0/3129128135 >> 172.21.15.153:0/3129128135 conn(0x7f74ac0fb030 msgr2=0x7f74ac107a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:04:32.510 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:32.507+0000 7f74b1400700 1 -- 172.21.15.153:0/3129128135 shutdown_connections 2024-03-21T10:04:32.510 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:32.508+0000 7f74b1400700 1 -- 172.21.15.153:0/3129128135 wait complete. 2024-03-21T10:04:32.511 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 4 2024-03-21T10:04:33.070 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:32 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:04:33.070 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:32 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:04:33.071 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:32 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:04:33.071 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:32 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi174.yyimst", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T10:04:33.071 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:32 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi174.yyimst", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T10:04:33.071 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:32 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:04:33.071 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:32 smithi174 ceph-mon[37496]: Deploying daemon mds.cephfs.smithi174.yyimst on smithi174 2024-03-21T10:04:33.071 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:32 smithi174 ceph-mon[37496]: mds.? [v2:172.21.15.153:6826/1353876997,v1:172.21.15.153:6827/1353876997] up:boot 2024-03-21T10:04:33.071 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:32 smithi174 ceph-mon[37496]: daemon mds.cephfs.smithi153.zipckr assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-03-21T10:04:33.071 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:32 smithi174 ceph-mon[37496]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-21T10:04:33.071 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:32 smithi174 ceph-mon[37496]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-21T10:04:33.071 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:32 smithi174 ceph-mon[37496]: Cluster is now healthy 2024-03-21T10:04:33.071 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:32 smithi174 ceph-mon[37496]: fsmap cephfs:0 1 up:standby 2024-03-21T10:04:33.071 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:32 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi153.zipckr"}]: dispatch 2024-03-21T10:04:33.071 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:32 smithi174 ceph-mon[37496]: fsmap cephfs:1 {0=cephfs.smithi153.zipckr=up:creating} 2024-03-21T10:04:33.071 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:32 smithi174 ceph-mon[37496]: daemon mds.cephfs.smithi153.zipckr is now active in filesystem cephfs as rank 0 2024-03-21T10:04:33.071 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:32 smithi174 ceph-mon[37496]: pgmap v218: 65 pgs: 65 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:04:33.072 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:32 smithi174 ceph-mon[37496]: from='client.? 172.21.15.153:0/3129128135' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:04:33.155 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:32 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:04:33.155 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:32 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:04:33.155 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:32 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:04:33.155 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:32 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi174.yyimst", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T10:04:33.155 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:32 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi174.yyimst", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T10:04:33.155 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:32 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:04:33.155 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:32 smithi153 ceph-mon[29117]: Deploying daemon mds.cephfs.smithi174.yyimst on smithi174 2024-03-21T10:04:33.155 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:32 smithi153 ceph-mon[29117]: mds.? [v2:172.21.15.153:6826/1353876997,v1:172.21.15.153:6827/1353876997] up:boot 2024-03-21T10:04:33.155 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:32 smithi153 ceph-mon[29117]: daemon mds.cephfs.smithi153.zipckr assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-03-21T10:04:33.155 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:32 smithi153 ceph-mon[29117]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-21T10:04:33.155 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:32 smithi153 ceph-mon[29117]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-21T10:04:33.156 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:32 smithi153 ceph-mon[29117]: Cluster is now healthy 2024-03-21T10:04:33.156 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:32 smithi153 ceph-mon[29117]: fsmap cephfs:0 1 up:standby 2024-03-21T10:04:33.156 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:32 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi153.zipckr"}]: dispatch 2024-03-21T10:04:33.156 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:32 smithi153 ceph-mon[29117]: fsmap cephfs:1 {0=cephfs.smithi153.zipckr=up:creating} 2024-03-21T10:04:33.156 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:32 smithi153 ceph-mon[29117]: daemon mds.cephfs.smithi153.zipckr is now active in filesystem cephfs as rank 0 2024-03-21T10:04:33.156 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:32 smithi153 ceph-mon[29117]: pgmap v218: 65 pgs: 65 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:04:33.156 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:32 smithi153 ceph-mon[29117]: from='client.? 172.21.15.153:0/3129128135' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:04:33.212 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-21T10:04:33.221 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi153.front.sepia.ceph.com 2024-03-21T10:04:33.221 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 070ed3e4-e769-11ee-95cd-87774f69a715 -- bash -c 'ceph fs set cephfs max_mds 2' 2024-03-21T10:04:33.630 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi153/config 2024-03-21T10:04:34.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:34 smithi174 ceph-mon[37496]: mds.? [v2:172.21.15.153:6826/1353876997,v1:172.21.15.153:6827/1353876997] up:active 2024-03-21T10:04:34.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:34 smithi174 ceph-mon[37496]: fsmap cephfs:1 {0=cephfs.smithi153.zipckr=up:active} 2024-03-21T10:04:34.542 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:34 smithi153 ceph-mon[29117]: mds.? [v2:172.21.15.153:6826/1353876997,v1:172.21.15.153:6827/1353876997] up:active 2024-03-21T10:04:34.542 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:34 smithi153 ceph-mon[29117]: fsmap cephfs:1 {0=cephfs.smithi153.zipckr=up:active} 2024-03-21T10:04:35.352 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:35 smithi174 ceph-mon[37496]: pgmap v219: 65 pgs: 65 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 141 B/s wr, 0 op/s 2024-03-21T10:04:35.483 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:35 smithi153 ceph-mon[29117]: pgmap v219: 65 pgs: 65 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 141 B/s wr, 0 op/s 2024-03-21T10:04:35.659 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:35.657+0000 7fc3d7c00700 1 -- 172.21.15.153:0/1598328449 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc3d0074d40 msgr2=0x7fc3d00731a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:04:35.659 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:35.657+0000 7fc3d7c00700 1 --2- 172.21.15.153:0/1598328449 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc3d0074d40 0x7fc3d00731a0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc3c4009910 tx=0x7fc3c4009c20 comp rx=0 tx=0).stop 2024-03-21T10:04:35.659 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:35.657+0000 7fc3d7c00700 1 -- 172.21.15.153:0/1598328449 shutdown_connections 2024-03-21T10:04:35.659 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:35.657+0000 7fc3d7c00700 1 --2- 172.21.15.153:0/1598328449 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc3d0073770 0x7fc3d0073bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:04:35.659 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:35.657+0000 7fc3d7c00700 1 --2- 172.21.15.153:0/1598328449 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc3d0074d40 0x7fc3d00731a0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:04:35.659 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:35.657+0000 7fc3d7c00700 1 -- 172.21.15.153:0/1598328449 >> 172.21.15.153:0/1598328449 conn(0x7fc3d00fc190 msgr2=0x7fc3d00fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:04:35.659 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:35.657+0000 7fc3d7c00700 1 -- 172.21.15.153:0/1598328449 shutdown_connections 2024-03-21T10:04:35.659 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:35.657+0000 7fc3d7c00700 1 -- 172.21.15.153:0/1598328449 wait complete. 2024-03-21T10:04:35.660 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:35.658+0000 7fc3d7c00700 1 Processor -- start 2024-03-21T10:04:35.660 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:35.659+0000 7fc3d7c00700 1 -- start start 2024-03-21T10:04:35.661 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:35.659+0000 7fc3d7c00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc3d0073770 0x7fc3d0198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:04:35.661 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:35.659+0000 7fc3d7c00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc3d0074d40 0x7fc3d0198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:04:35.661 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:35.659+0000 7fc3d7c00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d01990b0 con 0x7fc3d0073770 2024-03-21T10:04:35.661 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:35.659+0000 7fc3d7c00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d01991f0 con 0x7fc3d0074d40 2024-03-21T10:04:35.661 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:35.659+0000 7fc3d5200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc3d0073770 0x7fc3d0198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:04:35.661 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:35.659+0000 7fc3d5200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc3d0073770 0x7fc3d0198550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:57634/0 (socket says 172.21.15.153:57634) 2024-03-21T10:04:35.661 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:35.659+0000 7fc3d5200700 1 -- 172.21.15.153:0/3848861681 learned_addr learned my addr 172.21.15.153:0/3848861681 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:04:35.662 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:35.659+0000 7fc3cfe00700 1 --2- 172.21.15.153:0/3848861681 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc3d0074d40 0x7fc3d0198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:04:35.662 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:35.660+0000 7fc3d5200700 1 -- 172.21.15.153:0/3848861681 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc3d0074d40 msgr2=0x7fc3d0198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:04:35.662 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:35.660+0000 7fc3d5200700 1 --2- 172.21.15.153:0/3848861681 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc3d0074d40 0x7fc3d0198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:04:35.662 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:35.660+0000 7fc3d5200700 1 -- 172.21.15.153:0/3848861681 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3c40095f0 con 0x7fc3d0073770 2024-03-21T10:04:35.662 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:35.660+0000 7fc3d5200700 1 --2- 172.21.15.153:0/3848861681 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc3d0073770 0x7fc3d0198550 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc3c400b5c0 tx=0x7fc3c400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:04:35.662 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:35.660+0000 7fc3cd600700 1 -- 172.21.15.153:0/3848861681 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc3c401d070 con 0x7fc3d0073770 2024-03-21T10:04:35.662 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:35.660+0000 7fc3cd600700 1 -- 172.21.15.153:0/3848861681 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc3c400f460 con 0x7fc3d0073770 2024-03-21T10:04:35.662 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:35.660+0000 7fc3d7c00700 1 -- 172.21.15.153:0/3848861681 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3d019dc40 con 0x7fc3d0073770 2024-03-21T10:04:35.662 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:35.660+0000 7fc3cd600700 1 -- 172.21.15.153:0/3848861681 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc3c4021640 con 0x7fc3d0073770 2024-03-21T10:04:35.663 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:35.661+0000 7fc3d7c00700 1 -- 172.21.15.153:0/3848861681 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3d019e130 con 0x7fc3d0073770 2024-03-21T10:04:35.663 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:35.662+0000 7fc3d7c00700 1 -- 172.21.15.153:0/3848861681 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3d010b920 con 0x7fc3d0073770 2024-03-21T10:04:35.664 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:35.663+0000 7fc3cd600700 1 -- 172.21.15.153:0/3848861681 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fc3c400f5d0 con 0x7fc3d0073770 2024-03-21T10:04:35.665 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:35.664+0000 7fc3cd600700 1 --2- 172.21.15.153:0/3848861681 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fc3b806c4e0 0x7fc3b806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:04:35.665 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:35.664+0000 7fc3cd600700 1 -- 172.21.15.153:0/3848861681 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5468+0+0 (secure 0 0 0) 0x7fc3c408cc90 con 0x7fc3d0073770 2024-03-21T10:04:35.665 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:35.664+0000 7fc3cfe00700 1 --2- 172.21.15.153:0/3848861681 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fc3b806c4e0 0x7fc3b806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:04:35.666 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:35.665+0000 7fc3cfe00700 1 --2- 172.21.15.153:0/3848861681 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fc3b806c4e0 0x7fc3b806e9a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc005950 tx=0x7fc3bc0058e0 comp rx=0 tx=0).ready entity=mgr.14221 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:04:35.670 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:35.669+0000 7fc3cd600700 1 -- 172.21.15.153:0/3848861681 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fc3c405ae60 con 0x7fc3d0073770 2024-03-21T10:04:35.936 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:35.934+0000 7fc3d7c00700 1 -- 172.21.15.153:0/3848861681 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"} v 0) v1 -- 0x7fc3d019ec00 con 0x7fc3d0073770 2024-03-21T10:04:36.162 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:36.160+0000 7fc3cd600700 1 -- 172.21.15.153:0/3848861681 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]=0 v6) v1 ==== 105+0+0 (secure 0 0 0) 0x7fc3c405a9f0 con 0x7fc3d0073770 2024-03-21T10:04:36.165 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:36.163+0000 7fc3d7c00700 1 -- 172.21.15.153:0/3848861681 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fc3b806c4e0 msgr2=0x7fc3b806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:04:36.165 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:36.163+0000 7fc3d7c00700 1 --2- 172.21.15.153:0/3848861681 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fc3b806c4e0 0x7fc3b806e9a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc005950 tx=0x7fc3bc0058e0 comp rx=0 tx=0).stop 2024-03-21T10:04:36.165 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:36.163+0000 7fc3d7c00700 1 -- 172.21.15.153:0/3848861681 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc3d0073770 msgr2=0x7fc3d0198550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:04:36.165 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:36.163+0000 7fc3d7c00700 1 --2- 172.21.15.153:0/3848861681 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc3d0073770 0x7fc3d0198550 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc3c400b5c0 tx=0x7fc3c400be20 comp rx=0 tx=0).stop 2024-03-21T10:04:36.165 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:36.163+0000 7fc3d7c00700 1 -- 172.21.15.153:0/3848861681 shutdown_connections 2024-03-21T10:04:36.165 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:36.163+0000 7fc3d7c00700 1 --2- 172.21.15.153:0/3848861681 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fc3b806c4e0 0x7fc3b806e9a0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:04:36.166 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:36.163+0000 7fc3d7c00700 1 --2- 172.21.15.153:0/3848861681 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc3d0073770 0x7fc3d0198550 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:04:36.166 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:36.163+0000 7fc3d7c00700 1 --2- 172.21.15.153:0/3848861681 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc3d0074d40 0x7fc3d0198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:04:36.166 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:36.163+0000 7fc3d7c00700 1 -- 172.21.15.153:0/3848861681 >> 172.21.15.153:0/3848861681 conn(0x7fc3d00fc190 msgr2=0x7fc3d0106a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:04:36.166 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:36.164+0000 7fc3d7c00700 1 -- 172.21.15.153:0/3848861681 shutdown_connections 2024-03-21T10:04:36.166 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:36.164+0000 7fc3d7c00700 1 -- 172.21.15.153:0/3848861681 wait complete. 2024-03-21T10:04:36.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:36 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:04:36.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:36 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:04:36.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:36 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:04:36.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:36 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi153.iuyluj", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T10:04:36.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:36 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi153.iuyluj", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T10:04:36.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:36 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:04:36.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:36 smithi153 ceph-mon[29117]: Deploying daemon mds.cephfs.smithi153.iuyluj on smithi153 2024-03-21T10:04:36.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:36 smithi153 ceph-mon[29117]: from='client.? 172.21.15.153:0/3848861681' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]: dispatch 2024-03-21T10:04:36.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:36 smithi153 ceph-mon[29117]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-21T10:04:36.808 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:36 smithi153 ceph-mon[29117]: mds.? [v2:172.21.15.174:6824/1776947618,v1:172.21.15.174:6825/1776947618] up:boot 2024-03-21T10:04:36.808 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:36 smithi153 ceph-mon[29117]: from='client.? 172.21.15.153:0/3848861681' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]': finished 2024-03-21T10:04:36.808 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:36 smithi153 ceph-mon[29117]: daemon mds.cephfs.smithi174.yyimst assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-21T10:04:36.808 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:36 smithi153 ceph-mon[29117]: Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY) 2024-03-21T10:04:36.808 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:36 smithi153 ceph-mon[29117]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-21T10:04:36.808 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:36 smithi153 ceph-mon[29117]: fsmap cephfs:1 {0=cephfs.smithi153.zipckr=up:active} 1 up:standby 2024-03-21T10:04:36.808 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:36 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi174.yyimst"}]: dispatch 2024-03-21T10:04:36.808 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:36 smithi153 ceph-mon[29117]: fsmap cephfs:2 {0=cephfs.smithi153.zipckr=up:active,1=cephfs.smithi174.yyimst=up:creating} 2024-03-21T10:04:36.808 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:36 smithi153 ceph-mon[29117]: daemon mds.cephfs.smithi174.yyimst is now active in filesystem cephfs as rank 1 2024-03-21T10:04:36.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:36 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:04:36.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:36 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:04:36.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:36 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:04:36.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:36 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi153.iuyluj", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T10:04:36.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:36 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi153.iuyluj", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T10:04:36.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:36 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:04:36.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:36 smithi174 ceph-mon[37496]: Deploying daemon mds.cephfs.smithi153.iuyluj on smithi153 2024-03-21T10:04:36.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:36 smithi174 ceph-mon[37496]: from='client.? 172.21.15.153:0/3848861681' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]: dispatch 2024-03-21T10:04:36.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:36 smithi174 ceph-mon[37496]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-21T10:04:36.953 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:36 smithi174 ceph-mon[37496]: mds.? [v2:172.21.15.174:6824/1776947618,v1:172.21.15.174:6825/1776947618] up:boot 2024-03-21T10:04:36.953 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:36 smithi174 ceph-mon[37496]: from='client.? 172.21.15.153:0/3848861681' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]': finished 2024-03-21T10:04:36.953 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:36 smithi174 ceph-mon[37496]: daemon mds.cephfs.smithi174.yyimst assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-21T10:04:36.953 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:36 smithi174 ceph-mon[37496]: Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY) 2024-03-21T10:04:36.953 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:36 smithi174 ceph-mon[37496]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-21T10:04:36.953 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:36 smithi174 ceph-mon[37496]: fsmap cephfs:1 {0=cephfs.smithi153.zipckr=up:active} 1 up:standby 2024-03-21T10:04:36.953 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:36 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi174.yyimst"}]: dispatch 2024-03-21T10:04:36.953 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:36 smithi174 ceph-mon[37496]: fsmap cephfs:2 {0=cephfs.smithi153.zipckr=up:active,1=cephfs.smithi174.yyimst=up:creating} 2024-03-21T10:04:36.953 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:36 smithi174 ceph-mon[37496]: daemon mds.cephfs.smithi174.yyimst is now active in filesystem cephfs as rank 1 2024-03-21T10:04:37.003 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-21T10:04:37.012 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi153.front.sepia.ceph.com 2024-03-21T10:04:37.013 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 070ed3e4-e769-11ee-95cd-87774f69a715 -- bash -c 'ceph fs set cephfs allow_standby_replay true' 2024-03-21T10:04:37.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:37 smithi153 ceph-mon[29117]: pgmap v220: 65 pgs: 65 active+clean; 578 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 496 B/s wr, 2 op/s 2024-03-21T10:04:37.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:37 smithi153 ceph-mon[29117]: mds.? [v2:172.21.15.174:6824/1776947618,v1:172.21.15.174:6825/1776947618] up:active 2024-03-21T10:04:37.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:37 smithi153 ceph-mon[29117]: fsmap cephfs:2 {0=cephfs.smithi153.zipckr=up:active,1=cephfs.smithi174.yyimst=up:active} 2024-03-21T10:04:37.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:37 smithi174 ceph-mon[37496]: pgmap v220: 65 pgs: 65 active+clean; 578 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 496 B/s wr, 2 op/s 2024-03-21T10:04:37.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:37 smithi174 ceph-mon[37496]: mds.? [v2:172.21.15.174:6824/1776947618,v1:172.21.15.174:6825/1776947618] up:active 2024-03-21T10:04:37.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:37 smithi174 ceph-mon[37496]: fsmap cephfs:2 {0=cephfs.smithi153.zipckr=up:active,1=cephfs.smithi174.yyimst=up:active} 2024-03-21T10:04:38.485 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi153/config 2024-03-21T10:04:39.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:39 smithi174 ceph-mon[37496]: pgmap v221: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.8 KiB/s wr, 5 op/s 2024-03-21T10:04:39.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:39 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:04:39.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:39 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:04:39.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:39 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:04:39.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:39 smithi153 ceph-mon[29117]: pgmap v221: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.8 KiB/s wr, 5 op/s 2024-03-21T10:04:39.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:39 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:04:39.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:39 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:04:39.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:39 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:04:41.392 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:41.390+0000 7f7af9e00700 1 -- 172.21.15.153:0/152860723 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7af40737f0 msgr2=0x7f7af4073c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:04:41.393 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:41.390+0000 7f7af9e00700 1 --2- 172.21.15.153:0/152860723 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7af40737f0 0x7f7af4073c70 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7ae4009910 tx=0x7f7ae4009c20 comp rx=0 tx=0).stop 2024-03-21T10:04:41.393 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:41.390+0000 7f7af9e00700 1 -- 172.21.15.153:0/152860723 shutdown_connections 2024-03-21T10:04:41.393 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:41.390+0000 7f7af9e00700 1 --2- 172.21.15.153:0/152860723 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7af40737f0 0x7f7af4073c70 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:04:41.393 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:41.390+0000 7f7af9e00700 1 --2- 172.21.15.153:0/152860723 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7af4074dc0 0x7f7af4073220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:04:41.393 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:41.390+0000 7f7af9e00700 1 -- 172.21.15.153:0/152860723 >> 172.21.15.153:0/152860723 conn(0x7f7af40fbf60 msgr2=0x7f7af40fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:04:41.393 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:41.390+0000 7f7af9e00700 1 -- 172.21.15.153:0/152860723 shutdown_connections 2024-03-21T10:04:41.393 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:41.391+0000 7f7af9e00700 1 -- 172.21.15.153:0/152860723 wait complete. 2024-03-21T10:04:41.395 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:41.392+0000 7f7af9e00700 1 Processor -- start 2024-03-21T10:04:41.395 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:41.392+0000 7f7af9e00700 1 -- start start 2024-03-21T10:04:41.395 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:41.392+0000 7f7af9e00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7af40737f0 0x7f7af41983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:04:41.396 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:41.392+0000 7f7af9e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7af4074dc0 0x7f7af4198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:04:41.396 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:41.392+0000 7f7af9e00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7af4198f40 con 0x7f7af4074dc0 2024-03-21T10:04:41.396 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:41.392+0000 7f7af9e00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7af4199080 con 0x7f7af40737f0 2024-03-21T10:04:41.396 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:41.393+0000 7f7aebe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7af4074dc0 0x7f7af4198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:04:41.396 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:41.393+0000 7f7aebe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7af4074dc0 0x7f7af4198920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:57668/0 (socket says 172.21.15.153:57668) 2024-03-21T10:04:41.396 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:41.393+0000 7f7aebe00700 1 -- 172.21.15.153:0/638505967 learned_addr learned my addr 172.21.15.153:0/638505967 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:04:41.396 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:41.393+0000 7f7af8a00700 1 --2- 172.21.15.153:0/638505967 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7af40737f0 0x7f7af41983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:04:41.396 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:41.393+0000 7f7aebe00700 1 -- 172.21.15.153:0/638505967 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7af40737f0 msgr2=0x7f7af41983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:04:41.396 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:41.393+0000 7f7aebe00700 1 --2- 172.21.15.153:0/638505967 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7af40737f0 0x7f7af41983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:04:41.396 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:41.393+0000 7f7aebe00700 1 -- 172.21.15.153:0/638505967 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ae40095f0 con 0x7f7af4074dc0 2024-03-21T10:04:41.397 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:41.393+0000 7f7aebe00700 1 --2- 172.21.15.153:0/638505967 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7af4074dc0 0x7f7af4198920 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7ae400bd70 tx=0x7f7ae400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:04:41.397 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:41.394+0000 7f7af2000700 1 -- 172.21.15.153:0/638505967 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7ae401d070 con 0x7f7af4074dc0 2024-03-21T10:04:41.397 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:41.394+0000 7f7af2000700 1 -- 172.21.15.153:0/638505967 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f7ae400f460 con 0x7f7af4074dc0 2024-03-21T10:04:41.397 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:41.394+0000 7f7af9e00700 1 -- 172.21.15.153:0/638505967 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7af419dad0 con 0x7f7af4074dc0 2024-03-21T10:04:41.397 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:41.394+0000 7f7af2000700 1 -- 172.21.15.153:0/638505967 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7ae4021620 con 0x7f7af4074dc0 2024-03-21T10:04:41.397 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:41.394+0000 7f7af9e00700 1 -- 172.21.15.153:0/638505967 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7af419dfc0 con 0x7f7af4074dc0 2024-03-21T10:04:41.398 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:41.395+0000 7f7af9e00700 1 -- 172.21.15.153:0/638505967 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7af410b790 con 0x7f7af4074dc0 2024-03-21T10:04:41.398 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:41.396+0000 7f7af2000700 1 -- 172.21.15.153:0/638505967 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f7ae400fa40 con 0x7f7af4074dc0 2024-03-21T10:04:41.398 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:41.397+0000 7f7af2000700 1 --2- 172.21.15.153:0/638505967 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f7ad406c4e0 0x7f7ad406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:04:41.399 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:41.397+0000 7f7af2000700 1 -- 172.21.15.153:0/638505967 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f7ae408cc30 con 0x7f7af4074dc0 2024-03-21T10:04:41.399 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:41.397+0000 7f7af8a00700 1 --2- 172.21.15.153:0/638505967 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f7ad406c4e0 0x7f7ad406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:04:41.399 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:41.398+0000 7f7af8a00700 1 --2- 172.21.15.153:0/638505967 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f7ad406c4e0 0x7f7ad406e9a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7adc005950 tx=0x7f7adc0058e0 comp rx=0 tx=0).ready entity=mgr.14221 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:04:41.403 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:41.402+0000 7f7af2000700 1 -- 172.21.15.153:0/638505967 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f7ae405ae00 con 0x7f7af4074dc0 2024-03-21T10:04:41.667 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:41.665+0000 7f7af9e00700 1 -- 172.21.15.153:0/638505967 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7f7af419e630 con 0x7f7af4074dc0 2024-03-21T10:04:41.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:41 smithi153 ceph-mon[29117]: pgmap v222: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.6 KiB/s wr, 5 op/s 2024-03-21T10:04:41.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:41 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:04:41.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:41 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:04:41.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:41 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:04:41.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:41 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi174.nhozag", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T10:04:41.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:41 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi174.nhozag", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T10:04:41.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:41 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:04:41.818 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:41 smithi174 ceph-mon[37496]: pgmap v222: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.6 KiB/s wr, 5 op/s 2024-03-21T10:04:41.818 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:41 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:04:41.818 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:41 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:04:41.818 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:41 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:04:41.818 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:41 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi174.nhozag", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T10:04:41.818 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:41 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi174.nhozag", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T10:04:41.818 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:41 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:04:42.570 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:42.568+0000 7f7af2000700 1 -- 172.21.15.153:0/638505967 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v10) v1 ==== 121+0+0 (secure 0 0 0) 0x7f7ae405a990 con 0x7f7af4074dc0 2024-03-21T10:04:42.574 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:42.572+0000 7f7af9e00700 1 -- 172.21.15.153:0/638505967 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f7ad406c4e0 msgr2=0x7f7ad406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:04:42.574 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:42.572+0000 7f7af9e00700 1 --2- 172.21.15.153:0/638505967 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f7ad406c4e0 0x7f7ad406e9a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7adc005950 tx=0x7f7adc0058e0 comp rx=0 tx=0).stop 2024-03-21T10:04:42.575 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:42.572+0000 7f7af9e00700 1 -- 172.21.15.153:0/638505967 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7af4074dc0 msgr2=0x7f7af4198920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:04:42.575 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:42.572+0000 7f7af9e00700 1 --2- 172.21.15.153:0/638505967 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7af4074dc0 0x7f7af4198920 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7ae400bd70 tx=0x7f7ae400be50 comp rx=0 tx=0).stop 2024-03-21T10:04:42.575 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:42.572+0000 7f7af9e00700 1 -- 172.21.15.153:0/638505967 shutdown_connections 2024-03-21T10:04:42.575 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:42.572+0000 7f7af9e00700 1 --2- 172.21.15.153:0/638505967 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f7ad406c4e0 0x7f7ad406e9a0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:04:42.575 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:42.573+0000 7f7af9e00700 1 --2- 172.21.15.153:0/638505967 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7af40737f0 0x7f7af41983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:04:42.575 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:42.573+0000 7f7af9e00700 1 --2- 172.21.15.153:0/638505967 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7af4074dc0 0x7f7af4198920 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:04:42.575 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:42.573+0000 7f7af9e00700 1 -- 172.21.15.153:0/638505967 >> 172.21.15.153:0/638505967 conn(0x7f7af40fbf60 msgr2=0x7f7af41067e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:04:42.575 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:42.573+0000 7f7af9e00700 1 -- 172.21.15.153:0/638505967 shutdown_connections 2024-03-21T10:04:42.575 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:42.573+0000 7f7af9e00700 1 -- 172.21.15.153:0/638505967 wait complete. 2024-03-21T10:04:42.828 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:42 smithi174 ceph-mon[37496]: Deploying daemon mds.cephfs.smithi174.nhozag on smithi174 2024-03-21T10:04:42.828 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:42 smithi174 ceph-mon[37496]: Health check cleared: MDS_INSUFFICIENT_STANDBY (was: insufficient standby MDS daemons available) 2024-03-21T10:04:42.828 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:42 smithi174 ceph-mon[37496]: Cluster is now healthy 2024-03-21T10:04:42.828 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:42 smithi174 ceph-mon[37496]: mds.? [v2:172.21.15.153:6828/866138999,v1:172.21.15.153:6829/866138999] up:boot 2024-03-21T10:04:42.828 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:42 smithi174 ceph-mon[37496]: fsmap cephfs:2 {0=cephfs.smithi153.zipckr=up:active,1=cephfs.smithi174.yyimst=up:active} 1 up:standby 2024-03-21T10:04:42.828 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:42 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi153.iuyluj"}]: dispatch 2024-03-21T10:04:42.828 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:42 smithi174 ceph-mon[37496]: from='client.? 172.21.15.153:0/638505967' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-21T10:04:42.958 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:42 smithi153 ceph-mon[29117]: Deploying daemon mds.cephfs.smithi174.nhozag on smithi174 2024-03-21T10:04:42.959 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:42 smithi153 ceph-mon[29117]: Health check cleared: MDS_INSUFFICIENT_STANDBY (was: insufficient standby MDS daemons available) 2024-03-21T10:04:42.959 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:42 smithi153 ceph-mon[29117]: Cluster is now healthy 2024-03-21T10:04:42.959 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:42 smithi153 ceph-mon[29117]: mds.? [v2:172.21.15.153:6828/866138999,v1:172.21.15.153:6829/866138999] up:boot 2024-03-21T10:04:42.959 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:42 smithi153 ceph-mon[29117]: fsmap cephfs:2 {0=cephfs.smithi153.zipckr=up:active,1=cephfs.smithi174.yyimst=up:active} 1 up:standby 2024-03-21T10:04:42.959 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:42 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi153.iuyluj"}]: dispatch 2024-03-21T10:04:42.959 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:42 smithi153 ceph-mon[29117]: from='client.? 172.21.15.153:0/638505967' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-21T10:04:43.408 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-21T10:04:43.418 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi153.front.sepia.ceph.com 2024-03-21T10:04:43.418 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 070ed3e4-e769-11ee-95cd-87774f69a715 -- bash -c 'ceph fs set cephfs inline_data true --yes-i-really-really-mean-it' 2024-03-21T10:04:43.638 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:43 smithi174 ceph-mon[37496]: pgmap v223: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s wr, 7 op/s 2024-03-21T10:04:43.638 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:43 smithi174 ceph-mon[37496]: from='client.? 172.21.15.153:0/638505967' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-21T10:04:43.638 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:43 smithi174 ceph-mon[37496]: fsmap cephfs:2 {0=cephfs.smithi153.zipckr=up:active,1=cephfs.smithi174.yyimst=up:active} 1 up:standby 2024-03-21T10:04:43.638 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:43 smithi174 ceph-mon[37496]: fsmap cephfs:2 {0=cephfs.smithi153.zipckr=up:active,1=cephfs.smithi174.yyimst=up:active} 1 up:standby-replay 2024-03-21T10:04:43.688 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:43 smithi153 ceph-mon[29117]: pgmap v223: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s wr, 7 op/s 2024-03-21T10:04:43.688 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:43 smithi153 ceph-mon[29117]: from='client.? 172.21.15.153:0/638505967' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-21T10:04:43.688 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:43 smithi153 ceph-mon[29117]: fsmap cephfs:2 {0=cephfs.smithi153.zipckr=up:active,1=cephfs.smithi174.yyimst=up:active} 1 up:standby 2024-03-21T10:04:43.688 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:43 smithi153 ceph-mon[29117]: fsmap cephfs:2 {0=cephfs.smithi153.zipckr=up:active,1=cephfs.smithi174.yyimst=up:active} 1 up:standby-replay 2024-03-21T10:04:43.831 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi153/config 2024-03-21T10:04:45.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:45 smithi153 ceph-mon[29117]: pgmap v224: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 85 B/s rd, 2.2 KiB/s wr, 7 op/s 2024-03-21T10:04:45.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:45 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:04:45.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:45 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:04:45.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:45 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:04:45.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:45 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:04:45.808 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:45 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:04:45.808 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:45 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:04:45.808 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:45 smithi153 ceph-mon[29117]: mds.? [v2:172.21.15.174:6826/806095873,v1:172.21.15.174:6827/806095873] up:boot 2024-03-21T10:04:45.808 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:45 smithi153 ceph-mon[29117]: mds.? [v2:172.21.15.153:6828/866138999,v1:172.21.15.153:6829/866138999] up:standby-replay 2024-03-21T10:04:45.808 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:45 smithi153 ceph-mon[29117]: Dropping low affinity active daemon mds.cephfs.smithi153.zipckr in favor of higher affinity standby. 2024-03-21T10:04:45.808 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:45 smithi153 ceph-mon[29117]: Replacing daemon mds.cephfs.smithi153.zipckr as rank 0 with standby daemon mds.cephfs.smithi153.iuyluj 2024-03-21T10:04:45.808 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:45 smithi153 ceph-mon[29117]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-21T10:04:45.808 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:45 smithi153 ceph-mon[29117]: fsmap cephfs:2 {0=cephfs.smithi153.zipckr=up:active,1=cephfs.smithi174.yyimst=up:active} 1 up:standby-replay 1 up:standby 2024-03-21T10:04:45.808 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:45 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi174.nhozag"}]: dispatch 2024-03-21T10:04:45.808 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:45 smithi153 ceph-mon[29117]: osdmap e47: 6 total, 6 up, 6 in 2024-03-21T10:04:45.808 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:45 smithi153 ceph-mon[29117]: fsmap cephfs:2/2 {0=cephfs.smithi153.iuyluj=up:replay,1=cephfs.smithi174.yyimst=up:active} 1 up:standby 2024-03-21T10:04:45.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:45 smithi174 ceph-mon[37496]: pgmap v224: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 85 B/s rd, 2.2 KiB/s wr, 7 op/s 2024-03-21T10:04:45.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:45 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:04:45.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:45 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:04:45.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:45 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:04:45.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:45 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:04:45.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:45 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:04:45.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:45 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:04:45.953 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:45 smithi174 ceph-mon[37496]: mds.? [v2:172.21.15.174:6826/806095873,v1:172.21.15.174:6827/806095873] up:boot 2024-03-21T10:04:45.953 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:45 smithi174 ceph-mon[37496]: mds.? [v2:172.21.15.153:6828/866138999,v1:172.21.15.153:6829/866138999] up:standby-replay 2024-03-21T10:04:45.953 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:45 smithi174 ceph-mon[37496]: Dropping low affinity active daemon mds.cephfs.smithi153.zipckr in favor of higher affinity standby. 2024-03-21T10:04:45.953 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:45 smithi174 ceph-mon[37496]: Replacing daemon mds.cephfs.smithi153.zipckr as rank 0 with standby daemon mds.cephfs.smithi153.iuyluj 2024-03-21T10:04:45.953 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:45 smithi174 ceph-mon[37496]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-21T10:04:45.953 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:45 smithi174 ceph-mon[37496]: fsmap cephfs:2 {0=cephfs.smithi153.zipckr=up:active,1=cephfs.smithi174.yyimst=up:active} 1 up:standby-replay 1 up:standby 2024-03-21T10:04:45.953 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:45 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi174.nhozag"}]: dispatch 2024-03-21T10:04:45.953 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:45 smithi174 ceph-mon[37496]: osdmap e47: 6 total, 6 up, 6 in 2024-03-21T10:04:45.953 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:45 smithi174 ceph-mon[37496]: fsmap cephfs:2/2 {0=cephfs.smithi153.iuyluj=up:replay,1=cephfs.smithi174.yyimst=up:active} 1 up:standby 2024-03-21T10:04:45.978 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:45.976+0000 7fcfb0200700 1 -- 172.21.15.153:0/739074866 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcfa8101db0 msgr2=0x7fcfa8102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:04:45.979 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:45.976+0000 7fcfb0200700 1 --2- 172.21.15.153:0/739074866 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcfa8101db0 0x7fcfa8102210 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fcf94009910 tx=0x7fcf94009c20 comp rx=0 tx=0).stop 2024-03-21T10:04:45.979 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:45.977+0000 7fcfb0200700 1 -- 172.21.15.153:0/739074866 shutdown_connections 2024-03-21T10:04:45.979 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:45.977+0000 7fcfb0200700 1 --2- 172.21.15.153:0/739074866 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcfa8101db0 0x7fcfa8102210 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:04:45.979 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:45.977+0000 7fcfb0200700 1 --2- 172.21.15.153:0/739074866 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcfa8100bb0 0x7fcfa8100fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:04:45.979 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:45.977+0000 7fcfb0200700 1 -- 172.21.15.153:0/739074866 >> 172.21.15.153:0/739074866 conn(0x7fcfa80fc110 msgr2=0x7fcfa80fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:04:45.979 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:45.977+0000 7fcfb0200700 1 -- 172.21.15.153:0/739074866 shutdown_connections 2024-03-21T10:04:45.979 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:45.977+0000 7fcfb0200700 1 -- 172.21.15.153:0/739074866 wait complete. 2024-03-21T10:04:45.980 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:45.978+0000 7fcfb0200700 1 Processor -- start 2024-03-21T10:04:45.980 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:45.978+0000 7fcfb0200700 1 -- start start 2024-03-21T10:04:45.980 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:45.978+0000 7fcfb0200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcfa8100bb0 0x7fcfa8198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:04:45.980 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:45.978+0000 7fcfb0200700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcfa8101db0 0x7fcfa8198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:04:45.980 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:45.978+0000 7fcfb0200700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfa81990b0 con 0x7fcfa8100bb0 2024-03-21T10:04:45.980 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:45.978+0000 7fcfb0200700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfa81991f0 con 0x7fcfa8101db0 2024-03-21T10:04:45.981 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:45.979+0000 7fcfad800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcfa8100bb0 0x7fcfa8198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:04:45.981 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:45.979+0000 7fcfad800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcfa8100bb0 0x7fcfa8198550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:35722/0 (socket says 172.21.15.153:35722) 2024-03-21T10:04:45.981 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:45.979+0000 7fcfad800700 1 -- 172.21.15.153:0/1442478729 learned_addr learned my addr 172.21.15.153:0/1442478729 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:04:45.981 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:45.979+0000 7fcface00700 1 --2- 172.21.15.153:0/1442478729 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcfa8101db0 0x7fcfa8198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:04:45.981 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:45.979+0000 7fcfad800700 1 -- 172.21.15.153:0/1442478729 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcfa8101db0 msgr2=0x7fcfa8198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:04:45.982 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:45.979+0000 7fcfad800700 1 --2- 172.21.15.153:0/1442478729 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcfa8101db0 0x7fcfa8198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:04:45.982 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:45.979+0000 7fcfad800700 1 -- 172.21.15.153:0/1442478729 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf940095f0 con 0x7fcfa8100bb0 2024-03-21T10:04:45.982 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:45.979+0000 7fcface00700 1 --2- 172.21.15.153:0/1442478729 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcfa8101db0 0x7fcfa8198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:04:45.982 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:45.980+0000 7fcfad800700 1 --2- 172.21.15.153:0/1442478729 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcfa8100bb0 0x7fcfa8198550 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fcf9c00c8f0 tx=0x7fcf9c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:04:45.982 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:45.980+0000 7fcfa6000700 1 -- 172.21.15.153:0/1442478729 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcf9c007a70 con 0x7fcfa8100bb0 2024-03-21T10:04:45.982 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:45.980+0000 7fcfb0200700 1 -- 172.21.15.153:0/1442478729 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfa819dca0 con 0x7fcfa8100bb0 2024-03-21T10:04:45.982 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:45.980+0000 7fcfa6000700 1 -- 172.21.15.153:0/1442478729 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fcf9c007bd0 con 0x7fcfa8100bb0 2024-03-21T10:04:45.982 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:45.980+0000 7fcfa6000700 1 -- 172.21.15.153:0/1442478729 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcf9c018630 con 0x7fcfa8100bb0 2024-03-21T10:04:45.982 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:45.980+0000 7fcfb0200700 1 -- 172.21.15.153:0/1442478729 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfa819e1a0 con 0x7fcfa8100bb0 2024-03-21T10:04:45.983 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:45.982+0000 7fcfb0200700 1 -- 172.21.15.153:0/1442478729 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfa8105e90 con 0x7fcfa8100bb0 2024-03-21T10:04:45.985 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:45.983+0000 7fcfa6000700 1 -- 172.21.15.153:0/1442478729 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fcf9c00f450 con 0x7fcfa8100bb0 2024-03-21T10:04:45.985 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:45.984+0000 7fcfa6000700 1 --2- 172.21.15.153:0/1442478729 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fcf9006c530 0x7fcf9006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:04:45.986 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:45.984+0000 7fcfa6000700 1 -- 172.21.15.153:0/1442478729 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fcf9c08b930 con 0x7fcfa8100bb0 2024-03-21T10:04:45.986 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:45.985+0000 7fcface00700 1 --2- 172.21.15.153:0/1442478729 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fcf9006c530 0x7fcf9006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:04:45.987 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:45.986+0000 7fcface00700 1 --2- 172.21.15.153:0/1442478729 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fcf9006c530 0x7fcf9006e9f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fcf94006010 tx=0x7fcf9400b540 comp rx=0 tx=0).ready entity=mgr.14221 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:04:45.990 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:45.989+0000 7fcfa6000700 1 -- 172.21.15.153:0/1442478729 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fcf9c059b50 con 0x7fcfa8100bb0 2024-03-21T10:04:46.281 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:46.279+0000 7fcfb0200700 1 -- 172.21.15.153:0/1442478729 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fcfa802cc70 con 0x7fcfa8100bb0 2024-03-21T10:04:47.070 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:47.068+0000 7fcfa6000700 1 -- 172.21.15.153:0/1442478729 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]=0 inline data enabled v15) v1 ==== 168+0+0 (secure 0 0 0) 0x7fcf9c0596e0 con 0x7fcfa8100bb0 2024-03-21T10:04:47.073 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:47.072+0000 7fcfb0200700 1 -- 172.21.15.153:0/1442478729 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fcf9006c530 msgr2=0x7fcf9006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:04:47.073 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:47.072+0000 7fcfb0200700 1 --2- 172.21.15.153:0/1442478729 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fcf9006c530 0x7fcf9006e9f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fcf94006010 tx=0x7fcf9400b540 comp rx=0 tx=0).stop 2024-03-21T10:04:47.074 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:47.072+0000 7fcfb0200700 1 -- 172.21.15.153:0/1442478729 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcfa8100bb0 msgr2=0x7fcfa8198550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:04:47.074 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:47.072+0000 7fcfb0200700 1 --2- 172.21.15.153:0/1442478729 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcfa8100bb0 0x7fcfa8198550 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fcf9c00c8f0 tx=0x7fcf9c00ccb0 comp rx=0 tx=0).stop 2024-03-21T10:04:47.074 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:47.072+0000 7fcfb0200700 1 -- 172.21.15.153:0/1442478729 shutdown_connections 2024-03-21T10:04:47.075 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:47.072+0000 7fcfb0200700 1 --2- 172.21.15.153:0/1442478729 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fcf9006c530 0x7fcf9006e9f0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:04:47.075 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:47.072+0000 7fcfb0200700 1 --2- 172.21.15.153:0/1442478729 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcfa8100bb0 0x7fcfa8198550 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:04:47.075 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:47.072+0000 7fcfb0200700 1 --2- 172.21.15.153:0/1442478729 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcfa8101db0 0x7fcfa8198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:04:47.075 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:47.073+0000 7fcfb0200700 1 -- 172.21.15.153:0/1442478729 >> 172.21.15.153:0/1442478729 conn(0x7fcfa80fc110 msgr2=0x7fcfa8102fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:04:47.075 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:47.073+0000 7fcfb0200700 1 -- 172.21.15.153:0/1442478729 shutdown_connections 2024-03-21T10:04:47.075 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:47.073+0000 7fcfb0200700 1 -- 172.21.15.153:0/1442478729 wait complete. 2024-03-21T10:04:47.076 INFO:teuthology.orchestra.run.smithi153.stderr:inline data enabled 2024-03-21T10:04:47.333 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:47 smithi174 ceph-mon[37496]: mds.? [v2:172.21.15.153:6826/4033622615,v1:172.21.15.153:6827/4033622615] up:boot 2024-03-21T10:04:47.334 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:47 smithi174 ceph-mon[37496]: mds.? [v2:172.21.15.153:6828/866138999,v1:172.21.15.153:6829/866138999] up:resolve 2024-03-21T10:04:47.334 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:47 smithi174 ceph-mon[37496]: fsmap cephfs:2/2 {0=cephfs.smithi153.iuyluj=up:resolve,1=cephfs.smithi174.yyimst=up:active} 2 up:standby 2024-03-21T10:04:47.334 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:47 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi153.zipckr"}]: dispatch 2024-03-21T10:04:47.334 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:47 smithi174 ceph-mon[37496]: from='client.? 172.21.15.153:0/1442478729' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-21T10:04:47.334 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:47 smithi174 ceph-mon[37496]: pgmap v226: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1023 B/s rd, 2.2 KiB/s wr, 8 op/s 2024-03-21T10:04:47.425 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:47 smithi153 ceph-mon[29117]: mds.? [v2:172.21.15.153:6826/4033622615,v1:172.21.15.153:6827/4033622615] up:boot 2024-03-21T10:04:47.425 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:47 smithi153 ceph-mon[29117]: mds.? [v2:172.21.15.153:6828/866138999,v1:172.21.15.153:6829/866138999] up:resolve 2024-03-21T10:04:47.425 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:47 smithi153 ceph-mon[29117]: fsmap cephfs:2/2 {0=cephfs.smithi153.iuyluj=up:resolve,1=cephfs.smithi174.yyimst=up:active} 2 up:standby 2024-03-21T10:04:47.426 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:47 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi153.zipckr"}]: dispatch 2024-03-21T10:04:47.426 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:47 smithi153 ceph-mon[29117]: from='client.? 172.21.15.153:0/1442478729' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-21T10:04:47.426 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:47 smithi153 ceph-mon[29117]: pgmap v226: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1023 B/s rd, 2.2 KiB/s wr, 8 op/s 2024-03-21T10:04:47.861 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-21T10:04:47.885 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi153.front.sepia.ceph.com 2024-03-21T10:04:47.886 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 070ed3e4-e769-11ee-95cd-87774f69a715 -- bash -c 'ceph fs dump' 2024-03-21T10:04:48.333 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:48 smithi153 ceph-mon[29117]: Health check failed: 1 filesystem with deprecated feature inline_data (FS_INLINE_DATA_DEPRECATED) 2024-03-21T10:04:48.333 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:48 smithi153 ceph-mon[29117]: mds.? [v2:172.21.15.153:6828/866138999,v1:172.21.15.153:6829/866138999] up:reconnect 2024-03-21T10:04:48.333 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:48 smithi153 ceph-mon[29117]: from='client.? 172.21.15.153:0/1442478729' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]': finished 2024-03-21T10:04:48.333 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:48 smithi153 ceph-mon[29117]: fsmap cephfs:2/2 {0=cephfs.smithi153.iuyluj=up:reconnect,1=cephfs.smithi174.yyimst=up:active} 2 up:standby 2024-03-21T10:04:48.333 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:48 smithi153 ceph-mon[29117]: mds.? [v2:172.21.15.153:6828/866138999,v1:172.21.15.153:6829/866138999] up:rejoin 2024-03-21T10:04:48.333 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:48 smithi153 ceph-mon[29117]: fsmap cephfs:2/2 {0=cephfs.smithi153.iuyluj=up:rejoin,1=cephfs.smithi174.yyimst=up:active} 2 up:standby 2024-03-21T10:04:48.333 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:48 smithi153 ceph-mon[29117]: daemon mds.cephfs.smithi153.iuyluj is now active in filesystem cephfs as rank 0 2024-03-21T10:04:48.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:48 smithi174 ceph-mon[37496]: Health check failed: 1 filesystem with deprecated feature inline_data (FS_INLINE_DATA_DEPRECATED) 2024-03-21T10:04:48.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:48 smithi174 ceph-mon[37496]: mds.? [v2:172.21.15.153:6828/866138999,v1:172.21.15.153:6829/866138999] up:reconnect 2024-03-21T10:04:48.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:48 smithi174 ceph-mon[37496]: from='client.? 172.21.15.153:0/1442478729' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]': finished 2024-03-21T10:04:48.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:48 smithi174 ceph-mon[37496]: fsmap cephfs:2/2 {0=cephfs.smithi153.iuyluj=up:reconnect,1=cephfs.smithi174.yyimst=up:active} 2 up:standby 2024-03-21T10:04:48.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:48 smithi174 ceph-mon[37496]: mds.? [v2:172.21.15.153:6828/866138999,v1:172.21.15.153:6829/866138999] up:rejoin 2024-03-21T10:04:48.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:48 smithi174 ceph-mon[37496]: fsmap cephfs:2/2 {0=cephfs.smithi153.iuyluj=up:rejoin,1=cephfs.smithi174.yyimst=up:active} 2 up:standby 2024-03-21T10:04:48.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:48 smithi174 ceph-mon[37496]: daemon mds.cephfs.smithi153.iuyluj is now active in filesystem cephfs as rank 0 2024-03-21T10:04:48.904 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi153/config 2024-03-21T10:04:49.657 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:49 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:04:49.657 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:49 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:04:49.657 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:49 smithi153 ceph-mon[29117]: pgmap v227: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.6 KiB/s rd, 818 B/s wr, 4 op/s 2024-03-21T10:04:49.657 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:49 smithi153 ceph-mon[29117]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-21T10:04:49.658 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:49 smithi153 ceph-mon[29117]: mds.? [v2:172.21.15.153:6828/866138999,v1:172.21.15.153:6829/866138999] up:active 2024-03-21T10:04:49.658 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:49 smithi153 ceph-mon[29117]: mds.? [v2:172.21.15.174:6824/1776947618,v1:172.21.15.174:6825/1776947618] up:active 2024-03-21T10:04:49.658 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:49 smithi153 ceph-mon[29117]: mds.? [v2:172.21.15.174:6826/806095873,v1:172.21.15.174:6827/806095873] up:standby 2024-03-21T10:04:49.658 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:49 smithi153 ceph-mon[29117]: fsmap cephfs:2 {0=cephfs.smithi153.iuyluj=up:active,1=cephfs.smithi174.yyimst=up:active} 2 up:standby-replay 2024-03-21T10:04:49.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:49 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:04:49.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:49 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:04:49.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:49 smithi174 ceph-mon[37496]: pgmap v227: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.6 KiB/s rd, 818 B/s wr, 4 op/s 2024-03-21T10:04:49.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:49 smithi174 ceph-mon[37496]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-21T10:04:49.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:49 smithi174 ceph-mon[37496]: mds.? [v2:172.21.15.153:6828/866138999,v1:172.21.15.153:6829/866138999] up:active 2024-03-21T10:04:49.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:49 smithi174 ceph-mon[37496]: mds.? [v2:172.21.15.174:6824/1776947618,v1:172.21.15.174:6825/1776947618] up:active 2024-03-21T10:04:49.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:49 smithi174 ceph-mon[37496]: mds.? [v2:172.21.15.174:6826/806095873,v1:172.21.15.174:6827/806095873] up:standby 2024-03-21T10:04:49.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:49 smithi174 ceph-mon[37496]: fsmap cephfs:2 {0=cephfs.smithi153.iuyluj=up:active,1=cephfs.smithi174.yyimst=up:active} 2 up:standby-replay 2024-03-21T10:04:50.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:50 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:04:50.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:50 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:04:51.487 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:51 smithi153 ceph-mon[29117]: pgmap v228: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.6 KiB/s rd, 818 B/s wr, 4 op/s 2024-03-21T10:04:51.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:51 smithi174 ceph-mon[37496]: pgmap v228: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.6 KiB/s rd, 818 B/s wr, 4 op/s 2024-03-21T10:04:52.182 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:52.179+0000 7f6562e00700 1 -- 172.21.15.153:0/1953919848 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f655c074dc0 msgr2=0x7f655c073220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:04:52.182 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:52.179+0000 7f6562e00700 1 --2- 172.21.15.153:0/1953919848 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f655c074dc0 0x7f655c073220 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f654c009910 tx=0x7f654c009c20 comp rx=0 tx=0).stop 2024-03-21T10:04:52.182 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:52.180+0000 7f6562e00700 1 -- 172.21.15.153:0/1953919848 shutdown_connections 2024-03-21T10:04:52.182 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:52.180+0000 7f6562e00700 1 --2- 172.21.15.153:0/1953919848 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f655c0737f0 0x7f655c073c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:04:52.182 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:52.180+0000 7f6562e00700 1 --2- 172.21.15.153:0/1953919848 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f655c074dc0 0x7f655c073220 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:04:52.182 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:52.180+0000 7f6562e00700 1 -- 172.21.15.153:0/1953919848 >> 172.21.15.153:0/1953919848 conn(0x7f655c0fc110 msgr2=0x7f655c0fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:04:52.182 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:52.180+0000 7f6562e00700 1 -- 172.21.15.153:0/1953919848 shutdown_connections 2024-03-21T10:04:52.182 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:52.180+0000 7f6562e00700 1 -- 172.21.15.153:0/1953919848 wait complete. 2024-03-21T10:04:52.183 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:52.182+0000 7f6562e00700 1 Processor -- start 2024-03-21T10:04:52.183 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:52.182+0000 7f6562e00700 1 -- start start 2024-03-21T10:04:52.184 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:52.182+0000 7f6562e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f655c0737f0 0x7f655c1985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:04:52.184 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:52.183+0000 7f6562e00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f655c074dc0 0x7f655c198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:04:52.184 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:52.183+0000 7f6562e00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f655c199100 con 0x7f655c0737f0 2024-03-21T10:04:52.184 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:52.183+0000 7f6562e00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f655c199240 con 0x7f655c074dc0 2024-03-21T10:04:52.184 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:52.183+0000 7f655be00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f655c0737f0 0x7f655c1985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:04:52.185 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:52.183+0000 7f655be00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f655c0737f0 0x7f655c1985a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:35744/0 (socket says 172.21.15.153:35744) 2024-03-21T10:04:52.185 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:52.183+0000 7f655be00700 1 -- 172.21.15.153:0/2200084940 learned_addr learned my addr 172.21.15.153:0/2200084940 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:04:52.185 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:52.183+0000 7f655b400700 1 --2- 172.21.15.153:0/2200084940 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f655c074dc0 0x7f655c198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:04:52.185 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:52.183+0000 7f655be00700 1 -- 172.21.15.153:0/2200084940 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f655c074dc0 msgr2=0x7f655c198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:04:52.185 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:52.183+0000 7f655be00700 1 --2- 172.21.15.153:0/2200084940 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f655c074dc0 0x7f655c198ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:04:52.185 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:52.184+0000 7f655be00700 1 -- 172.21.15.153:0/2200084940 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6548009520 con 0x7f655c0737f0 2024-03-21T10:04:52.186 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:52.184+0000 7f655be00700 1 --2- 172.21.15.153:0/2200084940 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f655c0737f0 0x7f655c1985a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f654c005b40 tx=0x7f654c00fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:04:52.186 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:52.184+0000 7f6558c00700 1 -- 172.21.15.153:0/2200084940 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f654c01d070 con 0x7f655c0737f0 2024-03-21T10:04:52.186 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:52.184+0000 7f6558c00700 1 -- 172.21.15.153:0/2200084940 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f654c005dc0 con 0x7f655c0737f0 2024-03-21T10:04:52.186 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:52.184+0000 7f6562e00700 1 -- 172.21.15.153:0/2200084940 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f654c0095f0 con 0x7f655c0737f0 2024-03-21T10:04:52.187 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:52.185+0000 7f6562e00700 1 -- 172.21.15.153:0/2200084940 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f655c19dff0 con 0x7f655c0737f0 2024-03-21T10:04:52.187 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:52.185+0000 7f6558c00700 1 -- 172.21.15.153:0/2200084940 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f654c0226a0 con 0x7f655c0737f0 2024-03-21T10:04:52.188 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:52.186+0000 7f6562e00700 1 -- 172.21.15.153:0/2200084940 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f653c005320 con 0x7f655c0737f0 2024-03-21T10:04:52.189 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:52.187+0000 7f6558c00700 1 -- 172.21.15.153:0/2200084940 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f654c003680 con 0x7f655c0737f0 2024-03-21T10:04:52.190 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:52.189+0000 7f6558c00700 1 --2- 172.21.15.153:0/2200084940 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f654006c4e0 0x7f654006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:04:52.190 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:52.189+0000 7f6558c00700 1 -- 172.21.15.153:0/2200084940 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f654c08cf30 con 0x7f655c0737f0 2024-03-21T10:04:52.190 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:52.189+0000 7f655b400700 1 --2- 172.21.15.153:0/2200084940 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f654006c4e0 0x7f654006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:04:52.191 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:52.190+0000 7f655b400700 1 --2- 172.21.15.153:0/2200084940 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f654006c4e0 0x7f654006e9a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f655c074af0 tx=0x7f6548009260 comp rx=0 tx=0).ready entity=mgr.14221 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:04:52.195 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:52.193+0000 7f6558c00700 1 -- 172.21.15.153:0/2200084940 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f654c05b0a0 con 0x7f655c0737f0 2024-03-21T10:04:52.527 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:52.526+0000 7f6562e00700 1 -- 172.21.15.153:0/2200084940 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f653c006200 con 0x7f655c0737f0 2024-03-21T10:04:52.527 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:52.527+0000 7f6558c00700 1 -- 172.21.15.153:0/2200084940 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 17 v17) v1 ==== 76+0+1850 (secure 0 0 0) 0x7f654c003930 con 0x7f655c0737f0 2024-03-21T10:04:52.528 INFO:teuthology.orchestra.run.smithi153.stdout:e17 2024-03-21T10:04:52.528 INFO:teuthology.orchestra.run.smithi153.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-21T10:04:52.528 INFO:teuthology.orchestra.run.smithi153.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:04:52.528 INFO:teuthology.orchestra.run.smithi153.stdout:legacy client fscid: 1 2024-03-21T10:04:52.528 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:04:52.528 INFO:teuthology.orchestra.run.smithi153.stdout:Filesystem 'cephfs' (1) 2024-03-21T10:04:52.528 INFO:teuthology.orchestra.run.smithi153.stdout:fs_name cephfs 2024-03-21T10:04:52.528 INFO:teuthology.orchestra.run.smithi153.stdout:epoch 17 2024-03-21T10:04:52.528 INFO:teuthology.orchestra.run.smithi153.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-21T10:04:52.528 INFO:teuthology.orchestra.run.smithi153.stdout:created 2024-03-21T10:04:27.086576+0000 2024-03-21T10:04:52.529 INFO:teuthology.orchestra.run.smithi153.stdout:modified 2024-03-21T10:04:49.033187+0000 2024-03-21T10:04:52.529 INFO:teuthology.orchestra.run.smithi153.stdout:tableserver 0 2024-03-21T10:04:52.529 INFO:teuthology.orchestra.run.smithi153.stdout:root 0 2024-03-21T10:04:52.529 INFO:teuthology.orchestra.run.smithi153.stdout:session_timeout 60 2024-03-21T10:04:52.529 INFO:teuthology.orchestra.run.smithi153.stdout:session_autoclose 300 2024-03-21T10:04:52.529 INFO:teuthology.orchestra.run.smithi153.stdout:max_file_size 1099511627776 2024-03-21T10:04:52.529 INFO:teuthology.orchestra.run.smithi153.stdout:required_client_features {} 2024-03-21T10:04:52.529 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure 0 2024-03-21T10:04:52.529 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure_osd_epoch 47 2024-03-21T10:04:52.529 INFO:teuthology.orchestra.run.smithi153.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:04:52.529 INFO:teuthology.orchestra.run.smithi153.stdout:max_mds 2 2024-03-21T10:04:52.530 INFO:teuthology.orchestra.run.smithi153.stdout:in 0,1 2024-03-21T10:04:52.530 INFO:teuthology.orchestra.run.smithi153.stdout:up {0=14484,1=24273} 2024-03-21T10:04:52.530 INFO:teuthology.orchestra.run.smithi153.stdout:failed 2024-03-21T10:04:52.530 INFO:teuthology.orchestra.run.smithi153.stdout:damaged 2024-03-21T10:04:52.530 INFO:teuthology.orchestra.run.smithi153.stdout:stopped 2024-03-21T10:04:52.530 INFO:teuthology.orchestra.run.smithi153.stdout:data_pools [3] 2024-03-21T10:04:52.530 INFO:teuthology.orchestra.run.smithi153.stdout:metadata_pool 2 2024-03-21T10:04:52.530 INFO:teuthology.orchestra.run.smithi153.stdout:inline_data enabled 2024-03-21T10:04:52.530 INFO:teuthology.orchestra.run.smithi153.stdout:balancer 2024-03-21T10:04:52.530 INFO:teuthology.orchestra.run.smithi153.stdout:bal_rank_mask -1 2024-03-21T10:04:52.530 INFO:teuthology.orchestra.run.smithi153.stdout:standby_count_wanted 1 2024-03-21T10:04:52.531 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.iuyluj{0:14484} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.153:6828/866138999,v1:172.21.15.153:6829/866138999] compat {c=[1],r=[1],i=[7ff]}] 2024-03-21T10:04:52.531 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.zipckr{0:14494} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.153:6826/4033622615,v1:172.21.15.153:6827/4033622615] compat {c=[1],r=[1],i=[7ff]}] 2024-03-21T10:04:52.531 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.yyimst{1:24273} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.174:6824/1776947618,v1:172.21.15.174:6825/1776947618] compat {c=[1],r=[1],i=[7ff]}] 2024-03-21T10:04:52.531 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.nhozag{1:24285} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.174:6826/806095873,v1:172.21.15.174:6827/806095873] compat {c=[1],r=[1],i=[7ff]}] 2024-03-21T10:04:52.531 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:04:52.531 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:04:52.532 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:52.530+0000 7f6562e00700 1 -- 172.21.15.153:0/2200084940 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f654006c4e0 msgr2=0x7f654006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:04:52.532 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:52.530+0000 7f6562e00700 1 --2- 172.21.15.153:0/2200084940 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f654006c4e0 0x7f654006e9a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f655c074af0 tx=0x7f6548009260 comp rx=0 tx=0).stop 2024-03-21T10:04:52.532 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:52.530+0000 7f6562e00700 1 -- 172.21.15.153:0/2200084940 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f655c0737f0 msgr2=0x7f655c1985a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:04:52.533 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:52.530+0000 7f6562e00700 1 --2- 172.21.15.153:0/2200084940 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f655c0737f0 0x7f655c1985a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f654c005b40 tx=0x7f654c00fab0 comp rx=0 tx=0).stop 2024-03-21T10:04:52.533 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:52.531+0000 7f6562e00700 1 -- 172.21.15.153:0/2200084940 shutdown_connections 2024-03-21T10:04:52.533 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:52.531+0000 7f6562e00700 1 --2- 172.21.15.153:0/2200084940 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f654006c4e0 0x7f654006e9a0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:04:52.533 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:52.531+0000 7f6562e00700 1 --2- 172.21.15.153:0/2200084940 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f655c0737f0 0x7f655c1985a0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:04:52.533 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:52.531+0000 7f6562e00700 1 --2- 172.21.15.153:0/2200084940 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f655c074dc0 0x7f655c198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:04:52.533 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:52.531+0000 7f6562e00700 1 -- 172.21.15.153:0/2200084940 >> 172.21.15.153:0/2200084940 conn(0x7f655c0fc110 msgr2=0x7f655c106990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:04:52.533 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:52.531+0000 7f6562e00700 1 -- 172.21.15.153:0/2200084940 shutdown_connections 2024-03-21T10:04:52.533 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:52.531+0000 7f6562e00700 1 -- 172.21.15.153:0/2200084940 wait complete. 2024-03-21T10:04:52.534 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 17 2024-03-21T10:04:53.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:53 smithi174 ceph-mon[37496]: pgmap v229: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 9.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-21T10:04:53.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:53 smithi174 ceph-mon[37496]: from='client.? 172.21.15.153:0/2200084940' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:04:53.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:53 smithi153 ceph-mon[29117]: pgmap v229: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 9.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-21T10:04:53.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:53 smithi153 ceph-mon[29117]: from='client.? 172.21.15.153:0/2200084940' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:04:53.934 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 070ed3e4-e769-11ee-95cd-87774f69a715 -- bash -c 'ceph --format=json fs dump | jq -e ".filesystems | length == 1"' 2024-03-21T10:04:54.359 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi153/config 2024-03-21T10:04:55.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:55 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:04:55.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:55 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:04:55.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:55 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:04:55.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:55 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:04:55.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:55 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:04:55.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:55 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:04:55.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:55 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:04:55.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:55 smithi174 ceph-mon[37496]: pgmap v230: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 10 KiB/s rd, 0 B/s wr, 11 op/s 2024-03-21T10:04:55.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:55 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:04:55.766 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:55 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:04:55.766 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:55 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:04:55.766 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:55 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:04:55.766 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:55 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:04:55.766 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:55 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:04:55.767 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:55 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:04:55.767 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:55 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:04:55.767 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:55 smithi153 ceph-mon[29117]: pgmap v230: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 10 KiB/s rd, 0 B/s wr, 11 op/s 2024-03-21T10:04:55.767 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:55 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:04:56.572 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:56.571+0000 7f8a57600700 1 -- 172.21.15.153:0/3588840823 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8a500ff430 msgr2=0x7f8a500ff850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:04:56.572 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:56.571+0000 7f8a57600700 1 --2- 172.21.15.153:0/3588840823 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8a500ff430 0x7f8a500ff850 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f8a44009910 tx=0x7f8a44009c20 comp rx=0 tx=0).stop 2024-03-21T10:04:56.572 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:56.571+0000 7f8a57600700 1 -- 172.21.15.153:0/3588840823 shutdown_connections 2024-03-21T10:04:56.572 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:56.571+0000 7f8a57600700 1 --2- 172.21.15.153:0/3588840823 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8a500ffd90 0x7f8a50100210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:04:56.572 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:56.571+0000 7f8a57600700 1 --2- 172.21.15.153:0/3588840823 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8a500ff430 0x7f8a500ff850 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:04:56.572 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:56.571+0000 7f8a57600700 1 -- 172.21.15.153:0/3588840823 >> 172.21.15.153:0/3588840823 conn(0x7f8a500fb030 msgr2=0x7f8a500fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:04:56.573 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:56.571+0000 7f8a57600700 1 -- 172.21.15.153:0/3588840823 shutdown_connections 2024-03-21T10:04:56.573 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:56.571+0000 7f8a57600700 1 -- 172.21.15.153:0/3588840823 wait complete. 2024-03-21T10:04:56.574 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:56.572+0000 7f8a57600700 1 Processor -- start 2024-03-21T10:04:56.574 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:56.573+0000 7f8a57600700 1 -- start start 2024-03-21T10:04:56.574 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:56.573+0000 7f8a57600700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8a500ff430 0x7f8a50198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:04:56.574 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:56.573+0000 7f8a57600700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8a500ffd90 0x7f8a50198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:04:56.574 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:56.573+0000 7f8a57600700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a50199060 con 0x7f8a500ffd90 2024-03-21T10:04:56.574 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:56.573+0000 7f8a57600700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a501991a0 con 0x7f8a500ff430 2024-03-21T10:04:56.575 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:56.573+0000 7f8a4fe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8a500ffd90 0x7f8a50198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:04:56.575 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:56.573+0000 7f8a4fe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8a500ffd90 0x7f8a50198a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39866/0 (socket says 172.21.15.153:39866) 2024-03-21T10:04:56.575 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:56.573+0000 7f8a4fe00700 1 -- 172.21.15.153:0/3990867656 learned_addr learned my addr 172.21.15.153:0/3990867656 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:04:56.575 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:56.573+0000 7f8a54c00700 1 --2- 172.21.15.153:0/3990867656 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8a500ff430 0x7f8a50198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:04:56.575 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:56.574+0000 7f8a4fe00700 1 -- 172.21.15.153:0/3990867656 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8a500ff430 msgr2=0x7f8a50198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:04:56.575 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:56.574+0000 7f8a4fe00700 1 --2- 172.21.15.153:0/3990867656 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8a500ff430 0x7f8a50198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:04:56.575 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:56.574+0000 7f8a4fe00700 1 -- 172.21.15.153:0/3990867656 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a440095f0 con 0x7f8a500ffd90 2024-03-21T10:04:56.576 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:56.574+0000 7f8a4fe00700 1 --2- 172.21.15.153:0/3990867656 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8a500ffd90 0x7f8a50198a40 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c00d8d0 tx=0x7f8a3c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:04:56.576 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:56.574+0000 7f8a4d600700 1 -- 172.21.15.153:0/3990867656 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8a3c009940 con 0x7f8a500ffd90 2024-03-21T10:04:56.576 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:56.574+0000 7f8a4d600700 1 -- 172.21.15.153:0/3990867656 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f8a3c010460 con 0x7f8a500ffd90 2024-03-21T10:04:56.576 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:56.574+0000 7f8a57600700 1 -- 172.21.15.153:0/3990867656 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a5019dc50 con 0x7f8a500ffd90 2024-03-21T10:04:56.576 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:56.574+0000 7f8a4d600700 1 -- 172.21.15.153:0/3990867656 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8a3c00f5f0 con 0x7f8a500ffd90 2024-03-21T10:04:56.576 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:56.575+0000 7f8a57600700 1 -- 172.21.15.153:0/3990867656 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a5019e1a0 con 0x7f8a500ffd90 2024-03-21T10:04:56.576 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:56.576+0000 7f8a57600700 1 -- 172.21.15.153:0/3990867656 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a5010b900 con 0x7f8a500ffd90 2024-03-21T10:04:56.577 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:56.577+0000 7f8a4d600700 1 -- 172.21.15.153:0/3990867656 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f8a3c0105d0 con 0x7f8a500ffd90 2024-03-21T10:04:56.578 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:56.578+0000 7f8a4d600700 1 --2- 172.21.15.153:0/3990867656 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f8a3806c4e0 0x7f8a3806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:04:56.578 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:56.578+0000 7f8a4d600700 1 -- 172.21.15.153:0/3990867656 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f8a3c08b930 con 0x7f8a500ffd90 2024-03-21T10:04:56.579 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:56.578+0000 7f8a54c00700 1 --2- 172.21.15.153:0/3990867656 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f8a3806c4e0 0x7f8a3806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:04:56.580 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:56.579+0000 7f8a54c00700 1 --2- 172.21.15.153:0/3990867656 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f8a3806c4e0 0x7f8a3806e9a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8a44006010 tx=0x7f8a4400b540 comp rx=0 tx=0).ready entity=mgr.14221 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:04:56.583 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:56.583+0000 7f8a4d600700 1 -- 172.21.15.153:0/3990867656 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f8a3c059aa0 con 0x7f8a500ffd90 2024-03-21T10:04:56.848 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:56.848+0000 7f8a57600700 1 -- 172.21.15.153:0/3990867656 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8a5019e7b0 con 0x7f8a500ffd90 2024-03-21T10:04:56.849 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:56.849+0000 7f8a4d600700 1 -- 172.21.15.153:0/3990867656 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4776 (secure 0 0 0) 0x7f8a3c059630 con 0x7f8a500ffd90 2024-03-21T10:04:56.852 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:56.851+0000 7f8a57600700 1 -- 172.21.15.153:0/3990867656 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f8a3806c4e0 msgr2=0x7f8a3806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:04:56.852 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:56.851+0000 7f8a57600700 1 --2- 172.21.15.153:0/3990867656 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f8a3806c4e0 0x7f8a3806e9a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8a44006010 tx=0x7f8a4400b540 comp rx=0 tx=0).stop 2024-03-21T10:04:56.852 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:56.852+0000 7f8a57600700 1 -- 172.21.15.153:0/3990867656 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8a500ffd90 msgr2=0x7f8a50198a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:04:56.853 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:56.852+0000 7f8a57600700 1 --2- 172.21.15.153:0/3990867656 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8a500ffd90 0x7f8a50198a40 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c00d8d0 tx=0x7f8a3c00dc90 comp rx=0 tx=0).stop 2024-03-21T10:04:56.853 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:56.852+0000 7f8a57600700 1 -- 172.21.15.153:0/3990867656 shutdown_connections 2024-03-21T10:04:56.853 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:56.852+0000 7f8a57600700 1 --2- 172.21.15.153:0/3990867656 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f8a3806c4e0 0x7f8a3806e9a0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:04:56.853 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:56.852+0000 7f8a57600700 1 --2- 172.21.15.153:0/3990867656 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8a500ff430 0x7f8a50198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:04:56.853 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:56.852+0000 7f8a57600700 1 --2- 172.21.15.153:0/3990867656 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8a500ffd90 0x7f8a50198a40 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:04:56.853 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:56.852+0000 7f8a57600700 1 -- 172.21.15.153:0/3990867656 >> 172.21.15.153:0/3990867656 conn(0x7f8a500fb030 msgr2=0x7f8a50107a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:04:56.853 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:56.852+0000 7f8a57600700 1 -- 172.21.15.153:0/3990867656 shutdown_connections 2024-03-21T10:04:56.853 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:04:56.852+0000 7f8a57600700 1 -- 172.21.15.153:0/3990867656 wait complete. 2024-03-21T10:04:56.854 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 17 2024-03-21T10:04:56.869 INFO:teuthology.orchestra.run.smithi153.stdout:true 2024-03-21T10:04:57.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:57 smithi174 ceph-mon[37496]: pgmap v231: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 10 KiB/s rd, 0 B/s wr, 11 op/s 2024-03-21T10:04:57.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:57 smithi174 ceph-mon[37496]: from='client.? 172.21.15.153:0/3990867656' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:04:57.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:57 smithi153 ceph-mon[29117]: pgmap v231: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 10 KiB/s rd, 0 B/s wr, 11 op/s 2024-03-21T10:04:57.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:57 smithi153 ceph-mon[29117]: from='client.? 172.21.15.153:0/3990867656' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:04:57.958 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 070ed3e4-e769-11ee-95cd-87774f69a715 -- bash -c 'while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done' 2024-03-21T10:04:59.066 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi153/config 2024-03-21T10:04:59.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:58 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:04:59.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:58 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:04:59.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:04:58 smithi174 ceph-mon[37496]: pgmap v232: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 8.7 KiB/s rd, 0 B/s wr, 9 op/s 2024-03-21T10:04:59.219 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:58 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:04:59.219 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:58 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:04:59.219 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:04:58 smithi153 ceph-mon[29117]: pgmap v232: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 8.7 KiB/s rd, 0 B/s wr, 9 op/s 2024-03-21T10:05:01.599 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:01.597+0000 7f4ca1800700 1 -- 172.21.15.153:0/3856502930 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4c9c108400 msgr2=0x7f4c9c108860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:05:01.599 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:01.597+0000 7f4ca1800700 1 --2- 172.21.15.153:0/3856502930 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4c9c108400 0x7f4c9c108860 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f4c84009910 tx=0x7f4c84009c20 comp rx=0 tx=0).stop 2024-03-21T10:05:01.599 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:01.598+0000 7f4ca1800700 1 -- 172.21.15.153:0/3856502930 shutdown_connections 2024-03-21T10:05:01.599 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:01.598+0000 7f4ca1800700 1 --2- 172.21.15.153:0/3856502930 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4c9c108400 0x7f4c9c108860 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:05:01.600 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:01.598+0000 7f4ca1800700 1 --2- 172.21.15.153:0/3856502930 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4c9c107200 0x7f4c9c107620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:05:01.600 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:01.598+0000 7f4ca1800700 1 -- 172.21.15.153:0/3856502930 >> 172.21.15.153:0/3856502930 conn(0x7f4c9c0761b0 msgr2=0x7f4c9c078610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:05:01.600 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:01.598+0000 7f4ca1800700 1 -- 172.21.15.153:0/3856502930 shutdown_connections 2024-03-21T10:05:01.600 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:01.598+0000 7f4ca1800700 1 -- 172.21.15.153:0/3856502930 wait complete. 2024-03-21T10:05:01.600 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:01.599+0000 7f4ca1800700 1 Processor -- start 2024-03-21T10:05:01.600 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:01.599+0000 7f4ca1800700 1 -- start start 2024-03-21T10:05:01.602 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:01.599+0000 7f4ca1800700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4c9c107200 0x7f4c9c19c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:05:01.602 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:01.599+0000 7f4ca1800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4c9c108400 0x7f4c9c19cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:05:01.602 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:01.599+0000 7f4ca1800700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c9c19d430 con 0x7f4c9c108400 2024-03-21T10:05:01.602 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:01.599+0000 7f4ca1800700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c9c19d570 con 0x7f4c9c107200 2024-03-21T10:05:01.602 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:01.599+0000 7f4c92000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4c9c108400 0x7f4c9c19cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:05:01.602 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:01.600+0000 7f4c92000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4c9c108400 0x7f4c9c19cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39892/0 (socket says 172.21.15.153:39892) 2024-03-21T10:05:01.602 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:01.600+0000 7f4c92000700 1 -- 172.21.15.153:0/4179686903 learned_addr learned my addr 172.21.15.153:0/4179686903 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:05:01.602 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:01.600+0000 7f4c9aa00700 1 --2- 172.21.15.153:0/4179686903 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4c9c107200 0x7f4c9c19c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:05:01.602 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:01.600+0000 7f4c92000700 1 -- 172.21.15.153:0/4179686903 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4c9c107200 msgr2=0x7f4c9c19c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:05:01.602 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:01.600+0000 7f4c92000700 1 --2- 172.21.15.153:0/4179686903 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4c9c107200 0x7f4c9c19c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:05:01.602 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:01.600+0000 7f4c92000700 1 -- 172.21.15.153:0/4179686903 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c840095f0 con 0x7f4c9c108400 2024-03-21T10:05:01.603 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:01.600+0000 7f4c92000700 1 --2- 172.21.15.153:0/4179686903 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4c9c108400 0x7f4c9c19cea0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f4c8400b5c0 tx=0x7f4c8400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:05:01.603 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:01.601+0000 7f4c93e00700 1 -- 172.21.15.153:0/4179686903 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c8401d070 con 0x7f4c9c108400 2024-03-21T10:05:01.603 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:01.601+0000 7f4c93e00700 1 -- 172.21.15.153:0/4179686903 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f4c84022470 con 0x7f4c9c108400 2024-03-21T10:05:01.603 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:01.601+0000 7f4ca1800700 1 -- 172.21.15.153:0/4179686903 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c9c1a1fd0 con 0x7f4c9c108400 2024-03-21T10:05:01.603 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:01.601+0000 7f4c93e00700 1 -- 172.21.15.153:0/4179686903 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c8400f690 con 0x7f4c9c108400 2024-03-21T10:05:01.603 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:01.601+0000 7f4ca1800700 1 -- 172.21.15.153:0/4179686903 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c9c1a2490 con 0x7f4c9c108400 2024-03-21T10:05:01.603 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:01.602+0000 7f4ca1800700 1 -- 172.21.15.153:0/4179686903 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c9c10fd60 con 0x7f4c9c108400 2024-03-21T10:05:01.603 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:01.603+0000 7f4c93e00700 1 -- 172.21.15.153:0/4179686903 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f4c84004b70 con 0x7f4c9c108400 2024-03-21T10:05:01.605 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:01.605+0000 7f4c93e00700 1 --2- 172.21.15.153:0/4179686903 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f4c8006c4e0 0x7f4c8006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:05:01.605 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:01.605+0000 7f4c93e00700 1 -- 172.21.15.153:0/4179686903 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f4c8408cf20 con 0x7f4c9c108400 2024-03-21T10:05:01.605 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:01.605+0000 7f4c9aa00700 1 --2- 172.21.15.153:0/4179686903 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f4c8006c4e0 0x7f4c8006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:05:01.606 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:01.605+0000 7f4c9aa00700 1 --2- 172.21.15.153:0/4179686903 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f4c8006c4e0 0x7f4c8006e9a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c005fd0 tx=0x7f4c8c005e50 comp rx=0 tx=0).ready entity=mgr.14221 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:05:01.610 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:01.610+0000 7f4c93e00700 1 -- 172.21.15.153:0/4179686903 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f4c840c00e0 con 0x7f4c9c108400 2024-03-21T10:05:01.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:05:01 smithi174 ceph-mon[37496]: pgmap v233: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 8.2 KiB/s rd, 0 B/s wr, 9 op/s 2024-03-21T10:05:01.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:05:01 smithi153 ceph-mon[29117]: pgmap v233: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 8.2 KiB/s rd, 0 B/s wr, 9 op/s 2024-03-21T10:05:01.986 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:01.985+0000 7f4ca1800700 1 -- 172.21.15.153:0/4179686903 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "mds versions", "format": "json"} v 0) v1 -- 0x7f4c9c1a2af0 con 0x7f4c9c108400 2024-03-21T10:05:01.987 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:01.986+0000 7f4c93e00700 1 -- 172.21.15.153:0/4179686903 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "mds versions", "format": "json"}]=0 v17) v1 ==== 78+0+83 (secure 0 0 0) 0x7f4c84027090 con 0x7f4c9c108400 2024-03-21T10:05:01.991 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:01.990+0000 7f4ca1800700 1 -- 172.21.15.153:0/4179686903 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f4c8006c4e0 msgr2=0x7f4c8006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:05:01.991 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:01.990+0000 7f4ca1800700 1 --2- 172.21.15.153:0/4179686903 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f4c8006c4e0 0x7f4c8006e9a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c005fd0 tx=0x7f4c8c005e50 comp rx=0 tx=0).stop 2024-03-21T10:05:01.992 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:01.990+0000 7f4ca1800700 1 -- 172.21.15.153:0/4179686903 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4c9c108400 msgr2=0x7f4c9c19cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:05:01.992 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:01.990+0000 7f4ca1800700 1 --2- 172.21.15.153:0/4179686903 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4c9c108400 0x7f4c9c19cea0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f4c8400b5c0 tx=0x7f4c8400be20 comp rx=0 tx=0).stop 2024-03-21T10:05:01.992 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:01.990+0000 7f4ca1800700 1 -- 172.21.15.153:0/4179686903 shutdown_connections 2024-03-21T10:05:01.992 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:01.990+0000 7f4ca1800700 1 --2- 172.21.15.153:0/4179686903 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f4c8006c4e0 0x7f4c8006e9a0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:05:01.992 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:01.990+0000 7f4ca1800700 1 --2- 172.21.15.153:0/4179686903 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4c9c107200 0x7f4c9c19c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:05:01.992 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:01.990+0000 7f4ca1800700 1 --2- 172.21.15.153:0/4179686903 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4c9c108400 0x7f4c9c19cea0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:05:01.992 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:01.990+0000 7f4ca1800700 1 -- 172.21.15.153:0/4179686903 >> 172.21.15.153:0/4179686903 conn(0x7f4c9c0761b0 msgr2=0x7f4c9c10b630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:05:01.992 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:01.991+0000 7f4ca1800700 1 -- 172.21.15.153:0/4179686903 shutdown_connections 2024-03-21T10:05:01.992 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:01.991+0000 7f4ca1800700 1 -- 172.21.15.153:0/4179686903 wait complete. 2024-03-21T10:05:02.015 INFO:teuthology.orchestra.run.smithi153.stdout:true 2024-03-21T10:05:02.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:05:02 smithi153 ceph-mon[29117]: from='client.? 172.21.15.153:0/4179686903' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-03-21T10:05:02.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:05:02 smithi174 ceph-mon[37496]: from='client.? 172.21.15.153:0/4179686903' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-03-21T10:05:03.236 INFO:teuthology.run_tasks:Running task fs.pre_upgrade_save... 2024-03-21T10:05:03.249 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T10:05:03.475 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:03.474+0000 7ff422600640 1 -- 172.21.15.153:0/434152135 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff41c0fe1f0 msgr2=0x7ff41c0fe610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:05:03.475 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:03.474+0000 7ff422600640 1 --2- 172.21.15.153:0/434152135 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff41c0fe1f0 0x7ff41c0fe610 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7ff410009880 tx=0x7ff41002bce0 comp rx=0 tx=0).stop 2024-03-21T10:05:03.476 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:03.474+0000 7ff422600640 1 -- 172.21.15.153:0/434152135 shutdown_connections 2024-03-21T10:05:03.476 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:03.474+0000 7ff422600640 1 --2- 172.21.15.153:0/434152135 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff41c1011c0 0x7ff41c101640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:05:03.476 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:03.474+0000 7ff422600640 1 --2- 172.21.15.153:0/434152135 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff41c0fe1f0 0x7ff41c0fe610 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:05:03.476 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:03.474+0000 7ff422600640 1 -- 172.21.15.153:0/434152135 >> 172.21.15.153:0/434152135 conn(0x7ff41c0f9ae0 msgr2=0x7ff41c0fbf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:05:03.476 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:03.474+0000 7ff422600640 1 -- 172.21.15.153:0/434152135 shutdown_connections 2024-03-21T10:05:03.476 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:03.474+0000 7ff422600640 1 -- 172.21.15.153:0/434152135 wait complete. 2024-03-21T10:05:03.476 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:03.475+0000 7ff422600640 1 Processor -- start 2024-03-21T10:05:03.478 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:03.475+0000 7ff422600640 1 -- start start 2024-03-21T10:05:03.478 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:03.476+0000 7ff422600640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff41c0fe1f0 0x7ff41c100b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:05:03.478 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:03.476+0000 7ff422600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff41c1011c0 0x7ff41c0ff170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:05:03.479 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:03.476+0000 7ff422600640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff41c0ff6b0 con 0x7ff41c1011c0 2024-03-21T10:05:03.479 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:03.476+0000 7ff422600640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff41c0ff820 con 0x7ff41c0fe1f0 2024-03-21T10:05:03.479 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:03.476+0000 7ff41be00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff41c1011c0 0x7ff41c0ff170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:05:03.479 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:03.476+0000 7ff41be00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff41c1011c0 0x7ff41c0ff170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:54976/0 (socket says 172.21.15.153:54976) 2024-03-21T10:05:03.479 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:03.476+0000 7ff41be00640 1 -- 172.21.15.153:0/102602282 learned_addr learned my addr 172.21.15.153:0/102602282 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:05:03.479 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:03.476+0000 7ff421200640 1 --2- 172.21.15.153:0/102602282 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff41c0fe1f0 0x7ff41c100b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:05:03.479 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:03.476+0000 7ff41be00640 1 -- 172.21.15.153:0/102602282 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff41c0fe1f0 msgr2=0x7ff41c100b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:05:03.479 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:03.476+0000 7ff41be00640 1 --2- 172.21.15.153:0/102602282 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff41c0fe1f0 0x7ff41c100b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:05:03.479 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:03.476+0000 7ff41be00640 1 -- 172.21.15.153:0/102602282 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff410009560 con 0x7ff41c1011c0 2024-03-21T10:05:03.479 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:03.477+0000 7ff41be00640 1 --2- 172.21.15.153:0/102602282 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff41c1011c0 0x7ff41c0ff170 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ff40c00d6f0 tx=0x7ff40c00dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:05:03.479 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:03.477+0000 7ff419600640 1 -- 172.21.15.153:0/102602282 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff40c009a30 con 0x7ff41c1011c0 2024-03-21T10:05:03.479 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:03.477+0000 7ff419600640 1 -- 172.21.15.153:0/102602282 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff40c0054d0 con 0x7ff41c1011c0 2024-03-21T10:05:03.479 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:03.477+0000 7ff422600640 1 -- 172.21.15.153:0/102602282 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff41c0ffb00 con 0x7ff41c1011c0 2024-03-21T10:05:03.480 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:03.477+0000 7ff419600640 1 -- 172.21.15.153:0/102602282 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff40c010460 con 0x7ff41c1011c0 2024-03-21T10:05:03.480 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:03.477+0000 7ff422600640 1 -- 172.21.15.153:0/102602282 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff41c19d730 con 0x7ff41c1011c0 2024-03-21T10:05:03.480 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:03.478+0000 7ff422600640 1 -- 172.21.15.153:0/102602282 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff41c108ab0 con 0x7ff41c1011c0 2024-03-21T10:05:03.480 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:03.479+0000 7ff419600640 1 -- 172.21.15.153:0/102602282 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7ff40c009b90 con 0x7ff41c1011c0 2024-03-21T10:05:03.481 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:03.480+0000 7ff419600640 1 --2- 172.21.15.153:0/102602282 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7ff40406c510 0x7ff40406e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:05:03.481 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:03.480+0000 7ff419600640 1 -- 172.21.15.153:0/102602282 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7ff40c08b3d0 con 0x7ff41c1011c0 2024-03-21T10:05:03.481 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:03.480+0000 7ff421200640 1 --2- 172.21.15.153:0/102602282 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7ff40406c510 0x7ff40406e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:05:03.481 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:03.481+0000 7ff421200640 1 --2- 172.21.15.153:0/102602282 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7ff40406c510 0x7ff40406e9d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff410006010 tx=0x7ff41002cf90 comp rx=0 tx=0).ready entity=mgr.14221 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:05:03.485 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:03.485+0000 7ff419600640 1 -- 172.21.15.153:0/102602282 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7ff40c055b30 con 0x7ff41c1011c0 2024-03-21T10:05:03.689 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:03.689+0000 7ff422600640 1 -- 172.21.15.153:0/102602282 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff41c108ca0 con 0x7ff41c1011c0 2024-03-21T10:05:03.691 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:03.690+0000 7ff419600640 1 -- 172.21.15.153:0/102602282 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) v1 ==== 74+0+15165 (secure 0 0 0) 0x7ff40c059150 con 0x7ff41c1011c0 2024-03-21T10:05:03.691 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:05:03.691 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":47,"fsid":"070ed3e4-e769-11ee-95cd-87774f69a715","created":"2024-03-21T09:55:19.360240+0000","modified":"2024-03-21T10:04:45.049196+0000","last_up_change":"2024-03-21T10:02:55.420099+0000","last_in_change":"2024-03-21T10:02:32.306573+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T10:01:18.943639+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-03-21T10:04:25.621358+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-03-21T10:04:26.593674+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0f2e1224-6f45-4f23-b97a-5b89319a90f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":1050381948},{"type":"v1","addr":"172.21.15.153:6803","nonce":1050381948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":1050381948},{"type":"v1","addr":"172.21.15.153:6805","nonce":1050381948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":1050381948},{"type":"v1","addr":"172.21.15.153:6809","nonce":1050381948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":1050381948},{"type":"v1","addr":"172.21.15.153:6807","nonce":1050381948}]},"public_addr":"172.21.15.153:6803/1050381948","cluster_addr":"172.21.15.153:6805/1050381948","heartbeat_back_addr":"172.21.15.153:6809/1050381948","heartbeat_front_addr":"172.21.15.153:6807/1050381948","state":["exists","up"]},{"osd":1,"uuid":"76195edb-aa7a-4627-a211-09a578de65c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":3504985760},{"type":"v1","addr":"172.21.15.153:6811","nonce":3504985760}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":3504985760},{"type":"v1","addr":"172.21.15.153:6813","nonce":3504985760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":3504985760},{"type":"v1","addr":"172.21.15.153:6817","nonce":3504985760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":3504985760},{"type":"v1","addr":"172.21.15.153:6815","nonce":3504985760}]},"public_addr":"172.21.15.153:6811/3504985760","cluster_addr":"172.21.15.153:6813/3504985760","heartbeat_back_addr":"172.21.15.153:6817/3504985760","heartbeat_front_addr":"172.21.15.153:6815/3504985760","state":["exists","up"]},{"osd":2,"uuid":"3ec17cf8-f0ea-45c4-b084-b304024d6e92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":2844395219},{"type":"v1","addr":"172.21.15.153:6819","nonce":2844395219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":2844395219},{"type":"v1","addr":"172.21.15.153:6821","nonce":2844395219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":2844395219},{"type":"v1","addr":"172.21.15.153:6825","nonce":2844395219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":2844395219},{"type":"v1","addr":"172.21.15.153:6823","nonce":2844395219}]},"public_addr":"172.21.15.153:6819/2844395219","cluster_addr":"172.21.15.153:6821/2844395219","heartbeat_back_addr":"172.21.15.153:6825/2844395219","heartbeat_front_addr":"172.21.15.153:6823/2844395219","state":["exists","up"]},{"osd":3,"uuid":"285146f6-6209-47ee-9e5b-2e075c949231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":393592876},{"type":"v1","addr":"172.21.15.174:6801","nonce":393592876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":393592876},{"type":"v1","addr":"172.21.15.174:6803","nonce":393592876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":393592876},{"type":"v1","addr":"172.21.15.174:6807","nonce":393592876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":393592876},{"type":"v1","addr":"172.21.15.174:6805","nonce":393592876}]},"public_addr":"172.21.15.174:6801/393592876","cluster_addr":"172.21.15.174:6803/393592876","heartbeat_back_addr":"172.21.15.174:6807/393592876","heartbeat_front_addr":"172.21.15.174:6805/393592876","state":["exists","up"]},{"osd":4,"uuid":"f05cec2d-c649-489d-98a3-f24a75fe1f95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":116016676},{"type":"v1","addr":"172.21.15.174:6809","nonce":116016676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":116016676},{"type":"v1","addr":"172.21.15.174:6811","nonce":116016676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":116016676},{"type":"v1","addr":"172.21.15.174:6815","nonce":116016676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":116016676},{"type":"v1","addr":"172.21.15.174:6813","nonce":116016676}]},"public_addr":"172.21.15.174:6809/116016676","cluster_addr":"172.21.15.174:6811/116016676","heartbeat_back_addr":"172.21.15.174:6815/116016676","heartbeat_front_addr":"172.21.15.174:6813/116016676","state":["exists","up"]},{"osd":5,"uuid":"f430a3b2-e1b5-4698-9089-812fcf3a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":184680961},{"type":"v1","addr":"172.21.15.174:6817","nonce":184680961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":184680961},{"type":"v1","addr":"172.21.15.174:6819","nonce":184680961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":184680961},{"type":"v1","addr":"172.21.15.174:6823","nonce":184680961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":184680961},{"type":"v1","addr":"172.21.15.174:6821","nonce":184680961}]},"public_addr":"172.21.15.174:6817/184680961","cluster_addr":"172.21.15.174:6819/184680961","heartbeat_back_addr":"172.21.15.174:6823/184680961","heartbeat_front_addr":"172.21.15.174:6821/184680961","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:00:04.356688+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:00:39.988638+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:01:14.882381+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:01:48.072468+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:02:21.211701+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:02:54.161138+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.153:6827/1353876997":"2024-03-22T10:04:45.049156+0000","172.21.15.153:0/3435678189":"2024-03-22T09:56:00.849533+0000","172.21.15.153:0/816489762":"2024-03-22T09:56:43.480866+0000","172.21.15.153:6801/270837948":"2024-03-22T09:56:00.849533+0000","172.21.15.153:6800/270837948":"2024-03-22T09:56:00.849533+0000","172.21.15.153:6801/1143522828":"2024-03-22T09:58:24.250138+0000","172.21.15.153:6826/1353876997":"2024-03-22T10:04:45.049156+0000","172.21.15.153:0/623594592":"2024-03-22T09:56:00.849533+0000","172.21.15.153:6800/3178125243":"2024-03-22T09:56:43.480866+0000","172.21.15.153:6801/3178125243":"2024-03-22T09:56:43.480866+0000","172.21.15.153:0/3456116624":"2024-03-22T09:56:43.480866+0000","172.21.15.153:6800/1143522828":"2024-03-22T09:58:24.250138+0000","172.21.15.153:0/2407536999":"2024-03-22T09:56:00.849533+0000","172.21.15.153:0/405164318":"2024-03-22T09:58:24.250138+0000","172.21.15.153:0/1163205431":"2024-03-22T09:56:43.480866+0000","172.21.15.153:0/652104290":"2024-03-22T09:58:24.250138+0000","172.21.15.153:0/3278686288":"2024-03-22T09:58:24.250138+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T10:05:03.694 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:03.693+0000 7ff422600640 1 -- 172.21.15.153:0/102602282 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7ff40406c510 msgr2=0x7ff40406e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:05:03.694 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:03.693+0000 7ff422600640 1 --2- 172.21.15.153:0/102602282 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7ff40406c510 0x7ff40406e9d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff410006010 tx=0x7ff41002cf90 comp rx=0 tx=0).stop 2024-03-21T10:05:03.694 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:03.693+0000 7ff422600640 1 -- 172.21.15.153:0/102602282 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff41c1011c0 msgr2=0x7ff41c0ff170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:05:03.694 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:03.693+0000 7ff422600640 1 --2- 172.21.15.153:0/102602282 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff41c1011c0 0x7ff41c0ff170 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ff40c00d6f0 tx=0x7ff40c00dbb0 comp rx=0 tx=0).stop 2024-03-21T10:05:03.694 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:03.693+0000 7ff422600640 1 -- 172.21.15.153:0/102602282 shutdown_connections 2024-03-21T10:05:03.694 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:03.693+0000 7ff422600640 1 --2- 172.21.15.153:0/102602282 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7ff40406c510 0x7ff40406e9d0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:05:03.694 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:03.693+0000 7ff422600640 1 --2- 172.21.15.153:0/102602282 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff41c1011c0 0x7ff41c0ff170 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:05:03.694 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:03.693+0000 7ff422600640 1 --2- 172.21.15.153:0/102602282 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff41c0fe1f0 0x7ff41c100b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:05:03.694 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:03.693+0000 7ff422600640 1 -- 172.21.15.153:0/102602282 >> 172.21.15.153:0/102602282 conn(0x7ff41c0f9ae0 msgr2=0x7ff41c1043f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:05:03.694 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:03.693+0000 7ff422600640 1 -- 172.21.15.153:0/102602282 shutdown_connections 2024-03-21T10:05:03.694 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:03.693+0000 7ff422600640 1 -- 172.21.15.153:0/102602282 wait complete. 2024-03-21T10:05:03.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:05:03 smithi174 ceph-mon[37496]: pgmap v234: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 9.0 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-21T10:05:03.710 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:05:03 smithi153 ceph-mon[29117]: pgmap v234: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 9.0 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-21T10:05:03.714 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-21T10:01:18.943639+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-03-21T10:04:25.621358+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-03-21T10:04:26.593674+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-21T10:05:03.714 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-21T10:05:03.917 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:03.917+0000 7fc4e4e00640 1 -- 172.21.15.153:0/1393483885 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc4e01034c0 msgr2=0x7fc4e0103940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:05:03.918 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:03.917+0000 7fc4e4e00640 1 --2- 172.21.15.153:0/1393483885 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc4e01034c0 0x7fc4e0103940 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc009880 tx=0x7fc4cc02bcf0 comp rx=0 tx=0).stop 2024-03-21T10:05:03.918 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:03.917+0000 7fc4e4e00640 1 -- 172.21.15.153:0/1393483885 shutdown_connections 2024-03-21T10:05:03.918 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:03.917+0000 7fc4e4e00640 1 --2- 172.21.15.153:0/1393483885 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc4e01034c0 0x7fc4e0103940 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:05:03.918 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:03.917+0000 7fc4e4e00640 1 --2- 172.21.15.153:0/1393483885 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc4e01022d0 0x7fc4e01026f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:05:03.918 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:03.917+0000 7fc4e4e00640 1 -- 172.21.15.153:0/1393483885 >> 172.21.15.153:0/1393483885 conn(0x7fc4e00fd880 msgr2=0x7fc4e00ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:05:03.918 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:03.917+0000 7fc4e4e00640 1 -- 172.21.15.153:0/1393483885 shutdown_connections 2024-03-21T10:05:03.918 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:03.917+0000 7fc4e4e00640 1 -- 172.21.15.153:0/1393483885 wait complete. 2024-03-21T10:05:03.920 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:03.918+0000 7fc4e4e00640 1 Processor -- start 2024-03-21T10:05:03.920 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:03.918+0000 7fc4e4e00640 1 -- start start 2024-03-21T10:05:03.920 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:03.918+0000 7fc4e4e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc4e01022d0 0x7fc4e0197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:05:03.920 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:03.918+0000 7fc4e4e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc4e01034c0 0x7fc4e01980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:05:03.920 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:03.918+0000 7fc4e4e00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e0198690 con 0x7fc4e01034c0 2024-03-21T10:05:03.921 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:03.918+0000 7fc4e4e00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e0198800 con 0x7fc4e01022d0 2024-03-21T10:05:03.921 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:03.919+0000 7fc4dd400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc4e01034c0 0x7fc4e01980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:05:03.921 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:03.919+0000 7fc4dd400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc4e01034c0 0x7fc4e01980c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:54998/0 (socket says 172.21.15.153:54998) 2024-03-21T10:05:03.921 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:03.919+0000 7fc4dd400640 1 -- 172.21.15.153:0/2310998308 learned_addr learned my addr 172.21.15.153:0/2310998308 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:05:03.921 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:03.919+0000 7fc4dd400640 1 -- 172.21.15.153:0/2310998308 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc4e01022d0 msgr2=0x7fc4e0197b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:05:03.921 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:03.919+0000 7fc4dde00640 1 --2- 172.21.15.153:0/2310998308 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc4e01022d0 0x7fc4e0197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:05:03.921 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:03.919+0000 7fc4dd400640 1 --2- 172.21.15.153:0/2310998308 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc4e01022d0 0x7fc4e0197b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:05:03.921 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:03.919+0000 7fc4dd400640 1 -- 172.21.15.153:0/2310998308 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4cc009560 con 0x7fc4e01034c0 2024-03-21T10:05:03.921 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:03.919+0000 7fc4dd400640 1 --2- 172.21.15.153:0/2310998308 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc4e01034c0 0x7fc4e01980c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc036040 tx=0x7fc4cc004f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:05:03.921 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:03.920+0000 7fc4d6a00640 1 -- 172.21.15.153:0/2310998308 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc4cc004520 con 0x7fc4e01034c0 2024-03-21T10:05:03.921 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:03.920+0000 7fc4d6a00640 1 -- 172.21.15.153:0/2310998308 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc4cc0054b0 con 0x7fc4e01034c0 2024-03-21T10:05:03.921 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:03.920+0000 7fc4e4e00640 1 -- 172.21.15.153:0/2310998308 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4e019d230 con 0x7fc4e01034c0 2024-03-21T10:05:03.921 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:03.920+0000 7fc4d6a00640 1 -- 172.21.15.153:0/2310998308 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc4cc034440 con 0x7fc4e01034c0 2024-03-21T10:05:03.922 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:03.920+0000 7fc4e4e00640 1 -- 172.21.15.153:0/2310998308 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4e019d790 con 0x7fc4e01034c0 2024-03-21T10:05:03.922 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:03.921+0000 7fc4e4e00640 1 -- 172.21.15.153:0/2310998308 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4e010b020 con 0x7fc4e01034c0 2024-03-21T10:05:03.922 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:03.922+0000 7fc4d6a00640 1 -- 172.21.15.153:0/2310998308 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fc4cc003880 con 0x7fc4e01034c0 2024-03-21T10:05:03.923 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:03.923+0000 7fc4d6a00640 1 --2- 172.21.15.153:0/2310998308 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fc4c406c450 0x7fc4c406e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:05:03.924 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:03.923+0000 7fc4d6a00640 1 -- 172.21.15.153:0/2310998308 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fc4cc0adce0 con 0x7fc4e01034c0 2024-03-21T10:05:03.924 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:03.924+0000 7fc4dde00640 1 --2- 172.21.15.153:0/2310998308 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fc4c406c450 0x7fc4c406e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:05:03.924 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:03.924+0000 7fc4dde00640 1 --2- 172.21.15.153:0/2310998308 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fc4c406c450 0x7fc4c406e910 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc4d0005fd0 tx=0x7fc4d0005850 comp rx=0 tx=0).ready entity=mgr.14221 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:05:03.928 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:03.928+0000 7fc4d6a00640 1 -- 172.21.15.153:0/2310998308 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fc4cc0784e0 con 0x7fc4e01034c0 2024-03-21T10:05:04.131 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:04.131+0000 7fc4e4e00640 1 -- 172.21.15.153:0/2310998308 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc4e010b1b0 con 0x7fc4e01034c0 2024-03-21T10:05:04.132 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:04.131+0000 7fc4d6a00640 1 -- 172.21.15.153:0/2310998308 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v47) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc4cc041020 con 0x7fc4e01034c0 2024-03-21T10:05:04.132 INFO:teuthology.orchestra.run.smithi153.stdout:pg_num: 1 2024-03-21T10:05:04.135 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:04.134+0000 7fc4e4e00640 1 -- 172.21.15.153:0/2310998308 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fc4c406c450 msgr2=0x7fc4c406e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:05:04.135 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:04.134+0000 7fc4e4e00640 1 --2- 172.21.15.153:0/2310998308 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fc4c406c450 0x7fc4c406e910 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc4d0005fd0 tx=0x7fc4d0005850 comp rx=0 tx=0).stop 2024-03-21T10:05:04.135 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:04.134+0000 7fc4e4e00640 1 -- 172.21.15.153:0/2310998308 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc4e01034c0 msgr2=0x7fc4e01980c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:05:04.135 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:04.134+0000 7fc4e4e00640 1 --2- 172.21.15.153:0/2310998308 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc4e01034c0 0x7fc4e01980c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc036040 tx=0x7fc4cc004f00 comp rx=0 tx=0).stop 2024-03-21T10:05:04.135 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:04.134+0000 7fc4e4e00640 1 -- 172.21.15.153:0/2310998308 shutdown_connections 2024-03-21T10:05:04.135 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:04.134+0000 7fc4e4e00640 1 --2- 172.21.15.153:0/2310998308 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fc4c406c450 0x7fc4c406e910 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:05:04.135 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:04.134+0000 7fc4e4e00640 1 --2- 172.21.15.153:0/2310998308 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc4e01034c0 0x7fc4e01980c0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:05:04.135 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:04.134+0000 7fc4e4e00640 1 --2- 172.21.15.153:0/2310998308 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc4e01022d0 0x7fc4e0197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:05:04.135 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:04.134+0000 7fc4e4e00640 1 -- 172.21.15.153:0/2310998308 >> 172.21.15.153:0/2310998308 conn(0x7fc4e00fd880 msgr2=0x7fc4e01066f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:05:04.135 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:04.134+0000 7fc4e4e00640 1 -- 172.21.15.153:0/2310998308 shutdown_connections 2024-03-21T10:05:04.135 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:04.134+0000 7fc4e4e00640 1 -- 172.21.15.153:0/2310998308 wait complete. 2024-03-21T10:05:04.152 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-03-21T10:05:04.349 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:04.348+0000 7fb99cc00640 1 -- 172.21.15.153:0/1915568957 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb998101290 msgr2=0x7fb998103700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:05:04.349 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:04.348+0000 7fb99cc00640 1 --2- 172.21.15.153:0/1915568957 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb998101290 0x7fb998103700 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb984009880 tx=0x7fb98402bce0 comp rx=0 tx=0).stop 2024-03-21T10:05:04.349 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:04.348+0000 7fb99cc00640 1 -- 172.21.15.153:0/1915568957 shutdown_connections 2024-03-21T10:05:04.349 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:04.348+0000 7fb99cc00640 1 --2- 172.21.15.153:0/1915568957 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb998101290 0x7fb998103700 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:05:04.350 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:04.348+0000 7fb99cc00640 1 --2- 172.21.15.153:0/1915568957 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb9980fe920 0x7fb998100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:05:04.350 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:04.348+0000 7fb99cc00640 1 -- 172.21.15.153:0/1915568957 >> 172.21.15.153:0/1915568957 conn(0x7fb9980fa520 msgr2=0x7fb9980fc960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:05:04.350 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:04.348+0000 7fb99cc00640 1 -- 172.21.15.153:0/1915568957 shutdown_connections 2024-03-21T10:05:04.350 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:04.348+0000 7fb99cc00640 1 -- 172.21.15.153:0/1915568957 wait complete. 2024-03-21T10:05:04.352 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:04.349+0000 7fb99cc00640 1 Processor -- start 2024-03-21T10:05:04.352 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:04.349+0000 7fb99cc00640 1 -- start start 2024-03-21T10:05:04.352 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:04.350+0000 7fb99cc00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb9980fe920 0x7fb99819c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:05:04.352 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:04.350+0000 7fb99cc00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb998101290 0x7fb99819c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:05:04.352 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:04.350+0000 7fb99cc00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb99819cb60 con 0x7fb9980fe920 2024-03-21T10:05:04.353 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:04.350+0000 7fb99cc00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb99819ccd0 con 0x7fb998101290 2024-03-21T10:05:04.353 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:04.350+0000 7fb995e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb9980fe920 0x7fb99819c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:05:04.353 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:04.350+0000 7fb995e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb9980fe920 0x7fb99819c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:55008/0 (socket says 172.21.15.153:55008) 2024-03-21T10:05:04.353 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:04.350+0000 7fb995e00640 1 -- 172.21.15.153:0/3498530486 learned_addr learned my addr 172.21.15.153:0/3498530486 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:05:04.353 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:04.350+0000 7fb995400640 1 --2- 172.21.15.153:0/3498530486 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb998101290 0x7fb99819c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:05:04.353 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:04.350+0000 7fb995e00640 1 -- 172.21.15.153:0/3498530486 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb998101290 msgr2=0x7fb99819c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:05:04.353 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:04.350+0000 7fb995e00640 1 --2- 172.21.15.153:0/3498530486 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb998101290 0x7fb99819c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:05:04.353 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:04.351+0000 7fb995e00640 1 -- 172.21.15.153:0/3498530486 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb984009560 con 0x7fb9980fe920 2024-03-21T10:05:04.353 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:04.351+0000 7fb995e00640 1 --2- 172.21.15.153:0/3498530486 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb9980fe920 0x7fb99819c050 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb98800b420 tx=0x7fb98800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:05:04.353 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:04.351+0000 7fb98ea00640 1 -- 172.21.15.153:0/3498530486 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb988011800 con 0x7fb9980fe920 2024-03-21T10:05:04.353 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:04.351+0000 7fb99cc00640 1 -- 172.21.15.153:0/3498530486 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9981a1760 con 0x7fb9980fe920 2024-03-21T10:05:04.354 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:04.351+0000 7fb98ea00640 1 -- 172.21.15.153:0/3498530486 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fb988011e10 con 0x7fb9980fe920 2024-03-21T10:05:04.354 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:04.351+0000 7fb98ea00640 1 -- 172.21.15.153:0/3498530486 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb98800f2e0 con 0x7fb9980fe920 2024-03-21T10:05:04.354 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:04.351+0000 7fb99cc00640 1 -- 172.21.15.153:0/3498530486 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9981a1d40 con 0x7fb9980fe920 2024-03-21T10:05:04.355 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:04.352+0000 7fb99cc00640 1 -- 172.21.15.153:0/3498530486 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb998196070 con 0x7fb9980fe920 2024-03-21T10:05:04.355 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:04.353+0000 7fb98ea00640 1 -- 172.21.15.153:0/3498530486 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fb988010380 con 0x7fb9980fe920 2024-03-21T10:05:04.355 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:04.354+0000 7fb98ea00640 1 --2- 172.21.15.153:0/3498530486 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fb97c06c450 0x7fb97c06e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:05:04.355 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:04.354+0000 7fb98ea00640 1 -- 172.21.15.153:0/3498530486 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fb98808b300 con 0x7fb9980fe920 2024-03-21T10:05:04.355 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:04.354+0000 7fb995400640 1 --2- 172.21.15.153:0/3498530486 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fb97c06c450 0x7fb97c06e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:05:04.355 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:04.355+0000 7fb995400640 1 --2- 172.21.15.153:0/3498530486 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fb97c06c450 0x7fb97c06e910 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb99819d560 tx=0x7fb984000bc0 comp rx=0 tx=0).ready entity=mgr.14221 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:05:04.359 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:04.359+0000 7fb98ea00640 1 -- 172.21.15.153:0/3498530486 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fb9880559e0 con 0x7fb9980fe920 2024-03-21T10:05:04.587 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:04.586+0000 7fb99cc00640 1 -- 172.21.15.153:0/3498530486 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7fb99802c6b0 con 0x7fb9980fe920 2024-03-21T10:05:04.588 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:04.588+0000 7fb98ea00640 1 -- 172.21.15.153:0/3498530486 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v47) v1 ==== 107+0+11 (secure 0 0 0) 0x7fb988059000 con 0x7fb9980fe920 2024-03-21T10:05:04.588 INFO:teuthology.orchestra.run.smithi153.stdout:pg_num: 32 2024-03-21T10:05:04.591 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:04.591+0000 7fb99cc00640 1 -- 172.21.15.153:0/3498530486 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fb97c06c450 msgr2=0x7fb97c06e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:05:04.591 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:04.591+0000 7fb99cc00640 1 --2- 172.21.15.153:0/3498530486 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fb97c06c450 0x7fb97c06e910 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb99819d560 tx=0x7fb984000bc0 comp rx=0 tx=0).stop 2024-03-21T10:05:04.592 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:04.591+0000 7fb99cc00640 1 -- 172.21.15.153:0/3498530486 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb9980fe920 msgr2=0x7fb99819c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:05:04.592 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:04.591+0000 7fb99cc00640 1 --2- 172.21.15.153:0/3498530486 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb9980fe920 0x7fb99819c050 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb98800b420 tx=0x7fb98800b8e0 comp rx=0 tx=0).stop 2024-03-21T10:05:04.592 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:04.591+0000 7fb99cc00640 1 -- 172.21.15.153:0/3498530486 shutdown_connections 2024-03-21T10:05:04.592 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:04.591+0000 7fb99cc00640 1 --2- 172.21.15.153:0/3498530486 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fb97c06c450 0x7fb97c06e910 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:05:04.592 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:04.591+0000 7fb99cc00640 1 --2- 172.21.15.153:0/3498530486 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb998101290 0x7fb99819c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:05:04.592 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:04.591+0000 7fb99cc00640 1 --2- 172.21.15.153:0/3498530486 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb9980fe920 0x7fb99819c050 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:05:04.592 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:04.591+0000 7fb99cc00640 1 -- 172.21.15.153:0/3498530486 >> 172.21.15.153:0/3498530486 conn(0x7fb9980fa520 msgr2=0x7fb9980fc960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:05:04.592 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:04.591+0000 7fb99cc00640 1 -- 172.21.15.153:0/3498530486 shutdown_connections 2024-03-21T10:05:04.592 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:04.591+0000 7fb99cc00640 1 -- 172.21.15.153:0/3498530486 wait complete. 2024-03-21T10:05:04.610 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-03-21T10:05:04.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:05:04 smithi174 ceph-mon[37496]: from='client.? 172.21.15.153:0/102602282' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T10:05:04.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:05:04 smithi174 ceph-mon[37496]: from='client.? 172.21.15.153:0/2310998308' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T10:05:04.803 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:04.802+0000 7f5b1e000640 1 -- 172.21.15.153:0/2018026960 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5b181034b0 msgr2=0x7f5b18103930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:05:04.803 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:04.802+0000 7f5b1e000640 1 --2- 172.21.15.153:0/2018026960 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5b181034b0 0x7f5b18103930 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f5b04009880 tx=0x7f5b0402bcf0 comp rx=0 tx=0).stop 2024-03-21T10:05:04.803 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:04.802+0000 7f5b1e000640 1 -- 172.21.15.153:0/2018026960 shutdown_connections 2024-03-21T10:05:04.803 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:04.802+0000 7f5b1e000640 1 --2- 172.21.15.153:0/2018026960 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5b181034b0 0x7f5b18103930 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:05:04.803 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:04.802+0000 7f5b1e000640 1 --2- 172.21.15.153:0/2018026960 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b181022c0 0x7f5b181026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:05:04.803 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:04.802+0000 7f5b1e000640 1 -- 172.21.15.153:0/2018026960 >> 172.21.15.153:0/2018026960 conn(0x7f5b180fd810 msgr2=0x7f5b180ffc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:05:04.803 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:04.803+0000 7f5b1e000640 1 -- 172.21.15.153:0/2018026960 shutdown_connections 2024-03-21T10:05:04.804 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:04.803+0000 7f5b1e000640 1 -- 172.21.15.153:0/2018026960 wait complete. 2024-03-21T10:05:04.805 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:04.803+0000 7f5b1e000640 1 Processor -- start 2024-03-21T10:05:04.805 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:04.804+0000 7f5b1e000640 1 -- start start 2024-03-21T10:05:04.805 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:04.804+0000 7f5b1e000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b181022c0 0x7f5b18197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:05:04.805 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:04.804+0000 7f5b1e000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5b181034b0 0x7f5b18197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:05:04.805 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:04.804+0000 7f5b1e000640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b18198530 con 0x7f5b181034b0 2024-03-21T10:05:04.805 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:04.804+0000 7f5b1e000640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b181986a0 con 0x7f5b181022c0 2024-03-21T10:05:04.805 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:04.804+0000 7f5b16a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5b181034b0 0x7f5b18197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:05:04.805 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:04.804+0000 7f5b16a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5b181034b0 0x7f5b18197ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:55016/0 (socket says 172.21.15.153:55016) 2024-03-21T10:05:04.805 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:04.804+0000 7f5b16a00640 1 -- 172.21.15.153:0/3085812993 learned_addr learned my addr 172.21.15.153:0/3085812993 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:05:04.806 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:04.805+0000 7f5b17400640 1 --2- 172.21.15.153:0/3085812993 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b181022c0 0x7f5b18197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:05:04.806 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:04.805+0000 7f5b16a00640 1 -- 172.21.15.153:0/3085812993 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b181022c0 msgr2=0x7f5b18197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:05:04.806 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:04.805+0000 7f5b16a00640 1 --2- 172.21.15.153:0/3085812993 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b181022c0 0x7f5b18197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:05:04.806 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:04.805+0000 7f5b16a00640 1 -- 172.21.15.153:0/3085812993 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b04009560 con 0x7f5b181034b0 2024-03-21T10:05:04.806 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:04.805+0000 7f5b16a00640 1 --2- 172.21.15.153:0/3085812993 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5b181034b0 0x7f5b18197ff0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f5b04036040 tx=0x7f5b04004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:05:04.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:05:04 smithi153 ceph-mon[29117]: from='client.? 172.21.15.153:0/102602282' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T10:05:04.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:05:04 smithi153 ceph-mon[29117]: from='client.? 172.21.15.153:0/2310998308' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T10:05:04.807 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:04.806+0000 7f5b0fe00640 1 -- 172.21.15.153:0/3085812993 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5b040051d0 con 0x7f5b181034b0 2024-03-21T10:05:04.807 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:04.806+0000 7f5b0fe00640 1 -- 172.21.15.153:0/3085812993 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f5b04004aa0 con 0x7f5b181034b0 2024-03-21T10:05:04.807 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:04.806+0000 7f5b0fe00640 1 -- 172.21.15.153:0/3085812993 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5b04034670 con 0x7f5b181034b0 2024-03-21T10:05:04.807 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:04.806+0000 7f5b1e000640 1 -- 172.21.15.153:0/3085812993 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b1819d110 con 0x7f5b181034b0 2024-03-21T10:05:04.807 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:04.806+0000 7f5b1e000640 1 -- 172.21.15.153:0/3085812993 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b1819d5f0 con 0x7f5b181034b0 2024-03-21T10:05:04.808 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:04.807+0000 7f5b1e000640 1 -- 172.21.15.153:0/3085812993 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b1810aea0 con 0x7f5b181034b0 2024-03-21T10:05:04.810 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:04.810+0000 7f5b0fe00640 1 -- 172.21.15.153:0/3085812993 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f5b040035f0 con 0x7f5b181034b0 2024-03-21T10:05:04.811 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:04.811+0000 7f5b0fe00640 1 --2- 172.21.15.153:0/3085812993 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f5afc06c450 0x7f5afc06e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:05:04.811 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:04.811+0000 7f5b0fe00640 1 -- 172.21.15.153:0/3085812993 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f5b040adfe0 con 0x7f5b181034b0 2024-03-21T10:05:04.811 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:04.811+0000 7f5b17400640 1 --2- 172.21.15.153:0/3085812993 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f5afc06c450 0x7f5afc06e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:05:04.812 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:04.812+0000 7f5b17400640 1 --2- 172.21.15.153:0/3085812993 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f5afc06c450 0x7f5afc06e910 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5b080058c0 tx=0x7f5b08005850 comp rx=0 tx=0).ready entity=mgr.14221 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:05:04.816 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:04.816+0000 7f5b0fe00640 1 -- 172.21.15.153:0/3085812993 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f5b040786e0 con 0x7f5b181034b0 2024-03-21T10:05:05.008 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:05.008+0000 7f5b1e000640 1 -- 172.21.15.153:0/3085812993 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7f5b1819d920 con 0x7f5b181034b0 2024-03-21T10:05:05.009 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:05.009+0000 7f5b0fe00640 1 -- 172.21.15.153:0/3085812993 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v47) v1 ==== 107+0+11 (secure 0 0 0) 0x7f5b0407bd00 con 0x7f5b181034b0 2024-03-21T10:05:05.009 INFO:teuthology.orchestra.run.smithi153.stdout:pg_num: 32 2024-03-21T10:05:05.012 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:05.011+0000 7f5b1e000640 1 -- 172.21.15.153:0/3085812993 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f5afc06c450 msgr2=0x7f5afc06e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:05:05.012 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:05.011+0000 7f5b1e000640 1 --2- 172.21.15.153:0/3085812993 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f5afc06c450 0x7f5afc06e910 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5b080058c0 tx=0x7f5b08005850 comp rx=0 tx=0).stop 2024-03-21T10:05:05.012 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:05.011+0000 7f5b1e000640 1 -- 172.21.15.153:0/3085812993 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5b181034b0 msgr2=0x7f5b18197ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:05:05.012 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:05.011+0000 7f5b1e000640 1 --2- 172.21.15.153:0/3085812993 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5b181034b0 0x7f5b18197ff0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f5b04036040 tx=0x7f5b04004270 comp rx=0 tx=0).stop 2024-03-21T10:05:05.012 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:05.011+0000 7f5b1e000640 1 -- 172.21.15.153:0/3085812993 shutdown_connections 2024-03-21T10:05:05.012 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:05.011+0000 7f5b1e000640 1 --2- 172.21.15.153:0/3085812993 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f5afc06c450 0x7f5afc06e910 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:05:05.012 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:05.011+0000 7f5b1e000640 1 --2- 172.21.15.153:0/3085812993 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5b181034b0 0x7f5b18197ff0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:05:05.012 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:05.011+0000 7f5b1e000640 1 --2- 172.21.15.153:0/3085812993 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b181022c0 0x7f5b18197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:05:05.012 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:05.011+0000 7f5b1e000640 1 -- 172.21.15.153:0/3085812993 >> 172.21.15.153:0/3085812993 conn(0x7f5b180fd810 msgr2=0x7f5b181066e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:05:05.013 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:05.011+0000 7f5b1e000640 1 -- 172.21.15.153:0/3085812993 shutdown_connections 2024-03-21T10:05:05.013 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:05.012+0000 7f5b1e000640 1 -- 172.21.15.153:0/3085812993 wait complete. 2024-03-21T10:05:05.031 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T10:05:05.211 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:05.210+0000 7fdae8e00640 1 -- 172.21.15.153:0/425407913 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdae41034c0 msgr2=0x7fdae4103940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:05:05.211 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:05.210+0000 7fdae8e00640 1 --2- 172.21.15.153:0/425407913 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdae41034c0 0x7fdae4103940 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fdad0009880 tx=0x7fdad002bce0 comp rx=0 tx=0).stop 2024-03-21T10:05:05.211 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:05.210+0000 7fdae8e00640 1 -- 172.21.15.153:0/425407913 shutdown_connections 2024-03-21T10:05:05.211 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:05.210+0000 7fdae8e00640 1 --2- 172.21.15.153:0/425407913 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdae41034c0 0x7fdae4103940 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:05:05.211 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:05.210+0000 7fdae8e00640 1 --2- 172.21.15.153:0/425407913 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdae41022d0 0x7fdae41026f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:05:05.211 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:05.210+0000 7fdae8e00640 1 -- 172.21.15.153:0/425407913 >> 172.21.15.153:0/425407913 conn(0x7fdae40fd820 msgr2=0x7fdae40ffc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:05:05.211 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:05.210+0000 7fdae8e00640 1 -- 172.21.15.153:0/425407913 shutdown_connections 2024-03-21T10:05:05.211 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:05.210+0000 7fdae8e00640 1 -- 172.21.15.153:0/425407913 wait complete. 2024-03-21T10:05:05.213 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:05.211+0000 7fdae8e00640 1 Processor -- start 2024-03-21T10:05:05.213 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:05.211+0000 7fdae8e00640 1 -- start start 2024-03-21T10:05:05.213 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:05.212+0000 7fdae8e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdae41022d0 0x7fdae4197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:05:05.213 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:05.212+0000 7fdae8e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdae41034c0 0x7fdae4197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:05:05.213 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:05.212+0000 7fdae8e00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdae41985a0 con 0x7fdae41034c0 2024-03-21T10:05:05.214 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:05.212+0000 7fdae8e00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdae4198710 con 0x7fdae41022d0 2024-03-21T10:05:05.214 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:05.212+0000 7fdae1400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdae41034c0 0x7fdae4197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:05:05.214 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:05.212+0000 7fdae1400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdae41034c0 0x7fdae4197fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:55040/0 (socket says 172.21.15.153:55040) 2024-03-21T10:05:05.214 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:05.212+0000 7fdae1400640 1 -- 172.21.15.153:0/3506947600 learned_addr learned my addr 172.21.15.153:0/3506947600 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:05:05.214 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:05.212+0000 7fdae1e00640 1 --2- 172.21.15.153:0/3506947600 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdae41022d0 0x7fdae4197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:05:05.214 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:05.212+0000 7fdae1400640 1 -- 172.21.15.153:0/3506947600 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdae41022d0 msgr2=0x7fdae4197a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:05:05.214 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:05.212+0000 7fdae1400640 1 --2- 172.21.15.153:0/3506947600 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdae41022d0 0x7fdae4197a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:05:05.214 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:05.212+0000 7fdae1400640 1 -- 172.21.15.153:0/3506947600 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdad0009560 con 0x7fdae41034c0 2024-03-21T10:05:05.214 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:05.213+0000 7fdae1400640 1 --2- 172.21.15.153:0/3506947600 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdae41034c0 0x7fdae4197fd0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fdad0009430 tx=0x7fdad002cec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:05:05.214 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:05.213+0000 7fdadaa00640 1 -- 172.21.15.153:0/3506947600 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdad0004520 con 0x7fdae41034c0 2024-03-21T10:05:05.214 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:05.213+0000 7fdae8e00640 1 -- 172.21.15.153:0/3506947600 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdae419d140 con 0x7fdae41034c0 2024-03-21T10:05:05.214 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:05.213+0000 7fdadaa00640 1 -- 172.21.15.153:0/3506947600 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fdad00054b0 con 0x7fdae41034c0 2024-03-21T10:05:05.214 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:05.213+0000 7fdadaa00640 1 -- 172.21.15.153:0/3506947600 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdad0034440 con 0x7fdae41034c0 2024-03-21T10:05:05.214 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:05.213+0000 7fdae8e00640 1 -- 172.21.15.153:0/3506947600 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdae419d640 con 0x7fdae41034c0 2024-03-21T10:05:05.215 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:05.214+0000 7fdae8e00640 1 -- 172.21.15.153:0/3506947600 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdae410b020 con 0x7fdae41034c0 2024-03-21T10:05:05.215 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:05.215+0000 7fdadaa00640 1 -- 172.21.15.153:0/3506947600 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fdad0005000 con 0x7fdae41034c0 2024-03-21T10:05:05.217 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:05.216+0000 7fdadaa00640 1 --2- 172.21.15.153:0/3506947600 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fdac806c510 0x7fdac806e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:05:05.217 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:05.216+0000 7fdadaa00640 1 -- 172.21.15.153:0/3506947600 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fdad00adfe0 con 0x7fdae41034c0 2024-03-21T10:05:05.217 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:05.217+0000 7fdae1e00640 1 --2- 172.21.15.153:0/3506947600 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fdac806c510 0x7fdac806e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:05:05.217 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:05.217+0000 7fdae1e00640 1 --2- 172.21.15.153:0/3506947600 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fdac806c510 0x7fdac806e9d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fdad40058c0 tx=0x7fdad4005c80 comp rx=0 tx=0).ready entity=mgr.14221 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:05:05.220 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:05.220+0000 7fdadaa00640 1 -- 172.21.15.153:0/3506947600 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fdad0078720 con 0x7fdae41034c0 2024-03-21T10:05:05.454 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:05.452+0000 7fdae8e00640 1 -- 172.21.15.153:0/3506947600 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdae419dce0 con 0x7fdae41034c0 2024-03-21T10:05:05.454 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:05.453+0000 7fdadaa00640 1 -- 172.21.15.153:0/3506947600 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4776 (secure 0 0 0) 0x7fdad007bd40 con 0x7fdae41034c0 2024-03-21T10:05:05.454 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:05:05.455 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":17,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T10:04:27.086576+0000","modified":"2024-03-21T10:04:49.033187+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14484,"mds_1":24273},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14484":{"gid":14484,"name":"cephfs.smithi153.iuyluj","rank":0,"incarnation":13,"state":"up:active","state_seq":6,"addr":"172.21.15.153:6829/866138999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":866138999},{"type":"v1","addr":"172.21.15.153:6829","nonce":866138999}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14494":{"gid":14494,"name":"cephfs.smithi153.zipckr","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6827/4033622615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":4033622615},{"type":"v1","addr":"172.21.15.153:6827","nonce":4033622615}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24273":{"gid":24273,"name":"cephfs.smithi174.yyimst","rank":1,"incarnation":7,"state":"up:active","state_seq":5,"addr":"172.21.15.174:6825/1776947618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":1776947618},{"type":"v1","addr":"172.21.15.174:6825","nonce":1776947618}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24285":{"gid":24285,"name":"cephfs.smithi174.nhozag","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":2,"addr":"172.21.15.174:6827/806095873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":806095873},{"type":"v1","addr":"172.21.15.174:6827","nonce":806095873}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-21T10:05:05.455 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 17 2024-03-21T10:05:05.456 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:05.455+0000 7fdae8e00640 1 -- 172.21.15.153:0/3506947600 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fdac806c510 msgr2=0x7fdac806e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:05:05.456 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:05.455+0000 7fdae8e00640 1 --2- 172.21.15.153:0/3506947600 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fdac806c510 0x7fdac806e9d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fdad40058c0 tx=0x7fdad4005c80 comp rx=0 tx=0).stop 2024-03-21T10:05:05.456 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:05.455+0000 7fdae8e00640 1 -- 172.21.15.153:0/3506947600 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdae41034c0 msgr2=0x7fdae4197fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:05:05.456 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:05.455+0000 7fdae8e00640 1 --2- 172.21.15.153:0/3506947600 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdae41034c0 0x7fdae4197fd0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fdad0009430 tx=0x7fdad002cec0 comp rx=0 tx=0).stop 2024-03-21T10:05:05.456 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:05.456+0000 7fdae8e00640 1 -- 172.21.15.153:0/3506947600 shutdown_connections 2024-03-21T10:05:05.457 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:05.456+0000 7fdae8e00640 1 --2- 172.21.15.153:0/3506947600 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fdac806c510 0x7fdac806e9d0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:05:05.457 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:05.456+0000 7fdae8e00640 1 --2- 172.21.15.153:0/3506947600 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdae41034c0 0x7fdae4197fd0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:05:05.457 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:05.456+0000 7fdae8e00640 1 --2- 172.21.15.153:0/3506947600 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdae41022d0 0x7fdae4197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:05:05.457 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:05.456+0000 7fdae8e00640 1 -- 172.21.15.153:0/3506947600 >> 172.21.15.153:0/3506947600 conn(0x7fdae40fd820 msgr2=0x7fdae41066f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:05:05.457 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:05.456+0000 7fdae8e00640 1 -- 172.21.15.153:0/3506947600 shutdown_connections 2024-03-21T10:05:05.457 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:05.456+0000 7fdae8e00640 1 -- 172.21.15.153:0/3506947600 wait complete. 2024-03-21T10:05:05.473 DEBUG:tasks.fs:fs fscid=1,name=cephfs state = {'epoch': 17, 'max_mds': 2, 'flags': 50} 2024-03-21T10:05:05.473 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-03-21T10:05:05.500 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-03-21T10:05:05.500 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.1': {}} 2024-03-21T10:05:05.500 INFO:tasks.ceph_fuse:client.0 config is {} 2024-03-21T10:05:05.500 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-21T10:05:05.500 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-21T10:05:05.501 INFO:tasks.ceph_fuse:client.1 config is {} 2024-03-21T10:05:05.501 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-21T10:05:05.501 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-03-21T10:05:05.501 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T10:05:05.501 DEBUG:teuthology.orchestra.run.smithi153:> ip netns list 2024-03-21T10:05:05.548 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T10:05:05.548 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip link delete ceph-brx 2024-03-21T10:05:05.669 INFO:teuthology.orchestra.run.smithi153.stderr:Cannot find device "ceph-brx" 2024-03-21T10:05:05.672 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-21T10:05:05.673 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T10:05:05.673 DEBUG:teuthology.orchestra.run.smithi174:> ip netns list 2024-03-21T10:05:05.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:05:05 smithi174 ceph-mon[37496]: pgmap v235: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s rd, 3 op/s 2024-03-21T10:05:05.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:05:05 smithi174 ceph-mon[37496]: from='client.? 172.21.15.153:0/3498530486' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-21T10:05:05.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:05:05 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:05:05.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:05:05 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:05:05.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:05:05 smithi174 ceph-mon[37496]: from='client.? 172.21.15.153:0/3085812993' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-21T10:05:05.733 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T10:05:05.733 DEBUG:teuthology.orchestra.run.smithi174:> sudo ip link delete ceph-brx 2024-03-21T10:05:05.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:05:05 smithi153 ceph-mon[29117]: pgmap v235: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s rd, 3 op/s 2024-03-21T10:05:05.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:05:05 smithi153 ceph-mon[29117]: from='client.? 172.21.15.153:0/3498530486' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-21T10:05:05.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:05:05 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:05:05.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:05:05 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:05:05.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:05:05 smithi153 ceph-mon[29117]: from='client.? 172.21.15.153:0/3085812993' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-21T10:05:05.852 INFO:teuthology.orchestra.run.smithi174.stderr:Cannot find device "ceph-brx" 2024-03-21T10:05:05.855 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-21T10:05:05.856 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-03-21T10:05:05.856 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 070ed3e4-e769-11ee-95cd-87774f69a715 -- ceph fs ls 2024-03-21T10:05:06.282 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi153/config 2024-03-21T10:05:06.684 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:05:06 smithi153 ceph-mon[29117]: from='client.? 172.21.15.153:0/3506947600' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:05:06.684 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:05:06 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:05:06.684 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:05:06 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:05:06.684 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:05:06 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:05:06.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:05:06 smithi174 ceph-mon[37496]: from='client.? 172.21.15.153:0/3506947600' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:05:06.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:05:06 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:05:06.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:05:06 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:05:06.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:05:06 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:05:07.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:05:07 smithi153 ceph-mon[29117]: pgmap v236: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-21T10:05:07.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:05:07 smithi174 ceph-mon[37496]: pgmap v236: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-21T10:05:08.429 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:08.427+0000 7fe72f600700 1 -- 172.21.15.153:0/3254968764 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe728100b50 msgr2=0x7fe728100fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:05:08.430 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:08.427+0000 7fe72f600700 1 --2- 172.21.15.153:0/3254968764 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe728100b50 0x7fe728100fd0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fe718009910 tx=0x7fe718009c20 comp rx=0 tx=0).stop 2024-03-21T10:05:08.430 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:08.427+0000 7fe72f600700 1 -- 172.21.15.153:0/3254968764 shutdown_connections 2024-03-21T10:05:08.430 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:08.427+0000 7fe72f600700 1 --2- 172.21.15.153:0/3254968764 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe728100b50 0x7fe728100fd0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:05:08.430 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:08.427+0000 7fe72f600700 1 --2- 172.21.15.153:0/3254968764 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe728106c00 0x7fe728106fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:05:08.430 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:08.427+0000 7fe72f600700 1 -- 172.21.15.153:0/3254968764 >> 172.21.15.153:0/3254968764 conn(0x7fe7280fc640 msgr2=0x7fe7280fea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:05:08.430 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:08.427+0000 7fe72f600700 1 -- 172.21.15.153:0/3254968764 shutdown_connections 2024-03-21T10:05:08.430 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:08.427+0000 7fe72f600700 1 -- 172.21.15.153:0/3254968764 wait complete. 2024-03-21T10:05:08.430 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:08.428+0000 7fe72f600700 1 Processor -- start 2024-03-21T10:05:08.430 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:08.428+0000 7fe72f600700 1 -- start start 2024-03-21T10:05:08.432 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:08.428+0000 7fe72f600700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe728100b50 0x7fe72819e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:05:08.432 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:08.428+0000 7fe72f600700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe728106c00 0x7fe72819eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:05:08.432 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:08.428+0000 7fe72f600700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe72819f230 con 0x7fe728106c00 2024-03-21T10:05:08.432 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:08.428+0000 7fe72f600700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7281986e0 con 0x7fe728100b50 2024-03-21T10:05:08.432 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:08.429+0000 7fe727e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe728106c00 0x7fe72819eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:05:08.432 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:08.429+0000 7fe727e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe728106c00 0x7fe72819eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:55072/0 (socket says 172.21.15.153:55072) 2024-03-21T10:05:08.432 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:08.429+0000 7fe727e00700 1 -- 172.21.15.153:0/2840925961 learned_addr learned my addr 172.21.15.153:0/2840925961 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:05:08.432 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:08.429+0000 7fe72cc00700 1 --2- 172.21.15.153:0/2840925961 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe728100b50 0x7fe72819e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:05:08.432 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:08.429+0000 7fe727e00700 1 -- 172.21.15.153:0/2840925961 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe728100b50 msgr2=0x7fe72819e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:05:08.432 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:08.429+0000 7fe727e00700 1 --2- 172.21.15.153:0/2840925961 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe728100b50 0x7fe72819e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:05:08.432 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:08.429+0000 7fe727e00700 1 -- 172.21.15.153:0/2840925961 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7180095f0 con 0x7fe728106c00 2024-03-21T10:05:08.432 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:08.429+0000 7fe72cc00700 1 --2- 172.21.15.153:0/2840925961 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe728100b50 0x7fe72819e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:05:08.433 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:08.430+0000 7fe727e00700 1 --2- 172.21.15.153:0/2840925961 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe728106c00 0x7fe72819eba0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fe71800b5c0 tx=0x7fe71800be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:05:08.433 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:08.430+0000 7fe725600700 1 -- 172.21.15.153:0/2840925961 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe71801d070 con 0x7fe728106c00 2024-03-21T10:05:08.433 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:08.430+0000 7fe725600700 1 -- 172.21.15.153:0/2840925961 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fe718022470 con 0x7fe728106c00 2024-03-21T10:05:08.433 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:08.430+0000 7fe72f600700 1 -- 172.21.15.153:0/2840925961 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe728198960 con 0x7fe728106c00 2024-03-21T10:05:08.433 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:08.430+0000 7fe725600700 1 -- 172.21.15.153:0/2840925961 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe71800f670 con 0x7fe728106c00 2024-03-21T10:05:08.433 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:08.430+0000 7fe72f600700 1 -- 172.21.15.153:0/2840925961 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe728198e00 con 0x7fe728106c00 2024-03-21T10:05:08.433 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:08.431+0000 7fe72f600700 1 -- 172.21.15.153:0/2840925961 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe72804ea90 con 0x7fe728106c00 2024-03-21T10:05:08.434 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:08.432+0000 7fe725600700 1 -- 172.21.15.153:0/2840925961 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fe718004c30 con 0x7fe728106c00 2024-03-21T10:05:08.435 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:08.433+0000 7fe725600700 1 --2- 172.21.15.153:0/2840925961 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fe71006c4e0 0x7fe71006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:05:08.435 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:08.433+0000 7fe725600700 1 -- 172.21.15.153:0/2840925961 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fe71808dad0 con 0x7fe728106c00 2024-03-21T10:05:08.435 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:08.434+0000 7fe72cc00700 1 --2- 172.21.15.153:0/2840925961 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fe71006c4e0 0x7fe71006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:05:08.435 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:08.434+0000 7fe72cc00700 1 --2- 172.21.15.153:0/2840925961 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fe71006c4e0 0x7fe71006e9a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe71c005fd0 tx=0x7fe71c005dc0 comp rx=0 tx=0).ready entity=mgr.14221 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:05:08.438 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:08.437+0000 7fe725600700 1 -- 172.21.15.153:0/2840925961 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fe71805bcc0 con 0x7fe728106c00 2024-03-21T10:05:08.701 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:08.700+0000 7fe72f600700 1 -- 172.21.15.153:0/2840925961 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fe728199bf0 con 0x7fe728106c00 2024-03-21T10:05:08.702 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:08.700+0000 7fe725600700 1 -- 172.21.15.153:0/2840925961 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+83 (secure 0 0 0) 0x7fe718027020 con 0x7fe728106c00 2024-03-21T10:05:08.702 INFO:teuthology.orchestra.run.smithi153.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-03-21T10:05:08.705 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:08.703+0000 7fe72f600700 1 -- 172.21.15.153:0/2840925961 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fe71006c4e0 msgr2=0x7fe71006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:05:08.705 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:08.703+0000 7fe72f600700 1 --2- 172.21.15.153:0/2840925961 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fe71006c4e0 0x7fe71006e9a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe71c005fd0 tx=0x7fe71c005dc0 comp rx=0 tx=0).stop 2024-03-21T10:05:08.705 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:08.703+0000 7fe72f600700 1 -- 172.21.15.153:0/2840925961 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe728106c00 msgr2=0x7fe72819eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:05:08.705 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:08.703+0000 7fe72f600700 1 --2- 172.21.15.153:0/2840925961 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe728106c00 0x7fe72819eba0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fe71800b5c0 tx=0x7fe71800be00 comp rx=0 tx=0).stop 2024-03-21T10:05:08.706 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:08.704+0000 7fe72f600700 1 -- 172.21.15.153:0/2840925961 shutdown_connections 2024-03-21T10:05:08.706 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:08.704+0000 7fe72f600700 1 --2- 172.21.15.153:0/2840925961 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fe71006c4e0 0x7fe71006e9a0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:05:08.706 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:08.704+0000 7fe72f600700 1 --2- 172.21.15.153:0/2840925961 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe728100b50 0x7fe72819e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:05:08.706 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:08.704+0000 7fe72f600700 1 --2- 172.21.15.153:0/2840925961 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe728106c00 0x7fe72819eba0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:05:08.706 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:08.704+0000 7fe72f600700 1 -- 172.21.15.153:0/2840925961 >> 172.21.15.153:0/2840925961 conn(0x7fe7280fc640 msgr2=0x7fe7280fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:05:08.706 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:08.704+0000 7fe72f600700 1 -- 172.21.15.153:0/2840925961 shutdown_connections 2024-03-21T10:05:08.706 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:08.704+0000 7fe72f600700 1 -- 172.21.15.153:0/2840925961 wait complete. 2024-03-21T10:05:09.557 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-21T10:05:09.557 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-21T10:05:09.557 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi153.front.sepia.ceph.com 2024-03-21T10:05:09.557 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-21T10:05:09.557 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-21T10:05:09.557 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-21T10:05:09.557 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-21T10:05:09.557 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-21T10:05:09.557 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-21T10:05:09.558 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T10:05:09.558 DEBUG:teuthology.orchestra.run.smithi153:> ip addr 2024-03-21T10:05:09.614 INFO:teuthology.orchestra.run.smithi153.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-21T10:05:09.614 INFO:teuthology.orchestra.run.smithi153.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-21T10:05:09.614 INFO:teuthology.orchestra.run.smithi153.stdout: inet 127.0.0.1/8 scope host lo 2024-03-21T10:05:09.614 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2024-03-21T10:05:09.615 INFO:teuthology.orchestra.run.smithi153.stdout: inet6 ::1/128 scope host 2024-03-21T10:05:09.615 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2024-03-21T10:05:09.615 INFO:teuthology.orchestra.run.smithi153.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-21T10:05:09.615 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether 0c:c4:7a:da:6a:b2 brd ff:ff:ff:ff:ff:ff 2024-03-21T10:05:09.615 INFO:teuthology.orchestra.run.smithi153.stdout: altname enp7s0f0 2024-03-21T10:05:09.615 INFO:teuthology.orchestra.run.smithi153.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-21T10:05:09.615 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether 0c:c4:7a:da:6a:b3 brd ff:ff:ff:ff:ff:ff 2024-03-21T10:05:09.615 INFO:teuthology.orchestra.run.smithi153.stdout: altname enp7s0f1 2024-03-21T10:05:09.615 INFO:teuthology.orchestra.run.smithi153.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-21T10:05:09.615 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether 0c:c4:7a:88:6d:ea brd ff:ff:ff:ff:ff:ff 2024-03-21T10:05:09.615 INFO:teuthology.orchestra.run.smithi153.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-21T10:05:09.615 INFO:teuthology.orchestra.run.smithi153.stdout: link/ether 0c:c4:7a:88:6d:eb brd ff:ff:ff:ff:ff:ff 2024-03-21T10:05:09.615 INFO:teuthology.orchestra.run.smithi153.stdout: inet 172.21.15.153/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-21T10:05:09.615 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft 42041sec preferred_lft 42041sec 2024-03-21T10:05:09.615 INFO:teuthology.orchestra.run.smithi153.stdout: inet6 fe80::ec4:7aff:fe88:6deb/64 scope link 2024-03-21T10:05:09.615 INFO:teuthology.orchestra.run.smithi153.stdout: valid_lft forever preferred_lft forever 2024-03-21T10:05:09.616 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-21T10:05:09.617 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T10:05:09.617 DEBUG:teuthology.orchestra.run.smithi153:> (cd / && exec bash -c ' 2024-03-21T10:05:09.617 DEBUG:teuthology.orchestra.run.smithi153:> set -e 2024-03-21T10:05:09.617 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip link add name ceph-brx type bridge 2024-03-21T10:05:09.617 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip addr flush dev ceph-brx 2024-03-21T10:05:09.617 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip link set ceph-brx up 2024-03-21T10:05:09.617 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-21T10:05:09.617 DEBUG:teuthology.orchestra.run.smithi153:> ') 2024-03-21T10:05:09.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:05:09 smithi153 ceph-mon[29117]: pgmap v237: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-21T10:05:09.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:05:09 smithi153 ceph-mon[29117]: from='client.? 172.21.15.153:0/2840925961' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-21T10:05:09.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:05:09 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:05:09.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:05:09 smithi174 ceph-mon[37496]: pgmap v237: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-21T10:05:09.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:05:09 smithi174 ceph-mon[37496]: from='client.? 172.21.15.153:0/2840925961' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-21T10:05:09.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:05:09 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:05:10.059 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T10:05:10.192 DEBUG:teuthology.orchestra.run.smithi153:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-21T10:05:10.281 INFO:teuthology.orchestra.run.smithi153.stdout:1 2024-03-21T10:05:10.285 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T10:05:10.285 DEBUG:teuthology.orchestra.run.smithi153:> route 2024-03-21T10:05:10.355 INFO:teuthology.orchestra.run.smithi153.stdout:Kernel IP routing table 2024-03-21T10:05:10.355 INFO:teuthology.orchestra.run.smithi153.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-21T10:05:10.355 INFO:teuthology.orchestra.run.smithi153.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-21T10:05:10.355 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-21T10:05:10.355 INFO:teuthology.orchestra.run.smithi153.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-21T10:05:10.355 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T10:05:10.356 DEBUG:teuthology.orchestra.run.smithi153:> (cd / && exec bash -c ' 2024-03-21T10:05:10.356 DEBUG:teuthology.orchestra.run.smithi153:> set -e 2024-03-21T10:05:10.356 DEBUG:teuthology.orchestra.run.smithi153:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-21T10:05:10.356 DEBUG:teuthology.orchestra.run.smithi153:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-21T10:05:10.356 DEBUG:teuthology.orchestra.run.smithi153:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-21T10:05:10.356 DEBUG:teuthology.orchestra.run.smithi153:> ') 2024-03-21T10:05:11.066 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T10:05:11.066 DEBUG:teuthology.orchestra.run.smithi153:> ip netns list 2024-03-21T10:05:11.121 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T10:05:11.121 DEBUG:teuthology.orchestra.run.smithi153:> ip netns list-id 2024-03-21T10:05:11.174 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T10:05:11.175 DEBUG:teuthology.orchestra.run.smithi153:> (cd / && exec bash -c ' 2024-03-21T10:05:11.175 DEBUG:teuthology.orchestra.run.smithi153:> set -e 2024-03-21T10:05:11.175 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-21T10:05:11.175 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-03-21T10:05:11.175 DEBUG:teuthology.orchestra.run.smithi153:> ') 2024-03-21T10:05:11.318 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-03-21T10:05:11.318 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T10:05:11.318 DEBUG:teuthology.orchestra.run.smithi153:> (cd / && exec bash -c ' 2024-03-21T10:05:11.318 DEBUG:teuthology.orchestra.run.smithi153:> set -e 2024-03-21T10:05:11.319 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-03-21T10:05:11.319 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-21T10:05:11.319 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-03-21T10:05:11.319 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-03-21T10:05:11.319 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-03-21T10:05:11.319 DEBUG:teuthology.orchestra.run.smithi153:> ') 2024-03-21T10:05:11.594 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T10:05:11.594 DEBUG:teuthology.orchestra.run.smithi153:> (cd / && exec bash -c ' 2024-03-21T10:05:11.594 DEBUG:teuthology.orchestra.run.smithi153:> set -e 2024-03-21T10:05:11.594 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip link set brx.0 up 2024-03-21T10:05:11.594 DEBUG:teuthology.orchestra.run.smithi153:> sudo ip link set dev brx.0 master ceph-brx 2024-03-21T10:05:11.594 DEBUG:teuthology.orchestra.run.smithi153:> ') 2024-03-21T10:05:11.760 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-03-21T10:05:11.760 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-21T10:05:11.760 DEBUG:teuthology.orchestra.run.smithi153:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-21T10:05:11.808 INFO:teuthology.orchestra.run.smithi153.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-21T10:05:11.808 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-21T10:05:11.809 DEBUG:teuthology.orchestra.run.smithi153:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-21T10:05:11.902 DEBUG:teuthology.orchestra.run.smithi153:> sudo modprobe fuse 2024-03-21T10:05:11.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:05:11 smithi174 ceph-mon[37496]: pgmap v238: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-21T10:05:11.982 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:05:11 smithi153 ceph-mon[29117]: pgmap v238: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-21T10:05:11.996 DEBUG:teuthology.orchestra.run.smithi153:> cat /proc/self/mounts | awk '{print $2}' 2024-03-21T10:05:12.045 INFO:teuthology.orchestra.run.smithi153.stdout:/proc 2024-03-21T10:05:12.046 INFO:teuthology.orchestra.run.smithi153.stdout:/sys 2024-03-21T10:05:12.046 INFO:teuthology.orchestra.run.smithi153.stdout:/dev 2024-03-21T10:05:12.046 INFO:teuthology.orchestra.run.smithi153.stdout:/sys/kernel/security 2024-03-21T10:05:12.046 INFO:teuthology.orchestra.run.smithi153.stdout:/dev/shm 2024-03-21T10:05:12.046 INFO:teuthology.orchestra.run.smithi153.stdout:/dev/pts 2024-03-21T10:05:12.046 INFO:teuthology.orchestra.run.smithi153.stdout:/run 2024-03-21T10:05:12.046 INFO:teuthology.orchestra.run.smithi153.stdout:/sys/fs/cgroup 2024-03-21T10:05:12.046 INFO:teuthology.orchestra.run.smithi153.stdout:/sys/fs/pstore 2024-03-21T10:05:12.046 INFO:teuthology.orchestra.run.smithi153.stdout:/sys/fs/bpf 2024-03-21T10:05:12.046 INFO:teuthology.orchestra.run.smithi153.stdout:/ 2024-03-21T10:05:12.046 INFO:teuthology.orchestra.run.smithi153.stdout:/sys/fs/selinux 2024-03-21T10:05:12.046 INFO:teuthology.orchestra.run.smithi153.stdout:/proc/sys/fs/binfmt_misc 2024-03-21T10:05:12.046 INFO:teuthology.orchestra.run.smithi153.stdout:/dev/mqueue 2024-03-21T10:05:12.047 INFO:teuthology.orchestra.run.smithi153.stdout:/dev/hugepages 2024-03-21T10:05:12.047 INFO:teuthology.orchestra.run.smithi153.stdout:/sys/kernel/debug 2024-03-21T10:05:12.047 INFO:teuthology.orchestra.run.smithi153.stdout:/sys/kernel/tracing 2024-03-21T10:05:12.047 INFO:teuthology.orchestra.run.smithi153.stdout:/sys/fs/fuse/connections 2024-03-21T10:05:12.047 INFO:teuthology.orchestra.run.smithi153.stdout:/sys/kernel/config 2024-03-21T10:05:12.047 INFO:teuthology.orchestra.run.smithi153.stdout:/run/credentials/systemd-sysctl.service 2024-03-21T10:05:12.047 INFO:teuthology.orchestra.run.smithi153.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-21T10:05:12.047 INFO:teuthology.orchestra.run.smithi153.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-21T10:05:12.047 INFO:teuthology.orchestra.run.smithi153.stdout:/var/lib/nfs/rpc_pipefs 2024-03-21T10:05:12.047 INFO:teuthology.orchestra.run.smithi153.stdout:/run/user/1000 2024-03-21T10:05:12.047 INFO:teuthology.orchestra.run.smithi153.stdout:/var/lib/ceph 2024-03-21T10:05:12.047 INFO:teuthology.orchestra.run.smithi153.stdout:/var/lib/containers/storage/overlay 2024-03-21T10:05:12.047 INFO:teuthology.orchestra.run.smithi153.stdout:/var/lib/containers/storage/overlay/1a78b7d0d52de441bd07a7f84ea584762d6c49eadf71a4cd0c027ce32a2e3b2e/merged 2024-03-21T10:05:12.047 INFO:teuthology.orchestra.run.smithi153.stdout:/var/lib/containers/storage/overlay/5f40359e14914d8a9089623a93271c40fa107a407da72364352c8ce94199e767/merged 2024-03-21T10:05:12.048 INFO:teuthology.orchestra.run.smithi153.stdout:/run/user/0 2024-03-21T10:05:12.048 INFO:teuthology.orchestra.run.smithi153.stdout:/proc/sys/fs/binfmt_misc 2024-03-21T10:05:12.048 INFO:teuthology.orchestra.run.smithi153.stdout:/var/lib/containers/storage/overlay/d8ba1e5482dfd182ad7f24fe413a20e2c554b13c1a52e876d82a2209e45c0214/merged 2024-03-21T10:05:12.048 INFO:teuthology.orchestra.run.smithi153.stdout:/var/lib/containers/storage/overlay/a7c55df1e4c1ae02b6c8d644cac9bcf18e17cdf29f7c7235f591553bee326974/merged 2024-03-21T10:05:12.048 INFO:teuthology.orchestra.run.smithi153.stdout:/var/lib/containers/storage/overlay/64d5fe09dfec2268b9fa72961c83d20c15f6b0556040d5901f5c7cdc8b482402/merged 2024-03-21T10:05:12.048 INFO:teuthology.orchestra.run.smithi153.stdout:/var/lib/containers/storage/overlay/4fc50842e6e8317e9201dde6cc8fa545a88cdc3f8e2bc09b19d33350aca695dc/merged 2024-03-21T10:05:12.048 INFO:teuthology.orchestra.run.smithi153.stdout:/var/lib/containers/storage/overlay/f03b1eb0bfdd91d26d0d2ddb817f11d9a4dbcc83f95c8a3f162de46b1f772c0f/merged 2024-03-21T10:05:12.048 INFO:teuthology.orchestra.run.smithi153.stdout:/var/lib/containers/storage/overlay/3c772ec53b68f6cd651582122b0222525ad0a03911264d545aa802855e5a5da2/merged 2024-03-21T10:05:12.048 INFO:teuthology.orchestra.run.smithi153.stdout:/var/lib/containers/storage/overlay/f832be57993da0590147aff14477907bcdc781c3e7cc23435520878018f3ad8a/merged 2024-03-21T10:05:12.048 INFO:teuthology.orchestra.run.smithi153.stdout:/var/lib/containers/storage/overlay/89ab59ea3ec111882e9c4e576aaf147ecda735a40f1736c7977509be437943b8/merged 2024-03-21T10:05:12.048 INFO:teuthology.orchestra.run.smithi153.stdout:/var/lib/containers/storage/overlay/c3e78be25694fee98c53ade1da856cd8426e9a12d1c3a3ed4ead86e5f3edae0f/merged 2024-03-21T10:05:12.048 INFO:teuthology.orchestra.run.smithi153.stdout:/var/lib/containers/storage/overlay/9d3fbbf86b47bc4cb9ccc07b6a1a70c06ff4e5d2958de5d65cc362f327ac433f/merged 2024-03-21T10:05:12.048 INFO:teuthology.orchestra.run.smithi153.stdout:/var/lib/containers/storage/overlay/5c6d917a2a132ac2e878f295988eb2bb4c89853bc9f493f23a8310c71f610bd5/merged 2024-03-21T10:05:12.048 INFO:teuthology.orchestra.run.smithi153.stdout:/run/netns 2024-03-21T10:05:12.049 INFO:teuthology.orchestra.run.smithi153.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-21T10:05:12.049 INFO:teuthology.orchestra.run.smithi153.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-21T10:05:12.049 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T10:05:12.050 DEBUG:teuthology.orchestra.run.smithi153:> ls /sys/fs/fuse/connections 2024-03-21T10:05:12.094 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-21T10:05:12.095 DEBUG:teuthology.orchestra.run.smithi153:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-03-21T10:05:12.141 DEBUG:teuthology.orchestra.run.smithi153:> sudo modprobe fuse 2024-03-21T10:05:12.230 DEBUG:teuthology.orchestra.run.smithi153:> cat /proc/self/mounts | awk '{print $2}' 2024-03-21T10:05:12.279 INFO:teuthology.orchestra.run.smithi153.stdout:/proc 2024-03-21T10:05:12.279 INFO:teuthology.orchestra.run.smithi153.stdout:/sys 2024-03-21T10:05:12.279 INFO:teuthology.orchestra.run.smithi153.stdout:/dev 2024-03-21T10:05:12.280 INFO:teuthology.orchestra.run.smithi153.stdout:/sys/kernel/security 2024-03-21T10:05:12.280 INFO:teuthology.orchestra.run.smithi153.stdout:/dev/shm 2024-03-21T10:05:12.280 INFO:teuthology.orchestra.run.smithi153.stdout:/dev/pts 2024-03-21T10:05:12.280 INFO:teuthology.orchestra.run.smithi153.stdout:/run 2024-03-21T10:05:12.280 INFO:teuthology.orchestra.run.smithi153.stdout:/sys/fs/cgroup 2024-03-21T10:05:12.280 INFO:teuthology.orchestra.run.smithi153.stdout:/sys/fs/pstore 2024-03-21T10:05:12.280 INFO:teuthology.orchestra.run.smithi153.stdout:/sys/fs/bpf 2024-03-21T10:05:12.280 INFO:teuthology.orchestra.run.smithi153.stdout:/ 2024-03-21T10:05:12.280 INFO:teuthology.orchestra.run.smithi153.stdout:/sys/fs/selinux 2024-03-21T10:05:12.280 INFO:teuthology.orchestra.run.smithi153.stdout:/proc/sys/fs/binfmt_misc 2024-03-21T10:05:12.280 INFO:teuthology.orchestra.run.smithi153.stdout:/dev/mqueue 2024-03-21T10:05:12.280 INFO:teuthology.orchestra.run.smithi153.stdout:/dev/hugepages 2024-03-21T10:05:12.280 INFO:teuthology.orchestra.run.smithi153.stdout:/sys/kernel/debug 2024-03-21T10:05:12.280 INFO:teuthology.orchestra.run.smithi153.stdout:/sys/kernel/tracing 2024-03-21T10:05:12.280 INFO:teuthology.orchestra.run.smithi153.stdout:/sys/fs/fuse/connections 2024-03-21T10:05:12.281 INFO:teuthology.orchestra.run.smithi153.stdout:/sys/kernel/config 2024-03-21T10:05:12.281 INFO:teuthology.orchestra.run.smithi153.stdout:/run/credentials/systemd-sysctl.service 2024-03-21T10:05:12.281 INFO:teuthology.orchestra.run.smithi153.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-21T10:05:12.281 INFO:teuthology.orchestra.run.smithi153.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-21T10:05:12.281 INFO:teuthology.orchestra.run.smithi153.stdout:/var/lib/nfs/rpc_pipefs 2024-03-21T10:05:12.281 INFO:teuthology.orchestra.run.smithi153.stdout:/run/user/1000 2024-03-21T10:05:12.281 INFO:teuthology.orchestra.run.smithi153.stdout:/var/lib/ceph 2024-03-21T10:05:12.281 INFO:teuthology.orchestra.run.smithi153.stdout:/var/lib/containers/storage/overlay 2024-03-21T10:05:12.281 INFO:teuthology.orchestra.run.smithi153.stdout:/var/lib/containers/storage/overlay/1a78b7d0d52de441bd07a7f84ea584762d6c49eadf71a4cd0c027ce32a2e3b2e/merged 2024-03-21T10:05:12.281 INFO:teuthology.orchestra.run.smithi153.stdout:/var/lib/containers/storage/overlay/5f40359e14914d8a9089623a93271c40fa107a407da72364352c8ce94199e767/merged 2024-03-21T10:05:12.281 INFO:teuthology.orchestra.run.smithi153.stdout:/run/user/0 2024-03-21T10:05:12.281 INFO:teuthology.orchestra.run.smithi153.stdout:/proc/sys/fs/binfmt_misc 2024-03-21T10:05:12.281 INFO:teuthology.orchestra.run.smithi153.stdout:/var/lib/containers/storage/overlay/d8ba1e5482dfd182ad7f24fe413a20e2c554b13c1a52e876d82a2209e45c0214/merged 2024-03-21T10:05:12.281 INFO:teuthology.orchestra.run.smithi153.stdout:/var/lib/containers/storage/overlay/a7c55df1e4c1ae02b6c8d644cac9bcf18e17cdf29f7c7235f591553bee326974/merged 2024-03-21T10:05:12.281 INFO:teuthology.orchestra.run.smithi153.stdout:/var/lib/containers/storage/overlay/64d5fe09dfec2268b9fa72961c83d20c15f6b0556040d5901f5c7cdc8b482402/merged 2024-03-21T10:05:12.282 INFO:teuthology.orchestra.run.smithi153.stdout:/var/lib/containers/storage/overlay/4fc50842e6e8317e9201dde6cc8fa545a88cdc3f8e2bc09b19d33350aca695dc/merged 2024-03-21T10:05:12.282 INFO:teuthology.orchestra.run.smithi153.stdout:/var/lib/containers/storage/overlay/f03b1eb0bfdd91d26d0d2ddb817f11d9a4dbcc83f95c8a3f162de46b1f772c0f/merged 2024-03-21T10:05:12.282 INFO:teuthology.orchestra.run.smithi153.stdout:/var/lib/containers/storage/overlay/3c772ec53b68f6cd651582122b0222525ad0a03911264d545aa802855e5a5da2/merged 2024-03-21T10:05:12.282 INFO:teuthology.orchestra.run.smithi153.stdout:/var/lib/containers/storage/overlay/f832be57993da0590147aff14477907bcdc781c3e7cc23435520878018f3ad8a/merged 2024-03-21T10:05:12.282 INFO:teuthology.orchestra.run.smithi153.stdout:/var/lib/containers/storage/overlay/89ab59ea3ec111882e9c4e576aaf147ecda735a40f1736c7977509be437943b8/merged 2024-03-21T10:05:12.282 INFO:teuthology.orchestra.run.smithi153.stdout:/var/lib/containers/storage/overlay/c3e78be25694fee98c53ade1da856cd8426e9a12d1c3a3ed4ead86e5f3edae0f/merged 2024-03-21T10:05:12.282 INFO:teuthology.orchestra.run.smithi153.stdout:/var/lib/containers/storage/overlay/9d3fbbf86b47bc4cb9ccc07b6a1a70c06ff4e5d2958de5d65cc362f327ac433f/merged 2024-03-21T10:05:12.282 INFO:teuthology.orchestra.run.smithi153.stdout:/var/lib/containers/storage/overlay/5c6d917a2a132ac2e878f295988eb2bb4c89853bc9f493f23a8310c71f610bd5/merged 2024-03-21T10:05:12.282 INFO:teuthology.orchestra.run.smithi153.stdout:/run/netns 2024-03-21T10:05:12.282 INFO:teuthology.orchestra.run.smithi153.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-21T10:05:12.282 INFO:teuthology.orchestra.run.smithi153.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-21T10:05:12.283 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T10:05:12.283 DEBUG:teuthology.orchestra.run.smithi153:> ls /sys/fs/fuse/connections 2024-03-21T10:05:12.388 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi153.stderr:2024-03-21T10:05:12.386+0000 7fbf67440480 -1 init, newargv = 0x562d8a776510 newargc=15 2024-03-21T10:05:12.388 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi153.stderr:ceph-fuse[77502]: starting ceph client 2024-03-21T10:05:12.409 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi153.stderr:ceph-fuse[77502]: starting fuse 2024-03-21T10:05:13.431 DEBUG:teuthology.orchestra.run.smithi153:> sudo modprobe fuse 2024-03-21T10:05:13.535 DEBUG:teuthology.orchestra.run.smithi153:> cat /proc/self/mounts | awk '{print $2}' 2024-03-21T10:05:13.589 INFO:teuthology.orchestra.run.smithi153.stdout:/proc 2024-03-21T10:05:13.589 INFO:teuthology.orchestra.run.smithi153.stdout:/sys 2024-03-21T10:05:13.590 INFO:teuthology.orchestra.run.smithi153.stdout:/dev 2024-03-21T10:05:13.590 INFO:teuthology.orchestra.run.smithi153.stdout:/sys/kernel/security 2024-03-21T10:05:13.590 INFO:teuthology.orchestra.run.smithi153.stdout:/dev/shm 2024-03-21T10:05:13.590 INFO:teuthology.orchestra.run.smithi153.stdout:/dev/pts 2024-03-21T10:05:13.590 INFO:teuthology.orchestra.run.smithi153.stdout:/run 2024-03-21T10:05:13.590 INFO:teuthology.orchestra.run.smithi153.stdout:/sys/fs/cgroup 2024-03-21T10:05:13.590 INFO:teuthology.orchestra.run.smithi153.stdout:/sys/fs/pstore 2024-03-21T10:05:13.590 INFO:teuthology.orchestra.run.smithi153.stdout:/sys/fs/bpf 2024-03-21T10:05:13.590 INFO:teuthology.orchestra.run.smithi153.stdout:/ 2024-03-21T10:05:13.590 INFO:teuthology.orchestra.run.smithi153.stdout:/sys/fs/selinux 2024-03-21T10:05:13.590 INFO:teuthology.orchestra.run.smithi153.stdout:/proc/sys/fs/binfmt_misc 2024-03-21T10:05:13.590 INFO:teuthology.orchestra.run.smithi153.stdout:/dev/mqueue 2024-03-21T10:05:13.590 INFO:teuthology.orchestra.run.smithi153.stdout:/dev/hugepages 2024-03-21T10:05:13.590 INFO:teuthology.orchestra.run.smithi153.stdout:/sys/kernel/debug 2024-03-21T10:05:13.590 INFO:teuthology.orchestra.run.smithi153.stdout:/sys/kernel/tracing 2024-03-21T10:05:13.591 INFO:teuthology.orchestra.run.smithi153.stdout:/sys/fs/fuse/connections 2024-03-21T10:05:13.591 INFO:teuthology.orchestra.run.smithi153.stdout:/sys/kernel/config 2024-03-21T10:05:13.591 INFO:teuthology.orchestra.run.smithi153.stdout:/run/credentials/systemd-sysctl.service 2024-03-21T10:05:13.591 INFO:teuthology.orchestra.run.smithi153.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-21T10:05:13.591 INFO:teuthology.orchestra.run.smithi153.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-21T10:05:13.591 INFO:teuthology.orchestra.run.smithi153.stdout:/var/lib/nfs/rpc_pipefs 2024-03-21T10:05:13.591 INFO:teuthology.orchestra.run.smithi153.stdout:/run/user/1000 2024-03-21T10:05:13.591 INFO:teuthology.orchestra.run.smithi153.stdout:/var/lib/ceph 2024-03-21T10:05:13.591 INFO:teuthology.orchestra.run.smithi153.stdout:/var/lib/containers/storage/overlay 2024-03-21T10:05:13.591 INFO:teuthology.orchestra.run.smithi153.stdout:/var/lib/containers/storage/overlay/1a78b7d0d52de441bd07a7f84ea584762d6c49eadf71a4cd0c027ce32a2e3b2e/merged 2024-03-21T10:05:13.591 INFO:teuthology.orchestra.run.smithi153.stdout:/var/lib/containers/storage/overlay/5f40359e14914d8a9089623a93271c40fa107a407da72364352c8ce94199e767/merged 2024-03-21T10:05:13.591 INFO:teuthology.orchestra.run.smithi153.stdout:/run/user/0 2024-03-21T10:05:13.591 INFO:teuthology.orchestra.run.smithi153.stdout:/proc/sys/fs/binfmt_misc 2024-03-21T10:05:13.591 INFO:teuthology.orchestra.run.smithi153.stdout:/var/lib/containers/storage/overlay/d8ba1e5482dfd182ad7f24fe413a20e2c554b13c1a52e876d82a2209e45c0214/merged 2024-03-21T10:05:13.591 INFO:teuthology.orchestra.run.smithi153.stdout:/var/lib/containers/storage/overlay/a7c55df1e4c1ae02b6c8d644cac9bcf18e17cdf29f7c7235f591553bee326974/merged 2024-03-21T10:05:13.591 INFO:teuthology.orchestra.run.smithi153.stdout:/var/lib/containers/storage/overlay/64d5fe09dfec2268b9fa72961c83d20c15f6b0556040d5901f5c7cdc8b482402/merged 2024-03-21T10:05:13.592 INFO:teuthology.orchestra.run.smithi153.stdout:/var/lib/containers/storage/overlay/4fc50842e6e8317e9201dde6cc8fa545a88cdc3f8e2bc09b19d33350aca695dc/merged 2024-03-21T10:05:13.592 INFO:teuthology.orchestra.run.smithi153.stdout:/var/lib/containers/storage/overlay/f03b1eb0bfdd91d26d0d2ddb817f11d9a4dbcc83f95c8a3f162de46b1f772c0f/merged 2024-03-21T10:05:13.592 INFO:teuthology.orchestra.run.smithi153.stdout:/var/lib/containers/storage/overlay/3c772ec53b68f6cd651582122b0222525ad0a03911264d545aa802855e5a5da2/merged 2024-03-21T10:05:13.592 INFO:teuthology.orchestra.run.smithi153.stdout:/var/lib/containers/storage/overlay/f832be57993da0590147aff14477907bcdc781c3e7cc23435520878018f3ad8a/merged 2024-03-21T10:05:13.592 INFO:teuthology.orchestra.run.smithi153.stdout:/var/lib/containers/storage/overlay/89ab59ea3ec111882e9c4e576aaf147ecda735a40f1736c7977509be437943b8/merged 2024-03-21T10:05:13.592 INFO:teuthology.orchestra.run.smithi153.stdout:/var/lib/containers/storage/overlay/c3e78be25694fee98c53ade1da856cd8426e9a12d1c3a3ed4ead86e5f3edae0f/merged 2024-03-21T10:05:13.592 INFO:teuthology.orchestra.run.smithi153.stdout:/var/lib/containers/storage/overlay/9d3fbbf86b47bc4cb9ccc07b6a1a70c06ff4e5d2958de5d65cc362f327ac433f/merged 2024-03-21T10:05:13.592 INFO:teuthology.orchestra.run.smithi153.stdout:/var/lib/containers/storage/overlay/5c6d917a2a132ac2e878f295988eb2bb4c89853bc9f493f23a8310c71f610bd5/merged 2024-03-21T10:05:13.592 INFO:teuthology.orchestra.run.smithi153.stdout:/run/netns 2024-03-21T10:05:13.592 INFO:teuthology.orchestra.run.smithi153.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-21T10:05:13.592 INFO:teuthology.orchestra.run.smithi153.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-21T10:05:13.592 INFO:teuthology.orchestra.run.smithi153.stdout:/home/ubuntu/cephtest/mnt.0 2024-03-21T10:05:13.593 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T10:05:13.593 DEBUG:teuthology.orchestra.run.smithi153:> ls /sys/fs/fuse/connections 2024-03-21T10:05:13.642 INFO:teuthology.orchestra.run.smithi153.stdout:158 2024-03-21T10:05:13.642 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [158] 2024-03-21T10:05:13.642 DEBUG:teuthology.orchestra.run.smithi153:> sudo stdin-killer -- python3 -c ' 2024-03-21T10:05:13.642 DEBUG:teuthology.orchestra.run.smithi153:> import glob 2024-03-21T10:05:13.643 DEBUG:teuthology.orchestra.run.smithi153:> import re 2024-03-21T10:05:13.643 DEBUG:teuthology.orchestra.run.smithi153:> import os 2024-03-21T10:05:13.643 DEBUG:teuthology.orchestra.run.smithi153:> import subprocess 2024-03-21T10:05:13.643 DEBUG:teuthology.orchestra.run.smithi153:> 2024-03-21T10:05:13.643 DEBUG:teuthology.orchestra.run.smithi153:> def _find_admin_socket(client_name): 2024-03-21T10:05:13.643 DEBUG:teuthology.orchestra.run.smithi153:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-03-21T10:05:13.643 DEBUG:teuthology.orchestra.run.smithi153:> files = glob.glob(asok_path) 2024-03-21T10:05:13.643 DEBUG:teuthology.orchestra.run.smithi153:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-03-21T10:05:13.643 DEBUG:teuthology.orchestra.run.smithi153:> 2024-03-21T10:05:13.643 DEBUG:teuthology.orchestra.run.smithi153:> # Given a non-glob path, it better be there 2024-03-21T10:05:13.643 DEBUG:teuthology.orchestra.run.smithi153:> if "*" not in asok_path: 2024-03-21T10:05:13.643 DEBUG:teuthology.orchestra.run.smithi153:> assert(len(files) == 1) 2024-03-21T10:05:13.643 DEBUG:teuthology.orchestra.run.smithi153:> return files[0] 2024-03-21T10:05:13.643 DEBUG:teuthology.orchestra.run.smithi153:> 2024-03-21T10:05:13.643 DEBUG:teuthology.orchestra.run.smithi153:> for f in files: 2024-03-21T10:05:13.644 DEBUG:teuthology.orchestra.run.smithi153:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-21T10:05:13.644 DEBUG:teuthology.orchestra.run.smithi153:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-21T10:05:13.644 DEBUG:teuthology.orchestra.run.smithi153:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-21T10:05:13.644 DEBUG:teuthology.orchestra.run.smithi153:> contents = proc_f.read() 2024-03-21T10:05:13.644 DEBUG:teuthology.orchestra.run.smithi153:> if mountpoint in contents: 2024-03-21T10:05:13.644 DEBUG:teuthology.orchestra.run.smithi153:> return f 2024-03-21T10:05:13.644 DEBUG:teuthology.orchestra.run.smithi153:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-21T10:05:13.644 DEBUG:teuthology.orchestra.run.smithi153:> 2024-03-21T10:05:13.644 DEBUG:teuthology.orchestra.run.smithi153:> print(_find_admin_socket("client.0")) 2024-03-21T10:05:13.644 DEBUG:teuthology.orchestra.run.smithi153:> ' 2024-03-21T10:05:13.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:05:13 smithi153 ceph-mon[29117]: pgmap v239: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-21T10:05:13.816 INFO:teuthology.orchestra.run.smithi153.stdout:/var/run/ceph/ceph-client.0.77502.asok 2024-03-21T10:05:13.821 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-21T10:05:13.836 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.77502.asok 2024-03-21T10:05:13.837 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T10:05:13.837 DEBUG:teuthology.orchestra.run.smithi153:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.77502.asok status 2024-03-21T10:05:13.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:05:13 smithi174 ceph-mon[37496]: pgmap v239: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-21T10:05:14.074 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:05:14.074 INFO:teuthology.orchestra.run.smithi153.stdout: "metadata": { 2024-03-21T10:05:14.075 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_sha1": "7fe91d5d5842e04be3b4f514d6dd990c54b29c76", 2024-03-21T10:05:14.075 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph_version": "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)", 2024-03-21T10:05:14.075 INFO:teuthology.orchestra.run.smithi153.stdout: "entity_id": "0", 2024-03-21T10:05:14.075 INFO:teuthology.orchestra.run.smithi153.stdout: "hostname": "smithi153", 2024-03-21T10:05:14.075 INFO:teuthology.orchestra.run.smithi153.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-03-21T10:05:14.075 INFO:teuthology.orchestra.run.smithi153.stdout: "pid": "77502", 2024-03-21T10:05:14.075 INFO:teuthology.orchestra.run.smithi153.stdout: "root": "/" 2024-03-21T10:05:14.075 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:05:14.075 INFO:teuthology.orchestra.run.smithi153.stdout: "dentry_count": 0, 2024-03-21T10:05:14.075 INFO:teuthology.orchestra.run.smithi153.stdout: "dentry_pinned_count": 0, 2024-03-21T10:05:14.075 INFO:teuthology.orchestra.run.smithi153.stdout: "id": 14538, 2024-03-21T10:05:14.075 INFO:teuthology.orchestra.run.smithi153.stdout: "inst": { 2024-03-21T10:05:14.075 INFO:teuthology.orchestra.run.smithi153.stdout: "name": { 2024-03-21T10:05:14.075 INFO:teuthology.orchestra.run.smithi153.stdout: "type": "client", 2024-03-21T10:05:14.075 INFO:teuthology.orchestra.run.smithi153.stdout: "num": 14538 2024-03-21T10:05:14.075 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:05:14.076 INFO:teuthology.orchestra.run.smithi153.stdout: "addr": { 2024-03-21T10:05:14.076 INFO:teuthology.orchestra.run.smithi153.stdout: "type": "v1", 2024-03-21T10:05:14.076 INFO:teuthology.orchestra.run.smithi153.stdout: "addr": "192.168.0.1:0", 2024-03-21T10:05:14.076 INFO:teuthology.orchestra.run.smithi153.stdout: "nonce": 2069638377 2024-03-21T10:05:14.076 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-03-21T10:05:14.076 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:05:14.076 INFO:teuthology.orchestra.run.smithi153.stdout: "addr": { 2024-03-21T10:05:14.076 INFO:teuthology.orchestra.run.smithi153.stdout: "type": "v1", 2024-03-21T10:05:14.076 INFO:teuthology.orchestra.run.smithi153.stdout: "addr": "192.168.0.1:0", 2024-03-21T10:05:14.076 INFO:teuthology.orchestra.run.smithi153.stdout: "nonce": 2069638377 2024-03-21T10:05:14.076 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:05:14.076 INFO:teuthology.orchestra.run.smithi153.stdout: "inst_str": "client.14538 192.168.0.1:0/2069638377", 2024-03-21T10:05:14.076 INFO:teuthology.orchestra.run.smithi153.stdout: "addr_str": "192.168.0.1:0/2069638377", 2024-03-21T10:05:14.076 INFO:teuthology.orchestra.run.smithi153.stdout: "inode_count": 1, 2024-03-21T10:05:14.076 INFO:teuthology.orchestra.run.smithi153.stdout: "mds_epoch": 17, 2024-03-21T10:05:14.076 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_epoch": 47, 2024-03-21T10:05:14.077 INFO:teuthology.orchestra.run.smithi153.stdout: "osd_epoch_barrier": 0, 2024-03-21T10:05:14.077 INFO:teuthology.orchestra.run.smithi153.stdout: "blocklisted": false, 2024-03-21T10:05:14.077 INFO:teuthology.orchestra.run.smithi153.stdout: "fs_name": "cephfs" 2024-03-21T10:05:14.077 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:05:14.091 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid 070ed3e4-e769-11ee-95cd-87774f69a715 -- ceph fs ls 2024-03-21T10:05:14.509 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi153/config 2024-03-21T10:05:15.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:05:15 smithi153 ceph-mon[29117]: pgmap v240: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:05:15.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:05:15 smithi174 ceph-mon[37496]: pgmap v240: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:05:16.744 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:16.742+0000 7fc412400700 1 -- 172.21.15.153:0/4281100464 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc40c102fd0 msgr2=0x7fc40c1033b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:05:16.745 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:16.742+0000 7fc412400700 1 --2- 172.21.15.153:0/4281100464 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc40c102fd0 0x7fc40c1033b0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc009910 tx=0x7fc3fc009c20 comp rx=0 tx=0).stop 2024-03-21T10:05:16.745 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:16.743+0000 7fc412400700 1 -- 172.21.15.153:0/4281100464 shutdown_connections 2024-03-21T10:05:16.745 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:16.743+0000 7fc412400700 1 --2- 172.21.15.153:0/4281100464 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc40c103980 0x7fc40c1079d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:05:16.745 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:16.743+0000 7fc412400700 1 --2- 172.21.15.153:0/4281100464 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc40c102fd0 0x7fc40c1033b0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:05:16.745 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:16.743+0000 7fc412400700 1 -- 172.21.15.153:0/4281100464 >> 172.21.15.153:0/4281100464 conn(0x7fc40c0fe840 msgr2=0x7fc40c100c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:05:16.745 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:16.743+0000 7fc412400700 1 -- 172.21.15.153:0/4281100464 shutdown_connections 2024-03-21T10:05:16.745 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:16.743+0000 7fc412400700 1 -- 172.21.15.153:0/4281100464 wait complete. 2024-03-21T10:05:16.745 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:16.744+0000 7fc412400700 1 Processor -- start 2024-03-21T10:05:16.746 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:16.744+0000 7fc412400700 1 -- start start 2024-03-21T10:05:16.746 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:16.745+0000 7fc412400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc40c102fd0 0x7fc40c19e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:05:16.746 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:16.745+0000 7fc412400700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc40c103980 0x7fc40c19ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:05:16.746 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:16.745+0000 7fc412400700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc40c19f3b0 con 0x7fc40c102fd0 2024-03-21T10:05:16.746 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:16.745+0000 7fc412400700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc40c198860 con 0x7fc40c103980 2024-03-21T10:05:16.747 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:16.745+0000 7fc40b400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc40c102fd0 0x7fc40c19e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:05:16.747 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:16.745+0000 7fc40b400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc40c102fd0 0x7fc40c19e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:50462/0 (socket says 172.21.15.153:50462) 2024-03-21T10:05:16.747 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:16.745+0000 7fc40b400700 1 -- 172.21.15.153:0/2444626030 learned_addr learned my addr 172.21.15.153:0/2444626030 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:05:16.747 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:16.745+0000 7fc40aa00700 1 --2- 172.21.15.153:0/2444626030 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc40c103980 0x7fc40c19ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:05:16.747 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:16.746+0000 7fc40b400700 1 -- 172.21.15.153:0/2444626030 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc40c103980 msgr2=0x7fc40c19ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:05:16.747 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:16.746+0000 7fc40b400700 1 --2- 172.21.15.153:0/2444626030 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc40c103980 0x7fc40c19ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:05:16.748 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:16.746+0000 7fc40b400700 1 -- 172.21.15.153:0/2444626030 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3fc0095f0 con 0x7fc40c102fd0 2024-03-21T10:05:16.748 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:16.746+0000 7fc40aa00700 1 --2- 172.21.15.153:0/2444626030 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc40c103980 0x7fc40c19ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:05:16.748 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:16.746+0000 7fc40b400700 1 --2- 172.21.15.153:0/2444626030 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc40c102fd0 0x7fc40c19e7e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc00b5c0 tx=0x7fc3fc00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:05:16.748 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:16.746+0000 7fc403e00700 1 -- 172.21.15.153:0/2444626030 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc3fc01d070 con 0x7fc40c102fd0 2024-03-21T10:05:16.748 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:16.746+0000 7fc403e00700 1 -- 172.21.15.153:0/2444626030 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc3fc004fa0 con 0x7fc40c102fd0 2024-03-21T10:05:16.749 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:16.746+0000 7fc412400700 1 -- 172.21.15.153:0/2444626030 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc40c198ae0 con 0x7fc40c102fd0 2024-03-21T10:05:16.749 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:16.746+0000 7fc403e00700 1 -- 172.21.15.153:0/2444626030 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc3fc00f460 con 0x7fc40c102fd0 2024-03-21T10:05:16.749 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:16.747+0000 7fc412400700 1 -- 172.21.15.153:0/2444626030 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc40c198fd0 con 0x7fc40c102fd0 2024-03-21T10:05:16.749 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:16.748+0000 7fc412400700 1 -- 172.21.15.153:0/2444626030 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc40c10b370 con 0x7fc40c102fd0 2024-03-21T10:05:16.750 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:16.749+0000 7fc403e00700 1 -- 172.21.15.153:0/2444626030 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fc3fc022810 con 0x7fc40c102fd0 2024-03-21T10:05:16.751 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:16.750+0000 7fc403e00700 1 --2- 172.21.15.153:0/2444626030 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fc3f006c4e0 0x7fc3f006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:05:16.751 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:16.750+0000 7fc403e00700 1 -- 172.21.15.153:0/2444626030 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fc3fc08d240 con 0x7fc40c102fd0 2024-03-21T10:05:16.752 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:16.750+0000 7fc40aa00700 1 --2- 172.21.15.153:0/2444626030 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fc3f006c4e0 0x7fc3f006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:05:16.752 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:16.751+0000 7fc40aa00700 1 --2- 172.21.15.153:0/2444626030 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fc3f006c4e0 0x7fc3f006e9a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc3f8005950 tx=0x7fc3f80058e0 comp rx=0 tx=0).ready entity=mgr.14221 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:05:16.755 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:16.754+0000 7fc403e00700 1 -- 172.21.15.153:0/2444626030 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fc3fc05b4e0 con 0x7fc40c102fd0 2024-03-21T10:05:17.015 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:17.013+0000 7fc412400700 1 -- 172.21.15.153:0/2444626030 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fc40c04ea90 con 0x7fc40c102fd0 2024-03-21T10:05:17.016 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:17.014+0000 7fc403e00700 1 -- 172.21.15.153:0/2444626030 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+83 (secure 0 0 0) 0x7fc3fc027070 con 0x7fc40c102fd0 2024-03-21T10:05:17.016 INFO:teuthology.orchestra.run.smithi153.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-03-21T10:05:17.018 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:17.017+0000 7fc412400700 1 -- 172.21.15.153:0/2444626030 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fc3f006c4e0 msgr2=0x7fc3f006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:05:17.019 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:17.017+0000 7fc412400700 1 --2- 172.21.15.153:0/2444626030 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fc3f006c4e0 0x7fc3f006e9a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc3f8005950 tx=0x7fc3f80058e0 comp rx=0 tx=0).stop 2024-03-21T10:05:17.019 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:17.017+0000 7fc412400700 1 -- 172.21.15.153:0/2444626030 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc40c102fd0 msgr2=0x7fc40c19e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:05:17.019 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:17.017+0000 7fc412400700 1 --2- 172.21.15.153:0/2444626030 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc40c102fd0 0x7fc40c19e7e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc00b5c0 tx=0x7fc3fc00be20 comp rx=0 tx=0).stop 2024-03-21T10:05:17.019 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:17.017+0000 7fc412400700 1 -- 172.21.15.153:0/2444626030 shutdown_connections 2024-03-21T10:05:17.019 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:17.017+0000 7fc412400700 1 --2- 172.21.15.153:0/2444626030 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fc3f006c4e0 0x7fc3f006e9a0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:05:17.019 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:17.017+0000 7fc412400700 1 --2- 172.21.15.153:0/2444626030 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc40c102fd0 0x7fc40c19e7e0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:05:17.019 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:17.017+0000 7fc412400700 1 --2- 172.21.15.153:0/2444626030 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc40c103980 0x7fc40c19ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:05:17.020 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:17.017+0000 7fc412400700 1 -- 172.21.15.153:0/2444626030 >> 172.21.15.153:0/2444626030 conn(0x7fc40c0fe840 msgr2=0x7fc40c0ffe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:05:17.020 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:17.018+0000 7fc412400700 1 -- 172.21.15.153:0/2444626030 shutdown_connections 2024-03-21T10:05:17.020 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:17.018+0000 7fc412400700 1 -- 172.21.15.153:0/2444626030 wait complete. 2024-03-21T10:05:17.649 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:05:17 smithi153 ceph-mon[29117]: pgmap v241: 65 pgs: 65 active+clean; 581 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.6 KiB/s rd, 255 B/s wr, 4 op/s 2024-03-21T10:05:17.649 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:05:17 smithi153 ceph-mon[29117]: from='client.? 172.21.15.153:0/2444626030' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-21T10:05:17.699 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-21T10:05:17.699 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-21T10:05:17.699 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi174.front.sepia.ceph.com 2024-03-21T10:05:17.699 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-03-21T10:05:17.699 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-03-21T10:05:17.699 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-21T10:05:17.699 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-21T10:05:17.699 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-21T10:05:17.700 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-03-21T10:05:17.700 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T10:05:17.700 DEBUG:teuthology.orchestra.run.smithi174:> ip addr 2024-03-21T10:05:17.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:05:17 smithi174 ceph-mon[37496]: pgmap v241: 65 pgs: 65 active+clean; 581 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.6 KiB/s rd, 255 B/s wr, 4 op/s 2024-03-21T10:05:17.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:05:17 smithi174 ceph-mon[37496]: from='client.? 172.21.15.153:0/2444626030' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-21T10:05:17.759 INFO:teuthology.orchestra.run.smithi174.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-21T10:05:17.759 INFO:teuthology.orchestra.run.smithi174.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-21T10:05:17.759 INFO:teuthology.orchestra.run.smithi174.stdout: inet 127.0.0.1/8 scope host lo 2024-03-21T10:05:17.759 INFO:teuthology.orchestra.run.smithi174.stdout: valid_lft forever preferred_lft forever 2024-03-21T10:05:17.759 INFO:teuthology.orchestra.run.smithi174.stdout: inet6 ::1/128 scope host 2024-03-21T10:05:17.759 INFO:teuthology.orchestra.run.smithi174.stdout: valid_lft forever preferred_lft forever 2024-03-21T10:05:17.759 INFO:teuthology.orchestra.run.smithi174.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-21T10:05:17.759 INFO:teuthology.orchestra.run.smithi174.stdout: link/ether 0c:c4:7a:d9:36:e0 brd ff:ff:ff:ff:ff:ff 2024-03-21T10:05:17.759 INFO:teuthology.orchestra.run.smithi174.stdout: altname enp7s0f0 2024-03-21T10:05:17.759 INFO:teuthology.orchestra.run.smithi174.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-21T10:05:17.759 INFO:teuthology.orchestra.run.smithi174.stdout: link/ether 0c:c4:7a:d9:36:e1 brd ff:ff:ff:ff:ff:ff 2024-03-21T10:05:17.759 INFO:teuthology.orchestra.run.smithi174.stdout: altname enp7s0f1 2024-03-21T10:05:17.759 INFO:teuthology.orchestra.run.smithi174.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-21T10:05:17.760 INFO:teuthology.orchestra.run.smithi174.stdout: link/ether 0c:c4:7a:88:77:1a brd ff:ff:ff:ff:ff:ff 2024-03-21T10:05:17.760 INFO:teuthology.orchestra.run.smithi174.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-21T10:05:17.760 INFO:teuthology.orchestra.run.smithi174.stdout: link/ether 0c:c4:7a:88:77:1b brd ff:ff:ff:ff:ff:ff 2024-03-21T10:05:17.760 INFO:teuthology.orchestra.run.smithi174.stdout: inet 172.21.15.174/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-21T10:05:17.760 INFO:teuthology.orchestra.run.smithi174.stdout: valid_lft 42009sec preferred_lft 42009sec 2024-03-21T10:05:17.760 INFO:teuthology.orchestra.run.smithi174.stdout: inet6 fe80::ec4:7aff:fe88:771b/64 scope link 2024-03-21T10:05:17.760 INFO:teuthology.orchestra.run.smithi174.stdout: valid_lft forever preferred_lft forever 2024-03-21T10:05:17.761 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-21T10:05:17.761 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T10:05:17.761 DEBUG:teuthology.orchestra.run.smithi174:> (cd / && exec bash -c ' 2024-03-21T10:05:17.761 DEBUG:teuthology.orchestra.run.smithi174:> set -e 2024-03-21T10:05:17.761 DEBUG:teuthology.orchestra.run.smithi174:> sudo ip link add name ceph-brx type bridge 2024-03-21T10:05:17.761 DEBUG:teuthology.orchestra.run.smithi174:> sudo ip addr flush dev ceph-brx 2024-03-21T10:05:17.761 DEBUG:teuthology.orchestra.run.smithi174:> sudo ip link set ceph-brx up 2024-03-21T10:05:17.761 DEBUG:teuthology.orchestra.run.smithi174:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-21T10:05:17.761 DEBUG:teuthology.orchestra.run.smithi174:> ') 2024-03-21T10:05:18.178 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T10:05:18.179 DEBUG:teuthology.orchestra.run.smithi174:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-21T10:05:18.253 INFO:teuthology.orchestra.run.smithi174.stdout:1 2024-03-21T10:05:18.256 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T10:05:18.256 DEBUG:teuthology.orchestra.run.smithi174:> route 2024-03-21T10:05:18.312 INFO:teuthology.orchestra.run.smithi174.stdout:Kernel IP routing table 2024-03-21T10:05:18.312 INFO:teuthology.orchestra.run.smithi174.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-21T10:05:18.312 INFO:teuthology.orchestra.run.smithi174.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-21T10:05:18.313 INFO:teuthology.orchestra.run.smithi174.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-21T10:05:18.313 INFO:teuthology.orchestra.run.smithi174.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-21T10:05:18.313 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T10:05:18.313 DEBUG:teuthology.orchestra.run.smithi174:> (cd / && exec bash -c ' 2024-03-21T10:05:18.313 DEBUG:teuthology.orchestra.run.smithi174:> set -e 2024-03-21T10:05:18.313 DEBUG:teuthology.orchestra.run.smithi174:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-21T10:05:18.313 DEBUG:teuthology.orchestra.run.smithi174:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-21T10:05:18.313 DEBUG:teuthology.orchestra.run.smithi174:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-21T10:05:18.313 DEBUG:teuthology.orchestra.run.smithi174:> ') 2024-03-21T10:05:18.948 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T10:05:18.948 DEBUG:teuthology.orchestra.run.smithi174:> ip netns list 2024-03-21T10:05:18.998 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T10:05:18.998 DEBUG:teuthology.orchestra.run.smithi174:> ip netns list-id 2024-03-21T10:05:19.044 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T10:05:19.044 DEBUG:teuthology.orchestra.run.smithi174:> (cd / && exec bash -c ' 2024-03-21T10:05:19.044 DEBUG:teuthology.orchestra.run.smithi174:> set -e 2024-03-21T10:05:19.044 DEBUG:teuthology.orchestra.run.smithi174:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-21T10:05:19.044 DEBUG:teuthology.orchestra.run.smithi174:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-03-21T10:05:19.044 DEBUG:teuthology.orchestra.run.smithi174:> ') 2024-03-21T10:05:19.217 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-03-21T10:05:19.218 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T10:05:19.218 DEBUG:teuthology.orchestra.run.smithi174:> (cd / && exec bash -c ' 2024-03-21T10:05:19.218 DEBUG:teuthology.orchestra.run.smithi174:> set -e 2024-03-21T10:05:19.218 DEBUG:teuthology.orchestra.run.smithi174:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-03-21T10:05:19.218 DEBUG:teuthology.orchestra.run.smithi174:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-21T10:05:19.218 DEBUG:teuthology.orchestra.run.smithi174:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-03-21T10:05:19.218 DEBUG:teuthology.orchestra.run.smithi174:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-03-21T10:05:19.218 DEBUG:teuthology.orchestra.run.smithi174:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-03-21T10:05:19.218 DEBUG:teuthology.orchestra.run.smithi174:> ') 2024-03-21T10:05:19.463 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:05:19 smithi174 ceph-mon[37496]: pgmap v242: 65 pgs: 65 active+clean; 581 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.8 KiB/s rd, 341 B/s wr, 4 op/s 2024-03-21T10:05:19.476 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T10:05:19.476 DEBUG:teuthology.orchestra.run.smithi174:> (cd / && exec bash -c ' 2024-03-21T10:05:19.476 DEBUG:teuthology.orchestra.run.smithi174:> set -e 2024-03-21T10:05:19.476 DEBUG:teuthology.orchestra.run.smithi174:> sudo ip link set brx.0 up 2024-03-21T10:05:19.477 DEBUG:teuthology.orchestra.run.smithi174:> sudo ip link set dev brx.0 master ceph-brx 2024-03-21T10:05:19.477 DEBUG:teuthology.orchestra.run.smithi174:> ') 2024-03-21T10:05:19.621 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-03-21T10:05:19.621 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-21T10:05:19.621 DEBUG:teuthology.orchestra.run.smithi174:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-03-21T10:05:19.661 INFO:teuthology.orchestra.run.smithi174.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-03-21T10:05:19.661 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-21T10:05:19.662 DEBUG:teuthology.orchestra.run.smithi174:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-03-21T10:05:19.748 DEBUG:teuthology.orchestra.run.smithi174:> sudo modprobe fuse 2024-03-21T10:05:19.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:05:19 smithi153 ceph-mon[29117]: pgmap v242: 65 pgs: 65 active+clean; 581 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.8 KiB/s rd, 341 B/s wr, 4 op/s 2024-03-21T10:05:19.885 DEBUG:teuthology.orchestra.run.smithi174:> cat /proc/self/mounts | awk '{print $2}' 2024-03-21T10:05:19.934 INFO:teuthology.orchestra.run.smithi174.stdout:/proc 2024-03-21T10:05:19.934 INFO:teuthology.orchestra.run.smithi174.stdout:/sys 2024-03-21T10:05:19.934 INFO:teuthology.orchestra.run.smithi174.stdout:/dev 2024-03-21T10:05:19.935 INFO:teuthology.orchestra.run.smithi174.stdout:/sys/kernel/security 2024-03-21T10:05:19.935 INFO:teuthology.orchestra.run.smithi174.stdout:/dev/shm 2024-03-21T10:05:19.935 INFO:teuthology.orchestra.run.smithi174.stdout:/dev/pts 2024-03-21T10:05:19.935 INFO:teuthology.orchestra.run.smithi174.stdout:/run 2024-03-21T10:05:19.935 INFO:teuthology.orchestra.run.smithi174.stdout:/sys/fs/cgroup 2024-03-21T10:05:19.935 INFO:teuthology.orchestra.run.smithi174.stdout:/sys/fs/pstore 2024-03-21T10:05:19.935 INFO:teuthology.orchestra.run.smithi174.stdout:/sys/fs/bpf 2024-03-21T10:05:19.935 INFO:teuthology.orchestra.run.smithi174.stdout:/ 2024-03-21T10:05:19.935 INFO:teuthology.orchestra.run.smithi174.stdout:/sys/fs/selinux 2024-03-21T10:05:19.935 INFO:teuthology.orchestra.run.smithi174.stdout:/proc/sys/fs/binfmt_misc 2024-03-21T10:05:19.935 INFO:teuthology.orchestra.run.smithi174.stdout:/dev/hugepages 2024-03-21T10:05:19.935 INFO:teuthology.orchestra.run.smithi174.stdout:/dev/mqueue 2024-03-21T10:05:19.935 INFO:teuthology.orchestra.run.smithi174.stdout:/sys/kernel/debug 2024-03-21T10:05:19.935 INFO:teuthology.orchestra.run.smithi174.stdout:/sys/kernel/tracing 2024-03-21T10:05:19.935 INFO:teuthology.orchestra.run.smithi174.stdout:/sys/fs/fuse/connections 2024-03-21T10:05:19.936 INFO:teuthology.orchestra.run.smithi174.stdout:/sys/kernel/config 2024-03-21T10:05:19.936 INFO:teuthology.orchestra.run.smithi174.stdout:/run/credentials/systemd-sysctl.service 2024-03-21T10:05:19.936 INFO:teuthology.orchestra.run.smithi174.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-21T10:05:19.936 INFO:teuthology.orchestra.run.smithi174.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-21T10:05:19.936 INFO:teuthology.orchestra.run.smithi174.stdout:/var/lib/nfs/rpc_pipefs 2024-03-21T10:05:19.936 INFO:teuthology.orchestra.run.smithi174.stdout:/run/user/1000 2024-03-21T10:05:19.936 INFO:teuthology.orchestra.run.smithi174.stdout:/var/lib/ceph 2024-03-21T10:05:19.936 INFO:teuthology.orchestra.run.smithi174.stdout:/run/user/0 2024-03-21T10:05:19.936 INFO:teuthology.orchestra.run.smithi174.stdout:/proc/sys/fs/binfmt_misc 2024-03-21T10:05:19.936 INFO:teuthology.orchestra.run.smithi174.stdout:/var/lib/containers/storage/overlay 2024-03-21T10:05:19.936 INFO:teuthology.orchestra.run.smithi174.stdout:/var/lib/containers/storage/overlay/4af2e0f4d4bebabce98c7854f1a324e403609292eb8bf9b80611c360fde213b8/merged 2024-03-21T10:05:19.936 INFO:teuthology.orchestra.run.smithi174.stdout:/var/lib/containers/storage/overlay/f7c630ef647f8693671c3dc2ba420b303c184d6b869e5890d3e95967799f2dee/merged 2024-03-21T10:05:19.936 INFO:teuthology.orchestra.run.smithi174.stdout:/var/lib/containers/storage/overlay/5857c5bdfc5110e77fd6012dbca0ee30083f8a505ba1ff856035b122edcb4801/merged 2024-03-21T10:05:19.936 INFO:teuthology.orchestra.run.smithi174.stdout:/var/lib/containers/storage/overlay/3d41027f75c80a9b60fb28bf5a91c8df6f302d02608c1fe8a239dcd728ce6f0c/merged 2024-03-21T10:05:19.937 INFO:teuthology.orchestra.run.smithi174.stdout:/var/lib/containers/storage/overlay/e3466dfdaac4a594b246914319b856e82cea513b7b33edc48489b81eec771f47/merged 2024-03-21T10:05:19.937 INFO:teuthology.orchestra.run.smithi174.stdout:/var/lib/containers/storage/overlay/27ee0c8c05df53c700592b94cbb87a1b8e026358fda68e4d5e99c743ab020c18/merged 2024-03-21T10:05:19.937 INFO:teuthology.orchestra.run.smithi174.stdout:/var/lib/containers/storage/overlay/3835ea552b808b53d3d2e511b5d6d7f4095a9b9cf19e1505267e132fe775e28d/merged 2024-03-21T10:05:19.937 INFO:teuthology.orchestra.run.smithi174.stdout:/var/lib/containers/storage/overlay/dc245ffaac5709f87fcfb512b2504f463a1f3648399ee844eb4e0f52ff9105a2/merged 2024-03-21T10:05:19.937 INFO:teuthology.orchestra.run.smithi174.stdout:/var/lib/containers/storage/overlay/2bf6d4cca3e7a9abed8dc2c1631f2f6d30df0399565521ef9707a3094bd13347/merged 2024-03-21T10:05:19.937 INFO:teuthology.orchestra.run.smithi174.stdout:/var/lib/containers/storage/overlay/d2da08dd2cea116d1389c0767ea65e4ca874b89414f201974c8b4515d464d31e/merged 2024-03-21T10:05:19.937 INFO:teuthology.orchestra.run.smithi174.stdout:/run/netns 2024-03-21T10:05:19.937 INFO:teuthology.orchestra.run.smithi174.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-21T10:05:19.937 INFO:teuthology.orchestra.run.smithi174.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-21T10:05:19.938 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T10:05:19.938 DEBUG:teuthology.orchestra.run.smithi174:> ls /sys/fs/fuse/connections 2024-03-21T10:05:19.982 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-21T10:05:19.983 DEBUG:teuthology.orchestra.run.smithi174:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-03-21T10:05:20.030 DEBUG:teuthology.orchestra.run.smithi174:> sudo modprobe fuse 2024-03-21T10:05:20.109 DEBUG:teuthology.orchestra.run.smithi174:> cat /proc/self/mounts | awk '{print $2}' 2024-03-21T10:05:20.151 INFO:teuthology.orchestra.run.smithi174.stdout:/proc 2024-03-21T10:05:20.152 INFO:teuthology.orchestra.run.smithi174.stdout:/sys 2024-03-21T10:05:20.152 INFO:teuthology.orchestra.run.smithi174.stdout:/dev 2024-03-21T10:05:20.152 INFO:teuthology.orchestra.run.smithi174.stdout:/sys/kernel/security 2024-03-21T10:05:20.152 INFO:teuthology.orchestra.run.smithi174.stdout:/dev/shm 2024-03-21T10:05:20.152 INFO:teuthology.orchestra.run.smithi174.stdout:/dev/pts 2024-03-21T10:05:20.152 INFO:teuthology.orchestra.run.smithi174.stdout:/run 2024-03-21T10:05:20.152 INFO:teuthology.orchestra.run.smithi174.stdout:/sys/fs/cgroup 2024-03-21T10:05:20.152 INFO:teuthology.orchestra.run.smithi174.stdout:/sys/fs/pstore 2024-03-21T10:05:20.152 INFO:teuthology.orchestra.run.smithi174.stdout:/sys/fs/bpf 2024-03-21T10:05:20.152 INFO:teuthology.orchestra.run.smithi174.stdout:/ 2024-03-21T10:05:20.152 INFO:teuthology.orchestra.run.smithi174.stdout:/sys/fs/selinux 2024-03-21T10:05:20.152 INFO:teuthology.orchestra.run.smithi174.stdout:/proc/sys/fs/binfmt_misc 2024-03-21T10:05:20.153 INFO:teuthology.orchestra.run.smithi174.stdout:/dev/hugepages 2024-03-21T10:05:20.153 INFO:teuthology.orchestra.run.smithi174.stdout:/dev/mqueue 2024-03-21T10:05:20.153 INFO:teuthology.orchestra.run.smithi174.stdout:/sys/kernel/debug 2024-03-21T10:05:20.153 INFO:teuthology.orchestra.run.smithi174.stdout:/sys/kernel/tracing 2024-03-21T10:05:20.153 INFO:teuthology.orchestra.run.smithi174.stdout:/sys/fs/fuse/connections 2024-03-21T10:05:20.153 INFO:teuthology.orchestra.run.smithi174.stdout:/sys/kernel/config 2024-03-21T10:05:20.153 INFO:teuthology.orchestra.run.smithi174.stdout:/run/credentials/systemd-sysctl.service 2024-03-21T10:05:20.153 INFO:teuthology.orchestra.run.smithi174.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-21T10:05:20.153 INFO:teuthology.orchestra.run.smithi174.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-21T10:05:20.153 INFO:teuthology.orchestra.run.smithi174.stdout:/var/lib/nfs/rpc_pipefs 2024-03-21T10:05:20.153 INFO:teuthology.orchestra.run.smithi174.stdout:/run/user/1000 2024-03-21T10:05:20.153 INFO:teuthology.orchestra.run.smithi174.stdout:/var/lib/ceph 2024-03-21T10:05:20.153 INFO:teuthology.orchestra.run.smithi174.stdout:/run/user/0 2024-03-21T10:05:20.153 INFO:teuthology.orchestra.run.smithi174.stdout:/proc/sys/fs/binfmt_misc 2024-03-21T10:05:20.154 INFO:teuthology.orchestra.run.smithi174.stdout:/var/lib/containers/storage/overlay 2024-03-21T10:05:20.154 INFO:teuthology.orchestra.run.smithi174.stdout:/var/lib/containers/storage/overlay/4af2e0f4d4bebabce98c7854f1a324e403609292eb8bf9b80611c360fde213b8/merged 2024-03-21T10:05:20.154 INFO:teuthology.orchestra.run.smithi174.stdout:/var/lib/containers/storage/overlay/f7c630ef647f8693671c3dc2ba420b303c184d6b869e5890d3e95967799f2dee/merged 2024-03-21T10:05:20.154 INFO:teuthology.orchestra.run.smithi174.stdout:/var/lib/containers/storage/overlay/5857c5bdfc5110e77fd6012dbca0ee30083f8a505ba1ff856035b122edcb4801/merged 2024-03-21T10:05:20.154 INFO:teuthology.orchestra.run.smithi174.stdout:/var/lib/containers/storage/overlay/3d41027f75c80a9b60fb28bf5a91c8df6f302d02608c1fe8a239dcd728ce6f0c/merged 2024-03-21T10:05:20.154 INFO:teuthology.orchestra.run.smithi174.stdout:/var/lib/containers/storage/overlay/e3466dfdaac4a594b246914319b856e82cea513b7b33edc48489b81eec771f47/merged 2024-03-21T10:05:20.154 INFO:teuthology.orchestra.run.smithi174.stdout:/var/lib/containers/storage/overlay/27ee0c8c05df53c700592b94cbb87a1b8e026358fda68e4d5e99c743ab020c18/merged 2024-03-21T10:05:20.154 INFO:teuthology.orchestra.run.smithi174.stdout:/var/lib/containers/storage/overlay/3835ea552b808b53d3d2e511b5d6d7f4095a9b9cf19e1505267e132fe775e28d/merged 2024-03-21T10:05:20.154 INFO:teuthology.orchestra.run.smithi174.stdout:/var/lib/containers/storage/overlay/dc245ffaac5709f87fcfb512b2504f463a1f3648399ee844eb4e0f52ff9105a2/merged 2024-03-21T10:05:20.154 INFO:teuthology.orchestra.run.smithi174.stdout:/var/lib/containers/storage/overlay/2bf6d4cca3e7a9abed8dc2c1631f2f6d30df0399565521ef9707a3094bd13347/merged 2024-03-21T10:05:20.154 INFO:teuthology.orchestra.run.smithi174.stdout:/var/lib/containers/storage/overlay/d2da08dd2cea116d1389c0767ea65e4ca874b89414f201974c8b4515d464d31e/merged 2024-03-21T10:05:20.154 INFO:teuthology.orchestra.run.smithi174.stdout:/run/netns 2024-03-21T10:05:20.154 INFO:teuthology.orchestra.run.smithi174.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-21T10:05:20.155 INFO:teuthology.orchestra.run.smithi174.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-21T10:05:20.155 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T10:05:20.156 DEBUG:teuthology.orchestra.run.smithi174:> ls /sys/fs/fuse/connections 2024-03-21T10:05:20.270 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi174.stderr:2024-03-21T10:05:20.269+0000 7f6ab9a1f480 -1 init, newargv = 0x55fdcb3cd6b0 newargc=15 2024-03-21T10:05:20.270 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi174.stderr:ceph-fuse[63322]: starting ceph client 2024-03-21T10:05:20.285 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi174.stderr:ceph-fuse[63322]: starting fuse 2024-03-21T10:05:21.346 DEBUG:teuthology.orchestra.run.smithi174:> sudo modprobe fuse 2024-03-21T10:05:21.454 DEBUG:teuthology.orchestra.run.smithi174:> cat /proc/self/mounts | awk '{print $2}' 2024-03-21T10:05:21.512 INFO:teuthology.orchestra.run.smithi174.stdout:/proc 2024-03-21T10:05:21.512 INFO:teuthology.orchestra.run.smithi174.stdout:/sys 2024-03-21T10:05:21.512 INFO:teuthology.orchestra.run.smithi174.stdout:/dev 2024-03-21T10:05:21.512 INFO:teuthology.orchestra.run.smithi174.stdout:/sys/kernel/security 2024-03-21T10:05:21.513 INFO:teuthology.orchestra.run.smithi174.stdout:/dev/shm 2024-03-21T10:05:21.513 INFO:teuthology.orchestra.run.smithi174.stdout:/dev/pts 2024-03-21T10:05:21.513 INFO:teuthology.orchestra.run.smithi174.stdout:/run 2024-03-21T10:05:21.513 INFO:teuthology.orchestra.run.smithi174.stdout:/sys/fs/cgroup 2024-03-21T10:05:21.513 INFO:teuthology.orchestra.run.smithi174.stdout:/sys/fs/pstore 2024-03-21T10:05:21.513 INFO:teuthology.orchestra.run.smithi174.stdout:/sys/fs/bpf 2024-03-21T10:05:21.513 INFO:teuthology.orchestra.run.smithi174.stdout:/ 2024-03-21T10:05:21.513 INFO:teuthology.orchestra.run.smithi174.stdout:/sys/fs/selinux 2024-03-21T10:05:21.513 INFO:teuthology.orchestra.run.smithi174.stdout:/proc/sys/fs/binfmt_misc 2024-03-21T10:05:21.513 INFO:teuthology.orchestra.run.smithi174.stdout:/dev/hugepages 2024-03-21T10:05:21.513 INFO:teuthology.orchestra.run.smithi174.stdout:/dev/mqueue 2024-03-21T10:05:21.514 INFO:teuthology.orchestra.run.smithi174.stdout:/sys/kernel/debug 2024-03-21T10:05:21.514 INFO:teuthology.orchestra.run.smithi174.stdout:/sys/kernel/tracing 2024-03-21T10:05:21.514 INFO:teuthology.orchestra.run.smithi174.stdout:/sys/fs/fuse/connections 2024-03-21T10:05:21.514 INFO:teuthology.orchestra.run.smithi174.stdout:/sys/kernel/config 2024-03-21T10:05:21.514 INFO:teuthology.orchestra.run.smithi174.stdout:/run/credentials/systemd-sysctl.service 2024-03-21T10:05:21.514 INFO:teuthology.orchestra.run.smithi174.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-21T10:05:21.514 INFO:teuthology.orchestra.run.smithi174.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-21T10:05:21.514 INFO:teuthology.orchestra.run.smithi174.stdout:/var/lib/nfs/rpc_pipefs 2024-03-21T10:05:21.514 INFO:teuthology.orchestra.run.smithi174.stdout:/run/user/1000 2024-03-21T10:05:21.515 INFO:teuthology.orchestra.run.smithi174.stdout:/var/lib/ceph 2024-03-21T10:05:21.515 INFO:teuthology.orchestra.run.smithi174.stdout:/run/user/0 2024-03-21T10:05:21.515 INFO:teuthology.orchestra.run.smithi174.stdout:/proc/sys/fs/binfmt_misc 2024-03-21T10:05:21.515 INFO:teuthology.orchestra.run.smithi174.stdout:/var/lib/containers/storage/overlay 2024-03-21T10:05:21.515 INFO:teuthology.orchestra.run.smithi174.stdout:/var/lib/containers/storage/overlay/4af2e0f4d4bebabce98c7854f1a324e403609292eb8bf9b80611c360fde213b8/merged 2024-03-21T10:05:21.515 INFO:teuthology.orchestra.run.smithi174.stdout:/var/lib/containers/storage/overlay/f7c630ef647f8693671c3dc2ba420b303c184d6b869e5890d3e95967799f2dee/merged 2024-03-21T10:05:21.515 INFO:teuthology.orchestra.run.smithi174.stdout:/var/lib/containers/storage/overlay/5857c5bdfc5110e77fd6012dbca0ee30083f8a505ba1ff856035b122edcb4801/merged 2024-03-21T10:05:21.515 INFO:teuthology.orchestra.run.smithi174.stdout:/var/lib/containers/storage/overlay/3d41027f75c80a9b60fb28bf5a91c8df6f302d02608c1fe8a239dcd728ce6f0c/merged 2024-03-21T10:05:21.515 INFO:teuthology.orchestra.run.smithi174.stdout:/var/lib/containers/storage/overlay/e3466dfdaac4a594b246914319b856e82cea513b7b33edc48489b81eec771f47/merged 2024-03-21T10:05:21.515 INFO:teuthology.orchestra.run.smithi174.stdout:/var/lib/containers/storage/overlay/27ee0c8c05df53c700592b94cbb87a1b8e026358fda68e4d5e99c743ab020c18/merged 2024-03-21T10:05:21.515 INFO:teuthology.orchestra.run.smithi174.stdout:/var/lib/containers/storage/overlay/3835ea552b808b53d3d2e511b5d6d7f4095a9b9cf19e1505267e132fe775e28d/merged 2024-03-21T10:05:21.516 INFO:teuthology.orchestra.run.smithi174.stdout:/var/lib/containers/storage/overlay/dc245ffaac5709f87fcfb512b2504f463a1f3648399ee844eb4e0f52ff9105a2/merged 2024-03-21T10:05:21.516 INFO:teuthology.orchestra.run.smithi174.stdout:/var/lib/containers/storage/overlay/2bf6d4cca3e7a9abed8dc2c1631f2f6d30df0399565521ef9707a3094bd13347/merged 2024-03-21T10:05:21.516 INFO:teuthology.orchestra.run.smithi174.stdout:/var/lib/containers/storage/overlay/d2da08dd2cea116d1389c0767ea65e4ca874b89414f201974c8b4515d464d31e/merged 2024-03-21T10:05:21.516 INFO:teuthology.orchestra.run.smithi174.stdout:/run/netns 2024-03-21T10:05:21.516 INFO:teuthology.orchestra.run.smithi174.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-21T10:05:21.516 INFO:teuthology.orchestra.run.smithi174.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-21T10:05:21.516 INFO:teuthology.orchestra.run.smithi174.stdout:/home/ubuntu/cephtest/mnt.1 2024-03-21T10:05:21.517 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T10:05:21.517 DEBUG:teuthology.orchestra.run.smithi174:> ls /sys/fs/fuse/connections 2024-03-21T10:05:21.559 INFO:teuthology.orchestra.run.smithi174.stdout:125 2024-03-21T10:05:21.559 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [125] 2024-03-21T10:05:21.560 DEBUG:teuthology.orchestra.run.smithi174:> sudo stdin-killer -- python3 -c ' 2024-03-21T10:05:21.560 DEBUG:teuthology.orchestra.run.smithi174:> import glob 2024-03-21T10:05:21.560 DEBUG:teuthology.orchestra.run.smithi174:> import re 2024-03-21T10:05:21.560 DEBUG:teuthology.orchestra.run.smithi174:> import os 2024-03-21T10:05:21.560 DEBUG:teuthology.orchestra.run.smithi174:> import subprocess 2024-03-21T10:05:21.560 DEBUG:teuthology.orchestra.run.smithi174:> 2024-03-21T10:05:21.560 DEBUG:teuthology.orchestra.run.smithi174:> def _find_admin_socket(client_name): 2024-03-21T10:05:21.560 DEBUG:teuthology.orchestra.run.smithi174:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-03-21T10:05:21.560 DEBUG:teuthology.orchestra.run.smithi174:> files = glob.glob(asok_path) 2024-03-21T10:05:21.560 DEBUG:teuthology.orchestra.run.smithi174:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-03-21T10:05:21.560 DEBUG:teuthology.orchestra.run.smithi174:> 2024-03-21T10:05:21.560 DEBUG:teuthology.orchestra.run.smithi174:> # Given a non-glob path, it better be there 2024-03-21T10:05:21.561 DEBUG:teuthology.orchestra.run.smithi174:> if "*" not in asok_path: 2024-03-21T10:05:21.561 DEBUG:teuthology.orchestra.run.smithi174:> assert(len(files) == 1) 2024-03-21T10:05:21.561 DEBUG:teuthology.orchestra.run.smithi174:> return files[0] 2024-03-21T10:05:21.561 DEBUG:teuthology.orchestra.run.smithi174:> 2024-03-21T10:05:21.561 DEBUG:teuthology.orchestra.run.smithi174:> for f in files: 2024-03-21T10:05:21.561 DEBUG:teuthology.orchestra.run.smithi174:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-21T10:05:21.561 DEBUG:teuthology.orchestra.run.smithi174:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-21T10:05:21.561 DEBUG:teuthology.orchestra.run.smithi174:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-21T10:05:21.561 DEBUG:teuthology.orchestra.run.smithi174:> contents = proc_f.read() 2024-03-21T10:05:21.561 DEBUG:teuthology.orchestra.run.smithi174:> if mountpoint in contents: 2024-03-21T10:05:21.561 DEBUG:teuthology.orchestra.run.smithi174:> return f 2024-03-21T10:05:21.561 DEBUG:teuthology.orchestra.run.smithi174:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-21T10:05:21.561 DEBUG:teuthology.orchestra.run.smithi174:> 2024-03-21T10:05:21.561 DEBUG:teuthology.orchestra.run.smithi174:> print(_find_admin_socket("client.1")) 2024-03-21T10:05:21.562 DEBUG:teuthology.orchestra.run.smithi174:> ' 2024-03-21T10:05:21.688 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:05:21 smithi174 ceph-mon[37496]: pgmap v243: 65 pgs: 65 active+clean; 581 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.8 KiB/s rd, 341 B/s wr, 3 op/s 2024-03-21T10:05:21.779 INFO:teuthology.orchestra.run.smithi174.stdout:/var/run/ceph/ceph-client.1.63322.asok 2024-03-21T10:05:21.783 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-21T10:05:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-21T10:05:21.797 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.63322.asok 2024-03-21T10:05:21.797 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T10:05:21.797 DEBUG:teuthology.orchestra.run.smithi174:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.63322.asok status 2024-03-21T10:05:21.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:05:21 smithi153 ceph-mon[29117]: pgmap v243: 65 pgs: 65 active+clean; 581 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.8 KiB/s rd, 341 B/s wr, 3 op/s 2024-03-21T10:05:22.032 INFO:teuthology.orchestra.run.smithi174.stdout:{ 2024-03-21T10:05:22.032 INFO:teuthology.orchestra.run.smithi174.stdout: "metadata": { 2024-03-21T10:05:22.032 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_sha1": "7fe91d5d5842e04be3b4f514d6dd990c54b29c76", 2024-03-21T10:05:22.032 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_version": "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)", 2024-03-21T10:05:22.032 INFO:teuthology.orchestra.run.smithi174.stdout: "entity_id": "1", 2024-03-21T10:05:22.032 INFO:teuthology.orchestra.run.smithi174.stdout: "hostname": "smithi174", 2024-03-21T10:05:22.032 INFO:teuthology.orchestra.run.smithi174.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-03-21T10:05:22.032 INFO:teuthology.orchestra.run.smithi174.stdout: "pid": "63322", 2024-03-21T10:05:22.033 INFO:teuthology.orchestra.run.smithi174.stdout: "root": "/" 2024-03-21T10:05:22.033 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-21T10:05:22.033 INFO:teuthology.orchestra.run.smithi174.stdout: "dentry_count": 0, 2024-03-21T10:05:22.033 INFO:teuthology.orchestra.run.smithi174.stdout: "dentry_pinned_count": 0, 2024-03-21T10:05:22.033 INFO:teuthology.orchestra.run.smithi174.stdout: "id": 24319, 2024-03-21T10:05:22.033 INFO:teuthology.orchestra.run.smithi174.stdout: "inst": { 2024-03-21T10:05:22.033 INFO:teuthology.orchestra.run.smithi174.stdout: "name": { 2024-03-21T10:05:22.033 INFO:teuthology.orchestra.run.smithi174.stdout: "type": "client", 2024-03-21T10:05:22.033 INFO:teuthology.orchestra.run.smithi174.stdout: "num": 24319 2024-03-21T10:05:22.033 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-21T10:05:22.034 INFO:teuthology.orchestra.run.smithi174.stdout: "addr": { 2024-03-21T10:05:22.034 INFO:teuthology.orchestra.run.smithi174.stdout: "type": "v1", 2024-03-21T10:05:22.034 INFO:teuthology.orchestra.run.smithi174.stdout: "addr": "192.168.0.1:0", 2024-03-21T10:05:22.034 INFO:teuthology.orchestra.run.smithi174.stdout: "nonce": 482977964 2024-03-21T10:05:22.034 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-03-21T10:05:22.034 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-21T10:05:22.034 INFO:teuthology.orchestra.run.smithi174.stdout: "addr": { 2024-03-21T10:05:22.034 INFO:teuthology.orchestra.run.smithi174.stdout: "type": "v1", 2024-03-21T10:05:22.034 INFO:teuthology.orchestra.run.smithi174.stdout: "addr": "192.168.0.1:0", 2024-03-21T10:05:22.034 INFO:teuthology.orchestra.run.smithi174.stdout: "nonce": 482977964 2024-03-21T10:05:22.034 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-03-21T10:05:22.034 INFO:teuthology.orchestra.run.smithi174.stdout: "inst_str": "client.24319 192.168.0.1:0/482977964", 2024-03-21T10:05:22.034 INFO:teuthology.orchestra.run.smithi174.stdout: "addr_str": "192.168.0.1:0/482977964", 2024-03-21T10:05:22.035 INFO:teuthology.orchestra.run.smithi174.stdout: "inode_count": 1, 2024-03-21T10:05:22.035 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_epoch": 17, 2024-03-21T10:05:22.035 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_epoch": 47, 2024-03-21T10:05:22.035 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_epoch_barrier": 0, 2024-03-21T10:05:22.035 INFO:teuthology.orchestra.run.smithi174.stdout: "blocklisted": false, 2024-03-21T10:05:22.035 INFO:teuthology.orchestra.run.smithi174.stdout: "fs_name": "cephfs" 2024-03-21T10:05:22.035 INFO:teuthology.orchestra.run.smithi174.stdout:} 2024-03-21T10:05:22.047 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T10:05:22.047 DEBUG:teuthology.orchestra.run.smithi153:> stat --file-system '--printf=%T 2024-03-21T10:05:22.048 DEBUG:teuthology.orchestra.run.smithi153:> ' -- /home/ubuntu/cephtest/mnt.0 2024-03-21T10:05:22.104 INFO:teuthology.orchestra.run.smithi153.stdout:fuseblk 2024-03-21T10:05:22.105 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-03-21T10:05:22.105 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T10:05:22.105 DEBUG:teuthology.orchestra.run.smithi153:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-21T10:05:22.197 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T10:05:22.197 DEBUG:teuthology.orchestra.run.smithi174:> stat --file-system '--printf=%T 2024-03-21T10:05:22.198 DEBUG:teuthology.orchestra.run.smithi174:> ' -- /home/ubuntu/cephtest/mnt.1 2024-03-21T10:05:22.257 INFO:teuthology.orchestra.run.smithi174.stdout:fuseblk 2024-03-21T10:05:22.257 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-03-21T10:05:22.257 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T10:05:22.257 DEBUG:teuthology.orchestra.run.smithi174:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-03-21T10:05:23.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:05:23 smithi174 ceph-mon[37496]: pgmap v244: 65 pgs: 65 active+clean; 582 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.7 KiB/s rd, 511 B/s wr, 5 op/s 2024-03-21T10:05:23.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:05:23 smithi153 ceph-mon[29117]: pgmap v244: 65 pgs: 65 active+clean; 582 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.7 KiB/s rd, 511 B/s wr, 5 op/s 2024-03-21T10:05:25.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:05:25 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:05:25.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:05:25 smithi174 ceph-mon[37496]: pgmap v245: 65 pgs: 65 active+clean; 582 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.9 KiB/s rd, 511 B/s wr, 4 op/s 2024-03-21T10:05:25.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:05:25 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:05:25.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:05:25 smithi153 ceph-mon[29117]: pgmap v245: 65 pgs: 65 active+clean; 582 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.9 KiB/s rd, 511 B/s wr, 4 op/s 2024-03-21T10:05:25.726 INFO:teuthology.run_tasks:Running task print... 2024-03-21T10:05:25.736 INFO:teuthology.task.print:**** done client 2024-03-21T10:05:25.736 INFO:teuthology.run_tasks:Running task parallel... 2024-03-21T10:05:25.772 INFO:teuthology.task.parallel:starting parallel... 2024-03-21T10:05:25.772 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-03-21T10:05:25.773 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-03-21T10:05:25.773 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi153.front.sepia.ceph.com 2024-03-21T10:05:25.773 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 070ed3e4-e769-11ee-95cd-87774f69a715 -e sha1=2edfee68ad7243bed422a049b6f5f58619300f11 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force' 2024-03-21T10:05:25.774 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-03-21T10:05:25.774 INFO:teuthology.task.sequential:In sequential, running task workunit... 2024-03-21T10:05:25.777 INFO:tasks.workunit:Pulling workunits from ref 2edfee68ad7243bed422a049b6f5f58619300f11 2024-03-21T10:05:25.778 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-03-21T10:05:25.778 INFO:tasks.workunit:timeout=3h 2024-03-21T10:05:25.778 INFO:tasks.workunit:cleanup=True 2024-03-21T10:05:25.778 DEBUG:teuthology.orchestra.run.smithi153:> stat -- /home/ubuntu/cephtest/mnt.0 2024-03-21T10:05:25.836 INFO:teuthology.orchestra.run.smithi153.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-03-21T10:05:25.836 INFO:teuthology.orchestra.run.smithi153.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-03-21T10:05:25.837 INFO:teuthology.orchestra.run.smithi153.stdout:Device: 9eh/158d Inode: 1 Links: 2 2024-03-21T10:05:25.837 INFO:teuthology.orchestra.run.smithi153.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-21T10:05:25.837 INFO:teuthology.orchestra.run.smithi153.stdout:Context: system_u:object_r:fusefs_t:s0 2024-03-21T10:05:25.837 INFO:teuthology.orchestra.run.smithi153.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-03-21T10:05:25.837 INFO:teuthology.orchestra.run.smithi153.stdout:Modify: 2024-03-21 10:04:32.143484838 +0000 2024-03-21T10:05:25.837 INFO:teuthology.orchestra.run.smithi153.stdout:Change: 2024-03-21 10:05:22.188575470 +0000 2024-03-21T10:05:25.837 INFO:teuthology.orchestra.run.smithi153.stdout: Birth: - 2024-03-21T10:05:25.837 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-03-21T10:05:25.837 DEBUG:teuthology.orchestra.run.smithi153:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-03-21T10:05:25.983 DEBUG:teuthology.orchestra.run.smithi174:> stat -- /home/ubuntu/cephtest/mnt.1 2024-03-21T10:05:26.042 INFO:teuthology.orchestra.run.smithi174.stdout: File: /home/ubuntu/cephtest/mnt.1 2024-03-21T10:05:26.042 INFO:teuthology.orchestra.run.smithi174.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-03-21T10:05:26.042 INFO:teuthology.orchestra.run.smithi174.stdout:Device: 7dh/125d Inode: 1 Links: 3 2024-03-21T10:05:26.042 INFO:teuthology.orchestra.run.smithi174.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-21T10:05:26.042 INFO:teuthology.orchestra.run.smithi174.stdout:Context: system_u:object_r:fusefs_t:s0 2024-03-21T10:05:26.042 INFO:teuthology.orchestra.run.smithi174.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-03-21T10:05:26.042 INFO:teuthology.orchestra.run.smithi174.stdout:Modify: 2024-03-21 10:05:25.965395478 +0000 2024-03-21T10:05:26.042 INFO:teuthology.orchestra.run.smithi174.stdout:Change: 2024-03-21 10:05:25.965395478 +0000 2024-03-21T10:05:26.042 INFO:teuthology.orchestra.run.smithi174.stdout: Birth: - 2024-03-21T10:05:26.043 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.1 2024-03-21T10:05:26.043 DEBUG:teuthology.orchestra.run.smithi174:> cd -- /home/ubuntu/cephtest/mnt.1 && sudo install -d -m 0755 --owner=ubuntu -- client.1 2024-03-21T10:05:26.148 DEBUG:teuthology.orchestra.run.smithi153:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 2edfee68ad7243bed422a049b6f5f58619300f11 2024-03-21T10:05:26.149 DEBUG:teuthology.orchestra.run.smithi174:> rm -rf /home/ubuntu/cephtest/clone.client.1 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.1 && cd /home/ubuntu/cephtest/clone.client.1 && git checkout 2edfee68ad7243bed422a049b6f5f58619300f11 2024-03-21T10:05:26.184 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi153/config 2024-03-21T10:05:26.194 INFO:tasks.workunit.client.0.smithi153.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-03-21T10:05:26.208 INFO:tasks.workunit.client.1.smithi174.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.1'... 2024-03-21T10:05:27.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:05:27 smithi174 ceph-mon[37496]: pgmap v246: 65 pgs: 65 active+clean; 585 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.8 KiB/s rd, 767 B/s wr, 5 op/s 2024-03-21T10:05:27.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:05:27 smithi153 ceph-mon[29117]: pgmap v246: 65 pgs: 65 active+clean; 585 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.8 KiB/s rd, 767 B/s wr, 5 op/s 2024-03-21T10:05:28.385 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:28.382+0000 7f3cc9600700 1 -- 172.21.15.153:0/3965723277 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3cc4102dd0 msgr2=0x7f3cc41031f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:05:28.385 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:28.382+0000 7f3cc9600700 1 --2- 172.21.15.153:0/3965723277 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3cc4102dd0 0x7f3cc41031f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f3cb4009910 tx=0x7f3cb4009c20 comp rx=0 tx=0).stop 2024-03-21T10:05:28.385 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:28.383+0000 7f3cc9600700 1 -- 172.21.15.153:0/3965723277 shutdown_connections 2024-03-21T10:05:28.385 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:28.383+0000 7f3cc9600700 1 --2- 172.21.15.153:0/3965723277 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3cc4103fd0 0x7f3cc4104430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:05:28.385 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:28.383+0000 7f3cc9600700 1 --2- 172.21.15.153:0/3965723277 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3cc4102dd0 0x7f3cc41031f0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:05:28.385 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:28.383+0000 7f3cc9600700 1 -- 172.21.15.153:0/3965723277 >> 172.21.15.153:0/3965723277 conn(0x7f3cc40fe370 msgr2=0x7f3cc41007b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:05:28.385 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:28.383+0000 7f3cc9600700 1 -- 172.21.15.153:0/3965723277 shutdown_connections 2024-03-21T10:05:28.385 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:28.383+0000 7f3cc9600700 1 -- 172.21.15.153:0/3965723277 wait complete. 2024-03-21T10:05:28.386 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:28.384+0000 7f3cc9600700 1 Processor -- start 2024-03-21T10:05:28.386 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:28.385+0000 7f3cc9600700 1 -- start start 2024-03-21T10:05:28.387 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:28.385+0000 7f3cc9600700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3cc4102dd0 0x7f3cc4198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:05:28.387 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:28.385+0000 7f3cc9600700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3cc4103fd0 0x7f3cc4198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:05:28.387 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:28.385+0000 7f3cc9600700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc41990e0 con 0x7f3cc4102dd0 2024-03-21T10:05:28.387 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:28.385+0000 7f3cc9600700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc4199220 con 0x7f3cc4103fd0 2024-03-21T10:05:28.387 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:28.385+0000 7f3cc2a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3cc4102dd0 0x7f3cc4198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:05:28.387 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:28.386+0000 7f3cc2a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3cc4102dd0 0x7f3cc4198580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:51864/0 (socket says 172.21.15.153:51864) 2024-03-21T10:05:28.387 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:28.386+0000 7f3cc2a00700 1 -- 172.21.15.153:0/830354920 learned_addr learned my addr 172.21.15.153:0/830354920 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:05:28.388 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:28.386+0000 7f3cc2000700 1 --2- 172.21.15.153:0/830354920 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3cc4103fd0 0x7f3cc4198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:05:28.388 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:28.386+0000 7f3cc2a00700 1 -- 172.21.15.153:0/830354920 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3cc4103fd0 msgr2=0x7f3cc4198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:05:28.388 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:28.386+0000 7f3cc2a00700 1 --2- 172.21.15.153:0/830354920 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3cc4103fd0 0x7f3cc4198ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:05:28.388 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:28.386+0000 7f3cc2a00700 1 -- 172.21.15.153:0/830354920 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cb40095f0 con 0x7f3cc4102dd0 2024-03-21T10:05:28.388 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:28.386+0000 7f3cc2a00700 1 --2- 172.21.15.153:0/830354920 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3cc4102dd0 0x7f3cc4198580 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f3cb400bda0 tx=0x7f3cb400be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:05:28.389 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:28.387+0000 7f3cbb400700 1 -- 172.21.15.153:0/830354920 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3cb401d070 con 0x7f3cc4102dd0 2024-03-21T10:05:28.389 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:28.387+0000 7f3cbb400700 1 -- 172.21.15.153:0/830354920 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f3cb400f460 con 0x7f3cc4102dd0 2024-03-21T10:05:28.389 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:28.387+0000 7f3cc9600700 1 -- 172.21.15.153:0/830354920 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cc419dc70 con 0x7f3cc4102dd0 2024-03-21T10:05:28.389 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:28.387+0000 7f3cbb400700 1 -- 172.21.15.153:0/830354920 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3cb4021620 con 0x7f3cc4102dd0 2024-03-21T10:05:28.389 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:28.387+0000 7f3cc9600700 1 -- 172.21.15.153:0/830354920 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cc419e160 con 0x7f3cc4102dd0 2024-03-21T10:05:28.389 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:28.388+0000 7f3cc9600700 1 -- 172.21.15.153:0/830354920 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cc410b930 con 0x7f3cc4102dd0 2024-03-21T10:05:28.390 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:28.389+0000 7f3cbb400700 1 -- 172.21.15.153:0/830354920 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f3cb400f5d0 con 0x7f3cc4102dd0 2024-03-21T10:05:28.391 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:28.390+0000 7f3cbb400700 1 --2- 172.21.15.153:0/830354920 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f3ca806c4e0 0x7f3ca806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:05:28.391 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:28.390+0000 7f3cbb400700 1 -- 172.21.15.153:0/830354920 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f3cb408cc30 con 0x7f3cc4102dd0 2024-03-21T10:05:28.392 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:28.390+0000 7f3cc2000700 1 --2- 172.21.15.153:0/830354920 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f3ca806c4e0 0x7f3ca806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:05:28.392 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:28.391+0000 7f3cc2000700 1 --2- 172.21.15.153:0/830354920 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f3ca806c4e0 0x7f3ca806e9a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3cb0005950 tx=0x7f3cb00058e0 comp rx=0 tx=0).ready entity=mgr.14221 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:05:28.397 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:28.395+0000 7f3cbb400700 1 -- 172.21.15.153:0/830354920 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f3cb405ae40 con 0x7f3cc4102dd0 2024-03-21T10:05:28.614 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:28.612+0000 7f3cc9600700 1 -- 172.21.15.153:0/830354920 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}] v 0) v1 -- 0x7f3cc419e770 con 0x7f3cc4102dd0 2024-03-21T10:05:28.615 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:28.613+0000 7f3cbb400700 1 -- 172.21.15.153:0/830354920 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}]=0 v21)=0 v21) v1 ==== 155+0+0 (secure 0 0 0) 0x7f3cb405a9d0 con 0x7f3cc4102dd0 2024-03-21T10:05:28.617 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:28.616+0000 7f3cc9600700 1 -- 172.21.15.153:0/830354920 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f3ca806c4e0 msgr2=0x7f3ca806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:05:28.618 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:28.616+0000 7f3cc9600700 1 --2- 172.21.15.153:0/830354920 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f3ca806c4e0 0x7f3ca806e9a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3cb0005950 tx=0x7f3cb00058e0 comp rx=0 tx=0).stop 2024-03-21T10:05:28.618 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:28.616+0000 7f3cc9600700 1 -- 172.21.15.153:0/830354920 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3cc4102dd0 msgr2=0x7f3cc4198580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:05:28.618 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:28.616+0000 7f3cc9600700 1 --2- 172.21.15.153:0/830354920 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3cc4102dd0 0x7f3cc4198580 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f3cb400bda0 tx=0x7f3cb400be80 comp rx=0 tx=0).stop 2024-03-21T10:05:28.618 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:28.616+0000 7f3cc9600700 1 -- 172.21.15.153:0/830354920 shutdown_connections 2024-03-21T10:05:28.619 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:28.616+0000 7f3cc9600700 1 --2- 172.21.15.153:0/830354920 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f3ca806c4e0 0x7f3ca806e9a0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:05:28.619 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:28.616+0000 7f3cc9600700 1 --2- 172.21.15.153:0/830354920 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3cc4102dd0 0x7f3cc4198580 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:05:28.619 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:28.616+0000 7f3cc9600700 1 --2- 172.21.15.153:0/830354920 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3cc4103fd0 0x7f3cc4198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:05:28.619 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:28.616+0000 7f3cc9600700 1 -- 172.21.15.153:0/830354920 >> 172.21.15.153:0/830354920 conn(0x7f3cc40fe370 msgr2=0x7f3cc4107200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:05:28.619 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:28.617+0000 7f3cc9600700 1 -- 172.21.15.153:0/830354920 shutdown_connections 2024-03-21T10:05:28.619 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:28.617+0000 7f3cc9600700 1 -- 172.21.15.153:0/830354920 wait complete. 2024-03-21T10:05:29.444 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 070ed3e4-e769-11ee-95cd-87774f69a715 -e sha1=2edfee68ad7243bed422a049b6f5f58619300f11 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force' 2024-03-21T10:05:29.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:05:29 smithi174 ceph-mon[37496]: pgmap v247: 65 pgs: 65 active+clean; 585 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.9 KiB/s rd, 682 B/s wr, 4 op/s 2024-03-21T10:05:29.707 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:05:29 smithi153 ceph-mon[29117]: pgmap v247: 65 pgs: 65 active+clean; 585 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.9 KiB/s rd, 682 B/s wr, 4 op/s 2024-03-21T10:05:29.901 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi153/config 2024-03-21T10:05:31.699 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:05:31 smithi153 ceph-mon[29117]: pgmap v248: 65 pgs: 65 active+clean; 585 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.8 KiB/s rd, 596 B/s wr, 4 op/s 2024-03-21T10:05:31.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:05:31 smithi174 ceph-mon[37496]: pgmap v248: 65 pgs: 65 active+clean; 585 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.8 KiB/s rd, 596 B/s wr, 4 op/s 2024-03-21T10:05:31.997 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:31.995+0000 7fc035e00700 1 -- 172.21.15.153:0/1245378062 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc030107200 msgr2=0x7fc030107620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:05:31.998 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:31.995+0000 7fc035e00700 1 --2- 172.21.15.153:0/1245378062 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc030107200 0x7fc030107620 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fc020009910 tx=0x7fc020009c20 comp rx=0 tx=0).stop 2024-03-21T10:05:31.998 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:31.995+0000 7fc035e00700 1 -- 172.21.15.153:0/1245378062 shutdown_connections 2024-03-21T10:05:31.998 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:31.995+0000 7fc035e00700 1 --2- 172.21.15.153:0/1245378062 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc030108400 0x7fc030108860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:05:31.998 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:31.995+0000 7fc035e00700 1 --2- 172.21.15.153:0/1245378062 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc030107200 0x7fc030107620 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:05:31.998 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:31.995+0000 7fc035e00700 1 -- 172.21.15.153:0/1245378062 >> 172.21.15.153:0/1245378062 conn(0x7fc0300761b0 msgr2=0x7fc030078610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:05:31.998 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:31.996+0000 7fc035e00700 1 -- 172.21.15.153:0/1245378062 shutdown_connections 2024-03-21T10:05:31.998 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:31.996+0000 7fc035e00700 1 -- 172.21.15.153:0/1245378062 wait complete. 2024-03-21T10:05:31.999 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:31.997+0000 7fc035e00700 1 Processor -- start 2024-03-21T10:05:31.999 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:31.998+0000 7fc035e00700 1 -- start start 2024-03-21T10:05:32.000 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:31.998+0000 7fc035e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc030107200 0x7fc03019ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:05:32.000 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:31.998+0000 7fc035e00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc030108400 0x7fc03019cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:05:32.000 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:31.998+0000 7fc035e00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc03019d590 con 0x7fc030107200 2024-03-21T10:05:32.001 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:31.998+0000 7fc035e00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc03019d6d0 con 0x7fc030108400 2024-03-21T10:05:32.001 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:31.999+0000 7fc02ea00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc030107200 0x7fc03019ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:05:32.001 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:31.999+0000 7fc02ea00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc030107200 0x7fc03019ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:51874/0 (socket says 172.21.15.153:51874) 2024-03-21T10:05:32.001 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:31.999+0000 7fc02ea00700 1 -- 172.21.15.153:0/2702617053 learned_addr learned my addr 172.21.15.153:0/2702617053 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:05:32.001 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:31.999+0000 7fc02ea00700 1 -- 172.21.15.153:0/2702617053 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc030108400 msgr2=0x7fc03019cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:05:32.001 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:31.999+0000 7fc02ea00700 1 --2- 172.21.15.153:0/2702617053 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc030108400 0x7fc03019cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:05:32.001 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:31.999+0000 7fc02ea00700 1 -- 172.21.15.153:0/2702617053 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0200095f0 con 0x7fc030107200 2024-03-21T10:05:32.001 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:31.999+0000 7fc02ea00700 1 --2- 172.21.15.153:0/2702617053 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc030107200 0x7fc03019ca30 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fc02000bda0 tx=0x7fc02000be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:05:32.002 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:32.000+0000 7fc034a00700 1 -- 172.21.15.153:0/2702617053 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc02001d070 con 0x7fc030107200 2024-03-21T10:05:32.002 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:32.000+0000 7fc034a00700 1 -- 172.21.15.153:0/2702617053 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc020004c50 con 0x7fc030107200 2024-03-21T10:05:32.002 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:32.000+0000 7fc034a00700 1 -- 172.21.15.153:0/2702617053 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc02000f690 con 0x7fc030107200 2024-03-21T10:05:32.002 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:32.000+0000 7fc035e00700 1 -- 172.21.15.153:0/2702617053 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0301a2120 con 0x7fc030107200 2024-03-21T10:05:32.002 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:32.000+0000 7fc035e00700 1 -- 172.21.15.153:0/2702617053 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0301a2690 con 0x7fc030107200 2024-03-21T10:05:32.003 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:32.001+0000 7fc035e00700 1 -- 172.21.15.153:0/2702617053 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc03010fde0 con 0x7fc030107200 2024-03-21T10:05:32.003 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:32.002+0000 7fc034a00700 1 -- 172.21.15.153:0/2702617053 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fc02000f7f0 con 0x7fc030107200 2024-03-21T10:05:32.004 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:32.003+0000 7fc034a00700 1 --2- 172.21.15.153:0/2702617053 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fc01406c4e0 0x7fc01406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:05:32.004 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:32.003+0000 7fc034a00700 1 -- 172.21.15.153:0/2702617053 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fc02008cfd0 con 0x7fc030107200 2024-03-21T10:05:32.004 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:32.003+0000 7fc027e00700 1 --2- 172.21.15.153:0/2702617053 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fc01406c4e0 0x7fc01406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:05:32.005 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:32.004+0000 7fc027e00700 1 --2- 172.21.15.153:0/2702617053 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fc01406c4e0 0x7fc01406e9a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc0180097b0 tx=0x7fc018006d20 comp rx=0 tx=0).ready entity=mgr.14221 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:05:32.009 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:32.008+0000 7fc034a00700 1 -- 172.21.15.153:0/2702617053 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fc02005b380 con 0x7fc030107200 2024-03-21T10:05:32.230 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:32.228+0000 7fc035e00700 1 -- 172.21.15.153:0/2702617053 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}] v 0) v1 -- 0x7fc0301a2d20 con 0x7fc030107200 2024-03-21T10:05:32.230 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:32.229+0000 7fc034a00700 1 -- 172.21.15.153:0/2702617053 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}]=0 v21)=0 v21) v1 ==== 163+0+0 (secure 0 0 0) 0x7fc02005af10 con 0x7fc030107200 2024-03-21T10:05:32.234 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:32.231+0000 7fc035e00700 1 -- 172.21.15.153:0/2702617053 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fc01406c4e0 msgr2=0x7fc01406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:05:32.234 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:32.231+0000 7fc035e00700 1 --2- 172.21.15.153:0/2702617053 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fc01406c4e0 0x7fc01406e9a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc0180097b0 tx=0x7fc018006d20 comp rx=0 tx=0).stop 2024-03-21T10:05:32.234 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:32.231+0000 7fc035e00700 1 -- 172.21.15.153:0/2702617053 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc030107200 msgr2=0x7fc03019ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:05:32.234 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:32.231+0000 7fc035e00700 1 --2- 172.21.15.153:0/2702617053 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc030107200 0x7fc03019ca30 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fc02000bda0 tx=0x7fc02000be80 comp rx=0 tx=0).stop 2024-03-21T10:05:32.234 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:32.232+0000 7fc035e00700 1 -- 172.21.15.153:0/2702617053 shutdown_connections 2024-03-21T10:05:32.234 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:32.232+0000 7fc035e00700 1 --2- 172.21.15.153:0/2702617053 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fc01406c4e0 0x7fc01406e9a0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:05:32.234 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:32.232+0000 7fc035e00700 1 --2- 172.21.15.153:0/2702617053 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc030107200 0x7fc03019ca30 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:05:32.234 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:32.232+0000 7fc035e00700 1 --2- 172.21.15.153:0/2702617053 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc030108400 0x7fc03019cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:05:32.234 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:32.232+0000 7fc035e00700 1 -- 172.21.15.153:0/2702617053 >> 172.21.15.153:0/2702617053 conn(0x7fc0300761b0 msgr2=0x7fc03010b630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:05:32.234 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:32.232+0000 7fc035e00700 1 -- 172.21.15.153:0/2702617053 shutdown_connections 2024-03-21T10:05:32.235 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:32.232+0000 7fc035e00700 1 -- 172.21.15.153:0/2702617053 wait complete. 2024-03-21T10:05:32.979 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 070ed3e4-e769-11ee-95cd-87774f69a715 -e sha1=2edfee68ad7243bed422a049b6f5f58619300f11 -- bash -c 'ceph config set global log_to_journald false --force' 2024-03-21T10:05:33.392 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi153/config 2024-03-21T10:05:33.719 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:05:33 smithi153 ceph-mon[29117]: pgmap v249: 65 pgs: 65 active+clean; 587 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 3.1 KiB/s rd, 852 B/s wr, 6 op/s 2024-03-21T10:05:33.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:05:33 smithi174 ceph-mon[37496]: pgmap v249: 65 pgs: 65 active+clean; 587 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 3.1 KiB/s rd, 852 B/s wr, 6 op/s 2024-03-21T10:05:35.338 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:05:35 smithi153 ceph-mon[29117]: pgmap v250: 65 pgs: 65 active+clean; 587 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s rd, 682 B/s wr, 4 op/s 2024-03-21T10:05:35.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:05:35 smithi174 ceph-mon[37496]: pgmap v250: 65 pgs: 65 active+clean; 587 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s rd, 682 B/s wr, 4 op/s 2024-03-21T10:05:35.514 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:35.512+0000 7fc499600700 1 -- 172.21.15.153:0/1690420890 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc4940ff430 msgr2=0x7fc4940ff850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:05:35.515 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:35.512+0000 7fc499600700 1 --2- 172.21.15.153:0/1690420890 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc4940ff430 0x7fc4940ff850 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fc484009910 tx=0x7fc484009c20 comp rx=0 tx=0).stop 2024-03-21T10:05:35.515 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:35.512+0000 7fc499600700 1 -- 172.21.15.153:0/1690420890 shutdown_connections 2024-03-21T10:05:35.515 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:35.512+0000 7fc499600700 1 --2- 172.21.15.153:0/1690420890 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc4940ffd90 0x7fc494100210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:05:35.515 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:35.512+0000 7fc499600700 1 --2- 172.21.15.153:0/1690420890 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc4940ff430 0x7fc4940ff850 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:05:35.515 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:35.513+0000 7fc499600700 1 -- 172.21.15.153:0/1690420890 >> 172.21.15.153:0/1690420890 conn(0x7fc4940fb030 msgr2=0x7fc4940fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:05:35.515 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:35.513+0000 7fc499600700 1 -- 172.21.15.153:0/1690420890 shutdown_connections 2024-03-21T10:05:35.515 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:35.513+0000 7fc499600700 1 -- 172.21.15.153:0/1690420890 wait complete. 2024-03-21T10:05:35.516 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:35.514+0000 7fc499600700 1 Processor -- start 2024-03-21T10:05:35.516 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:35.514+0000 7fc499600700 1 -- start start 2024-03-21T10:05:35.516 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:35.514+0000 7fc499600700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc4940ff430 0x7fc494198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:05:35.516 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:35.514+0000 7fc499600700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc4940ffd90 0x7fc494198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:05:35.516 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:35.514+0000 7fc499600700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc494199060 con 0x7fc4940ffd90 2024-03-21T10:05:35.516 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:35.514+0000 7fc499600700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4941991a0 con 0x7fc4940ff430 2024-03-21T10:05:35.517 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:35.515+0000 7fc492000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc4940ffd90 0x7fc494198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:05:35.517 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:35.515+0000 7fc492000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc4940ffd90 0x7fc494198a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:55006/0 (socket says 172.21.15.153:55006) 2024-03-21T10:05:35.517 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:35.515+0000 7fc492000700 1 -- 172.21.15.153:0/1931844058 learned_addr learned my addr 172.21.15.153:0/1931844058 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:05:35.518 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:35.515+0000 7fc492a00700 1 --2- 172.21.15.153:0/1931844058 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc4940ff430 0x7fc494198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:05:35.518 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:35.515+0000 7fc492000700 1 -- 172.21.15.153:0/1931844058 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc4940ff430 msgr2=0x7fc494198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:05:35.518 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:35.515+0000 7fc492000700 1 --2- 172.21.15.153:0/1931844058 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc4940ff430 0x7fc494198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:05:35.518 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:35.515+0000 7fc492000700 1 -- 172.21.15.153:0/1931844058 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4840095f0 con 0x7fc4940ffd90 2024-03-21T10:05:35.518 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:35.515+0000 7fc492a00700 1 --2- 172.21.15.153:0/1931844058 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc4940ff430 0x7fc494198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:05:35.518 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:35.516+0000 7fc492000700 1 --2- 172.21.15.153:0/1931844058 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc4940ffd90 0x7fc494198a40 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fc48000d8d0 tx=0x7fc48000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:05:35.518 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:35.516+0000 7fc48b400700 1 -- 172.21.15.153:0/1931844058 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc480009940 con 0x7fc4940ffd90 2024-03-21T10:05:35.518 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:35.516+0000 7fc48b400700 1 -- 172.21.15.153:0/1931844058 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc480010460 con 0x7fc4940ffd90 2024-03-21T10:05:35.518 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:35.516+0000 7fc48b400700 1 -- 172.21.15.153:0/1931844058 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc48000f5f0 con 0x7fc4940ffd90 2024-03-21T10:05:35.519 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:35.516+0000 7fc499600700 1 -- 172.21.15.153:0/1931844058 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc49419dc50 con 0x7fc4940ffd90 2024-03-21T10:05:35.519 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:35.516+0000 7fc499600700 1 -- 172.21.15.153:0/1931844058 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc49419e220 con 0x7fc4940ffd90 2024-03-21T10:05:35.519 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:35.517+0000 7fc499600700 1 -- 172.21.15.153:0/1931844058 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc49410b900 con 0x7fc4940ffd90 2024-03-21T10:05:35.519 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:35.518+0000 7fc48b400700 1 -- 172.21.15.153:0/1931844058 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fc4800105d0 con 0x7fc4940ffd90 2024-03-21T10:05:35.521 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:35.520+0000 7fc48b400700 1 --2- 172.21.15.153:0/1931844058 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fc47806c4e0 0x7fc47806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:05:35.521 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:35.520+0000 7fc48b400700 1 -- 172.21.15.153:0/1931844058 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fc48008b830 con 0x7fc4940ffd90 2024-03-21T10:05:35.521 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:35.520+0000 7fc492a00700 1 --2- 172.21.15.153:0/1931844058 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fc47806c4e0 0x7fc47806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:05:35.522 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:35.521+0000 7fc492a00700 1 --2- 172.21.15.153:0/1931844058 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fc47806c4e0 0x7fc47806e9a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc484005090 tx=0x7fc48400b560 comp rx=0 tx=0).ready entity=mgr.14221 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:05:35.526 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:35.525+0000 7fc48b400700 1 -- 172.21.15.153:0/1931844058 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fc480059a20 con 0x7fc4940ffd90 2024-03-21T10:05:35.741 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:35.739+0000 7fc499600700 1 -- 172.21.15.153:0/1931844058 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command([{prefix=config set, name=log_to_journald}] v 0) v1 -- 0x7fc49419e9d0 con 0x7fc4940ffd90 2024-03-21T10:05:35.741 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:35.740+0000 7fc48b400700 1 -- 172.21.15.153:0/1931844058 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{prefix=config set, name=log_to_journald}]=0 v21)=0 v21) v1 ==== 135+0+0 (secure 0 0 0) 0x7fc48008b0c0 con 0x7fc4940ffd90 2024-03-21T10:05:35.745 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:35.743+0000 7fc499600700 1 -- 172.21.15.153:0/1931844058 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fc47806c4e0 msgr2=0x7fc47806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:05:35.745 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:35.743+0000 7fc499600700 1 --2- 172.21.15.153:0/1931844058 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fc47806c4e0 0x7fc47806e9a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc484005090 tx=0x7fc48400b560 comp rx=0 tx=0).stop 2024-03-21T10:05:35.745 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:35.743+0000 7fc499600700 1 -- 172.21.15.153:0/1931844058 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc4940ffd90 msgr2=0x7fc494198a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:05:35.745 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:35.743+0000 7fc499600700 1 --2- 172.21.15.153:0/1931844058 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc4940ffd90 0x7fc494198a40 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fc48000d8d0 tx=0x7fc48000dc90 comp rx=0 tx=0).stop 2024-03-21T10:05:35.745 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:35.743+0000 7fc499600700 1 -- 172.21.15.153:0/1931844058 shutdown_connections 2024-03-21T10:05:35.745 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:35.743+0000 7fc499600700 1 --2- 172.21.15.153:0/1931844058 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fc47806c4e0 0x7fc47806e9a0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:05:35.745 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:35.743+0000 7fc499600700 1 --2- 172.21.15.153:0/1931844058 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc4940ff430 0x7fc494198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:05:35.745 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:35.743+0000 7fc499600700 1 --2- 172.21.15.153:0/1931844058 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc4940ffd90 0x7fc494198a40 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:05:35.746 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:35.743+0000 7fc499600700 1 -- 172.21.15.153:0/1931844058 >> 172.21.15.153:0/1931844058 conn(0x7fc4940fb030 msgr2=0x7fc494107a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:05:35.746 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:35.744+0000 7fc499600700 1 -- 172.21.15.153:0/1931844058 shutdown_connections 2024-03-21T10:05:35.746 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:35.744+0000 7fc499600700 1 -- 172.21.15.153:0/1931844058 wait complete. 2024-03-21T10:05:36.597 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 070ed3e4-e769-11ee-95cd-87774f69a715 -e sha1=2edfee68ad7243bed422a049b6f5f58619300f11 -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 --daemon-types mgr' 2024-03-21T10:05:37.029 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi153/config 2024-03-21T10:05:37.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:05:37 smithi174 ceph-mon[37496]: pgmap v251: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 3.0 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-03-21T10:05:37.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:05:37 smithi153 ceph-mon[29117]: pgmap v251: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 3.0 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-03-21T10:05:39.319 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:39.317+0000 7f5a2d400700 1 -- 172.21.15.153:0/210883445 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5a28074dc0 msgr2=0x7f5a28073220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:05:39.319 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:39.317+0000 7f5a2d400700 1 --2- 172.21.15.153:0/210883445 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5a28074dc0 0x7f5a28073220 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f5a18009910 tx=0x7f5a18009c20 comp rx=0 tx=0).stop 2024-03-21T10:05:39.319 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:39.317+0000 7f5a2d400700 1 -- 172.21.15.153:0/210883445 shutdown_connections 2024-03-21T10:05:39.320 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:39.317+0000 7f5a2d400700 1 --2- 172.21.15.153:0/210883445 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5a280737f0 0x7f5a28073c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:05:39.320 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:39.317+0000 7f5a2d400700 1 --2- 172.21.15.153:0/210883445 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5a28074dc0 0x7f5a28073220 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:05:39.320 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:39.317+0000 7f5a2d400700 1 -- 172.21.15.153:0/210883445 >> 172.21.15.153:0/210883445 conn(0x7f5a280fc140 msgr2=0x7f5a280fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:05:39.320 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:39.317+0000 7f5a2d400700 1 -- 172.21.15.153:0/210883445 shutdown_connections 2024-03-21T10:05:39.320 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:39.317+0000 7f5a2d400700 1 -- 172.21.15.153:0/210883445 wait complete. 2024-03-21T10:05:39.321 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:39.319+0000 7f5a2d400700 1 Processor -- start 2024-03-21T10:05:39.321 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:39.319+0000 7f5a2d400700 1 -- start start 2024-03-21T10:05:39.321 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:39.319+0000 7f5a2d400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5a280737f0 0x7f5a28198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:05:39.321 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:39.319+0000 7f5a2d400700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5a28074dc0 0x7f5a28198b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:05:39.321 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:39.319+0000 7f5a2d400700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a281991b0 con 0x7f5a280737f0 2024-03-21T10:05:39.321 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:39.319+0000 7f5a2d400700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a281992f0 con 0x7f5a28074dc0 2024-03-21T10:05:39.321 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:39.320+0000 7f5a26800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5a280737f0 0x7f5a28198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:05:39.321 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:39.320+0000 7f5a26800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5a280737f0 0x7f5a28198650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:55020/0 (socket says 172.21.15.153:55020) 2024-03-21T10:05:39.321 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:39.320+0000 7f5a26800700 1 -- 172.21.15.153:0/1036455165 learned_addr learned my addr 172.21.15.153:0/1036455165 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:05:39.323 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:39.320+0000 7f5a1de00700 1 --2- 172.21.15.153:0/1036455165 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5a28074dc0 0x7f5a28198b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:05:39.323 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:39.320+0000 7f5a26800700 1 -- 172.21.15.153:0/1036455165 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5a28074dc0 msgr2=0x7f5a28198b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:05:39.323 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:39.320+0000 7f5a26800700 1 --2- 172.21.15.153:0/1036455165 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5a28074dc0 0x7f5a28198b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:05:39.323 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:39.320+0000 7f5a26800700 1 -- 172.21.15.153:0/1036455165 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a180095f0 con 0x7f5a280737f0 2024-03-21T10:05:39.323 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:39.320+0000 7f5a1de00700 1 --2- 172.21.15.153:0/1036455165 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5a28074dc0 0x7f5a28198b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:05:39.323 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:39.320+0000 7f5a26800700 1 --2- 172.21.15.153:0/1036455165 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5a280737f0 0x7f5a28198650 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f5a1800bd70 tx=0x7f5a1800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:05:39.323 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:39.321+0000 7f5a1fe00700 1 -- 172.21.15.153:0/1036455165 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5a1801d070 con 0x7f5a280737f0 2024-03-21T10:05:39.323 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:39.321+0000 7f5a1fe00700 1 -- 172.21.15.153:0/1036455165 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f5a1800f460 con 0x7f5a280737f0 2024-03-21T10:05:39.323 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:39.321+0000 7f5a1fe00700 1 -- 172.21.15.153:0/1036455165 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5a18021640 con 0x7f5a280737f0 2024-03-21T10:05:39.323 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:39.321+0000 7f5a2d400700 1 -- 172.21.15.153:0/1036455165 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a2819dd40 con 0x7f5a280737f0 2024-03-21T10:05:39.323 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:39.321+0000 7f5a2d400700 1 -- 172.21.15.153:0/1036455165 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a2819e190 con 0x7f5a280737f0 2024-03-21T10:05:39.324 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:39.322+0000 7f5a2d400700 1 -- 172.21.15.153:0/1036455165 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a2810b9d0 con 0x7f5a280737f0 2024-03-21T10:05:39.324 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:39.323+0000 7f5a1fe00700 1 -- 172.21.15.153:0/1036455165 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f5a1800fab0 con 0x7f5a280737f0 2024-03-21T10:05:39.325 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:39.324+0000 7f5a1fe00700 1 --2- 172.21.15.153:0/1036455165 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f5a0c06c4e0 0x7f5a0c06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:05:39.326 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:39.324+0000 7f5a1fe00700 1 -- 172.21.15.153:0/1036455165 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f5a1808cdb0 con 0x7f5a280737f0 2024-03-21T10:05:39.326 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:39.324+0000 7f5a1de00700 1 --2- 172.21.15.153:0/1036455165 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f5a0c06c4e0 0x7f5a0c06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:05:39.326 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:39.325+0000 7f5a1de00700 1 --2- 172.21.15.153:0/1036455165 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f5a0c06c4e0 0x7f5a0c06e9a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5a28074af0 tx=0x7f5a10008040 comp rx=0 tx=0).ready entity=mgr.14221 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:05:39.330 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:39.329+0000 7f5a1fe00700 1 -- 172.21.15.153:0/1036455165 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f5a1805af20 con 0x7f5a280737f0 2024-03-21T10:05:39.579 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:05:39.577+0000 7f5a2d400700 1 -- 172.21.15.153:0/1036455165 --> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] -- mgr_command(tid 0: {"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11", "daemon_types": "mgr", "target": ["mon-mgr", ""]}) v1 -- 0x7f5a28108170 con 0x7f5a0c06c4e0 2024-03-21T10:05:39.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:05:39 smithi174 ceph-mon[37496]: pgmap v252: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 938 B/s wr, 4 op/s 2024-03-21T10:05:39.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:05:39 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:05:39.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:05:39 smithi153 ceph-mon[29117]: pgmap v252: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 938 B/s wr, 4 op/s 2024-03-21T10:05:39.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:05:39 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:05:40.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:05:40 smithi153 ceph-mon[29117]: from='client.14560 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11", "daemon_types": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:05:40.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:05:40 smithi174 ceph-mon[37496]: from='client.14560 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11", "daemon_types": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:05:41.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:05:41 smithi153 ceph-mon[29117]: pgmap v253: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 767 B/s wr, 4 op/s 2024-03-21T10:05:41.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:05:41 smithi174 ceph-mon[37496]: pgmap v253: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 767 B/s wr, 4 op/s 2024-03-21T10:05:43.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:05:43 smithi153 ceph-mon[29117]: pgmap v254: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 3.2 KiB/s rd, 767 B/s wr, 5 op/s 2024-03-21T10:05:43.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:05:43 smithi174 ceph-mon[37496]: pgmap v254: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 3.2 KiB/s rd, 767 B/s wr, 5 op/s 2024-03-21T10:05:45.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:05:45 smithi174 ceph-mon[37496]: pgmap v255: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 511 B/s wr, 3 op/s 2024-03-21T10:05:45.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:05:45 smithi153 ceph-mon[29117]: pgmap v255: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 511 B/s wr, 3 op/s 2024-03-21T10:05:47.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:05:47 smithi174 ceph-mon[37496]: pgmap v256: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.8 KiB/s rd, 511 B/s wr, 4 op/s 2024-03-21T10:05:47.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:05:47 smithi153 ceph-mon[29117]: pgmap v256: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.8 KiB/s rd, 511 B/s wr, 4 op/s 2024-03-21T10:05:49.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:05:49 smithi174 ceph-mon[37496]: pgmap v257: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 85 B/s wr, 3 op/s 2024-03-21T10:05:49.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:05:49 smithi153 ceph-mon[29117]: pgmap v257: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 85 B/s wr, 3 op/s 2024-03-21T10:05:51.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:05:51 smithi153 ceph-mon[29117]: pgmap v258: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-03-21T10:05:51.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:05:51 smithi174 ceph-mon[37496]: pgmap v258: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-03-21T10:05:53.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:05:53 smithi153 ceph-mon[29117]: pgmap v259: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.8 KiB/s rd, 5 op/s 2024-03-21T10:05:53.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:05:53 smithi174 ceph-mon[37496]: pgmap v259: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.8 KiB/s rd, 5 op/s 2024-03-21T10:05:55.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:05:55 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:05:55.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:05:55 smithi174 ceph-mon[37496]: pgmap v260: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:05:55.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:05:55 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:05:55.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:05:55 smithi153 ceph-mon[29117]: pgmap v260: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:05:57.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:05:57 smithi153 ceph-mon[29117]: pgmap v261: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:05:57.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:05:57 smithi174 ceph-mon[37496]: pgmap v261: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:05:59.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:05:59 smithi153 ceph-mon[29117]: pgmap v262: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:05:59.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:05:59 smithi174 ceph-mon[37496]: pgmap v262: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:06:01.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:06:01 smithi153 ceph-mon[29117]: pgmap v263: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:06:01.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:06:01 smithi174 ceph-mon[37496]: pgmap v263: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:06:03.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:06:03 smithi153 ceph-mon[29117]: pgmap v264: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:06:03.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:06:03 smithi174 ceph-mon[37496]: pgmap v264: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:06:05.454 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:06:05 smithi174 ceph-mon[37496]: pgmap v265: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:06:05.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:06:05 smithi153 ceph-mon[29117]: pgmap v265: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:06:06.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:06:06 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:06:06.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:06:06 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:06:07.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:06:07 smithi174 ceph-mon[37496]: pgmap v266: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:06:07.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:06:07 smithi153 ceph-mon[29117]: pgmap v266: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:06:09.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:06:09 smithi153 ceph-mon[29117]: pgmap v267: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:06:09.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:06:09 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:06:09.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:06:09 smithi174 ceph-mon[37496]: pgmap v267: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:06:09.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:06:09 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:06:11.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:06:11 smithi153 ceph-mon[29117]: pgmap v268: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:06:11.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:06:11 smithi174 ceph-mon[37496]: pgmap v268: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:06:13.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:06:13 smithi153 ceph-mon[29117]: pgmap v269: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:06:13.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:06:13 smithi174 ceph-mon[37496]: pgmap v269: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:06:15.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:06:15 smithi174 ceph-mon[37496]: pgmap v270: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:06:15.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:06:15 smithi153 ceph-mon[29117]: pgmap v270: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:06:17.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:06:17 smithi153 ceph-mon[29117]: pgmap v271: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:06:17.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:06:17 smithi174 ceph-mon[37496]: pgmap v271: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:06:18.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:06:18 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:06:18.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:06:18 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:06:18.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:06:18 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:06:18.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:06:18 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:06:18.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:06:18 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:06:18.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:06:18 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:06:19.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:06:19 smithi153 ceph-mon[29117]: pgmap v272: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:06:19.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:06:19 smithi174 ceph-mon[37496]: pgmap v272: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:06:21.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:06:21 smithi153 ceph-mon[29117]: pgmap v273: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:06:21.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:06:21 smithi174 ceph-mon[37496]: pgmap v273: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:06:23.649 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:06:23 smithi153 ceph-mon[29117]: pgmap v274: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:06:23.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:06:23 smithi174 ceph-mon[37496]: pgmap v274: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:06:24.615 INFO:teuthology.orchestra.run.smithi153.stdout:Initiating upgrade to quay.ceph.io/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 2024-03-21T10:06:24.615 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:24.613+0000 7f5a1fe00700 1 -- 172.21.15.153:0/1036455165 <== mgr.14221 v2:172.21.15.153:6800/1639929315 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+89 (secure 0 0 0) 0x7f5a28108170 con 0x7f5a0c06c4e0 2024-03-21T10:06:24.618 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:24.616+0000 7f5a2d400700 1 -- 172.21.15.153:0/1036455165 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f5a0c06c4e0 msgr2=0x7f5a0c06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:06:24.618 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:24.616+0000 7f5a2d400700 1 --2- 172.21.15.153:0/1036455165 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f5a0c06c4e0 0x7f5a0c06e9a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5a28074af0 tx=0x7f5a10008040 comp rx=0 tx=0).stop 2024-03-21T10:06:24.618 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:24.616+0000 7f5a2d400700 1 -- 172.21.15.153:0/1036455165 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5a280737f0 msgr2=0x7f5a28198650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:06:24.618 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:24.616+0000 7f5a2d400700 1 --2- 172.21.15.153:0/1036455165 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5a280737f0 0x7f5a28198650 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f5a1800bd70 tx=0x7f5a1800be50 comp rx=0 tx=0).stop 2024-03-21T10:06:24.619 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:24.617+0000 7f5a2d400700 1 -- 172.21.15.153:0/1036455165 shutdown_connections 2024-03-21T10:06:24.619 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:24.617+0000 7f5a2d400700 1 --2- 172.21.15.153:0/1036455165 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f5a0c06c4e0 0x7f5a0c06e9a0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:06:24.619 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:24.617+0000 7f5a2d400700 1 --2- 172.21.15.153:0/1036455165 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5a280737f0 0x7f5a28198650 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:06:24.619 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:24.617+0000 7f5a2d400700 1 --2- 172.21.15.153:0/1036455165 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5a28074dc0 0x7f5a28198b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:06:24.619 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:24.617+0000 7f5a2d400700 1 -- 172.21.15.153:0/1036455165 >> 172.21.15.153:0/1036455165 conn(0x7f5a280fc140 msgr2=0x7f5a281069a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:06:24.619 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:24.617+0000 7f5a2d400700 1 -- 172.21.15.153:0/1036455165 shutdown_connections 2024-03-21T10:06:24.619 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:24.617+0000 7f5a2d400700 1 -- 172.21.15.153:0/1036455165 wait complete. 2024-03-21T10:06:25.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:06:25 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:06:25.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:06:25 smithi153 ceph-mon[29117]: pgmap v275: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:06:25.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:06:25 smithi153 ceph-mon[29117]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 2024-03-21T10:06:25.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:06:25 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:06:25.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:06:25 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:06:25.432 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 070ed3e4-e769-11ee-95cd-87774f69a715 -e sha1=2edfee68ad7243bed422a049b6f5f58619300f11 -- bash -c 'while ceph orch upgrade status | jq '"'"'.in_progress'"'"' | grep true && ! ceph orch upgrade status | jq '"'"'.message'"'"' | grep Error ; do ceph orch ps ; ceph versions ; ceph orch upgrade status ; sleep 30 ; done' 2024-03-21T10:06:25.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:06:25 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:06:25.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:06:25 smithi174 ceph-mon[37496]: pgmap v275: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:06:25.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:06:25 smithi174 ceph-mon[37496]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 2024-03-21T10:06:25.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:06:25 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:06:25.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:06:25 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:06:25.908 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi153/config 2024-03-21T10:06:26.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:06:26 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:06:26.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:06:26 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:06:26.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:06:26 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:06:26.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:06:26 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:06:26.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:06:26 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:06:26.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:06:26 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:06:26.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:06:26 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:06:26.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:06:26 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:06:26.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:06:26 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:06:26.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:06:26 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:06:27.892 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:06:27 smithi153 ceph-mon[29117]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 2024-03-21T10:06:27.892 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:06:27 smithi153 ceph-mon[29117]: pgmap v276: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:06:27.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:06:27 smithi174 ceph-mon[37496]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 2024-03-21T10:06:27.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:06:27 smithi174 ceph-mon[37496]: pgmap v276: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:06:28.328 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:28.326+0000 7fa10ac00700 1 -- 172.21.15.153:0/2915151455 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa1040737f0 msgr2=0x7fa104073c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:06:28.328 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:28.326+0000 7fa10ac00700 1 --2- 172.21.15.153:0/2915151455 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa1040737f0 0x7fa104073c70 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa0f0009910 tx=0x7fa0f0009c20 comp rx=0 tx=0).stop 2024-03-21T10:06:28.328 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:28.326+0000 7fa10ac00700 1 -- 172.21.15.153:0/2915151455 shutdown_connections 2024-03-21T10:06:28.328 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:28.326+0000 7fa10ac00700 1 --2- 172.21.15.153:0/2915151455 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa1040737f0 0x7fa104073c70 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:06:28.329 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:28.326+0000 7fa10ac00700 1 --2- 172.21.15.153:0/2915151455 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa104074dc0 0x7fa104073220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:06:28.329 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:28.326+0000 7fa10ac00700 1 -- 172.21.15.153:0/2915151455 >> 172.21.15.153:0/2915151455 conn(0x7fa1040fc110 msgr2=0x7fa1040fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:06:28.329 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:28.326+0000 7fa10ac00700 1 -- 172.21.15.153:0/2915151455 shutdown_connections 2024-03-21T10:06:28.329 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:28.326+0000 7fa10ac00700 1 -- 172.21.15.153:0/2915151455 wait complete. 2024-03-21T10:06:28.329 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:28.327+0000 7fa10ac00700 1 Processor -- start 2024-03-21T10:06:28.330 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:28.327+0000 7fa10ac00700 1 -- start start 2024-03-21T10:06:28.330 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:28.328+0000 7fa10ac00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa1040737f0 0x7fa1041985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:06:28.330 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:28.328+0000 7fa10ac00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa104074dc0 0x7fa104198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:06:28.330 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:28.328+0000 7fa10ac00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa104199110 con 0x7fa1040737f0 2024-03-21T10:06:28.330 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:28.328+0000 7fa10ac00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa104199250 con 0x7fa104074dc0 2024-03-21T10:06:28.331 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:28.328+0000 7fa103e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa1040737f0 0x7fa1041985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:06:28.331 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:28.328+0000 7fa103400700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa104074dc0 0x7fa104198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:06:28.331 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:28.328+0000 7fa103e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa1040737f0 0x7fa1041985b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:57414/0 (socket says 172.21.15.153:57414) 2024-03-21T10:06:28.331 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:28.328+0000 7fa103400700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa104074dc0 0x7fa104198af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.153:60134/0 (socket says 172.21.15.153:60134) 2024-03-21T10:06:28.331 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:28.328+0000 7fa103e00700 1 -- 172.21.15.153:0/291583058 learned_addr learned my addr 172.21.15.153:0/291583058 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:06:28.331 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:28.329+0000 7fa103e00700 1 -- 172.21.15.153:0/291583058 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa104074dc0 msgr2=0x7fa104198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:06:28.331 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:28.329+0000 7fa103e00700 1 --2- 172.21.15.153:0/291583058 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa104074dc0 0x7fa104198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:06:28.331 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:28.329+0000 7fa103e00700 1 -- 172.21.15.153:0/291583058 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0f00095f0 con 0x7fa1040737f0 2024-03-21T10:06:28.331 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:28.329+0000 7fa103400700 1 --2- 172.21.15.153:0/291583058 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa104074dc0 0x7fa104198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:06:28.332 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:28.329+0000 7fa103e00700 1 --2- 172.21.15.153:0/291583058 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa1040737f0 0x7fa1041985b0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa0f400e8f0 tx=0x7fa0f400ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:06:28.332 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:28.330+0000 7fa100c00700 1 -- 172.21.15.153:0/291583058 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa0f4009790 con 0x7fa1040737f0 2024-03-21T10:06:28.332 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:28.330+0000 7fa10ac00700 1 -- 172.21.15.153:0/291583058 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa10419dd00 con 0x7fa1040737f0 2024-03-21T10:06:28.332 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:28.330+0000 7fa10ac00700 1 -- 172.21.15.153:0/291583058 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa10419e1f0 con 0x7fa1040737f0 2024-03-21T10:06:28.332 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:28.330+0000 7fa100c00700 1 -- 172.21.15.153:0/291583058 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa0f400c810 con 0x7fa1040737f0 2024-03-21T10:06:28.332 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:28.330+0000 7fa100c00700 1 -- 172.21.15.153:0/291583058 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa0f40055b0 con 0x7fa1040737f0 2024-03-21T10:06:28.333 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:28.331+0000 7fa10ac00700 1 -- 172.21.15.153:0/291583058 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0e4005320 con 0x7fa1040737f0 2024-03-21T10:06:28.334 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:28.332+0000 7fa100c00700 1 -- 172.21.15.153:0/291583058 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fa0f400c980 con 0x7fa1040737f0 2024-03-21T10:06:28.334 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:28.333+0000 7fa100c00700 1 --2- 172.21.15.153:0/291583058 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fa0e806c4e0 0x7fa0e806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:06:28.334 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:28.333+0000 7fa100c00700 1 -- 172.21.15.153:0/291583058 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fa0f4014070 con 0x7fa1040737f0 2024-03-21T10:06:28.336 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:28.335+0000 7fa103400700 1 --2- 172.21.15.153:0/291583058 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fa0e806c4e0 0x7fa0e806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:06:28.337 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:28.335+0000 7fa103400700 1 --2- 172.21.15.153:0/291583058 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fa0e806c4e0 0x7fa0e806e9a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa0f0009fd0 tx=0x7fa0f0005fd0 comp rx=0 tx=0).ready entity=mgr.14221 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:06:28.339 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:28.338+0000 7fa100c00700 1 -- 172.21.15.153:0/291583058 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fa0f4059c90 con 0x7fa1040737f0 2024-03-21T10:06:28.581 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:28.579+0000 7fa10ac00700 1 -- 172.21.15.153:0/291583058 --> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa0e4000bf0 con 0x7fa0e806c4e0 2024-03-21T10:06:28.582 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:28.580+0000 7fa100c00700 1 -- 172.21.15.153:0/291583058 <== mgr.14221 v2:172.21.15.153:6800/1639929315 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+351 (secure 0 0 0) 0x7fa0e4000bf0 con 0x7fa0e806c4e0 2024-03-21T10:06:28.584 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:28.583+0000 7fa10ac00700 1 -- 172.21.15.153:0/291583058 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fa0e806c4e0 msgr2=0x7fa0e806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:06:28.584 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:28.583+0000 7fa10ac00700 1 --2- 172.21.15.153:0/291583058 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fa0e806c4e0 0x7fa0e806e9a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa0f0009fd0 tx=0x7fa0f0005fd0 comp rx=0 tx=0).stop 2024-03-21T10:06:28.584 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:28.583+0000 7fa10ac00700 1 -- 172.21.15.153:0/291583058 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa1040737f0 msgr2=0x7fa1041985b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:06:28.585 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:28.583+0000 7fa10ac00700 1 --2- 172.21.15.153:0/291583058 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa1040737f0 0x7fa1041985b0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa0f400e8f0 tx=0x7fa0f400ec00 comp rx=0 tx=0).stop 2024-03-21T10:06:28.585 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:28.583+0000 7fa10ac00700 1 -- 172.21.15.153:0/291583058 shutdown_connections 2024-03-21T10:06:28.585 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:28.583+0000 7fa10ac00700 1 --2- 172.21.15.153:0/291583058 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fa0e806c4e0 0x7fa0e806e9a0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:06:28.585 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:28.583+0000 7fa10ac00700 1 --2- 172.21.15.153:0/291583058 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa1040737f0 0x7fa1041985b0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:06:28.586 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:28.583+0000 7fa10ac00700 1 --2- 172.21.15.153:0/291583058 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa104074dc0 0x7fa104198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:06:28.586 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:28.583+0000 7fa10ac00700 1 -- 172.21.15.153:0/291583058 >> 172.21.15.153:0/291583058 conn(0x7fa1040fc110 msgr2=0x7fa104106990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:06:28.586 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:28.584+0000 7fa10ac00700 1 -- 172.21.15.153:0/291583058 shutdown_connections 2024-03-21T10:06:28.586 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:28.584+0000 7fa10ac00700 1 -- 172.21.15.153:0/291583058 wait complete. 2024-03-21T10:06:28.600 INFO:teuthology.orchestra.run.smithi153.stdout:true 2024-03-21T10:06:28.746 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:28.744+0000 7f414e200700 1 -- 172.21.15.153:0/1845359282 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4148102df0 msgr2=0x7f4148103210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:06:28.746 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:28.744+0000 7f414e200700 1 --2- 172.21.15.153:0/1845359282 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4148102df0 0x7f4148103210 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f4138009910 tx=0x7f4138009c20 comp rx=0 tx=0).stop 2024-03-21T10:06:28.746 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:28.744+0000 7f414e200700 1 -- 172.21.15.153:0/1845359282 shutdown_connections 2024-03-21T10:06:28.746 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:28.744+0000 7f414e200700 1 --2- 172.21.15.153:0/1845359282 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4148103ff0 0x7f4148104450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:06:28.746 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:28.744+0000 7f414e200700 1 --2- 172.21.15.153:0/1845359282 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4148102df0 0x7f4148103210 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:06:28.746 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:28.744+0000 7f414e200700 1 -- 172.21.15.153:0/1845359282 >> 172.21.15.153:0/1845359282 conn(0x7f41480fe390 msgr2=0x7f41481007d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:06:28.746 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:28.744+0000 7f414e200700 1 -- 172.21.15.153:0/1845359282 shutdown_connections 2024-03-21T10:06:28.746 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:28.745+0000 7f414e200700 1 -- 172.21.15.153:0/1845359282 wait complete. 2024-03-21T10:06:28.747 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:28.746+0000 7f414e200700 1 Processor -- start 2024-03-21T10:06:28.748 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:28.746+0000 7f414e200700 1 -- start start 2024-03-21T10:06:28.748 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:28.747+0000 7f414e200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4148102df0 0x7f4148078b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:06:28.748 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:28.747+0000 7f414e200700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4148103ff0 0x7f4148079080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:06:28.749 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:28.747+0000 7f414e200700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4148075630 con 0x7f4148102df0 2024-03-21T10:06:28.749 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:28.747+0000 7f414e200700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41480757a0 con 0x7f4148103ff0 2024-03-21T10:06:28.749 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:28.747+0000 7f4147400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4148102df0 0x7f4148078b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:06:28.749 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:28.747+0000 7f4147400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4148102df0 0x7f4148078b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:57432/0 (socket says 172.21.15.153:57432) 2024-03-21T10:06:28.750 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:28.747+0000 7f4147400700 1 -- 172.21.15.153:0/612359168 learned_addr learned my addr 172.21.15.153:0/612359168 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:06:28.750 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:28.748+0000 7f4147400700 1 -- 172.21.15.153:0/612359168 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4148103ff0 msgr2=0x7f4148079080 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-21T10:06:28.750 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:28.748+0000 7f4147400700 1 --2- 172.21.15.153:0/612359168 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4148103ff0 0x7f4148079080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:06:28.750 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:28.748+0000 7f4147400700 1 -- 172.21.15.153:0/612359168 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41380095f0 con 0x7f4148102df0 2024-03-21T10:06:28.750 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:28.748+0000 7f4147400700 1 --2- 172.21.15.153:0/612359168 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4148102df0 0x7f4148078b40 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f413800bd70 tx=0x7f413800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:06:28.750 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:28.748+0000 7f413fe00700 1 -- 172.21.15.153:0/612359168 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f413801d070 con 0x7f4148102df0 2024-03-21T10:06:28.750 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:28.748+0000 7f413fe00700 1 -- 172.21.15.153:0/612359168 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f413800f460 con 0x7f4148102df0 2024-03-21T10:06:28.751 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:28.748+0000 7f413fe00700 1 -- 172.21.15.153:0/612359168 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4138021620 con 0x7f4148102df0 2024-03-21T10:06:28.751 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:28.748+0000 7f414e200700 1 -- 172.21.15.153:0/612359168 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4148075a20 con 0x7f4148102df0 2024-03-21T10:06:28.751 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:28.749+0000 7f414e200700 1 -- 172.21.15.153:0/612359168 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4148075f10 con 0x7f4148102df0 2024-03-21T10:06:28.752 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:28.751+0000 7f414e200700 1 -- 172.21.15.153:0/612359168 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4128005320 con 0x7f4148102df0 2024-03-21T10:06:28.755 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:28.753+0000 7f413fe00700 1 -- 172.21.15.153:0/612359168 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f413800fa40 con 0x7f4148102df0 2024-03-21T10:06:28.755 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:28.753+0000 7f413fe00700 1 --2- 172.21.15.153:0/612359168 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f412c06c530 0x7f412c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:06:28.755 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:28.754+0000 7f413fe00700 1 -- 172.21.15.153:0/612359168 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f413808c630 con 0x7f4148102df0 2024-03-21T10:06:28.756 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:28.754+0000 7f4146a00700 1 --2- 172.21.15.153:0/612359168 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f412c06c530 0x7f412c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:06:28.756 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:28.755+0000 7f4146a00700 1 --2- 172.21.15.153:0/612359168 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f412c06c530 0x7f412c06e9f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4134005fd0 tx=0x7f4134005f60 comp rx=0 tx=0).ready entity=mgr.14221 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:06:28.763 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:28.761+0000 7f413fe00700 1 -- 172.21.15.153:0/612359168 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f413805a8a0 con 0x7f4148102df0 2024-03-21T10:06:29.211 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:29.209+0000 7f414e200700 1 -- 172.21.15.153:0/612359168 --> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4128000bf0 con 0x7f412c06c530 2024-03-21T10:06:29.212 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:29.211+0000 7f413fe00700 1 -- 172.21.15.153:0/612359168 <== mgr.14221 v2:172.21.15.153:6800/1639929315 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7f4128000bf0 con 0x7f412c06c530 2024-03-21T10:06:29.215 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:29.214+0000 7f414e200700 1 -- 172.21.15.153:0/612359168 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f412c06c530 msgr2=0x7f412c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:06:29.215 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:29.214+0000 7f414e200700 1 --2- 172.21.15.153:0/612359168 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f412c06c530 0x7f412c06e9f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4134005fd0 tx=0x7f4134005f60 comp rx=0 tx=0).stop 2024-03-21T10:06:29.215 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:29.214+0000 7f414e200700 1 -- 172.21.15.153:0/612359168 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4148102df0 msgr2=0x7f4148078b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:06:29.215 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:29.214+0000 7f414e200700 1 --2- 172.21.15.153:0/612359168 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4148102df0 0x7f4148078b40 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f413800bd70 tx=0x7f413800be50 comp rx=0 tx=0).stop 2024-03-21T10:06:29.216 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:29.214+0000 7f414e200700 1 -- 172.21.15.153:0/612359168 shutdown_connections 2024-03-21T10:06:29.216 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:29.214+0000 7f414e200700 1 --2- 172.21.15.153:0/612359168 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f412c06c530 0x7f412c06e9f0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:06:29.216 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:29.214+0000 7f414e200700 1 --2- 172.21.15.153:0/612359168 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4148102df0 0x7f4148078b40 secure :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f413800bd70 tx=0x7f413800be50 comp rx=0 tx=0).stop 2024-03-21T10:06:29.216 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:29.214+0000 7f414e200700 1 --2- 172.21.15.153:0/612359168 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4148103ff0 0x7f4148079080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:06:29.216 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:29.214+0000 7f414e200700 1 -- 172.21.15.153:0/612359168 >> 172.21.15.153:0/612359168 conn(0x7f41480fe390 msgr2=0x7f4148107220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:06:29.216 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:29.214+0000 7f414e200700 1 -- 172.21.15.153:0/612359168 shutdown_connections 2024-03-21T10:06:29.216 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:29.214+0000 7f414e200700 1 -- 172.21.15.153:0/612359168 wait complete. 2024-03-21T10:06:29.358 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:29.356+0000 7f6bdb200700 1 -- 172.21.15.153:0/517809165 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6bdc0ff260 msgr2=0x7f6bdc101690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:06:29.358 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:29.356+0000 7f6bdb200700 1 --2- 172.21.15.153:0/517809165 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6bdc0ff260 0x7f6bdc101690 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc009910 tx=0x7f6bcc009c20 comp rx=0 tx=0).stop 2024-03-21T10:06:29.358 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:29.356+0000 7f6bdb200700 1 -- 172.21.15.153:0/517809165 shutdown_connections 2024-03-21T10:06:29.358 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:29.356+0000 7f6bdb200700 1 --2- 172.21.15.153:0/517809165 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6bdc101bd0 0x7f6bdc104060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:06:29.358 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:29.356+0000 7f6bdb200700 1 --2- 172.21.15.153:0/517809165 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6bdc0ff260 0x7f6bdc101690 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:06:29.358 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:29.356+0000 7f6bdb200700 1 -- 172.21.15.153:0/517809165 >> 172.21.15.153:0/517809165 conn(0x7f6bdc0fae40 msgr2=0x7f6bdc0fd2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:06:29.358 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:29.356+0000 7f6bdb200700 1 -- 172.21.15.153:0/517809165 shutdown_connections 2024-03-21T10:06:29.359 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:29.356+0000 7f6bdb200700 1 -- 172.21.15.153:0/517809165 wait complete. 2024-03-21T10:06:29.359 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:29.357+0000 7f6bdb200700 1 Processor -- start 2024-03-21T10:06:29.359 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:29.357+0000 7f6bdb200700 1 -- start start 2024-03-21T10:06:29.359 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:29.358+0000 7f6bdb200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6bdc0ff260 0x7f6bdc19c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:06:29.360 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:29.358+0000 7f6bdb200700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6bdc101bd0 0x7f6bdc19ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:06:29.360 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:29.358+0000 7f6bdb200700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bdc19d310 con 0x7f6bdc0ff260 2024-03-21T10:06:29.360 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:29.358+0000 7f6bdb200700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bdc19d450 con 0x7f6bdc101bd0 2024-03-21T10:06:29.361 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:29.358+0000 7f6bd9e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6bdc0ff260 0x7f6bdc19c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:06:29.361 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:29.359+0000 7f6bd9e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6bdc0ff260 0x7f6bdc19c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:57448/0 (socket says 172.21.15.153:57448) 2024-03-21T10:06:29.361 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:29.359+0000 7f6bd9e00700 1 -- 172.21.15.153:0/315999399 learned_addr learned my addr 172.21.15.153:0/315999399 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:06:29.361 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:29.359+0000 7f6bd9400700 1 --2- 172.21.15.153:0/315999399 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6bdc101bd0 0x7f6bdc19ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:06:29.361 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:29.359+0000 7f6bd9e00700 1 -- 172.21.15.153:0/315999399 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6bdc101bd0 msgr2=0x7f6bdc19ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:06:29.361 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:29.359+0000 7f6bd9e00700 1 --2- 172.21.15.153:0/315999399 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6bdc101bd0 0x7f6bdc19ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:06:29.361 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:29.359+0000 7f6bd9e00700 1 -- 172.21.15.153:0/315999399 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bcc0095f0 con 0x7f6bdc0ff260 2024-03-21T10:06:29.361 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:29.359+0000 7f6bd9e00700 1 --2- 172.21.15.153:0/315999399 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6bdc0ff260 0x7f6bdc19c7b0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc000c00 tx=0x7f6bcc00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:06:29.362 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:29.360+0000 7f6bd2a00700 1 -- 172.21.15.153:0/315999399 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6bcc01d070 con 0x7f6bdc0ff260 2024-03-21T10:06:29.362 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:29.360+0000 7f6bd2a00700 1 -- 172.21.15.153:0/315999399 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f6bcc005dc0 con 0x7f6bdc0ff260 2024-03-21T10:06:29.362 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:29.360+0000 7f6bd2a00700 1 -- 172.21.15.153:0/315999399 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6bcc0226a0 con 0x7f6bdc0ff260 2024-03-21T10:06:29.363 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:29.360+0000 7f6bdb200700 1 -- 172.21.15.153:0/315999399 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bdc1a1ea0 con 0x7f6bdc0ff260 2024-03-21T10:06:29.363 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:29.360+0000 7f6bdb200700 1 -- 172.21.15.153:0/315999399 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bdc1a2310 con 0x7f6bdc0ff260 2024-03-21T10:06:29.363 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:29.362+0000 7f6bdb200700 1 -- 172.21.15.153:0/315999399 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bdc196940 con 0x7f6bdc0ff260 2024-03-21T10:06:29.365 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:29.364+0000 7f6bd2a00700 1 -- 172.21.15.153:0/315999399 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f6bcc0038f0 con 0x7f6bdc0ff260 2024-03-21T10:06:29.367 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:29.364+0000 7f6bd2a00700 1 --2- 172.21.15.153:0/315999399 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f6bc006c4e0 0x7f6bc006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:06:29.367 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:29.365+0000 7f6bd2a00700 1 -- 172.21.15.153:0/315999399 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f6bcc08cdb0 con 0x7f6bdc0ff260 2024-03-21T10:06:29.367 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:29.365+0000 7f6bd9400700 1 --2- 172.21.15.153:0/315999399 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f6bc006c4e0 0x7f6bc006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:06:29.367 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:29.366+0000 7f6bd9400700 1 --2- 172.21.15.153:0/315999399 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f6bc006c4e0 0x7f6bc006e9a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f6bc8009c60 tx=0x7f6bc8009260 comp rx=0 tx=0).ready entity=mgr.14221 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:06:29.375 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:29.374+0000 7f6bd2a00700 1 -- 172.21.15.153:0/315999399 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f6bcc05afa0 con 0x7f6bdc0ff260 2024-03-21T10:06:29.660 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:29.658+0000 7f6bdb200700 1 -- 172.21.15.153:0/315999399 --> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f6bdc0611d0 con 0x7f6bc006c4e0 2024-03-21T10:06:29.672 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:29.671+0000 7f6bd2a00700 1 -- 172.21.15.153:0/315999399 <== mgr.14221 v2:172.21.15.153:6800/1639929315 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3528 (secure 0 0 0) 0x7f6bdc0611d0 con 0x7f6bc006c4e0 2024-03-21T10:06:29.673 INFO:teuthology.orchestra.run.smithi153.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-21T10:06:29.673 INFO:teuthology.orchestra.run.smithi153.stdout:alertmanager.smithi153 smithi153 *:9093,9094 running (6m) 85s ago 9m 13.0M - 0.25.0 c8568f914cd2 2cfcc225c7e6 2024-03-21T10:06:29.673 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi153 smithi153 running (9m) 85s ago 9m 7587k - 18.2.1 5be31c24972a 5b61994396a6 2024-03-21T10:06:29.673 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi174 smithi174 running (7m) 91s ago 7m 7428k - 18.2.1 5be31c24972a 4f7a5fc3ac28 2024-03-21T10:06:29.673 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi153 smithi153 running (9m) 85s ago 9m 6665k - 18.2.1 5be31c24972a 90a8b8ea03d1 2024-03-21T10:06:29.673 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi174 smithi174 running (7m) 91s ago 7m 6665k - 18.2.1 5be31c24972a a4c929e4dc77 2024-03-21T10:06:29.674 INFO:teuthology.orchestra.run.smithi153.stdout:grafana.smithi153 smithi153 *:3000 running (6m) 85s ago 8m 73.6M - 9.4.7 954c08fa6188 3af5800ecfef 2024-03-21T10:06:29.674 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.iuyluj smithi153 running (109s) 85s ago 108s 15.0M - 18.2.1 5be31c24972a 02f667cee979 2024-03-21T10:06:29.674 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.zipckr smithi153 running (118s) 85s ago 117s 12.9M - 18.2.1 5be31c24972a 68a853929f8f 2024-03-21T10:06:29.674 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.nhozag smithi174 running (105s) 91s ago 105s 12.9M - 18.2.1 5be31c24972a 2fee4000308f 2024-03-21T10:06:29.674 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.yyimst smithi174 running (114s) 91s ago 114s 13.9M - 18.2.1 5be31c24972a d78a729220d1 2024-03-21T10:06:29.674 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi153.tluxni smithi153 *:9283,8765,8443 running (10m) 85s ago 10m 491M - 18.2.1 5be31c24972a 4978543154ff 2024-03-21T10:06:29.674 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi174.qpcwat smithi174 *:8443,9283,8765 running (7m) 91s ago 7m 441M - 18.2.1 5be31c24972a 6d0c64be5134 2024-03-21T10:06:29.674 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi153 smithi153 running (10m) 85s ago 11m 43.7M 2048M 18.2.1 5be31c24972a 813963a95d69 2024-03-21T10:06:29.674 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi174 smithi174 running (7m) 91s ago 7m 34.2M 2048M 18.2.1 5be31c24972a 99204923eec6 2024-03-21T10:06:29.674 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi153 smithi153 *:9100 running (9m) 85s ago 9m 7608k - 1.5.0 0da6a335fe13 babbc29a0eaf 2024-03-21T10:06:29.674 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi174 smithi174 *:9100 running (7m) 91s ago 7m 7952k - 1.5.0 0da6a335fe13 86841435042a 2024-03-21T10:06:29.674 INFO:teuthology.orchestra.run.smithi153.stdout:osd.0 smithi153 running (6m) 85s ago 6m 46.3M 3947M 18.2.1 5be31c24972a 28912a2319b4 2024-03-21T10:06:29.674 INFO:teuthology.orchestra.run.smithi153.stdout:osd.1 smithi153 running (5m) 85s ago 5m 47.2M 3947M 18.2.1 5be31c24972a ca60681e2314 2024-03-21T10:06:29.674 INFO:teuthology.orchestra.run.smithi153.stdout:osd.2 smithi153 running (5m) 85s ago 5m 45.4M 3947M 18.2.1 5be31c24972a 8db799f8962a 2024-03-21T10:06:29.674 INFO:teuthology.orchestra.run.smithi153.stdout:osd.3 smithi174 running (4m) 91s ago 4m 45.0M 4971M 18.2.1 5be31c24972a dafc89e0658d 2024-03-21T10:06:29.675 INFO:teuthology.orchestra.run.smithi153.stdout:osd.4 smithi174 running (4m) 91s ago 4m 43.1M 4971M 18.2.1 5be31c24972a c4cd15701f05 2024-03-21T10:06:29.675 INFO:teuthology.orchestra.run.smithi153.stdout:osd.5 smithi174 running (3m) 91s ago 3m 44.4M 4971M 18.2.1 5be31c24972a 3b69886ce22e 2024-03-21T10:06:29.675 INFO:teuthology.orchestra.run.smithi153.stdout:prometheus.smithi153 smithi153 *:9095 running (6m) 85s ago 8m 53.9M - 2.43.0 a07b618ecd1d c30cdeb81cc9 2024-03-21T10:06:29.676 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:29.674+0000 7f6bdb200700 1 -- 172.21.15.153:0/315999399 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f6bc006c4e0 msgr2=0x7f6bc006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:06:29.676 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:29.674+0000 7f6bdb200700 1 --2- 172.21.15.153:0/315999399 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f6bc006c4e0 0x7f6bc006e9a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f6bc8009c60 tx=0x7f6bc8009260 comp rx=0 tx=0).stop 2024-03-21T10:06:29.676 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:29.674+0000 7f6bdb200700 1 -- 172.21.15.153:0/315999399 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6bdc0ff260 msgr2=0x7f6bdc19c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:06:29.676 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:29.674+0000 7f6bdb200700 1 --2- 172.21.15.153:0/315999399 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6bdc0ff260 0x7f6bdc19c7b0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc000c00 tx=0x7f6bcc00f690 comp rx=0 tx=0).stop 2024-03-21T10:06:29.677 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:29.675+0000 7f6bdb200700 1 -- 172.21.15.153:0/315999399 shutdown_connections 2024-03-21T10:06:29.677 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:29.675+0000 7f6bdb200700 1 --2- 172.21.15.153:0/315999399 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f6bc006c4e0 0x7f6bc006e9a0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:06:29.677 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:29.675+0000 7f6bdb200700 1 --2- 172.21.15.153:0/315999399 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6bdc0ff260 0x7f6bdc19c7b0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:06:29.677 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:29.675+0000 7f6bdb200700 1 --2- 172.21.15.153:0/315999399 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6bdc101bd0 0x7f6bdc19ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:06:29.677 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:29.675+0000 7f6bdb200700 1 -- 172.21.15.153:0/315999399 >> 172.21.15.153:0/315999399 conn(0x7f6bdc0fae40 msgr2=0x7f6bdc0fd2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:06:29.677 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:29.675+0000 7f6bdb200700 1 -- 172.21.15.153:0/315999399 shutdown_connections 2024-03-21T10:06:29.677 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:29.675+0000 7f6bdb200700 1 -- 172.21.15.153:0/315999399 wait complete. 2024-03-21T10:06:29.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:06:29 smithi174 ceph-mon[37496]: pgmap v277: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:06:29.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:06:29 smithi174 ceph-mon[37496]: from='client.14564 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:06:29.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:06:29 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:06:29.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:06:29 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:06:29.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:06:29 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:06:29.830 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:29.828+0000 7f4e4b200700 1 -- 172.21.15.153:0/381697957 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e4c101c10 msgr2=0x7f4e4c102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:06:29.830 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:29.828+0000 7f4e4b200700 1 --2- 172.21.15.153:0/381697957 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e4c101c10 0x7f4e4c102070 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f4e38009910 tx=0x7f4e38009c20 comp rx=0 tx=0).stop 2024-03-21T10:06:29.830 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:29.828+0000 7f4e4b200700 1 -- 172.21.15.153:0/381697957 shutdown_connections 2024-03-21T10:06:29.831 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:29.828+0000 7f4e4b200700 1 --2- 172.21.15.153:0/381697957 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e4c101c10 0x7f4e4c102070 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:06:29.831 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:29.828+0000 7f4e4b200700 1 --2- 172.21.15.153:0/381697957 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4e4c100a10 0x7f4e4c100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:06:29.831 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:29.828+0000 7f4e4b200700 1 -- 172.21.15.153:0/381697957 >> 172.21.15.153:0/381697957 conn(0x7f4e4c0fbf70 msgr2=0x7f4e4c0fe3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:06:29.831 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:29.828+0000 7f4e4b200700 1 -- 172.21.15.153:0/381697957 shutdown_connections 2024-03-21T10:06:29.831 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:29.828+0000 7f4e4b200700 1 -- 172.21.15.153:0/381697957 wait complete. 2024-03-21T10:06:29.831 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:29.830+0000 7f4e4b200700 1 Processor -- start 2024-03-21T10:06:29.832 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:29.830+0000 7f4e4b200700 1 -- start start 2024-03-21T10:06:29.832 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:29.830+0000 7f4e4b200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e4c100a10 0x7f4e4c074b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:06:29.832 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:29.831+0000 7f4e4b200700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4e4c101c10 0x7f4e4c073180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:06:29.832 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:29.831+0000 7f4e4b200700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e4c0736c0 con 0x7f4e4c100a10 2024-03-21T10:06:29.832 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:29.831+0000 7f4e4b200700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e4c073800 con 0x7f4e4c101c10 2024-03-21T10:06:29.832 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:29.831+0000 7f4e49e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e4c100a10 0x7f4e4c074b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:06:29.833 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:29.831+0000 7f4e49e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e4c100a10 0x7f4e4c074b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:57472/0 (socket says 172.21.15.153:57472) 2024-03-21T10:06:29.833 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:29.831+0000 7f4e49e00700 1 -- 172.21.15.153:0/4251422866 learned_addr learned my addr 172.21.15.153:0/4251422866 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:06:29.834 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:29.831+0000 7f4e49400700 1 --2- 172.21.15.153:0/4251422866 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4e4c101c10 0x7f4e4c073180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:06:29.834 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:29.831+0000 7f4e49e00700 1 -- 172.21.15.153:0/4251422866 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4e4c101c10 msgr2=0x7f4e4c073180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:06:29.834 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:29.831+0000 7f4e49e00700 1 --2- 172.21.15.153:0/4251422866 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4e4c101c10 0x7f4e4c073180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:06:29.834 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:29.831+0000 7f4e49e00700 1 -- 172.21.15.153:0/4251422866 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e380095f0 con 0x7f4e4c100a10 2024-03-21T10:06:29.834 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:29.832+0000 7f4e49e00700 1 --2- 172.21.15.153:0/4251422866 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e4c100a10 0x7f4e4c074b30 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c00e8f0 tx=0x7f4e3c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:06:29.834 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:29.832+0000 7f4e42a00700 1 -- 172.21.15.153:0/4251422866 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4e3c009790 con 0x7f4e4c100a10 2024-03-21T10:06:29.835 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:29.832+0000 7f4e42a00700 1 -- 172.21.15.153:0/4251422866 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f4e3c0044d0 con 0x7f4e4c100a10 2024-03-21T10:06:29.835 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:29.832+0000 7f4e42a00700 1 -- 172.21.15.153:0/4251422866 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4e3c003da0 con 0x7f4e4c100a10 2024-03-21T10:06:29.835 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:29.833+0000 7f4e4b200700 1 -- 172.21.15.153:0/4251422866 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e4c073ae0 con 0x7f4e4c100a10 2024-03-21T10:06:29.835 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:29.833+0000 7f4e4b200700 1 -- 172.21.15.153:0/4251422866 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e4c19e1b0 con 0x7f4e4c100a10 2024-03-21T10:06:29.835 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:29.834+0000 7f4e4b200700 1 -- 172.21.15.153:0/4251422866 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e4c109570 con 0x7f4e4c100a10 2024-03-21T10:06:29.837 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:29.836+0000 7f4e42a00700 1 -- 172.21.15.153:0/4251422866 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f4e3c00cba0 con 0x7f4e4c100a10 2024-03-21T10:06:29.839 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:29.837+0000 7f4e42a00700 1 --2- 172.21.15.153:0/4251422866 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f4e3006c4e0 0x7f4e3006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:06:29.839 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:29.837+0000 7f4e42a00700 1 -- 172.21.15.153:0/4251422866 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f4e3c014070 con 0x7f4e4c100a10 2024-03-21T10:06:29.839 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:29.838+0000 7f4e49400700 1 --2- 172.21.15.153:0/4251422866 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f4e3006c4e0 0x7f4e3006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:06:29.840 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:29.839+0000 7f4e49400700 1 --2- 172.21.15.153:0/4251422866 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f4e3006c4e0 0x7f4e3006e9a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4e38009fd0 tx=0x7f4e38005fd0 comp rx=0 tx=0).ready entity=mgr.14221 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:06:29.843 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:29.842+0000 7f4e42a00700 1 -- 172.21.15.153:0/4251422866 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f4e3c059e90 con 0x7f4e4c100a10 2024-03-21T10:06:30.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:06:29 smithi153 ceph-mon[29117]: pgmap v277: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:06:30.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:06:29 smithi153 ceph-mon[29117]: from='client.14564 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:06:30.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:06:29 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:06:30.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:06:29 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:06:30.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:06:29 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:06:30.161 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:30.160+0000 7f4e4b200700 1 -- 172.21.15.153:0/4251422866 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f4e4c19e760 con 0x7f4e4c100a10 2024-03-21T10:06:30.162 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:30.160+0000 7f4e42a00700 1 -- 172.21.15.153:0/4251422866 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+558 (secure 0 0 0) 0x7f4e3c059a20 con 0x7f4e4c100a10 2024-03-21T10:06:30.162 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:06:30.162 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": { 2024-03-21T10:06:30.162 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-03-21T10:06:30.162 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:06:30.162 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": { 2024-03-21T10:06:30.162 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-03-21T10:06:30.162 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:06:30.163 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": { 2024-03-21T10:06:30.163 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-03-21T10:06:30.163 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:06:30.163 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": { 2024-03-21T10:06:30.163 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-03-21T10:06:30.163 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:06:30.163 INFO:teuthology.orchestra.run.smithi153.stdout: "overall": { 2024-03-21T10:06:30.163 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 14 2024-03-21T10:06:30.163 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-03-21T10:06:30.163 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:06:30.165 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:30.163+0000 7f4e4b200700 1 -- 172.21.15.153:0/4251422866 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f4e3006c4e0 msgr2=0x7f4e3006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:06:30.165 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:30.163+0000 7f4e4b200700 1 --2- 172.21.15.153:0/4251422866 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f4e3006c4e0 0x7f4e3006e9a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4e38009fd0 tx=0x7f4e38005fd0 comp rx=0 tx=0).stop 2024-03-21T10:06:30.165 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:30.163+0000 7f4e4b200700 1 -- 172.21.15.153:0/4251422866 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e4c100a10 msgr2=0x7f4e4c074b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:06:30.165 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:30.163+0000 7f4e4b200700 1 --2- 172.21.15.153:0/4251422866 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e4c100a10 0x7f4e4c074b30 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c00e8f0 tx=0x7f4e3c00ecb0 comp rx=0 tx=0).stop 2024-03-21T10:06:30.165 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:30.163+0000 7f4e4b200700 1 -- 172.21.15.153:0/4251422866 shutdown_connections 2024-03-21T10:06:30.165 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:30.163+0000 7f4e4b200700 1 --2- 172.21.15.153:0/4251422866 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f4e3006c4e0 0x7f4e3006e9a0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:06:30.165 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:30.163+0000 7f4e4b200700 1 --2- 172.21.15.153:0/4251422866 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e4c100a10 0x7f4e4c074b30 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:06:30.165 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:30.163+0000 7f4e4b200700 1 --2- 172.21.15.153:0/4251422866 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4e4c101c10 0x7f4e4c073180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:06:30.165 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:30.163+0000 7f4e4b200700 1 -- 172.21.15.153:0/4251422866 >> 172.21.15.153:0/4251422866 conn(0x7f4e4c0fbf70 msgr2=0x7f4e4c104e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:06:30.166 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:30.164+0000 7f4e4b200700 1 -- 172.21.15.153:0/4251422866 shutdown_connections 2024-03-21T10:06:30.166 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:30.164+0000 7f4e4b200700 1 -- 172.21.15.153:0/4251422866 wait complete. 2024-03-21T10:06:30.325 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:30.323+0000 7fb348200700 1 -- 172.21.15.153:0/2824259120 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb340103f70 msgr2=0x7fb340106360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:06:30.325 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:30.323+0000 7fb348200700 1 --2- 172.21.15.153:0/2824259120 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb340103f70 0x7fb340106360 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fb32c00b0a0 tx=0x7fb32c00b3b0 comp rx=0 tx=0).stop 2024-03-21T10:06:30.325 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:30.323+0000 7fb348200700 1 -- 172.21.15.153:0/2824259120 shutdown_connections 2024-03-21T10:06:30.325 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:30.323+0000 7fb348200700 1 --2- 172.21.15.153:0/2824259120 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb340103f70 0x7fb340106360 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:06:30.325 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:30.323+0000 7fb348200700 1 --2- 172.21.15.153:0/2824259120 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb340101640 0x7fb340103a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:06:30.325 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:30.323+0000 7fb348200700 1 -- 172.21.15.153:0/2824259120 >> 172.21.15.153:0/2824259120 conn(0x7fb3400fb030 msgr2=0x7fb3400fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:06:30.325 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:30.323+0000 7fb348200700 1 -- 172.21.15.153:0/2824259120 shutdown_connections 2024-03-21T10:06:30.326 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:30.323+0000 7fb348200700 1 -- 172.21.15.153:0/2824259120 wait complete. 2024-03-21T10:06:30.326 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:30.325+0000 7fb348200700 1 Processor -- start 2024-03-21T10:06:30.327 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:30.325+0000 7fb348200700 1 -- start start 2024-03-21T10:06:30.327 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:30.325+0000 7fb348200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb340101640 0x7fb3401926f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:06:30.327 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:30.325+0000 7fb348200700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb340103f70 0x7fb340192c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:06:30.327 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:30.325+0000 7fb348200700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb34018f1e0 con 0x7fb340101640 2024-03-21T10:06:30.327 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:30.325+0000 7fb348200700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb34018f350 con 0x7fb340103f70 2024-03-21T10:06:30.327 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:30.325+0000 7fb345800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb340101640 0x7fb3401926f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:06:30.327 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:30.325+0000 7fb344e00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb340103f70 0x7fb340192c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:06:30.327 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:30.326+0000 7fb344e00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb340103f70 0x7fb340192c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.153:60184/0 (socket says 172.21.15.153:60184) 2024-03-21T10:06:30.328 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:30.326+0000 7fb344e00700 1 -- 172.21.15.153:0/3662140960 learned_addr learned my addr 172.21.15.153:0/3662140960 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:06:30.328 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:30.326+0000 7fb345800700 1 -- 172.21.15.153:0/3662140960 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb340103f70 msgr2=0x7fb340192c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:06:30.328 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:30.326+0000 7fb345800700 1 --2- 172.21.15.153:0/3662140960 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb340103f70 0x7fb340192c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:06:30.328 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:30.326+0000 7fb345800700 1 -- 172.21.15.153:0/3662140960 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb32c009d00 con 0x7fb340101640 2024-03-21T10:06:30.328 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:30.327+0000 7fb345800700 1 --2- 172.21.15.153:0/3662140960 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb340101640 0x7fb3401926f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb33800d8d0 tx=0x7fb33800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:06:30.329 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:30.327+0000 7fb33e000700 1 -- 172.21.15.153:0/3662140960 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb33800f860 con 0x7fb340101640 2024-03-21T10:06:30.329 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:30.327+0000 7fb33e000700 1 -- 172.21.15.153:0/3662140960 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fb33800fe70 con 0x7fb340101640 2024-03-21T10:06:30.329 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:30.327+0000 7fb33e000700 1 -- 172.21.15.153:0/3662140960 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb33800e5e0 con 0x7fb340101640 2024-03-21T10:06:30.329 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:30.327+0000 7fb348200700 1 -- 172.21.15.153:0/3662140960 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb34018f630 con 0x7fb340101640 2024-03-21T10:06:30.329 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:30.327+0000 7fb348200700 1 -- 172.21.15.153:0/3662140960 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb34018fc00 con 0x7fb340101640 2024-03-21T10:06:30.331 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:30.330+0000 7fb348200700 1 -- 172.21.15.153:0/3662140960 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb320005320 con 0x7fb340101640 2024-03-21T10:06:30.332 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:30.331+0000 7fb33e000700 1 -- 172.21.15.153:0/3662140960 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fb338010460 con 0x7fb340101640 2024-03-21T10:06:30.334 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:30.332+0000 7fb33e000700 1 --2- 172.21.15.153:0/3662140960 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fb32406c4e0 0x7fb32406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:06:30.334 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:30.333+0000 7fb33e000700 1 -- 172.21.15.153:0/3662140960 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fb33808b450 con 0x7fb340101640 2024-03-21T10:06:30.335 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:30.333+0000 7fb344e00700 1 --2- 172.21.15.153:0/3662140960 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fb32406c4e0 0x7fb32406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:06:30.335 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:30.334+0000 7fb344e00700 1 --2- 172.21.15.153:0/3662140960 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fb32406c4e0 0x7fb32406e9a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb32c009480 tx=0x7fb32c0061f0 comp rx=0 tx=0).ready entity=mgr.14221 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:06:30.342 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:30.341+0000 7fb33e000700 1 -- 172.21.15.153:0/3662140960 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fb338059640 con 0x7fb340101640 2024-03-21T10:06:30.574 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:30.572+0000 7fb348200700 1 -- 172.21.15.153:0/3662140960 --> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb320000bf0 con 0x7fb32406c4e0 2024-03-21T10:06:30.575 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:30.574+0000 7fb33e000700 1 -- 172.21.15.153:0/3662140960 <== mgr.14221 v2:172.21.15.153:6800/1639929315 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7fb320000bf0 con 0x7fb32406c4e0 2024-03-21T10:06:30.575 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:06:30.576 INFO:teuthology.orchestra.run.smithi153.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232", 2024-03-21T10:06:30.576 INFO:teuthology.orchestra.run.smithi153.stdout: "in_progress": true, 2024-03-21T10:06:30.576 INFO:teuthology.orchestra.run.smithi153.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-03-21T10:06:30.576 INFO:teuthology.orchestra.run.smithi153.stdout: "services_complete": [], 2024-03-21T10:06:30.576 INFO:teuthology.orchestra.run.smithi153.stdout: "progress": "0/2 daemons upgraded", 2024-03-21T10:06:30.576 INFO:teuthology.orchestra.run.smithi153.stdout: "message": "Pulling quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 image on host smithi174", 2024-03-21T10:06:30.576 INFO:teuthology.orchestra.run.smithi153.stdout: "is_paused": false 2024-03-21T10:06:30.576 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:06:30.578 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:30.576+0000 7fb348200700 1 -- 172.21.15.153:0/3662140960 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fb32406c4e0 msgr2=0x7fb32406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:06:30.578 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:30.576+0000 7fb348200700 1 --2- 172.21.15.153:0/3662140960 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fb32406c4e0 0x7fb32406e9a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb32c009480 tx=0x7fb32c0061f0 comp rx=0 tx=0).stop 2024-03-21T10:06:30.578 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:30.576+0000 7fb348200700 1 -- 172.21.15.153:0/3662140960 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb340101640 msgr2=0x7fb3401926f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:06:30.579 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:30.576+0000 7fb348200700 1 --2- 172.21.15.153:0/3662140960 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb340101640 0x7fb3401926f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb33800d8d0 tx=0x7fb33800dbe0 comp rx=0 tx=0).stop 2024-03-21T10:06:30.579 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:30.576+0000 7fb348200700 1 -- 172.21.15.153:0/3662140960 shutdown_connections 2024-03-21T10:06:30.579 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:30.576+0000 7fb348200700 1 --2- 172.21.15.153:0/3662140960 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fb32406c4e0 0x7fb32406e9a0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:06:30.579 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:30.576+0000 7fb348200700 1 --2- 172.21.15.153:0/3662140960 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb340101640 0x7fb3401926f0 secure :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb33800d8d0 tx=0x7fb33800dbe0 comp rx=0 tx=0).stop 2024-03-21T10:06:30.579 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:30.576+0000 7fb348200700 1 --2- 172.21.15.153:0/3662140960 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb340103f70 0x7fb340192c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:06:30.579 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:30.577+0000 7fb348200700 1 -- 172.21.15.153:0/3662140960 >> 172.21.15.153:0/3662140960 conn(0x7fb3400fb030 msgr2=0x7fb340100eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:06:30.579 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:30.577+0000 7fb348200700 1 -- 172.21.15.153:0/3662140960 shutdown_connections 2024-03-21T10:06:30.579 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:06:30.577+0000 7fb348200700 1 -- 172.21.15.153:0/3662140960 wait complete. 2024-03-21T10:06:30.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:06:30 smithi174 ceph-mon[37496]: Upgrade: Target is version 19.0.0-2231-g2edfee68 (unknown) 2024-03-21T10:06:30.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:06:30 smithi174 ceph-mon[37496]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, digests ['quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232'] 2024-03-21T10:06:30.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:06:30 smithi174 ceph-mon[37496]: Upgrade: Need to upgrade myself (mgr.smithi153.tluxni) 2024-03-21T10:06:30.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:06:30 smithi174 ceph-mon[37496]: from='client.14568 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:06:30.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:06:30 smithi174 ceph-mon[37496]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 on smithi174 2024-03-21T10:06:30.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:06:30 smithi174 ceph-mon[37496]: from='client.? 172.21.15.153:0/4251422866' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:06:31.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:06:30 smithi153 ceph-mon[29117]: Upgrade: Target is version 19.0.0-2231-g2edfee68 (unknown) 2024-03-21T10:06:31.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:06:30 smithi153 ceph-mon[29117]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, digests ['quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232'] 2024-03-21T10:06:31.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:06:30 smithi153 ceph-mon[29117]: Upgrade: Need to upgrade myself (mgr.smithi153.tluxni) 2024-03-21T10:06:31.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:06:30 smithi153 ceph-mon[29117]: from='client.14568 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:06:31.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:06:30 smithi153 ceph-mon[29117]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 on smithi174 2024-03-21T10:06:31.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:06:30 smithi153 ceph-mon[29117]: from='client.? 172.21.15.153:0/4251422866' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:06:31.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:06:31 smithi174 ceph-mon[37496]: from='client.14572 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:06:31.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:06:31 smithi174 ceph-mon[37496]: pgmap v278: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:06:31.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:06:31 smithi174 ceph-mon[37496]: from='client.14580 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:06:32.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:06:31 smithi153 ceph-mon[29117]: from='client.14572 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:06:32.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:06:31 smithi153 ceph-mon[29117]: pgmap v278: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:06:32.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:06:31 smithi153 ceph-mon[29117]: from='client.14580 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:06:33.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:06:33 smithi174 ceph-mon[37496]: pgmap v279: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:06:34.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:06:33 smithi153 ceph-mon[29117]: pgmap v279: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:06:35.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:06:35 smithi174 ceph-mon[37496]: pgmap v280: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:06:35.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:06:35 smithi153 ceph-mon[29117]: pgmap v280: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:06:37.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:06:37 smithi153 ceph-mon[29117]: pgmap v281: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:06:37.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:06:37 smithi174 ceph-mon[37496]: pgmap v281: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:06:39.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:06:39 smithi153 ceph-mon[29117]: pgmap v282: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:06:39.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:06:39 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:06:39.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:06:39 smithi174 ceph-mon[37496]: pgmap v282: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:06:39.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:06:39 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:06:41.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:06:41 smithi153 ceph-mon[29117]: pgmap v283: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:06:41.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:06:41 smithi174 ceph-mon[37496]: pgmap v283: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:06:43.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:06:43 smithi153 ceph-mon[29117]: pgmap v284: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:06:43.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:06:43 smithi174 ceph-mon[37496]: pgmap v284: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:06:45.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:06:45 smithi174 ceph-mon[37496]: pgmap v285: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:06:45.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:06:45 smithi153 ceph-mon[29117]: pgmap v285: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:06:47.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:06:47 smithi153 ceph-mon[29117]: pgmap v286: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:06:47.944 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:06:47 smithi174 ceph-mon[37496]: pgmap v286: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:06:49.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:06:49 smithi153 ceph-mon[29117]: pgmap v287: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:06:49.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:06:49 smithi174 ceph-mon[37496]: pgmap v287: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:06:51.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:06:51 smithi153 ceph-mon[29117]: pgmap v288: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:06:51.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:06:51 smithi174 ceph-mon[37496]: pgmap v288: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:06:53.202 INFO:tasks.workunit.client.0.smithi153.stderr:Updating files: 61% (7503/12271) Updating files: 62% (7609/12271) Updating files: 63% (7731/12271) Updating files: 64% (7854/12271) Updating files: 65% (7977/12271) Updating files: 66% (8099/12271) Updating files: 67% (8222/12271) Updating files: 68% (8345/12271) Updating files: 69% (8467/12271) Updating files: 70% (8590/12271) Updating files: 71% (8713/12271) Updating files: 72% (8836/12271) Updating files: 73% (8958/12271) Updating files: 74% (9081/12271) Updating files: 75% (9204/12271) Updating files: 76% (9326/12271) Updating files: 77% (9449/12271) Updating files: 78% (9572/12271) Updating files: 79% (9695/12271) Updating files: 80% (9817/12271) Updating files: 81% (9940/12271) Updating files: 82% (10063/12271) Updating files: 83% (10185/12271) Updating files: 84% (10308/12271) Updating files: 85% (10431/12271) Updating files: 86% (10554/12271) Updating files: 87% (10676/12271) Updating files: 88% (10799/12271) Updating files: 89% (10922/12271) Updating files: 90% (11044/12271) Updating files: 91% (11167/12271) Updating files: 92% (11290/12271) Updating files: 93% (11413/12271) Updating files: 94% (11535/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-03-21T10:06:53.637 INFO:tasks.workunit.client.0.smithi153.stderr:Note: switching to '2edfee68ad7243bed422a049b6f5f58619300f11'. 2024-03-21T10:06:53.637 INFO:tasks.workunit.client.0.smithi153.stderr: 2024-03-21T10:06:53.637 INFO:tasks.workunit.client.0.smithi153.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-03-21T10:06:53.637 INFO:tasks.workunit.client.0.smithi153.stderr:changes and commit them, and you can discard any commits you make in this 2024-03-21T10:06:53.637 INFO:tasks.workunit.client.0.smithi153.stderr:state without impacting any branches by switching back to a branch. 2024-03-21T10:06:53.637 INFO:tasks.workunit.client.0.smithi153.stderr: 2024-03-21T10:06:53.638 INFO:tasks.workunit.client.0.smithi153.stderr:If you want to create a new branch to retain commits you create, you may 2024-03-21T10:06:53.638 INFO:tasks.workunit.client.0.smithi153.stderr:do so (now or later) by using -c with the switch command. Example: 2024-03-21T10:06:53.638 INFO:tasks.workunit.client.0.smithi153.stderr: 2024-03-21T10:06:53.638 INFO:tasks.workunit.client.0.smithi153.stderr: git switch -c 2024-03-21T10:06:53.638 INFO:tasks.workunit.client.0.smithi153.stderr: 2024-03-21T10:06:53.638 INFO:tasks.workunit.client.0.smithi153.stderr:Or undo this operation with: 2024-03-21T10:06:53.638 INFO:tasks.workunit.client.0.smithi153.stderr: 2024-03-21T10:06:53.638 INFO:tasks.workunit.client.0.smithi153.stderr: git switch - 2024-03-21T10:06:53.638 INFO:tasks.workunit.client.0.smithi153.stderr: 2024-03-21T10:06:53.638 INFO:tasks.workunit.client.0.smithi153.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-03-21T10:06:53.638 INFO:tasks.workunit.client.0.smithi153.stderr: 2024-03-21T10:06:53.638 INFO:tasks.workunit.client.0.smithi153.stderr:HEAD is now at 2edfee68ad7 Merge PR #55768 into wip-vshankar-testing-20240320.053002 2024-03-21T10:06:53.645 DEBUG:teuthology.orchestra.run.smithi153:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-03-21T10:06:53.715 INFO:tasks.workunit.client.0.smithi153.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-03-21T10:06:53.724 INFO:tasks.workunit.client.0.smithi153.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-21T10:06:53.724 INFO:tasks.workunit.client.0.smithi153.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-03-21T10:06:53.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:06:53 smithi153 ceph-mon[29117]: pgmap v289: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:06:53.826 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:06:53 smithi174 ceph-mon[37496]: pgmap v289: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:06:54.693 INFO:tasks.workunit.client.0.smithi153.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-03-21T10:06:54.810 INFO:tasks.workunit.client.0.smithi153.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-03-21T10:06:54.897 INFO:tasks.workunit.client.0.smithi153.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-21T10:06:54.903 INFO:tasks.workunit.client.0.smithi153.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-21T10:06:54.903 INFO:tasks.workunit.client.0.smithi153.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-03-21T10:06:54.986 INFO:tasks.workunit.client.0.smithi153.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-21T10:06:54.995 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-03-21T10:06:54.995 DEBUG:teuthology.orchestra.run.smithi153:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-03-21T10:06:55.044 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.0... 2024-03-21T10:06:55.046 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-03-21T10:06:55.046 DEBUG:teuthology.orchestra.run.smithi153:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=2edfee68ad7243bed422a049b6f5f58619300f11 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh 2024-03-21T10:06:55.175 INFO:tasks.workunit.client.0.smithi153.stderr:+ mkdir -p fsstress 2024-03-21T10:06:55.183 INFO:tasks.workunit.client.0.smithi153.stderr:+ pushd fsstress 2024-03-21T10:06:55.188 INFO:tasks.workunit.client.0.smithi153.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-03-21T10:06:55.188 INFO:tasks.workunit.client.0.smithi153.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-03-21T10:06:55.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:06:55 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:06:55.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:06:55 smithi174 ceph-mon[37496]: pgmap v290: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:06:55.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:06:55 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:06:55.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:06:55 smithi153 ceph-mon[29117]: pgmap v290: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:06:55.827 INFO:tasks.workunit.client.1.smithi174.stderr:Updating files: 54% (6707/12271) Updating files: 55% (6750/12271) Updating files: 56% (6872/12271) Updating files: 57% (6995/12271) Updating files: 58% (7118/12271) Updating files: 59% (7240/12271) Updating files: 60% (7363/12271) Updating files: 61% (7486/12271) Updating files: 62% (7609/12271) Updating files: 63% (7731/12271) Updating files: 64% (7854/12271) Updating files: 65% (7977/12271) Updating files: 66% (8099/12271) Updating files: 67% (8222/12271) Updating files: 68% (8345/12271) Updating files: 69% (8467/12271) Updating files: 70% (8590/12271) Updating files: 71% (8713/12271) Updating files: 72% (8836/12271) Updating files: 73% (8958/12271) Updating files: 74% (9081/12271) Updating files: 75% (9204/12271) Updating files: 76% (9326/12271) Updating files: 77% (9449/12271) Updating files: 78% (9572/12271) Updating files: 79% (9695/12271) Updating files: 79% (9778/12271) Updating files: 80% (9817/12271) Updating files: 81% (9940/12271) Updating files: 82% (10063/12271) Updating files: 83% (10185/12271) Updating files: 84% (10308/12271) Updating files: 85% (10431/12271) Updating files: 86% (10554/12271) Updating files: 87% (10676/12271) Updating files: 88% (10799/12271) Updating files: 89% (10922/12271) Updating files: 90% (11044/12271) Updating files: 91% (11167/12271) Updating files: 92% (11290/12271) Updating files: 93% (11413/12271) Updating files: 94% (11535/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-03-21T10:06:56.626 INFO:tasks.workunit.client.1.smithi174.stderr:Note: switching to '2edfee68ad7243bed422a049b6f5f58619300f11'. 2024-03-21T10:06:56.626 INFO:tasks.workunit.client.1.smithi174.stderr: 2024-03-21T10:06:56.626 INFO:tasks.workunit.client.1.smithi174.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-03-21T10:06:56.626 INFO:tasks.workunit.client.1.smithi174.stderr:changes and commit them, and you can discard any commits you make in this 2024-03-21T10:06:56.626 INFO:tasks.workunit.client.1.smithi174.stderr:state without impacting any branches by switching back to a branch. 2024-03-21T10:06:56.626 INFO:tasks.workunit.client.1.smithi174.stderr: 2024-03-21T10:06:56.626 INFO:tasks.workunit.client.1.smithi174.stderr:If you want to create a new branch to retain commits you create, you may 2024-03-21T10:06:56.627 INFO:tasks.workunit.client.1.smithi174.stderr:do so (now or later) by using -c with the switch command. Example: 2024-03-21T10:06:56.627 INFO:tasks.workunit.client.1.smithi174.stderr: 2024-03-21T10:06:56.627 INFO:tasks.workunit.client.1.smithi174.stderr: git switch -c 2024-03-21T10:06:56.627 INFO:tasks.workunit.client.1.smithi174.stderr: 2024-03-21T10:06:56.627 INFO:tasks.workunit.client.1.smithi174.stderr:Or undo this operation with: 2024-03-21T10:06:56.627 INFO:tasks.workunit.client.1.smithi174.stderr: 2024-03-21T10:06:56.627 INFO:tasks.workunit.client.1.smithi174.stderr: git switch - 2024-03-21T10:06:56.627 INFO:tasks.workunit.client.1.smithi174.stderr: 2024-03-21T10:06:56.627 INFO:tasks.workunit.client.1.smithi174.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-03-21T10:06:56.627 INFO:tasks.workunit.client.1.smithi174.stderr: 2024-03-21T10:06:56.627 INFO:tasks.workunit.client.1.smithi174.stderr:HEAD is now at 2edfee68ad7 Merge PR #55768 into wip-vshankar-testing-20240320.053002 2024-03-21T10:06:56.644 DEBUG:teuthology.orchestra.run.smithi174:> cd -- /home/ubuntu/cephtest/clone.client.1/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.1 2024-03-21T10:06:56.714 INFO:tasks.workunit.client.1.smithi174.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-03-21T10:06:56.721 INFO:tasks.workunit.client.1.smithi174.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-03-21T10:06:56.721 INFO:tasks.workunit.client.1.smithi174.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-03-21T10:06:57.171 INFO:tasks.workunit.client.0.smithi153.stderr:+ tar xzf ltp-full.tgz 2024-03-21T10:06:57.606 INFO:tasks.workunit.client.1.smithi174.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-03-21T10:06:57.735 INFO:tasks.workunit.client.1.smithi174.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-03-21T10:06:57.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:06:57 smithi153 ceph-mon[29117]: pgmap v291: 65 pgs: 65 active+clean; 599 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 853 B/s wr, 4 op/s 2024-03-21T10:06:57.828 INFO:tasks.workunit.client.1.smithi174.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-03-21T10:06:57.832 INFO:tasks.workunit.client.1.smithi174.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-03-21T10:06:57.832 INFO:tasks.workunit.client.1.smithi174.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-03-21T10:06:57.914 INFO:tasks.workunit.client.1.smithi174.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-03-21T10:06:57.922 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-03-21T10:06:57.922 DEBUG:teuthology.orchestra.run.smithi174:> dd if=/home/ubuntu/cephtest/workunits.list.client.1 of=/dev/stdout 2024-03-21T10:06:57.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:06:57 smithi174 ceph-mon[37496]: pgmap v291: 65 pgs: 65 active+clean; 599 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 853 B/s wr, 4 op/s 2024-03-21T10:06:57.978 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.1... 2024-03-21T10:06:57.979 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-03-21T10:06:57.980 DEBUG:teuthology.orchestra.run.smithi174:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=2edfee68ad7243bed422a049b6f5f58619300f11 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/suites/fsstress.sh 2024-03-21T10:06:58.095 INFO:tasks.workunit.client.1.smithi174.stderr:+ mkdir -p fsstress 2024-03-21T10:06:58.103 INFO:tasks.workunit.client.1.smithi174.stderr:+ pushd fsstress 2024-03-21T10:06:58.106 INFO:tasks.workunit.client.1.smithi174.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-03-21T10:06:58.106 INFO:tasks.workunit.client.1.smithi174.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-03-21T10:06:59.464 INFO:tasks.workunit.client.1.smithi174.stderr:+ tar xzf ltp-full.tgz 2024-03-21T10:06:59.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:06:59 smithi153 ceph-mon[29117]: pgmap v292: 65 pgs: 65 active+clean; 599 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 938 B/s wr, 4 op/s 2024-03-21T10:06:59.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:06:59 smithi174 ceph-mon[37496]: pgmap v292: 65 pgs: 65 active+clean; 599 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 938 B/s wr, 4 op/s 2024-03-21T10:07:00.776 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:00.774+0000 7f8375e00700 1 -- 172.21.15.153:0/3242713311 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8370104050 msgr2=0x7f8370106440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:07:00.776 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:00.774+0000 7f8375e00700 1 --2- 172.21.15.153:0/3242713311 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8370104050 0x7f8370106440 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8358009910 tx=0x7f8358009c20 comp rx=0 tx=0).stop 2024-03-21T10:07:00.776 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:00.774+0000 7f8375e00700 1 -- 172.21.15.153:0/3242713311 shutdown_connections 2024-03-21T10:07:00.776 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:00.774+0000 7f8375e00700 1 --2- 172.21.15.153:0/3242713311 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8370104050 0x7f8370106440 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:07:00.777 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:00.774+0000 7f8375e00700 1 --2- 172.21.15.153:0/3242713311 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8370101720 0x7f8370103b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:07:00.777 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:00.774+0000 7f8375e00700 1 -- 172.21.15.153:0/3242713311 >> 172.21.15.153:0/3242713311 conn(0x7f83700fb070 msgr2=0x7f83700fd4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:07:00.777 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:00.774+0000 7f8375e00700 1 -- 172.21.15.153:0/3242713311 shutdown_connections 2024-03-21T10:07:00.777 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:00.774+0000 7f8375e00700 1 -- 172.21.15.153:0/3242713311 wait complete. 2024-03-21T10:07:00.777 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:00.776+0000 7f8375e00700 1 Processor -- start 2024-03-21T10:07:00.778 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:00.776+0000 7f8375e00700 1 -- start start 2024-03-21T10:07:00.779 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:00.777+0000 7f8375e00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8370101720 0x7f8370192740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:07:00.779 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:00.777+0000 7f8375e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8370104050 0x7f8370192c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:07:00.779 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:00.777+0000 7f8375e00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f837018f230 con 0x7f8370104050 2024-03-21T10:07:00.779 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:00.777+0000 7f8375e00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f837018f3a0 con 0x7f8370101720 2024-03-21T10:07:00.779 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:00.777+0000 7f836ea00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8370101720 0x7f8370192740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:07:00.779 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:00.777+0000 7f8367e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8370104050 0x7f8370192c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:07:00.779 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:00.777+0000 7f836ea00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8370101720 0x7f8370192740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.153:44046/0 (socket says 172.21.15.153:44046) 2024-03-21T10:07:00.780 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:00.777+0000 7f836ea00700 1 -- 172.21.15.153:0/2264720864 learned_addr learned my addr 172.21.15.153:0/2264720864 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:07:00.781 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:00.778+0000 7f8367e00700 1 -- 172.21.15.153:0/2264720864 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8370101720 msgr2=0x7f8370192740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:07:00.781 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:00.778+0000 7f8367e00700 1 --2- 172.21.15.153:0/2264720864 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8370101720 0x7f8370192740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:07:00.781 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:00.778+0000 7f8367e00700 1 -- 172.21.15.153:0/2264720864 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8360009520 con 0x7f8370104050 2024-03-21T10:07:00.781 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:00.778+0000 7f8367e00700 1 --2- 172.21.15.153:0/2264720864 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8370104050 0x7f8370192c80 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8358009910 tx=0x7f835800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:07:00.782 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:00.779+0000 7f8374a00700 1 -- 172.21.15.153:0/2264720864 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f835801d070 con 0x7f8370104050 2024-03-21T10:07:00.782 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:00.779+0000 7f8375e00700 1 -- 172.21.15.153:0/2264720864 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83580095f0 con 0x7f8370104050 2024-03-21T10:07:00.782 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:00.779+0000 7f8375e00700 1 -- 172.21.15.153:0/2264720864 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f837018f980 con 0x7f8370104050 2024-03-21T10:07:00.782 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:00.779+0000 7f8374a00700 1 -- 172.21.15.153:0/2264720864 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f8358005dc0 con 0x7f8370104050 2024-03-21T10:07:00.782 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:00.779+0000 7f8374a00700 1 -- 172.21.15.153:0/2264720864 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8358017400 con 0x7f8370104050 2024-03-21T10:07:00.784 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:00.781+0000 7f8375e00700 1 -- 172.21.15.153:0/2264720864 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f834c005320 con 0x7f8370104050 2024-03-21T10:07:00.785 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:00.784+0000 7f8374a00700 1 -- 172.21.15.153:0/2264720864 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f8358003680 con 0x7f8370104050 2024-03-21T10:07:00.786 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:00.785+0000 7f8374a00700 1 --2- 172.21.15.153:0/2264720864 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f835406c4e0 0x7f835406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:07:00.787 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:00.785+0000 7f8374a00700 1 -- 172.21.15.153:0/2264720864 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f835808cdf0 con 0x7f8370104050 2024-03-21T10:07:00.787 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:00.785+0000 7f836ea00700 1 --2- 172.21.15.153:0/2264720864 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f835406c4e0 0x7f835406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:07:00.787 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:00.786+0000 7f836ea00700 1 --2- 172.21.15.153:0/2264720864 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f835406c4e0 0x7f835406e9a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f8360009ef0 tx=0x7f8360009260 comp rx=0 tx=0).ready entity=mgr.14221 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:07:00.795 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:00.793+0000 7f8374a00700 1 -- 172.21.15.153:0/2264720864 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f835805b020 con 0x7f8370104050 2024-03-21T10:07:01.133 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:01.131+0000 7f8375e00700 1 -- 172.21.15.153:0/2264720864 --> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f834c000bf0 con 0x7f835406c4e0 2024-03-21T10:07:01.135 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:01.134+0000 7f8374a00700 1 -- 172.21.15.153:0/2264720864 <== mgr.14221 v2:172.21.15.153:6800/1639929315 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7f834c000bf0 con 0x7f835406c4e0 2024-03-21T10:07:01.139 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:01.137+0000 7f8375e00700 1 -- 172.21.15.153:0/2264720864 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f835406c4e0 msgr2=0x7f835406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:07:01.139 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:01.137+0000 7f8375e00700 1 --2- 172.21.15.153:0/2264720864 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f835406c4e0 0x7f835406e9a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f8360009ef0 tx=0x7f8360009260 comp rx=0 tx=0).stop 2024-03-21T10:07:01.139 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:01.137+0000 7f8375e00700 1 -- 172.21.15.153:0/2264720864 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8370104050 msgr2=0x7f8370192c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:07:01.139 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:01.137+0000 7f8375e00700 1 --2- 172.21.15.153:0/2264720864 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8370104050 0x7f8370192c80 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8358009910 tx=0x7f835800f690 comp rx=0 tx=0).stop 2024-03-21T10:07:01.139 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:01.137+0000 7f8375e00700 1 -- 172.21.15.153:0/2264720864 shutdown_connections 2024-03-21T10:07:01.139 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:01.137+0000 7f8375e00700 1 --2- 172.21.15.153:0/2264720864 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f835406c4e0 0x7f835406e9a0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:07:01.139 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:01.137+0000 7f8375e00700 1 --2- 172.21.15.153:0/2264720864 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8370101720 0x7f8370192740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:07:01.139 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:01.137+0000 7f8375e00700 1 --2- 172.21.15.153:0/2264720864 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8370104050 0x7f8370192c80 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:07:01.140 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:01.137+0000 7f8375e00700 1 -- 172.21.15.153:0/2264720864 >> 172.21.15.153:0/2264720864 conn(0x7f83700fb070 msgr2=0x7f8370104cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:07:01.140 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:01.138+0000 7f8375e00700 1 -- 172.21.15.153:0/2264720864 shutdown_connections 2024-03-21T10:07:01.140 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:01.138+0000 7f8375e00700 1 -- 172.21.15.153:0/2264720864 wait complete. 2024-03-21T10:07:01.158 INFO:teuthology.orchestra.run.smithi153.stdout:true 2024-03-21T10:07:01.324 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:01.322+0000 7f42dbe00700 1 -- 172.21.15.153:0/2881898162 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f42d4074d40 msgr2=0x7f42d40731a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:07:01.324 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:01.322+0000 7f42dbe00700 1 --2- 172.21.15.153:0/2881898162 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f42d4074d40 0x7f42d40731a0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f42c8009910 tx=0x7f42c8009c20 comp rx=0 tx=0).stop 2024-03-21T10:07:01.324 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:01.322+0000 7f42dbe00700 1 -- 172.21.15.153:0/2881898162 shutdown_connections 2024-03-21T10:07:01.324 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:01.322+0000 7f42dbe00700 1 --2- 172.21.15.153:0/2881898162 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f42d4073770 0x7f42d4073bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:07:01.324 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:01.322+0000 7f42dbe00700 1 --2- 172.21.15.153:0/2881898162 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f42d4074d40 0x7f42d40731a0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:07:01.324 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:01.322+0000 7f42dbe00700 1 -- 172.21.15.153:0/2881898162 >> 172.21.15.153:0/2881898162 conn(0x7f42d40fc190 msgr2=0x7f42d40fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:07:01.324 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:01.322+0000 7f42dbe00700 1 -- 172.21.15.153:0/2881898162 shutdown_connections 2024-03-21T10:07:01.324 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:01.322+0000 7f42dbe00700 1 -- 172.21.15.153:0/2881898162 wait complete. 2024-03-21T10:07:01.325 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:01.324+0000 7f42dbe00700 1 Processor -- start 2024-03-21T10:07:01.325 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:01.324+0000 7f42dbe00700 1 -- start start 2024-03-21T10:07:01.326 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:01.324+0000 7f42dbe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f42d4073770 0x7f42d4198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:07:01.326 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:01.324+0000 7f42dbe00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f42d4074d40 0x7f42d4198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:07:01.326 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:01.324+0000 7f42dbe00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42d41990d0 con 0x7f42d4073770 2024-03-21T10:07:01.326 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:01.324+0000 7f42dbe00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42d4199210 con 0x7f42d4074d40 2024-03-21T10:07:01.327 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:01.325+0000 7f42d9400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f42d4073770 0x7f42d4198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:07:01.327 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:01.325+0000 7f42d9400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f42d4073770 0x7f42d4198570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:33268/0 (socket says 172.21.15.153:33268) 2024-03-21T10:07:01.327 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:01.325+0000 7f42d9400700 1 -- 172.21.15.153:0/4218663120 learned_addr learned my addr 172.21.15.153:0/4218663120 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:07:01.327 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:01.325+0000 7f42d8a00700 1 --2- 172.21.15.153:0/4218663120 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f42d4074d40 0x7f42d4198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:07:01.327 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:01.325+0000 7f42d9400700 1 -- 172.21.15.153:0/4218663120 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f42d4074d40 msgr2=0x7f42d4198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:07:01.327 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:01.325+0000 7f42d9400700 1 --2- 172.21.15.153:0/4218663120 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f42d4074d40 0x7f42d4198ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:07:01.327 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:01.325+0000 7f42d9400700 1 -- 172.21.15.153:0/4218663120 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42c80095f0 con 0x7f42d4073770 2024-03-21T10:07:01.327 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:01.325+0000 7f42d8a00700 1 --2- 172.21.15.153:0/4218663120 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f42d4074d40 0x7f42d4198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:07:01.328 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:01.326+0000 7f42d9400700 1 --2- 172.21.15.153:0/4218663120 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f42d4073770 0x7f42d4198570 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f42c800b5c0 tx=0x7f42c800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:07:01.328 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:01.326+0000 7f42d2000700 1 -- 172.21.15.153:0/4218663120 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f42c801d070 con 0x7f42d4073770 2024-03-21T10:07:01.328 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:01.326+0000 7f42dbe00700 1 -- 172.21.15.153:0/4218663120 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42d419dc60 con 0x7f42d4073770 2024-03-21T10:07:01.328 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:01.326+0000 7f42d2000700 1 -- 172.21.15.153:0/4218663120 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f42c800f460 con 0x7f42d4073770 2024-03-21T10:07:01.328 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:01.326+0000 7f42d2000700 1 -- 172.21.15.153:0/4218663120 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f42c8021640 con 0x7f42d4073770 2024-03-21T10:07:01.328 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:01.326+0000 7f42dbe00700 1 -- 172.21.15.153:0/4218663120 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42d419e150 con 0x7f42d4073770 2024-03-21T10:07:01.329 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:01.327+0000 7f42dbe00700 1 -- 172.21.15.153:0/4218663120 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42d410b940 con 0x7f42d4073770 2024-03-21T10:07:01.330 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:01.329+0000 7f42d2000700 1 -- 172.21.15.153:0/4218663120 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f42c800f5d0 con 0x7f42d4073770 2024-03-21T10:07:01.331 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:01.330+0000 7f42d2000700 1 --2- 172.21.15.153:0/4218663120 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f42bc06c4e0 0x7f42bc06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:07:01.331 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:01.330+0000 7f42d2000700 1 -- 172.21.15.153:0/4218663120 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f42c808ccf0 con 0x7f42d4073770 2024-03-21T10:07:01.332 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:01.330+0000 7f42d8a00700 1 --2- 172.21.15.153:0/4218663120 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f42bc06c4e0 0x7f42bc06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:07:01.332 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:01.331+0000 7f42d8a00700 1 --2- 172.21.15.153:0/4218663120 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f42bc06c4e0 0x7f42bc06e9a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f42c000bcf0 tx=0x7f42c000b2c0 comp rx=0 tx=0).ready entity=mgr.14221 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:07:01.338 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:01.337+0000 7f42d2000700 1 -- 172.21.15.153:0/4218663120 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f42c805ae60 con 0x7f42d4073770 2024-03-21T10:07:01.649 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:01.647+0000 7f42dbe00700 1 -- 172.21.15.153:0/4218663120 --> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f42d4108150 con 0x7f42bc06c4e0 2024-03-21T10:07:01.652 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:01.651+0000 7f42d2000700 1 -- 172.21.15.153:0/4218663120 <== mgr.14221 v2:172.21.15.153:6800/1639929315 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7f42d4108150 con 0x7f42bc06c4e0 2024-03-21T10:07:01.655 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:01.653+0000 7f42dbe00700 1 -- 172.21.15.153:0/4218663120 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f42bc06c4e0 msgr2=0x7f42bc06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:07:01.656 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:01.653+0000 7f42dbe00700 1 --2- 172.21.15.153:0/4218663120 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f42bc06c4e0 0x7f42bc06e9a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f42c000bcf0 tx=0x7f42c000b2c0 comp rx=0 tx=0).stop 2024-03-21T10:07:01.656 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:01.654+0000 7f42dbe00700 1 -- 172.21.15.153:0/4218663120 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f42d4073770 msgr2=0x7f42d4198570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:07:01.656 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:01.654+0000 7f42dbe00700 1 --2- 172.21.15.153:0/4218663120 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f42d4073770 0x7f42d4198570 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f42c800b5c0 tx=0x7f42c800be20 comp rx=0 tx=0).stop 2024-03-21T10:07:01.656 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:01.654+0000 7f42dbe00700 1 -- 172.21.15.153:0/4218663120 shutdown_connections 2024-03-21T10:07:01.656 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:01.654+0000 7f42dbe00700 1 --2- 172.21.15.153:0/4218663120 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f42bc06c4e0 0x7f42bc06e9a0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:07:01.656 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:01.654+0000 7f42dbe00700 1 --2- 172.21.15.153:0/4218663120 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f42d4073770 0x7f42d4198570 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:07:01.656 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:01.654+0000 7f42dbe00700 1 --2- 172.21.15.153:0/4218663120 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f42d4074d40 0x7f42d4198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:07:01.656 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:01.654+0000 7f42dbe00700 1 -- 172.21.15.153:0/4218663120 >> 172.21.15.153:0/4218663120 conn(0x7f42d40fc190 msgr2=0x7f42d4106a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:07:01.656 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:01.654+0000 7f42dbe00700 1 -- 172.21.15.153:0/4218663120 shutdown_connections 2024-03-21T10:07:01.656 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:01.654+0000 7f42dbe00700 1 -- 172.21.15.153:0/4218663120 wait complete. 2024-03-21T10:07:01.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:01 smithi153 ceph-mon[29117]: pgmap v293: 65 pgs: 65 active+clean; 604 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 1.4 KiB/s wr, 4 op/s 2024-03-21T10:07:01.837 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:01.835+0000 7f1d38400700 1 -- 172.21.15.153:0/3940343566 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1d30103ff0 msgr2=0x7f1d30104450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:07:01.837 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:01.835+0000 7f1d38400700 1 --2- 172.21.15.153:0/3940343566 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1d30103ff0 0x7f1d30104450 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f1d1c009910 tx=0x7f1d1c009c20 comp rx=0 tx=0).stop 2024-03-21T10:07:01.837 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:01.836+0000 7f1d38400700 1 -- 172.21.15.153:0/3940343566 shutdown_connections 2024-03-21T10:07:01.837 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:01.836+0000 7f1d38400700 1 --2- 172.21.15.153:0/3940343566 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1d30103ff0 0x7f1d30104450 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:07:01.837 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:01.836+0000 7f1d38400700 1 --2- 172.21.15.153:0/3940343566 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1d30102df0 0x7f1d30103210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:07:01.837 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:01.836+0000 7f1d38400700 1 -- 172.21.15.153:0/3940343566 >> 172.21.15.153:0/3940343566 conn(0x7f1d300fe390 msgr2=0x7f1d301007d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:07:01.838 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:01.836+0000 7f1d38400700 1 -- 172.21.15.153:0/3940343566 shutdown_connections 2024-03-21T10:07:01.838 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:01.836+0000 7f1d38400700 1 -- 172.21.15.153:0/3940343566 wait complete. 2024-03-21T10:07:01.839 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:01.838+0000 7f1d38400700 1 Processor -- start 2024-03-21T10:07:01.839 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:01.838+0000 7f1d38400700 1 -- start start 2024-03-21T10:07:01.839 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:01.838+0000 7f1d38400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1d30102df0 0x7f1d301985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:07:01.840 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:01.838+0000 7f1d38400700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1d30103ff0 0x7f1d30198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:07:01.840 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:01.838+0000 7f1d38400700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d30199100 con 0x7f1d30102df0 2024-03-21T10:07:01.840 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:01.838+0000 7f1d38400700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d30199240 con 0x7f1d30103ff0 2024-03-21T10:07:01.840 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:01.839+0000 7f1d35a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1d30102df0 0x7f1d301985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:07:01.841 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:01.839+0000 7f1d35a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1d30102df0 0x7f1d301985a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:33282/0 (socket says 172.21.15.153:33282) 2024-03-21T10:07:01.841 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:01.839+0000 7f1d35a00700 1 -- 172.21.15.153:0/3081419206 learned_addr learned my addr 172.21.15.153:0/3081419206 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:07:01.841 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:01.839+0000 7f1d35000700 1 --2- 172.21.15.153:0/3081419206 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1d30103ff0 0x7f1d30198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:07:01.841 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:01.839+0000 7f1d35a00700 1 -- 172.21.15.153:0/3081419206 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1d30103ff0 msgr2=0x7f1d30198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:07:01.841 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:01.839+0000 7f1d35a00700 1 --2- 172.21.15.153:0/3081419206 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1d30103ff0 0x7f1d30198ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:07:01.841 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:01.839+0000 7f1d35a00700 1 -- 172.21.15.153:0/3081419206 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d1c0095f0 con 0x7f1d30102df0 2024-03-21T10:07:01.842 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:01.840+0000 7f1d35a00700 1 --2- 172.21.15.153:0/3081419206 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1d30102df0 0x7f1d301985a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f1d2400ada0 tx=0x7f1d2400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:07:01.842 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:01.840+0000 7f1d2e000700 1 -- 172.21.15.153:0/3081419206 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1d2400f070 con 0x7f1d30102df0 2024-03-21T10:07:01.842 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:01.840+0000 7f1d2e000700 1 -- 172.21.15.153:0/3081419206 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f1d24007be0 con 0x7f1d30102df0 2024-03-21T10:07:01.842 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:01.840+0000 7f1d38400700 1 -- 172.21.15.153:0/3081419206 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d3019dcf0 con 0x7f1d30102df0 2024-03-21T10:07:01.842 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:01.840+0000 7f1d38400700 1 -- 172.21.15.153:0/3081419206 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d3019e1e0 con 0x7f1d30102df0 2024-03-21T10:07:01.842 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:01.840+0000 7f1d2e000700 1 -- 172.21.15.153:0/3081419206 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1d240083f0 con 0x7f1d30102df0 2024-03-21T10:07:01.843 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:01.842+0000 7f1d38400700 1 -- 172.21.15.153:0/3081419206 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d3010b9c0 con 0x7f1d30102df0 2024-03-21T10:07:01.844 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:01.843+0000 7f1d2e000700 1 -- 172.21.15.153:0/3081419206 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f1d2400cad0 con 0x7f1d30102df0 2024-03-21T10:07:01.846 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:01.844+0000 7f1d2e000700 1 --2- 172.21.15.153:0/3081419206 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f1d1806c4e0 0x7f1d1806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:07:01.846 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:01.844+0000 7f1d2e000700 1 -- 172.21.15.153:0/3081419206 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f1d2408b330 con 0x7f1d30102df0 2024-03-21T10:07:01.846 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:01.845+0000 7f1d35000700 1 --2- 172.21.15.153:0/3081419206 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f1d1806c4e0 0x7f1d1806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:07:01.847 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:01.845+0000 7f1d35000700 1 --2- 172.21.15.153:0/3081419206 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f1d1806c4e0 0x7f1d1806e9a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1d1c006010 tx=0x7f1d1c00b580 comp rx=0 tx=0).ready entity=mgr.14221 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:07:01.851 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:01.849+0000 7f1d2e000700 1 -- 172.21.15.153:0/3081419206 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f1d240594a0 con 0x7f1d30102df0 2024-03-21T10:07:01.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:01 smithi174 ceph-mon[37496]: pgmap v293: 65 pgs: 65 active+clean; 604 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 1.4 KiB/s wr, 4 op/s 2024-03-21T10:07:02.117 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:02.115+0000 7f1d38400700 1 -- 172.21.15.153:0/3081419206 --> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f1d30108940 con 0x7f1d1806c4e0 2024-03-21T10:07:02.131 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:02.129+0000 7f1d2e000700 1 -- 172.21.15.153:0/3081419206 <== mgr.14221 v2:172.21.15.153:6800/1639929315 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3480 (secure 0 0 0) 0x7f1d30108940 con 0x7f1d1806c4e0 2024-03-21T10:07:02.131 INFO:teuthology.orchestra.run.smithi153.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-21T10:07:02.131 INFO:teuthology.orchestra.run.smithi153.stdout:alertmanager.smithi153 smithi153 *:9093,9094 running (7m) 117s ago 9m 13.0M - 0.25.0 c8568f914cd2 2cfcc225c7e6 2024-03-21T10:07:02.131 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi153 smithi153 running (9m) 117s ago 9m 7587k - 18.2.1 5be31c24972a 5b61994396a6 2024-03-21T10:07:02.131 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi174 smithi174 running (8m) 2m ago 8m 7428k - 18.2.1 5be31c24972a 4f7a5fc3ac28 2024-03-21T10:07:02.132 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi153 smithi153 running (9m) 117s ago 9m 6665k - 18.2.1 5be31c24972a 90a8b8ea03d1 2024-03-21T10:07:02.132 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi174 smithi174 running (8m) 2m ago 8m 6665k - 18.2.1 5be31c24972a a4c929e4dc77 2024-03-21T10:07:02.132 INFO:teuthology.orchestra.run.smithi153.stdout:grafana.smithi153 smithi153 *:3000 running (7m) 117s ago 9m 73.6M - 9.4.7 954c08fa6188 3af5800ecfef 2024-03-21T10:07:02.132 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.iuyluj smithi153 running (2m) 117s ago 2m 15.0M - 18.2.1 5be31c24972a 02f667cee979 2024-03-21T10:07:02.132 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.zipckr smithi153 running (2m) 117s ago 2m 12.9M - 18.2.1 5be31c24972a 68a853929f8f 2024-03-21T10:07:02.132 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.nhozag smithi174 running (2m) 2m ago 2m 12.9M - 18.2.1 5be31c24972a 2fee4000308f 2024-03-21T10:07:02.132 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.yyimst smithi174 running (2m) 2m ago 2m 13.9M - 18.2.1 5be31c24972a d78a729220d1 2024-03-21T10:07:02.132 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi153.tluxni smithi153 *:9283,8765,8443 running (11m) 117s ago 11m 491M - 18.2.1 5be31c24972a 4978543154ff 2024-03-21T10:07:02.132 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi174.qpcwat smithi174 *:8443,9283,8765 running (8m) 2m ago 8m 441M - 18.2.1 5be31c24972a 6d0c64be5134 2024-03-21T10:07:02.132 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi153 smithi153 running (11m) 117s ago 11m 43.7M 2048M 18.2.1 5be31c24972a 813963a95d69 2024-03-21T10:07:02.132 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi174 smithi174 running (7m) 2m ago 7m 34.2M 2048M 18.2.1 5be31c24972a 99204923eec6 2024-03-21T10:07:02.132 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi153 smithi153 *:9100 running (9m) 117s ago 9m 7608k - 1.5.0 0da6a335fe13 babbc29a0eaf 2024-03-21T10:07:02.132 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi174 smithi174 *:9100 running (8m) 2m ago 8m 7952k - 1.5.0 0da6a335fe13 86841435042a 2024-03-21T10:07:02.132 INFO:teuthology.orchestra.run.smithi153.stdout:osd.0 smithi153 running (7m) 117s ago 7m 46.3M 3947M 18.2.1 5be31c24972a 28912a2319b4 2024-03-21T10:07:02.133 INFO:teuthology.orchestra.run.smithi153.stdout:osd.1 smithi153 running (6m) 117s ago 6m 47.2M 3947M 18.2.1 5be31c24972a ca60681e2314 2024-03-21T10:07:02.133 INFO:teuthology.orchestra.run.smithi153.stdout:osd.2 smithi153 running (5m) 117s ago 5m 45.4M 3947M 18.2.1 5be31c24972a 8db799f8962a 2024-03-21T10:07:02.133 INFO:teuthology.orchestra.run.smithi153.stdout:osd.3 smithi174 running (5m) 2m ago 5m 45.0M 4971M 18.2.1 5be31c24972a dafc89e0658d 2024-03-21T10:07:02.133 INFO:teuthology.orchestra.run.smithi153.stdout:osd.4 smithi174 running (4m) 2m ago 4m 43.1M 4971M 18.2.1 5be31c24972a c4cd15701f05 2024-03-21T10:07:02.133 INFO:teuthology.orchestra.run.smithi153.stdout:osd.5 smithi174 running (4m) 2m ago 4m 44.4M 4971M 18.2.1 5be31c24972a 3b69886ce22e 2024-03-21T10:07:02.133 INFO:teuthology.orchestra.run.smithi153.stdout:prometheus.smithi153 smithi153 *:9095 running (7m) 117s ago 8m 53.9M - 2.43.0 a07b618ecd1d c30cdeb81cc9 2024-03-21T10:07:02.134 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:02.132+0000 7f1d38400700 1 -- 172.21.15.153:0/3081419206 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f1d1806c4e0 msgr2=0x7f1d1806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:07:02.134 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:02.132+0000 7f1d38400700 1 --2- 172.21.15.153:0/3081419206 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f1d1806c4e0 0x7f1d1806e9a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1d1c006010 tx=0x7f1d1c00b580 comp rx=0 tx=0).stop 2024-03-21T10:07:02.134 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:02.132+0000 7f1d38400700 1 -- 172.21.15.153:0/3081419206 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1d30102df0 msgr2=0x7f1d301985a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:07:02.134 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:02.132+0000 7f1d38400700 1 --2- 172.21.15.153:0/3081419206 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1d30102df0 0x7f1d301985a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f1d2400ada0 tx=0x7f1d2400c330 comp rx=0 tx=0).stop 2024-03-21T10:07:02.134 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:02.132+0000 7f1d38400700 1 -- 172.21.15.153:0/3081419206 shutdown_connections 2024-03-21T10:07:02.134 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:02.132+0000 7f1d38400700 1 --2- 172.21.15.153:0/3081419206 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7f1d1806c4e0 0x7f1d1806e9a0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:07:02.134 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:02.132+0000 7f1d38400700 1 --2- 172.21.15.153:0/3081419206 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1d30102df0 0x7f1d301985a0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:07:02.134 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:02.132+0000 7f1d38400700 1 --2- 172.21.15.153:0/3081419206 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1d30103ff0 0x7f1d30198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:07:02.134 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:02.132+0000 7f1d38400700 1 -- 172.21.15.153:0/3081419206 >> 172.21.15.153:0/3081419206 conn(0x7f1d300fe390 msgr2=0x7f1d30107220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:07:02.135 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:02.133+0000 7f1d38400700 1 -- 172.21.15.153:0/3081419206 shutdown_connections 2024-03-21T10:07:02.135 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:02.133+0000 7f1d38400700 1 -- 172.21.15.153:0/3081419206 wait complete. 2024-03-21T10:07:02.307 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:02.306+0000 7fc71e000700 1 -- 172.21.15.153:0/2857310438 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc718100cb0 msgr2=0x7fc718101110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:07:02.308 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:02.306+0000 7fc71e000700 1 --2- 172.21.15.153:0/2857310438 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc718100cb0 0x7fc718101110 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fc704009910 tx=0x7fc704009c20 comp rx=0 tx=0).stop 2024-03-21T10:07:02.308 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:02.306+0000 7fc71e000700 1 -- 172.21.15.153:0/2857310438 shutdown_connections 2024-03-21T10:07:02.308 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:02.306+0000 7fc71e000700 1 --2- 172.21.15.153:0/2857310438 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc718100cb0 0x7fc718101110 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:07:02.308 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:02.306+0000 7fc71e000700 1 --2- 172.21.15.153:0/2857310438 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc7180ffab0 0x7fc7180ffed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:07:02.308 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:02.306+0000 7fc71e000700 1 -- 172.21.15.153:0/2857310438 >> 172.21.15.153:0/2857310438 conn(0x7fc7180fb050 msgr2=0x7fc7180fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:07:02.308 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:02.306+0000 7fc71e000700 1 -- 172.21.15.153:0/2857310438 shutdown_connections 2024-03-21T10:07:02.308 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:02.306+0000 7fc71e000700 1 -- 172.21.15.153:0/2857310438 wait complete. 2024-03-21T10:07:02.309 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:02.307+0000 7fc71e000700 1 Processor -- start 2024-03-21T10:07:02.309 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:02.307+0000 7fc71e000700 1 -- start start 2024-03-21T10:07:02.309 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:02.308+0000 7fc71e000700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc7180ffab0 0x7fc718198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:07:02.309 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:02.308+0000 7fc71e000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc718100cb0 0x7fc718198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:07:02.309 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:02.308+0000 7fc71e000700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7181990b0 con 0x7fc718100cb0 2024-03-21T10:07:02.309 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:02.308+0000 7fc71e000700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7181991f0 con 0x7fc7180ffab0 2024-03-21T10:07:02.310 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:02.308+0000 7fc716a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc718100cb0 0x7fc718198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:07:02.310 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:02.308+0000 7fc716a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc718100cb0 0x7fc718198a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:33298/0 (socket says 172.21.15.153:33298) 2024-03-21T10:07:02.310 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:02.308+0000 7fc716a00700 1 -- 172.21.15.153:0/2348918107 learned_addr learned my addr 172.21.15.153:0/2348918107 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:07:02.310 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:02.308+0000 7fc717400700 1 --2- 172.21.15.153:0/2348918107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc7180ffab0 0x7fc718198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:07:02.310 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:02.309+0000 7fc716a00700 1 -- 172.21.15.153:0/2348918107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc7180ffab0 msgr2=0x7fc718198550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:07:02.310 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:02.309+0000 7fc716a00700 1 --2- 172.21.15.153:0/2348918107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc7180ffab0 0x7fc718198550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:07:02.310 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:02.309+0000 7fc716a00700 1 -- 172.21.15.153:0/2348918107 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7040095f0 con 0x7fc718100cb0 2024-03-21T10:07:02.311 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:02.309+0000 7fc716a00700 1 --2- 172.21.15.153:0/2348918107 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc718100cb0 0x7fc718198a90 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fc704006010 tx=0x7fc70400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:07:02.311 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:02.309+0000 7fc70fe00700 1 -- 172.21.15.153:0/2348918107 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc70401d070 con 0x7fc718100cb0 2024-03-21T10:07:02.311 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:02.309+0000 7fc70fe00700 1 -- 172.21.15.153:0/2348918107 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc70400f460 con 0x7fc718100cb0 2024-03-21T10:07:02.311 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:02.309+0000 7fc71e000700 1 -- 172.21.15.153:0/2348918107 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc71819dc40 con 0x7fc718100cb0 2024-03-21T10:07:02.311 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:02.309+0000 7fc70fe00700 1 -- 172.21.15.153:0/2348918107 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc704017630 con 0x7fc718100cb0 2024-03-21T10:07:02.311 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:02.310+0000 7fc71e000700 1 -- 172.21.15.153:0/2348918107 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc71819e130 con 0x7fc718100cb0 2024-03-21T10:07:02.312 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:02.311+0000 7fc71e000700 1 -- 172.21.15.153:0/2348918107 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc718108610 con 0x7fc718100cb0 2024-03-21T10:07:02.314 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:02.312+0000 7fc70fe00700 1 -- 172.21.15.153:0/2348918107 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fc704003a60 con 0x7fc718100cb0 2024-03-21T10:07:02.315 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:02.313+0000 7fc70fe00700 1 --2- 172.21.15.153:0/2348918107 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fc6fc06c4e0 0x7fc6fc06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:07:02.315 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:02.313+0000 7fc70fe00700 1 -- 172.21.15.153:0/2348918107 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fc70408cd90 con 0x7fc718100cb0 2024-03-21T10:07:02.315 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:02.314+0000 7fc717400700 1 --2- 172.21.15.153:0/2348918107 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fc6fc06c4e0 0x7fc6fc06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:07:02.316 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:02.315+0000 7fc717400700 1 --2- 172.21.15.153:0/2348918107 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fc6fc06c4e0 0x7fc6fc06e9a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc708005950 tx=0x7fc7080058e0 comp rx=0 tx=0).ready entity=mgr.14221 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:07:02.320 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:02.319+0000 7fc70fe00700 1 -- 172.21.15.153:0/2348918107 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fc70405af80 con 0x7fc718100cb0 2024-03-21T10:07:02.697 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:02.695+0000 7fc71e000700 1 -- 172.21.15.153:0/2348918107 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fc718108800 con 0x7fc718100cb0 2024-03-21T10:07:02.697 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:02.696+0000 7fc70fe00700 1 -- 172.21.15.153:0/2348918107 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+558 (secure 0 0 0) 0x7fc70405ab10 con 0x7fc718100cb0 2024-03-21T10:07:02.698 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:07:02.698 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": { 2024-03-21T10:07:02.698 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-03-21T10:07:02.698 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:07:02.698 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": { 2024-03-21T10:07:02.698 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-03-21T10:07:02.698 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:07:02.698 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": { 2024-03-21T10:07:02.698 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-03-21T10:07:02.698 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:07:02.699 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": { 2024-03-21T10:07:02.699 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-03-21T10:07:02.699 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:07:02.699 INFO:teuthology.orchestra.run.smithi153.stdout: "overall": { 2024-03-21T10:07:02.699 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 14 2024-03-21T10:07:02.699 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-03-21T10:07:02.699 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:07:02.701 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:02.699+0000 7fc71e000700 1 -- 172.21.15.153:0/2348918107 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fc6fc06c4e0 msgr2=0x7fc6fc06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:07:02.701 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:02.699+0000 7fc71e000700 1 --2- 172.21.15.153:0/2348918107 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fc6fc06c4e0 0x7fc6fc06e9a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc708005950 tx=0x7fc7080058e0 comp rx=0 tx=0).stop 2024-03-21T10:07:02.701 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:02.699+0000 7fc71e000700 1 -- 172.21.15.153:0/2348918107 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc718100cb0 msgr2=0x7fc718198a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:07:02.701 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:02.699+0000 7fc71e000700 1 --2- 172.21.15.153:0/2348918107 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc718100cb0 0x7fc718198a90 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fc704006010 tx=0x7fc70400be20 comp rx=0 tx=0).stop 2024-03-21T10:07:02.701 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:02.699+0000 7fc71e000700 1 -- 172.21.15.153:0/2348918107 shutdown_connections 2024-03-21T10:07:02.701 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:02.699+0000 7fc71e000700 1 --2- 172.21.15.153:0/2348918107 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7fc6fc06c4e0 0x7fc6fc06e9a0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:07:02.701 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:02.699+0000 7fc71e000700 1 --2- 172.21.15.153:0/2348918107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc7180ffab0 0x7fc718198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:07:02.701 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:02.699+0000 7fc71e000700 1 --2- 172.21.15.153:0/2348918107 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc718100cb0 0x7fc718198a90 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:07:02.701 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:02.699+0000 7fc71e000700 1 -- 172.21.15.153:0/2348918107 >> 172.21.15.153:0/2348918107 conn(0x7fc7180fb050 msgr2=0x7fc718068ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:07:02.702 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:02.700+0000 7fc71e000700 1 -- 172.21.15.153:0/2348918107 shutdown_connections 2024-03-21T10:07:02.702 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:02.700+0000 7fc71e000700 1 -- 172.21.15.153:0/2348918107 wait complete. 2024-03-21T10:07:02.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:02 smithi153 ceph-mon[29117]: from='client.14584 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:07:02.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:02 smithi153 ceph-mon[29117]: from='client.14588 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:07:02.880 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:02.878+0000 7ff3a8000700 1 -- 172.21.15.153:0/3027865605 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff3a0103fd0 msgr2=0x7ff3a0104430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:07:02.880 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:02.878+0000 7ff3a8000700 1 --2- 172.21.15.153:0/3027865605 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff3a0103fd0 0x7ff3a0104430 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ff38c009910 tx=0x7ff38c009c20 comp rx=0 tx=0).stop 2024-03-21T10:07:02.880 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:02.878+0000 7ff3a8000700 1 -- 172.21.15.153:0/3027865605 shutdown_connections 2024-03-21T10:07:02.880 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:02.878+0000 7ff3a8000700 1 --2- 172.21.15.153:0/3027865605 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff3a0103fd0 0x7ff3a0104430 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:07:02.880 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:02.878+0000 7ff3a8000700 1 --2- 172.21.15.153:0/3027865605 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff3a0102dd0 0x7ff3a01031f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:07:02.881 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:02.878+0000 7ff3a8000700 1 -- 172.21.15.153:0/3027865605 >> 172.21.15.153:0/3027865605 conn(0x7ff3a00fe370 msgr2=0x7ff3a01007b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:07:02.881 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:02.879+0000 7ff3a8000700 1 -- 172.21.15.153:0/3027865605 shutdown_connections 2024-03-21T10:07:02.881 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:02.879+0000 7ff3a8000700 1 -- 172.21.15.153:0/3027865605 wait complete. 2024-03-21T10:07:02.881 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:02.880+0000 7ff3a8000700 1 Processor -- start 2024-03-21T10:07:02.881 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:02.880+0000 7ff3a8000700 1 -- start start 2024-03-21T10:07:02.882 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:02.880+0000 7ff3a8000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff3a0102dd0 0x7ff3a01985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:07:02.882 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:02.880+0000 7ff3a8000700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff3a0103fd0 0x7ff3a0198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:07:02.882 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:02.880+0000 7ff3a8000700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a0199150 con 0x7ff3a0102dd0 2024-03-21T10:07:02.882 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:02.880+0000 7ff3a8000700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a0199290 con 0x7ff3a0103fd0 2024-03-21T10:07:02.885 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:02.881+0000 7ff3a5600700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff3a0102dd0 0x7ff3a01985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:07:02.885 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:02.881+0000 7ff3a5600700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff3a0102dd0 0x7ff3a01985f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:33318/0 (socket says 172.21.15.153:33318) 2024-03-21T10:07:02.885 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:02.881+0000 7ff3a5600700 1 -- 172.21.15.153:0/4213406619 learned_addr learned my addr 172.21.15.153:0/4213406619 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:07:02.885 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:02.881+0000 7ff3a4c00700 1 --2- 172.21.15.153:0/4213406619 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff3a0103fd0 0x7ff3a0198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:07:02.886 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:02.881+0000 7ff3a5600700 1 -- 172.21.15.153:0/4213406619 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff3a0103fd0 msgr2=0x7ff3a0198b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:07:02.886 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:02.881+0000 7ff3a5600700 1 --2- 172.21.15.153:0/4213406619 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff3a0103fd0 0x7ff3a0198b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:07:02.886 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:02.881+0000 7ff3a5600700 1 -- 172.21.15.153:0/4213406619 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff38c0095f0 con 0x7ff3a0102dd0 2024-03-21T10:07:02.887 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:02.882+0000 7ff3a5600700 1 --2- 172.21.15.153:0/4213406619 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff3a0102dd0 0x7ff3a01985f0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff39400b480 tx=0x7ff39400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:07:02.887 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:02.882+0000 7ff39e000700 1 -- 172.21.15.153:0/4213406619 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff3940118c0 con 0x7ff3a0102dd0 2024-03-21T10:07:02.887 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:02.882+0000 7ff3a8000700 1 -- 172.21.15.153:0/4213406619 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3a019dd40 con 0x7ff3a0102dd0 2024-03-21T10:07:02.887 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:02.882+0000 7ff39e000700 1 -- 172.21.15.153:0/4213406619 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff394010380 con 0x7ff3a0102dd0 2024-03-21T10:07:02.887 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:02.882+0000 7ff39e000700 1 -- 172.21.15.153:0/4213406619 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff39400f4f0 con 0x7ff3a0102dd0 2024-03-21T10:07:02.887 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:02.882+0000 7ff3a8000700 1 -- 172.21.15.153:0/4213406619 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3a019e2b0 con 0x7ff3a0102dd0 2024-03-21T10:07:02.887 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:02.883+0000 7ff3a8000700 1 -- 172.21.15.153:0/4213406619 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3a010b9d0 con 0x7ff3a0102dd0 2024-03-21T10:07:02.887 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:02.885+0000 7ff39e000700 1 -- 172.21.15.153:0/4213406619 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7ff3940104f0 con 0x7ff3a0102dd0 2024-03-21T10:07:02.888 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:02.886+0000 7ff39e000700 1 --2- 172.21.15.153:0/4213406619 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7ff38806c4e0 0x7ff38806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:07:02.888 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:02.886+0000 7ff39e000700 1 -- 172.21.15.153:0/4213406619 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7ff39408b4b0 con 0x7ff3a0102dd0 2024-03-21T10:07:02.888 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:02.886+0000 7ff3a4c00700 1 --2- 172.21.15.153:0/4213406619 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7ff38806c4e0 0x7ff38806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:07:02.889 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:02.887+0000 7ff3a4c00700 1 --2- 172.21.15.153:0/4213406619 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7ff38806c4e0 0x7ff38806e9a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff38c006010 tx=0x7ff38c00b540 comp rx=0 tx=0).ready entity=mgr.14221 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:07:02.892 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:02.891+0000 7ff39e000700 1 -- 172.21.15.153:0/4213406619 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7ff3940596c0 con 0x7ff3a0102dd0 2024-03-21T10:07:02.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:02 smithi174 ceph-mon[37496]: from='client.14584 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:07:02.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:02 smithi174 ceph-mon[37496]: from='client.14588 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:07:03.172 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:03.170+0000 7ff3a8000700 1 -- 172.21.15.153:0/4213406619 --> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff3a0108920 con 0x7ff38806c4e0 2024-03-21T10:07:03.173 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:03.172+0000 7ff39e000700 1 -- 172.21.15.153:0/4213406619 <== mgr.14221 v2:172.21.15.153:6800/1639929315 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7ff3a0108920 con 0x7ff38806c4e0 2024-03-21T10:07:03.174 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:07:03.174 INFO:teuthology.orchestra.run.smithi153.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232", 2024-03-21T10:07:03.174 INFO:teuthology.orchestra.run.smithi153.stdout: "in_progress": true, 2024-03-21T10:07:03.174 INFO:teuthology.orchestra.run.smithi153.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-03-21T10:07:03.174 INFO:teuthology.orchestra.run.smithi153.stdout: "services_complete": [], 2024-03-21T10:07:03.174 INFO:teuthology.orchestra.run.smithi153.stdout: "progress": "0/2 daemons upgraded", 2024-03-21T10:07:03.174 INFO:teuthology.orchestra.run.smithi153.stdout: "message": "Pulling quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 image on host smithi174", 2024-03-21T10:07:03.174 INFO:teuthology.orchestra.run.smithi153.stdout: "is_paused": false 2024-03-21T10:07:03.174 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:07:03.178 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:03.176+0000 7ff3a8000700 1 -- 172.21.15.153:0/4213406619 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7ff38806c4e0 msgr2=0x7ff38806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:07:03.179 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:03.176+0000 7ff3a8000700 1 --2- 172.21.15.153:0/4213406619 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7ff38806c4e0 0x7ff38806e9a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff38c006010 tx=0x7ff38c00b540 comp rx=0 tx=0).stop 2024-03-21T10:07:03.179 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:03.176+0000 7ff3a8000700 1 -- 172.21.15.153:0/4213406619 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff3a0102dd0 msgr2=0x7ff3a01985f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:07:03.179 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:03.176+0000 7ff3a8000700 1 --2- 172.21.15.153:0/4213406619 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff3a0102dd0 0x7ff3a01985f0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff39400b480 tx=0x7ff39400b840 comp rx=0 tx=0).stop 2024-03-21T10:07:03.179 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:03.177+0000 7ff3a8000700 1 -- 172.21.15.153:0/4213406619 shutdown_connections 2024-03-21T10:07:03.179 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:03.177+0000 7ff3a8000700 1 --2- 172.21.15.153:0/4213406619 >> [v2:172.21.15.153:6800/1639929315,v1:172.21.15.153:6801/1639929315] conn(0x7ff38806c4e0 0x7ff38806e9a0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:07:03.179 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:03.177+0000 7ff3a8000700 1 --2- 172.21.15.153:0/4213406619 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff3a0102dd0 0x7ff3a01985f0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:07:03.179 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:03.177+0000 7ff3a8000700 1 --2- 172.21.15.153:0/4213406619 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff3a0103fd0 0x7ff3a0198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:07:03.179 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:03.177+0000 7ff3a8000700 1 -- 172.21.15.153:0/4213406619 >> 172.21.15.153:0/4213406619 conn(0x7ff3a00fe370 msgr2=0x7ff3a0107200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:07:03.179 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:03.177+0000 7ff3a8000700 1 -- 172.21.15.153:0/4213406619 shutdown_connections 2024-03-21T10:07:03.179 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:03.177+0000 7ff3a8000700 1 -- 172.21.15.153:0/4213406619 wait complete. 2024-03-21T10:07:03.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:03 smithi153 ceph-mon[29117]: from='client.14592 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:07:03.915 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:03 smithi153 ceph-mon[29117]: pgmap v294: 65 pgs: 65 active+clean; 24 MiB data, 226 MiB used, 536 GiB / 536 GiB avail; 45 KiB/s rd, 2.0 MiB/s wr, 30 op/s 2024-03-21T10:07:03.915 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:03 smithi153 ceph-mon[29117]: from='client.? 172.21.15.153:0/2348918107' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:07:03.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:03 smithi174 ceph-mon[37496]: from='client.14592 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:07:03.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:03 smithi174 ceph-mon[37496]: pgmap v294: 65 pgs: 65 active+clean; 24 MiB data, 226 MiB used, 536 GiB / 536 GiB avail; 45 KiB/s rd, 2.0 MiB/s wr, 30 op/s 2024-03-21T10:07:03.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:03 smithi174 ceph-mon[37496]: from='client.? 172.21.15.153:0/2348918107' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:07:04.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:04 smithi153 ceph-mon[29117]: from='client.14600 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:07:04.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:04 smithi174 ceph-mon[37496]: from='client.14600 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:07:05.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:05 smithi153 ceph-mon[29117]: pgmap v295: 65 pgs: 65 active+clean; 25 MiB data, 234 MiB used, 536 GiB / 536 GiB avail; 44 KiB/s rd, 2.0 MiB/s wr, 31 op/s 2024-03-21T10:07:05.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:05 smithi174 ceph-mon[37496]: pgmap v295: 65 pgs: 65 active+clean; 25 MiB data, 234 MiB used, 536 GiB / 536 GiB avail; 44 KiB/s rd, 2.0 MiB/s wr, 31 op/s 2024-03-21T10:07:07.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:07 smithi153 ceph-mon[29117]: pgmap v296: 65 pgs: 65 active+clean; 28 MiB data, 238 MiB used, 536 GiB / 536 GiB avail; 343 KiB/s rd, 2.3 MiB/s wr, 50 op/s 2024-03-21T10:07:07.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:07 smithi153 ceph-mon[29117]: fsmap cephfs:2 {0=cephfs.smithi153.iuyluj=up:active,1=cephfs.smithi174.yyimst=up:active} 2 up:standby-replay 2024-03-21T10:07:07.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:07 smithi174 ceph-mon[37496]: pgmap v296: 65 pgs: 65 active+clean; 28 MiB data, 238 MiB used, 536 GiB / 536 GiB avail; 343 KiB/s rd, 2.3 MiB/s wr, 50 op/s 2024-03-21T10:07:07.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:07 smithi174 ceph-mon[37496]: fsmap cephfs:2 {0=cephfs.smithi153.iuyluj=up:active,1=cephfs.smithi174.yyimst=up:active} 2 up:standby-replay 2024-03-21T10:07:09.703 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:09 smithi174 ceph-mon[37496]: pgmap v297: 65 pgs: 65 active+clean; 30 MiB data, 243 MiB used, 536 GiB / 536 GiB avail; 342 KiB/s rd, 2.5 MiB/s wr, 63 op/s 2024-03-21T10:07:09.703 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:09 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:07:09.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:09 smithi153 ceph-mon[29117]: pgmap v297: 65 pgs: 65 active+clean; 30 MiB data, 243 MiB used, 536 GiB / 536 GiB avail; 342 KiB/s rd, 2.5 MiB/s wr, 63 op/s 2024-03-21T10:07:09.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:09 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:07:11.641 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:11 smithi174 ceph-mon[37496]: pgmap v298: 65 pgs: 65 active+clean; 30 MiB data, 246 MiB used, 536 GiB / 536 GiB avail; 342 KiB/s rd, 2.5 MiB/s wr, 64 op/s 2024-03-21T10:07:11.641 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:11 smithi174 ceph-mon[37496]: Upgrade: Updating mgr.smithi174.qpcwat 2024-03-21T10:07:11.641 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:11 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:07:11.641 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:11 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi174.qpcwat", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-21T10:07:11.641 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:11 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-21T10:07:11.641 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:11 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:07:11.641 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:11 smithi174 ceph-mon[37496]: Deploying daemon mgr.smithi174.qpcwat on smithi174 2024-03-21T10:07:11.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:11 smithi153 ceph-mon[29117]: pgmap v298: 65 pgs: 65 active+clean; 30 MiB data, 246 MiB used, 536 GiB / 536 GiB avail; 342 KiB/s rd, 2.5 MiB/s wr, 64 op/s 2024-03-21T10:07:11.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:11 smithi153 ceph-mon[29117]: Upgrade: Updating mgr.smithi174.qpcwat 2024-03-21T10:07:11.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:11 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:07:11.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:11 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi174.qpcwat", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-21T10:07:11.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:11 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-21T10:07:11.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:11 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:07:11.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:11 smithi153 ceph-mon[29117]: Deploying daemon mgr.smithi174.qpcwat on smithi174 2024-03-21T10:07:13.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:13 smithi153 ceph-mon[29117]: pgmap v299: 65 pgs: 65 active+clean; 36 MiB data, 269 MiB used, 536 GiB / 536 GiB avail; 780 KiB/s rd, 3.0 MiB/s wr, 90 op/s 2024-03-21T10:07:13.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:13 smithi174 ceph-mon[37496]: pgmap v299: 65 pgs: 65 active+clean; 36 MiB data, 269 MiB used, 536 GiB / 536 GiB avail; 780 KiB/s rd, 3.0 MiB/s wr, 90 op/s 2024-03-21T10:07:15.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:15 smithi174 ceph-mon[37496]: pgmap v300: 65 pgs: 65 active+clean; 36 MiB data, 271 MiB used, 536 GiB / 536 GiB avail; 736 KiB/s rd, 1.0 MiB/s wr, 64 op/s 2024-03-21T10:07:15.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:15 smithi153 ceph-mon[29117]: pgmap v300: 65 pgs: 65 active+clean; 36 MiB data, 271 MiB used, 536 GiB / 536 GiB avail; 736 KiB/s rd, 1.0 MiB/s wr, 64 op/s 2024-03-21T10:07:17.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:17 smithi153 ceph-mon[29117]: pgmap v301: 65 pgs: 65 active+clean; 39 MiB data, 281 MiB used, 536 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 72 op/s 2024-03-21T10:07:17.794 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:17 smithi174 ceph-mon[37496]: pgmap v301: 65 pgs: 65 active+clean; 39 MiB data, 281 MiB used, 536 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 72 op/s 2024-03-21T10:07:19.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:19 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:07:19.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:19 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:07:19.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:19 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:07:19.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:19 smithi174 ceph-mon[37496]: pgmap v302: 65 pgs: 65 active+clean; 41 MiB data, 287 MiB used, 536 GiB / 536 GiB avail; 889 KiB/s rd, 1.1 MiB/s wr, 66 op/s 2024-03-21T10:07:19.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:19 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:07:19.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:19 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:07:19.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:19 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:07:19.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:19 smithi153 ceph-mon[29117]: pgmap v302: 65 pgs: 65 active+clean; 41 MiB data, 287 MiB used, 536 GiB / 536 GiB avail; 889 KiB/s rd, 1.1 MiB/s wr, 66 op/s 2024-03-21T10:07:21.699 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:21 smithi174 ceph-mon[37496]: pgmap v303: 65 pgs: 65 active+clean; 41 MiB data, 291 MiB used, 536 GiB / 536 GiB avail; 889 KiB/s rd, 977 KiB/s wr, 53 op/s 2024-03-21T10:07:21.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:21 smithi153 ceph-mon[29117]: pgmap v303: 65 pgs: 65 active+clean; 41 MiB data, 291 MiB used, 536 GiB / 536 GiB avail; 889 KiB/s rd, 977 KiB/s wr, 53 op/s 2024-03-21T10:07:23.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:23 smithi153 ceph-mon[29117]: pgmap v304: 65 pgs: 65 active+clean; 49 MiB data, 317 MiB used, 536 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 96 op/s 2024-03-21T10:07:23.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:23 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:07:23.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:23 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:07:23.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:23 smithi174 ceph-mon[37496]: pgmap v304: 65 pgs: 65 active+clean; 49 MiB data, 317 MiB used, 536 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 96 op/s 2024-03-21T10:07:23.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:23 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:07:23.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:23 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:07:25.415 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:25 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:07:25.415 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:25 smithi174 ceph-mon[37496]: pgmap v305: 65 pgs: 65 active+clean; 50 MiB data, 328 MiB used, 536 GiB / 536 GiB avail; 966 KiB/s rd, 1.1 MiB/s wr, 73 op/s 2024-03-21T10:07:25.416 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:25 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:07:25.416 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:25 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:07:25.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:25 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:07:25.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:25 smithi153 ceph-mon[29117]: pgmap v305: 65 pgs: 65 active+clean; 50 MiB data, 328 MiB used, 536 GiB / 536 GiB avail; 966 KiB/s rd, 1.1 MiB/s wr, 73 op/s 2024-03-21T10:07:25.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:25 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:07:25.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:25 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:07:27.570 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:27 smithi174 ceph-mon[37496]: pgmap v306: 65 pgs: 65 active+clean; 57 MiB data, 349 MiB used, 536 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 92 op/s 2024-03-21T10:07:27.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:27 smithi153 ceph-mon[29117]: pgmap v306: 65 pgs: 65 active+clean; 57 MiB data, 349 MiB used, 536 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 92 op/s 2024-03-21T10:07:29.660 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:29 smithi174 ceph-mon[37496]: pgmap v307: 65 pgs: 65 active+clean; 62 MiB data, 365 MiB used, 536 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 92 op/s 2024-03-21T10:07:29.660 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:29 smithi174 ceph-mon[37496]: Standby manager daemon smithi174.qpcwat restarted 2024-03-21T10:07:29.660 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:29 smithi174 ceph-mon[37496]: Standby manager daemon smithi174.qpcwat started 2024-03-21T10:07:29.661 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:29 smithi174 ceph-mon[37496]: from='mgr.? 172.21.15.174:0/1831991646' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi174.qpcwat/crt"}]: dispatch 2024-03-21T10:07:29.661 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:29 smithi174 ceph-mon[37496]: from='mgr.? 172.21.15.174:0/1831991646' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-21T10:07:29.661 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:29 smithi174 ceph-mon[37496]: from='mgr.? 172.21.15.174:0/1831991646' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi174.qpcwat/key"}]: dispatch 2024-03-21T10:07:29.661 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:29 smithi174 ceph-mon[37496]: from='mgr.? 172.21.15.174:0/1831991646' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-21T10:07:29.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:29 smithi153 ceph-mon[29117]: pgmap v307: 65 pgs: 65 active+clean; 62 MiB data, 365 MiB used, 536 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 92 op/s 2024-03-21T10:07:29.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:29 smithi153 ceph-mon[29117]: Standby manager daemon smithi174.qpcwat restarted 2024-03-21T10:07:29.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:29 smithi153 ceph-mon[29117]: Standby manager daemon smithi174.qpcwat started 2024-03-21T10:07:29.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:29 smithi153 ceph-mon[29117]: from='mgr.? 172.21.15.174:0/1831991646' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi174.qpcwat/crt"}]: dispatch 2024-03-21T10:07:29.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:29 smithi153 ceph-mon[29117]: from='mgr.? 172.21.15.174:0/1831991646' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-21T10:07:29.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:29 smithi153 ceph-mon[29117]: from='mgr.? 172.21.15.174:0/1831991646' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi174.qpcwat/key"}]: dispatch 2024-03-21T10:07:29.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:29 smithi153 ceph-mon[29117]: from='mgr.? 172.21.15.174:0/1831991646' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-21T10:07:31.120 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:31 smithi153 ceph-mon[29117]: mgrmap e20: smithi153.tluxni(active, since 9m), standbys: smithi174.qpcwat 2024-03-21T10:07:31.120 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:31 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:07:31.120 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:31 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:07:31.121 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:31 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:07:31.121 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:31 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:07:31.121 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:31 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:07:31.121 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:31 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:07:31.121 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:31 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mgr fail", "who": "smithi153.tluxni"}]: dispatch 2024-03-21T10:07:31.121 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:31 smithi153 ceph-mon[29117]: osdmap e48: 6 total, 6 up, 6 in 2024-03-21T10:07:31.122 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:31 smithi153 ceph-mon[29117]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "mgr fail", "who": "smithi153.tluxni"}]': finished 2024-03-21T10:07:31.122 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:31 smithi153 ceph-mon[29117]: mgrmap e21: smithi174.qpcwat(active, starting, since 0.0105372s) 2024-03-21T10:07:31.122 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:31 smithi153 ceph-mon[29117]: from='mgr.24361 172.21.15.174:0/1831991646' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "mon metadata", "id": "smithi153"}]: dispatch 2024-03-21T10:07:31.122 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:31 smithi153 ceph-mon[29117]: from='mgr.24361 172.21.15.174:0/1831991646' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "mon metadata", "id": "smithi174"}]: dispatch 2024-03-21T10:07:31.122 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:31 smithi153 ceph-mon[29117]: from='mgr.24361 172.21.15.174:0/1831991646' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi153.iuyluj"}]: dispatch 2024-03-21T10:07:31.122 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:31 smithi153 ceph-mon[29117]: from='mgr.24361 172.21.15.174:0/1831991646' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi153.zipckr"}]: dispatch 2024-03-21T10:07:31.123 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:31 smithi153 ceph-mon[29117]: from='mgr.24361 172.21.15.174:0/1831991646' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi174.yyimst"}]: dispatch 2024-03-21T10:07:31.123 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:31 smithi153 ceph-mon[29117]: from='mgr.24361 172.21.15.174:0/1831991646' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi174.nhozag"}]: dispatch 2024-03-21T10:07:31.123 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:31 smithi153 ceph-mon[29117]: from='mgr.24361 172.21.15.174:0/1831991646' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "mgr metadata", "who": "smithi174.qpcwat", "id": "smithi174.qpcwat"}]: dispatch 2024-03-21T10:07:31.123 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:31 smithi153 ceph-mon[29117]: from='mgr.24361 172.21.15.174:0/1831991646' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T10:07:31.123 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:31 smithi153 ceph-mon[29117]: from='mgr.24361 172.21.15.174:0/1831991646' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T10:07:31.123 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:31 smithi153 ceph-mon[29117]: from='mgr.24361 172.21.15.174:0/1831991646' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T10:07:31.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:31 smithi174 ceph-mon[37496]: mgrmap e20: smithi153.tluxni(active, since 9m), standbys: smithi174.qpcwat 2024-03-21T10:07:31.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:31 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:07:31.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:31 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:07:31.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:31 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:07:31.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:31 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:07:31.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:31 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' 2024-03-21T10:07:31.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:31 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:07:31.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:31 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mgr fail", "who": "smithi153.tluxni"}]: dispatch 2024-03-21T10:07:31.453 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:31 smithi174 ceph-mon[37496]: osdmap e48: 6 total, 6 up, 6 in 2024-03-21T10:07:31.453 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:31 smithi174 ceph-mon[37496]: from='mgr.14221 172.21.15.153:0/1209125341' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "mgr fail", "who": "smithi153.tluxni"}]': finished 2024-03-21T10:07:31.453 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:31 smithi174 ceph-mon[37496]: mgrmap e21: smithi174.qpcwat(active, starting, since 0.0105372s) 2024-03-21T10:07:31.453 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:31 smithi174 ceph-mon[37496]: from='mgr.24361 172.21.15.174:0/1831991646' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "mon metadata", "id": "smithi153"}]: dispatch 2024-03-21T10:07:31.453 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:31 smithi174 ceph-mon[37496]: from='mgr.24361 172.21.15.174:0/1831991646' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "mon metadata", "id": "smithi174"}]: dispatch 2024-03-21T10:07:31.453 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:31 smithi174 ceph-mon[37496]: from='mgr.24361 172.21.15.174:0/1831991646' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi153.iuyluj"}]: dispatch 2024-03-21T10:07:31.453 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:31 smithi174 ceph-mon[37496]: from='mgr.24361 172.21.15.174:0/1831991646' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi153.zipckr"}]: dispatch 2024-03-21T10:07:31.453 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:31 smithi174 ceph-mon[37496]: from='mgr.24361 172.21.15.174:0/1831991646' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi174.yyimst"}]: dispatch 2024-03-21T10:07:31.453 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:31 smithi174 ceph-mon[37496]: from='mgr.24361 172.21.15.174:0/1831991646' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi174.nhozag"}]: dispatch 2024-03-21T10:07:31.453 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:31 smithi174 ceph-mon[37496]: from='mgr.24361 172.21.15.174:0/1831991646' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "mgr metadata", "who": "smithi174.qpcwat", "id": "smithi174.qpcwat"}]: dispatch 2024-03-21T10:07:31.453 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:31 smithi174 ceph-mon[37496]: from='mgr.24361 172.21.15.174:0/1831991646' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T10:07:31.453 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:31 smithi174 ceph-mon[37496]: from='mgr.24361 172.21.15.174:0/1831991646' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T10:07:31.453 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:31 smithi174 ceph-mon[37496]: from='mgr.24361 172.21.15.174:0/1831991646' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T10:07:31.453 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:31 smithi174 ceph-mon[37496]: from='mgr.24361 172.21.15.174:0/1831991646' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T10:07:31.454 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:31 smithi174 ceph-mon[37496]: from='mgr.24361 172.21.15.174:0/1831991646' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T10:07:31.454 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:31 smithi174 ceph-mon[37496]: from='mgr.24361 172.21.15.174:0/1831991646' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T10:07:31.454 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:31 smithi174 ceph-mon[37496]: from='mgr.24361 172.21.15.174:0/1831991646' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-21T10:07:31.454 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:31 smithi174 ceph-mon[37496]: from='mgr.24361 172.21.15.174:0/1831991646' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-21T10:07:31.454 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:31 smithi174 ceph-mon[37496]: from='mgr.24361 172.21.15.174:0/1831991646' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-21T10:07:31.454 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:31 smithi174 ceph-mon[37496]: Manager daemon smithi174.qpcwat is now available 2024-03-21T10:07:31.454 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:31 smithi174 ceph-mon[37496]: from='mgr.24361 172.21.15.174:0/1831991646' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:07:31.454 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:31 smithi174 ceph-mon[37496]: from='mgr.24361 172.21.15.174:0/1831991646' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:07:31.454 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:31 smithi174 ceph-mon[37496]: from='mgr.24361 172.21.15.174:0/1831991646' entity='mgr.smithi174.qpcwat' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi174.qpcwat/mirror_snapshot_schedule"}]: dispatch 2024-03-21T10:07:31.454 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:31 smithi174 ceph-mon[37496]: from='mgr.24361 ' entity='mgr.smithi174.qpcwat' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi174.qpcwat/mirror_snapshot_schedule"}]: dispatch 2024-03-21T10:07:31.454 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:31 smithi174 ceph-mon[37496]: from='mgr.24361 172.21.15.174:0/1831991646' entity='mgr.smithi174.qpcwat' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi174.qpcwat/trash_purge_schedule"}]: dispatch 2024-03-21T10:07:31.454 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:31 smithi174 ceph-mon[37496]: from='mgr.24361 ' entity='mgr.smithi174.qpcwat' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi174.qpcwat/trash_purge_schedule"}]: dispatch 2024-03-21T10:07:31.558 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:31 smithi153 ceph-mon[29117]: from='mgr.24361 172.21.15.174:0/1831991646' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T10:07:31.558 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:31 smithi153 ceph-mon[29117]: from='mgr.24361 172.21.15.174:0/1831991646' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T10:07:31.559 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:31 smithi153 ceph-mon[29117]: from='mgr.24361 172.21.15.174:0/1831991646' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T10:07:31.559 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:31 smithi153 ceph-mon[29117]: from='mgr.24361 172.21.15.174:0/1831991646' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-21T10:07:31.559 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:31 smithi153 ceph-mon[29117]: from='mgr.24361 172.21.15.174:0/1831991646' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-21T10:07:31.559 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:31 smithi153 ceph-mon[29117]: from='mgr.24361 172.21.15.174:0/1831991646' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-21T10:07:31.559 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:31 smithi153 ceph-mon[29117]: Manager daemon smithi174.qpcwat is now available 2024-03-21T10:07:31.559 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:31 smithi153 ceph-mon[29117]: from='mgr.24361 172.21.15.174:0/1831991646' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:07:31.559 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:31 smithi153 ceph-mon[29117]: from='mgr.24361 172.21.15.174:0/1831991646' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:07:31.559 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:31 smithi153 ceph-mon[29117]: from='mgr.24361 172.21.15.174:0/1831991646' entity='mgr.smithi174.qpcwat' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi174.qpcwat/mirror_snapshot_schedule"}]: dispatch 2024-03-21T10:07:31.559 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:31 smithi153 ceph-mon[29117]: from='mgr.24361 ' entity='mgr.smithi174.qpcwat' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi174.qpcwat/mirror_snapshot_schedule"}]: dispatch 2024-03-21T10:07:31.559 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:31 smithi153 ceph-mon[29117]: from='mgr.24361 172.21.15.174:0/1831991646' entity='mgr.smithi174.qpcwat' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi174.qpcwat/trash_purge_schedule"}]: dispatch 2024-03-21T10:07:31.559 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:31 smithi153 ceph-mon[29117]: from='mgr.24361 ' entity='mgr.smithi174.qpcwat' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi174.qpcwat/trash_purge_schedule"}]: dispatch 2024-03-21T10:07:32.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:32 smithi153 ceph-mon[29117]: Deploying cephadm binary to smithi174 2024-03-21T10:07:32.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:32 smithi153 ceph-mon[29117]: Deploying cephadm binary to smithi153 2024-03-21T10:07:32.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:32 smithi153 ceph-mon[29117]: mgrmap e22: smithi174.qpcwat(active, since 1.01722s) 2024-03-21T10:07:32.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:32 smithi153 ceph-mon[29117]: pgmap v3: 65 pgs: 65 active+clean; 80 MiB data, 428 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:07:32.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:32 smithi174 ceph-mon[37496]: Deploying cephadm binary to smithi174 2024-03-21T10:07:32.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:32 smithi174 ceph-mon[37496]: Deploying cephadm binary to smithi153 2024-03-21T10:07:32.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:32 smithi174 ceph-mon[37496]: mgrmap e22: smithi174.qpcwat(active, since 1.01722s) 2024-03-21T10:07:32.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:32 smithi174 ceph-mon[37496]: pgmap v3: 65 pgs: 65 active+clean; 80 MiB data, 428 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:07:33.363 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:33.359+0000 7f275e400700 1 -- 172.21.15.153:0/314563343 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2758102e00 msgr2=0x7f2758103220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:07:33.363 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:33.359+0000 7f275e400700 1 --2- 172.21.15.153:0/314563343 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2758102e00 0x7f2758103220 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f2744009910 tx=0x7f2744009c20 comp rx=0 tx=0).stop 2024-03-21T10:07:33.363 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:33.359+0000 7f275e400700 1 -- 172.21.15.153:0/314563343 shutdown_connections 2024-03-21T10:07:33.364 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:33.359+0000 7f275e400700 1 --2- 172.21.15.153:0/314563343 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2758103f60 0x7f27581043e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:07:33.364 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:33.359+0000 7f275e400700 1 --2- 172.21.15.153:0/314563343 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2758102e00 0x7f2758103220 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:07:33.364 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:33.359+0000 7f275e400700 1 -- 172.21.15.153:0/314563343 >> 172.21.15.153:0/314563343 conn(0x7f27580fe3a0 msgr2=0x7f27581007e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:07:33.364 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:33.359+0000 7f275e400700 1 -- 172.21.15.153:0/314563343 shutdown_connections 2024-03-21T10:07:33.364 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:33.359+0000 7f275e400700 1 -- 172.21.15.153:0/314563343 wait complete. 2024-03-21T10:07:33.364 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:33.360+0000 7f275e400700 1 Processor -- start 2024-03-21T10:07:33.364 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:33.361+0000 7f275e400700 1 -- start start 2024-03-21T10:07:33.364 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:33.361+0000 7f275e400700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2758102e00 0x7f2758198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:07:33.364 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:33.361+0000 7f275e400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2758103f60 0x7f2758198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:07:33.364 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:33.361+0000 7f275e400700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27581990c0 con 0x7f2758103f60 2024-03-21T10:07:33.364 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:33.361+0000 7f275e400700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2758199200 con 0x7f2758102e00 2024-03-21T10:07:33.364 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:33.361+0000 7f2756a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2758103f60 0x7f2758198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:07:33.364 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:33.362+0000 7f2756a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2758103f60 0x7f2758198aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:53708/0 (socket says 172.21.15.153:53708) 2024-03-21T10:07:33.365 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:33.362+0000 7f2756a00700 1 -- 172.21.15.153:0/188884364 learned_addr learned my addr 172.21.15.153:0/188884364 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:07:33.365 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:33.362+0000 7f2757400700 1 --2- 172.21.15.153:0/188884364 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2758102e00 0x7f2758198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:07:33.365 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:33.362+0000 7f2756a00700 1 -- 172.21.15.153:0/188884364 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2758102e00 msgr2=0x7f2758198560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:07:33.365 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:33.362+0000 7f2756a00700 1 --2- 172.21.15.153:0/188884364 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2758102e00 0x7f2758198560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:07:33.366 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:33.362+0000 7f2756a00700 1 -- 172.21.15.153:0/188884364 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27440095f0 con 0x7f2758103f60 2024-03-21T10:07:33.366 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:33.363+0000 7f2756a00700 1 --2- 172.21.15.153:0/188884364 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2758103f60 0x7f2758198aa0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f274800d8d0 tx=0x7f274800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:07:33.366 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:33.363+0000 7f274fe00700 1 -- 172.21.15.153:0/188884364 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2748009940 con 0x7f2758103f60 2024-03-21T10:07:33.366 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:33.363+0000 7f274fe00700 1 -- 172.21.15.153:0/188884364 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f2748010460 con 0x7f2758103f60 2024-03-21T10:07:33.366 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:33.363+0000 7f275e400700 1 -- 172.21.15.153:0/188884364 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f275819dcb0 con 0x7f2758103f60 2024-03-21T10:07:33.366 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:33.364+0000 7f274fe00700 1 -- 172.21.15.153:0/188884364 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f274800f5f0 con 0x7f2758103f60 2024-03-21T10:07:33.366 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:33.364+0000 7f275e400700 1 -- 172.21.15.153:0/188884364 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f275819e200 con 0x7f2758103f60 2024-03-21T10:07:33.369 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:33.366+0000 7f275e400700 1 -- 172.21.15.153:0/188884364 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2738005320 con 0x7f2758103f60 2024-03-21T10:07:33.369 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:33.367+0000 7f274fe00700 1 -- 172.21.15.153:0/188884364 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 22) v1 ==== 46379+0+0 (secure 0 0 0) 0x7f2748009aa0 con 0x7f2758103f60 2024-03-21T10:07:33.369 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:33.367+0000 7f274fe00700 1 --2- 172.21.15.153:0/188884364 >> [v2:172.21.15.174:6828/1155541738,v1:172.21.15.174:6829/1155541738] conn(0x7f273c039410 0x7f273c03b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:07:33.370 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:33.368+0000 7f2757400700 1 --2- 172.21.15.153:0/188884364 >> [v2:172.21.15.174:6828/1155541738,v1:172.21.15.174:6829/1155541738] conn(0x7f273c039410 0x7f273c03b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:07:33.371 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:33.370+0000 7f274fe00700 1 -- 172.21.15.153:0/188884364 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f274800fe90 con 0x7f2758103f60 2024-03-21T10:07:33.371 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:33.370+0000 7f2757400700 1 --2- 172.21.15.153:0/188884364 >> [v2:172.21.15.174:6828/1155541738,v1:172.21.15.174:6829/1155541738] conn(0x7f273c039410 0x7f273c03b8d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2744006010 tx=0x7f274400b540 comp rx=0 tx=0).ready entity=mgr.24361 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:07:33.379 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:33.377+0000 7f274fe00700 1 -- 172.21.15.153:0/188884364 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185454 (secure 0 0 0) 0x7f2748005030 con 0x7f2758103f60 2024-03-21T10:07:33.558 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:33.557+0000 7f274fe00700 1 -- 172.21.15.153:0/188884364 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mgrmap(e 23) v1 ==== 46435+0+0 (secure 0 0 0) 0x7f2748018db0 con 0x7f2758103f60 2024-03-21T10:07:33.567 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:33 smithi153 ceph-mon[29117]: [21/Mar/2024:10:07:32] ENGINE Bus STARTING 2024-03-21T10:07:33.567 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:33 smithi153 ceph-mon[29117]: pgmap v4: 65 pgs: 65 active+clean; 80 MiB data, 428 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:07:33.568 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:33 smithi153 ceph-mon[29117]: [21/Mar/2024:10:07:32] ENGINE Serving on http://172.21.15.174:8765 2024-03-21T10:07:33.568 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:33 smithi153 ceph-mon[29117]: [21/Mar/2024:10:07:32] ENGINE Serving on https://172.21.15.174:7150 2024-03-21T10:07:33.568 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:33 smithi153 ceph-mon[29117]: [21/Mar/2024:10:07:32] ENGINE Bus STARTED 2024-03-21T10:07:33.742 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:33.740+0000 7f275e400700 1 -- 172.21.15.153:0/188884364 --> [v2:172.21.15.174:6828/1155541738,v1:172.21.15.174:6829/1155541738] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2738000bf0 con 0x7f273c039410 2024-03-21T10:07:33.755 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:33.753+0000 7f274fe00700 1 -- 172.21.15.153:0/188884364 <== mgr.24361 v2:172.21.15.174:6828/1155541738 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f2738000bf0 con 0x7f273c039410 2024-03-21T10:07:33.759 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:33.757+0000 7f275e400700 1 -- 172.21.15.153:0/188884364 >> [v2:172.21.15.174:6828/1155541738,v1:172.21.15.174:6829/1155541738] conn(0x7f273c039410 msgr2=0x7f273c03b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:07:33.759 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:33.757+0000 7f275e400700 1 --2- 172.21.15.153:0/188884364 >> [v2:172.21.15.174:6828/1155541738,v1:172.21.15.174:6829/1155541738] conn(0x7f273c039410 0x7f273c03b8d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2744006010 tx=0x7f274400b540 comp rx=0 tx=0).stop 2024-03-21T10:07:33.759 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:33.757+0000 7f275e400700 1 -- 172.21.15.153:0/188884364 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2758103f60 msgr2=0x7f2758198aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:07:33.759 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:33.757+0000 7f275e400700 1 --2- 172.21.15.153:0/188884364 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2758103f60 0x7f2758198aa0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f274800d8d0 tx=0x7f274800dc90 comp rx=0 tx=0).stop 2024-03-21T10:07:33.759 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:33.757+0000 7f275e400700 1 -- 172.21.15.153:0/188884364 shutdown_connections 2024-03-21T10:07:33.760 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:33.757+0000 7f275e400700 1 --2- 172.21.15.153:0/188884364 >> [v2:172.21.15.174:6828/1155541738,v1:172.21.15.174:6829/1155541738] conn(0x7f273c039410 0x7f273c03b8d0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:07:33.760 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:33.757+0000 7f275e400700 1 --2- 172.21.15.153:0/188884364 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2758102e00 0x7f2758198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:07:33.760 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:33.757+0000 7f275e400700 1 --2- 172.21.15.153:0/188884364 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2758103f60 0x7f2758198aa0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:07:33.760 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:33.757+0000 7f275e400700 1 -- 172.21.15.153:0/188884364 >> 172.21.15.153:0/188884364 conn(0x7f27580fe3a0 msgr2=0x7f2758107220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:07:33.760 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:33.757+0000 7f275e400700 1 -- 172.21.15.153:0/188884364 shutdown_connections 2024-03-21T10:07:33.760 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:33.757+0000 7f275e400700 1 -- 172.21.15.153:0/188884364 wait complete. 2024-03-21T10:07:33.783 INFO:teuthology.orchestra.run.smithi153.stdout:true 2024-03-21T10:07:33.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:33 smithi174 ceph-mon[37496]: [21/Mar/2024:10:07:32] ENGINE Bus STARTING 2024-03-21T10:07:33.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:33 smithi174 ceph-mon[37496]: pgmap v4: 65 pgs: 65 active+clean; 80 MiB data, 428 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:07:33.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:33 smithi174 ceph-mon[37496]: [21/Mar/2024:10:07:32] ENGINE Serving on http://172.21.15.174:8765 2024-03-21T10:07:33.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:33 smithi174 ceph-mon[37496]: [21/Mar/2024:10:07:32] ENGINE Serving on https://172.21.15.174:7150 2024-03-21T10:07:33.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:33 smithi174 ceph-mon[37496]: [21/Mar/2024:10:07:32] ENGINE Bus STARTED 2024-03-21T10:07:33.985 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:33.984+0000 7f8d8c000700 1 -- 172.21.15.153:0/2233582308 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d84103ff0 msgr2=0x7f8d84104450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:07:33.986 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:33.984+0000 7f8d8c000700 1 --2- 172.21.15.153:0/2233582308 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d84103ff0 0x7f8d84104450 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f8d70009910 tx=0x7f8d70009c20 comp rx=0 tx=0).stop 2024-03-21T10:07:33.986 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:33.984+0000 7f8d8c000700 1 -- 172.21.15.153:0/2233582308 shutdown_connections 2024-03-21T10:07:33.986 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:33.984+0000 7f8d8c000700 1 --2- 172.21.15.153:0/2233582308 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d84103ff0 0x7f8d84104450 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:07:33.987 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:33.984+0000 7f8d8c000700 1 --2- 172.21.15.153:0/2233582308 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8d84102df0 0x7f8d84103210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:07:33.987 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:33.984+0000 7f8d8c000700 1 -- 172.21.15.153:0/2233582308 >> 172.21.15.153:0/2233582308 conn(0x7f8d840fe390 msgr2=0x7f8d841007d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:07:33.987 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:33.985+0000 7f8d8c000700 1 -- 172.21.15.153:0/2233582308 shutdown_connections 2024-03-21T10:07:33.987 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:33.985+0000 7f8d8c000700 1 -- 172.21.15.153:0/2233582308 wait complete. 2024-03-21T10:07:33.987 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:33.986+0000 7f8d8c000700 1 Processor -- start 2024-03-21T10:07:33.988 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:33.986+0000 7f8d8c000700 1 -- start start 2024-03-21T10:07:33.988 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:33.986+0000 7f8d8c000700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8d84102df0 0x7f8d84198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:07:33.988 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:33.986+0000 7f8d8c000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d84103ff0 0x7f8d84198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:07:33.988 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:33.987+0000 7f8d8c000700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d841990f0 con 0x7f8d84103ff0 2024-03-21T10:07:33.988 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:33.987+0000 7f8d8c000700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d84199230 con 0x7f8d84102df0 2024-03-21T10:07:33.989 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:33.987+0000 7f8d89600700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8d84102df0 0x7f8d84198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:07:33.989 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:33.987+0000 7f8d88c00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d84103ff0 0x7f8d84198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:07:33.989 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:33.987+0000 7f8d89600700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8d84102df0 0x7f8d84198590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.153:50946/0 (socket says 172.21.15.153:50946) 2024-03-21T10:07:33.989 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:33.987+0000 7f8d89600700 1 -- 172.21.15.153:0/2507305633 learned_addr learned my addr 172.21.15.153:0/2507305633 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:07:33.990 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:33.988+0000 7f8d89600700 1 -- 172.21.15.153:0/2507305633 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d84103ff0 msgr2=0x7f8d84198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:07:33.990 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:33.988+0000 7f8d89600700 1 --2- 172.21.15.153:0/2507305633 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d84103ff0 0x7f8d84198ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:07:33.990 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:33.988+0000 7f8d89600700 1 -- 172.21.15.153:0/2507305633 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d700095f0 con 0x7f8d84102df0 2024-03-21T10:07:33.990 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:33.989+0000 7f8d88c00700 1 --2- 172.21.15.153:0/2507305633 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d84103ff0 0x7f8d84198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:07:33.991 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:33.989+0000 7f8d89600700 1 --2- 172.21.15.153:0/2507305633 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8d84102df0 0x7f8d84198590 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8d7800e8d0 tx=0x7f8d7800ec90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:07:33.991 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:33.990+0000 7f8d82000700 1 -- 172.21.15.153:0/2507305633 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d78009770 con 0x7f8d84102df0 2024-03-21T10:07:33.991 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:33.990+0000 7f8d8c000700 1 -- 172.21.15.153:0/2507305633 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d8419dce0 con 0x7f8d84102df0 2024-03-21T10:07:33.991 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:33.990+0000 7f8d8c000700 1 -- 172.21.15.153:0/2507305633 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d8419e1d0 con 0x7f8d84102df0 2024-03-21T10:07:33.992 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:33.991+0000 7f8d82000700 1 -- 172.21.15.153:0/2507305633 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f8d7800eed0 con 0x7f8d84102df0 2024-03-21T10:07:33.992 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:33.991+0000 7f8d82000700 1 -- 172.21.15.153:0/2507305633 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d78003e40 con 0x7f8d84102df0 2024-03-21T10:07:33.994 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:33.992+0000 7f8d8c000700 1 -- 172.21.15.153:0/2507305633 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d8410b9e0 con 0x7f8d84102df0 2024-03-21T10:07:33.996 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:33.993+0000 7f8d82000700 1 -- 172.21.15.153:0/2507305633 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 23) v1 ==== 46435+0+0 (secure 0 0 0) 0x7f8d780104c0 con 0x7f8d84102df0 2024-03-21T10:07:33.996 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:33.993+0000 7f8d82000700 1 --2- 172.21.15.153:0/2507305633 >> [v2:172.21.15.174:6828/1155541738,v1:172.21.15.174:6829/1155541738] conn(0x7f8d6c039490 0x7f8d6c03b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:07:33.996 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:33.993+0000 7f8d82000700 1 -- 172.21.15.153:0/2507305633 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f8d78014070 con 0x7f8d84102df0 2024-03-21T10:07:33.996 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:33.994+0000 7f8d88c00700 1 --2- 172.21.15.153:0/2507305633 >> [v2:172.21.15.174:6828/1155541738,v1:172.21.15.174:6829/1155541738] conn(0x7f8d6c039490 0x7f8d6c03b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:07:33.996 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:33.995+0000 7f8d88c00700 1 --2- 172.21.15.153:0/2507305633 >> [v2:172.21.15.174:6828/1155541738,v1:172.21.15.174:6829/1155541738] conn(0x7f8d6c039490 0x7f8d6c03b950 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8d70009fd0 tx=0x7f8d70005a90 comp rx=0 tx=0).ready entity=mgr.24361 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:07:34.005 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:34.003+0000 7f8d82000700 1 -- 172.21.15.153:0/2507305633 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185454 (secure 0 0 0) 0x7f8d7801f340 con 0x7f8d84102df0 2024-03-21T10:07:34.343 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:34.341+0000 7f8d8c000700 1 -- 172.21.15.153:0/2507305633 --> [v2:172.21.15.174:6828/1155541738,v1:172.21.15.174:6829/1155541738] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8d84108940 con 0x7f8d6c039490 2024-03-21T10:07:34.352 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:34.350+0000 7f8d82000700 1 -- 172.21.15.153:0/2507305633 <== mgr.24361 v2:172.21.15.174:6828/1155541738 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f8d84108940 con 0x7f8d6c039490 2024-03-21T10:07:34.354 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:34.353+0000 7f8d8c000700 1 -- 172.21.15.153:0/2507305633 >> [v2:172.21.15.174:6828/1155541738,v1:172.21.15.174:6829/1155541738] conn(0x7f8d6c039490 msgr2=0x7f8d6c03b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:07:34.355 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:34.353+0000 7f8d8c000700 1 --2- 172.21.15.153:0/2507305633 >> [v2:172.21.15.174:6828/1155541738,v1:172.21.15.174:6829/1155541738] conn(0x7f8d6c039490 0x7f8d6c03b950 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8d70009fd0 tx=0x7f8d70005a90 comp rx=0 tx=0).stop 2024-03-21T10:07:34.355 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:34.353+0000 7f8d8c000700 1 -- 172.21.15.153:0/2507305633 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8d84102df0 msgr2=0x7f8d84198590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:07:34.355 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:34.353+0000 7f8d8c000700 1 --2- 172.21.15.153:0/2507305633 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8d84102df0 0x7f8d84198590 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8d7800e8d0 tx=0x7f8d7800ec90 comp rx=0 tx=0).stop 2024-03-21T10:07:34.356 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:34.354+0000 7f8d8c000700 1 -- 172.21.15.153:0/2507305633 shutdown_connections 2024-03-21T10:07:34.356 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:34.354+0000 7f8d8c000700 1 --2- 172.21.15.153:0/2507305633 >> [v2:172.21.15.174:6828/1155541738,v1:172.21.15.174:6829/1155541738] conn(0x7f8d6c039490 0x7f8d6c03b950 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:07:34.356 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:34.354+0000 7f8d8c000700 1 --2- 172.21.15.153:0/2507305633 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8d84102df0 0x7f8d84198590 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:07:34.356 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:34.354+0000 7f8d8c000700 1 --2- 172.21.15.153:0/2507305633 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d84103ff0 0x7f8d84198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:07:34.357 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:34.354+0000 7f8d8c000700 1 -- 172.21.15.153:0/2507305633 >> 172.21.15.153:0/2507305633 conn(0x7f8d840fe390 msgr2=0x7f8d84107220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:07:34.357 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:34.354+0000 7f8d8c000700 1 -- 172.21.15.153:0/2507305633 shutdown_connections 2024-03-21T10:07:34.357 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:34.354+0000 7f8d8c000700 1 -- 172.21.15.153:0/2507305633 wait complete. 2024-03-21T10:07:34.531 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:34.529+0000 7f2363200700 1 -- 172.21.15.153:0/878314144 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f235c0ffab0 msgr2=0x7f235c0ffed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:07:34.531 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:34.529+0000 7f2363200700 1 --2- 172.21.15.153:0/878314144 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f235c0ffab0 0x7f235c0ffed0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f234c009910 tx=0x7f234c009c20 comp rx=0 tx=0).stop 2024-03-21T10:07:34.531 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:34.529+0000 7f2363200700 1 -- 172.21.15.153:0/878314144 shutdown_connections 2024-03-21T10:07:34.532 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:34.529+0000 7f2363200700 1 --2- 172.21.15.153:0/878314144 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f235c100cb0 0x7f235c101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:07:34.532 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:34.529+0000 7f2363200700 1 --2- 172.21.15.153:0/878314144 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f235c0ffab0 0x7f235c0ffed0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:07:34.532 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:34.529+0000 7f2363200700 1 -- 172.21.15.153:0/878314144 >> 172.21.15.153:0/878314144 conn(0x7f235c0fb030 msgr2=0x7f235c0fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:07:34.532 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:34.530+0000 7f2363200700 1 -- 172.21.15.153:0/878314144 shutdown_connections 2024-03-21T10:07:34.532 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:34.530+0000 7f2363200700 1 -- 172.21.15.153:0/878314144 wait complete. 2024-03-21T10:07:34.532 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:34.531+0000 7f2363200700 1 Processor -- start 2024-03-21T10:07:34.533 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:34.531+0000 7f2363200700 1 -- start start 2024-03-21T10:07:34.533 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:34.532+0000 7f2363200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f235c0ffab0 0x7f235c198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:07:34.533 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:34.532+0000 7f2363200700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f235c100cb0 0x7f235c198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:07:34.533 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:34.532+0000 7f2363200700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f235c199080 con 0x7f235c0ffab0 2024-03-21T10:07:34.534 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:34.532+0000 7f2363200700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f235c1991c0 con 0x7f235c100cb0 2024-03-21T10:07:34.534 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:34.532+0000 7f235be00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f235c0ffab0 0x7f235c198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:07:34.534 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:34.532+0000 7f235be00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f235c0ffab0 0x7f235c198520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:53762/0 (socket says 172.21.15.153:53762) 2024-03-21T10:07:34.534 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:34.532+0000 7f235be00700 1 -- 172.21.15.153:0/2024444913 learned_addr learned my addr 172.21.15.153:0/2024444913 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:07:34.534 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:34.532+0000 7f235b400700 1 --2- 172.21.15.153:0/2024444913 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f235c100cb0 0x7f235c198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:07:34.535 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:34.533+0000 7f235be00700 1 -- 172.21.15.153:0/2024444913 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f235c100cb0 msgr2=0x7f235c198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:07:34.535 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:34.533+0000 7f235be00700 1 --2- 172.21.15.153:0/2024444913 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f235c100cb0 0x7f235c198a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:07:34.535 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:34.533+0000 7f235be00700 1 -- 172.21.15.153:0/2024444913 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f234c0095f0 con 0x7f235c0ffab0 2024-03-21T10:07:34.535 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:34.533+0000 7f235be00700 1 --2- 172.21.15.153:0/2024444913 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f235c0ffab0 0x7f235c198520 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f234c00bd70 tx=0x7f234c00bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:07:34.535 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:34.534+0000 7f2358c00700 1 -- 172.21.15.153:0/2024444913 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f234c01d070 con 0x7f235c0ffab0 2024-03-21T10:07:34.535 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:34.534+0000 7f2358c00700 1 -- 172.21.15.153:0/2024444913 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f234c005a00 con 0x7f235c0ffab0 2024-03-21T10:07:34.536 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:34.534+0000 7f2358c00700 1 -- 172.21.15.153:0/2024444913 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f234c021410 con 0x7f235c0ffab0 2024-03-21T10:07:34.536 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:34.534+0000 7f2363200700 1 -- 172.21.15.153:0/2024444913 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f235c19dc10 con 0x7f235c0ffab0 2024-03-21T10:07:34.536 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:34.534+0000 7f2363200700 1 -- 172.21.15.153:0/2024444913 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f235c19e100 con 0x7f235c0ffab0 2024-03-21T10:07:34.536 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:34.535+0000 7f2363200700 1 -- 172.21.15.153:0/2024444913 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f235c108610 con 0x7f235c0ffab0 2024-03-21T10:07:34.537 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:34.536+0000 7f2358c00700 1 -- 172.21.15.153:0/2024444913 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 23) v1 ==== 46435+0+0 (secure 0 0 0) 0x7f234c00f860 con 0x7f235c0ffab0 2024-03-21T10:07:34.538 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:34.537+0000 7f2358c00700 1 --2- 172.21.15.153:0/2024444913 >> [v2:172.21.15.174:6828/1155541738,v1:172.21.15.174:6829/1155541738] conn(0x7f2340039490 0x7f234003b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:07:34.538 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:34.537+0000 7f2358c00700 1 -- 172.21.15.153:0/2024444913 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f234c04ec60 con 0x7f235c0ffab0 2024-03-21T10:07:34.539 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:34.537+0000 7f235b400700 1 --2- 172.21.15.153:0/2024444913 >> [v2:172.21.15.174:6828/1155541738,v1:172.21.15.174:6829/1155541738] conn(0x7f2340039490 0x7f234003b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:07:34.540 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:34.538+0000 7f235b400700 1 --2- 172.21.15.153:0/2024444913 >> [v2:172.21.15.174:6828/1155541738,v1:172.21.15.174:6829/1155541738] conn(0x7f2340039490 0x7f234003b950 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2348005950 tx=0x7f234800a300 comp rx=0 tx=0).ready entity=mgr.24361 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:07:34.544 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:34.543+0000 7f2358c00700 1 -- 172.21.15.153:0/2024444913 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185454 (secure 0 0 0) 0x7f234c029910 con 0x7f235c0ffab0 2024-03-21T10:07:34.861 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:34.859+0000 7f2363200700 1 -- 172.21.15.153:0/2024444913 --> [v2:172.21.15.174:6828/1155541738,v1:172.21.15.174:6829/1155541738] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f235c068ae0 con 0x7f2340039490 2024-03-21T10:07:34.861 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:34 smithi153 ceph-mon[29117]: mgrmap e23: smithi174.qpcwat(active, since 3s) 2024-03-21T10:07:34.861 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:34 smithi153 ceph-mon[29117]: from='client.14626 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:07:34.878 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:34.876+0000 7f2358c00700 1 -- 172.21.15.153:0/2024444913 <== mgr.24361 v2:172.21.15.174:6828/1155541738 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f235c068ae0 con 0x7f2340039490 2024-03-21T10:07:34.878 INFO:teuthology.orchestra.run.smithi153.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-21T10:07:34.878 INFO:teuthology.orchestra.run.smithi153.stdout:alertmanager.smithi153 smithi153 *:9093,9094 running (8m) 2m ago 10m 13.0M - 0.25.0 c8568f914cd2 2cfcc225c7e6 2024-03-21T10:07:34.878 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi153 smithi153 running (10m) 2m ago 10m 7587k - 18.2.1 5be31c24972a 5b61994396a6 2024-03-21T10:07:34.878 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi174 smithi174 running (8m) 12s ago 8m 7680k - 18.2.1 5be31c24972a 4f7a5fc3ac28 2024-03-21T10:07:34.878 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi153 smithi153 running (10m) 2m ago 10m 6665k - 18.2.1 5be31c24972a 90a8b8ea03d1 2024-03-21T10:07:34.878 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi174 smithi174 running (8m) 12s ago 8m 6665k - 18.2.1 5be31c24972a a4c929e4dc77 2024-03-21T10:07:34.878 INFO:teuthology.orchestra.run.smithi153.stdout:grafana.smithi153 smithi153 *:3000 running (7m) 2m ago 9m 73.6M - 9.4.7 954c08fa6188 3af5800ecfef 2024-03-21T10:07:34.878 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.iuyluj smithi153 running (2m) 2m ago 2m 15.0M - 18.2.1 5be31c24972a 02f667cee979 2024-03-21T10:07:34.879 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.zipckr smithi153 running (3m) 2m ago 3m 12.9M - 18.2.1 5be31c24972a 68a853929f8f 2024-03-21T10:07:34.879 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.nhozag smithi174 running (2m) 12s ago 2m 30.8M - 18.2.1 5be31c24972a 2fee4000308f 2024-03-21T10:07:34.879 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.yyimst smithi174 running (2m) 12s ago 2m 46.2M - 18.2.1 5be31c24972a d78a729220d1 2024-03-21T10:07:34.879 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi153.tluxni smithi153 *:9283,8765,8443 running (11m) 2m ago 11m 491M - 18.2.1 5be31c24972a 4978543154ff 2024-03-21T10:07:34.879 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi174.qpcwat smithi174 *:8443,9283,8765 running (17s) 12s ago 8m 55.8M - 19.0.0-2231-g2edfee68 237e43c7a65f 2f7934875e11 2024-03-21T10:07:34.879 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi153 smithi153 running (12m) 2m ago 12m 43.7M 2048M 18.2.1 5be31c24972a 813963a95d69 2024-03-21T10:07:34.879 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi174 smithi174 running (8m) 12s ago 8m 35.0M 2048M 18.2.1 5be31c24972a 99204923eec6 2024-03-21T10:07:34.879 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi153 smithi153 *:9100 running (10m) 2m ago 10m 7608k - 1.5.0 0da6a335fe13 babbc29a0eaf 2024-03-21T10:07:34.879 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi174 smithi174 *:9100 running (8m) 12s ago 8m 7868k - 1.5.0 0da6a335fe13 86841435042a 2024-03-21T10:07:34.879 INFO:teuthology.orchestra.run.smithi153.stdout:osd.0 smithi153 running (7m) 2m ago 7m 46.3M 3947M 18.2.1 5be31c24972a 28912a2319b4 2024-03-21T10:07:34.879 INFO:teuthology.orchestra.run.smithi153.stdout:osd.1 smithi153 running (7m) 2m ago 6m 47.2M 3947M 18.2.1 5be31c24972a ca60681e2314 2024-03-21T10:07:34.879 INFO:teuthology.orchestra.run.smithi153.stdout:osd.2 smithi153 running (6m) 2m ago 6m 45.4M 3947M 18.2.1 5be31c24972a 8db799f8962a 2024-03-21T10:07:34.879 INFO:teuthology.orchestra.run.smithi153.stdout:osd.3 smithi174 running (5m) 12s ago 5m 57.6M 4971M 18.2.1 5be31c24972a dafc89e0658d 2024-03-21T10:07:34.879 INFO:teuthology.orchestra.run.smithi153.stdout:osd.4 smithi174 running (5m) 12s ago 5m 55.9M 4971M 18.2.1 5be31c24972a c4cd15701f05 2024-03-21T10:07:34.880 INFO:teuthology.orchestra.run.smithi153.stdout:osd.5 smithi174 running (4m) 12s ago 4m 57.0M 4971M 18.2.1 5be31c24972a 3b69886ce22e 2024-03-21T10:07:34.880 INFO:teuthology.orchestra.run.smithi153.stdout:prometheus.smithi153 smithi153 *:9095 running (7m) 2m ago 9m 53.9M - 2.43.0 a07b618ecd1d c30cdeb81cc9 2024-03-21T10:07:34.881 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:34.879+0000 7f2363200700 1 -- 172.21.15.153:0/2024444913 >> [v2:172.21.15.174:6828/1155541738,v1:172.21.15.174:6829/1155541738] conn(0x7f2340039490 msgr2=0x7f234003b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:07:34.881 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:34.879+0000 7f2363200700 1 --2- 172.21.15.153:0/2024444913 >> [v2:172.21.15.174:6828/1155541738,v1:172.21.15.174:6829/1155541738] conn(0x7f2340039490 0x7f234003b950 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2348005950 tx=0x7f234800a300 comp rx=0 tx=0).stop 2024-03-21T10:07:34.881 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:34.879+0000 7f2363200700 1 -- 172.21.15.153:0/2024444913 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f235c0ffab0 msgr2=0x7f235c198520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:07:34.881 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:34.879+0000 7f2363200700 1 --2- 172.21.15.153:0/2024444913 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f235c0ffab0 0x7f235c198520 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f234c00bd70 tx=0x7f234c00bda0 comp rx=0 tx=0).stop 2024-03-21T10:07:34.881 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:34.879+0000 7f2363200700 1 -- 172.21.15.153:0/2024444913 shutdown_connections 2024-03-21T10:07:34.881 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:34.879+0000 7f2363200700 1 --2- 172.21.15.153:0/2024444913 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f235c0ffab0 0x7f235c198520 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:07:34.881 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:34.879+0000 7f2363200700 1 --2- 172.21.15.153:0/2024444913 >> [v2:172.21.15.174:6828/1155541738,v1:172.21.15.174:6829/1155541738] conn(0x7f2340039490 0x7f234003b950 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:07:34.881 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:34.879+0000 7f2363200700 1 --2- 172.21.15.153:0/2024444913 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f235c100cb0 0x7f235c198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:07:34.881 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:34.879+0000 7f2363200700 1 -- 172.21.15.153:0/2024444913 >> 172.21.15.153:0/2024444913 conn(0x7f235c0fb030 msgr2=0x7f235c0683d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:07:34.882 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:34.880+0000 7f2363200700 1 -- 172.21.15.153:0/2024444913 shutdown_connections 2024-03-21T10:07:34.882 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:34.880+0000 7f2363200700 1 -- 172.21.15.153:0/2024444913 wait complete. 2024-03-21T10:07:34.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:34 smithi174 ceph-mon[37496]: mgrmap e23: smithi174.qpcwat(active, since 3s) 2024-03-21T10:07:34.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:34 smithi174 ceph-mon[37496]: from='client.14626 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:07:35.056 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:35.054+0000 7f3100400700 1 -- 172.21.15.153:0/1130713941 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f30f8073790 msgr2=0x7f30f8073c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:07:35.056 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:35.054+0000 7f3100400700 1 --2- 172.21.15.153:0/1130713941 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f30f8073790 0x7f30f8073c10 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f30e4009910 tx=0x7f30e4009c20 comp rx=0 tx=0).stop 2024-03-21T10:07:35.056 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:35.054+0000 7f3100400700 1 -- 172.21.15.153:0/1130713941 shutdown_connections 2024-03-21T10:07:35.057 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:35.054+0000 7f3100400700 1 --2- 172.21.15.153:0/1130713941 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f30f8073790 0x7f30f8073c10 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:07:35.057 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:35.054+0000 7f3100400700 1 --2- 172.21.15.153:0/1130713941 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f30f8074d60 0x7f30f80731c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:07:35.057 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:35.054+0000 7f3100400700 1 -- 172.21.15.153:0/1130713941 >> 172.21.15.153:0/1130713941 conn(0x7f30f80fc110 msgr2=0x7f30f80fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:07:35.057 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:35.055+0000 7f3100400700 1 -- 172.21.15.153:0/1130713941 shutdown_connections 2024-03-21T10:07:35.057 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:35.055+0000 7f3100400700 1 -- 172.21.15.153:0/1130713941 wait complete. 2024-03-21T10:07:35.057 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:35.056+0000 7f3100400700 1 Processor -- start 2024-03-21T10:07:35.057 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:35.056+0000 7f3100400700 1 -- start start 2024-03-21T10:07:35.058 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:35.056+0000 7f3100400700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f30f8073790 0x7f30f8198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:07:35.058 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:35.056+0000 7f3100400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f30f8074d60 0x7f30f8198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:07:35.058 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:35.056+0000 7f3100400700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30f8199080 con 0x7f30f8074d60 2024-03-21T10:07:35.058 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:35.056+0000 7f3100400700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30f81991c0 con 0x7f30f8073790 2024-03-21T10:07:35.058 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:35.057+0000 7f30fd000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f30f8074d60 0x7f30f8198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:07:35.058 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:35.057+0000 7f30fda00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f30f8073790 0x7f30f8198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:07:35.059 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:35.057+0000 7f30fd000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f30f8074d60 0x7f30f8198a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:53784/0 (socket says 172.21.15.153:53784) 2024-03-21T10:07:35.059 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:35.057+0000 7f30fd000700 1 -- 172.21.15.153:0/2504156760 learned_addr learned my addr 172.21.15.153:0/2504156760 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:07:35.059 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:35.057+0000 7f30fda00700 1 -- 172.21.15.153:0/2504156760 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f30f8074d60 msgr2=0x7f30f8198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:07:35.059 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:35.057+0000 7f30fda00700 1 --2- 172.21.15.153:0/2504156760 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f30f8074d60 0x7f30f8198a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:07:35.059 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:35.057+0000 7f30fda00700 1 -- 172.21.15.153:0/2504156760 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30e40095f0 con 0x7f30f8073790 2024-03-21T10:07:35.060 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:35.058+0000 7f30fd000700 1 --2- 172.21.15.153:0/2504156760 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f30f8074d60 0x7f30f8198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:07:35.060 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:35.058+0000 7f30fda00700 1 --2- 172.21.15.153:0/2504156760 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f30f8073790 0x7f30f8198520 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f30ec00e970 tx=0x7f30ec00ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:07:35.060 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:35.058+0000 7f30f6000700 1 -- 172.21.15.153:0/2504156760 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f30ec00c9d0 con 0x7f30f8073790 2024-03-21T10:07:35.060 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:35.058+0000 7f3100400700 1 -- 172.21.15.153:0/2504156760 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30f819dc70 con 0x7f30f8073790 2024-03-21T10:07:35.060 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:35.058+0000 7f30f6000700 1 -- 172.21.15.153:0/2504156760 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f30ec010430 con 0x7f30f8073790 2024-03-21T10:07:35.060 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:35.058+0000 7f30f6000700 1 -- 172.21.15.153:0/2504156760 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f30ec018650 con 0x7f30f8073790 2024-03-21T10:07:35.060 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:35.059+0000 7f3100400700 1 -- 172.21.15.153:0/2504156760 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30f819e1c0 con 0x7f30f8073790 2024-03-21T10:07:35.061 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:35.059+0000 7f3100400700 1 -- 172.21.15.153:0/2504156760 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30f810b900 con 0x7f30f8073790 2024-03-21T10:07:35.061 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:35.060+0000 7f30f6000700 1 -- 172.21.15.153:0/2504156760 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 23) v1 ==== 46435+0+0 (secure 0 0 0) 0x7f30ec0105a0 con 0x7f30f8073790 2024-03-21T10:07:35.062 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:35.060+0000 7f30f6000700 1 --2- 172.21.15.153:0/2504156760 >> [v2:172.21.15.174:6828/1155541738,v1:172.21.15.174:6829/1155541738] conn(0x7f30e0039490 0x7f30e003b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:07:35.062 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:35.061+0000 7f30f6000700 1 -- 172.21.15.153:0/2504156760 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f30ec014070 con 0x7f30f8073790 2024-03-21T10:07:35.062 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:35.061+0000 7f30fd000700 1 --2- 172.21.15.153:0/2504156760 >> [v2:172.21.15.174:6828/1155541738,v1:172.21.15.174:6829/1155541738] conn(0x7f30e0039490 0x7f30e003b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:07:35.063 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:35.062+0000 7f30fd000700 1 --2- 172.21.15.153:0/2504156760 >> [v2:172.21.15.174:6828/1155541738,v1:172.21.15.174:6829/1155541738] conn(0x7f30e0039490 0x7f30e003b950 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f30e4009fd0 tx=0x7f30e4005940 comp rx=0 tx=0).ready entity=mgr.24361 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:07:35.070 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:35.069+0000 7f30f6000700 1 -- 172.21.15.153:0/2504156760 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185454 (secure 0 0 0) 0x7f30ec01fe10 con 0x7f30f8073790 2024-03-21T10:07:35.522 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:35.521+0000 7f3100400700 1 -- 172.21.15.153:0/2504156760 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f30f819e7d0 con 0x7f30f8073790 2024-03-21T10:07:35.524 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:35.522+0000 7f30f6000700 1 -- 172.21.15.153:0/2504156760 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7f30ec01f310 con 0x7f30f8073790 2024-03-21T10:07:35.524 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:07:35.524 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": { 2024-03-21T10:07:35.524 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-03-21T10:07:35.524 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:07:35.525 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": { 2024-03-21T10:07:35.525 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 1 2024-03-21T10:07:35.525 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:07:35.525 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": { 2024-03-21T10:07:35.525 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-03-21T10:07:35.525 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:07:35.525 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": { 2024-03-21T10:07:35.525 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-03-21T10:07:35.525 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:07:35.525 INFO:teuthology.orchestra.run.smithi153.stdout: "overall": { 2024-03-21T10:07:35.525 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 12, 2024-03-21T10:07:35.525 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 1 2024-03-21T10:07:35.525 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-03-21T10:07:35.526 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:07:35.528 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:35.526+0000 7f3100400700 1 -- 172.21.15.153:0/2504156760 >> [v2:172.21.15.174:6828/1155541738,v1:172.21.15.174:6829/1155541738] conn(0x7f30e0039490 msgr2=0x7f30e003b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:07:35.528 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:35.526+0000 7f3100400700 1 --2- 172.21.15.153:0/2504156760 >> [v2:172.21.15.174:6828/1155541738,v1:172.21.15.174:6829/1155541738] conn(0x7f30e0039490 0x7f30e003b950 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f30e4009fd0 tx=0x7f30e4005940 comp rx=0 tx=0).stop 2024-03-21T10:07:35.528 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:35.526+0000 7f3100400700 1 -- 172.21.15.153:0/2504156760 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f30f8073790 msgr2=0x7f30f8198520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:07:35.528 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:35.526+0000 7f3100400700 1 --2- 172.21.15.153:0/2504156760 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f30f8073790 0x7f30f8198520 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f30ec00e970 tx=0x7f30ec00ec80 comp rx=0 tx=0).stop 2024-03-21T10:07:35.529 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:35.527+0000 7f3100400700 1 -- 172.21.15.153:0/2504156760 shutdown_connections 2024-03-21T10:07:35.529 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:35.527+0000 7f3100400700 1 --2- 172.21.15.153:0/2504156760 >> [v2:172.21.15.174:6828/1155541738,v1:172.21.15.174:6829/1155541738] conn(0x7f30e0039490 0x7f30e003b950 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:07:35.529 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:35.527+0000 7f3100400700 1 --2- 172.21.15.153:0/2504156760 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f30f8073790 0x7f30f8198520 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:07:35.529 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:35.527+0000 7f3100400700 1 --2- 172.21.15.153:0/2504156760 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f30f8074d60 0x7f30f8198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:07:35.529 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:35.527+0000 7f3100400700 1 -- 172.21.15.153:0/2504156760 >> 172.21.15.153:0/2504156760 conn(0x7f30f80fc110 msgr2=0x7f30f81069f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:07:35.529 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:35.527+0000 7f3100400700 1 -- 172.21.15.153:0/2504156760 shutdown_connections 2024-03-21T10:07:35.529 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:35.528+0000 7f3100400700 1 -- 172.21.15.153:0/2504156760 wait complete. 2024-03-21T10:07:35.737 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:35.735+0000 7f76aa200700 1 -- 172.21.15.153:0/701940159 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f76a4108400 msgr2=0x7f76a4108860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:07:35.737 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:35.735+0000 7f76a2000700 1 -- 172.21.15.153:0/701940159 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f76940054b0 con 0x7f76a4108400 2024-03-21T10:07:35.737 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:35.735+0000 7f76aa200700 1 --2- 172.21.15.153:0/701940159 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f76a4108400 0x7f76a4108860 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f7694009910 tx=0x7f7694009c20 comp rx=0 tx=0).stop 2024-03-21T10:07:35.737 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:35.735+0000 7f76aa200700 1 -- 172.21.15.153:0/701940159 shutdown_connections 2024-03-21T10:07:35.737 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:35.735+0000 7f76aa200700 1 --2- 172.21.15.153:0/701940159 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f76a4108400 0x7f76a4108860 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:07:35.737 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:35.735+0000 7f76aa200700 1 --2- 172.21.15.153:0/701940159 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f76a4107200 0x7f76a4107620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:07:35.737 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:35.735+0000 7f76aa200700 1 -- 172.21.15.153:0/701940159 >> 172.21.15.153:0/701940159 conn(0x7f76a406a1d0 msgr2=0x7f76a4078790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:07:35.738 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:35.736+0000 7f76aa200700 1 -- 172.21.15.153:0/701940159 shutdown_connections 2024-03-21T10:07:35.739 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:35.736+0000 7f76aa200700 1 -- 172.21.15.153:0/701940159 wait complete. 2024-03-21T10:07:35.739 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:35.737+0000 7f76aa200700 1 Processor -- start 2024-03-21T10:07:35.739 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:35.738+0000 7f76aa200700 1 -- start start 2024-03-21T10:07:35.741 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:35.739+0000 7f76aa200700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f76a4107200 0x7f76a4080930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:07:35.741 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:35.739+0000 7f76aa200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f76a4108400 0x7f76a4080e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:07:35.741 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:35.739+0000 7f76aa200700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76a4081490 con 0x7f76a4108400 2024-03-21T10:07:35.741 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:35.739+0000 7f76aa200700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76a40815d0 con 0x7f76a4107200 2024-03-21T10:07:35.741 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:35.739+0000 7f76a3400700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f76a4107200 0x7f76a4080930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:07:35.741 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:35.739+0000 7f76a2a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f76a4108400 0x7f76a4080e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:07:35.741 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:35.739+0000 7f76a2a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f76a4108400 0x7f76a4080e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:53802/0 (socket says 172.21.15.153:53802) 2024-03-21T10:07:35.741 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:35.739+0000 7f76a3400700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f76a4107200 0x7f76a4080930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.153:50996/0 (socket says 172.21.15.153:50996) 2024-03-21T10:07:35.741 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:35.739+0000 7f76a2a00700 1 -- 172.21.15.153:0/4038015492 learned_addr learned my addr 172.21.15.153:0/4038015492 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:07:35.742 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:35.740+0000 7f76a3400700 1 -- 172.21.15.153:0/4038015492 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f76a4108400 msgr2=0x7f76a4080e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:07:35.742 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:35.740+0000 7f76a3400700 1 --2- 172.21.15.153:0/4038015492 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f76a4108400 0x7f76a4080e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:07:35.742 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:35.740+0000 7f76a3400700 1 -- 172.21.15.153:0/4038015492 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76940095f0 con 0x7f76a4107200 2024-03-21T10:07:35.742 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:35.741+0000 7f76a3400700 1 --2- 172.21.15.153:0/4038015492 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f76a4107200 0x7f76a4080930 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f769c00bcc0 tx=0x7f769c00c040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:07:35.742 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:35.741+0000 7f769be00700 1 -- 172.21.15.153:0/4038015492 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f769c00cba0 con 0x7f76a4107200 2024-03-21T10:07:35.742 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:35.741+0000 7f769be00700 1 -- 172.21.15.153:0/4038015492 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f769c00cd00 con 0x7f76a4107200 2024-03-21T10:07:35.743 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:35.741+0000 7f769be00700 1 -- 172.21.15.153:0/4038015492 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f769c0126e0 con 0x7f76a4107200 2024-03-21T10:07:35.743 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:35.741+0000 7f76aa200700 1 -- 172.21.15.153:0/4038015492 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76a41aa7c0 con 0x7f76a4107200 2024-03-21T10:07:35.743 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:35.741+0000 7f76aa200700 1 -- 172.21.15.153:0/4038015492 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76a41aad90 con 0x7f76a4107200 2024-03-21T10:07:35.744 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:35.743+0000 7f76aa200700 1 -- 172.21.15.153:0/4038015492 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7684005320 con 0x7f76a4107200 2024-03-21T10:07:35.744 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:35.743+0000 7f769be00700 1 -- 172.21.15.153:0/4038015492 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 23) v1 ==== 46435+0+0 (secure 0 0 0) 0x7f769c0128c0 con 0x7f76a4107200 2024-03-21T10:07:35.745 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:35.744+0000 7f769be00700 1 --2- 172.21.15.153:0/4038015492 >> [v2:172.21.15.174:6828/1155541738,v1:172.21.15.174:6829/1155541738] conn(0x7f76880394e0 0x7f768803b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:07:35.745 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:35.744+0000 7f76a2a00700 1 --2- 172.21.15.153:0/4038015492 >> [v2:172.21.15.174:6828/1155541738,v1:172.21.15.174:6829/1155541738] conn(0x7f76880394e0 0x7f768803b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:07:35.746 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:35.745+0000 7f769be00700 1 -- 172.21.15.153:0/4038015492 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f769c006e40 con 0x7f76a4107200 2024-03-21T10:07:35.746 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:35.745+0000 7f76a2a00700 1 --2- 172.21.15.153:0/4038015492 >> [v2:172.21.15.174:6828/1155541738,v1:172.21.15.174:6829/1155541738] conn(0x7f76880394e0 0x7f768803b9a0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f7694005090 tx=0x7f769401a040 comp rx=0 tx=0).ready entity=mgr.24361 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:07:35.752 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:35.750+0000 7f769be00700 1 -- 172.21.15.153:0/4038015492 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185454 (secure 0 0 0) 0x7f769c01d3b0 con 0x7f76a4107200 2024-03-21T10:07:35.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:35 smithi153 ceph-mon[29117]: from='client.24389 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:07:35.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:35 smithi153 ceph-mon[29117]: pgmap v5: 65 pgs: 65 active+clean; 80 MiB data, 428 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:07:35.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:35 smithi153 ceph-mon[29117]: from='client.14634 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:07:35.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:35 smithi153 ceph-mon[29117]: from='client.? 172.21.15.153:0/2504156760' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:07:35.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:35 smithi174 ceph-mon[37496]: from='client.24389 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:07:35.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:35 smithi174 ceph-mon[37496]: pgmap v5: 65 pgs: 65 active+clean; 80 MiB data, 428 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:07:35.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:35 smithi174 ceph-mon[37496]: from='client.14634 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:07:35.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:35 smithi174 ceph-mon[37496]: from='client.? 172.21.15.153:0/2504156760' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:07:36.048 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:36.046+0000 7f76aa200700 1 -- 172.21.15.153:0/4038015492 --> [v2:172.21.15.174:6828/1155541738,v1:172.21.15.174:6829/1155541738] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7684000bf0 con 0x7f76880394e0 2024-03-21T10:07:36.053 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:36.052+0000 7f769be00700 1 -- 172.21.15.153:0/4038015492 <== mgr.24361 v2:172.21.15.174:6828/1155541738 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f7684000bf0 con 0x7f76880394e0 2024-03-21T10:07:36.053 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:07:36.054 INFO:teuthology.orchestra.run.smithi153.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232", 2024-03-21T10:07:36.054 INFO:teuthology.orchestra.run.smithi153.stdout: "in_progress": true, 2024-03-21T10:07:36.054 INFO:teuthology.orchestra.run.smithi153.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-03-21T10:07:36.054 INFO:teuthology.orchestra.run.smithi153.stdout: "services_complete": [], 2024-03-21T10:07:36.054 INFO:teuthology.orchestra.run.smithi153.stdout: "progress": "1/2 daemons upgraded", 2024-03-21T10:07:36.054 INFO:teuthology.orchestra.run.smithi153.stdout: "message": "", 2024-03-21T10:07:36.054 INFO:teuthology.orchestra.run.smithi153.stdout: "is_paused": false 2024-03-21T10:07:36.054 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:07:36.056 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:36.055+0000 7f76aa200700 1 -- 172.21.15.153:0/4038015492 >> [v2:172.21.15.174:6828/1155541738,v1:172.21.15.174:6829/1155541738] conn(0x7f76880394e0 msgr2=0x7f768803b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:07:36.056 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:36.055+0000 7f76aa200700 1 --2- 172.21.15.153:0/4038015492 >> [v2:172.21.15.174:6828/1155541738,v1:172.21.15.174:6829/1155541738] conn(0x7f76880394e0 0x7f768803b9a0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f7694005090 tx=0x7f769401a040 comp rx=0 tx=0).stop 2024-03-21T10:07:36.056 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:36.055+0000 7f76aa200700 1 -- 172.21.15.153:0/4038015492 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f76a4107200 msgr2=0x7f76a4080930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:07:36.056 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:36.055+0000 7f76aa200700 1 --2- 172.21.15.153:0/4038015492 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f76a4107200 0x7f76a4080930 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f769c00bcc0 tx=0x7f769c00c040 comp rx=0 tx=0).stop 2024-03-21T10:07:36.056 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:36.055+0000 7f76aa200700 1 -- 172.21.15.153:0/4038015492 shutdown_connections 2024-03-21T10:07:36.057 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:36.055+0000 7f76aa200700 1 --2- 172.21.15.153:0/4038015492 >> [v2:172.21.15.174:6828/1155541738,v1:172.21.15.174:6829/1155541738] conn(0x7f76880394e0 0x7f768803b9a0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:07:36.057 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:36.055+0000 7f76aa200700 1 --2- 172.21.15.153:0/4038015492 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f76a4107200 0x7f76a4080930 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:07:36.057 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:36.055+0000 7f76aa200700 1 --2- 172.21.15.153:0/4038015492 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f76a4108400 0x7f76a4080e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:07:36.057 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:36.055+0000 7f76aa200700 1 -- 172.21.15.153:0/4038015492 >> 172.21.15.153:0/4038015492 conn(0x7f76a406a1d0 msgr2=0x7f76a40785c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:07:36.057 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:36.055+0000 7f76aa200700 1 -- 172.21.15.153:0/4038015492 shutdown_connections 2024-03-21T10:07:36.057 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:07:36.055+0000 7f76aa200700 1 -- 172.21.15.153:0/4038015492 wait complete. 2024-03-21T10:07:37.767 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:37 smithi153 ceph-mon[29117]: from='client.24397 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:07:37.767 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:37 smithi153 ceph-mon[29117]: pgmap v6: 65 pgs: 65 active+clean; 83 MiB data, 447 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 646 KiB/s wr, 43 op/s 2024-03-21T10:07:37.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:37 smithi174 ceph-mon[37496]: from='client.24397 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:07:37.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:37 smithi174 ceph-mon[37496]: pgmap v6: 65 pgs: 65 active+clean; 83 MiB data, 447 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 646 KiB/s wr, 43 op/s 2024-03-21T10:07:39.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:39 smithi153 ceph-mon[29117]: pgmap v7: 65 pgs: 65 active+clean; 96 MiB data, 491 MiB used, 536 GiB / 536 GiB avail; 939 KiB/s rd, 2.3 MiB/s wr, 139 op/s 2024-03-21T10:07:39.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:39 smithi174 ceph-mon[37496]: pgmap v7: 65 pgs: 65 active+clean; 96 MiB data, 491 MiB used, 536 GiB / 536 GiB avail; 939 KiB/s rd, 2.3 MiB/s wr, 139 op/s 2024-03-21T10:07:41.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:41 smithi174 ceph-mon[37496]: pgmap v8: 65 pgs: 65 active+clean; 96 MiB data, 491 MiB used, 536 GiB / 536 GiB avail; 730 KiB/s rd, 1.8 MiB/s wr, 108 op/s 2024-03-21T10:07:41.833 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:41 smithi153 ceph-mon[29117]: pgmap v8: 65 pgs: 65 active+clean; 96 MiB data, 491 MiB used, 536 GiB / 536 GiB avail; 730 KiB/s rd, 1.8 MiB/s wr, 108 op/s 2024-03-21T10:07:42.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:42 smithi153 ceph-mon[29117]: Standby manager daemon smithi153.tluxni started 2024-03-21T10:07:42.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:42 smithi153 ceph-mon[29117]: from='mgr.? 172.21.15.153:0/3962338308' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi153.tluxni/crt"}]: dispatch 2024-03-21T10:07:42.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:42 smithi153 ceph-mon[29117]: from='mgr.? 172.21.15.153:0/3962338308' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-21T10:07:42.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:42 smithi153 ceph-mon[29117]: from='mgr.? 172.21.15.153:0/3962338308' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi153.tluxni/key"}]: dispatch 2024-03-21T10:07:42.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:42 smithi153 ceph-mon[29117]: from='mgr.? 172.21.15.153:0/3962338308' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-21T10:07:42.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:42 smithi174 ceph-mon[37496]: Standby manager daemon smithi153.tluxni started 2024-03-21T10:07:42.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:42 smithi174 ceph-mon[37496]: from='mgr.? 172.21.15.153:0/3962338308' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi153.tluxni/crt"}]: dispatch 2024-03-21T10:07:42.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:42 smithi174 ceph-mon[37496]: from='mgr.? 172.21.15.153:0/3962338308' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-21T10:07:42.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:42 smithi174 ceph-mon[37496]: from='mgr.? 172.21.15.153:0/3962338308' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi153.tluxni/key"}]: dispatch 2024-03-21T10:07:42.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:42 smithi174 ceph-mon[37496]: from='mgr.? 172.21.15.153:0/3962338308' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-21T10:07:43.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:43 smithi153 ceph-mon[29117]: pgmap v9: 65 pgs: 65 active+clean; 105 MiB data, 525 MiB used, 536 GiB / 536 GiB avail; 1.3 MiB/s rd, 2.3 MiB/s wr, 120 op/s 2024-03-21T10:07:43.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:43 smithi153 ceph-mon[29117]: mgrmap e24: smithi174.qpcwat(active, since 12s), standbys: smithi153.tluxni 2024-03-21T10:07:43.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:43 smithi153 ceph-mon[29117]: from='mgr.24361 172.21.15.174:0/1831991646' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "mgr metadata", "who": "smithi153.tluxni", "id": "smithi153.tluxni"}]: dispatch 2024-03-21T10:07:43.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:43 smithi153 ceph-mon[29117]: from='mgr.24361 ' entity='mgr.smithi174.qpcwat' 2024-03-21T10:07:43.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:43 smithi153 ceph-mon[29117]: from='mgr.24361 ' entity='mgr.smithi174.qpcwat' 2024-03-21T10:07:43.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:43 smithi174 ceph-mon[37496]: pgmap v9: 65 pgs: 65 active+clean; 105 MiB data, 525 MiB used, 536 GiB / 536 GiB avail; 1.3 MiB/s rd, 2.3 MiB/s wr, 120 op/s 2024-03-21T10:07:43.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:43 smithi174 ceph-mon[37496]: mgrmap e24: smithi174.qpcwat(active, since 12s), standbys: smithi153.tluxni 2024-03-21T10:07:43.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:43 smithi174 ceph-mon[37496]: from='mgr.24361 172.21.15.174:0/1831991646' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "mgr metadata", "who": "smithi153.tluxni", "id": "smithi153.tluxni"}]: dispatch 2024-03-21T10:07:43.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:43 smithi174 ceph-mon[37496]: from='mgr.24361 ' entity='mgr.smithi174.qpcwat' 2024-03-21T10:07:43.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:43 smithi174 ceph-mon[37496]: from='mgr.24361 ' entity='mgr.smithi174.qpcwat' 2024-03-21T10:07:45.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:45 smithi174 ceph-mon[37496]: pgmap v10: 65 pgs: 65 active+clean; 105 MiB data, 525 MiB used, 536 GiB / 536 GiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 110 op/s 2024-03-21T10:07:45.378 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:45 smithi153 ceph-mon[29117]: pgmap v10: 65 pgs: 65 active+clean; 105 MiB data, 525 MiB used, 536 GiB / 536 GiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 110 op/s 2024-03-21T10:07:46.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:46 smithi174 ceph-mon[37496]: from='mgr.24361 172.21.15.174:0/1831991646' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:07:46.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:46 smithi174 ceph-mon[37496]: from='mgr.24361 ' entity='mgr.smithi174.qpcwat' 2024-03-21T10:07:46.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:46 smithi174 ceph-mon[37496]: from='mgr.24361 ' entity='mgr.smithi174.qpcwat' 2024-03-21T10:07:46.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:46 smithi153 ceph-mon[29117]: from='mgr.24361 172.21.15.174:0/1831991646' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:07:46.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:46 smithi153 ceph-mon[29117]: from='mgr.24361 ' entity='mgr.smithi174.qpcwat' 2024-03-21T10:07:46.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:46 smithi153 ceph-mon[29117]: from='mgr.24361 ' entity='mgr.smithi174.qpcwat' 2024-03-21T10:07:47.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:47 smithi153 ceph-mon[29117]: from='mgr.24361 ' entity='mgr.smithi174.qpcwat' 2024-03-21T10:07:47.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:47 smithi153 ceph-mon[29117]: pgmap v11: 65 pgs: 65 active+clean; 112 MiB data, 542 MiB used, 536 GiB / 536 GiB avail; 1.8 MiB/s rd, 2.7 MiB/s wr, 136 op/s 2024-03-21T10:07:47.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:47 smithi153 ceph-mon[29117]: from='mgr.24361 ' entity='mgr.smithi174.qpcwat' 2024-03-21T10:07:47.813 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:47 smithi174 ceph-mon[37496]: from='mgr.24361 ' entity='mgr.smithi174.qpcwat' 2024-03-21T10:07:47.813 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:47 smithi174 ceph-mon[37496]: pgmap v11: 65 pgs: 65 active+clean; 112 MiB data, 542 MiB used, 536 GiB / 536 GiB avail; 1.8 MiB/s rd, 2.7 MiB/s wr, 136 op/s 2024-03-21T10:07:47.813 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:47 smithi174 ceph-mon[37496]: from='mgr.24361 ' entity='mgr.smithi174.qpcwat' 2024-03-21T10:07:49.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:49 smithi174 ceph-mon[37496]: pgmap v12: 65 pgs: 65 active+clean; 113 MiB data, 555 MiB used, 536 GiB / 536 GiB avail; 1.8 MiB/s rd, 2.5 MiB/s wr, 140 op/s 2024-03-21T10:07:49.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:49 smithi174 ceph-mon[37496]: from='mgr.24361 ' entity='mgr.smithi174.qpcwat' 2024-03-21T10:07:49.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:49 smithi174 ceph-mon[37496]: from='mgr.24361 ' entity='mgr.smithi174.qpcwat' 2024-03-21T10:07:49.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:49 smithi174 ceph-mon[37496]: from='mgr.24361 172.21.15.174:0/1831991646' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:07:49.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:49 smithi174 ceph-mon[37496]: from='mgr.24361 ' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:07:49.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:49 smithi174 ceph-mon[37496]: from='mgr.24361 172.21.15.174:0/1831991646' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:07:49.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:49 smithi174 ceph-mon[37496]: from='mgr.24361 ' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:07:49.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:49 smithi174 ceph-mon[37496]: from='mgr.24361 172.21.15.174:0/1831991646' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:07:49.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:49 smithi174 ceph-mon[37496]: from='mgr.24361 ' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:07:49.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:49 smithi174 ceph-mon[37496]: Adjusting osd_memory_target on smithi153 to 1217M 2024-03-21T10:07:49.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:49 smithi174 ceph-mon[37496]: from='mgr.24361 ' entity='mgr.smithi174.qpcwat' 2024-03-21T10:07:50.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:49 smithi153 ceph-mon[29117]: pgmap v12: 65 pgs: 65 active+clean; 113 MiB data, 555 MiB used, 536 GiB / 536 GiB avail; 1.8 MiB/s rd, 2.5 MiB/s wr, 140 op/s 2024-03-21T10:07:50.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:49 smithi153 ceph-mon[29117]: from='mgr.24361 ' entity='mgr.smithi174.qpcwat' 2024-03-21T10:07:50.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:49 smithi153 ceph-mon[29117]: from='mgr.24361 ' entity='mgr.smithi174.qpcwat' 2024-03-21T10:07:50.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:49 smithi153 ceph-mon[29117]: from='mgr.24361 172.21.15.174:0/1831991646' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:07:50.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:49 smithi153 ceph-mon[29117]: from='mgr.24361 ' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:07:50.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:49 smithi153 ceph-mon[29117]: from='mgr.24361 172.21.15.174:0/1831991646' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:07:50.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:49 smithi153 ceph-mon[29117]: from='mgr.24361 ' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:07:50.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:49 smithi153 ceph-mon[29117]: from='mgr.24361 172.21.15.174:0/1831991646' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:07:50.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:49 smithi153 ceph-mon[29117]: from='mgr.24361 ' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:07:50.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:49 smithi153 ceph-mon[29117]: Adjusting osd_memory_target on smithi153 to 1217M 2024-03-21T10:07:50.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:49 smithi153 ceph-mon[29117]: from='mgr.24361 ' entity='mgr.smithi174.qpcwat' 2024-03-21T10:07:51.811 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:51 smithi174 ceph-mon[37496]: pgmap v13: 65 pgs: 65 active+clean; 113 MiB data, 555 MiB used, 536 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 77 op/s 2024-03-21T10:07:51.811 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:51 smithi174 ceph-mon[37496]: from='mgr.24361 ' entity='mgr.smithi174.qpcwat' 2024-03-21T10:07:51.811 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:51 smithi174 ceph-mon[37496]: from='mgr.24361 ' entity='mgr.smithi174.qpcwat' 2024-03-21T10:07:51.811 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:51 smithi174 ceph-mon[37496]: from='mgr.24361 172.21.15.174:0/1831991646' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:07:51.811 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:51 smithi174 ceph-mon[37496]: from='mgr.24361 ' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:07:51.811 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:51 smithi174 ceph-mon[37496]: from='mgr.24361 172.21.15.174:0/1831991646' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:07:51.811 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:51 smithi174 ceph-mon[37496]: from='mgr.24361 ' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:07:51.811 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:51 smithi174 ceph-mon[37496]: from='mgr.24361 172.21.15.174:0/1831991646' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:07:51.811 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:51 smithi174 ceph-mon[37496]: from='mgr.24361 ' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:07:51.811 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:51 smithi174 ceph-mon[37496]: from='mgr.24361 ' entity='mgr.smithi174.qpcwat' 2024-03-21T10:07:51.811 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:51 smithi174 ceph-mon[37496]: from='mgr.24361 172.21.15.174:0/1831991646' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:07:51.812 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:51 smithi174 ceph-mon[37496]: from='mgr.24361 172.21.15.174:0/1831991646' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:07:52.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:51 smithi153 ceph-mon[29117]: pgmap v13: 65 pgs: 65 active+clean; 113 MiB data, 555 MiB used, 536 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 77 op/s 2024-03-21T10:07:52.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:51 smithi153 ceph-mon[29117]: from='mgr.24361 ' entity='mgr.smithi174.qpcwat' 2024-03-21T10:07:52.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:51 smithi153 ceph-mon[29117]: from='mgr.24361 ' entity='mgr.smithi174.qpcwat' 2024-03-21T10:07:52.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:51 smithi153 ceph-mon[29117]: from='mgr.24361 172.21.15.174:0/1831991646' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:07:52.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:51 smithi153 ceph-mon[29117]: from='mgr.24361 ' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:07:52.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:51 smithi153 ceph-mon[29117]: from='mgr.24361 172.21.15.174:0/1831991646' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:07:52.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:51 smithi153 ceph-mon[29117]: from='mgr.24361 ' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:07:52.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:51 smithi153 ceph-mon[29117]: from='mgr.24361 172.21.15.174:0/1831991646' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:07:52.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:51 smithi153 ceph-mon[29117]: from='mgr.24361 ' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T10:07:52.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:51 smithi153 ceph-mon[29117]: from='mgr.24361 ' entity='mgr.smithi174.qpcwat' 2024-03-21T10:07:52.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:51 smithi153 ceph-mon[29117]: from='mgr.24361 172.21.15.174:0/1831991646' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:07:52.058 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:51 smithi153 ceph-mon[29117]: from='mgr.24361 172.21.15.174:0/1831991646' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:07:52.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:52 smithi174 ceph-mon[37496]: Adjusting osd_memory_target on smithi174 to 2241M 2024-03-21T10:07:52.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:52 smithi174 ceph-mon[37496]: Updating smithi153:/etc/ceph/ceph.conf 2024-03-21T10:07:52.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:52 smithi174 ceph-mon[37496]: Updating smithi174:/etc/ceph/ceph.conf 2024-03-21T10:07:52.955 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:52 smithi153 ceph-mon[29117]: Adjusting osd_memory_target on smithi174 to 2241M 2024-03-21T10:07:52.956 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:52 smithi153 ceph-mon[29117]: Updating smithi153:/etc/ceph/ceph.conf 2024-03-21T10:07:52.956 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:52 smithi153 ceph-mon[29117]: Updating smithi174:/etc/ceph/ceph.conf 2024-03-21T10:07:53.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:53 smithi174 ceph-mon[37496]: Updating smithi153:/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T10:07:53.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:53 smithi174 ceph-mon[37496]: Updating smithi174:/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T10:07:53.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:53 smithi174 ceph-mon[37496]: pgmap v14: 65 pgs: 65 active+clean; 121 MiB data, 584 MiB used, 536 GiB / 536 GiB avail; 1.8 MiB/s rd, 2.2 MiB/s wr, 110 op/s 2024-03-21T10:07:53.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:53 smithi174 ceph-mon[37496]: Updating smithi153:/etc/ceph/ceph.client.admin.keyring 2024-03-21T10:07:53.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:53 smithi174 ceph-mon[37496]: Updating smithi174:/etc/ceph/ceph.client.admin.keyring 2024-03-21T10:07:54.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:53 smithi153 ceph-mon[29117]: Updating smithi153:/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T10:07:54.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:53 smithi153 ceph-mon[29117]: Updating smithi174:/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T10:07:54.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:53 smithi153 ceph-mon[29117]: pgmap v14: 65 pgs: 65 active+clean; 121 MiB data, 584 MiB used, 536 GiB / 536 GiB avail; 1.8 MiB/s rd, 2.2 MiB/s wr, 110 op/s 2024-03-21T10:07:54.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:53 smithi153 ceph-mon[29117]: Updating smithi153:/etc/ceph/ceph.client.admin.keyring 2024-03-21T10:07:54.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:53 smithi153 ceph-mon[29117]: Updating smithi174:/etc/ceph/ceph.client.admin.keyring 2024-03-21T10:07:54.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:54 smithi174 ceph-mon[37496]: Updating smithi153:/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-21T10:07:54.994 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:54 smithi174 ceph-mon[37496]: Updating smithi174:/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-21T10:07:54.994 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:54 smithi174 ceph-mon[37496]: from='mgr.24361 ' entity='mgr.smithi174.qpcwat' 2024-03-21T10:07:54.994 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:54 smithi174 ceph-mon[37496]: from='mgr.24361 ' entity='mgr.smithi174.qpcwat' 2024-03-21T10:07:54.994 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:54 smithi174 ceph-mon[37496]: from='mgr.24361 ' entity='mgr.smithi174.qpcwat' 2024-03-21T10:07:54.994 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:54 smithi174 ceph-mon[37496]: from='mgr.24361 ' entity='mgr.smithi174.qpcwat' 2024-03-21T10:07:54.994 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:54 smithi174 ceph-mon[37496]: from='mgr.24361 ' entity='mgr.smithi174.qpcwat' 2024-03-21T10:07:55.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:54 smithi153 ceph-mon[29117]: Updating smithi153:/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-21T10:07:55.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:54 smithi153 ceph-mon[29117]: Updating smithi174:/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-21T10:07:55.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:54 smithi153 ceph-mon[29117]: from='mgr.24361 ' entity='mgr.smithi174.qpcwat' 2024-03-21T10:07:55.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:54 smithi153 ceph-mon[29117]: from='mgr.24361 ' entity='mgr.smithi174.qpcwat' 2024-03-21T10:07:55.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:54 smithi153 ceph-mon[29117]: from='mgr.24361 ' entity='mgr.smithi174.qpcwat' 2024-03-21T10:07:55.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:54 smithi153 ceph-mon[29117]: from='mgr.24361 ' entity='mgr.smithi174.qpcwat' 2024-03-21T10:07:55.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:54 smithi153 ceph-mon[29117]: from='mgr.24361 ' entity='mgr.smithi174.qpcwat' 2024-03-21T10:07:55.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:55 smithi174 ceph-mon[37496]: Reconfiguring prometheus.smithi153 (dependencies changed)... 2024-03-21T10:07:55.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:55 smithi174 ceph-mon[37496]: pgmap v15: 65 pgs: 65 active+clean; 121 MiB data, 584 MiB used, 536 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 82 op/s 2024-03-21T10:07:55.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:55 smithi174 ceph-mon[37496]: Reconfiguring daemon prometheus.smithi153 on smithi153 2024-03-21T10:07:55.952 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:55 smithi153 ceph-mon[29117]: Reconfiguring prometheus.smithi153 (dependencies changed)... 2024-03-21T10:07:55.952 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:55 smithi153 ceph-mon[29117]: pgmap v15: 65 pgs: 65 active+clean; 121 MiB data, 584 MiB used, 536 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 82 op/s 2024-03-21T10:07:55.952 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:55 smithi153 ceph-mon[29117]: Reconfiguring daemon prometheus.smithi153 on smithi153 2024-03-21T10:07:57.587 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:57 smithi153 ceph-mon[29117]: fsmap cephfs:2 {0=cephfs.smithi153.iuyluj=up:active,1=cephfs.smithi174.yyimst=up:active} 2 up:standby-replay 2024-03-21T10:07:57.587 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:57 smithi153 ceph-mon[29117]: pgmap v16: 65 pgs: 65 active+clean; 130 MiB data, 604 MiB used, 536 GiB / 536 GiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 107 op/s 2024-03-21T10:07:57.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:57 smithi174 ceph-mon[37496]: fsmap cephfs:2 {0=cephfs.smithi153.iuyluj=up:active,1=cephfs.smithi174.yyimst=up:active} 2 up:standby-replay 2024-03-21T10:07:57.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:57 smithi174 ceph-mon[37496]: pgmap v16: 65 pgs: 65 active+clean; 130 MiB data, 604 MiB used, 536 GiB / 536 GiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 107 op/s 2024-03-21T10:07:59.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:07:59 smithi153 ceph-mon[29117]: pgmap v17: 65 pgs: 65 active+clean; 135 MiB data, 620 MiB used, 536 GiB / 536 GiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 112 op/s 2024-03-21T10:07:59.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:07:59 smithi174 ceph-mon[37496]: pgmap v17: 65 pgs: 65 active+clean; 135 MiB data, 620 MiB used, 536 GiB / 536 GiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 112 op/s 2024-03-21T10:08:01.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:01 smithi174 ceph-mon[37496]: pgmap v18: 65 pgs: 65 active+clean; 135 MiB data, 620 MiB used, 536 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 90 op/s 2024-03-21T10:08:01.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:01 smithi174 ceph-mon[37496]: from='mgr.24361 172.21.15.174:0/1831991646' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:08:02.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:01 smithi153 ceph-mon[29117]: pgmap v18: 65 pgs: 65 active+clean; 135 MiB data, 620 MiB used, 536 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 90 op/s 2024-03-21T10:08:02.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:01 smithi153 ceph-mon[29117]: from='mgr.24361 172.21.15.174:0/1831991646' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:08:03.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:03 smithi153 ceph-mon[29117]: from='mgr.24361 ' entity='mgr.smithi174.qpcwat' 2024-03-21T10:08:03.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:03 smithi153 ceph-mon[29117]: from='mgr.24361 ' entity='mgr.smithi174.qpcwat' 2024-03-21T10:08:03.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:03 smithi153 ceph-mon[29117]: from='mgr.24361 172.21.15.174:0/1831991646' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-21T10:08:03.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:03 smithi153 ceph-mon[29117]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-21T10:08:03.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:03 smithi153 ceph-mon[29117]: from='mgr.24361 172.21.15.174:0/1831991646' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:08:03.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:03 smithi153 ceph-mon[29117]: Upgrade: Need to upgrade myself (mgr.smithi174.qpcwat) 2024-03-21T10:08:03.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:03 smithi153 ceph-mon[29117]: Upgrade: Need to upgrade myself (mgr.smithi174.qpcwat) 2024-03-21T10:08:03.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:03 smithi153 ceph-mon[29117]: pgmap v19: 65 pgs: 65 active+clean; 148 MiB data, 673 MiB used, 536 GiB / 536 GiB avail; 1.8 MiB/s rd, 3.0 MiB/s wr, 150 op/s 2024-03-21T10:08:03.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:03 smithi174 ceph-mon[37496]: from='mgr.24361 ' entity='mgr.smithi174.qpcwat' 2024-03-21T10:08:03.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:03 smithi174 ceph-mon[37496]: from='mgr.24361 ' entity='mgr.smithi174.qpcwat' 2024-03-21T10:08:03.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:03 smithi174 ceph-mon[37496]: from='mgr.24361 172.21.15.174:0/1831991646' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-21T10:08:03.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:03 smithi174 ceph-mon[37496]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-21T10:08:03.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:03 smithi174 ceph-mon[37496]: from='mgr.24361 172.21.15.174:0/1831991646' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:08:03.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:03 smithi174 ceph-mon[37496]: Upgrade: Need to upgrade myself (mgr.smithi174.qpcwat) 2024-03-21T10:08:03.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:03 smithi174 ceph-mon[37496]: Upgrade: Need to upgrade myself (mgr.smithi174.qpcwat) 2024-03-21T10:08:03.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:03 smithi174 ceph-mon[37496]: pgmap v19: 65 pgs: 65 active+clean; 148 MiB data, 673 MiB used, 536 GiB / 536 GiB avail; 1.8 MiB/s rd, 3.0 MiB/s wr, 150 op/s 2024-03-21T10:08:06.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:05 smithi174 ceph-mon[37496]: pgmap v20: 65 pgs: 65 active+clean; 148 MiB data, 673 MiB used, 536 GiB / 536 GiB avail; 1.3 MiB/s rd, 2.3 MiB/s wr, 116 op/s 2024-03-21T10:08:06.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:05 smithi174 ceph-mon[37496]: Upgrade: Updating mgr.smithi153.tluxni 2024-03-21T10:08:06.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:05 smithi174 ceph-mon[37496]: from='mgr.24361 ' entity='mgr.smithi174.qpcwat' 2024-03-21T10:08:06.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:05 smithi174 ceph-mon[37496]: from='mgr.24361 172.21.15.174:0/1831991646' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi153.tluxni", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-21T10:08:06.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:05 smithi174 ceph-mon[37496]: from='mgr.24361 ' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi153.tluxni", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-21T10:08:06.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:05 smithi174 ceph-mon[37496]: from='mgr.24361 172.21.15.174:0/1831991646' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-21T10:08:06.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:05 smithi174 ceph-mon[37496]: from='mgr.24361 172.21.15.174:0/1831991646' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:08:06.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:05 smithi174 ceph-mon[37496]: Deploying daemon mgr.smithi153.tluxni on smithi153 2024-03-21T10:08:06.239 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:06.237+0000 7f00dd000700 1 -- 172.21.15.153:0/2066410950 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f00d8101db0 msgr2=0x7f00d8102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:08:06.239 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:06.237+0000 7f00dd000700 1 --2- 172.21.15.153:0/2066410950 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f00d8101db0 0x7f00d8102210 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f00c4009910 tx=0x7f00c4009c20 comp rx=0 tx=0).stop 2024-03-21T10:08:06.239 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:05 smithi153 ceph-mon[29117]: pgmap v20: 65 pgs: 65 active+clean; 148 MiB data, 673 MiB used, 536 GiB / 536 GiB avail; 1.3 MiB/s rd, 2.3 MiB/s wr, 116 op/s 2024-03-21T10:08:06.239 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:05 smithi153 ceph-mon[29117]: Upgrade: Updating mgr.smithi153.tluxni 2024-03-21T10:08:06.239 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:05 smithi153 ceph-mon[29117]: from='mgr.24361 ' entity='mgr.smithi174.qpcwat' 2024-03-21T10:08:06.240 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:05 smithi153 ceph-mon[29117]: from='mgr.24361 172.21.15.174:0/1831991646' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi153.tluxni", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-21T10:08:06.240 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:05 smithi153 ceph-mon[29117]: from='mgr.24361 ' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi153.tluxni", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-21T10:08:06.240 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:05 smithi153 ceph-mon[29117]: from='mgr.24361 172.21.15.174:0/1831991646' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-21T10:08:06.240 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:05 smithi153 ceph-mon[29117]: from='mgr.24361 172.21.15.174:0/1831991646' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:08:06.240 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:05 smithi153 ceph-mon[29117]: Deploying daemon mgr.smithi153.tluxni on smithi153 2024-03-21T10:08:06.241 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:06.237+0000 7f00dd000700 1 -- 172.21.15.153:0/2066410950 shutdown_connections 2024-03-21T10:08:06.241 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:06.237+0000 7f00dd000700 1 --2- 172.21.15.153:0/2066410950 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f00d8101db0 0x7f00d8102210 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:08:06.241 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:06.237+0000 7f00dd000700 1 --2- 172.21.15.153:0/2066410950 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f00d8100bb0 0x7f00d8100fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:08:06.241 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:06.237+0000 7f00dd000700 1 -- 172.21.15.153:0/2066410950 >> 172.21.15.153:0/2066410950 conn(0x7f00d80fc110 msgr2=0x7f00d80fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:08:06.242 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:06.237+0000 7f00dd000700 1 -- 172.21.15.153:0/2066410950 shutdown_connections 2024-03-21T10:08:06.242 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:06.238+0000 7f00dd000700 1 -- 172.21.15.153:0/2066410950 wait complete. 2024-03-21T10:08:06.242 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:06.239+0000 7f00dd000700 1 Processor -- start 2024-03-21T10:08:06.242 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:06.239+0000 7f00dd000700 1 -- start start 2024-03-21T10:08:06.242 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:06.239+0000 7f00dd000700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f00d8100bb0 0x7f00d81985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:08:06.242 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:06.239+0000 7f00dd000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f00d8101db0 0x7f00d8198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:08:06.242 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:06.239+0000 7f00dd000700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00d8199140 con 0x7f00d8101db0 2024-03-21T10:08:06.242 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:06.239+0000 7f00dd000700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00d8199280 con 0x7f00d8100bb0 2024-03-21T10:08:06.242 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:06.239+0000 7f00d5400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f00d8101db0 0x7f00d8198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:08:06.242 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:06.239+0000 7f00d5400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f00d8101db0 0x7f00d8198b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:47636/0 (socket says 172.21.15.153:47636) 2024-03-21T10:08:06.242 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:06.239+0000 7f00d5400700 1 -- 172.21.15.153:0/2948188916 learned_addr learned my addr 172.21.15.153:0/2948188916 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:08:06.242 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:06.240+0000 7f00d5e00700 1 --2- 172.21.15.153:0/2948188916 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f00d8100bb0 0x7f00d81985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:08:06.242 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:06.240+0000 7f00d5400700 1 -- 172.21.15.153:0/2948188916 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f00d8100bb0 msgr2=0x7f00d81985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:08:06.243 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:06.240+0000 7f00d5400700 1 --2- 172.21.15.153:0/2948188916 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f00d8100bb0 0x7f00d81985e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:08:06.243 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:06.240+0000 7f00d5400700 1 -- 172.21.15.153:0/2948188916 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00c40095f0 con 0x7f00d8101db0 2024-03-21T10:08:06.243 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:06.240+0000 7f00d5400700 1 --2- 172.21.15.153:0/2948188916 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f00d8101db0 0x7f00d8198b20 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f00c4005b40 tx=0x7f00c400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:08:06.243 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:06.240+0000 7f00cea00700 1 -- 172.21.15.153:0/2948188916 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f00c401d070 con 0x7f00d8101db0 2024-03-21T10:08:06.243 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:06.240+0000 7f00cea00700 1 -- 172.21.15.153:0/2948188916 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f00c4022470 con 0x7f00d8101db0 2024-03-21T10:08:06.243 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:06.240+0000 7f00dd000700 1 -- 172.21.15.153:0/2948188916 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00d819dcd0 con 0x7f00d8101db0 2024-03-21T10:08:06.244 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:06.240+0000 7f00cea00700 1 -- 172.21.15.153:0/2948188916 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f00c400f690 con 0x7f00d8101db0 2024-03-21T10:08:06.244 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:06.241+0000 7f00dd000700 1 -- 172.21.15.153:0/2948188916 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00d819e240 con 0x7f00d8101db0 2024-03-21T10:08:06.244 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:06.242+0000 7f00dd000700 1 -- 172.21.15.153:0/2948188916 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00d8105e90 con 0x7f00d8101db0 2024-03-21T10:08:06.244 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:06.242+0000 7f00cea00700 1 -- 172.21.15.153:0/2948188916 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 24) v1 ==== 91295+0+0 (secure 0 0 0) 0x7f00c40225e0 con 0x7f00d8101db0 2024-03-21T10:08:06.245 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:06.244+0000 7f00cea00700 1 --2- 172.21.15.153:0/2948188916 >> [v2:172.21.15.174:6828/1155541738,v1:172.21.15.174:6829/1155541738] conn(0x7f00bc06d410 0x7f00bc06f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:08:06.245 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:06.244+0000 7f00cea00700 1 -- 172.21.15.153:0/2948188916 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f00c408e5d0 con 0x7f00d8101db0 2024-03-21T10:08:06.246 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:06.244+0000 7f00d5e00700 1 --2- 172.21.15.153:0/2948188916 >> [v2:172.21.15.174:6828/1155541738,v1:172.21.15.174:6829/1155541738] conn(0x7f00bc06d410 0x7f00bc06f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:08:06.247 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:06.246+0000 7f00d5e00700 1 --2- 172.21.15.153:0/2948188916 >> [v2:172.21.15.174:6828/1155541738,v1:172.21.15.174:6829/1155541738] conn(0x7f00bc06d410 0x7f00bc06f8d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f00c8006fd0 tx=0x7f00c8008040 comp rx=0 tx=0).ready entity=mgr.24361 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:08:06.251 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:06.250+0000 7f00cea00700 1 -- 172.21.15.153:0/2948188916 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185454 (secure 0 0 0) 0x7f00c405b890 con 0x7f00d8101db0 2024-03-21T10:08:06.528 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:06.526+0000 7f00dd000700 1 -- 172.21.15.153:0/2948188916 --> [v2:172.21.15.174:6828/1155541738,v1:172.21.15.174:6829/1155541738] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f00d80611d0 con 0x7f00bc06d410 2024-03-21T10:08:06.531 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:06.529+0000 7f00cea00700 1 -- 172.21.15.153:0/2948188916 <== mgr.24361 v2:172.21.15.174:6828/1155541738 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7f00d80611d0 con 0x7f00bc06d410 2024-03-21T10:08:06.535 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:06.533+0000 7f00dd000700 1 -- 172.21.15.153:0/2948188916 >> [v2:172.21.15.174:6828/1155541738,v1:172.21.15.174:6829/1155541738] conn(0x7f00bc06d410 msgr2=0x7f00bc06f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:08:06.535 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:06.533+0000 7f00dd000700 1 --2- 172.21.15.153:0/2948188916 >> [v2:172.21.15.174:6828/1155541738,v1:172.21.15.174:6829/1155541738] conn(0x7f00bc06d410 0x7f00bc06f8d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f00c8006fd0 tx=0x7f00c8008040 comp rx=0 tx=0).stop 2024-03-21T10:08:06.535 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:06.533+0000 7f00dd000700 1 -- 172.21.15.153:0/2948188916 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f00d8101db0 msgr2=0x7f00d8198b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:08:06.535 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:06.533+0000 7f00dd000700 1 --2- 172.21.15.153:0/2948188916 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f00d8101db0 0x7f00d8198b20 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f00c4005b40 tx=0x7f00c400be20 comp rx=0 tx=0).stop 2024-03-21T10:08:06.536 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:06.534+0000 7f00dd000700 1 -- 172.21.15.153:0/2948188916 shutdown_connections 2024-03-21T10:08:06.536 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:06.534+0000 7f00dd000700 1 --2- 172.21.15.153:0/2948188916 >> [v2:172.21.15.174:6828/1155541738,v1:172.21.15.174:6829/1155541738] conn(0x7f00bc06d410 0x7f00bc06f8d0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:08:06.536 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:06.534+0000 7f00dd000700 1 --2- 172.21.15.153:0/2948188916 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f00d8100bb0 0x7f00d81985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:08:06.536 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:06.534+0000 7f00dd000700 1 --2- 172.21.15.153:0/2948188916 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f00d8101db0 0x7f00d8198b20 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:08:06.536 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:06.534+0000 7f00dd000700 1 -- 172.21.15.153:0/2948188916 >> 172.21.15.153:0/2948188916 conn(0x7f00d80fc110 msgr2=0x7f00d8102fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:08:06.536 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:06.534+0000 7f00dd000700 1 -- 172.21.15.153:0/2948188916 shutdown_connections 2024-03-21T10:08:06.536 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:06.534+0000 7f00dd000700 1 -- 172.21.15.153:0/2948188916 wait complete. 2024-03-21T10:08:06.550 INFO:teuthology.orchestra.run.smithi153.stdout:true 2024-03-21T10:08:06.708 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:06.706+0000 7fae04400700 1 -- 172.21.15.153:0/3675088147 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fadfc101640 msgr2=0x7fadfc103a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:08:06.708 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:06.706+0000 7fae04400700 1 --2- 172.21.15.153:0/3675088147 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fadfc101640 0x7fadfc103a30 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fadf0009910 tx=0x7fadf0009c20 comp rx=0 tx=0).stop 2024-03-21T10:08:06.708 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:06.706+0000 7fae04400700 1 -- 172.21.15.153:0/3675088147 shutdown_connections 2024-03-21T10:08:06.708 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:06.706+0000 7fae04400700 1 --2- 172.21.15.153:0/3675088147 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fadfc103f70 0x7fadfc106360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:08:06.708 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:06.706+0000 7fae04400700 1 --2- 172.21.15.153:0/3675088147 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fadfc101640 0x7fadfc103a30 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:08:06.708 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:06.706+0000 7fae04400700 1 -- 172.21.15.153:0/3675088147 >> 172.21.15.153:0/3675088147 conn(0x7fadfc0fb050 msgr2=0x7fadfc0fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:08:06.708 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:06.706+0000 7fae04400700 1 -- 172.21.15.153:0/3675088147 shutdown_connections 2024-03-21T10:08:06.708 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:06.706+0000 7fae04400700 1 -- 172.21.15.153:0/3675088147 wait complete. 2024-03-21T10:08:06.708 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:06.707+0000 7fae04400700 1 Processor -- start 2024-03-21T10:08:06.709 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:06.708+0000 7fae04400700 1 -- start start 2024-03-21T10:08:06.709 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:06.708+0000 7fae04400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fadfc101640 0x7fadfc100fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:08:06.710 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:06.708+0000 7fae04400700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fadfc103f70 0x7fadfc0ff5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:08:06.710 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:06.708+0000 7fae04400700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadfc0ffb30 con 0x7fadfc101640 2024-03-21T10:08:06.710 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:06.708+0000 7fae04400700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadfc0ffc70 con 0x7fadfc103f70 2024-03-21T10:08:06.710 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:06.708+0000 7fae01a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fadfc101640 0x7fadfc100fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:08:06.710 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:06.709+0000 7fae01a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fadfc101640 0x7fadfc100fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:47644/0 (socket says 172.21.15.153:47644) 2024-03-21T10:08:06.710 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:06.709+0000 7fae01a00700 1 -- 172.21.15.153:0/2977832580 learned_addr learned my addr 172.21.15.153:0/2977832580 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:08:06.711 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:06.709+0000 7fae01000700 1 --2- 172.21.15.153:0/2977832580 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fadfc103f70 0x7fadfc0ff5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:08:06.711 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:06.709+0000 7fae01a00700 1 -- 172.21.15.153:0/2977832580 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fadfc103f70 msgr2=0x7fadfc0ff5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:08:06.711 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:06.709+0000 7fae01a00700 1 --2- 172.21.15.153:0/2977832580 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fadfc103f70 0x7fadfc0ff5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:08:06.711 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:06.709+0000 7fae01a00700 1 -- 172.21.15.153:0/2977832580 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadf00095f0 con 0x7fadfc101640 2024-03-21T10:08:06.711 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:06.710+0000 7fae01a00700 1 --2- 172.21.15.153:0/2977832580 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fadfc101640 0x7fadfc100fa0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fadf0005b40 tx=0x7fadf000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:08:06.712 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:06.710+0000 7fadfa000700 1 -- 172.21.15.153:0/2977832580 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fadf001d070 con 0x7fadfc101640 2024-03-21T10:08:06.713 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:06.710+0000 7fadfa000700 1 -- 172.21.15.153:0/2977832580 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fadf000f460 con 0x7fadfc101640 2024-03-21T10:08:06.713 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:06.711+0000 7fadfa000700 1 -- 172.21.15.153:0/2977832580 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fadf0021620 con 0x7fadfc101640 2024-03-21T10:08:06.713 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:06.711+0000 7fae04400700 1 -- 172.21.15.153:0/2977832580 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadfc0ffef0 con 0x7fadfc101640 2024-03-21T10:08:06.713 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:06.711+0000 7fae04400700 1 -- 172.21.15.153:0/2977832580 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadfc1003e0 con 0x7fadfc101640 2024-03-21T10:08:06.713 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:06.712+0000 7fae04400700 1 -- 172.21.15.153:0/2977832580 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadfc190520 con 0x7fadfc101640 2024-03-21T10:08:06.715 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:06.714+0000 7fadfa000700 1 -- 172.21.15.153:0/2977832580 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 24) v1 ==== 91295+0+0 (secure 0 0 0) 0x7fadf000fa40 con 0x7fadfc101640 2024-03-21T10:08:06.718 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:06.715+0000 7fadfa000700 1 --2- 172.21.15.153:0/2977832580 >> [v2:172.21.15.174:6828/1155541738,v1:172.21.15.174:6829/1155541738] conn(0x7fade406d410 0x7fade406f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:08:06.718 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:06.715+0000 7fadfa000700 1 -- 172.21.15.153:0/2977832580 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7fadf008dfa0 con 0x7fadfc101640 2024-03-21T10:08:06.718 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:06.715+0000 7fae01000700 1 --2- 172.21.15.153:0/2977832580 >> [v2:172.21.15.174:6828/1155541738,v1:172.21.15.174:6829/1155541738] conn(0x7fade406d410 0x7fade406f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:08:06.718 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:06.716+0000 7fae01000700 1 --2- 172.21.15.153:0/2977832580 >> [v2:172.21.15.174:6828/1155541738,v1:172.21.15.174:6829/1155541738] conn(0x7fade406d410 0x7fade406f8d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fade8005950 tx=0x7fade800b410 comp rx=0 tx=0).ready entity=mgr.24361 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:08:06.722 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:06.720+0000 7fadfa000700 1 -- 172.21.15.153:0/2977832580 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185454 (secure 0 0 0) 0x7fadf005b160 con 0x7fadfc101640 2024-03-21T10:08:07.049 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:07.048+0000 7fae04400700 1 -- 172.21.15.153:0/2977832580 --> [v2:172.21.15.174:6828/1155541738,v1:172.21.15.174:6829/1155541738] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fadfc0611d0 con 0x7fade406d410 2024-03-21T10:08:07.053 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:07.052+0000 7fadfa000700 1 -- 172.21.15.153:0/2977832580 <== mgr.24361 v2:172.21.15.174:6828/1155541738 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7fadfc0611d0 con 0x7fade406d410 2024-03-21T10:08:07.057 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:07.055+0000 7fae04400700 1 -- 172.21.15.153:0/2977832580 >> [v2:172.21.15.174:6828/1155541738,v1:172.21.15.174:6829/1155541738] conn(0x7fade406d410 msgr2=0x7fade406f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:08:07.057 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:07.055+0000 7fae04400700 1 --2- 172.21.15.153:0/2977832580 >> [v2:172.21.15.174:6828/1155541738,v1:172.21.15.174:6829/1155541738] conn(0x7fade406d410 0x7fade406f8d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fade8005950 tx=0x7fade800b410 comp rx=0 tx=0).stop 2024-03-21T10:08:07.057 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:07.055+0000 7fae04400700 1 -- 172.21.15.153:0/2977832580 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fadfc101640 msgr2=0x7fadfc100fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:08:07.057 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:07.055+0000 7fae04400700 1 --2- 172.21.15.153:0/2977832580 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fadfc101640 0x7fadfc100fa0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fadf0005b40 tx=0x7fadf000bd70 comp rx=0 tx=0).stop 2024-03-21T10:08:07.057 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:07.055+0000 7fae04400700 1 -- 172.21.15.153:0/2977832580 shutdown_connections 2024-03-21T10:08:07.057 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:07.055+0000 7fae04400700 1 --2- 172.21.15.153:0/2977832580 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fadfc101640 0x7fadfc100fa0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:08:07.057 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:07.055+0000 7fae04400700 1 --2- 172.21.15.153:0/2977832580 >> [v2:172.21.15.174:6828/1155541738,v1:172.21.15.174:6829/1155541738] conn(0x7fade406d410 0x7fade406f8d0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:08:07.057 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:07.055+0000 7fae04400700 1 --2- 172.21.15.153:0/2977832580 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fadfc103f70 0x7fadfc0ff5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:08:07.057 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:07.055+0000 7fae04400700 1 -- 172.21.15.153:0/2977832580 >> 172.21.15.153:0/2977832580 conn(0x7fadfc0fb050 msgr2=0x7fadfc0fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:08:07.057 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:07.055+0000 7fae04400700 1 -- 172.21.15.153:0/2977832580 shutdown_connections 2024-03-21T10:08:07.057 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:07.055+0000 7fae04400700 1 -- 172.21.15.153:0/2977832580 wait complete. 2024-03-21T10:08:07.236 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:07.234+0000 7f7837200700 1 -- 172.21.15.153:0/2357070433 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f78300737f0 msgr2=0x7f7830073c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:08:07.236 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:07.234+0000 7f7837200700 1 --2- 172.21.15.153:0/2357070433 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f78300737f0 0x7f7830073c70 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f781c009910 tx=0x7f781c009c20 comp rx=0 tx=0).stop 2024-03-21T10:08:07.236 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:07.234+0000 7f7837200700 1 -- 172.21.15.153:0/2357070433 shutdown_connections 2024-03-21T10:08:07.237 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:07.234+0000 7f7837200700 1 --2- 172.21.15.153:0/2357070433 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f78300737f0 0x7f7830073c70 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:08:07.237 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:07.234+0000 7f7837200700 1 --2- 172.21.15.153:0/2357070433 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7830074dc0 0x7f7830073220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:08:07.237 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:07.234+0000 7f7837200700 1 -- 172.21.15.153:0/2357070433 >> 172.21.15.153:0/2357070433 conn(0x7f78300fc110 msgr2=0x7f78300fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:08:07.237 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:07.234+0000 7f7837200700 1 -- 172.21.15.153:0/2357070433 shutdown_connections 2024-03-21T10:08:07.237 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:07.234+0000 7f7837200700 1 -- 172.21.15.153:0/2357070433 wait complete. 2024-03-21T10:08:07.238 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:07.235+0000 7f7837200700 1 Processor -- start 2024-03-21T10:08:07.238 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:07.235+0000 7f7837200700 1 -- start start 2024-03-21T10:08:07.238 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:07.236+0000 7f7837200700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f78300737f0 0x7f7830198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:08:07.238 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:07.236+0000 7f7837200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7830074dc0 0x7f7830198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:08:07.238 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:07.236+0000 7f7837200700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7830199170 con 0x7f7830074dc0 2024-03-21T10:08:07.238 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:07.236+0000 7f7837200700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78301992b0 con 0x7f78300737f0 2024-03-21T10:08:07.238 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:07.236+0000 7f782f400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7830074dc0 0x7f7830198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:08:07.238 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:07.236+0000 7f782f400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7830074dc0 0x7f7830198b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:47666/0 (socket says 172.21.15.153:47666) 2024-03-21T10:08:07.239 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:07.236+0000 7f782f400700 1 -- 172.21.15.153:0/970965055 learned_addr learned my addr 172.21.15.153:0/970965055 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:08:07.239 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:07.237+0000 7f782fe00700 1 --2- 172.21.15.153:0/970965055 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f78300737f0 0x7f7830198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:08:07.239 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:07.237+0000 7f782f400700 1 -- 172.21.15.153:0/970965055 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f78300737f0 msgr2=0x7f7830198610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:08:07.239 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:07.237+0000 7f782f400700 1 --2- 172.21.15.153:0/970965055 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f78300737f0 0x7f7830198610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:08:07.239 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:07.237+0000 7f782f400700 1 -- 172.21.15.153:0/970965055 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f781c0095f0 con 0x7f7830074dc0 2024-03-21T10:08:07.239 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:07.238+0000 7f782f400700 1 --2- 172.21.15.153:0/970965055 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7830074dc0 0x7f7830198b50 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f781c00bd70 tx=0x7f781c00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:08:07.240 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:07.238+0000 7f782cc00700 1 -- 172.21.15.153:0/970965055 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f781c01d070 con 0x7f7830074dc0 2024-03-21T10:08:07.240 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:07.238+0000 7f782cc00700 1 -- 172.21.15.153:0/970965055 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f781c00f460 con 0x7f7830074dc0 2024-03-21T10:08:07.240 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:07.238+0000 7f782cc00700 1 -- 172.21.15.153:0/970965055 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f781c021640 con 0x7f7830074dc0 2024-03-21T10:08:07.240 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:07.238+0000 7f7837200700 1 -- 172.21.15.153:0/970965055 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f783019dd00 con 0x7f7830074dc0 2024-03-21T10:08:07.240 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:07.238+0000 7f7837200700 1 -- 172.21.15.153:0/970965055 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f783019e220 con 0x7f7830074dc0 2024-03-21T10:08:07.241 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:07.240+0000 7f7837200700 1 -- 172.21.15.153:0/970965055 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f783010b9c0 con 0x7f7830074dc0 2024-03-21T10:08:07.242 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:07.241+0000 7f782cc00700 1 -- 172.21.15.153:0/970965055 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 24) v1 ==== 91295+0+0 (secure 0 0 0) 0x7f781c00fab0 con 0x7f7830074dc0 2024-03-21T10:08:07.244 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:07.242+0000 7f782cc00700 1 --2- 172.21.15.153:0/970965055 >> [v2:172.21.15.174:6828/1155541738,v1:172.21.15.174:6829/1155541738] conn(0x7f781406d450 0x7f781406f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:08:07.244 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:07.242+0000 7f782cc00700 1 -- 172.21.15.153:0/970965055 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f781c08e0e0 con 0x7f7830074dc0 2024-03-21T10:08:07.245 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:07.243+0000 7f782fe00700 1 --2- 172.21.15.153:0/970965055 >> [v2:172.21.15.174:6828/1155541738,v1:172.21.15.174:6829/1155541738] conn(0x7f781406d450 0x7f781406f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:08:07.246 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:07.245+0000 7f782fe00700 1 --2- 172.21.15.153:0/970965055 >> [v2:172.21.15.174:6828/1155541738,v1:172.21.15.174:6829/1155541738] conn(0x7f781406d450 0x7f781406f910 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7820006fd0 tx=0x7f7820008040 comp rx=0 tx=0).ready entity=mgr.24361 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:08:07.248 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:07.247+0000 7f782cc00700 1 -- 172.21.15.153:0/970965055 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185454 (secure 0 0 0) 0x7f781c05b340 con 0x7f7830074dc0 2024-03-21T10:08:07.542 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:07.540+0000 7f7837200700 1 -- 172.21.15.153:0/970965055 --> [v2:172.21.15.174:6828/1155541738,v1:172.21.15.174:6829/1155541738] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f78301080b0 con 0x7f781406d450 2024-03-21T10:08:07.584 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:07.583+0000 7f782cc00700 1 -- 172.21.15.153:0/970965055 <== mgr.24361 v2:172.21.15.174:6828/1155541738 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f78301080b0 con 0x7f781406d450 2024-03-21T10:08:07.585 INFO:teuthology.orchestra.run.smithi153.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-21T10:08:07.585 INFO:teuthology.orchestra.run.smithi153.stdout:alertmanager.smithi153 smithi153 *:9093,9094 running (8m) 21s ago 10m 13.8M - 0.25.0 c8568f914cd2 2cfcc225c7e6 2024-03-21T10:08:07.585 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi153 smithi153 running (11m) 21s ago 11m 7818k - 18.2.1 5be31c24972a 5b61994396a6 2024-03-21T10:08:07.585 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi174 smithi174 running (9m) 24s ago 9m 7738k - 18.2.1 5be31c24972a 4f7a5fc3ac28 2024-03-21T10:08:07.585 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi153 smithi153 running (10m) 21s ago 10m 6665k - 18.2.1 5be31c24972a 90a8b8ea03d1 2024-03-21T10:08:07.585 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi174 smithi174 running (9m) 24s ago 9m 6665k - 18.2.1 5be31c24972a a4c929e4dc77 2024-03-21T10:08:07.585 INFO:teuthology.orchestra.run.smithi153.stdout:grafana.smithi153 smithi153 *:3000 running (8m) 21s ago 10m 74.0M - 9.4.7 954c08fa6188 3af5800ecfef 2024-03-21T10:08:07.585 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.iuyluj smithi153 running (3m) 21s ago 3m 76.4M - 18.2.1 5be31c24972a 02f667cee979 2024-03-21T10:08:07.585 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.zipckr smithi153 running (3m) 21s ago 3m 66.6M - 18.2.1 5be31c24972a 68a853929f8f 2024-03-21T10:08:07.585 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.nhozag smithi174 running (3m) 24s ago 3m 57.9M - 18.2.1 5be31c24972a 2fee4000308f 2024-03-21T10:08:07.585 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.yyimst smithi174 running (3m) 24s ago 3m 75.3M - 18.2.1 5be31c24972a d78a729220d1 2024-03-21T10:08:07.585 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi153.tluxni smithi153 *:9283,8765,8443 running (12m) 21s ago 12m 334M - 18.2.1 5be31c24972a 4978543154ff 2024-03-21T10:08:07.586 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi174.qpcwat smithi174 *:8443,9283,8765 running (50s) 24s ago 9m 487M - 19.0.0-2231-g2edfee68 237e43c7a65f 2f7934875e11 2024-03-21T10:08:07.586 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi153 smithi153 running (12m) 21s ago 12m 49.2M 2048M 18.2.1 5be31c24972a 813963a95d69 2024-03-21T10:08:07.586 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi174 smithi174 running (9m) 24s ago 9m 39.1M 2048M 18.2.1 5be31c24972a 99204923eec6 2024-03-21T10:08:07.586 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi153 smithi153 *:9100 running (10m) 21s ago 10m 7852k - 1.5.0 0da6a335fe13 babbc29a0eaf 2024-03-21T10:08:07.586 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi174 smithi174 *:9100 running (9m) 24s ago 9m 7834k - 1.5.0 0da6a335fe13 86841435042a 2024-03-21T10:08:07.586 INFO:teuthology.orchestra.run.smithi153.stdout:osd.0 smithi153 running (8m) 21s ago 8m 68.9M 1217M 18.2.1 5be31c24972a 28912a2319b4 2024-03-21T10:08:07.586 INFO:teuthology.orchestra.run.smithi153.stdout:osd.1 smithi153 running (7m) 21s ago 7m 73.2M 1217M 18.2.1 5be31c24972a ca60681e2314 2024-03-21T10:08:07.586 INFO:teuthology.orchestra.run.smithi153.stdout:osd.2 smithi153 running (6m) 21s ago 6m 64.2M 1217M 18.2.1 5be31c24972a 8db799f8962a 2024-03-21T10:08:07.586 INFO:teuthology.orchestra.run.smithi153.stdout:osd.3 smithi174 running (6m) 24s ago 6m 72.8M 2241M 18.2.1 5be31c24972a dafc89e0658d 2024-03-21T10:08:07.586 INFO:teuthology.orchestra.run.smithi153.stdout:osd.4 smithi174 running (5m) 24s ago 5m 67.4M 2241M 18.2.1 5be31c24972a c4cd15701f05 2024-03-21T10:08:07.586 INFO:teuthology.orchestra.run.smithi153.stdout:osd.5 smithi174 running (5m) 24s ago 5m 70.0M 2241M 18.2.1 5be31c24972a 3b69886ce22e 2024-03-21T10:08:07.586 INFO:teuthology.orchestra.run.smithi153.stdout:prometheus.smithi153 smithi153 *:9095 starting - - - - 2024-03-21T10:08:07.588 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:07.586+0000 7f7837200700 1 -- 172.21.15.153:0/970965055 >> [v2:172.21.15.174:6828/1155541738,v1:172.21.15.174:6829/1155541738] conn(0x7f781406d450 msgr2=0x7f781406f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:08:07.588 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:07.586+0000 7f7837200700 1 --2- 172.21.15.153:0/970965055 >> [v2:172.21.15.174:6828/1155541738,v1:172.21.15.174:6829/1155541738] conn(0x7f781406d450 0x7f781406f910 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7820006fd0 tx=0x7f7820008040 comp rx=0 tx=0).stop 2024-03-21T10:08:07.588 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:07.587+0000 7f7837200700 1 -- 172.21.15.153:0/970965055 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7830074dc0 msgr2=0x7f7830198b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:08:07.588 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:07.587+0000 7f7837200700 1 --2- 172.21.15.153:0/970965055 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7830074dc0 0x7f7830198b50 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f781c00bd70 tx=0x7f781c00be50 comp rx=0 tx=0).stop 2024-03-21T10:08:07.589 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:07.587+0000 7f7837200700 1 -- 172.21.15.153:0/970965055 shutdown_connections 2024-03-21T10:08:07.589 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:07.587+0000 7f7837200700 1 --2- 172.21.15.153:0/970965055 >> [v2:172.21.15.174:6828/1155541738,v1:172.21.15.174:6829/1155541738] conn(0x7f781406d450 0x7f781406f910 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:08:07.589 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:07.587+0000 7f7837200700 1 --2- 172.21.15.153:0/970965055 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f78300737f0 0x7f7830198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:08:07.589 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:07.587+0000 7f7837200700 1 --2- 172.21.15.153:0/970965055 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7830074dc0 0x7f7830198b50 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:08:07.589 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:07.587+0000 7f7837200700 1 -- 172.21.15.153:0/970965055 >> 172.21.15.153:0/970965055 conn(0x7f78300fc110 msgr2=0x7f7830106990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:08:07.589 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:07.587+0000 7f7837200700 1 -- 172.21.15.153:0/970965055 shutdown_connections 2024-03-21T10:08:07.589 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:07.587+0000 7f7837200700 1 -- 172.21.15.153:0/970965055 wait complete. 2024-03-21T10:08:07.774 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:07.772+0000 7efe0bc00700 1 -- 172.21.15.153:0/3618546330 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efe04073790 msgr2=0x7efe04073c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:08:07.774 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:07.772+0000 7efe0bc00700 1 --2- 172.21.15.153:0/3618546330 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efe04073790 0x7efe04073c10 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7efdf0009910 tx=0x7efdf0009c20 comp rx=0 tx=0).stop 2024-03-21T10:08:07.774 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:07.772+0000 7efe0bc00700 1 -- 172.21.15.153:0/3618546330 shutdown_connections 2024-03-21T10:08:07.774 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:07.772+0000 7efe0bc00700 1 --2- 172.21.15.153:0/3618546330 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efe04073790 0x7efe04073c10 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:08:07.774 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:07.772+0000 7efe0bc00700 1 --2- 172.21.15.153:0/3618546330 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efe04074d60 0x7efe040731c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:08:07.774 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:07.772+0000 7efe0bc00700 1 -- 172.21.15.153:0/3618546330 >> 172.21.15.153:0/3618546330 conn(0x7efe040fc110 msgr2=0x7efe040fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:08:07.775 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:07.773+0000 7efe0bc00700 1 -- 172.21.15.153:0/3618546330 shutdown_connections 2024-03-21T10:08:07.775 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:07.773+0000 7efe0bc00700 1 -- 172.21.15.153:0/3618546330 wait complete. 2024-03-21T10:08:07.775 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:07.774+0000 7efe0bc00700 1 Processor -- start 2024-03-21T10:08:07.776 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:07.774+0000 7efe0bc00700 1 -- start start 2024-03-21T10:08:07.776 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:07.775+0000 7efe0bc00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efe04073790 0x7efe04198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:08:07.776 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:07.775+0000 7efe0bc00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efe04074d60 0x7efe04198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:08:07.776 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:07.775+0000 7efe0bc00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe04199080 con 0x7efe04073790 2024-03-21T10:08:07.776 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:07.775+0000 7efe0bc00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe041991c0 con 0x7efe04074d60 2024-03-21T10:08:07.777 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:07.775+0000 7efe09200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efe04073790 0x7efe04198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:08:07.777 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:07.775+0000 7efe09200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efe04073790 0x7efe04198520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:47676/0 (socket says 172.21.15.153:47676) 2024-03-21T10:08:07.777 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:07.775+0000 7efe09200700 1 -- 172.21.15.153:0/98986945 learned_addr learned my addr 172.21.15.153:0/98986945 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:08:07.777 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:07.775+0000 7efe03e00700 1 --2- 172.21.15.153:0/98986945 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efe04074d60 0x7efe04198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:08:07.777 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:07.775+0000 7efe09200700 1 -- 172.21.15.153:0/98986945 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efe04074d60 msgr2=0x7efe04198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:08:07.777 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:07.775+0000 7efe09200700 1 --2- 172.21.15.153:0/98986945 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efe04074d60 0x7efe04198a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:08:07.777 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:07.775+0000 7efe09200700 1 -- 172.21.15.153:0/98986945 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdf00095f0 con 0x7efe04073790 2024-03-21T10:08:07.778 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:07.775+0000 7efe03e00700 1 --2- 172.21.15.153:0/98986945 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efe04074d60 0x7efe04198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:08:07.778 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:07.776+0000 7efe09200700 1 --2- 172.21.15.153:0/98986945 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efe04073790 0x7efe04198520 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7efdf800d8d0 tx=0x7efdf800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:08:07.778 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:07.776+0000 7efe01600700 1 -- 172.21.15.153:0/98986945 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efdf8009940 con 0x7efe04073790 2024-03-21T10:08:07.778 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:07.776+0000 7efe01600700 1 -- 172.21.15.153:0/98986945 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7efdf8010460 con 0x7efe04073790 2024-03-21T10:08:07.778 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:07.776+0000 7efe0bc00700 1 -- 172.21.15.153:0/98986945 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe0419dc70 con 0x7efe04073790 2024-03-21T10:08:07.778 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:07.776+0000 7efe01600700 1 -- 172.21.15.153:0/98986945 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efdf800f5f0 con 0x7efe04073790 2024-03-21T10:08:07.779 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:07.777+0000 7efe0bc00700 1 -- 172.21.15.153:0/98986945 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe0419e1c0 con 0x7efe04073790 2024-03-21T10:08:07.780 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:07.779+0000 7efe01600700 1 -- 172.21.15.153:0/98986945 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 24) v1 ==== 91295+0+0 (secure 0 0 0) 0x7efdf80105d0 con 0x7efe04073790 2024-03-21T10:08:07.780 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:07.779+0000 7efe0bc00700 1 -- 172.21.15.153:0/98986945 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe0410b900 con 0x7efe04073790 2024-03-21T10:08:07.781 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:07.780+0000 7efe01600700 1 --2- 172.21.15.153:0/98986945 >> [v2:172.21.15.174:6828/1155541738,v1:172.21.15.174:6829/1155541738] conn(0x7efdec06d410 0x7efdec06f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:08:07.781 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:07.780+0000 7efe01600700 1 -- 172.21.15.153:0/98986945 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7efdf808c730 con 0x7efe04073790 2024-03-21T10:08:07.782 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:07.781+0000 7efe03e00700 1 --2- 172.21.15.153:0/98986945 >> [v2:172.21.15.174:6828/1155541738,v1:172.21.15.174:6829/1155541738] conn(0x7efdec06d410 0x7efdec06f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:08:07.785 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:07.783+0000 7efe03e00700 1 --2- 172.21.15.153:0/98986945 >> [v2:172.21.15.174:6828/1155541738,v1:172.21.15.174:6829/1155541738] conn(0x7efdec06d410 0x7efdec06f8d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7efdf0006010 tx=0x7efdf000b560 comp rx=0 tx=0).ready entity=mgr.24361 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:08:07.786 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:07.785+0000 7efe01600700 1 -- 172.21.15.153:0/98986945 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185454 (secure 0 0 0) 0x7efdf80598f0 con 0x7efe04073790 2024-03-21T10:08:08.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:07 smithi153 ceph-mon[29117]: from='client.14646 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:08:08.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:07 smithi153 ceph-mon[29117]: pgmap v21: 65 pgs: 65 active+clean; 159 MiB data, 720 MiB used, 536 GiB / 536 GiB avail; 1.9 MiB/s rd, 3.2 MiB/s wr, 176 op/s 2024-03-21T10:08:08.195 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:08.193+0000 7efe0bc00700 1 -- 172.21.15.153:0/98986945 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7efe0410bb20 con 0x7efe04073790 2024-03-21T10:08:08.196 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:08.194+0000 7efe01600700 1 -- 172.21.15.153:0/98986945 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+766 (secure 0 0 0) 0x7efdf8059480 con 0x7efe04073790 2024-03-21T10:08:08.196 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:08:08.196 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": { 2024-03-21T10:08:08.196 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-03-21T10:08:08.196 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:08:08.196 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": { 2024-03-21T10:08:08.196 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-03-21T10:08:08.196 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 1 2024-03-21T10:08:08.197 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:08:08.197 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": { 2024-03-21T10:08:08.197 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-03-21T10:08:08.197 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:08:08.197 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": { 2024-03-21T10:08:08.197 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-03-21T10:08:08.197 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:08:08.197 INFO:teuthology.orchestra.run.smithi153.stdout: "overall": { 2024-03-21T10:08:08.197 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 13, 2024-03-21T10:08:08.197 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 1 2024-03-21T10:08:08.197 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-03-21T10:08:08.197 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:08:08.199 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:08.198+0000 7efe0bc00700 1 -- 172.21.15.153:0/98986945 >> [v2:172.21.15.174:6828/1155541738,v1:172.21.15.174:6829/1155541738] conn(0x7efdec06d410 msgr2=0x7efdec06f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:08:08.200 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:08.198+0000 7efe0bc00700 1 --2- 172.21.15.153:0/98986945 >> [v2:172.21.15.174:6828/1155541738,v1:172.21.15.174:6829/1155541738] conn(0x7efdec06d410 0x7efdec06f8d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7efdf0006010 tx=0x7efdf000b560 comp rx=0 tx=0).stop 2024-03-21T10:08:08.200 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:08.198+0000 7efe0bc00700 1 -- 172.21.15.153:0/98986945 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efe04073790 msgr2=0x7efe04198520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:08:08.200 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:08.198+0000 7efe0bc00700 1 --2- 172.21.15.153:0/98986945 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efe04073790 0x7efe04198520 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7efdf800d8d0 tx=0x7efdf800dc90 comp rx=0 tx=0).stop 2024-03-21T10:08:08.200 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:08.198+0000 7efe0bc00700 1 -- 172.21.15.153:0/98986945 shutdown_connections 2024-03-21T10:08:08.200 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:08.198+0000 7efe0bc00700 1 --2- 172.21.15.153:0/98986945 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efe04073790 0x7efe04198520 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:08:08.201 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:08.198+0000 7efe0bc00700 1 --2- 172.21.15.153:0/98986945 >> [v2:172.21.15.174:6828/1155541738,v1:172.21.15.174:6829/1155541738] conn(0x7efdec06d410 0x7efdec06f8d0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:08:08.201 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:08.198+0000 7efe0bc00700 1 --2- 172.21.15.153:0/98986945 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efe04074d60 0x7efe04198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:08:08.201 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:08.198+0000 7efe0bc00700 1 -- 172.21.15.153:0/98986945 >> 172.21.15.153:0/98986945 conn(0x7efe040fc110 msgr2=0x7efe041069f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:08:08.201 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:08.199+0000 7efe0bc00700 1 -- 172.21.15.153:0/98986945 shutdown_connections 2024-03-21T10:08:08.201 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:08.199+0000 7efe0bc00700 1 -- 172.21.15.153:0/98986945 wait complete. 2024-03-21T10:08:08.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:07 smithi174 ceph-mon[37496]: from='client.14646 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:08:08.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:07 smithi174 ceph-mon[37496]: pgmap v21: 65 pgs: 65 active+clean; 159 MiB data, 720 MiB used, 536 GiB / 536 GiB avail; 1.9 MiB/s rd, 3.2 MiB/s wr, 176 op/s 2024-03-21T10:08:08.394 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:08.392+0000 7f3e18600700 1 -- 172.21.15.153:0/3165045852 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3e10101640 msgr2=0x7f3e10103a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:08:08.394 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:08.392+0000 7f3e18600700 1 --2- 172.21.15.153:0/3165045852 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3e10101640 0x7f3e10103a30 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f3e04009910 tx=0x7f3e04009c20 comp rx=0 tx=0).stop 2024-03-21T10:08:08.394 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:08.392+0000 7f3e18600700 1 -- 172.21.15.153:0/3165045852 shutdown_connections 2024-03-21T10:08:08.394 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:08.392+0000 7f3e18600700 1 --2- 172.21.15.153:0/3165045852 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3e10103f70 0x7f3e10106360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:08:08.394 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:08.392+0000 7f3e18600700 1 --2- 172.21.15.153:0/3165045852 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3e10101640 0x7f3e10103a30 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:08:08.395 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:08.392+0000 7f3e18600700 1 -- 172.21.15.153:0/3165045852 >> 172.21.15.153:0/3165045852 conn(0x7f3e100fb030 msgr2=0x7f3e100fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:08:08.395 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:08.392+0000 7f3e18600700 1 -- 172.21.15.153:0/3165045852 shutdown_connections 2024-03-21T10:08:08.395 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:08.393+0000 7f3e18600700 1 -- 172.21.15.153:0/3165045852 wait complete. 2024-03-21T10:08:08.395 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:08.394+0000 7f3e18600700 1 Processor -- start 2024-03-21T10:08:08.395 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:08.394+0000 7f3e18600700 1 -- start start 2024-03-21T10:08:08.396 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:08.394+0000 7f3e18600700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3e10101640 0x7f3e10198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:08:08.396 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:08.394+0000 7f3e18600700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3e10103f70 0x7f3e10198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:08:08.396 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:08.394+0000 7f3e18600700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e101990e0 con 0x7f3e10103f70 2024-03-21T10:08:08.396 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:08.394+0000 7f3e18600700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e10199220 con 0x7f3e10101640 2024-03-21T10:08:08.396 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:08.395+0000 7f3e15200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3e10103f70 0x7f3e10198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:08:08.396 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:08.395+0000 7f3e15200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3e10103f70 0x7f3e10198ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:47704/0 (socket says 172.21.15.153:47704) 2024-03-21T10:08:08.397 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:08.395+0000 7f3e15200700 1 -- 172.21.15.153:0/3115687194 learned_addr learned my addr 172.21.15.153:0/3115687194 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:08:08.397 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:08.395+0000 7f3e15c00700 1 --2- 172.21.15.153:0/3115687194 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3e10101640 0x7f3e10198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:08:08.397 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:08.395+0000 7f3e15200700 1 -- 172.21.15.153:0/3115687194 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3e10101640 msgr2=0x7f3e10198580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:08:08.397 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:08.395+0000 7f3e15200700 1 --2- 172.21.15.153:0/3115687194 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3e10101640 0x7f3e10198580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:08:08.397 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:08.395+0000 7f3e15200700 1 -- 172.21.15.153:0/3115687194 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e040095f0 con 0x7f3e10103f70 2024-03-21T10:08:08.398 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:08.396+0000 7f3e15200700 1 --2- 172.21.15.153:0/3115687194 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3e10103f70 0x7f3e10198ac0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc00b4b0 tx=0x7f3dfc00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:08:08.398 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:08.396+0000 7f3e0e000700 1 -- 172.21.15.153:0/3115687194 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3dfc011800 con 0x7f3e10103f70 2024-03-21T10:08:08.398 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:08.396+0000 7f3e0e000700 1 -- 172.21.15.153:0/3115687194 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f3dfc011e10 con 0x7f3e10103f70 2024-03-21T10:08:08.398 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:08.396+0000 7f3e0e000700 1 -- 172.21.15.153:0/3115687194 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3dfc00f2e0 con 0x7f3e10103f70 2024-03-21T10:08:08.398 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:08.396+0000 7f3e18600700 1 -- 172.21.15.153:0/3115687194 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e1019dcd0 con 0x7f3e10103f70 2024-03-21T10:08:08.398 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:08.396+0000 7f3e18600700 1 -- 172.21.15.153:0/3115687194 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e1019e2a0 con 0x7f3e10103f70 2024-03-21T10:08:08.399 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:08.398+0000 7f3e18600700 1 -- 172.21.15.153:0/3115687194 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e100fcc20 con 0x7f3e10103f70 2024-03-21T10:08:08.401 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:08.399+0000 7f3e0e000700 1 -- 172.21.15.153:0/3115687194 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 24) v1 ==== 91295+0+0 (secure 0 0 0) 0x7f3dfc010380 con 0x7f3e10103f70 2024-03-21T10:08:08.401 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:08.400+0000 7f3e0e000700 1 --2- 172.21.15.153:0/3115687194 >> [v2:172.21.15.174:6828/1155541738,v1:172.21.15.174:6829/1155541738] conn(0x7f3df806d410 0x7f3df806f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:08:08.402 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:08.400+0000 7f3e0e000700 1 -- 172.21.15.153:0/3115687194 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f3dfc08c8c0 con 0x7f3e10103f70 2024-03-21T10:08:08.403 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:08.401+0000 7f3e15c00700 1 --2- 172.21.15.153:0/3115687194 >> [v2:172.21.15.174:6828/1155541738,v1:172.21.15.174:6829/1155541738] conn(0x7f3df806d410 0x7f3df806f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:08:08.404 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:08.403+0000 7f3e15c00700 1 --2- 172.21.15.153:0/3115687194 >> [v2:172.21.15.174:6828/1155541738,v1:172.21.15.174:6829/1155541738] conn(0x7f3df806d410 0x7f3df806f8d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f3e04006010 tx=0x7f3e0400b540 comp rx=0 tx=0).ready entity=mgr.24361 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:08:08.408 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:08.406+0000 7f3e0e000700 1 -- 172.21.15.153:0/3115687194 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185454 (secure 0 0 0) 0x7f3dfc059af0 con 0x7f3e10103f70 2024-03-21T10:08:08.801 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:08.799+0000 7f3e18600700 1 -- 172.21.15.153:0/3115687194 --> [v2:172.21.15.174:6828/1155541738,v1:172.21.15.174:6829/1155541738] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3e100611d0 con 0x7f3df806d410 2024-03-21T10:08:08.804 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:08.803+0000 7f3e0e000700 1 -- 172.21.15.153:0/3115687194 <== mgr.24361 v2:172.21.15.174:6828/1155541738 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7f3e100611d0 con 0x7f3df806d410 2024-03-21T10:08:08.805 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:08:08.805 INFO:teuthology.orchestra.run.smithi153.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232", 2024-03-21T10:08:08.805 INFO:teuthology.orchestra.run.smithi153.stdout: "in_progress": true, 2024-03-21T10:08:08.805 INFO:teuthology.orchestra.run.smithi153.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-03-21T10:08:08.805 INFO:teuthology.orchestra.run.smithi153.stdout: "services_complete": [], 2024-03-21T10:08:08.805 INFO:teuthology.orchestra.run.smithi153.stdout: "progress": "1/2 daemons upgraded", 2024-03-21T10:08:08.805 INFO:teuthology.orchestra.run.smithi153.stdout: "message": "Currently upgrading mgr daemons", 2024-03-21T10:08:08.805 INFO:teuthology.orchestra.run.smithi153.stdout: "is_paused": false 2024-03-21T10:08:08.805 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:08:08.809 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:08.807+0000 7f3e18600700 1 -- 172.21.15.153:0/3115687194 >> [v2:172.21.15.174:6828/1155541738,v1:172.21.15.174:6829/1155541738] conn(0x7f3df806d410 msgr2=0x7f3df806f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:08:08.809 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:08.807+0000 7f3e18600700 1 --2- 172.21.15.153:0/3115687194 >> [v2:172.21.15.174:6828/1155541738,v1:172.21.15.174:6829/1155541738] conn(0x7f3df806d410 0x7f3df806f8d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f3e04006010 tx=0x7f3e0400b540 comp rx=0 tx=0).stop 2024-03-21T10:08:08.809 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:08.807+0000 7f3e18600700 1 -- 172.21.15.153:0/3115687194 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3e10103f70 msgr2=0x7f3e10198ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:08:08.809 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:08.807+0000 7f3e18600700 1 --2- 172.21.15.153:0/3115687194 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3e10103f70 0x7f3e10198ac0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc00b4b0 tx=0x7f3dfc00b870 comp rx=0 tx=0).stop 2024-03-21T10:08:08.809 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:08.808+0000 7f3e18600700 1 -- 172.21.15.153:0/3115687194 shutdown_connections 2024-03-21T10:08:08.810 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:08.808+0000 7f3e18600700 1 --2- 172.21.15.153:0/3115687194 >> [v2:172.21.15.174:6828/1155541738,v1:172.21.15.174:6829/1155541738] conn(0x7f3df806d410 0x7f3df806f8d0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:08:08.810 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:08.808+0000 7f3e18600700 1 --2- 172.21.15.153:0/3115687194 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3e10101640 0x7f3e10198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:08:08.810 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:08.808+0000 7f3e18600700 1 --2- 172.21.15.153:0/3115687194 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3e10103f70 0x7f3e10198ac0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:08:08.810 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:08.808+0000 7f3e18600700 1 -- 172.21.15.153:0/3115687194 >> 172.21.15.153:0/3115687194 conn(0x7f3e100fb030 msgr2=0x7f3e100ffcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:08:08.810 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:08.808+0000 7f3e18600700 1 -- 172.21.15.153:0/3115687194 shutdown_connections 2024-03-21T10:08:08.810 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:08.808+0000 7f3e18600700 1 -- 172.21.15.153:0/3115687194 wait complete. 2024-03-21T10:08:09.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:08 smithi153 ceph-mon[29117]: from='client.14650 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:08:09.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:08 smithi153 ceph-mon[29117]: from='client.14654 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:08:09.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:08 smithi153 ceph-mon[29117]: from='client.? 172.21.15.153:0/98986945' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:08:09.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:08 smithi174 ceph-mon[37496]: from='client.14650 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:08:09.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:08 smithi174 ceph-mon[37496]: from='client.14654 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:08:09.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:08 smithi174 ceph-mon[37496]: from='client.? 172.21.15.153:0/98986945' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:08:10.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:09 smithi174 ceph-mon[37496]: pgmap v22: 65 pgs: 65 active+clean; 160 MiB data, 722 MiB used, 536 GiB / 536 GiB avail; 1.3 MiB/s rd, 2.6 MiB/s wr, 168 op/s 2024-03-21T10:08:10.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:09 smithi174 ceph-mon[37496]: from='client.14662 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:08:10.231 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:09 smithi153 ceph-mon[29117]: pgmap v22: 65 pgs: 65 active+clean; 160 MiB data, 722 MiB used, 536 GiB / 536 GiB avail; 1.3 MiB/s rd, 2.6 MiB/s wr, 168 op/s 2024-03-21T10:08:10.231 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:09 smithi153 ceph-mon[29117]: from='client.14662 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:08:12.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:11 smithi153 ceph-mon[29117]: pgmap v23: 65 pgs: 65 active+clean; 160 MiB data, 722 MiB used, 536 GiB / 536 GiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 136 op/s 2024-03-21T10:08:12.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:11 smithi174 ceph-mon[37496]: pgmap v23: 65 pgs: 65 active+clean; 160 MiB data, 722 MiB used, 536 GiB / 536 GiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 136 op/s 2024-03-21T10:08:14.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:13 smithi153 ceph-mon[29117]: pgmap v24: 65 pgs: 65 active+clean; 168 MiB data, 757 MiB used, 536 GiB / 536 GiB avail; 1.8 MiB/s rd, 2.8 MiB/s wr, 192 op/s 2024-03-21T10:08:14.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:13 smithi174 ceph-mon[37496]: pgmap v24: 65 pgs: 65 active+clean; 168 MiB data, 757 MiB used, 536 GiB / 536 GiB avail; 1.8 MiB/s rd, 2.8 MiB/s wr, 192 op/s 2024-03-21T10:08:15.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:15 smithi174 ceph-mon[37496]: pgmap v25: 65 pgs: 65 active+clean; 168 MiB data, 757 MiB used, 536 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 132 op/s 2024-03-21T10:08:15.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:15 smithi153 ceph-mon[29117]: pgmap v25: 65 pgs: 65 active+clean; 168 MiB data, 757 MiB used, 536 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 132 op/s 2024-03-21T10:08:16.404 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:16 smithi153 ceph-mon[29117]: from='mgr.24361 172.21.15.174:0/1831991646' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:08:16.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:16 smithi174 ceph-mon[37496]: from='mgr.24361 172.21.15.174:0/1831991646' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:08:17.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:17 smithi174 ceph-mon[37496]: pgmap v26: 65 pgs: 65 active+clean; 173 MiB data, 750 MiB used, 536 GiB / 536 GiB avail; 1.4 MiB/s rd, 2.1 MiB/s wr, 181 op/s 2024-03-21T10:08:17.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:17 smithi174 ceph-mon[37496]: from='mgr.24361 ' entity='mgr.smithi174.qpcwat' 2024-03-21T10:08:17.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:17 smithi174 ceph-mon[37496]: from='mgr.24361 ' entity='mgr.smithi174.qpcwat' 2024-03-21T10:08:17.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:17 smithi174 ceph-mon[37496]: from='mgr.24361 172.21.15.174:0/1831991646' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:08:18.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:17 smithi153 ceph-mon[29117]: pgmap v26: 65 pgs: 65 active+clean; 173 MiB data, 750 MiB used, 536 GiB / 536 GiB avail; 1.4 MiB/s rd, 2.1 MiB/s wr, 181 op/s 2024-03-21T10:08:18.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:17 smithi153 ceph-mon[29117]: from='mgr.24361 ' entity='mgr.smithi174.qpcwat' 2024-03-21T10:08:18.058 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:17 smithi153 ceph-mon[29117]: from='mgr.24361 ' entity='mgr.smithi174.qpcwat' 2024-03-21T10:08:18.058 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:17 smithi153 ceph-mon[29117]: from='mgr.24361 172.21.15.174:0/1831991646' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:08:19.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:19 smithi174 ceph-mon[37496]: pgmap v27: 65 pgs: 65 active+clean; 175 MiB data, 757 MiB used, 536 GiB / 536 GiB avail; 903 KiB/s rd, 1.4 MiB/s wr, 138 op/s 2024-03-21T10:08:19.970 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:19 smithi153 ceph-mon[29117]: pgmap v27: 65 pgs: 65 active+clean; 175 MiB data, 757 MiB used, 536 GiB / 536 GiB avail; 903 KiB/s rd, 1.4 MiB/s wr, 138 op/s 2024-03-21T10:08:21.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:21 smithi153 ceph-mon[29117]: pgmap v28: 65 pgs: 65 active+clean; 175 MiB data, 757 MiB used, 536 GiB / 536 GiB avail; 903 KiB/s rd, 1.3 MiB/s wr, 121 op/s 2024-03-21T10:08:21.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:21 smithi174 ceph-mon[37496]: pgmap v28: 65 pgs: 65 active+clean; 175 MiB data, 757 MiB used, 536 GiB / 536 GiB avail; 903 KiB/s rd, 1.3 MiB/s wr, 121 op/s 2024-03-21T10:08:23.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:23 smithi153 ceph-mon[29117]: pgmap v29: 65 pgs: 65 active+clean; 184 MiB data, 776 MiB used, 536 GiB / 536 GiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 227 op/s 2024-03-21T10:08:23.953 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:23 smithi174 ceph-mon[37496]: pgmap v29: 65 pgs: 65 active+clean; 184 MiB data, 776 MiB used, 536 GiB / 536 GiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 227 op/s 2024-03-21T10:08:25.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:25 smithi153 ceph-mon[29117]: from='mgr.24361 ' entity='mgr.smithi174.qpcwat' 2024-03-21T10:08:25.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:25 smithi153 ceph-mon[29117]: from='mgr.24361 ' entity='mgr.smithi174.qpcwat' 2024-03-21T10:08:25.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:25 smithi153 ceph-mon[29117]: pgmap v30: 65 pgs: 65 active+clean; 184 MiB data, 776 MiB used, 536 GiB / 536 GiB avail; 859 KiB/s rd, 1.4 MiB/s wr, 171 op/s 2024-03-21T10:08:25.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:25 smithi174 ceph-mon[37496]: from='mgr.24361 ' entity='mgr.smithi174.qpcwat' 2024-03-21T10:08:25.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:25 smithi174 ceph-mon[37496]: from='mgr.24361 ' entity='mgr.smithi174.qpcwat' 2024-03-21T10:08:25.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:25 smithi174 ceph-mon[37496]: pgmap v30: 65 pgs: 65 active+clean; 184 MiB data, 776 MiB used, 536 GiB / 536 GiB avail; 859 KiB/s rd, 1.4 MiB/s wr, 171 op/s 2024-03-21T10:08:27.899 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:27 smithi153 ceph-mon[29117]: pgmap v31: 65 pgs: 65 active+clean; 198 MiB data, 802 MiB used, 536 GiB / 536 GiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 288 op/s 2024-03-21T10:08:27.899 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:27 smithi153 ceph-mon[29117]: from='mgr.24361 ' entity='mgr.smithi174.qpcwat' 2024-03-21T10:08:27.899 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:27 smithi153 ceph-mon[29117]: from='mgr.24361 ' entity='mgr.smithi174.qpcwat' 2024-03-21T10:08:27.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:27 smithi174 ceph-mon[37496]: pgmap v31: 65 pgs: 65 active+clean; 198 MiB data, 802 MiB used, 536 GiB / 536 GiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 288 op/s 2024-03-21T10:08:27.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:27 smithi174 ceph-mon[37496]: from='mgr.24361 ' entity='mgr.smithi174.qpcwat' 2024-03-21T10:08:27.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:27 smithi174 ceph-mon[37496]: from='mgr.24361 ' entity='mgr.smithi174.qpcwat' 2024-03-21T10:08:28.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:28 smithi174 ceph-mon[37496]: Standby manager daemon smithi153.tluxni restarted 2024-03-21T10:08:28.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:28 smithi174 ceph-mon[37496]: Standby manager daemon smithi153.tluxni started 2024-03-21T10:08:28.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:28 smithi174 ceph-mon[37496]: from='mgr.? 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi153.tluxni/crt"}]: dispatch 2024-03-21T10:08:28.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:28 smithi174 ceph-mon[37496]: from='mgr.? 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-21T10:08:28.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:28 smithi174 ceph-mon[37496]: from='mgr.? 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi153.tluxni/key"}]: dispatch 2024-03-21T10:08:28.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:28 smithi174 ceph-mon[37496]: from='mgr.? 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-21T10:08:29.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:28 smithi153 ceph-mon[29117]: Standby manager daemon smithi153.tluxni restarted 2024-03-21T10:08:29.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:28 smithi153 ceph-mon[29117]: Standby manager daemon smithi153.tluxni started 2024-03-21T10:08:29.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:28 smithi153 ceph-mon[29117]: from='mgr.? 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi153.tluxni/crt"}]: dispatch 2024-03-21T10:08:29.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:28 smithi153 ceph-mon[29117]: from='mgr.? 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-21T10:08:29.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:28 smithi153 ceph-mon[29117]: from='mgr.? 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi153.tluxni/key"}]: dispatch 2024-03-21T10:08:29.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:28 smithi153 ceph-mon[29117]: from='mgr.? 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-21T10:08:29.790 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:29 smithi153 ceph-mon[29117]: pgmap v32: 65 pgs: 65 active+clean; 198 MiB data, 802 MiB used, 536 GiB / 536 GiB avail; 995 KiB/s rd, 2.2 MiB/s wr, 239 op/s 2024-03-21T10:08:29.791 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:29 smithi153 ceph-mon[29117]: mgrmap e25: smithi174.qpcwat(active, since 58s), standbys: smithi153.tluxni 2024-03-21T10:08:29.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:29 smithi174 ceph-mon[37496]: pgmap v32: 65 pgs: 65 active+clean; 198 MiB data, 802 MiB used, 536 GiB / 536 GiB avail; 995 KiB/s rd, 2.2 MiB/s wr, 239 op/s 2024-03-21T10:08:29.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:29 smithi174 ceph-mon[37496]: mgrmap e25: smithi174.qpcwat(active, since 58s), standbys: smithi153.tluxni 2024-03-21T10:08:31.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:31 smithi153 ceph-mon[29117]: pgmap v33: 65 pgs: 65 active+clean; 198 MiB data, 802 MiB used, 536 GiB / 536 GiB avail; 808 KiB/s rd, 1.9 MiB/s wr, 222 op/s 2024-03-21T10:08:31.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:31 smithi153 ceph-mon[29117]: from='mgr.24361 172.21.15.174:0/1831991646' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:08:31.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:31 smithi174 ceph-mon[37496]: pgmap v33: 65 pgs: 65 active+clean; 198 MiB data, 802 MiB used, 536 GiB / 536 GiB avail; 808 KiB/s rd, 1.9 MiB/s wr, 222 op/s 2024-03-21T10:08:31.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:31 smithi174 ceph-mon[37496]: from='mgr.24361 172.21.15.174:0/1831991646' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:08:33.168 INFO:tasks.workunit.client.0.smithi153.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-03-21T10:08:33.190 INFO:tasks.workunit.client.0.smithi153.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-03-21T10:08:33.190 INFO:tasks.workunit.client.0.smithi153.stderr:+ make 2024-03-21T10:08:33.286 INFO:tasks.workunit.client.0.smithi153.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-03-21T10:08:34.058 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:33 smithi153 ceph-mon[29117]: pgmap v34: 65 pgs: 65 active+clean; 214 MiB data, 828 MiB used, 536 GiB / 536 GiB avail; 1.5 MiB/s rd, 3.3 MiB/s wr, 296 op/s 2024-03-21T10:08:34.058 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:33 smithi153 ceph-mon[29117]: from='mgr.24361 ' entity='mgr.smithi174.qpcwat' 2024-03-21T10:08:34.058 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:33 smithi153 ceph-mon[29117]: from='mgr.24361 ' entity='mgr.smithi174.qpcwat' 2024-03-21T10:08:34.058 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:33 smithi153 ceph-mon[29117]: from='mgr.24361 172.21.15.174:0/1831991646' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:08:34.058 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:33 smithi153 ceph-mon[29117]: from='mgr.24361 172.21.15.174:0/1831991646' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:08:34.058 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:33 smithi153 ceph-mon[29117]: from='mgr.24361 ' entity='mgr.smithi174.qpcwat' 2024-03-21T10:08:34.058 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:33 smithi153 ceph-mon[29117]: from='mgr.24361 172.21.15.174:0/1831991646' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:08:34.058 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:33 smithi153 ceph-mon[29117]: from='mgr.24361 172.21.15.174:0/1831991646' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "mgr fail", "who": "smithi174.qpcwat"}]: dispatch 2024-03-21T10:08:34.059 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:33 smithi153 ceph-mon[29117]: from='mgr.24361 ' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "mgr fail", "who": "smithi174.qpcwat"}]: dispatch 2024-03-21T10:08:34.059 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:33 smithi153 ceph-mon[29117]: osdmap e49: 6 total, 6 up, 6 in 2024-03-21T10:08:34.059 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:33 smithi153 ceph-mon[29117]: from='mgr.24361 ' entity='mgr.smithi174.qpcwat' cmd='[{"prefix": "mgr fail", "who": "smithi174.qpcwat"}]': finished 2024-03-21T10:08:34.059 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:33 smithi153 ceph-mon[29117]: mgrmap e26: smithi153.tluxni(active, starting, since 0.0152174s) 2024-03-21T10:08:34.059 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:33 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mon metadata", "id": "smithi153"}]: dispatch 2024-03-21T10:08:34.059 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:33 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mon metadata", "id": "smithi174"}]: dispatch 2024-03-21T10:08:34.059 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:33 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi153.iuyluj"}]: dispatch 2024-03-21T10:08:34.059 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:33 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi153.zipckr"}]: dispatch 2024-03-21T10:08:34.059 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:33 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi174.yyimst"}]: dispatch 2024-03-21T10:08:34.059 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:33 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi174.nhozag"}]: dispatch 2024-03-21T10:08:34.059 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:33 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mgr metadata", "who": "smithi153.tluxni", "id": "smithi153.tluxni"}]: dispatch 2024-03-21T10:08:34.060 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:33 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T10:08:34.060 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:33 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T10:08:34.060 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:33 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T10:08:34.060 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:33 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T10:08:34.060 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:33 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T10:08:34.060 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:33 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T10:08:34.060 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:33 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-21T10:08:34.060 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:33 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-21T10:08:34.060 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:33 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-21T10:08:34.060 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:33 smithi153 ceph-mon[29117]: Manager daemon smithi153.tluxni is now available 2024-03-21T10:08:34.060 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:33 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:08:34.060 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:33 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:08:34.060 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:33 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi153.tluxni/mirror_snapshot_schedule"}]: dispatch 2024-03-21T10:08:34.060 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:33 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi153.tluxni/trash_purge_schedule"}]: dispatch 2024-03-21T10:08:34.164 INFO:tasks.workunit.client.0.smithi153.stderr:++ readlink -f fsstress 2024-03-21T10:08:34.169 INFO:tasks.workunit.client.0.smithi153.stderr:+ BIN=/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-03-21T10:08:34.169 INFO:tasks.workunit.client.0.smithi153.stderr:+ popd 2024-03-21T10:08:34.172 INFO:tasks.workunit.client.0.smithi153.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-03-21T10:08:34.172 INFO:tasks.workunit.client.0.smithi153.stderr:+ popd 2024-03-21T10:08:34.174 INFO:tasks.workunit.client.0.smithi153.stdout:~/cephtest/mnt.0/client.0/tmp 2024-03-21T10:08:34.174 INFO:tasks.workunit.client.0.smithi153.stderr:++ mktemp -d -p . 2024-03-21T10:08:34.179 INFO:tasks.workunit.client.0.smithi153.stderr:+ T=./tmp.2EStReJkf8 2024-03-21T10:08:34.179 INFO:tasks.workunit.client.0.smithi153.stderr:+ /home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.2EStReJkf8 -l 1 -n 1000 -p 10 -v 2024-03-21T10:08:34.186 INFO:tasks.workunit.client.0.smithi153.stdout:seed = 1711198680 2024-03-21T10:08:34.192 INFO:tasks.workunit.client.0.smithi153.stdout:0/0: chown . 5604 1 2024-03-21T10:08:34.193 INFO:tasks.workunit.client.0.smithi153.stdout:1/0: dwrite - no filename 2024-03-21T10:08:34.193 INFO:tasks.workunit.client.0.smithi153.stdout:1/1: write - no filename 2024-03-21T10:08:34.194 INFO:tasks.workunit.client.0.smithi153.stdout:1/2: link - no file 2024-03-21T10:08:34.194 INFO:tasks.workunit.client.0.smithi153.stdout:1/3: rename - no filename 2024-03-21T10:08:34.194 INFO:tasks.workunit.client.0.smithi153.stdout:1/4: rename - no filename 2024-03-21T10:08:34.194 INFO:tasks.workunit.client.0.smithi153.stdout:0/1: mknod c0 0 2024-03-21T10:08:34.196 INFO:tasks.workunit.client.0.smithi153.stdout:1/5: getdents . 0 2024-03-21T10:08:34.196 INFO:tasks.workunit.client.0.smithi153.stdout:1/6: dread - no filename 2024-03-21T10:08:34.196 INFO:tasks.workunit.client.0.smithi153.stdout:1/7: write - no filename 2024-03-21T10:08:34.197 INFO:tasks.workunit.client.0.smithi153.stdout:0/2: creat f1 x:0 0 0 2024-03-21T10:08:34.197 INFO:tasks.workunit.client.0.smithi153.stdout:2/0: rename - no filename 2024-03-21T10:08:34.198 INFO:tasks.workunit.client.0.smithi153.stdout:0/3: dread - f1 zero size 2024-03-21T10:08:34.199 INFO:tasks.workunit.client.0.smithi153.stdout:1/8: creat f0 x:0 0 0 2024-03-21T10:08:34.199 INFO:tasks.workunit.client.0.smithi153.stdout:0/4: dread - f1 zero size 2024-03-21T10:08:34.201 INFO:tasks.workunit.client.0.smithi153.stdout:1/9: symlink l1 0 2024-03-21T10:08:34.201 INFO:tasks.workunit.client.0.smithi153.stdout:0/5: dread - f1 zero size 2024-03-21T10:08:34.201 INFO:tasks.workunit.client.0.smithi153.stdout:3/0: write - no filename 2024-03-21T10:08:34.201 INFO:tasks.workunit.client.0.smithi153.stdout:3/1: rename - no filename 2024-03-21T10:08:34.201 INFO:tasks.workunit.client.0.smithi153.stdout:3/2: dread - no filename 2024-03-21T10:08:34.201 INFO:tasks.workunit.client.0.smithi153.stdout:3/3: write - no filename 2024-03-21T10:08:34.201 INFO:tasks.workunit.client.0.smithi153.stdout:3/4: dwrite - no filename 2024-03-21T10:08:34.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:33 smithi174 ceph-mon[37496]: pgmap v34: 65 pgs: 65 active+clean; 214 MiB data, 828 MiB used, 536 GiB / 536 GiB avail; 1.5 MiB/s rd, 3.3 MiB/s wr, 296 op/s 2024-03-21T10:08:34.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:33 smithi174 ceph-mon[37496]: from='mgr.24361 ' entity='mgr.smithi174.qpcwat' 2024-03-21T10:08:34.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:33 smithi174 ceph-mon[37496]: from='mgr.24361 ' entity='mgr.smithi174.qpcwat' 2024-03-21T10:08:34.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:33 smithi174 ceph-mon[37496]: from='mgr.24361 172.21.15.174:0/1831991646' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:08:34.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:33 smithi174 ceph-mon[37496]: from='mgr.24361 172.21.15.174:0/1831991646' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:08:34.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:33 smithi174 ceph-mon[37496]: from='mgr.24361 ' entity='mgr.smithi174.qpcwat' 2024-03-21T10:08:34.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:33 smithi174 ceph-mon[37496]: from='mgr.24361 172.21.15.174:0/1831991646' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:08:34.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:33 smithi174 ceph-mon[37496]: from='mgr.24361 172.21.15.174:0/1831991646' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "mgr fail", "who": "smithi174.qpcwat"}]: dispatch 2024-03-21T10:08:34.203 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:33 smithi174 ceph-mon[37496]: from='mgr.24361 ' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "mgr fail", "who": "smithi174.qpcwat"}]: dispatch 2024-03-21T10:08:34.203 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:33 smithi174 ceph-mon[37496]: osdmap e49: 6 total, 6 up, 6 in 2024-03-21T10:08:34.203 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:33 smithi174 ceph-mon[37496]: from='mgr.24361 ' entity='mgr.smithi174.qpcwat' cmd='[{"prefix": "mgr fail", "who": "smithi174.qpcwat"}]': finished 2024-03-21T10:08:34.203 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:33 smithi174 ceph-mon[37496]: mgrmap e26: smithi153.tluxni(active, starting, since 0.0152174s) 2024-03-21T10:08:34.203 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:33 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mon metadata", "id": "smithi153"}]: dispatch 2024-03-21T10:08:34.203 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:33 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mon metadata", "id": "smithi174"}]: dispatch 2024-03-21T10:08:34.203 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:33 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi153.iuyluj"}]: dispatch 2024-03-21T10:08:34.203 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:33 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi153.zipckr"}]: dispatch 2024-03-21T10:08:34.203 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:33 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi174.yyimst"}]: dispatch 2024-03-21T10:08:34.203 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:33 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi174.nhozag"}]: dispatch 2024-03-21T10:08:34.203 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:33 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mgr metadata", "who": "smithi153.tluxni", "id": "smithi153.tluxni"}]: dispatch 2024-03-21T10:08:34.203 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:33 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T10:08:34.203 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:33 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T10:08:34.203 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:33 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T10:08:34.204 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:33 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T10:08:34.204 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:33 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T10:08:34.204 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:33 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T10:08:34.204 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:33 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-21T10:08:34.204 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:33 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-21T10:08:34.204 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:33 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-21T10:08:34.204 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:33 smithi174 ceph-mon[37496]: Manager daemon smithi153.tluxni is now available 2024-03-21T10:08:34.204 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:33 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:08:34.204 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:33 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:08:34.204 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:33 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi153.tluxni/mirror_snapshot_schedule"}]: dispatch 2024-03-21T10:08:34.204 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:33 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi153.tluxni/trash_purge_schedule"}]: dispatch 2024-03-21T10:08:34.205 INFO:tasks.workunit.client.0.smithi153.stdout:2/1: mkdir d0 0 2024-03-21T10:08:34.205 INFO:tasks.workunit.client.0.smithi153.stdout:0/6: dread - f1 zero size 2024-03-21T10:08:34.205 INFO:tasks.workunit.client.0.smithi153.stdout:1/10: creat f2 x:0 0 0 2024-03-21T10:08:34.205 INFO:tasks.workunit.client.0.smithi153.stdout:4/0: dwrite - no filename 2024-03-21T10:08:34.207 INFO:tasks.workunit.client.0.smithi153.stdout:3/5: creat f0 x:0 0 0 2024-03-21T10:08:34.207 INFO:tasks.workunit.client.0.smithi153.stdout:3/6: rmdir - no directory 2024-03-21T10:08:34.208 INFO:tasks.workunit.client.0.smithi153.stdout:3/7: dread - f0 zero size 2024-03-21T10:08:34.208 INFO:tasks.workunit.client.0.smithi153.stdout:1/11: creat f3 x:0 0 0 2024-03-21T10:08:34.209 INFO:tasks.workunit.client.0.smithi153.stdout:3/8: dread - f0 zero size 2024-03-21T10:08:34.211 INFO:tasks.workunit.client.0.smithi153.stdout:3/9: dread - f0 zero size 2024-03-21T10:08:34.211 INFO:tasks.workunit.client.0.smithi153.stdout:6/0: dwrite - no filename 2024-03-21T10:08:34.212 INFO:tasks.workunit.client.0.smithi153.stdout:4/1: creat f0 x:0 0 0 2024-03-21T10:08:34.212 INFO:tasks.workunit.client.0.smithi153.stdout:1/12: creat f4 x:0 0 0 2024-03-21T10:08:34.213 INFO:tasks.workunit.client.0.smithi153.stdout:2/2: rmdir d0 0 2024-03-21T10:08:34.213 INFO:tasks.workunit.client.0.smithi153.stdout:1/13: stat f2 0 2024-03-21T10:08:34.214 INFO:tasks.workunit.client.0.smithi153.stdout:1/14: dread - f0 zero size 2024-03-21T10:08:34.215 INFO:tasks.workunit.client.0.smithi153.stdout:4/2: write f0 [557780,23988] 0 2024-03-21T10:08:34.218 INFO:tasks.workunit.client.0.smithi153.stdout:7/0: chown . 507 1 2024-03-21T10:08:34.218 INFO:tasks.workunit.client.0.smithi153.stdout:7/1: rmdir - no directory 2024-03-21T10:08:34.218 INFO:tasks.workunit.client.0.smithi153.stdout:5/0: creat f0 x:0 0 0 2024-03-21T10:08:34.218 INFO:tasks.workunit.client.0.smithi153.stdout:3/10: creat f1 x:0 0 0 2024-03-21T10:08:34.219 INFO:tasks.workunit.client.0.smithi153.stdout:2/3: symlink l1 0 2024-03-21T10:08:34.220 INFO:tasks.workunit.client.0.smithi153.stdout:1/15: creat f5 x:0 0 0 2024-03-21T10:08:34.223 INFO:tasks.workunit.client.0.smithi153.stdout:6/1: mknod c0 0 2024-03-21T10:08:34.228 INFO:tasks.workunit.client.0.smithi153.stdout:7/2: mknod c0 0 2024-03-21T10:08:34.228 INFO:tasks.workunit.client.0.smithi153.stdout:3/11: rename f1 to f2 0 2024-03-21T10:08:34.228 INFO:tasks.workunit.client.0.smithi153.stdout:8/0: chown . 1843664 1 2024-03-21T10:08:34.228 INFO:tasks.workunit.client.0.smithi153.stdout:4/3: dread f0 [0,4194304] 0 2024-03-21T10:08:34.229 INFO:tasks.workunit.client.0.smithi153.stdout:8/1: chown . 18325 1 2024-03-21T10:08:34.229 INFO:tasks.workunit.client.0.smithi153.stdout:7/3: stat c0 0 2024-03-21T10:08:34.229 INFO:tasks.workunit.client.0.smithi153.stdout:7/4: dread - no filename 2024-03-21T10:08:34.230 INFO:tasks.workunit.client.0.smithi153.stdout:8/2: chown . 30518 1 2024-03-21T10:08:34.230 INFO:tasks.workunit.client.0.smithi153.stdout:3/12: dread - f2 zero size 2024-03-21T10:08:34.230 INFO:tasks.workunit.client.0.smithi153.stdout:1/16: mkdir d6 0 2024-03-21T10:08:34.231 INFO:tasks.workunit.client.0.smithi153.stdout:3/13: dread - f0 zero size 2024-03-21T10:08:34.233 INFO:tasks.workunit.client.0.smithi153.stdout:6/2: creat f1 x:0 0 0 2024-03-21T10:08:34.234 INFO:tasks.workunit.client.0.smithi153.stdout:4/4: dread f0 [0,4194304] 0 2024-03-21T10:08:34.237 INFO:tasks.workunit.client.0.smithi153.stdout:7/5: mknod c1 0 2024-03-21T10:08:34.237 INFO:tasks.workunit.client.0.smithi153.stdout:7/6: dread - no filename 2024-03-21T10:08:34.237 INFO:tasks.workunit.client.0.smithi153.stdout:7/7: fsync - no filename 2024-03-21T10:08:34.238 INFO:tasks.workunit.client.0.smithi153.stdout:3/14: write f0 [713908,82628] 0 2024-03-21T10:08:34.239 INFO:tasks.workunit.client.0.smithi153.stdout:6/3: write f1 [84782,47011] 0 2024-03-21T10:08:34.243 INFO:tasks.workunit.client.0.smithi153.stdout:9/0: mknod c0 0 2024-03-21T10:08:34.245 INFO:tasks.workunit.client.0.smithi153.stdout:4/5: creat f1 x:0 0 0 2024-03-21T10:08:34.248 INFO:tasks.workunit.client.0.smithi153.stdout:1/17: creat d6/f7 x:0 0 0 2024-03-21T10:08:34.248 INFO:tasks.workunit.client.0.smithi153.stdout:7/8: creat f2 x:0 0 0 2024-03-21T10:08:34.248 INFO:tasks.workunit.client.0.smithi153.stdout:6/4: fdatasync f1 0 2024-03-21T10:08:34.254 INFO:tasks.workunit.client.0.smithi153.stdout:9/1: mkdir d1 0 2024-03-21T10:08:34.254 INFO:tasks.workunit.client.0.smithi153.stdout:9/2: fsync - no filename 2024-03-21T10:08:34.254 INFO:tasks.workunit.client.0.smithi153.stdout:9/3: dwrite - no filename 2024-03-21T10:08:34.255 INFO:tasks.workunit.client.0.smithi153.stdout:8/3: creat f0 x:0 0 0 2024-03-21T10:08:34.255 INFO:tasks.workunit.client.0.smithi153.stdout:7/9: mknod c3 0 2024-03-21T10:08:34.257 INFO:tasks.workunit.client.0.smithi153.stdout:1/18: write d6/f7 [337747,111552] 0 2024-03-21T10:08:34.258 INFO:tasks.workunit.client.0.smithi153.stdout:3/15: dread f0 [0,4194304] 0 2024-03-21T10:08:34.262 INFO:tasks.workunit.client.0.smithi153.stdout:7/10: chown c1 7410860 1 2024-03-21T10:08:34.264 INFO:tasks.workunit.client.0.smithi153.stdout:7/11: fdatasync f2 0 2024-03-21T10:08:34.269 INFO:tasks.workunit.client.0.smithi153.stdout:9/4: mknod d1/c2 0 2024-03-21T10:08:34.272 INFO:tasks.workunit.client.0.smithi153.stdout:3/16: write f0 [943146,75206] 0 2024-03-21T10:08:34.272 INFO:tasks.workunit.client.0.smithi153.stdout:7/12: mkdir d4 0 2024-03-21T10:08:34.273 INFO:tasks.workunit.client.0.smithi153.stdout:5/1: dwrite f0 [0,4194304] 0 2024-03-21T10:08:34.275 INFO:tasks.workunit.client.0.smithi153.stdout:7/13: fdatasync f2 0 2024-03-21T10:08:34.281 INFO:tasks.workunit.client.0.smithi153.stdout:3/17: symlink l3 0 2024-03-21T10:08:34.287 INFO:tasks.workunit.client.0.smithi153.stdout:3/18: symlink l4 0 2024-03-21T10:08:34.292 INFO:tasks.workunit.client.0.smithi153.stdout:3/19: chown f2 169834 1 2024-03-21T10:08:34.297 INFO:tasks.workunit.client.0.smithi153.stdout:8/4: dwrite f0 [0,4194304] 0 2024-03-21T10:08:34.298 INFO:tasks.workunit.client.0.smithi153.stdout:7/14: link c0 d4/c5 0 2024-03-21T10:08:34.300 INFO:tasks.workunit.client.0.smithi153.stdout:8/5: creat f1 x:0 0 0 2024-03-21T10:08:34.301 INFO:tasks.workunit.client.0.smithi153.stdout:3/20: write f2 [341271,41761] 0 2024-03-21T10:08:34.305 INFO:tasks.workunit.client.0.smithi153.stdout:7/15: unlink c0 0 2024-03-21T10:08:34.308 INFO:tasks.workunit.client.0.smithi153.stdout:7/16: write f2 [668274,102084] 0 2024-03-21T10:08:34.309 INFO:tasks.workunit.client.0.smithi153.stdout:8/6: rename f1 to f2 0 2024-03-21T10:08:34.314 INFO:tasks.workunit.client.0.smithi153.stdout:3/21: link f0 f5 0 2024-03-21T10:08:34.325 INFO:tasks.workunit.client.0.smithi153.stdout:3/22: link f2 f6 0 2024-03-21T10:08:34.328 INFO:tasks.workunit.client.0.smithi153.stdout:7/17: dread f2 [0,4194304] 0 2024-03-21T10:08:34.336 INFO:tasks.workunit.client.0.smithi153.stdout:7/18: mknod d4/c6 0 2024-03-21T10:08:34.336 INFO:tasks.workunit.client.0.smithi153.stdout:8/7: dwrite f0 [0,4194304] 0 2024-03-21T10:08:34.347 INFO:tasks.workunit.client.0.smithi153.stdout:0/7: getdents . 0 2024-03-21T10:08:34.351 INFO:tasks.workunit.client.0.smithi153.stdout:2/4: getdents . 0 2024-03-21T10:08:34.354 INFO:tasks.workunit.client.0.smithi153.stdout:0/8: rename f1 to f2 0 2024-03-21T10:08:34.356 INFO:tasks.workunit.client.0.smithi153.stdout:0/9: dread - f2 zero size 2024-03-21T10:08:34.357 INFO:tasks.workunit.client.0.smithi153.stdout:2/5: mknod c2 0 2024-03-21T10:08:34.358 INFO:tasks.workunit.client.0.smithi153.stdout:2/6: dwrite - no filename 2024-03-21T10:08:34.359 INFO:tasks.workunit.client.0.smithi153.stdout:6/5: getdents . 0 2024-03-21T10:08:34.361 INFO:tasks.workunit.client.0.smithi153.stdout:0/10: dread - f2 zero size 2024-03-21T10:08:34.375 INFO:tasks.workunit.client.0.smithi153.stdout:2/7: mknod c3 0 2024-03-21T10:08:34.376 INFO:tasks.workunit.client.0.smithi153.stdout:2/8: truncate - no filename 2024-03-21T10:08:34.376 INFO:tasks.workunit.client.0.smithi153.stdout:4/6: getdents . 0 2024-03-21T10:08:34.376 INFO:tasks.workunit.client.0.smithi153.stdout:2/9: chown l1 1758136 1 2024-03-21T10:08:34.377 INFO:tasks.workunit.client.0.smithi153.stdout:1/19: rmdir d6 39 2024-03-21T10:08:34.381 INFO:tasks.workunit.client.0.smithi153.stdout:9/5: getdents d1 0 2024-03-21T10:08:34.384 INFO:tasks.workunit.client.0.smithi153.stdout:7/19: dwrite f2 [0,4194304] 0 2024-03-21T10:08:34.387 INFO:tasks.workunit.client.0.smithi153.stdout:5/2: truncate f0 1275190 0 2024-03-21T10:08:34.389 INFO:tasks.workunit.client.0.smithi153.stdout:9/6: chown d1 1268339 1 2024-03-21T10:08:34.389 INFO:tasks.workunit.client.0.smithi153.stdout:9/7: dread - no filename 2024-03-21T10:08:34.392 INFO:tasks.workunit.client.0.smithi153.stdout:7/20: write f2 [3912723,15991] 0 2024-03-21T10:08:34.393 INFO:tasks.workunit.client.0.smithi153.stdout:8/8: dwrite f2 [0,4194304] 0 2024-03-21T10:08:34.397 INFO:tasks.workunit.client.0.smithi153.stdout:3/23: fsync f2 0 2024-03-21T10:08:34.400 INFO:tasks.workunit.client.0.smithi153.stdout:4/7: write f0 [1063944,130166] 0 2024-03-21T10:08:34.400 INFO:tasks.workunit.client.0.smithi153.stdout:3/24: read f2 [40284,999] 0 2024-03-21T10:08:34.401 INFO:tasks.workunit.client.0.smithi153.stdout:5/3: creat f1 x:0 0 0 2024-03-21T10:08:34.404 INFO:tasks.workunit.client.0.smithi153.stdout:9/8: unlink d1/c2 0 2024-03-21T10:08:34.405 INFO:tasks.workunit.client.0.smithi153.stdout:1/20: rmdir d6 39 2024-03-21T10:08:34.405 INFO:tasks.workunit.client.0.smithi153.stdout:8/9: write f2 [3279996,39557] 0 2024-03-21T10:08:34.406 INFO:tasks.workunit.client.0.smithi153.stdout:2/10: link c2 c4 0 2024-03-21T10:08:34.406 INFO:tasks.workunit.client.0.smithi153.stdout:2/11: fdatasync - no filename 2024-03-21T10:08:34.407 INFO:tasks.workunit.client.0.smithi153.stdout:5/4: dread - f1 zero size 2024-03-21T10:08:34.414 INFO:tasks.workunit.client.0.smithi153.stdout:1/21: dread - f2 zero size 2024-03-21T10:08:34.416 INFO:tasks.workunit.client.0.smithi153.stdout:9/9: creat d1/f3 x:0 0 0 2024-03-21T10:08:34.417 INFO:tasks.workunit.client.0.smithi153.stdout:6/6: dwrite f1 [0,4194304] 0 2024-03-21T10:08:34.419 INFO:tasks.workunit.client.0.smithi153.stdout:8/10: creat f3 x:0 0 0 2024-03-21T10:08:34.420 INFO:tasks.workunit.client.0.smithi153.stdout:2/12: creat f5 x:0 0 0 2024-03-21T10:08:34.420 INFO:tasks.workunit.client.0.smithi153.stdout:5/5: dread - f1 zero size 2024-03-21T10:08:34.429 INFO:tasks.workunit.client.0.smithi153.stdout:2/13: write f5 [348612,52928] 0 2024-03-21T10:08:34.436 INFO:tasks.workunit.client.0.smithi153.stdout:2/14: readlink l1 0 2024-03-21T10:08:34.436 INFO:tasks.workunit.client.0.smithi153.stdout:9/10: mkdir d1/d4 0 2024-03-21T10:08:34.436 INFO:tasks.workunit.client.0.smithi153.stdout:3/25: dread f6 [0,4194304] 0 2024-03-21T10:08:34.439 INFO:tasks.workunit.client.0.smithi153.stdout:1/22: creat d6/f8 x:0 0 0 2024-03-21T10:08:34.439 INFO:tasks.workunit.client.0.smithi153.stdout:5/6: dread - f1 zero size 2024-03-21T10:08:34.451 INFO:tasks.workunit.client.0.smithi153.stdout:9/11: write d1/f3 [956767,26928] 0 2024-03-21T10:08:34.454 INFO:tasks.workunit.client.0.smithi153.stdout:4/8: dwrite f1 [0,4194304] 0 2024-03-21T10:08:34.455 INFO:tasks.workunit.client.0.smithi153.stdout:2/15: chown c2 12022 1 2024-03-21T10:08:34.457 INFO:tasks.workunit.client.0.smithi153.stdout:7/21: mkdir d4/d7 0 2024-03-21T10:08:34.460 INFO:tasks.workunit.client.0.smithi153.stdout:1/23: rmdir d6 39 2024-03-21T10:08:34.460 INFO:tasks.workunit.client.0.smithi153.stdout:6/7: truncate f1 1825898 0 2024-03-21T10:08:34.464 INFO:tasks.workunit.client.0.smithi153.stdout:4/9: creat f2 x:0 0 0 2024-03-21T10:08:34.470 INFO:tasks.workunit.client.0.smithi153.stdout:8/11: dwrite f3 [0,4194304] 0 2024-03-21T10:08:34.472 INFO:tasks.workunit.client.0.smithi153.stdout:4/10: mknod c3 0 2024-03-21T10:08:34.474 INFO:tasks.workunit.client.0.smithi153.stdout:9/12: mknod d1/d4/c5 0 2024-03-21T10:08:34.478 INFO:tasks.workunit.client.0.smithi153.stdout:8/12: write f2 [1080967,108879] 0 2024-03-21T10:08:34.482 INFO:tasks.workunit.client.0.smithi153.stdout:4/11: chown f0 0 1 2024-03-21T10:08:34.483 INFO:tasks.workunit.client.0.smithi153.stdout:8/13: chown f3 282 1 2024-03-21T10:08:34.486 INFO:tasks.workunit.client.0.smithi153.stdout:5/7: truncate f0 271436 0 2024-03-21T10:08:34.486 INFO:tasks.workunit.client.0.smithi153.stdout:5/8: readlink - no filename 2024-03-21T10:08:34.488 INFO:tasks.workunit.client.0.smithi153.stdout:9/13: unlink d1/f3 0 2024-03-21T10:08:34.494 INFO:tasks.workunit.client.0.smithi153.stdout:4/12: mknod c4 0 2024-03-21T10:08:34.496 INFO:tasks.workunit.client.0.smithi153.stdout:3/26: dwrite f0 [0,4194304] 0 2024-03-21T10:08:34.498 INFO:tasks.workunit.client.0.smithi153.stdout:4/13: fsync f0 0 2024-03-21T10:08:34.499 INFO:tasks.workunit.client.0.smithi153.stdout:7/22: getdents d4/d7 0 2024-03-21T10:08:34.504 INFO:tasks.workunit.client.0.smithi153.stdout:2/16: dwrite f5 [0,4194304] 0 2024-03-21T10:08:34.506 INFO:tasks.workunit.client.0.smithi153.stdout:4/14: write f1 [3724385,23845] 0 2024-03-21T10:08:34.508 INFO:tasks.workunit.client.0.smithi153.stdout:6/8: truncate f1 129492 0 2024-03-21T10:08:34.508 INFO:tasks.workunit.client.0.smithi153.stdout:2/17: readlink l1 0 2024-03-21T10:08:34.512 INFO:tasks.workunit.client.0.smithi153.stdout:2/18: readlink l1 0 2024-03-21T10:08:34.512 INFO:tasks.workunit.client.0.smithi153.stdout:5/9: fsync f0 0 2024-03-21T10:08:34.512 INFO:tasks.workunit.client.0.smithi153.stdout:4/15: chown c3 14727 1 2024-03-21T10:08:34.512 INFO:tasks.workunit.client.0.smithi153.stdout:2/19: write f5 [2568719,69173] 0 2024-03-21T10:08:34.517 INFO:tasks.workunit.client.0.smithi153.stdout:5/10: fdatasync f1 0 2024-03-21T10:08:34.517 INFO:tasks.workunit.client.0.smithi153.stdout:9/14: link c0 d1/d4/c6 0 2024-03-21T10:08:34.521 INFO:tasks.workunit.client.0.smithi153.stdout:3/27: rename f5 to f7 0 2024-03-21T10:08:34.522 INFO:tasks.workunit.client.0.smithi153.stdout:4/16: mknod c5 0 2024-03-21T10:08:34.522 INFO:tasks.workunit.client.0.smithi153.stdout:3/28: readlink l4 0 2024-03-21T10:08:34.523 INFO:tasks.workunit.client.0.smithi153.stdout:5/11: dread - f1 zero size 2024-03-21T10:08:34.524 INFO:tasks.workunit.client.0.smithi153.stdout:8/14: dwrite f0 [0,4194304] 0 2024-03-21T10:08:34.525 INFO:tasks.workunit.client.0.smithi153.stdout:2/20: symlink l6 0 2024-03-21T10:08:34.528 INFO:tasks.workunit.client.0.smithi153.stdout:4/17: fdatasync f0 0 2024-03-21T10:08:34.529 INFO:tasks.workunit.client.0.smithi153.stdout:9/15: creat d1/d4/f7 x:0 0 0 2024-03-21T10:08:34.532 INFO:tasks.workunit.client.0.smithi153.stdout:3/29: creat f8 x:0 0 0 2024-03-21T10:08:34.532 INFO:tasks.workunit.client.0.smithi153.stdout:4/18: stat c4 0 2024-03-21T10:08:34.532 INFO:tasks.workunit.client.0.smithi153.stdout:5/12: write f0 [643380,31854] 0 2024-03-21T10:08:34.534 INFO:tasks.workunit.client.0.smithi153.stdout:9/16: fdatasync d1/d4/f7 0 2024-03-21T10:08:34.535 INFO:tasks.workunit.client.0.smithi153.stdout:3/30: fdatasync f8 0 2024-03-21T10:08:34.539 INFO:tasks.workunit.client.0.smithi153.stdout:5/13: write f0 [1672538,52869] 0 2024-03-21T10:08:34.539 INFO:tasks.workunit.client.0.smithi153.stdout:2/21: chown f5 2579949 1 2024-03-21T10:08:34.540 INFO:tasks.workunit.client.0.smithi153.stdout:9/17: write d1/d4/f7 [733671,78075] 0 2024-03-21T10:08:34.542 INFO:tasks.workunit.client.0.smithi153.stdout:4/19: unlink f0 0 2024-03-21T10:08:34.543 INFO:tasks.workunit.client.0.smithi153.stdout:7/23: dread f2 [0,4194304] 0 2024-03-21T10:08:34.544 INFO:tasks.workunit.client.0.smithi153.stdout:7/24: readlink - no filename 2024-03-21T10:08:34.547 INFO:tasks.workunit.client.0.smithi153.stdout:2/22: creat f7 x:0 0 0 2024-03-21T10:08:34.550 INFO:tasks.workunit.client.0.smithi153.stdout:6/9: dread f1 [0,4194304] 0 2024-03-21T10:08:34.554 INFO:tasks.workunit.client.0.smithi153.stdout:4/20: creat f6 x:0 0 0 2024-03-21T10:08:34.558 INFO:tasks.workunit.client.0.smithi153.stdout:7/25: fsync f2 0 2024-03-21T10:08:34.565 INFO:tasks.workunit.client.0.smithi153.stdout:9/18: mknod d1/c8 0 2024-03-21T10:08:34.569 INFO:tasks.workunit.client.0.smithi153.stdout:6/10: fdatasync f1 0 2024-03-21T10:08:34.570 INFO:tasks.workunit.client.0.smithi153.stdout:4/21: creat f7 x:0 0 0 2024-03-21T10:08:34.574 INFO:tasks.workunit.client.0.smithi153.stdout:8/15: dwrite f3 [0,4194304] 0 2024-03-21T10:08:34.576 INFO:tasks.workunit.client.0.smithi153.stdout:4/22: stat f1 0 2024-03-21T10:08:34.579 INFO:tasks.workunit.client.0.smithi153.stdout:2/23: truncate f5 3324571 0 2024-03-21T10:08:34.588 INFO:tasks.workunit.client.0.smithi153.stdout:3/31: dwrite f8 [0,4194304] 0 2024-03-21T10:08:34.588 INFO:tasks.workunit.client.0.smithi153.stdout:8/16: rename f0 to f4 0 2024-03-21T10:08:34.589 INFO:tasks.workunit.client.0.smithi153.stdout:5/14: dwrite f1 [0,4194304] 0 2024-03-21T10:08:34.590 INFO:tasks.workunit.client.0.smithi153.stdout:6/11: write f1 [758023,20437] 0 2024-03-21T10:08:34.590 INFO:tasks.workunit.client.0.smithi153.stdout:4/23: unlink c4 0 2024-03-21T10:08:34.590 INFO:tasks.workunit.client.0.smithi153.stdout:4/24: readlink - no filename 2024-03-21T10:08:34.595 INFO:tasks.workunit.client.0.smithi153.stdout:3/32: dread f6 [0,4194304] 0 2024-03-21T10:08:34.596 INFO:tasks.workunit.client.0.smithi153.stdout:3/33: chown f8 105895158 1 2024-03-21T10:08:34.597 INFO:tasks.workunit.client.0.smithi153.stdout:3/34: chown f2 5 1 2024-03-21T10:08:34.599 INFO:tasks.workunit.client.0.smithi153.stdout:3/35: readlink l3 0 2024-03-21T10:08:34.600 INFO:tasks.workunit.client.0.smithi153.stdout:7/26: rmdir d4/d7 0 2024-03-21T10:08:34.603 INFO:tasks.workunit.client.0.smithi153.stdout:8/17: unlink f2 0 2024-03-21T10:08:34.603 INFO:tasks.workunit.client.0.smithi153.stdout:3/36: write f8 [3678066,48503] 0 2024-03-21T10:08:34.605 INFO:tasks.workunit.client.0.smithi153.stdout:3/37: chown f0 2084513 1 2024-03-21T10:08:34.605 INFO:tasks.workunit.client.0.smithi153.stdout:8/18: chown f3 859778218 1 2024-03-21T10:08:34.605 INFO:tasks.workunit.client.0.smithi153.stdout:5/15: symlink l2 0 2024-03-21T10:08:34.608 INFO:tasks.workunit.client.0.smithi153.stdout:4/25: mknod c8 0 2024-03-21T10:08:34.611 INFO:tasks.workunit.client.0.smithi153.stdout:4/26: chown f6 57789096 1 2024-03-21T10:08:34.619 INFO:tasks.workunit.client.0.smithi153.stdout:8/19: mkdir d5 0 2024-03-21T10:08:34.621 INFO:tasks.workunit.client.0.smithi153.stdout:5/16: unlink f0 0 2024-03-21T10:08:34.621 INFO:tasks.workunit.client.0.smithi153.stdout:9/19: link d1/d4/c6 d1/d4/c9 0 2024-03-21T10:08:34.625 INFO:tasks.workunit.client.0.smithi153.stdout:9/20: fsync d1/d4/f7 0 2024-03-21T10:08:34.627 INFO:tasks.workunit.client.0.smithi153.stdout:4/27: symlink l9 0 2024-03-21T10:08:34.632 INFO:tasks.workunit.client.0.smithi153.stdout:3/38: dread f8 [0,4194304] 0 2024-03-21T10:08:34.632 INFO:tasks.workunit.client.0.smithi153.stdout:7/27: mknod d4/c8 0 2024-03-21T10:08:34.637 INFO:tasks.workunit.client.0.smithi153.stdout:5/17: unlink l2 0 2024-03-21T10:08:34.640 INFO:tasks.workunit.client.0.smithi153.stdout:6/12: dwrite f1 [0,4194304] 0 2024-03-21T10:08:34.643 INFO:tasks.workunit.client.0.smithi153.stdout:6/13: chown c0 4117199 1 2024-03-21T10:08:34.643 INFO:tasks.workunit.client.0.smithi153.stdout:2/24: dread f5 [0,4194304] 0 2024-03-21T10:08:34.644 INFO:tasks.workunit.client.0.smithi153.stdout:4/28: mknod ca 0 2024-03-21T10:08:34.646 INFO:tasks.workunit.client.0.smithi153.stdout:3/39: symlink l9 0 2024-03-21T10:08:34.650 INFO:tasks.workunit.client.0.smithi153.stdout:5/18: mknod c3 0 2024-03-21T10:08:34.654 INFO:tasks.workunit.client.0.smithi153.stdout:9/21: unlink d1/d4/c6 0 2024-03-21T10:08:34.658 INFO:tasks.workunit.client.0.smithi153.stdout:2/25: symlink l8 0 2024-03-21T10:08:34.660 INFO:tasks.workunit.client.0.smithi153.stdout:4/29: creat fb x:0 0 0 2024-03-21T10:08:34.662 INFO:tasks.workunit.client.0.smithi153.stdout:3/40: write f0 [3481063,40157] 0 2024-03-21T10:08:34.666 INFO:tasks.workunit.client.0.smithi153.stdout:2/26: mknod c9 0 2024-03-21T10:08:34.666 INFO:tasks.workunit.client.0.smithi153.stdout:2/27: readlink l8 0 2024-03-21T10:08:34.667 INFO:tasks.workunit.client.0.smithi153.stdout:2/28: read f5 [2011634,93182] 0 2024-03-21T10:08:34.668 INFO:tasks.workunit.client.0.smithi153.stdout:6/14: fsync f1 0 2024-03-21T10:08:34.670 INFO:tasks.workunit.client.0.smithi153.stdout:3/41: fsync f7 0 2024-03-21T10:08:34.670 INFO:tasks.workunit.client.0.smithi153.stdout:4/30: creat fc x:0 0 0 2024-03-21T10:08:34.672 INFO:tasks.workunit.client.0.smithi153.stdout:3/42: truncate f2 504037 0 2024-03-21T10:08:34.677 INFO:tasks.workunit.client.0.smithi153.stdout:9/22: truncate d1/d4/f7 789963 0 2024-03-21T10:08:34.680 INFO:tasks.workunit.client.0.smithi153.stdout:8/20: truncate f4 745870 0 2024-03-21T10:08:34.682 INFO:tasks.workunit.client.0.smithi153.stdout:4/31: creat fd x:0 0 0 2024-03-21T10:08:34.684 INFO:tasks.workunit.client.0.smithi153.stdout:3/43: dread f0 [0,4194304] 0 2024-03-21T10:08:34.685 INFO:tasks.workunit.client.0.smithi153.stdout:9/23: symlink d1/d4/la 0 2024-03-21T10:08:34.686 INFO:tasks.workunit.client.0.smithi153.stdout:3/44: chown f0 495487 1 2024-03-21T10:08:34.687 INFO:tasks.workunit.client.0.smithi153.stdout:6/15: rename f1 to f2 0 2024-03-21T10:08:34.690 INFO:tasks.workunit.client.0.smithi153.stdout:4/32: rename f6 to fe 0 2024-03-21T10:08:34.691 INFO:tasks.workunit.client.0.smithi153.stdout:4/33: fsync f2 0 2024-03-21T10:08:34.692 INFO:tasks.workunit.client.0.smithi153.stdout:9/24: chown d1/d4/c9 1653 1 2024-03-21T10:08:34.692 INFO:tasks.workunit.client.0.smithi153.stdout:5/19: truncate f1 2992748 0 2024-03-21T10:08:34.694 INFO:tasks.workunit.client.0.smithi153.stdout:6/16: rename c0 to c3 0 2024-03-21T10:08:34.696 INFO:tasks.workunit.client.0.smithi153.stdout:2/29: getdents . 0 2024-03-21T10:08:34.700 INFO:tasks.workunit.client.0.smithi153.stdout:5/20: mkdir d4 0 2024-03-21T10:08:34.703 INFO:tasks.workunit.client.0.smithi153.stdout:6/17: rename f2 to f4 0 2024-03-21T10:08:34.706 INFO:tasks.workunit.client.0.smithi153.stdout:2/30: mknod ca 0 2024-03-21T10:08:34.713 INFO:tasks.workunit.client.0.smithi153.stdout:6/18: creat f5 x:0 0 0 2024-03-21T10:08:34.714 INFO:tasks.workunit.client.0.smithi153.stdout:6/19: readlink - no filename 2024-03-21T10:08:34.716 INFO:tasks.workunit.client.0.smithi153.stdout:3/45: getdents . 0 2024-03-21T10:08:34.719 INFO:tasks.workunit.client.0.smithi153.stdout:6/20: write f4 [4818551,77992] 0 2024-03-21T10:08:34.719 INFO:tasks.workunit.client.0.smithi153.stdout:3/46: mknod ca 0 2024-03-21T10:08:34.722 INFO:tasks.workunit.client.0.smithi153.stdout:3/47: chown f6 119 1 2024-03-21T10:08:34.723 INFO:tasks.workunit.client.0.smithi153.stdout:9/25: dread d1/d4/f7 [0,4194304] 0 2024-03-21T10:08:34.733 INFO:tasks.workunit.client.0.smithi153.stdout:5/21: write f1 [3720411,95493] 0 2024-03-21T10:08:34.735 INFO:tasks.workunit.client.0.smithi153.stdout:2/31: dwrite f7 [0,4194304] 0 2024-03-21T10:08:34.747 INFO:tasks.workunit.client.0.smithi153.stdout:2/32: creat fb x:0 0 0 2024-03-21T10:08:34.749 INFO:tasks.workunit.client.0.smithi153.stdout:6/21: dwrite f5 [0,4194304] 0 2024-03-21T10:08:34.749 INFO:tasks.workunit.client.0.smithi153.stdout:5/22: mkdir d4/d5 0 2024-03-21T10:08:34.751 INFO:tasks.workunit.client.0.smithi153.stdout:3/48: dread f8 [0,4194304] 0 2024-03-21T10:08:34.751 INFO:tasks.workunit.client.0.smithi153.stdout:5/23: chown d4/d5 1467664 1 2024-03-21T10:08:34.754 INFO:tasks.workunit.client.0.smithi153.stdout:3/49: write f7 [1036304,17178] 0 2024-03-21T10:08:34.755 INFO:tasks.workunit.client.0.smithi153.stdout:3/50: stat f6 0 2024-03-21T10:08:34.759 INFO:tasks.workunit.client.0.smithi153.stdout:3/51: dread f6 [0,4194304] 0 2024-03-21T10:08:34.762 INFO:tasks.workunit.client.0.smithi153.stdout:3/52: symlink lb 0 2024-03-21T10:08:34.764 INFO:tasks.workunit.client.0.smithi153.stdout:6/22: dread f4 [4194304,4194304] 0 2024-03-21T10:08:34.764 INFO:tasks.workunit.client.0.smithi153.stdout:2/33: dread f5 [0,4194304] 0 2024-03-21T10:08:34.765 INFO:tasks.workunit.client.0.smithi153.stdout:3/53: chown f0 55 1 2024-03-21T10:08:34.766 INFO:tasks.workunit.client.0.smithi153.stdout:5/24: dread f1 [0,4194304] 0 2024-03-21T10:08:34.770 INFO:tasks.workunit.client.0.smithi153.stdout:3/54: chown l3 771428798 1 2024-03-21T10:08:34.771 INFO:tasks.workunit.client.0.smithi153.stdout:5/25: write f1 [4173492,5124] 0 2024-03-21T10:08:34.775 INFO:tasks.workunit.client.0.smithi153.stdout:5/26: creat d4/d5/f6 x:0 0 0 2024-03-21T10:08:34.778 INFO:tasks.workunit.client.0.smithi153.stdout:6/23: dread f5 [0,4194304] 0 2024-03-21T10:08:34.781 INFO:tasks.workunit.client.0.smithi153.stdout:2/34: dread f5 [0,4194304] 0 2024-03-21T10:08:34.782 INFO:tasks.workunit.client.0.smithi153.stdout:5/27: fsync d4/d5/f6 0 2024-03-21T10:08:34.787 INFO:tasks.workunit.client.0.smithi153.stdout:5/28: creat d4/d5/f7 x:0 0 0 2024-03-21T10:08:34.789 INFO:tasks.workunit.client.0.smithi153.stdout:6/24: truncate f4 4690848 0 2024-03-21T10:08:34.790 INFO:tasks.workunit.client.0.smithi153.stdout:5/29: mkdir d4/d8 0 2024-03-21T10:08:34.791 INFO:tasks.workunit.client.0.smithi153.stdout:2/35: truncate f7 4182678 0 2024-03-21T10:08:34.792 INFO:tasks.workunit.client.0.smithi153.stdout:3/55: dwrite f8 [0,4194304] 0 2024-03-21T10:08:34.793 INFO:tasks.workunit.client.0.smithi153.stdout:3/56: readlink l9 0 2024-03-21T10:08:34.794 INFO:tasks.workunit.client.0.smithi153.stdout:2/36: creat fc x:0 0 0 2024-03-21T10:08:34.794 INFO:tasks.workunit.client.0.smithi153.stdout:6/25: rename c3 to c6 0 2024-03-21T10:08:34.796 INFO:tasks.workunit.client.0.smithi153.stdout:3/57: creat fc x:0 0 0 2024-03-21T10:08:34.798 INFO:tasks.workunit.client.0.smithi153.stdout:6/26: link f5 f7 0 2024-03-21T10:08:34.799 INFO:tasks.workunit.client.0.smithi153.stdout:3/58: mkdir dd 0 2024-03-21T10:08:34.801 INFO:tasks.workunit.client.0.smithi153.stdout:5/30: mknod d4/d8/c9 0 2024-03-21T10:08:34.805 INFO:tasks.workunit.client.0.smithi153.stdout:3/59: mknod dd/ce 0 2024-03-21T10:08:34.805 INFO:tasks.workunit.client.0.smithi153.stdout:5/31: rmdir d4 39 2024-03-21T10:08:34.805 INFO:tasks.workunit.client.0.smithi153.stdout:3/60: stat f7 0 2024-03-21T10:08:34.809 INFO:tasks.workunit.client.0.smithi153.stdout:3/61: mknod dd/cf 0 2024-03-21T10:08:34.815 INFO:tasks.workunit.client.0.smithi153.stdout:5/32: rename d4/d5/f6 to d4/fa 0 2024-03-21T10:08:34.818 INFO:tasks.workunit.client.0.smithi153.stdout:2/37: getdents . 0 2024-03-21T10:08:34.822 INFO:tasks.workunit.client.0.smithi153.stdout:2/38: mkdir dd 0 2024-03-21T10:08:34.823 INFO:tasks.workunit.client.0.smithi153.stdout:6/27: dwrite f5 [0,4194304] 0 2024-03-21T10:08:34.826 INFO:tasks.workunit.client.0.smithi153.stdout:2/39: write fb [41310,35142] 0 2024-03-21T10:08:34.830 INFO:tasks.workunit.client.0.smithi153.stdout:6/28: creat f8 x:0 0 0 2024-03-21T10:08:34.831 INFO:tasks.workunit.client.0.smithi153.stdout:3/62: write f2 [165569,86647] 0 2024-03-21T10:08:34.832 INFO:tasks.workunit.client.0.smithi153.stdout:2/40: symlink dd/le 0 2024-03-21T10:08:34.837 INFO:tasks.workunit.client.0.smithi153.stdout:5/33: dread f1 [0,4194304] 0 2024-03-21T10:08:34.839 INFO:tasks.workunit.client.0.smithi153.stdout:2/41: mknod dd/cf 0 2024-03-21T10:08:34.839 INFO:tasks.workunit.client.0.smithi153.stdout:3/63: creat dd/f10 x:0 0 0 2024-03-21T10:08:34.845 INFO:tasks.workunit.client.0.smithi153.stdout:2/42: creat dd/f10 x:0 0 0 2024-03-21T10:08:34.845 INFO:tasks.workunit.client.0.smithi153.stdout:2/43: stat fb 0 2024-03-21T10:08:34.845 INFO:tasks.workunit.client.0.smithi153.stdout:3/64: rename dd/ce to dd/c11 0 2024-03-21T10:08:34.847 INFO:tasks.workunit.client.0.smithi153.stdout:6/29: chown f4 103 1 2024-03-21T10:08:34.848 INFO:tasks.workunit.client.0.smithi153.stdout:6/30: stat f5 0 2024-03-21T10:08:34.848 INFO:tasks.workunit.client.0.smithi153.stdout:6/31: readlink - no filename 2024-03-21T10:08:34.852 INFO:tasks.workunit.client.0.smithi153.stdout:6/32: mknod c9 0 2024-03-21T10:08:34.859 INFO:tasks.workunit.client.0.smithi153.stdout:3/65: getdents dd 0 2024-03-21T10:08:34.859 INFO:tasks.workunit.client.0.smithi153.stdout:6/33: symlink la 0 2024-03-21T10:08:34.861 INFO:tasks.workunit.client.0.smithi153.stdout:2/44: write f7 [5221203,69073] 0 2024-03-21T10:08:34.861 INFO:tasks.workunit.client.0.smithi153.stdout:5/34: dwrite d4/fa [0,4194304] 0 2024-03-21T10:08:34.862 INFO:tasks.workunit.client.0.smithi153.stdout:3/66: write f0 [957485,12784] 0 2024-03-21T10:08:34.865 INFO:tasks.workunit.client.0.smithi153.stdout:6/34: symlink lb 0 2024-03-21T10:08:34.867 INFO:tasks.workunit.client.0.smithi153.stdout:5/35: rmdir d4/d8 39 2024-03-21T10:08:34.867 INFO:tasks.workunit.client.0.smithi153.stdout:6/35: write f5 [2906452,77717] 0 2024-03-21T10:08:34.868 INFO:tasks.workunit.client.0.smithi153.stdout:5/36: read f1 [3936553,9588] 0 2024-03-21T10:08:34.870 INFO:tasks.workunit.client.0.smithi153.stdout:3/67: rmdir dd 39 2024-03-21T10:08:34.871 INFO:tasks.workunit.client.0.smithi153.stdout:6/36: fsync f4 0 2024-03-21T10:08:34.878 INFO:tasks.workunit.client.0.smithi153.stdout:5/37: link d4/d5/f7 d4/fb 0 2024-03-21T10:08:34.878 INFO:tasks.workunit.client.0.smithi153.stdout:2/45: dread f5 [0,4194304] 0 2024-03-21T10:08:34.881 INFO:tasks.workunit.client.0.smithi153.stdout:3/68: creat dd/f12 x:0 0 0 2024-03-21T10:08:34.881 INFO:tasks.workunit.client.0.smithi153.stdout:3/69: fdatasync dd/f10 0 2024-03-21T10:08:34.881 INFO:tasks.workunit.client.0.smithi153.stdout:5/38: mknod d4/d5/cc 0 2024-03-21T10:08:34.883 INFO:tasks.workunit.client.0.smithi153.stdout:3/70: creat dd/f13 x:0 0 0 2024-03-21T10:08:34.884 INFO:tasks.workunit.client.0.smithi153.stdout:3/71: write dd/f10 [952713,58221] 0 2024-03-21T10:08:34.885 INFO:tasks.workunit.client.0.smithi153.stdout:5/39: mknod d4/d5/cd 0 2024-03-21T10:08:34.889 INFO:tasks.workunit.client.0.smithi153.stdout:3/72: mknod dd/c14 0 2024-03-21T10:08:34.890 INFO:tasks.workunit.client.0.smithi153.stdout:6/37: dread f7 [0,4194304] 0 2024-03-21T10:08:34.891 INFO:tasks.workunit.client.0.smithi153.stdout:3/73: write dd/f12 [985454,62115] 0 2024-03-21T10:08:34.893 INFO:tasks.workunit.client.0.smithi153.stdout:6/38: symlink lc 0 2024-03-21T10:08:34.894 INFO:tasks.workunit.client.0.smithi153.stdout:3/74: truncate dd/f13 382842 0 2024-03-21T10:08:34.896 INFO:tasks.workunit.client.0.smithi153.stdout:3/75: creat dd/f15 x:0 0 0 2024-03-21T10:08:34.899 INFO:tasks.workunit.client.0.smithi153.stdout:6/39: truncate f4 2159821 0 2024-03-21T10:08:34.901 INFO:tasks.workunit.client.0.smithi153.stdout:5/40: dread f1 [0,4194304] 0 2024-03-21T10:08:34.903 INFO:tasks.workunit.client.0.smithi153.stdout:6/40: rename f8 to fd 0 2024-03-21T10:08:34.904 INFO:tasks.workunit.client.0.smithi153.stdout:5/41: rmdir d4 39 2024-03-21T10:08:34.909 INFO:tasks.workunit.client.0.smithi153.stdout:5/42: mkdir d4/d5/de 0 2024-03-21T10:08:34.912 INFO:tasks.workunit.client.0.smithi153.stdout:5/43: read d4/fa [3109392,16989] 0 2024-03-21T10:08:34.913 INFO:tasks.workunit.client.0.smithi153.stdout:5/44: dread - d4/fb zero size 2024-03-21T10:08:34.917 INFO:tasks.workunit.client.0.smithi153.stdout:3/76: dwrite f8 [0,4194304] 0 2024-03-21T10:08:34.922 INFO:tasks.workunit.client.0.smithi153.stdout:6/41: truncate f5 2944681 0 2024-03-21T10:08:34.926 INFO:tasks.workunit.client.0.smithi153.stdout:5/45: mkdir d4/d5/df 0 2024-03-21T10:08:34.927 INFO:tasks.workunit.client.0.smithi153.stdout:3/77: mknod dd/c16 0 2024-03-21T10:08:34.928 INFO:tasks.workunit.client.0.smithi153.stdout:5/46: chown d4/d5/df 46125 1 2024-03-21T10:08:34.928 INFO:tasks.workunit.client.0.smithi153.stdout:6/42: creat fe x:0 0 0 2024-03-21T10:08:34.929 INFO:tasks.workunit.client.0.smithi153.stdout:3/78: readlink l4 0 2024-03-21T10:08:34.931 INFO:tasks.workunit.client.0.smithi153.stdout:3/79: truncate dd/f15 745083 0 2024-03-21T10:08:34.931 INFO:tasks.workunit.client.0.smithi153.stdout:5/47: mknod d4/d5/de/c10 0 2024-03-21T10:08:34.932 INFO:tasks.workunit.client.0.smithi153.stdout:6/43: chown f5 4010 1 2024-03-21T10:08:34.934 INFO:tasks.workunit.client.0.smithi153.stdout:3/80: truncate dd/f15 832330 0 2024-03-21T10:08:34.934 INFO:tasks.workunit.client.0.smithi153.stdout:6/44: mknod cf 0 2024-03-21T10:08:34.934 INFO:tasks.workunit.client.0.smithi153.stdout:6/45: rmdir - no directory 2024-03-21T10:08:34.937 INFO:tasks.workunit.client.0.smithi153.stdout:6/46: mknod c10 0 2024-03-21T10:08:34.943 INFO:tasks.workunit.client.0.smithi153.stdout:5/48: getdents d4/d8 0 2024-03-21T10:08:34.943 INFO:tasks.workunit.client.0.smithi153.stdout:6/47: rename fe to f11 0 2024-03-21T10:08:34.943 INFO:tasks.workunit.client.0.smithi153.stdout:5/49: creat d4/d8/f11 x:0 0 0 2024-03-21T10:08:34.948 INFO:tasks.workunit.client.0.smithi153.stdout:3/81: getdents dd 0 2024-03-21T10:08:34.950 INFO:tasks.workunit.client.0.smithi153.stdout:5/50: creat d4/d5/f12 x:0 0 0 2024-03-21T10:08:34.952 INFO:tasks.workunit.client.0.smithi153.stdout:3/82: dread f6 [0,4194304] 0 2024-03-21T10:08:34.954 INFO:tasks.workunit.client.0.smithi153.stdout:3/83: dread - fc zero size 2024-03-21T10:08:34.956 INFO:tasks.workunit.client.0.smithi153.stdout:5/51: creat d4/d5/f13 x:0 0 0 2024-03-21T10:08:34.956 INFO:tasks.workunit.client.0.smithi153.stdout:5/52: readlink - no filename 2024-03-21T10:08:34.957 INFO:tasks.workunit.client.0.smithi153.stdout:3/84: symlink dd/l17 0 2024-03-21T10:08:34.960 INFO:tasks.workunit.client.0.smithi153.stdout:3/85: read dd/f15 [654801,22210] 0 2024-03-21T10:08:34.963 INFO:tasks.workunit.client.0.smithi153.stdout:3/86: truncate f7 4890220 0 2024-03-21T10:08:34.966 INFO:tasks.workunit.client.0.smithi153.stdout:3/87: truncate f2 1230792 0 2024-03-21T10:08:34.967 INFO:tasks.workunit.client.0.smithi153.stdout:6/48: dwrite fd [0,4194304] 0 2024-03-21T10:08:34.970 INFO:tasks.workunit.client.0.smithi153.stdout:3/88: unlink dd/c14 0 2024-03-21T10:08:34.971 INFO:tasks.workunit.client.0.smithi153.stdout:6/49: creat f12 x:0 0 0 2024-03-21T10:08:34.974 INFO:tasks.workunit.client.0.smithi153.stdout:3/89: creat dd/f18 x:0 0 0 2024-03-21T10:08:34.975 INFO:tasks.workunit.client.0.smithi153.stdout:6/50: symlink l13 0 2024-03-21T10:08:34.975 INFO:tasks.workunit.client.0.smithi153.stdout:3/90: chown dd/f15 1118426 1 2024-03-21T10:08:34.978 INFO:tasks.workunit.client.0.smithi153.stdout:3/91: mknod dd/c19 0 2024-03-21T10:08:34.981 INFO:tasks.workunit.client.0.smithi153.stdout:3/92: creat dd/f1a x:0 0 0 2024-03-21T10:08:34.984 INFO:tasks.workunit.client.0.smithi153.stdout:6/51: dread f4 [0,4194304] 0 2024-03-21T10:08:34.986 INFO:tasks.workunit.client.0.smithi153.stdout:6/52: unlink lc 0 2024-03-21T10:08:34.988 INFO:tasks.workunit.client.0.smithi153.stdout:6/53: symlink l14 0 2024-03-21T10:08:34.988 INFO:tasks.workunit.client.0.smithi153.stdout:6/54: dread - f11 zero size 2024-03-21T10:08:34.989 INFO:tasks.workunit.client.0.smithi153.stdout:6/55: dread - f11 zero size 2024-03-21T10:08:34.991 INFO:tasks.workunit.client.0.smithi153.stdout:6/56: chown f7 3708006 1 2024-03-21T10:08:34.992 INFO:tasks.workunit.client.0.smithi153.stdout:3/93: truncate f7 928650 0 2024-03-21T10:08:34.995 INFO:tasks.workunit.client.0.smithi153.stdout:3/94: chown f7 11941889 1 2024-03-21T10:08:34.996 INFO:tasks.workunit.client.0.smithi153.stdout:3/95: truncate f6 1270762 0 2024-03-21T10:08:34.998 INFO:tasks.workunit.client.0.smithi153.stdout:3/96: mknod dd/c1b 0 2024-03-21T10:08:34.998 INFO:tasks.workunit.client.0.smithi153.stdout:3/97: fdatasync fc 0 2024-03-21T10:08:34.999 INFO:tasks.workunit.client.0.smithi153.stdout:3/98: truncate fc 928062 0 2024-03-21T10:08:35.000 INFO:tasks.workunit.client.0.smithi153.stdout:3/99: dread - dd/f1a zero size 2024-03-21T10:08:35.002 INFO:tasks.workunit.client.0.smithi153.stdout:3/100: write fc [1506526,21992] 0 2024-03-21T10:08:35.006 INFO:tasks.workunit.client.0.smithi153.stdout:3/101: rename dd/f1a to dd/f1c 0 2024-03-21T10:08:35.020 INFO:tasks.workunit.client.0.smithi153.stdout:3/102: mknod dd/c1d 0 2024-03-21T10:08:35.021 INFO:tasks.workunit.client.0.smithi153.stdout:6/57: dwrite f11 [0,4194304] 0 2024-03-21T10:08:35.031 INFO:tasks.workunit.client.0.smithi153.stdout:6/58: dread fd [0,4194304] 0 2024-03-21T10:08:35.032 INFO:tasks.workunit.client.0.smithi153.stdout:6/59: write f11 [2310498,31727] 0 2024-03-21T10:08:35.083 INFO:tasks.workunit.client.0.smithi153.stdout:6/60: creat f15 x:0 0 0 2024-03-21T10:08:35.133 INFO:tasks.workunit.client.0.smithi153.stdout:6/61: dwrite f7 [0,4194304] 0 2024-03-21T10:08:35.133 INFO:tasks.workunit.client.0.smithi153.stdout:6/62: stat lb 0 2024-03-21T10:08:35.134 INFO:tasks.workunit.client.0.smithi153.stdout:6/63: chown c9 0 1 2024-03-21T10:08:35.157 INFO:tasks.workunit.client.0.smithi153.stdout:6/64: fsync f5 0 2024-03-21T10:08:35.162 INFO:tasks.workunit.client.0.smithi153.stdout:6/65: write fd [1728771,127322] 0 2024-03-21T10:08:35.166 INFO:tasks.workunit.client.0.smithi153.stdout:6/66: getdents . 0 2024-03-21T10:08:35.174 INFO:tasks.workunit.client.0.smithi153.stdout:6/67: link c6 c16 0 2024-03-21T10:08:35.174 INFO:tasks.workunit.client.0.smithi153.stdout:6/68: rmdir - no directory 2024-03-21T10:08:35.182 INFO:tasks.workunit.client.0.smithi153.stdout:6/69: dread f7 [0,4194304] 0 2024-03-21T10:08:35.190 INFO:tasks.workunit.client.0.smithi153.stdout:6/70: getdents . 0 2024-03-21T10:08:35.206 INFO:tasks.workunit.client.0.smithi153.stdout:6/71: dwrite f7 [4194304,4194304] 0 2024-03-21T10:08:35.223 INFO:tasks.workunit.client.0.smithi153.stdout:6/72: dwrite f11 [0,4194304] 0 2024-03-21T10:08:35.230 INFO:tasks.workunit.client.0.smithi153.stdout:6/73: link la l17 0 2024-03-21T10:08:35.231 INFO:tasks.workunit.client.0.smithi153.stdout:6/74: write f15 [95428,111057] 0 2024-03-21T10:08:35.234 INFO:tasks.workunit.client.0.smithi153.stdout:6/75: creat f18 x:0 0 0 2024-03-21T10:08:35.252 INFO:tasks.workunit.client.0.smithi153.stdout:6/76: dwrite f4 [0,4194304] 0 2024-03-21T10:08:35.253 INFO:tasks.workunit.client.0.smithi153.stdout:6/77: fsync f5 0 2024-03-21T10:08:35.258 INFO:tasks.workunit.client.0.smithi153.stdout:6/78: rename f12 to f19 0 2024-03-21T10:08:35.263 INFO:tasks.workunit.client.0.smithi153.stdout:6/79: link f11 f1a 0 2024-03-21T10:08:35.265 INFO:tasks.workunit.client.0.smithi153.stdout:6/80: chown l14 168 1 2024-03-21T10:08:35.276 INFO:tasks.workunit.client.0.smithi153.stdout:6/81: dread fd [0,4194304] 0 2024-03-21T10:08:35.279 INFO:tasks.workunit.client.0.smithi153.stdout:6/82: unlink lb 0 2024-03-21T10:08:35.280 INFO:tasks.workunit.client.0.smithi153.stdout:6/83: creat f1b x:0 0 0 2024-03-21T10:08:35.282 INFO:tasks.workunit.client.0.smithi153.stdout:6/84: mknod c1c 0 2024-03-21T10:08:35.297 INFO:tasks.workunit.client.0.smithi153.stdout:6/85: dwrite f4 [0,4194304] 0 2024-03-21T10:08:35.302 INFO:tasks.workunit.client.0.smithi153.stdout:6/86: getdents . 0 2024-03-21T10:08:35.302 INFO:tasks.workunit.client.0.smithi153.stdout:6/87: chown l14 670 1 2024-03-21T10:08:35.323 INFO:tasks.workunit.client.0.smithi153.stdout:6/88: dwrite f19 [0,4194304] 0 2024-03-21T10:08:35.325 INFO:tasks.workunit.client.0.smithi153.stdout:6/89: write fd [3951660,26669] 0 2024-03-21T10:08:35.330 INFO:tasks.workunit.client.0.smithi153.stdout:6/90: creat f1d x:0 0 0 2024-03-21T10:08:35.337 INFO:tasks.workunit.client.0.smithi153.stdout:6/91: getdents . 0 2024-03-21T10:08:35.337 INFO:tasks.workunit.client.0.smithi153.stdout:6/92: fsync f18 0 2024-03-21T10:08:35.338 INFO:tasks.workunit.client.0.smithi153.stdout:6/93: dread - f1d zero size 2024-03-21T10:08:35.339 INFO:tasks.workunit.client.0.smithi153.stdout:6/94: symlink l1e 0 2024-03-21T10:08:35.346 INFO:tasks.workunit.client.0.smithi153.stdout:6/95: truncate f4 2420479 0 2024-03-21T10:08:35.347 INFO:tasks.workunit.client.0.smithi153.stdout:6/96: mkdir d1f 0 2024-03-21T10:08:35.348 INFO:tasks.workunit.client.0.smithi153.stdout:6/97: chown f1d 23083 1 2024-03-21T10:08:35.358 INFO:tasks.workunit.client.0.smithi153.stdout:6/98: dread f11 [0,4194304] 0 2024-03-21T10:08:35.358 INFO:tasks.workunit.client.0.smithi153.stdout:6/99: fdatasync fd 0 2024-03-21T10:08:35.359 INFO:tasks.workunit.client.0.smithi153.stdout:6/100: chown d1f 8 1 2024-03-21T10:08:35.360 INFO:tasks.workunit.client.0.smithi153.stdout:6/101: chown f15 4109744 1 2024-03-21T10:08:35.360 INFO:tasks.workunit.client.0.smithi153.stdout:6/102: dread - f1d zero size 2024-03-21T10:08:35.367 INFO:tasks.workunit.client.0.smithi153.stdout:6/103: rename f4 to d1f/f20 0 2024-03-21T10:08:35.377 INFO:tasks.workunit.client.0.smithi153.stdout:6/104: dread fd [0,4194304] 0 2024-03-21T10:08:35.378 INFO:tasks.workunit.client.0.smithi153.stdout:6/105: dread - f1d zero size 2024-03-21T10:08:35.380 INFO:tasks.workunit.client.0.smithi153.stdout:6/106: mknod d1f/c21 0 2024-03-21T10:08:35.382 INFO:tasks.workunit.client.0.smithi153.stdout:6/107: mkdir d1f/d22 0 2024-03-21T10:08:35.391 INFO:tasks.workunit.client.0.smithi153.stdout:6/108: dread fd [0,4194304] 0 2024-03-21T10:08:35.393 INFO:tasks.workunit.client.0.smithi153.stdout:6/109: creat d1f/f23 x:0 0 0 2024-03-21T10:08:35.395 INFO:tasks.workunit.client.0.smithi153.stdout:6/110: symlink d1f/l24 0 2024-03-21T10:08:35.410 INFO:tasks.workunit.client.0.smithi153.stdout:6/111: dwrite f1d [0,4194304] 0 2024-03-21T10:08:35.435 INFO:tasks.workunit.client.0.smithi153.stdout:6/112: dwrite d1f/f23 [0,4194304] 0 2024-03-21T10:08:35.441 INFO:tasks.workunit.client.0.smithi153.stdout:6/113: creat d1f/f25 x:0 0 0 2024-03-21T10:08:35.444 INFO:tasks.workunit.client.0.smithi153.stdout:6/114: link d1f/f25 d1f/f26 0 2024-03-21T10:08:35.448 INFO:tasks.workunit.client.0.smithi153.stdout:6/115: rmdir d1f/d22 0 2024-03-21T10:08:35.449 INFO:tasks.workunit.client.0.smithi153.stdout:6/116: fdatasync f11 0 2024-03-21T10:08:35.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:35 smithi174 ceph-mon[37496]: [21/Mar/2024:10:08:33] ENGINE Bus STARTING 2024-03-21T10:08:35.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:35 smithi174 ceph-mon[37496]: [21/Mar/2024:10:08:33] ENGINE Serving on https://172.21.15.153:7150 2024-03-21T10:08:35.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:35 smithi174 ceph-mon[37496]: [21/Mar/2024:10:08:34] ENGINE Serving on http://172.21.15.153:8765 2024-03-21T10:08:35.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:35 smithi174 ceph-mon[37496]: [21/Mar/2024:10:08:34] ENGINE Bus STARTED 2024-03-21T10:08:35.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:35 smithi174 ceph-mon[37496]: mgrmap e27: smithi153.tluxni(active, since 1.02275s) 2024-03-21T10:08:35.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:35 smithi174 ceph-mon[37496]: pgmap v3: 65 pgs: 65 active+clean; 220 MiB data, 878 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:08:35.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:35 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:08:35.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:35 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:08:35.452 INFO:tasks.workunit.client.0.smithi153.stdout:6/117: creat d1f/f27 x:0 0 0 2024-03-21T10:08:35.453 INFO:tasks.workunit.client.0.smithi153.stdout:6/118: chown f1a 104433 1 2024-03-21T10:08:35.454 INFO:tasks.workunit.client.0.smithi153.stdout:6/119: creat d1f/f28 x:0 0 0 2024-03-21T10:08:35.455 INFO:tasks.workunit.client.0.smithi153.stdout:6/120: stat f15 0 2024-03-21T10:08:35.464 INFO:tasks.workunit.client.0.smithi153.stdout:6/121: getdents d1f 0 2024-03-21T10:08:35.466 INFO:tasks.workunit.client.0.smithi153.stdout:6/122: chown l17 401534503 1 2024-03-21T10:08:35.470 INFO:tasks.workunit.client.0.smithi153.stdout:6/123: truncate f5 1898666 0 2024-03-21T10:08:35.471 INFO:tasks.workunit.client.0.smithi153.stdout:6/124: readlink d1f/l24 0 2024-03-21T10:08:35.472 INFO:tasks.workunit.client.0.smithi153.stdout:6/125: write f1a [3115176,61365] 0 2024-03-21T10:08:35.473 INFO:tasks.workunit.client.0.smithi153.stdout:6/126: dread - f18 zero size 2024-03-21T10:08:35.475 INFO:tasks.workunit.client.0.smithi153.stdout:6/127: mknod d1f/c29 0 2024-03-21T10:08:35.498 INFO:tasks.workunit.client.0.smithi153.stdout:6/128: dwrite f5 [0,4194304] 0 2024-03-21T10:08:35.501 INFO:tasks.workunit.client.0.smithi153.stdout:6/129: symlink d1f/l2a 0 2024-03-21T10:08:35.504 INFO:tasks.workunit.client.0.smithi153.stdout:6/130: mkdir d1f/d2b 0 2024-03-21T10:08:35.505 INFO:tasks.workunit.client.0.smithi153.stdout:6/131: dread - d1f/f27 zero size 2024-03-21T10:08:35.520 INFO:tasks.workunit.client.0.smithi153.stdout:6/132: dwrite f5 [0,4194304] 0 2024-03-21T10:08:35.521 INFO:tasks.workunit.client.0.smithi153.stdout:6/133: readlink l1e 0 2024-03-21T10:08:35.522 INFO:tasks.workunit.client.0.smithi153.stdout:6/134: chown d1f/f27 758762898 1 2024-03-21T10:08:35.524 INFO:tasks.workunit.client.0.smithi153.stdout:6/135: read - d1f/f25 zero size 2024-03-21T10:08:35.542 INFO:tasks.workunit.client.0.smithi153.stdout:6/136: dwrite f11 [0,4194304] 0 2024-03-21T10:08:35.544 INFO:tasks.workunit.client.0.smithi153.stdout:6/137: dread - d1f/f27 zero size 2024-03-21T10:08:35.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:35 smithi153 ceph-mon[29117]: [21/Mar/2024:10:08:33] ENGINE Bus STARTING 2024-03-21T10:08:35.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:35 smithi153 ceph-mon[29117]: [21/Mar/2024:10:08:33] ENGINE Serving on https://172.21.15.153:7150 2024-03-21T10:08:35.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:35 smithi153 ceph-mon[29117]: [21/Mar/2024:10:08:34] ENGINE Serving on http://172.21.15.153:8765 2024-03-21T10:08:35.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:35 smithi153 ceph-mon[29117]: [21/Mar/2024:10:08:34] ENGINE Bus STARTED 2024-03-21T10:08:35.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:35 smithi153 ceph-mon[29117]: mgrmap e27: smithi153.tluxni(active, since 1.02275s) 2024-03-21T10:08:35.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:35 smithi153 ceph-mon[29117]: pgmap v3: 65 pgs: 65 active+clean; 220 MiB data, 878 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:08:35.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:35 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:08:35.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:35 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:08:35.561 INFO:tasks.workunit.client.0.smithi153.stdout:6/138: dwrite f5 [0,4194304] 0 2024-03-21T10:08:35.563 INFO:tasks.workunit.client.0.smithi153.stdout:6/139: chown d1f/c21 5 1 2024-03-21T10:08:35.564 INFO:tasks.workunit.client.0.smithi153.stdout:6/140: readlink l1e 0 2024-03-21T10:08:35.566 INFO:tasks.workunit.client.0.smithi153.stdout:6/141: chown d1f/f27 44008 1 2024-03-21T10:08:35.585 INFO:tasks.workunit.client.0.smithi153.stdout:6/142: dwrite d1f/f28 [0,4194304] 0 2024-03-21T10:08:35.590 INFO:tasks.workunit.client.0.smithi153.stdout:6/143: rename f1d to d1f/d2b/f2c 0 2024-03-21T10:08:35.593 INFO:tasks.workunit.client.0.smithi153.stdout:6/144: creat d1f/f2d x:0 0 0 2024-03-21T10:08:35.595 INFO:tasks.workunit.client.0.smithi153.stdout:6/145: chown d1f/f25 2912581 1 2024-03-21T10:08:35.618 INFO:tasks.workunit.client.0.smithi153.stdout:6/146: dwrite fd [0,4194304] 0 2024-03-21T10:08:35.656 INFO:tasks.workunit.client.0.smithi153.stdout:6/147: dwrite d1f/f26 [0,4194304] 0 2024-03-21T10:08:35.659 INFO:tasks.workunit.client.0.smithi153.stdout:6/148: symlink d1f/l2e 0 2024-03-21T10:08:35.661 INFO:tasks.workunit.client.0.smithi153.stdout:6/149: mkdir d1f/d2f 0 2024-03-21T10:08:35.664 INFO:tasks.workunit.client.0.smithi153.stdout:6/150: mknod d1f/d2b/c30 0 2024-03-21T10:08:35.668 INFO:tasks.workunit.client.0.smithi153.stdout:6/151: symlink d1f/l31 0 2024-03-21T10:08:35.672 INFO:tasks.workunit.client.0.smithi153.stdout:6/152: creat d1f/d2f/f32 x:0 0 0 2024-03-21T10:08:35.688 INFO:tasks.workunit.client.0.smithi153.stdout:6/153: rmdir d1f 39 2024-03-21T10:08:35.693 INFO:tasks.workunit.client.0.smithi153.stdout:6/154: unlink d1f/f26 0 2024-03-21T10:08:35.696 INFO:tasks.workunit.client.0.smithi153.stdout:6/155: mkdir d1f/d2b/d33 0 2024-03-21T10:08:35.697 INFO:tasks.workunit.client.0.smithi153.stdout:6/156: read - f1b zero size 2024-03-21T10:08:35.701 INFO:tasks.workunit.client.0.smithi153.stdout:6/157: creat d1f/d2b/d33/f34 x:0 0 0 2024-03-21T10:08:35.702 INFO:tasks.workunit.client.0.smithi153.stdout:6/158: readlink d1f/l2e 0 2024-03-21T10:08:35.704 INFO:tasks.workunit.client.0.smithi153.stdout:6/159: chown d1f/l2e 1 1 2024-03-21T10:08:35.708 INFO:tasks.workunit.client.0.smithi153.stdout:6/160: creat d1f/f35 x:0 0 0 2024-03-21T10:08:35.726 INFO:tasks.workunit.client.0.smithi153.stdout:6/161: truncate f5 32073 0 2024-03-21T10:08:35.754 INFO:tasks.workunit.client.0.smithi153.stdout:6/162: creat d1f/f36 x:0 0 0 2024-03-21T10:08:35.774 INFO:tasks.workunit.client.0.smithi153.stdout:6/163: dwrite f11 [0,4194304] 0 2024-03-21T10:08:35.790 INFO:tasks.workunit.client.0.smithi153.stdout:6/164: dwrite d1f/f2d [0,4194304] 0 2024-03-21T10:08:35.806 INFO:tasks.workunit.client.0.smithi153.stdout:6/165: dwrite f19 [0,4194304] 0 2024-03-21T10:08:35.809 INFO:tasks.workunit.client.0.smithi153.stdout:6/166: symlink d1f/d2b/l37 0 2024-03-21T10:08:35.811 INFO:tasks.workunit.client.0.smithi153.stdout:6/167: dread - f1b zero size 2024-03-21T10:08:35.815 INFO:tasks.workunit.client.0.smithi153.stdout:6/168: creat d1f/d2b/f38 x:0 0 0 2024-03-21T10:08:35.818 INFO:tasks.workunit.client.0.smithi153.stdout:6/169: write d1f/f23 [3238893,22107] 0 2024-03-21T10:08:35.820 INFO:tasks.workunit.client.0.smithi153.stdout:6/170: write f18 [815424,36958] 0 2024-03-21T10:08:35.823 INFO:tasks.workunit.client.0.smithi153.stdout:6/171: creat d1f/f39 x:0 0 0 2024-03-21T10:08:35.824 INFO:tasks.workunit.client.0.smithi153.stdout:6/172: chown la 385 1 2024-03-21T10:08:35.828 INFO:tasks.workunit.client.0.smithi153.stdout:6/173: getdents d1f/d2b 0 2024-03-21T10:08:35.831 INFO:tasks.workunit.client.0.smithi153.stdout:6/174: unlink f7 0 2024-03-21T10:08:35.837 INFO:tasks.workunit.client.0.smithi153.stdout:6/175: truncate fd 3661088 0 2024-03-21T10:08:35.838 INFO:tasks.workunit.client.0.smithi153.stdout:6/176: dread - d1f/d2f/f32 zero size 2024-03-21T10:08:36.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:36 smithi174 ceph-mon[37496]: pgmap v4: 65 pgs: 65 active+clean; 220 MiB data, 878 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:08:36.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:36 smithi153 ceph-mon[29117]: pgmap v4: 65 pgs: 65 active+clean; 220 MiB data, 878 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:08:37.375 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:37 smithi174 ceph-mon[37496]: mgrmap e28: smithi153.tluxni(active, since 3s) 2024-03-21T10:08:37.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:37 smithi153 ceph-mon[29117]: mgrmap e28: smithi153.tluxni(active, since 3s) 2024-03-21T10:08:38.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:38 smithi174 ceph-mon[37496]: pgmap v5: 65 pgs: 65 active+clean; 220 MiB data, 878 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:08:38.480 INFO:tasks.workunit.client.0.smithi153.stdout:0/11: sync 2024-03-21T10:08:38.483 INFO:tasks.workunit.client.0.smithi153.stdout:0/12: fsync f2 0 2024-03-21T10:08:38.485 INFO:tasks.workunit.client.0.smithi153.stdout:0/13: fsync f2 0 2024-03-21T10:08:38.487 INFO:tasks.workunit.client.0.smithi153.stdout:0/14: chown f2 3572 1 2024-03-21T10:08:38.491 INFO:tasks.workunit.client.0.smithi153.stdout:0/15: mknod c3 0 2024-03-21T10:08:38.493 INFO:tasks.workunit.client.0.smithi153.stdout:0/16: chown c0 83585 1 2024-03-21T10:08:38.497 INFO:tasks.workunit.client.0.smithi153.stdout:0/17: dread - f2 zero size 2024-03-21T10:08:38.500 INFO:tasks.workunit.client.0.smithi153.stdout:0/18: mkdir d4 0 2024-03-21T10:08:38.500 INFO:tasks.workunit.client.0.smithi153.stdout:0/19: readlink - no filename 2024-03-21T10:08:38.503 INFO:tasks.workunit.client.0.smithi153.stdout:0/20: truncate f2 468723 0 2024-03-21T10:08:38.509 INFO:tasks.workunit.client.0.smithi153.stdout:0/21: rename c0 to d4/c5 0 2024-03-21T10:08:38.512 INFO:tasks.workunit.client.0.smithi153.stdout:0/22: read f2 [139207,38439] 0 2024-03-21T10:08:38.552 INFO:tasks.workunit.client.0.smithi153.stdout:6/177: sync 2024-03-21T10:08:38.552 INFO:tasks.workunit.client.0.smithi153.stdout:1/24: sync 2024-03-21T10:08:38.552 INFO:tasks.workunit.client.0.smithi153.stdout:7/28: sync 2024-03-21T10:08:38.552 INFO:tasks.workunit.client.0.smithi153.stdout:3/103: sync 2024-03-21T10:08:38.552 INFO:tasks.workunit.client.0.smithi153.stdout:9/26: sync 2024-03-21T10:08:38.552 INFO:tasks.workunit.client.0.smithi153.stdout:4/34: sync 2024-03-21T10:08:38.552 INFO:tasks.workunit.client.0.smithi153.stdout:2/46: sync 2024-03-21T10:08:38.552 INFO:tasks.workunit.client.0.smithi153.stdout:8/21: sync 2024-03-21T10:08:38.553 INFO:tasks.workunit.client.0.smithi153.stdout:5/53: sync 2024-03-21T10:08:38.556 INFO:tasks.workunit.client.0.smithi153.stdout:1/25: dread - f5 zero size 2024-03-21T10:08:38.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:38 smithi153 ceph-mon[29117]: pgmap v5: 65 pgs: 65 active+clean; 220 MiB data, 878 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:08:38.559 INFO:tasks.workunit.client.0.smithi153.stdout:2/47: symlink dd/l11 0 2024-03-21T10:08:38.559 INFO:tasks.workunit.client.0.smithi153.stdout:1/26: write f0 [449455,64392] 0 2024-03-21T10:08:38.561 INFO:tasks.workunit.client.0.smithi153.stdout:2/48: chown l1 229184 1 2024-03-21T10:08:38.563 INFO:tasks.workunit.client.0.smithi153.stdout:5/54: mknod d4/d8/c14 0 2024-03-21T10:08:38.563 INFO:tasks.workunit.client.0.smithi153.stdout:9/27: dread d1/d4/f7 [0,4194304] 0 2024-03-21T10:08:38.565 INFO:tasks.workunit.client.0.smithi153.stdout:4/35: mknod cf 0 2024-03-21T10:08:38.568 INFO:tasks.workunit.client.0.smithi153.stdout:6/178: rename l14 to d1f/d2b/l3a 0 2024-03-21T10:08:38.568 INFO:tasks.workunit.client.0.smithi153.stdout:6/179: readlink l17 0 2024-03-21T10:08:38.570 INFO:tasks.workunit.client.0.smithi153.stdout:3/104: dread dd/f12 [0,4194304] 0 2024-03-21T10:08:38.573 INFO:tasks.workunit.client.0.smithi153.stdout:2/49: creat dd/f12 x:0 0 0 2024-03-21T10:08:38.574 INFO:tasks.workunit.client.0.smithi153.stdout:2/50: chown f7 447957 1 2024-03-21T10:08:38.576 INFO:tasks.workunit.client.0.smithi153.stdout:4/36: creat f10 x:0 0 0 2024-03-21T10:08:38.577 INFO:tasks.workunit.client.0.smithi153.stdout:8/22: mkdir d5/d6 0 2024-03-21T10:08:38.580 INFO:tasks.workunit.client.0.smithi153.stdout:8/23: fdatasync f3 0 2024-03-21T10:08:38.582 INFO:tasks.workunit.client.0.smithi153.stdout:1/27: creat d6/f9 x:0 0 0 2024-03-21T10:08:38.584 INFO:tasks.workunit.client.0.smithi153.stdout:3/105: unlink fc 0 2024-03-21T10:08:38.585 INFO:tasks.workunit.client.0.smithi153.stdout:1/28: dread - d6/f9 zero size 2024-03-21T10:08:38.586 INFO:tasks.workunit.client.0.smithi153.stdout:7/29: creat d4/f9 x:0 0 0 2024-03-21T10:08:38.588 INFO:tasks.workunit.client.0.smithi153.stdout:1/29: write f0 [850534,22250] 0 2024-03-21T10:08:38.590 INFO:tasks.workunit.client.0.smithi153.stdout:2/51: creat dd/f13 x:0 0 0 2024-03-21T10:08:38.592 INFO:tasks.workunit.client.0.smithi153.stdout:6/180: dwrite d1f/f2d [0,4194304] 0 2024-03-21T10:08:38.592 INFO:tasks.workunit.client.0.smithi153.stdout:4/37: mkdir d11 0 2024-03-21T10:08:38.593 INFO:tasks.workunit.client.0.smithi153.stdout:4/38: chown f10 1915303 1 2024-03-21T10:08:38.597 INFO:tasks.workunit.client.0.smithi153.stdout:3/106: mkdir dd/d1e 0 2024-03-21T10:08:38.597 INFO:tasks.workunit.client.0.smithi153.stdout:3/107: stat l9 0 2024-03-21T10:08:38.603 INFO:tasks.workunit.client.0.smithi153.stdout:1/30: unlink f0 0 2024-03-21T10:08:38.605 INFO:tasks.workunit.client.0.smithi153.stdout:9/28: link c0 d1/d4/cb 0 2024-03-21T10:08:38.606 INFO:tasks.workunit.client.0.smithi153.stdout:2/52: fsync f5 0 2024-03-21T10:08:38.608 INFO:tasks.workunit.client.0.smithi153.stdout:1/31: truncate f2 959860 0 2024-03-21T10:08:38.608 INFO:tasks.workunit.client.0.smithi153.stdout:8/24: creat d5/d6/f7 x:0 0 0 2024-03-21T10:08:38.611 INFO:tasks.workunit.client.0.smithi153.stdout:6/181: creat d1f/d2f/f3b x:0 0 0 2024-03-21T10:08:38.611 INFO:tasks.workunit.client.0.smithi153.stdout:5/55: getdents d4 0 2024-03-21T10:08:38.613 INFO:tasks.workunit.client.0.smithi153.stdout:3/108: creat dd/f1f x:0 0 0 2024-03-21T10:08:38.614 INFO:tasks.workunit.client.0.smithi153.stdout:4/39: dwrite f2 [0,4194304] 0 2024-03-21T10:08:38.615 INFO:tasks.workunit.client.0.smithi153.stdout:3/109: chown lb 27903 1 2024-03-21T10:08:38.615 INFO:tasks.workunit.client.0.smithi153.stdout:4/40: chown f1 17 1 2024-03-21T10:08:38.617 INFO:tasks.workunit.client.0.smithi153.stdout:2/53: creat dd/f14 x:0 0 0 2024-03-21T10:08:38.618 INFO:tasks.workunit.client.0.smithi153.stdout:2/54: readlink dd/l11 0 2024-03-21T10:08:38.620 INFO:tasks.workunit.client.0.smithi153.stdout:1/32: unlink d6/f8 0 2024-03-21T10:08:38.621 INFO:tasks.workunit.client.0.smithi153.stdout:9/29: creat d1/d4/fc x:0 0 0 2024-03-21T10:08:38.623 INFO:tasks.workunit.client.0.smithi153.stdout:6/182: unlink d1f/d2f/f3b 0 2024-03-21T10:08:38.625 INFO:tasks.workunit.client.0.smithi153.stdout:5/56: mknod d4/d5/de/c15 0 2024-03-21T10:08:38.628 INFO:tasks.workunit.client.0.smithi153.stdout:8/25: fsync f4 0 2024-03-21T10:08:38.630 INFO:tasks.workunit.client.0.smithi153.stdout:2/55: mkdir dd/d15 0 2024-03-21T10:08:38.630 INFO:tasks.workunit.client.0.smithi153.stdout:2/56: stat ca 0 2024-03-21T10:08:38.636 INFO:tasks.workunit.client.0.smithi153.stdout:5/57: mkdir d4/d5/d16 0 2024-03-21T10:08:38.638 INFO:tasks.workunit.client.0.smithi153.stdout:4/41: creat d11/f12 x:0 0 0 2024-03-21T10:08:38.640 INFO:tasks.workunit.client.0.smithi153.stdout:4/42: truncate f1 4582187 0 2024-03-21T10:08:38.640 INFO:tasks.workunit.client.0.smithi153.stdout:5/58: truncate d4/d5/f12 15345 0 2024-03-21T10:08:38.641 INFO:tasks.workunit.client.0.smithi153.stdout:4/43: readlink l9 0 2024-03-21T10:08:38.643 INFO:tasks.workunit.client.0.smithi153.stdout:1/33: link d6/f9 d6/fa 0 2024-03-21T10:08:38.644 INFO:tasks.workunit.client.0.smithi153.stdout:4/44: readlink l9 0 2024-03-21T10:08:38.649 INFO:tasks.workunit.client.0.smithi153.stdout:4/45: read f1 [2665267,87137] 0 2024-03-21T10:08:38.652 INFO:tasks.workunit.client.0.smithi153.stdout:5/59: creat d4/d5/f17 x:0 0 0 2024-03-21T10:08:38.652 INFO:tasks.workunit.client.0.smithi153.stdout:1/34: creat d6/fb x:0 0 0 2024-03-21T10:08:38.655 INFO:tasks.workunit.client.0.smithi153.stdout:8/26: dwrite d5/d6/f7 [0,4194304] 0 2024-03-21T10:08:38.655 INFO:tasks.workunit.client.0.smithi153.stdout:5/60: dread - d4/d5/f17 zero size 2024-03-21T10:08:38.655 INFO:tasks.workunit.client.0.smithi153.stdout:1/35: write f2 [250143,24486] 0 2024-03-21T10:08:38.657 INFO:tasks.workunit.client.0.smithi153.stdout:4/46: symlink d11/l13 0 2024-03-21T10:08:38.663 INFO:tasks.workunit.client.0.smithi153.stdout:4/47: symlink d11/l14 0 2024-03-21T10:08:38.667 INFO:tasks.workunit.client.0.smithi153.stdout:4/48: symlink d11/l15 0 2024-03-21T10:08:38.669 INFO:tasks.workunit.client.0.smithi153.stdout:4/49: dread - d11/f12 zero size 2024-03-21T10:08:38.678 INFO:tasks.workunit.client.0.smithi153.stdout:5/61: dread f1 [0,4194304] 0 2024-03-21T10:08:38.686 INFO:tasks.workunit.client.0.smithi153.stdout:5/62: write d4/d8/f11 [166444,58269] 0 2024-03-21T10:08:38.686 INFO:tasks.workunit.client.0.smithi153.stdout:1/36: dwrite f5 [0,4194304] 0 2024-03-21T10:08:38.692 INFO:tasks.workunit.client.0.smithi153.stdout:5/63: chown d4/d5/df 209832617 1 2024-03-21T10:08:38.698 INFO:tasks.workunit.client.0.smithi153.stdout:5/64: rename d4/d5/f13 to d4/d5/de/f18 0 2024-03-21T10:08:38.708 INFO:tasks.workunit.client.0.smithi153.stdout:5/65: creat d4/d5/d16/f19 x:0 0 0 2024-03-21T10:08:38.713 INFO:tasks.workunit.client.0.smithi153.stdout:5/66: creat d4/d5/f1a x:0 0 0 2024-03-21T10:08:38.715 INFO:tasks.workunit.client.0.smithi153.stdout:5/67: stat c3 0 2024-03-21T10:08:38.718 INFO:tasks.workunit.client.0.smithi153.stdout:1/37: dwrite d6/fb [0,4194304] 0 2024-03-21T10:08:38.726 INFO:tasks.workunit.client.0.smithi153.stdout:1/38: creat d6/fc x:0 0 0 2024-03-21T10:08:38.745 INFO:tasks.workunit.client.0.smithi153.stdout:5/68: dwrite d4/d5/f12 [0,4194304] 0 2024-03-21T10:08:38.751 INFO:tasks.workunit.client.0.smithi153.stdout:5/69: creat d4/d5/df/f1b x:0 0 0 2024-03-21T10:08:38.756 INFO:tasks.workunit.client.0.smithi153.stdout:0/23: truncate f2 863018 0 2024-03-21T10:08:38.759 INFO:tasks.workunit.client.0.smithi153.stdout:0/24: creat d4/f6 x:0 0 0 2024-03-21T10:08:38.762 INFO:tasks.workunit.client.0.smithi153.stdout:0/25: symlink d4/l7 0 2024-03-21T10:08:38.764 INFO:tasks.workunit.client.0.smithi153.stdout:0/26: symlink d4/l8 0 2024-03-21T10:08:38.767 INFO:tasks.workunit.client.0.smithi153.stdout:0/27: chown d4/c5 876942 1 2024-03-21T10:08:38.768 INFO:tasks.workunit.client.0.smithi153.stdout:7/30: getdents d4 0 2024-03-21T10:08:38.769 INFO:tasks.workunit.client.0.smithi153.stdout:0/28: symlink d4/l9 0 2024-03-21T10:08:38.773 INFO:tasks.workunit.client.0.smithi153.stdout:7/31: truncate f2 1285775 0 2024-03-21T10:08:38.776 INFO:tasks.workunit.client.0.smithi153.stdout:7/32: creat d4/fa x:0 0 0 2024-03-21T10:08:38.778 INFO:tasks.workunit.client.0.smithi153.stdout:3/110: rmdir dd 39 2024-03-21T10:08:38.782 INFO:tasks.workunit.client.0.smithi153.stdout:7/33: mknod d4/cb 0 2024-03-21T10:08:38.783 INFO:tasks.workunit.client.0.smithi153.stdout:3/111: fsync f0 0 2024-03-21T10:08:38.784 INFO:tasks.workunit.client.0.smithi153.stdout:7/34: write d4/fa [40740,123557] 0 2024-03-21T10:08:38.788 INFO:tasks.workunit.client.0.smithi153.stdout:2/57: getdents dd 0 2024-03-21T10:08:38.789 INFO:tasks.workunit.client.0.smithi153.stdout:2/58: stat l8 0 2024-03-21T10:08:38.790 INFO:tasks.workunit.client.0.smithi153.stdout:9/30: write d1/d4/f7 [1247983,112432] 0 2024-03-21T10:08:38.791 INFO:tasks.workunit.client.0.smithi153.stdout:2/59: read - dd/f10 zero size 2024-03-21T10:08:38.791 INFO:tasks.workunit.client.0.smithi153.stdout:3/112: creat dd/f20 x:0 0 0 2024-03-21T10:08:38.793 INFO:tasks.workunit.client.0.smithi153.stdout:2/60: write dd/f14 [866452,82808] 0 2024-03-21T10:08:38.795 INFO:tasks.workunit.client.0.smithi153.stdout:2/61: write f7 [704881,71683] 0 2024-03-21T10:08:38.796 INFO:tasks.workunit.client.0.smithi153.stdout:9/31: rename d1/d4/c5 to d1/cd 0 2024-03-21T10:08:38.797 INFO:tasks.workunit.client.0.smithi153.stdout:2/62: dread - dd/f13 zero size 2024-03-21T10:08:38.799 INFO:tasks.workunit.client.0.smithi153.stdout:9/32: chown d1/d4/f7 64353410 1 2024-03-21T10:08:38.800 INFO:tasks.workunit.client.0.smithi153.stdout:3/113: write dd/f1c [187778,9727] 0 2024-03-21T10:08:38.801 INFO:tasks.workunit.client.0.smithi153.stdout:8/27: write d5/d6/f7 [4907543,108797] 0 2024-03-21T10:08:38.806 INFO:tasks.workunit.client.0.smithi153.stdout:4/50: getdents d11 0 2024-03-21T10:08:38.811 INFO:tasks.workunit.client.0.smithi153.stdout:3/114: unlink dd/l17 0 2024-03-21T10:08:38.812 INFO:tasks.workunit.client.0.smithi153.stdout:2/63: dread dd/f14 [0,4194304] 0 2024-03-21T10:08:38.813 INFO:tasks.workunit.client.0.smithi153.stdout:9/33: dread d1/d4/f7 [0,4194304] 0 2024-03-21T10:08:38.817 INFO:tasks.workunit.client.0.smithi153.stdout:3/115: truncate dd/f1c 1235288 0 2024-03-21T10:08:38.820 INFO:tasks.workunit.client.0.smithi153.stdout:1/39: getdents d6 0 2024-03-21T10:08:38.820 INFO:tasks.workunit.client.0.smithi153.stdout:6/183: dwrite d1f/f25 [4194304,4194304] 0 2024-03-21T10:08:38.823 INFO:tasks.workunit.client.0.smithi153.stdout:2/64: dread dd/f14 [0,4194304] 0 2024-03-21T10:08:38.826 INFO:tasks.workunit.client.0.smithi153.stdout:9/34: creat d1/d4/fe x:0 0 0 2024-03-21T10:08:38.828 INFO:tasks.workunit.client.0.smithi153.stdout:3/116: rename dd/cf to dd/c21 0 2024-03-21T10:08:38.828 INFO:tasks.workunit.client.0.smithi153.stdout:6/184: chown d1f/c21 48878 1 2024-03-21T10:08:38.830 INFO:tasks.workunit.client.0.smithi153.stdout:1/40: truncate d6/fc 206217 0 2024-03-21T10:08:38.831 INFO:tasks.workunit.client.0.smithi153.stdout:2/65: chown dd/cf 609 1 2024-03-21T10:08:38.833 INFO:tasks.workunit.client.0.smithi153.stdout:8/28: write f4 [661888,122545] 0 2024-03-21T10:08:38.835 INFO:tasks.workunit.client.0.smithi153.stdout:9/35: chown d1/d4/la 2095 1 2024-03-21T10:08:38.840 INFO:tasks.workunit.client.0.smithi153.stdout:2/66: read - dd/f12 zero size 2024-03-21T10:08:38.844 INFO:tasks.workunit.client.0.smithi153.stdout:1/41: write d6/f7 [1231367,66524] 0 2024-03-21T10:08:38.844 INFO:tasks.workunit.client.0.smithi153.stdout:6/185: creat d1f/d2b/d33/f3c x:0 0 0 2024-03-21T10:08:38.844 INFO:tasks.workunit.client.0.smithi153.stdout:9/36: dread - d1/d4/fc zero size 2024-03-21T10:08:38.844 INFO:tasks.workunit.client.0.smithi153.stdout:4/51: dwrite fb [0,4194304] 0 2024-03-21T10:08:38.845 INFO:tasks.workunit.client.0.smithi153.stdout:8/29: creat d5/f8 x:0 0 0 2024-03-21T10:08:38.848 INFO:tasks.workunit.client.0.smithi153.stdout:4/52: dread - fe zero size 2024-03-21T10:08:38.852 INFO:tasks.workunit.client.0.smithi153.stdout:8/30: write f3 [4684930,101119] 0 2024-03-21T10:08:38.854 INFO:tasks.workunit.client.0.smithi153.stdout:9/37: truncate d1/d4/fe 673652 0 2024-03-21T10:08:38.857 INFO:tasks.workunit.client.0.smithi153.stdout:4/53: write f2 [2629679,78598] 0 2024-03-21T10:08:38.861 INFO:tasks.workunit.client.0.smithi153.stdout:3/117: truncate f2 1087564 0 2024-03-21T10:08:38.862 INFO:tasks.workunit.client.0.smithi153.stdout:3/118: read - dd/f18 zero size 2024-03-21T10:08:38.862 INFO:tasks.workunit.client.0.smithi153.stdout:9/38: truncate d1/d4/fc 648385 0 2024-03-21T10:08:38.866 INFO:tasks.workunit.client.0.smithi153.stdout:0/29: dwrite f2 [0,4194304] 0 2024-03-21T10:08:38.869 INFO:tasks.workunit.client.0.smithi153.stdout:9/39: dread d1/d4/fe [0,4194304] 0 2024-03-21T10:08:38.870 INFO:tasks.workunit.client.0.smithi153.stdout:6/186: rename fd to d1f/d2f/f3d 0 2024-03-21T10:08:38.871 INFO:tasks.workunit.client.0.smithi153.stdout:6/187: fsync d1f/d2b/d33/f3c 0 2024-03-21T10:08:38.879 INFO:tasks.workunit.client.0.smithi153.stdout:8/31: rename d5/d6 to d5/d9 0 2024-03-21T10:08:38.879 INFO:tasks.workunit.client.0.smithi153.stdout:3/119: creat dd/f22 x:0 0 0 2024-03-21T10:08:38.883 INFO:tasks.workunit.client.0.smithi153.stdout:0/30: rmdir d4 39 2024-03-21T10:08:38.887 INFO:tasks.workunit.client.0.smithi153.stdout:2/67: rename l6 to dd/d15/l16 0 2024-03-21T10:08:38.887 INFO:tasks.workunit.client.0.smithi153.stdout:6/188: symlink d1f/d2b/d33/l3e 0 2024-03-21T10:08:38.887 INFO:tasks.workunit.client.0.smithi153.stdout:9/40: creat d1/d4/ff x:0 0 0 2024-03-21T10:08:38.894 INFO:tasks.workunit.client.0.smithi153.stdout:6/189: fdatasync f1a 0 2024-03-21T10:08:38.896 INFO:tasks.workunit.client.0.smithi153.stdout:1/42: dwrite f4 [0,4194304] 0 2024-03-21T10:08:38.896 INFO:tasks.workunit.client.0.smithi153.stdout:4/54: rename fc to d11/f16 0 2024-03-21T10:08:38.899 INFO:tasks.workunit.client.0.smithi153.stdout:6/190: write d1f/f36 [895661,44642] 0 2024-03-21T10:08:38.899 INFO:tasks.workunit.client.0.smithi153.stdout:4/55: write f7 [726946,85367] 0 2024-03-21T10:08:38.906 INFO:tasks.workunit.client.0.smithi153.stdout:3/120: symlink dd/d1e/l23 0 2024-03-21T10:08:38.910 INFO:tasks.workunit.client.0.smithi153.stdout:0/31: symlink d4/la 0 2024-03-21T10:08:38.916 INFO:tasks.workunit.client.0.smithi153.stdout:9/41: dread d1/d4/fe [0,4194304] 0 2024-03-21T10:08:38.927 INFO:tasks.workunit.client.0.smithi153.stdout:0/32: mkdir d4/db 0 2024-03-21T10:08:38.927 INFO:tasks.workunit.client.0.smithi153.stdout:2/68: dwrite fc [0,4194304] 0 2024-03-21T10:08:38.927 INFO:tasks.workunit.client.0.smithi153.stdout:9/42: symlink d1/d4/l10 0 2024-03-21T10:08:38.930 INFO:tasks.workunit.client.0.smithi153.stdout:0/33: write d4/f6 [615644,77335] 0 2024-03-21T10:08:38.931 INFO:tasks.workunit.client.0.smithi153.stdout:4/56: link l9 d11/l17 0 2024-03-21T10:08:38.931 INFO:tasks.workunit.client.0.smithi153.stdout:3/121: link dd/c1b dd/d1e/c24 0 2024-03-21T10:08:38.932 INFO:tasks.workunit.client.0.smithi153.stdout:2/69: mknod dd/d15/c17 0 2024-03-21T10:08:38.932 INFO:tasks.workunit.client.0.smithi153.stdout:2/70: stat dd/f14 0 2024-03-21T10:08:38.933 INFO:tasks.workunit.client.0.smithi153.stdout:1/43: dread f5 [0,4194304] 0 2024-03-21T10:08:38.936 INFO:tasks.workunit.client.0.smithi153.stdout:9/43: rename d1/d4/l10 to d1/l11 0 2024-03-21T10:08:38.938 INFO:tasks.workunit.client.0.smithi153.stdout:0/34: symlink d4/lc 0 2024-03-21T10:08:38.940 INFO:tasks.workunit.client.0.smithi153.stdout:4/57: rmdir d11 39 2024-03-21T10:08:38.944 INFO:tasks.workunit.client.0.smithi153.stdout:2/71: mkdir dd/d15/d18 0 2024-03-21T10:08:38.945 INFO:tasks.workunit.client.0.smithi153.stdout:3/122: unlink l3 0 2024-03-21T10:08:38.949 INFO:tasks.workunit.client.0.smithi153.stdout:9/44: dread d1/d4/fe [0,4194304] 0 2024-03-21T10:08:38.950 INFO:tasks.workunit.client.0.smithi153.stdout:0/35: rename d4/la to d4/db/ld 0 2024-03-21T10:08:38.951 INFO:tasks.workunit.client.0.smithi153.stdout:1/44: creat d6/fd x:0 0 0 2024-03-21T10:08:38.953 INFO:tasks.workunit.client.0.smithi153.stdout:6/191: dwrite d1f/d2f/f3d [0,4194304] 0 2024-03-21T10:08:38.954 INFO:tasks.workunit.client.0.smithi153.stdout:4/58: mkdir d11/d18 0 2024-03-21T10:08:38.957 INFO:tasks.workunit.client.0.smithi153.stdout:3/123: chown dd/f10 49067375 1 2024-03-21T10:08:38.957 INFO:tasks.workunit.client.0.smithi153.stdout:1/45: chown f3 914383 1 2024-03-21T10:08:38.958 INFO:tasks.workunit.client.0.smithi153.stdout:9/45: truncate d1/d4/ff 347280 0 2024-03-21T10:08:38.960 INFO:tasks.workunit.client.0.smithi153.stdout:3/124: read dd/f12 [299407,43584] 0 2024-03-21T10:08:38.962 INFO:tasks.workunit.client.0.smithi153.stdout:9/46: read d1/d4/ff [249559,121893] 0 2024-03-21T10:08:38.965 INFO:tasks.workunit.client.0.smithi153.stdout:0/36: unlink d4/lc 0 2024-03-21T10:08:38.968 INFO:tasks.workunit.client.0.smithi153.stdout:0/37: chown d4 1034542070 1 2024-03-21T10:08:38.968 INFO:tasks.workunit.client.0.smithi153.stdout:1/46: creat d6/fe x:0 0 0 2024-03-21T10:08:38.968 INFO:tasks.workunit.client.0.smithi153.stdout:6/192: symlink d1f/d2f/l3f 0 2024-03-21T10:08:38.969 INFO:tasks.workunit.client.0.smithi153.stdout:1/47: stat d6/fd 0 2024-03-21T10:08:38.970 INFO:tasks.workunit.client.0.smithi153.stdout:6/193: chown d1f/f35 1 1 2024-03-21T10:08:38.972 INFO:tasks.workunit.client.0.smithi153.stdout:3/125: creat dd/f25 x:0 0 0 2024-03-21T10:08:38.973 INFO:tasks.workunit.client.0.smithi153.stdout:9/47: mkdir d1/d12 0 2024-03-21T10:08:38.978 INFO:tasks.workunit.client.0.smithi153.stdout:4/59: dread fb [0,4194304] 0 2024-03-21T10:08:38.978 INFO:tasks.workunit.client.0.smithi153.stdout:1/48: rename f2 to d6/ff 0 2024-03-21T10:08:38.982 INFO:tasks.workunit.client.0.smithi153.stdout:6/194: mkdir d1f/d2b/d33/d40 0 2024-03-21T10:08:38.987 INFO:tasks.workunit.client.0.smithi153.stdout:3/126: write dd/f12 [1248878,55007] 0 2024-03-21T10:08:38.991 INFO:tasks.workunit.client.0.smithi153.stdout:2/72: getdents dd/d15 0 2024-03-21T10:08:38.992 INFO:tasks.workunit.client.0.smithi153.stdout:4/60: creat d11/f19 x:0 0 0 2024-03-21T10:08:38.995 INFO:tasks.workunit.client.0.smithi153.stdout:6/195: creat d1f/d2b/d33/f41 x:0 0 0 2024-03-21T10:08:38.995 INFO:tasks.workunit.client.0.smithi153.stdout:0/38: dwrite f2 [0,4194304] 0 2024-03-21T10:08:38.998 INFO:tasks.workunit.client.0.smithi153.stdout:3/127: write dd/f20 [923704,76632] 0 2024-03-21T10:08:38.999 INFO:tasks.workunit.client.0.smithi153.stdout:2/73: truncate fc 4412204 0 2024-03-21T10:08:39.005 INFO:tasks.workunit.client.0.smithi153.stdout:3/128: write dd/f18 [461117,89470] 0 2024-03-21T10:08:39.006 INFO:tasks.workunit.client.0.smithi153.stdout:6/196: write d1f/f35 [501013,11002] 0 2024-03-21T10:08:39.009 INFO:tasks.workunit.client.0.smithi153.stdout:4/61: symlink d11/d18/l1a 0 2024-03-21T10:08:39.009 INFO:tasks.workunit.client.0.smithi153.stdout:6/197: fdatasync d1f/f36 0 2024-03-21T10:08:39.011 INFO:tasks.workunit.client.0.smithi153.stdout:0/39: mknod d4/db/ce 0 2024-03-21T10:08:39.013 INFO:tasks.workunit.client.0.smithi153.stdout:9/48: link d1/cd d1/c13 0 2024-03-21T10:08:39.017 INFO:tasks.workunit.client.0.smithi153.stdout:4/62: chown d11/l13 24144223 1 2024-03-21T10:08:39.017 INFO:tasks.workunit.client.0.smithi153.stdout:1/49: dwrite f3 [0,4194304] 0 2024-03-21T10:08:39.020 INFO:tasks.workunit.client.0.smithi153.stdout:2/74: rmdir dd 39 2024-03-21T10:08:39.020 INFO:tasks.workunit.client.0.smithi153.stdout:1/50: fdatasync d6/fc 0 2024-03-21T10:08:39.025 INFO:tasks.workunit.client.0.smithi153.stdout:1/51: chown f5 12 1 2024-03-21T10:08:39.025 INFO:tasks.workunit.client.0.smithi153.stdout:3/129: mknod dd/d1e/c26 0 2024-03-21T10:08:39.025 INFO:tasks.workunit.client.0.smithi153.stdout:2/75: read f7 [4169915,108685] 0 2024-03-21T10:08:39.035 INFO:tasks.workunit.client.0.smithi153.stdout:9/49: creat d1/d4/f14 x:0 0 0 2024-03-21T10:08:39.040 INFO:tasks.workunit.client.0.smithi153.stdout:1/52: mkdir d6/d10 0 2024-03-21T10:08:39.049 INFO:tasks.workunit.client.0.smithi153.stdout:4/63: dwrite f2 [0,4194304] 0 2024-03-21T10:08:39.051 INFO:tasks.workunit.client.0.smithi153.stdout:0/40: dread f2 [0,4194304] 0 2024-03-21T10:08:39.060 INFO:tasks.workunit.client.0.smithi153.stdout:1/53: creat d6/d10/f11 x:0 0 0 2024-03-21T10:08:39.061 INFO:tasks.workunit.client.0.smithi153.stdout:3/130: link f2 dd/f27 0 2024-03-21T10:08:39.065 INFO:tasks.workunit.client.0.smithi153.stdout:4/64: write fe [679982,6462] 0 2024-03-21T10:08:39.066 INFO:tasks.workunit.client.0.smithi153.stdout:2/76: getdents dd 0 2024-03-21T10:08:39.069 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:39.066+0000 7f0f68e00700 1 -- 172.21.15.153:0/3861721494 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0f64101da0 msgr2=0x7f0f64104230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:08:39.069 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:39.066+0000 7f0f68e00700 1 --2- 172.21.15.153:0/3861721494 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0f64101da0 0x7f0f64104230 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f0f50009910 tx=0x7f0f50009c20 comp rx=0 tx=0).stop 2024-03-21T10:08:39.069 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:39.066+0000 7f0f68e00700 1 -- 172.21.15.153:0/3861721494 shutdown_connections 2024-03-21T10:08:39.069 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:39.066+0000 7f0f68e00700 1 --2- 172.21.15.153:0/3861721494 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0f64101da0 0x7f0f64104230 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:08:39.069 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:39.066+0000 7f0f68e00700 1 --2- 172.21.15.153:0/3861721494 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0f640ff430 0x7f0f64101860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:08:39.069 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:39.066+0000 7f0f68e00700 1 -- 172.21.15.153:0/3861721494 >> 172.21.15.153:0/3861721494 conn(0x7f0f640fb030 msgr2=0x7f0f640fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:08:39.069 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:39.066+0000 7f0f68e00700 1 -- 172.21.15.153:0/3861721494 shutdown_connections 2024-03-21T10:08:39.069 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:39.066+0000 7f0f68e00700 1 -- 172.21.15.153:0/3861721494 wait complete. 2024-03-21T10:08:39.069 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:39.068+0000 7f0f68e00700 1 Processor -- start 2024-03-21T10:08:39.069 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:39.068+0000 7f0f68e00700 1 -- start start 2024-03-21T10:08:39.070 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:39.069+0000 7f0f68e00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0f640ff430 0x7f0f6419c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:08:39.070 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:39.069+0000 7f0f68e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0f64101da0 0x7f0f6419ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:08:39.070 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:39.069+0000 7f0f68e00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f6419d470 con 0x7f0f64101da0 2024-03-21T10:08:39.070 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:39.069+0000 7f0f68e00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f6419d5b0 con 0x7f0f640ff430 2024-03-21T10:08:39.071 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:39.069+0000 7f0f61400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0f64101da0 0x7f0f6419ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:08:39.071 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:39.070+0000 7f0f61e00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0f640ff430 0x7f0f6419c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:08:39.071 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:39.070+0000 7f0f61e00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0f640ff430 0x7f0f6419c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.153:57322/0 (socket says 172.21.15.153:57322) 2024-03-21T10:08:39.071 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:39.070+0000 7f0f61e00700 1 -- 172.21.15.153:0/2936201231 learned_addr learned my addr 172.21.15.153:0/2936201231 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:08:39.072 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:39.070+0000 7f0f61e00700 1 -- 172.21.15.153:0/2936201231 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0f64101da0 msgr2=0x7f0f6419ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:08:39.072 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:39.070+0000 7f0f61e00700 1 --2- 172.21.15.153:0/2936201231 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0f64101da0 0x7f0f6419ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:08:39.072 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:39.070+0000 7f0f61e00700 1 -- 172.21.15.153:0/2936201231 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f500095f0 con 0x7f0f640ff430 2024-03-21T10:08:39.072 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:39.071+0000 7f0f61e00700 1 --2- 172.21.15.153:0/2936201231 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0f640ff430 0x7f0f6419c910 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0f5400e940 tx=0x7f0f5400ec50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:08:39.072 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:39.071+0000 7f0f5aa00700 1 -- 172.21.15.153:0/2936201231 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f5400cba0 con 0x7f0f640ff430 2024-03-21T10:08:39.072 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:39.071+0000 7f0f5aa00700 1 -- 172.21.15.153:0/2936201231 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0f5400ee90 con 0x7f0f640ff430 2024-03-21T10:08:39.072 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:39.071+0000 7f0f5aa00700 1 -- 172.21.15.153:0/2936201231 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f54005650 con 0x7f0f640ff430 2024-03-21T10:08:39.072 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:39.071+0000 7f0f68e00700 1 -- 172.21.15.153:0/2936201231 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f641a2060 con 0x7f0f640ff430 2024-03-21T10:08:39.072 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:39.071+0000 7f0f68e00700 1 -- 172.21.15.153:0/2936201231 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f641a2560 con 0x7f0f640ff430 2024-03-21T10:08:39.073 INFO:tasks.workunit.client.0.smithi153.stdout:1/54: write d6/fc [268810,89906] 0 2024-03-21T10:08:39.073 INFO:tasks.workunit.client.0.smithi153.stdout:0/41: chown d4/db/ld 6342432 1 2024-03-21T10:08:39.073 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:39.073+0000 7f0f68e00700 1 -- 172.21.15.153:0/2936201231 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f64196b30 con 0x7f0f640ff430 2024-03-21T10:08:39.074 INFO:tasks.workunit.client.0.smithi153.stdout:4/65: chown f10 44 1 2024-03-21T10:08:39.074 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:39.073+0000 7f0f5aa00700 1 -- 172.21.15.153:0/2936201231 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 28) v1 ==== 46435+0+0 (secure 0 0 0) 0x7f0f54004750 con 0x7f0f640ff430 2024-03-21T10:08:39.074 INFO:tasks.workunit.client.0.smithi153.stdout:6/198: dwrite d1f/f20 [0,4194304] 0 2024-03-21T10:08:39.075 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:39.074+0000 7f0f5aa00700 1 --2- 172.21.15.153:0/2936201231 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f0f480394e0 0x7f0f4803b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:08:39.076 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:39.074+0000 7f0f5aa00700 1 -- 172.21.15.153:0/2936201231 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f0f54014070 con 0x7f0f640ff430 2024-03-21T10:08:39.076 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:39.075+0000 7f0f61400700 1 --2- 172.21.15.153:0/2936201231 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f0f480394e0 0x7f0f4803b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:08:39.076 INFO:tasks.workunit.client.0.smithi153.stdout:2/77: readlink dd/le 0 2024-03-21T10:08:39.079 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:39.077+0000 7f0f61400700 1 --2- 172.21.15.153:0/2936201231 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f0f480394e0 0x7f0f4803b9a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f0f5000b6f0 tx=0x7f0f50009f70 comp rx=0 tx=0).ready entity=mgr.14666 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:08:39.079 INFO:tasks.workunit.client.0.smithi153.stdout:9/50: dwrite d1/d4/fc [0,4194304] 0 2024-03-21T10:08:39.080 INFO:tasks.workunit.client.0.smithi153.stdout:4/66: symlink d11/l1b 0 2024-03-21T10:08:39.083 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:39.082+0000 7f0f5aa00700 1 -- 172.21.15.153:0/2936201231 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185454 (secure 0 0 0) 0x7f0f54020390 con 0x7f0f640ff430 2024-03-21T10:08:39.088 INFO:tasks.workunit.client.0.smithi153.stdout:0/42: rename c3 to d4/db/cf 0 2024-03-21T10:08:39.095 INFO:tasks.workunit.client.0.smithi153.stdout:7/35: sync 2024-03-21T10:08:39.095 INFO:tasks.workunit.client.0.smithi153.stdout:8/32: sync 2024-03-21T10:08:39.095 INFO:tasks.workunit.client.0.smithi153.stdout:5/70: sync 2024-03-21T10:08:39.097 INFO:tasks.workunit.client.0.smithi153.stdout:1/55: mknod d6/d10/c12 0 2024-03-21T10:08:39.097 INFO:tasks.workunit.client.0.smithi153.stdout:2/78: mknod dd/c19 0 2024-03-21T10:08:39.101 INFO:tasks.workunit.client.0.smithi153.stdout:2/79: read fc [2803418,35636] 0 2024-03-21T10:08:39.101 INFO:tasks.workunit.client.0.smithi153.stdout:4/67: rename d11/l1b to d11/l1c 0 2024-03-21T10:08:39.105 INFO:tasks.workunit.client.0.smithi153.stdout:6/199: dread d1f/f36 [0,4194304] 0 2024-03-21T10:08:39.110 INFO:tasks.workunit.client.0.smithi153.stdout:6/200: readlink d1f/l31 0 2024-03-21T10:08:39.117 INFO:tasks.workunit.client.0.smithi153.stdout:8/33: mkdir d5/da 0 2024-03-21T10:08:39.120 INFO:tasks.workunit.client.0.smithi153.stdout:7/36: dwrite d4/fa [0,4194304] 0 2024-03-21T10:08:39.121 INFO:tasks.workunit.client.0.smithi153.stdout:9/51: getdents d1/d12 0 2024-03-21T10:08:39.122 INFO:tasks.workunit.client.0.smithi153.stdout:3/131: link lb dd/d1e/l28 0 2024-03-21T10:08:39.122 INFO:tasks.workunit.client.0.smithi153.stdout:4/68: creat d11/d18/f1d x:0 0 0 2024-03-21T10:08:39.125 INFO:tasks.workunit.client.0.smithi153.stdout:6/201: rmdir d1f 39 2024-03-21T10:08:39.135 INFO:tasks.workunit.client.0.smithi153.stdout:2/80: creat dd/d15/d18/f1a x:0 0 0 2024-03-21T10:08:39.136 INFO:tasks.workunit.client.0.smithi153.stdout:3/132: dread dd/f20 [0,4194304] 0 2024-03-21T10:08:39.140 INFO:tasks.workunit.client.0.smithi153.stdout:4/69: creat d11/f1e x:0 0 0 2024-03-21T10:08:39.141 INFO:tasks.workunit.client.0.smithi153.stdout:9/52: truncate d1/d4/ff 1359594 0 2024-03-21T10:08:39.143 INFO:tasks.workunit.client.0.smithi153.stdout:8/34: mknod d5/da/cb 0 2024-03-21T10:08:39.146 INFO:tasks.workunit.client.0.smithi153.stdout:4/70: write d11/f1e [604166,111191] 0 2024-03-21T10:08:39.146 INFO:tasks.workunit.client.0.smithi153.stdout:0/43: getdents d4 0 2024-03-21T10:08:39.149 INFO:tasks.workunit.client.0.smithi153.stdout:4/71: fsync f2 0 2024-03-21T10:08:39.152 INFO:tasks.workunit.client.0.smithi153.stdout:3/133: mknod dd/c29 0 2024-03-21T10:08:39.155 INFO:tasks.workunit.client.0.smithi153.stdout:9/53: write d1/d4/f7 [172373,38985] 0 2024-03-21T10:08:39.155 INFO:tasks.workunit.client.0.smithi153.stdout:0/44: rmdir d4/db 39 2024-03-21T10:08:39.159 INFO:tasks.workunit.client.0.smithi153.stdout:6/202: dwrite f15 [0,4194304] 0 2024-03-21T10:08:39.159 INFO:tasks.workunit.client.0.smithi153.stdout:7/37: write f2 [707006,42912] 0 2024-03-21T10:08:39.160 INFO:tasks.workunit.client.0.smithi153.stdout:8/35: mkdir d5/d9/dc 0 2024-03-21T10:08:39.163 INFO:tasks.workunit.client.0.smithi153.stdout:4/72: creat d11/d18/f1f x:0 0 0 2024-03-21T10:08:39.163 INFO:tasks.workunit.client.0.smithi153.stdout:9/54: write d1/d4/f7 [2140958,64020] 0 2024-03-21T10:08:39.165 INFO:tasks.workunit.client.0.smithi153.stdout:8/36: write f4 [1760088,48451] 0 2024-03-21T10:08:39.169 INFO:tasks.workunit.client.0.smithi153.stdout:2/81: getdents dd/d15/d18 0 2024-03-21T10:08:39.171 INFO:tasks.workunit.client.0.smithi153.stdout:6/203: creat d1f/f42 x:0 0 0 2024-03-21T10:08:39.173 INFO:tasks.workunit.client.0.smithi153.stdout:4/73: dread fe [0,4194304] 0 2024-03-21T10:08:39.174 INFO:tasks.workunit.client.0.smithi153.stdout:4/74: dread - f10 zero size 2024-03-21T10:08:39.176 INFO:tasks.workunit.client.0.smithi153.stdout:4/75: write f10 [690555,36513] 0 2024-03-21T10:08:39.179 INFO:tasks.workunit.client.0.smithi153.stdout:2/82: unlink dd/cf 0 2024-03-21T10:08:39.181 INFO:tasks.workunit.client.0.smithi153.stdout:6/204: unlink c9 0 2024-03-21T10:08:39.184 INFO:tasks.workunit.client.0.smithi153.stdout:0/45: truncate d4/f6 649151 0 2024-03-21T10:08:39.185 INFO:tasks.workunit.client.0.smithi153.stdout:2/83: dread dd/f14 [0,4194304] 0 2024-03-21T10:08:39.185 INFO:tasks.workunit.client.0.smithi153.stdout:9/55: rename d1/d4/cb to d1/c15 0 2024-03-21T10:08:39.186 INFO:tasks.workunit.client.0.smithi153.stdout:0/46: chown d4/l7 67407934 1 2024-03-21T10:08:39.187 INFO:tasks.workunit.client.0.smithi153.stdout:3/134: getdents dd/d1e 0 2024-03-21T10:08:39.187 INFO:tasks.workunit.client.0.smithi153.stdout:8/37: mkdir d5/d9/dc/dd 0 2024-03-21T10:08:39.189 INFO:tasks.workunit.client.0.smithi153.stdout:7/38: rename d4/cb to d4/cc 0 2024-03-21T10:08:39.193 INFO:tasks.workunit.client.0.smithi153.stdout:9/56: creat d1/f16 x:0 0 0 2024-03-21T10:08:39.196 INFO:tasks.workunit.client.0.smithi153.stdout:9/57: chown d1/d12 7770596 1 2024-03-21T10:08:39.198 INFO:tasks.workunit.client.0.smithi153.stdout:4/76: link f1 d11/f20 0 2024-03-21T10:08:39.205 INFO:tasks.workunit.client.0.smithi153.stdout:4/77: creat d11/d18/f21 x:0 0 0 2024-03-21T10:08:39.205 INFO:tasks.workunit.client.0.smithi153.stdout:8/38: dread f4 [0,4194304] 0 2024-03-21T10:08:39.208 INFO:tasks.workunit.client.0.smithi153.stdout:3/135: link dd/f25 dd/d1e/f2a 0 2024-03-21T10:08:39.210 INFO:tasks.workunit.client.0.smithi153.stdout:4/78: chown d11/f12 556 1 2024-03-21T10:08:39.210 INFO:tasks.workunit.client.0.smithi153.stdout:6/205: write f5 [21661,53959] 0 2024-03-21T10:08:39.212 INFO:tasks.workunit.client.0.smithi153.stdout:0/47: dread d4/f6 [0,4194304] 0 2024-03-21T10:08:39.212 INFO:tasks.workunit.client.0.smithi153.stdout:9/58: mknod d1/d12/c17 0 2024-03-21T10:08:39.212 INFO:tasks.workunit.client.0.smithi153.stdout:7/39: truncate d4/fa 2045437 0 2024-03-21T10:08:39.213 INFO:tasks.workunit.client.0.smithi153.stdout:8/39: write d5/f8 [255281,5128] 0 2024-03-21T10:08:39.216 INFO:tasks.workunit.client.0.smithi153.stdout:6/206: write d1f/f39 [282353,101638] 0 2024-03-21T10:08:39.219 INFO:tasks.workunit.client.0.smithi153.stdout:9/59: mknod d1/c18 0 2024-03-21T10:08:39.223 INFO:tasks.workunit.client.0.smithi153.stdout:2/84: dwrite dd/f12 [0,4194304] 0 2024-03-21T10:08:39.223 INFO:tasks.workunit.client.0.smithi153.stdout:8/40: symlink d5/d9/le 0 2024-03-21T10:08:39.226 INFO:tasks.workunit.client.0.smithi153.stdout:6/207: mknod d1f/d2b/c43 0 2024-03-21T10:08:39.228 INFO:tasks.workunit.client.0.smithi153.stdout:6/208: write d1f/f28 [3687498,57799] 0 2024-03-21T10:08:39.229 INFO:tasks.workunit.client.0.smithi153.stdout:7/40: creat d4/fd x:0 0 0 2024-03-21T10:08:39.231 INFO:tasks.workunit.client.0.smithi153.stdout:6/209: write f1a [2577542,75224] 0 2024-03-21T10:08:39.231 INFO:tasks.workunit.client.0.smithi153.stdout:7/41: write d4/fd [141102,88439] 0 2024-03-21T10:08:39.235 INFO:tasks.workunit.client.0.smithi153.stdout:7/42: chown d4/fd 321 1 2024-03-21T10:08:39.235 INFO:tasks.workunit.client.0.smithi153.stdout:2/85: symlink dd/d15/l1b 0 2024-03-21T10:08:39.236 INFO:tasks.workunit.client.0.smithi153.stdout:4/79: getdents d11/d18 0 2024-03-21T10:08:39.238 INFO:tasks.workunit.client.0.smithi153.stdout:0/48: dread f2 [0,4194304] 0 2024-03-21T10:08:39.239 INFO:tasks.workunit.client.0.smithi153.stdout:6/210: creat d1f/d2b/d33/d40/f44 x:0 0 0 2024-03-21T10:08:39.240 INFO:tasks.workunit.client.0.smithi153.stdout:4/80: rename d11 to d11/d22 22 2024-03-21T10:08:39.242 INFO:tasks.workunit.client.0.smithi153.stdout:2/86: symlink dd/l1c 0 2024-03-21T10:08:39.249 INFO:tasks.workunit.client.0.smithi153.stdout:4/81: fdatasync d11/f12 0 2024-03-21T10:08:39.254 INFO:tasks.workunit.client.0.smithi153.stdout:6/211: creat d1f/d2b/d33/d40/f45 x:0 0 0 2024-03-21T10:08:39.256 INFO:tasks.workunit.client.0.smithi153.stdout:7/43: getdents d4 0 2024-03-21T10:08:39.256 INFO:tasks.workunit.client.0.smithi153.stdout:9/60: dwrite d1/f16 [0,4194304] 0 2024-03-21T10:08:39.258 INFO:tasks.workunit.client.0.smithi153.stdout:0/49: creat d4/f10 x:0 0 0 2024-03-21T10:08:39.258 INFO:tasks.workunit.client.0.smithi153.stdout:2/87: creat dd/d15/f1d x:0 0 0 2024-03-21T10:08:39.259 INFO:tasks.workunit.client.0.smithi153.stdout:6/212: write f19 [1263111,129640] 0 2024-03-21T10:08:39.261 INFO:tasks.workunit.client.0.smithi153.stdout:0/50: readlink d4/l9 0 2024-03-21T10:08:39.261 INFO:tasks.workunit.client.0.smithi153.stdout:4/82: rmdir d11/d18 39 2024-03-21T10:08:39.264 INFO:tasks.workunit.client.0.smithi153.stdout:6/213: creat d1f/d2b/f46 x:0 0 0 2024-03-21T10:08:39.265 INFO:tasks.workunit.client.0.smithi153.stdout:7/44: truncate d4/fd 719241 0 2024-03-21T10:08:39.269 INFO:tasks.workunit.client.0.smithi153.stdout:4/83: readlink d11/l17 0 2024-03-21T10:08:39.269 INFO:tasks.workunit.client.0.smithi153.stdout:8/41: dread d5/d9/f7 [0,4194304] 0 2024-03-21T10:08:39.269 INFO:tasks.workunit.client.0.smithi153.stdout:6/214: truncate f11 4239354 0 2024-03-21T10:08:39.270 INFO:tasks.workunit.client.0.smithi153.stdout:2/88: truncate fc 2712632 0 2024-03-21T10:08:39.271 INFO:tasks.workunit.client.0.smithi153.stdout:7/45: symlink d4/le 0 2024-03-21T10:08:39.274 INFO:tasks.workunit.client.0.smithi153.stdout:9/61: truncate d1/d4/fe 1563165 0 2024-03-21T10:08:39.275 INFO:tasks.workunit.client.0.smithi153.stdout:0/51: creat d4/db/f11 x:0 0 0 2024-03-21T10:08:39.277 INFO:tasks.workunit.client.0.smithi153.stdout:9/62: fsync d1/d4/fc 0 2024-03-21T10:08:39.277 INFO:tasks.workunit.client.0.smithi153.stdout:6/215: creat d1f/d2f/f47 x:0 0 0 2024-03-21T10:08:39.278 INFO:tasks.workunit.client.0.smithi153.stdout:4/84: rmdir d11/d18 39 2024-03-21T10:08:39.279 INFO:tasks.workunit.client.0.smithi153.stdout:8/42: dread f4 [0,4194304] 0 2024-03-21T10:08:39.280 INFO:tasks.workunit.client.0.smithi153.stdout:4/85: stat ca 0 2024-03-21T10:08:39.282 INFO:tasks.workunit.client.0.smithi153.stdout:6/216: symlink d1f/d2b/l48 0 2024-03-21T10:08:39.284 INFO:tasks.workunit.client.0.smithi153.stdout:6/217: stat d1f/l24 0 2024-03-21T10:08:39.287 INFO:tasks.workunit.client.0.smithi153.stdout:9/63: creat d1/d12/f19 x:0 0 0 2024-03-21T10:08:39.291 INFO:tasks.workunit.client.0.smithi153.stdout:9/64: dread - d1/d12/f19 zero size 2024-03-21T10:08:39.294 INFO:tasks.workunit.client.0.smithi153.stdout:9/65: write d1/d4/fc [442640,84073] 0 2024-03-21T10:08:39.295 INFO:tasks.workunit.client.0.smithi153.stdout:2/89: getdents dd 0 2024-03-21T10:08:39.297 INFO:tasks.workunit.client.0.smithi153.stdout:2/90: chown dd/d15/f1d 1 1 2024-03-21T10:08:39.322 INFO:tasks.workunit.client.0.smithi153.stdout:0/52: rename d4/db/ce to d4/c12 0 2024-03-21T10:08:39.328 INFO:tasks.workunit.client.0.smithi153.stdout:6/218: dwrite d1f/d2b/d33/f3c [0,4194304] 0 2024-03-21T10:08:39.328 INFO:tasks.workunit.client.0.smithi153.stdout:9/66: creat d1/d4/f1a x:0 0 0 2024-03-21T10:08:39.330 INFO:tasks.workunit.client.0.smithi153.stdout:6/219: read - d1f/d2f/f47 zero size 2024-03-21T10:08:39.333 INFO:tasks.workunit.client.0.smithi153.stdout:2/91: creat dd/d15/f1e x:0 0 0 2024-03-21T10:08:39.334 INFO:tasks.workunit.client.0.smithi153.stdout:4/86: getdents d11/d18 0 2024-03-21T10:08:39.337 INFO:tasks.workunit.client.0.smithi153.stdout:6/220: mknod d1f/d2b/c49 0 2024-03-21T10:08:39.337 INFO:tasks.workunit.client.0.smithi153.stdout:8/43: getdents d5/d9 0 2024-03-21T10:08:39.338 INFO:tasks.workunit.client.0.smithi153.stdout:6/221: stat d1f/c29 0 2024-03-21T10:08:39.339 INFO:tasks.workunit.client.0.smithi153.stdout:8/44: stat d5/d9 0 2024-03-21T10:08:39.350 INFO:tasks.workunit.client.0.smithi153.stdout:4/87: link c5 d11/c23 0 2024-03-21T10:08:39.350 INFO:tasks.workunit.client.0.smithi153.stdout:8/45: dread f4 [0,4194304] 0 2024-03-21T10:08:39.351 INFO:tasks.workunit.client.0.smithi153.stdout:6/222: rmdir d1f 39 2024-03-21T10:08:39.355 INFO:tasks.workunit.client.0.smithi153.stdout:4/88: read f2 [1412751,56505] 0 2024-03-21T10:08:39.355 INFO:tasks.workunit.client.0.smithi153.stdout:2/92: truncate dd/f14 563084 0 2024-03-21T10:08:39.415 INFO:tasks.workunit.client.0.smithi153.stdout:9/67: dwrite d1/d4/f1a [0,4194304] 0 2024-03-21T10:08:39.417 INFO:tasks.workunit.client.0.smithi153.stdout:2/93: truncate dd/f13 550211 0 2024-03-21T10:08:39.418 INFO:tasks.workunit.client.0.smithi153.stdout:4/89: write d11/d18/f21 [479279,79530] 0 2024-03-21T10:08:39.419 INFO:tasks.workunit.client.0.smithi153.stdout:8/46: symlink d5/d9/dc/lf 0 2024-03-21T10:08:39.420 INFO:tasks.workunit.client.0.smithi153.stdout:6/223: creat d1f/f4a x:0 0 0 2024-03-21T10:08:39.421 INFO:tasks.workunit.client.0.smithi153.stdout:2/94: chown l8 1 1 2024-03-21T10:08:39.424 INFO:tasks.workunit.client.0.smithi153.stdout:4/90: write f2 [4307448,48578] 0 2024-03-21T10:08:39.431 INFO:tasks.workunit.client.0.smithi153.stdout:4/91: stat ca 0 2024-03-21T10:08:39.431 INFO:tasks.workunit.client.0.smithi153.stdout:6/224: chown d1f/d2b/l37 3765 1 2024-03-21T10:08:39.431 INFO:tasks.workunit.client.0.smithi153.stdout:8/47: creat d5/f10 x:0 0 0 2024-03-21T10:08:39.431 INFO:tasks.workunit.client.0.smithi153.stdout:8/48: stat d5 0 2024-03-21T10:08:39.432 INFO:tasks.workunit.client.0.smithi153.stdout:6/225: readlink d1f/d2b/d33/l3e 0 2024-03-21T10:08:39.432 INFO:tasks.workunit.client.0.smithi153.stdout:6/226: dread - d1f/f42 zero size 2024-03-21T10:08:39.434 INFO:tasks.workunit.client.0.smithi153.stdout:6/227: creat d1f/f4b x:0 0 0 2024-03-21T10:08:39.438 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:39.437+0000 7f0f68e00700 1 -- 172.21.15.153:0/2936201231 --> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0f640611d0 con 0x7f0f480394e0 2024-03-21T10:08:39.440 INFO:tasks.workunit.client.0.smithi153.stdout:4/92: dread f10 [0,4194304] 0 2024-03-21T10:08:39.440 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:39.440+0000 7f0f5aa00700 1 -- 172.21.15.153:0/2936201231 <== mgr.14666 v2:172.21.15.153:6800/2524145044 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+329 (secure 0 0 0) 0x7f0f640611d0 con 0x7f0f480394e0 2024-03-21T10:08:39.444 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:39.443+0000 7f0f68e00700 1 -- 172.21.15.153:0/2936201231 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f0f480394e0 msgr2=0x7f0f4803b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:08:39.444 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:39.443+0000 7f0f68e00700 1 --2- 172.21.15.153:0/2936201231 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f0f480394e0 0x7f0f4803b9a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f0f5000b6f0 tx=0x7f0f50009f70 comp rx=0 tx=0).stop 2024-03-21T10:08:39.444 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:39.443+0000 7f0f68e00700 1 -- 172.21.15.153:0/2936201231 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0f640ff430 msgr2=0x7f0f6419c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:08:39.444 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:39.443+0000 7f0f68e00700 1 --2- 172.21.15.153:0/2936201231 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0f640ff430 0x7f0f6419c910 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0f5400e940 tx=0x7f0f5400ec50 comp rx=0 tx=0).stop 2024-03-21T10:08:39.445 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:39.444+0000 7f0f68e00700 1 -- 172.21.15.153:0/2936201231 shutdown_connections 2024-03-21T10:08:39.445 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:39.444+0000 7f0f68e00700 1 --2- 172.21.15.153:0/2936201231 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f0f480394e0 0x7f0f4803b9a0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:08:39.445 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:39.444+0000 7f0f68e00700 1 --2- 172.21.15.153:0/2936201231 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0f640ff430 0x7f0f6419c910 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:08:39.445 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:39.444+0000 7f0f68e00700 1 --2- 172.21.15.153:0/2936201231 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0f64101da0 0x7f0f6419ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:08:39.445 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:39.444+0000 7f0f68e00700 1 -- 172.21.15.153:0/2936201231 >> 172.21.15.153:0/2936201231 conn(0x7f0f640fb030 msgr2=0x7f0f640fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:08:39.445 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:39.444+0000 7f0f68e00700 1 -- 172.21.15.153:0/2936201231 shutdown_connections 2024-03-21T10:08:39.445 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:39.444+0000 7f0f68e00700 1 -- 172.21.15.153:0/2936201231 wait complete. 2024-03-21T10:08:39.450 INFO:tasks.workunit.client.0.smithi153.stdout:8/49: dread d5/d9/f7 [0,4194304] 0 2024-03-21T10:08:39.453 INFO:tasks.workunit.client.0.smithi153.stdout:9/68: dwrite d1/d12/f19 [0,4194304] 0 2024-03-21T10:08:39.463 INFO:tasks.workunit.client.0.smithi153.stdout:9/69: creat d1/f1b x:0 0 0 2024-03-21T10:08:39.472 INFO:teuthology.orchestra.run.smithi153.stdout:true 2024-03-21T10:08:39.478 INFO:tasks.workunit.client.0.smithi153.stdout:2/95: dwrite f7 [0,4194304] 0 2024-03-21T10:08:39.482 INFO:tasks.workunit.client.0.smithi153.stdout:6/228: dwrite d1f/d2b/d33/d40/f44 [0,4194304] 0 2024-03-21T10:08:39.489 INFO:tasks.workunit.client.0.smithi153.stdout:9/70: getdents d1 0 2024-03-21T10:08:39.490 INFO:tasks.workunit.client.0.smithi153.stdout:6/229: truncate f1b 641605 0 2024-03-21T10:08:39.490 INFO:tasks.workunit.client.0.smithi153.stdout:2/96: chown dd/d15/f1d 3997442 1 2024-03-21T10:08:39.492 INFO:tasks.workunit.client.0.smithi153.stdout:4/93: dwrite d11/d18/f21 [0,4194304] 0 2024-03-21T10:08:39.492 INFO:tasks.workunit.client.0.smithi153.stdout:9/71: fdatasync d1/f16 0 2024-03-21T10:08:39.500 INFO:tasks.workunit.client.0.smithi153.stdout:6/230: dread f5 [0,4194304] 0 2024-03-21T10:08:39.503 INFO:tasks.workunit.client.0.smithi153.stdout:9/72: mkdir d1/d12/d1c 0 2024-03-21T10:08:39.505 INFO:tasks.workunit.client.0.smithi153.stdout:8/50: dwrite f3 [0,4194304] 0 2024-03-21T10:08:39.511 INFO:tasks.workunit.client.0.smithi153.stdout:6/231: creat d1f/d2f/f4c x:0 0 0 2024-03-21T10:08:39.521 INFO:tasks.workunit.client.0.smithi153.stdout:6/232: creat d1f/d2b/d33/d40/f4d x:0 0 0 2024-03-21T10:08:39.521 INFO:tasks.workunit.client.0.smithi153.stdout:8/51: creat d5/d9/f11 x:0 0 0 2024-03-21T10:08:39.526 INFO:tasks.workunit.client.0.smithi153.stdout:6/233: stat d1f/f39 0 2024-03-21T10:08:39.530 INFO:tasks.workunit.client.0.smithi153.stdout:2/97: write dd/f14 [659587,33352] 0 2024-03-21T10:08:39.530 INFO:tasks.workunit.client.0.smithi153.stdout:8/52: write d5/d9/f11 [775702,8150] 0 2024-03-21T10:08:39.537 INFO:tasks.workunit.client.0.smithi153.stdout:6/234: creat d1f/f4e x:0 0 0 2024-03-21T10:08:39.539 INFO:tasks.workunit.client.0.smithi153.stdout:4/94: dwrite f2 [4194304,4194304] 0 2024-03-21T10:08:39.541 INFO:tasks.workunit.client.0.smithi153.stdout:6/235: readlink d1f/d2b/l48 0 2024-03-21T10:08:39.543 INFO:tasks.workunit.client.0.smithi153.stdout:2/98: truncate f5 236872 0 2024-03-21T10:08:39.549 INFO:tasks.workunit.client.0.smithi153.stdout:8/53: truncate f4 526638 0 2024-03-21T10:08:39.550 INFO:tasks.workunit.client.0.smithi153.stdout:9/73: dwrite d1/d12/f19 [4194304,4194304] 0 2024-03-21T10:08:39.552 INFO:tasks.workunit.client.0.smithi153.stdout:4/95: creat d11/f24 x:0 0 0 2024-03-21T10:08:39.556 INFO:tasks.workunit.client.0.smithi153.stdout:4/96: chown d11/f1e 525486287 1 2024-03-21T10:08:39.566 INFO:tasks.workunit.client.0.smithi153.stdout:2/99: dread dd/f12 [0,4194304] 0 2024-03-21T10:08:39.567 INFO:tasks.workunit.client.0.smithi153.stdout:6/236: dwrite d1f/d2f/f3d [0,4194304] 0 2024-03-21T10:08:39.570 INFO:tasks.workunit.client.0.smithi153.stdout:4/97: link cf d11/d18/c25 0 2024-03-21T10:08:39.574 INFO:tasks.workunit.client.0.smithi153.stdout:2/100: mknod dd/d15/d18/c1f 0 2024-03-21T10:08:39.575 INFO:tasks.workunit.client.0.smithi153.stdout:9/74: truncate d1/d12/f19 5689615 0 2024-03-21T10:08:39.576 INFO:tasks.workunit.client.0.smithi153.stdout:6/237: chown c16 0 1 2024-03-21T10:08:39.578 INFO:tasks.workunit.client.0.smithi153.stdout:9/75: read - d1/f1b zero size 2024-03-21T10:08:39.579 INFO:tasks.workunit.client.0.smithi153.stdout:2/101: mknod dd/d15/c20 0 2024-03-21T10:08:39.581 INFO:tasks.workunit.client.0.smithi153.stdout:6/238: mkdir d1f/d2b/d33/d40/d4f 0 2024-03-21T10:08:39.581 INFO:tasks.workunit.client.0.smithi153.stdout:4/98: creat d11/f26 x:0 0 0 2024-03-21T10:08:39.584 INFO:tasks.workunit.client.0.smithi153.stdout:6/239: mknod d1f/c50 0 2024-03-21T10:08:39.588 INFO:tasks.workunit.client.0.smithi153.stdout:9/76: dread d1/d4/ff [0,4194304] 0 2024-03-21T10:08:39.590 INFO:tasks.workunit.client.0.smithi153.stdout:6/240: mkdir d1f/d2b/d33/d40/d51 0 2024-03-21T10:08:39.592 INFO:tasks.workunit.client.0.smithi153.stdout:8/54: truncate f4 326289 0 2024-03-21T10:08:39.592 INFO:tasks.workunit.client.0.smithi153.stdout:6/241: dread - d1f/d2b/d33/d40/f4d zero size 2024-03-21T10:08:39.593 INFO:tasks.workunit.client.0.smithi153.stdout:9/77: fdatasync d1/d4/f14 0 2024-03-21T10:08:39.603 INFO:tasks.workunit.client.0.smithi153.stdout:6/242: creat d1f/d2b/d33/d40/d51/f52 x:0 0 0 2024-03-21T10:08:39.603 INFO:tasks.workunit.client.0.smithi153.stdout:2/102: dwrite f7 [4194304,4194304] 0 2024-03-21T10:08:39.609 INFO:tasks.workunit.client.0.smithi153.stdout:8/55: truncate f3 3604231 0 2024-03-21T10:08:39.610 INFO:tasks.workunit.client.0.smithi153.stdout:2/103: write dd/f10 [529035,51954] 0 2024-03-21T10:08:39.613 INFO:tasks.workunit.client.0.smithi153.stdout:2/104: stat c2 0 2024-03-21T10:08:39.614 INFO:tasks.workunit.client.0.smithi153.stdout:4/99: dwrite d11/d18/f1d [0,4194304] 0 2024-03-21T10:08:39.614 INFO:tasks.workunit.client.0.smithi153.stdout:8/56: chown d5/d9/dc/lf 68625 1 2024-03-21T10:08:39.617 INFO:tasks.workunit.client.0.smithi153.stdout:4/100: read - d11/d18/f1f zero size 2024-03-21T10:08:39.617 INFO:tasks.workunit.client.0.smithi153.stdout:8/57: chown d5/f10 91087924 1 2024-03-21T10:08:39.619 INFO:tasks.workunit.client.0.smithi153.stdout:9/78: dread d1/f16 [0,4194304] 0 2024-03-21T10:08:39.624 INFO:tasks.workunit.client.0.smithi153.stdout:2/105: rename dd/l1c to dd/d15/l21 0 2024-03-21T10:08:39.624 INFO:tasks.workunit.client.0.smithi153.stdout:6/243: link d1f/d2b/d33/d40/f4d d1f/d2b/f53 0 2024-03-21T10:08:39.626 INFO:tasks.workunit.client.0.smithi153.stdout:4/101: stat d11/c23 0 2024-03-21T10:08:39.626 INFO:tasks.workunit.client.0.smithi153.stdout:8/58: rmdir d5/d9 39 2024-03-21T10:08:39.628 INFO:tasks.workunit.client.0.smithi153.stdout:2/106: chown dd/d15/d18/f1a 3807 1 2024-03-21T10:08:39.628 INFO:tasks.workunit.client.0.smithi153.stdout:6/244: write d1f/d2f/f32 [999170,75654] 0 2024-03-21T10:08:39.628 INFO:tasks.workunit.client.0.smithi153.stdout:9/79: creat d1/d12/f1d x:0 0 0 2024-03-21T10:08:39.629 INFO:tasks.workunit.client.0.smithi153.stdout:4/102: dread - d11/f16 zero size 2024-03-21T10:08:39.632 INFO:tasks.workunit.client.0.smithi153.stdout:4/103: chown fe 29 1 2024-03-21T10:08:39.633 INFO:tasks.workunit.client.0.smithi153.stdout:8/59: write d5/f10 [546935,106894] 0 2024-03-21T10:08:39.635 INFO:tasks.workunit.client.0.smithi153.stdout:4/104: chown d11/d18/f1d 1589427 1 2024-03-21T10:08:39.637 INFO:tasks.workunit.client.0.smithi153.stdout:4/105: truncate d11/f26 623169 0 2024-03-21T10:08:39.640 INFO:tasks.workunit.client.0.smithi153.stdout:2/107: rename dd/d15/c17 to dd/c22 0 2024-03-21T10:08:39.641 INFO:tasks.workunit.client.0.smithi153.stdout:2/108: read - dd/d15/f1e zero size 2024-03-21T10:08:39.649 INFO:tasks.workunit.client.0.smithi153.stdout:2/109: write dd/d15/f1d [409281,96440] 0 2024-03-21T10:08:39.651 INFO:tasks.workunit.client.0.smithi153.stdout:8/60: dread d5/f10 [0,4194304] 0 2024-03-21T10:08:39.654 INFO:tasks.workunit.client.0.smithi153.stdout:4/106: write f1 [4078871,75793] 0 2024-03-21T10:08:39.654 INFO:tasks.workunit.client.0.smithi153.stdout:9/80: getdents d1/d4 0 2024-03-21T10:08:39.658 INFO:tasks.workunit.client.0.smithi153.stdout:8/61: creat d5/d9/f12 x:0 0 0 2024-03-21T10:08:39.659 INFO:tasks.workunit.client.0.smithi153.stdout:9/81: mknod d1/d12/c1e 0 2024-03-21T10:08:39.664 INFO:tasks.workunit.client.0.smithi153.stdout:6/245: dwrite d1f/f2d [4194304,4194304] 0 2024-03-21T10:08:39.667 INFO:tasks.workunit.client.0.smithi153.stdout:8/62: mknod d5/c13 0 2024-03-21T10:08:39.667 INFO:tasks.workunit.client.0.smithi153.stdout:9/82: mkdir d1/d12/d1f 0 2024-03-21T10:08:39.668 INFO:tasks.workunit.client.0.smithi153.stdout:4/107: rename cf to d11/c27 0 2024-03-21T10:08:39.671 INFO:tasks.workunit.client.0.smithi153.stdout:6/246: rename d1f/d2b/c49 to d1f/d2b/c54 0 2024-03-21T10:08:39.672 INFO:tasks.workunit.client.0.smithi153.stdout:4/108: chown d11/d18/f21 695445 1 2024-03-21T10:08:39.675 INFO:tasks.workunit.client.0.smithi153.stdout:6/247: creat d1f/d2b/d33/d40/d51/f55 x:0 0 0 2024-03-21T10:08:39.679 INFO:tasks.workunit.client.0.smithi153.stdout:8/63: creat d5/f14 x:0 0 0 2024-03-21T10:08:39.680 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:39.679+0000 7fa0d2200700 1 -- 172.21.15.153:0/90942029 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa0cc0ff430 msgr2=0x7fa0cc101860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:08:39.680 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:39.679+0000 7fa0d2200700 1 --2- 172.21.15.153:0/90942029 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa0cc0ff430 0x7fa0cc101860 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc009910 tx=0x7fa0bc009c20 comp rx=0 tx=0).stop 2024-03-21T10:08:39.680 INFO:tasks.workunit.client.0.smithi153.stdout:6/248: chown d1f/d2f/f47 22 1 2024-03-21T10:08:39.681 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:39.680+0000 7fa0d2200700 1 -- 172.21.15.153:0/90942029 shutdown_connections 2024-03-21T10:08:39.681 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:39.680+0000 7fa0d2200700 1 --2- 172.21.15.153:0/90942029 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa0cc101da0 0x7fa0cc104230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:08:39.681 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:39.680+0000 7fa0d2200700 1 --2- 172.21.15.153:0/90942029 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa0cc0ff430 0x7fa0cc101860 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:08:39.681 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:39.680+0000 7fa0d2200700 1 -- 172.21.15.153:0/90942029 >> 172.21.15.153:0/90942029 conn(0x7fa0cc0fb030 msgr2=0x7fa0cc0fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:08:39.681 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:39.681+0000 7fa0d2200700 1 -- 172.21.15.153:0/90942029 shutdown_connections 2024-03-21T10:08:39.682 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:39.681+0000 7fa0d2200700 1 -- 172.21.15.153:0/90942029 wait complete. 2024-03-21T10:08:39.682 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:39.682+0000 7fa0d2200700 1 Processor -- start 2024-03-21T10:08:39.682 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:39.682+0000 7fa0d2200700 1 -- start start 2024-03-21T10:08:39.683 INFO:tasks.workunit.client.0.smithi153.stdout:6/249: chown d1f/f4e 2488697 1 2024-03-21T10:08:39.684 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:39.683+0000 7fa0d2200700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa0cc0ff430 0x7fa0cc10d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:08:39.684 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:39.683+0000 7fa0d2200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa0cc101da0 0x7fa0cc10d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:08:39.684 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:39.684+0000 7fa0d2200700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0cc10db60 con 0x7fa0cc101da0 2024-03-21T10:08:39.685 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:39.684+0000 7fa0caa00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa0cc101da0 0x7fa0cc10d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:08:39.685 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:39.684+0000 7fa0d2200700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0cc10dca0 con 0x7fa0cc0ff430 2024-03-21T10:08:39.685 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:39.684+0000 7fa0caa00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa0cc101da0 0x7fa0cc10d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:58888/0 (socket says 172.21.15.153:58888) 2024-03-21T10:08:39.685 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:39.684+0000 7fa0caa00700 1 -- 172.21.15.153:0/742145548 learned_addr learned my addr 172.21.15.153:0/742145548 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:08:39.685 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:39.685+0000 7fa0cb400700 1 --2- 172.21.15.153:0/742145548 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa0cc0ff430 0x7fa0cc10d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:08:39.686 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:39.685+0000 7fa0caa00700 1 -- 172.21.15.153:0/742145548 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa0cc0ff430 msgr2=0x7fa0cc10d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:08:39.686 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:39.685+0000 7fa0caa00700 1 --2- 172.21.15.153:0/742145548 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa0cc0ff430 0x7fa0cc10d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:08:39.686 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:39.686+0000 7fa0caa00700 1 -- 172.21.15.153:0/742145548 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0bc0095f0 con 0x7fa0cc101da0 2024-03-21T10:08:39.686 INFO:tasks.workunit.client.0.smithi153.stdout:4/109: link d11/f24 d11/d18/f28 0 2024-03-21T10:08:39.689 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:39.688+0000 7fa0caa00700 1 --2- 172.21.15.153:0/742145548 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa0cc101da0 0x7fa0cc10d540 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fa0b800d8d0 tx=0x7fa0b800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:08:39.690 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:39.689+0000 7fa0c3e00700 1 -- 172.21.15.153:0/742145548 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa0b8009940 con 0x7fa0cc101da0 2024-03-21T10:08:39.690 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:39.689+0000 7fa0c3e00700 1 -- 172.21.15.153:0/742145548 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa0b8010460 con 0x7fa0cc101da0 2024-03-21T10:08:39.690 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:39.689+0000 7fa0c3e00700 1 -- 172.21.15.153:0/742145548 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa0b800f5f0 con 0x7fa0cc101da0 2024-03-21T10:08:39.691 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:39.689+0000 7fa0d2200700 1 -- 172.21.15.153:0/742145548 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0cc1aa870 con 0x7fa0cc101da0 2024-03-21T10:08:39.692 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:39.691+0000 7fa0d2200700 1 -- 172.21.15.153:0/742145548 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0cc1aadc0 con 0x7fa0cc101da0 2024-03-21T10:08:39.693 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:39.693+0000 7fa0c3e00700 1 -- 172.21.15.153:0/742145548 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 28) v1 ==== 46435+0+0 (secure 0 0 0) 0x7fa0b80105d0 con 0x7fa0cc101da0 2024-03-21T10:08:39.693 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:39.693+0000 7fa0d2200700 1 -- 172.21.15.153:0/742145548 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0cc1071f0 con 0x7fa0cc101da0 2024-03-21T10:08:39.694 INFO:tasks.workunit.client.0.smithi153.stdout:4/110: write d11/d18/f1f [499159,111671] 0 2024-03-21T10:08:39.694 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:39.693+0000 7fa0c3e00700 1 --2- 172.21.15.153:0/742145548 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7fa0b00394e0 0x7fa0b003b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:08:39.694 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:39.693+0000 7fa0c3e00700 1 -- 172.21.15.153:0/742145548 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fa0b804d570 con 0x7fa0cc101da0 2024-03-21T10:08:39.694 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:39.694+0000 7fa0cb400700 1 --2- 172.21.15.153:0/742145548 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7fa0b00394e0 0x7fa0b003b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:08:39.696 INFO:tasks.workunit.client.0.smithi153.stdout:9/83: dwrite d1/d4/f1a [0,4194304] 0 2024-03-21T10:08:39.697 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:39.696+0000 7fa0cb400700 1 --2- 172.21.15.153:0/742145548 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7fa0b00394e0 0x7fa0b003b9a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc006010 tx=0x7fa0bc00b540 comp rx=0 tx=0).ready entity=mgr.14666 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:08:39.698 INFO:tasks.workunit.client.0.smithi153.stdout:4/111: write f1 [283430,1794] 0 2024-03-21T10:08:39.698 INFO:tasks.workunit.client.0.smithi153.stdout:6/250: truncate d1f/f28 1988045 0 2024-03-21T10:08:39.700 INFO:tasks.workunit.client.0.smithi153.stdout:6/251: dread - d1f/f4a zero size 2024-03-21T10:08:39.701 INFO:tasks.workunit.client.0.smithi153.stdout:4/112: stat d11/d18/f28 0 2024-03-21T10:08:39.706 INFO:tasks.workunit.client.0.smithi153.stdout:6/252: fsync f18 0 2024-03-21T10:08:39.707 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:39.706+0000 7fa0c3e00700 1 -- 172.21.15.153:0/742145548 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185454 (secure 0 0 0) 0x7fa0b801ae90 con 0x7fa0cc101da0 2024-03-21T10:08:39.707 INFO:tasks.workunit.client.0.smithi153.stdout:4/113: symlink d11/l29 0 2024-03-21T10:08:39.708 INFO:tasks.workunit.client.0.smithi153.stdout:4/114: chown l9 1 1 2024-03-21T10:08:39.709 INFO:tasks.workunit.client.0.smithi153.stdout:4/115: stat l9 0 2024-03-21T10:08:39.710 INFO:tasks.workunit.client.0.smithi153.stdout:8/64: dread f3 [0,4194304] 0 2024-03-21T10:08:39.711 INFO:tasks.workunit.client.0.smithi153.stdout:4/116: chown fe 47 1 2024-03-21T10:08:39.712 INFO:tasks.workunit.client.0.smithi153.stdout:6/253: symlink d1f/d2f/l56 0 2024-03-21T10:08:39.717 INFO:tasks.workunit.client.0.smithi153.stdout:8/65: mkdir d5/da/d15 0 2024-03-21T10:08:39.718 INFO:tasks.workunit.client.0.smithi153.stdout:6/254: creat d1f/d2b/d33/d40/d51/f57 x:0 0 0 2024-03-21T10:08:39.721 INFO:tasks.workunit.client.0.smithi153.stdout:9/84: dwrite d1/d4/f7 [0,4194304] 0 2024-03-21T10:08:39.727 INFO:tasks.workunit.client.0.smithi153.stdout:6/255: creat d1f/d2b/d33/d40/f58 x:0 0 0 2024-03-21T10:08:39.729 INFO:tasks.workunit.client.0.smithi153.stdout:9/85: creat d1/d4/f20 x:0 0 0 2024-03-21T10:08:39.734 INFO:tasks.workunit.client.0.smithi153.stdout:9/86: dread - d1/d12/f1d zero size 2024-03-21T10:08:39.736 INFO:tasks.workunit.client.0.smithi153.stdout:6/256: getdents d1f/d2b/d33/d40/d4f 0 2024-03-21T10:08:39.737 INFO:tasks.workunit.client.0.smithi153.stdout:8/66: dread f3 [0,4194304] 0 2024-03-21T10:08:39.739 INFO:tasks.workunit.client.0.smithi153.stdout:4/117: dwrite fe [0,4194304] 0 2024-03-21T10:08:39.739 INFO:tasks.workunit.client.0.smithi153.stdout:6/257: write f11 [3113145,126653] 0 2024-03-21T10:08:39.741 INFO:tasks.workunit.client.0.smithi153.stdout:6/258: readlink d1f/d2b/l37 0 2024-03-21T10:08:39.742 INFO:tasks.workunit.client.0.smithi153.stdout:8/67: write d5/d9/f11 [658198,123442] 0 2024-03-21T10:08:39.743 INFO:tasks.workunit.client.0.smithi153.stdout:6/259: write f11 [1655744,46381] 0 2024-03-21T10:08:39.743 INFO:tasks.workunit.client.0.smithi153.stdout:6/260: readlink l1e 0 2024-03-21T10:08:39.744 INFO:tasks.workunit.client.0.smithi153.stdout:8/68: fsync d5/d9/f12 0 2024-03-21T10:08:39.745 INFO:tasks.workunit.client.0.smithi153.stdout:4/118: unlink d11/c23 0 2024-03-21T10:08:39.750 INFO:tasks.workunit.client.0.smithi153.stdout:6/261: mkdir d1f/d2f/d59 0 2024-03-21T10:08:39.753 INFO:tasks.workunit.client.0.smithi153.stdout:8/69: write d5/d9/f7 [2617310,93950] 0 2024-03-21T10:08:39.754 INFO:tasks.workunit.client.0.smithi153.stdout:9/87: rmdir d1/d4 39 2024-03-21T10:08:39.755 INFO:tasks.workunit.client.0.smithi153.stdout:6/262: write d1f/d2b/d33/f3c [4986779,68854] 0 2024-03-21T10:08:39.758 INFO:tasks.workunit.client.0.smithi153.stdout:6/263: fdatasync d1f/d2b/f38 0 2024-03-21T10:08:39.758 INFO:tasks.workunit.client.0.smithi153.stdout:8/70: truncate d5/d9/f11 1559797 0 2024-03-21T10:08:39.765 INFO:tasks.workunit.client.0.smithi153.stdout:4/119: rename d11/d18/c25 to d11/c2a 0 2024-03-21T10:08:39.765 INFO:tasks.workunit.client.0.smithi153.stdout:6/264: mknod d1f/d2f/c5a 0 2024-03-21T10:08:39.767 INFO:tasks.workunit.client.0.smithi153.stdout:6/265: chown cf 1 1 2024-03-21T10:08:39.768 INFO:tasks.workunit.client.0.smithi153.stdout:9/88: dread d1/d12/f19 [0,4194304] 0 2024-03-21T10:08:39.768 INFO:tasks.workunit.client.0.smithi153.stdout:4/120: mknod d11/d18/c2b 0 2024-03-21T10:08:39.771 INFO:tasks.workunit.client.0.smithi153.stdout:6/266: symlink d1f/d2b/d33/d40/d4f/l5b 0 2024-03-21T10:08:39.772 INFO:tasks.workunit.client.0.smithi153.stdout:9/89: rmdir d1/d4 39 2024-03-21T10:08:39.773 INFO:tasks.workunit.client.0.smithi153.stdout:6/267: truncate d1f/f27 413574 0 2024-03-21T10:08:39.774 INFO:tasks.workunit.client.0.smithi153.stdout:6/268: dread f5 [0,4194304] 0 2024-03-21T10:08:39.776 INFO:tasks.workunit.client.0.smithi153.stdout:4/121: link f1 d11/d18/f2c 0 2024-03-21T10:08:39.778 INFO:tasks.workunit.client.0.smithi153.stdout:9/90: symlink d1/d12/d1c/l21 0 2024-03-21T10:08:39.780 INFO:tasks.workunit.client.0.smithi153.stdout:9/91: symlink d1/d12/d1c/l22 0 2024-03-21T10:08:39.781 INFO:tasks.workunit.client.0.smithi153.stdout:9/92: dread - d1/f1b zero size 2024-03-21T10:08:39.785 INFO:tasks.workunit.client.0.smithi153.stdout:6/269: rmdir d1f 39 2024-03-21T10:08:39.786 INFO:tasks.workunit.client.0.smithi153.stdout:9/93: mknod d1/d12/d1c/c23 0 2024-03-21T10:08:39.787 INFO:tasks.workunit.client.0.smithi153.stdout:5/71: sync 2024-03-21T10:08:39.787 INFO:tasks.workunit.client.0.smithi153.stdout:1/56: sync 2024-03-21T10:08:39.788 INFO:tasks.workunit.client.0.smithi153.stdout:3/136: sync 2024-03-21T10:08:39.788 INFO:tasks.workunit.client.0.smithi153.stdout:0/53: sync 2024-03-21T10:08:39.788 INFO:tasks.workunit.client.0.smithi153.stdout:7/46: sync 2024-03-21T10:08:39.791 INFO:tasks.workunit.client.0.smithi153.stdout:5/72: stat d4/d5/df 0 2024-03-21T10:08:39.791 INFO:tasks.workunit.client.0.smithi153.stdout:7/47: fdatasync f2 0 2024-03-21T10:08:39.794 INFO:tasks.workunit.client.0.smithi153.stdout:6/270: symlink d1f/d2b/d33/d40/d4f/l5c 0 2024-03-21T10:08:39.795 INFO:tasks.workunit.client.0.smithi153.stdout:9/94: mkdir d1/d12/d1c/d24 0 2024-03-21T10:08:39.797 INFO:tasks.workunit.client.0.smithi153.stdout:8/71: dwrite f3 [0,4194304] 0 2024-03-21T10:08:39.801 INFO:tasks.workunit.client.0.smithi153.stdout:9/95: creat d1/d12/d1f/f25 x:0 0 0 2024-03-21T10:08:39.806 INFO:tasks.workunit.client.0.smithi153.stdout:8/72: stat d5/d9/dc 0 2024-03-21T10:08:39.806 INFO:tasks.workunit.client.0.smithi153.stdout:3/137: getdents dd/d1e 0 2024-03-21T10:08:39.807 INFO:tasks.workunit.client.0.smithi153.stdout:1/57: chown d6/f7 1640 1 2024-03-21T10:08:39.810 INFO:tasks.workunit.client.0.smithi153.stdout:9/96: creat d1/d12/f26 x:0 0 0 2024-03-21T10:08:39.813 INFO:tasks.workunit.client.0.smithi153.stdout:8/73: chown d5/c13 477149 1 2024-03-21T10:08:39.815 INFO:tasks.workunit.client.0.smithi153.stdout:4/122: dwrite f1 [0,4194304] 0 2024-03-21T10:08:39.816 INFO:tasks.workunit.client.0.smithi153.stdout:3/138: creat dd/f2b x:0 0 0 2024-03-21T10:08:39.822 INFO:tasks.workunit.client.0.smithi153.stdout:1/58: fdatasync d6/f9 0 2024-03-21T10:08:39.825 INFO:tasks.workunit.client.0.smithi153.stdout:3/139: creat dd/f2c x:0 0 0 2024-03-21T10:08:39.837 INFO:tasks.workunit.client.0.smithi153.stdout:0/54: dwrite d4/f10 [0,4194304] 0 2024-03-21T10:08:39.837 INFO:tasks.workunit.client.0.smithi153.stdout:8/74: link f3 d5/d9/dc/f16 0 2024-03-21T10:08:39.841 INFO:tasks.workunit.client.0.smithi153.stdout:1/59: mkdir d6/d13 0 2024-03-21T10:08:39.841 INFO:tasks.workunit.client.0.smithi153.stdout:3/140: truncate dd/f2b 854972 0 2024-03-21T10:08:39.851 INFO:tasks.workunit.client.0.smithi153.stdout:3/141: fdatasync f8 0 2024-03-21T10:08:39.857 INFO:tasks.workunit.client.0.smithi153.stdout:5/73: dread d4/fa [0,4194304] 0 2024-03-21T10:08:39.857 INFO:tasks.workunit.client.0.smithi153.stdout:0/55: read d4/f6 [294097,10474] 0 2024-03-21T10:08:39.861 INFO:tasks.workunit.client.0.smithi153.stdout:1/60: symlink d6/d10/l14 0 2024-03-21T10:08:39.861 INFO:tasks.workunit.client.0.smithi153.stdout:7/48: dwrite f2 [0,4194304] 0 2024-03-21T10:08:39.868 INFO:tasks.workunit.client.0.smithi153.stdout:3/142: dread - dd/f25 zero size 2024-03-21T10:08:39.871 INFO:tasks.workunit.client.0.smithi153.stdout:6/271: dwrite d1f/d2b/f46 [0,4194304] 0 2024-03-21T10:08:39.871 INFO:tasks.workunit.client.0.smithi153.stdout:5/74: write d4/d5/df/f1b [521257,1367] 0 2024-03-21T10:08:39.875 INFO:tasks.workunit.client.0.smithi153.stdout:1/61: creat d6/f15 x:0 0 0 2024-03-21T10:08:39.877 INFO:tasks.workunit.client.0.smithi153.stdout:9/97: dwrite d1/d12/f26 [0,4194304] 0 2024-03-21T10:08:39.883 INFO:tasks.workunit.client.0.smithi153.stdout:7/49: chown d4/c5 341869714 1 2024-03-21T10:08:39.885 INFO:tasks.workunit.client.0.smithi153.stdout:8/75: getdents d5/da 0 2024-03-21T10:08:39.887 INFO:tasks.workunit.client.0.smithi153.stdout:1/62: write d6/f7 [1119523,106532] 0 2024-03-21T10:08:39.888 INFO:tasks.workunit.client.0.smithi153.stdout:5/75: write d4/d5/f12 [2102194,69649] 0 2024-03-21T10:08:39.891 INFO:tasks.workunit.client.0.smithi153.stdout:9/98: read d1/d4/f7 [667507,105997] 0 2024-03-21T10:08:39.894 INFO:tasks.workunit.client.0.smithi153.stdout:4/123: dwrite d11/f24 [0,4194304] 0 2024-03-21T10:08:39.896 INFO:tasks.workunit.client.0.smithi153.stdout:6/272: getdents d1f/d2f 0 2024-03-21T10:08:39.898 INFO:tasks.workunit.client.0.smithi153.stdout:7/50: chown d4/c8 948098 1 2024-03-21T10:08:39.899 INFO:tasks.workunit.client.0.smithi153.stdout:3/143: dread f2 [0,4194304] 0 2024-03-21T10:08:39.905 INFO:tasks.workunit.client.0.smithi153.stdout:5/76: fsync d4/fb 0 2024-03-21T10:08:39.907 INFO:tasks.workunit.client.0.smithi153.stdout:3/144: creat dd/f2d x:0 0 0 2024-03-21T10:08:39.908 INFO:tasks.workunit.client.0.smithi153.stdout:0/56: dread f2 [0,4194304] 0 2024-03-21T10:08:39.913 INFO:tasks.workunit.client.0.smithi153.stdout:6/273: link d1f/d2b/d33/f3c d1f/d2b/d33/d40/f5d 0 2024-03-21T10:08:39.918 INFO:tasks.workunit.client.0.smithi153.stdout:5/77: getdents d4 0 2024-03-21T10:08:39.920 INFO:tasks.workunit.client.0.smithi153.stdout:6/274: mkdir d1f/d2f/d59/d5e 0 2024-03-21T10:08:39.924 INFO:tasks.workunit.client.0.smithi153.stdout:7/51: dread f2 [0,4194304] 0 2024-03-21T10:08:39.931 INFO:tasks.workunit.client.0.smithi153.stdout:5/78: truncate d4/fb 942436 0 2024-03-21T10:08:39.934 INFO:tasks.workunit.client.0.smithi153.stdout:9/99: dread d1/d4/f1a [0,4194304] 0 2024-03-21T10:08:39.935 INFO:tasks.workunit.client.0.smithi153.stdout:6/275: mkdir d1f/d2b/d33/d40/d5f 0 2024-03-21T10:08:39.941 INFO:tasks.workunit.client.0.smithi153.stdout:4/124: dwrite d11/f16 [0,4194304] 0 2024-03-21T10:08:39.941 INFO:tasks.workunit.client.0.smithi153.stdout:8/76: dwrite d5/d9/f12 [0,4194304] 0 2024-03-21T10:08:39.941 INFO:tasks.workunit.client.0.smithi153.stdout:6/276: chown d1f 46 1 2024-03-21T10:08:39.942 INFO:tasks.workunit.client.0.smithi153.stdout:5/79: chown d4/d5/f12 21966 1 2024-03-21T10:08:39.945 INFO:tasks.workunit.client.0.smithi153.stdout:6/277: symlink d1f/d2b/d33/d40/l60 0 2024-03-21T10:08:39.946 INFO:tasks.workunit.client.0.smithi153.stdout:4/125: chown d11/l17 4 1 2024-03-21T10:08:39.949 INFO:tasks.workunit.client.0.smithi153.stdout:6/278: write d1f/d2b/d33/d40/d51/f57 [304619,79126] 0 2024-03-21T10:08:39.952 INFO:tasks.workunit.client.0.smithi153.stdout:5/80: truncate d4/d5/f1a 721930 0 2024-03-21T10:08:39.954 INFO:tasks.workunit.client.0.smithi153.stdout:8/77: write d5/f10 [985980,16251] 0 2024-03-21T10:08:39.954 INFO:tasks.workunit.client.0.smithi153.stdout:7/52: rename d4/le to d4/lf 0 2024-03-21T10:08:39.957 INFO:tasks.workunit.client.0.smithi153.stdout:0/57: dwrite d4/f10 [0,4194304] 0 2024-03-21T10:08:39.958 INFO:tasks.workunit.client.0.smithi153.stdout:4/126: mkdir d11/d2d 0 2024-03-21T10:08:39.959 INFO:tasks.workunit.client.0.smithi153.stdout:9/100: creat d1/f27 x:0 0 0 2024-03-21T10:08:39.968 INFO:tasks.workunit.client.0.smithi153.stdout:6/279: mknod d1f/d2b/d33/d40/d4f/c61 0 2024-03-21T10:08:39.968 INFO:tasks.workunit.client.0.smithi153.stdout:8/78: write d5/d9/dc/f16 [3079811,96969] 0 2024-03-21T10:08:39.968 INFO:tasks.workunit.client.0.smithi153.stdout:6/280: read d1f/f36 [208933,2080] 0 2024-03-21T10:08:39.971 INFO:tasks.workunit.client.0.smithi153.stdout:6/281: dread - d1f/d2f/f47 zero size 2024-03-21T10:08:39.973 INFO:tasks.workunit.client.0.smithi153.stdout:7/53: creat d4/f10 x:0 0 0 2024-03-21T10:08:39.974 INFO:tasks.workunit.client.0.smithi153.stdout:8/79: creat d5/d9/f17 x:0 0 0 2024-03-21T10:08:39.977 INFO:tasks.workunit.client.0.smithi153.stdout:4/127: link d11/d18/c2b d11/d18/c2e 0 2024-03-21T10:08:39.988 INFO:tasks.workunit.client.0.smithi153.stdout:6/282: rename d1f/d2b/c43 to d1f/d2f/d59/c62 0 2024-03-21T10:08:39.988 INFO:tasks.workunit.client.0.smithi153.stdout:4/128: read d11/f1e [563053,91189] 0 2024-03-21T10:08:39.989 INFO:tasks.workunit.client.0.smithi153.stdout:8/80: creat d5/da/f18 x:0 0 0 2024-03-21T10:08:39.990 INFO:tasks.workunit.client.0.smithi153.stdout:7/54: dread d4/fd [0,4194304] 0 2024-03-21T10:08:39.992 INFO:tasks.workunit.client.0.smithi153.stdout:5/81: dwrite f1 [0,4194304] 0 2024-03-21T10:08:39.996 INFO:tasks.workunit.client.0.smithi153.stdout:8/81: dread f4 [0,4194304] 0 2024-03-21T10:08:39.997 INFO:tasks.workunit.client.0.smithi153.stdout:6/283: creat d1f/d2b/f63 x:0 0 0 2024-03-21T10:08:39.999 INFO:tasks.workunit.client.0.smithi153.stdout:5/82: chown d4/fb 0 1 2024-03-21T10:08:40.001 INFO:tasks.workunit.client.0.smithi153.stdout:7/55: mkdir d4/d11 0 2024-03-21T10:08:40.004 INFO:tasks.workunit.client.0.smithi153.stdout:5/83: dread - d4/d5/de/f18 zero size 2024-03-21T10:08:40.009 INFO:tasks.workunit.client.0.smithi153.stdout:9/101: dwrite d1/d4/fe [0,4194304] 0 2024-03-21T10:08:40.009 INFO:tasks.workunit.client.0.smithi153.stdout:5/84: unlink d4/d5/f17 0 2024-03-21T10:08:40.011 INFO:tasks.workunit.client.0.smithi153.stdout:7/56: creat d4/f12 x:0 0 0 2024-03-21T10:08:40.017 INFO:tasks.workunit.client.0.smithi153.stdout:5/85: rmdir d4/d5/de 39 2024-03-21T10:08:40.018 INFO:tasks.workunit.client.0.smithi153.stdout:7/57: write f2 [3099683,42381] 0 2024-03-21T10:08:40.019 INFO:tasks.workunit.client.1.smithi174.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-03-21T10:08:40.020 INFO:tasks.workunit.client.0.smithi153.stdout:9/102: symlink d1/d12/d1f/l28 0 2024-03-21T10:08:40.024 INFO:tasks.workunit.client.0.smithi153.stdout:4/129: dwrite d11/f20 [0,4194304] 0 2024-03-21T10:08:40.025 INFO:tasks.workunit.client.0.smithi153.stdout:7/58: dread - d4/f10 zero size 2024-03-21T10:08:40.027 INFO:tasks.workunit.client.0.smithi153.stdout:6/284: dwrite f1b [0,4194304] 0 2024-03-21T10:08:40.029 INFO:tasks.workunit.client.0.smithi153.stdout:5/86: write d4/d5/f1a [15274,126360] 0 2024-03-21T10:08:40.029 INFO:tasks.workunit.client.0.smithi153.stdout:9/103: write d1/d12/f26 [4436253,97944] 0 2024-03-21T10:08:40.037 INFO:tasks.workunit.client.0.smithi153.stdout:9/104: truncate d1/f27 98704 0 2024-03-21T10:08:40.040 INFO:tasks.workunit.client.0.smithi153.stdout:5/87: unlink d4/fb 0 2024-03-21T10:08:40.041 INFO:tasks.workunit.client.0.smithi153.stdout:7/59: symlink d4/d11/l13 0 2024-03-21T10:08:40.045 INFO:tasks.workunit.client.0.smithi153.stdout:7/60: rmdir d4/d11 39 2024-03-21T10:08:40.045 INFO:tasks.workunit.client.0.smithi153.stdout:6/285: creat d1f/d2b/f64 x:0 0 0 2024-03-21T10:08:40.047 INFO:tasks.workunit.client.0.smithi153.stdout:5/88: creat d4/d5/d16/f1c x:0 0 0 2024-03-21T10:08:40.047 INFO:tasks.workunit.client.0.smithi153.stdout:6/286: chown f5 898276 1 2024-03-21T10:08:40.048 INFO:tasks.workunit.client.0.smithi153.stdout:7/61: truncate d4/f10 510182 0 2024-03-21T10:08:40.048 INFO:tasks.workunit.client.0.smithi153.stdout:5/89: stat d4/d8/f11 0 2024-03-21T10:08:40.049 INFO:tasks.workunit.client.0.smithi153.stdout:6/287: dread - d1f/d2b/f64 zero size 2024-03-21T10:08:40.051 INFO:tasks.workunit.client.0.smithi153.stdout:4/130: dread fb [0,4194304] 0 2024-03-21T10:08:40.051 INFO:tasks.workunit.client.0.smithi153.stdout:5/90: write d4/d8/f11 [325680,39892] 0 2024-03-21T10:08:40.054 INFO:tasks.workunit.client.0.smithi153.stdout:6/288: dread - d1f/d2b/f64 zero size 2024-03-21T10:08:40.054 INFO:tasks.workunit.client.1.smithi174.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-03-21T10:08:40.054 INFO:tasks.workunit.client.1.smithi174.stderr:+ make 2024-03-21T10:08:40.055 INFO:tasks.workunit.client.0.smithi153.stdout:4/131: write f1 [4465986,11727] 0 2024-03-21T10:08:40.055 INFO:tasks.workunit.client.0.smithi153.stdout:5/91: unlink d4/d5/cd 0 2024-03-21T10:08:40.055 INFO:tasks.workunit.client.0.smithi153.stdout:5/92: readlink - no filename 2024-03-21T10:08:40.057 INFO:tasks.workunit.client.0.smithi153.stdout:4/132: write d11/f19 [926345,90130] 0 2024-03-21T10:08:40.058 INFO:tasks.workunit.client.0.smithi153.stdout:6/289: mkdir d1f/d2b/d33/d40/d65 0 2024-03-21T10:08:40.060 INFO:tasks.workunit.client.0.smithi153.stdout:5/93: creat d4/d8/f1d x:0 0 0 2024-03-21T10:08:40.061 INFO:tasks.workunit.client.0.smithi153.stdout:6/290: read d1f/f20 [1142282,120009] 0 2024-03-21T10:08:40.062 INFO:tasks.workunit.client.0.smithi153.stdout:5/94: chown d4/d5/f12 111639508 1 2024-03-21T10:08:40.063 INFO:tasks.workunit.client.0.smithi153.stdout:7/62: dread f2 [0,4194304] 0 2024-03-21T10:08:40.064 INFO:tasks.workunit.client.0.smithi153.stdout:6/291: mkdir d1f/d2b/d33/d40/d4f/d66 0 2024-03-21T10:08:40.067 INFO:tasks.workunit.client.0.smithi153.stdout:7/63: mkdir d4/d14 0 2024-03-21T10:08:40.071 INFO:tasks.workunit.client.0.smithi153.stdout:7/64: readlink d4/d11/l13 0 2024-03-21T10:08:40.073 INFO:tasks.workunit.client.0.smithi153.stdout:7/65: write d4/f12 [70439,68468] 0 2024-03-21T10:08:40.075 INFO:tasks.workunit.client.0.smithi153.stdout:2/110: sync 2024-03-21T10:08:40.082 INFO:tasks.workunit.client.0.smithi153.stdout:4/133: dwrite d11/f1e [0,4194304] 0 2024-03-21T10:08:40.090 INFO:tasks.workunit.client.0.smithi153.stdout:4/134: creat d11/d2d/f2f x:0 0 0 2024-03-21T10:08:40.093 INFO:tasks.workunit.client.0.smithi153.stdout:6/292: dwrite f5 [0,4194304] 0 2024-03-21T10:08:40.094 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:40.093+0000 7fa0d2200700 1 -- 172.21.15.153:0/742145548 --> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa0cc0611d0 con 0x7fa0b00394e0 2024-03-21T10:08:40.096 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:40.095+0000 7fa0c3e00700 1 -- 172.21.15.153:0/742145548 <== mgr.14666 v2:172.21.15.153:6800/2524145044 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+329 (secure 0 0 0) 0x7fa0cc0611d0 con 0x7fa0b00394e0 2024-03-21T10:08:40.100 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:40.099+0000 7fa0d2200700 1 -- 172.21.15.153:0/742145548 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7fa0b00394e0 msgr2=0x7fa0b003b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:08:40.100 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:40.099+0000 7fa0d2200700 1 --2- 172.21.15.153:0/742145548 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7fa0b00394e0 0x7fa0b003b9a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc006010 tx=0x7fa0bc00b540 comp rx=0 tx=0).stop 2024-03-21T10:08:40.100 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:40.099+0000 7fa0d2200700 1 -- 172.21.15.153:0/742145548 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa0cc101da0 msgr2=0x7fa0cc10d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:08:40.100 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:40.099+0000 7fa0d2200700 1 --2- 172.21.15.153:0/742145548 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa0cc101da0 0x7fa0cc10d540 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fa0b800d8d0 tx=0x7fa0b800dc90 comp rx=0 tx=0).stop 2024-03-21T10:08:40.101 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:40.100+0000 7fa0d2200700 1 -- 172.21.15.153:0/742145548 shutdown_connections 2024-03-21T10:08:40.101 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:40.100+0000 7fa0d2200700 1 --2- 172.21.15.153:0/742145548 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7fa0b00394e0 0x7fa0b003b9a0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:08:40.101 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:40.100+0000 7fa0d2200700 1 --2- 172.21.15.153:0/742145548 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa0cc0ff430 0x7fa0cc10d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:08:40.101 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:40.100+0000 7fa0d2200700 1 --2- 172.21.15.153:0/742145548 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa0cc101da0 0x7fa0cc10d540 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:08:40.101 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:40.100+0000 7fa0d2200700 1 -- 172.21.15.153:0/742145548 >> 172.21.15.153:0/742145548 conn(0x7fa0cc0fb030 msgr2=0x7fa0cc0fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:08:40.101 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:40.100+0000 7fa0d2200700 1 -- 172.21.15.153:0/742145548 shutdown_connections 2024-03-21T10:08:40.101 INFO:tasks.workunit.client.0.smithi153.stdout:4/135: creat d11/d2d/f30 x:0 0 0 2024-03-21T10:08:40.102 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:40.101+0000 7fa0d2200700 1 -- 172.21.15.153:0/742145548 wait complete. 2024-03-21T10:08:40.105 INFO:tasks.workunit.client.0.smithi153.stdout:4/136: fsync d11/d18/f21 0 2024-03-21T10:08:40.107 INFO:tasks.workunit.client.0.smithi153.stdout:6/293: write d1f/f35 [882992,47904] 0 2024-03-21T10:08:40.111 INFO:tasks.workunit.client.0.smithi153.stdout:7/66: dwrite f2 [4194304,4194304] 0 2024-03-21T10:08:40.112 INFO:tasks.workunit.client.0.smithi153.stdout:2/111: dwrite dd/d15/f1d [0,4194304] 0 2024-03-21T10:08:40.113 INFO:tasks.workunit.client.0.smithi153.stdout:4/137: write d11/d2d/f30 [653237,46106] 0 2024-03-21T10:08:40.116 INFO:tasks.workunit.client.0.smithi153.stdout:6/294: creat d1f/d2b/d33/d40/d4f/f67 x:0 0 0 2024-03-21T10:08:40.117 INFO:tasks.workunit.client.0.smithi153.stdout:7/67: write d4/f12 [1168521,69423] 0 2024-03-21T10:08:40.127 INFO:tasks.workunit.client.0.smithi153.stdout:2/112: symlink dd/l23 0 2024-03-21T10:08:40.131 INFO:tasks.workunit.client.0.smithi153.stdout:6/295: dread d1f/f35 [0,4194304] 0 2024-03-21T10:08:40.131 INFO:tasks.workunit.client.0.smithi153.stdout:7/68: chown d4/cc 258639089 1 2024-03-21T10:08:40.133 INFO:tasks.workunit.client.0.smithi153.stdout:2/113: creat dd/d15/d18/f24 x:0 0 0 2024-03-21T10:08:40.140 INFO:tasks.workunit.client.0.smithi153.stdout:6/296: mknod d1f/d2f/d59/c68 0 2024-03-21T10:08:40.141 INFO:tasks.workunit.client.0.smithi153.stdout:4/138: getdents d11/d2d 0 2024-03-21T10:08:40.142 INFO:tasks.workunit.client.0.smithi153.stdout:6/297: readlink d1f/d2f/l3f 0 2024-03-21T10:08:40.144 INFO:tasks.workunit.client.0.smithi153.stdout:6/298: write d1f/f23 [2413270,37025] 0 2024-03-21T10:08:40.146 INFO:tasks.workunit.client.0.smithi153.stdout:6/299: fsync f15 0 2024-03-21T10:08:40.150 INFO:tasks.workunit.client.0.smithi153.stdout:7/69: dread f2 [0,4194304] 0 2024-03-21T10:08:40.151 INFO:tasks.workunit.client.0.smithi153.stdout:4/139: rename d11/l15 to d11/d18/l31 0 2024-03-21T10:08:40.152 INFO:tasks.workunit.client.0.smithi153.stdout:4/140: read f2 [4077965,99244] 0 2024-03-21T10:08:40.153 INFO:tasks.workunit.client.0.smithi153.stdout:6/300: creat d1f/d2b/d33/d40/d65/f69 x:0 0 0 2024-03-21T10:08:40.154 INFO:tasks.workunit.client.0.smithi153.stdout:4/141: fdatasync d11/f1e 0 2024-03-21T10:08:40.155 INFO:tasks.workunit.client.0.smithi153.stdout:7/70: symlink d4/d14/l15 0 2024-03-21T10:08:40.157 INFO:tasks.workunit.client.0.smithi153.stdout:6/301: symlink d1f/l6a 0 2024-03-21T10:08:40.159 INFO:tasks.workunit.client.0.smithi153.stdout:4/142: getdents d11/d2d 0 2024-03-21T10:08:40.159 INFO:tasks.workunit.client.0.smithi153.stdout:7/71: dread d4/f10 [0,4194304] 0 2024-03-21T10:08:40.159 INFO:tasks.workunit.client.0.smithi153.stdout:2/114: dwrite dd/d15/f1d [0,4194304] 0 2024-03-21T10:08:40.161 INFO:tasks.workunit.client.0.smithi153.stdout:4/143: dread - fd zero size 2024-03-21T10:08:40.161 INFO:tasks.workunit.client.0.smithi153.stdout:2/115: dread - dd/d15/f1e zero size 2024-03-21T10:08:40.163 INFO:tasks.workunit.client.0.smithi153.stdout:7/72: readlink d4/lf 0 2024-03-21T10:08:40.167 INFO:tasks.workunit.client.0.smithi153.stdout:6/302: mknod d1f/d2b/d33/d40/d65/c6b 0 2024-03-21T10:08:40.169 INFO:tasks.workunit.client.0.smithi153.stdout:7/73: mkdir d4/d14/d16 0 2024-03-21T10:08:40.169 INFO:tasks.workunit.client.0.smithi153.stdout:6/303: fsync d1f/d2b/d33/f41 0 2024-03-21T10:08:40.169 INFO:tasks.workunit.client.0.smithi153.stdout:4/144: dread d11/f26 [0,4194304] 0 2024-03-21T10:08:40.171 INFO:tasks.workunit.client.0.smithi153.stdout:6/304: readlink d1f/d2f/l56 0 2024-03-21T10:08:40.172 INFO:tasks.workunit.client.0.smithi153.stdout:4/145: dread - d11/d2d/f2f zero size 2024-03-21T10:08:40.172 INFO:tasks.workunit.client.0.smithi153.stdout:7/74: symlink d4/d11/l17 0 2024-03-21T10:08:40.174 INFO:tasks.workunit.client.0.smithi153.stdout:6/305: unlink c1c 0 2024-03-21T10:08:40.175 INFO:tasks.workunit.client.0.smithi153.stdout:7/75: fdatasync d4/f9 0 2024-03-21T10:08:40.177 INFO:tasks.workunit.client.0.smithi153.stdout:7/76: readlink d4/lf 0 2024-03-21T10:08:40.178 INFO:tasks.workunit.client.0.smithi153.stdout:6/306: creat d1f/f6c x:0 0 0 2024-03-21T10:08:40.181 INFO:tasks.workunit.client.0.smithi153.stdout:2/116: dwrite dd/f10 [0,4194304] 0 2024-03-21T10:08:40.181 INFO:tasks.workunit.client.0.smithi153.stdout:7/77: mknod d4/d14/d16/c18 0 2024-03-21T10:08:40.183 INFO:tasks.workunit.client.0.smithi153.stdout:7/78: chown c3 248 1 2024-03-21T10:08:40.184 INFO:tasks.workunit.client.0.smithi153.stdout:4/146: dread d11/f19 [0,4194304] 0 2024-03-21T10:08:40.187 INFO:tasks.workunit.client.0.smithi153.stdout:6/307: rename d1f/f2d to d1f/d2b/d33/d40/d5f/f6d 0 2024-03-21T10:08:40.188 INFO:tasks.workunit.client.0.smithi153.stdout:2/117: mknod dd/d15/d18/c25 0 2024-03-21T10:08:40.189 INFO:tasks.workunit.client.0.smithi153.stdout:7/79: fdatasync d4/f10 0 2024-03-21T10:08:40.193 INFO:tasks.workunit.client.0.smithi153.stdout:4/147: dread d11/d2d/f30 [0,4194304] 0 2024-03-21T10:08:40.194 INFO:tasks.workunit.client.0.smithi153.stdout:2/118: rename dd/d15/f1e to dd/f26 0 2024-03-21T10:08:40.197 INFO:tasks.workunit.client.0.smithi153.stdout:7/80: fsync d4/fa 0 2024-03-21T10:08:40.199 INFO:tasks.workunit.client.0.smithi153.stdout:2/119: unlink l1 0 2024-03-21T10:08:40.204 INFO:tasks.workunit.client.0.smithi153.stdout:2/120: symlink dd/d15/l27 0 2024-03-21T10:08:40.209 INFO:tasks.workunit.client.0.smithi153.stdout:6/308: dwrite d1f/d2b/f46 [0,4194304] 0 2024-03-21T10:08:40.210 INFO:tasks.workunit.client.0.smithi153.stdout:2/121: dread fb [0,4194304] 0 2024-03-21T10:08:40.217 INFO:tasks.workunit.client.0.smithi153.stdout:2/122: symlink dd/d15/d18/l28 0 2024-03-21T10:08:40.217 INFO:tasks.workunit.client.0.smithi153.stdout:2/123: readlink l8 0 2024-03-21T10:08:40.220 INFO:tasks.workunit.client.0.smithi153.stdout:4/148: truncate d11/d18/f28 2295178 0 2024-03-21T10:08:40.223 INFO:tasks.workunit.client.0.smithi153.stdout:2/124: dread dd/f14 [0,4194304] 0 2024-03-21T10:08:40.226 INFO:tasks.workunit.client.0.smithi153.stdout:4/149: write d11/d18/f21 [3831772,43091] 0 2024-03-21T10:08:40.228 INFO:tasks.workunit.client.0.smithi153.stdout:7/81: dwrite d4/f9 [0,4194304] 0 2024-03-21T10:08:40.229 INFO:tasks.workunit.client.0.smithi153.stdout:4/150: chown d11 770900 1 2024-03-21T10:08:40.231 INFO:tasks.workunit.client.0.smithi153.stdout:2/125: creat dd/d15/d18/f29 x:0 0 0 2024-03-21T10:08:40.234 INFO:tasks.workunit.client.0.smithi153.stdout:7/82: write f2 [3905603,83742] 0 2024-03-21T10:08:40.240 INFO:tasks.workunit.client.0.smithi153.stdout:4/151: write d11/f12 [960744,69582] 0 2024-03-21T10:08:40.240 INFO:tasks.workunit.client.0.smithi153.stdout:6/309: dwrite d1f/f4b [0,4194304] 0 2024-03-21T10:08:40.241 INFO:tasks.workunit.client.0.smithi153.stdout:7/83: stat d4/lf 0 2024-03-21T10:08:40.241 INFO:tasks.workunit.client.0.smithi153.stdout:4/152: chown d11/f26 53000 1 2024-03-21T10:08:40.246 INFO:tasks.workunit.client.0.smithi153.stdout:6/310: dread d1f/f35 [0,4194304] 0 2024-03-21T10:08:40.248 INFO:tasks.workunit.client.0.smithi153.stdout:4/153: mknod d11/d2d/c32 0 2024-03-21T10:08:40.249 INFO:tasks.workunit.client.0.smithi153.stdout:7/84: mkdir d4/d19 0 2024-03-21T10:08:40.250 INFO:tasks.workunit.client.0.smithi153.stdout:6/311: read - d1f/d2b/d33/f41 zero size 2024-03-21T10:08:40.256 INFO:tasks.workunit.client.0.smithi153.stdout:6/312: unlink cf 0 2024-03-21T10:08:40.256 INFO:tasks.workunit.client.0.smithi153.stdout:4/154: mknod d11/c33 0 2024-03-21T10:08:40.259 INFO:tasks.workunit.client.0.smithi153.stdout:2/126: dwrite dd/f12 [0,4194304] 0 2024-03-21T10:08:40.261 INFO:tasks.workunit.client.0.smithi153.stdout:7/85: creat d4/f1a x:0 0 0 2024-03-21T10:08:40.261 INFO:tasks.workunit.client.0.smithi153.stdout:6/313: fsync d1f/d2b/d33/d40/f5d 0 2024-03-21T10:08:40.263 INFO:tasks.workunit.client.0.smithi153.stdout:7/86: read d4/fa [1699884,111766] 0 2024-03-21T10:08:40.266 INFO:tasks.workunit.client.0.smithi153.stdout:7/87: mkdir d4/d1b 0 2024-03-21T10:08:40.269 INFO:tasks.workunit.client.0.smithi153.stdout:4/155: getdents d11/d2d 0 2024-03-21T10:08:40.271 INFO:tasks.workunit.client.0.smithi153.stdout:7/88: link d4/d11/l13 d4/d1b/l1c 0 2024-03-21T10:08:40.272 INFO:tasks.workunit.client.0.smithi153.stdout:4/156: mknod d11/d18/c34 0 2024-03-21T10:08:40.275 INFO:tasks.workunit.client.0.smithi153.stdout:7/89: rename c1 to d4/d11/c1d 0 2024-03-21T10:08:40.277 INFO:tasks.workunit.client.0.smithi153.stdout:4/157: creat d11/d18/f35 x:0 0 0 2024-03-21T10:08:40.278 INFO:tasks.workunit.client.0.smithi153.stdout:4/158: chown d11/d18/l1a 74 1 2024-03-21T10:08:40.281 INFO:tasks.workunit.client.0.smithi153.stdout:6/314: truncate d1f/d2b/d33/d40/d5f/f6d 3623891 0 2024-03-21T10:08:40.281 INFO:tasks.workunit.client.0.smithi153.stdout:4/159: truncate fe 4554347 0 2024-03-21T10:08:40.282 INFO:tasks.workunit.client.0.smithi153.stdout:4/160: creat d11/d2d/f36 x:0 0 0 2024-03-21T10:08:40.283 INFO:tasks.workunit.client.0.smithi153.stdout:6/315: link d1f/d2b/f38 d1f/d2f/d59/f6e 0 2024-03-21T10:08:40.287 INFO:tasks.workunit.client.0.smithi153.stdout:6/316: creat d1f/d2b/d33/d40/d4f/d66/f6f x:0 0 0 2024-03-21T10:08:40.290 INFO:tasks.workunit.client.0.smithi153.stdout:6/317: mkdir d1f/d2f/d59/d70 0 2024-03-21T10:08:40.296 INFO:tasks.workunit.client.0.smithi153.stdout:6/318: link d1f/d2b/d33/d40/d4f/d66/f6f d1f/d2b/d33/d40/d51/f71 0 2024-03-21T10:08:40.297 INFO:tasks.workunit.client.0.smithi153.stdout:4/161: dwrite d11/d18/f1f [0,4194304] 0 2024-03-21T10:08:40.297 INFO:tasks.workunit.client.0.smithi153.stdout:6/319: dread - d1f/f4e zero size 2024-03-21T10:08:40.299 INFO:tasks.workunit.client.0.smithi153.stdout:6/320: fdatasync d1f/f39 0 2024-03-21T10:08:40.299 INFO:tasks.workunit.client.0.smithi153.stdout:4/162: chown d11/f24 266896 1 2024-03-21T10:08:40.306 INFO:tasks.workunit.client.0.smithi153.stdout:6/321: creat d1f/d2b/d33/d40/d5f/f72 x:0 0 0 2024-03-21T10:08:40.310 INFO:tasks.workunit.client.0.smithi153.stdout:6/322: mknod d1f/d2b/d33/c73 0 2024-03-21T10:08:40.311 INFO:tasks.workunit.client.0.smithi153.stdout:6/323: stat f18 0 2024-03-21T10:08:40.313 INFO:tasks.workunit.client.0.smithi153.stdout:6/324: creat d1f/d2f/f74 x:0 0 0 2024-03-21T10:08:40.314 INFO:tasks.workunit.client.0.smithi153.stdout:4/163: dwrite d11/f20 [0,4194304] 0 2024-03-21T10:08:40.315 INFO:tasks.workunit.client.0.smithi153.stdout:4/164: dread - fd zero size 2024-03-21T10:08:40.320 INFO:tasks.workunit.client.0.smithi153.stdout:6/325: link d1f/d2b/d33/d40/f58 d1f/d2b/d33/d40/d4f/d66/f75 0 2024-03-21T10:08:40.321 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:40.320+0000 7f5fb7a00700 1 -- 172.21.15.153:0/1771980341 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5fb00ff430 msgr2=0x7f5fb0101860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:08:40.321 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:40.320+0000 7f5fb7a00700 1 --2- 172.21.15.153:0/1771980341 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5fb00ff430 0x7f5fb0101860 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f5fa4009910 tx=0x7f5fa4009c20 comp rx=0 tx=0).stop 2024-03-21T10:08:40.321 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:40.320+0000 7f5fb7a00700 1 -- 172.21.15.153:0/1771980341 shutdown_connections 2024-03-21T10:08:40.321 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:40.320+0000 7f5fb7a00700 1 --2- 172.21.15.153:0/1771980341 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5fb0101da0 0x7f5fb0104230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:08:40.321 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:40.320+0000 7f5fb7a00700 1 --2- 172.21.15.153:0/1771980341 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5fb00ff430 0x7f5fb0101860 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:08:40.321 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:40.320+0000 7f5fb7a00700 1 -- 172.21.15.153:0/1771980341 >> 172.21.15.153:0/1771980341 conn(0x7f5fb00fb030 msgr2=0x7f5fb00fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:08:40.321 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:40.320+0000 7f5fb7a00700 1 -- 172.21.15.153:0/1771980341 shutdown_connections 2024-03-21T10:08:40.321 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:40.321+0000 7f5fb7a00700 1 -- 172.21.15.153:0/1771980341 wait complete. 2024-03-21T10:08:40.322 INFO:tasks.workunit.client.0.smithi153.stdout:6/326: read f1b [582707,50657] 0 2024-03-21T10:08:40.322 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:40.322+0000 7f5fb7a00700 1 Processor -- start 2024-03-21T10:08:40.323 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:40.322+0000 7f5fb7a00700 1 -- start start 2024-03-21T10:08:40.323 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:40.322+0000 7f5fb7a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5fb00ff430 0x7f5fb019c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:08:40.323 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:40.322+0000 7f5fb7a00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5fb0101da0 0x7f5fb019ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:08:40.323 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:40.322+0000 7f5fb7a00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fb019d470 con 0x7f5fb00ff430 2024-03-21T10:08:40.323 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:40.322+0000 7f5fb7a00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fb019d5b0 con 0x7f5fb0101da0 2024-03-21T10:08:40.323 INFO:tasks.workunit.client.0.smithi153.stdout:6/327: write f11 [4497237,77976] 0 2024-03-21T10:08:40.324 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:40.323+0000 7f5fb5000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5fb00ff430 0x7f5fb019c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:08:40.324 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:40.323+0000 7f5fb5000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5fb00ff430 0x7f5fb019c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:58916/0 (socket says 172.21.15.153:58916) 2024-03-21T10:08:40.324 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:40.323+0000 7f5fb5000700 1 -- 172.21.15.153:0/3922679859 learned_addr learned my addr 172.21.15.153:0/3922679859 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:08:40.324 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:40.323+0000 7f5fafe00700 1 --2- 172.21.15.153:0/3922679859 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5fb0101da0 0x7f5fb019ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:08:40.324 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:40.324+0000 7f5fb5000700 1 -- 172.21.15.153:0/3922679859 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5fb0101da0 msgr2=0x7f5fb019ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:08:40.324 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:40.324+0000 7f5fb5000700 1 --2- 172.21.15.153:0/3922679859 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5fb0101da0 0x7f5fb019ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:08:40.324 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:40.324+0000 7f5fb5000700 1 -- 172.21.15.153:0/3922679859 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fa40095f0 con 0x7f5fb00ff430 2024-03-21T10:08:40.324 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:40.324+0000 7f5fafe00700 1 --2- 172.21.15.153:0/3922679859 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5fb0101da0 0x7f5fb019ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:08:40.325 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:40.324+0000 7f5fb5000700 1 --2- 172.21.15.153:0/3922679859 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5fb00ff430 0x7f5fb019c910 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f5fa400bd70 tx=0x7f5fa400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:08:40.325 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:40.324+0000 7f5fad600700 1 -- 172.21.15.153:0/3922679859 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5fa401d070 con 0x7f5fb00ff430 2024-03-21T10:08:40.325 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:40.325+0000 7f5fad600700 1 -- 172.21.15.153:0/3922679859 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f5fa400f460 con 0x7f5fb00ff430 2024-03-21T10:08:40.325 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:40.325+0000 7f5fb7a00700 1 -- 172.21.15.153:0/3922679859 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fb0105b90 con 0x7f5fb00ff430 2024-03-21T10:08:40.325 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:40.325+0000 7f5fad600700 1 -- 172.21.15.153:0/3922679859 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5fa4021640 con 0x7f5fb00ff430 2024-03-21T10:08:40.326 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:40.325+0000 7f5fb7a00700 1 -- 172.21.15.153:0/3922679859 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fb0106080 con 0x7f5fb00ff430 2024-03-21T10:08:40.327 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:40.326+0000 7f5fad600700 1 -- 172.21.15.153:0/3922679859 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 28) v1 ==== 46435+0+0 (secure 0 0 0) 0x7f5fa402b440 con 0x7f5fb00ff430 2024-03-21T10:08:40.327 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:40.326+0000 7f5fb7a00700 1 -- 172.21.15.153:0/3922679859 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fb0196b10 con 0x7f5fb00ff430 2024-03-21T10:08:40.327 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:40.326+0000 7f5fad600700 1 --2- 172.21.15.153:0/3922679859 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f5f980394e0 0x7f5f9803b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:08:40.327 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:40.327+0000 7f5fad600700 1 -- 172.21.15.153:0/3922679859 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f5fa404eb50 con 0x7f5fb00ff430 2024-03-21T10:08:40.328 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:40.327+0000 7f5fafe00700 1 --2- 172.21.15.153:0/3922679859 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f5f980394e0 0x7f5f9803b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:08:40.328 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:40.328+0000 7f5fafe00700 1 --2- 172.21.15.153:0/3922679859 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f5f980394e0 0x7f5f9803b9a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c005950 tx=0x7f5f9c0058e0 comp rx=0 tx=0).ready entity=mgr.14666 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:08:40.330 INFO:tasks.workunit.client.0.smithi153.stdout:6/328: rename d1f/d2b/c30 to d1f/d2b/d33/d40/d51/c76 0 2024-03-21T10:08:40.333 INFO:tasks.workunit.client.0.smithi153.stdout:6/329: creat d1f/d2b/d33/d40/d4f/f77 x:0 0 0 2024-03-21T10:08:40.337 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:40.336+0000 7f5fad600700 1 -- 172.21.15.153:0/3922679859 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185454 (secure 0 0 0) 0x7f5fa4029910 con 0x7f5fb00ff430 2024-03-21T10:08:40.345 INFO:tasks.workunit.client.0.smithi153.stdout:6/330: rename d1f/d2b/l3a to d1f/d2b/d33/d40/d4f/l78 0 2024-03-21T10:08:40.348 INFO:tasks.workunit.client.0.smithi153.stdout:4/165: write d11/d18/f28 [973535,21107] 0 2024-03-21T10:08:40.349 INFO:tasks.workunit.client.0.smithi153.stdout:6/331: symlink d1f/d2b/d33/d40/l79 0 2024-03-21T10:08:40.351 INFO:tasks.workunit.client.0.smithi153.stdout:4/166: truncate d11/d2d/f2f 155379 0 2024-03-21T10:08:40.353 INFO:tasks.workunit.client.0.smithi153.stdout:6/332: rename d1f/c21 to d1f/d2b/d33/c7a 0 2024-03-21T10:08:40.353 INFO:tasks.workunit.client.1.smithi174.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-03-21T10:08:40.354 INFO:tasks.workunit.client.0.smithi153.stdout:6/333: fdatasync d1f/f27 0 2024-03-21T10:08:40.354 INFO:tasks.workunit.client.0.smithi153.stdout:4/167: symlink d11/d18/l37 0 2024-03-21T10:08:40.356 INFO:tasks.workunit.client.0.smithi153.stdout:4/168: creat d11/d2d/f38 x:0 0 0 2024-03-21T10:08:40.357 INFO:tasks.workunit.client.0.smithi153.stdout:6/334: mkdir d1f/d2b/d33/d40/d7b 0 2024-03-21T10:08:40.359 INFO:tasks.workunit.client.0.smithi153.stdout:4/169: creat d11/f39 x:0 0 0 2024-03-21T10:08:40.360 INFO:tasks.workunit.client.0.smithi153.stdout:6/335: creat d1f/d2b/d33/d40/d4f/d66/f7c x:0 0 0 2024-03-21T10:08:40.362 INFO:tasks.workunit.client.0.smithi153.stdout:6/336: creat d1f/d2f/f7d x:0 0 0 2024-03-21T10:08:40.365 INFO:tasks.workunit.client.0.smithi153.stdout:6/337: unlink d1f/d2b/l37 0 2024-03-21T10:08:40.367 INFO:tasks.workunit.client.0.smithi153.stdout:4/170: rmdir d11/d18 39 2024-03-21T10:08:40.375 INFO:tasks.workunit.client.0.smithi153.stdout:4/171: rename d11/d18/f2c to d11/d2d/f3a 0 2024-03-21T10:08:40.378 INFO:tasks.workunit.client.0.smithi153.stdout:4/172: symlink d11/d18/l3b 0 2024-03-21T10:08:40.378 INFO:tasks.workunit.client.0.smithi153.stdout:4/173: chown ca 273625 1 2024-03-21T10:08:40.379 INFO:tasks.workunit.client.0.smithi153.stdout:4/174: readlink d11/l29 0 2024-03-21T10:08:40.389 INFO:tasks.workunit.client.0.smithi153.stdout:4/175: dread d11/d18/f1f [0,4194304] 0 2024-03-21T10:08:40.391 INFO:tasks.workunit.client.0.smithi153.stdout:4/176: readlink d11/d18/l37 0 2024-03-21T10:08:40.405 INFO:tasks.workunit.client.0.smithi153.stdout:4/177: dwrite d11/d2d/f30 [0,4194304] 0 2024-03-21T10:08:40.407 INFO:tasks.workunit.client.0.smithi153.stdout:4/178: symlink d11/d18/l3c 0 2024-03-21T10:08:40.408 INFO:tasks.workunit.client.0.smithi153.stdout:4/179: readlink d11/l17 0 2024-03-21T10:08:40.427 INFO:tasks.workunit.client.0.smithi153.stdout:4/180: dwrite d11/d2d/f2f [0,4194304] 0 2024-03-21T10:08:40.432 INFO:tasks.workunit.client.0.smithi153.stdout:4/181: dread d11/f19 [0,4194304] 0 2024-03-21T10:08:40.435 INFO:tasks.workunit.client.0.smithi153.stdout:4/182: write d11/d18/f1f [4482047,34413] 0 2024-03-21T10:08:40.447 INFO:tasks.workunit.client.0.smithi153.stdout:4/183: dread d11/d2d/f3a [0,4194304] 0 2024-03-21T10:08:40.459 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:40 smithi153 ceph-mon[29117]: pgmap v6: 65 pgs: 65 active+clean; 220 MiB data, 878 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:08:40.459 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:40 smithi153 ceph-mon[29117]: from='client.24443 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:08:40.459 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:40 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:08:40.459 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:40 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:08:40.462 INFO:tasks.workunit.client.0.smithi153.stdout:4/184: dwrite d11/d18/f28 [0,4194304] 0 2024-03-21T10:08:40.465 INFO:tasks.workunit.client.0.smithi153.stdout:4/185: mkdir d11/d2d/d3d 0 2024-03-21T10:08:40.477 INFO:tasks.workunit.client.0.smithi153.stdout:4/186: write d11/f26 [1403022,67022] 0 2024-03-21T10:08:40.479 INFO:tasks.workunit.client.0.smithi153.stdout:4/187: mknod d11/d18/c3e 0 2024-03-21T10:08:40.480 INFO:tasks.workunit.client.0.smithi153.stdout:4/188: stat d11/d2d 0 2024-03-21T10:08:40.489 INFO:tasks.workunit.client.0.smithi153.stdout:4/189: truncate d11/f16 1009555 0 2024-03-21T10:08:40.490 INFO:tasks.workunit.client.0.smithi153.stdout:4/190: read d11/f1e [334966,103294] 0 2024-03-21T10:08:40.492 INFO:tasks.workunit.client.0.smithi153.stdout:4/191: symlink d11/d2d/l3f 0 2024-03-21T10:08:40.499 INFO:tasks.workunit.client.0.smithi153.stdout:4/192: fsync d11/f16 0 2024-03-21T10:08:40.501 INFO:tasks.workunit.client.0.smithi153.stdout:4/193: symlink d11/d2d/l40 0 2024-03-21T10:08:40.502 INFO:tasks.workunit.client.0.smithi153.stdout:4/194: chown d11/d18/f35 1387917549 1 2024-03-21T10:08:40.504 INFO:tasks.workunit.client.0.smithi153.stdout:4/195: creat d11/d18/f41 x:0 0 0 2024-03-21T10:08:40.517 INFO:tasks.workunit.client.0.smithi153.stdout:4/196: dwrite fb [0,4194304] 0 2024-03-21T10:08:40.524 INFO:tasks.workunit.client.0.smithi153.stdout:4/197: getdents d11 0 2024-03-21T10:08:40.527 INFO:tasks.workunit.client.0.smithi153.stdout:4/198: creat d11/d2d/f42 x:0 0 0 2024-03-21T10:08:40.529 INFO:tasks.workunit.client.0.smithi153.stdout:4/199: creat d11/d2d/f43 x:0 0 0 2024-03-21T10:08:40.540 INFO:tasks.workunit.client.0.smithi153.stdout:4/200: truncate d11/d18/f1f 1226224 0 2024-03-21T10:08:40.552 INFO:tasks.workunit.client.0.smithi153.stdout:4/201: link d11/d18/f1f d11/f44 0 2024-03-21T10:08:40.553 INFO:tasks.workunit.client.0.smithi153.stdout:4/202: fsync f1 0 2024-03-21T10:08:40.559 INFO:tasks.workunit.client.0.smithi153.stdout:1/63: sync 2024-03-21T10:08:40.563 INFO:tasks.workunit.client.0.smithi153.stdout:4/203: unlink d11/f12 0 2024-03-21T10:08:40.566 INFO:tasks.workunit.client.0.smithi153.stdout:4/204: write d11/d2d/f36 [412967,88418] 0 2024-03-21T10:08:40.574 INFO:tasks.workunit.client.0.smithi153.stdout:4/205: dread d11/d2d/f36 [0,4194304] 0 2024-03-21T10:08:40.575 INFO:tasks.workunit.client.0.smithi153.stdout:1/64: dread f5 [0,4194304] 0 2024-03-21T10:08:40.576 INFO:tasks.workunit.client.0.smithi153.stdout:4/206: read - d11/d18/f41 zero size 2024-03-21T10:08:40.576 INFO:tasks.workunit.client.0.smithi153.stdout:1/65: read - d6/fd zero size 2024-03-21T10:08:40.578 INFO:tasks.workunit.client.0.smithi153.stdout:1/66: write d6/fc [22864,8004] 0 2024-03-21T10:08:40.581 INFO:tasks.workunit.client.0.smithi153.stdout:1/67: creat d6/f16 x:0 0 0 2024-03-21T10:08:40.582 INFO:tasks.workunit.client.0.smithi153.stdout:4/207: creat d11/d2d/d3d/f45 x:0 0 0 2024-03-21T10:08:40.591 INFO:tasks.workunit.client.0.smithi153.stdout:1/68: write d6/fa [110886,74614] 0 2024-03-21T10:08:40.596 INFO:tasks.workunit.client.0.smithi153.stdout:1/69: link d6/fe d6/f17 0 2024-03-21T10:08:40.598 INFO:tasks.workunit.client.0.smithi153.stdout:4/208: dwrite fe [0,4194304] 0 2024-03-21T10:08:40.603 INFO:tasks.workunit.client.0.smithi153.stdout:4/209: symlink d11/l46 0 2024-03-21T10:08:40.609 INFO:tasks.workunit.client.0.smithi153.stdout:4/210: unlink d11/d18/l31 0 2024-03-21T10:08:40.612 INFO:tasks.workunit.client.0.smithi153.stdout:4/211: truncate d11/d2d/f2f 5118528 0 2024-03-21T10:08:40.616 INFO:tasks.workunit.client.0.smithi153.stdout:1/70: dwrite d6/fd [0,4194304] 0 2024-03-21T10:08:40.622 INFO:tasks.workunit.client.0.smithi153.stdout:1/71: write d6/f17 [238753,31832] 0 2024-03-21T10:08:40.625 INFO:tasks.workunit.client.0.smithi153.stdout:1/72: creat d6/f18 x:0 0 0 2024-03-21T10:08:40.627 INFO:tasks.workunit.client.0.smithi153.stdout:1/73: mknod d6/d10/c19 0 2024-03-21T10:08:40.630 INFO:tasks.workunit.client.0.smithi153.stdout:1/74: symlink d6/d10/l1a 0 2024-03-21T10:08:40.630 INFO:tasks.workunit.client.0.smithi153.stdout:1/75: chown d6/d10 109 1 2024-03-21T10:08:40.631 INFO:tasks.workunit.client.0.smithi153.stdout:1/76: rename d6/d13 to d6/d13/d1b 22 2024-03-21T10:08:40.633 INFO:tasks.workunit.client.0.smithi153.stdout:1/77: truncate d6/ff 1997778 0 2024-03-21T10:08:40.641 INFO:tasks.workunit.client.0.smithi153.stdout:1/78: truncate f3 3908934 0 2024-03-21T10:08:40.642 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:40.642+0000 7f5fb7a00700 1 -- 172.21.15.153:0/3922679859 --> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f5fb00611d0 con 0x7f5f980394e0 2024-03-21T10:08:40.643 INFO:tasks.workunit.client.0.smithi153.stdout:1/79: write d6/fd [2830448,52942] 0 2024-03-21T10:08:40.648 INFO:tasks.workunit.client.0.smithi153.stdout:1/80: dread d6/fa [0,4194304] 0 2024-03-21T10:08:40.650 INFO:tasks.workunit.client.0.smithi153.stdout:1/81: creat d6/f1c x:0 0 0 2024-03-21T10:08:40.660 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:40.660+0000 7f5fad600700 1 -- 172.21.15.153:0/3922679859 <== mgr.14666 v2:172.21.15.153:6800/2524145044 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f5fb00611d0 con 0x7f5f980394e0 2024-03-21T10:08:40.661 INFO:teuthology.orchestra.run.smithi153.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-21T10:08:40.661 INFO:teuthology.orchestra.run.smithi153.stdout:alertmanager.smithi153 smithi153 *:9093,9094 running (9m) 16s ago 11m 13.8M - 0.25.0 c8568f914cd2 2cfcc225c7e6 2024-03-21T10:08:40.661 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi153 smithi153 running (11m) 16s ago 11m 7864k - 18.2.1 5be31c24972a 5b61994396a6 2024-03-21T10:08:40.661 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi174 smithi174 running (9m) 1s ago 9m 7814k - 18.2.1 5be31c24972a 4f7a5fc3ac28 2024-03-21T10:08:40.661 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi153 smithi153 running (11m) 16s ago 11m 6665k - 18.2.1 5be31c24972a 90a8b8ea03d1 2024-03-21T10:08:40.661 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi174 smithi174 running (9m) 1s ago 9m 6665k - 18.2.1 5be31c24972a a4c929e4dc77 2024-03-21T10:08:40.661 INFO:teuthology.orchestra.run.smithi153.stdout:grafana.smithi153 smithi153 *:3000 running (8m) 16s ago 10m 74.6M - 9.4.7 954c08fa6188 3af5800ecfef 2024-03-21T10:08:40.661 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.iuyluj smithi153 running (4m) 16s ago 3m 127M - 18.2.1 5be31c24972a 02f667cee979 2024-03-21T10:08:40.661 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.zipckr smithi153 running (4m) 16s ago 4m 123M - 18.2.1 5be31c24972a 68a853929f8f 2024-03-21T10:08:40.661 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.nhozag smithi174 running (3m) 1s ago 3m 119M - 18.2.1 5be31c24972a 2fee4000308f 2024-03-21T10:08:40.661 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.yyimst smithi174 running (4m) 1s ago 4m 117M - 18.2.1 5be31c24972a d78a729220d1 2024-03-21T10:08:40.661 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi153.tluxni smithi153 *:8443,9283,8765 running (28s) 16s ago 13m 63.7M - 19.0.0-2231-g2edfee68 237e43c7a65f 7ffee0e55382 2024-03-21T10:08:40.661 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi174.qpcwat smithi174 *:8443,9283,8765 running (83s) 1s ago 9m 99.0M - 19.0.0-2231-g2edfee68 237e43c7a65f 2f7934875e11 2024-03-21T10:08:40.662 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi153 smithi153 running (13m) 16s ago 13m 49.2M 2048M 18.2.1 5be31c24972a 813963a95d69 2024-03-21T10:08:40.662 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi174 smithi174 running (9m) 1s ago 9m 40.9M 2048M 18.2.1 5be31c24972a 99204923eec6 2024-03-21T10:08:40.662 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi153 smithi153 *:9100 running (11m) 16s ago 11m 7788k - 1.5.0 0da6a335fe13 babbc29a0eaf 2024-03-21T10:08:40.662 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi174 smithi174 *:9100 running (9m) 1s ago 9m 8166k - 1.5.0 0da6a335fe13 86841435042a 2024-03-21T10:08:40.662 INFO:teuthology.orchestra.run.smithi153.stdout:osd.0 smithi153 running (8m) 16s ago 8m 82.2M 1217M 18.2.1 5be31c24972a 28912a2319b4 2024-03-21T10:08:40.662 INFO:teuthology.orchestra.run.smithi153.stdout:osd.1 smithi153 running (8m) 16s ago 8m 86.3M 1217M 18.2.1 5be31c24972a ca60681e2314 2024-03-21T10:08:40.662 INFO:teuthology.orchestra.run.smithi153.stdout:osd.2 smithi153 running (7m) 16s ago 7m 74.9M 1217M 18.2.1 5be31c24972a 8db799f8962a 2024-03-21T10:08:40.662 INFO:teuthology.orchestra.run.smithi153.stdout:osd.3 smithi174 running (6m) 1s ago 6m 132M 2241M 18.2.1 5be31c24972a dafc89e0658d 2024-03-21T10:08:40.662 INFO:teuthology.orchestra.run.smithi153.stdout:osd.4 smithi174 running (6m) 1s ago 6m 118M 2241M 18.2.1 5be31c24972a c4cd15701f05 2024-03-21T10:08:40.662 INFO:teuthology.orchestra.run.smithi153.stdout:osd.5 smithi174 running (5m) 1s ago 5m 111M 2241M 18.2.1 5be31c24972a 3b69886ce22e 2024-03-21T10:08:40.662 INFO:teuthology.orchestra.run.smithi153.stdout:prometheus.smithi153 smithi153 *:9095 running (39s) 16s ago 10m 44.3M - 2.43.0 a07b618ecd1d 08adc18937d9 2024-03-21T10:08:40.663 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:40.663+0000 7f5fb7a00700 1 -- 172.21.15.153:0/3922679859 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f5f980394e0 msgr2=0x7f5f9803b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:08:40.663 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:40.663+0000 7f5fb7a00700 1 --2- 172.21.15.153:0/3922679859 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f5f980394e0 0x7f5f9803b9a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c005950 tx=0x7f5f9c0058e0 comp rx=0 tx=0).stop 2024-03-21T10:08:40.663 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:40.663+0000 7f5fb7a00700 1 -- 172.21.15.153:0/3922679859 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5fb00ff430 msgr2=0x7f5fb019c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:08:40.663 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:40.663+0000 7f5fb7a00700 1 --2- 172.21.15.153:0/3922679859 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5fb00ff430 0x7f5fb019c910 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f5fa400bd70 tx=0x7f5fa400be50 comp rx=0 tx=0).stop 2024-03-21T10:08:40.664 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:40.663+0000 7f5fb7a00700 1 -- 172.21.15.153:0/3922679859 shutdown_connections 2024-03-21T10:08:40.664 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:40.663+0000 7f5fb7a00700 1 --2- 172.21.15.153:0/3922679859 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f5f980394e0 0x7f5f9803b9a0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:08:40.664 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:40.663+0000 7f5fb7a00700 1 --2- 172.21.15.153:0/3922679859 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5fb00ff430 0x7f5fb019c910 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:08:40.664 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:40.663+0000 7f5fb7a00700 1 --2- 172.21.15.153:0/3922679859 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5fb0101da0 0x7f5fb019ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:08:40.664 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:40.663+0000 7f5fb7a00700 1 -- 172.21.15.153:0/3922679859 >> 172.21.15.153:0/3922679859 conn(0x7f5fb00fb030 msgr2=0x7f5fb00fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:08:40.664 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:40.663+0000 7f5fb7a00700 1 -- 172.21.15.153:0/3922679859 shutdown_connections 2024-03-21T10:08:40.664 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:40.664+0000 7f5fb7a00700 1 -- 172.21.15.153:0/3922679859 wait complete. 2024-03-21T10:08:40.667 INFO:tasks.workunit.client.0.smithi153.stdout:1/82: dwrite d6/f9 [0,4194304] 0 2024-03-21T10:08:40.687 INFO:tasks.workunit.client.0.smithi153.stdout:1/83: dwrite d6/f15 [0,4194304] 0 2024-03-21T10:08:40.712 INFO:tasks.workunit.client.0.smithi153.stdout:1/84: dwrite d6/fc [0,4194304] 0 2024-03-21T10:08:40.722 INFO:tasks.workunit.client.0.smithi153.stdout:1/85: dread d6/f17 [0,4194304] 0 2024-03-21T10:08:40.746 INFO:tasks.workunit.client.0.smithi153.stdout:1/86: creat d6/d13/f1d x:0 0 0 2024-03-21T10:08:40.748 INFO:tasks.workunit.client.0.smithi153.stdout:1/87: write d6/f15 [1712491,3448] 0 2024-03-21T10:08:40.748 INFO:tasks.workunit.client.0.smithi153.stdout:3/145: sync 2024-03-21T10:08:40.750 INFO:tasks.workunit.client.0.smithi153.stdout:1/88: dread - d6/f16 zero size 2024-03-21T10:08:40.802 INFO:tasks.workunit.client.0.smithi153.stdout:1/89: truncate d6/fb 3056531 0 2024-03-21T10:08:40.806 INFO:tasks.workunit.client.0.smithi153.stdout:0/58: sync 2024-03-21T10:08:40.806 INFO:tasks.workunit.client.0.smithi153.stdout:5/95: sync 2024-03-21T10:08:40.806 INFO:tasks.workunit.client.0.smithi153.stdout:1/90: write d6/f7 [1995517,36729] 0 2024-03-21T10:08:40.811 INFO:tasks.workunit.client.0.smithi153.stdout:0/59: fdatasync f2 0 2024-03-21T10:08:40.816 INFO:tasks.workunit.client.0.smithi153.stdout:5/96: creat d4/d5/d16/f1e x:0 0 0 2024-03-21T10:08:40.817 INFO:tasks.workunit.client.0.smithi153.stdout:0/60: write d4/f6 [540433,67751] 0 2024-03-21T10:08:40.817 INFO:tasks.workunit.client.0.smithi153.stdout:3/146: dwrite f6 [0,4194304] 0 2024-03-21T10:08:40.821 INFO:tasks.workunit.client.0.smithi153.stdout:0/61: write d4/f10 [1713379,119493] 0 2024-03-21T10:08:40.823 INFO:tasks.workunit.client.0.smithi153.stdout:3/147: symlink dd/l2e 0 2024-03-21T10:08:40.829 INFO:tasks.workunit.client.0.smithi153.stdout:1/91: rmdir d6 39 2024-03-21T10:08:40.834 INFO:tasks.workunit.client.0.smithi153.stdout:5/97: dread d4/fa [0,4194304] 0 2024-03-21T10:08:40.837 INFO:tasks.workunit.client.0.smithi153.stdout:5/98: mknod d4/d8/c1f 0 2024-03-21T10:08:40.837 INFO:tasks.workunit.client.0.smithi153.stdout:1/92: mknod d6/d13/c1e 0 2024-03-21T10:08:40.837 INFO:tasks.workunit.client.0.smithi153.stdout:0/62: dread f2 [0,4194304] 0 2024-03-21T10:08:40.840 INFO:tasks.workunit.client.0.smithi153.stdout:0/63: chown f2 11269519 1 2024-03-21T10:08:40.843 INFO:tasks.workunit.client.0.smithi153.stdout:0/64: creat d4/db/f13 x:0 0 0 2024-03-21T10:08:40.844 INFO:tasks.workunit.client.0.smithi153.stdout:5/99: symlink d4/d5/de/l20 0 2024-03-21T10:08:40.845 INFO:tasks.workunit.client.0.smithi153.stdout:0/65: chown d4/db/f13 869 1 2024-03-21T10:08:40.850 INFO:tasks.workunit.client.0.smithi153.stdout:5/100: dread d4/d5/f1a [0,4194304] 0 2024-03-21T10:08:40.851 INFO:tasks.workunit.client.0.smithi153.stdout:3/148: write dd/f15 [167906,87497] 0 2024-03-21T10:08:40.854 INFO:tasks.workunit.client.0.smithi153.stdout:3/149: write dd/f18 [320514,125963] 0 2024-03-21T10:08:40.855 INFO:tasks.workunit.client.0.smithi153.stdout:3/150: readlink dd/l2e 0 2024-03-21T10:08:40.856 INFO:tasks.workunit.client.0.smithi153.stdout:5/101: creat d4/d5/de/f21 x:0 0 0 2024-03-21T10:08:40.858 INFO:tasks.workunit.client.0.smithi153.stdout:5/102: stat d4/d8/c1f 0 2024-03-21T10:08:40.860 INFO:tasks.workunit.client.0.smithi153.stdout:3/151: mkdir dd/d1e/d2f 0 2024-03-21T10:08:40.863 INFO:tasks.workunit.client.0.smithi153.stdout:0/66: rename d4/c12 to d4/c14 0 2024-03-21T10:08:40.864 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:40.863+0000 7f9587e00700 1 -- 172.21.15.153:0/1135349789 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9580101db0 msgr2=0x7f9580102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:08:40.864 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:40.863+0000 7f9587e00700 1 --2- 172.21.15.153:0/1135349789 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9580101db0 0x7f9580102210 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f9574009910 tx=0x7f9574009c20 comp rx=0 tx=0).stop 2024-03-21T10:08:40.864 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:40.864+0000 7f9587e00700 1 -- 172.21.15.153:0/1135349789 shutdown_connections 2024-03-21T10:08:40.864 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:40.864+0000 7f9587e00700 1 --2- 172.21.15.153:0/1135349789 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9580101db0 0x7f9580102210 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:08:40.864 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:40.864+0000 7f9587e00700 1 --2- 172.21.15.153:0/1135349789 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9580100bb0 0x7f9580100fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:08:40.864 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:40.864+0000 7f9587e00700 1 -- 172.21.15.153:0/1135349789 >> 172.21.15.153:0/1135349789 conn(0x7f95800fc110 msgr2=0x7f95800fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:08:40.865 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:40.864+0000 7f9587e00700 1 -- 172.21.15.153:0/1135349789 shutdown_connections 2024-03-21T10:08:40.865 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:40.864+0000 7f9587e00700 1 -- 172.21.15.153:0/1135349789 wait complete. 2024-03-21T10:08:40.867 INFO:tasks.workunit.client.0.smithi153.stdout:3/152: fdatasync dd/f13 0 2024-03-21T10:08:40.868 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:40.866+0000 7f9587e00700 1 Processor -- start 2024-03-21T10:08:40.868 INFO:tasks.workunit.client.0.smithi153.stdout:1/93: truncate d6/fd 804934 0 2024-03-21T10:08:40.868 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:40.866+0000 7f9587e00700 1 -- start start 2024-03-21T10:08:40.869 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:40.867+0000 7f9587e00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9580100bb0 0x7f9580194160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:08:40.869 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:40.867+0000 7f9587e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9580101db0 0x7f95801946a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:08:40.869 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:40.867+0000 7f9587e00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9580194cc0 con 0x7f9580101db0 2024-03-21T10:08:40.869 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:40.867+0000 7f9587e00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9580194e00 con 0x7f9580100bb0 2024-03-21T10:08:40.869 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:40.867+0000 7f9585400700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9580100bb0 0x7f9580194160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:08:40.869 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:40.867+0000 7f9585400700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9580100bb0 0x7f9580194160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.153:57364/0 (socket says 172.21.15.153:57364) 2024-03-21T10:08:40.869 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:40.867+0000 7f9585400700 1 -- 172.21.15.153:0/1262239475 learned_addr learned my addr 172.21.15.153:0/1262239475 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:08:40.869 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:40.868+0000 7f9585400700 1 -- 172.21.15.153:0/1262239475 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9580101db0 msgr2=0x7f95801946a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:08:40.869 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:40.868+0000 7f9585400700 1 --2- 172.21.15.153:0/1262239475 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9580101db0 0x7f95801946a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:08:40.869 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:40.868+0000 7f9585400700 1 -- 172.21.15.153:0/1262239475 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95740095f0 con 0x7f9580100bb0 2024-03-21T10:08:40.870 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:40.869+0000 7f9585400700 1 --2- 172.21.15.153:0/1262239475 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9580100bb0 0x7f9580194160 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9578007e90 tx=0x7f957800d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:08:40.870 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:40.869+0000 7f957e000700 1 -- 172.21.15.153:0/1262239475 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9578017070 con 0x7f9580100bb0 2024-03-21T10:08:40.870 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:40.869+0000 7f957e000700 1 -- 172.21.15.153:0/1262239475 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f957800f040 con 0x7f9580100bb0 2024-03-21T10:08:40.870 INFO:tasks.workunit.client.0.smithi153.stdout:3/153: chown dd/d1e/f2a 6813 1 2024-03-21T10:08:40.870 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:40.869+0000 7f9587e00700 1 -- 172.21.15.153:0/1262239475 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95801998b0 con 0x7f9580100bb0 2024-03-21T10:08:40.870 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:40.869+0000 7f957e000700 1 -- 172.21.15.153:0/1262239475 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f95780048b0 con 0x7f9580100bb0 2024-03-21T10:08:40.871 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:40.870+0000 7f9587e00700 1 -- 172.21.15.153:0/1262239475 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9580199e00 con 0x7f9580100bb0 2024-03-21T10:08:40.871 INFO:tasks.workunit.client.0.smithi153.stdout:0/67: mkdir d4/d15 0 2024-03-21T10:08:40.872 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:40.871+0000 7f957e000700 1 -- 172.21.15.153:0/1262239475 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 28) v1 ==== 46435+0+0 (secure 0 0 0) 0x7f957801c410 con 0x7f9580100bb0 2024-03-21T10:08:40.872 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:40.871+0000 7f9587e00700 1 -- 172.21.15.153:0/1262239475 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f958019b810 con 0x7f9580100bb0 2024-03-21T10:08:40.873 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:40.872+0000 7f957e000700 1 --2- 172.21.15.153:0/1262239475 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f9568039490 0x7f956803b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:08:40.873 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:40.873+0000 7f957e000700 1 -- 172.21.15.153:0/1262239475 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f957804d260 con 0x7f9580100bb0 2024-03-21T10:08:40.874 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:40.874+0000 7f9584a00700 1 --2- 172.21.15.153:0/1262239475 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f9568039490 0x7f956803b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:08:40.875 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:40.875+0000 7f9584a00700 1 --2- 172.21.15.153:0/1262239475 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f9568039490 0x7f956803b950 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f9574005090 tx=0x7f9574005fb0 comp rx=0 tx=0).ready entity=mgr.14666 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:08:40.875 INFO:tasks.workunit.client.0.smithi153.stdout:1/94: creat d6/d13/f1f x:0 0 0 2024-03-21T10:08:40.878 INFO:tasks.workunit.client.0.smithi153.stdout:1/95: chown d6/d13/f1d 0 1 2024-03-21T10:08:40.883 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:40.882+0000 7f957e000700 1 -- 172.21.15.153:0/1262239475 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185454 (secure 0 0 0) 0x7f95780133f0 con 0x7f9580100bb0 2024-03-21T10:08:40.883 INFO:tasks.workunit.client.0.smithi153.stdout:3/154: creat dd/d1e/f30 x:0 0 0 2024-03-21T10:08:40.886 INFO:tasks.workunit.client.0.smithi153.stdout:1/96: mknod d6/c20 0 2024-03-21T10:08:40.889 INFO:tasks.workunit.client.0.smithi153.stdout:5/103: dwrite d4/d5/de/f21 [0,4194304] 0 2024-03-21T10:08:40.905 INFO:tasks.workunit.client.0.smithi153.stdout:0/68: dwrite d4/f6 [0,4194304] 0 2024-03-21T10:08:40.911 INFO:tasks.workunit.client.0.smithi153.stdout:6/338: sync 2024-03-21T10:08:40.911 INFO:tasks.workunit.client.0.smithi153.stdout:8/82: sync 2024-03-21T10:08:40.912 INFO:tasks.workunit.client.0.smithi153.stdout:4/212: sync 2024-03-21T10:08:40.912 INFO:tasks.workunit.client.0.smithi153.stdout:7/90: sync 2024-03-21T10:08:40.912 INFO:tasks.workunit.client.0.smithi153.stdout:2/127: sync 2024-03-21T10:08:40.912 INFO:tasks.workunit.client.0.smithi153.stdout:9/105: sync 2024-03-21T10:08:40.914 INFO:tasks.workunit.client.0.smithi153.stdout:3/155: dread f2 [0,4194304] 0 2024-03-21T10:08:40.915 INFO:tasks.workunit.client.0.smithi153.stdout:0/69: read - d4/db/f11 zero size 2024-03-21T10:08:40.917 INFO:tasks.workunit.client.0.smithi153.stdout:6/339: write d1f/f42 [933303,105396] 0 2024-03-21T10:08:40.917 INFO:tasks.workunit.client.0.smithi153.stdout:4/213: stat d11/f24 0 2024-03-21T10:08:40.919 INFO:tasks.workunit.client.0.smithi153.stdout:7/91: stat d4/d1b/l1c 0 2024-03-21T10:08:40.921 INFO:tasks.workunit.client.0.smithi153.stdout:6/340: chown d1f 472923874 1 2024-03-21T10:08:40.921 INFO:tasks.workunit.client.0.smithi153.stdout:9/106: unlink d1/d4/f7 0 2024-03-21T10:08:40.921 INFO:tasks.workunit.client.0.smithi153.stdout:1/97: dread d6/f7 [0,4194304] 0 2024-03-21T10:08:40.922 INFO:tasks.workunit.client.0.smithi153.stdout:0/70: truncate d4/db/f11 796749 0 2024-03-21T10:08:40.923 INFO:tasks.workunit.client.0.smithi153.stdout:3/156: mknod dd/c31 0 2024-03-21T10:08:40.932 INFO:tasks.workunit.client.0.smithi153.stdout:1/98: mknod d6/d13/c21 0 2024-03-21T10:08:40.932 INFO:tasks.workunit.client.0.smithi153.stdout:4/214: readlink d11/l29 0 2024-03-21T10:08:40.933 INFO:tasks.workunit.client.0.smithi153.stdout:2/128: creat dd/d15/d18/f2a x:0 0 0 2024-03-21T10:08:40.934 INFO:tasks.workunit.client.0.smithi153.stdout:7/92: dread d4/f10 [0,4194304] 0 2024-03-21T10:08:40.939 INFO:tasks.workunit.client.0.smithi153.stdout:3/157: mknod dd/d1e/d2f/c32 0 2024-03-21T10:08:40.940 INFO:tasks.workunit.client.0.smithi153.stdout:8/83: write f4 [280979,40234] 0 2024-03-21T10:08:40.941 INFO:tasks.workunit.client.0.smithi153.stdout:7/93: fsync d4/f1a 0 2024-03-21T10:08:40.942 INFO:tasks.workunit.client.0.smithi153.stdout:0/71: mkdir d4/d16 0 2024-03-21T10:08:40.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:40 smithi174 ceph-mon[37496]: pgmap v6: 65 pgs: 65 active+clean; 220 MiB data, 878 MiB used, 536 GiB / 536 GiB avail 2024-03-21T10:08:40.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:40 smithi174 ceph-mon[37496]: from='client.24443 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:08:40.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:40 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:08:40.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:40 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:08:40.951 INFO:tasks.workunit.client.0.smithi153.stdout:6/341: symlink d1f/d2f/d59/d70/l7e 0 2024-03-21T10:08:40.952 INFO:tasks.workunit.client.0.smithi153.stdout:1/99: unlink d6/c20 0 2024-03-21T10:08:40.955 INFO:tasks.workunit.client.0.smithi153.stdout:5/104: dwrite d4/d8/f1d [0,4194304] 0 2024-03-21T10:08:40.955 INFO:tasks.workunit.client.0.smithi153.stdout:1/100: readlink d6/d10/l1a 0 2024-03-21T10:08:40.964 INFO:tasks.workunit.client.0.smithi153.stdout:0/72: symlink d4/d15/l17 0 2024-03-21T10:08:40.965 INFO:tasks.workunit.client.0.smithi153.stdout:3/158: write dd/f20 [589068,111776] 0 2024-03-21T10:08:40.972 INFO:tasks.workunit.client.0.smithi153.stdout:9/107: dwrite d1/d4/f14 [0,4194304] 0 2024-03-21T10:08:40.973 INFO:tasks.workunit.client.0.smithi153.stdout:7/94: mknod d4/d1b/c1e 0 2024-03-21T10:08:40.982 INFO:tasks.workunit.client.0.smithi153.stdout:0/73: symlink d4/l18 0 2024-03-21T10:08:40.986 INFO:tasks.workunit.client.0.smithi153.stdout:7/95: fsync d4/fa 0 2024-03-21T10:08:40.986 INFO:tasks.workunit.client.0.smithi153.stdout:5/105: dread d4/d5/f7 [0,4194304] 0 2024-03-21T10:08:40.994 INFO:tasks.workunit.client.0.smithi153.stdout:0/74: readlink d4/db/ld 0 2024-03-21T10:08:40.995 INFO:tasks.workunit.client.0.smithi153.stdout:9/108: symlink d1/d12/d1f/l29 0 2024-03-21T10:08:41.000 INFO:tasks.workunit.client.0.smithi153.stdout:2/129: dwrite dd/d15/d18/f24 [0,4194304] 0 2024-03-21T10:08:41.002 INFO:tasks.workunit.client.0.smithi153.stdout:6/342: dread f5 [0,4194304] 0 2024-03-21T10:08:41.011 INFO:tasks.workunit.client.0.smithi153.stdout:8/84: dwrite d5/d9/f7 [0,4194304] 0 2024-03-21T10:08:41.013 INFO:tasks.workunit.client.0.smithi153.stdout:6/343: write d1f/f23 [3526402,44587] 0 2024-03-21T10:08:41.016 INFO:tasks.workunit.client.0.smithi153.stdout:1/101: dwrite d6/f17 [0,4194304] 0 2024-03-21T10:08:41.019 INFO:tasks.workunit.client.0.smithi153.stdout:9/109: write d1/d4/ff [1604801,13032] 0 2024-03-21T10:08:41.021 INFO:tasks.workunit.client.0.smithi153.stdout:8/85: chown f4 508 1 2024-03-21T10:08:41.023 INFO:tasks.workunit.client.0.smithi153.stdout:3/159: dwrite f8 [4194304,4194304] 0 2024-03-21T10:08:41.028 INFO:tasks.workunit.client.0.smithi153.stdout:9/110: readlink d1/l11 0 2024-03-21T10:08:41.030 INFO:tasks.workunit.client.0.smithi153.stdout:8/86: unlink d5/c13 0 2024-03-21T10:08:41.032 INFO:tasks.workunit.client.0.smithi153.stdout:2/130: getdents dd 0 2024-03-21T10:08:41.034 INFO:tasks.workunit.client.0.smithi153.stdout:1/102: symlink d6/d13/l22 0 2024-03-21T10:08:41.039 INFO:tasks.workunit.client.0.smithi153.stdout:8/87: creat d5/d9/f19 x:0 0 0 2024-03-21T10:08:41.039 INFO:tasks.workunit.client.0.smithi153.stdout:4/215: dwrite d11/f44 [0,4194304] 0 2024-03-21T10:08:41.039 INFO:tasks.workunit.client.0.smithi153.stdout:6/344: mknod d1f/d2b/d33/d40/d7b/c7f 0 2024-03-21T10:08:41.045 INFO:tasks.workunit.client.0.smithi153.stdout:9/111: write d1/d12/f26 [986485,45159] 0 2024-03-21T10:08:41.048 INFO:tasks.workunit.client.0.smithi153.stdout:2/131: rename dd/c22 to dd/d15/c2b 0 2024-03-21T10:08:41.053 INFO:tasks.workunit.client.0.smithi153.stdout:4/216: unlink d11/d18/l3b 0 2024-03-21T10:08:41.054 INFO:tasks.workunit.client.0.smithi153.stdout:3/160: creat dd/d1e/d2f/f33 x:0 0 0 2024-03-21T10:08:41.059 INFO:tasks.workunit.client.0.smithi153.stdout:6/345: creat d1f/d2b/d33/d40/d5f/f80 x:0 0 0 2024-03-21T10:08:41.059 INFO:tasks.workunit.client.0.smithi153.stdout:2/132: fsync dd/d15/d18/f24 0 2024-03-21T10:08:41.060 INFO:tasks.workunit.client.0.smithi153.stdout:0/75: dwrite d4/f10 [0,4194304] 0 2024-03-21T10:08:41.061 INFO:tasks.workunit.client.0.smithi153.stdout:9/112: chown d1/d12/f19 765 1 2024-03-21T10:08:41.062 INFO:tasks.workunit.client.0.smithi153.stdout:8/88: write d5/f10 [1255091,97546] 0 2024-03-21T10:08:41.064 INFO:tasks.workunit.client.0.smithi153.stdout:4/217: chown d11/d18/c3e 14 1 2024-03-21T10:08:41.071 INFO:tasks.workunit.client.0.smithi153.stdout:9/113: write d1/f16 [3931513,19560] 0 2024-03-21T10:08:41.077 INFO:tasks.workunit.client.0.smithi153.stdout:6/346: read d1f/f23 [1626033,86153] 0 2024-03-21T10:08:41.078 INFO:tasks.workunit.client.0.smithi153.stdout:5/106: dwrite d4/fa [0,4194304] 0 2024-03-21T10:08:41.083 INFO:tasks.workunit.client.0.smithi153.stdout:9/114: chown d1/d12 1634493349 1 2024-03-21T10:08:41.083 INFO:tasks.workunit.client.0.smithi153.stdout:2/133: mkdir dd/d15/d18/d2c 0 2024-03-21T10:08:41.084 INFO:tasks.workunit.client.0.smithi153.stdout:1/103: dwrite d6/fe [4194304,4194304] 0 2024-03-21T10:08:41.084 INFO:tasks.workunit.client.0.smithi153.stdout:6/347: readlink d1f/d2b/l48 0 2024-03-21T10:08:41.084 INFO:tasks.workunit.client.0.smithi153.stdout:4/218: symlink d11/d18/l47 0 2024-03-21T10:08:41.088 INFO:tasks.workunit.client.0.smithi153.stdout:9/115: chown d1/d4/f20 0 1 2024-03-21T10:08:41.089 INFO:tasks.workunit.client.0.smithi153.stdout:1/104: stat d6/d13/c21 0 2024-03-21T10:08:41.090 INFO:tasks.workunit.client.0.smithi153.stdout:6/348: truncate d1f/f27 1278530 0 2024-03-21T10:08:41.091 INFO:tasks.workunit.client.0.smithi153.stdout:5/107: write d4/d5/f1a [631699,5360] 0 2024-03-21T10:08:41.093 INFO:tasks.workunit.client.0.smithi153.stdout:4/219: symlink d11/l48 0 2024-03-21T10:08:41.093 INFO:tasks.workunit.client.0.smithi153.stdout:0/76: getdents d4/db 0 2024-03-21T10:08:41.102 INFO:tasks.workunit.client.0.smithi153.stdout:5/108: write d4/d8/f11 [597017,46208] 0 2024-03-21T10:08:41.105 INFO:tasks.workunit.client.0.smithi153.stdout:4/220: read d11/f19 [432332,29364] 0 2024-03-21T10:08:41.106 INFO:tasks.workunit.client.0.smithi153.stdout:0/77: write d4/db/f11 [1244459,121012] 0 2024-03-21T10:08:41.108 INFO:tasks.workunit.client.0.smithi153.stdout:2/134: truncate dd/f14 244668 0 2024-03-21T10:08:41.108 INFO:tasks.workunit.client.0.smithi153.stdout:1/105: truncate d6/d13/f1d 613015 0 2024-03-21T10:08:41.109 INFO:tasks.workunit.client.0.smithi153.stdout:3/161: dwrite dd/f1f [0,4194304] 0 2024-03-21T10:08:41.112 INFO:tasks.workunit.client.0.smithi153.stdout:5/109: creat d4/d5/de/f22 x:0 0 0 2024-03-21T10:08:41.115 INFO:tasks.workunit.client.0.smithi153.stdout:8/89: dread d5/d9/f7 [0,4194304] 0 2024-03-21T10:08:41.115 INFO:tasks.workunit.client.0.smithi153.stdout:4/221: truncate d11/f44 4541096 0 2024-03-21T10:08:41.118 INFO:tasks.workunit.client.0.smithi153.stdout:6/349: link d1f/d2b/f46 d1f/d2f/d59/f81 0 2024-03-21T10:08:41.119 INFO:tasks.workunit.client.0.smithi153.stdout:5/110: dread - d4/d5/d16/f1e zero size 2024-03-21T10:08:41.123 INFO:tasks.workunit.client.0.smithi153.stdout:6/350: dread - d1f/d2b/f53 zero size 2024-03-21T10:08:41.124 INFO:tasks.workunit.client.0.smithi153.stdout:5/111: fsync d4/d5/de/f21 0 2024-03-21T10:08:41.124 INFO:tasks.workunit.client.0.smithi153.stdout:8/90: chown d5/d9/f11 3881 1 2024-03-21T10:08:41.127 INFO:tasks.workunit.client.0.smithi153.stdout:4/222: fsync d11/d18/f28 0 2024-03-21T10:08:41.127 INFO:tasks.workunit.client.0.smithi153.stdout:9/116: dwrite d1/d4/fc [4194304,4194304] 0 2024-03-21T10:08:41.128 INFO:tasks.workunit.client.0.smithi153.stdout:1/106: mknod d6/d13/c23 0 2024-03-21T10:08:41.128 INFO:tasks.workunit.client.0.smithi153.stdout:8/91: truncate d5/f8 491521 0 2024-03-21T10:08:41.131 INFO:tasks.workunit.client.0.smithi153.stdout:2/135: creat dd/d15/d18/d2c/f2d x:0 0 0 2024-03-21T10:08:41.132 INFO:tasks.workunit.client.0.smithi153.stdout:0/78: mkdir d4/d16/d19 0 2024-03-21T10:08:41.133 INFO:tasks.workunit.client.0.smithi153.stdout:8/92: write d5/d9/dc/f16 [529367,94559] 0 2024-03-21T10:08:41.135 INFO:tasks.workunit.client.0.smithi153.stdout:9/117: write d1/d12/f26 [566234,42826] 0 2024-03-21T10:08:41.141 INFO:tasks.workunit.client.0.smithi153.stdout:2/136: dread dd/f13 [0,4194304] 0 2024-03-21T10:08:41.144 INFO:tasks.workunit.client.0.smithi153.stdout:4/223: creat d11/d18/f49 x:0 0 0 2024-03-21T10:08:41.156 INFO:tasks.workunit.client.0.smithi153.stdout:3/162: dwrite dd/f20 [0,4194304] 0 2024-03-21T10:08:41.159 INFO:tasks.workunit.client.0.smithi153.stdout:0/79: dwrite d4/db/f11 [0,4194304] 0 2024-03-21T10:08:41.163 INFO:tasks.workunit.client.0.smithi153.stdout:5/112: truncate d4/d5/de/f21 1116444 0 2024-03-21T10:08:41.164 INFO:tasks.workunit.client.0.smithi153.stdout:3/163: truncate dd/f12 1785478 0 2024-03-21T10:08:41.169 INFO:tasks.workunit.client.0.smithi153.stdout:6/351: write d1f/d2b/d33/d40/d4f/d66/f75 [106786,48191] 0 2024-03-21T10:08:41.170 INFO:tasks.workunit.client.0.smithi153.stdout:4/224: creat d11/d2d/f4a x:0 0 0 2024-03-21T10:08:41.173 INFO:tasks.workunit.client.0.smithi153.stdout:2/137: rename dd/f10 to dd/f2e 0 2024-03-21T10:08:41.179 INFO:tasks.workunit.client.0.smithi153.stdout:5/113: mknod d4/c23 0 2024-03-21T10:08:41.180 INFO:tasks.workunit.client.0.smithi153.stdout:5/114: fdatasync d4/fa 0 2024-03-21T10:08:41.184 INFO:tasks.workunit.client.0.smithi153.stdout:3/164: mknod dd/d1e/d2f/c34 0 2024-03-21T10:08:41.185 INFO:tasks.workunit.client.0.smithi153.stdout:8/93: mknod d5/d9/dc/dd/c1a 0 2024-03-21T10:08:41.188 INFO:tasks.workunit.client.0.smithi153.stdout:1/107: truncate d6/f7 91503 0 2024-03-21T10:08:41.191 INFO:tasks.workunit.client.0.smithi153.stdout:8/94: truncate d5/f14 154937 0 2024-03-21T10:08:41.192 INFO:tasks.workunit.client.0.smithi153.stdout:0/80: dwrite d4/f10 [4194304,4194304] 0 2024-03-21T10:08:41.192 INFO:tasks.workunit.client.0.smithi153.stdout:2/138: creat dd/d15/d18/d2c/f2f x:0 0 0 2024-03-21T10:08:41.192 INFO:tasks.workunit.client.0.smithi153.stdout:6/352: creat d1f/d2b/d33/d40/d5f/f82 x:0 0 0 2024-03-21T10:08:41.195 INFO:tasks.workunit.client.0.smithi153.stdout:3/165: mkdir dd/d1e/d2f/d35 0 2024-03-21T10:08:41.195 INFO:tasks.workunit.client.0.smithi153.stdout:5/115: mkdir d4/d5/d16/d24 0 2024-03-21T10:08:41.197 INFO:tasks.workunit.client.0.smithi153.stdout:0/81: readlink d4/l18 0 2024-03-21T10:08:41.198 INFO:tasks.workunit.client.0.smithi153.stdout:1/108: symlink d6/d13/l24 0 2024-03-21T10:08:41.199 INFO:tasks.workunit.client.0.smithi153.stdout:2/139: fdatasync dd/d15/d18/f2a 0 2024-03-21T10:08:41.200 INFO:tasks.workunit.client.0.smithi153.stdout:4/225: dwrite fe [4194304,4194304] 0 2024-03-21T10:08:41.201 INFO:tasks.workunit.client.0.smithi153.stdout:8/95: truncate d5/d9/f19 295824 0 2024-03-21T10:08:41.202 INFO:tasks.workunit.client.0.smithi153.stdout:9/118: truncate d1/d4/fc 341263 0 2024-03-21T10:08:41.203 INFO:tasks.workunit.client.0.smithi153.stdout:0/82: chown d4/l18 4121 1 2024-03-21T10:08:41.205 INFO:tasks.workunit.client.0.smithi153.stdout:0/83: truncate f2 4833716 0 2024-03-21T10:08:41.205 INFO:tasks.workunit.client.0.smithi153.stdout:2/140: dread - dd/d15/d18/d2c/f2d zero size 2024-03-21T10:08:41.208 INFO:tasks.workunit.client.0.smithi153.stdout:8/96: stat d5/d9/dc 0 2024-03-21T10:08:41.210 INFO:tasks.workunit.client.0.smithi153.stdout:6/353: creat d1f/d2b/d33/d40/d4f/f83 x:0 0 0 2024-03-21T10:08:41.211 INFO:tasks.workunit.client.0.smithi153.stdout:0/84: dread - d4/db/f13 zero size 2024-03-21T10:08:41.213 INFO:tasks.workunit.client.0.smithi153.stdout:5/116: rename d4/d5/f12 to d4/d5/d16/d24/f25 0 2024-03-21T10:08:41.213 INFO:tasks.workunit.client.0.smithi153.stdout:2/141: write dd/d15/d18/f29 [324845,99452] 0 2024-03-21T10:08:41.213 INFO:tasks.workunit.client.0.smithi153.stdout:3/166: creat dd/d1e/d2f/f36 x:0 0 0 2024-03-21T10:08:41.214 INFO:tasks.workunit.client.0.smithi153.stdout:6/354: stat d1f/d2b/d33/d40/d65/c6b 0 2024-03-21T10:08:41.214 INFO:tasks.workunit.client.0.smithi153.stdout:9/119: rmdir d1 39 2024-03-21T10:08:41.222 INFO:tasks.workunit.client.1.smithi174.stderr:++ readlink -f fsstress 2024-03-21T10:08:41.224 INFO:tasks.workunit.client.0.smithi153.stdout:0/85: symlink d4/d15/l1a 0 2024-03-21T10:08:41.227 INFO:tasks.workunit.client.1.smithi174.stderr:+ BIN=/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-03-21T10:08:41.228 INFO:tasks.workunit.client.1.smithi174.stderr:+ popd 2024-03-21T10:08:41.229 INFO:tasks.workunit.client.0.smithi153.stdout:9/120: dread - d1/d4/f20 zero size 2024-03-21T10:08:41.230 INFO:tasks.workunit.client.0.smithi153.stdout:3/167: creat dd/d1e/d2f/f37 x:0 0 0 2024-03-21T10:08:41.230 INFO:tasks.workunit.client.0.smithi153.stdout:6/355: creat d1f/d2b/d33/d40/d4f/d66/f84 x:0 0 0 2024-03-21T10:08:41.230 INFO:tasks.workunit.client.1.smithi174.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-03-21T10:08:41.231 INFO:tasks.workunit.client.1.smithi174.stderr:+ popd 2024-03-21T10:08:41.232 INFO:tasks.workunit.client.0.smithi153.stdout:2/142: truncate dd/d15/d18/d2c/f2d 872822 0 2024-03-21T10:08:41.232 INFO:tasks.workunit.client.1.smithi174.stdout:~/cephtest/mnt.1/client.1/tmp 2024-03-21T10:08:41.233 INFO:tasks.workunit.client.1.smithi174.stderr:++ mktemp -d -p . 2024-03-21T10:08:41.234 INFO:tasks.workunit.client.0.smithi153.stdout:0/86: creat d4/db/f1b x:0 0 0 2024-03-21T10:08:41.235 INFO:tasks.workunit.client.0.smithi153.stdout:4/226: dwrite d11/d2d/f2f [4194304,4194304] 0 2024-03-21T10:08:41.235 INFO:tasks.workunit.client.0.smithi153.stdout:1/109: rmdir d6/d13 39 2024-03-21T10:08:41.239 INFO:tasks.workunit.client.0.smithi153.stdout:0/87: read d4/f10 [736949,103308] 0 2024-03-21T10:08:41.239 INFO:tasks.workunit.client.0.smithi153.stdout:4/227: truncate f7 1584329 0 2024-03-21T10:08:41.242 INFO:tasks.workunit.client.1.smithi174.stderr:+ T=./tmp.pIui9Xc5jq 2024-03-21T10:08:41.242 INFO:tasks.workunit.client.1.smithi174.stderr:+ /home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.pIui9Xc5jq -l 1 -n 1000 -p 10 -v 2024-03-21T10:08:41.243 INFO:tasks.workunit.client.0.smithi153.stdout:5/117: rmdir d4/d5/d16 39 2024-03-21T10:08:41.243 INFO:tasks.workunit.client.0.smithi153.stdout:6/356: creat d1f/d2b/f85 x:0 0 0 2024-03-21T10:08:41.245 INFO:tasks.workunit.client.0.smithi153.stdout:8/97: dwrite f3 [4194304,4194304] 0 2024-03-21T10:08:41.247 INFO:tasks.workunit.client.0.smithi153.stdout:5/118: stat d4/d5/de/f22 0 2024-03-21T10:08:41.248 INFO:tasks.workunit.client.0.smithi153.stdout:8/98: readlink d5/d9/le 0 2024-03-21T10:08:41.252 INFO:tasks.workunit.client.1.smithi174.stdout:seed = 1711267357 2024-03-21T10:08:41.254 INFO:tasks.workunit.client.0.smithi153.stdout:0/88: mknod d4/db/c1c 0 2024-03-21T10:08:41.256 INFO:tasks.workunit.client.0.smithi153.stdout:7/96: sync 2024-03-21T10:08:41.258 INFO:tasks.workunit.client.0.smithi153.stdout:6/357: dread d1f/d2b/d33/d40/d4f/d66/f75 [0,4194304] 0 2024-03-21T10:08:41.261 INFO:tasks.workunit.client.0.smithi153.stdout:3/168: mknod dd/d1e/d2f/d35/c38 0 2024-03-21T10:08:41.261 INFO:tasks.workunit.client.0.smithi153.stdout:0/89: stat d4/d15/l1a 0 2024-03-21T10:08:41.265 INFO:tasks.workunit.client.0.smithi153.stdout:6/358: readlink d1f/d2b/d33/d40/l79 0 2024-03-21T10:08:41.267 INFO:tasks.workunit.client.1.smithi174.stdout:1/0: readlink - no filename 2024-03-21T10:08:41.267 INFO:tasks.workunit.client.1.smithi174.stdout:1/1: readlink - no filename 2024-03-21T10:08:41.267 INFO:tasks.workunit.client.1.smithi174.stdout:1/2: truncate - no filename 2024-03-21T10:08:41.267 INFO:tasks.workunit.client.1.smithi174.stdout:1/3: chown . 1055008108 1 2024-03-21T10:08:41.267 INFO:tasks.workunit.client.1.smithi174.stdout:1/4: write - no filename 2024-03-21T10:08:41.269 INFO:tasks.workunit.client.1.smithi174.stdout:0/0: dread - no filename 2024-03-21T10:08:41.269 INFO:tasks.workunit.client.1.smithi174.stdout:0/1: rename - no filename 2024-03-21T10:08:41.270 INFO:tasks.workunit.client.0.smithi153.stdout:9/121: dwrite d1/d4/ff [0,4194304] 0 2024-03-21T10:08:41.271 INFO:tasks.workunit.client.1.smithi174.stdout:1/5: creat f0 x:0 0 0 2024-03-21T10:08:41.273 INFO:tasks.workunit.client.0.smithi153.stdout:4/228: link d11/d2d/f3a d11/d18/f4b 0 2024-03-21T10:08:41.273 INFO:tasks.workunit.client.0.smithi153.stdout:6/359: truncate d1f/d2f/f3d 4343298 0 2024-03-21T10:08:41.274 INFO:tasks.workunit.client.1.smithi174.stdout:2/0: chown . 54 1 2024-03-21T10:08:41.274 INFO:tasks.workunit.client.1.smithi174.stdout:2/1: read - no filename 2024-03-21T10:08:41.274 INFO:tasks.workunit.client.1.smithi174.stdout:2/2: fsync - no filename 2024-03-21T10:08:41.276 INFO:tasks.workunit.client.1.smithi174.stdout:0/2: creat f0 x:0 0 0 2024-03-21T10:08:41.278 INFO:tasks.workunit.client.1.smithi174.stdout:2/3: mknod c0 0 2024-03-21T10:08:41.279 INFO:tasks.workunit.client.0.smithi153.stdout:3/169: mknod dd/c39 0 2024-03-21T10:08:41.282 INFO:tasks.workunit.client.0.smithi153.stdout:5/119: write d4/d5/d16/f1c [707080,13483] 0 2024-03-21T10:08:41.283 INFO:tasks.workunit.client.1.smithi174.stdout:4/0: read - no filename 2024-03-21T10:08:41.283 INFO:tasks.workunit.client.1.smithi174.stdout:4/1: fdatasync - no filename 2024-03-21T10:08:41.283 INFO:tasks.workunit.client.1.smithi174.stdout:3/0: mkdir d0 0 2024-03-21T10:08:41.284 INFO:tasks.workunit.client.0.smithi153.stdout:2/143: write f5 [1107042,36918] 0 2024-03-21T10:08:41.284 INFO:tasks.workunit.client.1.smithi174.stdout:4/2: chown . 1623 1 2024-03-21T10:08:41.287 INFO:tasks.workunit.client.1.smithi174.stdout:5/0: rmdir - no directory 2024-03-21T10:08:41.287 INFO:tasks.workunit.client.1.smithi174.stdout:5/1: dwrite - no filename 2024-03-21T10:08:41.288 INFO:tasks.workunit.client.0.smithi153.stdout:1/110: dwrite f5 [0,4194304] 0 2024-03-21T10:08:41.288 INFO:tasks.workunit.client.0.smithi153.stdout:6/360: write d1f/f4b [3018563,85197] 0 2024-03-21T10:08:41.290 INFO:tasks.workunit.client.0.smithi153.stdout:0/90: write d4/f10 [1268077,33691] 0 2024-03-21T10:08:41.292 INFO:tasks.workunit.client.1.smithi174.stdout:2/4: link c0 c1 0 2024-03-21T10:08:41.292 INFO:tasks.workunit.client.1.smithi174.stdout:2/5: write - no filename 2024-03-21T10:08:41.292 INFO:tasks.workunit.client.1.smithi174.stdout:2/6: fdatasync - no filename 2024-03-21T10:08:41.292 INFO:tasks.workunit.client.0.smithi153.stdout:8/99: dwrite f3 [0,4194304] 0 2024-03-21T10:08:41.293 INFO:tasks.workunit.client.0.smithi153.stdout:3/170: creat dd/d1e/d2f/f3a x:0 0 0 2024-03-21T10:08:41.296 INFO:tasks.workunit.client.1.smithi174.stdout:6/0: rmdir - no directory 2024-03-21T10:08:41.296 INFO:tasks.workunit.client.1.smithi174.stdout:6/1: rmdir - no directory 2024-03-21T10:08:41.296 INFO:tasks.workunit.client.1.smithi174.stdout:6/2: unlink - no file 2024-03-21T10:08:41.296 INFO:tasks.workunit.client.1.smithi174.stdout:6/3: dwrite - no filename 2024-03-21T10:08:41.297 INFO:tasks.workunit.client.1.smithi174.stdout:3/1: symlink d0/l1 0 2024-03-21T10:08:41.298 INFO:tasks.workunit.client.0.smithi153.stdout:4/229: write d11/f1e [696622,40889] 0 2024-03-21T10:08:41.302 INFO:tasks.workunit.client.1.smithi174.stdout:4/3: creat f0 x:0 0 0 2024-03-21T10:08:41.302 INFO:tasks.workunit.client.1.smithi174.stdout:5/2: creat f0 x:0 0 0 2024-03-21T10:08:41.303 INFO:tasks.workunit.client.1.smithi174.stdout:3/2: chown d0/l1 729540 1 2024-03-21T10:08:41.303 INFO:tasks.workunit.client.1.smithi174.stdout:3/3: dread - no filename 2024-03-21T10:08:41.303 INFO:tasks.workunit.client.1.smithi174.stdout:3/4: dwrite - no filename 2024-03-21T10:08:41.303 INFO:tasks.workunit.client.1.smithi174.stdout:2/7: creat f2 x:0 0 0 2024-03-21T10:08:41.304 INFO:tasks.workunit.client.0.smithi153.stdout:0/91: rmdir d4/d16 39 2024-03-21T10:08:41.305 INFO:tasks.workunit.client.0.smithi153.stdout:7/97: dwrite d4/f1a [0,4194304] 0 2024-03-21T10:08:41.310 INFO:tasks.workunit.client.0.smithi153.stdout:3/171: creat dd/d1e/d2f/f3b x:0 0 0 2024-03-21T10:08:41.312 INFO:tasks.workunit.client.0.smithi153.stdout:5/120: link d4/d5/de/f22 d4/d8/f26 0 2024-03-21T10:08:41.316 INFO:tasks.workunit.client.1.smithi174.stdout:1/6: dwrite f0 [0,4194304] 0 2024-03-21T10:08:41.317 INFO:tasks.workunit.client.0.smithi153.stdout:8/100: rename d5/da to d5/d1b 0 2024-03-21T10:08:41.318 INFO:tasks.workunit.client.0.smithi153.stdout:0/92: truncate d4/db/f13 573981 0 2024-03-21T10:08:41.323 INFO:tasks.workunit.client.1.smithi174.stdout:6/4: mkdir d0 0 2024-03-21T10:08:41.323 INFO:tasks.workunit.client.1.smithi174.stdout:3/5: chown d0/l1 1896041 1 2024-03-21T10:08:41.323 INFO:tasks.workunit.client.1.smithi174.stdout:3/6: write - no filename 2024-03-21T10:08:41.324 INFO:tasks.workunit.client.1.smithi174.stdout:3/7: chown d0 43 1 2024-03-21T10:08:41.324 INFO:tasks.workunit.client.0.smithi153.stdout:3/172: mknod dd/d1e/c3c 0 2024-03-21T10:08:41.329 INFO:tasks.workunit.client.1.smithi174.stdout:5/3: mkdir d1 0 2024-03-21T10:08:41.330 INFO:tasks.workunit.client.0.smithi153.stdout:0/93: mknod d4/c1d 0 2024-03-21T10:08:41.331 INFO:tasks.workunit.client.1.smithi174.stdout:4/4: creat f1 x:0 0 0 2024-03-21T10:08:41.335 INFO:tasks.workunit.client.0.smithi153.stdout:2/144: dwrite dd/d15/d18/f2a [0,4194304] 0 2024-03-21T10:08:41.336 INFO:tasks.workunit.client.1.smithi174.stdout:1/7: rename f0 to f1 0 2024-03-21T10:08:41.336 INFO:tasks.workunit.client.0.smithi153.stdout:6/361: dwrite d1f/d2b/d33/d40/d4f/d66/f7c [0,4194304] 0 2024-03-21T10:08:41.339 INFO:tasks.workunit.client.1.smithi174.stdout:9/0: chown . 502 1 2024-03-21T10:08:41.340 INFO:tasks.workunit.client.1.smithi174.stdout:9/1: chown . 57 1 2024-03-21T10:08:41.341 INFO:tasks.workunit.client.0.smithi153.stdout:3/173: chown dd/c21 1166279 1 2024-03-21T10:08:41.341 INFO:tasks.workunit.client.0.smithi153.stdout:2/145: fdatasync dd/d15/d18/f29 0 2024-03-21T10:08:41.344 INFO:tasks.workunit.client.0.smithi153.stdout:1/111: dwrite d6/f1c [0,4194304] 0 2024-03-21T10:08:41.345 INFO:tasks.workunit.client.0.smithi153.stdout:6/362: truncate d1f/d2b/f38 534651 0 2024-03-21T10:08:41.345 INFO:tasks.workunit.client.1.smithi174.stdout:4/5: creat f2 x:0 0 0 2024-03-21T10:08:41.346 INFO:tasks.workunit.client.0.smithi153.stdout:0/94: mknod d4/db/c1e 0 2024-03-21T10:08:41.347 INFO:tasks.workunit.client.0.smithi153.stdout:9/122: truncate d1/d4/f14 3767846 0 2024-03-21T10:08:41.348 INFO:tasks.workunit.client.0.smithi153.stdout:0/95: chown d4/db 12712 1 2024-03-21T10:08:41.353 INFO:tasks.workunit.client.1.smithi174.stdout:6/5: symlink d0/l1 0 2024-03-21T10:08:41.355 INFO:tasks.workunit.client.0.smithi153.stdout:3/174: mkdir dd/d1e/d3d 0 2024-03-21T10:08:41.355 INFO:tasks.workunit.client.0.smithi153.stdout:0/96: fsync d4/db/f1b 0 2024-03-21T10:08:41.356 INFO:tasks.workunit.client.0.smithi153.stdout:6/363: symlink d1f/d2b/d33/d40/d4f/l86 0 2024-03-21T10:08:41.356 INFO:tasks.workunit.client.1.smithi174.stdout:5/4: creat d1/f2 x:0 0 0 2024-03-21T10:08:41.357 INFO:tasks.workunit.client.0.smithi153.stdout:1/112: symlink d6/d13/l25 0 2024-03-21T10:08:41.358 INFO:tasks.workunit.client.0.smithi153.stdout:5/121: dwrite f1 [0,4194304] 0 2024-03-21T10:08:41.359 INFO:tasks.workunit.client.0.smithi153.stdout:9/123: truncate d1/d4/ff 4489519 0 2024-03-21T10:08:41.359 INFO:tasks.workunit.client.1.smithi174.stdout:8/0: creat f0 x:0 0 0 2024-03-21T10:08:41.361 INFO:tasks.workunit.client.1.smithi174.stdout:5/5: read - f0 zero size 2024-03-21T10:08:41.364 INFO:tasks.workunit.client.1.smithi174.stdout:4/6: creat f3 x:0 0 0 2024-03-21T10:08:41.364 INFO:tasks.workunit.client.1.smithi174.stdout:8/1: truncate f0 355060 0 2024-03-21T10:08:41.365 INFO:tasks.workunit.client.1.smithi174.stdout:5/6: dread - d1/f2 zero size 2024-03-21T10:08:41.366 INFO:tasks.workunit.client.1.smithi174.stdout:5/7: stat f0 0 2024-03-21T10:08:41.367 INFO:tasks.workunit.client.1.smithi174.stdout:9/2: creat f0 x:0 0 0 2024-03-21T10:08:41.368 INFO:tasks.workunit.client.0.smithi153.stdout:8/101: dwrite d5/d9/f7 [0,4194304] 0 2024-03-21T10:08:41.371 INFO:tasks.workunit.client.1.smithi174.stdout:6/6: mknod d0/c2 0 2024-03-21T10:08:41.372 INFO:tasks.workunit.client.1.smithi174.stdout:8/2: write f0 [33493,82153] 0 2024-03-21T10:08:41.372 INFO:tasks.workunit.client.1.smithi174.stdout:2/8: dwrite f2 [0,4194304] 0 2024-03-21T10:08:41.382 INFO:tasks.workunit.client.1.smithi174.stdout:9/3: rename f0 to f1 0 2024-03-21T10:08:41.382 INFO:tasks.workunit.client.0.smithi153.stdout:1/113: symlink d6/d10/l26 0 2024-03-21T10:08:41.382 INFO:tasks.workunit.client.0.smithi153.stdout:5/122: mknod d4/d8/c27 0 2024-03-21T10:08:41.383 INFO:tasks.workunit.client.1.smithi174.stdout:8/3: creat f1 x:0 0 0 2024-03-21T10:08:41.384 INFO:tasks.workunit.client.1.smithi174.stdout:6/7: mkdir d0/d3 0 2024-03-21T10:08:41.385 INFO:tasks.workunit.client.0.smithi153.stdout:9/124: symlink d1/d12/d1f/l2a 0 2024-03-21T10:08:41.386 INFO:tasks.workunit.client.0.smithi153.stdout:3/175: mknod dd/d1e/d2f/c3e 0 2024-03-21T10:08:41.389 INFO:tasks.workunit.client.0.smithi153.stdout:7/98: write d4/fa [2413704,127294] 0 2024-03-21T10:08:41.389 INFO:tasks.workunit.client.0.smithi153.stdout:0/97: rename d4/d15 to d4/d16/d1f 0 2024-03-21T10:08:41.396 INFO:tasks.workunit.client.0.smithi153.stdout:7/99: fdatasync f2 0 2024-03-21T10:08:41.403 INFO:tasks.workunit.client.0.smithi153.stdout:5/123: mkdir d4/d5/de/d28 0 2024-03-21T10:08:41.404 INFO:tasks.workunit.client.1.smithi174.stdout:2/9: dread f2 [0,4194304] 0 2024-03-21T10:08:41.404 INFO:tasks.workunit.client.1.smithi174.stdout:6/8: link d0/l1 d0/l4 0 2024-03-21T10:08:41.410 INFO:tasks.workunit.client.0.smithi153.stdout:7/100: fdatasync d4/f12 0 2024-03-21T10:08:41.411 INFO:tasks.workunit.client.0.smithi153.stdout:0/98: fsync d4/db/f11 0 2024-03-21T10:08:41.413 INFO:tasks.workunit.client.0.smithi153.stdout:8/102: link f3 d5/d9/dc/dd/f1c 0 2024-03-21T10:08:41.417 INFO:tasks.workunit.client.0.smithi153.stdout:0/99: write d4/db/f13 [331257,3458] 0 2024-03-21T10:08:41.418 INFO:tasks.workunit.client.0.smithi153.stdout:6/364: dwrite d1f/d2b/f63 [0,4194304] 0 2024-03-21T10:08:41.424 INFO:tasks.workunit.client.0.smithi153.stdout:5/124: chown d4/d8/c27 132192880 1 2024-03-21T10:08:41.425 INFO:tasks.workunit.client.0.smithi153.stdout:4/230: dwrite fd [0,4194304] 0 2024-03-21T10:08:41.426 INFO:tasks.workunit.client.1.smithi174.stdout:6/9: mkdir d0/d3/d5 0 2024-03-21T10:08:41.426 INFO:tasks.workunit.client.1.smithi174.stdout:6/10: dwrite - no filename 2024-03-21T10:08:41.427 INFO:tasks.workunit.client.1.smithi174.stdout:4/7: dwrite f3 [0,4194304] 0 2024-03-21T10:08:41.427 INFO:tasks.workunit.client.0.smithi153.stdout:0/100: chown d4/c14 31649842 1 2024-03-21T10:08:41.429 INFO:tasks.workunit.client.0.smithi153.stdout:5/125: fdatasync d4/d5/d16/f19 0 2024-03-21T10:08:41.430 INFO:tasks.workunit.client.0.smithi153.stdout:7/101: mknod d4/c1f 0 2024-03-21T10:08:41.432 INFO:tasks.workunit.client.1.smithi174.stdout:4/8: write f2 [214606,103842] 0 2024-03-21T10:08:41.435 INFO:tasks.workunit.client.1.smithi174.stdout:6/11: mkdir d0/d3/d5/d6 0 2024-03-21T10:08:41.436 INFO:tasks.workunit.client.1.smithi174.stdout:4/9: write f3 [1596166,102172] 0 2024-03-21T10:08:41.437 INFO:tasks.workunit.client.0.smithi153.stdout:3/176: getdents dd/d1e 0 2024-03-21T10:08:41.442 INFO:tasks.workunit.client.0.smithi153.stdout:3/177: fdatasync dd/f18 0 2024-03-21T10:08:41.442 INFO:tasks.workunit.client.0.smithi153.stdout:5/126: mknod d4/d8/c29 0 2024-03-21T10:08:41.442 INFO:tasks.workunit.client.1.smithi174.stdout:2/10: dread f2 [0,4194304] 0 2024-03-21T10:08:41.443 INFO:tasks.workunit.client.1.smithi174.stdout:4/10: truncate f0 191651 0 2024-03-21T10:08:41.443 INFO:tasks.workunit.client.1.smithi174.stdout:6/12: symlink d0/d3/l7 0 2024-03-21T10:08:41.447 INFO:tasks.workunit.client.0.smithi153.stdout:2/146: dwrite dd/d15/d18/f2a [4194304,4194304] 0 2024-03-21T10:08:41.449 INFO:tasks.workunit.client.1.smithi174.stdout:4/11: write f2 [191885,15597] 0 2024-03-21T10:08:41.449 INFO:tasks.workunit.client.0.smithi153.stdout:0/101: rmdir d4/db 39 2024-03-21T10:08:41.450 INFO:tasks.workunit.client.1.smithi174.stdout:8/4: dread f0 [0,4194304] 0 2024-03-21T10:08:41.450 INFO:tasks.workunit.client.1.smithi174.stdout:4/12: dread - f1 zero size 2024-03-21T10:08:41.452 INFO:tasks.workunit.client.0.smithi153.stdout:7/102: mkdir d4/d1b/d20 0 2024-03-21T10:08:41.452 INFO:tasks.workunit.client.0.smithi153.stdout:3/178: truncate dd/f22 457274 0 2024-03-21T10:08:41.452 INFO:tasks.workunit.client.0.smithi153.stdout:8/103: rename d5/f10 to d5/d1b/f1d 0 2024-03-21T10:08:41.452 INFO:tasks.workunit.client.1.smithi174.stdout:8/5: chown f0 0 1 2024-03-21T10:08:41.453 INFO:tasks.workunit.client.0.smithi153.stdout:2/147: chown dd/d15/d18/d2c 58 1 2024-03-21T10:08:41.456 INFO:tasks.workunit.client.1.smithi174.stdout:6/13: creat d0/d3/f8 x:0 0 0 2024-03-21T10:08:41.456 INFO:tasks.workunit.client.0.smithi153.stdout:5/127: creat d4/d5/de/f2a x:0 0 0 2024-03-21T10:08:41.460 INFO:tasks.workunit.client.0.smithi153.stdout:1/114: dwrite d6/d13/f1d [0,4194304] 0 2024-03-21T10:08:41.462 INFO:tasks.workunit.client.1.smithi174.stdout:4/13: mkdir d4 0 2024-03-21T10:08:41.465 INFO:tasks.workunit.client.1.smithi174.stdout:8/6: creat f2 x:0 0 0 2024-03-21T10:08:41.469 INFO:tasks.workunit.client.1.smithi174.stdout:4/14: write f1 [723270,90889] 0 2024-03-21T10:08:41.471 INFO:tasks.workunit.client.0.smithi153.stdout:3/179: dread - dd/d1e/d2f/f33 zero size 2024-03-21T10:08:41.473 INFO:tasks.workunit.client.1.smithi174.stdout:4/15: truncate f2 833057 0 2024-03-21T10:08:41.475 INFO:tasks.workunit.client.0.smithi153.stdout:3/180: dread - dd/f25 zero size 2024-03-21T10:08:41.476 INFO:tasks.workunit.client.1.smithi174.stdout:2/11: dread f2 [0,4194304] 0 2024-03-21T10:08:41.476 INFO:tasks.workunit.client.0.smithi153.stdout:8/104: mknod d5/d1b/c1e 0 2024-03-21T10:08:41.483 INFO:tasks.workunit.client.0.smithi153.stdout:2/148: rmdir dd 39 2024-03-21T10:08:41.484 INFO:tasks.workunit.client.0.smithi153.stdout:9/125: dread d1/d4/f1a [0,4194304] 0 2024-03-21T10:08:41.484 INFO:tasks.workunit.client.0.smithi153.stdout:1/115: mknod d6/d10/c27 0 2024-03-21T10:08:41.487 INFO:tasks.workunit.client.1.smithi174.stdout:2/12: write f2 [2863500,115540] 0 2024-03-21T10:08:41.488 INFO:tasks.workunit.client.0.smithi153.stdout:7/103: symlink d4/l21 0 2024-03-21T10:08:41.488 INFO:tasks.workunit.client.0.smithi153.stdout:6/365: dwrite d1f/f42 [0,4194304] 0 2024-03-21T10:08:41.494 INFO:tasks.workunit.client.0.smithi153.stdout:1/116: chown d6/f17 11 1 2024-03-21T10:08:41.495 INFO:tasks.workunit.client.1.smithi174.stdout:4/16: creat d4/f5 x:0 0 0 2024-03-21T10:08:41.496 INFO:tasks.workunit.client.1.smithi174.stdout:4/17: chown d4/f5 604 1 2024-03-21T10:08:41.501 INFO:tasks.workunit.client.0.smithi153.stdout:6/366: fsync d1f/f4b 0 2024-03-21T10:08:41.502 INFO:tasks.workunit.client.0.smithi153.stdout:9/126: unlink d1/d4/ff 0 2024-03-21T10:08:41.504 INFO:tasks.workunit.client.0.smithi153.stdout:3/181: getdents dd/d1e/d3d 0 2024-03-21T10:08:41.504 INFO:tasks.workunit.client.0.smithi153.stdout:6/367: read - d1f/f4a zero size 2024-03-21T10:08:41.505 INFO:tasks.workunit.client.0.smithi153.stdout:9/127: readlink d1/d12/d1f/l29 0 2024-03-21T10:08:41.507 INFO:tasks.workunit.client.1.smithi174.stdout:4/18: unlink f1 0 2024-03-21T10:08:41.507 INFO:tasks.workunit.client.0.smithi153.stdout:5/128: link c3 d4/d5/d16/d24/c2b 0 2024-03-21T10:08:41.507 INFO:tasks.workunit.client.0.smithi153.stdout:1/117: mknod d6/d13/c28 0 2024-03-21T10:08:41.511 INFO:tasks.workunit.client.1.smithi174.stdout:2/13: chown c0 2 1 2024-03-21T10:08:41.511 INFO:tasks.workunit.client.0.smithi153.stdout:2/149: rename c2 to dd/d15/c30 0 2024-03-21T10:08:41.512 INFO:tasks.workunit.client.0.smithi153.stdout:6/368: symlink d1f/d2f/d59/l87 0 2024-03-21T10:08:41.513 INFO:tasks.workunit.client.0.smithi153.stdout:3/182: chown dd/d1e/l28 210 1 2024-03-21T10:08:41.520 INFO:tasks.workunit.client.0.smithi153.stdout:2/150: fsync dd/d15/d18/f24 0 2024-03-21T10:08:41.521 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:41.519+0000 7f9587e00700 1 -- 172.21.15.153:0/1262239475 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f958002cfe0 con 0x7f9580100bb0 2024-03-21T10:08:41.521 INFO:tasks.workunit.client.0.smithi153.stdout:1/118: dread d6/fd [0,4194304] 0 2024-03-21T10:08:41.522 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:41.522+0000 7f957e000700 1 -- 172.21.15.153:0/1262239475 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7f957801f100 con 0x7f9580100bb0 2024-03-21T10:08:41.523 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:08:41.523 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": { 2024-03-21T10:08:41.523 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-03-21T10:08:41.523 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:08:41.523 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": { 2024-03-21T10:08:41.523 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 1 2024-03-21T10:08:41.523 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:08:41.523 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": { 2024-03-21T10:08:41.523 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-03-21T10:08:41.523 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:08:41.523 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": { 2024-03-21T10:08:41.524 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-03-21T10:08:41.524 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:08:41.524 INFO:teuthology.orchestra.run.smithi153.stdout: "overall": { 2024-03-21T10:08:41.524 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 12, 2024-03-21T10:08:41.524 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 1 2024-03-21T10:08:41.524 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-03-21T10:08:41.524 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:08:41.527 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:41.526+0000 7f9587e00700 1 -- 172.21.15.153:0/1262239475 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f9568039490 msgr2=0x7f956803b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:08:41.527 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:41.526+0000 7f9587e00700 1 --2- 172.21.15.153:0/1262239475 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f9568039490 0x7f956803b950 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f9574005090 tx=0x7f9574005fb0 comp rx=0 tx=0).stop 2024-03-21T10:08:41.527 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:41.526+0000 7f9587e00700 1 -- 172.21.15.153:0/1262239475 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9580100bb0 msgr2=0x7f9580194160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:08:41.527 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:41.526+0000 7f9587e00700 1 --2- 172.21.15.153:0/1262239475 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9580100bb0 0x7f9580194160 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9578007e90 tx=0x7f957800d3b0 comp rx=0 tx=0).stop 2024-03-21T10:08:41.527 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:41.526+0000 7f9587e00700 1 -- 172.21.15.153:0/1262239475 shutdown_connections 2024-03-21T10:08:41.527 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:41.526+0000 7f9587e00700 1 --2- 172.21.15.153:0/1262239475 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f9568039490 0x7f956803b950 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:08:41.527 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:41.526+0000 7f9587e00700 1 --2- 172.21.15.153:0/1262239475 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9580100bb0 0x7f9580194160 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:08:41.527 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:41.526+0000 7f9587e00700 1 --2- 172.21.15.153:0/1262239475 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9580101db0 0x7f95801946a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:08:41.527 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:41.526+0000 7f9587e00700 1 -- 172.21.15.153:0/1262239475 >> 172.21.15.153:0/1262239475 conn(0x7f95800fc110 msgr2=0x7f9580102fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:08:41.527 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:41.526+0000 7f9587e00700 1 -- 172.21.15.153:0/1262239475 shutdown_connections 2024-03-21T10:08:41.528 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:41.526+0000 7f9587e00700 1 -- 172.21.15.153:0/1262239475 wait complete. 2024-03-21T10:08:41.528 INFO:tasks.workunit.client.0.smithi153.stdout:1/119: chown d6/d13/l22 153642 1 2024-03-21T10:08:41.531 INFO:tasks.workunit.client.0.smithi153.stdout:6/369: fdatasync d1f/d2b/d33/d40/d51/f71 0 2024-03-21T10:08:41.531 INFO:tasks.workunit.client.0.smithi153.stdout:3/183: symlink dd/d1e/d2f/d35/l3f 0 2024-03-21T10:08:41.531 INFO:tasks.workunit.client.0.smithi153.stdout:8/105: dwrite d5/d1b/f18 [0,4194304] 0 2024-03-21T10:08:41.534 INFO:tasks.workunit.client.0.smithi153.stdout:2/151: dread dd/f13 [0,4194304] 0 2024-03-21T10:08:41.536 INFO:tasks.workunit.client.0.smithi153.stdout:4/231: write d11/f16 [53156,10391] 0 2024-03-21T10:08:41.536 INFO:tasks.workunit.client.0.smithi153.stdout:3/184: chown dd/c16 89365378 1 2024-03-21T10:08:41.540 INFO:tasks.workunit.client.1.smithi174.stdout:4/19: dwrite f0 [0,4194304] 0 2024-03-21T10:08:41.542 INFO:tasks.workunit.client.0.smithi153.stdout:1/120: symlink d6/l29 0 2024-03-21T10:08:41.545 INFO:tasks.workunit.client.0.smithi153.stdout:7/104: write d4/fd [455037,55340] 0 2024-03-21T10:08:41.550 INFO:tasks.workunit.client.1.smithi174.stdout:4/20: mknod d4/c6 0 2024-03-21T10:08:41.550 INFO:tasks.workunit.client.1.smithi174.stdout:4/21: readlink - no filename 2024-03-21T10:08:41.551 INFO:tasks.workunit.client.0.smithi153.stdout:6/370: dread d1f/f27 [0,4194304] 0 2024-03-21T10:08:41.552 INFO:tasks.workunit.client.0.smithi153.stdout:4/232: rename fe to d11/d18/f4c 0 2024-03-21T10:08:41.554 INFO:tasks.workunit.client.0.smithi153.stdout:9/128: dwrite d1/d4/f20 [0,4194304] 0 2024-03-21T10:08:41.555 INFO:tasks.workunit.client.1.smithi174.stdout:4/22: creat d4/f7 x:0 0 0 2024-03-21T10:08:41.558 INFO:tasks.workunit.client.0.smithi153.stdout:9/129: chown d1/d12 223294972 1 2024-03-21T10:08:41.559 INFO:tasks.workunit.client.0.smithi153.stdout:8/106: creat d5/d1b/d15/f1f x:0 0 0 2024-03-21T10:08:41.562 INFO:tasks.workunit.client.1.smithi174.stdout:4/23: rename f3 to d4/f8 0 2024-03-21T10:08:41.565 INFO:tasks.workunit.client.0.smithi153.stdout:3/185: symlink dd/d1e/d2f/d35/l40 0 2024-03-21T10:08:41.566 INFO:tasks.workunit.client.1.smithi174.stdout:4/24: mknod d4/c9 0 2024-03-21T10:08:41.567 INFO:tasks.workunit.client.1.smithi174.stdout:4/25: truncate f2 1241706 0 2024-03-21T10:08:41.569 INFO:tasks.workunit.client.0.smithi153.stdout:6/371: unlink d1f/d2b/d33/f34 0 2024-03-21T10:08:41.572 INFO:tasks.workunit.client.0.smithi153.stdout:1/121: mknod d6/c2a 0 2024-03-21T10:08:41.573 INFO:tasks.workunit.client.1.smithi174.stdout:4/26: rename f2 to d4/fa 0 2024-03-21T10:08:41.578 INFO:tasks.workunit.client.0.smithi153.stdout:8/107: mkdir d5/d9/dc/dd/d20 0 2024-03-21T10:08:41.579 INFO:tasks.workunit.client.1.smithi174.stdout:0/3: getdents . 0 2024-03-21T10:08:41.579 INFO:tasks.workunit.client.1.smithi174.stdout:4/27: symlink d4/lb 0 2024-03-21T10:08:41.580 INFO:tasks.workunit.client.1.smithi174.stdout:0/4: dread - f0 zero size 2024-03-21T10:08:41.581 INFO:tasks.workunit.client.1.smithi174.stdout:0/5: stat f0 0 2024-03-21T10:08:41.582 INFO:tasks.workunit.client.0.smithi153.stdout:5/129: dwrite d4/d5/f7 [0,4194304] 0 2024-03-21T10:08:41.586 INFO:tasks.workunit.client.1.smithi174.stdout:0/6: write f0 [429035,113463] 0 2024-03-21T10:08:41.586 INFO:tasks.workunit.client.1.smithi174.stdout:3/8: getdents d0 0 2024-03-21T10:08:41.586 INFO:tasks.workunit.client.1.smithi174.stdout:4/28: creat d4/fc x:0 0 0 2024-03-21T10:08:41.589 INFO:tasks.workunit.client.0.smithi153.stdout:1/122: write d6/fe [3009475,77647] 0 2024-03-21T10:08:41.589 INFO:tasks.workunit.client.0.smithi153.stdout:8/108: write d5/d9/f17 [656880,72564] 0 2024-03-21T10:08:41.589 INFO:tasks.workunit.client.1.smithi174.stdout:0/7: chown f0 5092287 1 2024-03-21T10:08:41.589 INFO:tasks.workunit.client.1.smithi174.stdout:2/14: getdents . 0 2024-03-21T10:08:41.589 INFO:tasks.workunit.client.1.smithi174.stdout:1/8: fsync f1 0 2024-03-21T10:08:41.590 INFO:tasks.workunit.client.1.smithi174.stdout:1/9: rmdir - no directory 2024-03-21T10:08:41.593 INFO:tasks.workunit.client.0.smithi153.stdout:6/372: fsync d1f/d2b/d33/d40/d51/f71 0 2024-03-21T10:08:41.594 INFO:tasks.workunit.client.1.smithi174.stdout:0/8: symlink l1 0 2024-03-21T10:08:41.595 INFO:tasks.workunit.client.0.smithi153.stdout:5/130: readlink d4/d5/de/l20 0 2024-03-21T10:08:41.596 INFO:tasks.workunit.client.1.smithi174.stdout:0/9: chown l1 0 1 2024-03-21T10:08:41.600 INFO:tasks.workunit.client.1.smithi174.stdout:3/9: creat d0/f2 x:0 0 0 2024-03-21T10:08:41.600 INFO:tasks.workunit.client.1.smithi174.stdout:5/8: rmdir d1 39 2024-03-21T10:08:41.600 INFO:tasks.workunit.client.0.smithi153.stdout:2/152: write dd/f14 [418738,109637] 0 2024-03-21T10:08:41.603 INFO:tasks.workunit.client.0.smithi153.stdout:9/130: getdents d1/d12 0 2024-03-21T10:08:41.604 INFO:tasks.workunit.client.0.smithi153.stdout:2/153: chown dd/l23 3136 1 2024-03-21T10:08:41.604 INFO:tasks.workunit.client.1.smithi174.stdout:2/15: rename c1 to c3 0 2024-03-21T10:08:41.606 INFO:tasks.workunit.client.0.smithi153.stdout:4/233: dread f7 [0,4194304] 0 2024-03-21T10:08:41.607 INFO:tasks.workunit.client.1.smithi174.stdout:0/10: symlink l2 0 2024-03-21T10:08:41.609 INFO:tasks.workunit.client.0.smithi153.stdout:7/105: dwrite d4/f9 [0,4194304] 0 2024-03-21T10:08:41.612 INFO:tasks.workunit.client.0.smithi153.stdout:3/186: dread f6 [0,4194304] 0 2024-03-21T10:08:41.614 INFO:tasks.workunit.client.1.smithi174.stdout:9/4: unlink f1 0 2024-03-21T10:08:41.615 INFO:tasks.workunit.client.1.smithi174.stdout:1/10: truncate f1 528764 0 2024-03-21T10:08:41.619 INFO:tasks.workunit.client.1.smithi174.stdout:2/16: mknod c4 0 2024-03-21T10:08:41.619 INFO:tasks.workunit.client.1.smithi174.stdout:2/17: readlink - no filename 2024-03-21T10:08:41.619 INFO:tasks.workunit.client.1.smithi174.stdout:2/18: rmdir - no directory 2024-03-21T10:08:41.620 INFO:tasks.workunit.client.1.smithi174.stdout:0/11: symlink l3 0 2024-03-21T10:08:41.623 INFO:tasks.workunit.client.0.smithi153.stdout:7/106: fdatasync d4/f1a 0 2024-03-21T10:08:41.624 INFO:tasks.workunit.client.1.smithi174.stdout:6/14: getdents d0 0 2024-03-21T10:08:41.624 INFO:tasks.workunit.client.1.smithi174.stdout:4/29: dwrite d4/f7 [0,4194304] 0 2024-03-21T10:08:41.627 INFO:tasks.workunit.client.1.smithi174.stdout:0/12: write f0 [416775,31625] 0 2024-03-21T10:08:41.628 INFO:tasks.workunit.client.1.smithi174.stdout:9/5: mkdir d2 0 2024-03-21T10:08:41.630 INFO:tasks.workunit.client.0.smithi153.stdout:1/123: link d6/ff d6/d13/f2b 0 2024-03-21T10:08:41.630 INFO:tasks.workunit.client.0.smithi153.stdout:6/373: unlink d1f/d2b/d33/d40/d4f/l78 0 2024-03-21T10:08:41.632 INFO:tasks.workunit.client.1.smithi174.stdout:2/19: write f2 [456069,70804] 0 2024-03-21T10:08:41.633 INFO:tasks.workunit.client.0.smithi153.stdout:8/109: rename d5/d9/dc/dd/c1a to d5/d9/dc/dd/c21 0 2024-03-21T10:08:41.634 INFO:tasks.workunit.client.1.smithi174.stdout:3/10: link d0/f2 d0/f3 0 2024-03-21T10:08:41.635 INFO:tasks.workunit.client.0.smithi153.stdout:2/154: mknod dd/d15/c31 0 2024-03-21T10:08:41.637 INFO:tasks.workunit.client.0.smithi153.stdout:8/110: fsync d5/d9/f17 0 2024-03-21T10:08:41.638 INFO:tasks.workunit.client.1.smithi174.stdout:1/11: symlink l2 0 2024-03-21T10:08:41.639 INFO:tasks.workunit.client.0.smithi153.stdout:4/234: symlink d11/d18/l4d 0 2024-03-21T10:08:41.639 INFO:tasks.workunit.client.0.smithi153.stdout:9/131: fdatasync d1/d4/fc 0 2024-03-21T10:08:41.640 INFO:tasks.workunit.client.0.smithi153.stdout:5/131: mkdir d4/d5/de/d28/d2c 0 2024-03-21T10:08:41.640 INFO:tasks.workunit.client.0.smithi153.stdout:4/235: readlink d11/d18/l47 0 2024-03-21T10:08:41.642 INFO:tasks.workunit.client.0.smithi153.stdout:8/111: write f4 [454932,77708] 0 2024-03-21T10:08:41.647 INFO:tasks.workunit.client.0.smithi153.stdout:4/236: chown d11/d2d/f3a 12 1 2024-03-21T10:08:41.650 INFO:tasks.workunit.client.1.smithi174.stdout:0/13: creat f4 x:0 0 0 2024-03-21T10:08:41.650 INFO:tasks.workunit.client.1.smithi174.stdout:2/20: mkdir d5 0 2024-03-21T10:08:41.650 INFO:tasks.workunit.client.0.smithi153.stdout:5/132: truncate d4/d5/de/f2a 185930 0 2024-03-21T10:08:41.652 INFO:tasks.workunit.client.0.smithi153.stdout:8/112: fsync f3 0 2024-03-21T10:08:41.653 INFO:tasks.workunit.client.1.smithi174.stdout:3/11: symlink d0/l4 0 2024-03-21T10:08:41.654 INFO:tasks.workunit.client.0.smithi153.stdout:4/237: symlink d11/d18/l4e 0 2024-03-21T10:08:41.654 INFO:tasks.workunit.client.0.smithi153.stdout:5/133: readlink d4/d5/de/l20 0 2024-03-21T10:08:41.654 INFO:tasks.workunit.client.1.smithi174.stdout:5/9: dwrite f0 [0,4194304] 0 2024-03-21T10:08:41.655 INFO:tasks.workunit.client.0.smithi153.stdout:4/238: dread - d11/d2d/f42 zero size 2024-03-21T10:08:41.656 INFO:tasks.workunit.client.1.smithi174.stdout:3/12: dread - d0/f3 zero size 2024-03-21T10:08:41.656 INFO:tasks.workunit.client.0.smithi153.stdout:8/113: fdatasync f3 0 2024-03-21T10:08:41.657 INFO:tasks.workunit.client.0.smithi153.stdout:9/132: link d1/d12/d1f/l29 d1/d12/d1f/l2b 0 2024-03-21T10:08:41.659 INFO:tasks.workunit.client.1.smithi174.stdout:0/14: truncate f0 1303646 0 2024-03-21T10:08:41.659 INFO:tasks.workunit.client.0.smithi153.stdout:5/134: creat d4/d8/f2d x:0 0 0 2024-03-21T10:08:41.660 INFO:tasks.workunit.client.0.smithi153.stdout:3/187: dread dd/f27 [0,4194304] 0 2024-03-21T10:08:41.660 INFO:tasks.workunit.client.0.smithi153.stdout:9/133: write d1/f16 [5235226,77456] 0 2024-03-21T10:08:41.660 INFO:tasks.workunit.client.1.smithi174.stdout:1/12: fsync f1 0 2024-03-21T10:08:41.661 INFO:tasks.workunit.client.1.smithi174.stdout:9/6: creat d2/f3 x:0 0 0 2024-03-21T10:08:41.663 INFO:tasks.workunit.client.0.smithi153.stdout:3/188: chown dd/f13 9289 1 2024-03-21T10:08:41.666 INFO:tasks.workunit.client.0.smithi153.stdout:3/189: dread - dd/f2d zero size 2024-03-21T10:08:41.667 INFO:tasks.workunit.client.1.smithi174.stdout:3/13: chown d0/f3 13 1 2024-03-21T10:08:41.671 INFO:tasks.workunit.client.1.smithi174.stdout:1/13: dread f1 [0,4194304] 0 2024-03-21T10:08:41.673 INFO:tasks.workunit.client.0.smithi153.stdout:3/190: mkdir dd/d41 0 2024-03-21T10:08:41.674 INFO:tasks.workunit.client.1.smithi174.stdout:9/7: truncate d2/f3 305173 0 2024-03-21T10:08:41.675 INFO:tasks.workunit.client.0.smithi153.stdout:5/135: getdents d4/d5/df 0 2024-03-21T10:08:41.675 INFO:tasks.workunit.client.1.smithi174.stdout:1/14: chown l2 16642 1 2024-03-21T10:08:41.676 INFO:tasks.workunit.client.1.smithi174.stdout:9/8: chown d2/f3 1673171 1 2024-03-21T10:08:41.680 INFO:tasks.workunit.client.1.smithi174.stdout:2/21: rename c3 to d5/c6 0 2024-03-21T10:08:41.681 INFO:tasks.workunit.client.0.smithi153.stdout:4/239: dread d11/f24 [0,4194304] 0 2024-03-21T10:08:41.682 INFO:tasks.workunit.client.0.smithi153.stdout:5/136: read d4/d5/df/f1b [272281,50668] 0 2024-03-21T10:08:41.682 INFO:tasks.workunit.client.0.smithi153.stdout:9/134: link d1/d12/d1f/l2a d1/d12/d1f/l2c 0 2024-03-21T10:08:41.682 INFO:tasks.workunit.client.0.smithi153.stdout:4/240: chown f7 1592 1 2024-03-21T10:08:41.684 INFO:tasks.workunit.client.1.smithi174.stdout:4/30: dwrite d4/f8 [0,4194304] 0 2024-03-21T10:08:41.684 INFO:tasks.workunit.client.0.smithi153.stdout:8/114: dread d5/d1b/f18 [0,4194304] 0 2024-03-21T10:08:41.685 INFO:tasks.workunit.client.1.smithi174.stdout:9/9: write d2/f3 [1108710,25002] 0 2024-03-21T10:08:41.685 INFO:tasks.workunit.client.1.smithi174.stdout:3/14: mknod d0/c5 0 2024-03-21T10:08:41.685 INFO:tasks.workunit.client.1.smithi174.stdout:2/22: read f2 [777483,96373] 0 2024-03-21T10:08:41.686 INFO:tasks.workunit.client.1.smithi174.stdout:5/10: rename d1/f2 to d1/f3 0 2024-03-21T10:08:41.688 INFO:tasks.workunit.client.0.smithi153.stdout:1/124: rmdir d6 39 2024-03-21T10:08:41.688 INFO:tasks.workunit.client.0.smithi153.stdout:5/137: mknod d4/c2e 0 2024-03-21T10:08:41.689 INFO:tasks.workunit.client.1.smithi174.stdout:9/10: chown d2 2375890 1 2024-03-21T10:08:41.689 INFO:tasks.workunit.client.1.smithi174.stdout:0/15: dread f0 [0,4194304] 0 2024-03-21T10:08:41.689 INFO:tasks.workunit.client.1.smithi174.stdout:1/15: symlink l3 0 2024-03-21T10:08:41.691 INFO:tasks.workunit.client.0.smithi153.stdout:2/155: write dd/f13 [1105189,104133] 0 2024-03-21T10:08:41.691 INFO:tasks.workunit.client.1.smithi174.stdout:5/11: read - d1/f3 zero size 2024-03-21T10:08:41.692 INFO:tasks.workunit.client.1.smithi174.stdout:0/16: chown l3 6157392 1 2024-03-21T10:08:41.693 INFO:tasks.workunit.client.0.smithi153.stdout:7/107: dread d4/fa [0,4194304] 0 2024-03-21T10:08:41.694 INFO:tasks.workunit.client.1.smithi174.stdout:4/31: creat d4/fd x:0 0 0 2024-03-21T10:08:41.695 INFO:tasks.workunit.client.1.smithi174.stdout:4/32: fdatasync f0 0 2024-03-21T10:08:41.695 INFO:tasks.workunit.client.0.smithi153.stdout:8/115: unlink d5/d9/dc/dd/c21 0 2024-03-21T10:08:41.695 INFO:tasks.workunit.client.0.smithi153.stdout:9/135: creat d1/d12/d1f/f2d x:0 0 0 2024-03-21T10:08:41.696 INFO:tasks.workunit.client.1.smithi174.stdout:4/33: fsync d4/f7 0 2024-03-21T10:08:41.698 INFO:tasks.workunit.client.0.smithi153.stdout:7/108: fsync d4/fa 0 2024-03-21T10:08:41.699 INFO:tasks.workunit.client.0.smithi153.stdout:1/125: chown d6/f18 1123455820 1 2024-03-21T10:08:41.701 INFO:tasks.workunit.client.0.smithi153.stdout:2/156: dread dd/d15/d18/d2c/f2d [0,4194304] 0 2024-03-21T10:08:41.701 INFO:tasks.workunit.client.0.smithi153.stdout:1/126: chown d6/l29 49328 1 2024-03-21T10:08:41.704 INFO:tasks.workunit.client.0.smithi153.stdout:4/241: rename d11/l14 to d11/l4f 0 2024-03-21T10:08:41.705 INFO:tasks.workunit.client.0.smithi153.stdout:4/242: readlink d11/d18/l3c 0 2024-03-21T10:08:41.706 INFO:tasks.workunit.client.0.smithi153.stdout:4/243: fdatasync d11/d2d/f30 0 2024-03-21T10:08:41.707 INFO:tasks.workunit.client.0.smithi153.stdout:2/157: chown dd/d15/c30 4137 1 2024-03-21T10:08:41.709 INFO:tasks.workunit.client.0.smithi153.stdout:1/127: mkdir d6/d10/d2c 0 2024-03-21T10:08:41.709 INFO:tasks.workunit.client.0.smithi153.stdout:4/244: chown d11/d18/c3e 10151374 1 2024-03-21T10:08:41.711 INFO:tasks.workunit.client.1.smithi174.stdout:2/23: mknod d5/c7 0 2024-03-21T10:08:41.712 INFO:tasks.workunit.client.0.smithi153.stdout:1/128: write d6/d13/f1d [4457497,36228] 0 2024-03-21T10:08:41.715 INFO:tasks.workunit.client.0.smithi153.stdout:2/158: rename dd/le to dd/d15/d18/d2c/l32 0 2024-03-21T10:08:41.716 INFO:tasks.workunit.client.0.smithi153.stdout:4/245: mkdir d11/d18/d50 0 2024-03-21T10:08:41.716 INFO:tasks.workunit.client.0.smithi153.stdout:7/109: link c3 d4/c22 0 2024-03-21T10:08:41.721 INFO:tasks.workunit.client.0.smithi153.stdout:4/246: mknod d11/c51 0 2024-03-21T10:08:41.721 INFO:tasks.workunit.client.0.smithi153.stdout:7/110: mknod d4/d14/d16/c23 0 2024-03-21T10:08:41.728 INFO:tasks.workunit.client.1.smithi174.stdout:5/12: dwrite f0 [0,4194304] 0 2024-03-21T10:08:41.734 INFO:tasks.workunit.client.1.smithi174.stdout:2/24: mkdir d5/d8 0 2024-03-21T10:08:41.735 INFO:tasks.workunit.client.1.smithi174.stdout:3/15: link d0/c5 d0/c6 0 2024-03-21T10:08:41.736 INFO:tasks.workunit.client.1.smithi174.stdout:5/13: dread - d1/f3 zero size 2024-03-21T10:08:41.736 INFO:tasks.workunit.client.1.smithi174.stdout:4/34: rmdir d4 39 2024-03-21T10:08:41.738 INFO:tasks.workunit.client.0.smithi153.stdout:1/129: dwrite d6/f18 [0,4194304] 0 2024-03-21T10:08:41.739 INFO:tasks.workunit.client.1.smithi174.stdout:2/25: chown d5/d8 2500714 1 2024-03-21T10:08:41.740 INFO:tasks.workunit.client.0.smithi153.stdout:2/159: dwrite dd/d15/d18/f29 [0,4194304] 0 2024-03-21T10:08:41.741 INFO:tasks.workunit.client.0.smithi153.stdout:1/130: stat d6/d13/l22 0 2024-03-21T10:08:41.741 INFO:tasks.workunit.client.1.smithi174.stdout:5/14: stat d1 0 2024-03-21T10:08:41.742 INFO:tasks.workunit.client.1.smithi174.stdout:9/11: dwrite d2/f3 [0,4194304] 0 2024-03-21T10:08:41.744 INFO:tasks.workunit.client.1.smithi174.stdout:3/16: dread - d0/f2 zero size 2024-03-21T10:08:41.747 INFO:tasks.workunit.client.1.smithi174.stdout:5/15: rename d1 to d1/d4 22 2024-03-21T10:08:41.751 INFO:tasks.workunit.client.0.smithi153.stdout:8/116: truncate d5/d9/f12 2246241 0 2024-03-21T10:08:41.753 INFO:tasks.workunit.client.0.smithi153.stdout:1/131: creat d6/d13/f2d x:0 0 0 2024-03-21T10:08:41.753 INFO:tasks.workunit.client.0.smithi153.stdout:4/247: getdents d11 0 2024-03-21T10:08:41.754 INFO:tasks.workunit.client.1.smithi174.stdout:5/16: chown d1/f3 3 1 2024-03-21T10:08:41.754 INFO:tasks.workunit.client.1.smithi174.stdout:2/26: symlink d5/d8/l9 0 2024-03-21T10:08:41.759 INFO:tasks.workunit.client.0.smithi153.stdout:8/117: fsync d5/d9/f7 0 2024-03-21T10:08:41.766 INFO:tasks.workunit.client.0.smithi153.stdout:2/160: mkdir dd/d15/d18/d2c/d33 0 2024-03-21T10:08:41.766 INFO:tasks.workunit.client.0.smithi153.stdout:4/248: symlink d11/d2d/l52 0 2024-03-21T10:08:41.767 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:41.766+0000 7f17f6600700 1 -- 172.21.15.153:0/2077892506 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f17f0100bb0 msgr2=0x7f17f0100fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:08:41.767 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:41.766+0000 7f17f6600700 1 --2- 172.21.15.153:0/2077892506 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f17f0100bb0 0x7f17f0100fd0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f17e0009910 tx=0x7f17e0009c20 comp rx=0 tx=0).stop 2024-03-21T10:08:41.767 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:41.766+0000 7f17f6600700 1 -- 172.21.15.153:0/2077892506 shutdown_connections 2024-03-21T10:08:41.767 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:41.766+0000 7f17f6600700 1 --2- 172.21.15.153:0/2077892506 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f17f0101db0 0x7f17f0102210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:08:41.767 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:41.766+0000 7f17f6600700 1 --2- 172.21.15.153:0/2077892506 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f17f0100bb0 0x7f17f0100fd0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:08:41.767 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:41.766+0000 7f17f6600700 1 -- 172.21.15.153:0/2077892506 >> 172.21.15.153:0/2077892506 conn(0x7f17f00fc110 msgr2=0x7f17f00fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:08:41.767 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:41.767+0000 7f17f6600700 1 -- 172.21.15.153:0/2077892506 shutdown_connections 2024-03-21T10:08:41.768 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:41.767+0000 7f17f6600700 1 -- 172.21.15.153:0/2077892506 wait complete. 2024-03-21T10:08:41.769 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:41 smithi153 ceph-mon[29117]: from='client.14700 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:08:41.772 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:41.769+0000 7f17f6600700 1 Processor -- start 2024-03-21T10:08:41.772 INFO:tasks.workunit.client.0.smithi153.stdout:2/161: unlink dd/d15/f1d 0 2024-03-21T10:08:41.772 INFO:tasks.workunit.client.0.smithi153.stdout:4/249: stat d11/d18/f49 0 2024-03-21T10:08:41.774 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:41.773+0000 7f17f6600700 1 -- start start 2024-03-21T10:08:41.774 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:41.774+0000 7f17f6600700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f17f0100bb0 0x7f17f0074b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:08:41.774 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:41.774+0000 7f17f6600700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f17f0101db0 0x7f17f0073180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:08:41.774 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:41.774+0000 7f17f6600700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17f0199fb0 con 0x7f17f0101db0 2024-03-21T10:08:41.774 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:41.774+0000 7f17f6600700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17f019a0f0 con 0x7f17f0100bb0 2024-03-21T10:08:41.775 INFO:tasks.workunit.client.1.smithi174.stdout:5/17: rmdir d1 39 2024-03-21T10:08:41.776 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:41.775+0000 7f17eea00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f17f0101db0 0x7f17f0073180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:08:41.778 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:41.775+0000 7f17eea00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f17f0101db0 0x7f17f0073180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:58950/0 (socket says 172.21.15.153:58950) 2024-03-21T10:08:41.778 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:41.776+0000 7f17eea00700 1 -- 172.21.15.153:0/2611252182 learned_addr learned my addr 172.21.15.153:0/2611252182 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:08:41.778 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:41.776+0000 7f17ef400700 1 --2- 172.21.15.153:0/2611252182 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f17f0100bb0 0x7f17f0074b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:08:41.781 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:41.778+0000 7f17eea00700 1 -- 172.21.15.153:0/2611252182 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f17f0100bb0 msgr2=0x7f17f0074b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:08:41.781 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:41.779+0000 7f17eea00700 1 --2- 172.21.15.153:0/2611252182 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f17f0100bb0 0x7f17f0074b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:08:41.781 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:41.779+0000 7f17eea00700 1 -- 172.21.15.153:0/2611252182 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17e00095f0 con 0x7f17f0101db0 2024-03-21T10:08:41.781 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:41.779+0000 7f17ef400700 1 --2- 172.21.15.153:0/2611252182 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f17f0100bb0 0x7f17f0074b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:08:41.781 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:41.780+0000 7f17eea00700 1 --2- 172.21.15.153:0/2611252182 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f17f0101db0 0x7f17f0073180 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f17d800e920 tx=0x7f17d800ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:08:41.782 INFO:tasks.workunit.client.1.smithi174.stdout:2/27: creat d5/fa x:0 0 0 2024-03-21T10:08:41.783 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:41.783+0000 7f17e7e00700 1 -- 172.21.15.153:0/2611252182 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f17d800c9d0 con 0x7f17f0101db0 2024-03-21T10:08:41.783 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:41.783+0000 7f17e7e00700 1 -- 172.21.15.153:0/2611252182 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f17d8010430 con 0x7f17f0101db0 2024-03-21T10:08:41.783 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:41.783+0000 7f17e7e00700 1 -- 172.21.15.153:0/2611252182 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f17d8018650 con 0x7f17f0101db0 2024-03-21T10:08:41.784 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:41.784+0000 7f17f6600700 1 -- 172.21.15.153:0/2611252182 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17f00736c0 con 0x7f17f0101db0 2024-03-21T10:08:41.785 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:41.784+0000 7f17f6600700 1 -- 172.21.15.153:0/2611252182 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17f0073be0 con 0x7f17f0101db0 2024-03-21T10:08:41.785 INFO:tasks.workunit.client.0.smithi153.stdout:1/132: dwrite d6/f18 [0,4194304] 0 2024-03-21T10:08:41.786 INFO:tasks.workunit.client.1.smithi174.stdout:0/17: dwrite f0 [0,4194304] 0 2024-03-21T10:08:41.787 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:41.786+0000 7f17f6600700 1 -- 172.21.15.153:0/2611252182 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17f0073dc0 con 0x7f17f0101db0 2024-03-21T10:08:41.791 INFO:tasks.workunit.client.0.smithi153.stdout:2/162: symlink dd/d15/d18/l34 0 2024-03-21T10:08:41.791 INFO:tasks.workunit.client.0.smithi153.stdout:9/136: dread d1/d4/fe [0,4194304] 0 2024-03-21T10:08:41.791 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:41.791+0000 7f17e7e00700 1 -- 172.21.15.153:0/2611252182 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 28) v1 ==== 46435+0+0 (secure 0 0 0) 0x7f17d80105a0 con 0x7f17f0101db0 2024-03-21T10:08:41.793 INFO:tasks.workunit.client.1.smithi174.stdout:1/16: dwrite f1 [0,4194304] 0 2024-03-21T10:08:41.794 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:41.793+0000 7f17e7e00700 1 --2- 172.21.15.153:0/2611252182 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f17d4039520 0x7f17d403b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:08:41.794 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:41.793+0000 7f17e7e00700 1 -- 172.21.15.153:0/2611252182 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f17d8014070 con 0x7f17f0101db0 2024-03-21T10:08:41.794 INFO:tasks.workunit.client.1.smithi174.stdout:0/18: write f0 [4679268,122380] 0 2024-03-21T10:08:41.795 INFO:tasks.workunit.client.0.smithi153.stdout:1/133: creat d6/d10/d2c/f2e x:0 0 0 2024-03-21T10:08:41.795 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:41.793+0000 7f17ef400700 1 --2- 172.21.15.153:0/2611252182 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f17d4039520 0x7f17d403b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:08:41.795 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:41.795+0000 7f17ef400700 1 --2- 172.21.15.153:0/2611252182 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f17d4039520 0x7f17d403b9e0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f17e000b6f0 tx=0x7f17e0009f70 comp rx=0 tx=0).ready entity=mgr.14666 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:08:41.799 INFO:tasks.workunit.client.0.smithi153.stdout:1/134: write f4 [441987,50957] 0 2024-03-21T10:08:41.803 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:41.802+0000 7f17e7e00700 1 -- 172.21.15.153:0/2611252182 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185454 (secure 0 0 0) 0x7f17d8010a60 con 0x7f17f0101db0 2024-03-21T10:08:41.812 INFO:tasks.workunit.client.0.smithi153.stdout:9/137: fsync d1/d4/f14 0 2024-03-21T10:08:41.816 INFO:tasks.workunit.client.0.smithi153.stdout:9/138: chown d1/d12/f26 0 1 2024-03-21T10:08:41.818 INFO:tasks.workunit.client.0.smithi153.stdout:4/250: dread f1 [4194304,4194304] 0 2024-03-21T10:08:41.820 INFO:tasks.workunit.client.0.smithi153.stdout:9/139: mkdir d1/d12/d1c/d2e 0 2024-03-21T10:08:41.821 INFO:tasks.workunit.client.0.smithi153.stdout:2/163: dwrite dd/d15/d18/f29 [0,4194304] 0 2024-03-21T10:08:41.822 INFO:tasks.workunit.client.0.smithi153.stdout:9/140: read d1/f27 [37224,45268] 0 2024-03-21T10:08:41.823 INFO:tasks.workunit.client.1.smithi174.stdout:0/19: dwrite f4 [0,4194304] 0 2024-03-21T10:08:41.825 INFO:tasks.workunit.client.0.smithi153.stdout:1/135: dwrite d6/d13/f1d [4194304,4194304] 0 2024-03-21T10:08:41.826 INFO:tasks.workunit.client.0.smithi153.stdout:4/251: rename fb to d11/d18/d50/f53 0 2024-03-21T10:08:41.829 INFO:tasks.workunit.client.0.smithi153.stdout:1/136: write d6/f16 [84283,29103] 0 2024-03-21T10:08:41.829 INFO:tasks.workunit.client.0.smithi153.stdout:1/137: write d6/f9 [1333083,87605] 0 2024-03-21T10:08:41.830 INFO:tasks.workunit.client.0.smithi153.stdout:8/118: write d5/d9/f12 [492859,30773] 0 2024-03-21T10:08:41.850 INFO:tasks.workunit.client.0.smithi153.stdout:4/252: dread d11/d18/d50/f53 [0,4194304] 0 2024-03-21T10:08:41.852 INFO:tasks.workunit.client.0.smithi153.stdout:4/253: chown d11/d2d/f3a 249803064 1 2024-03-21T10:08:41.852 INFO:tasks.workunit.client.0.smithi153.stdout:1/138: dwrite d6/d10/d2c/f2e [0,4194304] 0 2024-03-21T10:08:41.857 INFO:tasks.workunit.client.0.smithi153.stdout:1/139: write d6/fe [6971857,44426] 0 2024-03-21T10:08:41.859 INFO:tasks.workunit.client.0.smithi153.stdout:1/140: readlink d6/d13/l25 0 2024-03-21T10:08:41.862 INFO:tasks.workunit.client.0.smithi153.stdout:1/141: read d6/d10/d2c/f2e [1173806,75102] 0 2024-03-21T10:08:41.874 INFO:tasks.workunit.client.0.smithi153.stdout:2/164: write dd/d15/d18/d2c/f2d [1468636,93071] 0 2024-03-21T10:08:41.876 INFO:tasks.workunit.client.0.smithi153.stdout:1/142: creat d6/f2f x:0 0 0 2024-03-21T10:08:41.876 INFO:tasks.workunit.client.0.smithi153.stdout:4/254: dwrite d11/d18/f4b [4194304,4194304] 0 2024-03-21T10:08:41.881 INFO:tasks.workunit.client.0.smithi153.stdout:1/143: mknod d6/d13/c30 0 2024-03-21T10:08:41.883 INFO:tasks.workunit.client.1.smithi174.stdout:0/20: fsync f4 0 2024-03-21T10:08:41.883 INFO:tasks.workunit.client.0.smithi153.stdout:2/165: rename dd/d15/d18/f29 to dd/d15/d18/d2c/f35 0 2024-03-21T10:08:41.886 INFO:tasks.workunit.client.0.smithi153.stdout:2/166: stat dd/d15/c2b 0 2024-03-21T10:08:41.886 INFO:tasks.workunit.client.0.smithi153.stdout:8/119: dwrite d5/d9/dc/f16 [0,4194304] 0 2024-03-21T10:08:41.894 INFO:tasks.workunit.client.0.smithi153.stdout:8/120: creat d5/d9/dc/dd/f22 x:0 0 0 2024-03-21T10:08:41.894 INFO:tasks.workunit.client.0.smithi153.stdout:2/167: creat dd/d15/d18/d2c/d33/f36 x:0 0 0 2024-03-21T10:08:41.895 INFO:tasks.workunit.client.0.smithi153.stdout:8/121: fsync d5/f8 0 2024-03-21T10:08:41.895 INFO:tasks.workunit.client.0.smithi153.stdout:2/168: fdatasync dd/d15/d18/f2a 0 2024-03-21T10:08:41.897 INFO:tasks.workunit.client.0.smithi153.stdout:0/102: sync 2024-03-21T10:08:41.899 INFO:tasks.workunit.client.1.smithi174.stdout:0/21: dread f0 [0,4194304] 0 2024-03-21T10:08:41.900 INFO:tasks.workunit.client.0.smithi153.stdout:8/122: stat d5/d9/dc/dd/f22 0 2024-03-21T10:08:41.904 INFO:tasks.workunit.client.0.smithi153.stdout:2/169: creat dd/f37 x:0 0 0 2024-03-21T10:08:41.910 INFO:tasks.workunit.client.0.smithi153.stdout:9/141: dread d1/d12/f26 [0,4194304] 0 2024-03-21T10:08:41.910 INFO:tasks.workunit.client.0.smithi153.stdout:2/170: unlink dd/d15/d18/d2c/f2f 0 2024-03-21T10:08:41.914 INFO:tasks.workunit.client.0.smithi153.stdout:9/142: write d1/d4/f1a [2860435,48182] 0 2024-03-21T10:08:41.917 INFO:tasks.workunit.client.0.smithi153.stdout:9/143: readlink d1/d12/d1f/l2c 0 2024-03-21T10:08:41.918 INFO:tasks.workunit.client.0.smithi153.stdout:8/123: dread d5/d9/dc/dd/f1c [0,4194304] 0 2024-03-21T10:08:41.919 INFO:tasks.workunit.client.0.smithi153.stdout:0/103: dread f2 [0,4194304] 0 2024-03-21T10:08:41.921 INFO:tasks.workunit.client.0.smithi153.stdout:0/104: chown d4/c14 210924744 1 2024-03-21T10:08:41.922 INFO:tasks.workunit.client.0.smithi153.stdout:8/124: creat d5/d1b/d15/f23 x:0 0 0 2024-03-21T10:08:41.924 INFO:tasks.workunit.client.0.smithi153.stdout:0/105: dread - d4/db/f1b zero size 2024-03-21T10:08:41.928 INFO:tasks.workunit.client.0.smithi153.stdout:2/171: truncate dd/d15/d18/f2a 4179886 0 2024-03-21T10:08:41.930 INFO:tasks.workunit.client.0.smithi153.stdout:8/125: mkdir d5/d1b/d15/d24 0 2024-03-21T10:08:41.934 INFO:tasks.workunit.client.0.smithi153.stdout:9/144: dwrite d1/d4/f20 [4194304,4194304] 0 2024-03-21T10:08:41.936 INFO:tasks.workunit.client.0.smithi153.stdout:9/145: write d1/f1b [854349,49523] 0 2024-03-21T10:08:41.937 INFO:tasks.workunit.client.0.smithi153.stdout:2/172: rename dd/f14 to dd/d15/d18/f38 0 2024-03-21T10:08:41.939 INFO:tasks.workunit.client.1.smithi174.stdout:4/35: getdents d4 0 2024-03-21T10:08:41.940 INFO:tasks.workunit.client.0.smithi153.stdout:8/126: rename d5/d9/dc/dd/d20 to d5/d9/dc/dd/d20/d25 22 2024-03-21T10:08:41.940 INFO:tasks.workunit.client.0.smithi153.stdout:0/106: dread d4/f6 [0,4194304] 0 2024-03-21T10:08:41.941 INFO:tasks.workunit.client.0.smithi153.stdout:8/127: stat d5/d1b/f18 0 2024-03-21T10:08:41.943 INFO:tasks.workunit.client.0.smithi153.stdout:2/173: creat dd/d15/d18/d2c/f39 x:0 0 0 2024-03-21T10:08:41.944 INFO:tasks.workunit.client.1.smithi174.stdout:2/28: mknod d5/d8/cb 0 2024-03-21T10:08:41.945 INFO:tasks.workunit.client.0.smithi153.stdout:0/107: creat d4/d16/d1f/f20 x:0 0 0 2024-03-21T10:08:41.947 INFO:tasks.workunit.client.0.smithi153.stdout:0/108: chown d4/db/c1e 105433273 1 2024-03-21T10:08:41.948 INFO:tasks.workunit.client.0.smithi153.stdout:2/174: mkdir dd/d15/d18/d3a 0 2024-03-21T10:08:41.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:41 smithi174 ceph-mon[37496]: from='client.14700 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:08:41.953 INFO:tasks.workunit.client.0.smithi153.stdout:8/128: getdents d5/d1b/d15 0 2024-03-21T10:08:41.954 INFO:tasks.workunit.client.1.smithi174.stdout:9/12: getdents d2 0 2024-03-21T10:08:41.954 INFO:tasks.workunit.client.1.smithi174.stdout:0/22: creat f5 x:0 0 0 2024-03-21T10:08:41.956 INFO:tasks.workunit.client.1.smithi174.stdout:2/29: symlink d5/d8/lc 0 2024-03-21T10:08:41.958 INFO:tasks.workunit.client.0.smithi153.stdout:8/129: rename d5/d1b/d15 to d5/d9/dc/dd/d20/d26 0 2024-03-21T10:08:41.961 INFO:tasks.workunit.client.1.smithi174.stdout:4/36: link d4/f8 d4/fe 0 2024-03-21T10:08:41.962 INFO:tasks.workunit.client.0.smithi153.stdout:0/109: dwrite d4/db/f13 [0,4194304] 0 2024-03-21T10:08:41.962 INFO:tasks.workunit.client.0.smithi153.stdout:8/130: symlink d5/d9/dc/dd/d20/d26/d24/l27 0 2024-03-21T10:08:41.963 INFO:tasks.workunit.client.0.smithi153.stdout:0/110: stat d4/db/c1c 0 2024-03-21T10:08:41.965 INFO:tasks.workunit.client.0.smithi153.stdout:0/111: chown d4/d16/d19 8277257 1 2024-03-21T10:08:41.965 INFO:tasks.workunit.client.0.smithi153.stdout:0/112: chown d4/c5 8 1 2024-03-21T10:08:41.966 INFO:tasks.workunit.client.1.smithi174.stdout:2/30: mknod d5/cd 0 2024-03-21T10:08:41.966 INFO:tasks.workunit.client.1.smithi174.stdout:9/13: creat d2/f4 x:0 0 0 2024-03-21T10:08:41.966 INFO:tasks.workunit.client.1.smithi174.stdout:4/37: creat d4/ff x:0 0 0 2024-03-21T10:08:41.970 INFO:tasks.workunit.client.0.smithi153.stdout:9/146: truncate d1/f16 4849994 0 2024-03-21T10:08:41.971 INFO:tasks.workunit.client.0.smithi153.stdout:0/113: symlink d4/d16/l21 0 2024-03-21T10:08:41.972 INFO:tasks.workunit.client.0.smithi153.stdout:9/147: chown d1/d12/d1c/c23 2327721 1 2024-03-21T10:08:41.976 INFO:tasks.workunit.client.0.smithi153.stdout:9/148: readlink d1/d12/d1f/l2b 0 2024-03-21T10:08:41.977 INFO:tasks.workunit.client.0.smithi153.stdout:8/131: dread d5/d9/dc/f16 [0,4194304] 0 2024-03-21T10:08:41.978 INFO:tasks.workunit.client.0.smithi153.stdout:8/132: chown d5/d9/dc/dd 4165 1 2024-03-21T10:08:41.980 INFO:tasks.workunit.client.0.smithi153.stdout:8/133: unlink d5/d9/f12 0 2024-03-21T10:08:41.981 INFO:tasks.workunit.client.0.smithi153.stdout:8/134: fdatasync d5/d1b/f1d 0 2024-03-21T10:08:41.983 INFO:tasks.workunit.client.1.smithi174.stdout:4/38: dread d4/fa [0,4194304] 0 2024-03-21T10:08:41.985 INFO:tasks.workunit.client.1.smithi174.stdout:2/31: link f2 d5/d8/fe 0 2024-03-21T10:08:41.990 INFO:tasks.workunit.client.0.smithi153.stdout:0/114: rmdir d4 39 2024-03-21T10:08:41.991 INFO:tasks.workunit.client.1.smithi174.stdout:5/18: dwrite d1/f3 [0,4194304] 0 2024-03-21T10:08:41.993 INFO:tasks.workunit.client.0.smithi153.stdout:9/149: truncate d1/d12/f26 1279553 0 2024-03-21T10:08:41.996 INFO:tasks.workunit.client.0.smithi153.stdout:9/150: write d1/d4/f20 [5556910,84343] 0 2024-03-21T10:08:41.997 INFO:tasks.workunit.client.0.smithi153.stdout:0/115: mkdir d4/d16/d1f/d22 0 2024-03-21T10:08:41.998 INFO:tasks.workunit.client.0.smithi153.stdout:8/135: dwrite d5/d9/f17 [0,4194304] 0 2024-03-21T10:08:41.999 INFO:tasks.workunit.client.0.smithi153.stdout:9/151: symlink d1/d4/l2f 0 2024-03-21T10:08:41.999 INFO:tasks.workunit.client.1.smithi174.stdout:9/14: dread d2/f3 [0,4194304] 0 2024-03-21T10:08:42.000 INFO:tasks.workunit.client.1.smithi174.stdout:2/32: creat d5/d8/ff x:0 0 0 2024-03-21T10:08:42.002 INFO:tasks.workunit.client.0.smithi153.stdout:0/116: write d4/d16/d1f/f20 [453508,125305] 0 2024-03-21T10:08:42.003 INFO:tasks.workunit.client.0.smithi153.stdout:8/136: write d5/d9/dc/dd/d20/d26/f23 [457990,114211] 0 2024-03-21T10:08:42.005 INFO:tasks.workunit.client.0.smithi153.stdout:9/152: chown d1/d12/d1f/l2a 356345 1 2024-03-21T10:08:42.006 INFO:tasks.workunit.client.0.smithi153.stdout:8/137: readlink d5/d9/dc/dd/d20/d26/d24/l27 0 2024-03-21T10:08:42.006 INFO:tasks.workunit.client.1.smithi174.stdout:9/15: fsync d2/f3 0 2024-03-21T10:08:42.009 INFO:tasks.workunit.client.0.smithi153.stdout:9/153: rename d1 to d1/d12/d1c/d30 22 2024-03-21T10:08:42.010 INFO:tasks.workunit.client.0.smithi153.stdout:0/117: creat d4/f23 x:0 0 0 2024-03-21T10:08:42.011 INFO:tasks.workunit.client.1.smithi174.stdout:4/39: dread d4/f7 [0,4194304] 0 2024-03-21T10:08:42.014 INFO:tasks.workunit.client.0.smithi153.stdout:8/138: creat d5/d9/dc/dd/d20/d26/d24/f28 x:0 0 0 2024-03-21T10:08:42.018 INFO:tasks.workunit.client.0.smithi153.stdout:8/139: unlink d5/d9/dc/dd/f1c 0 2024-03-21T10:08:42.019 INFO:tasks.workunit.client.1.smithi174.stdout:5/19: fsync d1/f3 0 2024-03-21T10:08:42.020 INFO:tasks.workunit.client.0.smithi153.stdout:8/140: write d5/d1b/f1d [1773252,56084] 0 2024-03-21T10:08:42.026 INFO:tasks.workunit.client.0.smithi153.stdout:9/154: dwrite d1/f1b [0,4194304] 0 2024-03-21T10:08:42.027 INFO:tasks.workunit.client.1.smithi174.stdout:2/33: dread f2 [0,4194304] 0 2024-03-21T10:08:42.031 INFO:tasks.workunit.client.1.smithi174.stdout:2/34: stat d5/d8/l9 0 2024-03-21T10:08:42.035 INFO:tasks.workunit.client.1.smithi174.stdout:2/35: write d5/fa [565114,98851] 0 2024-03-21T10:08:42.037 INFO:tasks.workunit.client.1.smithi174.stdout:4/40: dwrite d4/ff [0,4194304] 0 2024-03-21T10:08:42.040 INFO:tasks.workunit.client.1.smithi174.stdout:4/41: dread - d4/fd zero size 2024-03-21T10:08:42.041 INFO:tasks.workunit.client.0.smithi153.stdout:9/155: dwrite d1/d12/f1d [0,4194304] 0 2024-03-21T10:08:42.042 INFO:tasks.workunit.client.1.smithi174.stdout:2/36: creat d5/f10 x:0 0 0 2024-03-21T10:08:42.044 INFO:tasks.workunit.client.1.smithi174.stdout:4/42: creat d4/f10 x:0 0 0 2024-03-21T10:08:42.046 INFO:tasks.workunit.client.1.smithi174.stdout:4/43: write f0 [5060919,107917] 0 2024-03-21T10:08:42.046 INFO:tasks.workunit.client.0.smithi153.stdout:9/156: creat d1/d12/d1c/f31 x:0 0 0 2024-03-21T10:08:42.071 INFO:tasks.workunit.client.1.smithi174.stdout:4/44: dread d4/f7 [0,4194304] 0 2024-03-21T10:08:42.072 INFO:tasks.workunit.client.0.smithi153.stdout:8/141: read d5/d1b/f1d [316611,1277] 0 2024-03-21T10:08:42.072 INFO:tasks.workunit.client.1.smithi174.stdout:1/17: truncate f1 3868089 0 2024-03-21T10:08:42.078 INFO:tasks.workunit.client.1.smithi174.stdout:1/18: chown l3 0 1 2024-03-21T10:08:42.082 INFO:tasks.workunit.client.1.smithi174.stdout:9/16: fsync d2/f4 0 2024-03-21T10:08:42.085 INFO:tasks.workunit.client.0.smithi153.stdout:8/142: dwrite d5/d9/f11 [0,4194304] 0 2024-03-21T10:08:42.085 INFO:tasks.workunit.client.0.smithi153.stdout:0/118: truncate d4/f10 6497398 0 2024-03-21T10:08:42.086 INFO:tasks.workunit.client.1.smithi174.stdout:9/17: mknod d2/c5 0 2024-03-21T10:08:42.086 INFO:tasks.workunit.client.0.smithi153.stdout:0/119: read d4/db/f11 [1538049,43240] 0 2024-03-21T10:08:42.088 INFO:tasks.workunit.client.0.smithi153.stdout:8/143: symlink d5/d9/dc/dd/d20/d26/d24/l29 0 2024-03-21T10:08:42.089 INFO:tasks.workunit.client.0.smithi153.stdout:9/157: write d1/d12/f26 [1685142,105329] 0 2024-03-21T10:08:42.092 INFO:tasks.workunit.client.0.smithi153.stdout:3/191: sync 2024-03-21T10:08:42.092 INFO:tasks.workunit.client.0.smithi153.stdout:6/374: sync 2024-03-21T10:08:42.093 INFO:tasks.workunit.client.1.smithi174.stdout:9/18: creat d2/f6 x:0 0 0 2024-03-21T10:08:42.094 INFO:tasks.workunit.client.0.smithi153.stdout:6/375: dread - d1f/d2b/d33/f41 zero size 2024-03-21T10:08:42.094 INFO:tasks.workunit.client.0.smithi153.stdout:3/192: stat dd/d1e/d2f/d35/l3f 0 2024-03-21T10:08:42.095 INFO:tasks.workunit.client.0.smithi153.stdout:0/120: creat d4/d16/d1f/f24 x:0 0 0 2024-03-21T10:08:42.098 INFO:tasks.workunit.client.0.smithi153.stdout:9/158: creat d1/d12/d1c/d24/f32 x:0 0 0 2024-03-21T10:08:42.099 INFO:tasks.workunit.client.0.smithi153.stdout:6/376: symlink d1f/d2b/d33/d40/d5f/l88 0 2024-03-21T10:08:42.100 INFO:tasks.workunit.client.0.smithi153.stdout:3/193: symlink dd/d1e/d2f/l42 0 2024-03-21T10:08:42.105 INFO:tasks.workunit.client.1.smithi174.stdout:9/19: mknod d2/c7 0 2024-03-21T10:08:42.111 INFO:tasks.workunit.client.1.smithi174.stdout:9/20: dread - d2/f6 zero size 2024-03-21T10:08:42.113 INFO:tasks.workunit.client.0.smithi153.stdout:8/144: dwrite d5/d1b/f1d [0,4194304] 0 2024-03-21T10:08:42.116 INFO:tasks.workunit.client.0.smithi153.stdout:0/121: read d4/f10 [576305,94114] 0 2024-03-21T10:08:42.116 INFO:tasks.workunit.client.1.smithi174.stdout:4/45: dwrite d4/fd [0,4194304] 0 2024-03-21T10:08:42.117 INFO:tasks.workunit.client.0.smithi153.stdout:6/377: dwrite d1f/d2f/f7d [0,4194304] 0 2024-03-21T10:08:42.117 INFO:tasks.workunit.client.0.smithi153.stdout:8/145: write d5/d9/f7 [5681796,56319] 0 2024-03-21T10:08:42.123 INFO:tasks.workunit.client.1.smithi174.stdout:9/21: creat d2/f8 x:0 0 0 2024-03-21T10:08:42.126 INFO:tasks.workunit.client.0.smithi153.stdout:9/159: rmdir d1/d12/d1c/d24 39 2024-03-21T10:08:42.126 INFO:tasks.workunit.client.1.smithi174.stdout:5/20: dwrite d1/f3 [4194304,4194304] 0 2024-03-21T10:08:42.129 INFO:tasks.workunit.client.1.smithi174.stdout:2/37: dwrite d5/d8/fe [0,4194304] 0 2024-03-21T10:08:42.132 INFO:tasks.workunit.client.1.smithi174.stdout:4/46: dread d4/fa [0,4194304] 0 2024-03-21T10:08:42.132 INFO:tasks.workunit.client.1.smithi174.stdout:9/22: creat d2/f9 x:0 0 0 2024-03-21T10:08:42.132 INFO:tasks.workunit.client.1.smithi174.stdout:9/23: stat d2/c7 0 2024-03-21T10:08:42.136 INFO:tasks.workunit.client.0.smithi153.stdout:0/122: symlink d4/l25 0 2024-03-21T10:08:42.137 INFO:tasks.workunit.client.1.smithi174.stdout:1/19: truncate f1 4561977 0 2024-03-21T10:08:42.139 INFO:tasks.workunit.client.0.smithi153.stdout:0/123: chown d4/db/c1c 28 1 2024-03-21T10:08:42.142 INFO:tasks.workunit.client.1.smithi174.stdout:5/21: symlink d1/l5 0 2024-03-21T10:08:42.143 INFO:tasks.workunit.client.1.smithi174.stdout:4/47: symlink d4/l11 0 2024-03-21T10:08:42.145 INFO:tasks.workunit.client.1.smithi174.stdout:1/20: mkdir d4 0 2024-03-21T10:08:42.145 INFO:tasks.workunit.client.0.smithi153.stdout:0/124: mknod d4/d16/d1f/d22/c26 0 2024-03-21T10:08:42.146 INFO:tasks.workunit.client.0.smithi153.stdout:3/194: dwrite dd/f13 [0,4194304] 0 2024-03-21T10:08:42.150 INFO:tasks.workunit.client.1.smithi174.stdout:4/48: symlink d4/l12 0 2024-03-21T10:08:42.151 INFO:tasks.workunit.client.0.smithi153.stdout:3/195: write dd/f20 [4450735,101082] 0 2024-03-21T10:08:42.160 INFO:tasks.workunit.client.1.smithi174.stdout:1/21: mkdir d4/d5 0 2024-03-21T10:08:42.170 INFO:tasks.workunit.client.0.smithi153.stdout:6/378: write d1f/d2b/d33/d40/f5d [714598,97390] 0 2024-03-21T10:08:42.170 INFO:tasks.workunit.client.1.smithi174.stdout:9/24: dwrite d2/f9 [0,4194304] 0 2024-03-21T10:08:42.173 INFO:tasks.workunit.client.0.smithi153.stdout:3/196: dread dd/f15 [0,4194304] 0 2024-03-21T10:08:42.174 INFO:tasks.workunit.client.0.smithi153.stdout:3/197: write dd/f1f [3782095,5599] 0 2024-03-21T10:08:42.175 INFO:tasks.workunit.client.1.smithi174.stdout:9/25: rename d2 to d2/da 22 2024-03-21T10:08:42.175 INFO:tasks.workunit.client.1.smithi174.stdout:9/26: readlink - no filename 2024-03-21T10:08:42.175 INFO:tasks.workunit.client.1.smithi174.stdout:2/38: fsync d5/d8/fe 0 2024-03-21T10:08:42.177 INFO:tasks.workunit.client.1.smithi174.stdout:2/39: fdatasync d5/d8/ff 0 2024-03-21T10:08:42.179 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:42.178+0000 7f17f6600700 1 -- 172.21.15.153:0/2611252182 --> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f17f0106700 con 0x7f17d4039520 2024-03-21T10:08:42.180 INFO:tasks.workunit.client.1.smithi174.stdout:9/27: read d2/f9 [3229505,26901] 0 2024-03-21T10:08:42.183 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:42.182+0000 7f17e7e00700 1 -- 172.21.15.153:0/2611252182 <== mgr.14666 v2:172.21.15.153:6800/2524145044 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+329 (secure 0 0 0) 0x7f17f0106700 con 0x7f17d4039520 2024-03-21T10:08:42.183 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:08:42.183 INFO:teuthology.orchestra.run.smithi153.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232", 2024-03-21T10:08:42.183 INFO:teuthology.orchestra.run.smithi153.stdout: "in_progress": true, 2024-03-21T10:08:42.183 INFO:teuthology.orchestra.run.smithi153.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-03-21T10:08:42.183 INFO:teuthology.orchestra.run.smithi153.stdout: "services_complete": [ 2024-03-21T10:08:42.183 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr" 2024-03-21T10:08:42.183 INFO:teuthology.orchestra.run.smithi153.stdout: ], 2024-03-21T10:08:42.184 INFO:teuthology.orchestra.run.smithi153.stdout: "progress": "2/2 daemons upgraded", 2024-03-21T10:08:42.184 INFO:teuthology.orchestra.run.smithi153.stdout: "message": "", 2024-03-21T10:08:42.184 INFO:teuthology.orchestra.run.smithi153.stdout: "is_paused": false 2024-03-21T10:08:42.184 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:08:42.185 INFO:tasks.workunit.client.1.smithi174.stdout:4/49: fdatasync d4/fd 0 2024-03-21T10:08:42.185 INFO:tasks.workunit.client.0.smithi153.stdout:6/379: dwrite d1f/d2b/f2c [0,4194304] 0 2024-03-21T10:08:42.186 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:42.186+0000 7f17f6600700 1 -- 172.21.15.153:0/2611252182 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f17d4039520 msgr2=0x7f17d403b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:08:42.186 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:42.186+0000 7f17f6600700 1 --2- 172.21.15.153:0/2611252182 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f17d4039520 0x7f17d403b9e0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f17e000b6f0 tx=0x7f17e0009f70 comp rx=0 tx=0).stop 2024-03-21T10:08:42.186 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:42.186+0000 7f17f6600700 1 -- 172.21.15.153:0/2611252182 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f17f0101db0 msgr2=0x7f17f0073180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:08:42.187 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:42.186+0000 7f17f6600700 1 --2- 172.21.15.153:0/2611252182 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f17f0101db0 0x7f17f0073180 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f17d800e920 tx=0x7f17d800ec30 comp rx=0 tx=0).stop 2024-03-21T10:08:42.187 INFO:tasks.workunit.client.1.smithi174.stdout:4/50: dread - d4/fc zero size 2024-03-21T10:08:42.187 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:42.186+0000 7f17f6600700 1 -- 172.21.15.153:0/2611252182 shutdown_connections 2024-03-21T10:08:42.187 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:42.186+0000 7f17f6600700 1 --2- 172.21.15.153:0/2611252182 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f17d4039520 0x7f17d403b9e0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:08:42.187 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:42.186+0000 7f17f6600700 1 --2- 172.21.15.153:0/2611252182 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f17f0100bb0 0x7f17f0074b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:08:42.188 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:42.186+0000 7f17f6600700 1 --2- 172.21.15.153:0/2611252182 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f17f0101db0 0x7f17f0073180 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:08:42.188 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:42.186+0000 7f17f6600700 1 -- 172.21.15.153:0/2611252182 >> 172.21.15.153:0/2611252182 conn(0x7f17f00fc110 msgr2=0x7f17f0104fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:08:42.188 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:42.187+0000 7f17f6600700 1 -- 172.21.15.153:0/2611252182 shutdown_connections 2024-03-21T10:08:42.188 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:08:42.187+0000 7f17f6600700 1 -- 172.21.15.153:0/2611252182 wait complete. 2024-03-21T10:08:42.192 INFO:tasks.workunit.client.0.smithi153.stdout:6/380: rename d1f/f42 to d1f/d2b/f89 0 2024-03-21T10:08:42.194 INFO:tasks.workunit.client.0.smithi153.stdout:6/381: dread - d1f/f6c zero size 2024-03-21T10:08:42.196 INFO:tasks.workunit.client.0.smithi153.stdout:3/198: dread dd/f1c [0,4194304] 0 2024-03-21T10:08:42.198 INFO:tasks.workunit.client.0.smithi153.stdout:6/382: mknod d1f/d2b/c8a 0 2024-03-21T10:08:42.200 INFO:tasks.workunit.client.0.smithi153.stdout:3/199: chown dd/d1e/c24 46249587 1 2024-03-21T10:08:42.204 INFO:tasks.workunit.client.1.smithi174.stdout:9/28: write d2/f3 [4380599,55843] 0 2024-03-21T10:08:42.204 INFO:tasks.workunit.client.1.smithi174.stdout:5/22: dwrite d1/f3 [4194304,4194304] 0 2024-03-21T10:08:42.205 INFO:tasks.workunit.client.0.smithi153.stdout:3/200: write f6 [868378,60654] 0 2024-03-21T10:08:42.205 INFO:tasks.workunit.client.0.smithi153.stdout:6/383: fdatasync d1f/d2b/d33/d40/d5f/f6d 0 2024-03-21T10:08:42.209 INFO:tasks.workunit.client.0.smithi153.stdout:3/201: truncate f0 1749137 0 2024-03-21T10:08:42.210 INFO:tasks.workunit.client.1.smithi174.stdout:9/29: dread - d2/f4 zero size 2024-03-21T10:08:42.218 INFO:tasks.workunit.client.0.smithi153.stdout:3/202: truncate dd/f15 780360 0 2024-03-21T10:08:42.221 INFO:tasks.workunit.client.1.smithi174.stdout:9/30: link d2/c5 d2/cb 0 2024-03-21T10:08:42.224 INFO:tasks.workunit.client.1.smithi174.stdout:1/22: dwrite f1 [4194304,4194304] 0 2024-03-21T10:08:42.224 INFO:tasks.workunit.client.1.smithi174.stdout:2/40: dread f2 [0,4194304] 0 2024-03-21T10:08:42.227 INFO:tasks.workunit.client.0.smithi153.stdout:7/111: sync 2024-03-21T10:08:42.227 INFO:tasks.workunit.client.0.smithi153.stdout:1/144: sync 2024-03-21T10:08:42.228 INFO:tasks.workunit.client.0.smithi153.stdout:5/138: sync 2024-03-21T10:08:42.228 INFO:tasks.workunit.client.0.smithi153.stdout:4/255: sync 2024-03-21T10:08:42.229 INFO:tasks.workunit.client.0.smithi153.stdout:4/256: readlink d11/d18/l4e 0 2024-03-21T10:08:42.232 INFO:tasks.workunit.client.1.smithi174.stdout:4/51: dread d4/f7 [0,4194304] 0 2024-03-21T10:08:42.233 INFO:tasks.workunit.client.0.smithi153.stdout:7/112: mkdir d4/d11/d24 0 2024-03-21T10:08:42.233 INFO:tasks.workunit.client.1.smithi174.stdout:4/52: stat d4/l11 0 2024-03-21T10:08:42.234 INFO:tasks.workunit.client.1.smithi174.stdout:2/41: creat d5/f11 x:0 0 0 2024-03-21T10:08:42.237 INFO:tasks.workunit.client.0.smithi153.stdout:5/139: rename d4/d5/de/d28 to d4/d5/de/d2f 0 2024-03-21T10:08:42.240 INFO:tasks.workunit.client.0.smithi153.stdout:7/113: truncate d4/f1a 1402186 0 2024-03-21T10:08:42.241 INFO:tasks.workunit.client.0.smithi153.stdout:5/140: rmdir d4/d5/d16 39 2024-03-21T10:08:42.242 INFO:tasks.workunit.client.0.smithi153.stdout:7/114: chown d4/f9 77070 1 2024-03-21T10:08:42.242 INFO:tasks.workunit.client.1.smithi174.stdout:5/23: dwrite f0 [0,4194304] 0 2024-03-21T10:08:42.245 INFO:tasks.workunit.client.0.smithi153.stdout:5/141: creat d4/d5/df/f30 x:0 0 0 2024-03-21T10:08:42.245 INFO:tasks.workunit.client.1.smithi174.stdout:4/53: unlink d4/l11 0 2024-03-21T10:08:42.246 INFO:tasks.workunit.client.0.smithi153.stdout:7/115: chown d4/c22 25064 1 2024-03-21T10:08:42.247 INFO:tasks.workunit.client.0.smithi153.stdout:4/257: dwrite fd [0,4194304] 0 2024-03-21T10:08:42.251 INFO:tasks.workunit.client.1.smithi174.stdout:1/23: dread f1 [4194304,4194304] 0 2024-03-21T10:08:42.253 INFO:tasks.workunit.client.1.smithi174.stdout:9/31: dwrite d2/f3 [0,4194304] 0 2024-03-21T10:08:42.257 INFO:tasks.workunit.client.1.smithi174.stdout:2/42: link d5/fa d5/f12 0 2024-03-21T10:08:42.258 INFO:tasks.workunit.client.0.smithi153.stdout:1/145: truncate f4 2715640 0 2024-03-21T10:08:42.263 INFO:tasks.workunit.client.0.smithi153.stdout:7/116: dread d4/f12 [0,4194304] 0 2024-03-21T10:08:42.264 INFO:tasks.workunit.client.0.smithi153.stdout:1/146: creat d6/f31 x:0 0 0 2024-03-21T10:08:42.265 INFO:tasks.workunit.client.1.smithi174.stdout:9/32: creat d2/fc x:0 0 0 2024-03-21T10:08:42.265 INFO:tasks.workunit.client.0.smithi153.stdout:4/258: dread f1 [4194304,4194304] 0 2024-03-21T10:08:42.267 INFO:tasks.workunit.client.1.smithi174.stdout:5/24: dwrite f0 [0,4194304] 0 2024-03-21T10:08:42.269 INFO:tasks.workunit.client.0.smithi153.stdout:4/259: readlink d11/d18/l37 0 2024-03-21T10:08:42.274 INFO:tasks.workunit.client.1.smithi174.stdout:1/24: dread f1 [4194304,4194304] 0 2024-03-21T10:08:42.277 INFO:tasks.workunit.client.0.smithi153.stdout:5/142: truncate d4/d8/f11 633808 0 2024-03-21T10:08:42.279 INFO:tasks.workunit.client.1.smithi174.stdout:4/54: write d4/f8 [3628286,128374] 0 2024-03-21T10:08:42.281 INFO:tasks.workunit.client.1.smithi174.stdout:5/25: creat d1/f6 x:0 0 0 2024-03-21T10:08:42.282 INFO:tasks.workunit.client.0.smithi153.stdout:1/147: dread f5 [0,4194304] 0 2024-03-21T10:08:42.284 INFO:tasks.workunit.client.0.smithi153.stdout:5/143: link d4/d5/de/l20 d4/d5/l31 0 2024-03-21T10:08:42.285 INFO:tasks.workunit.client.1.smithi174.stdout:4/55: creat d4/f13 x:0 0 0 2024-03-21T10:08:42.286 INFO:tasks.workunit.client.0.smithi153.stdout:1/148: mkdir d6/d32 0 2024-03-21T10:08:42.289 INFO:tasks.workunit.client.0.smithi153.stdout:5/144: mkdir d4/d5/df/d32 0 2024-03-21T10:08:42.290 INFO:tasks.workunit.client.0.smithi153.stdout:1/149: creat d6/d10/d2c/f33 x:0 0 0 2024-03-21T10:08:42.290 INFO:tasks.workunit.client.0.smithi153.stdout:1/150: fdatasync d6/fa 0 2024-03-21T10:08:42.292 INFO:tasks.workunit.client.1.smithi174.stdout:4/56: symlink d4/l14 0 2024-03-21T10:08:42.300 INFO:tasks.workunit.client.1.smithi174.stdout:9/33: dwrite d2/f8 [0,4194304] 0 2024-03-21T10:08:42.303 INFO:tasks.workunit.client.0.smithi153.stdout:5/145: dwrite d4/d5/d16/f19 [0,4194304] 0 2024-03-21T10:08:42.304 INFO:tasks.workunit.client.0.smithi153.stdout:5/146: fsync d4/d5/f1a 0 2024-03-21T10:08:42.307 INFO:tasks.workunit.client.1.smithi174.stdout:4/57: rename d4/fd to d4/f15 0 2024-03-21T10:08:42.307 INFO:tasks.workunit.client.1.smithi174.stdout:1/25: dread f1 [4194304,4194304] 0 2024-03-21T10:08:42.307 INFO:tasks.workunit.client.0.smithi153.stdout:5/147: chown d4/d5/l31 29222 1 2024-03-21T10:08:42.315 INFO:tasks.workunit.client.1.smithi174.stdout:2/43: dwrite d5/f12 [0,4194304] 0 2024-03-21T10:08:42.316 INFO:tasks.workunit.client.0.smithi153.stdout:1/151: write d6/fb [3287431,129614] 0 2024-03-21T10:08:42.318 INFO:tasks.workunit.client.0.smithi153.stdout:1/152: truncate d6/f16 950392 0 2024-03-21T10:08:42.320 INFO:tasks.workunit.client.1.smithi174.stdout:9/34: rmdir d2 39 2024-03-21T10:08:42.325 INFO:tasks.workunit.client.1.smithi174.stdout:5/26: dwrite d1/f6 [0,4194304] 0 2024-03-21T10:08:42.329 INFO:tasks.workunit.client.1.smithi174.stdout:1/26: rename l2 to d4/d5/l6 0 2024-03-21T10:08:42.331 INFO:tasks.workunit.client.1.smithi174.stdout:2/44: stat c0 0 2024-03-21T10:08:42.333 INFO:tasks.workunit.client.1.smithi174.stdout:1/27: mkdir d4/d7 0 2024-03-21T10:08:42.337 INFO:tasks.workunit.client.1.smithi174.stdout:9/35: dread d2/f3 [4194304,4194304] 0 2024-03-21T10:08:42.338 INFO:tasks.workunit.client.1.smithi174.stdout:2/45: mknod d5/c13 0 2024-03-21T10:08:42.348 INFO:tasks.workunit.client.0.smithi153.stdout:5/148: dread d4/d8/f1d [0,4194304] 0 2024-03-21T10:08:42.352 INFO:tasks.workunit.client.0.smithi153.stdout:5/149: rename d4/d5/df/d32 to d4/d5/de/d2f/d33 0 2024-03-21T10:08:42.354 INFO:tasks.workunit.client.0.smithi153.stdout:5/150: write d4/d5/de/f2a [871895,105902] 0 2024-03-21T10:08:42.356 INFO:tasks.workunit.client.0.smithi153.stdout:5/151: rmdir d4/d8 39 2024-03-21T10:08:42.358 INFO:tasks.workunit.client.1.smithi174.stdout:4/58: rmdir d4 39 2024-03-21T10:08:42.358 INFO:tasks.workunit.client.0.smithi153.stdout:5/152: creat d4/d5/df/f34 x:0 0 0 2024-03-21T10:08:42.364 INFO:tasks.workunit.client.0.smithi153.stdout:5/153: dread d4/d5/de/f2a [0,4194304] 0 2024-03-21T10:08:42.364 INFO:tasks.workunit.client.0.smithi153.stdout:7/117: dread f2 [4194304,4194304] 0 2024-03-21T10:08:42.367 INFO:tasks.workunit.client.0.smithi153.stdout:5/154: write d4/d8/f2d [109125,104303] 0 2024-03-21T10:08:42.373 INFO:tasks.workunit.client.0.smithi153.stdout:5/155: truncate d4/d5/f1a 228950 0 2024-03-21T10:08:42.374 INFO:tasks.workunit.client.0.smithi153.stdout:5/156: rmdir d4/d5/d16 39 2024-03-21T10:08:42.374 INFO:tasks.workunit.client.1.smithi174.stdout:4/59: symlink d4/l16 0 2024-03-21T10:08:42.375 INFO:tasks.workunit.client.0.smithi153.stdout:5/157: symlink d4/l35 0 2024-03-21T10:08:42.378 INFO:tasks.workunit.client.1.smithi174.stdout:1/28: dwrite f1 [0,4194304] 0 2024-03-21T10:08:42.378 INFO:tasks.workunit.client.0.smithi153.stdout:5/158: creat d4/d5/de/f36 x:0 0 0 2024-03-21T10:08:42.379 INFO:tasks.workunit.client.1.smithi174.stdout:9/36: dwrite d2/f4 [0,4194304] 0 2024-03-21T10:08:42.379 INFO:tasks.workunit.client.0.smithi153.stdout:7/118: dwrite d4/f10 [0,4194304] 0 2024-03-21T10:08:42.379 INFO:tasks.workunit.client.1.smithi174.stdout:4/60: write d4/f8 [5116384,29077] 0 2024-03-21T10:08:42.380 INFO:tasks.workunit.client.0.smithi153.stdout:7/119: write f2 [8164663,18548] 0 2024-03-21T10:08:42.382 INFO:tasks.workunit.client.0.smithi153.stdout:7/120: chown d4/d1b/d20 1469 1 2024-03-21T10:08:42.383 INFO:tasks.workunit.client.1.smithi174.stdout:2/46: dwrite d5/f11 [0,4194304] 0 2024-03-21T10:08:42.384 INFO:tasks.workunit.client.0.smithi153.stdout:7/121: write d4/f12 [827983,7338] 0 2024-03-21T10:08:42.384 INFO:tasks.workunit.client.0.smithi153.stdout:5/159: symlink d4/d5/de/d2f/l37 0 2024-03-21T10:08:42.388 INFO:tasks.workunit.client.1.smithi174.stdout:4/61: rmdir d4 39 2024-03-21T10:08:42.388 INFO:tasks.workunit.client.0.smithi153.stdout:7/122: unlink f2 0 2024-03-21T10:08:42.389 INFO:tasks.workunit.client.0.smithi153.stdout:5/160: readlink d4/d5/l31 0 2024-03-21T10:08:42.391 INFO:tasks.workunit.client.0.smithi153.stdout:5/161: mkdir d4/d5/de/d38 0 2024-03-21T10:08:42.393 INFO:tasks.workunit.client.0.smithi153.stdout:7/123: creat d4/f25 x:0 0 0 2024-03-21T10:08:42.395 INFO:tasks.workunit.client.0.smithi153.stdout:5/162: mkdir d4/d5/d16/d24/d39 0 2024-03-21T10:08:42.395 INFO:tasks.workunit.client.1.smithi174.stdout:2/47: rmdir d5/d8 39 2024-03-21T10:08:42.398 INFO:tasks.workunit.client.0.smithi153.stdout:7/124: rename d4/d11/c1d to d4/d11/d24/c26 0 2024-03-21T10:08:42.399 INFO:tasks.workunit.client.0.smithi153.stdout:5/163: unlink d4/d5/de/c15 0 2024-03-21T10:08:42.401 INFO:tasks.workunit.client.0.smithi153.stdout:7/125: truncate d4/f12 1362499 0 2024-03-21T10:08:42.407 INFO:tasks.workunit.client.0.smithi153.stdout:7/126: dread d4/fa [0,4194304] 0 2024-03-21T10:08:42.408 INFO:tasks.workunit.client.1.smithi174.stdout:4/62: getdents d4 0 2024-03-21T10:08:42.413 INFO:tasks.workunit.client.0.smithi153.stdout:7/127: dread d4/f1a [0,4194304] 0 2024-03-21T10:08:42.415 INFO:tasks.workunit.client.0.smithi153.stdout:7/128: mknod d4/d1b/d20/c27 0 2024-03-21T10:08:42.416 INFO:tasks.workunit.client.1.smithi174.stdout:2/48: truncate d5/d8/fe 865000 0 2024-03-21T10:08:42.416 INFO:tasks.workunit.client.1.smithi174.stdout:1/29: dwrite f1 [4194304,4194304] 0 2024-03-21T10:08:42.418 INFO:tasks.workunit.client.0.smithi153.stdout:8/146: sync 2024-03-21T10:08:42.418 INFO:tasks.workunit.client.0.smithi153.stdout:2/175: sync 2024-03-21T10:08:42.420 INFO:tasks.workunit.client.1.smithi174.stdout:9/37: dwrite d2/f6 [0,4194304] 0 2024-03-21T10:08:42.420 INFO:tasks.workunit.client.1.smithi174.stdout:9/38: readlink - no filename 2024-03-21T10:08:42.421 INFO:tasks.workunit.client.1.smithi174.stdout:1/30: creat d4/d5/f8 x:0 0 0 2024-03-21T10:08:42.430 INFO:tasks.workunit.client.1.smithi174.stdout:9/39: symlink d2/ld 0 2024-03-21T10:08:42.430 INFO:tasks.workunit.client.1.smithi174.stdout:2/49: link d5/f11 d5/f14 0 2024-03-21T10:08:42.433 INFO:tasks.workunit.client.1.smithi174.stdout:4/63: dwrite f0 [0,4194304] 0 2024-03-21T10:08:42.436 INFO:tasks.workunit.client.0.smithi153.stdout:7/129: dwrite d4/fa [0,4194304] 0 2024-03-21T10:08:42.440 INFO:tasks.workunit.client.0.smithi153.stdout:8/147: dwrite d5/f8 [0,4194304] 0 2024-03-21T10:08:42.443 INFO:tasks.workunit.client.0.smithi153.stdout:7/130: creat d4/d11/d24/f28 x:0 0 0 2024-03-21T10:08:42.443 INFO:tasks.workunit.client.0.smithi153.stdout:2/176: dwrite dd/d15/d18/f38 [0,4194304] 0 2024-03-21T10:08:42.446 INFO:tasks.workunit.client.1.smithi174.stdout:2/50: truncate d5/f10 416649 0 2024-03-21T10:08:42.446 INFO:tasks.workunit.client.1.smithi174.stdout:2/51: symlink d5/d8/l15 0 2024-03-21T10:08:42.446 INFO:tasks.workunit.client.1.smithi174.stdout:2/52: dread d5/f10 [0,4194304] 0 2024-03-21T10:08:42.447 INFO:tasks.workunit.client.0.smithi153.stdout:2/177: truncate dd/d15/d18/d2c/d33/f36 751617 0 2024-03-21T10:08:42.448 INFO:tasks.workunit.client.0.smithi153.stdout:2/178: fsync dd/f13 0 2024-03-21T10:08:42.452 INFO:tasks.workunit.client.1.smithi174.stdout:4/64: getdents d4 0 2024-03-21T10:08:42.454 INFO:tasks.workunit.client.1.smithi174.stdout:1/31: fdatasync f1 0 2024-03-21T10:08:42.459 INFO:tasks.workunit.client.1.smithi174.stdout:4/65: symlink d4/l17 0 2024-03-21T10:08:42.463 INFO:tasks.workunit.client.0.smithi153.stdout:8/148: dread d5/d1b/f18 [0,4194304] 0 2024-03-21T10:08:42.464 INFO:tasks.workunit.client.1.smithi174.stdout:4/66: creat d4/f18 x:0 0 0 2024-03-21T10:08:42.466 INFO:tasks.workunit.client.1.smithi174.stdout:4/67: fsync d4/ff 0 2024-03-21T10:08:42.467 INFO:tasks.workunit.client.1.smithi174.stdout:2/53: rename d5/d8/cb to d5/c16 0 2024-03-21T10:08:42.467 INFO:tasks.workunit.client.0.smithi153.stdout:8/149: write d5/f8 [3326788,79856] 0 2024-03-21T10:08:42.467 INFO:tasks.workunit.client.1.smithi174.stdout:1/32: getdents d4/d7 0 2024-03-21T10:08:42.468 INFO:tasks.workunit.client.1.smithi174.stdout:1/33: chown d4 2815 1 2024-03-21T10:08:42.470 INFO:tasks.workunit.client.1.smithi174.stdout:4/68: fsync d4/f15 0 2024-03-21T10:08:42.470 INFO:tasks.workunit.client.1.smithi174.stdout:1/34: read - d4/d5/f8 zero size 2024-03-21T10:08:42.471 INFO:tasks.workunit.client.0.smithi153.stdout:8/150: symlink d5/d9/dc/dd/d20/l2a 0 2024-03-21T10:08:42.471 INFO:tasks.workunit.client.1.smithi174.stdout:4/69: fdatasync d4/ff 0 2024-03-21T10:08:42.472 INFO:tasks.workunit.client.0.smithi153.stdout:2/179: dwrite dd/f37 [0,4194304] 0 2024-03-21T10:08:42.472 INFO:tasks.workunit.client.0.smithi153.stdout:8/151: fdatasync d5/f14 0 2024-03-21T10:08:42.473 INFO:tasks.workunit.client.0.smithi153.stdout:8/152: readlink d5/d9/dc/lf 0 2024-03-21T10:08:42.474 INFO:tasks.workunit.client.1.smithi174.stdout:4/70: fsync d4/ff 0 2024-03-21T10:08:42.475 INFO:tasks.workunit.client.1.smithi174.stdout:1/35: creat d4/d5/f9 x:0 0 0 2024-03-21T10:08:42.475 INFO:tasks.workunit.client.0.smithi153.stdout:8/153: chown d5/d1b/c1e 12736 1 2024-03-21T10:08:42.475 INFO:tasks.workunit.client.0.smithi153.stdout:2/180: unlink dd/d15/l1b 0 2024-03-21T10:08:42.477 INFO:tasks.workunit.client.1.smithi174.stdout:4/71: mkdir d4/d19 0 2024-03-21T10:08:42.477 INFO:tasks.workunit.client.1.smithi174.stdout:1/36: truncate d4/d5/f8 679996 0 2024-03-21T10:08:42.478 INFO:tasks.workunit.client.0.smithi153.stdout:2/181: write dd/d15/d18/d2c/f2d [1499777,54553] 0 2024-03-21T10:08:42.481 INFO:tasks.workunit.client.0.smithi153.stdout:2/182: mkdir dd/d3b 0 2024-03-21T10:08:42.484 INFO:tasks.workunit.client.0.smithi153.stdout:2/183: mkdir dd/d15/d3c 0 2024-03-21T10:08:42.486 INFO:tasks.workunit.client.1.smithi174.stdout:2/54: dread d5/f14 [0,4194304] 0 2024-03-21T10:08:42.487 INFO:tasks.workunit.client.0.smithi153.stdout:2/184: write dd/f2e [1702798,101219] 0 2024-03-21T10:08:42.487 INFO:tasks.workunit.client.1.smithi174.stdout:9/40: truncate d2/f4 2785281 0 2024-03-21T10:08:42.491 INFO:tasks.workunit.client.0.smithi153.stdout:8/154: dwrite f3 [4194304,4194304] 0 2024-03-21T10:08:42.492 INFO:tasks.workunit.client.1.smithi174.stdout:9/41: fdatasync d2/f3 0 2024-03-21T10:08:42.494 INFO:tasks.workunit.client.1.smithi174.stdout:9/42: dread - d2/fc zero size 2024-03-21T10:08:42.497 INFO:tasks.workunit.client.0.smithi153.stdout:8/155: rename d5/f14 to d5/d9/dc/f2b 0 2024-03-21T10:08:42.501 INFO:tasks.workunit.client.1.smithi174.stdout:2/55: link d5/f12 d5/d8/f17 0 2024-03-21T10:08:42.504 INFO:tasks.workunit.client.0.smithi153.stdout:8/156: link d5/f8 d5/d9/dc/f2c 0 2024-03-21T10:08:42.504 INFO:tasks.workunit.client.0.smithi153.stdout:2/185: dwrite dd/d15/d18/f24 [0,4194304] 0 2024-03-21T10:08:42.505 INFO:tasks.workunit.client.0.smithi153.stdout:8/157: readlink d5/d9/le 0 2024-03-21T10:08:42.506 INFO:tasks.workunit.client.1.smithi174.stdout:2/56: chown d5/d8/f17 66 1 2024-03-21T10:08:42.507 INFO:tasks.workunit.client.1.smithi174.stdout:1/37: dwrite d4/d5/f9 [0,4194304] 0 2024-03-21T10:08:42.507 INFO:tasks.workunit.client.1.smithi174.stdout:9/43: mknod d2/ce 0 2024-03-21T10:08:42.513 INFO:tasks.workunit.client.1.smithi174.stdout:1/38: chown d4/d5/f8 190990 1 2024-03-21T10:08:42.514 INFO:tasks.workunit.client.1.smithi174.stdout:2/57: fdatasync d5/fa 0 2024-03-21T10:08:42.519 INFO:tasks.workunit.client.1.smithi174.stdout:9/44: rename d2/f6 to d2/ff 0 2024-03-21T10:08:42.524 INFO:tasks.workunit.client.0.smithi153.stdout:8/158: dwrite d5/d9/dc/dd/d20/d26/f23 [0,4194304] 0 2024-03-21T10:08:42.527 INFO:tasks.workunit.client.0.smithi153.stdout:8/159: mkdir d5/d9/dc/dd/d2d 0 2024-03-21T10:08:42.529 INFO:tasks.workunit.client.1.smithi174.stdout:9/45: dread d2/f9 [0,4194304] 0 2024-03-21T10:08:42.531 INFO:tasks.workunit.client.1.smithi174.stdout:2/58: getdents d5 0 2024-03-21T10:08:42.533 INFO:tasks.workunit.client.0.smithi153.stdout:2/186: getdents dd 0 2024-03-21T10:08:42.533 INFO:tasks.workunit.client.0.smithi153.stdout:8/160: rename d5/d1b/c1e to d5/d9/dc/dd/d2d/c2e 0 2024-03-21T10:08:42.533 INFO:tasks.workunit.client.0.smithi153.stdout:2/187: truncate f7 9144084 0 2024-03-21T10:08:42.537 INFO:tasks.workunit.client.1.smithi174.stdout:1/39: fsync d4/d5/f9 0 2024-03-21T10:08:42.541 INFO:tasks.workunit.client.1.smithi174.stdout:1/40: truncate d4/d5/f8 1069116 0 2024-03-21T10:08:42.545 INFO:tasks.workunit.client.1.smithi174.stdout:9/46: dread d2/f9 [0,4194304] 0 2024-03-21T10:08:42.548 INFO:tasks.workunit.client.1.smithi174.stdout:9/47: read - d2/fc zero size 2024-03-21T10:08:42.560 INFO:tasks.workunit.client.1.smithi174.stdout:2/59: dwrite d5/f10 [0,4194304] 0 2024-03-21T10:08:42.565 INFO:tasks.workunit.client.1.smithi174.stdout:2/60: truncate d5/d8/ff 878618 0 2024-03-21T10:08:42.567 INFO:tasks.workunit.client.1.smithi174.stdout:1/41: dwrite d4/d5/f9 [0,4194304] 0 2024-03-21T10:08:42.569 INFO:tasks.workunit.client.1.smithi174.stdout:9/48: dread d2/f3 [0,4194304] 0 2024-03-21T10:08:42.571 INFO:tasks.workunit.client.1.smithi174.stdout:2/61: symlink d5/d8/l18 0 2024-03-21T10:08:42.575 INFO:tasks.workunit.client.1.smithi174.stdout:1/42: rmdir d4 39 2024-03-21T10:08:42.577 INFO:tasks.workunit.client.0.smithi153.stdout:9/160: sync 2024-03-21T10:08:42.587 INFO:tasks.workunit.client.1.smithi174.stdout:1/43: dread f1 [0,4194304] 0 2024-03-21T10:08:42.589 INFO:tasks.workunit.client.1.smithi174.stdout:2/62: dread d5/fa [0,4194304] 0 2024-03-21T10:08:42.593 INFO:tasks.workunit.client.1.smithi174.stdout:9/49: dwrite d2/fc [0,4194304] 0 2024-03-21T10:08:42.593 INFO:tasks.workunit.client.1.smithi174.stdout:2/63: rename c4 to d5/c19 0 2024-03-21T10:08:42.596 INFO:tasks.workunit.client.1.smithi174.stdout:9/50: mkdir d2/d10 0 2024-03-21T10:08:42.599 INFO:tasks.workunit.client.1.smithi174.stdout:2/64: truncate f2 302202 0 2024-03-21T10:08:42.611 INFO:tasks.workunit.client.0.smithi153.stdout:9/161: fdatasync d1/d4/fc 0 2024-03-21T10:08:42.615 INFO:tasks.workunit.client.1.smithi174.stdout:2/65: stat d5/c16 0 2024-03-21T10:08:42.619 INFO:tasks.workunit.client.1.smithi174.stdout:9/51: write d2/f4 [3785649,92831] 0 2024-03-21T10:08:42.621 INFO:tasks.workunit.client.0.smithi153.stdout:9/162: rename d1/d4/f20 to d1/d12/d1f/f33 0 2024-03-21T10:08:42.622 INFO:tasks.workunit.client.1.smithi174.stdout:2/66: mknod d5/d8/c1a 0 2024-03-21T10:08:42.622 INFO:tasks.workunit.client.0.smithi153.stdout:2/188: dwrite dd/d15/d18/d2c/d33/f36 [0,4194304] 0 2024-03-21T10:08:42.625 INFO:tasks.workunit.client.1.smithi174.stdout:9/52: rmdir d2/d10 0 2024-03-21T10:08:42.629 INFO:tasks.workunit.client.0.smithi153.stdout:2/189: mkdir dd/d15/d18/d3a/d3d 0 2024-03-21T10:08:42.631 INFO:tasks.workunit.client.0.smithi153.stdout:2/190: write dd/f2e [5054202,92400] 0 2024-03-21T10:08:42.637 INFO:tasks.workunit.client.0.smithi153.stdout:2/191: creat dd/d15/d18/d3a/f3e x:0 0 0 2024-03-21T10:08:42.638 INFO:tasks.workunit.client.1.smithi174.stdout:2/67: write f2 [273980,63936] 0 2024-03-21T10:08:42.639 INFO:tasks.workunit.client.1.smithi174.stdout:9/53: dread d2/fc [0,4194304] 0 2024-03-21T10:08:42.640 INFO:tasks.workunit.client.0.smithi153.stdout:2/192: creat dd/d15/d18/d2c/f3f x:0 0 0 2024-03-21T10:08:42.641 INFO:tasks.workunit.client.1.smithi174.stdout:2/68: dread f2 [0,4194304] 0 2024-03-21T10:08:42.644 INFO:tasks.workunit.client.1.smithi174.stdout:2/69: symlink d5/l1b 0 2024-03-21T10:08:42.647 INFO:tasks.workunit.client.0.smithi153.stdout:2/193: link dd/d15/d18/f24 dd/d15/f40 0 2024-03-21T10:08:42.648 INFO:tasks.workunit.client.1.smithi174.stdout:2/70: read d5/f12 [430927,57831] 0 2024-03-21T10:08:42.648 INFO:tasks.workunit.client.1.smithi174.stdout:9/54: truncate d2/f4 2335656 0 2024-03-21T10:08:42.649 INFO:tasks.workunit.client.1.smithi174.stdout:9/55: rmdir d2 39 2024-03-21T10:08:42.649 INFO:tasks.workunit.client.0.smithi153.stdout:9/163: dwrite d1/d12/d1c/d24/f32 [0,4194304] 0 2024-03-21T10:08:42.650 INFO:tasks.workunit.client.1.smithi174.stdout:2/71: write d5/f10 [1806097,98106] 0 2024-03-21T10:08:42.654 INFO:tasks.workunit.client.0.smithi153.stdout:2/194: fsync dd/d15/d18/f2a 0 2024-03-21T10:08:42.655 INFO:tasks.workunit.client.0.smithi153.stdout:2/195: readlink dd/l23 0 2024-03-21T10:08:42.656 INFO:tasks.workunit.client.1.smithi174.stdout:9/56: mkdir d2/d11 0 2024-03-21T10:08:42.658 INFO:tasks.workunit.client.1.smithi174.stdout:2/72: chown d5/f11 1 1 2024-03-21T10:08:42.658 INFO:tasks.workunit.client.0.smithi153.stdout:2/196: creat dd/d15/d18/d2c/f41 x:0 0 0 2024-03-21T10:08:42.663 INFO:tasks.workunit.client.1.smithi174.stdout:2/73: creat d5/d8/f1c x:0 0 0 2024-03-21T10:08:42.668 INFO:tasks.workunit.client.0.smithi153.stdout:2/197: link dd/d15/l21 dd/d15/d18/d3a/d3d/l42 0 2024-03-21T10:08:42.668 INFO:tasks.workunit.client.1.smithi174.stdout:2/74: creat d5/d8/f1d x:0 0 0 2024-03-21T10:08:42.670 INFO:tasks.workunit.client.0.smithi153.stdout:2/198: chown fc 0 1 2024-03-21T10:08:42.671 INFO:tasks.workunit.client.1.smithi174.stdout:2/75: mkdir d5/d8/d1e 0 2024-03-21T10:08:42.674 INFO:tasks.workunit.client.1.smithi174.stdout:2/76: chown d5/c7 960 1 2024-03-21T10:08:42.674 INFO:tasks.workunit.client.0.smithi153.stdout:2/199: creat dd/d15/d18/d2c/d33/f43 x:0 0 0 2024-03-21T10:08:42.675 INFO:tasks.workunit.client.1.smithi174.stdout:2/77: stat d5/d8/lc 0 2024-03-21T10:08:42.677 INFO:tasks.workunit.client.0.smithi153.stdout:2/200: mknod dd/d15/d18/c44 0 2024-03-21T10:08:42.678 INFO:tasks.workunit.client.1.smithi174.stdout:9/57: dwrite d2/f9 [0,4194304] 0 2024-03-21T10:08:42.694 INFO:tasks.workunit.client.1.smithi174.stdout:2/78: dwrite d5/fa [0,4194304] 0 2024-03-21T10:08:42.694 INFO:tasks.workunit.client.1.smithi174.stdout:9/58: chown d2/f4 5 1 2024-03-21T10:08:42.697 INFO:tasks.workunit.client.1.smithi174.stdout:2/79: symlink d5/l1f 0 2024-03-21T10:08:42.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:42 smithi174 ceph-mon[37496]: from='client.14704 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:08:42.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:42 smithi174 ceph-mon[37496]: pgmap v7: 65 pgs: 65 active+clean; 369 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.8 MiB/s rd, 36 MiB/s wr, 150 op/s 2024-03-21T10:08:42.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:42 smithi174 ceph-mon[37496]: from='client.? 172.21.15.153:0/1262239475' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:08:42.703 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:42 smithi174 ceph-mon[37496]: from='client.14710 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:08:42.703 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:42 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:08:42.703 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:42 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:08:42.704 INFO:tasks.workunit.client.0.smithi153.stdout:2/201: dwrite dd/d15/f40 [0,4194304] 0 2024-03-21T10:08:42.705 INFO:tasks.workunit.client.1.smithi174.stdout:9/59: dread d2/ff [0,4194304] 0 2024-03-21T10:08:42.706 INFO:tasks.workunit.client.0.smithi153.stdout:2/202: chown f7 134026179 1 2024-03-21T10:08:42.708 INFO:tasks.workunit.client.1.smithi174.stdout:9/60: write d2/ff [1312843,72861] 0 2024-03-21T10:08:42.708 INFO:tasks.workunit.client.0.smithi153.stdout:2/203: truncate dd/f13 2003452 0 2024-03-21T10:08:42.721 INFO:tasks.workunit.client.1.smithi174.stdout:2/80: dwrite d5/f11 [4194304,4194304] 0 2024-03-21T10:08:42.725 INFO:tasks.workunit.client.1.smithi174.stdout:9/61: dread d2/f3 [0,4194304] 0 2024-03-21T10:08:42.728 INFO:tasks.workunit.client.0.smithi153.stdout:2/204: dwrite dd/d15/d18/f1a [0,4194304] 0 2024-03-21T10:08:42.728 INFO:tasks.workunit.client.1.smithi174.stdout:9/62: write d2/fc [1036942,34931] 0 2024-03-21T10:08:42.730 INFO:tasks.workunit.client.1.smithi174.stdout:9/63: fdatasync d2/ff 0 2024-03-21T10:08:42.731 INFO:tasks.workunit.client.1.smithi174.stdout:9/64: chown d2/c7 18 1 2024-03-21T10:08:42.739 INFO:tasks.workunit.client.1.smithi174.stdout:2/81: dread d5/f14 [0,4194304] 0 2024-03-21T10:08:42.743 INFO:tasks.workunit.client.1.smithi174.stdout:2/82: mknod d5/c20 0 2024-03-21T10:08:42.744 INFO:tasks.workunit.client.1.smithi174.stdout:2/83: chown d5/f14 417128149 1 2024-03-21T10:08:42.745 INFO:tasks.workunit.client.1.smithi174.stdout:9/65: truncate d2/f9 269372 0 2024-03-21T10:08:42.746 INFO:tasks.workunit.client.1.smithi174.stdout:2/84: chown d5/d8/l15 2779032 1 2024-03-21T10:08:42.747 INFO:tasks.workunit.client.0.smithi153.stdout:2/205: dread dd/d15/d18/d2c/d33/f36 [0,4194304] 0 2024-03-21T10:08:42.748 INFO:tasks.workunit.client.1.smithi174.stdout:9/66: getdents d2/d11 0 2024-03-21T10:08:42.750 INFO:tasks.workunit.client.1.smithi174.stdout:2/85: unlink d5/c19 0 2024-03-21T10:08:42.751 INFO:tasks.workunit.client.0.smithi153.stdout:2/206: creat dd/d15/d18/f45 x:0 0 0 2024-03-21T10:08:42.756 INFO:tasks.workunit.client.1.smithi174.stdout:2/86: link d5/fa d5/d8/d1e/f21 0 2024-03-21T10:08:42.757 INFO:tasks.workunit.client.1.smithi174.stdout:9/67: dread d2/f3 [0,4194304] 0 2024-03-21T10:08:42.759 INFO:tasks.workunit.client.0.smithi153.stdout:2/207: rename dd/d15/c2b to dd/c46 0 2024-03-21T10:08:42.761 INFO:tasks.workunit.client.1.smithi174.stdout:9/68: unlink d2/ce 0 2024-03-21T10:08:42.762 INFO:tasks.workunit.client.0.smithi153.stdout:2/208: fsync fc 0 2024-03-21T10:08:42.763 INFO:tasks.workunit.client.0.smithi153.stdout:2/209: chown dd/d15/l27 5 1 2024-03-21T10:08:42.764 INFO:tasks.workunit.client.1.smithi174.stdout:9/69: mknod d2/d11/c12 0 2024-03-21T10:08:42.765 INFO:tasks.workunit.client.0.smithi153.stdout:2/210: write dd/d15/d18/f1a [3991915,66062] 0 2024-03-21T10:08:42.769 INFO:tasks.workunit.client.0.smithi153.stdout:2/211: rmdir dd/d15/d18/d3a/d3d 39 2024-03-21T10:08:42.771 INFO:tasks.workunit.client.1.smithi174.stdout:2/87: dwrite d5/f10 [0,4194304] 0 2024-03-21T10:08:42.772 INFO:tasks.workunit.client.0.smithi153.stdout:2/212: symlink dd/d15/d18/d2c/l47 0 2024-03-21T10:08:42.773 INFO:tasks.workunit.client.0.smithi153.stdout:2/213: chown dd/d15 50752814 1 2024-03-21T10:08:42.777 INFO:tasks.workunit.client.0.smithi153.stdout:2/214: getdents dd 0 2024-03-21T10:08:42.778 INFO:tasks.workunit.client.0.smithi153.stdout:0/125: sync 2024-03-21T10:08:42.778 INFO:tasks.workunit.client.0.smithi153.stdout:6/384: sync 2024-03-21T10:08:42.782 INFO:tasks.workunit.client.0.smithi153.stdout:2/215: symlink dd/d15/l48 0 2024-03-21T10:08:42.782 INFO:tasks.workunit.client.0.smithi153.stdout:0/126: write d4/d16/d1f/f20 [1227859,19061] 0 2024-03-21T10:08:42.783 INFO:tasks.workunit.client.1.smithi174.stdout:9/70: getdents d2 0 2024-03-21T10:08:42.785 INFO:tasks.workunit.client.0.smithi153.stdout:2/216: mknod dd/d15/d3c/c49 0 2024-03-21T10:08:42.788 INFO:tasks.workunit.client.0.smithi153.stdout:6/385: getdents d1f/d2b/d33/d40/d65 0 2024-03-21T10:08:42.788 INFO:tasks.workunit.client.0.smithi153.stdout:0/127: read d4/d16/d1f/f20 [664076,71094] 0 2024-03-21T10:08:42.790 INFO:tasks.workunit.client.0.smithi153.stdout:2/217: creat dd/d15/d18/d3a/d3d/f4a x:0 0 0 2024-03-21T10:08:42.791 INFO:tasks.workunit.client.0.smithi153.stdout:0/128: mkdir d4/d16/d27 0 2024-03-21T10:08:42.793 INFO:tasks.workunit.client.0.smithi153.stdout:6/386: creat d1f/d2b/d33/d40/d5f/f8b x:0 0 0 2024-03-21T10:08:42.794 INFO:tasks.workunit.client.0.smithi153.stdout:6/387: chown d1f/f6c 23 1 2024-03-21T10:08:42.794 INFO:tasks.workunit.client.0.smithi153.stdout:2/218: mkdir dd/d15/d18/d2c/d4b 0 2024-03-21T10:08:42.798 INFO:tasks.workunit.client.0.smithi153.stdout:6/388: truncate d1f/d2b/f53 622060 0 2024-03-21T10:08:42.804 INFO:tasks.workunit.client.0.smithi153.stdout:0/129: dread d4/f10 [0,4194304] 0 2024-03-21T10:08:42.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:42 smithi153 ceph-mon[29117]: from='client.14704 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:08:42.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:42 smithi153 ceph-mon[29117]: pgmap v7: 65 pgs: 65 active+clean; 369 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.8 MiB/s rd, 36 MiB/s wr, 150 op/s 2024-03-21T10:08:42.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:42 smithi153 ceph-mon[29117]: from='client.? 172.21.15.153:0/1262239475' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:08:42.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:42 smithi153 ceph-mon[29117]: from='client.14710 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:08:42.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:42 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:08:42.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:42 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:08:42.809 INFO:tasks.workunit.client.0.smithi153.stdout:2/219: dwrite dd/d15/d18/f38 [0,4194304] 0 2024-03-21T10:08:42.810 INFO:tasks.workunit.client.0.smithi153.stdout:0/130: write d4/f10 [681507,19677] 0 2024-03-21T10:08:42.814 INFO:tasks.workunit.client.0.smithi153.stdout:0/131: unlink d4/l18 0 2024-03-21T10:08:42.815 INFO:tasks.workunit.client.0.smithi153.stdout:2/220: creat dd/d3b/f4c x:0 0 0 2024-03-21T10:08:42.815 INFO:tasks.workunit.client.0.smithi153.stdout:0/132: dread - d4/d16/d1f/f24 zero size 2024-03-21T10:08:42.817 INFO:tasks.workunit.client.0.smithi153.stdout:0/133: chown d4/d16/d1f/d22/c26 53331332 1 2024-03-21T10:08:42.825 INFO:tasks.workunit.client.0.smithi153.stdout:2/221: getdents dd/d3b 0 2024-03-21T10:08:42.828 INFO:tasks.workunit.client.0.smithi153.stdout:0/134: dread d4/db/f11 [0,4194304] 0 2024-03-21T10:08:42.832 INFO:tasks.workunit.client.0.smithi153.stdout:0/135: link d4/f6 d4/f28 0 2024-03-21T10:08:42.834 INFO:tasks.workunit.client.0.smithi153.stdout:0/136: write d4/db/f1b [167872,2820] 0 2024-03-21T10:08:42.837 INFO:tasks.workunit.client.0.smithi153.stdout:0/137: fsync d4/d16/d1f/f20 0 2024-03-21T10:08:42.840 INFO:tasks.workunit.client.0.smithi153.stdout:2/222: dwrite dd/d15/d18/d3a/d3d/f4a [0,4194304] 0 2024-03-21T10:08:42.840 INFO:tasks.workunit.client.0.smithi153.stdout:0/138: read d4/db/f1b [145566,40730] 0 2024-03-21T10:08:42.843 INFO:tasks.workunit.client.0.smithi153.stdout:0/139: creat d4/db/f29 x:0 0 0 2024-03-21T10:08:42.847 INFO:tasks.workunit.client.0.smithi153.stdout:2/223: rename dd/d15/d18/d3a/d3d/f4a to dd/d15/d18/d2c/d4b/f4d 0 2024-03-21T10:08:42.849 INFO:tasks.workunit.client.0.smithi153.stdout:0/140: symlink d4/d16/d1f/l2a 0 2024-03-21T10:08:42.855 INFO:tasks.workunit.client.0.smithi153.stdout:0/141: rename d4/d16/d1f/l17 to d4/d16/d19/l2b 0 2024-03-21T10:08:42.856 INFO:tasks.workunit.client.0.smithi153.stdout:0/142: fsync d4/d16/d1f/f20 0 2024-03-21T10:08:42.857 INFO:tasks.workunit.client.0.smithi153.stdout:0/143: chown d4 0 1 2024-03-21T10:08:42.858 INFO:tasks.workunit.client.0.smithi153.stdout:2/224: dread dd/d15/d18/f1a [0,4194304] 0 2024-03-21T10:08:42.859 INFO:tasks.workunit.client.0.smithi153.stdout:0/144: mknod d4/d16/d19/c2c 0 2024-03-21T10:08:42.861 INFO:tasks.workunit.client.0.smithi153.stdout:2/225: truncate dd/d15/d18/d2c/f39 138557 0 2024-03-21T10:08:42.861 INFO:tasks.workunit.client.0.smithi153.stdout:0/145: mknod d4/d16/d1f/c2d 0 2024-03-21T10:08:42.862 INFO:tasks.workunit.client.1.smithi174.stdout:9/71: mknod d2/c13 0 2024-03-21T10:08:42.865 INFO:tasks.workunit.client.0.smithi153.stdout:2/226: getdents dd/d15/d18/d3a/d3d 0 2024-03-21T10:08:42.866 INFO:tasks.workunit.client.1.smithi174.stdout:9/72: mknod d2/d11/c14 0 2024-03-21T10:08:42.868 INFO:tasks.workunit.client.0.smithi153.stdout:2/227: chown dd/d15/d18/d3a/d3d/l42 1007073107 1 2024-03-21T10:08:42.870 INFO:tasks.workunit.client.0.smithi153.stdout:2/228: creat dd/d15/d18/d2c/f4e x:0 0 0 2024-03-21T10:08:42.871 INFO:tasks.workunit.client.0.smithi153.stdout:0/146: dread d4/f6 [0,4194304] 0 2024-03-21T10:08:42.873 INFO:tasks.workunit.client.0.smithi153.stdout:2/229: creat dd/d15/d18/d3a/d3d/f4f x:0 0 0 2024-03-21T10:08:42.875 INFO:tasks.workunit.client.0.smithi153.stdout:0/147: link d4/f28 d4/d16/d1f/d22/f2e 0 2024-03-21T10:08:42.876 INFO:tasks.workunit.client.0.smithi153.stdout:0/148: chown d4/d16/d1f/f24 2036 1 2024-03-21T10:08:42.877 INFO:tasks.workunit.client.0.smithi153.stdout:2/230: getdents dd/d15/d18/d2c/d4b 0 2024-03-21T10:08:42.878 INFO:tasks.workunit.client.0.smithi153.stdout:2/231: chown dd/d15/d18/l28 32346 1 2024-03-21T10:08:42.879 INFO:tasks.workunit.client.0.smithi153.stdout:0/149: symlink d4/d16/d19/l2f 0 2024-03-21T10:08:42.880 INFO:tasks.workunit.client.0.smithi153.stdout:2/232: readlink dd/d15/d18/d3a/d3d/l42 0 2024-03-21T10:08:42.880 INFO:tasks.workunit.client.0.smithi153.stdout:0/150: dread - d4/db/f29 zero size 2024-03-21T10:08:42.880 INFO:tasks.workunit.client.0.smithi153.stdout:2/233: dread - dd/d3b/f4c zero size 2024-03-21T10:08:42.882 INFO:tasks.workunit.client.0.smithi153.stdout:0/151: write d4/db/f1b [581490,63567] 0 2024-03-21T10:08:42.883 INFO:tasks.workunit.client.0.smithi153.stdout:2/234: mkdir dd/d3b/d50 0 2024-03-21T10:08:42.884 INFO:tasks.workunit.client.0.smithi153.stdout:0/152: chown d4/d16/l21 1832 1 2024-03-21T10:08:42.886 INFO:tasks.workunit.client.0.smithi153.stdout:0/153: mkdir d4/d16/d19/d30 0 2024-03-21T10:08:42.893 INFO:tasks.workunit.client.0.smithi153.stdout:2/235: dread dd/f13 [0,4194304] 0 2024-03-21T10:08:42.894 INFO:tasks.workunit.client.0.smithi153.stdout:2/236: rmdir dd 39 2024-03-21T10:08:42.895 INFO:tasks.workunit.client.0.smithi153.stdout:2/237: readlink dd/l11 0 2024-03-21T10:08:42.897 INFO:tasks.workunit.client.0.smithi153.stdout:2/238: unlink dd/d15/d18/l34 0 2024-03-21T10:08:42.898 INFO:tasks.workunit.client.0.smithi153.stdout:2/239: truncate f5 2178011 0 2024-03-21T10:08:42.903 INFO:tasks.workunit.client.0.smithi153.stdout:2/240: rename dd/d15/d18/f38 to dd/d15/d18/d2c/d33/f51 0 2024-03-21T10:08:42.912 INFO:tasks.workunit.client.0.smithi153.stdout:1/153: sync 2024-03-21T10:08:42.913 INFO:tasks.workunit.client.0.smithi153.stdout:4/260: sync 2024-03-21T10:08:42.913 INFO:tasks.workunit.client.0.smithi153.stdout:3/203: sync 2024-03-21T10:08:42.914 INFO:tasks.workunit.client.0.smithi153.stdout:4/261: dread - d11/f39 zero size 2024-03-21T10:08:42.916 INFO:tasks.workunit.client.0.smithi153.stdout:3/204: rename f6 to dd/f43 0 2024-03-21T10:08:42.918 INFO:tasks.workunit.client.1.smithi174.stdout:9/73: write d2/f3 [3231724,16025] 0 2024-03-21T10:08:42.919 INFO:tasks.workunit.client.0.smithi153.stdout:1/154: creat d6/d10/f34 x:0 0 0 2024-03-21T10:08:42.920 INFO:tasks.workunit.client.0.smithi153.stdout:2/241: write fc [2606842,63048] 0 2024-03-21T10:08:42.921 INFO:tasks.workunit.client.0.smithi153.stdout:1/155: write d6/fe [3659324,63026] 0 2024-03-21T10:08:42.922 INFO:tasks.workunit.client.0.smithi153.stdout:4/262: rmdir d11/d2d 39 2024-03-21T10:08:42.924 INFO:tasks.workunit.client.0.smithi153.stdout:3/205: rename dd/d1e/c26 to dd/d1e/d2f/c44 0 2024-03-21T10:08:42.924 INFO:tasks.workunit.client.1.smithi174.stdout:9/74: getdents d2 0 2024-03-21T10:08:42.925 INFO:tasks.workunit.client.0.smithi153.stdout:1/156: write d6/f31 [359331,76599] 0 2024-03-21T10:08:42.926 INFO:tasks.workunit.client.0.smithi153.stdout:2/242: creat dd/d15/f52 x:0 0 0 2024-03-21T10:08:42.927 INFO:tasks.workunit.client.0.smithi153.stdout:1/157: write d6/fa [3877679,107796] 0 2024-03-21T10:08:42.929 INFO:tasks.workunit.client.0.smithi153.stdout:1/158: chown d6/d13/f1d 4286 1 2024-03-21T10:08:42.929 INFO:tasks.workunit.client.1.smithi174.stdout:9/75: link d2/ld d2/l15 0 2024-03-21T10:08:42.930 INFO:tasks.workunit.client.1.smithi174.stdout:9/76: chown d2/c13 217261062 1 2024-03-21T10:08:42.931 INFO:tasks.workunit.client.0.smithi153.stdout:4/263: mkdir d11/d18/d54 0 2024-03-21T10:08:42.933 INFO:tasks.workunit.client.0.smithi153.stdout:4/264: creat d11/d18/f55 x:0 0 0 2024-03-21T10:08:42.934 INFO:tasks.workunit.client.1.smithi174.stdout:9/77: symlink d2/d11/l16 0 2024-03-21T10:08:42.936 INFO:tasks.workunit.client.0.smithi153.stdout:3/206: chown dd/d1e/d2f/c44 18809 1 2024-03-21T10:08:42.943 INFO:tasks.workunit.client.0.smithi153.stdout:1/159: dread f5 [0,4194304] 0 2024-03-21T10:08:42.946 INFO:tasks.workunit.client.0.smithi153.stdout:1/160: getdents d6/d32 0 2024-03-21T10:08:42.947 INFO:tasks.workunit.client.0.smithi153.stdout:2/243: dwrite fb [0,4194304] 0 2024-03-21T10:08:42.949 INFO:tasks.workunit.client.1.smithi174.stdout:9/78: dwrite d2/f4 [0,4194304] 0 2024-03-21T10:08:42.951 INFO:tasks.workunit.client.0.smithi153.stdout:2/244: write dd/f37 [3348048,77543] 0 2024-03-21T10:08:42.952 INFO:tasks.workunit.client.0.smithi153.stdout:1/161: write d6/d10/d2c/f33 [844853,96564] 0 2024-03-21T10:08:42.953 INFO:tasks.workunit.client.1.smithi174.stdout:9/79: write d2/f8 [4306651,16876] 0 2024-03-21T10:08:42.955 INFO:tasks.workunit.client.0.smithi153.stdout:1/162: truncate d6/d13/f1f 445926 0 2024-03-21T10:08:42.956 INFO:tasks.workunit.client.0.smithi153.stdout:2/245: creat dd/d15/d18/d3a/d3d/f53 x:0 0 0 2024-03-21T10:08:42.957 INFO:tasks.workunit.client.1.smithi174.stdout:9/80: stat d2/cb 0 2024-03-21T10:08:42.957 INFO:tasks.workunit.client.0.smithi153.stdout:3/207: dwrite dd/d1e/f30 [0,4194304] 0 2024-03-21T10:08:42.961 INFO:tasks.workunit.client.0.smithi153.stdout:2/246: mknod dd/d15/d18/d2c/d4b/c54 0 2024-03-21T10:08:42.961 INFO:tasks.workunit.client.0.smithi153.stdout:3/208: mkdir dd/d1e/d45 0 2024-03-21T10:08:42.964 INFO:tasks.workunit.client.0.smithi153.stdout:3/209: readlink dd/d1e/d2f/d35/l40 0 2024-03-21T10:08:42.964 INFO:tasks.workunit.client.0.smithi153.stdout:4/265: dread d11/d18/f4c [4194304,4194304] 0 2024-03-21T10:08:42.973 INFO:tasks.workunit.client.0.smithi153.stdout:3/210: link dd/d1e/d2f/f3a dd/d1e/f46 0 2024-03-21T10:08:42.980 INFO:tasks.workunit.client.0.smithi153.stdout:4/266: getdents d11/d2d 0 2024-03-21T10:08:42.980 INFO:tasks.workunit.client.0.smithi153.stdout:3/211: mkdir dd/d41/d47 0 2024-03-21T10:08:42.981 INFO:tasks.workunit.client.1.smithi174.stdout:9/81: truncate d2/f4 1889267 0 2024-03-21T10:08:42.985 INFO:tasks.workunit.client.0.smithi153.stdout:3/212: fdatasync dd/f12 0 2024-03-21T10:08:42.986 INFO:tasks.workunit.client.1.smithi174.stdout:9/82: unlink d2/f8 0 2024-03-21T10:08:42.986 INFO:tasks.workunit.client.0.smithi153.stdout:4/267: chown d11/d18/f1f 97181560 1 2024-03-21T10:08:42.986 INFO:tasks.workunit.client.0.smithi153.stdout:1/163: dwrite d6/d10/f34 [0,4194304] 0 2024-03-21T10:08:42.987 INFO:tasks.workunit.client.1.smithi174.stdout:9/83: creat d2/f17 x:0 0 0 2024-03-21T10:08:42.989 INFO:tasks.workunit.client.0.smithi153.stdout:3/213: creat dd/f48 x:0 0 0 2024-03-21T10:08:42.989 INFO:tasks.workunit.client.0.smithi153.stdout:2/247: dwrite dd/d15/d18/d2c/f4e [0,4194304] 0 2024-03-21T10:08:42.995 INFO:tasks.workunit.client.0.smithi153.stdout:2/248: dread - dd/d15/d18/d2c/f3f zero size 2024-03-21T10:08:42.996 INFO:tasks.workunit.client.0.smithi153.stdout:1/164: fsync d6/d13/f2b 0 2024-03-21T10:08:42.997 INFO:tasks.workunit.client.0.smithi153.stdout:3/214: creat dd/d41/f49 x:0 0 0 2024-03-21T10:08:42.998 INFO:tasks.workunit.client.1.smithi174.stdout:9/84: dread d2/f4 [0,4194304] 0 2024-03-21T10:08:43.002 INFO:tasks.workunit.client.0.smithi153.stdout:1/165: mkdir d6/d10/d35 0 2024-03-21T10:08:43.006 INFO:tasks.workunit.client.0.smithi153.stdout:1/166: write d6/f2f [714585,85585] 0 2024-03-21T10:08:43.006 INFO:tasks.workunit.client.0.smithi153.stdout:2/249: creat dd/d15/d18/d3a/f55 x:0 0 0 2024-03-21T10:08:43.007 INFO:tasks.workunit.client.0.smithi153.stdout:2/250: truncate fc 2893615 0 2024-03-21T10:08:43.007 INFO:tasks.workunit.client.0.smithi153.stdout:3/215: rmdir dd/d41/d47 0 2024-03-21T10:08:43.008 INFO:tasks.workunit.client.0.smithi153.stdout:2/251: readlink dd/d15/d18/l28 0 2024-03-21T10:08:43.009 INFO:tasks.workunit.client.0.smithi153.stdout:4/268: dwrite d11/f20 [4194304,4194304] 0 2024-03-21T10:08:43.011 INFO:tasks.workunit.client.0.smithi153.stdout:4/269: write d11/d18/f28 [3180452,50166] 0 2024-03-21T10:08:43.011 INFO:tasks.workunit.client.0.smithi153.stdout:2/252: dread - dd/d15/d18/d2c/d33/f43 zero size 2024-03-21T10:08:43.012 INFO:tasks.workunit.client.0.smithi153.stdout:1/167: creat d6/f36 x:0 0 0 2024-03-21T10:08:43.014 INFO:tasks.workunit.client.0.smithi153.stdout:4/270: truncate f10 1226280 0 2024-03-21T10:08:43.017 INFO:tasks.workunit.client.0.smithi153.stdout:3/216: write dd/f10 [1263080,107028] 0 2024-03-21T10:08:43.019 INFO:tasks.workunit.client.0.smithi153.stdout:2/253: mkdir dd/d15/d18/d2c/d4b/d56 0 2024-03-21T10:08:43.020 INFO:tasks.workunit.client.0.smithi153.stdout:1/168: link f5 d6/d32/f37 0 2024-03-21T10:08:43.021 INFO:tasks.workunit.client.0.smithi153.stdout:2/254: truncate dd/f37 4878667 0 2024-03-21T10:08:43.024 INFO:tasks.workunit.client.0.smithi153.stdout:3/217: dread dd/f15 [0,4194304] 0 2024-03-21T10:08:43.026 INFO:tasks.workunit.client.0.smithi153.stdout:1/169: link d6/d10/l1a d6/d13/l38 0 2024-03-21T10:08:43.028 INFO:tasks.workunit.client.0.smithi153.stdout:1/170: fdatasync d6/fd 0 2024-03-21T10:08:43.029 INFO:tasks.workunit.client.0.smithi153.stdout:1/171: fdatasync d6/f16 0 2024-03-21T10:08:43.032 INFO:tasks.workunit.client.0.smithi153.stdout:1/172: symlink d6/d10/l39 0 2024-03-21T10:08:43.034 INFO:tasks.workunit.client.0.smithi153.stdout:9/164: sync 2024-03-21T10:08:43.034 INFO:tasks.workunit.client.0.smithi153.stdout:8/161: sync 2024-03-21T10:08:43.034 INFO:tasks.workunit.client.0.smithi153.stdout:6/389: sync 2024-03-21T10:08:43.034 INFO:tasks.workunit.client.0.smithi153.stdout:7/131: sync 2024-03-21T10:08:43.034 INFO:tasks.workunit.client.0.smithi153.stdout:5/164: sync 2024-03-21T10:08:43.034 INFO:tasks.workunit.client.0.smithi153.stdout:3/218: link dd/d1e/d2f/c3e dd/d1e/d2f/c4a 0 2024-03-21T10:08:43.060 INFO:tasks.workunit.client.0.smithi153.stdout:1/173: unlink d6/c2a 0 2024-03-21T10:08:43.062 INFO:tasks.workunit.client.0.smithi153.stdout:5/165: readlink d4/d5/de/d2f/l37 0 2024-03-21T10:08:43.062 INFO:tasks.workunit.client.0.smithi153.stdout:8/162: write d5/d9/f17 [1284811,6986] 0 2024-03-21T10:08:43.062 INFO:tasks.workunit.client.0.smithi153.stdout:3/219: symlink dd/d1e/l4b 0 2024-03-21T10:08:43.064 INFO:tasks.workunit.client.0.smithi153.stdout:9/165: truncate d1/d12/d1f/f2d 758784 0 2024-03-21T10:08:43.065 INFO:tasks.workunit.client.0.smithi153.stdout:6/390: creat d1f/d2f/f8c x:0 0 0 2024-03-21T10:08:43.065 INFO:tasks.workunit.client.0.smithi153.stdout:8/163: read f3 [403954,61900] 0 2024-03-21T10:08:43.066 INFO:tasks.workunit.client.0.smithi153.stdout:7/132: creat d4/d11/d24/f29 x:0 0 0 2024-03-21T10:08:43.067 INFO:tasks.workunit.client.0.smithi153.stdout:1/174: fsync d6/d10/d2c/f2e 0 2024-03-21T10:08:43.067 INFO:tasks.workunit.client.0.smithi153.stdout:5/166: truncate d4/d8/f26 617229 0 2024-03-21T10:08:43.068 INFO:tasks.workunit.client.0.smithi153.stdout:3/220: creat dd/f4c x:0 0 0 2024-03-21T10:08:43.068 INFO:tasks.workunit.client.0.smithi153.stdout:8/164: chown d5/d9/f7 29561 1 2024-03-21T10:08:43.069 INFO:tasks.workunit.client.0.smithi153.stdout:5/167: readlink d4/d5/de/d2f/l37 0 2024-03-21T10:08:43.070 INFO:tasks.workunit.client.0.smithi153.stdout:4/271: dread d11/d18/f1d [0,4194304] 0 2024-03-21T10:08:43.072 INFO:tasks.workunit.client.0.smithi153.stdout:6/391: creat d1f/d2f/d59/f8d x:0 0 0 2024-03-21T10:08:43.074 INFO:tasks.workunit.client.0.smithi153.stdout:6/392: dread - d1f/d2b/d33/d40/d5f/f72 zero size 2024-03-21T10:08:43.076 INFO:tasks.workunit.client.0.smithi153.stdout:8/165: truncate d5/d9/dc/f2b 892287 0 2024-03-21T10:08:43.078 INFO:tasks.workunit.client.0.smithi153.stdout:8/166: write d5/d9/f11 [3313998,2244] 0 2024-03-21T10:08:43.078 INFO:tasks.workunit.client.0.smithi153.stdout:6/393: symlink d1f/d2f/d59/d70/l8e 0 2024-03-21T10:08:43.080 INFO:tasks.workunit.client.0.smithi153.stdout:6/394: chown d1f/d2f/f7d 35256 1 2024-03-21T10:08:43.081 INFO:tasks.workunit.client.0.smithi153.stdout:1/175: dread d6/f18 [0,4194304] 0 2024-03-21T10:08:43.082 INFO:tasks.workunit.client.0.smithi153.stdout:5/168: rename d4/d5/de/d2f/d33 to d4/d5/d3a 0 2024-03-21T10:08:43.083 INFO:tasks.workunit.client.0.smithi153.stdout:6/395: write d1f/d2f/d59/f8d [308923,22994] 0 2024-03-21T10:08:43.084 INFO:tasks.workunit.client.0.smithi153.stdout:1/176: truncate d6/f2f 1557281 0 2024-03-21T10:08:43.088 INFO:tasks.workunit.client.0.smithi153.stdout:8/167: chown d5/d9/dc/dd/d2d/c2e 151022143 1 2024-03-21T10:08:43.088 INFO:tasks.workunit.client.0.smithi153.stdout:1/177: chown d6/d13/l24 4523 1 2024-03-21T10:08:43.089 INFO:tasks.workunit.client.0.smithi153.stdout:1/178: fdatasync f3 0 2024-03-21T10:08:43.094 INFO:tasks.workunit.client.0.smithi153.stdout:3/221: dwrite dd/d1e/f2a [0,4194304] 0 2024-03-21T10:08:43.097 INFO:tasks.workunit.client.0.smithi153.stdout:6/396: truncate d1f/d2b/f63 3641583 0 2024-03-21T10:08:43.100 INFO:tasks.workunit.client.0.smithi153.stdout:6/397: dread - d1f/f4e zero size 2024-03-21T10:08:43.101 INFO:tasks.workunit.client.0.smithi153.stdout:1/179: chown d6/d13/l38 1330175 1 2024-03-21T10:08:43.102 INFO:tasks.workunit.client.0.smithi153.stdout:1/180: dread - d6/d10/f11 zero size 2024-03-21T10:08:43.103 INFO:tasks.workunit.client.0.smithi153.stdout:5/169: dwrite d4/d5/df/f30 [0,4194304] 0 2024-03-21T10:08:43.106 INFO:tasks.workunit.client.0.smithi153.stdout:3/222: creat dd/d1e/d45/f4d x:0 0 0 2024-03-21T10:08:43.110 INFO:tasks.workunit.client.0.smithi153.stdout:8/168: dwrite d5/d9/dc/dd/d20/d26/f1f [0,4194304] 0 2024-03-21T10:08:43.114 INFO:tasks.workunit.client.0.smithi153.stdout:1/181: mkdir d6/d3a 0 2024-03-21T10:08:43.114 INFO:tasks.workunit.client.0.smithi153.stdout:3/223: creat dd/d1e/d2f/f4e x:0 0 0 2024-03-21T10:08:43.115 INFO:tasks.workunit.client.0.smithi153.stdout:1/182: fsync d6/f15 0 2024-03-21T10:08:43.116 INFO:tasks.workunit.client.0.smithi153.stdout:3/224: chown dd/c19 3439899 1 2024-03-21T10:08:43.117 INFO:tasks.workunit.client.0.smithi153.stdout:6/398: link d1f/d2b/f85 d1f/d2b/d33/d40/f8f 0 2024-03-21T10:08:43.125 INFO:tasks.workunit.client.0.smithi153.stdout:8/169: creat d5/f2f x:0 0 0 2024-03-21T10:08:43.126 INFO:tasks.workunit.client.0.smithi153.stdout:3/225: link dd/f27 dd/d1e/f4f 0 2024-03-21T10:08:43.129 INFO:tasks.workunit.client.0.smithi153.stdout:8/170: chown d5/d1b/f1d 2 1 2024-03-21T10:08:43.136 INFO:tasks.workunit.client.0.smithi153.stdout:5/170: dwrite d4/d5/f7 [0,4194304] 0 2024-03-21T10:08:43.137 INFO:tasks.workunit.client.0.smithi153.stdout:6/399: write d1f/d2b/f38 [1429852,31159] 0 2024-03-21T10:08:43.143 INFO:tasks.workunit.client.0.smithi153.stdout:5/171: write d4/d5/d16/f19 [2231142,17374] 0 2024-03-21T10:08:43.144 INFO:tasks.workunit.client.0.smithi153.stdout:3/226: getdents dd/d1e/d2f 0 2024-03-21T10:08:43.150 INFO:tasks.workunit.client.0.smithi153.stdout:6/400: rmdir d1f/d2f/d59/d5e 0 2024-03-21T10:08:43.150 INFO:tasks.workunit.client.0.smithi153.stdout:3/227: write dd/f12 [353038,125508] 0 2024-03-21T10:08:43.152 INFO:tasks.workunit.client.0.smithi153.stdout:5/172: fsync d4/d5/d16/d24/f25 0 2024-03-21T10:08:43.156 INFO:tasks.workunit.client.0.smithi153.stdout:1/183: dwrite d6/fc [4194304,4194304] 0 2024-03-21T10:08:43.157 INFO:tasks.workunit.client.0.smithi153.stdout:8/171: dwrite f4 [0,4194304] 0 2024-03-21T10:08:43.159 INFO:tasks.workunit.client.0.smithi153.stdout:5/173: write d4/d8/f2d [551248,75812] 0 2024-03-21T10:08:43.160 INFO:tasks.workunit.client.0.smithi153.stdout:6/401: mknod d1f/d2b/d33/d40/d4f/d66/c90 0 2024-03-21T10:08:43.163 INFO:tasks.workunit.client.0.smithi153.stdout:3/228: truncate dd/f15 386166 0 2024-03-21T10:08:43.164 INFO:tasks.workunit.client.0.smithi153.stdout:5/174: read - d4/d5/df/f34 zero size 2024-03-21T10:08:43.165 INFO:tasks.workunit.client.0.smithi153.stdout:1/184: mkdir d6/d3a/d3b 0 2024-03-21T10:08:43.166 INFO:tasks.workunit.client.0.smithi153.stdout:6/402: chown d1f/d2b/d33/d40/d65/c6b 40 1 2024-03-21T10:08:43.168 INFO:tasks.workunit.client.0.smithi153.stdout:3/229: truncate dd/d1e/d2f/f37 597079 0 2024-03-21T10:08:43.173 INFO:tasks.workunit.client.0.smithi153.stdout:5/175: mkdir d4/d8/d3b 0 2024-03-21T10:08:43.173 INFO:tasks.workunit.client.0.smithi153.stdout:1/185: mkdir d6/d3a/d3c 0 2024-03-21T10:08:43.178 INFO:tasks.workunit.client.0.smithi153.stdout:3/230: rename dd/d1e/d2f/c4a to dd/c50 0 2024-03-21T10:08:43.189 INFO:tasks.workunit.client.0.smithi153.stdout:6/403: dwrite d1f/d2b/d33/d40/f45 [0,4194304] 0 2024-03-21T10:08:43.191 INFO:tasks.workunit.client.0.smithi153.stdout:8/172: truncate d5/d1b/f1d 865942 0 2024-03-21T10:08:43.192 INFO:tasks.workunit.client.0.smithi153.stdout:5/176: dwrite d4/d5/de/f18 [0,4194304] 0 2024-03-21T10:08:43.193 INFO:tasks.workunit.client.0.smithi153.stdout:8/173: chown d5/d9/f17 1395959 1 2024-03-21T10:08:43.200 INFO:tasks.workunit.client.0.smithi153.stdout:0/154: sync 2024-03-21T10:08:43.205 INFO:tasks.workunit.client.0.smithi153.stdout:6/404: symlink d1f/d2f/d59/d70/l91 0 2024-03-21T10:08:43.207 INFO:tasks.workunit.client.0.smithi153.stdout:3/231: dwrite dd/f22 [0,4194304] 0 2024-03-21T10:08:43.209 INFO:tasks.workunit.client.0.smithi153.stdout:6/405: write d1f/d2b/f38 [190759,4668] 0 2024-03-21T10:08:43.212 INFO:tasks.workunit.client.0.smithi153.stdout:0/155: rmdir d4/d16/d19 39 2024-03-21T10:08:43.222 INFO:tasks.workunit.client.0.smithi153.stdout:5/177: dread d4/d5/df/f30 [0,4194304] 0 2024-03-21T10:08:43.222 INFO:tasks.workunit.client.0.smithi153.stdout:5/178: readlink d4/d5/l31 0 2024-03-21T10:08:43.226 INFO:tasks.workunit.client.0.smithi153.stdout:6/406: creat d1f/d2f/d59/d70/f92 x:0 0 0 2024-03-21T10:08:43.227 INFO:tasks.workunit.client.0.smithi153.stdout:5/179: fsync d4/d8/f1d 0 2024-03-21T10:08:43.232 INFO:tasks.workunit.client.0.smithi153.stdout:5/180: creat d4/f3c x:0 0 0 2024-03-21T10:08:43.232 INFO:tasks.workunit.client.0.smithi153.stdout:6/407: write d1f/d2b/d33/d40/d5f/f72 [461138,58453] 0 2024-03-21T10:08:43.233 INFO:tasks.workunit.client.0.smithi153.stdout:1/186: dread d6/fa [0,4194304] 0 2024-03-21T10:08:43.236 INFO:tasks.workunit.client.0.smithi153.stdout:6/408: chown c10 24074 1 2024-03-21T10:08:43.237 INFO:tasks.workunit.client.0.smithi153.stdout:1/187: fsync d6/fe 0 2024-03-21T10:08:43.238 INFO:tasks.workunit.client.0.smithi153.stdout:5/181: mknod d4/d5/de/d2f/d2c/c3d 0 2024-03-21T10:08:43.242 INFO:tasks.workunit.client.0.smithi153.stdout:8/174: dwrite f3 [4194304,4194304] 0 2024-03-21T10:08:43.245 INFO:tasks.workunit.client.0.smithi153.stdout:1/188: symlink d6/d10/d35/l3d 0 2024-03-21T10:08:43.247 INFO:tasks.workunit.client.0.smithi153.stdout:3/232: dwrite f7 [0,4194304] 0 2024-03-21T10:08:43.247 INFO:tasks.workunit.client.0.smithi153.stdout:8/175: mknod d5/d9/dc/c30 0 2024-03-21T10:08:43.247 INFO:tasks.workunit.client.0.smithi153.stdout:0/156: dwrite d4/d16/d1f/d22/f2e [4194304,4194304] 0 2024-03-21T10:08:43.249 INFO:tasks.workunit.client.0.smithi153.stdout:0/157: read - d4/d16/d1f/f24 zero size 2024-03-21T10:08:43.251 INFO:tasks.workunit.client.0.smithi153.stdout:0/158: chown d4/db/f1b 63069237 1 2024-03-21T10:08:43.254 INFO:tasks.workunit.client.0.smithi153.stdout:8/176: creat d5/d9/dc/dd/d2d/f31 x:0 0 0 2024-03-21T10:08:43.255 INFO:tasks.workunit.client.0.smithi153.stdout:8/177: dread - d5/f2f zero size 2024-03-21T10:08:43.256 INFO:tasks.workunit.client.0.smithi153.stdout:1/189: truncate d6/d13/f2b 1890537 0 2024-03-21T10:08:43.257 INFO:tasks.workunit.client.0.smithi153.stdout:8/178: chown d5/d9/dc/dd/d20/d26/d24 16 1 2024-03-21T10:08:43.257 INFO:tasks.workunit.client.0.smithi153.stdout:1/190: readlink l1 0 2024-03-21T10:08:43.257 INFO:tasks.workunit.client.0.smithi153.stdout:3/233: mknod dd/d1e/d45/c51 0 2024-03-21T10:08:43.263 INFO:tasks.workunit.client.0.smithi153.stdout:1/191: mkdir d6/d32/d3e 0 2024-03-21T10:08:43.264 INFO:tasks.workunit.client.0.smithi153.stdout:3/234: read dd/f27 [3621611,93379] 0 2024-03-21T10:08:43.267 INFO:tasks.workunit.client.0.smithi153.stdout:1/192: chown d6/f1c 17 1 2024-03-21T10:08:43.271 INFO:tasks.workunit.client.0.smithi153.stdout:6/409: dwrite d1f/d2b/d33/d40/d51/f52 [0,4194304] 0 2024-03-21T10:08:43.272 INFO:tasks.workunit.client.0.smithi153.stdout:1/193: creat d6/d13/f3f x:0 0 0 2024-03-21T10:08:43.276 INFO:tasks.workunit.client.0.smithi153.stdout:3/235: rename dd/d1e/f2a to dd/d1e/d45/f52 0 2024-03-21T10:08:43.276 INFO:tasks.workunit.client.0.smithi153.stdout:1/194: readlink d6/d10/l14 0 2024-03-21T10:08:43.276 INFO:tasks.workunit.client.0.smithi153.stdout:1/195: stat d6/fe 0 2024-03-21T10:08:43.276 INFO:tasks.workunit.client.0.smithi153.stdout:6/410: mkdir d1f/d93 0 2024-03-21T10:08:43.277 INFO:tasks.workunit.client.0.smithi153.stdout:8/179: dread d5/d9/dc/f16 [0,4194304] 0 2024-03-21T10:08:43.281 INFO:tasks.workunit.client.0.smithi153.stdout:3/236: mkdir dd/d1e/d3d/d53 0 2024-03-21T10:08:43.282 INFO:tasks.workunit.client.0.smithi153.stdout:1/196: write d6/fc [6817608,81656] 0 2024-03-21T10:08:43.284 INFO:tasks.workunit.client.0.smithi153.stdout:5/182: dwrite d4/d5/d16/f19 [0,4194304] 0 2024-03-21T10:08:43.284 INFO:tasks.workunit.client.0.smithi153.stdout:8/180: write d5/d9/dc/dd/d2d/f31 [292927,12611] 0 2024-03-21T10:08:43.285 INFO:tasks.workunit.client.0.smithi153.stdout:8/181: readlink d5/d9/le 0 2024-03-21T10:08:43.288 INFO:tasks.workunit.client.0.smithi153.stdout:8/182: chown d5/d9/dc/dd/d2d 0 1 2024-03-21T10:08:43.288 INFO:tasks.workunit.client.0.smithi153.stdout:6/411: creat d1f/d2f/f94 x:0 0 0 2024-03-21T10:08:43.290 INFO:tasks.workunit.client.0.smithi153.stdout:1/197: rename d6/d10/c12 to d6/d3a/d3c/c40 0 2024-03-21T10:08:43.294 INFO:tasks.workunit.client.0.smithi153.stdout:8/183: creat d5/d9/dc/dd/d20/d26/d24/f32 x:0 0 0 2024-03-21T10:08:43.295 INFO:tasks.workunit.client.0.smithi153.stdout:5/183: mknod d4/d5/de/d38/c3e 0 2024-03-21T10:08:43.296 INFO:tasks.workunit.client.0.smithi153.stdout:6/412: fsync d1f/f28 0 2024-03-21T10:08:43.303 INFO:tasks.workunit.client.0.smithi153.stdout:1/198: truncate f5 703660 0 2024-03-21T10:08:43.303 INFO:tasks.workunit.client.0.smithi153.stdout:8/184: mkdir d5/d9/dc/dd/d20/d26/d33 0 2024-03-21T10:08:43.304 INFO:tasks.workunit.client.0.smithi153.stdout:3/237: dwrite dd/f2b [0,4194304] 0 2024-03-21T10:08:43.304 INFO:tasks.workunit.client.0.smithi153.stdout:8/185: write d5/d9/f7 [6252849,29225] 0 2024-03-21T10:08:43.306 INFO:tasks.workunit.client.0.smithi153.stdout:3/238: chown dd/f2b 55517 1 2024-03-21T10:08:43.308 INFO:tasks.workunit.client.0.smithi153.stdout:6/413: symlink d1f/d2b/d33/l95 0 2024-03-21T10:08:43.309 INFO:tasks.workunit.client.0.smithi153.stdout:1/199: creat d6/d3a/d3c/f41 x:0 0 0 2024-03-21T10:08:43.315 INFO:tasks.workunit.client.0.smithi153.stdout:8/186: fdatasync f3 0 2024-03-21T10:08:43.318 INFO:tasks.workunit.client.0.smithi153.stdout:8/187: fsync d5/d9/dc/dd/d20/d26/d24/f28 0 2024-03-21T10:08:43.320 INFO:tasks.workunit.client.0.smithi153.stdout:6/414: symlink d1f/d2f/d59/d70/l96 0 2024-03-21T10:08:43.321 INFO:tasks.workunit.client.0.smithi153.stdout:8/188: chown d5/d9/dc/dd/d20 466 1 2024-03-21T10:08:43.326 INFO:tasks.workunit.client.0.smithi153.stdout:3/239: dwrite dd/f13 [0,4194304] 0 2024-03-21T10:08:43.328 INFO:tasks.workunit.client.0.smithi153.stdout:3/240: stat dd/c11 0 2024-03-21T10:08:43.332 INFO:tasks.workunit.client.0.smithi153.stdout:5/184: truncate f1 2909982 0 2024-03-21T10:08:43.332 INFO:tasks.workunit.client.0.smithi153.stdout:8/189: mkdir d5/d34 0 2024-03-21T10:08:43.332 INFO:tasks.workunit.client.0.smithi153.stdout:6/415: rename d1f/d2f/d59/d70/l8e to d1f/d2f/d59/l97 0 2024-03-21T10:08:43.338 INFO:tasks.workunit.client.0.smithi153.stdout:8/190: rename d5/d1b/f18 to d5/d9/dc/f35 0 2024-03-21T10:08:43.340 INFO:tasks.workunit.client.0.smithi153.stdout:6/416: mkdir d1f/d2b/d33/d40/d51/d98 0 2024-03-21T10:08:43.347 INFO:tasks.workunit.client.0.smithi153.stdout:8/191: rename d5/d9/dc/c30 to d5/c36 0 2024-03-21T10:08:43.351 INFO:tasks.workunit.client.0.smithi153.stdout:1/200: dread d6/fc [0,4194304] 0 2024-03-21T10:08:43.353 INFO:tasks.workunit.client.0.smithi153.stdout:8/192: write d5/d9/dc/dd/d20/d26/d24/f32 [955308,83042] 0 2024-03-21T10:08:43.354 INFO:tasks.workunit.client.0.smithi153.stdout:5/185: dwrite d4/d5/d16/f19 [0,4194304] 0 2024-03-21T10:08:43.360 INFO:tasks.workunit.client.0.smithi153.stdout:6/417: truncate d1f/d2b/d33/d40/f45 3003519 0 2024-03-21T10:08:43.364 INFO:tasks.workunit.client.0.smithi153.stdout:5/186: unlink d4/d5/d16/f1c 0 2024-03-21T10:08:43.365 INFO:tasks.workunit.client.0.smithi153.stdout:8/193: truncate d5/d9/dc/f2b 602793 0 2024-03-21T10:08:43.366 INFO:tasks.workunit.client.0.smithi153.stdout:5/187: dread - d4/d5/d16/f1e zero size 2024-03-21T10:08:43.367 INFO:tasks.workunit.client.0.smithi153.stdout:3/241: dread dd/f20 [0,4194304] 0 2024-03-21T10:08:43.369 INFO:tasks.workunit.client.0.smithi153.stdout:3/242: truncate dd/d1e/d45/f4d 200826 0 2024-03-21T10:08:43.369 INFO:tasks.workunit.client.0.smithi153.stdout:5/188: creat d4/d5/de/f3f x:0 0 0 2024-03-21T10:08:43.373 INFO:tasks.workunit.client.0.smithi153.stdout:6/418: dread d1f/d2b/d33/d40/d4f/d66/f75 [0,4194304] 0 2024-03-21T10:08:43.374 INFO:tasks.workunit.client.0.smithi153.stdout:5/189: creat d4/d5/df/f40 x:0 0 0 2024-03-21T10:08:43.378 INFO:tasks.workunit.client.0.smithi153.stdout:6/419: mkdir d1f/d99 0 2024-03-21T10:08:43.380 INFO:tasks.workunit.client.0.smithi153.stdout:5/190: creat d4/d5/de/d2f/f41 x:0 0 0 2024-03-21T10:08:43.380 INFO:tasks.workunit.client.0.smithi153.stdout:1/201: write f5 [1402120,51672] 0 2024-03-21T10:08:43.387 INFO:tasks.workunit.client.0.smithi153.stdout:1/202: creat d6/d32/d3e/f42 x:0 0 0 2024-03-21T10:08:43.393 INFO:tasks.workunit.client.0.smithi153.stdout:8/194: dwrite d5/d9/dc/f2c [0,4194304] 0 2024-03-21T10:08:43.396 INFO:tasks.workunit.client.0.smithi153.stdout:5/191: dread d4/d5/d16/f19 [0,4194304] 0 2024-03-21T10:08:43.399 INFO:tasks.workunit.client.0.smithi153.stdout:5/192: mknod d4/c42 0 2024-03-21T10:08:43.401 INFO:tasks.workunit.client.0.smithi153.stdout:5/193: dread - d4/d5/de/f36 zero size 2024-03-21T10:08:43.405 INFO:tasks.workunit.client.0.smithi153.stdout:5/194: read d4/d5/f7 [3927803,50779] 0 2024-03-21T10:08:43.409 INFO:tasks.workunit.client.0.smithi153.stdout:8/195: dwrite d5/d9/f7 [0,4194304] 0 2024-03-21T10:08:43.417 INFO:tasks.workunit.client.0.smithi153.stdout:8/196: link d5/d1b/f1d d5/d1b/f37 0 2024-03-21T10:08:43.421 INFO:tasks.workunit.client.0.smithi153.stdout:8/197: read d5/d9/dc/f2c [310777,86995] 0 2024-03-21T10:08:43.422 INFO:tasks.workunit.client.0.smithi153.stdout:5/195: dwrite d4/d5/df/f34 [0,4194304] 0 2024-03-21T10:08:43.425 INFO:tasks.workunit.client.0.smithi153.stdout:8/198: chown d5/d9/dc/dd/d20/d26/f1f 12447121 1 2024-03-21T10:08:43.427 INFO:tasks.workunit.client.0.smithi153.stdout:8/199: read d5/d9/f7 [706033,96904] 0 2024-03-21T10:08:43.427 INFO:tasks.workunit.client.0.smithi153.stdout:1/203: dread f3 [0,4194304] 0 2024-03-21T10:08:43.429 INFO:tasks.workunit.client.0.smithi153.stdout:1/204: write d6/fe [4039685,22761] 0 2024-03-21T10:08:43.433 INFO:tasks.workunit.client.0.smithi153.stdout:1/205: rmdir d6/d10/d2c 39 2024-03-21T10:08:43.435 INFO:tasks.workunit.client.0.smithi153.stdout:9/166: sync 2024-03-21T10:08:43.435 INFO:tasks.workunit.client.0.smithi153.stdout:2/255: sync 2024-03-21T10:08:43.438 INFO:tasks.workunit.client.0.smithi153.stdout:1/206: fsync d6/f7 0 2024-03-21T10:08:43.442 INFO:tasks.workunit.client.0.smithi153.stdout:5/196: truncate d4/d5/de/f22 126274 0 2024-03-21T10:08:43.444 INFO:tasks.workunit.client.0.smithi153.stdout:1/207: creat d6/d3a/d3c/f43 x:0 0 0 2024-03-21T10:08:43.444 INFO:tasks.workunit.client.0.smithi153.stdout:2/256: truncate dd/d15/d18/d2c/f2d 1975552 0 2024-03-21T10:08:43.449 INFO:tasks.workunit.client.0.smithi153.stdout:5/197: truncate d4/d5/f1a 1210176 0 2024-03-21T10:08:43.451 INFO:tasks.workunit.client.0.smithi153.stdout:2/257: write dd/d15/d18/d2c/d4b/f4d [7452,69933] 0 2024-03-21T10:08:43.457 INFO:tasks.workunit.client.0.smithi153.stdout:9/167: truncate d1/d12/d1f/f2d 55553 0 2024-03-21T10:08:43.458 INFO:tasks.workunit.client.0.smithi153.stdout:8/200: dwrite d5/d9/dc/dd/d2d/f31 [0,4194304] 0 2024-03-21T10:08:43.458 INFO:tasks.workunit.client.0.smithi153.stdout:1/208: dread d6/f2f [0,4194304] 0 2024-03-21T10:08:43.469 INFO:tasks.workunit.client.0.smithi153.stdout:8/201: rename f4 to d5/d9/dc/dd/f38 0 2024-03-21T10:08:43.471 INFO:tasks.workunit.client.0.smithi153.stdout:1/209: unlink d6/ff 0 2024-03-21T10:08:43.476 INFO:tasks.workunit.client.0.smithi153.stdout:1/210: write d6/d10/d2c/f2e [2586951,110890] 0 2024-03-21T10:08:43.478 INFO:tasks.workunit.client.0.smithi153.stdout:2/258: truncate dd/d15/d18/f24 1679776 0 2024-03-21T10:08:43.479 INFO:tasks.workunit.client.0.smithi153.stdout:1/211: dread - d6/d13/f2d zero size 2024-03-21T10:08:43.479 INFO:tasks.workunit.client.0.smithi153.stdout:5/198: truncate d4/d8/f26 17219 0 2024-03-21T10:08:43.479 INFO:tasks.workunit.client.0.smithi153.stdout:2/259: fdatasync dd/f37 0 2024-03-21T10:08:43.480 INFO:tasks.workunit.client.0.smithi153.stdout:9/168: truncate d1/d12/f19 1722229 0 2024-03-21T10:08:43.483 INFO:tasks.workunit.client.0.smithi153.stdout:2/260: dread - dd/d15/d18/f45 zero size 2024-03-21T10:08:43.483 INFO:tasks.workunit.client.0.smithi153.stdout:1/212: write d6/fc [6462862,11312] 0 2024-03-21T10:08:43.486 INFO:tasks.workunit.client.0.smithi153.stdout:8/202: dread d5/d9/dc/dd/d2d/f31 [0,4194304] 0 2024-03-21T10:08:43.491 INFO:tasks.workunit.client.0.smithi153.stdout:9/169: creat d1/d12/d1f/f34 x:0 0 0 2024-03-21T10:08:43.496 INFO:tasks.workunit.client.0.smithi153.stdout:2/261: creat dd/d15/d18/d2c/d33/f57 x:0 0 0 2024-03-21T10:08:43.500 INFO:tasks.workunit.client.0.smithi153.stdout:9/170: unlink d1/d12/d1c/l22 0 2024-03-21T10:08:43.502 INFO:tasks.workunit.client.0.smithi153.stdout:5/199: dwrite d4/d5/d16/d24/f25 [0,4194304] 0 2024-03-21T10:08:43.507 INFO:tasks.workunit.client.0.smithi153.stdout:1/213: truncate d6/f15 1494093 0 2024-03-21T10:08:43.513 INFO:tasks.workunit.client.0.smithi153.stdout:1/214: mknod d6/d10/d2c/c44 0 2024-03-21T10:08:43.515 INFO:tasks.workunit.client.0.smithi153.stdout:1/215: dread - d6/d10/f11 zero size 2024-03-21T10:08:43.516 INFO:tasks.workunit.client.0.smithi153.stdout:9/171: symlink d1/d12/d1c/d2e/l35 0 2024-03-21T10:08:43.519 INFO:tasks.workunit.client.0.smithi153.stdout:5/200: dwrite d4/d8/f1d [0,4194304] 0 2024-03-21T10:08:43.521 INFO:tasks.workunit.client.0.smithi153.stdout:8/203: getdents d5/d9/dc/dd/d20/d26/d24 0 2024-03-21T10:08:43.527 INFO:tasks.workunit.client.0.smithi153.stdout:2/262: getdents dd/d15/d18/d2c/d4b 0 2024-03-21T10:08:43.531 INFO:tasks.workunit.client.0.smithi153.stdout:9/172: creat d1/d12/d1c/f36 x:0 0 0 2024-03-21T10:08:43.532 INFO:tasks.workunit.client.0.smithi153.stdout:2/263: chown dd/d15/d18/f1a 6868058 1 2024-03-21T10:08:43.534 INFO:tasks.workunit.client.0.smithi153.stdout:8/204: truncate d5/d1b/f1d 1019815 0 2024-03-21T10:08:43.536 INFO:tasks.workunit.client.0.smithi153.stdout:1/216: dwrite d6/d32/f37 [0,4194304] 0 2024-03-21T10:08:43.538 INFO:tasks.workunit.client.0.smithi153.stdout:9/173: creat d1/d12/f37 x:0 0 0 2024-03-21T10:08:43.538 INFO:tasks.workunit.client.0.smithi153.stdout:5/201: dread d4/d5/f1a [0,4194304] 0 2024-03-21T10:08:43.538 INFO:tasks.workunit.client.0.smithi153.stdout:8/205: write d5/d9/f19 [1244390,58843] 0 2024-03-21T10:08:43.539 INFO:tasks.workunit.client.0.smithi153.stdout:2/264: mkdir dd/d3b/d50/d58 0 2024-03-21T10:08:43.541 INFO:tasks.workunit.client.0.smithi153.stdout:5/202: write d4/d5/de/f36 [555636,105914] 0 2024-03-21T10:08:43.543 INFO:tasks.workunit.client.0.smithi153.stdout:9/174: creat d1/d12/d1c/d24/f38 x:0 0 0 2024-03-21T10:08:43.543 INFO:tasks.workunit.client.0.smithi153.stdout:2/265: dread - dd/d15/d18/d2c/d33/f43 zero size 2024-03-21T10:08:43.543 INFO:tasks.workunit.client.0.smithi153.stdout:8/206: rmdir d5/d9/dc/dd/d20/d26 39 2024-03-21T10:08:43.546 INFO:tasks.workunit.client.0.smithi153.stdout:2/266: readlink dd/d15/l48 0 2024-03-21T10:08:43.548 INFO:tasks.workunit.client.0.smithi153.stdout:1/217: rmdir d6/d3a/d3b 0 2024-03-21T10:08:43.549 INFO:tasks.workunit.client.0.smithi153.stdout:5/203: creat d4/d5/de/f43 x:0 0 0 2024-03-21T10:08:43.550 INFO:tasks.workunit.client.0.smithi153.stdout:9/175: unlink d1/f16 0 2024-03-21T10:08:43.551 INFO:tasks.workunit.client.0.smithi153.stdout:1/218: write d6/d10/d2c/f2e [3076246,35465] 0 2024-03-21T10:08:43.553 INFO:tasks.workunit.client.0.smithi153.stdout:9/176: mknod d1/d4/c39 0 2024-03-21T10:08:43.557 INFO:tasks.workunit.client.0.smithi153.stdout:1/219: read d6/f17 [1635114,18829] 0 2024-03-21T10:08:43.558 INFO:tasks.workunit.client.0.smithi153.stdout:9/177: rename c0 to d1/d12/d1f/c3a 0 2024-03-21T10:08:43.559 INFO:tasks.workunit.client.0.smithi153.stdout:8/207: dwrite d5/d9/f17 [0,4194304] 0 2024-03-21T10:08:43.560 INFO:tasks.workunit.client.0.smithi153.stdout:9/178: stat d1/d12/f26 0 2024-03-21T10:08:43.564 INFO:tasks.workunit.client.0.smithi153.stdout:8/208: rmdir d5/d34 0 2024-03-21T10:08:43.571 INFO:tasks.workunit.client.0.smithi153.stdout:9/179: dread d1/d4/fc [0,4194304] 0 2024-03-21T10:08:43.571 INFO:tasks.workunit.client.0.smithi153.stdout:8/209: symlink d5/d9/l39 0 2024-03-21T10:08:43.572 INFO:tasks.workunit.client.0.smithi153.stdout:5/204: dwrite d4/f3c [0,4194304] 0 2024-03-21T10:08:43.578 INFO:tasks.workunit.client.0.smithi153.stdout:9/180: fdatasync d1/d12/d1c/d24/f32 0 2024-03-21T10:08:43.578 INFO:tasks.workunit.client.0.smithi153.stdout:5/205: stat d4/d5/de/d2f/d2c/c3d 0 2024-03-21T10:08:43.579 INFO:tasks.workunit.client.0.smithi153.stdout:5/206: chown d4/d5/de/f3f 315155 1 2024-03-21T10:08:43.581 INFO:tasks.workunit.client.0.smithi153.stdout:8/210: creat d5/d9/dc/dd/d20/d26/d24/f3a x:0 0 0 2024-03-21T10:08:43.584 INFO:tasks.workunit.client.0.smithi153.stdout:5/207: mkdir d4/d5/de/d44 0 2024-03-21T10:08:43.586 INFO:tasks.workunit.client.0.smithi153.stdout:5/208: stat d4/c42 0 2024-03-21T10:08:43.587 INFO:tasks.workunit.client.0.smithi153.stdout:9/181: rmdir d1/d12/d1f 39 2024-03-21T10:08:43.588 INFO:tasks.workunit.client.0.smithi153.stdout:8/211: mkdir d5/d3b 0 2024-03-21T10:08:43.589 INFO:tasks.workunit.client.0.smithi153.stdout:1/220: write d6/f7 [482763,40517] 0 2024-03-21T10:08:43.590 INFO:tasks.workunit.client.0.smithi153.stdout:5/209: rmdir d4/d5/de/d2f 39 2024-03-21T10:08:43.598 INFO:tasks.workunit.client.0.smithi153.stdout:5/210: chown d4/d5/de/d2f/d2c 1159 1 2024-03-21T10:08:43.599 INFO:tasks.workunit.client.0.smithi153.stdout:1/221: dread d6/f7 [0,4194304] 0 2024-03-21T10:08:43.599 INFO:tasks.workunit.client.0.smithi153.stdout:1/222: chown d6/d10/d2c/f33 101888 1 2024-03-21T10:08:43.601 INFO:tasks.workunit.client.0.smithi153.stdout:1/223: fsync d6/d32/f37 0 2024-03-21T10:08:43.605 INFO:tasks.workunit.client.0.smithi153.stdout:8/212: dread d5/d9/dc/f16 [4194304,4194304] 0 2024-03-21T10:08:43.608 INFO:tasks.workunit.client.0.smithi153.stdout:9/182: dread d1/d12/f19 [0,4194304] 0 2024-03-21T10:08:43.609 INFO:tasks.workunit.client.0.smithi153.stdout:9/183: dread - d1/d12/d1c/f36 zero size 2024-03-21T10:08:43.614 INFO:tasks.workunit.client.0.smithi153.stdout:9/184: fdatasync d1/d12/d1f/f2d 0 2024-03-21T10:08:43.615 INFO:tasks.workunit.client.0.smithi153.stdout:1/224: dread d6/d10/f34 [0,4194304] 0 2024-03-21T10:08:43.616 INFO:tasks.workunit.client.0.smithi153.stdout:1/225: read d6/fa [90282,94903] 0 2024-03-21T10:08:43.621 INFO:tasks.workunit.client.0.smithi153.stdout:8/213: dwrite d5/d9/dc/f35 [0,4194304] 0 2024-03-21T10:08:43.626 INFO:tasks.workunit.client.0.smithi153.stdout:8/214: read d5/d1b/f1d [808993,77324] 0 2024-03-21T10:08:43.631 INFO:tasks.workunit.client.0.smithi153.stdout:9/185: dwrite d1/d12/d1c/d24/f38 [0,4194304] 0 2024-03-21T10:08:43.636 INFO:tasks.workunit.client.0.smithi153.stdout:9/186: fdatasync d1/d12/d1f/f2d 0 2024-03-21T10:08:43.647 INFO:tasks.workunit.client.0.smithi153.stdout:8/215: dwrite d5/d1b/f37 [0,4194304] 0 2024-03-21T10:08:43.662 INFO:tasks.workunit.client.0.smithi153.stdout:9/187: dwrite d1/d12/d1f/f25 [0,4194304] 0 2024-03-21T10:08:43.666 INFO:tasks.workunit.client.0.smithi153.stdout:1/226: dread d6/d10/d2c/f2e [0,4194304] 0 2024-03-21T10:08:43.666 INFO:tasks.workunit.client.0.smithi153.stdout:9/188: chown d1/d12/d1f/f2d 3126 1 2024-03-21T10:08:43.674 INFO:tasks.workunit.client.0.smithi153.stdout:9/189: mkdir d1/d3b 0 2024-03-21T10:08:43.678 INFO:tasks.workunit.client.0.smithi153.stdout:9/190: fsync d1/f27 0 2024-03-21T10:08:43.678 INFO:tasks.workunit.client.0.smithi153.stdout:8/216: dwrite f3 [4194304,4194304] 0 2024-03-21T10:08:43.678 INFO:tasks.workunit.client.0.smithi153.stdout:1/227: creat d6/d13/f45 x:0 0 0 2024-03-21T10:08:43.684 INFO:tasks.workunit.client.0.smithi153.stdout:1/228: rmdir d6/d10 39 2024-03-21T10:08:43.688 INFO:tasks.workunit.client.0.smithi153.stdout:9/191: mknod d1/d3b/c3c 0 2024-03-21T10:08:43.692 INFO:tasks.workunit.client.0.smithi153.stdout:9/192: unlink d1/d12/d1c/l21 0 2024-03-21T10:08:43.698 INFO:tasks.workunit.client.0.smithi153.stdout:9/193: creat d1/d12/d1c/f3d x:0 0 0 2024-03-21T10:08:43.700 INFO:tasks.workunit.client.0.smithi153.stdout:8/217: dread d5/d9/dc/f2c [0,4194304] 0 2024-03-21T10:08:43.701 INFO:tasks.workunit.client.0.smithi153.stdout:9/194: write d1/d12/d1f/f34 [505041,87882] 0 2024-03-21T10:08:43.703 INFO:tasks.workunit.client.0.smithi153.stdout:9/195: chown d1/d12/d1c/d24 5016114 1 2024-03-21T10:08:43.703 INFO:tasks.workunit.client.0.smithi153.stdout:8/218: write d5/d9/f17 [2269196,97045] 0 2024-03-21T10:08:43.708 INFO:tasks.workunit.client.0.smithi153.stdout:9/196: unlink d1/d12/d1c/d2e/l35 0 2024-03-21T10:08:43.709 INFO:tasks.workunit.client.0.smithi153.stdout:9/197: fdatasync d1/d12/d1f/f25 0 2024-03-21T10:08:43.711 INFO:tasks.workunit.client.0.smithi153.stdout:1/229: dwrite d6/d13/f2b [0,4194304] 0 2024-03-21T10:08:43.712 INFO:tasks.workunit.client.0.smithi153.stdout:9/198: chown d1/d12/d1f/f34 9689 1 2024-03-21T10:08:43.715 INFO:tasks.workunit.client.0.smithi153.stdout:1/230: symlink d6/d3a/l46 0 2024-03-21T10:08:43.718 INFO:tasks.workunit.client.0.smithi153.stdout:9/199: unlink d1/d12/d1c/c23 0 2024-03-21T10:08:43.718 INFO:tasks.workunit.client.0.smithi153.stdout:9/200: readlink d1/d4/la 0 2024-03-21T10:08:43.724 INFO:tasks.workunit.client.0.smithi153.stdout:9/201: getdents d1 0 2024-03-21T10:08:43.740 INFO:tasks.workunit.client.0.smithi153.stdout:9/202: dwrite d1/d12/d1c/f36 [0,4194304] 0 2024-03-21T10:08:43.740 INFO:tasks.workunit.client.0.smithi153.stdout:8/219: dread d5/d9/f11 [0,4194304] 0 2024-03-21T10:08:43.749 INFO:tasks.workunit.client.0.smithi153.stdout:8/220: rename d5/d9/dc/dd/d20/d26/d24/f3a to d5/d9/dc/dd/d20/d26/d33/f3c 0 2024-03-21T10:08:43.752 INFO:tasks.workunit.client.0.smithi153.stdout:8/221: chown d5/d9/dc/dd/f22 12578 1 2024-03-21T10:08:43.758 INFO:tasks.workunit.client.0.smithi153.stdout:9/203: dwrite d1/d12/d1c/f3d [0,4194304] 0 2024-03-21T10:08:43.766 INFO:tasks.workunit.client.0.smithi153.stdout:8/222: dread d5/d9/dc/dd/d20/d26/f1f [0,4194304] 0 2024-03-21T10:08:43.771 INFO:tasks.workunit.client.0.smithi153.stdout:8/223: mknod d5/d9/dc/dd/d20/c3d 0 2024-03-21T10:08:43.775 INFO:tasks.workunit.client.0.smithi153.stdout:9/204: dwrite d1/d12/d1c/f36 [0,4194304] 0 2024-03-21T10:08:43.778 INFO:tasks.workunit.client.0.smithi153.stdout:9/205: creat d1/d12/d1f/f3e x:0 0 0 2024-03-21T10:08:43.784 INFO:tasks.workunit.client.0.smithi153.stdout:9/206: mknod d1/d12/c3f 0 2024-03-21T10:08:43.784 INFO:tasks.workunit.client.0.smithi153.stdout:8/224: dread d5/d9/dc/f16 [0,4194304] 0 2024-03-21T10:08:43.789 INFO:tasks.workunit.client.0.smithi153.stdout:8/225: rmdir d5/d9/dc/dd/d20/d26/d24 39 2024-03-21T10:08:43.790 INFO:tasks.workunit.client.0.smithi153.stdout:9/207: dread d1/d12/d1f/f34 [0,4194304] 0 2024-03-21T10:08:43.795 INFO:tasks.workunit.client.0.smithi153.stdout:9/208: write d1/d4/f14 [3921381,39893] 0 2024-03-21T10:08:43.798 INFO:tasks.workunit.client.0.smithi153.stdout:9/209: symlink d1/d3b/l40 0 2024-03-21T10:08:43.802 INFO:tasks.workunit.client.0.smithi153.stdout:9/210: mkdir d1/d12/d1c/d24/d41 0 2024-03-21T10:08:43.804 INFO:tasks.workunit.client.0.smithi153.stdout:8/226: dwrite d5/f2f [0,4194304] 0 2024-03-21T10:08:43.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:43 smithi153 ceph-mon[29117]: pgmap v8: 65 pgs: 65 active+clean; 369 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 28 MiB/s wr, 116 op/s 2024-03-21T10:08:43.807 INFO:tasks.workunit.client.0.smithi153.stdout:9/211: rename d1/d12/c3f to d1/d12/d1c/c42 0 2024-03-21T10:08:43.826 INFO:tasks.workunit.client.0.smithi153.stdout:8/227: dwrite d5/d9/dc/f2c [0,4194304] 0 2024-03-21T10:08:43.843 INFO:tasks.workunit.client.0.smithi153.stdout:8/228: dwrite d5/f2f [4194304,4194304] 0 2024-03-21T10:08:43.848 INFO:tasks.workunit.client.0.smithi153.stdout:8/229: symlink d5/d9/dc/l3e 0 2024-03-21T10:08:43.851 INFO:tasks.workunit.client.0.smithi153.stdout:9/212: dread d1/f1b [0,4194304] 0 2024-03-21T10:08:43.852 INFO:tasks.workunit.client.0.smithi153.stdout:8/230: unlink d5/d9/dc/dd/d20/c3d 0 2024-03-21T10:08:43.853 INFO:tasks.workunit.client.0.smithi153.stdout:9/213: write d1/d12/f1d [1151939,61079] 0 2024-03-21T10:08:43.855 INFO:tasks.workunit.client.0.smithi153.stdout:9/214: mkdir d1/d43 0 2024-03-21T10:08:43.856 INFO:tasks.workunit.client.0.smithi153.stdout:8/231: write d5/d9/dc/dd/d2d/f31 [3721098,90864] 0 2024-03-21T10:08:43.861 INFO:tasks.workunit.client.0.smithi153.stdout:8/232: write d5/d9/dc/dd/d20/d26/d24/f32 [991118,35050] 0 2024-03-21T10:08:43.865 INFO:tasks.workunit.client.0.smithi153.stdout:8/233: creat d5/d9/dc/dd/d2d/f3f x:0 0 0 2024-03-21T10:08:43.866 INFO:tasks.workunit.client.0.smithi153.stdout:8/234: stat d5/d9/dc/l3e 0 2024-03-21T10:08:43.868 INFO:tasks.workunit.client.0.smithi153.stdout:9/215: rename d1/d12/d1f to d1/d43/d44 0 2024-03-21T10:08:43.871 INFO:tasks.workunit.client.0.smithi153.stdout:9/216: symlink d1/d43/l45 0 2024-03-21T10:08:43.872 INFO:tasks.workunit.client.0.smithi153.stdout:8/235: link d5/d9/dc/dd/d20/d26/d24/l27 d5/d1b/l40 0 2024-03-21T10:08:43.876 INFO:tasks.workunit.client.0.smithi153.stdout:8/236: creat d5/d1b/f41 x:0 0 0 2024-03-21T10:08:43.877 INFO:tasks.workunit.client.0.smithi153.stdout:8/237: fdatasync d5/d1b/f41 0 2024-03-21T10:08:43.878 INFO:tasks.workunit.client.0.smithi153.stdout:7/133: sync 2024-03-21T10:08:43.878 INFO:tasks.workunit.client.0.smithi153.stdout:9/217: rename d1/d12 to d1/d43/d44/d46 0 2024-03-21T10:08:43.879 INFO:tasks.workunit.client.0.smithi153.stdout:8/238: write d5/d9/f11 [3714152,75940] 0 2024-03-21T10:08:43.882 INFO:tasks.workunit.client.0.smithi153.stdout:8/239: symlink d5/d9/dc/dd/d2d/l42 0 2024-03-21T10:08:43.890 INFO:tasks.workunit.client.0.smithi153.stdout:9/218: rename d1/d43/d44/f2d to d1/d43/d44/d46/d1c/d24/d41/f47 0 2024-03-21T10:08:43.891 INFO:tasks.workunit.client.0.smithi153.stdout:8/240: rename d5/d9/dc to d5/d9/d43 0 2024-03-21T10:08:43.893 INFO:tasks.workunit.client.0.smithi153.stdout:7/134: dwrite d4/d11/d24/f29 [0,4194304] 0 2024-03-21T10:08:43.895 INFO:tasks.workunit.client.0.smithi153.stdout:9/219: rename d1/d43/d44 to d1/d43/d44/d46/d48 22 2024-03-21T10:08:43.897 INFO:tasks.workunit.client.0.smithi153.stdout:7/135: mkdir d4/d2a 0 2024-03-21T10:08:43.898 INFO:tasks.workunit.client.0.smithi153.stdout:8/241: rmdir d5/d9 39 2024-03-21T10:08:43.902 INFO:tasks.workunit.client.0.smithi153.stdout:8/242: write d5/d9/f11 [4715638,69081] 0 2024-03-21T10:08:43.903 INFO:tasks.workunit.client.0.smithi153.stdout:7/136: rename d4/d1b/l1c to d4/l2b 0 2024-03-21T10:08:43.905 INFO:tasks.workunit.client.0.smithi153.stdout:7/137: rmdir d4/d1b/d20 39 2024-03-21T10:08:43.907 INFO:tasks.workunit.client.0.smithi153.stdout:8/243: mknod d5/c44 0 2024-03-21T10:08:43.912 INFO:tasks.workunit.client.0.smithi153.stdout:9/220: dwrite d1/d43/d44/d46/f37 [0,4194304] 0 2024-03-21T10:08:43.914 INFO:tasks.workunit.client.0.smithi153.stdout:6/420: sync 2024-03-21T10:08:43.914 INFO:tasks.workunit.client.0.smithi153.stdout:3/243: sync 2024-03-21T10:08:43.914 INFO:tasks.workunit.client.0.smithi153.stdout:2/267: sync 2024-03-21T10:08:43.914 INFO:tasks.workunit.client.0.smithi153.stdout:4/272: sync 2024-03-21T10:08:43.915 INFO:tasks.workunit.client.0.smithi153.stdout:1/231: sync 2024-03-21T10:08:43.915 INFO:tasks.workunit.client.0.smithi153.stdout:5/211: sync 2024-03-21T10:08:43.915 INFO:tasks.workunit.client.0.smithi153.stdout:0/159: sync 2024-03-21T10:08:43.916 INFO:tasks.workunit.client.0.smithi153.stdout:8/244: truncate d5/d9/d43/dd/d20/d26/d24/f32 1987426 0 2024-03-21T10:08:43.919 INFO:tasks.workunit.client.0.smithi153.stdout:3/244: chown dd/f43 1 1 2024-03-21T10:08:43.919 INFO:tasks.workunit.client.0.smithi153.stdout:1/232: chown d6/f31 238 1 2024-03-21T10:08:43.922 INFO:tasks.workunit.client.0.smithi153.stdout:5/212: readlink d4/d5/de/l20 0 2024-03-21T10:08:43.924 INFO:tasks.workunit.client.0.smithi153.stdout:9/221: creat d1/d43/d44/d46/d1c/d24/f49 x:0 0 0 2024-03-21T10:08:43.931 INFO:tasks.workunit.client.0.smithi153.stdout:5/213: fdatasync d4/fa 0 2024-03-21T10:08:43.933 INFO:tasks.workunit.client.0.smithi153.stdout:9/222: mknod d1/d43/c4a 0 2024-03-21T10:08:43.936 INFO:tasks.workunit.client.0.smithi153.stdout:3/245: truncate dd/f15 137864 0 2024-03-21T10:08:43.941 INFO:tasks.workunit.client.0.smithi153.stdout:7/138: dwrite d4/f10 [0,4194304] 0 2024-03-21T10:08:43.948 INFO:tasks.workunit.client.0.smithi153.stdout:5/214: symlink d4/d5/de/d2f/d2c/l45 0 2024-03-21T10:08:43.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:43 smithi174 ceph-mon[37496]: pgmap v8: 65 pgs: 65 active+clean; 369 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 28 MiB/s wr, 116 op/s 2024-03-21T10:08:43.952 INFO:tasks.workunit.client.0.smithi153.stdout:3/246: creat dd/d41/f54 x:0 0 0 2024-03-21T10:08:43.954 INFO:tasks.workunit.client.0.smithi153.stdout:1/233: link d6/d13/f1d d6/d10/d2c/f47 0 2024-03-21T10:08:43.956 INFO:tasks.workunit.client.0.smithi153.stdout:3/247: read f7 [681221,35545] 0 2024-03-21T10:08:43.962 INFO:tasks.workunit.client.0.smithi153.stdout:6/421: dwrite d1f/d2b/d33/d40/f5d [4194304,4194304] 0 2024-03-21T10:08:43.963 INFO:tasks.workunit.client.0.smithi153.stdout:4/273: dwrite d11/d18/f1d [4194304,4194304] 0 2024-03-21T10:08:43.965 INFO:tasks.workunit.client.0.smithi153.stdout:8/245: dread d5/d9/f11 [0,4194304] 0 2024-03-21T10:08:43.967 INFO:tasks.workunit.client.0.smithi153.stdout:9/223: write d1/d43/d44/d46/f19 [2349029,52954] 0 2024-03-21T10:08:43.968 INFO:tasks.workunit.client.0.smithi153.stdout:6/422: chown d1f/d2b/d33/d40/d51 17444 1 2024-03-21T10:08:43.970 INFO:tasks.workunit.client.0.smithi153.stdout:3/248: mknod dd/c55 0 2024-03-21T10:08:43.970 INFO:tasks.workunit.client.0.smithi153.stdout:8/246: write d5/d1b/f37 [3697010,42970] 0 2024-03-21T10:08:43.971 INFO:tasks.workunit.client.0.smithi153.stdout:2/268: dread dd/d15/d18/d2c/f4e [0,4194304] 0 2024-03-21T10:08:43.980 INFO:tasks.workunit.client.0.smithi153.stdout:1/234: symlink d6/d13/l48 0 2024-03-21T10:08:43.980 INFO:tasks.workunit.client.0.smithi153.stdout:5/215: creat d4/f46 x:0 0 0 2024-03-21T10:08:43.985 INFO:tasks.workunit.client.0.smithi153.stdout:6/423: rmdir d1f 39 2024-03-21T10:08:43.988 INFO:tasks.workunit.client.0.smithi153.stdout:7/139: dwrite d4/f1a [0,4194304] 0 2024-03-21T10:08:43.988 INFO:tasks.workunit.client.0.smithi153.stdout:4/274: creat d11/d2d/d3d/f56 x:0 0 0 2024-03-21T10:08:43.994 INFO:tasks.workunit.client.0.smithi153.stdout:3/249: mkdir dd/d1e/d45/d56 0 2024-03-21T10:08:43.994 INFO:tasks.workunit.client.0.smithi153.stdout:8/247: mkdir d5/d9/d43/dd/d2d/d45 0 2024-03-21T10:08:43.994 INFO:tasks.workunit.client.0.smithi153.stdout:2/269: mkdir dd/d3b/d50/d59 0 2024-03-21T10:08:43.996 INFO:tasks.workunit.client.0.smithi153.stdout:2/270: fdatasync f5 0 2024-03-21T10:08:43.999 INFO:tasks.workunit.client.0.smithi153.stdout:8/248: write d5/d9/d43/dd/d20/d26/d24/f28 [821282,8991] 0 2024-03-21T10:08:44.000 INFO:tasks.workunit.client.0.smithi153.stdout:0/160: dread d4/db/f13 [0,4194304] 0 2024-03-21T10:08:44.000 INFO:tasks.workunit.client.1.smithi174.stdout:7/0: sync 2024-03-21T10:08:44.003 INFO:tasks.workunit.client.1.smithi174.stdout:7/1: mkdir d0 0 2024-03-21T10:08:44.003 INFO:tasks.workunit.client.0.smithi153.stdout:8/249: truncate d5/f2f 9423490 0 2024-03-21T10:08:44.004 INFO:tasks.workunit.client.0.smithi153.stdout:7/140: chown d4/c6 53200973 1 2024-03-21T10:08:44.006 INFO:tasks.workunit.client.1.smithi174.stdout:7/2: creat d0/f1 x:0 0 0 2024-03-21T10:08:44.006 INFO:tasks.workunit.client.0.smithi153.stdout:8/250: fdatasync d5/d9/f17 0 2024-03-21T10:08:44.007 INFO:tasks.workunit.client.1.smithi174.stdout:7/3: write d0/f1 [333422,13249] 0 2024-03-21T10:08:44.008 INFO:tasks.workunit.client.0.smithi153.stdout:8/251: fsync d5/d9/d43/f2c 0 2024-03-21T10:08:44.010 INFO:tasks.workunit.client.1.smithi174.stdout:7/4: truncate d0/f1 431741 0 2024-03-21T10:08:44.013 INFO:tasks.workunit.client.1.smithi174.stdout:7/5: mkdir d0/d2 0 2024-03-21T10:08:44.015 INFO:tasks.workunit.client.1.smithi174.stdout:7/6: fdatasync d0/f1 0 2024-03-21T10:08:44.016 INFO:tasks.workunit.client.0.smithi153.stdout:9/224: mkdir d1/d43/d44/d46/d1c/d4b 0 2024-03-21T10:08:44.018 INFO:tasks.workunit.client.0.smithi153.stdout:5/216: mknod d4/c47 0 2024-03-21T10:08:44.020 INFO:tasks.workunit.client.1.smithi174.stdout:7/7: dread d0/f1 [0,4194304] 0 2024-03-21T10:08:44.021 INFO:tasks.workunit.client.1.smithi174.stdout:7/8: chown d0/d2 12007031 1 2024-03-21T10:08:44.021 INFO:tasks.workunit.client.0.smithi153.stdout:0/161: creat d4/d16/d1f/f31 x:0 0 0 2024-03-21T10:08:44.024 INFO:tasks.workunit.client.1.smithi174.stdout:7/9: creat d0/d2/f3 x:0 0 0 2024-03-21T10:08:44.026 INFO:tasks.workunit.client.0.smithi153.stdout:6/424: symlink d1f/d2b/d33/l9a 0 2024-03-21T10:08:44.027 INFO:tasks.workunit.client.0.smithi153.stdout:4/275: dread d11/f26 [0,4194304] 0 2024-03-21T10:08:44.028 INFO:tasks.workunit.client.0.smithi153.stdout:6/425: readlink d1f/d2f/d59/d70/l7e 0 2024-03-21T10:08:44.034 INFO:tasks.workunit.client.0.smithi153.stdout:3/250: write dd/f1c [281063,105297] 0 2024-03-21T10:08:44.034 INFO:tasks.workunit.client.0.smithi153.stdout:9/225: stat d1/d3b 0 2024-03-21T10:08:44.038 INFO:tasks.workunit.client.0.smithi153.stdout:5/217: rename d4/d5/d3a to d4/d5/de/d44/d48 0 2024-03-21T10:08:44.040 INFO:tasks.workunit.client.0.smithi153.stdout:5/218: readlink d4/d5/de/d2f/l37 0 2024-03-21T10:08:44.041 INFO:tasks.workunit.client.0.smithi153.stdout:4/276: mkdir d11/d2d/d57 0 2024-03-21T10:08:44.043 INFO:tasks.workunit.client.1.smithi174.stdout:7/10: dwrite d0/d2/f3 [0,4194304] 0 2024-03-21T10:08:44.044 INFO:tasks.workunit.client.0.smithi153.stdout:1/235: getdents d6/d3a 0 2024-03-21T10:08:44.046 INFO:tasks.workunit.client.0.smithi153.stdout:7/141: creat d4/d1b/d20/f2c x:0 0 0 2024-03-21T10:08:44.049 INFO:tasks.workunit.client.0.smithi153.stdout:2/271: truncate dd/f2e 3451202 0 2024-03-21T10:08:44.054 INFO:tasks.workunit.client.0.smithi153.stdout:6/426: rename d1f/f27 to d1f/d2b/d33/d40/d5f/f9b 0 2024-03-21T10:08:44.054 INFO:tasks.workunit.client.1.smithi174.stdout:7/11: rename d0/d2 to d0/d4 0 2024-03-21T10:08:44.055 INFO:tasks.workunit.client.0.smithi153.stdout:5/219: write d4/d5/df/f30 [4242042,37233] 0 2024-03-21T10:08:44.060 INFO:tasks.workunit.client.0.smithi153.stdout:3/251: mknod dd/d1e/d45/d56/c57 0 2024-03-21T10:08:44.061 INFO:tasks.workunit.client.0.smithi153.stdout:1/236: stat d6/d13/f3f 0 2024-03-21T10:08:44.062 INFO:tasks.workunit.client.0.smithi153.stdout:1/237: dread - d6/d13/f45 zero size 2024-03-21T10:08:44.062 INFO:tasks.workunit.client.0.smithi153.stdout:3/252: chown dd/c31 3793 1 2024-03-21T10:08:44.062 INFO:tasks.workunit.client.0.smithi153.stdout:9/226: symlink d1/d43/d44/d46/d1c/l4c 0 2024-03-21T10:08:44.063 INFO:tasks.workunit.client.0.smithi153.stdout:2/272: rename dd/d15/c20 to dd/d3b/d50/c5a 0 2024-03-21T10:08:44.063 INFO:tasks.workunit.client.0.smithi153.stdout:3/253: fsync f8 0 2024-03-21T10:08:44.064 INFO:tasks.workunit.client.0.smithi153.stdout:0/162: getdents d4/d16/d19/d30 0 2024-03-21T10:08:44.065 INFO:tasks.workunit.client.0.smithi153.stdout:5/220: mknod d4/d5/de/d2f/d2c/c49 0 2024-03-21T10:08:44.066 INFO:tasks.workunit.client.0.smithi153.stdout:7/142: creat d4/d2a/f2d x:0 0 0 2024-03-21T10:08:44.068 INFO:tasks.workunit.client.0.smithi153.stdout:5/221: fdatasync d4/d5/df/f30 0 2024-03-21T10:08:44.070 INFO:tasks.workunit.client.1.smithi174.stdout:7/12: fsync d0/d4/f3 0 2024-03-21T10:08:44.071 INFO:tasks.workunit.client.0.smithi153.stdout:5/222: chown d4/c47 1193350 1 2024-03-21T10:08:44.072 INFO:tasks.workunit.client.0.smithi153.stdout:7/143: truncate d4/d1b/d20/f2c 2408 0 2024-03-21T10:08:44.073 INFO:tasks.workunit.client.0.smithi153.stdout:0/163: rmdir d4/d16/d1f 39 2024-03-21T10:08:44.073 INFO:tasks.workunit.client.1.smithi174.stdout:7/13: symlink d0/d4/l5 0 2024-03-21T10:08:44.074 INFO:tasks.workunit.client.0.smithi153.stdout:2/273: rmdir dd/d15/d18/d3a/d3d 39 2024-03-21T10:08:44.079 INFO:tasks.workunit.client.1.smithi174.stdout:7/14: rename d0/d4/l5 to d0/l6 0 2024-03-21T10:08:44.080 INFO:tasks.workunit.client.0.smithi153.stdout:4/277: getdents d11/d2d/d3d 0 2024-03-21T10:08:44.080 INFO:tasks.workunit.client.0.smithi153.stdout:9/227: mkdir d1/d43/d44/d46/d4d 0 2024-03-21T10:08:44.082 INFO:tasks.workunit.client.1.smithi174.stdout:7/15: dread d0/f1 [0,4194304] 0 2024-03-21T10:08:44.087 INFO:tasks.workunit.client.0.smithi153.stdout:3/254: dwrite dd/f2c [0,4194304] 0 2024-03-21T10:08:44.090 INFO:tasks.workunit.client.0.smithi153.stdout:0/164: write d4/d16/d1f/f24 [395846,15132] 0 2024-03-21T10:08:44.093 INFO:tasks.workunit.client.0.smithi153.stdout:9/228: unlink d1/d43/d44/d46/d1c/f36 0 2024-03-21T10:08:44.093 INFO:tasks.workunit.client.0.smithi153.stdout:5/223: symlink d4/d5/de/l4a 0 2024-03-21T10:08:44.095 INFO:tasks.workunit.client.1.smithi174.stdout:7/16: dread d0/d4/f3 [0,4194304] 0 2024-03-21T10:08:44.098 INFO:tasks.workunit.client.0.smithi153.stdout:4/278: mkdir d11/d58 0 2024-03-21T10:08:44.098 INFO:tasks.workunit.client.1.smithi174.stdout:7/17: truncate d0/f1 752613 0 2024-03-21T10:08:44.099 INFO:tasks.workunit.client.0.smithi153.stdout:5/224: write d4/d5/d16/f1e [948117,127721] 0 2024-03-21T10:08:44.100 INFO:tasks.workunit.client.1.smithi174.stdout:7/18: write d0/f1 [1622606,123843] 0 2024-03-21T10:08:44.105 INFO:tasks.workunit.client.0.smithi153.stdout:2/274: dwrite f5 [0,4194304] 0 2024-03-21T10:08:44.106 INFO:tasks.workunit.client.0.smithi153.stdout:3/255: mkdir dd/d1e/d45/d56/d58 0 2024-03-21T10:08:44.107 INFO:tasks.workunit.client.1.smithi174.stdout:7/19: rename d0/d4/f3 to d0/d4/f7 0 2024-03-21T10:08:44.111 INFO:tasks.workunit.client.1.smithi174.stdout:6/15: sync 2024-03-21T10:08:44.111 INFO:tasks.workunit.client.1.smithi174.stdout:1/44: sync 2024-03-21T10:08:44.111 INFO:tasks.workunit.client.1.smithi174.stdout:4/72: sync 2024-03-21T10:08:44.111 INFO:tasks.workunit.client.1.smithi174.stdout:5/27: sync 2024-03-21T10:08:44.111 INFO:tasks.workunit.client.1.smithi174.stdout:3/17: sync 2024-03-21T10:08:44.112 INFO:tasks.workunit.client.1.smithi174.stdout:0/23: sync 2024-03-21T10:08:44.112 INFO:tasks.workunit.client.0.smithi153.stdout:6/427: truncate d1f/d2f/f7d 1331348 0 2024-03-21T10:08:44.113 INFO:tasks.workunit.client.1.smithi174.stdout:8/7: sync 2024-03-21T10:08:44.113 INFO:tasks.workunit.client.1.smithi174.stdout:9/85: sync 2024-03-21T10:08:44.113 INFO:tasks.workunit.client.1.smithi174.stdout:2/88: sync 2024-03-21T10:08:44.113 INFO:tasks.workunit.client.1.smithi174.stdout:7/20: write d0/d4/f7 [4819706,69849] 0 2024-03-21T10:08:44.115 INFO:tasks.workunit.client.1.smithi174.stdout:7/21: stat d0/d4 0 2024-03-21T10:08:44.117 INFO:tasks.workunit.client.0.smithi153.stdout:2/275: creat dd/d3b/f5b x:0 0 0 2024-03-21T10:08:44.118 INFO:tasks.workunit.client.1.smithi174.stdout:4/73: chown d4/d19 3391211 1 2024-03-21T10:08:44.118 INFO:tasks.workunit.client.1.smithi174.stdout:3/18: dread - d0/f2 zero size 2024-03-21T10:08:44.119 INFO:tasks.workunit.client.1.smithi174.stdout:8/8: symlink l3 0 2024-03-21T10:08:44.120 INFO:tasks.workunit.client.1.smithi174.stdout:0/24: creat f6 x:0 0 0 2024-03-21T10:08:44.120 INFO:tasks.workunit.client.0.smithi153.stdout:5/225: creat d4/d8/d3b/f4b x:0 0 0 2024-03-21T10:08:44.123 INFO:tasks.workunit.client.0.smithi153.stdout:9/229: mkdir d1/d4e 0 2024-03-21T10:08:44.124 INFO:tasks.workunit.client.1.smithi174.stdout:1/45: rename d4/d7 to d4/d7/da 22 2024-03-21T10:08:44.125 INFO:tasks.workunit.client.0.smithi153.stdout:0/165: dwrite d4/db/f11 [0,4194304] 0 2024-03-21T10:08:44.127 INFO:tasks.workunit.client.1.smithi174.stdout:9/86: creat d2/f18 x:0 0 0 2024-03-21T10:08:44.127 INFO:tasks.workunit.client.0.smithi153.stdout:7/144: dread d4/f9 [0,4194304] 0 2024-03-21T10:08:44.128 INFO:tasks.workunit.client.1.smithi174.stdout:3/19: creat d0/f7 x:0 0 0 2024-03-21T10:08:44.129 INFO:tasks.workunit.client.1.smithi174.stdout:9/87: stat d2/c5 0 2024-03-21T10:08:44.130 INFO:tasks.workunit.client.0.smithi153.stdout:4/279: dwrite d11/d18/f4b [4194304,4194304] 0 2024-03-21T10:08:44.131 INFO:tasks.workunit.client.1.smithi174.stdout:0/25: unlink f0 0 2024-03-21T10:08:44.131 INFO:tasks.workunit.client.0.smithi153.stdout:7/145: write d4/fa [2936887,1010] 0 2024-03-21T10:08:44.134 INFO:tasks.workunit.client.0.smithi153.stdout:6/428: creat d1f/d99/f9c x:0 0 0 2024-03-21T10:08:44.134 INFO:tasks.workunit.client.1.smithi174.stdout:3/20: dread - d0/f7 zero size 2024-03-21T10:08:44.135 INFO:tasks.workunit.client.0.smithi153.stdout:0/166: unlink d4/l25 0 2024-03-21T10:08:44.138 INFO:tasks.workunit.client.0.smithi153.stdout:1/238: dwrite d6/f15 [0,4194304] 0 2024-03-21T10:08:44.139 INFO:tasks.workunit.client.1.smithi174.stdout:6/16: rename d0/l1 to d0/l9 0 2024-03-21T10:08:44.141 INFO:tasks.workunit.client.1.smithi174.stdout:8/9: truncate f0 1013797 0 2024-03-21T10:08:44.144 INFO:tasks.workunit.client.1.smithi174.stdout:8/10: readlink l3 0 2024-03-21T10:08:44.146 INFO:tasks.workunit.client.1.smithi174.stdout:0/26: rename f5 to f7 0 2024-03-21T10:08:44.147 INFO:tasks.workunit.client.0.smithi153.stdout:7/146: mknod d4/d2a/c2e 0 2024-03-21T10:08:44.148 INFO:tasks.workunit.client.1.smithi174.stdout:9/88: readlink d2/ld 0 2024-03-21T10:08:44.151 INFO:tasks.workunit.client.0.smithi153.stdout:7/147: readlink d4/l21 0 2024-03-21T10:08:44.151 INFO:tasks.workunit.client.1.smithi174.stdout:1/46: chown d4/d5/l6 27 1 2024-03-21T10:08:44.153 INFO:tasks.workunit.client.1.smithi174.stdout:3/21: creat d0/f8 x:0 0 0 2024-03-21T10:08:44.155 INFO:tasks.workunit.client.1.smithi174.stdout:8/11: write f2 [409446,21658] 0 2024-03-21T10:08:44.156 INFO:tasks.workunit.client.0.smithi153.stdout:3/256: dwrite f7 [0,4194304] 0 2024-03-21T10:08:44.158 INFO:tasks.workunit.client.1.smithi174.stdout:2/89: dwrite d5/f10 [0,4194304] 0 2024-03-21T10:08:44.158 INFO:tasks.workunit.client.1.smithi174.stdout:4/74: dread d4/fe [0,4194304] 0 2024-03-21T10:08:44.160 INFO:tasks.workunit.client.1.smithi174.stdout:5/28: dwrite f0 [0,4194304] 0 2024-03-21T10:08:44.162 INFO:tasks.workunit.client.1.smithi174.stdout:5/29: stat d1/l5 0 2024-03-21T10:08:44.163 INFO:tasks.workunit.client.1.smithi174.stdout:5/30: stat f0 0 2024-03-21T10:08:44.164 INFO:tasks.workunit.client.1.smithi174.stdout:4/75: write d4/f18 [104635,19293] 0 2024-03-21T10:08:44.165 INFO:tasks.workunit.client.0.smithi153.stdout:5/226: creat d4/d5/de/f4c x:0 0 0 2024-03-21T10:08:44.166 INFO:tasks.workunit.client.0.smithi153.stdout:7/148: mkdir d4/d14/d2f 0 2024-03-21T10:08:44.171 INFO:tasks.workunit.client.1.smithi174.stdout:1/47: creat d4/fb x:0 0 0 2024-03-21T10:08:44.172 INFO:tasks.workunit.client.0.smithi153.stdout:6/429: write d1f/f35 [124416,1974] 0 2024-03-21T10:08:44.174 INFO:tasks.workunit.client.1.smithi174.stdout:1/48: chown d4/d7 974 1 2024-03-21T10:08:44.174 INFO:tasks.workunit.client.0.smithi153.stdout:0/167: creat d4/f32 x:0 0 0 2024-03-21T10:08:44.175 INFO:tasks.workunit.client.0.smithi153.stdout:7/149: unlink d4/f9 0 2024-03-21T10:08:44.176 INFO:tasks.workunit.client.1.smithi174.stdout:3/22: symlink d0/l9 0 2024-03-21T10:08:44.179 INFO:tasks.workunit.client.0.smithi153.stdout:9/230: dwrite d1/d4/f1a [0,4194304] 0 2024-03-21T10:08:44.179 INFO:tasks.workunit.client.0.smithi153.stdout:7/150: chown d4/c1f 10670544 1 2024-03-21T10:08:44.181 INFO:tasks.workunit.client.1.smithi174.stdout:6/17: symlink d0/d3/d5/d6/la 0 2024-03-21T10:08:44.183 INFO:tasks.workunit.client.0.smithi153.stdout:7/151: fdatasync d4/fa 0 2024-03-21T10:08:44.184 INFO:tasks.workunit.client.0.smithi153.stdout:4/280: dwrite f1 [4194304,4194304] 0 2024-03-21T10:08:44.185 INFO:tasks.workunit.client.0.smithi153.stdout:2/276: dwrite dd/d15/d18/d2c/f4e [0,4194304] 0 2024-03-21T10:08:44.186 INFO:tasks.workunit.client.0.smithi153.stdout:7/152: write d4/fd [1203858,114380] 0 2024-03-21T10:08:44.187 INFO:tasks.workunit.client.0.smithi153.stdout:5/227: unlink d4/d5/de/d2f/d2c/l45 0 2024-03-21T10:08:44.188 INFO:tasks.workunit.client.1.smithi174.stdout:8/12: creat f4 x:0 0 0 2024-03-21T10:08:44.190 INFO:tasks.workunit.client.0.smithi153.stdout:7/153: truncate d4/f25 818862 0 2024-03-21T10:08:44.192 INFO:tasks.workunit.client.1.smithi174.stdout:7/22: truncate d0/d4/f7 4416421 0 2024-03-21T10:08:44.192 INFO:tasks.workunit.client.0.smithi153.stdout:3/257: getdents dd/d1e/d2f/d35 0 2024-03-21T10:08:44.192 INFO:tasks.workunit.client.1.smithi174.stdout:2/90: write f2 [758232,103307] 0 2024-03-21T10:08:44.197 INFO:tasks.workunit.client.0.smithi153.stdout:0/168: mknod d4/db/c33 0 2024-03-21T10:08:44.202 INFO:tasks.workunit.client.1.smithi174.stdout:5/31: symlink d1/l7 0 2024-03-21T10:08:44.204 INFO:tasks.workunit.client.1.smithi174.stdout:1/49: symlink d4/lc 0 2024-03-21T10:08:44.204 INFO:tasks.workunit.client.1.smithi174.stdout:5/32: chown d1/l7 215 1 2024-03-21T10:08:44.205 INFO:tasks.workunit.client.1.smithi174.stdout:3/23: creat d0/fa x:0 0 0 2024-03-21T10:08:44.209 INFO:tasks.workunit.client.1.smithi174.stdout:5/33: write d1/f3 [8976993,8136] 0 2024-03-21T10:08:44.211 INFO:tasks.workunit.client.1.smithi174.stdout:8/13: creat f5 x:0 0 0 2024-03-21T10:08:44.211 INFO:tasks.workunit.client.1.smithi174.stdout:6/18: mknod d0/cb 0 2024-03-21T10:08:44.211 INFO:tasks.workunit.client.0.smithi153.stdout:4/281: dwrite d11/d2d/f2f [0,4194304] 0 2024-03-21T10:08:44.213 INFO:tasks.workunit.client.1.smithi174.stdout:7/23: creat d0/d4/f8 x:0 0 0 2024-03-21T10:08:44.216 INFO:tasks.workunit.client.0.smithi153.stdout:8/252: sync 2024-03-21T10:08:44.217 INFO:tasks.workunit.client.1.smithi174.stdout:2/91: creat d5/d8/f22 x:0 0 0 2024-03-21T10:08:44.219 INFO:tasks.workunit.client.0.smithi153.stdout:8/253: read d5/f2f [8236160,102281] 0 2024-03-21T10:08:44.220 INFO:tasks.workunit.client.1.smithi174.stdout:9/89: dwrite d2/f9 [0,4194304] 0 2024-03-21T10:08:44.220 INFO:tasks.workunit.client.1.smithi174.stdout:3/24: creat d0/fb x:0 0 0 2024-03-21T10:08:44.221 INFO:tasks.workunit.client.1.smithi174.stdout:1/50: creat d4/d5/fd x:0 0 0 2024-03-21T10:08:44.225 INFO:tasks.workunit.client.1.smithi174.stdout:8/14: symlink l6 0 2024-03-21T10:08:44.226 INFO:tasks.workunit.client.1.smithi174.stdout:1/51: fsync d4/d5/f9 0 2024-03-21T10:08:44.227 INFO:tasks.workunit.client.1.smithi174.stdout:2/92: write d5/d8/ff [993370,15507] 0 2024-03-21T10:08:44.228 INFO:tasks.workunit.client.1.smithi174.stdout:6/19: unlink d0/cb 0 2024-03-21T10:08:44.228 INFO:tasks.workunit.client.1.smithi174.stdout:8/15: truncate f2 926500 0 2024-03-21T10:08:44.228 INFO:tasks.workunit.client.1.smithi174.stdout:8/16: rmdir - no directory 2024-03-21T10:08:44.231 INFO:tasks.workunit.client.1.smithi174.stdout:3/25: mkdir d0/dc 0 2024-03-21T10:08:44.231 INFO:tasks.workunit.client.0.smithi153.stdout:5/228: creat d4/d5/de/d38/f4d x:0 0 0 2024-03-21T10:08:44.232 INFO:tasks.workunit.client.1.smithi174.stdout:9/90: mknod d2/d11/c19 0 2024-03-21T10:08:44.234 INFO:tasks.workunit.client.1.smithi174.stdout:6/20: read - d0/d3/f8 zero size 2024-03-21T10:08:44.235 INFO:tasks.workunit.client.1.smithi174.stdout:1/52: chown d4/d5/fd 42 1 2024-03-21T10:08:44.236 INFO:tasks.workunit.client.1.smithi174.stdout:3/26: fdatasync d0/f7 0 2024-03-21T10:08:44.238 INFO:tasks.workunit.client.1.smithi174.stdout:2/93: creat d5/d8/f23 x:0 0 0 2024-03-21T10:08:44.240 INFO:tasks.workunit.client.0.smithi153.stdout:8/254: dread d5/f8 [0,4194304] 0 2024-03-21T10:08:44.241 INFO:tasks.workunit.client.0.smithi153.stdout:6/430: link d1f/f4e d1f/d2f/d59/d70/f9d 0 2024-03-21T10:08:44.241 INFO:tasks.workunit.client.1.smithi174.stdout:2/94: readlink d5/d8/l15 0 2024-03-21T10:08:44.241 INFO:tasks.workunit.client.1.smithi174.stdout:9/91: creat d2/f1a x:0 0 0 2024-03-21T10:08:44.245 INFO:tasks.workunit.client.1.smithi174.stdout:2/95: write d5/d8/f1c [836938,22836] 0 2024-03-21T10:08:44.249 INFO:tasks.workunit.client.1.smithi174.stdout:5/34: dread d1/f3 [4194304,4194304] 0 2024-03-21T10:08:44.250 INFO:tasks.workunit.client.0.smithi153.stdout:0/169: rename d4/db/cf to d4/d16/d1f/d22/c34 0 2024-03-21T10:08:44.252 INFO:tasks.workunit.client.1.smithi174.stdout:1/53: mknod d4/d7/ce 0 2024-03-21T10:08:44.255 INFO:tasks.workunit.client.1.smithi174.stdout:9/92: creat d2/f1b x:0 0 0 2024-03-21T10:08:44.259 INFO:tasks.workunit.client.1.smithi174.stdout:1/54: write d4/d5/fd [787443,74156] 0 2024-03-21T10:08:44.259 INFO:tasks.workunit.client.1.smithi174.stdout:6/21: rename d0/l9 to d0/d3/d5/d6/lc 0 2024-03-21T10:08:44.260 INFO:tasks.workunit.client.0.smithi153.stdout:2/277: symlink dd/d15/d18/d3a/l5c 0 2024-03-21T10:08:44.263 INFO:tasks.workunit.client.0.smithi153.stdout:6/431: creat d1f/d2b/d33/d40/f9e x:0 0 0 2024-03-21T10:08:44.263 INFO:tasks.workunit.client.0.smithi153.stdout:5/229: creat d4/d5/de/d2f/f4e x:0 0 0 2024-03-21T10:08:44.265 INFO:tasks.workunit.client.1.smithi174.stdout:9/93: rename d2/cb to d2/c1c 0 2024-03-21T10:08:44.266 INFO:tasks.workunit.client.0.smithi153.stdout:8/255: rename d5/d9/f17 to d5/d9/d43/dd/d20/d26/d33/f46 0 2024-03-21T10:08:44.268 INFO:tasks.workunit.client.1.smithi174.stdout:1/55: creat d4/d7/ff x:0 0 0 2024-03-21T10:08:44.269 INFO:tasks.workunit.client.0.smithi153.stdout:0/170: creat d4/d16/d1f/f35 x:0 0 0 2024-03-21T10:08:44.270 INFO:tasks.workunit.client.1.smithi174.stdout:2/96: creat d5/f24 x:0 0 0 2024-03-21T10:08:44.271 INFO:tasks.workunit.client.0.smithi153.stdout:7/154: creat d4/f30 x:0 0 0 2024-03-21T10:08:44.273 INFO:tasks.workunit.client.0.smithi153.stdout:3/258: getdents dd/d1e/d3d 0 2024-03-21T10:08:44.273 INFO:tasks.workunit.client.0.smithi153.stdout:0/171: stat d4/d16/d1f/d22/c26 0 2024-03-21T10:08:44.274 INFO:tasks.workunit.client.1.smithi174.stdout:9/94: mkdir d2/d11/d1d 0 2024-03-21T10:08:44.275 INFO:tasks.workunit.client.0.smithi153.stdout:0/172: fsync d4/f28 0 2024-03-21T10:08:44.276 INFO:tasks.workunit.client.0.smithi153.stdout:6/432: creat d1f/d93/f9f x:0 0 0 2024-03-21T10:08:44.278 INFO:tasks.workunit.client.1.smithi174.stdout:2/97: mkdir d5/d25 0 2024-03-21T10:08:44.278 INFO:tasks.workunit.client.0.smithi153.stdout:7/155: write d4/f10 [3394527,98860] 0 2024-03-21T10:08:44.282 INFO:tasks.workunit.client.1.smithi174.stdout:9/95: unlink d2/ld 0 2024-03-21T10:08:44.282 INFO:tasks.workunit.client.1.smithi174.stdout:5/35: dwrite f0 [0,4194304] 0 2024-03-21T10:08:44.284 INFO:tasks.workunit.client.1.smithi174.stdout:9/96: stat d2/ff 0 2024-03-21T10:08:44.288 INFO:tasks.workunit.client.1.smithi174.stdout:9/97: chown d2/c13 559066963 1 2024-03-21T10:08:44.289 INFO:tasks.workunit.client.0.smithi153.stdout:3/259: dread - dd/d1e/d2f/f33 zero size 2024-03-21T10:08:44.289 INFO:tasks.workunit.client.0.smithi153.stdout:2/278: getdents dd/d15/d18 0 2024-03-21T10:08:44.291 INFO:tasks.workunit.client.0.smithi153.stdout:0/173: rmdir d4/d16 39 2024-03-21T10:08:44.292 INFO:tasks.workunit.client.1.smithi174.stdout:9/98: read - d2/f17 zero size 2024-03-21T10:08:44.292 INFO:tasks.workunit.client.0.smithi153.stdout:2/279: readlink dd/d15/l48 0 2024-03-21T10:08:44.293 INFO:tasks.workunit.client.0.smithi153.stdout:4/282: dwrite f10 [0,4194304] 0 2024-03-21T10:08:44.295 INFO:tasks.workunit.client.1.smithi174.stdout:4/76: truncate d4/fe 3243833 0 2024-03-21T10:08:44.297 INFO:tasks.workunit.client.1.smithi174.stdout:1/56: dwrite f1 [0,4194304] 0 2024-03-21T10:08:44.297 INFO:tasks.workunit.client.1.smithi174.stdout:9/99: creat d2/f1e x:0 0 0 2024-03-21T10:08:44.298 INFO:tasks.workunit.client.0.smithi153.stdout:9/231: write d1/f27 [492103,41955] 0 2024-03-21T10:08:44.300 INFO:tasks.workunit.client.0.smithi153.stdout:3/260: symlink dd/d1e/l59 0 2024-03-21T10:08:44.300 INFO:tasks.workunit.client.0.smithi153.stdout:2/280: chown fc 2613365 1 2024-03-21T10:08:44.311 INFO:tasks.workunit.client.0.smithi153.stdout:3/261: stat dd/c21 0 2024-03-21T10:08:44.311 INFO:tasks.workunit.client.0.smithi153.stdout:7/156: creat d4/d19/f31 x:0 0 0 2024-03-21T10:08:44.316 INFO:tasks.workunit.client.0.smithi153.stdout:5/230: dwrite d4/d5/de/d2f/f4e [0,4194304] 0 2024-03-21T10:08:44.316 INFO:tasks.workunit.client.0.smithi153.stdout:4/283: mknod d11/d2d/d57/c59 0 2024-03-21T10:08:44.319 INFO:tasks.workunit.client.1.smithi174.stdout:2/98: dwrite d5/d8/f1c [0,4194304] 0 2024-03-21T10:08:44.319 INFO:tasks.workunit.client.0.smithi153.stdout:4/284: stat d11/f26 0 2024-03-21T10:08:44.321 INFO:tasks.workunit.client.1.smithi174.stdout:6/22: getdents d0/d3/d5/d6 0 2024-03-21T10:08:44.321 INFO:tasks.workunit.client.0.smithi153.stdout:6/433: dwrite f11 [0,4194304] 0 2024-03-21T10:08:44.322 INFO:tasks.workunit.client.0.smithi153.stdout:0/174: mknod d4/d16/d1f/d22/c36 0 2024-03-21T10:08:44.322 INFO:tasks.workunit.client.0.smithi153.stdout:4/285: write f10 [79817,32222] 0 2024-03-21T10:08:44.322 INFO:tasks.workunit.client.0.smithi153.stdout:1/239: dread d6/d13/f1d [0,4194304] 0 2024-03-21T10:08:44.323 INFO:tasks.workunit.client.1.smithi174.stdout:9/100: creat d2/f1f x:0 0 0 2024-03-21T10:08:44.327 INFO:tasks.workunit.client.0.smithi153.stdout:2/281: mkdir dd/d3b/d50/d59/d5d 0 2024-03-21T10:08:44.329 INFO:tasks.workunit.client.0.smithi153.stdout:7/157: creat d4/d19/f32 x:0 0 0 2024-03-21T10:08:44.330 INFO:tasks.workunit.client.0.smithi153.stdout:4/286: stat d11/d2d/f30 0 2024-03-21T10:08:44.331 INFO:tasks.workunit.client.1.smithi174.stdout:6/23: mknod d0/cd 0 2024-03-21T10:08:44.332 INFO:tasks.workunit.client.0.smithi153.stdout:5/231: mknod d4/d5/de/d44/c4f 0 2024-03-21T10:08:44.333 INFO:tasks.workunit.client.0.smithi153.stdout:6/434: mkdir d1f/da0 0 2024-03-21T10:08:44.334 INFO:tasks.workunit.client.1.smithi174.stdout:5/36: dwrite f0 [0,4194304] 0 2024-03-21T10:08:44.337 INFO:tasks.workunit.client.1.smithi174.stdout:0/27: sync 2024-03-21T10:08:44.338 INFO:tasks.workunit.client.0.smithi153.stdout:2/282: mkdir dd/d15/d5e 0 2024-03-21T10:08:44.339 INFO:tasks.workunit.client.0.smithi153.stdout:4/287: creat d11/d2d/f5a x:0 0 0 2024-03-21T10:08:44.342 INFO:tasks.workunit.client.0.smithi153.stdout:6/435: symlink d1f/d2f/d59/la1 0 2024-03-21T10:08:44.343 INFO:tasks.workunit.client.0.smithi153.stdout:5/232: mknod d4/c50 0 2024-03-21T10:08:44.344 INFO:tasks.workunit.client.0.smithi153.stdout:0/175: creat d4/f37 x:0 0 0 2024-03-21T10:08:44.346 INFO:tasks.workunit.client.1.smithi174.stdout:6/24: fsync d0/d3/f8 0 2024-03-21T10:08:44.347 INFO:tasks.workunit.client.1.smithi174.stdout:4/77: read d4/fe [1284024,124515] 0 2024-03-21T10:08:44.348 INFO:tasks.workunit.client.0.smithi153.stdout:2/283: fsync fb 0 2024-03-21T10:08:44.348 INFO:tasks.workunit.client.0.smithi153.stdout:5/233: chown d4/d5/de/f18 1 1 2024-03-21T10:08:44.350 INFO:tasks.workunit.client.0.smithi153.stdout:4/288: mkdir d11/d2d/d57/d5b 0 2024-03-21T10:08:44.354 INFO:tasks.workunit.client.0.smithi153.stdout:6/436: creat d1f/d2b/d33/fa2 x:0 0 0 2024-03-21T10:08:44.356 INFO:tasks.workunit.client.0.smithi153.stdout:8/256: truncate d5/d9/d43/dd/d20/d26/d33/f46 3030846 0 2024-03-21T10:08:44.358 INFO:tasks.workunit.client.1.smithi174.stdout:8/17: dwrite f0 [0,4194304] 0 2024-03-21T10:08:44.358 INFO:tasks.workunit.client.1.smithi174.stdout:3/27: dwrite d0/f2 [0,4194304] 0 2024-03-21T10:08:44.358 INFO:tasks.workunit.client.1.smithi174.stdout:1/57: truncate d4/d5/fd 635998 0 2024-03-21T10:08:44.358 INFO:tasks.workunit.client.1.smithi174.stdout:6/25: write d0/d3/f8 [752741,7436] 0 2024-03-21T10:08:44.363 INFO:tasks.workunit.client.1.smithi174.stdout:4/78: symlink d4/l1a 0 2024-03-21T10:08:44.364 INFO:tasks.workunit.client.0.smithi153.stdout:4/289: symlink d11/d18/l5c 0 2024-03-21T10:08:44.366 INFO:tasks.workunit.client.1.smithi174.stdout:8/18: creat f7 x:0 0 0 2024-03-21T10:08:44.368 INFO:tasks.workunit.client.0.smithi153.stdout:0/176: creat d4/d16/d27/f38 x:0 0 0 2024-03-21T10:08:44.370 INFO:tasks.workunit.client.1.smithi174.stdout:4/79: fdatasync d4/f7 0 2024-03-21T10:08:44.371 INFO:tasks.workunit.client.0.smithi153.stdout:6/437: symlink d1f/da0/la3 0 2024-03-21T10:08:44.372 INFO:tasks.workunit.client.1.smithi174.stdout:1/58: write d4/d5/f8 [1054740,35529] 0 2024-03-21T10:08:44.373 INFO:tasks.workunit.client.1.smithi174.stdout:8/19: creat f8 x:0 0 0 2024-03-21T10:08:44.374 INFO:tasks.workunit.client.0.smithi153.stdout:8/257: mkdir d5/d9/d43/dd/d20/d26/d47 0 2024-03-21T10:08:44.379 INFO:tasks.workunit.client.1.smithi174.stdout:3/28: rename d0/l1 to d0/ld 0 2024-03-21T10:08:44.381 INFO:tasks.workunit.client.0.smithi153.stdout:0/177: fdatasync f2 0 2024-03-21T10:08:44.386 INFO:tasks.workunit.client.1.smithi174.stdout:4/80: write d4/f18 [732364,73293] 0 2024-03-21T10:08:44.388 INFO:tasks.workunit.client.0.smithi153.stdout:4/290: mknod d11/d2d/d57/d5b/c5d 0 2024-03-21T10:08:44.388 INFO:tasks.workunit.client.0.smithi153.stdout:5/234: dwrite d4/d5/de/d38/f4d [0,4194304] 0 2024-03-21T10:08:44.389 INFO:tasks.workunit.client.0.smithi153.stdout:9/232: dwrite d1/f1b [0,4194304] 0 2024-03-21T10:08:44.389 INFO:tasks.workunit.client.1.smithi174.stdout:6/26: creat d0/d3/d5/fe x:0 0 0 2024-03-21T10:08:44.390 INFO:tasks.workunit.client.1.smithi174.stdout:8/20: mkdir d9 0 2024-03-21T10:08:44.390 INFO:tasks.workunit.client.1.smithi174.stdout:9/101: rename d2/ff to d2/f20 0 2024-03-21T10:08:44.391 INFO:tasks.workunit.client.1.smithi174.stdout:7/24: dwrite d0/d4/f7 [0,4194304] 0 2024-03-21T10:08:44.392 INFO:tasks.workunit.client.1.smithi174.stdout:7/25: chown d0 627447811 1 2024-03-21T10:08:44.393 INFO:tasks.workunit.client.1.smithi174.stdout:2/99: dwrite d5/f11 [0,4194304] 0 2024-03-21T10:08:44.394 INFO:tasks.workunit.client.0.smithi153.stdout:8/258: symlink d5/d9/d43/dd/l48 0 2024-03-21T10:08:44.398 INFO:tasks.workunit.client.0.smithi153.stdout:9/233: readlink d1/d43/d44/l29 0 2024-03-21T10:08:44.402 INFO:tasks.workunit.client.1.smithi174.stdout:5/37: dwrite f0 [0,4194304] 0 2024-03-21T10:08:44.402 INFO:tasks.workunit.client.0.smithi153.stdout:9/234: fsync d1/f27 0 2024-03-21T10:08:44.404 INFO:tasks.workunit.client.1.smithi174.stdout:7/26: rmdir d0/d4 39 2024-03-21T10:08:44.405 INFO:tasks.workunit.client.0.smithi153.stdout:6/438: mknod d1f/d2b/d33/d40/d51/d98/ca4 0 2024-03-21T10:08:44.407 INFO:tasks.workunit.client.1.smithi174.stdout:6/27: truncate d0/d3/d5/fe 828932 0 2024-03-21T10:08:44.407 INFO:tasks.workunit.client.1.smithi174.stdout:7/27: chown d0 8 1 2024-03-21T10:08:44.408 INFO:tasks.workunit.client.1.smithi174.stdout:0/28: dread f4 [0,4194304] 0 2024-03-21T10:08:44.410 INFO:tasks.workunit.client.1.smithi174.stdout:8/21: mknod d9/ca 0 2024-03-21T10:08:44.414 INFO:tasks.workunit.client.1.smithi174.stdout:5/38: chown f0 29885841 1 2024-03-21T10:08:44.415 INFO:tasks.workunit.client.1.smithi174.stdout:8/22: truncate f8 971132 0 2024-03-21T10:08:44.417 INFO:tasks.workunit.client.1.smithi174.stdout:9/102: symlink d2/d11/d1d/l21 0 2024-03-21T10:08:44.424 INFO:tasks.workunit.client.1.smithi174.stdout:6/28: mkdir d0/d3/d5/df 0 2024-03-21T10:08:44.424 INFO:tasks.workunit.client.1.smithi174.stdout:5/39: chown d1/f3 2 1 2024-03-21T10:08:44.424 INFO:tasks.workunit.client.1.smithi174.stdout:0/29: rename l2 to l8 0 2024-03-21T10:08:44.425 INFO:tasks.workunit.client.1.smithi174.stdout:1/59: dwrite d4/d7/ff [0,4194304] 0 2024-03-21T10:08:44.428 INFO:tasks.workunit.client.1.smithi174.stdout:3/29: readlink d0/ld 0 2024-03-21T10:08:44.429 INFO:tasks.workunit.client.1.smithi174.stdout:0/30: stat f4 0 2024-03-21T10:08:44.439 INFO:tasks.workunit.client.1.smithi174.stdout:8/23: rename l3 to d9/lb 0 2024-03-21T10:08:44.440 INFO:tasks.workunit.client.1.smithi174.stdout:2/100: dwrite d5/d8/f23 [0,4194304] 0 2024-03-21T10:08:44.440 INFO:tasks.workunit.client.1.smithi174.stdout:4/81: dread d4/f15 [0,4194304] 0 2024-03-21T10:08:44.449 INFO:tasks.workunit.client.1.smithi174.stdout:5/40: mknod d1/c8 0 2024-03-21T10:08:44.450 INFO:tasks.workunit.client.1.smithi174.stdout:0/31: mkdir d9 0 2024-03-21T10:08:44.452 INFO:tasks.workunit.client.1.smithi174.stdout:5/41: chown d1/l7 247 1 2024-03-21T10:08:44.454 INFO:tasks.workunit.client.1.smithi174.stdout:6/29: chown d0/d3/d5/d6/lc 53171463 1 2024-03-21T10:08:44.454 INFO:tasks.workunit.client.1.smithi174.stdout:4/82: mknod d4/c1b 0 2024-03-21T10:08:44.454 INFO:tasks.workunit.client.1.smithi174.stdout:5/42: truncate d1/f6 4615173 0 2024-03-21T10:08:44.454 INFO:tasks.workunit.client.1.smithi174.stdout:9/103: chown d2/fc 5 1 2024-03-21T10:08:44.455 INFO:tasks.workunit.client.1.smithi174.stdout:4/83: dread - d4/f13 zero size 2024-03-21T10:08:44.455 INFO:tasks.workunit.client.1.smithi174.stdout:9/104: readlink d2/d11/d1d/l21 0 2024-03-21T10:08:44.459 INFO:tasks.workunit.client.1.smithi174.stdout:7/28: rename d0/f1 to d0/f9 0 2024-03-21T10:08:44.466 INFO:tasks.workunit.client.1.smithi174.stdout:5/43: write d1/f3 [6558402,47460] 0 2024-03-21T10:08:44.466 INFO:tasks.workunit.client.1.smithi174.stdout:9/105: rmdir d2/d11/d1d 39 2024-03-21T10:08:44.469 INFO:tasks.workunit.client.0.smithi153.stdout:8/259: mkdir d5/d1b/d49 0 2024-03-21T10:08:44.469 INFO:tasks.workunit.client.1.smithi174.stdout:1/60: link d4/d7/ce d4/d5/c10 0 2024-03-21T10:08:44.470 INFO:tasks.workunit.client.0.smithi153.stdout:9/235: chown d1/d43/d44/l29 596358 1 2024-03-21T10:08:44.471 INFO:tasks.workunit.client.0.smithi153.stdout:5/235: rename d4/d5/de/f18 to d4/d5/de/d2f/d2c/f51 0 2024-03-21T10:08:44.471 INFO:tasks.workunit.client.1.smithi174.stdout:0/32: mknod d9/ca 0 2024-03-21T10:08:44.472 INFO:tasks.workunit.client.1.smithi174.stdout:1/61: stat d4/d7/ff 0 2024-03-21T10:08:44.472 INFO:tasks.workunit.client.0.smithi153.stdout:7/158: dwrite d4/d2a/f2d [0,4194304] 0 2024-03-21T10:08:44.473 INFO:tasks.workunit.client.1.smithi174.stdout:2/101: creat d5/f26 x:0 0 0 2024-03-21T10:08:44.474 INFO:tasks.workunit.client.0.smithi153.stdout:1/240: dread d6/f17 [4194304,4194304] 0 2024-03-21T10:08:44.475 INFO:tasks.workunit.client.1.smithi174.stdout:3/30: truncate d0/f3 379110 0 2024-03-21T10:08:44.476 INFO:tasks.workunit.client.1.smithi174.stdout:2/102: dread - d5/f24 zero size 2024-03-21T10:08:44.477 INFO:tasks.workunit.client.1.smithi174.stdout:4/84: creat d4/d19/f1c x:0 0 0 2024-03-21T10:08:44.478 INFO:tasks.workunit.client.1.smithi174.stdout:9/106: truncate d2/f1a 595665 0 2024-03-21T10:08:44.478 INFO:tasks.workunit.client.0.smithi153.stdout:8/260: dread - d5/d9/d43/dd/f22 zero size 2024-03-21T10:08:44.481 INFO:tasks.workunit.client.0.smithi153.stdout:1/241: readlink d6/d10/d35/l3d 0 2024-03-21T10:08:44.484 INFO:tasks.workunit.client.0.smithi153.stdout:2/284: dwrite dd/d15/d18/d3a/d3d/f53 [0,4194304] 0 2024-03-21T10:08:44.486 INFO:tasks.workunit.client.1.smithi174.stdout:0/33: truncate f4 1136661 0 2024-03-21T10:08:44.487 INFO:tasks.workunit.client.0.smithi153.stdout:6/439: fsync d1f/d2b/d33/d40/f58 0 2024-03-21T10:08:44.487 INFO:tasks.workunit.client.1.smithi174.stdout:2/103: mkdir d5/d25/d27 0 2024-03-21T10:08:44.488 INFO:tasks.workunit.client.0.smithi153.stdout:6/440: fsync f19 0 2024-03-21T10:08:44.489 INFO:tasks.workunit.client.1.smithi174.stdout:1/62: rmdir d4/d5 39 2024-03-21T10:08:44.492 INFO:tasks.workunit.client.1.smithi174.stdout:4/85: symlink d4/d19/l1d 0 2024-03-21T10:08:44.493 INFO:tasks.workunit.client.0.smithi153.stdout:1/242: symlink d6/d32/l49 0 2024-03-21T10:08:44.493 INFO:tasks.workunit.client.1.smithi174.stdout:6/30: dwrite d0/d3/d5/fe [0,4194304] 0 2024-03-21T10:08:44.494 INFO:tasks.workunit.client.0.smithi153.stdout:0/178: dwrite d4/d16/d1f/f20 [0,4194304] 0 2024-03-21T10:08:44.499 INFO:tasks.workunit.client.1.smithi174.stdout:4/86: write d4/ff [614598,11431] 0 2024-03-21T10:08:44.500 INFO:tasks.workunit.client.1.smithi174.stdout:7/29: dwrite d0/f9 [0,4194304] 0 2024-03-21T10:08:44.501 INFO:tasks.workunit.client.0.smithi153.stdout:5/236: getdents d4/d5/df 0 2024-03-21T10:08:44.502 INFO:tasks.workunit.client.0.smithi153.stdout:6/441: creat d1f/d2b/d33/d40/fa5 x:0 0 0 2024-03-21T10:08:44.504 INFO:tasks.workunit.client.1.smithi174.stdout:4/87: write d4/f13 [791829,103274] 0 2024-03-21T10:08:44.506 INFO:tasks.workunit.client.0.smithi153.stdout:1/243: rename d6/d10/l26 to d6/d10/d35/l4a 0 2024-03-21T10:08:44.508 INFO:tasks.workunit.client.1.smithi174.stdout:0/34: mknod d9/cb 0 2024-03-21T10:08:44.509 INFO:tasks.workunit.client.1.smithi174.stdout:6/31: symlink d0/d3/l10 0 2024-03-21T10:08:44.513 INFO:tasks.workunit.client.1.smithi174.stdout:1/63: symlink d4/l11 0 2024-03-21T10:08:44.514 INFO:tasks.workunit.client.1.smithi174.stdout:5/44: truncate d1/f6 1070342 0 2024-03-21T10:08:44.515 INFO:tasks.workunit.client.1.smithi174.stdout:5/45: fdatasync d1/f3 0 2024-03-21T10:08:44.517 INFO:tasks.workunit.client.1.smithi174.stdout:5/46: stat d1 0 2024-03-21T10:08:44.517 INFO:tasks.workunit.client.0.smithi153.stdout:2/285: link dd/d15/l16 dd/d15/d18/d2c/d4b/d56/l5f 0 2024-03-21T10:08:44.518 INFO:tasks.workunit.client.1.smithi174.stdout:4/88: unlink d4/f13 0 2024-03-21T10:08:44.522 INFO:tasks.workunit.client.0.smithi153.stdout:0/179: link d4/d16/d1f/d22/f2e d4/d16/d27/f39 0 2024-03-21T10:08:44.523 INFO:tasks.workunit.client.0.smithi153.stdout:7/159: dwrite d4/f25 [0,4194304] 0 2024-03-21T10:08:44.524 INFO:tasks.workunit.client.0.smithi153.stdout:8/261: dwrite d5/d9/d43/dd/d20/d26/d24/f32 [0,4194304] 0 2024-03-21T10:08:44.525 INFO:tasks.workunit.client.1.smithi174.stdout:5/47: mknod d1/c9 0 2024-03-21T10:08:44.526 INFO:tasks.workunit.client.0.smithi153.stdout:9/236: dwrite d1/d43/d44/f33 [4194304,4194304] 0 2024-03-21T10:08:44.526 INFO:tasks.workunit.client.0.smithi153.stdout:1/244: rmdir d6/d3a/d3c 39 2024-03-21T10:08:44.527 INFO:tasks.workunit.client.1.smithi174.stdout:4/89: symlink d4/d19/l1e 0 2024-03-21T10:08:44.529 INFO:tasks.workunit.client.1.smithi174.stdout:1/64: mknod d4/d5/c12 0 2024-03-21T10:08:44.531 INFO:tasks.workunit.client.0.smithi153.stdout:1/245: write d6/f16 [1493334,101315] 0 2024-03-21T10:08:44.531 INFO:tasks.workunit.client.0.smithi153.stdout:5/237: rename d4/d5/de/d2f/f4e to d4/d5/f52 0 2024-03-21T10:08:44.531 INFO:tasks.workunit.client.1.smithi174.stdout:7/30: creat d0/fa x:0 0 0 2024-03-21T10:08:44.531 INFO:tasks.workunit.client.0.smithi153.stdout:0/180: mknod d4/db/c3a 0 2024-03-21T10:08:44.533 INFO:tasks.workunit.client.0.smithi153.stdout:0/181: stat d4/d16/d19 0 2024-03-21T10:08:44.533 INFO:tasks.workunit.client.1.smithi174.stdout:9/107: link d2/l15 d2/d11/d1d/l22 0 2024-03-21T10:08:44.534 INFO:tasks.workunit.client.1.smithi174.stdout:4/90: creat d4/d19/f1f x:0 0 0 2024-03-21T10:08:44.535 INFO:tasks.workunit.client.0.smithi153.stdout:7/160: unlink d4/fd 0 2024-03-21T10:08:44.536 INFO:tasks.workunit.client.0.smithi153.stdout:8/262: rmdir d5/d9 39 2024-03-21T10:08:44.539 INFO:tasks.workunit.client.0.smithi153.stdout:5/238: write d4/d5/d16/f1e [573537,27356] 0 2024-03-21T10:08:44.539 INFO:tasks.workunit.client.0.smithi153.stdout:7/161: dread - d4/d19/f32 zero size 2024-03-21T10:08:44.539 INFO:tasks.workunit.client.0.smithi153.stdout:2/286: write dd/d15/d18/d2c/d33/f36 [4775574,92555] 0 2024-03-21T10:08:44.540 INFO:tasks.workunit.client.1.smithi174.stdout:9/108: dread d2/f1a [0,4194304] 0 2024-03-21T10:08:44.541 INFO:tasks.workunit.client.1.smithi174.stdout:4/91: creat d4/f20 x:0 0 0 2024-03-21T10:08:44.542 INFO:tasks.workunit.client.0.smithi153.stdout:6/442: rename d1f/d2b/d33/d40/d4f/d66/f84 to d1f/da0/fa6 0 2024-03-21T10:08:44.542 INFO:tasks.workunit.client.0.smithi153.stdout:1/246: creat d6/d32/d3e/f4b x:0 0 0 2024-03-21T10:08:44.544 INFO:tasks.workunit.client.1.smithi174.stdout:4/92: read - d4/d19/f1c zero size 2024-03-21T10:08:44.546 INFO:tasks.workunit.client.1.smithi174.stdout:1/65: dread f1 [4194304,4194304] 0 2024-03-21T10:08:44.549 INFO:tasks.workunit.client.1.smithi174.stdout:4/93: mkdir d4/d21 0 2024-03-21T10:08:44.550 INFO:tasks.workunit.client.1.smithi174.stdout:4/94: chown d4/c9 236294908 1 2024-03-21T10:08:44.551 INFO:tasks.workunit.client.0.smithi153.stdout:9/237: rename d1/d4/f1a to d1/d43/f4f 0 2024-03-21T10:08:44.551 INFO:tasks.workunit.client.0.smithi153.stdout:0/182: mknod d4/d16/c3b 0 2024-03-21T10:08:44.551 INFO:tasks.workunit.client.0.smithi153.stdout:2/287: truncate dd/d15/d18/f2a 2292227 0 2024-03-21T10:08:44.553 INFO:tasks.workunit.client.1.smithi174.stdout:9/109: dread d2/f4 [0,4194304] 0 2024-03-21T10:08:44.553 INFO:tasks.workunit.client.0.smithi153.stdout:9/238: chown d1 49178 1 2024-03-21T10:08:44.557 INFO:tasks.workunit.client.1.smithi174.stdout:6/32: write d0/d3/d5/fe [4530840,106838] 0 2024-03-21T10:08:44.559 INFO:tasks.workunit.client.1.smithi174.stdout:1/66: truncate d4/d5/f9 1223579 0 2024-03-21T10:08:44.560 INFO:tasks.workunit.client.0.smithi153.stdout:7/162: symlink d4/d14/d2f/l33 0 2024-03-21T10:08:44.560 INFO:tasks.workunit.client.1.smithi174.stdout:6/33: write d0/d3/d5/fe [2005698,52583] 0 2024-03-21T10:08:44.562 INFO:tasks.workunit.client.0.smithi153.stdout:7/163: stat d4/d2a/f2d 0 2024-03-21T10:08:44.562 INFO:tasks.workunit.client.1.smithi174.stdout:4/95: fsync d4/f8 0 2024-03-21T10:08:44.562 INFO:tasks.workunit.client.1.smithi174.stdout:6/34: fdatasync d0/d3/f8 0 2024-03-21T10:08:44.563 INFO:tasks.workunit.client.0.smithi153.stdout:8/263: rename d5/d1b/f41 to d5/d9/d43/dd/d2d/d45/f4a 0 2024-03-21T10:08:44.564 INFO:tasks.workunit.client.1.smithi174.stdout:4/96: chown d4/fa 4945 1 2024-03-21T10:08:44.566 INFO:tasks.workunit.client.1.smithi174.stdout:4/97: fdatasync d4/f10 0 2024-03-21T10:08:44.566 INFO:tasks.workunit.client.1.smithi174.stdout:6/35: write d0/d3/f8 [558370,25561] 0 2024-03-21T10:08:44.568 INFO:tasks.workunit.client.1.smithi174.stdout:9/110: write d2/f3 [1187586,54103] 0 2024-03-21T10:08:44.568 INFO:tasks.workunit.client.0.smithi153.stdout:0/183: getdents d4/d16/d19/d30 0 2024-03-21T10:08:44.570 INFO:tasks.workunit.client.1.smithi174.stdout:1/67: symlink d4/d7/l13 0 2024-03-21T10:08:44.571 INFO:tasks.workunit.client.0.smithi153.stdout:5/239: getdents d4/d5/de/d38 0 2024-03-21T10:08:44.571 INFO:tasks.workunit.client.0.smithi153.stdout:2/288: rename dd/d15/d18/d2c/d4b/f4d to dd/d15/d18/d2c/d4b/d56/f60 0 2024-03-21T10:08:44.572 INFO:tasks.workunit.client.0.smithi153.stdout:0/184: read - d4/d16/d1f/f31 zero size 2024-03-21T10:08:44.573 INFO:tasks.workunit.client.0.smithi153.stdout:8/264: truncate d5/d9/d43/dd/f38 4554698 0 2024-03-21T10:08:44.576 INFO:tasks.workunit.client.0.smithi153.stdout:3/262: sync 2024-03-21T10:08:44.579 INFO:tasks.workunit.client.1.smithi174.stdout:2/104: dwrite d5/fa [0,4194304] 0 2024-03-21T10:08:44.582 INFO:tasks.workunit.client.0.smithi153.stdout:8/265: creat d5/d9/f4b x:0 0 0 2024-03-21T10:08:44.585 INFO:tasks.workunit.client.0.smithi153.stdout:5/240: rename d4/f3c to d4/d5/d16/d24/f53 0 2024-03-21T10:08:44.589 INFO:tasks.workunit.client.1.smithi174.stdout:9/111: readlink d2/l15 0 2024-03-21T10:08:44.591 INFO:tasks.workunit.client.1.smithi174.stdout:1/68: rename d4/d7/l13 to d4/d7/l14 0 2024-03-21T10:08:44.595 INFO:tasks.workunit.client.1.smithi174.stdout:8/24: sync 2024-03-21T10:08:44.595 INFO:tasks.workunit.client.1.smithi174.stdout:3/31: sync 2024-03-21T10:08:44.596 INFO:tasks.workunit.client.1.smithi174.stdout:2/105: creat d5/d8/f28 x:0 0 0 2024-03-21T10:08:44.596 INFO:tasks.workunit.client.1.smithi174.stdout:8/25: chown d9 5 1 2024-03-21T10:08:44.597 INFO:tasks.workunit.client.1.smithi174.stdout:7/31: rmdir d0 39 2024-03-21T10:08:44.597 INFO:tasks.workunit.client.0.smithi153.stdout:3/263: write dd/d1e/d45/f52 [4987700,38818] 0 2024-03-21T10:08:44.598 INFO:tasks.workunit.client.0.smithi153.stdout:1/247: dwrite d6/d32/d3e/f4b [0,4194304] 0 2024-03-21T10:08:44.599 INFO:tasks.workunit.client.1.smithi174.stdout:3/32: chown d0/f8 3906 1 2024-03-21T10:08:44.601 INFO:tasks.workunit.client.0.smithi153.stdout:0/185: dread d4/db/f13 [0,4194304] 0 2024-03-21T10:08:44.601 INFO:tasks.workunit.client.1.smithi174.stdout:4/98: dwrite d4/f18 [0,4194304] 0 2024-03-21T10:08:44.602 INFO:tasks.workunit.client.1.smithi174.stdout:1/69: mkdir d4/d7/d15 0 2024-03-21T10:08:44.602 INFO:tasks.workunit.client.0.smithi153.stdout:9/239: truncate d1/d43/d44/d46/f1d 684992 0 2024-03-21T10:08:44.603 INFO:tasks.workunit.client.0.smithi153.stdout:7/164: dread d4/d11/d24/f29 [0,4194304] 0 2024-03-21T10:08:44.605 INFO:tasks.workunit.client.0.smithi153.stdout:1/248: dread - d6/d10/f11 zero size 2024-03-21T10:08:44.606 INFO:tasks.workunit.client.1.smithi174.stdout:4/99: write d4/f10 [121503,55345] 0 2024-03-21T10:08:44.606 INFO:tasks.workunit.client.0.smithi153.stdout:8/266: dread d5/d9/d43/f2b [0,4194304] 0 2024-03-21T10:08:44.607 INFO:tasks.workunit.client.1.smithi174.stdout:4/100: fdatasync d4/f7 0 2024-03-21T10:08:44.609 INFO:tasks.workunit.client.1.smithi174.stdout:8/26: chown d9/lb 1 1 2024-03-21T10:08:44.615 INFO:tasks.workunit.client.1.smithi174.stdout:2/106: rename d5/d8/f17 to d5/d8/f29 0 2024-03-21T10:08:44.617 INFO:tasks.workunit.client.0.smithi153.stdout:2/289: dwrite dd/d15/f52 [0,4194304] 0 2024-03-21T10:08:44.617 INFO:tasks.workunit.client.0.smithi153.stdout:3/264: creat dd/d1e/d2f/d35/f5a x:0 0 0 2024-03-21T10:08:44.620 INFO:tasks.workunit.client.1.smithi174.stdout:9/112: getdents d2 0 2024-03-21T10:08:44.621 INFO:tasks.workunit.client.1.smithi174.stdout:1/70: symlink d4/d7/l16 0 2024-03-21T10:08:44.627 INFO:tasks.workunit.client.1.smithi174.stdout:9/113: truncate d2/f1e 432770 0 2024-03-21T10:08:44.627 INFO:tasks.workunit.client.0.smithi153.stdout:0/186: truncate f2 2697195 0 2024-03-21T10:08:44.630 INFO:tasks.workunit.client.0.smithi153.stdout:9/240: write d1/d43/d44/d46/f37 [3135628,32085] 0 2024-03-21T10:08:44.630 INFO:tasks.workunit.client.0.smithi153.stdout:6/443: dwrite d1f/d2b/d33/d40/f8f [0,4194304] 0 2024-03-21T10:08:44.630 INFO:tasks.workunit.client.1.smithi174.stdout:0/35: dwrite f4 [0,4194304] 0 2024-03-21T10:08:44.631 INFO:tasks.workunit.client.0.smithi153.stdout:3/265: write dd/d41/f54 [159054,130177] 0 2024-03-21T10:08:44.631 INFO:tasks.workunit.client.1.smithi174.stdout:4/101: dread f0 [0,4194304] 0 2024-03-21T10:08:44.631 INFO:tasks.workunit.client.0.smithi153.stdout:8/267: creat d5/d9/d43/dd/d20/d26/d33/f4c x:0 0 0 2024-03-21T10:08:44.632 INFO:tasks.workunit.client.1.smithi174.stdout:9/114: chown d2/f1a 1836 1 2024-03-21T10:08:44.632 INFO:tasks.workunit.client.1.smithi174.stdout:5/48: truncate d1/f6 1323558 0 2024-03-21T10:08:44.632 INFO:tasks.workunit.client.1.smithi174.stdout:8/27: creat d9/fc x:0 0 0 2024-03-21T10:08:44.633 INFO:tasks.workunit.client.0.smithi153.stdout:3/266: readlink dd/l2e 0 2024-03-21T10:08:44.636 INFO:tasks.workunit.client.1.smithi174.stdout:0/36: truncate f7 427205 0 2024-03-21T10:08:44.638 INFO:tasks.workunit.client.1.smithi174.stdout:2/107: chown d5/c6 182 1 2024-03-21T10:08:44.641 INFO:tasks.workunit.client.0.smithi153.stdout:6/444: creat d1f/d2b/fa7 x:0 0 0 2024-03-21T10:08:44.642 INFO:tasks.workunit.client.0.smithi153.stdout:2/290: mknod dd/d3b/d50/d58/c61 0 2024-03-21T10:08:44.643 INFO:tasks.workunit.client.0.smithi153.stdout:7/165: dread d4/f1a [0,4194304] 0 2024-03-21T10:08:44.643 INFO:tasks.workunit.client.0.smithi153.stdout:0/187: symlink d4/l3c 0 2024-03-21T10:08:44.644 INFO:tasks.workunit.client.0.smithi153.stdout:2/291: dread - dd/d15/d18/f45 zero size 2024-03-21T10:08:44.646 INFO:tasks.workunit.client.0.smithi153.stdout:3/267: creat dd/d1e/d2f/f5b x:0 0 0 2024-03-21T10:08:44.647 INFO:tasks.workunit.client.1.smithi174.stdout:8/28: dread f2 [0,4194304] 0 2024-03-21T10:08:44.648 INFO:tasks.workunit.client.0.smithi153.stdout:8/268: mknod d5/d9/d43/dd/d20/d26/d33/c4d 0 2024-03-21T10:08:44.650 INFO:tasks.workunit.client.0.smithi153.stdout:0/188: read d4/f10 [6026556,61059] 0 2024-03-21T10:08:44.653 INFO:tasks.workunit.client.0.smithi153.stdout:0/189: chown d4/d16/d1f/d22/c26 1723 1 2024-03-21T10:08:44.656 INFO:tasks.workunit.client.1.smithi174.stdout:4/102: creat d4/d19/f22 x:0 0 0 2024-03-21T10:08:44.658 INFO:tasks.workunit.client.1.smithi174.stdout:9/115: mknod d2/d11/d1d/c23 0 2024-03-21T10:08:44.659 INFO:tasks.workunit.client.0.smithi153.stdout:7/166: symlink d4/d14/d16/l34 0 2024-03-21T10:08:44.659 INFO:tasks.workunit.client.1.smithi174.stdout:0/37: creat d9/fc x:0 0 0 2024-03-21T10:08:44.660 INFO:tasks.workunit.client.1.smithi174.stdout:4/103: dread - d4/fc zero size 2024-03-21T10:08:44.662 INFO:tasks.workunit.client.1.smithi174.stdout:6/36: truncate d0/d3/d5/fe 3634001 0 2024-03-21T10:08:44.663 INFO:tasks.workunit.client.1.smithi174.stdout:4/104: chown d4/l16 0 1 2024-03-21T10:08:44.664 INFO:tasks.workunit.client.0.smithi153.stdout:3/268: symlink dd/d1e/d3d/l5c 0 2024-03-21T10:08:44.669 INFO:tasks.workunit.client.1.smithi174.stdout:7/32: creat d0/fb x:0 0 0 2024-03-21T10:08:44.671 INFO:tasks.workunit.client.1.smithi174.stdout:9/116: mkdir d2/d24 0 2024-03-21T10:08:44.672 INFO:tasks.workunit.client.0.smithi153.stdout:7/167: creat d4/d11/d24/f35 x:0 0 0 2024-03-21T10:08:44.672 INFO:tasks.workunit.client.1.smithi174.stdout:7/33: dread - d0/fb zero size 2024-03-21T10:08:44.673 INFO:tasks.workunit.client.1.smithi174.stdout:2/108: creat d5/d25/d27/f2a x:0 0 0 2024-03-21T10:08:44.674 INFO:tasks.workunit.client.0.smithi153.stdout:1/249: truncate d6/d10/d2c/f47 1156289 0 2024-03-21T10:08:44.675 INFO:tasks.workunit.client.1.smithi174.stdout:8/29: dwrite f7 [0,4194304] 0 2024-03-21T10:08:44.675 INFO:tasks.workunit.client.1.smithi174.stdout:4/105: creat d4/d19/f23 x:0 0 0 2024-03-21T10:08:44.676 INFO:tasks.workunit.client.0.smithi153.stdout:4/291: sync 2024-03-21T10:08:44.678 INFO:tasks.workunit.client.0.smithi153.stdout:9/241: getdents d1/d43/d44/d46/d1c/d24/d41 0 2024-03-21T10:08:44.678 INFO:tasks.workunit.client.0.smithi153.stdout:5/241: dread d4/d5/d16/d24/f53 [0,4194304] 0 2024-03-21T10:08:44.679 INFO:tasks.workunit.client.0.smithi153.stdout:8/269: creat d5/d3b/f4e x:0 0 0 2024-03-21T10:08:44.679 INFO:tasks.workunit.client.1.smithi174.stdout:8/30: dread - f1 zero size 2024-03-21T10:08:44.679 INFO:tasks.workunit.client.1.smithi174.stdout:7/34: write d0/fa [888912,7850] 0 2024-03-21T10:08:44.680 INFO:tasks.workunit.client.0.smithi153.stdout:3/269: symlink dd/d1e/d45/d56/l5d 0 2024-03-21T10:08:44.683 INFO:tasks.workunit.client.1.smithi174.stdout:7/35: fdatasync d0/fb 0 2024-03-21T10:08:44.684 INFO:tasks.workunit.client.0.smithi153.stdout:6/445: dwrite d1f/d2b/d33/d40/d5f/f72 [0,4194304] 0 2024-03-21T10:08:44.687 INFO:tasks.workunit.client.0.smithi153.stdout:9/242: chown d1/d43/d44/d46/f26 1 1 2024-03-21T10:08:44.687 INFO:tasks.workunit.client.0.smithi153.stdout:2/292: dwrite dd/f26 [0,4194304] 0 2024-03-21T10:08:44.689 INFO:tasks.workunit.client.1.smithi174.stdout:2/109: rmdir d5/d8/d1e 39 2024-03-21T10:08:44.689 INFO:tasks.workunit.client.1.smithi174.stdout:8/31: rename f2 to d9/fd 0 2024-03-21T10:08:44.689 INFO:tasks.workunit.client.1.smithi174.stdout:2/110: readlink d5/d8/lc 0 2024-03-21T10:08:44.689 INFO:tasks.workunit.client.0.smithi153.stdout:6/446: fsync d1f/d2f/f32 0 2024-03-21T10:08:44.691 INFO:tasks.workunit.client.1.smithi174.stdout:0/38: dwrite f4 [0,4194304] 0 2024-03-21T10:08:44.695 INFO:tasks.workunit.client.1.smithi174.stdout:4/106: mkdir d4/d21/d24 0 2024-03-21T10:08:44.699 INFO:tasks.workunit.client.0.smithi153.stdout:5/242: mkdir d4/d5/de/d2f/d2c/d54 0 2024-03-21T10:08:44.700 INFO:tasks.workunit.client.1.smithi174.stdout:0/39: truncate f6 577319 0 2024-03-21T10:08:44.700 INFO:tasks.workunit.client.1.smithi174.stdout:5/49: truncate d1/f6 1573578 0 2024-03-21T10:08:44.700 INFO:tasks.workunit.client.1.smithi174.stdout:4/107: chown d4/d19/l1e 99034317 1 2024-03-21T10:08:44.702 INFO:tasks.workunit.client.1.smithi174.stdout:1/71: truncate f1 5718955 0 2024-03-21T10:08:44.702 INFO:tasks.workunit.client.1.smithi174.stdout:3/33: write d0/f3 [312534,124379] 0 2024-03-21T10:08:44.708 INFO:tasks.workunit.client.0.smithi153.stdout:7/168: link d4/d1b/c1e d4/d19/c36 0 2024-03-21T10:08:44.709 INFO:tasks.workunit.client.0.smithi153.stdout:7/169: chown d4/d19 719 1 2024-03-21T10:08:44.709 INFO:tasks.workunit.client.0.smithi153.stdout:0/190: dwrite d4/d16/d27/f38 [0,4194304] 0 2024-03-21T10:08:44.710 INFO:tasks.workunit.client.0.smithi153.stdout:9/243: truncate d1/d43/d44/d46/d1c/f31 771199 0 2024-03-21T10:08:44.712 INFO:tasks.workunit.client.0.smithi153.stdout:4/292: getdents d11/d18/d50 0 2024-03-21T10:08:44.714 INFO:tasks.workunit.client.0.smithi153.stdout:6/447: read d1f/d2b/d33/d40/d4f/d66/f7c [342768,109383] 0 2024-03-21T10:08:44.714 INFO:tasks.workunit.client.1.smithi174.stdout:2/111: dread d5/f11 [4194304,4194304] 0 2024-03-21T10:08:44.715 INFO:tasks.workunit.client.0.smithi153.stdout:2/293: creat dd/d15/d18/d2c/d33/f62 x:0 0 0 2024-03-21T10:08:44.716 INFO:tasks.workunit.client.0.smithi153.stdout:3/270: write dd/f27 [1334282,65443] 0 2024-03-21T10:08:44.718 INFO:tasks.workunit.client.1.smithi174.stdout:7/36: symlink d0/lc 0 2024-03-21T10:08:44.724 INFO:tasks.workunit.client.0.smithi153.stdout:4/293: truncate d11/d18/f49 281840 0 2024-03-21T10:08:44.724 INFO:tasks.workunit.client.0.smithi153.stdout:7/170: write d4/f1a [2627071,112356] 0 2024-03-21T10:08:44.724 INFO:tasks.workunit.client.1.smithi174.stdout:3/34: readlink d0/ld 0 2024-03-21T10:08:44.725 INFO:tasks.workunit.client.0.smithi153.stdout:6/448: mkdir d1f/d2b/d33/d40/da8 0 2024-03-21T10:08:44.729 INFO:tasks.workunit.client.0.smithi153.stdout:4/294: truncate d11/d2d/f38 649523 0 2024-03-21T10:08:44.729 INFO:tasks.workunit.client.0.smithi153.stdout:2/294: mkdir dd/d15/d18/d2c/d4b/d56/d63 0 2024-03-21T10:08:44.730 INFO:tasks.workunit.client.1.smithi174.stdout:0/40: dwrite f6 [0,4194304] 0 2024-03-21T10:08:44.730 INFO:tasks.workunit.client.0.smithi153.stdout:5/243: dread d4/d5/f52 [0,4194304] 0 2024-03-21T10:08:44.731 INFO:tasks.workunit.client.0.smithi153.stdout:2/295: readlink dd/d15/l21 0 2024-03-21T10:08:44.732 INFO:tasks.workunit.client.1.smithi174.stdout:3/35: write d0/fa [51869,40102] 0 2024-03-21T10:08:44.735 INFO:tasks.workunit.client.0.smithi153.stdout:2/296: truncate dd/d15/d18/d3a/f3e 349869 0 2024-03-21T10:08:44.735 INFO:tasks.workunit.client.1.smithi174.stdout:0/41: write f6 [3795990,27086] 0 2024-03-21T10:08:44.737 INFO:tasks.workunit.client.1.smithi174.stdout:2/112: dread d5/d8/ff [0,4194304] 0 2024-03-21T10:08:44.738 INFO:tasks.workunit.client.1.smithi174.stdout:4/108: mknod d4/d21/d24/c25 0 2024-03-21T10:08:44.741 INFO:tasks.workunit.client.0.smithi153.stdout:1/250: truncate d6/d10/d2c/f47 1044072 0 2024-03-21T10:08:44.741 INFO:tasks.workunit.client.1.smithi174.stdout:9/117: mknod d2/d11/c25 0 2024-03-21T10:08:44.744 INFO:tasks.workunit.client.0.smithi153.stdout:2/297: symlink dd/d15/d18/d2c/l64 0 2024-03-21T10:08:44.745 INFO:tasks.workunit.client.0.smithi153.stdout:3/271: link dd/c21 dd/d1e/d3d/d53/c5e 0 2024-03-21T10:08:44.746 INFO:tasks.workunit.client.0.smithi153.stdout:3/272: stat dd/f48 0 2024-03-21T10:08:44.748 INFO:tasks.workunit.client.0.smithi153.stdout:3/273: chown f8 34120733 1 2024-03-21T10:08:44.750 INFO:tasks.workunit.client.1.smithi174.stdout:7/37: creat d0/d4/fd x:0 0 0 2024-03-21T10:08:44.750 INFO:tasks.workunit.client.1.smithi174.stdout:3/36: mkdir d0/de 0 2024-03-21T10:08:44.751 INFO:tasks.workunit.client.0.smithi153.stdout:3/274: chown dd/c55 34185 1 2024-03-21T10:08:44.751 INFO:tasks.workunit.client.1.smithi174.stdout:8/32: getdents d9 0 2024-03-21T10:08:44.753 INFO:tasks.workunit.client.1.smithi174.stdout:5/50: dread d1/f6 [0,4194304] 0 2024-03-21T10:08:44.754 INFO:tasks.workunit.client.1.smithi174.stdout:0/42: symlink d9/ld 0 2024-03-21T10:08:44.755 INFO:tasks.workunit.client.1.smithi174.stdout:4/109: mkdir d4/d26 0 2024-03-21T10:08:44.757 INFO:tasks.workunit.client.1.smithi174.stdout:5/51: write d1/f3 [1461711,36429] 0 2024-03-21T10:08:44.758 INFO:tasks.workunit.client.1.smithi174.stdout:0/43: fdatasync d9/fc 0 2024-03-21T10:08:44.758 INFO:tasks.workunit.client.0.smithi153.stdout:5/244: creat d4/d5/de/d2f/d2c/d54/f55 x:0 0 0 2024-03-21T10:08:44.760 INFO:tasks.workunit.client.1.smithi174.stdout:5/52: write f0 [1932916,8361] 0 2024-03-21T10:08:44.761 INFO:tasks.workunit.client.0.smithi153.stdout:5/245: stat d4/d5/de/f36 0 2024-03-21T10:08:44.762 INFO:tasks.workunit.client.0.smithi153.stdout:1/251: mknod d6/d10/d2c/c4c 0 2024-03-21T10:08:44.763 INFO:tasks.workunit.client.0.smithi153.stdout:0/191: write f2 [3715223,76936] 0 2024-03-21T10:08:44.765 INFO:tasks.workunit.client.1.smithi174.stdout:5/53: chown d1/f3 6696 1 2024-03-21T10:08:44.765 INFO:tasks.workunit.client.1.smithi174.stdout:7/38: mkdir d0/d4/de 0 2024-03-21T10:08:44.766 INFO:tasks.workunit.client.0.smithi153.stdout:0/192: fsync d4/f37 0 2024-03-21T10:08:44.766 INFO:tasks.workunit.client.0.smithi153.stdout:0/193: fsync d4/f32 0 2024-03-21T10:08:44.766 INFO:tasks.workunit.client.0.smithi153.stdout:5/246: truncate d4/d5/df/f30 5269498 0 2024-03-21T10:08:44.769 INFO:tasks.workunit.client.1.smithi174.stdout:8/33: rmdir d9 39 2024-03-21T10:08:44.770 INFO:tasks.workunit.client.1.smithi174.stdout:7/39: write d0/d4/f7 [206853,70531] 0 2024-03-21T10:08:44.772 INFO:tasks.workunit.client.1.smithi174.stdout:6/37: dwrite d0/d3/d5/fe [0,4194304] 0 2024-03-21T10:08:44.774 INFO:tasks.workunit.client.1.smithi174.stdout:2/113: mknod d5/d8/c2b 0 2024-03-21T10:08:44.774 INFO:tasks.workunit.client.0.smithi153.stdout:9/244: truncate d1/d43/d44/d46/f37 2848371 0 2024-03-21T10:08:44.775 INFO:tasks.workunit.client.0.smithi153.stdout:4/295: rmdir d11/d58 0 2024-03-21T10:08:44.776 INFO:tasks.workunit.client.0.smithi153.stdout:3/275: stat dd/f15 0 2024-03-21T10:08:44.778 INFO:tasks.workunit.client.0.smithi153.stdout:3/276: fdatasync f0 0 2024-03-21T10:08:44.780 INFO:tasks.workunit.client.0.smithi153.stdout:9/245: stat d1/d43/d44/d46/d1c/d24/d41 0 2024-03-21T10:08:44.780 INFO:tasks.workunit.client.0.smithi153.stdout:7/171: dread d4/fa [0,4194304] 0 2024-03-21T10:08:44.781 INFO:tasks.workunit.client.1.smithi174.stdout:1/72: dwrite d4/d5/f9 [0,4194304] 0 2024-03-21T10:08:44.784 INFO:tasks.workunit.client.0.smithi153.stdout:4/296: chown d11/d18/l5c 2744350 1 2024-03-21T10:08:44.789 INFO:tasks.workunit.client.0.smithi153.stdout:2/298: getdents dd/d15 0 2024-03-21T10:08:44.792 INFO:tasks.workunit.client.0.smithi153.stdout:7/172: symlink d4/d2a/l37 0 2024-03-21T10:08:44.796 INFO:tasks.workunit.client.0.smithi153.stdout:2/299: chown dd/d15/d18/d3a/f55 245 1 2024-03-21T10:08:44.796 INFO:tasks.workunit.client.0.smithi153.stdout:9/246: symlink d1/d43/d44/d46/d1c/d2e/l50 0 2024-03-21T10:08:44.799 INFO:tasks.workunit.client.0.smithi153.stdout:5/247: dwrite d4/fa [0,4194304] 0 2024-03-21T10:08:44.799 INFO:tasks.workunit.client.0.smithi153.stdout:7/173: fdatasync d4/f30 0 2024-03-21T10:08:44.799 INFO:tasks.workunit.client.0.smithi153.stdout:0/194: getdents d4/d16/d19 0 2024-03-21T10:08:44.802 INFO:tasks.workunit.client.1.smithi174.stdout:2/114: mknod d5/d25/c2c 0 2024-03-21T10:08:44.803 INFO:tasks.workunit.client.1.smithi174.stdout:3/37: link d0/l4 d0/lf 0 2024-03-21T10:08:44.803 INFO:tasks.workunit.client.1.smithi174.stdout:4/110: rmdir d4/d26 0 2024-03-21T10:08:44.804 INFO:tasks.workunit.client.0.smithi153.stdout:6/449: dwrite d1f/d2b/d33/d40/f45 [0,4194304] 0 2024-03-21T10:08:44.805 INFO:tasks.workunit.client.1.smithi174.stdout:4/111: readlink d4/l17 0 2024-03-21T10:08:44.805 INFO:tasks.workunit.client.1.smithi174.stdout:8/34: creat d9/fe x:0 0 0 2024-03-21T10:08:44.806 INFO:tasks.workunit.client.1.smithi174.stdout:3/38: fdatasync d0/fb 0 2024-03-21T10:08:44.806 INFO:tasks.workunit.client.0.smithi153.stdout:5/248: write d4/d5/de/f43 [284876,59910] 0 2024-03-21T10:08:44.807 INFO:tasks.workunit.client.0.smithi153.stdout:3/277: getdents dd/d1e/d45 0 2024-03-21T10:08:44.809 INFO:tasks.workunit.client.0.smithi153.stdout:2/300: symlink dd/d15/d18/d2c/d4b/d56/l65 0 2024-03-21T10:08:44.811 INFO:tasks.workunit.client.0.smithi153.stdout:9/247: rmdir d1/d3b 39 2024-03-21T10:08:44.813 INFO:tasks.workunit.client.0.smithi153.stdout:1/252: dread d6/fe [0,4194304] 0 2024-03-21T10:08:44.814 INFO:tasks.workunit.client.0.smithi153.stdout:0/195: fsync d4/d16/d1f/d22/f2e 0 2024-03-21T10:08:44.816 INFO:tasks.workunit.client.0.smithi153.stdout:9/248: read - d1/d43/d44/d46/d1c/d24/f49 zero size 2024-03-21T10:08:44.817 INFO:tasks.workunit.client.1.smithi174.stdout:5/54: link d1/c8 d1/ca 0 2024-03-21T10:08:44.819 INFO:tasks.workunit.client.1.smithi174.stdout:1/73: read d4/d5/fd [287041,17938] 0 2024-03-21T10:08:44.820 INFO:tasks.workunit.client.1.smithi174.stdout:9/118: dread d2/f20 [0,4194304] 0 2024-03-21T10:08:44.821 INFO:tasks.workunit.client.0.smithi153.stdout:4/297: rename d11/d2d/d57/d5b to d11/d5e 0 2024-03-21T10:08:44.822 INFO:tasks.workunit.client.0.smithi153.stdout:7/174: dwrite d4/d2a/f2d [0,4194304] 0 2024-03-21T10:08:44.824 INFO:tasks.workunit.client.1.smithi174.stdout:0/44: dwrite f4 [0,4194304] 0 2024-03-21T10:08:44.828 INFO:tasks.workunit.client.0.smithi153.stdout:5/249: mkdir d4/d8/d3b/d56 0 2024-03-21T10:08:44.834 INFO:tasks.workunit.client.0.smithi153.stdout:6/450: symlink d1f/d2b/d33/d40/d4f/la9 0 2024-03-21T10:08:44.838 INFO:tasks.workunit.client.0.smithi153.stdout:6/451: fdatasync d1f/d2f/f74 0 2024-03-21T10:08:44.839 INFO:tasks.workunit.client.0.smithi153.stdout:9/249: unlink d1/d43/d44/d46/d1c/d2e/l50 0 2024-03-21T10:08:44.842 INFO:tasks.workunit.client.0.smithi153.stdout:0/196: dwrite d4/f32 [0,4194304] 0 2024-03-21T10:08:44.843 INFO:tasks.workunit.client.0.smithi153.stdout:2/301: dwrite dd/d15/f52 [4194304,4194304] 0 2024-03-21T10:08:44.843 INFO:tasks.workunit.client.0.smithi153.stdout:7/175: mknod d4/d1b/d20/c38 0 2024-03-21T10:08:44.843 INFO:tasks.workunit.client.0.smithi153.stdout:8/270: sync 2024-03-21T10:08:44.847 INFO:tasks.workunit.client.0.smithi153.stdout:3/278: creat dd/d1e/d45/d56/d58/f5f x:0 0 0 2024-03-21T10:08:44.849 INFO:tasks.workunit.client.0.smithi153.stdout:5/250: write d4/d5/df/f1b [1067791,62379] 0 2024-03-21T10:08:44.850 INFO:tasks.workunit.client.0.smithi153.stdout:0/197: write d4/d16/d27/f38 [2112108,8086] 0 2024-03-21T10:08:44.851 INFO:tasks.workunit.client.0.smithi153.stdout:1/253: symlink d6/d3a/d3c/l4d 0 2024-03-21T10:08:44.855 INFO:tasks.workunit.client.0.smithi153.stdout:6/452: mknod d1f/d2b/d33/d40/d7b/caa 0 2024-03-21T10:08:44.856 INFO:tasks.workunit.client.0.smithi153.stdout:0/198: truncate d4/f23 463665 0 2024-03-21T10:08:44.857 INFO:tasks.workunit.client.0.smithi153.stdout:8/271: creat d5/d3b/f4f x:0 0 0 2024-03-21T10:08:44.857 INFO:tasks.workunit.client.0.smithi153.stdout:6/453: chown f19 0 1 2024-03-21T10:08:44.857 INFO:tasks.workunit.client.0.smithi153.stdout:0/199: fdatasync f2 0 2024-03-21T10:08:44.858 INFO:tasks.workunit.client.0.smithi153.stdout:0/200: stat d4/d16/d19 0 2024-03-21T10:08:44.861 INFO:tasks.workunit.client.0.smithi153.stdout:9/250: rename d1/d4/fe to d1/d43/d44/d46/d1c/d2e/f51 0 2024-03-21T10:08:44.861 INFO:tasks.workunit.client.0.smithi153.stdout:3/279: mkdir dd/d1e/d60 0 2024-03-21T10:08:44.863 INFO:tasks.workunit.client.0.smithi153.stdout:5/251: chown d4/d5/de/f21 92989918 1 2024-03-21T10:08:44.867 INFO:tasks.workunit.client.0.smithi153.stdout:2/302: creat dd/d15/d18/d2c/d4b/f66 x:0 0 0 2024-03-21T10:08:44.868 INFO:tasks.workunit.client.0.smithi153.stdout:4/298: creat d11/f5f x:0 0 0 2024-03-21T10:08:44.870 INFO:tasks.workunit.client.0.smithi153.stdout:6/454: creat d1f/d2f/d59/fab x:0 0 0 2024-03-21T10:08:44.876 INFO:tasks.workunit.client.0.smithi153.stdout:2/303: creat dd/d15/d18/d2c/d4b/f67 x:0 0 0 2024-03-21T10:08:44.879 INFO:tasks.workunit.client.0.smithi153.stdout:7/176: dwrite d4/d2a/f2d [0,4194304] 0 2024-03-21T10:08:44.882 INFO:tasks.workunit.client.0.smithi153.stdout:7/177: dread - d4/d19/f31 zero size 2024-03-21T10:08:44.883 INFO:tasks.workunit.client.0.smithi153.stdout:3/280: rmdir dd/d1e/d60 0 2024-03-21T10:08:44.883 INFO:tasks.workunit.client.0.smithi153.stdout:6/455: mkdir d1f/d2b/d33/d40/d5f/dac 0 2024-03-21T10:08:44.883 INFO:tasks.workunit.client.0.smithi153.stdout:6/456: stat c10 0 2024-03-21T10:08:44.888 INFO:tasks.workunit.client.0.smithi153.stdout:8/272: rename d5/d9/d43/f2c to d5/d9/d43/dd/d20/f50 0 2024-03-21T10:08:44.890 INFO:tasks.workunit.client.0.smithi153.stdout:7/178: write d4/d1b/d20/f2c [699418,68943] 0 2024-03-21T10:08:44.890 INFO:tasks.workunit.client.0.smithi153.stdout:6/457: write f11 [382417,36828] 0 2024-03-21T10:08:44.890 INFO:tasks.workunit.client.0.smithi153.stdout:0/201: link d4/d16/d1f/l1a d4/d16/d19/d30/l3d 0 2024-03-21T10:08:44.891 INFO:tasks.workunit.client.0.smithi153.stdout:2/304: symlink dd/d3b/d50/d58/l68 0 2024-03-21T10:08:44.896 INFO:tasks.workunit.client.0.smithi153.stdout:3/281: rmdir dd/d1e/d45/d56 39 2024-03-21T10:08:44.896 INFO:tasks.workunit.client.0.smithi153.stdout:5/252: dwrite d4/d5/df/f34 [0,4194304] 0 2024-03-21T10:08:44.896 INFO:tasks.workunit.client.0.smithi153.stdout:0/202: chown d4/d16/d19/l2b 2154 1 2024-03-21T10:08:44.899 INFO:tasks.workunit.client.0.smithi153.stdout:2/305: rmdir dd/d3b/d50 39 2024-03-21T10:08:44.902 INFO:tasks.workunit.client.0.smithi153.stdout:7/179: creat d4/d14/d16/f39 x:0 0 0 2024-03-21T10:08:44.902 INFO:tasks.workunit.client.0.smithi153.stdout:1/254: write d6/fd [1769403,117902] 0 2024-03-21T10:08:44.905 INFO:tasks.workunit.client.0.smithi153.stdout:8/273: mknod d5/d9/d43/dd/d20/d26/d24/c51 0 2024-03-21T10:08:44.907 INFO:tasks.workunit.client.1.smithi174.stdout:7/40: rename d0/fb to d0/ff 0 2024-03-21T10:08:44.908 INFO:tasks.workunit.client.0.smithi153.stdout:1/255: write d6/d32/d3e/f42 [913827,121118] 0 2024-03-21T10:08:44.908 INFO:tasks.workunit.client.0.smithi153.stdout:9/251: truncate d1/d43/d44/d46/f37 2384272 0 2024-03-21T10:08:44.909 INFO:tasks.workunit.client.0.smithi153.stdout:8/274: write d5/d3b/f4e [849863,4733] 0 2024-03-21T10:08:44.909 INFO:tasks.workunit.client.1.smithi174.stdout:4/112: creat d4/d21/f27 x:0 0 0 2024-03-21T10:08:44.911 INFO:tasks.workunit.client.1.smithi174.stdout:8/35: symlink d9/lf 0 2024-03-21T10:08:44.911 INFO:tasks.workunit.client.0.smithi153.stdout:7/180: rmdir d4/d2a 39 2024-03-21T10:08:44.915 INFO:tasks.workunit.client.1.smithi174.stdout:2/115: rename d5/d8/d1e to d5/d25/d27/d2d 0 2024-03-21T10:08:44.915 INFO:tasks.workunit.client.0.smithi153.stdout:0/203: mkdir d4/d16/d3e 0 2024-03-21T10:08:44.916 INFO:tasks.workunit.client.0.smithi153.stdout:9/252: chown d1/d43/f4f 12330027 1 2024-03-21T10:08:44.916 INFO:tasks.workunit.client.1.smithi174.stdout:2/116: stat d5/l1f 0 2024-03-21T10:08:44.918 INFO:tasks.workunit.client.1.smithi174.stdout:9/119: chown d2/f18 1715675 1 2024-03-21T10:08:44.918 INFO:tasks.workunit.client.1.smithi174.stdout:0/45: symlink d9/le 0 2024-03-21T10:08:44.919 INFO:tasks.workunit.client.0.smithi153.stdout:6/458: rmdir d1f/d2b/d33/d40/da8 0 2024-03-21T10:08:44.920 INFO:tasks.workunit.client.1.smithi174.stdout:7/41: mkdir d0/d4/d10 0 2024-03-21T10:08:44.922 INFO:tasks.workunit.client.1.smithi174.stdout:4/113: unlink f0 0 2024-03-21T10:08:44.923 INFO:tasks.workunit.client.1.smithi174.stdout:7/42: write d0/d4/fd [98902,125927] 0 2024-03-21T10:08:44.924 INFO:tasks.workunit.client.1.smithi174.stdout:3/39: mkdir d0/de/d10 0 2024-03-21T10:08:44.924 INFO:tasks.workunit.client.0.smithi153.stdout:5/253: rename d4/d5/de/f4c to d4/d5/f57 0 2024-03-21T10:08:44.925 INFO:tasks.workunit.client.0.smithi153.stdout:4/299: truncate d11/f19 650064 0 2024-03-21T10:08:44.925 INFO:tasks.workunit.client.0.smithi153.stdout:3/282: dwrite dd/f2c [4194304,4194304] 0 2024-03-21T10:08:44.927 INFO:tasks.workunit.client.1.smithi174.stdout:5/55: unlink d1/ca 0 2024-03-21T10:08:44.928 INFO:tasks.workunit.client.0.smithi153.stdout:9/253: rmdir d1/d43/d44 39 2024-03-21T10:08:44.928 INFO:tasks.workunit.client.0.smithi153.stdout:7/181: write d4/d11/d24/f29 [3245945,55403] 0 2024-03-21T10:08:44.928 INFO:tasks.workunit.client.1.smithi174.stdout:1/74: mknod d4/d7/d15/c17 0 2024-03-21T10:08:44.928 INFO:tasks.workunit.client.0.smithi153.stdout:8/275: mkdir d5/d9/d43/dd/d20/d26/d24/d52 0 2024-03-21T10:08:44.929 INFO:tasks.workunit.client.1.smithi174.stdout:6/38: link d0/d3/d5/d6/lc d0/d3/d5/d6/l11 0 2024-03-21T10:08:44.931 INFO:tasks.workunit.client.1.smithi174.stdout:4/114: mkdir d4/d21/d24/d28 0 2024-03-21T10:08:44.933 INFO:tasks.workunit.client.1.smithi174.stdout:9/120: mknod d2/d24/c26 0 2024-03-21T10:08:44.934 INFO:tasks.workunit.client.0.smithi153.stdout:7/182: fdatasync d4/d19/f32 0 2024-03-21T10:08:44.938 INFO:tasks.workunit.client.1.smithi174.stdout:1/75: unlink d4/d7/d15/c17 0 2024-03-21T10:08:44.939 INFO:tasks.workunit.client.0.smithi153.stdout:2/306: truncate dd/f12 2926330 0 2024-03-21T10:08:44.940 INFO:tasks.workunit.client.0.smithi153.stdout:3/283: symlink dd/d41/l61 0 2024-03-21T10:08:44.942 INFO:tasks.workunit.client.1.smithi174.stdout:4/115: symlink d4/l29 0 2024-03-21T10:08:44.944 INFO:tasks.workunit.client.1.smithi174.stdout:1/76: readlink d4/lc 0 2024-03-21T10:08:44.945 INFO:tasks.workunit.client.1.smithi174.stdout:3/40: link d0/l9 d0/l11 0 2024-03-21T10:08:44.946 INFO:tasks.workunit.client.1.smithi174.stdout:9/121: symlink d2/d11/d1d/l27 0 2024-03-21T10:08:44.951 INFO:tasks.workunit.client.1.smithi174.stdout:4/116: symlink d4/l2a 0 2024-03-21T10:08:44.951 INFO:tasks.workunit.client.0.smithi153.stdout:3/284: read - dd/d1e/d2f/f33 zero size 2024-03-21T10:08:44.953 INFO:tasks.workunit.client.1.smithi174.stdout:0/46: dwrite d9/fc [0,4194304] 0 2024-03-21T10:08:44.957 INFO:tasks.workunit.client.0.smithi153.stdout:3/285: dread - dd/f2d zero size 2024-03-21T10:08:44.960 INFO:tasks.workunit.client.1.smithi174.stdout:3/41: dread d0/fa [0,4194304] 0 2024-03-21T10:08:44.961 INFO:tasks.workunit.client.1.smithi174.stdout:4/117: truncate d4/f10 894602 0 2024-03-21T10:08:44.962 INFO:tasks.workunit.client.0.smithi153.stdout:1/256: truncate d6/d10/f34 2931728 0 2024-03-21T10:08:44.962 INFO:tasks.workunit.client.1.smithi174.stdout:7/43: dread d0/fa [0,4194304] 0 2024-03-21T10:08:44.962 INFO:tasks.workunit.client.0.smithi153.stdout:5/254: dwrite d4/d5/d16/d24/f25 [0,4194304] 0 2024-03-21T10:08:44.963 INFO:tasks.workunit.client.1.smithi174.stdout:0/47: fsync f6 0 2024-03-21T10:08:44.963 INFO:tasks.workunit.client.0.smithi153.stdout:2/307: dread dd/d15/d18/d2c/f39 [0,4194304] 0 2024-03-21T10:08:44.967 INFO:tasks.workunit.client.1.smithi174.stdout:3/42: mkdir d0/d12 0 2024-03-21T10:08:44.968 INFO:tasks.workunit.client.1.smithi174.stdout:8/36: dwrite d9/fd [0,4194304] 0 2024-03-21T10:08:44.969 INFO:tasks.workunit.client.0.smithi153.stdout:8/276: dread d5/d9/f11 [0,4194304] 0 2024-03-21T10:08:44.969 INFO:tasks.workunit.client.0.smithi153.stdout:6/459: dwrite d1f/d2b/d33/d40/d5f/f72 [0,4194304] 0 2024-03-21T10:08:44.971 INFO:tasks.workunit.client.0.smithi153.stdout:4/300: write d11/f26 [871695,30645] 0 2024-03-21T10:08:44.976 INFO:tasks.workunit.client.1.smithi174.stdout:6/39: dread d0/d3/d5/fe [0,4194304] 0 2024-03-21T10:08:44.979 INFO:tasks.workunit.client.1.smithi174.stdout:7/44: dread d0/d4/fd [0,4194304] 0 2024-03-21T10:08:44.981 INFO:tasks.workunit.client.1.smithi174.stdout:3/43: creat d0/f13 x:0 0 0 2024-03-21T10:08:44.982 INFO:tasks.workunit.client.1.smithi174.stdout:8/37: write f5 [304680,47213] 0 2024-03-21T10:08:44.982 INFO:tasks.workunit.client.0.smithi153.stdout:0/204: dread f2 [0,4194304] 0 2024-03-21T10:08:44.984 INFO:tasks.workunit.client.1.smithi174.stdout:6/40: fdatasync d0/d3/f8 0 2024-03-21T10:08:44.985 INFO:tasks.workunit.client.1.smithi174.stdout:4/118: creat d4/d21/d24/d28/f2b x:0 0 0 2024-03-21T10:08:44.985 INFO:tasks.workunit.client.1.smithi174.stdout:8/38: readlink d9/lf 0 2024-03-21T10:08:44.987 INFO:tasks.workunit.client.0.smithi153.stdout:7/183: dwrite d4/fa [4194304,4194304] 0 2024-03-21T10:08:44.989 INFO:tasks.workunit.client.1.smithi174.stdout:8/39: dread - d9/fe zero size 2024-03-21T10:08:44.989 INFO:tasks.workunit.client.0.smithi153.stdout:2/308: creat dd/d15/d3c/f69 x:0 0 0 2024-03-21T10:08:44.991 INFO:tasks.workunit.client.0.smithi153.stdout:5/255: creat d4/d5/df/f58 x:0 0 0 2024-03-21T10:08:44.993 INFO:tasks.workunit.client.1.smithi174.stdout:4/119: creat d4/f2c x:0 0 0 2024-03-21T10:08:44.993 INFO:tasks.workunit.client.0.smithi153.stdout:6/460: mknod d1f/d2b/d33/d40/d51/cad 0 2024-03-21T10:08:44.994 INFO:tasks.workunit.client.1.smithi174.stdout:6/41: creat d0/d3/d5/f12 x:0 0 0 2024-03-21T10:08:44.997 INFO:tasks.workunit.client.1.smithi174.stdout:4/120: stat d4/fa 0 2024-03-21T10:08:44.997 INFO:tasks.workunit.client.0.smithi153.stdout:1/257: dread d6/d32/d3e/f42 [0,4194304] 0 2024-03-21T10:08:44.998 INFO:tasks.workunit.client.1.smithi174.stdout:9/122: dwrite d2/fc [0,4194304] 0 2024-03-21T10:08:44.999 INFO:tasks.workunit.client.0.smithi153.stdout:5/256: creat d4/d5/de/d44/d48/f59 x:0 0 0 2024-03-21T10:08:45.003 INFO:tasks.workunit.client.1.smithi174.stdout:1/77: dread f1 [0,4194304] 0 2024-03-21T10:08:45.003 INFO:tasks.workunit.client.1.smithi174.stdout:7/45: rename d0/d4/f7 to d0/f11 0 2024-03-21T10:08:45.004 INFO:tasks.workunit.client.1.smithi174.stdout:6/42: truncate d0/d3/d5/f12 748766 0 2024-03-21T10:08:45.004 INFO:tasks.workunit.client.0.smithi153.stdout:9/254: dread d1/d4/f14 [0,4194304] 0 2024-03-21T10:08:45.004 INFO:tasks.workunit.client.0.smithi153.stdout:2/309: mknod dd/c6a 0 2024-03-21T10:08:45.005 INFO:tasks.workunit.client.0.smithi153.stdout:3/286: dwrite dd/f43 [0,4194304] 0 2024-03-21T10:08:45.008 INFO:tasks.workunit.client.0.smithi153.stdout:8/277: rename d5/d9/d43/dd/d20/d26/d24/c51 to d5/d9/d43/dd/d20/c53 0 2024-03-21T10:08:45.008 INFO:tasks.workunit.client.1.smithi174.stdout:8/40: symlink d9/l10 0 2024-03-21T10:08:45.009 INFO:tasks.workunit.client.0.smithi153.stdout:9/255: chown d1/d4/c39 118547 1 2024-03-21T10:08:45.010 INFO:tasks.workunit.client.1.smithi174.stdout:8/41: chown d9/fe 14 1 2024-03-21T10:08:45.010 INFO:tasks.workunit.client.0.smithi153.stdout:7/184: link d4/f30 d4/d11/d24/f3a 0 2024-03-21T10:08:45.013 INFO:tasks.workunit.client.1.smithi174.stdout:4/121: creat d4/d19/f2d x:0 0 0 2024-03-21T10:08:45.014 INFO:tasks.workunit.client.0.smithi153.stdout:1/258: fdatasync d6/f9 0 2024-03-21T10:08:45.015 INFO:tasks.workunit.client.1.smithi174.stdout:7/46: read d0/d4/fd [38836,25598] 0 2024-03-21T10:08:45.015 INFO:tasks.workunit.client.0.smithi153.stdout:6/461: symlink d1f/d2b/d33/d40/d4f/d66/lae 0 2024-03-21T10:08:45.016 INFO:tasks.workunit.client.1.smithi174.stdout:1/78: creat d4/d5/f18 x:0 0 0 2024-03-21T10:08:45.018 INFO:tasks.workunit.client.1.smithi174.stdout:8/42: symlink d9/l11 0 2024-03-21T10:08:45.021 INFO:tasks.workunit.client.0.smithi153.stdout:5/257: rename d4/d5/de/d2f/d2c to d4/d8/d5a 0 2024-03-21T10:08:45.022 INFO:tasks.workunit.client.0.smithi153.stdout:5/258: dread - d4/f46 zero size 2024-03-21T10:08:45.023 INFO:tasks.workunit.client.0.smithi153.stdout:9/256: chown d1/d43/d44/d46/d1c/d24/f32 826024 1 2024-03-21T10:08:45.023 INFO:tasks.workunit.client.1.smithi174.stdout:0/48: dwrite f7 [0,4194304] 0 2024-03-21T10:08:45.024 INFO:tasks.workunit.client.0.smithi153.stdout:5/259: fsync d4/d8/d5a/d54/f55 0 2024-03-21T10:08:45.024 INFO:tasks.workunit.client.1.smithi174.stdout:4/122: mknod d4/d21/d24/d28/c2e 0 2024-03-21T10:08:45.025 INFO:tasks.workunit.client.0.smithi153.stdout:4/301: getdents d11/d18 0 2024-03-21T10:08:45.026 INFO:tasks.workunit.client.1.smithi174.stdout:0/49: chown l3 104200 1 2024-03-21T10:08:45.026 INFO:tasks.workunit.client.1.smithi174.stdout:0/50: stat l1 0 2024-03-21T10:08:45.027 INFO:tasks.workunit.client.0.smithi153.stdout:4/302: chown d11 1214062 1 2024-03-21T10:08:45.027 INFO:tasks.workunit.client.0.smithi153.stdout:6/462: unlink d1f/d2f/f8c 0 2024-03-21T10:08:45.029 INFO:tasks.workunit.client.1.smithi174.stdout:7/47: creat d0/d4/f12 x:0 0 0 2024-03-21T10:08:45.029 INFO:tasks.workunit.client.0.smithi153.stdout:9/257: read d1/d43/d44/d46/d1c/d24/f38 [2744476,69682] 0 2024-03-21T10:08:45.031 INFO:tasks.workunit.client.1.smithi174.stdout:9/123: creat d2/f28 x:0 0 0 2024-03-21T10:08:45.032 INFO:tasks.workunit.client.0.smithi153.stdout:3/287: rename dd/d1e/d2f/f4e to dd/f62 0 2024-03-21T10:08:45.032 INFO:tasks.workunit.client.0.smithi153.stdout:8/278: creat d5/d9/d43/dd/d20/d26/d24/d52/f54 x:0 0 0 2024-03-21T10:08:45.032 INFO:tasks.workunit.client.0.smithi153.stdout:0/205: link d4/d16/d1f/d22/c36 d4/d16/d19/c3f 0 2024-03-21T10:08:45.036 INFO:tasks.workunit.client.1.smithi174.stdout:7/48: dread d0/fa [0,4194304] 0 2024-03-21T10:08:45.036 INFO:tasks.workunit.client.0.smithi153.stdout:0/206: write d4/f37 [534987,3427] 0 2024-03-21T10:08:45.036 INFO:tasks.workunit.client.0.smithi153.stdout:1/259: mkdir d6/d3a/d4e 0 2024-03-21T10:08:45.039 INFO:tasks.workunit.client.0.smithi153.stdout:6/463: creat d1f/d99/faf x:0 0 0 2024-03-21T10:08:45.041 INFO:tasks.workunit.client.0.smithi153.stdout:2/310: getdents dd/d15/d3c 0 2024-03-21T10:08:45.043 INFO:tasks.workunit.client.1.smithi174.stdout:4/123: mkdir d4/d21/d24/d2f 0 2024-03-21T10:08:45.044 INFO:tasks.workunit.client.0.smithi153.stdout:2/311: chown dd/d15/d18/d2c/d4b/f66 13950 1 2024-03-21T10:08:45.044 INFO:tasks.workunit.client.1.smithi174.stdout:4/124: chown d4/l14 179 1 2024-03-21T10:08:45.045 INFO:tasks.workunit.client.0.smithi153.stdout:5/260: truncate d4/d5/de/f22 154240 0 2024-03-21T10:08:45.046 INFO:tasks.workunit.client.0.smithi153.stdout:0/207: symlink d4/d16/d27/l40 0 2024-03-21T10:08:45.046 INFO:tasks.workunit.client.0.smithi153.stdout:1/260: mkdir d6/d32/d3e/d4f 0 2024-03-21T10:08:45.047 INFO:tasks.workunit.client.0.smithi153.stdout:8/279: dread d5/d9/d43/dd/d20/d26/d24/f28 [0,4194304] 0 2024-03-21T10:08:45.047 INFO:tasks.workunit.client.1.smithi174.stdout:9/124: stat d2/f9 0 2024-03-21T10:08:45.048 INFO:tasks.workunit.client.0.smithi153.stdout:4/303: truncate d11/f19 846146 0 2024-03-21T10:08:45.051 INFO:tasks.workunit.client.0.smithi153.stdout:3/288: symlink dd/d1e/d45/d56/l63 0 2024-03-21T10:08:45.053 INFO:tasks.workunit.client.0.smithi153.stdout:3/289: readlink dd/d1e/l28 0 2024-03-21T10:08:45.056 INFO:tasks.workunit.client.1.smithi174.stdout:8/43: link f8 d9/f12 0 2024-03-21T10:08:45.059 INFO:tasks.workunit.client.0.smithi153.stdout:1/261: mkdir d6/d3a/d3c/d50 0 2024-03-21T10:08:45.060 INFO:tasks.workunit.client.0.smithi153.stdout:8/280: mkdir d5/d9/d43/dd/d20/d26/d24/d52/d55 0 2024-03-21T10:08:45.060 INFO:tasks.workunit.client.1.smithi174.stdout:4/125: symlink d4/d19/l30 0 2024-03-21T10:08:45.062 INFO:tasks.workunit.client.0.smithi153.stdout:5/261: dread d4/fa [0,4194304] 0 2024-03-21T10:08:45.063 INFO:tasks.workunit.client.1.smithi174.stdout:9/125: chown d2/c13 1 1 2024-03-21T10:08:45.063 INFO:tasks.workunit.client.0.smithi153.stdout:8/281: chown d5/d9/f4b 53690 1 2024-03-21T10:08:45.066 INFO:tasks.workunit.client.1.smithi174.stdout:4/126: chown d4/d19/l1d 4 1 2024-03-21T10:08:45.067 INFO:tasks.workunit.client.1.smithi174.stdout:9/126: dread - d2/f28 zero size 2024-03-21T10:08:45.067 INFO:tasks.workunit.client.1.smithi174.stdout:1/79: creat d4/f19 x:0 0 0 2024-03-21T10:08:45.070 INFO:tasks.workunit.client.0.smithi153.stdout:2/312: getdents dd/d15/d18/d2c 0 2024-03-21T10:08:45.071 INFO:tasks.workunit.client.0.smithi153.stdout:5/262: creat d4/d8/d5a/d54/f5b x:0 0 0 2024-03-21T10:08:45.074 INFO:tasks.workunit.client.1.smithi174.stdout:9/127: rmdir d2/d24 39 2024-03-21T10:08:45.076 INFO:tasks.workunit.client.1.smithi174.stdout:7/49: dwrite d0/fa [0,4194304] 0 2024-03-21T10:08:45.077 INFO:tasks.workunit.client.0.smithi153.stdout:1/262: mkdir d6/d3a/d3c/d50/d51 0 2024-03-21T10:08:45.080 INFO:tasks.workunit.client.0.smithi153.stdout:3/290: getdents dd/d1e/d45 0 2024-03-21T10:08:45.081 INFO:tasks.workunit.client.1.smithi174.stdout:7/50: truncate d0/d4/f8 291181 0 2024-03-21T10:08:45.081 INFO:tasks.workunit.client.1.smithi174.stdout:1/80: creat d4/d7/f1a x:0 0 0 2024-03-21T10:08:45.085 INFO:tasks.workunit.client.1.smithi174.stdout:4/127: truncate d4/d19/f23 216954 0 2024-03-21T10:08:45.086 INFO:tasks.workunit.client.0.smithi153.stdout:2/313: getdents dd/d15/d18/d3a 0 2024-03-21T10:08:45.087 INFO:tasks.workunit.client.1.smithi174.stdout:7/51: symlink d0/l13 0 2024-03-21T10:08:45.091 INFO:tasks.workunit.client.1.smithi174.stdout:1/81: creat d4/d7/f1b x:0 0 0 2024-03-21T10:08:45.095 INFO:tasks.workunit.client.0.smithi153.stdout:2/314: fsync dd/f13 0 2024-03-21T10:08:45.097 INFO:tasks.workunit.client.1.smithi174.stdout:4/128: mkdir d4/d21/d31 0 2024-03-21T10:08:45.098 INFO:tasks.workunit.client.1.smithi174.stdout:7/52: mknod d0/d4/c14 0 2024-03-21T10:08:45.098 INFO:tasks.workunit.client.0.smithi153.stdout:5/263: dwrite d4/d5/df/f58 [0,4194304] 0 2024-03-21T10:08:45.100 INFO:tasks.workunit.client.1.smithi174.stdout:4/129: write d4/f18 [2678813,107904] 0 2024-03-21T10:08:45.106 INFO:tasks.workunit.client.0.smithi153.stdout:4/304: truncate d11/d18/f21 520579 0 2024-03-21T10:08:45.112 INFO:tasks.workunit.client.0.smithi153.stdout:8/282: truncate f3 6037422 0 2024-03-21T10:08:45.112 INFO:tasks.workunit.client.1.smithi174.stdout:1/82: dread d4/d7/ff [0,4194304] 0 2024-03-21T10:08:45.115 INFO:tasks.workunit.client.1.smithi174.stdout:1/83: chown d4/d5/f8 5507 1 2024-03-21T10:08:45.119 INFO:tasks.workunit.client.0.smithi153.stdout:1/263: dwrite d6/d13/f3f [0,4194304] 0 2024-03-21T10:08:45.120 INFO:tasks.workunit.client.1.smithi174.stdout:1/84: unlink d4/d5/l6 0 2024-03-21T10:08:45.122 INFO:tasks.workunit.client.0.smithi153.stdout:4/305: write d11/d18/f41 [224900,50145] 0 2024-03-21T10:08:45.124 INFO:tasks.workunit.client.1.smithi174.stdout:4/130: dwrite d4/f7 [0,4194304] 0 2024-03-21T10:08:45.124 INFO:tasks.workunit.client.1.smithi174.stdout:1/85: unlink f1 0 2024-03-21T10:08:45.124 INFO:tasks.workunit.client.0.smithi153.stdout:2/315: rename dd/d3b/d50/d58/c61 to dd/d15/d5e/c6b 0 2024-03-21T10:08:45.126 INFO:tasks.workunit.client.1.smithi174.stdout:4/131: read - d4/d19/f22 zero size 2024-03-21T10:08:45.126 INFO:tasks.workunit.client.0.smithi153.stdout:0/208: dwrite d4/f28 [0,4194304] 0 2024-03-21T10:08:45.129 INFO:tasks.workunit.client.0.smithi153.stdout:9/258: dwrite d1/d43/d44/d46/f1d [0,4194304] 0 2024-03-21T10:08:45.129 INFO:tasks.workunit.client.0.smithi153.stdout:5/264: getdents d4/d8/d3b 0 2024-03-21T10:08:45.130 INFO:tasks.workunit.client.0.smithi153.stdout:8/283: getdents d5/d1b/d49 0 2024-03-21T10:08:45.130 INFO:tasks.workunit.client.0.smithi153.stdout:6/464: dwrite d1f/d2b/d33/d40/f4d [0,4194304] 0 2024-03-21T10:08:45.131 INFO:tasks.workunit.client.1.smithi174.stdout:3/44: rename d0/de to d0/d12/d14 0 2024-03-21T10:08:45.133 INFO:tasks.workunit.client.1.smithi174.stdout:3/45: chown d0/f3 28 1 2024-03-21T10:08:45.134 INFO:tasks.workunit.client.0.smithi153.stdout:1/264: creat d6/f52 x:0 0 0 2024-03-21T10:08:45.134 INFO:tasks.workunit.client.1.smithi174.stdout:1/86: mknod d4/d7/d15/c1c 0 2024-03-21T10:08:45.135 INFO:tasks.workunit.client.1.smithi174.stdout:3/46: write d0/fb [206815,59599] 0 2024-03-21T10:08:45.135 INFO:tasks.workunit.client.1.smithi174.stdout:5/56: rmdir d1 39 2024-03-21T10:08:45.138 INFO:tasks.workunit.client.1.smithi174.stdout:4/132: mknod d4/c32 0 2024-03-21T10:08:45.141 INFO:tasks.workunit.client.0.smithi153.stdout:4/306: rename d11/c2a to d11/d18/c60 0 2024-03-21T10:08:45.144 INFO:tasks.workunit.client.1.smithi174.stdout:1/87: read d4/d5/fd [546474,88800] 0 2024-03-21T10:08:45.144 INFO:tasks.workunit.client.1.smithi174.stdout:7/53: rename d0/l13 to d0/d4/l15 0 2024-03-21T10:08:45.149 INFO:tasks.workunit.client.0.smithi153.stdout:8/284: mkdir d5/d9/d43/dd/d20/d26/d24/d52/d56 0 2024-03-21T10:08:45.150 INFO:tasks.workunit.client.0.smithi153.stdout:8/285: chown d5 16617543 1 2024-03-21T10:08:45.154 INFO:tasks.workunit.client.0.smithi153.stdout:1/265: stat d6/d13/f2b 0 2024-03-21T10:08:45.155 INFO:tasks.workunit.client.1.smithi174.stdout:3/47: symlink d0/d12/l15 0 2024-03-21T10:08:45.157 INFO:tasks.workunit.client.0.smithi153.stdout:8/286: chown d5/d9/d43/dd/d2d/d45 2 1 2024-03-21T10:08:45.159 INFO:tasks.workunit.client.1.smithi174.stdout:4/133: mknod d4/d21/c33 0 2024-03-21T10:08:45.163 INFO:tasks.workunit.client.1.smithi174.stdout:2/117: dwrite d5/d8/f1c [4194304,4194304] 0 2024-03-21T10:08:45.164 INFO:tasks.workunit.client.1.smithi174.stdout:4/134: dread - d4/d21/f27 zero size 2024-03-21T10:08:45.166 INFO:tasks.workunit.client.1.smithi174.stdout:0/51: write d9/fc [4935774,79819] 0 2024-03-21T10:08:45.167 INFO:tasks.workunit.client.0.smithi153.stdout:6/465: dread d1f/d2b/d33/d40/f8f [0,4194304] 0 2024-03-21T10:08:45.167 INFO:tasks.workunit.client.1.smithi174.stdout:4/135: write d4/f20 [804169,93636] 0 2024-03-21T10:08:45.170 INFO:tasks.workunit.client.1.smithi174.stdout:1/88: mknod d4/d7/c1d 0 2024-03-21T10:08:45.170 INFO:tasks.workunit.client.1.smithi174.stdout:5/57: symlink d1/lb 0 2024-03-21T10:08:45.173 INFO:tasks.workunit.client.1.smithi174.stdout:5/58: fsync d1/f3 0 2024-03-21T10:08:45.173 INFO:tasks.workunit.client.0.smithi153.stdout:0/209: dwrite d4/db/f29 [0,4194304] 0 2024-03-21T10:08:45.173 INFO:tasks.workunit.client.0.smithi153.stdout:5/265: dwrite d4/d5/df/f34 [0,4194304] 0 2024-03-21T10:08:45.174 INFO:tasks.workunit.client.0.smithi153.stdout:8/287: write d5/d9/f7 [5914398,62786] 0 2024-03-21T10:08:45.180 INFO:tasks.workunit.client.1.smithi174.stdout:7/54: unlink d0/l6 0 2024-03-21T10:08:45.180 INFO:tasks.workunit.client.0.smithi153.stdout:6/466: creat d1f/da0/fb0 x:0 0 0 2024-03-21T10:08:45.182 INFO:tasks.workunit.client.0.smithi153.stdout:1/266: creat d6/d3a/f53 x:0 0 0 2024-03-21T10:08:45.186 INFO:tasks.workunit.client.1.smithi174.stdout:2/118: mkdir d5/d25/d27/d2e 0 2024-03-21T10:08:45.189 INFO:tasks.workunit.client.1.smithi174.stdout:0/52: dread f6 [0,4194304] 0 2024-03-21T10:08:45.194 INFO:tasks.workunit.client.1.smithi174.stdout:4/136: symlink d4/l34 0 2024-03-21T10:08:45.195 INFO:tasks.workunit.client.0.smithi153.stdout:8/288: creat d5/d9/d43/dd/d20/f57 x:0 0 0 2024-03-21T10:08:45.196 INFO:tasks.workunit.client.1.smithi174.stdout:4/137: fsync d4/f15 0 2024-03-21T10:08:45.197 INFO:tasks.workunit.client.1.smithi174.stdout:4/138: fsync d4/f2c 0 2024-03-21T10:08:45.198 INFO:tasks.workunit.client.1.smithi174.stdout:6/43: truncate d0/d3/d5/fe 1140748 0 2024-03-21T10:08:45.199 INFO:tasks.workunit.client.1.smithi174.stdout:4/139: fsync d4/d21/d24/d28/f2b 0 2024-03-21T10:08:45.201 INFO:tasks.workunit.client.1.smithi174.stdout:3/48: creat d0/d12/d14/d10/f16 x:0 0 0 2024-03-21T10:08:45.204 INFO:tasks.workunit.client.0.smithi153.stdout:0/210: getdents d4/db 0 2024-03-21T10:08:45.206 INFO:tasks.workunit.client.0.smithi153.stdout:8/289: stat d5/d9/d43/dd/d2d/d45 0 2024-03-21T10:08:45.206 INFO:tasks.workunit.client.1.smithi174.stdout:1/89: creat d4/d7/f1e x:0 0 0 2024-03-21T10:08:45.211 INFO:tasks.workunit.client.1.smithi174.stdout:5/59: creat d1/fc x:0 0 0 2024-03-21T10:08:45.213 INFO:tasks.workunit.client.1.smithi174.stdout:7/55: creat d0/d4/f16 x:0 0 0 2024-03-21T10:08:45.215 INFO:tasks.workunit.client.1.smithi174.stdout:0/53: mknod d9/cf 0 2024-03-21T10:08:45.218 INFO:tasks.workunit.client.1.smithi174.stdout:8/44: truncate d9/fd 3249387 0 2024-03-21T10:08:45.219 INFO:tasks.workunit.client.0.smithi153.stdout:2/316: dwrite dd/d15/d18/f24 [0,4194304] 0 2024-03-21T10:08:45.219 INFO:tasks.workunit.client.0.smithi153.stdout:9/259: dwrite d1/d43/d44/d46/f37 [0,4194304] 0 2024-03-21T10:08:45.223 INFO:tasks.workunit.client.0.smithi153.stdout:2/317: stat c4 0 2024-03-21T10:08:45.225 INFO:tasks.workunit.client.1.smithi174.stdout:1/90: creat d4/d7/d15/f1f x:0 0 0 2024-03-21T10:08:45.226 INFO:tasks.workunit.client.0.smithi153.stdout:5/266: dwrite d4/f46 [0,4194304] 0 2024-03-21T10:08:45.228 INFO:tasks.workunit.client.1.smithi174.stdout:5/60: creat d1/fd x:0 0 0 2024-03-21T10:08:45.230 INFO:tasks.workunit.client.1.smithi174.stdout:5/61: truncate d1/fc 628177 0 2024-03-21T10:08:45.230 INFO:tasks.workunit.client.0.smithi153.stdout:1/267: dwrite d6/d10/d2c/f2e [0,4194304] 0 2024-03-21T10:08:45.237 INFO:tasks.workunit.client.0.smithi153.stdout:2/318: symlink dd/d15/d3c/l6c 0 2024-03-21T10:08:45.238 INFO:tasks.workunit.client.1.smithi174.stdout:7/56: creat d0/d4/f17 x:0 0 0 2024-03-21T10:08:45.239 INFO:tasks.workunit.client.1.smithi174.stdout:7/57: dread - d0/ff zero size 2024-03-21T10:08:45.241 INFO:tasks.workunit.client.0.smithi153.stdout:5/267: mkdir d4/d8/d5c 0 2024-03-21T10:08:45.242 INFO:tasks.workunit.client.1.smithi174.stdout:7/58: write d0/d4/f12 [897704,39449] 0 2024-03-21T10:08:45.242 INFO:tasks.workunit.client.0.smithi153.stdout:9/260: write d1/d43/d44/f34 [1283131,103108] 0 2024-03-21T10:08:45.244 INFO:tasks.workunit.client.0.smithi153.stdout:4/307: dread d11/d2d/f3a [0,4194304] 0 2024-03-21T10:08:45.246 INFO:tasks.workunit.client.0.smithi153.stdout:9/261: readlink d1/d4/l2f 0 2024-03-21T10:08:45.246 INFO:tasks.workunit.client.1.smithi174.stdout:8/45: creat d9/f13 x:0 0 0 2024-03-21T10:08:45.247 INFO:tasks.workunit.client.0.smithi153.stdout:2/319: mknod dd/d15/d3c/c6d 0 2024-03-21T10:08:45.247 INFO:tasks.workunit.client.1.smithi174.stdout:4/140: mknod d4/d21/d31/c35 0 2024-03-21T10:08:45.248 INFO:tasks.workunit.client.0.smithi153.stdout:6/467: dread f1b [0,4194304] 0 2024-03-21T10:08:45.248 INFO:tasks.workunit.client.0.smithi153.stdout:8/290: dread d5/f8 [0,4194304] 0 2024-03-21T10:08:45.249 INFO:tasks.workunit.client.1.smithi174.stdout:4/141: write d4/f2c [48642,110289] 0 2024-03-21T10:08:45.250 INFO:tasks.workunit.client.0.smithi153.stdout:4/308: chown d11/d18/f49 6459214 1 2024-03-21T10:08:45.255 INFO:tasks.workunit.client.0.smithi153.stdout:9/262: mkdir d1/d52 0 2024-03-21T10:08:45.257 INFO:tasks.workunit.client.0.smithi153.stdout:2/320: symlink dd/d15/d18/d2c/l6e 0 2024-03-21T10:08:45.258 INFO:tasks.workunit.client.1.smithi174.stdout:5/62: mknod d1/ce 0 2024-03-21T10:08:45.259 INFO:tasks.workunit.client.1.smithi174.stdout:7/59: creat d0/d4/f18 x:0 0 0 2024-03-21T10:08:45.260 INFO:tasks.workunit.client.0.smithi153.stdout:4/309: readlink d11/l17 0 2024-03-21T10:08:45.262 INFO:tasks.workunit.client.0.smithi153.stdout:2/321: read dd/d15/d18/f1a [3302718,2666] 0 2024-03-21T10:08:45.266 INFO:tasks.workunit.client.0.smithi153.stdout:9/263: rename d1/d43/d44/d46/d1c/d24/f49 to d1/d4e/f53 0 2024-03-21T10:08:45.266 INFO:tasks.workunit.client.0.smithi153.stdout:2/322: dread - dd/d15/d18/d2c/d4b/f66 zero size 2024-03-21T10:08:45.266 INFO:tasks.workunit.client.1.smithi174.stdout:8/46: unlink f0 0 2024-03-21T10:08:45.268 INFO:tasks.workunit.client.0.smithi153.stdout:4/310: chown d11/c33 2018068110 1 2024-03-21T10:08:45.270 INFO:tasks.workunit.client.1.smithi174.stdout:4/142: mknod d4/d21/d24/c36 0 2024-03-21T10:08:45.274 INFO:tasks.workunit.client.0.smithi153.stdout:9/264: rename d1/d4/f14 to d1/d4e/f54 0 2024-03-21T10:08:45.275 INFO:tasks.workunit.client.1.smithi174.stdout:0/54: link d9/ca d9/c10 0 2024-03-21T10:08:45.276 INFO:tasks.workunit.client.1.smithi174.stdout:4/143: truncate d4/d19/f2d 165694 0 2024-03-21T10:08:45.277 INFO:tasks.workunit.client.0.smithi153.stdout:9/265: truncate d1/d43/d44/f3e 935166 0 2024-03-21T10:08:45.278 INFO:tasks.workunit.client.0.smithi153.stdout:0/211: dread d4/f6 [4194304,4194304] 0 2024-03-21T10:08:45.278 INFO:tasks.workunit.client.0.smithi153.stdout:2/323: mknod dd/d15/d18/d2c/d4b/d56/d63/c6f 0 2024-03-21T10:08:45.281 INFO:tasks.workunit.client.1.smithi174.stdout:4/144: fdatasync d4/d19/f23 0 2024-03-21T10:08:45.282 INFO:tasks.workunit.client.0.smithi153.stdout:5/268: truncate d4/d5/df/f30 1579642 0 2024-03-21T10:08:45.282 INFO:tasks.workunit.client.0.smithi153.stdout:1/268: dread d6/fb [0,4194304] 0 2024-03-21T10:08:45.286 INFO:tasks.workunit.client.0.smithi153.stdout:1/269: truncate d6/d10/d2c/f33 1054655 0 2024-03-21T10:08:45.286 INFO:tasks.workunit.client.0.smithi153.stdout:6/468: truncate d1f/f35 165792 0 2024-03-21T10:08:45.287 INFO:tasks.workunit.client.1.smithi174.stdout:3/49: getdents d0 0 2024-03-21T10:08:45.288 INFO:tasks.workunit.client.0.smithi153.stdout:2/324: symlink dd/d15/d18/d2c/d33/l70 0 2024-03-21T10:08:45.288 INFO:tasks.workunit.client.0.smithi153.stdout:8/291: write d5/d9/d43/dd/d20/d26/d33/f46 [3025653,128502] 0 2024-03-21T10:08:45.292 INFO:tasks.workunit.client.0.smithi153.stdout:0/212: creat d4/d16/d19/d30/f41 x:0 0 0 2024-03-21T10:08:45.293 INFO:tasks.workunit.client.0.smithi153.stdout:4/311: dread d11/f16 [0,4194304] 0 2024-03-21T10:08:45.293 INFO:tasks.workunit.client.0.smithi153.stdout:5/269: rename d4/d8/d5a/d54 to d4/d8/d5a/d5d 0 2024-03-21T10:08:45.293 INFO:tasks.workunit.client.1.smithi174.stdout:7/60: dwrite d0/d4/f12 [0,4194304] 0 2024-03-21T10:08:45.294 INFO:tasks.workunit.client.0.smithi153.stdout:5/270: readlink d4/l35 0 2024-03-21T10:08:45.296 INFO:tasks.workunit.client.0.smithi153.stdout:1/270: mkdir d6/d10/d2c/d54 0 2024-03-21T10:08:45.296 INFO:tasks.workunit.client.0.smithi153.stdout:0/213: read d4/d16/d1f/f24 [266995,20082] 0 2024-03-21T10:08:45.297 INFO:tasks.workunit.client.0.smithi153.stdout:6/469: fdatasync d1f/d2f/d59/d70/f9d 0 2024-03-21T10:08:45.298 INFO:tasks.workunit.client.1.smithi174.stdout:2/119: truncate d5/f11 5974263 0 2024-03-21T10:08:45.299 INFO:tasks.workunit.client.0.smithi153.stdout:2/325: creat dd/d15/f71 x:0 0 0 2024-03-21T10:08:45.302 INFO:tasks.workunit.client.0.smithi153.stdout:6/470: chown d1f/d2b/d33/d40/d4f/d66/lae 299545 1 2024-03-21T10:08:45.302 INFO:tasks.workunit.client.0.smithi153.stdout:2/326: chown dd/d15/d18/d3a/l5c 47056 1 2024-03-21T10:08:45.303 INFO:tasks.workunit.client.1.smithi174.stdout:1/91: write d4/d5/fd [1035524,82794] 0 2024-03-21T10:08:45.303 INFO:tasks.workunit.client.1.smithi174.stdout:8/47: dwrite f4 [0,4194304] 0 2024-03-21T10:08:45.303 INFO:tasks.workunit.client.0.smithi153.stdout:6/471: stat l17 0 2024-03-21T10:08:45.308 INFO:tasks.workunit.client.0.smithi153.stdout:5/271: mkdir d4/d5/df/d5e 0 2024-03-21T10:08:45.310 INFO:tasks.workunit.client.0.smithi153.stdout:8/292: rename d5/c36 to d5/d9/c58 0 2024-03-21T10:08:45.311 INFO:tasks.workunit.client.1.smithi174.stdout:0/55: rmdir d9 39 2024-03-21T10:08:45.311 INFO:tasks.workunit.client.0.smithi153.stdout:8/293: readlink d5/d9/d43/dd/d20/l2a 0 2024-03-21T10:08:45.311 INFO:tasks.workunit.client.1.smithi174.stdout:5/63: dwrite d1/f6 [0,4194304] 0 2024-03-21T10:08:45.314 INFO:tasks.workunit.client.0.smithi153.stdout:2/327: unlink dd/d15/d18/c44 0 2024-03-21T10:08:45.316 INFO:tasks.workunit.client.0.smithi153.stdout:6/472: rmdir d1f/da0 39 2024-03-21T10:08:45.316 INFO:tasks.workunit.client.1.smithi174.stdout:8/48: rmdir d9 39 2024-03-21T10:08:45.323 INFO:tasks.workunit.client.1.smithi174.stdout:6/44: truncate d0/d3/d5/fe 786102 0 2024-03-21T10:08:45.323 INFO:tasks.workunit.client.1.smithi174.stdout:1/92: mkdir d4/d7/d20 0 2024-03-21T10:08:45.325 INFO:tasks.workunit.client.0.smithi153.stdout:6/473: dread - d1f/d2b/d33/d40/d5f/f80 zero size 2024-03-21T10:08:45.327 INFO:tasks.workunit.client.1.smithi174.stdout:2/120: mkdir d5/d8/d2f 0 2024-03-21T10:08:45.330 INFO:tasks.workunit.client.0.smithi153.stdout:8/294: write d5/f8 [2694228,55735] 0 2024-03-21T10:08:45.330 INFO:tasks.workunit.client.0.smithi153.stdout:2/328: creat dd/d15/d18/d2c/f72 x:0 0 0 2024-03-21T10:08:45.330 INFO:tasks.workunit.client.1.smithi174.stdout:5/64: creat d1/ff x:0 0 0 2024-03-21T10:08:45.330 INFO:tasks.workunit.client.1.smithi174.stdout:2/121: readlink d5/d8/lc 0 2024-03-21T10:08:45.331 INFO:tasks.workunit.client.1.smithi174.stdout:1/93: write d4/d7/f1a [224181,55868] 0 2024-03-21T10:08:45.333 INFO:tasks.workunit.client.1.smithi174.stdout:1/94: chown d4/fb 7200418 1 2024-03-21T10:08:45.334 INFO:tasks.workunit.client.0.smithi153.stdout:1/271: getdents d6/d3a 0 2024-03-21T10:08:45.335 INFO:tasks.workunit.client.1.smithi174.stdout:5/65: dread d1/fc [0,4194304] 0 2024-03-21T10:08:45.335 INFO:tasks.workunit.client.1.smithi174.stdout:1/95: fsync d4/d7/f1e 0 2024-03-21T10:08:45.336 INFO:tasks.workunit.client.0.smithi153.stdout:8/295: creat d5/d1b/f59 x:0 0 0 2024-03-21T10:08:45.337 INFO:tasks.workunit.client.1.smithi174.stdout:4/145: dwrite d4/f2c [0,4194304] 0 2024-03-21T10:08:45.337 INFO:tasks.workunit.client.0.smithi153.stdout:5/272: dwrite d4/d5/de/f43 [0,4194304] 0 2024-03-21T10:08:45.340 INFO:tasks.workunit.client.1.smithi174.stdout:5/66: rename d1 to d1/d10 22 2024-03-21T10:08:45.340 INFO:tasks.workunit.client.1.smithi174.stdout:7/61: creat d0/d4/d10/f19 x:0 0 0 2024-03-21T10:08:45.340 INFO:tasks.workunit.client.1.smithi174.stdout:7/62: stat d0/f9 0 2024-03-21T10:08:45.342 INFO:tasks.workunit.client.0.smithi153.stdout:1/272: mknod d6/d3a/d3c/c55 0 2024-03-21T10:08:45.342 INFO:tasks.workunit.client.0.smithi153.stdout:9/266: dread d1/d43/d44/d46/d1c/f3d [0,4194304] 0 2024-03-21T10:08:45.343 INFO:tasks.workunit.client.1.smithi174.stdout:5/67: write d1/fd [387425,129894] 0 2024-03-21T10:08:45.344 INFO:tasks.workunit.client.1.smithi174.stdout:7/63: chown d0/d4/c14 28802324 1 2024-03-21T10:08:45.345 INFO:tasks.workunit.client.0.smithi153.stdout:4/312: dread d11/d18/d50/f53 [0,4194304] 0 2024-03-21T10:08:45.345 INFO:tasks.workunit.client.1.smithi174.stdout:7/64: stat d0/fa 0 2024-03-21T10:08:45.346 INFO:tasks.workunit.client.0.smithi153.stdout:8/296: unlink d5/c44 0 2024-03-21T10:08:45.348 INFO:tasks.workunit.client.0.smithi153.stdout:2/329: rename dd/f12 to dd/d3b/d50/f73 0 2024-03-21T10:08:45.348 INFO:tasks.workunit.client.1.smithi174.stdout:5/68: write d1/ff [126310,122594] 0 2024-03-21T10:08:45.352 INFO:tasks.workunit.client.0.smithi153.stdout:8/297: write d5/d9/f7 [3817744,83894] 0 2024-03-21T10:08:45.353 INFO:tasks.workunit.client.1.smithi174.stdout:6/45: rmdir d0/d3 39 2024-03-21T10:08:45.355 INFO:tasks.workunit.client.1.smithi174.stdout:5/69: truncate d1/fd 1126818 0 2024-03-21T10:08:45.355 INFO:tasks.workunit.client.0.smithi153.stdout:9/267: mkdir d1/d43/d44/d46/d1c/d24/d41/d55 0 2024-03-21T10:08:45.358 INFO:tasks.workunit.client.1.smithi174.stdout:8/49: fsync d9/f12 0 2024-03-21T10:08:45.358 INFO:tasks.workunit.client.0.smithi153.stdout:2/330: read dd/d15/d18/f1a [3577740,41404] 0 2024-03-21T10:08:45.358 INFO:tasks.workunit.client.0.smithi153.stdout:5/273: rename d4/d5/df/f34 to d4/d8/d5a/d5d/f5f 0 2024-03-21T10:08:45.359 INFO:tasks.workunit.client.0.smithi153.stdout:1/273: mknod d6/d10/d2c/d54/c56 0 2024-03-21T10:08:45.360 INFO:tasks.workunit.client.1.smithi174.stdout:1/96: creat d4/d7/f21 x:0 0 0 2024-03-21T10:08:45.362 INFO:tasks.workunit.client.0.smithi153.stdout:1/274: chown d6/d32/l49 348 1 2024-03-21T10:08:45.362 INFO:tasks.workunit.client.0.smithi153.stdout:2/331: write dd/d3b/f5b [421989,20823] 0 2024-03-21T10:08:45.362 INFO:tasks.workunit.client.0.smithi153.stdout:5/274: stat d4/d5/de/d2f/l37 0 2024-03-21T10:08:45.363 INFO:tasks.workunit.client.1.smithi174.stdout:2/122: symlink d5/d8/d2f/l30 0 2024-03-21T10:08:45.364 INFO:tasks.workunit.client.0.smithi153.stdout:5/275: chown d4/d8/d5c 7 1 2024-03-21T10:08:45.366 INFO:tasks.workunit.client.0.smithi153.stdout:8/298: creat d5/d9/d43/dd/d20/d26/d24/f5a x:0 0 0 2024-03-21T10:08:45.368 INFO:tasks.workunit.client.0.smithi153.stdout:5/276: dread - d4/d8/d5a/d5d/f5b zero size 2024-03-21T10:08:45.368 INFO:tasks.workunit.client.0.smithi153.stdout:4/313: rename d11/f26 to d11/d18/f61 0 2024-03-21T10:08:45.369 INFO:tasks.workunit.client.0.smithi153.stdout:0/214: dread d4/f10 [0,4194304] 0 2024-03-21T10:08:45.372 INFO:tasks.workunit.client.1.smithi174.stdout:5/70: creat d1/f11 x:0 0 0 2024-03-21T10:08:45.373 INFO:tasks.workunit.client.1.smithi174.stdout:4/146: dwrite d4/d19/f1f [0,4194304] 0 2024-03-21T10:08:45.373 INFO:tasks.workunit.client.1.smithi174.stdout:1/97: unlink d4/lc 0 2024-03-21T10:08:45.374 INFO:tasks.workunit.client.0.smithi153.stdout:9/268: link d1/d43/l45 d1/d43/d44/d46/d1c/d2e/l56 0 2024-03-21T10:08:45.374 INFO:tasks.workunit.client.0.smithi153.stdout:8/299: rename d5/d9/le to d5/d3b/l5b 0 2024-03-21T10:08:45.375 INFO:tasks.workunit.client.0.smithi153.stdout:0/215: chown d4/d16/d1f/f31 53221222 1 2024-03-21T10:08:45.375 INFO:tasks.workunit.client.1.smithi174.stdout:1/98: chown d4/d5 15 1 2024-03-21T10:08:45.377 INFO:tasks.workunit.client.0.smithi153.stdout:0/216: read - d4/d16/d1f/f35 zero size 2024-03-21T10:08:45.378 INFO:tasks.workunit.client.1.smithi174.stdout:7/65: rmdir d0/d4/de 0 2024-03-21T10:08:45.382 INFO:tasks.workunit.client.1.smithi174.stdout:5/71: creat d1/f12 x:0 0 0 2024-03-21T10:08:45.383 INFO:tasks.workunit.client.1.smithi174.stdout:0/56: truncate f4 773444 0 2024-03-21T10:08:45.386 INFO:tasks.workunit.client.1.smithi174.stdout:7/66: dread - d0/d4/f17 zero size 2024-03-21T10:08:45.386 INFO:tasks.workunit.client.1.smithi174.stdout:1/99: fsync d4/d7/ff 0 2024-03-21T10:08:45.386 INFO:tasks.workunit.client.0.smithi153.stdout:3/291: sync 2024-03-21T10:08:45.387 INFO:tasks.workunit.client.0.smithi153.stdout:7/185: sync 2024-03-21T10:08:45.389 INFO:tasks.workunit.client.0.smithi153.stdout:3/292: stat dd/d1e/d2f/d35/l40 0 2024-03-21T10:08:45.392 INFO:tasks.workunit.client.1.smithi174.stdout:7/67: write d0/d4/f16 [525461,109251] 0 2024-03-21T10:08:45.392 INFO:tasks.workunit.client.1.smithi174.stdout:8/50: getdents d9 0 2024-03-21T10:08:45.393 INFO:tasks.workunit.client.0.smithi153.stdout:4/314: dwrite d11/d18/f35 [0,4194304] 0 2024-03-21T10:08:45.394 INFO:tasks.workunit.client.0.smithi153.stdout:9/269: mknod d1/d4/c57 0 2024-03-21T10:08:45.396 INFO:tasks.workunit.client.0.smithi153.stdout:6/474: dread f11 [0,4194304] 0 2024-03-21T10:08:45.396 INFO:tasks.workunit.client.0.smithi153.stdout:1/275: dwrite d6/f17 [4194304,4194304] 0 2024-03-21T10:08:45.398 INFO:tasks.workunit.client.0.smithi153.stdout:0/217: write d4/d16/d1f/d22/f2e [7606505,45464] 0 2024-03-21T10:08:45.399 INFO:tasks.workunit.client.1.smithi174.stdout:1/100: rename d4/d5/f9 to d4/d5/f22 0 2024-03-21T10:08:45.400 INFO:tasks.workunit.client.1.smithi174.stdout:2/123: dwrite d5/d25/d27/f2a [0,4194304] 0 2024-03-21T10:08:45.402 INFO:tasks.workunit.client.0.smithi153.stdout:3/293: creat dd/d41/f64 x:0 0 0 2024-03-21T10:08:45.402 INFO:tasks.workunit.client.0.smithi153.stdout:4/315: rmdir d11/d18 39 2024-03-21T10:08:45.403 INFO:tasks.workunit.client.0.smithi153.stdout:7/186: mkdir d4/d2a/d3b 0 2024-03-21T10:08:45.404 INFO:tasks.workunit.client.1.smithi174.stdout:2/124: chown d5/c7 45463718 1 2024-03-21T10:08:45.407 INFO:tasks.workunit.client.0.smithi153.stdout:2/332: getdents dd/d15/d5e 0 2024-03-21T10:08:45.410 INFO:tasks.workunit.client.0.smithi153.stdout:2/333: fsync dd/f37 0 2024-03-21T10:08:45.410 INFO:tasks.workunit.client.0.smithi153.stdout:0/218: mknod d4/db/c42 0 2024-03-21T10:08:45.415 INFO:tasks.workunit.client.0.smithi153.stdout:3/294: mkdir dd/d1e/d65 0 2024-03-21T10:08:45.418 INFO:tasks.workunit.client.1.smithi174.stdout:8/51: unlink d9/f13 0 2024-03-21T10:08:45.418 INFO:tasks.workunit.client.1.smithi174.stdout:7/68: mknod d0/d4/d10/c1a 0 2024-03-21T10:08:45.418 INFO:tasks.workunit.client.0.smithi153.stdout:7/187: mknod d4/d2a/c3c 0 2024-03-21T10:08:45.419 INFO:tasks.workunit.client.0.smithi153.stdout:1/276: read f5 [2616624,103974] 0 2024-03-21T10:08:45.421 INFO:tasks.workunit.client.0.smithi153.stdout:7/188: write d4/f25 [3270425,73011] 0 2024-03-21T10:08:45.423 INFO:tasks.workunit.client.0.smithi153.stdout:0/219: symlink d4/d16/d19/l43 0 2024-03-21T10:08:45.427 INFO:tasks.workunit.client.0.smithi153.stdout:3/295: mkdir dd/d1e/d2f/d35/d66 0 2024-03-21T10:08:45.428 INFO:tasks.workunit.client.0.smithi153.stdout:7/189: truncate d4/d11/d24/f29 4326855 0 2024-03-21T10:08:45.431 INFO:tasks.workunit.client.1.smithi174.stdout:8/52: write f1 [981700,36754] 0 2024-03-21T10:08:45.432 INFO:tasks.workunit.client.1.smithi174.stdout:5/72: dwrite d1/fd [0,4194304] 0 2024-03-21T10:08:45.433 INFO:tasks.workunit.client.0.smithi153.stdout:1/277: unlink d6/d3a/d3c/f41 0 2024-03-21T10:08:45.434 INFO:tasks.workunit.client.0.smithi153.stdout:2/334: dread dd/f26 [0,4194304] 0 2024-03-21T10:08:45.435 INFO:tasks.workunit.client.1.smithi174.stdout:2/125: symlink d5/d25/d27/d2e/l31 0 2024-03-21T10:08:45.440 INFO:tasks.workunit.client.1.smithi174.stdout:2/126: dread - d5/d8/f1d zero size 2024-03-21T10:08:45.445 INFO:tasks.workunit.client.0.smithi153.stdout:0/220: mknod d4/db/c44 0 2024-03-21T10:08:45.445 INFO:tasks.workunit.client.0.smithi153.stdout:8/300: dread d5/d9/d43/dd/f38 [0,4194304] 0 2024-03-21T10:08:45.446 INFO:tasks.workunit.client.0.smithi153.stdout:3/296: unlink dd/d1e/d2f/c44 0 2024-03-21T10:08:45.447 INFO:tasks.workunit.client.0.smithi153.stdout:9/270: rmdir d1/d43/d44/d46/d1c/d24 39 2024-03-21T10:08:45.448 INFO:tasks.workunit.client.1.smithi174.stdout:2/127: write d5/f10 [2027904,36952] 0 2024-03-21T10:08:45.449 INFO:tasks.workunit.client.0.smithi153.stdout:7/190: mknod d4/d2a/c3d 0 2024-03-21T10:08:45.450 INFO:tasks.workunit.client.1.smithi174.stdout:1/101: dwrite d4/d7/f1a [0,4194304] 0 2024-03-21T10:08:45.452 INFO:tasks.workunit.client.0.smithi153.stdout:9/271: chown d1/d43/d44/l28 17758 1 2024-03-21T10:08:45.453 INFO:tasks.workunit.client.1.smithi174.stdout:0/57: dwrite d9/fc [0,4194304] 0 2024-03-21T10:08:45.453 INFO:tasks.workunit.client.0.smithi153.stdout:1/278: unlink d6/f18 0 2024-03-21T10:08:45.455 INFO:tasks.workunit.client.0.smithi153.stdout:7/191: read d4/f25 [2479521,36622] 0 2024-03-21T10:08:45.455 INFO:tasks.workunit.client.0.smithi153.stdout:1/279: chown d6/d10 13004 1 2024-03-21T10:08:45.456 INFO:tasks.workunit.client.1.smithi174.stdout:0/58: write f6 [1161333,26282] 0 2024-03-21T10:08:45.458 INFO:tasks.workunit.client.1.smithi174.stdout:8/53: creat d9/f14 x:0 0 0 2024-03-21T10:08:45.458 INFO:tasks.workunit.client.0.smithi153.stdout:2/335: fsync dd/d15/d18/d2c/f39 0 2024-03-21T10:08:45.464 INFO:tasks.workunit.client.0.smithi153.stdout:7/192: chown d4/d11/d24/f3a 1319 1 2024-03-21T10:08:45.466 INFO:tasks.workunit.client.1.smithi174.stdout:2/128: stat d5/c16 0 2024-03-21T10:08:45.468 INFO:tasks.workunit.client.1.smithi174.stdout:1/102: mknod d4/d5/c23 0 2024-03-21T10:08:45.468 INFO:tasks.workunit.client.0.smithi153.stdout:0/221: dread d4/db/f29 [0,4194304] 0 2024-03-21T10:08:45.468 INFO:tasks.workunit.client.0.smithi153.stdout:7/193: write d4/d14/d16/f39 [786349,103870] 0 2024-03-21T10:08:45.474 INFO:tasks.workunit.client.0.smithi153.stdout:5/277: write d4/d5/de/f21 [1976387,120485] 0 2024-03-21T10:08:45.475 INFO:tasks.workunit.client.0.smithi153.stdout:6/475: dread d1f/d2b/d33/f3c [4194304,4194304] 0 2024-03-21T10:08:45.476 INFO:tasks.workunit.client.1.smithi174.stdout:6/46: dwrite d0/d3/d5/fe [0,4194304] 0 2024-03-21T10:08:45.477 INFO:tasks.workunit.client.0.smithi153.stdout:6/476: write d1f/f4b [1584016,35851] 0 2024-03-21T10:08:45.480 INFO:tasks.workunit.client.0.smithi153.stdout:9/272: creat d1/d43/d44/f58 x:0 0 0 2024-03-21T10:08:45.480 INFO:tasks.workunit.client.1.smithi174.stdout:1/103: creat d4/d7/d15/f24 x:0 0 0 2024-03-21T10:08:45.482 INFO:tasks.workunit.client.0.smithi153.stdout:2/336: creat dd/d15/d18/d3a/d3d/f74 x:0 0 0 2024-03-21T10:08:45.484 INFO:tasks.workunit.client.0.smithi153.stdout:3/297: unlink dd/c21 0 2024-03-21T10:08:45.485 INFO:tasks.workunit.client.1.smithi174.stdout:7/69: getdents d0 0 2024-03-21T10:08:45.486 INFO:tasks.workunit.client.1.smithi174.stdout:1/104: chown d4/d7/d15/c1c 2207443 1 2024-03-21T10:08:45.491 INFO:tasks.workunit.client.0.smithi153.stdout:4/316: truncate d11/d18/f28 528663 0 2024-03-21T10:08:45.494 INFO:tasks.workunit.client.0.smithi153.stdout:8/301: creat d5/d9/d43/dd/d20/d26/d24/d52/d55/f5c x:0 0 0 2024-03-21T10:08:45.494 INFO:tasks.workunit.client.1.smithi174.stdout:0/59: getdents d9 0 2024-03-21T10:08:45.494 INFO:tasks.workunit.client.1.smithi174.stdout:6/47: creat d0/d3/d5/d6/f13 x:0 0 0 2024-03-21T10:08:45.496 INFO:tasks.workunit.client.0.smithi153.stdout:7/194: dwrite d4/d2a/f2d [0,4194304] 0 2024-03-21T10:08:45.497 INFO:tasks.workunit.client.0.smithi153.stdout:9/273: rmdir d1 39 2024-03-21T10:08:45.498 INFO:tasks.workunit.client.1.smithi174.stdout:5/73: dwrite d1/fd [4194304,4194304] 0 2024-03-21T10:08:45.499 INFO:tasks.workunit.client.1.smithi174.stdout:7/70: mkdir d0/d4/d10/d1b 0 2024-03-21T10:08:45.500 INFO:tasks.workunit.client.0.smithi153.stdout:3/298: creat dd/f67 x:0 0 0 2024-03-21T10:08:45.501 INFO:tasks.workunit.client.0.smithi153.stdout:2/337: fsync dd/d15/d18/d2c/d33/f51 0 2024-03-21T10:08:45.501 INFO:tasks.workunit.client.0.smithi153.stdout:4/317: rmdir d11/d5e 39 2024-03-21T10:08:45.503 INFO:tasks.workunit.client.0.smithi153.stdout:1/280: dwrite d6/f7 [0,4194304] 0 2024-03-21T10:08:45.503 INFO:tasks.workunit.client.1.smithi174.stdout:0/60: write f7 [4849795,114691] 0 2024-03-21T10:08:45.504 INFO:tasks.workunit.client.1.smithi174.stdout:6/48: readlink d0/d3/d5/d6/lc 0 2024-03-21T10:08:45.506 INFO:tasks.workunit.client.1.smithi174.stdout:7/71: chown d0/d4/f18 12059 1 2024-03-21T10:08:45.506 INFO:tasks.workunit.client.1.smithi174.stdout:0/61: stat d9/cf 0 2024-03-21T10:08:45.507 INFO:tasks.workunit.client.1.smithi174.stdout:8/54: dread d9/fd [0,4194304] 0 2024-03-21T10:08:45.508 INFO:tasks.workunit.client.1.smithi174.stdout:6/49: fsync d0/d3/f8 0 2024-03-21T10:08:45.508 INFO:tasks.workunit.client.1.smithi174.stdout:5/74: creat d1/f13 x:0 0 0 2024-03-21T10:08:45.514 INFO:tasks.workunit.client.0.smithi153.stdout:8/302: creat d5/d9/d43/dd/d20/d26/d33/f5d x:0 0 0 2024-03-21T10:08:45.516 INFO:tasks.workunit.client.1.smithi174.stdout:6/50: chown d0/d3/d5/df 292 1 2024-03-21T10:08:45.516 INFO:tasks.workunit.client.1.smithi174.stdout:8/55: fsync d9/fe 0 2024-03-21T10:08:45.518 INFO:tasks.workunit.client.0.smithi153.stdout:7/195: dread d4/fa [0,4194304] 0 2024-03-21T10:08:45.518 INFO:tasks.workunit.client.0.smithi153.stdout:2/338: symlink dd/d15/d3c/l75 0 2024-03-21T10:08:45.524 INFO:tasks.workunit.client.1.smithi174.stdout:5/75: symlink d1/l14 0 2024-03-21T10:08:45.524 INFO:tasks.workunit.client.0.smithi153.stdout:2/339: fdatasync dd/d15/d3c/f69 0 2024-03-21T10:08:45.526 INFO:tasks.workunit.client.1.smithi174.stdout:8/56: chown d9/fe 187 1 2024-03-21T10:08:45.530 INFO:tasks.workunit.client.1.smithi174.stdout:1/105: dwrite d4/d7/f1b [0,4194304] 0 2024-03-21T10:08:45.530 INFO:tasks.workunit.client.1.smithi174.stdout:7/72: link d0/d4/fd d0/d4/d10/f1c 0 2024-03-21T10:08:45.532 INFO:tasks.workunit.client.1.smithi174.stdout:6/51: rename d0/l4 to d0/d3/d5/df/l14 0 2024-03-21T10:08:45.533 INFO:tasks.workunit.client.1.smithi174.stdout:5/76: mknod d1/c15 0 2024-03-21T10:08:45.533 INFO:tasks.workunit.client.0.smithi153.stdout:2/340: write dd/d15/f52 [5344988,57921] 0 2024-03-21T10:08:45.535 INFO:tasks.workunit.client.0.smithi153.stdout:6/477: getdents d1f/d2b 0 2024-03-21T10:08:45.536 INFO:tasks.workunit.client.0.smithi153.stdout:4/318: dwrite d11/d18/f35 [4194304,4194304] 0 2024-03-21T10:08:45.537 INFO:tasks.workunit.client.1.smithi174.stdout:8/57: rename f4 to d9/f15 0 2024-03-21T10:08:45.537 INFO:tasks.workunit.client.1.smithi174.stdout:1/106: fsync d4/d5/f18 0 2024-03-21T10:08:45.537 INFO:tasks.workunit.client.0.smithi153.stdout:0/222: write d4/f10 [4066923,24328] 0 2024-03-21T10:08:45.538 INFO:tasks.workunit.client.1.smithi174.stdout:5/77: mkdir d1/d16 0 2024-03-21T10:08:45.541 INFO:tasks.workunit.client.1.smithi174.stdout:6/52: rmdir d0/d3 39 2024-03-21T10:08:45.543 INFO:tasks.workunit.client.0.smithi153.stdout:6/478: dread - d1f/d2b/fa7 zero size 2024-03-21T10:08:45.543 INFO:tasks.workunit.client.1.smithi174.stdout:0/62: truncate d9/fc 1974996 0 2024-03-21T10:08:45.547 INFO:tasks.workunit.client.0.smithi153.stdout:0/223: readlink d4/d16/d27/l40 0 2024-03-21T10:08:45.547 INFO:tasks.workunit.client.0.smithi153.stdout:9/274: creat d1/f59 x:0 0 0 2024-03-21T10:08:45.547 INFO:tasks.workunit.client.1.smithi174.stdout:9/128: sync 2024-03-21T10:08:45.547 INFO:tasks.workunit.client.1.smithi174.stdout:8/58: creat d9/f16 x:0 0 0 2024-03-21T10:08:45.552 INFO:tasks.workunit.client.1.smithi174.stdout:8/59: write f1 [1654072,56364] 0 2024-03-21T10:08:45.553 INFO:tasks.workunit.client.1.smithi174.stdout:1/107: symlink d4/d7/d15/l25 0 2024-03-21T10:08:45.555 INFO:tasks.workunit.client.0.smithi153.stdout:3/299: getdents dd/d1e/d2f/d35 0 2024-03-21T10:08:45.556 INFO:tasks.workunit.client.0.smithi153.stdout:1/281: link d6/d13/f1f d6/d10/d2c/d54/f57 0 2024-03-21T10:08:45.557 INFO:tasks.workunit.client.1.smithi174.stdout:0/63: mknod d9/c11 0 2024-03-21T10:08:45.558 INFO:tasks.workunit.client.0.smithi153.stdout:3/300: chown f2 117557 1 2024-03-21T10:08:45.560 INFO:tasks.workunit.client.0.smithi153.stdout:8/303: dwrite d5/d9/f11 [0,4194304] 0 2024-03-21T10:08:45.561 INFO:tasks.workunit.client.1.smithi174.stdout:8/60: readlink d9/l10 0 2024-03-21T10:08:45.562 INFO:tasks.workunit.client.1.smithi174.stdout:1/108: truncate d4/d5/f18 158549 0 2024-03-21T10:08:45.563 INFO:tasks.workunit.client.0.smithi153.stdout:3/301: readlink dd/d1e/d45/d56/l63 0 2024-03-21T10:08:45.565 INFO:tasks.workunit.client.0.smithi153.stdout:7/196: dwrite d4/d19/f32 [0,4194304] 0 2024-03-21T10:08:45.565 INFO:tasks.workunit.client.1.smithi174.stdout:6/53: symlink d0/d3/l15 0 2024-03-21T10:08:45.566 INFO:tasks.workunit.client.0.smithi153.stdout:0/224: mknod d4/d16/d19/c45 0 2024-03-21T10:08:45.568 INFO:tasks.workunit.client.0.smithi153.stdout:5/278: dread f1 [0,4194304] 0 2024-03-21T10:08:45.568 INFO:tasks.workunit.client.0.smithi153.stdout:3/302: dread - dd/d1e/d45/d56/d58/f5f zero size 2024-03-21T10:08:45.568 INFO:tasks.workunit.client.0.smithi153.stdout:5/279: chown d4/d5/df 81 1 2024-03-21T10:08:45.571 INFO:tasks.workunit.client.1.smithi174.stdout:0/64: stat l8 0 2024-03-21T10:08:45.572 INFO:tasks.workunit.client.1.smithi174.stdout:9/129: creat d2/d11/f29 x:0 0 0 2024-03-21T10:08:45.572 INFO:tasks.workunit.client.1.smithi174.stdout:7/73: dwrite d0/d4/f17 [0,4194304] 0 2024-03-21T10:08:45.573 INFO:tasks.workunit.client.1.smithi174.stdout:5/78: dread d1/fd [4194304,4194304] 0 2024-03-21T10:08:45.575 INFO:tasks.workunit.client.0.smithi153.stdout:5/280: chown d4/d8/d3b 741124 1 2024-03-21T10:08:45.575 INFO:tasks.workunit.client.0.smithi153.stdout:4/319: mknod d11/d18/d54/c62 0 2024-03-21T10:08:45.580 INFO:tasks.workunit.client.0.smithi153.stdout:4/320: write d11/d2d/f5a [398487,19390] 0 2024-03-21T10:08:45.581 INFO:tasks.workunit.client.0.smithi153.stdout:1/282: creat d6/d32/d3e/d4f/f58 x:0 0 0 2024-03-21T10:08:45.583 INFO:tasks.workunit.client.0.smithi153.stdout:9/275: dread d1/f27 [0,4194304] 0 2024-03-21T10:08:45.585 INFO:tasks.workunit.client.0.smithi153.stdout:0/225: symlink d4/db/l46 0 2024-03-21T10:08:45.592 INFO:tasks.workunit.client.0.smithi153.stdout:8/304: unlink d5/d3b/l5b 0 2024-03-21T10:08:45.593 INFO:tasks.workunit.client.0.smithi153.stdout:1/283: symlink d6/d3a/d3c/d50/l59 0 2024-03-21T10:08:45.595 INFO:tasks.workunit.client.1.smithi174.stdout:5/79: rename d1/lb to d1/d16/l17 0 2024-03-21T10:08:45.596 INFO:tasks.workunit.client.0.smithi153.stdout:3/303: dwrite dd/d1e/d45/f52 [4194304,4194304] 0 2024-03-21T10:08:45.597 INFO:tasks.workunit.client.0.smithi153.stdout:7/197: dread d4/d1b/d20/f2c [0,4194304] 0 2024-03-21T10:08:45.600 INFO:tasks.workunit.client.0.smithi153.stdout:0/226: creat d4/d16/d1f/d22/f47 x:0 0 0 2024-03-21T10:08:45.607 INFO:tasks.workunit.client.0.smithi153.stdout:1/284: truncate d6/d3a/f53 720211 0 2024-03-21T10:08:45.611 INFO:tasks.workunit.client.0.smithi153.stdout:2/341: truncate dd/d15/d18/d3a/d3d/f53 3158262 0 2024-03-21T10:08:45.612 INFO:tasks.workunit.client.1.smithi174.stdout:5/80: symlink d1/d16/l18 0 2024-03-21T10:08:45.613 INFO:tasks.workunit.client.0.smithi153.stdout:8/305: creat d5/d1b/f5e x:0 0 0 2024-03-21T10:08:45.613 INFO:tasks.workunit.client.0.smithi153.stdout:4/321: creat d11/f63 x:0 0 0 2024-03-21T10:08:45.613 INFO:tasks.workunit.client.1.smithi174.stdout:1/109: dwrite d4/f19 [0,4194304] 0 2024-03-21T10:08:45.614 INFO:tasks.workunit.client.0.smithi153.stdout:2/342: dread - dd/d3b/f4c zero size 2024-03-21T10:08:45.617 INFO:tasks.workunit.client.0.smithi153.stdout:5/281: dwrite d4/d8/d3b/f4b [0,4194304] 0 2024-03-21T10:08:45.617 INFO:tasks.workunit.client.0.smithi153.stdout:7/198: creat d4/d2a/f3e x:0 0 0 2024-03-21T10:08:45.618 INFO:tasks.workunit.client.1.smithi174.stdout:5/81: dread - d1/f12 zero size 2024-03-21T10:08:45.620 INFO:tasks.workunit.client.1.smithi174.stdout:5/82: stat d1/fc 0 2024-03-21T10:08:45.621 INFO:tasks.workunit.client.1.smithi174.stdout:6/54: dwrite d0/d3/d5/fe [0,4194304] 0 2024-03-21T10:08:45.622 INFO:tasks.workunit.client.1.smithi174.stdout:1/110: rename d4 to d4/d7/d20/d26 22 2024-03-21T10:08:45.629 INFO:tasks.workunit.client.1.smithi174.stdout:5/83: write d1/f3 [1958848,52836] 0 2024-03-21T10:08:45.631 INFO:tasks.workunit.client.0.smithi153.stdout:4/322: write d11/d18/f1d [3566607,100462] 0 2024-03-21T10:08:45.631 INFO:tasks.workunit.client.0.smithi153.stdout:6/479: dwrite d1f/f28 [0,4194304] 0 2024-03-21T10:08:45.631 INFO:tasks.workunit.client.0.smithi153.stdout:7/199: fdatasync d4/d11/d24/f35 0 2024-03-21T10:08:45.635 INFO:tasks.workunit.client.1.smithi174.stdout:1/111: symlink d4/l27 0 2024-03-21T10:08:45.637 INFO:tasks.workunit.client.0.smithi153.stdout:5/282: symlink d4/d5/df/l60 0 2024-03-21T10:08:45.638 INFO:tasks.workunit.client.1.smithi174.stdout:5/84: symlink d1/d16/l19 0 2024-03-21T10:08:45.638 INFO:tasks.workunit.client.1.smithi174.stdout:9/130: dwrite d2/f18 [0,4194304] 0 2024-03-21T10:08:45.640 INFO:tasks.workunit.client.0.smithi153.stdout:4/323: unlink l9 0 2024-03-21T10:08:45.642 INFO:tasks.workunit.client.1.smithi174.stdout:1/112: truncate d4/d5/f22 4291176 0 2024-03-21T10:08:45.645 INFO:tasks.workunit.client.0.smithi153.stdout:3/304: getdents dd/d1e/d2f/d35 0 2024-03-21T10:08:45.647 INFO:tasks.workunit.client.1.smithi174.stdout:5/85: creat d1/d16/f1a x:0 0 0 2024-03-21T10:08:45.647 INFO:tasks.workunit.client.1.smithi174.stdout:1/113: write d4/fb [506744,25260] 0 2024-03-21T10:08:45.649 INFO:tasks.workunit.client.0.smithi153.stdout:1/285: getdents d6/d3a/d3c/d50 0 2024-03-21T10:08:45.650 INFO:tasks.workunit.client.1.smithi174.stdout:9/131: mkdir d2/d11/d1d/d2a 0 2024-03-21T10:08:45.651 INFO:tasks.workunit.client.0.smithi153.stdout:9/276: dwrite d1/d43/d44/d46/d1c/f3d [0,4194304] 0 2024-03-21T10:08:45.652 INFO:tasks.workunit.client.0.smithi153.stdout:6/480: mknod d1f/d93/cb1 0 2024-03-21T10:08:45.654 INFO:tasks.workunit.client.1.smithi174.stdout:7/74: dwrite d0/d4/d10/f1c [0,4194304] 0 2024-03-21T10:08:45.654 INFO:tasks.workunit.client.0.smithi153.stdout:7/200: read - d4/d11/d24/f3a zero size 2024-03-21T10:08:45.654 INFO:tasks.workunit.client.0.smithi153.stdout:8/306: creat d5/d9/d43/dd/d20/d26/d24/d52/d55/f5f x:0 0 0 2024-03-21T10:08:45.660 INFO:tasks.workunit.client.1.smithi174.stdout:1/114: write d4/d7/f1e [51689,20782] 0 2024-03-21T10:08:45.664 INFO:tasks.workunit.client.0.smithi153.stdout:2/343: dwrite dd/d15/f52 [4194304,4194304] 0 2024-03-21T10:08:45.664 INFO:tasks.workunit.client.0.smithi153.stdout:4/324: creat d11/d18/f64 x:0 0 0 2024-03-21T10:08:45.666 INFO:tasks.workunit.client.0.smithi153.stdout:1/286: creat d6/d32/f5a x:0 0 0 2024-03-21T10:08:45.668 INFO:tasks.workunit.client.0.smithi153.stdout:0/227: dwrite d4/d16/d27/f38 [0,4194304] 0 2024-03-21T10:08:45.680 INFO:tasks.workunit.client.1.smithi174.stdout:9/132: rename d2/d11/d1d/l22 to d2/l2b 0 2024-03-21T10:08:45.680 INFO:tasks.workunit.client.1.smithi174.stdout:6/55: dwrite d0/d3/f8 [0,4194304] 0 2024-03-21T10:08:45.683 INFO:tasks.workunit.client.0.smithi153.stdout:4/325: write f1 [6138975,54553] 0 2024-03-21T10:08:45.684 INFO:tasks.workunit.client.0.smithi153.stdout:1/287: mkdir d6/d10/d35/d5b 0 2024-03-21T10:08:45.686 INFO:tasks.workunit.client.0.smithi153.stdout:5/283: dwrite d4/d5/de/f21 [0,4194304] 0 2024-03-21T10:08:45.689 INFO:tasks.workunit.client.1.smithi174.stdout:6/56: mkdir d0/d3/d5/df/d16 0 2024-03-21T10:08:45.691 INFO:tasks.workunit.client.1.smithi174.stdout:1/115: dread d4/d7/f1b [0,4194304] 0 2024-03-21T10:08:45.694 INFO:tasks.workunit.client.1.smithi174.stdout:7/75: dread d0/d4/f12 [0,4194304] 0 2024-03-21T10:08:45.694 INFO:tasks.workunit.client.0.smithi153.stdout:5/284: chown d4/d8/c14 31734132 1 2024-03-21T10:08:45.701 INFO:tasks.workunit.client.0.smithi153.stdout:1/288: write d6/f16 [2595651,117142] 0 2024-03-21T10:08:45.701 INFO:tasks.workunit.client.0.smithi153.stdout:2/344: rename dd/d15/d18/f45 to dd/d15/d18/d3a/d3d/f76 0 2024-03-21T10:08:45.702 INFO:tasks.workunit.client.1.smithi174.stdout:9/133: rename d2/f1a to d2/d11/f2c 0 2024-03-21T10:08:45.704 INFO:tasks.workunit.client.0.smithi153.stdout:4/326: mkdir d11/d18/d50/d65 0 2024-03-21T10:08:45.705 INFO:tasks.workunit.client.1.smithi174.stdout:1/116: mkdir d4/d7/d28 0 2024-03-21T10:08:45.706 INFO:tasks.workunit.client.1.smithi174.stdout:0/65: truncate d9/fc 2063810 0 2024-03-21T10:08:45.710 INFO:tasks.workunit.client.0.smithi153.stdout:0/228: write d4/db/f13 [1284531,60195] 0 2024-03-21T10:08:45.710 INFO:tasks.workunit.client.0.smithi153.stdout:5/285: fsync d4/d5/de/d44/d48/f59 0 2024-03-21T10:08:45.712 INFO:tasks.workunit.client.0.smithi153.stdout:3/305: dwrite dd/d1e/d2f/f37 [0,4194304] 0 2024-03-21T10:08:45.714 INFO:tasks.workunit.client.0.smithi153.stdout:0/229: dread - d4/d16/d19/d30/f41 zero size 2024-03-21T10:08:45.714 INFO:tasks.workunit.client.1.smithi174.stdout:7/76: symlink d0/d4/d10/d1b/l1d 0 2024-03-21T10:08:45.715 INFO:tasks.workunit.client.1.smithi174.stdout:9/134: mknod d2/d24/c2d 0 2024-03-21T10:08:45.716 INFO:tasks.workunit.client.0.smithi153.stdout:6/481: dwrite d1f/d2b/d33/d40/d5f/f80 [0,4194304] 0 2024-03-21T10:08:45.716 INFO:tasks.workunit.client.0.smithi153.stdout:9/277: dwrite d1/f27 [0,4194304] 0 2024-03-21T10:08:45.718 INFO:tasks.workunit.client.1.smithi174.stdout:7/77: chown d0/d4/f18 710817 1 2024-03-21T10:08:45.718 INFO:tasks.workunit.client.1.smithi174.stdout:1/117: rmdir d4/d5 39 2024-03-21T10:08:45.722 INFO:tasks.workunit.client.0.smithi153.stdout:6/482: stat f18 0 2024-03-21T10:08:45.724 INFO:tasks.workunit.client.1.smithi174.stdout:6/57: dwrite d0/d3/d5/fe [0,4194304] 0 2024-03-21T10:08:45.726 INFO:tasks.workunit.client.1.smithi174.stdout:9/135: mkdir d2/d24/d2e 0 2024-03-21T10:08:45.732 INFO:tasks.workunit.client.1.smithi174.stdout:7/78: rmdir d0/d4/d10/d1b 39 2024-03-21T10:08:45.733 INFO:tasks.workunit.client.1.smithi174.stdout:5/86: truncate f0 3409492 0 2024-03-21T10:08:45.736 INFO:tasks.workunit.client.1.smithi174.stdout:9/136: fsync d2/f1e 0 2024-03-21T10:08:45.738 INFO:tasks.workunit.client.1.smithi174.stdout:6/58: readlink d0/d3/d5/d6/lc 0 2024-03-21T10:08:45.742 INFO:tasks.workunit.client.1.smithi174.stdout:5/87: write d1/ff [513930,65941] 0 2024-03-21T10:08:45.745 INFO:tasks.workunit.client.1.smithi174.stdout:5/88: read d1/fd [2629410,129886] 0 2024-03-21T10:08:45.746 INFO:tasks.workunit.client.1.smithi174.stdout:7/79: rename d0/d4 to d0/d4/d10/d1e 22 2024-03-21T10:08:45.751 INFO:tasks.workunit.client.1.smithi174.stdout:5/89: chown d1/d16/l19 9598 1 2024-03-21T10:08:45.752 INFO:tasks.workunit.client.1.smithi174.stdout:3/50: sync 2024-03-21T10:08:45.755 INFO:tasks.workunit.client.1.smithi174.stdout:1/118: dread d4/d5/f22 [0,4194304] 0 2024-03-21T10:08:45.756 INFO:tasks.workunit.client.0.smithi153.stdout:5/286: dwrite d4/fa [0,4194304] 0 2024-03-21T10:08:45.756 INFO:tasks.workunit.client.1.smithi174.stdout:3/51: truncate d0/fa 254603 0 2024-03-21T10:08:45.758 INFO:tasks.workunit.client.0.smithi153.stdout:8/307: dwrite d5/d9/d43/f2b [0,4194304] 0 2024-03-21T10:08:45.758 INFO:tasks.workunit.client.1.smithi174.stdout:9/137: getdents d2/d24/d2e 0 2024-03-21T10:08:45.759 INFO:tasks.workunit.client.0.smithi153.stdout:6/483: dwrite d1f/d2b/d33/d40/d4f/f77 [0,4194304] 0 2024-03-21T10:08:45.759 INFO:tasks.workunit.client.1.smithi174.stdout:5/90: mknod d1/d16/c1b 0 2024-03-21T10:08:45.761 INFO:tasks.workunit.client.1.smithi174.stdout:1/119: creat d4/d7/f29 x:0 0 0 2024-03-21T10:08:45.763 INFO:tasks.workunit.client.1.smithi174.stdout:1/120: read d4/d7/ff [565135,38949] 0 2024-03-21T10:08:45.764 INFO:tasks.workunit.client.0.smithi153.stdout:4/327: dwrite d11/f1e [4194304,4194304] 0 2024-03-21T10:08:45.765 INFO:tasks.workunit.client.1.smithi174.stdout:6/59: dread d0/d3/d5/fe [0,4194304] 0 2024-03-21T10:08:45.767 INFO:tasks.workunit.client.0.smithi153.stdout:6/484: write d1f/d2f/d59/d70/f92 [646881,116336] 0 2024-03-21T10:08:45.772 INFO:tasks.workunit.client.0.smithi153.stdout:3/306: mkdir dd/d1e/d2f/d35/d68 0 2024-03-21T10:08:45.783 INFO:tasks.workunit.client.1.smithi174.stdout:7/80: dread d0/d4/f16 [0,4194304] 0 2024-03-21T10:08:45.784 INFO:tasks.workunit.client.0.smithi153.stdout:7/201: truncate d4/f10 1979453 0 2024-03-21T10:08:45.786 INFO:tasks.workunit.client.0.smithi153.stdout:2/345: fdatasync dd/d15/d18/d2c/f39 0 2024-03-21T10:08:45.790 INFO:tasks.workunit.client.0.smithi153.stdout:5/287: dread d4/d5/df/f1b [0,4194304] 0 2024-03-21T10:08:45.791 INFO:tasks.workunit.client.0.smithi153.stdout:1/289: mkdir d6/d10/d5c 0 2024-03-21T10:08:45.792 INFO:tasks.workunit.client.0.smithi153.stdout:5/288: readlink d4/l35 0 2024-03-21T10:08:45.792 INFO:tasks.workunit.client.0.smithi153.stdout:1/290: stat d6/d3a/d3c/d50/d51 0 2024-03-21T10:08:45.792 INFO:tasks.workunit.client.0.smithi153.stdout:5/289: readlink d4/d5/l31 0 2024-03-21T10:08:45.795 INFO:tasks.workunit.client.0.smithi153.stdout:5/290: write d4/d5/df/f58 [1517413,123053] 0 2024-03-21T10:08:45.797 INFO:tasks.workunit.client.0.smithi153.stdout:0/230: symlink d4/d16/d3e/l48 0 2024-03-21T10:08:45.799 INFO:tasks.workunit.client.1.smithi174.stdout:3/52: dwrite d0/d12/d14/d10/f16 [0,4194304] 0 2024-03-21T10:08:45.801 INFO:tasks.workunit.client.1.smithi174.stdout:0/66: dwrite f4 [0,4194304] 0 2024-03-21T10:08:45.810 INFO:tasks.workunit.client.0.smithi153.stdout:6/485: symlink d1f/d2b/lb2 0 2024-03-21T10:08:45.813 INFO:tasks.workunit.client.0.smithi153.stdout:6/486: dread - d1f/f4e zero size 2024-03-21T10:08:45.813 INFO:tasks.workunit.client.0.smithi153.stdout:3/307: creat dd/d1e/d3d/f69 x:0 0 0 2024-03-21T10:08:45.814 INFO:tasks.workunit.client.1.smithi174.stdout:5/91: write d1/fc [366537,65083] 0 2024-03-21T10:08:45.818 INFO:tasks.workunit.client.0.smithi153.stdout:0/231: dwrite d4/d16/d1f/d22/f47 [0,4194304] 0 2024-03-21T10:08:45.819 INFO:tasks.workunit.client.1.smithi174.stdout:9/138: truncate d2/f20 2102081 0 2024-03-21T10:08:45.820 INFO:tasks.workunit.client.0.smithi153.stdout:7/202: symlink d4/d11/l3f 0 2024-03-21T10:08:45.822 INFO:tasks.workunit.client.0.smithi153.stdout:7/203: read d4/f12 [317504,90689] 0 2024-03-21T10:08:45.823 INFO:tasks.workunit.client.0.smithi153.stdout:7/204: fdatasync d4/f12 0 2024-03-21T10:08:45.825 INFO:tasks.workunit.client.0.smithi153.stdout:7/205: chown d4/d11/d24/f35 12 1 2024-03-21T10:08:45.828 INFO:tasks.workunit.client.1.smithi174.stdout:7/81: dread d0/d4/d10/f1c [0,4194304] 0 2024-03-21T10:08:45.831 INFO:tasks.workunit.client.0.smithi153.stdout:5/291: unlink d4/d5/de/f36 0 2024-03-21T10:08:45.837 INFO:tasks.workunit.client.1.smithi174.stdout:5/92: dwrite d1/f11 [0,4194304] 0 2024-03-21T10:08:45.839 INFO:tasks.workunit.client.1.smithi174.stdout:5/93: fdatasync d1/f12 0 2024-03-21T10:08:45.840 INFO:tasks.workunit.client.0.smithi153.stdout:3/308: mknod dd/c6a 0 2024-03-21T10:08:45.840 INFO:tasks.workunit.client.0.smithi153.stdout:6/487: chown d1f/d2f/f7d 1546 1 2024-03-21T10:08:45.845 INFO:tasks.workunit.client.0.smithi153.stdout:0/232: mknod d4/d16/d3e/c49 0 2024-03-21T10:08:45.846 INFO:tasks.workunit.client.1.smithi174.stdout:3/53: creat d0/d12/f17 x:0 0 0 2024-03-21T10:08:45.847 INFO:tasks.workunit.client.0.smithi153.stdout:4/328: write d11/d18/f35 [1363206,90328] 0 2024-03-21T10:08:45.850 INFO:tasks.workunit.client.1.smithi174.stdout:3/54: truncate d0/f3 520938 0 2024-03-21T10:08:45.850 INFO:tasks.workunit.client.1.smithi174.stdout:7/82: readlink d0/d4/d10/d1b/l1d 0 2024-03-21T10:08:45.854 INFO:tasks.workunit.client.0.smithi153.stdout:7/206: mkdir d4/d2a/d40 0 2024-03-21T10:08:45.854 INFO:tasks.workunit.client.1.smithi174.stdout:3/55: write d0/f7 [977342,119611] 0 2024-03-21T10:08:45.854 INFO:tasks.workunit.client.1.smithi174.stdout:5/94: mkdir d1/d16/d1c 0 2024-03-21T10:08:45.854 INFO:tasks.workunit.client.1.smithi174.stdout:7/83: write d0/d4/f8 [803570,66188] 0 2024-03-21T10:08:45.855 INFO:tasks.workunit.client.1.smithi174.stdout:3/56: fdatasync d0/f8 0 2024-03-21T10:08:45.857 INFO:tasks.workunit.client.0.smithi153.stdout:7/207: dread - d4/d11/d24/f35 zero size 2024-03-21T10:08:45.857 INFO:tasks.workunit.client.1.smithi174.stdout:5/95: truncate d1/f12 77718 0 2024-03-21T10:08:45.859 INFO:tasks.workunit.client.1.smithi174.stdout:3/57: fdatasync d0/f8 0 2024-03-21T10:08:45.861 INFO:tasks.workunit.client.1.smithi174.stdout:5/96: creat d1/d16/d1c/f1d x:0 0 0 2024-03-21T10:08:45.863 INFO:tasks.workunit.client.1.smithi174.stdout:3/58: dread - d0/d12/f17 zero size 2024-03-21T10:08:45.864 INFO:tasks.workunit.client.1.smithi174.stdout:8/61: sync 2024-03-21T10:08:45.864 INFO:tasks.workunit.client.1.smithi174.stdout:2/129: sync 2024-03-21T10:08:45.864 INFO:tasks.workunit.client.1.smithi174.stdout:4/147: sync 2024-03-21T10:08:45.868 INFO:tasks.workunit.client.1.smithi174.stdout:2/130: fsync d5/d8/fe 0 2024-03-21T10:08:45.869 INFO:tasks.workunit.client.1.smithi174.stdout:8/62: fsync d9/fe 0 2024-03-21T10:08:45.869 INFO:tasks.workunit.client.1.smithi174.stdout:0/67: dread d9/fc [0,4194304] 0 2024-03-21T10:08:45.870 INFO:tasks.workunit.client.1.smithi174.stdout:3/59: symlink d0/d12/l18 0 2024-03-21T10:08:45.871 INFO:tasks.workunit.client.1.smithi174.stdout:2/131: chown d5/d8/f1c 961080 1 2024-03-21T10:08:45.872 INFO:tasks.workunit.client.1.smithi174.stdout:0/68: chown l8 70575534 1 2024-03-21T10:08:45.873 INFO:tasks.workunit.client.1.smithi174.stdout:4/148: creat d4/d21/f37 x:0 0 0 2024-03-21T10:08:45.874 INFO:tasks.workunit.client.0.smithi153.stdout:2/346: creat dd/d3b/f77 x:0 0 0 2024-03-21T10:08:45.874 INFO:tasks.workunit.client.0.smithi153.stdout:1/291: link d6/d3a/d3c/l4d d6/d3a/d3c/d50/l5d 0 2024-03-21T10:08:45.875 INFO:tasks.workunit.client.1.smithi174.stdout:8/63: mkdir d9/d17 0 2024-03-21T10:08:45.876 INFO:tasks.workunit.client.1.smithi174.stdout:9/139: dwrite d2/fc [0,4194304] 0 2024-03-21T10:08:45.877 INFO:tasks.workunit.client.0.smithi153.stdout:1/292: readlink d6/d32/l49 0 2024-03-21T10:08:45.877 INFO:tasks.workunit.client.1.smithi174.stdout:3/60: mknod d0/c19 0 2024-03-21T10:08:45.878 INFO:tasks.workunit.client.0.smithi153.stdout:6/488: truncate d1f/d2f/d59/f81 1703484 0 2024-03-21T10:08:45.881 INFO:tasks.workunit.client.0.smithi153.stdout:5/292: rmdir d4/d5/d16/d24/d39 0 2024-03-21T10:08:45.882 INFO:tasks.workunit.client.0.smithi153.stdout:5/293: chown d4/d5/de/d44 49964 1 2024-03-21T10:08:45.883 INFO:tasks.workunit.client.1.smithi174.stdout:3/61: chown d0/d12/d14/d10/f16 1 1 2024-03-21T10:08:45.883 INFO:tasks.workunit.client.1.smithi174.stdout:2/132: symlink d5/d25/d27/l32 0 2024-03-21T10:08:45.883 INFO:tasks.workunit.client.1.smithi174.stdout:4/149: mkdir d4/d21/d24/d28/d38 0 2024-03-21T10:08:45.884 INFO:tasks.workunit.client.0.smithi153.stdout:5/294: stat d4/c23 0 2024-03-21T10:08:45.884 INFO:tasks.workunit.client.0.smithi153.stdout:3/309: dread dd/f12 [0,4194304] 0 2024-03-21T10:08:45.887 INFO:tasks.workunit.client.0.smithi153.stdout:2/347: symlink dd/d15/d18/d3a/l78 0 2024-03-21T10:08:45.892 INFO:tasks.workunit.client.0.smithi153.stdout:6/489: mkdir d1f/d2b/db3 0 2024-03-21T10:08:45.893 INFO:tasks.workunit.client.1.smithi174.stdout:2/133: fsync d5/f12 0 2024-03-21T10:08:45.893 INFO:tasks.workunit.client.1.smithi174.stdout:5/97: dwrite d1/ff [0,4194304] 0 2024-03-21T10:08:45.900 INFO:tasks.workunit.client.0.smithi153.stdout:7/208: rename d4/c6 to d4/d11/d24/c41 0 2024-03-21T10:08:45.900 INFO:tasks.workunit.client.0.smithi153.stdout:6/490: creat d1f/d2f/fb4 x:0 0 0 2024-03-21T10:08:45.905 INFO:tasks.workunit.client.0.smithi153.stdout:2/348: mkdir dd/d3b/d50/d58/d79 0 2024-03-21T10:08:45.907 INFO:tasks.workunit.client.1.smithi174.stdout:0/69: getdents d9 0 2024-03-21T10:08:45.907 INFO:tasks.workunit.client.1.smithi174.stdout:5/98: mknod d1/d16/c1e 0 2024-03-21T10:08:45.909 INFO:tasks.workunit.client.1.smithi174.stdout:3/62: rmdir d0/dc 0 2024-03-21T10:08:45.910 INFO:tasks.workunit.client.1.smithi174.stdout:5/99: fsync d1/f3 0 2024-03-21T10:08:45.910 INFO:tasks.workunit.client.1.smithi174.stdout:0/70: chown f4 12645 1 2024-03-21T10:08:45.911 INFO:tasks.workunit.client.0.smithi153.stdout:7/209: truncate d4/d11/d24/f28 308178 0 2024-03-21T10:08:45.912 INFO:tasks.workunit.client.1.smithi174.stdout:0/71: readlink l8 0 2024-03-21T10:08:45.913 INFO:tasks.workunit.client.1.smithi174.stdout:7/84: truncate d0/d4/f17 2957715 0 2024-03-21T10:08:45.917 INFO:tasks.workunit.client.1.smithi174.stdout:1/121: sync 2024-03-21T10:08:45.917 INFO:tasks.workunit.client.1.smithi174.stdout:0/72: write f4 [652927,3294] 0 2024-03-21T10:08:45.917 INFO:tasks.workunit.client.1.smithi174.stdout:4/150: rmdir d4/d21/d24/d2f 0 2024-03-21T10:08:45.920 INFO:tasks.workunit.client.0.smithi153.stdout:6/491: getdents d1f/d2f/d59 0 2024-03-21T10:08:45.921 INFO:tasks.workunit.client.1.smithi174.stdout:5/100: read d1/fc [573205,90949] 0 2024-03-21T10:08:45.922 INFO:tasks.workunit.client.0.smithi153.stdout:7/210: creat d4/d14/d16/f42 x:0 0 0 2024-03-21T10:08:45.924 INFO:tasks.workunit.client.1.smithi174.stdout:8/64: dwrite d9/fc [0,4194304] 0 2024-03-21T10:08:45.925 INFO:tasks.workunit.client.0.smithi153.stdout:5/295: dwrite d4/d5/de/d44/d48/f59 [0,4194304] 0 2024-03-21T10:08:45.925 INFO:tasks.workunit.client.1.smithi174.stdout:3/63: dread d0/fb [0,4194304] 0 2024-03-21T10:08:45.925 INFO:tasks.workunit.client.0.smithi153.stdout:6/492: stat d1f/d2f/d59/l87 0 2024-03-21T10:08:45.929 INFO:tasks.workunit.client.0.smithi153.stdout:3/310: dwrite dd/d1e/d2f/f3b [0,4194304] 0 2024-03-21T10:08:45.929 INFO:tasks.workunit.client.0.smithi153.stdout:6/493: chown f1a 209099331 1 2024-03-21T10:08:45.930 INFO:tasks.workunit.client.0.smithi153.stdout:7/211: dread - d4/d11/d24/f35 zero size 2024-03-21T10:08:45.930 INFO:tasks.workunit.client.1.smithi174.stdout:9/140: truncate d2/d11/f2c 215353 0 2024-03-21T10:08:45.931 INFO:tasks.workunit.client.0.smithi153.stdout:4/329: getdents d11/d2d/d3d 0 2024-03-21T10:08:45.933 INFO:tasks.workunit.client.1.smithi174.stdout:2/134: dread d5/d8/f29 [0,4194304] 0 2024-03-21T10:08:45.934 INFO:tasks.workunit.client.1.smithi174.stdout:1/122: creat d4/d5/f2a x:0 0 0 2024-03-21T10:08:45.934 INFO:tasks.workunit.client.1.smithi174.stdout:4/151: creat d4/d21/d24/d28/f39 x:0 0 0 2024-03-21T10:08:45.936 INFO:tasks.workunit.client.0.smithi153.stdout:5/296: creat d4/d8/d5a/f61 x:0 0 0 2024-03-21T10:08:45.937 INFO:tasks.workunit.client.1.smithi174.stdout:4/152: dread - d4/f5 zero size 2024-03-21T10:08:45.937 INFO:tasks.workunit.client.1.smithi174.stdout:4/153: write d4/f18 [4051205,44581] 0 2024-03-21T10:08:45.938 INFO:tasks.workunit.client.1.smithi174.stdout:4/154: write d4/fc [154835,20584] 0 2024-03-21T10:08:45.938 INFO:tasks.workunit.client.1.smithi174.stdout:5/101: mknod d1/d16/c1f 0 2024-03-21T10:08:45.939 INFO:tasks.workunit.client.0.smithi153.stdout:4/330: symlink d11/d2d/l66 0 2024-03-21T10:08:45.939 INFO:tasks.workunit.client.0.smithi153.stdout:4/331: fdatasync f1 0 2024-03-21T10:08:45.941 INFO:tasks.workunit.client.0.smithi153.stdout:3/311: readlink dd/d1e/d3d/l5c 0 2024-03-21T10:08:45.941 INFO:tasks.workunit.client.0.smithi153.stdout:6/494: write d1f/d2f/f32 [849226,102671] 0 2024-03-21T10:08:45.941 INFO:tasks.workunit.client.0.smithi153.stdout:2/349: dwrite dd/d15/d3c/f69 [0,4194304] 0 2024-03-21T10:08:45.942 INFO:tasks.workunit.client.1.smithi174.stdout:3/64: mkdir d0/d12/d14/d1a 0 2024-03-21T10:08:45.947 INFO:tasks.workunit.client.0.smithi153.stdout:3/312: fsync dd/f2b 0 2024-03-21T10:08:45.951 INFO:tasks.workunit.client.0.smithi153.stdout:4/332: creat d11/d2d/d3d/f67 x:0 0 0 2024-03-21T10:08:45.953 INFO:tasks.workunit.client.1.smithi174.stdout:5/102: mknod d1/d16/c20 0 2024-03-21T10:08:45.953 INFO:tasks.workunit.client.1.smithi174.stdout:4/155: mkdir d4/d21/d31/d3a 0 2024-03-21T10:08:45.954 INFO:tasks.workunit.client.0.smithi153.stdout:5/297: creat d4/d8/d3b/d56/f62 x:0 0 0 2024-03-21T10:08:45.955 INFO:tasks.workunit.client.1.smithi174.stdout:3/65: creat d0/d12/d14/d10/f1b x:0 0 0 2024-03-21T10:08:45.956 INFO:tasks.workunit.client.1.smithi174.stdout:0/73: link d9/cb d9/c12 0 2024-03-21T10:08:45.959 INFO:tasks.workunit.client.1.smithi174.stdout:2/135: symlink d5/l33 0 2024-03-21T10:08:45.960 INFO:tasks.workunit.client.1.smithi174.stdout:3/66: stat d0/d12/f17 0 2024-03-21T10:08:45.962 INFO:tasks.workunit.client.0.smithi153.stdout:3/313: mkdir dd/d1e/d6b 0 2024-03-21T10:08:45.962 INFO:tasks.workunit.client.0.smithi153.stdout:5/298: dread - d4/d8/d5a/d5d/f5b zero size 2024-03-21T10:08:45.963 INFO:tasks.workunit.client.1.smithi174.stdout:3/67: write d0/f13 [714619,77974] 0 2024-03-21T10:08:45.965 INFO:tasks.workunit.client.0.smithi153.stdout:5/299: chown d4/f46 1 1 2024-03-21T10:08:45.965 INFO:tasks.workunit.client.0.smithi153.stdout:6/495: link d1f/d2b/d33/d40/d51/f52 d1f/d2b/fb5 0 2024-03-21T10:08:45.966 INFO:tasks.workunit.client.0.smithi153.stdout:2/350: dread - dd/d15/d18/d3a/d3d/f74 zero size 2024-03-21T10:08:45.969 INFO:tasks.workunit.client.0.smithi153.stdout:4/333: rmdir d11/d5e 39 2024-03-21T10:08:45.969 INFO:tasks.workunit.client.1.smithi174.stdout:3/68: dread d0/fa [0,4194304] 0 2024-03-21T10:08:45.972 INFO:tasks.workunit.client.0.smithi153.stdout:6/496: mknod d1f/d2f/d59/cb6 0 2024-03-21T10:08:45.973 INFO:tasks.workunit.client.1.smithi174.stdout:3/69: truncate d0/fa 727352 0 2024-03-21T10:08:45.974 INFO:tasks.workunit.client.0.smithi153.stdout:2/351: creat dd/d15/d18/f7a x:0 0 0 2024-03-21T10:08:45.976 INFO:tasks.workunit.client.0.smithi153.stdout:3/314: creat dd/d1e/d6b/f6c x:0 0 0 2024-03-21T10:08:45.977 INFO:tasks.workunit.client.1.smithi174.stdout:3/70: fdatasync d0/d12/d14/d10/f16 0 2024-03-21T10:08:45.978 INFO:tasks.workunit.client.0.smithi153.stdout:2/352: stat dd/d15/d18/d2c/l32 0 2024-03-21T10:08:45.978 INFO:tasks.workunit.client.1.smithi174.stdout:5/103: link d1/ce d1/c21 0 2024-03-21T10:08:45.979 INFO:tasks.workunit.client.1.smithi174.stdout:0/74: dread f4 [0,4194304] 0 2024-03-21T10:08:45.982 INFO:tasks.workunit.client.1.smithi174.stdout:4/156: getdents d4/d19 0 2024-03-21T10:08:45.982 INFO:tasks.workunit.client.0.smithi153.stdout:4/334: creat d11/d18/d54/f68 x:0 0 0 2024-03-21T10:08:45.985 INFO:tasks.workunit.client.1.smithi174.stdout:0/75: chown d9/cf 255945183 1 2024-03-21T10:08:45.987 INFO:tasks.workunit.client.1.smithi174.stdout:0/76: chown f7 532360 1 2024-03-21T10:08:45.987 INFO:tasks.workunit.client.0.smithi153.stdout:2/353: write dd/d15/d18/d2c/d4b/f67 [54204,875] 0 2024-03-21T10:08:45.988 INFO:tasks.workunit.client.0.smithi153.stdout:5/300: dwrite d4/d8/d5a/d5d/f5b [0,4194304] 0 2024-03-21T10:08:45.992 INFO:tasks.workunit.client.0.smithi153.stdout:7/212: dwrite d4/f25 [0,4194304] 0 2024-03-21T10:08:45.993 INFO:tasks.workunit.client.0.smithi153.stdout:7/213: fdatasync d4/d11/d24/f35 0 2024-03-21T10:08:45.994 INFO:tasks.workunit.client.0.smithi153.stdout:2/354: chown dd/d15/d18/d2c/d33/f57 0 1 2024-03-21T10:08:45.994 INFO:tasks.workunit.client.1.smithi174.stdout:1/123: truncate d4/d7/f1b 1061312 0 2024-03-21T10:08:45.995 INFO:tasks.workunit.client.1.smithi174.stdout:2/136: dwrite d5/d8/fe [0,4194304] 0 2024-03-21T10:08:45.996 INFO:tasks.workunit.client.0.smithi153.stdout:4/335: creat d11/d2d/f69 x:0 0 0 2024-03-21T10:08:45.997 INFO:tasks.workunit.client.0.smithi153.stdout:3/315: link dd/c50 dd/d1e/d45/d56/c6d 0 2024-03-21T10:08:45.999 INFO:tasks.workunit.client.0.smithi153.stdout:2/355: truncate dd/d15/d18/d2c/f72 248670 0 2024-03-21T10:08:46.000 INFO:tasks.workunit.client.1.smithi174.stdout:4/157: link d4/d19/f22 d4/d21/d24/d28/d38/f3b 0 2024-03-21T10:08:46.002 INFO:tasks.workunit.client.0.smithi153.stdout:4/336: truncate d11/f63 16132 0 2024-03-21T10:08:46.002 INFO:tasks.workunit.client.0.smithi153.stdout:7/214: getdents d4/d11/d24 0 2024-03-21T10:08:46.004 INFO:tasks.workunit.client.1.smithi174.stdout:1/124: mknod d4/d7/d28/c2b 0 2024-03-21T10:08:46.004 INFO:tasks.workunit.client.0.smithi153.stdout:5/301: dread d4/d5/f1a [0,4194304] 0 2024-03-21T10:08:46.005 INFO:tasks.workunit.client.0.smithi153.stdout:4/337: write d11/d2d/d3d/f56 [144966,49995] 0 2024-03-21T10:08:46.005 INFO:tasks.workunit.client.0.smithi153.stdout:2/356: creat dd/d15/d18/d2c/d4b/d56/f7b x:0 0 0 2024-03-21T10:08:46.006 INFO:tasks.workunit.client.1.smithi174.stdout:9/141: dread d2/f20 [0,4194304] 0 2024-03-21T10:08:46.008 INFO:tasks.workunit.client.1.smithi174.stdout:0/77: dread d9/fc [0,4194304] 0 2024-03-21T10:08:46.009 INFO:tasks.workunit.client.0.smithi153.stdout:2/357: symlink dd/d3b/d50/l7c 0 2024-03-21T10:08:46.010 INFO:tasks.workunit.client.0.smithi153.stdout:4/338: truncate d11/f24 987069 0 2024-03-21T10:08:46.010 INFO:tasks.workunit.client.0.smithi153.stdout:2/358: write f5 [3419736,27863] 0 2024-03-21T10:08:46.011 INFO:tasks.workunit.client.1.smithi174.stdout:8/65: dwrite d9/fd [0,4194304] 0 2024-03-21T10:08:46.015 INFO:tasks.workunit.client.1.smithi174.stdout:1/125: write d4/d5/f2a [327042,75694] 0 2024-03-21T10:08:46.015 INFO:tasks.workunit.client.1.smithi174.stdout:9/142: chown d2/d11/d1d/d2a 126995699 1 2024-03-21T10:08:46.016 INFO:tasks.workunit.client.0.smithi153.stdout:4/339: link f10 d11/d18/d50/d65/f6a 0 2024-03-21T10:08:46.017 INFO:tasks.workunit.client.1.smithi174.stdout:8/66: mknod d9/c18 0 2024-03-21T10:08:46.018 INFO:tasks.workunit.client.0.smithi153.stdout:4/340: readlink d11/d2d/l52 0 2024-03-21T10:08:46.019 INFO:tasks.workunit.client.0.smithi153.stdout:2/359: creat dd/d15/d18/d2c/d33/f7d x:0 0 0 2024-03-21T10:08:46.019 INFO:tasks.workunit.client.1.smithi174.stdout:1/126: chown d4/d7/d28/c2b 442630 1 2024-03-21T10:08:46.021 INFO:tasks.workunit.client.0.smithi153.stdout:7/215: dread d4/f25 [0,4194304] 0 2024-03-21T10:08:46.022 INFO:tasks.workunit.client.1.smithi174.stdout:1/127: fsync d4/d7/f21 0 2024-03-21T10:08:46.023 INFO:tasks.workunit.client.1.smithi174.stdout:8/67: unlink d9/l11 0 2024-03-21T10:08:46.024 INFO:tasks.workunit.client.1.smithi174.stdout:1/128: creat d4/d7/f2c x:0 0 0 2024-03-21T10:08:46.025 INFO:tasks.workunit.client.0.smithi153.stdout:2/360: readlink dd/d15/d18/d2c/d4b/d56/l5f 0 2024-03-21T10:08:46.028 INFO:tasks.workunit.client.1.smithi174.stdout:6/60: sync 2024-03-21T10:08:46.030 INFO:tasks.workunit.client.0.smithi153.stdout:7/216: getdents d4/d14 0 2024-03-21T10:08:46.031 INFO:tasks.workunit.client.1.smithi174.stdout:1/129: mkdir d4/d2d 0 2024-03-21T10:08:46.032 INFO:tasks.workunit.client.0.smithi153.stdout:2/361: creat dd/d3b/d50/d58/d79/f7e x:0 0 0 2024-03-21T10:08:46.035 INFO:tasks.workunit.client.1.smithi174.stdout:6/61: dread - d0/d3/d5/d6/f13 zero size 2024-03-21T10:08:46.041 INFO:tasks.workunit.client.0.smithi153.stdout:4/341: dwrite d11/d18/f55 [0,4194304] 0 2024-03-21T10:08:46.041 INFO:tasks.workunit.client.0.smithi153.stdout:4/342: readlink d11/d2d/l66 0 2024-03-21T10:08:46.044 INFO:tasks.workunit.client.0.smithi153.stdout:4/343: unlink d11/d2d/l66 0 2024-03-21T10:08:46.045 INFO:tasks.workunit.client.0.smithi153.stdout:2/362: truncate dd/d15/d18/d2c/d33/f51 185264 0 2024-03-21T10:08:46.046 INFO:tasks.workunit.client.1.smithi174.stdout:1/130: chown d4/d7/d15/l25 78 1 2024-03-21T10:08:46.046 INFO:tasks.workunit.client.0.smithi153.stdout:2/363: stat dd/d15/d18/d2c/l47 0 2024-03-21T10:08:46.047 INFO:tasks.workunit.client.0.smithi153.stdout:4/344: readlink d11/l1c 0 2024-03-21T10:08:46.049 INFO:tasks.workunit.client.1.smithi174.stdout:0/78: dwrite f6 [4194304,4194304] 0 2024-03-21T10:08:46.051 INFO:tasks.workunit.client.0.smithi153.stdout:2/364: write dd/d15/d18/d3a/d3d/f4f [539624,13536] 0 2024-03-21T10:08:46.052 INFO:tasks.workunit.client.0.smithi153.stdout:4/345: mkdir d11/d2d/d6b 0 2024-03-21T10:08:46.052 INFO:tasks.workunit.client.1.smithi174.stdout:0/79: fdatasync f4 0 2024-03-21T10:08:46.053 INFO:tasks.workunit.client.1.smithi174.stdout:8/68: dwrite d9/fd [0,4194304] 0 2024-03-21T10:08:46.057 INFO:tasks.workunit.client.1.smithi174.stdout:8/69: mkdir d9/d17/d19 0 2024-03-21T10:08:46.058 INFO:tasks.workunit.client.0.smithi153.stdout:4/346: write d11/f16 [692929,66502] 0 2024-03-21T10:08:46.059 INFO:tasks.workunit.client.0.smithi153.stdout:4/347: chown ca 60111372 1 2024-03-21T10:08:46.061 INFO:tasks.workunit.client.0.smithi153.stdout:5/302: dread d4/d5/de/d38/f4d [0,4194304] 0 2024-03-21T10:08:46.062 INFO:tasks.workunit.client.1.smithi174.stdout:0/80: rename f6 to d9/f13 0 2024-03-21T10:08:46.062 INFO:tasks.workunit.client.1.smithi174.stdout:8/70: fdatasync f8 0 2024-03-21T10:08:46.069 INFO:tasks.workunit.client.1.smithi174.stdout:8/71: getdents d9/d17/d19 0 2024-03-21T10:08:46.070 INFO:tasks.workunit.client.0.smithi153.stdout:5/303: rename d4/d5/de/d44/d48/f59 to d4/d5/d16/f63 0 2024-03-21T10:08:46.070 INFO:tasks.workunit.client.1.smithi174.stdout:6/62: dwrite d0/d3/f8 [0,4194304] 0 2024-03-21T10:08:46.074 INFO:tasks.workunit.client.1.smithi174.stdout:8/72: dread d9/f12 [0,4194304] 0 2024-03-21T10:08:46.076 INFO:tasks.workunit.client.0.smithi153.stdout:7/217: dread d4/d11/d24/f29 [0,4194304] 0 2024-03-21T10:08:46.078 INFO:tasks.workunit.client.0.smithi153.stdout:2/365: dwrite dd/d15/d18/d3a/f55 [0,4194304] 0 2024-03-21T10:08:46.087 INFO:tasks.workunit.client.0.smithi153.stdout:7/218: mknod d4/d14/d2f/c43 0 2024-03-21T10:08:46.100 INFO:tasks.workunit.client.1.smithi174.stdout:6/63: dwrite d0/d3/d5/d6/f13 [0,4194304] 0 2024-03-21T10:08:46.101 INFO:tasks.workunit.client.1.smithi174.stdout:3/71: fsync d0/fa 0 2024-03-21T10:08:46.103 INFO:tasks.workunit.client.1.smithi174.stdout:2/137: write f2 [4355870,72421] 0 2024-03-21T10:08:46.104 INFO:tasks.workunit.client.1.smithi174.stdout:5/104: truncate d1/f6 2150901 0 2024-03-21T10:08:46.105 INFO:tasks.workunit.client.0.smithi153.stdout:7/219: unlink d4/d19/f32 0 2024-03-21T10:08:46.112 INFO:tasks.workunit.client.1.smithi174.stdout:3/72: mknod d0/d12/d14/d1a/c1c 0 2024-03-21T10:08:46.114 INFO:tasks.workunit.client.1.smithi174.stdout:6/64: creat d0/d3/d5/df/d16/f17 x:0 0 0 2024-03-21T10:08:46.115 INFO:tasks.workunit.client.1.smithi174.stdout:4/158: truncate d4/d19/f1f 2621938 0 2024-03-21T10:08:46.123 INFO:tasks.workunit.client.1.smithi174.stdout:0/81: truncate d9/f13 4783728 0 2024-03-21T10:08:46.125 INFO:tasks.workunit.client.1.smithi174.stdout:5/105: getdents d1/d16 0 2024-03-21T10:08:46.126 INFO:tasks.workunit.client.1.smithi174.stdout:6/65: write d0/d3/d5/f12 [1408035,56705] 0 2024-03-21T10:08:46.131 INFO:tasks.workunit.client.1.smithi174.stdout:3/73: dread d0/f13 [0,4194304] 0 2024-03-21T10:08:46.133 INFO:tasks.workunit.client.1.smithi174.stdout:2/138: dwrite f2 [4194304,4194304] 0 2024-03-21T10:08:46.136 INFO:tasks.workunit.client.0.smithi153.stdout:7/220: dwrite d4/d2a/f3e [0,4194304] 0 2024-03-21T10:08:46.136 INFO:tasks.workunit.client.1.smithi174.stdout:3/74: rename d0/d12 to d0/d12/d14/d1d 22 2024-03-21T10:08:46.136 INFO:tasks.workunit.client.1.smithi174.stdout:8/73: truncate f7 3024968 0 2024-03-21T10:08:46.140 INFO:tasks.workunit.client.0.smithi153.stdout:7/221: chown d4/d2a 0 1 2024-03-21T10:08:46.143 INFO:tasks.workunit.client.1.smithi174.stdout:6/66: creat d0/d3/d5/df/d16/f18 x:0 0 0 2024-03-21T10:08:46.147 INFO:tasks.workunit.client.1.smithi174.stdout:6/67: readlink d0/d3/d5/d6/l11 0 2024-03-21T10:08:46.147 INFO:tasks.workunit.client.1.smithi174.stdout:3/75: creat d0/d12/d14/d1a/f1e x:0 0 0 2024-03-21T10:08:46.148 INFO:tasks.workunit.client.0.smithi153.stdout:5/304: dwrite d4/d5/f1a [0,4194304] 0 2024-03-21T10:08:46.149 INFO:tasks.workunit.client.1.smithi174.stdout:6/68: write d0/d3/f8 [3896120,23351] 0 2024-03-21T10:08:46.149 INFO:tasks.workunit.client.1.smithi174.stdout:3/76: chown d0/d12/f17 5 1 2024-03-21T10:08:46.152 INFO:tasks.workunit.client.1.smithi174.stdout:6/69: dread - d0/d3/d5/df/d16/f18 zero size 2024-03-21T10:08:46.155 INFO:tasks.workunit.client.1.smithi174.stdout:2/139: mknod d5/c34 0 2024-03-21T10:08:46.157 INFO:tasks.workunit.client.0.smithi153.stdout:2/366: dwrite dd/d15/d18/f2a [0,4194304] 0 2024-03-21T10:08:46.157 INFO:tasks.workunit.client.0.smithi153.stdout:2/367: chown dd/d15/d18/d2c/d4b/d56/l65 465 1 2024-03-21T10:08:46.157 INFO:tasks.workunit.client.0.smithi153.stdout:7/222: creat d4/f44 x:0 0 0 2024-03-21T10:08:46.157 INFO:tasks.workunit.client.0.smithi153.stdout:7/223: readlink d4/lf 0 2024-03-21T10:08:46.163 INFO:tasks.workunit.client.1.smithi174.stdout:0/82: link d9/ca d9/c14 0 2024-03-21T10:08:46.163 INFO:tasks.workunit.client.1.smithi174.stdout:3/77: dread d0/f2 [0,4194304] 0 2024-03-21T10:08:46.169 INFO:tasks.workunit.client.1.smithi174.stdout:8/74: dwrite f5 [0,4194304] 0 2024-03-21T10:08:46.169 INFO:tasks.workunit.client.1.smithi174.stdout:7/85: sync 2024-03-21T10:08:46.170 INFO:tasks.workunit.client.1.smithi174.stdout:3/78: dread d0/f2 [0,4194304] 0 2024-03-21T10:08:46.171 INFO:tasks.workunit.client.0.smithi153.stdout:7/224: truncate d4/d11/d24/f29 3064750 0 2024-03-21T10:08:46.172 INFO:tasks.workunit.client.1.smithi174.stdout:6/70: unlink d0/d3/d5/df/d16/f18 0 2024-03-21T10:08:46.179 INFO:tasks.workunit.client.1.smithi174.stdout:3/79: dread - d0/d12/d14/d1a/f1e zero size 2024-03-21T10:08:46.179 INFO:tasks.workunit.client.1.smithi174.stdout:6/71: readlink d0/d3/l7 0 2024-03-21T10:08:46.182 INFO:tasks.workunit.client.0.smithi153.stdout:5/305: dwrite d4/d5/f57 [0,4194304] 0 2024-03-21T10:08:46.186 INFO:tasks.workunit.client.1.smithi174.stdout:6/72: readlink d0/d3/d5/df/l14 0 2024-03-21T10:08:46.189 INFO:tasks.workunit.client.1.smithi174.stdout:2/140: dwrite d5/d8/f22 [0,4194304] 0 2024-03-21T10:08:46.190 INFO:tasks.workunit.client.0.smithi153.stdout:2/368: dwrite dd/d15/d18/d3a/f3e [0,4194304] 0 2024-03-21T10:08:46.192 INFO:tasks.workunit.client.0.smithi153.stdout:7/225: rmdir d4 39 2024-03-21T10:08:46.200 INFO:tasks.workunit.client.0.smithi153.stdout:9/278: sync 2024-03-21T10:08:46.202 INFO:tasks.workunit.client.0.smithi153.stdout:5/306: link d4/d5/df/f30 d4/d5/de/d38/f64 0 2024-03-21T10:08:46.202 INFO:tasks.workunit.client.1.smithi174.stdout:8/75: write d9/f15 [463781,42923] 0 2024-03-21T10:08:46.204 INFO:tasks.workunit.client.0.smithi153.stdout:7/226: creat d4/d14/d2f/f45 x:0 0 0 2024-03-21T10:08:46.205 INFO:tasks.workunit.client.0.smithi153.stdout:9/279: chown d1/d43/d44/f34 2998767 1 2024-03-21T10:08:46.205 INFO:tasks.workunit.client.0.smithi153.stdout:2/369: getdents dd/d15/d3c 0 2024-03-21T10:08:46.210 INFO:tasks.workunit.client.0.smithi153.stdout:7/227: dread - d4/d11/d24/f35 zero size 2024-03-21T10:08:46.211 INFO:tasks.workunit.client.1.smithi174.stdout:7/86: mknod d0/d4/d10/d1b/c1f 0 2024-03-21T10:08:46.214 INFO:tasks.workunit.client.0.smithi153.stdout:5/307: mknod d4/d5/df/d5e/c65 0 2024-03-21T10:08:46.214 INFO:tasks.workunit.client.0.smithi153.stdout:5/308: chown d4/d8/d3b/f4b 1702458 1 2024-03-21T10:08:46.215 INFO:tasks.workunit.client.1.smithi174.stdout:3/80: fsync d0/f13 0 2024-03-21T10:08:46.216 INFO:tasks.workunit.client.0.smithi153.stdout:7/228: dread - d4/d14/d16/f42 zero size 2024-03-21T10:08:46.217 INFO:tasks.workunit.client.1.smithi174.stdout:3/81: chown d0/f7 199605 1 2024-03-21T10:08:46.221 INFO:tasks.workunit.client.1.smithi174.stdout:4/159: write d4/fe [633929,101940] 0 2024-03-21T10:08:46.222 INFO:tasks.workunit.client.0.smithi153.stdout:7/229: write d4/fa [2131231,55675] 0 2024-03-21T10:08:46.224 INFO:tasks.workunit.client.1.smithi174.stdout:6/73: dread d0/d3/f8 [0,4194304] 0 2024-03-21T10:08:46.224 INFO:tasks.workunit.client.1.smithi174.stdout:2/141: dwrite d5/d8/f1c [0,4194304] 0 2024-03-21T10:08:46.224 INFO:tasks.workunit.client.0.smithi153.stdout:7/230: chown d4/d1b/d20 1 1 2024-03-21T10:08:46.230 INFO:tasks.workunit.client.1.smithi174.stdout:9/143: sync 2024-03-21T10:08:46.233 INFO:tasks.workunit.client.0.smithi153.stdout:9/280: getdents d1/d43/d44/d46/d1c/d24 0 2024-03-21T10:08:46.233 INFO:tasks.workunit.client.1.smithi174.stdout:3/82: creat d0/d12/d14/d10/f1f x:0 0 0 2024-03-21T10:08:46.235 INFO:tasks.workunit.client.1.smithi174.stdout:3/83: write d0/fa [1724411,27593] 0 2024-03-21T10:08:46.236 INFO:tasks.workunit.client.0.smithi153.stdout:7/231: dread d4/d11/d24/f28 [0,4194304] 0 2024-03-21T10:08:46.238 INFO:tasks.workunit.client.0.smithi153.stdout:9/281: symlink d1/d4/l5a 0 2024-03-21T10:08:46.239 INFO:tasks.workunit.client.1.smithi174.stdout:2/142: rename d5/c7 to d5/d25/c35 0 2024-03-21T10:08:46.250 INFO:tasks.workunit.client.1.smithi174.stdout:9/144: chown d2/f1f 206776 1 2024-03-21T10:08:46.250 INFO:tasks.workunit.client.1.smithi174.stdout:7/87: dwrite d0/d4/d10/f1c [0,4194304] 0 2024-03-21T10:08:46.251 INFO:tasks.workunit.client.1.smithi174.stdout:7/88: write d0/d4/fd [424731,54021] 0 2024-03-21T10:08:46.253 INFO:tasks.workunit.client.1.smithi174.stdout:7/89: dread - d0/ff zero size 2024-03-21T10:08:46.257 INFO:tasks.workunit.client.1.smithi174.stdout:3/84: creat d0/d12/d14/f20 x:0 0 0 2024-03-21T10:08:46.260 INFO:tasks.workunit.client.1.smithi174.stdout:5/106: write d1/fc [844354,80246] 0 2024-03-21T10:08:46.266 INFO:tasks.workunit.client.0.smithi153.stdout:7/232: write d4/d11/d24/f3a [219310,130798] 0 2024-03-21T10:08:46.266 INFO:tasks.workunit.client.0.smithi153.stdout:7/233: truncate d4/f44 487696 0 2024-03-21T10:08:46.270 INFO:tasks.workunit.client.0.smithi153.stdout:7/234: mknod d4/d2a/d40/c46 0 2024-03-21T10:08:46.271 INFO:tasks.workunit.client.0.smithi153.stdout:7/235: chown d4/d14/d2f/f45 1589 1 2024-03-21T10:08:46.272 INFO:tasks.workunit.client.1.smithi174.stdout:1/131: sync 2024-03-21T10:08:46.272 INFO:tasks.workunit.client.0.smithi153.stdout:7/236: fdatasync d4/f44 0 2024-03-21T10:08:46.275 INFO:tasks.workunit.client.0.smithi153.stdout:5/309: dread d4/d8/d5a/d5d/f5f [0,4194304] 0 2024-03-21T10:08:46.277 INFO:tasks.workunit.client.1.smithi174.stdout:4/160: mknod d4/d21/d24/d28/d38/c3c 0 2024-03-21T10:08:46.278 INFO:tasks.workunit.client.0.smithi153.stdout:7/237: rename d4/cc to d4/d19/c47 0 2024-03-21T10:08:46.282 INFO:tasks.workunit.client.1.smithi174.stdout:6/74: link d0/d3/d5/d6/f13 d0/d3/d5/f19 0 2024-03-21T10:08:46.283 INFO:tasks.workunit.client.0.smithi153.stdout:7/238: creat d4/d11/f48 x:0 0 0 2024-03-21T10:08:46.283 INFO:tasks.workunit.client.0.smithi153.stdout:2/370: dread dd/f37 [0,4194304] 0 2024-03-21T10:08:46.287 INFO:tasks.workunit.client.0.smithi153.stdout:7/239: rename d4/d2a/c3c to d4/d2a/d40/c49 0 2024-03-21T10:08:46.288 INFO:tasks.workunit.client.0.smithi153.stdout:2/371: creat dd/d3b/d50/d59/d5d/f7f x:0 0 0 2024-03-21T10:08:46.289 INFO:tasks.workunit.client.0.smithi153.stdout:9/282: dread d1/d4/fc [0,4194304] 0 2024-03-21T10:08:46.291 INFO:tasks.workunit.client.0.smithi153.stdout:7/240: rename d4/c5 to d4/d1b/d20/c4a 0 2024-03-21T10:08:46.291 INFO:tasks.workunit.client.1.smithi174.stdout:7/90: symlink d0/d4/l20 0 2024-03-21T10:08:46.293 INFO:tasks.workunit.client.0.smithi153.stdout:9/283: chown d1/d43/d44/d46/d1c/d2e/l56 117768 1 2024-03-21T10:08:46.295 INFO:tasks.workunit.client.0.smithi153.stdout:7/241: creat d4/d14/f4b x:0 0 0 2024-03-21T10:08:46.299 INFO:tasks.workunit.client.1.smithi174.stdout:9/145: dread d2/f3 [0,4194304] 0 2024-03-21T10:08:46.300 INFO:tasks.workunit.client.0.smithi153.stdout:7/242: rename d4/f30 to d4/d2a/d40/f4c 0 2024-03-21T10:08:46.300 INFO:tasks.workunit.client.1.smithi174.stdout:5/107: rename d1/f13 to d1/f22 0 2024-03-21T10:08:46.301 INFO:tasks.workunit.client.1.smithi174.stdout:5/108: read d1/ff [3237707,96231] 0 2024-03-21T10:08:46.302 INFO:tasks.workunit.client.0.smithi153.stdout:7/243: readlink d4/l2b 0 2024-03-21T10:08:46.302 INFO:tasks.workunit.client.0.smithi153.stdout:7/244: chown d4/d1b 8632 1 2024-03-21T10:08:46.303 INFO:tasks.workunit.client.1.smithi174.stdout:1/132: creat d4/d7/f2e x:0 0 0 2024-03-21T10:08:46.304 INFO:tasks.workunit.client.0.smithi153.stdout:2/372: write dd/d15/d18/d3a/d3d/f53 [1902622,100616] 0 2024-03-21T10:08:46.305 INFO:tasks.workunit.client.0.smithi153.stdout:9/284: dread d1/d43/d44/d46/d1c/f3d [0,4194304] 0 2024-03-21T10:08:46.307 INFO:tasks.workunit.client.0.smithi153.stdout:2/373: rmdir dd/d3b/d50/d59 39 2024-03-21T10:08:46.309 INFO:tasks.workunit.client.0.smithi153.stdout:2/374: chown dd/d15/d3c/l6c 160 1 2024-03-21T10:08:46.309 INFO:tasks.workunit.client.1.smithi174.stdout:1/133: dread d4/d5/f18 [0,4194304] 0 2024-03-21T10:08:46.310 INFO:tasks.workunit.client.0.smithi153.stdout:9/285: creat d1/d43/d44/d46/d1c/d24/d41/f5b x:0 0 0 2024-03-21T10:08:46.311 INFO:tasks.workunit.client.1.smithi174.stdout:6/75: creat d0/d3/d5/df/d16/f1a x:0 0 0 2024-03-21T10:08:46.314 INFO:tasks.workunit.client.0.smithi153.stdout:9/286: write d1/d43/d44/f33 [8703146,13346] 0 2024-03-21T10:08:46.314 INFO:tasks.workunit.client.1.smithi174.stdout:7/91: dwrite d0/d4/f8 [0,4194304] 0 2024-03-21T10:08:46.315 INFO:tasks.workunit.client.1.smithi174.stdout:6/76: chown d0/d3 2274176 1 2024-03-21T10:08:46.318 INFO:tasks.workunit.client.0.smithi153.stdout:9/287: mkdir d1/d43/d44/d46/d4d/d5c 0 2024-03-21T10:08:46.320 INFO:tasks.workunit.client.1.smithi174.stdout:3/85: dwrite d0/f2 [0,4194304] 0 2024-03-21T10:08:46.322 INFO:tasks.workunit.client.0.smithi153.stdout:9/288: creat d1/d43/d44/f5d x:0 0 0 2024-03-21T10:08:46.323 INFO:tasks.workunit.client.0.smithi153.stdout:7/245: dwrite d4/d11/d24/f28 [0,4194304] 0 2024-03-21T10:08:46.325 INFO:tasks.workunit.client.0.smithi153.stdout:2/375: dwrite fb [0,4194304] 0 2024-03-21T10:08:46.327 INFO:tasks.workunit.client.0.smithi153.stdout:9/289: read d1/d43/d44/d46/d1c/d24/f38 [1873170,126901] 0 2024-03-21T10:08:46.329 INFO:tasks.workunit.client.0.smithi153.stdout:2/376: rmdir dd/d3b/d50/d58/d79 39 2024-03-21T10:08:46.332 INFO:tasks.workunit.client.1.smithi174.stdout:4/161: creat d4/d21/d31/d3a/f3d x:0 0 0 2024-03-21T10:08:46.332 INFO:tasks.workunit.client.0.smithi153.stdout:9/290: mknod d1/d3b/c5e 0 2024-03-21T10:08:46.333 INFO:tasks.workunit.client.0.smithi153.stdout:2/377: write dd/d3b/f4c [617334,68444] 0 2024-03-21T10:08:46.334 INFO:tasks.workunit.client.0.smithi153.stdout:9/291: chown d1/d4/la 250032 1 2024-03-21T10:08:46.336 INFO:tasks.workunit.client.1.smithi174.stdout:5/109: rename d1/f12 to d1/d16/d1c/f23 0 2024-03-21T10:08:46.338 INFO:tasks.workunit.client.0.smithi153.stdout:2/378: creat dd/d15/d18/d2c/d4b/d56/d63/f80 x:0 0 0 2024-03-21T10:08:46.341 INFO:tasks.workunit.client.1.smithi174.stdout:0/83: truncate f7 3695791 0 2024-03-21T10:08:46.342 INFO:tasks.workunit.client.0.smithi153.stdout:2/379: rmdir dd/d3b/d50/d59 39 2024-03-21T10:08:46.342 INFO:tasks.workunit.client.1.smithi174.stdout:0/84: readlink l1 0 2024-03-21T10:08:46.344 INFO:tasks.workunit.client.1.smithi174.stdout:2/143: rmdir d5 39 2024-03-21T10:08:46.347 INFO:tasks.workunit.client.1.smithi174.stdout:0/85: chown d9/le 685 1 2024-03-21T10:08:46.350 INFO:tasks.workunit.client.1.smithi174.stdout:3/86: chown d0/l4 5 1 2024-03-21T10:08:46.351 INFO:tasks.workunit.client.1.smithi174.stdout:8/76: truncate f5 3248852 0 2024-03-21T10:08:46.353 INFO:tasks.workunit.client.1.smithi174.stdout:9/146: dwrite d2/f1b [0,4194304] 0 2024-03-21T10:08:46.353 INFO:tasks.workunit.client.1.smithi174.stdout:3/87: stat d0/d12/d14/d1a/c1c 0 2024-03-21T10:08:46.354 INFO:tasks.workunit.client.1.smithi174.stdout:8/77: truncate d9/f16 977969 0 2024-03-21T10:08:46.364 INFO:tasks.workunit.client.1.smithi174.stdout:1/134: link d4/d7/f29 d4/d7/d15/f2f 0 2024-03-21T10:08:46.367 INFO:tasks.workunit.client.1.smithi174.stdout:5/110: creat d1/d16/f24 x:0 0 0 2024-03-21T10:08:46.370 INFO:tasks.workunit.client.1.smithi174.stdout:2/144: fsync d5/f12 0 2024-03-21T10:08:46.371 INFO:tasks.workunit.client.1.smithi174.stdout:2/145: stat d5/f26 0 2024-03-21T10:08:46.371 INFO:tasks.workunit.client.1.smithi174.stdout:7/92: dwrite d0/ff [0,4194304] 0 2024-03-21T10:08:46.375 INFO:tasks.workunit.client.1.smithi174.stdout:7/93: dread - d0/d4/d10/f19 zero size 2024-03-21T10:08:46.381 INFO:tasks.workunit.client.1.smithi174.stdout:9/147: readlink d2/l15 0 2024-03-21T10:08:46.381 INFO:tasks.workunit.client.1.smithi174.stdout:6/77: rename d0/d3/d5/f19 to d0/d3/d5/d6/f1b 0 2024-03-21T10:08:46.388 INFO:tasks.workunit.client.1.smithi174.stdout:1/135: creat d4/d7/d28/f30 x:0 0 0 2024-03-21T10:08:46.389 INFO:tasks.workunit.client.1.smithi174.stdout:5/111: creat d1/d16/d1c/f25 x:0 0 0 2024-03-21T10:08:46.393 INFO:tasks.workunit.client.1.smithi174.stdout:5/112: write d1/d16/f1a [985181,48319] 0 2024-03-21T10:08:46.401 INFO:tasks.workunit.client.1.smithi174.stdout:6/78: creat d0/d3/d5/f1c x:0 0 0 2024-03-21T10:08:46.404 INFO:tasks.workunit.client.1.smithi174.stdout:1/136: chown d4/d7/f1b 16916229 1 2024-03-21T10:08:46.404 INFO:tasks.workunit.client.1.smithi174.stdout:3/88: link d0/lf d0/d12/l21 0 2024-03-21T10:08:46.405 INFO:tasks.workunit.client.1.smithi174.stdout:2/146: dwrite d5/fa [0,4194304] 0 2024-03-21T10:08:46.405 INFO:tasks.workunit.client.1.smithi174.stdout:9/148: dwrite d2/f28 [0,4194304] 0 2024-03-21T10:08:46.408 INFO:tasks.workunit.client.1.smithi174.stdout:9/149: readlink d2/d11/l16 0 2024-03-21T10:08:46.409 INFO:tasks.workunit.client.1.smithi174.stdout:9/150: readlink d2/d11/l16 0 2024-03-21T10:08:46.412 INFO:tasks.workunit.client.1.smithi174.stdout:9/151: write d2/f1f [323581,31837] 0 2024-03-21T10:08:46.416 INFO:tasks.workunit.client.0.smithi153.stdout:8/308: sync 2024-03-21T10:08:46.416 INFO:tasks.workunit.client.0.smithi153.stdout:0/233: sync 2024-03-21T10:08:46.419 INFO:tasks.workunit.client.1.smithi174.stdout:6/79: rmdir d0/d3/d5/df/d16 39 2024-03-21T10:08:46.423 INFO:tasks.workunit.client.1.smithi174.stdout:6/80: dread - d0/d3/d5/f1c zero size 2024-03-21T10:08:46.424 INFO:tasks.workunit.client.0.smithi153.stdout:0/234: rmdir d4/d16/d1f 39 2024-03-21T10:08:46.424 INFO:tasks.workunit.client.1.smithi174.stdout:8/78: link f7 d9/f1a 0 2024-03-21T10:08:46.425 INFO:tasks.workunit.client.0.smithi153.stdout:2/380: creat dd/d15/d18/d2c/d4b/f81 x:0 0 0 2024-03-21T10:08:46.427 INFO:tasks.workunit.client.0.smithi153.stdout:8/309: creat d5/d9/d43/dd/d20/d26/d24/d52/d55/f60 x:0 0 0 2024-03-21T10:08:46.427 INFO:tasks.workunit.client.1.smithi174.stdout:7/94: creat d0/d4/d10/f21 x:0 0 0 2024-03-21T10:08:46.428 INFO:tasks.workunit.client.1.smithi174.stdout:9/152: creat d2/d24/f2f x:0 0 0 2024-03-21T10:08:46.431 INFO:tasks.workunit.client.0.smithi153.stdout:2/381: creat dd/d15/f82 x:0 0 0 2024-03-21T10:08:46.433 INFO:tasks.workunit.client.1.smithi174.stdout:1/137: creat d4/d7/d20/f31 x:0 0 0 2024-03-21T10:08:46.434 INFO:tasks.workunit.client.0.smithi153.stdout:8/310: symlink d5/d9/d43/dd/d20/d26/l61 0 2024-03-21T10:08:46.435 INFO:tasks.workunit.client.0.smithi153.stdout:2/382: creat dd/d15/d18/d2c/d4b/d56/d63/f83 x:0 0 0 2024-03-21T10:08:46.436 INFO:tasks.workunit.client.1.smithi174.stdout:3/89: link d0/f3 d0/d12/f22 0 2024-03-21T10:08:46.436 INFO:tasks.workunit.client.0.smithi153.stdout:8/311: chown d5/d9/l39 3 1 2024-03-21T10:08:46.439 INFO:tasks.workunit.client.1.smithi174.stdout:4/162: write d4/d19/f1f [1447170,36955] 0 2024-03-21T10:08:46.439 INFO:tasks.workunit.client.1.smithi174.stdout:3/90: truncate d0/f13 1633433 0 2024-03-21T10:08:46.439 INFO:tasks.workunit.client.1.smithi174.stdout:6/81: symlink d0/d3/d5/d6/l1d 0 2024-03-21T10:08:46.441 INFO:tasks.workunit.client.1.smithi174.stdout:0/86: dwrite f7 [0,4194304] 0 2024-03-21T10:08:46.442 INFO:tasks.workunit.client.1.smithi174.stdout:8/79: symlink d9/d17/l1b 0 2024-03-21T10:08:46.447 INFO:tasks.workunit.client.0.smithi153.stdout:8/312: rename d5/d9/d43/dd/d2d/f3f to d5/d3b/f62 0 2024-03-21T10:08:46.447 INFO:tasks.workunit.client.0.smithi153.stdout:0/235: dwrite d4/db/f13 [0,4194304] 0 2024-03-21T10:08:46.447 INFO:tasks.workunit.client.1.smithi174.stdout:5/113: link d1/ff d1/d16/d1c/f26 0 2024-03-21T10:08:46.448 INFO:tasks.workunit.client.1.smithi174.stdout:1/138: mkdir d4/d7/d32 0 2024-03-21T10:08:46.449 INFO:tasks.workunit.client.1.smithi174.stdout:2/147: rename d5/d8/d2f/l30 to d5/l36 0 2024-03-21T10:08:46.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:46 smithi174 ceph-mon[37496]: pgmap v9: 65 pgs: 65 active+clean; 924 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 10 MiB/s rd, 105 MiB/s wr, 237 op/s 2024-03-21T10:08:46.452 INFO:tasks.workunit.client.1.smithi174.stdout:4/163: creat d4/f3e x:0 0 0 2024-03-21T10:08:46.452 INFO:tasks.workunit.client.0.smithi153.stdout:8/313: fdatasync d5/d9/d43/f16 0 2024-03-21T10:08:46.454 INFO:tasks.workunit.client.1.smithi174.stdout:3/91: creat d0/d12/d14/d1a/f23 x:0 0 0 2024-03-21T10:08:46.456 INFO:tasks.workunit.client.1.smithi174.stdout:6/82: dread - d0/d3/d5/df/d16/f1a zero size 2024-03-21T10:08:46.458 INFO:tasks.workunit.client.0.smithi153.stdout:0/236: mknod d4/d16/d1f/d22/c4a 0 2024-03-21T10:08:46.462 INFO:tasks.workunit.client.0.smithi153.stdout:8/314: chown d5/d9/d43/dd/d20/d26/d24/d52/f54 1228462 1 2024-03-21T10:08:46.462 INFO:tasks.workunit.client.0.smithi153.stdout:2/383: dwrite dd/d3b/d50/f73 [0,4194304] 0 2024-03-21T10:08:46.462 INFO:tasks.workunit.client.1.smithi174.stdout:9/153: dwrite d2/f1f [0,4194304] 0 2024-03-21T10:08:46.463 INFO:tasks.workunit.client.1.smithi174.stdout:4/164: write d4/d21/d24/d28/f2b [866564,5743] 0 2024-03-21T10:08:46.463 INFO:tasks.workunit.client.0.smithi153.stdout:8/315: creat d5/d3b/f63 x:0 0 0 2024-03-21T10:08:46.464 INFO:tasks.workunit.client.0.smithi153.stdout:2/384: write dd/d15/d18/d3a/f3e [4023872,21055] 0 2024-03-21T10:08:46.464 INFO:tasks.workunit.client.1.smithi174.stdout:9/154: read d2/f20 [1982197,63190] 0 2024-03-21T10:08:46.469 INFO:tasks.workunit.client.0.smithi153.stdout:8/316: creat d5/d9/d43/dd/d20/d26/d24/d52/f64 x:0 0 0 2024-03-21T10:08:46.470 INFO:tasks.workunit.client.0.smithi153.stdout:8/317: readlink d5/d9/d43/l3e 0 2024-03-21T10:08:46.474 INFO:tasks.workunit.client.1.smithi174.stdout:2/148: symlink d5/d25/d27/d2e/l37 0 2024-03-21T10:08:46.474 INFO:tasks.workunit.client.0.smithi153.stdout:8/318: unlink d5/d9/d43/dd/d20/f57 0 2024-03-21T10:08:46.476 INFO:tasks.workunit.client.1.smithi174.stdout:1/139: dread d4/fb [0,4194304] 0 2024-03-21T10:08:46.476 INFO:tasks.workunit.client.1.smithi174.stdout:9/155: chown d2/c7 0 1 2024-03-21T10:08:46.478 INFO:tasks.workunit.client.1.smithi174.stdout:8/80: dread d9/fc [0,4194304] 0 2024-03-21T10:08:46.478 INFO:tasks.workunit.client.1.smithi174.stdout:9/156: chown d2 273193 1 2024-03-21T10:08:46.478 INFO:tasks.workunit.client.0.smithi153.stdout:8/319: write d5/d9/d43/dd/d20/d26/d24/d52/f54 [148386,119424] 0 2024-03-21T10:08:46.482 INFO:tasks.workunit.client.0.smithi153.stdout:8/320: symlink d5/d9/l65 0 2024-03-21T10:08:46.485 INFO:tasks.workunit.client.0.smithi153.stdout:2/385: dwrite dd/d15/d18/d2c/d33/f62 [0,4194304] 0 2024-03-21T10:08:46.488 INFO:tasks.workunit.client.0.smithi153.stdout:9/292: dread d1/f1b [0,4194304] 0 2024-03-21T10:08:46.488 INFO:tasks.workunit.client.1.smithi174.stdout:1/140: dread - d4/d7/d28/f30 zero size 2024-03-21T10:08:46.489 INFO:tasks.workunit.client.0.smithi153.stdout:2/386: read dd/d15/d18/d2c/f39 [122837,2974] 0 2024-03-21T10:08:46.489 INFO:tasks.workunit.client.0.smithi153.stdout:8/321: read d5/d9/d43/dd/d20/d26/d24/f28 [156549,80682] 0 2024-03-21T10:08:46.491 INFO:tasks.workunit.client.0.smithi153.stdout:0/237: truncate d4/d16/d1f/d22/f2e 6999764 0 2024-03-21T10:08:46.493 INFO:tasks.workunit.client.1.smithi174.stdout:0/87: dread f7 [0,4194304] 0 2024-03-21T10:08:46.495 INFO:tasks.workunit.client.0.smithi153.stdout:2/387: unlink f7 0 2024-03-21T10:08:46.496 INFO:tasks.workunit.client.1.smithi174.stdout:4/165: dwrite d4/d21/f27 [0,4194304] 0 2024-03-21T10:08:46.500 INFO:tasks.workunit.client.0.smithi153.stdout:9/293: write d1/d43/d44/d46/d1c/f3d [4223847,130813] 0 2024-03-21T10:08:46.501 INFO:tasks.workunit.client.1.smithi174.stdout:4/166: write d4/f20 [1306224,55744] 0 2024-03-21T10:08:46.502 INFO:tasks.workunit.client.1.smithi174.stdout:1/141: mkdir d4/d7/d33 0 2024-03-21T10:08:46.504 INFO:tasks.workunit.client.0.smithi153.stdout:2/388: mknod dd/d15/d18/d2c/d33/c84 0 2024-03-21T10:08:46.505 INFO:tasks.workunit.client.0.smithi153.stdout:1/293: sync 2024-03-21T10:08:46.506 INFO:tasks.workunit.client.0.smithi153.stdout:9/294: mkdir d1/d4/d5f 0 2024-03-21T10:08:46.507 INFO:tasks.workunit.client.1.smithi174.stdout:2/149: link d5/f10 d5/f38 0 2024-03-21T10:08:46.508 INFO:tasks.workunit.client.0.smithi153.stdout:1/294: stat d6/f9 0 2024-03-21T10:08:46.509 INFO:tasks.workunit.client.1.smithi174.stdout:0/88: creat d9/f15 x:0 0 0 2024-03-21T10:08:46.510 INFO:tasks.workunit.client.1.smithi174.stdout:2/150: write d5/d8/f28 [262367,111556] 0 2024-03-21T10:08:46.510 INFO:tasks.workunit.client.0.smithi153.stdout:8/322: dwrite d5/d9/d43/dd/d2d/f31 [0,4194304] 0 2024-03-21T10:08:46.511 INFO:tasks.workunit.client.0.smithi153.stdout:9/295: symlink d1/d4e/l60 0 2024-03-21T10:08:46.512 INFO:tasks.workunit.client.1.smithi174.stdout:8/81: dread f1 [0,4194304] 0 2024-03-21T10:08:46.513 INFO:tasks.workunit.client.0.smithi153.stdout:8/323: chown d5/f2f 7530172 1 2024-03-21T10:08:46.514 INFO:tasks.workunit.client.1.smithi174.stdout:7/95: truncate d0/ff 1732359 0 2024-03-21T10:08:46.514 INFO:tasks.workunit.client.1.smithi174.stdout:9/157: creat d2/d11/f30 x:0 0 0 2024-03-21T10:08:46.515 INFO:tasks.workunit.client.0.smithi153.stdout:0/238: dwrite d4/d16/d27/f38 [0,4194304] 0 2024-03-21T10:08:46.518 INFO:tasks.workunit.client.1.smithi174.stdout:4/167: unlink d4/l14 0 2024-03-21T10:08:46.521 INFO:tasks.workunit.client.1.smithi174.stdout:6/83: truncate d0/d3/d5/fe 3349566 0 2024-03-21T10:08:46.521 INFO:tasks.workunit.client.0.smithi153.stdout:2/389: rename dd/d15/f82 to dd/d15/d18/d2c/d4b/d56/f85 0 2024-03-21T10:08:46.521 INFO:tasks.workunit.client.0.smithi153.stdout:1/295: truncate f5 1831314 0 2024-03-21T10:08:46.526 INFO:tasks.workunit.client.1.smithi174.stdout:0/89: write d9/f13 [5665668,77373] 0 2024-03-21T10:08:46.527 INFO:tasks.workunit.client.1.smithi174.stdout:2/151: rename d5/d8/fe to d5/d25/d27/f39 0 2024-03-21T10:08:46.530 INFO:tasks.workunit.client.0.smithi153.stdout:9/296: mkdir d1/d43/d44/d46/d1c/d61 0 2024-03-21T10:08:46.532 INFO:tasks.workunit.client.0.smithi153.stdout:2/390: mknod dd/d3b/d50/c86 0 2024-03-21T10:08:46.534 INFO:tasks.workunit.client.1.smithi174.stdout:9/158: stat d2/f17 0 2024-03-21T10:08:46.719 INFO:tasks.workunit.client.1.smithi174.stdout:7/96: readlink d0/d4/l15 0 2024-03-21T10:08:46.720 INFO:tasks.workunit.client.0.smithi153.stdout:0/239: rename d4/l8 to d4/d16/l4b 0 2024-03-21T10:08:46.720 INFO:tasks.workunit.client.0.smithi153.stdout:1/296: creat d6/d13/f5e x:0 0 0 2024-03-21T10:08:46.721 INFO:tasks.workunit.client.0.smithi153.stdout:1/297: readlink d6/d10/l1a 0 2024-03-21T10:08:46.721 INFO:tasks.workunit.client.0.smithi153.stdout:0/240: chown d4/d16/d1f/l2a 1324 1 2024-03-21T10:08:46.721 INFO:tasks.workunit.client.0.smithi153.stdout:8/324: dwrite d5/d9/d43/dd/d20/d26/d24/d52/d55/f5c [0,4194304] 0 2024-03-21T10:08:46.721 INFO:tasks.workunit.client.0.smithi153.stdout:8/325: readlink d5/d9/d43/l3e 0 2024-03-21T10:08:46.722 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:46 smithi153 ceph-mon[29117]: pgmap v9: 65 pgs: 65 active+clean; 924 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 10 MiB/s rd, 105 MiB/s wr, 237 op/s 2024-03-21T10:08:46.731 INFO:tasks.workunit.client.1.smithi174.stdout:1/142: creat d4/d2d/f34 x:0 0 0 2024-03-21T10:08:46.731 INFO:tasks.workunit.client.1.smithi174.stdout:7/97: stat d0/d4/d10/f21 0 2024-03-21T10:08:46.731 INFO:tasks.workunit.client.1.smithi174.stdout:0/90: mknod d9/c16 0 2024-03-21T10:08:46.731 INFO:tasks.workunit.client.1.smithi174.stdout:4/168: rename d4/d21/d24/d28/d38/c3c to d4/d21/d31/d3a/c3f 0 2024-03-21T10:08:46.731 INFO:tasks.workunit.client.1.smithi174.stdout:1/143: chown d4/d5/fd 50596 1 2024-03-21T10:08:46.732 INFO:tasks.workunit.client.1.smithi174.stdout:4/169: write d4/fe [708639,127722] 0 2024-03-21T10:08:46.732 INFO:tasks.workunit.client.1.smithi174.stdout:6/84: symlink d0/d3/d5/df/l1e 0 2024-03-21T10:08:46.732 INFO:tasks.workunit.client.1.smithi174.stdout:7/98: write d0/d4/d10/f19 [385664,25909] 0 2024-03-21T10:08:46.732 INFO:tasks.workunit.client.1.smithi174.stdout:4/170: dread - d4/d19/f1c zero size 2024-03-21T10:08:46.732 INFO:tasks.workunit.client.1.smithi174.stdout:8/82: link f8 d9/f1c 0 2024-03-21T10:08:46.732 INFO:tasks.workunit.client.1.smithi174.stdout:5/114: dread d1/f3 [0,4194304] 0 2024-03-21T10:08:46.732 INFO:tasks.workunit.client.1.smithi174.stdout:0/91: creat d9/f17 x:0 0 0 2024-03-21T10:08:46.732 INFO:tasks.workunit.client.1.smithi174.stdout:8/83: symlink d9/l1d 0 2024-03-21T10:08:46.732 INFO:tasks.workunit.client.1.smithi174.stdout:1/144: fsync d4/d2d/f34 0 2024-03-21T10:08:46.732 INFO:tasks.workunit.client.1.smithi174.stdout:1/145: stat d4/d7/f2e 0 2024-03-21T10:08:46.732 INFO:tasks.workunit.client.1.smithi174.stdout:2/152: dwrite d5/f26 [0,4194304] 0 2024-03-21T10:08:46.732 INFO:tasks.workunit.client.1.smithi174.stdout:0/92: creat d9/f18 x:0 0 0 2024-03-21T10:08:46.732 INFO:tasks.workunit.client.1.smithi174.stdout:1/146: stat d4/l27 0 2024-03-21T10:08:46.732 INFO:tasks.workunit.client.1.smithi174.stdout:8/84: dread d9/f16 [0,4194304] 0 2024-03-21T10:08:46.733 INFO:tasks.workunit.client.1.smithi174.stdout:6/85: mknod d0/d3/d5/df/d16/c1f 0 2024-03-21T10:08:46.733 INFO:tasks.workunit.client.1.smithi174.stdout:2/153: fsync f2 0 2024-03-21T10:08:46.733 INFO:tasks.workunit.client.1.smithi174.stdout:1/147: symlink d4/d7/l35 0 2024-03-21T10:08:46.733 INFO:tasks.workunit.client.1.smithi174.stdout:5/115: fsync d1/d16/d1c/f26 0 2024-03-21T10:08:46.733 INFO:tasks.workunit.client.1.smithi174.stdout:9/159: write d2/d11/f2c [628264,56588] 0 2024-03-21T10:08:46.733 INFO:tasks.workunit.client.1.smithi174.stdout:6/86: mkdir d0/d3/d20 0 2024-03-21T10:08:46.733 INFO:tasks.workunit.client.1.smithi174.stdout:5/116: creat d1/d16/f27 x:0 0 0 2024-03-21T10:08:46.733 INFO:tasks.workunit.client.1.smithi174.stdout:1/148: rename d4/d7/d20/f31 to d4/d7/d33/f36 0 2024-03-21T10:08:46.734 INFO:tasks.workunit.client.1.smithi174.stdout:6/87: mkdir d0/d3/d5/d21 0 2024-03-21T10:08:46.734 INFO:tasks.workunit.client.1.smithi174.stdout:9/160: dwrite d2/f1b [0,4194304] 0 2024-03-21T10:08:46.734 INFO:tasks.workunit.client.1.smithi174.stdout:4/171: truncate d4/f7 327103 0 2024-03-21T10:08:46.734 INFO:tasks.workunit.client.1.smithi174.stdout:6/88: chown d0/d3/d5/df/d16/f1a 37 1 2024-03-21T10:08:46.734 INFO:tasks.workunit.client.1.smithi174.stdout:1/149: symlink d4/d2d/l37 0 2024-03-21T10:08:46.734 INFO:tasks.workunit.client.1.smithi174.stdout:5/117: unlink d1/d16/d1c/f1d 0 2024-03-21T10:08:46.734 INFO:tasks.workunit.client.1.smithi174.stdout:7/99: dread d0/ff [0,4194304] 0 2024-03-21T10:08:46.734 INFO:tasks.workunit.client.1.smithi174.stdout:1/150: write d4/d7/f2e [952577,31022] 0 2024-03-21T10:08:46.734 INFO:tasks.workunit.client.1.smithi174.stdout:0/93: getdents d9 0 2024-03-21T10:08:46.734 INFO:tasks.workunit.client.1.smithi174.stdout:1/151: dread - d4/d7/f29 zero size 2024-03-21T10:08:46.734 INFO:tasks.workunit.client.1.smithi174.stdout:7/100: creat d0/d4/f22 x:0 0 0 2024-03-21T10:08:46.734 INFO:tasks.workunit.client.1.smithi174.stdout:9/161: link d2/d11/f30 d2/d24/d2e/f31 0 2024-03-21T10:08:46.734 INFO:tasks.workunit.client.1.smithi174.stdout:6/89: unlink d0/d3/d5/d6/f1b 0 2024-03-21T10:08:46.735 INFO:tasks.workunit.client.1.smithi174.stdout:6/90: chown d0/d3/d5/f1c 4580498 1 2024-03-21T10:08:46.735 INFO:tasks.workunit.client.1.smithi174.stdout:2/154: dread d5/f11 [0,4194304] 0 2024-03-21T10:08:46.735 INFO:tasks.workunit.client.1.smithi174.stdout:7/101: chown d0/d4/f17 952452 1 2024-03-21T10:08:46.735 INFO:tasks.workunit.client.1.smithi174.stdout:7/102: readlink d0/d4/l20 0 2024-03-21T10:08:46.735 INFO:tasks.workunit.client.1.smithi174.stdout:4/172: getdents d4 0 2024-03-21T10:08:46.735 INFO:tasks.workunit.client.1.smithi174.stdout:4/173: readlink d4/l2a 0 2024-03-21T10:08:46.735 INFO:tasks.workunit.client.1.smithi174.stdout:4/174: write d4/d21/d24/d28/f39 [764422,103951] 0 2024-03-21T10:08:46.735 INFO:tasks.workunit.client.1.smithi174.stdout:9/162: dwrite d2/fc [0,4194304] 0 2024-03-21T10:08:46.735 INFO:tasks.workunit.client.1.smithi174.stdout:7/103: unlink d0/d4/c14 0 2024-03-21T10:08:46.735 INFO:tasks.workunit.client.1.smithi174.stdout:1/152: getdents d4/d7/d33 0 2024-03-21T10:08:46.735 INFO:tasks.workunit.client.1.smithi174.stdout:4/175: mknod d4/d21/c40 0 2024-03-21T10:08:46.735 INFO:tasks.workunit.client.1.smithi174.stdout:6/91: dwrite d0/d3/d5/df/d16/f17 [0,4194304] 0 2024-03-21T10:08:46.735 INFO:tasks.workunit.client.1.smithi174.stdout:9/163: dread d2/f18 [0,4194304] 0 2024-03-21T10:08:46.735 INFO:tasks.workunit.client.1.smithi174.stdout:9/164: read d2/f1f [3532776,51800] 0 2024-03-21T10:08:46.735 INFO:tasks.workunit.client.1.smithi174.stdout:2/155: dread d5/d8/f22 [0,4194304] 0 2024-03-21T10:08:46.736 INFO:tasks.workunit.client.1.smithi174.stdout:1/153: dread d4/d5/f18 [0,4194304] 0 2024-03-21T10:08:46.736 INFO:tasks.workunit.client.1.smithi174.stdout:2/156: chown d5/l1f 1 1 2024-03-21T10:08:46.736 INFO:tasks.workunit.client.1.smithi174.stdout:5/118: rmdir d1/d16 39 2024-03-21T10:08:46.736 INFO:tasks.workunit.client.1.smithi174.stdout:4/176: creat d4/d19/f41 x:0 0 0 2024-03-21T10:08:46.736 INFO:tasks.workunit.client.1.smithi174.stdout:1/154: write d4/d7/f29 [569313,21501] 0 2024-03-21T10:08:46.736 INFO:tasks.workunit.client.1.smithi174.stdout:0/94: write d9/fc [1525461,20120] 0 2024-03-21T10:08:46.736 INFO:tasks.workunit.client.1.smithi174.stdout:4/177: creat d4/d21/d24/f42 x:0 0 0 2024-03-21T10:08:46.736 INFO:tasks.workunit.client.1.smithi174.stdout:2/157: link d5/d8/f22 d5/d25/d27/d2e/f3a 0 2024-03-21T10:08:46.736 INFO:tasks.workunit.client.1.smithi174.stdout:7/104: getdents d0/d4 0 2024-03-21T10:08:46.738 INFO:tasks.workunit.client.1.smithi174.stdout:2/158: write d5/f12 [3684325,119167] 0 2024-03-21T10:08:46.738 INFO:tasks.workunit.client.1.smithi174.stdout:0/95: creat d9/f19 x:0 0 0 2024-03-21T10:08:46.738 INFO:tasks.workunit.client.1.smithi174.stdout:5/119: mknod d1/d16/c28 0 2024-03-21T10:08:46.740 INFO:tasks.workunit.client.0.smithi153.stdout:6/497: sync 2024-03-21T10:08:46.740 INFO:tasks.workunit.client.0.smithi153.stdout:3/316: sync 2024-03-21T10:08:46.740 INFO:tasks.workunit.client.0.smithi153.stdout:8/326: dread - d5/d9/d43/dd/d2d/d45/f4a zero size 2024-03-21T10:08:46.740 INFO:tasks.workunit.client.0.smithi153.stdout:5/310: sync 2024-03-21T10:08:46.740 INFO:tasks.workunit.client.0.smithi153.stdout:7/246: sync 2024-03-21T10:08:46.740 INFO:tasks.workunit.client.0.smithi153.stdout:4/348: sync 2024-03-21T10:08:46.740 INFO:tasks.workunit.client.0.smithi153.stdout:2/391: dread dd/d15/d18/f2a [0,4194304] 0 2024-03-21T10:08:46.740 INFO:tasks.workunit.client.0.smithi153.stdout:8/327: fdatasync d5/d1b/f5e 0 2024-03-21T10:08:46.740 INFO:tasks.workunit.client.0.smithi153.stdout:1/298: link d6/f15 d6/d3a/d3c/d50/d51/f5f 0 2024-03-21T10:08:46.740 INFO:tasks.workunit.client.0.smithi153.stdout:5/311: chown d4/c42 354831 1 2024-03-21T10:08:46.740 INFO:tasks.workunit.client.0.smithi153.stdout:7/247: chown d4/d1b/d20/c27 248 1 2024-03-21T10:08:46.741 INFO:tasks.workunit.client.0.smithi153.stdout:6/498: read d1f/d2f/f7d [1128180,62363] 0 2024-03-21T10:08:46.741 INFO:tasks.workunit.client.0.smithi153.stdout:9/297: dwrite d1/d43/d44/f5d [0,4194304] 0 2024-03-21T10:08:46.741 INFO:tasks.workunit.client.0.smithi153.stdout:9/298: fdatasync d1/f27 0 2024-03-21T10:08:46.741 INFO:tasks.workunit.client.0.smithi153.stdout:7/248: write d4/d11/d24/f35 [21428,106034] 0 2024-03-21T10:08:46.741 INFO:tasks.workunit.client.0.smithi153.stdout:5/312: creat d4/d8/d3b/f66 x:0 0 0 2024-03-21T10:08:46.741 INFO:tasks.workunit.client.0.smithi153.stdout:2/392: symlink dd/d15/d18/d3a/l87 0 2024-03-21T10:08:46.741 INFO:tasks.workunit.client.0.smithi153.stdout:0/241: dwrite d4/d16/d1f/d22/f47 [0,4194304] 0 2024-03-21T10:08:46.741 INFO:tasks.workunit.client.0.smithi153.stdout:3/317: dwrite f8 [0,4194304] 0 2024-03-21T10:08:46.741 INFO:tasks.workunit.client.0.smithi153.stdout:7/249: unlink d4/f12 0 2024-03-21T10:08:46.741 INFO:tasks.workunit.client.0.smithi153.stdout:3/318: write dd/f43 [689325,56008] 0 2024-03-21T10:08:46.741 INFO:tasks.workunit.client.0.smithi153.stdout:9/299: creat d1/d43/d44/d46/d1c/d2e/f62 x:0 0 0 2024-03-21T10:08:46.741 INFO:tasks.workunit.client.0.smithi153.stdout:3/319: readlink dd/d1e/l4b 0 2024-03-21T10:08:46.741 INFO:tasks.workunit.client.0.smithi153.stdout:5/313: mkdir d4/d5/df/d67 0 2024-03-21T10:08:46.741 INFO:tasks.workunit.client.0.smithi153.stdout:5/314: read f1 [2502188,25897] 0 2024-03-21T10:08:46.741 INFO:tasks.workunit.client.0.smithi153.stdout:1/299: rename d6/d13/f1f to d6/f60 0 2024-03-21T10:08:46.742 INFO:tasks.workunit.client.0.smithi153.stdout:9/300: symlink d1/d4/l63 0 2024-03-21T10:08:46.742 INFO:tasks.workunit.client.0.smithi153.stdout:5/315: rmdir d4/d5/de 39 2024-03-21T10:08:46.742 INFO:tasks.workunit.client.0.smithi153.stdout:3/320: creat dd/d41/f6e x:0 0 0 2024-03-21T10:08:46.742 INFO:tasks.workunit.client.0.smithi153.stdout:3/321: chown lb 19702 1 2024-03-21T10:08:46.742 INFO:tasks.workunit.client.0.smithi153.stdout:1/300: mknod d6/d32/d3e/c61 0 2024-03-21T10:08:46.742 INFO:tasks.workunit.client.0.smithi153.stdout:9/301: dread d1/d4/fc [0,4194304] 0 2024-03-21T10:08:46.742 INFO:tasks.workunit.client.0.smithi153.stdout:3/322: chown lb 3 1 2024-03-21T10:08:46.742 INFO:tasks.workunit.client.0.smithi153.stdout:2/393: dread dd/d15/d18/d3a/f55 [0,4194304] 0 2024-03-21T10:08:46.742 INFO:tasks.workunit.client.0.smithi153.stdout:0/242: getdents d4/d16/d19 0 2024-03-21T10:08:46.743 INFO:tasks.workunit.client.0.smithi153.stdout:6/499: dwrite d1f/f36 [0,4194304] 0 2024-03-21T10:08:46.743 INFO:tasks.workunit.client.0.smithi153.stdout:0/243: stat d4/db/c42 0 2024-03-21T10:08:46.743 INFO:tasks.workunit.client.0.smithi153.stdout:7/250: dread d4/d11/d24/f29 [0,4194304] 0 2024-03-21T10:08:46.743 INFO:tasks.workunit.client.0.smithi153.stdout:0/244: write f2 [2752313,37066] 0 2024-03-21T10:08:46.743 INFO:tasks.workunit.client.0.smithi153.stdout:3/323: mknod dd/d1e/d6b/c6f 0 2024-03-21T10:08:46.743 INFO:tasks.workunit.client.0.smithi153.stdout:9/302: symlink d1/d43/l64 0 2024-03-21T10:08:46.743 INFO:tasks.workunit.client.0.smithi153.stdout:7/251: mkdir d4/d19/d4d 0 2024-03-21T10:08:46.743 INFO:tasks.workunit.client.0.smithi153.stdout:6/500: dread - d1f/d2b/d33/d40/f9e zero size 2024-03-21T10:08:46.743 INFO:tasks.workunit.client.0.smithi153.stdout:4/349: dread d11/d18/d50/d65/f6a [0,4194304] 0 2024-03-21T10:08:46.743 INFO:tasks.workunit.client.0.smithi153.stdout:3/324: symlink dd/d41/l70 0 2024-03-21T10:08:46.743 INFO:tasks.workunit.client.0.smithi153.stdout:0/245: mkdir d4/d4c 0 2024-03-21T10:08:46.743 INFO:tasks.workunit.client.0.smithi153.stdout:6/501: creat d1f/d99/fb7 x:0 0 0 2024-03-21T10:08:46.743 INFO:tasks.workunit.client.0.smithi153.stdout:8/328: dwrite d5/d9/d43/dd/d20/d26/d24/f32 [4194304,4194304] 0 2024-03-21T10:08:46.743 INFO:tasks.workunit.client.0.smithi153.stdout:6/502: stat d1f/d2b/f46 0 2024-03-21T10:08:46.743 INFO:tasks.workunit.client.0.smithi153.stdout:5/316: dwrite d4/d8/f2d [0,4194304] 0 2024-03-21T10:08:46.744 INFO:tasks.workunit.client.0.smithi153.stdout:6/503: dread - d1f/d2b/d33/d40/d4f/f83 zero size 2024-03-21T10:08:46.744 INFO:tasks.workunit.client.0.smithi153.stdout:9/303: link d1/d4/c57 d1/d43/d44/d46/d1c/d24/d41/c65 0 2024-03-21T10:08:46.744 INFO:tasks.workunit.client.0.smithi153.stdout:9/304: chown d1/d3b/c5e 353957 1 2024-03-21T10:08:46.744 INFO:tasks.workunit.client.0.smithi153.stdout:8/329: rename d5/d9/d43/dd/d20/d26/d24/f5a to d5/d9/d43/dd/d2d/f66 0 2024-03-21T10:08:46.744 INFO:tasks.workunit.client.0.smithi153.stdout:7/252: dread d4/d2a/f3e [0,4194304] 0 2024-03-21T10:08:46.744 INFO:tasks.workunit.client.0.smithi153.stdout:5/317: creat d4/d8/f68 x:0 0 0 2024-03-21T10:08:46.744 INFO:tasks.workunit.client.0.smithi153.stdout:9/305: chown d1/c8 519593790 1 2024-03-21T10:08:46.744 INFO:tasks.workunit.client.0.smithi153.stdout:0/246: getdents d4/d16/d19 0 2024-03-21T10:08:46.744 INFO:tasks.workunit.client.0.smithi153.stdout:4/350: link d11/c27 d11/c6c 0 2024-03-21T10:08:46.744 INFO:tasks.workunit.client.0.smithi153.stdout:4/351: fdatasync d11/f63 0 2024-03-21T10:08:46.744 INFO:tasks.workunit.client.0.smithi153.stdout:5/318: truncate d4/d8/d5a/d5d/f55 122598 0 2024-03-21T10:08:46.744 INFO:tasks.workunit.client.0.smithi153.stdout:6/504: unlink d1f/f35 0 2024-03-21T10:08:46.744 INFO:tasks.workunit.client.0.smithi153.stdout:0/247: creat d4/d16/d27/f4d x:0 0 0 2024-03-21T10:08:46.744 INFO:tasks.workunit.client.0.smithi153.stdout:6/505: chown d1f/d2b/d33/d40/d4f/l86 3054246 1 2024-03-21T10:08:46.745 INFO:tasks.workunit.client.0.smithi153.stdout:9/306: creat d1/d3b/f66 x:0 0 0 2024-03-21T10:08:46.745 INFO:tasks.workunit.client.0.smithi153.stdout:5/319: dread - d4/d5/de/f3f zero size 2024-03-21T10:08:46.745 INFO:tasks.workunit.client.0.smithi153.stdout:8/330: link d5/d9/d43/f35 d5/d1b/d49/f67 0 2024-03-21T10:08:46.745 INFO:tasks.workunit.client.0.smithi153.stdout:1/301: dread d6/fb [0,4194304] 0 2024-03-21T10:08:46.745 INFO:tasks.workunit.client.0.smithi153.stdout:6/506: read - d1f/d2b/d33/d40/fa5 zero size 2024-03-21T10:08:46.745 INFO:tasks.workunit.client.0.smithi153.stdout:0/248: creat d4/d4c/f4e x:0 0 0 2024-03-21T10:08:46.745 INFO:tasks.workunit.client.0.smithi153.stdout:1/302: write d6/d13/f3f [2142520,48246] 0 2024-03-21T10:08:46.745 INFO:tasks.workunit.client.0.smithi153.stdout:5/320: chown d4/d8/f26 4 1 2024-03-21T10:08:46.745 INFO:tasks.workunit.client.0.smithi153.stdout:8/331: symlink d5/d1b/d49/l68 0 2024-03-21T10:08:46.745 INFO:tasks.workunit.client.0.smithi153.stdout:5/321: readlink d4/d5/l31 0 2024-03-21T10:08:46.745 INFO:tasks.workunit.client.0.smithi153.stdout:8/332: fdatasync d5/d9/f4b 0 2024-03-21T10:08:46.745 INFO:tasks.workunit.client.0.smithi153.stdout:0/249: rmdir d4/d16/d19/d30 39 2024-03-21T10:08:46.745 INFO:tasks.workunit.client.0.smithi153.stdout:3/325: dread dd/f20 [0,4194304] 0 2024-03-21T10:08:46.745 INFO:tasks.workunit.client.0.smithi153.stdout:5/322: mkdir d4/d5/df/d5e/d69 0 2024-03-21T10:08:46.745 INFO:tasks.workunit.client.0.smithi153.stdout:4/352: dwrite d11/d18/d54/f68 [0,4194304] 0 2024-03-21T10:08:46.745 INFO:tasks.workunit.client.0.smithi153.stdout:8/333: mkdir d5/d9/d43/dd/d20/d26/d47/d69 0 2024-03-21T10:08:46.746 INFO:tasks.workunit.client.0.smithi153.stdout:3/326: fsync dd/d1e/d2f/f37 0 2024-03-21T10:08:46.746 INFO:tasks.workunit.client.0.smithi153.stdout:6/507: dwrite d1f/f28 [0,4194304] 0 2024-03-21T10:08:46.746 INFO:tasks.workunit.client.0.smithi153.stdout:5/323: truncate d4/d8/d3b/d56/f62 265559 0 2024-03-21T10:08:46.746 INFO:tasks.workunit.client.0.smithi153.stdout:3/327: truncate dd/f1c 2267237 0 2024-03-21T10:08:46.746 INFO:tasks.workunit.client.0.smithi153.stdout:5/324: write d4/d8/d5a/d5d/f55 [1022142,124859] 0 2024-03-21T10:08:46.746 INFO:tasks.workunit.client.0.smithi153.stdout:3/328: dread - dd/d1e/d2f/d35/f5a zero size 2024-03-21T10:08:46.747 INFO:tasks.workunit.client.0.smithi153.stdout:6/508: dread - d1f/d2b/d33/d40/d51/f55 zero size 2024-03-21T10:08:46.747 INFO:tasks.workunit.client.0.smithi153.stdout:1/303: truncate d6/f2f 789616 0 2024-03-21T10:08:46.747 INFO:tasks.workunit.client.0.smithi153.stdout:5/325: getdents d4/d5/df/d5e/d69 0 2024-03-21T10:08:46.747 INFO:tasks.workunit.client.1.smithi174.stdout:3/92: sync 2024-03-21T10:08:46.747 INFO:tasks.workunit.client.1.smithi174.stdout:1/155: rename d4/d7/d15/f24 to d4/d7/d15/f38 0 2024-03-21T10:08:46.747 INFO:tasks.workunit.client.1.smithi174.stdout:7/105: creat d0/d4/f23 x:0 0 0 2024-03-21T10:08:46.750 INFO:tasks.workunit.client.0.smithi153.stdout:5/326: chown d4/d8/d3b/f4b 5274054 1 2024-03-21T10:08:46.750 INFO:tasks.workunit.client.1.smithi174.stdout:7/106: fdatasync d0/d4/f18 0 2024-03-21T10:08:46.751 INFO:tasks.workunit.client.0.smithi153.stdout:4/353: dread d11/d18/f21 [0,4194304] 0 2024-03-21T10:08:46.752 INFO:tasks.workunit.client.0.smithi153.stdout:7/253: dread d4/f1a [0,4194304] 0 2024-03-21T10:08:46.752 INFO:tasks.workunit.client.1.smithi174.stdout:0/96: mknod d9/c1a 0 2024-03-21T10:08:46.753 INFO:tasks.workunit.client.1.smithi174.stdout:4/178: dwrite d4/d19/f41 [0,4194304] 0 2024-03-21T10:08:46.755 INFO:tasks.workunit.client.0.smithi153.stdout:9/307: dwrite d1/d43/d44/d46/d1c/d24/f38 [0,4194304] 0 2024-03-21T10:08:46.755 INFO:tasks.workunit.client.1.smithi174.stdout:2/159: creat d5/d25/d27/f3b x:0 0 0 2024-03-21T10:08:46.755 INFO:tasks.workunit.client.1.smithi174.stdout:4/179: chown d4/d21/f27 955 1 2024-03-21T10:08:46.755 INFO:tasks.workunit.client.1.smithi174.stdout:6/92: getdents d0/d3/d5/df 0 2024-03-21T10:08:46.756 INFO:tasks.workunit.client.1.smithi174.stdout:3/93: symlink d0/l24 0 2024-03-21T10:08:46.757 INFO:tasks.workunit.client.1.smithi174.stdout:2/160: fdatasync d5/d8/f28 0 2024-03-21T10:08:46.759 INFO:tasks.workunit.client.1.smithi174.stdout:1/156: creat d4/d7/d15/f39 x:0 0 0 2024-03-21T10:08:46.759 INFO:tasks.workunit.client.1.smithi174.stdout:7/107: mknod d0/c24 0 2024-03-21T10:08:46.760 INFO:tasks.workunit.client.1.smithi174.stdout:1/157: fdatasync d4/d7/f1e 0 2024-03-21T10:08:46.761 INFO:tasks.workunit.client.0.smithi153.stdout:8/334: dwrite f3 [4194304,4194304] 0 2024-03-21T10:08:46.761 INFO:tasks.workunit.client.0.smithi153.stdout:4/354: mkdir d11/d18/d6d 0 2024-03-21T10:08:46.767 INFO:tasks.workunit.client.0.smithi153.stdout:4/355: chown d11/d18/l5c 24054 1 2024-03-21T10:08:46.767 INFO:tasks.workunit.client.0.smithi153.stdout:4/356: dread - d11/d2d/f69 zero size 2024-03-21T10:08:46.768 INFO:tasks.workunit.client.0.smithi153.stdout:9/308: mkdir d1/d43/d44/d46/d1c/d2e/d67 0 2024-03-21T10:08:46.768 INFO:tasks.workunit.client.0.smithi153.stdout:8/335: chown d5/d9/d43/dd/d20/d26/d33 9198 1 2024-03-21T10:08:46.770 INFO:tasks.workunit.client.0.smithi153.stdout:4/357: readlink d11/d2d/l3f 0 2024-03-21T10:08:46.771 INFO:tasks.workunit.client.0.smithi153.stdout:9/309: write d1/d43/d44/f25 [3175504,30445] 0 2024-03-21T10:08:46.772 INFO:tasks.workunit.client.0.smithi153.stdout:8/336: chown d5/d9/c58 116 1 2024-03-21T10:08:46.774 INFO:tasks.workunit.client.1.smithi174.stdout:4/180: symlink d4/d21/d24/d28/l43 0 2024-03-21T10:08:46.774 INFO:tasks.workunit.client.0.smithi153.stdout:4/358: chown d11/c51 8000650 1 2024-03-21T10:08:46.777 INFO:tasks.workunit.client.0.smithi153.stdout:9/310: unlink d1/d3b/c5e 0 2024-03-21T10:08:46.778 INFO:tasks.workunit.client.0.smithi153.stdout:7/254: dwrite d4/d11/d24/f28 [0,4194304] 0 2024-03-21T10:08:46.779 INFO:tasks.workunit.client.0.smithi153.stdout:9/311: dread - d1/d43/d44/d46/d1c/d2e/f62 zero size 2024-03-21T10:08:46.779 INFO:tasks.workunit.client.0.smithi153.stdout:8/337: fsync d5/d9/d43/dd/d20/d26/d24/d52/d55/f5f 0 2024-03-21T10:08:46.781 INFO:tasks.workunit.client.1.smithi174.stdout:6/93: dread d0/d3/d5/df/d16/f17 [0,4194304] 0 2024-03-21T10:08:46.784 INFO:tasks.workunit.client.1.smithi174.stdout:2/161: creat d5/d25/d27/f3c x:0 0 0 2024-03-21T10:08:46.786 INFO:tasks.workunit.client.0.smithi153.stdout:7/255: mknod d4/d11/d24/c4e 0 2024-03-21T10:08:46.786 INFO:tasks.workunit.client.1.smithi174.stdout:2/162: readlink d5/d8/l18 0 2024-03-21T10:08:46.789 INFO:tasks.workunit.client.0.smithi153.stdout:9/312: chown d1/c18 2257 1 2024-03-21T10:08:46.791 INFO:tasks.workunit.client.0.smithi153.stdout:7/256: read d4/d11/d24/f29 [621782,17024] 0 2024-03-21T10:08:46.792 INFO:tasks.workunit.client.0.smithi153.stdout:6/509: dread d1f/d2b/d33/d40/d5f/f6d [0,4194304] 0 2024-03-21T10:08:46.794 INFO:tasks.workunit.client.1.smithi174.stdout:7/108: creat d0/d4/d10/d1b/f25 x:0 0 0 2024-03-21T10:08:46.797 INFO:tasks.workunit.client.1.smithi174.stdout:5/120: creat d1/f29 x:0 0 0 2024-03-21T10:08:46.800 INFO:tasks.workunit.client.0.smithi153.stdout:5/327: dwrite d4/d8/f11 [0,4194304] 0 2024-03-21T10:08:46.800 INFO:tasks.workunit.client.1.smithi174.stdout:0/97: dwrite f7 [0,4194304] 0 2024-03-21T10:08:46.802 INFO:tasks.workunit.client.1.smithi174.stdout:5/121: dread d1/d16/d1c/f23 [0,4194304] 0 2024-03-21T10:08:46.802 INFO:tasks.workunit.client.1.smithi174.stdout:4/181: mknod d4/c44 0 2024-03-21T10:08:46.803 INFO:tasks.workunit.client.1.smithi174.stdout:7/109: write d0/d4/d10/f1c [5147528,5596] 0 2024-03-21T10:08:46.807 INFO:tasks.workunit.client.0.smithi153.stdout:7/257: mknod d4/d2a/d3b/c4f 0 2024-03-21T10:08:46.809 INFO:tasks.workunit.client.0.smithi153.stdout:5/328: write d4/d8/d5a/d5d/f55 [716529,7955] 0 2024-03-21T10:08:46.812 INFO:tasks.workunit.client.0.smithi153.stdout:6/510: getdents d1f/d2b/d33/d40/d51 0 2024-03-21T10:08:46.812 INFO:tasks.workunit.client.0.smithi153.stdout:4/359: dwrite d11/d2d/d3d/f45 [0,4194304] 0 2024-03-21T10:08:46.814 INFO:tasks.workunit.client.0.smithi153.stdout:6/511: chown d1f/l2a 467 1 2024-03-21T10:08:46.814 INFO:tasks.workunit.client.0.smithi153.stdout:7/258: write d4/f1a [1732751,43650] 0 2024-03-21T10:08:46.815 INFO:tasks.workunit.client.1.smithi174.stdout:6/94: dwrite d0/d3/d5/df/d16/f1a [0,4194304] 0 2024-03-21T10:08:46.815 INFO:tasks.workunit.client.0.smithi153.stdout:0/250: dread d4/d16/d1f/f20 [0,4194304] 0 2024-03-21T10:08:46.816 INFO:tasks.workunit.client.1.smithi174.stdout:0/98: mknod d9/c1b 0 2024-03-21T10:08:46.818 INFO:tasks.workunit.client.0.smithi153.stdout:7/259: read d4/d11/d24/f35 [82171,11320] 0 2024-03-21T10:08:46.818 INFO:tasks.workunit.client.0.smithi153.stdout:6/512: write d1f/d2f/d59/f8d [471391,86793] 0 2024-03-21T10:08:46.819 INFO:tasks.workunit.client.1.smithi174.stdout:4/182: unlink d4/l12 0 2024-03-21T10:08:46.820 INFO:tasks.workunit.client.1.smithi174.stdout:2/163: dread d5/d25/d27/d2d/f21 [0,4194304] 0 2024-03-21T10:08:46.820 INFO:tasks.workunit.client.1.smithi174.stdout:7/110: creat d0/f26 x:0 0 0 2024-03-21T10:08:46.821 INFO:tasks.workunit.client.0.smithi153.stdout:8/338: dread d5/d9/d43/dd/d20/d26/d24/d52/d55/f5c [0,4194304] 0 2024-03-21T10:08:46.822 INFO:tasks.workunit.client.1.smithi174.stdout:5/122: mknod d1/c2a 0 2024-03-21T10:08:46.825 INFO:tasks.workunit.client.0.smithi153.stdout:8/339: fdatasync d5/d9/d43/dd/d20/f50 0 2024-03-21T10:08:46.825 INFO:tasks.workunit.client.0.smithi153.stdout:8/340: stat d5/d9/d43/dd/d2d/c2e 0 2024-03-21T10:08:46.826 INFO:tasks.workunit.client.1.smithi174.stdout:6/95: read - d0/d3/d5/f1c zero size 2024-03-21T10:08:46.829 INFO:tasks.workunit.client.1.smithi174.stdout:1/158: dwrite d4/d7/d33/f36 [0,4194304] 0 2024-03-21T10:08:46.831 INFO:tasks.workunit.client.1.smithi174.stdout:7/111: stat d0/d4/d10/f1c 0 2024-03-21T10:08:46.834 INFO:tasks.workunit.client.1.smithi174.stdout:7/112: chown d0/d4/d10/f19 20857 1 2024-03-21T10:08:46.835 INFO:tasks.workunit.client.0.smithi153.stdout:9/313: fdatasync d1/d43/d44/d46/d1c/d24/f38 0 2024-03-21T10:08:46.835 INFO:tasks.workunit.client.0.smithi153.stdout:4/360: symlink d11/l6e 0 2024-03-21T10:08:46.836 INFO:tasks.workunit.client.1.smithi174.stdout:1/159: write d4/d2d/f34 [382405,97378] 0 2024-03-21T10:08:46.836 INFO:tasks.workunit.client.1.smithi174.stdout:4/183: unlink d4/d21/d24/d28/d38/f3b 0 2024-03-21T10:08:46.838 INFO:tasks.workunit.client.0.smithi153.stdout:0/251: symlink d4/d16/l4f 0 2024-03-21T10:08:46.843 INFO:tasks.workunit.client.0.smithi153.stdout:6/513: dwrite d1f/d2f/f3d [0,4194304] 0 2024-03-21T10:08:46.844 INFO:tasks.workunit.client.0.smithi153.stdout:8/341: mknod d5/d9/d43/c6a 0 2024-03-21T10:08:46.846 INFO:tasks.workunit.client.1.smithi174.stdout:6/96: creat d0/d3/d5/df/f22 x:0 0 0 2024-03-21T10:08:46.847 INFO:tasks.workunit.client.0.smithi153.stdout:9/314: mkdir d1/d4e/d68 0 2024-03-21T10:08:46.851 INFO:tasks.workunit.client.0.smithi153.stdout:0/252: symlink d4/d16/d1f/d22/l50 0 2024-03-21T10:08:46.852 INFO:tasks.workunit.client.1.smithi174.stdout:5/123: link d1/d16/c28 d1/d16/d1c/c2b 0 2024-03-21T10:08:46.854 INFO:tasks.workunit.client.0.smithi153.stdout:8/342: dread - d5/d3b/f4f zero size 2024-03-21T10:08:46.857 INFO:tasks.workunit.client.0.smithi153.stdout:6/514: mknod d1f/da0/cb8 0 2024-03-21T10:08:46.858 INFO:tasks.workunit.client.0.smithi153.stdout:8/343: stat d5/d1b 0 2024-03-21T10:08:46.858 INFO:tasks.workunit.client.0.smithi153.stdout:5/329: dwrite d4/d5/de/d38/f4d [4194304,4194304] 0 2024-03-21T10:08:46.860 INFO:tasks.workunit.client.0.smithi153.stdout:0/253: read d4/d16/d1f/f20 [616174,114242] 0 2024-03-21T10:08:46.861 INFO:tasks.workunit.client.1.smithi174.stdout:0/99: dwrite d9/f19 [0,4194304] 0 2024-03-21T10:08:46.863 INFO:tasks.workunit.client.0.smithi153.stdout:7/260: dwrite d4/f25 [0,4194304] 0 2024-03-21T10:08:46.863 INFO:tasks.workunit.client.0.smithi153.stdout:9/315: write d1/d43/d44/d46/d1c/d24/f32 [2668249,52155] 0 2024-03-21T10:08:46.865 INFO:tasks.workunit.client.1.smithi174.stdout:1/160: rename d4/d7/f1b to d4/d7/d32/f3a 0 2024-03-21T10:08:46.868 INFO:tasks.workunit.client.1.smithi174.stdout:2/164: dwrite d5/d25/d27/f3b [0,4194304] 0 2024-03-21T10:08:46.873 INFO:tasks.workunit.client.0.smithi153.stdout:8/344: creat d5/d9/d43/dd/d20/d26/d33/f6b x:0 0 0 2024-03-21T10:08:46.874 INFO:tasks.workunit.client.1.smithi174.stdout:4/184: dwrite d4/f8 [0,4194304] 0 2024-03-21T10:08:46.875 INFO:tasks.workunit.client.0.smithi153.stdout:5/330: write d4/d5/f52 [3587596,114926] 0 2024-03-21T10:08:46.878 INFO:tasks.workunit.client.1.smithi174.stdout:0/100: truncate d9/f18 734908 0 2024-03-21T10:08:46.878 INFO:tasks.workunit.client.1.smithi174.stdout:4/185: write d4/d19/f23 [380161,36370] 0 2024-03-21T10:08:46.878 INFO:tasks.workunit.client.0.smithi153.stdout:4/361: dwrite f1 [4194304,4194304] 0 2024-03-21T10:08:46.878 INFO:tasks.workunit.client.0.smithi153.stdout:6/515: rename d1f/da0/cb8 to d1f/d2f/d59/d70/cb9 0 2024-03-21T10:08:46.880 INFO:tasks.workunit.client.1.smithi174.stdout:1/161: creat d4/d7/d28/f3b x:0 0 0 2024-03-21T10:08:46.881 INFO:tasks.workunit.client.0.smithi153.stdout:5/331: mkdir d4/d5/df/d67/d6a 0 2024-03-21T10:08:46.882 INFO:tasks.workunit.client.1.smithi174.stdout:5/124: write d1/fd [8584472,32511] 0 2024-03-21T10:08:46.882 INFO:tasks.workunit.client.1.smithi174.stdout:4/186: write d4/d19/f23 [1259267,72020] 0 2024-03-21T10:08:46.883 INFO:tasks.workunit.client.0.smithi153.stdout:6/516: dread - d1f/d2b/d33/fa2 zero size 2024-03-21T10:08:46.885 INFO:tasks.workunit.client.0.smithi153.stdout:9/316: rename d1/d4e/l60 to d1/d43/d44/d46/d1c/d2e/l69 0 2024-03-21T10:08:46.887 INFO:tasks.workunit.client.1.smithi174.stdout:4/187: write d4/fc [322258,57019] 0 2024-03-21T10:08:46.887 INFO:tasks.workunit.client.0.smithi153.stdout:8/345: getdents d5 0 2024-03-21T10:08:46.891 INFO:tasks.workunit.client.0.smithi153.stdout:4/362: creat d11/f6f x:0 0 0 2024-03-21T10:08:46.891 INFO:tasks.workunit.client.0.smithi153.stdout:5/332: symlink d4/d5/df/d5e/d69/l6b 0 2024-03-21T10:08:46.893 INFO:tasks.workunit.client.0.smithi153.stdout:6/517: chown d1f/d2b/d33/d40/d4f/c61 232 1 2024-03-21T10:08:46.896 INFO:tasks.workunit.client.1.smithi174.stdout:1/162: write d4/d5/f22 [1525179,81993] 0 2024-03-21T10:08:46.896 INFO:tasks.workunit.client.0.smithi153.stdout:8/346: creat d5/d9/d43/f6c x:0 0 0 2024-03-21T10:08:46.899 INFO:tasks.workunit.client.1.smithi174.stdout:5/125: fsync d1/f22 0 2024-03-21T10:08:46.900 INFO:tasks.workunit.client.0.smithi153.stdout:7/261: dwrite d4/d2a/d40/f4c [0,4194304] 0 2024-03-21T10:08:46.901 INFO:tasks.workunit.client.0.smithi153.stdout:9/317: creat d1/d43/d44/d46/d1c/d24/d41/f6a x:0 0 0 2024-03-21T10:08:46.901 INFO:tasks.workunit.client.1.smithi174.stdout:4/188: unlink d4/d21/d24/c36 0 2024-03-21T10:08:46.901 INFO:tasks.workunit.client.0.smithi153.stdout:4/363: mknod d11/d2d/d6b/c70 0 2024-03-21T10:08:46.905 INFO:tasks.workunit.client.1.smithi174.stdout:1/163: creat d4/d7/d32/f3c x:0 0 0 2024-03-21T10:08:46.907 INFO:tasks.workunit.client.0.smithi153.stdout:9/318: rmdir d1/d43/d44/d46/d4d 39 2024-03-21T10:08:46.908 INFO:tasks.workunit.client.0.smithi153.stdout:6/518: mkdir d1f/d2b/d33/d40/d51/d98/dba 0 2024-03-21T10:08:46.908 INFO:tasks.workunit.client.1.smithi174.stdout:6/97: dread d0/d3/d5/df/d16/f1a [0,4194304] 0 2024-03-21T10:08:46.909 INFO:tasks.workunit.client.1.smithi174.stdout:7/113: truncate d0/f9 2587386 0 2024-03-21T10:08:46.912 INFO:tasks.workunit.client.0.smithi153.stdout:8/347: symlink d5/d9/d43/dd/d20/d26/l6d 0 2024-03-21T10:08:46.913 INFO:tasks.workunit.client.0.smithi153.stdout:4/364: mkdir d11/d71 0 2024-03-21T10:08:46.914 INFO:tasks.workunit.client.1.smithi174.stdout:5/126: creat d1/f2c x:0 0 0 2024-03-21T10:08:46.917 INFO:tasks.workunit.client.0.smithi153.stdout:8/348: dread - d5/d9/d43/dd/d2d/d45/f4a zero size 2024-03-21T10:08:46.917 INFO:tasks.workunit.client.1.smithi174.stdout:4/189: dread d4/f10 [0,4194304] 0 2024-03-21T10:08:46.919 INFO:tasks.workunit.client.1.smithi174.stdout:1/164: fsync d4/d5/f22 0 2024-03-21T10:08:46.919 INFO:tasks.workunit.client.1.smithi174.stdout:7/114: rmdir d0/d4/d10/d1b 39 2024-03-21T10:08:46.920 INFO:tasks.workunit.client.0.smithi153.stdout:4/365: getdents d11/d18/d6d 0 2024-03-21T10:08:46.920 INFO:tasks.workunit.client.1.smithi174.stdout:8/85: sync 2024-03-21T10:08:46.921 INFO:tasks.workunit.client.1.smithi174.stdout:4/190: truncate d4/fc 1117081 0 2024-03-21T10:08:46.921 INFO:tasks.workunit.client.0.smithi153.stdout:9/319: rmdir d1/d52 0 2024-03-21T10:08:46.923 INFO:tasks.workunit.client.1.smithi174.stdout:4/191: fsync d4/ff 0 2024-03-21T10:08:46.924 INFO:tasks.workunit.client.0.smithi153.stdout:3/329: sync 2024-03-21T10:08:46.924 INFO:tasks.workunit.client.0.smithi153.stdout:2/394: sync 2024-03-21T10:08:46.924 INFO:tasks.workunit.client.0.smithi153.stdout:1/304: sync 2024-03-21T10:08:46.924 INFO:tasks.workunit.client.0.smithi153.stdout:8/349: creat d5/d9/d43/dd/d20/d26/d33/f6e x:0 0 0 2024-03-21T10:08:46.926 INFO:tasks.workunit.client.0.smithi153.stdout:3/330: readlink l9 0 2024-03-21T10:08:46.926 INFO:tasks.workunit.client.0.smithi153.stdout:4/366: mknod d11/d2d/c72 0 2024-03-21T10:08:46.927 INFO:tasks.workunit.client.1.smithi174.stdout:7/115: read d0/d4/fd [287655,6332] 0 2024-03-21T10:08:46.927 INFO:tasks.workunit.client.1.smithi174.stdout:6/98: creat d0/d3/d20/f23 x:0 0 0 2024-03-21T10:08:46.931 INFO:tasks.workunit.client.0.smithi153.stdout:8/350: stat d5/d9/f7 0 2024-03-21T10:08:46.932 INFO:tasks.workunit.client.0.smithi153.stdout:0/254: truncate d4/f32 3592853 0 2024-03-21T10:08:46.932 INFO:tasks.workunit.client.1.smithi174.stdout:5/127: write d1/d16/d1c/f23 [339430,62285] 0 2024-03-21T10:08:46.932 INFO:tasks.workunit.client.0.smithi153.stdout:1/305: read - d6/d32/d3e/d4f/f58 zero size 2024-03-21T10:08:46.933 INFO:tasks.workunit.client.0.smithi153.stdout:4/367: creat d11/d2d/f73 x:0 0 0 2024-03-21T10:08:46.934 INFO:tasks.workunit.client.1.smithi174.stdout:5/128: write d1/fc [1927502,107628] 0 2024-03-21T10:08:46.936 INFO:tasks.workunit.client.1.smithi174.stdout:5/129: readlink d1/l14 0 2024-03-21T10:08:46.939 INFO:tasks.workunit.client.0.smithi153.stdout:0/255: unlink d4/l3c 0 2024-03-21T10:08:46.940 INFO:tasks.workunit.client.0.smithi153.stdout:2/395: creat dd/d15/d18/d2c/d33/f88 x:0 0 0 2024-03-21T10:08:46.940 INFO:tasks.workunit.client.1.smithi174.stdout:0/101: truncate d9/f13 1829142 0 2024-03-21T10:08:46.940 INFO:tasks.workunit.client.0.smithi153.stdout:0/256: read d4/d16/d1f/f20 [3383617,85993] 0 2024-03-21T10:08:46.941 INFO:tasks.workunit.client.1.smithi174.stdout:5/130: truncate d1/f29 182745 0 2024-03-21T10:08:46.942 INFO:tasks.workunit.client.0.smithi153.stdout:1/306: readlink d6/d13/l38 0 2024-03-21T10:08:46.944 INFO:tasks.workunit.client.0.smithi153.stdout:2/396: creat dd/d15/d18/d2c/d4b/d56/d63/f89 x:0 0 0 2024-03-21T10:08:46.945 INFO:tasks.workunit.client.0.smithi153.stdout:1/307: write d6/fd [528609,17991] 0 2024-03-21T10:08:46.946 INFO:tasks.workunit.client.0.smithi153.stdout:4/368: write d11/d18/d50/f53 [3677315,19414] 0 2024-03-21T10:08:46.946 INFO:tasks.workunit.client.0.smithi153.stdout:6/519: dread f19 [0,4194304] 0 2024-03-21T10:08:46.947 INFO:tasks.workunit.client.1.smithi174.stdout:1/165: dread d4/f19 [0,4194304] 0 2024-03-21T10:08:46.948 INFO:tasks.workunit.client.0.smithi153.stdout:2/397: chown dd/d3b/f4c 262569 1 2024-03-21T10:08:46.951 INFO:tasks.workunit.client.0.smithi153.stdout:7/262: write d4/f10 [1932023,11471] 0 2024-03-21T10:08:46.952 INFO:tasks.workunit.client.1.smithi174.stdout:8/86: fsync f8 0 2024-03-21T10:08:46.953 INFO:tasks.workunit.client.1.smithi174.stdout:7/116: dwrite d0/f26 [0,4194304] 0 2024-03-21T10:08:46.955 INFO:tasks.workunit.client.1.smithi174.stdout:8/87: chown d9/f16 710 1 2024-03-21T10:08:46.956 INFO:tasks.workunit.client.0.smithi153.stdout:6/520: mkdir d1f/d99/dbb 0 2024-03-21T10:08:46.962 INFO:tasks.workunit.client.0.smithi153.stdout:4/369: unlink d11/d2d/f2f 0 2024-03-21T10:08:46.962 INFO:tasks.workunit.client.1.smithi174.stdout:0/102: mkdir d9/d1c 0 2024-03-21T10:08:46.963 INFO:tasks.workunit.client.0.smithi153.stdout:8/351: dwrite d5/d9/d43/dd/d20/f50 [0,4194304] 0 2024-03-21T10:08:46.964 INFO:tasks.workunit.client.1.smithi174.stdout:5/131: mknod d1/d16/c2d 0 2024-03-21T10:08:46.965 INFO:tasks.workunit.client.1.smithi174.stdout:1/166: creat d4/d7/d33/f3d x:0 0 0 2024-03-21T10:08:46.966 INFO:tasks.workunit.client.1.smithi174.stdout:0/103: dread - d9/f17 zero size 2024-03-21T10:08:46.966 INFO:tasks.workunit.client.1.smithi174.stdout:8/88: fsync f5 0 2024-03-21T10:08:46.966 INFO:tasks.workunit.client.1.smithi174.stdout:5/132: fdatasync d1/d16/f24 0 2024-03-21T10:08:46.967 INFO:tasks.workunit.client.0.smithi153.stdout:8/352: write d5/d1b/f59 [393133,81724] 0 2024-03-21T10:08:46.967 INFO:tasks.workunit.client.1.smithi174.stdout:4/192: link d4/l29 d4/d21/d31/d3a/l45 0 2024-03-21T10:08:46.969 INFO:tasks.workunit.client.0.smithi153.stdout:9/320: write d1/d4/fc [1172351,103176] 0 2024-03-21T10:08:46.971 INFO:tasks.workunit.client.1.smithi174.stdout:6/99: getdents d0/d3/d5 0 2024-03-21T10:08:46.973 INFO:tasks.workunit.client.1.smithi174.stdout:8/89: rename d9/f16 to d9/d17/d19/f1e 0 2024-03-21T10:08:46.974 INFO:tasks.workunit.client.0.smithi153.stdout:7/263: dwrite d4/d11/d24/f3a [0,4194304] 0 2024-03-21T10:08:46.975 INFO:tasks.workunit.client.1.smithi174.stdout:8/90: fdatasync d9/f14 0 2024-03-21T10:08:46.979 INFO:tasks.workunit.client.1.smithi174.stdout:4/193: creat d4/f46 x:0 0 0 2024-03-21T10:08:46.980 INFO:tasks.workunit.client.1.smithi174.stdout:6/100: symlink d0/d3/d5/d6/l24 0 2024-03-21T10:08:46.982 INFO:tasks.workunit.client.1.smithi174.stdout:8/91: creat d9/d17/d19/f1f x:0 0 0 2024-03-21T10:08:46.984 INFO:tasks.workunit.client.0.smithi153.stdout:1/308: dwrite d6/d3a/f53 [0,4194304] 0 2024-03-21T10:08:46.985 INFO:tasks.workunit.client.1.smithi174.stdout:4/194: dread - d4/d19/f1c zero size 2024-03-21T10:08:46.985 INFO:tasks.workunit.client.0.smithi153.stdout:6/521: dwrite d1f/d2f/f94 [0,4194304] 0 2024-03-21T10:08:46.985 INFO:tasks.workunit.client.0.smithi153.stdout:2/398: write dd/d15/d18/d2c/d4b/d56/f85 [140039,70899] 0 2024-03-21T10:08:46.988 INFO:tasks.workunit.client.1.smithi174.stdout:5/133: getdents d1 0 2024-03-21T10:08:46.991 INFO:tasks.workunit.client.1.smithi174.stdout:8/92: creat d9/d17/f20 x:0 0 0 2024-03-21T10:08:46.994 INFO:tasks.workunit.client.1.smithi174.stdout:1/167: dwrite d4/d7/d32/f3c [0,4194304] 0 2024-03-21T10:08:46.995 INFO:tasks.workunit.client.0.smithi153.stdout:4/370: symlink d11/d18/d50/l74 0 2024-03-21T10:08:47.000 INFO:tasks.workunit.client.1.smithi174.stdout:4/195: mknod d4/d21/d31/c47 0 2024-03-21T10:08:47.000 INFO:tasks.workunit.client.1.smithi174.stdout:5/134: rmdir d1/d16 39 2024-03-21T10:08:47.001 INFO:tasks.workunit.client.1.smithi174.stdout:8/93: creat d9/d17/f21 x:0 0 0 2024-03-21T10:08:47.003 INFO:tasks.workunit.client.1.smithi174.stdout:1/168: creat d4/d7/d33/f3e x:0 0 0 2024-03-21T10:08:47.004 INFO:tasks.workunit.client.0.smithi153.stdout:0/257: truncate d4/db/f13 2992120 0 2024-03-21T10:08:47.004 INFO:tasks.workunit.client.0.smithi153.stdout:7/264: mkdir d4/d2a/d40/d50 0 2024-03-21T10:08:47.007 INFO:tasks.workunit.client.0.smithi153.stdout:7/265: write d4/d11/f48 [274245,110394] 0 2024-03-21T10:08:47.007 INFO:tasks.workunit.client.0.smithi153.stdout:5/333: dread d4/d5/de/d38/f64 [0,4194304] 0 2024-03-21T10:08:47.009 INFO:tasks.workunit.client.0.smithi153.stdout:7/266: dread - d4/d14/d2f/f45 zero size 2024-03-21T10:08:47.009 INFO:tasks.workunit.client.1.smithi174.stdout:4/196: creat d4/d21/d31/f48 x:0 0 0 2024-03-21T10:08:47.012 INFO:tasks.workunit.client.0.smithi153.stdout:6/522: creat d1f/d2f/d59/fbc x:0 0 0 2024-03-21T10:08:47.012 INFO:tasks.workunit.client.1.smithi174.stdout:8/94: write d9/f1c [1536409,94464] 0 2024-03-21T10:08:47.013 INFO:tasks.workunit.client.1.smithi174.stdout:1/169: truncate d4/d5/fd 1179517 0 2024-03-21T10:08:47.017 INFO:tasks.workunit.client.0.smithi153.stdout:6/523: write d1f/d2f/f94 [526832,90414] 0 2024-03-21T10:08:47.018 INFO:tasks.workunit.client.0.smithi153.stdout:1/309: creat d6/d10/d2c/d54/f62 x:0 0 0 2024-03-21T10:08:47.018 INFO:tasks.workunit.client.1.smithi174.stdout:5/135: chown d1/d16/c1e 0 1 2024-03-21T10:08:47.019 INFO:tasks.workunit.client.1.smithi174.stdout:1/170: dread - d4/d7/f21 zero size 2024-03-21T10:08:47.019 INFO:tasks.workunit.client.1.smithi174.stdout:4/197: write d4/d21/d24/f42 [477808,105665] 0 2024-03-21T10:08:47.019 INFO:tasks.workunit.client.1.smithi174.stdout:7/117: dwrite d0/f26 [4194304,4194304] 0 2024-03-21T10:08:47.020 INFO:tasks.workunit.client.1.smithi174.stdout:8/95: chown d9/d17/d19/f1f 7757069 1 2024-03-21T10:08:47.023 INFO:tasks.workunit.client.0.smithi153.stdout:9/321: symlink d1/d43/d44/d46/l6b 0 2024-03-21T10:08:47.024 INFO:tasks.workunit.client.1.smithi174.stdout:1/171: chown d4/fb 3518 1 2024-03-21T10:08:47.026 INFO:tasks.workunit.client.1.smithi174.stdout:5/136: write d1/f2c [849860,102634] 0 2024-03-21T10:08:47.027 INFO:tasks.workunit.client.0.smithi153.stdout:2/399: dwrite dd/d15/d18/d2c/d4b/f81 [0,4194304] 0 2024-03-21T10:08:47.031 INFO:tasks.workunit.client.0.smithi153.stdout:5/334: rename d4/d8/d3b/f66 to d4/d8/f6c 0 2024-03-21T10:08:47.031 INFO:tasks.workunit.client.0.smithi153.stdout:5/335: stat f1 0 2024-03-21T10:08:47.031 INFO:tasks.workunit.client.0.smithi153.stdout:4/371: rmdir d11/d5e 39 2024-03-21T10:08:47.036 INFO:tasks.workunit.client.0.smithi153.stdout:0/258: dread d4/f37 [0,4194304] 0 2024-03-21T10:08:47.036 INFO:tasks.workunit.client.0.smithi153.stdout:6/524: rename d1f/d2b/d33/d40/d51/d98/dba to d1f/d2b/d33/d40/d51/dbd 0 2024-03-21T10:08:47.037 INFO:tasks.workunit.client.1.smithi174.stdout:6/101: dread d0/d3/d5/f12 [0,4194304] 0 2024-03-21T10:08:47.038 INFO:tasks.workunit.client.0.smithi153.stdout:4/372: mknod d11/d2d/c75 0 2024-03-21T10:08:47.040 INFO:tasks.workunit.client.1.smithi174.stdout:4/198: creat d4/d21/f49 x:0 0 0 2024-03-21T10:08:47.040 INFO:tasks.workunit.client.1.smithi174.stdout:9/165: sync 2024-03-21T10:08:47.041 INFO:tasks.workunit.client.0.smithi153.stdout:2/400: mknod dd/d15/d18/d3a/c8a 0 2024-03-21T10:08:47.041 INFO:tasks.workunit.client.1.smithi174.stdout:7/118: dread d0/f26 [0,4194304] 0 2024-03-21T10:08:47.042 INFO:tasks.workunit.client.1.smithi174.stdout:1/172: symlink d4/d7/d20/l3f 0 2024-03-21T10:08:47.043 INFO:tasks.workunit.client.1.smithi174.stdout:9/166: readlink d2/d11/d1d/l27 0 2024-03-21T10:08:47.044 INFO:tasks.workunit.client.0.smithi153.stdout:6/525: mknod d1f/d2b/d33/d40/d51/cbe 0 2024-03-21T10:08:47.045 INFO:tasks.workunit.client.1.smithi174.stdout:5/137: unlink d1/d16/c1b 0 2024-03-21T10:08:47.047 INFO:tasks.workunit.client.1.smithi174.stdout:6/102: unlink d0/d3/d5/f12 0 2024-03-21T10:08:47.049 INFO:tasks.workunit.client.0.smithi153.stdout:4/373: creat d11/f76 x:0 0 0 2024-03-21T10:08:47.051 INFO:tasks.workunit.client.1.smithi174.stdout:1/173: write d4/fb [219399,128286] 0 2024-03-21T10:08:47.051 INFO:tasks.workunit.client.0.smithi153.stdout:0/259: rename d4/c1d to d4/d16/d19/c51 0 2024-03-21T10:08:47.052 INFO:tasks.workunit.client.1.smithi174.stdout:7/119: rename d0/d4/f16 to d0/f27 0 2024-03-21T10:08:47.053 INFO:tasks.workunit.client.1.smithi174.stdout:6/103: chown d0/d3/d5/d6/la 54637421 1 2024-03-21T10:08:47.053 INFO:tasks.workunit.client.1.smithi174.stdout:5/138: mkdir d1/d16/d2e 0 2024-03-21T10:08:47.055 INFO:tasks.workunit.client.1.smithi174.stdout:0/104: dwrite d9/f13 [0,4194304] 0 2024-03-21T10:08:47.059 INFO:tasks.workunit.client.0.smithi153.stdout:4/374: creat d11/d18/f77 x:0 0 0 2024-03-21T10:08:47.059 INFO:tasks.workunit.client.0.smithi153.stdout:9/322: dwrite d1/d43/d44/d46/d1c/d24/d41/f5b [0,4194304] 0 2024-03-21T10:08:47.063 INFO:tasks.workunit.client.1.smithi174.stdout:6/104: read - d0/d3/d5/df/f22 zero size 2024-03-21T10:08:47.063 INFO:tasks.workunit.client.0.smithi153.stdout:8/353: truncate d5/d9/f19 648067 0 2024-03-21T10:08:47.065 INFO:tasks.workunit.client.1.smithi174.stdout:1/174: mknod d4/d7/d20/c40 0 2024-03-21T10:08:47.066 INFO:tasks.workunit.client.0.smithi153.stdout:9/323: stat d1/d4e 0 2024-03-21T10:08:47.066 INFO:tasks.workunit.client.1.smithi174.stdout:2/165: sync 2024-03-21T10:08:47.067 INFO:tasks.workunit.client.1.smithi174.stdout:3/94: sync 2024-03-21T10:08:47.068 INFO:tasks.workunit.client.1.smithi174.stdout:6/105: read - d0/d3/d5/f1c zero size 2024-03-21T10:08:47.069 INFO:tasks.workunit.client.1.smithi174.stdout:8/96: dread d9/f1c [0,4194304] 0 2024-03-21T10:08:47.071 INFO:tasks.workunit.client.1.smithi174.stdout:5/139: mkdir d1/d2f 0 2024-03-21T10:08:47.072 INFO:tasks.workunit.client.1.smithi174.stdout:2/166: write d5/d8/f1d [444637,90072] 0 2024-03-21T10:08:47.073 INFO:tasks.workunit.client.1.smithi174.stdout:3/95: readlink d0/d12/l15 0 2024-03-21T10:08:47.076 INFO:tasks.workunit.client.1.smithi174.stdout:4/199: dwrite d4/d21/d31/d3a/f3d [0,4194304] 0 2024-03-21T10:08:47.077 INFO:tasks.workunit.client.0.smithi153.stdout:6/526: creat d1f/fbf x:0 0 0 2024-03-21T10:08:47.077 INFO:tasks.workunit.client.0.smithi153.stdout:7/267: dwrite d4/d2a/f3e [0,4194304] 0 2024-03-21T10:08:47.080 INFO:tasks.workunit.client.0.smithi153.stdout:5/336: dwrite d4/d8/f2d [0,4194304] 0 2024-03-21T10:08:47.081 INFO:tasks.workunit.client.1.smithi174.stdout:0/105: truncate f4 3839768 0 2024-03-21T10:08:47.081 INFO:tasks.workunit.client.1.smithi174.stdout:1/175: creat d4/d7/d15/f41 x:0 0 0 2024-03-21T10:08:47.082 INFO:tasks.workunit.client.1.smithi174.stdout:7/120: link d0/d4/f23 d0/d4/d10/d1b/f28 0 2024-03-21T10:08:47.082 INFO:tasks.workunit.client.1.smithi174.stdout:9/167: link d2/d11/c14 d2/d11/c32 0 2024-03-21T10:08:47.082 INFO:tasks.workunit.client.1.smithi174.stdout:0/106: fsync f7 0 2024-03-21T10:08:47.083 INFO:tasks.workunit.client.0.smithi153.stdout:9/324: fdatasync d1/d43/d44/d46/d1c/d2e/f51 0 2024-03-21T10:08:47.086 INFO:tasks.workunit.client.0.smithi153.stdout:0/260: getdents d4/d4c 0 2024-03-21T10:08:47.087 INFO:tasks.workunit.client.1.smithi174.stdout:1/176: dread - d4/d7/d28/f3b zero size 2024-03-21T10:08:47.087 INFO:tasks.workunit.client.1.smithi174.stdout:2/167: write d5/d8/f23 [1041692,99557] 0 2024-03-21T10:08:47.087 INFO:tasks.workunit.client.1.smithi174.stdout:2/168: chown d5 175084 1 2024-03-21T10:08:47.087 INFO:tasks.workunit.client.1.smithi174.stdout:0/107: readlink l8 0 2024-03-21T10:08:47.088 INFO:tasks.workunit.client.1.smithi174.stdout:3/96: stat d0/l4 0 2024-03-21T10:08:47.088 INFO:tasks.workunit.client.1.smithi174.stdout:5/140: truncate d1/d16/f1a 1124390 0 2024-03-21T10:08:47.088 INFO:tasks.workunit.client.0.smithi153.stdout:7/268: creat d4/d11/d24/f51 x:0 0 0 2024-03-21T10:08:47.089 INFO:tasks.workunit.client.1.smithi174.stdout:8/97: symlink d9/d17/l22 0 2024-03-21T10:08:47.091 INFO:tasks.workunit.client.0.smithi153.stdout:9/325: chown d1/d43/d44/d46/f19 35 1 2024-03-21T10:08:47.091 INFO:tasks.workunit.client.1.smithi174.stdout:1/177: dread - d4/d7/d15/f1f zero size 2024-03-21T10:08:47.093 INFO:tasks.workunit.client.1.smithi174.stdout:6/106: rename d0/cd to d0/d3/d20/c25 0 2024-03-21T10:08:47.093 INFO:tasks.workunit.client.0.smithi153.stdout:0/261: write d4/d4c/f4e [230221,25953] 0 2024-03-21T10:08:47.094 INFO:tasks.workunit.client.1.smithi174.stdout:0/108: write d9/f17 [707149,54080] 0 2024-03-21T10:08:47.096 INFO:tasks.workunit.client.1.smithi174.stdout:7/121: mknod d0/d4/c29 0 2024-03-21T10:08:47.097 INFO:tasks.workunit.client.1.smithi174.stdout:0/109: chown d9/f17 6 1 2024-03-21T10:08:47.098 INFO:tasks.workunit.client.0.smithi153.stdout:4/375: dwrite d11/d2d/d3d/f56 [0,4194304] 0 2024-03-21T10:08:47.098 INFO:tasks.workunit.client.1.smithi174.stdout:6/107: stat d0/d3/d5/df/d16 0 2024-03-21T10:08:47.099 INFO:tasks.workunit.client.1.smithi174.stdout:0/110: readlink d9/le 0 2024-03-21T10:08:47.099 INFO:tasks.workunit.client.1.smithi174.stdout:5/141: write d1/d16/d1c/f23 [529099,43420] 0 2024-03-21T10:08:47.100 INFO:tasks.workunit.client.1.smithi174.stdout:8/98: write d9/d17/d19/f1f [973181,123683] 0 2024-03-21T10:08:47.100 INFO:tasks.workunit.client.0.smithi153.stdout:5/337: creat d4/d5/df/d5e/d69/f6d x:0 0 0 2024-03-21T10:08:47.102 INFO:tasks.workunit.client.1.smithi174.stdout:3/97: symlink d0/d12/d14/d10/l25 0 2024-03-21T10:08:47.103 INFO:tasks.workunit.client.0.smithi153.stdout:8/354: dread d5/d9/d43/dd/d20/d26/d24/d52/d55/f5c [0,4194304] 0 2024-03-21T10:08:47.108 INFO:tasks.workunit.client.1.smithi174.stdout:3/98: write d0/d12/d14/d1a/f1e [279487,79285] 0 2024-03-21T10:08:47.110 INFO:tasks.workunit.client.1.smithi174.stdout:0/111: creat d9/f1d x:0 0 0 2024-03-21T10:08:47.112 INFO:tasks.workunit.client.0.smithi153.stdout:0/262: truncate d4/d16/d19/d30/f41 25942 0 2024-03-21T10:08:47.113 INFO:tasks.workunit.client.0.smithi153.stdout:5/338: creat d4/d5/df/d5e/f6e x:0 0 0 2024-03-21T10:08:47.115 INFO:tasks.workunit.client.0.smithi153.stdout:2/401: dread dd/d15/d18/d2c/d33/f36 [0,4194304] 0 2024-03-21T10:08:47.116 INFO:tasks.workunit.client.1.smithi174.stdout:9/168: getdents d2/d11/d1d 0 2024-03-21T10:08:47.117 INFO:tasks.workunit.client.1.smithi174.stdout:0/112: write d9/f15 [161023,110000] 0 2024-03-21T10:08:47.118 INFO:tasks.workunit.client.0.smithi153.stdout:2/402: chown dd/d15/d18/f24 2032 1 2024-03-21T10:08:47.120 INFO:tasks.workunit.client.1.smithi174.stdout:0/113: readlink d9/le 0 2024-03-21T10:08:47.122 INFO:tasks.workunit.client.1.smithi174.stdout:5/142: rename d1/f3 to d1/d16/f30 0 2024-03-21T10:08:47.122 INFO:tasks.workunit.client.1.smithi174.stdout:0/114: write d9/f1d [801026,43105] 0 2024-03-21T10:08:47.126 INFO:tasks.workunit.client.0.smithi153.stdout:8/355: creat d5/d9/d43/dd/d20/d26/d24/d52/d56/f6f x:0 0 0 2024-03-21T10:08:47.127 INFO:tasks.workunit.client.0.smithi153.stdout:6/527: write d1f/d2b/d33/d40/d4f/d66/f75 [433436,21476] 0 2024-03-21T10:08:47.129 INFO:tasks.workunit.client.1.smithi174.stdout:6/108: getdents d0/d3/d5/df 0 2024-03-21T10:08:47.132 INFO:tasks.workunit.client.1.smithi174.stdout:9/169: creat d2/d11/d1d/f33 x:0 0 0 2024-03-21T10:08:47.132 INFO:tasks.workunit.client.1.smithi174.stdout:6/109: read - d0/d3/d5/f1c zero size 2024-03-21T10:08:47.132 INFO:tasks.workunit.client.0.smithi153.stdout:0/263: rename d4/d16/c3b to d4/d16/d1f/d22/c52 0 2024-03-21T10:08:47.132 INFO:tasks.workunit.client.0.smithi153.stdout:3/331: sync 2024-03-21T10:08:47.133 INFO:tasks.workunit.client.1.smithi174.stdout:9/170: dread - d2/d11/f29 zero size 2024-03-21T10:08:47.136 INFO:tasks.workunit.client.0.smithi153.stdout:9/326: rmdir d1/d43/d44/d46/d1c/d24/d41/d55 0 2024-03-21T10:08:47.136 INFO:tasks.workunit.client.1.smithi174.stdout:7/122: write d0/d4/fd [4988960,55781] 0 2024-03-21T10:08:47.137 INFO:tasks.workunit.client.0.smithi153.stdout:4/376: creat d11/f78 x:0 0 0 2024-03-21T10:08:47.139 INFO:tasks.workunit.client.0.smithi153.stdout:4/377: chown d11/d18/l5c 604 1 2024-03-21T10:08:47.142 INFO:tasks.workunit.client.0.smithi153.stdout:1/310: dread d6/d13/f3f [0,4194304] 0 2024-03-21T10:08:47.144 INFO:tasks.workunit.client.1.smithi174.stdout:7/123: symlink d0/d4/d10/d1b/l2a 0 2024-03-21T10:08:47.144 INFO:tasks.workunit.client.0.smithi153.stdout:6/528: unlink d1f/d2b/d33/f41 0 2024-03-21T10:08:47.146 INFO:tasks.workunit.client.0.smithi153.stdout:8/356: rmdir d5/d9/d43/dd/d20/d26/d47 39 2024-03-21T10:08:47.146 INFO:tasks.workunit.client.1.smithi174.stdout:0/115: dwrite d9/f19 [4194304,4194304] 0 2024-03-21T10:08:47.149 INFO:tasks.workunit.client.1.smithi174.stdout:9/171: rmdir d2/d11 39 2024-03-21T10:08:47.150 INFO:tasks.workunit.client.1.smithi174.stdout:7/124: creat d0/f2b x:0 0 0 2024-03-21T10:08:47.150 INFO:tasks.workunit.client.1.smithi174.stdout:2/169: dread d5/f38 [0,4194304] 0 2024-03-21T10:08:47.151 INFO:tasks.workunit.client.0.smithi153.stdout:2/403: link dd/d15/d18/d3a/d3d/f74 dd/d15/d18/d2c/d4b/d56/d63/f8b 0 2024-03-21T10:08:47.152 INFO:tasks.workunit.client.1.smithi174.stdout:6/110: dread d0/d3/d5/df/d16/f1a [0,4194304] 0 2024-03-21T10:08:47.153 INFO:tasks.workunit.client.1.smithi174.stdout:0/116: creat d9/d1c/f1e x:0 0 0 2024-03-21T10:08:47.154 INFO:tasks.workunit.client.1.smithi174.stdout:9/172: creat d2/d24/f34 x:0 0 0 2024-03-21T10:08:47.155 INFO:tasks.workunit.client.0.smithi153.stdout:3/332: creat dd/d41/f71 x:0 0 0 2024-03-21T10:08:47.156 INFO:tasks.workunit.client.1.smithi174.stdout:6/111: chown d0/d3/f8 28701 1 2024-03-21T10:08:47.156 INFO:tasks.workunit.client.1.smithi174.stdout:7/125: truncate d0/d4/f23 872788 0 2024-03-21T10:08:47.156 INFO:tasks.workunit.client.1.smithi174.stdout:3/99: dwrite d0/f3 [0,4194304] 0 2024-03-21T10:08:47.157 INFO:tasks.workunit.client.1.smithi174.stdout:0/117: creat d9/f1f x:0 0 0 2024-03-21T10:08:47.157 INFO:tasks.workunit.client.0.smithi153.stdout:4/378: rmdir d11/d2d/d57 39 2024-03-21T10:08:47.159 INFO:tasks.workunit.client.0.smithi153.stdout:7/269: truncate d4/f25 2929584 0 2024-03-21T10:08:47.160 INFO:tasks.workunit.client.1.smithi174.stdout:9/173: chown d2/d11/f29 830169851 1 2024-03-21T10:08:47.161 INFO:tasks.workunit.client.0.smithi153.stdout:4/379: write d11/f16 [203280,130012] 0 2024-03-21T10:08:47.161 INFO:tasks.workunit.client.1.smithi174.stdout:9/174: readlink d2/l15 0 2024-03-21T10:08:47.162 INFO:tasks.workunit.client.1.smithi174.stdout:6/112: truncate d0/d3/d5/df/f22 362604 0 2024-03-21T10:08:47.164 INFO:tasks.workunit.client.1.smithi174.stdout:7/126: dread d0/ff [0,4194304] 0 2024-03-21T10:08:47.165 INFO:tasks.workunit.client.0.smithi153.stdout:6/529: symlink d1f/d2b/d33/d40/d4f/d66/lc0 0 2024-03-21T10:08:47.167 INFO:tasks.workunit.client.1.smithi174.stdout:0/118: link d9/fc d9/d1c/f20 0 2024-03-21T10:08:47.168 INFO:tasks.workunit.client.1.smithi174.stdout:2/170: dread d5/f12 [0,4194304] 0 2024-03-21T10:08:47.169 INFO:tasks.workunit.client.0.smithi153.stdout:6/530: write d1f/d2b/f2c [3817284,84963] 0 2024-03-21T10:08:47.169 INFO:tasks.workunit.client.0.smithi153.stdout:5/339: dread d4/d5/d16/f63 [0,4194304] 0 2024-03-21T10:08:47.169 INFO:tasks.workunit.client.0.smithi153.stdout:9/327: dread d1/d4e/f54 [0,4194304] 0 2024-03-21T10:08:47.172 INFO:tasks.workunit.client.1.smithi174.stdout:2/171: write d5/d8/f1d [1366346,30587] 0 2024-03-21T10:08:47.173 INFO:tasks.workunit.client.1.smithi174.stdout:0/119: write d9/fc [1316135,52193] 0 2024-03-21T10:08:47.176 INFO:tasks.workunit.client.1.smithi174.stdout:7/127: mknod d0/d4/d10/c2c 0 2024-03-21T10:08:47.177 INFO:tasks.workunit.client.0.smithi153.stdout:8/357: link d5/d9/d43/f16 d5/d9/d43/dd/d2d/d45/f70 0 2024-03-21T10:08:47.177 INFO:tasks.workunit.client.1.smithi174.stdout:2/172: truncate d5/d25/d27/f3c 669405 0 2024-03-21T10:08:47.180 INFO:tasks.workunit.client.1.smithi174.stdout:6/113: getdents d0 0 2024-03-21T10:08:47.180 INFO:tasks.workunit.client.0.smithi153.stdout:2/404: mkdir dd/d15/d18/d3a/d8c 0 2024-03-21T10:08:47.181 INFO:tasks.workunit.client.1.smithi174.stdout:7/128: creat d0/d4/f2d x:0 0 0 2024-03-21T10:08:47.182 INFO:tasks.workunit.client.1.smithi174.stdout:2/173: mknod d5/d8/c3d 0 2024-03-21T10:08:47.182 INFO:tasks.workunit.client.1.smithi174.stdout:9/175: dread d2/f1b [0,4194304] 0 2024-03-21T10:08:47.183 INFO:tasks.workunit.client.0.smithi153.stdout:8/358: truncate d5/d9/d43/dd/d20/d26/d24/d52/f64 324045 0 2024-03-21T10:08:47.184 INFO:tasks.workunit.client.0.smithi153.stdout:6/531: rmdir d1f/d2b/d33/d40/d51 39 2024-03-21T10:08:47.185 INFO:tasks.workunit.client.1.smithi174.stdout:7/129: creat d0/d4/d10/d1b/f2e x:0 0 0 2024-03-21T10:08:47.186 INFO:tasks.workunit.client.0.smithi153.stdout:7/270: symlink d4/l52 0 2024-03-21T10:08:47.188 INFO:tasks.workunit.client.1.smithi174.stdout:9/176: creat d2/d24/d2e/f35 x:0 0 0 2024-03-21T10:08:47.189 INFO:tasks.workunit.client.0.smithi153.stdout:2/405: write dd/f26 [1272648,71458] 0 2024-03-21T10:08:47.190 INFO:tasks.workunit.client.1.smithi174.stdout:9/177: chown d2/fc 3 1 2024-03-21T10:08:47.191 INFO:tasks.workunit.client.1.smithi174.stdout:7/130: creat d0/f2f x:0 0 0 2024-03-21T10:08:47.192 INFO:tasks.workunit.client.0.smithi153.stdout:7/271: chown d4/d2a/d40/c49 823816244 1 2024-03-21T10:08:47.193 INFO:tasks.workunit.client.0.smithi153.stdout:2/406: chown dd/d15/d18/c1f 1967034 1 2024-03-21T10:08:47.194 INFO:tasks.workunit.client.1.smithi174.stdout:7/131: read d0/f26 [7458658,87367] 0 2024-03-21T10:08:47.194 INFO:tasks.workunit.client.0.smithi153.stdout:8/359: creat d5/d9/d43/dd/d20/d26/d24/d52/d55/f71 x:0 0 0 2024-03-21T10:08:47.195 INFO:tasks.workunit.client.0.smithi153.stdout:4/380: link d11/d2d/d57/c59 d11/d2d/c79 0 2024-03-21T10:08:47.196 INFO:tasks.workunit.client.0.smithi153.stdout:4/381: stat d11/d18/f64 0 2024-03-21T10:08:47.196 INFO:tasks.workunit.client.0.smithi153.stdout:7/272: creat d4/d14/f53 x:0 0 0 2024-03-21T10:08:47.198 INFO:tasks.workunit.client.0.smithi153.stdout:6/532: read d1f/f20 [2894709,8515] 0 2024-03-21T10:08:47.199 INFO:tasks.workunit.client.0.smithi153.stdout:8/360: dread - d5/d9/d43/dd/d20/d26/d33/f5d zero size 2024-03-21T10:08:47.200 INFO:tasks.workunit.client.0.smithi153.stdout:2/407: mkdir dd/d15/d18/d2c/d4b/d56/d63/d8d 0 2024-03-21T10:08:47.201 INFO:tasks.workunit.client.1.smithi174.stdout:2/174: dwrite d5/f24 [0,4194304] 0 2024-03-21T10:08:47.206 INFO:tasks.workunit.client.1.smithi174.stdout:2/175: write d5/f26 [3530826,59032] 0 2024-03-21T10:08:47.206 INFO:tasks.workunit.client.1.smithi174.stdout:0/120: rmdir d9/d1c 39 2024-03-21T10:08:47.209 INFO:tasks.workunit.client.1.smithi174.stdout:3/100: truncate d0/f2 2167533 0 2024-03-21T10:08:47.210 INFO:tasks.workunit.client.0.smithi153.stdout:7/273: creat d4/d11/d24/f54 x:0 0 0 2024-03-21T10:08:47.211 INFO:tasks.workunit.client.1.smithi174.stdout:0/121: write d9/f17 [179259,125969] 0 2024-03-21T10:08:47.213 INFO:tasks.workunit.client.0.smithi153.stdout:2/408: stat dd/d15/d18/d2c/d4b/d56/d63/f89 0 2024-03-21T10:08:47.215 INFO:tasks.workunit.client.1.smithi174.stdout:9/178: dwrite d2/f9 [4194304,4194304] 0 2024-03-21T10:08:47.216 INFO:tasks.workunit.client.0.smithi153.stdout:8/361: write d5/d9/d43/dd/d20/d26/d24/d52/d55/f71 [358920,109824] 0 2024-03-21T10:08:47.217 INFO:tasks.workunit.client.1.smithi174.stdout:2/176: truncate d5/d25/d27/f39 541683 0 2024-03-21T10:08:47.219 INFO:tasks.workunit.client.0.smithi153.stdout:0/264: dread d4/f32 [0,4194304] 0 2024-03-21T10:08:47.220 INFO:tasks.workunit.client.1.smithi174.stdout:0/122: mkdir d9/d21 0 2024-03-21T10:08:47.221 INFO:tasks.workunit.client.0.smithi153.stdout:9/328: dread d1/d43/d44/d46/f26 [0,4194304] 0 2024-03-21T10:08:47.222 INFO:tasks.workunit.client.1.smithi174.stdout:0/123: readlink l1 0 2024-03-21T10:08:47.224 INFO:tasks.workunit.client.1.smithi174.stdout:0/124: write d9/f19 [4463775,98779] 0 2024-03-21T10:08:47.225 INFO:tasks.workunit.client.0.smithi153.stdout:0/265: dread - d4/d16/d1f/f31 zero size 2024-03-21T10:08:47.225 INFO:tasks.workunit.client.0.smithi153.stdout:7/274: mknod d4/d11/d24/c55 0 2024-03-21T10:08:47.226 INFO:tasks.workunit.client.0.smithi153.stdout:8/362: write d5/d9/d43/dd/d20/d26/d24/d52/f54 [392840,116801] 0 2024-03-21T10:08:47.226 INFO:tasks.workunit.client.1.smithi174.stdout:2/177: dread d5/d8/f28 [0,4194304] 0 2024-03-21T10:08:47.230 INFO:tasks.workunit.client.1.smithi174.stdout:9/179: creat d2/d11/f36 x:0 0 0 2024-03-21T10:08:47.231 INFO:tasks.workunit.client.0.smithi153.stdout:9/329: chown d1/d43/d44/d46/d1c/d24/d41/f6a 189487596 1 2024-03-21T10:08:47.232 INFO:tasks.workunit.client.1.smithi174.stdout:3/101: getdents d0/d12/d14/d1a 0 2024-03-21T10:08:47.233 INFO:tasks.workunit.client.0.smithi153.stdout:8/363: dread - d5/d9/d43/dd/d20/d26/d24/d52/d55/f60 zero size 2024-03-21T10:08:47.233 INFO:tasks.workunit.client.0.smithi153.stdout:6/533: link d1f/fbf d1f/d2f/d59/d70/fc1 0 2024-03-21T10:08:47.233 INFO:tasks.workunit.client.1.smithi174.stdout:2/178: mknod d5/d25/d27/d2d/c3e 0 2024-03-21T10:08:47.234 INFO:tasks.workunit.client.1.smithi174.stdout:9/180: mknod d2/d24/d2e/c37 0 2024-03-21T10:08:47.235 INFO:tasks.workunit.client.1.smithi174.stdout:0/125: symlink d9/d1c/l22 0 2024-03-21T10:08:47.236 INFO:tasks.workunit.client.1.smithi174.stdout:0/126: read - d9/f1f zero size 2024-03-21T10:08:47.236 INFO:tasks.workunit.client.0.smithi153.stdout:6/534: dread - d1f/d2f/d59/d70/fc1 zero size 2024-03-21T10:08:47.237 INFO:tasks.workunit.client.1.smithi174.stdout:3/102: mknod d0/d12/d14/c26 0 2024-03-21T10:08:47.237 INFO:tasks.workunit.client.1.smithi174.stdout:0/127: chown l3 1470662 1 2024-03-21T10:08:47.238 INFO:tasks.workunit.client.0.smithi153.stdout:7/275: mkdir d4/d11/d56 0 2024-03-21T10:08:47.239 INFO:tasks.workunit.client.1.smithi174.stdout:2/179: unlink d5/d8/l18 0 2024-03-21T10:08:47.240 INFO:tasks.workunit.client.1.smithi174.stdout:9/181: symlink d2/d24/l38 0 2024-03-21T10:08:47.240 INFO:tasks.workunit.client.0.smithi153.stdout:4/382: getdents d11/d2d/d57 0 2024-03-21T10:08:47.241 INFO:tasks.workunit.client.1.smithi174.stdout:0/128: truncate d9/d1c/f1e 715596 0 2024-03-21T10:08:47.241 INFO:tasks.workunit.client.0.smithi153.stdout:2/409: mkdir dd/d15/d18/d2c/d4b/d56/d63/d8d/d8e 0 2024-03-21T10:08:47.249 INFO:tasks.workunit.client.1.smithi174.stdout:3/103: unlink d0/d12/d14/d10/f16 0 2024-03-21T10:08:47.249 INFO:tasks.workunit.client.1.smithi174.stdout:2/180: creat d5/d25/d27/d2e/f3f x:0 0 0 2024-03-21T10:08:47.249 INFO:tasks.workunit.client.1.smithi174.stdout:9/182: write d2/f1b [3020335,125849] 0 2024-03-21T10:08:47.251 INFO:tasks.workunit.client.0.smithi153.stdout:8/364: write d5/d9/d43/dd/d2d/d45/f70 [2586295,86821] 0 2024-03-21T10:08:47.252 INFO:tasks.workunit.client.1.smithi174.stdout:9/183: write d2/f9 [7233861,66434] 0 2024-03-21T10:08:47.254 INFO:tasks.workunit.client.0.smithi153.stdout:9/330: symlink d1/d4/d5f/l6c 0 2024-03-21T10:08:47.254 INFO:tasks.workunit.client.1.smithi174.stdout:2/181: creat d5/d25/d27/f40 x:0 0 0 2024-03-21T10:08:47.255 INFO:tasks.workunit.client.1.smithi174.stdout:9/184: write d2/f1b [1156543,37573] 0 2024-03-21T10:08:47.255 INFO:tasks.workunit.client.0.smithi153.stdout:2/410: rmdir dd/d3b/d50/d58 39 2024-03-21T10:08:47.259 INFO:tasks.workunit.client.1.smithi174.stdout:9/185: chown d2/d24/d2e/f35 7 1 2024-03-21T10:08:47.260 INFO:tasks.workunit.client.0.smithi153.stdout:8/365: truncate d5/d9/d43/dd/f22 387642 0 2024-03-21T10:08:47.260 INFO:tasks.workunit.client.0.smithi153.stdout:6/535: creat d1f/d2b/d33/d40/d51/fc2 x:0 0 0 2024-03-21T10:08:47.262 INFO:tasks.workunit.client.1.smithi174.stdout:0/129: dwrite d9/f17 [0,4194304] 0 2024-03-21T10:08:47.262 INFO:tasks.workunit.client.0.smithi153.stdout:7/276: unlink d4/d19/c36 0 2024-03-21T10:08:47.264 INFO:tasks.workunit.client.0.smithi153.stdout:4/383: dread f10 [0,4194304] 0 2024-03-21T10:08:47.264 INFO:tasks.workunit.client.0.smithi153.stdout:9/331: mknod d1/d43/d44/d46/d1c/d2e/c6d 0 2024-03-21T10:08:47.266 INFO:tasks.workunit.client.0.smithi153.stdout:2/411: fsync dd/d15/d18/d2c/f39 0 2024-03-21T10:08:47.268 INFO:tasks.workunit.client.0.smithi153.stdout:6/536: mknod d1f/d99/cc3 0 2024-03-21T10:08:47.271 INFO:tasks.workunit.client.0.smithi153.stdout:8/366: creat d5/d9/d43/dd/d20/d26/d24/d52/f72 x:0 0 0 2024-03-21T10:08:47.272 INFO:tasks.workunit.client.1.smithi174.stdout:9/186: rename d2/c5 to d2/c39 0 2024-03-21T10:08:47.272 INFO:tasks.workunit.client.1.smithi174.stdout:3/104: truncate d0/f7 154105 0 2024-03-21T10:08:47.275 INFO:tasks.workunit.client.1.smithi174.stdout:3/105: read - d0/d12/d14/d1a/f23 zero size 2024-03-21T10:08:47.276 INFO:tasks.workunit.client.1.smithi174.stdout:3/106: chown d0 19679150 1 2024-03-21T10:08:47.276 INFO:tasks.workunit.client.1.smithi174.stdout:9/187: truncate d2/d11/d1d/f33 600283 0 2024-03-21T10:08:47.276 INFO:tasks.workunit.client.0.smithi153.stdout:6/537: stat d1f/d2f/d59/l97 0 2024-03-21T10:08:47.277 INFO:tasks.workunit.client.0.smithi153.stdout:7/277: symlink d4/d1b/l57 0 2024-03-21T10:08:47.277 INFO:tasks.workunit.client.0.smithi153.stdout:3/333: dwrite dd/d1e/d2f/f3a [0,4194304] 0 2024-03-21T10:08:47.279 INFO:tasks.workunit.client.1.smithi174.stdout:0/130: dwrite d9/f17 [0,4194304] 0 2024-03-21T10:08:47.279 INFO:tasks.workunit.client.0.smithi153.stdout:1/311: write d6/d32/f37 [653490,82787] 0 2024-03-21T10:08:47.282 INFO:tasks.workunit.client.1.smithi174.stdout:9/188: creat d2/d11/f3a x:0 0 0 2024-03-21T10:08:47.283 INFO:tasks.workunit.client.0.smithi153.stdout:2/412: creat dd/d15/d5e/f8f x:0 0 0 2024-03-21T10:08:47.286 INFO:tasks.workunit.client.1.smithi174.stdout:9/189: unlink d2/c13 0 2024-03-21T10:08:47.287 INFO:tasks.workunit.client.0.smithi153.stdout:2/413: chown dd/d15/d18/c25 91139620 1 2024-03-21T10:08:47.287 INFO:tasks.workunit.client.0.smithi153.stdout:4/384: truncate d11/d2d/f4a 941229 0 2024-03-21T10:08:47.288 INFO:tasks.workunit.client.1.smithi174.stdout:0/131: mkdir d9/d21/d23 0 2024-03-21T10:08:47.291 INFO:tasks.workunit.client.1.smithi174.stdout:0/132: rename d9/c11 to d9/d1c/c24 0 2024-03-21T10:08:47.294 INFO:tasks.workunit.client.0.smithi153.stdout:8/367: creat d5/d9/d43/dd/d20/d26/d33/f73 x:0 0 0 2024-03-21T10:08:47.294 INFO:tasks.workunit.client.1.smithi174.stdout:0/133: chown d9/c10 178185003 1 2024-03-21T10:08:47.298 INFO:tasks.workunit.client.0.smithi153.stdout:8/368: write d5/d1b/f37 [4260990,73140] 0 2024-03-21T10:08:47.299 INFO:tasks.workunit.client.1.smithi174.stdout:9/190: link d2/c39 d2/d11/d1d/d2a/c3b 0 2024-03-21T10:08:47.300 INFO:tasks.workunit.client.0.smithi153.stdout:1/312: creat d6/d32/d3e/f63 x:0 0 0 2024-03-21T10:08:47.301 INFO:tasks.workunit.client.1.smithi174.stdout:0/134: creat d9/d21/d23/f25 x:0 0 0 2024-03-21T10:08:47.301 INFO:tasks.workunit.client.1.smithi174.stdout:9/191: chown d2/d24/c26 50596 1 2024-03-21T10:08:47.301 INFO:tasks.workunit.client.0.smithi153.stdout:6/538: symlink d1f/da0/lc4 0 2024-03-21T10:08:47.303 INFO:tasks.workunit.client.0.smithi153.stdout:3/334: mknod dd/d1e/d2f/d35/c72 0 2024-03-21T10:08:47.304 INFO:tasks.workunit.client.1.smithi174.stdout:3/107: truncate d0/f7 241701 0 2024-03-21T10:08:47.306 INFO:tasks.workunit.client.1.smithi174.stdout:9/192: creat d2/f3c x:0 0 0 2024-03-21T10:08:47.306 INFO:tasks.workunit.client.1.smithi174.stdout:3/108: mkdir d0/d12/d14/d1a/d27 0 2024-03-21T10:08:47.307 INFO:tasks.workunit.client.0.smithi153.stdout:5/340: dwrite d4/d8/f26 [0,4194304] 0 2024-03-21T10:08:47.308 INFO:tasks.workunit.client.0.smithi153.stdout:9/332: symlink d1/d43/d44/d46/d1c/d4b/l6e 0 2024-03-21T10:08:47.310 INFO:tasks.workunit.client.1.smithi174.stdout:3/109: rename d0/c19 to d0/c28 0 2024-03-21T10:08:47.320 INFO:tasks.workunit.client.0.smithi153.stdout:2/414: dwrite dd/d15/d18/d2c/d4b/d56/f85 [0,4194304] 0 2024-03-21T10:08:47.328 INFO:tasks.workunit.client.0.smithi153.stdout:6/539: dwrite d1f/d2f/fb4 [0,4194304] 0 2024-03-21T10:08:47.331 INFO:tasks.workunit.client.0.smithi153.stdout:9/333: symlink d1/d43/d44/d46/d1c/d24/d41/l6f 0 2024-03-21T10:08:47.339 INFO:tasks.workunit.client.0.smithi153.stdout:3/335: mkdir dd/d1e/d2f/d35/d66/d73 0 2024-03-21T10:08:47.343 INFO:tasks.workunit.client.0.smithi153.stdout:4/385: dread d11/d18/f28 [0,4194304] 0 2024-03-21T10:08:47.348 INFO:tasks.workunit.client.0.smithi153.stdout:9/334: symlink d1/d4/l70 0 2024-03-21T10:08:47.349 INFO:tasks.workunit.client.0.smithi153.stdout:5/341: dread d4/d8/d5a/d5d/f5b [0,4194304] 0 2024-03-21T10:08:47.351 INFO:tasks.workunit.client.0.smithi153.stdout:5/342: write d4/d8/f2d [3560753,58829] 0 2024-03-21T10:08:47.354 INFO:tasks.workunit.client.0.smithi153.stdout:3/336: rename dd/f10 to dd/d1e/d45/f74 0 2024-03-21T10:08:47.356 INFO:tasks.workunit.client.0.smithi153.stdout:4/386: creat d11/d18/d50/f7a x:0 0 0 2024-03-21T10:08:47.356 INFO:tasks.workunit.client.0.smithi153.stdout:5/343: dread d4/d8/d3b/d56/f62 [0,4194304] 0 2024-03-21T10:08:47.357 INFO:tasks.workunit.client.0.smithi153.stdout:1/313: dread d6/d3a/d3c/d50/d51/f5f [0,4194304] 0 2024-03-21T10:08:47.357 INFO:tasks.workunit.client.0.smithi153.stdout:4/387: readlink d11/d2d/l3f 0 2024-03-21T10:08:47.362 INFO:tasks.workunit.client.0.smithi153.stdout:8/369: getdents d5/d9/d43/dd/d2d/d45 0 2024-03-21T10:08:47.362 INFO:tasks.workunit.client.0.smithi153.stdout:6/540: getdents d1f/d2b/d33/d40/d5f/dac 0 2024-03-21T10:08:47.363 INFO:tasks.workunit.client.0.smithi153.stdout:4/388: symlink d11/d18/l7b 0 2024-03-21T10:08:47.365 INFO:tasks.workunit.client.0.smithi153.stdout:4/389: chown d11/f1e 7939 1 2024-03-21T10:08:47.369 INFO:tasks.workunit.client.0.smithi153.stdout:5/344: getdents d4/d5/de/d2f 0 2024-03-21T10:08:47.374 INFO:tasks.workunit.client.0.smithi153.stdout:5/345: write d4/d5/de/f43 [656842,12860] 0 2024-03-21T10:08:47.377 INFO:tasks.workunit.client.0.smithi153.stdout:3/337: getdents dd/d1e/d45 0 2024-03-21T10:08:47.377 INFO:tasks.workunit.client.0.smithi153.stdout:6/541: truncate d1f/d2f/f3d 1093927 0 2024-03-21T10:08:47.379 INFO:tasks.workunit.client.0.smithi153.stdout:5/346: read d4/d5/de/d38/f4d [5627982,9160] 0 2024-03-21T10:08:47.382 INFO:tasks.workunit.client.0.smithi153.stdout:6/542: write d1f/d2f/f32 [850481,115865] 0 2024-03-21T10:08:47.382 INFO:tasks.workunit.client.0.smithi153.stdout:3/338: write dd/f25 [4809363,51652] 0 2024-03-21T10:08:47.383 INFO:tasks.workunit.client.1.smithi174.stdout:3/110: creat d0/d12/d14/d1a/f29 x:0 0 0 2024-03-21T10:08:47.383 INFO:tasks.workunit.client.0.smithi153.stdout:1/314: dwrite d6/d3a/f53 [4194304,4194304] 0 2024-03-21T10:08:47.383 INFO:tasks.workunit.client.0.smithi153.stdout:5/347: readlink d4/d5/de/l20 0 2024-03-21T10:08:47.387 INFO:tasks.workunit.client.0.smithi153.stdout:9/335: dread d1/d43/d44/f33 [0,4194304] 0 2024-03-21T10:08:47.388 INFO:tasks.workunit.client.1.smithi174.stdout:0/135: dwrite d9/d1c/f1e [0,4194304] 0 2024-03-21T10:08:47.392 INFO:tasks.workunit.client.0.smithi153.stdout:3/339: write dd/f20 [1879447,59094] 0 2024-03-21T10:08:47.392 INFO:tasks.workunit.client.1.smithi174.stdout:0/136: rmdir d9 39 2024-03-21T10:08:47.393 INFO:tasks.workunit.client.0.smithi153.stdout:5/348: write d4/d8/d5a/f51 [843396,24119] 0 2024-03-21T10:08:47.397 INFO:tasks.workunit.client.0.smithi153.stdout:9/336: write d1/d4e/f53 [439001,50930] 0 2024-03-21T10:08:47.401 INFO:tasks.workunit.client.1.smithi174.stdout:9/193: dwrite d2/f20 [0,4194304] 0 2024-03-21T10:08:47.401 INFO:tasks.workunit.client.1.smithi174.stdout:0/137: truncate d9/f17 4697966 0 2024-03-21T10:08:47.402 INFO:tasks.workunit.client.0.smithi153.stdout:5/349: unlink d4/d8/d5a/c49 0 2024-03-21T10:08:47.404 INFO:tasks.workunit.client.0.smithi153.stdout:9/337: rmdir d1/d43/d44/d46/d1c 39 2024-03-21T10:08:47.404 INFO:tasks.workunit.client.0.smithi153.stdout:5/350: readlink d4/l35 0 2024-03-21T10:08:47.410 INFO:tasks.workunit.client.1.smithi174.stdout:0/138: unlink d9/c1a 0 2024-03-21T10:08:47.411 INFO:tasks.workunit.client.1.smithi174.stdout:0/139: chown d9/c14 529700334 1 2024-03-21T10:08:47.412 INFO:tasks.workunit.client.1.smithi174.stdout:3/111: dwrite d0/d12/f17 [0,4194304] 0 2024-03-21T10:08:47.415 INFO:tasks.workunit.client.1.smithi174.stdout:3/112: chown d0/d12/d14/c26 488136 1 2024-03-21T10:08:47.418 INFO:tasks.workunit.client.0.smithi153.stdout:5/351: creat d4/d5/de/d44/f6f x:0 0 0 2024-03-21T10:08:47.420 INFO:tasks.workunit.client.0.smithi153.stdout:9/338: creat d1/d43/d44/d46/d1c/d61/f71 x:0 0 0 2024-03-21T10:08:47.420 INFO:tasks.workunit.client.0.smithi153.stdout:5/352: read - d4/d8/d5a/f61 zero size 2024-03-21T10:08:47.423 INFO:tasks.workunit.client.1.smithi174.stdout:3/113: rename d0/d12/d14/f20 to d0/d12/d14/d1a/d27/f2a 0 2024-03-21T10:08:47.423 INFO:tasks.workunit.client.0.smithi153.stdout:6/543: dwrite d1f/d2b/d33/d40/d5f/f82 [0,4194304] 0 2024-03-21T10:08:47.429 INFO:tasks.workunit.client.0.smithi153.stdout:5/353: read d4/d5/d16/f1e [294851,44469] 0 2024-03-21T10:08:47.429 INFO:tasks.workunit.client.0.smithi153.stdout:9/339: mkdir d1/d43/d72 0 2024-03-21T10:08:47.429 INFO:tasks.workunit.client.0.smithi153.stdout:6/544: fsync d1f/d2b/d33/d40/d4f/f67 0 2024-03-21T10:08:47.432 INFO:tasks.workunit.client.1.smithi174.stdout:0/140: dwrite d9/f13 [0,4194304] 0 2024-03-21T10:08:47.432 INFO:tasks.workunit.client.0.smithi153.stdout:9/340: write d1/d43/d44/d46/f37 [3031134,61173] 0 2024-03-21T10:08:47.432 INFO:tasks.workunit.client.0.smithi153.stdout:6/545: dread - d1f/d2b/d33/d40/d51/fc2 zero size 2024-03-21T10:08:47.433 INFO:tasks.workunit.client.0.smithi153.stdout:5/354: mkdir d4/d5/d16/d24/d70 0 2024-03-21T10:08:47.434 INFO:tasks.workunit.client.1.smithi174.stdout:3/114: dread d0/d12/f22 [0,4194304] 0 2024-03-21T10:08:47.434 INFO:tasks.workunit.client.1.smithi174.stdout:0/141: read - d9/d21/d23/f25 zero size 2024-03-21T10:08:47.436 INFO:tasks.workunit.client.0.smithi153.stdout:9/341: chown d1/d43/d44/d46/d1c/d24/d41/l6f 47 1 2024-03-21T10:08:47.439 INFO:tasks.workunit.client.1.smithi174.stdout:0/142: write d9/d21/d23/f25 [639187,79472] 0 2024-03-21T10:08:47.442 INFO:tasks.workunit.client.1.smithi174.stdout:9/194: dwrite d2/f3 [0,4194304] 0 2024-03-21T10:08:47.443 INFO:tasks.workunit.client.1.smithi174.stdout:3/115: getdents d0 0 2024-03-21T10:08:47.444 INFO:tasks.workunit.client.0.smithi153.stdout:9/342: creat d1/d43/d44/d46/d1c/d4b/f73 x:0 0 0 2024-03-21T10:08:47.445 INFO:tasks.workunit.client.1.smithi174.stdout:0/143: creat d9/d21/f26 x:0 0 0 2024-03-21T10:08:47.446 INFO:tasks.workunit.client.0.smithi153.stdout:9/343: write d1/f59 [680280,124147] 0 2024-03-21T10:08:47.447 INFO:tasks.workunit.client.1.smithi174.stdout:3/116: symlink d0/d12/d14/d1a/l2b 0 2024-03-21T10:08:47.449 INFO:tasks.workunit.client.0.smithi153.stdout:9/344: dread - d1/d43/d44/f58 zero size 2024-03-21T10:08:47.450 INFO:tasks.workunit.client.1.smithi174.stdout:9/195: link d2/d11/d1d/l21 d2/d11/d1d/d2a/l3d 0 2024-03-21T10:08:47.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:47 smithi174 ceph-mon[37496]: Standby manager daemon smithi174.qpcwat started 2024-03-21T10:08:47.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:47 smithi174 ceph-mon[37496]: from='mgr.? 172.21.15.174:0/3540956956' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi174.qpcwat/crt"}]: dispatch 2024-03-21T10:08:47.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:47 smithi174 ceph-mon[37496]: from='mgr.? 172.21.15.174:0/3540956956' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-21T10:08:47.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:47 smithi174 ceph-mon[37496]: from='mgr.? 172.21.15.174:0/3540956956' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi174.qpcwat/key"}]: dispatch 2024-03-21T10:08:47.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:47 smithi174 ceph-mon[37496]: from='mgr.? 172.21.15.174:0/3540956956' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-21T10:08:47.452 INFO:tasks.workunit.client.1.smithi174.stdout:3/117: chown d0/d12/d14/d1a/c1c 945 1 2024-03-21T10:08:47.452 INFO:tasks.workunit.client.1.smithi174.stdout:9/196: dread - d2/d24/f2f zero size 2024-03-21T10:08:47.452 INFO:tasks.workunit.client.1.smithi174.stdout:0/144: creat d9/f27 x:0 0 0 2024-03-21T10:08:47.453 INFO:tasks.workunit.client.1.smithi174.stdout:0/145: fdatasync d9/f18 0 2024-03-21T10:08:47.455 INFO:tasks.workunit.client.1.smithi174.stdout:9/197: creat d2/d24/f3e x:0 0 0 2024-03-21T10:08:47.456 INFO:tasks.workunit.client.1.smithi174.stdout:0/146: rename d9/ld to d9/d21/l28 0 2024-03-21T10:08:47.457 INFO:tasks.workunit.client.1.smithi174.stdout:9/198: dread - d2/d24/f3e zero size 2024-03-21T10:08:47.458 INFO:tasks.workunit.client.0.smithi153.stdout:5/355: write d4/d5/de/f2a [998494,115044] 0 2024-03-21T10:08:47.461 INFO:tasks.workunit.client.1.smithi174.stdout:9/199: symlink d2/d11/l3f 0 2024-03-21T10:08:47.461 INFO:tasks.workunit.client.0.smithi153.stdout:9/345: creat d1/d43/d44/d46/d4d/f74 x:0 0 0 2024-03-21T10:08:47.463 INFO:tasks.workunit.client.1.smithi174.stdout:0/147: link d9/f1d d9/d21/f29 0 2024-03-21T10:08:47.465 INFO:tasks.workunit.client.1.smithi174.stdout:9/200: dread d2/f1e [0,4194304] 0 2024-03-21T10:08:47.467 INFO:tasks.workunit.client.1.smithi174.stdout:9/201: mknod d2/c40 0 2024-03-21T10:08:47.468 INFO:tasks.workunit.client.0.smithi153.stdout:9/346: dread - d1/d43/d44/d46/d4d/f74 zero size 2024-03-21T10:08:47.469 INFO:tasks.workunit.client.0.smithi153.stdout:5/356: unlink d4/d5/df/f1b 0 2024-03-21T10:08:47.470 INFO:tasks.workunit.client.1.smithi174.stdout:9/202: unlink d2/d11/f29 0 2024-03-21T10:08:47.470 INFO:tasks.workunit.client.0.smithi153.stdout:1/315: dread d6/f17 [0,4194304] 0 2024-03-21T10:08:47.471 INFO:tasks.workunit.client.0.smithi153.stdout:9/347: creat d1/d4/f75 x:0 0 0 2024-03-21T10:08:47.472 INFO:tasks.workunit.client.0.smithi153.stdout:5/357: symlink d4/d5/d16/l71 0 2024-03-21T10:08:47.472 INFO:tasks.workunit.client.0.smithi153.stdout:1/316: fdatasync d6/d32/f37 0 2024-03-21T10:08:47.474 INFO:tasks.workunit.client.0.smithi153.stdout:5/358: mknod d4/d5/de/d44/c72 0 2024-03-21T10:08:47.474 INFO:tasks.workunit.client.1.smithi174.stdout:9/203: rename d2/d24/d2e/f35 to d2/d11/d1d/f41 0 2024-03-21T10:08:47.475 INFO:tasks.workunit.client.1.smithi174.stdout:9/204: chown d2/d11/l3f 5 1 2024-03-21T10:08:47.476 INFO:tasks.workunit.client.0.smithi153.stdout:9/348: rename d1/f27 to d1/d43/d44/d46/d1c/d2e/f76 0 2024-03-21T10:08:47.477 INFO:tasks.workunit.client.0.smithi153.stdout:1/317: link d6/fe d6/d3a/d3c/f64 0 2024-03-21T10:08:47.478 INFO:tasks.workunit.client.1.smithi174.stdout:9/205: read d2/d11/f2c [11547,113924] 0 2024-03-21T10:08:47.479 INFO:tasks.workunit.client.0.smithi153.stdout:9/349: symlink d1/d43/d44/l77 0 2024-03-21T10:08:47.480 INFO:tasks.workunit.client.1.smithi174.stdout:0/148: dwrite d9/f15 [0,4194304] 0 2024-03-21T10:08:47.482 INFO:tasks.workunit.client.0.smithi153.stdout:9/350: mkdir d1/d4e/d68/d78 0 2024-03-21T10:08:47.485 INFO:tasks.workunit.client.1.smithi174.stdout:9/206: rename d2/d24/d2e/c37 to d2/d11/d1d/d2a/c42 0 2024-03-21T10:08:47.487 INFO:tasks.workunit.client.1.smithi174.stdout:0/149: symlink d9/l2a 0 2024-03-21T10:08:47.490 INFO:tasks.workunit.client.0.smithi153.stdout:1/318: read d6/f16 [62928,119705] 0 2024-03-21T10:08:47.494 INFO:tasks.workunit.client.1.smithi174.stdout:0/150: creat d9/d21/f2b x:0 0 0 2024-03-21T10:08:47.494 INFO:tasks.workunit.client.1.smithi174.stdout:9/207: truncate d2/f1e 396934 0 2024-03-21T10:08:47.495 INFO:tasks.workunit.client.0.smithi153.stdout:1/319: symlink d6/d3a/l65 0 2024-03-21T10:08:47.496 INFO:tasks.workunit.client.0.smithi153.stdout:5/359: dwrite d4/d5/df/f40 [0,4194304] 0 2024-03-21T10:08:47.499 INFO:tasks.workunit.client.0.smithi153.stdout:6/546: dread d1f/d2b/d33/d40/f44 [0,4194304] 0 2024-03-21T10:08:47.499 INFO:tasks.workunit.client.0.smithi153.stdout:5/360: chown d4/d5/d16/d24/f53 99080 1 2024-03-21T10:08:47.502 INFO:tasks.workunit.client.0.smithi153.stdout:6/547: write f11 [4703833,86400] 0 2024-03-21T10:08:47.503 INFO:tasks.workunit.client.0.smithi153.stdout:9/351: dwrite d1/d43/d44/d46/f37 [0,4194304] 0 2024-03-21T10:08:47.505 INFO:tasks.workunit.client.1.smithi174.stdout:0/151: dwrite d9/f19 [0,4194304] 0 2024-03-21T10:08:47.507 INFO:tasks.workunit.client.0.smithi153.stdout:6/548: chown d1f/d2b/d33/d40/d4f/c61 149397645 1 2024-03-21T10:08:47.509 INFO:tasks.workunit.client.0.smithi153.stdout:9/352: creat d1/d4e/d68/f79 x:0 0 0 2024-03-21T10:08:47.513 INFO:tasks.workunit.client.0.smithi153.stdout:6/549: truncate d1f/f4e 107677 0 2024-03-21T10:08:47.516 INFO:tasks.workunit.client.1.smithi174.stdout:9/208: chown d2/d11/d1d/d2a/c42 582 1 2024-03-21T10:08:47.517 INFO:tasks.workunit.client.0.smithi153.stdout:1/320: mkdir d6/d3a/d66 0 2024-03-21T10:08:47.518 INFO:tasks.workunit.client.0.smithi153.stdout:6/550: unlink d1f/d2f/c5a 0 2024-03-21T10:08:47.519 INFO:tasks.workunit.client.0.smithi153.stdout:5/361: getdents d4/d8/d5a/d5d 0 2024-03-21T10:08:47.521 INFO:tasks.workunit.client.1.smithi174.stdout:0/152: dread f4 [0,4194304] 0 2024-03-21T10:08:47.522 INFO:tasks.workunit.client.1.smithi174.stdout:0/153: stat d9/d1c/f20 0 2024-03-21T10:08:47.523 INFO:tasks.workunit.client.1.smithi174.stdout:0/154: write d9/f18 [762543,82989] 0 2024-03-21T10:08:47.525 INFO:tasks.workunit.client.0.smithi153.stdout:5/362: creat d4/d8/d5c/f73 x:0 0 0 2024-03-21T10:08:47.527 INFO:tasks.workunit.client.0.smithi153.stdout:1/321: creat d6/d3a/d3c/d50/f67 x:0 0 0 2024-03-21T10:08:47.528 INFO:tasks.workunit.client.0.smithi153.stdout:6/551: link d1f/f25 d1f/d2b/d33/d40/d51/fc5 0 2024-03-21T10:08:47.528 INFO:tasks.workunit.client.0.smithi153.stdout:5/363: stat d4/d5/df/d5e/c65 0 2024-03-21T10:08:47.530 INFO:tasks.workunit.client.0.smithi153.stdout:6/552: unlink d1f/f23 0 2024-03-21T10:08:47.532 INFO:tasks.workunit.client.0.smithi153.stdout:5/364: unlink d4/c47 0 2024-03-21T10:08:47.533 INFO:tasks.workunit.client.0.smithi153.stdout:5/365: write d4/d5/f1a [3268314,27093] 0 2024-03-21T10:08:47.533 INFO:tasks.workunit.client.0.smithi153.stdout:1/322: unlink d6/d13/c21 0 2024-03-21T10:08:47.535 INFO:tasks.workunit.client.0.smithi153.stdout:6/553: fsync d1f/d2b/f63 0 2024-03-21T10:08:47.539 INFO:tasks.workunit.client.0.smithi153.stdout:1/323: readlink d6/d13/l24 0 2024-03-21T10:08:47.540 INFO:tasks.workunit.client.0.smithi153.stdout:6/554: creat d1f/d2b/fc6 x:0 0 0 2024-03-21T10:08:47.546 INFO:tasks.workunit.client.1.smithi174.stdout:0/155: dwrite d9/d21/f29 [0,4194304] 0 2024-03-21T10:08:47.546 INFO:tasks.workunit.client.1.smithi174.stdout:0/156: chown f7 1631467663 1 2024-03-21T10:08:47.548 INFO:tasks.workunit.client.0.smithi153.stdout:6/555: write d1f/d2b/d33/d40/f5d [1247353,109569] 0 2024-03-21T10:08:47.550 INFO:tasks.workunit.client.1.smithi174.stdout:0/157: mknod d9/c2c 0 2024-03-21T10:08:47.552 INFO:tasks.workunit.client.0.smithi153.stdout:5/366: truncate d4/d5/d16/d24/f25 3149890 0 2024-03-21T10:08:47.552 INFO:tasks.workunit.client.0.smithi153.stdout:1/324: write d6/d10/d2c/f47 [1114445,54961] 0 2024-03-21T10:08:47.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:47 smithi153 ceph-mon[29117]: Standby manager daemon smithi174.qpcwat started 2024-03-21T10:08:47.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:47 smithi153 ceph-mon[29117]: from='mgr.? 172.21.15.174:0/3540956956' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi174.qpcwat/crt"}]: dispatch 2024-03-21T10:08:47.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:47 smithi153 ceph-mon[29117]: from='mgr.? 172.21.15.174:0/3540956956' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-21T10:08:47.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:47 smithi153 ceph-mon[29117]: from='mgr.? 172.21.15.174:0/3540956956' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi174.qpcwat/key"}]: dispatch 2024-03-21T10:08:47.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:47 smithi153 ceph-mon[29117]: from='mgr.? 172.21.15.174:0/3540956956' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-21T10:08:47.565 INFO:tasks.workunit.client.1.smithi174.stdout:0/158: dwrite d9/f19 [8388608,4194304] 0 2024-03-21T10:08:47.571 INFO:tasks.workunit.client.1.smithi174.stdout:0/159: creat d9/d21/d23/f2d x:0 0 0 2024-03-21T10:08:47.580 INFO:tasks.workunit.client.1.smithi174.stdout:0/160: getdents d9 0 2024-03-21T10:08:47.581 INFO:tasks.workunit.client.0.smithi153.stdout:6/556: dread d1f/d2b/d33/d40/d4f/d66/f7c [0,4194304] 0 2024-03-21T10:08:47.582 INFO:tasks.workunit.client.0.smithi153.stdout:5/367: dread d4/d5/f7 [0,4194304] 0 2024-03-21T10:08:47.586 INFO:tasks.workunit.client.0.smithi153.stdout:5/368: read d4/d5/df/f30 [384266,127959] 0 2024-03-21T10:08:47.586 INFO:tasks.workunit.client.1.smithi174.stdout:5/143: sync 2024-03-21T10:08:47.586 INFO:tasks.workunit.client.1.smithi174.stdout:4/200: sync 2024-03-21T10:08:47.586 INFO:tasks.workunit.client.1.smithi174.stdout:8/99: sync 2024-03-21T10:08:47.586 INFO:tasks.workunit.client.1.smithi174.stdout:1/178: sync 2024-03-21T10:08:47.587 INFO:tasks.workunit.client.0.smithi153.stdout:5/369: creat d4/d5/d16/d24/f74 x:0 0 0 2024-03-21T10:08:47.588 INFO:tasks.workunit.client.1.smithi174.stdout:8/100: unlink l6 0 2024-03-21T10:08:47.590 INFO:tasks.workunit.client.0.smithi153.stdout:5/370: fdatasync d4/d8/d3b/d56/f62 0 2024-03-21T10:08:47.591 INFO:tasks.workunit.client.1.smithi174.stdout:8/101: chown d9/fc 154695 1 2024-03-21T10:08:47.594 INFO:tasks.workunit.client.1.smithi174.stdout:5/144: mkdir d1/d16/d2e/d31 0 2024-03-21T10:08:47.594 INFO:tasks.workunit.client.1.smithi174.stdout:1/179: creat d4/d7/d20/f42 x:0 0 0 2024-03-21T10:08:47.599 INFO:tasks.workunit.client.1.smithi174.stdout:0/161: dwrite d9/d21/f29 [0,4194304] 0 2024-03-21T10:08:47.600 INFO:tasks.workunit.client.1.smithi174.stdout:5/145: mkdir d1/d2f/d32 0 2024-03-21T10:08:47.607 INFO:tasks.workunit.client.1.smithi174.stdout:0/162: unlink d9/le 0 2024-03-21T10:08:47.608 INFO:tasks.workunit.client.1.smithi174.stdout:4/201: dwrite d4/d21/d31/d3a/f3d [0,4194304] 0 2024-03-21T10:08:47.610 INFO:tasks.workunit.client.1.smithi174.stdout:4/202: fdatasync d4/fe 0 2024-03-21T10:08:47.612 INFO:tasks.workunit.client.1.smithi174.stdout:8/102: truncate d9/fc 2687534 0 2024-03-21T10:08:47.620 INFO:tasks.workunit.client.1.smithi174.stdout:5/146: creat d1/d16/d2e/d31/f33 x:0 0 0 2024-03-21T10:08:47.621 INFO:tasks.workunit.client.1.smithi174.stdout:0/163: write d9/d21/d23/f2d [193712,48983] 0 2024-03-21T10:08:47.622 INFO:tasks.workunit.client.1.smithi174.stdout:1/180: dwrite d4/d7/f29 [0,4194304] 0 2024-03-21T10:08:47.622 INFO:tasks.workunit.client.1.smithi174.stdout:5/147: readlink d1/d16/l17 0 2024-03-21T10:08:47.624 INFO:tasks.workunit.client.1.smithi174.stdout:1/181: readlink d4/l27 0 2024-03-21T10:08:47.625 INFO:tasks.workunit.client.1.smithi174.stdout:1/182: chown d4/d7/d33 0 1 2024-03-21T10:08:47.626 INFO:tasks.workunit.client.1.smithi174.stdout:0/164: mknod d9/d21/c2e 0 2024-03-21T10:08:47.630 INFO:tasks.workunit.client.1.smithi174.stdout:5/148: creat d1/d16/d1c/f34 x:0 0 0 2024-03-21T10:08:47.631 INFO:tasks.workunit.client.1.smithi174.stdout:1/183: write d4/d5/f2a [727684,45298] 0 2024-03-21T10:08:47.633 INFO:tasks.workunit.client.1.smithi174.stdout:5/149: readlink d1/d16/l19 0 2024-03-21T10:08:47.633 INFO:tasks.workunit.client.1.smithi174.stdout:0/165: chown d9/d1c/f20 57 1 2024-03-21T10:08:47.636 INFO:tasks.workunit.client.1.smithi174.stdout:1/184: write d4/d7/f2e [136473,68258] 0 2024-03-21T10:08:47.644 INFO:tasks.workunit.client.1.smithi174.stdout:8/103: dwrite d9/d17/d19/f1f [0,4194304] 0 2024-03-21T10:08:47.648 INFO:tasks.workunit.client.1.smithi174.stdout:8/104: truncate d9/d17/f20 995039 0 2024-03-21T10:08:47.660 INFO:tasks.workunit.client.1.smithi174.stdout:8/105: unlink d9/d17/d19/f1f 0 2024-03-21T10:08:47.664 INFO:tasks.workunit.client.1.smithi174.stdout:5/150: dwrite d1/fc [0,4194304] 0 2024-03-21T10:08:47.666 INFO:tasks.workunit.client.1.smithi174.stdout:0/166: dwrite d9/f1d [0,4194304] 0 2024-03-21T10:08:47.669 INFO:tasks.workunit.client.1.smithi174.stdout:2/182: sync 2024-03-21T10:08:47.669 INFO:tasks.workunit.client.1.smithi174.stdout:6/114: sync 2024-03-21T10:08:47.669 INFO:tasks.workunit.client.1.smithi174.stdout:7/132: sync 2024-03-21T10:08:47.672 INFO:tasks.workunit.client.1.smithi174.stdout:1/185: dwrite d4/f19 [0,4194304] 0 2024-03-21T10:08:47.672 INFO:tasks.workunit.client.1.smithi174.stdout:8/106: creat d9/d17/d19/f23 x:0 0 0 2024-03-21T10:08:47.674 INFO:tasks.workunit.client.1.smithi174.stdout:5/151: creat d1/d16/d2e/d31/f35 x:0 0 0 2024-03-21T10:08:47.677 INFO:tasks.workunit.client.1.smithi174.stdout:1/186: stat d4/d7/d15/f38 0 2024-03-21T10:08:47.680 INFO:tasks.workunit.client.1.smithi174.stdout:0/167: truncate d9/f18 1516825 0 2024-03-21T10:08:47.680 INFO:tasks.workunit.client.1.smithi174.stdout:8/107: chown d9/f1c 463674 1 2024-03-21T10:08:47.683 INFO:tasks.workunit.client.1.smithi174.stdout:6/115: creat d0/d3/f26 x:0 0 0 2024-03-21T10:08:47.685 INFO:tasks.workunit.client.1.smithi174.stdout:5/152: write d1/d16/d2e/d31/f33 [569400,105643] 0 2024-03-21T10:08:47.686 INFO:tasks.workunit.client.0.smithi153.stdout:7/278: sync 2024-03-21T10:08:47.686 INFO:tasks.workunit.client.0.smithi153.stdout:0/266: sync 2024-03-21T10:08:47.686 INFO:tasks.workunit.client.1.smithi174.stdout:1/187: truncate d4/d7/d15/f1f 161444 0 2024-03-21T10:08:47.690 INFO:tasks.workunit.client.1.smithi174.stdout:2/183: link d5/d25/d27/f3c d5/d25/d27/d2d/f41 0 2024-03-21T10:08:47.692 INFO:tasks.workunit.client.1.smithi174.stdout:1/188: dread - d4/d7/d20/f42 zero size 2024-03-21T10:08:47.695 INFO:tasks.workunit.client.1.smithi174.stdout:4/203: dread d4/d21/d24/f42 [0,4194304] 0 2024-03-21T10:08:47.697 INFO:tasks.workunit.client.1.smithi174.stdout:6/116: creat d0/d3/d5/df/d16/f27 x:0 0 0 2024-03-21T10:08:47.699 INFO:tasks.workunit.client.1.smithi174.stdout:9/209: sync 2024-03-21T10:08:47.699 INFO:tasks.workunit.client.1.smithi174.stdout:3/118: sync 2024-03-21T10:08:47.703 INFO:tasks.workunit.client.1.smithi174.stdout:8/108: getdents d9 0 2024-03-21T10:08:47.703 INFO:tasks.workunit.client.0.smithi153.stdout:0/267: write d4/f23 [1285332,23210] 0 2024-03-21T10:08:47.703 INFO:tasks.workunit.client.0.smithi153.stdout:5/371: getdents d4/d5/df 0 2024-03-21T10:08:47.703 INFO:tasks.workunit.client.0.smithi153.stdout:0/268: symlink d4/d16/d3e/l53 0 2024-03-21T10:08:47.706 INFO:tasks.workunit.client.1.smithi174.stdout:8/109: fsync d9/fe 0 2024-03-21T10:08:47.706 INFO:tasks.workunit.client.1.smithi174.stdout:2/184: rename d5/l1f to d5/d25/d27/d2d/l42 0 2024-03-21T10:08:47.706 INFO:tasks.workunit.client.1.smithi174.stdout:7/133: dwrite d0/f27 [0,4194304] 0 2024-03-21T10:08:47.707 INFO:tasks.workunit.client.1.smithi174.stdout:3/119: rmdir d0 39 2024-03-21T10:08:47.707 INFO:tasks.workunit.client.0.smithi153.stdout:0/269: write d4/d4c/f4e [1181201,60538] 0 2024-03-21T10:08:47.707 INFO:tasks.workunit.client.0.smithi153.stdout:5/372: creat d4/f75 x:0 0 0 2024-03-21T10:08:47.709 INFO:tasks.workunit.client.0.smithi153.stdout:0/270: fsync d4/f23 0 2024-03-21T10:08:47.714 INFO:tasks.workunit.client.1.smithi174.stdout:7/134: dread - d0/f2b zero size 2024-03-21T10:08:47.717 INFO:tasks.workunit.client.1.smithi174.stdout:0/168: write f4 [450163,34652] 0 2024-03-21T10:08:47.718 INFO:tasks.workunit.client.0.smithi153.stdout:6/557: dread d1f/d2b/fb5 [0,4194304] 0 2024-03-21T10:08:47.721 INFO:tasks.workunit.client.1.smithi174.stdout:1/189: rename d4/d7/d20 to d4/d7/d33/d43 0 2024-03-21T10:08:47.724 INFO:tasks.workunit.client.1.smithi174.stdout:2/185: creat d5/d25/d27/d2d/f43 x:0 0 0 2024-03-21T10:08:47.727 INFO:tasks.workunit.client.0.smithi153.stdout:7/279: dwrite d4/d14/d2f/f45 [0,4194304] 0 2024-03-21T10:08:47.727 INFO:tasks.workunit.client.1.smithi174.stdout:7/135: write d0/d4/fd [522573,78748] 0 2024-03-21T10:08:47.730 INFO:tasks.workunit.client.1.smithi174.stdout:9/210: rename d2/d11/l3f to d2/d11/d1d/l43 0 2024-03-21T10:08:47.730 INFO:tasks.workunit.client.1.smithi174.stdout:1/190: dread - d4/d7/f21 zero size 2024-03-21T10:08:47.733 INFO:tasks.workunit.client.0.smithi153.stdout:5/373: dwrite d4/d5/de/d2f/f41 [0,4194304] 0 2024-03-21T10:08:47.736 INFO:tasks.workunit.client.0.smithi153.stdout:7/280: rename d4/f44 to d4/d11/d24/f58 0 2024-03-21T10:08:47.736 INFO:tasks.workunit.client.1.smithi174.stdout:0/169: unlink d9/f27 0 2024-03-21T10:08:47.737 INFO:tasks.workunit.client.1.smithi174.stdout:5/153: dwrite d1/d16/f1a [0,4194304] 0 2024-03-21T10:08:47.740 INFO:tasks.workunit.client.0.smithi153.stdout:0/271: write d4/db/f13 [1305988,99358] 0 2024-03-21T10:08:47.746 INFO:tasks.workunit.client.0.smithi153.stdout:7/281: link d4/d14/d16/c23 d4/d2a/d40/c59 0 2024-03-21T10:08:47.747 INFO:tasks.workunit.client.1.smithi174.stdout:7/136: write d0/d4/f2d [390758,29746] 0 2024-03-21T10:08:47.751 INFO:tasks.workunit.client.1.smithi174.stdout:6/117: dwrite d0/d3/d5/df/d16/f1a [0,4194304] 0 2024-03-21T10:08:47.753 INFO:tasks.workunit.client.0.smithi153.stdout:7/282: rename d4/d2a/c2e to d4/d19/d4d/c5a 0 2024-03-21T10:08:47.753 INFO:tasks.workunit.client.1.smithi174.stdout:9/211: chown d2/d11/c12 28 1 2024-03-21T10:08:47.754 INFO:tasks.workunit.client.0.smithi153.stdout:5/374: dread d4/d8/f2d [0,4194304] 0 2024-03-21T10:08:47.755 INFO:tasks.workunit.client.1.smithi174.stdout:1/191: rmdir d4/d5 39 2024-03-21T10:08:47.757 INFO:tasks.workunit.client.0.smithi153.stdout:5/375: dread - d4/f75 zero size 2024-03-21T10:08:47.761 INFO:tasks.workunit.client.1.smithi174.stdout:5/154: creat d1/d16/d2e/d31/f36 x:0 0 0 2024-03-21T10:08:47.763 INFO:tasks.workunit.client.0.smithi153.stdout:6/558: dread f11 [0,4194304] 0 2024-03-21T10:08:47.765 INFO:tasks.workunit.client.0.smithi153.stdout:0/272: dwrite d4/db/f1b [0,4194304] 0 2024-03-21T10:08:47.767 INFO:tasks.workunit.client.1.smithi174.stdout:8/110: dwrite f8 [0,4194304] 0 2024-03-21T10:08:47.772 INFO:tasks.workunit.client.0.smithi153.stdout:6/559: rename d1f/d2b/f38 to d1f/d2b/d33/d40/d4f/fc7 0 2024-03-21T10:08:47.773 INFO:tasks.workunit.client.0.smithi153.stdout:0/273: mknod d4/d16/d3e/c54 0 2024-03-21T10:08:47.775 INFO:tasks.workunit.client.0.smithi153.stdout:0/274: mknod d4/db/c55 0 2024-03-21T10:08:47.776 INFO:tasks.workunit.client.1.smithi174.stdout:3/120: dwrite d0/d12/d14/d10/f1b [0,4194304] 0 2024-03-21T10:08:47.776 INFO:tasks.workunit.client.1.smithi174.stdout:9/212: unlink d2/d24/c2d 0 2024-03-21T10:08:47.777 INFO:tasks.workunit.client.0.smithi153.stdout:6/560: creat d1f/d2b/db3/fc8 x:0 0 0 2024-03-21T10:08:47.778 INFO:tasks.workunit.client.1.smithi174.stdout:1/192: mknod d4/d7/d32/c44 0 2024-03-21T10:08:47.780 INFO:tasks.workunit.client.1.smithi174.stdout:2/186: dwrite d5/d25/d27/f2a [0,4194304] 0 2024-03-21T10:08:47.780 INFO:tasks.workunit.client.0.smithi153.stdout:6/561: write d1f/d2b/d33/d40/d4f/f77 [4727611,118596] 0 2024-03-21T10:08:47.781 INFO:tasks.workunit.client.0.smithi153.stdout:0/275: rename d4/d16/d1f/d22/c26 to d4/d16/d1f/d22/c56 0 2024-03-21T10:08:47.785 INFO:tasks.workunit.client.1.smithi174.stdout:7/137: rename d0/d4/d10/d1b/l2a to d0/d4/d10/d1b/l30 0 2024-03-21T10:08:47.785 INFO:tasks.workunit.client.1.smithi174.stdout:3/121: write d0/d12/d14/d1a/f1e [157112,28855] 0 2024-03-21T10:08:47.785 INFO:tasks.workunit.client.1.smithi174.stdout:5/155: rename d1/d2f/d32 to d1/d2f/d37 0 2024-03-21T10:08:47.788 INFO:tasks.workunit.client.0.smithi153.stdout:0/276: symlink d4/d16/l57 0 2024-03-21T10:08:47.788 INFO:tasks.workunit.client.0.smithi153.stdout:6/562: link d1f/d2b/d33/d40/d7b/c7f d1f/d2f/cc9 0 2024-03-21T10:08:47.789 INFO:tasks.workunit.client.1.smithi174.stdout:6/118: link d0/d3/l7 d0/d3/d5/l28 0 2024-03-21T10:08:47.789 INFO:tasks.workunit.client.0.smithi153.stdout:6/563: readlink d1f/d2f/d59/l87 0 2024-03-21T10:08:47.790 INFO:tasks.workunit.client.1.smithi174.stdout:2/187: write d5/d8/f23 [5216218,111968] 0 2024-03-21T10:08:47.790 INFO:tasks.workunit.client.1.smithi174.stdout:1/193: truncate d4/d7/d33/f3d 38066 0 2024-03-21T10:08:47.792 INFO:tasks.workunit.client.1.smithi174.stdout:0/170: dwrite f7 [0,4194304] 0 2024-03-21T10:08:47.792 INFO:tasks.workunit.client.0.smithi153.stdout:6/564: stat d1f/d2b/d33/d40/d5f/f6d 0 2024-03-21T10:08:47.796 INFO:tasks.workunit.client.0.smithi153.stdout:0/277: mkdir d4/d16/d19/d58 0 2024-03-21T10:08:47.798 INFO:tasks.workunit.client.1.smithi174.stdout:6/119: chown d0/d3/d5/df/f22 15 1 2024-03-21T10:08:47.798 INFO:tasks.workunit.client.1.smithi174.stdout:0/171: dread - d9/d21/f26 zero size 2024-03-21T10:08:47.800 INFO:tasks.workunit.client.0.smithi153.stdout:7/283: truncate d4/d2a/d40/f4c 581543 0 2024-03-21T10:08:47.801 INFO:tasks.workunit.client.1.smithi174.stdout:4/204: sync 2024-03-21T10:08:47.801 INFO:tasks.workunit.client.0.smithi153.stdout:5/376: truncate d4/d8/f2d 1725144 0 2024-03-21T10:08:47.803 INFO:tasks.workunit.client.1.smithi174.stdout:6/120: chown d0/d3/d5/df/d16/c1f 29558 1 2024-03-21T10:08:47.804 INFO:tasks.workunit.client.1.smithi174.stdout:9/213: link d2/d11/d1d/f41 d2/f44 0 2024-03-21T10:08:47.806 INFO:tasks.workunit.client.1.smithi174.stdout:3/122: rename d0/d12/f22 to d0/d12/d14/d1a/d27/f2c 0 2024-03-21T10:08:47.807 INFO:tasks.workunit.client.1.smithi174.stdout:9/214: write d2/f1b [971877,79390] 0 2024-03-21T10:08:47.809 INFO:tasks.workunit.client.1.smithi174.stdout:0/172: unlink d9/f18 0 2024-03-21T10:08:47.809 INFO:tasks.workunit.client.0.smithi153.stdout:7/284: creat d4/d2a/f5b x:0 0 0 2024-03-21T10:08:47.810 INFO:tasks.workunit.client.1.smithi174.stdout:7/138: link d0/c24 d0/c31 0 2024-03-21T10:08:47.811 INFO:tasks.workunit.client.1.smithi174.stdout:9/215: dread - d2/d11/d1d/f41 zero size 2024-03-21T10:08:47.818 INFO:tasks.workunit.client.1.smithi174.stdout:4/205: creat d4/d21/d31/f4a x:0 0 0 2024-03-21T10:08:47.818 INFO:tasks.workunit.client.0.smithi153.stdout:7/285: link d4/d1b/d20/c4a d4/d14/d2f/c5c 0 2024-03-21T10:08:47.820 INFO:tasks.workunit.client.0.smithi153.stdout:7/286: chown d4/d2a/d40 0 1 2024-03-21T10:08:47.820 INFO:tasks.workunit.client.1.smithi174.stdout:7/139: chown d0/d4/d10/f21 19625933 1 2024-03-21T10:08:47.821 INFO:tasks.workunit.client.0.smithi153.stdout:5/377: write f1 [2275913,19773] 0 2024-03-21T10:08:47.822 INFO:tasks.workunit.client.0.smithi153.stdout:5/378: chown d4/d8/f6c 6 1 2024-03-21T10:08:47.825 INFO:tasks.workunit.client.1.smithi174.stdout:9/216: write d2/d11/d1d/f33 [770005,99880] 0 2024-03-21T10:08:47.826 INFO:tasks.workunit.client.0.smithi153.stdout:5/379: mknod d4/d5/df/c76 0 2024-03-21T10:08:47.829 INFO:tasks.workunit.client.1.smithi174.stdout:0/173: mknod d9/d1c/c2f 0 2024-03-21T10:08:47.829 INFO:tasks.workunit.client.1.smithi174.stdout:4/206: write d4/d21/d31/f4a [927440,127754] 0 2024-03-21T10:08:47.836 INFO:tasks.workunit.client.1.smithi174.stdout:8/111: truncate d9/f1c 2097587 0 2024-03-21T10:08:47.836 INFO:tasks.workunit.client.1.smithi174.stdout:5/156: dwrite d1/d16/f27 [0,4194304] 0 2024-03-21T10:08:47.836 INFO:tasks.workunit.client.1.smithi174.stdout:2/188: dwrite d5/f12 [0,4194304] 0 2024-03-21T10:08:47.840 INFO:tasks.workunit.client.1.smithi174.stdout:7/140: creat d0/f32 x:0 0 0 2024-03-21T10:08:47.845 INFO:tasks.workunit.client.1.smithi174.stdout:1/194: dwrite d4/d7/d33/f36 [0,4194304] 0 2024-03-21T10:08:47.849 INFO:tasks.workunit.client.1.smithi174.stdout:5/157: read d1/d16/d1c/f23 [376809,71835] 0 2024-03-21T10:08:47.851 INFO:tasks.workunit.client.0.smithi153.stdout:7/287: dwrite d4/d11/d24/f35 [0,4194304] 0 2024-03-21T10:08:47.854 INFO:tasks.workunit.client.1.smithi174.stdout:5/158: truncate d1/d16/d1c/f25 979025 0 2024-03-21T10:08:47.857 INFO:tasks.workunit.client.1.smithi174.stdout:0/174: write d9/f19 [7337750,71857] 0 2024-03-21T10:08:47.858 INFO:tasks.workunit.client.0.smithi153.stdout:7/288: creat d4/d1b/f5d x:0 0 0 2024-03-21T10:08:47.860 INFO:tasks.workunit.client.1.smithi174.stdout:6/121: link d0/d3/d5/d6/l11 d0/d3/d20/l29 0 2024-03-21T10:08:47.861 INFO:tasks.workunit.client.1.smithi174.stdout:8/112: dwrite d9/f14 [0,4194304] 0 2024-03-21T10:08:47.864 INFO:tasks.workunit.client.0.smithi153.stdout:7/289: creat d4/d14/d2f/f5e x:0 0 0 2024-03-21T10:08:47.867 INFO:tasks.workunit.client.0.smithi153.stdout:7/290: write d4/d14/d16/f39 [250919,84039] 0 2024-03-21T10:08:47.869 INFO:tasks.workunit.client.1.smithi174.stdout:2/189: mkdir d5/d25/d27/d44 0 2024-03-21T10:08:47.870 INFO:tasks.workunit.client.0.smithi153.stdout:0/278: dread d4/d16/d1f/d22/f47 [0,4194304] 0 2024-03-21T10:08:47.872 INFO:tasks.workunit.client.1.smithi174.stdout:1/195: symlink d4/d7/d15/l45 0 2024-03-21T10:08:47.874 INFO:tasks.workunit.client.0.smithi153.stdout:7/291: mknod d4/d1b/c5f 0 2024-03-21T10:08:47.874 INFO:tasks.workunit.client.0.smithi153.stdout:5/380: dread d4/d8/f2d [0,4194304] 0 2024-03-21T10:08:47.877 INFO:tasks.workunit.client.0.smithi153.stdout:0/279: symlink d4/d16/l59 0 2024-03-21T10:08:47.877 INFO:tasks.workunit.client.1.smithi174.stdout:5/159: chown d1/c9 0 1 2024-03-21T10:08:47.878 INFO:tasks.workunit.client.0.smithi153.stdout:7/292: mkdir d4/d2a/d3b/d60 0 2024-03-21T10:08:47.879 INFO:tasks.workunit.client.1.smithi174.stdout:0/175: dread f4 [0,4194304] 0 2024-03-21T10:08:47.879 INFO:tasks.workunit.client.0.smithi153.stdout:5/381: creat d4/d5/df/d67/d6a/f77 x:0 0 0 2024-03-21T10:08:47.880 INFO:tasks.workunit.client.1.smithi174.stdout:5/160: dread - d1/d16/d2e/d31/f36 zero size 2024-03-21T10:08:47.882 INFO:tasks.workunit.client.1.smithi174.stdout:3/123: truncate d0/f3 380776 0 2024-03-21T10:08:47.884 INFO:tasks.workunit.client.0.smithi153.stdout:2/415: sync 2024-03-21T10:08:47.884 INFO:tasks.workunit.client.0.smithi153.stdout:2/416: readlink dd/d15/d18/d2c/d4b/d56/l5f 0 2024-03-21T10:08:47.884 INFO:tasks.workunit.client.0.smithi153.stdout:5/382: mknod d4/d8/d3b/c78 0 2024-03-21T10:08:47.887 INFO:tasks.workunit.client.0.smithi153.stdout:5/383: mknod d4/d5/d16/c79 0 2024-03-21T10:08:47.888 INFO:tasks.workunit.client.1.smithi174.stdout:8/113: creat d9/d17/d19/f24 x:0 0 0 2024-03-21T10:08:47.890 INFO:tasks.workunit.client.0.smithi153.stdout:2/417: rename dd/d15/d18/d2c/d4b/f81 to dd/f90 0 2024-03-21T10:08:47.890 INFO:tasks.workunit.client.1.smithi174.stdout:9/217: rename d2/d24/f3e to d2/f45 0 2024-03-21T10:08:47.891 INFO:tasks.workunit.client.1.smithi174.stdout:6/122: chown d0/d3/d5/d6/lc 31204 1 2024-03-21T10:08:47.891 INFO:tasks.workunit.client.0.smithi153.stdout:5/384: symlink d4/d5/df/d67/d6a/l7a 0 2024-03-21T10:08:47.893 INFO:tasks.workunit.client.1.smithi174.stdout:2/190: mkdir d5/d25/d27/d2d/d45 0 2024-03-21T10:08:47.894 INFO:tasks.workunit.client.1.smithi174.stdout:5/161: unlink d1/c9 0 2024-03-21T10:08:47.895 INFO:tasks.workunit.client.1.smithi174.stdout:9/218: fsync d2/d24/f2f 0 2024-03-21T10:08:47.896 INFO:tasks.workunit.client.0.smithi153.stdout:7/293: dwrite d4/fa [0,4194304] 0 2024-03-21T10:08:47.902 INFO:tasks.workunit.client.1.smithi174.stdout:2/191: chown d5/d25/d27/d2e/f3f 7651163 1 2024-03-21T10:08:47.902 INFO:tasks.workunit.client.0.smithi153.stdout:7/294: chown d4/d2a/d40/d50 247273 1 2024-03-21T10:08:47.902 INFO:tasks.workunit.client.0.smithi153.stdout:0/280: write d4/f6 [5625601,15752] 0 2024-03-21T10:08:47.904 INFO:tasks.workunit.client.1.smithi174.stdout:6/123: fdatasync d0/d3/f8 0 2024-03-21T10:08:47.904 INFO:tasks.workunit.client.1.smithi174.stdout:4/207: truncate d4/f20 94235 0 2024-03-21T10:08:47.904 INFO:tasks.workunit.client.0.smithi153.stdout:5/385: write d4/d5/f7 [964438,31181] 0 2024-03-21T10:08:47.905 INFO:tasks.workunit.client.0.smithi153.stdout:0/281: symlink d4/d16/d27/l5a 0 2024-03-21T10:08:47.907 INFO:tasks.workunit.client.1.smithi174.stdout:1/196: dread d4/d5/f8 [0,4194304] 0 2024-03-21T10:08:47.908 INFO:tasks.workunit.client.1.smithi174.stdout:5/162: symlink d1/d16/d2e/l38 0 2024-03-21T10:08:47.910 INFO:tasks.workunit.client.0.smithi153.stdout:2/418: dwrite dd/d15/d18/d2c/f72 [0,4194304] 0 2024-03-21T10:08:47.913 INFO:tasks.workunit.client.1.smithi174.stdout:0/176: rename d9/d21/l28 to d9/d1c/l30 0 2024-03-21T10:08:47.914 INFO:tasks.workunit.client.0.smithi153.stdout:0/282: read d4/f6 [3086360,39736] 0 2024-03-21T10:08:47.914 INFO:tasks.workunit.client.0.smithi153.stdout:7/295: link d4/d11/d24/f54 d4/d11/f61 0 2024-03-21T10:08:47.915 INFO:tasks.workunit.client.1.smithi174.stdout:3/124: dwrite d0/d12/d14/d10/f1f [0,4194304] 0 2024-03-21T10:08:47.915 INFO:tasks.workunit.client.1.smithi174.stdout:9/219: symlink d2/l46 0 2024-03-21T10:08:47.918 INFO:tasks.workunit.client.0.smithi153.stdout:2/419: rmdir dd/d15/d5e 39 2024-03-21T10:08:47.918 INFO:tasks.workunit.client.0.smithi153.stdout:7/296: write d4/d14/f53 [451205,48059] 0 2024-03-21T10:08:47.918 INFO:tasks.workunit.client.0.smithi153.stdout:0/283: mkdir d4/d16/d3e/d5b 0 2024-03-21T10:08:47.918 INFO:tasks.workunit.client.0.smithi153.stdout:2/420: chown dd/d15/d18/d2c/l6e 4607 1 2024-03-21T10:08:47.919 INFO:tasks.workunit.client.1.smithi174.stdout:5/163: write d1/d16/d2e/d31/f36 [199222,61733] 0 2024-03-21T10:08:47.920 INFO:tasks.workunit.client.1.smithi174.stdout:4/208: truncate d4/d19/f1c 145948 0 2024-03-21T10:08:47.920 INFO:tasks.workunit.client.0.smithi153.stdout:7/297: write d4/d14/d2f/f45 [3295555,53057] 0 2024-03-21T10:08:47.921 INFO:tasks.workunit.client.0.smithi153.stdout:2/421: chown dd/d3b/f5b 1595348 1 2024-03-21T10:08:47.924 INFO:tasks.workunit.client.0.smithi153.stdout:0/284: creat d4/d16/d27/f5c x:0 0 0 2024-03-21T10:08:47.925 INFO:tasks.workunit.client.1.smithi174.stdout:7/141: dwrite d0/d4/f12 [4194304,4194304] 0 2024-03-21T10:08:47.926 INFO:tasks.workunit.client.1.smithi174.stdout:1/197: symlink d4/d7/d15/l46 0 2024-03-21T10:08:47.928 INFO:tasks.workunit.client.0.smithi153.stdout:7/298: write d4/d2a/f2d [1167812,38457] 0 2024-03-21T10:08:47.929 INFO:tasks.workunit.client.0.smithi153.stdout:2/422: truncate dd/d15/d18/d2c/d33/f88 572237 0 2024-03-21T10:08:47.931 INFO:tasks.workunit.client.0.smithi153.stdout:0/285: chown d4/d16/d19/d30/l3d 3221 1 2024-03-21T10:08:47.932 INFO:tasks.workunit.client.0.smithi153.stdout:7/299: dread - d4/d1b/f5d zero size 2024-03-21T10:08:47.932 INFO:tasks.workunit.client.1.smithi174.stdout:3/125: unlink d0/d12/d14/d10/l25 0 2024-03-21T10:08:47.934 INFO:tasks.workunit.client.1.smithi174.stdout:1/198: stat d4/d2d/l37 0 2024-03-21T10:08:47.934 INFO:tasks.workunit.client.0.smithi153.stdout:5/386: dwrite d4/d5/d16/f19 [0,4194304] 0 2024-03-21T10:08:47.934 INFO:tasks.workunit.client.0.smithi153.stdout:7/300: chown d4/d14/d2f/c43 27594651 1 2024-03-21T10:08:47.935 INFO:tasks.workunit.client.0.smithi153.stdout:2/423: write dd/d15/d18/d2c/f3f [518362,80199] 0 2024-03-21T10:08:47.936 INFO:tasks.workunit.client.1.smithi174.stdout:5/164: write d1/f22 [247057,27437] 0 2024-03-21T10:08:47.939 INFO:tasks.workunit.client.0.smithi153.stdout:7/301: truncate d4/d14/f4b 809265 0 2024-03-21T10:08:47.940 INFO:tasks.workunit.client.1.smithi174.stdout:8/114: dwrite f1 [0,4194304] 0 2024-03-21T10:08:47.944 INFO:tasks.workunit.client.0.smithi153.stdout:2/424: mkdir dd/d15/d18/d2c/d4b/d56/d63/d8d/d8e/d91 0 2024-03-21T10:08:47.945 INFO:tasks.workunit.client.1.smithi174.stdout:0/177: dread d9/d21/d23/f25 [0,4194304] 0 2024-03-21T10:08:47.947 INFO:tasks.workunit.client.0.smithi153.stdout:7/302: rename d4/d2a/d40/c49 to d4/d19/c62 0 2024-03-21T10:08:47.948 INFO:tasks.workunit.client.0.smithi153.stdout:2/425: mknod dd/d15/d18/c92 0 2024-03-21T10:08:47.950 INFO:tasks.workunit.client.0.smithi153.stdout:7/303: creat d4/d2a/f63 x:0 0 0 2024-03-21T10:08:47.950 INFO:tasks.workunit.client.0.smithi153.stdout:2/426: fdatasync dd/d15/d18/f7a 0 2024-03-21T10:08:47.951 INFO:tasks.workunit.client.0.smithi153.stdout:5/387: dread d4/d8/f26 [0,4194304] 0 2024-03-21T10:08:47.952 INFO:tasks.workunit.client.1.smithi174.stdout:6/124: dwrite d0/d3/f8 [0,4194304] 0 2024-03-21T10:08:47.953 INFO:tasks.workunit.client.0.smithi153.stdout:2/427: dread dd/d15/d18/d2c/f39 [0,4194304] 0 2024-03-21T10:08:47.954 INFO:tasks.workunit.client.0.smithi153.stdout:7/304: creat d4/d2a/d3b/d60/f64 x:0 0 0 2024-03-21T10:08:47.955 INFO:tasks.workunit.client.0.smithi153.stdout:2/428: chown dd/d15/d18/c92 81459 1 2024-03-21T10:08:47.961 INFO:tasks.workunit.client.1.smithi174.stdout:9/220: dwrite d2/f28 [0,4194304] 0 2024-03-21T10:08:47.961 INFO:tasks.workunit.client.0.smithi153.stdout:7/305: symlink d4/l65 0 2024-03-21T10:08:47.965 INFO:tasks.workunit.client.0.smithi153.stdout:7/306: stat d4/d14/d16/l34 0 2024-03-21T10:08:47.966 INFO:tasks.workunit.client.1.smithi174.stdout:1/199: truncate d4/d5/f18 1011608 0 2024-03-21T10:08:47.966 INFO:tasks.workunit.client.0.smithi153.stdout:7/307: readlink d4/lf 0 2024-03-21T10:08:47.968 INFO:tasks.workunit.client.0.smithi153.stdout:7/308: fsync d4/f1a 0 2024-03-21T10:08:47.969 INFO:tasks.workunit.client.0.smithi153.stdout:2/429: link dd/d15/d18/c1f dd/d3b/d50/d58/c93 0 2024-03-21T10:08:47.970 INFO:tasks.workunit.client.1.smithi174.stdout:3/126: link d0/fa d0/d12/f2d 0 2024-03-21T10:08:47.971 INFO:tasks.workunit.client.0.smithi153.stdout:5/388: dwrite d4/d5/de/f21 [0,4194304] 0 2024-03-21T10:08:47.972 INFO:tasks.workunit.client.1.smithi174.stdout:4/209: dwrite d4/f18 [0,4194304] 0 2024-03-21T10:08:47.976 INFO:tasks.workunit.client.1.smithi174.stdout:8/115: rename d9/lb to d9/l25 0 2024-03-21T10:08:47.977 INFO:tasks.workunit.client.1.smithi174.stdout:6/125: mkdir d0/d3/d2a 0 2024-03-21T10:08:47.978 INFO:tasks.workunit.client.1.smithi174.stdout:2/192: write d5/d25/d27/f39 [1165543,42647] 0 2024-03-21T10:08:47.978 INFO:tasks.workunit.client.0.smithi153.stdout:0/286: dwrite d4/f32 [0,4194304] 0 2024-03-21T10:08:47.982 INFO:tasks.workunit.client.1.smithi174.stdout:2/193: write d5/d8/f29 [266718,41428] 0 2024-03-21T10:08:47.990 INFO:tasks.workunit.client.1.smithi174.stdout:9/221: unlink d2/d11/d1d/l27 0 2024-03-21T10:08:47.991 INFO:tasks.workunit.client.0.smithi153.stdout:8/370: sync 2024-03-21T10:08:47.993 INFO:tasks.workunit.client.0.smithi153.stdout:8/371: truncate d5/d3b/f63 681836 0 2024-03-21T10:08:47.994 INFO:tasks.workunit.client.1.smithi174.stdout:4/210: mknod d4/d21/d24/c4b 0 2024-03-21T10:08:47.995 INFO:tasks.workunit.client.0.smithi153.stdout:8/372: chown d5/d9/d43/dd/d20/d26/d33/f46 685125548 1 2024-03-21T10:08:47.995 INFO:tasks.workunit.client.1.smithi174.stdout:6/126: creat d0/d3/d5/df/d16/f2b x:0 0 0 2024-03-21T10:08:47.996 INFO:tasks.workunit.client.1.smithi174.stdout:2/194: mknod d5/d8/d2f/c46 0 2024-03-21T10:08:47.999 INFO:tasks.workunit.client.1.smithi174.stdout:3/127: dread d0/fa [0,4194304] 0 2024-03-21T10:08:48.002 INFO:tasks.workunit.client.1.smithi174.stdout:4/211: read d4/d21/d24/d28/f39 [367845,97494] 0 2024-03-21T10:08:48.004 INFO:tasks.workunit.client.1.smithi174.stdout:8/116: link d9/l1d d9/l26 0 2024-03-21T10:08:48.006 INFO:tasks.workunit.client.1.smithi174.stdout:2/195: mknod d5/d25/d27/c47 0 2024-03-21T10:08:48.008 INFO:tasks.workunit.client.0.smithi153.stdout:6/565: sync 2024-03-21T10:08:48.009 INFO:tasks.workunit.client.0.smithi153.stdout:1/325: sync 2024-03-21T10:08:48.009 INFO:tasks.workunit.client.0.smithi153.stdout:3/340: sync 2024-03-21T10:08:48.009 INFO:tasks.workunit.client.0.smithi153.stdout:4/390: sync 2024-03-21T10:08:48.009 INFO:tasks.workunit.client.0.smithi153.stdout:9/353: sync 2024-03-21T10:08:48.010 INFO:tasks.workunit.client.1.smithi174.stdout:2/196: fdatasync d5/d25/d27/d2d/f43 0 2024-03-21T10:08:48.011 INFO:tasks.workunit.client.0.smithi153.stdout:4/391: write f1 [1120298,125199] 0 2024-03-21T10:08:48.013 INFO:tasks.workunit.client.1.smithi174.stdout:4/212: creat d4/d21/d31/d3a/f4c x:0 0 0 2024-03-21T10:08:48.014 INFO:tasks.workunit.client.0.smithi153.stdout:4/392: write d11/d2d/f73 [55971,103546] 0 2024-03-21T10:08:48.014 INFO:tasks.workunit.client.0.smithi153.stdout:0/287: dread d4/f10 [4194304,4194304] 0 2024-03-21T10:08:48.015 INFO:tasks.workunit.client.0.smithi153.stdout:6/566: creat d1f/d2b/d33/d40/d65/fca x:0 0 0 2024-03-21T10:08:48.016 INFO:tasks.workunit.client.0.smithi153.stdout:4/393: fsync d11/f78 0 2024-03-21T10:08:48.017 INFO:tasks.workunit.client.0.smithi153.stdout:1/326: rename d6/f15 to d6/d3a/d66/f68 0 2024-03-21T10:08:48.018 INFO:tasks.workunit.client.0.smithi153.stdout:4/394: readlink d11/d18/l47 0 2024-03-21T10:08:48.018 INFO:tasks.workunit.client.0.smithi153.stdout:3/341: mkdir dd/d1e/d2f/d35/d68/d75 0 2024-03-21T10:08:48.019 INFO:tasks.workunit.client.0.smithi153.stdout:9/354: rename d1 to d1/d43/d44/d46/d1c/d61/d7a 22 2024-03-21T10:08:48.019 INFO:tasks.workunit.client.1.smithi174.stdout:3/128: getdents d0/d12/d14/d1a/d27 0 2024-03-21T10:08:48.020 INFO:tasks.workunit.client.0.smithi153.stdout:0/288: mknod d4/db/c5d 0 2024-03-21T10:08:48.020 INFO:tasks.workunit.client.0.smithi153.stdout:4/395: chown d11/f6f 142769853 1 2024-03-21T10:08:48.022 INFO:tasks.workunit.client.0.smithi153.stdout:1/327: symlink d6/d3a/d3c/d50/l69 0 2024-03-21T10:08:48.023 INFO:tasks.workunit.client.0.smithi153.stdout:6/567: write d1f/d2b/db3/fc8 [10411,86556] 0 2024-03-21T10:08:48.024 INFO:tasks.workunit.client.1.smithi174.stdout:2/197: link d5/d25/d27/d2d/f43 d5/d25/d27/d44/f48 0 2024-03-21T10:08:48.025 INFO:tasks.workunit.client.0.smithi153.stdout:3/342: symlink dd/d1e/d2f/d35/l76 0 2024-03-21T10:08:48.026 INFO:tasks.workunit.client.0.smithi153.stdout:0/289: mkdir d4/d16/d3e/d5e 0 2024-03-21T10:08:48.026 INFO:tasks.workunit.client.1.smithi174.stdout:3/129: symlink d0/d12/l2e 0 2024-03-21T10:08:48.027 INFO:tasks.workunit.client.1.smithi174.stdout:2/198: chown d5/f24 401080 1 2024-03-21T10:08:48.028 INFO:tasks.workunit.client.0.smithi153.stdout:9/355: write d1/d43/d44/d46/d1c/d24/f32 [128959,8700] 0 2024-03-21T10:08:48.029 INFO:tasks.workunit.client.0.smithi153.stdout:2/430: dread dd/f26 [0,4194304] 0 2024-03-21T10:08:48.029 INFO:tasks.workunit.client.1.smithi174.stdout:2/199: write d5/f12 [2030183,65478] 0 2024-03-21T10:08:48.031 INFO:tasks.workunit.client.0.smithi153.stdout:6/568: dread d1f/f4e [0,4194304] 0 2024-03-21T10:08:48.031 INFO:tasks.workunit.client.0.smithi153.stdout:0/290: dread d4/d16/d19/d30/f41 [0,4194304] 0 2024-03-21T10:08:48.034 INFO:tasks.workunit.client.0.smithi153.stdout:0/291: write d4/db/f11 [3028621,53618] 0 2024-03-21T10:08:48.039 INFO:tasks.workunit.client.0.smithi153.stdout:3/343: symlink dd/d1e/d65/l77 0 2024-03-21T10:08:48.039 INFO:tasks.workunit.client.0.smithi153.stdout:6/569: creat d1f/d99/dbb/fcb x:0 0 0 2024-03-21T10:08:48.042 INFO:tasks.workunit.client.1.smithi174.stdout:3/130: getdents d0/d12/d14/d1a 0 2024-03-21T10:08:48.043 INFO:tasks.workunit.client.0.smithi153.stdout:7/309: write d4/d11/d24/f3a [1355154,110295] 0 2024-03-21T10:08:48.043 INFO:tasks.workunit.client.0.smithi153.stdout:2/431: creat dd/d3b/d50/d59/d5d/f94 x:0 0 0 2024-03-21T10:08:48.044 INFO:tasks.workunit.client.0.smithi153.stdout:0/292: unlink d4/d16/d19/l2b 0 2024-03-21T10:08:48.046 INFO:tasks.workunit.client.0.smithi153.stdout:6/570: creat d1f/d2b/db3/fcc x:0 0 0 2024-03-21T10:08:48.047 INFO:tasks.workunit.client.0.smithi153.stdout:3/344: mknod dd/d1e/d3d/c78 0 2024-03-21T10:08:48.047 INFO:tasks.workunit.client.0.smithi153.stdout:2/432: chown dd/d15/d18/l28 1723028 1 2024-03-21T10:08:48.047 INFO:tasks.workunit.client.0.smithi153.stdout:0/293: fsync d4/d16/d1f/f20 0 2024-03-21T10:08:48.048 INFO:tasks.workunit.client.1.smithi174.stdout:7/142: truncate d0/d4/f12 3534786 0 2024-03-21T10:08:48.048 INFO:tasks.workunit.client.1.smithi174.stdout:5/165: write d1/ff [2454783,70040] 0 2024-03-21T10:08:48.050 INFO:tasks.workunit.client.0.smithi153.stdout:9/356: link d1/l11 d1/d43/d44/d46/d1c/l7b 0 2024-03-21T10:08:48.050 INFO:tasks.workunit.client.1.smithi174.stdout:6/127: dread d0/d3/d5/d6/f13 [0,4194304] 0 2024-03-21T10:08:48.050 INFO:tasks.workunit.client.0.smithi153.stdout:0/294: chown d4/db/f1b 802876146 1 2024-03-21T10:08:48.052 INFO:tasks.workunit.client.1.smithi174.stdout:4/213: getdents d4/d21/d24 0 2024-03-21T10:08:48.053 INFO:tasks.workunit.client.1.smithi174.stdout:6/128: chown d0/d3/d5/f1c 22943 1 2024-03-21T10:08:48.053 INFO:tasks.workunit.client.1.smithi174.stdout:5/166: chown d1/d16/c1f 195 1 2024-03-21T10:08:48.053 INFO:tasks.workunit.client.0.smithi153.stdout:3/345: creat dd/d1e/d45/d56/f79 x:0 0 0 2024-03-21T10:08:48.053 INFO:tasks.workunit.client.0.smithi153.stdout:7/310: link d4/lf d4/d1b/d20/l66 0 2024-03-21T10:08:48.054 INFO:tasks.workunit.client.1.smithi174.stdout:7/143: creat d0/d4/f33 x:0 0 0 2024-03-21T10:08:48.057 INFO:tasks.workunit.client.1.smithi174.stdout:4/214: stat d4/d21/d24/d28/d38 0 2024-03-21T10:08:48.058 INFO:tasks.workunit.client.0.smithi153.stdout:0/295: fsync d4/d16/d1f/d22/f2e 0 2024-03-21T10:08:48.058 INFO:tasks.workunit.client.0.smithi153.stdout:3/346: write dd/f12 [1627510,19573] 0 2024-03-21T10:08:48.058 INFO:tasks.workunit.client.0.smithi153.stdout:9/357: rename d1/d4/la to d1/d43/d44/d46/d1c/d24/l7c 0 2024-03-21T10:08:48.059 INFO:tasks.workunit.client.1.smithi174.stdout:6/129: mknod d0/d3/d5/df/c2c 0 2024-03-21T10:08:48.059 INFO:tasks.workunit.client.1.smithi174.stdout:7/144: mknod d0/c34 0 2024-03-21T10:08:48.059 INFO:tasks.workunit.client.0.smithi153.stdout:8/373: write d5/f2f [5302771,75386] 0 2024-03-21T10:08:48.061 INFO:tasks.workunit.client.1.smithi174.stdout:3/131: dread d0/d12/d14/d10/f1b [0,4194304] 0 2024-03-21T10:08:48.061 INFO:tasks.workunit.client.1.smithi174.stdout:6/130: chown d0/d3/d20/f23 21843579 1 2024-03-21T10:08:48.063 INFO:tasks.workunit.client.1.smithi174.stdout:5/167: write d1/d16/f30 [1065879,72904] 0 2024-03-21T10:08:48.063 INFO:tasks.workunit.client.1.smithi174.stdout:4/215: creat d4/d21/d24/f4d x:0 0 0 2024-03-21T10:08:48.064 INFO:tasks.workunit.client.1.smithi174.stdout:9/222: truncate d2/d11/d1d/f33 518449 0 2024-03-21T10:08:48.065 INFO:tasks.workunit.client.0.smithi153.stdout:3/347: chown dd/d1e/f4f 130419 1 2024-03-21T10:08:48.068 INFO:tasks.workunit.client.1.smithi174.stdout:4/216: write d4/d19/f2d [27668,28423] 0 2024-03-21T10:08:48.068 INFO:tasks.workunit.client.1.smithi174.stdout:3/132: stat d0/c6 0 2024-03-21T10:08:48.068 INFO:tasks.workunit.client.1.smithi174.stdout:9/223: write d2/f17 [339663,130433] 0 2024-03-21T10:08:48.068 INFO:tasks.workunit.client.1.smithi174.stdout:4/217: fdatasync d4/f18 0 2024-03-21T10:08:48.068 INFO:tasks.workunit.client.1.smithi174.stdout:5/168: symlink d1/d16/d2e/d31/l39 0 2024-03-21T10:08:48.069 INFO:tasks.workunit.client.0.smithi153.stdout:0/296: mknod d4/d16/d27/c5f 0 2024-03-21T10:08:48.069 INFO:tasks.workunit.client.1.smithi174.stdout:9/224: chown d2/d11/d1d/d2a 1 1 2024-03-21T10:08:48.070 INFO:tasks.workunit.client.1.smithi174.stdout:8/117: write f7 [2413877,52367] 0 2024-03-21T10:08:48.070 INFO:tasks.workunit.client.1.smithi174.stdout:4/218: stat d4/d19/f41 0 2024-03-21T10:08:48.071 INFO:tasks.workunit.client.1.smithi174.stdout:9/225: stat d2/f9 0 2024-03-21T10:08:48.073 INFO:tasks.workunit.client.1.smithi174.stdout:8/118: write d9/d17/d19/f23 [1001495,58326] 0 2024-03-21T10:08:48.074 INFO:tasks.workunit.client.0.smithi153.stdout:0/297: dread - d4/d16/d27/f5c zero size 2024-03-21T10:08:48.075 INFO:tasks.workunit.client.1.smithi174.stdout:3/133: creat d0/d12/d14/d1a/f2f x:0 0 0 2024-03-21T10:08:48.075 INFO:tasks.workunit.client.1.smithi174.stdout:5/169: chown d1/l5 0 1 2024-03-21T10:08:48.078 INFO:tasks.workunit.client.0.smithi153.stdout:8/374: rename d5/d9/d43/dd/d2d/l42 to d5/d1b/d49/l74 0 2024-03-21T10:08:48.080 INFO:tasks.workunit.client.1.smithi174.stdout:4/219: rename d4/d21/d31/f4a to d4/d21/d24/d28/f4e 0 2024-03-21T10:08:48.080 INFO:tasks.workunit.client.0.smithi153.stdout:3/348: creat dd/d1e/d45/f7a x:0 0 0 2024-03-21T10:08:48.081 INFO:tasks.workunit.client.1.smithi174.stdout:5/170: chown d1/d16/d2e/d31/l39 34616999 1 2024-03-21T10:08:48.083 INFO:tasks.workunit.client.1.smithi174.stdout:8/119: mkdir d9/d17/d19/d27 0 2024-03-21T10:08:48.083 INFO:tasks.workunit.client.0.smithi153.stdout:7/311: dwrite d4/fa [4194304,4194304] 0 2024-03-21T10:08:48.084 INFO:tasks.workunit.client.0.smithi153.stdout:4/396: write d11/d2d/f42 [1024800,85381] 0 2024-03-21T10:08:48.084 INFO:tasks.workunit.client.1.smithi174.stdout:3/134: mknod d0/d12/d14/c30 0 2024-03-21T10:08:48.085 INFO:tasks.workunit.client.1.smithi174.stdout:4/220: chown d4/c32 30467 1 2024-03-21T10:08:48.085 INFO:tasks.workunit.client.1.smithi174.stdout:5/171: fdatasync d1/d16/f24 0 2024-03-21T10:08:48.086 INFO:tasks.workunit.client.1.smithi174.stdout:9/226: creat d2/f47 x:0 0 0 2024-03-21T10:08:48.087 INFO:tasks.workunit.client.0.smithi153.stdout:3/349: read dd/f1c [201364,46019] 0 2024-03-21T10:08:48.093 INFO:tasks.workunit.client.0.smithi153.stdout:1/328: truncate d6/d3a/f53 2248840 0 2024-03-21T10:08:48.093 INFO:tasks.workunit.client.0.smithi153.stdout:3/350: chown dd/d1e/d3d/f69 375656 1 2024-03-21T10:08:48.095 INFO:tasks.workunit.client.1.smithi174.stdout:2/200: dwrite d5/f14 [4194304,4194304] 0 2024-03-21T10:08:48.095 INFO:tasks.workunit.client.1.smithi174.stdout:3/135: creat d0/d12/d14/d10/f31 x:0 0 0 2024-03-21T10:08:48.096 INFO:tasks.workunit.client.0.smithi153.stdout:3/351: fdatasync dd/d1e/f4f 0 2024-03-21T10:08:48.099 INFO:tasks.workunit.client.1.smithi174.stdout:7/145: dwrite d0/ff [0,4194304] 0 2024-03-21T10:08:48.100 INFO:tasks.workunit.client.0.smithi153.stdout:9/358: dwrite d1/d43/d44/d46/d1c/d24/d41/f5b [4194304,4194304] 0 2024-03-21T10:08:48.101 INFO:tasks.workunit.client.1.smithi174.stdout:1/200: dwrite d4/d5/f18 [0,4194304] 0 2024-03-21T10:08:48.103 INFO:tasks.workunit.client.0.smithi153.stdout:2/433: dread dd/d15/f52 [0,4194304] 0 2024-03-21T10:08:48.105 INFO:tasks.workunit.client.1.smithi174.stdout:1/201: chown d4/d2d 11400522 1 2024-03-21T10:08:48.106 INFO:tasks.workunit.client.0.smithi153.stdout:8/375: rename d5/d9/d43/dd/d20/d26/l6d to d5/d1b/d49/l75 0 2024-03-21T10:08:48.106 INFO:tasks.workunit.client.1.smithi174.stdout:3/136: symlink d0/d12/d14/d1a/l32 0 2024-03-21T10:08:48.106 INFO:tasks.workunit.client.0.smithi153.stdout:3/352: truncate dd/d41/f64 822385 0 2024-03-21T10:08:48.107 INFO:tasks.workunit.client.1.smithi174.stdout:5/172: link d1/d16/d2e/d31/l39 d1/d2f/l3a 0 2024-03-21T10:08:48.109 INFO:tasks.workunit.client.1.smithi174.stdout:7/146: write d0/f11 [1059009,14630] 0 2024-03-21T10:08:48.109 INFO:tasks.workunit.client.0.smithi153.stdout:0/298: getdents d4/d16/d1f/d22 0 2024-03-21T10:08:48.110 INFO:tasks.workunit.client.1.smithi174.stdout:1/202: creat d4/d7/d33/d43/f47 x:0 0 0 2024-03-21T10:08:48.112 INFO:tasks.workunit.client.0.smithi153.stdout:1/329: creat d6/d3a/d66/f6a x:0 0 0 2024-03-21T10:08:48.115 INFO:tasks.workunit.client.1.smithi174.stdout:8/120: dwrite d9/f15 [0,4194304] 0 2024-03-21T10:08:48.117 INFO:tasks.workunit.client.1.smithi174.stdout:3/137: dread d0/d12/d14/d1a/d27/f2c [0,4194304] 0 2024-03-21T10:08:48.117 INFO:tasks.workunit.client.0.smithi153.stdout:9/359: chown d1/d43/l64 450 1 2024-03-21T10:08:48.118 INFO:tasks.workunit.client.1.smithi174.stdout:2/201: rename d5/f10 to d5/f49 0 2024-03-21T10:08:48.119 INFO:tasks.workunit.client.0.smithi153.stdout:8/376: dread - d5/d9/f4b zero size 2024-03-21T10:08:48.122 INFO:tasks.workunit.client.1.smithi174.stdout:7/147: mkdir d0/d4/d10/d1b/d35 0 2024-03-21T10:08:48.123 INFO:tasks.workunit.client.0.smithi153.stdout:0/299: rmdir d4/d16/d19 39 2024-03-21T10:08:48.128 INFO:tasks.workunit.client.1.smithi174.stdout:1/203: mknod d4/d2d/c48 0 2024-03-21T10:08:48.128 INFO:tasks.workunit.client.1.smithi174.stdout:5/173: dread d1/d16/d2e/d31/f33 [0,4194304] 0 2024-03-21T10:08:48.130 INFO:tasks.workunit.client.1.smithi174.stdout:8/121: mknod d9/d17/d19/c28 0 2024-03-21T10:08:48.131 INFO:tasks.workunit.client.1.smithi174.stdout:3/138: creat d0/f33 x:0 0 0 2024-03-21T10:08:48.133 INFO:tasks.workunit.client.1.smithi174.stdout:8/122: write f5 [1793639,59017] 0 2024-03-21T10:08:48.134 INFO:tasks.workunit.client.0.smithi153.stdout:5/389: dread d4/d5/de/f43 [0,4194304] 0 2024-03-21T10:08:48.135 INFO:tasks.workunit.client.1.smithi174.stdout:7/148: creat d0/d4/f36 x:0 0 0 2024-03-21T10:08:48.136 INFO:tasks.workunit.client.0.smithi153.stdout:3/353: unlink dd/d1e/d2f/f5b 0 2024-03-21T10:08:48.138 INFO:tasks.workunit.client.0.smithi153.stdout:7/312: dwrite d4/d2a/f3e [0,4194304] 0 2024-03-21T10:08:48.138 INFO:tasks.workunit.client.1.smithi174.stdout:3/139: write d0/d12/d14/d10/f1f [1417350,100931] 0 2024-03-21T10:08:48.141 INFO:tasks.workunit.client.1.smithi174.stdout:8/123: creat d9/d17/f29 x:0 0 0 2024-03-21T10:08:48.142 INFO:tasks.workunit.client.1.smithi174.stdout:5/174: write d1/f29 [1023652,53567] 0 2024-03-21T10:08:48.143 INFO:tasks.workunit.client.0.smithi153.stdout:9/360: write d1/d43/d44/f5d [3441229,108151] 0 2024-03-21T10:08:48.147 INFO:tasks.workunit.client.1.smithi174.stdout:7/149: truncate d0/d4/f36 996172 0 2024-03-21T10:08:48.149 INFO:tasks.workunit.client.0.smithi153.stdout:5/390: stat d4/d5/de/d44/f6f 0 2024-03-21T10:08:48.150 INFO:tasks.workunit.client.1.smithi174.stdout:8/124: write d9/d17/d19/f23 [1554909,60498] 0 2024-03-21T10:08:48.153 INFO:tasks.workunit.client.1.smithi174.stdout:9/227: write d2/d11/f30 [427529,103828] 0 2024-03-21T10:08:48.155 INFO:tasks.workunit.client.1.smithi174.stdout:2/202: dwrite d5/d8/f29 [0,4194304] 0 2024-03-21T10:08:48.161 INFO:tasks.workunit.client.1.smithi174.stdout:5/175: symlink d1/d16/d2e/d31/l3b 0 2024-03-21T10:08:48.161 INFO:tasks.workunit.client.0.smithi153.stdout:5/391: write d4/d8/f11 [4961805,10937] 0 2024-03-21T10:08:48.163 INFO:tasks.workunit.client.1.smithi174.stdout:7/150: symlink d0/d4/l37 0 2024-03-21T10:08:48.163 INFO:tasks.workunit.client.1.smithi174.stdout:8/125: mkdir d9/d2a 0 2024-03-21T10:08:48.164 INFO:tasks.workunit.client.1.smithi174.stdout:5/176: stat d1/d16/d2e/d31/f33 0 2024-03-21T10:08:48.166 INFO:tasks.workunit.client.1.smithi174.stdout:1/204: dwrite d4/d7/d28/f30 [0,4194304] 0 2024-03-21T10:08:48.167 INFO:tasks.workunit.client.0.smithi153.stdout:6/571: dwrite f15 [4194304,4194304] 0 2024-03-21T10:08:48.171 INFO:tasks.workunit.client.1.smithi174.stdout:9/228: creat d2/d24/d2e/f48 x:0 0 0 2024-03-21T10:08:48.172 INFO:tasks.workunit.client.1.smithi174.stdout:1/205: fdatasync d4/d7/d28/f3b 0 2024-03-21T10:08:48.173 INFO:tasks.workunit.client.0.smithi153.stdout:5/392: read - d4/d5/df/d5e/d69/f6d zero size 2024-03-21T10:08:48.173 INFO:tasks.workunit.client.1.smithi174.stdout:3/140: link d0/f13 d0/d12/d14/d1a/f34 0 2024-03-21T10:08:48.174 INFO:tasks.workunit.client.0.smithi153.stdout:6/572: dread - d1f/d99/faf zero size 2024-03-21T10:08:48.175 INFO:tasks.workunit.client.1.smithi174.stdout:1/206: read d4/d7/d15/f1f [57269,16641] 0 2024-03-21T10:08:48.175 INFO:tasks.workunit.client.0.smithi153.stdout:3/354: rename f0 to dd/d1e/d2f/d35/f7b 0 2024-03-21T10:08:48.176 INFO:tasks.workunit.client.0.smithi153.stdout:7/313: unlink d4/d14/f53 0 2024-03-21T10:08:48.177 INFO:tasks.workunit.client.1.smithi174.stdout:8/126: symlink d9/l2b 0 2024-03-21T10:08:48.177 INFO:tasks.workunit.client.0.smithi153.stdout:8/377: write d5/d9/d43/dd/d20/d26/f1f [480688,71401] 0 2024-03-21T10:08:48.179 INFO:tasks.workunit.client.0.smithi153.stdout:2/434: dwrite dd/d15/d18/d2c/f3f [0,4194304] 0 2024-03-21T10:08:48.181 INFO:tasks.workunit.client.0.smithi153.stdout:9/361: symlink d1/d4/l7d 0 2024-03-21T10:08:48.182 INFO:tasks.workunit.client.0.smithi153.stdout:6/573: unlink d1f/d93/f9f 0 2024-03-21T10:08:48.182 INFO:tasks.workunit.client.0.smithi153.stdout:5/393: write d4/d5/d16/f19 [4221294,52479] 0 2024-03-21T10:08:48.185 INFO:tasks.workunit.client.1.smithi174.stdout:2/203: creat d5/f4a x:0 0 0 2024-03-21T10:08:48.185 INFO:tasks.workunit.client.1.smithi174.stdout:5/177: readlink d1/d16/d2e/d31/l39 0 2024-03-21T10:08:48.189 INFO:tasks.workunit.client.0.smithi153.stdout:2/435: fdatasync dd/d15/d18/d3a/d3d/f53 0 2024-03-21T10:08:48.190 INFO:tasks.workunit.client.0.smithi153.stdout:0/300: dwrite d4/d16/d1f/f20 [0,4194304] 0 2024-03-21T10:08:48.191 INFO:tasks.workunit.client.0.smithi153.stdout:7/314: rename d4/d19/d4d/c5a to d4/d11/d56/c67 0 2024-03-21T10:08:48.195 INFO:tasks.workunit.client.1.smithi174.stdout:9/229: truncate d2/d11/d1d/f41 142077 0 2024-03-21T10:08:48.195 INFO:tasks.workunit.client.0.smithi153.stdout:6/574: chown d1f/d2f/d59/f81 38 1 2024-03-21T10:08:48.197 INFO:tasks.workunit.client.1.smithi174.stdout:7/151: unlink d0/c24 0 2024-03-21T10:08:48.202 INFO:tasks.workunit.client.0.smithi153.stdout:6/575: dread - d1f/d2b/fc6 zero size 2024-03-21T10:08:48.202 INFO:tasks.workunit.client.1.smithi174.stdout:3/141: symlink d0/d12/d14/d1a/d27/l35 0 2024-03-21T10:08:48.202 INFO:tasks.workunit.client.1.smithi174.stdout:7/152: write d0/f2f [236890,102601] 0 2024-03-21T10:08:48.203 INFO:tasks.workunit.client.0.smithi153.stdout:5/394: mkdir d4/d8/d5a/d5d/d7b 0 2024-03-21T10:08:48.203 INFO:tasks.workunit.client.1.smithi174.stdout:3/142: stat d0/f7 0 2024-03-21T10:08:48.207 INFO:tasks.workunit.client.0.smithi153.stdout:1/330: dread d6/d10/d2c/f33 [0,4194304] 0 2024-03-21T10:08:48.208 INFO:tasks.workunit.client.1.smithi174.stdout:3/143: dread - d0/d12/d14/d10/f31 zero size 2024-03-21T10:08:48.208 INFO:tasks.workunit.client.1.smithi174.stdout:8/127: rename d9/l2b to d9/d17/d19/d27/l2c 0 2024-03-21T10:08:48.208 INFO:tasks.workunit.client.0.smithi153.stdout:5/395: dread - d4/d5/de/f3f zero size 2024-03-21T10:08:48.209 INFO:tasks.workunit.client.1.smithi174.stdout:1/207: dwrite d4/fb [0,4194304] 0 2024-03-21T10:08:48.209 INFO:tasks.workunit.client.0.smithi153.stdout:3/355: dwrite dd/f2d [0,4194304] 0 2024-03-21T10:08:48.211 INFO:tasks.workunit.client.1.smithi174.stdout:3/144: fdatasync d0/d12/f17 0 2024-03-21T10:08:48.211 INFO:tasks.workunit.client.1.smithi174.stdout:2/204: fdatasync d5/d25/d27/d2d/f41 0 2024-03-21T10:08:48.216 INFO:tasks.workunit.client.1.smithi174.stdout:2/205: write d5/f11 [5935980,99606] 0 2024-03-21T10:08:48.219 INFO:tasks.workunit.client.1.smithi174.stdout:5/178: symlink d1/d2f/d37/l3c 0 2024-03-21T10:08:48.219 INFO:tasks.workunit.client.1.smithi174.stdout:3/145: dread - d0/d12/d14/d1a/f29 zero size 2024-03-21T10:08:48.219 INFO:tasks.workunit.client.0.smithi153.stdout:8/378: rename d5/d9/d43/dd/d20/f50 to d5/d9/d43/dd/d20/d26/d33/f76 0 2024-03-21T10:08:48.220 INFO:tasks.workunit.client.0.smithi153.stdout:4/397: dread f2 [4194304,4194304] 0 2024-03-21T10:08:48.220 INFO:tasks.workunit.client.1.smithi174.stdout:5/179: chown d1/d16 3 1 2024-03-21T10:08:48.226 INFO:tasks.workunit.client.0.smithi153.stdout:2/436: truncate dd/f37 1338986 0 2024-03-21T10:08:48.226 INFO:tasks.workunit.client.1.smithi174.stdout:7/153: dwrite d0/d4/f36 [0,4194304] 0 2024-03-21T10:08:48.227 INFO:tasks.workunit.client.1.smithi174.stdout:7/154: readlink d0/d4/l15 0 2024-03-21T10:08:48.229 INFO:tasks.workunit.client.0.smithi153.stdout:5/396: mknod d4/d5/df/c7c 0 2024-03-21T10:08:48.238 INFO:tasks.workunit.client.1.smithi174.stdout:8/128: symlink d9/l2d 0 2024-03-21T10:08:48.238 INFO:tasks.workunit.client.0.smithi153.stdout:3/356: dwrite dd/f20 [0,4194304] 0 2024-03-21T10:08:48.238 INFO:tasks.workunit.client.0.smithi153.stdout:5/397: write d4/d5/de/f3f [262132,52448] 0 2024-03-21T10:08:48.239 INFO:tasks.workunit.client.0.smithi153.stdout:5/398: readlink d4/d5/df/l60 0 2024-03-21T10:08:48.240 INFO:tasks.workunit.client.0.smithi153.stdout:0/301: rename d4/d16/d1f/d22/l50 to d4/d16/d3e/l60 0 2024-03-21T10:08:48.242 INFO:tasks.workunit.client.1.smithi174.stdout:0/178: sync 2024-03-21T10:08:48.243 INFO:tasks.workunit.client.1.smithi174.stdout:2/206: mkdir d5/d8/d2f/d4b 0 2024-03-21T10:08:48.243 INFO:tasks.workunit.client.0.smithi153.stdout:8/379: creat d5/d9/d43/f77 x:0 0 0 2024-03-21T10:08:48.243 INFO:tasks.workunit.client.0.smithi153.stdout:9/362: getdents d1/d43/d44/d46/d4d 0 2024-03-21T10:08:48.245 INFO:tasks.workunit.client.1.smithi174.stdout:9/230: mknod d2/d11/c49 0 2024-03-21T10:08:48.249 INFO:tasks.workunit.client.1.smithi174.stdout:2/207: symlink d5/d25/l4c 0 2024-03-21T10:08:48.252 INFO:tasks.workunit.client.1.smithi174.stdout:5/180: rename d1/c8 to d1/d2f/c3d 0 2024-03-21T10:08:48.252 INFO:tasks.workunit.client.1.smithi174.stdout:3/146: dwrite d0/d12/d14/d1a/f23 [0,4194304] 0 2024-03-21T10:08:48.252 INFO:tasks.workunit.client.1.smithi174.stdout:1/208: link d4/d5/c12 d4/d7/d28/c49 0 2024-03-21T10:08:48.260 INFO:tasks.workunit.client.0.smithi153.stdout:2/437: unlink dd/d15/l48 0 2024-03-21T10:08:48.261 INFO:tasks.workunit.client.0.smithi153.stdout:4/398: dwrite d11/d2d/f5a [0,4194304] 0 2024-03-21T10:08:48.262 INFO:tasks.workunit.client.1.smithi174.stdout:7/155: rename d0/d4/d10/d1b/d35 to d0/d4/d38 0 2024-03-21T10:08:48.264 INFO:tasks.workunit.client.1.smithi174.stdout:5/181: mknod d1/d16/d2e/c3e 0 2024-03-21T10:08:48.266 INFO:tasks.workunit.client.1.smithi174.stdout:3/147: dread d0/d12/d14/d1a/f1e [0,4194304] 0 2024-03-21T10:08:48.266 INFO:tasks.workunit.client.1.smithi174.stdout:0/179: dread f7 [0,4194304] 0 2024-03-21T10:08:48.270 INFO:tasks.workunit.client.1.smithi174.stdout:4/221: sync 2024-03-21T10:08:48.271 INFO:tasks.workunit.client.1.smithi174.stdout:6/131: sync 2024-03-21T10:08:48.272 INFO:tasks.workunit.client.0.smithi153.stdout:0/302: dwrite d4/f28 [0,4194304] 0 2024-03-21T10:08:48.273 INFO:tasks.workunit.client.0.smithi153.stdout:7/315: link d4/d1b/c1e d4/d2a/d3b/d60/c68 0 2024-03-21T10:08:48.274 INFO:tasks.workunit.client.0.smithi153.stdout:2/438: fsync dd/d15/d18/d2c/f3f 0 2024-03-21T10:08:48.274 INFO:tasks.workunit.client.1.smithi174.stdout:0/180: read d9/f19 [5968245,103205] 0 2024-03-21T10:08:48.278 INFO:tasks.workunit.client.1.smithi174.stdout:9/231: creat d2/d11/f4a x:0 0 0 2024-03-21T10:08:48.279 INFO:tasks.workunit.client.0.smithi153.stdout:1/331: dwrite d6/f31 [0,4194304] 0 2024-03-21T10:08:48.279 INFO:tasks.workunit.client.0.smithi153.stdout:6/576: rename d1f/d2f/f3d to d1f/d2b/d33/d40/d51/d98/fcd 0 2024-03-21T10:08:48.280 INFO:tasks.workunit.client.0.smithi153.stdout:1/332: dread - d6/f52 zero size 2024-03-21T10:08:48.283 INFO:tasks.workunit.client.0.smithi153.stdout:3/357: creat dd/d1e/d2f/d35/d66/d73/f7c x:0 0 0 2024-03-21T10:08:48.284 INFO:tasks.workunit.client.0.smithi153.stdout:8/380: truncate d5/d9/f19 1482036 0 2024-03-21T10:08:48.285 INFO:tasks.workunit.client.1.smithi174.stdout:8/129: rename d9/c18 to d9/d17/d19/d27/c2e 0 2024-03-21T10:08:48.291 INFO:tasks.workunit.client.1.smithi174.stdout:1/209: dwrite d4/d7/d33/d43/f47 [0,4194304] 0 2024-03-21T10:08:48.291 INFO:tasks.workunit.client.1.smithi174.stdout:5/182: creat d1/d2f/d37/f3f x:0 0 0 2024-03-21T10:08:48.292 INFO:tasks.workunit.client.0.smithi153.stdout:3/358: write dd/f43 [3943786,50209] 0 2024-03-21T10:08:48.292 INFO:tasks.workunit.client.0.smithi153.stdout:1/333: write d6/f52 [799369,27637] 0 2024-03-21T10:08:48.293 INFO:tasks.workunit.client.1.smithi174.stdout:3/148: mknod d0/d12/d14/c36 0 2024-03-21T10:08:48.294 INFO:tasks.workunit.client.0.smithi153.stdout:0/303: unlink d4/db/c1e 0 2024-03-21T10:08:48.296 INFO:tasks.workunit.client.0.smithi153.stdout:4/399: symlink d11/d2d/d6b/l7c 0 2024-03-21T10:08:48.299 INFO:tasks.workunit.client.0.smithi153.stdout:7/316: creat d4/d2a/d40/d50/f69 x:0 0 0 2024-03-21T10:08:48.299 INFO:tasks.workunit.client.1.smithi174.stdout:4/222: mknod d4/d21/d31/d3a/c4f 0 2024-03-21T10:08:48.299 INFO:tasks.workunit.client.0.smithi153.stdout:1/334: fsync d6/d32/f5a 0 2024-03-21T10:08:48.300 INFO:tasks.workunit.client.1.smithi174.stdout:0/181: mkdir d9/d21/d31 0 2024-03-21T10:08:48.302 INFO:tasks.workunit.client.1.smithi174.stdout:9/232: mkdir d2/d11/d1d/d4b 0 2024-03-21T10:08:48.302 INFO:tasks.workunit.client.0.smithi153.stdout:9/363: getdents d1/d43/d44/d46 0 2024-03-21T10:08:48.306 INFO:tasks.workunit.client.0.smithi153.stdout:4/400: fsync d11/d2d/f36 0 2024-03-21T10:08:48.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:48 smithi153 ceph-mon[29117]: pgmap v10: 65 pgs: 65 active+clean; 924 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 9.5 MiB/s rd, 96 MiB/s wr, 217 op/s 2024-03-21T10:08:48.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:48 smithi153 ceph-mon[29117]: mgrmap e29: smithi153.tluxni(active, since 14s), standbys: smithi174.qpcwat 2024-03-21T10:08:48.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:48 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mgr metadata", "who": "smithi174.qpcwat", "id": "smithi174.qpcwat"}]: dispatch 2024-03-21T10:08:48.309 INFO:tasks.workunit.client.1.smithi174.stdout:2/208: rename d5/d25/c2c to d5/d25/d27/d2d/c4d 0 2024-03-21T10:08:48.310 INFO:tasks.workunit.client.1.smithi174.stdout:1/210: dread d4/fb [0,4194304] 0 2024-03-21T10:08:48.310 INFO:tasks.workunit.client.1.smithi174.stdout:0/182: rename d9/d1c to d9/d1c/d32 22 2024-03-21T10:08:48.310 INFO:tasks.workunit.client.1.smithi174.stdout:8/130: creat d9/d17/d19/f2f x:0 0 0 2024-03-21T10:08:48.312 INFO:tasks.workunit.client.1.smithi174.stdout:5/183: creat d1/d16/f40 x:0 0 0 2024-03-21T10:08:48.312 INFO:tasks.workunit.client.1.smithi174.stdout:0/183: read f4 [25645,120631] 0 2024-03-21T10:08:48.312 INFO:tasks.workunit.client.1.smithi174.stdout:2/209: read - d5/d25/d27/d44/f48 zero size 2024-03-21T10:08:48.312 INFO:tasks.workunit.client.0.smithi153.stdout:4/401: chown f1 83465883 1 2024-03-21T10:08:48.312 INFO:tasks.workunit.client.0.smithi153.stdout:2/439: dwrite dd/d15/d18/d2c/f3f [0,4194304] 0 2024-03-21T10:08:48.313 INFO:tasks.workunit.client.1.smithi174.stdout:8/131: write d9/d17/f20 [79454,19338] 0 2024-03-21T10:08:48.314 INFO:tasks.workunit.client.0.smithi153.stdout:7/317: unlink d4/d14/l15 0 2024-03-21T10:08:48.319 INFO:tasks.workunit.client.1.smithi174.stdout:4/223: creat d4/d19/f50 x:0 0 0 2024-03-21T10:08:48.320 INFO:tasks.workunit.client.1.smithi174.stdout:9/233: symlink d2/d24/l4c 0 2024-03-21T10:08:48.322 INFO:tasks.workunit.client.0.smithi153.stdout:9/364: creat d1/d43/d44/d46/d1c/d61/f7e x:0 0 0 2024-03-21T10:08:48.326 INFO:tasks.workunit.client.1.smithi174.stdout:9/234: read d2/f20 [1040686,48927] 0 2024-03-21T10:08:48.330 INFO:tasks.workunit.client.1.smithi174.stdout:5/184: mkdir d1/d16/d1c/d41 0 2024-03-21T10:08:48.332 INFO:tasks.workunit.client.1.smithi174.stdout:2/210: creat d5/d25/d27/f4e x:0 0 0 2024-03-21T10:08:48.334 INFO:tasks.workunit.client.1.smithi174.stdout:4/224: unlink d4/c32 0 2024-03-21T10:08:48.335 INFO:tasks.workunit.client.1.smithi174.stdout:6/132: dread d0/d3/d5/df/d16/f1a [0,4194304] 0 2024-03-21T10:08:48.336 INFO:tasks.workunit.client.0.smithi153.stdout:2/440: read - dd/d15/d18/d2c/d4b/d56/d63/f8b zero size 2024-03-21T10:08:48.336 INFO:tasks.workunit.client.0.smithi153.stdout:6/577: dwrite d1f/d2b/f53 [0,4194304] 0 2024-03-21T10:08:48.337 INFO:tasks.workunit.client.1.smithi174.stdout:8/132: rename d9/d17/d19/f24 to d9/f30 0 2024-03-21T10:08:48.337 INFO:tasks.workunit.client.1.smithi174.stdout:6/133: dread - d0/d3/d5/df/d16/f27 zero size 2024-03-21T10:08:48.340 INFO:tasks.workunit.client.1.smithi174.stdout:5/185: mknod d1/d16/d2e/d31/c42 0 2024-03-21T10:08:48.340 INFO:tasks.workunit.client.1.smithi174.stdout:2/211: dread d5/d25/d27/d2d/f41 [0,4194304] 0 2024-03-21T10:08:48.341 INFO:tasks.workunit.client.0.smithi153.stdout:9/365: symlink d1/l7f 0 2024-03-21T10:08:48.344 INFO:tasks.workunit.client.1.smithi174.stdout:9/235: mknod d2/d11/d1d/d4b/c4d 0 2024-03-21T10:08:48.344 INFO:tasks.workunit.client.1.smithi174.stdout:3/149: getdents d0 0 2024-03-21T10:08:48.347 INFO:tasks.workunit.client.0.smithi153.stdout:5/399: dread d4/d8/d3b/f4b [0,4194304] 0 2024-03-21T10:08:48.349 INFO:tasks.workunit.client.1.smithi174.stdout:4/225: rename d4/d19/f1c to d4/d21/d24/d28/f51 0 2024-03-21T10:08:48.349 INFO:tasks.workunit.client.0.smithi153.stdout:0/304: dwrite d4/f6 [0,4194304] 0 2024-03-21T10:08:48.350 INFO:tasks.workunit.client.1.smithi174.stdout:6/134: creat d0/d3/d5/d6/f2d x:0 0 0 2024-03-21T10:08:48.351 INFO:tasks.workunit.client.1.smithi174.stdout:2/212: unlink d5/d8/f28 0 2024-03-21T10:08:48.354 INFO:tasks.workunit.client.1.smithi174.stdout:3/150: rename d0/c28 to d0/c37 0 2024-03-21T10:08:48.357 INFO:tasks.workunit.client.0.smithi153.stdout:3/359: dwrite dd/f25 [4194304,4194304] 0 2024-03-21T10:08:48.357 INFO:tasks.workunit.client.0.smithi153.stdout:9/366: chown d1/d43/d44/d46/f37 2071491890 1 2024-03-21T10:08:48.357 INFO:tasks.workunit.client.0.smithi153.stdout:3/360: write f8 [107771,107006] 0 2024-03-21T10:08:48.357 INFO:tasks.workunit.client.0.smithi153.stdout:0/305: chown d4/d16/d27/l5a 4127281 1 2024-03-21T10:08:48.358 INFO:tasks.workunit.client.0.smithi153.stdout:5/400: dread - d4/f75 zero size 2024-03-21T10:08:48.360 INFO:tasks.workunit.client.1.smithi174.stdout:5/186: rename d1/d16/l17 to d1/d2f/d37/l43 0 2024-03-21T10:08:48.364 INFO:tasks.workunit.client.1.smithi174.stdout:4/226: creat d4/f52 x:0 0 0 2024-03-21T10:08:48.365 INFO:tasks.workunit.client.1.smithi174.stdout:9/236: truncate d2/d11/d1d/f41 550772 0 2024-03-21T10:08:48.365 INFO:tasks.workunit.client.0.smithi153.stdout:0/306: readlink d4/d16/l59 0 2024-03-21T10:08:48.366 INFO:tasks.workunit.client.0.smithi153.stdout:7/318: rmdir d4/d2a/d40/d50 39 2024-03-21T10:08:48.366 INFO:tasks.workunit.client.1.smithi174.stdout:4/227: dread - d4/f5 zero size 2024-03-21T10:08:48.368 INFO:tasks.workunit.client.0.smithi153.stdout:3/361: mknod dd/c7d 0 2024-03-21T10:08:48.369 INFO:tasks.workunit.client.0.smithi153.stdout:4/402: truncate d11/f1e 2680017 0 2024-03-21T10:08:48.369 INFO:tasks.workunit.client.0.smithi153.stdout:9/367: fsync d1/d43/d44/d46/d1c/f3d 0 2024-03-21T10:08:48.373 INFO:tasks.workunit.client.1.smithi174.stdout:0/184: dread d9/d1c/f20 [0,4194304] 0 2024-03-21T10:08:48.374 INFO:tasks.workunit.client.0.smithi153.stdout:6/578: write f1b [2875253,43255] 0 2024-03-21T10:08:48.375 INFO:tasks.workunit.client.0.smithi153.stdout:4/403: read d11/d18/f28 [504181,9007] 0 2024-03-21T10:08:48.375 INFO:tasks.workunit.client.1.smithi174.stdout:0/185: chown d9/d21/f2b 21812649 1 2024-03-21T10:08:48.375 INFO:tasks.workunit.client.0.smithi153.stdout:5/401: fsync d4/d5/de/d38/f64 0 2024-03-21T10:08:48.376 INFO:tasks.workunit.client.1.smithi174.stdout:6/135: creat d0/d3/d5/df/f2e x:0 0 0 2024-03-21T10:08:48.378 INFO:tasks.workunit.client.0.smithi153.stdout:0/307: stat d4/d16/d19/c3f 0 2024-03-21T10:08:48.378 INFO:tasks.workunit.client.1.smithi174.stdout:9/237: unlink d2/d11/c32 0 2024-03-21T10:08:48.380 INFO:tasks.workunit.client.0.smithi153.stdout:1/335: dread d6/f9 [0,4194304] 0 2024-03-21T10:08:48.381 INFO:tasks.workunit.client.1.smithi174.stdout:6/136: dread - d0/d3/d5/df/d16/f27 zero size 2024-03-21T10:08:48.384 INFO:tasks.workunit.client.1.smithi174.stdout:0/186: creat d9/f33 x:0 0 0 2024-03-21T10:08:48.387 INFO:tasks.workunit.client.0.smithi153.stdout:3/362: rename dd/d1e/d2f/d35/d66 to dd/d7e 0 2024-03-21T10:08:48.387 INFO:tasks.workunit.client.0.smithi153.stdout:9/368: write d1/d43/d44/d46/d1c/f3d [3697360,91956] 0 2024-03-21T10:08:48.387 INFO:tasks.workunit.client.0.smithi153.stdout:1/336: chown d6/d3a/d3c/d50/d51 107 1 2024-03-21T10:08:48.387 INFO:tasks.workunit.client.1.smithi174.stdout:4/228: read d4/d19/f23 [1021607,120250] 0 2024-03-21T10:08:48.390 INFO:tasks.workunit.client.0.smithi153.stdout:8/381: dread d5/d9/d43/f2b [0,4194304] 0 2024-03-21T10:08:48.391 INFO:tasks.workunit.client.1.smithi174.stdout:4/229: read - d4/f52 zero size 2024-03-21T10:08:48.391 INFO:tasks.workunit.client.1.smithi174.stdout:9/238: link d2/d11/f4a d2/d11/d1d/f4e 0 2024-03-21T10:08:48.392 INFO:tasks.workunit.client.0.smithi153.stdout:6/579: creat d1f/d2b/d33/d40/d4f/fce x:0 0 0 2024-03-21T10:08:48.393 INFO:tasks.workunit.client.0.smithi153.stdout:4/404: creat d11/d5e/f7d x:0 0 0 2024-03-21T10:08:48.395 INFO:tasks.workunit.client.1.smithi174.stdout:5/187: dwrite d1/fd [0,4194304] 0 2024-03-21T10:08:48.396 INFO:tasks.workunit.client.0.smithi153.stdout:2/441: dwrite dd/f2e [0,4194304] 0 2024-03-21T10:08:48.396 INFO:tasks.workunit.client.0.smithi153.stdout:5/402: symlink d4/d8/d5a/d5d/l7d 0 2024-03-21T10:08:48.397 INFO:tasks.workunit.client.1.smithi174.stdout:2/213: getdents d5/d25 0 2024-03-21T10:08:48.397 INFO:tasks.workunit.client.1.smithi174.stdout:1/211: write d4/d7/d33/d43/f47 [4489545,126869] 0 2024-03-21T10:08:48.398 INFO:tasks.workunit.client.0.smithi153.stdout:3/363: mkdir dd/d7e/d73/d7f 0 2024-03-21T10:08:48.399 INFO:tasks.workunit.client.0.smithi153.stdout:9/369: write d1/d43/d44/d46/d1c/d24/f38 [706803,92250] 0 2024-03-21T10:08:48.401 INFO:tasks.workunit.client.1.smithi174.stdout:0/187: creat d9/d21/d31/f34 x:0 0 0 2024-03-21T10:08:48.402 INFO:tasks.workunit.client.1.smithi174.stdout:0/188: readlink l3 0 2024-03-21T10:08:48.405 INFO:tasks.workunit.client.1.smithi174.stdout:7/156: dread d0/fa [0,4194304] 0 2024-03-21T10:08:48.410 INFO:tasks.workunit.client.0.smithi153.stdout:1/337: rmdir d6/d10/d2c 39 2024-03-21T10:08:48.411 INFO:tasks.workunit.client.1.smithi174.stdout:2/214: write d5/f38 [427915,71331] 0 2024-03-21T10:08:48.417 INFO:tasks.workunit.client.1.smithi174.stdout:1/212: chown d4/d7/l14 619979 1 2024-03-21T10:08:48.417 INFO:tasks.workunit.client.0.smithi153.stdout:6/580: fsync d1f/d2b/f89 0 2024-03-21T10:08:48.419 INFO:tasks.workunit.client.0.smithi153.stdout:5/403: creat d4/d5/df/d5e/d69/f7e x:0 0 0 2024-03-21T10:08:48.419 INFO:tasks.workunit.client.0.smithi153.stdout:2/442: mkdir dd/d15/d18/d2c/d4b/d56/d63/d8d/d8e/d95 0 2024-03-21T10:08:48.420 INFO:tasks.workunit.client.0.smithi153.stdout:3/364: mkdir dd/d41/d80 0 2024-03-21T10:08:48.420 INFO:tasks.workunit.client.0.smithi153.stdout:4/405: dwrite d11/d18/f77 [0,4194304] 0 2024-03-21T10:08:48.421 INFO:tasks.workunit.client.0.smithi153.stdout:7/319: getdents d4/d11 0 2024-03-21T10:08:48.421 INFO:tasks.workunit.client.1.smithi174.stdout:4/230: dwrite d4/d21/f49 [0,4194304] 0 2024-03-21T10:08:48.426 INFO:tasks.workunit.client.1.smithi174.stdout:5/188: dwrite d1/d16/f40 [0,4194304] 0 2024-03-21T10:08:48.432 INFO:tasks.workunit.client.1.smithi174.stdout:0/189: mkdir d9/d35 0 2024-03-21T10:08:48.436 INFO:tasks.workunit.client.0.smithi153.stdout:7/320: dread d4/d14/f4b [0,4194304] 0 2024-03-21T10:08:48.436 INFO:tasks.workunit.client.0.smithi153.stdout:6/581: mkdir d1f/d99/dbb/dcf 0 2024-03-21T10:08:48.438 INFO:tasks.workunit.client.1.smithi174.stdout:9/239: dread d2/f4 [0,4194304] 0 2024-03-21T10:08:48.439 INFO:tasks.workunit.client.0.smithi153.stdout:0/308: getdents d4/d16/d1f/d22 0 2024-03-21T10:08:48.439 INFO:tasks.workunit.client.0.smithi153.stdout:3/365: symlink dd/d1e/d6b/l81 0 2024-03-21T10:08:48.441 INFO:tasks.workunit.client.0.smithi153.stdout:6/582: fsync d1f/f36 0 2024-03-21T10:08:48.442 INFO:tasks.workunit.client.0.smithi153.stdout:4/406: fdatasync d11/d18/f4c 0 2024-03-21T10:08:48.443 INFO:tasks.workunit.client.0.smithi153.stdout:1/338: unlink d6/f60 0 2024-03-21T10:08:48.443 INFO:tasks.workunit.client.1.smithi174.stdout:2/215: dread f2 [0,4194304] 0 2024-03-21T10:08:48.449 INFO:tasks.workunit.client.0.smithi153.stdout:8/382: dwrite d5/d9/d43/dd/d2d/f31 [0,4194304] 0 2024-03-21T10:08:48.449 INFO:tasks.workunit.client.0.smithi153.stdout:6/583: write d1f/d99/fb7 [528551,22436] 0 2024-03-21T10:08:48.449 INFO:tasks.workunit.client.0.smithi153.stdout:4/407: write d11/d18/d50/f53 [130990,73293] 0 2024-03-21T10:08:48.450 INFO:tasks.workunit.client.1.smithi174.stdout:9/240: write d2/d24/f2f [721611,104857] 0 2024-03-21T10:08:48.457 INFO:tasks.workunit.client.1.smithi174.stdout:4/231: dwrite d4/f52 [0,4194304] 0 2024-03-21T10:08:48.458 INFO:tasks.workunit.client.1.smithi174.stdout:5/189: rmdir d1/d16 39 2024-03-21T10:08:48.460 INFO:tasks.workunit.client.0.smithi153.stdout:8/383: read - d5/d9/f4b zero size 2024-03-21T10:08:48.460 INFO:tasks.workunit.client.0.smithi153.stdout:5/404: fsync d4/d5/de/d38/f64 0 2024-03-21T10:08:48.460 INFO:tasks.workunit.client.0.smithi153.stdout:7/321: creat d4/d1b/d20/f6a x:0 0 0 2024-03-21T10:08:48.461 INFO:tasks.workunit.client.0.smithi153.stdout:5/405: stat d4/d5/d16/d24/f53 0 2024-03-21T10:08:48.462 INFO:tasks.workunit.client.0.smithi153.stdout:0/309: mkdir d4/d16/d1f/d22/d61 0 2024-03-21T10:08:48.462 INFO:tasks.workunit.client.1.smithi174.stdout:3/151: truncate d0/d12/d14/d1a/f23 3927156 0 2024-03-21T10:08:48.463 INFO:tasks.workunit.client.0.smithi153.stdout:3/366: symlink dd/d1e/d2f/d35/l82 0 2024-03-21T10:08:48.464 INFO:tasks.workunit.client.0.smithi153.stdout:8/384: chown d5/f2f 2782458 1 2024-03-21T10:08:48.465 INFO:tasks.workunit.client.0.smithi153.stdout:5/406: chown d4/d5/de/d2f/f41 4726321 1 2024-03-21T10:08:48.466 INFO:tasks.workunit.client.1.smithi174.stdout:7/157: dwrite d0/d4/f8 [0,4194304] 0 2024-03-21T10:08:48.466 INFO:tasks.workunit.client.1.smithi174.stdout:1/213: creat d4/f4a x:0 0 0 2024-03-21T10:08:48.467 INFO:tasks.workunit.client.0.smithi153.stdout:5/407: chown d4/d5/f57 2 1 2024-03-21T10:08:48.467 INFO:tasks.workunit.client.1.smithi174.stdout:9/241: symlink d2/d24/d2e/l4f 0 2024-03-21T10:08:48.468 INFO:tasks.workunit.client.0.smithi153.stdout:2/443: mknod dd/d15/d18/d2c/d4b/d56/d63/d8d/d8e/d91/c96 0 2024-03-21T10:08:48.469 INFO:tasks.workunit.client.0.smithi153.stdout:6/584: creat d1f/d2b/d33/d40/d4f/d66/fd0 x:0 0 0 2024-03-21T10:08:48.471 INFO:tasks.workunit.client.0.smithi153.stdout:3/367: symlink dd/d7e/l83 0 2024-03-21T10:08:48.473 INFO:tasks.workunit.client.1.smithi174.stdout:7/158: dread - d0/d4/d10/f21 zero size 2024-03-21T10:08:48.473 INFO:tasks.workunit.client.1.smithi174.stdout:2/216: dwrite d5/d25/d27/f3b [0,4194304] 0 2024-03-21T10:08:48.474 INFO:tasks.workunit.client.1.smithi174.stdout:4/232: creat d4/f53 x:0 0 0 2024-03-21T10:08:48.475 INFO:tasks.workunit.client.0.smithi153.stdout:5/408: rename d4/d8/c27 to d4/d5/de/d38/c7f 0 2024-03-21T10:08:48.479 INFO:tasks.workunit.client.1.smithi174.stdout:1/214: mknod d4/d7/d32/c4b 0 2024-03-21T10:08:48.479 INFO:tasks.workunit.client.1.smithi174.stdout:1/215: stat d4/d7/d15/f41 0 2024-03-21T10:08:48.481 INFO:tasks.workunit.client.0.smithi153.stdout:0/310: creat d4/f62 x:0 0 0 2024-03-21T10:08:48.486 INFO:tasks.workunit.client.0.smithi153.stdout:1/339: rmdir d6/d3a/d4e 0 2024-03-21T10:08:48.486 INFO:tasks.workunit.client.0.smithi153.stdout:5/409: chown d4/d5/f57 1872707819 1 2024-03-21T10:08:48.491 INFO:tasks.workunit.client.1.smithi174.stdout:2/217: dread - d5/d25/d27/f40 zero size 2024-03-21T10:08:48.491 INFO:tasks.workunit.client.1.smithi174.stdout:7/159: truncate d0/d4/f17 525157 0 2024-03-21T10:08:48.493 INFO:tasks.workunit.client.0.smithi153.stdout:0/311: stat d4/d16/d19/c45 0 2024-03-21T10:08:48.494 INFO:tasks.workunit.client.0.smithi153.stdout:5/410: chown d4/d5/de/d2f/l37 122 1 2024-03-21T10:08:48.499 INFO:tasks.workunit.client.0.smithi153.stdout:1/340: link d6/d10/l1a d6/d3a/d3c/d50/l6b 0 2024-03-21T10:08:48.502 INFO:tasks.workunit.client.1.smithi174.stdout:2/218: mkdir d5/d25/d27/d2e/d4f 0 2024-03-21T10:08:48.504 INFO:tasks.workunit.client.0.smithi153.stdout:0/312: creat d4/d16/d3e/f63 x:0 0 0 2024-03-21T10:08:48.504 INFO:tasks.workunit.client.0.smithi153.stdout:5/411: chown d4/d5/d16/l71 156514398 1 2024-03-21T10:08:48.507 INFO:tasks.workunit.client.1.smithi174.stdout:3/152: dwrite d0/d12/d14/d1a/f29 [0,4194304] 0 2024-03-21T10:08:48.509 INFO:tasks.workunit.client.0.smithi153.stdout:7/322: dwrite d4/d11/d24/f58 [0,4194304] 0 2024-03-21T10:08:48.510 INFO:tasks.workunit.client.0.smithi153.stdout:5/412: fsync d4/d5/de/f22 0 2024-03-21T10:08:48.514 INFO:tasks.workunit.client.0.smithi153.stdout:0/313: mknod d4/d16/c64 0 2024-03-21T10:08:48.514 INFO:tasks.workunit.client.1.smithi174.stdout:2/219: symlink d5/d25/d27/d44/l50 0 2024-03-21T10:08:48.515 INFO:tasks.workunit.client.0.smithi153.stdout:1/341: read d6/d13/f2b [3158126,30960] 0 2024-03-21T10:08:48.517 INFO:tasks.workunit.client.1.smithi174.stdout:4/233: dwrite d4/d21/f27 [0,4194304] 0 2024-03-21T10:08:48.518 INFO:tasks.workunit.client.0.smithi153.stdout:5/413: read d4/d5/de/d2f/f41 [288776,87846] 0 2024-03-21T10:08:48.519 INFO:tasks.workunit.client.1.smithi174.stdout:6/137: dread d0/d3/d5/fe [0,4194304] 0 2024-03-21T10:08:48.519 INFO:tasks.workunit.client.1.smithi174.stdout:2/220: write d5/f26 [1384505,62247] 0 2024-03-21T10:08:48.520 INFO:tasks.workunit.client.1.smithi174.stdout:0/190: write d9/f19 [4997552,62903] 0 2024-03-21T10:08:48.520 INFO:tasks.workunit.client.0.smithi153.stdout:3/368: dwrite dd/d1e/d2f/d35/f5a [0,4194304] 0 2024-03-21T10:08:48.524 INFO:tasks.workunit.client.1.smithi174.stdout:7/160: rename d0/c34 to d0/d4/d10/c39 0 2024-03-21T10:08:48.524 INFO:tasks.workunit.client.1.smithi174.stdout:9/242: rmdir d2/d24/d2e 39 2024-03-21T10:08:48.525 INFO:tasks.workunit.client.0.smithi153.stdout:9/370: truncate d1/d43/d44/d46/d1c/d24/f38 1539466 0 2024-03-21T10:08:48.526 INFO:tasks.workunit.client.1.smithi174.stdout:1/216: dwrite d4/d5/f18 [4194304,4194304] 0 2024-03-21T10:08:48.526 INFO:tasks.workunit.client.1.smithi174.stdout:4/234: read d4/fe [206404,29364] 0 2024-03-21T10:08:48.528 INFO:tasks.workunit.client.1.smithi174.stdout:6/138: fsync d0/d3/d5/df/d16/f27 0 2024-03-21T10:08:48.528 INFO:tasks.workunit.client.0.smithi153.stdout:5/414: fsync d4/d8/d5a/f61 0 2024-03-21T10:08:48.530 INFO:tasks.workunit.client.1.smithi174.stdout:7/161: stat d0/d4/d38 0 2024-03-21T10:08:48.530 INFO:tasks.workunit.client.1.smithi174.stdout:8/133: sync 2024-03-21T10:08:48.530 INFO:tasks.workunit.client.1.smithi174.stdout:4/235: chown d4/f15 16 1 2024-03-21T10:08:48.533 INFO:tasks.workunit.client.1.smithi174.stdout:4/236: dread - d4/d21/d24/f4d zero size 2024-03-21T10:08:48.538 INFO:tasks.workunit.client.1.smithi174.stdout:4/237: write d4/d19/f41 [124225,128053] 0 2024-03-21T10:08:48.541 INFO:tasks.workunit.client.0.smithi153.stdout:5/415: rename d4/d5/d16/c79 to d4/d8/c80 0 2024-03-21T10:08:48.542 INFO:tasks.workunit.client.1.smithi174.stdout:2/221: symlink d5/d8/d2f/d4b/l51 0 2024-03-21T10:08:48.543 INFO:tasks.workunit.client.0.smithi153.stdout:1/342: getdents d6/d32/d3e/d4f 0 2024-03-21T10:08:48.550 INFO:tasks.workunit.client.1.smithi174.stdout:5/190: dwrite d1/d16/d1c/f23 [0,4194304] 0 2024-03-21T10:08:48.551 INFO:tasks.workunit.client.1.smithi174.stdout:6/139: rename d0/d3/f8 to d0/d3/d2a/f2f 0 2024-03-21T10:08:48.554 INFO:tasks.workunit.client.1.smithi174.stdout:0/191: creat d9/d35/f36 x:0 0 0 2024-03-21T10:08:48.555 INFO:tasks.workunit.client.0.smithi153.stdout:5/416: fdatasync d4/d5/df/d5e/d69/f7e 0 2024-03-21T10:08:48.563 INFO:tasks.workunit.client.1.smithi174.stdout:9/243: symlink d2/d24/d2e/l50 0 2024-03-21T10:08:48.567 INFO:tasks.workunit.client.0.smithi153.stdout:8/385: dread d5/f8 [0,4194304] 0 2024-03-21T10:08:48.572 INFO:tasks.workunit.client.1.smithi174.stdout:3/153: write d0/d12/d14/d1a/d27/f2c [128896,43491] 0 2024-03-21T10:08:48.573 INFO:tasks.workunit.client.1.smithi174.stdout:0/192: readlink d9/d1c/l30 0 2024-03-21T10:08:48.575 INFO:tasks.workunit.client.1.smithi174.stdout:1/217: link l3 d4/d7/d32/l4c 0 2024-03-21T10:08:48.577 INFO:tasks.workunit.client.1.smithi174.stdout:6/140: dread d0/d3/d5/df/d16/f1a [0,4194304] 0 2024-03-21T10:08:48.580 INFO:tasks.workunit.client.1.smithi174.stdout:8/134: getdents d9/d17/d19 0 2024-03-21T10:08:48.586 INFO:tasks.workunit.client.0.smithi153.stdout:0/314: dwrite d4/f37 [0,4194304] 0 2024-03-21T10:08:48.588 INFO:tasks.workunit.client.0.smithi153.stdout:5/417: read d4/d5/df/f58 [3088690,66569] 0 2024-03-21T10:08:48.589 INFO:tasks.workunit.client.1.smithi174.stdout:3/154: rename d0/d12/d14/d1a/f2f to d0/d12/d14/d10/f38 0 2024-03-21T10:08:48.589 INFO:tasks.workunit.client.1.smithi174.stdout:8/135: unlink d9/fc 0 2024-03-21T10:08:48.591 INFO:tasks.workunit.client.0.smithi153.stdout:9/371: dwrite d1/d43/d44/d46/d1c/d2e/f62 [0,4194304] 0 2024-03-21T10:08:48.592 INFO:tasks.workunit.client.1.smithi174.stdout:8/136: fdatasync f5 0 2024-03-21T10:08:48.594 INFO:tasks.workunit.client.0.smithi153.stdout:6/585: dread d1f/d2b/d33/d40/d5f/f80 [0,4194304] 0 2024-03-21T10:08:48.594 INFO:tasks.workunit.client.0.smithi153.stdout:0/315: chown d4/d16/d1f/l2a 88357 1 2024-03-21T10:08:48.595 INFO:tasks.workunit.client.0.smithi153.stdout:5/418: dread - d4/d8/f6c zero size 2024-03-21T10:08:48.597 INFO:tasks.workunit.client.1.smithi174.stdout:8/137: mkdir d9/d17/d19/d31 0 2024-03-21T10:08:48.597 INFO:tasks.workunit.client.0.smithi153.stdout:2/444: dwrite dd/d15/d18/d2c/f39 [0,4194304] 0 2024-03-21T10:08:48.600 INFO:tasks.workunit.client.0.smithi153.stdout:0/316: write d4/d16/d27/f38 [5082765,64316] 0 2024-03-21T10:08:48.601 INFO:tasks.workunit.client.1.smithi174.stdout:1/218: dwrite d4/d7/d15/f2f [0,4194304] 0 2024-03-21T10:08:48.601 INFO:tasks.workunit.client.1.smithi174.stdout:8/138: creat d9/d2a/f32 x:0 0 0 2024-03-21T10:08:48.610 INFO:tasks.workunit.client.1.smithi174.stdout:8/139: mkdir d9/d17/d19/d33 0 2024-03-21T10:08:48.610 INFO:tasks.workunit.client.1.smithi174.stdout:1/219: symlink d4/d7/d28/l4d 0 2024-03-21T10:08:48.612 INFO:tasks.workunit.client.1.smithi174.stdout:8/140: chown d9/l10 145532131 1 2024-03-21T10:08:48.612 INFO:tasks.workunit.client.0.smithi153.stdout:8/386: symlink d5/d9/d43/dd/d20/d26/d47/l78 0 2024-03-21T10:08:48.613 INFO:tasks.workunit.client.1.smithi174.stdout:1/220: fsync d4/d7/d33/f36 0 2024-03-21T10:08:48.613 INFO:tasks.workunit.client.0.smithi153.stdout:6/586: creat d1f/d2f/fd1 x:0 0 0 2024-03-21T10:08:48.615 INFO:tasks.workunit.client.0.smithi153.stdout:5/419: write d4/d8/d3b/f4b [4132778,51072] 0 2024-03-21T10:08:48.618 INFO:tasks.workunit.client.0.smithi153.stdout:1/343: truncate d6/d32/d3e/f42 864441 0 2024-03-21T10:08:48.619 INFO:tasks.workunit.client.0.smithi153.stdout:8/387: symlink d5/d9/l79 0 2024-03-21T10:08:48.621 INFO:tasks.workunit.client.1.smithi174.stdout:8/141: rename d9/d2a/f32 to d9/d2a/f34 0 2024-03-21T10:08:48.622 INFO:tasks.workunit.client.1.smithi174.stdout:1/221: symlink d4/d7/d33/l4e 0 2024-03-21T10:08:48.625 INFO:tasks.workunit.client.1.smithi174.stdout:8/142: mknod d9/d17/c35 0 2024-03-21T10:08:48.627 INFO:tasks.workunit.client.1.smithi174.stdout:1/222: mkdir d4/d7/d15/d4f 0 2024-03-21T10:08:48.627 INFO:tasks.workunit.client.1.smithi174.stdout:2/222: truncate d5/d25/d27/d2e/f3a 1765572 0 2024-03-21T10:08:48.628 INFO:tasks.workunit.client.1.smithi174.stdout:8/143: creat d9/f36 x:0 0 0 2024-03-21T10:08:48.629 INFO:tasks.workunit.client.0.smithi153.stdout:2/445: rename dd/d15/d18/f1a to dd/d15/f97 0 2024-03-21T10:08:48.629 INFO:tasks.workunit.client.1.smithi174.stdout:2/223: chown d5/d25/d27/f4e 4735874 1 2024-03-21T10:08:48.630 INFO:tasks.workunit.client.1.smithi174.stdout:5/191: truncate d1/d16/d2e/d31/f33 650243 0 2024-03-21T10:08:48.630 INFO:tasks.workunit.client.1.smithi174.stdout:3/155: rmdir d0/d12/d14/d1a 39 2024-03-21T10:08:48.631 INFO:tasks.workunit.client.1.smithi174.stdout:8/144: chown d9/d17/l22 39 1 2024-03-21T10:08:48.632 INFO:tasks.workunit.client.0.smithi153.stdout:7/323: dread d4/d11/d24/f28 [0,4194304] 0 2024-03-21T10:08:48.632 INFO:tasks.workunit.client.1.smithi174.stdout:5/192: fdatasync d1/d16/f40 0 2024-03-21T10:08:48.634 INFO:tasks.workunit.client.1.smithi174.stdout:1/223: dread d4/d5/f8 [0,4194304] 0 2024-03-21T10:08:48.635 INFO:tasks.workunit.client.1.smithi174.stdout:6/141: write d0/d3/d5/df/d16/f17 [1935071,16921] 0 2024-03-21T10:08:48.637 INFO:tasks.workunit.client.1.smithi174.stdout:8/145: mkdir d9/d37 0 2024-03-21T10:08:48.638 INFO:tasks.workunit.client.0.smithi153.stdout:5/420: getdents d4/d5/df/d67 0 2024-03-21T10:08:48.639 INFO:tasks.workunit.client.1.smithi174.stdout:3/156: chown d0/d12/d14/d10/f1b 57766294 1 2024-03-21T10:08:48.639 INFO:tasks.workunit.client.1.smithi174.stdout:0/193: truncate d9/f1d 2501340 0 2024-03-21T10:08:48.642 INFO:tasks.workunit.client.0.smithi153.stdout:2/446: mkdir dd/d15/d18/d2c/d33/d98 0 2024-03-21T10:08:48.644 INFO:tasks.workunit.client.0.smithi153.stdout:1/344: link d6/d3a/d3c/l4d d6/d32/d3e/l6c 0 2024-03-21T10:08:48.645 INFO:tasks.workunit.client.1.smithi174.stdout:6/142: symlink d0/d3/d5/d21/l30 0 2024-03-21T10:08:48.647 INFO:tasks.workunit.client.1.smithi174.stdout:9/244: dwrite d2/f44 [0,4194304] 0 2024-03-21T10:08:48.647 INFO:tasks.workunit.client.1.smithi174.stdout:3/157: symlink d0/l39 0 2024-03-21T10:08:48.650 INFO:tasks.workunit.client.1.smithi174.stdout:6/143: chown d0/d3/d5/fe 43551524 1 2024-03-21T10:08:48.651 INFO:tasks.workunit.client.0.smithi153.stdout:9/372: rename d1/d43/d44/d46/d1c/d4b/l6e to d1/l80 0 2024-03-21T10:08:48.651 INFO:tasks.workunit.client.1.smithi174.stdout:0/194: creat d9/d21/f37 x:0 0 0 2024-03-21T10:08:48.652 INFO:tasks.workunit.client.1.smithi174.stdout:6/144: fdatasync d0/d3/d5/df/d16/f27 0 2024-03-21T10:08:48.653 INFO:tasks.workunit.client.1.smithi174.stdout:2/224: link d5/d25/c35 d5/d25/d27/d2d/d45/c52 0 2024-03-21T10:08:48.655 INFO:tasks.workunit.client.1.smithi174.stdout:9/245: symlink d2/d24/d2e/l51 0 2024-03-21T10:08:48.656 INFO:tasks.workunit.client.1.smithi174.stdout:3/158: dread d0/d12/d14/d1a/d27/f2c [0,4194304] 0 2024-03-21T10:08:48.656 INFO:tasks.workunit.client.1.smithi174.stdout:1/224: dread d4/d7/d33/f36 [0,4194304] 0 2024-03-21T10:08:48.659 INFO:tasks.workunit.client.0.smithi153.stdout:2/447: unlink dd/d3b/d50/f73 0 2024-03-21T10:08:48.659 INFO:tasks.workunit.client.0.smithi153.stdout:5/421: truncate d4/d5/df/f30 2227082 0 2024-03-21T10:08:48.659 INFO:tasks.workunit.client.0.smithi153.stdout:6/587: dwrite f1a [0,4194304] 0 2024-03-21T10:08:48.662 INFO:tasks.workunit.client.1.smithi174.stdout:2/225: symlink d5/d25/d27/d2e/l53 0 2024-03-21T10:08:48.664 INFO:tasks.workunit.client.0.smithi153.stdout:8/388: dwrite d5/d9/d43/dd/d20/d26/f1f [0,4194304] 0 2024-03-21T10:08:48.672 INFO:tasks.workunit.client.1.smithi174.stdout:1/225: mknod d4/d7/d15/d4f/c50 0 2024-03-21T10:08:48.675 INFO:tasks.workunit.client.0.smithi153.stdout:0/317: dwrite d4/d16/d1f/f24 [0,4194304] 0 2024-03-21T10:08:48.678 INFO:tasks.workunit.client.1.smithi174.stdout:6/145: dwrite d0/d3/d5/df/f2e [0,4194304] 0 2024-03-21T10:08:48.680 INFO:tasks.workunit.client.0.smithi153.stdout:9/373: creat d1/d4e/d68/f81 x:0 0 0 2024-03-21T10:08:48.680 INFO:tasks.workunit.client.0.smithi153.stdout:5/422: creat d4/d5/df/d5e/f81 x:0 0 0 2024-03-21T10:08:48.685 INFO:tasks.workunit.client.1.smithi174.stdout:9/246: rename d2/l46 to d2/d24/l52 0 2024-03-21T10:08:48.689 INFO:tasks.workunit.client.0.smithi153.stdout:0/318: fsync d4/db/f13 0 2024-03-21T10:08:48.690 INFO:tasks.workunit.client.1.smithi174.stdout:1/226: mkdir d4/d7/d33/d43/d51 0 2024-03-21T10:08:48.690 INFO:tasks.workunit.client.0.smithi153.stdout:7/324: rename d4/d1b/d20/c4a to d4/d2a/d3b/d60/c6b 0 2024-03-21T10:08:48.690 INFO:tasks.workunit.client.0.smithi153.stdout:1/345: creat d6/d3a/d3c/d50/d51/f6d x:0 0 0 2024-03-21T10:08:48.699 INFO:tasks.workunit.client.1.smithi174.stdout:2/226: rename f2 to d5/d8/d2f/d4b/f54 0 2024-03-21T10:08:48.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:48 smithi174 ceph-mon[37496]: pgmap v10: 65 pgs: 65 active+clean; 924 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 9.5 MiB/s rd, 96 MiB/s wr, 217 op/s 2024-03-21T10:08:48.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:48 smithi174 ceph-mon[37496]: mgrmap e29: smithi153.tluxni(active, since 14s), standbys: smithi174.qpcwat 2024-03-21T10:08:48.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:48 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mgr metadata", "who": "smithi174.qpcwat", "id": "smithi174.qpcwat"}]: dispatch 2024-03-21T10:08:48.701 INFO:tasks.workunit.client.1.smithi174.stdout:5/193: truncate d1/d16/d1c/f26 2530521 0 2024-03-21T10:08:48.703 INFO:tasks.workunit.client.1.smithi174.stdout:1/227: symlink d4/d7/l52 0 2024-03-21T10:08:48.705 INFO:tasks.workunit.client.0.smithi153.stdout:9/374: mkdir d1/d82 0 2024-03-21T10:08:48.706 INFO:tasks.workunit.client.0.smithi153.stdout:0/319: symlink d4/l65 0 2024-03-21T10:08:48.708 INFO:tasks.workunit.client.1.smithi174.stdout:2/227: mkdir d5/d8/d55 0 2024-03-21T10:08:48.709 INFO:tasks.workunit.client.0.smithi153.stdout:6/588: dwrite f15 [4194304,4194304] 0 2024-03-21T10:08:48.710 INFO:tasks.workunit.client.1.smithi174.stdout:5/194: symlink d1/d2f/l44 0 2024-03-21T10:08:48.710 INFO:tasks.workunit.client.1.smithi174.stdout:1/228: mkdir d4/d7/d15/d53 0 2024-03-21T10:08:48.713 INFO:tasks.workunit.client.0.smithi153.stdout:5/423: link d4/d5/de/f3f d4/d5/df/d5e/f82 0 2024-03-21T10:08:48.713 INFO:tasks.workunit.client.1.smithi174.stdout:3/159: dread d0/d12/f17 [0,4194304] 0 2024-03-21T10:08:48.714 INFO:tasks.workunit.client.1.smithi174.stdout:8/146: dread d9/f1a [0,4194304] 0 2024-03-21T10:08:48.714 INFO:tasks.workunit.client.0.smithi153.stdout:2/448: dwrite dd/d3b/f5b [0,4194304] 0 2024-03-21T10:08:48.717 INFO:tasks.workunit.client.1.smithi174.stdout:6/146: dwrite d0/d3/d5/df/d16/f2b [0,4194304] 0 2024-03-21T10:08:48.718 INFO:tasks.workunit.client.1.smithi174.stdout:1/229: chown d4/d7/d15/f38 30257 1 2024-03-21T10:08:48.719 INFO:tasks.workunit.client.0.smithi153.stdout:8/389: dread d5/f8 [0,4194304] 0 2024-03-21T10:08:48.722 INFO:tasks.workunit.client.0.smithi153.stdout:7/325: creat d4/f6c x:0 0 0 2024-03-21T10:08:48.723 INFO:tasks.workunit.client.1.smithi174.stdout:3/160: write d0/d12/d14/d10/f31 [991871,107508] 0 2024-03-21T10:08:48.723 INFO:tasks.workunit.client.0.smithi153.stdout:0/320: mknod d4/d16/d3e/c66 0 2024-03-21T10:08:48.725 INFO:tasks.workunit.client.1.smithi174.stdout:5/195: dread d1/f2c [0,4194304] 0 2024-03-21T10:08:48.725 INFO:tasks.workunit.client.1.smithi174.stdout:6/147: symlink d0/d3/d5/df/l31 0 2024-03-21T10:08:48.738 INFO:tasks.workunit.client.1.smithi174.stdout:1/230: link d4/d7/d15/f2f d4/d7/d15/d53/f54 0 2024-03-21T10:08:48.739 INFO:tasks.workunit.client.1.smithi174.stdout:5/196: symlink d1/d16/l45 0 2024-03-21T10:08:48.739 INFO:tasks.workunit.client.0.smithi153.stdout:6/589: rmdir d1f/d2b/d33/d40/d51 39 2024-03-21T10:08:48.740 INFO:tasks.workunit.client.1.smithi174.stdout:2/228: dwrite d5/d8/d2f/d4b/f54 [0,4194304] 0 2024-03-21T10:08:48.741 INFO:tasks.workunit.client.1.smithi174.stdout:9/247: write d2/f1f [3382663,96155] 0 2024-03-21T10:08:48.742 INFO:tasks.workunit.client.0.smithi153.stdout:1/346: dwrite d6/d13/f5e [0,4194304] 0 2024-03-21T10:08:48.743 INFO:tasks.workunit.client.1.smithi174.stdout:1/231: write d4/d7/d33/f36 [2062770,55667] 0 2024-03-21T10:08:48.745 INFO:tasks.workunit.client.0.smithi153.stdout:9/375: fdatasync d1/d43/d44/d46/d1c/d24/f38 0 2024-03-21T10:08:48.746 INFO:tasks.workunit.client.0.smithi153.stdout:0/321: chown d4/db/c42 33242 1 2024-03-21T10:08:48.750 INFO:tasks.workunit.client.1.smithi174.stdout:1/232: fsync d4/d7/d33/f3e 0 2024-03-21T10:08:48.751 INFO:tasks.workunit.client.1.smithi174.stdout:5/197: unlink d1/d16/f27 0 2024-03-21T10:08:48.755 INFO:tasks.workunit.client.1.smithi174.stdout:0/195: truncate d9/d21/f29 1587072 0 2024-03-21T10:08:48.755 INFO:tasks.workunit.client.1.smithi174.stdout:5/198: chown d1/l5 156809 1 2024-03-21T10:08:48.757 INFO:tasks.workunit.client.1.smithi174.stdout:8/147: truncate d9/d17/f20 674465 0 2024-03-21T10:08:48.757 INFO:tasks.workunit.client.1.smithi174.stdout:6/148: dwrite d0/d3/d5/df/f2e [4194304,4194304] 0 2024-03-21T10:08:48.758 INFO:tasks.workunit.client.0.smithi153.stdout:9/376: write d1/d43/d44/d46/f1d [3810204,35290] 0 2024-03-21T10:08:48.760 INFO:tasks.workunit.client.1.smithi174.stdout:0/196: symlink d9/d1c/l38 0 2024-03-21T10:08:48.761 INFO:tasks.workunit.client.1.smithi174.stdout:5/199: read d1/f22 [98183,112974] 0 2024-03-21T10:08:48.763 INFO:tasks.workunit.client.1.smithi174.stdout:3/161: dwrite d0/d12/d14/d10/f1f [0,4194304] 0 2024-03-21T10:08:48.764 INFO:tasks.workunit.client.1.smithi174.stdout:9/248: creat d2/d11/f53 x:0 0 0 2024-03-21T10:08:48.765 INFO:tasks.workunit.client.0.smithi153.stdout:1/347: creat d6/d32/f6e x:0 0 0 2024-03-21T10:08:48.766 INFO:tasks.workunit.client.1.smithi174.stdout:8/148: write d9/d2a/f34 [151697,72561] 0 2024-03-21T10:08:48.767 INFO:tasks.workunit.client.1.smithi174.stdout:5/200: readlink d1/d16/l18 0 2024-03-21T10:08:48.768 INFO:tasks.workunit.client.1.smithi174.stdout:3/162: dread - d0/f8 zero size 2024-03-21T10:08:48.769 INFO:tasks.workunit.client.0.smithi153.stdout:2/449: rename dd/d3b/d50/c5a to dd/d15/d18/d2c/d4b/d56/d63/d8d/d8e/d91/c99 0 2024-03-21T10:08:48.773 INFO:tasks.workunit.client.0.smithi153.stdout:6/590: symlink d1f/d2b/d33/d40/d51/d98/ld2 0 2024-03-21T10:08:48.773 INFO:tasks.workunit.client.1.smithi174.stdout:5/201: write d1/d16/d1c/f25 [1542769,33674] 0 2024-03-21T10:08:48.775 INFO:tasks.workunit.client.0.smithi153.stdout:7/326: getdents d4/d19/d4d 0 2024-03-21T10:08:48.777 INFO:tasks.workunit.client.1.smithi174.stdout:8/149: write d9/f1c [741093,70528] 0 2024-03-21T10:08:48.780 INFO:tasks.workunit.client.1.smithi174.stdout:5/202: creat d1/d16/f46 x:0 0 0 2024-03-21T10:08:48.780 INFO:tasks.workunit.client.0.smithi153.stdout:1/348: chown d6/d3a/d3c/c55 9460 1 2024-03-21T10:08:48.781 INFO:tasks.workunit.client.1.smithi174.stdout:8/150: chown d9/f1a 4 1 2024-03-21T10:08:48.781 INFO:tasks.workunit.client.0.smithi153.stdout:9/377: dread d1/d43/d44/f3e [0,4194304] 0 2024-03-21T10:08:48.781 INFO:tasks.workunit.client.0.smithi153.stdout:5/424: dwrite d4/d8/d5a/d5d/f55 [0,4194304] 0 2024-03-21T10:08:48.781 INFO:tasks.workunit.client.0.smithi153.stdout:6/591: write d1f/f28 [4219245,76801] 0 2024-03-21T10:08:48.782 INFO:tasks.workunit.client.1.smithi174.stdout:0/197: rename d9/ca to d9/c39 0 2024-03-21T10:08:48.785 INFO:tasks.workunit.client.1.smithi174.stdout:3/163: rename d0/d12/d14/d1a/f29 to d0/d12/d14/f3a 0 2024-03-21T10:08:48.787 INFO:tasks.workunit.client.0.smithi153.stdout:8/390: dwrite d5/d9/d43/dd/d20/d26/d24/d52/f64 [0,4194304] 0 2024-03-21T10:08:48.791 INFO:tasks.workunit.client.1.smithi174.stdout:8/151: creat d9/d17/d19/d27/f38 x:0 0 0 2024-03-21T10:08:48.792 INFO:tasks.workunit.client.0.smithi153.stdout:5/425: chown d4/d5/df/f58 7002151 1 2024-03-21T10:08:48.796 INFO:tasks.workunit.client.1.smithi174.stdout:8/152: write d9/f15 [2795455,77060] 0 2024-03-21T10:08:48.797 INFO:tasks.workunit.client.0.smithi153.stdout:1/349: rename l1 to d6/d10/d35/l6f 0 2024-03-21T10:08:48.798 INFO:tasks.workunit.client.1.smithi174.stdout:3/164: mknod d0/d12/c3b 0 2024-03-21T10:08:48.799 INFO:tasks.workunit.client.0.smithi153.stdout:9/378: symlink d1/d43/d44/d46/d1c/d4b/l83 0 2024-03-21T10:08:48.801 INFO:tasks.workunit.client.0.smithi153.stdout:9/379: readlink d1/d4/l5a 0 2024-03-21T10:08:48.801 INFO:tasks.workunit.client.1.smithi174.stdout:6/149: dwrite d0/d3/d5/df/d16/f2b [0,4194304] 0 2024-03-21T10:08:48.808 INFO:tasks.workunit.client.1.smithi174.stdout:0/198: link d9/d1c/l38 d9/d21/d31/l3a 0 2024-03-21T10:08:48.809 INFO:tasks.workunit.client.1.smithi174.stdout:9/249: dwrite d2/d24/d2e/f48 [0,4194304] 0 2024-03-21T10:08:48.810 INFO:tasks.workunit.client.1.smithi174.stdout:2/229: dread d5/d25/d27/d2e/f3a [0,4194304] 0 2024-03-21T10:08:48.811 INFO:tasks.workunit.client.0.smithi153.stdout:0/322: truncate d4/d16/d1f/f24 2378139 0 2024-03-21T10:08:48.811 INFO:tasks.workunit.client.1.smithi174.stdout:3/165: chown d0/d12/d14/d1a/c1c 949 1 2024-03-21T10:08:48.814 INFO:tasks.workunit.client.0.smithi153.stdout:5/426: creat d4/d5/d16/d24/f83 x:0 0 0 2024-03-21T10:08:48.814 INFO:tasks.workunit.client.0.smithi153.stdout:9/380: chown d1/d43/d44/d46/d1c/d24/f38 7058827 1 2024-03-21T10:08:48.816 INFO:tasks.workunit.client.1.smithi174.stdout:6/150: symlink d0/l32 0 2024-03-21T10:08:48.821 INFO:tasks.workunit.client.1.smithi174.stdout:0/199: rmdir d9/d35 39 2024-03-21T10:08:48.822 INFO:tasks.workunit.client.1.smithi174.stdout:3/166: mkdir d0/d12/d14/d1a/d27/d3c 0 2024-03-21T10:08:48.823 INFO:tasks.workunit.client.1.smithi174.stdout:5/203: dwrite d1/d16/f30 [4194304,4194304] 0 2024-03-21T10:08:48.825 INFO:tasks.workunit.client.1.smithi174.stdout:0/200: unlink f4 0 2024-03-21T10:08:48.826 INFO:tasks.workunit.client.1.smithi174.stdout:2/230: chown d5/l36 12 1 2024-03-21T10:08:48.828 INFO:tasks.workunit.client.0.smithi153.stdout:1/350: write d6/d10/d2c/f2e [1394692,130660] 0 2024-03-21T10:08:48.832 INFO:tasks.workunit.client.1.smithi174.stdout:3/167: creat d0/d12/d14/d1a/f3d x:0 0 0 2024-03-21T10:08:48.833 INFO:tasks.workunit.client.0.smithi153.stdout:5/427: fdatasync d4/d5/d16/d24/f53 0 2024-03-21T10:08:48.834 INFO:tasks.workunit.client.1.smithi174.stdout:0/201: write d9/d1c/f20 [1339022,19588] 0 2024-03-21T10:08:48.835 INFO:tasks.workunit.client.1.smithi174.stdout:0/202: stat f7 0 2024-03-21T10:08:48.835 INFO:tasks.workunit.client.1.smithi174.stdout:2/231: symlink d5/l56 0 2024-03-21T10:08:48.836 INFO:tasks.workunit.client.1.smithi174.stdout:8/153: read d9/d17/f20 [120633,26454] 0 2024-03-21T10:08:48.837 INFO:tasks.workunit.client.1.smithi174.stdout:3/168: symlink d0/d12/d14/d10/l3e 0 2024-03-21T10:08:48.838 INFO:tasks.workunit.client.1.smithi174.stdout:2/232: dread - d5/d25/d27/d2e/f3f zero size 2024-03-21T10:08:48.838 INFO:tasks.workunit.client.1.smithi174.stdout:0/203: creat d9/d21/f3b x:0 0 0 2024-03-21T10:08:48.840 INFO:tasks.workunit.client.0.smithi153.stdout:0/323: mknod d4/d16/d3e/d5b/c67 0 2024-03-21T10:08:48.841 INFO:tasks.workunit.client.1.smithi174.stdout:8/154: write d9/d17/d19/f23 [1536757,59076] 0 2024-03-21T10:08:48.842 INFO:tasks.workunit.client.0.smithi153.stdout:2/450: dwrite dd/f90 [0,4194304] 0 2024-03-21T10:08:48.844 INFO:tasks.workunit.client.1.smithi174.stdout:3/169: creat d0/d12/d14/d1a/f3f x:0 0 0 2024-03-21T10:08:48.845 INFO:tasks.workunit.client.0.smithi153.stdout:7/327: dwrite d4/d2a/f5b [0,4194304] 0 2024-03-21T10:08:48.845 INFO:tasks.workunit.client.0.smithi153.stdout:9/381: mkdir d1/d43/d44/d46/d1c/d4b/d84 0 2024-03-21T10:08:48.846 INFO:tasks.workunit.client.1.smithi174.stdout:0/204: mkdir d9/d21/d31/d3c 0 2024-03-21T10:08:48.848 INFO:tasks.workunit.client.0.smithi153.stdout:5/428: creat d4/d5/de/d44/d48/f84 x:0 0 0 2024-03-21T10:08:48.848 INFO:tasks.workunit.client.1.smithi174.stdout:8/155: rmdir d9/d17 39 2024-03-21T10:08:48.851 INFO:tasks.workunit.client.0.smithi153.stdout:9/382: creat d1/d43/d44/f85 x:0 0 0 2024-03-21T10:08:48.851 INFO:tasks.workunit.client.0.smithi153.stdout:1/351: dread d6/d10/d2c/d54/f57 [0,4194304] 0 2024-03-21T10:08:48.853 INFO:tasks.workunit.client.1.smithi174.stdout:3/170: fdatasync d0/fa 0 2024-03-21T10:08:48.853 INFO:tasks.workunit.client.1.smithi174.stdout:5/204: dwrite d1/f2c [0,4194304] 0 2024-03-21T10:08:48.855 INFO:tasks.workunit.client.0.smithi153.stdout:7/328: chown d4/d11/d24/f29 401338 1 2024-03-21T10:08:48.858 INFO:tasks.workunit.client.1.smithi174.stdout:6/151: symlink d0/d3/d20/l33 0 2024-03-21T10:08:48.859 INFO:tasks.workunit.client.0.smithi153.stdout:8/391: dwrite d5/d1b/d49/f67 [0,4194304] 0 2024-03-21T10:08:48.859 INFO:tasks.workunit.client.1.smithi174.stdout:3/171: creat d0/d12/d14/d1a/f40 x:0 0 0 2024-03-21T10:08:48.859 INFO:tasks.workunit.client.1.smithi174.stdout:5/205: dread d1/f22 [0,4194304] 0 2024-03-21T10:08:48.861 INFO:tasks.workunit.client.1.smithi174.stdout:6/152: truncate d0/d3/d20/f23 1042282 0 2024-03-21T10:08:48.863 INFO:tasks.workunit.client.0.smithi153.stdout:1/352: mknod d6/d10/d35/c70 0 2024-03-21T10:08:48.864 INFO:tasks.workunit.client.1.smithi174.stdout:9/250: write d2/d11/d1d/f33 [1190823,120472] 0 2024-03-21T10:08:48.866 INFO:tasks.workunit.client.1.smithi174.stdout:9/251: fdatasync d2/f47 0 2024-03-21T10:08:48.867 INFO:tasks.workunit.client.0.smithi153.stdout:1/353: write d6/d13/f1d [899453,90770] 0 2024-03-21T10:08:48.872 INFO:tasks.workunit.client.0.smithi153.stdout:6/592: write d1f/d2b/d33/d40/d51/fc5 [8646608,63356] 0 2024-03-21T10:08:48.873 INFO:tasks.workunit.client.1.smithi174.stdout:8/156: getdents d9/d17/d19 0 2024-03-21T10:08:48.874 INFO:tasks.workunit.client.0.smithi153.stdout:6/593: chown d1f/d2b/f64 39 1 2024-03-21T10:08:48.875 INFO:tasks.workunit.client.0.smithi153.stdout:0/324: link d4/db/c33 d4/d16/c68 0 2024-03-21T10:08:48.878 INFO:tasks.workunit.client.1.smithi174.stdout:0/205: dwrite d9/d21/d23/f2d [0,4194304] 0 2024-03-21T10:08:48.878 INFO:tasks.workunit.client.1.smithi174.stdout:2/233: write d5/d25/d27/f3c [733405,102417] 0 2024-03-21T10:08:48.882 INFO:tasks.workunit.client.0.smithi153.stdout:7/329: mkdir d4/d2a/d3b/d6d 0 2024-03-21T10:08:48.883 INFO:tasks.workunit.client.1.smithi174.stdout:7/162: sync 2024-03-21T10:08:48.883 INFO:tasks.workunit.client.1.smithi174.stdout:2/234: symlink d5/d8/d2f/d4b/l57 0 2024-03-21T10:08:48.883 INFO:tasks.workunit.client.1.smithi174.stdout:0/206: rmdir d9/d35 39 2024-03-21T10:08:48.884 INFO:tasks.workunit.client.0.smithi153.stdout:2/451: getdents dd/d3b/d50/d58 0 2024-03-21T10:08:48.886 INFO:tasks.workunit.client.1.smithi174.stdout:3/172: write d0/d12/d14/d1a/f1e [746052,53596] 0 2024-03-21T10:08:48.893 INFO:tasks.workunit.client.1.smithi174.stdout:7/163: symlink d0/d4/d38/l3a 0 2024-03-21T10:08:48.893 INFO:tasks.workunit.client.1.smithi174.stdout:6/153: dread d0/d3/d5/d6/f13 [0,4194304] 0 2024-03-21T10:08:48.896 INFO:tasks.workunit.client.1.smithi174.stdout:9/252: dwrite d2/f17 [0,4194304] 0 2024-03-21T10:08:48.896 INFO:tasks.workunit.client.1.smithi174.stdout:2/235: creat d5/d8/d55/f58 x:0 0 0 2024-03-21T10:08:48.897 INFO:tasks.workunit.client.0.smithi153.stdout:0/325: creat d4/d16/d3e/f69 x:0 0 0 2024-03-21T10:08:48.897 INFO:tasks.workunit.client.0.smithi153.stdout:6/594: dread - d1f/f6c zero size 2024-03-21T10:08:48.897 INFO:tasks.workunit.client.0.smithi153.stdout:8/392: mkdir d5/d9/d43/dd/d20/d26/d7a 0 2024-03-21T10:08:48.898 INFO:tasks.workunit.client.1.smithi174.stdout:3/173: symlink d0/d12/d14/l41 0 2024-03-21T10:08:48.899 INFO:tasks.workunit.client.0.smithi153.stdout:7/330: mknod d4/d2a/c6e 0 2024-03-21T10:08:48.899 INFO:tasks.workunit.client.1.smithi174.stdout:2/236: stat d5/d8/f23 0 2024-03-21T10:08:48.900 INFO:tasks.workunit.client.1.smithi174.stdout:7/164: creat d0/d4/d10/d1b/f3b x:0 0 0 2024-03-21T10:08:48.902 INFO:tasks.workunit.client.1.smithi174.stdout:0/207: read d9/f13 [68627,17310] 0 2024-03-21T10:08:48.903 INFO:tasks.workunit.client.0.smithi153.stdout:1/354: dwrite d6/d32/f37 [0,4194304] 0 2024-03-21T10:08:48.904 INFO:tasks.workunit.client.1.smithi174.stdout:6/154: creat d0/d3/d5/df/d16/f34 x:0 0 0 2024-03-21T10:08:48.908 INFO:tasks.workunit.client.1.smithi174.stdout:3/174: unlink d0/d12/d14/d1a/d27/l35 0 2024-03-21T10:08:48.908 INFO:tasks.workunit.client.1.smithi174.stdout:3/175: stat d0/f33 0 2024-03-21T10:08:48.911 INFO:tasks.workunit.client.1.smithi174.stdout:0/208: symlink d9/d1c/l3d 0 2024-03-21T10:08:48.912 INFO:tasks.workunit.client.1.smithi174.stdout:9/253: mkdir d2/d54 0 2024-03-21T10:08:48.913 INFO:tasks.workunit.client.1.smithi174.stdout:5/206: dread d1/f29 [0,4194304] 0 2024-03-21T10:08:48.914 INFO:tasks.workunit.client.1.smithi174.stdout:3/176: creat d0/d12/d14/d1a/f42 x:0 0 0 2024-03-21T10:08:48.915 INFO:tasks.workunit.client.0.smithi153.stdout:8/393: write d5/d9/d43/dd/d20/d26/d33/f6b [721045,47176] 0 2024-03-21T10:08:48.922 INFO:tasks.workunit.client.0.smithi153.stdout:0/326: chown d4/d16/d19/d30/f41 17429457 1 2024-03-21T10:08:48.922 INFO:tasks.workunit.client.0.smithi153.stdout:2/452: getdents dd/d15/d18 0 2024-03-21T10:08:48.922 INFO:tasks.workunit.client.1.smithi174.stdout:5/207: write d1/d16/f30 [1937228,81739] 0 2024-03-21T10:08:48.922 INFO:tasks.workunit.client.1.smithi174.stdout:6/155: rename d0/d3/d5/d6/l24 to d0/l35 0 2024-03-21T10:08:48.922 INFO:tasks.workunit.client.1.smithi174.stdout:0/209: fsync d9/d21/f29 0 2024-03-21T10:08:48.922 INFO:tasks.workunit.client.1.smithi174.stdout:2/237: dread d5/f12 [0,4194304] 0 2024-03-21T10:08:48.923 INFO:tasks.workunit.client.1.smithi174.stdout:9/254: mknod d2/c55 0 2024-03-21T10:08:48.925 INFO:tasks.workunit.client.1.smithi174.stdout:3/177: creat d0/d12/d14/d1a/d27/f43 x:0 0 0 2024-03-21T10:08:48.926 INFO:tasks.workunit.client.1.smithi174.stdout:5/208: stat d1/c2a 0 2024-03-21T10:08:48.928 INFO:tasks.workunit.client.0.smithi153.stdout:2/453: read - dd/d15/d5e/f8f zero size 2024-03-21T10:08:48.928 INFO:tasks.workunit.client.0.smithi153.stdout:9/383: dread d1/d43/d44/f34 [0,4194304] 0 2024-03-21T10:08:48.930 INFO:tasks.workunit.client.1.smithi174.stdout:2/238: creat d5/d25/d27/d44/f59 x:0 0 0 2024-03-21T10:08:48.930 INFO:tasks.workunit.client.1.smithi174.stdout:0/210: fdatasync d9/fc 0 2024-03-21T10:08:48.931 INFO:tasks.workunit.client.0.smithi153.stdout:7/331: dwrite d4/d11/d24/f35 [0,4194304] 0 2024-03-21T10:08:48.933 INFO:tasks.workunit.client.1.smithi174.stdout:5/209: mknod d1/d2f/c47 0 2024-03-21T10:08:48.933 INFO:tasks.workunit.client.1.smithi174.stdout:9/255: creat d2/d11/d1d/d4b/f56 x:0 0 0 2024-03-21T10:08:48.933 INFO:tasks.workunit.client.1.smithi174.stdout:3/178: mkdir d0/d12/d14/d1a/d44 0 2024-03-21T10:08:48.935 INFO:tasks.workunit.client.1.smithi174.stdout:6/156: mknod d0/c36 0 2024-03-21T10:08:48.936 INFO:tasks.workunit.client.1.smithi174.stdout:3/179: fsync d0/d12/d14/d10/f1f 0 2024-03-21T10:08:48.938 INFO:tasks.workunit.client.0.smithi153.stdout:7/332: creat d4/d11/d24/f6f x:0 0 0 2024-03-21T10:08:48.938 INFO:tasks.workunit.client.1.smithi174.stdout:5/210: fsync d1/f11 0 2024-03-21T10:08:48.939 INFO:tasks.workunit.client.1.smithi174.stdout:9/256: write d2/d24/d2e/f31 [918725,28795] 0 2024-03-21T10:08:48.940 INFO:tasks.workunit.client.1.smithi174.stdout:3/180: stat d0/d12/d14/c26 0 2024-03-21T10:08:48.943 INFO:tasks.workunit.client.0.smithi153.stdout:0/327: dread d4/f28 [0,4194304] 0 2024-03-21T10:08:48.944 INFO:tasks.workunit.client.1.smithi174.stdout:9/257: write d2/d24/f34 [593401,70669] 0 2024-03-21T10:08:48.945 INFO:tasks.workunit.client.0.smithi153.stdout:6/595: write d1f/d2b/f64 [314814,114973] 0 2024-03-21T10:08:48.947 INFO:tasks.workunit.client.0.smithi153.stdout:0/328: stat d4/d16/d1f/f35 0 2024-03-21T10:08:48.949 INFO:tasks.workunit.client.0.smithi153.stdout:7/333: rename d4/d1b/c1e to d4/d14/d2f/c70 0 2024-03-21T10:08:48.950 INFO:tasks.workunit.client.1.smithi174.stdout:9/258: write d2/d24/f34 [49345,98180] 0 2024-03-21T10:08:48.951 INFO:tasks.workunit.client.1.smithi174.stdout:9/259: fdatasync d2/f9 0 2024-03-21T10:08:48.952 INFO:tasks.workunit.client.0.smithi153.stdout:0/329: write d4/f62 [210715,37823] 0 2024-03-21T10:08:48.952 INFO:tasks.workunit.client.0.smithi153.stdout:9/384: truncate d1/d43/d44/f25 2076922 0 2024-03-21T10:08:48.952 INFO:tasks.workunit.client.1.smithi174.stdout:0/211: link d9/f19 d9/d35/f3e 0 2024-03-21T10:08:48.953 INFO:tasks.workunit.client.0.smithi153.stdout:4/408: sync 2024-03-21T10:08:48.953 INFO:tasks.workunit.client.1.smithi174.stdout:5/211: creat d1/d16/d2e/d31/f48 x:0 0 0 2024-03-21T10:08:48.953 INFO:tasks.workunit.client.0.smithi153.stdout:6/596: truncate d1f/d2b/f46 2001637 0 2024-03-21T10:08:48.954 INFO:tasks.workunit.client.1.smithi174.stdout:6/157: creat d0/f37 x:0 0 0 2024-03-21T10:08:48.958 INFO:tasks.workunit.client.0.smithi153.stdout:6/597: stat d1f/d2b/d33/d40/d65/c6b 0 2024-03-21T10:08:48.959 INFO:tasks.workunit.client.1.smithi174.stdout:2/239: dwrite d5/f24 [4194304,4194304] 0 2024-03-21T10:08:48.959 INFO:tasks.workunit.client.0.smithi153.stdout:4/409: read - d11/f5f zero size 2024-03-21T10:08:48.960 INFO:tasks.workunit.client.0.smithi153.stdout:2/454: dwrite dd/d15/d18/d2c/d4b/d56/d63/f80 [0,4194304] 0 2024-03-21T10:08:48.961 INFO:tasks.workunit.client.1.smithi174.stdout:2/240: chown d5/c20 322139 1 2024-03-21T10:08:48.962 INFO:tasks.workunit.client.1.smithi174.stdout:9/260: mkdir d2/d11/d1d/d2a/d57 0 2024-03-21T10:08:48.962 INFO:tasks.workunit.client.0.smithi153.stdout:0/330: fsync d4/f62 0 2024-03-21T10:08:48.965 INFO:tasks.workunit.client.0.smithi153.stdout:8/394: dwrite d5/d9/d43/dd/d20/d26/f23 [4194304,4194304] 0 2024-03-21T10:08:48.965 INFO:tasks.workunit.client.1.smithi174.stdout:6/158: write d0/d3/f26 [670493,100377] 0 2024-03-21T10:08:48.966 INFO:tasks.workunit.client.1.smithi174.stdout:3/181: creat d0/d12/d14/f45 x:0 0 0 2024-03-21T10:08:48.967 INFO:tasks.workunit.client.0.smithi153.stdout:6/598: creat d1f/d99/dbb/fd3 x:0 0 0 2024-03-21T10:08:48.968 INFO:tasks.workunit.client.0.smithi153.stdout:9/385: truncate d1/d43/d44/d46/d1c/d61/f7e 127288 0 2024-03-21T10:08:48.969 INFO:tasks.workunit.client.1.smithi174.stdout:0/212: rename d9/d21/f29 to d9/d1c/f3f 0 2024-03-21T10:08:48.971 INFO:tasks.workunit.client.1.smithi174.stdout:2/241: chown d5/d25/d27/d2d/c4d 5968390 1 2024-03-21T10:08:48.972 INFO:tasks.workunit.client.0.smithi153.stdout:9/386: read - d1/d4/f75 zero size 2024-03-21T10:08:48.974 INFO:tasks.workunit.client.0.smithi153.stdout:9/387: chown d1/d4e/d68 5584 1 2024-03-21T10:08:48.975 INFO:tasks.workunit.client.1.smithi174.stdout:2/242: dread - d5/d25/d27/d44/f48 zero size 2024-03-21T10:08:48.977 INFO:tasks.workunit.client.0.smithi153.stdout:6/599: symlink d1f/d2b/d33/d40/d4f/ld4 0 2024-03-21T10:08:48.978 INFO:tasks.workunit.client.1.smithi174.stdout:2/243: fsync d5/f4a 0 2024-03-21T10:08:48.979 INFO:tasks.workunit.client.0.smithi153.stdout:2/455: symlink dd/d15/d18/d2c/d4b/d56/d63/d8d/d8e/l9a 0 2024-03-21T10:08:48.980 INFO:tasks.workunit.client.0.smithi153.stdout:0/331: creat d4/d16/d19/d58/f6a x:0 0 0 2024-03-21T10:08:48.981 INFO:tasks.workunit.client.1.smithi174.stdout:0/213: creat d9/d21/d31/f40 x:0 0 0 2024-03-21T10:08:48.983 INFO:tasks.workunit.client.0.smithi153.stdout:6/600: mknod d1f/d93/cd5 0 2024-03-21T10:08:48.985 INFO:tasks.workunit.client.0.smithi153.stdout:8/395: dread d5/d9/d43/f35 [0,4194304] 0 2024-03-21T10:08:48.986 INFO:tasks.workunit.client.0.smithi153.stdout:9/388: creat d1/d3b/f86 x:0 0 0 2024-03-21T10:08:48.987 INFO:tasks.workunit.client.1.smithi174.stdout:9/261: rmdir d2/d11/d1d/d2a/d57 0 2024-03-21T10:08:48.987 INFO:tasks.workunit.client.1.smithi174.stdout:2/244: write d5/d8/d55/f58 [834924,59377] 0 2024-03-21T10:08:48.989 INFO:tasks.workunit.client.1.smithi174.stdout:0/214: dread - d9/d21/d31/f40 zero size 2024-03-21T10:08:48.990 INFO:tasks.workunit.client.0.smithi153.stdout:8/396: chown d5/d9/d43/dd/d20/d26/d24/d52/d56 2399758 1 2024-03-21T10:08:48.992 INFO:tasks.workunit.client.1.smithi174.stdout:5/212: rmdir d1/d16/d2e/d31 39 2024-03-21T10:08:48.994 INFO:tasks.workunit.client.1.smithi174.stdout:2/245: chown d5/fa 6587 1 2024-03-21T10:08:48.994 INFO:tasks.workunit.client.0.smithi153.stdout:0/332: link d4/f28 d4/d16/d3e/d5e/f6b 0 2024-03-21T10:08:48.995 INFO:tasks.workunit.client.0.smithi153.stdout:9/389: creat d1/d43/d44/d46/d1c/d61/f87 x:0 0 0 2024-03-21T10:08:48.997 INFO:tasks.workunit.client.0.smithi153.stdout:8/397: mkdir d5/d9/d43/d7b 0 2024-03-21T10:08:48.998 INFO:tasks.workunit.client.0.smithi153.stdout:0/333: write d4/f62 [305729,128787] 0 2024-03-21T10:08:48.999 INFO:tasks.workunit.client.1.smithi174.stdout:9/262: write d2/d11/d1d/d4b/f56 [473962,57132] 0 2024-03-21T10:08:48.999 INFO:tasks.workunit.client.0.smithi153.stdout:4/410: dwrite d11/f24 [0,4194304] 0 2024-03-21T10:08:49.000 INFO:tasks.workunit.client.1.smithi174.stdout:6/159: dwrite d0/d3/d20/f23 [0,4194304] 0 2024-03-21T10:08:49.004 INFO:tasks.workunit.client.0.smithi153.stdout:8/398: write d5/d9/d43/f6c [577832,35802] 0 2024-03-21T10:08:49.004 INFO:tasks.workunit.client.0.smithi153.stdout:0/334: unlink f2 0 2024-03-21T10:08:49.004 INFO:tasks.workunit.client.1.smithi174.stdout:3/182: dwrite d0/d12/d14/d10/f1b [0,4194304] 0 2024-03-21T10:08:49.005 INFO:tasks.workunit.client.0.smithi153.stdout:9/390: mkdir d1/d43/d44/d46/d4d/d88 0 2024-03-21T10:08:49.008 INFO:tasks.workunit.client.0.smithi153.stdout:4/411: mkdir d11/d18/d7e 0 2024-03-21T10:08:49.009 INFO:tasks.workunit.client.0.smithi153.stdout:0/335: mkdir d4/d16/d19/d30/d6c 0 2024-03-21T10:08:49.010 INFO:tasks.workunit.client.1.smithi174.stdout:2/246: rename d5/d8/l15 to d5/d25/l5a 0 2024-03-21T10:08:49.011 INFO:tasks.workunit.client.0.smithi153.stdout:8/399: truncate d5/d9/d43/dd/d20/d26/d33/f73 566858 0 2024-03-21T10:08:49.011 INFO:tasks.workunit.client.0.smithi153.stdout:9/391: read d1/d43/f4f [2877138,45619] 0 2024-03-21T10:08:49.013 INFO:tasks.workunit.client.0.smithi153.stdout:0/336: fdatasync d4/d16/d1f/f20 0 2024-03-21T10:08:49.018 INFO:tasks.workunit.client.1.smithi174.stdout:9/263: dread d2/d24/d2e/f48 [0,4194304] 0 2024-03-21T10:08:49.021 INFO:tasks.workunit.client.0.smithi153.stdout:0/337: write d4/d16/d1f/f31 [29126,88077] 0 2024-03-21T10:08:49.021 INFO:tasks.workunit.client.1.smithi174.stdout:3/183: mkdir d0/d12/d46 0 2024-03-21T10:08:49.021 INFO:tasks.workunit.client.1.smithi174.stdout:0/215: getdents d9/d21/d31 0 2024-03-21T10:08:49.023 INFO:tasks.workunit.client.1.smithi174.stdout:3/184: stat d0/f3 0 2024-03-21T10:08:49.024 INFO:tasks.workunit.client.1.smithi174.stdout:5/213: getdents d1/d2f/d37 0 2024-03-21T10:08:49.027 INFO:tasks.workunit.client.0.smithi153.stdout:9/392: truncate d1/d43/d44/d46/d1c/d24/f38 1578559 0 2024-03-21T10:08:49.027 INFO:tasks.workunit.client.0.smithi153.stdout:6/601: dwrite d1f/d2b/d33/d40/d51/f57 [0,4194304] 0 2024-03-21T10:08:49.030 INFO:tasks.workunit.client.1.smithi174.stdout:1/233: sync 2024-03-21T10:08:49.030 INFO:tasks.workunit.client.1.smithi174.stdout:4/238: sync 2024-03-21T10:08:49.033 INFO:tasks.workunit.client.0.smithi153.stdout:4/412: dwrite d11/d2d/f3a [0,4194304] 0 2024-03-21T10:08:49.035 INFO:tasks.workunit.client.1.smithi174.stdout:4/239: chown d4 0 1 2024-03-21T10:08:49.035 INFO:tasks.workunit.client.1.smithi174.stdout:0/216: getdents d9/d21/d31/d3c 0 2024-03-21T10:08:49.035 INFO:tasks.workunit.client.1.smithi174.stdout:5/214: write d1/d16/d2e/d31/f48 [69221,43634] 0 2024-03-21T10:08:49.036 INFO:tasks.workunit.client.0.smithi153.stdout:6/602: rmdir d1f/d2b/d33/d40/d4f 39 2024-03-21T10:08:49.037 INFO:tasks.workunit.client.0.smithi153.stdout:8/400: dread d5/d9/d43/dd/d20/d26/f23 [4194304,4194304] 0 2024-03-21T10:08:49.038 INFO:tasks.workunit.client.1.smithi174.stdout:2/247: getdents d5/d25/d27/d2e 0 2024-03-21T10:08:49.040 INFO:tasks.workunit.client.0.smithi153.stdout:9/393: unlink d1/d43/d44/d46/d1c/d2e/l56 0 2024-03-21T10:08:49.040 INFO:tasks.workunit.client.1.smithi174.stdout:0/217: creat d9/d21/d23/f41 x:0 0 0 2024-03-21T10:08:49.042 INFO:tasks.workunit.client.1.smithi174.stdout:6/160: dwrite d0/d3/d5/d6/f13 [0,4194304] 0 2024-03-21T10:08:49.042 INFO:tasks.workunit.client.1.smithi174.stdout:0/218: readlink l3 0 2024-03-21T10:08:49.045 INFO:tasks.workunit.client.1.smithi174.stdout:4/240: creat d4/d21/d24/d28/d38/f54 x:0 0 0 2024-03-21T10:08:49.047 INFO:tasks.workunit.client.1.smithi174.stdout:9/264: truncate d2/f4 788216 0 2024-03-21T10:08:49.052 INFO:tasks.workunit.client.1.smithi174.stdout:7/165: sync 2024-03-21T10:08:49.053 INFO:tasks.workunit.client.1.smithi174.stdout:8/157: sync 2024-03-21T10:08:49.054 INFO:tasks.workunit.client.1.smithi174.stdout:0/219: creat d9/d21/d31/f42 x:0 0 0 2024-03-21T10:08:49.055 INFO:tasks.workunit.client.1.smithi174.stdout:4/241: mknod d4/d21/c55 0 2024-03-21T10:08:49.055 INFO:tasks.workunit.client.0.smithi153.stdout:8/401: mkdir d5/d9/d43/d7b/d7c 0 2024-03-21T10:08:49.055 INFO:tasks.workunit.client.0.smithi153.stdout:0/338: dread d4/d16/d1f/d22/f2e [4194304,4194304] 0 2024-03-21T10:08:49.056 INFO:tasks.workunit.client.1.smithi174.stdout:1/234: truncate d4/d7/d15/f2f 1515531 0 2024-03-21T10:08:49.061 INFO:tasks.workunit.client.0.smithi153.stdout:0/339: chown d4/db/c1c 3182005 1 2024-03-21T10:08:49.062 INFO:tasks.workunit.client.1.smithi174.stdout:8/158: mkdir d9/d2a/d39 0 2024-03-21T10:08:49.063 INFO:tasks.workunit.client.1.smithi174.stdout:5/215: rename d1/d16/d1c/d41 to d1/d49 0 2024-03-21T10:08:49.065 INFO:tasks.workunit.client.1.smithi174.stdout:3/185: dwrite d0/f2 [0,4194304] 0 2024-03-21T10:08:49.066 INFO:tasks.workunit.client.1.smithi174.stdout:0/220: mknod d9/d21/c43 0 2024-03-21T10:08:49.067 INFO:tasks.workunit.client.1.smithi174.stdout:4/242: mknod d4/d21/d24/d28/d38/c56 0 2024-03-21T10:08:49.069 INFO:tasks.workunit.client.0.smithi153.stdout:0/340: fdatasync d4/d16/d27/f4d 0 2024-03-21T10:08:49.071 INFO:tasks.workunit.client.1.smithi174.stdout:7/166: fsync d0/fa 0 2024-03-21T10:08:49.074 INFO:tasks.workunit.client.1.smithi174.stdout:2/248: rename d5/d8 to d5/d25/d27/d2d/d45/d5b 0 2024-03-21T10:08:49.078 INFO:tasks.workunit.client.1.smithi174.stdout:1/235: symlink d4/l55 0 2024-03-21T10:08:49.079 INFO:tasks.workunit.client.1.smithi174.stdout:4/243: read - d4/d21/d24/d28/d38/f54 zero size 2024-03-21T10:08:49.079 INFO:tasks.workunit.client.0.smithi153.stdout:9/394: dwrite d1/d43/d44/f5d [4194304,4194304] 0 2024-03-21T10:08:49.079 INFO:tasks.workunit.client.1.smithi174.stdout:8/159: truncate d9/d17/d19/f1e 917439 0 2024-03-21T10:08:49.082 INFO:tasks.workunit.client.0.smithi153.stdout:4/413: dwrite d11/d18/f61 [0,4194304] 0 2024-03-21T10:08:49.084 INFO:tasks.workunit.client.1.smithi174.stdout:7/167: read - d0/d4/f22 zero size 2024-03-21T10:08:49.084 INFO:tasks.workunit.client.1.smithi174.stdout:8/160: readlink d9/d17/l1b 0 2024-03-21T10:08:49.087 INFO:tasks.workunit.client.0.smithi153.stdout:4/414: fdatasync d11/f76 0 2024-03-21T10:08:49.088 INFO:tasks.workunit.client.0.smithi153.stdout:6/603: dwrite d1f/d2b/d33/d40/d4f/f77 [0,4194304] 0 2024-03-21T10:08:49.088 INFO:tasks.workunit.client.0.smithi153.stdout:3/369: sync 2024-03-21T10:08:49.088 INFO:tasks.workunit.client.0.smithi153.stdout:5/429: sync 2024-03-21T10:08:49.089 INFO:tasks.workunit.client.1.smithi174.stdout:3/186: write d0/d12/d14/d1a/d27/f2c [3948651,96263] 0 2024-03-21T10:08:49.092 INFO:tasks.workunit.client.0.smithi153.stdout:9/395: write d1/d43/d44/f34 [1235541,75507] 0 2024-03-21T10:08:49.094 INFO:tasks.workunit.client.1.smithi174.stdout:6/161: dwrite d0/d3/d5/df/d16/f1a [0,4194304] 0 2024-03-21T10:08:49.094 INFO:tasks.workunit.client.1.smithi174.stdout:9/265: rename d2/f44 to d2/d24/d2e/f58 0 2024-03-21T10:08:49.094 INFO:tasks.workunit.client.0.smithi153.stdout:3/370: symlink dd/d1e/d6b/l84 0 2024-03-21T10:08:49.095 INFO:tasks.workunit.client.1.smithi174.stdout:1/236: mknod d4/d2d/c56 0 2024-03-21T10:08:49.096 INFO:tasks.workunit.client.1.smithi174.stdout:2/249: symlink d5/d25/d27/d2d/d45/d5b/d2f/l5c 0 2024-03-21T10:08:49.097 INFO:tasks.workunit.client.0.smithi153.stdout:9/396: fsync d1/d43/d44/f85 0 2024-03-21T10:08:49.098 INFO:tasks.workunit.client.0.smithi153.stdout:5/430: chown d4/d5/de/f21 304341777 1 2024-03-21T10:08:49.098 INFO:tasks.workunit.client.0.smithi153.stdout:6/604: chown d1f/d2b/d33/d40/d51/d98/ld2 494252 1 2024-03-21T10:08:49.099 INFO:tasks.workunit.client.1.smithi174.stdout:8/161: mknod d9/d17/c3a 0 2024-03-21T10:08:49.100 INFO:tasks.workunit.client.0.smithi153.stdout:8/402: dwrite d5/d9/d43/dd/d2d/f31 [4194304,4194304] 0 2024-03-21T10:08:49.101 INFO:tasks.workunit.client.0.smithi153.stdout:5/431: fsync d4/d5/d16/d24/f74 0 2024-03-21T10:08:49.105 INFO:tasks.workunit.client.0.smithi153.stdout:6/605: dread - d1f/d99/dbb/fd3 zero size 2024-03-21T10:08:49.108 INFO:tasks.workunit.client.0.smithi153.stdout:3/371: symlink dd/d1e/d3d/d53/l85 0 2024-03-21T10:08:49.109 INFO:tasks.workunit.client.0.smithi153.stdout:6/606: fdatasync d1f/d2f/d59/f8d 0 2024-03-21T10:08:49.110 INFO:tasks.workunit.client.0.smithi153.stdout:3/372: chown dd/f12 0 1 2024-03-21T10:08:49.113 INFO:tasks.workunit.client.0.smithi153.stdout:2/456: dread dd/d15/d18/d2c/d33/f62 [0,4194304] 0 2024-03-21T10:08:49.114 INFO:tasks.workunit.client.0.smithi153.stdout:4/415: dread f1 [0,4194304] 0 2024-03-21T10:08:49.114 INFO:tasks.workunit.client.1.smithi174.stdout:1/237: write d4/d7/d15/f38 [572366,12425] 0 2024-03-21T10:08:49.120 INFO:tasks.workunit.client.0.smithi153.stdout:2/457: chown dd/d15/d18/d2c/f35 27962630 1 2024-03-21T10:08:49.120 INFO:tasks.workunit.client.0.smithi153.stdout:0/341: dread d4/db/f13 [0,4194304] 0 2024-03-21T10:08:49.120 INFO:tasks.workunit.client.1.smithi174.stdout:5/216: rename d1/d16/d2e/d31/f35 to d1/f4a 0 2024-03-21T10:08:49.121 INFO:tasks.workunit.client.0.smithi153.stdout:6/607: truncate d1f/d2b/d33/d40/d4f/d66/fd0 151250 0 2024-03-21T10:08:49.123 INFO:tasks.workunit.client.0.smithi153.stdout:0/342: fdatasync d4/f37 0 2024-03-21T10:08:49.125 INFO:tasks.workunit.client.0.smithi153.stdout:5/432: creat d4/d5/f85 x:0 0 0 2024-03-21T10:08:49.127 INFO:tasks.workunit.client.1.smithi174.stdout:4/244: dread d4/fa [0,4194304] 0 2024-03-21T10:08:49.128 INFO:tasks.workunit.client.1.smithi174.stdout:5/217: chown d1/d16/f30 361 1 2024-03-21T10:08:49.132 INFO:tasks.workunit.client.1.smithi174.stdout:7/168: dwrite d0/fa [4194304,4194304] 0 2024-03-21T10:08:49.132 INFO:tasks.workunit.client.1.smithi174.stdout:8/162: creat d9/d17/d19/d31/f3b x:0 0 0 2024-03-21T10:08:49.133 INFO:tasks.workunit.client.1.smithi174.stdout:4/245: write d4/f18 [2912718,30548] 0 2024-03-21T10:08:49.135 INFO:tasks.workunit.client.0.smithi153.stdout:0/343: truncate d4/d16/d27/f4d 392248 0 2024-03-21T10:08:49.135 INFO:tasks.workunit.client.0.smithi153.stdout:5/433: write d4/d8/f11 [2032902,31084] 0 2024-03-21T10:08:49.141 INFO:tasks.workunit.client.1.smithi174.stdout:6/162: dwrite d0/d3/d5/df/d16/f17 [0,4194304] 0 2024-03-21T10:08:49.144 INFO:tasks.workunit.client.1.smithi174.stdout:4/246: dread - d4/d21/d24/f4d zero size 2024-03-21T10:08:49.144 INFO:tasks.workunit.client.1.smithi174.stdout:3/187: rename d0/f8 to d0/d12/d14/d1a/d27/f47 0 2024-03-21T10:08:49.145 INFO:tasks.workunit.client.0.smithi153.stdout:1/355: sync 2024-03-21T10:08:49.145 INFO:tasks.workunit.client.0.smithi153.stdout:7/334: sync 2024-03-21T10:08:49.145 INFO:tasks.workunit.client.0.smithi153.stdout:4/416: dwrite d11/d18/f55 [0,4194304] 0 2024-03-21T10:08:49.145 INFO:tasks.workunit.client.0.smithi153.stdout:8/403: rename d5/d1b/f1d to d5/d9/d43/dd/d20/f7d 0 2024-03-21T10:08:49.146 INFO:tasks.workunit.client.1.smithi174.stdout:4/247: fdatasync d4/f3e 0 2024-03-21T10:08:49.147 INFO:tasks.workunit.client.1.smithi174.stdout:9/266: dwrite d2/d11/d1d/f41 [0,4194304] 0 2024-03-21T10:08:49.148 INFO:tasks.workunit.client.1.smithi174.stdout:6/163: chown d0/d3/d5/d6/f2d 399 1 2024-03-21T10:08:49.149 INFO:tasks.workunit.client.1.smithi174.stdout:5/218: creat d1/d16/d2e/d31/f4b x:0 0 0 2024-03-21T10:08:49.149 INFO:tasks.workunit.client.1.smithi174.stdout:4/248: truncate d4/f5 389463 0 2024-03-21T10:08:49.150 INFO:tasks.workunit.client.1.smithi174.stdout:3/188: truncate d0/d12/d14/f3a 4993810 0 2024-03-21T10:08:49.150 INFO:tasks.workunit.client.0.smithi153.stdout:9/397: dwrite d1/d43/d44/d46/d1c/d24/d41/f5b [4194304,4194304] 0 2024-03-21T10:08:49.152 INFO:tasks.workunit.client.0.smithi153.stdout:5/434: mkdir d4/d5/df/d67/d86 0 2024-03-21T10:08:49.153 INFO:tasks.workunit.client.0.smithi153.stdout:3/373: creat dd/d1e/d45/d56/d58/f86 x:0 0 0 2024-03-21T10:08:49.154 INFO:tasks.workunit.client.1.smithi174.stdout:4/249: dread - d4/d21/d31/d3a/f4c zero size 2024-03-21T10:08:49.154 INFO:tasks.workunit.client.0.smithi153.stdout:7/335: readlink d4/d11/l17 0 2024-03-21T10:08:49.155 INFO:tasks.workunit.client.1.smithi174.stdout:0/221: write d9/d1c/f3f [1909546,100033] 0 2024-03-21T10:08:49.156 INFO:tasks.workunit.client.1.smithi174.stdout:4/250: dread - d4/d21/d24/f4d zero size 2024-03-21T10:08:49.159 INFO:tasks.workunit.client.0.smithi153.stdout:8/404: creat d5/f7e x:0 0 0 2024-03-21T10:08:49.160 INFO:tasks.workunit.client.0.smithi153.stdout:6/608: rename d1f/d2f/l3f to d1f/d2f/d59/ld6 0 2024-03-21T10:08:49.162 INFO:tasks.workunit.client.1.smithi174.stdout:6/164: creat d0/d3/d5/d6/f38 x:0 0 0 2024-03-21T10:08:49.166 INFO:tasks.workunit.client.1.smithi174.stdout:4/251: symlink d4/l57 0 2024-03-21T10:08:49.167 INFO:tasks.workunit.client.0.smithi153.stdout:7/336: readlink d4/d1b/d20/l66 0 2024-03-21T10:08:49.170 INFO:tasks.workunit.client.1.smithi174.stdout:0/222: mknod d9/d21/d31/d3c/c44 0 2024-03-21T10:08:49.170 INFO:tasks.workunit.client.0.smithi153.stdout:7/337: dread - d4/d11/d24/f51 zero size 2024-03-21T10:08:49.172 INFO:tasks.workunit.client.1.smithi174.stdout:6/165: unlink d0/d3/d20/l29 0 2024-03-21T10:08:49.173 INFO:tasks.workunit.client.0.smithi153.stdout:3/374: mknod dd/d1e/d45/d56/c87 0 2024-03-21T10:08:49.173 INFO:tasks.workunit.client.0.smithi153.stdout:5/435: fdatasync d4/d8/d3b/d56/f62 0 2024-03-21T10:08:49.173 INFO:tasks.workunit.client.0.smithi153.stdout:7/338: chown d4/d2a/d40 297126833 1 2024-03-21T10:08:49.173 INFO:tasks.workunit.client.0.smithi153.stdout:4/417: getdents d11/d18/d6d 0 2024-03-21T10:08:49.175 INFO:tasks.workunit.client.1.smithi174.stdout:0/223: mknod d9/d1c/c45 0 2024-03-21T10:08:49.176 INFO:tasks.workunit.client.0.smithi153.stdout:3/375: fsync dd/d41/f64 0 2024-03-21T10:08:49.177 INFO:tasks.workunit.client.1.smithi174.stdout:6/166: write d0/d3/d5/df/d16/f27 [752261,56235] 0 2024-03-21T10:08:49.178 INFO:tasks.workunit.client.1.smithi174.stdout:9/267: dwrite d2/f17 [0,4194304] 0 2024-03-21T10:08:49.183 INFO:tasks.workunit.client.0.smithi153.stdout:6/609: unlink d1f/d2b/f46 0 2024-03-21T10:08:49.183 INFO:tasks.workunit.client.1.smithi174.stdout:9/268: chown d2/d24/l38 1626601805 1 2024-03-21T10:08:49.183 INFO:tasks.workunit.client.1.smithi174.stdout:0/224: creat d9/d35/f46 x:0 0 0 2024-03-21T10:08:49.184 INFO:tasks.workunit.client.0.smithi153.stdout:0/344: getdents d4/d16/d3e/d5e 0 2024-03-21T10:08:49.185 INFO:tasks.workunit.client.1.smithi174.stdout:9/269: write d2/f1f [4245941,12087] 0 2024-03-21T10:08:49.187 INFO:tasks.workunit.client.1.smithi174.stdout:8/163: dread f8 [0,4194304] 0 2024-03-21T10:08:49.187 INFO:tasks.workunit.client.1.smithi174.stdout:6/167: symlink d0/d3/l39 0 2024-03-21T10:08:49.188 INFO:tasks.workunit.client.0.smithi153.stdout:5/436: creat d4/d5/de/d44/d48/f87 x:0 0 0 2024-03-21T10:08:49.191 INFO:tasks.workunit.client.1.smithi174.stdout:0/225: creat d9/d21/d31/f47 x:0 0 0 2024-03-21T10:08:49.193 INFO:tasks.workunit.client.0.smithi153.stdout:2/458: dread dd/d15/d18/f2a [0,4194304] 0 2024-03-21T10:08:49.194 INFO:tasks.workunit.client.0.smithi153.stdout:4/418: unlink d11/d18/d54/c62 0 2024-03-21T10:08:49.195 INFO:tasks.workunit.client.0.smithi153.stdout:6/610: symlink d1f/da0/ld7 0 2024-03-21T10:08:49.195 INFO:tasks.workunit.client.1.smithi174.stdout:8/164: creat d9/d17/f3c x:0 0 0 2024-03-21T10:08:49.198 INFO:tasks.workunit.client.1.smithi174.stdout:4/252: dwrite d4/d21/d24/d28/f51 [0,4194304] 0 2024-03-21T10:08:49.199 INFO:tasks.workunit.client.0.smithi153.stdout:3/376: mknod dd/d1e/d45/c88 0 2024-03-21T10:08:49.201 INFO:tasks.workunit.client.1.smithi174.stdout:6/168: symlink d0/d3/d5/df/l3a 0 2024-03-21T10:08:49.203 INFO:tasks.workunit.client.0.smithi153.stdout:6/611: write d1f/d99/faf [563311,77573] 0 2024-03-21T10:08:49.204 INFO:tasks.workunit.client.0.smithi153.stdout:9/398: dwrite d1/d43/d44/f58 [0,4194304] 0 2024-03-21T10:08:49.206 INFO:tasks.workunit.client.0.smithi153.stdout:5/437: rename d4/d5/de/d38/f4d to d4/d8/d3b/f88 0 2024-03-21T10:08:49.208 INFO:tasks.workunit.client.1.smithi174.stdout:2/250: truncate d5/f24 2639333 0 2024-03-21T10:08:49.209 INFO:tasks.workunit.client.1.smithi174.stdout:6/169: readlink d0/d3/d5/d21/l30 0 2024-03-21T10:08:49.210 INFO:tasks.workunit.client.0.smithi153.stdout:0/345: mkdir d4/d16/d19/d30/d6c/d6d 0 2024-03-21T10:08:49.210 INFO:tasks.workunit.client.1.smithi174.stdout:9/270: creat d2/d24/f59 x:0 0 0 2024-03-21T10:08:49.210 INFO:tasks.workunit.client.1.smithi174.stdout:2/251: chown d5 36409373 1 2024-03-21T10:08:49.212 INFO:tasks.workunit.client.0.smithi153.stdout:2/459: mkdir dd/d15/d18/d2c/d4b/d56/d63/d8d/d9b 0 2024-03-21T10:08:49.212 INFO:tasks.workunit.client.1.smithi174.stdout:6/170: stat d0/d3/l10 0 2024-03-21T10:08:49.219 INFO:tasks.workunit.client.0.smithi153.stdout:5/438: symlink d4/d8/l89 0 2024-03-21T10:08:49.221 INFO:tasks.workunit.client.1.smithi174.stdout:8/165: rename d9/d17/c3a to d9/c3d 0 2024-03-21T10:08:49.222 INFO:tasks.workunit.client.1.smithi174.stdout:4/253: dwrite d4/d21/d31/d3a/f3d [0,4194304] 0 2024-03-21T10:08:49.226 INFO:tasks.workunit.client.0.smithi153.stdout:0/346: mknod d4/d16/d3e/c6e 0 2024-03-21T10:08:49.227 INFO:tasks.workunit.client.1.smithi174.stdout:6/171: mkdir d0/d3/d5/d3b 0 2024-03-21T10:08:49.232 INFO:tasks.workunit.client.1.smithi174.stdout:8/166: rename d9/d17/f29 to d9/d2a/f3e 0 2024-03-21T10:08:49.234 INFO:tasks.workunit.client.1.smithi174.stdout:4/254: creat d4/d19/f58 x:0 0 0 2024-03-21T10:08:49.242 INFO:tasks.workunit.client.0.smithi153.stdout:0/347: mknod d4/d16/d3e/d5e/c6f 0 2024-03-21T10:08:49.243 INFO:tasks.workunit.client.1.smithi174.stdout:9/271: getdents d2/d24/d2e 0 2024-03-21T10:08:49.246 INFO:tasks.workunit.client.1.smithi174.stdout:5/219: rmdir d1 39 2024-03-21T10:08:49.248 INFO:tasks.workunit.client.0.smithi153.stdout:0/348: truncate d4/f62 1012238 0 2024-03-21T10:08:49.248 INFO:tasks.workunit.client.1.smithi174.stdout:9/272: fdatasync d2/d11/f3a 0 2024-03-21T10:08:49.249 INFO:tasks.workunit.client.0.smithi153.stdout:0/349: readlink d4/d16/d19/l2f 0 2024-03-21T10:08:49.250 INFO:tasks.workunit.client.1.smithi174.stdout:4/255: mknod d4/d21/d24/c59 0 2024-03-21T10:08:49.250 INFO:tasks.workunit.client.0.smithi153.stdout:5/439: truncate d4/d5/df/f30 3269827 0 2024-03-21T10:08:49.251 INFO:tasks.workunit.client.1.smithi174.stdout:1/238: dwrite d4/d7/d32/f3a [0,4194304] 0 2024-03-21T10:08:49.254 INFO:tasks.workunit.client.1.smithi174.stdout:2/252: dwrite d5/f4a [0,4194304] 0 2024-03-21T10:08:49.257 INFO:tasks.workunit.client.1.smithi174.stdout:6/172: getdents d0/d3/d20 0 2024-03-21T10:08:49.257 INFO:tasks.workunit.client.0.smithi153.stdout:6/612: rename d1f/d2f to d1f/d93/dd8 0 2024-03-21T10:08:49.260 INFO:tasks.workunit.client.1.smithi174.stdout:6/173: chown d0/d3/d5/df/d16/f17 1294 1 2024-03-21T10:08:49.261 INFO:tasks.workunit.client.0.smithi153.stdout:8/405: read d5/d9/d43/dd/f22 [284002,39951] 0 2024-03-21T10:08:49.263 INFO:tasks.workunit.client.0.smithi153.stdout:8/406: readlink d5/d9/l39 0 2024-03-21T10:08:49.264 INFO:tasks.workunit.client.1.smithi174.stdout:4/256: unlink d4/d19/f1f 0 2024-03-21T10:08:49.265 INFO:tasks.workunit.client.1.smithi174.stdout:9/273: symlink d2/d24/d2e/l5a 0 2024-03-21T10:08:49.266 INFO:tasks.workunit.client.0.smithi153.stdout:1/356: truncate d6/d13/f5e 1180182 0 2024-03-21T10:08:49.267 INFO:tasks.workunit.client.0.smithi153.stdout:9/399: dread d1/d43/d44/d46/d1c/d24/f38 [0,4194304] 0 2024-03-21T10:08:49.267 INFO:tasks.workunit.client.0.smithi153.stdout:6/613: mknod d1f/d2b/d33/d40/d65/cd9 0 2024-03-21T10:08:49.268 INFO:tasks.workunit.client.1.smithi174.stdout:1/239: read d4/d5/f2a [367526,126212] 0 2024-03-21T10:08:49.269 INFO:tasks.workunit.client.1.smithi174.stdout:9/274: fsync d2/d11/f30 0 2024-03-21T10:08:49.270 INFO:tasks.workunit.client.1.smithi174.stdout:1/240: readlink d4/d7/l35 0 2024-03-21T10:08:49.271 INFO:tasks.workunit.client.1.smithi174.stdout:8/167: creat d9/d17/d19/d33/f3f x:0 0 0 2024-03-21T10:08:49.275 INFO:tasks.workunit.client.1.smithi174.stdout:6/174: readlink d0/l35 0 2024-03-21T10:08:49.276 INFO:tasks.workunit.client.0.smithi153.stdout:1/357: dread - d6/d3a/d3c/d50/f67 zero size 2024-03-21T10:08:49.276 INFO:tasks.workunit.client.0.smithi153.stdout:8/407: dread - d5/d9/d43/dd/d20/d26/d24/d52/d56/f6f zero size 2024-03-21T10:08:49.277 INFO:tasks.workunit.client.0.smithi153.stdout:7/339: dread d4/d14/d16/f39 [0,4194304] 0 2024-03-21T10:08:49.277 INFO:tasks.workunit.client.0.smithi153.stdout:6/614: mknod d1f/d2b/d33/d40/d51/cda 0 2024-03-21T10:08:49.277 INFO:tasks.workunit.client.0.smithi153.stdout:9/400: fdatasync d1/d43/d44/d46/d1c/d24/d41/f6a 0 2024-03-21T10:08:49.278 INFO:tasks.workunit.client.1.smithi174.stdout:5/220: fsync d1/d16/d2e/d31/f33 0 2024-03-21T10:08:49.283 INFO:tasks.workunit.client.1.smithi174.stdout:6/175: write d0/d3/d5/df/d16/f34 [981799,115018] 0 2024-03-21T10:08:49.285 INFO:tasks.workunit.client.0.smithi153.stdout:6/615: rmdir d1f/d93/dd8/d59/d70 39 2024-03-21T10:08:49.288 INFO:tasks.workunit.client.1.smithi174.stdout:1/241: unlink d4/d7/d33/f3e 0 2024-03-21T10:08:49.289 INFO:tasks.workunit.client.1.smithi174.stdout:4/257: dwrite d4/f3e [0,4194304] 0 2024-03-21T10:08:49.289 INFO:tasks.workunit.client.0.smithi153.stdout:0/350: dwrite d4/d16/d3e/f63 [0,4194304] 0 2024-03-21T10:08:49.290 INFO:tasks.workunit.client.1.smithi174.stdout:8/168: creat d9/d17/d19/f40 x:0 0 0 2024-03-21T10:08:49.292 INFO:tasks.workunit.client.0.smithi153.stdout:9/401: creat d1/d4e/f89 x:0 0 0 2024-03-21T10:08:49.293 INFO:tasks.workunit.client.1.smithi174.stdout:2/253: dwrite d5/f38 [4194304,4194304] 0 2024-03-21T10:08:49.294 INFO:tasks.workunit.client.0.smithi153.stdout:7/340: creat d4/d11/d24/f71 x:0 0 0 2024-03-21T10:08:49.296 INFO:tasks.workunit.client.1.smithi174.stdout:8/169: read d9/f15 [735472,91028] 0 2024-03-21T10:08:49.296 INFO:tasks.workunit.client.1.smithi174.stdout:5/221: stat d1/d2f/c3d 0 2024-03-21T10:08:49.298 INFO:tasks.workunit.client.0.smithi153.stdout:1/358: unlink d6/d13/c30 0 2024-03-21T10:08:49.299 INFO:tasks.workunit.client.1.smithi174.stdout:9/275: creat d2/d54/f5b x:0 0 0 2024-03-21T10:08:49.301 INFO:tasks.workunit.client.0.smithi153.stdout:9/402: fsync d1/f59 0 2024-03-21T10:08:49.302 INFO:tasks.workunit.client.0.smithi153.stdout:8/408: mknod d5/d3b/c7f 0 2024-03-21T10:08:49.303 INFO:tasks.workunit.client.0.smithi153.stdout:6/616: mknod d1f/da0/cdb 0 2024-03-21T10:08:49.304 INFO:tasks.workunit.client.1.smithi174.stdout:7/169: sync 2024-03-21T10:08:49.306 INFO:tasks.workunit.client.1.smithi174.stdout:6/176: creat d0/d3/d5/f3c x:0 0 0 2024-03-21T10:08:49.306 INFO:tasks.workunit.client.1.smithi174.stdout:1/242: creat d4/d2d/f57 x:0 0 0 2024-03-21T10:08:49.307 INFO:tasks.workunit.client.0.smithi153.stdout:7/341: chown d4/d11/d24/f29 0 1 2024-03-21T10:08:49.308 INFO:tasks.workunit.client.0.smithi153.stdout:5/440: dwrite d4/d8/d5a/f61 [0,4194304] 0 2024-03-21T10:08:49.310 INFO:tasks.workunit.client.0.smithi153.stdout:0/351: creat d4/d16/d3e/d5e/f70 x:0 0 0 2024-03-21T10:08:49.312 INFO:tasks.workunit.client.1.smithi174.stdout:5/222: creat d1/d2f/f4c x:0 0 0 2024-03-21T10:08:49.314 INFO:tasks.workunit.client.1.smithi174.stdout:9/276: creat d2/d54/f5c x:0 0 0 2024-03-21T10:08:49.314 INFO:tasks.workunit.client.1.smithi174.stdout:2/254: dread d5/d25/d27/d2d/d45/d5b/d2f/d4b/f54 [0,4194304] 0 2024-03-21T10:08:49.317 INFO:tasks.workunit.client.0.smithi153.stdout:4/419: dwrite f7 [0,4194304] 0 2024-03-21T10:08:49.320 INFO:tasks.workunit.client.0.smithi153.stdout:8/409: mkdir d5/d3b/d80 0 2024-03-21T10:08:49.321 INFO:tasks.workunit.client.0.smithi153.stdout:5/441: chown d4/d5/d16/d24/d70 3 1 2024-03-21T10:08:49.326 INFO:tasks.workunit.client.1.smithi174.stdout:2/255: creat d5/d25/d27/d2d/d45/f5d x:0 0 0 2024-03-21T10:08:49.328 INFO:tasks.workunit.client.0.smithi153.stdout:7/342: rename d4/f1a to d4/d11/f72 0 2024-03-21T10:08:49.330 INFO:tasks.workunit.client.1.smithi174.stdout:8/170: dwrite f5 [0,4194304] 0 2024-03-21T10:08:49.342 INFO:tasks.workunit.client.1.smithi174.stdout:4/258: dread d4/f7 [0,4194304] 0 2024-03-21T10:08:49.342 INFO:tasks.workunit.client.1.smithi174.stdout:9/277: link d2/l2b d2/d11/d1d/l5d 0 2024-03-21T10:08:49.343 INFO:tasks.workunit.client.0.smithi153.stdout:7/343: mkdir d4/d14/d73 0 2024-03-21T10:08:49.345 INFO:tasks.workunit.client.1.smithi174.stdout:0/226: dread d9/d1c/f1e [0,4194304] 0 2024-03-21T10:08:49.346 INFO:tasks.workunit.client.1.smithi174.stdout:8/171: truncate d9/d17/d19/d31/f3b 238993 0 2024-03-21T10:08:49.347 INFO:tasks.workunit.client.1.smithi174.stdout:2/256: symlink d5/l5e 0 2024-03-21T10:08:49.349 INFO:tasks.workunit.client.1.smithi174.stdout:1/243: link d4/d7/d32/l4c d4/d7/d15/d4f/l58 0 2024-03-21T10:08:49.351 INFO:tasks.workunit.client.1.smithi174.stdout:9/278: chown d2/d11/d1d/d4b/c4d 0 1 2024-03-21T10:08:49.351 INFO:tasks.workunit.client.1.smithi174.stdout:8/172: chown f7 951 1 2024-03-21T10:08:49.353 INFO:tasks.workunit.client.0.smithi153.stdout:2/460: dwrite dd/d15/d18/d2c/d33/f51 [0,4194304] 0 2024-03-21T10:08:49.353 INFO:tasks.workunit.client.1.smithi174.stdout:7/170: dwrite d0/ff [0,4194304] 0 2024-03-21T10:08:49.359 INFO:tasks.workunit.client.0.smithi153.stdout:9/403: getdents d1/d43/d44/d46/d1c 0 2024-03-21T10:08:49.359 INFO:tasks.workunit.client.1.smithi174.stdout:6/177: dwrite d0/d3/d2a/f2f [0,4194304] 0 2024-03-21T10:08:49.360 INFO:tasks.workunit.client.0.smithi153.stdout:8/410: creat d5/d9/d43/dd/d20/d26/d7a/f81 x:0 0 0 2024-03-21T10:08:49.360 INFO:tasks.workunit.client.1.smithi174.stdout:9/279: dread - d2/f3c zero size 2024-03-21T10:08:49.361 INFO:tasks.workunit.client.0.smithi153.stdout:0/352: dwrite d4/d16/d27/f4d [0,4194304] 0 2024-03-21T10:08:49.362 INFO:tasks.workunit.client.0.smithi153.stdout:4/420: dwrite d11/f6f [0,4194304] 0 2024-03-21T10:08:49.362 INFO:tasks.workunit.client.1.smithi174.stdout:1/244: mkdir d4/d7/d28/d59 0 2024-03-21T10:08:49.362 INFO:tasks.workunit.client.1.smithi174.stdout:3/189: sync 2024-03-21T10:08:49.363 INFO:tasks.workunit.client.0.smithi153.stdout:6/617: getdents d1f/d2b/d33/d40/d4f 0 2024-03-21T10:08:49.367 INFO:tasks.workunit.client.1.smithi174.stdout:9/280: dread - d2/d24/f59 zero size 2024-03-21T10:08:49.369 INFO:tasks.workunit.client.1.smithi174.stdout:3/190: write d0/d12/d14/d10/f1b [4015755,56881] 0 2024-03-21T10:08:49.369 INFO:tasks.workunit.client.0.smithi153.stdout:6/618: chown d1f/d2b/f53 1577149 1 2024-03-21T10:08:49.371 INFO:tasks.workunit.client.0.smithi153.stdout:2/461: mknod dd/d3b/d50/d58/d79/c9c 0 2024-03-21T10:08:49.371 INFO:tasks.workunit.client.0.smithi153.stdout:4/421: chown d11/d18/l3c 4 1 2024-03-21T10:08:49.372 INFO:tasks.workunit.client.1.smithi174.stdout:1/245: read d4/d5/f8 [122672,26145] 0 2024-03-21T10:08:49.372 INFO:tasks.workunit.client.1.smithi174.stdout:8/173: creat d9/d17/d19/d27/f41 x:0 0 0 2024-03-21T10:08:49.373 INFO:tasks.workunit.client.1.smithi174.stdout:7/171: mknod d0/d4/d10/d1b/c3c 0 2024-03-21T10:08:49.377 INFO:tasks.workunit.client.0.smithi153.stdout:9/404: mkdir d1/d8a 0 2024-03-21T10:08:49.378 INFO:tasks.workunit.client.1.smithi174.stdout:1/246: write d4/d7/d15/f41 [827382,99132] 0 2024-03-21T10:08:49.379 INFO:tasks.workunit.client.0.smithi153.stdout:7/344: creat d4/d2a/d40/f74 x:0 0 0 2024-03-21T10:08:49.381 INFO:tasks.workunit.client.0.smithi153.stdout:6/619: rmdir d1f/d2b/d33/d40/d65 39 2024-03-21T10:08:49.382 INFO:tasks.workunit.client.1.smithi174.stdout:2/257: mknod d5/d25/d27/d2e/d4f/c5f 0 2024-03-21T10:08:49.384 INFO:tasks.workunit.client.1.smithi174.stdout:1/247: fdatasync d4/d7/f2e 0 2024-03-21T10:08:49.384 INFO:tasks.workunit.client.1.smithi174.stdout:9/281: mkdir d2/d11/d1d/d5e 0 2024-03-21T10:08:49.386 INFO:tasks.workunit.client.1.smithi174.stdout:3/191: symlink d0/d12/d14/d10/l48 0 2024-03-21T10:08:49.388 INFO:tasks.workunit.client.0.smithi153.stdout:2/462: write dd/d15/d18/d2c/d4b/d56/f60 [3826963,73443] 0 2024-03-21T10:08:49.392 INFO:tasks.workunit.client.1.smithi174.stdout:8/174: unlink d9/fd 0 2024-03-21T10:08:49.396 INFO:tasks.workunit.client.0.smithi153.stdout:5/442: dwrite d4/d8/d3b/d56/f62 [0,4194304] 0 2024-03-21T10:08:49.397 INFO:tasks.workunit.client.0.smithi153.stdout:7/345: write d4/d2a/d3b/d60/f64 [380647,95448] 0 2024-03-21T10:08:49.399 INFO:tasks.workunit.client.1.smithi174.stdout:7/172: symlink d0/d4/d38/l3d 0 2024-03-21T10:08:49.399 INFO:tasks.workunit.client.1.smithi174.stdout:6/178: dwrite d0/d3/d5/f1c [0,4194304] 0 2024-03-21T10:08:49.402 INFO:tasks.workunit.client.1.smithi174.stdout:0/227: getdents d9/d21/d23 0 2024-03-21T10:08:49.402 INFO:tasks.workunit.client.0.smithi153.stdout:2/463: write dd/d3b/f5b [3707540,26677] 0 2024-03-21T10:08:49.403 INFO:tasks.workunit.client.0.smithi153.stdout:6/620: write d1f/d2b/d33/d40/d5f/f72 [2735313,6571] 0 2024-03-21T10:08:49.410 INFO:tasks.workunit.client.1.smithi174.stdout:1/248: unlink d4/d7/d32/c4b 0 2024-03-21T10:08:49.411 INFO:tasks.workunit.client.1.smithi174.stdout:0/228: write d9/d21/d31/f42 [791065,81273] 0 2024-03-21T10:08:49.411 INFO:tasks.workunit.client.1.smithi174.stdout:5/223: dwrite d1/d16/d1c/f26 [0,4194304] 0 2024-03-21T10:08:49.417 INFO:tasks.workunit.client.1.smithi174.stdout:4/259: dwrite d4/f7 [0,4194304] 0 2024-03-21T10:08:49.417 INFO:tasks.workunit.client.0.smithi153.stdout:4/422: dwrite d11/d18/d50/f53 [4194304,4194304] 0 2024-03-21T10:08:49.418 INFO:tasks.workunit.client.0.smithi153.stdout:0/353: dread d4/f10 [4194304,4194304] 0 2024-03-21T10:08:49.420 INFO:tasks.workunit.client.0.smithi153.stdout:5/443: truncate d4/d5/de/d44/d48/f84 644718 0 2024-03-21T10:08:49.423 INFO:tasks.workunit.client.0.smithi153.stdout:0/354: write d4/d16/d1f/f20 [2765635,27380] 0 2024-03-21T10:08:49.423 INFO:tasks.workunit.client.0.smithi153.stdout:2/464: rename dd/d15/d18/d2c/d33/f62 to dd/d15/d18/d2c/d33/f9d 0 2024-03-21T10:08:49.425 INFO:tasks.workunit.client.0.smithi153.stdout:7/346: link d4/d11/d24/c26 d4/d1b/d20/c75 0 2024-03-21T10:08:49.425 INFO:tasks.workunit.client.1.smithi174.stdout:8/175: rename d9/f36 to d9/d17/f42 0 2024-03-21T10:08:49.426 INFO:tasks.workunit.client.0.smithi153.stdout:8/411: dwrite d5/d1b/f5e [0,4194304] 0 2024-03-21T10:08:49.428 INFO:tasks.workunit.client.0.smithi153.stdout:6/621: mkdir d1f/d93/dd8/d59/d70/ddc 0 2024-03-21T10:08:49.428 INFO:tasks.workunit.client.0.smithi153.stdout:2/465: stat dd/d15/d3c/l6c 0 2024-03-21T10:08:49.429 INFO:tasks.workunit.client.0.smithi153.stdout:0/355: chown d4/d16/d1f/f31 2463 1 2024-03-21T10:08:49.430 INFO:tasks.workunit.client.1.smithi174.stdout:7/173: unlink d0/d4/d10/d1b/c3c 0 2024-03-21T10:08:49.433 INFO:tasks.workunit.client.0.smithi153.stdout:5/444: creat d4/d5/d16/f8a x:0 0 0 2024-03-21T10:08:49.435 INFO:tasks.workunit.client.0.smithi153.stdout:2/466: read dd/d15/d18/d2c/d33/f88 [201391,60968] 0 2024-03-21T10:08:49.436 INFO:tasks.workunit.client.1.smithi174.stdout:0/229: symlink d9/d21/l48 0 2024-03-21T10:08:49.437 INFO:tasks.workunit.client.1.smithi174.stdout:7/174: read - d0/d4/f22 zero size 2024-03-21T10:08:49.443 INFO:tasks.workunit.client.0.smithi153.stdout:8/412: rename d5/d9/d43/dd/f22 to d5/d9/d43/dd/d20/d26/d24/f82 0 2024-03-21T10:08:49.446 INFO:tasks.workunit.client.1.smithi174.stdout:4/260: symlink d4/d21/d31/l5a 0 2024-03-21T10:08:49.447 INFO:tasks.workunit.client.1.smithi174.stdout:3/192: dwrite d0/f3 [4194304,4194304] 0 2024-03-21T10:08:49.448 INFO:tasks.workunit.client.0.smithi153.stdout:8/413: fsync d5/d9/d43/f35 0 2024-03-21T10:08:49.450 INFO:tasks.workunit.client.1.smithi174.stdout:6/179: getdents d0/d3/d5/d3b 0 2024-03-21T10:08:49.452 INFO:tasks.workunit.client.1.smithi174.stdout:3/193: fsync d0/f33 0 2024-03-21T10:08:49.454 INFO:tasks.workunit.client.1.smithi174.stdout:4/261: chown d4/d19/l30 10910 1 2024-03-21T10:08:49.454 INFO:tasks.workunit.client.1.smithi174.stdout:2/258: getdents d5/d25 0 2024-03-21T10:08:49.457 INFO:tasks.workunit.client.1.smithi174.stdout:9/282: link d2/d11/d1d/l43 d2/d24/d2e/l5f 0 2024-03-21T10:08:49.457 INFO:tasks.workunit.client.0.smithi153.stdout:8/414: symlink d5/d9/d43/dd/d20/l83 0 2024-03-21T10:08:49.458 INFO:tasks.workunit.client.1.smithi174.stdout:1/249: dwrite d4/d7/d32/f3c [4194304,4194304] 0 2024-03-21T10:08:49.461 INFO:tasks.workunit.client.1.smithi174.stdout:7/175: rename d0/f9 to d0/d4/d10/d1b/f3e 0 2024-03-21T10:08:49.461 INFO:tasks.workunit.client.0.smithi153.stdout:6/622: dwrite d1f/d2b/fa7 [0,4194304] 0 2024-03-21T10:08:49.464 INFO:tasks.workunit.client.0.smithi153.stdout:2/467: link dd/d15/d18/c1f dd/d15/d18/d2c/d4b/d56/d63/d8d/d9b/c9e 0 2024-03-21T10:08:49.464 INFO:tasks.workunit.client.0.smithi153.stdout:5/445: dwrite d4/d5/de/f43 [0,4194304] 0 2024-03-21T10:08:49.468 INFO:tasks.workunit.client.1.smithi174.stdout:4/262: creat d4/d21/f5b x:0 0 0 2024-03-21T10:08:49.469 INFO:tasks.workunit.client.1.smithi174.stdout:7/176: readlink d0/d4/l15 0 2024-03-21T10:08:49.470 INFO:tasks.workunit.client.1.smithi174.stdout:5/224: link d1/d16/c28 d1/d16/d2e/c4d 0 2024-03-21T10:08:49.470 INFO:tasks.workunit.client.0.smithi153.stdout:2/468: fsync dd/d15/d18/d3a/d3d/f4f 0 2024-03-21T10:08:49.471 INFO:tasks.workunit.client.1.smithi174.stdout:1/250: mknod d4/d7/c5a 0 2024-03-21T10:08:49.472 INFO:tasks.workunit.client.1.smithi174.stdout:0/230: rename d9/d21/d23/f25 to d9/d21/d31/d3c/f49 0 2024-03-21T10:08:49.472 INFO:tasks.workunit.client.0.smithi153.stdout:4/423: dread d11/d18/f1d [4194304,4194304] 0 2024-03-21T10:08:49.476 INFO:tasks.workunit.client.1.smithi174.stdout:2/259: creat d5/d25/d27/d2d/d45/d5b/f60 x:0 0 0 2024-03-21T10:08:49.479 INFO:tasks.workunit.client.0.smithi153.stdout:5/446: mknod d4/d8/d5c/c8b 0 2024-03-21T10:08:49.479 INFO:tasks.workunit.client.1.smithi174.stdout:8/176: dwrite d9/d17/f21 [0,4194304] 0 2024-03-21T10:08:49.480 INFO:tasks.workunit.client.1.smithi174.stdout:2/260: write d5/f38 [6160182,31487] 0 2024-03-21T10:08:49.483 INFO:tasks.workunit.client.1.smithi174.stdout:1/251: unlink d4/d2d/c56 0 2024-03-21T10:08:49.485 INFO:tasks.workunit.client.0.smithi153.stdout:5/447: creat d4/d5/d16/d24/f8c x:0 0 0 2024-03-21T10:08:49.489 INFO:tasks.workunit.client.1.smithi174.stdout:2/261: fsync d5/d25/d27/d2e/f3f 0 2024-03-21T10:08:49.489 INFO:tasks.workunit.client.0.smithi153.stdout:0/356: truncate d4/d16/d27/f39 3104697 0 2024-03-21T10:08:49.489 INFO:tasks.workunit.client.0.smithi153.stdout:5/448: chown d4/d5/de/d38 99614305 1 2024-03-21T10:08:49.492 INFO:tasks.workunit.client.0.smithi153.stdout:5/449: chown d4/d5/df/d5e/f6e 906 1 2024-03-21T10:08:49.493 INFO:tasks.workunit.client.0.smithi153.stdout:0/357: write d4/d16/d1f/f31 [522353,76608] 0 2024-03-21T10:08:49.493 INFO:tasks.workunit.client.1.smithi174.stdout:8/177: mknod d9/d2a/c43 0 2024-03-21T10:08:49.495 INFO:tasks.workunit.client.1.smithi174.stdout:3/194: dread d0/f2 [4194304,4194304] 0 2024-03-21T10:08:49.496 INFO:tasks.workunit.client.1.smithi174.stdout:4/263: fdatasync d4/d21/d24/d28/f39 0 2024-03-21T10:08:49.497 INFO:tasks.workunit.client.1.smithi174.stdout:1/252: creat d4/d7/f5b x:0 0 0 2024-03-21T10:08:49.497 INFO:tasks.workunit.client.0.smithi153.stdout:2/469: dwrite dd/d15/d18/d2c/d33/f7d [0,4194304] 0 2024-03-21T10:08:49.498 INFO:tasks.workunit.client.0.smithi153.stdout:8/415: dread d5/d9/d43/dd/d20/d26/d24/d52/d55/f71 [0,4194304] 0 2024-03-21T10:08:49.500 INFO:tasks.workunit.client.0.smithi153.stdout:5/450: mkdir d4/d5/d16/d24/d8d 0 2024-03-21T10:08:49.501 INFO:tasks.workunit.client.1.smithi174.stdout:4/264: read - d4/d19/f58 zero size 2024-03-21T10:08:49.503 INFO:tasks.workunit.client.1.smithi174.stdout:2/262: creat d5/d25/d27/d44/f61 x:0 0 0 2024-03-21T10:08:49.503 INFO:tasks.workunit.client.1.smithi174.stdout:8/178: rmdir d9/d17/d19/d33 39 2024-03-21T10:08:49.506 INFO:tasks.workunit.client.0.smithi153.stdout:8/416: mknod d5/d9/d43/c84 0 2024-03-21T10:08:49.507 INFO:tasks.workunit.client.0.smithi153.stdout:8/417: stat d5/d3b/f4f 0 2024-03-21T10:08:49.508 INFO:tasks.workunit.client.1.smithi174.stdout:9/283: dwrite d2/d24/f2f [0,4194304] 0 2024-03-21T10:08:49.509 INFO:tasks.workunit.client.0.smithi153.stdout:9/405: dread d1/f1b [0,4194304] 0 2024-03-21T10:08:49.509 INFO:tasks.workunit.client.0.smithi153.stdout:5/451: unlink d4/d5/df/l60 0 2024-03-21T10:08:49.510 INFO:tasks.workunit.client.0.smithi153.stdout:8/418: stat d5/d9/d43/dd/d20/d26/d47 0 2024-03-21T10:08:49.510 INFO:tasks.workunit.client.1.smithi174.stdout:8/179: readlink d9/l25 0 2024-03-21T10:08:49.512 INFO:tasks.workunit.client.1.smithi174.stdout:4/265: chown d4/d21/d24/f4d 1544110 1 2024-03-21T10:08:49.513 INFO:tasks.workunit.client.1.smithi174.stdout:4/266: readlink d4/l2a 0 2024-03-21T10:08:49.514 INFO:tasks.workunit.client.1.smithi174.stdout:3/195: truncate d0/d12/d14/f45 128828 0 2024-03-21T10:08:49.517 INFO:tasks.workunit.client.1.smithi174.stdout:1/253: mknod d4/d7/d33/d43/d51/c5c 0 2024-03-21T10:08:49.522 INFO:tasks.workunit.client.1.smithi174.stdout:3/196: chown d0/d12/d14/d1a/l2b 999 1 2024-03-21T10:08:49.522 INFO:tasks.workunit.client.1.smithi174.stdout:2/263: creat d5/d25/d27/d2d/d45/f62 x:0 0 0 2024-03-21T10:08:49.523 INFO:tasks.workunit.client.1.smithi174.stdout:0/231: dwrite d9/fc [0,4194304] 0 2024-03-21T10:08:49.523 INFO:tasks.workunit.client.0.smithi153.stdout:5/452: mknod d4/d5/df/d5e/d69/c8e 0 2024-03-21T10:08:49.524 INFO:tasks.workunit.client.0.smithi153.stdout:8/419: mkdir d5/d9/d43/dd/d20/d26/d33/d85 0 2024-03-21T10:08:49.527 INFO:tasks.workunit.client.1.smithi174.stdout:8/180: mkdir d9/d37/d44 0 2024-03-21T10:08:49.528 INFO:tasks.workunit.client.0.smithi153.stdout:9/406: rename d1/d4e/d68/d78 to d1/d43/d44/d46/d1c/d4b/d84/d8b 0 2024-03-21T10:08:49.528 INFO:tasks.workunit.client.0.smithi153.stdout:5/453: mknod d4/d5/df/d67/d6a/c8f 0 2024-03-21T10:08:49.530 INFO:tasks.workunit.client.1.smithi174.stdout:1/254: rmdir d4/d7/d15 39 2024-03-21T10:08:49.531 INFO:tasks.workunit.client.1.smithi174.stdout:9/284: mkdir d2/d11/d60 0 2024-03-21T10:08:49.534 INFO:tasks.workunit.client.1.smithi174.stdout:8/181: write d9/d17/f21 [3265575,3878] 0 2024-03-21T10:08:49.535 INFO:tasks.workunit.client.0.smithi153.stdout:2/470: dwrite dd/d15/d18/d2c/d4b/d56/d63/f89 [0,4194304] 0 2024-03-21T10:08:49.535 INFO:tasks.workunit.client.0.smithi153.stdout:4/424: dwrite fd [0,4194304] 0 2024-03-21T10:08:49.535 INFO:tasks.workunit.client.0.smithi153.stdout:9/407: mknod d1/d3b/c8c 0 2024-03-21T10:08:49.535 INFO:tasks.workunit.client.1.smithi174.stdout:0/232: mkdir d9/d35/d4a 0 2024-03-21T10:08:49.538 INFO:tasks.workunit.client.0.smithi153.stdout:8/420: getdents d5/d9/d43/d7b 0 2024-03-21T10:08:49.539 INFO:tasks.workunit.client.0.smithi153.stdout:2/471: unlink dd/d15/d18/d2c/l64 0 2024-03-21T10:08:49.541 INFO:tasks.workunit.client.1.smithi174.stdout:0/233: chown d9/f33 384122296 1 2024-03-21T10:08:49.543 INFO:tasks.workunit.client.0.smithi153.stdout:2/472: stat dd/d3b/f77 0 2024-03-21T10:08:49.543 INFO:tasks.workunit.client.0.smithi153.stdout:9/408: write d1/d4e/f54 [124894,62469] 0 2024-03-21T10:08:49.544 INFO:tasks.workunit.client.1.smithi174.stdout:9/285: symlink d2/d11/d60/l61 0 2024-03-21T10:08:49.544 INFO:tasks.workunit.client.1.smithi174.stdout:5/225: dwrite d1/f22 [0,4194304] 0 2024-03-21T10:08:49.546 INFO:tasks.workunit.client.0.smithi153.stdout:2/473: rmdir dd/d15/d5e 39 2024-03-21T10:08:49.548 INFO:tasks.workunit.client.0.smithi153.stdout:4/425: mkdir d11/d18/d7e/d7f 0 2024-03-21T10:08:49.549 INFO:tasks.workunit.client.1.smithi174.stdout:8/182: rename d9/l25 to d9/d17/d19/d27/l45 0 2024-03-21T10:08:49.550 INFO:tasks.workunit.client.0.smithi153.stdout:9/409: mknod d1/d8a/c8d 0 2024-03-21T10:08:49.553 INFO:tasks.workunit.client.1.smithi174.stdout:8/183: chown f5 226 1 2024-03-21T10:08:49.555 INFO:tasks.workunit.client.1.smithi174.stdout:2/264: dread d5/d25/d27/d2d/d45/d5b/f29 [0,4194304] 0 2024-03-21T10:08:49.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:49 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:08:49.556 INFO:tasks.workunit.client.0.smithi153.stdout:4/426: link d11/d18/l4d d11/d18/d50/d65/l80 0 2024-03-21T10:08:49.559 INFO:tasks.workunit.client.1.smithi174.stdout:9/286: rename d2/d11/f53 to d2/f62 0 2024-03-21T10:08:49.559 INFO:tasks.workunit.client.0.smithi153.stdout:2/474: truncate dd/d15/d18/d3a/d3d/f76 79891 0 2024-03-21T10:08:49.561 INFO:tasks.workunit.client.0.smithi153.stdout:4/427: write d11/d2d/f69 [555003,105571] 0 2024-03-21T10:08:49.564 INFO:tasks.workunit.client.1.smithi174.stdout:9/287: mkdir d2/d24/d2e/d63 0 2024-03-21T10:08:49.564 INFO:tasks.workunit.client.1.smithi174.stdout:3/197: dwrite d0/d12/d14/d1a/f3d [0,4194304] 0 2024-03-21T10:08:49.565 INFO:tasks.workunit.client.0.smithi153.stdout:9/410: link d1/c13 d1/d43/d44/d46/d1c/d2e/d67/c8e 0 2024-03-21T10:08:49.565 INFO:tasks.workunit.client.0.smithi153.stdout:4/428: fdatasync d11/d18/f41 0 2024-03-21T10:08:49.567 INFO:tasks.workunit.client.0.smithi153.stdout:0/358: dwrite d4/d16/d1f/d22/f2e [0,4194304] 0 2024-03-21T10:08:49.568 INFO:tasks.workunit.client.0.smithi153.stdout:8/421: dwrite d5/d1b/f5e [0,4194304] 0 2024-03-21T10:08:49.571 INFO:tasks.workunit.client.0.smithi153.stdout:5/454: dread d4/d5/df/f30 [0,4194304] 0 2024-03-21T10:08:49.571 INFO:tasks.workunit.client.1.smithi174.stdout:9/288: truncate d2/d11/f2c 1610041 0 2024-03-21T10:08:49.571 INFO:tasks.workunit.client.1.smithi174.stdout:0/234: rename d9/d1c/l3d to d9/d21/l4b 0 2024-03-21T10:08:49.572 INFO:tasks.workunit.client.0.smithi153.stdout:0/359: truncate d4/d16/d3e/f69 351898 0 2024-03-21T10:08:49.573 INFO:tasks.workunit.client.1.smithi174.stdout:3/198: rmdir d0/d12/d14 39 2024-03-21T10:08:49.573 INFO:tasks.workunit.client.0.smithi153.stdout:9/411: rename d1/d82 to d1/d43/d44/d46/d1c/d2e/d8f 0 2024-03-21T10:08:49.574 INFO:tasks.workunit.client.0.smithi153.stdout:5/455: mkdir d4/d8/d5c/d90 0 2024-03-21T10:08:49.575 INFO:tasks.workunit.client.1.smithi174.stdout:5/226: dread d1/ff [0,4194304] 0 2024-03-21T10:08:49.576 INFO:tasks.workunit.client.1.smithi174.stdout:0/235: unlink l3 0 2024-03-21T10:08:49.577 INFO:tasks.workunit.client.0.smithi153.stdout:0/360: readlink d4/d16/d3e/l60 0 2024-03-21T10:08:49.577 INFO:tasks.workunit.client.0.smithi153.stdout:5/456: fsync d4/d5/de/f43 0 2024-03-21T10:08:49.579 INFO:tasks.workunit.client.0.smithi153.stdout:4/429: link d11/d18/c3e d11/d18/d54/c81 0 2024-03-21T10:08:49.580 INFO:tasks.workunit.client.0.smithi153.stdout:8/422: creat d5/d9/d43/dd/d20/d26/d47/d69/f86 x:0 0 0 2024-03-21T10:08:49.581 INFO:tasks.workunit.client.0.smithi153.stdout:9/412: symlink d1/d43/d44/d46/d1c/d2e/d67/l90 0 2024-03-21T10:08:49.583 INFO:tasks.workunit.client.1.smithi174.stdout:8/184: dwrite f1 [0,4194304] 0 2024-03-21T10:08:49.585 INFO:tasks.workunit.client.1.smithi174.stdout:9/289: creat d2/d11/d60/f64 x:0 0 0 2024-03-21T10:08:49.587 INFO:tasks.workunit.client.0.smithi153.stdout:5/457: symlink d4/d5/df/d67/d86/l91 0 2024-03-21T10:08:49.589 INFO:tasks.workunit.client.0.smithi153.stdout:9/413: link d1/d43/d44/d46/d1c/l4c d1/d4/d5f/l91 0 2024-03-21T10:08:49.589 INFO:tasks.workunit.client.0.smithi153.stdout:5/458: getdents d4/d8/d5c/d90 0 2024-03-21T10:08:49.592 INFO:tasks.workunit.client.0.smithi153.stdout:5/459: mkdir d4/d8/d5a/d92 0 2024-03-21T10:08:49.593 INFO:tasks.workunit.client.1.smithi174.stdout:0/236: creat d9/d21/d31/f4c x:0 0 0 2024-03-21T10:08:49.593 INFO:tasks.workunit.client.0.smithi153.stdout:9/414: chown d1/c13 45514 1 2024-03-21T10:08:49.594 INFO:tasks.workunit.client.0.smithi153.stdout:5/460: chown d4/d5/de/d44/f6f 9891051 1 2024-03-21T10:08:49.596 INFO:tasks.workunit.client.1.smithi174.stdout:4/267: dread d4/fe [0,4194304] 0 2024-03-21T10:08:49.596 INFO:tasks.workunit.client.0.smithi153.stdout:9/415: truncate d1/d43/d44/d46/d1c/d61/f7e 654922 0 2024-03-21T10:08:49.597 INFO:tasks.workunit.client.1.smithi174.stdout:5/227: truncate d1/d16/f40 176165 0 2024-03-21T10:08:49.598 INFO:tasks.workunit.client.0.smithi153.stdout:5/461: mknod d4/d5/d16/c93 0 2024-03-21T10:08:49.600 INFO:tasks.workunit.client.0.smithi153.stdout:5/462: write d4/f75 [709906,31439] 0 2024-03-21T10:08:49.600 INFO:tasks.workunit.client.0.smithi153.stdout:9/416: readlink d1/d4/d5f/l91 0 2024-03-21T10:08:49.601 INFO:tasks.workunit.client.1.smithi174.stdout:9/290: mknod d2/d24/c65 0 2024-03-21T10:08:49.602 INFO:tasks.workunit.client.0.smithi153.stdout:5/463: write d4/d8/f11 [5591580,21367] 0 2024-03-21T10:08:49.603 INFO:tasks.workunit.client.0.smithi153.stdout:9/417: symlink d1/d8a/l92 0 2024-03-21T10:08:49.607 INFO:tasks.workunit.client.0.smithi153.stdout:5/464: rmdir d4/d8/d3b/d56 39 2024-03-21T10:08:49.608 INFO:tasks.workunit.client.0.smithi153.stdout:5/465: write d4/d8/d3b/f4b [3955221,1249] 0 2024-03-21T10:08:49.614 INFO:tasks.workunit.client.1.smithi174.stdout:4/268: dread d4/d21/d24/d28/f39 [0,4194304] 0 2024-03-21T10:08:49.615 INFO:tasks.workunit.client.1.smithi174.stdout:8/185: dwrite d9/d17/f20 [0,4194304] 0 2024-03-21T10:08:49.616 INFO:tasks.workunit.client.1.smithi174.stdout:8/186: chown f1 12 1 2024-03-21T10:08:49.616 INFO:tasks.workunit.client.0.smithi153.stdout:8/423: truncate d5/d9/d43/f16 6590818 0 2024-03-21T10:08:49.620 INFO:tasks.workunit.client.1.smithi174.stdout:4/269: mknod d4/d21/d24/d28/c5c 0 2024-03-21T10:08:49.621 INFO:tasks.workunit.client.1.smithi174.stdout:4/270: dread - d4/d19/f50 zero size 2024-03-21T10:08:49.623 INFO:tasks.workunit.client.0.smithi153.stdout:3/377: sync 2024-03-21T10:08:49.633 INFO:tasks.workunit.client.0.smithi153.stdout:8/424: rmdir d5/d9/d43/dd/d20/d26/d24 39 2024-03-21T10:08:49.635 INFO:tasks.workunit.client.1.smithi174.stdout:4/271: dwrite d4/d19/f23 [0,4194304] 0 2024-03-21T10:08:49.636 INFO:tasks.workunit.client.1.smithi174.stdout:4/272: chown d4/lb 36681723 1 2024-03-21T10:08:49.637 INFO:tasks.workunit.client.0.smithi153.stdout:4/430: dwrite d11/d18/f1f [0,4194304] 0 2024-03-21T10:08:49.638 INFO:tasks.workunit.client.1.smithi174.stdout:4/273: write d4/f18 [2949190,102290] 0 2024-03-21T10:08:49.640 INFO:tasks.workunit.client.0.smithi153.stdout:9/418: dwrite d1/d43/d44/f33 [4194304,4194304] 0 2024-03-21T10:08:49.649 INFO:tasks.workunit.client.0.smithi153.stdout:5/466: dwrite d4/d5/de/d2f/f41 [0,4194304] 0 2024-03-21T10:08:49.653 INFO:tasks.workunit.client.0.smithi153.stdout:8/425: chown d5/d9/d43/dd/d20/d26/d24/d52/f64 330501331 1 2024-03-21T10:08:49.656 INFO:tasks.workunit.client.0.smithi153.stdout:9/419: dread d1/d43/d44/d46/d1c/f31 [0,4194304] 0 2024-03-21T10:08:49.660 INFO:tasks.workunit.client.0.smithi153.stdout:8/426: creat d5/d1b/f87 x:0 0 0 2024-03-21T10:08:49.667 INFO:tasks.workunit.client.0.smithi153.stdout:9/420: symlink d1/d43/d44/d46/d1c/d4b/l93 0 2024-03-21T10:08:49.670 INFO:tasks.workunit.client.1.smithi174.stdout:6/180: truncate d0/d3/d5/df/d16/f34 898077 0 2024-03-21T10:08:49.673 INFO:tasks.workunit.client.1.smithi174.stdout:7/177: truncate d0/d4/fd 3325371 0 2024-03-21T10:08:49.674 INFO:tasks.workunit.client.1.smithi174.stdout:6/181: chown d0/d3/d5/df/f2e 75625 1 2024-03-21T10:08:49.674 INFO:tasks.workunit.client.0.smithi153.stdout:9/421: symlink d1/l94 0 2024-03-21T10:08:49.676 INFO:tasks.workunit.client.1.smithi174.stdout:6/182: write d0/d3/d5/f1c [937176,95120] 0 2024-03-21T10:08:49.678 INFO:tasks.workunit.client.0.smithi153.stdout:8/427: rename d5/d9/d43/dd/d20/d26/d33 to d5/d9/d43/dd/d2d/d45/d88 0 2024-03-21T10:08:49.679 INFO:tasks.workunit.client.1.smithi174.stdout:7/178: getdents d0/d4 0 2024-03-21T10:08:49.680 INFO:tasks.workunit.client.1.smithi174.stdout:6/183: mkdir d0/d3/d20/d3d 0 2024-03-21T10:08:49.687 INFO:tasks.workunit.client.1.smithi174.stdout:6/184: truncate d0/d3/d5/fe 1332816 0 2024-03-21T10:08:49.687 INFO:tasks.workunit.client.0.smithi153.stdout:4/431: dread d11/f24 [0,4194304] 0 2024-03-21T10:08:49.688 INFO:tasks.workunit.client.1.smithi174.stdout:6/185: write d0/d3/f26 [1755532,54505] 0 2024-03-21T10:08:49.690 INFO:tasks.workunit.client.0.smithi153.stdout:8/428: write d5/d9/d43/dd/d2d/d45/d88/f76 [2761962,46731] 0 2024-03-21T10:08:49.691 INFO:tasks.workunit.client.0.smithi153.stdout:4/432: chown d11/d71 0 1 2024-03-21T10:08:49.693 INFO:tasks.workunit.client.1.smithi174.stdout:6/186: mknod d0/d3/d5/df/c3e 0 2024-03-21T10:08:49.694 INFO:tasks.workunit.client.1.smithi174.stdout:6/187: chown d0/d3/l10 141402660 1 2024-03-21T10:08:49.699 INFO:tasks.workunit.client.0.smithi153.stdout:2/475: read dd/d15/d18/d2c/d4b/f67 [50497,126259] 0 2024-03-21T10:08:49.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:49 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:08:49.701 INFO:tasks.workunit.client.0.smithi153.stdout:5/467: dwrite d4/d8/f2d [0,4194304] 0 2024-03-21T10:08:49.703 INFO:tasks.workunit.client.1.smithi174.stdout:6/188: link d0/d3/d5/d6/l11 d0/d3/d5/df/d16/l3f 0 2024-03-21T10:08:49.706 INFO:tasks.workunit.client.0.smithi153.stdout:8/429: rename d5/d9/f11 to d5/d9/d43/dd/d20/d26/d24/d52/d55/f89 0 2024-03-21T10:08:49.712 INFO:tasks.workunit.client.1.smithi174.stdout:2/265: rename d5/d25/d27/d44 to d5/d25/d27/d2d/d45/d5b/d63 0 2024-03-21T10:08:49.718 INFO:tasks.workunit.client.1.smithi174.stdout:2/266: chown d5/d25/d27/d2d/l42 136331 1 2024-03-21T10:08:49.718 INFO:tasks.workunit.client.0.smithi153.stdout:5/468: creat d4/d8/d5a/d92/f94 x:0 0 0 2024-03-21T10:08:49.720 INFO:tasks.workunit.client.1.smithi174.stdout:3/199: rename d0/d12/d14/f45 to d0/d12/f49 0 2024-03-21T10:08:49.720 INFO:tasks.workunit.client.1.smithi174.stdout:6/189: dwrite d0/d3/d5/f3c [0,4194304] 0 2024-03-21T10:08:49.724 INFO:tasks.workunit.client.0.smithi153.stdout:9/422: dread d1/d43/d44/d46/d1c/d24/f32 [0,4194304] 0 2024-03-21T10:08:49.725 INFO:tasks.workunit.client.0.smithi153.stdout:3/378: dread dd/f27 [0,4194304] 0 2024-03-21T10:08:49.725 INFO:tasks.workunit.client.1.smithi174.stdout:2/267: rename d5/d25/d27/d2d/d45/d5b/d55 to d5/d25/d27/d2d/d45/d5b/d55/d64 22 2024-03-21T10:08:49.726 INFO:tasks.workunit.client.0.smithi153.stdout:4/433: dwrite d11/d2d/f38 [0,4194304] 0 2024-03-21T10:08:49.731 INFO:tasks.workunit.client.0.smithi153.stdout:5/469: mkdir d4/d8/d5c/d90/d95 0 2024-03-21T10:08:49.733 INFO:tasks.workunit.client.0.smithi153.stdout:3/379: read dd/f20 [1639226,20110] 0 2024-03-21T10:08:49.734 INFO:tasks.workunit.client.1.smithi174.stdout:3/200: dread d0/f13 [0,4194304] 0 2024-03-21T10:08:49.738 INFO:tasks.workunit.client.0.smithi153.stdout:5/470: read d4/d8/d5a/f51 [3320119,2826] 0 2024-03-21T10:08:49.739 INFO:tasks.workunit.client.0.smithi153.stdout:9/423: symlink d1/d43/d44/d46/d1c/d4b/d84/l95 0 2024-03-21T10:08:49.740 INFO:tasks.workunit.client.1.smithi174.stdout:1/255: truncate d4/d7/d28/f30 364836 0 2024-03-21T10:08:49.742 INFO:tasks.workunit.client.0.smithi153.stdout:3/380: symlink dd/d1e/d3d/l89 0 2024-03-21T10:08:49.743 INFO:tasks.workunit.client.0.smithi153.stdout:9/424: stat d1/d4e/d68/f79 0 2024-03-21T10:08:49.745 INFO:tasks.workunit.client.0.smithi153.stdout:5/471: write d4/d5/f7 [4292406,18083] 0 2024-03-21T10:08:49.748 INFO:tasks.workunit.client.0.smithi153.stdout:8/430: dwrite d5/d9/d43/dd/d2d/f66 [0,4194304] 0 2024-03-21T10:08:49.748 INFO:tasks.workunit.client.1.smithi174.stdout:9/291: rmdir d2 39 2024-03-21T10:08:49.751 INFO:tasks.workunit.client.1.smithi174.stdout:8/187: write d9/f15 [4608169,79031] 0 2024-03-21T10:08:49.751 INFO:tasks.workunit.client.0.smithi153.stdout:8/431: fsync d5/d9/d43/f35 0 2024-03-21T10:08:49.754 INFO:tasks.workunit.client.0.smithi153.stdout:2/476: dwrite dd/d15/f97 [4194304,4194304] 0 2024-03-21T10:08:49.755 INFO:tasks.workunit.client.0.smithi153.stdout:3/381: rename dd/d1e/d6b/c6f to dd/d7e/d73/c8a 0 2024-03-21T10:08:49.759 INFO:tasks.workunit.client.0.smithi153.stdout:9/425: unlink d1/d43/d44/d46/d1c/d61/f7e 0 2024-03-21T10:08:49.764 INFO:tasks.workunit.client.1.smithi174.stdout:9/292: rename d2/c1c to d2/d24/d2e/d63/c66 0 2024-03-21T10:08:49.765 INFO:tasks.workunit.client.0.smithi153.stdout:5/472: rename d4/d5/d16/d24/d70 to d4/d5/df/d67/d86/d96 0 2024-03-21T10:08:49.767 INFO:tasks.workunit.client.0.smithi153.stdout:3/382: chown dd/d1e/d3d/d53 717 1 2024-03-21T10:08:49.767 INFO:tasks.workunit.client.1.smithi174.stdout:7/179: write d0/d4/f17 [853382,91013] 0 2024-03-21T10:08:49.770 INFO:tasks.workunit.client.0.smithi153.stdout:8/432: creat d5/d9/d43/dd/d20/d26/d24/d52/d56/f8a x:0 0 0 2024-03-21T10:08:49.771 INFO:tasks.workunit.client.0.smithi153.stdout:4/434: dread d11/f19 [0,4194304] 0 2024-03-21T10:08:49.772 INFO:tasks.workunit.client.0.smithi153.stdout:5/473: write d4/d5/d16/d24/f8c [176537,31338] 0 2024-03-21T10:08:49.775 INFO:tasks.workunit.client.1.smithi174.stdout:9/293: creat d2/d24/d2e/f67 x:0 0 0 2024-03-21T10:08:49.776 INFO:tasks.workunit.client.0.smithi153.stdout:5/474: write d4/d5/df/f40 [656126,72911] 0 2024-03-21T10:08:49.776 INFO:tasks.workunit.client.0.smithi153.stdout:2/477: rename dd/d15/d3c/l6c to dd/d15/d18/d2c/d4b/d56/d63/d8d/d9b/l9f 0 2024-03-21T10:08:49.777 INFO:tasks.workunit.client.1.smithi174.stdout:7/180: unlink d0/f32 0 2024-03-21T10:08:49.777 INFO:tasks.workunit.client.1.smithi174.stdout:9/294: write d2/fc [3299857,53669] 0 2024-03-21T10:08:49.781 INFO:tasks.workunit.client.0.smithi153.stdout:1/359: sync 2024-03-21T10:08:49.782 INFO:tasks.workunit.client.1.smithi174.stdout:2/268: rmdir d5/d25/d27/d2d/d45/d5b 39 2024-03-21T10:08:49.782 INFO:tasks.workunit.client.0.smithi153.stdout:4/435: read d11/d2d/f4a [885037,119134] 0 2024-03-21T10:08:49.783 INFO:tasks.workunit.client.0.smithi153.stdout:3/383: creat dd/d1e/d45/d56/d58/f8b x:0 0 0 2024-03-21T10:08:49.783 INFO:tasks.workunit.client.0.smithi153.stdout:5/475: fdatasync d4/d8/f6c 0 2024-03-21T10:08:49.783 INFO:tasks.workunit.client.0.smithi153.stdout:2/478: write dd/d15/d18/d3a/d3d/f53 [1018675,96714] 0 2024-03-21T10:08:49.785 INFO:tasks.workunit.client.0.smithi153.stdout:3/384: read - dd/d1e/d45/f7a zero size 2024-03-21T10:08:49.786 INFO:tasks.workunit.client.0.smithi153.stdout:4/436: read - d11/f39 zero size 2024-03-21T10:08:49.787 INFO:tasks.workunit.client.1.smithi174.stdout:7/181: symlink d0/l3f 0 2024-03-21T10:08:49.788 INFO:tasks.workunit.client.0.smithi153.stdout:2/479: stat dd/d15/d18/d2c/d4b/d56/d63 0 2024-03-21T10:08:49.788 INFO:tasks.workunit.client.0.smithi153.stdout:8/433: rename d5/d3b to d5/d9/d43/dd/d20/d26/d8b 0 2024-03-21T10:08:49.789 INFO:tasks.workunit.client.1.smithi174.stdout:7/182: readlink d0/d4/d38/l3d 0 2024-03-21T10:08:49.789 INFO:tasks.workunit.client.1.smithi174.stdout:5/228: dread d1/d16/f1a [0,4194304] 0 2024-03-21T10:08:49.789 INFO:tasks.workunit.client.0.smithi153.stdout:4/437: write f7 [4043778,119657] 0 2024-03-21T10:08:49.790 INFO:tasks.workunit.client.0.smithi153.stdout:5/476: mknod d4/d5/df/c97 0 2024-03-21T10:08:49.792 INFO:tasks.workunit.client.1.smithi174.stdout:2/269: mkdir d5/d25/d27/d2e/d4f/d65 0 2024-03-21T10:08:49.792 INFO:tasks.workunit.client.1.smithi174.stdout:3/201: write d0/f7 [1178459,2867] 0 2024-03-21T10:08:49.794 INFO:tasks.workunit.client.1.smithi174.stdout:3/202: chown d0/fb 94 1 2024-03-21T10:08:49.795 INFO:tasks.workunit.client.0.smithi153.stdout:1/360: creat d6/d13/f71 x:0 0 0 2024-03-21T10:08:49.802 INFO:tasks.workunit.client.0.smithi153.stdout:4/438: creat d11/d18/f82 x:0 0 0 2024-03-21T10:08:49.803 INFO:tasks.workunit.client.0.smithi153.stdout:1/361: fdatasync d6/d3a/d66/f68 0 2024-03-21T10:08:49.804 INFO:tasks.workunit.client.0.smithi153.stdout:5/477: read d4/d8/d3b/f88 [7074390,89727] 0 2024-03-21T10:08:49.804 INFO:tasks.workunit.client.1.smithi174.stdout:8/188: truncate d9/f12 1966127 0 2024-03-21T10:08:49.805 INFO:tasks.workunit.client.1.smithi174.stdout:8/189: dread - d9/fe zero size 2024-03-21T10:08:49.807 INFO:tasks.workunit.client.1.smithi174.stdout:2/270: mkdir d5/d25/d27/d2d/d45/d5b/d2f/d66 0 2024-03-21T10:08:49.810 INFO:tasks.workunit.client.1.smithi174.stdout:5/229: dread d1/f22 [0,4194304] 0 2024-03-21T10:08:49.810 INFO:tasks.workunit.client.0.smithi153.stdout:9/426: dwrite d1/d43/d44/d46/d1c/d24/f38 [0,4194304] 0 2024-03-21T10:08:49.813 INFO:tasks.workunit.client.0.smithi153.stdout:2/480: link dd/d15/d18/d2c/d4b/d56/d63/d8d/d8e/l9a dd/d3b/d50/d58/la0 0 2024-03-21T10:08:49.814 INFO:tasks.workunit.client.1.smithi174.stdout:5/230: write d1/d16/d2e/d31/f36 [235614,120645] 0 2024-03-21T10:08:49.814 INFO:tasks.workunit.client.0.smithi153.stdout:4/439: symlink d11/d18/d6d/l83 0 2024-03-21T10:08:49.819 INFO:tasks.workunit.client.0.smithi153.stdout:8/434: creat d5/d9/d43/f8c x:0 0 0 2024-03-21T10:08:49.820 INFO:tasks.workunit.client.1.smithi174.stdout:5/231: chown d1/d16/d2e/d31/f48 46865875 1 2024-03-21T10:08:49.820 INFO:tasks.workunit.client.1.smithi174.stdout:3/203: dwrite d0/d12/d14/d1a/f1e [0,4194304] 0 2024-03-21T10:08:49.823 INFO:tasks.workunit.client.0.smithi153.stdout:4/440: mknod d11/d5e/c84 0 2024-03-21T10:08:49.823 INFO:tasks.workunit.client.0.smithi153.stdout:1/362: creat d6/d13/f72 x:0 0 0 2024-03-21T10:08:49.827 INFO:tasks.workunit.client.0.smithi153.stdout:3/385: dwrite dd/f48 [0,4194304] 0 2024-03-21T10:08:49.827 INFO:tasks.workunit.client.0.smithi153.stdout:4/441: fsync d11/f63 0 2024-03-21T10:08:49.827 INFO:tasks.workunit.client.0.smithi153.stdout:8/435: dread - d5/d9/d43/dd/d20/d26/d24/d52/f72 zero size 2024-03-21T10:08:49.829 INFO:tasks.workunit.client.1.smithi174.stdout:9/295: dread d2/f3 [0,4194304] 0 2024-03-21T10:08:49.830 INFO:tasks.workunit.client.0.smithi153.stdout:3/386: dread - dd/d1e/d45/d56/f79 zero size 2024-03-21T10:08:49.831 INFO:tasks.workunit.client.1.smithi174.stdout:2/271: link d5/cd d5/d25/d27/d2d/d45/d5b/d2f/d66/c67 0 2024-03-21T10:08:49.832 INFO:tasks.workunit.client.1.smithi174.stdout:9/296: readlink d2/d24/d2e/l5a 0 2024-03-21T10:08:49.833 INFO:tasks.workunit.client.0.smithi153.stdout:3/387: creat dd/d1e/d2f/f8c x:0 0 0 2024-03-21T10:08:49.834 INFO:tasks.workunit.client.0.smithi153.stdout:1/363: creat d6/d10/f73 x:0 0 0 2024-03-21T10:08:49.835 INFO:tasks.workunit.client.0.smithi153.stdout:3/388: chown dd/d1e/l59 0 1 2024-03-21T10:08:49.836 INFO:tasks.workunit.client.1.smithi174.stdout:9/297: dread - d2/f45 zero size 2024-03-21T10:08:49.836 INFO:tasks.workunit.client.1.smithi174.stdout:8/190: dread d9/d17/d19/f23 [0,4194304] 0 2024-03-21T10:08:49.838 INFO:tasks.workunit.client.1.smithi174.stdout:3/204: symlink d0/d12/d14/d10/l4a 0 2024-03-21T10:08:49.838 INFO:tasks.workunit.client.1.smithi174.stdout:8/191: fdatasync d9/fe 0 2024-03-21T10:08:49.839 INFO:tasks.workunit.client.1.smithi174.stdout:8/192: fsync d9/d17/f21 0 2024-03-21T10:08:49.839 INFO:tasks.workunit.client.0.smithi153.stdout:1/364: readlink d6/d3a/d3c/d50/l59 0 2024-03-21T10:08:49.839 INFO:tasks.workunit.client.0.smithi153.stdout:8/436: getdents d5/d9 0 2024-03-21T10:08:49.839 INFO:tasks.workunit.client.0.smithi153.stdout:8/437: fdatasync d5/d9/f4b 0 2024-03-21T10:08:49.840 INFO:tasks.workunit.client.0.smithi153.stdout:3/389: symlink dd/d41/d80/l8d 0 2024-03-21T10:08:49.841 INFO:tasks.workunit.client.0.smithi153.stdout:1/365: creat d6/d32/f74 x:0 0 0 2024-03-21T10:08:49.843 INFO:tasks.workunit.client.1.smithi174.stdout:9/298: write d2/f3 [1851060,109634] 0 2024-03-21T10:08:49.843 INFO:tasks.workunit.client.1.smithi174.stdout:8/193: fdatasync d9/d17/f3c 0 2024-03-21T10:08:49.843 INFO:tasks.workunit.client.1.smithi174.stdout:3/205: write d0/d12/f49 [432262,85914] 0 2024-03-21T10:08:49.845 INFO:tasks.workunit.client.0.smithi153.stdout:1/366: write d6/f7 [831559,106423] 0 2024-03-21T10:08:49.847 INFO:tasks.workunit.client.1.smithi174.stdout:8/194: write d9/f30 [566701,112142] 0 2024-03-21T10:08:49.849 INFO:tasks.workunit.client.0.smithi153.stdout:9/427: dwrite d1/d43/d44/d46/d1c/d61/f87 [0,4194304] 0 2024-03-21T10:08:49.849 INFO:tasks.workunit.client.1.smithi174.stdout:9/299: symlink d2/d24/d2e/d63/l68 0 2024-03-21T10:08:49.850 INFO:tasks.workunit.client.1.smithi174.stdout:8/195: write d9/f30 [36165,41139] 0 2024-03-21T10:08:49.852 INFO:tasks.workunit.client.1.smithi174.stdout:9/300: chown d2/f4 12338721 1 2024-03-21T10:08:49.857 INFO:tasks.workunit.client.1.smithi174.stdout:2/272: dwrite d5/f14 [0,4194304] 0 2024-03-21T10:08:49.862 INFO:tasks.workunit.client.0.smithi153.stdout:9/428: getdents d1/d43/d44/d46/d4d/d88 0 2024-03-21T10:08:49.867 INFO:tasks.workunit.client.1.smithi174.stdout:7/183: write d0/d4/d10/f1c [1239292,130557] 0 2024-03-21T10:08:49.868 INFO:tasks.workunit.client.0.smithi153.stdout:4/442: dread d11/d2d/f36 [0,4194304] 0 2024-03-21T10:08:49.873 INFO:tasks.workunit.client.0.smithi153.stdout:1/367: write f4 [1098845,108877] 0 2024-03-21T10:08:49.873 INFO:tasks.workunit.client.1.smithi174.stdout:7/184: truncate d0/d4/f23 1084998 0 2024-03-21T10:08:49.874 INFO:tasks.workunit.client.0.smithi153.stdout:4/443: getdents d11/d2d/d6b 0 2024-03-21T10:08:49.877 INFO:tasks.workunit.client.0.smithi153.stdout:4/444: mkdir d11/d2d/d57/d85 0 2024-03-21T10:08:49.880 INFO:tasks.workunit.client.1.smithi174.stdout:7/185: getdents d0/d4/d38 0 2024-03-21T10:08:49.884 INFO:tasks.workunit.client.0.smithi153.stdout:8/438: truncate d5/d9/d43/dd/d20/d26/d24/d52/f64 524855 0 2024-03-21T10:08:49.888 INFO:tasks.workunit.client.0.smithi153.stdout:8/439: write d5/d1b/f5e [4481893,35546] 0 2024-03-21T10:08:49.889 INFO:tasks.workunit.client.0.smithi153.stdout:2/481: dread dd/d15/d18/d3a/f3e [0,4194304] 0 2024-03-21T10:08:49.889 INFO:tasks.workunit.client.1.smithi174.stdout:3/206: dread d0/d12/f2d [0,4194304] 0 2024-03-21T10:08:49.891 INFO:tasks.workunit.client.1.smithi174.stdout:3/207: chown d0/d12/f49 64 1 2024-03-21T10:08:49.894 INFO:tasks.workunit.client.0.smithi153.stdout:8/440: chown d5/d9/d43/dd/d20/d26/d24/l27 3971709 1 2024-03-21T10:08:49.894 INFO:tasks.workunit.client.0.smithi153.stdout:2/482: symlink dd/d3b/d50/la1 0 2024-03-21T10:08:49.895 INFO:tasks.workunit.client.0.smithi153.stdout:4/445: dwrite d11/d2d/d3d/f45 [0,4194304] 0 2024-03-21T10:08:49.896 INFO:tasks.workunit.client.0.smithi153.stdout:9/429: write d1/d43/d44/d46/f19 [2570726,82287] 0 2024-03-21T10:08:49.900 INFO:tasks.workunit.client.1.smithi174.stdout:9/301: write d2/f1e [1280847,115680] 0 2024-03-21T10:08:49.901 INFO:tasks.workunit.client.0.smithi153.stdout:8/441: creat d5/d9/f8d x:0 0 0 2024-03-21T10:08:49.902 INFO:tasks.workunit.client.0.smithi153.stdout:9/430: write d1/d43/d44/d46/d1c/d24/f38 [3468752,103837] 0 2024-03-21T10:08:49.904 INFO:tasks.workunit.client.1.smithi174.stdout:2/273: truncate d5/d25/d27/d2d/d45/d5b/f29 3451834 0 2024-03-21T10:08:49.904 INFO:tasks.workunit.client.1.smithi174.stdout:7/186: truncate d0/fa 4258447 0 2024-03-21T10:08:49.904 INFO:tasks.workunit.client.1.smithi174.stdout:3/208: write d0/d12/d14/d1a/f34 [2249445,90296] 0 2024-03-21T10:08:49.905 INFO:tasks.workunit.client.1.smithi174.stdout:7/187: stat d0/d4/f36 0 2024-03-21T10:08:49.906 INFO:tasks.workunit.client.1.smithi174.stdout:1/256: sync 2024-03-21T10:08:49.907 INFO:tasks.workunit.client.1.smithi174.stdout:0/237: sync 2024-03-21T10:08:49.907 INFO:tasks.workunit.client.1.smithi174.stdout:4/274: sync 2024-03-21T10:08:49.907 INFO:tasks.workunit.client.1.smithi174.stdout:6/190: sync 2024-03-21T10:08:49.908 INFO:tasks.workunit.client.0.smithi153.stdout:9/431: write d1/d43/f4f [347245,107195] 0 2024-03-21T10:08:49.908 INFO:tasks.workunit.client.1.smithi174.stdout:5/232: dwrite d1/d16/f40 [0,4194304] 0 2024-03-21T10:08:49.909 INFO:tasks.workunit.client.0.smithi153.stdout:4/446: rmdir d11/d18/d7e/d7f 0 2024-03-21T10:08:49.911 INFO:tasks.workunit.client.1.smithi174.stdout:7/188: symlink d0/d4/l40 0 2024-03-21T10:08:49.911 INFO:tasks.workunit.client.1.smithi174.stdout:6/191: write d0/d3/d5/df/d16/f1a [5061665,56526] 0 2024-03-21T10:08:49.912 INFO:tasks.workunit.client.0.smithi153.stdout:9/432: write d1/d4e/f53 [391583,100084] 0 2024-03-21T10:08:49.913 INFO:tasks.workunit.client.1.smithi174.stdout:4/275: unlink d4/fe 0 2024-03-21T10:08:49.916 INFO:tasks.workunit.client.0.smithi153.stdout:9/433: write d1/d43/d44/d46/d1c/d2e/f51 [3373372,88841] 0 2024-03-21T10:08:49.917 INFO:tasks.workunit.client.0.smithi153.stdout:2/483: dwrite dd/d15/d18/d2c/f4e [0,4194304] 0 2024-03-21T10:08:49.917 INFO:tasks.workunit.client.1.smithi174.stdout:0/238: write d9/d21/d31/d3c/f49 [829479,81275] 0 2024-03-21T10:08:49.917 INFO:tasks.workunit.client.1.smithi174.stdout:7/189: write d0/d4/d10/f1c [3380668,42765] 0 2024-03-21T10:08:49.918 INFO:tasks.workunit.client.0.smithi153.stdout:8/442: rename d5/d9/d43/l3e to d5/d9/d43/dd/d20/d26/d24/d52/d56/l8e 0 2024-03-21T10:08:49.918 INFO:tasks.workunit.client.0.smithi153.stdout:1/368: dread f3 [0,4194304] 0 2024-03-21T10:08:49.919 INFO:tasks.workunit.client.1.smithi174.stdout:7/190: chown d0/l3f 4695 1 2024-03-21T10:08:49.920 INFO:tasks.workunit.client.1.smithi174.stdout:0/239: read - d9/d21/d23/f41 zero size 2024-03-21T10:08:49.921 INFO:tasks.workunit.client.0.smithi153.stdout:4/447: truncate d11/d18/f21 99867 0 2024-03-21T10:08:49.925 INFO:tasks.workunit.client.0.smithi153.stdout:9/434: write d1/d43/d44/d46/d1c/d2e/f51 [3852018,56005] 0 2024-03-21T10:08:49.926 INFO:tasks.workunit.client.0.smithi153.stdout:2/484: symlink dd/d15/d18/d2c/d4b/d56/la2 0 2024-03-21T10:08:49.926 INFO:tasks.workunit.client.1.smithi174.stdout:0/240: write d9/fc [1315895,100406] 0 2024-03-21T10:08:49.926 INFO:tasks.workunit.client.1.smithi174.stdout:4/276: symlink d4/d21/d24/d28/l5d 0 2024-03-21T10:08:49.926 INFO:tasks.workunit.client.1.smithi174.stdout:5/233: link d1/d16/c1f d1/d16/d2e/c4e 0 2024-03-21T10:08:49.931 INFO:tasks.workunit.client.1.smithi174.stdout:4/277: dread - d4/d21/f37 zero size 2024-03-21T10:08:49.937 INFO:tasks.workunit.client.0.smithi153.stdout:7/347: sync 2024-03-21T10:08:49.937 INFO:tasks.workunit.client.0.smithi153.stdout:6/623: sync 2024-03-21T10:08:49.937 INFO:tasks.workunit.client.0.smithi153.stdout:0/361: sync 2024-03-21T10:08:49.938 INFO:tasks.workunit.client.1.smithi174.stdout:2/274: dwrite d5/d25/d27/d2d/d45/f62 [0,4194304] 0 2024-03-21T10:08:49.941 INFO:tasks.workunit.client.0.smithi153.stdout:9/435: unlink d1/d43/d44/d46/d1c/d2e/d67/c8e 0 2024-03-21T10:08:49.941 INFO:tasks.workunit.client.0.smithi153.stdout:1/369: rename d6/d10/d2c/d54 to d6/d13/d75 0 2024-03-21T10:08:49.942 INFO:tasks.workunit.client.0.smithi153.stdout:6/624: dread - d1f/d2b/db3/fcc zero size 2024-03-21T10:08:49.943 INFO:tasks.workunit.client.0.smithi153.stdout:9/436: dread - d1/d4/f75 zero size 2024-03-21T10:08:49.943 INFO:tasks.workunit.client.0.smithi153.stdout:3/390: dread dd/d1e/d2f/f37 [0,4194304] 0 2024-03-21T10:08:49.946 INFO:tasks.workunit.client.1.smithi174.stdout:3/209: dwrite d0/d12/d14/d1a/f34 [0,4194304] 0 2024-03-21T10:08:49.946 INFO:tasks.workunit.client.0.smithi153.stdout:2/485: link dd/d15/d3c/c49 dd/d15/d18/d2c/d4b/d56/d63/d8d/d8e/ca3 0 2024-03-21T10:08:49.946 INFO:tasks.workunit.client.0.smithi153.stdout:8/443: link d5/d9/d43/dd/d20/d26/d47/d69/f86 d5/d1b/f8f 0 2024-03-21T10:08:49.947 INFO:tasks.workunit.client.0.smithi153.stdout:0/362: creat d4/d16/d3e/d5e/f71 x:0 0 0 2024-03-21T10:08:49.947 INFO:tasks.workunit.client.0.smithi153.stdout:3/391: dread - dd/d1e/d45/f7a zero size 2024-03-21T10:08:49.948 INFO:tasks.workunit.client.0.smithi153.stdout:7/348: rename d4/d11/l17 to d4/d19/l76 0 2024-03-21T10:08:49.949 INFO:tasks.workunit.client.0.smithi153.stdout:7/349: readlink d4/lf 0 2024-03-21T10:08:49.951 INFO:tasks.workunit.client.0.smithi153.stdout:6/625: creat d1f/d93/fdd x:0 0 0 2024-03-21T10:08:49.957 INFO:tasks.workunit.client.0.smithi153.stdout:9/437: symlink d1/d43/d44/d46/d4d/d88/l96 0 2024-03-21T10:08:49.957 INFO:tasks.workunit.client.0.smithi153.stdout:0/363: fdatasync d4/d16/d1f/d22/f47 0 2024-03-21T10:08:49.964 INFO:tasks.workunit.client.1.smithi174.stdout:8/196: dwrite d9/f1c [0,4194304] 0 2024-03-21T10:08:49.969 INFO:tasks.workunit.client.0.smithi153.stdout:0/364: symlink d4/db/l72 0 2024-03-21T10:08:49.971 INFO:tasks.workunit.client.0.smithi153.stdout:8/444: dwrite d5/d1b/f59 [0,4194304] 0 2024-03-21T10:08:49.971 INFO:tasks.workunit.client.0.smithi153.stdout:6/626: creat d1f/d2b/d33/d40/d65/fde x:0 0 0 2024-03-21T10:08:49.974 INFO:tasks.workunit.client.0.smithi153.stdout:0/365: write d4/d16/d1f/d22/f47 [3623912,113071] 0 2024-03-21T10:08:49.976 INFO:tasks.workunit.client.0.smithi153.stdout:0/366: fsync d4/f62 0 2024-03-21T10:08:49.977 INFO:tasks.workunit.client.1.smithi174.stdout:2/275: dread d5/d25/d27/d2d/d45/d5b/d55/f58 [0,4194304] 0 2024-03-21T10:08:49.977 INFO:tasks.workunit.client.0.smithi153.stdout:6/627: mkdir d1f/d93/ddf 0 2024-03-21T10:08:49.977 INFO:tasks.workunit.client.0.smithi153.stdout:8/445: write d5/d9/d43/dd/d2d/d45/d88/f73 [1325599,93748] 0 2024-03-21T10:08:49.979 INFO:tasks.workunit.client.1.smithi174.stdout:3/210: dread d0/d12/f49 [0,4194304] 0 2024-03-21T10:08:49.981 INFO:tasks.workunit.client.0.smithi153.stdout:9/438: creat d1/f97 x:0 0 0 2024-03-21T10:08:49.987 INFO:tasks.workunit.client.0.smithi153.stdout:0/367: truncate d4/d16/d1f/d22/f2e 4585849 0 2024-03-21T10:08:49.991 INFO:tasks.workunit.client.0.smithi153.stdout:6/628: dread d1f/d2b/d33/d40/d4f/d66/fd0 [0,4194304] 0 2024-03-21T10:08:49.993 INFO:tasks.workunit.client.0.smithi153.stdout:3/392: dwrite dd/f43 [4194304,4194304] 0 2024-03-21T10:08:49.994 INFO:tasks.workunit.client.0.smithi153.stdout:0/368: chown d4/d16/d27/f4d 0 1 2024-03-21T10:08:49.996 INFO:tasks.workunit.client.0.smithi153.stdout:0/369: fsync d4/db/f13 0 2024-03-21T10:08:49.999 INFO:tasks.workunit.client.0.smithi153.stdout:0/370: fdatasync d4/d16/d27/f5c 0 2024-03-21T10:08:50.002 INFO:tasks.workunit.client.0.smithi153.stdout:8/446: creat d5/d9/d43/dd/d20/d26/d24/d52/d55/f90 x:0 0 0 2024-03-21T10:08:50.003 INFO:tasks.workunit.client.0.smithi153.stdout:2/486: truncate dd/d15/d18/d3a/f55 1735678 0 2024-03-21T10:08:50.006 INFO:tasks.workunit.client.0.smithi153.stdout:4/448: write d11/d18/f21 [332693,10324] 0 2024-03-21T10:08:50.006 INFO:tasks.workunit.client.0.smithi153.stdout:3/393: mkdir dd/d7e/d8e 0 2024-03-21T10:08:50.008 INFO:tasks.workunit.client.0.smithi153.stdout:7/350: dread d4/f10 [0,4194304] 0 2024-03-21T10:08:50.009 INFO:tasks.workunit.client.0.smithi153.stdout:4/449: chown d11/d18/d6d 6 1 2024-03-21T10:08:50.009 INFO:tasks.workunit.client.1.smithi174.stdout:3/211: dread d0/d12/d14/d10/f1f [0,4194304] 0 2024-03-21T10:08:50.011 INFO:tasks.workunit.client.0.smithi153.stdout:9/439: dwrite d1/d3b/f86 [0,4194304] 0 2024-03-21T10:08:50.011 INFO:tasks.workunit.client.0.smithi153.stdout:6/629: mkdir d1f/de0 0 2024-03-21T10:08:50.012 INFO:tasks.workunit.client.0.smithi153.stdout:7/351: dread - d4/d11/d24/f54 zero size 2024-03-21T10:08:50.016 INFO:tasks.workunit.client.0.smithi153.stdout:6/630: fdatasync d1f/d2b/f89 0 2024-03-21T10:08:50.016 INFO:tasks.workunit.client.0.smithi153.stdout:8/447: write d5/d9/d43/dd/d2d/d45/d88/f3c [357581,11103] 0 2024-03-21T10:08:50.023 INFO:tasks.workunit.client.0.smithi153.stdout:7/352: dread d4/f10 [0,4194304] 0 2024-03-21T10:08:50.026 INFO:tasks.workunit.client.0.smithi153.stdout:0/371: dwrite d4/db/f13 [0,4194304] 0 2024-03-21T10:08:50.031 INFO:tasks.workunit.client.0.smithi153.stdout:9/440: creat d1/d4e/f98 x:0 0 0 2024-03-21T10:08:50.032 INFO:tasks.workunit.client.0.smithi153.stdout:3/394: getdents dd/d7e/d73/d7f 0 2024-03-21T10:08:50.034 INFO:tasks.workunit.client.0.smithi153.stdout:6/631: mkdir d1f/de0/de1 0 2024-03-21T10:08:50.036 INFO:tasks.workunit.client.0.smithi153.stdout:8/448: write d5/d9/d43/dd/d20/f7d [5005947,39460] 0 2024-03-21T10:08:50.038 INFO:tasks.workunit.client.0.smithi153.stdout:8/449: chown d5/d1b 3 1 2024-03-21T10:08:50.040 INFO:tasks.workunit.client.0.smithi153.stdout:9/441: chown d1/d43/d44/d46/d1c/d2e/f51 867762 1 2024-03-21T10:08:50.045 INFO:tasks.workunit.client.0.smithi153.stdout:9/442: truncate d1/f97 556958 0 2024-03-21T10:08:50.046 INFO:tasks.workunit.client.0.smithi153.stdout:2/487: dwrite dd/d15/d18/f2a [0,4194304] 0 2024-03-21T10:08:50.046 INFO:tasks.workunit.client.0.smithi153.stdout:3/395: rename dd/d41/l70 to dd/d1e/d2f/d35/d68/d75/l8f 0 2024-03-21T10:08:50.051 INFO:tasks.workunit.client.0.smithi153.stdout:4/450: dwrite d11/d2d/f30 [0,4194304] 0 2024-03-21T10:08:50.052 INFO:tasks.workunit.client.0.smithi153.stdout:8/450: creat d5/d9/d43/d7b/d7c/f91 x:0 0 0 2024-03-21T10:08:50.052 INFO:tasks.workunit.client.0.smithi153.stdout:2/488: stat dd/d15/d18/d2c/d4b/d56/d63/d8d/d8e 0 2024-03-21T10:08:50.056 INFO:tasks.workunit.client.0.smithi153.stdout:4/451: stat d11/d2d/c75 0 2024-03-21T10:08:50.063 INFO:tasks.workunit.client.0.smithi153.stdout:8/451: read - d5/d9/d43/dd/d20/d26/d24/d52/d55/f90 zero size 2024-03-21T10:08:50.066 INFO:tasks.workunit.client.1.smithi174.stdout:4/278: rmdir d4/d21/d24/d28 39 2024-03-21T10:08:50.067 INFO:tasks.workunit.client.0.smithi153.stdout:6/632: link d1f/d2b/d33/d40/d4f/d66/lc0 d1f/de0/le2 0 2024-03-21T10:08:50.069 INFO:tasks.workunit.client.0.smithi153.stdout:2/489: fdatasync dd/d15/d18/d2c/d33/f9d 0 2024-03-21T10:08:50.070 INFO:tasks.workunit.client.1.smithi174.stdout:2/276: mkdir d5/d25/d27/d2d/d45/d5b/d2f/d68 0 2024-03-21T10:08:50.071 INFO:tasks.workunit.client.0.smithi153.stdout:4/452: symlink d11/d18/d7e/l86 0 2024-03-21T10:08:50.074 INFO:tasks.workunit.client.0.smithi153.stdout:3/396: readlink dd/d1e/d2f/d35/d68/d75/l8f 0 2024-03-21T10:08:50.074 INFO:tasks.workunit.client.1.smithi174.stdout:8/197: symlink d9/d2a/d39/l46 0 2024-03-21T10:08:50.074 INFO:tasks.workunit.client.1.smithi174.stdout:0/241: truncate f7 1682366 0 2024-03-21T10:08:50.075 INFO:tasks.workunit.client.0.smithi153.stdout:4/453: write d11/f44 [2821316,67262] 0 2024-03-21T10:08:50.075 INFO:tasks.workunit.client.1.smithi174.stdout:3/212: creat d0/d12/d46/f4b x:0 0 0 2024-03-21T10:08:50.077 INFO:tasks.workunit.client.1.smithi174.stdout:8/198: chown d9/d17/f42 3455 1 2024-03-21T10:08:50.078 INFO:tasks.workunit.client.1.smithi174.stdout:4/279: rename d4/ff to d4/d21/d24/d28/f5e 0 2024-03-21T10:08:50.079 INFO:tasks.workunit.client.0.smithi153.stdout:7/353: dread d4/d11/d24/f58 [0,4194304] 0 2024-03-21T10:08:50.081 INFO:tasks.workunit.client.0.smithi153.stdout:4/454: write d11/f44 [3918327,115564] 0 2024-03-21T10:08:50.082 INFO:tasks.workunit.client.1.smithi174.stdout:3/213: unlink d0/d12/d14/d1a/l2b 0 2024-03-21T10:08:50.084 INFO:tasks.workunit.client.1.smithi174.stdout:2/277: link d5/d25/d27/d2d/d45/d5b/f1c d5/d25/d27/d2d/d45/d5b/d55/f69 0 2024-03-21T10:08:50.084 INFO:tasks.workunit.client.1.smithi174.stdout:5/234: dwrite d1/f6 [0,4194304] 0 2024-03-21T10:08:50.085 INFO:tasks.workunit.client.0.smithi153.stdout:7/354: creat d4/d2a/f77 x:0 0 0 2024-03-21T10:08:50.086 INFO:tasks.workunit.client.0.smithi153.stdout:2/490: creat dd/d15/d5e/fa4 x:0 0 0 2024-03-21T10:08:50.087 INFO:tasks.workunit.client.0.smithi153.stdout:0/372: dwrite d4/db/f1b [4194304,4194304] 0 2024-03-21T10:08:50.088 INFO:tasks.workunit.client.0.smithi153.stdout:7/355: readlink d4/l52 0 2024-03-21T10:08:50.088 INFO:tasks.workunit.client.1.smithi174.stdout:3/214: unlink d0/d12/l18 0 2024-03-21T10:08:50.091 INFO:tasks.workunit.client.0.smithi153.stdout:5/478: sync 2024-03-21T10:08:50.091 INFO:tasks.workunit.client.0.smithi153.stdout:1/370: sync 2024-03-21T10:08:50.092 INFO:tasks.workunit.client.0.smithi153.stdout:4/455: readlink d11/d18/l4d 0 2024-03-21T10:08:50.092 INFO:tasks.workunit.client.0.smithi153.stdout:0/373: stat d4/d16/d3e/c66 0 2024-03-21T10:08:50.094 INFO:tasks.workunit.client.1.smithi174.stdout:0/242: link d9/d1c/l22 d9/d21/d31/d3c/l4d 0 2024-03-21T10:08:50.094 INFO:tasks.workunit.client.1.smithi174.stdout:8/199: rmdir d9/d37/d44 0 2024-03-21T10:08:50.095 INFO:tasks.workunit.client.0.smithi153.stdout:8/452: link d5/d9/l65 d5/d9/d43/dd/d20/d26/d8b/d80/l92 0 2024-03-21T10:08:50.095 INFO:tasks.workunit.client.1.smithi174.stdout:8/200: write d9/fe [660509,67793] 0 2024-03-21T10:08:50.096 INFO:tasks.workunit.client.0.smithi153.stdout:5/479: write d4/d5/d16/d24/f8c [1174850,27845] 0 2024-03-21T10:08:50.096 INFO:tasks.workunit.client.0.smithi153.stdout:4/456: stat d11/d2d/d3d/f45 0 2024-03-21T10:08:50.096 INFO:tasks.workunit.client.1.smithi174.stdout:0/243: write d9/d21/d23/f41 [359979,124988] 0 2024-03-21T10:08:50.097 INFO:tasks.workunit.client.0.smithi153.stdout:1/371: write d6/f9 [2211492,108636] 0 2024-03-21T10:08:50.100 INFO:tasks.workunit.client.0.smithi153.stdout:2/491: creat dd/d15/d18/d2c/d4b/d56/d63/d8d/d9b/fa5 x:0 0 0 2024-03-21T10:08:50.105 INFO:tasks.workunit.client.1.smithi174.stdout:8/201: dread d9/d2a/f34 [0,4194304] 0 2024-03-21T10:08:50.105 INFO:tasks.workunit.client.1.smithi174.stdout:8/202: readlink d9/lf 0 2024-03-21T10:08:50.106 INFO:tasks.workunit.client.0.smithi153.stdout:7/356: rename d4/d11/d24 to d4/d14/d2f/d78 0 2024-03-21T10:08:50.110 INFO:tasks.workunit.client.0.smithi153.stdout:6/633: write d1f/d2b/d33/d40/d51/f71 [191595,81444] 0 2024-03-21T10:08:50.113 INFO:tasks.workunit.client.0.smithi153.stdout:8/453: creat d5/d1b/f93 x:0 0 0 2024-03-21T10:08:50.114 INFO:tasks.workunit.client.1.smithi174.stdout:5/235: dwrite d1/d16/d1c/f26 [0,4194304] 0 2024-03-21T10:08:50.119 INFO:tasks.workunit.client.0.smithi153.stdout:3/397: dwrite dd/f22 [0,4194304] 0 2024-03-21T10:08:50.120 INFO:tasks.workunit.client.1.smithi174.stdout:3/215: dwrite d0/d12/d14/d10/f38 [0,4194304] 0 2024-03-21T10:08:50.122 INFO:tasks.workunit.client.1.smithi174.stdout:4/280: getdents d4 0 2024-03-21T10:08:50.124 INFO:tasks.workunit.client.0.smithi153.stdout:4/457: dwrite d11/f16 [0,4194304] 0 2024-03-21T10:08:50.124 INFO:tasks.workunit.client.0.smithi153.stdout:9/443: dread d1/d43/d44/f33 [0,4194304] 0 2024-03-21T10:08:50.125 INFO:tasks.workunit.client.0.smithi153.stdout:3/398: readlink dd/d1e/d2f/l42 0 2024-03-21T10:08:50.125 INFO:tasks.workunit.client.1.smithi174.stdout:5/236: mkdir d1/d49/d4f 0 2024-03-21T10:08:50.126 INFO:tasks.workunit.client.0.smithi153.stdout:0/374: rename d4/d16/d27/f38 to d4/d16/d1f/d22/f73 0 2024-03-21T10:08:50.126 INFO:tasks.workunit.client.1.smithi174.stdout:0/244: dwrite d9/d21/d23/f2d [0,4194304] 0 2024-03-21T10:08:50.129 INFO:tasks.workunit.client.0.smithi153.stdout:3/399: fdatasync dd/d1e/d45/d56/d58/f8b 0 2024-03-21T10:08:50.130 INFO:tasks.workunit.client.0.smithi153.stdout:2/492: dread - dd/d15/d18/d2c/d33/f57 zero size 2024-03-21T10:08:50.134 INFO:tasks.workunit.client.0.smithi153.stdout:4/458: rmdir d11/d18/d54 39 2024-03-21T10:08:50.135 INFO:tasks.workunit.client.1.smithi174.stdout:3/216: mkdir d0/d12/d14/d4c 0 2024-03-21T10:08:50.136 INFO:tasks.workunit.client.0.smithi153.stdout:8/454: stat d5/d9/d43/dd/d2d/d45/f70 0 2024-03-21T10:08:50.148 INFO:tasks.workunit.client.1.smithi174.stdout:4/281: dwrite d4/d21/d31/f48 [0,4194304] 0 2024-03-21T10:08:50.150 INFO:tasks.workunit.client.1.smithi174.stdout:5/237: link d1/f22 d1/d16/f50 0 2024-03-21T10:08:50.151 INFO:tasks.workunit.client.0.smithi153.stdout:4/459: dread - d11/f39 zero size 2024-03-21T10:08:50.151 INFO:tasks.workunit.client.0.smithi153.stdout:0/375: chown d4/d16/d1f/f24 7 1 2024-03-21T10:08:50.152 INFO:tasks.workunit.client.1.smithi174.stdout:5/238: dread - d1/d16/f24 zero size 2024-03-21T10:08:50.154 INFO:tasks.workunit.client.1.smithi174.stdout:4/282: fsync d4/d19/f22 0 2024-03-21T10:08:50.156 INFO:tasks.workunit.client.1.smithi174.stdout:0/245: dwrite d9/d21/d23/f41 [0,4194304] 0 2024-03-21T10:08:50.160 INFO:tasks.workunit.client.1.smithi174.stdout:5/239: mkdir d1/d49/d4f/d51 0 2024-03-21T10:08:50.160 INFO:tasks.workunit.client.1.smithi174.stdout:0/246: chown d9/f13 20675244 1 2024-03-21T10:08:50.162 INFO:tasks.workunit.client.0.smithi153.stdout:6/634: dwrite d1f/f25 [8388608,4194304] 0 2024-03-21T10:08:50.165 INFO:tasks.workunit.client.1.smithi174.stdout:8/203: truncate f7 1116164 0 2024-03-21T10:08:50.166 INFO:tasks.workunit.client.1.smithi174.stdout:0/247: mknod d9/d1c/c4e 0 2024-03-21T10:08:50.167 INFO:tasks.workunit.client.0.smithi153.stdout:1/372: dwrite d6/d3a/d3c/f64 [0,4194304] 0 2024-03-21T10:08:50.167 INFO:tasks.workunit.client.0.smithi153.stdout:6/635: chown c6 2 1 2024-03-21T10:08:50.168 INFO:tasks.workunit.client.0.smithi153.stdout:1/373: fdatasync d6/d13/f71 0 2024-03-21T10:08:50.169 INFO:tasks.workunit.client.1.smithi174.stdout:5/240: rename d1/d16/c2d to d1/d16/d1c/c52 0 2024-03-21T10:08:50.170 INFO:tasks.workunit.client.1.smithi174.stdout:1/257: sync 2024-03-21T10:08:50.171 INFO:tasks.workunit.client.1.smithi174.stdout:9/302: sync 2024-03-21T10:08:50.171 INFO:tasks.workunit.client.1.smithi174.stdout:0/248: read d9/fc [892520,76421] 0 2024-03-21T10:08:50.173 INFO:tasks.workunit.client.1.smithi174.stdout:8/204: creat d9/d2a/d39/f47 x:0 0 0 2024-03-21T10:08:50.173 INFO:tasks.workunit.client.1.smithi174.stdout:3/217: dwrite d0/d12/d14/f3a [0,4194304] 0 2024-03-21T10:08:50.174 INFO:tasks.workunit.client.1.smithi174.stdout:4/283: getdents d4 0 2024-03-21T10:08:50.175 INFO:tasks.workunit.client.1.smithi174.stdout:0/249: fsync d9/f13 0 2024-03-21T10:08:50.176 INFO:tasks.workunit.client.0.smithi153.stdout:1/374: write d6/f17 [5149546,22314] 0 2024-03-21T10:08:50.176 INFO:tasks.workunit.client.0.smithi153.stdout:1/375: stat d6/d3a/d3c/d50 0 2024-03-21T10:08:50.177 INFO:tasks.workunit.client.1.smithi174.stdout:1/258: creat d4/d7/d33/d43/f5d x:0 0 0 2024-03-21T10:08:50.177 INFO:tasks.workunit.client.1.smithi174.stdout:3/218: fdatasync d0/d12/d14/d1a/f40 0 2024-03-21T10:08:50.179 INFO:tasks.workunit.client.1.smithi174.stdout:2/278: write d5/fa [1141065,76981] 0 2024-03-21T10:08:50.181 INFO:tasks.workunit.client.0.smithi153.stdout:2/493: dwrite dd/d15/d18/d2c/d33/f7d [0,4194304] 0 2024-03-21T10:08:50.182 INFO:tasks.workunit.client.1.smithi174.stdout:2/279: dread - d5/d25/d27/f40 zero size 2024-03-21T10:08:50.182 INFO:tasks.workunit.client.1.smithi174.stdout:9/303: link d2/d24/d2e/f58 d2/d24/d2e/f69 0 2024-03-21T10:08:50.183 INFO:tasks.workunit.client.0.smithi153.stdout:9/444: dread d1/d3b/f86 [0,4194304] 0 2024-03-21T10:08:50.183 INFO:tasks.workunit.client.1.smithi174.stdout:4/284: chown d4/d21/d24/c59 100502915 1 2024-03-21T10:08:50.184 INFO:tasks.workunit.client.1.smithi174.stdout:1/259: truncate d4/d2d/f34 1123662 0 2024-03-21T10:08:50.184 INFO:tasks.workunit.client.0.smithi153.stdout:2/494: write dd/d15/f40 [3116155,86431] 0 2024-03-21T10:08:50.188 INFO:tasks.workunit.client.1.smithi174.stdout:8/205: mknod d9/c48 0 2024-03-21T10:08:50.190 INFO:tasks.workunit.client.0.smithi153.stdout:8/455: symlink d5/d9/l94 0 2024-03-21T10:08:50.191 INFO:tasks.workunit.client.1.smithi174.stdout:8/206: chown d9/f12 18468 1 2024-03-21T10:08:50.192 INFO:tasks.workunit.client.1.smithi174.stdout:0/250: mkdir d9/d35/d4f 0 2024-03-21T10:08:50.193 INFO:tasks.workunit.client.0.smithi153.stdout:5/480: dread d4/d5/f7 [0,4194304] 0 2024-03-21T10:08:50.193 INFO:tasks.workunit.client.0.smithi153.stdout:7/357: getdents d4/d2a 0 2024-03-21T10:08:50.196 INFO:tasks.workunit.client.1.smithi174.stdout:9/304: stat d2/d24/l52 0 2024-03-21T10:08:50.196 INFO:tasks.workunit.client.0.smithi153.stdout:7/358: fdatasync d4/d2a/f2d 0 2024-03-21T10:08:50.197 INFO:tasks.workunit.client.1.smithi174.stdout:3/219: rename d0/d12/d14/c30 to d0/d12/d14/d10/c4d 0 2024-03-21T10:08:50.202 INFO:tasks.workunit.client.0.smithi153.stdout:4/460: fdatasync d11/f19 0 2024-03-21T10:08:50.203 INFO:tasks.workunit.client.1.smithi174.stdout:1/260: creat d4/d2d/f5e x:0 0 0 2024-03-21T10:08:50.204 INFO:tasks.workunit.client.1.smithi174.stdout:0/251: mknod d9/d21/c50 0 2024-03-21T10:08:50.206 INFO:tasks.workunit.client.1.smithi174.stdout:5/241: dwrite d1/d16/d1c/f34 [0,4194304] 0 2024-03-21T10:08:50.207 INFO:tasks.workunit.client.1.smithi174.stdout:0/252: fsync d9/d21/f37 0 2024-03-21T10:08:50.207 INFO:tasks.workunit.client.0.smithi153.stdout:4/461: chown d11/f39 1 1 2024-03-21T10:08:50.208 INFO:tasks.workunit.client.1.smithi174.stdout:2/280: rename d5/d25/d27/d2d/f21 to d5/d25/d27/f6a 0 2024-03-21T10:08:50.214 INFO:tasks.workunit.client.1.smithi174.stdout:5/242: creat d1/d49/f53 x:0 0 0 2024-03-21T10:08:50.217 INFO:tasks.workunit.client.0.smithi153.stdout:1/376: symlink d6/d3a/d3c/d50/l76 0 2024-03-21T10:08:50.217 INFO:tasks.workunit.client.1.smithi174.stdout:8/207: rename d9/ca to d9/d17/d19/d31/c49 0 2024-03-21T10:08:50.218 INFO:tasks.workunit.client.1.smithi174.stdout:0/253: symlink d9/d1c/l51 0 2024-03-21T10:08:50.221 INFO:tasks.workunit.client.1.smithi174.stdout:8/208: write d9/d17/d19/d27/f41 [355048,24523] 0 2024-03-21T10:08:50.222 INFO:tasks.workunit.client.1.smithi174.stdout:3/220: creat d0/d12/d14/f4e x:0 0 0 2024-03-21T10:08:50.223 INFO:tasks.workunit.client.1.smithi174.stdout:1/261: symlink d4/d7/d15/l5f 0 2024-03-21T10:08:50.224 INFO:tasks.workunit.client.1.smithi174.stdout:3/221: stat d0/d12/f49 0 2024-03-21T10:08:50.226 INFO:tasks.workunit.client.1.smithi174.stdout:2/281: unlink d5/d25/d27/d2d/d45/f62 0 2024-03-21T10:08:50.227 INFO:tasks.workunit.client.1.smithi174.stdout:2/282: dread - d5/d25/d27/d2d/d45/d5b/d63/f61 zero size 2024-03-21T10:08:50.230 INFO:tasks.workunit.client.1.smithi174.stdout:7/191: sync 2024-03-21T10:08:50.230 INFO:tasks.workunit.client.1.smithi174.stdout:6/192: sync 2024-03-21T10:08:50.231 INFO:tasks.workunit.client.0.smithi153.stdout:9/445: symlink d1/d43/d44/d46/d1c/l99 0 2024-03-21T10:08:50.231 INFO:tasks.workunit.client.0.smithi153.stdout:3/400: dwrite dd/d1e/f30 [0,4194304] 0 2024-03-21T10:08:50.232 INFO:tasks.workunit.client.1.smithi174.stdout:5/243: rmdir d1/d2f 39 2024-03-21T10:08:50.232 INFO:tasks.workunit.client.1.smithi174.stdout:4/285: getdents d4/d21/d24/d28 0 2024-03-21T10:08:50.234 INFO:tasks.workunit.client.1.smithi174.stdout:1/262: mkdir d4/d7/d33/d43/d51/d60 0 2024-03-21T10:08:50.234 INFO:tasks.workunit.client.1.smithi174.stdout:4/286: write d4/d19/f23 [3883189,100832] 0 2024-03-21T10:08:50.235 INFO:tasks.workunit.client.0.smithi153.stdout:2/495: readlink dd/d15/d18/d2c/d4b/d56/d63/d8d/d8e/l9a 0 2024-03-21T10:08:50.235 INFO:tasks.workunit.client.0.smithi153.stdout:3/401: stat dd/f43 0 2024-03-21T10:08:50.236 INFO:tasks.workunit.client.1.smithi174.stdout:3/222: mkdir d0/d12/d14/d1a/d27/d4f 0 2024-03-21T10:08:50.238 INFO:tasks.workunit.client.1.smithi174.stdout:5/244: mknod d1/d16/c54 0 2024-03-21T10:08:50.241 INFO:tasks.workunit.client.0.smithi153.stdout:8/456: symlink d5/d9/d43/dd/d20/d26/d7a/l95 0 2024-03-21T10:08:50.247 INFO:tasks.workunit.client.0.smithi153.stdout:3/402: chown dd/d1e/d45/d56/c87 170485 1 2024-03-21T10:08:50.248 INFO:tasks.workunit.client.0.smithi153.stdout:7/359: mkdir d4/d14/d16/d79 0 2024-03-21T10:08:50.252 INFO:tasks.workunit.client.1.smithi174.stdout:7/192: symlink d0/l41 0 2024-03-21T10:08:50.252 INFO:tasks.workunit.client.1.smithi174.stdout:6/193: mknod d0/d3/d5/d3b/c40 0 2024-03-21T10:08:50.257 INFO:tasks.workunit.client.1.smithi174.stdout:5/245: dwrite d1/d16/d1c/f26 [4194304,4194304] 0 2024-03-21T10:08:50.259 INFO:tasks.workunit.client.1.smithi174.stdout:6/194: write d0/d3/d5/df/d16/f27 [1438832,4473] 0 2024-03-21T10:08:50.262 INFO:tasks.workunit.client.1.smithi174.stdout:4/287: dwrite d4/d21/d31/d3a/f3d [0,4194304] 0 2024-03-21T10:08:50.263 INFO:tasks.workunit.client.0.smithi153.stdout:4/462: dwrite fd [4194304,4194304] 0 2024-03-21T10:08:50.264 INFO:tasks.workunit.client.0.smithi153.stdout:1/377: dwrite d6/f7 [0,4194304] 0 2024-03-21T10:08:50.265 INFO:tasks.workunit.client.1.smithi174.stdout:4/288: chown d4/d21/c55 0 1 2024-03-21T10:08:50.267 INFO:tasks.workunit.client.0.smithi153.stdout:0/376: creat d4/d16/d19/f74 x:0 0 0 2024-03-21T10:08:50.270 INFO:tasks.workunit.client.1.smithi174.stdout:5/246: symlink d1/d16/d1c/l55 0 2024-03-21T10:08:50.273 INFO:tasks.workunit.client.1.smithi174.stdout:7/193: dread d0/ff [0,4194304] 0 2024-03-21T10:08:50.276 INFO:tasks.workunit.client.1.smithi174.stdout:6/195: rename d0/d3/l39 to d0/d3/d2a/l41 0 2024-03-21T10:08:50.276 INFO:tasks.workunit.client.1.smithi174.stdout:0/254: rmdir d9/d35 39 2024-03-21T10:08:50.277 INFO:tasks.workunit.client.0.smithi153.stdout:8/457: symlink d5/d9/d43/dd/d20/d26/d8b/l96 0 2024-03-21T10:08:50.277 INFO:tasks.workunit.client.1.smithi174.stdout:4/289: creat d4/d21/d24/d28/d38/f5f x:0 0 0 2024-03-21T10:08:50.280 INFO:tasks.workunit.client.0.smithi153.stdout:3/403: creat dd/d1e/d2f/f90 x:0 0 0 2024-03-21T10:08:50.280 INFO:tasks.workunit.client.0.smithi153.stdout:1/378: read f4 [2570214,22579] 0 2024-03-21T10:08:50.283 INFO:tasks.workunit.client.1.smithi174.stdout:4/290: truncate d4/d21/d24/d28/f39 1658453 0 2024-03-21T10:08:50.287 INFO:tasks.workunit.client.1.smithi174.stdout:5/247: dread d1/d16/d1c/f26 [0,4194304] 0 2024-03-21T10:08:50.288 INFO:tasks.workunit.client.1.smithi174.stdout:8/209: dread f7 [0,4194304] 0 2024-03-21T10:08:50.289 INFO:tasks.workunit.client.1.smithi174.stdout:7/194: creat d0/d4/f42 x:0 0 0 2024-03-21T10:08:50.290 INFO:tasks.workunit.client.0.smithi153.stdout:6/636: dwrite d1f/fbf [0,4194304] 0 2024-03-21T10:08:50.293 INFO:tasks.workunit.client.1.smithi174.stdout:0/255: mknod d9/d21/d31/c52 0 2024-03-21T10:08:50.297 INFO:tasks.workunit.client.0.smithi153.stdout:7/360: mknod d4/c7a 0 2024-03-21T10:08:50.298 INFO:tasks.workunit.client.1.smithi174.stdout:9/305: truncate d2/d24/d2e/f48 2997631 0 2024-03-21T10:08:50.304 INFO:tasks.workunit.client.1.smithi174.stdout:8/210: mknod d9/d17/c4a 0 2024-03-21T10:08:50.304 INFO:tasks.workunit.client.1.smithi174.stdout:5/248: chown d1/d2f/d37/l3c 313 1 2024-03-21T10:08:50.307 INFO:tasks.workunit.client.1.smithi174.stdout:6/196: getdents d0/d3/d20/d3d 0 2024-03-21T10:08:50.307 INFO:tasks.workunit.client.0.smithi153.stdout:0/377: creat d4/d16/d3e/d5e/f75 x:0 0 0 2024-03-21T10:08:50.308 INFO:tasks.workunit.client.1.smithi174.stdout:5/249: fsync d1/d16/f24 0 2024-03-21T10:08:50.309 INFO:tasks.workunit.client.0.smithi153.stdout:4/463: dwrite d11/d2d/d3d/f67 [0,4194304] 0 2024-03-21T10:08:50.309 INFO:tasks.workunit.client.1.smithi174.stdout:1/263: rmdir d4 39 2024-03-21T10:08:50.314 INFO:tasks.workunit.client.1.smithi174.stdout:6/197: write d0/d3/d5/d6/f38 [486585,81952] 0 2024-03-21T10:08:50.314 INFO:tasks.workunit.client.1.smithi174.stdout:7/195: link d0/d4/d10/d1b/f3b d0/d4/f43 0 2024-03-21T10:08:50.315 INFO:tasks.workunit.client.0.smithi153.stdout:8/458: mknod d5/d9/d43/dd/d2d/c97 0 2024-03-21T10:08:50.317 INFO:tasks.workunit.client.0.smithi153.stdout:8/459: chown d5/d9/d43/f16 151486186 1 2024-03-21T10:08:50.318 INFO:tasks.workunit.client.1.smithi174.stdout:0/256: dwrite d9/d21/d31/f42 [0,4194304] 0 2024-03-21T10:08:50.321 INFO:tasks.workunit.client.1.smithi174.stdout:9/306: unlink d2/l15 0 2024-03-21T10:08:50.322 INFO:tasks.workunit.client.0.smithi153.stdout:5/481: getdents d4/d5/df/d67 0 2024-03-21T10:08:50.322 INFO:tasks.workunit.client.1.smithi174.stdout:5/250: unlink d1/d16/f50 0 2024-03-21T10:08:50.324 INFO:tasks.workunit.client.1.smithi174.stdout:1/264: truncate d4/d5/fd 1605393 0 2024-03-21T10:08:50.325 INFO:tasks.workunit.client.1.smithi174.stdout:3/223: dwrite d0/fa [0,4194304] 0 2024-03-21T10:08:50.326 INFO:tasks.workunit.client.1.smithi174.stdout:8/211: unlink d9/d17/d19/d33/f3f 0 2024-03-21T10:08:50.328 INFO:tasks.workunit.client.1.smithi174.stdout:5/251: chown d1/d2f/c3d 191615756 1 2024-03-21T10:08:50.329 INFO:tasks.workunit.client.0.smithi153.stdout:9/446: write d1/d43/d44/d46/d1c/d4b/f73 [878450,70362] 0 2024-03-21T10:08:50.329 INFO:tasks.workunit.client.1.smithi174.stdout:4/291: getdents d4/d21 0 2024-03-21T10:08:50.331 INFO:tasks.workunit.client.0.smithi153.stdout:6/637: creat d1f/d2b/d33/d40/d7b/fe3 x:0 0 0 2024-03-21T10:08:50.332 INFO:tasks.workunit.client.1.smithi174.stdout:8/212: stat d9/f1a 0 2024-03-21T10:08:50.333 INFO:tasks.workunit.client.0.smithi153.stdout:1/379: dwrite f4 [0,4194304] 0 2024-03-21T10:08:50.334 INFO:tasks.workunit.client.1.smithi174.stdout:4/292: dread - d4/d21/f5b zero size 2024-03-21T10:08:50.335 INFO:tasks.workunit.client.0.smithi153.stdout:2/496: dwrite dd/d15/d18/d2c/d33/f43 [0,4194304] 0 2024-03-21T10:08:50.335 INFO:tasks.workunit.client.0.smithi153.stdout:8/460: fdatasync d5/d9/d43/dd/d2d/d45/d88/f46 0 2024-03-21T10:08:50.336 INFO:tasks.workunit.client.1.smithi174.stdout:8/213: chown d9/fe 640815 1 2024-03-21T10:08:50.336 INFO:tasks.workunit.client.1.smithi174.stdout:2/283: dwrite d5/f24 [0,4194304] 0 2024-03-21T10:08:50.336 INFO:tasks.workunit.client.0.smithi153.stdout:5/482: write d4/d8/d3b/f4b [4617153,100591] 0 2024-03-21T10:08:50.337 INFO:tasks.workunit.client.1.smithi174.stdout:3/224: chown d0/d12/d14/d1a/d27/d4f 46774 1 2024-03-21T10:08:50.338 INFO:tasks.workunit.client.0.smithi153.stdout:3/404: mkdir dd/d7e/d73/d7f/d91 0 2024-03-21T10:08:50.343 INFO:tasks.workunit.client.0.smithi153.stdout:1/380: truncate d6/fd 2602190 0 2024-03-21T10:08:50.346 INFO:tasks.workunit.client.0.smithi153.stdout:7/361: creat d4/d2a/d40/d50/f7b x:0 0 0 2024-03-21T10:08:50.346 INFO:tasks.workunit.client.1.smithi174.stdout:8/214: write d9/d17/d19/d27/f38 [970646,73898] 0 2024-03-21T10:08:50.347 INFO:tasks.workunit.client.1.smithi174.stdout:6/198: dwrite d0/d3/d5/d6/f38 [0,4194304] 0 2024-03-21T10:08:50.348 INFO:tasks.workunit.client.0.smithi153.stdout:3/405: dread dd/f15 [0,4194304] 0 2024-03-21T10:08:50.348 INFO:tasks.workunit.client.1.smithi174.stdout:1/265: creat d4/d7/d15/d4f/f61 x:0 0 0 2024-03-21T10:08:50.349 INFO:tasks.workunit.client.1.smithi174.stdout:5/252: fsync d1/ff 0 2024-03-21T10:08:50.349 INFO:tasks.workunit.client.1.smithi174.stdout:7/196: mkdir d0/d4/d10/d44 0 2024-03-21T10:08:50.352 INFO:tasks.workunit.client.1.smithi174.stdout:4/293: fsync d4/d21/d24/d28/d38/f54 0 2024-03-21T10:08:50.354 INFO:tasks.workunit.client.1.smithi174.stdout:3/225: truncate d0/d12/d14/d1a/f3f 1012314 0 2024-03-21T10:08:50.355 INFO:tasks.workunit.client.1.smithi174.stdout:0/257: rename d9/d21/f3b to d9/d35/d4a/f53 0 2024-03-21T10:08:50.355 INFO:tasks.workunit.client.1.smithi174.stdout:1/266: chown d4/d2d/f34 1966757 1 2024-03-21T10:08:50.357 INFO:tasks.workunit.client.0.smithi153.stdout:0/378: fsync d4/d16/d3e/d5e/f75 0 2024-03-21T10:08:50.364 INFO:tasks.workunit.client.0.smithi153.stdout:9/447: write d1/d43/d44/d46/d1c/d24/d41/f5b [6916335,87761] 0 2024-03-21T10:08:50.364 INFO:tasks.workunit.client.0.smithi153.stdout:5/483: symlink d4/d5/d16/d24/l98 0 2024-03-21T10:08:50.368 INFO:tasks.workunit.client.0.smithi153.stdout:1/381: rmdir d6/d3a/d66 39 2024-03-21T10:08:50.372 INFO:tasks.workunit.client.1.smithi174.stdout:6/199: unlink d0/d3/d5/df/c3e 0 2024-03-21T10:08:50.375 INFO:tasks.workunit.client.1.smithi174.stdout:8/215: chown d9/d17/d19/d27/l45 600 1 2024-03-21T10:08:50.376 INFO:tasks.workunit.client.1.smithi174.stdout:6/200: write d0/d3/f26 [1144296,107361] 0 2024-03-21T10:08:50.377 INFO:tasks.workunit.client.0.smithi153.stdout:8/461: unlink d5/d9/d43/dd/d2d/d45/d88/f6b 0 2024-03-21T10:08:50.381 INFO:tasks.workunit.client.1.smithi174.stdout:5/253: dread d1/f22 [0,4194304] 0 2024-03-21T10:08:50.382 INFO:tasks.workunit.client.0.smithi153.stdout:7/362: unlink d4/d11/f48 0 2024-03-21T10:08:50.383 INFO:tasks.workunit.client.1.smithi174.stdout:2/284: dwrite d5/d25/d27/d2d/d45/d5b/f23 [4194304,4194304] 0 2024-03-21T10:08:50.386 INFO:tasks.workunit.client.1.smithi174.stdout:9/307: link d2/d11/d1d/l5d d2/d11/d1d/d5e/l6a 0 2024-03-21T10:08:50.387 INFO:tasks.workunit.client.1.smithi174.stdout:2/285: readlink d5/d25/d27/l32 0 2024-03-21T10:08:50.388 INFO:tasks.workunit.client.1.smithi174.stdout:7/197: write d0/f26 [7179421,60142] 0 2024-03-21T10:08:50.388 INFO:tasks.workunit.client.1.smithi174.stdout:4/294: dread d4/d21/d24/d28/f4e [0,4194304] 0 2024-03-21T10:08:50.389 INFO:tasks.workunit.client.1.smithi174.stdout:6/201: mkdir d0/d3/d5/d6/d42 0 2024-03-21T10:08:50.391 INFO:tasks.workunit.client.0.smithi153.stdout:3/406: symlink dd/d7e/d73/l92 0 2024-03-21T10:08:50.394 INFO:tasks.workunit.client.1.smithi174.stdout:2/286: write d5/d25/d27/d2e/f3f [825170,53173] 0 2024-03-21T10:08:50.394 INFO:tasks.workunit.client.1.smithi174.stdout:0/258: link d9/d21/d31/f34 d9/f54 0 2024-03-21T10:08:50.398 INFO:tasks.workunit.client.0.smithi153.stdout:0/379: rename d4/d16/d27/f39 to d4/d16/d27/f76 0 2024-03-21T10:08:50.400 INFO:tasks.workunit.client.1.smithi174.stdout:6/202: write d0/d3/d5/df/d16/f2b [1428119,119384] 0 2024-03-21T10:08:50.400 INFO:tasks.workunit.client.1.smithi174.stdout:3/226: truncate d0/fa 3309480 0 2024-03-21T10:08:50.403 INFO:tasks.workunit.client.1.smithi174.stdout:6/203: write d0/d3/d20/f23 [905717,91843] 0 2024-03-21T10:08:50.404 INFO:tasks.workunit.client.1.smithi174.stdout:5/254: rename d1/d16/d1c/f26 to d1/d49/d4f/d51/f56 0 2024-03-21T10:08:50.407 INFO:tasks.workunit.client.1.smithi174.stdout:4/295: creat d4/d21/d31/d3a/f60 x:0 0 0 2024-03-21T10:08:50.410 INFO:tasks.workunit.client.0.smithi153.stdout:6/638: dwrite d1f/d99/f9c [0,4194304] 0 2024-03-21T10:08:50.411 INFO:tasks.workunit.client.1.smithi174.stdout:6/204: fdatasync d0/d3/d5/df/d16/f17 0 2024-03-21T10:08:50.412 INFO:tasks.workunit.client.0.smithi153.stdout:0/380: truncate d4/d16/d19/f74 929206 0 2024-03-21T10:08:50.413 INFO:tasks.workunit.client.1.smithi174.stdout:4/296: write d4/f3e [4839744,35535] 0 2024-03-21T10:08:50.419 INFO:tasks.workunit.client.1.smithi174.stdout:8/216: dwrite f1 [0,4194304] 0 2024-03-21T10:08:50.419 INFO:tasks.workunit.client.0.smithi153.stdout:0/381: chown d4/d4c 81481507 1 2024-03-21T10:08:50.419 INFO:tasks.workunit.client.0.smithi153.stdout:6/639: chown d1f/d2b/f85 26 1 2024-03-21T10:08:50.419 INFO:tasks.workunit.client.0.smithi153.stdout:7/363: rename d4/d1b/f5d to d4/d2a/d3b/d60/f7c 0 2024-03-21T10:08:50.421 INFO:tasks.workunit.client.0.smithi153.stdout:5/484: creat d4/d8/d5c/d90/d95/f99 x:0 0 0 2024-03-21T10:08:50.421 INFO:tasks.workunit.client.0.smithi153.stdout:8/462: dwrite d5/d9/d43/dd/d20/d26/d24/f32 [4194304,4194304] 0 2024-03-21T10:08:50.421 INFO:tasks.workunit.client.0.smithi153.stdout:7/364: dread - d4/f6c zero size 2024-03-21T10:08:50.422 INFO:tasks.workunit.client.1.smithi174.stdout:1/267: dread d4/fb [0,4194304] 0 2024-03-21T10:08:50.426 INFO:tasks.workunit.client.0.smithi153.stdout:8/463: fdatasync d5/d9/d43/dd/d2d/f31 0 2024-03-21T10:08:50.426 INFO:tasks.workunit.client.0.smithi153.stdout:4/464: link d11/d18/d54/c81 d11/d2d/d57/d85/c87 0 2024-03-21T10:08:50.427 INFO:tasks.workunit.client.1.smithi174.stdout:2/287: dwrite d5/f49 [0,4194304] 0 2024-03-21T10:08:50.430 INFO:tasks.workunit.client.0.smithi153.stdout:5/485: mknod d4/d5/de/d44/c9a 0 2024-03-21T10:08:50.431 INFO:tasks.workunit.client.0.smithi153.stdout:0/382: mkdir d4/d16/d1f/d22/d61/d77 0 2024-03-21T10:08:50.439 INFO:tasks.workunit.client.0.smithi153.stdout:8/464: creat d5/d9/d43/d7b/f98 x:0 0 0 2024-03-21T10:08:50.442 INFO:tasks.workunit.client.0.smithi153.stdout:5/486: write d4/d5/de/d44/d48/f87 [917770,114730] 0 2024-03-21T10:08:50.442 INFO:tasks.workunit.client.0.smithi153.stdout:8/465: dread - d5/d9/f8d zero size 2024-03-21T10:08:50.446 INFO:tasks.workunit.client.0.smithi153.stdout:4/465: creat d11/d2d/d57/f88 x:0 0 0 2024-03-21T10:08:50.446 INFO:tasks.workunit.client.0.smithi153.stdout:0/383: rmdir d4/d16/d1f/d22/d61 39 2024-03-21T10:08:50.446 INFO:tasks.workunit.client.0.smithi153.stdout:8/466: chown d5/d9/c58 137689 1 2024-03-21T10:08:50.450 INFO:tasks.workunit.client.0.smithi153.stdout:6/640: dwrite d1f/d93/dd8/fd1 [0,4194304] 0 2024-03-21T10:08:50.465 INFO:tasks.workunit.client.0.smithi153.stdout:7/365: rename d4/d2a/d40/d50/f69 to d4/d1b/f7d 0 2024-03-21T10:08:50.466 INFO:tasks.workunit.client.0.smithi153.stdout:3/407: dread dd/f2b [0,4194304] 0 2024-03-21T10:08:50.469 INFO:tasks.workunit.client.0.smithi153.stdout:2/497: truncate dd/d15/d18/d2c/d33/f43 4117131 0 2024-03-21T10:08:50.470 INFO:tasks.workunit.client.0.smithi153.stdout:7/366: write d4/d14/d2f/f5e [403500,98499] 0 2024-03-21T10:08:50.472 INFO:tasks.workunit.client.1.smithi174.stdout:5/255: creat d1/d49/d4f/f57 x:0 0 0 2024-03-21T10:08:50.477 INFO:tasks.workunit.client.0.smithi153.stdout:1/382: truncate f4 2449620 0 2024-03-21T10:08:50.478 INFO:tasks.workunit.client.1.smithi174.stdout:9/308: dwrite d2/d54/f5b [0,4194304] 0 2024-03-21T10:08:50.483 INFO:tasks.workunit.client.0.smithi153.stdout:0/384: creat d4/d16/d3e/d5e/f78 x:0 0 0 2024-03-21T10:08:50.484 INFO:tasks.workunit.client.1.smithi174.stdout:1/268: symlink d4/d7/d28/l62 0 2024-03-21T10:08:50.485 INFO:tasks.workunit.client.1.smithi174.stdout:2/288: mknod d5/d25/d27/d2d/d45/d5b/d2f/c6b 0 2024-03-21T10:08:50.487 INFO:tasks.workunit.client.0.smithi153.stdout:3/408: mkdir dd/d7e/d93 0 2024-03-21T10:08:50.488 INFO:tasks.workunit.client.1.smithi174.stdout:5/256: mkdir d1/d49/d58 0 2024-03-21T10:08:50.489 INFO:tasks.workunit.client.1.smithi174.stdout:1/269: read d4/d5/fd [1355858,8387] 0 2024-03-21T10:08:50.490 INFO:tasks.workunit.client.1.smithi174.stdout:5/257: read d1/f22 [2697578,34967] 0 2024-03-21T10:08:50.491 INFO:tasks.workunit.client.0.smithi153.stdout:8/467: mkdir d5/d9/d43/dd/d20/d99 0 2024-03-21T10:08:50.493 INFO:tasks.workunit.client.0.smithi153.stdout:4/466: getdents d11/d71 0 2024-03-21T10:08:50.494 INFO:tasks.workunit.client.0.smithi153.stdout:3/409: rmdir dd/d7e/d73/d7f 39 2024-03-21T10:08:50.499 INFO:tasks.workunit.client.1.smithi174.stdout:4/297: getdents d4/d21/d24/d28 0 2024-03-21T10:08:50.499 INFO:tasks.workunit.client.1.smithi174.stdout:2/289: truncate d5/d25/d27/d2d/d45/d5b/f22 902340 0 2024-03-21T10:08:50.499 INFO:tasks.workunit.client.0.smithi153.stdout:3/410: write dd/f43 [7413270,85484] 0 2024-03-21T10:08:50.501 INFO:tasks.workunit.client.1.smithi174.stdout:6/205: getdents d0/d3/d5/d3b 0 2024-03-21T10:08:50.502 INFO:tasks.workunit.client.0.smithi153.stdout:8/468: chown d5/d9/d43/dd/d20/d26/d8b/f62 20534130 1 2024-03-21T10:08:50.503 INFO:tasks.workunit.client.1.smithi174.stdout:6/206: chown d0/l35 732 1 2024-03-21T10:08:50.511 INFO:tasks.workunit.client.1.smithi174.stdout:6/207: unlink d0/d3/d5/df/d16/c1f 0 2024-03-21T10:08:50.511 INFO:tasks.workunit.client.1.smithi174.stdout:5/258: dwrite d1/d2f/f4c [0,4194304] 0 2024-03-21T10:08:50.512 INFO:tasks.workunit.client.1.smithi174.stdout:4/298: dread d4/f3e [0,4194304] 0 2024-03-21T10:08:50.514 INFO:tasks.workunit.client.1.smithi174.stdout:2/290: dread d5/f24 [0,4194304] 0 2024-03-21T10:08:50.514 INFO:tasks.workunit.client.1.smithi174.stdout:4/299: truncate d4/d21/d24/f4d 984579 0 2024-03-21T10:08:50.516 INFO:tasks.workunit.client.1.smithi174.stdout:6/208: rmdir d0/d3/d5/d3b 39 2024-03-21T10:08:50.517 INFO:tasks.workunit.client.1.smithi174.stdout:5/259: symlink d1/d49/d4f/l59 0 2024-03-21T10:08:50.519 INFO:tasks.workunit.client.1.smithi174.stdout:4/300: creat d4/d21/d24/d28/d38/f61 x:0 0 0 2024-03-21T10:08:50.521 INFO:tasks.workunit.client.1.smithi174.stdout:6/209: readlink d0/d3/d5/l28 0 2024-03-21T10:08:50.521 INFO:tasks.workunit.client.1.smithi174.stdout:5/260: symlink d1/d49/d4f/d51/l5a 0 2024-03-21T10:08:50.528 INFO:tasks.workunit.client.1.smithi174.stdout:3/227: rename d0/d12/d14/c36 to d0/d12/d14/d1a/d27/c50 0 2024-03-21T10:08:50.533 INFO:tasks.workunit.client.0.smithi153.stdout:2/498: dwrite dd/d15/d18/d2c/d4b/f66 [0,4194304] 0 2024-03-21T10:08:50.535 INFO:tasks.workunit.client.1.smithi174.stdout:8/217: rename d9/d17/l1b to d9/d2a/d39/l4b 0 2024-03-21T10:08:50.536 INFO:tasks.workunit.client.0.smithi153.stdout:9/448: dread d1/d43/d44/d46/d1c/d2e/f51 [0,4194304] 0 2024-03-21T10:08:50.539 INFO:tasks.workunit.client.1.smithi174.stdout:2/291: dwrite d5/d25/d27/f4e [0,4194304] 0 2024-03-21T10:08:50.539 INFO:tasks.workunit.client.0.smithi153.stdout:0/385: getdents d4/db 0 2024-03-21T10:08:50.540 INFO:tasks.workunit.client.1.smithi174.stdout:3/228: symlink d0/d12/d14/d1a/d44/l51 0 2024-03-21T10:08:50.541 INFO:tasks.workunit.client.0.smithi153.stdout:8/469: mkdir d5/d9/d43/dd/d20/d26/d47/d9a 0 2024-03-21T10:08:50.543 INFO:tasks.workunit.client.0.smithi153.stdout:7/367: dwrite d4/d14/d16/f42 [0,4194304] 0 2024-03-21T10:08:50.543 INFO:tasks.workunit.client.1.smithi174.stdout:6/210: dread d0/d3/d5/df/d16/f27 [0,4194304] 0 2024-03-21T10:08:50.545 INFO:tasks.workunit.client.1.smithi174.stdout:7/198: truncate d0/d4/f17 677464 0 2024-03-21T10:08:50.545 INFO:tasks.workunit.client.0.smithi153.stdout:5/487: dwrite d4/d5/de/d38/f64 [0,4194304] 0 2024-03-21T10:08:50.548 INFO:tasks.workunit.client.1.smithi174.stdout:5/261: dread d1/d16/f40 [0,4194304] 0 2024-03-21T10:08:50.553 INFO:tasks.workunit.client.1.smithi174.stdout:5/262: dread - d1/d2f/d37/f3f zero size 2024-03-21T10:08:50.557 INFO:tasks.workunit.client.0.smithi153.stdout:8/470: creat d5/d9/f9b x:0 0 0 2024-03-21T10:08:50.557 INFO:tasks.workunit.client.0.smithi153.stdout:6/641: dwrite d1f/d2b/d33/d40/d65/f69 [0,4194304] 0 2024-03-21T10:08:50.558 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:50 smithi153 ceph-mon[29117]: pgmap v11: 65 pgs: 65 active+clean; 924 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 9.5 MiB/s rd, 96 MiB/s wr, 217 op/s 2024-03-21T10:08:50.558 INFO:tasks.workunit.client.1.smithi174.stdout:4/301: dwrite d4/d21/d31/d3a/f4c [0,4194304] 0 2024-03-21T10:08:50.559 INFO:tasks.workunit.client.1.smithi174.stdout:5/263: fdatasync d1/f11 0 2024-03-21T10:08:50.559 INFO:tasks.workunit.client.0.smithi153.stdout:5/488: truncate d4/d5/d16/f19 4675688 0 2024-03-21T10:08:50.562 INFO:tasks.workunit.client.0.smithi153.stdout:2/499: fsync dd/d15/d18/d2c/d4b/f67 0 2024-03-21T10:08:50.564 INFO:tasks.workunit.client.0.smithi153.stdout:6/642: symlink d1f/da0/le4 0 2024-03-21T10:08:50.564 INFO:tasks.workunit.client.1.smithi174.stdout:3/229: creat d0/f52 x:0 0 0 2024-03-21T10:08:50.564 INFO:tasks.workunit.client.1.smithi174.stdout:6/211: creat d0/d3/d5/d6/d42/f43 x:0 0 0 2024-03-21T10:08:50.567 INFO:tasks.workunit.client.0.smithi153.stdout:0/386: rename d4/d16/d1f/f31 to d4/d16/d27/f79 0 2024-03-21T10:08:50.567 INFO:tasks.workunit.client.0.smithi153.stdout:9/449: symlink d1/d43/d44/d46/d1c/d4b/d84/d8b/l9a 0 2024-03-21T10:08:50.567 INFO:tasks.workunit.client.0.smithi153.stdout:1/383: dwrite d6/d13/f45 [0,4194304] 0 2024-03-21T10:08:50.568 INFO:tasks.workunit.client.1.smithi174.stdout:3/230: chown d0/d12/d14/d1a/f42 241708 1 2024-03-21T10:08:50.572 INFO:tasks.workunit.client.0.smithi153.stdout:2/500: chown dd/l23 1 1 2024-03-21T10:08:50.572 INFO:tasks.workunit.client.0.smithi153.stdout:5/489: chown d4/d5/de/d44/f6f 4050 1 2024-03-21T10:08:50.575 INFO:tasks.workunit.client.0.smithi153.stdout:2/501: chown dd/d15/c30 879178652 1 2024-03-21T10:08:50.576 INFO:tasks.workunit.client.1.smithi174.stdout:5/264: creat d1/d49/d58/f5b x:0 0 0 2024-03-21T10:08:50.576 INFO:tasks.workunit.client.1.smithi174.stdout:2/292: dread d5/d25/d27/f4e [0,4194304] 0 2024-03-21T10:08:50.576 INFO:tasks.workunit.client.0.smithi153.stdout:8/471: mknod d5/d9/d43/dd/d20/d26/d47/c9c 0 2024-03-21T10:08:50.578 INFO:tasks.workunit.client.1.smithi174.stdout:1/270: rename l3 to d4/d7/d33/d43/d51/l63 0 2024-03-21T10:08:50.579 INFO:tasks.workunit.client.1.smithi174.stdout:2/293: chown d5/d25/d27/d2d/d45/d5b/f1c 3 1 2024-03-21T10:08:50.582 INFO:tasks.workunit.client.0.smithi153.stdout:5/490: write d4/d5/df/d67/d6a/f77 [407264,115520] 0 2024-03-21T10:08:50.583 INFO:tasks.workunit.client.1.smithi174.stdout:3/231: mknod d0/d12/d14/d1a/d27/c53 0 2024-03-21T10:08:50.585 INFO:tasks.workunit.client.0.smithi153.stdout:9/450: write d1/f1b [1068965,51647] 0 2024-03-21T10:08:50.585 INFO:tasks.workunit.client.0.smithi153.stdout:5/491: fdatasync f1 0 2024-03-21T10:08:50.586 INFO:tasks.workunit.client.1.smithi174.stdout:9/309: rename d2/d11/c12 to d2/d11/d1d/d2a/c6b 0 2024-03-21T10:08:50.586 INFO:tasks.workunit.client.0.smithi153.stdout:0/387: dread - d4/d16/d19/d58/f6a zero size 2024-03-21T10:08:50.590 INFO:tasks.workunit.client.1.smithi174.stdout:1/271: creat d4/d7/d33/d43/d51/f64 x:0 0 0 2024-03-21T10:08:50.590 INFO:tasks.workunit.client.1.smithi174.stdout:1/272: dread d4/d7/d33/f3d [0,4194304] 0 2024-03-21T10:08:50.591 INFO:tasks.workunit.client.1.smithi174.stdout:1/273: chown d4/d7/f21 3 1 2024-03-21T10:08:50.592 INFO:tasks.workunit.client.0.smithi153.stdout:3/411: dwrite dd/d1e/d45/d56/d58/f86 [0,4194304] 0 2024-03-21T10:08:50.593 INFO:tasks.workunit.client.1.smithi174.stdout:5/265: mkdir d1/d5c 0 2024-03-21T10:08:50.593 INFO:tasks.workunit.client.0.smithi153.stdout:6/643: link d1f/f6c d1f/d2b/d33/d40/d51/d98/fe5 0 2024-03-21T10:08:50.595 INFO:tasks.workunit.client.1.smithi174.stdout:5/266: write d1/f11 [130148,117716] 0 2024-03-21T10:08:50.599 INFO:tasks.workunit.client.0.smithi153.stdout:9/451: symlink d1/d4/l9b 0 2024-03-21T10:08:50.601 INFO:tasks.workunit.client.0.smithi153.stdout:0/388: write d4/d16/d1f/f20 [2349708,16452] 0 2024-03-21T10:08:50.602 INFO:tasks.workunit.client.0.smithi153.stdout:6/644: stat d1f/d2b/d33/d40 0 2024-03-21T10:08:50.603 INFO:tasks.workunit.client.1.smithi174.stdout:9/310: creat d2/d24/d2e/d63/f6c x:0 0 0 2024-03-21T10:08:50.603 INFO:tasks.workunit.client.1.smithi174.stdout:5/267: creat d1/d2f/d37/f5d x:0 0 0 2024-03-21T10:08:50.604 INFO:tasks.workunit.client.0.smithi153.stdout:7/368: truncate d4/d14/d2f/d78/f35 689006 0 2024-03-21T10:08:50.604 INFO:tasks.workunit.client.0.smithi153.stdout:9/452: truncate d1/d4/fc 1992530 0 2024-03-21T10:08:50.608 INFO:tasks.workunit.client.0.smithi153.stdout:3/412: rename dd/d7e to dd/d1e/d3d/d94 0 2024-03-21T10:08:50.611 INFO:tasks.workunit.client.1.smithi174.stdout:3/232: getdents d0/d12/d14/d10 0 2024-03-21T10:08:50.612 INFO:tasks.workunit.client.0.smithi153.stdout:9/453: read - d1/d4/f75 zero size 2024-03-21T10:08:50.613 INFO:tasks.workunit.client.0.smithi153.stdout:7/369: symlink d4/d2a/l7e 0 2024-03-21T10:08:50.614 INFO:tasks.workunit.client.1.smithi174.stdout:9/311: read d2/f28 [2002157,38335] 0 2024-03-21T10:08:50.615 INFO:tasks.workunit.client.1.smithi174.stdout:3/233: stat d0/d12/d14/d10 0 2024-03-21T10:08:50.615 INFO:tasks.workunit.client.0.smithi153.stdout:3/413: fsync dd/d1e/f46 0 2024-03-21T10:08:50.617 INFO:tasks.workunit.client.1.smithi174.stdout:9/312: chown d2/d24/c65 2707 1 2024-03-21T10:08:50.619 INFO:tasks.workunit.client.0.smithi153.stdout:9/454: creat d1/d4/f9c x:0 0 0 2024-03-21T10:08:50.619 INFO:tasks.workunit.client.1.smithi174.stdout:2/294: rename d5/d25/c35 to d5/d25/d27/d2d/d45/c6c 0 2024-03-21T10:08:50.620 INFO:tasks.workunit.client.1.smithi174.stdout:4/302: write d4/f10 [880708,26720] 0 2024-03-21T10:08:50.621 INFO:tasks.workunit.client.1.smithi174.stdout:7/199: truncate d0/d4/f8 3521377 0 2024-03-21T10:08:50.623 INFO:tasks.workunit.client.0.smithi153.stdout:0/389: link d4/f62 d4/d4c/f7a 0 2024-03-21T10:08:50.624 INFO:tasks.workunit.client.0.smithi153.stdout:5/492: dwrite d4/d8/f6c [0,4194304] 0 2024-03-21T10:08:50.625 INFO:tasks.workunit.client.1.smithi174.stdout:1/274: rename d4/d7/d28/l62 to d4/d7/d15/l65 0 2024-03-21T10:08:50.626 INFO:tasks.workunit.client.1.smithi174.stdout:5/268: dread d1/ff [4194304,4194304] 0 2024-03-21T10:08:50.626 INFO:tasks.workunit.client.0.smithi153.stdout:1/384: dwrite d6/d32/f37 [4194304,4194304] 0 2024-03-21T10:08:50.627 INFO:tasks.workunit.client.0.smithi153.stdout:8/472: dread d5/d9/d43/dd/d20/d26/d24/d52/f54 [0,4194304] 0 2024-03-21T10:08:50.629 INFO:tasks.workunit.client.0.smithi153.stdout:0/390: chown d4/db/l46 11140171 1 2024-03-21T10:08:50.630 INFO:tasks.workunit.client.0.smithi153.stdout:9/455: write d1/d43/d44/d46/f19 [2607981,124254] 0 2024-03-21T10:08:50.631 INFO:tasks.workunit.client.0.smithi153.stdout:2/502: truncate dd/d15/d3c/f69 815143 0 2024-03-21T10:08:50.634 INFO:tasks.workunit.client.1.smithi174.stdout:4/303: symlink d4/d21/l62 0 2024-03-21T10:08:50.634 INFO:tasks.workunit.client.0.smithi153.stdout:5/493: chown d4/d8/d5a/d92/f94 1130 1 2024-03-21T10:08:50.634 INFO:tasks.workunit.client.0.smithi153.stdout:6/645: write d1f/d93/dd8/f4c [66078,29107] 0 2024-03-21T10:08:50.635 INFO:tasks.workunit.client.0.smithi153.stdout:1/385: truncate d6/fc 6230755 0 2024-03-21T10:08:50.639 INFO:tasks.workunit.client.0.smithi153.stdout:0/391: chown d4/d16/d1f/d22/d61/d77 2 1 2024-03-21T10:08:50.641 INFO:tasks.workunit.client.1.smithi174.stdout:6/212: dread d0/d3/d20/f23 [0,4194304] 0 2024-03-21T10:08:50.641 INFO:tasks.workunit.client.1.smithi174.stdout:5/269: creat d1/d2f/f5e x:0 0 0 2024-03-21T10:08:50.641 INFO:tasks.workunit.client.0.smithi153.stdout:2/503: mknod dd/d15/d18/d2c/d4b/d56/ca6 0 2024-03-21T10:08:50.642 INFO:tasks.workunit.client.0.smithi153.stdout:8/473: dread d5/d1b/f5e [4194304,4194304] 0 2024-03-21T10:08:50.642 INFO:tasks.workunit.client.0.smithi153.stdout:5/494: mknod d4/d8/d5a/c9b 0 2024-03-21T10:08:50.644 INFO:tasks.workunit.client.0.smithi153.stdout:9/456: mkdir d1/d43/d72/d9d 0 2024-03-21T10:08:50.647 INFO:tasks.workunit.client.0.smithi153.stdout:6/646: mknod d1f/d2b/ce6 0 2024-03-21T10:08:50.648 INFO:tasks.workunit.client.1.smithi174.stdout:7/200: fsync d0/fa 0 2024-03-21T10:08:50.649 INFO:tasks.workunit.client.1.smithi174.stdout:5/270: dread d1/d16/d2e/d31/f48 [0,4194304] 0 2024-03-21T10:08:50.650 INFO:tasks.workunit.client.1.smithi174.stdout:3/234: creat d0/d12/f54 x:0 0 0 2024-03-21T10:08:50.652 INFO:tasks.workunit.client.0.smithi153.stdout:0/392: fsync d4/f28 0 2024-03-21T10:08:50.654 INFO:tasks.workunit.client.1.smithi174.stdout:9/313: link d2/d11/f30 d2/d11/d60/f6d 0 2024-03-21T10:08:50.656 INFO:tasks.workunit.client.0.smithi153.stdout:5/495: symlink d4/d8/d5c/l9c 0 2024-03-21T10:08:50.656 INFO:tasks.workunit.client.1.smithi174.stdout:1/275: creat d4/d7/d28/d59/f66 x:0 0 0 2024-03-21T10:08:50.657 INFO:tasks.workunit.client.1.smithi174.stdout:6/213: creat d0/d3/d5/d6/f44 x:0 0 0 2024-03-21T10:08:50.658 INFO:tasks.workunit.client.0.smithi153.stdout:7/370: dwrite d4/d11/f61 [0,4194304] 0 2024-03-21T10:08:50.658 INFO:tasks.workunit.client.0.smithi153.stdout:0/393: dread - d4/d16/d3e/d5e/f75 zero size 2024-03-21T10:08:50.661 INFO:tasks.workunit.client.0.smithi153.stdout:1/386: rename d6/d10/f34 to d6/d3a/d3c/d50/f77 0 2024-03-21T10:08:50.661 INFO:tasks.workunit.client.0.smithi153.stdout:5/496: write d4/d5/df/d5e/f6e [604876,92295] 0 2024-03-21T10:08:50.661 INFO:tasks.workunit.client.1.smithi174.stdout:7/201: truncate d0/d4/f12 4578108 0 2024-03-21T10:08:50.662 INFO:tasks.workunit.client.1.smithi174.stdout:0/259: sync 2024-03-21T10:08:50.664 INFO:tasks.workunit.client.1.smithi174.stdout:3/235: rename d0/d12/d14/d10 to d0/d12/d14/d1a/d27/d4f/d55 0 2024-03-21T10:08:50.664 INFO:tasks.workunit.client.0.smithi153.stdout:8/474: mkdir d5/d9/d43/dd/d20/d26/d24/d52/d56/d9d 0 2024-03-21T10:08:50.664 INFO:tasks.workunit.client.1.smithi174.stdout:7/202: truncate d0/d4/d10/f21 1007900 0 2024-03-21T10:08:50.666 INFO:tasks.workunit.client.0.smithi153.stdout:9/457: rename d1/d43/d44/d46/d1c to d1/d43/d44/d46/d9e 0 2024-03-21T10:08:50.670 INFO:tasks.workunit.client.0.smithi153.stdout:7/371: creat d4/d11/f7f x:0 0 0 2024-03-21T10:08:50.672 INFO:tasks.workunit.client.0.smithi153.stdout:2/504: rename dd/d15/d18/d2c/d4b/d56/d63/d8d/d8e to dd/d15/d3c/da7 0 2024-03-21T10:08:50.674 INFO:tasks.workunit.client.1.smithi174.stdout:0/260: mknod d9/d21/c55 0 2024-03-21T10:08:50.675 INFO:tasks.workunit.client.1.smithi174.stdout:4/304: link d4/d21/d24/d28/c2e d4/d21/d24/d28/d38/c63 0 2024-03-21T10:08:50.677 INFO:tasks.workunit.client.0.smithi153.stdout:0/394: unlink d4/d16/d19/f74 0 2024-03-21T10:08:50.678 INFO:tasks.workunit.client.1.smithi174.stdout:3/236: mkdir d0/d12/d14/d1a/d44/d56 0 2024-03-21T10:08:50.680 INFO:tasks.workunit.client.0.smithi153.stdout:5/497: symlink d4/d5/df/l9d 0 2024-03-21T10:08:50.680 INFO:tasks.workunit.client.1.smithi174.stdout:1/276: mkdir d4/d67 0 2024-03-21T10:08:50.681 INFO:tasks.workunit.client.1.smithi174.stdout:2/295: write d5/d25/d27/d2d/f43 [408360,114738] 0 2024-03-21T10:08:50.682 INFO:tasks.workunit.client.1.smithi174.stdout:1/277: readlink d4/d7/d15/l5f 0 2024-03-21T10:08:50.682 INFO:tasks.workunit.client.0.smithi153.stdout:5/498: chown d4/d5/df/c76 152 1 2024-03-21T10:08:50.683 INFO:tasks.workunit.client.1.smithi174.stdout:4/305: symlink d4/d21/d31/d3a/l64 0 2024-03-21T10:08:50.685 INFO:tasks.workunit.client.0.smithi153.stdout:7/372: rmdir d4/d14/d2f/d78 39 2024-03-21T10:08:50.685 INFO:tasks.workunit.client.0.smithi153.stdout:9/458: dread d1/d43/d44/d46/d9e/f31 [0,4194304] 0 2024-03-21T10:08:50.686 INFO:tasks.workunit.client.1.smithi174.stdout:3/237: mkdir d0/d12/d14/d1a/d44/d57 0 2024-03-21T10:08:50.688 INFO:tasks.workunit.client.0.smithi153.stdout:1/387: symlink d6/d13/d75/l78 0 2024-03-21T10:08:50.689 INFO:tasks.workunit.client.1.smithi174.stdout:3/238: chown d0/d12/d14/d1a/f1e 1571 1 2024-03-21T10:08:50.691 INFO:tasks.workunit.client.0.smithi153.stdout:0/395: symlink d4/d16/d1f/d22/d61/l7b 0 2024-03-21T10:08:50.692 INFO:tasks.workunit.client.1.smithi174.stdout:7/203: link d0/f27 d0/d4/d10/f45 0 2024-03-21T10:08:50.692 INFO:tasks.workunit.client.0.smithi153.stdout:6/647: link d1f/d2b/d33/d40/d51/d98/fcd d1f/d99/fe7 0 2024-03-21T10:08:50.696 INFO:tasks.workunit.client.0.smithi153.stdout:1/388: rmdir d6/d32/d3e/d4f 39 2024-03-21T10:08:50.696 INFO:tasks.workunit.client.1.smithi174.stdout:1/278: dread d4/d7/d28/f30 [0,4194304] 0 2024-03-21T10:08:50.697 INFO:tasks.workunit.client.1.smithi174.stdout:3/239: symlink d0/d12/d14/d1a/d27/d3c/l58 0 2024-03-21T10:08:50.699 INFO:tasks.workunit.client.1.smithi174.stdout:8/218: sync 2024-03-21T10:08:50.700 INFO:tasks.workunit.client.1.smithi174.stdout:7/204: rename d0/c31 to d0/d4/d10/d44/c46 0 2024-03-21T10:08:50.700 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:50 smithi174 ceph-mon[37496]: pgmap v11: 65 pgs: 65 active+clean; 924 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 9.5 MiB/s rd, 96 MiB/s wr, 217 op/s 2024-03-21T10:08:50.704 INFO:tasks.workunit.client.0.smithi153.stdout:7/373: fdatasync d4/d14/d2f/d78/f28 0 2024-03-21T10:08:50.704 INFO:tasks.workunit.client.1.smithi174.stdout:3/240: mkdir d0/d12/d46/d59 0 2024-03-21T10:08:50.705 INFO:tasks.workunit.client.0.smithi153.stdout:8/475: dread d5/d9/d43/dd/d2d/d45/d88/f46 [0,4194304] 0 2024-03-21T10:08:50.707 INFO:tasks.workunit.client.1.smithi174.stdout:7/205: creat d0/d4/f47 x:0 0 0 2024-03-21T10:08:50.709 INFO:tasks.workunit.client.0.smithi153.stdout:2/505: creat dd/d3b/d50/d58/fa8 x:0 0 0 2024-03-21T10:08:50.711 INFO:tasks.workunit.client.0.smithi153.stdout:9/459: rename d1/d43/d44/d46/f19 to d1/d43/d44/d46/d9e/f9f 0 2024-03-21T10:08:50.714 INFO:tasks.workunit.client.0.smithi153.stdout:2/506: rmdir dd/d15/d18 39 2024-03-21T10:08:50.715 INFO:tasks.workunit.client.0.smithi153.stdout:5/499: truncate d4/d5/de/d2f/f41 3185851 0 2024-03-21T10:08:50.715 INFO:tasks.workunit.client.1.smithi174.stdout:3/241: creat d0/d12/d14/d1a/d27/d3c/f5a x:0 0 0 2024-03-21T10:08:50.717 INFO:tasks.workunit.client.1.smithi174.stdout:6/214: dread d0/d3/d5/d6/f13 [0,4194304] 0 2024-03-21T10:08:50.717 INFO:tasks.workunit.client.1.smithi174.stdout:1/279: dread d4/d7/d15/f38 [0,4194304] 0 2024-03-21T10:08:50.717 INFO:tasks.workunit.client.0.smithi153.stdout:1/389: dwrite d6/fe [0,4194304] 0 2024-03-21T10:08:50.718 INFO:tasks.workunit.client.0.smithi153.stdout:8/476: fsync d5/d9/d43/dd/d20/d26/d24/d52/f54 0 2024-03-21T10:08:50.719 INFO:tasks.workunit.client.1.smithi174.stdout:8/219: getdents d9/d17/d19/d31 0 2024-03-21T10:08:50.720 INFO:tasks.workunit.client.1.smithi174.stdout:2/296: truncate d5/d25/d27/f3c 52368 0 2024-03-21T10:08:50.720 INFO:tasks.workunit.client.1.smithi174.stdout:5/271: dwrite f0 [0,4194304] 0 2024-03-21T10:08:50.725 INFO:tasks.workunit.client.1.smithi174.stdout:9/314: dwrite d2/d11/d1d/f4e [0,4194304] 0 2024-03-21T10:08:50.727 INFO:tasks.workunit.client.0.smithi153.stdout:9/460: creat d1/d43/d72/d9d/fa0 x:0 0 0 2024-03-21T10:08:50.727 INFO:tasks.workunit.client.1.smithi174.stdout:3/242: truncate d0/d12/d14/d1a/d27/f47 87431 0 2024-03-21T10:08:50.728 INFO:tasks.workunit.client.1.smithi174.stdout:0/261: dread d9/f13 [0,4194304] 0 2024-03-21T10:08:50.729 INFO:tasks.workunit.client.0.smithi153.stdout:6/648: dread d1f/d2b/d33/d40/d51/f52 [0,4194304] 0 2024-03-21T10:08:50.729 INFO:tasks.workunit.client.0.smithi153.stdout:0/396: dread d4/f37 [0,4194304] 0 2024-03-21T10:08:50.730 INFO:tasks.workunit.client.1.smithi174.stdout:2/297: readlink d5/d25/d27/d2d/d45/d5b/d2f/d4b/l51 0 2024-03-21T10:08:50.732 INFO:tasks.workunit.client.0.smithi153.stdout:0/397: fsync d4/d16/d1f/f20 0 2024-03-21T10:08:50.734 INFO:tasks.workunit.client.1.smithi174.stdout:2/298: chown d5/l5e 2488 1 2024-03-21T10:08:50.735 INFO:tasks.workunit.client.1.smithi174.stdout:5/272: creat d1/d16/f5f x:0 0 0 2024-03-21T10:08:50.738 INFO:tasks.workunit.client.0.smithi153.stdout:5/500: dread d4/d5/df/f30 [0,4194304] 0 2024-03-21T10:08:50.743 INFO:tasks.workunit.client.1.smithi174.stdout:9/315: creat d2/d11/d1d/d4b/f6e x:0 0 0 2024-03-21T10:08:50.746 INFO:tasks.workunit.client.1.smithi174.stdout:2/299: creat d5/d25/d27/d2e/d4f/f6d x:0 0 0 2024-03-21T10:08:50.748 INFO:tasks.workunit.client.1.smithi174.stdout:5/273: creat d1/d16/f60 x:0 0 0 2024-03-21T10:08:50.752 INFO:tasks.workunit.client.1.smithi174.stdout:1/280: dwrite d4/d7/f5b [0,4194304] 0 2024-03-21T10:08:50.754 INFO:tasks.workunit.client.1.smithi174.stdout:9/316: creat d2/d11/d60/f6f x:0 0 0 2024-03-21T10:08:50.755 INFO:tasks.workunit.client.1.smithi174.stdout:2/300: chown d5/l36 5132470 1 2024-03-21T10:08:50.761 INFO:tasks.workunit.client.1.smithi174.stdout:6/215: dwrite d0/d3/d5/df/d16/f2b [0,4194304] 0 2024-03-21T10:08:50.763 INFO:tasks.workunit.client.1.smithi174.stdout:9/317: dread - d2/f3c zero size 2024-03-21T10:08:50.763 INFO:tasks.workunit.client.1.smithi174.stdout:5/274: read d1/f2c [2950456,13321] 0 2024-03-21T10:08:50.765 INFO:tasks.workunit.client.1.smithi174.stdout:9/318: chown d2/d54 38 1 2024-03-21T10:08:50.765 INFO:tasks.workunit.client.1.smithi174.stdout:6/216: stat d0/d3/d5/df/d16/f1a 0 2024-03-21T10:08:50.766 INFO:tasks.workunit.client.1.smithi174.stdout:3/243: dwrite d0/f52 [0,4194304] 0 2024-03-21T10:08:50.787 INFO:tasks.workunit.client.0.smithi153.stdout:1/390: creat d6/d13/f79 x:0 0 0 2024-03-21T10:08:50.790 INFO:tasks.workunit.client.1.smithi174.stdout:3/244: dwrite d0/d12/d14/d1a/f34 [0,4194304] 0 2024-03-21T10:08:50.798 INFO:tasks.workunit.client.0.smithi153.stdout:8/477: link d5/d9/l94 d5/d9/d43/dd/d2d/d45/d88/d85/l9e 0 2024-03-21T10:08:50.808 INFO:tasks.workunit.client.0.smithi153.stdout:2/507: mknod dd/d3b/d50/d58/ca9 0 2024-03-21T10:08:50.813 INFO:tasks.workunit.client.0.smithi153.stdout:0/398: rmdir d4/d16/d19/d30/d6c/d6d 0 2024-03-21T10:08:50.819 INFO:tasks.workunit.client.0.smithi153.stdout:6/649: getdents d1f/d93/dd8/d59/d70 0 2024-03-21T10:08:50.819 INFO:tasks.workunit.client.0.smithi153.stdout:0/399: readlink d4/d16/l4b 0 2024-03-21T10:08:50.821 INFO:tasks.workunit.client.0.smithi153.stdout:2/508: dread - dd/d15/d18/d2c/f41 zero size 2024-03-21T10:08:50.822 INFO:tasks.workunit.client.0.smithi153.stdout:1/391: getdents d6/d13/d75 0 2024-03-21T10:08:50.822 INFO:tasks.workunit.client.0.smithi153.stdout:2/509: readlink dd/d15/d18/l28 0 2024-03-21T10:08:50.823 INFO:tasks.workunit.client.1.smithi174.stdout:1/281: mkdir d4/d7/d68 0 2024-03-21T10:08:50.823 INFO:tasks.workunit.client.0.smithi153.stdout:2/510: readlink dd/d15/l21 0 2024-03-21T10:08:50.825 INFO:tasks.workunit.client.0.smithi153.stdout:6/650: readlink d1f/d93/dd8/d59/la1 0 2024-03-21T10:08:50.827 INFO:tasks.workunit.client.0.smithi153.stdout:6/651: chown d1f/f20 496 1 2024-03-21T10:08:50.830 INFO:tasks.workunit.client.0.smithi153.stdout:8/478: getdents d5/d9/d43 0 2024-03-21T10:08:50.831 INFO:tasks.workunit.client.1.smithi174.stdout:9/319: mknod d2/d24/d2e/c70 0 2024-03-21T10:08:50.832 INFO:tasks.workunit.client.1.smithi174.stdout:3/245: fdatasync d0/d12/d14/d1a/d27/f2c 0 2024-03-21T10:08:50.834 INFO:tasks.workunit.client.1.smithi174.stdout:0/262: write d9/f15 [4557701,77771] 0 2024-03-21T10:08:50.834 INFO:tasks.workunit.client.1.smithi174.stdout:9/320: read d2/d11/f4a [1394574,43853] 0 2024-03-21T10:08:50.836 INFO:tasks.workunit.client.1.smithi174.stdout:2/301: creat d5/d25/d27/d2d/d45/f6e x:0 0 0 2024-03-21T10:08:50.837 INFO:tasks.workunit.client.1.smithi174.stdout:2/302: stat d5/c6 0 2024-03-21T10:08:50.839 INFO:tasks.workunit.client.0.smithi153.stdout:7/374: rename d4/d14/d2f/d78/c4e to d4/d1b/c80 0 2024-03-21T10:08:50.843 INFO:tasks.workunit.client.0.smithi153.stdout:5/501: dread d4/d8/f1d [0,4194304] 0 2024-03-21T10:08:50.843 INFO:tasks.workunit.client.0.smithi153.stdout:1/392: dwrite d6/d13/d75/f62 [0,4194304] 0 2024-03-21T10:08:50.844 INFO:tasks.workunit.client.0.smithi153.stdout:2/511: dread dd/d15/d18/d2c/d33/f7d [0,4194304] 0 2024-03-21T10:08:50.845 INFO:tasks.workunit.client.0.smithi153.stdout:5/502: dread - d4/d5/d16/f8a zero size 2024-03-21T10:08:50.848 INFO:tasks.workunit.client.1.smithi174.stdout:9/321: fsync d2/f1b 0 2024-03-21T10:08:50.849 INFO:tasks.workunit.client.1.smithi174.stdout:3/246: symlink d0/d12/d46/l5b 0 2024-03-21T10:08:50.850 INFO:tasks.workunit.client.0.smithi153.stdout:6/652: dwrite f11 [0,4194304] 0 2024-03-21T10:08:50.850 INFO:tasks.workunit.client.0.smithi153.stdout:9/461: rename d1/d3b/f86 to d1/d43/d44/d46/d9e/d24/fa1 0 2024-03-21T10:08:50.852 INFO:tasks.workunit.client.1.smithi174.stdout:6/217: link d0/d3/d5/df/d16/f27 d0/d3/d5/f45 0 2024-03-21T10:08:50.853 INFO:tasks.workunit.client.1.smithi174.stdout:3/247: write d0/d12/d14/d1a/f1e [1612469,80363] 0 2024-03-21T10:08:50.854 INFO:tasks.workunit.client.0.smithi153.stdout:9/462: dread - d1/d4e/d68/f81 zero size 2024-03-21T10:08:50.857 INFO:tasks.workunit.client.0.smithi153.stdout:1/393: chown d6/d10/d35/l4a 244 1 2024-03-21T10:08:50.858 INFO:tasks.workunit.client.1.smithi174.stdout:3/248: write d0/f7 [1952584,52106] 0 2024-03-21T10:08:50.859 INFO:tasks.workunit.client.0.smithi153.stdout:1/394: fdatasync d6/d10/d2c/f47 0 2024-03-21T10:08:50.861 INFO:tasks.workunit.client.0.smithi153.stdout:6/653: mknod d1f/d2b/d33/d40/d7b/ce8 0 2024-03-21T10:08:50.864 INFO:tasks.workunit.client.0.smithi153.stdout:2/512: creat dd/d15/d18/d3a/d8c/faa x:0 0 0 2024-03-21T10:08:50.868 INFO:tasks.workunit.client.1.smithi174.stdout:3/249: mkdir d0/d12/d14/d1a/d5c 0 2024-03-21T10:08:50.874 INFO:tasks.workunit.client.1.smithi174.stdout:0/263: dwrite d9/f54 [0,4194304] 0 2024-03-21T10:08:50.876 INFO:tasks.workunit.client.1.smithi174.stdout:0/264: chown d9/d21/f2b 8074 1 2024-03-21T10:08:50.878 INFO:tasks.workunit.client.1.smithi174.stdout:4/306: sync 2024-03-21T10:08:50.879 INFO:tasks.workunit.client.0.smithi153.stdout:0/400: rename d4/d16/l4b to d4/l7c 0 2024-03-21T10:08:50.879 INFO:tasks.workunit.client.1.smithi174.stdout:3/250: rename d0/d12/d14/d1a/d27/d4f/d55 to d0/d12/d14/d1a/d27/d4f/d5d 0 2024-03-21T10:08:50.881 INFO:tasks.workunit.client.0.smithi153.stdout:0/401: fdatasync d4/d16/d3e/d5e/f78 0 2024-03-21T10:08:50.881 INFO:tasks.workunit.client.0.smithi153.stdout:9/463: dread d1/d4/fc [0,4194304] 0 2024-03-21T10:08:50.883 INFO:tasks.workunit.client.1.smithi174.stdout:2/303: dwrite d5/f24 [0,4194304] 0 2024-03-21T10:08:50.883 INFO:tasks.workunit.client.1.smithi174.stdout:1/282: truncate d4/d7/d33/d43/f47 2861957 0 2024-03-21T10:08:50.886 INFO:tasks.workunit.client.1.smithi174.stdout:4/307: rename d4/f5 to d4/d19/f65 0 2024-03-21T10:08:50.886 INFO:tasks.workunit.client.0.smithi153.stdout:6/654: rename d1f/d2b/f53 to d1f/d2b/d33/d40/d65/fe9 0 2024-03-21T10:08:50.886 INFO:tasks.workunit.client.0.smithi153.stdout:2/513: dwrite dd/d15/d18/d2c/d4b/d56/f85 [0,4194304] 0 2024-03-21T10:08:50.889 INFO:tasks.workunit.client.0.smithi153.stdout:8/479: rmdir d5/d9/d43/dd/d20/d26/d24/d52 39 2024-03-21T10:08:50.892 INFO:tasks.workunit.client.1.smithi174.stdout:3/251: mknod d0/d12/d14/c5e 0 2024-03-21T10:08:50.894 INFO:tasks.workunit.client.1.smithi174.stdout:9/322: dwrite d2/f1e [0,4194304] 0 2024-03-21T10:08:50.894 INFO:tasks.workunit.client.0.smithi153.stdout:2/514: write dd/d15/d18/d2c/f35 [49041,124628] 0 2024-03-21T10:08:50.897 INFO:tasks.workunit.client.0.smithi153.stdout:8/480: rmdir d5/d9/d43/dd/d20/d26/d8b 39 2024-03-21T10:08:50.897 INFO:tasks.workunit.client.1.smithi174.stdout:0/265: dread d9/d21/d31/d3c/f49 [0,4194304] 0 2024-03-21T10:08:50.902 INFO:tasks.workunit.client.0.smithi153.stdout:0/402: rename d4/db/f1b to d4/d16/d1f/d22/f7d 0 2024-03-21T10:08:50.905 INFO:tasks.workunit.client.0.smithi153.stdout:1/395: rmdir d6/d10/d2c 39 2024-03-21T10:08:50.905 INFO:tasks.workunit.client.1.smithi174.stdout:9/323: dread - d2/f45 zero size 2024-03-21T10:08:50.908 INFO:tasks.workunit.client.1.smithi174.stdout:0/266: mkdir d9/d35/d4f/d56 0 2024-03-21T10:08:50.909 INFO:tasks.workunit.client.0.smithi153.stdout:2/515: creat dd/d15/fab x:0 0 0 2024-03-21T10:08:50.909 INFO:tasks.workunit.client.0.smithi153.stdout:0/403: mknod d4/d16/d1f/d22/d61/c7e 0 2024-03-21T10:08:50.911 INFO:tasks.workunit.client.1.smithi174.stdout:2/304: dwrite d5/d25/d27/f3b [4194304,4194304] 0 2024-03-21T10:08:50.915 INFO:tasks.workunit.client.0.smithi153.stdout:8/481: link d5/d9/d43/f2b d5/d9/d43/dd/f9f 0 2024-03-21T10:08:50.917 INFO:tasks.workunit.client.0.smithi153.stdout:6/655: rename d1f/de0/le2 to d1f/d93/lea 0 2024-03-21T10:08:50.920 INFO:tasks.workunit.client.0.smithi153.stdout:1/396: dread d6/fe [0,4194304] 0 2024-03-21T10:08:50.921 INFO:tasks.workunit.client.1.smithi174.stdout:4/308: dwrite d4/d21/d24/d28/f4e [0,4194304] 0 2024-03-21T10:08:50.922 INFO:tasks.workunit.client.0.smithi153.stdout:9/464: rename d1/c18 to d1/d43/d44/d46/d9e/d61/ca2 0 2024-03-21T10:08:50.922 INFO:tasks.workunit.client.1.smithi174.stdout:0/267: creat d9/d21/d31/f57 x:0 0 0 2024-03-21T10:08:50.925 INFO:tasks.workunit.client.1.smithi174.stdout:9/324: rmdir d2/d11/d1d/d5e 39 2024-03-21T10:08:50.926 INFO:tasks.workunit.client.0.smithi153.stdout:6/656: write d1f/d2b/d33/d40/d4f/f83 [811952,125998] 0 2024-03-21T10:08:50.926 INFO:tasks.workunit.client.1.smithi174.stdout:2/305: fsync d5/d25/d27/d2d/d45/d5b/d55/f58 0 2024-03-21T10:08:50.926 INFO:tasks.workunit.client.0.smithi153.stdout:0/404: truncate d4/f6 3686066 0 2024-03-21T10:08:50.928 INFO:tasks.workunit.client.1.smithi174.stdout:4/309: creat d4/d21/f66 x:0 0 0 2024-03-21T10:08:50.928 INFO:tasks.workunit.client.1.smithi174.stdout:9/325: fsync d2/f17 0 2024-03-21T10:08:50.929 INFO:tasks.workunit.client.0.smithi153.stdout:8/482: rename d5/d9/d43/dd/f9f to d5/d9/d43/fa0 0 2024-03-21T10:08:50.930 INFO:tasks.workunit.client.1.smithi174.stdout:0/268: mkdir d9/d58 0 2024-03-21T10:08:50.933 INFO:tasks.workunit.client.0.smithi153.stdout:1/397: truncate d6/d32/d3e/f4b 5035175 0 2024-03-21T10:08:50.935 INFO:tasks.workunit.client.1.smithi174.stdout:0/269: mknod d9/d21/c59 0 2024-03-21T10:08:50.936 INFO:tasks.workunit.client.0.smithi153.stdout:5/503: mkdir d4/d5/d9e 0 2024-03-21T10:08:50.940 INFO:tasks.workunit.client.1.smithi174.stdout:1/283: dwrite d4/fb [4194304,4194304] 0 2024-03-21T10:08:50.941 INFO:tasks.workunit.client.0.smithi153.stdout:9/465: creat d1/d43/d44/d46/d9e/d24/d41/fa3 x:0 0 0 2024-03-21T10:08:50.941 INFO:tasks.workunit.client.0.smithi153.stdout:2/516: dwrite dd/d15/d18/d2c/d33/f9d [0,4194304] 0 2024-03-21T10:08:50.941 INFO:tasks.workunit.client.0.smithi153.stdout:8/483: write d5/f2f [10252588,27165] 0 2024-03-21T10:08:50.946 INFO:tasks.workunit.client.0.smithi153.stdout:8/484: readlink d5/d9/d43/dd/d20/l83 0 2024-03-21T10:08:50.948 INFO:tasks.workunit.client.1.smithi174.stdout:4/310: dwrite d4/f3e [0,4194304] 0 2024-03-21T10:08:50.948 INFO:tasks.workunit.client.0.smithi153.stdout:6/657: creat d1f/d99/dbb/dcf/feb x:0 0 0 2024-03-21T10:08:50.950 INFO:tasks.workunit.client.1.smithi174.stdout:1/284: creat d4/d7/d15/d4f/f69 x:0 0 0 2024-03-21T10:08:50.952 INFO:tasks.workunit.client.0.smithi153.stdout:9/466: unlink d1/d43/d44/d46/c17 0 2024-03-21T10:08:50.953 INFO:tasks.workunit.client.1.smithi174.stdout:1/285: write d4/d2d/f34 [2045008,17992] 0 2024-03-21T10:08:50.954 INFO:tasks.workunit.client.0.smithi153.stdout:5/504: write d4/d5/df/f30 [3428235,11383] 0 2024-03-21T10:08:50.958 INFO:tasks.workunit.client.0.smithi153.stdout:7/375: dwrite d4/d14/d2f/d78/f35 [0,4194304] 0 2024-03-21T10:08:50.958 INFO:tasks.workunit.client.1.smithi174.stdout:0/270: dwrite d9/f1d [0,4194304] 0 2024-03-21T10:08:50.961 INFO:tasks.workunit.client.0.smithi153.stdout:6/658: creat d1f/d99/fec x:0 0 0 2024-03-21T10:08:50.966 INFO:tasks.workunit.client.0.smithi153.stdout:9/467: truncate d1/d4e/f53 1319073 0 2024-03-21T10:08:50.968 INFO:tasks.workunit.client.0.smithi153.stdout:7/376: write d4/d2a/f63 [477939,96491] 0 2024-03-21T10:08:50.969 INFO:tasks.workunit.client.0.smithi153.stdout:0/405: dwrite d4/d16/d1f/f35 [0,4194304] 0 2024-03-21T10:08:50.969 INFO:tasks.workunit.client.1.smithi174.stdout:0/271: unlink d9/d21/d31/c52 0 2024-03-21T10:08:50.972 INFO:tasks.workunit.client.0.smithi153.stdout:3/414: sync 2024-03-21T10:08:50.973 INFO:tasks.workunit.client.1.smithi174.stdout:0/272: mknod d9/d1c/c5a 0 2024-03-21T10:08:50.975 INFO:tasks.workunit.client.0.smithi153.stdout:4/467: sync 2024-03-21T10:08:50.977 INFO:tasks.workunit.client.0.smithi153.stdout:8/485: creat d5/d9/d43/dd/d20/d26/d8b/d80/fa1 x:0 0 0 2024-03-21T10:08:50.981 INFO:tasks.workunit.client.0.smithi153.stdout:2/517: dread dd/f13 [0,4194304] 0 2024-03-21T10:08:50.995 INFO:tasks.workunit.client.0.smithi153.stdout:5/505: dwrite d4/d5/f1a [4194304,4194304] 0 2024-03-21T10:08:50.997 INFO:tasks.workunit.client.1.smithi174.stdout:4/311: dread d4/d19/f41 [0,4194304] 0 2024-03-21T10:08:51.002 INFO:tasks.workunit.client.0.smithi153.stdout:0/406: truncate d4/f37 478880 0 2024-03-21T10:08:51.002 INFO:tasks.workunit.client.1.smithi174.stdout:4/312: mkdir d4/d21/d31/d3a/d67 0 2024-03-21T10:08:51.004 INFO:tasks.workunit.client.0.smithi153.stdout:0/407: readlink d4/db/ld 0 2024-03-21T10:08:51.005 INFO:tasks.workunit.client.1.smithi174.stdout:4/313: truncate d4/d21/f66 883679 0 2024-03-21T10:08:51.010 INFO:tasks.workunit.client.0.smithi153.stdout:0/408: dread - d4/d16/d19/d58/f6a zero size 2024-03-21T10:08:51.010 INFO:tasks.workunit.client.0.smithi153.stdout:2/518: mknod dd/d15/d18/d2c/d33/cac 0 2024-03-21T10:08:51.012 INFO:tasks.workunit.client.1.smithi174.stdout:2/306: write d5/d25/d27/d2d/d45/d5b/f1c [865616,115230] 0 2024-03-21T10:08:51.012 INFO:tasks.workunit.client.0.smithi153.stdout:1/398: dread d6/f1c [0,4194304] 0 2024-03-21T10:08:51.013 INFO:tasks.workunit.client.0.smithi153.stdout:6/659: rmdir d1f/d93/ddf 0 2024-03-21T10:08:51.013 INFO:tasks.workunit.client.1.smithi174.stdout:4/314: symlink d4/d21/d31/d3a/l68 0 2024-03-21T10:08:51.014 INFO:tasks.workunit.client.0.smithi153.stdout:8/486: dwrite d5/d9/d43/f77 [0,4194304] 0 2024-03-21T10:08:51.015 INFO:tasks.workunit.client.0.smithi153.stdout:0/409: write d4/d16/d3e/f69 [270820,127083] 0 2024-03-21T10:08:51.017 INFO:tasks.workunit.client.0.smithi153.stdout:6/660: chown f1b 113 1 2024-03-21T10:08:51.019 INFO:tasks.workunit.client.0.smithi153.stdout:5/506: dread d4/d8/f11 [4194304,4194304] 0 2024-03-21T10:08:51.020 INFO:tasks.workunit.client.1.smithi174.stdout:1/286: dread d4/d7/ff [0,4194304] 0 2024-03-21T10:08:51.022 INFO:tasks.workunit.client.0.smithi153.stdout:9/468: dread d1/d43/d44/d46/f26 [0,4194304] 0 2024-03-21T10:08:51.024 INFO:tasks.workunit.client.1.smithi174.stdout:9/326: dwrite d2/f20 [4194304,4194304] 0 2024-03-21T10:08:51.024 INFO:tasks.workunit.client.1.smithi174.stdout:0/273: dread f7 [0,4194304] 0 2024-03-21T10:08:51.025 INFO:tasks.workunit.client.0.smithi153.stdout:3/415: rmdir dd/d1e/d3d/d94/d93 0 2024-03-21T10:08:51.025 INFO:tasks.workunit.client.0.smithi153.stdout:9/469: fsync d1/d4/f75 0 2024-03-21T10:08:51.028 INFO:tasks.workunit.client.1.smithi174.stdout:9/327: chown d2/c40 5863051 1 2024-03-21T10:08:51.028 INFO:tasks.workunit.client.1.smithi174.stdout:7/206: sync 2024-03-21T10:08:51.029 INFO:tasks.workunit.client.0.smithi153.stdout:3/416: chown dd/f15 8 1 2024-03-21T10:08:51.029 INFO:tasks.workunit.client.0.smithi153.stdout:1/399: rmdir d6/d13/d75 39 2024-03-21T10:08:51.030 INFO:tasks.workunit.client.1.smithi174.stdout:0/274: truncate d9/d21/d31/f47 932308 0 2024-03-21T10:08:51.032 INFO:tasks.workunit.client.1.smithi174.stdout:9/328: rmdir d2/d24 39 2024-03-21T10:08:51.033 INFO:tasks.workunit.client.0.smithi153.stdout:6/661: write d1f/d2b/fb5 [2091459,129047] 0 2024-03-21T10:08:51.034 INFO:tasks.workunit.client.1.smithi174.stdout:1/287: link d4/d7/d15/d4f/f69 d4/d7/d15/d53/f6a 0 2024-03-21T10:08:51.035 INFO:tasks.workunit.client.0.smithi153.stdout:4/468: getdents d11 0 2024-03-21T10:08:51.036 INFO:tasks.workunit.client.1.smithi174.stdout:4/315: dread d4/fc [0,4194304] 0 2024-03-21T10:08:51.037 INFO:tasks.workunit.client.0.smithi153.stdout:3/417: rmdir dd/d1e/d2f/d35/d68 39 2024-03-21T10:08:51.038 INFO:tasks.workunit.client.0.smithi153.stdout:4/469: stat d11/d18/d50 0 2024-03-21T10:08:51.039 INFO:tasks.workunit.client.1.smithi174.stdout:7/207: symlink d0/d4/d38/l48 0 2024-03-21T10:08:51.043 INFO:tasks.workunit.client.1.smithi174.stdout:0/275: read d9/f17 [4136618,52152] 0 2024-03-21T10:08:51.043 INFO:tasks.workunit.client.1.smithi174.stdout:1/288: mkdir d4/d7/d15/d53/d6b 0 2024-03-21T10:08:51.044 INFO:tasks.workunit.client.0.smithi153.stdout:2/519: rename ca to dd/d15/cad 0 2024-03-21T10:08:51.044 INFO:tasks.workunit.client.1.smithi174.stdout:9/329: chown d2/d24/d2e/f58 47904 1 2024-03-21T10:08:51.047 INFO:tasks.workunit.client.0.smithi153.stdout:0/410: truncate d4/f28 1208327 0 2024-03-21T10:08:51.048 INFO:tasks.workunit.client.0.smithi153.stdout:3/418: read dd/d1e/d2f/f3b [927907,63229] 0 2024-03-21T10:08:51.048 INFO:tasks.workunit.client.1.smithi174.stdout:1/289: rmdir d4 39 2024-03-21T10:08:51.048 INFO:tasks.workunit.client.1.smithi174.stdout:7/208: read d0/d4/d10/f45 [2821869,80695] 0 2024-03-21T10:08:51.049 INFO:tasks.workunit.client.0.smithi153.stdout:4/470: unlink d11/d2d/c75 0 2024-03-21T10:08:51.050 INFO:tasks.workunit.client.1.smithi174.stdout:0/276: creat d9/d21/d31/d3c/f5b x:0 0 0 2024-03-21T10:08:51.051 INFO:tasks.workunit.client.1.smithi174.stdout:8/220: sync 2024-03-21T10:08:51.051 INFO:tasks.workunit.client.1.smithi174.stdout:5/275: sync 2024-03-21T10:08:51.054 INFO:tasks.workunit.client.1.smithi174.stdout:8/221: stat d9/l2d 0 2024-03-21T10:08:51.055 INFO:tasks.workunit.client.1.smithi174.stdout:6/218: sync 2024-03-21T10:08:51.056 INFO:tasks.workunit.client.1.smithi174.stdout:3/252: sync 2024-03-21T10:08:51.056 INFO:tasks.workunit.client.1.smithi174.stdout:1/290: write d4/f19 [2098299,129177] 0 2024-03-21T10:08:51.056 INFO:tasks.workunit.client.1.smithi174.stdout:5/276: readlink d1/d16/d2e/l38 0 2024-03-21T10:08:51.058 INFO:tasks.workunit.client.0.smithi153.stdout:9/470: dwrite d1/d4e/f53 [0,4194304] 0 2024-03-21T10:08:51.060 INFO:tasks.workunit.client.1.smithi174.stdout:9/330: rmdir d2/d11/d60 39 2024-03-21T10:08:51.061 INFO:tasks.workunit.client.0.smithi153.stdout:9/471: write d1/f1b [1328767,47968] 0 2024-03-21T10:08:51.061 INFO:tasks.workunit.client.0.smithi153.stdout:3/419: unlink dd/f2d 0 2024-03-21T10:08:51.061 INFO:tasks.workunit.client.0.smithi153.stdout:1/400: dread d6/f17 [0,4194304] 0 2024-03-21T10:08:51.064 INFO:tasks.workunit.client.1.smithi174.stdout:4/316: dwrite d4/d21/d24/d28/d38/f5f [0,4194304] 0 2024-03-21T10:08:51.065 INFO:tasks.workunit.client.1.smithi174.stdout:0/277: mkdir d9/d21/d31/d5c 0 2024-03-21T10:08:51.067 INFO:tasks.workunit.client.0.smithi153.stdout:5/507: dread d4/d5/f57 [0,4194304] 0 2024-03-21T10:08:51.069 INFO:tasks.workunit.client.0.smithi153.stdout:4/471: mkdir d11/d71/d89 0 2024-03-21T10:08:51.073 INFO:tasks.workunit.client.1.smithi174.stdout:3/253: creat d0/f5f x:0 0 0 2024-03-21T10:08:51.077 INFO:tasks.workunit.client.1.smithi174.stdout:8/222: rmdir d9/d2a/d39 39 2024-03-21T10:08:51.078 INFO:tasks.workunit.client.0.smithi153.stdout:7/377: dread d4/fa [4194304,4194304] 0 2024-03-21T10:08:51.078 INFO:tasks.workunit.client.0.smithi153.stdout:8/487: dread d5/d9/d43/f35 [0,4194304] 0 2024-03-21T10:08:51.079 INFO:tasks.workunit.client.1.smithi174.stdout:8/223: chown d9/d17/d19/d31 7440453 1 2024-03-21T10:08:51.079 INFO:tasks.workunit.client.1.smithi174.stdout:1/291: dwrite d4/d2d/f5e [0,4194304] 0 2024-03-21T10:08:51.080 INFO:tasks.workunit.client.1.smithi174.stdout:6/219: unlink d0/l32 0 2024-03-21T10:08:51.084 INFO:tasks.workunit.client.0.smithi153.stdout:5/508: mkdir d4/d8/d5a/d92/d9f 0 2024-03-21T10:08:51.085 INFO:tasks.workunit.client.1.smithi174.stdout:2/307: truncate d5/f12 2673319 0 2024-03-21T10:08:51.085 INFO:tasks.workunit.client.1.smithi174.stdout:6/220: truncate d0/d3/d5/f1c 4384942 0 2024-03-21T10:08:51.086 INFO:tasks.workunit.client.0.smithi153.stdout:7/378: write d4/d14/d16/f39 [1199634,50303] 0 2024-03-21T10:08:51.087 INFO:tasks.workunit.client.0.smithi153.stdout:3/420: creat dd/d1e/d3d/d94/d73/d7f/f95 x:0 0 0 2024-03-21T10:08:51.088 INFO:tasks.workunit.client.0.smithi153.stdout:5/509: truncate d4/d5/d16/d24/f53 4338591 0 2024-03-21T10:08:51.089 INFO:tasks.workunit.client.0.smithi153.stdout:7/379: write d4/d2a/f5b [487248,82562] 0 2024-03-21T10:08:51.090 INFO:tasks.workunit.client.1.smithi174.stdout:7/209: write d0/d4/f43 [804659,17329] 0 2024-03-21T10:08:51.091 INFO:tasks.workunit.client.0.smithi153.stdout:5/510: chown d4/d8/d5c/d90 51075 1 2024-03-21T10:08:51.091 INFO:tasks.workunit.client.1.smithi174.stdout:2/308: chown d5/d25/d27/d2e/f3f 393169632 1 2024-03-21T10:08:51.095 INFO:tasks.workunit.client.1.smithi174.stdout:7/210: chown d0/d4/d10/c1a 1729913 1 2024-03-21T10:08:51.095 INFO:tasks.workunit.client.1.smithi174.stdout:3/254: dread d0/fa [0,4194304] 0 2024-03-21T10:08:51.095 INFO:tasks.workunit.client.1.smithi174.stdout:9/331: write d2/d11/d60/f64 [308892,115213] 0 2024-03-21T10:08:51.095 INFO:tasks.workunit.client.1.smithi174.stdout:4/317: creat d4/d21/d24/d28/f69 x:0 0 0 2024-03-21T10:08:51.101 INFO:tasks.workunit.client.0.smithi153.stdout:8/488: link d5/d9/d43/dd/d20/d26/d8b/f4e d5/d9/d43/d7b/fa2 0 2024-03-21T10:08:51.102 INFO:tasks.workunit.client.1.smithi174.stdout:4/318: write d4/f46 [784890,38592] 0 2024-03-21T10:08:51.104 INFO:tasks.workunit.client.0.smithi153.stdout:5/511: read d4/d8/d3b/f88 [6779918,42855] 0 2024-03-21T10:08:51.105 INFO:tasks.workunit.client.0.smithi153.stdout:3/421: creat dd/d1e/d2f/d35/d68/f96 x:0 0 0 2024-03-21T10:08:51.109 INFO:tasks.workunit.client.1.smithi174.stdout:4/319: truncate d4/d19/f23 4243087 0 2024-03-21T10:08:51.109 INFO:tasks.workunit.client.1.smithi174.stdout:4/320: write d4/d21/f37 [824389,45019] 0 2024-03-21T10:08:51.109 INFO:tasks.workunit.client.1.smithi174.stdout:9/332: symlink d2/d11/d1d/d4b/l71 0 2024-03-21T10:08:51.110 INFO:tasks.workunit.client.1.smithi174.stdout:8/224: rmdir d9/d2a/d39 39 2024-03-21T10:08:51.115 INFO:tasks.workunit.client.1.smithi174.stdout:7/211: read d0/d4/f12 [2231551,57678] 0 2024-03-21T10:08:51.117 INFO:tasks.workunit.client.1.smithi174.stdout:7/212: write d0/d4/f22 [426790,22954] 0 2024-03-21T10:08:51.118 INFO:tasks.workunit.client.0.smithi153.stdout:3/422: write dd/d1e/d45/d56/f79 [164500,5598] 0 2024-03-21T10:08:51.119 INFO:tasks.workunit.client.1.smithi174.stdout:6/221: creat d0/d3/d5/d3b/f46 x:0 0 0 2024-03-21T10:08:51.121 INFO:tasks.workunit.client.0.smithi153.stdout:1/401: dwrite d6/d10/d2c/f33 [0,4194304] 0 2024-03-21T10:08:51.122 INFO:tasks.workunit.client.0.smithi153.stdout:3/423: readlink dd/d1e/l59 0 2024-03-21T10:08:51.122 INFO:tasks.workunit.client.1.smithi174.stdout:0/278: link d9/d21/c2e d9/d35/d4f/d56/c5d 0 2024-03-21T10:08:51.123 INFO:tasks.workunit.client.0.smithi153.stdout:2/520: dwrite dd/f13 [0,4194304] 0 2024-03-21T10:08:51.123 INFO:tasks.workunit.client.0.smithi153.stdout:8/489: creat d5/d9/d43/dd/d20/d26/d24/fa3 x:0 0 0 2024-03-21T10:08:51.125 INFO:tasks.workunit.client.1.smithi174.stdout:9/333: chown d2/f62 104739 1 2024-03-21T10:08:51.125 INFO:tasks.workunit.client.0.smithi153.stdout:2/521: dread - dd/d15/fab zero size 2024-03-21T10:08:51.126 INFO:tasks.workunit.client.1.smithi174.stdout:0/279: write d9/d21/f37 [963818,67708] 0 2024-03-21T10:08:51.127 INFO:tasks.workunit.client.1.smithi174.stdout:9/334: chown d2/d11/d60/f6f 8648 1 2024-03-21T10:08:51.130 INFO:tasks.workunit.client.1.smithi174.stdout:0/280: write d9/d21/f37 [874511,65468] 0 2024-03-21T10:08:51.132 INFO:tasks.workunit.client.1.smithi174.stdout:3/255: symlink d0/d12/d14/d1a/d44/d57/l60 0 2024-03-21T10:08:51.134 INFO:tasks.workunit.client.1.smithi174.stdout:7/213: mknod d0/d4/d10/d1b/c49 0 2024-03-21T10:08:51.135 INFO:tasks.workunit.client.0.smithi153.stdout:4/472: dread d11/d2d/d3d/f45 [0,4194304] 0 2024-03-21T10:08:51.135 INFO:tasks.workunit.client.0.smithi153.stdout:0/411: dread d4/d16/d1f/f20 [0,4194304] 0 2024-03-21T10:08:51.136 INFO:tasks.workunit.client.0.smithi153.stdout:3/424: mknod dd/c97 0 2024-03-21T10:08:51.137 INFO:tasks.workunit.client.1.smithi174.stdout:5/277: getdents d1/d16/d1c 0 2024-03-21T10:08:51.138 INFO:tasks.workunit.client.0.smithi153.stdout:9/472: dread d1/d43/d44/f58 [0,4194304] 0 2024-03-21T10:08:51.140 INFO:tasks.workunit.client.0.smithi153.stdout:8/490: stat d5/d9/d43/dd/d20/d26/d24/d52/d56/f6f 0 2024-03-21T10:08:51.142 INFO:tasks.workunit.client.1.smithi174.stdout:7/214: chown d0/d4/d38/l48 187 1 2024-03-21T10:08:51.143 INFO:tasks.workunit.client.1.smithi174.stdout:8/225: rename d9/l26 to d9/d17/d19/d27/l4c 0 2024-03-21T10:08:51.145 INFO:tasks.workunit.client.0.smithi153.stdout:2/522: rename dd/d3b to dd/d15/d3c/da7/d91/dae 0 2024-03-21T10:08:51.148 INFO:tasks.workunit.client.1.smithi174.stdout:1/292: truncate d4/d2d/f5e 3866948 0 2024-03-21T10:08:51.149 INFO:tasks.workunit.client.1.smithi174.stdout:9/335: mkdir d2/d11/d1d/d4b/d72 0 2024-03-21T10:08:51.153 INFO:tasks.workunit.client.0.smithi153.stdout:3/425: symlink dd/d1e/d3d/d94/d73/d7f/l98 0 2024-03-21T10:08:51.153 INFO:tasks.workunit.client.1.smithi174.stdout:6/222: rename d0/d3/l10 to d0/d3/d5/d3b/l47 0 2024-03-21T10:08:51.153 INFO:tasks.workunit.client.1.smithi174.stdout:2/309: dwrite d5/d25/d27/d2d/d45/d5b/ff [0,4194304] 0 2024-03-21T10:08:51.159 INFO:tasks.workunit.client.1.smithi174.stdout:7/215: symlink d0/d4/d38/l4a 0 2024-03-21T10:08:51.163 INFO:tasks.workunit.client.1.smithi174.stdout:0/281: link d9/fc d9/d35/d4a/f5e 0 2024-03-21T10:08:51.166 INFO:tasks.workunit.client.1.smithi174.stdout:3/256: dwrite d0/d12/d14/d1a/d27/f43 [0,4194304] 0 2024-03-21T10:08:51.166 INFO:tasks.workunit.client.1.smithi174.stdout:1/293: mkdir d4/d5/d6c 0 2024-03-21T10:08:51.170 INFO:tasks.workunit.client.1.smithi174.stdout:0/282: readlink d9/d1c/l30 0 2024-03-21T10:08:51.170 INFO:tasks.workunit.client.0.smithi153.stdout:0/412: dread d4/f10 [4194304,4194304] 0 2024-03-21T10:08:51.170 INFO:tasks.workunit.client.0.smithi153.stdout:1/402: dwrite d6/fe [4194304,4194304] 0 2024-03-21T10:08:51.172 INFO:tasks.workunit.client.1.smithi174.stdout:0/283: stat d9/d1c/c5a 0 2024-03-21T10:08:51.172 INFO:tasks.workunit.client.0.smithi153.stdout:4/473: unlink d11/d2d/d57/d85/c87 0 2024-03-21T10:08:51.175 INFO:tasks.workunit.client.0.smithi153.stdout:2/523: creat dd/faf x:0 0 0 2024-03-21T10:08:51.175 INFO:tasks.workunit.client.0.smithi153.stdout:7/380: dread d4/d14/d2f/d78/f28 [0,4194304] 0 2024-03-21T10:08:51.176 INFO:tasks.workunit.client.0.smithi153.stdout:8/491: creat d5/d9/d43/dd/d20/d26/d8b/fa4 x:0 0 0 2024-03-21T10:08:51.176 INFO:tasks.workunit.client.1.smithi174.stdout:3/257: chown d0/d12/d14/d1a/d27/d4f/d5d/f1f 195 1 2024-03-21T10:08:51.177 INFO:tasks.workunit.client.1.smithi174.stdout:4/321: dwrite d4/fc [0,4194304] 0 2024-03-21T10:08:51.179 INFO:tasks.workunit.client.1.smithi174.stdout:8/226: rename d9/d17/d19/d33 to d9/d2a/d4d 0 2024-03-21T10:08:51.180 INFO:tasks.workunit.client.1.smithi174.stdout:3/258: write d0/d12/d14/d1a/f3d [3710017,65035] 0 2024-03-21T10:08:51.182 INFO:tasks.workunit.client.0.smithi153.stdout:2/524: chown dd/d15/d18/d3a/c8a 55 1 2024-03-21T10:08:51.182 INFO:tasks.workunit.client.1.smithi174.stdout:8/227: chown d9/f15 1181 1 2024-03-21T10:08:51.183 INFO:tasks.workunit.client.0.smithi153.stdout:4/474: rename d11/d18/f4c to d11/d2d/d6b/f8a 0 2024-03-21T10:08:51.185 INFO:tasks.workunit.client.1.smithi174.stdout:5/278: dread d1/f2c [0,4194304] 0 2024-03-21T10:08:51.185 INFO:tasks.workunit.client.1.smithi174.stdout:7/216: dread d0/ff [0,4194304] 0 2024-03-21T10:08:51.186 INFO:tasks.workunit.client.0.smithi153.stdout:7/381: mkdir d4/d2a/d3b/d81 0 2024-03-21T10:08:51.186 INFO:tasks.workunit.client.0.smithi153.stdout:1/403: symlink d6/d32/d3e/d4f/l7a 0 2024-03-21T10:08:51.190 INFO:tasks.workunit.client.0.smithi153.stdout:3/426: creat dd/d1e/d45/f99 x:0 0 0 2024-03-21T10:08:51.190 INFO:tasks.workunit.client.1.smithi174.stdout:5/279: chown d1/d49/d4f/d51 13484802 1 2024-03-21T10:08:51.192 INFO:tasks.workunit.client.0.smithi153.stdout:4/475: symlink d11/d2d/d3d/l8b 0 2024-03-21T10:08:51.192 INFO:tasks.workunit.client.1.smithi174.stdout:4/322: mkdir d4/d21/d31/d6a 0 2024-03-21T10:08:51.193 INFO:tasks.workunit.client.0.smithi153.stdout:7/382: read d4/d14/d2f/d78/f35 [442953,102633] 0 2024-03-21T10:08:51.197 INFO:tasks.workunit.client.0.smithi153.stdout:3/427: chown dd/d1e/f4f 86 1 2024-03-21T10:08:51.198 INFO:tasks.workunit.client.1.smithi174.stdout:3/259: creat d0/d12/d14/d1a/d27/d3c/f61 x:0 0 0 2024-03-21T10:08:51.198 INFO:tasks.workunit.client.1.smithi174.stdout:0/284: dread d9/f54 [0,4194304] 0 2024-03-21T10:08:51.200 INFO:tasks.workunit.client.0.smithi153.stdout:8/492: dwrite d5/d9/d43/f35 [0,4194304] 0 2024-03-21T10:08:51.204 INFO:tasks.workunit.client.1.smithi174.stdout:9/336: rename d2/f20 to d2/d11/d1d/d2a/f73 0 2024-03-21T10:08:51.208 INFO:tasks.workunit.client.1.smithi174.stdout:3/260: write d0/d12/d14/d1a/d27/f43 [2128802,36535] 0 2024-03-21T10:08:51.208 INFO:tasks.workunit.client.1.smithi174.stdout:7/217: symlink d0/d4/l4b 0 2024-03-21T10:08:51.210 INFO:tasks.workunit.client.1.smithi174.stdout:4/323: mknod d4/d19/c6b 0 2024-03-21T10:08:51.214 INFO:tasks.workunit.client.1.smithi174.stdout:6/223: rename d0/d3/d5/df/l31 to d0/d3/l48 0 2024-03-21T10:08:51.221 INFO:tasks.workunit.client.1.smithi174.stdout:2/310: dread d5/f14 [4194304,4194304] 0 2024-03-21T10:08:51.223 INFO:tasks.workunit.client.1.smithi174.stdout:4/324: mkdir d4/d6c 0 2024-03-21T10:08:51.224 INFO:tasks.workunit.client.1.smithi174.stdout:5/280: dwrite d1/d49/d4f/f57 [0,4194304] 0 2024-03-21T10:08:51.226 INFO:tasks.workunit.client.1.smithi174.stdout:8/228: rename d9/d17/d19/d31/c49 to d9/d2a/d39/c4e 0 2024-03-21T10:08:51.226 INFO:tasks.workunit.client.1.smithi174.stdout:6/224: mknod d0/d3/d5/d6/d42/c49 0 2024-03-21T10:08:51.232 INFO:tasks.workunit.client.1.smithi174.stdout:4/325: write d4/d21/d24/d28/d38/f61 [340632,114156] 0 2024-03-21T10:08:51.232 INFO:tasks.workunit.client.1.smithi174.stdout:9/337: creat d2/d24/f74 x:0 0 0 2024-03-21T10:08:51.238 INFO:tasks.workunit.client.1.smithi174.stdout:3/261: dwrite d0/d12/f17 [0,4194304] 0 2024-03-21T10:08:51.242 INFO:tasks.workunit.client.1.smithi174.stdout:1/294: dwrite d4/d7/d15/f2f [0,4194304] 0 2024-03-21T10:08:51.246 INFO:tasks.workunit.client.1.smithi174.stdout:5/281: symlink d1/d2f/d37/l61 0 2024-03-21T10:08:51.251 INFO:tasks.workunit.client.1.smithi174.stdout:8/229: rename f7 to d9/d17/d19/d31/f4f 0 2024-03-21T10:08:51.261 INFO:tasks.workunit.client.1.smithi174.stdout:6/225: dwrite d0/d3/d5/d6/f13 [4194304,4194304] 0 2024-03-21T10:08:51.270 INFO:tasks.workunit.client.0.smithi153.stdout:7/383: symlink d4/d2a/d40/l82 0 2024-03-21T10:08:51.271 INFO:tasks.workunit.client.0.smithi153.stdout:3/428: rmdir dd/d1e/d2f/d35 39 2024-03-21T10:08:51.271 INFO:tasks.workunit.client.1.smithi174.stdout:2/311: write d5/d25/d27/f6a [589841,98894] 0 2024-03-21T10:08:51.272 INFO:tasks.workunit.client.0.smithi153.stdout:7/384: mkdir d4/d1b/d83 0 2024-03-21T10:08:51.272 INFO:tasks.workunit.client.0.smithi153.stdout:2/525: dwrite dd/d15/f40 [0,4194304] 0 2024-03-21T10:08:51.272 INFO:tasks.workunit.client.1.smithi174.stdout:8/230: creat d9/d2a/d4d/f50 x:0 0 0 2024-03-21T10:08:51.276 INFO:tasks.workunit.client.0.smithi153.stdout:9/473: dread d1/d43/d44/f5d [4194304,4194304] 0 2024-03-21T10:08:51.276 INFO:tasks.workunit.client.1.smithi174.stdout:2/312: write d5/d25/d27/d2d/d45/f6e [909429,40409] 0 2024-03-21T10:08:51.279 INFO:tasks.workunit.client.0.smithi153.stdout:8/493: fdatasync d5/d9/d43/dd/d20/d26/d24/d52/f64 0 2024-03-21T10:08:51.284 INFO:tasks.workunit.client.0.smithi153.stdout:1/404: dwrite d6/d10/d2c/f2e [0,4194304] 0 2024-03-21T10:08:51.285 INFO:tasks.workunit.client.1.smithi174.stdout:8/231: write d9/d2a/d4d/f50 [679948,28371] 0 2024-03-21T10:08:51.285 INFO:tasks.workunit.client.1.smithi174.stdout:3/262: truncate d0/d12/d14/d1a/f23 2555254 0 2024-03-21T10:08:51.288 INFO:tasks.workunit.client.0.smithi153.stdout:9/474: creat d1/d43/d44/d46/d9e/d4b/d84/fa4 x:0 0 0 2024-03-21T10:08:51.289 INFO:tasks.workunit.client.1.smithi174.stdout:6/226: link d0/d3/d20/f23 d0/d3/d5/d6/f4a 0 2024-03-21T10:08:51.290 INFO:tasks.workunit.client.1.smithi174.stdout:4/326: link d4/d21/d31/d3a/c4f d4/d21/d31/d3a/c6d 0 2024-03-21T10:08:51.293 INFO:tasks.workunit.client.1.smithi174.stdout:1/295: creat d4/d7/f6d x:0 0 0 2024-03-21T10:08:51.299 INFO:tasks.workunit.client.1.smithi174.stdout:1/296: fsync d4/d5/f18 0 2024-03-21T10:08:51.299 INFO:tasks.workunit.client.0.smithi153.stdout:3/429: dwrite dd/d1e/d45/f74 [0,4194304] 0 2024-03-21T10:08:51.303 INFO:tasks.workunit.client.0.smithi153.stdout:9/475: truncate d1/d43/d44/d46/d9e/d24/d41/f6a 958625 0 2024-03-21T10:08:51.304 INFO:tasks.workunit.client.1.smithi174.stdout:4/327: creat d4/d21/d31/d6a/f6e x:0 0 0 2024-03-21T10:08:51.304 INFO:tasks.workunit.client.0.smithi153.stdout:7/385: dwrite d4/d14/d2f/d78/f28 [0,4194304] 0 2024-03-21T10:08:51.305 INFO:tasks.workunit.client.1.smithi174.stdout:5/282: dwrite d1/d16/f1a [0,4194304] 0 2024-03-21T10:08:51.305 INFO:tasks.workunit.client.1.smithi174.stdout:2/313: creat d5/d25/d27/d2d/d45/d5b/f6f x:0 0 0 2024-03-21T10:08:51.305 INFO:tasks.workunit.client.0.smithi153.stdout:8/494: dwrite d5/d9/f8d [0,4194304] 0 2024-03-21T10:08:51.306 INFO:tasks.workunit.client.0.smithi153.stdout:1/405: dread d6/f17 [4194304,4194304] 0 2024-03-21T10:08:51.307 INFO:tasks.workunit.client.0.smithi153.stdout:7/386: dread - d4/f6c zero size 2024-03-21T10:08:51.308 INFO:tasks.workunit.client.0.smithi153.stdout:9/476: write d1/d43/d44/d46/d9e/d61/f87 [185995,99153] 0 2024-03-21T10:08:51.310 INFO:tasks.workunit.client.1.smithi174.stdout:3/263: creat d0/d12/d14/f62 x:0 0 0 2024-03-21T10:08:51.310 INFO:tasks.workunit.client.0.smithi153.stdout:1/406: chown d6/d3a/d3c/f43 1550 1 2024-03-21T10:08:51.310 INFO:tasks.workunit.client.0.smithi153.stdout:3/430: creat dd/d1e/d2f/d35/d68/f9a x:0 0 0 2024-03-21T10:08:51.311 INFO:tasks.workunit.client.1.smithi174.stdout:2/314: readlink d5/d25/d27/l32 0 2024-03-21T10:08:51.314 INFO:tasks.workunit.client.0.smithi153.stdout:7/387: rename d4/d14/d16/d79 to d4/d14/d2f/d78/d84 0 2024-03-21T10:08:51.315 INFO:tasks.workunit.client.1.smithi174.stdout:2/315: write d5/d25/d27/f2a [1302944,17289] 0 2024-03-21T10:08:51.316 INFO:tasks.workunit.client.0.smithi153.stdout:9/477: symlink d1/d43/d44/d46/d9e/d4b/d84/la5 0 2024-03-21T10:08:51.320 INFO:tasks.workunit.client.1.smithi174.stdout:4/328: mknod d4/d21/c6f 0 2024-03-21T10:08:51.320 INFO:tasks.workunit.client.1.smithi174.stdout:1/297: readlink d4/d7/d32/l4c 0 2024-03-21T10:08:51.320 INFO:tasks.workunit.client.0.smithi153.stdout:8/495: creat d5/d9/d43/dd/d2d/fa5 x:0 0 0 2024-03-21T10:08:51.322 INFO:tasks.workunit.client.0.smithi153.stdout:9/478: fdatasync d1/d43/d44/d46/d9e/d24/d41/f5b 0 2024-03-21T10:08:51.324 INFO:tasks.workunit.client.0.smithi153.stdout:9/479: creat d1/d4e/fa6 x:0 0 0 2024-03-21T10:08:51.324 INFO:tasks.workunit.client.0.smithi153.stdout:1/407: dread d6/d32/d3e/f42 [0,4194304] 0 2024-03-21T10:08:51.328 INFO:tasks.workunit.client.0.smithi153.stdout:9/480: symlink d1/d43/d44/d46/d9e/d2e/d8f/la7 0 2024-03-21T10:08:51.332 INFO:tasks.workunit.client.0.smithi153.stdout:4/476: dread d11/f24 [0,4194304] 0 2024-03-21T10:08:51.332 INFO:tasks.workunit.client.0.smithi153.stdout:2/526: truncate dd/d15/d18/d2c/d33/f9d 3845869 0 2024-03-21T10:08:51.334 INFO:tasks.workunit.client.0.smithi153.stdout:1/408: getdents d6/d10/d35/d5b 0 2024-03-21T10:08:51.334 INFO:tasks.workunit.client.1.smithi174.stdout:3/264: dread d0/d12/d14/d1a/d27/d4f/d5d/f31 [0,4194304] 0 2024-03-21T10:08:51.337 INFO:tasks.workunit.client.0.smithi153.stdout:9/481: rename d1/d4/l70 to d1/d43/d44/d46/d9e/d4b/d84/la8 0 2024-03-21T10:08:51.339 INFO:tasks.workunit.client.0.smithi153.stdout:7/388: dwrite d4/d2a/f63 [0,4194304] 0 2024-03-21T10:08:51.340 INFO:tasks.workunit.client.1.smithi174.stdout:8/232: dread d9/d17/f20 [0,4194304] 0 2024-03-21T10:08:51.340 INFO:tasks.workunit.client.1.smithi174.stdout:3/265: unlink d0/f52 0 2024-03-21T10:08:51.340 INFO:tasks.workunit.client.1.smithi174.stdout:5/283: getdents d1/d2f/d37 0 2024-03-21T10:08:51.340 INFO:tasks.workunit.client.0.smithi153.stdout:2/527: creat dd/d15/d3c/da7/d91/dae/d50/d59/d5d/fb0 x:0 0 0 2024-03-21T10:08:51.341 INFO:tasks.workunit.client.1.smithi174.stdout:3/266: chown d0/f33 9678310 1 2024-03-21T10:08:51.342 INFO:tasks.workunit.client.0.smithi153.stdout:4/477: rename d11/d18/f1f to d11/d18/d50/f8c 0 2024-03-21T10:08:51.343 INFO:tasks.workunit.client.0.smithi153.stdout:9/482: write d1/d43/d44/f5d [3321306,42845] 0 2024-03-21T10:08:51.345 INFO:tasks.workunit.client.1.smithi174.stdout:5/284: write d1/d16/f40 [2908206,45759] 0 2024-03-21T10:08:51.345 INFO:tasks.workunit.client.1.smithi174.stdout:3/267: write d0/d12/f54 [661768,86267] 0 2024-03-21T10:08:51.345 INFO:tasks.workunit.client.0.smithi153.stdout:2/528: mknod dd/d15/cb1 0 2024-03-21T10:08:51.347 INFO:tasks.workunit.client.1.smithi174.stdout:5/285: chown d1/d49/d4f/l59 1602596296 1 2024-03-21T10:08:51.348 INFO:tasks.workunit.client.0.smithi153.stdout:2/529: truncate dd/d15/fab 90963 0 2024-03-21T10:08:51.349 INFO:tasks.workunit.client.0.smithi153.stdout:9/483: creat d1/d43/d44/d46/d4d/d5c/fa9 x:0 0 0 2024-03-21T10:08:51.355 INFO:tasks.workunit.client.1.smithi174.stdout:5/286: rmdir d1/d16/d1c 39 2024-03-21T10:08:51.355 INFO:tasks.workunit.client.1.smithi174.stdout:3/268: rmdir d0/d12/d14/d1a/d27 39 2024-03-21T10:08:51.355 INFO:tasks.workunit.client.1.smithi174.stdout:8/233: rename d9/d17/d19/f1e to d9/d2a/f51 0 2024-03-21T10:08:51.356 INFO:tasks.workunit.client.0.smithi153.stdout:9/484: write d1/d4e/f89 [849898,69324] 0 2024-03-21T10:08:51.357 INFO:tasks.workunit.client.1.smithi174.stdout:7/218: sync 2024-03-21T10:08:51.357 INFO:tasks.workunit.client.1.smithi174.stdout:0/285: sync 2024-03-21T10:08:51.358 INFO:tasks.workunit.client.1.smithi174.stdout:5/287: chown d1/d16/c1e 20152 1 2024-03-21T10:08:51.359 INFO:tasks.workunit.client.0.smithi153.stdout:3/431: truncate f2 6139364 0 2024-03-21T10:08:51.359 INFO:tasks.workunit.client.0.smithi153.stdout:2/530: creat dd/d15/d18/d3a/d8c/fb2 x:0 0 0 2024-03-21T10:08:51.362 INFO:tasks.workunit.client.1.smithi174.stdout:3/269: mknod d0/d12/d14/d1a/d27/c63 0 2024-03-21T10:08:51.362 INFO:tasks.workunit.client.0.smithi153.stdout:3/432: readlink dd/d1e/l59 0 2024-03-21T10:08:51.363 INFO:tasks.workunit.client.1.smithi174.stdout:0/286: write d9/f13 [1972928,27905] 0 2024-03-21T10:08:51.364 INFO:tasks.workunit.client.0.smithi153.stdout:9/485: fsync d1/d43/d44/d46/d4d/f74 0 2024-03-21T10:08:51.368 INFO:tasks.workunit.client.1.smithi174.stdout:2/316: dread d5/f12 [0,4194304] 0 2024-03-21T10:08:51.369 INFO:tasks.workunit.client.1.smithi174.stdout:0/287: dread - d9/f1f zero size 2024-03-21T10:08:51.369 INFO:tasks.workunit.client.0.smithi153.stdout:7/389: write d4/d19/f31 [468776,76202] 0 2024-03-21T10:08:51.369 INFO:tasks.workunit.client.1.smithi174.stdout:5/288: rename d1/d49/d4f/d51/l5a to d1/d2f/d37/l62 0 2024-03-21T10:08:51.371 INFO:tasks.workunit.client.1.smithi174.stdout:7/219: fsync d0/d4/f8 0 2024-03-21T10:08:51.374 INFO:tasks.workunit.client.1.smithi174.stdout:0/288: write d9/f1d [2985186,64920] 0 2024-03-21T10:08:51.374 INFO:tasks.workunit.client.0.smithi153.stdout:2/531: mkdir dd/d15/d5e/db3 0 2024-03-21T10:08:51.375 INFO:tasks.workunit.client.0.smithi153.stdout:9/486: mkdir d1/d3b/daa 0 2024-03-21T10:08:51.376 INFO:tasks.workunit.client.1.smithi174.stdout:3/270: mkdir d0/d12/d14/d1a/d27/d4f/d64 0 2024-03-21T10:08:51.377 INFO:tasks.workunit.client.1.smithi174.stdout:5/289: rmdir d1/d16 39 2024-03-21T10:08:51.378 INFO:tasks.workunit.client.1.smithi174.stdout:8/234: dwrite f8 [0,4194304] 0 2024-03-21T10:08:51.381 INFO:tasks.workunit.client.1.smithi174.stdout:0/289: truncate d9/d35/f46 910364 0 2024-03-21T10:08:51.383 INFO:tasks.workunit.client.1.smithi174.stdout:0/290: write d9/f15 [4592590,65881] 0 2024-03-21T10:08:51.386 INFO:tasks.workunit.client.1.smithi174.stdout:8/235: write d9/f30 [1690610,74240] 0 2024-03-21T10:08:51.387 INFO:tasks.workunit.client.0.smithi153.stdout:3/433: link dd/l2e dd/d1e/d3d/l9b 0 2024-03-21T10:08:51.388 INFO:tasks.workunit.client.0.smithi153.stdout:8/496: dwrite d5/d9/d43/dd/f38 [0,4194304] 0 2024-03-21T10:08:51.388 INFO:tasks.workunit.client.0.smithi153.stdout:7/390: truncate d4/d14/f4b 789403 0 2024-03-21T10:08:51.389 INFO:tasks.workunit.client.0.smithi153.stdout:9/487: write d1/d43/d44/d46/d9e/f3d [1876345,71275] 0 2024-03-21T10:08:51.389 INFO:tasks.workunit.client.1.smithi174.stdout:3/271: mkdir d0/d12/d14/d1a/d44/d65 0 2024-03-21T10:08:51.392 INFO:tasks.workunit.client.1.smithi174.stdout:2/317: dread d5/f24 [0,4194304] 0 2024-03-21T10:08:51.392 INFO:tasks.workunit.client.0.smithi153.stdout:9/488: stat d1/d4 0 2024-03-21T10:08:51.393 INFO:tasks.workunit.client.1.smithi174.stdout:0/291: mkdir d9/d21/d5f 0 2024-03-21T10:08:51.395 INFO:tasks.workunit.client.0.smithi153.stdout:1/409: dread d6/fa [0,4194304] 0 2024-03-21T10:08:51.396 INFO:tasks.workunit.client.1.smithi174.stdout:8/236: symlink d9/d37/l52 0 2024-03-21T10:08:51.398 INFO:tasks.workunit.client.1.smithi174.stdout:8/237: fsync f5 0 2024-03-21T10:08:51.401 INFO:tasks.workunit.client.1.smithi174.stdout:5/290: link d1/l14 d1/d5c/l63 0 2024-03-21T10:08:51.402 INFO:tasks.workunit.client.0.smithi153.stdout:3/434: creat dd/d1e/d6b/f9c x:0 0 0 2024-03-21T10:08:51.402 INFO:tasks.workunit.client.0.smithi153.stdout:8/497: symlink d5/d9/d43/dd/d2d/la6 0 2024-03-21T10:08:51.404 INFO:tasks.workunit.client.0.smithi153.stdout:7/391: symlink d4/l85 0 2024-03-21T10:08:51.410 INFO:tasks.workunit.client.0.smithi153.stdout:9/489: creat d1/d43/fab x:0 0 0 2024-03-21T10:08:51.411 INFO:tasks.workunit.client.0.smithi153.stdout:7/392: write d4/d2a/f63 [571293,42716] 0 2024-03-21T10:08:51.412 INFO:tasks.workunit.client.0.smithi153.stdout:2/532: link dd/d15/d3c/da7/d91/dae/d50/d58/c93 dd/d15/cb4 0 2024-03-21T10:08:51.415 INFO:tasks.workunit.client.0.smithi153.stdout:4/478: dread d11/d18/f35 [0,4194304] 0 2024-03-21T10:08:51.419 INFO:tasks.workunit.client.0.smithi153.stdout:3/435: link dd/d1e/d3d/d94/d73/d7f/f95 dd/d1e/d3d/d94/d73/d7f/d91/f9d 0 2024-03-21T10:08:51.422 INFO:tasks.workunit.client.1.smithi174.stdout:2/318: dwrite d5/d25/d27/d2d/d45/d5b/d63/f61 [0,4194304] 0 2024-03-21T10:08:51.425 INFO:tasks.workunit.client.1.smithi174.stdout:0/292: dwrite d9/d35/f36 [0,4194304] 0 2024-03-21T10:08:51.426 INFO:tasks.workunit.client.0.smithi153.stdout:8/498: dread d5/d9/f19 [0,4194304] 0 2024-03-21T10:08:51.426 INFO:tasks.workunit.client.1.smithi174.stdout:5/291: dread d1/f22 [0,4194304] 0 2024-03-21T10:08:51.427 INFO:tasks.workunit.client.0.smithi153.stdout:2/533: unlink dd/d15/d18/d2c/d4b/d56/f85 0 2024-03-21T10:08:51.427 INFO:tasks.workunit.client.0.smithi153.stdout:7/393: symlink d4/d2a/d40/l86 0 2024-03-21T10:08:51.428 INFO:tasks.workunit.client.1.smithi174.stdout:2/319: write d5/f14 [4639399,125723] 0 2024-03-21T10:08:51.428 INFO:tasks.workunit.client.0.smithi153.stdout:4/479: chown d11/f63 404413 1 2024-03-21T10:08:51.428 INFO:tasks.workunit.client.1.smithi174.stdout:0/293: chown d9/d21/f37 4 1 2024-03-21T10:08:51.430 INFO:tasks.workunit.client.0.smithi153.stdout:7/394: dread - d4/d2a/d3b/d60/f7c zero size 2024-03-21T10:08:51.431 INFO:tasks.workunit.client.1.smithi174.stdout:0/294: write d9/d21/d31/f42 [2536138,58048] 0 2024-03-21T10:08:51.432 INFO:tasks.workunit.client.1.smithi174.stdout:5/292: rename d1/fc to d1/d49/d58/f64 0 2024-03-21T10:08:51.432 INFO:tasks.workunit.client.0.smithi153.stdout:2/534: unlink dd/d15/l27 0 2024-03-21T10:08:51.437 INFO:tasks.workunit.client.1.smithi174.stdout:2/320: mknod d5/d25/d27/d2d/c70 0 2024-03-21T10:08:51.437 INFO:tasks.workunit.client.0.smithi153.stdout:8/499: mkdir d5/d9/d43/dd/d20/d26/d24/d52/d56/da7 0 2024-03-21T10:08:51.438 INFO:tasks.workunit.client.0.smithi153.stdout:9/490: dwrite d1/d43/d44/d46/d9e/d24/d41/f5b [0,4194304] 0 2024-03-21T10:08:51.445 INFO:tasks.workunit.client.0.smithi153.stdout:7/395: creat d4/d2a/d40/f87 x:0 0 0 2024-03-21T10:08:51.445 INFO:tasks.workunit.client.0.smithi153.stdout:7/396: readlink d4/d11/l3f 0 2024-03-21T10:08:51.446 INFO:tasks.workunit.client.0.smithi153.stdout:2/535: rmdir dd/d15/d3c/da7/d95 0 2024-03-21T10:08:51.447 INFO:tasks.workunit.client.1.smithi174.stdout:9/338: dwrite d2/d24/d2e/f48 [0,4194304] 0 2024-03-21T10:08:51.447 INFO:tasks.workunit.client.1.smithi174.stdout:6/227: write d0/d3/d5/f45 [2399218,128399] 0 2024-03-21T10:08:51.452 INFO:tasks.workunit.client.0.smithi153.stdout:7/397: write d4/d2a/d40/f74 [428832,81081] 0 2024-03-21T10:08:51.456 INFO:tasks.workunit.client.1.smithi174.stdout:1/298: truncate d4/d7/d33/f36 3281619 0 2024-03-21T10:08:51.460 INFO:tasks.workunit.client.0.smithi153.stdout:8/500: link d5/d9/d43/dd/d2d/d45/d88/f5d d5/d9/fa8 0 2024-03-21T10:08:51.462 INFO:tasks.workunit.client.1.smithi174.stdout:6/228: rmdir d0/d3/d5/d6 39 2024-03-21T10:08:51.466 INFO:tasks.workunit.client.1.smithi174.stdout:5/293: dwrite d1/d49/d4f/f57 [0,4194304] 0 2024-03-21T10:08:51.472 INFO:tasks.workunit.client.0.smithi153.stdout:7/398: dwrite d4/d2a/d40/d50/f7b [0,4194304] 0 2024-03-21T10:08:51.473 INFO:tasks.workunit.client.0.smithi153.stdout:8/501: symlink d5/d9/d43/dd/d20/d26/d47/la9 0 2024-03-21T10:08:51.473 INFO:tasks.workunit.client.1.smithi174.stdout:2/321: fdatasync d5/d25/d27/d2d/d45/d5b/f22 0 2024-03-21T10:08:51.474 INFO:tasks.workunit.client.0.smithi153.stdout:7/399: stat d4/d14/d2f/d78/c41 0 2024-03-21T10:08:51.475 INFO:tasks.workunit.client.1.smithi174.stdout:4/329: dwrite d4/f2c [0,4194304] 0 2024-03-21T10:08:51.476 INFO:tasks.workunit.client.1.smithi174.stdout:0/295: dwrite d9/d1c/f1e [0,4194304] 0 2024-03-21T10:08:51.482 INFO:tasks.workunit.client.0.smithi153.stdout:9/491: link d1/d43/l45 d1/d43/d44/d46/d9e/lac 0 2024-03-21T10:08:51.482 INFO:tasks.workunit.client.1.smithi174.stdout:2/322: truncate d5/d25/d27/d2d/d45/d5b/d63/f59 431168 0 2024-03-21T10:08:51.487 INFO:tasks.workunit.client.1.smithi174.stdout:5/294: mknod d1/d49/c65 0 2024-03-21T10:08:51.488 INFO:tasks.workunit.client.1.smithi174.stdout:5/295: read d1/f22 [732799,82102] 0 2024-03-21T10:08:51.489 INFO:tasks.workunit.client.1.smithi174.stdout:9/339: truncate d2/d24/d2e/f69 1646705 0 2024-03-21T10:08:51.489 INFO:tasks.workunit.client.0.smithi153.stdout:8/502: mkdir d5/d9/d43/d7b/daa 0 2024-03-21T10:08:51.492 INFO:tasks.workunit.client.1.smithi174.stdout:1/299: symlink d4/d7/l6e 0 2024-03-21T10:08:51.496 INFO:tasks.workunit.client.0.smithi153.stdout:7/400: mknod d4/d14/d73/c88 0 2024-03-21T10:08:51.496 INFO:tasks.workunit.client.0.smithi153.stdout:8/503: write d5/d9/d43/dd/d20/d26/d7a/f81 [959342,20756] 0 2024-03-21T10:08:51.496 INFO:tasks.workunit.client.0.smithi153.stdout:3/436: truncate dd/d1e/d2f/d35/f7b 3087930 0 2024-03-21T10:08:51.503 INFO:tasks.workunit.client.1.smithi174.stdout:4/330: dwrite d4/d21/d24/d28/f4e [0,4194304] 0 2024-03-21T10:08:51.506 INFO:tasks.workunit.client.1.smithi174.stdout:7/220: dwrite d0/d4/d10/f45 [0,4194304] 0 2024-03-21T10:08:51.507 INFO:tasks.workunit.client.0.smithi153.stdout:3/437: write dd/d1e/d45/d56/d58/f8b [1034577,125386] 0 2024-03-21T10:08:51.508 INFO:tasks.workunit.client.0.smithi153.stdout:1/410: dwrite d6/fc [4194304,4194304] 0 2024-03-21T10:08:51.512 INFO:tasks.workunit.client.0.smithi153.stdout:9/492: symlink d1/d3b/daa/lad 0 2024-03-21T10:08:51.514 INFO:tasks.workunit.client.1.smithi174.stdout:7/221: truncate d0/d4/d10/f19 727920 0 2024-03-21T10:08:51.516 INFO:tasks.workunit.client.1.smithi174.stdout:4/331: write d4/d21/d24/d28/f2b [1008603,127178] 0 2024-03-21T10:08:51.516 INFO:tasks.workunit.client.1.smithi174.stdout:6/229: rename d0/d3/d20/l33 to d0/l4b 0 2024-03-21T10:08:51.517 INFO:tasks.workunit.client.1.smithi174.stdout:2/323: rename d5/d25/d27 to d5/d25/d27/d2e/d71 22 2024-03-21T10:08:51.519 INFO:tasks.workunit.client.1.smithi174.stdout:1/300: dwrite d4/d2d/f57 [0,4194304] 0 2024-03-21T10:08:51.524 INFO:tasks.workunit.client.1.smithi174.stdout:1/301: fsync d4/d7/f2e 0 2024-03-21T10:08:51.525 INFO:tasks.workunit.client.0.smithi153.stdout:2/536: dread dd/d15/d18/d2c/d4b/d56/f60 [0,4194304] 0 2024-03-21T10:08:51.527 INFO:tasks.workunit.client.0.smithi153.stdout:2/537: chown dd/d15/d18/d2c/d4b/d56 20 1 2024-03-21T10:08:51.529 INFO:tasks.workunit.client.0.smithi153.stdout:3/438: dwrite dd/f22 [0,4194304] 0 2024-03-21T10:08:51.534 INFO:tasks.workunit.client.0.smithi153.stdout:1/411: truncate d6/d10/f11 73069 0 2024-03-21T10:08:51.534 INFO:tasks.workunit.client.0.smithi153.stdout:9/493: mknod d1/d43/d44/d46/d4d/cae 0 2024-03-21T10:08:51.534 INFO:tasks.workunit.client.1.smithi174.stdout:4/332: dwrite d4/f18 [0,4194304] 0 2024-03-21T10:08:51.535 INFO:tasks.workunit.client.1.smithi174.stdout:4/333: write d4/d21/d24/d28/f4e [2167085,93502] 0 2024-03-21T10:08:51.536 INFO:tasks.workunit.client.0.smithi153.stdout:1/412: read - d6/d32/d3e/f63 zero size 2024-03-21T10:08:51.539 INFO:tasks.workunit.client.1.smithi174.stdout:2/324: mknod d5/d25/c72 0 2024-03-21T10:08:51.539 INFO:tasks.workunit.client.1.smithi174.stdout:7/222: chown d0/f2b 0 1 2024-03-21T10:08:51.539 INFO:tasks.workunit.client.1.smithi174.stdout:8/238: write d9/d2a/f51 [1868990,119411] 0 2024-03-21T10:08:51.539 INFO:tasks.workunit.client.0.smithi153.stdout:3/439: creat dd/d1e/d45/d56/f9e x:0 0 0 2024-03-21T10:08:51.542 INFO:tasks.workunit.client.1.smithi174.stdout:2/325: write d5/f49 [3453049,37916] 0 2024-03-21T10:08:51.544 INFO:tasks.workunit.client.1.smithi174.stdout:0/296: link d9/f19 d9/d21/d31/d3c/f60 0 2024-03-21T10:08:51.546 INFO:tasks.workunit.client.0.smithi153.stdout:7/401: rmdir d4/d2a/d3b/d6d 0 2024-03-21T10:08:51.549 INFO:tasks.workunit.client.1.smithi174.stdout:2/326: fsync d5/f26 0 2024-03-21T10:08:51.550 INFO:tasks.workunit.client.1.smithi174.stdout:5/296: mkdir d1/d16/d2e/d66 0 2024-03-21T10:08:51.552 INFO:tasks.workunit.client.0.smithi153.stdout:6/662: sync 2024-03-21T10:08:51.552 INFO:tasks.workunit.client.0.smithi153.stdout:5/512: sync 2024-03-21T10:08:51.552 INFO:tasks.workunit.client.0.smithi153.stdout:3/440: dread dd/d1e/d45/f74 [0,4194304] 0 2024-03-21T10:08:51.553 INFO:tasks.workunit.client.0.smithi153.stdout:2/538: creat dd/d15/d18/d2c/d33/d98/fb5 x:0 0 0 2024-03-21T10:08:51.554 INFO:tasks.workunit.client.1.smithi174.stdout:1/302: creat d4/d5/d6c/f6f x:0 0 0 2024-03-21T10:08:51.556 INFO:tasks.workunit.client.0.smithi153.stdout:6/663: dread - d1f/d2b/d33/d40/f9e zero size 2024-03-21T10:08:51.556 INFO:tasks.workunit.client.1.smithi174.stdout:0/297: mkdir d9/d35/d4f/d61 0 2024-03-21T10:08:51.560 INFO:tasks.workunit.client.1.smithi174.stdout:4/334: dwrite d4/d21/f37 [0,4194304] 0 2024-03-21T10:08:51.560 INFO:tasks.workunit.client.1.smithi174.stdout:5/297: truncate d1/f22 1244623 0 2024-03-21T10:08:51.560 INFO:tasks.workunit.client.1.smithi174.stdout:0/298: fdatasync d9/f1f 0 2024-03-21T10:08:51.561 INFO:tasks.workunit.client.1.smithi174.stdout:9/340: rename d2/f62 to d2/d11/f75 0 2024-03-21T10:08:51.562 INFO:tasks.workunit.client.0.smithi153.stdout:2/539: dread dd/d15/d18/d3a/d3d/f76 [0,4194304] 0 2024-03-21T10:08:51.562 INFO:tasks.workunit.client.0.smithi153.stdout:9/494: symlink d1/d43/d44/d46/d4d/d5c/laf 0 2024-03-21T10:08:51.563 INFO:tasks.workunit.client.1.smithi174.stdout:8/239: mkdir d9/d53 0 2024-03-21T10:08:51.566 INFO:tasks.workunit.client.0.smithi153.stdout:9/495: readlink d1/d4/l9b 0 2024-03-21T10:08:51.566 INFO:tasks.workunit.client.1.smithi174.stdout:1/303: readlink d4/d7/d15/l65 0 2024-03-21T10:08:51.567 INFO:tasks.workunit.client.0.smithi153.stdout:5/513: mkdir d4/d5/df/d67/da0 0 2024-03-21T10:08:51.567 INFO:tasks.workunit.client.1.smithi174.stdout:4/335: write d4/d21/d24/d28/f39 [63983,34175] 0 2024-03-21T10:08:51.569 INFO:tasks.workunit.client.1.smithi174.stdout:9/341: fsync d2/f1f 0 2024-03-21T10:08:51.571 INFO:tasks.workunit.client.1.smithi174.stdout:0/299: rename d9/d1c/f3f to d9/d21/d31/f62 0 2024-03-21T10:08:51.572 INFO:tasks.workunit.client.1.smithi174.stdout:8/240: write d9/d17/f21 [4026522,90731] 0 2024-03-21T10:08:51.573 INFO:tasks.workunit.client.0.smithi153.stdout:2/540: truncate dd/d15/d18/d3a/f3e 4877968 0 2024-03-21T10:08:51.573 INFO:tasks.workunit.client.0.smithi153.stdout:8/504: dwrite d5/d9/d43/d7b/fa2 [0,4194304] 0 2024-03-21T10:08:51.576 INFO:tasks.workunit.client.1.smithi174.stdout:8/241: write d9/f30 [1241364,68007] 0 2024-03-21T10:08:51.578 INFO:tasks.workunit.client.1.smithi174.stdout:1/304: mkdir d4/d7/d70 0 2024-03-21T10:08:51.580 INFO:tasks.workunit.client.1.smithi174.stdout:1/305: dread - d4/f4a zero size 2024-03-21T10:08:51.580 INFO:tasks.workunit.client.1.smithi174.stdout:5/298: dwrite d1/d2f/d37/f5d [0,4194304] 0 2024-03-21T10:08:51.582 INFO:tasks.workunit.client.0.smithi153.stdout:7/402: creat d4/d1b/d83/f89 x:0 0 0 2024-03-21T10:08:51.582 INFO:tasks.workunit.client.1.smithi174.stdout:1/306: chown d4/d7/d15/d53 5286 1 2024-03-21T10:08:51.583 INFO:tasks.workunit.client.0.smithi153.stdout:7/403: chown d4/d2a/d40/f74 246846235 1 2024-03-21T10:08:51.584 INFO:tasks.workunit.client.1.smithi174.stdout:1/307: stat d4/fb 0 2024-03-21T10:08:51.584 INFO:tasks.workunit.client.1.smithi174.stdout:9/342: symlink d2/l76 0 2024-03-21T10:08:51.585 INFO:tasks.workunit.client.1.smithi174.stdout:4/336: symlink d4/d21/d31/d3a/d67/l70 0 2024-03-21T10:08:51.586 INFO:tasks.workunit.client.0.smithi153.stdout:5/514: symlink d4/d8/d5c/d90/la1 0 2024-03-21T10:08:51.587 INFO:tasks.workunit.client.1.smithi174.stdout:9/343: chown d2/d24/d2e/c70 580 1 2024-03-21T10:08:51.587 INFO:tasks.workunit.client.1.smithi174.stdout:1/308: dread - d4/d7/d15/f39 zero size 2024-03-21T10:08:51.587 INFO:tasks.workunit.client.0.smithi153.stdout:8/505: creat d5/d9/d43/dd/d2d/d45/fab x:0 0 0 2024-03-21T10:08:51.587 INFO:tasks.workunit.client.1.smithi174.stdout:1/309: readlink d4/d7/d33/d43/l3f 0 2024-03-21T10:08:51.588 INFO:tasks.workunit.client.1.smithi174.stdout:8/242: rmdir d9/d17 39 2024-03-21T10:08:51.588 INFO:tasks.workunit.client.0.smithi153.stdout:6/664: creat d1f/fed x:0 0 0 2024-03-21T10:08:51.591 INFO:tasks.workunit.client.1.smithi174.stdout:1/310: creat d4/d7/d32/f71 x:0 0 0 2024-03-21T10:08:51.593 INFO:tasks.workunit.client.1.smithi174.stdout:5/299: link d1/d49/d58/f64 d1/d49/f67 0 2024-03-21T10:08:51.593 INFO:tasks.workunit.client.1.smithi174.stdout:4/337: mknod d4/d6c/c71 0 2024-03-21T10:08:51.595 INFO:tasks.workunit.client.1.smithi174.stdout:9/344: rename d2/f1b to d2/d11/d60/f77 0 2024-03-21T10:08:51.596 INFO:tasks.workunit.client.1.smithi174.stdout:1/311: fdatasync d4/d7/d15/d4f/f69 0 2024-03-21T10:08:51.597 INFO:tasks.workunit.client.1.smithi174.stdout:3/272: truncate d0/d12/d14/d1a/f23 703723 0 2024-03-21T10:08:51.601 INFO:tasks.workunit.client.0.smithi153.stdout:5/515: dread d4/f75 [0,4194304] 0 2024-03-21T10:08:51.602 INFO:tasks.workunit.client.1.smithi174.stdout:3/273: read d0/f2 [6536227,116378] 0 2024-03-21T10:08:51.602 INFO:tasks.workunit.client.1.smithi174.stdout:9/345: write d2/f3c [385818,92695] 0 2024-03-21T10:08:51.603 INFO:tasks.workunit.client.0.smithi153.stdout:5/516: truncate d4/d8/f68 106607 0 2024-03-21T10:08:51.604 INFO:tasks.workunit.client.1.smithi174.stdout:5/300: rename d1/d16/f30 to d1/d49/d4f/f68 0 2024-03-21T10:08:51.605 INFO:tasks.workunit.client.0.smithi153.stdout:2/541: dwrite dd/d15/d18/d3a/d8c/faa [0,4194304] 0 2024-03-21T10:08:51.606 INFO:tasks.workunit.client.1.smithi174.stdout:5/301: chown d1/d16/c54 121276 1 2024-03-21T10:08:51.608 INFO:tasks.workunit.client.0.smithi153.stdout:7/404: mknod d4/d1b/d83/c8a 0 2024-03-21T10:08:51.610 INFO:tasks.workunit.client.1.smithi174.stdout:1/312: dread d4/d7/f5b [0,4194304] 0 2024-03-21T10:08:51.611 INFO:tasks.workunit.client.1.smithi174.stdout:3/274: unlink d0/d12/l21 0 2024-03-21T10:08:51.623 INFO:tasks.workunit.client.0.smithi153.stdout:6/665: dwrite d1f/d2b/d33/d40/d5f/f80 [0,4194304] 0 2024-03-21T10:08:51.624 INFO:tasks.workunit.client.1.smithi174.stdout:3/275: link d0/d12/f54 d0/d12/d14/d1a/d27/f66 0 2024-03-21T10:08:51.624 INFO:tasks.workunit.client.1.smithi174.stdout:6/230: truncate d0/d3/d5/df/f2e 1178625 0 2024-03-21T10:08:51.626 INFO:tasks.workunit.client.1.smithi174.stdout:7/223: dwrite d0/d4/f8 [0,4194304] 0 2024-03-21T10:08:51.630 INFO:tasks.workunit.client.0.smithi153.stdout:2/542: rename dd/d15/d3c/c6d to dd/d15/d18/d2c/d4b/d56/cb6 0 2024-03-21T10:08:51.633 INFO:tasks.workunit.client.1.smithi174.stdout:3/276: mkdir d0/d12/d14/d67 0 2024-03-21T10:08:51.634 INFO:tasks.workunit.client.0.smithi153.stdout:2/543: dread - dd/d15/d3c/da7/d91/dae/d50/d58/fa8 zero size 2024-03-21T10:08:51.639 INFO:tasks.workunit.client.1.smithi174.stdout:6/231: rmdir d0/d3/d5/d6 39 2024-03-21T10:08:51.643 INFO:tasks.workunit.client.1.smithi174.stdout:6/232: write d0/d3/d5/df/d16/f27 [2491956,17717] 0 2024-03-21T10:08:51.646 INFO:tasks.workunit.client.0.smithi153.stdout:8/506: link d5/d1b/f87 d5/d9/d43/dd/d2d/d45/fac 0 2024-03-21T10:08:51.646 INFO:tasks.workunit.client.0.smithi153.stdout:5/517: unlink d4/d5/d16/d24/f25 0 2024-03-21T10:08:51.647 INFO:tasks.workunit.client.0.smithi153.stdout:6/666: creat d1f/d2b/d33/d40/d7b/fee x:0 0 0 2024-03-21T10:08:51.649 INFO:tasks.workunit.client.1.smithi174.stdout:4/338: dread d4/f10 [0,4194304] 0 2024-03-21T10:08:51.649 INFO:tasks.workunit.client.1.smithi174.stdout:7/224: link d0/d4/d38/l3d d0/d4/d10/d44/l4c 0 2024-03-21T10:08:51.651 INFO:tasks.workunit.client.0.smithi153.stdout:2/544: rename dd/d15/d18/d2c/d4b/d56/ca6 to dd/d15/d18/d3a/d8c/cb7 0 2024-03-21T10:08:51.651 INFO:tasks.workunit.client.1.smithi174.stdout:6/233: symlink d0/d3/d5/d21/l4c 0 2024-03-21T10:08:51.651 INFO:tasks.workunit.client.1.smithi174.stdout:7/225: chown d0/d4/c29 12299 1 2024-03-21T10:08:51.653 INFO:tasks.workunit.client.0.smithi153.stdout:6/667: write d1f/d2b/d33/d40/d51/f57 [5197937,63389] 0 2024-03-21T10:08:51.653 INFO:tasks.workunit.client.1.smithi174.stdout:4/339: creat d4/d21/d31/d6a/f72 x:0 0 0 2024-03-21T10:08:51.653 INFO:tasks.workunit.client.1.smithi174.stdout:0/300: write d9/d21/d31/d3c/f60 [10263308,113431] 0 2024-03-21T10:08:51.655 INFO:tasks.workunit.client.1.smithi174.stdout:7/226: dread - d0/d4/d10/d1b/f2e zero size 2024-03-21T10:08:51.656 INFO:tasks.workunit.client.1.smithi174.stdout:7/227: readlink d0/d4/d38/l4a 0 2024-03-21T10:08:51.659 INFO:tasks.workunit.client.0.smithi153.stdout:1/413: dwrite d6/f36 [0,4194304] 0 2024-03-21T10:08:51.665 INFO:tasks.workunit.client.1.smithi174.stdout:5/302: truncate d1/d49/d4f/f68 6818738 0 2024-03-21T10:08:51.667 INFO:tasks.workunit.client.1.smithi174.stdout:0/301: creat d9/d58/f63 x:0 0 0 2024-03-21T10:08:51.669 INFO:tasks.workunit.client.1.smithi174.stdout:7/228: chown d0/d4/f17 675096 1 2024-03-21T10:08:51.672 INFO:tasks.workunit.client.0.smithi153.stdout:9/496: write d1/d43/d44/f25 [240123,52448] 0 2024-03-21T10:08:51.672 INFO:tasks.workunit.client.1.smithi174.stdout:5/303: write d1/f4a [152557,98508] 0 2024-03-21T10:08:51.674 INFO:tasks.workunit.client.1.smithi174.stdout:7/229: chown d0/d4/d10/f19 344251 1 2024-03-21T10:08:51.676 INFO:tasks.workunit.client.0.smithi153.stdout:6/668: rmdir d1f/d2b/d33/d40/d65 39 2024-03-21T10:08:51.677 INFO:tasks.workunit.client.0.smithi153.stdout:9/497: truncate d1/d43/d44/d46/d9e/d4b/d84/fa4 834677 0 2024-03-21T10:08:51.679 INFO:tasks.workunit.client.1.smithi174.stdout:3/277: write d0/fb [755662,109064] 0 2024-03-21T10:08:51.682 INFO:tasks.workunit.client.1.smithi174.stdout:7/230: read d0/f26 [5038124,41715] 0 2024-03-21T10:08:51.687 INFO:tasks.workunit.client.1.smithi174.stdout:6/234: dwrite d0/d3/d5/d3b/f46 [0,4194304] 0 2024-03-21T10:08:51.687 INFO:tasks.workunit.client.1.smithi174.stdout:4/340: dwrite d4/d21/d24/d28/f39 [0,4194304] 0 2024-03-21T10:08:51.688 INFO:tasks.workunit.client.0.smithi153.stdout:7/405: getdents d4/d1b/d20 0 2024-03-21T10:08:51.690 INFO:tasks.workunit.client.1.smithi174.stdout:3/278: rename d0/c6 to d0/d12/d14/d1a/d44/d56/c68 0 2024-03-21T10:08:51.691 INFO:tasks.workunit.client.1.smithi174.stdout:4/341: readlink d4/l1a 0 2024-03-21T10:08:51.691 INFO:tasks.workunit.client.1.smithi174.stdout:7/231: symlink d0/d4/d38/l4d 0 2024-03-21T10:08:51.701 INFO:tasks.workunit.client.0.smithi153.stdout:9/498: creat d1/d43/d44/d46/d9e/d4b/d84/d8b/fb0 x:0 0 0 2024-03-21T10:08:51.702 INFO:tasks.workunit.client.1.smithi174.stdout:3/279: truncate d0/d12/d14/d1a/f40 675039 0 2024-03-21T10:08:51.706 INFO:tasks.workunit.client.0.smithi153.stdout:9/499: write d1/d43/d44/d46/f37 [731364,18874] 0 2024-03-21T10:08:51.706 INFO:tasks.workunit.client.1.smithi174.stdout:0/302: dwrite d9/d21/d31/f57 [0,4194304] 0 2024-03-21T10:08:51.707 INFO:tasks.workunit.client.0.smithi153.stdout:0/413: sync 2024-03-21T10:08:51.707 INFO:tasks.workunit.client.1.smithi174.stdout:4/342: truncate d4/d21/d24/d28/d38/f61 944777 0 2024-03-21T10:08:51.708 INFO:tasks.workunit.client.1.smithi174.stdout:6/235: creat d0/d3/d5/f4d x:0 0 0 2024-03-21T10:08:51.710 INFO:tasks.workunit.client.0.smithi153.stdout:9/500: truncate d1/f97 613161 0 2024-03-21T10:08:51.711 INFO:tasks.workunit.client.1.smithi174.stdout:3/280: fsync d0/d12/d14/d1a/f1e 0 2024-03-21T10:08:51.713 INFO:tasks.workunit.client.0.smithi153.stdout:9/501: dread - d1/d4e/fa6 zero size 2024-03-21T10:08:51.713 INFO:tasks.workunit.client.1.smithi174.stdout:5/304: dwrite d1/d16/d1c/f34 [4194304,4194304] 0 2024-03-21T10:08:51.718 INFO:tasks.workunit.client.1.smithi174.stdout:4/343: write d4/d21/d24/d28/d38/f5f [2130142,80477] 0 2024-03-21T10:08:51.719 INFO:tasks.workunit.client.0.smithi153.stdout:8/507: dwrite d5/d9/d43/dd/d20/d26/d47/d69/f86 [0,4194304] 0 2024-03-21T10:08:51.719 INFO:tasks.workunit.client.0.smithi153.stdout:0/414: unlink d4/d16/d3e/l48 0 2024-03-21T10:08:51.721 INFO:tasks.workunit.client.0.smithi153.stdout:7/406: dwrite d4/d19/f31 [0,4194304] 0 2024-03-21T10:08:51.722 INFO:tasks.workunit.client.1.smithi174.stdout:9/346: dwrite d2/d11/d1d/f41 [0,4194304] 0 2024-03-21T10:08:51.723 INFO:tasks.workunit.client.1.smithi174.stdout:6/236: creat d0/d3/d5/df/f4e x:0 0 0 2024-03-21T10:08:51.723 INFO:tasks.workunit.client.1.smithi174.stdout:0/303: mknod d9/d21/d5f/c64 0 2024-03-21T10:08:51.723 INFO:tasks.workunit.client.0.smithi153.stdout:9/502: creat d1/d4e/d68/fb1 x:0 0 0 2024-03-21T10:08:51.725 INFO:tasks.workunit.client.0.smithi153.stdout:0/415: creat d4/d16/d19/d30/d6c/f7f x:0 0 0 2024-03-21T10:08:51.731 INFO:tasks.workunit.client.0.smithi153.stdout:9/503: write d1/d43/d44/d46/d9e/f3d [4853732,74567] 0 2024-03-21T10:08:51.732 INFO:tasks.workunit.client.0.smithi153.stdout:8/508: truncate d5/d9/d43/dd/d20/d26/d24/d52/d56/f8a 928278 0 2024-03-21T10:08:51.737 INFO:tasks.workunit.client.0.smithi153.stdout:2/545: truncate dd/d15/d3c/da7/d91/dae/f5b 4002737 0 2024-03-21T10:08:51.738 INFO:tasks.workunit.client.1.smithi174.stdout:9/347: getdents d2 0 2024-03-21T10:08:51.739 INFO:tasks.workunit.client.1.smithi174.stdout:0/304: creat d9/f65 x:0 0 0 2024-03-21T10:08:51.743 INFO:tasks.workunit.client.1.smithi174.stdout:3/281: dwrite d0/d12/d14/d1a/d27/d4f/d5d/f38 [0,4194304] 0 2024-03-21T10:08:51.746 INFO:tasks.workunit.client.1.smithi174.stdout:0/305: read d9/f19 [4335870,38171] 0 2024-03-21T10:08:51.750 INFO:tasks.workunit.client.0.smithi153.stdout:2/546: creat dd/d15/d5e/fb8 x:0 0 0 2024-03-21T10:08:51.750 INFO:tasks.workunit.client.0.smithi153.stdout:5/518: dread d4/d8/d3b/f4b [0,4194304] 0 2024-03-21T10:08:51.754 INFO:tasks.workunit.client.1.smithi174.stdout:3/282: rename d0/l11 to d0/d12/d14/d1a/d44/d56/l69 0 2024-03-21T10:08:51.755 INFO:tasks.workunit.client.1.smithi174.stdout:0/306: creat d9/d35/d4f/d56/f66 x:0 0 0 2024-03-21T10:08:51.756 INFO:tasks.workunit.client.1.smithi174.stdout:3/283: dread - d0/d12/d14/d1a/d27/d3c/f61 zero size 2024-03-21T10:08:51.757 INFO:tasks.workunit.client.1.smithi174.stdout:0/307: write d9/d35/d4f/d56/f66 [12815,16483] 0 2024-03-21T10:08:51.758 INFO:tasks.workunit.client.0.smithi153.stdout:8/509: rmdir d5/d9/d43/dd/d20/d26/d24/d52/d56/d9d 0 2024-03-21T10:08:51.758 INFO:tasks.workunit.client.0.smithi153.stdout:5/519: truncate d4/d5/f57 5035362 0 2024-03-21T10:08:51.758 INFO:tasks.workunit.client.0.smithi153.stdout:6/669: truncate d1f/d2b/f2c 3839919 0 2024-03-21T10:08:51.762 INFO:tasks.workunit.client.1.smithi174.stdout:3/284: symlink d0/d12/d14/l6a 0 2024-03-21T10:08:51.763 INFO:tasks.workunit.client.0.smithi153.stdout:7/407: dwrite d4/d14/d2f/f5e [0,4194304] 0 2024-03-21T10:08:51.763 INFO:tasks.workunit.client.1.smithi174.stdout:0/308: mkdir d9/d35/d4a/d67 0 2024-03-21T10:08:51.768 INFO:tasks.workunit.client.0.smithi153.stdout:2/547: chown dd/d15/d18/d2c/d4b/d56/cb6 547334 1 2024-03-21T10:08:51.769 INFO:tasks.workunit.client.0.smithi153.stdout:7/408: chown d4/d14/d2f/c43 0 1 2024-03-21T10:08:51.769 INFO:tasks.workunit.client.1.smithi174.stdout:0/309: rename d9/c10 to d9/d35/d4f/c68 0 2024-03-21T10:08:51.771 INFO:tasks.workunit.client.1.smithi174.stdout:0/310: mknod d9/d35/d4f/c69 0 2024-03-21T10:08:51.774 INFO:tasks.workunit.client.0.smithi153.stdout:5/520: symlink d4/d5/df/d67/d6a/la2 0 2024-03-21T10:08:51.776 INFO:tasks.workunit.client.1.smithi174.stdout:3/285: dread d0/d12/d14/d1a/f34 [0,4194304] 0 2024-03-21T10:08:51.781 INFO:tasks.workunit.client.1.smithi174.stdout:3/286: read d0/d12/d14/d1a/f3d [4046848,18898] 0 2024-03-21T10:08:51.781 INFO:tasks.workunit.client.0.smithi153.stdout:5/521: chown d4/d8/d5a/d5d/f55 758 1 2024-03-21T10:08:51.786 INFO:tasks.workunit.client.1.smithi174.stdout:3/287: write d0/d12/d14/d1a/d27/f47 [948079,96539] 0 2024-03-21T10:08:51.787 INFO:tasks.workunit.client.0.smithi153.stdout:5/522: write d4/d5/df/d5e/f81 [308667,113214] 0 2024-03-21T10:08:51.787 INFO:tasks.workunit.client.0.smithi153.stdout:6/670: symlink d1f/d2b/d33/d40/d7b/lef 0 2024-03-21T10:08:51.787 INFO:tasks.workunit.client.0.smithi153.stdout:5/523: chown d4 162830 1 2024-03-21T10:08:51.787 INFO:tasks.workunit.client.0.smithi153.stdout:8/510: symlink d5/lad 0 2024-03-21T10:08:51.787 INFO:tasks.workunit.client.1.smithi174.stdout:3/288: fdatasync d0/f3 0 2024-03-21T10:08:51.791 INFO:tasks.workunit.client.0.smithi153.stdout:5/524: chown d4/d5/df/f58 5010683 1 2024-03-21T10:08:51.792 INFO:tasks.workunit.client.1.smithi174.stdout:3/289: creat d0/d12/d14/d1a/d44/d65/f6b x:0 0 0 2024-03-21T10:08:51.793 INFO:tasks.workunit.client.1.smithi174.stdout:4/344: write d4/f15 [4112464,123339] 0 2024-03-21T10:08:51.794 INFO:tasks.workunit.client.1.smithi174.stdout:3/290: chown d0/d12/d14/d1a/f42 15 1 2024-03-21T10:08:51.795 INFO:tasks.workunit.client.1.smithi174.stdout:4/345: write d4/d21/d24/f4d [729751,114090] 0 2024-03-21T10:08:51.798 INFO:tasks.workunit.client.0.smithi153.stdout:0/416: dwrite d4/db/f29 [0,4194304] 0 2024-03-21T10:08:51.802 INFO:tasks.workunit.client.1.smithi174.stdout:5/305: dwrite d1/d16/d2e/d31/f48 [0,4194304] 0 2024-03-21T10:08:51.803 INFO:tasks.workunit.client.1.smithi174.stdout:3/291: truncate d0/d12/d14/d1a/f1e 491744 0 2024-03-21T10:08:51.806 INFO:tasks.workunit.client.0.smithi153.stdout:0/417: dread - d4/d16/d19/d58/f6a zero size 2024-03-21T10:08:51.808 INFO:tasks.workunit.client.0.smithi153.stdout:7/409: link d4/l85 d4/d14/d2f/d78/d84/l8b 0 2024-03-21T10:08:51.808 INFO:tasks.workunit.client.0.smithi153.stdout:2/548: dwrite dd/d15/d18/d2c/d33/f7d [0,4194304] 0 2024-03-21T10:08:51.810 INFO:tasks.workunit.client.0.smithi153.stdout:6/671: creat d1f/d2b/d33/d40/d51/ff0 x:0 0 0 2024-03-21T10:08:51.810 INFO:tasks.workunit.client.1.smithi174.stdout:6/237: truncate d0/d3/d5/d3b/f46 3260547 0 2024-03-21T10:08:51.811 INFO:tasks.workunit.client.0.smithi153.stdout:5/525: mknod d4/d8/d5a/d5d/ca3 0 2024-03-21T10:08:51.813 INFO:tasks.workunit.client.1.smithi174.stdout:3/292: write d0/d12/f17 [3352118,67197] 0 2024-03-21T10:08:51.813 INFO:tasks.workunit.client.1.smithi174.stdout:5/306: truncate d1/d16/f24 288103 0 2024-03-21T10:08:51.816 INFO:tasks.workunit.client.1.smithi174.stdout:3/293: chown d0/d12/d14/d1a/d44 808723283 1 2024-03-21T10:08:51.816 INFO:tasks.workunit.client.0.smithi153.stdout:6/672: truncate d1f/d2b/d33/d40/d4f/f83 1713901 0 2024-03-21T10:08:51.818 INFO:tasks.workunit.client.0.smithi153.stdout:0/418: creat d4/d16/d3e/f80 x:0 0 0 2024-03-21T10:08:51.819 INFO:tasks.workunit.client.1.smithi174.stdout:6/238: mknod d0/d3/d5/d6/c4f 0 2024-03-21T10:08:51.820 INFO:tasks.workunit.client.0.smithi153.stdout:9/504: truncate d1/d43/d44/d46/d9e/d24/d41/f5b 1584969 0 2024-03-21T10:08:51.821 INFO:tasks.workunit.client.0.smithi153.stdout:7/410: chown d4/d1b/d20/c75 24186 1 2024-03-21T10:08:51.822 INFO:tasks.workunit.client.1.smithi174.stdout:4/346: dwrite d4/d21/f49 [0,4194304] 0 2024-03-21T10:08:51.825 INFO:tasks.workunit.client.1.smithi174.stdout:3/294: creat d0/d12/d14/d1a/d44/f6c x:0 0 0 2024-03-21T10:08:51.826 INFO:tasks.workunit.client.0.smithi153.stdout:5/526: fsync d4/f75 0 2024-03-21T10:08:51.826 INFO:tasks.workunit.client.1.smithi174.stdout:4/347: fsync d4/d21/d31/d6a/f6e 0 2024-03-21T10:08:51.827 INFO:tasks.workunit.client.0.smithi153.stdout:7/411: read - d4/d2a/d40/f87 zero size 2024-03-21T10:08:51.827 INFO:tasks.workunit.client.0.smithi153.stdout:4/480: sync 2024-03-21T10:08:51.827 INFO:tasks.workunit.client.0.smithi153.stdout:3/441: sync 2024-03-21T10:08:51.831 INFO:tasks.workunit.client.0.smithi153.stdout:3/442: write dd/d1e/d3d/d94/d73/f7c [897029,95973] 0 2024-03-21T10:08:51.832 INFO:tasks.workunit.client.0.smithi153.stdout:7/412: write d4/d14/d2f/d78/f6f [427308,117697] 0 2024-03-21T10:08:51.835 INFO:tasks.workunit.client.1.smithi174.stdout:0/311: write d9/d1c/f20 [2884022,6029] 0 2024-03-21T10:08:51.837 INFO:tasks.workunit.client.0.smithi153.stdout:2/549: rename dd/d15/d18/d2c/d4b/d56/d63/f83 to dd/d15/d5e/db3/fb9 0 2024-03-21T10:08:51.840 INFO:tasks.workunit.client.0.smithi153.stdout:2/550: stat dd/d15/d18/c92 0 2024-03-21T10:08:51.842 INFO:tasks.workunit.client.0.smithi153.stdout:9/505: symlink d1/d43/d44/lb2 0 2024-03-21T10:08:51.844 INFO:tasks.workunit.client.1.smithi174.stdout:2/327: sync 2024-03-21T10:08:51.845 INFO:tasks.workunit.client.0.smithi153.stdout:9/506: readlink d1/d43/d44/l2a 0 2024-03-21T10:08:51.845 INFO:tasks.workunit.client.1.smithi174.stdout:0/312: creat d9/d35/d4a/f6a x:0 0 0 2024-03-21T10:08:51.847 INFO:tasks.workunit.client.1.smithi174.stdout:6/239: link d0/c36 d0/d3/d5/c50 0 2024-03-21T10:08:51.847 INFO:tasks.workunit.client.0.smithi153.stdout:8/511: write d5/d9/fa8 [53117,72793] 0 2024-03-21T10:08:51.848 INFO:tasks.workunit.client.1.smithi174.stdout:4/348: creat d4/d21/d24/f73 x:0 0 0 2024-03-21T10:08:51.850 INFO:tasks.workunit.client.0.smithi153.stdout:9/507: truncate d1/d43/d44/d46/d4d/f74 138784 0 2024-03-21T10:08:51.851 INFO:tasks.workunit.client.1.smithi174.stdout:0/313: mknod d9/d21/d31/c6b 0 2024-03-21T10:08:51.851 INFO:tasks.workunit.client.1.smithi174.stdout:2/328: rename d5/d25/d27/d2d/d45/d5b to d5/d25/d27/d2d/d45/d5b/d2f/d66/d73 22 2024-03-21T10:08:51.852 INFO:tasks.workunit.client.0.smithi153.stdout:1/414: sync 2024-03-21T10:08:51.855 INFO:tasks.workunit.client.1.smithi174.stdout:3/295: dread d0/d12/f54 [0,4194304] 0 2024-03-21T10:08:51.856 INFO:tasks.workunit.client.1.smithi174.stdout:2/329: mkdir d5/d25/d27/d74 0 2024-03-21T10:08:51.859 INFO:tasks.workunit.client.1.smithi174.stdout:3/296: chown d0/d12/d14/l41 0 1 2024-03-21T10:08:51.859 INFO:tasks.workunit.client.1.smithi174.stdout:0/314: write d9/d35/f36 [4594245,51100] 0 2024-03-21T10:08:51.860 INFO:tasks.workunit.client.1.smithi174.stdout:6/240: getdents d0/d3/d20/d3d 0 2024-03-21T10:08:51.862 INFO:tasks.workunit.client.0.smithi153.stdout:2/551: symlink dd/d15/d18/d2c/d4b/d56/d63/d8d/d9b/lba 0 2024-03-21T10:08:51.863 INFO:tasks.workunit.client.0.smithi153.stdout:9/508: creat d1/d4e/d68/fb3 x:0 0 0 2024-03-21T10:08:51.864 INFO:tasks.workunit.client.1.smithi174.stdout:2/330: fsync d5/f12 0 2024-03-21T10:08:51.868 INFO:tasks.workunit.client.1.smithi174.stdout:0/315: creat d9/d21/d5f/f6c x:0 0 0 2024-03-21T10:08:51.869 INFO:tasks.workunit.client.1.smithi174.stdout:6/241: creat d0/d3/d5/d3b/f51 x:0 0 0 2024-03-21T10:08:51.869 INFO:tasks.workunit.client.0.smithi153.stdout:2/552: creat dd/d15/d5e/fbb x:0 0 0 2024-03-21T10:08:51.870 INFO:tasks.workunit.client.1.smithi174.stdout:0/316: fdatasync d9/d21/f26 0 2024-03-21T10:08:51.872 INFO:tasks.workunit.client.0.smithi153.stdout:5/527: link d4/d8/d5a/d5d/ca3 d4/d5/de/ca4 0 2024-03-21T10:08:51.873 INFO:tasks.workunit.client.1.smithi174.stdout:5/307: write d1/d49/d4f/f68 [1863910,10135] 0 2024-03-21T10:08:51.873 INFO:tasks.workunit.client.1.smithi174.stdout:6/242: symlink d0/d3/d20/l52 0 2024-03-21T10:08:51.874 INFO:tasks.workunit.client.0.smithi153.stdout:6/673: rename d1f/da0/fa6 to d1f/ff1 0 2024-03-21T10:08:51.875 INFO:tasks.workunit.client.1.smithi174.stdout:4/349: dwrite d4/d19/f23 [0,4194304] 0 2024-03-21T10:08:51.876 INFO:tasks.workunit.client.0.smithi153.stdout:5/528: chown d4/d8/d5a/d5d/f55 29043 1 2024-03-21T10:08:51.876 INFO:tasks.workunit.client.0.smithi153.stdout:8/512: link d5/d9/d43/dd/d20/d26/d24/l27 d5/d9/d43/dd/d20/d26/d24/d52/d55/lae 0 2024-03-21T10:08:51.876 INFO:tasks.workunit.client.0.smithi153.stdout:6/674: dread - d1f/fed zero size 2024-03-21T10:08:51.877 INFO:tasks.workunit.client.1.smithi174.stdout:4/350: fdatasync d4/f7 0 2024-03-21T10:08:51.879 INFO:tasks.workunit.client.1.smithi174.stdout:5/308: mknod d1/d2f/c69 0 2024-03-21T10:08:51.883 INFO:tasks.workunit.client.0.smithi153.stdout:8/513: chown d5/d9/d43/dd/d20/d26/f23 493007050 1 2024-03-21T10:08:51.885 INFO:tasks.workunit.client.0.smithi153.stdout:3/443: write dd/f18 [810750,94694] 0 2024-03-21T10:08:51.886 INFO:tasks.workunit.client.1.smithi174.stdout:4/351: read d4/d21/d24/f4d [396921,6928] 0 2024-03-21T10:08:51.889 INFO:tasks.workunit.client.0.smithi153.stdout:0/419: rename d4/d16/d1f/d22/f2e to d4/d4c/f81 0 2024-03-21T10:08:51.891 INFO:tasks.workunit.client.0.smithi153.stdout:6/675: mkdir d1f/de0/de1/df2 0 2024-03-21T10:08:51.891 INFO:tasks.workunit.client.1.smithi174.stdout:3/297: dread d0/d12/d14/f3a [0,4194304] 0 2024-03-21T10:08:51.894 INFO:tasks.workunit.client.0.smithi153.stdout:3/444: write dd/d1e/d45/f7a [388485,22939] 0 2024-03-21T10:08:51.895 INFO:tasks.workunit.client.1.smithi174.stdout:6/243: rename d0/d3/d20/c25 to d0/d3/c53 0 2024-03-21T10:08:51.895 INFO:tasks.workunit.client.0.smithi153.stdout:2/553: dread dd/d15/d18/f24 [0,4194304] 0 2024-03-21T10:08:51.896 INFO:tasks.workunit.client.0.smithi153.stdout:7/413: rename d4/d1b/d20/l66 to d4/d2a/d3b/d81/l8c 0 2024-03-21T10:08:51.897 INFO:tasks.workunit.client.1.smithi174.stdout:3/298: read d0/fa [1978199,14380] 0 2024-03-21T10:08:51.897 INFO:tasks.workunit.client.1.smithi174.stdout:6/244: fsync d0/d3/d5/df/d16/f2b 0 2024-03-21T10:08:51.897 INFO:tasks.workunit.client.1.smithi174.stdout:5/309: mknod d1/d16/d2e/d31/c6a 0 2024-03-21T10:08:51.900 INFO:tasks.workunit.client.0.smithi153.stdout:4/481: dwrite d11/d18/f64 [0,4194304] 0 2024-03-21T10:08:51.900 INFO:tasks.workunit.client.0.smithi153.stdout:9/509: dread d1/d43/d44/d46/d9e/d2e/f76 [0,4194304] 0 2024-03-21T10:08:51.901 INFO:tasks.workunit.client.0.smithi153.stdout:7/414: rmdir d4/d11 39 2024-03-21T10:08:51.902 INFO:tasks.workunit.client.0.smithi153.stdout:4/482: chown d11/d18/f64 1013874104 1 2024-03-21T10:08:51.902 INFO:tasks.workunit.client.1.smithi174.stdout:0/317: dwrite d9/f54 [0,4194304] 0 2024-03-21T10:08:51.902 INFO:tasks.workunit.client.0.smithi153.stdout:3/445: mkdir dd/d1e/d45/d9f 0 2024-03-21T10:08:51.903 INFO:tasks.workunit.client.0.smithi153.stdout:2/554: rmdir dd/d15/d3c/da7 39 2024-03-21T10:08:51.903 INFO:tasks.workunit.client.1.smithi174.stdout:3/299: stat d0/d12/d14/d1a/f23 0 2024-03-21T10:08:51.908 INFO:tasks.workunit.client.1.smithi174.stdout:6/245: symlink d0/d3/d5/l54 0 2024-03-21T10:08:51.909 INFO:tasks.workunit.client.1.smithi174.stdout:3/300: mknod d0/d12/d14/d1a/d44/c6d 0 2024-03-21T10:08:51.909 INFO:tasks.workunit.client.1.smithi174.stdout:5/310: read d1/d2f/f4c [3915748,81759] 0 2024-03-21T10:08:51.910 INFO:tasks.workunit.client.0.smithi153.stdout:4/483: unlink d11/c51 0 2024-03-21T10:08:51.910 INFO:tasks.workunit.client.1.smithi174.stdout:4/352: dread d4/d21/d24/d28/d38/f5f [0,4194304] 0 2024-03-21T10:08:51.914 INFO:tasks.workunit.client.1.smithi174.stdout:0/318: creat d9/f6d x:0 0 0 2024-03-21T10:08:51.914 INFO:tasks.workunit.client.1.smithi174.stdout:3/301: dread - d0/d12/d14/d1a/f42 zero size 2024-03-21T10:08:51.915 INFO:tasks.workunit.client.1.smithi174.stdout:0/319: write d9/fc [2325313,58696] 0 2024-03-21T10:08:51.915 INFO:tasks.workunit.client.0.smithi153.stdout:7/415: rename d4/d14/d73/c88 to d4/d14/d16/c8d 0 2024-03-21T10:08:51.916 INFO:tasks.workunit.client.1.smithi174.stdout:3/302: chown d0/d12/d14/d1a/d27/d4f/d5d/f1f 521022302 1 2024-03-21T10:08:51.917 INFO:tasks.workunit.client.1.smithi174.stdout:5/311: mknod d1/d16/c6b 0 2024-03-21T10:08:51.918 INFO:tasks.workunit.client.1.smithi174.stdout:0/320: creat d9/d21/f6e x:0 0 0 2024-03-21T10:08:51.920 INFO:tasks.workunit.client.1.smithi174.stdout:5/312: dread - d1/d2f/f5e zero size 2024-03-21T10:08:51.921 INFO:tasks.workunit.client.1.smithi174.stdout:4/353: link d4/d21/d31/d3a/f3d d4/d19/f74 0 2024-03-21T10:08:51.925 INFO:tasks.workunit.client.0.smithi153.stdout:0/420: link d4/d16/d27/l5a d4/d16/d27/l82 0 2024-03-21T10:08:51.925 INFO:tasks.workunit.client.0.smithi153.stdout:2/555: write dd/d15/d3c/da7/d91/dae/d50/d59/d5d/f94 [620079,64066] 0 2024-03-21T10:08:51.926 INFO:tasks.workunit.client.1.smithi174.stdout:0/321: creat d9/d35/d4f/d56/f6f x:0 0 0 2024-03-21T10:08:51.927 INFO:tasks.workunit.client.1.smithi174.stdout:4/354: fsync d4/d19/f41 0 2024-03-21T10:08:51.927 INFO:tasks.workunit.client.1.smithi174.stdout:3/303: symlink d0/d12/d14/d1a/d27/d4f/d64/l6e 0 2024-03-21T10:08:51.931 INFO:tasks.workunit.client.1.smithi174.stdout:5/313: creat d1/d16/d2e/d66/f6c x:0 0 0 2024-03-21T10:08:51.931 INFO:tasks.workunit.client.0.smithi153.stdout:8/514: dwrite d5/d9/d43/dd/d2d/d45/d88/f46 [0,4194304] 0 2024-03-21T10:08:51.932 INFO:tasks.workunit.client.0.smithi153.stdout:5/529: write d4/d8/f1d [3176143,107092] 0 2024-03-21T10:08:51.933 INFO:tasks.workunit.client.1.smithi174.stdout:0/322: mknod d9/d35/c70 0 2024-03-21T10:08:51.934 INFO:tasks.workunit.client.1.smithi174.stdout:4/355: mknod d4/d21/d31/c75 0 2024-03-21T10:08:51.937 INFO:tasks.workunit.client.0.smithi153.stdout:9/510: dwrite d1/d43/d44/f25 [0,4194304] 0 2024-03-21T10:08:51.938 INFO:tasks.workunit.client.1.smithi174.stdout:5/314: creat d1/d16/d1c/f6d x:0 0 0 2024-03-21T10:08:51.939 INFO:tasks.workunit.client.1.smithi174.stdout:0/323: symlink d9/d35/l71 0 2024-03-21T10:08:51.941 INFO:tasks.workunit.client.1.smithi174.stdout:4/356: mkdir d4/d21/d31/d3a/d67/d76 0 2024-03-21T10:08:51.942 INFO:tasks.workunit.client.0.smithi153.stdout:6/676: dwrite d1f/d93/dd8/d59/d70/fc1 [0,4194304] 0 2024-03-21T10:08:51.944 INFO:tasks.workunit.client.1.smithi174.stdout:3/304: dread d0/f7 [0,4194304] 0 2024-03-21T10:08:51.946 INFO:tasks.workunit.client.1.smithi174.stdout:0/324: dread f7 [0,4194304] 0 2024-03-21T10:08:51.947 INFO:tasks.workunit.client.1.smithi174.stdout:4/357: mkdir d4/d21/d31/d3a/d77 0 2024-03-21T10:08:51.947 INFO:tasks.workunit.client.0.smithi153.stdout:4/484: readlink d11/d18/l1a 0 2024-03-21T10:08:51.949 INFO:tasks.workunit.client.0.smithi153.stdout:1/415: dwrite d6/f2f [0,4194304] 0 2024-03-21T10:08:51.949 INFO:tasks.workunit.client.0.smithi153.stdout:0/421: rename d4/d16/d1f to d4/d4c/d83 0 2024-03-21T10:08:51.950 INFO:tasks.workunit.client.1.smithi174.stdout:3/305: readlink d0/d12/d14/d1a/d27/d4f/d5d/l48 0 2024-03-21T10:08:51.953 INFO:tasks.workunit.client.1.smithi174.stdout:4/358: mknod d4/d19/c78 0 2024-03-21T10:08:51.954 INFO:tasks.workunit.client.0.smithi153.stdout:2/556: symlink dd/d15/d3c/da7/d91/dae/d50/d59/lbc 0 2024-03-21T10:08:51.955 INFO:tasks.workunit.client.0.smithi153.stdout:0/422: fsync d4/d16/d3e/d5e/f78 0 2024-03-21T10:08:51.957 INFO:tasks.workunit.client.1.smithi174.stdout:3/306: creat d0/d12/d46/f6f x:0 0 0 2024-03-21T10:08:51.958 INFO:tasks.workunit.client.1.smithi174.stdout:6/246: dwrite d0/d3/d5/df/f2e [0,4194304] 0 2024-03-21T10:08:51.958 INFO:tasks.workunit.client.1.smithi174.stdout:4/359: chown d4/f8 53065578 1 2024-03-21T10:08:51.963 INFO:tasks.workunit.client.1.smithi174.stdout:3/307: creat d0/f70 x:0 0 0 2024-03-21T10:08:51.964 INFO:tasks.workunit.client.1.smithi174.stdout:6/247: creat d0/d3/d5/d3b/f55 x:0 0 0 2024-03-21T10:08:51.964 INFO:tasks.workunit.client.1.smithi174.stdout:4/360: mkdir d4/d21/d24/d28/d79 0 2024-03-21T10:08:51.969 INFO:tasks.workunit.client.0.smithi153.stdout:9/511: fdatasync d1/d43/d44/f58 0 2024-03-21T10:08:51.981 INFO:tasks.workunit.client.1.smithi174.stdout:4/361: mkdir d4/d21/d31/d3a/d7a 0 2024-03-21T10:08:51.983 INFO:tasks.workunit.client.1.smithi174.stdout:4/362: chown d4/l16 253736 1 2024-03-21T10:08:51.988 INFO:tasks.workunit.client.1.smithi174.stdout:4/363: creat d4/d21/f7b x:0 0 0 2024-03-21T10:08:51.989 INFO:tasks.workunit.client.0.smithi153.stdout:6/677: creat d1f/d2b/d33/d40/d51/ff3 x:0 0 0 2024-03-21T10:08:51.989 INFO:tasks.workunit.client.0.smithi153.stdout:7/416: write d4/d14/d2f/d78/f29 [190080,40723] 0 2024-03-21T10:08:51.990 INFO:tasks.workunit.client.0.smithi153.stdout:8/515: rename d5/f7e to d5/d9/d43/dd/d20/d26/d8b/faf 0 2024-03-21T10:08:51.990 INFO:tasks.workunit.client.0.smithi153.stdout:6/678: fdatasync d1f/d93/dd8/f4c 0 2024-03-21T10:08:51.990 INFO:tasks.workunit.client.0.smithi153.stdout:1/416: dread - d6/d32/f6e zero size 2024-03-21T10:08:51.990 INFO:tasks.workunit.client.0.smithi153.stdout:2/557: symlink dd/d15/d18/lbd 0 2024-03-21T10:08:51.990 INFO:tasks.workunit.client.0.smithi153.stdout:9/512: chown d1/d4e/f89 1 1 2024-03-21T10:08:51.990 INFO:tasks.workunit.client.0.smithi153.stdout:0/423: mknod d4/d16/d3e/c84 0 2024-03-21T10:08:51.990 INFO:tasks.workunit.client.0.smithi153.stdout:6/679: dread - d1f/d2b/d33/d40/fa5 zero size 2024-03-21T10:08:51.991 INFO:tasks.workunit.client.0.smithi153.stdout:2/558: readlink dd/d15/d18/d3a/d3d/l42 0 2024-03-21T10:08:51.991 INFO:tasks.workunit.client.0.smithi153.stdout:4/485: getdents d11 0 2024-03-21T10:08:51.992 INFO:tasks.workunit.client.0.smithi153.stdout:9/513: fdatasync d1/d43/d44/d46/f37 0 2024-03-21T10:08:51.992 INFO:tasks.workunit.client.0.smithi153.stdout:0/424: unlink d4/f32 0 2024-03-21T10:08:51.992 INFO:tasks.workunit.client.1.smithi174.stdout:4/364: fsync d4/d21/d24/d28/d38/f61 0 2024-03-21T10:08:51.993 INFO:tasks.workunit.client.1.smithi174.stdout:4/365: rename d4/d21/f66 to d4/d19/f7c 0 2024-03-21T10:08:51.993 INFO:tasks.workunit.client.1.smithi174.stdout:6/248: dwrite d0/d3/d5/f3c [0,4194304] 0 2024-03-21T10:08:51.993 INFO:tasks.workunit.client.1.smithi174.stdout:1/313: sync 2024-03-21T10:08:51.993 INFO:tasks.workunit.client.1.smithi174.stdout:8/243: sync 2024-03-21T10:08:51.993 INFO:tasks.workunit.client.1.smithi174.stdout:4/366: write d4/d21/f27 [413931,23835] 0 2024-03-21T10:08:51.994 INFO:tasks.workunit.client.1.smithi174.stdout:1/314: rmdir d4/d7 39 2024-03-21T10:08:51.995 INFO:tasks.workunit.client.0.smithi153.stdout:8/516: symlink d5/d9/d43/dd/d20/d26/d24/d52/d55/lb0 0 2024-03-21T10:08:51.995 INFO:tasks.workunit.client.0.smithi153.stdout:3/446: dwrite dd/f1c [0,4194304] 0 2024-03-21T10:08:51.995 INFO:tasks.workunit.client.0.smithi153.stdout:4/486: rmdir d11/d18/d50 39 2024-03-21T10:08:51.998 INFO:tasks.workunit.client.1.smithi174.stdout:4/367: chown d4/d21/d31/d3a/c6d 383767111 1 2024-03-21T10:08:52.002 INFO:tasks.workunit.client.0.smithi153.stdout:6/680: truncate d1f/d2b/d33/d40/f9e 479321 0 2024-03-21T10:08:52.003 INFO:tasks.workunit.client.1.smithi174.stdout:5/315: dwrite d1/f22 [0,4194304] 0 2024-03-21T10:08:52.003 INFO:tasks.workunit.client.1.smithi174.stdout:8/244: write d9/d17/d19/d27/f41 [999450,53977] 0 2024-03-21T10:08:52.003 INFO:tasks.workunit.client.1.smithi174.stdout:1/315: creat d4/d5/f72 x:0 0 0 2024-03-21T10:08:52.003 INFO:tasks.workunit.client.1.smithi174.stdout:4/368: chown d4/d21/c55 0 1 2024-03-21T10:08:52.004 INFO:tasks.workunit.client.0.smithi153.stdout:2/559: rmdir dd/d15/d18/d2c 39 2024-03-21T10:08:52.004 INFO:tasks.workunit.client.1.smithi174.stdout:8/245: chown d9/d17/d19/f2f 1454 1 2024-03-21T10:08:52.005 INFO:tasks.workunit.client.0.smithi153.stdout:7/417: dread d4/d2a/d3b/d60/f64 [0,4194304] 0 2024-03-21T10:08:52.009 INFO:tasks.workunit.client.0.smithi153.stdout:3/447: write dd/d1e/d45/d56/f79 [1173543,33552] 0 2024-03-21T10:08:52.010 INFO:tasks.workunit.client.0.smithi153.stdout:0/425: stat d4/f28 0 2024-03-21T10:08:52.010 INFO:tasks.workunit.client.0.smithi153.stdout:1/417: rename d6/d3a/d3c/c55 to d6/d10/d35/d5b/c7b 0 2024-03-21T10:08:52.010 INFO:tasks.workunit.client.0.smithi153.stdout:0/426: chown d4/d16/d27/l40 92373 1 2024-03-21T10:08:52.010 INFO:tasks.workunit.client.1.smithi174.stdout:1/316: write d4/d7/d28/f3b [417084,92741] 0 2024-03-21T10:08:52.011 INFO:tasks.workunit.client.1.smithi174.stdout:0/325: dwrite d9/d35/f3e [8388608,4194304] 0 2024-03-21T10:08:52.011 INFO:tasks.workunit.client.1.smithi174.stdout:5/316: dread d1/f4a [0,4194304] 0 2024-03-21T10:08:52.011 INFO:tasks.workunit.client.0.smithi153.stdout:4/487: link d11/d18/f41 d11/f8d 0 2024-03-21T10:08:52.012 INFO:tasks.workunit.client.0.smithi153.stdout:1/418: write d6/d13/f79 [38674,79506] 0 2024-03-21T10:08:52.013 INFO:tasks.workunit.client.0.smithi153.stdout:2/560: symlink dd/d15/d18/d3a/d3d/lbe 0 2024-03-21T10:08:52.015 INFO:tasks.workunit.client.1.smithi174.stdout:4/369: read d4/f7 [1611200,125503] 0 2024-03-21T10:08:52.015 INFO:tasks.workunit.client.1.smithi174.stdout:0/326: chown d9/d21/d31/c6b 91595388 1 2024-03-21T10:08:52.016 INFO:tasks.workunit.client.0.smithi153.stdout:7/418: mknod d4/d11/c8e 0 2024-03-21T10:08:52.016 INFO:tasks.workunit.client.1.smithi174.stdout:8/246: read d9/d17/d19/d27/f41 [593103,36868] 0 2024-03-21T10:08:52.017 INFO:tasks.workunit.client.0.smithi153.stdout:0/427: rmdir d4/d4c/d83/d22 39 2024-03-21T10:08:52.018 INFO:tasks.workunit.client.1.smithi174.stdout:6/249: dwrite d0/d3/d5/f3c [0,4194304] 0 2024-03-21T10:08:52.019 INFO:tasks.workunit.client.0.smithi153.stdout:1/419: chown d6/d3a/d3c/d50/d51 189 1 2024-03-21T10:08:52.020 INFO:tasks.workunit.client.1.smithi174.stdout:6/250: dread - d0/d3/d5/d3b/f55 zero size 2024-03-21T10:08:52.022 INFO:tasks.workunit.client.0.smithi153.stdout:0/428: readlink d4/d16/l4f 0 2024-03-21T10:08:52.023 INFO:tasks.workunit.client.0.smithi153.stdout:6/681: rename d1f/d2b/f2c to d1f/d93/ff4 0 2024-03-21T10:08:52.024 INFO:tasks.workunit.client.1.smithi174.stdout:8/247: symlink d9/d17/d19/d27/l54 0 2024-03-21T10:08:52.027 INFO:tasks.workunit.client.0.smithi153.stdout:8/517: dwrite d5/d9/f8d [0,4194304] 0 2024-03-21T10:08:52.027 INFO:tasks.workunit.client.1.smithi174.stdout:6/251: mkdir d0/d3/d2a/d56 0 2024-03-21T10:08:52.029 INFO:tasks.workunit.client.1.smithi174.stdout:8/248: symlink d9/d17/d19/d31/l55 0 2024-03-21T10:08:52.031 INFO:tasks.workunit.client.1.smithi174.stdout:8/249: dread - d9/d2a/d39/f47 zero size 2024-03-21T10:08:52.035 INFO:tasks.workunit.client.1.smithi174.stdout:8/250: dread - d9/d2a/d39/f47 zero size 2024-03-21T10:08:52.035 INFO:tasks.workunit.client.1.smithi174.stdout:8/251: truncate d9/d17/d19/f2f 396226 0 2024-03-21T10:08:52.036 INFO:tasks.workunit.client.1.smithi174.stdout:8/252: chown d9/d17/c35 12 1 2024-03-21T10:08:52.037 INFO:tasks.workunit.client.0.smithi153.stdout:7/419: dread d4/d14/f4b [0,4194304] 0 2024-03-21T10:08:52.037 INFO:tasks.workunit.client.1.smithi174.stdout:8/253: chown d9/d17/f21 276 1 2024-03-21T10:08:52.038 INFO:tasks.workunit.client.1.smithi174.stdout:8/254: chown d9/d17/d19/d31/f4f 55 1 2024-03-21T10:08:52.040 INFO:tasks.workunit.client.1.smithi174.stdout:8/255: creat d9/d37/f56 x:0 0 0 2024-03-21T10:08:52.042 INFO:tasks.workunit.client.1.smithi174.stdout:8/256: stat d9/d2a/d39/f47 0 2024-03-21T10:08:52.042 INFO:tasks.workunit.client.0.smithi153.stdout:9/514: dwrite d1/d43/d44/d46/d9e/d24/f38 [0,4194304] 0 2024-03-21T10:08:52.043 INFO:tasks.workunit.client.0.smithi153.stdout:2/561: symlink dd/d15/d18/d2c/d33/d98/lbf 0 2024-03-21T10:08:52.045 INFO:tasks.workunit.client.1.smithi174.stdout:8/257: symlink d9/d17/l57 0 2024-03-21T10:08:52.050 INFO:tasks.workunit.client.0.smithi153.stdout:1/420: creat d6/d13/f7c x:0 0 0 2024-03-21T10:08:52.050 INFO:tasks.workunit.client.0.smithi153.stdout:8/518: dread d5/d9/d43/dd/d20/d26/d24/d52/f54 [0,4194304] 0 2024-03-21T10:08:52.051 INFO:tasks.workunit.client.0.smithi153.stdout:9/515: chown d1/l11 242621 1 2024-03-21T10:08:52.055 INFO:tasks.workunit.client.0.smithi153.stdout:6/682: link d1f/d2b/d33/d40/d51/f71 d1f/d2b/d33/d40/d51/dbd/ff5 0 2024-03-21T10:08:52.056 INFO:tasks.workunit.client.0.smithi153.stdout:3/448: write dd/d1e/d3d/f69 [588272,77781] 0 2024-03-21T10:08:52.059 INFO:tasks.workunit.client.0.smithi153.stdout:3/449: read - dd/d1e/d2f/f90 zero size 2024-03-21T10:08:52.060 INFO:tasks.workunit.client.0.smithi153.stdout:0/429: creat d4/d16/d19/f85 x:0 0 0 2024-03-21T10:08:52.060 INFO:tasks.workunit.client.0.smithi153.stdout:2/562: creat dd/d15/d5e/fc0 x:0 0 0 2024-03-21T10:08:52.061 INFO:tasks.workunit.client.0.smithi153.stdout:0/430: readlink d4/d16/l21 0 2024-03-21T10:08:52.066 INFO:tasks.workunit.client.0.smithi153.stdout:1/421: stat d6/d10/d35/d5b/c7b 0 2024-03-21T10:08:52.066 INFO:tasks.workunit.client.0.smithi153.stdout:9/516: creat d1/d43/d44/d46/d9e/d24/fb4 x:0 0 0 2024-03-21T10:08:52.068 INFO:tasks.workunit.client.0.smithi153.stdout:6/683: write d1f/d2b/d33/d40/f4d [2913482,5648] 0 2024-03-21T10:08:52.074 INFO:tasks.workunit.client.0.smithi153.stdout:0/431: creat d4/d16/d19/d30/d6c/f86 x:0 0 0 2024-03-21T10:08:52.074 INFO:tasks.workunit.client.0.smithi153.stdout:0/432: write d4/db/f11 [3424064,29914] 0 2024-03-21T10:08:52.074 INFO:tasks.workunit.client.0.smithi153.stdout:2/563: symlink dd/d15/d3c/da7/d91/dae/d50/d59/d5d/lc1 0 2024-03-21T10:08:52.076 INFO:tasks.workunit.client.0.smithi153.stdout:2/564: fdatasync dd/d15/d18/d3a/d8c/fb2 0 2024-03-21T10:08:52.081 INFO:tasks.workunit.client.0.smithi153.stdout:2/565: write dd/d15/d18/d2c/f4e [1500436,20145] 0 2024-03-21T10:08:52.083 INFO:tasks.workunit.client.0.smithi153.stdout:7/420: truncate d4/d2a/f3e 595355 0 2024-03-21T10:08:52.083 INFO:tasks.workunit.client.1.smithi174.stdout:8/258: rmdir d9 39 2024-03-21T10:08:52.086 INFO:tasks.workunit.client.1.smithi174.stdout:8/259: fsync d9/d17/f20 0 2024-03-21T10:08:52.086 INFO:tasks.workunit.client.1.smithi174.stdout:4/370: dwrite d4/f20 [0,4194304] 0 2024-03-21T10:08:52.088 INFO:tasks.workunit.client.1.smithi174.stdout:8/260: write d9/d17/f20 [1622753,99022] 0 2024-03-21T10:08:52.088 INFO:tasks.workunit.client.1.smithi174.stdout:4/371: stat d4/d21/d24/d28/c5c 0 2024-03-21T10:08:52.089 INFO:tasks.workunit.client.1.smithi174.stdout:8/261: fdatasync d9/f15 0 2024-03-21T10:08:52.089 INFO:tasks.workunit.client.0.smithi153.stdout:7/421: fdatasync d4/d14/d2f/d78/f71 0 2024-03-21T10:08:52.090 INFO:tasks.workunit.client.0.smithi153.stdout:6/684: dwrite d1f/d99/dbb/fcb [0,4194304] 0 2024-03-21T10:08:52.090 INFO:tasks.workunit.client.0.smithi153.stdout:8/519: truncate d5/d9/d43/dd/d2d/d45/d88/f46 362660 0 2024-03-21T10:08:52.090 INFO:tasks.workunit.client.1.smithi174.stdout:8/262: read d9/f1c [75404,64067] 0 2024-03-21T10:08:52.091 INFO:tasks.workunit.client.0.smithi153.stdout:0/433: dread d4/f37 [0,4194304] 0 2024-03-21T10:08:52.093 INFO:tasks.workunit.client.0.smithi153.stdout:7/422: stat d4/d2a 0 2024-03-21T10:08:52.093 INFO:tasks.workunit.client.1.smithi174.stdout:8/263: mkdir d9/d2a/d39/d58 0 2024-03-21T10:08:52.095 INFO:tasks.workunit.client.0.smithi153.stdout:3/450: rename dd/d1e/d45/d56/d58/f5f to dd/d1e/d45/fa0 0 2024-03-21T10:08:52.098 INFO:tasks.workunit.client.0.smithi153.stdout:3/451: chown dd/d1e/d2f/f90 0 1 2024-03-21T10:08:52.099 INFO:tasks.workunit.client.1.smithi174.stdout:8/264: link d9/d2a/d39/l46 d9/d2a/d4d/l59 0 2024-03-21T10:08:52.101 INFO:tasks.workunit.client.0.smithi153.stdout:8/520: symlink d5/d9/d43/d7b/lb1 0 2024-03-21T10:08:52.102 INFO:tasks.workunit.client.1.smithi174.stdout:8/265: write f5 [2381595,127279] 0 2024-03-21T10:08:52.104 INFO:tasks.workunit.client.0.smithi153.stdout:8/521: chown d5/d9/d43/f2b 182 1 2024-03-21T10:08:52.104 INFO:tasks.workunit.client.0.smithi153.stdout:0/434: chown d4/d16/d19/c3f 9176325 1 2024-03-21T10:08:52.105 INFO:tasks.workunit.client.0.smithi153.stdout:7/423: creat d4/d1b/d83/f8f x:0 0 0 2024-03-21T10:08:52.105 INFO:tasks.workunit.client.1.smithi174.stdout:8/266: fdatasync d9/d17/d19/d31/f4f 0 2024-03-21T10:08:52.109 INFO:tasks.workunit.client.1.smithi174.stdout:4/372: dwrite d4/d21/d24/f42 [0,4194304] 0 2024-03-21T10:08:52.111 INFO:tasks.workunit.client.1.smithi174.stdout:4/373: readlink d4/d19/l1d 0 2024-03-21T10:08:52.111 INFO:tasks.workunit.client.0.smithi153.stdout:2/566: symlink dd/d15/d18/d3a/lc2 0 2024-03-21T10:08:52.111 INFO:tasks.workunit.client.0.smithi153.stdout:3/452: creat dd/d1e/d6b/fa1 x:0 0 0 2024-03-21T10:08:52.116 INFO:tasks.workunit.client.0.smithi153.stdout:1/422: dread d6/d3a/f53 [0,4194304] 0 2024-03-21T10:08:52.117 INFO:tasks.workunit.client.0.smithi153.stdout:4/488: rename d11/l1c to d11/d18/d50/d65/l8e 0 2024-03-21T10:08:52.118 INFO:tasks.workunit.client.1.smithi174.stdout:8/267: getdents d9/d17/d19/d27 0 2024-03-21T10:08:52.118 INFO:tasks.workunit.client.1.smithi174.stdout:8/268: fdatasync d9/d2a/f34 0 2024-03-21T10:08:52.130 INFO:tasks.workunit.client.0.smithi153.stdout:0/435: dread d4/d4c/f81 [0,4194304] 0 2024-03-21T10:08:52.131 INFO:tasks.workunit.client.1.smithi174.stdout:8/269: dwrite d9/d17/f42 [0,4194304] 0 2024-03-21T10:08:52.134 INFO:tasks.workunit.client.0.smithi153.stdout:9/517: dread d1/d43/d44/d46/d9e/d2e/f62 [0,4194304] 0 2024-03-21T10:08:52.136 INFO:tasks.workunit.client.0.smithi153.stdout:9/518: readlink d1/d43/d44/l2c 0 2024-03-21T10:08:52.138 INFO:tasks.workunit.client.0.smithi153.stdout:2/567: mkdir dd/d15/d18/d2c/d4b/d56/d63/d8d/d9b/dc3 0 2024-03-21T10:08:52.138 INFO:tasks.workunit.client.0.smithi153.stdout:3/453: dread - dd/d41/f49 zero size 2024-03-21T10:08:52.141 INFO:tasks.workunit.client.0.smithi153.stdout:8/522: dwrite d5/d9/d43/dd/d2d/d45/d88/f3c [0,4194304] 0 2024-03-21T10:08:52.142 INFO:tasks.workunit.client.0.smithi153.stdout:0/436: chown d4/d4c/d83/d22/d61/l7b 439 1 2024-03-21T10:08:52.144 INFO:tasks.workunit.client.0.smithi153.stdout:9/519: dread - d1/d43/d72/d9d/fa0 zero size 2024-03-21T10:08:52.146 INFO:tasks.workunit.client.0.smithi153.stdout:9/520: chown d1/c8 0 1 2024-03-21T10:08:52.152 INFO:tasks.workunit.client.1.smithi174.stdout:4/374: write d4/d21/d24/f4d [1696456,121598] 0 2024-03-21T10:08:52.156 INFO:tasks.workunit.client.1.smithi174.stdout:4/375: read d4/d19/f74 [1619741,32958] 0 2024-03-21T10:08:52.158 INFO:tasks.workunit.client.1.smithi174.stdout:4/376: symlink d4/d21/l7d 0 2024-03-21T10:08:52.160 INFO:tasks.workunit.client.0.smithi153.stdout:8/523: rename d5/lad to d5/d9/d43/dd/d2d/d45/lb2 0 2024-03-21T10:08:52.160 INFO:tasks.workunit.client.0.smithi153.stdout:0/437: dread d4/f10 [4194304,4194304] 0 2024-03-21T10:08:52.160 INFO:tasks.workunit.client.1.smithi174.stdout:7/232: sync 2024-03-21T10:08:52.161 INFO:tasks.workunit.client.1.smithi174.stdout:2/331: sync 2024-03-21T10:08:52.161 INFO:tasks.workunit.client.0.smithi153.stdout:9/521: creat d1/d43/d44/d46/d9e/d4b/d84/fb5 x:0 0 0 2024-03-21T10:08:52.161 INFO:tasks.workunit.client.1.smithi174.stdout:9/348: sync 2024-03-21T10:08:52.162 INFO:tasks.workunit.client.1.smithi174.stdout:4/377: rename d4/f53 to d4/d21/d24/d28/d38/f7e 0 2024-03-21T10:08:52.163 INFO:tasks.workunit.client.0.smithi153.stdout:0/438: readlink d4/db/l46 0 2024-03-21T10:08:52.163 INFO:tasks.workunit.client.0.smithi153.stdout:9/522: chown d1/d43/d44/d46/d9e/d4b 1 1 2024-03-21T10:08:52.164 INFO:tasks.workunit.client.1.smithi174.stdout:9/349: chown d2/d24/c26 773166 1 2024-03-21T10:08:52.164 INFO:tasks.workunit.client.1.smithi174.stdout:4/378: chown d4/l34 106 1 2024-03-21T10:08:52.165 INFO:tasks.workunit.client.1.smithi174.stdout:7/233: truncate d0/d4/d10/f21 1933888 0 2024-03-21T10:08:52.167 INFO:tasks.workunit.client.1.smithi174.stdout:2/332: mkdir d5/d25/d27/d2d/d45/d5b/d55/d75 0 2024-03-21T10:08:52.167 INFO:tasks.workunit.client.0.smithi153.stdout:1/423: dwrite d6/d13/f2b [0,4194304] 0 2024-03-21T10:08:52.168 INFO:tasks.workunit.client.1.smithi174.stdout:2/333: chown d5/f12 134 1 2024-03-21T10:08:52.168 INFO:tasks.workunit.client.1.smithi174.stdout:9/350: symlink d2/d24/d2e/l78 0 2024-03-21T10:08:52.169 INFO:tasks.workunit.client.1.smithi174.stdout:7/234: read d0/d4/f17 [402248,10669] 0 2024-03-21T10:08:52.170 INFO:tasks.workunit.client.0.smithi153.stdout:6/685: truncate d1f/d2b/d33/d40/d4f/f83 1302018 0 2024-03-21T10:08:52.172 INFO:tasks.workunit.client.1.smithi174.stdout:8/270: dread d9/d17/f21 [0,4194304] 0 2024-03-21T10:08:52.173 INFO:tasks.workunit.client.0.smithi153.stdout:6/686: dread - d1f/d2b/d33/d40/d51/ff3 zero size 2024-03-21T10:08:52.174 INFO:tasks.workunit.client.1.smithi174.stdout:7/235: unlink d0/d4/l4b 0 2024-03-21T10:08:52.177 INFO:tasks.workunit.client.1.smithi174.stdout:7/236: chown d0/d4/d10/f19 9 1 2024-03-21T10:08:52.178 INFO:tasks.workunit.client.1.smithi174.stdout:8/271: dread d9/d17/d19/f2f [0,4194304] 0 2024-03-21T10:08:52.181 INFO:tasks.workunit.client.1.smithi174.stdout:7/237: dread - d0/d4/f47 zero size 2024-03-21T10:08:52.185 INFO:tasks.workunit.client.0.smithi153.stdout:3/454: creat dd/d1e/d3d/d94/d8e/fa2 x:0 0 0 2024-03-21T10:08:52.186 INFO:tasks.workunit.client.1.smithi174.stdout:7/238: rename d0/d4/f36 to d0/d4/d10/d1b/f4e 0 2024-03-21T10:08:52.187 INFO:tasks.workunit.client.0.smithi153.stdout:2/568: symlink dd/d15/d3c/da7/d91/dae/d50/d58/lc4 0 2024-03-21T10:08:52.187 INFO:tasks.workunit.client.0.smithi153.stdout:3/455: fdatasync f8 0 2024-03-21T10:08:52.189 INFO:tasks.workunit.client.0.smithi153.stdout:8/524: dread - d5/d9/d43/dd/d2d/d45/f4a zero size 2024-03-21T10:08:52.190 INFO:tasks.workunit.client.1.smithi174.stdout:2/334: dwrite d5/d25/d27/d2d/d45/d5b/ff [4194304,4194304] 0 2024-03-21T10:08:52.195 INFO:tasks.workunit.client.0.smithi153.stdout:9/523: rmdir d1 39 2024-03-21T10:08:52.195 INFO:tasks.workunit.client.0.smithi153.stdout:6/687: rename d1f/d2b/d33/d40/d51/cbe to d1f/d2b/d33/d40/d4f/d66/cf6 0 2024-03-21T10:08:52.195 INFO:tasks.workunit.client.1.smithi174.stdout:8/272: getdents d9/d2a/d39 0 2024-03-21T10:08:52.195 INFO:tasks.workunit.client.1.smithi174.stdout:7/239: dread d0/d4/d10/d1b/f3b [0,4194304] 0 2024-03-21T10:08:52.196 INFO:tasks.workunit.client.1.smithi174.stdout:2/335: mkdir d5/d25/d27/d2d/d45/d5b/d2f/d4b/d76 0 2024-03-21T10:08:52.197 INFO:tasks.workunit.client.1.smithi174.stdout:7/240: chown d0/f11 231 1 2024-03-21T10:08:52.199 INFO:tasks.workunit.client.0.smithi153.stdout:8/525: chown d5/d1b/f59 72 1 2024-03-21T10:08:52.200 INFO:tasks.workunit.client.1.smithi174.stdout:7/241: mkdir d0/d4/d38/d4f 0 2024-03-21T10:08:52.204 INFO:tasks.workunit.client.1.smithi174.stdout:8/273: dwrite d9/f14 [0,4194304] 0 2024-03-21T10:08:52.205 INFO:tasks.workunit.client.1.smithi174.stdout:7/242: rename d0/d4/d38/l48 to d0/d4/d38/l50 0 2024-03-21T10:08:52.205 INFO:tasks.workunit.client.0.smithi153.stdout:1/424: creat d6/d32/d3e/f7d x:0 0 0 2024-03-21T10:08:52.206 INFO:tasks.workunit.client.0.smithi153.stdout:0/439: link d4/d16/l4f d4/d16/l87 0 2024-03-21T10:08:52.206 INFO:tasks.workunit.client.1.smithi174.stdout:8/274: write d9/f14 [4984501,48583] 0 2024-03-21T10:08:52.207 INFO:tasks.workunit.client.1.smithi174.stdout:7/243: write d0/d4/f47 [246525,46128] 0 2024-03-21T10:08:52.207 INFO:tasks.workunit.client.1.smithi174.stdout:8/275: write f1 [2497141,28009] 0 2024-03-21T10:08:52.208 INFO:tasks.workunit.client.1.smithi174.stdout:7/244: mknod d0/d4/c51 0 2024-03-21T10:08:52.209 INFO:tasks.workunit.client.0.smithi153.stdout:9/524: mkdir d1/d4/d5f/db6 0 2024-03-21T10:08:52.210 INFO:tasks.workunit.client.0.smithi153.stdout:4/489: dread d11/f24 [0,4194304] 0 2024-03-21T10:08:52.211 INFO:tasks.workunit.client.1.smithi174.stdout:7/245: write d0/d4/d10/d1b/f4e [3377383,53511] 0 2024-03-21T10:08:52.211 INFO:tasks.workunit.client.0.smithi153.stdout:6/688: truncate d1f/d2b/d33/d40/d5f/f9b 2209512 0 2024-03-21T10:08:52.212 INFO:tasks.workunit.client.0.smithi153.stdout:8/526: mknod d5/d9/d43/dd/d2d/cb3 0 2024-03-21T10:08:52.213 INFO:tasks.workunit.client.1.smithi174.stdout:7/246: write d0/f11 [1502011,74047] 0 2024-03-21T10:08:52.213 INFO:tasks.workunit.client.0.smithi153.stdout:3/456: dwrite dd/f48 [0,4194304] 0 2024-03-21T10:08:52.214 INFO:tasks.workunit.client.1.smithi174.stdout:7/247: write d0/d4/f22 [481242,91166] 0 2024-03-21T10:08:52.216 INFO:tasks.workunit.client.0.smithi153.stdout:9/525: creat d1/d43/d44/d46/d4d/fb7 x:0 0 0 2024-03-21T10:08:52.216 INFO:tasks.workunit.client.0.smithi153.stdout:4/490: rmdir d11/d18/d50 39 2024-03-21T10:08:52.220 INFO:tasks.workunit.client.0.smithi153.stdout:2/569: dwrite dd/d15/d18/d3a/d8c/faa [0,4194304] 0 2024-03-21T10:08:52.220 INFO:tasks.workunit.client.0.smithi153.stdout:3/457: write dd/d1e/d2f/f90 [15859,126353] 0 2024-03-21T10:08:52.227 INFO:tasks.workunit.client.0.smithi153.stdout:6/689: mkdir d1f/d93/df7 0 2024-03-21T10:08:52.228 INFO:tasks.workunit.client.0.smithi153.stdout:4/491: fdatasync d11/d2d/f43 0 2024-03-21T10:08:52.230 INFO:tasks.workunit.client.0.smithi153.stdout:3/458: unlink dd/f1c 0 2024-03-21T10:08:52.232 INFO:tasks.workunit.client.0.smithi153.stdout:2/570: rmdir dd/d15/d18/d3a 39 2024-03-21T10:08:52.233 INFO:tasks.workunit.client.0.smithi153.stdout:4/492: write d11/d2d/f42 [458224,62120] 0 2024-03-21T10:08:52.237 INFO:tasks.workunit.client.0.smithi153.stdout:8/527: dread d5/d1b/d49/f67 [0,4194304] 0 2024-03-21T10:08:52.240 INFO:tasks.workunit.client.0.smithi153.stdout:3/459: chown dd/d1e/d2f/d35/d68/d75/l8f 30929985 1 2024-03-21T10:08:52.240 INFO:tasks.workunit.client.0.smithi153.stdout:4/493: mknod d11/d18/d54/c8f 0 2024-03-21T10:08:52.241 INFO:tasks.workunit.client.0.smithi153.stdout:2/571: truncate dd/d15/d5e/fc0 222704 0 2024-03-21T10:08:52.241 INFO:tasks.workunit.client.0.smithi153.stdout:6/690: creat d1f/d93/dd8/d59/d70/ddc/ff8 x:0 0 0 2024-03-21T10:08:52.242 INFO:tasks.workunit.client.0.smithi153.stdout:8/528: read - d5/d9/d43/dd/d20/d26/d8b/faf zero size 2024-03-21T10:08:52.248 INFO:tasks.workunit.client.0.smithi153.stdout:3/460: creat dd/d1e/d3d/d53/fa3 x:0 0 0 2024-03-21T10:08:52.252 INFO:tasks.workunit.client.1.smithi174.stdout:7/248: dread d0/d4/fd [0,4194304] 0 2024-03-21T10:08:52.254 INFO:tasks.workunit.client.0.smithi153.stdout:3/461: creat dd/d1e/d3d/d94/d73/fa4 x:0 0 0 2024-03-21T10:08:52.257 INFO:tasks.workunit.client.0.smithi153.stdout:3/462: write dd/f48 [2236208,22718] 0 2024-03-21T10:08:52.265 INFO:tasks.workunit.client.0.smithi153.stdout:3/463: creat dd/d1e/d45/d56/fa5 x:0 0 0 2024-03-21T10:08:52.267 INFO:tasks.workunit.client.0.smithi153.stdout:3/464: readlink dd/d1e/l23 0 2024-03-21T10:08:52.268 INFO:tasks.workunit.client.1.smithi174.stdout:7/249: getdents d0/d4/d10 0 2024-03-21T10:08:52.268 INFO:tasks.workunit.client.0.smithi153.stdout:8/529: dwrite d5/d9/d43/dd/d20/d26/f1f [0,4194304] 0 2024-03-21T10:08:52.269 INFO:tasks.workunit.client.0.smithi153.stdout:3/465: dread - dd/d1e/d2f/d35/d68/f96 zero size 2024-03-21T10:08:52.273 INFO:tasks.workunit.client.0.smithi153.stdout:3/466: mkdir dd/d1e/da6 0 2024-03-21T10:08:52.274 INFO:tasks.workunit.client.0.smithi153.stdout:8/530: mknod d5/d9/d43/dd/d2d/d45/d88/cb4 0 2024-03-21T10:08:52.286 INFO:tasks.workunit.client.0.smithi153.stdout:8/531: getdents d5/d9/d43/dd/d2d/d45 0 2024-03-21T10:08:52.287 INFO:tasks.workunit.client.1.smithi174.stdout:2/336: write d5/d25/d27/d2d/d45/d5b/f22 [1438408,106075] 0 2024-03-21T10:08:52.289 INFO:tasks.workunit.client.0.smithi153.stdout:8/532: dread - d5/d9/f9b zero size 2024-03-21T10:08:52.289 INFO:tasks.workunit.client.0.smithi153.stdout:4/494: dread f1 [4194304,4194304] 0 2024-03-21T10:08:52.290 INFO:tasks.workunit.client.0.smithi153.stdout:3/467: dread dd/d1e/d45/f7a [0,4194304] 0 2024-03-21T10:08:52.290 INFO:tasks.workunit.client.1.smithi174.stdout:7/250: rename d0/f2f to d0/f52 0 2024-03-21T10:08:52.291 INFO:tasks.workunit.client.0.smithi153.stdout:1/425: write d6/d3a/d3c/d50/d51/f6d [2294,21985] 0 2024-03-21T10:08:52.293 INFO:tasks.workunit.client.1.smithi174.stdout:4/379: dread d4/d19/f41 [0,4194304] 0 2024-03-21T10:08:52.297 INFO:tasks.workunit.client.0.smithi153.stdout:8/533: creat d5/d9/d43/d7b/daa/fb5 x:0 0 0 2024-03-21T10:08:52.298 INFO:tasks.workunit.client.1.smithi174.stdout:4/380: symlink d4/d21/l7f 0 2024-03-21T10:08:52.301 INFO:tasks.workunit.client.0.smithi153.stdout:8/534: mknod d5/d9/d43/dd/d2d/d45/d88/d85/cb6 0 2024-03-21T10:08:52.304 INFO:tasks.workunit.client.0.smithi153.stdout:4/495: dread d11/f8d [0,4194304] 0 2024-03-21T10:08:52.306 INFO:tasks.workunit.client.0.smithi153.stdout:8/535: mknod d5/cb7 0 2024-03-21T10:08:52.308 INFO:tasks.workunit.client.0.smithi153.stdout:4/496: creat d11/d2d/d3d/f90 x:0 0 0 2024-03-21T10:08:52.308 INFO:tasks.workunit.client.1.smithi174.stdout:2/337: truncate d5/d25/d27/d2d/d45/d5b/d55/f69 6304737 0 2024-03-21T10:08:52.309 INFO:tasks.workunit.client.1.smithi174.stdout:2/338: stat d5/l33 0 2024-03-21T10:08:52.313 INFO:tasks.workunit.client.1.smithi174.stdout:9/351: dread d2/f1e [0,4194304] 0 2024-03-21T10:08:52.314 INFO:tasks.workunit.client.0.smithi153.stdout:4/497: getdents d11/d18/d50 0 2024-03-21T10:08:52.315 INFO:tasks.workunit.client.1.smithi174.stdout:7/251: dwrite d0/d4/f47 [0,4194304] 0 2024-03-21T10:08:52.317 INFO:tasks.workunit.client.1.smithi174.stdout:2/339: mkdir d5/d25/d27/d2e/d4f/d65/d77 0 2024-03-21T10:08:52.318 INFO:tasks.workunit.client.1.smithi174.stdout:7/252: chown d0/d4/f18 60 1 2024-03-21T10:08:52.318 INFO:tasks.workunit.client.0.smithi153.stdout:4/498: write d11/f6f [2202316,88039] 0 2024-03-21T10:08:52.321 INFO:tasks.workunit.client.0.smithi153.stdout:5/530: sync 2024-03-21T10:08:52.323 INFO:tasks.workunit.client.0.smithi153.stdout:4/499: write d11/d18/d50/d65/f6a [1256567,26796] 0 2024-03-21T10:08:52.326 INFO:tasks.workunit.client.0.smithi153.stdout:5/531: mkdir d4/d8/d3b/d56/da5 0 2024-03-21T10:08:52.327 INFO:tasks.workunit.client.0.smithi153.stdout:4/500: stat d11/d5e 0 2024-03-21T10:08:52.328 INFO:tasks.workunit.client.0.smithi153.stdout:2/572: dwrite dd/d15/d18/d2c/d33/f88 [0,4194304] 0 2024-03-21T10:08:52.329 INFO:tasks.workunit.client.0.smithi153.stdout:3/468: write dd/d1e/d45/f4d [211388,46232] 0 2024-03-21T10:08:52.331 INFO:tasks.workunit.client.1.smithi174.stdout:2/340: link d5/d25/d27/d2d/d45/f6e d5/d25/d27/d2d/d45/d5b/d2f/d4b/f78 0 2024-03-21T10:08:52.336 INFO:tasks.workunit.client.1.smithi174.stdout:3/308: sync 2024-03-21T10:08:52.337 INFO:tasks.workunit.client.1.smithi174.stdout:9/352: dwrite d2/d11/f4a [0,4194304] 0 2024-03-21T10:08:52.338 INFO:tasks.workunit.client.1.smithi174.stdout:7/253: write d0/d4/d10/d1b/f3e [2764662,8319] 0 2024-03-21T10:08:52.340 INFO:tasks.workunit.client.1.smithi174.stdout:3/309: symlink d0/d12/d14/d1a/d27/d4f/d64/l71 0 2024-03-21T10:08:52.341 INFO:tasks.workunit.client.1.smithi174.stdout:2/341: dread d5/f12 [0,4194304] 0 2024-03-21T10:08:52.342 INFO:tasks.workunit.client.1.smithi174.stdout:9/353: unlink d2/d11/d60/f64 0 2024-03-21T10:08:52.343 INFO:tasks.workunit.client.1.smithi174.stdout:7/254: unlink d0/d4/f18 0 2024-03-21T10:08:52.344 INFO:tasks.workunit.client.1.smithi174.stdout:9/354: write d2/d24/f59 [582170,32881] 0 2024-03-21T10:08:52.345 INFO:tasks.workunit.client.1.smithi174.stdout:3/310: creat d0/d12/d14/d1a/d27/d3c/f72 x:0 0 0 2024-03-21T10:08:52.346 INFO:tasks.workunit.client.0.smithi153.stdout:2/573: dread - dd/d15/d5e/fbb zero size 2024-03-21T10:08:52.346 INFO:tasks.workunit.client.0.smithi153.stdout:1/426: truncate d6/d10/d2c/f2e 1176804 0 2024-03-21T10:08:52.346 INFO:tasks.workunit.client.0.smithi153.stdout:5/532: symlink d4/d5/df/d67/d86/d96/la6 0 2024-03-21T10:08:52.348 INFO:tasks.workunit.client.1.smithi174.stdout:7/255: fdatasync d0/d4/f17 0 2024-03-21T10:08:52.348 INFO:tasks.workunit.client.1.smithi174.stdout:2/342: dread d5/d25/d27/f3c [0,4194304] 0 2024-03-21T10:08:52.349 INFO:tasks.workunit.client.1.smithi174.stdout:7/256: stat d0/d4/d10/d44 0 2024-03-21T10:08:52.351 INFO:tasks.workunit.client.1.smithi174.stdout:9/355: creat d2/d11/d1d/f79 x:0 0 0 2024-03-21T10:08:52.351 INFO:tasks.workunit.client.0.smithi153.stdout:2/574: truncate fc 2123955 0 2024-03-21T10:08:52.353 INFO:tasks.workunit.client.1.smithi174.stdout:2/343: write d5/d25/d27/f4e [4666324,127160] 0 2024-03-21T10:08:52.353 INFO:tasks.workunit.client.0.smithi153.stdout:1/427: creat d6/d10/d2c/f7e x:0 0 0 2024-03-21T10:08:52.360 INFO:tasks.workunit.client.0.smithi153.stdout:2/575: creat dd/d15/d3c/da7/d91/dae/d50/d59/fc5 x:0 0 0 2024-03-21T10:08:52.361 INFO:tasks.workunit.client.1.smithi174.stdout:2/344: getdents d5/d25/d27/d2e/d4f/d65 0 2024-03-21T10:08:52.363 INFO:tasks.workunit.client.0.smithi153.stdout:4/501: dread d11/d18/f64 [0,4194304] 0 2024-03-21T10:08:52.365 INFO:tasks.workunit.client.0.smithi153.stdout:4/502: stat d11/d18/l7b 0 2024-03-21T10:08:52.365 INFO:tasks.workunit.client.0.smithi153.stdout:6/691: dwrite d1f/d2b/d33/d40/d5f/f9b [0,4194304] 0 2024-03-21T10:08:52.367 INFO:tasks.workunit.client.0.smithi153.stdout:6/692: write d1f/d2b/d33/d40/fa5 [534046,75783] 0 2024-03-21T10:08:52.368 INFO:tasks.workunit.client.1.smithi174.stdout:9/356: dwrite d2/d11/d1d/f4e [4194304,4194304] 0 2024-03-21T10:08:52.369 INFO:tasks.workunit.client.0.smithi153.stdout:4/503: mknod d11/d2d/d6b/c91 0 2024-03-21T10:08:52.370 INFO:tasks.workunit.client.0.smithi153.stdout:4/504: fdatasync d11/d18/d50/f53 0 2024-03-21T10:08:52.371 INFO:tasks.workunit.client.0.smithi153.stdout:6/693: rename d1f/ff1 to d1f/d99/ff9 0 2024-03-21T10:08:52.375 INFO:tasks.workunit.client.0.smithi153.stdout:6/694: fdatasync d1f/d2b/d33/d40/d65/fde 0 2024-03-21T10:08:52.376 INFO:tasks.workunit.client.1.smithi174.stdout:9/357: write d2/d24/d2e/d63/f6c [722882,63434] 0 2024-03-21T10:08:52.377 INFO:tasks.workunit.client.0.smithi153.stdout:6/695: dread - d1f/d99/dbb/dcf/feb zero size 2024-03-21T10:08:52.382 INFO:tasks.workunit.client.0.smithi153.stdout:6/696: creat d1f/d2b/d33/d40/d65/ffa x:0 0 0 2024-03-21T10:08:52.382 INFO:tasks.workunit.client.1.smithi174.stdout:9/358: mknod d2/d11/c7a 0 2024-03-21T10:08:52.383 INFO:tasks.workunit.client.1.smithi174.stdout:2/345: dwrite d5/d25/d27/f39 [0,4194304] 0 2024-03-21T10:08:52.384 INFO:tasks.workunit.client.0.smithi153.stdout:6/697: creat d1f/d93/dd8/ffb x:0 0 0 2024-03-21T10:08:52.385 INFO:tasks.workunit.client.1.smithi174.stdout:9/359: symlink d2/d24/l7b 0 2024-03-21T10:08:52.388 INFO:tasks.workunit.client.0.smithi153.stdout:6/698: truncate d1f/d93/dd8/d59/f81 825403 0 2024-03-21T10:08:52.391 INFO:tasks.workunit.client.1.smithi174.stdout:9/360: symlink d2/d11/d1d/d4b/d72/l7c 0 2024-03-21T10:08:52.394 INFO:tasks.workunit.client.1.smithi174.stdout:9/361: mkdir d2/d11/d7d 0 2024-03-21T10:08:52.400 INFO:tasks.workunit.client.0.smithi153.stdout:6/699: dwrite d1f/d93/dd8/d59/d70/ddc/ff8 [0,4194304] 0 2024-03-21T10:08:52.404 INFO:tasks.workunit.client.0.smithi153.stdout:6/700: mkdir d1f/d93/dd8/d59/dfc 0 2024-03-21T10:08:52.406 INFO:tasks.workunit.client.0.smithi153.stdout:6/701: creat d1f/d2b/d33/ffd x:0 0 0 2024-03-21T10:08:52.413 INFO:tasks.workunit.client.0.smithi153.stdout:6/702: rename d1f/d99/dbb/dcf/feb to d1f/d2b/d33/d40/d51/d98/ffe 0 2024-03-21T10:08:52.418 INFO:tasks.workunit.client.0.smithi153.stdout:6/703: link d1f/d2b/d33/d40/d4f/d66/f6f d1f/d93/fff 0 2024-03-21T10:08:52.439 INFO:tasks.workunit.client.1.smithi174.stdout:2/346: write d5/d25/d27/d2d/d45/d5b/d55/f69 [2243122,113190] 0 2024-03-21T10:08:52.440 INFO:tasks.workunit.client.0.smithi153.stdout:3/469: write dd/f43 [1770848,539] 0 2024-03-21T10:08:52.440 INFO:tasks.workunit.client.0.smithi153.stdout:2/576: truncate dd/d15/d18/d2c/d4b/d56/d63/f89 3528260 0 2024-03-21T10:08:52.441 INFO:tasks.workunit.client.0.smithi153.stdout:3/470: fsync dd/f25 0 2024-03-21T10:08:52.442 INFO:tasks.workunit.client.1.smithi174.stdout:2/347: mkdir d5/d25/d27/d2d/d79 0 2024-03-21T10:08:52.443 INFO:tasks.workunit.client.0.smithi153.stdout:2/577: dread - dd/d15/d5e/fa4 zero size 2024-03-21T10:08:52.444 INFO:tasks.workunit.client.0.smithi153.stdout:4/505: truncate d11/d2d/f42 743872 0 2024-03-21T10:08:52.448 INFO:tasks.workunit.client.0.smithi153.stdout:3/471: symlink dd/d1e/d2f/la7 0 2024-03-21T10:08:52.453 INFO:tasks.workunit.client.0.smithi153.stdout:2/578: mknod dd/d15/d3c/da7/d91/dae/d50/cc6 0 2024-03-21T10:08:52.453 INFO:tasks.workunit.client.0.smithi153.stdout:4/506: creat d11/d2d/f92 x:0 0 0 2024-03-21T10:08:52.454 INFO:tasks.workunit.client.0.smithi153.stdout:4/507: chown d11/d2d/d57 30 1 2024-03-21T10:08:52.455 INFO:tasks.workunit.client.1.smithi174.stdout:2/348: dwrite d5/d25/d27/f6a [0,4194304] 0 2024-03-21T10:08:52.456 INFO:tasks.workunit.client.0.smithi153.stdout:3/472: rename dd/d1e/d3d/d94/d73/d7f/d91/f9d to dd/d1e/d3d/d94/d73/d7f/d91/fa8 0 2024-03-21T10:08:52.456 INFO:tasks.workunit.client.1.smithi174.stdout:2/349: readlink d5/l5e 0 2024-03-21T10:08:52.460 INFO:tasks.workunit.client.1.smithi174.stdout:2/350: getdents d5/d25/d27/d2d/d79 0 2024-03-21T10:08:52.460 INFO:tasks.workunit.client.0.smithi153.stdout:4/508: mkdir d11/d18/d7e/d93 0 2024-03-21T10:08:52.463 INFO:tasks.workunit.client.0.smithi153.stdout:3/473: mkdir dd/d1e/d3d/d94/d8e/da9 0 2024-03-21T10:08:52.467 INFO:tasks.workunit.client.0.smithi153.stdout:4/509: fdatasync d11/f8d 0 2024-03-21T10:08:52.468 INFO:tasks.workunit.client.0.smithi153.stdout:2/579: read dd/d15/d18/d3a/f3e [86745,56389] 0 2024-03-21T10:08:52.471 INFO:tasks.workunit.client.0.smithi153.stdout:6/704: dwrite d1f/d93/dd8/d59/fab [0,4194304] 0 2024-03-21T10:08:52.474 INFO:tasks.workunit.client.0.smithi153.stdout:3/474: mkdir dd/d1e/d2f/d35/d68/daa 0 2024-03-21T10:08:52.475 INFO:tasks.workunit.client.0.smithi153.stdout:4/510: rmdir d11/d2d/d3d 39 2024-03-21T10:08:52.479 INFO:tasks.workunit.client.0.smithi153.stdout:2/580: mkdir dd/d15/d18/d2c/d33/d98/dc7 0 2024-03-21T10:08:52.480 INFO:tasks.workunit.client.0.smithi153.stdout:2/581: readlink dd/d15/d3c/l75 0 2024-03-21T10:08:52.486 INFO:tasks.workunit.client.0.smithi153.stdout:6/705: link d1f/d93/dd8/d59/d70/fc1 d1f/da0/f100 0 2024-03-21T10:08:52.487 INFO:tasks.workunit.client.0.smithi153.stdout:2/582: dread dd/d15/d5e/fc0 [0,4194304] 0 2024-03-21T10:08:52.492 INFO:tasks.workunit.client.0.smithi153.stdout:4/511: mknod d11/d71/d89/c94 0 2024-03-21T10:08:52.497 INFO:tasks.workunit.client.1.smithi174.stdout:5/317: sync 2024-03-21T10:08:52.498 INFO:tasks.workunit.client.1.smithi174.stdout:1/317: sync 2024-03-21T10:08:52.498 INFO:tasks.workunit.client.0.smithi153.stdout:4/512: unlink d11/d18/d50/f7a 0 2024-03-21T10:08:52.499 INFO:tasks.workunit.client.0.smithi153.stdout:2/583: rename dd/d15/d18/d2c/f3f to dd/d15/d18/fc8 0 2024-03-21T10:08:52.499 INFO:tasks.workunit.client.0.smithi153.stdout:4/513: stat d11/f6f 0 2024-03-21T10:08:52.502 INFO:tasks.workunit.client.0.smithi153.stdout:2/584: dread - dd/d15/d18/d2c/f41 zero size 2024-03-21T10:08:52.502 INFO:tasks.workunit.client.1.smithi174.stdout:1/318: rename d4/fb to d4/d5/d6c/f73 0 2024-03-21T10:08:52.503 INFO:tasks.workunit.client.0.smithi153.stdout:4/514: fdatasync d11/d18/f35 0 2024-03-21T10:08:52.504 INFO:tasks.workunit.client.0.smithi153.stdout:4/515: chown d11/d18/d6d 1854 1 2024-03-21T10:08:52.505 INFO:tasks.workunit.client.0.smithi153.stdout:2/585: dread dd/d15/d18/d3a/d3d/f76 [0,4194304] 0 2024-03-21T10:08:52.508 INFO:tasks.workunit.client.1.smithi174.stdout:5/318: rename d1/l5 to d1/d16/d2e/d31/l6e 0 2024-03-21T10:08:52.509 INFO:tasks.workunit.client.1.smithi174.stdout:1/319: creat d4/d67/f74 x:0 0 0 2024-03-21T10:08:52.510 INFO:tasks.workunit.client.1.smithi174.stdout:5/319: creat d1/d16/d2e/d66/f6f x:0 0 0 2024-03-21T10:08:52.512 INFO:tasks.workunit.client.0.smithi153.stdout:3/475: dread dd/d1e/d2f/f3b [0,4194304] 0 2024-03-21T10:08:52.515 INFO:tasks.workunit.client.1.smithi174.stdout:5/320: write d1/d16/d1c/f23 [1565418,81219] 0 2024-03-21T10:08:52.515 INFO:tasks.workunit.client.0.smithi153.stdout:4/516: creat d11/d18/f95 x:0 0 0 2024-03-21T10:08:52.516 INFO:tasks.workunit.client.1.smithi174.stdout:5/321: fdatasync d1/d49/d58/f5b 0 2024-03-21T10:08:52.517 INFO:tasks.workunit.client.1.smithi174.stdout:1/320: rename d4/d7/d28/c2b to d4/d7/c75 0 2024-03-21T10:08:52.517 INFO:tasks.workunit.client.0.smithi153.stdout:3/476: read - dd/d1e/d2f/d35/d68/f9a zero size 2024-03-21T10:08:52.518 INFO:tasks.workunit.client.0.smithi153.stdout:6/706: dwrite d1f/d2b/d33/d40/d51/f71 [0,4194304] 0 2024-03-21T10:08:52.518 INFO:tasks.workunit.client.1.smithi174.stdout:1/321: fsync d4/d5/f18 0 2024-03-21T10:08:52.519 INFO:tasks.workunit.client.0.smithi153.stdout:4/517: creat d11/f96 x:0 0 0 2024-03-21T10:08:52.519 INFO:tasks.workunit.client.1.smithi174.stdout:5/322: fdatasync d1/d49/d58/f64 0 2024-03-21T10:08:52.520 INFO:tasks.workunit.client.0.smithi153.stdout:3/477: chown dd/c7d 132399 1 2024-03-21T10:08:52.520 INFO:tasks.workunit.client.1.smithi174.stdout:1/322: chown d4/d7/d15/d53 1 1 2024-03-21T10:08:52.523 INFO:tasks.workunit.client.1.smithi174.stdout:5/323: symlink d1/d49/d4f/l70 0 2024-03-21T10:08:52.525 INFO:tasks.workunit.client.1.smithi174.stdout:5/324: write d1/d16/d1c/f34 [2012745,114335] 0 2024-03-21T10:08:52.532 INFO:tasks.workunit.client.1.smithi174.stdout:5/325: rename d1/d16/f46 to d1/f71 0 2024-03-21T10:08:52.534 INFO:tasks.workunit.client.0.smithi153.stdout:2/586: dwrite dd/d15/d18/f24 [4194304,4194304] 0 2024-03-21T10:08:52.536 INFO:tasks.workunit.client.0.smithi153.stdout:2/587: stat dd/d15/d3c/l75 0 2024-03-21T10:08:52.536 INFO:tasks.workunit.client.1.smithi174.stdout:5/326: symlink d1/d2f/l72 0 2024-03-21T10:08:52.539 INFO:tasks.workunit.client.0.smithi153.stdout:2/588: chown dd/d15/d18/d2c/d4b/c54 639 1 2024-03-21T10:08:52.540 INFO:tasks.workunit.client.1.smithi174.stdout:5/327: mkdir d1/d5c/d73 0 2024-03-21T10:08:52.542 INFO:tasks.workunit.client.0.smithi153.stdout:3/478: write dd/f20 [3315684,115504] 0 2024-03-21T10:08:52.546 INFO:tasks.workunit.client.1.smithi174.stdout:5/328: mknod d1/c74 0 2024-03-21T10:08:52.551 INFO:tasks.workunit.client.0.smithi153.stdout:3/479: rmdir dd/d1e/d45/d9f 0 2024-03-21T10:08:52.553 INFO:tasks.workunit.client.0.smithi153.stdout:3/480: dread - dd/d1e/d45/fa0 zero size 2024-03-21T10:08:52.555 INFO:tasks.workunit.client.0.smithi153.stdout:3/481: fsync dd/d1e/d45/d56/fa5 0 2024-03-21T10:08:52.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:52 smithi153 ceph-mon[29117]: pgmap v12: 65 pgs: 65 active+clean; 1.6 GiB data, 4.6 GiB used, 532 GiB / 536 GiB avail; 37 MiB/s rd, 216 MiB/s wr, 417 op/s 2024-03-21T10:08:52.556 INFO:tasks.workunit.client.1.smithi174.stdout:1/323: dwrite d4/d7/f1e [0,4194304] 0 2024-03-21T10:08:52.559 INFO:tasks.workunit.client.0.smithi153.stdout:4/518: dread d11/d2d/f30 [0,4194304] 0 2024-03-21T10:08:52.560 INFO:tasks.workunit.client.0.smithi153.stdout:2/589: dwrite dd/d15/d18/d2c/d33/f9d [0,4194304] 0 2024-03-21T10:08:52.561 INFO:tasks.workunit.client.1.smithi174.stdout:1/324: truncate d4/d7/d33/d43/d51/f64 734249 0 2024-03-21T10:08:52.563 INFO:tasks.workunit.client.0.smithi153.stdout:3/482: dread dd/d1e/d45/f4d [0,4194304] 0 2024-03-21T10:08:52.563 INFO:tasks.workunit.client.1.smithi174.stdout:1/325: write d4/d2d/f34 [1915028,34868] 0 2024-03-21T10:08:52.565 INFO:tasks.workunit.client.1.smithi174.stdout:5/329: dwrite d1/d16/d1c/f23 [0,4194304] 0 2024-03-21T10:08:52.565 INFO:tasks.workunit.client.1.smithi174.stdout:1/326: dread - d4/d7/d28/d59/f66 zero size 2024-03-21T10:08:52.566 INFO:tasks.workunit.client.1.smithi174.stdout:5/330: stat d1/d16/c1e 0 2024-03-21T10:08:52.567 INFO:tasks.workunit.client.0.smithi153.stdout:3/483: write dd/d1e/d2f/f3a [5086522,92372] 0 2024-03-21T10:08:52.568 INFO:tasks.workunit.client.1.smithi174.stdout:1/327: chown d4/d7/d15/f2f 1234 1 2024-03-21T10:08:52.571 INFO:tasks.workunit.client.1.smithi174.stdout:5/331: creat d1/f75 x:0 0 0 2024-03-21T10:08:52.575 INFO:tasks.workunit.client.1.smithi174.stdout:1/328: write d4/d7/d32/f3c [2986275,103677] 0 2024-03-21T10:08:52.577 INFO:tasks.workunit.client.1.smithi174.stdout:5/332: chown d1/d16/d2e/c4e 255770366 1 2024-03-21T10:08:52.579 INFO:tasks.workunit.client.0.smithi153.stdout:2/590: rename dd/d15/d18/d2c/f35 to dd/d15/d18/d2c/d4b/d56/fc9 0 2024-03-21T10:08:52.581 INFO:tasks.workunit.client.1.smithi174.stdout:1/329: rename d4/d5/fd to d4/d2d/f76 0 2024-03-21T10:08:52.581 INFO:tasks.workunit.client.1.smithi174.stdout:5/333: creat d1/d16/d2e/d66/f76 x:0 0 0 2024-03-21T10:08:52.584 INFO:tasks.workunit.client.0.smithi153.stdout:4/519: dwrite f7 [0,4194304] 0 2024-03-21T10:08:52.589 INFO:tasks.workunit.client.1.smithi174.stdout:5/334: getdents d1/d2f 0 2024-03-21T10:08:52.591 INFO:tasks.workunit.client.1.smithi174.stdout:5/335: chown d1/d16/f60 2488083 1 2024-03-21T10:08:52.596 INFO:tasks.workunit.client.0.smithi153.stdout:4/520: mknod d11/d2d/d6b/c97 0 2024-03-21T10:08:52.597 INFO:tasks.workunit.client.0.smithi153.stdout:2/591: getdents dd/d15/d3c/da7 0 2024-03-21T10:08:52.597 INFO:tasks.workunit.client.0.smithi153.stdout:3/484: dwrite dd/d1e/d2f/f3a [0,4194304] 0 2024-03-21T10:08:52.598 INFO:tasks.workunit.client.1.smithi174.stdout:5/336: link d1/d16/d2e/c4d d1/d49/c77 0 2024-03-21T10:08:52.603 INFO:tasks.workunit.client.1.smithi174.stdout:5/337: dread - d1/f75 zero size 2024-03-21T10:08:52.603 INFO:tasks.workunit.client.1.smithi174.stdout:6/252: sync 2024-03-21T10:08:52.604 INFO:tasks.workunit.client.1.smithi174.stdout:0/327: sync 2024-03-21T10:08:52.605 INFO:tasks.workunit.client.0.smithi153.stdout:2/592: chown dd/d15/d18/d3a/d3d/f76 954 1 2024-03-21T10:08:52.605 INFO:tasks.workunit.client.0.smithi153.stdout:4/521: chown d11/d2d/d57 188 1 2024-03-21T10:08:52.605 INFO:tasks.workunit.client.0.smithi153.stdout:3/485: rmdir dd/d1e/d45/d56/d58 39 2024-03-21T10:08:52.606 INFO:tasks.workunit.client.1.smithi174.stdout:0/328: mkdir d9/d21/d31/d3c/d72 0 2024-03-21T10:08:52.606 INFO:tasks.workunit.client.1.smithi174.stdout:0/329: chown d9/d1c/c4e 167 1 2024-03-21T10:08:52.606 INFO:tasks.workunit.client.1.smithi174.stdout:5/338: creat d1/d16/f78 x:0 0 0 2024-03-21T10:08:52.606 INFO:tasks.workunit.client.1.smithi174.stdout:5/339: readlink d1/d16/l19 0 2024-03-21T10:08:52.610 INFO:tasks.workunit.client.0.smithi153.stdout:2/593: mkdir dd/d15/d18/dca 0 2024-03-21T10:08:52.615 INFO:tasks.workunit.client.1.smithi174.stdout:6/253: dwrite d0/d3/d5/f3c [0,4194304] 0 2024-03-21T10:08:52.619 INFO:tasks.workunit.client.0.smithi153.stdout:2/594: rename dd/d15/d3c/da7/d91/dae/f4c to dd/d15/d3c/fcb 0 2024-03-21T10:08:52.622 INFO:tasks.workunit.client.1.smithi174.stdout:5/340: dwrite d1/d49/d58/f5b [0,4194304] 0 2024-03-21T10:08:52.626 INFO:tasks.workunit.client.0.smithi153.stdout:2/595: truncate dd/d15/d18/d2c/d4b/d56/f7b 62406 0 2024-03-21T10:08:52.629 INFO:tasks.workunit.client.1.smithi174.stdout:6/254: getdents d0/d3/d5/d6/d42 0 2024-03-21T10:08:52.630 INFO:tasks.workunit.client.1.smithi174.stdout:1/330: dread d4/d5/f18 [0,4194304] 0 2024-03-21T10:08:52.633 INFO:tasks.workunit.client.0.smithi153.stdout:2/596: write dd/d15/d18/d2c/d4b/d56/d63/f80 [4724983,11744] 0 2024-03-21T10:08:52.637 INFO:tasks.workunit.client.0.smithi153.stdout:2/597: write dd/d15/d3c/da7/d91/dae/d50/d58/fa8 [612738,126890] 0 2024-03-21T10:08:52.638 INFO:tasks.workunit.client.1.smithi174.stdout:5/341: rename d1/d2f/l3a to d1/d16/d2e/l79 0 2024-03-21T10:08:52.640 INFO:tasks.workunit.client.1.smithi174.stdout:6/255: creat d0/d3/d5/f57 x:0 0 0 2024-03-21T10:08:52.640 INFO:tasks.workunit.client.1.smithi174.stdout:5/342: truncate d1/f75 308477 0 2024-03-21T10:08:52.641 INFO:tasks.workunit.client.1.smithi174.stdout:6/256: stat d0/d3/d5/d3b/f55 0 2024-03-21T10:08:52.645 INFO:tasks.workunit.client.0.smithi153.stdout:2/598: truncate dd/d15/d18/d2c/d4b/d56/d63/d8d/d9b/fa5 332097 0 2024-03-21T10:08:52.647 INFO:tasks.workunit.client.1.smithi174.stdout:1/331: dread d4/d7/f1e [0,4194304] 0 2024-03-21T10:08:52.648 INFO:tasks.workunit.client.0.smithi153.stdout:3/486: write dd/f15 [321423,6714] 0 2024-03-21T10:08:52.649 INFO:tasks.workunit.client.0.smithi153.stdout:4/522: dread d11/f44 [0,4194304] 0 2024-03-21T10:08:52.651 INFO:tasks.workunit.client.0.smithi153.stdout:3/487: chown dd/d1e/d2f/f37 104 1 2024-03-21T10:08:52.655 INFO:tasks.workunit.client.0.smithi153.stdout:2/599: creat dd/d15/d3c/da7/d91/dae/d50/fcc x:0 0 0 2024-03-21T10:08:52.656 INFO:tasks.workunit.client.0.smithi153.stdout:3/488: read dd/d1e/d3d/d94/d73/f7c [779734,67831] 0 2024-03-21T10:08:52.660 INFO:tasks.workunit.client.0.smithi153.stdout:2/600: rmdir dd/d15/d5e/db3 39 2024-03-21T10:08:52.661 INFO:tasks.workunit.client.0.smithi153.stdout:3/489: mkdir dd/d41/dab 0 2024-03-21T10:08:52.664 INFO:tasks.workunit.client.0.smithi153.stdout:2/601: write dd/d15/d18/d2c/d4b/f66 [1922724,120773] 0 2024-03-21T10:08:52.670 INFO:tasks.workunit.client.0.smithi153.stdout:3/490: rename f7 to dd/d1e/d2f/d35/d68/daa/fac 0 2024-03-21T10:08:52.670 INFO:tasks.workunit.client.0.smithi153.stdout:4/523: dread d11/d18/f4b [4194304,4194304] 0 2024-03-21T10:08:52.671 INFO:tasks.workunit.client.1.smithi174.stdout:5/343: getdents d1/d16 0 2024-03-21T10:08:52.672 INFO:tasks.workunit.client.0.smithi153.stdout:3/491: read f8 [7656483,99561] 0 2024-03-21T10:08:52.673 INFO:tasks.workunit.client.0.smithi153.stdout:2/602: symlink dd/d15/d3c/da7/d91/dae/d50/d58/d79/lcd 0 2024-03-21T10:08:52.674 INFO:tasks.workunit.client.1.smithi174.stdout:6/257: truncate d0/d3/d5/df/f2e 2361668 0 2024-03-21T10:08:52.674 INFO:tasks.workunit.client.0.smithi153.stdout:3/492: stat dd/d1e/d2f/c34 0 2024-03-21T10:08:52.682 INFO:tasks.workunit.client.1.smithi174.stdout:1/332: dwrite d4/d7/d33/d43/f47 [0,4194304] 0 2024-03-21T10:08:52.682 INFO:tasks.workunit.client.0.smithi153.stdout:3/493: fsync dd/d1e/d2f/d35/d68/daa/fac 0 2024-03-21T10:08:52.685 INFO:tasks.workunit.client.1.smithi174.stdout:1/333: stat d4/d7/d33/d43/l3f 0 2024-03-21T10:08:52.686 INFO:tasks.workunit.client.1.smithi174.stdout:6/258: dread d0/d3/d5/df/d16/f27 [0,4194304] 0 2024-03-21T10:08:52.688 INFO:tasks.workunit.client.0.smithi153.stdout:4/524: dwrite d11/f20 [4194304,4194304] 0 2024-03-21T10:08:52.688 INFO:tasks.workunit.client.1.smithi174.stdout:1/334: unlink d4/d7/d15/l45 0 2024-03-21T10:08:52.689 INFO:tasks.workunit.client.1.smithi174.stdout:5/344: dwrite d1/f11 [0,4194304] 0 2024-03-21T10:08:52.691 INFO:tasks.workunit.client.1.smithi174.stdout:6/259: creat d0/d3/d5/d6/d42/f58 x:0 0 0 2024-03-21T10:08:52.692 INFO:tasks.workunit.client.1.smithi174.stdout:1/335: read d4/d7/f29 [4033920,15041] 0 2024-03-21T10:08:52.693 INFO:tasks.workunit.client.0.smithi153.stdout:7/424: sync 2024-03-21T10:08:52.695 INFO:tasks.workunit.client.0.smithi153.stdout:4/525: link d11/f1e d11/d18/d7e/d93/f98 0 2024-03-21T10:08:52.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:52 smithi174 ceph-mon[37496]: pgmap v12: 65 pgs: 65 active+clean; 1.6 GiB data, 4.6 GiB used, 532 GiB / 536 GiB avail; 37 MiB/s rd, 216 MiB/s wr, 417 op/s 2024-03-21T10:08:52.701 INFO:tasks.workunit.client.1.smithi174.stdout:5/345: getdents d1/d16/d2e/d31 0 2024-03-21T10:08:52.702 INFO:tasks.workunit.client.0.smithi153.stdout:3/494: truncate dd/d1e/d3d/d94/d73/f7c 81813 0 2024-03-21T10:08:52.705 INFO:tasks.workunit.client.0.smithi153.stdout:4/526: truncate d11/d18/f95 603780 0 2024-03-21T10:08:52.707 INFO:tasks.workunit.client.1.smithi174.stdout:5/346: write d1/f71 [800758,88458] 0 2024-03-21T10:08:52.711 INFO:tasks.workunit.client.1.smithi174.stdout:5/347: creat d1/d5c/f7a x:0 0 0 2024-03-21T10:08:52.712 INFO:tasks.workunit.client.0.smithi153.stdout:3/495: creat dd/d1e/d45/d56/d58/fad x:0 0 0 2024-03-21T10:08:52.713 INFO:tasks.workunit.client.1.smithi174.stdout:1/336: dread d4/d7/f1e [0,4194304] 0 2024-03-21T10:08:52.716 INFO:tasks.workunit.client.1.smithi174.stdout:6/260: dwrite d0/d3/d5/df/d16/f2b [0,4194304] 0 2024-03-21T10:08:52.718 INFO:tasks.workunit.client.1.smithi174.stdout:1/337: write d4/d7/d33/d43/f5d [96794,1242] 0 2024-03-21T10:08:52.719 INFO:tasks.workunit.client.0.smithi153.stdout:3/496: mkdir dd/d1e/d3d/d53/dae 0 2024-03-21T10:08:52.722 INFO:tasks.workunit.client.1.smithi174.stdout:1/338: read - d4/d7/d15/d53/f6a zero size 2024-03-21T10:08:52.722 INFO:tasks.workunit.client.1.smithi174.stdout:6/261: creat d0/d3/d20/d3d/f59 x:0 0 0 2024-03-21T10:08:52.725 INFO:tasks.workunit.client.1.smithi174.stdout:8/276: sync 2024-03-21T10:08:52.726 INFO:tasks.workunit.client.0.smithi153.stdout:3/497: read dd/d1e/d2f/d35/f5a [1076558,29074] 0 2024-03-21T10:08:52.729 INFO:tasks.workunit.client.1.smithi174.stdout:1/339: fsync d4/d7/d33/f36 0 2024-03-21T10:08:52.731 INFO:tasks.workunit.client.1.smithi174.stdout:8/277: write d9/d17/f42 [3482506,4673] 0 2024-03-21T10:08:52.732 INFO:tasks.workunit.client.0.smithi153.stdout:4/527: truncate d11/d18/d50/d65/f6a 3598572 0 2024-03-21T10:08:52.734 INFO:tasks.workunit.client.1.smithi174.stdout:5/348: dwrite d1/f29 [0,4194304] 0 2024-03-21T10:08:52.738 INFO:tasks.workunit.client.0.smithi153.stdout:4/528: chown d11/f6f 147060075 1 2024-03-21T10:08:52.738 INFO:tasks.workunit.client.1.smithi174.stdout:5/349: fsync d1/d16/f5f 0 2024-03-21T10:08:52.740 INFO:tasks.workunit.client.0.smithi153.stdout:3/498: mkdir dd/d1e/d2f/d35/daf 0 2024-03-21T10:08:52.744 INFO:tasks.workunit.client.0.smithi153.stdout:7/425: dread d4/d2a/f5b [0,4194304] 0 2024-03-21T10:08:52.746 INFO:tasks.workunit.client.1.smithi174.stdout:1/340: dread d4/d7/d33/d43/f47 [0,4194304] 0 2024-03-21T10:08:52.748 INFO:tasks.workunit.client.0.smithi153.stdout:3/499: creat dd/d1e/fb0 x:0 0 0 2024-03-21T10:08:52.753 INFO:tasks.workunit.client.1.smithi174.stdout:6/262: dread d0/d3/d5/f1c [0,4194304] 0 2024-03-21T10:08:52.754 INFO:tasks.workunit.client.1.smithi174.stdout:5/350: dwrite d1/f11 [0,4194304] 0 2024-03-21T10:08:52.756 INFO:tasks.workunit.client.0.smithi153.stdout:3/500: truncate dd/d41/f6e 571936 0 2024-03-21T10:08:52.761 INFO:tasks.workunit.client.0.smithi153.stdout:7/426: getdents d4/d2a/d40 0 2024-03-21T10:08:52.762 INFO:tasks.workunit.client.1.smithi174.stdout:6/263: link d0/d3/d5/df/f22 d0/d3/f5a 0 2024-03-21T10:08:52.765 INFO:tasks.workunit.client.1.smithi174.stdout:6/264: getdents d0/d3/d5/d6 0 2024-03-21T10:08:52.766 INFO:tasks.workunit.client.1.smithi174.stdout:1/341: dread d4/d5/f8 [0,4194304] 0 2024-03-21T10:08:52.769 INFO:tasks.workunit.client.1.smithi174.stdout:3/311: sync 2024-03-21T10:08:52.769 INFO:tasks.workunit.client.1.smithi174.stdout:9/362: sync 2024-03-21T10:08:52.769 INFO:tasks.workunit.client.1.smithi174.stdout:7/257: sync 2024-03-21T10:08:52.769 INFO:tasks.workunit.client.1.smithi174.stdout:4/381: sync 2024-03-21T10:08:52.769 INFO:tasks.workunit.client.1.smithi174.stdout:2/351: sync 2024-03-21T10:08:52.769 INFO:tasks.workunit.client.1.smithi174.stdout:0/330: sync 2024-03-21T10:08:52.770 INFO:tasks.workunit.client.0.smithi153.stdout:4/529: dwrite d11/f8d [0,4194304] 0 2024-03-21T10:08:52.771 INFO:tasks.workunit.client.1.smithi174.stdout:3/312: dread - d0/f5f zero size 2024-03-21T10:08:52.771 INFO:tasks.workunit.client.0.smithi153.stdout:7/427: link d4/d1b/d83/f8f d4/d2a/d3b/d81/f90 0 2024-03-21T10:08:52.773 INFO:tasks.workunit.client.1.smithi174.stdout:4/382: dread - d4/d21/f7b zero size 2024-03-21T10:08:52.775 INFO:tasks.workunit.client.1.smithi174.stdout:7/258: chown d0/d4/f42 101186 1 2024-03-21T10:08:52.775 INFO:tasks.workunit.client.1.smithi174.stdout:0/331: write d9/fc [1499680,111741] 0 2024-03-21T10:08:52.776 INFO:tasks.workunit.client.0.smithi153.stdout:7/428: symlink d4/d1b/d83/l91 0 2024-03-21T10:08:52.776 INFO:tasks.workunit.client.1.smithi174.stdout:6/265: getdents d0/d3/d20 0 2024-03-21T10:08:52.778 INFO:tasks.workunit.client.0.smithi153.stdout:7/429: read d4/d2a/d40/f4c [690518,33286] 0 2024-03-21T10:08:52.778 INFO:tasks.workunit.client.1.smithi174.stdout:5/351: truncate d1/fd 3370321 0 2024-03-21T10:08:52.781 INFO:tasks.workunit.client.1.smithi174.stdout:4/383: write d4/d21/f49 [158569,129372] 0 2024-03-21T10:08:52.781 INFO:tasks.workunit.client.1.smithi174.stdout:7/259: fdatasync d0/d4/f2d 0 2024-03-21T10:08:52.782 INFO:tasks.workunit.client.1.smithi174.stdout:1/342: rename d4/d7/d33/d43/d51/d60 to d4/d7/d33/d43/d77 0 2024-03-21T10:08:52.786 INFO:tasks.workunit.client.1.smithi174.stdout:9/363: mkdir d2/d24/d2e/d7e 0 2024-03-21T10:08:52.789 INFO:tasks.workunit.client.1.smithi174.stdout:8/278: dread d9/f30 [0,4194304] 0 2024-03-21T10:08:52.790 INFO:tasks.workunit.client.1.smithi174.stdout:9/364: chown d2/d24/f74 1 1 2024-03-21T10:08:52.793 INFO:tasks.workunit.client.1.smithi174.stdout:2/352: truncate d5/d25/d27/d2d/d45/d5b/d55/f58 282796 0 2024-03-21T10:08:52.796 INFO:tasks.workunit.client.1.smithi174.stdout:8/279: chown d9/d17/f20 204483683 1 2024-03-21T10:08:52.798 INFO:tasks.workunit.client.1.smithi174.stdout:6/266: symlink d0/l5b 0 2024-03-21T10:08:52.798 INFO:tasks.workunit.client.0.smithi153.stdout:7/430: dwrite d4/d14/d2f/d78/f71 [0,4194304] 0 2024-03-21T10:08:52.799 INFO:tasks.workunit.client.0.smithi153.stdout:3/501: dread dd/f25 [0,4194304] 0 2024-03-21T10:08:52.799 INFO:tasks.workunit.client.1.smithi174.stdout:7/260: mkdir d0/d53 0 2024-03-21T10:08:52.801 INFO:tasks.workunit.client.0.smithi153.stdout:7/431: chown d4/d2a/d40/f87 6068 1 2024-03-21T10:08:52.806 INFO:tasks.workunit.client.0.smithi153.stdout:7/432: readlink d4/l65 0 2024-03-21T10:08:52.813 INFO:tasks.workunit.client.1.smithi174.stdout:4/384: mknod d4/d21/d31/d3a/d67/d76/c80 0 2024-03-21T10:08:52.814 INFO:tasks.workunit.client.0.smithi153.stdout:7/433: symlink d4/d11/d56/l92 0 2024-03-21T10:08:52.817 INFO:tasks.workunit.client.0.smithi153.stdout:3/502: write dd/d41/f6e [1423900,50712] 0 2024-03-21T10:08:52.818 INFO:tasks.workunit.client.0.smithi153.stdout:3/503: dread - dd/d1e/d45/fa0 zero size 2024-03-21T10:08:52.819 INFO:tasks.workunit.client.1.smithi174.stdout:8/280: creat d9/d17/f5a x:0 0 0 2024-03-21T10:08:52.819 INFO:tasks.workunit.client.1.smithi174.stdout:3/313: dwrite d0/f2 [4194304,4194304] 0 2024-03-21T10:08:52.820 INFO:tasks.workunit.client.0.smithi153.stdout:4/530: dread d11/d18/f1d [4194304,4194304] 0 2024-03-21T10:08:52.821 INFO:tasks.workunit.client.0.smithi153.stdout:3/504: write dd/d41/f71 [562142,111994] 0 2024-03-21T10:08:52.821 INFO:tasks.workunit.client.1.smithi174.stdout:2/353: truncate d5/d25/d27/d2d/d45/d5b/f22 1682903 0 2024-03-21T10:08:52.822 INFO:tasks.workunit.client.1.smithi174.stdout:7/261: mknod d0/d4/c54 0 2024-03-21T10:08:52.823 INFO:tasks.workunit.client.0.smithi153.stdout:4/531: fdatasync d11/d2d/f30 0 2024-03-21T10:08:52.825 INFO:tasks.workunit.client.1.smithi174.stdout:4/385: fdatasync d4/d19/f65 0 2024-03-21T10:08:52.826 INFO:tasks.workunit.client.0.smithi153.stdout:3/505: mkdir dd/d1e/d3d/d94/d73/d7f/d91/db1 0 2024-03-21T10:08:52.828 INFO:tasks.workunit.client.1.smithi174.stdout:1/343: dwrite d4/d5/f72 [0,4194304] 0 2024-03-21T10:08:52.829 INFO:tasks.workunit.client.1.smithi174.stdout:5/352: dwrite d1/d16/f5f [0,4194304] 0 2024-03-21T10:08:52.831 INFO:tasks.workunit.client.1.smithi174.stdout:9/365: dread d2/d11/f4a [0,4194304] 0 2024-03-21T10:08:52.831 INFO:tasks.workunit.client.0.smithi153.stdout:3/506: symlink dd/d1e/d3d/lb2 0 2024-03-21T10:08:52.833 INFO:tasks.workunit.client.0.smithi153.stdout:4/532: link d11/d18/f41 d11/d2d/d3d/f99 0 2024-03-21T10:08:52.834 INFO:tasks.workunit.client.1.smithi174.stdout:9/366: dread - d2/d11/f36 zero size 2024-03-21T10:08:52.836 INFO:tasks.workunit.client.0.smithi153.stdout:3/507: dread - dd/f4c zero size 2024-03-21T10:08:52.836 INFO:tasks.workunit.client.0.smithi153.stdout:4/533: chown d11/d18/d50/f53 101583133 1 2024-03-21T10:08:52.838 INFO:tasks.workunit.client.0.smithi153.stdout:3/508: dread - dd/d1e/d45/f99 zero size 2024-03-21T10:08:52.842 INFO:tasks.workunit.client.0.smithi153.stdout:3/509: symlink dd/lb3 0 2024-03-21T10:08:52.846 INFO:tasks.workunit.client.1.smithi174.stdout:3/314: mkdir d0/d12/d14/d1a/d27/d3c/d73 0 2024-03-21T10:08:52.846 INFO:tasks.workunit.client.1.smithi174.stdout:6/267: dread d0/d3/d5/df/d16/f2b [0,4194304] 0 2024-03-21T10:08:52.846 INFO:tasks.workunit.client.1.smithi174.stdout:1/344: unlink d4/d7/f5b 0 2024-03-21T10:08:52.847 INFO:tasks.workunit.client.0.smithi153.stdout:4/534: rename d11/l48 to d11/d2d/d3d/l9a 0 2024-03-21T10:08:52.849 INFO:tasks.workunit.client.1.smithi174.stdout:0/332: dread d9/d21/d23/f41 [0,4194304] 0 2024-03-21T10:08:52.849 INFO:tasks.workunit.client.1.smithi174.stdout:7/262: rmdir d0/d4/d38/d4f 0 2024-03-21T10:08:52.850 INFO:tasks.workunit.client.1.smithi174.stdout:2/354: rename d5/d25/d27/d2e/f3f to d5/d25/d27/d2d/d45/d5b/d2f/d68/f7a 0 2024-03-21T10:08:52.851 INFO:tasks.workunit.client.1.smithi174.stdout:1/345: write d4/d7/d15/d4f/f61 [898356,95865] 0 2024-03-21T10:08:52.854 INFO:tasks.workunit.client.0.smithi153.stdout:3/510: creat dd/d1e/d2f/d35/daf/fb4 x:0 0 0 2024-03-21T10:08:52.854 INFO:tasks.workunit.client.0.smithi153.stdout:4/535: truncate d11/d2d/f43 637297 0 2024-03-21T10:08:52.854 INFO:tasks.workunit.client.0.smithi153.stdout:3/511: fdatasync dd/d41/f6e 0 2024-03-21T10:08:52.857 INFO:tasks.workunit.client.0.smithi153.stdout:4/536: creat d11/d18/d6d/f9b x:0 0 0 2024-03-21T10:08:52.857 INFO:tasks.workunit.client.0.smithi153.stdout:3/512: dread - dd/d1e/d2f/d35/daf/fb4 zero size 2024-03-21T10:08:52.858 INFO:tasks.workunit.client.1.smithi174.stdout:4/386: getdents d4/d21/d24/d28/d38 0 2024-03-21T10:08:52.858 INFO:tasks.workunit.client.1.smithi174.stdout:8/281: dread d9/f14 [0,4194304] 0 2024-03-21T10:08:52.859 INFO:tasks.workunit.client.0.smithi153.stdout:4/537: read d11/d18/f41 [2715760,115101] 0 2024-03-21T10:08:52.860 INFO:tasks.workunit.client.1.smithi174.stdout:2/355: mkdir d5/d25/d27/d2e/d7b 0 2024-03-21T10:08:52.862 INFO:tasks.workunit.client.1.smithi174.stdout:1/346: mkdir d4/d7/d15/d53/d78 0 2024-03-21T10:08:52.862 INFO:tasks.workunit.client.0.smithi153.stdout:3/513: write dd/d1e/d45/d56/d58/f8b [1376294,22813] 0 2024-03-21T10:08:52.865 INFO:tasks.workunit.client.1.smithi174.stdout:0/333: symlink d9/d21/d31/d3c/d72/l73 0 2024-03-21T10:08:52.866 INFO:tasks.workunit.client.1.smithi174.stdout:6/268: getdents d0/d3/d20 0 2024-03-21T10:08:52.868 INFO:tasks.workunit.client.1.smithi174.stdout:4/387: unlink d4/d21/f37 0 2024-03-21T10:08:52.869 INFO:tasks.workunit.client.0.smithi153.stdout:3/514: creat dd/d1e/d2f/d35/fb5 x:0 0 0 2024-03-21T10:08:52.869 INFO:tasks.workunit.client.0.smithi153.stdout:3/515: stat dd/f27 0 2024-03-21T10:08:52.870 INFO:tasks.workunit.client.1.smithi174.stdout:5/353: dwrite d1/d16/d2e/d31/f4b [0,4194304] 0 2024-03-21T10:08:52.873 INFO:tasks.workunit.client.1.smithi174.stdout:9/367: dwrite d2/d24/d2e/f69 [0,4194304] 0 2024-03-21T10:08:52.874 INFO:tasks.workunit.client.0.smithi153.stdout:3/516: rmdir dd/d1e/d3d/d94/d8e/da9 0 2024-03-21T10:08:52.876 INFO:tasks.workunit.client.1.smithi174.stdout:4/388: write d4/d21/d24/d28/f39 [950907,58763] 0 2024-03-21T10:08:52.876 INFO:tasks.workunit.client.1.smithi174.stdout:3/315: rmdir d0/d12/d14 39 2024-03-21T10:08:52.877 INFO:tasks.workunit.client.1.smithi174.stdout:5/354: chown d1/d2f/d37/l3c 337363 1 2024-03-21T10:08:52.877 INFO:tasks.workunit.client.0.smithi153.stdout:3/517: read - dd/d1e/d45/d56/f9e zero size 2024-03-21T10:08:52.879 INFO:tasks.workunit.client.1.smithi174.stdout:0/334: dread f7 [0,4194304] 0 2024-03-21T10:08:52.879 INFO:tasks.workunit.client.1.smithi174.stdout:1/347: write d4/d7/d15/d53/f54 [1207296,22387] 0 2024-03-21T10:08:52.879 INFO:tasks.workunit.client.1.smithi174.stdout:2/356: creat d5/d25/d27/d2d/d45/d5b/d2f/d4b/f7c x:0 0 0 2024-03-21T10:08:52.881 INFO:tasks.workunit.client.0.smithi153.stdout:3/518: dread dd/d1e/d2f/f90 [0,4194304] 0 2024-03-21T10:08:52.882 INFO:tasks.workunit.client.1.smithi174.stdout:3/316: chown d0/f5f 0 1 2024-03-21T10:08:52.883 INFO:tasks.workunit.client.1.smithi174.stdout:0/335: write d9/d21/f37 [276,55775] 0 2024-03-21T10:08:52.887 INFO:tasks.workunit.client.1.smithi174.stdout:9/368: stat d2/d11/d1d/d5e 0 2024-03-21T10:08:52.890 INFO:tasks.workunit.client.1.smithi174.stdout:4/389: creat d4/d21/d24/d28/f81 x:0 0 0 2024-03-21T10:08:52.892 INFO:tasks.workunit.client.1.smithi174.stdout:2/357: unlink d5/fa 0 2024-03-21T10:08:52.895 INFO:tasks.workunit.client.1.smithi174.stdout:6/269: creat d0/d3/d5/f5c x:0 0 0 2024-03-21T10:08:52.897 INFO:tasks.workunit.client.1.smithi174.stdout:1/348: symlink d4/d7/d28/l79 0 2024-03-21T10:08:52.900 INFO:tasks.workunit.client.1.smithi174.stdout:3/317: readlink d0/l4 0 2024-03-21T10:08:52.900 INFO:tasks.workunit.client.1.smithi174.stdout:0/336: dread - d9/d21/f2b zero size 2024-03-21T10:08:52.901 INFO:tasks.workunit.client.1.smithi174.stdout:0/337: chown d9/d1c 29 1 2024-03-21T10:08:52.904 INFO:tasks.workunit.client.1.smithi174.stdout:0/338: write d9/d21/f26 [920046,16397] 0 2024-03-21T10:08:52.904 INFO:tasks.workunit.client.1.smithi174.stdout:8/282: dwrite d9/f1c [0,4194304] 0 2024-03-21T10:08:52.906 INFO:tasks.workunit.client.0.smithi153.stdout:0/440: sync 2024-03-21T10:08:52.908 INFO:tasks.workunit.client.1.smithi174.stdout:1/349: unlink d4/d7/f1e 0 2024-03-21T10:08:52.911 INFO:tasks.workunit.client.0.smithi153.stdout:4/538: dread d11/d18/d54/f68 [0,4194304] 0 2024-03-21T10:08:52.913 INFO:tasks.workunit.client.1.smithi174.stdout:9/369: unlink d2/d11/d1d/d2a/c6b 0 2024-03-21T10:08:52.913 INFO:tasks.workunit.client.1.smithi174.stdout:2/358: symlink d5/d25/d27/d74/l7d 0 2024-03-21T10:08:52.914 INFO:tasks.workunit.client.1.smithi174.stdout:5/355: link d1/d2f/l44 d1/d49/d4f/d51/l7b 0 2024-03-21T10:08:52.916 INFO:tasks.workunit.client.0.smithi153.stdout:4/539: mkdir d11/d2d/d6b/d9c 0 2024-03-21T10:08:52.917 INFO:tasks.workunit.client.1.smithi174.stdout:0/339: unlink d9/d21/d31/f47 0 2024-03-21T10:08:52.917 INFO:tasks.workunit.client.1.smithi174.stdout:0/340: readlink l1 0 2024-03-21T10:08:52.924 INFO:tasks.workunit.client.1.smithi174.stdout:3/318: rename d0/d12/d14/d1a/d27/f2a to d0/d12/d14/d1a/d27/d4f/d64/f74 0 2024-03-21T10:08:52.927 INFO:tasks.workunit.client.0.smithi153.stdout:3/519: dwrite dd/f4c [0,4194304] 0 2024-03-21T10:08:52.927 INFO:tasks.workunit.client.0.smithi153.stdout:0/441: dwrite d4/db/f11 [4194304,4194304] 0 2024-03-21T10:08:52.928 INFO:tasks.workunit.client.1.smithi174.stdout:8/283: mkdir d9/d53/d5b 0 2024-03-21T10:08:52.928 INFO:tasks.workunit.client.1.smithi174.stdout:0/341: symlink d9/d21/d23/l74 0 2024-03-21T10:08:52.932 INFO:tasks.workunit.client.1.smithi174.stdout:3/319: symlink d0/d12/d46/l75 0 2024-03-21T10:08:52.933 INFO:tasks.workunit.client.1.smithi174.stdout:0/342: mkdir d9/d21/d75 0 2024-03-21T10:08:52.934 INFO:tasks.workunit.client.1.smithi174.stdout:3/320: readlink d0/d12/d46/l75 0 2024-03-21T10:08:52.934 INFO:tasks.workunit.client.1.smithi174.stdout:8/284: mknod d9/d17/d19/d31/c5c 0 2024-03-21T10:08:52.934 INFO:tasks.workunit.client.1.smithi174.stdout:4/390: dwrite d4/f8 [0,4194304] 0 2024-03-21T10:08:52.935 INFO:tasks.workunit.client.0.smithi153.stdout:0/442: mknod d4/d16/d3e/d5b/c88 0 2024-03-21T10:08:52.935 INFO:tasks.workunit.client.1.smithi174.stdout:8/285: stat d9 0 2024-03-21T10:08:52.936 INFO:tasks.workunit.client.0.smithi153.stdout:3/520: creat dd/d1e/d2f/d35/d68/fb6 x:0 0 0 2024-03-21T10:08:52.937 INFO:tasks.workunit.client.0.smithi153.stdout:0/443: truncate d4/d4c/f7a 1920030 0 2024-03-21T10:08:52.943 INFO:tasks.workunit.client.0.smithi153.stdout:0/444: stat d4/d4c/d83/d22/c52 0 2024-03-21T10:08:52.943 INFO:tasks.workunit.client.1.smithi174.stdout:0/343: mkdir d9/d1c/d76 0 2024-03-21T10:08:52.944 INFO:tasks.workunit.client.1.smithi174.stdout:4/391: mkdir d4/d21/d31/d3a/d67/d76/d82 0 2024-03-21T10:08:52.944 INFO:tasks.workunit.client.0.smithi153.stdout:3/521: symlink dd/d1e/d3d/d94/d73/lb7 0 2024-03-21T10:08:52.945 INFO:tasks.workunit.client.1.smithi174.stdout:9/370: dwrite d2/d11/f2c [0,4194304] 0 2024-03-21T10:08:52.945 INFO:tasks.workunit.client.1.smithi174.stdout:3/321: chown d0/c5 64694320 1 2024-03-21T10:08:52.946 INFO:tasks.workunit.client.0.smithi153.stdout:4/540: dwrite d11/d2d/f43 [0,4194304] 0 2024-03-21T10:08:52.946 INFO:tasks.workunit.client.0.smithi153.stdout:0/445: symlink d4/l89 0 2024-03-21T10:08:52.947 INFO:tasks.workunit.client.1.smithi174.stdout:0/344: rmdir d9/d35/d4f/d56 39 2024-03-21T10:08:52.948 INFO:tasks.workunit.client.0.smithi153.stdout:4/541: stat d11/d18/d50/d65/l8e 0 2024-03-21T10:08:52.950 INFO:tasks.workunit.client.1.smithi174.stdout:0/345: chown d9/d1c/f20 62 1 2024-03-21T10:08:52.952 INFO:tasks.workunit.client.1.smithi174.stdout:0/346: chown d9/d21/d31/f57 27677318 1 2024-03-21T10:08:52.957 INFO:tasks.workunit.client.1.smithi174.stdout:9/371: creat d2/f7f x:0 0 0 2024-03-21T10:08:52.957 INFO:tasks.workunit.client.1.smithi174.stdout:4/392: mknod d4/d21/d24/c83 0 2024-03-21T10:08:52.958 INFO:tasks.workunit.client.0.smithi153.stdout:4/542: stat d11/d5e/f7d 0 2024-03-21T10:08:52.963 INFO:tasks.workunit.client.1.smithi174.stdout:4/393: dread - d4/d21/d31/d6a/f6e zero size 2024-03-21T10:08:52.964 INFO:tasks.workunit.client.1.smithi174.stdout:8/286: dwrite d9/d2a/f3e [0,4194304] 0 2024-03-21T10:08:52.968 INFO:tasks.workunit.client.1.smithi174.stdout:8/287: write d9/d17/f20 [1237882,44257] 0 2024-03-21T10:08:52.969 INFO:tasks.workunit.client.0.smithi153.stdout:3/522: dwrite dd/d1e/d6b/fa1 [0,4194304] 0 2024-03-21T10:08:52.970 INFO:tasks.workunit.client.1.smithi174.stdout:5/356: dread d1/f2c [0,4194304] 0 2024-03-21T10:08:52.973 INFO:tasks.workunit.client.1.smithi174.stdout:4/394: mknod d4/d19/c84 0 2024-03-21T10:08:52.975 INFO:tasks.workunit.client.0.smithi153.stdout:3/523: mkdir dd/d1e/d3d/d94/db8 0 2024-03-21T10:08:52.979 INFO:tasks.workunit.client.1.smithi174.stdout:5/357: creat d1/d16/d2e/d66/f7c x:0 0 0 2024-03-21T10:08:52.982 INFO:tasks.workunit.client.0.smithi153.stdout:3/524: rename dd/c39 to dd/d1e/d6b/cb9 0 2024-03-21T10:08:52.982 INFO:tasks.workunit.client.1.smithi174.stdout:4/395: rename d4/l16 to d4/d21/d31/d3a/d77/l85 0 2024-03-21T10:08:52.984 INFO:tasks.workunit.client.1.smithi174.stdout:0/347: dwrite d9/d21/f6e [0,4194304] 0 2024-03-21T10:08:52.988 INFO:tasks.workunit.client.1.smithi174.stdout:4/396: write d4/d21/f7b [172303,66870] 0 2024-03-21T10:08:52.988 INFO:tasks.workunit.client.0.smithi153.stdout:3/525: creat dd/d1e/d3d/d53/fba x:0 0 0 2024-03-21T10:08:52.992 INFO:tasks.workunit.client.1.smithi174.stdout:4/397: chown d4/l1a 2416092 1 2024-03-21T10:08:52.992 INFO:tasks.workunit.client.1.smithi174.stdout:5/358: write d1/ff [7847237,126657] 0 2024-03-21T10:08:52.993 INFO:tasks.workunit.client.1.smithi174.stdout:0/348: truncate d9/d58/f63 439547 0 2024-03-21T10:08:52.993 INFO:tasks.workunit.client.0.smithi153.stdout:3/526: chown dd/d1e/d2f/d35/l82 308304907 1 2024-03-21T10:08:52.997 INFO:tasks.workunit.client.1.smithi174.stdout:0/349: write d9/d35/f46 [835077,96151] 0 2024-03-21T10:08:52.999 INFO:tasks.workunit.client.0.smithi153.stdout:8/536: sync 2024-03-21T10:08:52.999 INFO:tasks.workunit.client.0.smithi153.stdout:9/526: sync 2024-03-21T10:08:52.999 INFO:tasks.workunit.client.0.smithi153.stdout:5/533: sync 2024-03-21T10:08:53.000 INFO:tasks.workunit.client.0.smithi153.stdout:3/527: mkdir dd/d41/d80/dbb 0 2024-03-21T10:08:53.001 INFO:tasks.workunit.client.1.smithi174.stdout:8/288: dwrite d9/d17/d19/d31/f3b [0,4194304] 0 2024-03-21T10:08:53.008 INFO:tasks.workunit.client.1.smithi174.stdout:4/398: creat d4/d21/d24/d28/d79/f86 x:0 0 0 2024-03-21T10:08:53.008 INFO:tasks.workunit.client.1.smithi174.stdout:0/350: stat d9/c2c 0 2024-03-21T10:08:53.008 INFO:tasks.workunit.client.0.smithi153.stdout:0/446: dwrite d4/d16/d27/f76 [0,4194304] 0 2024-03-21T10:08:53.009 INFO:tasks.workunit.client.0.smithi153.stdout:9/527: symlink d1/d43/d44/d46/d9e/d2e/d67/lb8 0 2024-03-21T10:08:53.009 INFO:tasks.workunit.client.0.smithi153.stdout:4/543: dwrite d11/f39 [0,4194304] 0 2024-03-21T10:08:53.014 INFO:tasks.workunit.client.1.smithi174.stdout:7/263: sync 2024-03-21T10:08:53.014 INFO:tasks.workunit.client.0.smithi153.stdout:9/528: read d1/d43/d44/f25 [3233119,48953] 0 2024-03-21T10:08:53.017 INFO:tasks.workunit.client.0.smithi153.stdout:0/447: truncate d4/d16/d3e/d5e/f78 1014475 0 2024-03-21T10:08:53.019 INFO:tasks.workunit.client.1.smithi174.stdout:0/351: creat d9/d35/d4f/d61/f77 x:0 0 0 2024-03-21T10:08:53.020 INFO:tasks.workunit.client.0.smithi153.stdout:0/448: write d4/d16/d3e/d5e/f78 [1021978,103512] 0 2024-03-21T10:08:53.023 INFO:tasks.workunit.client.1.smithi174.stdout:3/322: dread d0/f2 [0,4194304] 0 2024-03-21T10:08:53.025 INFO:tasks.workunit.client.0.smithi153.stdout:9/529: creat d1/d8a/fb9 x:0 0 0 2024-03-21T10:08:53.028 INFO:tasks.workunit.client.0.smithi153.stdout:3/528: mknod dd/d41/d80/dbb/cbc 0 2024-03-21T10:08:53.032 INFO:tasks.workunit.client.0.smithi153.stdout:5/534: symlink d4/d8/d5a/d92/d9f/la7 0 2024-03-21T10:08:53.032 INFO:tasks.workunit.client.0.smithi153.stdout:3/529: fsync dd/d1e/d45/f99 0 2024-03-21T10:08:53.039 INFO:tasks.workunit.client.0.smithi153.stdout:5/535: fsync d4/d5/f85 0 2024-03-21T10:08:53.039 INFO:tasks.workunit.client.0.smithi153.stdout:8/537: dwrite d5/d9/d43/dd/d20/d26/d24/fa3 [0,4194304] 0 2024-03-21T10:08:53.039 INFO:tasks.workunit.client.0.smithi153.stdout:9/530: mkdir d1/d43/d72/dba 0 2024-03-21T10:08:53.043 INFO:tasks.workunit.client.0.smithi153.stdout:0/449: mknod d4/d4c/d83/d22/d61/d77/c8a 0 2024-03-21T10:08:53.044 INFO:tasks.workunit.client.1.smithi174.stdout:8/289: dwrite d9/f1c [0,4194304] 0 2024-03-21T10:08:53.044 INFO:tasks.workunit.client.1.smithi174.stdout:5/359: dwrite d1/f22 [0,4194304] 0 2024-03-21T10:08:53.044 INFO:tasks.workunit.client.1.smithi174.stdout:6/270: sync 2024-03-21T10:08:53.044 INFO:tasks.workunit.client.1.smithi174.stdout:4/399: dwrite d4/d21/d31/d6a/f72 [0,4194304] 0 2024-03-21T10:08:53.045 INFO:tasks.workunit.client.0.smithi153.stdout:8/538: chown d5/d9/d43/dd/d2d/c97 0 1 2024-03-21T10:08:53.046 INFO:tasks.workunit.client.0.smithi153.stdout:4/544: dwrite d11/f16 [0,4194304] 0 2024-03-21T10:08:53.046 INFO:tasks.workunit.client.0.smithi153.stdout:3/530: creat dd/d1e/d3d/d94/d73/d7f/d91/fbd x:0 0 0 2024-03-21T10:08:53.047 INFO:tasks.workunit.client.1.smithi174.stdout:4/400: write d4/f52 [3114533,44552] 0 2024-03-21T10:08:53.051 INFO:tasks.workunit.client.1.smithi174.stdout:3/323: rmdir d0/d12/d14/d1a/d44/d56 39 2024-03-21T10:08:53.051 INFO:tasks.workunit.client.0.smithi153.stdout:3/531: fsync dd/d1e/d45/d56/d58/f8b 0 2024-03-21T10:08:53.054 INFO:tasks.workunit.client.1.smithi174.stdout:0/352: mknod d9/d21/d31/d5c/c78 0 2024-03-21T10:08:53.056 INFO:tasks.workunit.client.0.smithi153.stdout:4/545: mkdir d11/d9d 0 2024-03-21T10:08:53.058 INFO:tasks.workunit.client.1.smithi174.stdout:6/271: chown d0/c2 268 1 2024-03-21T10:08:53.059 INFO:tasks.workunit.client.1.smithi174.stdout:0/353: fdatasync d9/d21/d31/f42 0 2024-03-21T10:08:53.061 INFO:tasks.workunit.client.0.smithi153.stdout:3/532: link dd/f62 dd/d1e/d3d/d94/d8e/fbe 0 2024-03-21T10:08:53.061 INFO:tasks.workunit.client.1.smithi174.stdout:8/290: write f5 [2604673,18092] 0 2024-03-21T10:08:53.065 INFO:tasks.workunit.client.0.smithi153.stdout:4/546: creat d11/d2d/f9e x:0 0 0 2024-03-21T10:08:53.069 INFO:tasks.workunit.client.0.smithi153.stdout:3/533: dread - dd/d1e/d3d/d94/d73/d7f/d91/fa8 zero size 2024-03-21T10:08:53.070 INFO:tasks.workunit.client.0.smithi153.stdout:0/450: dwrite d4/d16/d3e/f63 [0,4194304] 0 2024-03-21T10:08:53.074 INFO:tasks.workunit.client.0.smithi153.stdout:4/547: mknod d11/d18/d7e/c9f 0 2024-03-21T10:08:53.075 INFO:tasks.workunit.client.0.smithi153.stdout:9/531: dwrite d1/d43/d44/d46/d9e/d4b/d84/fa4 [0,4194304] 0 2024-03-21T10:08:53.078 INFO:tasks.workunit.client.0.smithi153.stdout:8/539: dread d5/d9/d43/dd/d20/d26/f1f [0,4194304] 0 2024-03-21T10:08:53.078 INFO:tasks.workunit.client.0.smithi153.stdout:3/534: write dd/d1e/d3d/d94/d73/fa4 [525176,40342] 0 2024-03-21T10:08:53.080 INFO:tasks.workunit.client.0.smithi153.stdout:7/434: sync 2024-03-21T10:08:53.080 INFO:tasks.workunit.client.0.smithi153.stdout:2/603: sync 2024-03-21T10:08:53.080 INFO:tasks.workunit.client.0.smithi153.stdout:1/428: sync 2024-03-21T10:08:53.080 INFO:tasks.workunit.client.0.smithi153.stdout:6/707: sync 2024-03-21T10:08:53.083 INFO:tasks.workunit.client.1.smithi174.stdout:9/372: rename d2/f28 to d2/d11/d1d/f80 0 2024-03-21T10:08:53.084 INFO:tasks.workunit.client.0.smithi153.stdout:8/540: chown d5/d9/d43/dd/d2d/d45/d88/d85 38 1 2024-03-21T10:08:53.084 INFO:tasks.workunit.client.0.smithi153.stdout:1/429: chown d6/d32/l49 182002 1 2024-03-21T10:08:53.088 INFO:tasks.workunit.client.0.smithi153.stdout:5/536: write d4/d8/d5a/d5d/f5f [4138302,83786] 0 2024-03-21T10:08:53.088 INFO:tasks.workunit.client.0.smithi153.stdout:9/532: stat d1/d43/d44/d46/d9e/c42 0 2024-03-21T10:08:53.090 INFO:tasks.workunit.client.0.smithi153.stdout:3/535: mkdir dd/d41/d80/dbb/dbf 0 2024-03-21T10:08:53.092 INFO:tasks.workunit.client.0.smithi153.stdout:5/537: chown d4/d8/d3b/f88 176 1 2024-03-21T10:08:53.092 INFO:tasks.workunit.client.0.smithi153.stdout:3/536: stat dd/d1e/d6b/f9c 0 2024-03-21T10:08:53.096 INFO:tasks.workunit.client.0.smithi153.stdout:2/604: creat dd/d15/d18/d2c/d4b/fce x:0 0 0 2024-03-21T10:08:53.098 INFO:tasks.workunit.client.0.smithi153.stdout:7/435: rmdir d4/d2a/d3b/d60 39 2024-03-21T10:08:53.102 INFO:tasks.workunit.client.0.smithi153.stdout:6/708: unlink d1f/d2b/d33/l95 0 2024-03-21T10:08:53.105 INFO:tasks.workunit.client.1.smithi174.stdout:5/360: dwrite d1/d16/f24 [0,4194304] 0 2024-03-21T10:08:53.106 INFO:tasks.workunit.client.1.smithi174.stdout:7/264: dread d0/d4/d10/d1b/f3e [0,4194304] 0 2024-03-21T10:08:53.108 INFO:tasks.workunit.client.0.smithi153.stdout:7/436: chown d4/d14/d2f/d78/f3a 868064894 1 2024-03-21T10:08:53.113 INFO:tasks.workunit.client.1.smithi174.stdout:3/324: dwrite d0/d12/d46/f6f [0,4194304] 0 2024-03-21T10:08:53.114 INFO:tasks.workunit.client.1.smithi174.stdout:6/272: write d0/d3/d5/df/f22 [391183,99765] 0 2024-03-21T10:08:53.116 INFO:tasks.workunit.client.1.smithi174.stdout:1/350: sync 2024-03-21T10:08:53.116 INFO:tasks.workunit.client.1.smithi174.stdout:2/359: sync 2024-03-21T10:08:53.119 INFO:tasks.workunit.client.1.smithi174.stdout:0/354: dwrite d9/d21/f6e [0,4194304] 0 2024-03-21T10:08:53.120 INFO:tasks.workunit.client.1.smithi174.stdout:0/355: readlink l8 0 2024-03-21T10:08:53.121 INFO:tasks.workunit.client.0.smithi153.stdout:4/548: write d11/f24 [4247363,117378] 0 2024-03-21T10:08:53.121 INFO:tasks.workunit.client.0.smithi153.stdout:8/541: dwrite d5/d9/d43/d7b/d7c/f91 [0,4194304] 0 2024-03-21T10:08:53.122 INFO:tasks.workunit.client.0.smithi153.stdout:9/533: rmdir d1/d4/d5f 39 2024-03-21T10:08:53.126 INFO:tasks.workunit.client.1.smithi174.stdout:5/361: chown d1/d49/d58/f5b 19 1 2024-03-21T10:08:53.126 INFO:tasks.workunit.client.0.smithi153.stdout:5/538: mknod d4/d8/ca8 0 2024-03-21T10:08:53.127 INFO:tasks.workunit.client.1.smithi174.stdout:8/291: dwrite f8 [0,4194304] 0 2024-03-21T10:08:53.129 INFO:tasks.workunit.client.0.smithi153.stdout:8/542: chown d5/d9/d43/dd/d20/d26/d47/l78 6202277 1 2024-03-21T10:08:53.129 INFO:tasks.workunit.client.1.smithi174.stdout:2/360: creat d5/d25/f7e x:0 0 0 2024-03-21T10:08:53.130 INFO:tasks.workunit.client.0.smithi153.stdout:3/537: rmdir dd/d1e/d3d/d94/d73/d7f/d91 39 2024-03-21T10:08:53.130 INFO:tasks.workunit.client.1.smithi174.stdout:7/265: write d0/d4/f33 [754895,33930] 0 2024-03-21T10:08:53.134 INFO:tasks.workunit.client.0.smithi153.stdout:3/538: read dd/d1e/d45/f7a [239889,34164] 0 2024-03-21T10:08:53.137 INFO:tasks.workunit.client.1.smithi174.stdout:8/292: write d9/f1c [2069869,5678] 0 2024-03-21T10:08:53.138 INFO:tasks.workunit.client.1.smithi174.stdout:0/356: chown d9/d21/d31/d3c/l4d 0 1 2024-03-21T10:08:53.139 INFO:tasks.workunit.client.1.smithi174.stdout:6/273: rename d0/d3/d5/d6/la to d0/d3/d5/d6/d42/l5d 0 2024-03-21T10:08:53.140 INFO:tasks.workunit.client.1.smithi174.stdout:7/266: symlink d0/d4/d10/d44/l55 0 2024-03-21T10:08:53.149 INFO:tasks.workunit.client.0.smithi153.stdout:7/437: creat d4/d1b/d20/f93 x:0 0 0 2024-03-21T10:08:53.152 INFO:tasks.workunit.client.1.smithi174.stdout:3/325: mkdir d0/d12/d14/d1a/d44/d56/d76 0 2024-03-21T10:08:53.157 INFO:tasks.workunit.client.1.smithi174.stdout:2/361: readlink d5/d25/d27/d2d/d45/d5b/d2f/l5c 0 2024-03-21T10:08:53.157 INFO:tasks.workunit.client.1.smithi174.stdout:7/267: dread d0/d4/d10/f21 [0,4194304] 0 2024-03-21T10:08:53.160 INFO:tasks.workunit.client.0.smithi153.stdout:0/451: write d4/d4c/f7a [2493045,95033] 0 2024-03-21T10:08:53.163 INFO:tasks.workunit.client.1.smithi174.stdout:7/268: fdatasync d0/d4/d10/f19 0 2024-03-21T10:08:53.163 INFO:tasks.workunit.client.1.smithi174.stdout:0/357: mkdir d9/d35/d4f/d79 0 2024-03-21T10:08:53.169 INFO:tasks.workunit.client.1.smithi174.stdout:0/358: write d9/d35/d4a/f5e [2968448,95347] 0 2024-03-21T10:08:53.172 INFO:tasks.workunit.client.1.smithi174.stdout:0/359: chown d9/d1c/l51 188475352 1 2024-03-21T10:08:53.172 INFO:tasks.workunit.client.0.smithi153.stdout:8/543: fsync d5/d9/d43/dd/d20/d26/f23 0 2024-03-21T10:08:53.174 INFO:tasks.workunit.client.1.smithi174.stdout:9/373: getdents d2/d24/d2e 0 2024-03-21T10:08:53.174 INFO:tasks.workunit.client.1.smithi174.stdout:6/274: dwrite d0/d3/d5/f57 [0,4194304] 0 2024-03-21T10:08:53.177 INFO:tasks.workunit.client.1.smithi174.stdout:3/326: rmdir d0/d12/d14/d1a/d27/d4f 39 2024-03-21T10:08:53.177 INFO:tasks.workunit.client.1.smithi174.stdout:1/351: link d4/d7/ce d4/d5/c7a 0 2024-03-21T10:08:53.178 INFO:tasks.workunit.client.0.smithi153.stdout:4/549: mkdir d11/d2d/d6b/d9c/da0 0 2024-03-21T10:08:53.181 INFO:tasks.workunit.client.0.smithi153.stdout:7/438: fdatasync d4/d2a/d3b/d81/f90 0 2024-03-21T10:08:53.181 INFO:tasks.workunit.client.0.smithi153.stdout:3/539: dwrite dd/d1e/d3d/d53/fba [0,4194304] 0 2024-03-21T10:08:53.183 INFO:tasks.workunit.client.0.smithi153.stdout:8/544: truncate d5/d9/d43/d7b/daa/fb5 837455 0 2024-03-21T10:08:53.186 INFO:tasks.workunit.client.1.smithi174.stdout:4/401: dread d4/d21/d31/d3a/f3d [0,4194304] 0 2024-03-21T10:08:53.186 INFO:tasks.workunit.client.1.smithi174.stdout:3/327: readlink d0/d12/d14/d1a/d27/d4f/d5d/l3e 0 2024-03-21T10:08:53.189 INFO:tasks.workunit.client.1.smithi174.stdout:1/352: mkdir d4/d7/d68/d7b 0 2024-03-21T10:08:53.190 INFO:tasks.workunit.client.1.smithi174.stdout:2/362: chown d5/d25/d27/d2d/d45/d5b/d55/f58 23697 1 2024-03-21T10:08:53.192 INFO:tasks.workunit.client.1.smithi174.stdout:7/269: rename d0/f2b to d0/f56 0 2024-03-21T10:08:53.192 INFO:tasks.workunit.client.1.smithi174.stdout:9/374: write d2/d11/d1d/f80 [2319261,92362] 0 2024-03-21T10:08:53.193 INFO:tasks.workunit.client.1.smithi174.stdout:6/275: dread d0/d3/d5/df/f22 [0,4194304] 0 2024-03-21T10:08:53.195 INFO:tasks.workunit.client.1.smithi174.stdout:1/353: fsync d4/d7/d33/d43/d51/f64 0 2024-03-21T10:08:53.197 INFO:tasks.workunit.client.0.smithi153.stdout:6/709: dwrite f19 [0,4194304] 0 2024-03-21T10:08:53.200 INFO:tasks.workunit.client.1.smithi174.stdout:3/328: write d0/f3 [8012838,63204] 0 2024-03-21T10:08:53.203 INFO:tasks.workunit.client.0.smithi153.stdout:1/430: truncate d6/fc 3572041 0 2024-03-21T10:08:53.205 INFO:tasks.workunit.client.1.smithi174.stdout:9/375: write d2/d24/f74 [939945,107401] 0 2024-03-21T10:08:53.205 INFO:tasks.workunit.client.1.smithi174.stdout:6/276: stat d0/d3/d5/df/l3a 0 2024-03-21T10:08:53.207 INFO:tasks.workunit.client.0.smithi153.stdout:2/605: write dd/d15/d3c/f69 [1483038,15778] 0 2024-03-21T10:08:53.210 INFO:tasks.workunit.client.1.smithi174.stdout:0/360: dwrite d9/d21/f6e [0,4194304] 0 2024-03-21T10:08:53.212 INFO:tasks.workunit.client.1.smithi174.stdout:7/270: creat d0/f57 x:0 0 0 2024-03-21T10:08:53.214 INFO:tasks.workunit.client.0.smithi153.stdout:7/439: dwrite d4/d14/d16/f42 [0,4194304] 0 2024-03-21T10:08:53.218 INFO:tasks.workunit.client.1.smithi174.stdout:6/277: fsync d0/d3/d20/d3d/f59 0 2024-03-21T10:08:53.221 INFO:tasks.workunit.client.1.smithi174.stdout:4/402: dwrite d4/f20 [0,4194304] 0 2024-03-21T10:08:53.225 INFO:tasks.workunit.client.1.smithi174.stdout:6/278: read - d0/d3/d5/df/f4e zero size 2024-03-21T10:08:53.229 INFO:tasks.workunit.client.1.smithi174.stdout:4/403: chown d4/d21/d24/d28/l43 1406972 1 2024-03-21T10:08:53.230 INFO:tasks.workunit.client.1.smithi174.stdout:3/329: mkdir d0/d12/d14/d1a/d27/d4f/d64/d77 0 2024-03-21T10:08:53.230 INFO:tasks.workunit.client.1.smithi174.stdout:5/362: dread d1/d49/d58/f64 [0,4194304] 0 2024-03-21T10:08:53.231 INFO:tasks.workunit.client.1.smithi174.stdout:7/271: creat d0/d4/d38/f58 x:0 0 0 2024-03-21T10:08:53.233 INFO:tasks.workunit.client.1.smithi174.stdout:4/404: chown d4/fa 556936030 1 2024-03-21T10:08:53.233 INFO:tasks.workunit.client.1.smithi174.stdout:9/376: creat d2/d24/f81 x:0 0 0 2024-03-21T10:08:53.238 INFO:tasks.workunit.client.0.smithi153.stdout:4/550: mkdir d11/d2d/d3d/da1 0 2024-03-21T10:08:53.241 INFO:tasks.workunit.client.1.smithi174.stdout:2/363: dread d5/f12 [0,4194304] 0 2024-03-21T10:08:53.241 INFO:tasks.workunit.client.1.smithi174.stdout:5/363: write d1/f29 [227853,64020] 0 2024-03-21T10:08:53.241 INFO:tasks.workunit.client.1.smithi174.stdout:8/293: dwrite d9/d17/d19/f2f [0,4194304] 0 2024-03-21T10:08:53.241 INFO:tasks.workunit.client.1.smithi174.stdout:4/405: chown d4/f2c 26780 1 2024-03-21T10:08:53.241 INFO:tasks.workunit.client.0.smithi153.stdout:5/539: dread d4/d5/d16/d24/f53 [0,4194304] 0 2024-03-21T10:08:53.244 INFO:tasks.workunit.client.1.smithi174.stdout:4/406: readlink d4/l34 0 2024-03-21T10:08:53.245 INFO:tasks.workunit.client.1.smithi174.stdout:2/364: chown d5/d25/d27/d2e/d7b 146761175 1 2024-03-21T10:08:53.251 INFO:tasks.workunit.client.1.smithi174.stdout:3/330: creat d0/d12/d14/d1a/d27/d4f/d5d/f78 x:0 0 0 2024-03-21T10:08:53.251 INFO:tasks.workunit.client.1.smithi174.stdout:0/361: dread d9/d21/d23/f41 [0,4194304] 0 2024-03-21T10:08:53.252 INFO:tasks.workunit.client.1.smithi174.stdout:5/364: write d1/d16/d2e/d31/f36 [1322501,59870] 0 2024-03-21T10:08:53.255 INFO:tasks.workunit.client.1.smithi174.stdout:7/272: rename d0/d4/d10/d1b/c49 to d0/d4/d10/d44/c59 0 2024-03-21T10:08:53.259 INFO:tasks.workunit.client.1.smithi174.stdout:3/331: write d0/d12/d14/f4e [647059,91613] 0 2024-03-21T10:08:53.259 INFO:tasks.workunit.client.1.smithi174.stdout:8/294: creat d9/d53/f5d x:0 0 0 2024-03-21T10:08:53.260 INFO:tasks.workunit.client.1.smithi174.stdout:0/362: mknod d9/d58/c7a 0 2024-03-21T10:08:53.263 INFO:tasks.workunit.client.0.smithi153.stdout:0/452: dwrite d4/d16/d27/f79 [0,4194304] 0 2024-03-21T10:08:53.263 INFO:tasks.workunit.client.0.smithi153.stdout:9/534: symlink d1/d4/d5f/db6/lbb 0 2024-03-21T10:08:53.264 INFO:tasks.workunit.client.1.smithi174.stdout:4/407: mknod d4/d21/d24/d28/d79/c87 0 2024-03-21T10:08:53.264 INFO:tasks.workunit.client.1.smithi174.stdout:6/279: dread d0/d3/d5/f1c [0,4194304] 0 2024-03-21T10:08:53.269 INFO:tasks.workunit.client.1.smithi174.stdout:7/273: mknod d0/d4/c5a 0 2024-03-21T10:08:53.273 INFO:tasks.workunit.client.0.smithi153.stdout:7/440: symlink d4/d11/l94 0 2024-03-21T10:08:53.273 INFO:tasks.workunit.client.0.smithi153.stdout:4/551: unlink f7 0 2024-03-21T10:08:53.273 INFO:tasks.workunit.client.0.smithi153.stdout:5/540: creat d4/d5/d16/d24/fa9 x:0 0 0 2024-03-21T10:08:53.276 INFO:tasks.workunit.client.1.smithi174.stdout:1/354: dwrite d4/d7/d15/f1f [0,4194304] 0 2024-03-21T10:08:53.279 INFO:tasks.workunit.client.1.smithi174.stdout:6/280: write d0/d3/d20/d3d/f59 [418112,105982] 0 2024-03-21T10:08:53.280 INFO:tasks.workunit.client.1.smithi174.stdout:4/408: creat d4/d21/d24/d28/d38/f88 x:0 0 0 2024-03-21T10:08:53.280 INFO:tasks.workunit.client.0.smithi153.stdout:0/453: creat d4/db/f8b x:0 0 0 2024-03-21T10:08:53.281 INFO:tasks.workunit.client.1.smithi174.stdout:7/274: write d0/d4/d38/f58 [812842,15459] 0 2024-03-21T10:08:53.282 INFO:tasks.workunit.client.0.smithi153.stdout:9/535: symlink d1/d43/d44/d46/d9e/d4b/d84/d8b/lbc 0 2024-03-21T10:08:53.284 INFO:tasks.workunit.client.0.smithi153.stdout:0/454: fsync d4/d16/d3e/f80 0 2024-03-21T10:08:53.285 INFO:tasks.workunit.client.1.smithi174.stdout:9/377: dwrite d2/fc [4194304,4194304] 0 2024-03-21T10:08:53.287 INFO:tasks.workunit.client.1.smithi174.stdout:0/363: creat d9/f7b x:0 0 0 2024-03-21T10:08:53.287 INFO:tasks.workunit.client.1.smithi174.stdout:8/295: link d9/d2a/d4d/f50 d9/d2a/f5e 0 2024-03-21T10:08:53.288 INFO:tasks.workunit.client.0.smithi153.stdout:9/536: write d1/d43/d72/d9d/fa0 [10349,128675] 0 2024-03-21T10:08:53.289 INFO:tasks.workunit.client.1.smithi174.stdout:1/355: mknod d4/d5/d6c/c7c 0 2024-03-21T10:08:53.290 INFO:tasks.workunit.client.0.smithi153.stdout:0/455: fdatasync d4/d16/d3e/f69 0 2024-03-21T10:08:53.292 INFO:tasks.workunit.client.1.smithi174.stdout:6/281: chown d0/d3/d5/d6/d42/c49 1 1 2024-03-21T10:08:53.295 INFO:tasks.workunit.client.1.smithi174.stdout:7/275: mknod d0/d4/d10/d1b/c5b 0 2024-03-21T10:08:53.295 INFO:tasks.workunit.client.1.smithi174.stdout:0/364: fsync d9/d58/f63 0 2024-03-21T10:08:53.296 INFO:tasks.workunit.client.0.smithi153.stdout:0/456: chown d4/d16/d19/c2c 248 1 2024-03-21T10:08:53.296 INFO:tasks.workunit.client.1.smithi174.stdout:1/356: chown d4/l55 880570 1 2024-03-21T10:08:53.298 INFO:tasks.workunit.client.0.smithi153.stdout:0/457: stat d4/db/f13 0 2024-03-21T10:08:53.299 INFO:tasks.workunit.client.1.smithi174.stdout:4/409: rename d4/d21/d31/d3a/f60 to d4/d21/d31/d3a/d77/f89 0 2024-03-21T10:08:53.307 INFO:tasks.workunit.client.0.smithi153.stdout:4/552: dread d11/d18/f1d [4194304,4194304] 0 2024-03-21T10:08:53.308 INFO:tasks.workunit.client.1.smithi174.stdout:1/357: mkdir d4/d7/d32/d7d 0 2024-03-21T10:08:53.309 INFO:tasks.workunit.client.0.smithi153.stdout:4/553: chown d11/d2d/f3a 64 1 2024-03-21T10:08:53.310 INFO:tasks.workunit.client.0.smithi153.stdout:9/537: mkdir d1/d43/d44/d46/d9e/d24/d41/dbd 0 2024-03-21T10:08:53.310 INFO:tasks.workunit.client.1.smithi174.stdout:7/276: fsync d0/d4/d10/d1b/f3e 0 2024-03-21T10:08:53.311 INFO:tasks.workunit.client.0.smithi153.stdout:5/541: dwrite d4/d8/f1d [0,4194304] 0 2024-03-21T10:08:53.311 INFO:tasks.workunit.client.1.smithi174.stdout:3/332: dwrite d0/d12/d14/d1a/f34 [0,4194304] 0 2024-03-21T10:08:53.312 INFO:tasks.workunit.client.1.smithi174.stdout:4/410: write d4/d21/d24/d28/d38/f88 [600803,10828] 0 2024-03-21T10:08:53.312 INFO:tasks.workunit.client.1.smithi174.stdout:2/365: dwrite d5/d25/d27/d2d/d45/d5b/f6f [0,4194304] 0 2024-03-21T10:08:53.316 INFO:tasks.workunit.client.0.smithi153.stdout:9/538: chown d1/d43/d44/d46/d4d/f74 92836055 1 2024-03-21T10:08:53.328 INFO:tasks.workunit.client.1.smithi174.stdout:8/296: dwrite d9/d17/f5a [0,4194304] 0 2024-03-21T10:08:53.330 INFO:tasks.workunit.client.0.smithi153.stdout:4/554: mknod d11/d2d/d57/ca2 0 2024-03-21T10:08:53.332 INFO:tasks.workunit.client.0.smithi153.stdout:0/458: dread d4/f62 [0,4194304] 0 2024-03-21T10:08:53.333 INFO:tasks.workunit.client.0.smithi153.stdout:5/542: symlink d4/d5/d16/laa 0 2024-03-21T10:08:53.334 INFO:tasks.workunit.client.1.smithi174.stdout:1/358: unlink d4/d7/l16 0 2024-03-21T10:08:53.341 INFO:tasks.workunit.client.1.smithi174.stdout:0/365: dwrite d9/f13 [0,4194304] 0 2024-03-21T10:08:53.341 INFO:tasks.workunit.client.1.smithi174.stdout:7/277: mknod d0/d4/d38/c5c 0 2024-03-21T10:08:53.344 INFO:tasks.workunit.client.0.smithi153.stdout:2/606: link dd/d15/d3c/fcb dd/d15/d3c/fcf 0 2024-03-21T10:08:53.345 INFO:tasks.workunit.client.1.smithi174.stdout:1/359: symlink d4/d7/d33/d43/d51/l7e 0 2024-03-21T10:08:53.346 INFO:tasks.workunit.client.1.smithi174.stdout:3/333: creat d0/d12/d14/d4c/f79 x:0 0 0 2024-03-21T10:08:53.350 INFO:tasks.workunit.client.1.smithi174.stdout:7/278: mknod d0/c5d 0 2024-03-21T10:08:53.351 INFO:tasks.workunit.client.1.smithi174.stdout:4/411: link d4/d19/f22 d4/d21/d24/d28/f8a 0 2024-03-21T10:08:53.351 INFO:tasks.workunit.client.1.smithi174.stdout:1/360: chown d4/d7/d32/l4c 387557 1 2024-03-21T10:08:53.351 INFO:tasks.workunit.client.1.smithi174.stdout:9/378: dwrite d2/f3 [4194304,4194304] 0 2024-03-21T10:08:53.354 INFO:tasks.workunit.client.0.smithi153.stdout:0/459: write d4/f10 [4905295,23305] 0 2024-03-21T10:08:53.356 INFO:tasks.workunit.client.1.smithi174.stdout:9/379: readlink d2/d11/d60/l61 0 2024-03-21T10:08:53.356 INFO:tasks.workunit.client.1.smithi174.stdout:7/279: stat d0/d4/f43 0 2024-03-21T10:08:53.357 INFO:tasks.workunit.client.1.smithi174.stdout:4/412: fdatasync d4/d19/f65 0 2024-03-21T10:08:53.360 INFO:tasks.workunit.client.1.smithi174.stdout:5/365: dwrite d1/d16/d2e/d31/f33 [0,4194304] 0 2024-03-21T10:08:53.361 INFO:tasks.workunit.client.0.smithi153.stdout:2/607: readlink dd/d15/d18/d2c/d33/l70 0 2024-03-21T10:08:53.361 INFO:tasks.workunit.client.0.smithi153.stdout:4/555: mkdir d11/d2d/da3 0 2024-03-21T10:08:53.361 INFO:tasks.workunit.client.0.smithi153.stdout:3/540: truncate dd/f27 2397835 0 2024-03-21T10:08:53.362 INFO:tasks.workunit.client.1.smithi174.stdout:3/334: fsync d0/f13 0 2024-03-21T10:08:53.365 INFO:tasks.workunit.client.1.smithi174.stdout:6/282: write d0/d3/d5/d3b/f46 [3189610,108868] 0 2024-03-21T10:08:53.366 INFO:tasks.workunit.client.0.smithi153.stdout:1/431: truncate d6/d13/f2b 2360254 0 2024-03-21T10:08:53.371 INFO:tasks.workunit.client.1.smithi174.stdout:8/297: truncate d9/d17/f21 2568019 0 2024-03-21T10:08:53.372 INFO:tasks.workunit.client.0.smithi153.stdout:4/556: mknod d11/d2d/d3d/ca4 0 2024-03-21T10:08:53.373 INFO:tasks.workunit.client.1.smithi174.stdout:9/380: mknod d2/d24/d2e/d63/c82 0 2024-03-21T10:08:53.373 INFO:tasks.workunit.client.1.smithi174.stdout:4/413: dread - d4/d21/d24/d28/f8a zero size 2024-03-21T10:08:53.376 INFO:tasks.workunit.client.0.smithi153.stdout:6/710: dwrite d1f/d2b/f63 [0,4194304] 0 2024-03-21T10:08:53.376 INFO:tasks.workunit.client.0.smithi153.stdout:8/545: dwrite d5/d9/d43/dd/d2d/d45/d88/f6e [0,4194304] 0 2024-03-21T10:08:53.377 INFO:tasks.workunit.client.1.smithi174.stdout:8/298: write d9/d17/f5a [1395292,29823] 0 2024-03-21T10:08:53.378 INFO:tasks.workunit.client.0.smithi153.stdout:2/608: write dd/d15/d18/d3a/d8c/faa [819952,61582] 0 2024-03-21T10:08:53.379 INFO:tasks.workunit.client.1.smithi174.stdout:3/335: mkdir d0/d12/d14/d4c/d7a 0 2024-03-21T10:08:53.381 INFO:tasks.workunit.client.0.smithi153.stdout:2/609: chown dd/d15/d18/d2c/d33/d98 6 1 2024-03-21T10:08:53.382 INFO:tasks.workunit.client.0.smithi153.stdout:9/539: dwrite d1/d43/d44/d46/d9e/d24/fa1 [4194304,4194304] 0 2024-03-21T10:08:53.385 INFO:tasks.workunit.client.1.smithi174.stdout:6/283: mkdir d0/d3/d2a/d56/d5e 0 2024-03-21T10:08:53.386 INFO:tasks.workunit.client.0.smithi153.stdout:2/610: read - dd/d15/d18/d3a/d3d/f74 zero size 2024-03-21T10:08:53.386 INFO:tasks.workunit.client.0.smithi153.stdout:9/540: write d1/d8a/fb9 [141443,36427] 0 2024-03-21T10:08:53.388 INFO:tasks.workunit.client.1.smithi174.stdout:6/284: chown d0/d3/d5/df/d16 121 1 2024-03-21T10:08:53.389 INFO:tasks.workunit.client.1.smithi174.stdout:5/366: link d1/d5c/f7a d1/d16/d2e/d66/f7d 0 2024-03-21T10:08:53.390 INFO:tasks.workunit.client.0.smithi153.stdout:9/541: fsync d1/d43/d44/d46/d9e/d4b/f73 0 2024-03-21T10:08:53.392 INFO:tasks.workunit.client.1.smithi174.stdout:9/381: read d2/f3c [46160,8313] 0 2024-03-21T10:08:53.393 INFO:tasks.workunit.client.1.smithi174.stdout:3/336: fdatasync d0/d12/d14/d1a/f23 0 2024-03-21T10:08:53.394 INFO:tasks.workunit.client.1.smithi174.stdout:6/285: fsync d0/d3/d5/df/d16/f27 0 2024-03-21T10:08:53.394 INFO:tasks.workunit.client.1.smithi174.stdout:3/337: dread - d0/d12/d14/f62 zero size 2024-03-21T10:08:53.396 INFO:tasks.workunit.client.0.smithi153.stdout:8/546: rmdir d5/d9/d43/d7b 39 2024-03-21T10:08:53.398 INFO:tasks.workunit.client.1.smithi174.stdout:3/338: fsync d0/d12/d46/f6f 0 2024-03-21T10:08:53.399 INFO:tasks.workunit.client.0.smithi153.stdout:8/547: chown d5/d9/d43/f6c 1052207248 1 2024-03-21T10:08:53.400 INFO:tasks.workunit.client.0.smithi153.stdout:6/711: creat d1f/da0/f101 x:0 0 0 2024-03-21T10:08:53.400 INFO:tasks.workunit.client.0.smithi153.stdout:7/441: dwrite d4/d14/d2f/f45 [0,4194304] 0 2024-03-21T10:08:53.402 INFO:tasks.workunit.client.1.smithi174.stdout:9/382: mknod d2/d24/c83 0 2024-03-21T10:08:53.402 INFO:tasks.workunit.client.1.smithi174.stdout:5/367: link d1/c2a d1/d49/c7e 0 2024-03-21T10:08:53.403 INFO:tasks.workunit.client.0.smithi153.stdout:4/557: symlink d11/d2d/d57/d85/la5 0 2024-03-21T10:08:53.403 INFO:tasks.workunit.client.0.smithi153.stdout:7/442: chown d4/d2a/d40/f4c 4518672 1 2024-03-21T10:08:53.403 INFO:tasks.workunit.client.0.smithi153.stdout:8/548: dread - d5/d9/d43/dd/d20/d26/d24/d52/f72 zero size 2024-03-21T10:08:53.404 INFO:tasks.workunit.client.0.smithi153.stdout:4/558: chown d11/d18/d7e/d93 1022 1 2024-03-21T10:08:53.408 INFO:tasks.workunit.client.1.smithi174.stdout:6/286: mknod d0/d3/d5/df/c5f 0 2024-03-21T10:08:53.409 INFO:tasks.workunit.client.0.smithi153.stdout:8/549: read d5/d9/d43/dd/d20/f7d [1369858,94028] 0 2024-03-21T10:08:53.410 INFO:tasks.workunit.client.1.smithi174.stdout:8/299: dwrite d9/d17/f5a [0,4194304] 0 2024-03-21T10:08:53.411 INFO:tasks.workunit.client.1.smithi174.stdout:9/383: chown d2/f9 1 1 2024-03-21T10:08:53.412 INFO:tasks.workunit.client.1.smithi174.stdout:3/339: fsync d0/d12/d14/d1a/f23 0 2024-03-21T10:08:53.415 INFO:tasks.workunit.client.1.smithi174.stdout:6/287: readlink d0/d3/d5/df/l1e 0 2024-03-21T10:08:53.415 INFO:tasks.workunit.client.1.smithi174.stdout:3/340: chown d0/fa 1591712 1 2024-03-21T10:08:53.416 INFO:tasks.workunit.client.0.smithi153.stdout:2/611: creat dd/d15/d18/d2c/d4b/fd0 x:0 0 0 2024-03-21T10:08:53.417 INFO:tasks.workunit.client.0.smithi153.stdout:3/541: mknod dd/d1e/d45/cc0 0 2024-03-21T10:08:53.424 INFO:tasks.workunit.client.0.smithi153.stdout:9/542: mkdir d1/d43/d44/d46/d9e/d24/d41/dbe 0 2024-03-21T10:08:53.425 INFO:tasks.workunit.client.1.smithi174.stdout:0/366: dread d9/d35/f36 [0,4194304] 0 2024-03-21T10:08:53.426 INFO:tasks.workunit.client.1.smithi174.stdout:5/368: creat d1/d5c/d73/f7f x:0 0 0 2024-03-21T10:08:53.426 INFO:tasks.workunit.client.1.smithi174.stdout:0/367: chown d9/l2a 30560 1 2024-03-21T10:08:53.426 INFO:tasks.workunit.client.1.smithi174.stdout:8/300: rename d9/d2a/f5e to d9/d2a/d39/f5f 0 2024-03-21T10:08:53.427 INFO:tasks.workunit.client.0.smithi153.stdout:6/712: mknod d1f/d2b/d33/d40/d7b/c102 0 2024-03-21T10:08:53.430 INFO:tasks.workunit.client.1.smithi174.stdout:9/384: rename d2/f3 to d2/f84 0 2024-03-21T10:08:53.436 INFO:tasks.workunit.client.1.smithi174.stdout:0/368: rmdir d9/d21/d31/d3c 39 2024-03-21T10:08:53.446 INFO:tasks.workunit.client.1.smithi174.stdout:8/301: dread d9/f14 [0,4194304] 0 2024-03-21T10:08:53.446 INFO:tasks.workunit.client.0.smithi153.stdout:5/543: dread d4/d5/de/f43 [0,4194304] 0 2024-03-21T10:08:53.449 INFO:tasks.workunit.client.1.smithi174.stdout:5/369: creat d1/d16/d2e/f80 x:0 0 0 2024-03-21T10:08:53.449 INFO:tasks.workunit.client.1.smithi174.stdout:7/280: dwrite d0/f56 [0,4194304] 0 2024-03-21T10:08:53.450 INFO:tasks.workunit.client.0.smithi153.stdout:3/542: creat dd/d41/d80/dbb/fc1 x:0 0 0 2024-03-21T10:08:53.456 INFO:tasks.workunit.client.1.smithi174.stdout:5/370: readlink d1/d2f/l44 0 2024-03-21T10:08:53.458 INFO:tasks.workunit.client.1.smithi174.stdout:7/281: mknod d0/d4/d10/d44/c5e 0 2024-03-21T10:08:53.458 INFO:tasks.workunit.client.1.smithi174.stdout:5/371: readlink d1/d16/d2e/d31/l6e 0 2024-03-21T10:08:53.458 INFO:tasks.workunit.client.0.smithi153.stdout:7/443: dread d4/d14/d2f/d78/f29 [0,4194304] 0 2024-03-21T10:08:53.459 INFO:tasks.workunit.client.0.smithi153.stdout:5/544: read d4/f75 [168250,103033] 0 2024-03-21T10:08:53.460 INFO:tasks.workunit.client.0.smithi153.stdout:8/550: creat d5/d9/d43/d7b/daa/fb8 x:0 0 0 2024-03-21T10:08:53.461 INFO:tasks.workunit.client.0.smithi153.stdout:2/612: fdatasync dd/d15/d3c/fcf 0 2024-03-21T10:08:53.463 INFO:tasks.workunit.client.1.smithi174.stdout:8/302: rename d9/d37 to d9/d17/d19/d27/d60 0 2024-03-21T10:08:53.466 INFO:tasks.workunit.client.0.smithi153.stdout:4/559: creat d11/fa6 x:0 0 0 2024-03-21T10:08:53.466 INFO:tasks.workunit.client.1.smithi174.stdout:0/369: getdents d9 0 2024-03-21T10:08:53.466 INFO:tasks.workunit.client.0.smithi153.stdout:5/545: rmdir d4 39 2024-03-21T10:08:53.467 INFO:tasks.workunit.client.1.smithi174.stdout:5/372: mkdir d1/d16/d1c/d81 0 2024-03-21T10:08:53.468 INFO:tasks.workunit.client.1.smithi174.stdout:3/341: dread d0/d12/d14/d1a/d27/d4f/d5d/f38 [0,4194304] 0 2024-03-21T10:08:53.469 INFO:tasks.workunit.client.1.smithi174.stdout:5/373: read - d1/d49/f53 zero size 2024-03-21T10:08:53.469 INFO:tasks.workunit.client.0.smithi153.stdout:9/543: getdents d1/d43/d72 0 2024-03-21T10:08:53.470 INFO:tasks.workunit.client.1.smithi174.stdout:0/370: creat d9/d21/d31/d5c/f7c x:0 0 0 2024-03-21T10:08:53.473 INFO:tasks.workunit.client.1.smithi174.stdout:8/303: creat d9/d2a/f61 x:0 0 0 2024-03-21T10:08:53.473 INFO:tasks.workunit.client.0.smithi153.stdout:3/543: rename dd/d1e/d65 to dd/d1e/d3d/d94/d73/dc2 0 2024-03-21T10:08:53.474 INFO:tasks.workunit.client.0.smithi153.stdout:8/551: dread - d5/d9/d43/dd/d20/d26/d24/d52/d56/f6f zero size 2024-03-21T10:08:53.474 INFO:tasks.workunit.client.0.smithi153.stdout:4/560: readlink d11/d2d/l52 0 2024-03-21T10:08:53.474 INFO:tasks.workunit.client.1.smithi174.stdout:5/374: write d1/f11 [3755413,111239] 0 2024-03-21T10:08:53.474 INFO:tasks.workunit.client.1.smithi174.stdout:3/342: unlink d0/d12/l15 0 2024-03-21T10:08:53.475 INFO:tasks.workunit.client.0.smithi153.stdout:9/544: readlink d1/l80 0 2024-03-21T10:08:53.475 INFO:tasks.workunit.client.0.smithi153.stdout:5/546: mknod d4/d5/de/d2f/cab 0 2024-03-21T10:08:53.476 INFO:tasks.workunit.client.0.smithi153.stdout:4/561: write d11/f16 [2452480,92260] 0 2024-03-21T10:08:53.478 INFO:tasks.workunit.client.0.smithi153.stdout:9/545: readlink d1/l7f 0 2024-03-21T10:08:53.479 INFO:tasks.workunit.client.0.smithi153.stdout:3/544: truncate dd/f15 665730 0 2024-03-21T10:08:53.482 INFO:tasks.workunit.client.1.smithi174.stdout:6/288: dwrite d0/d3/d20/f23 [0,4194304] 0 2024-03-21T10:08:53.483 INFO:tasks.workunit.client.0.smithi153.stdout:4/562: write d11/d18/d50/f53 [3341170,56675] 0 2024-03-21T10:08:53.484 INFO:tasks.workunit.client.1.smithi174.stdout:5/375: write d1/f2c [845651,60902] 0 2024-03-21T10:08:53.487 INFO:tasks.workunit.client.0.smithi153.stdout:0/460: truncate d4/d4c/d83/f35 2798481 0 2024-03-21T10:08:53.487 INFO:tasks.workunit.client.1.smithi174.stdout:6/289: write d0/d3/d5/d6/d42/f43 [805571,31770] 0 2024-03-21T10:08:53.490 INFO:tasks.workunit.client.1.smithi174.stdout:9/385: unlink d2/d11/d1d/l21 0 2024-03-21T10:08:53.492 INFO:tasks.workunit.client.1.smithi174.stdout:5/376: mknod d1/d5c/d73/c82 0 2024-03-21T10:08:53.493 INFO:tasks.workunit.client.1.smithi174.stdout:6/290: stat d0/d3/d20/l52 0 2024-03-21T10:08:53.493 INFO:tasks.workunit.client.1.smithi174.stdout:0/371: dwrite d9/d1c/f20 [0,4194304] 0 2024-03-21T10:08:53.493 INFO:tasks.workunit.client.0.smithi153.stdout:0/461: read - d4/d16/d19/f85 zero size 2024-03-21T10:08:53.495 INFO:tasks.workunit.client.1.smithi174.stdout:8/304: dread d9/d17/d19/f2f [0,4194304] 0 2024-03-21T10:08:53.495 INFO:tasks.workunit.client.0.smithi153.stdout:3/545: mknod dd/d41/cc3 0 2024-03-21T10:08:53.496 INFO:tasks.workunit.client.1.smithi174.stdout:9/386: symlink d2/d24/d2e/l85 0 2024-03-21T10:08:53.497 INFO:tasks.workunit.client.1.smithi174.stdout:5/377: truncate d1/f75 676505 0 2024-03-21T10:08:53.498 INFO:tasks.workunit.client.0.smithi153.stdout:0/462: dread - d4/d16/d19/d58/f6a zero size 2024-03-21T10:08:53.498 INFO:tasks.workunit.client.0.smithi153.stdout:7/444: dwrite d4/d2a/d40/f4c [0,4194304] 0 2024-03-21T10:08:53.499 INFO:tasks.workunit.client.0.smithi153.stdout:9/546: unlink d1/d4e/d68/f79 0 2024-03-21T10:08:53.500 INFO:tasks.workunit.client.1.smithi174.stdout:5/378: write d1/d49/d4f/d51/f56 [1599175,84005] 0 2024-03-21T10:08:53.501 INFO:tasks.workunit.client.1.smithi174.stdout:9/387: creat d2/d11/d60/f86 x:0 0 0 2024-03-21T10:08:53.503 INFO:tasks.workunit.client.0.smithi153.stdout:6/713: dread d1f/d2b/d33/d40/d5f/f6d [0,4194304] 0 2024-03-21T10:08:53.504 INFO:tasks.workunit.client.1.smithi174.stdout:5/379: chown d1/d16/d2e/d31/f36 5 1 2024-03-21T10:08:53.508 INFO:tasks.workunit.client.1.smithi174.stdout:5/380: fsync d1/d16/d2e/d31/f48 0 2024-03-21T10:08:53.510 INFO:tasks.workunit.client.1.smithi174.stdout:8/305: getdents d9/d17/d19/d27 0 2024-03-21T10:08:53.510 INFO:tasks.workunit.client.0.smithi153.stdout:5/547: rename d4/d5/df/f30 to d4/d5/df/d67/d86/fac 0 2024-03-21T10:08:53.511 INFO:tasks.workunit.client.0.smithi153.stdout:4/563: write d11/d2d/d3d/f45 [543551,81522] 0 2024-03-21T10:08:53.514 INFO:tasks.workunit.client.1.smithi174.stdout:8/306: write d9/d17/d19/f23 [86078,71788] 0 2024-03-21T10:08:53.517 INFO:tasks.workunit.client.1.smithi174.stdout:6/291: dwrite d0/d3/d5/d3b/f46 [0,4194304] 0 2024-03-21T10:08:53.520 INFO:tasks.workunit.client.1.smithi174.stdout:6/292: symlink d0/d3/d20/l60 0 2024-03-21T10:08:53.521 INFO:tasks.workunit.client.0.smithi153.stdout:8/552: mkdir d5/d9/d43/dd/d20/d26/d47/d9a/db9 0 2024-03-21T10:08:53.526 INFO:tasks.workunit.client.0.smithi153.stdout:2/613: dread dd/d15/d3c/da7/d91/dae/d50/d58/fa8 [0,4194304] 0 2024-03-21T10:08:53.526 INFO:tasks.workunit.client.1.smithi174.stdout:5/381: dread d1/f2c [0,4194304] 0 2024-03-21T10:08:53.529 INFO:tasks.workunit.client.1.smithi174.stdout:5/382: creat d1/d5c/f83 x:0 0 0 2024-03-21T10:08:53.532 INFO:tasks.workunit.client.1.smithi174.stdout:5/383: mkdir d1/d16/d2e/d31/d84 0 2024-03-21T10:08:53.535 INFO:tasks.workunit.client.0.smithi153.stdout:7/445: creat d4/d19/d4d/f95 x:0 0 0 2024-03-21T10:08:53.540 INFO:tasks.workunit.client.0.smithi153.stdout:9/547: dread - d1/d43/d44/d46/d9e/d61/f71 zero size 2024-03-21T10:08:53.543 INFO:tasks.workunit.client.0.smithi153.stdout:9/548: chown d1/d43/d44/d46/f37 17085343 1 2024-03-21T10:08:53.544 INFO:tasks.workunit.client.0.smithi153.stdout:4/564: rmdir d11/d2d/d3d 39 2024-03-21T10:08:53.545 INFO:tasks.workunit.client.1.smithi174.stdout:5/384: dwrite d1/d16/d1c/f6d [0,4194304] 0 2024-03-21T10:08:53.546 INFO:tasks.workunit.client.0.smithi153.stdout:6/714: read d1f/d93/dd8/d59/f81 [499861,90978] 0 2024-03-21T10:08:53.547 INFO:tasks.workunit.client.0.smithi153.stdout:2/614: dread dd/d15/d18/f24 [4194304,4194304] 0 2024-03-21T10:08:53.548 INFO:tasks.workunit.client.0.smithi153.stdout:5/548: truncate d4/d8/d5a/d5d/f5b 730647 0 2024-03-21T10:08:53.550 INFO:tasks.workunit.client.0.smithi153.stdout:5/549: dread - d4/d8/d5c/d90/d95/f99 zero size 2024-03-21T10:08:53.550 INFO:tasks.workunit.client.0.smithi153.stdout:8/553: unlink d5/d1b/f5e 0 2024-03-21T10:08:53.551 INFO:tasks.workunit.client.1.smithi174.stdout:5/385: rename d1/d49/d58 to d1/d16/d2e/d31/d85 0 2024-03-21T10:08:53.554 INFO:tasks.workunit.client.1.smithi174.stdout:5/386: creat d1/f86 x:0 0 0 2024-03-21T10:08:53.556 INFO:tasks.workunit.client.1.smithi174.stdout:5/387: write d1/f75 [404471,86412] 0 2024-03-21T10:08:53.557 INFO:tasks.workunit.client.0.smithi153.stdout:0/463: mknod d4/d4c/d83/c8c 0 2024-03-21T10:08:53.558 INFO:tasks.workunit.client.1.smithi174.stdout:3/343: truncate d0/d12/d14/d1a/d27/f43 2752725 0 2024-03-21T10:08:53.560 INFO:tasks.workunit.client.0.smithi153.stdout:7/446: rename d4/d2a/d40/f87 to d4/d14/d2f/d78/d84/f96 0 2024-03-21T10:08:53.561 INFO:tasks.workunit.client.1.smithi174.stdout:5/388: mkdir d1/d16/d2e/d87 0 2024-03-21T10:08:53.561 INFO:tasks.workunit.client.1.smithi174.stdout:3/344: read d0/d12/f17 [2479702,85340] 0 2024-03-21T10:08:53.562 INFO:tasks.workunit.client.0.smithi153.stdout:7/447: write d4/d2a/f2d [2426326,52900] 0 2024-03-21T10:08:53.566 INFO:tasks.workunit.client.1.smithi174.stdout:5/389: mknod d1/d49/c88 0 2024-03-21T10:08:53.567 INFO:tasks.workunit.client.1.smithi174.stdout:3/345: creat d0/d12/d14/d4c/f7b x:0 0 0 2024-03-21T10:08:53.569 INFO:tasks.workunit.client.0.smithi153.stdout:9/549: readlink d1/d43/d44/d46/d9e/d24/l7c 0 2024-03-21T10:08:53.570 INFO:tasks.workunit.client.1.smithi174.stdout:3/346: chown d0/d12/d14/d1a/d27/d3c/f5a 2549 1 2024-03-21T10:08:53.572 INFO:tasks.workunit.client.0.smithi153.stdout:9/550: write d1/d4/f75 [631055,61114] 0 2024-03-21T10:08:53.573 INFO:tasks.workunit.client.1.smithi174.stdout:0/372: truncate d9/f15 428290 0 2024-03-21T10:08:53.575 INFO:tasks.workunit.client.0.smithi153.stdout:6/715: mkdir d1f/d93/dd8/d103 0 2024-03-21T10:08:53.577 INFO:tasks.workunit.client.1.smithi174.stdout:9/388: truncate d2/d11/d1d/f4e 4974657 0 2024-03-21T10:08:53.578 INFO:tasks.workunit.client.0.smithi153.stdout:6/716: chown d1f/d2b/d33/d40/d4f/fce 2431918 1 2024-03-21T10:08:53.580 INFO:tasks.workunit.client.0.smithi153.stdout:6/717: dread - d1f/d2b/d33/d40/d51/d98/ffe zero size 2024-03-21T10:08:53.581 INFO:tasks.workunit.client.0.smithi153.stdout:5/550: rmdir d4/d5/d16 39 2024-03-21T10:08:53.582 INFO:tasks.workunit.client.0.smithi153.stdout:8/554: dread d5/d9/d43/dd/d2d/f66 [0,4194304] 0 2024-03-21T10:08:53.583 INFO:tasks.workunit.client.1.smithi174.stdout:5/390: dwrite d1/d16/d1c/f23 [0,4194304] 0 2024-03-21T10:08:53.585 INFO:tasks.workunit.client.1.smithi174.stdout:5/391: chown d1/f29 672352978 1 2024-03-21T10:08:53.589 INFO:tasks.workunit.client.1.smithi174.stdout:5/392: rename d1/d16/d2e/d66/f76 to d1/d16/d2e/d66/f89 0 2024-03-21T10:08:53.591 INFO:tasks.workunit.client.1.smithi174.stdout:5/393: write d1/d2f/f5e [837377,7787] 0 2024-03-21T10:08:53.591 INFO:tasks.workunit.client.0.smithi153.stdout:0/464: unlink d4/d16/d19/d30/l3d 0 2024-03-21T10:08:53.593 INFO:tasks.workunit.client.1.smithi174.stdout:5/394: chown d1/d16/c1f 0 1 2024-03-21T10:08:53.595 INFO:tasks.workunit.client.0.smithi153.stdout:3/546: dwrite dd/d1e/d2f/d35/f7b [0,4194304] 0 2024-03-21T10:08:53.596 INFO:tasks.workunit.client.0.smithi153.stdout:7/448: dread - d4/d1b/d83/f8f zero size 2024-03-21T10:08:53.596 INFO:tasks.workunit.client.1.smithi174.stdout:3/347: rmdir d0/d12 39 2024-03-21T10:08:53.604 INFO:tasks.workunit.client.1.smithi174.stdout:0/373: truncate d9/d35/d4a/f5e 2820698 0 2024-03-21T10:08:53.604 INFO:tasks.workunit.client.0.smithi153.stdout:2/615: fsync fc 0 2024-03-21T10:08:53.606 INFO:tasks.workunit.client.0.smithi153.stdout:9/551: chown d1/d43/d44/d46/d9e/lac 4418516 1 2024-03-21T10:08:53.610 INFO:tasks.workunit.client.0.smithi153.stdout:2/616: write dd/d15/d5e/fa4 [738996,62170] 0 2024-03-21T10:08:53.610 INFO:tasks.workunit.client.0.smithi153.stdout:9/552: chown d1/d43/d44/d46/d9e/d4b/l93 0 1 2024-03-21T10:08:53.611 INFO:tasks.workunit.client.0.smithi153.stdout:6/718: fsync d1f/d2b/d33/d40/d4f/d66/fd0 0 2024-03-21T10:08:53.612 INFO:tasks.workunit.client.0.smithi153.stdout:8/555: fdatasync d5/d9/f19 0 2024-03-21T10:08:53.617 INFO:tasks.workunit.client.1.smithi174.stdout:0/374: creat d9/d1c/f7d x:0 0 0 2024-03-21T10:08:53.619 INFO:tasks.workunit.client.0.smithi153.stdout:1/432: sync 2024-03-21T10:08:53.619 INFO:tasks.workunit.client.1.smithi174.stdout:3/348: link d0/d12/d14/d1a/d27/d4f/d64/l6e d0/d12/d14/d1a/d27/d4f/l7c 0 2024-03-21T10:08:53.622 INFO:tasks.workunit.client.0.smithi153.stdout:0/465: mknod d4/db/c8d 0 2024-03-21T10:08:53.623 INFO:tasks.workunit.client.1.smithi174.stdout:5/395: dwrite d1/d16/f78 [0,4194304] 0 2024-03-21T10:08:53.625 INFO:tasks.workunit.client.1.smithi174.stdout:9/389: dwrite d2/d11/d60/f6d [0,4194304] 0 2024-03-21T10:08:53.625 INFO:tasks.workunit.client.1.smithi174.stdout:0/375: read d9/f19 [10665135,49435] 0 2024-03-21T10:08:53.626 INFO:tasks.workunit.client.1.smithi174.stdout:3/349: mkdir d0/d12/d14/d1a/d27/d4f/d5d/d7d 0 2024-03-21T10:08:53.629 INFO:tasks.workunit.client.0.smithi153.stdout:0/466: write d4/db/f13 [447674,11456] 0 2024-03-21T10:08:53.630 INFO:tasks.workunit.client.1.smithi174.stdout:0/376: rmdir d9/d35/d4f/d61 39 2024-03-21T10:08:53.631 INFO:tasks.workunit.client.0.smithi153.stdout:4/565: rename d11/d18/d50/d65/f6a to d11/d18/d54/fa7 0 2024-03-21T10:08:53.631 INFO:tasks.workunit.client.0.smithi153.stdout:7/449: rmdir d4/d2a 39 2024-03-21T10:08:53.632 INFO:tasks.workunit.client.1.smithi174.stdout:2/366: sync 2024-03-21T10:08:53.632 INFO:tasks.workunit.client.0.smithi153.stdout:0/467: fdatasync d4/d16/d19/d58/f6a 0 2024-03-21T10:08:53.634 INFO:tasks.workunit.client.1.smithi174.stdout:5/396: rename d1/d2f/f5e to d1/d16/d1c/d81/f8a 0 2024-03-21T10:08:53.638 INFO:tasks.workunit.client.1.smithi174.stdout:3/350: mkdir d0/d12/d14/d1a/d44/d65/d7e 0 2024-03-21T10:08:53.641 INFO:tasks.workunit.client.0.smithi153.stdout:8/556: dwrite d5/d9/d43/dd/d2d/d45/d88/f5d [0,4194304] 0 2024-03-21T10:08:53.643 INFO:tasks.workunit.client.0.smithi153.stdout:6/719: symlink d1f/d2b/l104 0 2024-03-21T10:08:53.644 INFO:tasks.workunit.client.0.smithi153.stdout:5/551: symlink d4/d5/df/d67/da0/lad 0 2024-03-21T10:08:53.645 INFO:tasks.workunit.client.1.smithi174.stdout:9/390: link d2/d24/f2f d2/d11/f87 0 2024-03-21T10:08:53.647 INFO:tasks.workunit.client.1.smithi174.stdout:5/397: dread d1/f71 [0,4194304] 0 2024-03-21T10:08:53.648 INFO:tasks.workunit.client.1.smithi174.stdout:2/367: truncate d5/d25/d27/f6a 2506168 0 2024-03-21T10:08:53.652 INFO:tasks.workunit.client.0.smithi153.stdout:1/433: creat d6/d3a/d3c/d50/f7f x:0 0 0 2024-03-21T10:08:53.653 INFO:tasks.workunit.client.0.smithi153.stdout:3/547: getdents dd/d1e/da6 0 2024-03-21T10:08:53.653 INFO:tasks.workunit.client.0.smithi153.stdout:8/557: truncate d5/d9/d43/dd/d20/d26/d8b/faf 887777 0 2024-03-21T10:08:53.654 INFO:tasks.workunit.client.0.smithi153.stdout:4/566: creat d11/d5e/fa8 x:0 0 0 2024-03-21T10:08:53.656 INFO:tasks.workunit.client.1.smithi174.stdout:0/377: rename d9/d21/d31/f57 to d9/d21/d31/d3c/d72/f7e 0 2024-03-21T10:08:53.658 INFO:tasks.workunit.client.1.smithi174.stdout:9/391: chown d2/d11/c14 168901899 1 2024-03-21T10:08:53.658 INFO:tasks.workunit.client.0.smithi153.stdout:2/617: dread dd/d15/d18/d2c/f2d [0,4194304] 0 2024-03-21T10:08:53.658 INFO:tasks.workunit.client.0.smithi153.stdout:0/468: unlink d4/d16/d3e/f63 0 2024-03-21T10:08:53.659 INFO:tasks.workunit.client.1.smithi174.stdout:5/398: rmdir d1/d16/d2e/d31 39 2024-03-21T10:08:53.660 INFO:tasks.workunit.client.1.smithi174.stdout:3/351: mkdir d0/d12/d14/d1a/d27/d7f 0 2024-03-21T10:08:53.663 INFO:tasks.workunit.client.0.smithi153.stdout:6/720: mkdir d1f/de0/de1/d105 0 2024-03-21T10:08:53.668 INFO:tasks.workunit.client.1.smithi174.stdout:9/392: creat d2/d24/f88 x:0 0 0 2024-03-21T10:08:53.670 INFO:tasks.workunit.client.0.smithi153.stdout:8/558: creat d5/d9/d43/dd/d2d/d45/fba x:0 0 0 2024-03-21T10:08:53.672 INFO:tasks.workunit.client.1.smithi174.stdout:2/368: dwrite d5/f4a [0,4194304] 0 2024-03-21T10:08:53.672 INFO:tasks.workunit.client.1.smithi174.stdout:5/399: rename d1/d16/d2e/d31 to d1/d16/d2e/d66/d8b 0 2024-03-21T10:08:53.676 INFO:tasks.workunit.client.0.smithi153.stdout:5/552: mknod d4/d5/cae 0 2024-03-21T10:08:53.677 INFO:tasks.workunit.client.0.smithi153.stdout:1/434: mkdir d6/d3a/d66/d80 0 2024-03-21T10:08:53.678 INFO:tasks.workunit.client.0.smithi153.stdout:1/435: readlink d6/d3a/d3c/d50/l59 0 2024-03-21T10:08:53.679 INFO:tasks.workunit.client.1.smithi174.stdout:3/352: creat d0/d12/d14/f80 x:0 0 0 2024-03-21T10:08:53.682 INFO:tasks.workunit.client.1.smithi174.stdout:3/353: chown d0/d12/d14/d1a/d44/d56 255 1 2024-03-21T10:08:53.682 INFO:tasks.workunit.client.0.smithi153.stdout:3/548: fdatasync dd/d1e/f4f 0 2024-03-21T10:08:53.683 INFO:tasks.workunit.client.1.smithi174.stdout:9/393: mknod d2/d11/d1d/d4b/c89 0 2024-03-21T10:08:53.684 INFO:tasks.workunit.client.1.smithi174.stdout:5/400: mknod d1/d16/d2e/d66/d8b/c8c 0 2024-03-21T10:08:53.686 INFO:tasks.workunit.client.1.smithi174.stdout:0/378: dwrite d9/d21/d31/d3c/f5b [0,4194304] 0 2024-03-21T10:08:53.691 INFO:tasks.workunit.client.1.smithi174.stdout:2/369: creat d5/d25/d27/d2d/d45/d5b/d2f/d4b/d76/f7f x:0 0 0 2024-03-21T10:08:53.692 INFO:tasks.workunit.client.0.smithi153.stdout:5/553: mknod d4/d8/d5c/caf 0 2024-03-21T10:08:53.694 INFO:tasks.workunit.client.0.smithi153.stdout:9/553: truncate d1/d43/d44/d46/d9e/d61/f87 786736 0 2024-03-21T10:08:53.694 INFO:tasks.workunit.client.0.smithi153.stdout:7/450: mkdir d4/d2a/d40/d97 0 2024-03-21T10:08:53.695 INFO:tasks.workunit.client.1.smithi174.stdout:2/370: unlink d5/c6 0 2024-03-21T10:08:53.699 INFO:tasks.workunit.client.1.smithi174.stdout:5/401: mkdir d1/d16/d2e/d8d 0 2024-03-21T10:08:53.703 INFO:tasks.workunit.client.0.smithi153.stdout:6/721: rmdir d1f/de0/de1/d105 0 2024-03-21T10:08:53.703 INFO:tasks.workunit.client.0.smithi153.stdout:0/469: getdents d4/d16/d19/d58 0 2024-03-21T10:08:53.704 INFO:tasks.workunit.client.0.smithi153.stdout:1/436: fsync d6/d13/f5e 0 2024-03-21T10:08:53.704 INFO:tasks.workunit.client.0.smithi153.stdout:3/549: mkdir dd/d1e/d3d/d94/d73/dc4 0 2024-03-21T10:08:53.705 INFO:tasks.workunit.client.1.smithi174.stdout:2/371: creat d5/d25/d27/f80 x:0 0 0 2024-03-21T10:08:53.706 INFO:tasks.workunit.client.1.smithi174.stdout:0/379: link d9/d35/d4f/d56/f66 d9/d21/f7f 0 2024-03-21T10:08:53.708 INFO:tasks.workunit.client.0.smithi153.stdout:9/554: unlink d1/d43/d44/d46/d9e/d2e/d67/lb8 0 2024-03-21T10:08:53.709 INFO:tasks.workunit.client.0.smithi153.stdout:0/470: write d4/db/f29 [2607920,53918] 0 2024-03-21T10:08:53.709 INFO:tasks.workunit.client.0.smithi153.stdout:1/437: rename d6 to d6/d3a/d3c/d81 22 2024-03-21T10:08:53.714 INFO:tasks.workunit.client.1.smithi174.stdout:3/354: dread d0/d12/d14/d1a/d27/f2c [0,4194304] 0 2024-03-21T10:08:53.714 INFO:tasks.workunit.client.1.smithi174.stdout:9/394: dwrite d2/d11/d60/f86 [0,4194304] 0 2024-03-21T10:08:53.714 INFO:tasks.workunit.client.0.smithi153.stdout:8/559: rmdir d5/d9/d43/dd/d2d/d45 39 2024-03-21T10:08:53.715 INFO:tasks.workunit.client.1.smithi174.stdout:2/372: truncate d5/d25/d27/d2d/d45/d5b/d2f/d4b/f78 843155 0 2024-03-21T10:08:53.716 INFO:tasks.workunit.client.1.smithi174.stdout:0/380: getdents d9/d21/d23 0 2024-03-21T10:08:53.716 INFO:tasks.workunit.client.1.smithi174.stdout:3/355: unlink d0/l24 0 2024-03-21T10:08:53.717 INFO:tasks.workunit.client.1.smithi174.stdout:9/395: readlink d2/d24/d2e/d63/l68 0 2024-03-21T10:08:53.721 INFO:tasks.workunit.client.1.smithi174.stdout:0/381: mkdir d9/d21/d23/d80 0 2024-03-21T10:08:53.721 INFO:tasks.workunit.client.0.smithi153.stdout:4/567: truncate d11/d18/f28 918802 0 2024-03-21T10:08:53.725 INFO:tasks.workunit.client.1.smithi174.stdout:3/356: write d0/d12/f2d [4266922,126609] 0 2024-03-21T10:08:53.727 INFO:tasks.workunit.client.0.smithi153.stdout:0/471: symlink d4/d4c/l8e 0 2024-03-21T10:08:53.727 INFO:tasks.workunit.client.1.smithi174.stdout:5/402: dwrite d1/d16/d2e/f80 [0,4194304] 0 2024-03-21T10:08:53.731 INFO:tasks.workunit.client.0.smithi153.stdout:9/555: symlink d1/d43/d72/lbf 0 2024-03-21T10:08:53.732 INFO:tasks.workunit.client.1.smithi174.stdout:9/396: symlink d2/d24/d2e/l8a 0 2024-03-21T10:08:53.732 INFO:tasks.workunit.client.1.smithi174.stdout:0/382: mkdir d9/d21/d81 0 2024-03-21T10:08:53.733 INFO:tasks.workunit.client.1.smithi174.stdout:2/373: mkdir d5/d25/d27/d2e/d7b/d81 0 2024-03-21T10:08:53.734 INFO:tasks.workunit.client.0.smithi153.stdout:9/556: chown d1/d43/d44/d46/d9e/d24/f38 2 1 2024-03-21T10:08:53.736 INFO:tasks.workunit.client.1.smithi174.stdout:5/403: unlink d1/d16/d2e/d66/d8b/f4b 0 2024-03-21T10:08:53.737 INFO:tasks.workunit.client.0.smithi153.stdout:6/722: unlink d1f/d2b/d33/d40/d7b/c7f 0 2024-03-21T10:08:53.738 INFO:tasks.workunit.client.1.smithi174.stdout:0/383: mkdir d9/d35/d4f/d79/d82 0 2024-03-21T10:08:53.740 INFO:tasks.workunit.client.1.smithi174.stdout:9/397: mkdir d2/d11/d1d/d8b 0 2024-03-21T10:08:53.740 INFO:tasks.workunit.client.1.smithi174.stdout:2/374: mknod d5/d25/d27/d2d/d45/d5b/d2f/d4b/c82 0 2024-03-21T10:08:53.742 INFO:tasks.workunit.client.0.smithi153.stdout:7/451: creat d4/d2a/d3b/f98 x:0 0 0 2024-03-21T10:08:53.742 INFO:tasks.workunit.client.1.smithi174.stdout:0/384: chown d9/d35/d4f/d61 127495 1 2024-03-21T10:08:53.746 INFO:tasks.workunit.client.0.smithi153.stdout:5/554: getdents d4/d8/d5a/d92 0 2024-03-21T10:08:53.747 INFO:tasks.workunit.client.1.smithi174.stdout:3/357: dwrite d0/d12/d14/d1a/d27/d3c/f72 [0,4194304] 0 2024-03-21T10:08:53.747 INFO:tasks.workunit.client.1.smithi174.stdout:0/385: unlink d9/d21/d23/f41 0 2024-03-21T10:08:53.747 INFO:tasks.workunit.client.0.smithi153.stdout:7/452: chown d4/d1b/d20/c38 1292 1 2024-03-21T10:08:53.748 INFO:tasks.workunit.client.1.smithi174.stdout:3/358: fsync d0/fa 0 2024-03-21T10:08:53.750 INFO:tasks.workunit.client.0.smithi153.stdout:4/568: dwrite d11/f6f [4194304,4194304] 0 2024-03-21T10:08:53.750 INFO:tasks.workunit.client.1.smithi174.stdout:3/359: write d0/d12/d46/f4b [426511,21552] 0 2024-03-21T10:08:53.751 INFO:tasks.workunit.client.0.smithi153.stdout:3/550: mkdir dd/d41/dab/dc5 0 2024-03-21T10:08:53.755 INFO:tasks.workunit.client.1.smithi174.stdout:2/375: link d5/f38 d5/d25/d27/d2d/d45/d5b/d55/f83 0 2024-03-21T10:08:53.758 INFO:tasks.workunit.client.1.smithi174.stdout:3/360: creat d0/d12/d14/d1a/d27/d3c/f81 x:0 0 0 2024-03-21T10:08:53.760 INFO:tasks.workunit.client.1.smithi174.stdout:3/361: write d0/fa [2934795,73137] 0 2024-03-21T10:08:53.767 INFO:tasks.workunit.client.1.smithi174.stdout:3/362: symlink d0/d12/d14/d1a/d44/l82 0 2024-03-21T10:08:53.767 INFO:tasks.workunit.client.1.smithi174.stdout:2/376: link d5/d25/d27/d2d/d45/d5b/d2f/d4b/c82 d5/d25/d27/d2d/d45/d5b/d55/d75/c84 0 2024-03-21T10:08:53.770 INFO:tasks.workunit.client.1.smithi174.stdout:3/363: dread - d0/d12/d14/d1a/d27/d4f/d5d/f78 zero size 2024-03-21T10:08:53.771 INFO:tasks.workunit.client.1.smithi174.stdout:2/377: write d5/d25/d27/d2d/d45/d5b/d63/f59 [1170646,90960] 0 2024-03-21T10:08:53.775 INFO:tasks.workunit.client.0.smithi153.stdout:5/555: dread d4/d8/f1d [0,4194304] 0 2024-03-21T10:08:53.779 INFO:tasks.workunit.client.0.smithi153.stdout:7/453: rename d4/d14/d2f/d78/f54 to d4/d2a/d3b/d81/f99 0 2024-03-21T10:08:53.783 INFO:tasks.workunit.client.0.smithi153.stdout:6/723: truncate d1f/d2b/f89 260911 0 2024-03-21T10:08:53.786 INFO:tasks.workunit.client.1.smithi174.stdout:5/404: truncate d1/d49/d4f/f57 4084944 0 2024-03-21T10:08:53.787 INFO:tasks.workunit.client.0.smithi153.stdout:9/557: mknod d1/d43/d44/d46/d9e/cc0 0 2024-03-21T10:08:53.790 INFO:tasks.workunit.client.0.smithi153.stdout:8/560: write d5/d9/d43/dd/d20/d26/d24/d52/d55/f5c [1965200,83504] 0 2024-03-21T10:08:53.791 INFO:tasks.workunit.client.0.smithi153.stdout:4/569: creat d11/d2d/d6b/d9c/da0/fa9 x:0 0 0 2024-03-21T10:08:53.791 INFO:tasks.workunit.client.0.smithi153.stdout:9/558: read d1/f1b [4137462,119529] 0 2024-03-21T10:08:53.792 INFO:tasks.workunit.client.1.smithi174.stdout:0/386: truncate d9/d21/d31/d3c/d72/f7e 94680 0 2024-03-21T10:08:53.797 INFO:tasks.workunit.client.0.smithi153.stdout:4/570: truncate d11/d18/f95 871830 0 2024-03-21T10:08:53.798 INFO:tasks.workunit.client.0.smithi153.stdout:0/472: write d4/f37 [1500195,17237] 0 2024-03-21T10:08:53.798 INFO:tasks.workunit.client.1.smithi174.stdout:2/378: dwrite d5/d25/d27/d2d/d45/d5b/f22 [0,4194304] 0 2024-03-21T10:08:53.801 INFO:tasks.workunit.client.1.smithi174.stdout:3/364: dwrite d0/d12/d46/f6f [4194304,4194304] 0 2024-03-21T10:08:53.801 INFO:tasks.workunit.client.1.smithi174.stdout:1/361: sync 2024-03-21T10:08:53.801 INFO:tasks.workunit.client.1.smithi174.stdout:4/414: sync 2024-03-21T10:08:53.802 INFO:tasks.workunit.client.1.smithi174.stdout:5/405: rename d1/fd to d1/d16/d2e/d66/d8b/d85/f8e 0 2024-03-21T10:08:53.805 INFO:tasks.workunit.client.1.smithi174.stdout:1/362: chown d4/d67/f74 1938169240 1 2024-03-21T10:08:53.805 INFO:tasks.workunit.client.1.smithi174.stdout:0/387: fsync d9/d21/d31/d3c/f60 0 2024-03-21T10:08:53.806 INFO:tasks.workunit.client.1.smithi174.stdout:4/415: chown d4/d21/d24/d28 0 1 2024-03-21T10:08:53.808 INFO:tasks.workunit.client.1.smithi174.stdout:2/379: write d5/d25/d27/d2d/f43 [31261,120668] 0 2024-03-21T10:08:53.810 INFO:tasks.workunit.client.1.smithi174.stdout:3/365: fsync d0/d12/f54 0 2024-03-21T10:08:53.810 INFO:tasks.workunit.client.1.smithi174.stdout:9/398: dwrite d2/d24/f2f [0,4194304] 0 2024-03-21T10:08:53.815 INFO:tasks.workunit.client.1.smithi174.stdout:1/363: unlink d4/d7/l6e 0 2024-03-21T10:08:53.815 INFO:tasks.workunit.client.1.smithi174.stdout:5/406: chown d1/d16/d2e/d66/d8b/d85/f5b 10 1 2024-03-21T10:08:53.817 INFO:tasks.workunit.client.0.smithi153.stdout:1/438: dwrite d6/d13/f2d [0,4194304] 0 2024-03-21T10:08:53.825 INFO:tasks.workunit.client.1.smithi174.stdout:4/416: unlink d4/d21/d31/d3a/d67/d76/c80 0 2024-03-21T10:08:53.825 INFO:tasks.workunit.client.1.smithi174.stdout:2/380: chown d5/d25/d27/f6a 8752976 1 2024-03-21T10:08:53.825 INFO:tasks.workunit.client.1.smithi174.stdout:1/364: mknod d4/d7/d15/c7f 0 2024-03-21T10:08:53.826 INFO:tasks.workunit.client.0.smithi153.stdout:5/556: dread d4/d8/d5a/d5d/f5b [0,4194304] 0 2024-03-21T10:08:53.830 INFO:tasks.workunit.client.0.smithi153.stdout:7/454: dwrite d4/d2a/d3b/d81/f99 [0,4194304] 0 2024-03-21T10:08:53.831 INFO:tasks.workunit.client.0.smithi153.stdout:6/724: mknod d1f/d2b/d33/d40/c106 0 2024-03-21T10:08:53.831 INFO:tasks.workunit.client.0.smithi153.stdout:0/473: symlink d4/d16/d3e/d5b/l8f 0 2024-03-21T10:08:53.831 INFO:tasks.workunit.client.0.smithi153.stdout:4/571: mknod d11/d2d/d6b/caa 0 2024-03-21T10:08:53.832 INFO:tasks.workunit.client.1.smithi174.stdout:3/366: mknod d0/c83 0 2024-03-21T10:08:53.832 INFO:tasks.workunit.client.0.smithi153.stdout:4/572: readlink d11/d18/l47 0 2024-03-21T10:08:53.833 INFO:tasks.workunit.client.1.smithi174.stdout:9/399: creat d2/d24/d2e/d7e/f8c x:0 0 0 2024-03-21T10:08:53.835 INFO:tasks.workunit.client.0.smithi153.stdout:8/561: dread d5/d9/d43/dd/d20/d26/d24/f28 [0,4194304] 0 2024-03-21T10:08:53.836 INFO:tasks.workunit.client.0.smithi153.stdout:9/559: link d1/d4e/f98 d1/d4e/fc1 0 2024-03-21T10:08:53.837 INFO:tasks.workunit.client.1.smithi174.stdout:3/367: dread - d0/d12/d14/d1a/d27/d3c/f81 zero size 2024-03-21T10:08:53.839 INFO:tasks.workunit.client.0.smithi153.stdout:1/439: mknod d6/d3a/d66/c82 0 2024-03-21T10:08:53.841 INFO:tasks.workunit.client.1.smithi174.stdout:1/365: unlink d4/d7/c5a 0 2024-03-21T10:08:53.842 INFO:tasks.workunit.client.0.smithi153.stdout:5/557: mknod d4/d8/d3b/d56/cb0 0 2024-03-21T10:08:53.843 INFO:tasks.workunit.client.1.smithi174.stdout:7/282: sync 2024-03-21T10:08:53.843 INFO:tasks.workunit.client.1.smithi174.stdout:8/307: sync 2024-03-21T10:08:53.843 INFO:tasks.workunit.client.1.smithi174.stdout:6/293: sync 2024-03-21T10:08:53.845 INFO:tasks.workunit.client.1.smithi174.stdout:1/366: write d4/d7/d33/d43/f5d [1126212,44227] 0 2024-03-21T10:08:53.846 INFO:tasks.workunit.client.1.smithi174.stdout:2/381: unlink d5/d25/d27/d74/l7d 0 2024-03-21T10:08:53.846 INFO:tasks.workunit.client.1.smithi174.stdout:5/407: dwrite d1/d16/d2e/d66/d8b/f48 [0,4194304] 0 2024-03-21T10:08:53.847 INFO:tasks.workunit.client.1.smithi174.stdout:6/294: write d0/d3/d20/d3d/f59 [777969,127065] 0 2024-03-21T10:08:53.847 INFO:tasks.workunit.client.0.smithi153.stdout:4/573: mkdir d11/d18/d54/dab 0 2024-03-21T10:08:53.849 INFO:tasks.workunit.client.1.smithi174.stdout:4/417: rename d4/c6 to d4/c8b 0 2024-03-21T10:08:53.851 INFO:tasks.workunit.client.1.smithi174.stdout:5/408: chown d1/d16/f60 88203201 1 2024-03-21T10:08:53.852 INFO:tasks.workunit.client.1.smithi174.stdout:0/388: getdents d9/d21/d5f 0 2024-03-21T10:08:53.853 INFO:tasks.workunit.client.0.smithi153.stdout:4/574: dread - d11/d5e/fa8 zero size 2024-03-21T10:08:53.858 INFO:tasks.workunit.client.0.smithi153.stdout:5/558: write d4/d5/f57 [5607528,85915] 0 2024-03-21T10:08:53.861 INFO:tasks.workunit.client.0.smithi153.stdout:8/562: unlink d5/d9/d43/dd/d2d/d45/d88/f3c 0 2024-03-21T10:08:53.864 INFO:tasks.workunit.client.1.smithi174.stdout:3/368: symlink d0/d12/d14/d1a/d27/d3c/l84 0 2024-03-21T10:08:53.868 INFO:tasks.workunit.client.1.smithi174.stdout:7/283: fdatasync d0/d4/d10/d1b/f25 0 2024-03-21T10:08:53.868 INFO:tasks.workunit.client.0.smithi153.stdout:0/474: dwrite d4/f62 [0,4194304] 0 2024-03-21T10:08:53.868 INFO:tasks.workunit.client.0.smithi153.stdout:1/440: dread d6/d13/f5e [0,4194304] 0 2024-03-21T10:08:53.869 INFO:tasks.workunit.client.1.smithi174.stdout:1/367: write d4/d7/d33/d43/f47 [4040651,86933] 0 2024-03-21T10:08:53.873 INFO:tasks.workunit.client.1.smithi174.stdout:8/308: rename d9/d17/d19/d27/c2e to d9/d2a/d39/c62 0 2024-03-21T10:08:53.873 INFO:tasks.workunit.client.1.smithi174.stdout:4/418: unlink d4/d19/l1e 0 2024-03-21T10:08:53.874 INFO:tasks.workunit.client.1.smithi174.stdout:2/382: mkdir d5/d25/d27/d2e/d85 0 2024-03-21T10:08:53.875 INFO:tasks.workunit.client.0.smithi153.stdout:8/563: write d5/d9/d43/dd/d20/d26/d8b/f62 [230113,102182] 0 2024-03-21T10:08:53.877 INFO:tasks.workunit.client.1.smithi174.stdout:4/419: write d4/d21/d24/d28/f4e [5204832,122051] 0 2024-03-21T10:08:53.877 INFO:tasks.workunit.client.1.smithi174.stdout:0/389: unlink d9/d21/d31/d3c/d72/l73 0 2024-03-21T10:08:53.880 INFO:tasks.workunit.client.1.smithi174.stdout:9/400: creat d2/d11/d7d/f8d x:0 0 0 2024-03-21T10:08:53.880 INFO:tasks.workunit.client.1.smithi174.stdout:3/369: fdatasync d0/f7 0 2024-03-21T10:08:53.883 INFO:tasks.workunit.client.0.smithi153.stdout:1/441: chown d6/d3a/d3c/d50/l6b 339 1 2024-03-21T10:08:53.883 INFO:tasks.workunit.client.0.smithi153.stdout:7/455: dread d4/f10 [0,4194304] 0 2024-03-21T10:08:53.884 INFO:tasks.workunit.client.1.smithi174.stdout:1/368: rmdir d4/d7/d28/d59 39 2024-03-21T10:08:53.884 INFO:tasks.workunit.client.1.smithi174.stdout:5/409: rename d1/d16/d2e/d66/d8b/d84 to d1/d49/d4f/d51/d8f 0 2024-03-21T10:08:53.885 INFO:tasks.workunit.client.0.smithi153.stdout:4/575: getdents d11/d18/d6d 0 2024-03-21T10:08:53.886 INFO:tasks.workunit.client.1.smithi174.stdout:0/390: dread - d9/d21/d5f/f6c zero size 2024-03-21T10:08:53.887 INFO:tasks.workunit.client.1.smithi174.stdout:4/420: unlink d4/fc 0 2024-03-21T10:08:53.888 INFO:tasks.workunit.client.1.smithi174.stdout:3/370: chown d0/d12/d14/d1a/d27/d4f/d5d/l4a 2 1 2024-03-21T10:08:53.889 INFO:tasks.workunit.client.1.smithi174.stdout:1/369: creat d4/d2d/f80 x:0 0 0 2024-03-21T10:08:53.891 INFO:tasks.workunit.client.1.smithi174.stdout:2/383: rename d5/f24 to d5/d25/d27/d2d/d45/d5b/d2f/d4b/d76/f86 0 2024-03-21T10:08:53.892 INFO:tasks.workunit.client.0.smithi153.stdout:7/456: write d4/d19/f31 [623540,99090] 0 2024-03-21T10:08:53.893 INFO:tasks.workunit.client.0.smithi153.stdout:6/725: dwrite d1f/d93/dd8/fd1 [4194304,4194304] 0 2024-03-21T10:08:53.893 INFO:tasks.workunit.client.0.smithi153.stdout:8/564: creat d5/d9/d43/dd/d2d/d45/d88/d85/fbb x:0 0 0 2024-03-21T10:08:53.895 INFO:tasks.workunit.client.1.smithi174.stdout:3/371: fsync d0/d12/d14/d1a/d27/f47 0 2024-03-21T10:08:53.896 INFO:tasks.workunit.client.1.smithi174.stdout:5/410: creat d1/d16/d2e/d66/d8b/d85/f90 x:0 0 0 2024-03-21T10:08:53.897 INFO:tasks.workunit.client.1.smithi174.stdout:1/370: truncate d4/d7/f2e 1011621 0 2024-03-21T10:08:53.897 INFO:tasks.workunit.client.1.smithi174.stdout:8/309: creat d9/f63 x:0 0 0 2024-03-21T10:08:53.897 INFO:tasks.workunit.client.0.smithi153.stdout:6/726: chown d1f/c50 1678925998 1 2024-03-21T10:08:53.898 INFO:tasks.workunit.client.1.smithi174.stdout:0/391: write d9/d35/f36 [409111,109779] 0 2024-03-21T10:08:53.900 INFO:tasks.workunit.client.1.smithi174.stdout:9/401: creat d2/f8e x:0 0 0 2024-03-21T10:08:53.903 INFO:tasks.workunit.client.0.smithi153.stdout:5/559: dwrite d4/d5/df/d5e/f81 [0,4194304] 0 2024-03-21T10:08:53.903 INFO:tasks.workunit.client.0.smithi153.stdout:4/576: mkdir d11/d2d/da3/dac 0 2024-03-21T10:08:53.905 INFO:tasks.workunit.client.0.smithi153.stdout:7/457: truncate d4/d1b/f7d 1004447 0 2024-03-21T10:08:53.906 INFO:tasks.workunit.client.0.smithi153.stdout:2/618: sync 2024-03-21T10:08:53.908 INFO:tasks.workunit.client.1.smithi174.stdout:1/371: rename d4/d2d/l37 to d4/d7/d68/l81 0 2024-03-21T10:08:53.909 INFO:tasks.workunit.client.0.smithi153.stdout:0/475: dwrite d4/d16/d27/f79 [0,4194304] 0 2024-03-21T10:08:53.912 INFO:tasks.workunit.client.0.smithi153.stdout:6/727: truncate d1f/d2b/d33/d40/d51/d98/ffe 216184 0 2024-03-21T10:08:53.915 INFO:tasks.workunit.client.1.smithi174.stdout:0/392: creat d9/d21/d31/d3c/f83 x:0 0 0 2024-03-21T10:08:53.918 INFO:tasks.workunit.client.0.smithi153.stdout:1/442: dwrite d6/d13/f45 [0,4194304] 0 2024-03-21T10:08:53.918 INFO:tasks.workunit.client.0.smithi153.stdout:6/728: truncate d1f/f28 4387191 0 2024-03-21T10:08:53.921 INFO:tasks.workunit.client.1.smithi174.stdout:6/295: dread d0/d3/d5/df/d16/f2b [0,4194304] 0 2024-03-21T10:08:53.921 INFO:tasks.workunit.client.1.smithi174.stdout:2/384: dread d5/d25/d27/d2d/f43 [0,4194304] 0 2024-03-21T10:08:53.923 INFO:tasks.workunit.client.0.smithi153.stdout:8/565: dread - d5/d9/d43/dd/d20/d26/d24/d52/d55/f5f zero size 2024-03-21T10:08:53.923 INFO:tasks.workunit.client.0.smithi153.stdout:9/560: dread d1/d43/d44/d46/d9e/d24/f38 [0,4194304] 0 2024-03-21T10:08:53.924 INFO:tasks.workunit.client.1.smithi174.stdout:4/421: dwrite d4/d21/f27 [0,4194304] 0 2024-03-21T10:08:53.925 INFO:tasks.workunit.client.0.smithi153.stdout:4/577: mknod d11/d2d/d3d/cad 0 2024-03-21T10:08:53.925 INFO:tasks.workunit.client.0.smithi153.stdout:5/560: mkdir d4/d8/d5a/d5d/db1 0 2024-03-21T10:08:53.927 INFO:tasks.workunit.client.0.smithi153.stdout:4/578: rename d11 to d11/d2d/d6b/dae 22 2024-03-21T10:08:53.930 INFO:tasks.workunit.client.0.smithi153.stdout:1/443: readlink d6/d3a/d3c/l4d 0 2024-03-21T10:08:53.930 INFO:tasks.workunit.client.1.smithi174.stdout:1/372: rename d4/f4a to d4/d7/d15/d4f/f82 0 2024-03-21T10:08:53.932 INFO:tasks.workunit.client.1.smithi174.stdout:0/393: mknod d9/d21/d31/d3c/c84 0 2024-03-21T10:08:53.937 INFO:tasks.workunit.client.1.smithi174.stdout:6/296: creat d0/d3/d5/d3b/f61 x:0 0 0 2024-03-21T10:08:53.940 INFO:tasks.workunit.client.1.smithi174.stdout:3/372: dwrite d0/d12/d14/d1a/d44/d65/f6b [0,4194304] 0 2024-03-21T10:08:53.940 INFO:tasks.workunit.client.1.smithi174.stdout:9/402: dwrite d2/f45 [0,4194304] 0 2024-03-21T10:08:53.942 INFO:tasks.workunit.client.1.smithi174.stdout:6/297: read d0/d3/d5/f57 [4032901,116017] 0 2024-03-21T10:08:53.948 INFO:tasks.workunit.client.1.smithi174.stdout:1/373: unlink d4/d7/d15/c7f 0 2024-03-21T10:08:53.950 INFO:tasks.workunit.client.1.smithi174.stdout:2/385: dread d5/d25/d27/d2d/d45/d5b/d2f/d68/f7a [0,4194304] 0 2024-03-21T10:08:53.951 INFO:tasks.workunit.client.1.smithi174.stdout:8/310: getdents d9/d17/d19/d27/d60 0 2024-03-21T10:08:53.952 INFO:tasks.workunit.client.1.smithi174.stdout:7/284: write d0/d4/d10/d1b/f25 [142460,106464] 0 2024-03-21T10:08:53.954 INFO:tasks.workunit.client.1.smithi174.stdout:0/394: creat d9/d21/d31/d3c/d72/f85 x:0 0 0 2024-03-21T10:08:53.955 INFO:tasks.workunit.client.1.smithi174.stdout:0/395: fdatasync d9/f65 0 2024-03-21T10:08:53.957 INFO:tasks.workunit.client.1.smithi174.stdout:8/311: fsync d9/d17/d19/d27/f38 0 2024-03-21T10:08:53.958 INFO:tasks.workunit.client.0.smithi153.stdout:2/619: symlink dd/d15/d5e/db3/ld1 0 2024-03-21T10:08:53.958 INFO:tasks.workunit.client.1.smithi174.stdout:8/312: readlink d9/lf 0 2024-03-21T10:08:53.959 INFO:tasks.workunit.client.0.smithi153.stdout:4/579: mknod d11/d2d/d57/caf 0 2024-03-21T10:08:53.960 INFO:tasks.workunit.client.1.smithi174.stdout:6/298: readlink d0/d3/d2a/l41 0 2024-03-21T10:08:53.960 INFO:tasks.workunit.client.0.smithi153.stdout:6/729: link d1f/d2b/d33/d40/d4f/f67 d1f/d2b/db3/f107 0 2024-03-21T10:08:53.961 INFO:tasks.workunit.client.0.smithi153.stdout:7/458: dwrite d4/d2a/f77 [0,4194304] 0 2024-03-21T10:08:53.961 INFO:tasks.workunit.client.1.smithi174.stdout:1/374: creat d4/d7/d15/d4f/f83 x:0 0 0 2024-03-21T10:08:53.961 INFO:tasks.workunit.client.1.smithi174.stdout:8/313: write f5 [2658434,122109] 0 2024-03-21T10:08:53.963 INFO:tasks.workunit.client.1.smithi174.stdout:2/386: creat d5/d25/d27/d2d/d45/d5b/d2f/f87 x:0 0 0 2024-03-21T10:08:53.963 INFO:tasks.workunit.client.0.smithi153.stdout:1/444: dread - d6/d32/d3e/d4f/f58 zero size 2024-03-21T10:08:53.966 INFO:tasks.workunit.client.0.smithi153.stdout:0/476: dread d4/f6 [0,4194304] 0 2024-03-21T10:08:53.967 INFO:tasks.workunit.client.1.smithi174.stdout:9/403: rmdir d2/d11/d1d/d5e 39 2024-03-21T10:08:53.967 INFO:tasks.workunit.client.1.smithi174.stdout:7/285: fsync d0/d4/d10/f1c 0 2024-03-21T10:08:53.968 INFO:tasks.workunit.client.0.smithi153.stdout:6/730: write d1f/d2b/d33/d40/d65/f69 [2725438,29526] 0 2024-03-21T10:08:53.968 INFO:tasks.workunit.client.0.smithi153.stdout:2/620: read - dd/d15/d3c/da7/d91/dae/d50/d58/d79/f7e zero size 2024-03-21T10:08:53.971 INFO:tasks.workunit.client.1.smithi174.stdout:0/396: rename d9/d58 to d9/d21/d31/d5c/d86 0 2024-03-21T10:08:53.976 INFO:tasks.workunit.client.1.smithi174.stdout:7/286: truncate d0/d4/d38/f58 1166853 0 2024-03-21T10:08:53.983 INFO:tasks.workunit.client.1.smithi174.stdout:6/299: symlink d0/d3/d5/d3b/l62 0 2024-03-21T10:08:53.984 INFO:tasks.workunit.client.0.smithi153.stdout:4/580: creat d11/fb0 x:0 0 0 2024-03-21T10:08:53.985 INFO:tasks.workunit.client.1.smithi174.stdout:8/314: creat d9/d17/d19/d27/f64 x:0 0 0 2024-03-21T10:08:53.987 INFO:tasks.workunit.client.1.smithi174.stdout:6/300: fsync d0/d3/d5/d6/d42/f58 0 2024-03-21T10:08:53.988 INFO:tasks.workunit.client.1.smithi174.stdout:2/387: mknod d5/d25/d27/d2d/d45/d5b/d63/c88 0 2024-03-21T10:08:53.990 INFO:tasks.workunit.client.1.smithi174.stdout:6/301: fsync d0/d3/d5/df/d16/f27 0 2024-03-21T10:08:53.991 INFO:tasks.workunit.client.0.smithi153.stdout:2/621: dwrite dd/d15/d18/d2c/d4b/d56/d63/f80 [0,4194304] 0 2024-03-21T10:08:53.994 INFO:tasks.workunit.client.1.smithi174.stdout:1/375: truncate d4/d7/f2c 533822 0 2024-03-21T10:08:53.998 INFO:tasks.workunit.client.1.smithi174.stdout:4/422: getdents d4/d21/d31/d6a 0 2024-03-21T10:08:53.999 INFO:tasks.workunit.client.0.smithi153.stdout:2/622: dread dd/d15/d3c/da7/d91/dae/d50/d58/fa8 [0,4194304] 0 2024-03-21T10:08:54.001 INFO:tasks.workunit.client.1.smithi174.stdout:7/287: mkdir d0/d53/d5f 0 2024-03-21T10:08:54.002 INFO:tasks.workunit.client.1.smithi174.stdout:4/423: read - d4/d21/d24/d28/d38/f7e zero size 2024-03-21T10:08:54.003 INFO:tasks.workunit.client.1.smithi174.stdout:9/404: creat d2/d24/d2e/d7e/f8f x:0 0 0 2024-03-21T10:08:54.003 INFO:tasks.workunit.client.1.smithi174.stdout:5/411: write d1/d49/f67 [1494282,77086] 0 2024-03-21T10:08:54.004 INFO:tasks.workunit.client.0.smithi153.stdout:2/623: write dd/d15/d18/d2c/f4e [1153598,59427] 0 2024-03-21T10:08:54.004 INFO:tasks.workunit.client.0.smithi153.stdout:2/624: dread - dd/faf zero size 2024-03-21T10:08:54.005 INFO:tasks.workunit.client.1.smithi174.stdout:4/424: chown d4/d19/f23 375121 1 2024-03-21T10:08:54.007 INFO:tasks.workunit.client.1.smithi174.stdout:6/302: dwrite d0/d3/d5/f3c [0,4194304] 0 2024-03-21T10:08:54.008 INFO:tasks.workunit.client.0.smithi153.stdout:9/561: dread d1/d43/f4f [0,4194304] 0 2024-03-21T10:08:54.009 INFO:tasks.workunit.client.0.smithi153.stdout:5/561: write d4/d5/df/d5e/f82 [134410,100237] 0 2024-03-21T10:08:54.010 INFO:tasks.workunit.client.1.smithi174.stdout:2/388: rename d5/d25/d27/d2d/d45/d5b/l9 to d5/d25/d27/d2e/l89 0 2024-03-21T10:08:54.015 INFO:tasks.workunit.client.0.smithi153.stdout:4/581: mkdir d11/db1 0 2024-03-21T10:08:54.015 INFO:tasks.workunit.client.0.smithi153.stdout:7/459: dread d4/d14/d2f/f5e [0,4194304] 0 2024-03-21T10:08:54.016 INFO:tasks.workunit.client.1.smithi174.stdout:4/425: rename d4/d21/d24/d28/d38/f61 to d4/d21/f8c 0 2024-03-21T10:08:54.017 INFO:tasks.workunit.client.0.smithi153.stdout:0/477: link d4/d16/d19/d30/f41 d4/d16/d27/f90 0 2024-03-21T10:08:54.018 INFO:tasks.workunit.client.0.smithi153.stdout:2/625: rename dd/d15/d3c/da7/l9a to dd/d15/d18/d2c/d33/d98/ld2 0 2024-03-21T10:08:54.018 INFO:tasks.workunit.client.0.smithi153.stdout:7/460: write d4/d11/f7f [958201,96590] 0 2024-03-21T10:08:54.019 INFO:tasks.workunit.client.1.smithi174.stdout:9/405: write d2/f84 [1877098,92598] 0 2024-03-21T10:08:54.019 INFO:tasks.workunit.client.1.smithi174.stdout:7/288: dwrite d0/d4/f33 [0,4194304] 0 2024-03-21T10:08:54.021 INFO:tasks.workunit.client.1.smithi174.stdout:2/389: mknod d5/d25/d27/d2d/d45/d5b/d55/c8a 0 2024-03-21T10:08:54.023 INFO:tasks.workunit.client.1.smithi174.stdout:4/426: fsync d4/d21/d24/d28/d79/f86 0 2024-03-21T10:08:54.024 INFO:tasks.workunit.client.0.smithi153.stdout:2/626: readlink dd/d15/d18/d3a/d3d/l42 0 2024-03-21T10:08:54.024 INFO:tasks.workunit.client.0.smithi153.stdout:0/478: creat d4/d4c/d83/f91 x:0 0 0 2024-03-21T10:08:54.024 INFO:tasks.workunit.client.1.smithi174.stdout:1/376: link d4/d7/ff d4/d7/d33/d43/d77/f84 0 2024-03-21T10:08:54.026 INFO:tasks.workunit.client.0.smithi153.stdout:2/627: mknod dd/d15/d18/d2c/d33/d98/cd3 0 2024-03-21T10:08:54.026 INFO:tasks.workunit.client.0.smithi153.stdout:7/461: fdatasync d4/d11/f61 0 2024-03-21T10:08:54.026 INFO:tasks.workunit.client.1.smithi174.stdout:5/412: creat d1/f91 x:0 0 0 2024-03-21T10:08:54.028 INFO:tasks.workunit.client.0.smithi153.stdout:2/628: readlink dd/d15/d3c/da7/d91/dae/d50/d59/lbc 0 2024-03-21T10:08:54.029 INFO:tasks.workunit.client.1.smithi174.stdout:3/373: truncate d0/f2 6077357 0 2024-03-21T10:08:54.030 INFO:tasks.workunit.client.1.smithi174.stdout:1/377: chown d4/d7/d15/d53/d6b 2928 1 2024-03-21T10:08:54.030 INFO:tasks.workunit.client.1.smithi174.stdout:7/289: fdatasync d0/d4/d10/d1b/f25 0 2024-03-21T10:08:54.031 INFO:tasks.workunit.client.0.smithi153.stdout:0/479: unlink d4/d4c/d83/d22/d61/c7e 0 2024-03-21T10:08:54.033 INFO:tasks.workunit.client.1.smithi174.stdout:4/427: write d4/d19/f58 [548553,51743] 0 2024-03-21T10:08:54.033 INFO:tasks.workunit.client.0.smithi153.stdout:7/462: write d4/d2a/f5b [4203487,130362] 0 2024-03-21T10:08:54.033 INFO:tasks.workunit.client.1.smithi174.stdout:5/413: write d1/ff [1619188,77310] 0 2024-03-21T10:08:54.038 INFO:tasks.workunit.client.0.smithi153.stdout:0/480: chown d4/d4c/d83/d22/c4a 218743125 1 2024-03-21T10:08:54.038 INFO:tasks.workunit.client.1.smithi174.stdout:3/374: symlink d0/d12/d14/l85 0 2024-03-21T10:08:54.038 INFO:tasks.workunit.client.1.smithi174.stdout:9/406: creat d2/d11/f90 x:0 0 0 2024-03-21T10:08:54.038 INFO:tasks.workunit.client.1.smithi174.stdout:6/303: dwrite d0/d3/d5/d3b/f55 [0,4194304] 0 2024-03-21T10:08:54.040 INFO:tasks.workunit.client.1.smithi174.stdout:1/378: dread - d4/d7/f21 zero size 2024-03-21T10:08:54.040 INFO:tasks.workunit.client.1.smithi174.stdout:5/414: symlink d1/d16/l92 0 2024-03-21T10:08:54.042 INFO:tasks.workunit.client.1.smithi174.stdout:0/397: mkdir d9/d1c/d76/d87 0 2024-03-21T10:08:54.043 INFO:tasks.workunit.client.1.smithi174.stdout:5/415: stat d1/d49/d4f/l59 0 2024-03-21T10:08:54.044 INFO:tasks.workunit.client.1.smithi174.stdout:6/304: write d0/d3/d5/df/d16/f1a [5929562,80532] 0 2024-03-21T10:08:54.045 INFO:tasks.workunit.client.0.smithi153.stdout:0/481: creat d4/d4c/f92 x:0 0 0 2024-03-21T10:08:54.046 INFO:tasks.workunit.client.1.smithi174.stdout:2/390: getdents d5/d25/d27/d2d/d45/d5b/d63 0 2024-03-21T10:08:54.048 INFO:tasks.workunit.client.1.smithi174.stdout:9/407: mkdir d2/d11/d1d/d2a/d91 0 2024-03-21T10:08:54.049 INFO:tasks.workunit.client.1.smithi174.stdout:6/305: symlink d0/d3/l63 0 2024-03-21T10:08:54.050 INFO:tasks.workunit.client.1.smithi174.stdout:9/408: fsync d2/d54/f5c 0 2024-03-21T10:08:54.050 INFO:tasks.workunit.client.1.smithi174.stdout:0/398: mknod d9/d21/d5f/c88 0 2024-03-21T10:08:54.051 INFO:tasks.workunit.client.0.smithi153.stdout:0/482: mknod d4/d16/d27/c93 0 2024-03-21T10:08:54.052 INFO:tasks.workunit.client.1.smithi174.stdout:5/416: unlink d1/d16/f24 0 2024-03-21T10:08:54.054 INFO:tasks.workunit.client.1.smithi174.stdout:2/391: creat d5/d25/d27/f8b x:0 0 0 2024-03-21T10:08:54.057 INFO:tasks.workunit.client.1.smithi174.stdout:5/417: stat d1/d16/c54 0 2024-03-21T10:08:54.057 INFO:tasks.workunit.client.1.smithi174.stdout:9/409: fdatasync d2/f3c 0 2024-03-21T10:08:54.058 INFO:tasks.workunit.client.0.smithi153.stdout:0/483: fdatasync d4/d16/d27/f90 0 2024-03-21T10:08:54.059 INFO:tasks.workunit.client.1.smithi174.stdout:6/306: fsync d0/d3/d5/df/d16/f34 0 2024-03-21T10:08:54.060 INFO:tasks.workunit.client.0.smithi153.stdout:3/551: sync 2024-03-21T10:08:54.060 INFO:tasks.workunit.client.1.smithi174.stdout:0/399: mkdir d9/d35/d4a/d67/d89 0 2024-03-21T10:08:54.061 INFO:tasks.workunit.client.1.smithi174.stdout:2/392: mknod d5/d25/d27/d2e/d7b/c8c 0 2024-03-21T10:08:54.061 INFO:tasks.workunit.client.1.smithi174.stdout:9/410: chown d2/f3c 458 1 2024-03-21T10:08:54.062 INFO:tasks.workunit.client.0.smithi153.stdout:4/582: dread fd [4194304,4194304] 0 2024-03-21T10:08:54.062 INFO:tasks.workunit.client.0.smithi153.stdout:7/463: dread d4/d14/d2f/f45 [0,4194304] 0 2024-03-21T10:08:54.062 INFO:tasks.workunit.client.0.smithi153.stdout:3/552: chown dd/d1e/d2f/d35/d68/f9a 2905 1 2024-03-21T10:08:54.063 INFO:tasks.workunit.client.1.smithi174.stdout:5/418: mkdir d1/d49/d93 0 2024-03-21T10:08:54.068 INFO:tasks.workunit.client.1.smithi174.stdout:5/419: write d1/d16/d2e/d66/d8b/d85/f90 [647769,105742] 0 2024-03-21T10:08:54.071 INFO:tasks.workunit.client.0.smithi153.stdout:4/583: unlink d11/d18/d7e/d93/f98 0 2024-03-21T10:08:54.071 INFO:tasks.workunit.client.1.smithi174.stdout:0/400: unlink d9/c2c 0 2024-03-21T10:08:54.072 INFO:tasks.workunit.client.1.smithi174.stdout:5/420: truncate d1/d16/f78 4920288 0 2024-03-21T10:08:54.073 INFO:tasks.workunit.client.0.smithi153.stdout:3/553: link dd/d1e/d3d/d94/d73/fa4 dd/d1e/d2f/d35/d68/fc6 0 2024-03-21T10:08:54.075 INFO:tasks.workunit.client.0.smithi153.stdout:0/484: link d4/d16/d19/f85 d4/d4c/d83/d22/f94 0 2024-03-21T10:08:54.077 INFO:tasks.workunit.client.0.smithi153.stdout:4/584: fsync d11/d18/d54/f68 0 2024-03-21T10:08:54.078 INFO:tasks.workunit.client.0.smithi153.stdout:3/554: creat dd/d1e/d2f/d35/d68/daa/fc7 x:0 0 0 2024-03-21T10:08:54.078 INFO:tasks.workunit.client.0.smithi153.stdout:4/585: write d11/fb0 [1008209,125593] 0 2024-03-21T10:08:54.080 INFO:tasks.workunit.client.1.smithi174.stdout:5/421: symlink d1/d16/d2e/d87/l94 0 2024-03-21T10:08:54.080 INFO:tasks.workunit.client.1.smithi174.stdout:9/411: dwrite d2/d11/d1d/f41 [0,4194304] 0 2024-03-21T10:08:54.084 INFO:tasks.workunit.client.1.smithi174.stdout:1/379: dread d4/d7/ff [0,4194304] 0 2024-03-21T10:08:54.085 INFO:tasks.workunit.client.0.smithi153.stdout:3/555: rename dd/d1e/d45/f99 to dd/d1e/d45/fc8 0 2024-03-21T10:08:54.088 INFO:tasks.workunit.client.0.smithi153.stdout:3/556: readlink dd/d1e/d2f/l42 0 2024-03-21T10:08:54.088 INFO:tasks.workunit.client.1.smithi174.stdout:5/422: rename d1/f75 to d1/d16/d1c/d81/f95 0 2024-03-21T10:08:54.090 INFO:tasks.workunit.client.1.smithi174.stdout:1/380: mknod d4/d7/d70/c85 0 2024-03-21T10:08:54.091 INFO:tasks.workunit.client.0.smithi153.stdout:3/557: mkdir dd/d1e/d3d/dc9 0 2024-03-21T10:08:54.092 INFO:tasks.workunit.client.1.smithi174.stdout:5/423: dread - d1/d16/d2e/d66/f7c zero size 2024-03-21T10:08:54.094 INFO:tasks.workunit.client.0.smithi153.stdout:3/558: rmdir dd/d1e/d2f/d35/d68/d75 39 2024-03-21T10:08:54.095 INFO:tasks.workunit.client.1.smithi174.stdout:0/401: dwrite d9/d1c/f7d [0,4194304] 0 2024-03-21T10:08:54.096 INFO:tasks.workunit.client.1.smithi174.stdout:5/424: chown d1/d16/c28 3 1 2024-03-21T10:08:54.096 INFO:tasks.workunit.client.0.smithi153.stdout:3/559: readlink dd/d1e/d3d/l9b 0 2024-03-21T10:08:54.097 INFO:tasks.workunit.client.1.smithi174.stdout:9/412: dwrite d2/d54/f5c [0,4194304] 0 2024-03-21T10:08:54.098 INFO:tasks.workunit.client.0.smithi153.stdout:3/560: fdatasync dd/f2c 0 2024-03-21T10:08:54.101 INFO:tasks.workunit.client.0.smithi153.stdout:3/561: mkdir dd/d1e/d3d/d53/dca 0 2024-03-21T10:08:54.102 INFO:tasks.workunit.client.0.smithi153.stdout:3/562: fdatasync dd/d41/f71 0 2024-03-21T10:08:54.104 INFO:tasks.workunit.client.0.smithi153.stdout:3/563: fdatasync dd/d1e/d2f/d35/daf/fb4 0 2024-03-21T10:08:54.105 INFO:tasks.workunit.client.1.smithi174.stdout:0/402: mkdir d9/d35/d4a/d8a 0 2024-03-21T10:08:54.108 INFO:tasks.workunit.client.1.smithi174.stdout:9/413: link d2/d24/d2e/f67 d2/d54/f92 0 2024-03-21T10:08:54.108 INFO:tasks.workunit.client.0.smithi153.stdout:3/564: write dd/d1e/d2f/d35/d68/daa/fac [3531571,53051] 0 2024-03-21T10:08:54.111 INFO:tasks.workunit.client.0.smithi153.stdout:3/565: symlink dd/d1e/d45/d56/lcb 0 2024-03-21T10:08:54.115 INFO:tasks.workunit.client.1.smithi174.stdout:1/381: dwrite d4/d7/d15/d4f/f82 [0,4194304] 0 2024-03-21T10:08:54.119 INFO:tasks.workunit.client.1.smithi174.stdout:5/425: dwrite d1/d16/d1c/f6d [0,4194304] 0 2024-03-21T10:08:54.122 INFO:tasks.workunit.client.1.smithi174.stdout:1/382: mkdir d4/d7/d32/d86 0 2024-03-21T10:08:54.125 INFO:tasks.workunit.client.1.smithi174.stdout:0/403: link d9/d21/d31/l3a d9/d35/l8b 0 2024-03-21T10:08:54.126 INFO:tasks.workunit.client.1.smithi174.stdout:5/426: read - d1/d16/d2e/d66/f89 zero size 2024-03-21T10:08:54.133 INFO:tasks.workunit.client.1.smithi174.stdout:1/383: mkdir d4/d7/d33/d43/d77/d87 0 2024-03-21T10:08:54.139 INFO:tasks.workunit.client.1.smithi174.stdout:1/384: mknod d4/d7/d33/d43/c88 0 2024-03-21T10:08:54.147 INFO:tasks.workunit.client.1.smithi174.stdout:5/427: dwrite d1/d16/d2e/d66/f7c [0,4194304] 0 2024-03-21T10:08:54.161 INFO:tasks.workunit.client.1.smithi174.stdout:9/414: dread d2/d11/f4a [0,4194304] 0 2024-03-21T10:08:54.163 INFO:tasks.workunit.client.1.smithi174.stdout:5/428: dread d1/d16/d2e/d66/f7c [0,4194304] 0 2024-03-21T10:08:54.164 INFO:tasks.workunit.client.0.smithi153.stdout:5/562: read d4/d5/df/d5e/f82 [54200,40728] 0 2024-03-21T10:08:54.166 INFO:tasks.workunit.client.1.smithi174.stdout:9/415: mknod d2/d24/d2e/d63/c93 0 2024-03-21T10:08:54.167 INFO:tasks.workunit.client.0.smithi153.stdout:5/563: fsync d4/d5/df/d5e/d69/f7e 0 2024-03-21T10:08:54.176 INFO:tasks.workunit.client.1.smithi174.stdout:9/416: rename d2/l76 to d2/d11/l94 0 2024-03-21T10:08:54.181 INFO:tasks.workunit.client.0.smithi153.stdout:1/445: dwrite d6/d32/d3e/d4f/f58 [0,4194304] 0 2024-03-21T10:08:54.182 INFO:tasks.workunit.client.1.smithi174.stdout:5/429: dwrite d1/d5c/d73/f7f [0,4194304] 0 2024-03-21T10:08:54.186 INFO:tasks.workunit.client.1.smithi174.stdout:9/417: creat d2/d24/d2e/d7e/f95 x:0 0 0 2024-03-21T10:08:54.188 INFO:tasks.workunit.client.0.smithi153.stdout:1/446: write d6/d3a/d66/f68 [1228837,22819] 0 2024-03-21T10:08:54.189 INFO:tasks.workunit.client.0.smithi153.stdout:6/731: dwrite d1f/d2b/d33/d40/d51/d98/fcd [0,4194304] 0 2024-03-21T10:08:54.190 INFO:tasks.workunit.client.1.smithi174.stdout:7/290: truncate d0/d4/f33 3052692 0 2024-03-21T10:08:54.197 INFO:tasks.workunit.client.0.smithi153.stdout:9/562: dwrite d1/d43/d44/d46/d9e/d24/d41/f47 [0,4194304] 0 2024-03-21T10:08:54.204 INFO:tasks.workunit.client.0.smithi153.stdout:2/629: dwrite dd/d15/d5e/fc0 [0,4194304] 0 2024-03-21T10:08:54.214 INFO:tasks.workunit.client.0.smithi153.stdout:0/485: truncate d4/d16/d27/f76 4092225 0 2024-03-21T10:08:54.218 INFO:tasks.workunit.client.1.smithi174.stdout:9/418: dwrite d2/d54/f92 [0,4194304] 0 2024-03-21T10:08:54.218 INFO:tasks.workunit.client.1.smithi174.stdout:2/393: truncate d5/d25/d27/d2d/d45/d5b/d63/f48 103540 0 2024-03-21T10:08:54.222 INFO:tasks.workunit.client.1.smithi174.stdout:6/307: dwrite d0/d3/d5/fe [0,4194304] 0 2024-03-21T10:08:54.224 INFO:tasks.workunit.client.1.smithi174.stdout:7/291: dwrite d0/f26 [0,4194304] 0 2024-03-21T10:08:54.225 INFO:tasks.workunit.client.1.smithi174.stdout:2/394: unlink d5/d25/d27/f6a 0 2024-03-21T10:08:54.230 INFO:tasks.workunit.client.0.smithi153.stdout:4/586: dwrite d11/d2d/d3d/f99 [0,4194304] 0 2024-03-21T10:08:54.235 INFO:tasks.workunit.client.0.smithi153.stdout:1/447: dread d6/d3a/d3c/f64 [4194304,4194304] 0 2024-03-21T10:08:54.236 INFO:tasks.workunit.client.1.smithi174.stdout:2/395: stat d5/d25/d27/d2d/d45/d5b/d2f/d4b/d76 0 2024-03-21T10:08:54.236 INFO:tasks.workunit.client.1.smithi174.stdout:7/292: write d0/d4/fd [669637,27163] 0 2024-03-21T10:08:54.239 INFO:tasks.workunit.client.0.smithi153.stdout:3/566: rename dd/d1e/d45/d56 to dd/d1e/d2f/d35/daf/dcc 0 2024-03-21T10:08:54.241 INFO:tasks.workunit.client.1.smithi174.stdout:9/419: link d2/f3c d2/d11/d1d/f96 0 2024-03-21T10:08:54.242 INFO:tasks.workunit.client.0.smithi153.stdout:3/567: truncate dd/d1e/f30 4522918 0 2024-03-21T10:08:54.246 INFO:tasks.workunit.client.1.smithi174.stdout:0/404: write d9/f33 [986894,27452] 0 2024-03-21T10:08:54.247 INFO:tasks.workunit.client.0.smithi153.stdout:3/568: fdatasync dd/d1e/d2f/d35/d68/daa/fc7 0 2024-03-21T10:08:54.249 INFO:tasks.workunit.client.1.smithi174.stdout:2/396: truncate d5/f38 7057493 0 2024-03-21T10:08:54.250 INFO:tasks.workunit.client.0.smithi153.stdout:9/563: creat d1/d3b/fc2 x:0 0 0 2024-03-21T10:08:54.251 INFO:tasks.workunit.client.1.smithi174.stdout:1/385: write d4/d7/d15/d4f/f69 [907700,109833] 0 2024-03-21T10:08:54.254 INFO:tasks.workunit.client.1.smithi174.stdout:9/420: rename d2/d11/d1d/d4b/c4d to d2/d11/d1d/d4b/c97 0 2024-03-21T10:08:54.255 INFO:tasks.workunit.client.1.smithi174.stdout:7/293: getdents d0/d53 0 2024-03-21T10:08:54.255 INFO:tasks.workunit.client.0.smithi153.stdout:4/587: write d11/d18/f55 [1855336,115718] 0 2024-03-21T10:08:54.260 INFO:tasks.workunit.client.0.smithi153.stdout:9/564: fsync d1/d43/d44/d46/f26 0 2024-03-21T10:08:54.263 INFO:tasks.workunit.client.0.smithi153.stdout:9/565: fsync d1/d4e/f98 0 2024-03-21T10:08:54.264 INFO:tasks.workunit.client.1.smithi174.stdout:1/386: mkdir d4/d7/d68/d89 0 2024-03-21T10:08:54.266 INFO:tasks.workunit.client.1.smithi174.stdout:9/421: write d2/d11/f75 [134576,40137] 0 2024-03-21T10:08:54.267 INFO:tasks.workunit.client.0.smithi153.stdout:9/566: mkdir d1/d43/d44/dc3 0 2024-03-21T10:08:54.269 INFO:tasks.workunit.client.1.smithi174.stdout:7/294: dread d0/d4/f23 [0,4194304] 0 2024-03-21T10:08:54.271 INFO:tasks.workunit.client.1.smithi174.stdout:5/430: dread d1/d16/f5f [0,4194304] 0 2024-03-21T10:08:54.272 INFO:tasks.workunit.client.0.smithi153.stdout:2/630: dwrite dd/d15/d18/d3a/f3e [0,4194304] 0 2024-03-21T10:08:54.275 INFO:tasks.workunit.client.1.smithi174.stdout:7/295: write d0/d4/d10/d1b/f25 [714943,52563] 0 2024-03-21T10:08:54.275 INFO:tasks.workunit.client.1.smithi174.stdout:1/387: creat d4/d7/d32/d86/f8a x:0 0 0 2024-03-21T10:08:54.276 INFO:tasks.workunit.client.1.smithi174.stdout:9/422: symlink d2/d24/d2e/d7e/l98 0 2024-03-21T10:08:54.277 INFO:tasks.workunit.client.0.smithi153.stdout:2/631: chown dd/d15/d3c/da7/d91/dae/d50/d59 0 1 2024-03-21T10:08:54.280 INFO:tasks.workunit.client.1.smithi174.stdout:5/431: unlink d1/d16/f78 0 2024-03-21T10:08:54.280 INFO:tasks.workunit.client.1.smithi174.stdout:9/423: chown d2/d54/f5b 6 1 2024-03-21T10:08:54.281 INFO:tasks.workunit.client.1.smithi174.stdout:7/296: write d0/d4/d10/d1b/f2e [159588,84064] 0 2024-03-21T10:08:54.285 INFO:tasks.workunit.client.1.smithi174.stdout:7/297: read d0/d4/d10/d1b/f28 [579436,92435] 0 2024-03-21T10:08:54.286 INFO:tasks.workunit.client.0.smithi153.stdout:2/632: mknod dd/d15/d18/cd4 0 2024-03-21T10:08:54.290 INFO:tasks.workunit.client.1.smithi174.stdout:5/432: mkdir d1/d5c/d96 0 2024-03-21T10:08:54.292 INFO:tasks.workunit.client.1.smithi174.stdout:7/298: mkdir d0/d4/d10/d60 0 2024-03-21T10:08:54.294 INFO:tasks.workunit.client.0.smithi153.stdout:2/633: creat dd/d15/d18/d2c/d4b/d56/d63/d8d/d9b/dc3/fd5 x:0 0 0 2024-03-21T10:08:54.297 INFO:tasks.workunit.client.1.smithi174.stdout:5/433: stat d1/d16/d2e/d66/d8b/c8c 0 2024-03-21T10:08:54.298 INFO:tasks.workunit.client.0.smithi153.stdout:2/634: stat dd/d15/d18/d2c/d33/d98/cd3 0 2024-03-21T10:08:54.300 INFO:tasks.workunit.client.1.smithi174.stdout:9/424: creat d2/d24/f99 x:0 0 0 2024-03-21T10:08:54.301 INFO:tasks.workunit.client.1.smithi174.stdout:1/388: dwrite d4/d5/f72 [0,4194304] 0 2024-03-21T10:08:54.303 INFO:tasks.workunit.client.0.smithi153.stdout:8/566: sync 2024-03-21T10:08:54.303 INFO:tasks.workunit.client.1.smithi174.stdout:9/425: chown d2/d11/f36 10150 1 2024-03-21T10:08:54.304 INFO:tasks.workunit.client.1.smithi174.stdout:8/315: sync 2024-03-21T10:08:54.306 INFO:tasks.workunit.client.1.smithi174.stdout:1/389: write d4/d7/d15/f2f [228,40889] 0 2024-03-21T10:08:54.306 INFO:tasks.workunit.client.0.smithi153.stdout:6/732: dread d1f/f39 [0,4194304] 0 2024-03-21T10:08:54.309 INFO:tasks.workunit.client.1.smithi174.stdout:5/434: creat d1/d2f/d37/f97 x:0 0 0 2024-03-21T10:08:54.310 INFO:tasks.workunit.client.1.smithi174.stdout:1/390: write d4/d7/d15/d4f/f83 [109285,129356] 0 2024-03-21T10:08:54.315 INFO:tasks.workunit.client.1.smithi174.stdout:8/316: fdatasync d9/f1a 0 2024-03-21T10:08:54.316 INFO:tasks.workunit.client.1.smithi174.stdout:8/317: chown d9/d2a 4767930 1 2024-03-21T10:08:54.317 INFO:tasks.workunit.client.1.smithi174.stdout:5/435: symlink d1/d16/d1c/d81/l98 0 2024-03-21T10:08:54.321 INFO:tasks.workunit.client.1.smithi174.stdout:5/436: chown d1/d16/d2e/d66/d8b/c6a 276246213 1 2024-03-21T10:08:54.325 INFO:tasks.workunit.client.1.smithi174.stdout:1/391: mkdir d4/d7/d68/d7b/d8b 0 2024-03-21T10:08:54.325 INFO:tasks.workunit.client.0.smithi153.stdout:2/635: dwrite dd/faf [0,4194304] 0 2024-03-21T10:08:54.328 INFO:tasks.workunit.client.0.smithi153.stdout:2/636: stat dd/d15/d5e/db3/fb9 0 2024-03-21T10:08:54.328 INFO:tasks.workunit.client.1.smithi174.stdout:1/392: fdatasync d4/d7/d33/d43/d51/f64 0 2024-03-21T10:08:54.329 INFO:tasks.workunit.client.1.smithi174.stdout:1/393: chown d4/d5 719995 1 2024-03-21T10:08:54.331 INFO:tasks.workunit.client.0.smithi153.stdout:8/567: fdatasync d5/d9/d43/dd/d20/d26/d24/d52/d55/f89 0 2024-03-21T10:08:54.331 INFO:tasks.workunit.client.0.smithi153.stdout:2/637: chown dd/d15/d18/d2c/d33/d98/lbf 14 1 2024-03-21T10:08:54.332 INFO:tasks.workunit.client.0.smithi153.stdout:3/569: dread dd/d1e/d2f/d35/f5a [0,4194304] 0 2024-03-21T10:08:54.334 INFO:tasks.workunit.client.1.smithi174.stdout:5/437: mkdir d1/d16/d2e/d8d/d99 0 2024-03-21T10:08:54.334 INFO:tasks.workunit.client.1.smithi174.stdout:1/394: write d4/f19 [417894,1846] 0 2024-03-21T10:08:54.336 INFO:tasks.workunit.client.1.smithi174.stdout:1/395: chown d4/l27 31722418 1 2024-03-21T10:08:54.337 INFO:tasks.workunit.client.1.smithi174.stdout:8/318: link d9/d2a/f51 d9/d2a/f65 0 2024-03-21T10:08:54.339 INFO:tasks.workunit.client.0.smithi153.stdout:8/568: symlink d5/d9/d43/dd/d2d/d45/d88/d85/lbc 0 2024-03-21T10:08:54.341 INFO:tasks.workunit.client.1.smithi174.stdout:1/396: chown d4/d5/d6c/f73 156311165 1 2024-03-21T10:08:54.341 INFO:tasks.workunit.client.1.smithi174.stdout:9/426: dwrite d2/f1e [0,4194304] 0 2024-03-21T10:08:54.341 INFO:tasks.workunit.client.0.smithi153.stdout:8/569: readlink d5/d9/d43/dd/d20/d26/d8b/l96 0 2024-03-21T10:08:54.347 INFO:tasks.workunit.client.0.smithi153.stdout:4/588: truncate d11/d18/f1d 2659557 0 2024-03-21T10:08:54.347 INFO:tasks.workunit.client.1.smithi174.stdout:5/438: rename d1/d5c/f83 to d1/d49/d4f/d51/f9a 0 2024-03-21T10:08:54.348 INFO:tasks.workunit.client.0.smithi153.stdout:3/570: link dd/c50 dd/d1e/d3d/d53/dae/ccd 0 2024-03-21T10:08:54.350 INFO:tasks.workunit.client.0.smithi153.stdout:6/733: write d1f/d2b/d33/d40/d5f/f82 [2722428,34048] 0 2024-03-21T10:08:54.353 INFO:tasks.workunit.client.0.smithi153.stdout:8/570: creat d5/d9/d43/dd/d2d/d45/fbd x:0 0 0 2024-03-21T10:08:54.354 INFO:tasks.workunit.client.1.smithi174.stdout:5/439: mkdir d1/d2f/d37/d9b 0 2024-03-21T10:08:54.356 INFO:tasks.workunit.client.0.smithi153.stdout:3/571: stat dd/d1e/d2f/d35/d68/fb6 0 2024-03-21T10:08:54.357 INFO:tasks.workunit.client.1.smithi174.stdout:9/427: creat d2/d11/d1d/d8b/f9a x:0 0 0 2024-03-21T10:08:54.359 INFO:tasks.workunit.client.0.smithi153.stdout:4/589: link d11/d18/l47 d11/d18/d7e/d93/lb2 0 2024-03-21T10:08:54.360 INFO:tasks.workunit.client.1.smithi174.stdout:8/319: dread d9/f1a [0,4194304] 0 2024-03-21T10:08:54.360 INFO:tasks.workunit.client.0.smithi153.stdout:1/448: dwrite d6/d13/f5e [0,4194304] 0 2024-03-21T10:08:54.362 INFO:tasks.workunit.client.1.smithi174.stdout:5/440: mkdir d1/d49/d93/d9c 0 2024-03-21T10:08:54.363 INFO:tasks.workunit.client.1.smithi174.stdout:9/428: rmdir d2/d11/d60 39 2024-03-21T10:08:54.364 INFO:tasks.workunit.client.1.smithi174.stdout:0/405: write d9/d35/d4f/d56/f66 [270438,101449] 0 2024-03-21T10:08:54.366 INFO:tasks.workunit.client.1.smithi174.stdout:5/441: chown d1/c15 69079395 1 2024-03-21T10:08:54.368 INFO:tasks.workunit.client.1.smithi174.stdout:8/320: unlink d9/d2a/f65 0 2024-03-21T10:08:54.368 INFO:tasks.workunit.client.0.smithi153.stdout:2/638: dwrite dd/d15/d18/d3a/d8c/fb2 [0,4194304] 0 2024-03-21T10:08:54.372 INFO:tasks.workunit.client.1.smithi174.stdout:5/442: write d1/d16/d2e/d66/d8b/d85/f5b [1830068,53421] 0 2024-03-21T10:08:54.373 INFO:tasks.workunit.client.1.smithi174.stdout:9/429: link d2/d11/f75 d2/d24/d2e/f9b 0 2024-03-21T10:08:54.375 INFO:tasks.workunit.client.0.smithi153.stdout:8/571: rmdir d5/d9/d43/dd/d20/d26/d24 39 2024-03-21T10:08:54.377 INFO:tasks.workunit.client.1.smithi174.stdout:0/406: mknod d9/d21/d75/c8c 0 2024-03-21T10:08:54.380 INFO:tasks.workunit.client.0.smithi153.stdout:8/572: fdatasync d5/d9/d43/dd/d2d/d45/d88/d85/fbb 0 2024-03-21T10:08:54.384 INFO:tasks.workunit.client.1.smithi174.stdout:0/407: chown d9/c16 3551 1 2024-03-21T10:08:54.385 INFO:tasks.workunit.client.0.smithi153.stdout:2/639: dread dd/faf [0,4194304] 0 2024-03-21T10:08:54.387 INFO:tasks.workunit.client.1.smithi174.stdout:8/321: creat d9/d53/d5b/f66 x:0 0 0 2024-03-21T10:08:54.387 INFO:tasks.workunit.client.0.smithi153.stdout:1/449: creat d6/d13/d75/f83 x:0 0 0 2024-03-21T10:08:54.388 INFO:tasks.workunit.client.1.smithi174.stdout:9/430: write d2/d11/d60/f77 [1631700,13265] 0 2024-03-21T10:08:54.389 INFO:tasks.workunit.client.0.smithi153.stdout:6/734: write d1f/d93/dd8/f74 [374164,110187] 0 2024-03-21T10:08:54.392 INFO:tasks.workunit.client.0.smithi153.stdout:3/572: truncate dd/d1e/d3d/d94/d73/f7c 376099 0 2024-03-21T10:08:54.392 INFO:tasks.workunit.client.1.smithi174.stdout:0/408: creat d9/d1c/d76/d87/f8d x:0 0 0 2024-03-21T10:08:54.393 INFO:tasks.workunit.client.1.smithi174.stdout:8/322: symlink d9/d17/d19/d31/l67 0 2024-03-21T10:08:54.394 INFO:tasks.workunit.client.0.smithi153.stdout:3/573: fsync dd/d1e/d3d/d53/fba 0 2024-03-21T10:08:54.394 INFO:tasks.workunit.client.0.smithi153.stdout:8/573: creat d5/d9/d43/dd/d2d/d45/d88/fbe x:0 0 0 2024-03-21T10:08:54.397 INFO:tasks.workunit.client.1.smithi174.stdout:0/409: dread - d9/d21/d31/d3c/f83 zero size 2024-03-21T10:08:54.397 INFO:tasks.workunit.client.1.smithi174.stdout:2/397: write d5/d25/d27/d2d/d45/d5b/d55/f83 [5394235,58607] 0 2024-03-21T10:08:54.402 INFO:tasks.workunit.client.1.smithi174.stdout:3/375: sync 2024-03-21T10:08:54.402 INFO:tasks.workunit.client.1.smithi174.stdout:4/428: sync 2024-03-21T10:08:54.404 INFO:tasks.workunit.client.0.smithi153.stdout:3/574: read dd/f2c [4315589,82723] 0 2024-03-21T10:08:54.405 INFO:tasks.workunit.client.0.smithi153.stdout:4/590: getdents d11/d71/d89 0 2024-03-21T10:08:54.405 INFO:tasks.workunit.client.0.smithi153.stdout:1/450: mknod d6/d3a/d66/d80/c84 0 2024-03-21T10:08:54.406 INFO:tasks.workunit.client.1.smithi174.stdout:5/443: dwrite d1/d16/d2e/d66/f89 [0,4194304] 0 2024-03-21T10:08:54.408 INFO:tasks.workunit.client.0.smithi153.stdout:4/591: write d11/d2d/d3d/f90 [983978,9564] 0 2024-03-21T10:08:54.409 INFO:tasks.workunit.client.1.smithi174.stdout:2/398: read d5/d25/d27/f4e [2270779,73203] 0 2024-03-21T10:08:54.409 INFO:tasks.workunit.client.1.smithi174.stdout:0/410: write d9/d21/d31/d3c/f49 [831422,1597] 0 2024-03-21T10:08:54.410 INFO:tasks.workunit.client.0.smithi153.stdout:2/640: getdents dd/d15/d18/d3a/d3d 0 2024-03-21T10:08:54.414 INFO:tasks.workunit.client.1.smithi174.stdout:1/397: dwrite d4/d7/d28/f30 [0,4194304] 0 2024-03-21T10:08:54.415 INFO:tasks.workunit.client.1.smithi174.stdout:4/429: creat d4/d21/d31/d3a/f8d x:0 0 0 2024-03-21T10:08:54.417 INFO:tasks.workunit.client.0.smithi153.stdout:1/451: creat d6/d3a/d66/f85 x:0 0 0 2024-03-21T10:08:54.422 INFO:tasks.workunit.client.1.smithi174.stdout:5/444: mknod d1/d16/c9d 0 2024-03-21T10:08:54.422 INFO:tasks.workunit.client.0.smithi153.stdout:8/574: dwrite d5/d9/d43/d7b/daa/fb8 [0,4194304] 0 2024-03-21T10:08:54.423 INFO:tasks.workunit.client.0.smithi153.stdout:2/641: creat dd/d15/d5e/db3/fd6 x:0 0 0 2024-03-21T10:08:54.426 INFO:tasks.workunit.client.0.smithi153.stdout:4/592: mknod d11/d9d/cb3 0 2024-03-21T10:08:54.427 INFO:tasks.workunit.client.1.smithi174.stdout:5/445: rmdir d1/d16/d1c/d81 39 2024-03-21T10:08:54.427 INFO:tasks.workunit.client.1.smithi174.stdout:2/399: creat d5/d25/d27/d2d/d45/d5b/d55/d75/f8d x:0 0 0 2024-03-21T10:08:54.430 INFO:tasks.workunit.client.1.smithi174.stdout:4/430: unlink d4/d21/d31/d3a/d77/f89 0 2024-03-21T10:08:54.433 INFO:tasks.workunit.client.1.smithi174.stdout:5/446: creat d1/d2f/d37/f9e x:0 0 0 2024-03-21T10:08:54.434 INFO:tasks.workunit.client.1.smithi174.stdout:4/431: unlink d4/l1a 0 2024-03-21T10:08:54.437 INFO:tasks.workunit.client.1.smithi174.stdout:5/447: symlink d1/d5c/l9f 0 2024-03-21T10:08:54.438 INFO:tasks.workunit.client.1.smithi174.stdout:4/432: dread - d4/d21/d24/d28/d79/f86 zero size 2024-03-21T10:08:54.440 INFO:tasks.workunit.client.0.smithi153.stdout:6/735: dwrite d1f/d93/dd8/f4c [0,4194304] 0 2024-03-21T10:08:54.441 INFO:tasks.workunit.client.0.smithi153.stdout:4/593: dread d11/d18/f49 [0,4194304] 0 2024-03-21T10:08:54.441 INFO:tasks.workunit.client.1.smithi174.stdout:4/433: write d4/f3e [3554999,93992] 0 2024-03-21T10:08:54.442 INFO:tasks.workunit.client.0.smithi153.stdout:3/575: dwrite dd/f2b [0,4194304] 0 2024-03-21T10:08:54.443 INFO:tasks.workunit.client.1.smithi174.stdout:8/323: dread d9/d17/f21 [0,4194304] 0 2024-03-21T10:08:54.445 INFO:tasks.workunit.client.1.smithi174.stdout:0/411: dwrite d9/f19 [0,4194304] 0 2024-03-21T10:08:54.445 INFO:tasks.workunit.client.1.smithi174.stdout:5/448: write d1/d16/d1c/d81/f95 [1143018,24217] 0 2024-03-21T10:08:54.446 INFO:tasks.workunit.client.1.smithi174.stdout:9/431: dread d2/d11/d1d/f80 [0,4194304] 0 2024-03-21T10:08:54.450 INFO:tasks.workunit.client.1.smithi174.stdout:4/434: creat d4/d21/d31/d3a/f8e x:0 0 0 2024-03-21T10:08:54.451 INFO:tasks.workunit.client.0.smithi153.stdout:8/575: rename d5/d9/d43/dd/d20/c53 to d5/d9/d43/dd/cbf 0 2024-03-21T10:08:54.451 INFO:tasks.workunit.client.0.smithi153.stdout:3/576: truncate dd/d1e/d3d/d53/fa3 478045 0 2024-03-21T10:08:54.451 INFO:tasks.workunit.client.0.smithi153.stdout:2/642: mkdir dd/d15/d18/d2c/dd7 0 2024-03-21T10:08:54.451 INFO:tasks.workunit.client.1.smithi174.stdout:0/412: write d9/d21/d31/d3c/d72/f85 [747593,30511] 0 2024-03-21T10:08:54.453 INFO:tasks.workunit.client.1.smithi174.stdout:0/413: readlink d9/d1c/l30 0 2024-03-21T10:08:54.453 INFO:tasks.workunit.client.1.smithi174.stdout:9/432: creat d2/d54/f9c x:0 0 0 2024-03-21T10:08:54.454 INFO:tasks.workunit.client.0.smithi153.stdout:2/643: chown dd/d15/d3c/da7/d91/dae/d50/d58/d79/lcd 4 1 2024-03-21T10:08:54.462 INFO:tasks.workunit.client.0.smithi153.stdout:6/736: mknod d1f/d2b/d33/d40/d5f/c108 0 2024-03-21T10:08:54.463 INFO:tasks.workunit.client.1.smithi174.stdout:3/376: truncate d0/d12/d14/d1a/f3d 3255600 0 2024-03-21T10:08:54.466 INFO:tasks.workunit.client.0.smithi153.stdout:8/576: creat d5/d9/d43/dd/d2d/d45/fc0 x:0 0 0 2024-03-21T10:08:54.467 INFO:tasks.workunit.client.0.smithi153.stdout:1/452: write d6/f1c [2422023,129612] 0 2024-03-21T10:08:54.471 INFO:tasks.workunit.client.1.smithi174.stdout:3/377: rmdir d0/d12/d14/d1a/d27/d3c 39 2024-03-21T10:08:54.474 INFO:tasks.workunit.client.0.smithi153.stdout:2/644: dread dd/d15/d18/d2c/d4b/d56/f7b [0,4194304] 0 2024-03-21T10:08:54.475 INFO:tasks.workunit.client.1.smithi174.stdout:4/435: dwrite d4/d21/d24/d28/d38/f88 [0,4194304] 0 2024-03-21T10:08:54.477 INFO:tasks.workunit.client.0.smithi153.stdout:6/737: rmdir d1f/da0 39 2024-03-21T10:08:54.478 INFO:tasks.workunit.client.0.smithi153.stdout:3/577: dwrite dd/d1e/d2f/d35/daf/dcc/f79 [0,4194304] 0 2024-03-21T10:08:54.480 INFO:tasks.workunit.client.0.smithi153.stdout:2/645: chown dd/d15/d3c/da7/d91/dae/d50/d58/d79 15189424 1 2024-03-21T10:08:54.482 INFO:tasks.workunit.client.0.smithi153.stdout:1/453: chown d6/d13/c1e 28904913 1 2024-03-21T10:08:54.482 INFO:tasks.workunit.client.0.smithi153.stdout:3/578: readlink dd/d1e/d3d/d94/l83 0 2024-03-21T10:08:54.482 INFO:tasks.workunit.client.0.smithi153.stdout:6/738: chown d1f/d2b/d33/c73 340811 1 2024-03-21T10:08:54.490 INFO:tasks.workunit.client.0.smithi153.stdout:7/464: sync 2024-03-21T10:08:54.491 INFO:tasks.workunit.client.1.smithi174.stdout:3/378: dwrite d0/f13 [0,4194304] 0 2024-03-21T10:08:54.493 INFO:tasks.workunit.client.0.smithi153.stdout:4/594: link d11/d18/l47 d11/d18/d6d/lb4 0 2024-03-21T10:08:54.495 INFO:tasks.workunit.client.0.smithi153.stdout:8/577: unlink d5/d9/d43/dd/d20/d26/d24/d52/d55/f5f 0 2024-03-21T10:08:54.501 INFO:tasks.workunit.client.0.smithi153.stdout:8/578: write d5/d9/d43/d7b/f98 [788950,27974] 0 2024-03-21T10:08:54.502 INFO:tasks.workunit.client.0.smithi153.stdout:7/465: fsync d4/d1b/d20/f2c 0 2024-03-21T10:08:54.506 INFO:tasks.workunit.client.1.smithi174.stdout:8/324: dread f5 [0,4194304] 0 2024-03-21T10:08:54.508 INFO:tasks.workunit.client.0.smithi153.stdout:2/646: mknod dd/d15/d18/d2c/d33/d98/dc7/cd8 0 2024-03-21T10:08:54.509 INFO:tasks.workunit.client.0.smithi153.stdout:3/579: mknod dd/d1e/d3d/dc9/cce 0 2024-03-21T10:08:54.511 INFO:tasks.workunit.client.1.smithi174.stdout:8/325: rename d9/d17/d19/d27/l45 to d9/d53/l68 0 2024-03-21T10:08:54.512 INFO:tasks.workunit.client.0.smithi153.stdout:7/466: fdatasync d4/d14/d2f/f5e 0 2024-03-21T10:08:54.512 INFO:tasks.workunit.client.0.smithi153.stdout:1/454: dwrite d6/d3a/d66/f85 [0,4194304] 0 2024-03-21T10:08:54.512 INFO:tasks.workunit.client.1.smithi174.stdout:3/379: dread d0/d12/d14/d1a/d27/f2c [0,4194304] 0 2024-03-21T10:08:54.514 INFO:tasks.workunit.client.1.smithi174.stdout:8/326: creat d9/d17/f69 x:0 0 0 2024-03-21T10:08:54.514 INFO:tasks.workunit.client.0.smithi153.stdout:4/595: truncate d11/f76 746497 0 2024-03-21T10:08:54.517 INFO:tasks.workunit.client.0.smithi153.stdout:7/467: write d4/d14/d2f/d78/f3a [731799,489] 0 2024-03-21T10:08:54.517 INFO:tasks.workunit.client.0.smithi153.stdout:8/579: creat d5/d9/d43/dd/d2d/d45/fc1 x:0 0 0 2024-03-21T10:08:54.518 INFO:tasks.workunit.client.1.smithi174.stdout:3/380: mknod d0/d12/d14/d1a/c86 0 2024-03-21T10:08:54.518 INFO:tasks.workunit.client.1.smithi174.stdout:8/327: creat d9/d17/f6a x:0 0 0 2024-03-21T10:08:54.523 INFO:tasks.workunit.client.0.smithi153.stdout:3/580: fsync dd/d1e/d2f/f90 0 2024-03-21T10:08:54.523 INFO:tasks.workunit.client.0.smithi153.stdout:2/647: fdatasync dd/d15/d18/d2c/d4b/d56/f60 0 2024-03-21T10:08:54.525 INFO:tasks.workunit.client.1.smithi174.stdout:0/414: dread d9/d21/d31/f34 [0,4194304] 0 2024-03-21T10:08:54.525 INFO:tasks.workunit.client.1.smithi174.stdout:8/328: getdents d9/d17/d19 0 2024-03-21T10:08:54.529 INFO:tasks.workunit.client.1.smithi174.stdout:0/415: unlink d9/d1c/l51 0 2024-03-21T10:08:54.532 INFO:tasks.workunit.client.1.smithi174.stdout:0/416: write d9/f65 [311595,1489] 0 2024-03-21T10:08:54.532 INFO:tasks.workunit.client.1.smithi174.stdout:8/329: mknod d9/d2a/d39/d58/c6b 0 2024-03-21T10:08:54.532 INFO:tasks.workunit.client.0.smithi153.stdout:6/739: link d1f/l6a d1f/d93/dd8/d59/l109 0 2024-03-21T10:08:54.533 INFO:tasks.workunit.client.1.smithi174.stdout:3/381: link d0/d12/d14/d1a/d27/c50 d0/d12/d14/d1a/d27/d7f/c87 0 2024-03-21T10:08:54.533 INFO:tasks.workunit.client.1.smithi174.stdout:6/308: sync 2024-03-21T10:08:54.533 INFO:tasks.workunit.client.1.smithi174.stdout:7/299: sync 2024-03-21T10:08:54.534 INFO:tasks.workunit.client.0.smithi153.stdout:3/581: unlink dd/d1e/d2f/la7 0 2024-03-21T10:08:54.535 INFO:tasks.workunit.client.1.smithi174.stdout:7/300: readlink d0/l41 0 2024-03-21T10:08:54.535 INFO:tasks.workunit.client.1.smithi174.stdout:8/330: write f1 [2204095,53021] 0 2024-03-21T10:08:54.535 INFO:tasks.workunit.client.0.smithi153.stdout:3/582: stat dd/d1e/f30 0 2024-03-21T10:08:54.536 INFO:tasks.workunit.client.0.smithi153.stdout:4/596: creat d11/d2d/d3d/da1/fb5 x:0 0 0 2024-03-21T10:08:54.538 INFO:tasks.workunit.client.0.smithi153.stdout:7/468: chown d4/d2a/d3b/d60/c6b 9356265 1 2024-03-21T10:08:54.539 INFO:tasks.workunit.client.1.smithi174.stdout:6/309: fdatasync d0/d3/d5/d3b/f61 0 2024-03-21T10:08:54.545 INFO:tasks.workunit.client.0.smithi153.stdout:6/740: rename d1f/d2b/d33 to d1f/d2b/d10a 0 2024-03-21T10:08:54.549 INFO:tasks.workunit.client.1.smithi174.stdout:0/417: dread d9/d1c/f7d [0,4194304] 0 2024-03-21T10:08:54.549 INFO:tasks.workunit.client.0.smithi153.stdout:3/583: chown dd/d1e/d2f/d35/d68/d75/l8f 7 1 2024-03-21T10:08:54.549 INFO:tasks.workunit.client.1.smithi174.stdout:7/301: rename d0/d4/c29 to d0/d4/d10/d44/c61 0 2024-03-21T10:08:54.551 INFO:tasks.workunit.client.0.smithi153.stdout:4/597: rename d11/d2d/d6b/d9c/da0 to d11/d2d/d6b/db6 0 2024-03-21T10:08:54.552 INFO:tasks.workunit.client.1.smithi174.stdout:6/310: getdents d0/d3/d5/d6/d42 0 2024-03-21T10:08:54.553 INFO:tasks.workunit.client.0.smithi153.stdout:1/455: dwrite d6/d13/f3f [4194304,4194304] 0 2024-03-21T10:08:54.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:54 smithi153 ceph-mon[29117]: pgmap v13: 65 pgs: 65 active+clean; 1.6 GiB data, 4.6 GiB used, 532 GiB / 536 GiB avail; 36 MiB/s rd, 195 MiB/s wr, 329 op/s 2024-03-21T10:08:54.557 INFO:tasks.workunit.client.1.smithi174.stdout:6/311: symlink d0/d3/d2a/d56/l64 0 2024-03-21T10:08:54.559 INFO:tasks.workunit.client.1.smithi174.stdout:3/382: write d0/d12/d14/d1a/f23 [967718,2709] 0 2024-03-21T10:08:54.561 INFO:tasks.workunit.client.0.smithi153.stdout:7/469: write d4/d1b/d20/f6a [518055,115126] 0 2024-03-21T10:08:54.561 INFO:tasks.workunit.client.0.smithi153.stdout:7/470: stat d4/l2b 0 2024-03-21T10:08:54.565 INFO:tasks.workunit.client.0.smithi153.stdout:2/648: link dd/d15/d18/d2c/d33/l70 dd/d15/d3c/da7/d91/dae/ld9 0 2024-03-21T10:08:54.565 INFO:tasks.workunit.client.0.smithi153.stdout:4/598: symlink d11/lb7 0 2024-03-21T10:08:54.566 INFO:tasks.workunit.client.0.smithi153.stdout:3/584: link dd/f62 dd/d1e/da6/fcf 0 2024-03-21T10:08:54.570 INFO:tasks.workunit.client.0.smithi153.stdout:4/599: creat d11/d18/d7e/d93/fb8 x:0 0 0 2024-03-21T10:08:54.574 INFO:tasks.workunit.client.0.smithi153.stdout:3/585: mkdir dd/d1e/d2f/d35/dd0 0 2024-03-21T10:08:54.575 INFO:tasks.workunit.client.0.smithi153.stdout:1/456: dwrite d6/d3a/d3c/d50/f7f [0,4194304] 0 2024-03-21T10:08:54.577 INFO:tasks.workunit.client.0.smithi153.stdout:2/649: rename dd/d15/d18/d2c/d4b/f67 to dd/d15/d18/d2c/d4b/d56/d63/d8d/fda 0 2024-03-21T10:08:54.579 INFO:tasks.workunit.client.0.smithi153.stdout:4/600: creat d11/d71/d89/fb9 x:0 0 0 2024-03-21T10:08:54.580 INFO:tasks.workunit.client.0.smithi153.stdout:4/601: stat d11/d5e/c84 0 2024-03-21T10:08:54.583 INFO:tasks.workunit.client.0.smithi153.stdout:4/602: chown d11/d18/l7b 3912186 1 2024-03-21T10:08:54.583 INFO:tasks.workunit.client.0.smithi153.stdout:3/586: rmdir dd/d1e/da6 39 2024-03-21T10:08:54.584 INFO:tasks.workunit.client.0.smithi153.stdout:6/741: write d1f/d2b/f89 [813194,22386] 0 2024-03-21T10:08:54.585 INFO:tasks.workunit.client.0.smithi153.stdout:1/457: mknod d6/d10/d35/d5b/c86 0 2024-03-21T10:08:54.589 INFO:tasks.workunit.client.0.smithi153.stdout:6/742: fsync d1f/d2b/d10a/d40/f44 0 2024-03-21T10:08:54.591 INFO:tasks.workunit.client.0.smithi153.stdout:3/587: getdents dd/d1e/d3d/d94/db8 0 2024-03-21T10:08:54.591 INFO:tasks.workunit.client.0.smithi153.stdout:2/650: creat dd/d15/d18/d3a/fdb x:0 0 0 2024-03-21T10:08:54.595 INFO:tasks.workunit.client.0.smithi153.stdout:1/458: link d6/d3a/d3c/d50/l5d d6/d10/d5c/l87 0 2024-03-21T10:08:54.595 INFO:tasks.workunit.client.0.smithi153.stdout:2/651: mkdir dd/d15/d18/d3a/d3d/ddc 0 2024-03-21T10:08:54.597 INFO:tasks.workunit.client.0.smithi153.stdout:6/743: truncate d1f/d2b/d10a/d40/d4f/f77 3151478 0 2024-03-21T10:08:54.599 INFO:tasks.workunit.client.0.smithi153.stdout:2/652: rmdir dd/d15/d18/d2c/d33/d98 39 2024-03-21T10:08:54.603 INFO:tasks.workunit.client.0.smithi153.stdout:6/744: rmdir d1f/d99 39 2024-03-21T10:08:54.603 INFO:tasks.workunit.client.0.smithi153.stdout:2/653: chown dd/d15/d18/c1f 4340797 1 2024-03-21T10:08:54.605 INFO:tasks.workunit.client.0.smithi153.stdout:6/745: mknod d1f/d99/dbb/c10b 0 2024-03-21T10:08:54.606 INFO:tasks.workunit.client.1.smithi174.stdout:7/302: creat d0/d4/d10/f62 x:0 0 0 2024-03-21T10:08:54.608 INFO:tasks.workunit.client.0.smithi153.stdout:3/588: dread dd/f18 [0,4194304] 0 2024-03-21T10:08:54.609 INFO:tasks.workunit.client.1.smithi174.stdout:7/303: write d0/d4/f22 [866631,130501] 0 2024-03-21T10:08:54.609 INFO:tasks.workunit.client.0.smithi153.stdout:2/654: rename dd/d15/d18/d2c/d4b/d56/cb6 to dd/d15/d5e/cdd 0 2024-03-21T10:08:54.611 INFO:tasks.workunit.client.1.smithi174.stdout:7/304: write d0/f26 [2945695,76533] 0 2024-03-21T10:08:54.611 INFO:tasks.workunit.client.1.smithi174.stdout:6/312: rename d0/d3/d5/d6/d42/f43 to d0/d3/d5/df/d16/f65 0 2024-03-21T10:08:54.612 INFO:tasks.workunit.client.0.smithi153.stdout:3/589: dread - dd/d1e/d2f/d35/daf/dcc/d58/fad zero size 2024-03-21T10:08:54.612 INFO:tasks.workunit.client.1.smithi174.stdout:8/331: dread f8 [0,4194304] 0 2024-03-21T10:08:54.613 INFO:tasks.workunit.client.0.smithi153.stdout:6/746: rename d1f/da0/le4 to d1f/d93/df7/l10c 0 2024-03-21T10:08:54.615 INFO:tasks.workunit.client.0.smithi153.stdout:4/603: dread d11/d18/f35 [0,4194304] 0 2024-03-21T10:08:54.616 INFO:tasks.workunit.client.0.smithi153.stdout:2/655: mkdir dd/d15/d18/d2c/d4b/d56/d63/d8d/d9b/dde 0 2024-03-21T10:08:54.617 INFO:tasks.workunit.client.1.smithi174.stdout:8/332: creat d9/d53/f6c x:0 0 0 2024-03-21T10:08:54.617 INFO:tasks.workunit.client.1.smithi174.stdout:6/313: mkdir d0/d3/d5/df/d66 0 2024-03-21T10:08:54.619 INFO:tasks.workunit.client.0.smithi153.stdout:3/590: creat dd/d1e/fd1 x:0 0 0 2024-03-21T10:08:54.621 INFO:tasks.workunit.client.1.smithi174.stdout:7/305: getdents d0/d4/d10/d44 0 2024-03-21T10:08:54.623 INFO:tasks.workunit.client.1.smithi174.stdout:3/383: dread d0/f2 [0,4194304] 0 2024-03-21T10:08:54.624 INFO:tasks.workunit.client.0.smithi153.stdout:6/747: creat d1f/d2b/d10a/d40/f10d x:0 0 0 2024-03-21T10:08:54.624 INFO:tasks.workunit.client.1.smithi174.stdout:7/306: creat d0/d53/f63 x:0 0 0 2024-03-21T10:08:54.625 INFO:tasks.workunit.client.0.smithi153.stdout:2/656: creat dd/d15/d18/d2c/d4b/d56/d63/fdf x:0 0 0 2024-03-21T10:08:54.626 INFO:tasks.workunit.client.1.smithi174.stdout:6/314: getdents d0/d3/d2a 0 2024-03-21T10:08:54.627 INFO:tasks.workunit.client.1.smithi174.stdout:0/418: dread d9/d21/d23/f2d [0,4194304] 0 2024-03-21T10:08:54.629 INFO:tasks.workunit.client.0.smithi153.stdout:2/657: symlink dd/d15/d18/d2c/le0 0 2024-03-21T10:08:54.631 INFO:tasks.workunit.client.1.smithi174.stdout:3/384: getdents d0/d12/d14 0 2024-03-21T10:08:54.631 INFO:tasks.workunit.client.0.smithi153.stdout:3/591: link dd/d1e/da6/fcf dd/d41/d80/fd2 0 2024-03-21T10:08:54.632 INFO:tasks.workunit.client.1.smithi174.stdout:8/333: write d9/d2a/d39/f5f [1692786,15960] 0 2024-03-21T10:08:54.634 INFO:tasks.workunit.client.0.smithi153.stdout:6/748: getdents d1f 0 2024-03-21T10:08:54.634 INFO:tasks.workunit.client.1.smithi174.stdout:8/334: chown d9/d17/d19/f23 32182 1 2024-03-21T10:08:54.635 INFO:tasks.workunit.client.0.smithi153.stdout:4/604: dwrite d11/f20 [4194304,4194304] 0 2024-03-21T10:08:54.636 INFO:tasks.workunit.client.1.smithi174.stdout:8/335: write d9/d53/d5b/f66 [391886,122499] 0 2024-03-21T10:08:54.638 INFO:tasks.workunit.client.0.smithi153.stdout:6/749: chown d1f/d2b/d10a/d40/d4f/d66/lae 14870 1 2024-03-21T10:08:54.638 INFO:tasks.workunit.client.0.smithi153.stdout:3/592: unlink dd/d1e/d2f/d35/d68/fb6 0 2024-03-21T10:08:54.639 INFO:tasks.workunit.client.0.smithi153.stdout:9/567: sync 2024-03-21T10:08:54.639 INFO:tasks.workunit.client.0.smithi153.stdout:0/486: sync 2024-03-21T10:08:54.639 INFO:tasks.workunit.client.0.smithi153.stdout:5/564: sync 2024-03-21T10:08:54.640 INFO:tasks.workunit.client.0.smithi153.stdout:4/605: mknod d11/d2d/d57/d85/cba 0 2024-03-21T10:08:54.640 INFO:tasks.workunit.client.0.smithi153.stdout:6/750: write d1f/d2b/d10a/d40/d7b/fe3 [76980,1090] 0 2024-03-21T10:08:54.640 INFO:tasks.workunit.client.0.smithi153.stdout:3/593: chown dd/d1e/d3d/d53/dae 43078 1 2024-03-21T10:08:54.644 INFO:tasks.workunit.client.0.smithi153.stdout:9/568: mknod d1/d3b/daa/cc4 0 2024-03-21T10:08:54.644 INFO:tasks.workunit.client.1.smithi174.stdout:7/307: dwrite d0/d4/f23 [0,4194304] 0 2024-03-21T10:08:54.647 INFO:tasks.workunit.client.0.smithi153.stdout:6/751: unlink d1f/f20 0 2024-03-21T10:08:54.647 INFO:tasks.workunit.client.0.smithi153.stdout:9/569: chown d1/d43/d44/d46/d4d/cae 609258 1 2024-03-21T10:08:54.648 INFO:tasks.workunit.client.1.smithi174.stdout:7/308: truncate d0/d4/f23 4761954 0 2024-03-21T10:08:54.648 INFO:tasks.workunit.client.0.smithi153.stdout:5/565: mknod d4/d5/cb2 0 2024-03-21T10:08:54.652 INFO:tasks.workunit.client.1.smithi174.stdout:3/385: dwrite d0/d12/d14/f3a [0,4194304] 0 2024-03-21T10:08:54.653 INFO:tasks.workunit.client.0.smithi153.stdout:0/487: mkdir d4/d16/d19/d95 0 2024-03-21T10:08:54.656 INFO:tasks.workunit.client.1.smithi174.stdout:7/309: creat d0/d4/d10/d44/f64 x:0 0 0 2024-03-21T10:08:54.662 INFO:tasks.workunit.client.1.smithi174.stdout:7/310: mkdir d0/d53/d5f/d65 0 2024-03-21T10:08:54.667 INFO:tasks.workunit.client.1.smithi174.stdout:7/311: rename d0/f56 to d0/d4/d38/f66 0 2024-03-21T10:08:54.671 INFO:tasks.workunit.client.1.smithi174.stdout:7/312: chown d0/d4/d10/d1b/c5b 14234 1 2024-03-21T10:08:54.673 INFO:tasks.workunit.client.1.smithi174.stdout:3/386: dwrite d0/d12/f49 [0,4194304] 0 2024-03-21T10:08:54.673 INFO:tasks.workunit.client.1.smithi174.stdout:1/398: sync 2024-03-21T10:08:54.673 INFO:tasks.workunit.client.0.smithi153.stdout:0/488: creat d4/db/f96 x:0 0 0 2024-03-21T10:08:54.675 INFO:tasks.workunit.client.0.smithi153.stdout:0/489: dread - d4/db/f96 zero size 2024-03-21T10:08:54.678 INFO:tasks.workunit.client.1.smithi174.stdout:7/313: creat d0/d4/d38/f67 x:0 0 0 2024-03-21T10:08:54.680 INFO:tasks.workunit.client.1.smithi174.stdout:7/314: chown d0/f27 24 1 2024-03-21T10:08:54.681 INFO:tasks.workunit.client.0.smithi153.stdout:5/566: rename d4/d5/d16/d24 to d4/d5/df/d67/da0/db3 0 2024-03-21T10:08:54.683 INFO:tasks.workunit.client.0.smithi153.stdout:0/490: read d4/db/f29 [3522323,32877] 0 2024-03-21T10:08:54.686 INFO:tasks.workunit.client.0.smithi153.stdout:3/594: link dd/d1e/d3d/d94/d73/lb7 dd/d1e/d2f/d35/d68/d75/ld3 0 2024-03-21T10:08:54.687 INFO:tasks.workunit.client.0.smithi153.stdout:0/491: stat d4/d16/d27/c5f 0 2024-03-21T10:08:54.690 INFO:tasks.workunit.client.1.smithi174.stdout:7/315: link d0/d4/d10/f21 d0/d4/d10/d1b/f68 0 2024-03-21T10:08:54.691 INFO:tasks.workunit.client.0.smithi153.stdout:5/567: read - d4/d5/df/d67/da0/db3/f83 zero size 2024-03-21T10:08:54.693 INFO:tasks.workunit.client.1.smithi174.stdout:3/387: dread d0/f13 [0,4194304] 0 2024-03-21T10:08:54.694 INFO:tasks.workunit.client.1.smithi174.stdout:7/316: mknod d0/d53/d5f/c69 0 2024-03-21T10:08:54.695 INFO:tasks.workunit.client.1.smithi174.stdout:1/399: dwrite d4/d7/d32/d86/f8a [0,4194304] 0 2024-03-21T10:08:54.698 INFO:tasks.workunit.client.0.smithi153.stdout:1/459: dwrite d6/d3a/f53 [0,4194304] 0 2024-03-21T10:08:54.698 INFO:tasks.workunit.client.1.smithi174.stdout:3/388: mkdir d0/d12/d46/d59/d88 0 2024-03-21T10:08:54.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:54 smithi174 ceph-mon[37496]: pgmap v13: 65 pgs: 65 active+clean; 1.6 GiB data, 4.6 GiB used, 532 GiB / 536 GiB avail; 36 MiB/s rd, 195 MiB/s wr, 329 op/s 2024-03-21T10:08:54.701 INFO:tasks.workunit.client.0.smithi153.stdout:5/568: symlink d4/d5/d16/lb4 0 2024-03-21T10:08:54.703 INFO:tasks.workunit.client.0.smithi153.stdout:0/492: symlink d4/d16/l97 0 2024-03-21T10:08:54.708 INFO:tasks.workunit.client.0.smithi153.stdout:0/493: write d4/db/f8b [743492,94086] 0 2024-03-21T10:08:54.709 INFO:tasks.workunit.client.1.smithi174.stdout:1/400: write d4/d7/d33/d43/f42 [710148,62418] 0 2024-03-21T10:08:54.710 INFO:tasks.workunit.client.1.smithi174.stdout:3/389: dread d0/d12/d14/d1a/d27/d4f/d5d/f31 [0,4194304] 0 2024-03-21T10:08:54.714 INFO:tasks.workunit.client.1.smithi174.stdout:3/390: write d0/d12/d14/d1a/f42 [844562,82436] 0 2024-03-21T10:08:54.714 INFO:tasks.workunit.client.0.smithi153.stdout:1/460: unlink d6/d13/c23 0 2024-03-21T10:08:54.716 INFO:tasks.workunit.client.0.smithi153.stdout:3/595: dwrite dd/d1e/d2f/d35/d68/daa/fac [0,4194304] 0 2024-03-21T10:08:54.717 INFO:tasks.workunit.client.1.smithi174.stdout:7/317: dwrite d0/d4/f33 [0,4194304] 0 2024-03-21T10:08:54.718 INFO:tasks.workunit.client.0.smithi153.stdout:4/606: dread d11/d18/d54/fa7 [0,4194304] 0 2024-03-21T10:08:54.719 INFO:tasks.workunit.client.0.smithi153.stdout:4/607: chown d11/d2d 11 1 2024-03-21T10:08:54.719 INFO:tasks.workunit.client.0.smithi153.stdout:3/596: chown dd/d1e/d2f/d35/daf/fb4 371468 1 2024-03-21T10:08:54.722 INFO:tasks.workunit.client.0.smithi153.stdout:5/569: truncate d4/f75 827550 0 2024-03-21T10:08:54.723 INFO:tasks.workunit.client.0.smithi153.stdout:3/597: write dd/f2b [4887041,39142] 0 2024-03-21T10:08:54.724 INFO:tasks.workunit.client.1.smithi174.stdout:3/391: fdatasync d0/d12/d14/d1a/d27/d3c/f72 0 2024-03-21T10:08:54.724 INFO:tasks.workunit.client.0.smithi153.stdout:5/570: chown d4/d8/d3b/f4b 31510 1 2024-03-21T10:08:54.726 INFO:tasks.workunit.client.0.smithi153.stdout:1/461: mknod d6/d10/d35/c88 0 2024-03-21T10:08:54.728 INFO:tasks.workunit.client.1.smithi174.stdout:7/318: creat d0/d4/d10/d60/f6a x:0 0 0 2024-03-21T10:08:54.728 INFO:tasks.workunit.client.0.smithi153.stdout:9/570: rmdir d1/d3b 39 2024-03-21T10:08:54.732 INFO:tasks.workunit.client.1.smithi174.stdout:3/392: symlink d0/d12/d14/d1a/d5c/l89 0 2024-03-21T10:08:54.734 INFO:tasks.workunit.client.0.smithi153.stdout:0/494: rename d4/d16/d3e to d4/d16/d19/d30/d6c/d98 0 2024-03-21T10:08:54.737 INFO:tasks.workunit.client.0.smithi153.stdout:9/571: dread d1/d8a/fb9 [0,4194304] 0 2024-03-21T10:08:54.737 INFO:tasks.workunit.client.0.smithi153.stdout:0/495: readlink d4/d4c/d83/l2a 0 2024-03-21T10:08:54.738 INFO:tasks.workunit.client.1.smithi174.stdout:1/401: dwrite d4/d7/d15/d53/f6a [0,4194304] 0 2024-03-21T10:08:54.740 INFO:tasks.workunit.client.0.smithi153.stdout:0/496: fdatasync d4/db/f11 0 2024-03-21T10:08:54.743 INFO:tasks.workunit.client.1.smithi174.stdout:1/402: chown d4/d7/d33/d43/d51/c5c 434734 1 2024-03-21T10:08:54.744 INFO:tasks.workunit.client.0.smithi153.stdout:5/571: rmdir d4/d8/d5a/d92 39 2024-03-21T10:08:54.746 INFO:tasks.workunit.client.1.smithi174.stdout:1/403: read d4/d7/d33/f36 [2160451,23227] 0 2024-03-21T10:08:54.747 INFO:tasks.workunit.client.1.smithi174.stdout:7/319: write d0/fa [1015479,56413] 0 2024-03-21T10:08:54.748 INFO:tasks.workunit.client.1.smithi174.stdout:3/393: truncate d0/d12/f49 2294114 0 2024-03-21T10:08:54.753 INFO:tasks.workunit.client.1.smithi174.stdout:7/320: dread d0/d4/d10/d1b/f25 [0,4194304] 0 2024-03-21T10:08:54.757 INFO:tasks.workunit.client.1.smithi174.stdout:7/321: fsync d0/d4/d10/f19 0 2024-03-21T10:08:54.758 INFO:tasks.workunit.client.0.smithi153.stdout:6/752: fsync d1f/d2b/d10a/d40/fa5 0 2024-03-21T10:08:54.760 INFO:tasks.workunit.client.0.smithi153.stdout:9/572: symlink d1/d43/d44/d46/d9e/d4b/lc5 0 2024-03-21T10:08:54.761 INFO:tasks.workunit.client.0.smithi153.stdout:4/608: creat d11/db1/fbb x:0 0 0 2024-03-21T10:08:54.761 INFO:tasks.workunit.client.0.smithi153.stdout:0/497: creat d4/d16/d19/d30/d6c/d98/d5e/f99 x:0 0 0 2024-03-21T10:08:54.763 INFO:tasks.workunit.client.1.smithi174.stdout:7/322: rename d0/d4/d10/c2c to d0/d4/c6b 0 2024-03-21T10:08:54.765 INFO:tasks.workunit.client.1.smithi174.stdout:1/404: dwrite d4/d7/d33/d43/f42 [0,4194304] 0 2024-03-21T10:08:54.765 INFO:tasks.workunit.client.0.smithi153.stdout:3/598: mknod dd/d1e/cd4 0 2024-03-21T10:08:54.766 INFO:tasks.workunit.client.0.smithi153.stdout:5/572: dread - d4/d5/df/d67/da0/db3/f74 zero size 2024-03-21T10:08:54.766 INFO:tasks.workunit.client.0.smithi153.stdout:0/498: write d4/db/f8b [702514,68183] 0 2024-03-21T10:08:54.767 INFO:tasks.workunit.client.0.smithi153.stdout:4/609: write d11/d18/f55 [16225,3369] 0 2024-03-21T10:08:54.770 INFO:tasks.workunit.client.0.smithi153.stdout:9/573: symlink d1/d43/d44/d46/d9e/d61/lc6 0 2024-03-21T10:08:54.773 INFO:tasks.workunit.client.1.smithi174.stdout:7/323: rename d0/d4/f17 to d0/d53/d5f/d65/f6c 0 2024-03-21T10:08:54.773 INFO:tasks.workunit.client.1.smithi174.stdout:3/394: write d0/d12/d14/d1a/d27/f66 [1203639,22319] 0 2024-03-21T10:08:54.774 INFO:tasks.workunit.client.0.smithi153.stdout:5/573: rmdir d4/d8/d3b 39 2024-03-21T10:08:54.774 INFO:tasks.workunit.client.0.smithi153.stdout:6/753: symlink d1f/d99/dbb/dcf/l10e 0 2024-03-21T10:08:54.775 INFO:tasks.workunit.client.1.smithi174.stdout:1/405: creat d4/d7/d68/d7b/f8c x:0 0 0 2024-03-21T10:08:54.778 INFO:tasks.workunit.client.0.smithi153.stdout:6/754: readlink d1f/d2b/d10a/d40/d4f/l5b 0 2024-03-21T10:08:54.779 INFO:tasks.workunit.client.1.smithi174.stdout:3/395: write d0/d12/d14/d1a/d27/d4f/d5d/f1b [3289578,63850] 0 2024-03-21T10:08:54.780 INFO:tasks.workunit.client.1.smithi174.stdout:1/406: write d4/d7/d28/f30 [5059248,41262] 0 2024-03-21T10:08:54.781 INFO:tasks.workunit.client.1.smithi174.stdout:1/407: fsync d4/d7/d15/f2f 0 2024-03-21T10:08:54.786 INFO:tasks.workunit.client.0.smithi153.stdout:3/599: fdatasync dd/d1e/f4f 0 2024-03-21T10:08:54.790 INFO:tasks.workunit.client.0.smithi153.stdout:1/462: truncate d6/f1c 4005933 0 2024-03-21T10:08:54.792 INFO:tasks.workunit.client.1.smithi174.stdout:7/324: dwrite d0/d4/f23 [4194304,4194304] 0 2024-03-21T10:08:54.792 INFO:tasks.workunit.client.0.smithi153.stdout:3/600: write dd/d1e/d6b/f9c [198133,117774] 0 2024-03-21T10:08:54.792 INFO:tasks.workunit.client.0.smithi153.stdout:9/574: unlink d1/d4e/f89 0 2024-03-21T10:08:54.797 INFO:tasks.workunit.client.1.smithi174.stdout:7/325: fdatasync d0/d4/d10/d1b/f4e 0 2024-03-21T10:08:54.798 INFO:tasks.workunit.client.0.smithi153.stdout:1/463: rmdir d6/d32 39 2024-03-21T10:08:54.801 INFO:tasks.workunit.client.1.smithi174.stdout:1/408: dwrite d4/d7/d32/f71 [0,4194304] 0 2024-03-21T10:08:54.802 INFO:tasks.workunit.client.1.smithi174.stdout:3/396: dwrite d0/fa [4194304,4194304] 0 2024-03-21T10:08:54.803 INFO:tasks.workunit.client.1.smithi174.stdout:7/326: chown d0/d53/d5f/d65/f6c 17688353 1 2024-03-21T10:08:54.807 INFO:tasks.workunit.client.0.smithi153.stdout:3/601: symlink dd/d1e/d2f/d35/ld5 0 2024-03-21T10:08:54.808 INFO:tasks.workunit.client.0.smithi153.stdout:4/610: dwrite d11/d2d/f43 [0,4194304] 0 2024-03-21T10:08:54.809 INFO:tasks.workunit.client.1.smithi174.stdout:3/397: dread d0/d12/d14/d1a/f3f [0,4194304] 0 2024-03-21T10:08:54.812 INFO:tasks.workunit.client.0.smithi153.stdout:8/580: sync 2024-03-21T10:08:54.814 INFO:tasks.workunit.client.0.smithi153.stdout:0/499: dwrite d4/d16/d27/f79 [0,4194304] 0 2024-03-21T10:08:54.814 INFO:tasks.workunit.client.0.smithi153.stdout:5/574: truncate d4/d5/f7 1740236 0 2024-03-21T10:08:54.817 INFO:tasks.workunit.client.0.smithi153.stdout:6/755: getdents d1f/de0 0 2024-03-21T10:08:54.818 INFO:tasks.workunit.client.1.smithi174.stdout:3/398: rmdir d0/d12/d14/d1a/d44/d56 39 2024-03-21T10:08:54.819 INFO:tasks.workunit.client.0.smithi153.stdout:3/602: fsync dd/f13 0 2024-03-21T10:08:54.821 INFO:tasks.workunit.client.1.smithi174.stdout:2/400: sync 2024-03-21T10:08:54.822 INFO:tasks.workunit.client.1.smithi174.stdout:7/327: link d0/d4/d10/d44/c46 d0/d4/c6d 0 2024-03-21T10:08:54.823 INFO:tasks.workunit.client.0.smithi153.stdout:1/464: symlink d6/d32/d3e/d4f/l89 0 2024-03-21T10:08:54.828 INFO:tasks.workunit.client.0.smithi153.stdout:1/465: write d6/d32/d3e/f63 [577843,89918] 0 2024-03-21T10:08:54.829 INFO:tasks.workunit.client.1.smithi174.stdout:1/409: dwrite d4/f19 [0,4194304] 0 2024-03-21T10:08:54.830 INFO:tasks.workunit.client.0.smithi153.stdout:0/500: rmdir d4/db 39 2024-03-21T10:08:54.832 INFO:tasks.workunit.client.0.smithi153.stdout:9/575: dwrite d1/d4e/f98 [0,4194304] 0 2024-03-21T10:08:54.839 INFO:tasks.workunit.client.1.smithi174.stdout:1/410: creat d4/d7/d33/d43/d51/f8d x:0 0 0 2024-03-21T10:08:54.841 INFO:tasks.workunit.client.0.smithi153.stdout:3/603: fsync dd/d1e/d45/f74 0 2024-03-21T10:08:54.841 INFO:tasks.workunit.client.1.smithi174.stdout:3/399: link d0/d12/d14/d1a/f23 d0/d12/d14/d1a/d27/d3c/d73/f8a 0 2024-03-21T10:08:54.842 INFO:tasks.workunit.client.0.smithi153.stdout:6/756: dwrite d1f/d2b/d10a/d40/f10d [0,4194304] 0 2024-03-21T10:08:54.846 INFO:tasks.workunit.client.0.smithi153.stdout:6/757: truncate d1f/d99/dbb/fcb 5059831 0 2024-03-21T10:08:54.847 INFO:tasks.workunit.client.1.smithi174.stdout:1/411: chown d4/d7/d15/f39 2131 1 2024-03-21T10:08:54.848 INFO:tasks.workunit.client.1.smithi174.stdout:1/412: write d4/d7/d15/d4f/f83 [873901,96764] 0 2024-03-21T10:08:54.850 INFO:tasks.workunit.client.0.smithi153.stdout:3/604: truncate dd/d1e/d2f/d35/daf/fb4 98967 0 2024-03-21T10:08:54.850 INFO:tasks.workunit.client.0.smithi153.stdout:1/466: dread d6/d13/f5e [0,4194304] 0 2024-03-21T10:08:54.853 INFO:tasks.workunit.client.0.smithi153.stdout:1/467: write d6/d3a/d66/f85 [1876121,101245] 0 2024-03-21T10:08:54.855 INFO:tasks.workunit.client.0.smithi153.stdout:0/501: mkdir d4/d16/d19/d58/d9a 0 2024-03-21T10:08:54.857 INFO:tasks.workunit.client.0.smithi153.stdout:0/502: chown d4/d4c/d83/d22/d61 793929583 1 2024-03-21T10:08:54.862 INFO:tasks.workunit.client.0.smithi153.stdout:9/576: write d1/d43/d44/d46/d9e/d24/f38 [944707,93594] 0 2024-03-21T10:08:54.862 INFO:tasks.workunit.client.1.smithi174.stdout:1/413: mkdir d4/d7/d68/d8e 0 2024-03-21T10:08:54.862 INFO:tasks.workunit.client.1.smithi174.stdout:1/414: chown d4/d7/d32/l4c 1410267 1 2024-03-21T10:08:54.863 INFO:tasks.workunit.client.1.smithi174.stdout:3/400: getdents d0/d12/d14/d1a/d27/d7f 0 2024-03-21T10:08:54.866 INFO:tasks.workunit.client.1.smithi174.stdout:1/415: truncate d4/d5/d6c/f6f 109729 0 2024-03-21T10:08:54.871 INFO:tasks.workunit.client.1.smithi174.stdout:5/449: sync 2024-03-21T10:08:54.872 INFO:tasks.workunit.client.0.smithi153.stdout:7/471: sync 2024-03-21T10:08:54.872 INFO:tasks.workunit.client.0.smithi153.stdout:1/468: symlink d6/d32/d3e/l8a 0 2024-03-21T10:08:54.874 INFO:tasks.workunit.client.0.smithi153.stdout:2/658: sync 2024-03-21T10:08:54.874 INFO:tasks.workunit.client.0.smithi153.stdout:5/575: link d4/d8/d3b/d56/cb0 d4/d5/df/d67/d86/d96/cb5 0 2024-03-21T10:08:54.880 INFO:tasks.workunit.client.1.smithi174.stdout:5/450: read d1/d49/d4f/f57 [2936299,109215] 0 2024-03-21T10:08:54.880 INFO:tasks.workunit.client.0.smithi153.stdout:5/576: stat d4/d5/d9e 0 2024-03-21T10:08:54.881 INFO:tasks.workunit.client.0.smithi153.stdout:9/577: rename d1/l80 to d1/d3b/lc7 0 2024-03-21T10:08:54.882 INFO:tasks.workunit.client.1.smithi174.stdout:5/451: fsync d1/d2f/d37/f3f 0 2024-03-21T10:08:54.884 INFO:tasks.workunit.client.1.smithi174.stdout:2/401: truncate d5/f49 2783323 0 2024-03-21T10:08:54.884 INFO:tasks.workunit.client.0.smithi153.stdout:3/605: dwrite dd/d41/f64 [0,4194304] 0 2024-03-21T10:08:54.885 INFO:tasks.workunit.client.1.smithi174.stdout:7/328: write d0/d4/d10/d1b/f68 [2170455,24017] 0 2024-03-21T10:08:54.885 INFO:tasks.workunit.client.1.smithi174.stdout:3/401: dwrite d0/d12/d14/d1a/d27/d3c/d73/f8a [0,4194304] 0 2024-03-21T10:08:54.886 INFO:tasks.workunit.client.0.smithi153.stdout:8/581: dread d5/d1b/f37 [0,4194304] 0 2024-03-21T10:08:54.886 INFO:tasks.workunit.client.1.smithi174.stdout:1/416: dwrite d4/d5/f72 [0,4194304] 0 2024-03-21T10:08:54.886 INFO:tasks.workunit.client.1.smithi174.stdout:2/402: readlink d5/d25/l4c 0 2024-03-21T10:08:54.887 INFO:tasks.workunit.client.0.smithi153.stdout:6/758: dwrite d1f/d2b/d10a/d40/d51/ff3 [0,4194304] 0 2024-03-21T10:08:54.887 INFO:tasks.workunit.client.0.smithi153.stdout:8/582: chown d5/d1b/d49 739314770 1 2024-03-21T10:08:54.887 INFO:tasks.workunit.client.0.smithi153.stdout:4/611: truncate d11/d2d/f38 479911 0 2024-03-21T10:08:54.889 INFO:tasks.workunit.client.0.smithi153.stdout:2/659: mkdir dd/d15/d18/d2c/d33/d98/dc7/de1 0 2024-03-21T10:08:54.889 INFO:tasks.workunit.client.1.smithi174.stdout:5/452: mknod d1/d49/d4f/ca0 0 2024-03-21T10:08:54.899 INFO:tasks.workunit.client.0.smithi153.stdout:9/578: creat d1/d4/d5f/fc8 x:0 0 0 2024-03-21T10:08:54.901 INFO:tasks.workunit.client.1.smithi174.stdout:1/417: rmdir d4/d7/d68 39 2024-03-21T10:08:54.903 INFO:tasks.workunit.client.1.smithi174.stdout:7/329: rename d0/d4/f47 to d0/d53/f6e 0 2024-03-21T10:08:54.904 INFO:tasks.workunit.client.0.smithi153.stdout:7/472: dwrite d4/d1b/d20/f93 [0,4194304] 0 2024-03-21T10:08:54.904 INFO:tasks.workunit.client.0.smithi153.stdout:2/660: symlink dd/d15/d18/d2c/d4b/d56/d63/le2 0 2024-03-21T10:08:54.912 INFO:tasks.workunit.client.0.smithi153.stdout:6/759: mkdir d1f/de0/de1/df2/d10f 0 2024-03-21T10:08:54.915 INFO:tasks.workunit.client.0.smithi153.stdout:6/760: read d1f/d2b/d10a/d40/d51/ff3 [1007027,110919] 0 2024-03-21T10:08:54.917 INFO:tasks.workunit.client.1.smithi174.stdout:1/418: dread d4/d7/d15/d53/f6a [0,4194304] 0 2024-03-21T10:08:54.917 INFO:tasks.workunit.client.0.smithi153.stdout:0/503: dwrite d4/d16/d27/f76 [0,4194304] 0 2024-03-21T10:08:54.921 INFO:tasks.workunit.client.0.smithi153.stdout:8/583: getdents d5/d9/d43/dd/d20/d26/d24/d52/d56/da7 0 2024-03-21T10:08:54.923 INFO:tasks.workunit.client.0.smithi153.stdout:5/577: dread d4/d5/df/d67/d6a/f77 [0,4194304] 0 2024-03-21T10:08:54.925 INFO:tasks.workunit.client.0.smithi153.stdout:1/469: rename d6/d3a/d3c/d50/l69 to d6/d13/l8b 0 2024-03-21T10:08:54.928 INFO:tasks.workunit.client.0.smithi153.stdout:8/584: write d5/d9/d43/dd/d20/d26/d7a/f81 [551637,45039] 0 2024-03-21T10:08:54.935 INFO:tasks.workunit.client.0.smithi153.stdout:2/661: link dd/d15/d18/d2c/d4b/d56/f60 dd/d15/d18/d2c/d4b/fe3 0 2024-03-21T10:08:54.935 INFO:tasks.workunit.client.0.smithi153.stdout:5/578: symlink d4/d5/df/d67/d6a/lb6 0 2024-03-21T10:08:54.936 INFO:tasks.workunit.client.0.smithi153.stdout:3/606: dwrite dd/d1e/d45/fc8 [0,4194304] 0 2024-03-21T10:08:54.941 INFO:tasks.workunit.client.0.smithi153.stdout:9/579: rename d1/d43/d44/d46/d9e/d2e/d67/l90 to d1/d43/d44/d46/d9e/d4b/d84/lc9 0 2024-03-21T10:08:54.945 INFO:tasks.workunit.client.0.smithi153.stdout:6/761: truncate d1f/f4b 392043 0 2024-03-21T10:08:54.947 INFO:tasks.workunit.client.1.smithi174.stdout:2/403: unlink d5/d25/d27/d2d/d45/d5b/d55/d75/c84 0 2024-03-21T10:08:54.948 INFO:tasks.workunit.client.0.smithi153.stdout:9/580: write d1/d4e/f98 [4375752,71992] 0 2024-03-21T10:08:54.949 INFO:tasks.workunit.client.0.smithi153.stdout:6/762: dread - d1f/d99/ff9 zero size 2024-03-21T10:08:54.950 INFO:tasks.workunit.client.1.smithi174.stdout:7/330: write d0/d4/d10/d1b/f3b [894318,114041] 0 2024-03-21T10:08:54.951 INFO:tasks.workunit.client.1.smithi174.stdout:3/402: creat d0/d12/d14/d1a/d27/d4f/d5d/d7d/f8b x:0 0 0 2024-03-21T10:08:54.951 INFO:tasks.workunit.client.1.smithi174.stdout:2/404: write d5/d25/d27/f2a [2075446,37199] 0 2024-03-21T10:08:54.952 INFO:tasks.workunit.client.1.smithi174.stdout:3/403: fdatasync d0/f70 0 2024-03-21T10:08:54.953 INFO:tasks.workunit.client.0.smithi153.stdout:4/612: write d11/d2d/f36 [1091840,56440] 0 2024-03-21T10:08:54.954 INFO:tasks.workunit.client.0.smithi153.stdout:2/662: mknod dd/d15/d18/d3a/d8c/ce4 0 2024-03-21T10:08:54.955 INFO:tasks.workunit.client.0.smithi153.stdout:5/579: read d4/f75 [375437,107132] 0 2024-03-21T10:08:54.955 INFO:tasks.workunit.client.1.smithi174.stdout:7/331: creat d0/d4/d10/d1b/f6f x:0 0 0 2024-03-21T10:08:54.956 INFO:tasks.workunit.client.1.smithi174.stdout:1/419: creat d4/d7/f8f x:0 0 0 2024-03-21T10:08:54.958 INFO:tasks.workunit.client.0.smithi153.stdout:6/763: write d1f/d2b/d10a/d40/d65/f69 [4932019,78250] 0 2024-03-21T10:08:54.958 INFO:tasks.workunit.client.1.smithi174.stdout:2/405: creat d5/d25/d27/d2e/d7b/f8e x:0 0 0 2024-03-21T10:08:54.958 INFO:tasks.workunit.client.1.smithi174.stdout:1/420: chown d4/d7/d33/f36 99494 1 2024-03-21T10:08:54.959 INFO:tasks.workunit.client.0.smithi153.stdout:8/585: dread d5/d9/d43/dd/d2d/d45/d88/f5d [0,4194304] 0 2024-03-21T10:08:54.960 INFO:tasks.workunit.client.1.smithi174.stdout:7/332: creat d0/d4/d38/f70 x:0 0 0 2024-03-21T10:08:54.962 INFO:tasks.workunit.client.0.smithi153.stdout:0/504: rename d4/d4c/f92 to d4/d16/d19/d30/d6c/f9b 0 2024-03-21T10:08:54.965 INFO:tasks.workunit.client.1.smithi174.stdout:7/333: creat d0/d53/d5f/f71 x:0 0 0 2024-03-21T10:08:54.968 INFO:tasks.workunit.client.1.smithi174.stdout:1/421: creat d4/d7/f90 x:0 0 0 2024-03-21T10:08:54.969 INFO:tasks.workunit.client.0.smithi153.stdout:2/663: creat dd/d15/d3c/da7/fe5 x:0 0 0 2024-03-21T10:08:54.969 INFO:tasks.workunit.client.1.smithi174.stdout:7/334: creat d0/d4/d10/d44/f72 x:0 0 0 2024-03-21T10:08:54.973 INFO:tasks.workunit.client.1.smithi174.stdout:7/335: creat d0/d4/d10/d44/f73 x:0 0 0 2024-03-21T10:08:54.977 INFO:tasks.workunit.client.1.smithi174.stdout:3/404: truncate d0/d12/f2d 5030862 0 2024-03-21T10:08:54.979 INFO:tasks.workunit.client.0.smithi153.stdout:9/581: dwrite d1/d43/d44/d46/f26 [0,4194304] 0 2024-03-21T10:08:54.979 INFO:tasks.workunit.client.0.smithi153.stdout:5/580: symlink d4/d5/lb7 0 2024-03-21T10:08:54.979 INFO:tasks.workunit.client.1.smithi174.stdout:3/405: dread - d0/d12/d14/d1a/d27/d3c/f61 zero size 2024-03-21T10:08:54.979 INFO:tasks.workunit.client.0.smithi153.stdout:1/470: link d6/d3a/d66/f6a d6/f8c 0 2024-03-21T10:08:54.980 INFO:tasks.workunit.client.1.smithi174.stdout:7/336: link d0/d4/d10/d1b/c1f d0/c74 0 2024-03-21T10:08:54.981 INFO:tasks.workunit.client.1.smithi174.stdout:1/422: dread d4/d7/d32/d86/f8a [0,4194304] 0 2024-03-21T10:08:54.984 INFO:tasks.workunit.client.1.smithi174.stdout:7/337: write d0/d4/f2d [1100978,70028] 0 2024-03-21T10:08:54.986 INFO:tasks.workunit.client.1.smithi174.stdout:0/419: sync 2024-03-21T10:08:54.986 INFO:tasks.workunit.client.1.smithi174.stdout:8/336: sync 2024-03-21T10:08:54.986 INFO:tasks.workunit.client.1.smithi174.stdout:6/315: sync 2024-03-21T10:08:54.986 INFO:tasks.workunit.client.1.smithi174.stdout:4/436: sync 2024-03-21T10:08:54.986 INFO:tasks.workunit.client.1.smithi174.stdout:9/433: sync 2024-03-21T10:08:54.987 INFO:tasks.workunit.client.0.smithi153.stdout:6/764: mkdir d1f/d93/dd8/d59/d70/d110 0 2024-03-21T10:08:54.987 INFO:tasks.workunit.client.1.smithi174.stdout:3/406: mknod d0/d12/d14/d4c/d7a/c8c 0 2024-03-21T10:08:54.990 INFO:tasks.workunit.client.0.smithi153.stdout:9/582: creat d1/d43/d44/d46/d9e/d24/d41/fca x:0 0 0 2024-03-21T10:08:54.992 INFO:tasks.workunit.client.0.smithi153.stdout:5/581: creat d4/d5/df/d67/d86/d96/fb8 x:0 0 0 2024-03-21T10:08:54.994 INFO:tasks.workunit.client.1.smithi174.stdout:7/338: chown d0/d4/d10/d44/c46 85666872 1 2024-03-21T10:08:54.994 INFO:tasks.workunit.client.1.smithi174.stdout:3/407: unlink d0/d12/d14/d1a/f42 0 2024-03-21T10:08:54.994 INFO:tasks.workunit.client.0.smithi153.stdout:3/607: dread dd/d1e/d2f/f3a [0,4194304] 0 2024-03-21T10:08:54.995 INFO:tasks.workunit.client.1.smithi174.stdout:2/406: dwrite d5/f12 [0,4194304] 0 2024-03-21T10:08:54.995 INFO:tasks.workunit.client.1.smithi174.stdout:8/337: creat d9/d2a/d4d/f6d x:0 0 0 2024-03-21T10:08:54.996 INFO:tasks.workunit.client.0.smithi153.stdout:9/583: chown d1/d43/l45 5 1 2024-03-21T10:08:54.997 INFO:tasks.workunit.client.0.smithi153.stdout:5/582: readlink d4/d5/de/l4a 0 2024-03-21T10:08:54.997 INFO:tasks.workunit.client.0.smithi153.stdout:0/505: getdents d4/d16 0 2024-03-21T10:08:54.998 INFO:tasks.workunit.client.1.smithi174.stdout:4/437: dread - d4/d21/d31/d3a/f8e zero size 2024-03-21T10:08:54.999 INFO:tasks.workunit.client.1.smithi174.stdout:8/338: chown d9/d17/f6a 1 1 2024-03-21T10:08:55.000 INFO:tasks.workunit.client.1.smithi174.stdout:0/420: truncate d9/f1f 333471 0 2024-03-21T10:08:55.000 INFO:tasks.workunit.client.1.smithi174.stdout:6/316: creat d0/d3/d5/df/d16/f67 x:0 0 0 2024-03-21T10:08:55.001 INFO:tasks.workunit.client.0.smithi153.stdout:0/506: fdatasync d4/d4c/d83/f20 0 2024-03-21T10:08:55.001 INFO:tasks.workunit.client.0.smithi153.stdout:6/765: dread - d1f/d2b/d10a/d40/d4f/f67 zero size 2024-03-21T10:08:55.001 INFO:tasks.workunit.client.1.smithi174.stdout:2/407: fdatasync d5/d25/d27/d2d/d45/d5b/d55/f69 0 2024-03-21T10:08:55.001 INFO:tasks.workunit.client.1.smithi174.stdout:8/339: fdatasync d9/d17/d19/d27/d60/f56 0 2024-03-21T10:08:55.003 INFO:tasks.workunit.client.0.smithi153.stdout:2/664: dwrite dd/d15/d3c/da7/d91/dae/d50/d59/d5d/fb0 [0,4194304] 0 2024-03-21T10:08:55.003 INFO:tasks.workunit.client.0.smithi153.stdout:5/583: fdatasync d4/d5/df/d67/d6a/f77 0 2024-03-21T10:08:55.005 INFO:tasks.workunit.client.0.smithi153.stdout:9/584: dread - d1/d43/d44/d46/d9e/d24/fb4 zero size 2024-03-21T10:08:55.007 INFO:tasks.workunit.client.0.smithi153.stdout:3/608: mkdir dd/d1e/d2f/d35/daf/dcc/dd6 0 2024-03-21T10:08:55.009 INFO:tasks.workunit.client.0.smithi153.stdout:3/609: chown dd/d1e/d6b 120278101 1 2024-03-21T10:08:55.009 INFO:tasks.workunit.client.0.smithi153.stdout:0/507: chown d4/d16/d19/d30/d6c/d98/d5b/l8f 18 1 2024-03-21T10:08:55.010 INFO:tasks.workunit.client.0.smithi153.stdout:5/584: symlink d4/d8/d5a/d5d/lb9 0 2024-03-21T10:08:55.013 INFO:tasks.workunit.client.0.smithi153.stdout:0/508: creat d4/d16/d19/d58/f9c x:0 0 0 2024-03-21T10:08:55.018 INFO:tasks.workunit.client.0.smithi153.stdout:3/610: dread - dd/d1e/d3d/d94/d73/d7f/d91/fa8 zero size 2024-03-21T10:08:55.019 INFO:tasks.workunit.client.0.smithi153.stdout:0/509: creat d4/d16/d19/d30/f9d x:0 0 0 2024-03-21T10:08:55.019 INFO:tasks.workunit.client.1.smithi174.stdout:4/438: mkdir d4/d21/d31/d6a/d8f 0 2024-03-21T10:08:55.019 INFO:tasks.workunit.client.1.smithi174.stdout:3/408: creat d0/d12/d14/d1a/d5c/f8d x:0 0 0 2024-03-21T10:08:55.021 INFO:tasks.workunit.client.1.smithi174.stdout:3/409: write d0/f70 [924892,82309] 0 2024-03-21T10:08:55.023 INFO:tasks.workunit.client.1.smithi174.stdout:0/421: mknod d9/d21/d31/c8e 0 2024-03-21T10:08:55.023 INFO:tasks.workunit.client.1.smithi174.stdout:4/439: unlink d4/d21/l62 0 2024-03-21T10:08:55.024 INFO:tasks.workunit.client.0.smithi153.stdout:1/471: dwrite d6/d3a/d3c/d50/d51/f5f [0,4194304] 0 2024-03-21T10:08:55.024 INFO:tasks.workunit.client.0.smithi153.stdout:3/611: write f8 [6207432,10175] 0 2024-03-21T10:08:55.026 INFO:tasks.workunit.client.0.smithi153.stdout:2/665: link dd/d15/d3c/da7/d91/dae/d50/d58/d79/f7e dd/d15/d3c/fe6 0 2024-03-21T10:08:55.027 INFO:tasks.workunit.client.1.smithi174.stdout:2/408: getdents d5/d25/d27/d2d/d45/d5b/d2f/d68 0 2024-03-21T10:08:55.028 INFO:tasks.workunit.client.1.smithi174.stdout:4/440: creat d4/d21/d24/d28/d38/f90 x:0 0 0 2024-03-21T10:08:55.029 INFO:tasks.workunit.client.1.smithi174.stdout:7/339: rmdir d0 39 2024-03-21T10:08:55.031 INFO:tasks.workunit.client.1.smithi174.stdout:2/409: symlink d5/d25/d27/d2d/l8f 0 2024-03-21T10:08:55.032 INFO:tasks.workunit.client.1.smithi174.stdout:4/441: write d4/d21/d31/d3a/f8e [418864,100836] 0 2024-03-21T10:08:55.035 INFO:tasks.workunit.client.1.smithi174.stdout:7/340: dread - d0/d53/f63 zero size 2024-03-21T10:08:55.037 INFO:tasks.workunit.client.1.smithi174.stdout:4/442: mknod d4/d19/c91 0 2024-03-21T10:08:55.038 INFO:tasks.workunit.client.1.smithi174.stdout:7/341: truncate d0/f57 551814 0 2024-03-21T10:08:55.039 INFO:tasks.workunit.client.0.smithi153.stdout:5/585: link d4/d5/de/d38/c7f d4/cba 0 2024-03-21T10:08:55.039 INFO:tasks.workunit.client.0.smithi153.stdout:2/666: truncate dd/d15/d18/f24 3091664 0 2024-03-21T10:08:55.041 INFO:tasks.workunit.client.1.smithi174.stdout:2/410: truncate d5/d25/d27/d2d/d45/f5d 600790 0 2024-03-21T10:08:55.042 INFO:tasks.workunit.client.0.smithi153.stdout:5/586: chown d4/d8/d5a/c9b 47 1 2024-03-21T10:08:55.042 INFO:tasks.workunit.client.0.smithi153.stdout:9/585: dwrite d1/d43/d44/d46/d4d/f74 [0,4194304] 0 2024-03-21T10:08:55.043 INFO:tasks.workunit.client.1.smithi174.stdout:7/342: symlink d0/d4/d10/l75 0 2024-03-21T10:08:55.044 INFO:tasks.workunit.client.0.smithi153.stdout:2/667: fsync dd/d15/d3c/da7/d91/dae/d50/fcc 0 2024-03-21T10:08:55.046 INFO:tasks.workunit.client.0.smithi153.stdout:1/472: rename d6/fa to d6/d3a/d3c/f8d 0 2024-03-21T10:08:55.046 INFO:tasks.workunit.client.0.smithi153.stdout:9/586: truncate d1/d4/f9c 378788 0 2024-03-21T10:08:55.047 INFO:tasks.workunit.client.0.smithi153.stdout:3/612: mknod dd/d41/d80/dbb/dbf/cd7 0 2024-03-21T10:08:55.048 INFO:tasks.workunit.client.1.smithi174.stdout:2/411: mkdir d5/d25/d27/d2e/d90 0 2024-03-21T10:08:55.048 INFO:tasks.workunit.client.1.smithi174.stdout:7/343: rename d0/d4/d10/d1b/f28 to d0/d53/f76 0 2024-03-21T10:08:55.049 INFO:tasks.workunit.client.0.smithi153.stdout:9/587: read - d1/d4/d5f/fc8 zero size 2024-03-21T10:08:55.052 INFO:tasks.workunit.client.1.smithi174.stdout:2/412: symlink d5/d25/d27/d2d/d79/l91 0 2024-03-21T10:08:55.056 INFO:tasks.workunit.client.0.smithi153.stdout:3/613: mknod dd/d41/dab/cd8 0 2024-03-21T10:08:55.056 INFO:tasks.workunit.client.1.smithi174.stdout:9/434: dwrite d2/d11/f4a [4194304,4194304] 0 2024-03-21T10:08:55.058 INFO:tasks.workunit.client.0.smithi153.stdout:5/587: mknod d4/d8/d5a/d92/d9f/cbb 0 2024-03-21T10:08:55.059 INFO:tasks.workunit.client.0.smithi153.stdout:1/473: write d6/d3a/d3c/d50/d51/f5f [730852,27163] 0 2024-03-21T10:08:55.066 INFO:tasks.workunit.client.1.smithi174.stdout:9/435: write d2/f8e [551926,71883] 0 2024-03-21T10:08:55.067 INFO:tasks.workunit.client.0.smithi153.stdout:6/766: dread d1f/d2b/d10a/d40/d5f/f80 [0,4194304] 0 2024-03-21T10:08:55.067 INFO:tasks.workunit.client.0.smithi153.stdout:9/588: getdents d1/d43/d44/d46/d9e/d2e 0 2024-03-21T10:08:55.069 INFO:tasks.workunit.client.0.smithi153.stdout:1/474: mkdir d6/d13/d8e 0 2024-03-21T10:08:55.070 INFO:tasks.workunit.client.1.smithi174.stdout:4/443: dwrite d4/d21/d24/d28/f69 [0,4194304] 0 2024-03-21T10:08:55.070 INFO:tasks.workunit.client.0.smithi153.stdout:5/588: write d4/d5/de/f3f [1157729,35634] 0 2024-03-21T10:08:55.073 INFO:tasks.workunit.client.1.smithi174.stdout:7/344: dwrite d0/d4/f8 [0,4194304] 0 2024-03-21T10:08:55.074 INFO:tasks.workunit.client.0.smithi153.stdout:3/614: truncate dd/d1e/d3d/d94/d73/d7f/d91/fbd 381450 0 2024-03-21T10:08:55.075 INFO:tasks.workunit.client.0.smithi153.stdout:3/615: stat f2 0 2024-03-21T10:08:55.076 INFO:tasks.workunit.client.1.smithi174.stdout:9/436: rename d2/d24/d2e/c70 to d2/d24/d2e/d7e/c9d 0 2024-03-21T10:08:55.076 INFO:tasks.workunit.client.1.smithi174.stdout:7/345: write d0/d53/d5f/f71 [1010447,98150] 0 2024-03-21T10:08:55.078 INFO:tasks.workunit.client.0.smithi153.stdout:1/475: creat d6/d3a/d3c/d50/d51/f8f x:0 0 0 2024-03-21T10:08:55.081 INFO:tasks.workunit.client.1.smithi174.stdout:4/444: unlink d4/d21/d24/d28/f2b 0 2024-03-21T10:08:55.082 INFO:tasks.workunit.client.0.smithi153.stdout:5/589: symlink d4/d5/df/d67/d86/d96/lbc 0 2024-03-21T10:08:55.085 INFO:tasks.workunit.client.0.smithi153.stdout:0/510: dread d4/f62 [0,4194304] 0 2024-03-21T10:08:55.086 INFO:tasks.workunit.client.0.smithi153.stdout:2/668: dwrite dd/d15/d3c/da7/d91/dae/d50/d59/d5d/fb0 [4194304,4194304] 0 2024-03-21T10:08:55.087 INFO:tasks.workunit.client.0.smithi153.stdout:5/590: fsync d4/d8/f68 0 2024-03-21T10:08:55.092 INFO:tasks.workunit.client.0.smithi153.stdout:3/616: getdents dd/d41/d80/dbb 0 2024-03-21T10:08:55.094 INFO:tasks.workunit.client.0.smithi153.stdout:0/511: fsync d4/d4c/f7a 0 2024-03-21T10:08:55.095 INFO:tasks.workunit.client.1.smithi174.stdout:7/346: getdents d0/d4/d10/d44 0 2024-03-21T10:08:55.095 INFO:tasks.workunit.client.1.smithi174.stdout:9/437: read d2/d11/d1d/d2a/f73 [4762077,56894] 0 2024-03-21T10:08:55.097 INFO:tasks.workunit.client.1.smithi174.stdout:7/347: dread - d0/d4/d38/f67 zero size 2024-03-21T10:08:55.098 INFO:tasks.workunit.client.0.smithi153.stdout:1/476: truncate d6/d3a/d3c/f43 844284 0 2024-03-21T10:08:55.103 INFO:tasks.workunit.client.0.smithi153.stdout:9/589: dwrite d1/d43/d44/d46/d9e/d2e/f62 [0,4194304] 0 2024-03-21T10:08:55.103 INFO:tasks.workunit.client.0.smithi153.stdout:2/669: truncate dd/d15/d5e/fb8 949592 0 2024-03-21T10:08:55.103 INFO:tasks.workunit.client.1.smithi174.stdout:9/438: fdatasync d2/f47 0 2024-03-21T10:08:55.103 INFO:tasks.workunit.client.1.smithi174.stdout:7/348: rmdir d0/d4/d10/d60 39 2024-03-21T10:08:55.104 INFO:tasks.workunit.client.0.smithi153.stdout:0/512: mknod d4/d16/d19/d95/c9e 0 2024-03-21T10:08:55.105 INFO:tasks.workunit.client.1.smithi174.stdout:4/445: dwrite d4/d21/d24/f73 [0,4194304] 0 2024-03-21T10:08:55.106 INFO:tasks.workunit.client.1.smithi174.stdout:9/439: chown d2/d24/d2e/f48 20847111 1 2024-03-21T10:08:55.108 INFO:tasks.workunit.client.1.smithi174.stdout:4/446: fsync d4/d21/d24/d28/d38/f88 0 2024-03-21T10:08:55.110 INFO:tasks.workunit.client.1.smithi174.stdout:7/349: truncate d0/d4/d10/d1b/f68 3192360 0 2024-03-21T10:08:55.110 INFO:tasks.workunit.client.0.smithi153.stdout:3/617: symlink dd/d1e/d2f/d35/daf/dcc/dd6/ld9 0 2024-03-21T10:08:55.111 INFO:tasks.workunit.client.1.smithi174.stdout:4/447: stat d4/d21/d24/d28 0 2024-03-21T10:08:55.112 INFO:tasks.workunit.client.0.smithi153.stdout:2/670: rmdir dd/d15/d18/d2c/d33/d98 39 2024-03-21T10:08:55.114 INFO:tasks.workunit.client.0.smithi153.stdout:6/767: truncate d1f/d99/faf 545798 0 2024-03-21T10:08:55.115 INFO:tasks.workunit.client.1.smithi174.stdout:2/413: dread d5/d25/d27/d2d/d45/d5b/d55/f83 [0,4194304] 0 2024-03-21T10:08:55.115 INFO:tasks.workunit.client.1.smithi174.stdout:9/440: symlink d2/d11/d1d/d2a/d91/l9e 0 2024-03-21T10:08:55.115 INFO:tasks.workunit.client.1.smithi174.stdout:7/350: write d0/d53/f6e [4616301,1320] 0 2024-03-21T10:08:55.121 INFO:tasks.workunit.client.0.smithi153.stdout:6/768: creat d1f/d2b/db3/f111 x:0 0 0 2024-03-21T10:08:55.121 INFO:tasks.workunit.client.0.smithi153.stdout:1/477: creat d6/f90 x:0 0 0 2024-03-21T10:08:55.122 INFO:tasks.workunit.client.1.smithi174.stdout:5/453: sync 2024-03-21T10:08:55.122 INFO:tasks.workunit.client.0.smithi153.stdout:2/671: creat dd/d15/d3c/da7/d91/dae/d50/d59/fe7 x:0 0 0 2024-03-21T10:08:55.123 INFO:tasks.workunit.client.1.smithi174.stdout:2/414: rename d5/f4a to d5/d25/d27/d2e/d4f/d65/f92 0 2024-03-21T10:08:55.125 INFO:tasks.workunit.client.1.smithi174.stdout:7/351: read d0/d4/d10/f45 [2799047,22912] 0 2024-03-21T10:08:55.126 INFO:tasks.workunit.client.1.smithi174.stdout:4/448: truncate d4/d21/d24/d28/f39 491240 0 2024-03-21T10:08:55.129 INFO:tasks.workunit.client.1.smithi174.stdout:4/449: readlink d4/d21/d31/d3a/l64 0 2024-03-21T10:08:55.131 INFO:tasks.workunit.client.1.smithi174.stdout:9/441: rename d2/d24/c83 to d2/d11/d1d/d5e/c9f 0 2024-03-21T10:08:55.135 INFO:tasks.workunit.client.1.smithi174.stdout:2/415: link d5/f14 d5/d25/d27/d2d/d45/d5b/d2f/d66/f93 0 2024-03-21T10:08:55.138 INFO:tasks.workunit.client.1.smithi174.stdout:9/442: rename d2/d24/l52 to d2/d11/d1d/d8b/la0 0 2024-03-21T10:08:55.140 INFO:tasks.workunit.client.0.smithi153.stdout:3/618: dwrite dd/d41/f6e [0,4194304] 0 2024-03-21T10:08:55.141 INFO:tasks.workunit.client.1.smithi174.stdout:2/416: chown d5/d25/d27/d2d/d45/d5b/c1a 726356032 1 2024-03-21T10:08:55.141 INFO:tasks.workunit.client.1.smithi174.stdout:9/443: write d2/d24/f88 [548991,103430] 0 2024-03-21T10:08:55.145 INFO:tasks.workunit.client.0.smithi153.stdout:2/672: mknod dd/d15/d3c/da7/d91/dae/d50/ce8 0 2024-03-21T10:08:55.145 INFO:tasks.workunit.client.0.smithi153.stdout:9/590: write d1/d43/d44/d46/d9e/d61/f71 [626144,9982] 0 2024-03-21T10:08:55.146 INFO:tasks.workunit.client.1.smithi174.stdout:9/444: mknod d2/d24/d2e/ca1 0 2024-03-21T10:08:55.146 INFO:tasks.workunit.client.1.smithi174.stdout:7/352: dwrite d0/d4/f2d [0,4194304] 0 2024-03-21T10:08:55.147 INFO:tasks.workunit.client.0.smithi153.stdout:1/478: creat d6/d13/d75/f91 x:0 0 0 2024-03-21T10:08:55.148 INFO:tasks.workunit.client.1.smithi174.stdout:9/445: write d2/d24/f99 [1031660,31622] 0 2024-03-21T10:08:55.149 INFO:tasks.workunit.client.0.smithi153.stdout:0/513: truncate d4/d4c/d83/d22/f47 281453 0 2024-03-21T10:08:55.149 INFO:tasks.workunit.client.0.smithi153.stdout:1/479: chown d6/d13/d75/f83 30 1 2024-03-21T10:08:55.151 INFO:tasks.workunit.client.0.smithi153.stdout:9/591: symlink d1/d43/d44/d46/d9e/d4b/lcb 0 2024-03-21T10:08:55.152 INFO:tasks.workunit.client.0.smithi153.stdout:1/480: dread - d6/d13/d75/f83 zero size 2024-03-21T10:08:55.153 INFO:tasks.workunit.client.1.smithi174.stdout:7/353: unlink d0/d4/d10/c1a 0 2024-03-21T10:08:55.155 INFO:tasks.workunit.client.0.smithi153.stdout:2/673: creat dd/d15/d3c/da7/d91/dae/d50/d59/fe9 x:0 0 0 2024-03-21T10:08:55.156 INFO:tasks.workunit.client.1.smithi174.stdout:9/446: unlink d2/d24/f34 0 2024-03-21T10:08:55.156 INFO:tasks.workunit.client.1.smithi174.stdout:2/417: link d5/d25/d27/d2e/l89 d5/d25/d27/d2d/d45/d5b/d2f/l94 0 2024-03-21T10:08:55.157 INFO:tasks.workunit.client.0.smithi153.stdout:5/591: dread d4/d5/f7 [0,4194304] 0 2024-03-21T10:08:55.157 INFO:tasks.workunit.client.0.smithi153.stdout:3/619: rmdir dd/d41/dab/dc5 0 2024-03-21T10:08:55.159 INFO:tasks.workunit.client.0.smithi153.stdout:0/514: rmdir d4/d16/d19/d30/d6c/d98/d5b 39 2024-03-21T10:08:55.160 INFO:tasks.workunit.client.1.smithi174.stdout:2/418: write d5/d25/d27/d2e/d4f/f6d [233670,60682] 0 2024-03-21T10:08:55.161 INFO:tasks.workunit.client.0.smithi153.stdout:3/620: readlink dd/d1e/l59 0 2024-03-21T10:08:55.162 INFO:tasks.workunit.client.0.smithi153.stdout:1/481: mkdir d6/d10/d35/d5b/d92 0 2024-03-21T10:08:55.162 INFO:tasks.workunit.client.1.smithi174.stdout:7/354: creat d0/d4/d10/d60/f77 x:0 0 0 2024-03-21T10:08:55.164 INFO:tasks.workunit.client.0.smithi153.stdout:3/621: chown dd/d1e/d45 0 1 2024-03-21T10:08:55.167 INFO:tasks.workunit.client.1.smithi174.stdout:4/450: fsync d4/d21/d24/d28/f39 0 2024-03-21T10:08:55.169 INFO:tasks.workunit.client.0.smithi153.stdout:2/674: rmdir dd/d15/d18/d2c/d4b/d56/d63/d8d/d9b 39 2024-03-21T10:08:55.169 INFO:tasks.workunit.client.0.smithi153.stdout:9/592: write d1/d43/d44/f33 [2079429,63972] 0 2024-03-21T10:08:55.170 INFO:tasks.workunit.client.1.smithi174.stdout:5/454: truncate d1/f2c 2221762 0 2024-03-21T10:08:55.170 INFO:tasks.workunit.client.0.smithi153.stdout:6/769: dwrite d1f/da0/f100 [0,4194304] 0 2024-03-21T10:08:55.172 INFO:tasks.workunit.client.1.smithi174.stdout:7/355: symlink d0/d53/l78 0 2024-03-21T10:08:55.173 INFO:tasks.workunit.client.0.smithi153.stdout:9/593: chown d1/d43/d44/l29 278573 1 2024-03-21T10:08:55.174 INFO:tasks.workunit.client.1.smithi174.stdout:2/419: unlink d5/d25/d27/d2d/d45/d5b/d55/f58 0 2024-03-21T10:08:55.175 INFO:tasks.workunit.client.1.smithi174.stdout:9/447: dwrite d2/d54/f5b [0,4194304] 0 2024-03-21T10:08:55.179 INFO:tasks.workunit.client.1.smithi174.stdout:5/455: truncate d1/d16/f60 57450 0 2024-03-21T10:08:55.185 INFO:tasks.workunit.client.1.smithi174.stdout:7/356: mkdir d0/d4/d10/d79 0 2024-03-21T10:08:55.188 INFO:tasks.workunit.client.1.smithi174.stdout:2/420: dread d5/d25/d27/d2e/d4f/f6d [0,4194304] 0 2024-03-21T10:08:55.194 INFO:tasks.workunit.client.1.smithi174.stdout:4/451: dread d4/d21/d24/d28/f39 [0,4194304] 0 2024-03-21T10:08:55.196 INFO:tasks.workunit.client.1.smithi174.stdout:7/357: mknod d0/d4/d38/c7a 0 2024-03-21T10:08:55.199 INFO:tasks.workunit.client.1.smithi174.stdout:5/456: rename d1/d16/d2e/d66/d8b/l39 to d1/d49/d93/d9c/la1 0 2024-03-21T10:08:55.201 INFO:tasks.workunit.client.1.smithi174.stdout:5/457: read - d1/d2f/d37/f9e zero size 2024-03-21T10:08:55.205 INFO:tasks.workunit.client.1.smithi174.stdout:5/458: creat d1/d2f/d37/fa2 x:0 0 0 2024-03-21T10:08:55.209 INFO:tasks.workunit.client.1.smithi174.stdout:5/459: write d1/f4a [1101518,16972] 0 2024-03-21T10:08:55.212 INFO:tasks.workunit.client.1.smithi174.stdout:9/448: dwrite d2/f47 [0,4194304] 0 2024-03-21T10:08:55.219 INFO:tasks.workunit.client.1.smithi174.stdout:7/358: dwrite d0/d4/d10/f19 [0,4194304] 0 2024-03-21T10:08:55.226 INFO:tasks.workunit.client.0.smithi153.stdout:2/675: write dd/d15/d18/d2c/d33/f7d [1029730,101906] 0 2024-03-21T10:08:55.226 INFO:tasks.workunit.client.0.smithi153.stdout:5/592: creat d4/d8/fbd x:0 0 0 2024-03-21T10:08:55.232 INFO:tasks.workunit.client.0.smithi153.stdout:9/594: unlink d1/d43/d72/d9d/fa0 0 2024-03-21T10:08:55.232 INFO:tasks.workunit.client.0.smithi153.stdout:3/622: link dd/c19 dd/d1e/d3d/cda 0 2024-03-21T10:08:55.232 INFO:tasks.workunit.client.0.smithi153.stdout:2/676: fsync dd/d15/d18/d2c/d4b/fd0 0 2024-03-21T10:08:55.236 INFO:tasks.workunit.client.1.smithi174.stdout:5/460: dwrite d1/d5c/d73/f7f [0,4194304] 0 2024-03-21T10:08:55.237 INFO:tasks.workunit.client.0.smithi153.stdout:3/623: chown dd/d1e/d45/c88 1712979 1 2024-03-21T10:08:55.239 INFO:tasks.workunit.client.0.smithi153.stdout:1/482: link d6/d10/l1a d6/d13/l93 0 2024-03-21T10:08:55.239 INFO:tasks.workunit.client.0.smithi153.stdout:0/515: dwrite d4/d16/d19/d58/f9c [0,4194304] 0 2024-03-21T10:08:55.240 INFO:tasks.workunit.client.1.smithi174.stdout:4/452: dread d4/d21/d24/d28/f5e [0,4194304] 0 2024-03-21T10:08:55.242 INFO:tasks.workunit.client.0.smithi153.stdout:2/677: symlink dd/d15/d18/d2c/lea 0 2024-03-21T10:08:55.247 INFO:tasks.workunit.client.0.smithi153.stdout:3/624: unlink dd/d1e/d3d/d53/c5e 0 2024-03-21T10:08:55.247 INFO:tasks.workunit.client.1.smithi174.stdout:5/461: rename d1/d2f/d37/fa2 to d1/d2f/d37/d9b/fa3 0 2024-03-21T10:08:55.248 INFO:tasks.workunit.client.1.smithi174.stdout:9/449: dwrite d2/d11/d1d/d4b/f56 [0,4194304] 0 2024-03-21T10:08:55.248 INFO:tasks.workunit.client.1.smithi174.stdout:4/453: symlink d4/d21/d24/d28/d79/l92 0 2024-03-21T10:08:55.250 INFO:tasks.workunit.client.0.smithi153.stdout:0/516: fdatasync d4/db/f29 0 2024-03-21T10:08:55.254 INFO:tasks.workunit.client.0.smithi153.stdout:5/593: dwrite d4/d5/de/d44/d48/f87 [0,4194304] 0 2024-03-21T10:08:55.255 INFO:tasks.workunit.client.1.smithi174.stdout:9/450: creat d2/d11/d1d/d8b/fa2 x:0 0 0 2024-03-21T10:08:55.257 INFO:tasks.workunit.client.1.smithi174.stdout:5/462: read d1/d16/d2e/d66/d8b/f36 [62998,6970] 0 2024-03-21T10:08:55.257 INFO:tasks.workunit.client.1.smithi174.stdout:4/454: symlink d4/d21/d31/d6a/d8f/l93 0 2024-03-21T10:08:55.257 INFO:tasks.workunit.client.0.smithi153.stdout:0/517: mkdir d4/d16/d9f 0 2024-03-21T10:08:55.259 INFO:tasks.workunit.client.1.smithi174.stdout:9/451: symlink d2/d11/d1d/d2a/d91/la3 0 2024-03-21T10:08:55.260 INFO:tasks.workunit.client.0.smithi153.stdout:5/594: mkdir d4/d5/df/d67/dbe 0 2024-03-21T10:08:55.263 INFO:tasks.workunit.client.1.smithi174.stdout:1/423: sync 2024-03-21T10:08:55.265 INFO:tasks.workunit.client.1.smithi174.stdout:4/455: creat d4/d21/d31/d6a/d8f/f94 x:0 0 0 2024-03-21T10:08:55.266 INFO:tasks.workunit.client.1.smithi174.stdout:1/424: readlink d4/d7/d33/d43/l3f 0 2024-03-21T10:08:55.277 INFO:tasks.workunit.client.1.smithi174.stdout:4/456: link d4/d21/d31/d6a/f72 d4/d21/d31/d3a/f95 0 2024-03-21T10:08:55.279 INFO:tasks.workunit.client.0.smithi153.stdout:0/518: dread d4/d4c/f7a [0,4194304] 0 2024-03-21T10:08:55.279 INFO:tasks.workunit.client.0.smithi153.stdout:1/483: truncate d6/d32/d3e/d4f/f58 3643300 0 2024-03-21T10:08:55.280 INFO:tasks.workunit.client.0.smithi153.stdout:2/678: write dd/d15/d18/d2c/d4b/d56/f7b [119945,117410] 0 2024-03-21T10:08:55.282 INFO:tasks.workunit.client.0.smithi153.stdout:5/595: dwrite d4/d5/df/d5e/f82 [0,4194304] 0 2024-03-21T10:08:55.285 INFO:tasks.workunit.client.0.smithi153.stdout:3/625: dread dd/d1e/f4f [0,4194304] 0 2024-03-21T10:08:55.286 INFO:tasks.workunit.client.0.smithi153.stdout:1/484: mknod d6/d10/d2c/c94 0 2024-03-21T10:08:55.286 INFO:tasks.workunit.client.1.smithi174.stdout:5/463: dwrite d1/d16/d2e/d66/f89 [0,4194304] 0 2024-03-21T10:08:55.286 INFO:tasks.workunit.client.1.smithi174.stdout:4/457: dread d4/d21/f7b [0,4194304] 0 2024-03-21T10:08:55.287 INFO:tasks.workunit.client.0.smithi153.stdout:5/596: mkdir d4/d5/df/d67/d86/dbf 0 2024-03-21T10:08:55.289 INFO:tasks.workunit.client.1.smithi174.stdout:7/359: dread d0/d4/d38/f66 [0,4194304] 0 2024-03-21T10:08:55.289 INFO:tasks.workunit.client.1.smithi174.stdout:1/425: truncate d4/d2d/f5e 71760 0 2024-03-21T10:08:55.293 INFO:tasks.workunit.client.0.smithi153.stdout:9/595: dread d1/d43/d44/d46/d9e/d4b/d84/fa4 [0,4194304] 0 2024-03-21T10:08:55.293 INFO:tasks.workunit.client.0.smithi153.stdout:0/519: chown d4/d16/d19 9090474 1 2024-03-21T10:08:55.295 INFO:tasks.workunit.client.0.smithi153.stdout:0/520: fdatasync d4/db/f96 0 2024-03-21T10:08:55.297 INFO:tasks.workunit.client.0.smithi153.stdout:5/597: dread d4/d5/de/d44/d48/f84 [0,4194304] 0 2024-03-21T10:08:55.300 INFO:tasks.workunit.client.1.smithi174.stdout:5/464: creat d1/d16/d2e/d66/d8b/fa4 x:0 0 0 2024-03-21T10:08:55.303 INFO:tasks.workunit.client.0.smithi153.stdout:5/598: creat d4/d5/df/d5e/fc0 x:0 0 0 2024-03-21T10:08:55.304 INFO:tasks.workunit.client.0.smithi153.stdout:9/596: getdents d1/d4e/d68 0 2024-03-21T10:08:55.306 INFO:tasks.workunit.client.0.smithi153.stdout:2/679: dwrite dd/d15/d18/d2c/d4b/d56/d63/f80 [4194304,4194304] 0 2024-03-21T10:08:55.307 INFO:tasks.workunit.client.0.smithi153.stdout:5/599: chown d4/f46 10544094 1 2024-03-21T10:08:55.311 INFO:tasks.workunit.client.0.smithi153.stdout:3/626: truncate dd/f4c 2547184 0 2024-03-21T10:08:55.312 INFO:tasks.workunit.client.1.smithi174.stdout:9/452: dwrite d2/d24/d2e/f9b [0,4194304] 0 2024-03-21T10:08:55.313 INFO:tasks.workunit.client.0.smithi153.stdout:0/521: write d4/d16/d27/f4d [4654648,62559] 0 2024-03-21T10:08:55.314 INFO:tasks.workunit.client.0.smithi153.stdout:3/627: chown dd/d1e/d2f/d35/ld5 11715631 1 2024-03-21T10:08:55.320 INFO:tasks.workunit.client.1.smithi174.stdout:1/426: dread d4/d2d/f76 [0,4194304] 0 2024-03-21T10:08:55.323 INFO:tasks.workunit.client.0.smithi153.stdout:9/597: rename d1/d43/d44/d46/d9e/d4b/d84 to d1/d43/d44/d46/d9e/d24/d41/dbe/dcc 0 2024-03-21T10:08:55.323 INFO:tasks.workunit.client.0.smithi153.stdout:3/628: creat dd/d1e/d3d/d94/fdb x:0 0 0 2024-03-21T10:08:55.324 INFO:tasks.workunit.client.0.smithi153.stdout:2/680: truncate dd/d15/d3c/da7/d91/dae/d50/d59/d5d/f7f 206297 0 2024-03-21T10:08:55.324 INFO:tasks.workunit.client.1.smithi174.stdout:7/360: dwrite d0/d4/d10/f21 [0,4194304] 0 2024-03-21T10:08:55.324 INFO:tasks.workunit.client.1.smithi174.stdout:4/458: dwrite d4/d21/d24/d28/d38/f5f [0,4194304] 0 2024-03-21T10:08:55.327 INFO:tasks.workunit.client.0.smithi153.stdout:5/600: mkdir d4/d8/d3b/dc1 0 2024-03-21T10:08:55.328 INFO:tasks.workunit.client.0.smithi153.stdout:3/629: readlink dd/d1e/d2f/d35/daf/dcc/dd6/ld9 0 2024-03-21T10:08:55.330 INFO:tasks.workunit.client.1.smithi174.stdout:8/340: sync 2024-03-21T10:08:55.330 INFO:tasks.workunit.client.1.smithi174.stdout:6/317: sync 2024-03-21T10:08:55.331 INFO:tasks.workunit.client.0.smithi153.stdout:3/630: read dd/d41/f6e [1455070,71729] 0 2024-03-21T10:08:55.333 INFO:tasks.workunit.client.1.smithi174.stdout:8/341: fsync d9/f63 0 2024-03-21T10:08:55.339 INFO:tasks.workunit.client.1.smithi174.stdout:5/465: dwrite d1/d49/f67 [4194304,4194304] 0 2024-03-21T10:08:55.339 INFO:tasks.workunit.client.1.smithi174.stdout:7/361: rename d0/d4/d38/c7a to d0/d53/d5f/d65/c7b 0 2024-03-21T10:08:55.339 INFO:tasks.workunit.client.1.smithi174.stdout:1/427: dread d4/d7/f2c [0,4194304] 0 2024-03-21T10:08:55.342 INFO:tasks.workunit.client.1.smithi174.stdout:8/342: chown d9/f63 8855326 1 2024-03-21T10:08:55.346 INFO:tasks.workunit.client.0.smithi153.stdout:0/522: dwrite d4/d16/d27/f79 [0,4194304] 0 2024-03-21T10:08:55.349 INFO:tasks.workunit.client.1.smithi174.stdout:9/453: dwrite d2/f84 [0,4194304] 0 2024-03-21T10:08:55.350 INFO:tasks.workunit.client.1.smithi174.stdout:8/343: rename d9/f1c to d9/d17/f6e 0 2024-03-21T10:08:55.353 INFO:tasks.workunit.client.1.smithi174.stdout:6/318: symlink d0/d3/d2a/d56/d5e/l68 0 2024-03-21T10:08:55.353 INFO:tasks.workunit.client.0.smithi153.stdout:2/681: mkdir dd/d15/d18/d2c/d33/deb 0 2024-03-21T10:08:55.356 INFO:tasks.workunit.client.1.smithi174.stdout:1/428: mknod d4/d7/d15/c91 0 2024-03-21T10:08:55.356 INFO:tasks.workunit.client.1.smithi174.stdout:7/362: creat d0/d4/d10/d1b/f7c x:0 0 0 2024-03-21T10:08:55.356 INFO:tasks.workunit.client.1.smithi174.stdout:9/454: write d2/d24/d2e/f69 [4469335,16229] 0 2024-03-21T10:08:55.362 INFO:tasks.workunit.client.1.smithi174.stdout:9/455: write d2/d24/d2e/f31 [3355315,3430] 0 2024-03-21T10:08:55.362 INFO:tasks.workunit.client.0.smithi153.stdout:2/682: fdatasync dd/d15/d3c/da7/d91/dae/d50/d59/fe7 0 2024-03-21T10:08:55.363 INFO:tasks.workunit.client.1.smithi174.stdout:3/410: sync 2024-03-21T10:08:55.363 INFO:tasks.workunit.client.1.smithi174.stdout:0/422: sync 2024-03-21T10:08:55.364 INFO:tasks.workunit.client.1.smithi174.stdout:8/344: creat d9/d53/f6f x:0 0 0 2024-03-21T10:08:55.364 INFO:tasks.workunit.client.1.smithi174.stdout:7/363: write d0/f11 [492702,93685] 0 2024-03-21T10:08:55.364 INFO:tasks.workunit.client.1.smithi174.stdout:2/421: sync 2024-03-21T10:08:55.364 INFO:tasks.workunit.client.1.smithi174.stdout:6/319: write d0/d3/d5/d6/f4a [506518,67337] 0 2024-03-21T10:08:55.370 INFO:tasks.workunit.client.0.smithi153.stdout:2/683: chown dd/d15/d18 973 1 2024-03-21T10:08:55.370 INFO:tasks.workunit.client.0.smithi153.stdout:1/485: dread d6/d3a/d3c/f8d [0,4194304] 0 2024-03-21T10:08:55.370 INFO:tasks.workunit.client.0.smithi153.stdout:3/631: dwrite dd/d1e/d2f/d35/daf/dcc/f9e [0,4194304] 0 2024-03-21T10:08:55.371 INFO:tasks.workunit.client.1.smithi174.stdout:4/459: dwrite d4/d21/d24/d28/d38/f90 [0,4194304] 0 2024-03-21T10:08:55.375 INFO:tasks.workunit.client.0.smithi153.stdout:2/684: mknod dd/d15/d18/d2c/cec 0 2024-03-21T10:08:55.375 INFO:tasks.workunit.client.0.smithi153.stdout:1/486: rmdir d6/d10/d35 39 2024-03-21T10:08:55.376 INFO:tasks.workunit.client.1.smithi174.stdout:1/429: mkdir d4/d7/d33/d43/d77/d87/d92 0 2024-03-21T10:08:55.378 INFO:tasks.workunit.client.1.smithi174.stdout:2/422: chown d5/d25/d27/d2d/d45/d5b/d2f/d66 33 1 2024-03-21T10:08:55.378 INFO:tasks.workunit.client.0.smithi153.stdout:3/632: chown dd/d1e/d3d/d94/d73/d7f/d91/db1 7204527 1 2024-03-21T10:08:55.378 INFO:tasks.workunit.client.1.smithi174.stdout:7/364: symlink d0/d53/d5f/l7d 0 2024-03-21T10:08:55.384 INFO:tasks.workunit.client.1.smithi174.stdout:3/411: dread - d0/d12/d14/d1a/d27/d4f/d64/f74 zero size 2024-03-21T10:08:55.386 INFO:tasks.workunit.client.1.smithi174.stdout:2/423: readlink d5/l1b 0 2024-03-21T10:08:55.387 INFO:tasks.workunit.client.1.smithi174.stdout:7/365: fdatasync d0/d4/d10/f19 0 2024-03-21T10:08:55.388 INFO:tasks.workunit.client.1.smithi174.stdout:5/466: write d1/d5c/f7a [344110,29860] 0 2024-03-21T10:08:55.389 INFO:tasks.workunit.client.0.smithi153.stdout:1/487: chown d6/d13/d75/l78 115 1 2024-03-21T10:08:55.391 INFO:tasks.workunit.client.1.smithi174.stdout:1/430: write d4/d7/d15/d4f/f83 [634957,124644] 0 2024-03-21T10:08:55.395 INFO:tasks.workunit.client.1.smithi174.stdout:6/320: link d0/d3/d5/fe d0/d3/d5/d6/d42/f69 0 2024-03-21T10:08:55.395 INFO:tasks.workunit.client.0.smithi153.stdout:3/633: rename dd/f20 to dd/d1e/d2f/d35/dd0/fdc 0 2024-03-21T10:08:55.395 INFO:tasks.workunit.client.1.smithi174.stdout:8/345: getdents d9/d17/d19/d31 0 2024-03-21T10:08:55.397 INFO:tasks.workunit.client.1.smithi174.stdout:2/424: creat d5/d25/d27/d74/f95 x:0 0 0 2024-03-21T10:08:55.397 INFO:tasks.workunit.client.0.smithi153.stdout:3/634: write f8 [8316427,59767] 0 2024-03-21T10:08:55.400 INFO:tasks.workunit.client.1.smithi174.stdout:2/425: stat d5/f14 0 2024-03-21T10:08:55.401 INFO:tasks.workunit.client.1.smithi174.stdout:8/346: creat d9/d17/f70 x:0 0 0 2024-03-21T10:08:55.402 INFO:tasks.workunit.client.0.smithi153.stdout:2/685: mkdir dd/d15/d18/d2c/d33/d98/dc7/ded 0 2024-03-21T10:08:55.405 INFO:tasks.workunit.client.0.smithi153.stdout:0/523: dwrite d4/d16/d19/d30/d6c/d98/d5e/f6b [0,4194304] 0 2024-03-21T10:08:55.408 INFO:tasks.workunit.client.1.smithi174.stdout:5/467: creat d1/d5c/d73/fa5 x:0 0 0 2024-03-21T10:08:55.409 INFO:tasks.workunit.client.1.smithi174.stdout:1/431: unlink d4/d7/d32/f71 0 2024-03-21T10:08:55.416 INFO:tasks.workunit.client.0.smithi153.stdout:1/488: symlink d6/d10/d35/d5b/l95 0 2024-03-21T10:08:55.416 INFO:tasks.workunit.client.1.smithi174.stdout:8/347: rename d9/d17/d19/d31/l55 to d9/d53/d5b/l71 0 2024-03-21T10:08:55.418 INFO:tasks.workunit.client.1.smithi174.stdout:2/426: creat d5/d25/d27/d2d/d45/d5b/d2f/d68/f96 x:0 0 0 2024-03-21T10:08:55.419 INFO:tasks.workunit.client.0.smithi153.stdout:1/489: truncate d6/d32/d3e/f63 1157322 0 2024-03-21T10:08:55.420 INFO:tasks.workunit.client.1.smithi174.stdout:0/423: dread d9/d35/d4f/d56/f66 [0,4194304] 0 2024-03-21T10:08:55.420 INFO:tasks.workunit.client.0.smithi153.stdout:0/524: dread - d4/d16/d19/d30/d6c/d98/d5e/f71 zero size 2024-03-21T10:08:55.421 INFO:tasks.workunit.client.1.smithi174.stdout:9/456: truncate d2/d11/f4a 5461200 0 2024-03-21T10:08:55.422 INFO:tasks.workunit.client.0.smithi153.stdout:2/686: mknod dd/d15/d18/d2c/d33/d98/dc7/de1/cee 0 2024-03-21T10:08:55.422 INFO:tasks.workunit.client.1.smithi174.stdout:1/432: symlink d4/d2d/l93 0 2024-03-21T10:08:55.424 INFO:tasks.workunit.client.0.smithi153.stdout:1/490: mknod d6/d10/c96 0 2024-03-21T10:08:55.426 INFO:tasks.workunit.client.1.smithi174.stdout:1/433: write d4/d67/f74 [598511,112740] 0 2024-03-21T10:08:55.427 INFO:tasks.workunit.client.0.smithi153.stdout:1/491: readlink d6/d3a/d3c/d50/l76 0 2024-03-21T10:08:55.427 INFO:tasks.workunit.client.0.smithi153.stdout:0/525: link d4/d16/d19/d30/d6c/f9b d4/db/fa0 0 2024-03-21T10:08:55.429 INFO:tasks.workunit.client.1.smithi174.stdout:3/412: dwrite d0/d12/d14/d1a/d44/d65/f6b [0,4194304] 0 2024-03-21T10:08:55.433 INFO:tasks.workunit.client.0.smithi153.stdout:1/492: stat d6/d10/d35/l6f 0 2024-03-21T10:08:55.434 INFO:tasks.workunit.client.0.smithi153.stdout:2/687: truncate dd/d15/d18/d2c/d4b/d56/d63/fdf 1047864 0 2024-03-21T10:08:55.435 INFO:tasks.workunit.client.0.smithi153.stdout:3/635: dwrite dd/d1e/d45/f52 [0,4194304] 0 2024-03-21T10:08:55.435 INFO:tasks.workunit.client.0.smithi153.stdout:1/493: dread - d6/f90 zero size 2024-03-21T10:08:55.444 INFO:tasks.workunit.client.1.smithi174.stdout:2/427: dwrite d5/d25/d27/f8b [0,4194304] 0 2024-03-21T10:08:55.444 INFO:tasks.workunit.client.1.smithi174.stdout:3/413: dread d0/f2 [4194304,4194304] 0 2024-03-21T10:08:55.445 INFO:tasks.workunit.client.0.smithi153.stdout:5/601: dwrite d4/d8/d5c/f73 [0,4194304] 0 2024-03-21T10:08:55.447 INFO:tasks.workunit.client.1.smithi174.stdout:2/428: dread - d5/d25/d27/d2d/d45/d5b/d2f/d68/f96 zero size 2024-03-21T10:08:55.450 INFO:tasks.workunit.client.1.smithi174.stdout:2/429: dread - d5/d25/d27/f80 zero size 2024-03-21T10:08:55.451 INFO:tasks.workunit.client.0.smithi153.stdout:0/526: getdents d4/d16/d27 0 2024-03-21T10:08:55.451 INFO:tasks.workunit.client.0.smithi153.stdout:6/770: sync 2024-03-21T10:08:55.451 INFO:tasks.workunit.client.0.smithi153.stdout:7/473: sync 2024-03-21T10:08:55.451 INFO:tasks.workunit.client.0.smithi153.stdout:4/613: sync 2024-03-21T10:08:55.451 INFO:tasks.workunit.client.0.smithi153.stdout:8/586: sync 2024-03-21T10:08:55.453 INFO:tasks.workunit.client.0.smithi153.stdout:4/614: chown d11/d18/d50 38553415 1 2024-03-21T10:08:55.455 INFO:tasks.workunit.client.0.smithi153.stdout:3/636: rename dd/d1e/d3d/d94/d73/d7f/d91 to dd/d1e/d2f/d35/d68/d75/ddd 0 2024-03-21T10:08:55.456 INFO:tasks.workunit.client.0.smithi153.stdout:4/615: readlink d11/d2d/l40 0 2024-03-21T10:08:55.457 INFO:tasks.workunit.client.1.smithi174.stdout:5/468: rename d1/d16/d2e/d66/d8b/d85/f5b to d1/d16/d1c/d81/fa6 0 2024-03-21T10:08:55.457 INFO:tasks.workunit.client.1.smithi174.stdout:7/366: dread d0/fa [0,4194304] 0 2024-03-21T10:08:55.457 INFO:tasks.workunit.client.0.smithi153.stdout:2/688: fsync dd/d15/f40 0 2024-03-21T10:08:55.458 INFO:tasks.workunit.client.0.smithi153.stdout:0/527: mkdir d4/d16/d19/d58/da1 0 2024-03-21T10:08:55.460 INFO:tasks.workunit.client.0.smithi153.stdout:7/474: mkdir d4/d1b/d20/d9a 0 2024-03-21T10:08:55.461 INFO:tasks.workunit.client.1.smithi174.stdout:1/434: dwrite d4/d7/d33/d43/d51/f64 [0,4194304] 0 2024-03-21T10:08:55.461 INFO:tasks.workunit.client.1.smithi174.stdout:7/367: chown d0/d4/d10/d1b/f3b 7961 1 2024-03-21T10:08:55.463 INFO:tasks.workunit.client.0.smithi153.stdout:5/602: creat d4/d5/df/d67/d6a/fc2 x:0 0 0 2024-03-21T10:08:55.467 INFO:tasks.workunit.client.0.smithi153.stdout:3/637: creat dd/d1e/d2f/d35/daf/dcc/d58/fde x:0 0 0 2024-03-21T10:08:55.470 INFO:tasks.workunit.client.0.smithi153.stdout:6/771: mknod d1f/de0/de1/df2/d10f/c112 0 2024-03-21T10:08:55.471 INFO:tasks.workunit.client.0.smithi153.stdout:5/603: dread - d4/d5/df/d5e/d69/f6d zero size 2024-03-21T10:08:55.471 INFO:tasks.workunit.client.0.smithi153.stdout:9/598: sync 2024-03-21T10:08:55.478 INFO:tasks.workunit.client.0.smithi153.stdout:2/689: creat dd/d15/d3c/da7/d91/dae/d50/d59/d5d/fef x:0 0 0 2024-03-21T10:08:55.479 INFO:tasks.workunit.client.0.smithi153.stdout:5/604: write d4/d5/df/d5e/d69/f7e [145295,80742] 0 2024-03-21T10:08:55.481 INFO:tasks.workunit.client.0.smithi153.stdout:0/528: creat d4/d16/d19/fa2 x:0 0 0 2024-03-21T10:08:55.484 INFO:tasks.workunit.client.0.smithi153.stdout:0/529: stat d4/d16/d19/d30/d6c/d98/l53 0 2024-03-21T10:08:55.485 INFO:tasks.workunit.client.1.smithi174.stdout:9/457: fsync d2/fc 0 2024-03-21T10:08:55.487 INFO:tasks.workunit.client.1.smithi174.stdout:0/424: mknod d9/d21/d5f/c8f 0 2024-03-21T10:08:55.487 INFO:tasks.workunit.client.0.smithi153.stdout:8/587: dread d5/d9/f7 [4194304,4194304] 0 2024-03-21T10:08:55.490 INFO:tasks.workunit.client.1.smithi174.stdout:4/460: sync 2024-03-21T10:08:55.493 INFO:tasks.workunit.client.0.smithi153.stdout:9/599: truncate d1/f1b 1061361 0 2024-03-21T10:08:55.493 INFO:tasks.workunit.client.0.smithi153.stdout:3/638: truncate dd/f62 178890 0 2024-03-21T10:08:55.497 INFO:tasks.workunit.client.0.smithi153.stdout:7/475: creat d4/d2a/d40/d50/f9b x:0 0 0 2024-03-21T10:08:55.498 INFO:tasks.workunit.client.0.smithi153.stdout:7/476: chown d4/l21 10994659 1 2024-03-21T10:08:55.498 INFO:tasks.workunit.client.0.smithi153.stdout:5/605: creat d4/d5/df/d67/d6a/fc3 x:0 0 0 2024-03-21T10:08:55.499 INFO:tasks.workunit.client.1.smithi174.stdout:6/321: truncate d0/d3/f26 1104564 0 2024-03-21T10:08:55.500 INFO:tasks.workunit.client.0.smithi153.stdout:1/494: write d6/d3a/d3c/d50/f67 [559774,936] 0 2024-03-21T10:08:55.500 INFO:tasks.workunit.client.0.smithi153.stdout:5/606: fsync d4/d8/d5c/f73 0 2024-03-21T10:08:55.502 INFO:tasks.workunit.client.0.smithi153.stdout:8/588: mkdir d5/d9/d43/dd/d20/d26/d24/d52/d56/dc2 0 2024-03-21T10:08:55.503 INFO:tasks.workunit.client.1.smithi174.stdout:5/469: creat d1/d2f/d37/fa7 x:0 0 0 2024-03-21T10:08:55.503 INFO:tasks.workunit.client.1.smithi174.stdout:6/322: write d0/d3/d5/f45 [937306,19587] 0 2024-03-21T10:08:55.515 INFO:tasks.workunit.client.1.smithi174.stdout:3/414: dwrite d0/d12/d14/d1a/d27/d4f/d5d/f31 [0,4194304] 0 2024-03-21T10:08:55.515 INFO:tasks.workunit.client.1.smithi174.stdout:9/458: truncate d2/f17 5157997 0 2024-03-21T10:08:55.518 INFO:tasks.workunit.client.1.smithi174.stdout:2/430: truncate d5/d25/d27/d2d/d45/f6e 237536 0 2024-03-21T10:08:55.519 INFO:tasks.workunit.client.1.smithi174.stdout:1/435: mknod d4/d7/d32/d7d/c94 0 2024-03-21T10:08:55.523 INFO:tasks.workunit.client.1.smithi174.stdout:7/368: link d0/d4/d10/d1b/f68 d0/d4/f7e 0 2024-03-21T10:08:55.533 INFO:tasks.workunit.client.1.smithi174.stdout:7/369: creat d0/d4/d10/d44/f7f x:0 0 0 2024-03-21T10:08:55.535 INFO:tasks.workunit.client.1.smithi174.stdout:3/415: dread d0/d12/d14/d1a/f34 [0,4194304] 0 2024-03-21T10:08:55.537 INFO:tasks.workunit.client.1.smithi174.stdout:7/370: creat d0/d53/f80 x:0 0 0 2024-03-21T10:08:55.537 INFO:tasks.workunit.client.1.smithi174.stdout:3/416: chown d0/d12/d14/d1a/d27/d4f/d5d/f78 248 1 2024-03-21T10:08:55.541 INFO:tasks.workunit.client.1.smithi174.stdout:7/371: mkdir d0/d4/d10/d44/d81 0 2024-03-21T10:08:55.542 INFO:tasks.workunit.client.1.smithi174.stdout:9/459: creat d2/d11/d1d/fa4 x:0 0 0 2024-03-21T10:08:55.546 INFO:tasks.workunit.client.1.smithi174.stdout:3/417: link d0/d12/f17 d0/d12/d14/d1a/d5c/f8e 0 2024-03-21T10:08:55.548 INFO:tasks.workunit.client.1.smithi174.stdout:3/418: chown d0/d12/d14/d1a/c1c 65783 1 2024-03-21T10:08:55.548 INFO:tasks.workunit.client.1.smithi174.stdout:8/348: mknod d9/d2a/c72 0 2024-03-21T10:08:55.548 INFO:tasks.workunit.client.1.smithi174.stdout:6/323: dread d0/d3/d20/f23 [0,4194304] 0 2024-03-21T10:08:55.550 INFO:tasks.workunit.client.1.smithi174.stdout:6/324: dread - d0/d3/d5/f4d zero size 2024-03-21T10:08:55.558 INFO:tasks.workunit.client.1.smithi174.stdout:6/325: link d0/d3/d20/d3d/f59 d0/d3/d5/d6/f6a 0 2024-03-21T10:08:55.561 INFO:tasks.workunit.client.1.smithi174.stdout:9/460: dwrite d2/d11/f2c [0,4194304] 0 2024-03-21T10:08:55.561 INFO:tasks.workunit.client.1.smithi174.stdout:6/326: chown d0/d3/d5/df/f22 9994 1 2024-03-21T10:08:55.563 INFO:tasks.workunit.client.1.smithi174.stdout:9/461: chown d2/d24/d2e/d63/f6c 208920569 1 2024-03-21T10:08:55.568 INFO:tasks.workunit.client.1.smithi174.stdout:6/327: mknod d0/d3/d20/d3d/c6b 0 2024-03-21T10:08:55.575 INFO:tasks.workunit.client.1.smithi174.stdout:4/461: dwrite d4/d19/f50 [0,4194304] 0 2024-03-21T10:08:55.577 INFO:tasks.workunit.client.1.smithi174.stdout:5/470: symlink d1/d16/la8 0 2024-03-21T10:08:55.580 INFO:tasks.workunit.client.1.smithi174.stdout:0/425: rename d9/c39 to d9/d21/c90 0 2024-03-21T10:08:55.583 INFO:tasks.workunit.client.1.smithi174.stdout:4/462: symlink d4/d21/d31/d6a/d8f/l96 0 2024-03-21T10:08:55.584 INFO:tasks.workunit.client.1.smithi174.stdout:5/471: fdatasync d1/d49/d4f/f57 0 2024-03-21T10:08:55.585 INFO:tasks.workunit.client.1.smithi174.stdout:4/463: chown d4/d21/d31/d3a/f8e 3805 1 2024-03-21T10:08:55.588 INFO:tasks.workunit.client.1.smithi174.stdout:1/436: rename d4/d7/d33/f3d to d4/d7/d32/d7d/f95 0 2024-03-21T10:08:55.590 INFO:tasks.workunit.client.1.smithi174.stdout:5/472: mknod d1/d49/d4f/d51/ca9 0 2024-03-21T10:08:55.590 INFO:tasks.workunit.client.0.smithi153.stdout:2/690: creat dd/d15/d18/d2c/d33/d98/dc7/ded/ff0 x:0 0 0 2024-03-21T10:08:55.590 INFO:tasks.workunit.client.0.smithi153.stdout:4/616: write d11/d18/f82 [438632,3187] 0 2024-03-21T10:08:55.591 INFO:tasks.workunit.client.1.smithi174.stdout:1/437: read d4/d7/d28/f3b [125401,25332] 0 2024-03-21T10:08:55.592 INFO:tasks.workunit.client.1.smithi174.stdout:5/473: read d1/f71 [21882,72771] 0 2024-03-21T10:08:55.593 INFO:tasks.workunit.client.1.smithi174.stdout:4/464: unlink d4/d21/d24/d28/d38/f7e 0 2024-03-21T10:08:55.593 INFO:tasks.workunit.client.1.smithi174.stdout:3/419: write d0/fa [2969642,35671] 0 2024-03-21T10:08:55.593 INFO:tasks.workunit.client.1.smithi174.stdout:1/438: readlink d4/d2d/l93 0 2024-03-21T10:08:55.595 INFO:tasks.workunit.client.1.smithi174.stdout:2/431: rename d5/d25/d27/d2d/d45/d5b/d2f/d4b/f7c to d5/d25/d27/d2d/d45/d5b/d2f/d4b/d76/f97 0 2024-03-21T10:08:55.596 INFO:tasks.workunit.client.0.smithi153.stdout:1/495: fsync d6/d3a/d3c/d50/d51/f5f 0 2024-03-21T10:08:55.597 INFO:tasks.workunit.client.0.smithi153.stdout:5/607: creat d4/d8/d3b/d56/da5/fc4 x:0 0 0 2024-03-21T10:08:55.598 INFO:tasks.workunit.client.0.smithi153.stdout:8/589: chown d5/d9/d43/dd/d20/d26/d24/d52/d55/lb0 301762 1 2024-03-21T10:08:55.599 INFO:tasks.workunit.client.1.smithi174.stdout:5/474: write d1/d16/f5f [4601319,56639] 0 2024-03-21T10:08:55.603 INFO:tasks.workunit.client.1.smithi174.stdout:7/372: rename d0/d4/d38/f58 to d0/d4/d10/d44/d81/f82 0 2024-03-21T10:08:55.607 INFO:tasks.workunit.client.0.smithi153.stdout:4/617: write d11/d2d/f73 [15707,31346] 0 2024-03-21T10:08:55.607 INFO:tasks.workunit.client.0.smithi153.stdout:9/600: dwrite d1/d4e/f53 [0,4194304] 0 2024-03-21T10:08:55.607 INFO:tasks.workunit.client.1.smithi174.stdout:3/420: creat d0/d12/d14/d1a/d27/d4f/f8f x:0 0 0 2024-03-21T10:08:55.608 INFO:tasks.workunit.client.0.smithi153.stdout:2/691: dread - dd/d15/d18/d2c/d4b/d56/d63/d8d/d9b/dc3/fd5 zero size 2024-03-21T10:08:55.609 INFO:tasks.workunit.client.0.smithi153.stdout:1/496: creat d6/d10/f97 x:0 0 0 2024-03-21T10:08:55.610 INFO:tasks.workunit.client.1.smithi174.stdout:2/432: creat d5/d25/d27/d2d/d45/d5b/d55/f98 x:0 0 0 2024-03-21T10:08:55.615 INFO:tasks.workunit.client.0.smithi153.stdout:3/639: dwrite dd/d1e/d2f/d35/d68/fc6 [0,4194304] 0 2024-03-21T10:08:55.617 INFO:tasks.workunit.client.0.smithi153.stdout:7/477: dread d4/d14/d2f/d78/f29 [0,4194304] 0 2024-03-21T10:08:55.618 INFO:tasks.workunit.client.1.smithi174.stdout:4/465: rmdir d4/d21/d31/d6a/d8f 39 2024-03-21T10:08:55.619 INFO:tasks.workunit.client.0.smithi153.stdout:9/601: truncate d1/d4/d5f/fc8 652108 0 2024-03-21T10:08:55.624 INFO:tasks.workunit.client.1.smithi174.stdout:6/328: dwrite d0/d3/d5/fe [0,4194304] 0 2024-03-21T10:08:55.627 INFO:tasks.workunit.client.0.smithi153.stdout:9/602: mknod d1/d43/d44/d46/d4d/d88/ccd 0 2024-03-21T10:08:55.627 INFO:tasks.workunit.client.1.smithi174.stdout:2/433: fsync d5/d25/d27/d2d/d45/d5b/d2f/d4b/d76/f86 0 2024-03-21T10:08:55.628 INFO:tasks.workunit.client.0.smithi153.stdout:3/640: mknod dd/d1e/d3d/d94/d73/dc2/cdf 0 2024-03-21T10:08:55.628 INFO:tasks.workunit.client.0.smithi153.stdout:4/618: link d11/d18/c3e d11/d71/cbc 0 2024-03-21T10:08:55.629 INFO:tasks.workunit.client.0.smithi153.stdout:7/478: symlink d4/l9c 0 2024-03-21T10:08:55.629 INFO:tasks.workunit.client.0.smithi153.stdout:8/590: dwrite d5/d9/d43/dd/d20/d26/d8b/f4e [0,4194304] 0 2024-03-21T10:08:55.633 INFO:tasks.workunit.client.0.smithi153.stdout:4/619: chown d11/d18/f41 105025843 1 2024-03-21T10:08:55.635 INFO:tasks.workunit.client.1.smithi174.stdout:7/373: dwrite d0/fa [0,4194304] 0 2024-03-21T10:08:55.636 INFO:tasks.workunit.client.0.smithi153.stdout:3/641: write dd/d1e/d2f/d35/daf/dcc/d58/f8b [1390962,77312] 0 2024-03-21T10:08:55.639 INFO:tasks.workunit.client.0.smithi153.stdout:9/603: rename d1/d43/d44/d46/d9e/d24/f38 to d1/d4/d5f/fce 0 2024-03-21T10:08:55.640 INFO:tasks.workunit.client.0.smithi153.stdout:1/497: dwrite d6/d3a/f53 [0,4194304] 0 2024-03-21T10:08:55.641 INFO:tasks.workunit.client.0.smithi153.stdout:5/608: truncate d4/d5/d16/f19 2817167 0 2024-03-21T10:08:55.644 INFO:tasks.workunit.client.0.smithi153.stdout:2/692: dwrite dd/d15/d5e/db3/fb9 [0,4194304] 0 2024-03-21T10:08:55.645 INFO:tasks.workunit.client.1.smithi174.stdout:9/462: rename d2/d24/d2e/l8a to d2/d11/d1d/d5e/la5 0 2024-03-21T10:08:55.645 INFO:tasks.workunit.client.1.smithi174.stdout:4/466: dread d4/d19/f50 [0,4194304] 0 2024-03-21T10:08:55.646 INFO:tasks.workunit.client.1.smithi174.stdout:3/421: truncate d0/d12/d14/d1a/d5c/f8e 2287195 0 2024-03-21T10:08:55.650 INFO:tasks.workunit.client.0.smithi153.stdout:8/591: symlink d5/d9/d43/dd/d20/d26/lc3 0 2024-03-21T10:08:55.650 INFO:tasks.workunit.client.0.smithi153.stdout:4/620: fsync d11/d2d/f42 0 2024-03-21T10:08:55.651 INFO:tasks.workunit.client.1.smithi174.stdout:1/439: dwrite d4/d7/f2c [0,4194304] 0 2024-03-21T10:08:55.654 INFO:tasks.workunit.client.1.smithi174.stdout:3/422: dread - d0/d12/d14/d1a/d27/d4f/d5d/f78 zero size 2024-03-21T10:08:55.656 INFO:tasks.workunit.client.0.smithi153.stdout:3/642: rename dd/d1e/d3d/cda to dd/d41/ce0 0 2024-03-21T10:08:55.658 INFO:tasks.workunit.client.1.smithi174.stdout:3/423: dread - d0/d12/d14/f80 zero size 2024-03-21T10:08:55.658 INFO:tasks.workunit.client.1.smithi174.stdout:6/329: creat d0/d3/d5/d21/f6c x:0 0 0 2024-03-21T10:08:55.662 INFO:tasks.workunit.client.0.smithi153.stdout:7/479: dread d4/d2a/f3e [0,4194304] 0 2024-03-21T10:08:55.662 INFO:tasks.workunit.client.0.smithi153.stdout:9/604: symlink d1/d43/d72/dba/lcf 0 2024-03-21T10:08:55.664 INFO:tasks.workunit.client.1.smithi174.stdout:0/426: rename d9/d35/d4a/f53 to d9/d21/d23/f91 0 2024-03-21T10:08:55.665 INFO:tasks.workunit.client.0.smithi153.stdout:1/498: creat d6/d10/f98 x:0 0 0 2024-03-21T10:08:55.666 INFO:tasks.workunit.client.1.smithi174.stdout:0/427: readlink l8 0 2024-03-21T10:08:55.669 INFO:tasks.workunit.client.1.smithi174.stdout:9/463: unlink d2/d24/d2e/l50 0 2024-03-21T10:08:55.670 INFO:tasks.workunit.client.1.smithi174.stdout:5/475: write d1/f2c [2353511,66590] 0 2024-03-21T10:08:55.671 INFO:tasks.workunit.client.1.smithi174.stdout:9/464: write d2/d24/d2e/d7e/f8c [91030,77717] 0 2024-03-21T10:08:55.673 INFO:tasks.workunit.client.1.smithi174.stdout:3/424: symlink d0/d12/d14/l90 0 2024-03-21T10:08:55.675 INFO:tasks.workunit.client.1.smithi174.stdout:7/374: creat d0/d4/d10/d79/f83 x:0 0 0 2024-03-21T10:08:55.678 INFO:tasks.workunit.client.0.smithi153.stdout:3/643: dwrite dd/f25 [0,4194304] 0 2024-03-21T10:08:55.682 INFO:tasks.workunit.client.0.smithi153.stdout:1/499: rmdir d6/d32 39 2024-03-21T10:08:55.682 INFO:tasks.workunit.client.1.smithi174.stdout:1/440: mknod d4/d7/d15/d53/d78/c96 0 2024-03-21T10:08:55.683 INFO:tasks.workunit.client.0.smithi153.stdout:2/693: dwrite dd/d15/d18/d2c/d4b/d56/d63/fdf [0,4194304] 0 2024-03-21T10:08:55.689 INFO:tasks.workunit.client.0.smithi153.stdout:5/609: write d4/f46 [3478288,7015] 0 2024-03-21T10:08:55.690 INFO:tasks.workunit.client.0.smithi153.stdout:8/592: getdents d5/d9/d43/dd/d20/d26/d8b/d80 0 2024-03-21T10:08:55.691 INFO:tasks.workunit.client.1.smithi174.stdout:2/434: write d5/f14 [6025290,10635] 0 2024-03-21T10:08:55.695 INFO:tasks.workunit.client.0.smithi153.stdout:3/644: symlink dd/d1e/d2f/d35/daf/dcc/dd6/le1 0 2024-03-21T10:08:55.699 INFO:tasks.workunit.client.0.smithi153.stdout:9/605: link d1/f1b d1/d43/d72/dba/fd0 0 2024-03-21T10:08:55.702 INFO:tasks.workunit.client.0.smithi153.stdout:3/645: mkdir dd/d1e/d3d/d94/de2 0 2024-03-21T10:08:55.702 INFO:tasks.workunit.client.0.smithi153.stdout:3/646: readlink l9 0 2024-03-21T10:08:55.705 INFO:tasks.workunit.client.0.smithi153.stdout:8/593: rename d5/d9/d43/dd/d20/d26/d24/d52/d56 to d5/d9/d43/d7b/dc4 0 2024-03-21T10:08:55.716 INFO:tasks.workunit.client.1.smithi174.stdout:4/467: truncate d4/d19/f50 1100058 0 2024-03-21T10:08:55.716 INFO:tasks.workunit.client.0.smithi153.stdout:7/480: truncate d4/d2a/f5b 1843558 0 2024-03-21T10:08:55.717 INFO:tasks.workunit.client.0.smithi153.stdout:3/647: creat dd/d1e/d3d/d94/d8e/fe3 x:0 0 0 2024-03-21T10:08:55.719 INFO:tasks.workunit.client.0.smithi153.stdout:9/606: rename d1/d43/d44/d46/d9e/d24/l7c to d1/d43/d44/d46/d9e/d2e/d67/ld1 0 2024-03-21T10:08:55.720 INFO:tasks.workunit.client.1.smithi174.stdout:1/441: creat d4/d7/d28/f97 x:0 0 0 2024-03-21T10:08:55.721 INFO:tasks.workunit.client.1.smithi174.stdout:9/465: getdents d2/d11/d1d 0 2024-03-21T10:08:55.721 INFO:tasks.workunit.client.1.smithi174.stdout:7/375: getdents d0/d53/d5f/d65 0 2024-03-21T10:08:55.722 INFO:tasks.workunit.client.0.smithi153.stdout:1/500: write d6/d10/f73 [390452,24639] 0 2024-03-21T10:08:55.722 INFO:tasks.workunit.client.0.smithi153.stdout:7/481: readlink d4/d11/d56/l92 0 2024-03-21T10:08:55.722 INFO:tasks.workunit.client.1.smithi174.stdout:2/435: fsync d5/d25/d27/d2d/d45/d5b/f1c 0 2024-03-21T10:08:55.723 INFO:tasks.workunit.client.0.smithi153.stdout:2/694: write dd/d15/f71 [953341,46463] 0 2024-03-21T10:08:55.723 INFO:tasks.workunit.client.1.smithi174.stdout:4/468: unlink d4/d21/d24/f73 0 2024-03-21T10:08:55.726 INFO:tasks.workunit.client.1.smithi174.stdout:3/425: read - d0/d12/d14/d1a/d27/d4f/f8f zero size 2024-03-21T10:08:55.727 INFO:tasks.workunit.client.0.smithi153.stdout:8/594: mknod d5/d9/d43/dd/d20/d26/d24/d52/cc5 0 2024-03-21T10:08:55.728 INFO:tasks.workunit.client.0.smithi153.stdout:2/695: dread - dd/d15/d5e/fbb zero size 2024-03-21T10:08:55.730 INFO:tasks.workunit.client.1.smithi174.stdout:5/476: dwrite d1/d16/f40 [4194304,4194304] 0 2024-03-21T10:08:55.730 INFO:tasks.workunit.client.1.smithi174.stdout:1/442: mkdir d4/d7/d28/d59/d98 0 2024-03-21T10:08:55.732 INFO:tasks.workunit.client.0.smithi153.stdout:4/621: dread d11/d2d/f5a [0,4194304] 0 2024-03-21T10:08:55.732 INFO:tasks.workunit.client.1.smithi174.stdout:2/436: creat d5/d25/d27/d2e/d7b/f99 x:0 0 0 2024-03-21T10:08:55.733 INFO:tasks.workunit.client.1.smithi174.stdout:7/376: write d0/d4/d10/f62 [761448,52911] 0 2024-03-21T10:08:55.737 INFO:tasks.workunit.client.1.smithi174.stdout:7/377: read - d0/d4/d38/f70 zero size 2024-03-21T10:08:55.739 INFO:tasks.workunit.client.0.smithi153.stdout:0/530: sync 2024-03-21T10:08:55.739 INFO:tasks.workunit.client.0.smithi153.stdout:6/772: sync 2024-03-21T10:08:55.739 INFO:tasks.workunit.client.1.smithi174.stdout:7/378: stat d0/d4/d10/d1b/f2e 0 2024-03-21T10:08:55.739 INFO:tasks.workunit.client.1.smithi174.stdout:4/469: rename d4/d21/d31/f48 to d4/d19/f97 0 2024-03-21T10:08:55.743 INFO:tasks.workunit.client.0.smithi153.stdout:3/648: dwrite dd/d41/f71 [0,4194304] 0 2024-03-21T10:08:55.744 INFO:tasks.workunit.client.1.smithi174.stdout:4/470: write d4/d19/f41 [1428120,130586] 0 2024-03-21T10:08:55.746 INFO:tasks.workunit.client.0.smithi153.stdout:3/649: write dd/f2b [958745,30502] 0 2024-03-21T10:08:55.747 INFO:tasks.workunit.client.1.smithi174.stdout:1/443: fdatasync d4/d5/f2a 0 2024-03-21T10:08:55.748 INFO:tasks.workunit.client.1.smithi174.stdout:5/477: symlink d1/d2f/d37/d9b/laa 0 2024-03-21T10:08:55.750 INFO:tasks.workunit.client.1.smithi174.stdout:3/426: dread d0/d12/d14/d1a/d27/d3c/d73/f8a [0,4194304] 0 2024-03-21T10:08:55.754 INFO:tasks.workunit.client.1.smithi174.stdout:4/471: creat d4/d19/f98 x:0 0 0 2024-03-21T10:08:55.759 INFO:tasks.workunit.client.1.smithi174.stdout:2/437: creat d5/d25/d27/d2e/d85/f9a x:0 0 0 2024-03-21T10:08:55.759 INFO:tasks.workunit.client.1.smithi174.stdout:9/466: getdents d2/d11/d1d/d8b 0 2024-03-21T10:08:55.760 INFO:tasks.workunit.client.0.smithi153.stdout:2/696: dwrite dd/d15/d18/d2c/d4b/d56/d63/f80 [0,4194304] 0 2024-03-21T10:08:55.761 INFO:tasks.workunit.client.1.smithi174.stdout:9/467: dread - d2/d11/d1d/f79 zero size 2024-03-21T10:08:55.761 INFO:tasks.workunit.client.1.smithi174.stdout:6/330: dread d0/d3/d5/d6/f38 [0,4194304] 0 2024-03-21T10:08:55.764 INFO:tasks.workunit.client.0.smithi153.stdout:2/697: chown dd/d15/d18/d2c/d4b/d56/la2 12195 1 2024-03-21T10:08:55.765 INFO:tasks.workunit.client.1.smithi174.stdout:4/472: creat d4/d6c/f99 x:0 0 0 2024-03-21T10:08:55.767 INFO:tasks.workunit.client.1.smithi174.stdout:6/331: chown d0/d3/d5/d3b/f61 13 1 2024-03-21T10:08:55.770 INFO:tasks.workunit.client.1.smithi174.stdout:4/473: chown d4/d21/d24/d28/f8a 0 1 2024-03-21T10:08:55.771 INFO:tasks.workunit.client.0.smithi153.stdout:2/698: read - dd/d15/d3c/da7/d91/dae/d50/d59/fe9 zero size 2024-03-21T10:08:55.773 INFO:tasks.workunit.client.1.smithi174.stdout:8/349: sync 2024-03-21T10:08:55.774 INFO:tasks.workunit.client.1.smithi174.stdout:1/444: link d4/d7/d15/f2f d4/d67/f99 0 2024-03-21T10:08:55.774 INFO:tasks.workunit.client.0.smithi153.stdout:2/699: read dd/d15/d18/d3a/d3d/f76 [192,112772] 0 2024-03-21T10:08:55.777 INFO:tasks.workunit.client.1.smithi174.stdout:9/468: rename d2/d11/d60/f6d to d2/d11/d1d/d8b/fa6 0 2024-03-21T10:08:55.778 INFO:tasks.workunit.client.1.smithi174.stdout:3/427: creat d0/f91 x:0 0 0 2024-03-21T10:08:55.781 INFO:tasks.workunit.client.1.smithi174.stdout:6/332: stat d0/d3/d5/d6/l11 0 2024-03-21T10:08:55.789 INFO:tasks.workunit.client.1.smithi174.stdout:4/474: fsync d4/d19/f7c 0 2024-03-21T10:08:55.789 INFO:tasks.workunit.client.1.smithi174.stdout:2/438: dwrite d5/d25/d27/d2e/d4f/d65/f92 [0,4194304] 0 2024-03-21T10:08:55.790 INFO:tasks.workunit.client.1.smithi174.stdout:8/350: mknod d9/d17/d19/d27/c73 0 2024-03-21T10:08:55.802 INFO:tasks.workunit.client.1.smithi174.stdout:6/333: mkdir d0/d3/d20/d6d 0 2024-03-21T10:08:55.803 INFO:tasks.workunit.client.1.smithi174.stdout:7/379: write d0/d4/d10/d1b/f68 [1160371,15227] 0 2024-03-21T10:08:55.805 INFO:tasks.workunit.client.1.smithi174.stdout:1/445: dwrite d4/d7/d15/f1f [4194304,4194304] 0 2024-03-21T10:08:55.809 INFO:tasks.workunit.client.1.smithi174.stdout:8/351: dread d9/d2a/d39/f5f [0,4194304] 0 2024-03-21T10:08:55.813 INFO:tasks.workunit.client.0.smithi153.stdout:2/700: chown dd/d15/d3c/da7/d91/dae/d50/d59/d5d/f7f 876258426 1 2024-03-21T10:08:55.814 INFO:tasks.workunit.client.1.smithi174.stdout:9/469: mkdir d2/d11/d1d/d2a/da7 0 2024-03-21T10:08:55.817 INFO:tasks.workunit.client.1.smithi174.stdout:6/334: unlink d0/d3/l7 0 2024-03-21T10:08:55.820 INFO:tasks.workunit.client.1.smithi174.stdout:6/335: truncate d0/d3/d5/f5c 962103 0 2024-03-21T10:08:55.820 INFO:tasks.workunit.client.1.smithi174.stdout:4/475: write d4/d19/f74 [38686,94917] 0 2024-03-21T10:08:55.831 INFO:tasks.workunit.client.0.smithi153.stdout:0/531: creat d4/d16/d19/d58/d9a/fa3 x:0 0 0 2024-03-21T10:08:55.832 INFO:tasks.workunit.client.0.smithi153.stdout:3/650: symlink dd/d1e/le4 0 2024-03-21T10:08:55.833 INFO:tasks.workunit.client.0.smithi153.stdout:8/595: link d5/d1b/f59 d5/d9/d43/d7b/dc4/da7/fc6 0 2024-03-21T10:08:55.839 INFO:tasks.workunit.client.0.smithi153.stdout:7/482: getdents d4/d19 0 2024-03-21T10:08:55.849 INFO:tasks.workunit.client.0.smithi153.stdout:3/651: unlink dd/d1e/l4b 0 2024-03-21T10:08:55.852 INFO:tasks.workunit.client.0.smithi153.stdout:2/701: dread dd/d15/d18/d3a/d8c/fb2 [0,4194304] 0 2024-03-21T10:08:55.865 INFO:tasks.workunit.client.0.smithi153.stdout:3/652: getdents dd/d1e/d2f/d35/daf/dcc/dd6 0 2024-03-21T10:08:55.868 INFO:tasks.workunit.client.0.smithi153.stdout:7/483: symlink d4/d2a/d3b/l9d 0 2024-03-21T10:08:55.871 INFO:tasks.workunit.client.1.smithi174.stdout:8/352: unlink d9/d17/d19/d27/l2c 0 2024-03-21T10:08:55.871 INFO:tasks.workunit.client.1.smithi174.stdout:1/446: creat d4/d7/d33/d43/d77/f9a x:0 0 0 2024-03-21T10:08:55.871 INFO:tasks.workunit.client.1.smithi174.stdout:2/439: symlink d5/d25/d27/d2e/d90/l9b 0 2024-03-21T10:08:55.872 INFO:tasks.workunit.client.1.smithi174.stdout:4/476: mknod d4/d6c/c9a 0 2024-03-21T10:08:55.873 INFO:tasks.workunit.client.1.smithi174.stdout:6/336: mknod d0/d3/d5/d6/d42/c6e 0 2024-03-21T10:08:55.873 INFO:tasks.workunit.client.1.smithi174.stdout:1/447: readlink d4/d7/l14 0 2024-03-21T10:08:55.876 INFO:tasks.workunit.client.1.smithi174.stdout:4/477: write d4/d21/f27 [2992893,62547] 0 2024-03-21T10:08:55.876 INFO:tasks.workunit.client.1.smithi174.stdout:6/337: stat d0/d3/d5/d3b/l62 0 2024-03-21T10:08:55.882 INFO:tasks.workunit.client.1.smithi174.stdout:3/428: rename d0/d12/d14/d4c/d7a/c8c to d0/d12/d14/d1a/d44/c92 0 2024-03-21T10:08:55.885 INFO:tasks.workunit.client.1.smithi174.stdout:9/470: link d2/d24/f59 d2/d11/d1d/d4b/fa8 0 2024-03-21T10:08:55.886 INFO:tasks.workunit.client.1.smithi174.stdout:8/353: mknod d9/d53/d5b/c74 0 2024-03-21T10:08:55.887 INFO:tasks.workunit.client.1.smithi174.stdout:3/429: read d0/d12/d14/d1a/d27/d4f/d5d/f1b [1665444,30198] 0 2024-03-21T10:08:55.887 INFO:tasks.workunit.client.1.smithi174.stdout:1/448: creat d4/d7/d15/f9b x:0 0 0 2024-03-21T10:08:55.888 INFO:tasks.workunit.client.0.smithi153.stdout:9/607: truncate d1/d4/d5f/fce 2389395 0 2024-03-21T10:08:55.888 INFO:tasks.workunit.client.1.smithi174.stdout:2/440: mkdir d5/d25/d27/d2d/d45/d5b/d55/d9c 0 2024-03-21T10:08:55.889 INFO:tasks.workunit.client.1.smithi174.stdout:6/338: mknod d0/d3/d2a/d56/d5e/c6f 0 2024-03-21T10:08:55.890 INFO:tasks.workunit.client.0.smithi153.stdout:6/773: truncate d1f/fbf 262332 0 2024-03-21T10:08:55.890 INFO:tasks.workunit.client.1.smithi174.stdout:4/478: dread d4/d21/d24/d28/f5e [0,4194304] 0 2024-03-21T10:08:55.891 INFO:tasks.workunit.client.0.smithi153.stdout:4/622: rename d11/d2d/f43 to d11/d9d/fbd 0 2024-03-21T10:08:55.892 INFO:tasks.workunit.client.1.smithi174.stdout:9/471: dread d2/f8e [0,4194304] 0 2024-03-21T10:08:55.892 INFO:tasks.workunit.client.0.smithi153.stdout:9/608: write d1/f97 [1571768,80377] 0 2024-03-21T10:08:55.892 INFO:tasks.workunit.client.1.smithi174.stdout:2/441: write d5/d25/d27/f2a [4609242,92098] 0 2024-03-21T10:08:55.893 INFO:tasks.workunit.client.1.smithi174.stdout:8/354: creat d9/d2a/d39/f75 x:0 0 0 2024-03-21T10:08:55.894 INFO:tasks.workunit.client.0.smithi153.stdout:4/623: dread - d11/db1/fbb zero size 2024-03-21T10:08:55.896 INFO:tasks.workunit.client.1.smithi174.stdout:6/339: read d0/d3/d5/df/d16/f27 [1901552,115475] 0 2024-03-21T10:08:55.897 INFO:tasks.workunit.client.1.smithi174.stdout:1/449: truncate d4/d7/f21 367590 0 2024-03-21T10:08:55.898 INFO:tasks.workunit.client.1.smithi174.stdout:2/442: dread d5/d25/d27/f3c [0,4194304] 0 2024-03-21T10:08:55.899 INFO:tasks.workunit.client.1.smithi174.stdout:3/430: rename d0/d12/d14/d1a/d27/d4f/d5d to d0/d12/d14/d1a/d44/d57/d93 0 2024-03-21T10:08:55.901 INFO:tasks.workunit.client.0.smithi153.stdout:8/596: write d5/d9/d43/fa0 [2757710,102066] 0 2024-03-21T10:08:55.902 INFO:tasks.workunit.client.1.smithi174.stdout:9/472: unlink d2/d11/d7d/f8d 0 2024-03-21T10:08:55.903 INFO:tasks.workunit.client.0.smithi153.stdout:9/609: mknod d1/d3b/cd2 0 2024-03-21T10:08:55.904 INFO:tasks.workunit.client.1.smithi174.stdout:9/473: fdatasync d2/f47 0 2024-03-21T10:08:55.907 INFO:tasks.workunit.client.0.smithi153.stdout:1/501: dwrite d6/d10/d2c/f2e [0,4194304] 0 2024-03-21T10:08:55.909 INFO:tasks.workunit.client.0.smithi153.stdout:7/484: rename d4/d19/f31 to d4/d14/d2f/f9e 0 2024-03-21T10:08:55.910 INFO:tasks.workunit.client.0.smithi153.stdout:5/610: dwrite d4/d5/d16/f19 [0,4194304] 0 2024-03-21T10:08:55.910 INFO:tasks.workunit.client.0.smithi153.stdout:8/597: write d5/d9/d43/d7b/daa/fb5 [1217958,77491] 0 2024-03-21T10:08:55.910 INFO:tasks.workunit.client.0.smithi153.stdout:4/624: mknod d11/d18/cbe 0 2024-03-21T10:08:55.913 INFO:tasks.workunit.client.1.smithi174.stdout:1/450: unlink d4/d7/d32/d7d/f95 0 2024-03-21T10:08:55.915 INFO:tasks.workunit.client.0.smithi153.stdout:4/625: dread - d11/d2d/f9e zero size 2024-03-21T10:08:55.917 INFO:tasks.workunit.client.1.smithi174.stdout:8/355: rename d9/d17 to d9/d2a/d39/d76 0 2024-03-21T10:08:55.920 INFO:tasks.workunit.client.0.smithi153.stdout:1/502: chown d6/f8c 0 1 2024-03-21T10:08:55.921 INFO:tasks.workunit.client.1.smithi174.stdout:3/431: mkdir d0/d12/d14/d1a/d44/d94 0 2024-03-21T10:08:55.922 INFO:tasks.workunit.client.0.smithi153.stdout:9/610: write d1/d43/d44/d46/d9e/d2e/f76 [4393488,804] 0 2024-03-21T10:08:55.922 INFO:tasks.workunit.client.0.smithi153.stdout:0/532: dwrite d4/f23 [0,4194304] 0 2024-03-21T10:08:55.927 INFO:tasks.workunit.client.0.smithi153.stdout:6/774: write d1f/f6c [708957,9550] 0 2024-03-21T10:08:55.928 INFO:tasks.workunit.client.1.smithi174.stdout:2/443: dwrite d5/d25/d27/d2d/d45/d5b/d55/f98 [0,4194304] 0 2024-03-21T10:08:55.933 INFO:tasks.workunit.client.1.smithi174.stdout:0/428: sync 2024-03-21T10:08:55.933 INFO:tasks.workunit.client.0.smithi153.stdout:5/611: creat d4/d5/df/d67/d6a/fc5 x:0 0 0 2024-03-21T10:08:55.934 INFO:tasks.workunit.client.0.smithi153.stdout:1/503: write d6/d10/d2c/f33 [3633498,102648] 0 2024-03-21T10:08:55.938 INFO:tasks.workunit.client.1.smithi174.stdout:6/340: dread d0/d3/d5/df/d16/f1a [0,4194304] 0 2024-03-21T10:08:55.943 INFO:tasks.workunit.client.0.smithi153.stdout:2/702: dwrite dd/d15/d3c/da7/d91/dae/f77 [0,4194304] 0 2024-03-21T10:08:55.943 INFO:tasks.workunit.client.0.smithi153.stdout:8/598: creat d5/d9/d43/d7b/d7c/fc7 x:0 0 0 2024-03-21T10:08:55.947 INFO:tasks.workunit.client.1.smithi174.stdout:0/429: mkdir d9/d21/d31/d5c/d92 0 2024-03-21T10:08:55.949 INFO:tasks.workunit.client.1.smithi174.stdout:3/432: getdents d0/d12/d14/d1a/d27/d7f 0 2024-03-21T10:08:55.952 INFO:tasks.workunit.client.0.smithi153.stdout:1/504: truncate d6/d13/f7c 96255 0 2024-03-21T10:08:55.952 INFO:tasks.workunit.client.1.smithi174.stdout:8/356: dwrite d9/d2a/d39/d76/d19/d31/f3b [0,4194304] 0 2024-03-21T10:08:55.953 INFO:tasks.workunit.client.0.smithi153.stdout:3/653: dwrite dd/d1e/d2f/f36 [0,4194304] 0 2024-03-21T10:08:55.953 INFO:tasks.workunit.client.1.smithi174.stdout:9/474: rename d2/d54/f9c to d2/d11/d1d/d2a/fa9 0 2024-03-21T10:08:55.954 INFO:tasks.workunit.client.0.smithi153.stdout:7/485: getdents d4/d11 0 2024-03-21T10:08:55.956 INFO:tasks.workunit.client.1.smithi174.stdout:5/478: sync 2024-03-21T10:08:55.962 INFO:tasks.workunit.client.1.smithi174.stdout:5/479: dread d1/d16/f60 [0,4194304] 0 2024-03-21T10:08:55.964 INFO:tasks.workunit.client.1.smithi174.stdout:2/444: rename d5/f38 to d5/d25/d27/d2d/d45/d5b/d2f/f9d 0 2024-03-21T10:08:55.964 INFO:tasks.workunit.client.0.smithi153.stdout:4/626: write d11/f19 [735770,102429] 0 2024-03-21T10:08:55.970 INFO:tasks.workunit.client.0.smithi153.stdout:2/703: rename c9 to dd/d15/d5e/db3/cf1 0 2024-03-21T10:08:55.976 INFO:tasks.workunit.client.1.smithi174.stdout:5/480: mknod d1/d16/d2e/d8d/cab 0 2024-03-21T10:08:55.979 INFO:tasks.workunit.client.0.smithi153.stdout:0/533: dread d4/f28 [0,4194304] 0 2024-03-21T10:08:55.979 INFO:tasks.workunit.client.1.smithi174.stdout:2/445: rmdir d5/d25/d27/d2d/d45/d5b/d2f 39 2024-03-21T10:08:55.980 INFO:tasks.workunit.client.1.smithi174.stdout:3/433: truncate d0/d12/f17 2317868 0 2024-03-21T10:08:55.981 INFO:tasks.workunit.client.1.smithi174.stdout:4/479: write d4/d21/d31/d3a/f95 [2941655,84507] 0 2024-03-21T10:08:55.982 INFO:tasks.workunit.client.0.smithi153.stdout:7/486: creat d4/d2a/d40/d50/f9f x:0 0 0 2024-03-21T10:08:55.983 INFO:tasks.workunit.client.1.smithi174.stdout:5/481: chown d1/d2f/d37/d9b/fa3 226719 1 2024-03-21T10:08:55.983 INFO:tasks.workunit.client.1.smithi174.stdout:9/475: dwrite d2/d11/d1d/f41 [0,4194304] 0 2024-03-21T10:08:55.983 INFO:tasks.workunit.client.0.smithi153.stdout:7/487: chown d4/d19 14 1 2024-03-21T10:08:55.983 INFO:tasks.workunit.client.0.smithi153.stdout:5/612: link d4/d8/f2d d4/d5/de/d44/d48/fc6 0 2024-03-21T10:08:55.985 INFO:tasks.workunit.client.1.smithi174.stdout:0/430: rename d9/c1b to d9/d35/d4a/d8a/c93 0 2024-03-21T10:08:55.986 INFO:tasks.workunit.client.1.smithi174.stdout:0/431: stat d9/d35/l71 0 2024-03-21T10:08:55.987 INFO:tasks.workunit.client.0.smithi153.stdout:9/611: rename d1/d43/d44/l77 to d1/d43/d44/d46/d9e/d4b/ld3 0 2024-03-21T10:08:55.989 INFO:tasks.workunit.client.0.smithi153.stdout:1/505: fdatasync d6/d13/d75/f57 0 2024-03-21T10:08:55.990 INFO:tasks.workunit.client.0.smithi153.stdout:9/612: chown d1/d43/d44/dc3 26 1 2024-03-21T10:08:55.996 INFO:tasks.workunit.client.0.smithi153.stdout:4/627: creat d11/d2d/da3/dac/fbf x:0 0 0 2024-03-21T10:08:55.997 INFO:tasks.workunit.client.0.smithi153.stdout:9/613: read d1/d43/d44/d46/f37 [1812083,72660] 0 2024-03-21T10:08:55.997 INFO:tasks.workunit.client.1.smithi174.stdout:9/476: mknod d2/d11/d1d/d4b/caa 0 2024-03-21T10:08:55.998 INFO:tasks.workunit.client.0.smithi153.stdout:1/506: dread d6/d3a/d3c/d50/d51/f6d [0,4194304] 0 2024-03-21T10:08:55.999 INFO:tasks.workunit.client.1.smithi174.stdout:0/432: mknod d9/d1c/d76/c94 0 2024-03-21T10:08:56.000 INFO:tasks.workunit.client.0.smithi153.stdout:5/613: mkdir d4/d5/de/d44/dc7 0 2024-03-21T10:08:56.001 INFO:tasks.workunit.client.0.smithi153.stdout:7/488: rename d4/d1b/d83/f8f to d4/d14/fa0 0 2024-03-21T10:08:56.001 INFO:tasks.workunit.client.0.smithi153.stdout:4/628: fdatasync d11/d18/f95 0 2024-03-21T10:08:56.003 INFO:tasks.workunit.client.0.smithi153.stdout:6/775: link d1f/d93/dd8/f47 d1f/d2b/d10a/d40/d51/dbd/f113 0 2024-03-21T10:08:56.005 INFO:tasks.workunit.client.1.smithi174.stdout:2/446: truncate d5/d25/d27/d2d/d45/d5b/d63/f48 1072932 0 2024-03-21T10:08:56.006 INFO:tasks.workunit.client.0.smithi153.stdout:9/614: readlink d1/d43/d44/d46/d9e/l4c 0 2024-03-21T10:08:56.007 INFO:tasks.workunit.client.0.smithi153.stdout:7/489: read d4/d14/d2f/d78/f3a [3913289,17247] 0 2024-03-21T10:08:56.008 INFO:tasks.workunit.client.1.smithi174.stdout:9/477: mkdir d2/d11/d1d/dab 0 2024-03-21T10:08:56.009 INFO:tasks.workunit.client.0.smithi153.stdout:6/776: fdatasync d1f/d2b/d10a/d40/f9e 0 2024-03-21T10:08:56.011 INFO:tasks.workunit.client.1.smithi174.stdout:1/451: dwrite d4/d7/d15/d53/f54 [0,4194304] 0 2024-03-21T10:08:56.011 INFO:tasks.workunit.client.1.smithi174.stdout:8/357: write d9/d2a/d39/d76/d19/f40 [342750,110955] 0 2024-03-21T10:08:56.012 INFO:tasks.workunit.client.0.smithi153.stdout:9/615: truncate d1/d3b/fc2 372555 0 2024-03-21T10:08:56.013 INFO:tasks.workunit.client.1.smithi174.stdout:3/434: dwrite d0/fa [0,4194304] 0 2024-03-21T10:08:56.014 INFO:tasks.workunit.client.0.smithi153.stdout:3/654: write f2 [1242956,31839] 0 2024-03-21T10:08:56.024 INFO:tasks.workunit.client.1.smithi174.stdout:1/452: creat d4/d7/d33/d43/d51/f9c x:0 0 0 2024-03-21T10:08:56.026 INFO:tasks.workunit.client.0.smithi153.stdout:7/490: mkdir d4/d11/da1 0 2024-03-21T10:08:56.029 INFO:tasks.workunit.client.0.smithi153.stdout:6/777: stat d1f/d2b/d10a/d40/d4f/f83 0 2024-03-21T10:08:56.032 INFO:tasks.workunit.client.1.smithi174.stdout:3/435: getdents d0/d12/d14/d67 0 2024-03-21T10:08:56.032 INFO:tasks.workunit.client.0.smithi153.stdout:5/614: dread d4/d5/de/d44/d48/f87 [0,4194304] 0 2024-03-21T10:08:56.033 INFO:tasks.workunit.client.1.smithi174.stdout:0/433: dwrite d9/d21/d31/f42 [0,4194304] 0 2024-03-21T10:08:56.034 INFO:tasks.workunit.client.0.smithi153.stdout:4/629: rename d11/d18/d7e/l86 to d11/d2d/d3d/lc0 0 2024-03-21T10:08:56.035 INFO:tasks.workunit.client.1.smithi174.stdout:2/447: dread d5/d25/d27/d2d/d45/d5b/d63/f59 [0,4194304] 0 2024-03-21T10:08:56.038 INFO:tasks.workunit.client.0.smithi153.stdout:7/491: creat d4/d14/d73/fa2 x:0 0 0 2024-03-21T10:08:56.038 INFO:tasks.workunit.client.0.smithi153.stdout:6/778: truncate d1f/d2b/d10a/d40/f44 685017 0 2024-03-21T10:08:56.039 INFO:tasks.workunit.client.0.smithi153.stdout:3/655: creat dd/d1e/d2f/d35/d68/d75/ddd/fe5 x:0 0 0 2024-03-21T10:08:56.040 INFO:tasks.workunit.client.1.smithi174.stdout:1/453: rename d4/d7 to d4/d7/d68/d7b/d8b/d9d 22 2024-03-21T10:08:56.043 INFO:tasks.workunit.client.1.smithi174.stdout:4/480: dwrite d4/d21/d24/d28/f39 [0,4194304] 0 2024-03-21T10:08:56.044 INFO:tasks.workunit.client.1.smithi174.stdout:3/436: creat d0/d12/d46/d59/f95 x:0 0 0 2024-03-21T10:08:56.045 INFO:tasks.workunit.client.0.smithi153.stdout:3/656: chown dd/d1e/d6b/f9c 7351606 1 2024-03-21T10:08:56.047 INFO:tasks.workunit.client.1.smithi174.stdout:1/454: chown d4/d2d/f57 9074 1 2024-03-21T10:08:56.047 INFO:tasks.workunit.client.0.smithi153.stdout:9/616: dwrite d1/d43/d44/d46/d9e/d61/f71 [0,4194304] 0 2024-03-21T10:08:56.048 INFO:tasks.workunit.client.1.smithi174.stdout:9/478: getdents d2/d11/d1d 0 2024-03-21T10:08:56.049 INFO:tasks.workunit.client.1.smithi174.stdout:0/434: chown d9/d21/d31/d5c/d86/f63 69345600 1 2024-03-21T10:08:56.050 INFO:tasks.workunit.client.0.smithi153.stdout:7/492: creat d4/d2a/fa3 x:0 0 0 2024-03-21T10:08:56.051 INFO:tasks.workunit.client.0.smithi153.stdout:2/704: dread dd/d15/d18/d2c/d4b/fe3 [0,4194304] 0 2024-03-21T10:08:56.052 INFO:tasks.workunit.client.1.smithi174.stdout:4/481: getdents d4/d21/d31/d3a/d7a 0 2024-03-21T10:08:56.054 INFO:tasks.workunit.client.1.smithi174.stdout:8/358: dwrite d9/d53/f6c [0,4194304] 0 2024-03-21T10:08:56.055 INFO:tasks.workunit.client.1.smithi174.stdout:1/455: write d4/d7/d33/d43/f47 [3091753,3099] 0 2024-03-21T10:08:56.055 INFO:tasks.workunit.client.1.smithi174.stdout:9/479: mkdir d2/d24/d2e/dac 0 2024-03-21T10:08:56.057 INFO:tasks.workunit.client.0.smithi153.stdout:1/507: rename d6/d13/l22 to d6/d32/d3e/l99 0 2024-03-21T10:08:56.059 INFO:tasks.workunit.client.1.smithi174.stdout:3/437: symlink d0/d12/d14/d1a/d44/d94/l96 0 2024-03-21T10:08:56.059 INFO:tasks.workunit.client.1.smithi174.stdout:8/359: rmdir d9/d2a/d39/d58 39 2024-03-21T10:08:56.064 INFO:tasks.workunit.client.1.smithi174.stdout:1/456: creat d4/d5/f9e x:0 0 0 2024-03-21T10:08:56.065 INFO:tasks.workunit.client.0.smithi153.stdout:7/493: creat d4/d14/fa4 x:0 0 0 2024-03-21T10:08:56.065 INFO:tasks.workunit.client.1.smithi174.stdout:2/448: rename d5/d25/d27/d2d/d45/d5b/d55/d9c to d5/d25/d27/d2d/d45/d5b/d2f/d68/d9e 0 2024-03-21T10:08:56.066 INFO:tasks.workunit.client.1.smithi174.stdout:0/435: link d9/d35/l71 d9/d35/d4f/d61/l95 0 2024-03-21T10:08:56.067 INFO:tasks.workunit.client.1.smithi174.stdout:3/438: fsync d0/d12/d14/f3a 0 2024-03-21T10:08:56.068 INFO:tasks.workunit.client.1.smithi174.stdout:1/457: fdatasync d4/d5/f72 0 2024-03-21T10:08:56.075 INFO:tasks.workunit.client.0.smithi153.stdout:4/630: unlink d11/d18/d7e/d93/lb2 0 2024-03-21T10:08:56.077 INFO:tasks.workunit.client.1.smithi174.stdout:6/341: dread d0/d3/d5/f3c [0,4194304] 0 2024-03-21T10:08:56.079 INFO:tasks.workunit.client.1.smithi174.stdout:8/360: creat d9/d53/d5b/f77 x:0 0 0 2024-03-21T10:08:56.080 INFO:tasks.workunit.client.1.smithi174.stdout:4/482: dwrite d4/d6c/f99 [0,4194304] 0 2024-03-21T10:08:56.080 INFO:tasks.workunit.client.0.smithi153.stdout:6/779: link d1f/d2b/d10a/d40/d65/fe9 d1f/d99/dbb/f114 0 2024-03-21T10:08:56.080 INFO:tasks.workunit.client.0.smithi153.stdout:1/508: creat d6/d32/f9a x:0 0 0 2024-03-21T10:08:56.083 INFO:tasks.workunit.client.0.smithi153.stdout:4/631: readlink d11/d18/d50/l74 0 2024-03-21T10:08:56.085 INFO:tasks.workunit.client.1.smithi174.stdout:8/361: fsync d9/d2a/d39/d76/d19/f23 0 2024-03-21T10:08:56.086 INFO:tasks.workunit.client.0.smithi153.stdout:0/534: dwrite d4/d4c/d83/f35 [0,4194304] 0 2024-03-21T10:08:56.087 INFO:tasks.workunit.client.1.smithi174.stdout:3/439: creat d0/d12/d14/f97 x:0 0 0 2024-03-21T10:08:56.088 INFO:tasks.workunit.client.1.smithi174.stdout:4/483: mknod d4/d6c/c9b 0 2024-03-21T10:08:56.090 INFO:tasks.workunit.client.1.smithi174.stdout:9/480: dwrite d2/d24/f2f [0,4194304] 0 2024-03-21T10:08:56.090 INFO:tasks.workunit.client.0.smithi153.stdout:2/705: stat dd/d15/d18/d2c/d4b/d56/d63/d8d/d9b/l9f 0 2024-03-21T10:08:56.091 INFO:tasks.workunit.client.0.smithi153.stdout:6/780: chown d1f/d2b/d10a/ffd 17752114 1 2024-03-21T10:08:56.091 INFO:tasks.workunit.client.0.smithi153.stdout:3/657: creat dd/d1e/d3d/d53/dca/fe6 x:0 0 0 2024-03-21T10:08:56.092 INFO:tasks.workunit.client.1.smithi174.stdout:1/458: dread d4/d7/f2c [0,4194304] 0 2024-03-21T10:08:56.093 INFO:tasks.workunit.client.0.smithi153.stdout:9/617: dwrite d1/d4e/f98 [0,4194304] 0 2024-03-21T10:08:56.094 INFO:tasks.workunit.client.0.smithi153.stdout:0/535: write d4/d4c/d83/f35 [1107338,109648] 0 2024-03-21T10:08:56.096 INFO:tasks.workunit.client.0.smithi153.stdout:7/494: creat d4/d2a/d40/d50/fa5 x:0 0 0 2024-03-21T10:08:56.097 INFO:tasks.workunit.client.1.smithi174.stdout:8/362: dread d9/d2a/d4d/f50 [0,4194304] 0 2024-03-21T10:08:56.099 INFO:tasks.workunit.client.0.smithi153.stdout:6/781: fdatasync d1f/d2b/d10a/d40/d51/f57 0 2024-03-21T10:08:56.102 INFO:tasks.workunit.client.1.smithi174.stdout:6/342: dread d0/d3/d20/f23 [0,4194304] 0 2024-03-21T10:08:56.103 INFO:tasks.workunit.client.0.smithi153.stdout:5/615: write d4/d8/d3b/d56/f62 [4521777,6243] 0 2024-03-21T10:08:56.103 INFO:tasks.workunit.client.1.smithi174.stdout:1/459: mknod d4/d7/d33/d43/d51/c9f 0 2024-03-21T10:08:56.104 INFO:tasks.workunit.client.0.smithi153.stdout:2/706: rmdir dd/d15 39 2024-03-21T10:08:56.106 INFO:tasks.workunit.client.0.smithi153.stdout:3/658: creat dd/d1e/d3d/fe7 x:0 0 0 2024-03-21T10:08:56.107 INFO:tasks.workunit.client.0.smithi153.stdout:3/659: stat f2 0 2024-03-21T10:08:56.108 INFO:tasks.workunit.client.1.smithi174.stdout:7/380: sync 2024-03-21T10:08:56.108 INFO:tasks.workunit.client.0.smithi153.stdout:9/618: readlink d1/d43/d44/d46/d9e/d4b/ld3 0 2024-03-21T10:08:56.111 INFO:tasks.workunit.client.1.smithi174.stdout:3/440: unlink d0/d12/d14/d1a/d27/d4f/f8f 0 2024-03-21T10:08:56.113 INFO:tasks.workunit.client.0.smithi153.stdout:6/782: fsync d1f/d2b/d10a/d40/d65/fca 0 2024-03-21T10:08:56.115 INFO:tasks.workunit.client.0.smithi153.stdout:1/509: creat d6/f9b x:0 0 0 2024-03-21T10:08:56.117 INFO:tasks.workunit.client.0.smithi153.stdout:4/632: creat d11/d2d/fc1 x:0 0 0 2024-03-21T10:08:56.119 INFO:tasks.workunit.client.1.smithi174.stdout:8/363: creat d9/d53/f78 x:0 0 0 2024-03-21T10:08:56.123 INFO:tasks.workunit.client.1.smithi174.stdout:5/482: sync 2024-03-21T10:08:56.123 INFO:tasks.workunit.client.0.smithi153.stdout:5/616: symlink d4/d5/df/d67/d86/d96/lc8 0 2024-03-21T10:08:56.126 INFO:tasks.workunit.client.1.smithi174.stdout:1/460: rename d4/d7/d15/l5f to d4/d7/d28/d59/la0 0 2024-03-21T10:08:56.127 INFO:tasks.workunit.client.0.smithi153.stdout:0/536: dwrite d4/db/f96 [0,4194304] 0 2024-03-21T10:08:56.127 INFO:tasks.workunit.client.0.smithi153.stdout:6/783: chown d1f/l6a 0 1 2024-03-21T10:08:56.129 INFO:tasks.workunit.client.0.smithi153.stdout:7/495: mkdir d4/d11/da1/da6 0 2024-03-21T10:08:56.134 INFO:tasks.workunit.client.1.smithi174.stdout:7/381: dwrite d0/d4/f2d [0,4194304] 0 2024-03-21T10:08:56.134 INFO:tasks.workunit.client.1.smithi174.stdout:8/364: mkdir d9/d2a/d39/d76/d19/d27/d79 0 2024-03-21T10:08:56.135 INFO:tasks.workunit.client.0.smithi153.stdout:9/619: write d1/d43/d44/d46/d9e/f9f [988687,91492] 0 2024-03-21T10:08:56.140 INFO:tasks.workunit.client.0.smithi153.stdout:4/633: creat d11/d2d/da3/dac/fc2 x:0 0 0 2024-03-21T10:08:56.142 INFO:tasks.workunit.client.1.smithi174.stdout:5/483: symlink d1/d2f/lac 0 2024-03-21T10:08:56.145 INFO:tasks.workunit.client.0.smithi153.stdout:3/660: dwrite dd/d1e/d3d/f69 [0,4194304] 0 2024-03-21T10:08:56.149 INFO:tasks.workunit.client.0.smithi153.stdout:6/784: fdatasync d1f/d2b/d10a/d40/d51/d98/ffe 0 2024-03-21T10:08:56.151 INFO:tasks.workunit.client.1.smithi174.stdout:3/441: mkdir d0/d12/d14/d98 0 2024-03-21T10:08:56.154 INFO:tasks.workunit.client.0.smithi153.stdout:0/537: getdents d4/d16/d27 0 2024-03-21T10:08:56.154 INFO:tasks.workunit.client.0.smithi153.stdout:6/785: fsync d1f/d2b/d10a/d40/d51/f57 0 2024-03-21T10:08:56.156 INFO:tasks.workunit.client.1.smithi174.stdout:2/449: dwrite d5/d25/d27/d2d/d45/f5d [0,4194304] 0 2024-03-21T10:08:56.157 INFO:tasks.workunit.client.0.smithi153.stdout:2/707: rename dd/d15/d18/d3a/d3d/l42 to dd/d15/d3c/da7/d91/dae/lf2 0 2024-03-21T10:08:56.157 INFO:tasks.workunit.client.0.smithi153.stdout:9/620: creat d1/d43/d44/dc3/fd4 x:0 0 0 2024-03-21T10:08:56.164 INFO:tasks.workunit.client.1.smithi174.stdout:1/461: rename d4/d7/f29 to d4/d7/d68/d8e/fa1 0 2024-03-21T10:08:56.164 INFO:tasks.workunit.client.0.smithi153.stdout:0/538: symlink d4/la4 0 2024-03-21T10:08:56.164 INFO:tasks.workunit.client.1.smithi174.stdout:2/450: write d5/d25/d27/d2d/d45/d5b/d2f/d4b/d76/f86 [4645714,89597] 0 2024-03-21T10:08:56.165 INFO:tasks.workunit.client.1.smithi174.stdout:0/436: truncate d9/f54 1469145 0 2024-03-21T10:08:56.166 INFO:tasks.workunit.client.0.smithi153.stdout:7/496: dwrite d4/d2a/f77 [0,4194304] 0 2024-03-21T10:08:56.169 INFO:tasks.workunit.client.1.smithi174.stdout:9/481: getdents d2/d11/d1d/d4b 0 2024-03-21T10:08:56.169 INFO:tasks.workunit.client.0.smithi153.stdout:8/599: sync 2024-03-21T10:08:56.169 INFO:tasks.workunit.client.0.smithi153.stdout:6/786: mkdir d1f/d2b/d10a/d40/d115 0 2024-03-21T10:08:56.171 INFO:tasks.workunit.client.1.smithi174.stdout:7/382: dwrite d0/d53/f63 [0,4194304] 0 2024-03-21T10:08:56.173 INFO:tasks.workunit.client.1.smithi174.stdout:4/484: dread d4/d19/f74 [0,4194304] 0 2024-03-21T10:08:56.174 INFO:tasks.workunit.client.1.smithi174.stdout:7/383: readlink d0/d53/l78 0 2024-03-21T10:08:56.175 INFO:tasks.workunit.client.0.smithi153.stdout:9/621: truncate d1/d43/d44/f25 4574196 0 2024-03-21T10:08:56.175 INFO:tasks.workunit.client.1.smithi174.stdout:6/343: dread d0/d3/d5/fe [0,4194304] 0 2024-03-21T10:08:56.177 INFO:tasks.workunit.client.1.smithi174.stdout:8/365: rename d9/d2a/d39/d76/d19/d27/d60/f56 to d9/d53/d5b/f7a 0 2024-03-21T10:08:56.178 INFO:tasks.workunit.client.0.smithi153.stdout:9/622: fdatasync d1/d4e/d68/fb1 0 2024-03-21T10:08:56.178 INFO:tasks.workunit.client.0.smithi153.stdout:6/787: chown d1f/d2b/d10a/d40/d7b/c102 85 1 2024-03-21T10:08:56.180 INFO:tasks.workunit.client.1.smithi174.stdout:1/462: mkdir d4/d7/d32/da2 0 2024-03-21T10:08:56.182 INFO:tasks.workunit.client.1.smithi174.stdout:1/463: write d4/d7/f90 [509181,17461] 0 2024-03-21T10:08:56.184 INFO:tasks.workunit.client.1.smithi174.stdout:9/482: read d2/f1f [1433879,17769] 0 2024-03-21T10:08:56.186 INFO:tasks.workunit.client.1.smithi174.stdout:2/451: mknod d5/d25/d27/d2d/c9f 0 2024-03-21T10:08:56.188 INFO:tasks.workunit.client.0.smithi153.stdout:2/708: dwrite dd/d15/d18/d2c/d4b/d56/d63/f80 [8388608,4194304] 0 2024-03-21T10:08:56.189 INFO:tasks.workunit.client.0.smithi153.stdout:2/709: chown dd/f90 7847235 1 2024-03-21T10:08:56.194 INFO:tasks.workunit.client.0.smithi153.stdout:9/623: creat d1/d43/d44/d46/d9e/d2e/d67/fd5 x:0 0 0 2024-03-21T10:08:56.195 INFO:tasks.workunit.client.0.smithi153.stdout:0/539: dread d4/db/f96 [0,4194304] 0 2024-03-21T10:08:56.195 INFO:tasks.workunit.client.0.smithi153.stdout:1/510: write d6/d32/d3e/f4b [3609512,68642] 0 2024-03-21T10:08:56.198 INFO:tasks.workunit.client.1.smithi174.stdout:4/485: unlink d4/d21/d31/d3a/f4c 0 2024-03-21T10:08:56.199 INFO:tasks.workunit.client.0.smithi153.stdout:5/617: dwrite d4/d8/d3b/f88 [4194304,4194304] 0 2024-03-21T10:08:56.200 INFO:tasks.workunit.client.1.smithi174.stdout:7/384: dwrite d0/d4/d10/d1b/f4e [0,4194304] 0 2024-03-21T10:08:56.201 INFO:tasks.workunit.client.1.smithi174.stdout:5/484: link d1/d16/d1c/c2b d1/d49/d93/d9c/cad 0 2024-03-21T10:08:56.205 INFO:tasks.workunit.client.0.smithi153.stdout:6/788: creat d1f/d93/dd8/d59/dfc/f116 x:0 0 0 2024-03-21T10:08:56.206 INFO:tasks.workunit.client.1.smithi174.stdout:8/366: chown d9/d53/l68 1 1 2024-03-21T10:08:56.208 INFO:tasks.workunit.client.1.smithi174.stdout:7/385: truncate d0/d4/d38/f67 50991 0 2024-03-21T10:08:56.209 INFO:tasks.workunit.client.1.smithi174.stdout:0/437: mknod d9/d21/d31/d5c/d92/c96 0 2024-03-21T10:08:56.211 INFO:tasks.workunit.client.0.smithi153.stdout:7/497: rename d4/d2a/d40/f74 to d4/fa7 0 2024-03-21T10:08:56.213 INFO:tasks.workunit.client.1.smithi174.stdout:8/367: dread - d9/d53/d5b/f7a zero size 2024-03-21T10:08:56.213 INFO:tasks.workunit.client.1.smithi174.stdout:5/485: chown d1/d16/d2e/d66/d8b/c6a 68699 1 2024-03-21T10:08:56.215 INFO:tasks.workunit.client.0.smithi153.stdout:2/710: creat dd/d15/d18/d2c/d33/d98/dc7/ded/ff3 x:0 0 0 2024-03-21T10:08:56.217 INFO:tasks.workunit.client.1.smithi174.stdout:6/344: creat d0/d3/d5/d6/f70 x:0 0 0 2024-03-21T10:08:56.220 INFO:tasks.workunit.client.0.smithi153.stdout:0/540: dwrite d4/f10 [0,4194304] 0 2024-03-21T10:08:56.220 INFO:tasks.workunit.client.0.smithi153.stdout:3/661: truncate dd/d1e/f30 2038869 0 2024-03-21T10:08:56.220 INFO:tasks.workunit.client.0.smithi153.stdout:4/634: dread d11/d18/d54/f68 [0,4194304] 0 2024-03-21T10:08:56.223 INFO:tasks.workunit.client.1.smithi174.stdout:7/386: unlink d0/d4/d10/d1b/l1d 0 2024-03-21T10:08:56.223 INFO:tasks.workunit.client.0.smithi153.stdout:3/662: read - dd/d1e/d3d/d94/d8e/fe3 zero size 2024-03-21T10:08:56.224 INFO:tasks.workunit.client.1.smithi174.stdout:1/464: dread d4/d7/d15/f2f [0,4194304] 0 2024-03-21T10:08:56.224 INFO:tasks.workunit.client.1.smithi174.stdout:0/438: creat d9/d21/d31/d5c/d92/f97 x:0 0 0 2024-03-21T10:08:56.229 INFO:tasks.workunit.client.1.smithi174.stdout:5/486: write d1/d2f/f4c [4474922,88301] 0 2024-03-21T10:08:56.230 INFO:tasks.workunit.client.1.smithi174.stdout:4/486: mkdir d4/d21/d31/d6a/d8f/d9c 0 2024-03-21T10:08:56.232 INFO:tasks.workunit.client.0.smithi153.stdout:1/511: write d6/d3a/d3c/f64 [1630507,73343] 0 2024-03-21T10:08:56.234 INFO:tasks.workunit.client.1.smithi174.stdout:9/483: dread d2/d11/f75 [0,4194304] 0 2024-03-21T10:08:56.235 INFO:tasks.workunit.client.0.smithi153.stdout:8/600: rename d5/d9/d43/dd/d2d/c2e to d5/d9/d43/dd/cc8 0 2024-03-21T10:08:56.235 INFO:tasks.workunit.client.1.smithi174.stdout:5/487: fdatasync d1/d16/d1c/d81/f95 0 2024-03-21T10:08:56.237 INFO:tasks.workunit.client.1.smithi174.stdout:7/387: unlink d0/d53/f6e 0 2024-03-21T10:08:56.239 INFO:tasks.workunit.client.1.smithi174.stdout:0/439: mkdir d9/d21/d31/d5c/d92/d98 0 2024-03-21T10:08:56.242 INFO:tasks.workunit.client.0.smithi153.stdout:8/601: stat d5/d9/d43/dd/d2d/cb3 0 2024-03-21T10:08:56.243 INFO:tasks.workunit.client.1.smithi174.stdout:7/388: chown d0/d4/l37 382574 1 2024-03-21T10:08:56.245 INFO:tasks.workunit.client.1.smithi174.stdout:2/452: dwrite d5/d25/d27/d74/f95 [0,4194304] 0 2024-03-21T10:08:56.248 INFO:tasks.workunit.client.1.smithi174.stdout:7/389: read - d0/d4/d10/d44/f73 zero size 2024-03-21T10:08:56.248 INFO:tasks.workunit.client.1.smithi174.stdout:5/488: chown d1/d49/d4f/l70 52 1 2024-03-21T10:08:56.251 INFO:tasks.workunit.client.1.smithi174.stdout:4/487: mkdir d4/d21/d31/d3a/d7a/d9d 0 2024-03-21T10:08:56.254 INFO:tasks.workunit.client.1.smithi174.stdout:4/488: chown d4/d21/c55 901 1 2024-03-21T10:08:56.256 INFO:tasks.workunit.client.1.smithi174.stdout:8/368: dwrite f8 [0,4194304] 0 2024-03-21T10:08:56.256 INFO:tasks.workunit.client.0.smithi153.stdout:9/624: dread d1/f1b [0,4194304] 0 2024-03-21T10:08:56.256 INFO:tasks.workunit.client.1.smithi174.stdout:5/489: chown d1/d16/d2e/d66/f6f 1486638 1 2024-03-21T10:08:56.257 INFO:tasks.workunit.client.1.smithi174.stdout:9/484: rename d2/f7f to d2/d24/d2e/d63/fad 0 2024-03-21T10:08:56.257 INFO:tasks.workunit.client.0.smithi153.stdout:9/625: fdatasync d1/d43/d44/d46/d9e/d24/fa1 0 2024-03-21T10:08:56.269 INFO:tasks.workunit.client.1.smithi174.stdout:1/465: dwrite d4/d7/f2c [0,4194304] 0 2024-03-21T10:08:56.270 INFO:tasks.workunit.client.1.smithi174.stdout:2/453: mkdir d5/d25/d27/d2e/d4f/da0 0 2024-03-21T10:08:56.270 INFO:tasks.workunit.client.0.smithi153.stdout:7/498: dwrite d4/d11/f72 [0,4194304] 0 2024-03-21T10:08:56.276 INFO:tasks.workunit.client.0.smithi153.stdout:2/711: write dd/d15/d18/d2c/d33/f57 [413285,131026] 0 2024-03-21T10:08:56.280 INFO:tasks.workunit.client.1.smithi174.stdout:7/390: symlink d0/d4/d10/d1b/l84 0 2024-03-21T10:08:56.280 INFO:tasks.workunit.client.1.smithi174.stdout:6/345: dread d0/d3/d5/f45 [0,4194304] 0 2024-03-21T10:08:56.283 INFO:tasks.workunit.client.1.smithi174.stdout:3/442: sync 2024-03-21T10:08:56.285 INFO:tasks.workunit.client.0.smithi153.stdout:0/541: creat d4/db/fa5 x:0 0 0 2024-03-21T10:08:56.286 INFO:tasks.workunit.client.0.smithi153.stdout:6/789: truncate d1f/f4a 911826 0 2024-03-21T10:08:56.288 INFO:tasks.workunit.client.1.smithi174.stdout:9/485: symlink d2/d11/d1d/d2a/d91/lae 0 2024-03-21T10:08:56.291 INFO:tasks.workunit.client.1.smithi174.stdout:5/490: dwrite d1/d16/d2e/f80 [0,4194304] 0 2024-03-21T10:08:56.291 INFO:tasks.workunit.client.0.smithi153.stdout:1/512: creat d6/d3a/d3c/f9c x:0 0 0 2024-03-21T10:08:56.294 INFO:tasks.workunit.client.1.smithi174.stdout:4/489: dwrite d4/d21/d24/d28/d38/f5f [4194304,4194304] 0 2024-03-21T10:08:56.295 INFO:tasks.workunit.client.1.smithi174.stdout:9/486: chown d2/d11/d1d/d4b/c97 51854389 1 2024-03-21T10:08:56.295 INFO:tasks.workunit.client.1.smithi174.stdout:0/440: mkdir d9/d35/d4a/d67/d89/d99 0 2024-03-21T10:08:56.296 INFO:tasks.workunit.client.0.smithi153.stdout:3/663: dwrite dd/f1f [0,4194304] 0 2024-03-21T10:08:56.296 INFO:tasks.workunit.client.0.smithi153.stdout:5/618: rename d4/d5/df/d5e/d69/c8e to d4/d5/de/d44/cc9 0 2024-03-21T10:08:56.300 INFO:tasks.workunit.client.1.smithi174.stdout:7/391: creat d0/d4/d10/d60/f85 x:0 0 0 2024-03-21T10:08:56.305 INFO:tasks.workunit.client.1.smithi174.stdout:2/454: dwrite d5/d25/d27/d2d/d45/d5b/d2f/f87 [0,4194304] 0 2024-03-21T10:08:56.305 INFO:tasks.workunit.client.1.smithi174.stdout:8/369: creat d9/d2a/d39/d58/f7b x:0 0 0 2024-03-21T10:08:56.307 INFO:tasks.workunit.client.0.smithi153.stdout:4/635: fdatasync d11/d18/f28 0 2024-03-21T10:08:56.307 INFO:tasks.workunit.client.1.smithi174.stdout:8/370: read d9/d2a/d39/f5f [106553,9126] 0 2024-03-21T10:08:56.308 INFO:tasks.workunit.client.1.smithi174.stdout:3/443: symlink d0/d12/d14/d1a/d27/d3c/l99 0 2024-03-21T10:08:56.310 INFO:tasks.workunit.client.1.smithi174.stdout:5/491: creat d1/d2f/fae x:0 0 0 2024-03-21T10:08:56.312 INFO:tasks.workunit.client.1.smithi174.stdout:5/492: chown d1/d16/d2e/c3e 0 1 2024-03-21T10:08:56.313 INFO:tasks.workunit.client.1.smithi174.stdout:4/490: mknod d4/d21/d24/d28/c9e 0 2024-03-21T10:08:56.315 INFO:tasks.workunit.client.0.smithi153.stdout:1/513: unlink d6/d10/d2c/f2e 0 2024-03-21T10:08:56.322 INFO:tasks.workunit.client.0.smithi153.stdout:5/619: creat d4/d8/d3b/d56/fca x:0 0 0 2024-03-21T10:08:56.330 INFO:tasks.workunit.client.0.smithi153.stdout:4/636: mkdir d11/d9d/dc3 0 2024-03-21T10:08:56.332 INFO:tasks.workunit.client.1.smithi174.stdout:2/455: chown d5/d25/d27/d2d/d45/d5b/f1d 7252970 1 2024-03-21T10:08:56.333 INFO:tasks.workunit.client.1.smithi174.stdout:9/487: truncate d2/f1f 2616232 0 2024-03-21T10:08:56.334 INFO:tasks.workunit.client.1.smithi174.stdout:5/493: rename d1/d49/d4f/f68 to d1/d16/d2e/d8d/d99/faf 0 2024-03-21T10:08:56.340 INFO:tasks.workunit.client.0.smithi153.stdout:9/626: creat d1/d4/fd6 x:0 0 0 2024-03-21T10:08:56.341 INFO:tasks.workunit.client.1.smithi174.stdout:1/466: write d4/d7/d33/f36 [3111513,99448] 0 2024-03-21T10:08:56.342 INFO:tasks.workunit.client.0.smithi153.stdout:0/542: link d4/db/fa0 d4/d16/d19/d58/fa6 0 2024-03-21T10:08:56.342 INFO:tasks.workunit.client.0.smithi153.stdout:9/627: read - d1/d43/d44/dc3/fd4 zero size 2024-03-21T10:08:56.344 INFO:tasks.workunit.client.0.smithi153.stdout:5/620: dread d4/d8/d5c/f73 [0,4194304] 0 2024-03-21T10:08:56.345 INFO:tasks.workunit.client.1.smithi174.stdout:5/494: readlink d1/d16/l45 0 2024-03-21T10:08:56.345 INFO:tasks.workunit.client.1.smithi174.stdout:4/491: rename d4/d21/d24/d28/d79/c87 to d4/d21/d31/d3a/d77/c9f 0 2024-03-21T10:08:56.345 INFO:tasks.workunit.client.1.smithi174.stdout:6/346: write d0/d3/d5/d6/f6a [1028125,39177] 0 2024-03-21T10:08:56.346 INFO:tasks.workunit.client.0.smithi153.stdout:6/790: link d1f/d2b/d10a/d40/d7b/ce8 d1f/d2b/d10a/d40/d51/c117 0 2024-03-21T10:08:56.348 INFO:tasks.workunit.client.0.smithi153.stdout:8/602: rename d5/d9/d43/dd/d2d/d45/d88/f5d to d5/fc9 0 2024-03-21T10:08:56.349 INFO:tasks.workunit.client.0.smithi153.stdout:4/637: read d11/d18/f61 [2726018,80052] 0 2024-03-21T10:08:56.350 INFO:tasks.workunit.client.1.smithi174.stdout:2/456: mkdir d5/d25/d27/d2e/d7b/d81/da1 0 2024-03-21T10:08:56.353 INFO:tasks.workunit.client.1.smithi174.stdout:8/371: dread d9/fe [0,4194304] 0 2024-03-21T10:08:56.353 INFO:tasks.workunit.client.1.smithi174.stdout:1/467: rename d4/d7/l52 to d4/d7/d33/d43/la3 0 2024-03-21T10:08:56.354 INFO:tasks.workunit.client.0.smithi153.stdout:0/543: mkdir d4/d16/d19/d30/da7 0 2024-03-21T10:08:56.354 INFO:tasks.workunit.client.0.smithi153.stdout:9/628: stat d1/cd 0 2024-03-21T10:08:56.355 INFO:tasks.workunit.client.1.smithi174.stdout:0/441: dwrite d9/d35/d4f/d56/f66 [0,4194304] 0 2024-03-21T10:08:56.355 INFO:tasks.workunit.client.0.smithi153.stdout:3/664: getdents dd/d1e/da6 0 2024-03-21T10:08:56.356 INFO:tasks.workunit.client.0.smithi153.stdout:7/499: rename d4/d14/d2f/d78/f28 to d4/d19/fa8 0 2024-03-21T10:08:56.357 INFO:tasks.workunit.client.0.smithi153.stdout:4/638: symlink d11/d2d/d3d/lc4 0 2024-03-21T10:08:56.357 INFO:tasks.workunit.client.1.smithi174.stdout:1/468: chown d4/d7/d15/d4f/f83 5009343 1 2024-03-21T10:08:56.358 INFO:tasks.workunit.client.0.smithi153.stdout:3/665: chown dd/d1e/d2f 6339 1 2024-03-21T10:08:56.358 INFO:tasks.workunit.client.0.smithi153.stdout:0/544: rmdir d4/d16/d19/d95 39 2024-03-21T10:08:56.359 INFO:tasks.workunit.client.1.smithi174.stdout:8/372: rmdir d9 39 2024-03-21T10:08:56.360 INFO:tasks.workunit.client.0.smithi153.stdout:6/791: mkdir d1f/d2b/d10a/d40/d118 0 2024-03-21T10:08:56.362 INFO:tasks.workunit.client.0.smithi153.stdout:3/666: chown dd/d1e/d2f/d35 1800 1 2024-03-21T10:08:56.368 INFO:tasks.workunit.client.1.smithi174.stdout:0/442: mkdir d9/d35/d4a/d67/d89/d9a 0 2024-03-21T10:08:56.370 INFO:tasks.workunit.client.0.smithi153.stdout:7/500: creat d4/d2a/d3b/d81/fa9 x:0 0 0 2024-03-21T10:08:56.370 INFO:tasks.workunit.client.0.smithi153.stdout:5/621: dwrite d4/d5/df/d67/da0/db3/fa9 [0,4194304] 0 2024-03-21T10:08:56.370 INFO:tasks.workunit.client.1.smithi174.stdout:9/488: dwrite d2/f45 [4194304,4194304] 0 2024-03-21T10:08:56.379 INFO:tasks.workunit.client.1.smithi174.stdout:3/444: dread d0/d12/f49 [0,4194304] 0 2024-03-21T10:08:56.379 INFO:tasks.workunit.client.1.smithi174.stdout:1/469: mkdir d4/d7/d33/d43/d51/da4 0 2024-03-21T10:08:56.382 INFO:tasks.workunit.client.0.smithi153.stdout:0/545: symlink d4/d16/d19/d30/da7/la8 0 2024-03-21T10:08:56.388 INFO:tasks.workunit.client.0.smithi153.stdout:3/667: mknod dd/d1e/d3d/d94/de2/ce8 0 2024-03-21T10:08:56.389 INFO:tasks.workunit.client.1.smithi174.stdout:5/495: dwrite d1/d16/d2e/d66/d8b/f48 [0,4194304] 0 2024-03-21T10:08:56.391 INFO:tasks.workunit.client.1.smithi174.stdout:3/445: creat d0/d12/d14/d1a/d27/f9a x:0 0 0 2024-03-21T10:08:56.394 INFO:tasks.workunit.client.1.smithi174.stdout:5/496: stat d1/ff 0 2024-03-21T10:08:56.395 INFO:tasks.workunit.client.1.smithi174.stdout:0/443: dread d9/d21/d31/d3c/f49 [0,4194304] 0 2024-03-21T10:08:56.396 INFO:tasks.workunit.client.1.smithi174.stdout:2/457: write d5/d25/d27/d2d/f43 [586718,42668] 0 2024-03-21T10:08:56.397 INFO:tasks.workunit.client.0.smithi153.stdout:5/622: link d4/d8/d5a/d5d/l7d d4/d5/de/d2f/lcb 0 2024-03-21T10:08:56.399 INFO:tasks.workunit.client.0.smithi153.stdout:8/603: dwrite d5/d9/d43/dd/d20/d26/d24/d52/d55/f71 [0,4194304] 0 2024-03-21T10:08:56.400 INFO:tasks.workunit.client.1.smithi174.stdout:3/446: chown d0/d12/d14/d1a/f40 12186284 1 2024-03-21T10:08:56.401 INFO:tasks.workunit.client.0.smithi153.stdout:3/668: truncate dd/d1e/d2f/d35/d68/f9a 270330 0 2024-03-21T10:08:56.403 INFO:tasks.workunit.client.0.smithi153.stdout:0/546: link d4/d16/d19/d30/d6c/d98/d5e/f78 d4/d16/d19/d30/d6c/d98/d5b/fa9 0 2024-03-21T10:08:56.403 INFO:tasks.workunit.client.0.smithi153.stdout:6/792: write d1f/d2b/f85 [4228564,90168] 0 2024-03-21T10:08:56.404 INFO:tasks.workunit.client.1.smithi174.stdout:5/497: creat d1/d49/d4f/d51/d8f/fb0 x:0 0 0 2024-03-21T10:08:56.404 INFO:tasks.workunit.client.1.smithi174.stdout:0/444: chown d9/d35/d4f/d79 0 1 2024-03-21T10:08:56.405 INFO:tasks.workunit.client.0.smithi153.stdout:3/669: chown dd/d41/d80/dbb/fc1 3360 1 2024-03-21T10:08:56.407 INFO:tasks.workunit.client.0.smithi153.stdout:0/547: fsync d4/d4c/d83/f20 0 2024-03-21T10:08:56.407 INFO:tasks.workunit.client.1.smithi174.stdout:3/447: mkdir d0/d12/d46/d9b 0 2024-03-21T10:08:56.409 INFO:tasks.workunit.client.1.smithi174.stdout:6/347: dwrite d0/d3/d5/df/d16/f2b [0,4194304] 0 2024-03-21T10:08:56.410 INFO:tasks.workunit.client.0.smithi153.stdout:5/623: creat d4/d5/fcc x:0 0 0 2024-03-21T10:08:56.415 INFO:tasks.workunit.client.1.smithi174.stdout:5/498: mknod d1/d49/d4f/d51/d8f/cb1 0 2024-03-21T10:08:56.424 INFO:tasks.workunit.client.1.smithi174.stdout:0/445: getdents d9/d21/d31/d5c/d92/d98 0 2024-03-21T10:08:56.426 INFO:tasks.workunit.client.1.smithi174.stdout:6/348: unlink d0/l4b 0 2024-03-21T10:08:56.426 INFO:tasks.workunit.client.1.smithi174.stdout:4/492: dread d4/d19/f97 [0,4194304] 0 2024-03-21T10:08:56.435 INFO:tasks.workunit.client.1.smithi174.stdout:0/446: unlink d9/f7b 0 2024-03-21T10:08:56.439 INFO:tasks.workunit.client.1.smithi174.stdout:4/493: mknod d4/d21/d31/d3a/ca0 0 2024-03-21T10:08:56.440 INFO:tasks.workunit.client.0.smithi153.stdout:8/604: mkdir d5/d9/d43/d7b/daa/dca 0 2024-03-21T10:08:56.441 INFO:tasks.workunit.client.1.smithi174.stdout:2/458: truncate d5/d25/d27/f2a 292320 0 2024-03-21T10:08:56.442 INFO:tasks.workunit.client.0.smithi153.stdout:9/629: dwrite d1/d4/d5f/fce [0,4194304] 0 2024-03-21T10:08:56.448 INFO:tasks.workunit.client.1.smithi174.stdout:6/349: rename d0/d3/f26 to d0/d3/d5/d6/d42/f71 0 2024-03-21T10:08:56.448 INFO:tasks.workunit.client.0.smithi153.stdout:3/670: dread dd/d1e/d2f/d35/daf/fb4 [0,4194304] 0 2024-03-21T10:08:56.450 INFO:tasks.workunit.client.1.smithi174.stdout:0/447: mkdir d9/d21/d31/d5c/d86/d9b 0 2024-03-21T10:08:56.452 INFO:tasks.workunit.client.1.smithi174.stdout:1/470: dread d4/d7/d32/f3c [0,4194304] 0 2024-03-21T10:08:56.454 INFO:tasks.workunit.client.0.smithi153.stdout:9/630: dread - d1/d43/d44/d46/d9e/d24/fb4 zero size 2024-03-21T10:08:56.455 INFO:tasks.workunit.client.0.smithi153.stdout:6/793: rename d1f/d2b/d10a/d40/d4f/d66/fd0 to d1f/d2b/d10a/d40/d4f/f119 0 2024-03-21T10:08:56.456 INFO:tasks.workunit.client.1.smithi174.stdout:9/489: dread d2/d54/f5b [0,4194304] 0 2024-03-21T10:08:56.456 INFO:tasks.workunit.client.0.smithi153.stdout:5/624: symlink d4/d8/d5a/d5d/db1/lcd 0 2024-03-21T10:08:56.459 INFO:tasks.workunit.client.1.smithi174.stdout:4/494: rename d4/d21/d31/d3a/c3f to d4/d6c/ca1 0 2024-03-21T10:08:56.462 INFO:tasks.workunit.client.1.smithi174.stdout:6/350: write d0/d3/d5/f3c [3112051,16284] 0 2024-03-21T10:08:56.462 INFO:tasks.workunit.client.0.smithi153.stdout:9/631: fdatasync d1/d4/fc 0 2024-03-21T10:08:56.465 INFO:tasks.workunit.client.0.smithi153.stdout:7/501: dwrite d4/d2a/d40/f4c [4194304,4194304] 0 2024-03-21T10:08:56.465 INFO:tasks.workunit.client.0.smithi153.stdout:5/625: dread - d4/d5/df/d67/da0/db3/f74 zero size 2024-03-21T10:08:56.469 INFO:tasks.workunit.client.1.smithi174.stdout:4/495: write d4/d21/d31/d6a/f72 [857311,119921] 0 2024-03-21T10:08:56.469 INFO:tasks.workunit.client.1.smithi174.stdout:8/373: dread d9/d2a/d39/d76/f5a [0,4194304] 0 2024-03-21T10:08:56.469 INFO:tasks.workunit.client.0.smithi153.stdout:6/794: symlink d1f/d93/dd8/d103/l11a 0 2024-03-21T10:08:56.473 INFO:tasks.workunit.client.0.smithi153.stdout:9/632: symlink d1/ld7 0 2024-03-21T10:08:56.473 INFO:tasks.workunit.client.1.smithi174.stdout:9/490: write d2/d24/d2e/f31 [3285327,76898] 0 2024-03-21T10:08:56.474 INFO:tasks.workunit.client.1.smithi174.stdout:6/351: chown d0/d3/d5/d6/f13 137043 1 2024-03-21T10:08:56.475 INFO:tasks.workunit.client.0.smithi153.stdout:0/548: dwrite d4/d16/d19/d30/d6c/d98/f80 [0,4194304] 0 2024-03-21T10:08:56.476 INFO:tasks.workunit.client.1.smithi174.stdout:0/448: symlink d9/d21/d5f/l9c 0 2024-03-21T10:08:56.476 INFO:tasks.workunit.client.1.smithi174.stdout:8/374: write d9/d53/f6f [799429,71752] 0 2024-03-21T10:08:56.477 INFO:tasks.workunit.client.0.smithi153.stdout:6/795: mknod d1f/d93/dd8/d59/d70/ddc/c11b 0 2024-03-21T10:08:56.478 INFO:tasks.workunit.client.0.smithi153.stdout:8/605: dwrite d5/fc9 [0,4194304] 0 2024-03-21T10:08:56.478 INFO:tasks.workunit.client.1.smithi174.stdout:2/459: getdents d5/d25/d27/d2d/d45/d5b/d55 0 2024-03-21T10:08:56.480 INFO:tasks.workunit.client.1.smithi174.stdout:9/491: write d2/d11/d60/f77 [2356099,74105] 0 2024-03-21T10:08:56.480 INFO:tasks.workunit.client.1.smithi174.stdout:8/375: read d9/d2a/d39/f5f [485593,18470] 0 2024-03-21T10:08:56.483 INFO:tasks.workunit.client.1.smithi174.stdout:8/376: write f1 [444872,32968] 0 2024-03-21T10:08:56.484 INFO:tasks.workunit.client.0.smithi153.stdout:0/549: creat d4/d4c/d83/d22/d61/faa x:0 0 0 2024-03-21T10:08:56.485 INFO:tasks.workunit.client.0.smithi153.stdout:3/671: dwrite dd/d1e/d2f/d35/daf/dcc/d58/f8b [0,4194304] 0 2024-03-21T10:08:56.487 INFO:tasks.workunit.client.1.smithi174.stdout:0/449: write d9/d21/d31/d3c/f5b [5146517,97818] 0 2024-03-21T10:08:56.488 INFO:tasks.workunit.client.0.smithi153.stdout:3/672: write dd/d1e/d3d/d53/fa3 [1410197,64764] 0 2024-03-21T10:08:56.488 INFO:tasks.workunit.client.1.smithi174.stdout:2/460: write d5/d25/f7e [285517,91441] 0 2024-03-21T10:08:56.491 INFO:tasks.workunit.client.0.smithi153.stdout:6/796: unlink d1f/f28 0 2024-03-21T10:08:56.493 INFO:tasks.workunit.client.0.smithi153.stdout:0/550: unlink d4/d16/d19/l43 0 2024-03-21T10:08:56.503 INFO:tasks.workunit.client.1.smithi174.stdout:0/450: mknod d9/d35/c9d 0 2024-03-21T10:08:56.503 INFO:tasks.workunit.client.1.smithi174.stdout:3/448: dread d0/d12/d14/d1a/d5c/f8e [0,4194304] 0 2024-03-21T10:08:56.503 INFO:tasks.workunit.client.1.smithi174.stdout:9/492: write d2/d11/f75 [5032570,35432] 0 2024-03-21T10:08:56.504 INFO:tasks.workunit.client.0.smithi153.stdout:9/633: dwrite d1/d43/d44/d46/d9e/d24/fa1 [0,4194304] 0 2024-03-21T10:08:56.506 INFO:tasks.workunit.client.0.smithi153.stdout:0/551: mkdir d4/d16/d19/d30/d6c/d98/d5b/dab 0 2024-03-21T10:08:56.508 INFO:tasks.workunit.client.0.smithi153.stdout:9/634: unlink d1/d43/d44/d46/d9e/l99 0 2024-03-21T10:08:56.509 INFO:tasks.workunit.client.0.smithi153.stdout:0/552: chown d4/d16/d19/d30/d6c/d98/c84 3576 1 2024-03-21T10:08:56.510 INFO:tasks.workunit.client.0.smithi153.stdout:9/635: creat d1/d8a/fd8 x:0 0 0 2024-03-21T10:08:56.512 INFO:tasks.workunit.client.1.smithi174.stdout:9/493: chown d2/d11/l16 0 1 2024-03-21T10:08:56.513 INFO:tasks.workunit.client.0.smithi153.stdout:6/797: truncate d1f/d93/dd8/ffb 749821 0 2024-03-21T10:08:56.515 INFO:tasks.workunit.client.0.smithi153.stdout:9/636: creat d1/fd9 x:0 0 0 2024-03-21T10:08:56.516 INFO:tasks.workunit.client.1.smithi174.stdout:0/451: readlink d9/d1c/l38 0 2024-03-21T10:08:56.517 INFO:tasks.workunit.client.0.smithi153.stdout:7/502: write d4/d1b/d20/f2c [1371894,88842] 0 2024-03-21T10:08:56.518 INFO:tasks.workunit.client.1.smithi174.stdout:2/461: chown d5/d25/d27/d2d/d45/d5b/d2f/d4b/f78 3390 1 2024-03-21T10:08:56.520 INFO:tasks.workunit.client.1.smithi174.stdout:4/496: dwrite d4/d21/f27 [0,4194304] 0 2024-03-21T10:08:56.523 INFO:tasks.workunit.client.0.smithi153.stdout:8/606: write d5/d9/d43/d7b/dc4/da7/fc6 [3240642,28566] 0 2024-03-21T10:08:56.526 INFO:tasks.workunit.client.0.smithi153.stdout:5/626: dwrite d4/d5/de/f21 [0,4194304] 0 2024-03-21T10:08:56.531 INFO:tasks.workunit.client.1.smithi174.stdout:1/471: dwrite d4/d7/d32/d86/f8a [0,4194304] 0 2024-03-21T10:08:56.531 INFO:tasks.workunit.client.1.smithi174.stdout:0/452: creat d9/d35/d4a/f9e x:0 0 0 2024-03-21T10:08:56.532 INFO:tasks.workunit.client.0.smithi153.stdout:5/627: truncate d4/d8/f68 223157 0 2024-03-21T10:08:56.532 INFO:tasks.workunit.client.0.smithi153.stdout:0/553: dwrite d4/d4c/d83/f20 [0,4194304] 0 2024-03-21T10:08:56.533 INFO:tasks.workunit.client.0.smithi153.stdout:9/637: link d1/d43/d44/f34 d1/d4/d5f/fda 0 2024-03-21T10:08:56.533 INFO:tasks.workunit.client.1.smithi174.stdout:7/392: sync 2024-03-21T10:08:56.534 INFO:tasks.workunit.client.0.smithi153.stdout:5/628: symlink d4/d5/d9e/lce 0 2024-03-21T10:08:56.535 INFO:tasks.workunit.client.0.smithi153.stdout:0/554: mknod d4/d4c/d83/d22/d61/d77/cac 0 2024-03-21T10:08:56.537 INFO:tasks.workunit.client.1.smithi174.stdout:6/352: dwrite d0/d3/d5/f5c [0,4194304] 0 2024-03-21T10:08:56.537 INFO:tasks.workunit.client.0.smithi153.stdout:5/629: chown d4/d5/df/d5e/d69 12316295 1 2024-03-21T10:08:56.541 INFO:tasks.workunit.client.1.smithi174.stdout:4/497: symlink d4/d21/d31/d3a/d67/la2 0 2024-03-21T10:08:56.541 INFO:tasks.workunit.client.0.smithi153.stdout:9/638: symlink d1/d43/d44/d46/d4d/d5c/ldb 0 2024-03-21T10:08:56.543 INFO:tasks.workunit.client.0.smithi153.stdout:6/798: write d1f/d99/fb7 [1198,86693] 0 2024-03-21T10:08:56.545 INFO:tasks.workunit.client.0.smithi153.stdout:0/555: mkdir d4/d16/d19/dad 0 2024-03-21T10:08:56.547 INFO:tasks.workunit.client.0.smithi153.stdout:5/630: mkdir d4/d5/dcf 0 2024-03-21T10:08:56.550 INFO:tasks.workunit.client.0.smithi153.stdout:7/503: dwrite d4/d2a/d3b/f98 [0,4194304] 0 2024-03-21T10:08:56.550 INFO:tasks.workunit.client.0.smithi153.stdout:6/799: dread - d1f/d2b/d10a/d40/d65/fca zero size 2024-03-21T10:08:56.553 INFO:tasks.workunit.client.1.smithi174.stdout:8/377: dwrite d9/f1a [0,4194304] 0 2024-03-21T10:08:56.553 INFO:tasks.workunit.client.0.smithi153.stdout:9/639: mkdir d1/d43/d72/d9d/ddc 0 2024-03-21T10:08:56.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:56 smithi153 ceph-mon[29117]: pgmap v14: 65 pgs: 65 active+clean; 2.2 GiB data, 6.5 GiB used, 530 GiB / 536 GiB avail; 71 MiB/s rd, 310 MiB/s wr, 547 op/s 2024-03-21T10:08:56.557 INFO:tasks.workunit.client.0.smithi153.stdout:5/631: fsync d4/d5/df/d67/da0/db3/f53 0 2024-03-21T10:08:56.557 INFO:tasks.workunit.client.0.smithi153.stdout:7/504: fsync d4/d14/d16/f39 0 2024-03-21T10:08:56.557 INFO:tasks.workunit.client.1.smithi174.stdout:3/449: rmdir d0/d12/d14/d1a/d27/d4f/d64/d77 0 2024-03-21T10:08:56.557 INFO:tasks.workunit.client.0.smithi153.stdout:6/800: chown d1f/d2b/d10a/d40/d51/c76 1786 1 2024-03-21T10:08:56.562 INFO:tasks.workunit.client.0.smithi153.stdout:9/640: dread d1/d4/f9c [0,4194304] 0 2024-03-21T10:08:56.562 INFO:tasks.workunit.client.0.smithi153.stdout:7/505: write d4/f10 [1907090,69449] 0 2024-03-21T10:08:56.565 INFO:tasks.workunit.client.0.smithi153.stdout:7/506: chown d4/d2a/f2d 1 1 2024-03-21T10:08:56.565 INFO:tasks.workunit.client.1.smithi174.stdout:9/494: dwrite d2/d54/f92 [0,4194304] 0 2024-03-21T10:08:56.565 INFO:tasks.workunit.client.1.smithi174.stdout:7/393: mkdir d0/d4/d10/d1b/d86 0 2024-03-21T10:08:56.569 INFO:tasks.workunit.client.0.smithi153.stdout:7/507: chown d4/c8 15672669 1 2024-03-21T10:08:56.569 INFO:tasks.workunit.client.1.smithi174.stdout:5/499: sync 2024-03-21T10:08:56.569 INFO:tasks.workunit.client.1.smithi174.stdout:4/498: write d4/d19/f97 [1385997,75430] 0 2024-03-21T10:08:56.569 INFO:tasks.workunit.client.0.smithi153.stdout:6/801: fdatasync d1f/d2b/d10a/d40/d5f/f8b 0 2024-03-21T10:08:56.574 INFO:tasks.workunit.client.0.smithi153.stdout:9/641: read d1/d43/d44/f33 [2959227,52714] 0 2024-03-21T10:08:56.576 INFO:tasks.workunit.client.0.smithi153.stdout:0/556: dwrite d4/f6 [0,4194304] 0 2024-03-21T10:08:56.579 INFO:tasks.workunit.client.1.smithi174.stdout:1/472: read d4/d7/d32/f3a [2956781,39671] 0 2024-03-21T10:08:56.580 INFO:tasks.workunit.client.1.smithi174.stdout:8/378: write d9/f30 [2204696,5860] 0 2024-03-21T10:08:56.580 INFO:tasks.workunit.client.1.smithi174.stdout:5/500: chown d1/d2f/l44 482 1 2024-03-21T10:08:56.581 INFO:tasks.workunit.client.0.smithi153.stdout:0/557: write d4/d16/d19/fa2 [458498,8787] 0 2024-03-21T10:08:56.581 INFO:tasks.workunit.client.0.smithi153.stdout:9/642: write d1/d43/fab [103122,82235] 0 2024-03-21T10:08:56.582 INFO:tasks.workunit.client.1.smithi174.stdout:7/394: unlink d0/d4/d38/l4a 0 2024-03-21T10:08:56.582 INFO:tasks.workunit.client.1.smithi174.stdout:2/462: dwrite d5/d25/f7e [0,4194304] 0 2024-03-21T10:08:56.587 INFO:tasks.workunit.client.1.smithi174.stdout:8/379: readlink d9/d2a/d39/d76/l57 0 2024-03-21T10:08:56.591 INFO:tasks.workunit.client.1.smithi174.stdout:4/499: write d4/d21/d31/d6a/f6e [432368,39442] 0 2024-03-21T10:08:56.592 INFO:tasks.workunit.client.0.smithi153.stdout:6/802: rename d1f/d2b/c8a to d1f/d2b/d10a/d40/d118/c11c 0 2024-03-21T10:08:56.593 INFO:tasks.workunit.client.0.smithi153.stdout:9/643: creat d1/d43/d72/fdd x:0 0 0 2024-03-21T10:08:56.593 INFO:tasks.workunit.client.0.smithi153.stdout:7/508: getdents d4/d11/da1/da6 0 2024-03-21T10:08:56.595 INFO:tasks.workunit.client.1.smithi174.stdout:0/453: rename d9/d35/d4f to d9/d21/d31/d9f 0 2024-03-21T10:08:56.597 INFO:tasks.workunit.client.0.smithi153.stdout:5/632: link d4/d8/d3b/d56/fca d4/d5/fd0 0 2024-03-21T10:08:56.600 INFO:tasks.workunit.client.0.smithi153.stdout:5/633: read - d4/d5/df/d67/da0/db3/f74 zero size 2024-03-21T10:08:56.601 INFO:tasks.workunit.client.0.smithi153.stdout:5/634: chown d4/d8/f26 1165 1 2024-03-21T10:08:56.603 INFO:tasks.workunit.client.1.smithi174.stdout:0/454: stat d9/d35/d4a/d67/d89/d99 0 2024-03-21T10:08:56.603 INFO:tasks.workunit.client.1.smithi174.stdout:3/450: symlink d0/d12/d14/d1a/d44/d65/d7e/l9c 0 2024-03-21T10:08:56.610 INFO:tasks.workunit.client.1.smithi174.stdout:6/353: dwrite d0/d3/d5/d21/f6c [0,4194304] 0 2024-03-21T10:08:56.611 INFO:tasks.workunit.client.0.smithi153.stdout:9/644: truncate d1/d8a/fb9 621489 0 2024-03-21T10:08:56.611 INFO:tasks.workunit.client.0.smithi153.stdout:0/558: link d4/d16/d19/d30/d6c/d98/l60 d4/d16/d19/d30/da7/lae 0 2024-03-21T10:08:56.618 INFO:tasks.workunit.client.0.smithi153.stdout:9/645: read d1/d43/d44/d46/d9e/d61/f71 [2555343,74270] 0 2024-03-21T10:08:56.619 INFO:tasks.workunit.client.1.smithi174.stdout:5/501: creat d1/d49/d93/d9c/fb2 x:0 0 0 2024-03-21T10:08:56.620 INFO:tasks.workunit.client.0.smithi153.stdout:8/607: dread d5/d1b/d49/f67 [0,4194304] 0 2024-03-21T10:08:56.621 INFO:tasks.workunit.client.0.smithi153.stdout:8/608: stat f3 0 2024-03-21T10:08:56.621 INFO:tasks.workunit.client.1.smithi174.stdout:7/395: creat d0/d4/d10/d44/d81/f87 x:0 0 0 2024-03-21T10:08:56.622 INFO:tasks.workunit.client.0.smithi153.stdout:0/559: rename d4/d16/d19/d30/d6c/d98/l53 to d4/d16/d19/d30/da7/laf 0 2024-03-21T10:08:56.623 INFO:tasks.workunit.client.0.smithi153.stdout:8/609: fsync d5/d9/d43/d7b/fa2 0 2024-03-21T10:08:56.624 INFO:tasks.workunit.client.1.smithi174.stdout:5/502: chown d1/d2f/f4c 74983 1 2024-03-21T10:08:56.624 INFO:tasks.workunit.client.0.smithi153.stdout:9/646: unlink d1/d4/d5f/l6c 0 2024-03-21T10:08:56.625 INFO:tasks.workunit.client.0.smithi153.stdout:0/560: write d4/d4c/f81 [380655,34462] 0 2024-03-21T10:08:56.631 INFO:tasks.workunit.client.0.smithi153.stdout:0/561: chown d4/d16/d19/c51 5491 1 2024-03-21T10:08:56.632 INFO:tasks.workunit.client.1.smithi174.stdout:9/495: dwrite d2/d11/d1d/d2a/f73 [4194304,4194304] 0 2024-03-21T10:08:56.636 INFO:tasks.workunit.client.0.smithi153.stdout:6/803: write d1f/d2b/db3/fcc [597054,77678] 0 2024-03-21T10:08:56.637 INFO:tasks.workunit.client.0.smithi153.stdout:7/509: write d4/d2a/f3e [1047355,102549] 0 2024-03-21T10:08:56.638 INFO:tasks.workunit.client.1.smithi174.stdout:1/473: write d4/d7/d15/f41 [1950918,69771] 0 2024-03-21T10:08:56.639 INFO:tasks.workunit.client.1.smithi174.stdout:3/451: write d0/d12/d14/f3a [5433495,5494] 0 2024-03-21T10:08:56.640 INFO:tasks.workunit.client.0.smithi153.stdout:8/610: rename d5/d9/d43/dd/d20/d26/d8b/f63 to d5/d9/d43/dd/d20/d26/d47/d9a/db9/fcb 0 2024-03-21T10:08:56.642 INFO:tasks.workunit.client.0.smithi153.stdout:7/510: rename d4/d2a to d4/d2a/d3b/d81/daa 22 2024-03-21T10:08:56.643 INFO:tasks.workunit.client.1.smithi174.stdout:5/503: mknod d1/d49/d4f/d51/d8f/cb3 0 2024-03-21T10:08:56.643 INFO:tasks.workunit.client.1.smithi174.stdout:4/500: mkdir d4/d21/d31/d3a/d7a/d9d/da3 0 2024-03-21T10:08:56.643 INFO:tasks.workunit.client.0.smithi153.stdout:6/804: mkdir d1f/d2b/d10a/d40/d51/d11d 0 2024-03-21T10:08:56.647 INFO:tasks.workunit.client.0.smithi153.stdout:8/611: dread - d5/d9/d43/dd/d20/d26/d8b/f4f zero size 2024-03-21T10:08:56.648 INFO:tasks.workunit.client.0.smithi153.stdout:0/562: link d4/d16/d19/d30/d6c/d98/c84 d4/d4c/d83/d22/d61/cb0 0 2024-03-21T10:08:56.648 INFO:tasks.workunit.client.0.smithi153.stdout:6/805: rmdir d1f/d2b/d10a/d40/d118 39 2024-03-21T10:08:56.650 INFO:tasks.workunit.client.0.smithi153.stdout:0/563: read d4/f28 [1865200,56431] 0 2024-03-21T10:08:56.650 INFO:tasks.workunit.client.1.smithi174.stdout:8/380: creat d9/f7c x:0 0 0 2024-03-21T10:08:56.650 INFO:tasks.workunit.client.1.smithi174.stdout:2/463: write d5/d25/d27/f40 [858989,69373] 0 2024-03-21T10:08:56.651 INFO:tasks.workunit.client.1.smithi174.stdout:9/496: creat d2/d24/d2e/d7e/faf x:0 0 0 2024-03-21T10:08:56.652 INFO:tasks.workunit.client.0.smithi153.stdout:5/635: dwrite d4/d5/de/d2f/f41 [0,4194304] 0 2024-03-21T10:08:56.653 INFO:tasks.workunit.client.1.smithi174.stdout:8/381: readlink d9/d2a/d39/d76/l57 0 2024-03-21T10:08:56.653 INFO:tasks.workunit.client.0.smithi153.stdout:7/511: dread d4/fa7 [0,4194304] 0 2024-03-21T10:08:56.654 INFO:tasks.workunit.client.0.smithi153.stdout:6/806: creat d1f/d2b/d10a/d40/d4f/f11e x:0 0 0 2024-03-21T10:08:56.655 INFO:tasks.workunit.client.1.smithi174.stdout:9/497: stat d2/d24/d2e/l51 0 2024-03-21T10:08:56.656 INFO:tasks.workunit.client.0.smithi153.stdout:7/512: fsync d4/d19/d4d/f95 0 2024-03-21T10:08:56.658 INFO:tasks.workunit.client.1.smithi174.stdout:9/498: dread - d2/d24/d2e/d63/fad zero size 2024-03-21T10:08:56.659 INFO:tasks.workunit.client.0.smithi153.stdout:8/612: chown d5/d9/d43/dd/d20/d26/d24/f82 4659 1 2024-03-21T10:08:56.661 INFO:tasks.workunit.client.0.smithi153.stdout:5/636: unlink d4/d5/d16/f1e 0 2024-03-21T10:08:56.661 INFO:tasks.workunit.client.1.smithi174.stdout:6/354: write d0/d3/d5/d6/f2d [997186,95535] 0 2024-03-21T10:08:56.662 INFO:tasks.workunit.client.0.smithi153.stdout:8/613: chown d5/d9/d43/dd/d2d/cb3 1322 1 2024-03-21T10:08:56.662 INFO:tasks.workunit.client.0.smithi153.stdout:0/564: creat d4/d4c/d83/d22/d61/d77/fb1 x:0 0 0 2024-03-21T10:08:56.664 INFO:tasks.workunit.client.1.smithi174.stdout:3/452: chown d0/d12/d14/d1a/d27/d4f/l7c 19037 1 2024-03-21T10:08:56.664 INFO:tasks.workunit.client.0.smithi153.stdout:7/513: mkdir d4/d2a/dab 0 2024-03-21T10:08:56.664 INFO:tasks.workunit.client.1.smithi174.stdout:5/504: unlink d1/d2f/d37/f3f 0 2024-03-21T10:08:56.665 INFO:tasks.workunit.client.0.smithi153.stdout:5/637: creat d4/d5/df/d67/da0/db3/d8d/fd1 x:0 0 0 2024-03-21T10:08:56.666 INFO:tasks.workunit.client.0.smithi153.stdout:6/807: mknod d1f/de0/de1/c11f 0 2024-03-21T10:08:56.667 INFO:tasks.workunit.client.1.smithi174.stdout:4/501: creat d4/d19/fa4 x:0 0 0 2024-03-21T10:08:56.668 INFO:tasks.workunit.client.1.smithi174.stdout:5/505: write d1/d16/f40 [4180313,71724] 0 2024-03-21T10:08:56.669 INFO:tasks.workunit.client.1.smithi174.stdout:4/502: stat d4/d6c 0 2024-03-21T10:08:56.670 INFO:tasks.workunit.client.0.smithi153.stdout:0/565: mkdir d4/d16/d19/d30/d6c/d98/d5e/db2 0 2024-03-21T10:08:56.670 INFO:tasks.workunit.client.1.smithi174.stdout:4/503: readlink d4/l57 0 2024-03-21T10:08:56.672 INFO:tasks.workunit.client.0.smithi153.stdout:7/514: symlink d4/d2a/d40/d97/lac 0 2024-03-21T10:08:56.673 INFO:tasks.workunit.client.1.smithi174.stdout:7/396: rename d0/d4/d10/d60/f6a to d0/f88 0 2024-03-21T10:08:56.677 INFO:tasks.workunit.client.0.smithi153.stdout:7/515: mknod d4/d1b/d20/cad 0 2024-03-21T10:08:56.681 INFO:tasks.workunit.client.1.smithi174.stdout:6/355: write d0/d3/f5a [1233856,9645] 0 2024-03-21T10:08:56.681 INFO:tasks.workunit.client.0.smithi153.stdout:7/516: creat d4/d2a/d40/d50/fae x:0 0 0 2024-03-21T10:08:56.683 INFO:tasks.workunit.client.0.smithi153.stdout:7/517: readlink d4/l52 0 2024-03-21T10:08:56.684 INFO:tasks.workunit.client.1.smithi174.stdout:3/453: creat d0/d12/d14/d1a/d44/d94/f9d x:0 0 0 2024-03-21T10:08:56.687 INFO:tasks.workunit.client.1.smithi174.stdout:0/455: getdents d9/d21/d31 0 2024-03-21T10:08:56.687 INFO:tasks.workunit.client.0.smithi153.stdout:7/518: chown d4/d2a/f2d 55018 1 2024-03-21T10:08:56.687 INFO:tasks.workunit.client.0.smithi153.stdout:6/808: write d1f/d93/dd8/d59/fbc [596356,81535] 0 2024-03-21T10:08:56.693 INFO:tasks.workunit.client.1.smithi174.stdout:2/464: rename d5/d25/d27/d2d/f43 to d5/d25/d27/d2e/d4f/d65/fa2 0 2024-03-21T10:08:56.693 INFO:tasks.workunit.client.0.smithi153.stdout:6/809: fdatasync d1f/d2b/d10a/f3c 0 2024-03-21T10:08:56.697 INFO:tasks.workunit.client.0.smithi153.stdout:0/566: dwrite d4/f10 [4194304,4194304] 0 2024-03-21T10:08:56.698 INFO:tasks.workunit.client.0.smithi153.stdout:9/647: dread d1/d43/d44/d46/d9e/d24/d41/f47 [0,4194304] 0 2024-03-21T10:08:56.698 INFO:tasks.workunit.client.0.smithi153.stdout:7/519: read d4/d14/d2f/d78/f35 [2530401,27944] 0 2024-03-21T10:08:56.700 INFO:tasks.workunit.client.0.smithi153.stdout:6/810: stat d1f/d2b/d10a/d40/d4f/ld4 0 2024-03-21T10:08:56.700 INFO:tasks.workunit.client.0.smithi153.stdout:8/614: dwrite d5/d9/d43/dd/d20/d26/d24/f82 [0,4194304] 0 2024-03-21T10:08:56.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:56 smithi174 ceph-mon[37496]: pgmap v14: 65 pgs: 65 active+clean; 2.2 GiB data, 6.5 GiB used, 530 GiB / 536 GiB avail; 71 MiB/s rd, 310 MiB/s wr, 547 op/s 2024-03-21T10:08:56.702 INFO:tasks.workunit.client.1.smithi174.stdout:1/474: getdents d4/d7/d33/d43/d51 0 2024-03-21T10:08:56.703 INFO:tasks.workunit.client.1.smithi174.stdout:0/456: creat d9/d21/d31/d9f/d79/fa0 x:0 0 0 2024-03-21T10:08:56.704 INFO:tasks.workunit.client.0.smithi153.stdout:0/567: fdatasync d4/d4c/d83/f91 0 2024-03-21T10:08:56.707 INFO:tasks.workunit.client.0.smithi153.stdout:9/648: write d1/fd9 [1010772,4389] 0 2024-03-21T10:08:56.707 INFO:tasks.workunit.client.1.smithi174.stdout:8/382: rename d9/d53/d5b/f7a to d9/d2a/d39/d76/d19/f7d 0 2024-03-21T10:08:56.710 INFO:tasks.workunit.client.0.smithi153.stdout:7/520: symlink d4/d11/da1/da6/laf 0 2024-03-21T10:08:56.710 INFO:tasks.workunit.client.1.smithi174.stdout:2/465: symlink d5/d25/d27/d2e/d85/la3 0 2024-03-21T10:08:56.711 INFO:tasks.workunit.client.0.smithi153.stdout:5/638: dread d4/d5/de/f3f [0,4194304] 0 2024-03-21T10:08:56.712 INFO:tasks.workunit.client.1.smithi174.stdout:3/454: mkdir d0/d12/d14/d4c/d7a/d9e 0 2024-03-21T10:08:56.713 INFO:tasks.workunit.client.0.smithi153.stdout:9/649: chown d1/d43/d44/d46/d9e/d24/d41/dbe/dcc/l95 14 1 2024-03-21T10:08:56.713 INFO:tasks.workunit.client.1.smithi174.stdout:4/504: link d4/d21/d31/d6a/d8f/l96 d4/d21/d24/d28/d38/la5 0 2024-03-21T10:08:56.714 INFO:tasks.workunit.client.1.smithi174.stdout:0/457: mkdir d9/d21/d75/da1 0 2024-03-21T10:08:56.715 INFO:tasks.workunit.client.0.smithi153.stdout:9/650: dread - d1/d4/fd6 zero size 2024-03-21T10:08:56.715 INFO:tasks.workunit.client.1.smithi174.stdout:2/466: read d5/f12 [1429732,5456] 0 2024-03-21T10:08:56.715 INFO:tasks.workunit.client.1.smithi174.stdout:8/383: mkdir d9/d53/d7e 0 2024-03-21T10:08:56.718 INFO:tasks.workunit.client.0.smithi153.stdout:6/811: link d1f/d2b/db3/fcc d1f/d2b/d10a/d40/d51/d98/f120 0 2024-03-21T10:08:56.718 INFO:tasks.workunit.client.0.smithi153.stdout:7/521: truncate d4/d14/d73/fa2 628175 0 2024-03-21T10:08:56.719 INFO:tasks.workunit.client.1.smithi174.stdout:4/505: fsync d4/d21/d31/d3a/f8e 0 2024-03-21T10:08:56.719 INFO:tasks.workunit.client.1.smithi174.stdout:5/506: rename d1/d16/c54 to d1/d49/d4f/d51/d8f/cb4 0 2024-03-21T10:08:56.720 INFO:tasks.workunit.client.0.smithi153.stdout:7/522: stat d4/d2a/d40/d50/f7b 0 2024-03-21T10:08:56.721 INFO:tasks.workunit.client.0.smithi153.stdout:6/812: chown d1f/d2b/d10a/d40/d7b/c102 54 1 2024-03-21T10:08:56.723 INFO:tasks.workunit.client.0.smithi153.stdout:8/615: getdents d5/d9/d43/d7b/dc4/da7 0 2024-03-21T10:08:56.723 INFO:tasks.workunit.client.1.smithi174.stdout:5/507: stat d1/d16/d1c 0 2024-03-21T10:08:56.724 INFO:tasks.workunit.client.0.smithi153.stdout:9/651: symlink d1/d43/lde 0 2024-03-21T10:08:56.725 INFO:tasks.workunit.client.0.smithi153.stdout:0/568: getdents d4/d4c/d83 0 2024-03-21T10:08:56.729 INFO:tasks.workunit.client.1.smithi174.stdout:2/467: read d5/d25/d27/d2d/d45/d5b/d2f/d66/f93 [2941404,10080] 0 2024-03-21T10:08:56.730 INFO:tasks.workunit.client.0.smithi153.stdout:0/569: symlink d4/d16/d19/dad/lb3 0 2024-03-21T10:08:56.730 INFO:tasks.workunit.client.1.smithi174.stdout:5/508: unlink d1/c74 0 2024-03-21T10:08:56.731 INFO:tasks.workunit.client.0.smithi153.stdout:8/616: chown d5/d9/d43/dd/d20/d26/d24/l27 6438 1 2024-03-21T10:08:56.731 INFO:tasks.workunit.client.1.smithi174.stdout:1/475: dwrite d4/d7/d68/d7b/f8c [0,4194304] 0 2024-03-21T10:08:56.732 INFO:tasks.workunit.client.1.smithi174.stdout:9/499: dwrite d2/d24/f59 [0,4194304] 0 2024-03-21T10:08:56.733 INFO:tasks.workunit.client.1.smithi174.stdout:5/509: dread - d1/d49/f53 zero size 2024-03-21T10:08:56.734 INFO:tasks.workunit.client.0.smithi153.stdout:9/652: rmdir d1/d43/d44/d46/d9e/d24/d41/dbd 0 2024-03-21T10:08:56.734 INFO:tasks.workunit.client.1.smithi174.stdout:2/468: write d5/d25/d27/d2e/d7b/f99 [736614,104582] 0 2024-03-21T10:08:56.736 INFO:tasks.workunit.client.0.smithi153.stdout:8/617: mknod d5/d9/d43/d7b/daa/dca/ccc 0 2024-03-21T10:08:56.738 INFO:tasks.workunit.client.0.smithi153.stdout:5/639: dwrite d4/d8/d5a/d5d/f5f [0,4194304] 0 2024-03-21T10:08:56.742 INFO:tasks.workunit.client.0.smithi153.stdout:0/570: rename d4/d16/d19/d58/f6a to d4/d16/d19/d58/d9a/fb4 0 2024-03-21T10:08:56.742 INFO:tasks.workunit.client.1.smithi174.stdout:1/476: chown d4/d67/f74 4643 1 2024-03-21T10:08:56.744 INFO:tasks.workunit.client.1.smithi174.stdout:6/356: truncate d0/d3/d5/df/d16/f1a 3460891 0 2024-03-21T10:08:56.747 INFO:tasks.workunit.client.0.smithi153.stdout:7/523: dwrite d4/d14/d16/f42 [0,4194304] 0 2024-03-21T10:08:56.753 INFO:tasks.workunit.client.1.smithi174.stdout:1/477: chown d4/d7/d68/d8e 0 1 2024-03-21T10:08:56.755 INFO:tasks.workunit.client.1.smithi174.stdout:0/458: truncate d9/f33 579336 0 2024-03-21T10:08:56.756 INFO:tasks.workunit.client.1.smithi174.stdout:3/455: dwrite d0/d12/d14/d1a/d27/d3c/f81 [0,4194304] 0 2024-03-21T10:08:56.757 INFO:tasks.workunit.client.0.smithi153.stdout:6/813: write d1f/d2b/d10a/d40/d51/fc2 [839526,129566] 0 2024-03-21T10:08:56.759 INFO:tasks.workunit.client.1.smithi174.stdout:8/384: dwrite d9/d2a/d39/d76/d19/d31/f3b [4194304,4194304] 0 2024-03-21T10:08:56.761 INFO:tasks.workunit.client.0.smithi153.stdout:7/524: rename d4/d2a/d40/c46 to d4/d11/d56/cb0 0 2024-03-21T10:08:56.763 INFO:tasks.workunit.client.0.smithi153.stdout:8/618: dread d5/d9/d43/dd/d20/d26/d24/d52/d55/f71 [0,4194304] 0 2024-03-21T10:08:56.769 INFO:tasks.workunit.client.0.smithi153.stdout:0/571: rmdir d4/d16/d19/d30/d6c/d98/d5b/dab 0 2024-03-21T10:08:56.770 INFO:tasks.workunit.client.1.smithi174.stdout:1/478: read d4/d7/d28/f3b [162138,30170] 0 2024-03-21T10:08:56.771 INFO:tasks.workunit.client.1.smithi174.stdout:8/385: mknod d9/d2a/d39/d76/d19/d27/c7f 0 2024-03-21T10:08:56.774 INFO:tasks.workunit.client.1.smithi174.stdout:2/469: dwrite d5/d25/d27/d2e/d7b/f8e [0,4194304] 0 2024-03-21T10:08:56.774 INFO:tasks.workunit.client.1.smithi174.stdout:8/386: stat d9/d2a/d39/c4e 0 2024-03-21T10:08:56.775 INFO:tasks.workunit.client.0.smithi153.stdout:6/814: symlink d1f/d93/dd8/l121 0 2024-03-21T10:08:56.776 INFO:tasks.workunit.client.0.smithi153.stdout:9/653: write d1/d43/d44/d46/d9e/d61/f71 [3544727,60684] 0 2024-03-21T10:08:56.777 INFO:tasks.workunit.client.1.smithi174.stdout:3/456: rename d0/d12/d14/l41 to d0/d12/d14/d1a/d44/d65/l9f 0 2024-03-21T10:08:56.782 INFO:tasks.workunit.client.1.smithi174.stdout:3/457: write d0/d12/d14/d1a/d44/d94/f9d [512066,17839] 0 2024-03-21T10:08:56.783 INFO:tasks.workunit.client.0.smithi153.stdout:8/619: rename d5/d9/d43/dd/d2d/d45/d88/d85/lbc to d5/d9/d43/dd/d2d/d45/d88/lcd 0 2024-03-21T10:08:56.784 INFO:tasks.workunit.client.1.smithi174.stdout:1/479: symlink d4/d7/d33/la5 0 2024-03-21T10:08:56.784 INFO:tasks.workunit.client.0.smithi153.stdout:8/620: readlink d5/d9/l79 0 2024-03-21T10:08:56.787 INFO:tasks.workunit.client.0.smithi153.stdout:0/572: truncate d4/db/f11 6305777 0 2024-03-21T10:08:56.789 INFO:tasks.workunit.client.0.smithi153.stdout:9/654: symlink d1/d4/d5f/db6/ldf 0 2024-03-21T10:08:56.790 INFO:tasks.workunit.client.0.smithi153.stdout:6/815: chown d1f/d93/dd8/f7d 439 1 2024-03-21T10:08:56.794 INFO:tasks.workunit.client.1.smithi174.stdout:6/357: write d0/d3/d5/d6/f44 [834806,57484] 0 2024-03-21T10:08:56.796 INFO:tasks.workunit.client.1.smithi174.stdout:1/480: mknod d4/d7/d15/d53/d78/ca6 0 2024-03-21T10:08:56.797 INFO:tasks.workunit.client.0.smithi153.stdout:6/816: rename d1f/d2b/d10a/d40/fa5 to d1f/d2b/d10a/d40/d7b/f122 0 2024-03-21T10:08:56.801 INFO:tasks.workunit.client.1.smithi174.stdout:2/470: getdents d5/d25/d27/d2e 0 2024-03-21T10:08:56.802 INFO:tasks.workunit.client.1.smithi174.stdout:3/458: dread d0/d12/d14/d1a/d27/d3c/f81 [0,4194304] 0 2024-03-21T10:08:56.802 INFO:tasks.workunit.client.1.smithi174.stdout:6/358: fdatasync d0/d3/d5/d6/f4a 0 2024-03-21T10:08:56.803 INFO:tasks.workunit.client.1.smithi174.stdout:1/481: write d4/d7/d32/f3c [6339295,21843] 0 2024-03-21T10:08:56.805 INFO:tasks.workunit.client.0.smithi153.stdout:9/655: rename d1/d43/d44/d46/d9e/d4b/lc5 to d1/d4e/d68/le0 0 2024-03-21T10:08:56.806 INFO:tasks.workunit.client.0.smithi153.stdout:6/817: mknod d1f/d93/df7/c123 0 2024-03-21T10:08:56.806 INFO:tasks.workunit.client.1.smithi174.stdout:8/387: dread d9/d2a/f51 [0,4194304] 0 2024-03-21T10:08:56.809 INFO:tasks.workunit.client.0.smithi153.stdout:9/656: symlink d1/d4e/d68/le1 0 2024-03-21T10:08:56.814 INFO:tasks.workunit.client.0.smithi153.stdout:8/621: dwrite d5/d9/d43/dd/d20/d26/d8b/faf [0,4194304] 0 2024-03-21T10:08:56.815 INFO:tasks.workunit.client.1.smithi174.stdout:1/482: creat d4/d5/d6c/fa7 x:0 0 0 2024-03-21T10:08:56.816 INFO:tasks.workunit.client.0.smithi153.stdout:9/657: creat d1/d4/d5f/db6/fe2 x:0 0 0 2024-03-21T10:08:56.817 INFO:tasks.workunit.client.1.smithi174.stdout:6/359: rename d0/d3/d20/l52 to d0/d3/d5/l72 0 2024-03-21T10:08:56.820 INFO:tasks.workunit.client.1.smithi174.stdout:1/483: rmdir d4/d7/d68/d7b 39 2024-03-21T10:08:56.821 INFO:tasks.workunit.client.0.smithi153.stdout:0/573: write d4/d16/d19/d30/d6c/d98/d5e/f78 [1290458,47422] 0 2024-03-21T10:08:56.822 INFO:tasks.workunit.client.0.smithi153.stdout:9/658: fdatasync d1/d43/d44/f3e 0 2024-03-21T10:08:56.823 INFO:tasks.workunit.client.1.smithi174.stdout:0/459: dread d9/f33 [0,4194304] 0 2024-03-21T10:08:56.824 INFO:tasks.workunit.client.1.smithi174.stdout:5/510: dread d1/d16/f1a [0,4194304] 0 2024-03-21T10:08:56.824 INFO:tasks.workunit.client.1.smithi174.stdout:6/360: rmdir d0/d3/d5/d3b 39 2024-03-21T10:08:56.829 INFO:tasks.workunit.client.0.smithi153.stdout:0/574: unlink d4/d16/d19/c45 0 2024-03-21T10:08:56.829 INFO:tasks.workunit.client.0.smithi153.stdout:7/525: dread d4/d2a/d40/d50/f7b [0,4194304] 0 2024-03-21T10:08:56.832 INFO:tasks.workunit.client.1.smithi174.stdout:2/471: dwrite d5/d25/d27/d2d/d45/d5b/d2f/d4b/d76/f7f [0,4194304] 0 2024-03-21T10:08:56.833 INFO:tasks.workunit.client.0.smithi153.stdout:9/659: symlink d1/d43/d44/d46/d9e/d24/le3 0 2024-03-21T10:08:56.833 INFO:tasks.workunit.client.0.smithi153.stdout:0/575: chown d4/d16/d27/f4d 402096 1 2024-03-21T10:08:56.834 INFO:tasks.workunit.client.0.smithi153.stdout:7/526: mkdir d4/d11/d56/db1 0 2024-03-21T10:08:56.837 INFO:tasks.workunit.client.0.smithi153.stdout:0/576: write d4/f62 [48626,122344] 0 2024-03-21T10:08:56.841 INFO:tasks.workunit.client.1.smithi174.stdout:0/460: mkdir d9/da2 0 2024-03-21T10:08:56.841 INFO:tasks.workunit.client.0.smithi153.stdout:6/818: dwrite d1f/d2b/d10a/d40/d4f/f67 [0,4194304] 0 2024-03-21T10:08:56.841 INFO:tasks.workunit.client.0.smithi153.stdout:2/712: sync 2024-03-21T10:08:56.842 INFO:tasks.workunit.client.0.smithi153.stdout:1/514: sync 2024-03-21T10:08:56.842 INFO:tasks.workunit.client.0.smithi153.stdout:7/527: symlink d4/lb2 0 2024-03-21T10:08:56.844 INFO:tasks.workunit.client.1.smithi174.stdout:2/472: creat d5/d25/fa4 x:0 0 0 2024-03-21T10:08:56.845 INFO:tasks.workunit.client.1.smithi174.stdout:5/511: write d1/d16/f60 [629211,93116] 0 2024-03-21T10:08:56.846 INFO:tasks.workunit.client.1.smithi174.stdout:6/361: dread d0/d3/d5/d21/f6c [0,4194304] 0 2024-03-21T10:08:56.849 INFO:tasks.workunit.client.0.smithi153.stdout:7/528: truncate d4/d2a/d3b/d81/fa9 770649 0 2024-03-21T10:08:56.850 INFO:tasks.workunit.client.1.smithi174.stdout:3/459: dwrite d0/d12/f49 [0,4194304] 0 2024-03-21T10:08:56.852 INFO:tasks.workunit.client.0.smithi153.stdout:8/622: dwrite d5/d9/d43/f8c [0,4194304] 0 2024-03-21T10:08:56.856 INFO:tasks.workunit.client.0.smithi153.stdout:8/623: readlink d5/d9/d43/dd/d20/l83 0 2024-03-21T10:08:56.859 INFO:tasks.workunit.client.0.smithi153.stdout:1/515: truncate d6/fc 350708 0 2024-03-21T10:08:56.859 INFO:tasks.workunit.client.1.smithi174.stdout:8/388: dwrite d9/d2a/d39/f5f [0,4194304] 0 2024-03-21T10:08:56.859 INFO:tasks.workunit.client.1.smithi174.stdout:5/512: creat d1/d16/fb5 x:0 0 0 2024-03-21T10:08:56.862 INFO:tasks.workunit.client.1.smithi174.stdout:5/513: read - d1/f91 zero size 2024-03-21T10:08:56.863 INFO:tasks.workunit.client.0.smithi153.stdout:1/516: dread - d6/d32/f9a zero size 2024-03-21T10:08:56.864 INFO:tasks.workunit.client.0.smithi153.stdout:9/660: link d1/d43/d44/l28 d1/d43/d44/dc3/le4 0 2024-03-21T10:08:56.865 INFO:tasks.workunit.client.1.smithi174.stdout:3/460: readlink d0/d12/d14/d1a/d44/d57/d93/l3e 0 2024-03-21T10:08:56.866 INFO:tasks.workunit.client.0.smithi153.stdout:7/529: mknod d4/d2a/d3b/cb3 0 2024-03-21T10:08:56.867 INFO:tasks.workunit.client.0.smithi153.stdout:9/661: stat d1/d43/lde 0 2024-03-21T10:08:56.872 INFO:tasks.workunit.client.0.smithi153.stdout:6/819: symlink d1f/d93/dd8/d59/d70/d110/l124 0 2024-03-21T10:08:56.872 INFO:tasks.workunit.client.0.smithi153.stdout:2/713: creat dd/d15/d3c/da7/d91/dae/ff4 x:0 0 0 2024-03-21T10:08:56.873 INFO:tasks.workunit.client.0.smithi153.stdout:0/577: dwrite d4/d4c/f7a [0,4194304] 0 2024-03-21T10:08:56.877 INFO:tasks.workunit.client.0.smithi153.stdout:8/624: rename d5/d9/d43/d7b/dc4/f6f to d5/d9/d43/dd/d20/fce 0 2024-03-21T10:08:56.878 INFO:tasks.workunit.client.1.smithi174.stdout:5/514: rename d1/d16/fb5 to d1/d49/d4f/d51/d8f/fb6 0 2024-03-21T10:08:56.880 INFO:tasks.workunit.client.0.smithi153.stdout:0/578: stat d4/d4c/d83/d22/d61/d77/c8a 0 2024-03-21T10:08:56.883 INFO:tasks.workunit.client.1.smithi174.stdout:2/473: write d5/d25/d27/d2d/d45/d5b/f1d [442020,34398] 0 2024-03-21T10:08:56.883 INFO:tasks.workunit.client.0.smithi153.stdout:7/530: rename d4/d19/c47 to d4/d14/d2f/d78/d84/cb4 0 2024-03-21T10:08:56.883 INFO:tasks.workunit.client.1.smithi174.stdout:5/515: readlink d1/d16/l45 0 2024-03-21T10:08:56.884 INFO:tasks.workunit.client.0.smithi153.stdout:6/820: rename d1f to d1f/d99/dbb/dcf/d125 22 2024-03-21T10:08:56.885 INFO:tasks.workunit.client.0.smithi153.stdout:9/662: creat d1/d43/d44/d46/d9e/d61/fe5 x:0 0 0 2024-03-21T10:08:56.887 INFO:tasks.workunit.client.0.smithi153.stdout:6/821: readlink d1f/d2b/l104 0 2024-03-21T10:08:56.888 INFO:tasks.workunit.client.1.smithi174.stdout:1/484: dwrite d4/d2d/f5e [0,4194304] 0 2024-03-21T10:08:56.888 INFO:tasks.workunit.client.1.smithi174.stdout:6/362: dread d0/d3/d5/df/d16/f65 [0,4194304] 0 2024-03-21T10:08:56.889 INFO:tasks.workunit.client.0.smithi153.stdout:7/531: write d4/d2a/d40/d50/fae [145142,80244] 0 2024-03-21T10:08:56.891 INFO:tasks.workunit.client.0.smithi153.stdout:8/625: rename d5/d9/d43/dd/cc8 to d5/d9/d43/dd/d20/d26/d47/d9a/ccf 0 2024-03-21T10:08:56.895 INFO:tasks.workunit.client.0.smithi153.stdout:9/663: creat d1/d4/d5f/fe6 x:0 0 0 2024-03-21T10:08:56.895 INFO:tasks.workunit.client.0.smithi153.stdout:0/579: creat d4/d16/d19/d95/fb5 x:0 0 0 2024-03-21T10:08:56.895 INFO:tasks.workunit.client.0.smithi153.stdout:6/822: symlink d1f/de0/de1/df2/d10f/l126 0 2024-03-21T10:08:56.895 INFO:tasks.workunit.client.1.smithi174.stdout:0/461: truncate d9/d21/d31/d3c/f49 61966 0 2024-03-21T10:08:56.901 INFO:tasks.workunit.client.0.smithi153.stdout:2/714: dread dd/d15/d18/d2c/d4b/d56/f7b [0,4194304] 0 2024-03-21T10:08:56.901 INFO:tasks.workunit.client.0.smithi153.stdout:0/580: dread - d4/d16/d19/f85 zero size 2024-03-21T10:08:56.903 INFO:tasks.workunit.client.0.smithi153.stdout:1/517: dwrite d6/fe [0,4194304] 0 2024-03-21T10:08:56.903 INFO:tasks.workunit.client.1.smithi174.stdout:8/389: rename d9/d2a/d39/d76/f6e to d9/d2a/d39/d76/d19/d27/d60/f80 0 2024-03-21T10:08:56.905 INFO:tasks.workunit.client.1.smithi174.stdout:3/461: creat d0/d12/d14/d1a/d44/d56/fa0 x:0 0 0 2024-03-21T10:08:56.905 INFO:tasks.workunit.client.0.smithi153.stdout:2/715: chown dd/d15/d3c/f69 5 1 2024-03-21T10:08:56.906 INFO:tasks.workunit.client.1.smithi174.stdout:5/516: creat d1/d16/d2e/d66/d8b/fb7 x:0 0 0 2024-03-21T10:08:56.906 INFO:tasks.workunit.client.1.smithi174.stdout:2/474: rmdir d5/d25/d27/d2e/d90 39 2024-03-21T10:08:56.908 INFO:tasks.workunit.client.1.smithi174.stdout:2/475: chown d5/l1b 600 1 2024-03-21T10:08:56.908 INFO:tasks.workunit.client.1.smithi174.stdout:6/363: creat d0/d3/d2a/d56/d5e/f73 x:0 0 0 2024-03-21T10:08:56.909 INFO:tasks.workunit.client.0.smithi153.stdout:6/823: creat d1f/de0/de1/df2/d10f/f127 x:0 0 0 2024-03-21T10:08:56.910 INFO:tasks.workunit.client.0.smithi153.stdout:2/716: stat dd/d15/d3c/da7/d91/dae/d50/d59/d5d/f7f 0 2024-03-21T10:08:56.911 INFO:tasks.workunit.client.0.smithi153.stdout:1/518: write d6/f9 [4192797,38509] 0 2024-03-21T10:08:56.914 INFO:tasks.workunit.client.1.smithi174.stdout:8/390: truncate d9/d2a/d39/d76/d19/f23 2412647 0 2024-03-21T10:08:56.914 INFO:tasks.workunit.client.1.smithi174.stdout:3/462: fdatasync d0/d12/d14/d1a/f40 0 2024-03-21T10:08:56.915 INFO:tasks.workunit.client.0.smithi153.stdout:6/824: rmdir d1f/d93/dd8/d59/dfc 39 2024-03-21T10:08:56.916 INFO:tasks.workunit.client.0.smithi153.stdout:4/639: sync 2024-03-21T10:08:56.916 INFO:tasks.workunit.client.0.smithi153.stdout:3/673: sync 2024-03-21T10:08:56.916 INFO:tasks.workunit.client.0.smithi153.stdout:1/519: mkdir d6/d32/d3e/d4f/d9d 0 2024-03-21T10:08:56.916 INFO:tasks.workunit.client.1.smithi174.stdout:1/485: mkdir d4/d7/d32/da2/da8 0 2024-03-21T10:08:56.917 INFO:tasks.workunit.client.1.smithi174.stdout:5/517: symlink d1/d49/d93/lb8 0 2024-03-21T10:08:56.919 INFO:tasks.workunit.client.1.smithi174.stdout:7/397: sync 2024-03-21T10:08:56.919 INFO:tasks.workunit.client.0.smithi153.stdout:0/581: dread d4/f62 [0,4194304] 0 2024-03-21T10:08:56.921 INFO:tasks.workunit.client.0.smithi153.stdout:9/664: dread d1/d43/d44/d46/d9e/d24/fa1 [0,4194304] 0 2024-03-21T10:08:56.922 INFO:tasks.workunit.client.1.smithi174.stdout:6/364: rename d0/d3/d5/d6/l11 to d0/d3/d5/d3b/l74 0 2024-03-21T10:08:56.922 INFO:tasks.workunit.client.0.smithi153.stdout:4/640: mkdir d11/d18/d7e/dc5 0 2024-03-21T10:08:56.925 INFO:tasks.workunit.client.1.smithi174.stdout:9/500: sync 2024-03-21T10:08:56.925 INFO:tasks.workunit.client.1.smithi174.stdout:4/506: sync 2024-03-21T10:08:56.925 INFO:tasks.workunit.client.0.smithi153.stdout:4/641: dread - d11/d2d/da3/dac/fbf zero size 2024-03-21T10:08:56.927 INFO:tasks.workunit.client.0.smithi153.stdout:3/674: symlink dd/d1e/d2f/d35/daf/dcc/le9 0 2024-03-21T10:08:56.930 INFO:tasks.workunit.client.1.smithi174.stdout:2/476: truncate d5/d25/d27/d2d/d45/d5b/d2f/d68/f7a 1128049 0 2024-03-21T10:08:56.933 INFO:tasks.workunit.client.1.smithi174.stdout:9/501: symlink d2/d11/d1d/lb0 0 2024-03-21T10:08:56.934 INFO:tasks.workunit.client.0.smithi153.stdout:6/825: link d1f/d93/dd8/d59/la1 d1f/d2b/db3/l128 0 2024-03-21T10:08:56.934 INFO:tasks.workunit.client.1.smithi174.stdout:4/507: truncate d4/d21/f7b 741381 0 2024-03-21T10:08:56.936 INFO:tasks.workunit.client.1.smithi174.stdout:7/398: getdents d0/d53 0 2024-03-21T10:08:56.936 INFO:tasks.workunit.client.0.smithi153.stdout:2/717: getdents dd/d15/d3c/da7/d91/dae/d50/d58/d79 0 2024-03-21T10:08:56.936 INFO:tasks.workunit.client.0.smithi153.stdout:6/826: fdatasync d1f/d2b/d10a/d40/d5f/f82 0 2024-03-21T10:08:56.939 INFO:tasks.workunit.client.1.smithi174.stdout:9/502: fsync d2/d24/f88 0 2024-03-21T10:08:56.939 INFO:tasks.workunit.client.1.smithi174.stdout:2/477: chown d5/d25/d27/d2d/d45/d5b/lc 131233182 1 2024-03-21T10:08:56.941 INFO:tasks.workunit.client.0.smithi153.stdout:4/642: creat d11/d2d/d6b/fc6 x:0 0 0 2024-03-21T10:08:56.943 INFO:tasks.workunit.client.0.smithi153.stdout:1/520: rename d6/d13/d75/f91 to d6/d13/d75/f9e 0 2024-03-21T10:08:56.948 INFO:tasks.workunit.client.0.smithi153.stdout:6/827: fsync d1f/d2b/d10a/d40/d4f/f83 0 2024-03-21T10:08:56.950 INFO:tasks.workunit.client.1.smithi174.stdout:9/503: rename d2/f47 to d2/d24/d2e/fb1 0 2024-03-21T10:08:56.950 INFO:tasks.workunit.client.1.smithi174.stdout:7/399: symlink d0/d4/d10/d1b/d86/l89 0 2024-03-21T10:08:56.950 INFO:tasks.workunit.client.1.smithi174.stdout:8/391: dwrite d9/d2a/d4d/f6d [0,4194304] 0 2024-03-21T10:08:56.950 INFO:tasks.workunit.client.0.smithi153.stdout:7/532: write d4/d14/d16/f39 [1932253,89498] 0 2024-03-21T10:08:56.952 INFO:tasks.workunit.client.1.smithi174.stdout:7/400: readlink d0/d4/l20 0 2024-03-21T10:08:56.956 INFO:tasks.workunit.client.1.smithi174.stdout:7/401: write d0/d4/d10/d60/f77 [922950,37534] 0 2024-03-21T10:08:56.957 INFO:tasks.workunit.client.0.smithi153.stdout:5/640: sync 2024-03-21T10:08:56.957 INFO:tasks.workunit.client.0.smithi153.stdout:2/718: rename dd/d15/d3c/da7/d91 to dd/d15/d18/d2c/d4b/d56/d63/d8d/d9b/dc3/df5 0 2024-03-21T10:08:56.960 INFO:tasks.workunit.client.1.smithi174.stdout:2/478: rename d5/d25/d27/l32 to d5/d25/d27/d2d/d45/d5b/d2f/d68/d9e/la5 0 2024-03-21T10:08:56.963 INFO:tasks.workunit.client.0.smithi153.stdout:8/626: write d5/d1b/f87 [402055,99329] 0 2024-03-21T10:08:56.963 INFO:tasks.workunit.client.0.smithi153.stdout:4/643: mkdir d11/d2d/d3d/dc7 0 2024-03-21T10:08:56.964 INFO:tasks.workunit.client.0.smithi153.stdout:3/675: mknod dd/d1e/d3d/d53/cea 0 2024-03-21T10:08:56.964 INFO:tasks.workunit.client.0.smithi153.stdout:1/521: symlink d6/d10/d2c/l9f 0 2024-03-21T10:08:56.964 INFO:tasks.workunit.client.0.smithi153.stdout:4/644: fsync d11/f8d 0 2024-03-21T10:08:56.965 INFO:tasks.workunit.client.0.smithi153.stdout:3/676: stat dd/d1e/da6 0 2024-03-21T10:08:56.967 INFO:tasks.workunit.client.0.smithi153.stdout:3/677: write dd/d41/f71 [4280258,55097] 0 2024-03-21T10:08:56.967 INFO:tasks.workunit.client.1.smithi174.stdout:4/508: link d4/c9 d4/d21/d31/d3a/d7a/d9d/ca6 0 2024-03-21T10:08:56.970 INFO:tasks.workunit.client.1.smithi174.stdout:4/509: rename d4 to d4/da7 22 2024-03-21T10:08:56.970 INFO:tasks.workunit.client.1.smithi174.stdout:2/479: truncate d5/d25/d27/d2d/d45/d5b/d2f/d4b/d76/f97 788390 0 2024-03-21T10:08:56.971 INFO:tasks.workunit.client.1.smithi174.stdout:7/402: creat d0/d4/d10/d60/f8a x:0 0 0 2024-03-21T10:08:56.972 INFO:tasks.workunit.client.0.smithi153.stdout:2/719: rename dd/d15/d18/d3a/d3d/f4f to dd/d15/d18/d2c/ff6 0 2024-03-21T10:08:56.973 INFO:tasks.workunit.client.0.smithi153.stdout:8/627: symlink d5/d9/ld0 0 2024-03-21T10:08:56.975 INFO:tasks.workunit.client.1.smithi174.stdout:5/518: dread d1/ff [4194304,4194304] 0 2024-03-21T10:08:56.977 INFO:tasks.workunit.client.0.smithi153.stdout:2/720: dread dd/d15/d18/d2c/d4b/d56/d63/d8d/d9b/fa5 [0,4194304] 0 2024-03-21T10:08:56.977 INFO:tasks.workunit.client.1.smithi174.stdout:5/519: fdatasync d1/d16/d2e/f80 0 2024-03-21T10:08:56.978 INFO:tasks.workunit.client.0.smithi153.stdout:0/582: write d4/d16/d19/d30/d6c/d98/d5e/f70 [505195,126261] 0 2024-03-21T10:08:56.978 INFO:tasks.workunit.client.0.smithi153.stdout:4/645: dread - d11/d5e/f7d zero size 2024-03-21T10:08:56.979 INFO:tasks.workunit.client.1.smithi174.stdout:2/480: chown d5/d25/d27/d2d/d45/d5b/d2f/d4b/d76/f7f 28866 1 2024-03-21T10:08:56.980 INFO:tasks.workunit.client.0.smithi153.stdout:6/828: creat d1f/d93/dd8/d59/dfc/f129 x:0 0 0 2024-03-21T10:08:56.981 INFO:tasks.workunit.client.1.smithi174.stdout:2/481: write d5/d25/d27/f8b [3493808,63806] 0 2024-03-21T10:08:56.982 INFO:tasks.workunit.client.0.smithi153.stdout:7/533: mknod d4/d11/d56/db1/cb5 0 2024-03-21T10:08:56.983 INFO:tasks.workunit.client.1.smithi174.stdout:4/510: unlink d4/d21/d31/d3a/f8d 0 2024-03-21T10:08:56.985 INFO:tasks.workunit.client.1.smithi174.stdout:6/365: dread d0/d3/d5/d3b/f55 [0,4194304] 0 2024-03-21T10:08:56.985 INFO:tasks.workunit.client.0.smithi153.stdout:6/829: readlink d1f/d2b/d10a/d40/d4f/ld4 0 2024-03-21T10:08:56.987 INFO:tasks.workunit.client.0.smithi153.stdout:3/678: chown dd/d1e/da6/fcf 748 1 2024-03-21T10:08:56.987 INFO:tasks.workunit.client.0.smithi153.stdout:8/628: creat d5/d1b/fd1 x:0 0 0 2024-03-21T10:08:56.989 INFO:tasks.workunit.client.1.smithi174.stdout:8/392: dwrite d9/f30 [0,4194304] 0 2024-03-21T10:08:56.992 INFO:tasks.workunit.client.1.smithi174.stdout:4/511: rmdir d4/d21/d31/d3a/d7a/d9d 39 2024-03-21T10:08:56.993 INFO:tasks.workunit.client.0.smithi153.stdout:3/679: fsync dd/d1e/f4f 0 2024-03-21T10:08:56.994 INFO:tasks.workunit.client.1.smithi174.stdout:5/520: symlink d1/lb9 0 2024-03-21T10:08:56.996 INFO:tasks.workunit.client.0.smithi153.stdout:4/646: chown d11/c6c 2 1 2024-03-21T10:08:56.996 INFO:tasks.workunit.client.0.smithi153.stdout:9/665: dwrite d1/d43/d72/dba/fd0 [0,4194304] 0 2024-03-21T10:08:56.997 INFO:tasks.workunit.client.1.smithi174.stdout:7/403: rename d0/l3f to d0/l8b 0 2024-03-21T10:08:56.999 INFO:tasks.workunit.client.1.smithi174.stdout:6/366: rename d0/d3/d5/df/c5f to d0/d3/d20/d3d/c75 0 2024-03-21T10:08:57.000 INFO:tasks.workunit.client.0.smithi153.stdout:7/534: rename d4/d14/d73/fa2 to d4/d19/fb6 0 2024-03-21T10:08:57.000 INFO:tasks.workunit.client.0.smithi153.stdout:4/647: write d11/f8d [1059858,84620] 0 2024-03-21T10:08:57.006 INFO:tasks.workunit.client.1.smithi174.stdout:5/521: symlink d1/d16/d2e/d66/lba 0 2024-03-21T10:08:57.010 INFO:tasks.workunit.client.0.smithi153.stdout:8/629: rmdir d5/d9/d43/dd/d20/d26/d24 39 2024-03-21T10:08:57.012 INFO:tasks.workunit.client.0.smithi153.stdout:7/535: write d4/d19/d4d/f95 [676890,32300] 0 2024-03-21T10:08:57.012 INFO:tasks.workunit.client.0.smithi153.stdout:2/721: creat dd/d15/ff7 x:0 0 0 2024-03-21T10:08:57.014 INFO:tasks.workunit.client.0.smithi153.stdout:9/666: fsync d1/d8a/fb9 0 2024-03-21T10:08:57.017 INFO:tasks.workunit.client.0.smithi153.stdout:1/522: write d6/d13/f2b [792557,57864] 0 2024-03-21T10:08:57.018 INFO:tasks.workunit.client.1.smithi174.stdout:6/367: write d0/d3/d5/df/d16/f27 [3084135,53024] 0 2024-03-21T10:08:57.018 INFO:tasks.workunit.client.1.smithi174.stdout:7/404: symlink d0/d4/d10/l8c 0 2024-03-21T10:08:57.019 INFO:tasks.workunit.client.0.smithi153.stdout:0/583: dwrite d4/d16/d19/fa2 [0,4194304] 0 2024-03-21T10:08:57.019 INFO:tasks.workunit.client.1.smithi174.stdout:5/522: mknod d1/d49/d93/d9c/cbb 0 2024-03-21T10:08:57.020 INFO:tasks.workunit.client.1.smithi174.stdout:1/486: dwrite d4/d7/d33/d43/d77/f84 [4194304,4194304] 0 2024-03-21T10:08:57.022 INFO:tasks.workunit.client.1.smithi174.stdout:7/405: chown d0/d4/d10/d60/f77 1100785 1 2024-03-21T10:08:57.024 INFO:tasks.workunit.client.0.smithi153.stdout:6/830: dwrite d1f/de0/de1/df2/d10f/f127 [0,4194304] 0 2024-03-21T10:08:57.025 INFO:tasks.workunit.client.0.smithi153.stdout:9/667: read d1/d43/d44/d46/d9e/f31 [72073,29967] 0 2024-03-21T10:08:57.027 INFO:tasks.workunit.client.0.smithi153.stdout:9/668: dread - d1/d43/d44/d46/d4d/fb7 zero size 2024-03-21T10:08:57.029 INFO:tasks.workunit.client.0.smithi153.stdout:5/641: dread d4/d8/f6c [0,4194304] 0 2024-03-21T10:08:57.031 INFO:tasks.workunit.client.1.smithi174.stdout:1/487: rmdir d4/d7/d32/d86 39 2024-03-21T10:08:57.031 INFO:tasks.workunit.client.0.smithi153.stdout:0/584: rmdir d4/d16/d19 39 2024-03-21T10:08:57.033 INFO:tasks.workunit.client.1.smithi174.stdout:9/504: write d2/d11/f4a [5145461,20752] 0 2024-03-21T10:08:57.033 INFO:tasks.workunit.client.0.smithi153.stdout:7/536: rename d4/d14/d2f/d78/d84 to d4/d1b/d20/d9a/db7 0 2024-03-21T10:08:57.034 INFO:tasks.workunit.client.0.smithi153.stdout:1/523: mkdir d6/d10/da0 0 2024-03-21T10:08:57.037 INFO:tasks.workunit.client.0.smithi153.stdout:4/648: write d11/d18/f61 [356405,83011] 0 2024-03-21T10:08:57.038 INFO:tasks.workunit.client.1.smithi174.stdout:8/393: rename d9/d2a/d39/d76/d19/d27/d60/f80 to d9/d2a/d39/f81 0 2024-03-21T10:08:57.038 INFO:tasks.workunit.client.0.smithi153.stdout:5/642: creat d4/d5/df/d67/da0/db3/fd2 x:0 0 0 2024-03-21T10:08:57.039 INFO:tasks.workunit.client.1.smithi174.stdout:9/505: mkdir d2/d11/d60/db2 0 2024-03-21T10:08:57.039 INFO:tasks.workunit.client.1.smithi174.stdout:1/488: creat d4/d7/d70/fa9 x:0 0 0 2024-03-21T10:08:57.041 INFO:tasks.workunit.client.1.smithi174.stdout:7/406: getdents d0/d53 0 2024-03-21T10:08:57.043 INFO:tasks.workunit.client.1.smithi174.stdout:9/506: write d2/d11/f87 [4424970,98296] 0 2024-03-21T10:08:57.044 INFO:tasks.workunit.client.0.smithi153.stdout:1/524: creat d6/d10/da0/fa1 x:0 0 0 2024-03-21T10:08:57.045 INFO:tasks.workunit.client.0.smithi153.stdout:5/643: rename d4/c42 to d4/d8/d5a/d92/d9f/cd3 0 2024-03-21T10:08:57.049 INFO:tasks.workunit.client.0.smithi153.stdout:7/537: getdents d4/d1b/d20/d9a/db7 0 2024-03-21T10:08:57.050 INFO:tasks.workunit.client.0.smithi153.stdout:7/538: read d4/fa7 [226352,27906] 0 2024-03-21T10:08:57.055 INFO:tasks.workunit.client.0.smithi153.stdout:7/539: fdatasync d4/f10 0 2024-03-21T10:08:57.057 INFO:tasks.workunit.client.1.smithi174.stdout:1/489: mknod d4/d7/d33/d43/d51/caa 0 2024-03-21T10:08:57.058 INFO:tasks.workunit.client.1.smithi174.stdout:9/507: creat d2/d11/d1d/d2a/da7/fb3 x:0 0 0 2024-03-21T10:08:57.059 INFO:tasks.workunit.client.1.smithi174.stdout:8/394: mknod d9/d2a/d39/d76/d19/d27/d79/c82 0 2024-03-21T10:08:57.060 INFO:tasks.workunit.client.1.smithi174.stdout:9/508: chown d2/d11/f87 0 1 2024-03-21T10:08:57.062 INFO:tasks.workunit.client.0.smithi153.stdout:5/644: rmdir d4/d5/df/d5e 39 2024-03-21T10:08:57.062 INFO:tasks.workunit.client.0.smithi153.stdout:7/540: fsync d4/d2a/d40/d50/fa5 0 2024-03-21T10:08:57.063 INFO:tasks.workunit.client.1.smithi174.stdout:8/395: unlink d9/f14 0 2024-03-21T10:08:57.066 INFO:tasks.workunit.client.1.smithi174.stdout:9/509: mknod d2/d11/d1d/d8b/cb4 0 2024-03-21T10:08:57.066 INFO:tasks.workunit.client.0.smithi153.stdout:9/669: dwrite d1/d43/d44/d46/d9e/f9f [0,4194304] 0 2024-03-21T10:08:57.069 INFO:tasks.workunit.client.1.smithi174.stdout:1/490: rmdir d4/d7/d15/d53/d6b 0 2024-03-21T10:08:57.069 INFO:tasks.workunit.client.1.smithi174.stdout:9/510: fsync d2/d11/d1d/f41 0 2024-03-21T10:08:57.070 INFO:tasks.workunit.client.0.smithi153.stdout:4/649: dwrite d11/d2d/d3d/f99 [0,4194304] 0 2024-03-21T10:08:57.074 INFO:tasks.workunit.client.1.smithi174.stdout:4/512: truncate d4/d21/d24/f4d 1150421 0 2024-03-21T10:08:57.075 INFO:tasks.workunit.client.1.smithi174.stdout:8/396: unlink d9/c3d 0 2024-03-21T10:08:57.078 INFO:tasks.workunit.client.0.smithi153.stdout:8/630: dread d5/d9/d43/dd/d20/d26/d24/d52/d55/f89 [4194304,4194304] 0 2024-03-21T10:08:57.078 INFO:tasks.workunit.client.1.smithi174.stdout:8/397: dread - d9/d2a/d39/d76/d19/d27/f64 zero size 2024-03-21T10:08:57.079 INFO:tasks.workunit.client.0.smithi153.stdout:7/541: truncate d4/d14/d2f/d78/f3a 8795722 0 2024-03-21T10:08:57.079 INFO:tasks.workunit.client.1.smithi174.stdout:1/491: creat d4/d7/d15/d53/d78/fab x:0 0 0 2024-03-21T10:08:57.079 INFO:tasks.workunit.client.1.smithi174.stdout:9/511: read d2/d24/d2e/f48 [1626022,86703] 0 2024-03-21T10:08:57.081 INFO:tasks.workunit.client.0.smithi153.stdout:9/670: mknod d1/d43/d44/d46/d9e/d4b/ce7 0 2024-03-21T10:08:57.085 INFO:tasks.workunit.client.0.smithi153.stdout:7/542: readlink d4/d11/l94 0 2024-03-21T10:08:57.086 INFO:tasks.workunit.client.1.smithi174.stdout:5/523: truncate d1/d16/d1c/f23 2351902 0 2024-03-21T10:08:57.088 INFO:tasks.workunit.client.0.smithi153.stdout:0/585: dwrite d4/d16/d19/d30/d6c/d98/d5e/f75 [0,4194304] 0 2024-03-21T10:08:57.088 INFO:tasks.workunit.client.0.smithi153.stdout:9/671: chown d1/d43/d44/d46/d9e/lac 34173495 1 2024-03-21T10:08:57.089 INFO:tasks.workunit.client.0.smithi153.stdout:4/650: link d11/d5e/f7d d11/d2d/d3d/da1/fc8 0 2024-03-21T10:08:57.090 INFO:tasks.workunit.client.1.smithi174.stdout:4/513: getdents d4 0 2024-03-21T10:08:57.094 INFO:tasks.workunit.client.1.smithi174.stdout:8/398: symlink d9/d2a/d39/d76/d19/l83 0 2024-03-21T10:08:57.097 INFO:tasks.workunit.client.0.smithi153.stdout:2/722: dread dd/d15/d3c/f69 [0,4194304] 0 2024-03-21T10:08:57.098 INFO:tasks.workunit.client.0.smithi153.stdout:8/631: creat d5/d9/d43/dd/d2d/d45/d88/fd2 x:0 0 0 2024-03-21T10:08:57.100 INFO:tasks.workunit.client.0.smithi153.stdout:1/525: dread d6/d13/f5e [0,4194304] 0 2024-03-21T10:08:57.100 INFO:tasks.workunit.client.0.smithi153.stdout:7/543: creat d4/d14/d73/fb8 x:0 0 0 2024-03-21T10:08:57.101 INFO:tasks.workunit.client.1.smithi174.stdout:9/512: mknod d2/d11/d7d/cb5 0 2024-03-21T10:08:57.101 INFO:tasks.workunit.client.0.smithi153.stdout:4/651: creat d11/d18/d54/fc9 x:0 0 0 2024-03-21T10:08:57.103 INFO:tasks.workunit.client.0.smithi153.stdout:9/672: creat d1/d4/fe8 x:0 0 0 2024-03-21T10:08:57.105 INFO:tasks.workunit.client.1.smithi174.stdout:6/368: dwrite d0/d3/d5/d3b/f51 [0,4194304] 0 2024-03-21T10:08:57.107 INFO:tasks.workunit.client.0.smithi153.stdout:7/544: mkdir d4/d2a/d40/d97/db9 0 2024-03-21T10:08:57.108 INFO:tasks.workunit.client.1.smithi174.stdout:7/407: dread d0/d4/d10/f1c [0,4194304] 0 2024-03-21T10:08:57.108 INFO:tasks.workunit.client.1.smithi174.stdout:0/462: sync 2024-03-21T10:08:57.109 INFO:tasks.workunit.client.0.smithi153.stdout:4/652: dread - d11/d2d/f92 zero size 2024-03-21T10:08:57.109 INFO:tasks.workunit.client.0.smithi153.stdout:8/632: creat d5/d9/d43/d7b/daa/fd3 x:0 0 0 2024-03-21T10:08:57.109 INFO:tasks.workunit.client.0.smithi153.stdout:1/526: mkdir d6/d3a/da2 0 2024-03-21T10:08:57.110 INFO:tasks.workunit.client.0.smithi153.stdout:0/586: link d4/d16/d19/d30/d6c/d98/c49 d4/d16/cb6 0 2024-03-21T10:08:57.112 INFO:tasks.workunit.client.1.smithi174.stdout:8/399: creat d9/d2a/d39/d76/d19/d27/f84 x:0 0 0 2024-03-21T10:08:57.112 INFO:tasks.workunit.client.0.smithi153.stdout:2/723: unlink dd/d15/d18/d2c/d4b/d56/d63/d8d/d9b/dc3/df5/dae/d50/d58/d79/c9c 0 2024-03-21T10:08:57.114 INFO:tasks.workunit.client.0.smithi153.stdout:0/587: write d4/d16/d19/d30/d6c/d98/f69 [1051978,100395] 0 2024-03-21T10:08:57.114 INFO:tasks.workunit.client.1.smithi174.stdout:9/513: dread - d2/d24/d2e/d63/fad zero size 2024-03-21T10:08:57.118 INFO:tasks.workunit.client.0.smithi153.stdout:0/588: chown d4/d16/d19/c51 26 1 2024-03-21T10:08:57.120 INFO:tasks.workunit.client.1.smithi174.stdout:0/463: mknod d9/d1c/ca3 0 2024-03-21T10:08:57.121 INFO:tasks.workunit.client.0.smithi153.stdout:7/545: stat d4/d2a/d3b/d81/f90 0 2024-03-21T10:08:57.121 INFO:tasks.workunit.client.0.smithi153.stdout:5/645: dread d4/d5/f7 [0,4194304] 0 2024-03-21T10:08:57.122 INFO:tasks.workunit.client.0.smithi153.stdout:3/680: dread dd/d1e/d45/fc8 [0,4194304] 0 2024-03-21T10:08:57.122 INFO:tasks.workunit.client.1.smithi174.stdout:1/492: dwrite d4/d7/d28/d59/f66 [0,4194304] 0 2024-03-21T10:08:57.122 INFO:tasks.workunit.client.1.smithi174.stdout:9/514: write d2/d24/d2e/f9b [5919089,109559] 0 2024-03-21T10:08:57.123 INFO:tasks.workunit.client.0.smithi153.stdout:1/527: symlink d6/d32/d3e/d4f/la3 0 2024-03-21T10:08:57.123 INFO:tasks.workunit.client.0.smithi153.stdout:8/633: mknod d5/d9/d43/d7b/d7c/cd4 0 2024-03-21T10:08:57.125 INFO:tasks.workunit.client.0.smithi153.stdout:9/673: truncate d1/d43/d44/d46/d9e/d61/f87 1343790 0 2024-03-21T10:08:57.126 INFO:tasks.workunit.client.0.smithi153.stdout:8/634: chown d5/d9/d43/lf 1 1 2024-03-21T10:08:57.126 INFO:tasks.workunit.client.1.smithi174.stdout:6/369: write d0/d3/d5/d6/f38 [1684080,13990] 0 2024-03-21T10:08:57.126 INFO:tasks.workunit.client.0.smithi153.stdout:1/528: write d6/d10/f98 [148006,6153] 0 2024-03-21T10:08:57.128 INFO:tasks.workunit.client.0.smithi153.stdout:2/724: creat dd/d15/d18/d2c/dd7/ff8 x:0 0 0 2024-03-21T10:08:57.128 INFO:tasks.workunit.client.0.smithi153.stdout:7/546: creat d4/d11/d56/db1/fba x:0 0 0 2024-03-21T10:08:57.128 INFO:tasks.workunit.client.1.smithi174.stdout:7/408: rename d0/d53/f76 to d0/d53/f8d 0 2024-03-21T10:08:57.128 INFO:tasks.workunit.client.0.smithi153.stdout:0/589: write d4/d4c/d83/d22/f73 [2010913,56476] 0 2024-03-21T10:08:57.130 INFO:tasks.workunit.client.1.smithi174.stdout:9/515: read d2/f3c [55034,124476] 0 2024-03-21T10:08:57.136 INFO:tasks.workunit.client.0.smithi153.stdout:9/674: write d1/d43/d44/d46/d9e/d2e/d67/fd5 [165828,130733] 0 2024-03-21T10:08:57.137 INFO:tasks.workunit.client.0.smithi153.stdout:3/681: creat dd/d1e/d2f/d35/d68/daa/feb x:0 0 0 2024-03-21T10:08:57.139 INFO:tasks.workunit.client.1.smithi174.stdout:5/524: dwrite d1/d16/d2e/d66/d8b/f36 [0,4194304] 0 2024-03-21T10:08:57.140 INFO:tasks.workunit.client.1.smithi174.stdout:0/464: mknod d9/d21/d31/d9f/d56/ca4 0 2024-03-21T10:08:57.141 INFO:tasks.workunit.client.1.smithi174.stdout:4/514: truncate d4/f3e 1212009 0 2024-03-21T10:08:57.142 INFO:tasks.workunit.client.0.smithi153.stdout:7/547: write d4/d14/d2f/d78/f71 [2608261,85741] 0 2024-03-21T10:08:57.142 INFO:tasks.workunit.client.0.smithi153.stdout:9/675: chown d1/d43/d72/dba/lcf 0 1 2024-03-21T10:08:57.143 INFO:tasks.workunit.client.1.smithi174.stdout:3/463: sync 2024-03-21T10:08:57.145 INFO:tasks.workunit.client.1.smithi174.stdout:7/409: rename d0/d4/d10/d44/f7f to d0/d4/d10/d44/f8e 0 2024-03-21T10:08:57.149 INFO:tasks.workunit.client.1.smithi174.stdout:4/515: fdatasync d4/d21/d24/d28/d38/f5f 0 2024-03-21T10:08:57.149 INFO:tasks.workunit.client.1.smithi174.stdout:9/516: chown d2/d11/d1d/d2a/l3d 1720146367 1 2024-03-21T10:08:57.149 INFO:tasks.workunit.client.0.smithi153.stdout:7/548: write d4/d14/d2f/d78/f3a [41643,63961] 0 2024-03-21T10:08:57.149 INFO:tasks.workunit.client.0.smithi153.stdout:4/653: write d11/f24 [500638,101257] 0 2024-03-21T10:08:57.150 INFO:tasks.workunit.client.0.smithi153.stdout:9/676: read d1/d43/d44/d46/f37 [1913879,11061] 0 2024-03-21T10:08:57.151 INFO:tasks.workunit.client.0.smithi153.stdout:2/725: creat dd/d15/d18/d2c/d4b/d56/d63/ff9 x:0 0 0 2024-03-21T10:08:57.151 INFO:tasks.workunit.client.1.smithi174.stdout:0/465: unlink d9/d1c/l22 0 2024-03-21T10:08:57.151 INFO:tasks.workunit.client.1.smithi174.stdout:9/517: stat d2/d24/d2e/f48 0 2024-03-21T10:08:57.153 INFO:tasks.workunit.client.0.smithi153.stdout:5/646: rename d4/d5/df/f40 to d4/d5/fd4 0 2024-03-21T10:08:57.153 INFO:tasks.workunit.client.0.smithi153.stdout:2/726: stat fc 0 2024-03-21T10:08:57.155 INFO:tasks.workunit.client.0.smithi153.stdout:0/590: dread d4/d4c/f81 [0,4194304] 0 2024-03-21T10:08:57.156 INFO:tasks.workunit.client.0.smithi153.stdout:4/654: mkdir d11/d71/d89/dca 0 2024-03-21T10:08:57.158 INFO:tasks.workunit.client.0.smithi153.stdout:7/549: mknod d4/d1b/d83/cbb 0 2024-03-21T10:08:57.158 INFO:tasks.workunit.client.1.smithi174.stdout:5/525: symlink d1/d49/lbc 0 2024-03-21T10:08:57.160 INFO:tasks.workunit.client.1.smithi174.stdout:8/400: getdents d9 0 2024-03-21T10:08:57.161 INFO:tasks.workunit.client.0.smithi153.stdout:5/647: dread - d4/d5/df/d67/da0/db3/fd2 zero size 2024-03-21T10:08:57.164 INFO:tasks.workunit.client.0.smithi153.stdout:0/591: symlink d4/d4c/lb7 0 2024-03-21T10:08:57.165 INFO:tasks.workunit.client.0.smithi153.stdout:9/677: fdatasync d1/d43/d44/d46/d9e/d24/d41/f5b 0 2024-03-21T10:08:57.168 INFO:tasks.workunit.client.1.smithi174.stdout:6/370: dwrite d0/d3/f5a [0,4194304] 0 2024-03-21T10:08:57.170 INFO:tasks.workunit.client.1.smithi174.stdout:4/516: mkdir d4/d6c/da8 0 2024-03-21T10:08:57.170 INFO:tasks.workunit.client.0.smithi153.stdout:7/550: truncate d4/d2a/d40/d50/f9b 936866 0 2024-03-21T10:08:57.170 INFO:tasks.workunit.client.1.smithi174.stdout:9/518: creat d2/d11/d60/fb6 x:0 0 0 2024-03-21T10:08:57.170 INFO:tasks.workunit.client.0.smithi153.stdout:1/529: dwrite d6/d10/f73 [0,4194304] 0 2024-03-21T10:08:57.172 INFO:tasks.workunit.client.1.smithi174.stdout:1/493: dwrite d4/d7/d33/d43/d51/f8d [0,4194304] 0 2024-03-21T10:08:57.174 INFO:tasks.workunit.client.1.smithi174.stdout:0/466: rename d9/d21/d31/d9f/d56 to d9/d21/d23/da5 0 2024-03-21T10:08:57.176 INFO:tasks.workunit.client.0.smithi153.stdout:7/551: read d4/d14/d2f/d78/f3a [7566635,86377] 0 2024-03-21T10:08:57.176 INFO:tasks.workunit.client.1.smithi174.stdout:9/519: chown d2/d24/d2e/ca1 1 1 2024-03-21T10:08:57.177 INFO:tasks.workunit.client.1.smithi174.stdout:0/467: dread - d9/d21/d31/f40 zero size 2024-03-21T10:08:57.178 INFO:tasks.workunit.client.1.smithi174.stdout:5/526: mkdir d1/d16/d2e/d66/dbd 0 2024-03-21T10:08:57.179 INFO:tasks.workunit.client.0.smithi153.stdout:9/678: fsync d1/d8a/fb9 0 2024-03-21T10:08:57.183 INFO:tasks.workunit.client.1.smithi174.stdout:7/410: link d0/d53/f8d d0/d4/d10/d44/d81/f8f 0 2024-03-21T10:08:57.183 INFO:tasks.workunit.client.1.smithi174.stdout:2/482: sync 2024-03-21T10:08:57.185 INFO:tasks.workunit.client.1.smithi174.stdout:6/371: dread d0/d3/d5/df/d16/f34 [0,4194304] 0 2024-03-21T10:08:57.186 INFO:tasks.workunit.client.1.smithi174.stdout:7/411: write d0/d4/d10/d60/f8a [654045,107647] 0 2024-03-21T10:08:57.187 INFO:tasks.workunit.client.1.smithi174.stdout:7/412: stat d0/d53/l78 0 2024-03-21T10:08:57.192 INFO:tasks.workunit.client.0.smithi153.stdout:9/679: write d1/d43/d44/d46/d4d/f74 [1784492,31892] 0 2024-03-21T10:08:57.194 INFO:tasks.workunit.client.0.smithi153.stdout:8/635: dwrite d5/d9/d43/dd/d2d/d45/f70 [0,4194304] 0 2024-03-21T10:08:57.198 INFO:tasks.workunit.client.1.smithi174.stdout:5/527: fsync d1/d16/d1c/f25 0 2024-03-21T10:08:57.201 INFO:tasks.workunit.client.1.smithi174.stdout:3/464: dwrite d0/d12/f17 [0,4194304] 0 2024-03-21T10:08:57.201 INFO:tasks.workunit.client.0.smithi153.stdout:0/592: dread d4/f6 [0,4194304] 0 2024-03-21T10:08:57.202 INFO:tasks.workunit.client.0.smithi153.stdout:4/655: dwrite d11/f20 [4194304,4194304] 0 2024-03-21T10:08:57.203 INFO:tasks.workunit.client.1.smithi174.stdout:1/494: dread d4/d7/d28/d59/f66 [0,4194304] 0 2024-03-21T10:08:57.206 INFO:tasks.workunit.client.0.smithi153.stdout:4/656: chown d11/d2d/l52 7 1 2024-03-21T10:08:57.207 INFO:tasks.workunit.client.1.smithi174.stdout:3/465: dread d0/d12/d14/d1a/f40 [0,4194304] 0 2024-03-21T10:08:57.209 INFO:tasks.workunit.client.0.smithi153.stdout:2/727: dwrite dd/d15/d18/d2c/d4b/d56/d63/d8d/d9b/dc3/df5/dae/d50/d59/fe7 [0,4194304] 0 2024-03-21T10:08:57.209 INFO:tasks.workunit.client.0.smithi153.stdout:4/657: fsync d11/d18/d6d/f9b 0 2024-03-21T10:08:57.215 INFO:tasks.workunit.client.0.smithi153.stdout:1/530: creat d6/d10/d35/d5b/d92/fa4 x:0 0 0 2024-03-21T10:08:57.216 INFO:tasks.workunit.client.0.smithi153.stdout:9/680: creat d1/d4e/fe9 x:0 0 0 2024-03-21T10:08:57.219 INFO:tasks.workunit.client.1.smithi174.stdout:6/372: dwrite d0/d3/d5/d6/f4a [4194304,4194304] 0 2024-03-21T10:08:57.219 INFO:tasks.workunit.client.1.smithi174.stdout:2/483: unlink d5/d25/d27/d2d/d45/d5b/f1d 0 2024-03-21T10:08:57.220 INFO:tasks.workunit.client.0.smithi153.stdout:5/648: link d4/d5/df/d67/da0/db3/l98 d4/d5/de/d44/d48/ld5 0 2024-03-21T10:08:57.220 INFO:tasks.workunit.client.0.smithi153.stdout:9/681: dread - d1/d4e/fa6 zero size 2024-03-21T10:08:57.221 INFO:tasks.workunit.client.1.smithi174.stdout:7/413: mknod d0/d53/c90 0 2024-03-21T10:08:57.223 INFO:tasks.workunit.client.1.smithi174.stdout:4/517: link d4/d21/f7b d4/d21/fa9 0 2024-03-21T10:08:57.228 INFO:tasks.workunit.client.0.smithi153.stdout:2/728: write dd/d15/d18/d2c/d4b/fe3 [1138928,112950] 0 2024-03-21T10:08:57.229 INFO:tasks.workunit.client.0.smithi153.stdout:1/531: rename d6/d32/d3e to d6/d10/d35/da5 0 2024-03-21T10:08:57.231 INFO:tasks.workunit.client.0.smithi153.stdout:7/552: creat d4/d2a/d3b/fbc x:0 0 0 2024-03-21T10:08:57.232 INFO:tasks.workunit.client.1.smithi174.stdout:5/528: creat d1/d16/d2e/d66/fbe x:0 0 0 2024-03-21T10:08:57.233 INFO:tasks.workunit.client.1.smithi174.stdout:1/495: write d4/d67/f99 [2545030,123345] 0 2024-03-21T10:08:57.244 INFO:tasks.workunit.client.1.smithi174.stdout:7/414: dwrite d0/d4/d10/d44/f64 [0,4194304] 0 2024-03-21T10:08:57.247 INFO:tasks.workunit.client.0.smithi153.stdout:9/682: rmdir d1/d4/d5f 39 2024-03-21T10:08:57.249 INFO:tasks.workunit.client.1.smithi174.stdout:8/401: link d9/d53/l68 d9/d2a/d39/d76/d19/d27/d79/l85 0 2024-03-21T10:08:57.249 INFO:tasks.workunit.client.0.smithi153.stdout:0/593: mknod d4/d4c/d83/d22/cb8 0 2024-03-21T10:08:57.251 INFO:tasks.workunit.client.1.smithi174.stdout:2/484: creat d5/d25/d27/d2d/d45/d5b/d2f/d68/d9e/fa6 x:0 0 0 2024-03-21T10:08:57.252 INFO:tasks.workunit.client.0.smithi153.stdout:4/658: dwrite d11/f19 [0,4194304] 0 2024-03-21T10:08:57.255 INFO:tasks.workunit.client.0.smithi153.stdout:1/532: write d6/d10/d35/da5/f4b [122278,67267] 0 2024-03-21T10:08:57.255 INFO:tasks.workunit.client.1.smithi174.stdout:0/468: dread d9/d21/d31/f42 [0,4194304] 0 2024-03-21T10:08:57.257 INFO:tasks.workunit.client.1.smithi174.stdout:6/373: dwrite d0/d3/d5/df/d16/f2b [4194304,4194304] 0 2024-03-21T10:08:57.258 INFO:tasks.workunit.client.0.smithi153.stdout:6/831: sync 2024-03-21T10:08:57.261 INFO:tasks.workunit.client.0.smithi153.stdout:1/533: chown d6/d10/da0/fa1 13 1 2024-03-21T10:08:57.266 INFO:tasks.workunit.client.0.smithi153.stdout:7/553: read d4/d14/d2f/f45 [3680392,33271] 0 2024-03-21T10:08:57.269 INFO:tasks.workunit.client.1.smithi174.stdout:4/518: dwrite d4/d21/d24/d28/d38/f88 [0,4194304] 0 2024-03-21T10:08:57.269 INFO:tasks.workunit.client.1.smithi174.stdout:7/415: creat d0/d4/d38/f91 x:0 0 0 2024-03-21T10:08:57.270 INFO:tasks.workunit.client.0.smithi153.stdout:5/649: dwrite d4/d5/df/d67/da0/db3/d8d/fd1 [0,4194304] 0 2024-03-21T10:08:57.272 INFO:tasks.workunit.client.0.smithi153.stdout:3/682: dread dd/f25 [4194304,4194304] 0 2024-03-21T10:08:57.273 INFO:tasks.workunit.client.1.smithi174.stdout:9/520: getdents d2/d54 0 2024-03-21T10:08:57.273 INFO:tasks.workunit.client.1.smithi174.stdout:8/402: symlink d9/d2a/d39/d76/d19/d27/d60/l86 0 2024-03-21T10:08:57.275 INFO:tasks.workunit.client.0.smithi153.stdout:6/832: read d1f/d2b/d10a/d40/f8f [521361,98311] 0 2024-03-21T10:08:57.277 INFO:tasks.workunit.client.0.smithi153.stdout:3/683: chown dd/d1e/d2f/d35/daf/dcc/d58/fde 987461719 1 2024-03-21T10:08:57.278 INFO:tasks.workunit.client.0.smithi153.stdout:3/684: stat dd/d1e/d3d/f69 0 2024-03-21T10:08:57.280 INFO:tasks.workunit.client.1.smithi174.stdout:5/529: rename d1/d5c/l63 to d1/d2f/d37/lbf 0 2024-03-21T10:08:57.285 INFO:tasks.workunit.client.1.smithi174.stdout:5/530: chown d1/d16/l92 141 1 2024-03-21T10:08:57.288 INFO:tasks.workunit.client.1.smithi174.stdout:2/485: dwrite d5/d25/d27/d2d/d45/d5b/d55/d75/f8d [0,4194304] 0 2024-03-21T10:08:57.288 INFO:tasks.workunit.client.1.smithi174.stdout:6/374: creat d0/d3/d20/d3d/f76 x:0 0 0 2024-03-21T10:08:57.296 INFO:tasks.workunit.client.1.smithi174.stdout:8/403: dread d9/d2a/d39/d76/d19/f40 [0,4194304] 0 2024-03-21T10:08:57.304 INFO:tasks.workunit.client.0.smithi153.stdout:9/683: dread d1/d43/f4f [0,4194304] 0 2024-03-21T10:08:57.308 INFO:tasks.workunit.client.1.smithi174.stdout:2/486: dread d5/d25/d27/d2e/d4f/f6d [0,4194304] 0 2024-03-21T10:08:57.310 INFO:tasks.workunit.client.0.smithi153.stdout:2/729: fdatasync dd/d15/d18/d2c/d4b/d56/fc9 0 2024-03-21T10:08:57.311 INFO:tasks.workunit.client.0.smithi153.stdout:8/636: getdents d5/d9/d43/dd/d20/d26/d47/d9a/db9 0 2024-03-21T10:08:57.314 INFO:tasks.workunit.client.0.smithi153.stdout:0/594: readlink d4/d16/d27/l40 0 2024-03-21T10:08:57.315 INFO:tasks.workunit.client.0.smithi153.stdout:4/659: creat d11/d71/fcb x:0 0 0 2024-03-21T10:08:57.315 INFO:tasks.workunit.client.1.smithi174.stdout:9/521: symlink d2/d24/lb7 0 2024-03-21T10:08:57.316 INFO:tasks.workunit.client.1.smithi174.stdout:3/466: dwrite d0/f3 [4194304,4194304] 0 2024-03-21T10:08:57.317 INFO:tasks.workunit.client.0.smithi153.stdout:1/534: creat d6/d3a/d3c/d50/d51/fa6 x:0 0 0 2024-03-21T10:08:57.322 INFO:tasks.workunit.client.1.smithi174.stdout:8/404: rename d9/d2a/f61 to d9/d2a/d39/d76/f87 0 2024-03-21T10:08:57.323 INFO:tasks.workunit.client.1.smithi174.stdout:1/496: getdents d4/d7/d33/d43/d51 0 2024-03-21T10:08:57.324 INFO:tasks.workunit.client.1.smithi174.stdout:2/487: write d5/d25/d27/d2d/d45/d5b/d2f/d68/f96 [739474,53846] 0 2024-03-21T10:08:57.326 INFO:tasks.workunit.client.1.smithi174.stdout:8/405: chown d9/d2a/d39/d58/c6b 15 1 2024-03-21T10:08:57.327 INFO:tasks.workunit.client.1.smithi174.stdout:4/519: link d4/d21/l7d d4/d21/d24/laa 0 2024-03-21T10:08:57.329 INFO:tasks.workunit.client.1.smithi174.stdout:4/520: fsync d4/f7 0 2024-03-21T10:08:57.332 INFO:tasks.workunit.client.0.smithi153.stdout:6/833: mknod d1f/d2b/c12a 0 2024-03-21T10:08:57.336 INFO:tasks.workunit.client.1.smithi174.stdout:2/488: mkdir d5/d25/d27/d2d/d45/d5b/d2f/d4b/da7 0 2024-03-21T10:08:57.339 INFO:tasks.workunit.client.0.smithi153.stdout:8/637: unlink d5/d9/c58 0 2024-03-21T10:08:57.341 INFO:tasks.workunit.client.0.smithi153.stdout:6/834: fdatasync d1f/de0/de1/df2/d10f/f127 0 2024-03-21T10:08:57.341 INFO:tasks.workunit.client.0.smithi153.stdout:7/554: dwrite d4/d2a/d3b/d60/f7c [0,4194304] 0 2024-03-21T10:08:57.341 INFO:tasks.workunit.client.1.smithi174.stdout:1/497: fsync d4/d7/f21 0 2024-03-21T10:08:57.341 INFO:tasks.workunit.client.1.smithi174.stdout:8/406: creat d9/d2a/f88 x:0 0 0 2024-03-21T10:08:57.341 INFO:tasks.workunit.client.1.smithi174.stdout:3/467: mkdir d0/d12/d14/d67/da1 0 2024-03-21T10:08:57.345 INFO:tasks.workunit.client.1.smithi174.stdout:8/407: dread - d9/d2a/d39/d76/f69 zero size 2024-03-21T10:08:57.345 INFO:tasks.workunit.client.0.smithi153.stdout:1/535: creat d6/d3a/d3c/fa7 x:0 0 0 2024-03-21T10:08:57.347 INFO:tasks.workunit.client.1.smithi174.stdout:6/375: dwrite d0/d3/d5/f4d [0,4194304] 0 2024-03-21T10:08:57.347 INFO:tasks.workunit.client.1.smithi174.stdout:4/521: rename d4/d6c to d4/d21/d31/d6a/dab 0 2024-03-21T10:08:57.348 INFO:tasks.workunit.client.0.smithi153.stdout:4/660: dwrite d11/d2d/f9e [0,4194304] 0 2024-03-21T10:08:57.350 INFO:tasks.workunit.client.1.smithi174.stdout:8/408: write d9/d2a/d39/d76/d19/d27/f64 [205255,61037] 0 2024-03-21T10:08:57.350 INFO:tasks.workunit.client.1.smithi174.stdout:1/498: creat d4/d7/d33/fac x:0 0 0 2024-03-21T10:08:57.354 INFO:tasks.workunit.client.0.smithi153.stdout:4/661: readlink d11/lb7 0 2024-03-21T10:08:57.354 INFO:tasks.workunit.client.0.smithi153.stdout:8/638: creat d5/d9/d43/dd/d20/d26/d47/fd5 x:0 0 0 2024-03-21T10:08:57.354 INFO:tasks.workunit.client.0.smithi153.stdout:1/536: write d6/d10/f73 [4154491,121635] 0 2024-03-21T10:08:57.355 INFO:tasks.workunit.client.1.smithi174.stdout:4/522: unlink d4/d21/d31/d3a/f95 0 2024-03-21T10:08:57.355 INFO:tasks.workunit.client.1.smithi174.stdout:3/468: creat d0/d12/d14/d1a/d44/d56/fa2 x:0 0 0 2024-03-21T10:08:57.357 INFO:tasks.workunit.client.0.smithi153.stdout:2/730: creat dd/ffa x:0 0 0 2024-03-21T10:08:57.359 INFO:tasks.workunit.client.1.smithi174.stdout:6/376: truncate d0/d3/d5/d3b/f46 4195958 0 2024-03-21T10:08:57.366 INFO:tasks.workunit.client.1.smithi174.stdout:8/409: symlink d9/d2a/d4d/l89 0 2024-03-21T10:08:57.366 INFO:tasks.workunit.client.0.smithi153.stdout:1/537: mkdir d6/d3a/d3c/d50/da8 0 2024-03-21T10:08:57.367 INFO:tasks.workunit.client.1.smithi174.stdout:6/377: mknod d0/d3/d5/d3b/c77 0 2024-03-21T10:08:57.369 INFO:tasks.workunit.client.0.smithi153.stdout:3/685: dwrite dd/d1e/d45/fc8 [4194304,4194304] 0 2024-03-21T10:08:57.370 INFO:tasks.workunit.client.1.smithi174.stdout:6/378: write d0/d3/d20/d3d/f59 [835002,28939] 0 2024-03-21T10:08:57.373 INFO:tasks.workunit.client.0.smithi153.stdout:7/555: creat d4/fbd x:0 0 0 2024-03-21T10:08:57.374 INFO:tasks.workunit.client.1.smithi174.stdout:5/531: chown d1/d2f/d37/lbf 6889134 1 2024-03-21T10:08:57.374 INFO:tasks.workunit.client.1.smithi174.stdout:1/499: creat d4/d7/d28/fad x:0 0 0 2024-03-21T10:08:57.375 INFO:tasks.workunit.client.1.smithi174.stdout:5/532: stat d1/d5c/d73 0 2024-03-21T10:08:57.377 INFO:tasks.workunit.client.1.smithi174.stdout:3/469: dwrite d0/d12/d14/d1a/d27/d3c/f72 [0,4194304] 0 2024-03-21T10:08:57.377 INFO:tasks.workunit.client.1.smithi174.stdout:5/533: truncate d1/d2f/f4c 5485081 0 2024-03-21T10:08:57.378 INFO:tasks.workunit.client.1.smithi174.stdout:3/470: readlink d0/d12/d46/l5b 0 2024-03-21T10:08:57.378 INFO:tasks.workunit.client.0.smithi153.stdout:0/595: rename d4/d16/l4f to d4/d16/d19/lb9 0 2024-03-21T10:08:57.379 INFO:tasks.workunit.client.1.smithi174.stdout:8/410: creat d9/d2a/d39/d76/d19/d27/f8a x:0 0 0 2024-03-21T10:08:57.383 INFO:tasks.workunit.client.0.smithi153.stdout:6/835: truncate d1f/d2b/d10a/d40/d4f/d66/f7c 4141418 0 2024-03-21T10:08:57.384 INFO:tasks.workunit.client.1.smithi174.stdout:0/469: truncate d9/d21/d23/da5/f66 2249046 0 2024-03-21T10:08:57.384 INFO:tasks.workunit.client.0.smithi153.stdout:2/731: mknod dd/d15/d18/d2c/d4b/d56/d63/cfb 0 2024-03-21T10:08:57.384 INFO:tasks.workunit.client.1.smithi174.stdout:1/500: creat d4/d2d/fae x:0 0 0 2024-03-21T10:08:57.389 INFO:tasks.workunit.client.0.smithi153.stdout:4/662: dwrite d11/d18/f82 [0,4194304] 0 2024-03-21T10:08:57.390 INFO:tasks.workunit.client.1.smithi174.stdout:5/534: mknod d1/d49/d4f/cc0 0 2024-03-21T10:08:57.391 INFO:tasks.workunit.client.0.smithi153.stdout:4/663: write d11/d18/d54/fc9 [121176,12848] 0 2024-03-21T10:08:57.392 INFO:tasks.workunit.client.1.smithi174.stdout:7/416: truncate d0/d4/d10/d1b/f68 3658317 0 2024-03-21T10:08:57.393 INFO:tasks.workunit.client.1.smithi174.stdout:3/471: truncate d0/d12/d14/f80 108885 0 2024-03-21T10:08:57.395 INFO:tasks.workunit.client.1.smithi174.stdout:0/470: creat d9/d21/d31/d9f/d79/fa6 x:0 0 0 2024-03-21T10:08:57.396 INFO:tasks.workunit.client.1.smithi174.stdout:7/417: chown d0/d53/f8d 0 1 2024-03-21T10:08:57.396 INFO:tasks.workunit.client.1.smithi174.stdout:1/501: mknod d4/d7/d32/d7d/caf 0 2024-03-21T10:08:57.399 INFO:tasks.workunit.client.1.smithi174.stdout:0/471: chown d9/d1c/d76/d87 840 1 2024-03-21T10:08:57.403 INFO:tasks.workunit.client.0.smithi153.stdout:0/596: symlink d4/d16/d19/d30/da7/lba 0 2024-03-21T10:08:57.406 INFO:tasks.workunit.client.0.smithi153.stdout:6/836: mknod d1f/d93/dd8/d59/d70/c12b 0 2024-03-21T10:08:57.411 INFO:tasks.workunit.client.1.smithi174.stdout:3/472: write d0/d12/d14/d1a/d27/d3c/f81 [4841195,55405] 0 2024-03-21T10:08:57.413 INFO:tasks.workunit.client.0.smithi153.stdout:4/664: symlink d11/d18/d7e/dc5/lcc 0 2024-03-21T10:08:57.414 INFO:tasks.workunit.client.0.smithi153.stdout:4/665: fdatasync d11/d2d/f92 0 2024-03-21T10:08:57.415 INFO:tasks.workunit.client.1.smithi174.stdout:8/411: getdents d9/d53 0 2024-03-21T10:08:57.415 INFO:tasks.workunit.client.0.smithi153.stdout:3/686: creat dd/d1e/d45/fec x:0 0 0 2024-03-21T10:08:57.416 INFO:tasks.workunit.client.0.smithi153.stdout:6/837: rmdir d1f/d93/dd8/d59/d70/d110 39 2024-03-21T10:08:57.418 INFO:tasks.workunit.client.1.smithi174.stdout:0/472: creat d9/d35/d4a/d67/d89/fa7 x:0 0 0 2024-03-21T10:08:57.418 INFO:tasks.workunit.client.0.smithi153.stdout:7/556: rename d4/d14/fa0 to d4/d14/d2f/fbe 0 2024-03-21T10:08:57.419 INFO:tasks.workunit.client.1.smithi174.stdout:7/418: dread d0/d4/d10/d60/f8a [0,4194304] 0 2024-03-21T10:08:57.421 INFO:tasks.workunit.client.0.smithi153.stdout:0/597: creat d4/d16/d9f/fbb x:0 0 0 2024-03-21T10:08:57.422 INFO:tasks.workunit.client.0.smithi153.stdout:3/687: mknod dd/d1e/d2f/d35/daf/ced 0 2024-03-21T10:08:57.423 INFO:tasks.workunit.client.1.smithi174.stdout:4/523: write d4/f15 [5078525,56943] 0 2024-03-21T10:08:57.424 INFO:tasks.workunit.client.1.smithi174.stdout:9/522: dwrite d2/f3c [0,4194304] 0 2024-03-21T10:08:57.424 INFO:tasks.workunit.client.0.smithi153.stdout:0/598: fdatasync d4/db/fa5 0 2024-03-21T10:08:57.424 INFO:tasks.workunit.client.0.smithi153.stdout:7/557: write d4/d2a/f2d [3678003,60535] 0 2024-03-21T10:08:57.433 INFO:tasks.workunit.client.0.smithi153.stdout:7/558: read d4/d2a/d3b/d81/f99 [752300,101571] 0 2024-03-21T10:08:57.433 INFO:tasks.workunit.client.1.smithi174.stdout:8/412: unlink d9/d2a/d39/d76/d19/f2f 0 2024-03-21T10:08:57.434 INFO:tasks.workunit.client.1.smithi174.stdout:9/523: chown d2/d11/d1d/d2a/d91 0 1 2024-03-21T10:08:57.435 INFO:tasks.workunit.client.0.smithi153.stdout:3/688: rmdir dd/d1e/d3d 39 2024-03-21T10:08:57.435 INFO:tasks.workunit.client.0.smithi153.stdout:0/599: creat d4/d16/d19/d30/da7/fbc x:0 0 0 2024-03-21T10:08:57.436 INFO:tasks.workunit.client.1.smithi174.stdout:1/502: dwrite d4/d7/d33/fac [0,4194304] 0 2024-03-21T10:08:57.440 INFO:tasks.workunit.client.0.smithi153.stdout:7/559: read d4/d2a/d3b/f98 [2925974,13437] 0 2024-03-21T10:08:57.441 INFO:tasks.workunit.client.1.smithi174.stdout:3/473: creat d0/d12/d14/d1a/d27/d3c/fa3 x:0 0 0 2024-03-21T10:08:57.446 INFO:tasks.workunit.client.1.smithi174.stdout:0/473: rename f7 to d9/d21/d75/da1/fa8 0 2024-03-21T10:08:57.449 INFO:tasks.workunit.client.0.smithi153.stdout:7/560: write d4/d2a/d40/d50/f9f [921118,111466] 0 2024-03-21T10:08:57.451 INFO:tasks.workunit.client.1.smithi174.stdout:7/419: creat d0/d4/d10/d1b/d86/f92 x:0 0 0 2024-03-21T10:08:57.453 INFO:tasks.workunit.client.0.smithi153.stdout:7/561: truncate d4/d19/fb6 1303888 0 2024-03-21T10:08:57.455 INFO:tasks.workunit.client.1.smithi174.stdout:6/379: dread d0/d3/d5/d6/f13 [4194304,4194304] 0 2024-03-21T10:08:57.457 INFO:tasks.workunit.client.1.smithi174.stdout:8/413: creat d9/d2a/d39/d76/f8b x:0 0 0 2024-03-21T10:08:57.457 INFO:tasks.workunit.client.0.smithi153.stdout:8/639: truncate d5/fc9 1775053 0 2024-03-21T10:08:57.459 INFO:tasks.workunit.client.1.smithi174.stdout:5/535: dwrite d1/d16/d1c/f23 [0,4194304] 0 2024-03-21T10:08:57.461 INFO:tasks.workunit.client.0.smithi153.stdout:7/562: dread - d4/d14/d73/fb8 zero size 2024-03-21T10:08:57.462 INFO:tasks.workunit.client.0.smithi153.stdout:4/666: truncate d11/f24 539878 0 2024-03-21T10:08:57.465 INFO:tasks.workunit.client.0.smithi153.stdout:7/563: stat d4/d2a/d3b/c4f 0 2024-03-21T10:08:57.465 INFO:tasks.workunit.client.1.smithi174.stdout:0/474: write d9/f33 [461572,24794] 0 2024-03-21T10:08:57.466 INFO:tasks.workunit.client.1.smithi174.stdout:3/474: mknod d0/d12/d14/d1a/d27/d3c/d73/ca4 0 2024-03-21T10:08:57.467 INFO:tasks.workunit.client.0.smithi153.stdout:5/650: dwrite d4/d5/de/d44/d48/f84 [0,4194304] 0 2024-03-21T10:08:57.467 INFO:tasks.workunit.client.0.smithi153.stdout:8/640: fsync d5/d9/d43/dd/f38 0 2024-03-21T10:08:57.468 INFO:tasks.workunit.client.1.smithi174.stdout:5/536: mkdir d1/d2f/dc1 0 2024-03-21T10:08:57.472 INFO:tasks.workunit.client.1.smithi174.stdout:6/380: chown d0/d3/d5/df/d16/f1a 2449 1 2024-03-21T10:08:57.477 INFO:tasks.workunit.client.0.smithi153.stdout:2/732: dread fb [0,4194304] 0 2024-03-21T10:08:57.478 INFO:tasks.workunit.client.0.smithi153.stdout:0/600: dread d4/d4c/f7a [0,4194304] 0 2024-03-21T10:08:57.481 INFO:tasks.workunit.client.1.smithi174.stdout:4/524: write d4/d21/d24/d28/f51 [4945087,9470] 0 2024-03-21T10:08:57.482 INFO:tasks.workunit.client.0.smithi153.stdout:7/564: unlink d4/fa 0 2024-03-21T10:08:57.485 INFO:tasks.workunit.client.1.smithi174.stdout:0/475: chown d9/d21/d31/d3c 6688412 1 2024-03-21T10:08:57.486 INFO:tasks.workunit.client.0.smithi153.stdout:0/601: creat d4/d4c/d83/d22/d61/fbd x:0 0 0 2024-03-21T10:08:57.487 INFO:tasks.workunit.client.1.smithi174.stdout:8/414: dwrite d9/d53/f5d [0,4194304] 0 2024-03-21T10:08:57.487 INFO:tasks.workunit.client.1.smithi174.stdout:0/476: fsync d9/f19 0 2024-03-21T10:08:57.492 INFO:tasks.workunit.client.1.smithi174.stdout:6/381: chown d0/d3/d5/df/f2e 12081008 1 2024-03-21T10:08:57.492 INFO:tasks.workunit.client.0.smithi153.stdout:6/838: dwrite d1f/f4a [0,4194304] 0 2024-03-21T10:08:57.493 INFO:tasks.workunit.client.0.smithi153.stdout:2/733: chown dd/d15/d18/d2c/d4b/d56/d63/d8d/d9b/dc3/df5/dae/d50/d59/fc5 1752992 1 2024-03-21T10:08:57.493 INFO:tasks.workunit.client.1.smithi174.stdout:1/503: dread d4/d7/d15/f2f [0,4194304] 0 2024-03-21T10:08:57.493 INFO:tasks.workunit.client.1.smithi174.stdout:7/420: getdents d0/d4 0 2024-03-21T10:08:57.498 INFO:tasks.workunit.client.1.smithi174.stdout:1/504: stat d4/d7/d15/d53/d78/ca6 0 2024-03-21T10:08:57.499 INFO:tasks.workunit.client.1.smithi174.stdout:0/477: creat d9/d21/fa9 x:0 0 0 2024-03-21T10:08:57.499 INFO:tasks.workunit.client.1.smithi174.stdout:4/525: mkdir d4/d21/d31/d6a/dac 0 2024-03-21T10:08:57.500 INFO:tasks.workunit.client.1.smithi174.stdout:1/505: dread - d4/d5/f9e zero size 2024-03-21T10:08:57.504 INFO:tasks.workunit.client.1.smithi174.stdout:3/475: link d0/fb d0/d12/d14/fa5 0 2024-03-21T10:08:57.512 INFO:tasks.workunit.client.1.smithi174.stdout:9/524: dread d2/f17 [0,4194304] 0 2024-03-21T10:08:57.515 INFO:tasks.workunit.client.1.smithi174.stdout:0/478: creat d9/d1c/d76/d87/faa x:0 0 0 2024-03-21T10:08:57.517 INFO:tasks.workunit.client.1.smithi174.stdout:8/415: link d9/d2a/d39/d76/d19/d27/f41 d9/f8c 0 2024-03-21T10:08:57.519 INFO:tasks.workunit.client.1.smithi174.stdout:3/476: truncate d0/d12/d14/d1a/d44/d56/fa0 920921 0 2024-03-21T10:08:57.519 INFO:tasks.workunit.client.1.smithi174.stdout:8/416: chown d9/c48 94 1 2024-03-21T10:08:57.524 INFO:tasks.workunit.client.1.smithi174.stdout:8/417: truncate d9/d53/d5b/f77 892825 0 2024-03-21T10:08:57.526 INFO:tasks.workunit.client.1.smithi174.stdout:9/525: rename d2/d11/d1d/f4e to d2/d11/fb8 0 2024-03-21T10:08:57.527 INFO:tasks.workunit.client.1.smithi174.stdout:2/489: sync 2024-03-21T10:08:57.528 INFO:tasks.workunit.client.1.smithi174.stdout:1/506: dread d4/d2d/f34 [0,4194304] 0 2024-03-21T10:08:57.531 INFO:tasks.workunit.client.1.smithi174.stdout:2/490: fdatasync d5/d25/d27/f40 0 2024-03-21T10:08:57.532 INFO:tasks.workunit.client.1.smithi174.stdout:7/421: dwrite d0/d4/d10/f45 [4194304,4194304] 0 2024-03-21T10:08:57.533 INFO:tasks.workunit.client.1.smithi174.stdout:5/537: truncate d1/d5c/d73/f7f 1229580 0 2024-03-21T10:08:57.535 INFO:tasks.workunit.client.1.smithi174.stdout:1/507: write d4/d2d/fae [169529,61654] 0 2024-03-21T10:08:57.538 INFO:tasks.workunit.client.1.smithi174.stdout:9/526: symlink d2/d11/d60/lb9 0 2024-03-21T10:08:57.539 INFO:tasks.workunit.client.1.smithi174.stdout:4/526: write d4/f18 [1115028,33242] 0 2024-03-21T10:08:57.541 INFO:tasks.workunit.client.1.smithi174.stdout:5/538: symlink d1/d2f/lc2 0 2024-03-21T10:08:57.544 INFO:tasks.workunit.client.1.smithi174.stdout:3/477: dread d0/f7 [0,4194304] 0 2024-03-21T10:08:57.544 INFO:tasks.workunit.client.1.smithi174.stdout:4/527: readlink d4/l57 0 2024-03-21T10:08:57.545 INFO:tasks.workunit.client.1.smithi174.stdout:2/491: chown d5/d25/d27/d2e/d4f/da0 925378 1 2024-03-21T10:08:57.546 INFO:tasks.workunit.client.1.smithi174.stdout:7/422: unlink d0/d4/d10/l75 0 2024-03-21T10:08:57.548 INFO:tasks.workunit.client.1.smithi174.stdout:0/479: dread d9/f54 [0,4194304] 0 2024-03-21T10:08:57.549 INFO:tasks.workunit.client.1.smithi174.stdout:4/528: fdatasync d4/d21/d31/d6a/f6e 0 2024-03-21T10:08:57.557 INFO:tasks.workunit.client.1.smithi174.stdout:9/527: mknod d2/d11/d60/db2/cba 0 2024-03-21T10:08:57.558 INFO:tasks.workunit.client.1.smithi174.stdout:4/529: read d4/d21/d31/d6a/f6e [302991,31452] 0 2024-03-21T10:08:57.560 INFO:tasks.workunit.client.1.smithi174.stdout:0/480: dread - d9/f6d zero size 2024-03-21T10:08:57.560 INFO:tasks.workunit.client.1.smithi174.stdout:6/382: dwrite d0/d3/d5/d6/f13 [0,4194304] 0 2024-03-21T10:08:57.565 INFO:tasks.workunit.client.1.smithi174.stdout:5/539: rename d1/d16/d2e/d66/f7c to d1/d16/d2e/d66/dbd/fc3 0 2024-03-21T10:08:57.567 INFO:tasks.workunit.client.1.smithi174.stdout:2/492: mkdir d5/d25/d27/d2d/d45/d5b/d2f/da8 0 2024-03-21T10:08:57.567 INFO:tasks.workunit.client.1.smithi174.stdout:7/423: symlink d0/d4/d10/l93 0 2024-03-21T10:08:57.578 INFO:tasks.workunit.client.1.smithi174.stdout:1/508: dwrite d4/d7/d28/f30 [4194304,4194304] 0 2024-03-21T10:08:57.579 INFO:tasks.workunit.client.1.smithi174.stdout:8/418: dread f1 [0,4194304] 0 2024-03-21T10:08:57.580 INFO:tasks.workunit.client.0.smithi153.stdout:0/602: mkdir d4/d16/d19/d30/d6c/d98/d5e/dbe 0 2024-03-21T10:08:57.582 INFO:tasks.workunit.client.0.smithi153.stdout:4/667: write f10 [2755760,8316] 0 2024-03-21T10:08:57.583 INFO:tasks.workunit.client.0.smithi153.stdout:0/603: stat d4/db/fa5 0 2024-03-21T10:08:57.584 INFO:tasks.workunit.client.1.smithi174.stdout:6/383: mknod d0/d3/d5/df/d16/c78 0 2024-03-21T10:08:57.586 INFO:tasks.workunit.client.0.smithi153.stdout:3/689: dread dd/f2b [0,4194304] 0 2024-03-21T10:08:57.587 INFO:tasks.workunit.client.1.smithi174.stdout:1/509: truncate d4/d7/d33/d43/d77/f9a 287213 0 2024-03-21T10:08:57.587 INFO:tasks.workunit.client.1.smithi174.stdout:5/540: creat d1/d16/d2e/d8d/d99/fc4 x:0 0 0 2024-03-21T10:08:57.587 INFO:tasks.workunit.client.0.smithi153.stdout:8/641: dwrite d5/d9/d43/dd/d20/d26/d7a/f81 [0,4194304] 0 2024-03-21T10:08:57.588 INFO:tasks.workunit.client.0.smithi153.stdout:0/604: dread - d4/d4c/d83/d22/d61/d77/fb1 zero size 2024-03-21T10:08:57.590 INFO:tasks.workunit.client.1.smithi174.stdout:2/493: write d5/d25/d27/d2e/d4f/d65/fa2 [1083452,36447] 0 2024-03-21T10:08:57.590 INFO:tasks.workunit.client.1.smithi174.stdout:1/510: chown d4/d7/f2e 13 1 2024-03-21T10:08:57.591 INFO:tasks.workunit.client.0.smithi153.stdout:3/690: creat dd/d1e/d2f/d35/daf/dcc/d58/fee x:0 0 0 2024-03-21T10:08:57.593 INFO:tasks.workunit.client.0.smithi153.stdout:8/642: readlink d5/d9/d43/d7b/dc4/l8e 0 2024-03-21T10:08:57.593 INFO:tasks.workunit.client.1.smithi174.stdout:3/478: getdents d0/d12/d14/d1a/d44/d56 0 2024-03-21T10:08:57.593 INFO:tasks.workunit.client.1.smithi174.stdout:4/530: dread d4/d21/d24/d28/d38/f88 [0,4194304] 0 2024-03-21T10:08:57.597 INFO:tasks.workunit.client.0.smithi153.stdout:0/605: chown d4/d16/d19/d30/d6c/d98/c6e 356016468 1 2024-03-21T10:08:57.597 INFO:tasks.workunit.client.1.smithi174.stdout:4/531: readlink d4/d21/d31/d6a/d8f/l93 0 2024-03-21T10:08:57.597 INFO:tasks.workunit.client.1.smithi174.stdout:4/532: chown d4 15778 1 2024-03-21T10:08:57.598 INFO:tasks.workunit.client.0.smithi153.stdout:3/691: mkdir dd/d41/d80/dbb/def 0 2024-03-21T10:08:57.602 INFO:tasks.workunit.client.1.smithi174.stdout:5/541: symlink d1/d16/d1c/d81/lc5 0 2024-03-21T10:08:57.602 INFO:tasks.workunit.client.0.smithi153.stdout:3/692: fsync dd/d1e/d2f/d35/daf/fb4 0 2024-03-21T10:08:57.603 INFO:tasks.workunit.client.1.smithi174.stdout:6/384: rmdir d0/d3/d2a 39 2024-03-21T10:08:57.603 INFO:tasks.workunit.client.1.smithi174.stdout:0/481: truncate d9/d21/f7f 641729 0 2024-03-21T10:08:57.604 INFO:tasks.workunit.client.1.smithi174.stdout:1/511: mknod d4/d2d/cb0 0 2024-03-21T10:08:57.606 INFO:tasks.workunit.client.0.smithi153.stdout:3/693: truncate dd/d1e/f4f 3124258 0 2024-03-21T10:08:57.606 INFO:tasks.workunit.client.0.smithi153.stdout:0/606: rename d4/d16/d19/d30/da7/fbc to d4/d16/d19/d58/fbf 0 2024-03-21T10:08:57.607 INFO:tasks.workunit.client.1.smithi174.stdout:0/482: fdatasync d9/d35/f46 0 2024-03-21T10:08:57.608 INFO:tasks.workunit.client.0.smithi153.stdout:3/694: dread - dd/d1e/d3d/d94/fdb zero size 2024-03-21T10:08:57.609 INFO:tasks.workunit.client.1.smithi174.stdout:6/385: chown d0/d3/d5/d6/d42/c6e 0 1 2024-03-21T10:08:57.610 INFO:tasks.workunit.client.1.smithi174.stdout:7/424: creat d0/d4/d10/f94 x:0 0 0 2024-03-21T10:08:57.611 INFO:tasks.workunit.client.0.smithi153.stdout:3/695: dread dd/d1e/d2f/d35/d68/d75/ddd/fbd [0,4194304] 0 2024-03-21T10:08:57.613 INFO:tasks.workunit.client.0.smithi153.stdout:3/696: dread - dd/d1e/d3d/d53/dca/fe6 zero size 2024-03-21T10:08:57.613 INFO:tasks.workunit.client.1.smithi174.stdout:9/528: dread d2/d54/f5c [0,4194304] 0 2024-03-21T10:08:57.615 INFO:tasks.workunit.client.1.smithi174.stdout:5/542: symlink d1/d49/d93/lc6 0 2024-03-21T10:08:57.616 INFO:tasks.workunit.client.1.smithi174.stdout:2/494: mkdir d5/d25/d27/d2e/d90/da9 0 2024-03-21T10:08:57.616 INFO:tasks.workunit.client.1.smithi174.stdout:1/512: mknod d4/d7/d68/d8e/cb1 0 2024-03-21T10:08:57.619 INFO:tasks.workunit.client.0.smithi153.stdout:3/697: creat dd/d1e/ff0 x:0 0 0 2024-03-21T10:08:57.621 INFO:tasks.workunit.client.1.smithi174.stdout:5/543: read - d1/d16/d2e/d66/fbe zero size 2024-03-21T10:08:57.622 INFO:tasks.workunit.client.1.smithi174.stdout:1/513: write d4/d7/f2c [1434391,117077] 0 2024-03-21T10:08:57.622 INFO:tasks.workunit.client.1.smithi174.stdout:4/533: mknod d4/d21/d31/d6a/d8f/d9c/cad 0 2024-03-21T10:08:57.622 INFO:tasks.workunit.client.0.smithi153.stdout:3/698: mkdir dd/d41/df1 0 2024-03-21T10:08:57.625 INFO:tasks.workunit.client.0.smithi153.stdout:0/607: dwrite d4/f23 [4194304,4194304] 0 2024-03-21T10:08:57.630 INFO:tasks.workunit.client.1.smithi174.stdout:5/544: write d1/ff [4507857,55713] 0 2024-03-21T10:08:57.630 INFO:tasks.workunit.client.0.smithi153.stdout:4/668: write d11/d2d/d3d/f56 [4194697,126983] 0 2024-03-21T10:08:57.631 INFO:tasks.workunit.client.0.smithi153.stdout:6/839: dwrite d1f/d93/fdd [0,4194304] 0 2024-03-21T10:08:57.631 INFO:tasks.workunit.client.0.smithi153.stdout:3/699: truncate dd/f67 66129 0 2024-03-21T10:08:57.631 INFO:tasks.workunit.client.0.smithi153.stdout:2/734: truncate dd/d15/d18/d3a/f3e 2923850 0 2024-03-21T10:08:57.632 INFO:tasks.workunit.client.0.smithi153.stdout:8/643: write d5/f8 [2643575,122870] 0 2024-03-21T10:08:57.635 INFO:tasks.workunit.client.1.smithi174.stdout:8/419: dread d9/d2a/d39/d76/f20 [0,4194304] 0 2024-03-21T10:08:57.636 INFO:tasks.workunit.client.1.smithi174.stdout:7/425: dwrite d0/d4/d10/d60/f85 [0,4194304] 0 2024-03-21T10:08:57.636 INFO:tasks.workunit.client.1.smithi174.stdout:2/495: mknod d5/d25/d27/d74/caa 0 2024-03-21T10:08:57.637 INFO:tasks.workunit.client.1.smithi174.stdout:0/483: symlink d9/d21/d31/d3c/lab 0 2024-03-21T10:08:57.637 INFO:tasks.workunit.client.1.smithi174.stdout:1/514: mknod d4/d7/d15/d4f/cb2 0 2024-03-21T10:08:57.639 INFO:tasks.workunit.client.0.smithi153.stdout:0/608: symlink d4/d16/d19/d30/d6c/lc0 0 2024-03-21T10:08:57.639 INFO:tasks.workunit.client.0.smithi153.stdout:6/840: write d1f/d2b/d10a/d40/d4f/f67 [4383198,60299] 0 2024-03-21T10:08:57.641 INFO:tasks.workunit.client.0.smithi153.stdout:4/669: dread d11/f76 [0,4194304] 0 2024-03-21T10:08:57.642 INFO:tasks.workunit.client.1.smithi174.stdout:0/484: fsync d9/d1c/d76/d87/f8d 0 2024-03-21T10:08:57.642 INFO:tasks.workunit.client.0.smithi153.stdout:5/651: dread d4/d5/d16/f19 [0,4194304] 0 2024-03-21T10:08:57.644 INFO:tasks.workunit.client.1.smithi174.stdout:0/485: stat d9/f65 0 2024-03-21T10:08:57.644 INFO:tasks.workunit.client.0.smithi153.stdout:4/670: dread - d11/d2d/d6b/fc6 zero size 2024-03-21T10:08:57.646 INFO:tasks.workunit.client.0.smithi153.stdout:2/735: rename dd/l11 to dd/d15/d18/d3a/d8c/lfc 0 2024-03-21T10:08:57.647 INFO:tasks.workunit.client.0.smithi153.stdout:0/609: read d4/f10 [3963825,108935] 0 2024-03-21T10:08:57.648 INFO:tasks.workunit.client.1.smithi174.stdout:7/426: mknod d0/d4/d10/d44/c95 0 2024-03-21T10:08:57.649 INFO:tasks.workunit.client.0.smithi153.stdout:2/736: truncate dd/ffa 619382 0 2024-03-21T10:08:57.650 INFO:tasks.workunit.client.0.smithi153.stdout:5/652: write d4/d5/df/d67/da0/db3/fd2 [585570,9124] 0 2024-03-21T10:08:57.655 INFO:tasks.workunit.client.0.smithi153.stdout:8/644: creat d5/d9/d43/dd/d20/d26/d24/fd6 x:0 0 0 2024-03-21T10:08:57.657 INFO:tasks.workunit.client.0.smithi153.stdout:2/737: chown dd/d15/d18/d2c/d33/f7d 2959 1 2024-03-21T10:08:57.658 INFO:tasks.workunit.client.0.smithi153.stdout:5/653: chown d4/d5/df/d67/d6a/la2 120982924 1 2024-03-21T10:08:57.659 INFO:tasks.workunit.client.1.smithi174.stdout:1/515: rename d4/d2d/f80 to d4/d7/d68/d89/fb3 0 2024-03-21T10:08:57.660 INFO:tasks.workunit.client.0.smithi153.stdout:8/645: chown d5/d1b/f8f 10398083 1 2024-03-21T10:08:57.660 INFO:tasks.workunit.client.0.smithi153.stdout:4/671: symlink d11/d5e/lcd 0 2024-03-21T10:08:57.662 INFO:tasks.workunit.client.1.smithi174.stdout:9/529: write d2/f17 [2121216,86912] 0 2024-03-21T10:08:57.664 INFO:tasks.workunit.client.0.smithi153.stdout:3/700: dwrite dd/f62 [0,4194304] 0 2024-03-21T10:08:57.666 INFO:tasks.workunit.client.1.smithi174.stdout:5/545: dwrite d1/d16/d2e/d8d/d99/faf [0,4194304] 0 2024-03-21T10:08:57.667 INFO:tasks.workunit.client.1.smithi174.stdout:3/479: dwrite d0/d12/d14/d1a/f3d [0,4194304] 0 2024-03-21T10:08:57.672 INFO:tasks.workunit.client.0.smithi153.stdout:6/841: write d1f/d2b/d10a/d40/f45 [5037031,55256] 0 2024-03-21T10:08:57.672 INFO:tasks.workunit.client.1.smithi174.stdout:3/480: chown d0/c83 1 1 2024-03-21T10:08:57.674 INFO:tasks.workunit.client.0.smithi153.stdout:0/610: rename d4/d16/d19/d30/d6c/f7f to d4/d16/fc1 0 2024-03-21T10:08:57.676 INFO:tasks.workunit.client.0.smithi153.stdout:5/654: truncate d4/d8/f11 1533606 0 2024-03-21T10:08:57.677 INFO:tasks.workunit.client.0.smithi153.stdout:4/672: mkdir d11/d2d/da3/dac/dce 0 2024-03-21T10:08:57.682 INFO:tasks.workunit.client.0.smithi153.stdout:8/646: creat d5/d9/d43/d7b/daa/dca/fd7 x:0 0 0 2024-03-21T10:08:57.684 INFO:tasks.workunit.client.1.smithi174.stdout:2/496: getdents d5/d25/d27/d2e/d4f/d65 0 2024-03-21T10:08:57.686 INFO:tasks.workunit.client.0.smithi153.stdout:6/842: rename d1f/d2b/fa7 to d1f/d2b/d10a/d40/d5f/f12c 0 2024-03-21T10:08:57.686 INFO:tasks.workunit.client.0.smithi153.stdout:9/684: sync 2024-03-21T10:08:57.688 INFO:tasks.workunit.client.0.smithi153.stdout:0/611: mknod d4/d16/d27/cc2 0 2024-03-21T10:08:57.689 INFO:tasks.workunit.client.0.smithi153.stdout:5/655: mkdir d4/d8/d5c/d90/dd6 0 2024-03-21T10:08:57.690 INFO:tasks.workunit.client.0.smithi153.stdout:2/738: dwrite dd/d15/d18/d2c/d4b/fd0 [0,4194304] 0 2024-03-21T10:08:57.691 INFO:tasks.workunit.client.1.smithi174.stdout:8/420: truncate d9/d53/f6f 784086 0 2024-03-21T10:08:57.693 INFO:tasks.workunit.client.1.smithi174.stdout:1/516: dwrite d4/d7/d70/fa9 [0,4194304] 0 2024-03-21T10:08:57.696 INFO:tasks.workunit.client.0.smithi153.stdout:0/612: stat d4/d16/d19/f85 0 2024-03-21T10:08:57.697 INFO:tasks.workunit.client.0.smithi153.stdout:4/673: mknod d11/d2d/da3/ccf 0 2024-03-21T10:08:57.698 INFO:tasks.workunit.client.0.smithi153.stdout:8/647: readlink d5/d9/d43/dd/d20/d26/d8b/d80/l92 0 2024-03-21T10:08:57.699 INFO:tasks.workunit.client.1.smithi174.stdout:0/486: dread d9/d21/d31/d3c/d72/f7e [0,4194304] 0 2024-03-21T10:08:57.700 INFO:tasks.workunit.client.0.smithi153.stdout:5/656: rename d4/d8/d3b/f4b to d4/d5/df/d67/d86/fd7 0 2024-03-21T10:08:57.704 INFO:tasks.workunit.client.0.smithi153.stdout:6/843: link d1f/d99/cc3 d1f/d99/dbb/c12d 0 2024-03-21T10:08:57.704 INFO:tasks.workunit.client.1.smithi174.stdout:8/421: creat d9/d2a/d39/d76/f8d x:0 0 0 2024-03-21T10:08:57.706 INFO:tasks.workunit.client.0.smithi153.stdout:0/613: read d4/d4c/d83/f24 [1806693,122940] 0 2024-03-21T10:08:57.709 INFO:tasks.workunit.client.0.smithi153.stdout:4/674: link d11/d2d/d6b/fc6 d11/d2d/fd0 0 2024-03-21T10:08:57.710 INFO:tasks.workunit.client.1.smithi174.stdout:8/422: chown d9/d2a/d39/d76/d19/d27/d60/l86 4737 1 2024-03-21T10:08:57.711 INFO:tasks.workunit.client.0.smithi153.stdout:6/844: read d1f/d93/dd8/fd1 [1550365,42688] 0 2024-03-21T10:08:57.711 INFO:tasks.workunit.client.0.smithi153.stdout:4/675: chown d11/f19 5 1 2024-03-21T10:08:57.712 INFO:tasks.workunit.client.1.smithi174.stdout:8/423: dread - d9/d2a/d39/f47 zero size 2024-03-21T10:08:57.714 INFO:tasks.workunit.client.1.smithi174.stdout:1/517: dread d4/d7/d33/d43/f5d [0,4194304] 0 2024-03-21T10:08:57.718 INFO:tasks.workunit.client.0.smithi153.stdout:6/845: creat d1f/d93/dd8/d103/f12e x:0 0 0 2024-03-21T10:08:57.718 INFO:tasks.workunit.client.1.smithi174.stdout:2/497: getdents d5/d25/d27/d2d/d45/d5b/d2f/d4b/d76 0 2024-03-21T10:08:57.719 INFO:tasks.workunit.client.1.smithi174.stdout:8/424: write d9/d2a/d39/d76/d19/d31/f3b [4400137,37583] 0 2024-03-21T10:08:57.722 INFO:tasks.workunit.client.0.smithi153.stdout:4/676: truncate d11/f78 775391 0 2024-03-21T10:08:57.723 INFO:tasks.workunit.client.0.smithi153.stdout:6/846: fdatasync d1f/d2b/d10a/d40/d65/ffa 0 2024-03-21T10:08:57.724 INFO:tasks.workunit.client.1.smithi174.stdout:1/518: creat d4/d7/d33/d43/d77/d87/fb4 x:0 0 0 2024-03-21T10:08:57.725 INFO:tasks.workunit.client.0.smithi153.stdout:8/648: dwrite d5/d9/d43/dd/d20/d26/d24/f82 [0,4194304] 0 2024-03-21T10:08:57.726 INFO:tasks.workunit.client.1.smithi174.stdout:1/519: stat d4/d7/d32 0 2024-03-21T10:08:57.727 INFO:tasks.workunit.client.0.smithi153.stdout:2/739: dwrite dd/d15/d18/d2c/d4b/d56/d63/f80 [4194304,4194304] 0 2024-03-21T10:08:57.729 INFO:tasks.workunit.client.1.smithi174.stdout:7/427: dread d0/d4/d10/f45 [0,4194304] 0 2024-03-21T10:08:57.734 INFO:tasks.workunit.client.0.smithi153.stdout:8/649: chown d5/d9/d43/d7b/dc4/f8a 1424767 1 2024-03-21T10:08:57.735 INFO:tasks.workunit.client.0.smithi153.stdout:3/701: write dd/d41/f54 [40335,7307] 0 2024-03-21T10:08:57.736 INFO:tasks.workunit.client.1.smithi174.stdout:8/425: link d9/d2a/d39/d76/f87 d9/d53/d7e/f8e 0 2024-03-21T10:08:57.738 INFO:tasks.workunit.client.0.smithi153.stdout:8/650: chown d5/d9/d43/dd/d20/d26/f1f 10916 1 2024-03-21T10:08:57.741 INFO:tasks.workunit.client.0.smithi153.stdout:6/847: symlink d1f/d2b/d10a/d40/d51/d11d/l12f 0 2024-03-21T10:08:57.741 INFO:tasks.workunit.client.0.smithi153.stdout:3/702: dread - dd/d1e/d2f/d35/daf/dcc/d58/fde zero size 2024-03-21T10:08:57.743 INFO:tasks.workunit.client.0.smithi153.stdout:8/651: chown d5/d9/d43/dd/d2d/d45/d88 2555 1 2024-03-21T10:08:57.744 INFO:tasks.workunit.client.0.smithi153.stdout:9/685: dread d1/d4e/f53 [0,4194304] 0 2024-03-21T10:08:57.746 INFO:tasks.workunit.client.0.smithi153.stdout:2/740: symlink dd/d15/d18/d2c/d4b/d56/d63/d8d/d9b/dc3/df5/dae/lfd 0 2024-03-21T10:08:57.748 INFO:tasks.workunit.client.1.smithi174.stdout:2/498: getdents d5/d25/d27/d2e/d90 0 2024-03-21T10:08:57.748 INFO:tasks.workunit.client.0.smithi153.stdout:8/652: write d5/d9/d43/d7b/daa/dca/fd7 [1044916,95365] 0 2024-03-21T10:08:57.750 INFO:tasks.workunit.client.0.smithi153.stdout:6/848: write d1f/d2b/d10a/d40/d7b/f122 [807114,103246] 0 2024-03-21T10:08:57.751 INFO:tasks.workunit.client.1.smithi174.stdout:8/426: rename d9/d2a/f88 to d9/d2a/d39/d76/d19/d31/f8f 0 2024-03-21T10:08:57.751 INFO:tasks.workunit.client.1.smithi174.stdout:3/481: dwrite d0/d12/d14/d1a/d27/f43 [0,4194304] 0 2024-03-21T10:08:57.751 INFO:tasks.workunit.client.0.smithi153.stdout:5/657: write d4/d5/f7 [1483584,37897] 0 2024-03-21T10:08:57.754 INFO:tasks.workunit.client.0.smithi153.stdout:3/703: mknod dd/d41/d80/dbb/def/cf2 0 2024-03-21T10:08:57.754 INFO:tasks.workunit.client.1.smithi174.stdout:2/499: truncate d5/d25/fa4 691977 0 2024-03-21T10:08:57.755 INFO:tasks.workunit.client.1.smithi174.stdout:5/546: dread d1/d49/d4f/d51/f56 [0,4194304] 0 2024-03-21T10:08:57.756 INFO:tasks.workunit.client.0.smithi153.stdout:6/849: fsync d1f/d93/dd8/ffb 0 2024-03-21T10:08:57.757 INFO:tasks.workunit.client.0.smithi153.stdout:2/741: mkdir dd/d15/d18/d2c/d33/d98/dc7/de1/dfe 0 2024-03-21T10:08:57.759 INFO:tasks.workunit.client.1.smithi174.stdout:8/427: write d9/d2a/d39/d76/d19/f7d [800726,90909] 0 2024-03-21T10:08:57.759 INFO:tasks.workunit.client.1.smithi174.stdout:3/482: mkdir d0/d12/d14/d1a/d44/d57/d93/d7d/da6 0 2024-03-21T10:08:57.761 INFO:tasks.workunit.client.1.smithi174.stdout:1/520: rmdir d4/d7/d33/d43/d77 39 2024-03-21T10:08:57.763 INFO:tasks.workunit.client.0.smithi153.stdout:3/704: fsync dd/d1e/d45/f4d 0 2024-03-21T10:08:57.764 INFO:tasks.workunit.client.1.smithi174.stdout:2/500: symlink d5/d25/d27/d2d/d45/d5b/d2f/d68/d9e/lab 0 2024-03-21T10:08:57.769 INFO:tasks.workunit.client.0.smithi153.stdout:8/653: mknod d5/d9/d43/dd/d20/d26/d47/d69/cd8 0 2024-03-21T10:08:57.770 INFO:tasks.workunit.client.1.smithi174.stdout:1/521: truncate d4/d67/f74 1073945 0 2024-03-21T10:08:57.770 INFO:tasks.workunit.client.1.smithi174.stdout:3/483: fdatasync d0/d12/d14/d1a/d44/d57/d93/f1b 0 2024-03-21T10:08:57.774 INFO:tasks.workunit.client.0.smithi153.stdout:4/677: dread d11/d18/f61 [0,4194304] 0 2024-03-21T10:08:57.775 INFO:tasks.workunit.client.1.smithi174.stdout:8/428: symlink d9/d2a/d39/d76/d19/d27/l90 0 2024-03-21T10:08:57.776 INFO:tasks.workunit.client.0.smithi153.stdout:7/565: sync 2024-03-21T10:08:57.776 INFO:tasks.workunit.client.0.smithi153.stdout:0/614: dwrite d4/d16/d19/fa2 [4194304,4194304] 0 2024-03-21T10:08:57.776 INFO:tasks.workunit.client.0.smithi153.stdout:1/538: sync 2024-03-21T10:08:57.776 INFO:tasks.workunit.client.0.smithi153.stdout:5/658: rename d4/d5/df/d67/da0/db3/f53 to d4/d5/df/d67/da0/db3/d8d/fd8 0 2024-03-21T10:08:57.780 INFO:tasks.workunit.client.0.smithi153.stdout:8/654: chown d5/d9/d43/dd/d20/d26/lc3 5 1 2024-03-21T10:08:57.781 INFO:tasks.workunit.client.1.smithi174.stdout:1/522: rename d4/d7/d32/d7d/caf to d4/d7/d15/d53/cb5 0 2024-03-21T10:08:57.782 INFO:tasks.workunit.client.1.smithi174.stdout:7/428: dwrite d0/d4/d10/d1b/f25 [0,4194304] 0 2024-03-21T10:08:57.783 INFO:tasks.workunit.client.1.smithi174.stdout:5/547: dwrite d1/d16/f40 [0,4194304] 0 2024-03-21T10:08:57.785 INFO:tasks.workunit.client.1.smithi174.stdout:1/523: chown d4/d7/d15/f9b 4 1 2024-03-21T10:08:57.786 INFO:tasks.workunit.client.1.smithi174.stdout:0/487: dwrite d9/d21/d23/da5/f66 [0,4194304] 0 2024-03-21T10:08:57.787 INFO:tasks.workunit.client.1.smithi174.stdout:3/484: mkdir d0/d12/d14/d67/da1/da7 0 2024-03-21T10:08:57.789 INFO:tasks.workunit.client.1.smithi174.stdout:2/501: getdents d5/d25/d27/d2d/d45/d5b/d55 0 2024-03-21T10:08:57.790 INFO:tasks.workunit.client.0.smithi153.stdout:1/539: fsync d6/d10/d2c/f33 0 2024-03-21T10:08:57.791 INFO:tasks.workunit.client.1.smithi174.stdout:1/524: dread - d4/d7/d15/f9b zero size 2024-03-21T10:08:57.793 INFO:tasks.workunit.client.0.smithi153.stdout:3/705: dwrite dd/f27 [0,4194304] 0 2024-03-21T10:08:57.794 INFO:tasks.workunit.client.1.smithi174.stdout:0/488: rename d9/d21/d31/d9f/d79 to d9/d21/d31/d9f/d79/d82/dac 22 2024-03-21T10:08:57.796 INFO:tasks.workunit.client.1.smithi174.stdout:7/429: write d0/f88 [419472,84530] 0 2024-03-21T10:08:57.798 INFO:tasks.workunit.client.0.smithi153.stdout:9/686: dwrite d1/d43/f4f [4194304,4194304] 0 2024-03-21T10:08:57.799 INFO:tasks.workunit.client.0.smithi153.stdout:7/566: dread d4/d19/fb6 [0,4194304] 0 2024-03-21T10:08:57.801 INFO:tasks.workunit.client.1.smithi174.stdout:3/485: mkdir d0/d12/d14/d1a/d44/d57/d93/d7d/da6/da8 0 2024-03-21T10:08:57.804 INFO:tasks.workunit.client.1.smithi174.stdout:3/486: fsync d0/d12/d14/d1a/d44/d57/d93/f78 0 2024-03-21T10:08:57.810 INFO:tasks.workunit.client.0.smithi153.stdout:2/742: dwrite dd/d15/d18/d3a/d8c/faa [0,4194304] 0 2024-03-21T10:08:57.813 INFO:tasks.workunit.client.1.smithi174.stdout:1/525: rename d4/d5/f9e to d4/d7/d28/d59/fb6 0 2024-03-21T10:08:57.815 INFO:tasks.workunit.client.1.smithi174.stdout:0/489: symlink d9/d21/d23/da5/lad 0 2024-03-21T10:08:57.818 INFO:tasks.workunit.client.0.smithi153.stdout:6/850: write d1f/f36 [4429406,12745] 0 2024-03-21T10:08:57.819 INFO:tasks.workunit.client.0.smithi153.stdout:2/743: read dd/d15/d18/d3a/d8c/faa [1886911,29081] 0 2024-03-21T10:08:57.824 INFO:tasks.workunit.client.1.smithi174.stdout:2/502: creat d5/d25/d27/d2d/d45/d5b/d2f/da8/fac x:0 0 0 2024-03-21T10:08:57.827 INFO:tasks.workunit.client.0.smithi153.stdout:5/659: read d4/d5/df/f58 [2225925,33209] 0 2024-03-21T10:08:57.828 INFO:tasks.workunit.client.0.smithi153.stdout:7/567: rename d4/d14/d2f/d78/f71 to d4/d2a/d40/d97/db9/fbf 0 2024-03-21T10:08:57.828 INFO:tasks.workunit.client.0.smithi153.stdout:8/655: link d5/d9/d43/dd/f38 d5/d9/d43/d7b/dc4/da7/fd9 0 2024-03-21T10:08:57.829 INFO:tasks.workunit.client.1.smithi174.stdout:9/530: sync 2024-03-21T10:08:57.829 INFO:tasks.workunit.client.1.smithi174.stdout:3/487: mkdir d0/d12/d14/da9 0 2024-03-21T10:08:57.829 INFO:tasks.workunit.client.1.smithi174.stdout:5/548: dwrite d1/d16/d2e/d66/d8b/f48 [0,4194304] 0 2024-03-21T10:08:57.831 INFO:tasks.workunit.client.1.smithi174.stdout:8/429: dwrite d9/d2a/d39/d76/d19/f7d [0,4194304] 0 2024-03-21T10:08:57.831 INFO:tasks.workunit.client.1.smithi174.stdout:0/490: write d9/d21/d31/d3c/f60 [6443177,77610] 0 2024-03-21T10:08:57.832 INFO:tasks.workunit.client.0.smithi153.stdout:2/744: symlink dd/d15/d18/d3a/d3d/lff 0 2024-03-21T10:08:57.834 INFO:tasks.workunit.client.1.smithi174.stdout:0/491: read - d9/f6d zero size 2024-03-21T10:08:57.835 INFO:tasks.workunit.client.0.smithi153.stdout:6/851: mkdir d1f/d2b/d10a/d40/d7b/d130 0 2024-03-21T10:08:57.836 INFO:tasks.workunit.client.1.smithi174.stdout:7/430: rename d0/d4/l37 to d0/l96 0 2024-03-21T10:08:57.837 INFO:tasks.workunit.client.0.smithi153.stdout:0/615: dwrite d4/f37 [0,4194304] 0 2024-03-21T10:08:57.838 INFO:tasks.workunit.client.1.smithi174.stdout:3/488: rmdir d0/d12/d14/d67/da1 39 2024-03-21T10:08:57.839 INFO:tasks.workunit.client.0.smithi153.stdout:4/678: dwrite d11/d2d/d3d/f99 [4194304,4194304] 0 2024-03-21T10:08:57.839 INFO:tasks.workunit.client.1.smithi174.stdout:8/430: dread - d9/d2a/d39/d76/f8d zero size 2024-03-21T10:08:57.841 INFO:tasks.workunit.client.1.smithi174.stdout:7/431: write d0/d53/f80 [390459,129870] 0 2024-03-21T10:08:57.841 INFO:tasks.workunit.client.1.smithi174.stdout:3/489: readlink d0/d12/d14/d1a/d44/d94/l96 0 2024-03-21T10:08:57.842 INFO:tasks.workunit.client.0.smithi153.stdout:3/706: dwrite dd/f25 [0,4194304] 0 2024-03-21T10:08:57.842 INFO:tasks.workunit.client.1.smithi174.stdout:5/549: unlink d1/d16/d2e/d66/d8b/fa4 0 2024-03-21T10:08:57.849 INFO:tasks.workunit.client.0.smithi153.stdout:1/540: dwrite d6/f17 [4194304,4194304] 0 2024-03-21T10:08:57.849 INFO:tasks.workunit.client.0.smithi153.stdout:4/679: chown d11/f44 2864 1 2024-03-21T10:08:57.851 INFO:tasks.workunit.client.1.smithi174.stdout:9/531: creat d2/d11/d1d/d4b/d72/fbb x:0 0 0 2024-03-21T10:08:57.855 INFO:tasks.workunit.client.0.smithi153.stdout:9/687: truncate d1/d4e/f98 1667080 0 2024-03-21T10:08:57.858 INFO:tasks.workunit.client.0.smithi153.stdout:8/656: truncate d5/d9/d43/dd/d20/d26/d24/d52/f72 715884 0 2024-03-21T10:08:57.860 INFO:tasks.workunit.client.1.smithi174.stdout:1/526: rename d4/d7/d33/d43/d51/f8d to d4/d7/d15/d53/fb7 0 2024-03-21T10:08:57.860 INFO:tasks.workunit.client.1.smithi174.stdout:1/527: read d4/d5/f2a [440845,6433] 0 2024-03-21T10:08:57.861 INFO:tasks.workunit.client.1.smithi174.stdout:8/431: symlink d9/d2a/d39/d76/d19/d27/l91 0 2024-03-21T10:08:57.865 INFO:tasks.workunit.client.0.smithi153.stdout:5/660: read d4/f46 [2435171,13166] 0 2024-03-21T10:08:57.865 INFO:tasks.workunit.client.0.smithi153.stdout:6/852: rename d1f/d93/dd8/d59/c68 to d1f/de0/c131 0 2024-03-21T10:08:57.867 INFO:tasks.workunit.client.1.smithi174.stdout:8/432: chown d9/d2a/d39/f75 27 1 2024-03-21T10:08:57.874 INFO:tasks.workunit.client.0.smithi153.stdout:1/541: dwrite d6/d10/d35/da5/f4b [0,4194304] 0 2024-03-21T10:08:57.874 INFO:tasks.workunit.client.1.smithi174.stdout:9/532: dwrite d2/d11/d1d/d8b/fa6 [0,4194304] 0 2024-03-21T10:08:57.877 INFO:tasks.workunit.client.1.smithi174.stdout:3/490: symlink d0/d12/laa 0 2024-03-21T10:08:57.878 INFO:tasks.workunit.client.0.smithi153.stdout:8/657: creat d5/d9/d43/dd/d20/d26/d47/d9a/fda x:0 0 0 2024-03-21T10:08:57.880 INFO:tasks.workunit.client.0.smithi153.stdout:9/688: truncate d1/d43/d44/d46/d9e/d24/d41/f6a 770401 0 2024-03-21T10:08:57.884 INFO:tasks.workunit.client.0.smithi153.stdout:6/853: unlink d1f/f4b 0 2024-03-21T10:08:57.885 INFO:tasks.workunit.client.1.smithi174.stdout:6/386: sync 2024-03-21T10:08:57.885 INFO:tasks.workunit.client.1.smithi174.stdout:4/534: sync 2024-03-21T10:08:57.887 INFO:tasks.workunit.client.1.smithi174.stdout:1/528: mkdir d4/d7/d68/d8e/db8 0 2024-03-21T10:08:57.890 INFO:tasks.workunit.client.1.smithi174.stdout:8/433: mknod d9/d53/c92 0 2024-03-21T10:08:57.891 INFO:tasks.workunit.client.0.smithi153.stdout:6/854: chown d1f/d2b/db3 72754 1 2024-03-21T10:08:57.891 INFO:tasks.workunit.client.0.smithi153.stdout:4/680: link d11/c27 d11/d2d/da3/cd1 0 2024-03-21T10:08:57.893 INFO:tasks.workunit.client.0.smithi153.stdout:3/707: getdents dd/d1e/d2f/d35/dd0 0 2024-03-21T10:08:57.893 INFO:tasks.workunit.client.1.smithi174.stdout:4/535: read d4/f2c [31579,100798] 0 2024-03-21T10:08:57.895 INFO:tasks.workunit.client.0.smithi153.stdout:4/681: dread - d11/d18/d7e/d93/fb8 zero size 2024-03-21T10:08:57.895 INFO:tasks.workunit.client.0.smithi153.stdout:3/708: dread - dd/d1e/ff0 zero size 2024-03-21T10:08:57.899 INFO:tasks.workunit.client.0.smithi153.stdout:5/661: dwrite d4/d8/d3b/d56/da5/fc4 [0,4194304] 0 2024-03-21T10:08:57.899 INFO:tasks.workunit.client.1.smithi174.stdout:0/492: link d9/d35/d4a/f5e d9/d21/d23/d80/fae 0 2024-03-21T10:08:57.901 INFO:tasks.workunit.client.1.smithi174.stdout:1/529: creat d4/d2d/fb9 x:0 0 0 2024-03-21T10:08:57.901 INFO:tasks.workunit.client.0.smithi153.stdout:1/542: getdents d6/d10/d5c 0 2024-03-21T10:08:57.904 INFO:tasks.workunit.client.1.smithi174.stdout:9/533: link d2/d11/d1d/d4b/f56 d2/d11/d1d/d4b/d72/fbc 0 2024-03-21T10:08:57.905 INFO:tasks.workunit.client.0.smithi153.stdout:6/855: rmdir d1f/d2b/d10a/d40/d7b 39 2024-03-21T10:08:57.906 INFO:tasks.workunit.client.0.smithi153.stdout:0/616: write d4/d16/d19/d30/f41 [419167,23737] 0 2024-03-21T10:08:57.906 INFO:tasks.workunit.client.1.smithi174.stdout:6/387: chown d0/d3/d5/d6/lc 1214 1 2024-03-21T10:08:57.907 INFO:tasks.workunit.client.0.smithi153.stdout:7/568: truncate d4/d2a/d40/d97/db9/fbf 1068233 0 2024-03-21T10:08:57.911 INFO:tasks.workunit.client.0.smithi153.stdout:4/682: chown d11/d2d/d6b/db6 96105053 1 2024-03-21T10:08:57.912 INFO:tasks.workunit.client.1.smithi174.stdout:0/493: mknod d9/d1c/caf 0 2024-03-21T10:08:57.913 INFO:tasks.workunit.client.1.smithi174.stdout:1/530: mknod d4/d5/d6c/cba 0 2024-03-21T10:08:57.914 INFO:tasks.workunit.client.0.smithi153.stdout:0/617: chown d4/d16/d19/d30/d6c/d98 105 1 2024-03-21T10:08:57.914 INFO:tasks.workunit.client.0.smithi153.stdout:8/658: dread d5/d9/d43/dd/d20/d26/d24/f82 [0,4194304] 0 2024-03-21T10:08:57.916 INFO:tasks.workunit.client.1.smithi174.stdout:3/491: symlink d0/d12/d14/d4c/d7a/d9e/lab 0 2024-03-21T10:08:57.916 INFO:tasks.workunit.client.1.smithi174.stdout:2/503: write d5/d25/d27/d2d/d45/d5b/d2f/d66/f93 [3196191,54371] 0 2024-03-21T10:08:57.916 INFO:tasks.workunit.client.0.smithi153.stdout:7/569: truncate d4/d1b/d20/d9a/db7/f96 526596 0 2024-03-21T10:08:57.918 INFO:tasks.workunit.client.1.smithi174.stdout:6/388: mkdir d0/d3/d5/d3b/d79 0 2024-03-21T10:08:57.918 INFO:tasks.workunit.client.0.smithi153.stdout:2/745: truncate dd/d15/d18/d2c/d4b/d56/d63/fdf 331805 0 2024-03-21T10:08:57.919 INFO:tasks.workunit.client.1.smithi174.stdout:2/504: chown d5/d25/f7e 122 1 2024-03-21T10:08:57.920 INFO:tasks.workunit.client.1.smithi174.stdout:8/434: symlink d9/d2a/l93 0 2024-03-21T10:08:57.922 INFO:tasks.workunit.client.1.smithi174.stdout:0/494: fsync d9/d1c/f1e 0 2024-03-21T10:08:57.924 INFO:tasks.workunit.client.1.smithi174.stdout:8/435: chown d9/d2a/d39/d76/d19/d27/d79/c82 649825 1 2024-03-21T10:08:57.928 INFO:tasks.workunit.client.1.smithi174.stdout:3/492: mkdir d0/d12/d14/d1a/d44/d65/d7e/dac 0 2024-03-21T10:08:57.938 INFO:tasks.workunit.client.1.smithi174.stdout:9/534: dwrite d2/d54/f5c [0,4194304] 0 2024-03-21T10:08:57.938 INFO:tasks.workunit.client.1.smithi174.stdout:2/505: dread d5/d25/d27/d2d/d45/d5b/d2f/d4b/f78 [0,4194304] 0 2024-03-21T10:08:57.943 INFO:tasks.workunit.client.1.smithi174.stdout:8/436: creat d9/d2a/d39/d76/d19/d31/f94 x:0 0 0 2024-03-21T10:08:57.943 INFO:tasks.workunit.client.1.smithi174.stdout:7/432: truncate d0/d4/d10/d1b/f4e 1976742 0 2024-03-21T10:08:57.943 INFO:tasks.workunit.client.1.smithi174.stdout:5/550: truncate d1/d16/d2e/d66/d8b/f33 1777213 0 2024-03-21T10:08:57.949 INFO:tasks.workunit.client.1.smithi174.stdout:2/506: write d5/d25/d27/d2d/d45/d5b/d2f/d4b/d76/f86 [1818433,23819] 0 2024-03-21T10:08:57.951 INFO:tasks.workunit.client.1.smithi174.stdout:4/536: truncate d4/d21/f49 3816080 0 2024-03-21T10:08:57.951 INFO:tasks.workunit.client.1.smithi174.stdout:8/437: creat d9/d53/f95 x:0 0 0 2024-03-21T10:08:57.957 INFO:tasks.workunit.client.1.smithi174.stdout:5/551: mknod d1/cc7 0 2024-03-21T10:08:57.958 INFO:tasks.workunit.client.1.smithi174.stdout:3/493: dwrite d0/d12/d46/d59/f95 [0,4194304] 0 2024-03-21T10:08:57.958 INFO:tasks.workunit.client.1.smithi174.stdout:2/507: mkdir d5/d25/d27/d2e/d85/dad 0 2024-03-21T10:08:57.965 INFO:tasks.workunit.client.1.smithi174.stdout:3/494: dread - d0/d12/d14/d1a/d5c/f8d zero size 2024-03-21T10:08:57.969 INFO:tasks.workunit.client.1.smithi174.stdout:1/531: truncate d4/d67/f74 537622 0 2024-03-21T10:08:57.977 INFO:tasks.workunit.client.1.smithi174.stdout:5/552: symlink d1/d5c/d96/lc8 0 2024-03-21T10:08:57.981 INFO:tasks.workunit.client.1.smithi174.stdout:2/508: getdents d5/d25/d27/d2e/d90 0 2024-03-21T10:08:57.982 INFO:tasks.workunit.client.1.smithi174.stdout:0/495: dread d9/d21/d31/d3c/f60 [8388608,4194304] 0 2024-03-21T10:08:57.983 INFO:tasks.workunit.client.1.smithi174.stdout:5/553: creat d1/d2f/fc9 x:0 0 0 2024-03-21T10:08:57.985 INFO:tasks.workunit.client.1.smithi174.stdout:5/554: dread - d1/d49/f53 zero size 2024-03-21T10:08:57.988 INFO:tasks.workunit.client.1.smithi174.stdout:2/509: fdatasync d5/d25/d27/d2d/d45/f6e 0 2024-03-21T10:08:57.990 INFO:tasks.workunit.client.1.smithi174.stdout:2/510: chown d5 5872 1 2024-03-21T10:08:57.992 INFO:tasks.workunit.client.1.smithi174.stdout:5/555: mkdir d1/dca 0 2024-03-21T10:08:57.994 INFO:tasks.workunit.client.0.smithi153.stdout:9/689: write d1/d4e/f54 [2427865,130051] 0 2024-03-21T10:08:57.995 INFO:tasks.workunit.client.1.smithi174.stdout:0/496: truncate d9/d21/d31/d5c/f7c 258257 0 2024-03-21T10:08:57.996 INFO:tasks.workunit.client.1.smithi174.stdout:3/495: dread d0/d12/d14/d1a/f3d [0,4194304] 0 2024-03-21T10:08:57.997 INFO:tasks.workunit.client.0.smithi153.stdout:8/659: mknod d5/d9/d43/dd/d2d/d45/d88/d85/cdb 0 2024-03-21T10:08:57.998 INFO:tasks.workunit.client.0.smithi153.stdout:7/570: creat d4/d2a/d40/d97/fc0 x:0 0 0 2024-03-21T10:08:57.998 INFO:tasks.workunit.client.0.smithi153.stdout:0/618: dread - d4/d16/d19/d30/d6c/f9b zero size 2024-03-21T10:08:57.999 INFO:tasks.workunit.client.1.smithi174.stdout:5/556: creat d1/d16/d2e/d66/d8b/d85/fcb x:0 0 0 2024-03-21T10:08:57.999 INFO:tasks.workunit.client.1.smithi174.stdout:4/537: dwrite d4/f2c [0,4194304] 0 2024-03-21T10:08:57.999 INFO:tasks.workunit.client.1.smithi174.stdout:0/497: stat d9/d21/d23/f91 0 2024-03-21T10:08:58.002 INFO:tasks.workunit.client.0.smithi153.stdout:3/709: write dd/d1e/d2f/d35/d68/d75/ddd/fa8 [708038,108616] 0 2024-03-21T10:08:58.004 INFO:tasks.workunit.client.0.smithi153.stdout:9/690: fsync d1/d43/d44/f85 0 2024-03-21T10:08:58.006 INFO:tasks.workunit.client.0.smithi153.stdout:7/571: rmdir d4/d11 39 2024-03-21T10:08:58.008 INFO:tasks.workunit.client.0.smithi153.stdout:5/662: dwrite d4/d8/f68 [0,4194304] 0 2024-03-21T10:08:58.011 INFO:tasks.workunit.client.1.smithi174.stdout:6/389: dread d0/d3/d2a/f2f [0,4194304] 0 2024-03-21T10:08:58.014 INFO:tasks.workunit.client.1.smithi174.stdout:5/557: write d1/d16/d2e/d66/fbe [794305,12691] 0 2024-03-21T10:08:58.015 INFO:tasks.workunit.client.0.smithi153.stdout:6/856: rename d1f/da0/cdb to d1f/c132 0 2024-03-21T10:08:58.016 INFO:tasks.workunit.client.0.smithi153.stdout:9/691: read d1/d43/d44/d46/f37 [1552686,21751] 0 2024-03-21T10:08:58.016 INFO:tasks.workunit.client.1.smithi174.stdout:7/433: dread d0/f26 [0,4194304] 0 2024-03-21T10:08:58.017 INFO:tasks.workunit.client.1.smithi174.stdout:2/511: rename d5/d25/d27/d2d/d45/d5b/d2f/da8 to d5/d25/d27/d2d/d45/dae 0 2024-03-21T10:08:58.018 INFO:tasks.workunit.client.0.smithi153.stdout:4/683: getdents d11/d2d 0 2024-03-21T10:08:58.020 INFO:tasks.workunit.client.0.smithi153.stdout:7/572: write d4/d14/d2f/d78/f29 [3363413,10106] 0 2024-03-21T10:08:58.021 INFO:tasks.workunit.client.0.smithi153.stdout:9/692: readlink d1/d43/d44/lb2 0 2024-03-21T10:08:58.022 INFO:tasks.workunit.client.0.smithi153.stdout:6/857: fdatasync d1f/f25 0 2024-03-21T10:08:58.023 INFO:tasks.workunit.client.1.smithi174.stdout:7/434: write d0/d4/d10/d79/f83 [600544,115748] 0 2024-03-21T10:08:58.024 INFO:tasks.workunit.client.1.smithi174.stdout:8/438: dwrite d9/d53/f6f [0,4194304] 0 2024-03-21T10:08:58.025 INFO:tasks.workunit.client.1.smithi174.stdout:4/538: mknod d4/d21/d31/d6a/cae 0 2024-03-21T10:08:58.027 INFO:tasks.workunit.client.0.smithi153.stdout:2/746: rename dd/d15/d18/d3a/d8c/faa to dd/d15/d18/d2c/d4b/d56/d63/d8d/d9b/dc3/df5/dae/f100 0 2024-03-21T10:08:58.028 INFO:tasks.workunit.client.0.smithi153.stdout:4/684: fsync d11/d18/d50/f8c 0 2024-03-21T10:08:58.029 INFO:tasks.workunit.client.0.smithi153.stdout:7/573: creat d4/d19/fc1 x:0 0 0 2024-03-21T10:08:58.031 INFO:tasks.workunit.client.1.smithi174.stdout:6/390: creat d0/f7a x:0 0 0 2024-03-21T10:08:58.031 INFO:tasks.workunit.client.0.smithi153.stdout:6/858: mknod d1f/d2b/db3/c133 0 2024-03-21T10:08:58.033 INFO:tasks.workunit.client.0.smithi153.stdout:5/663: rename d4/d5/de to d4/d8/d5a/d5d/dd9 0 2024-03-21T10:08:58.035 INFO:tasks.workunit.client.1.smithi174.stdout:5/558: mknod d1/d16/d2e/d8d/d99/ccc 0 2024-03-21T10:08:58.036 INFO:tasks.workunit.client.1.smithi174.stdout:6/391: chown d0/d3/d5/df/l1e 7081 1 2024-03-21T10:08:58.037 INFO:tasks.workunit.client.0.smithi153.stdout:3/710: dwrite dd/d1e/d2f/d35/daf/dcc/f9e [0,4194304] 0 2024-03-21T10:08:58.038 INFO:tasks.workunit.client.1.smithi174.stdout:0/498: write d9/f15 [674739,91686] 0 2024-03-21T10:08:58.038 INFO:tasks.workunit.client.1.smithi174.stdout:4/539: symlink d4/d21/d31/laf 0 2024-03-21T10:08:58.039 INFO:tasks.workunit.client.0.smithi153.stdout:9/693: link d1/fd9 d1/d3b/daa/fea 0 2024-03-21T10:08:58.041 INFO:tasks.workunit.client.1.smithi174.stdout:3/496: dwrite d0/f2 [0,4194304] 0 2024-03-21T10:08:58.041 INFO:tasks.workunit.client.1.smithi174.stdout:8/439: unlink d9/d2a/d39/d76/d19/d27/d60/l52 0 2024-03-21T10:08:58.043 INFO:tasks.workunit.client.0.smithi153.stdout:9/694: chown d1/d43/d44/d46/d4d/d5c/ldb 10 1 2024-03-21T10:08:58.045 INFO:tasks.workunit.client.1.smithi174.stdout:8/440: stat d9/d2a/d39/d76/f42 0 2024-03-21T10:08:58.045 INFO:tasks.workunit.client.1.smithi174.stdout:4/540: readlink d4/d21/d31/d3a/l64 0 2024-03-21T10:08:58.046 INFO:tasks.workunit.client.1.smithi174.stdout:3/497: chown d0/d12/d14/d1a/d44/d57/d93/l4a 1063431 1 2024-03-21T10:08:58.048 INFO:tasks.workunit.client.0.smithi153.stdout:4/685: creat d11/d18/d54/dab/fd2 x:0 0 0 2024-03-21T10:08:58.049 INFO:tasks.workunit.client.1.smithi174.stdout:4/541: truncate d4/d21/f8c 1003222 0 2024-03-21T10:08:58.051 INFO:tasks.workunit.client.1.smithi174.stdout:7/435: link d0/d4/d10/d1b/l84 d0/d4/d10/d1b/d86/l97 0 2024-03-21T10:08:58.054 INFO:tasks.workunit.client.1.smithi174.stdout:6/392: getdents d0/d3/d5/d3b/d79 0 2024-03-21T10:08:58.055 INFO:tasks.workunit.client.0.smithi153.stdout:2/747: creat dd/f101 x:0 0 0 2024-03-21T10:08:58.057 INFO:tasks.workunit.client.1.smithi174.stdout:4/542: readlink d4/d21/d24/d28/d38/la5 0 2024-03-21T10:08:58.057 INFO:tasks.workunit.client.0.smithi153.stdout:7/574: dwrite d4/d2a/d3b/d60/f7c [0,4194304] 0 2024-03-21T10:08:58.058 INFO:tasks.workunit.client.1.smithi174.stdout:6/393: write d0/d3/d20/f23 [6394022,41527] 0 2024-03-21T10:08:58.060 INFO:tasks.workunit.client.1.smithi174.stdout:7/436: write d0/d4/d10/d1b/f7c [736374,62808] 0 2024-03-21T10:08:58.061 INFO:tasks.workunit.client.0.smithi153.stdout:6/859: fsync d1f/d99/faf 0 2024-03-21T10:08:58.063 INFO:tasks.workunit.client.1.smithi174.stdout:5/559: rename d1/d16/c1e to d1/d16/d2e/ccd 0 2024-03-21T10:08:58.063 INFO:tasks.workunit.client.1.smithi174.stdout:0/499: link d9/f1d d9/d21/d31/d3c/fb0 0 2024-03-21T10:08:58.064 INFO:tasks.workunit.client.1.smithi174.stdout:6/394: write d0/d3/d5/f5c [715088,320] 0 2024-03-21T10:08:58.065 INFO:tasks.workunit.client.1.smithi174.stdout:7/437: readlink d0/d4/d10/d1b/d86/l89 0 2024-03-21T10:08:58.067 INFO:tasks.workunit.client.1.smithi174.stdout:8/441: getdents d9/d2a/d39/d76/d19/d31 0 2024-03-21T10:08:58.069 INFO:tasks.workunit.client.1.smithi174.stdout:5/560: creat d1/d49/d93/d9c/fce x:0 0 0 2024-03-21T10:08:58.069 INFO:tasks.workunit.client.1.smithi174.stdout:7/438: readlink d0/d4/l40 0 2024-03-21T10:08:58.070 INFO:tasks.workunit.client.0.smithi153.stdout:9/695: mknod d1/d43/d44/d46/d9e/d61/ceb 0 2024-03-21T10:08:58.072 INFO:tasks.workunit.client.1.smithi174.stdout:3/498: dread d0/d12/d14/d1a/f3d [0,4194304] 0 2024-03-21T10:08:58.081 INFO:tasks.workunit.client.0.smithi153.stdout:7/575: chown d4/d2a/d3b/d81/f90 0 1 2024-03-21T10:08:58.081 INFO:tasks.workunit.client.1.smithi174.stdout:4/543: rename d4/d21/d24/d28/f8a to d4/d21/d31/d6a/fb0 0 2024-03-21T10:08:58.081 INFO:tasks.workunit.client.0.smithi153.stdout:6/860: write d1f/d2b/d10a/d40/d4f/d66/f6f [1848919,88021] 0 2024-03-21T10:08:58.083 INFO:tasks.workunit.client.0.smithi153.stdout:5/664: dwrite d4/d5/d16/f19 [4194304,4194304] 0 2024-03-21T10:08:58.085 INFO:tasks.workunit.client.1.smithi174.stdout:7/439: creat d0/f98 x:0 0 0 2024-03-21T10:08:58.091 INFO:tasks.workunit.client.0.smithi153.stdout:4/686: dwrite d11/fb0 [0,4194304] 0 2024-03-21T10:08:58.092 INFO:tasks.workunit.client.1.smithi174.stdout:8/442: rename d9/d2a/f34 to d9/d53/f96 0 2024-03-21T10:08:58.098 INFO:tasks.workunit.client.1.smithi174.stdout:7/440: creat d0/d4/d10/d79/f99 x:0 0 0 2024-03-21T10:08:58.098 INFO:tasks.workunit.client.0.smithi153.stdout:4/687: readlink d11/d2d/d57/d85/la5 0 2024-03-21T10:08:58.100 INFO:tasks.workunit.client.0.smithi153.stdout:7/576: rmdir d4/d2a/dab 0 2024-03-21T10:08:58.102 INFO:tasks.workunit.client.1.smithi174.stdout:3/499: dwrite d0/d12/d14/d1a/d27/d3c/fa3 [0,4194304] 0 2024-03-21T10:08:58.102 INFO:tasks.workunit.client.1.smithi174.stdout:6/395: dwrite d0/d3/d5/d3b/f46 [4194304,4194304] 0 2024-03-21T10:08:58.103 INFO:tasks.workunit.client.0.smithi153.stdout:2/748: dwrite dd/d15/d18/d2c/d4b/fd0 [0,4194304] 0 2024-03-21T10:08:58.105 INFO:tasks.workunit.client.0.smithi153.stdout:5/665: creat d4/d5/df/d67/d86/fda x:0 0 0 2024-03-21T10:08:58.106 INFO:tasks.workunit.client.1.smithi174.stdout:5/561: dwrite d1/d2f/fae [0,4194304] 0 2024-03-21T10:08:58.108 INFO:tasks.workunit.client.0.smithi153.stdout:6/861: rename d1f/l6a to d1f/de0/de1/df2/d10f/l134 0 2024-03-21T10:08:58.108 INFO:tasks.workunit.client.0.smithi153.stdout:3/711: dwrite dd/d1e/d45/fa0 [0,4194304] 0 2024-03-21T10:08:58.108 INFO:tasks.workunit.client.0.smithi153.stdout:4/688: symlink d11/d2d/d6b/d9c/ld3 0 2024-03-21T10:08:58.109 INFO:tasks.workunit.client.0.smithi153.stdout:9/696: link d1/d43/d44/d46/d9e/d24/d41/dbe/dcc/l95 d1/d43/d72/d9d/ddc/lec 0 2024-03-21T10:08:58.111 INFO:tasks.workunit.client.0.smithi153.stdout:4/689: chown d11/d18/d6d 6141787 1 2024-03-21T10:08:58.111 INFO:tasks.workunit.client.1.smithi174.stdout:0/500: rename d9/d21/d31/d5c/d92/f97 to d9/d21/d31/d3c/d72/fb1 0 2024-03-21T10:08:58.114 INFO:tasks.workunit.client.0.smithi153.stdout:5/666: creat d4/d5/df/d67/da0/db3/fdb x:0 0 0 2024-03-21T10:08:58.115 INFO:tasks.workunit.client.1.smithi174.stdout:0/501: write d9/d21/d31/d5c/d86/f63 [1061245,63760] 0 2024-03-21T10:08:58.116 INFO:tasks.workunit.client.1.smithi174.stdout:7/441: mkdir d0/d4/d10/d44/d81/d9a 0 2024-03-21T10:08:58.119 INFO:tasks.workunit.client.0.smithi153.stdout:4/690: symlink d11/d18/d6d/ld4 0 2024-03-21T10:08:58.119 INFO:tasks.workunit.client.0.smithi153.stdout:9/697: chown d1/d4/d5f/fe6 23 1 2024-03-21T10:08:58.124 INFO:tasks.workunit.client.0.smithi153.stdout:4/691: write d11/f16 [1637182,94384] 0 2024-03-21T10:08:58.126 INFO:tasks.workunit.client.0.smithi153.stdout:6/862: mkdir d1f/d93/dd8/d59/dfc/d135 0 2024-03-21T10:08:58.128 INFO:tasks.workunit.client.0.smithi153.stdout:1/543: sync 2024-03-21T10:08:58.128 INFO:tasks.workunit.client.1.smithi174.stdout:1/532: sync 2024-03-21T10:08:58.128 INFO:tasks.workunit.client.1.smithi174.stdout:9/535: sync 2024-03-21T10:08:58.128 INFO:tasks.workunit.client.1.smithi174.stdout:2/512: sync 2024-03-21T10:08:58.129 INFO:tasks.workunit.client.0.smithi153.stdout:1/544: chown d6/d10/c96 13427 1 2024-03-21T10:08:58.130 INFO:tasks.workunit.client.1.smithi174.stdout:5/562: creat d1/d16/d2e/d87/fcf x:0 0 0 2024-03-21T10:08:58.131 INFO:tasks.workunit.client.1.smithi174.stdout:0/502: truncate d9/d1c/f1e 3757957 0 2024-03-21T10:08:58.131 INFO:tasks.workunit.client.0.smithi153.stdout:2/749: mkdir dd/d15/d18/d2c/d33/d98/dc7/de1/d102 0 2024-03-21T10:08:58.131 INFO:tasks.workunit.client.0.smithi153.stdout:8/660: dread d5/d9/d43/f77 [0,4194304] 0 2024-03-21T10:08:58.133 INFO:tasks.workunit.client.1.smithi174.stdout:6/396: fsync d0/d3/d5/df/d16/f17 0 2024-03-21T10:08:58.133 INFO:tasks.workunit.client.1.smithi174.stdout:2/513: write d5/d25/d27/f40 [286865,10887] 0 2024-03-21T10:08:58.133 INFO:tasks.workunit.client.1.smithi174.stdout:4/544: fsync d4/d19/f22 0 2024-03-21T10:08:58.138 INFO:tasks.workunit.client.0.smithi153.stdout:9/698: unlink d1/d43/f4f 0 2024-03-21T10:08:58.138 INFO:tasks.workunit.client.0.smithi153.stdout:4/692: creat d11/db1/fd5 x:0 0 0 2024-03-21T10:08:58.138 INFO:tasks.workunit.client.1.smithi174.stdout:8/443: rename d9/d2a/d39/d76/d19/d27/d79/l85 to d9/d2a/d39/d76/l97 0 2024-03-21T10:08:58.139 INFO:tasks.workunit.client.1.smithi174.stdout:2/514: readlink d5/d25/d27/d2d/d45/d5b/lc 0 2024-03-21T10:08:58.140 INFO:tasks.workunit.client.0.smithi153.stdout:6/863: unlink d1f/d93/fdd 0 2024-03-21T10:08:58.142 INFO:tasks.workunit.client.0.smithi153.stdout:9/699: read d1/d43/d44/f3e [701098,130587] 0 2024-03-21T10:08:58.143 INFO:tasks.workunit.client.0.smithi153.stdout:8/661: mkdir d5/d9/d43/dd/d2d/d45/d88/d85/ddc 0 2024-03-21T10:08:58.146 INFO:tasks.workunit.client.1.smithi174.stdout:7/442: creat d0/d4/d10/d44/d81/d9a/f9b x:0 0 0 2024-03-21T10:08:58.150 INFO:tasks.workunit.client.0.smithi153.stdout:6/864: rmdir d1f/d93/df7 39 2024-03-21T10:08:58.150 INFO:tasks.workunit.client.1.smithi174.stdout:5/563: chown d1/d49/d4f/d51/d8f/cb4 1 1 2024-03-21T10:08:58.152 INFO:tasks.workunit.client.0.smithi153.stdout:5/667: rename d4/d8/d5a/d5d/l7d to d4/d8/d3b/ldc 0 2024-03-21T10:08:58.155 INFO:tasks.workunit.client.0.smithi153.stdout:1/545: read d6/d13/d75/f62 [2419563,101591] 0 2024-03-21T10:08:58.157 INFO:tasks.workunit.client.0.smithi153.stdout:4/693: getdents d11/d2d/d3d 0 2024-03-21T10:08:58.159 INFO:tasks.workunit.client.0.smithi153.stdout:4/694: chown d11/f20 848588249 1 2024-03-21T10:08:58.159 INFO:tasks.workunit.client.1.smithi174.stdout:9/536: unlink d2/d11/d1d/l5d 0 2024-03-21T10:08:58.159 INFO:tasks.workunit.client.0.smithi153.stdout:8/662: rename d5/d9/d43/d7b/d7c to d5/d9/d43/dd/ddd 0 2024-03-21T10:08:58.160 INFO:tasks.workunit.client.0.smithi153.stdout:5/668: mknod d4/d8/d5c/d90/dd6/cdd 0 2024-03-21T10:08:58.161 INFO:tasks.workunit.client.1.smithi174.stdout:5/564: fdatasync d1/d2f/d37/f5d 0 2024-03-21T10:08:58.162 INFO:tasks.workunit.client.0.smithi153.stdout:1/546: rename d6/d10/c19 to d6/d3a/d3c/d50/ca9 0 2024-03-21T10:08:58.162 INFO:tasks.workunit.client.0.smithi153.stdout:4/695: fdatasync d11/d2d/d3d/f90 0 2024-03-21T10:08:58.162 INFO:tasks.workunit.client.0.smithi153.stdout:6/865: symlink d1f/d2b/d10a/d40/d115/l136 0 2024-03-21T10:08:58.165 INFO:tasks.workunit.client.0.smithi153.stdout:8/663: fsync d5/d9/d43/dd/d2d/d45/d88/f76 0 2024-03-21T10:08:58.166 INFO:tasks.workunit.client.1.smithi174.stdout:3/500: dread d0/fa [0,4194304] 0 2024-03-21T10:08:58.167 INFO:tasks.workunit.client.0.smithi153.stdout:4/696: mkdir d11/d18/d6d/dd6 0 2024-03-21T10:08:58.167 INFO:tasks.workunit.client.1.smithi174.stdout:5/565: dread - d1/d49/d93/d9c/fb2 zero size 2024-03-21T10:08:58.169 INFO:tasks.workunit.client.0.smithi153.stdout:5/669: mknod d4/d5/df/d67/dbe/cde 0 2024-03-21T10:08:58.169 INFO:tasks.workunit.client.1.smithi174.stdout:9/537: unlink d2/d11/d1d/d2a/fa9 0 2024-03-21T10:08:58.170 INFO:tasks.workunit.client.1.smithi174.stdout:6/397: link d0/d3/d2a/l41 d0/d3/d5/df/l7b 0 2024-03-21T10:08:58.170 INFO:tasks.workunit.client.0.smithi153.stdout:4/697: creat d11/d9d/fd7 x:0 0 0 2024-03-21T10:08:58.172 INFO:tasks.workunit.client.0.smithi153.stdout:1/547: truncate d6/d3a/d3c/d50/f77 3265824 0 2024-03-21T10:08:58.173 INFO:tasks.workunit.client.1.smithi174.stdout:3/501: stat d0/d12/d14/d1a/c1c 0 2024-03-21T10:08:58.177 INFO:tasks.workunit.client.0.smithi153.stdout:5/670: rename c3 to d4/d8/d5c/cdf 0 2024-03-21T10:08:58.178 INFO:tasks.workunit.client.1.smithi174.stdout:7/443: dwrite d0/d4/d10/f94 [0,4194304] 0 2024-03-21T10:08:58.178 INFO:tasks.workunit.client.1.smithi174.stdout:5/566: creat d1/d2f/fd0 x:0 0 0 2024-03-21T10:08:58.179 INFO:tasks.workunit.client.0.smithi153.stdout:4/698: creat d11/d71/d89/dca/fd8 x:0 0 0 2024-03-21T10:08:58.180 INFO:tasks.workunit.client.0.smithi153.stdout:0/619: dread d4/f23 [0,4194304] 0 2024-03-21T10:08:58.187 INFO:tasks.workunit.client.1.smithi174.stdout:2/515: dread d5/d25/d27/d2e/d4f/d65/f92 [0,4194304] 0 2024-03-21T10:08:58.188 INFO:tasks.workunit.client.0.smithi153.stdout:4/699: link d11/d2d/fd0 d11/d2d/fd9 0 2024-03-21T10:08:58.189 INFO:tasks.workunit.client.1.smithi174.stdout:5/567: mknod d1/d49/d4f/cd1 0 2024-03-21T10:08:58.196 INFO:tasks.workunit.client.0.smithi153.stdout:4/700: mkdir d11/d2d/d3d/dc7/dda 0 2024-03-21T10:08:58.196 INFO:tasks.workunit.client.1.smithi174.stdout:7/444: symlink d0/d4/d10/l9c 0 2024-03-21T10:08:58.197 INFO:tasks.workunit.client.0.smithi153.stdout:1/548: dread d6/f2f [0,4194304] 0 2024-03-21T10:08:58.198 INFO:tasks.workunit.client.0.smithi153.stdout:4/701: chown d11/d2d/d3d/dc7/dda 1612 1 2024-03-21T10:08:58.199 INFO:tasks.workunit.client.0.smithi153.stdout:4/702: dread - d11/f96 zero size 2024-03-21T10:08:58.200 INFO:tasks.workunit.client.1.smithi174.stdout:4/545: write d4/d21/f7b [254620,67155] 0 2024-03-21T10:08:58.200 INFO:tasks.workunit.client.0.smithi153.stdout:1/549: truncate d6/d10/d2c/f7e 689865 0 2024-03-21T10:08:58.205 INFO:tasks.workunit.client.0.smithi153.stdout:3/712: write dd/f67 [152404,52601] 0 2024-03-21T10:08:58.207 INFO:tasks.workunit.client.1.smithi174.stdout:9/538: getdents d2/d24 0 2024-03-21T10:08:58.207 INFO:tasks.workunit.client.1.smithi174.stdout:7/445: creat d0/d4/d10/d1b/d86/f9d x:0 0 0 2024-03-21T10:08:58.212 INFO:tasks.workunit.client.1.smithi174.stdout:2/516: mknod d5/caf 0 2024-03-21T10:08:58.214 INFO:tasks.workunit.client.0.smithi153.stdout:7/577: dwrite d4/d2a/d40/d97/db9/fbf [0,4194304] 0 2024-03-21T10:08:58.215 INFO:tasks.workunit.client.1.smithi174.stdout:3/502: link d0/d12/d14/d1a/d27/d3c/l58 d0/d12/d14/d67/da1/lad 0 2024-03-21T10:08:58.221 INFO:tasks.workunit.client.0.smithi153.stdout:4/703: dwrite d11/d2d/f36 [0,4194304] 0 2024-03-21T10:08:58.223 INFO:tasks.workunit.client.0.smithi153.stdout:7/578: chown d4/d11/f61 60561647 1 2024-03-21T10:08:58.226 INFO:tasks.workunit.client.1.smithi174.stdout:7/446: mkdir d0/d9e 0 2024-03-21T10:08:58.228 INFO:tasks.workunit.client.0.smithi153.stdout:3/713: getdents dd/d1e/d2f/d35/d68 0 2024-03-21T10:08:58.231 INFO:tasks.workunit.client.1.smithi174.stdout:5/568: dwrite d1/d16/d2e/d8d/d99/faf [4194304,4194304] 0 2024-03-21T10:08:58.233 INFO:tasks.workunit.client.1.smithi174.stdout:3/503: fdatasync d0/d12/d14/d1a/d5c/f8e 0 2024-03-21T10:08:58.235 INFO:tasks.workunit.client.1.smithi174.stdout:6/398: dread d0/d3/d5/df/f22 [0,4194304] 0 2024-03-21T10:08:58.235 INFO:tasks.workunit.client.1.smithi174.stdout:8/444: dwrite d9/d2a/d39/d76/f87 [0,4194304] 0 2024-03-21T10:08:58.236 INFO:tasks.workunit.client.1.smithi174.stdout:9/539: creat d2/d11/d7d/fbd x:0 0 0 2024-03-21T10:08:58.240 INFO:tasks.workunit.client.1.smithi174.stdout:4/546: dwrite d4/d19/f58 [0,4194304] 0 2024-03-21T10:08:58.243 INFO:tasks.workunit.client.0.smithi153.stdout:7/579: symlink d4/d2a/d3b/d60/lc2 0 2024-03-21T10:08:58.244 INFO:tasks.workunit.client.0.smithi153.stdout:2/750: dwrite dd/d15/d18/d3a/d3d/f74 [0,4194304] 0 2024-03-21T10:08:58.245 INFO:tasks.workunit.client.1.smithi174.stdout:8/445: fsync d9/d2a/d39/d76/f8d 0 2024-03-21T10:08:58.247 INFO:tasks.workunit.client.1.smithi174.stdout:3/504: write d0/d12/d14/d1a/d44/d57/d93/d7d/f8b [1024460,113783] 0 2024-03-21T10:08:58.248 INFO:tasks.workunit.client.1.smithi174.stdout:5/569: mknod d1/d16/d2e/d66/cd2 0 2024-03-21T10:08:58.250 INFO:tasks.workunit.client.1.smithi174.stdout:4/547: write d4/d21/d31/d6a/f6e [918031,123263] 0 2024-03-21T10:08:58.250 INFO:tasks.workunit.client.1.smithi174.stdout:7/447: truncate d0/d4/f42 614035 0 2024-03-21T10:08:58.250 INFO:tasks.workunit.client.1.smithi174.stdout:6/399: mknod d0/d3/d2a/c7c 0 2024-03-21T10:08:58.258 INFO:tasks.workunit.client.1.smithi174.stdout:3/505: fdatasync d0/f33 0 2024-03-21T10:08:58.259 INFO:tasks.workunit.client.1.smithi174.stdout:8/446: chown d9/d2a/d39/d76/d19/d31/f8f 0 1 2024-03-21T10:08:58.262 INFO:tasks.workunit.client.0.smithi153.stdout:4/704: creat d11/d18/fdb x:0 0 0 2024-03-21T10:08:58.262 INFO:tasks.workunit.client.1.smithi174.stdout:0/503: dread d9/d1c/f1e [0,4194304] 0 2024-03-21T10:08:58.262 INFO:tasks.workunit.client.1.smithi174.stdout:5/570: write d1/d16/d1c/f23 [2719833,55234] 0 2024-03-21T10:08:58.263 INFO:tasks.workunit.client.1.smithi174.stdout:9/540: fsync d2/d24/d2e/f31 0 2024-03-21T10:08:58.265 INFO:tasks.workunit.client.0.smithi153.stdout:5/671: rmdir d4/d5 39 2024-03-21T10:08:58.266 INFO:tasks.workunit.client.1.smithi174.stdout:3/506: chown d0/d12/f17 578 1 2024-03-21T10:08:58.271 INFO:tasks.workunit.client.1.smithi174.stdout:5/571: symlink d1/d5c/d96/ld3 0 2024-03-21T10:08:58.272 INFO:tasks.workunit.client.0.smithi153.stdout:9/700: dwrite d1/d43/d44/f3e [0,4194304] 0 2024-03-21T10:08:58.273 INFO:tasks.workunit.client.1.smithi174.stdout:7/448: chown d0/d4/d10/d44/d81/f87 4609003 1 2024-03-21T10:08:58.274 INFO:tasks.workunit.client.1.smithi174.stdout:2/517: dwrite d5/d25/d27/d2d/d45/d5b/d63/f48 [0,4194304] 0 2024-03-21T10:08:58.274 INFO:tasks.workunit.client.1.smithi174.stdout:9/541: symlink d2/d24/d2e/dac/lbe 0 2024-03-21T10:08:58.274 INFO:tasks.workunit.client.1.smithi174.stdout:4/548: mkdir d4/d21/d31/d3a/d67/d76/d82/db1 0 2024-03-21T10:08:58.277 INFO:tasks.workunit.client.1.smithi174.stdout:3/507: symlink d0/d12/d46/d9b/lae 0 2024-03-21T10:08:58.279 INFO:tasks.workunit.client.1.smithi174.stdout:9/542: chown d2/d11/f90 5029 1 2024-03-21T10:08:58.281 INFO:tasks.workunit.client.1.smithi174.stdout:5/572: rmdir d1/d49/d4f/d51/d8f 39 2024-03-21T10:08:58.281 INFO:tasks.workunit.client.0.smithi153.stdout:4/705: symlink d11/d2d/d6b/d9c/ldc 0 2024-03-21T10:08:58.282 INFO:tasks.workunit.client.1.smithi174.stdout:2/518: write d5/d25/d27/d2d/d45/d5b/d55/f98 [3343053,2422] 0 2024-03-21T10:08:58.283 INFO:tasks.workunit.client.1.smithi174.stdout:3/508: chown d0/d12/d46/d9b 81058387 1 2024-03-21T10:08:58.285 INFO:tasks.workunit.client.0.smithi153.stdout:0/620: dwrite d4/d16/d27/f5c [0,4194304] 0 2024-03-21T10:08:58.288 INFO:tasks.workunit.client.0.smithi153.stdout:1/550: dwrite d6/d3a/d3c/d50/d51/f6d [0,4194304] 0 2024-03-21T10:08:58.292 INFO:tasks.workunit.client.1.smithi174.stdout:0/504: dread d9/d21/f37 [0,4194304] 0 2024-03-21T10:08:58.293 INFO:tasks.workunit.client.0.smithi153.stdout:0/621: rmdir d4/d16/d19/d95 39 2024-03-21T10:08:58.293 INFO:tasks.workunit.client.1.smithi174.stdout:5/573: truncate d1/d2f/d37/f97 947484 0 2024-03-21T10:08:58.294 INFO:tasks.workunit.client.0.smithi153.stdout:7/580: link d4/d11/d56/c67 d4/d1b/d20/cc3 0 2024-03-21T10:08:58.294 INFO:tasks.workunit.client.1.smithi174.stdout:3/509: fdatasync d0/d12/d14/d1a/f3d 0 2024-03-21T10:08:58.295 INFO:tasks.workunit.client.1.smithi174.stdout:3/510: chown d0 3 1 2024-03-21T10:08:58.296 INFO:tasks.workunit.client.0.smithi153.stdout:2/751: dwrite dd/d15/d18/d2c/d4b/d56/fc9 [0,4194304] 0 2024-03-21T10:08:58.297 INFO:tasks.workunit.client.0.smithi153.stdout:0/622: chown d4/d4c/d83/d22/f73 6920806 1 2024-03-21T10:08:58.303 INFO:tasks.workunit.client.0.smithi153.stdout:9/701: dwrite d1/d43/d44/d46/d9e/f9f [0,4194304] 0 2024-03-21T10:08:58.304 INFO:tasks.workunit.client.1.smithi174.stdout:7/449: getdents d0/d4/d10/d44/d81 0 2024-03-21T10:08:58.306 INFO:tasks.workunit.client.0.smithi153.stdout:0/623: creat d4/d16/d19/fc3 x:0 0 0 2024-03-21T10:08:58.307 INFO:tasks.workunit.client.0.smithi153.stdout:7/581: truncate d4/d14/d2f/fbe 869444 0 2024-03-21T10:08:58.310 INFO:tasks.workunit.client.0.smithi153.stdout:0/624: chown d4/f28 7537 1 2024-03-21T10:08:58.313 INFO:tasks.workunit.client.1.smithi174.stdout:0/505: getdents d9 0 2024-03-21T10:08:58.313 INFO:tasks.workunit.client.1.smithi174.stdout:3/511: read d0/d12/d14/fa5 [203612,88248] 0 2024-03-21T10:08:58.328 INFO:tasks.workunit.client.1.smithi174.stdout:4/549: dwrite d4/d19/f98 [0,4194304] 0 2024-03-21T10:08:58.329 INFO:tasks.workunit.client.1.smithi174.stdout:2/519: dwrite d5/d25/fa4 [0,4194304] 0 2024-03-21T10:08:58.335 INFO:tasks.workunit.client.1.smithi174.stdout:0/506: rename d9/d35/d4a/d67/d89/d9a to d9/d35/d4a/d67/d89/d99/db2 0 2024-03-21T10:08:58.336 INFO:tasks.workunit.client.1.smithi174.stdout:2/520: write d5/d25/d27/f8b [4092974,908] 0 2024-03-21T10:08:58.342 INFO:tasks.workunit.client.1.smithi174.stdout:5/574: dwrite d1/d49/d93/d9c/fce [0,4194304] 0 2024-03-21T10:08:58.346 INFO:tasks.workunit.client.0.smithi153.stdout:2/752: dwrite dd/d15/d18/d2c/d4b/d56/d63/f8b [0,4194304] 0 2024-03-21T10:08:58.347 INFO:tasks.workunit.client.1.smithi174.stdout:7/450: dwrite d0/d4/f33 [4194304,4194304] 0 2024-03-21T10:08:58.347 INFO:tasks.workunit.client.1.smithi174.stdout:2/521: write d5/d25/d27/d2e/d7b/f99 [124456,116965] 0 2024-03-21T10:08:58.347 INFO:tasks.workunit.client.1.smithi174.stdout:0/507: fsync d9/f1d 0 2024-03-21T10:08:58.347 INFO:tasks.workunit.client.1.smithi174.stdout:1/533: sync 2024-03-21T10:08:58.347 INFO:tasks.workunit.client.1.smithi174.stdout:4/550: creat d4/d21/d31/d6a/d8f/d9c/fb2 x:0 0 0 2024-03-21T10:08:58.348 INFO:tasks.workunit.client.0.smithi153.stdout:2/753: rmdir dd/d15/d18 39 2024-03-21T10:08:58.352 INFO:tasks.workunit.client.1.smithi174.stdout:2/522: fdatasync d5/d25/d27/d2d/d45/d5b/d2f/d4b/d76/f86 0 2024-03-21T10:08:58.352 INFO:tasks.workunit.client.1.smithi174.stdout:1/534: write d4/d7/d15/f9b [816146,60067] 0 2024-03-21T10:08:58.357 INFO:tasks.workunit.client.1.smithi174.stdout:5/575: rmdir d1/d2f/d37 39 2024-03-21T10:08:58.361 INFO:tasks.workunit.client.1.smithi174.stdout:2/523: chown d5 36515003 1 2024-03-21T10:08:58.364 INFO:tasks.workunit.client.1.smithi174.stdout:9/543: truncate d2/d24/f2f 654621 0 2024-03-21T10:08:58.366 INFO:tasks.workunit.client.0.smithi153.stdout:2/754: creat dd/d15/d18/d2c/d4b/d56/d63/d8d/d9b/dc3/df5/dae/f103 x:0 0 0 2024-03-21T10:08:58.368 INFO:tasks.workunit.client.1.smithi174.stdout:9/544: write d2/d24/f74 [1532598,102979] 0 2024-03-21T10:08:58.372 INFO:tasks.workunit.client.0.smithi153.stdout:2/755: symlink dd/d15/d3c/l104 0 2024-03-21T10:08:58.374 INFO:tasks.workunit.client.1.smithi174.stdout:4/551: dread d4/d21/d24/d28/d38/f88 [0,4194304] 0 2024-03-21T10:08:58.375 INFO:tasks.workunit.client.1.smithi174.stdout:0/508: symlink d9/d21/d81/lb3 0 2024-03-21T10:08:58.376 INFO:tasks.workunit.client.0.smithi153.stdout:3/714: truncate dd/d1e/d2f/f3b 4112147 0 2024-03-21T10:08:58.377 INFO:tasks.workunit.client.1.smithi174.stdout:7/451: dwrite d0/d53/f80 [0,4194304] 0 2024-03-21T10:08:58.380 INFO:tasks.workunit.client.0.smithi153.stdout:3/715: chown dd/d1e/ff0 22 1 2024-03-21T10:08:58.382 INFO:tasks.workunit.client.0.smithi153.stdout:2/756: rename dd/d15/d18/d2c/d4b/d56/d63/d8d/d9b/dc3/df5/dae/d50/d59/fc5 to dd/d15/d5e/db3/f105 0 2024-03-21T10:08:58.383 INFO:tasks.workunit.client.1.smithi174.stdout:2/524: creat d5/d25/d27/d2e/d85/dad/fb0 x:0 0 0 2024-03-21T10:08:58.383 INFO:tasks.workunit.client.1.smithi174.stdout:0/509: symlink d9/d21/d31/lb4 0 2024-03-21T10:08:58.384 INFO:tasks.workunit.client.0.smithi153.stdout:3/716: creat dd/d41/d80/dbb/def/ff3 x:0 0 0 2024-03-21T10:08:58.385 INFO:tasks.workunit.client.1.smithi174.stdout:4/552: symlink d4/d21/d24/d28/d79/lb3 0 2024-03-21T10:08:58.387 INFO:tasks.workunit.client.1.smithi174.stdout:5/576: creat d1/fd4 x:0 0 0 2024-03-21T10:08:58.388 INFO:tasks.workunit.client.0.smithi153.stdout:2/757: write dd/d15/d18/d2c/d4b/d56/f7b [394401,102056] 0 2024-03-21T10:08:58.392 INFO:tasks.workunit.client.1.smithi174.stdout:0/510: chown d9/d1c/f7d 28716525 1 2024-03-21T10:08:58.394 INFO:tasks.workunit.client.0.smithi153.stdout:7/582: dread d4/d2a/d3b/d81/f90 [0,4194304] 0 2024-03-21T10:08:58.396 INFO:tasks.workunit.client.0.smithi153.stdout:9/702: truncate d1/d43/d44/d46/d9e/d2e/d67/fd5 135407 0 2024-03-21T10:08:58.397 INFO:tasks.workunit.client.1.smithi174.stdout:0/511: chown d9/d21/d23/da5/f6f 9 1 2024-03-21T10:08:58.397 INFO:tasks.workunit.client.1.smithi174.stdout:2/525: dread d5/d25/d27/d2d/d45/f6e [0,4194304] 0 2024-03-21T10:08:58.400 INFO:tasks.workunit.client.0.smithi153.stdout:2/758: truncate dd/d15/d18/d3a/d3d/f53 2860337 0 2024-03-21T10:08:58.403 INFO:tasks.workunit.client.0.smithi153.stdout:7/583: symlink d4/d11/d56/lc4 0 2024-03-21T10:08:58.405 INFO:tasks.workunit.client.0.smithi153.stdout:2/759: fsync dd/d15/d18/d3a/d8c/fb2 0 2024-03-21T10:08:58.408 INFO:tasks.workunit.client.1.smithi174.stdout:5/577: creat d1/d5c/d96/fd5 x:0 0 0 2024-03-21T10:08:58.410 INFO:tasks.workunit.client.0.smithi153.stdout:0/625: dread d4/d16/d19/d30/d6c/d98/f80 [0,4194304] 0 2024-03-21T10:08:58.411 INFO:tasks.workunit.client.1.smithi174.stdout:8/447: dread d9/d53/f6c [0,4194304] 0 2024-03-21T10:08:58.411 INFO:tasks.workunit.client.0.smithi153.stdout:3/717: dwrite dd/d1e/d2f/d35/d68/f9a [0,4194304] 0 2024-03-21T10:08:58.412 INFO:tasks.workunit.client.1.smithi174.stdout:7/452: dwrite d0/d53/f63 [0,4194304] 0 2024-03-21T10:08:58.413 INFO:tasks.workunit.client.0.smithi153.stdout:2/760: truncate dd/d15/d18/d2c/dd7/ff8 810866 0 2024-03-21T10:08:58.413 INFO:tasks.workunit.client.1.smithi174.stdout:0/512: mknod d9/d35/d4a/d8a/cb5 0 2024-03-21T10:08:58.413 INFO:tasks.workunit.client.0.smithi153.stdout:0/626: write d4/d16/d9f/fbb [650189,73514] 0 2024-03-21T10:08:58.416 INFO:tasks.workunit.client.1.smithi174.stdout:4/553: rename d4/d21/l7d to d4/d19/lb4 0 2024-03-21T10:08:58.419 INFO:tasks.workunit.client.0.smithi153.stdout:9/703: dread d1/d43/d44/f3e [0,4194304] 0 2024-03-21T10:08:58.425 INFO:tasks.workunit.client.1.smithi174.stdout:0/513: chown d9/d21/d31/d3c/f83 74 1 2024-03-21T10:08:58.428 INFO:tasks.workunit.client.1.smithi174.stdout:1/535: dread d4/d5/f18 [0,4194304] 0 2024-03-21T10:08:58.429 INFO:tasks.workunit.client.1.smithi174.stdout:7/453: rmdir d0/d4/d10/d44/d81 39 2024-03-21T10:08:58.431 INFO:tasks.workunit.client.1.smithi174.stdout:2/526: mknod d5/d25/d27/d2e/d4f/d65/d77/cb1 0 2024-03-21T10:08:58.431 INFO:tasks.workunit.client.0.smithi153.stdout:3/718: mkdir dd/d1e/d2f/d35/d68/d75/ddd/df4 0 2024-03-21T10:08:58.432 INFO:tasks.workunit.client.0.smithi153.stdout:2/761: rmdir dd/d15/d18/d2c/d4b/d56/d63/d8d/d9b/dc3 39 2024-03-21T10:08:58.434 INFO:tasks.workunit.client.1.smithi174.stdout:2/527: chown d5/d25/d27/f4e 56423 1 2024-03-21T10:08:58.437 INFO:tasks.workunit.client.0.smithi153.stdout:3/719: rmdir dd/d41/d80/dbb 39 2024-03-21T10:08:58.438 INFO:tasks.workunit.client.0.smithi153.stdout:1/551: dread d6/d10/d2c/f33 [0,4194304] 0 2024-03-21T10:08:58.441 INFO:tasks.workunit.client.1.smithi174.stdout:1/536: creat d4/d7/d68/d8e/db8/fbb x:0 0 0 2024-03-21T10:08:58.443 INFO:tasks.workunit.client.0.smithi153.stdout:2/762: fsync dd/d15/d18/d3a/f55 0 2024-03-21T10:08:58.444 INFO:tasks.workunit.client.1.smithi174.stdout:4/554: creat d4/d21/d24/fb5 x:0 0 0 2024-03-21T10:08:58.444 INFO:tasks.workunit.client.0.smithi153.stdout:0/627: dwrite d4/d4c/d83/d22/d61/faa [0,4194304] 0 2024-03-21T10:08:58.445 INFO:tasks.workunit.client.1.smithi174.stdout:4/555: write d4/d19/f65 [238793,82460] 0 2024-03-21T10:08:58.446 INFO:tasks.workunit.client.1.smithi174.stdout:7/454: dread d0/d4/d10/d79/f83 [0,4194304] 0 2024-03-21T10:08:58.446 INFO:tasks.workunit.client.0.smithi153.stdout:0/628: chown d4/d4c/d83 70179952 1 2024-03-21T10:08:58.447 INFO:tasks.workunit.client.1.smithi174.stdout:2/528: creat d5/d25/d27/d2d/d45/d5b/d2f/d66/fb2 x:0 0 0 2024-03-21T10:08:58.448 INFO:tasks.workunit.client.1.smithi174.stdout:1/537: mknod d4/d7/d33/cbc 0 2024-03-21T10:08:58.449 INFO:tasks.workunit.client.0.smithi153.stdout:0/629: stat d4/d16/d19/d30/d6c/d98/f80 0 2024-03-21T10:08:58.453 INFO:tasks.workunit.client.1.smithi174.stdout:5/578: dwrite d1/f6 [0,4194304] 0 2024-03-21T10:08:58.453 INFO:tasks.workunit.client.1.smithi174.stdout:7/455: creat d0/d4/d10/d79/f9f x:0 0 0 2024-03-21T10:08:58.455 INFO:tasks.workunit.client.1.smithi174.stdout:4/556: chown d4/d21/d31/d6a/dab/ca1 15078 1 2024-03-21T10:08:58.455 INFO:tasks.workunit.client.1.smithi174.stdout:0/514: dwrite d9/d35/f3e [0,4194304] 0 2024-03-21T10:08:58.458 INFO:tasks.workunit.client.1.smithi174.stdout:5/579: chown d1/d16/d1c/f23 4848991 1 2024-03-21T10:08:58.459 INFO:tasks.workunit.client.0.smithi153.stdout:3/720: creat dd/d1e/d3d/d94/d73/ff5 x:0 0 0 2024-03-21T10:08:58.460 INFO:tasks.workunit.client.1.smithi174.stdout:0/515: truncate d9/f13 5015682 0 2024-03-21T10:08:58.460 INFO:tasks.workunit.client.1.smithi174.stdout:1/538: truncate d4/d7/d15/d53/f6a 1821978 0 2024-03-21T10:08:58.463 INFO:tasks.workunit.client.0.smithi153.stdout:0/630: chown d4/d16/c68 801999 1 2024-03-21T10:08:58.465 INFO:tasks.workunit.client.1.smithi174.stdout:1/539: write d4/d7/d33/d43/d51/f64 [4641041,114388] 0 2024-03-21T10:08:58.466 INFO:tasks.workunit.client.0.smithi153.stdout:0/631: stat d4/d16/d19/dad/lb3 0 2024-03-21T10:08:58.467 INFO:tasks.workunit.client.1.smithi174.stdout:4/557: symlink d4/d21/d31/d6a/dab/da8/lb6 0 2024-03-21T10:08:58.468 INFO:tasks.workunit.client.1.smithi174.stdout:4/558: read d4/f2c [2852801,12977] 0 2024-03-21T10:08:58.470 INFO:tasks.workunit.client.0.smithi153.stdout:3/721: rename dd/d1e/d2f/f3a to dd/d1e/d2f/d35/d68/ff6 0 2024-03-21T10:08:58.471 INFO:tasks.workunit.client.0.smithi153.stdout:2/763: creat dd/d15/d3c/f106 x:0 0 0 2024-03-21T10:08:58.473 INFO:tasks.workunit.client.0.smithi153.stdout:7/584: dread d4/d14/d2f/d78/f58 [0,4194304] 0 2024-03-21T10:08:58.474 INFO:tasks.workunit.client.0.smithi153.stdout:2/764: mkdir dd/d15/d18/d2c/d33/d98/dc7/d107 0 2024-03-21T10:08:58.475 INFO:tasks.workunit.client.0.smithi153.stdout:7/585: stat d4/d11/d56/db1/cb5 0 2024-03-21T10:08:58.479 INFO:tasks.workunit.client.0.smithi153.stdout:7/586: creat d4/d19/fc5 x:0 0 0 2024-03-21T10:08:58.480 INFO:tasks.workunit.client.0.smithi153.stdout:2/765: read dd/d15/d18/d2c/f39 [1155239,99134] 0 2024-03-21T10:08:58.481 INFO:tasks.workunit.client.1.smithi174.stdout:5/580: rename d1/d49/d4f/d51/d8f/cb3 to d1/d49/d93/d9c/cd6 0 2024-03-21T10:08:58.484 INFO:tasks.workunit.client.0.smithi153.stdout:2/766: creat dd/d15/d18/d2c/dd7/f108 x:0 0 0 2024-03-21T10:08:58.491 INFO:tasks.workunit.client.1.smithi174.stdout:5/581: unlink d1/d16/d1c/f23 0 2024-03-21T10:08:58.492 INFO:tasks.workunit.client.0.smithi153.stdout:2/767: getdents dd/d15/d3c 0 2024-03-21T10:08:58.493 INFO:tasks.workunit.client.1.smithi174.stdout:4/559: dwrite d4/d21/d31/d3a/f8e [0,4194304] 0 2024-03-21T10:08:58.494 INFO:tasks.workunit.client.0.smithi153.stdout:9/704: dread d1/d43/d44/d46/f37 [0,4194304] 0 2024-03-21T10:08:58.496 INFO:tasks.workunit.client.1.smithi174.stdout:4/560: fdatasync d4/d19/f97 0 2024-03-21T10:08:58.501 INFO:tasks.workunit.client.0.smithi153.stdout:3/722: dread dd/d1e/d3d/f69 [0,4194304] 0 2024-03-21T10:08:58.505 INFO:tasks.workunit.client.0.smithi153.stdout:3/723: write dd/f27 [2496073,105351] 0 2024-03-21T10:08:58.506 INFO:tasks.workunit.client.0.smithi153.stdout:2/768: creat dd/d15/d18/d2c/d4b/d56/d63/d8d/d9b/dc3/df5/dae/f109 x:0 0 0 2024-03-21T10:08:58.506 INFO:tasks.workunit.client.1.smithi174.stdout:0/516: dread d9/d21/d31/f42 [0,4194304] 0 2024-03-21T10:08:58.507 INFO:tasks.workunit.client.1.smithi174.stdout:5/582: rename d1/f91 to d1/d2f/fd7 0 2024-03-21T10:08:58.509 INFO:tasks.workunit.client.1.smithi174.stdout:1/540: truncate d4/d7/d68/d7b/f8c 3135421 0 2024-03-21T10:08:58.511 INFO:tasks.workunit.client.1.smithi174.stdout:7/456: dwrite d0/d4/d10/d1b/f4e [0,4194304] 0 2024-03-21T10:08:58.512 INFO:tasks.workunit.client.1.smithi174.stdout:0/517: fdatasync d9/d1c/f7d 0 2024-03-21T10:08:58.513 INFO:tasks.workunit.client.0.smithi153.stdout:9/705: dwrite d1/d43/d44/d46/d9e/d61/f71 [4194304,4194304] 0 2024-03-21T10:08:58.521 INFO:tasks.workunit.client.1.smithi174.stdout:5/583: mknod d1/d16/d2e/d66/d8b/cd8 0 2024-03-21T10:08:58.521 INFO:tasks.workunit.client.0.smithi153.stdout:7/587: dread d4/d19/fa8 [0,4194304] 0 2024-03-21T10:08:58.523 INFO:tasks.workunit.client.1.smithi174.stdout:4/561: dread d4/d21/d31/d3a/f8e [0,4194304] 0 2024-03-21T10:08:58.525 INFO:tasks.workunit.client.0.smithi153.stdout:7/588: read d4/f25 [1309846,31194] 0 2024-03-21T10:08:58.529 INFO:tasks.workunit.client.0.smithi153.stdout:7/589: mkdir d4/d2a/dc6 0 2024-03-21T10:08:58.530 INFO:tasks.workunit.client.1.smithi174.stdout:1/541: creat d4/d7/d33/d43/d77/d87/d92/fbd x:0 0 0 2024-03-21T10:08:58.531 INFO:tasks.workunit.client.0.smithi153.stdout:7/590: write d4/d19/fc1 [71715,53767] 0 2024-03-21T10:08:58.531 INFO:tasks.workunit.client.0.smithi153.stdout:9/706: dwrite d1/d43/d44/d46/d9e/d61/f71 [0,4194304] 0 2024-03-21T10:08:58.534 INFO:tasks.workunit.client.1.smithi174.stdout:4/562: unlink d4/d21/d24/d28/d38/la5 0 2024-03-21T10:08:58.537 INFO:tasks.workunit.client.0.smithi153.stdout:3/724: dwrite dd/d1e/d2f/d35/d68/d75/ddd/fe5 [0,4194304] 0 2024-03-21T10:08:58.539 INFO:tasks.workunit.client.0.smithi153.stdout:9/707: rmdir d1/d43/d44/d46/d9e/d24/d41 39 2024-03-21T10:08:58.540 INFO:tasks.workunit.client.1.smithi174.stdout:0/518: write d9/d35/d4a/f6a [467446,108475] 0 2024-03-21T10:08:58.545 INFO:tasks.workunit.client.0.smithi153.stdout:3/725: chown dd/d1e/d2f/d35/dd0/fdc 2808168 1 2024-03-21T10:08:58.547 INFO:tasks.workunit.client.0.smithi153.stdout:9/708: mknod d1/d4e/ced 0 2024-03-21T10:08:58.548 INFO:tasks.workunit.client.0.smithi153.stdout:7/591: link d4/d2a/d40/l82 d4/d11/da1/da6/lc7 0 2024-03-21T10:08:58.548 INFO:tasks.workunit.client.1.smithi174.stdout:1/542: dwrite d4/d7/f8f [0,4194304] 0 2024-03-21T10:08:58.549 INFO:tasks.workunit.client.0.smithi153.stdout:3/726: creat dd/d1e/d3d/ff7 x:0 0 0 2024-03-21T10:08:58.549 INFO:tasks.workunit.client.0.smithi153.stdout:9/709: chown d1/d3b/fc2 820458786 1 2024-03-21T10:08:58.552 INFO:tasks.workunit.client.1.smithi174.stdout:1/543: stat d4/d2d/l93 0 2024-03-21T10:08:58.553 INFO:tasks.workunit.client.0.smithi153.stdout:3/727: mknod dd/d41/d80/dbb/dbf/cf8 0 2024-03-21T10:08:58.556 INFO:tasks.workunit.client.1.smithi174.stdout:1/544: chown d4/d7/d15/d4f/f61 1063350048 1 2024-03-21T10:08:58.564 INFO:tasks.workunit.client.0.smithi153.stdout:6/866: sync 2024-03-21T10:08:58.566 INFO:tasks.workunit.client.1.smithi174.stdout:5/584: write d1/d5c/d73/f7f [496835,32604] 0 2024-03-21T10:08:58.569 INFO:tasks.workunit.client.1.smithi174.stdout:4/563: write d4/f3e [647301,46944] 0 2024-03-21T10:08:58.570 INFO:tasks.workunit.client.1.smithi174.stdout:0/519: dwrite d9/d21/d23/f91 [0,4194304] 0 2024-03-21T10:08:58.572 INFO:tasks.workunit.client.1.smithi174.stdout:4/564: readlink d4/lb 0 2024-03-21T10:08:58.572 INFO:tasks.workunit.client.0.smithi153.stdout:9/710: rename d1/d43/d44/c3a to d1/cee 0 2024-03-21T10:08:58.573 INFO:tasks.workunit.client.0.smithi153.stdout:7/592: truncate d4/d1b/d20/f2c 883475 0 2024-03-21T10:08:58.574 INFO:tasks.workunit.client.0.smithi153.stdout:6/867: creat d1f/de0/de1/df2/d10f/f137 x:0 0 0 2024-03-21T10:08:58.576 INFO:tasks.workunit.client.1.smithi174.stdout:4/565: mknod d4/d21/d31/cb7 0 2024-03-21T10:08:58.576 INFO:tasks.workunit.client.0.smithi153.stdout:7/593: mknod d4/d11/cc8 0 2024-03-21T10:08:58.579 INFO:tasks.workunit.client.1.smithi174.stdout:1/545: dwrite d4/d7/d28/f30 [4194304,4194304] 0 2024-03-21T10:08:58.579 INFO:tasks.workunit.client.0.smithi153.stdout:9/711: rename d1/d3b/l40 to d1/d43/d44/d46/d4d/lef 0 2024-03-21T10:08:58.582 INFO:tasks.workunit.client.0.smithi153.stdout:7/594: fdatasync d4/d2a/d3b/d81/f90 0 2024-03-21T10:08:58.583 INFO:tasks.workunit.client.1.smithi174.stdout:4/566: fsync d4/d21/f5b 0 2024-03-21T10:08:58.586 INFO:tasks.workunit.client.0.smithi153.stdout:9/712: creat d1/ff0 x:0 0 0 2024-03-21T10:08:58.587 INFO:tasks.workunit.client.0.smithi153.stdout:7/595: mknod d4/d19/d4d/cc9 0 2024-03-21T10:08:58.589 INFO:tasks.workunit.client.0.smithi153.stdout:9/713: write d1/d4/d5f/fc8 [836615,107303] 0 2024-03-21T10:08:58.590 INFO:tasks.workunit.client.0.smithi153.stdout:3/728: truncate dd/f27 1611298 0 2024-03-21T10:08:58.592 INFO:tasks.workunit.client.1.smithi174.stdout:1/546: rename d4/d7/d33/d43/d51/f9c to d4/d7/d68/d8e/fbe 0 2024-03-21T10:08:58.594 INFO:tasks.workunit.client.1.smithi174.stdout:0/520: dwrite d9/d1c/f7d [4194304,4194304] 0 2024-03-21T10:08:58.594 INFO:tasks.workunit.client.1.smithi174.stdout:1/547: stat d4/d7/d33/d43/c40 0 2024-03-21T10:08:58.597 INFO:tasks.workunit.client.1.smithi174.stdout:0/521: chown d9/d1c/c4e 669040 1 2024-03-21T10:08:58.598 INFO:tasks.workunit.client.0.smithi153.stdout:3/729: rename dd/d41/f54 to dd/d1e/d2f/d35/dd0/ff9 0 2024-03-21T10:08:58.599 INFO:tasks.workunit.client.1.smithi174.stdout:1/548: truncate d4/d5/d6c/fa7 781296 0 2024-03-21T10:08:58.599 INFO:tasks.workunit.client.1.smithi174.stdout:5/585: truncate d1/d16/d2e/d66/d8b/f36 4193426 0 2024-03-21T10:08:58.600 INFO:tasks.workunit.client.0.smithi153.stdout:9/714: dread d1/d43/d44/d46/d9e/d2e/d67/fd5 [0,4194304] 0 2024-03-21T10:08:58.600 INFO:tasks.workunit.client.0.smithi153.stdout:7/596: getdents d4/d11 0 2024-03-21T10:08:58.602 INFO:tasks.workunit.client.1.smithi174.stdout:4/567: creat d4/d21/d31/d6a/fb8 x:0 0 0 2024-03-21T10:08:58.602 INFO:tasks.workunit.client.0.smithi153.stdout:6/868: write d1f/d93/dd8/d59/f81 [977700,12571] 0 2024-03-21T10:08:58.603 INFO:tasks.workunit.client.1.smithi174.stdout:5/586: readlink d1/d5c/l9f 0 2024-03-21T10:08:58.606 INFO:tasks.workunit.client.0.smithi153.stdout:7/597: rename d4/d2a/d40/d50/fae to d4/d2a/d40/d50/fca 0 2024-03-21T10:08:58.607 INFO:tasks.workunit.client.0.smithi153.stdout:6/869: mknod d1f/d99/c138 0 2024-03-21T10:08:58.609 INFO:tasks.workunit.client.1.smithi174.stdout:1/549: mknod d4/d7/d68/cbf 0 2024-03-21T10:08:58.610 INFO:tasks.workunit.client.1.smithi174.stdout:4/568: rename d4/d21/fa9 to d4/d21/d31/d3a/d67/fb9 0 2024-03-21T10:08:58.612 INFO:tasks.workunit.client.0.smithi153.stdout:7/598: unlink d4/d14/d2f/d78/f51 0 2024-03-21T10:08:58.614 INFO:tasks.workunit.client.1.smithi174.stdout:1/550: creat d4/d5/d6c/fc0 x:0 0 0 2024-03-21T10:08:58.616 INFO:tasks.workunit.client.0.smithi153.stdout:9/715: getdents d1/d4e/d68 0 2024-03-21T10:08:58.616 INFO:tasks.workunit.client.0.smithi153.stdout:6/870: dread d1f/d93/dd8/f74 [0,4194304] 0 2024-03-21T10:08:58.620 INFO:tasks.workunit.client.1.smithi174.stdout:1/551: creat d4/d7/d68/d89/fc1 x:0 0 0 2024-03-21T10:08:58.623 INFO:tasks.workunit.client.0.smithi153.stdout:6/871: mknod d1f/d2b/d10a/d40/d115/c139 0 2024-03-21T10:08:58.623 INFO:tasks.workunit.client.1.smithi174.stdout:4/569: write d4/d19/f23 [2247784,58969] 0 2024-03-21T10:08:58.624 INFO:tasks.workunit.client.0.smithi153.stdout:9/716: mknod d1/cf1 0 2024-03-21T10:08:58.627 INFO:tasks.workunit.client.0.smithi153.stdout:9/717: read - d1/d43/d44/d46/d9e/d61/fe5 zero size 2024-03-21T10:08:58.630 INFO:tasks.workunit.client.1.smithi174.stdout:1/552: creat d4/d7/d33/d43/d77/fc2 x:0 0 0 2024-03-21T10:08:58.631 INFO:tasks.workunit.client.0.smithi153.stdout:6/872: truncate d1f/d93/dd8/d59/f8d 1444539 0 2024-03-21T10:08:58.632 INFO:tasks.workunit.client.1.smithi174.stdout:0/522: dwrite d9/d21/d31/d3c/f49 [0,4194304] 0 2024-03-21T10:08:58.634 INFO:tasks.workunit.client.0.smithi153.stdout:9/718: fsync d1/d43/d44/d46/d9e/d24/d41/dbe/dcc/fa4 0 2024-03-21T10:08:58.634 INFO:tasks.workunit.client.0.smithi153.stdout:6/873: fsync d1f/d2b/d10a/d40/d51/ff3 0 2024-03-21T10:08:58.637 INFO:tasks.workunit.client.1.smithi174.stdout:0/523: creat d9/d21/d81/fb6 x:0 0 0 2024-03-21T10:08:58.637 INFO:tasks.workunit.client.0.smithi153.stdout:9/719: fsync d1/d4e/d68/f81 0 2024-03-21T10:08:58.639 INFO:tasks.workunit.client.0.smithi153.stdout:7/599: dwrite d4/d2a/f63 [0,4194304] 0 2024-03-21T10:08:58.639 INFO:tasks.workunit.client.1.smithi174.stdout:1/553: write d4/d5/f22 [2276706,24399] 0 2024-03-21T10:08:58.641 INFO:tasks.workunit.client.0.smithi153.stdout:6/874: creat d1f/d2b/d10a/d40/d5f/dac/f13a x:0 0 0 2024-03-21T10:08:58.642 INFO:tasks.workunit.client.0.smithi153.stdout:3/730: dread dd/d1e/d2f/f37 [0,4194304] 0 2024-03-21T10:08:58.643 INFO:tasks.workunit.client.0.smithi153.stdout:3/731: stat dd/c16 0 2024-03-21T10:08:58.643 INFO:tasks.workunit.client.0.smithi153.stdout:7/600: creat d4/d2a/d40/d50/fcb x:0 0 0 2024-03-21T10:08:58.644 INFO:tasks.workunit.client.1.smithi174.stdout:1/554: chown d4/d7/d68/d8e/db8/fbb 380526 1 2024-03-21T10:08:58.645 INFO:tasks.workunit.client.1.smithi174.stdout:0/524: truncate d9/f54 121949 0 2024-03-21T10:08:58.647 INFO:tasks.workunit.client.1.smithi174.stdout:4/570: dwrite d4/d21/d24/fb5 [0,4194304] 0 2024-03-21T10:08:58.647 INFO:tasks.workunit.client.1.smithi174.stdout:1/555: chown d4/d7/d15/f38 14384 1 2024-03-21T10:08:58.648 INFO:tasks.workunit.client.0.smithi153.stdout:7/601: mknod d4/d14/ccc 0 2024-03-21T10:08:58.650 INFO:tasks.workunit.client.0.smithi153.stdout:3/732: read - dd/d1e/d3d/d94/d8e/fa2 zero size 2024-03-21T10:08:58.652 INFO:tasks.workunit.client.1.smithi174.stdout:1/556: chown d4/d7/d33/d43/c88 13154 1 2024-03-21T10:08:58.652 INFO:tasks.workunit.client.1.smithi174.stdout:0/525: mkdir d9/d21/d31/d5c/d92/d98/db7 0 2024-03-21T10:08:58.653 INFO:tasks.workunit.client.0.smithi153.stdout:9/720: dread d1/d8a/fb9 [0,4194304] 0 2024-03-21T10:08:58.654 INFO:tasks.workunit.client.1.smithi174.stdout:1/557: chown d4/d7/f2c 4100173 1 2024-03-21T10:08:58.657 INFO:tasks.workunit.client.1.smithi174.stdout:1/558: write d4/d7/d15/d53/d78/fab [253951,10299] 0 2024-03-21T10:08:58.661 INFO:tasks.workunit.client.1.smithi174.stdout:0/526: creat d9/d21/d75/da1/fb8 x:0 0 0 2024-03-21T10:08:58.661 INFO:tasks.workunit.client.0.smithi153.stdout:3/733: link dd/c50 dd/d1e/d2f/d35/dd0/cfa 0 2024-03-21T10:08:58.662 INFO:tasks.workunit.client.1.smithi174.stdout:0/527: fdatasync d9/d21/d31/d9f/d79/fa6 0 2024-03-21T10:08:58.671 INFO:tasks.workunit.client.0.smithi153.stdout:9/721: dwrite d1/d43/fab [0,4194304] 0 2024-03-21T10:08:58.676 INFO:tasks.workunit.client.1.smithi174.stdout:0/528: mkdir d9/d21/d31/d9f/d61/db9 0 2024-03-21T10:08:58.677 INFO:tasks.workunit.client.1.smithi174.stdout:1/559: truncate d4/d67/f74 632569 0 2024-03-21T10:08:58.677 INFO:tasks.workunit.client.0.smithi153.stdout:9/722: creat d1/d4/ff2 x:0 0 0 2024-03-21T10:08:58.681 INFO:tasks.workunit.client.0.smithi153.stdout:6/875: write d1f/d2b/d10a/d40/d4f/f77 [500620,33982] 0 2024-03-21T10:08:58.682 INFO:tasks.workunit.client.1.smithi174.stdout:0/529: creat d9/d21/d31/d5c/fba x:0 0 0 2024-03-21T10:08:58.683 INFO:tasks.workunit.client.0.smithi153.stdout:9/723: chown d1/c15 32 1 2024-03-21T10:08:58.685 INFO:tasks.workunit.client.1.smithi174.stdout:1/560: read d4/d5/f8 [497609,65421] 0 2024-03-21T10:08:58.687 INFO:tasks.workunit.client.0.smithi153.stdout:9/724: mkdir d1/d4/d5f/df3 0 2024-03-21T10:08:58.690 INFO:tasks.workunit.client.0.smithi153.stdout:6/876: mknod d1f/d2b/d10a/d40/d7b/d130/c13b 0 2024-03-21T10:08:58.690 INFO:tasks.workunit.client.1.smithi174.stdout:0/530: mkdir d9/d21/d31/d9f/d79/d82/dbb 0 2024-03-21T10:08:58.694 INFO:tasks.workunit.client.0.smithi153.stdout:7/602: dwrite d4/d2a/d3b/d60/f64 [0,4194304] 0 2024-03-21T10:08:58.694 INFO:tasks.workunit.client.1.smithi174.stdout:1/561: read d4/d5/d6c/f6f [67815,15058] 0 2024-03-21T10:08:58.695 INFO:tasks.workunit.client.1.smithi174.stdout:4/571: dwrite d4/d21/d24/d28/d38/f54 [0,4194304] 0 2024-03-21T10:08:58.695 INFO:tasks.workunit.client.1.smithi174.stdout:1/562: chown d4/l11 13923073 1 2024-03-21T10:08:58.696 INFO:tasks.workunit.client.0.smithi153.stdout:9/725: getdents d1/d4e 0 2024-03-21T10:08:58.697 INFO:tasks.workunit.client.1.smithi174.stdout:4/572: read d4/d21/d31/d3a/f8e [4017924,109923] 0 2024-03-21T10:08:58.697 INFO:tasks.workunit.client.0.smithi153.stdout:6/877: mkdir d1f/d93/dd8/d59/d70/d13c 0 2024-03-21T10:08:58.699 INFO:tasks.workunit.client.0.smithi153.stdout:7/603: write d4/d1b/d20/d9a/db7/f96 [965385,66197] 0 2024-03-21T10:08:58.700 INFO:tasks.workunit.client.1.smithi174.stdout:1/563: rmdir d4/d7/d70 39 2024-03-21T10:08:58.700 INFO:tasks.workunit.client.1.smithi174.stdout:3/512: sync 2024-03-21T10:08:58.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:08:58 smithi174 ceph-mon[37496]: pgmap v15: 65 pgs: 65 active+clean; 2.2 GiB data, 6.5 GiB used, 530 GiB / 536 GiB avail; 62 MiB/s rd, 235 MiB/s wr, 417 op/s 2024-03-21T10:08:58.701 INFO:tasks.workunit.client.0.smithi153.stdout:3/734: dread dd/d1e/d2f/f36 [0,4194304] 0 2024-03-21T10:08:58.701 INFO:tasks.workunit.client.0.smithi153.stdout:9/726: chown d1/d43/d44/d46/d9e/d24/d41 1862761 1 2024-03-21T10:08:58.702 INFO:tasks.workunit.client.1.smithi174.stdout:9/545: sync 2024-03-21T10:08:58.702 INFO:tasks.workunit.client.1.smithi174.stdout:6/400: sync 2024-03-21T10:08:58.703 INFO:tasks.workunit.client.0.smithi153.stdout:6/878: mknod d1f/d93/dd8/d59/d70/d110/c13d 0 2024-03-21T10:08:58.706 INFO:tasks.workunit.client.1.smithi174.stdout:3/513: dread - d0/d12/d14/d1a/d5c/f8d zero size 2024-03-21T10:08:58.707 INFO:tasks.workunit.client.0.smithi153.stdout:7/604: dread d4/d2a/d3b/d81/fa9 [0,4194304] 0 2024-03-21T10:08:58.707 INFO:tasks.workunit.client.0.smithi153.stdout:9/727: creat d1/d4/d5f/df3/ff4 x:0 0 0 2024-03-21T10:08:58.708 INFO:tasks.workunit.client.1.smithi174.stdout:1/564: write d4/d2d/f34 [2563580,55046] 0 2024-03-21T10:08:58.709 INFO:tasks.workunit.client.0.smithi153.stdout:7/605: truncate d4/d11/f61 4909241 0 2024-03-21T10:08:58.710 INFO:tasks.workunit.client.1.smithi174.stdout:4/573: truncate d4/d21/d24/d28/d38/f90 4213595 0 2024-03-21T10:08:58.712 INFO:tasks.workunit.client.0.smithi153.stdout:9/728: mknod d1/d43/d44/d46/d4d/d5c/cf5 0 2024-03-21T10:08:58.712 INFO:tasks.workunit.client.1.smithi174.stdout:6/401: truncate d0/d3/d5/df/f4e 845950 0 2024-03-21T10:08:58.713 INFO:tasks.workunit.client.1.smithi174.stdout:1/565: chown d4/d7/d28/f97 742211 1 2024-03-21T10:08:58.715 INFO:tasks.workunit.client.1.smithi174.stdout:9/546: rename d2/d24/d2e/f69 to d2/d11/d1d/d4b/fbf 0 2024-03-21T10:08:58.715 INFO:tasks.workunit.client.1.smithi174.stdout:3/514: read d0/d12/d14/d1a/d44/d57/d93/f38 [3645570,80807] 0 2024-03-21T10:08:58.719 INFO:tasks.workunit.client.1.smithi174.stdout:3/515: chown d0/d12/d14/d1a/d27/f9a 102252008 1 2024-03-21T10:08:58.720 INFO:tasks.workunit.client.1.smithi174.stdout:1/566: rename d4/d7/d68/cbf to d4/d7/d33/d43/d77/cc3 0 2024-03-21T10:08:58.721 INFO:tasks.workunit.client.0.smithi153.stdout:8/664: sync 2024-03-21T10:08:58.721 INFO:tasks.workunit.client.1.smithi174.stdout:9/547: mknod d2/d11/d1d/d2a/d91/cc0 0 2024-03-21T10:08:58.724 INFO:tasks.workunit.client.1.smithi174.stdout:3/516: readlink d0/d12/d14/d1a/d27/d3c/l58 0 2024-03-21T10:08:58.724 INFO:tasks.workunit.client.1.smithi174.stdout:4/574: mknod d4/d21/d31/d3a/d7a/d9d/da3/cba 0 2024-03-21T10:08:58.725 INFO:tasks.workunit.client.1.smithi174.stdout:1/567: mkdir d4/d7/d32/da2/da8/dc4 0 2024-03-21T10:08:58.726 INFO:tasks.workunit.client.0.smithi153.stdout:7/606: write d4/d14/d2f/d78/f6f [233804,9957] 0 2024-03-21T10:08:58.726 INFO:tasks.workunit.client.1.smithi174.stdout:4/575: write d4/f3e [1253600,19949] 0 2024-03-21T10:08:58.730 INFO:tasks.workunit.client.1.smithi174.stdout:3/517: mknod d0/d12/d14/d1a/d27/d3c/d73/caf 0 2024-03-21T10:08:58.731 INFO:tasks.workunit.client.0.smithi153.stdout:7/607: rename d4/d2a/d3b/l9d to d4/d1b/d20/lcd 0 2024-03-21T10:08:58.733 INFO:tasks.workunit.client.0.smithi153.stdout:3/735: dwrite dd/d1e/d3d/d94/d8e/fa2 [0,4194304] 0 2024-03-21T10:08:58.734 INFO:tasks.workunit.client.1.smithi174.stdout:1/568: fdatasync d4/d7/d15/d4f/f69 0 2024-03-21T10:08:58.734 INFO:tasks.workunit.client.0.smithi153.stdout:7/608: truncate d4/d2a/f3e 1672108 0 2024-03-21T10:08:58.735 INFO:tasks.workunit.client.1.smithi174.stdout:9/548: rename d2/d11/d1d/d2a/d91 to d2/d11/d1d/dab/dc1 0 2024-03-21T10:08:58.737 INFO:tasks.workunit.client.0.smithi153.stdout:7/609: write d4/d2a/d40/d50/fcb [677631,18360] 0 2024-03-21T10:08:58.737 INFO:tasks.workunit.client.0.smithi153.stdout:3/736: creat dd/d41/d80/dbb/ffb x:0 0 0 2024-03-21T10:08:58.739 INFO:tasks.workunit.client.1.smithi174.stdout:3/518: creat d0/d12/d14/da9/fb0 x:0 0 0 2024-03-21T10:08:58.739 INFO:tasks.workunit.client.1.smithi174.stdout:9/549: mknod d2/d24/cc2 0 2024-03-21T10:08:58.739 INFO:tasks.workunit.client.0.smithi153.stdout:3/737: dread - dd/d41/d80/dbb/def/ff3 zero size 2024-03-21T10:08:58.741 INFO:tasks.workunit.client.1.smithi174.stdout:4/576: write d4/d19/f2d [680340,105291] 0 2024-03-21T10:08:58.741 INFO:tasks.workunit.client.1.smithi174.stdout:3/519: rmdir d0/d12/d14/d4c 39 2024-03-21T10:08:58.745 INFO:tasks.workunit.client.0.smithi153.stdout:7/610: rename d4/d14/d16/c23 to d4/d2a/d40/cce 0 2024-03-21T10:08:58.745 INFO:tasks.workunit.client.0.smithi153.stdout:3/738: fsync dd/d1e/d2f/d35/daf/dcc/fa5 0 2024-03-21T10:08:58.747 INFO:tasks.workunit.client.0.smithi153.stdout:3/739: stat dd/d1e/f46 0 2024-03-21T10:08:58.749 INFO:tasks.workunit.client.1.smithi174.stdout:9/550: creat d2/d11/d1d/d5e/fc3 x:0 0 0 2024-03-21T10:08:58.749 INFO:tasks.workunit.client.0.smithi153.stdout:7/611: link d4/d1b/l57 d4/d11/lcf 0 2024-03-21T10:08:58.751 INFO:tasks.workunit.client.0.smithi153.stdout:3/740: symlink dd/d1e/d2f/d35/lfc 0 2024-03-21T10:08:58.754 INFO:tasks.workunit.client.1.smithi174.stdout:1/569: dwrite d4/d7/d15/f41 [0,4194304] 0 2024-03-21T10:08:58.754 INFO:tasks.workunit.client.0.smithi153.stdout:3/741: fsync dd/d1e/d3d/f69 0 2024-03-21T10:08:58.758 INFO:tasks.workunit.client.0.smithi153.stdout:7/612: rename d4/d1b to d4/d14/dd0 0 2024-03-21T10:08:58.761 INFO:tasks.workunit.client.1.smithi174.stdout:1/570: mkdir d4/d7/d28/dc5 0 2024-03-21T10:08:58.764 INFO:tasks.workunit.client.1.smithi174.stdout:4/577: write d4/d19/f50 [923425,16055] 0 2024-03-21T10:08:58.765 INFO:tasks.workunit.client.1.smithi174.stdout:3/520: dwrite d0/d12/f49 [0,4194304] 0 2024-03-21T10:08:58.767 INFO:tasks.workunit.client.0.smithi153.stdout:3/742: dwrite dd/d1e/d3d/d53/fba [4194304,4194304] 0 2024-03-21T10:08:58.768 INFO:tasks.workunit.client.1.smithi174.stdout:3/521: write d0/d12/d14/d1a/d27/d3c/f72 [3868365,108003] 0 2024-03-21T10:08:58.769 INFO:tasks.workunit.client.0.smithi153.stdout:7/613: truncate d4/d19/fb6 36322 0 2024-03-21T10:08:58.769 INFO:tasks.workunit.client.0.smithi153.stdout:3/743: dread - dd/d1e/d3d/ff7 zero size 2024-03-21T10:08:58.772 INFO:tasks.workunit.client.0.smithi153.stdout:3/744: stat dd/d1e/d45/f52 0 2024-03-21T10:08:58.774 INFO:tasks.workunit.client.0.smithi153.stdout:3/745: write dd/d1e/d2f/d35/f7b [2312203,126644] 0 2024-03-21T10:08:58.774 INFO:tasks.workunit.client.0.smithi153.stdout:7/614: mknod d4/d14/d2f/cd1 0 2024-03-21T10:08:58.775 INFO:tasks.workunit.client.1.smithi174.stdout:1/571: truncate d4/d7/d15/d53/f6a 199976 0 2024-03-21T10:08:58.776 INFO:tasks.workunit.client.0.smithi153.stdout:3/746: chown dd/d1e/d3d/d94/d73/fa4 3388562 1 2024-03-21T10:08:58.778 INFO:tasks.workunit.client.1.smithi174.stdout:9/551: creat d2/d24/d2e/fc4 x:0 0 0 2024-03-21T10:08:58.779 INFO:tasks.workunit.client.1.smithi174.stdout:4/578: truncate d4/f2c 2109717 0 2024-03-21T10:08:58.779 INFO:tasks.workunit.client.1.smithi174.stdout:3/522: mknod d0/d12/d14/d1a/d27/d7f/cb1 0 2024-03-21T10:08:58.780 INFO:tasks.workunit.client.0.smithi153.stdout:7/615: rename d4/d2a/d40/d97 to d4/d14/dd0/d20/d9a/dd2 0 2024-03-21T10:08:58.781 INFO:tasks.workunit.client.0.smithi153.stdout:3/747: unlink dd/d1e/l59 0 2024-03-21T10:08:58.782 INFO:tasks.workunit.client.1.smithi174.stdout:3/523: dread - d0/d12/d14/d1a/d5c/f8d zero size 2024-03-21T10:08:58.783 INFO:tasks.workunit.client.0.smithi153.stdout:7/616: read d4/d14/dd0/d20/d9a/dd2/db9/fbf [3591946,1998] 0 2024-03-21T10:08:58.783 INFO:tasks.workunit.client.0.smithi153.stdout:3/748: write dd/f25 [8359159,115597] 0 2024-03-21T10:08:58.785 INFO:tasks.workunit.client.0.smithi153.stdout:3/749: stat dd/f13 0 2024-03-21T10:08:58.786 INFO:tasks.workunit.client.1.smithi174.stdout:9/552: rename d2/d11/d1d/dab/dc1/cc0 to d2/d11/d1d/d4b/cc5 0 2024-03-21T10:08:58.788 INFO:tasks.workunit.client.1.smithi174.stdout:3/524: chown d0/d12/d14/f62 933017 1 2024-03-21T10:08:58.788 INFO:tasks.workunit.client.1.smithi174.stdout:9/553: stat d2/d11/d1d/lb0 0 2024-03-21T10:08:58.789 INFO:tasks.workunit.client.1.smithi174.stdout:1/572: getdents d4/d7/d70 0 2024-03-21T10:08:58.789 INFO:tasks.workunit.client.0.smithi153.stdout:7/617: rename d4/d19/d4d/cc9 to d4/d11/cd3 0 2024-03-21T10:08:58.790 INFO:tasks.workunit.client.1.smithi174.stdout:4/579: getdents d4/d21/d24/d28/d79 0 2024-03-21T10:08:58.790 INFO:tasks.workunit.client.0.smithi153.stdout:3/750: truncate dd/d1e/d2f/f8c 316792 0 2024-03-21T10:08:58.790 INFO:tasks.workunit.client.1.smithi174.stdout:9/554: fdatasync d2/d11/d1d/d2a/f73 0 2024-03-21T10:08:58.792 INFO:tasks.workunit.client.1.smithi174.stdout:3/525: creat d0/d12/d46/d59/fb2 x:0 0 0 2024-03-21T10:08:58.793 INFO:tasks.workunit.client.1.smithi174.stdout:9/555: chown d2/d11/c7a 19 1 2024-03-21T10:08:58.794 INFO:tasks.workunit.client.0.smithi153.stdout:3/751: rename dd/d1e/d2f/d35/daf/dcc/l5d to dd/d1e/d3d/d94/de2/lfd 0 2024-03-21T10:08:58.797 INFO:tasks.workunit.client.1.smithi174.stdout:1/573: unlink d4/d7/d28/c49 0 2024-03-21T10:08:58.804 INFO:tasks.workunit.client.1.smithi174.stdout:9/556: mknod d2/cc6 0 2024-03-21T10:08:58.804 INFO:tasks.workunit.client.0.smithi153.stdout:7/618: dwrite d4/d2a/d3b/d60/f7c [0,4194304] 0 2024-03-21T10:08:58.805 INFO:tasks.workunit.client.1.smithi174.stdout:4/580: unlink d4/f2c 0 2024-03-21T10:08:58.805 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:08:58 smithi153 ceph-mon[29117]: pgmap v15: 65 pgs: 65 active+clean; 2.2 GiB data, 6.5 GiB used, 530 GiB / 536 GiB avail; 62 MiB/s rd, 235 MiB/s wr, 417 op/s 2024-03-21T10:08:58.807 INFO:tasks.workunit.client.0.smithi153.stdout:7/619: mknod d4/d11/d56/cd4 0 2024-03-21T10:08:58.807 INFO:tasks.workunit.client.1.smithi174.stdout:9/557: stat d2/d24/d2e/dac/lbe 0 2024-03-21T10:08:58.811 INFO:tasks.workunit.client.1.smithi174.stdout:9/558: chown d2/d11/d1d/d8b/f9a 91513902 1 2024-03-21T10:08:58.811 INFO:tasks.workunit.client.1.smithi174.stdout:4/581: dread d4/d19/f7c [0,4194304] 0 2024-03-21T10:08:58.813 INFO:tasks.workunit.client.1.smithi174.stdout:3/526: dwrite d0/d12/d14/f62 [0,4194304] 0 2024-03-21T10:08:58.813 INFO:tasks.workunit.client.0.smithi153.stdout:7/620: unlink d4/d14/dd0/d20/d9a/db7/l8b 0 2024-03-21T10:08:58.816 INFO:tasks.workunit.client.1.smithi174.stdout:3/527: mknod d0/d12/d46/cb3 0 2024-03-21T10:08:58.818 INFO:tasks.workunit.client.0.smithi153.stdout:3/752: read dd/d1e/d45/f74 [1079467,70259] 0 2024-03-21T10:08:58.818 INFO:tasks.workunit.client.1.smithi174.stdout:3/528: chown d0/f2 225 1 2024-03-21T10:08:58.820 INFO:tasks.workunit.client.1.smithi174.stdout:1/574: truncate d4/d7/d33/d43/d77/f84 157699 0 2024-03-21T10:08:58.825 INFO:tasks.workunit.client.1.smithi174.stdout:3/529: unlink d0/d12/d14/d1a/d27/f9a 0 2024-03-21T10:08:58.825 INFO:tasks.workunit.client.0.smithi153.stdout:3/753: creat dd/d41/d80/dbb/def/ffe x:0 0 0 2024-03-21T10:08:58.827 INFO:tasks.workunit.client.1.smithi174.stdout:3/530: stat d0/ld 0 2024-03-21T10:08:58.829 INFO:tasks.workunit.client.0.smithi153.stdout:3/754: symlink dd/d1e/d3d/d53/dae/lff 0 2024-03-21T10:08:58.830 INFO:tasks.workunit.client.1.smithi174.stdout:9/559: dwrite d2/d11/d1d/d4b/fa8 [4194304,4194304] 0 2024-03-21T10:08:58.831 INFO:tasks.workunit.client.0.smithi153.stdout:3/755: rmdir dd/d1e/d3d/d53 39 2024-03-21T10:08:58.831 INFO:tasks.workunit.client.1.smithi174.stdout:3/531: write d0/fa [5517676,10463] 0 2024-03-21T10:08:58.834 INFO:tasks.workunit.client.0.smithi153.stdout:3/756: dread dd/d1e/d2f/d35/d68/d75/ddd/fbd [0,4194304] 0 2024-03-21T10:08:58.839 INFO:tasks.workunit.client.1.smithi174.stdout:3/532: mkdir d0/d12/d14/d1a/d27/d3c/d73/db4 0 2024-03-21T10:08:58.840 INFO:tasks.workunit.client.1.smithi174.stdout:4/582: dwrite d4/f3e [0,4194304] 0 2024-03-21T10:08:58.841 INFO:tasks.workunit.client.0.smithi153.stdout:3/757: link dd/d1e/da6/fcf dd/d1e/d3d/d94/de2/f100 0 2024-03-21T10:08:58.844 INFO:tasks.workunit.client.0.smithi153.stdout:5/672: sync 2024-03-21T10:08:58.844 INFO:tasks.workunit.client.0.smithi153.stdout:1/552: sync 2024-03-21T10:08:58.844 INFO:tasks.workunit.client.0.smithi153.stdout:2/769: sync 2024-03-21T10:08:58.844 INFO:tasks.workunit.client.0.smithi153.stdout:0/632: sync 2024-03-21T10:08:58.845 INFO:tasks.workunit.client.1.smithi174.stdout:9/560: mknod d2/d11/cc7 0 2024-03-21T10:08:58.845 INFO:tasks.workunit.client.1.smithi174.stdout:1/575: unlink d4/d7/d15/d53/f6a 0 2024-03-21T10:08:58.845 INFO:tasks.workunit.client.0.smithi153.stdout:4/706: sync 2024-03-21T10:08:58.851 INFO:tasks.workunit.client.0.smithi153.stdout:4/707: mkdir d11/d18/d6d/dd6/ddd 0 2024-03-21T10:08:58.851 INFO:tasks.workunit.client.1.smithi174.stdout:9/561: creat d2/d11/d1d/d8b/fc8 x:0 0 0 2024-03-21T10:08:58.852 INFO:tasks.workunit.client.1.smithi174.stdout:1/576: mkdir d4/d7/d68/d89/dc6 0 2024-03-21T10:08:58.854 INFO:tasks.workunit.client.0.smithi153.stdout:0/633: dread d4/d16/d19/d30/f41 [0,4194304] 0 2024-03-21T10:08:58.856 INFO:tasks.workunit.client.0.smithi153.stdout:0/634: chown d4/db/c44 95 1 2024-03-21T10:08:58.856 INFO:tasks.workunit.client.0.smithi153.stdout:2/770: symlink dd/d15/d18/d2c/d4b/d56/d63/d8d/d9b/dc3/df5/l10a 0 2024-03-21T10:08:58.856 INFO:tasks.workunit.client.1.smithi174.stdout:9/562: chown d2/f8e 6769259 1 2024-03-21T10:08:58.857 INFO:tasks.workunit.client.0.smithi153.stdout:0/635: fdatasync d4/d4c/d83/f91 0 2024-03-21T10:08:58.858 INFO:tasks.workunit.client.1.smithi174.stdout:1/577: mkdir d4/d7/d15/d53/d78/dc7 0 2024-03-21T10:08:58.862 INFO:tasks.workunit.client.0.smithi153.stdout:5/673: getdents d4/d8/d5a/d5d/dd9/d2f 0 2024-03-21T10:08:58.862 INFO:tasks.workunit.client.1.smithi174.stdout:4/583: dwrite d4/d21/d31/d6a/dab/f99 [0,4194304] 0 2024-03-21T10:08:58.864 INFO:tasks.workunit.client.1.smithi174.stdout:1/578: symlink d4/d2d/lc8 0 2024-03-21T10:08:58.865 INFO:tasks.workunit.client.0.smithi153.stdout:4/708: link d11/d18/l37 d11/d2d/da3/lde 0 2024-03-21T10:08:58.866 INFO:tasks.workunit.client.0.smithi153.stdout:7/621: dread d4/d2a/d3b/f98 [0,4194304] 0 2024-03-21T10:08:58.868 INFO:tasks.workunit.client.1.smithi174.stdout:4/584: creat d4/d21/d24/d28/d79/fbb x:0 0 0 2024-03-21T10:08:58.869 INFO:tasks.workunit.client.1.smithi174.stdout:3/533: dwrite d0/d12/d14/d1a/d27/d3c/d73/f8a [0,4194304] 0 2024-03-21T10:08:58.873 INFO:tasks.workunit.client.1.smithi174.stdout:4/585: fdatasync d4/d21/f5b 0 2024-03-21T10:08:58.876 INFO:tasks.workunit.client.1.smithi174.stdout:4/586: fsync d4/d19/f97 0 2024-03-21T10:08:58.877 INFO:tasks.workunit.client.1.smithi174.stdout:4/587: chown d4/f7 1814 1 2024-03-21T10:08:58.879 INFO:tasks.workunit.client.1.smithi174.stdout:3/534: mknod d0/d12/d14/d1a/d44/d56/d76/cb5 0 2024-03-21T10:08:58.880 INFO:tasks.workunit.client.0.smithi153.stdout:0/636: rename d4/d16/c68 to d4/d4c/d83/d22/cc4 0 2024-03-21T10:08:58.883 INFO:tasks.workunit.client.0.smithi153.stdout:2/771: dwrite dd/d15/d5e/fc0 [0,4194304] 0 2024-03-21T10:08:58.884 INFO:tasks.workunit.client.1.smithi174.stdout:8/448: sync 2024-03-21T10:08:58.886 INFO:tasks.workunit.client.1.smithi174.stdout:8/449: write d9/f30 [3481248,8264] 0 2024-03-21T10:08:58.886 INFO:tasks.workunit.client.0.smithi153.stdout:4/709: link d11/d2d/d6b/l7c d11/d71/d89/dca/ldf 0 2024-03-21T10:08:58.886 INFO:tasks.workunit.client.0.smithi153.stdout:7/622: write d4/d19/fb6 [991670,71580] 0 2024-03-21T10:08:58.890 INFO:tasks.workunit.client.0.smithi153.stdout:3/758: dwrite dd/f43 [0,4194304] 0 2024-03-21T10:08:58.893 INFO:tasks.workunit.client.1.smithi174.stdout:3/535: rename d0/d12/d46/d9b to d0/d12/d14/d1a/d44/d94/db6 0 2024-03-21T10:08:58.893 INFO:tasks.workunit.client.1.smithi174.stdout:4/588: getdents d4/d21/d31/d3a/d67/d76/d82 0 2024-03-21T10:08:58.894 INFO:tasks.workunit.client.1.smithi174.stdout:1/579: write d4/d7/d15/d4f/f69 [46658,45541] 0 2024-03-21T10:08:58.894 INFO:tasks.workunit.client.0.smithi153.stdout:0/637: mkdir d4/d16/d27/dc5 0 2024-03-21T10:08:58.895 INFO:tasks.workunit.client.1.smithi174.stdout:8/450: write d9/d2a/d39/f47 [1032725,24336] 0 2024-03-21T10:08:58.898 INFO:tasks.workunit.client.1.smithi174.stdout:4/589: write d4/d21/d24/fb5 [475126,122091] 0 2024-03-21T10:08:58.899 INFO:tasks.workunit.client.0.smithi153.stdout:7/623: unlink d4/d2a/d40/d50/fcb 0 2024-03-21T10:08:58.900 INFO:tasks.workunit.client.0.smithi153.stdout:2/772: fdatasync dd/f37 0 2024-03-21T10:08:58.901 INFO:tasks.workunit.client.0.smithi153.stdout:0/638: readlink d4/d16/l59 0 2024-03-21T10:08:58.904 INFO:tasks.workunit.client.0.smithi153.stdout:5/674: dwrite d4/d5/df/d5e/d69/f7e [0,4194304] 0 2024-03-21T10:08:58.905 INFO:tasks.workunit.client.0.smithi153.stdout:2/773: stat dd/d15/d18/d2c/d33/d98 0 2024-03-21T10:08:58.908 INFO:tasks.workunit.client.0.smithi153.stdout:3/759: rename dd/d1e/d2f/d35/daf/dcc/dd6/le1 to dd/d1e/d2f/d35/daf/dcc/dd6/l101 0 2024-03-21T10:08:58.909 INFO:tasks.workunit.client.0.smithi153.stdout:0/639: dread - d4/d16/d19/d30/d6c/f86 zero size 2024-03-21T10:08:58.912 INFO:tasks.workunit.client.1.smithi174.stdout:3/536: dwrite d0/f2 [0,4194304] 0 2024-03-21T10:08:58.914 INFO:tasks.workunit.client.0.smithi153.stdout:7/624: chown d4/d14/dd0/d20/d9a/db7/f96 930937797 1 2024-03-21T10:08:58.914 INFO:tasks.workunit.client.0.smithi153.stdout:0/640: dread - d4/d16/d19/f85 zero size 2024-03-21T10:08:58.917 INFO:tasks.workunit.client.0.smithi153.stdout:4/710: write d11/d2d/f69 [440720,130858] 0 2024-03-21T10:08:58.920 INFO:tasks.workunit.client.0.smithi153.stdout:4/711: write d11/d18/f41 [5901778,71470] 0 2024-03-21T10:08:58.921 INFO:tasks.workunit.client.0.smithi153.stdout:7/625: rename d4/d11/da1/da6 to d4/d19/d4d/dd5 0 2024-03-21T10:08:58.925 INFO:tasks.workunit.client.0.smithi153.stdout:5/675: dread d4/d8/f68 [0,4194304] 0 2024-03-21T10:08:58.927 INFO:tasks.workunit.client.0.smithi153.stdout:7/626: chown d4/d2a/d3b/d81/l8c 53558916 1 2024-03-21T10:08:58.929 INFO:tasks.workunit.client.0.smithi153.stdout:4/712: mknod d11/d2d/da3/dac/dce/ce0 0 2024-03-21T10:08:58.930 INFO:tasks.workunit.client.0.smithi153.stdout:7/627: readlink d4/d19/d4d/dd5/laf 0 2024-03-21T10:08:58.932 INFO:tasks.workunit.client.1.smithi174.stdout:1/580: creat d4/d5/fc9 x:0 0 0 2024-03-21T10:08:58.932 INFO:tasks.workunit.client.1.smithi174.stdout:8/451: chown d9/d2a/d39/c62 42 1 2024-03-21T10:08:58.933 INFO:tasks.workunit.client.1.smithi174.stdout:4/590: creat d4/d21/d31/d6a/d8f/d9c/fbc x:0 0 0 2024-03-21T10:08:58.935 INFO:tasks.workunit.client.1.smithi174.stdout:3/537: rename d0/d12/d14/d1a/d44/d57/d93 to d0/d12/d14/d1a/d27/d3c/d73/db7 0 2024-03-21T10:08:58.936 INFO:tasks.workunit.client.0.smithi153.stdout:5/676: mknod d4/d5/df/d5e/d69/ce0 0 2024-03-21T10:08:58.937 INFO:tasks.workunit.client.1.smithi174.stdout:4/591: unlink d4/d21/d24/f42 0 2024-03-21T10:08:58.937 INFO:tasks.workunit.client.0.smithi153.stdout:3/760: dwrite dd/d1e/d2f/f90 [0,4194304] 0 2024-03-21T10:08:58.938 INFO:tasks.workunit.client.1.smithi174.stdout:8/452: unlink d9/d2a/d39/d76/d19/c28 0 2024-03-21T10:08:58.939 INFO:tasks.workunit.client.0.smithi153.stdout:4/713: creat d11/d2d/d6b/d9c/fe1 x:0 0 0 2024-03-21T10:08:58.941 INFO:tasks.workunit.client.1.smithi174.stdout:4/592: unlink d4/f18 0 2024-03-21T10:08:58.944 INFO:tasks.workunit.client.0.smithi153.stdout:0/641: dwrite d4/d16/d19/d30/d6c/d98/d5b/fa9 [0,4194304] 0 2024-03-21T10:08:58.946 INFO:tasks.workunit.client.1.smithi174.stdout:4/593: creat d4/d21/d31/d3a/d67/d76/fbd x:0 0 0 2024-03-21T10:08:58.947 INFO:tasks.workunit.client.1.smithi174.stdout:4/594: fdatasync d4/d21/d24/d28/f51 0 2024-03-21T10:08:58.948 INFO:tasks.workunit.client.0.smithi153.stdout:3/761: symlink dd/d1e/d3d/d94/d73/d7f/l102 0 2024-03-21T10:08:58.951 INFO:tasks.workunit.client.0.smithi153.stdout:3/762: read dd/f27 [1719779,126923] 0 2024-03-21T10:08:58.952 INFO:tasks.workunit.client.0.smithi153.stdout:2/774: write dd/d15/f40 [3597140,126712] 0 2024-03-21T10:08:58.957 INFO:tasks.workunit.client.1.smithi174.stdout:8/453: dread d9/d2a/d39/d76/d19/d27/f64 [0,4194304] 0 2024-03-21T10:08:58.958 INFO:tasks.workunit.client.0.smithi153.stdout:7/628: dwrite d4/d14/d16/f39 [0,4194304] 0 2024-03-21T10:08:58.959 INFO:tasks.workunit.client.0.smithi153.stdout:5/677: rename d4/d5/df/d67/da0/db3/d8d to d4/d8/d5a/d5d/dd9/d44/de1 0 2024-03-21T10:08:58.961 INFO:tasks.workunit.client.1.smithi174.stdout:8/454: mknod d9/d2a/d39/d76/c98 0 2024-03-21T10:08:58.961 INFO:tasks.workunit.client.1.smithi174.stdout:4/595: dread d4/d21/d24/d28/d38/f54 [0,4194304] 0 2024-03-21T10:08:58.965 INFO:tasks.workunit.client.1.smithi174.stdout:3/538: truncate d0/d12/d14/f3a 3125938 0 2024-03-21T10:08:58.971 INFO:tasks.workunit.client.0.smithi153.stdout:2/775: creat dd/d15/d18/d2c/d33/deb/f10b x:0 0 0 2024-03-21T10:08:58.972 INFO:tasks.workunit.client.1.smithi174.stdout:4/596: rmdir d4/d21/d31/d3a/d7a/d9d/da3 39 2024-03-21T10:08:58.972 INFO:tasks.workunit.client.1.smithi174.stdout:4/597: readlink d4/l17 0 2024-03-21T10:08:58.977 INFO:tasks.workunit.client.0.smithi153.stdout:3/763: creat dd/d41/d80/f103 x:0 0 0 2024-03-21T10:08:58.980 INFO:tasks.workunit.client.1.smithi174.stdout:1/581: dread d4/d7/d28/d59/f66 [0,4194304] 0 2024-03-21T10:08:58.980 INFO:tasks.workunit.client.0.smithi153.stdout:1/553: sync 2024-03-21T10:08:58.980 INFO:tasks.workunit.client.0.smithi153.stdout:6/879: sync 2024-03-21T10:08:58.980 INFO:tasks.workunit.client.0.smithi153.stdout:9/729: sync 2024-03-21T10:08:58.981 INFO:tasks.workunit.client.0.smithi153.stdout:8/665: sync 2024-03-21T10:08:58.981 INFO:tasks.workunit.client.0.smithi153.stdout:7/629: creat d4/fd6 x:0 0 0 2024-03-21T10:08:58.983 INFO:tasks.workunit.client.1.smithi174.stdout:8/455: dwrite d9/d2a/d39/d76/f69 [0,4194304] 0 2024-03-21T10:08:58.983 INFO:tasks.workunit.client.0.smithi153.stdout:1/554: mkdir d6/d32/daa 0 2024-03-21T10:08:58.987 INFO:tasks.workunit.client.0.smithi153.stdout:4/714: dwrite d11/d18/f21 [0,4194304] 0 2024-03-21T10:08:58.987 INFO:tasks.workunit.client.1.smithi174.stdout:4/598: rename d4/d21/d31/d6a/d8f/d9c/fbc to d4/d21/d31/d3a/d7a/d9d/da3/fbe 0 2024-03-21T10:08:58.988 INFO:tasks.workunit.client.0.smithi153.stdout:6/880: creat d1f/d2b/d10a/d40/d4f/d66/f13e x:0 0 0 2024-03-21T10:08:58.990 INFO:tasks.workunit.client.0.smithi153.stdout:7/630: truncate d4/d14/fa4 459630 0 2024-03-21T10:08:58.993 INFO:tasks.workunit.client.1.smithi174.stdout:3/539: dwrite d0/d12/d14/d1a/d27/d3c/f81 [4194304,4194304] 0 2024-03-21T10:08:58.996 INFO:tasks.workunit.client.1.smithi174.stdout:1/582: link d4/d7/f8f d4/d7/d32/d86/fca 0 2024-03-21T10:08:58.996 INFO:tasks.workunit.client.0.smithi153.stdout:3/764: rename dd/d1e/fd1 to dd/d1e/d3d/d94/db8/f104 0 2024-03-21T10:08:58.996 INFO:tasks.workunit.client.1.smithi174.stdout:4/599: truncate d4/d21/d31/d6a/fb0 904081 0 2024-03-21T10:08:58.998 INFO:tasks.workunit.client.1.smithi174.stdout:1/583: fdatasync d4/d7/d33/f36 0 2024-03-21T10:08:59.000 INFO:tasks.workunit.client.0.smithi153.stdout:4/715: unlink d11/d18/l4d 0 2024-03-21T10:08:59.002 INFO:tasks.workunit.client.1.smithi174.stdout:4/600: read - d4/d21/d31/d3a/d67/d76/fbd zero size 2024-03-21T10:08:59.003 INFO:tasks.workunit.client.1.smithi174.stdout:3/540: creat d0/d12/d14/d4c/d7a/d9e/fb8 x:0 0 0 2024-03-21T10:08:59.005 INFO:tasks.workunit.client.0.smithi153.stdout:4/716: read d11/fb0 [932991,83275] 0 2024-03-21T10:08:59.006 INFO:tasks.workunit.client.0.smithi153.stdout:6/881: mkdir d1f/d93/dd8/d103/d13f 0 2024-03-21T10:08:59.009 INFO:tasks.workunit.client.1.smithi174.stdout:1/584: link d4/d7/d33/d43/f47 d4/d5/d6c/fcb 0 2024-03-21T10:08:59.012 INFO:tasks.workunit.client.1.smithi174.stdout:4/601: creat d4/fbf x:0 0 0 2024-03-21T10:08:59.012 INFO:tasks.workunit.client.1.smithi174.stdout:1/585: fdatasync d4/d5/d6c/fa7 0 2024-03-21T10:08:59.012 INFO:tasks.workunit.client.0.smithi153.stdout:9/730: write d1/d4e/f98 [2586272,81910] 0 2024-03-21T10:08:59.012 INFO:tasks.workunit.client.0.smithi153.stdout:7/631: dread - d4/d14/dd0/d83/f89 zero size 2024-03-21T10:08:59.015 INFO:tasks.workunit.client.0.smithi153.stdout:9/731: chown d1/l11 739659 1 2024-03-21T10:08:59.017 INFO:tasks.workunit.client.0.smithi153.stdout:5/678: dwrite d4/d5/df/d67/d86/fd7 [0,4194304] 0 2024-03-21T10:08:59.017 INFO:tasks.workunit.client.0.smithi153.stdout:0/642: dread d4/d4c/f7a [0,4194304] 0 2024-03-21T10:08:59.017 INFO:tasks.workunit.client.0.smithi153.stdout:4/717: mknod d11/d2d/d3d/da1/ce2 0 2024-03-21T10:08:59.019 INFO:tasks.workunit.client.1.smithi174.stdout:1/586: mknod d4/d7/d15/d53/d78/dc7/ccc 0 2024-03-21T10:08:59.020 INFO:tasks.workunit.client.0.smithi153.stdout:3/765: mkdir dd/d1e/d45/d105 0 2024-03-21T10:08:59.020 INFO:tasks.workunit.client.1.smithi174.stdout:4/602: link d4/d21/d31/d6a/fb8 d4/d21/d31/d3a/d67/d76/fc0 0 2024-03-21T10:08:59.021 INFO:tasks.workunit.client.1.smithi174.stdout:1/587: chown d4/d7/d33/d43/d51/l63 27859 1 2024-03-21T10:08:59.024 INFO:tasks.workunit.client.0.smithi153.stdout:5/679: chown d4/d8/d5a/f61 50 1 2024-03-21T10:08:59.025 INFO:tasks.workunit.client.1.smithi174.stdout:1/588: chown d4/d5/c10 5826 1 2024-03-21T10:08:59.025 INFO:tasks.workunit.client.0.smithi153.stdout:6/882: symlink d1f/d2b/d10a/d40/d118/l140 0 2024-03-21T10:08:59.026 INFO:tasks.workunit.client.0.smithi153.stdout:7/632: mknod d4/d2a/d3b/cd7 0 2024-03-21T10:08:59.027 INFO:tasks.workunit.client.0.smithi153.stdout:7/633: readlink d4/l2b 0 2024-03-21T10:08:59.030 INFO:tasks.workunit.client.1.smithi174.stdout:1/589: mknod d4/d7/d28/dc5/ccd 0 2024-03-21T10:08:59.030 INFO:tasks.workunit.client.0.smithi153.stdout:4/718: mkdir d11/d18/d50/d65/de3 0 2024-03-21T10:08:59.034 INFO:tasks.workunit.client.0.smithi153.stdout:0/643: read d4/d16/d19/d58/f9c [1726902,129895] 0 2024-03-21T10:08:59.035 INFO:tasks.workunit.client.0.smithi153.stdout:2/776: dread dd/d15/d18/d2c/d4b/d56/d63/d8d/d9b/dc3/df5/dae/d50/d59/fe7 [0,4194304] 0 2024-03-21T10:08:59.036 INFO:tasks.workunit.client.1.smithi174.stdout:1/590: rename d4/d7/d33/d43/d51/f64 to d4/d2d/fce 0 2024-03-21T10:08:59.036 INFO:tasks.workunit.client.0.smithi153.stdout:7/634: symlink d4/d11/d56/ld8 0 2024-03-21T10:08:59.039 INFO:tasks.workunit.client.1.smithi174.stdout:1/591: write d4/d5/d6c/fc0 [891481,11600] 0 2024-03-21T10:08:59.041 INFO:tasks.workunit.client.1.smithi174.stdout:4/603: dwrite d4/d21/d31/d3a/d67/d76/fbd [0,4194304] 0 2024-03-21T10:08:59.043 INFO:tasks.workunit.client.0.smithi153.stdout:3/766: rename dd/d1e/d2f/d35/c72 to dd/d41/d80/dbb/c106 0 2024-03-21T10:08:59.045 INFO:tasks.workunit.client.1.smithi174.stdout:1/592: symlink d4/d7/d70/lcf 0 2024-03-21T10:08:59.047 INFO:tasks.workunit.client.0.smithi153.stdout:2/777: mknod dd/d15/d5e/db3/c10c 0 2024-03-21T10:08:59.047 INFO:tasks.workunit.client.0.smithi153.stdout:7/635: chown d4/l85 234621 1 2024-03-21T10:08:59.048 INFO:tasks.workunit.client.1.smithi174.stdout:1/593: mknod d4/d7/d33/cd0 0 2024-03-21T10:08:59.049 INFO:tasks.workunit.client.0.smithi153.stdout:3/767: write dd/d1e/d3d/ff7 [651369,90184] 0 2024-03-21T10:08:59.050 INFO:tasks.workunit.client.0.smithi153.stdout:1/555: write d6/f1c [3528702,9122] 0 2024-03-21T10:08:59.050 INFO:tasks.workunit.client.1.smithi174.stdout:1/594: creat d4/d5/d6c/fd1 x:0 0 0 2024-03-21T10:08:59.051 INFO:tasks.workunit.client.0.smithi153.stdout:0/644: creat d4/d16/d19/d30/d6c/d98/d5e/db2/fc6 x:0 0 0 2024-03-21T10:08:59.054 INFO:tasks.workunit.client.1.smithi174.stdout:2/529: sync 2024-03-21T10:08:59.055 INFO:tasks.workunit.client.1.smithi174.stdout:2/530: write d5/d25/fa4 [4794919,47031] 0 2024-03-21T10:08:59.060 INFO:tasks.workunit.client.1.smithi174.stdout:4/604: rename d4/d19/f22 to d4/d21/d24/d28/fc1 0 2024-03-21T10:08:59.062 INFO:tasks.workunit.client.0.smithi153.stdout:8/666: dread d5/d9/d43/fa0 [0,4194304] 0 2024-03-21T10:08:59.062 INFO:tasks.workunit.client.1.smithi174.stdout:2/531: creat d5/d25/d27/d2e/d85/dad/fb3 x:0 0 0 2024-03-21T10:08:59.063 INFO:tasks.workunit.client.0.smithi153.stdout:4/719: dwrite d11/f96 [0,4194304] 0 2024-03-21T10:08:59.067 INFO:tasks.workunit.client.0.smithi153.stdout:5/680: write d4/d5/d16/f63 [4894502,29047] 0 2024-03-21T10:08:59.068 INFO:tasks.workunit.client.1.smithi174.stdout:3/541: write d0/d12/d14/d1a/d44/f6c [947247,7906] 0 2024-03-21T10:08:59.070 INFO:tasks.workunit.client.1.smithi174.stdout:2/532: mkdir d5/d25/d27/d2d/db4 0 2024-03-21T10:08:59.071 INFO:tasks.workunit.client.0.smithi153.stdout:3/768: creat dd/d1e/d3d/d94/d8e/f107 x:0 0 0 2024-03-21T10:08:59.072 INFO:tasks.workunit.client.1.smithi174.stdout:1/595: dwrite d4/d2d/fb9 [0,4194304] 0 2024-03-21T10:08:59.075 INFO:tasks.workunit.client.1.smithi174.stdout:3/542: truncate d0/d12/d14/f97 772415 0 2024-03-21T10:08:59.076 INFO:tasks.workunit.client.0.smithi153.stdout:8/667: mknod d5/d9/cde 0 2024-03-21T10:08:59.078 INFO:tasks.workunit.client.0.smithi153.stdout:4/720: dread d11/d2d/f4a [0,4194304] 0 2024-03-21T10:08:59.078 INFO:tasks.workunit.client.1.smithi174.stdout:1/596: dread d4/d7/d33/d43/d77/f9a [0,4194304] 0 2024-03-21T10:08:59.079 INFO:tasks.workunit.client.0.smithi153.stdout:2/778: mkdir dd/d15/d18/d3a/d10d 0 2024-03-21T10:08:59.079 INFO:tasks.workunit.client.1.smithi174.stdout:1/597: fdatasync d4/f19 0 2024-03-21T10:08:59.081 INFO:tasks.workunit.client.0.smithi153.stdout:9/732: dread d1/d43/d44/d46/f26 [0,4194304] 0 2024-03-21T10:08:59.082 INFO:tasks.workunit.client.0.smithi153.stdout:0/645: symlink d4/d16/d19/d30/lc7 0 2024-03-21T10:08:59.083 INFO:tasks.workunit.client.1.smithi174.stdout:3/543: truncate d0/d12/d14/d4c/d7a/d9e/fb8 849124 0 2024-03-21T10:08:59.085 INFO:tasks.workunit.client.0.smithi153.stdout:3/769: fsync dd/d1e/d2f/f36 0 2024-03-21T10:08:59.085 INFO:tasks.workunit.client.0.smithi153.stdout:7/636: dwrite d4/d2a/d3b/d81/f99 [0,4194304] 0 2024-03-21T10:08:59.085 INFO:tasks.workunit.client.1.smithi174.stdout:3/544: fdatasync d0/d12/d14/d1a/f40 0 2024-03-21T10:08:59.088 INFO:tasks.workunit.client.1.smithi174.stdout:2/533: mkdir d5/d25/d27/d2d/d45/d5b/d2f/d66/db5 0 2024-03-21T10:08:59.089 INFO:tasks.workunit.client.1.smithi174.stdout:3/545: write d0/d12/d14/d1a/d27/d3c/fa3 [1442339,54164] 0 2024-03-21T10:08:59.090 INFO:tasks.workunit.client.0.smithi153.stdout:9/733: readlink d1/d43/d72/d9d/ddc/lec 0 2024-03-21T10:08:59.091 INFO:tasks.workunit.client.0.smithi153.stdout:2/779: write dd/d15/f52 [8535946,12712] 0 2024-03-21T10:08:59.092 INFO:tasks.workunit.client.1.smithi174.stdout:3/546: write d0/d12/d14/d1a/f40 [1356175,96888] 0 2024-03-21T10:08:59.093 INFO:tasks.workunit.client.0.smithi153.stdout:6/883: dwrite d1f/d2b/d10a/d40/d65/fde [0,4194304] 0 2024-03-21T10:08:59.093 INFO:tasks.workunit.client.0.smithi153.stdout:7/637: chown d4/d14/dd0/c5f 8317168 1 2024-03-21T10:08:59.094 INFO:tasks.workunit.client.1.smithi174.stdout:1/598: rename d4/d7/d28/f3b to d4/d7/d32/d86/fd2 0 2024-03-21T10:08:59.094 INFO:tasks.workunit.client.0.smithi153.stdout:4/721: truncate d11/d2d/fd0 1027988 0 2024-03-21T10:08:59.094 INFO:tasks.workunit.client.0.smithi153.stdout:0/646: fdatasync d4/d16/d19/d30/d6c/d98/d5e/f99 0 2024-03-21T10:08:59.095 INFO:tasks.workunit.client.0.smithi153.stdout:8/668: truncate d5/d9/d43/dd/d2d/d45/d88/f73 1821028 0 2024-03-21T10:08:59.096 INFO:tasks.workunit.client.0.smithi153.stdout:9/734: fdatasync d1/d43/d44/d46/d9e/d24/fb4 0 2024-03-21T10:08:59.100 INFO:tasks.workunit.client.0.smithi153.stdout:9/735: chown d1/d4e/d68/f81 222698805 1 2024-03-21T10:08:59.102 INFO:tasks.workunit.client.0.smithi153.stdout:4/722: mkdir d11/d9d/de4 0 2024-03-21T10:08:59.105 INFO:tasks.workunit.client.1.smithi174.stdout:4/605: truncate d4/d19/f23 545791 0 2024-03-21T10:08:59.107 INFO:tasks.workunit.client.0.smithi153.stdout:6/884: dread - d1f/d99/fec zero size 2024-03-21T10:08:59.107 INFO:tasks.workunit.client.1.smithi174.stdout:2/534: creat d5/d25/d27/d2d/d45/d5b/fb6 x:0 0 0 2024-03-21T10:08:59.107 INFO:tasks.workunit.client.0.smithi153.stdout:0/647: chown d4/d16/d19/d30/d6c/f86 101702 1 2024-03-21T10:08:59.108 INFO:tasks.workunit.client.1.smithi174.stdout:4/606: chown d4/d21/d31/cb7 2121622 1 2024-03-21T10:08:59.108 INFO:tasks.workunit.client.1.smithi174.stdout:1/599: dread d4/d2d/f76 [0,4194304] 0 2024-03-21T10:08:59.108 INFO:tasks.workunit.client.0.smithi153.stdout:8/669: symlink d5/d9/d43/dd/ddd/ldf 0 2024-03-21T10:08:59.108 INFO:tasks.workunit.client.0.smithi153.stdout:4/723: symlink d11/d2d/d3d/le5 0 2024-03-21T10:08:59.110 INFO:tasks.workunit.client.0.smithi153.stdout:6/885: fdatasync f19 0 2024-03-21T10:08:59.114 INFO:tasks.workunit.client.1.smithi174.stdout:1/600: creat d4/d7/d33/d43/d77/d87/fd3 x:0 0 0 2024-03-21T10:08:59.115 INFO:tasks.workunit.client.0.smithi153.stdout:1/556: write d6/d13/f79 [66436,46754] 0 2024-03-21T10:08:59.116 INFO:tasks.workunit.client.1.smithi174.stdout:1/601: chown d4/d2d/f34 111 1 2024-03-21T10:08:59.117 INFO:tasks.workunit.client.1.smithi174.stdout:2/535: rename d5/cd to d5/d25/d27/d2e/d4f/d65/d77/cb7 0 2024-03-21T10:08:59.117 INFO:tasks.workunit.client.0.smithi153.stdout:6/886: fsync d1f/d2b/d10a/d40/d51/dbd/ff5 0 2024-03-21T10:08:59.118 INFO:tasks.workunit.client.0.smithi153.stdout:0/648: creat d4/d16/d19/d58/d9a/fc8 x:0 0 0 2024-03-21T10:08:59.123 INFO:tasks.workunit.client.1.smithi174.stdout:1/602: creat d4/d7/d70/fd4 x:0 0 0 2024-03-21T10:08:59.124 INFO:tasks.workunit.client.1.smithi174.stdout:2/536: mknod d5/d25/d27/d2e/d7b/cb8 0 2024-03-21T10:08:59.126 INFO:tasks.workunit.client.0.smithi153.stdout:1/557: rmdir d6/d3a/d66/d80 39 2024-03-21T10:08:59.127 INFO:tasks.workunit.client.0.smithi153.stdout:6/887: chown d1f/c132 517809 1 2024-03-21T10:08:59.129 INFO:tasks.workunit.client.1.smithi174.stdout:3/547: dread d0/f7 [0,4194304] 0 2024-03-21T10:08:59.129 INFO:tasks.workunit.client.1.smithi174.stdout:3/548: readlink d0/l39 0 2024-03-21T10:08:59.132 INFO:tasks.workunit.client.0.smithi153.stdout:4/724: mknod d11/d9d/de4/ce6 0 2024-03-21T10:08:59.133 INFO:tasks.workunit.client.0.smithi153.stdout:7/638: getdents d4/d14/dd0/d20/d9a/dd2 0 2024-03-21T10:08:59.133 INFO:tasks.workunit.client.1.smithi174.stdout:1/603: creat d4/d7/d68/d8e/db8/fd5 x:0 0 0 2024-03-21T10:08:59.134 INFO:tasks.workunit.client.0.smithi153.stdout:9/736: getdents d1/d43/d44/d46/d9e/d24/d41/dbe/dcc 0 2024-03-21T10:08:59.137 INFO:tasks.workunit.client.1.smithi174.stdout:1/604: write d4/d7/f90 [159545,107547] 0 2024-03-21T10:08:59.138 INFO:tasks.workunit.client.0.smithi153.stdout:5/681: dread d4/fa [0,4194304] 0 2024-03-21T10:08:59.143 INFO:tasks.workunit.client.1.smithi174.stdout:4/607: dread d4/f52 [0,4194304] 0 2024-03-21T10:08:59.144 INFO:tasks.workunit.client.0.smithi153.stdout:1/558: truncate d6/d3a/d3c/f43 1811517 0 2024-03-21T10:08:59.144 INFO:tasks.workunit.client.0.smithi153.stdout:2/780: write dd/d15/d18/d2c/f2d [2958224,74479] 0 2024-03-21T10:08:59.145 INFO:tasks.workunit.client.1.smithi174.stdout:3/549: rename d0/d12/d14/d1a/d27/d3c/d73/db7/d7d/da6/da8 to d0/d12/d14/db9 0 2024-03-21T10:08:59.149 INFO:tasks.workunit.client.1.smithi174.stdout:2/537: write d5/f26 [752790,118693] 0 2024-03-21T10:08:59.149 INFO:tasks.workunit.client.0.smithi153.stdout:8/670: dwrite d5/d9/d43/dd/d2d/d45/d88/fbe [0,4194304] 0 2024-03-21T10:08:59.151 INFO:tasks.workunit.client.0.smithi153.stdout:7/639: write d4/d14/dd0/f7d [1603143,56577] 0 2024-03-21T10:08:59.152 INFO:tasks.workunit.client.0.smithi153.stdout:4/725: rename d11/d2d/d3d/f45 to d11/d2d/da3/dac/dce/fe7 0 2024-03-21T10:08:59.154 INFO:tasks.workunit.client.1.smithi174.stdout:4/608: rename d4/d21/d31/d3a/f8e to d4/d21/d31/d3a/d7a/d9d/fc2 0 2024-03-21T10:08:59.156 INFO:tasks.workunit.client.0.smithi153.stdout:9/737: creat d1/d43/d44/d46/d4d/d5c/ff6 x:0 0 0 2024-03-21T10:08:59.158 INFO:tasks.workunit.client.1.smithi174.stdout:3/550: rename d0/d12/d14/d4c/d7a/d9e/lab to d0/d12/d46/lba 0 2024-03-21T10:08:59.160 INFO:tasks.workunit.client.0.smithi153.stdout:2/781: read dd/f2e [2022437,48866] 0 2024-03-21T10:08:59.162 INFO:tasks.workunit.client.1.smithi174.stdout:3/551: dread d0/d12/d14/d4c/d7a/d9e/fb8 [0,4194304] 0 2024-03-21T10:08:59.163 INFO:tasks.workunit.client.0.smithi153.stdout:1/559: creat d6/d10/d35/fab x:0 0 0 2024-03-21T10:08:59.164 INFO:tasks.workunit.client.0.smithi153.stdout:9/738: stat d1/d43/d44/d46/d9e/d4b/l93 0 2024-03-21T10:08:59.167 INFO:tasks.workunit.client.0.smithi153.stdout:7/640: rename d4/d14/dd0/d20/c38 to d4/d19/d4d/cd9 0 2024-03-21T10:08:59.168 INFO:tasks.workunit.client.1.smithi174.stdout:4/609: dread d4/fa [0,4194304] 0 2024-03-21T10:08:59.171 INFO:tasks.workunit.client.0.smithi153.stdout:5/682: mkdir d4/d8/d5a/d5d/dd9/d44/dc7/de2 0 2024-03-21T10:08:59.171 INFO:tasks.workunit.client.0.smithi153.stdout:7/641: readlink d4/d2a/l37 0 2024-03-21T10:08:59.171 INFO:tasks.workunit.client.0.smithi153.stdout:1/560: write d6/d3a/d66/f68 [302418,69138] 0 2024-03-21T10:08:59.175 INFO:tasks.workunit.client.0.smithi153.stdout:9/739: chown d1/d43/d44/d46/d4d/d5c/ldb 47990842 1 2024-03-21T10:08:59.178 INFO:tasks.workunit.client.1.smithi174.stdout:7/457: sync 2024-03-21T10:08:59.180 INFO:tasks.workunit.client.0.smithi153.stdout:0/649: dread d4/f6 [0,4194304] 0 2024-03-21T10:08:59.180 INFO:tasks.workunit.client.0.smithi153.stdout:1/561: write d6/d3a/d66/f85 [1801295,68747] 0 2024-03-21T10:08:59.180 INFO:tasks.workunit.client.0.smithi153.stdout:9/740: chown d1/d43/d44/d46/c1e 192516 1 2024-03-21T10:08:59.182 INFO:tasks.workunit.client.0.smithi153.stdout:7/642: mkdir d4/d14/dd0/d20/d9a/db7/dda 0 2024-03-21T10:08:59.184 INFO:tasks.workunit.client.0.smithi153.stdout:4/726: dread d11/d2d/f42 [0,4194304] 0 2024-03-21T10:08:59.185 INFO:tasks.workunit.client.1.smithi174.stdout:3/552: dwrite d0/d12/f49 [0,4194304] 0 2024-03-21T10:08:59.187 INFO:tasks.workunit.client.1.smithi174.stdout:1/605: dwrite d4/d7/d15/f41 [4194304,4194304] 0 2024-03-21T10:08:59.187 INFO:tasks.workunit.client.1.smithi174.stdout:2/538: dwrite d5/d25/d27/d2d/d45/d5b/d2f/d4b/f78 [0,4194304] 0 2024-03-21T10:08:59.191 INFO:tasks.workunit.client.0.smithi153.stdout:4/727: dread - d11/d9d/fd7 zero size 2024-03-21T10:08:59.191 INFO:tasks.workunit.client.0.smithi153.stdout:4/728: fdatasync d11/db1/fd5 0 2024-03-21T10:08:59.194 INFO:tasks.workunit.client.1.smithi174.stdout:2/539: stat d5/d25/d27/d2d/d45/d5b/d2f/d4b/d76/f97 0 2024-03-21T10:08:59.194 INFO:tasks.workunit.client.0.smithi153.stdout:6/888: dread f11 [0,4194304] 0 2024-03-21T10:08:59.199 INFO:tasks.workunit.client.1.smithi174.stdout:1/606: mknod d4/d7/d32/da2/cd6 0 2024-03-21T10:08:59.199 INFO:tasks.workunit.client.0.smithi153.stdout:8/671: truncate d5/d9/d43/d7b/dc4/da7/fc6 3570799 0 2024-03-21T10:08:59.201 INFO:tasks.workunit.client.1.smithi174.stdout:3/553: dread d0/f7 [0,4194304] 0 2024-03-21T10:08:59.203 INFO:tasks.workunit.client.0.smithi153.stdout:1/562: creat d6/d13/d8e/fac x:0 0 0 2024-03-21T10:08:59.206 INFO:tasks.workunit.client.1.smithi174.stdout:6/402: sync 2024-03-21T10:08:59.207 INFO:tasks.workunit.client.1.smithi174.stdout:0/531: sync 2024-03-21T10:08:59.207 INFO:tasks.workunit.client.1.smithi174.stdout:8/456: sync 2024-03-21T10:08:59.207 INFO:tasks.workunit.client.1.smithi174.stdout:5/587: sync 2024-03-21T10:08:59.207 INFO:tasks.workunit.client.1.smithi174.stdout:9/563: sync 2024-03-21T10:08:59.207 INFO:tasks.workunit.client.0.smithi153.stdout:3/770: dread dd/d1e/d2f/d35/daf/dcc/d58/f8b [0,4194304] 0 2024-03-21T10:08:59.213 INFO:tasks.workunit.client.1.smithi174.stdout:1/607: read d4/d7/d15/d4f/f69 [96801,73724] 0 2024-03-21T10:08:59.217 INFO:tasks.workunit.client.0.smithi153.stdout:6/889: rename d1f/d93/dd8/f7d to d1f/d2b/d10a/d40/d5f/f141 0 2024-03-21T10:08:59.218 INFO:tasks.workunit.client.1.smithi174.stdout:9/564: truncate d2/d11/d60/f77 4211383 0 2024-03-21T10:08:59.221 INFO:tasks.workunit.client.0.smithi153.stdout:9/741: getdents d1/d43/d44 0 2024-03-21T10:08:59.221 INFO:tasks.workunit.client.0.smithi153.stdout:5/683: dread d4/d8/f6c [0,4194304] 0 2024-03-21T10:08:59.223 INFO:tasks.workunit.client.1.smithi174.stdout:6/403: mknod d0/d3/d5/d21/c7d 0 2024-03-21T10:08:59.226 INFO:tasks.workunit.client.1.smithi174.stdout:5/588: dread - d1/d16/d2e/d66/d8b/fb7 zero size 2024-03-21T10:08:59.226 INFO:tasks.workunit.client.0.smithi153.stdout:3/771: mknod dd/d41/c108 0 2024-03-21T10:08:59.229 INFO:tasks.workunit.client.0.smithi153.stdout:7/643: write d4/d14/dd0/d83/f89 [790154,110890] 0 2024-03-21T10:08:59.236 INFO:tasks.workunit.client.1.smithi174.stdout:7/458: dwrite d0/d4/d10/d1b/f7c [0,4194304] 0 2024-03-21T10:08:59.236 INFO:tasks.workunit.client.1.smithi174.stdout:8/457: read - d9/d2a/d39/d76/f3c zero size 2024-03-21T10:08:59.237 INFO:tasks.workunit.client.1.smithi174.stdout:3/554: creat d0/d12/d14/d1a/d27/d3c/d73/fbb x:0 0 0 2024-03-21T10:08:59.237 INFO:tasks.workunit.client.1.smithi174.stdout:1/608: creat d4/d7/d28/dc5/fd7 x:0 0 0 2024-03-21T10:08:59.237 INFO:tasks.workunit.client.1.smithi174.stdout:4/610: dwrite d4/f20 [0,4194304] 0 2024-03-21T10:08:59.237 INFO:tasks.workunit.client.1.smithi174.stdout:0/532: fsync d9/d21/d31/d3c/f60 0 2024-03-21T10:08:59.237 INFO:tasks.workunit.client.0.smithi153.stdout:7/644: unlink d4/fa7 0 2024-03-21T10:08:59.241 INFO:tasks.workunit.client.1.smithi174.stdout:2/540: dwrite d5/d25/d27/d2e/d85/dad/fb0 [0,4194304] 0 2024-03-21T10:08:59.243 INFO:tasks.workunit.client.1.smithi174.stdout:0/533: dread - d9/d21/d31/f40 zero size 2024-03-21T10:08:59.244 INFO:tasks.workunit.client.1.smithi174.stdout:5/589: symlink d1/d16/d1c/ld9 0 2024-03-21T10:08:59.244 INFO:tasks.workunit.client.0.smithi153.stdout:3/772: unlink dd/d1e/d45/f74 0 2024-03-21T10:08:59.247 INFO:tasks.workunit.client.0.smithi153.stdout:0/650: dread d4/d16/d19/d58/f9c [0,4194304] 0 2024-03-21T10:08:59.247 INFO:tasks.workunit.client.0.smithi153.stdout:1/563: dwrite d6/d3a/d3c/f8d [0,4194304] 0 2024-03-21T10:08:59.250 INFO:tasks.workunit.client.0.smithi153.stdout:7/645: creat d4/d14/d16/fdb x:0 0 0 2024-03-21T10:08:59.250 INFO:tasks.workunit.client.1.smithi174.stdout:7/459: write d0/d4/f2d [3572497,86375] 0 2024-03-21T10:08:59.252 INFO:tasks.workunit.client.1.smithi174.stdout:4/611: write d4/d19/f74 [3578861,39765] 0 2024-03-21T10:08:59.253 INFO:tasks.workunit.client.0.smithi153.stdout:3/773: mkdir dd/d1e/d2f/d35/dd0/d109 0 2024-03-21T10:08:59.253 INFO:tasks.workunit.client.1.smithi174.stdout:0/534: chown d9/d21/d31/d5c/c78 25400628 1 2024-03-21T10:08:59.256 INFO:tasks.workunit.client.0.smithi153.stdout:7/646: chown d4/d14/dd0/d83 1222485 1 2024-03-21T10:08:59.258 INFO:tasks.workunit.client.0.smithi153.stdout:1/564: creat d6/d10/d35/d5b/fad x:0 0 0 2024-03-21T10:08:59.259 INFO:tasks.workunit.client.1.smithi174.stdout:1/609: creat d4/d7/d68/d8e/db8/fd8 x:0 0 0 2024-03-21T10:08:59.259 INFO:tasks.workunit.client.1.smithi174.stdout:8/458: symlink d9/d53/d7e/l99 0 2024-03-21T10:08:59.261 INFO:tasks.workunit.client.1.smithi174.stdout:9/565: read d2/d11/d1d/d4b/fbf [3040273,34644] 0 2024-03-21T10:08:59.265 INFO:tasks.workunit.client.1.smithi174.stdout:6/404: getdents d0/d3/d20 0 2024-03-21T10:08:59.267 INFO:tasks.workunit.client.1.smithi174.stdout:8/459: chown d9/d2a/d39/f47 57 1 2024-03-21T10:08:59.269 INFO:tasks.workunit.client.1.smithi174.stdout:2/541: rename d5/d25/d27/d2d/d45/d5b/d63/f59 to d5/d25/d27/d2d/d45/d5b/d2f/d66/fb9 0 2024-03-21T10:08:59.272 INFO:tasks.workunit.client.0.smithi153.stdout:0/651: mknod d4/d16/d19/d95/cc9 0 2024-03-21T10:08:59.272 INFO:tasks.workunit.client.1.smithi174.stdout:8/460: readlink d9/lf 0 2024-03-21T10:08:59.274 INFO:tasks.workunit.client.1.smithi174.stdout:6/405: mknod d0/d3/d5/d6/c7e 0 2024-03-21T10:08:59.278 INFO:tasks.workunit.client.1.smithi174.stdout:8/461: fdatasync d9/f7c 0 2024-03-21T10:08:59.279 INFO:tasks.workunit.client.0.smithi153.stdout:5/684: write d4/d5/fd4 [1357886,28985] 0 2024-03-21T10:08:59.282 INFO:tasks.workunit.client.1.smithi174.stdout:4/612: truncate d4/d21/d24/d28/fc1 392933 0 2024-03-21T10:08:59.285 INFO:tasks.workunit.client.1.smithi174.stdout:2/542: creat d5/d25/d27/d2e/d7b/d81/fba x:0 0 0 2024-03-21T10:08:59.290 INFO:tasks.workunit.client.0.smithi153.stdout:4/729: dwrite f2 [4194304,4194304] 0 2024-03-21T10:08:59.290 INFO:tasks.workunit.client.1.smithi174.stdout:2/543: chown d5/d25/d27/d2d/d45/f6e 146527272 1 2024-03-21T10:08:59.291 INFO:tasks.workunit.client.1.smithi174.stdout:6/406: creat d0/d3/d20/f7f x:0 0 0 2024-03-21T10:08:59.291 INFO:tasks.workunit.client.1.smithi174.stdout:0/535: dwrite d9/f13 [0,4194304] 0 2024-03-21T10:08:59.291 INFO:tasks.workunit.client.1.smithi174.stdout:4/613: rename d4/d21/d24/d28/f51 to d4/d21/d31/d6a/dab/da8/fc3 0 2024-03-21T10:08:59.297 INFO:tasks.workunit.client.0.smithi153.stdout:8/672: dwrite d5/d1b/f8f [4194304,4194304] 0 2024-03-21T10:08:59.299 INFO:tasks.workunit.client.1.smithi174.stdout:2/544: chown d5/d25/d27/d2d/d45/d5b/d2f/f9d 15860 1 2024-03-21T10:08:59.300 INFO:tasks.workunit.client.0.smithi153.stdout:2/782: dread dd/d15/d18/d2c/f4e [0,4194304] 0 2024-03-21T10:08:59.305 INFO:tasks.workunit.client.1.smithi174.stdout:8/462: dwrite d9/d53/d5b/f77 [0,4194304] 0 2024-03-21T10:08:59.306 INFO:tasks.workunit.client.0.smithi153.stdout:9/742: dwrite d1/d3b/daa/fea [0,4194304] 0 2024-03-21T10:08:59.310 INFO:tasks.workunit.client.0.smithi153.stdout:3/774: dwrite dd/f1f [0,4194304] 0 2024-03-21T10:08:59.312 INFO:tasks.workunit.client.0.smithi153.stdout:0/652: rmdir d4/d16/d19/d95 39 2024-03-21T10:08:59.316 INFO:tasks.workunit.client.1.smithi174.stdout:2/545: truncate d5/d25/d27/d2d/d45/d5b/d2f/f9d 2306424 0 2024-03-21T10:08:59.317 INFO:tasks.workunit.client.0.smithi153.stdout:0/653: truncate d4/d16/d27/f90 743738 0 2024-03-21T10:08:59.317 INFO:tasks.workunit.client.0.smithi153.stdout:2/783: dread - dd/d15/d5e/f8f zero size 2024-03-21T10:08:59.318 INFO:tasks.workunit.client.1.smithi174.stdout:3/555: dread d0/d12/d14/d1a/d44/d94/f9d [0,4194304] 0 2024-03-21T10:08:59.318 INFO:tasks.workunit.client.0.smithi153.stdout:3/775: fdatasync dd/d1e/d3d/d94/d8e/fa2 0 2024-03-21T10:08:59.322 INFO:tasks.workunit.client.0.smithi153.stdout:8/673: mkdir d5/d9/d43/dd/d2d/d45/d88/de0 0 2024-03-21T10:08:59.326 INFO:tasks.workunit.client.1.smithi174.stdout:6/407: dwrite d0/d3/d5/d6/f6a [0,4194304] 0 2024-03-21T10:08:59.326 INFO:tasks.workunit.client.1.smithi174.stdout:8/463: getdents d9/d2a/d39/d58 0 2024-03-21T10:08:59.326 INFO:tasks.workunit.client.0.smithi153.stdout:1/565: creat d6/d13/d75/fae x:0 0 0 2024-03-21T10:08:59.330 INFO:tasks.workunit.client.1.smithi174.stdout:1/610: write d4/d7/d15/d4f/f61 [1609374,67220] 0 2024-03-21T10:08:59.330 INFO:tasks.workunit.client.1.smithi174.stdout:1/611: chown d4/d7 1 1 2024-03-21T10:08:59.335 INFO:tasks.workunit.client.1.smithi174.stdout:3/556: stat d0/d12/d14/d1a/d27/d3c/d73/db7/f38 0 2024-03-21T10:08:59.336 INFO:tasks.workunit.client.1.smithi174.stdout:6/408: symlink d0/d3/d5/d6/d42/l80 0 2024-03-21T10:08:59.338 INFO:tasks.workunit.client.1.smithi174.stdout:8/464: stat d9/d2a/d39/d76/d19/d27/c7f 0 2024-03-21T10:08:59.339 INFO:tasks.workunit.client.1.smithi174.stdout:2/546: rename d5/d25/d27/d2d/d45/d5b/d2f/d4b/d76/f86 to d5/d25/d27/d2d/d45/d5b/d2f/d66/fbb 0 2024-03-21T10:08:59.339 INFO:tasks.workunit.client.0.smithi153.stdout:3/776: truncate f2 4342316 0 2024-03-21T10:08:59.342 INFO:tasks.workunit.client.0.smithi153.stdout:9/743: creat d1/d43/d44/d46/d9e/d24/ff7 x:0 0 0 2024-03-21T10:08:59.345 INFO:tasks.workunit.client.1.smithi174.stdout:6/409: write d0/d3/d5/f45 [22677,109656] 0 2024-03-21T10:08:59.346 INFO:tasks.workunit.client.0.smithi153.stdout:8/674: dread - d5/d9/d43/dd/d20/d26/d8b/fa4 zero size 2024-03-21T10:08:59.348 INFO:tasks.workunit.client.1.smithi174.stdout:5/590: dwrite d1/d49/d4f/f57 [0,4194304] 0 2024-03-21T10:08:59.349 INFO:tasks.workunit.client.0.smithi153.stdout:3/777: stat dd/d1e/d2f/d35/fb5 0 2024-03-21T10:08:59.353 INFO:tasks.workunit.client.1.smithi174.stdout:1/612: rmdir d4/d7/d32/da2/da8/dc4 0 2024-03-21T10:08:59.355 INFO:tasks.workunit.client.0.smithi153.stdout:1/566: truncate d6/d10/d35/da5/f42 1099579 0 2024-03-21T10:08:59.356 INFO:tasks.workunit.client.1.smithi174.stdout:7/460: dread d0/ff [0,4194304] 0 2024-03-21T10:08:59.356 INFO:tasks.workunit.client.0.smithi153.stdout:4/730: dwrite d11/d18/fdb [0,4194304] 0 2024-03-21T10:08:59.358 INFO:tasks.workunit.client.1.smithi174.stdout:2/547: mkdir d5/d25/d27/d2d/db4/dbc 0 2024-03-21T10:08:59.358 INFO:tasks.workunit.client.0.smithi153.stdout:5/685: getdents d4/d8/d5a/d5d/dd9 0 2024-03-21T10:08:59.359 INFO:tasks.workunit.client.1.smithi174.stdout:6/410: mkdir d0/d3/d20/d3d/d81 0 2024-03-21T10:08:59.359 INFO:tasks.workunit.client.0.smithi153.stdout:9/744: mkdir d1/d3b/df8 0 2024-03-21T10:08:59.360 INFO:tasks.workunit.client.1.smithi174.stdout:8/465: rename d9/f63 to d9/d2a/d39/d76/d19/f9a 0 2024-03-21T10:08:59.365 INFO:tasks.workunit.client.1.smithi174.stdout:3/557: creat d0/d12/d14/d1a/fbc x:0 0 0 2024-03-21T10:08:59.366 INFO:tasks.workunit.client.0.smithi153.stdout:7/647: dread d4/d14/d2f/d78/f3a [4194304,4194304] 0 2024-03-21T10:08:59.366 INFO:tasks.workunit.client.0.smithi153.stdout:5/686: rmdir d4/d8 39 2024-03-21T10:08:59.367 INFO:tasks.workunit.client.0.smithi153.stdout:3/778: write dd/d1e/d2f/d35/dd0/fdc [4302788,60071] 0 2024-03-21T10:08:59.370 INFO:tasks.workunit.client.0.smithi153.stdout:2/784: creat dd/d15/d18/d2c/d4b/d56/d63/d8d/d9b/dc3/df5/dae/d50/d58/f10e x:0 0 0 2024-03-21T10:08:59.375 INFO:tasks.workunit.client.1.smithi174.stdout:9/566: dwrite d2/d11/d1d/f33 [0,4194304] 0 2024-03-21T10:08:59.378 INFO:tasks.workunit.client.0.smithi153.stdout:9/745: rename d1/d43/d44/d46/d9e/d24/d41/dbe/dcc/d8b to d1/d3b/daa/df9 0 2024-03-21T10:08:59.379 INFO:tasks.workunit.client.1.smithi174.stdout:0/536: truncate d9/d35/f36 650747 0 2024-03-21T10:08:59.381 INFO:tasks.workunit.client.1.smithi174.stdout:2/548: creat d5/d25/fbd x:0 0 0 2024-03-21T10:08:59.382 INFO:tasks.workunit.client.0.smithi153.stdout:9/746: dread - d1/d4e/d68/fb3 zero size 2024-03-21T10:08:59.386 INFO:tasks.workunit.client.1.smithi174.stdout:3/558: creat d0/d12/d14/d4c/d7a/d9e/fbd x:0 0 0 2024-03-21T10:08:59.387 INFO:tasks.workunit.client.0.smithi153.stdout:7/648: creat d4/d2a/fdc x:0 0 0 2024-03-21T10:08:59.388 INFO:tasks.workunit.client.0.smithi153.stdout:3/779: symlink dd/d1e/d2f/d35/d68/d75/ddd/db1/l10a 0 2024-03-21T10:08:59.388 INFO:tasks.workunit.client.0.smithi153.stdout:7/649: readlink d4/lf 0 2024-03-21T10:08:59.390 INFO:tasks.workunit.client.0.smithi153.stdout:2/785: truncate dd/d15/d18/d2c/ff6 240043 0 2024-03-21T10:08:59.393 INFO:tasks.workunit.client.0.smithi153.stdout:8/675: fdatasync d5/d1b/f59 0 2024-03-21T10:08:59.394 INFO:tasks.workunit.client.0.smithi153.stdout:4/731: link d11/d2d/d3d/l8b d11/d18/d6d/le8 0 2024-03-21T10:08:59.395 INFO:tasks.workunit.client.0.smithi153.stdout:7/650: truncate d4/d14/d2f/fbe 936410 0 2024-03-21T10:08:59.397 INFO:tasks.workunit.client.1.smithi174.stdout:9/567: creat d2/d11/d1d/d2a/fc9 x:0 0 0 2024-03-21T10:08:59.399 INFO:tasks.workunit.client.1.smithi174.stdout:1/613: getdents d4/d7/d15/d53/d78/dc7 0 2024-03-21T10:08:59.403 INFO:tasks.workunit.client.0.smithi153.stdout:9/747: unlink d1/d43/d44/f34 0 2024-03-21T10:08:59.403 INFO:tasks.workunit.client.0.smithi153.stdout:8/676: truncate d5/d9/d43/dd/d2d/d45/fbd 160372 0 2024-03-21T10:08:59.403 INFO:tasks.workunit.client.0.smithi153.stdout:0/654: dread d4/f37 [0,4194304] 0 2024-03-21T10:08:59.409 INFO:tasks.workunit.client.0.smithi153.stdout:9/748: write d1/d4/d5f/fe6 [666580,123999] 0 2024-03-21T10:08:59.409 INFO:tasks.workunit.client.0.smithi153.stdout:5/687: rename d4/d5/df/d67/da0/db3/fdb to d4/fe3 0 2024-03-21T10:08:59.409 INFO:tasks.workunit.client.0.smithi153.stdout:7/651: mknod d4/d19/cdd 0 2024-03-21T10:08:59.410 INFO:tasks.workunit.client.1.smithi174.stdout:1/614: truncate d4/d67/f74 22064 0 2024-03-21T10:08:59.410 INFO:tasks.workunit.client.1.smithi174.stdout:7/461: dread d0/d53/d5f/f71 [0,4194304] 0 2024-03-21T10:08:59.410 INFO:tasks.workunit.client.0.smithi153.stdout:3/780: dread dd/d1e/d3d/f69 [0,4194304] 0 2024-03-21T10:08:59.413 INFO:tasks.workunit.client.1.smithi174.stdout:0/537: dwrite d9/d21/d31/d3c/f83 [0,4194304] 0 2024-03-21T10:08:59.414 INFO:tasks.workunit.client.1.smithi174.stdout:9/568: rename d2/d11/d60/f6f to d2/d11/d1d/dab/dc1/fca 0 2024-03-21T10:08:59.415 INFO:tasks.workunit.client.1.smithi174.stdout:1/615: dread - d4/d7/d33/d43/d77/d87/fb4 zero size 2024-03-21T10:08:59.415 INFO:tasks.workunit.client.0.smithi153.stdout:4/732: rename d11/d2d/da3/ccf to d11/d18/d54/dab/ce9 0 2024-03-21T10:08:59.417 INFO:tasks.workunit.client.0.smithi153.stdout:6/890: sync 2024-03-21T10:08:59.419 INFO:tasks.workunit.client.1.smithi174.stdout:0/538: read d9/d21/d31/f42 [2822240,40004] 0 2024-03-21T10:08:59.420 INFO:tasks.workunit.client.1.smithi174.stdout:1/616: chown d4/d7/d68 3 1 2024-03-21T10:08:59.420 INFO:tasks.workunit.client.1.smithi174.stdout:5/591: write d1/f86 [496543,118261] 0 2024-03-21T10:08:59.421 INFO:tasks.workunit.client.1.smithi174.stdout:9/569: rmdir d2/d24/d2e 39 2024-03-21T10:08:59.422 INFO:tasks.workunit.client.0.smithi153.stdout:9/749: write d1/d43/d44/d46/d9e/d24/d41/f47 [1137193,82655] 0 2024-03-21T10:08:59.423 INFO:tasks.workunit.client.1.smithi174.stdout:1/617: fsync d4/d7/f90 0 2024-03-21T10:08:59.425 INFO:tasks.workunit.client.0.smithi153.stdout:7/652: symlink d4/d11/d56/db1/lde 0 2024-03-21T10:08:59.425 INFO:tasks.workunit.client.0.smithi153.stdout:7/653: fdatasync d4/d11/f72 0 2024-03-21T10:08:59.428 INFO:tasks.workunit.client.1.smithi174.stdout:1/618: chown d4/d7/d32/da2/cd6 6268 1 2024-03-21T10:08:59.432 INFO:tasks.workunit.client.0.smithi153.stdout:8/677: creat d5/fe1 x:0 0 0 2024-03-21T10:08:59.432 INFO:tasks.workunit.client.0.smithi153.stdout:2/786: rename dd/d15/d18/d2c/d4b/fe3 to dd/d15/d18/d2c/d33/d98/dc7/d107/f10f 0 2024-03-21T10:08:59.433 INFO:tasks.workunit.client.1.smithi174.stdout:6/411: write d0/d3/d5/df/d16/f1a [2160971,16967] 0 2024-03-21T10:08:59.433 INFO:tasks.workunit.client.1.smithi174.stdout:8/466: write d9/d2a/d39/d76/d19/d27/f41 [1279149,35288] 0 2024-03-21T10:08:59.437 INFO:tasks.workunit.client.0.smithi153.stdout:1/567: write d6/d13/d75/f57 [681469,108361] 0 2024-03-21T10:08:59.437 INFO:tasks.workunit.client.1.smithi174.stdout:9/570: dread - d2/d24/d2e/fc4 zero size 2024-03-21T10:08:59.439 INFO:tasks.workunit.client.1.smithi174.stdout:4/614: dwrite d4/d21/d31/d6a/fb0 [0,4194304] 0 2024-03-21T10:08:59.442 INFO:tasks.workunit.client.1.smithi174.stdout:2/549: dread d5/d25/d27/d74/f95 [0,4194304] 0 2024-03-21T10:08:59.442 INFO:tasks.workunit.client.1.smithi174.stdout:9/571: write d2/d24/d2e/d7e/f8f [384671,97278] 0 2024-03-21T10:08:59.446 INFO:tasks.workunit.client.0.smithi153.stdout:5/688: dwrite d4/d8/d5a/d5d/dd9/d2f/f41 [4194304,4194304] 0 2024-03-21T10:08:59.446 INFO:tasks.workunit.client.0.smithi153.stdout:7/654: creat d4/d14/dd0/d83/fdf x:0 0 0 2024-03-21T10:08:59.446 INFO:tasks.workunit.client.1.smithi174.stdout:1/619: creat d4/d7/d33/d43/d77/fd9 x:0 0 0 2024-03-21T10:08:59.447 INFO:tasks.workunit.client.1.smithi174.stdout:5/592: rmdir d1/d2f/d37/d9b 39 2024-03-21T10:08:59.447 INFO:tasks.workunit.client.1.smithi174.stdout:1/620: stat d4/d7/l14 0 2024-03-21T10:08:59.453 INFO:tasks.workunit.client.0.smithi153.stdout:7/655: chown d4/d14/dd0/d83/fdf 3717 1 2024-03-21T10:08:59.454 INFO:tasks.workunit.client.1.smithi174.stdout:6/412: creat d0/d3/d5/d6/d42/f82 x:0 0 0 2024-03-21T10:08:59.459 INFO:tasks.workunit.client.1.smithi174.stdout:0/539: link d9/d21/d31/d3c/lab d9/d1c/d76/d87/lbc 0 2024-03-21T10:08:59.459 INFO:tasks.workunit.client.1.smithi174.stdout:0/540: fdatasync d9/f6d 0 2024-03-21T10:08:59.462 INFO:tasks.workunit.client.0.smithi153.stdout:6/891: fsync d1f/d2b/d10a/d40/d4f/f119 0 2024-03-21T10:08:59.462 INFO:tasks.workunit.client.0.smithi153.stdout:9/750: mkdir d1/d3b/df8/dfa 0 2024-03-21T10:08:59.466 INFO:tasks.workunit.client.0.smithi153.stdout:8/678: dwrite d5/d9/d43/dd/d2d/d45/fbd [0,4194304] 0 2024-03-21T10:08:59.466 INFO:tasks.workunit.client.0.smithi153.stdout:5/689: fdatasync d4/d5/f1a 0 2024-03-21T10:08:59.467 INFO:tasks.workunit.client.1.smithi174.stdout:9/572: creat d2/d11/d1d/d2a/fcb x:0 0 0 2024-03-21T10:08:59.470 INFO:tasks.workunit.client.1.smithi174.stdout:0/541: mkdir d9/d35/d4a/d67/d89/d99/db2/dbd 0 2024-03-21T10:08:59.470 INFO:tasks.workunit.client.0.smithi153.stdout:7/656: rename d4/d14/d16/c18 to d4/d14/ce0 0 2024-03-21T10:08:59.471 INFO:tasks.workunit.client.1.smithi174.stdout:0/542: stat d9/d21/d31/d5c 0 2024-03-21T10:08:59.472 INFO:tasks.workunit.client.1.smithi174.stdout:5/593: rename d1/d49/d4f/l70 to d1/d2f/lda 0 2024-03-21T10:08:59.473 INFO:tasks.workunit.client.0.smithi153.stdout:9/751: symlink d1/d43/d72/dba/lfb 0 2024-03-21T10:08:59.474 INFO:tasks.workunit.client.0.smithi153.stdout:5/690: chown d4/d8/d5a/d5d/dd9/d2f/f41 644932 1 2024-03-21T10:08:59.475 INFO:tasks.workunit.client.1.smithi174.stdout:9/573: creat d2/d24/d2e/dac/fcc x:0 0 0 2024-03-21T10:08:59.480 INFO:tasks.workunit.client.0.smithi153.stdout:0/655: write d4/d4c/d83/d22/f47 [224633,127138] 0 2024-03-21T10:08:59.481 INFO:tasks.workunit.client.1.smithi174.stdout:5/594: truncate d1/d5c/d73/f7f 1272890 0 2024-03-21T10:08:59.481 INFO:tasks.workunit.client.1.smithi174.stdout:1/621: rename d4/d7/d33/d43/c88 to d4/d7/d33/d43/d51/cda 0 2024-03-21T10:08:59.481 INFO:tasks.workunit.client.1.smithi174.stdout:9/574: chown d2/d11/d1d/dab/dc1 5 1 2024-03-21T10:08:59.481 INFO:tasks.workunit.client.1.smithi174.stdout:4/615: dwrite d4/d21/d24/d28/d38/f54 [0,4194304] 0 2024-03-21T10:08:59.482 INFO:tasks.workunit.client.0.smithi153.stdout:4/733: write d11/d18/f49 [90309,120430] 0 2024-03-21T10:08:59.483 INFO:tasks.workunit.client.0.smithi153.stdout:0/656: write d4/d4c/d83/f20 [4351722,122066] 0 2024-03-21T10:08:59.484 INFO:tasks.workunit.client.1.smithi174.stdout:1/622: fsync d4/d7/d33/f36 0 2024-03-21T10:08:59.485 INFO:tasks.workunit.client.1.smithi174.stdout:4/616: write d4/d19/f74 [5101024,103344] 0 2024-03-21T10:08:59.486 INFO:tasks.workunit.client.0.smithi153.stdout:7/657: mkdir d4/d2a/d3b/d60/de1 0 2024-03-21T10:08:59.487 INFO:tasks.workunit.client.1.smithi174.stdout:7/462: dread d0/d4/d10/f45 [4194304,4194304] 0 2024-03-21T10:08:59.492 INFO:tasks.workunit.client.0.smithi153.stdout:8/679: rename f3 to d5/d9/d43/dd/d2d/d45/d88/d85/fe2 0 2024-03-21T10:08:59.492 INFO:tasks.workunit.client.1.smithi174.stdout:0/543: stat d9/d21/d31/d9f/c68 0 2024-03-21T10:08:59.492 INFO:tasks.workunit.client.1.smithi174.stdout:8/467: write d9/d2a/d39/d76/f42 [3117082,92291] 0 2024-03-21T10:08:59.495 INFO:tasks.workunit.client.1.smithi174.stdout:1/623: mknod d4/d7/d68/d8e/db8/cdb 0 2024-03-21T10:08:59.495 INFO:tasks.workunit.client.0.smithi153.stdout:9/752: symlink d1/d43/d44/d46/d9e/d2e/d67/lfc 0 2024-03-21T10:08:59.497 INFO:tasks.workunit.client.0.smithi153.stdout:5/691: fsync d4/d5/df/d5e/f82 0 2024-03-21T10:08:59.497 INFO:tasks.workunit.client.0.smithi153.stdout:1/568: link f5 d6/d3a/d66/faf 0 2024-03-21T10:08:59.501 INFO:tasks.workunit.client.1.smithi174.stdout:5/595: creat d1/dca/fdb x:0 0 0 2024-03-21T10:08:59.502 INFO:tasks.workunit.client.0.smithi153.stdout:2/787: dwrite dd/f26 [0,4194304] 0 2024-03-21T10:08:59.503 INFO:tasks.workunit.client.1.smithi174.stdout:5/596: write d1/d16/f5f [4677494,2363] 0 2024-03-21T10:08:59.504 INFO:tasks.workunit.client.1.smithi174.stdout:4/617: mknod d4/d21/d31/d6a/cc4 0 2024-03-21T10:08:59.509 INFO:tasks.workunit.client.1.smithi174.stdout:2/550: dread d5/d25/d27/d2d/d45/d5b/d55/f69 [0,4194304] 0 2024-03-21T10:08:59.511 INFO:tasks.workunit.client.0.smithi153.stdout:3/781: truncate dd/d1e/d2f/d35/d68/d75/ddd/fe5 2675386 0 2024-03-21T10:08:59.512 INFO:tasks.workunit.client.1.smithi174.stdout:8/468: symlink d9/d2a/d39/d76/d19/l9b 0 2024-03-21T10:08:59.512 INFO:tasks.workunit.client.0.smithi153.stdout:4/734: creat d11/d9d/fea x:0 0 0 2024-03-21T10:08:59.513 INFO:tasks.workunit.client.0.smithi153.stdout:0/657: creat d4/d16/d19/dad/fca x:0 0 0 2024-03-21T10:08:59.514 INFO:tasks.workunit.client.1.smithi174.stdout:8/469: read - d9/d2a/d39/d76/f8d zero size 2024-03-21T10:08:59.518 INFO:tasks.workunit.client.1.smithi174.stdout:6/413: dread d0/d3/d5/d3b/f51 [0,4194304] 0 2024-03-21T10:08:59.519 INFO:tasks.workunit.client.1.smithi174.stdout:8/470: chown d9/d2a/d39/d76/d19/d27/l90 28280533 1 2024-03-21T10:08:59.521 INFO:tasks.workunit.client.1.smithi174.stdout:1/624: rename d4/d7/d15/d53/d78/dc7 to d4/d7/d68/d7b/d8b/ddc 0 2024-03-21T10:08:59.523 INFO:tasks.workunit.client.1.smithi174.stdout:7/463: link d0/d4/f12 d0/d9e/fa0 0 2024-03-21T10:08:59.523 INFO:tasks.workunit.client.0.smithi153.stdout:2/788: truncate dd/d15/d3c/fcb 826591 0 2024-03-21T10:08:59.523 INFO:tasks.workunit.client.1.smithi174.stdout:5/597: creat d1/d16/d2e/d66/fdc x:0 0 0 2024-03-21T10:08:59.524 INFO:tasks.workunit.client.1.smithi174.stdout:2/551: creat d5/d25/d27/d2d/d45/d5b/d2f/d4b/fbe x:0 0 0 2024-03-21T10:08:59.525 INFO:tasks.workunit.client.1.smithi174.stdout:1/625: chown d4/d7/f8f 16698054 1 2024-03-21T10:08:59.526 INFO:tasks.workunit.client.0.smithi153.stdout:0/658: stat d4/d16/l97 0 2024-03-21T10:08:59.526 INFO:tasks.workunit.client.1.smithi174.stdout:9/575: link d2/d11/d1d/dab/dc1/l9e d2/d11/lcd 0 2024-03-21T10:08:59.528 INFO:tasks.workunit.client.1.smithi174.stdout:7/464: rmdir d0/d4 39 2024-03-21T10:08:59.530 INFO:tasks.workunit.client.0.smithi153.stdout:6/892: getdents d1f/d2b/d10a/d40 0 2024-03-21T10:08:59.530 INFO:tasks.workunit.client.1.smithi174.stdout:1/626: write d4/d7/d33/fac [1540386,93554] 0 2024-03-21T10:08:59.531 INFO:tasks.workunit.client.0.smithi153.stdout:1/569: creat d6/d3a/fb0 x:0 0 0 2024-03-21T10:08:59.532 INFO:tasks.workunit.client.1.smithi174.stdout:1/627: dread - d4/d7/d28/fad zero size 2024-03-21T10:08:59.535 INFO:tasks.workunit.client.1.smithi174.stdout:5/598: creat d1/d49/fdd x:0 0 0 2024-03-21T10:08:59.535 INFO:tasks.workunit.client.1.smithi174.stdout:9/576: fdatasync d2/d24/d2e/d63/fad 0 2024-03-21T10:08:59.539 INFO:tasks.workunit.client.0.smithi153.stdout:0/659: mknod d4/d16/d19/d30/da7/ccb 0 2024-03-21T10:08:59.542 INFO:tasks.workunit.client.1.smithi174.stdout:8/471: link d9/d2a/l93 d9/d2a/d39/d76/d19/d27/d60/l9c 0 2024-03-21T10:08:59.544 INFO:tasks.workunit.client.1.smithi174.stdout:7/465: write d0/d4/d38/f66 [2980989,35359] 0 2024-03-21T10:08:59.545 INFO:tasks.workunit.client.0.smithi153.stdout:1/570: write d6/d3a/d66/f68 [4820444,52226] 0 2024-03-21T10:08:59.546 INFO:tasks.workunit.client.1.smithi174.stdout:5/599: unlink d1/d16/d2e/d8d/d99/fc4 0 2024-03-21T10:08:59.547 INFO:tasks.workunit.client.0.smithi153.stdout:7/658: dread d4/d14/dd0/d20/f93 [0,4194304] 0 2024-03-21T10:08:59.547 INFO:tasks.workunit.client.1.smithi174.stdout:7/466: write d0/d4/d10/d1b/f3b [375383,26854] 0 2024-03-21T10:08:59.550 INFO:tasks.workunit.client.1.smithi174.stdout:5/600: fsync d1/d49/f53 0 2024-03-21T10:08:59.551 INFO:tasks.workunit.client.0.smithi153.stdout:3/782: dwrite dd/d1e/d2f/d35/daf/dcc/d58/fee [0,4194304] 0 2024-03-21T10:08:59.554 INFO:tasks.workunit.client.1.smithi174.stdout:5/601: write d1/d16/d2e/d66/d8b/f48 [320276,40436] 0 2024-03-21T10:08:59.556 INFO:tasks.workunit.client.0.smithi153.stdout:1/571: truncate d6/d10/da0/fa1 699306 0 2024-03-21T10:08:59.566 INFO:tasks.workunit.client.0.smithi153.stdout:6/893: truncate d1f/d2b/d10a/d40/d51/d98/f120 143527 0 2024-03-21T10:08:59.570 INFO:tasks.workunit.client.0.smithi153.stdout:2/789: write dd/d15/d3c/fcf [236301,102212] 0 2024-03-21T10:08:59.570 INFO:tasks.workunit.client.0.smithi153.stdout:4/735: dread d11/d18/f1d [0,4194304] 0 2024-03-21T10:08:59.570 INFO:tasks.workunit.client.1.smithi174.stdout:2/552: dread d5/d25/d27/d2e/d4f/d65/f92 [0,4194304] 0 2024-03-21T10:08:59.570 INFO:tasks.workunit.client.1.smithi174.stdout:4/618: write d4/fa [2239801,87300] 0 2024-03-21T10:08:59.571 INFO:tasks.workunit.client.1.smithi174.stdout:7/467: fsync d0/d53/f8d 0 2024-03-21T10:08:59.576 INFO:tasks.workunit.client.1.smithi174.stdout:3/559: sync 2024-03-21T10:08:59.576 INFO:tasks.workunit.client.1.smithi174.stdout:7/468: chown d0/d4/c6b 1 1 2024-03-21T10:08:59.579 INFO:tasks.workunit.client.0.smithi153.stdout:9/753: dwrite d1/d4/d5f/fe6 [0,4194304] 0 2024-03-21T10:08:59.579 INFO:tasks.workunit.client.1.smithi174.stdout:0/544: dwrite d9/d21/d31/f62 [4194304,4194304] 0 2024-03-21T10:08:59.580 INFO:tasks.workunit.client.1.smithi174.stdout:6/414: dread d0/d3/d5/f1c [0,4194304] 0 2024-03-21T10:08:59.581 INFO:tasks.workunit.client.1.smithi174.stdout:2/553: truncate d5/d25/d27/d2e/d7b/d81/fba 368556 0 2024-03-21T10:08:59.582 INFO:tasks.workunit.client.1.smithi174.stdout:5/602: rename d1/d2f/d37/f9e to d1/d16/d2e/d66/dbd/fde 0 2024-03-21T10:08:59.584 INFO:tasks.workunit.client.0.smithi153.stdout:8/680: dwrite d5/d9/d43/dd/d20/d26/d24/f32 [0,4194304] 0 2024-03-21T10:08:59.584 INFO:tasks.workunit.client.1.smithi174.stdout:9/577: creat d2/fce x:0 0 0 2024-03-21T10:08:59.585 INFO:tasks.workunit.client.1.smithi174.stdout:6/415: chown d0/d3/d5/fe 145 1 2024-03-21T10:08:59.586 INFO:tasks.workunit.client.0.smithi153.stdout:1/572: truncate d6/d3a/d3c/d50/d51/f8f 776000 0 2024-03-21T10:08:59.587 INFO:tasks.workunit.client.0.smithi153.stdout:3/783: link dd/d1e/d2f/d35/daf/dcc/d58/fde dd/d1e/d3d/d94/d73/f10b 0 2024-03-21T10:08:59.588 INFO:tasks.workunit.client.0.smithi153.stdout:5/692: dwrite d4/d8/d3b/d56/fca [0,4194304] 0 2024-03-21T10:08:59.590 INFO:tasks.workunit.client.1.smithi174.stdout:2/554: write d5/d25/d27/d2d/d45/d5b/f1c [3822346,92723] 0 2024-03-21T10:08:59.591 INFO:tasks.workunit.client.0.smithi153.stdout:9/754: symlink d1/d4/lfd 0 2024-03-21T10:08:59.592 INFO:tasks.workunit.client.1.smithi174.stdout:8/472: write d9/d2a/f51 [209328,126403] 0 2024-03-21T10:08:59.596 INFO:tasks.workunit.client.1.smithi174.stdout:4/619: fsync d4/d21/d24/d28/f5e 0 2024-03-21T10:08:59.600 INFO:tasks.workunit.client.0.smithi153.stdout:8/681: fdatasync d5/fc9 0 2024-03-21T10:08:59.602 INFO:tasks.workunit.client.1.smithi174.stdout:3/560: mkdir d0/d12/d14/d1a/d27/d4f/dbe 0 2024-03-21T10:08:59.603 INFO:tasks.workunit.client.1.smithi174.stdout:9/578: mknod d2/d11/d1d/dab/ccf 0 2024-03-21T10:08:59.605 INFO:tasks.workunit.client.0.smithi153.stdout:1/573: symlink d6/lb1 0 2024-03-21T10:08:59.605 INFO:tasks.workunit.client.1.smithi174.stdout:4/620: rmdir d4/d21/d31/d3a/d77 39 2024-03-21T10:08:59.607 INFO:tasks.workunit.client.1.smithi174.stdout:0/545: creat d9/d21/d31/d5c/d86/d9b/fbe x:0 0 0 2024-03-21T10:08:59.610 INFO:tasks.workunit.client.1.smithi174.stdout:9/579: creat d2/d11/d7d/fd0 x:0 0 0 2024-03-21T10:08:59.612 INFO:tasks.workunit.client.1.smithi174.stdout:8/473: read d9/f30 [3666462,80046] 0 2024-03-21T10:08:59.613 INFO:tasks.workunit.client.0.smithi153.stdout:0/660: dwrite d4/f37 [4194304,4194304] 0 2024-03-21T10:08:59.616 INFO:tasks.workunit.client.1.smithi174.stdout:4/621: write d4/d21/d31/d6a/d8f/f94 [839511,59107] 0 2024-03-21T10:08:59.618 INFO:tasks.workunit.client.0.smithi153.stdout:2/790: rename dd/d15/d18/d2c/f39 to dd/d15/d18/d2c/d4b/d56/d63/d8d/f110 0 2024-03-21T10:08:59.618 INFO:tasks.workunit.client.1.smithi174.stdout:9/580: chown d2/d11/d1d/d2a/f73 1 1 2024-03-21T10:08:59.619 INFO:tasks.workunit.client.1.smithi174.stdout:1/628: dwrite d4/d67/f74 [0,4194304] 0 2024-03-21T10:08:59.621 INFO:tasks.workunit.client.1.smithi174.stdout:3/561: symlink d0/d12/d14/d1a/d27/d3c/d73/db7/lbf 0 2024-03-21T10:08:59.622 INFO:tasks.workunit.client.1.smithi174.stdout:0/546: write d9/d21/d75/da1/fb8 [177782,49] 0 2024-03-21T10:08:59.623 INFO:tasks.workunit.client.0.smithi153.stdout:5/693: truncate d4/d5/f1a 2545505 0 2024-03-21T10:08:59.626 INFO:tasks.workunit.client.0.smithi153.stdout:9/755: truncate d1/d43/d44/f5d 8101908 0 2024-03-21T10:08:59.627 INFO:tasks.workunit.client.1.smithi174.stdout:6/416: dwrite d0/d3/d5/df/d16/f27 [0,4194304] 0 2024-03-21T10:08:59.629 INFO:tasks.workunit.client.1.smithi174.stdout:9/581: rmdir d2/d24 39 2024-03-21T10:08:59.629 INFO:tasks.workunit.client.0.smithi153.stdout:7/659: dwrite d4/d2a/d40/d50/fca [0,4194304] 0 2024-03-21T10:08:59.631 INFO:tasks.workunit.client.0.smithi153.stdout:0/661: symlink d4/d16/d27/lcc 0 2024-03-21T10:08:59.631 INFO:tasks.workunit.client.1.smithi174.stdout:0/547: write d9/f1d [6396345,123783] 0 2024-03-21T10:08:59.635 INFO:tasks.workunit.client.0.smithi153.stdout:6/894: dwrite d1f/d93/dd8/f74 [0,4194304] 0 2024-03-21T10:08:59.636 INFO:tasks.workunit.client.0.smithi153.stdout:2/791: symlink dd/d15/d18/d3a/d3d/l111 0 2024-03-21T10:08:59.641 INFO:tasks.workunit.client.1.smithi174.stdout:8/474: dwrite d9/d2a/d39/f47 [0,4194304] 0 2024-03-21T10:08:59.642 INFO:tasks.workunit.client.1.smithi174.stdout:1/629: rename d4/d7/d33/d43/d51/caa to d4/d7/d32/da2/cdd 0 2024-03-21T10:08:59.642 INFO:tasks.workunit.client.1.smithi174.stdout:0/548: creat d9/d21/d31/d5c/d86/fbf x:0 0 0 2024-03-21T10:08:59.643 INFO:tasks.workunit.client.0.smithi153.stdout:1/574: mknod d6/d3a/d3c/d50/da8/cb2 0 2024-03-21T10:08:59.645 INFO:tasks.workunit.client.1.smithi174.stdout:4/622: symlink d4/d21/d31/d3a/d67/d76/d82/db1/lc5 0 2024-03-21T10:08:59.648 INFO:tasks.workunit.client.0.smithi153.stdout:7/660: chown d4/d2a/c3d 1 1 2024-03-21T10:08:59.648 INFO:tasks.workunit.client.0.smithi153.stdout:5/694: creat d4/d8/d5a/d5d/dd9/d2f/fe4 x:0 0 0 2024-03-21T10:08:59.648 INFO:tasks.workunit.client.0.smithi153.stdout:6/895: chown d1f/d2b/d10a/d40/d65 2278 1 2024-03-21T10:08:59.649 INFO:tasks.workunit.client.1.smithi174.stdout:8/475: fsync d9/d2a/d39/d76/f8d 0 2024-03-21T10:08:59.654 INFO:tasks.workunit.client.1.smithi174.stdout:1/630: creat d4/d67/fde x:0 0 0 2024-03-21T10:08:59.655 INFO:tasks.workunit.client.0.smithi153.stdout:0/662: mkdir d4/d16/d27/dc5/dcd 0 2024-03-21T10:08:59.656 INFO:tasks.workunit.client.1.smithi174.stdout:4/623: mknod d4/d21/d24/d28/d79/cc6 0 2024-03-21T10:08:59.656 INFO:tasks.workunit.client.1.smithi174.stdout:1/631: truncate d4/d7/d33/d43/d77/d87/fd3 108129 0 2024-03-21T10:08:59.661 INFO:tasks.workunit.client.0.smithi153.stdout:7/661: creat d4/d19/d4d/dd5/fe2 x:0 0 0 2024-03-21T10:08:59.661 INFO:tasks.workunit.client.0.smithi153.stdout:5/695: fdatasync d4/d5/df/d67/d86/d96/fb8 0 2024-03-21T10:08:59.662 INFO:tasks.workunit.client.1.smithi174.stdout:8/476: rename d9/fe to d9/d53/f9d 0 2024-03-21T10:08:59.664 INFO:tasks.workunit.client.0.smithi153.stdout:3/784: dwrite dd/d1e/d3d/d94/fdb [0,4194304] 0 2024-03-21T10:08:59.665 INFO:tasks.workunit.client.0.smithi153.stdout:5/696: readlink d4/d8/d5a/d5d/dd9/l20 0 2024-03-21T10:08:59.665 INFO:tasks.workunit.client.0.smithi153.stdout:0/663: rmdir d4/d4c 39 2024-03-21T10:08:59.669 INFO:tasks.workunit.client.1.smithi174.stdout:8/477: truncate d9/d2a/d39/f75 369686 0 2024-03-21T10:08:59.671 INFO:tasks.workunit.client.0.smithi153.stdout:1/575: creat d6/fb3 x:0 0 0 2024-03-21T10:08:59.671 INFO:tasks.workunit.client.1.smithi174.stdout:1/632: dread d4/d2d/fae [0,4194304] 0 2024-03-21T10:08:59.673 INFO:tasks.workunit.client.0.smithi153.stdout:1/576: chown d6/d13/d75/fae 2 1 2024-03-21T10:08:59.677 INFO:tasks.workunit.client.0.smithi153.stdout:2/792: rename dd/d15/d18/d2c/d33/d98/cd3 to dd/d15/d18/d3a/d3d/ddc/c112 0 2024-03-21T10:08:59.677 INFO:tasks.workunit.client.1.smithi174.stdout:0/549: dread d9/d21/d31/d3c/d72/f7e [0,4194304] 0 2024-03-21T10:08:59.680 INFO:tasks.workunit.client.0.smithi153.stdout:9/756: symlink d1/d43/d44/d46/d9e/d24/lfe 0 2024-03-21T10:08:59.683 INFO:tasks.workunit.client.1.smithi174.stdout:4/624: dwrite d4/d21/d24/fb5 [0,4194304] 0 2024-03-21T10:08:59.683 INFO:tasks.workunit.client.1.smithi174.stdout:1/633: symlink d4/d7/d68/d7b/ldf 0 2024-03-21T10:08:59.683 INFO:tasks.workunit.client.1.smithi174.stdout:8/478: chown d9/l1d 338667 1 2024-03-21T10:08:59.684 INFO:tasks.workunit.client.0.smithi153.stdout:9/757: readlink d1/d43/d44/l2b 0 2024-03-21T10:08:59.687 INFO:tasks.workunit.client.1.smithi174.stdout:8/479: readlink d9/d2a/d39/d76/d19/d27/d60/l86 0 2024-03-21T10:08:59.688 INFO:tasks.workunit.client.1.smithi174.stdout:4/625: truncate d4/d21/d24/d28/f69 5073434 0 2024-03-21T10:08:59.689 INFO:tasks.workunit.client.0.smithi153.stdout:7/662: dwrite d4/d2a/f3e [0,4194304] 0 2024-03-21T10:08:59.689 INFO:tasks.workunit.client.1.smithi174.stdout:0/550: mknod d9/d21/d31/d9f/cc0 0 2024-03-21T10:08:59.696 INFO:tasks.workunit.client.0.smithi153.stdout:7/663: write d4/d19/d4d/dd5/fe2 [151970,125186] 0 2024-03-21T10:08:59.696 INFO:tasks.workunit.client.1.smithi174.stdout:8/480: write d9/d2a/d39/d76/d19/d31/f4f [4676254,6947] 0 2024-03-21T10:08:59.699 INFO:tasks.workunit.client.1.smithi174.stdout:1/634: mkdir d4/d7/d68/d7b/de0 0 2024-03-21T10:08:59.700 INFO:tasks.workunit.client.0.smithi153.stdout:7/664: chown d4/d14/d2f/cd1 1147 1 2024-03-21T10:08:59.707 INFO:tasks.workunit.client.0.smithi153.stdout:1/577: rmdir d6/d10/d35/da5/d4f 39 2024-03-21T10:08:59.710 INFO:tasks.workunit.client.1.smithi174.stdout:0/551: creat d9/fc1 x:0 0 0 2024-03-21T10:08:59.710 INFO:tasks.workunit.client.0.smithi153.stdout:0/664: creat d4/d16/d19/d30/d6c/d98/d5e/db2/fce x:0 0 0 2024-03-21T10:08:59.714 INFO:tasks.workunit.client.0.smithi153.stdout:2/793: creat dd/d15/d18/d2c/d4b/d56/d63/f113 x:0 0 0 2024-03-21T10:08:59.715 INFO:tasks.workunit.client.1.smithi174.stdout:8/481: mknod d9/d2a/d39/d76/d19/c9e 0 2024-03-21T10:08:59.718 INFO:tasks.workunit.client.0.smithi153.stdout:6/896: link d1f/d99/fe7 d1f/d93/dd8/f142 0 2024-03-21T10:08:59.725 INFO:tasks.workunit.client.0.smithi153.stdout:8/682: dwrite d5/fc9 [0,4194304] 0 2024-03-21T10:08:59.727 INFO:tasks.workunit.client.1.smithi174.stdout:5/603: dread d1/d49/d4f/d51/f56 [0,4194304] 0 2024-03-21T10:08:59.727 INFO:tasks.workunit.client.1.smithi174.stdout:1/635: link d4/d7/d15/d53/fb7 d4/d2d/fe1 0 2024-03-21T10:08:59.728 INFO:tasks.workunit.client.0.smithi153.stdout:4/736: dread d11/f44 [0,4194304] 0 2024-03-21T10:08:59.729 INFO:tasks.workunit.client.0.smithi153.stdout:7/665: creat d4/d14/dd0/d20/d9a/dd2/db9/fe3 x:0 0 0 2024-03-21T10:08:59.732 INFO:tasks.workunit.client.1.smithi174.stdout:8/482: creat d9/d2a/d39/d76/d19/d27/d60/f9f x:0 0 0 2024-03-21T10:08:59.733 INFO:tasks.workunit.client.1.smithi174.stdout:7/469: dread d0/d4/f7e [0,4194304] 0 2024-03-21T10:08:59.733 INFO:tasks.workunit.client.0.smithi153.stdout:7/666: write d4/d14/d2f/d78/f29 [2728196,80637] 0 2024-03-21T10:08:59.735 INFO:tasks.workunit.client.0.smithi153.stdout:1/578: chown d6/fc 6203 1 2024-03-21T10:08:59.736 INFO:tasks.workunit.client.0.smithi153.stdout:0/665: creat d4/d16/d19/d30/d6c/fcf x:0 0 0 2024-03-21T10:08:59.738 INFO:tasks.workunit.client.0.smithi153.stdout:8/683: dread d5/d9/d43/dd/d20/d26/d24/d52/f72 [0,4194304] 0 2024-03-21T10:08:59.739 INFO:tasks.workunit.client.0.smithi153.stdout:6/897: rmdir d1f/d2b/d10a/d40/d4f 39 2024-03-21T10:08:59.740 INFO:tasks.workunit.client.1.smithi174.stdout:1/636: dread d4/d7/d15/d53/d78/fab [0,4194304] 0 2024-03-21T10:08:59.743 INFO:tasks.workunit.client.0.smithi153.stdout:2/794: unlink dd/d15/d18/d2c/d4b/d56/d63/d8d/d9b/dc3/df5/dae/f109 0 2024-03-21T10:08:59.743 INFO:tasks.workunit.client.1.smithi174.stdout:4/626: link d4/d21/d24/d28/d38/f88 d4/d21/d31/d3a/d7a/d9d/fc7 0 2024-03-21T10:08:59.744 INFO:tasks.workunit.client.1.smithi174.stdout:5/604: mknod d1/d5c/d73/cdf 0 2024-03-21T10:08:59.746 INFO:tasks.workunit.client.0.smithi153.stdout:9/758: link d1/d43/d44/f58 d1/d43/d44/fff 0 2024-03-21T10:08:59.746 INFO:tasks.workunit.client.1.smithi174.stdout:1/637: dread - d4/d7/d68/d89/fb3 zero size 2024-03-21T10:08:59.746 INFO:tasks.workunit.client.1.smithi174.stdout:9/582: dread d2/d24/d2e/f58 [0,4194304] 0 2024-03-21T10:08:59.747 INFO:tasks.workunit.client.0.smithi153.stdout:2/795: chown dd/d15/d18/d2c/cec 5992 1 2024-03-21T10:08:59.749 INFO:tasks.workunit.client.1.smithi174.stdout:7/470: rmdir d0/d4/d38 39 2024-03-21T10:08:59.749 INFO:tasks.workunit.client.1.smithi174.stdout:3/562: dread d0/d12/d14/d1a/d27/d3c/d73/db7/f1b [0,4194304] 0 2024-03-21T10:08:59.750 INFO:tasks.workunit.client.0.smithi153.stdout:3/785: write dd/f15 [749076,67578] 0 2024-03-21T10:08:59.751 INFO:tasks.workunit.client.0.smithi153.stdout:7/667: mknod d4/d2a/ce4 0 2024-03-21T10:08:59.753 INFO:tasks.workunit.client.0.smithi153.stdout:7/668: dread - d4/fd6 zero size 2024-03-21T10:08:59.754 INFO:tasks.workunit.client.0.smithi153.stdout:1/579: mkdir d6/d3a/d3c/d50/da8/db4 0 2024-03-21T10:08:59.757 INFO:tasks.workunit.client.1.smithi174.stdout:4/627: creat d4/d21/fc8 x:0 0 0 2024-03-21T10:08:59.759 INFO:tasks.workunit.client.0.smithi153.stdout:5/697: truncate d4/d8/f68 1461822 0 2024-03-21T10:08:59.759 INFO:tasks.workunit.client.0.smithi153.stdout:7/669: fdatasync d4/d14/d2f/d78/f29 0 2024-03-21T10:08:59.759 INFO:tasks.workunit.client.1.smithi174.stdout:9/583: fdatasync d2/d11/fb8 0 2024-03-21T10:08:59.761 INFO:tasks.workunit.client.0.smithi153.stdout:7/670: fdatasync d4/d2a/f2d 0 2024-03-21T10:08:59.762 INFO:tasks.workunit.client.1.smithi174.stdout:1/638: creat d4/d7/d68/d7b/d8b/fe2 x:0 0 0 2024-03-21T10:08:59.767 INFO:tasks.workunit.client.0.smithi153.stdout:2/796: mknod dd/d15/d18/d3a/d3d/ddc/c114 0 2024-03-21T10:08:59.767 INFO:tasks.workunit.client.0.smithi153.stdout:0/666: mkdir d4/dd0 0 2024-03-21T10:08:59.771 INFO:tasks.workunit.client.1.smithi174.stdout:5/605: symlink d1/d16/le0 0 2024-03-21T10:08:59.776 INFO:tasks.workunit.client.0.smithi153.stdout:1/580: read d6/d10/f11 [61192,29699] 0 2024-03-21T10:08:59.777 INFO:tasks.workunit.client.0.smithi153.stdout:5/698: creat d4/d5/df/d67/dbe/fe5 x:0 0 0 2024-03-21T10:08:59.779 INFO:tasks.workunit.client.0.smithi153.stdout:4/737: rename d11/d18/d50/f53 to d11/feb 0 2024-03-21T10:08:59.779 INFO:tasks.workunit.client.1.smithi174.stdout:8/483: link d9/d2a/d39/d76/f8b d9/d2a/d39/fa0 0 2024-03-21T10:08:59.780 INFO:tasks.workunit.client.1.smithi174.stdout:6/417: sync 2024-03-21T10:08:59.780 INFO:tasks.workunit.client.1.smithi174.stdout:2/555: sync 2024-03-21T10:08:59.783 INFO:tasks.workunit.client.0.smithi153.stdout:3/786: creat dd/d1e/d2f/d35/d68/daa/f10c x:0 0 0 2024-03-21T10:08:59.785 INFO:tasks.workunit.client.0.smithi153.stdout:2/797: symlink dd/d15/d18/d2c/d4b/d56/d63/d8d/d9b/dc3/df5/l115 0 2024-03-21T10:08:59.786 INFO:tasks.workunit.client.0.smithi153.stdout:0/667: mknod d4/d16/d27/dc5/cd1 0 2024-03-21T10:08:59.791 INFO:tasks.workunit.client.1.smithi174.stdout:4/628: rename d4/d19/f7c to d4/d21/d31/d3a/d67/d76/d82/db1/fc9 0 2024-03-21T10:08:59.794 INFO:tasks.workunit.client.0.smithi153.stdout:4/738: write d11/d18/d6d/f9b [915966,94987] 0 2024-03-21T10:08:59.794 INFO:tasks.workunit.client.0.smithi153.stdout:9/759: dwrite d1/d4/d5f/fc8 [0,4194304] 0 2024-03-21T10:08:59.795 INFO:tasks.workunit.client.1.smithi174.stdout:7/471: mknod d0/d9e/ca1 0 2024-03-21T10:08:59.799 INFO:tasks.workunit.client.0.smithi153.stdout:6/898: dread d1f/d2b/d10a/d40/d4f/f83 [0,4194304] 0 2024-03-21T10:08:59.799 INFO:tasks.workunit.client.0.smithi153.stdout:8/684: getdents d5/d9/d43/dd/d20/d26/d24 0 2024-03-21T10:08:59.800 INFO:tasks.workunit.client.1.smithi174.stdout:3/563: truncate d0/d12/d14/f3a 1064378 0 2024-03-21T10:08:59.800 INFO:tasks.workunit.client.1.smithi174.stdout:7/472: readlink d0/d4/l20 0 2024-03-21T10:08:59.800 INFO:tasks.workunit.client.1.smithi174.stdout:1/639: dread d4/d5/f18 [0,4194304] 0 2024-03-21T10:08:59.801 INFO:tasks.workunit.client.1.smithi174.stdout:1/640: fdatasync d4/d5/d6c/fc0 0 2024-03-21T10:08:59.801 INFO:tasks.workunit.client.1.smithi174.stdout:3/564: chown d0/d12/d14/d1a/d27/d3c/l58 87 1 2024-03-21T10:08:59.802 INFO:tasks.workunit.client.0.smithi153.stdout:1/581: mkdir d6/d13/d75/db5 0 2024-03-21T10:08:59.804 INFO:tasks.workunit.client.1.smithi174.stdout:6/418: dwrite d0/d3/d5/d6/f6a [0,4194304] 0 2024-03-21T10:08:59.805 INFO:tasks.workunit.client.0.smithi153.stdout:8/685: truncate d5/d1b/fd1 381415 0 2024-03-21T10:08:59.806 INFO:tasks.workunit.client.0.smithi153.stdout:7/671: dwrite d4/d14/dd0/d20/d9a/dd2/db9/fe3 [0,4194304] 0 2024-03-21T10:08:59.808 INFO:tasks.workunit.client.0.smithi153.stdout:2/798: mknod dd/d15/d18/d2c/d33/deb/c116 0 2024-03-21T10:08:59.808 INFO:tasks.workunit.client.0.smithi153.stdout:1/582: dread - d6/d10/f97 zero size 2024-03-21T10:08:59.811 INFO:tasks.workunit.client.1.smithi174.stdout:3/565: dread - d0/d12/d14/da9/fb0 zero size 2024-03-21T10:08:59.813 INFO:tasks.workunit.client.0.smithi153.stdout:9/760: read - d1/d43/d72/fdd zero size 2024-03-21T10:08:59.820 INFO:tasks.workunit.client.1.smithi174.stdout:2/556: rename d5/d25/d27/d2d/d45/d5b/d2f/d4b/f54 to d5/d25/d27/d2d/d45/d5b/d55/fbf 0 2024-03-21T10:08:59.820 INFO:tasks.workunit.client.1.smithi174.stdout:4/629: creat d4/d21/d24/d28/d79/fca x:0 0 0 2024-03-21T10:08:59.821 INFO:tasks.workunit.client.0.smithi153.stdout:8/686: mknod d5/d9/d43/dd/d20/d26/d24/d52/ce3 0 2024-03-21T10:08:59.824 INFO:tasks.workunit.client.0.smithi153.stdout:0/668: mknod d4/d16/d19/d95/cd2 0 2024-03-21T10:08:59.825 INFO:tasks.workunit.client.1.smithi174.stdout:7/473: mkdir d0/d4/d10/d60/da2 0 2024-03-21T10:08:59.825 INFO:tasks.workunit.client.0.smithi153.stdout:2/799: chown dd/d15/d18/d2c/d4b/d56/d63/d8d/d9b/dc3/df5/l10a 498 1 2024-03-21T10:08:59.828 INFO:tasks.workunit.client.1.smithi174.stdout:9/584: write d2/d11/d60/f86 [1844562,5517] 0 2024-03-21T10:08:59.832 INFO:tasks.workunit.client.1.smithi174.stdout:8/484: symlink d9/d2a/la1 0 2024-03-21T10:08:59.833 INFO:tasks.workunit.client.1.smithi174.stdout:1/641: dwrite d4/d7/d15/f9b [0,4194304] 0 2024-03-21T10:08:59.834 INFO:tasks.workunit.client.0.smithi153.stdout:5/699: dwrite d4/d8/d5a/d5d/dd9/d44/de1/fd8 [0,4194304] 0 2024-03-21T10:08:59.837 INFO:tasks.workunit.client.0.smithi153.stdout:3/787: dread dd/f1f [0,4194304] 0 2024-03-21T10:08:59.837 INFO:tasks.workunit.client.1.smithi174.stdout:2/557: write d5/d25/d27/d2d/d45/d5b/d2f/d4b/f78 [3135034,82162] 0 2024-03-21T10:08:59.837 INFO:tasks.workunit.client.1.smithi174.stdout:7/474: stat d0/d4/d38/f67 0 2024-03-21T10:08:59.837 INFO:tasks.workunit.client.1.smithi174.stdout:7/475: fsync d0/d4/d10/d79/f9f 0 2024-03-21T10:08:59.838 INFO:tasks.workunit.client.0.smithi153.stdout:9/761: rename d1/d43/d72/dba to d1/d43/d44/d46/d100 0 2024-03-21T10:08:59.839 INFO:tasks.workunit.client.0.smithi153.stdout:4/739: mkdir d11/d18/d50/d65/de3/dec 0 2024-03-21T10:08:59.841 INFO:tasks.workunit.client.1.smithi174.stdout:8/485: chown d9/d2a/d39/d76/f42 257603243 1 2024-03-21T10:08:59.841 INFO:tasks.workunit.client.1.smithi174.stdout:2/558: readlink d5/d25/d27/d2d/d45/d5b/lc 0 2024-03-21T10:08:59.843 INFO:tasks.workunit.client.1.smithi174.stdout:3/566: creat d0/d12/d14/d1a/d44/d65/d7e/dac/fc0 x:0 0 0 2024-03-21T10:08:59.843 INFO:tasks.workunit.client.0.smithi153.stdout:5/700: fsync d4/d8/d3b/d56/fca 0 2024-03-21T10:08:59.843 INFO:tasks.workunit.client.0.smithi153.stdout:9/762: dread - d1/d43/d44/d46/d9e/d24/fb4 zero size 2024-03-21T10:08:59.845 INFO:tasks.workunit.client.1.smithi174.stdout:7/476: chown d0/d4/d10/f1c 923 1 2024-03-21T10:08:59.845 INFO:tasks.workunit.client.0.smithi153.stdout:2/800: creat dd/d15/d18/d2c/d33/d98/dc7/f117 x:0 0 0 2024-03-21T10:08:59.848 INFO:tasks.workunit.client.1.smithi174.stdout:8/486: write d9/d2a/d4d/f6d [809316,57191] 0 2024-03-21T10:08:59.848 INFO:tasks.workunit.client.0.smithi153.stdout:3/788: symlink dd/d41/dab/l10d 0 2024-03-21T10:08:59.849 INFO:tasks.workunit.client.1.smithi174.stdout:5/606: getdents d1/d49 0 2024-03-21T10:08:59.850 INFO:tasks.workunit.client.0.smithi153.stdout:3/789: readlink dd/d1e/d2f/d35/l40 0 2024-03-21T10:08:59.851 INFO:tasks.workunit.client.1.smithi174.stdout:0/552: dwrite d9/d1c/d76/d87/f8d [0,4194304] 0 2024-03-21T10:08:59.851 INFO:tasks.workunit.client.1.smithi174.stdout:4/630: dread d4/f10 [0,4194304] 0 2024-03-21T10:08:59.852 INFO:tasks.workunit.client.0.smithi153.stdout:2/801: write dd/d15/d18/d2c/d4b/d56/d63/d8d/d9b/dc3/df5/dae/f77 [606586,90651] 0 2024-03-21T10:08:59.853 INFO:tasks.workunit.client.0.smithi153.stdout:9/763: rmdir d1/d43/d44/d46/d9e/d61 39 2024-03-21T10:08:59.854 INFO:tasks.workunit.client.0.smithi153.stdout:7/672: getdents d4/d14/dd0/d83 0 2024-03-21T10:08:59.855 INFO:tasks.workunit.client.1.smithi174.stdout:1/642: fdatasync d4/d2d/fce 0 2024-03-21T10:08:59.856 INFO:tasks.workunit.client.1.smithi174.stdout:3/567: stat d0/d12/d14/d1a/d27/d3c/d73/db7/c4d 0 2024-03-21T10:08:59.857 INFO:tasks.workunit.client.0.smithi153.stdout:9/764: stat d1/d43/d44/d46/d9e/cc0 0 2024-03-21T10:08:59.862 INFO:tasks.workunit.client.1.smithi174.stdout:2/559: mknod d5/d25/d27/d2e/d7b/d81/cc0 0 2024-03-21T10:08:59.862 INFO:tasks.workunit.client.1.smithi174.stdout:5/607: truncate d1/d16/d2e/d66/d8b/d85/f90 1548453 0 2024-03-21T10:08:59.868 INFO:tasks.workunit.client.1.smithi174.stdout:2/560: write d5/d25/d27/d2d/d45/d5b/fb6 [784372,114907] 0 2024-03-21T10:08:59.873 INFO:tasks.workunit.client.0.smithi153.stdout:7/673: mknod d4/d14/dd0/d20/d9a/dd2/ce5 0 2024-03-21T10:08:59.876 INFO:tasks.workunit.client.0.smithi153.stdout:2/802: mknod dd/d15/d18/d2c/d33/d98/dc7/de1/d102/c118 0 2024-03-21T10:08:59.880 INFO:tasks.workunit.client.0.smithi153.stdout:5/701: dwrite d4/d8/d3b/f88 [4194304,4194304] 0 2024-03-21T10:08:59.883 INFO:tasks.workunit.client.1.smithi174.stdout:3/568: creat d0/d12/d14/d1a/d27/d4f/fc1 x:0 0 0 2024-03-21T10:08:59.885 INFO:tasks.workunit.client.0.smithi153.stdout:0/669: dwrite d4/db/f96 [0,4194304] 0 2024-03-21T10:08:59.889 INFO:tasks.workunit.client.1.smithi174.stdout:5/608: dread d1/d16/d1c/d81/f8a [0,4194304] 0 2024-03-21T10:08:59.890 INFO:tasks.workunit.client.1.smithi174.stdout:2/561: mknod d5/d25/d27/d2d/d79/cc1 0 2024-03-21T10:08:59.890 INFO:tasks.workunit.client.1.smithi174.stdout:7/477: dwrite d0/d53/d5f/f71 [0,4194304] 0 2024-03-21T10:08:59.890 INFO:tasks.workunit.client.0.smithi153.stdout:7/674: mkdir d4/d14/d2f/d78/de6 0 2024-03-21T10:08:59.892 INFO:tasks.workunit.client.0.smithi153.stdout:7/675: readlink d4/d11/l13 0 2024-03-21T10:08:59.892 INFO:tasks.workunit.client.1.smithi174.stdout:0/553: creat d9/d35/d4a/d67/d89/fc2 x:0 0 0 2024-03-21T10:08:59.895 INFO:tasks.workunit.client.0.smithi153.stdout:9/765: dwrite d1/d4/d5f/fe6 [0,4194304] 0 2024-03-21T10:08:59.896 INFO:tasks.workunit.client.0.smithi153.stdout:5/702: chown d4/d5/df/d67/d86/d96/cb5 203 1 2024-03-21T10:08:59.901 INFO:tasks.workunit.client.0.smithi153.stdout:8/687: write d5/d9/d43/dd/d20/d26/d8b/fa4 [938008,8216] 0 2024-03-21T10:08:59.903 INFO:tasks.workunit.client.0.smithi153.stdout:2/803: unlink dd/d15/d18/d2c/d4b/d56/d63/d8d/d9b/dc3/df5/dae/d50/d59/d5d/f94 0 2024-03-21T10:08:59.904 INFO:tasks.workunit.client.0.smithi153.stdout:5/703: write d4/d8/d5c/f73 [4454995,6305] 0 2024-03-21T10:08:59.905 INFO:tasks.workunit.client.1.smithi174.stdout:8/487: dwrite f5 [0,4194304] 0 2024-03-21T10:08:59.906 INFO:tasks.workunit.client.1.smithi174.stdout:3/569: rmdir d0/d12/d14/d1a/d5c 39 2024-03-21T10:08:59.914 INFO:tasks.workunit.client.1.smithi174.stdout:3/570: dread - d0/d12/d14/d1a/fbc zero size 2024-03-21T10:08:59.914 INFO:tasks.workunit.client.1.smithi174.stdout:5/609: write d1/d16/d2e/d66/f7d [114955,51296] 0 2024-03-21T10:08:59.915 INFO:tasks.workunit.client.0.smithi153.stdout:5/704: rename d4/d5/fcc to d4/d8/d5c/d90/fe6 0 2024-03-21T10:08:59.918 INFO:tasks.workunit.client.1.smithi174.stdout:5/610: write d1/d16/d2e/d66/d8b/f48 [3998471,85943] 0 2024-03-21T10:08:59.919 INFO:tasks.workunit.client.0.smithi153.stdout:1/583: dwrite d6/d3a/d3c/f43 [0,4194304] 0 2024-03-21T10:08:59.920 INFO:tasks.workunit.client.0.smithi153.stdout:0/670: read d4/d16/d27/f79 [889836,126365] 0 2024-03-21T10:08:59.920 INFO:tasks.workunit.client.1.smithi174.stdout:3/571: truncate d0/d12/d46/d59/f95 5158966 0 2024-03-21T10:08:59.921 INFO:tasks.workunit.client.1.smithi174.stdout:3/572: fsync d0/f70 0 2024-03-21T10:08:59.922 INFO:tasks.workunit.client.1.smithi174.stdout:0/554: symlink d9/d35/d4a/lc3 0 2024-03-21T10:08:59.926 INFO:tasks.workunit.client.0.smithi153.stdout:8/688: truncate d5/d9/d43/dd/d2d/f31 887790 0 2024-03-21T10:08:59.927 INFO:tasks.workunit.client.0.smithi153.stdout:5/705: mknod d4/d5/df/ce7 0 2024-03-21T10:08:59.931 INFO:tasks.workunit.client.0.smithi153.stdout:2/804: rename dd/d15/d18/d2c/d33/f36 to dd/d15/d18/d2c/d33/d98/dc7/f119 0 2024-03-21T10:08:59.933 INFO:tasks.workunit.client.0.smithi153.stdout:7/676: dwrite d4/d2a/d40/d50/f7b [4194304,4194304] 0 2024-03-21T10:08:59.934 INFO:tasks.workunit.client.0.smithi153.stdout:0/671: mknod d4/d16/d19/dad/cd3 0 2024-03-21T10:08:59.934 INFO:tasks.workunit.client.0.smithi153.stdout:5/706: fdatasync d4/d5/df/d67/d6a/fc3 0 2024-03-21T10:08:59.935 INFO:tasks.workunit.client.1.smithi174.stdout:1/643: getdents d4/d7/d68/d7b/d8b 0 2024-03-21T10:08:59.935 INFO:tasks.workunit.client.1.smithi174.stdout:7/478: rename d0/d4/f2d to d0/d4/d10/d44/d81/d9a/fa3 0 2024-03-21T10:08:59.937 INFO:tasks.workunit.client.1.smithi174.stdout:0/555: fdatasync d9/d21/d23/f91 0 2024-03-21T10:08:59.940 INFO:tasks.workunit.client.1.smithi174.stdout:7/479: fdatasync d0/d4/d10/d44/d81/f87 0 2024-03-21T10:08:59.948 INFO:tasks.workunit.client.1.smithi174.stdout:8/488: getdents d9/d2a/d39/d76/d19/d27 0 2024-03-21T10:08:59.949 INFO:tasks.workunit.client.1.smithi174.stdout:0/556: creat d9/d21/d31/d5c/d92/d98/fc4 x:0 0 0 2024-03-21T10:08:59.950 INFO:tasks.workunit.client.0.smithi153.stdout:2/805: dread - dd/d15/d3c/fe6 zero size 2024-03-21T10:08:59.950 INFO:tasks.workunit.client.0.smithi153.stdout:0/672: unlink d4/db/fa0 0 2024-03-21T10:08:59.952 INFO:tasks.workunit.client.0.smithi153.stdout:9/766: dread d1/d4/d5f/fce [0,4194304] 0 2024-03-21T10:08:59.953 INFO:tasks.workunit.client.1.smithi174.stdout:1/644: write d4/d7/d68/d8e/fa1 [3829817,55185] 0 2024-03-21T10:08:59.956 INFO:tasks.workunit.client.1.smithi174.stdout:6/419: sync 2024-03-21T10:08:59.959 INFO:tasks.workunit.client.0.smithi153.stdout:4/740: dwrite d11/d2d/d6b/fc6 [0,4194304] 0 2024-03-21T10:08:59.959 INFO:tasks.workunit.client.0.smithi153.stdout:3/790: dwrite dd/d1e/d3d/d94/d8e/fbe [0,4194304] 0 2024-03-21T10:08:59.959 INFO:tasks.workunit.client.0.smithi153.stdout:1/584: rename d6/d3a/da2 to d6/d10/d35/da5/d4f/d9d/db6 0 2024-03-21T10:08:59.959 INFO:tasks.workunit.client.0.smithi153.stdout:9/767: read - d1/d4e/d68/f81 zero size 2024-03-21T10:08:59.960 INFO:tasks.workunit.client.1.smithi174.stdout:2/562: dwrite d5/d25/d27/d2d/d45/d5b/d2f/d66/fbb [0,4194304] 0 2024-03-21T10:08:59.961 INFO:tasks.workunit.client.1.smithi174.stdout:5/611: rename d1/d49/d4f/d51/d8f/fb0 to d1/fe1 0 2024-03-21T10:08:59.963 INFO:tasks.workunit.client.1.smithi174.stdout:7/480: mkdir d0/d4/d10/da4 0 2024-03-21T10:08:59.965 INFO:tasks.workunit.client.0.smithi153.stdout:2/806: symlink dd/d15/d18/d3a/d3d/l11a 0 2024-03-21T10:08:59.967 INFO:tasks.workunit.client.1.smithi174.stdout:5/612: chown d1/d16/d2e/d66/d8b/c42 510 1 2024-03-21T10:08:59.970 INFO:tasks.workunit.client.0.smithi153.stdout:0/673: rename d4/d16/d19/f85 to d4/db/fd4 0 2024-03-21T10:08:59.970 INFO:tasks.workunit.client.1.smithi174.stdout:0/557: truncate d9/d35/f3e 11608375 0 2024-03-21T10:08:59.971 INFO:tasks.workunit.client.1.smithi174.stdout:2/563: symlink d5/d25/d27/d2e/d90/da9/lc2 0 2024-03-21T10:08:59.972 INFO:tasks.workunit.client.0.smithi153.stdout:4/741: mkdir d11/d2d/d57/ded 0 2024-03-21T10:08:59.972 INFO:tasks.workunit.client.1.smithi174.stdout:4/631: dread d4/d21/d24/d28/f5e [0,4194304] 0 2024-03-21T10:08:59.975 INFO:tasks.workunit.client.0.smithi153.stdout:1/585: truncate d6/d32/f74 680266 0 2024-03-21T10:08:59.977 INFO:tasks.workunit.client.1.smithi174.stdout:6/420: rename d0/d3/d5/d6/l1d to d0/d3/d5/df/d16/l83 0 2024-03-21T10:08:59.978 INFO:tasks.workunit.client.1.smithi174.stdout:7/481: symlink d0/d4/d10/d44/d81/d9a/la5 0 2024-03-21T10:08:59.978 INFO:tasks.workunit.client.1.smithi174.stdout:4/632: unlink d4/f7 0 2024-03-21T10:08:59.979 INFO:tasks.workunit.client.0.smithi153.stdout:7/677: getdents d4/d19/d4d 0 2024-03-21T10:08:59.981 INFO:tasks.workunit.client.0.smithi153.stdout:0/674: truncate d4/d16/d27/f90 835609 0 2024-03-21T10:08:59.981 INFO:tasks.workunit.client.0.smithi153.stdout:4/742: dread - d11/d18/d7e/d93/fb8 zero size 2024-03-21T10:08:59.984 INFO:tasks.workunit.client.0.smithi153.stdout:1/586: creat d6/d10/da0/fb7 x:0 0 0 2024-03-21T10:08:59.984 INFO:tasks.workunit.client.1.smithi174.stdout:0/558: write d9/d21/d31/d3c/f5b [4321582,85895] 0 2024-03-21T10:08:59.986 INFO:tasks.workunit.client.0.smithi153.stdout:8/689: dwrite d5/d9/d43/dd/d2d/d45/f70 [4194304,4194304] 0 2024-03-21T10:08:59.987 INFO:tasks.workunit.client.1.smithi174.stdout:7/482: creat d0/d4/d10/d1b/fa6 x:0 0 0 2024-03-21T10:08:59.991 INFO:tasks.workunit.client.1.smithi174.stdout:4/633: symlink d4/d21/d24/d28/lcb 0 2024-03-21T10:09:00.001 INFO:tasks.workunit.client.1.smithi174.stdout:2/564: truncate d5/d25/d27/d2d/d45/d5b/d55/f83 948674 0 2024-03-21T10:09:00.001 INFO:tasks.workunit.client.0.smithi153.stdout:7/678: symlink d4/d2a/d3b/d81/le7 0 2024-03-21T10:09:00.003 INFO:tasks.workunit.client.1.smithi174.stdout:7/483: mkdir d0/d4/d10/d79/da7 0 2024-03-21T10:09:00.006 INFO:tasks.workunit.client.0.smithi153.stdout:5/707: chown d4/d8/f68 20684 1 2024-03-21T10:09:00.007 INFO:tasks.workunit.client.0.smithi153.stdout:0/675: creat d4/d4c/d83/fd5 x:0 0 0 2024-03-21T10:09:00.009 INFO:tasks.workunit.client.1.smithi174.stdout:4/634: creat d4/d21/d31/d3a/d67/fcc x:0 0 0 2024-03-21T10:09:00.010 INFO:tasks.workunit.client.0.smithi153.stdout:2/807: dwrite dd/d15/d5e/fbb [0,4194304] 0 2024-03-21T10:09:00.013 INFO:tasks.workunit.client.1.smithi174.stdout:0/559: rename d9/f54 to d9/d21/d31/d5c/d86/d9b/fc5 0 2024-03-21T10:09:00.014 INFO:tasks.workunit.client.1.smithi174.stdout:2/565: creat d5/d25/d27/d2e/d90/fc3 x:0 0 0 2024-03-21T10:09:00.015 INFO:tasks.workunit.client.0.smithi153.stdout:9/768: dwrite d1/d4/fd6 [0,4194304] 0 2024-03-21T10:09:00.015 INFO:tasks.workunit.client.0.smithi153.stdout:8/690: creat d5/d9/d43/d7b/fe4 x:0 0 0 2024-03-21T10:09:00.016 INFO:tasks.workunit.client.0.smithi153.stdout:2/808: chown dd/d15/d18/d2c/d33/deb/f10b 1234835 1 2024-03-21T10:09:00.018 INFO:tasks.workunit.client.1.smithi174.stdout:7/484: rmdir d0/d4/d10/d1b/d86 39 2024-03-21T10:09:00.021 INFO:tasks.workunit.client.1.smithi174.stdout:4/635: unlink d4/d21/d24/fb5 0 2024-03-21T10:09:00.023 INFO:tasks.workunit.client.0.smithi153.stdout:0/676: unlink d4/d16/d27/f4d 0 2024-03-21T10:09:00.027 INFO:tasks.workunit.client.0.smithi153.stdout:5/708: creat d4/d8/d5a/d5d/dd9/d44/dc7/fe8 x:0 0 0 2024-03-21T10:09:00.033 INFO:tasks.workunit.client.1.smithi174.stdout:1/645: dread d4/d5/f2a [0,4194304] 0 2024-03-21T10:09:00.033 INFO:tasks.workunit.client.0.smithi153.stdout:8/691: read d5/d9/d43/dd/d2d/d45/d88/f73 [1546376,38073] 0 2024-03-21T10:09:00.034 INFO:tasks.workunit.client.1.smithi174.stdout:4/636: creat d4/d19/fcd x:0 0 0 2024-03-21T10:09:00.037 INFO:tasks.workunit.client.1.smithi174.stdout:4/637: chown d4/d21/d31 2771 1 2024-03-21T10:09:00.038 INFO:tasks.workunit.client.0.smithi153.stdout:8/692: fdatasync d5/d9/d43/f8c 0 2024-03-21T10:09:00.039 INFO:tasks.workunit.client.1.smithi174.stdout:1/646: truncate d4/d7/d68/d89/fb3 554324 0 2024-03-21T10:09:00.040 INFO:tasks.workunit.client.0.smithi153.stdout:0/677: unlink d4/db/f11 0 2024-03-21T10:09:00.041 INFO:tasks.workunit.client.1.smithi174.stdout:7/485: rename d0/ff to d0/d4/fa8 0 2024-03-21T10:09:00.041 INFO:tasks.workunit.client.0.smithi153.stdout:9/769: dwrite d1/d43/d44/d46/d9e/d24/d41/f47 [0,4194304] 0 2024-03-21T10:09:00.042 INFO:tasks.workunit.client.0.smithi153.stdout:7/679: dwrite d4/d14/d2f/d78/f6f [0,4194304] 0 2024-03-21T10:09:00.049 INFO:tasks.workunit.client.0.smithi153.stdout:5/709: rmdir d4/d8/d5a/d5d/d7b 0 2024-03-21T10:09:00.049 INFO:tasks.workunit.client.0.smithi153.stdout:5/710: chown d4 369940 1 2024-03-21T10:09:00.050 INFO:tasks.workunit.client.0.smithi153.stdout:8/693: creat d5/d9/fe5 x:0 0 0 2024-03-21T10:09:00.051 INFO:tasks.workunit.client.1.smithi174.stdout:7/486: write d0/d4/d38/f66 [5149430,128117] 0 2024-03-21T10:09:00.051 INFO:tasks.workunit.client.1.smithi174.stdout:2/566: dwrite d5/d25/d27/d2e/d7b/d81/fba [0,4194304] 0 2024-03-21T10:09:00.051 INFO:tasks.workunit.client.1.smithi174.stdout:4/638: symlink d4/d21/d31/lce 0 2024-03-21T10:09:00.052 INFO:tasks.workunit.client.0.smithi153.stdout:9/770: dread - d1/d3b/f66 zero size 2024-03-21T10:09:00.055 INFO:tasks.workunit.client.0.smithi153.stdout:7/680: symlink d4/d2a/d3b/d60/le8 0 2024-03-21T10:09:00.057 INFO:tasks.workunit.client.0.smithi153.stdout:5/711: write d4/d5/df/d5e/fc0 [327955,124695] 0 2024-03-21T10:09:00.060 INFO:tasks.workunit.client.0.smithi153.stdout:4/743: write d11/f63 [411186,11730] 0 2024-03-21T10:09:00.060 INFO:tasks.workunit.client.0.smithi153.stdout:9/771: chown d1/d3b/daa/df9/fb0 4426 1 2024-03-21T10:09:00.061 INFO:tasks.workunit.client.0.smithi153.stdout:9/772: chown d1/d43/d44/d46/d4d 348 1 2024-03-21T10:09:00.063 INFO:tasks.workunit.client.0.smithi153.stdout:4/744: chown d11/d2d/d3d 693732 1 2024-03-21T10:09:00.064 INFO:tasks.workunit.client.0.smithi153.stdout:7/681: creat d4/d14/dd0/d20/fe9 x:0 0 0 2024-03-21T10:09:00.065 INFO:tasks.workunit.client.1.smithi174.stdout:1/647: rename d4/d7/d15/d53/fb7 to d4/d7/d70/fe3 0 2024-03-21T10:09:00.067 INFO:tasks.workunit.client.0.smithi153.stdout:7/682: fsync d4/d11/f61 0 2024-03-21T10:09:00.069 INFO:tasks.workunit.client.0.smithi153.stdout:4/745: dread d11/d18/f49 [0,4194304] 0 2024-03-21T10:09:00.069 INFO:tasks.workunit.client.0.smithi153.stdout:8/694: creat d5/d9/d43/fe6 x:0 0 0 2024-03-21T10:09:00.072 INFO:tasks.workunit.client.0.smithi153.stdout:8/695: chown d5/d9/d43/dd/d20/fce 446180 1 2024-03-21T10:09:00.073 INFO:tasks.workunit.client.1.smithi174.stdout:3/573: dread d0/d12/d14/f62 [0,4194304] 0 2024-03-21T10:09:00.074 INFO:tasks.workunit.client.0.smithi153.stdout:5/712: truncate d4/fa 1045617 0 2024-03-21T10:09:00.074 INFO:tasks.workunit.client.1.smithi174.stdout:8/489: dread d9/d2a/d39/d76/f87 [0,4194304] 0 2024-03-21T10:09:00.075 INFO:tasks.workunit.client.1.smithi174.stdout:6/421: truncate d0/d3/d5/d21/f6c 2588440 0 2024-03-21T10:09:00.076 INFO:tasks.workunit.client.1.smithi174.stdout:8/490: stat d9/lf 0 2024-03-21T10:09:00.076 INFO:tasks.workunit.client.0.smithi153.stdout:9/773: mknod d1/d4e/c101 0 2024-03-21T10:09:00.077 INFO:tasks.workunit.client.1.smithi174.stdout:6/422: read - d0/d3/d20/f7f zero size 2024-03-21T10:09:00.079 INFO:tasks.workunit.client.0.smithi153.stdout:0/678: link d4/d16/d19/d30/da7/lae d4/d16/d19/d30/d6c/d98/d5e/db2/ld6 0 2024-03-21T10:09:00.089 INFO:tasks.workunit.client.1.smithi174.stdout:9/585: sync 2024-03-21T10:09:00.089 INFO:tasks.workunit.client.1.smithi174.stdout:4/639: mkdir d4/d21/d31/d3a/d77/dcf 0 2024-03-21T10:09:00.091 INFO:tasks.workunit.client.0.smithi153.stdout:5/713: mknod d4/d8/d5a/d5d/dd9/d44/de1/ce9 0 2024-03-21T10:09:00.093 INFO:tasks.workunit.client.1.smithi174.stdout:1/648: mkdir d4/d7/d68/d7b/de0/de4 0 2024-03-21T10:09:00.097 INFO:tasks.workunit.client.1.smithi174.stdout:4/640: mknod d4/d21/cd0 0 2024-03-21T10:09:00.100 INFO:tasks.workunit.client.1.smithi174.stdout:6/423: creat d0/d3/d5/df/d66/f84 x:0 0 0 2024-03-21T10:09:00.104 INFO:tasks.workunit.client.0.smithi153.stdout:7/683: dwrite d4/d14/dd0/f7d [0,4194304] 0 2024-03-21T10:09:00.105 INFO:tasks.workunit.client.1.smithi174.stdout:0/560: fsync d9/d21/d23/da5/f6f 0 2024-03-21T10:09:00.108 INFO:tasks.workunit.client.0.smithi153.stdout:0/679: link d4/d16/d19/d30/d6c/d98/d5b/fa9 d4/d16/d27/dc5/fd7 0 2024-03-21T10:09:00.109 INFO:tasks.workunit.client.1.smithi174.stdout:6/424: write d0/d3/d5/df/f4e [867897,72367] 0 2024-03-21T10:09:00.110 INFO:tasks.workunit.client.1.smithi174.stdout:1/649: symlink d4/d7/d33/d43/d77/d87/d92/le5 0 2024-03-21T10:09:00.115 INFO:tasks.workunit.client.1.smithi174.stdout:3/574: dwrite d0/d12/d14/d1a/d27/d3c/f72 [0,4194304] 0 2024-03-21T10:09:00.116 INFO:tasks.workunit.client.0.smithi153.stdout:0/680: chown d4/d16/cb6 191793224 1 2024-03-21T10:09:00.120 INFO:tasks.workunit.client.1.smithi174.stdout:8/491: dwrite d9/d2a/d39/d76/d19/d27/f8a [0,4194304] 0 2024-03-21T10:09:00.122 INFO:tasks.workunit.client.1.smithi174.stdout:2/567: dread d5/d25/d27/d2d/d45/d5b/f6f [0,4194304] 0 2024-03-21T10:09:00.129 INFO:tasks.workunit.client.0.smithi153.stdout:8/696: dread d5/d9/d43/dd/d20/f7d [0,4194304] 0 2024-03-21T10:09:00.132 INFO:tasks.workunit.client.1.smithi174.stdout:0/561: rename d9/d21/d81/fb6 to d9/d21/d31/fc6 0 2024-03-21T10:09:00.133 INFO:tasks.workunit.client.1.smithi174.stdout:6/425: creat d0/d3/d5/d3b/f85 x:0 0 0 2024-03-21T10:09:00.136 INFO:tasks.workunit.client.0.smithi153.stdout:4/746: dwrite d11/d2d/d6b/db6/fa9 [0,4194304] 0 2024-03-21T10:09:00.137 INFO:tasks.workunit.client.0.smithi153.stdout:7/684: dwrite d4/d14/d16/f42 [0,4194304] 0 2024-03-21T10:09:00.138 INFO:tasks.workunit.client.0.smithi153.stdout:9/774: dwrite d1/d4/fc [0,4194304] 0 2024-03-21T10:09:00.140 INFO:tasks.workunit.client.0.smithi153.stdout:8/697: write d5/d9/d43/dd/d20/d26/d24/d52/d55/f89 [435744,130019] 0 2024-03-21T10:09:00.142 INFO:tasks.workunit.client.0.smithi153.stdout:6/899: sync 2024-03-21T10:09:00.142 INFO:tasks.workunit.client.0.smithi153.stdout:1/587: sync 2024-03-21T10:09:00.142 INFO:tasks.workunit.client.0.smithi153.stdout:3/791: sync 2024-03-21T10:09:00.142 INFO:tasks.workunit.client.0.smithi153.stdout:4/747: fsync d11/d71/d89/dca/fd8 0 2024-03-21T10:09:00.142 INFO:tasks.workunit.client.0.smithi153.stdout:7/685: stat d4/d11/d56/db1/cb5 0 2024-03-21T10:09:00.150 INFO:tasks.workunit.client.0.smithi153.stdout:1/588: unlink d6/f8c 0 2024-03-21T10:09:00.151 INFO:tasks.workunit.client.0.smithi153.stdout:8/698: mknod d5/d9/d43/d7b/ce7 0 2024-03-21T10:09:00.166 INFO:tasks.workunit.client.0.smithi153.stdout:5/714: write d4/d8/d5a/d5d/f5b [807640,38686] 0 2024-03-21T10:09:00.168 INFO:tasks.workunit.client.0.smithi153.stdout:7/686: truncate d4/d14/f4b 203264 0 2024-03-21T10:09:00.168 INFO:tasks.workunit.client.0.smithi153.stdout:9/775: write d1/d4/d5f/fce [948335,14529] 0 2024-03-21T10:09:00.174 INFO:tasks.workunit.client.0.smithi153.stdout:7/687: fdatasync d4/d2a/fa3 0 2024-03-21T10:09:00.175 INFO:tasks.workunit.client.0.smithi153.stdout:6/900: rmdir d1f/d2b/d10a/d40/d7b/d130 39 2024-03-21T10:09:00.179 INFO:tasks.workunit.client.1.smithi174.stdout:3/575: readlink d0/l9 0 2024-03-21T10:09:00.179 INFO:tasks.workunit.client.1.smithi174.stdout:2/568: rmdir d5/d25/d27/d2e/d4f/d65/d77 39 2024-03-21T10:09:00.180 INFO:tasks.workunit.client.0.smithi153.stdout:3/792: dwrite dd/d1e/d2f/d35/d68/d75/ddd/fa8 [0,4194304] 0 2024-03-21T10:09:00.180 INFO:tasks.workunit.client.1.smithi174.stdout:8/492: creat d9/d2a/d39/d76/fa2 x:0 0 0 2024-03-21T10:09:00.182 INFO:tasks.workunit.client.0.smithi153.stdout:4/748: dwrite d11/d2d/fd0 [4194304,4194304] 0 2024-03-21T10:09:00.185 INFO:tasks.workunit.client.1.smithi174.stdout:0/562: write d9/d21/d31/d3c/fb0 [7246201,55590] 0 2024-03-21T10:09:00.185 INFO:tasks.workunit.client.1.smithi174.stdout:6/426: creat d0/d3/d2a/d56/d5e/f86 x:0 0 0 2024-03-21T10:09:00.186 INFO:tasks.workunit.client.1.smithi174.stdout:3/576: chown d0/d12/d14/d1a/d27/d3c/d73/caf 725278505 1 2024-03-21T10:09:00.188 INFO:tasks.workunit.client.1.smithi174.stdout:6/427: write d0/d3/d5/f3c [1027126,79423] 0 2024-03-21T10:09:00.189 INFO:tasks.workunit.client.0.smithi153.stdout:7/688: unlink d4/d14/dd0/d83/l91 0 2024-03-21T10:09:00.190 INFO:tasks.workunit.client.1.smithi174.stdout:2/569: mknod d5/d25/d27/cc4 0 2024-03-21T10:09:00.191 INFO:tasks.workunit.client.0.smithi153.stdout:2/809: dread dd/d15/f97 [0,4194304] 0 2024-03-21T10:09:00.191 INFO:tasks.workunit.client.0.smithi153.stdout:1/589: link d6/d13/l24 d6/d10/d35/da5/lb8 0 2024-03-21T10:09:00.196 INFO:tasks.workunit.client.1.smithi174.stdout:0/563: creat d9/d21/d75/fc7 x:0 0 0 2024-03-21T10:09:00.199 INFO:tasks.workunit.client.0.smithi153.stdout:1/590: write d6/d3a/d3c/fa7 [205714,71944] 0 2024-03-21T10:09:00.199 INFO:tasks.workunit.client.1.smithi174.stdout:4/641: rename d4/d21/d31/d3a/d67/d76/d82/db1 to d4/d21/d31/d6a/dd1 0 2024-03-21T10:09:00.202 INFO:tasks.workunit.client.0.smithi153.stdout:7/689: symlink d4/d14/lea 0 2024-03-21T10:09:00.204 INFO:tasks.workunit.client.0.smithi153.stdout:8/699: dwrite d5/d9/d43/fa0 [0,4194304] 0 2024-03-21T10:09:00.204 INFO:tasks.workunit.client.0.smithi153.stdout:3/793: dread dd/d1e/d2f/f8c [0,4194304] 0 2024-03-21T10:09:00.204 INFO:tasks.workunit.client.1.smithi174.stdout:3/577: creat d0/d12/d14/d1a/fc2 x:0 0 0 2024-03-21T10:09:00.206 INFO:tasks.workunit.client.1.smithi174.stdout:9/586: dread d2/d24/d2e/f48 [0,4194304] 0 2024-03-21T10:09:00.207 INFO:tasks.workunit.client.1.smithi174.stdout:0/564: unlink d9/d21/d5f/f6c 0 2024-03-21T10:09:00.208 INFO:tasks.workunit.client.1.smithi174.stdout:8/493: rename d9/d2a/d39/d58/c6b to d9/ca3 0 2024-03-21T10:09:00.210 INFO:tasks.workunit.client.0.smithi153.stdout:9/776: link d1/d4/d5f/fe6 d1/d4/d5f/f102 0 2024-03-21T10:09:00.214 INFO:tasks.workunit.client.1.smithi174.stdout:9/587: creat d2/d11/d1d/d2a/da7/fd1 x:0 0 0 2024-03-21T10:09:00.216 INFO:tasks.workunit.client.1.smithi174.stdout:8/494: chown d9/d53/f96 5518 1 2024-03-21T10:09:00.217 INFO:tasks.workunit.client.0.smithi153.stdout:3/794: write dd/d41/d80/dbb/ffb [425394,29005] 0 2024-03-21T10:09:00.219 INFO:tasks.workunit.client.0.smithi153.stdout:0/681: dwrite d4/d16/d19/d30/d6c/d98/d5e/f71 [0,4194304] 0 2024-03-21T10:09:00.220 INFO:tasks.workunit.client.1.smithi174.stdout:8/495: write d9/d2a/d39/f75 [1288908,116052] 0 2024-03-21T10:09:00.222 INFO:tasks.workunit.client.0.smithi153.stdout:8/700: mknod d5/d9/d43/dd/d20/d26/d8b/ce8 0 2024-03-21T10:09:00.223 INFO:tasks.workunit.client.0.smithi153.stdout:3/795: dread - dd/d1e/d2f/d35/daf/dcc/fa5 zero size 2024-03-21T10:09:00.223 INFO:tasks.workunit.client.1.smithi174.stdout:5/613: sync 2024-03-21T10:09:00.226 INFO:tasks.workunit.client.0.smithi153.stdout:9/777: creat d1/d43/d44/d46/d9e/d2e/d67/f103 x:0 0 0 2024-03-21T10:09:00.228 INFO:tasks.workunit.client.0.smithi153.stdout:9/778: chown d1/d4/f9c 10460 1 2024-03-21T10:09:00.230 INFO:tasks.workunit.client.0.smithi153.stdout:6/901: link d1f/d2b/d10a/d40/d5f/l88 d1f/d99/dbb/dcf/l143 0 2024-03-21T10:09:00.235 INFO:tasks.workunit.client.0.smithi153.stdout:7/690: dread d4/d14/d16/f42 [0,4194304] 0 2024-03-21T10:09:00.235 INFO:tasks.workunit.client.0.smithi153.stdout:2/810: rename dd/d15/d18/d2c/d4b/d56/d63/d8d/d9b/dc3/df5/dae/d50/d58/d79/f7e to dd/d15/d18/d2c/d4b/d56/d63/d8d/d9b/dc3/df5/dae/f11b 0 2024-03-21T10:09:00.236 INFO:tasks.workunit.client.1.smithi174.stdout:9/588: unlink d2/d11/d1d/d8b/fa6 0 2024-03-21T10:09:00.237 INFO:tasks.workunit.client.0.smithi153.stdout:9/779: mknod d1/d8a/c104 0 2024-03-21T10:09:00.237 INFO:tasks.workunit.client.0.smithi153.stdout:2/811: write dd/d15/f52 [4791539,39619] 0 2024-03-21T10:09:00.241 INFO:tasks.workunit.client.0.smithi153.stdout:3/796: creat dd/d1e/d45/d105/f10e x:0 0 0 2024-03-21T10:09:00.242 INFO:tasks.workunit.client.1.smithi174.stdout:9/589: fdatasync d2/d24/d2e/f9b 0 2024-03-21T10:09:00.245 INFO:tasks.workunit.client.0.smithi153.stdout:0/682: write d4/d4c/d83/d22/f73 [3868348,27373] 0 2024-03-21T10:09:00.248 INFO:tasks.workunit.client.0.smithi153.stdout:7/691: mknod d4/d2a/d40/ceb 0 2024-03-21T10:09:00.250 INFO:tasks.workunit.client.0.smithi153.stdout:7/692: chown d4/d14/dd0/d20/c27 54562503 1 2024-03-21T10:09:00.250 INFO:tasks.workunit.client.0.smithi153.stdout:6/902: fdatasync d1f/d2b/d10a/d40/d51/dbd/f113 0 2024-03-21T10:09:00.253 INFO:tasks.workunit.client.0.smithi153.stdout:9/780: mknod d1/d43/d72/d9d/ddc/c105 0 2024-03-21T10:09:00.253 INFO:tasks.workunit.client.1.smithi174.stdout:8/496: dwrite d9/d2a/d39/d76/d19/d31/f94 [0,4194304] 0 2024-03-21T10:09:00.254 INFO:tasks.workunit.client.0.smithi153.stdout:6/903: chown d1f/d2b/db3/c133 2053869 1 2024-03-21T10:09:00.256 INFO:tasks.workunit.client.0.smithi153.stdout:2/812: creat dd/d15/d18/d2c/d4b/d56/d63/d8d/d9b/dc3/f11c x:0 0 0 2024-03-21T10:09:00.264 INFO:tasks.workunit.client.0.smithi153.stdout:3/797: mknod dd/d1e/d3d/d94/de2/c10f 0 2024-03-21T10:09:00.265 INFO:tasks.workunit.client.1.smithi174.stdout:6/428: dread d0/d3/d5/d6/f4a [0,4194304] 0 2024-03-21T10:09:00.267 INFO:tasks.workunit.client.0.smithi153.stdout:7/693: creat d4/d14/fec x:0 0 0 2024-03-21T10:09:00.268 INFO:tasks.workunit.client.1.smithi174.stdout:0/565: rename d9/d1c/d76/d87/lbc to d9/d21/d31/d9f/d79/d82/dbb/lc8 0 2024-03-21T10:09:00.270 INFO:tasks.workunit.client.1.smithi174.stdout:6/429: chown d0/d3/d5/d6/lc 5999 1 2024-03-21T10:09:00.277 INFO:tasks.workunit.client.0.smithi153.stdout:0/683: unlink d4/d16/d27/l5a 0 2024-03-21T10:09:00.277 INFO:tasks.workunit.client.0.smithi153.stdout:2/813: rename dd/d15/d18/d2c/f2d to dd/d15/d18/d2c/f11d 0 2024-03-21T10:09:00.279 INFO:tasks.workunit.client.1.smithi174.stdout:5/614: truncate d1/d16/d1c/d81/f95 1143494 0 2024-03-21T10:09:00.281 INFO:tasks.workunit.client.1.smithi174.stdout:9/590: dwrite d2/d11/d1d/d4b/fa8 [4194304,4194304] 0 2024-03-21T10:09:00.293 INFO:tasks.workunit.client.0.smithi153.stdout:9/781: dwrite d1/d43/d44/f33 [8388608,4194304] 0 2024-03-21T10:09:00.293 INFO:tasks.workunit.client.0.smithi153.stdout:4/749: dwrite d11/d18/f28 [0,4194304] 0 2024-03-21T10:09:00.294 INFO:tasks.workunit.client.1.smithi174.stdout:2/570: fsync d5/d25/d27/d2d/d45/d5b/d55/f83 0 2024-03-21T10:09:00.294 INFO:tasks.workunit.client.1.smithi174.stdout:0/566: symlink d9/d21/d31/d9f/d61/lc9 0 2024-03-21T10:09:00.296 INFO:tasks.workunit.client.1.smithi174.stdout:3/578: link d0/d12/d14/d1a/d44/c92 d0/d12/d14/d1a/d44/cc3 0 2024-03-21T10:09:00.298 INFO:tasks.workunit.client.1.smithi174.stdout:9/591: creat d2/d54/fd2 x:0 0 0 2024-03-21T10:09:00.299 INFO:tasks.workunit.client.0.smithi153.stdout:8/701: write d5/d1b/f93 [167646,94018] 0 2024-03-21T10:09:00.300 INFO:tasks.workunit.client.1.smithi174.stdout:5/615: write d1/ff [9045932,97885] 0 2024-03-21T10:09:00.301 INFO:tasks.workunit.client.0.smithi153.stdout:7/694: dread d4/d14/d2f/d78/f3a [4194304,4194304] 0 2024-03-21T10:09:00.307 INFO:tasks.workunit.client.1.smithi174.stdout:4/642: dread d4/d21/d31/d3a/d67/d76/fbd [0,4194304] 0 2024-03-21T10:09:00.307 INFO:tasks.workunit.client.1.smithi174.stdout:5/616: read d1/d16/f40 [5831603,13554] 0 2024-03-21T10:09:00.309 INFO:tasks.workunit.client.1.smithi174.stdout:4/643: dread - d4/d21/d31/d3a/d67/fcc zero size 2024-03-21T10:09:00.325 INFO:tasks.workunit.client.0.smithi153.stdout:6/904: fdatasync d1f/d93/dd8/d59/f6e 0 2024-03-21T10:09:00.332 INFO:tasks.workunit.client.0.smithi153.stdout:0/684: write d4/d16/d27/f76 [4684571,35034] 0 2024-03-21T10:09:00.332 INFO:tasks.workunit.client.0.smithi153.stdout:3/798: link dd/d1e/d45/fa0 dd/d1e/d2f/d35/f110 0 2024-03-21T10:09:00.349 INFO:tasks.workunit.client.0.smithi153.stdout:3/799: mkdir dd/d1e/d2f/d35/d111 0 2024-03-21T10:09:00.351 INFO:tasks.workunit.client.0.smithi153.stdout:3/800: stat dd/d1e/d3d/d94/db8 0 2024-03-21T10:09:00.353 INFO:tasks.workunit.client.0.smithi153.stdout:7/695: dread d4/d2a/f2d [0,4194304] 0 2024-03-21T10:09:00.357 INFO:tasks.workunit.client.0.smithi153.stdout:7/696: creat d4/d14/d16/fed x:0 0 0 2024-03-21T10:09:00.361 INFO:tasks.workunit.client.1.smithi174.stdout:7/487: sync 2024-03-21T10:09:00.364 INFO:tasks.workunit.client.1.smithi174.stdout:3/579: creat d0/d12/d14/d1a/d27/d7f/fc4 x:0 0 0 2024-03-21T10:09:00.365 INFO:tasks.workunit.client.0.smithi153.stdout:7/697: creat d4/d2a/d40/fee x:0 0 0 2024-03-21T10:09:00.370 INFO:tasks.workunit.client.0.smithi153.stdout:4/750: write d11/d18/d54/f68 [2374255,10809] 0 2024-03-21T10:09:00.370 INFO:tasks.workunit.client.1.smithi174.stdout:4/644: mkdir d4/d21/d31/d3a/dd2 0 2024-03-21T10:09:00.372 INFO:tasks.workunit.client.0.smithi153.stdout:9/782: write d1/d43/d44/f5d [6026139,51093] 0 2024-03-21T10:09:00.372 INFO:tasks.workunit.client.1.smithi174.stdout:5/617: link d1/d16/d2e/c3e d1/d49/d4f/ce2 0 2024-03-21T10:09:00.374 INFO:tasks.workunit.client.1.smithi174.stdout:3/580: chown d0/d12/d14/l85 14203 1 2024-03-21T10:09:00.375 INFO:tasks.workunit.client.1.smithi174.stdout:9/592: dwrite d2/d24/d2e/fc4 [0,4194304] 0 2024-03-21T10:09:00.377 INFO:tasks.workunit.client.0.smithi153.stdout:7/698: link d4/d19/d4d/f95 d4/d2a/dc6/fef 0 2024-03-21T10:09:00.377 INFO:tasks.workunit.client.0.smithi153.stdout:9/783: chown d1/d3b/c3c 3 1 2024-03-21T10:09:00.377 INFO:tasks.workunit.client.1.smithi174.stdout:6/430: dread d0/d3/d5/df/d16/f1a [0,4194304] 0 2024-03-21T10:09:00.378 INFO:tasks.workunit.client.1.smithi174.stdout:5/618: fdatasync d1/f86 0 2024-03-21T10:09:00.379 INFO:tasks.workunit.client.0.smithi153.stdout:7/699: chown d4/fd6 353493 1 2024-03-21T10:09:00.379 INFO:tasks.workunit.client.1.smithi174.stdout:0/567: link d9/d21/d23/da5/c5d d9/d21/d81/cca 0 2024-03-21T10:09:00.381 INFO:tasks.workunit.client.0.smithi153.stdout:4/751: symlink d11/d2d/d3d/dc7/dda/lee 0 2024-03-21T10:09:00.382 INFO:tasks.workunit.client.1.smithi174.stdout:4/645: symlink d4/d21/d31/d3a/d77/ld3 0 2024-03-21T10:09:00.386 INFO:tasks.workunit.client.1.smithi174.stdout:9/593: write d2/d11/d7d/fbd [155572,127949] 0 2024-03-21T10:09:00.388 INFO:tasks.workunit.client.1.smithi174.stdout:3/581: chown d0/d12/d14/d1a/d44/d57/l60 79568079 1 2024-03-21T10:09:00.389 INFO:tasks.workunit.client.1.smithi174.stdout:6/431: mkdir d0/d3/d5/df/d66/d87 0 2024-03-21T10:09:00.390 INFO:tasks.workunit.client.0.smithi153.stdout:6/905: dwrite f15 [0,4194304] 0 2024-03-21T10:09:00.391 INFO:tasks.workunit.client.1.smithi174.stdout:9/594: chown d2/d24/d2e/d7e/f95 376152614 1 2024-03-21T10:09:00.393 INFO:tasks.workunit.client.1.smithi174.stdout:3/582: truncate d0/d12/d14/da9/fb0 461623 0 2024-03-21T10:09:00.394 INFO:tasks.workunit.client.1.smithi174.stdout:7/488: dwrite d0/d4/d10/d60/f85 [0,4194304] 0 2024-03-21T10:09:00.395 INFO:tasks.workunit.client.0.smithi153.stdout:8/702: dwrite d5/d9/d43/dd/d2d/fa5 [0,4194304] 0 2024-03-21T10:09:00.397 INFO:tasks.workunit.client.1.smithi174.stdout:3/583: stat d0/d12/d14/d1a/d44/d65/d7e/dac/fc0 0 2024-03-21T10:09:00.398 INFO:tasks.workunit.client.1.smithi174.stdout:7/489: read d0/d4/d10/d1b/f68 [673613,38896] 0 2024-03-21T10:09:00.399 INFO:tasks.workunit.client.1.smithi174.stdout:5/619: rename d1/d16/d2e/d66/dbd to d1/d5c/d73/de3 0 2024-03-21T10:09:00.401 INFO:tasks.workunit.client.1.smithi174.stdout:0/568: creat d9/d35/d4a/d8a/fcb x:0 0 0 2024-03-21T10:09:00.405 INFO:tasks.workunit.client.0.smithi153.stdout:9/784: creat d1/d43/d44/d46/d9e/d24/d41/dbe/dcc/f106 x:0 0 0 2024-03-21T10:09:00.407 INFO:tasks.workunit.client.1.smithi174.stdout:6/432: write d0/d3/d20/f23 [8264094,11279] 0 2024-03-21T10:09:00.408 INFO:tasks.workunit.client.0.smithi153.stdout:8/703: truncate d5/d9/d43/dd/d20/d26/f23 6258083 0 2024-03-21T10:09:00.410 INFO:tasks.workunit.client.0.smithi153.stdout:4/752: write d11/f5f [584490,68477] 0 2024-03-21T10:09:00.410 INFO:tasks.workunit.client.1.smithi174.stdout:9/595: unlink d2/d11/d1d/dab/dc1/la3 0 2024-03-21T10:09:00.414 INFO:tasks.workunit.client.1.smithi174.stdout:7/490: dread - d0/d4/d10/d1b/d86/f92 zero size 2024-03-21T10:09:00.416 INFO:tasks.workunit.client.1.smithi174.stdout:7/491: truncate d0/f57 1216418 0 2024-03-21T10:09:00.417 INFO:tasks.workunit.client.1.smithi174.stdout:4/646: link d4/lb d4/d21/d31/d3a/dd2/ld4 0 2024-03-21T10:09:00.423 INFO:tasks.workunit.client.0.smithi153.stdout:9/785: symlink d1/d43/d44/d46/d9e/d2e/l107 0 2024-03-21T10:09:00.423 INFO:tasks.workunit.client.0.smithi153.stdout:5/715: sync 2024-03-21T10:09:00.427 INFO:tasks.workunit.client.1.smithi174.stdout:4/647: read - d4/d21/d24/d28/f81 zero size 2024-03-21T10:09:00.427 INFO:tasks.workunit.client.1.smithi174.stdout:4/648: fdatasync d4/d21/d31/d6a/dab/f99 0 2024-03-21T10:09:00.428 INFO:tasks.workunit.client.0.smithi153.stdout:7/700: dwrite d4/d2a/d3b/d81/fa9 [0,4194304] 0 2024-03-21T10:09:00.435 INFO:tasks.workunit.client.1.smithi174.stdout:1/650: sync 2024-03-21T10:09:00.435 INFO:tasks.workunit.client.1.smithi174.stdout:8/497: sync 2024-03-21T10:09:00.436 INFO:tasks.workunit.client.1.smithi174.stdout:9/596: dread d2/d24/d2e/fc4 [0,4194304] 0 2024-03-21T10:09:00.436 INFO:tasks.workunit.client.0.smithi153.stdout:8/704: link d5/d9/l94 d5/d9/d43/dd/d20/d26/d24/d52/le9 0 2024-03-21T10:09:00.436 INFO:tasks.workunit.client.1.smithi174.stdout:1/651: write d4/d7/f2c [5109842,31485] 0 2024-03-21T10:09:00.439 INFO:tasks.workunit.client.0.smithi153.stdout:9/786: rename d1/d43/d72/d9d/ddc to d1/d108 0 2024-03-21T10:09:00.441 INFO:tasks.workunit.client.0.smithi153.stdout:7/701: link d4/d14/d2f/d78/c55 d4/d14/dd0/d20/d9a/db7/cf0 0 2024-03-21T10:09:00.444 INFO:tasks.workunit.client.1.smithi174.stdout:6/433: dwrite d0/d3/d5/d6/d42/f69 [0,4194304] 0 2024-03-21T10:09:00.446 INFO:tasks.workunit.client.1.smithi174.stdout:6/434: chown d0/d3/d20/d6d 250204682 1 2024-03-21T10:09:00.447 INFO:tasks.workunit.client.0.smithi153.stdout:5/716: rename d4/d8/d5a/d5d/dd9/f22 to d4/d5/df/d67/da0/db3/fea 0 2024-03-21T10:09:00.448 INFO:tasks.workunit.client.0.smithi153.stdout:8/705: getdents d5/d9/d43/d7b/dc4/da7 0 2024-03-21T10:09:00.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:00 smithi174 ceph-mon[37496]: pgmap v16: 65 pgs: 65 active+clean; 2.4 GiB data, 6.8 GiB used, 530 GiB / 536 GiB avail; 68 MiB/s rd, 256 MiB/s wr, 442 op/s 2024-03-21T10:09:00.456 INFO:tasks.workunit.client.0.smithi153.stdout:8/706: symlink d5/d9/d43/dd/d2d/d45/lea 0 2024-03-21T10:09:00.461 INFO:tasks.workunit.client.1.smithi174.stdout:7/492: mkdir d0/d4/d38/da9 0 2024-03-21T10:09:00.465 INFO:tasks.workunit.client.0.smithi153.stdout:8/707: chown d5/d9/ld0 478162822 1 2024-03-21T10:09:00.468 INFO:tasks.workunit.client.0.smithi153.stdout:5/717: rename d4/d8/d3b to d4/d5/df/d67/d86/d96/deb 0 2024-03-21T10:09:00.473 INFO:tasks.workunit.client.1.smithi174.stdout:7/493: dread d0/f88 [0,4194304] 0 2024-03-21T10:09:00.474 INFO:tasks.workunit.client.0.smithi153.stdout:8/708: creat d5/d9/d43/dd/d2d/d45/d88/feb x:0 0 0 2024-03-21T10:09:00.475 INFO:tasks.workunit.client.0.smithi153.stdout:7/702: dwrite d4/d14/dd0/d20/d9a/db7/f96 [0,4194304] 0 2024-03-21T10:09:00.478 INFO:tasks.workunit.client.1.smithi174.stdout:3/584: dread d0/d12/d14/d1a/d27/d3c/d73/db7/f31 [0,4194304] 0 2024-03-21T10:09:00.479 INFO:tasks.workunit.client.0.smithi153.stdout:6/906: dread f1a [0,4194304] 0 2024-03-21T10:09:00.483 INFO:tasks.workunit.client.1.smithi174.stdout:2/571: sync 2024-03-21T10:09:00.486 INFO:tasks.workunit.client.1.smithi174.stdout:2/572: write d5/d25/d27/d2d/d45/d5b/d55/f69 [6258827,49301] 0 2024-03-21T10:09:00.489 INFO:tasks.workunit.client.1.smithi174.stdout:2/573: stat d5/d25/d27/d2e/l53 0 2024-03-21T10:09:00.490 INFO:tasks.workunit.client.0.smithi153.stdout:5/718: mknod d4/d5/df/d67/d86/d96/deb/d56/da5/cec 0 2024-03-21T10:09:00.491 INFO:tasks.workunit.client.0.smithi153.stdout:7/703: creat d4/d2a/d3b/d60/ff1 x:0 0 0 2024-03-21T10:09:00.494 INFO:tasks.workunit.client.0.smithi153.stdout:9/787: dread d1/d43/d44/d46/d9e/f9f [0,4194304] 0 2024-03-21T10:09:00.495 INFO:tasks.workunit.client.1.smithi174.stdout:0/569: link d9/d21/d81/cca d9/d35/d4a/d67/d89/d99/ccc 0 2024-03-21T10:09:00.496 INFO:tasks.workunit.client.0.smithi153.stdout:5/719: chown d4/d5/df/d67/d86/d96/lbc 1 1 2024-03-21T10:09:00.504 INFO:tasks.workunit.client.0.smithi153.stdout:9/788: unlink d1/d43/d44/d46/d9e/d4b/ce7 0 2024-03-21T10:09:00.504 INFO:tasks.workunit.client.0.smithi153.stdout:7/704: write d4/d2a/d3b/f98 [540972,118100] 0 2024-03-21T10:09:00.505 INFO:tasks.workunit.client.1.smithi174.stdout:4/649: mkdir d4/d19/dd5 0 2024-03-21T10:09:00.507 INFO:tasks.workunit.client.0.smithi153.stdout:7/705: chown d4/c8 33905900 1 2024-03-21T10:09:00.508 INFO:tasks.workunit.client.1.smithi174.stdout:8/498: mknod d9/d2a/d39/d58/ca4 0 2024-03-21T10:09:00.510 INFO:tasks.workunit.client.1.smithi174.stdout:8/499: fdatasync d9/d2a/d39/f75 0 2024-03-21T10:09:00.515 INFO:tasks.workunit.client.0.smithi153.stdout:8/709: dread d5/f8 [0,4194304] 0 2024-03-21T10:09:00.517 INFO:tasks.workunit.client.0.smithi153.stdout:9/789: truncate d1/d43/d44/f58 1212599 0 2024-03-21T10:09:00.521 INFO:tasks.workunit.client.1.smithi174.stdout:5/620: dwrite d1/d16/d2e/d66/f6c [0,4194304] 0 2024-03-21T10:09:00.528 INFO:tasks.workunit.client.0.smithi153.stdout:9/790: creat d1/d43/d44/d46/d9e/d24/d41/f109 x:0 0 0 2024-03-21T10:09:00.534 INFO:tasks.workunit.client.1.smithi174.stdout:6/435: dwrite d0/d3/d5/d3b/f51 [0,4194304] 0 2024-03-21T10:09:00.537 INFO:tasks.workunit.client.0.smithi153.stdout:6/907: dwrite d1f/da0/fb0 [0,4194304] 0 2024-03-21T10:09:00.539 INFO:tasks.workunit.client.1.smithi174.stdout:0/570: truncate d9/f6d 934099 0 2024-03-21T10:09:00.543 INFO:tasks.workunit.client.0.smithi153.stdout:7/706: write d4/d2a/f5b [1047146,62454] 0 2024-03-21T10:09:00.545 INFO:tasks.workunit.client.0.smithi153.stdout:5/720: dread d4/d5/df/d5e/f82 [0,4194304] 0 2024-03-21T10:09:00.549 INFO:tasks.workunit.client.1.smithi174.stdout:1/652: symlink d4/d7/d68/d7b/de0/de4/le6 0 2024-03-21T10:09:00.550 INFO:tasks.workunit.client.0.smithi153.stdout:6/908: symlink d1f/d2b/d10a/d40/d51/d11d/l144 0 2024-03-21T10:09:00.552 INFO:tasks.workunit.client.0.smithi153.stdout:9/791: link d1/d43/d44/d46/d9e/d61/f71 d1/d4e/d68/f10a 0 2024-03-21T10:09:00.554 INFO:tasks.workunit.client.0.smithi153.stdout:7/707: rmdir d4/d2a/d3b/d81 39 2024-03-21T10:09:00.554 INFO:tasks.workunit.client.1.smithi174.stdout:8/500: rmdir d9/d2a/d39/d76/d19/d27/d60 39 2024-03-21T10:09:00.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:00 smithi153 ceph-mon[29117]: pgmap v16: 65 pgs: 65 active+clean; 2.4 GiB data, 6.8 GiB used, 530 GiB / 536 GiB avail; 68 MiB/s rd, 256 MiB/s wr, 442 op/s 2024-03-21T10:09:00.556 INFO:tasks.workunit.client.1.smithi174.stdout:8/501: chown d9/l2d 0 1 2024-03-21T10:09:00.557 INFO:tasks.workunit.client.0.smithi153.stdout:8/710: dwrite d5/d9/d43/dd/d20/d26/d24/d52/f64 [0,4194304] 0 2024-03-21T10:09:00.559 INFO:tasks.workunit.client.0.smithi153.stdout:9/792: write d1/d3b/fc2 [1341051,91870] 0 2024-03-21T10:09:00.559 INFO:tasks.workunit.client.1.smithi174.stdout:5/621: read - d1/d2f/d37/d9b/fa3 zero size 2024-03-21T10:09:00.562 INFO:tasks.workunit.client.0.smithi153.stdout:7/708: write d4/d11/f72 [3717138,115589] 0 2024-03-21T10:09:00.563 INFO:tasks.workunit.client.1.smithi174.stdout:3/585: creat d0/d12/d46/d59/d88/fc5 x:0 0 0 2024-03-21T10:09:00.564 INFO:tasks.workunit.client.0.smithi153.stdout:6/909: dread - d1f/d2b/d10a/d40/d4f/fce zero size 2024-03-21T10:09:00.565 INFO:tasks.workunit.client.0.smithi153.stdout:8/711: chown d5/d9/d43/dd/d2d/d45/fc1 64395568 1 2024-03-21T10:09:00.567 INFO:tasks.workunit.client.0.smithi153.stdout:9/793: write d1/d43/d44/d46/d9e/d24/d41/dbe/dcc/fa4 [2926905,20383] 0 2024-03-21T10:09:00.567 INFO:tasks.workunit.client.1.smithi174.stdout:8/502: read d9/d53/f6f [3253549,117728] 0 2024-03-21T10:09:00.572 INFO:tasks.workunit.client.0.smithi153.stdout:8/712: symlink d5/d9/d43/d7b/dc4/lec 0 2024-03-21T10:09:00.574 INFO:tasks.workunit.client.0.smithi153.stdout:5/721: creat d4/d8/d5a/d5d/dd9/d44/dc7/de2/fed x:0 0 0 2024-03-21T10:09:00.578 INFO:tasks.workunit.client.1.smithi174.stdout:7/494: mkdir d0/d53/d5f/daa 0 2024-03-21T10:09:00.580 INFO:tasks.workunit.client.1.smithi174.stdout:7/495: fdatasync d0/d4/f8 0 2024-03-21T10:09:00.580 INFO:tasks.workunit.client.0.smithi153.stdout:5/722: mknod d4/cee 0 2024-03-21T10:09:00.582 INFO:tasks.workunit.client.1.smithi174.stdout:5/622: mkdir d1/d49/d4f/de4 0 2024-03-21T10:09:00.584 INFO:tasks.workunit.client.1.smithi174.stdout:5/623: dread - d1/d2f/d37/d9b/fa3 zero size 2024-03-21T10:09:00.586 INFO:tasks.workunit.client.0.smithi153.stdout:6/910: write d1f/d2b/d10a/d40/d5f/f12c [2982225,129283] 0 2024-03-21T10:09:00.595 INFO:tasks.workunit.client.1.smithi174.stdout:8/503: rename d9/d2a/d39/d76/f42 to d9/d2a/d39/d76/d19/d31/fa5 0 2024-03-21T10:09:00.601 INFO:tasks.workunit.client.1.smithi174.stdout:1/653: link d4/d7/d32/f3c d4/d7/d33/fe7 0 2024-03-21T10:09:00.603 INFO:tasks.workunit.client.1.smithi174.stdout:7/496: write d0/d4/d10/d44/d81/f82 [937390,79953] 0 2024-03-21T10:09:00.603 INFO:tasks.workunit.client.1.smithi174.stdout:3/586: mknod d0/d12/d14/d1a/d27/d3c/d73/db4/cc6 0 2024-03-21T10:09:00.605 INFO:tasks.workunit.client.1.smithi174.stdout:4/650: dread d4/f8 [0,4194304] 0 2024-03-21T10:09:00.607 INFO:tasks.workunit.client.1.smithi174.stdout:6/436: rmdir d0/d3/d5/d3b/d79 0 2024-03-21T10:09:00.608 INFO:tasks.workunit.client.0.smithi153.stdout:6/911: dwrite d1f/d99/ff9 [0,4194304] 0 2024-03-21T10:09:00.609 INFO:tasks.workunit.client.0.smithi153.stdout:8/713: dread d5/d9/d43/d7b/daa/fb8 [0,4194304] 0 2024-03-21T10:09:00.611 INFO:tasks.workunit.client.0.smithi153.stdout:6/912: write d1f/d93/dd8/ffb [1370424,1415] 0 2024-03-21T10:09:00.611 INFO:tasks.workunit.client.1.smithi174.stdout:6/437: dread - d0/d3/d5/df/d66/f84 zero size 2024-03-21T10:09:00.612 INFO:tasks.workunit.client.0.smithi153.stdout:8/714: stat d5/d9/d43/dd/d2d/d45/fab 0 2024-03-21T10:09:00.615 INFO:tasks.workunit.client.1.smithi174.stdout:5/624: rename d1/d16/l45 to d1/d2f/dc1/le5 0 2024-03-21T10:09:00.616 INFO:tasks.workunit.client.0.smithi153.stdout:8/715: dread - d5/d9/d43/dd/d2d/d45/d88/d85/fbb zero size 2024-03-21T10:09:00.619 INFO:tasks.workunit.client.0.smithi153.stdout:5/723: dread d4/d8/d5a/d5d/dd9/d38/f64 [0,4194304] 0 2024-03-21T10:09:00.619 INFO:tasks.workunit.client.1.smithi174.stdout:8/504: creat d9/d2a/d39/d76/d19/d27/d79/fa6 x:0 0 0 2024-03-21T10:09:00.621 INFO:tasks.workunit.client.1.smithi174.stdout:8/505: readlink d9/d2a/la1 0 2024-03-21T10:09:00.622 INFO:tasks.workunit.client.0.smithi153.stdout:5/724: write d4/d8/d5a/d5d/dd9/d44/dc7/de2/fed [396667,107658] 0 2024-03-21T10:09:00.624 INFO:tasks.workunit.client.1.smithi174.stdout:1/654: creat d4/d7/d70/fe8 x:0 0 0 2024-03-21T10:09:00.626 INFO:tasks.workunit.client.0.smithi153.stdout:5/725: readlink d4/d5/df/d67/d86/l91 0 2024-03-21T10:09:00.626 INFO:tasks.workunit.client.1.smithi174.stdout:1/655: stat d4/d7/d28/l4d 0 2024-03-21T10:09:00.627 INFO:tasks.workunit.client.1.smithi174.stdout:3/587: unlink d0/d12/d14/d1a/c86 0 2024-03-21T10:09:00.628 INFO:tasks.workunit.client.0.smithi153.stdout:5/726: creat d4/d8/d5a/d5d/fef x:0 0 0 2024-03-21T10:09:00.631 INFO:tasks.workunit.client.1.smithi174.stdout:6/438: dwrite d0/d3/d5/d3b/f85 [0,4194304] 0 2024-03-21T10:09:00.633 INFO:tasks.workunit.client.1.smithi174.stdout:3/588: chown d0/d12/d14/d4c/d7a/d9e/fb8 3 1 2024-03-21T10:09:00.634 INFO:tasks.workunit.client.0.smithi153.stdout:1/591: sync 2024-03-21T10:09:00.639 INFO:tasks.workunit.client.1.smithi174.stdout:5/625: dwrite d1/d49/d4f/d51/f56 [0,4194304] 0 2024-03-21T10:09:00.645 INFO:tasks.workunit.client.0.smithi153.stdout:8/716: dread d5/d9/d43/dd/d20/d26/d24/d52/d55/f5c [0,4194304] 0 2024-03-21T10:09:00.646 INFO:tasks.workunit.client.1.smithi174.stdout:7/497: getdents d0/d4/d10/d60/da2 0 2024-03-21T10:09:00.647 INFO:tasks.workunit.client.0.smithi153.stdout:8/717: chown d5/d1b/f87 11 1 2024-03-21T10:09:00.648 INFO:tasks.workunit.client.1.smithi174.stdout:3/589: dread d0/d12/d14/d1a/d44/f6c [0,4194304] 0 2024-03-21T10:09:00.649 INFO:tasks.workunit.client.0.smithi153.stdout:5/727: truncate d4/d8/d5a/d5d/f5f 774511 0 2024-03-21T10:09:00.649 INFO:tasks.workunit.client.0.smithi153.stdout:1/592: write f3 [812773,36845] 0 2024-03-21T10:09:00.651 INFO:tasks.workunit.client.0.smithi153.stdout:1/593: chown d6/f17 819 1 2024-03-21T10:09:00.658 INFO:tasks.workunit.client.1.smithi174.stdout:8/506: symlink d9/d2a/d39/d76/d19/d27/d60/la7 0 2024-03-21T10:09:00.659 INFO:tasks.workunit.client.0.smithi153.stdout:1/594: creat d6/d10/fb9 x:0 0 0 2024-03-21T10:09:00.662 INFO:tasks.workunit.client.0.smithi153.stdout:1/595: dread - d6/d3a/d66/f6a zero size 2024-03-21T10:09:00.663 INFO:tasks.workunit.client.1.smithi174.stdout:5/626: dwrite d1/d16/d2e/d66/fbe [0,4194304] 0 2024-03-21T10:09:00.672 INFO:tasks.workunit.client.0.smithi153.stdout:8/718: dwrite d5/f8 [0,4194304] 0 2024-03-21T10:09:00.675 INFO:tasks.workunit.client.0.smithi153.stdout:5/728: dwrite d4/d5/df/d67/d6a/fc3 [0,4194304] 0 2024-03-21T10:09:00.675 INFO:tasks.workunit.client.1.smithi174.stdout:6/439: link d0/d3/d5/df/d16/f1a d0/d3/d5/df/d16/f88 0 2024-03-21T10:09:00.678 INFO:tasks.workunit.client.1.smithi174.stdout:8/507: write d9/d2a/d39/f81 [3594866,89711] 0 2024-03-21T10:09:00.679 INFO:tasks.workunit.client.0.smithi153.stdout:8/719: stat d5/d9/d43/dd/d20/d26/d47/la9 0 2024-03-21T10:09:00.682 INFO:tasks.workunit.client.0.smithi153.stdout:8/720: read d5/d9/d43/dd/d20/f7d [1325909,16883] 0 2024-03-21T10:09:00.682 INFO:tasks.workunit.client.0.smithi153.stdout:5/729: stat d4/d8/d5a/d5d/dd9/d44/d48 0 2024-03-21T10:09:00.685 INFO:tasks.workunit.client.0.smithi153.stdout:1/596: rename d6/d3a/d3c/d50/l59 to d6/d13/d75/lba 0 2024-03-21T10:09:00.689 INFO:tasks.workunit.client.0.smithi153.stdout:8/721: stat d5/d9/d43/dd/d20/d26/d8b/f4f 0 2024-03-21T10:09:00.689 INFO:tasks.workunit.client.0.smithi153.stdout:1/597: mkdir d6/d13/d8e/dbb 0 2024-03-21T10:09:00.692 INFO:tasks.workunit.client.1.smithi174.stdout:3/590: link d0/d12/d14/d1a/d27/d4f/d64/l6e d0/d12/d46/d59/d88/lc7 0 2024-03-21T10:09:00.695 INFO:tasks.workunit.client.0.smithi153.stdout:5/730: truncate d4/d8/d5a/d92/f94 23132 0 2024-03-21T10:09:00.700 INFO:tasks.workunit.client.0.smithi153.stdout:1/598: symlink d6/d10/d35/da5/lbc 0 2024-03-21T10:09:00.702 INFO:tasks.workunit.client.0.smithi153.stdout:8/722: read d5/d9/d43/d7b/daa/fb5 [40544,74319] 0 2024-03-21T10:09:00.702 INFO:tasks.workunit.client.1.smithi174.stdout:1/656: symlink d4/le9 0 2024-03-21T10:09:00.703 INFO:tasks.workunit.client.0.smithi153.stdout:5/731: truncate d4/d5/df/d67/d86/d96/fb8 574858 0 2024-03-21T10:09:00.703 INFO:tasks.workunit.client.1.smithi174.stdout:2/574: write d5/f12 [1180302,39273] 0 2024-03-21T10:09:00.705 INFO:tasks.workunit.client.0.smithi153.stdout:1/599: write d6/d3a/d3c/d50/d51/f6d [662156,21675] 0 2024-03-21T10:09:00.709 INFO:tasks.workunit.client.0.smithi153.stdout:8/723: write d5/d9/d43/dd/d2d/d45/d88/fbe [2710617,1223] 0 2024-03-21T10:09:00.710 INFO:tasks.workunit.client.1.smithi174.stdout:0/571: write d9/d21/d31/d3c/d72/f7e [837160,15878] 0 2024-03-21T10:09:00.711 INFO:tasks.workunit.client.1.smithi174.stdout:6/440: dwrite d0/d3/d5/d6/f6a [0,4194304] 0 2024-03-21T10:09:00.712 INFO:tasks.workunit.client.1.smithi174.stdout:3/591: chown d0/d12/d14/d1a/d44/d56/d76/cb5 1175 1 2024-03-21T10:09:00.713 INFO:tasks.workunit.client.1.smithi174.stdout:6/441: readlink d0/l5b 0 2024-03-21T10:09:00.715 INFO:tasks.workunit.client.0.smithi153.stdout:5/732: mkdir d4/d5/df/d67/d86/d96/deb/df0 0 2024-03-21T10:09:00.718 INFO:tasks.workunit.client.1.smithi174.stdout:5/627: dwrite d1/d2f/d37/d9b/fa3 [0,4194304] 0 2024-03-21T10:09:00.719 INFO:tasks.workunit.client.1.smithi174.stdout:9/597: symlink d2/d11/ld3 0 2024-03-21T10:09:00.720 INFO:tasks.workunit.client.0.smithi153.stdout:8/724: creat d5/d9/d43/dd/d20/d26/d47/d9a/db9/fed x:0 0 0 2024-03-21T10:09:00.724 INFO:tasks.workunit.client.0.smithi153.stdout:8/725: chown d5/d9/d43/dd/d20/d26/d8b/d80/l92 0 1 2024-03-21T10:09:00.725 INFO:tasks.workunit.client.1.smithi174.stdout:2/575: dread d5/d25/d27/f3c [0,4194304] 0 2024-03-21T10:09:00.726 INFO:tasks.workunit.client.1.smithi174.stdout:5/628: creat d1/d5c/fe6 x:0 0 0 2024-03-21T10:09:00.726 INFO:tasks.workunit.client.0.smithi153.stdout:1/600: getdents d6/d10/d35 0 2024-03-21T10:09:00.728 INFO:tasks.workunit.client.0.smithi153.stdout:5/733: creat d4/d8/d5a/d5d/dd9/d2f/ff1 x:0 0 0 2024-03-21T10:09:00.729 INFO:tasks.workunit.client.1.smithi174.stdout:9/598: symlink d2/d11/d60/db2/ld4 0 2024-03-21T10:09:00.729 INFO:tasks.workunit.client.0.smithi153.stdout:1/601: chown d6/d10/d35/da5/d4f/d9d/db6 29685 1 2024-03-21T10:09:00.736 INFO:tasks.workunit.client.1.smithi174.stdout:2/576: symlink d5/d25/d27/d2d/d45/d5b/d63/lc5 0 2024-03-21T10:09:00.741 INFO:tasks.workunit.client.1.smithi174.stdout:2/577: dread - d5/d25/d27/d2d/d45/d5b/d2f/d68/d9e/fa6 zero size 2024-03-21T10:09:00.743 INFO:tasks.workunit.client.1.smithi174.stdout:5/629: creat d1/d2f/d37/fe7 x:0 0 0 2024-03-21T10:09:00.744 INFO:tasks.workunit.client.1.smithi174.stdout:0/572: dwrite d9/d21/d31/d5c/d92/d98/fc4 [0,4194304] 0 2024-03-21T10:09:00.745 INFO:tasks.workunit.client.1.smithi174.stdout:4/651: rename d4/d21/d31/d6a/fb8 to d4/d21/fd6 0 2024-03-21T10:09:00.747 INFO:tasks.workunit.client.1.smithi174.stdout:5/630: stat d1/d16/d2e/d66/d8b/d85/f90 0 2024-03-21T10:09:00.756 INFO:tasks.workunit.client.1.smithi174.stdout:0/573: readlink d9/d21/d5f/l9c 0 2024-03-21T10:09:00.762 INFO:tasks.workunit.client.1.smithi174.stdout:1/657: rename d4/d7/d68/d8e/fa1 to d4/d5/d6c/fea 0 2024-03-21T10:09:00.767 INFO:tasks.workunit.client.1.smithi174.stdout:7/498: dwrite d0/d4/d10/d1b/f2e [0,4194304] 0 2024-03-21T10:09:00.767 INFO:tasks.workunit.client.1.smithi174.stdout:2/578: creat d5/d25/d27/d2e/d4f/d65/d77/fc6 x:0 0 0 2024-03-21T10:09:00.774 INFO:tasks.workunit.client.1.smithi174.stdout:1/658: chown d4/d7/d15/d4f/l58 7 1 2024-03-21T10:09:00.775 INFO:tasks.workunit.client.1.smithi174.stdout:0/574: mkdir d9/d1c/d76/d87/dcd 0 2024-03-21T10:09:00.777 INFO:tasks.workunit.client.1.smithi174.stdout:8/508: dwrite d9/d2a/d39/d76/f3c [0,4194304] 0 2024-03-21T10:09:00.779 INFO:tasks.workunit.client.1.smithi174.stdout:7/499: write d0/d4/d10/f19 [3893311,56384] 0 2024-03-21T10:09:00.786 INFO:tasks.workunit.client.0.smithi153.stdout:8/726: unlink d5/d9/d43/dd/d2d/d45/d88/d85/cdb 0 2024-03-21T10:09:00.787 INFO:tasks.workunit.client.1.smithi174.stdout:8/509: chown d9/d2a/d39/d76/d19/d27/f64 1266111 1 2024-03-21T10:09:00.787 INFO:tasks.workunit.client.1.smithi174.stdout:4/652: dread d4/d21/d24/d28/fc1 [0,4194304] 0 2024-03-21T10:09:00.787 INFO:tasks.workunit.client.1.smithi174.stdout:0/575: mknod d9/d21/d5f/cce 0 2024-03-21T10:09:00.787 INFO:tasks.workunit.client.1.smithi174.stdout:9/599: dread d2/d11/f4a [0,4194304] 0 2024-03-21T10:09:00.790 INFO:tasks.workunit.client.1.smithi174.stdout:0/576: stat d9/d35/d4a/d8a/cb5 0 2024-03-21T10:09:00.790 INFO:tasks.workunit.client.1.smithi174.stdout:7/500: write d0/d4/d10/d60/f8a [376592,45744] 0 2024-03-21T10:09:00.790 INFO:tasks.workunit.client.1.smithi174.stdout:3/592: rename d0/d12/d46/d59/d88 to d0/d12/d14/d67/dc8 0 2024-03-21T10:09:00.798 INFO:tasks.workunit.client.1.smithi174.stdout:8/510: dread - d9/d2a/d39/d76/d19/d27/d79/fa6 zero size 2024-03-21T10:09:00.801 INFO:tasks.workunit.client.0.smithi153.stdout:2/814: sync 2024-03-21T10:09:00.801 INFO:tasks.workunit.client.1.smithi174.stdout:4/653: rmdir d4/d21/d31/d3a/d7a/d9d/da3 39 2024-03-21T10:09:00.804 INFO:tasks.workunit.client.0.smithi153.stdout:1/602: rmdir d6/d3a/d3c/d50/da8/db4 0 2024-03-21T10:09:00.805 INFO:tasks.workunit.client.0.smithi153.stdout:8/727: getdents d5/d9/d43/dd/d20/d26/d24 0 2024-03-21T10:09:00.806 INFO:tasks.workunit.client.1.smithi174.stdout:5/631: write d1/f22 [4040210,30242] 0 2024-03-21T10:09:00.807 INFO:tasks.workunit.client.1.smithi174.stdout:3/593: mknod d0/d12/d14/d67/cc9 0 2024-03-21T10:09:00.808 INFO:tasks.workunit.client.0.smithi153.stdout:2/815: creat dd/d15/d18/d2c/dd7/f11e x:0 0 0 2024-03-21T10:09:00.808 INFO:tasks.workunit.client.1.smithi174.stdout:8/511: creat d9/d2a/d39/d76/d19/d27/d60/fa8 x:0 0 0 2024-03-21T10:09:00.811 INFO:tasks.workunit.client.0.smithi153.stdout:8/728: creat d5/d9/fee x:0 0 0 2024-03-21T10:09:00.811 INFO:tasks.workunit.client.1.smithi174.stdout:5/632: read - d1/d5c/d73/fa5 zero size 2024-03-21T10:09:00.815 INFO:tasks.workunit.client.0.smithi153.stdout:2/816: rename dd/d15/d18/d2c/d33/d98/dc7/de1/d102 to dd/d15/d3c/da7/d11f 0 2024-03-21T10:09:00.815 INFO:tasks.workunit.client.1.smithi174.stdout:3/594: fdatasync d0/d12/d14/f62 0 2024-03-21T10:09:00.816 INFO:tasks.workunit.client.1.smithi174.stdout:7/501: link d0/d4/d10/d1b/l84 d0/d4/d10/d60/lab 0 2024-03-21T10:09:00.816 INFO:tasks.workunit.client.0.smithi153.stdout:8/729: mkdir d5/d9/d43/dd/d20/d26/d8b/d80/def 0 2024-03-21T10:09:00.816 INFO:tasks.workunit.client.1.smithi174.stdout:0/577: fsync d9/d21/d31/d5c/d92/d98/fc4 0 2024-03-21T10:09:00.818 INFO:tasks.workunit.client.1.smithi174.stdout:5/633: write d1/d2f/fae [4815318,95514] 0 2024-03-21T10:09:00.819 INFO:tasks.workunit.client.0.smithi153.stdout:2/817: mkdir dd/d15/d18/d3a/d3d/ddc/d120 0 2024-03-21T10:09:00.820 INFO:tasks.workunit.client.0.smithi153.stdout:8/730: write d5/d9/d43/dd/d20/d26/d8b/fa4 [1514148,15185] 0 2024-03-21T10:09:00.824 INFO:tasks.workunit.client.1.smithi174.stdout:4/654: creat d4/d21/d31/d6a/fd7 x:0 0 0 2024-03-21T10:09:00.825 INFO:tasks.workunit.client.1.smithi174.stdout:1/659: write d4/d2d/f57 [1946049,74401] 0 2024-03-21T10:09:00.825 INFO:tasks.workunit.client.0.smithi153.stdout:8/731: mknod d5/d1b/d49/cf0 0 2024-03-21T10:09:00.827 INFO:tasks.workunit.client.0.smithi153.stdout:2/818: unlink dd/d15/d18/d2c/d33/d98/dc7/de1/cee 0 2024-03-21T10:09:00.828 INFO:tasks.workunit.client.1.smithi174.stdout:7/502: chown d0/d4/d10/d44/d81/f87 33037572 1 2024-03-21T10:09:00.829 INFO:tasks.workunit.client.0.smithi153.stdout:2/819: chown dd/d15/d18/d3a/lc2 170244 1 2024-03-21T10:09:00.830 INFO:tasks.workunit.client.1.smithi174.stdout:3/595: chown d0/d12/d14/d1a/d27/d3c/f72 0 1 2024-03-21T10:09:00.832 INFO:tasks.workunit.client.1.smithi174.stdout:6/442: dread d0/d3/d5/f4d [0,4194304] 0 2024-03-21T10:09:00.835 INFO:tasks.workunit.client.1.smithi174.stdout:5/634: chown d1/d16/d2e/d66/d8b/f33 7129 1 2024-03-21T10:09:00.837 INFO:tasks.workunit.client.1.smithi174.stdout:4/655: chown d4/d19/f41 70079 1 2024-03-21T10:09:00.842 INFO:tasks.workunit.client.1.smithi174.stdout:9/600: dwrite d2/f84 [4194304,4194304] 0 2024-03-21T10:09:00.842 INFO:tasks.workunit.client.0.smithi153.stdout:8/732: dwrite d5/d9/d43/dd/d20/d26/d47/d69/f86 [0,4194304] 0 2024-03-21T10:09:00.848 INFO:tasks.workunit.client.1.smithi174.stdout:6/443: stat d0/d3/d5/l54 0 2024-03-21T10:09:00.848 INFO:tasks.workunit.client.1.smithi174.stdout:0/578: dread d9/d21/d31/d5c/d86/d9b/fc5 [0,4194304] 0 2024-03-21T10:09:00.853 INFO:tasks.workunit.client.1.smithi174.stdout:2/579: dwrite d5/d25/d27/f2a [0,4194304] 0 2024-03-21T10:09:00.856 INFO:tasks.workunit.client.1.smithi174.stdout:1/660: creat d4/d7/feb x:0 0 0 2024-03-21T10:09:00.861 INFO:tasks.workunit.client.1.smithi174.stdout:8/512: dwrite d9/d53/f95 [0,4194304] 0 2024-03-21T10:09:00.861 INFO:tasks.workunit.client.1.smithi174.stdout:5/635: creat d1/d2f/dc1/fe8 x:0 0 0 2024-03-21T10:09:00.866 INFO:tasks.workunit.client.1.smithi174.stdout:3/596: fsync d0/d12/d14/d1a/d27/d3c/d73/db7/f1b 0 2024-03-21T10:09:00.867 INFO:tasks.workunit.client.1.smithi174.stdout:1/661: symlink d4/d5/d6c/lec 0 2024-03-21T10:09:00.869 INFO:tasks.workunit.client.1.smithi174.stdout:5/636: write d1/d49/d93/d9c/fb2 [405275,17879] 0 2024-03-21T10:09:00.873 INFO:tasks.workunit.client.1.smithi174.stdout:2/580: stat d5/d25/d27/d2d/d45/d5b/d2f/l5c 0 2024-03-21T10:09:00.876 INFO:tasks.workunit.client.0.smithi153.stdout:5/734: mknod d4/d5/df/d67/da0/cf2 0 2024-03-21T10:09:00.878 INFO:tasks.workunit.client.1.smithi174.stdout:8/513: creat d9/fa9 x:0 0 0 2024-03-21T10:09:00.882 INFO:tasks.workunit.client.0.smithi153.stdout:0/685: sync 2024-03-21T10:09:00.882 INFO:tasks.workunit.client.0.smithi153.stdout:4/753: sync 2024-03-21T10:09:00.885 INFO:tasks.workunit.client.0.smithi153.stdout:3/801: sync 2024-03-21T10:09:00.885 INFO:tasks.workunit.client.0.smithi153.stdout:4/754: chown d11/d2d/d6b/db6/fa9 232366 1 2024-03-21T10:09:00.886 INFO:tasks.workunit.client.0.smithi153.stdout:4/755: dread - d11/db1/fd5 zero size 2024-03-21T10:09:00.891 INFO:tasks.workunit.client.0.smithi153.stdout:0/686: rename d4/d4c/d83/f24 to d4/d16/d19/d58/da1/fd8 0 2024-03-21T10:09:00.892 INFO:tasks.workunit.client.1.smithi174.stdout:4/656: dwrite d4/d21/f8c [0,4194304] 0 2024-03-21T10:09:00.893 INFO:tasks.workunit.client.0.smithi153.stdout:4/756: mknod d11/d9d/cef 0 2024-03-21T10:09:00.893 INFO:tasks.workunit.client.1.smithi174.stdout:3/597: rename d0/ld to d0/d12/d14/d1a/d27/d3c/d73/db4/lca 0 2024-03-21T10:09:00.894 INFO:tasks.workunit.client.1.smithi174.stdout:1/662: dread d4/d7/f2e [0,4194304] 0 2024-03-21T10:09:00.896 INFO:tasks.workunit.client.1.smithi174.stdout:4/657: read d4/d21/d24/d28/f5e [1085778,32799] 0 2024-03-21T10:09:00.897 INFO:tasks.workunit.client.0.smithi153.stdout:4/757: getdents d11/d2d/d57/ded 0 2024-03-21T10:09:00.901 INFO:tasks.workunit.client.1.smithi174.stdout:7/503: dread d0/f11 [0,4194304] 0 2024-03-21T10:09:00.904 INFO:tasks.workunit.client.0.smithi153.stdout:4/758: readlink d11/d18/l37 0 2024-03-21T10:09:00.912 INFO:tasks.workunit.client.0.smithi153.stdout:0/687: dwrite d4/f62 [0,4194304] 0 2024-03-21T10:09:00.921 INFO:tasks.workunit.client.0.smithi153.stdout:0/688: creat d4/d16/d19/d30/d6c/d98/d5e/db2/fd9 x:0 0 0 2024-03-21T10:09:00.929 INFO:tasks.workunit.client.0.smithi153.stdout:4/759: dwrite d11/f8d [0,4194304] 0 2024-03-21T10:09:00.932 INFO:tasks.workunit.client.0.smithi153.stdout:9/794: sync 2024-03-21T10:09:00.932 INFO:tasks.workunit.client.0.smithi153.stdout:6/913: sync 2024-03-21T10:09:00.932 INFO:tasks.workunit.client.0.smithi153.stdout:7/709: sync 2024-03-21T10:09:00.933 INFO:tasks.workunit.client.0.smithi153.stdout:4/760: readlink d11/l4f 0 2024-03-21T10:09:00.935 INFO:tasks.workunit.client.0.smithi153.stdout:4/761: dread - d11/db1/fd5 zero size 2024-03-21T10:09:00.936 INFO:tasks.workunit.client.1.smithi174.stdout:2/581: dread d5/d25/f7e [0,4194304] 0 2024-03-21T10:09:00.936 INFO:tasks.workunit.client.0.smithi153.stdout:7/710: write d4/d2a/d3b/d60/ff1 [208120,51590] 0 2024-03-21T10:09:00.945 INFO:tasks.workunit.client.0.smithi153.stdout:7/711: creat d4/d11/ff2 x:0 0 0 2024-03-21T10:09:00.949 INFO:tasks.workunit.client.0.smithi153.stdout:6/914: fsync d1f/d2b/d10a/d40/d51/d98/f120 0 2024-03-21T10:09:00.952 INFO:tasks.workunit.client.0.smithi153.stdout:6/915: stat d1f/d2b/d10a/d40/d51/d98/ca4 0 2024-03-21T10:09:00.954 INFO:tasks.workunit.client.0.smithi153.stdout:4/762: rename d11/d2d/d57/d85/la5 to d11/d2d/d3d/dc7/lf0 0 2024-03-21T10:09:00.955 INFO:tasks.workunit.client.0.smithi153.stdout:4/763: readlink d11/d2d/d3d/lc4 0 2024-03-21T10:09:00.958 INFO:tasks.workunit.client.0.smithi153.stdout:4/764: stat d11/f63 0 2024-03-21T10:09:00.958 INFO:tasks.workunit.client.0.smithi153.stdout:9/795: dread d1/d43/d44/d46/d9e/d24/d41/f5b [0,4194304] 0 2024-03-21T10:09:00.958 INFO:tasks.workunit.client.0.smithi153.stdout:7/712: unlink d4/d14/dd0/f7d 0 2024-03-21T10:09:00.965 INFO:tasks.workunit.client.0.smithi153.stdout:9/796: mkdir d1/d108/d10b 0 2024-03-21T10:09:00.970 INFO:tasks.workunit.client.0.smithi153.stdout:7/713: mknod d4/d2a/d3b/cf3 0 2024-03-21T10:09:00.971 INFO:tasks.workunit.client.0.smithi153.stdout:6/916: dread d1f/d93/dd8/d59/d70/fc1 [0,4194304] 0 2024-03-21T10:09:00.973 INFO:tasks.workunit.client.0.smithi153.stdout:9/797: fsync d1/d43/d44/d46/d9e/d24/d41/dbe/dcc/fb5 0 2024-03-21T10:09:00.973 INFO:tasks.workunit.client.0.smithi153.stdout:6/917: chown l17 517 1 2024-03-21T10:09:00.980 INFO:tasks.workunit.client.0.smithi153.stdout:7/714: dread d4/d2a/dc6/fef [0,4194304] 0 2024-03-21T10:09:00.982 INFO:tasks.workunit.client.0.smithi153.stdout:6/918: read d1f/d2b/d10a/d40/f10d [3961567,100390] 0 2024-03-21T10:09:00.983 INFO:tasks.workunit.client.1.smithi174.stdout:3/598: chown d0/d12/d14/l6a 66 1 2024-03-21T10:09:00.986 INFO:tasks.workunit.client.0.smithi153.stdout:8/733: write d5/d9/d43/dd/d2d/d45/fc1 [187724,96865] 0 2024-03-21T10:09:00.986 INFO:tasks.workunit.client.1.smithi174.stdout:4/658: symlink d4/d21/d24/d28/ld8 0 2024-03-21T10:09:00.987 INFO:tasks.workunit.client.0.smithi153.stdout:7/715: creat d4/d2a/d3b/d60/ff4 x:0 0 0 2024-03-21T10:09:00.987 INFO:tasks.workunit.client.1.smithi174.stdout:3/599: write d0/d12/d14/d1a/d27/d4f/fc1 [899421,80558] 0 2024-03-21T10:09:00.988 INFO:tasks.workunit.client.1.smithi174.stdout:4/659: write d4/f15 [5818205,9278] 0 2024-03-21T10:09:00.989 INFO:tasks.workunit.client.0.smithi153.stdout:5/735: write d4/d8/d5a/d92/f94 [347472,117020] 0 2024-03-21T10:09:00.989 INFO:tasks.workunit.client.0.smithi153.stdout:6/919: fdatasync d1f/d93/dd8/d59/dfc/f116 0 2024-03-21T10:09:00.993 INFO:tasks.workunit.client.0.smithi153.stdout:4/765: dread d11/d18/d50/f8c [0,4194304] 0 2024-03-21T10:09:00.994 INFO:tasks.workunit.client.0.smithi153.stdout:9/798: rmdir d1/d43/d44/d46 39 2024-03-21T10:09:00.996 INFO:tasks.workunit.client.0.smithi153.stdout:7/716: dread - d4/f6c zero size 2024-03-21T10:09:00.997 INFO:tasks.workunit.client.0.smithi153.stdout:3/802: write dd/d1e/d2f/d35/d68/daa/fc7 [79840,28051] 0 2024-03-21T10:09:00.998 INFO:tasks.workunit.client.1.smithi174.stdout:6/444: getdents d0/d3/d5/df 0 2024-03-21T10:09:01.000 INFO:tasks.workunit.client.0.smithi153.stdout:1/603: unlink d6/d10/c96 0 2024-03-21T10:09:01.006 INFO:tasks.workunit.client.0.smithi153.stdout:2/820: dwrite dd/d15/d18/d2c/d4b/d56/d63/d8d/d9b/dc3/df5/dae/f11b [0,4194304] 0 2024-03-21T10:09:01.006 INFO:tasks.workunit.client.0.smithi153.stdout:7/717: dread d4/d2a/dc6/fef [0,4194304] 0 2024-03-21T10:09:01.007 INFO:tasks.workunit.client.0.smithi153.stdout:2/821: dread - dd/d15/ff7 zero size 2024-03-21T10:09:01.009 INFO:tasks.workunit.client.0.smithi153.stdout:7/718: dread - d4/d11/d56/db1/fba zero size 2024-03-21T10:09:01.011 INFO:tasks.workunit.client.0.smithi153.stdout:8/734: write d5/d9/d43/dd/d20/d26/d24/d52/d55/f71 [4132622,24728] 0 2024-03-21T10:09:01.013 INFO:tasks.workunit.client.1.smithi174.stdout:1/663: truncate d4/d7/d32/f3a 3861139 0 2024-03-21T10:09:01.015 INFO:tasks.workunit.client.1.smithi174.stdout:7/504: dread d0/f88 [0,4194304] 0 2024-03-21T10:09:01.016 INFO:tasks.workunit.client.0.smithi153.stdout:7/719: truncate d4/d14/d16/fed 125312 0 2024-03-21T10:09:01.019 INFO:tasks.workunit.client.1.smithi174.stdout:5/637: link d1/d49/fdd d1/d16/d2e/fe9 0 2024-03-21T10:09:01.025 INFO:tasks.workunit.client.1.smithi174.stdout:6/445: rmdir d0/d3/d5/df 39 2024-03-21T10:09:01.025 INFO:tasks.workunit.client.0.smithi153.stdout:9/799: fsync d1/d43/d44/d46/d9e/d24/d41/f5b 0 2024-03-21T10:09:01.026 INFO:tasks.workunit.client.1.smithi174.stdout:1/664: mkdir d4/d7/d33/d43/d51/ded 0 2024-03-21T10:09:01.026 INFO:tasks.workunit.client.0.smithi153.stdout:2/822: truncate dd/d15/fab 663972 0 2024-03-21T10:09:01.027 INFO:tasks.workunit.client.1.smithi174.stdout:6/446: stat d0/d3/d5/d3b/l74 0 2024-03-21T10:09:01.027 INFO:tasks.workunit.client.1.smithi174.stdout:3/600: dread d0/d12/d46/d59/f95 [0,4194304] 0 2024-03-21T10:09:01.028 INFO:tasks.workunit.client.1.smithi174.stdout:2/582: creat d5/d25/d27/d2e/d4f/d65/fc7 x:0 0 0 2024-03-21T10:09:01.031 INFO:tasks.workunit.client.1.smithi174.stdout:5/638: creat d1/d2f/d37/fea x:0 0 0 2024-03-21T10:09:01.032 INFO:tasks.workunit.client.0.smithi153.stdout:9/800: truncate d1/d43/d44/d46/d4d/fb7 779699 0 2024-03-21T10:09:01.033 INFO:tasks.workunit.client.0.smithi153.stdout:2/823: chown dd/d15/d18/d2c/d4b/f66 29 1 2024-03-21T10:09:01.036 INFO:tasks.workunit.client.0.smithi153.stdout:6/920: link d1f/d93/cd5 d1f/d99/dbb/dcf/c145 0 2024-03-21T10:09:01.038 INFO:tasks.workunit.client.0.smithi153.stdout:9/801: write d1/d4e/d68/fb1 [134081,95439] 0 2024-03-21T10:09:01.046 INFO:tasks.workunit.client.0.smithi153.stdout:7/720: truncate d4/d14/dd0/d20/f93 2753424 0 2024-03-21T10:09:01.046 INFO:tasks.workunit.client.0.smithi153.stdout:8/735: write d5/d9/d43/dd/d2d/d45/d88/f6e [4452956,54531] 0 2024-03-21T10:09:01.047 INFO:tasks.workunit.client.0.smithi153.stdout:6/921: write d1f/d2b/d10a/d40/d4f/d66/f13e [44675,13962] 0 2024-03-21T10:09:01.048 INFO:tasks.workunit.client.0.smithi153.stdout:2/824: fdatasync dd/d15/d5e/db3/f105 0 2024-03-21T10:09:01.051 INFO:tasks.workunit.client.0.smithi153.stdout:9/802: chown d1/d43/d44/d46/d4d/d5c/fa9 1 1 2024-03-21T10:09:01.052 INFO:tasks.workunit.client.0.smithi153.stdout:0/689: dread d4/d4c/d83/d22/f7d [0,4194304] 0 2024-03-21T10:09:01.057 INFO:tasks.workunit.client.0.smithi153.stdout:9/803: write d1/d4/fd6 [1029270,86390] 0 2024-03-21T10:09:01.061 INFO:tasks.workunit.client.0.smithi153.stdout:7/721: creat d4/d19/ff5 x:0 0 0 2024-03-21T10:09:01.063 INFO:tasks.workunit.client.0.smithi153.stdout:1/604: dread d6/f17 [4194304,4194304] 0 2024-03-21T10:09:01.065 INFO:tasks.workunit.client.0.smithi153.stdout:8/736: write d5/d9/fee [446375,52549] 0 2024-03-21T10:09:01.066 INFO:tasks.workunit.client.0.smithi153.stdout:9/804: chown d1/d43/d44/d46/d9e/d24/lfe 514992861 1 2024-03-21T10:09:01.071 INFO:tasks.workunit.client.0.smithi153.stdout:4/766: dwrite d11/d2d/f42 [0,4194304] 0 2024-03-21T10:09:01.076 INFO:tasks.workunit.client.0.smithi153.stdout:7/722: chown d4/d19/d4d/dd5 64753 1 2024-03-21T10:09:01.076 INFO:tasks.workunit.client.0.smithi153.stdout:3/803: dwrite dd/d1e/d2f/d35/daf/fb4 [0,4194304] 0 2024-03-21T10:09:01.076 INFO:tasks.workunit.client.0.smithi153.stdout:0/690: mknod d4/d16/d19/d30/d6c/d98/d5e/db2/cda 0 2024-03-21T10:09:01.076 INFO:tasks.workunit.client.0.smithi153.stdout:9/805: fdatasync d1/d4/d5f/df3/ff4 0 2024-03-21T10:09:01.079 INFO:tasks.workunit.client.0.smithi153.stdout:1/605: mknod d6/d10/d35/da5/d4f/d9d/cbd 0 2024-03-21T10:09:01.079 INFO:tasks.workunit.client.0.smithi153.stdout:5/736: dwrite d4/d5/df/d5e/f81 [0,4194304] 0 2024-03-21T10:09:01.085 INFO:tasks.workunit.client.0.smithi153.stdout:4/767: write d11/d18/f95 [1297635,75339] 0 2024-03-21T10:09:01.087 INFO:tasks.workunit.client.0.smithi153.stdout:3/804: mkdir dd/d41/d80/dbb/d112 0 2024-03-21T10:09:01.089 INFO:tasks.workunit.client.0.smithi153.stdout:1/606: stat d6/d13 0 2024-03-21T10:09:01.095 INFO:tasks.workunit.client.0.smithi153.stdout:8/737: rename d5/d9/d43/dd/d20/d26/d8b/c7f to d5/d9/d43/dd/cf1 0 2024-03-21T10:09:01.099 INFO:tasks.workunit.client.0.smithi153.stdout:5/737: mknod d4/d5/df/d67/d6a/cf3 0 2024-03-21T10:09:01.100 INFO:tasks.workunit.client.0.smithi153.stdout:4/768: mknod d11/d18/d54/cf1 0 2024-03-21T10:09:01.102 INFO:tasks.workunit.client.0.smithi153.stdout:4/769: fdatasync d11/d18/d6d/f9b 0 2024-03-21T10:09:01.102 INFO:tasks.workunit.client.0.smithi153.stdout:3/805: rename dd/d1e/d2f/d35/daf/ced to dd/d1e/d3d/d94/d8e/c113 0 2024-03-21T10:09:01.105 INFO:tasks.workunit.client.0.smithi153.stdout:2/825: creat dd/d15/d18/d2c/d4b/d56/d63/d8d/d9b/dc3/df5/dae/d50/f121 x:0 0 0 2024-03-21T10:09:01.105 INFO:tasks.workunit.client.0.smithi153.stdout:4/770: stat d11/d2d/f9e 0 2024-03-21T10:09:01.105 INFO:tasks.workunit.client.0.smithi153.stdout:1/607: symlink d6/d13/d75/db5/lbe 0 2024-03-21T10:09:01.111 INFO:tasks.workunit.client.0.smithi153.stdout:1/608: write d6/d13/f7c [158739,3552] 0 2024-03-21T10:09:01.111 INFO:tasks.workunit.client.0.smithi153.stdout:8/738: rename d5/d9/d43/dd/ddd/f91 to d5/d9/d43/dd/d20/d26/d24/d52/d55/ff2 0 2024-03-21T10:09:01.114 INFO:tasks.workunit.client.0.smithi153.stdout:7/723: dread d4/d14/dd0/d20/f6a [0,4194304] 0 2024-03-21T10:09:01.114 INFO:tasks.workunit.client.0.smithi153.stdout:2/826: fsync dd/d15/d18/d2c/d4b/d56/d63/ff9 0 2024-03-21T10:09:01.116 INFO:tasks.workunit.client.0.smithi153.stdout:5/738: fsync d4/d8/d5a/d5d/dd9/d44/d48/f84 0 2024-03-21T10:09:01.116 INFO:tasks.workunit.client.0.smithi153.stdout:4/771: read d11/d18/f41 [2792308,7549] 0 2024-03-21T10:09:01.119 INFO:tasks.workunit.client.0.smithi153.stdout:9/806: dwrite d1/d3b/daa/fea [0,4194304] 0 2024-03-21T10:09:01.120 INFO:tasks.workunit.client.0.smithi153.stdout:7/724: chown d4/d2a/d40/d50/fca 417724 1 2024-03-21T10:09:01.120 INFO:tasks.workunit.client.0.smithi153.stdout:5/739: readlink d4/d5/df/d5e/d69/l6b 0 2024-03-21T10:09:01.122 INFO:tasks.workunit.client.0.smithi153.stdout:6/922: dread d1f/d2b/f63 [0,4194304] 0 2024-03-21T10:09:01.123 INFO:tasks.workunit.client.0.smithi153.stdout:0/691: link d4/la4 d4/d16/d27/dc5/ldb 0 2024-03-21T10:09:01.124 INFO:tasks.workunit.client.0.smithi153.stdout:2/827: unlink dd/c6a 0 2024-03-21T10:09:01.124 INFO:tasks.workunit.client.0.smithi153.stdout:8/739: rmdir d5/d9/d43/dd/d20/d26 39 2024-03-21T10:09:01.125 INFO:tasks.workunit.client.0.smithi153.stdout:4/772: chown d11/d18/d7e/c9f 515440 1 2024-03-21T10:09:01.130 INFO:tasks.workunit.client.0.smithi153.stdout:0/692: symlink d4/d16/d19/d95/ldc 0 2024-03-21T10:09:01.138 INFO:tasks.workunit.client.0.smithi153.stdout:7/725: dread d4/d19/fb6 [0,4194304] 0 2024-03-21T10:09:01.141 INFO:tasks.workunit.client.0.smithi153.stdout:4/773: dread d11/d18/d50/f8c [0,4194304] 0 2024-03-21T10:09:01.155 INFO:tasks.workunit.client.0.smithi153.stdout:4/774: dwrite d11/f96 [0,4194304] 0 2024-03-21T10:09:01.156 INFO:tasks.workunit.client.0.smithi153.stdout:9/807: dread d1/d43/d44/f3e [0,4194304] 0 2024-03-21T10:09:01.164 INFO:tasks.workunit.client.0.smithi153.stdout:6/923: dread d1f/d2b/d10a/d40/d65/fe9 [0,4194304] 0 2024-03-21T10:09:01.167 INFO:tasks.workunit.client.1.smithi174.stdout:8/514: truncate d9/d2a/d39/d76/d19/d27/f8a 629129 0 2024-03-21T10:09:01.167 INFO:tasks.workunit.client.1.smithi174.stdout:5/639: readlink d1/d16/d2e/d87/l94 0 2024-03-21T10:09:01.167 INFO:tasks.workunit.client.1.smithi174.stdout:3/601: read - d0/d12/d14/d1a/fc2 zero size 2024-03-21T10:09:01.169 INFO:tasks.workunit.client.1.smithi174.stdout:8/515: fdatasync f8 0 2024-03-21T10:09:01.171 INFO:tasks.workunit.client.1.smithi174.stdout:1/665: creat d4/d7/d32/fee x:0 0 0 2024-03-21T10:09:01.175 INFO:tasks.workunit.client.1.smithi174.stdout:6/447: rename d0/d3/d5/d6/lc to d0/d3/d5/d6/d42/l89 0 2024-03-21T10:09:01.176 INFO:tasks.workunit.client.1.smithi174.stdout:3/602: fsync d0/d12/d14/d1a/d27/d3c/d73/fbb 0 2024-03-21T10:09:01.179 INFO:tasks.workunit.client.1.smithi174.stdout:4/660: dwrite d4/d21/d31/d3a/d67/d76/fbd [4194304,4194304] 0 2024-03-21T10:09:01.179 INFO:tasks.workunit.client.1.smithi174.stdout:1/666: creat d4/d7/d68/d7b/de0/fef x:0 0 0 2024-03-21T10:09:01.185 INFO:tasks.workunit.client.1.smithi174.stdout:6/448: rmdir d0/d3/d5/d3b 39 2024-03-21T10:09:01.187 INFO:tasks.workunit.client.1.smithi174.stdout:3/603: dread - d0/f5f zero size 2024-03-21T10:09:01.191 INFO:tasks.workunit.client.1.smithi174.stdout:4/661: rename d4/d21/d24/c83 to d4/d21/d24/d28/cd9 0 2024-03-21T10:09:01.191 INFO:tasks.workunit.client.1.smithi174.stdout:2/583: dwrite d5/d25/d27/d2d/d45/f6e [0,4194304] 0 2024-03-21T10:09:01.192 INFO:tasks.workunit.client.1.smithi174.stdout:3/604: write d0/d12/f49 [3643830,91002] 0 2024-03-21T10:09:01.207 INFO:tasks.workunit.client.1.smithi174.stdout:5/640: truncate d1/ff 8964728 0 2024-03-21T10:09:01.220 INFO:tasks.workunit.client.1.smithi174.stdout:4/662: dwrite d4/fbf [0,4194304] 0 2024-03-21T10:09:01.225 INFO:tasks.workunit.client.1.smithi174.stdout:5/641: link d1/d16/d2e/fe9 d1/d49/d93/d9c/feb 0 2024-03-21T10:09:01.232 INFO:tasks.workunit.client.1.smithi174.stdout:6/449: dwrite d0/d3/d5/df/d16/f67 [0,4194304] 0 2024-03-21T10:09:01.237 INFO:tasks.workunit.client.1.smithi174.stdout:8/516: dread d9/d53/d5b/f77 [0,4194304] 0 2024-03-21T10:09:01.240 INFO:tasks.workunit.client.0.smithi153.stdout:0/693: unlink d4/d4c/d83/d22/cb8 0 2024-03-21T10:09:01.245 INFO:tasks.workunit.client.1.smithi174.stdout:4/663: rename d4/d19/f50 to d4/d21/d31/d3a/d77/fda 0 2024-03-21T10:09:01.248 INFO:tasks.workunit.client.0.smithi153.stdout:6/924: chown d1f/d2b/d10a/d40/d5f/l88 6 1 2024-03-21T10:09:01.251 INFO:tasks.workunit.client.1.smithi174.stdout:1/667: dread d4/f19 [0,4194304] 0 2024-03-21T10:09:01.252 INFO:tasks.workunit.client.0.smithi153.stdout:8/740: mkdir d5/d9/d43/dd/d20/df3 0 2024-03-21T10:09:01.255 INFO:tasks.workunit.client.1.smithi174.stdout:8/517: mkdir d9/d2a/d39/d76/d19/d27/d60/daa 0 2024-03-21T10:09:01.255 INFO:tasks.workunit.client.1.smithi174.stdout:2/584: dread d5/f12 [0,4194304] 0 2024-03-21T10:09:01.256 INFO:tasks.workunit.client.1.smithi174.stdout:8/518: fdatasync f5 0 2024-03-21T10:09:01.256 INFO:tasks.workunit.client.0.smithi153.stdout:7/726: mkdir d4/d2a/d3b/d81/df6 0 2024-03-21T10:09:01.260 INFO:tasks.workunit.client.1.smithi174.stdout:4/664: creat d4/d21/d31/d3a/d67/d76/fdb x:0 0 0 2024-03-21T10:09:01.260 INFO:tasks.workunit.client.0.smithi153.stdout:1/609: link d6/d13/l38 d6/d10/d35/da5/d4f/lbf 0 2024-03-21T10:09:01.260 INFO:tasks.workunit.client.1.smithi174.stdout:8/519: chown d9/d2a/d39/d76/d19/d27/d60/f9f 0 1 2024-03-21T10:09:01.262 INFO:tasks.workunit.client.1.smithi174.stdout:4/665: write d4/d19/f2d [1119462,92224] 0 2024-03-21T10:09:01.263 INFO:tasks.workunit.client.1.smithi174.stdout:1/668: mknod d4/d7/d70/cf0 0 2024-03-21T10:09:01.264 INFO:tasks.workunit.client.0.smithi153.stdout:1/610: chown d6/d10/d35/da5/d4f/l89 91786 1 2024-03-21T10:09:01.266 INFO:tasks.workunit.client.0.smithi153.stdout:6/925: creat d1f/d2b/d10a/d40/d7b/d130/f146 x:0 0 0 2024-03-21T10:09:01.266 INFO:tasks.workunit.client.0.smithi153.stdout:4/775: write d11/d9d/fbd [4366164,8595] 0 2024-03-21T10:09:01.270 INFO:tasks.workunit.client.1.smithi174.stdout:1/669: write d4/d7/d28/f30 [1565687,6870] 0 2024-03-21T10:09:01.271 INFO:tasks.workunit.client.1.smithi174.stdout:8/520: write d9/d53/f78 [488562,115006] 0 2024-03-21T10:09:01.271 INFO:tasks.workunit.client.0.smithi153.stdout:6/926: stat d1f/d2b/d10a/d40/l79 0 2024-03-21T10:09:01.274 INFO:tasks.workunit.client.1.smithi174.stdout:8/521: write f5 [1224103,105907] 0 2024-03-21T10:09:01.275 INFO:tasks.workunit.client.0.smithi153.stdout:8/741: dread d5/d9/d43/dd/d2d/fa5 [0,4194304] 0 2024-03-21T10:09:01.275 INFO:tasks.workunit.client.0.smithi153.stdout:4/776: truncate d11/d18/f49 1185317 0 2024-03-21T10:09:01.279 INFO:tasks.workunit.client.1.smithi174.stdout:4/666: rename d4/d21/d31/d3a/d67/la2 to d4/d21/d24/d28/d79/ldc 0 2024-03-21T10:09:01.283 INFO:tasks.workunit.client.0.smithi153.stdout:7/727: write d4/d2a/d3b/d81/f90 [652127,115199] 0 2024-03-21T10:09:01.286 INFO:tasks.workunit.client.0.smithi153.stdout:6/927: read - d1f/d2b/d10a/d40/d7b/d130/f146 zero size 2024-03-21T10:09:01.290 INFO:tasks.workunit.client.1.smithi174.stdout:8/522: mkdir d9/d2a/d39/d76/d19/d27/d60/daa/dab 0 2024-03-21T10:09:01.292 INFO:tasks.workunit.client.0.smithi153.stdout:0/694: dread d4/d16/d19/d58/da1/fd8 [0,4194304] 0 2024-03-21T10:09:01.293 INFO:tasks.workunit.client.1.smithi174.stdout:8/523: read d9/d2a/d39/d76/d19/d31/f94 [1184816,63827] 0 2024-03-21T10:09:01.293 INFO:tasks.workunit.client.1.smithi174.stdout:1/670: getdents d4/d7/d28 0 2024-03-21T10:09:01.297 INFO:tasks.workunit.client.0.smithi153.stdout:2/828: dwrite dd/d15/d18/d2c/d4b/d56/d63/d8d/f110 [4194304,4194304] 0 2024-03-21T10:09:01.299 INFO:tasks.workunit.client.0.smithi153.stdout:5/740: sync 2024-03-21T10:09:01.300 INFO:tasks.workunit.client.0.smithi153.stdout:3/806: sync 2024-03-21T10:09:01.300 INFO:tasks.workunit.client.0.smithi153.stdout:9/808: dwrite d1/d43/d44/f58 [0,4194304] 0 2024-03-21T10:09:01.300 INFO:tasks.workunit.client.1.smithi174.stdout:2/585: dread d5/d25/d27/f8b [0,4194304] 0 2024-03-21T10:09:01.301 INFO:tasks.workunit.client.1.smithi174.stdout:6/450: dwrite d0/d3/d5/d21/f6c [0,4194304] 0 2024-03-21T10:09:01.302 INFO:tasks.workunit.client.1.smithi174.stdout:4/667: dwrite d4/d19/f65 [0,4194304] 0 2024-03-21T10:09:01.302 INFO:tasks.workunit.client.0.smithi153.stdout:9/809: stat d1/d43/d44/f25 0 2024-03-21T10:09:01.304 INFO:tasks.workunit.client.0.smithi153.stdout:1/611: dread d6/fb [0,4194304] 0 2024-03-21T10:09:01.306 INFO:tasks.workunit.client.0.smithi153.stdout:4/777: creat d11/d5e/ff2 x:0 0 0 2024-03-21T10:09:01.309 INFO:tasks.workunit.client.1.smithi174.stdout:8/524: rename d9/d2a/d39/d76/d19/d27/f84 to d9/d2a/d39/d76/d19/d31/fac 0 2024-03-21T10:09:01.314 INFO:tasks.workunit.client.1.smithi174.stdout:4/668: mkdir d4/d21/d31/ddd 0 2024-03-21T10:09:01.314 INFO:tasks.workunit.client.0.smithi153.stdout:0/695: unlink d4/d16/d19/d30/d6c/d98/c6e 0 2024-03-21T10:09:01.314 INFO:tasks.workunit.client.1.smithi174.stdout:2/586: mknod d5/d25/d27/d2d/d45/d5b/d2f/cc8 0 2024-03-21T10:09:01.316 INFO:tasks.workunit.client.0.smithi153.stdout:5/741: creat d4/d5/df/d67/d86/ff4 x:0 0 0 2024-03-21T10:09:01.319 INFO:tasks.workunit.client.0.smithi153.stdout:9/810: dread - d1/d4/d5f/db6/fe2 zero size 2024-03-21T10:09:01.319 INFO:tasks.workunit.client.0.smithi153.stdout:5/742: chown d4/d8/d5a/d5d/dd9/d38 121 1 2024-03-21T10:09:01.319 INFO:tasks.workunit.client.1.smithi174.stdout:8/525: unlink d9/d2a/d39/d76/d19/f23 0 2024-03-21T10:09:01.322 INFO:tasks.workunit.client.1.smithi174.stdout:6/451: truncate d0/d3/d5/d6/d42/f58 270522 0 2024-03-21T10:09:01.322 INFO:tasks.workunit.client.0.smithi153.stdout:5/743: chown d4/d5/df/d67/d86/fd7 15564674 1 2024-03-21T10:09:01.322 INFO:tasks.workunit.client.0.smithi153.stdout:9/811: dread - d1/d43/d44/d46/d9e/d24/d41/dbe/dcc/fb5 zero size 2024-03-21T10:09:01.324 INFO:tasks.workunit.client.0.smithi153.stdout:8/742: mkdir d5/d9/d43/dd/d20/d26/d47/df4 0 2024-03-21T10:09:01.332 INFO:tasks.workunit.client.0.smithi153.stdout:0/696: fsync d4/f23 0 2024-03-21T10:09:01.332 INFO:tasks.workunit.client.0.smithi153.stdout:2/829: mkdir dd/d15/d18/d2c/d4b/d56/d63/d8d/d9b/dc3/df5/dae/d50/d122 0 2024-03-21T10:09:01.332 INFO:tasks.workunit.client.1.smithi174.stdout:6/452: chown d0/d3/d5/d6/f6a 257342702 1 2024-03-21T10:09:01.333 INFO:tasks.workunit.client.1.smithi174.stdout:4/669: truncate d4/d21/d24/d28/d38/f88 2443216 0 2024-03-21T10:09:01.335 INFO:tasks.workunit.client.1.smithi174.stdout:1/671: dwrite d4/d7/d28/f30 [4194304,4194304] 0 2024-03-21T10:09:01.336 INFO:tasks.workunit.client.0.smithi153.stdout:6/928: rename d1f/d2b/d10a/d40/d4f/f83 to d1f/d93/dd8/d59/d70/f147 0 2024-03-21T10:09:01.339 INFO:tasks.workunit.client.0.smithi153.stdout:9/812: creat d1/d4/d5f/df3/f10c x:0 0 0 2024-03-21T10:09:01.339 INFO:tasks.workunit.client.0.smithi153.stdout:3/807: link dd/d1e/d45/c88 dd/d41/dab/c114 0 2024-03-21T10:09:01.340 INFO:tasks.workunit.client.0.smithi153.stdout:7/728: getdents d4/d14/dd0/d20/d9a/dd2/db9 0 2024-03-21T10:09:01.340 INFO:tasks.workunit.client.0.smithi153.stdout:5/744: mknod d4/d5/df/cf5 0 2024-03-21T10:09:01.340 INFO:tasks.workunit.client.1.smithi174.stdout:4/670: write d4/f15 [2734986,69752] 0 2024-03-21T10:09:01.341 INFO:tasks.workunit.client.0.smithi153.stdout:0/697: fdatasync d4/d4c/d83/d22/f73 0 2024-03-21T10:09:01.341 INFO:tasks.workunit.client.1.smithi174.stdout:1/672: readlink d4/d7/d33/d43/d51/l7e 0 2024-03-21T10:09:01.342 INFO:tasks.workunit.client.0.smithi153.stdout:2/830: readlink dd/d15/d18/d2c/d4b/d56/d63/d8d/d9b/l9f 0 2024-03-21T10:09:01.344 INFO:tasks.workunit.client.0.smithi153.stdout:3/808: chown dd/d41/f49 7252456 1 2024-03-21T10:09:01.344 INFO:tasks.workunit.client.0.smithi153.stdout:5/745: read - d4/fe3 zero size 2024-03-21T10:09:01.348 INFO:tasks.workunit.client.0.smithi153.stdout:4/778: rename d11/d2d/d57/caf to d11/d18/d7e/cf3 0 2024-03-21T10:09:01.349 INFO:tasks.workunit.client.1.smithi174.stdout:8/526: dread d9/d53/f9d [0,4194304] 0 2024-03-21T10:09:01.351 INFO:tasks.workunit.client.0.smithi153.stdout:6/929: creat d1f/d2b/f148 x:0 0 0 2024-03-21T10:09:01.353 INFO:tasks.workunit.client.1.smithi174.stdout:4/671: write d4/d21/d31/d6a/dab/da8/fc3 [4064416,8733] 0 2024-03-21T10:09:01.356 INFO:tasks.workunit.client.0.smithi153.stdout:3/809: write dd/f1f [4060057,3164] 0 2024-03-21T10:09:01.357 INFO:tasks.workunit.client.1.smithi174.stdout:4/672: rmdir d4/d21/d31/d6a 39 2024-03-21T10:09:01.358 INFO:tasks.workunit.client.1.smithi174.stdout:8/527: creat d9/d2a/d39/fad x:0 0 0 2024-03-21T10:09:01.359 INFO:tasks.workunit.client.1.smithi174.stdout:1/673: getdents d4/d7/d32/da2 0 2024-03-21T10:09:01.361 INFO:tasks.workunit.client.1.smithi174.stdout:9/601: sync 2024-03-21T10:09:01.363 INFO:tasks.workunit.client.0.smithi153.stdout:9/813: rename d1/ld7 to d1/d43/d44/d46/d9e/d24/d41/l10d 0 2024-03-21T10:09:01.367 INFO:tasks.workunit.client.0.smithi153.stdout:0/698: mknod d4/d16/d27/cdd 0 2024-03-21T10:09:01.368 INFO:tasks.workunit.client.1.smithi174.stdout:1/674: mkdir d4/d7/d32/d7d/df1 0 2024-03-21T10:09:01.368 INFO:tasks.workunit.client.0.smithi153.stdout:7/729: dwrite d4/d2a/d40/d50/fa5 [0,4194304] 0 2024-03-21T10:09:01.371 INFO:tasks.workunit.client.1.smithi174.stdout:2/587: dread d5/d25/d27/d2d/d45/d5b/d63/f48 [0,4194304] 0 2024-03-21T10:09:01.375 INFO:tasks.workunit.client.0.smithi153.stdout:4/779: creat d11/ff4 x:0 0 0 2024-03-21T10:09:01.377 INFO:tasks.workunit.client.1.smithi174.stdout:8/528: getdents d9/d2a/d39/d76/d19/d27/d60/daa 0 2024-03-21T10:09:01.380 INFO:tasks.workunit.client.1.smithi174.stdout:9/602: truncate d2/d24/f88 579887 0 2024-03-21T10:09:01.383 INFO:tasks.workunit.client.0.smithi153.stdout:0/699: mknod d4/d16/d19/d30/d6c/d98/d5e/db2/cde 0 2024-03-21T10:09:01.383 INFO:tasks.workunit.client.0.smithi153.stdout:9/814: read d1/d43/d44/d46/f1d [2349762,38687] 0 2024-03-21T10:09:01.384 INFO:tasks.workunit.client.1.smithi174.stdout:1/675: rename d4/d2d/f34 to d4/d7/d33/d43/d77/d87/d92/ff2 0 2024-03-21T10:09:01.386 INFO:tasks.workunit.client.1.smithi174.stdout:2/588: symlink d5/d25/d27/d2d/d45/d5b/d2f/lc9 0 2024-03-21T10:09:01.387 INFO:tasks.workunit.client.1.smithi174.stdout:8/529: creat d9/fae x:0 0 0 2024-03-21T10:09:01.389 INFO:tasks.workunit.client.0.smithi153.stdout:1/612: dwrite d6/d13/f1d [0,4194304] 0 2024-03-21T10:09:01.391 INFO:tasks.workunit.client.1.smithi174.stdout:1/676: creat d4/d7/d32/da2/ff3 x:0 0 0 2024-03-21T10:09:01.397 INFO:tasks.workunit.client.1.smithi174.stdout:9/603: creat d2/d24/fd5 x:0 0 0 2024-03-21T10:09:01.397 INFO:tasks.workunit.client.1.smithi174.stdout:4/673: dread d4/d19/f74 [0,4194304] 0 2024-03-21T10:09:01.402 INFO:tasks.workunit.client.0.smithi153.stdout:5/746: write d4/d8/f68 [1993721,82561] 0 2024-03-21T10:09:01.404 INFO:tasks.workunit.client.1.smithi174.stdout:8/530: link d9/d2a/d39/f5f d9/d53/d7e/faf 0 2024-03-21T10:09:01.405 INFO:tasks.workunit.client.1.smithi174.stdout:9/604: dread - d2/d11/f3a zero size 2024-03-21T10:09:01.408 INFO:tasks.workunit.client.0.smithi153.stdout:3/810: dread dd/d1e/d2f/f90 [0,4194304] 0 2024-03-21T10:09:01.409 INFO:tasks.workunit.client.0.smithi153.stdout:9/815: creat d1/d43/d44/d46/d9e/d24/f10e x:0 0 0 2024-03-21T10:09:01.411 INFO:tasks.workunit.client.1.smithi174.stdout:8/531: fdatasync d9/d2a/d39/d76/f6a 0 2024-03-21T10:09:01.411 INFO:tasks.workunit.client.0.smithi153.stdout:6/930: write d1f/d99/dbb/f114 [1801530,24099] 0 2024-03-21T10:09:01.412 INFO:tasks.workunit.client.1.smithi174.stdout:4/674: symlink d4/d21/d31/d3a/d77/dcf/lde 0 2024-03-21T10:09:01.413 INFO:tasks.workunit.client.0.smithi153.stdout:1/613: creat d6/d10/d35/d5b/d92/fc0 x:0 0 0 2024-03-21T10:09:01.414 INFO:tasks.workunit.client.1.smithi174.stdout:9/605: symlink d2/d11/d1d/d2a/da7/ld6 0 2024-03-21T10:09:01.414 INFO:tasks.workunit.client.1.smithi174.stdout:4/675: chown d4/d19/l30 595 1 2024-03-21T10:09:01.415 INFO:tasks.workunit.client.1.smithi174.stdout:1/677: write d4/d7/d15/f39 [540958,116594] 0 2024-03-21T10:09:01.418 INFO:tasks.workunit.client.1.smithi174.stdout:9/606: chown d2/d11/d1d/f80 44109414 1 2024-03-21T10:09:01.418 INFO:tasks.workunit.client.0.smithi153.stdout:3/811: write dd/d41/d80/dbb/def/ffe [445596,53642] 0 2024-03-21T10:09:01.419 INFO:tasks.workunit.client.0.smithi153.stdout:9/816: creat d1/d4/d5f/df3/f10f x:0 0 0 2024-03-21T10:09:01.419 INFO:tasks.workunit.client.1.smithi174.stdout:4/676: symlink d4/d21/d24/ldf 0 2024-03-21T10:09:01.421 INFO:tasks.workunit.client.0.smithi153.stdout:0/700: dwrite d4/d16/d27/f5c [0,4194304] 0 2024-03-21T10:09:01.421 INFO:tasks.workunit.client.1.smithi174.stdout:8/532: unlink d9/ca3 0 2024-03-21T10:09:01.423 INFO:tasks.workunit.client.1.smithi174.stdout:9/607: mknod d2/d11/d1d/d2a/da7/cd7 0 2024-03-21T10:09:01.425 INFO:tasks.workunit.client.0.smithi153.stdout:5/747: rename d4/d5/df/d67/da0/db3/l98 to d4/d8/d5c/lf6 0 2024-03-21T10:09:01.428 INFO:tasks.workunit.client.1.smithi174.stdout:9/608: write d2/d11/d60/f86 [3684852,108607] 0 2024-03-21T10:09:01.428 INFO:tasks.workunit.client.0.smithi153.stdout:9/817: symlink d1/d43/d44/d46/d9e/d2e/l110 0 2024-03-21T10:09:01.431 INFO:tasks.workunit.client.0.smithi153.stdout:5/748: write d4/d5/df/d67/d6a/fc3 [893731,128684] 0 2024-03-21T10:09:01.431 INFO:tasks.workunit.client.0.smithi153.stdout:4/780: write d11/d2d/fc1 [677509,6914] 0 2024-03-21T10:09:01.432 INFO:tasks.workunit.client.1.smithi174.stdout:1/678: link d4/d2d/fae d4/d7/d33/d43/d77/d87/d92/ff4 0 2024-03-21T10:09:01.433 INFO:tasks.workunit.client.0.smithi153.stdout:6/931: dread f18 [0,4194304] 0 2024-03-21T10:09:01.434 INFO:tasks.workunit.client.0.smithi153.stdout:5/749: dread - d4/d8/d5c/d90/fe6 zero size 2024-03-21T10:09:01.436 INFO:tasks.workunit.client.1.smithi174.stdout:9/609: symlink d2/d11/d1d/d4b/ld8 0 2024-03-21T10:09:01.438 INFO:tasks.workunit.client.0.smithi153.stdout:5/750: chown d4/d8/d5a/c3d 7531 1 2024-03-21T10:09:01.439 INFO:tasks.workunit.client.0.smithi153.stdout:1/614: creat d6/d10/fc1 x:0 0 0 2024-03-21T10:09:01.440 INFO:tasks.workunit.client.1.smithi174.stdout:1/679: creat d4/d7/d33/d43/ff5 x:0 0 0 2024-03-21T10:09:01.440 INFO:tasks.workunit.client.0.smithi153.stdout:4/781: dread - d11/d2d/d57/f88 zero size 2024-03-21T10:09:01.441 INFO:tasks.workunit.client.1.smithi174.stdout:4/677: dwrite d4/d21/d24/d28/d38/f5f [0,4194304] 0 2024-03-21T10:09:01.442 INFO:tasks.workunit.client.0.smithi153.stdout:0/701: creat d4/d16/d19/d30/d6c/d98/d5e/dbe/fdf x:0 0 0 2024-03-21T10:09:01.443 INFO:tasks.workunit.client.0.smithi153.stdout:4/782: dread - d11/d9d/fd7 zero size 2024-03-21T10:09:01.444 INFO:tasks.workunit.client.1.smithi174.stdout:1/680: symlink d4/d7/d68/d89/lf6 0 2024-03-21T10:09:01.447 INFO:tasks.workunit.client.0.smithi153.stdout:1/615: dread d6/d3a/d3c/fa7 [0,4194304] 0 2024-03-21T10:09:01.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:01 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:09:01.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:01 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:09:01.451 INFO:tasks.workunit.client.1.smithi174.stdout:9/610: rename d2/d24/d2e/d63/c82 to d2/d24/cd9 0 2024-03-21T10:09:01.452 INFO:tasks.workunit.client.0.smithi153.stdout:6/932: rename d1f/d2b/d10a/d40/d51/d98/ld2 to d1f/d2b/d10a/d40/d7b/d130/l149 0 2024-03-21T10:09:01.454 INFO:tasks.workunit.client.1.smithi174.stdout:8/533: dwrite d9/d53/f96 [0,4194304] 0 2024-03-21T10:09:01.455 INFO:tasks.workunit.client.1.smithi174.stdout:4/678: symlink d4/d21/d31/d3a/d7a/d9d/le0 0 2024-03-21T10:09:01.455 INFO:tasks.workunit.client.1.smithi174.stdout:1/681: write d4/d7/d28/d59/f66 [690856,121924] 0 2024-03-21T10:09:01.456 INFO:tasks.workunit.client.0.smithi153.stdout:0/702: creat d4/d4c/d83/d22/d61/fe0 x:0 0 0 2024-03-21T10:09:01.458 INFO:tasks.workunit.client.1.smithi174.stdout:4/679: stat d4/d21/d31/d3a/d67/d76/fdb 0 2024-03-21T10:09:01.459 INFO:tasks.workunit.client.0.smithi153.stdout:5/751: creat d4/d8/d5a/d5d/dd9/ff7 x:0 0 0 2024-03-21T10:09:01.464 INFO:tasks.workunit.client.1.smithi174.stdout:9/611: symlink d2/d24/lda 0 2024-03-21T10:09:01.466 INFO:tasks.workunit.client.0.smithi153.stdout:6/933: creat d1f/d2b/d10a/d40/d5f/f14a x:0 0 0 2024-03-21T10:09:01.468 INFO:tasks.workunit.client.0.smithi153.stdout:3/812: write dd/f4c [785049,95018] 0 2024-03-21T10:09:01.473 INFO:tasks.workunit.client.0.smithi153.stdout:5/752: creat d4/ff8 x:0 0 0 2024-03-21T10:09:01.476 INFO:tasks.workunit.client.0.smithi153.stdout:6/934: stat d1f/d93/dd8/d59/l97 0 2024-03-21T10:09:01.478 INFO:tasks.workunit.client.0.smithi153.stdout:3/813: creat dd/d1e/d3d/d94/f115 x:0 0 0 2024-03-21T10:09:01.480 INFO:tasks.workunit.client.0.smithi153.stdout:1/616: link d6/d10/d35/da5/d4f/lbf d6/d10/d35/d5b/d92/lc2 0 2024-03-21T10:09:01.481 INFO:tasks.workunit.client.1.smithi174.stdout:1/682: dwrite d4/d7/d68/d89/fb3 [0,4194304] 0 2024-03-21T10:09:01.482 INFO:tasks.workunit.client.1.smithi174.stdout:4/680: dwrite d4/d21/d24/d28/f5e [0,4194304] 0 2024-03-21T10:09:01.483 INFO:tasks.workunit.client.0.smithi153.stdout:4/783: dwrite d11/fb0 [0,4194304] 0 2024-03-21T10:09:01.488 INFO:tasks.workunit.client.1.smithi174.stdout:4/681: chown d4/d21/d31/d3a/d67/fb9 0 1 2024-03-21T10:09:01.490 INFO:tasks.workunit.client.0.smithi153.stdout:1/617: mknod d6/d3a/d66/d80/cc3 0 2024-03-21T10:09:01.493 INFO:tasks.workunit.client.0.smithi153.stdout:5/753: rename d4/d5/df/d5e/d69/f6d to d4/d8/d5a/d5d/dd9/d44/ff9 0 2024-03-21T10:09:01.493 INFO:tasks.workunit.client.0.smithi153.stdout:0/703: dwrite d4/d16/d19/d95/fb5 [0,4194304] 0 2024-03-21T10:09:01.494 INFO:tasks.workunit.client.1.smithi174.stdout:9/612: dwrite d2/d11/d1d/f96 [4194304,4194304] 0 2024-03-21T10:09:01.498 INFO:tasks.workunit.client.1.smithi174.stdout:1/683: link d4/d7/ce d4/d7/d68/d89/dc6/cf7 0 2024-03-21T10:09:01.499 INFO:tasks.workunit.client.0.smithi153.stdout:0/704: fsync d4/d16/d19/fa2 0 2024-03-21T10:09:01.499 INFO:tasks.workunit.client.1.smithi174.stdout:4/682: write d4/d19/f74 [3245274,6890] 0 2024-03-21T10:09:01.501 INFO:tasks.workunit.client.0.smithi153.stdout:1/618: dread d6/d32/f74 [0,4194304] 0 2024-03-21T10:09:01.502 INFO:tasks.workunit.client.1.smithi174.stdout:9/613: chown d2/f4 13 1 2024-03-21T10:09:01.504 INFO:tasks.workunit.client.0.smithi153.stdout:0/705: fsync d4/d4c/d83/d22/f73 0 2024-03-21T10:09:01.506 INFO:tasks.workunit.client.1.smithi174.stdout:1/684: truncate d4/d5/d6c/fd1 549090 0 2024-03-21T10:09:01.506 INFO:tasks.workunit.client.1.smithi174.stdout:8/534: dwrite d9/d2a/d39/d76/f6a [0,4194304] 0 2024-03-21T10:09:01.509 INFO:tasks.workunit.client.0.smithi153.stdout:0/706: chown d4/d4c/d83/fd5 6189 1 2024-03-21T10:09:01.511 INFO:tasks.workunit.client.0.smithi153.stdout:5/754: mkdir d4/d5/df/d67/d86/d96/deb/df0/dfa 0 2024-03-21T10:09:01.511 INFO:tasks.workunit.client.1.smithi174.stdout:9/614: creat d2/d11/d60/fdb x:0 0 0 2024-03-21T10:09:01.513 INFO:tasks.workunit.client.1.smithi174.stdout:8/535: rename d9 to d9/d2a/d39/d76/d19/d27/d60/daa/dab/db0 22 2024-03-21T10:09:01.516 INFO:tasks.workunit.client.0.smithi153.stdout:0/707: dread - d4/db/fd4 zero size 2024-03-21T10:09:01.518 INFO:tasks.workunit.client.0.smithi153.stdout:1/619: link d6/d10/d2c/f7e d6/d10/da0/fc4 0 2024-03-21T10:09:01.519 INFO:tasks.workunit.client.0.smithi153.stdout:5/755: fsync d4/d5/d16/f8a 0 2024-03-21T10:09:01.526 INFO:tasks.workunit.client.1.smithi174.stdout:1/685: getdents d4/d7/d68/d7b/de0 0 2024-03-21T10:09:01.526 INFO:tasks.workunit.client.1.smithi174.stdout:9/615: mknod d2/d11/cdc 0 2024-03-21T10:09:01.528 INFO:tasks.workunit.client.0.smithi153.stdout:5/756: mkdir d4/d8/d5a/d5d/dd9/d44/de1/dfb 0 2024-03-21T10:09:01.528 INFO:tasks.workunit.client.1.smithi174.stdout:1/686: write d4/d5/f72 [3952436,39820] 0 2024-03-21T10:09:01.531 INFO:tasks.workunit.client.1.smithi174.stdout:0/579: sync 2024-03-21T10:09:01.532 INFO:tasks.workunit.client.1.smithi174.stdout:9/616: symlink d2/d24/d2e/d7e/ldd 0 2024-03-21T10:09:01.540 INFO:tasks.workunit.client.0.smithi153.stdout:6/935: write d1f/d2b/d10a/f3c [1378958,25044] 0 2024-03-21T10:09:01.541 INFO:tasks.workunit.client.1.smithi174.stdout:4/683: dwrite d4/d21/fc8 [0,4194304] 0 2024-03-21T10:09:01.541 INFO:tasks.workunit.client.1.smithi174.stdout:1/687: rename d4/le9 to d4/d7/d33/d43/d51/da4/lf8 0 2024-03-21T10:09:01.545 INFO:tasks.workunit.client.0.smithi153.stdout:6/936: stat d1f/d2b/d10a/d40/d5f/dac 0 2024-03-21T10:09:01.545 INFO:tasks.workunit.client.1.smithi174.stdout:9/617: creat d2/d11/d7d/fde x:0 0 0 2024-03-21T10:09:01.549 INFO:tasks.workunit.client.0.smithi153.stdout:0/708: dwrite d4/d16/d9f/fbb [0,4194304] 0 2024-03-21T10:09:01.550 INFO:tasks.workunit.client.1.smithi174.stdout:4/684: chown d4/d21/d31/d3a/d67/fcc 8022742 1 2024-03-21T10:09:01.553 INFO:tasks.workunit.client.1.smithi174.stdout:1/688: unlink d4/d67/f74 0 2024-03-21T10:09:01.555 INFO:tasks.workunit.client.1.smithi174.stdout:8/536: write d9/d2a/d39/d76/f21 [1681309,76688] 0 2024-03-21T10:09:01.555 INFO:tasks.workunit.client.0.smithi153.stdout:3/814: dwrite dd/d1e/d3d/d94/d73/fa4 [0,4194304] 0 2024-03-21T10:09:01.556 INFO:tasks.workunit.client.1.smithi174.stdout:0/580: dwrite d9/d1c/d76/d87/f8d [0,4194304] 0 2024-03-21T10:09:01.559 INFO:tasks.workunit.client.0.smithi153.stdout:8/743: sync 2024-03-21T10:09:01.559 INFO:tasks.workunit.client.0.smithi153.stdout:7/730: sync 2024-03-21T10:09:01.560 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:01 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:09:01.560 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:01 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:09:01.562 INFO:tasks.workunit.client.1.smithi174.stdout:1/689: creat d4/d67/ff9 x:0 0 0 2024-03-21T10:09:01.564 INFO:tasks.workunit.client.1.smithi174.stdout:8/537: mknod d9/d2a/d4d/cb1 0 2024-03-21T10:09:01.566 INFO:tasks.workunit.client.0.smithi153.stdout:3/815: stat dd/c19 0 2024-03-21T10:09:01.567 INFO:tasks.workunit.client.1.smithi174.stdout:1/690: dread - d4/d7/d70/fd4 zero size 2024-03-21T10:09:01.570 INFO:tasks.workunit.client.0.smithi153.stdout:7/731: fdatasync d4/d2a/fa3 0 2024-03-21T10:09:01.571 INFO:tasks.workunit.client.0.smithi153.stdout:7/732: write d4/d19/d4d/dd5/fe2 [784929,58986] 0 2024-03-21T10:09:01.573 INFO:tasks.workunit.client.1.smithi174.stdout:8/538: fsync d9/d2a/d39/d76/d19/d27/d79/fa6 0 2024-03-21T10:09:01.573 INFO:tasks.workunit.client.1.smithi174.stdout:0/581: creat d9/d1c/d76/d87/dcd/fcf x:0 0 0 2024-03-21T10:09:01.574 INFO:tasks.workunit.client.1.smithi174.stdout:4/685: rename d4/d21/d24/d28/d79/f86 to d4/d21/d31/d3a/fe1 0 2024-03-21T10:09:01.577 INFO:tasks.workunit.client.1.smithi174.stdout:8/539: chown d9/d53/f6f 368 1 2024-03-21T10:09:01.578 INFO:tasks.workunit.client.1.smithi174.stdout:9/618: dwrite d2/d24/fd5 [0,4194304] 0 2024-03-21T10:09:01.582 INFO:tasks.workunit.client.1.smithi174.stdout:9/619: fdatasync d2/f84 0 2024-03-21T10:09:01.586 INFO:tasks.workunit.client.0.smithi153.stdout:3/816: creat dd/d1e/d3d/d94/f116 x:0 0 0 2024-03-21T10:09:01.587 INFO:tasks.workunit.client.1.smithi174.stdout:8/540: fdatasync d9/d2a/d39/d76/d19/d31/fac 0 2024-03-21T10:09:01.589 INFO:tasks.workunit.client.1.smithi174.stdout:1/691: getdents d4/d7/d33/d43/d77 0 2024-03-21T10:09:01.590 INFO:tasks.workunit.client.0.smithi153.stdout:7/733: unlink d4/d2a/d3b/d81/f99 0 2024-03-21T10:09:01.593 INFO:tasks.workunit.client.1.smithi174.stdout:8/541: mkdir d9/d2a/d4d/db2 0 2024-03-21T10:09:01.593 INFO:tasks.workunit.client.1.smithi174.stdout:1/692: fdatasync d4/d67/f99 0 2024-03-21T10:09:01.596 INFO:tasks.workunit.client.0.smithi153.stdout:7/734: write d4/d2a/d3b/d60/f7c [2321041,22897] 0 2024-03-21T10:09:01.596 INFO:tasks.workunit.client.0.smithi153.stdout:0/709: dwrite d4/d4c/d83/d22/f7d [4194304,4194304] 0 2024-03-21T10:09:01.598 INFO:tasks.workunit.client.1.smithi174.stdout:9/620: creat d2/d11/fdf x:0 0 0 2024-03-21T10:09:01.600 INFO:tasks.workunit.client.0.smithi153.stdout:7/735: chown d4/d11/l13 0 1 2024-03-21T10:09:01.604 INFO:tasks.workunit.client.0.smithi153.stdout:3/817: mknod dd/d1e/d3d/d94/d73/dc2/c117 0 2024-03-21T10:09:01.605 INFO:tasks.workunit.client.0.smithi153.stdout:7/736: dread - d4/d14/dd0/d83/fdf zero size 2024-03-21T10:09:01.607 INFO:tasks.workunit.client.1.smithi174.stdout:4/686: getdents d4/d21/d31/d3a/d7a/d9d/da3 0 2024-03-21T10:09:01.610 INFO:tasks.workunit.client.0.smithi153.stdout:1/620: write d6/d3a/d66/faf [6334030,23945] 0 2024-03-21T10:09:01.611 INFO:tasks.workunit.client.1.smithi174.stdout:8/542: mknod d9/d2a/d39/d76/d19/d27/d60/daa/dab/cb3 0 2024-03-21T10:09:01.611 INFO:tasks.workunit.client.0.smithi153.stdout:8/744: getdents d5/d9/d43/dd/d20/d26/d47/d9a 0 2024-03-21T10:09:01.613 INFO:tasks.workunit.client.1.smithi174.stdout:8/543: fdatasync d9/d53/f96 0 2024-03-21T10:09:01.616 INFO:tasks.workunit.client.0.smithi153.stdout:6/937: write f1a [492060,99775] 0 2024-03-21T10:09:01.616 INFO:tasks.workunit.client.0.smithi153.stdout:0/710: symlink d4/d16/d19/dad/le1 0 2024-03-21T10:09:01.617 INFO:tasks.workunit.client.0.smithi153.stdout:3/818: creat dd/d1e/d3d/dc9/f118 x:0 0 0 2024-03-21T10:09:01.618 INFO:tasks.workunit.client.0.smithi153.stdout:0/711: fsync d4/d4c/f7a 0 2024-03-21T10:09:01.618 INFO:tasks.workunit.client.1.smithi174.stdout:4/687: dread d4/d21/d31/d6a/dd1/fc9 [0,4194304] 0 2024-03-21T10:09:01.620 INFO:tasks.workunit.client.1.smithi174.stdout:8/544: mkdir d9/d2a/d4d/db4 0 2024-03-21T10:09:01.621 INFO:tasks.workunit.client.1.smithi174.stdout:8/545: readlink d9/l2d 0 2024-03-21T10:09:01.627 INFO:tasks.workunit.client.0.smithi153.stdout:4/784: dread d11/d18/f21 [0,4194304] 0 2024-03-21T10:09:01.629 INFO:tasks.workunit.client.0.smithi153.stdout:1/621: symlink d6/d10/d35/da5/d4f/d9d/db6/lc5 0 2024-03-21T10:09:01.629 INFO:tasks.workunit.client.0.smithi153.stdout:8/745: creat d5/d9/d43/dd/ff5 x:0 0 0 2024-03-21T10:09:01.630 INFO:tasks.workunit.client.1.smithi174.stdout:8/546: rename d9/d2a/d39/f5f to d9/d2a/d39/d76/d19/d27/d60/fb5 0 2024-03-21T10:09:01.633 INFO:tasks.workunit.client.0.smithi153.stdout:4/785: readlink d11/d2d/d6b/d9c/ld3 0 2024-03-21T10:09:01.633 INFO:tasks.workunit.client.0.smithi153.stdout:6/938: mkdir d1f/d2b/d10a/d14b 0 2024-03-21T10:09:01.633 INFO:tasks.workunit.client.0.smithi153.stdout:1/622: dread - d6/d10/da0/fb7 zero size 2024-03-21T10:09:01.642 INFO:tasks.workunit.client.1.smithi174.stdout:0/582: truncate d9/d1c/d76/d87/f8d 2412112 0 2024-03-21T10:09:01.647 INFO:tasks.workunit.client.1.smithi174.stdout:0/583: fsync d9/d21/d31/d9f/d79/fa0 0 2024-03-21T10:09:01.648 INFO:tasks.workunit.client.0.smithi153.stdout:0/712: dread d4/d16/d19/d95/fb5 [0,4194304] 0 2024-03-21T10:09:01.650 INFO:tasks.workunit.client.1.smithi174.stdout:4/688: write d4/f8 [4103397,27491] 0 2024-03-21T10:09:01.650 INFO:tasks.workunit.client.0.smithi153.stdout:7/737: dwrite d4/d14/d2f/d78/f3a [4194304,4194304] 0 2024-03-21T10:09:01.650 INFO:tasks.workunit.client.0.smithi153.stdout:8/746: creat d5/d9/d43/dd/d20/d26/d8b/ff6 x:0 0 0 2024-03-21T10:09:01.652 INFO:tasks.workunit.client.1.smithi174.stdout:8/547: dwrite d9/d2a/d39/d76/d19/d27/d60/fa8 [0,4194304] 0 2024-03-21T10:09:01.655 INFO:tasks.workunit.client.0.smithi153.stdout:4/786: symlink d11/db1/lf5 0 2024-03-21T10:09:01.659 INFO:tasks.workunit.client.0.smithi153.stdout:4/787: write d11/d2d/f9e [1392992,40242] 0 2024-03-21T10:09:01.660 INFO:tasks.workunit.client.1.smithi174.stdout:9/621: dwrite d2/d11/d1d/fa4 [0,4194304] 0 2024-03-21T10:09:01.662 INFO:tasks.workunit.client.0.smithi153.stdout:4/788: chown d11/d5e/ff2 114497093 1 2024-03-21T10:09:01.662 INFO:tasks.workunit.client.1.smithi174.stdout:1/693: dwrite d4/d7/d32/f3a [0,4194304] 0 2024-03-21T10:09:01.664 INFO:tasks.workunit.client.0.smithi153.stdout:4/789: chown d11/f63 325567 1 2024-03-21T10:09:01.664 INFO:tasks.workunit.client.1.smithi174.stdout:4/689: truncate d4/d19/f74 5681234 0 2024-03-21T10:09:01.671 INFO:tasks.workunit.client.0.smithi153.stdout:3/819: rename dd/d1e/d3d/d94/f116 to dd/d1e/d2f/d35/daf/dcc/f119 0 2024-03-21T10:09:01.672 INFO:tasks.workunit.client.0.smithi153.stdout:0/713: symlink d4/d4c/d83/d22/d61/le2 0 2024-03-21T10:09:01.672 INFO:tasks.workunit.client.1.smithi174.stdout:1/694: write d4/d2d/f5e [5303,54935] 0 2024-03-21T10:09:01.673 INFO:tasks.workunit.client.0.smithi153.stdout:1/623: write d6/f31 [3965860,91500] 0 2024-03-21T10:09:01.675 INFO:tasks.workunit.client.0.smithi153.stdout:3/820: dread - dd/d1e/d3d/dc9/f118 zero size 2024-03-21T10:09:01.676 INFO:tasks.workunit.client.0.smithi153.stdout:7/738: mkdir d4/d19/d4d/dd5/df7 0 2024-03-21T10:09:01.676 INFO:tasks.workunit.client.0.smithi153.stdout:6/939: mkdir d1f/d93/dd8/d103/d13f/d14c 0 2024-03-21T10:09:01.677 INFO:tasks.workunit.client.1.smithi174.stdout:9/622: creat d2/d11/d1d/d4b/d72/fe0 x:0 0 0 2024-03-21T10:09:01.679 INFO:tasks.workunit.client.1.smithi174.stdout:4/690: dread - d4/d21/d31/d3a/d67/d76/fc0 zero size 2024-03-21T10:09:01.679 INFO:tasks.workunit.client.0.smithi153.stdout:4/790: symlink d11/d2d/da3/dac/lf6 0 2024-03-21T10:09:01.681 INFO:tasks.workunit.client.0.smithi153.stdout:3/821: stat dd/d41/df1 0 2024-03-21T10:09:01.681 INFO:tasks.workunit.client.1.smithi174.stdout:4/691: write d4/fbf [1313639,1071] 0 2024-03-21T10:09:01.684 INFO:tasks.workunit.client.1.smithi174.stdout:7/505: sync 2024-03-21T10:09:01.689 INFO:tasks.workunit.client.0.smithi153.stdout:3/822: write dd/d1e/d2f/d35/d68/daa/fc7 [789165,3045] 0 2024-03-21T10:09:01.694 INFO:tasks.workunit.client.0.smithi153.stdout:4/791: creat d11/d9d/dc3/ff7 x:0 0 0 2024-03-21T10:09:01.694 INFO:tasks.workunit.client.0.smithi153.stdout:8/747: link d5/d1b/d49/l68 d5/d9/d43/dd/d2d/d45/d88/lf7 0 2024-03-21T10:09:01.694 INFO:tasks.workunit.client.0.smithi153.stdout:7/739: mkdir d4/d2a/d40/df8 0 2024-03-21T10:09:01.695 INFO:tasks.workunit.client.0.smithi153.stdout:1/624: creat d6/fc6 x:0 0 0 2024-03-21T10:09:01.696 INFO:tasks.workunit.client.1.smithi174.stdout:0/584: rename d9/cb to d9/d35/d4a/d67/d89/d99/db2/dbd/cd0 0 2024-03-21T10:09:01.696 INFO:tasks.workunit.client.1.smithi174.stdout:1/695: creat d4/d7/d28/d59/ffa x:0 0 0 2024-03-21T10:09:01.696 INFO:tasks.workunit.client.1.smithi174.stdout:7/506: readlink d0/d4/d10/d60/lab 0 2024-03-21T10:09:01.697 INFO:tasks.workunit.client.1.smithi174.stdout:1/696: stat d4/d5/c7a 0 2024-03-21T10:09:01.700 INFO:tasks.workunit.client.0.smithi153.stdout:7/740: write d4/d2a/d40/d50/fa5 [4222568,116999] 0 2024-03-21T10:09:01.700 INFO:tasks.workunit.client.0.smithi153.stdout:9/818: sync 2024-03-21T10:09:01.700 INFO:tasks.workunit.client.0.smithi153.stdout:2/831: sync 2024-03-21T10:09:01.705 INFO:tasks.workunit.client.1.smithi174.stdout:4/692: creat d4/d21/d31/d6a/fe2 x:0 0 0 2024-03-21T10:09:01.706 INFO:tasks.workunit.client.0.smithi153.stdout:7/741: stat d4/d14/dd0/d20/d9a 0 2024-03-21T10:09:01.706 INFO:tasks.workunit.client.1.smithi174.stdout:1/697: mkdir d4/d7/d33/d43/d77/d87/dfb 0 2024-03-21T10:09:01.708 INFO:tasks.workunit.client.0.smithi153.stdout:1/625: rename d6/d13/d75/db5 to d6/d10/d35/da5/d4f/d9d/dc7 0 2024-03-21T10:09:01.711 INFO:tasks.workunit.client.1.smithi174.stdout:1/698: read d4/d7/d28/f30 [6027187,15120] 0 2024-03-21T10:09:01.714 INFO:tasks.workunit.client.1.smithi174.stdout:9/623: dwrite d2/d11/d60/fdb [0,4194304] 0 2024-03-21T10:09:01.716 INFO:tasks.workunit.client.1.smithi174.stdout:8/548: dwrite d9/d2a/d39/d76/f8b [0,4194304] 0 2024-03-21T10:09:01.718 INFO:tasks.workunit.client.1.smithi174.stdout:7/507: dread d0/d4/d10/d1b/f2e [0,4194304] 0 2024-03-21T10:09:01.722 INFO:tasks.workunit.client.0.smithi153.stdout:3/823: dwrite dd/d1e/d2f/d35/d68/daa/fac [0,4194304] 0 2024-03-21T10:09:01.726 INFO:tasks.workunit.client.1.smithi174.stdout:1/699: unlink d4/d7/d28/fad 0 2024-03-21T10:09:01.730 INFO:tasks.workunit.client.0.smithi153.stdout:6/940: write d1f/d2b/d10a/d40/d4f/fce [593340,756] 0 2024-03-21T10:09:01.731 INFO:tasks.workunit.client.1.smithi174.stdout:3/605: sync 2024-03-21T10:09:01.732 INFO:tasks.workunit.client.1.smithi174.stdout:6/453: sync 2024-03-21T10:09:01.732 INFO:tasks.workunit.client.1.smithi174.stdout:2/589: sync 2024-03-21T10:09:01.732 INFO:tasks.workunit.client.1.smithi174.stdout:5/642: sync 2024-03-21T10:09:01.732 INFO:tasks.workunit.client.0.smithi153.stdout:6/941: stat d1f/d2b/f64 0 2024-03-21T10:09:01.732 INFO:tasks.workunit.client.1.smithi174.stdout:8/549: unlink d9/d2a/d39/f81 0 2024-03-21T10:09:01.735 INFO:tasks.workunit.client.0.smithi153.stdout:9/819: dwrite d1/d43/d44/d46/d9e/d24/d41/dbe/dcc/f106 [0,4194304] 0 2024-03-21T10:09:01.735 INFO:tasks.workunit.client.0.smithi153.stdout:6/942: chown d1f/d93/dd8/d59/dfc 3564680 1 2024-03-21T10:09:01.736 INFO:tasks.workunit.client.0.smithi153.stdout:1/626: link d6/d10/fb9 d6/d3a/d3c/d50/d51/fc8 0 2024-03-21T10:09:01.736 INFO:tasks.workunit.client.1.smithi174.stdout:6/454: readlink d0/d3/d2a/d56/d5e/l68 0 2024-03-21T10:09:01.740 INFO:tasks.workunit.client.0.smithi153.stdout:3/824: symlink dd/d1e/d2f/d35/d68/d75/ddd/db1/l11a 0 2024-03-21T10:09:01.740 INFO:tasks.workunit.client.0.smithi153.stdout:2/832: getdents dd/d15/d18/d2c/d4b/d56/d63/d8d 0 2024-03-21T10:09:01.740 INFO:tasks.workunit.client.0.smithi153.stdout:7/742: read d4/d11/f72 [3542581,75567] 0 2024-03-21T10:09:01.747 INFO:tasks.workunit.client.1.smithi174.stdout:0/585: dread d9/d1c/d76/d87/f8d [0,4194304] 0 2024-03-21T10:09:01.747 INFO:tasks.workunit.client.0.smithi153.stdout:1/627: symlink d6/d10/d35/d5b/d92/lc9 0 2024-03-21T10:09:01.748 INFO:tasks.workunit.client.0.smithi153.stdout:6/943: dread d1f/d2b/d10a/d40/d51/d98/f120 [0,4194304] 0 2024-03-21T10:09:01.749 INFO:tasks.workunit.client.1.smithi174.stdout:3/606: dread d0/d12/d14/d4c/d7a/d9e/fb8 [0,4194304] 0 2024-03-21T10:09:01.750 INFO:tasks.workunit.client.1.smithi174.stdout:9/624: creat d2/d24/fe1 x:0 0 0 2024-03-21T10:09:01.751 INFO:tasks.workunit.client.1.smithi174.stdout:6/455: creat d0/d3/d5/df/d16/f8a x:0 0 0 2024-03-21T10:09:01.751 INFO:tasks.workunit.client.0.smithi153.stdout:9/820: rmdir d1/d43/d44/d46/d4d/d5c 39 2024-03-21T10:09:01.752 INFO:tasks.workunit.client.1.smithi174.stdout:7/508: link d0/f27 d0/d53/d5f/d65/fac 0 2024-03-21T10:09:01.755 INFO:tasks.workunit.client.1.smithi174.stdout:4/693: link d4/l29 d4/d21/d31/d3a/d7a/d9d/da3/le3 0 2024-03-21T10:09:01.758 INFO:tasks.workunit.client.1.smithi174.stdout:2/590: symlink d5/d25/d27/d2d/d45/lca 0 2024-03-21T10:09:01.760 INFO:tasks.workunit.client.0.smithi153.stdout:7/743: mknod d4/d14/dd0/d20/d9a/db7/cf9 0 2024-03-21T10:09:01.760 INFO:tasks.workunit.client.0.smithi153.stdout:5/757: sync 2024-03-21T10:09:01.761 INFO:tasks.workunit.client.1.smithi174.stdout:8/550: creat d9/d2a/d4d/db4/fb6 x:0 0 0 2024-03-21T10:09:01.761 INFO:tasks.workunit.client.1.smithi174.stdout:7/509: read d0/d4/d10/d44/f64 [221041,7100] 0 2024-03-21T10:09:01.763 INFO:tasks.workunit.client.0.smithi153.stdout:8/748: truncate d5/d9/d43/dd/d20/d26/d24/f32 1478185 0 2024-03-21T10:09:01.766 INFO:tasks.workunit.client.0.smithi153.stdout:1/628: symlink d6/d10/d35/da5/d4f/d9d/lca 0 2024-03-21T10:09:01.768 INFO:tasks.workunit.client.0.smithi153.stdout:3/825: mkdir dd/d1e/d3d/d11b 0 2024-03-21T10:09:01.769 INFO:tasks.workunit.client.1.smithi174.stdout:3/607: mknod d0/d12/d14/d1a/d44/d65/ccb 0 2024-03-21T10:09:01.770 INFO:tasks.workunit.client.0.smithi153.stdout:5/758: read d4/d8/d5c/f73 [2156245,72529] 0 2024-03-21T10:09:01.770 INFO:tasks.workunit.client.0.smithi153.stdout:0/714: dread d4/d4c/d83/f35 [0,4194304] 0 2024-03-21T10:09:01.771 INFO:tasks.workunit.client.1.smithi174.stdout:0/586: truncate d9/d21/d31/d5c/fba 734699 0 2024-03-21T10:09:01.772 INFO:tasks.workunit.client.1.smithi174.stdout:4/694: creat d4/d21/d31/d3a/d67/d76/d82/fe4 x:0 0 0 2024-03-21T10:09:01.773 INFO:tasks.workunit.client.1.smithi174.stdout:8/551: fdatasync d9/d2a/d39/d76/f5a 0 2024-03-21T10:09:01.775 INFO:tasks.workunit.client.0.smithi153.stdout:4/792: dread d11/d18/f35 [4194304,4194304] 0 2024-03-21T10:09:01.775 INFO:tasks.workunit.client.0.smithi153.stdout:7/744: symlink d4/d14/dd0/d20/d9a/dd2/db9/lfa 0 2024-03-21T10:09:01.776 INFO:tasks.workunit.client.1.smithi174.stdout:5/643: link d1/d49/d4f/l59 d1/d16/d2e/d66/d8b/d85/lec 0 2024-03-21T10:09:01.778 INFO:tasks.workunit.client.0.smithi153.stdout:3/826: unlink f8 0 2024-03-21T10:09:01.778 INFO:tasks.workunit.client.0.smithi153.stdout:0/715: truncate d4/d4c/f4e 1451495 0 2024-03-21T10:09:01.780 INFO:tasks.workunit.client.0.smithi153.stdout:5/759: symlink d4/d5/df/d67/lfc 0 2024-03-21T10:09:01.782 INFO:tasks.workunit.client.0.smithi153.stdout:8/749: symlink d5/d9/d43/dd/d20/df3/lf8 0 2024-03-21T10:09:01.784 INFO:tasks.workunit.client.0.smithi153.stdout:1/629: mknod d6/d13/d75/ccb 0 2024-03-21T10:09:01.785 INFO:tasks.workunit.client.1.smithi174.stdout:3/608: creat d0/d12/d14/d1a/d27/d7f/fcc x:0 0 0 2024-03-21T10:09:01.789 INFO:tasks.workunit.client.1.smithi174.stdout:3/609: write d0/d12/d14/da9/fb0 [1042951,3265] 0 2024-03-21T10:09:01.789 INFO:tasks.workunit.client.0.smithi153.stdout:6/944: write d1f/d99/faf [541415,94998] 0 2024-03-21T10:09:01.791 INFO:tasks.workunit.client.1.smithi174.stdout:1/700: stat d4/d7/d28/d59/la0 0 2024-03-21T10:09:01.794 INFO:tasks.workunit.client.0.smithi153.stdout:5/760: unlink d4/d5/df/c76 0 2024-03-21T10:09:01.795 INFO:tasks.workunit.client.1.smithi174.stdout:4/695: mkdir d4/d21/d24/d28/d38/de5 0 2024-03-21T10:09:01.796 INFO:tasks.workunit.client.0.smithi153.stdout:2/833: dwrite dd/d15/d18/d2c/d33/f7d [0,4194304] 0 2024-03-21T10:09:01.798 INFO:tasks.workunit.client.1.smithi174.stdout:4/696: read - d4/d21/f5b zero size 2024-03-21T10:09:01.799 INFO:tasks.workunit.client.1.smithi174.stdout:5/644: mknod d1/d16/d2e/d8d/ced 0 2024-03-21T10:09:01.802 INFO:tasks.workunit.client.1.smithi174.stdout:0/587: dwrite d9/f1d [0,4194304] 0 2024-03-21T10:09:01.802 INFO:tasks.workunit.client.1.smithi174.stdout:5/645: fdatasync d1/d49/f53 0 2024-03-21T10:09:01.803 INFO:tasks.workunit.client.0.smithi153.stdout:2/834: fdatasync dd/d15/d3c/fcb 0 2024-03-21T10:09:01.808 INFO:tasks.workunit.client.0.smithi153.stdout:7/745: rename d4/d14/dd0/d20 to d4/d2a/dc6/dfb 0 2024-03-21T10:09:01.813 INFO:tasks.workunit.client.0.smithi153.stdout:4/793: link d11/d18/f61 d11/d18/d50/d65/ff8 0 2024-03-21T10:09:01.813 INFO:tasks.workunit.client.0.smithi153.stdout:3/827: dwrite dd/d1e/d3d/d94/fdb [0,4194304] 0 2024-03-21T10:09:01.814 INFO:tasks.workunit.client.0.smithi153.stdout:1/630: dread d6/fb [0,4194304] 0 2024-03-21T10:09:01.816 INFO:tasks.workunit.client.1.smithi174.stdout:6/456: getdents d0/d3/d5/d21 0 2024-03-21T10:09:01.820 INFO:tasks.workunit.client.0.smithi153.stdout:8/750: dread d5/d9/d43/d7b/dc4/da7/fc6 [0,4194304] 0 2024-03-21T10:09:01.821 INFO:tasks.workunit.client.0.smithi153.stdout:5/761: write d4/d5/f52 [4947481,28143] 0 2024-03-21T10:09:01.826 INFO:tasks.workunit.client.0.smithi153.stdout:6/945: creat d1f/d2b/d10a/d40/d7b/f14d x:0 0 0 2024-03-21T10:09:01.826 INFO:tasks.workunit.client.0.smithi153.stdout:3/828: creat dd/d41/dab/f11c x:0 0 0 2024-03-21T10:09:01.827 INFO:tasks.workunit.client.0.smithi153.stdout:4/794: truncate d11/d18/f77 5059153 0 2024-03-21T10:09:01.827 INFO:tasks.workunit.client.0.smithi153.stdout:0/716: dwrite d4/d16/d19/d58/fbf [0,4194304] 0 2024-03-21T10:09:01.827 INFO:tasks.workunit.client.1.smithi174.stdout:2/591: truncate d5/d25/d27/f8b 2633525 0 2024-03-21T10:09:01.827 INFO:tasks.workunit.client.1.smithi174.stdout:4/697: creat d4/d21/d31/d3a/dd2/fe6 x:0 0 0 2024-03-21T10:09:01.828 INFO:tasks.workunit.client.0.smithi153.stdout:7/746: truncate d4/d11/ff2 769553 0 2024-03-21T10:09:01.831 INFO:tasks.workunit.client.1.smithi174.stdout:6/457: write d0/d3/d5/d6/f6a [2383799,94100] 0 2024-03-21T10:09:01.832 INFO:tasks.workunit.client.1.smithi174.stdout:3/610: dwrite d0/f70 [0,4194304] 0 2024-03-21T10:09:01.834 INFO:tasks.workunit.client.1.smithi174.stdout:5/646: rename d1/d2f/d37/d9b/fa3 to d1/d16/d2e/d66/d8b/d85/fee 0 2024-03-21T10:09:01.838 INFO:tasks.workunit.client.0.smithi153.stdout:2/835: truncate dd/d15/d18/d3a/d3d/f53 2530719 0 2024-03-21T10:09:01.838 INFO:tasks.workunit.client.0.smithi153.stdout:5/762: creat d4/d8/d5a/d92/ffd x:0 0 0 2024-03-21T10:09:01.838 INFO:tasks.workunit.client.1.smithi174.stdout:4/698: stat d4/d21/d31/d3a/d67/fcc 0 2024-03-21T10:09:01.841 INFO:tasks.workunit.client.0.smithi153.stdout:7/747: creat d4/d11/ffc x:0 0 0 2024-03-21T10:09:01.841 INFO:tasks.workunit.client.1.smithi174.stdout:3/611: dread - d0/d12/d14/d4c/f7b zero size 2024-03-21T10:09:01.843 INFO:tasks.workunit.client.0.smithi153.stdout:1/631: rename d6/d3a/d66 to d6/d10/d35/da5/d4f/d9d/dc7/dcc 0 2024-03-21T10:09:01.844 INFO:tasks.workunit.client.0.smithi153.stdout:3/829: creat dd/d1e/d2f/d35/dd0/f11d x:0 0 0 2024-03-21T10:09:01.845 INFO:tasks.workunit.client.0.smithi153.stdout:2/836: symlink dd/d15/d18/d2c/dd7/l123 0 2024-03-21T10:09:01.847 INFO:tasks.workunit.client.1.smithi174.stdout:8/552: dread f8 [0,4194304] 0 2024-03-21T10:09:01.847 INFO:tasks.workunit.client.1.smithi174.stdout:4/699: write d4/d19/f98 [1351240,70860] 0 2024-03-21T10:09:01.847 INFO:tasks.workunit.client.1.smithi174.stdout:9/625: dwrite d2/d24/d2e/f31 [0,4194304] 0 2024-03-21T10:09:01.848 INFO:tasks.workunit.client.0.smithi153.stdout:8/751: truncate d5/d9/d43/d7b/dc4/f8a 1078845 0 2024-03-21T10:09:01.852 INFO:tasks.workunit.client.0.smithi153.stdout:1/632: write d6/d3a/d3c/d50/d51/f5f [5482214,80641] 0 2024-03-21T10:09:01.853 INFO:tasks.workunit.client.1.smithi174.stdout:7/510: dwrite d0/d53/d5f/d65/f6c [0,4194304] 0 2024-03-21T10:09:01.853 INFO:tasks.workunit.client.0.smithi153.stdout:7/748: rename d4/d2a/dc6/dfb/c27 to d4/d2a/d3b/d60/de1/cfd 0 2024-03-21T10:09:01.855 INFO:tasks.workunit.client.0.smithi153.stdout:7/749: dread - d4/d19/ff5 zero size 2024-03-21T10:09:01.857 INFO:tasks.workunit.client.0.smithi153.stdout:3/830: mkdir dd/d1e/d3d/d94/de2/d11e 0 2024-03-21T10:09:01.861 INFO:tasks.workunit.client.0.smithi153.stdout:4/795: link d11/f76 d11/d18/d50/d65/de3/dec/ff9 0 2024-03-21T10:09:01.861 INFO:tasks.workunit.client.0.smithi153.stdout:1/633: read d6/d13/d75/f62 [118612,68294] 0 2024-03-21T10:09:01.863 INFO:tasks.workunit.client.0.smithi153.stdout:3/831: symlink dd/d41/d80/dbb/dbf/l11f 0 2024-03-21T10:09:01.863 INFO:tasks.workunit.client.1.smithi174.stdout:1/701: dwrite d4/d7/f2e [0,4194304] 0 2024-03-21T10:09:01.864 INFO:tasks.workunit.client.0.smithi153.stdout:4/796: fdatasync d11/d2d/da3/dac/dce/fe7 0 2024-03-21T10:09:01.868 INFO:tasks.workunit.client.1.smithi174.stdout:9/626: chown d2/f18 98488 1 2024-03-21T10:09:01.869 INFO:tasks.workunit.client.1.smithi174.stdout:1/702: write d4/d7/d68/d89/fc1 [101852,114974] 0 2024-03-21T10:09:01.870 INFO:tasks.workunit.client.0.smithi153.stdout:7/750: creat d4/d14/d2f/ffe x:0 0 0 2024-03-21T10:09:01.870 INFO:tasks.workunit.client.1.smithi174.stdout:2/592: getdents d5/d25/d27/d2e/d4f/d65 0 2024-03-21T10:09:01.871 INFO:tasks.workunit.client.0.smithi153.stdout:4/797: rmdir d11/d2d/d6b/db6 39 2024-03-21T10:09:01.872 INFO:tasks.workunit.client.0.smithi153.stdout:5/763: dwrite d4/d5/f52 [0,4194304] 0 2024-03-21T10:09:01.873 INFO:tasks.workunit.client.1.smithi174.stdout:0/588: write d9/d21/d23/d80/fae [3764525,122713] 0 2024-03-21T10:09:01.876 INFO:tasks.workunit.client.1.smithi174.stdout:9/627: write d2/d24/d2e/dac/fcc [429235,97531] 0 2024-03-21T10:09:01.877 INFO:tasks.workunit.client.0.smithi153.stdout:3/832: fsync dd/d1e/d2f/d35/fb5 0 2024-03-21T10:09:01.879 INFO:tasks.workunit.client.0.smithi153.stdout:1/634: link d6/d10/d35/d5b/fad d6/d3a/d3c/d50/fcd 0 2024-03-21T10:09:01.881 INFO:tasks.workunit.client.1.smithi174.stdout:2/593: chown d5/d25/d27/d2e/d4f/d65/d77/fc6 19894 1 2024-03-21T10:09:01.881 INFO:tasks.workunit.client.1.smithi174.stdout:5/647: dwrite d1/d49/d93/d9c/fb2 [0,4194304] 0 2024-03-21T10:09:01.883 INFO:tasks.workunit.client.0.smithi153.stdout:9/821: sync 2024-03-21T10:09:01.884 INFO:tasks.workunit.client.1.smithi174.stdout:8/553: symlink d9/d2a/d39/d76/d19/d27/d60/daa/dab/lb7 0 2024-03-21T10:09:01.886 INFO:tasks.workunit.client.0.smithi153.stdout:4/798: mkdir d11/d18/d50/dfa 0 2024-03-21T10:09:01.887 INFO:tasks.workunit.client.0.smithi153.stdout:5/764: write d4/d5/df/d67/da0/db3/fd2 [394750,8158] 0 2024-03-21T10:09:01.890 INFO:tasks.workunit.client.1.smithi174.stdout:3/612: mknod d0/d12/d14/d1a/d5c/ccd 0 2024-03-21T10:09:01.891 INFO:tasks.workunit.client.0.smithi153.stdout:7/751: creat d4/d19/d4d/fff x:0 0 0 2024-03-21T10:09:01.891 INFO:tasks.workunit.client.1.smithi174.stdout:0/589: mknod d9/d21/d31/d5c/d92/d98/cd1 0 2024-03-21T10:09:01.893 INFO:tasks.workunit.client.0.smithi153.stdout:3/833: creat dd/d1e/d2f/d35/daf/f120 x:0 0 0 2024-03-21T10:09:01.893 INFO:tasks.workunit.client.1.smithi174.stdout:3/613: chown d0/d12/d14/d67 57532474 1 2024-03-21T10:09:01.899 INFO:tasks.workunit.client.0.smithi153.stdout:5/765: unlink d4/d5/df/d5e/d69/f7e 0 2024-03-21T10:09:01.899 INFO:tasks.workunit.client.1.smithi174.stdout:3/614: chown d0/d12/d46/d59/fb2 202399 1 2024-03-21T10:09:01.900 INFO:tasks.workunit.client.1.smithi174.stdout:9/628: rename d2/d24/d2e/f9b to d2/d11/d1d/d4b/d72/fe2 0 2024-03-21T10:09:01.900 INFO:tasks.workunit.client.0.smithi153.stdout:4/799: write d11/d2d/d6b/d9c/fe1 [347155,36957] 0 2024-03-21T10:09:01.901 INFO:tasks.workunit.client.1.smithi174.stdout:2/594: rmdir d5/d25/d27/d2d/d45 39 2024-03-21T10:09:01.905 INFO:tasks.workunit.client.1.smithi174.stdout:4/700: rmdir d4/d21/d31/ddd 0 2024-03-21T10:09:01.906 INFO:tasks.workunit.client.1.smithi174.stdout:4/701: stat d4/d19/f65 0 2024-03-21T10:09:01.907 INFO:tasks.workunit.client.0.smithi153.stdout:7/752: write d4/d19/d4d/dd5/fe2 [439783,42150] 0 2024-03-21T10:09:01.908 INFO:tasks.workunit.client.0.smithi153.stdout:9/822: truncate d1/d43/d44/d46/d9e/d24/d41/fa3 21882 0 2024-03-21T10:09:01.908 INFO:tasks.workunit.client.1.smithi174.stdout:8/554: chown d9/d53/d5b/l71 13652480 1 2024-03-21T10:09:01.915 INFO:tasks.workunit.client.0.smithi153.stdout:1/635: dwrite d6/d10/f97 [0,4194304] 0 2024-03-21T10:09:01.921 INFO:tasks.workunit.client.1.smithi174.stdout:1/703: dwrite d4/d7/d68/d7b/de0/fef [0,4194304] 0 2024-03-21T10:09:01.923 INFO:tasks.workunit.client.0.smithi153.stdout:6/946: dwrite f5 [0,4194304] 0 2024-03-21T10:09:01.923 INFO:tasks.workunit.client.0.smithi153.stdout:0/717: write d4/d16/d19/d30/d6c/f86 [850301,80162] 0 2024-03-21T10:09:01.926 INFO:tasks.workunit.client.0.smithi153.stdout:7/753: mkdir d4/d2a/d100 0 2024-03-21T10:09:01.930 INFO:tasks.workunit.client.1.smithi174.stdout:5/648: dwrite d1/d16/d2e/d66/f6c [0,4194304] 0 2024-03-21T10:09:01.930 INFO:tasks.workunit.client.0.smithi153.stdout:6/947: write d1f/d93/dd8/ffb [1870862,23417] 0 2024-03-21T10:09:01.930 INFO:tasks.workunit.client.0.smithi153.stdout:3/834: link dd/d1e/d2f/d35/dd0/ff9 dd/d1e/d2f/d35/dd0/d109/f121 0 2024-03-21T10:09:01.933 INFO:tasks.workunit.client.0.smithi153.stdout:6/948: write d1f/de0/de1/df2/d10f/f137 [760268,93206] 0 2024-03-21T10:09:01.939 INFO:tasks.workunit.client.0.smithi153.stdout:5/766: dread d4/d5/df/d67/d6a/f77 [0,4194304] 0 2024-03-21T10:09:01.942 INFO:tasks.workunit.client.0.smithi153.stdout:9/823: rmdir d1/d43/d44/d46/d9e/d24/d41/dbe/dcc 39 2024-03-21T10:09:01.944 INFO:tasks.workunit.client.1.smithi174.stdout:6/458: dread d0/d3/d5/fe [0,4194304] 0 2024-03-21T10:09:01.947 INFO:tasks.workunit.client.0.smithi153.stdout:1/636: truncate d6/d13/f5e 454625 0 2024-03-21T10:09:01.948 INFO:tasks.workunit.client.0.smithi153.stdout:0/718: mkdir d4/d16/d19/d30/d6c/de3 0 2024-03-21T10:09:01.952 INFO:tasks.workunit.client.1.smithi174.stdout:0/590: rename d9/d21/d31/d5c/d86/d9b to d9/d35/dd2 0 2024-03-21T10:09:01.956 INFO:tasks.workunit.client.1.smithi174.stdout:8/555: creat d9/d2a/d39/d58/fb8 x:0 0 0 2024-03-21T10:09:01.957 INFO:tasks.workunit.client.1.smithi174.stdout:0/591: read d9/d35/d4a/f6a [417216,36854] 0 2024-03-21T10:09:01.957 INFO:tasks.workunit.client.0.smithi153.stdout:7/754: creat d4/d2a/d3b/d81/f101 x:0 0 0 2024-03-21T10:09:01.960 INFO:tasks.workunit.client.1.smithi174.stdout:0/592: stat d9/d21/d5f/c64 0 2024-03-21T10:09:01.962 INFO:tasks.workunit.client.0.smithi153.stdout:3/835: rmdir dd/d1e/d3d/dc9 39 2024-03-21T10:09:01.962 INFO:tasks.workunit.client.0.smithi153.stdout:4/800: link d11/d9d/fea d11/d18/d7e/ffb 0 2024-03-21T10:09:01.962 INFO:tasks.workunit.client.0.smithi153.stdout:6/949: symlink d1f/d2b/d10a/d40/d4f/d66/l14e 0 2024-03-21T10:09:01.967 INFO:tasks.workunit.client.0.smithi153.stdout:5/767: mkdir d4/d8/d5a/d5d/dd9/d38/dfe 0 2024-03-21T10:09:01.969 INFO:tasks.workunit.client.0.smithi153.stdout:2/837: write dd/d15/d18/d2c/d4b/d56/d63/d8d/d9b/dc3/df5/dae/d50/d59/d5d/fef [624779,113266] 0 2024-03-21T10:09:01.969 INFO:tasks.workunit.client.0.smithi153.stdout:3/836: creat dd/d1e/d2f/d35/daf/dcc/d58/f122 x:0 0 0 2024-03-21T10:09:01.971 INFO:tasks.workunit.client.1.smithi174.stdout:2/595: creat d5/d25/d27/d2d/d45/d5b/d2f/d4b/fcb x:0 0 0 2024-03-21T10:09:01.971 INFO:tasks.workunit.client.1.smithi174.stdout:7/511: dwrite d0/d4/f42 [0,4194304] 0 2024-03-21T10:09:01.971 INFO:tasks.workunit.client.1.smithi174.stdout:1/704: creat d4/d7/d32/da2/ffc x:0 0 0 2024-03-21T10:09:01.973 INFO:tasks.workunit.client.0.smithi153.stdout:9/824: link d1/d43/d44/fff d1/d43/d44/d46/d9e/d2e/f111 0 2024-03-21T10:09:01.974 INFO:tasks.workunit.client.1.smithi174.stdout:5/649: unlink d1/d49/d4f/d51/f9a 0 2024-03-21T10:09:01.979 INFO:tasks.workunit.client.1.smithi174.stdout:4/702: dwrite d4/d21/d24/d28/fc1 [0,4194304] 0 2024-03-21T10:09:01.982 INFO:tasks.workunit.client.0.smithi153.stdout:1/637: rename d6/d13/c28 to d6/d10/d35/d5b/d92/cce 0 2024-03-21T10:09:01.982 INFO:tasks.workunit.client.0.smithi153.stdout:5/768: symlink d4/d8/d5a/d92/lff 0 2024-03-21T10:09:01.984 INFO:tasks.workunit.client.1.smithi174.stdout:3/615: rename d0/f3 to d0/d12/d14/d1a/d27/d3c/fce 0 2024-03-21T10:09:01.987 INFO:tasks.workunit.client.1.smithi174.stdout:5/650: write d1/d2f/d37/fea [254133,21832] 0 2024-03-21T10:09:01.987 INFO:tasks.workunit.client.1.smithi174.stdout:6/459: write d0/d3/d5/d6/f2d [1458302,62306] 0 2024-03-21T10:09:01.988 INFO:tasks.workunit.client.0.smithi153.stdout:9/825: rmdir d1/d4 39 2024-03-21T10:09:01.988 INFO:tasks.workunit.client.0.smithi153.stdout:1/638: dread - d6/d3a/d3c/d50/d51/fa6 zero size 2024-03-21T10:09:01.994 INFO:tasks.workunit.client.0.smithi153.stdout:8/752: dwrite d5/d1b/f59 [0,4194304] 0 2024-03-21T10:09:01.994 INFO:tasks.workunit.client.1.smithi174.stdout:6/460: dread - d0/d3/d5/d6/d42/f82 zero size 2024-03-21T10:09:01.997 INFO:tasks.workunit.client.0.smithi153.stdout:5/769: readlink d4/d8/d5a/d5d/lb9 0 2024-03-21T10:09:01.998 INFO:tasks.workunit.client.1.smithi174.stdout:5/651: write d1/d5c/d73/fa5 [885799,81908] 0 2024-03-21T10:09:02.002 INFO:tasks.workunit.client.0.smithi153.stdout:0/719: dwrite d4/d4c/d83/d22/f73 [0,4194304] 0 2024-03-21T10:09:02.002 INFO:tasks.workunit.client.0.smithi153.stdout:9/826: read d1/d43/d44/f3e [1863398,91100] 0 2024-03-21T10:09:02.002 INFO:tasks.workunit.client.1.smithi174.stdout:0/593: dwrite d9/d35/d4a/f5e [0,4194304] 0 2024-03-21T10:09:02.002 INFO:tasks.workunit.client.1.smithi174.stdout:2/596: truncate d5/d25/d27/d2d/d45/d5b/d2f/f87 150984 0 2024-03-21T10:09:02.003 INFO:tasks.workunit.client.1.smithi174.stdout:7/512: dread d0/d4/d10/d60/f77 [0,4194304] 0 2024-03-21T10:09:02.005 INFO:tasks.workunit.client.1.smithi174.stdout:4/703: symlink d4/d21/le7 0 2024-03-21T10:09:02.006 INFO:tasks.workunit.client.1.smithi174.stdout:5/652: fsync d1/d16/d2e/d66/d8b/f48 0 2024-03-21T10:09:02.007 INFO:tasks.workunit.client.0.smithi153.stdout:2/838: mkdir dd/d15/d18/d2c/d4b/d56/d63/d8d/d9b/dc3/df5/dae/d50/d124 0 2024-03-21T10:09:02.008 INFO:tasks.workunit.client.1.smithi174.stdout:3/616: truncate d0/d12/d14/d4c/f79 949750 0 2024-03-21T10:09:02.008 INFO:tasks.workunit.client.0.smithi153.stdout:7/755: dwrite d4/d2a/fa3 [0,4194304] 0 2024-03-21T10:09:02.009 INFO:tasks.workunit.client.1.smithi174.stdout:0/594: chown d9/d1c/d76/d87/faa 9590315 1 2024-03-21T10:09:02.009 INFO:tasks.workunit.client.1.smithi174.stdout:7/513: rename d0/d4 to d0/d4/d10/d79/da7/dad 22 2024-03-21T10:09:02.010 INFO:tasks.workunit.client.0.smithi153.stdout:8/753: truncate d5/d1b/f93 1173860 0 2024-03-21T10:09:02.011 INFO:tasks.workunit.client.1.smithi174.stdout:5/653: truncate d1/d5c/f7a 398689 0 2024-03-21T10:09:02.012 INFO:tasks.workunit.client.0.smithi153.stdout:9/827: creat d1/d3b/df8/f112 x:0 0 0 2024-03-21T10:09:02.015 INFO:tasks.workunit.client.0.smithi153.stdout:7/756: fsync d4/d19/fc1 0 2024-03-21T10:09:02.016 INFO:tasks.workunit.client.1.smithi174.stdout:4/704: mknod d4/d21/d31/d6a/dab/ce8 0 2024-03-21T10:09:02.020 INFO:tasks.workunit.client.0.smithi153.stdout:9/828: write d1/d4e/fa6 [594019,79677] 0 2024-03-21T10:09:02.020 INFO:tasks.workunit.client.1.smithi174.stdout:6/461: link d0/f7a d0/d3/d5/d6/d42/f8b 0 2024-03-21T10:09:02.021 INFO:tasks.workunit.client.1.smithi174.stdout:2/597: dread d5/d25/d27/d2d/d45/d5b/d63/f48 [0,4194304] 0 2024-03-21T10:09:02.022 INFO:tasks.workunit.client.0.smithi153.stdout:7/757: truncate d4/d14/fa4 532421 0 2024-03-21T10:09:02.024 INFO:tasks.workunit.client.1.smithi174.stdout:7/514: mknod d0/d4/d10/d44/d81/cae 0 2024-03-21T10:09:02.024 INFO:tasks.workunit.client.1.smithi174.stdout:2/598: write d5/d25/d27/d2e/d90/fc3 [262068,12217] 0 2024-03-21T10:09:02.026 INFO:tasks.workunit.client.0.smithi153.stdout:9/829: write d1/d43/d44/d46/d9e/d24/ff7 [913171,2463] 0 2024-03-21T10:09:02.026 INFO:tasks.workunit.client.1.smithi174.stdout:9/629: write d2/d24/f99 [913536,69819] 0 2024-03-21T10:09:02.029 INFO:tasks.workunit.client.0.smithi153.stdout:2/839: mkdir dd/d15/d18/d3a/d3d/d125 0 2024-03-21T10:09:02.030 INFO:tasks.workunit.client.1.smithi174.stdout:9/630: stat d2/d11/d7d/cb5 0 2024-03-21T10:09:02.030 INFO:tasks.workunit.client.0.smithi153.stdout:0/720: dread d4/f62 [0,4194304] 0 2024-03-21T10:09:02.030 INFO:tasks.workunit.client.1.smithi174.stdout:3/617: symlink d0/d12/d14/lcf 0 2024-03-21T10:09:02.033 INFO:tasks.workunit.client.0.smithi153.stdout:5/770: creat d4/d8/d5a/d5d/dd9/f100 x:0 0 0 2024-03-21T10:09:02.038 INFO:tasks.workunit.client.0.smithi153.stdout:9/830: read d1/d4e/f98 [2337273,115244] 0 2024-03-21T10:09:02.038 INFO:tasks.workunit.client.1.smithi174.stdout:4/705: dread d4/d21/d24/d28/fc1 [0,4194304] 0 2024-03-21T10:09:02.044 INFO:tasks.workunit.client.1.smithi174.stdout:0/595: link d9/d21/d23/d80/fae d9/d35/d4a/d67/d89/fd3 0 2024-03-21T10:09:02.047 INFO:tasks.workunit.client.0.smithi153.stdout:1/639: write d6/f16 [3755911,114873] 0 2024-03-21T10:09:02.048 INFO:tasks.workunit.client.1.smithi174.stdout:0/596: write d9/d21/d31/d5c/d92/d98/fc4 [5030487,120168] 0 2024-03-21T10:09:02.048 INFO:tasks.workunit.client.1.smithi174.stdout:8/556: dread d9/d2a/d39/f47 [0,4194304] 0 2024-03-21T10:09:02.050 INFO:tasks.workunit.client.1.smithi174.stdout:6/462: mknod d0/c8c 0 2024-03-21T10:09:02.050 INFO:tasks.workunit.client.1.smithi174.stdout:0/597: readlink d9/d1c/l30 0 2024-03-21T10:09:02.052 INFO:tasks.workunit.client.1.smithi174.stdout:5/654: creat d1/fef x:0 0 0 2024-03-21T10:09:02.053 INFO:tasks.workunit.client.1.smithi174.stdout:2/599: dread d5/d25/d27/d2d/d45/d5b/d2f/d68/f7a [0,4194304] 0 2024-03-21T10:09:02.053 INFO:tasks.workunit.client.0.smithi153.stdout:0/721: read d4/d16/d19/d30/d6c/d98/d5e/f78 [54096,77419] 0 2024-03-21T10:09:02.054 INFO:tasks.workunit.client.1.smithi174.stdout:0/598: read - d9/d21/d31/f4c zero size 2024-03-21T10:09:02.055 INFO:tasks.workunit.client.1.smithi174.stdout:8/557: chown d9/d2a/d39/d76/d19/d27/d79/fa6 1 1 2024-03-21T10:09:02.056 INFO:tasks.workunit.client.1.smithi174.stdout:0/599: write d9/d21/d31/d3c/fb0 [2801690,120645] 0 2024-03-21T10:09:02.059 INFO:tasks.workunit.client.0.smithi153.stdout:1/640: symlink d6/d10/d35/d5b/lcf 0 2024-03-21T10:09:02.059 INFO:tasks.workunit.client.1.smithi174.stdout:5/655: dread d1/d2f/d37/f97 [0,4194304] 0 2024-03-21T10:09:02.059 INFO:tasks.workunit.client.1.smithi174.stdout:6/463: creat d0/d3/d2a/f8d x:0 0 0 2024-03-21T10:09:02.061 INFO:tasks.workunit.client.0.smithi153.stdout:1/641: readlink d6/d10/d35/l6f 0 2024-03-21T10:09:02.062 INFO:tasks.workunit.client.0.smithi153.stdout:4/801: dread d11/d2d/fd0 [0,4194304] 0 2024-03-21T10:09:02.065 INFO:tasks.workunit.client.0.smithi153.stdout:8/754: write d5/d9/d43/dd/d20/d26/d8b/d80/fa1 [181218,124064] 0 2024-03-21T10:09:02.070 INFO:tasks.workunit.client.0.smithi153.stdout:5/771: write d4/d8/d5a/d5d/dd9/d44/f6f [69996,40459] 0 2024-03-21T10:09:02.071 INFO:tasks.workunit.client.0.smithi153.stdout:5/772: chown d4/f75 17 1 2024-03-21T10:09:02.072 INFO:tasks.workunit.client.0.smithi153.stdout:8/755: unlink d5/d9/d43/dd/d20/d26/d24/d52/f64 0 2024-03-21T10:09:02.075 INFO:tasks.workunit.client.1.smithi174.stdout:8/558: unlink d9/d2a/d39/d76/f69 0 2024-03-21T10:09:02.076 INFO:tasks.workunit.client.1.smithi174.stdout:0/600: rmdir d9/d21/d31/d9f/d79 39 2024-03-21T10:09:02.076 INFO:tasks.workunit.client.1.smithi174.stdout:5/656: creat d1/d2f/d37/d9b/ff0 x:0 0 0 2024-03-21T10:09:02.076 INFO:tasks.workunit.client.0.smithi153.stdout:5/773: symlink d4/d5/l101 0 2024-03-21T10:09:02.078 INFO:tasks.workunit.client.1.smithi174.stdout:8/559: fsync d9/d2a/d4d/f50 0 2024-03-21T10:09:02.078 INFO:tasks.workunit.client.1.smithi174.stdout:0/601: write d9/d35/dd2/fbe [887861,78910] 0 2024-03-21T10:09:02.081 INFO:tasks.workunit.client.0.smithi153.stdout:6/950: sync 2024-03-21T10:09:02.081 INFO:tasks.workunit.client.1.smithi174.stdout:5/657: write d1/d16/d2e/d66/d8b/d85/f90 [978011,129734] 0 2024-03-21T10:09:02.083 INFO:tasks.workunit.client.1.smithi174.stdout:8/560: chown d9/d53/l68 436 1 2024-03-21T10:09:02.089 INFO:tasks.workunit.client.1.smithi174.stdout:6/464: dwrite d0/d3/d2a/d56/d5e/f73 [0,4194304] 0 2024-03-21T10:09:02.090 INFO:tasks.workunit.client.1.smithi174.stdout:0/602: mkdir d9/d21/d31/d5c/d92/d98/db7/dd4 0 2024-03-21T10:09:02.093 INFO:tasks.workunit.client.0.smithi153.stdout:3/837: dread dd/d1e/d2f/f37 [0,4194304] 0 2024-03-21T10:09:02.094 INFO:tasks.workunit.client.0.smithi153.stdout:8/756: truncate d5/d9/d43/dd/d2d/d45/f70 1342890 0 2024-03-21T10:09:02.095 INFO:tasks.workunit.client.1.smithi174.stdout:8/561: creat d9/d2a/d39/d76/d19/d27/d60/daa/fb9 x:0 0 0 2024-03-21T10:09:02.097 INFO:tasks.workunit.client.1.smithi174.stdout:0/603: stat d9/d21/d31/d9f/d79 0 2024-03-21T10:09:02.099 INFO:tasks.workunit.client.1.smithi174.stdout:6/465: mkdir d0/d3/d5/df/d66/d8e 0 2024-03-21T10:09:02.099 INFO:tasks.workunit.client.0.smithi153.stdout:4/802: dwrite d11/f63 [0,4194304] 0 2024-03-21T10:09:02.101 INFO:tasks.workunit.client.0.smithi153.stdout:9/831: dwrite d1/d43/fab [4194304,4194304] 0 2024-03-21T10:09:02.102 INFO:tasks.workunit.client.1.smithi174.stdout:5/658: truncate d1/d16/d2e/d66/d8b/d85/f8e 1104356 0 2024-03-21T10:09:02.102 INFO:tasks.workunit.client.0.smithi153.stdout:6/951: read d1f/d2b/f64 [137004,51032] 0 2024-03-21T10:09:02.106 INFO:tasks.workunit.client.0.smithi153.stdout:5/774: creat d4/d5/df/f102 x:0 0 0 2024-03-21T10:09:02.107 INFO:tasks.workunit.client.1.smithi174.stdout:0/604: mkdir d9/d21/d31/d5c/dd5 0 2024-03-21T10:09:02.110 INFO:tasks.workunit.client.0.smithi153.stdout:3/838: chown dd/d1e/d3d/l89 5 1 2024-03-21T10:09:02.111 INFO:tasks.workunit.client.1.smithi174.stdout:8/562: dwrite f5 [0,4194304] 0 2024-03-21T10:09:02.112 INFO:tasks.workunit.client.1.smithi174.stdout:5/659: readlink d1/d49/d93/d9c/la1 0 2024-03-21T10:09:02.116 INFO:tasks.workunit.client.0.smithi153.stdout:1/642: dwrite d6/d13/d75/f62 [0,4194304] 0 2024-03-21T10:09:02.116 INFO:tasks.workunit.client.1.smithi174.stdout:5/660: write d1/f86 [1030102,100677] 0 2024-03-21T10:09:02.117 INFO:tasks.workunit.client.0.smithi153.stdout:2/840: dwrite dd/d15/d18/d3a/d3d/f53 [0,4194304] 0 2024-03-21T10:09:02.117 INFO:tasks.workunit.client.1.smithi174.stdout:1/705: truncate d4/d7/d33/d43/d77/d87/d92/ff2 1291134 0 2024-03-21T10:09:02.118 INFO:tasks.workunit.client.1.smithi174.stdout:4/706: rename d4/d21/d31 to d4/d21/d24/d28/de9 0 2024-03-21T10:09:02.122 INFO:tasks.workunit.client.0.smithi153.stdout:5/775: symlink d4/d5/d16/l103 0 2024-03-21T10:09:02.122 INFO:tasks.workunit.client.0.smithi153.stdout:6/952: mkdir d1f/d2b/d14f 0 2024-03-21T10:09:02.124 INFO:tasks.workunit.client.1.smithi174.stdout:0/605: creat d9/d21/d31/d5c/dd5/fd6 x:0 0 0 2024-03-21T10:09:02.128 INFO:tasks.workunit.client.0.smithi153.stdout:1/643: unlink d6/d3a/d3c/d50/d51/fc8 0 2024-03-21T10:09:02.128 INFO:tasks.workunit.client.1.smithi174.stdout:4/707: truncate d4/d21/d24/d28/d79/fbb 704536 0 2024-03-21T10:09:02.129 INFO:tasks.workunit.client.0.smithi153.stdout:1/644: chown d6/f31 3071223 1 2024-03-21T10:09:02.129 INFO:tasks.workunit.client.0.smithi153.stdout:9/832: truncate d1/d43/d44/d46/d9e/f31 885559 0 2024-03-21T10:09:02.132 INFO:tasks.workunit.client.0.smithi153.stdout:5/776: write d4/d8/d5a/d5d/dd9/d44/d48/f84 [5100961,122147] 0 2024-03-21T10:09:02.135 INFO:tasks.workunit.client.0.smithi153.stdout:6/953: mkdir d1f/d2b/d10a/d40/d150 0 2024-03-21T10:09:02.141 INFO:tasks.workunit.client.1.smithi174.stdout:0/606: write d9/f6d [283858,47444] 0 2024-03-21T10:09:02.142 INFO:tasks.workunit.client.0.smithi153.stdout:9/833: mknod d1/d3b/daa/df9/c113 0 2024-03-21T10:09:02.146 INFO:tasks.workunit.client.0.smithi153.stdout:9/834: chown d1/ff0 7302871 1 2024-03-21T10:09:02.147 INFO:tasks.workunit.client.0.smithi153.stdout:2/841: rename dd/d15/d3c/da7/d11f to dd/d15/d18/d3a/d10d/d126 0 2024-03-21T10:09:02.148 INFO:tasks.workunit.client.1.smithi174.stdout:3/618: truncate d0/d12/d14/d1a/d27/d3c/d73/db7/d7d/f8b 1025802 0 2024-03-21T10:09:02.151 INFO:tasks.workunit.client.0.smithi153.stdout:4/803: dwrite d11/d9d/dc3/ff7 [0,4194304] 0 2024-03-21T10:09:02.151 INFO:tasks.workunit.client.1.smithi174.stdout:4/708: truncate d4/d21/d24/d28/d38/f88 3185786 0 2024-03-21T10:09:02.153 INFO:tasks.workunit.client.1.smithi174.stdout:7/515: dwrite d0/d4/f12 [0,4194304] 0 2024-03-21T10:09:02.158 INFO:tasks.workunit.client.1.smithi174.stdout:5/661: rename d1/d16/d2e/d66/d8b/d85/lec to d1/d16/d1c/lf1 0 2024-03-21T10:09:02.161 INFO:tasks.workunit.client.0.smithi153.stdout:8/757: dwrite d5/d9/d43/dd/d20/d26/d24/f28 [0,4194304] 0 2024-03-21T10:09:02.162 INFO:tasks.workunit.client.1.smithi174.stdout:9/631: dwrite d2/d24/f88 [0,4194304] 0 2024-03-21T10:09:02.162 INFO:tasks.workunit.client.0.smithi153.stdout:1/645: rename d6/d3a/d3c/d50/d51/f6d to d6/d3a/d3c/fd0 0 2024-03-21T10:09:02.162 INFO:tasks.workunit.client.0.smithi153.stdout:2/842: chown dd/d15/d18/d2c/d33/d98/dc7/f119 17772 1 2024-03-21T10:09:02.170 INFO:tasks.workunit.client.0.smithi153.stdout:5/777: getdents d4/d8/d5c/d90/d95 0 2024-03-21T10:09:02.170 INFO:tasks.workunit.client.0.smithi153.stdout:1/646: mknod d6/d32/cd1 0 2024-03-21T10:09:02.176 INFO:tasks.workunit.client.0.smithi153.stdout:6/954: link d1f/l24 d1f/d93/dd8/d59/d70/l151 0 2024-03-21T10:09:02.178 INFO:tasks.workunit.client.1.smithi174.stdout:4/709: dwrite d4/d19/f2d [0,4194304] 0 2024-03-21T10:09:02.181 INFO:tasks.workunit.client.0.smithi153.stdout:0/722: dread d4/d16/d27/f79 [0,4194304] 0 2024-03-21T10:09:02.181 INFO:tasks.workunit.client.1.smithi174.stdout:4/710: fsync d4/f20 0 2024-03-21T10:09:02.181 INFO:tasks.workunit.client.1.smithi174.stdout:7/516: dread d0/d4/f42 [0,4194304] 0 2024-03-21T10:09:02.185 INFO:tasks.workunit.client.0.smithi153.stdout:4/804: dwrite d11/d18/f82 [0,4194304] 0 2024-03-21T10:09:02.186 INFO:tasks.workunit.client.1.smithi174.stdout:5/662: creat d1/d16/d2e/d66/ff2 x:0 0 0 2024-03-21T10:09:02.186 INFO:tasks.workunit.client.1.smithi174.stdout:1/706: getdents d4/d7/d68/d8e/db8 0 2024-03-21T10:09:02.188 INFO:tasks.workunit.client.1.smithi174.stdout:5/663: chown d1/d49/lbc 2 1 2024-03-21T10:09:02.189 INFO:tasks.workunit.client.0.smithi153.stdout:7/758: dread d4/d2a/f77 [0,4194304] 0 2024-03-21T10:09:02.189 INFO:tasks.workunit.client.0.smithi153.stdout:6/955: write d1f/d2b/d10a/d40/d4f/fce [1476010,7120] 0 2024-03-21T10:09:02.191 INFO:tasks.workunit.client.1.smithi174.stdout:5/664: chown d1/d2f 402702 1 2024-03-21T10:09:02.194 INFO:tasks.workunit.client.0.smithi153.stdout:8/758: rename d5/d1b/l40 to d5/d9/d43/dd/d20/d26/d8b/lf9 0 2024-03-21T10:09:02.194 INFO:tasks.workunit.client.0.smithi153.stdout:6/956: fdatasync d1f/d2b/d10a/d40/d51/dbd/ff5 0 2024-03-21T10:09:02.195 INFO:tasks.workunit.client.1.smithi174.stdout:9/632: creat d2/d11/fe3 x:0 0 0 2024-03-21T10:09:02.195 INFO:tasks.workunit.client.1.smithi174.stdout:7/517: mkdir d0/d4/d10/da4/daf 0 2024-03-21T10:09:02.196 INFO:tasks.workunit.client.1.smithi174.stdout:1/707: truncate d4/d7/f21 225801 0 2024-03-21T10:09:02.197 INFO:tasks.workunit.client.0.smithi153.stdout:5/778: mkdir d4/d5/df/d67/d86/d96/d104 0 2024-03-21T10:09:02.200 INFO:tasks.workunit.client.0.smithi153.stdout:8/759: write d5/d1b/f93 [1904858,98986] 0 2024-03-21T10:09:02.201 INFO:tasks.workunit.client.1.smithi174.stdout:4/711: creat d4/d21/d24/fea x:0 0 0 2024-03-21T10:09:02.203 INFO:tasks.workunit.client.1.smithi174.stdout:9/633: write d2/d24/f59 [7144821,66012] 0 2024-03-21T10:09:02.203 INFO:tasks.workunit.client.1.smithi174.stdout:1/708: chown d4/d7/d32/l4c 110755 1 2024-03-21T10:09:02.204 INFO:tasks.workunit.client.0.smithi153.stdout:4/805: unlink c8 0 2024-03-21T10:09:02.207 INFO:tasks.workunit.client.1.smithi174.stdout:7/518: read d0/d4/d10/d44/f64 [3025672,22457] 0 2024-03-21T10:09:02.207 INFO:tasks.workunit.client.1.smithi174.stdout:9/634: creat d2/d54/fe4 x:0 0 0 2024-03-21T10:09:02.208 INFO:tasks.workunit.client.0.smithi153.stdout:1/647: read f3 [1711954,32635] 0 2024-03-21T10:09:02.208 INFO:tasks.workunit.client.0.smithi153.stdout:7/759: unlink d4/d19/d4d/cd9 0 2024-03-21T10:09:02.213 INFO:tasks.workunit.client.0.smithi153.stdout:0/723: mknod d4/d4c/d83/d22/d61/d77/ce4 0 2024-03-21T10:09:02.214 INFO:tasks.workunit.client.1.smithi174.stdout:6/466: truncate d0/d3/d5/df/f22 1579560 0 2024-03-21T10:09:02.216 INFO:tasks.workunit.client.1.smithi174.stdout:7/519: creat d0/d53/d5f/d65/fb0 x:0 0 0 2024-03-21T10:09:02.218 INFO:tasks.workunit.client.0.smithi153.stdout:0/724: fsync d4/f37 0 2024-03-21T10:09:02.218 INFO:tasks.workunit.client.0.smithi153.stdout:5/779: mkdir d4/d5/df/d67/d86/d96/deb/d56/d105 0 2024-03-21T10:09:02.220 INFO:tasks.workunit.client.1.smithi174.stdout:5/665: dread d1/ff [8388608,4194304] 0 2024-03-21T10:09:02.224 INFO:tasks.workunit.client.0.smithi153.stdout:8/760: rename d5/d1b/d49 to d5/d9/d43/d7b/dc4/da7/dfa 0 2024-03-21T10:09:02.224 INFO:tasks.workunit.client.0.smithi153.stdout:7/760: truncate d4/d14/d2f/ffe 515051 0 2024-03-21T10:09:02.225 INFO:tasks.workunit.client.0.smithi153.stdout:1/648: write d6/f1c [4054211,95700] 0 2024-03-21T10:09:02.228 INFO:tasks.workunit.client.1.smithi174.stdout:4/712: rename d4/d21/d24/d28/de9/d6a/cae to d4/d21/d24/d28/de9/d6a/dac/ceb 0 2024-03-21T10:09:02.228 INFO:tasks.workunit.client.1.smithi174.stdout:9/635: truncate d2/d24/d2e/d7e/f8c 383429 0 2024-03-21T10:09:02.234 INFO:tasks.workunit.client.1.smithi174.stdout:7/520: symlink d0/d4/d10/d60/lb1 0 2024-03-21T10:09:02.234 INFO:tasks.workunit.client.0.smithi153.stdout:7/761: mkdir d4/d19/d102 0 2024-03-21T10:09:02.235 INFO:tasks.workunit.client.0.smithi153.stdout:1/649: chown d6/d32/cd1 22301 1 2024-03-21T10:09:02.237 INFO:tasks.workunit.client.0.smithi153.stdout:5/780: rename f1 to d4/d5/df/d67/d86/dbf/f106 0 2024-03-21T10:09:02.238 INFO:tasks.workunit.client.1.smithi174.stdout:8/563: truncate d9/d53/f6c 3229508 0 2024-03-21T10:09:02.238 INFO:tasks.workunit.client.0.smithi153.stdout:8/761: mkdir d5/d9/d43/dd/d20/dfb 0 2024-03-21T10:09:02.240 INFO:tasks.workunit.client.1.smithi174.stdout:1/709: link d4/d7/d32/c44 d4/d7/d68/cfd 0 2024-03-21T10:09:02.244 INFO:tasks.workunit.client.1.smithi174.stdout:1/710: chown d4/d7/d68/d8e/db8 80811 1 2024-03-21T10:09:02.247 INFO:tasks.workunit.client.0.smithi153.stdout:5/781: readlink d4/d8/d5a/d5d/dd9/d2f/lcb 0 2024-03-21T10:09:02.247 INFO:tasks.workunit.client.1.smithi174.stdout:9/636: rename d2/d11/f3a to d2/d24/d2e/d7e/fe5 0 2024-03-21T10:09:02.251 INFO:tasks.workunit.client.0.smithi153.stdout:3/839: sync 2024-03-21T10:09:02.251 INFO:tasks.workunit.client.1.smithi174.stdout:7/521: write d0/d4/d10/d60/f77 [1864277,62292] 0 2024-03-21T10:09:02.254 INFO:tasks.workunit.client.1.smithi174.stdout:3/619: dwrite d0/d12/d14/d1a/d27/d3c/f61 [0,4194304] 0 2024-03-21T10:09:02.254 INFO:tasks.workunit.client.0.smithi153.stdout:7/762: creat d4/d2a/d3b/f103 x:0 0 0 2024-03-21T10:09:02.255 INFO:tasks.workunit.client.0.smithi153.stdout:4/806: dwrite d11/f19 [0,4194304] 0 2024-03-21T10:09:02.257 INFO:tasks.workunit.client.0.smithi153.stdout:2/843: dread dd/d15/d18/d3a/f3e [0,4194304] 0 2024-03-21T10:09:02.258 INFO:tasks.workunit.client.0.smithi153.stdout:9/835: dread d1/d43/d44/d46/d9e/d2e/f51 [0,4194304] 0 2024-03-21T10:09:02.259 INFO:tasks.workunit.client.0.smithi153.stdout:1/650: rename d6/d3a/d3c/f43 to d6/d10/d35/da5/d4f/d9d/fd2 0 2024-03-21T10:09:02.262 INFO:tasks.workunit.client.0.smithi153.stdout:5/782: symlink d4/d8/d5c/d90/dd6/l107 0 2024-03-21T10:09:02.262 INFO:tasks.workunit.client.1.smithi174.stdout:3/620: mknod d0/d12/d14/d4c/d7a/cd0 0 2024-03-21T10:09:02.266 INFO:tasks.workunit.client.1.smithi174.stdout:7/522: unlink d0/d4/d10/d1b/l30 0 2024-03-21T10:09:02.266 INFO:tasks.workunit.client.1.smithi174.stdout:9/637: dwrite d2/d24/f59 [0,4194304] 0 2024-03-21T10:09:02.267 INFO:tasks.workunit.client.1.smithi174.stdout:3/621: readlink d0/d12/d14/d1a/d44/d94/l96 0 2024-03-21T10:09:02.271 INFO:tasks.workunit.client.1.smithi174.stdout:4/713: rename d4/d21/d24/d28/de9/d6a/fb0 to d4/d21/d24/fec 0 2024-03-21T10:09:02.273 INFO:tasks.workunit.client.1.smithi174.stdout:3/622: write d0/d12/f17 [141871,126054] 0 2024-03-21T10:09:02.274 INFO:tasks.workunit.client.1.smithi174.stdout:4/714: write d4/f20 [145732,76994] 0 2024-03-21T10:09:02.280 INFO:tasks.workunit.client.0.smithi153.stdout:2/844: mkdir dd/d15/d18/d2c/d33/d98/d127 0 2024-03-21T10:09:02.280 INFO:tasks.workunit.client.1.smithi174.stdout:3/623: truncate d0/d12/d14/d1a/d27/d4f/fc1 1531614 0 2024-03-21T10:09:02.284 INFO:tasks.workunit.client.1.smithi174.stdout:7/523: mkdir d0/d4/d10/d79/db2 0 2024-03-21T10:09:02.286 INFO:tasks.workunit.client.1.smithi174.stdout:8/564: rename d9/f15 to d9/d2a/d39/d76/d19/d27/d79/fba 0 2024-03-21T10:09:02.290 INFO:tasks.workunit.client.1.smithi174.stdout:6/467: write d0/d3/d5/df/d16/f65 [25988,81840] 0 2024-03-21T10:09:02.295 INFO:tasks.workunit.client.0.smithi153.stdout:3/840: creat dd/d1e/d3d/d11b/f123 x:0 0 0 2024-03-21T10:09:02.299 INFO:tasks.workunit.client.0.smithi153.stdout:7/763: dwrite d4/d2a/fdc [0,4194304] 0 2024-03-21T10:09:02.299 INFO:tasks.workunit.client.0.smithi153.stdout:9/836: creat d1/d4/d5f/df3/f114 x:0 0 0 2024-03-21T10:09:02.301 INFO:tasks.workunit.client.1.smithi174.stdout:1/711: rename d4/d7/d68/d7b/de0/fef to d4/d7/d32/da2/da8/ffe 0 2024-03-21T10:09:02.302 INFO:tasks.workunit.client.1.smithi174.stdout:5/666: dread d1/d16/f40 [4194304,4194304] 0 2024-03-21T10:09:02.302 INFO:tasks.workunit.client.0.smithi153.stdout:6/957: dwrite d1f/d93/dd8/f47 [0,4194304] 0 2024-03-21T10:09:02.307 INFO:tasks.workunit.client.1.smithi174.stdout:4/715: link d4/d21/d24/d28/de9/d3a/d67/fcc d4/d21/d24/d28/fed 0 2024-03-21T10:09:02.308 INFO:tasks.workunit.client.0.smithi153.stdout:9/837: write d1/d4e/f54 [3519571,56380] 0 2024-03-21T10:09:02.309 INFO:tasks.workunit.client.0.smithi153.stdout:0/725: truncate d4/d16/d19/d95/fb5 1683759 0 2024-03-21T10:09:02.310 INFO:tasks.workunit.client.1.smithi174.stdout:7/524: readlink d0/d4/d10/d44/l4c 0 2024-03-21T10:09:02.312 INFO:tasks.workunit.client.1.smithi174.stdout:7/525: readlink d0/d4/d10/d44/l55 0 2024-03-21T10:09:02.315 INFO:tasks.workunit.client.0.smithi153.stdout:4/807: dwrite d11/d2d/fd9 [0,4194304] 0 2024-03-21T10:09:02.318 INFO:tasks.workunit.client.1.smithi174.stdout:3/624: dwrite d0/d12/d14/d1a/d27/d4f/fc1 [0,4194304] 0 2024-03-21T10:09:02.319 INFO:tasks.workunit.client.1.smithi174.stdout:6/468: dread d0/d3/d5/fe [0,4194304] 0 2024-03-21T10:09:02.321 INFO:tasks.workunit.client.1.smithi174.stdout:8/565: unlink d9/d2a/d39/d76/d19/d27/f8a 0 2024-03-21T10:09:02.321 INFO:tasks.workunit.client.1.smithi174.stdout:5/667: unlink d1/d16/d2e/f80 0 2024-03-21T10:09:02.323 INFO:tasks.workunit.client.0.smithi153.stdout:4/808: truncate d11/d18/d54/fc9 347005 0 2024-03-21T10:09:02.324 INFO:tasks.workunit.client.1.smithi174.stdout:1/712: symlink d4/d7/d15/d4f/lff 0 2024-03-21T10:09:02.329 INFO:tasks.workunit.client.1.smithi174.stdout:4/716: chown d4/d21/d24/d28/de9/d3a/d7a/d9d/da3/le3 2259427 1 2024-03-21T10:09:02.331 INFO:tasks.workunit.client.1.smithi174.stdout:3/625: chown d0/d12/d14/f4e 52199726 1 2024-03-21T10:09:02.334 INFO:tasks.workunit.client.1.smithi174.stdout:5/668: write d1/d16/d1c/d81/fa6 [664804,104025] 0 2024-03-21T10:09:02.335 INFO:tasks.workunit.client.1.smithi174.stdout:3/626: chown d0/d12/l2e 121859 1 2024-03-21T10:09:02.338 INFO:tasks.workunit.client.0.smithi153.stdout:1/651: dwrite d6/fd [0,4194304] 0 2024-03-21T10:09:02.344 INFO:tasks.workunit.client.1.smithi174.stdout:1/713: chown d4/d7/d68/cfd 3 1 2024-03-21T10:09:02.349 INFO:tasks.workunit.client.1.smithi174.stdout:6/469: getdents d0/d3/d5/df/d66/d8e 0 2024-03-21T10:09:02.351 INFO:tasks.workunit.client.1.smithi174.stdout:5/669: mkdir d1/d5c/d73/de3/df3 0 2024-03-21T10:09:02.351 INFO:tasks.workunit.client.0.smithi153.stdout:2/845: rmdir dd/d15/d18/d2c/d4b/d56/d63/d8d/d9b/dc3/df5 39 2024-03-21T10:09:02.352 INFO:tasks.workunit.client.0.smithi153.stdout:7/764: creat d4/d2a/d3b/d60/f104 x:0 0 0 2024-03-21T10:09:02.355 INFO:tasks.workunit.client.0.smithi153.stdout:7/765: write d4/d14/dd0/d83/fdf [502595,97788] 0 2024-03-21T10:09:02.355 INFO:tasks.workunit.client.1.smithi174.stdout:8/566: dwrite d9/d2a/d39/d76/fa2 [0,4194304] 0 2024-03-21T10:09:02.355 INFO:tasks.workunit.client.1.smithi174.stdout:7/526: link d0/d4/d10/d44/d81/cae d0/d4/d10/d1b/cb3 0 2024-03-21T10:09:02.356 INFO:tasks.workunit.client.0.smithi153.stdout:6/958: rename d1f/d2b/f63 to d1f/d93/dd8/d59/d70/d13c/f152 0 2024-03-21T10:09:02.357 INFO:tasks.workunit.client.0.smithi153.stdout:4/809: fsync d11/f20 0 2024-03-21T10:09:02.358 INFO:tasks.workunit.client.0.smithi153.stdout:4/810: fdatasync d11/fb0 0 2024-03-21T10:09:02.359 INFO:tasks.workunit.client.0.smithi153.stdout:7/766: chown d4/d14/dd0/d83 0 1 2024-03-21T10:09:02.360 INFO:tasks.workunit.client.0.smithi153.stdout:2/846: mkdir dd/d15/d3c/da7/d128 0 2024-03-21T10:09:02.361 INFO:tasks.workunit.client.1.smithi174.stdout:6/470: mkdir d0/d3/d2a/d8f 0 2024-03-21T10:09:02.364 INFO:tasks.workunit.client.0.smithi153.stdout:9/838: creat d1/d3b/df8/dfa/f115 x:0 0 0 2024-03-21T10:09:02.364 INFO:tasks.workunit.client.1.smithi174.stdout:9/638: dwrite d2/fc [0,4194304] 0 2024-03-21T10:09:02.365 INFO:tasks.workunit.client.1.smithi174.stdout:8/567: mkdir d9/d2a/d39/d58/dbb 0 2024-03-21T10:09:02.366 INFO:tasks.workunit.client.0.smithi153.stdout:3/841: rename dd/d1e/d3d/l5c to dd/d1e/d3d/l124 0 2024-03-21T10:09:02.366 INFO:tasks.workunit.client.1.smithi174.stdout:4/717: getdents d4/d21/d24/d28/de9/d3a/d67 0 2024-03-21T10:09:02.369 INFO:tasks.workunit.client.1.smithi174.stdout:6/471: stat d0/d3/d5/d3b 0 2024-03-21T10:09:02.372 INFO:tasks.workunit.client.0.smithi153.stdout:0/726: getdents d4/d4c/d83/d22/d61 0 2024-03-21T10:09:02.372 INFO:tasks.workunit.client.0.smithi153.stdout:2/847: fsync dd/d15/d18/d2c/d4b/d56/d63/d8d/d9b/dc3/fd5 0 2024-03-21T10:09:02.373 INFO:tasks.workunit.client.0.smithi153.stdout:7/767: getdents d4/d2a/d100 0 2024-03-21T10:09:02.373 INFO:tasks.workunit.client.1.smithi174.stdout:9/639: mkdir d2/d24/d2e/de6 0 2024-03-21T10:09:02.375 INFO:tasks.workunit.client.1.smithi174.stdout:8/568: creat d9/d53/fbc x:0 0 0 2024-03-21T10:09:02.376 INFO:tasks.workunit.client.1.smithi174.stdout:5/670: link d1/d49/d93/d9c/fce d1/d2f/d37/ff4 0 2024-03-21T10:09:02.376 INFO:tasks.workunit.client.0.smithi153.stdout:0/727: chown d4/d16/d19/d58 415960 1 2024-03-21T10:09:02.376 INFO:tasks.workunit.client.1.smithi174.stdout:6/472: mkdir d0/d3/d2a/d90 0 2024-03-21T10:09:02.378 INFO:tasks.workunit.client.0.smithi153.stdout:7/768: dread - d4/d14/fec zero size 2024-03-21T10:09:02.378 INFO:tasks.workunit.client.0.smithi153.stdout:3/842: symlink dd/d1e/d3d/dc9/l125 0 2024-03-21T10:09:02.380 INFO:tasks.workunit.client.0.smithi153.stdout:6/959: getdents d1f/d2b/d10a/d40/d7b 0 2024-03-21T10:09:02.382 INFO:tasks.workunit.client.1.smithi174.stdout:9/640: chown d2/d11/d1d/d4b/fa8 13796 1 2024-03-21T10:09:02.385 INFO:tasks.workunit.client.1.smithi174.stdout:9/641: stat d2/d11/d1d/dab/dc1/lae 0 2024-03-21T10:09:02.386 INFO:tasks.workunit.client.1.smithi174.stdout:6/473: mkdir d0/d3/d2a/d56/d5e/d91 0 2024-03-21T10:09:02.389 INFO:tasks.workunit.client.1.smithi174.stdout:5/671: mkdir d1/d2f/d37/df5 0 2024-03-21T10:09:02.394 INFO:tasks.workunit.client.0.smithi153.stdout:4/811: dwrite d11/d18/d54/fc9 [0,4194304] 0 2024-03-21T10:09:02.394 INFO:tasks.workunit.client.1.smithi174.stdout:7/527: dwrite d0/f57 [0,4194304] 0 2024-03-21T10:09:02.395 INFO:tasks.workunit.client.0.smithi153.stdout:8/762: sync 2024-03-21T10:09:02.395 INFO:tasks.workunit.client.0.smithi153.stdout:5/783: sync 2024-03-21T10:09:02.395 INFO:tasks.workunit.client.0.smithi153.stdout:7/769: creat d4/d2a/dc6/dfb/d9a/db7/dda/f105 x:0 0 0 2024-03-21T10:09:02.396 INFO:tasks.workunit.client.1.smithi174.stdout:5/672: symlink d1/dca/lf6 0 2024-03-21T10:09:02.399 INFO:tasks.workunit.client.1.smithi174.stdout:9/642: link d2/d11/d1d/d2a/fc9 d2/d24/fe7 0 2024-03-21T10:09:02.400 INFO:tasks.workunit.client.1.smithi174.stdout:4/718: dwrite d4/d21/d24/d28/de9/d6a/d8f/f94 [0,4194304] 0 2024-03-21T10:09:02.400 INFO:tasks.workunit.client.0.smithi153.stdout:3/843: mkdir dd/d1e/d3d/d53/dae/d126 0 2024-03-21T10:09:02.403 INFO:tasks.workunit.client.1.smithi174.stdout:6/474: rename d0/d3/d5/d21/c7d to d0/d3/d2a/c92 0 2024-03-21T10:09:02.404 INFO:tasks.workunit.client.0.smithi153.stdout:5/784: chown d4/d8/d5a/d5d/dd9/d38/dfe 30144279 1 2024-03-21T10:09:02.405 INFO:tasks.workunit.client.1.smithi174.stdout:8/569: dwrite d9/d2a/d39/d76/d19/d31/fac [0,4194304] 0 2024-03-21T10:09:02.405 INFO:tasks.workunit.client.0.smithi153.stdout:3/844: chown dd/d41/f71 47597749 1 2024-03-21T10:09:02.406 INFO:tasks.workunit.client.1.smithi174.stdout:7/528: unlink d0/d4/d10/d1b/f7c 0 2024-03-21T10:09:02.406 INFO:tasks.workunit.client.0.smithi153.stdout:2/848: mknod dd/d15/d18/d2c/d4b/d56/d63/d8d/d9b/dc3/df5/dae/d50/d59/c129 0 2024-03-21T10:09:02.407 INFO:tasks.workunit.client.0.smithi153.stdout:9/839: dwrite d1/d43/d44/d46/d9e/d24/d41/f47 [0,4194304] 0 2024-03-21T10:09:02.408 INFO:tasks.workunit.client.1.smithi174.stdout:6/475: chown d0/d3/d5/df/d16/f2b 31352 1 2024-03-21T10:09:02.410 INFO:tasks.workunit.client.1.smithi174.stdout:5/673: symlink d1/d5c/d73/de3/lf7 0 2024-03-21T10:09:02.412 INFO:tasks.workunit.client.0.smithi153.stdout:4/812: creat d11/d2d/d57/ffc x:0 0 0 2024-03-21T10:09:02.414 INFO:tasks.workunit.client.0.smithi153.stdout:9/840: readlink d1/d4/d5f/db6/ldf 0 2024-03-21T10:09:02.414 INFO:tasks.workunit.client.1.smithi174.stdout:9/643: creat d2/d11/d1d/dab/fe8 x:0 0 0 2024-03-21T10:09:02.418 INFO:tasks.workunit.client.1.smithi174.stdout:8/570: unlink d9/d2a/d39/d76/d19/f40 0 2024-03-21T10:09:02.420 INFO:tasks.workunit.client.1.smithi174.stdout:4/719: dread d4/d19/f65 [0,4194304] 0 2024-03-21T10:09:02.420 INFO:tasks.workunit.client.0.smithi153.stdout:0/728: dread d4/d16/d19/d58/fbf [0,4194304] 0 2024-03-21T10:09:02.421 INFO:tasks.workunit.client.1.smithi174.stdout:7/529: symlink d0/d4/d10/d44/d81/d9a/lb4 0 2024-03-21T10:09:02.422 INFO:tasks.workunit.client.1.smithi174.stdout:7/530: readlink d0/d53/l78 0 2024-03-21T10:09:02.422 INFO:tasks.workunit.client.0.smithi153.stdout:3/845: rmdir dd/d1e 39 2024-03-21T10:09:02.423 INFO:tasks.workunit.client.1.smithi174.stdout:7/531: stat d0/fa 0 2024-03-21T10:09:02.424 INFO:tasks.workunit.client.1.smithi174.stdout:5/674: stat d1/d49/c7e 0 2024-03-21T10:09:02.424 INFO:tasks.workunit.client.0.smithi153.stdout:9/841: read - d1/d8a/fd8 zero size 2024-03-21T10:09:02.425 INFO:tasks.workunit.client.0.smithi153.stdout:4/813: read d11/d18/f4b [471452,104186] 0 2024-03-21T10:09:02.427 INFO:tasks.workunit.client.1.smithi174.stdout:9/644: rmdir d2/d11/d1d/d8b 39 2024-03-21T10:09:02.429 INFO:tasks.workunit.client.1.smithi174.stdout:6/476: symlink d0/d3/d20/d3d/d81/l93 0 2024-03-21T10:09:02.431 INFO:tasks.workunit.client.1.smithi174.stdout:8/571: truncate d9/d53/f9d 913234 0 2024-03-21T10:09:02.431 INFO:tasks.workunit.client.1.smithi174.stdout:3/627: write d0/d12/d14/d1a/d27/d3c/d73/db7/f1f [1321725,78198] 0 2024-03-21T10:09:02.431 INFO:tasks.workunit.client.1.smithi174.stdout:4/720: rmdir d4/d21/d24/d28/de9 39 2024-03-21T10:09:02.432 INFO:tasks.workunit.client.1.smithi174.stdout:5/675: symlink d1/d5c/lf8 0 2024-03-21T10:09:02.434 INFO:tasks.workunit.client.1.smithi174.stdout:6/477: write d0/d3/d5/df/d16/f65 [1406359,7222] 0 2024-03-21T10:09:02.434 INFO:tasks.workunit.client.0.smithi153.stdout:2/849: mknod dd/d15/d18/d2c/d4b/d56/d63/d8d/d9b/dc3/df5/dae/d50/d58/c12a 0 2024-03-21T10:09:02.436 INFO:tasks.workunit.client.1.smithi174.stdout:7/532: creat d0/d53/d5f/d65/fb5 x:0 0 0 2024-03-21T10:09:02.438 INFO:tasks.workunit.client.1.smithi174.stdout:9/645: creat d2/d11/d1d/d2a/da7/fe9 x:0 0 0 2024-03-21T10:09:02.442 INFO:tasks.workunit.client.0.smithi153.stdout:7/770: link d4/d14/dd0/d83/cbb d4/d14/c106 0 2024-03-21T10:09:02.442 INFO:tasks.workunit.client.1.smithi174.stdout:4/721: write d4/d21/d24/fea [445727,79733] 0 2024-03-21T10:09:02.443 INFO:tasks.workunit.client.1.smithi174.stdout:9/646: dread - d2/d24/d2e/d63/fad zero size 2024-03-21T10:09:02.443 INFO:tasks.workunit.client.1.smithi174.stdout:4/722: fdatasync d4/d21/d24/d28/d79/fca 0 2024-03-21T10:09:02.443 INFO:tasks.workunit.client.1.smithi174.stdout:3/628: rename d0/d12/d14/d1a/d27/f43 to d0/d12/d14/d67/dc8/fd1 0 2024-03-21T10:09:02.443 INFO:tasks.workunit.client.0.smithi153.stdout:4/814: mknod d11/d2d/d6b/d9c/cfd 0 2024-03-21T10:09:02.445 INFO:tasks.workunit.client.0.smithi153.stdout:8/763: getdents d5/d9/d43/dd/d20/d26/d8b/d80 0 2024-03-21T10:09:02.448 INFO:tasks.workunit.client.1.smithi174.stdout:6/478: creat d0/d3/d5/d3b/f94 x:0 0 0 2024-03-21T10:09:02.449 INFO:tasks.workunit.client.0.smithi153.stdout:2/850: rmdir dd/d15/d18/d2c/d33/d98 39 2024-03-21T10:09:02.449 INFO:tasks.workunit.client.1.smithi174.stdout:8/572: symlink d9/d2a/d4d/db2/lbd 0 2024-03-21T10:09:02.449 INFO:tasks.workunit.client.0.smithi153.stdout:7/771: dread d4/d2a/d40/f4c [8388608,4194304] 0 2024-03-21T10:09:02.452 INFO:tasks.workunit.client.1.smithi174.stdout:9/647: creat d2/d54/fea x:0 0 0 2024-03-21T10:09:02.453 INFO:tasks.workunit.client.0.smithi153.stdout:7/772: readlink d4/d2a/d3b/d81/l8c 0 2024-03-21T10:09:02.457 INFO:tasks.workunit.client.0.smithi153.stdout:7/773: read - d4/d2a/d3b/d60/f104 zero size 2024-03-21T10:09:02.457 INFO:tasks.workunit.client.1.smithi174.stdout:3/629: creat d0/d12/d14/d1a/d27/d4f/d64/fd2 x:0 0 0 2024-03-21T10:09:02.457 INFO:tasks.workunit.client.1.smithi174.stdout:9/648: truncate d2/d11/d1d/d4b/d72/fe0 846487 0 2024-03-21T10:09:02.459 INFO:tasks.workunit.client.0.smithi153.stdout:1/652: dwrite d6/d10/da0/fc4 [0,4194304] 0 2024-03-21T10:09:02.460 INFO:tasks.workunit.client.1.smithi174.stdout:8/573: mknod d9/d2a/d39/d76/d19/cbe 0 2024-03-21T10:09:02.460 INFO:tasks.workunit.client.0.smithi153.stdout:4/815: creat d11/d2d/d3d/ffe x:0 0 0 2024-03-21T10:09:02.461 INFO:tasks.workunit.client.0.smithi153.stdout:9/842: creat d1/f116 x:0 0 0 2024-03-21T10:09:02.465 INFO:tasks.workunit.client.1.smithi174.stdout:9/649: unlink d2/d24/d2e/l5a 0 2024-03-21T10:09:02.467 INFO:tasks.workunit.client.0.smithi153.stdout:9/843: truncate d1/d4/d5f/db6/fe2 29799 0 2024-03-21T10:09:02.467 INFO:tasks.workunit.client.1.smithi174.stdout:6/479: rmdir d0/d3/d2a/d90 0 2024-03-21T10:09:02.468 INFO:tasks.workunit.client.1.smithi174.stdout:8/574: truncate d9/d2a/d39/d76/d19/d27/d60/f9f 2950 0 2024-03-21T10:09:02.469 INFO:tasks.workunit.client.0.smithi153.stdout:2/851: rename dd/d15/d18/d2c/d33/c84 to dd/d15/d18/d2c/d4b/d56/d63/d8d/d9b/dc3/df5/dae/d50/d59/c12b 0 2024-03-21T10:09:02.473 INFO:tasks.workunit.client.0.smithi153.stdout:6/960: write d1f/d2b/d10a/d40/d4f/f119 [598143,63448] 0 2024-03-21T10:09:02.473 INFO:tasks.workunit.client.1.smithi174.stdout:6/480: write d0/d3/d5/d6/f13 [9013748,50086] 0 2024-03-21T10:09:02.473 INFO:tasks.workunit.client.1.smithi174.stdout:8/575: mknod d9/d2a/d39/d76/d19/cbf 0 2024-03-21T10:09:02.473 INFO:tasks.workunit.client.1.smithi174.stdout:4/723: link d4/d21/d24/d28/de9/d3a/d7a/d9d/ca6 d4/d21/d24/d28/de9/d6a/d8f/d9c/cee 0 2024-03-21T10:09:02.476 INFO:tasks.workunit.client.1.smithi174.stdout:4/724: write d4/f8 [40210,80097] 0 2024-03-21T10:09:02.477 INFO:tasks.workunit.client.1.smithi174.stdout:6/481: creat d0/d3/d2a/d56/f95 x:0 0 0 2024-03-21T10:09:02.480 INFO:tasks.workunit.client.0.smithi153.stdout:5/785: write d4/d8/f11 [299920,26796] 0 2024-03-21T10:09:02.481 INFO:tasks.workunit.client.0.smithi153.stdout:9/844: symlink d1/d4/d5f/db6/l117 0 2024-03-21T10:09:02.481 INFO:tasks.workunit.client.0.smithi153.stdout:7/774: creat d4/d19/d4d/dd5/df7/f107 x:0 0 0 2024-03-21T10:09:02.482 INFO:tasks.workunit.client.1.smithi174.stdout:4/725: dread d4/d21/d24/d28/d79/fbb [0,4194304] 0 2024-03-21T10:09:02.484 INFO:tasks.workunit.client.1.smithi174.stdout:8/576: dread d9/d2a/d39/d76/d19/d27/f38 [0,4194304] 0 2024-03-21T10:09:02.486 INFO:tasks.workunit.client.1.smithi174.stdout:8/577: creat d9/d53/d5b/fc0 x:0 0 0 2024-03-21T10:09:02.487 INFO:tasks.workunit.client.0.smithi153.stdout:3/846: write dd/d1e/d2f/d35/f110 [3895702,72322] 0 2024-03-21T10:09:02.488 INFO:tasks.workunit.client.0.smithi153.stdout:0/729: write d4/d16/fc1 [155020,96491] 0 2024-03-21T10:09:02.490 INFO:tasks.workunit.client.0.smithi153.stdout:9/845: creat d1/d4/d5f/df3/f118 x:0 0 0 2024-03-21T10:09:02.492 INFO:tasks.workunit.client.0.smithi153.stdout:2/852: mkdir dd/d15/d18/d2c/d4b/d56/d63/d8d/d9b/dc3/df5/dae/d50/d124/d12c 0 2024-03-21T10:09:02.494 INFO:tasks.workunit.client.0.smithi153.stdout:0/730: dread - d4/d4c/d83/d22/f94 zero size 2024-03-21T10:09:02.494 INFO:tasks.workunit.client.0.smithi153.stdout:3/847: write dd/d41/d80/dbb/def/ffe [1447267,119682] 0 2024-03-21T10:09:02.496 INFO:tasks.workunit.client.0.smithi153.stdout:7/775: mkdir d4/d19/d4d/dd5/df7/d108 0 2024-03-21T10:09:02.500 INFO:tasks.workunit.client.0.smithi153.stdout:7/776: chown d4/f10 2393 1 2024-03-21T10:09:02.501 INFO:tasks.workunit.client.0.smithi153.stdout:4/816: dread f10 [0,4194304] 0 2024-03-21T10:09:02.502 INFO:tasks.workunit.client.0.smithi153.stdout:0/731: chown d4/d16/d19/d30/d6c/d98/d5e 463168 1 2024-03-21T10:09:02.502 INFO:tasks.workunit.client.0.smithi153.stdout:7/777: dread - d4/f6c zero size 2024-03-21T10:09:02.502 INFO:tasks.workunit.client.0.smithi153.stdout:5/786: rename d4/d8/d5a/d5d/dd9/d44/d48 to d4/d8/d5a/d5d/dd9/d44/de1/dfb/d108 0 2024-03-21T10:09:02.509 INFO:tasks.workunit.client.0.smithi153.stdout:5/787: readlink d4/d5/df/d67/d86/d96/lbc 0 2024-03-21T10:09:02.509 INFO:tasks.workunit.client.0.smithi153.stdout:9/846: fdatasync d1/d3b/daa/df9/fb0 0 2024-03-21T10:09:02.515 INFO:tasks.workunit.client.0.smithi153.stdout:5/788: write d4/d5/f7 [1674543,125553] 0 2024-03-21T10:09:02.515 INFO:tasks.workunit.client.0.smithi153.stdout:2/853: creat dd/d15/d18/dca/f12d x:0 0 0 2024-03-21T10:09:02.518 INFO:tasks.workunit.client.1.smithi174.stdout:8/578: dread d9/d2a/d39/d76/f20 [0,4194304] 0 2024-03-21T10:09:02.519 INFO:tasks.workunit.client.0.smithi153.stdout:3/848: mknod dd/d1e/d3d/c127 0 2024-03-21T10:09:02.521 INFO:tasks.workunit.client.1.smithi174.stdout:8/579: creat d9/d2a/d39/d76/fc1 x:0 0 0 2024-03-21T10:09:02.526 INFO:tasks.workunit.client.0.smithi153.stdout:8/764: dread d5/d9/d43/dd/f38 [0,4194304] 0 2024-03-21T10:09:02.528 INFO:tasks.workunit.client.0.smithi153.stdout:5/789: rename d4/d5/df/d67/d6a/fc2 to d4/d8/d5c/d90/f109 0 2024-03-21T10:09:02.528 INFO:tasks.workunit.client.0.smithi153.stdout:9/847: fsync d1/d4e/f53 0 2024-03-21T10:09:02.531 INFO:tasks.workunit.client.0.smithi153.stdout:2/854: unlink dd/d15/d18/d3a/d3d/l11a 0 2024-03-21T10:09:02.534 INFO:tasks.workunit.client.0.smithi153.stdout:6/961: dread d1f/d2b/d10a/d40/d51/f52 [0,4194304] 0 2024-03-21T10:09:02.543 INFO:tasks.workunit.client.0.smithi153.stdout:2/855: dread - dd/d15/d5e/db3/f105 zero size 2024-03-21T10:09:02.544 INFO:tasks.workunit.client.0.smithi153.stdout:6/962: stat d1f/d2b/d10a 0 2024-03-21T10:09:02.544 INFO:tasks.workunit.client.0.smithi153.stdout:1/653: dwrite d6/f36 [0,4194304] 0 2024-03-21T10:09:02.547 INFO:tasks.workunit.client.0.smithi153.stdout:0/732: dwrite d4/d16/d19/fa2 [4194304,4194304] 0 2024-03-21T10:09:02.551 INFO:tasks.workunit.client.1.smithi174.stdout:0/607: sync 2024-03-21T10:09:02.551 INFO:tasks.workunit.client.0.smithi153.stdout:4/817: dwrite d11/d9d/fbd [0,4194304] 0 2024-03-21T10:09:02.552 INFO:tasks.workunit.client.0.smithi153.stdout:8/765: getdents d5/d9/d43/dd/d2d/d45/d88/d85/ddc 0 2024-03-21T10:09:02.555 INFO:tasks.workunit.client.0.smithi153.stdout:9/848: rename d1/d3b/c8c to d1/d43/d44/c119 0 2024-03-21T10:09:02.558 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:02 smithi153 ceph-mon[29117]: pgmap v17: 65 pgs: 65 active+clean; 2.9 GiB data, 8.4 GiB used, 528 GiB / 536 GiB avail; 100 MiB/s rd, 350 MiB/s wr, 638 op/s 2024-03-21T10:09:02.565 INFO:tasks.workunit.client.0.smithi153.stdout:4/818: mkdir d11/d18/d50/dfa/dff 0 2024-03-21T10:09:02.565 INFO:tasks.workunit.client.0.smithi153.stdout:6/963: mkdir d1f/d2b/d10a/d40/d7b/d153 0 2024-03-21T10:09:02.568 INFO:tasks.workunit.client.0.smithi153.stdout:7/778: dwrite d4/d14/d16/fed [0,4194304] 0 2024-03-21T10:09:02.571 INFO:tasks.workunit.client.1.smithi174.stdout:8/580: dread d9/d53/d5b/f77 [0,4194304] 0 2024-03-21T10:09:02.572 INFO:tasks.workunit.client.1.smithi174.stdout:0/608: dwrite d9/d21/d31/d3c/d72/f7e [0,4194304] 0 2024-03-21T10:09:02.574 INFO:tasks.workunit.client.1.smithi174.stdout:8/581: rmdir d9/d53/d7e 39 2024-03-21T10:09:02.575 INFO:tasks.workunit.client.0.smithi153.stdout:3/849: dwrite dd/d1e/d6b/f6c [0,4194304] 0 2024-03-21T10:09:02.576 INFO:tasks.workunit.client.0.smithi153.stdout:0/733: mknod d4/dd0/ce5 0 2024-03-21T10:09:02.585 INFO:tasks.workunit.client.0.smithi153.stdout:8/766: stat d5/d9/d43/dd/d20/d26/d24/d52/d55 0 2024-03-21T10:09:02.588 INFO:tasks.workunit.client.0.smithi153.stdout:5/790: dwrite d4/f46 [4194304,4194304] 0 2024-03-21T10:09:02.589 INFO:tasks.workunit.client.0.smithi153.stdout:7/779: fdatasync d4/d14/dd0/d83/f89 0 2024-03-21T10:09:02.589 INFO:tasks.workunit.client.0.smithi153.stdout:0/734: chown d4/d4c/f7a 962427341 1 2024-03-21T10:09:02.590 INFO:tasks.workunit.client.1.smithi174.stdout:5/676: truncate d1/d16/d1c/d81/fa6 1265841 0 2024-03-21T10:09:02.593 INFO:tasks.workunit.client.1.smithi174.stdout:3/630: write d0/d12/d46/f4b [653357,127767] 0 2024-03-21T10:09:02.593 INFO:tasks.workunit.client.0.smithi153.stdout:6/964: chown d1f/d2b/d10a/d40/d5f/f9b 29242 1 2024-03-21T10:09:02.593 INFO:tasks.workunit.client.0.smithi153.stdout:4/819: mknod d11/d2d/d3d/dc7/dda/c100 0 2024-03-21T10:09:02.598 INFO:tasks.workunit.client.0.smithi153.stdout:3/850: write dd/d1e/d2f/d35/daf/dcc/f9e [2955656,63524] 0 2024-03-21T10:09:02.599 INFO:tasks.workunit.client.1.smithi174.stdout:8/582: read d9/d53/f9d [592803,16580] 0 2024-03-21T10:09:02.600 INFO:tasks.workunit.client.1.smithi174.stdout:6/482: write d0/f37 [9613,102005] 0 2024-03-21T10:09:02.601 INFO:tasks.workunit.client.1.smithi174.stdout:3/631: unlink d0/d12/c3b 0 2024-03-21T10:09:02.604 INFO:tasks.workunit.client.1.smithi174.stdout:4/726: truncate d4/d21/d24/d28/de9/d3a/d7a/d9d/fc2 1042285 0 2024-03-21T10:09:02.606 INFO:tasks.workunit.client.1.smithi174.stdout:6/483: chown d0/d3/d5/d3b/f55 4 1 2024-03-21T10:09:02.606 INFO:tasks.workunit.client.1.smithi174.stdout:0/609: write d9/d21/d75/da1/fa8 [896144,1122] 0 2024-03-21T10:09:02.610 INFO:tasks.workunit.client.0.smithi153.stdout:3/851: rename dd/d1e/d3d/d94/d73 to dd/d1e/d3d/d94/d73/dc2/d128 22 2024-03-21T10:09:02.611 INFO:tasks.workunit.client.0.smithi153.stdout:0/735: creat d4/d4c/d83/d22/d61/d77/fe6 x:0 0 0 2024-03-21T10:09:02.613 INFO:tasks.workunit.client.0.smithi153.stdout:2/856: dwrite dd/d15/d18/d2c/d33/f7d [0,4194304] 0 2024-03-21T10:09:02.613 INFO:tasks.workunit.client.0.smithi153.stdout:4/820: dread - d11/d71/d89/fb9 zero size 2024-03-21T10:09:02.614 INFO:tasks.workunit.client.1.smithi174.stdout:4/727: mknod d4/d21/d24/d28/de9/d3a/d67/cef 0 2024-03-21T10:09:02.615 INFO:tasks.workunit.client.0.smithi153.stdout:9/849: dwrite d1/d43/d44/d46/d9e/d61/fe5 [0,4194304] 0 2024-03-21T10:09:02.615 INFO:tasks.workunit.client.1.smithi174.stdout:6/484: creat d0/d3/d5/df/d66/f96 x:0 0 0 2024-03-21T10:09:02.615 INFO:tasks.workunit.client.1.smithi174.stdout:9/650: dwrite d2/d11/d1d/f80 [4194304,4194304] 0 2024-03-21T10:09:02.616 INFO:tasks.workunit.client.0.smithi153.stdout:1/654: dwrite d6/d13/d75/f57 [0,4194304] 0 2024-03-21T10:09:02.620 INFO:tasks.workunit.client.1.smithi174.stdout:6/485: write d0/d3/d5/d3b/f94 [211108,124138] 0 2024-03-21T10:09:02.623 INFO:tasks.workunit.client.1.smithi174.stdout:5/677: dwrite d1/d2f/fae [0,4194304] 0 2024-03-21T10:09:02.625 INFO:tasks.workunit.client.1.smithi174.stdout:6/486: chown d0/d3/d5/d6/d42/f69 1 1 2024-03-21T10:09:02.628 INFO:tasks.workunit.client.1.smithi174.stdout:4/728: mknod d4/d21/d24/cf0 0 2024-03-21T10:09:02.629 INFO:tasks.workunit.client.1.smithi174.stdout:9/651: write d2/d24/d2e/f58 [376834,52544] 0 2024-03-21T10:09:02.634 INFO:tasks.workunit.client.0.smithi153.stdout:4/821: chown d11/d2d/d57/d85/cba 35093348 1 2024-03-21T10:09:02.636 INFO:tasks.workunit.client.0.smithi153.stdout:5/791: fsync d4/d8/d5a/d5d/f5f 0 2024-03-21T10:09:02.637 INFO:tasks.workunit.client.1.smithi174.stdout:5/678: getdents d1 0 2024-03-21T10:09:02.648 INFO:tasks.workunit.client.0.smithi153.stdout:6/965: symlink d1f/de0/l154 0 2024-03-21T10:09:02.648 INFO:tasks.workunit.client.1.smithi174.stdout:4/729: link d4/d21/f7b d4/d21/d24/ff1 0 2024-03-21T10:09:02.649 INFO:tasks.workunit.client.0.smithi153.stdout:0/736: symlink d4/d16/d19/d30/le7 0 2024-03-21T10:09:02.650 INFO:tasks.workunit.client.0.smithi153.stdout:8/767: dread d5/d9/d43/dd/d20/d26/d8b/d80/fa1 [0,4194304] 0 2024-03-21T10:09:02.653 INFO:tasks.workunit.client.1.smithi174.stdout:4/730: getdents d4/d21/d24/d28/de9/d6a/dac 0 2024-03-21T10:09:02.654 INFO:tasks.workunit.client.0.smithi153.stdout:3/852: fdatasync dd/d1e/d2f/d35/daf/dcc/f119 0 2024-03-21T10:09:02.657 INFO:tasks.workunit.client.0.smithi153.stdout:1/655: truncate d6/d13/d75/f83 150997 0 2024-03-21T10:09:02.657 INFO:tasks.workunit.client.0.smithi153.stdout:8/768: stat d5/d9/d43/dd/d20/d26/d24/d52/d55/f71 0 2024-03-21T10:09:02.657 INFO:tasks.workunit.client.1.smithi174.stdout:4/731: symlink d4/d21/d24/d28/de9/d3a/d67/d76/d82/lf2 0 2024-03-21T10:09:02.660 INFO:tasks.workunit.client.0.smithi153.stdout:2/857: truncate dd/d15/d3c/f69 1398491 0 2024-03-21T10:09:02.663 INFO:tasks.workunit.client.1.smithi174.stdout:4/732: creat d4/d21/d24/d28/de9/d6a/dab/ff3 x:0 0 0 2024-03-21T10:09:02.668 INFO:tasks.workunit.client.0.smithi153.stdout:4/822: unlink f10 0 2024-03-21T10:09:02.670 INFO:tasks.workunit.client.1.smithi174.stdout:9/652: dread d2/d24/d2e/f48 [0,4194304] 0 2024-03-21T10:09:02.671 INFO:tasks.workunit.client.0.smithi153.stdout:4/823: stat d11/d2d/d6b/d9c/cfd 0 2024-03-21T10:09:02.672 INFO:tasks.workunit.client.0.smithi153.stdout:7/780: dwrite d4/d14/d2f/d78/f35 [0,4194304] 0 2024-03-21T10:09:02.673 INFO:tasks.workunit.client.1.smithi174.stdout:4/733: rename d4/d21/d24/d28/de9/d6a/dab/ca1 to d4/d21/d24/d28/de9/cf4 0 2024-03-21T10:09:02.673 INFO:tasks.workunit.client.1.smithi174.stdout:3/632: dwrite d0/d12/d14/d1a/d27/f66 [0,4194304] 0 2024-03-21T10:09:02.674 INFO:tasks.workunit.client.1.smithi174.stdout:9/653: write d2/d11/d1d/d5e/fc3 [708681,24183] 0 2024-03-21T10:09:02.675 INFO:tasks.workunit.client.0.smithi153.stdout:5/792: mkdir d4/d5/df/d67/d86/d96/deb/df0/d10a 0 2024-03-21T10:09:02.676 INFO:tasks.workunit.client.1.smithi174.stdout:3/633: read - d0/d12/d14/d1a/d5c/f8d zero size 2024-03-21T10:09:02.678 INFO:tasks.workunit.client.1.smithi174.stdout:1/714: sync 2024-03-21T10:09:02.678 INFO:tasks.workunit.client.1.smithi174.stdout:2/600: sync 2024-03-21T10:09:02.678 INFO:tasks.workunit.client.1.smithi174.stdout:3/634: fsync d0/d12/d14/d1a/d5c/f8d 0 2024-03-21T10:09:02.679 INFO:tasks.workunit.client.1.smithi174.stdout:9/654: mknod d2/d11/d1d/dab/ceb 0 2024-03-21T10:09:02.681 INFO:tasks.workunit.client.1.smithi174.stdout:2/601: truncate d5/d25/fbd 291369 0 2024-03-21T10:09:02.682 INFO:tasks.workunit.client.0.smithi153.stdout:6/966: creat d1f/de0/de1/df2/f155 x:0 0 0 2024-03-21T10:09:02.682 INFO:tasks.workunit.client.1.smithi174.stdout:3/635: write d0/d12/d14/d67/dc8/fc5 [296278,37160] 0 2024-03-21T10:09:02.684 INFO:tasks.workunit.client.0.smithi153.stdout:3/853: creat dd/d1e/d3d/d94/db8/f129 x:0 0 0 2024-03-21T10:09:02.685 INFO:tasks.workunit.client.1.smithi174.stdout:2/602: chown d5/d25/d27/d2d/d45/d5b/d2f/d4b/d76 89088467 1 2024-03-21T10:09:02.685 INFO:tasks.workunit.client.1.smithi174.stdout:9/655: unlink d2/d24/d2e/f48 0 2024-03-21T10:09:02.686 INFO:tasks.workunit.client.0.smithi153.stdout:1/656: dread - d6/d32/f6e zero size 2024-03-21T10:09:02.687 INFO:tasks.workunit.client.0.smithi153.stdout:3/854: readlink dd/d1e/le4 0 2024-03-21T10:09:02.688 INFO:tasks.workunit.client.1.smithi174.stdout:9/656: unlink d2/f18 0 2024-03-21T10:09:02.690 INFO:tasks.workunit.client.0.smithi153.stdout:3/855: dread - dd/d41/d80/dbb/def/ff3 zero size 2024-03-21T10:09:02.690 INFO:tasks.workunit.client.0.smithi153.stdout:8/769: rmdir d5/d9/d43/dd/d20/d26/d24/d52 39 2024-03-21T10:09:02.692 INFO:tasks.workunit.client.0.smithi153.stdout:3/856: fdatasync dd/d1e/d3d/d94/d73/ff5 0 2024-03-21T10:09:02.696 INFO:tasks.workunit.client.0.smithi153.stdout:3/857: fsync dd/d1e/d2f/d35/daf/f120 0 2024-03-21T10:09:02.697 INFO:tasks.workunit.client.1.smithi174.stdout:2/603: rename d5/d25/d27/d2e/d4f/d65/d77/fc6 to d5/d25/d27/d74/fcc 0 2024-03-21T10:09:02.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:02 smithi174 ceph-mon[37496]: pgmap v17: 65 pgs: 65 active+clean; 2.9 GiB data, 8.4 GiB used, 528 GiB / 536 GiB avail; 100 MiB/s rd, 350 MiB/s wr, 638 op/s 2024-03-21T10:09:02.701 INFO:tasks.workunit.client.1.smithi174.stdout:2/604: readlink d5/l33 0 2024-03-21T10:09:02.703 INFO:tasks.workunit.client.0.smithi153.stdout:3/858: write dd/d1e/d2f/d35/f110 [2224826,6204] 0 2024-03-21T10:09:02.703 INFO:tasks.workunit.client.0.smithi153.stdout:0/737: unlink d4/d16/d19/lb9 0 2024-03-21T10:09:02.705 INFO:tasks.workunit.client.0.smithi153.stdout:3/859: chown dd/d1e/ff0 182754412 1 2024-03-21T10:09:02.705 INFO:tasks.workunit.client.0.smithi153.stdout:1/657: unlink d6/d3a/d3c/d50/da8/cb2 0 2024-03-21T10:09:02.706 INFO:tasks.workunit.client.1.smithi174.stdout:8/583: dread d9/d2a/d39/d76/d19/d31/f94 [0,4194304] 0 2024-03-21T10:09:02.707 INFO:tasks.workunit.client.1.smithi174.stdout:9/657: dwrite d2/d24/d2e/dac/fcc [0,4194304] 0 2024-03-21T10:09:02.709 INFO:tasks.workunit.client.0.smithi153.stdout:8/770: mknod d5/d9/d43/d7b/cfc 0 2024-03-21T10:09:02.715 INFO:tasks.workunit.client.1.smithi174.stdout:2/605: rename d5/d25/d27/d2d/d45/d5b/d2f/f87 to d5/d25/d27/d2e/d85/dad/fcd 0 2024-03-21T10:09:02.717 INFO:tasks.workunit.client.1.smithi174.stdout:8/584: mkdir d9/d2a/d39/d76/d19/d27/d60/daa/dab/dc2 0 2024-03-21T10:09:02.718 INFO:tasks.workunit.client.0.smithi153.stdout:1/658: write d6/d10/d35/da5/d4f/d9d/fd2 [2687754,37833] 0 2024-03-21T10:09:02.719 INFO:tasks.workunit.client.1.smithi174.stdout:9/658: mknod d2/d24/d2e/de6/cec 0 2024-03-21T10:09:02.720 INFO:tasks.workunit.client.1.smithi174.stdout:0/610: dread d9/f13 [0,4194304] 0 2024-03-21T10:09:02.723 INFO:tasks.workunit.client.1.smithi174.stdout:5/679: write d1/d49/fdd [839744,45986] 0 2024-03-21T10:09:02.723 INFO:tasks.workunit.client.0.smithi153.stdout:7/781: rename d4/d11/d56/c67 to d4/d11/da1/c109 0 2024-03-21T10:09:02.724 INFO:tasks.workunit.client.0.smithi153.stdout:6/967: dwrite d1f/d93/dd8/f47 [0,4194304] 0 2024-03-21T10:09:02.725 INFO:tasks.workunit.client.1.smithi174.stdout:5/680: fsync d1/d49/d4f/f57 0 2024-03-21T10:09:02.727 INFO:tasks.workunit.client.1.smithi174.stdout:0/611: mknod d9/d1c/d76/d87/dcd/cd7 0 2024-03-21T10:09:02.727 INFO:tasks.workunit.client.0.smithi153.stdout:3/860: creat dd/f12a x:0 0 0 2024-03-21T10:09:02.728 INFO:tasks.workunit.client.0.smithi153.stdout:5/793: dread d4/d5/df/d5e/f82 [0,4194304] 0 2024-03-21T10:09:02.728 INFO:tasks.workunit.client.1.smithi174.stdout:9/659: creat d2/d11/d1d/d8b/fed x:0 0 0 2024-03-21T10:09:02.730 INFO:tasks.workunit.client.1.smithi174.stdout:5/681: symlink d1/d5c/d96/lf9 0 2024-03-21T10:09:02.733 INFO:tasks.workunit.client.0.smithi153.stdout:3/861: chown dd/d1e/d45/cc0 150 1 2024-03-21T10:09:02.733 INFO:tasks.workunit.client.0.smithi153.stdout:8/771: readlink d5/d9/d43/dd/d20/d26/d24/d52/le9 0 2024-03-21T10:09:02.735 INFO:tasks.workunit.client.1.smithi174.stdout:9/660: creat d2/d11/d1d/d4b/d72/fee x:0 0 0 2024-03-21T10:09:02.735 INFO:tasks.workunit.client.1.smithi174.stdout:0/612: mkdir d9/dd8 0 2024-03-21T10:09:02.736 INFO:tasks.workunit.client.1.smithi174.stdout:6/487: dread d0/d3/d5/d6/f4a [0,4194304] 0 2024-03-21T10:09:02.741 INFO:tasks.workunit.client.1.smithi174.stdout:1/715: write d4/d2d/f76 [574720,74937] 0 2024-03-21T10:09:02.744 INFO:tasks.workunit.client.0.smithi153.stdout:6/968: truncate d1f/d2b/d10a/d40/d51/ff3 3705944 0 2024-03-21T10:09:02.751 INFO:tasks.workunit.client.0.smithi153.stdout:1/659: dwrite d6/d13/f7c [0,4194304] 0 2024-03-21T10:09:02.751 INFO:tasks.workunit.client.1.smithi174.stdout:9/661: creat d2/d24/fef x:0 0 0 2024-03-21T10:09:02.752 INFO:tasks.workunit.client.1.smithi174.stdout:0/613: mkdir d9/d21/d5f/dd9 0 2024-03-21T10:09:02.752 INFO:tasks.workunit.client.0.smithi153.stdout:1/660: fsync f5 0 2024-03-21T10:09:02.754 INFO:tasks.workunit.client.1.smithi174.stdout:4/734: dwrite d4/d21/fd6 [0,4194304] 0 2024-03-21T10:09:02.755 INFO:tasks.workunit.client.1.smithi174.stdout:0/614: dread - d9/d21/d31/d5c/d86/fbf zero size 2024-03-21T10:09:02.757 INFO:tasks.workunit.client.0.smithi153.stdout:8/772: creat d5/d9/d43/dd/d20/d26/d8b/ffd x:0 0 0 2024-03-21T10:09:02.757 INFO:tasks.workunit.client.1.smithi174.stdout:6/488: creat d0/d3/d2a/d8f/f97 x:0 0 0 2024-03-21T10:09:02.758 INFO:tasks.workunit.client.0.smithi153.stdout:3/862: creat dd/d1e/d3d/d53/dae/d126/f12b x:0 0 0 2024-03-21T10:09:02.764 INFO:tasks.workunit.client.0.smithi153.stdout:6/969: read d1f/d2b/d10a/d40/d5f/f80 [811485,22979] 0 2024-03-21T10:09:02.769 INFO:tasks.workunit.client.0.smithi153.stdout:1/661: truncate d6/d10/fb9 742220 0 2024-03-21T10:09:02.769 INFO:tasks.workunit.client.1.smithi174.stdout:4/735: mknod d4/d21/d24/d28/de9/d3a/dd2/cf5 0 2024-03-21T10:09:02.770 INFO:tasks.workunit.client.0.smithi153.stdout:7/782: dread d4/d11/f7f [0,4194304] 0 2024-03-21T10:09:02.770 INFO:tasks.workunit.client.0.smithi153.stdout:2/858: dread dd/d15/d18/d2c/d4b/d56/d63/d8d/d9b/dc3/df5/dae/d50/d59/fe7 [0,4194304] 0 2024-03-21T10:09:02.771 INFO:tasks.workunit.client.1.smithi174.stdout:8/585: write d9/d2a/f3e [2508010,13561] 0 2024-03-21T10:09:02.771 INFO:tasks.workunit.client.1.smithi174.stdout:2/606: truncate d5/f12 1528896 0 2024-03-21T10:09:02.771 INFO:tasks.workunit.client.1.smithi174.stdout:4/736: read - d4/d21/f5b zero size 2024-03-21T10:09:02.776 INFO:tasks.workunit.client.1.smithi174.stdout:0/615: symlink d9/dd8/lda 0 2024-03-21T10:09:02.778 INFO:tasks.workunit.client.1.smithi174.stdout:4/737: read d4/d19/f74 [98581,125115] 0 2024-03-21T10:09:02.786 INFO:tasks.workunit.client.1.smithi174.stdout:4/738: symlink d4/d21/lf6 0 2024-03-21T10:09:02.788 INFO:tasks.workunit.client.0.smithi153.stdout:7/783: dread d4/d2a/d40/d50/f9f [0,4194304] 0 2024-03-21T10:09:02.791 INFO:tasks.workunit.client.1.smithi174.stdout:0/616: creat d9/d21/d31/d5c/d92/fdb x:0 0 0 2024-03-21T10:09:02.792 INFO:tasks.workunit.client.0.smithi153.stdout:7/784: readlink d4/d2a/l7e 0 2024-03-21T10:09:02.792 INFO:tasks.workunit.client.0.smithi153.stdout:6/970: symlink d1f/d93/dd8/d59/d70/d13c/l156 0 2024-03-21T10:09:02.795 INFO:tasks.workunit.client.1.smithi174.stdout:4/739: symlink d4/d21/lf7 0 2024-03-21T10:09:02.801 INFO:tasks.workunit.client.0.smithi153.stdout:5/794: dwrite d4/d8/d5a/d5d/dd9/d44/de1/fd1 [0,4194304] 0 2024-03-21T10:09:02.801 INFO:tasks.workunit.client.1.smithi174.stdout:9/662: dwrite d2/d24/d2e/fb1 [0,4194304] 0 2024-03-21T10:09:02.806 INFO:tasks.workunit.client.1.smithi174.stdout:5/682: dwrite d1/d49/d93/d9c/fce [4194304,4194304] 0 2024-03-21T10:09:02.806 INFO:tasks.workunit.client.1.smithi174.stdout:1/716: dwrite d4/d7/d15/f9b [4194304,4194304] 0 2024-03-21T10:09:02.806 INFO:tasks.workunit.client.1.smithi174.stdout:2/607: truncate d5/d25/d27/d2e/d85/dad/fb0 3464623 0 2024-03-21T10:09:02.808 INFO:tasks.workunit.client.1.smithi174.stdout:0/617: mkdir d9/d21/d31/d5c/d92/d98/db7/ddc 0 2024-03-21T10:09:02.809 INFO:tasks.workunit.client.0.smithi153.stdout:6/971: symlink d1f/d99/l157 0 2024-03-21T10:09:02.809 INFO:tasks.workunit.client.0.smithi153.stdout:7/785: creat d4/d2a/d3b/d81/f10a x:0 0 0 2024-03-21T10:09:02.811 INFO:tasks.workunit.client.1.smithi174.stdout:1/717: dread - d4/d67/fde zero size 2024-03-21T10:09:02.813 INFO:tasks.workunit.client.0.smithi153.stdout:3/863: truncate dd/d1e/d45/fc8 188201 0 2024-03-21T10:09:02.815 INFO:tasks.workunit.client.1.smithi174.stdout:9/663: symlink d2/d24/d2e/dac/lf0 0 2024-03-21T10:09:02.816 INFO:tasks.workunit.client.0.smithi153.stdout:1/662: write d6/d10/f11 [358133,125504] 0 2024-03-21T10:09:02.817 INFO:tasks.workunit.client.1.smithi174.stdout:4/740: symlink d4/d21/d24/lf8 0 2024-03-21T10:09:02.818 INFO:tasks.workunit.client.1.smithi174.stdout:1/718: chown d4/d5/fc9 142179 1 2024-03-21T10:09:02.820 INFO:tasks.workunit.client.0.smithi153.stdout:2/859: getdents dd/d15/d18/d2c/d33/d98/d127 0 2024-03-21T10:09:02.821 INFO:tasks.workunit.client.1.smithi174.stdout:5/683: creat d1/d16/d2e/d8d/d99/ffa x:0 0 0 2024-03-21T10:09:02.821 INFO:tasks.workunit.client.1.smithi174.stdout:9/664: rmdir d2/d11/d1d/d5e 39 2024-03-21T10:09:02.822 INFO:tasks.workunit.client.0.smithi153.stdout:4/824: sync 2024-03-21T10:09:02.822 INFO:tasks.workunit.client.0.smithi153.stdout:9/850: sync 2024-03-21T10:09:02.822 INFO:tasks.workunit.client.0.smithi153.stdout:7/786: mknod d4/d14/d73/c10b 0 2024-03-21T10:09:02.822 INFO:tasks.workunit.client.1.smithi174.stdout:0/618: creat d9/d35/d4a/d67/fdd x:0 0 0 2024-03-21T10:09:02.822 INFO:tasks.workunit.client.1.smithi174.stdout:5/684: stat d1/d49/d4f/ca0 0 2024-03-21T10:09:02.824 INFO:tasks.workunit.client.1.smithi174.stdout:9/665: chown d2/d24/d2e/l85 0 1 2024-03-21T10:09:02.825 INFO:tasks.workunit.client.1.smithi174.stdout:1/719: rmdir d4/d7/d33/d43/d51/da4 39 2024-03-21T10:09:02.826 INFO:tasks.workunit.client.0.smithi153.stdout:3/864: read dd/d1e/d45/f52 [1936647,32510] 0 2024-03-21T10:09:02.826 INFO:tasks.workunit.client.0.smithi153.stdout:4/825: chown d11/f96 1488 1 2024-03-21T10:09:02.826 INFO:tasks.workunit.client.0.smithi153.stdout:7/787: stat d4/d11/d56/db1 0 2024-03-21T10:09:02.828 INFO:tasks.workunit.client.0.smithi153.stdout:1/663: fsync d6/d10/d35/da5/d4f/d9d/dc7/dcc/f68 0 2024-03-21T10:09:02.829 INFO:tasks.workunit.client.1.smithi174.stdout:5/685: chown d1/d5c/d73/de3 22958325 1 2024-03-21T10:09:02.829 INFO:tasks.workunit.client.1.smithi174.stdout:1/720: readlink d4/d7/l35 0 2024-03-21T10:09:02.829 INFO:tasks.workunit.client.0.smithi153.stdout:9/851: write d1/d4/d5f/df3/f10c [843072,10957] 0 2024-03-21T10:09:02.830 INFO:tasks.workunit.client.1.smithi174.stdout:4/741: mknod d4/d21/d24/d28/de9/d3a/cf9 0 2024-03-21T10:09:02.830 INFO:tasks.workunit.client.0.smithi153.stdout:5/795: link d4/d5/f7 d4/d5/df/d67/d86/d96/deb/d56/d105/f10b 0 2024-03-21T10:09:02.833 INFO:tasks.workunit.client.0.smithi153.stdout:0/738: dread d4/f37 [0,4194304] 0 2024-03-21T10:09:02.837 INFO:tasks.workunit.client.1.smithi174.stdout:7/533: sync 2024-03-21T10:09:02.843 INFO:tasks.workunit.client.0.smithi153.stdout:3/865: truncate dd/d1e/d45/fec 536438 0 2024-03-21T10:09:02.843 INFO:tasks.workunit.client.0.smithi153.stdout:8/773: truncate d5/d9/d43/dd/d2d/d45/fc1 40948 0 2024-03-21T10:09:02.843 INFO:tasks.workunit.client.0.smithi153.stdout:7/788: rmdir d4/d11/da1 39 2024-03-21T10:09:02.844 INFO:tasks.workunit.client.1.smithi174.stdout:1/721: symlink d4/d7/d28/l100 0 2024-03-21T10:09:02.850 INFO:tasks.workunit.client.1.smithi174.stdout:7/534: readlink d0/d4/d10/d1b/l84 0 2024-03-21T10:09:02.850 INFO:tasks.workunit.client.1.smithi174.stdout:2/608: dwrite d5/d25/d27/d2e/d7b/f99 [0,4194304] 0 2024-03-21T10:09:02.850 INFO:tasks.workunit.client.1.smithi174.stdout:4/742: truncate d4/d21/d24/ff1 674074 0 2024-03-21T10:09:02.851 INFO:tasks.workunit.client.0.smithi153.stdout:1/664: mknod d6/d10/d35/da5/d4f/cd3 0 2024-03-21T10:09:02.851 INFO:tasks.workunit.client.0.smithi153.stdout:7/789: write d4/d14/d16/fed [1640925,28527] 0 2024-03-21T10:09:02.857 INFO:tasks.workunit.client.1.smithi174.stdout:8/586: dread d9/d2a/d39/d76/d19/d27/d60/fb5 [0,4194304] 0 2024-03-21T10:09:02.857 INFO:tasks.workunit.client.1.smithi174.stdout:9/666: rename d2/d24/f2f to d2/d11/d1d/d2a/da7/ff1 0 2024-03-21T10:09:02.861 INFO:tasks.workunit.client.1.smithi174.stdout:2/609: chown d5/d25/d27/d2e/d90/da9/lc2 782853 1 2024-03-21T10:09:02.862 INFO:tasks.workunit.client.1.smithi174.stdout:9/667: chown d2/d11/d7d/fde 636 1 2024-03-21T10:09:02.864 INFO:tasks.workunit.client.0.smithi153.stdout:5/796: dread d4/d8/d5a/d5d/f5f [0,4194304] 0 2024-03-21T10:09:02.865 INFO:tasks.workunit.client.1.smithi174.stdout:8/587: mkdir d9/d2a/d39/d58/dbb/dc3 0 2024-03-21T10:09:02.866 INFO:tasks.workunit.client.1.smithi174.stdout:0/619: dwrite d9/d1c/d76/d87/dcd/fcf [0,4194304] 0 2024-03-21T10:09:02.867 INFO:tasks.workunit.client.0.smithi153.stdout:4/826: creat d11/d18/d6d/dd6/ddd/f101 x:0 0 0 2024-03-21T10:09:02.870 INFO:tasks.workunit.client.0.smithi153.stdout:4/827: chown d11/d2d/da3 27018456 1 2024-03-21T10:09:02.872 INFO:tasks.workunit.client.0.smithi153.stdout:7/790: mkdir d4/d11/d56/d10c 0 2024-03-21T10:09:02.875 INFO:tasks.workunit.client.1.smithi174.stdout:5/686: dwrite d1/f86 [0,4194304] 0 2024-03-21T10:09:02.876 INFO:tasks.workunit.client.1.smithi174.stdout:8/588: truncate d9/f7c 34413 0 2024-03-21T10:09:02.876 INFO:tasks.workunit.client.0.smithi153.stdout:7/791: fdatasync d4/d19/fc5 0 2024-03-21T10:09:02.880 INFO:tasks.workunit.client.0.smithi153.stdout:7/792: truncate d4/fbd 111954 0 2024-03-21T10:09:02.880 INFO:tasks.workunit.client.0.smithi153.stdout:3/866: symlink dd/d1e/d3d/d94/d73/d7f/l12c 0 2024-03-21T10:09:02.885 INFO:tasks.workunit.client.1.smithi174.stdout:9/668: unlink d2/d11/d1d/d2a/c42 0 2024-03-21T10:09:02.888 INFO:tasks.workunit.client.0.smithi153.stdout:5/797: mkdir d4/d5/df/d67/d86/dbf/d10c 0 2024-03-21T10:09:02.889 INFO:tasks.workunit.client.1.smithi174.stdout:1/722: write d4/d7/d33/d43/f42 [2157449,24007] 0 2024-03-21T10:09:02.891 INFO:tasks.workunit.client.1.smithi174.stdout:8/589: mkdir d9/d2a/d39/d76/d19/d27/d60/daa/dab/dc4 0 2024-03-21T10:09:02.891 INFO:tasks.workunit.client.0.smithi153.stdout:8/774: dwrite d5/d9/d43/fe6 [0,4194304] 0 2024-03-21T10:09:02.891 INFO:tasks.workunit.client.0.smithi153.stdout:6/972: dread d1f/d2b/d10a/d40/d51/ff3 [0,4194304] 0 2024-03-21T10:09:02.893 INFO:tasks.workunit.client.0.smithi153.stdout:2/860: dwrite dd/d15/d3c/da7/fe5 [0,4194304] 0 2024-03-21T10:09:02.896 INFO:tasks.workunit.client.1.smithi174.stdout:0/620: creat d9/fde x:0 0 0 2024-03-21T10:09:02.897 INFO:tasks.workunit.client.1.smithi174.stdout:7/535: dwrite d0/d4/d10/d1b/fa6 [0,4194304] 0 2024-03-21T10:09:02.898 INFO:tasks.workunit.client.1.smithi174.stdout:7/536: chown d0/d4 470 1 2024-03-21T10:09:02.900 INFO:tasks.workunit.client.0.smithi153.stdout:1/665: rename d6/d10/d35/da5/lb8 to d6/d10/d35/ld4 0 2024-03-21T10:09:02.900 INFO:tasks.workunit.client.1.smithi174.stdout:0/621: mkdir d9/d21/d23/da5/ddf 0 2024-03-21T10:09:02.903 INFO:tasks.workunit.client.1.smithi174.stdout:8/590: rename d9/d2a/d39/d76/f8d to d9/d2a/d4d/fc5 0 2024-03-21T10:09:02.905 INFO:tasks.workunit.client.1.smithi174.stdout:7/537: rmdir d0/d53/d5f/d65 39 2024-03-21T10:09:02.906 INFO:tasks.workunit.client.0.smithi153.stdout:8/775: read d5/d9/d43/d7b/daa/dca/fd7 [410447,106537] 0 2024-03-21T10:09:02.907 INFO:tasks.workunit.client.1.smithi174.stdout:8/591: unlink d9/d2a/d39/d76/f3c 0 2024-03-21T10:09:02.908 INFO:tasks.workunit.client.0.smithi153.stdout:9/852: getdents d1/d3b/daa 0 2024-03-21T10:09:02.911 INFO:tasks.workunit.client.1.smithi174.stdout:7/538: mkdir d0/db6 0 2024-03-21T10:09:02.915 INFO:tasks.workunit.client.1.smithi174.stdout:8/592: rename d9/d2a/d39/d76/d19/d31 to d9/dc6 0 2024-03-21T10:09:02.916 INFO:tasks.workunit.client.0.smithi153.stdout:7/793: symlink d4/d2a/dc6/dfb/d9a/db7/l10d 0 2024-03-21T10:09:02.916 INFO:tasks.workunit.client.0.smithi153.stdout:3/867: dwrite dd/d1e/d2f/d35/d68/daa/f10c [0,4194304] 0 2024-03-21T10:09:02.921 INFO:tasks.workunit.client.0.smithi153.stdout:9/853: dread d1/d8a/fb9 [0,4194304] 0 2024-03-21T10:09:02.921 INFO:tasks.workunit.client.0.smithi153.stdout:3/868: dread - dd/d41/d80/dbb/def/ff3 zero size 2024-03-21T10:09:02.924 INFO:tasks.workunit.client.0.smithi153.stdout:1/666: dwrite d6/d10/da0/fc4 [0,4194304] 0 2024-03-21T10:09:02.928 INFO:tasks.workunit.client.0.smithi153.stdout:4/828: truncate d11/f16 1866174 0 2024-03-21T10:09:02.930 INFO:tasks.workunit.client.0.smithi153.stdout:4/829: chown d11/d18/d50/d65 50 1 2024-03-21T10:09:02.935 INFO:tasks.workunit.client.0.smithi153.stdout:5/798: mkdir d4/d5/df/d67/d86/d96/deb/d56/d105/d10d 0 2024-03-21T10:09:02.935 INFO:tasks.workunit.client.1.smithi174.stdout:2/610: dread d5/d25/d27/d2d/d45/d5b/d55/f98 [0,4194304] 0 2024-03-21T10:09:02.936 INFO:tasks.workunit.client.0.smithi153.stdout:7/794: rmdir d4/d14/dd0 39 2024-03-21T10:09:02.940 INFO:tasks.workunit.client.1.smithi174.stdout:0/622: dread d9/d21/d75/da1/fa8 [0,4194304] 0 2024-03-21T10:09:02.941 INFO:tasks.workunit.client.1.smithi174.stdout:4/743: truncate d4/d21/d24/d28/f5e 1245597 0 2024-03-21T10:09:02.942 INFO:tasks.workunit.client.0.smithi153.stdout:1/667: creat d6/d3a/d3c/d50/d51/fd5 x:0 0 0 2024-03-21T10:09:02.943 INFO:tasks.workunit.client.0.smithi153.stdout:3/869: fsync dd/d1e/d3d/d94/d8e/fe3 0 2024-03-21T10:09:02.950 INFO:tasks.workunit.client.0.smithi153.stdout:2/861: creat dd/d15/d18/d2c/d4b/d56/d63/d8d/d9b/dc3/f12e x:0 0 0 2024-03-21T10:09:02.950 INFO:tasks.workunit.client.0.smithi153.stdout:9/854: truncate d1/d4/d5f/fe6 492903 0 2024-03-21T10:09:02.952 INFO:tasks.workunit.client.1.smithi174.stdout:4/744: write d4/d21/d24/d28/de9/d6a/d8f/f94 [3017311,12201] 0 2024-03-21T10:09:02.953 INFO:tasks.workunit.client.0.smithi153.stdout:4/830: truncate d11/feb 1361613 0 2024-03-21T10:09:02.960 INFO:tasks.workunit.client.0.smithi153.stdout:2/862: rmdir dd/d15/d18/d2c/d4b/d56/d63/d8d 39 2024-03-21T10:09:02.961 INFO:tasks.workunit.client.1.smithi174.stdout:5/687: truncate d1/d16/d2e/d8d/d99/faf 6826010 0 2024-03-21T10:09:02.963 INFO:tasks.workunit.client.1.smithi174.stdout:0/623: dwrite d9/fc1 [0,4194304] 0 2024-03-21T10:09:02.966 INFO:tasks.workunit.client.0.smithi153.stdout:1/668: getdents d6/d32 0 2024-03-21T10:09:02.968 INFO:tasks.workunit.client.0.smithi153.stdout:1/669: dread - d6/d10/fc1 zero size 2024-03-21T10:09:02.969 INFO:tasks.workunit.client.0.smithi153.stdout:7/795: dwrite d4/d14/d2f/d78/f29 [0,4194304] 0 2024-03-21T10:09:02.970 INFO:tasks.workunit.client.0.smithi153.stdout:3/870: link dd/d1e/d3d/d94/d8e/f107 dd/d41/d80/f12d 0 2024-03-21T10:09:02.971 INFO:tasks.workunit.client.0.smithi153.stdout:8/776: dread d5/d9/d43/f16 [0,4194304] 0 2024-03-21T10:09:02.973 INFO:tasks.workunit.client.1.smithi174.stdout:1/723: dread d4/d7/d15/f41 [4194304,4194304] 0 2024-03-21T10:09:02.974 INFO:tasks.workunit.client.1.smithi174.stdout:7/539: dread d0/d4/d10/d1b/f25 [0,4194304] 0 2024-03-21T10:09:02.977 INFO:tasks.workunit.client.0.smithi153.stdout:4/831: link d11/d2d/d57/c59 d11/d5e/c102 0 2024-03-21T10:09:02.977 INFO:tasks.workunit.client.0.smithi153.stdout:9/855: dwrite d1/d43/d44/f33 [0,4194304] 0 2024-03-21T10:09:02.979 INFO:tasks.workunit.client.0.smithi153.stdout:4/832: chown d11/d18/d50/d65/de3 27343261 1 2024-03-21T10:09:02.984 INFO:tasks.workunit.client.1.smithi174.stdout:3/636: sync 2024-03-21T10:09:02.989 INFO:tasks.workunit.client.0.smithi153.stdout:1/670: symlink d6/d10/d35/da5/d4f/d9d/dc7/dcc/d80/ld6 0 2024-03-21T10:09:02.991 INFO:tasks.workunit.client.1.smithi174.stdout:8/593: write d9/d2a/d39/d76/d19/d27/d79/fba [1512372,36077] 0 2024-03-21T10:09:02.991 INFO:tasks.workunit.client.0.smithi153.stdout:8/777: mkdir d5/d9/dfe 0 2024-03-21T10:09:02.997 INFO:tasks.workunit.client.0.smithi153.stdout:2/863: mkdir dd/d15/d18/d2c/d4b/d56/d63/d8d/d9b/dc3/df5/d12f 0 2024-03-21T10:09:02.997 INFO:tasks.workunit.client.0.smithi153.stdout:7/796: truncate d4/d14/f4b 1114863 0 2024-03-21T10:09:02.998 INFO:tasks.workunit.client.1.smithi174.stdout:2/611: rename d5/d25/d27/d2e/d90/da9/lc2 to d5/d25/d27/d2d/d45/d5b/d2f/d4b/d76/lce 0 2024-03-21T10:09:03.001 INFO:tasks.workunit.client.0.smithi153.stdout:7/797: truncate d4/fd6 208376 0 2024-03-21T10:09:03.001 INFO:tasks.workunit.client.0.smithi153.stdout:4/833: dread d11/f78 [0,4194304] 0 2024-03-21T10:09:03.002 INFO:tasks.workunit.client.0.smithi153.stdout:8/778: symlink d5/d1b/lff 0 2024-03-21T10:09:03.002 INFO:tasks.workunit.client.1.smithi174.stdout:5/688: mknod d1/d5c/d73/de3/cfb 0 2024-03-21T10:09:03.002 INFO:tasks.workunit.client.1.smithi174.stdout:2/612: write d5/d25/d27/d2d/d45/f6e [4468167,127580] 0 2024-03-21T10:09:03.003 INFO:tasks.workunit.client.0.smithi153.stdout:3/871: rename dd/d1e/f46 to dd/d1e/d3d/d94/d73/d7f/f12e 0 2024-03-21T10:09:03.003 INFO:tasks.workunit.client.1.smithi174.stdout:1/724: unlink d4/d7/d15/l25 0 2024-03-21T10:09:03.004 INFO:tasks.workunit.client.1.smithi174.stdout:5/689: chown d1/d2f/dc1/fe8 3180 1 2024-03-21T10:09:03.006 INFO:tasks.workunit.client.1.smithi174.stdout:3/637: dread - d0/d12/d14/d1a/d44/d56/fa2 zero size 2024-03-21T10:09:03.009 INFO:tasks.workunit.client.1.smithi174.stdout:4/745: unlink d4/d21/d24/d28/de9/d6a/d8f/l96 0 2024-03-21T10:09:03.010 INFO:tasks.workunit.client.1.smithi174.stdout:2/613: mkdir d5/d25/d27/d2e/d4f/d65/d77/dcf 0 2024-03-21T10:09:03.010 INFO:tasks.workunit.client.0.smithi153.stdout:4/834: mkdir d11/d2d/da3/d103 0 2024-03-21T10:09:03.010 INFO:tasks.workunit.client.1.smithi174.stdout:5/690: read d1/d2f/d37/ff4 [2785312,56920] 0 2024-03-21T10:09:03.013 INFO:tasks.workunit.client.1.smithi174.stdout:7/540: truncate d0/d4/d38/f91 247539 0 2024-03-21T10:09:03.013 INFO:tasks.workunit.client.1.smithi174.stdout:5/691: chown d1/d49/c88 8 1 2024-03-21T10:09:03.013 INFO:tasks.workunit.client.0.smithi153.stdout:8/779: creat d5/d9/d43/dd/d20/d26/d24/d52/d55/f100 x:0 0 0 2024-03-21T10:09:03.020 INFO:tasks.workunit.client.0.smithi153.stdout:7/798: dread d4/d19/d4d/f95 [0,4194304] 0 2024-03-21T10:09:03.024 INFO:tasks.workunit.client.1.smithi174.stdout:1/725: link d4/d7/f2e d4/d5/f101 0 2024-03-21T10:09:03.024 INFO:tasks.workunit.client.0.smithi153.stdout:6/973: dread d1f/d93/dd8/d59/d70/d13c/f152 [0,4194304] 0 2024-03-21T10:09:03.024 INFO:tasks.workunit.client.0.smithi153.stdout:3/872: link dd/d1e/d45/fec dd/d1e/d2f/d35/d68/d75/f12f 0 2024-03-21T10:09:03.025 INFO:tasks.workunit.client.0.smithi153.stdout:1/671: dwrite d6/fc6 [0,4194304] 0 2024-03-21T10:09:03.025 INFO:tasks.workunit.client.1.smithi174.stdout:5/692: chown d1/d16/d1c/d81/f95 515585600 1 2024-03-21T10:09:03.026 INFO:tasks.workunit.client.1.smithi174.stdout:7/541: write d0/d4/d10/d44/d81/d9a/fa3 [569193,52142] 0 2024-03-21T10:09:03.026 INFO:tasks.workunit.client.0.smithi153.stdout:7/799: chown d4/d11/d56/db1/fba 1582438 1 2024-03-21T10:09:03.027 INFO:tasks.workunit.client.1.smithi174.stdout:8/594: rmdir d9/d2a/d39/d76/d19/d27/d60/daa/dab/dc2 0 2024-03-21T10:09:03.029 INFO:tasks.workunit.client.0.smithi153.stdout:4/835: rename d11/d71/d89/c94 to d11/d71/d89/dca/c104 0 2024-03-21T10:09:03.029 INFO:tasks.workunit.client.1.smithi174.stdout:8/595: write d9/d2a/d39/fa0 [3787338,22292] 0 2024-03-21T10:09:03.034 INFO:tasks.workunit.client.1.smithi174.stdout:2/614: mknod d5/d25/d27/d2d/d45/cd0 0 2024-03-21T10:09:03.035 INFO:tasks.workunit.client.0.smithi153.stdout:9/856: chown d1/d4/d5f/fe6 14 1 2024-03-21T10:09:03.036 INFO:tasks.workunit.client.1.smithi174.stdout:1/726: symlink d4/d7/d68/d7b/de0/l102 0 2024-03-21T10:09:03.039 INFO:tasks.workunit.client.0.smithi153.stdout:7/800: unlink d4/d19/l76 0 2024-03-21T10:09:03.039 INFO:tasks.workunit.client.1.smithi174.stdout:5/693: unlink d1/d5c/d73/f7f 0 2024-03-21T10:09:03.040 INFO:tasks.workunit.client.1.smithi174.stdout:2/615: chown d5/d25/l5a 7826438 1 2024-03-21T10:09:03.041 INFO:tasks.workunit.client.1.smithi174.stdout:4/746: creat d4/d21/d24/d28/de9/d3a/d7a/d9d/ffa x:0 0 0 2024-03-21T10:09:03.044 INFO:tasks.workunit.client.1.smithi174.stdout:4/747: chown d4/d21/d24/d28/de9/d6a 3147 1 2024-03-21T10:09:03.045 INFO:tasks.workunit.client.0.smithi153.stdout:8/780: creat d5/d9/d43/dd/d2d/d45/d88/de0/f101 x:0 0 0 2024-03-21T10:09:03.046 INFO:tasks.workunit.client.1.smithi174.stdout:2/616: truncate d5/d25/d27/d2d/d45/d5b/fb6 1786356 0 2024-03-21T10:09:03.046 INFO:tasks.workunit.client.0.smithi153.stdout:2/864: dread dd/d15/d18/d2c/f4e [0,4194304] 0 2024-03-21T10:09:03.047 INFO:tasks.workunit.client.1.smithi174.stdout:8/596: dread - d9/d2a/d39/d76/f70 zero size 2024-03-21T10:09:03.055 INFO:tasks.workunit.client.1.smithi174.stdout:1/727: symlink d4/d7/d33/d43/d77/d87/l103 0 2024-03-21T10:09:03.055 INFO:tasks.workunit.client.1.smithi174.stdout:8/597: dread - d9/d2a/d4d/db4/fb6 zero size 2024-03-21T10:09:03.056 INFO:tasks.workunit.client.0.smithi153.stdout:6/974: rename d1f/d99/l157 to d1f/d93/dd8/d59/d70/d13c/l158 0 2024-03-21T10:09:03.056 INFO:tasks.workunit.client.0.smithi153.stdout:7/801: rmdir d4/d14 39 2024-03-21T10:09:03.056 INFO:tasks.workunit.client.0.smithi153.stdout:2/865: symlink dd/d15/d5e/db3/l130 0 2024-03-21T10:09:03.056 INFO:tasks.workunit.client.1.smithi174.stdout:4/748: symlink d4/d21/lfb 0 2024-03-21T10:09:03.058 INFO:tasks.workunit.client.0.smithi153.stdout:9/857: fsync d1/d43/d44/d46/d9e/f31 0 2024-03-21T10:09:03.058 INFO:tasks.workunit.client.0.smithi153.stdout:3/873: write dd/d41/f49 [61283,50187] 0 2024-03-21T10:09:03.061 INFO:tasks.workunit.client.1.smithi174.stdout:7/542: dread d0/f11 [0,4194304] 0 2024-03-21T10:09:03.061 INFO:tasks.workunit.client.0.smithi153.stdout:1/672: dwrite d6/d13/f1d [0,4194304] 0 2024-03-21T10:09:03.063 INFO:tasks.workunit.client.1.smithi174.stdout:7/543: stat d0 0 2024-03-21T10:09:03.063 INFO:tasks.workunit.client.1.smithi174.stdout:5/694: dread d1/d16/d1c/f25 [0,4194304] 0 2024-03-21T10:09:03.069 INFO:tasks.workunit.client.1.smithi174.stdout:5/695: write d1/d16/d2e/d66/fbe [2743681,11195] 0 2024-03-21T10:09:03.069 INFO:tasks.workunit.client.1.smithi174.stdout:7/544: creat d0/d4/d38/fb7 x:0 0 0 2024-03-21T10:09:03.071 INFO:tasks.workunit.client.0.smithi153.stdout:0/739: sync 2024-03-21T10:09:03.073 INFO:tasks.workunit.client.1.smithi174.stdout:8/598: dwrite d9/fae [0,4194304] 0 2024-03-21T10:09:03.075 INFO:tasks.workunit.client.1.smithi174.stdout:8/599: chown d9/d2a/d39/d76/d19 54093180 1 2024-03-21T10:09:03.076 INFO:tasks.workunit.client.0.smithi153.stdout:1/673: mknod d6/d10/d35/cd7 0 2024-03-21T10:09:03.076 INFO:tasks.workunit.client.1.smithi174.stdout:6/489: sync 2024-03-21T10:09:03.076 INFO:tasks.workunit.client.1.smithi174.stdout:1/728: link d4/l11 d4/d7/l104 0 2024-03-21T10:09:03.079 INFO:tasks.workunit.client.0.smithi153.stdout:2/866: write dd/d15/d18/f2a [1123639,79348] 0 2024-03-21T10:09:03.081 INFO:tasks.workunit.client.1.smithi174.stdout:7/545: rename d0/d4/d10/d44/c46 to d0/d4/d38/cb8 0 2024-03-21T10:09:03.083 INFO:tasks.workunit.client.0.smithi153.stdout:1/674: fdatasync d6/f17 0 2024-03-21T10:09:03.089 INFO:tasks.workunit.client.1.smithi174.stdout:3/638: dwrite d0/d12/d14/d1a/d27/d3c/d73/db7/d7d/f8b [0,4194304] 0 2024-03-21T10:09:03.092 INFO:tasks.workunit.client.0.smithi153.stdout:6/975: dwrite d1f/d2b/db3/fcc [0,4194304] 0 2024-03-21T10:09:03.093 INFO:tasks.workunit.client.1.smithi174.stdout:1/729: creat d4/d5/f105 x:0 0 0 2024-03-21T10:09:03.093 INFO:tasks.workunit.client.0.smithi153.stdout:3/874: rename dd/d41/cc3 to dd/d1e/d2f/d35/c130 0 2024-03-21T10:09:03.097 INFO:tasks.workunit.client.0.smithi153.stdout:5/799: sync 2024-03-21T10:09:03.099 INFO:tasks.workunit.client.1.smithi174.stdout:7/546: creat d0/d4/d10/d60/da2/fb9 x:0 0 0 2024-03-21T10:09:03.099 INFO:tasks.workunit.client.1.smithi174.stdout:2/617: write d5/d25/d27/f4e [588682,104250] 0 2024-03-21T10:09:03.101 INFO:tasks.workunit.client.1.smithi174.stdout:5/696: dread d1/f4a [0,4194304] 0 2024-03-21T10:09:03.101 INFO:tasks.workunit.client.0.smithi153.stdout:2/867: creat dd/d15/d18/d3a/d3d/f131 x:0 0 0 2024-03-21T10:09:03.104 INFO:tasks.workunit.client.0.smithi153.stdout:5/800: dread - d4/ff8 zero size 2024-03-21T10:09:03.112 INFO:tasks.workunit.client.0.smithi153.stdout:6/976: mknod d1f/d2b/d10a/d14b/c159 0 2024-03-21T10:09:03.112 INFO:tasks.workunit.client.1.smithi174.stdout:0/624: sync 2024-03-21T10:09:03.112 INFO:tasks.workunit.client.1.smithi174.stdout:9/669: sync 2024-03-21T10:09:03.114 INFO:tasks.workunit.client.0.smithi153.stdout:3/875: symlink dd/d41/d80/dbb/d112/l131 0 2024-03-21T10:09:03.115 INFO:tasks.workunit.client.1.smithi174.stdout:2/618: creat d5/d25/d27/d2e/d4f/d65/fd1 x:0 0 0 2024-03-21T10:09:03.115 INFO:tasks.workunit.client.1.smithi174.stdout:4/749: dwrite d4/d21/d24/d28/de9/d3a/d7a/d9d/fc2 [0,4194304] 0 2024-03-21T10:09:03.117 INFO:tasks.workunit.client.1.smithi174.stdout:3/639: rename d0/d12/d14/d1a/d27/d3c/d73/db4/lca to d0/d12/d14/d1a/d44/d56/d76/ld3 0 2024-03-21T10:09:03.117 INFO:tasks.workunit.client.0.smithi153.stdout:1/675: link d6/d3a/d3c/fa7 d6/d10/d35/da5/d4f/d9d/dc7/dcc/fd8 0 2024-03-21T10:09:03.120 INFO:tasks.workunit.client.1.smithi174.stdout:5/697: creat d1/d2f/dc1/ffc x:0 0 0 2024-03-21T10:09:03.121 INFO:tasks.workunit.client.0.smithi153.stdout:8/781: dread d5/d9/d43/dd/d2d/fa5 [0,4194304] 0 2024-03-21T10:09:03.123 INFO:tasks.workunit.client.1.smithi174.stdout:9/670: readlink d2/d24/l38 0 2024-03-21T10:09:03.123 INFO:tasks.workunit.client.0.smithi153.stdout:7/802: dwrite d4/d14/d16/f42 [0,4194304] 0 2024-03-21T10:09:03.126 INFO:tasks.workunit.client.1.smithi174.stdout:7/547: symlink d0/d4/d10/d79/db2/lba 0 2024-03-21T10:09:03.127 INFO:tasks.workunit.client.0.smithi153.stdout:4/836: dread d11/d18/f55 [0,4194304] 0 2024-03-21T10:09:03.128 INFO:tasks.workunit.client.0.smithi153.stdout:3/876: mkdir dd/d1e/d3d/d53/dae/d126/d132 0 2024-03-21T10:09:03.131 INFO:tasks.workunit.client.1.smithi174.stdout:1/730: dwrite d4/d2d/f76 [0,4194304] 0 2024-03-21T10:09:03.131 INFO:tasks.workunit.client.1.smithi174.stdout:3/640: mknod d0/d12/d14/d67/da1/da7/cd4 0 2024-03-21T10:09:03.134 INFO:tasks.workunit.client.1.smithi174.stdout:5/698: rmdir d1/d16/d1c/d81 39 2024-03-21T10:09:03.137 INFO:tasks.workunit.client.0.smithi153.stdout:9/858: dwrite d1/d43/d44/f58 [0,4194304] 0 2024-03-21T10:09:03.141 INFO:tasks.workunit.client.0.smithi153.stdout:7/803: unlink d4/d2a/dc6/fef 0 2024-03-21T10:09:03.142 INFO:tasks.workunit.client.0.smithi153.stdout:1/676: link d6/d13/f7c d6/fd9 0 2024-03-21T10:09:03.143 INFO:tasks.workunit.client.0.smithi153.stdout:3/877: symlink dd/l133 0 2024-03-21T10:09:03.143 INFO:tasks.workunit.client.1.smithi174.stdout:6/490: dwrite d0/d3/d2a/f2f [4194304,4194304] 0 2024-03-21T10:09:03.144 INFO:tasks.workunit.client.1.smithi174.stdout:8/600: dwrite d9/f12 [4194304,4194304] 0 2024-03-21T10:09:03.145 INFO:tasks.workunit.client.0.smithi153.stdout:4/837: mknod d11/d2d/da3/dac/c105 0 2024-03-21T10:09:03.146 INFO:tasks.workunit.client.1.smithi174.stdout:7/548: symlink d0/d53/lbb 0 2024-03-21T10:09:03.146 INFO:tasks.workunit.client.0.smithi153.stdout:9/859: read d1/d43/d44/f25 [1842530,66118] 0 2024-03-21T10:09:03.147 INFO:tasks.workunit.client.0.smithi153.stdout:7/804: mkdir d4/d19/d4d/d10e 0 2024-03-21T10:09:03.151 INFO:tasks.workunit.client.0.smithi153.stdout:1/677: getdents d6/d10/da0 0 2024-03-21T10:09:03.152 INFO:tasks.workunit.client.0.smithi153.stdout:7/805: fsync d4/d2a/d3b/d60/f64 0 2024-03-21T10:09:03.153 INFO:tasks.workunit.client.1.smithi174.stdout:6/491: readlink d0/l5b 0 2024-03-21T10:09:03.154 INFO:tasks.workunit.client.1.smithi174.stdout:3/641: unlink d0/lf 0 2024-03-21T10:09:03.157 INFO:tasks.workunit.client.0.smithi153.stdout:3/878: truncate dd/d1e/d45/fc8 408131 0 2024-03-21T10:09:03.158 INFO:tasks.workunit.client.0.smithi153.stdout:1/678: creat d6/d13/d8e/fda x:0 0 0 2024-03-21T10:09:03.161 INFO:tasks.workunit.client.1.smithi174.stdout:0/625: dwrite d9/d21/d31/fc6 [0,4194304] 0 2024-03-21T10:09:03.162 INFO:tasks.workunit.client.0.smithi153.stdout:7/806: creat d4/d19/d4d/dd5/df7/f10f x:0 0 0 2024-03-21T10:09:03.162 INFO:tasks.workunit.client.0.smithi153.stdout:9/860: dread d1/d43/d44/d46/d9e/d24/d41/f6a [0,4194304] 0 2024-03-21T10:09:03.167 INFO:tasks.workunit.client.1.smithi174.stdout:2/619: symlink d5/d25/d27/d2e/d7b/d81/da1/ld2 0 2024-03-21T10:09:03.173 INFO:tasks.workunit.client.1.smithi174.stdout:6/492: rmdir d0/d3/d20/d3d/d81 39 2024-03-21T10:09:03.173 INFO:tasks.workunit.client.1.smithi174.stdout:4/750: dwrite d4/d21/d24/d28/de9/d3a/d67/d76/d82/fe4 [0,4194304] 0 2024-03-21T10:09:03.176 INFO:tasks.workunit.client.0.smithi153.stdout:0/740: dread d4/d16/d19/d58/da1/fd8 [0,4194304] 0 2024-03-21T10:09:03.178 INFO:tasks.workunit.client.1.smithi174.stdout:3/642: creat d0/d12/d14/d1a/d44/d65/d7e/fd5 x:0 0 0 2024-03-21T10:09:03.179 INFO:tasks.workunit.client.1.smithi174.stdout:7/549: mknod d0/d53/d5f/cbc 0 2024-03-21T10:09:03.187 INFO:tasks.workunit.client.1.smithi174.stdout:0/626: unlink d9/d21/d5f/l9c 0 2024-03-21T10:09:03.188 INFO:tasks.workunit.client.1.smithi174.stdout:1/731: link d4/d2d/lc8 d4/d7/d33/d43/d77/d87/l106 0 2024-03-21T10:09:03.188 INFO:tasks.workunit.client.1.smithi174.stdout:9/671: getdents d2/d24/d2e/dac 0 2024-03-21T10:09:03.188 INFO:tasks.workunit.client.1.smithi174.stdout:2/620: rmdir d5/d25/d27/d2d/d45 39 2024-03-21T10:09:03.191 INFO:tasks.workunit.client.1.smithi174.stdout:6/493: creat d0/d3/d2a/d56/d5e/f98 x:0 0 0 2024-03-21T10:09:03.191 INFO:tasks.workunit.client.1.smithi174.stdout:3/643: fsync d0/d12/d14/d4c/d7a/d9e/fb8 0 2024-03-21T10:09:03.197 INFO:tasks.workunit.client.0.smithi153.stdout:0/741: mkdir d4/d16/d27/dc5/dcd/de8 0 2024-03-21T10:09:03.199 INFO:tasks.workunit.client.0.smithi153.stdout:6/977: write d1f/d2b/f89 [277057,4413] 0 2024-03-21T10:09:03.199 INFO:tasks.workunit.client.0.smithi153.stdout:5/801: write d4/d8/d5a/f51 [647727,107094] 0 2024-03-21T10:09:03.200 INFO:tasks.workunit.client.0.smithi153.stdout:9/861: dwrite d1/d4/d5f/df3/f10f [0,4194304] 0 2024-03-21T10:09:03.201 INFO:tasks.workunit.client.1.smithi174.stdout:7/550: chown d0/d53/d5f/d65/fac 0 1 2024-03-21T10:09:03.203 INFO:tasks.workunit.client.1.smithi174.stdout:1/732: creat d4/d7/d32/d86/f107 x:0 0 0 2024-03-21T10:09:03.205 INFO:tasks.workunit.client.1.smithi174.stdout:2/621: fdatasync d5/d25/d27/d2e/d90/fc3 0 2024-03-21T10:09:03.211 INFO:tasks.workunit.client.1.smithi174.stdout:6/494: readlink d0/d3/d5/df/l7b 0 2024-03-21T10:09:03.211 INFO:tasks.workunit.client.1.smithi174.stdout:4/751: rename d4/d21/d24/d28/de9/d6a/f6e to d4/d21/d24/ffc 0 2024-03-21T10:09:03.220 INFO:tasks.workunit.client.0.smithi153.stdout:2/868: dread dd/d15/d18/d2c/d4b/d56/d63/f8b [0,4194304] 0 2024-03-21T10:09:03.225 INFO:tasks.workunit.client.0.smithi153.stdout:8/782: write d5/d9/d43/dd/d2d/d45/fc0 [404304,73920] 0 2024-03-21T10:09:03.227 INFO:tasks.workunit.client.1.smithi174.stdout:2/622: symlink d5/d25/d27/d2e/d90/da9/ld3 0 2024-03-21T10:09:03.227 INFO:tasks.workunit.client.0.smithi153.stdout:0/742: getdents d4/d4c 0 2024-03-21T10:09:03.228 INFO:tasks.workunit.client.0.smithi153.stdout:8/783: chown d5/d9/cde 16305 1 2024-03-21T10:09:03.228 INFO:tasks.workunit.client.1.smithi174.stdout:5/699: dwrite d1/d16/f40 [4194304,4194304] 0 2024-03-21T10:09:03.230 INFO:tasks.workunit.client.1.smithi174.stdout:2/623: stat d5/d25 0 2024-03-21T10:09:03.230 INFO:tasks.workunit.client.1.smithi174.stdout:0/627: dwrite d9/d35/d4a/d67/fdd [0,4194304] 0 2024-03-21T10:09:03.232 INFO:tasks.workunit.client.0.smithi153.stdout:4/838: dread d11/d2d/d6b/db6/fa9 [0,4194304] 0 2024-03-21T10:09:03.232 INFO:tasks.workunit.client.0.smithi153.stdout:6/978: getdents d1f/d2b/d10a/d40 0 2024-03-21T10:09:03.233 INFO:tasks.workunit.client.0.smithi153.stdout:0/743: chown d4/db/fd4 34625911 1 2024-03-21T10:09:03.237 INFO:tasks.workunit.client.0.smithi153.stdout:4/839: unlink d11/d5e/fa8 0 2024-03-21T10:09:03.238 INFO:tasks.workunit.client.0.smithi153.stdout:4/840: chown d11/d9d 1299 1 2024-03-21T10:09:03.241 INFO:tasks.workunit.client.0.smithi153.stdout:3/879: write dd/d1e/d2f/d35/daf/dcc/d58/f86 [4695569,115103] 0 2024-03-21T10:09:03.244 INFO:tasks.workunit.client.0.smithi153.stdout:6/979: creat d1f/d93/dd8/d59/dfc/f15a x:0 0 0 2024-03-21T10:09:03.244 INFO:tasks.workunit.client.0.smithi153.stdout:1/679: dwrite d6/d13/f2b [0,4194304] 0 2024-03-21T10:09:03.244 INFO:tasks.workunit.client.0.smithi153.stdout:4/841: rmdir d11/d9d/de4 39 2024-03-21T10:09:03.244 INFO:tasks.workunit.client.0.smithi153.stdout:7/807: write d4/d2a/f77 [811148,30052] 0 2024-03-21T10:09:03.246 INFO:tasks.workunit.client.0.smithi153.stdout:9/862: write d1/f1b [950455,89154] 0 2024-03-21T10:09:03.249 INFO:tasks.workunit.client.1.smithi174.stdout:3/644: dwrite d0/d12/d14/d1a/d27/d3c/fce [0,4194304] 0 2024-03-21T10:09:03.250 INFO:tasks.workunit.client.0.smithi153.stdout:9/863: readlink d1/d4/l7d 0 2024-03-21T10:09:03.259 INFO:tasks.workunit.client.0.smithi153.stdout:4/842: rename d11/f39 to d11/d2d/d3d/dc7/f106 0 2024-03-21T10:09:03.262 INFO:tasks.workunit.client.0.smithi153.stdout:1/680: creat d6/d32/fdb x:0 0 0 2024-03-21T10:09:03.263 INFO:tasks.workunit.client.1.smithi174.stdout:1/733: write d4/d5/d6c/f6f [220689,48723] 0 2024-03-21T10:09:03.265 INFO:tasks.workunit.client.1.smithi174.stdout:7/551: rename d0/d4/d10/d1b/f2e to d0/d4/d10/fbd 0 2024-03-21T10:09:03.265 INFO:tasks.workunit.client.0.smithi153.stdout:7/808: read d4/d2a/d3b/d60/f64 [239426,119146] 0 2024-03-21T10:09:03.266 INFO:tasks.workunit.client.1.smithi174.stdout:0/628: symlink d9/d21/d31/d3c/le0 0 2024-03-21T10:09:03.267 INFO:tasks.workunit.client.0.smithi153.stdout:8/784: dread d5/d9/d43/d7b/f98 [0,4194304] 0 2024-03-21T10:09:03.267 INFO:tasks.workunit.client.0.smithi153.stdout:6/980: mknod d1f/d2b/d10a/d40/d51/c15b 0 2024-03-21T10:09:03.267 INFO:tasks.workunit.client.1.smithi174.stdout:0/629: readlink l8 0 2024-03-21T10:09:03.269 INFO:tasks.workunit.client.0.smithi153.stdout:9/864: mkdir d1/d4/d5f/df3/d11a 0 2024-03-21T10:09:03.270 INFO:tasks.workunit.client.0.smithi153.stdout:5/802: write d4/d8/d5c/d90/f109 [971944,43590] 0 2024-03-21T10:09:03.270 INFO:tasks.workunit.client.0.smithi153.stdout:7/809: chown d4/d2a/d40/df8 207 1 2024-03-21T10:09:03.271 INFO:tasks.workunit.client.1.smithi174.stdout:4/752: write d4/d21/d24/d28/de9/d3a/fe1 [315820,58456] 0 2024-03-21T10:09:03.272 INFO:tasks.workunit.client.0.smithi153.stdout:2/869: write dd/d15/d18/d2c/ff6 [1286484,58483] 0 2024-03-21T10:09:03.272 INFO:tasks.workunit.client.0.smithi153.stdout:3/880: truncate dd/d1e/d2f/d35/daf/dcc/f79 4851767 0 2024-03-21T10:09:03.275 INFO:tasks.workunit.client.0.smithi153.stdout:1/681: creat d6/d32/fdc x:0 0 0 2024-03-21T10:09:03.278 INFO:tasks.workunit.client.1.smithi174.stdout:9/672: dread d2/d11/d1d/d4b/d72/fe2 [0,4194304] 0 2024-03-21T10:09:03.278 INFO:tasks.workunit.client.0.smithi153.stdout:9/865: write d1/d43/d44/d46/d9e/d2e/f111 [4767059,98082] 0 2024-03-21T10:09:03.278 INFO:tasks.workunit.client.0.smithi153.stdout:6/981: mkdir d1f/d93/dd8/d59/d70/d110/d15c 0 2024-03-21T10:09:03.280 INFO:tasks.workunit.client.0.smithi153.stdout:8/785: fdatasync d5/d9/d43/dd/d20/d26/d24/fa3 0 2024-03-21T10:09:03.282 INFO:tasks.workunit.client.1.smithi174.stdout:7/552: mknod d0/d53/d5f/d65/cbe 0 2024-03-21T10:09:03.287 INFO:tasks.workunit.client.1.smithi174.stdout:9/673: fsync d2/d11/d1d/f96 0 2024-03-21T10:09:03.287 INFO:tasks.workunit.client.1.smithi174.stdout:3/645: dread d0/d12/d14/d1a/d27/f66 [0,4194304] 0 2024-03-21T10:09:03.289 INFO:tasks.workunit.client.0.smithi153.stdout:1/682: mkdir d6/d32/ddd 0 2024-03-21T10:09:03.289 INFO:tasks.workunit.client.0.smithi153.stdout:0/744: dwrite d4/d16/d19/d58/fbf [4194304,4194304] 0 2024-03-21T10:09:03.289 INFO:tasks.workunit.client.0.smithi153.stdout:7/810: mkdir d4/d14/d110 0 2024-03-21T10:09:03.289 INFO:tasks.workunit.client.0.smithi153.stdout:5/803: truncate d4/d5/df/d67/da0/db3/fa9 4329184 0 2024-03-21T10:09:03.289 INFO:tasks.workunit.client.0.smithi153.stdout:3/881: symlink dd/d41/df1/l134 0 2024-03-21T10:09:03.290 INFO:tasks.workunit.client.1.smithi174.stdout:6/495: dread d0/d3/d5/f45 [0,4194304] 0 2024-03-21T10:09:03.294 INFO:tasks.workunit.client.0.smithi153.stdout:6/982: creat d1f/d93/dd8/d103/d13f/f15d x:0 0 0 2024-03-21T10:09:03.296 INFO:tasks.workunit.client.1.smithi174.stdout:5/700: link d1/d2f/c3d d1/d49/d4f/cfd 0 2024-03-21T10:09:03.300 INFO:tasks.workunit.client.1.smithi174.stdout:8/601: dread d9/f1a [0,4194304] 0 2024-03-21T10:09:03.303 INFO:tasks.workunit.client.0.smithi153.stdout:2/870: creat dd/d15/d18/d2c/d4b/d56/d63/d8d/d9b/dc3/f132 x:0 0 0 2024-03-21T10:09:03.307 INFO:tasks.workunit.client.0.smithi153.stdout:3/882: chown dd/f27 269156910 1 2024-03-21T10:09:03.307 INFO:tasks.workunit.client.0.smithi153.stdout:9/866: rmdir d1/d3b/daa/df9 39 2024-03-21T10:09:03.308 INFO:tasks.workunit.client.1.smithi174.stdout:5/701: symlink d1/d16/d2e/d66/d8b/lfe 0 2024-03-21T10:09:03.308 INFO:tasks.workunit.client.0.smithi153.stdout:7/811: chown d4/d19/d4d/f95 268510940 1 2024-03-21T10:09:03.311 INFO:tasks.workunit.client.1.smithi174.stdout:8/602: creat d9/d2a/d4d/db2/fc7 x:0 0 0 2024-03-21T10:09:03.312 INFO:tasks.workunit.client.1.smithi174.stdout:6/496: unlink d0/d3/d5/d3b/l74 0 2024-03-21T10:09:03.312 INFO:tasks.workunit.client.1.smithi174.stdout:1/734: link d4/d7/d15/d53/cb5 d4/d7/d68/d7b/c108 0 2024-03-21T10:09:03.314 INFO:tasks.workunit.client.1.smithi174.stdout:0/630: getdents d9/d21/d31/d5c/d92/d98 0 2024-03-21T10:09:03.318 INFO:tasks.workunit.client.1.smithi174.stdout:2/624: dwrite d5/d25/d27/d2e/d4f/d65/fa2 [0,4194304] 0 2024-03-21T10:09:03.321 INFO:tasks.workunit.client.1.smithi174.stdout:6/497: mknod d0/d3/d2a/c99 0 2024-03-21T10:09:03.323 INFO:tasks.workunit.client.1.smithi174.stdout:1/735: rename d4/d7/d15/c91 to d4/d67/c109 0 2024-03-21T10:09:03.324 INFO:tasks.workunit.client.0.smithi153.stdout:4/843: dwrite d11/d18/f4b [0,4194304] 0 2024-03-21T10:09:03.326 INFO:tasks.workunit.client.1.smithi174.stdout:2/625: chown d5/d25/d27/d2e/d85/dad/fb0 1819882 1 2024-03-21T10:09:03.329 INFO:tasks.workunit.client.1.smithi174.stdout:4/753: write d4/d19/f2d [4337325,27595] 0 2024-03-21T10:09:03.329 INFO:tasks.workunit.client.1.smithi174.stdout:7/553: write d0/d53/f8d [5773861,56436] 0 2024-03-21T10:09:03.333 INFO:tasks.workunit.client.1.smithi174.stdout:1/736: truncate d4/d7/d68/d89/fc1 913309 0 2024-03-21T10:09:03.333 INFO:tasks.workunit.client.1.smithi174.stdout:4/754: chown d4/d21/cd0 287864 1 2024-03-21T10:09:03.341 INFO:tasks.workunit.client.1.smithi174.stdout:8/603: dwrite d9/d53/f78 [0,4194304] 0 2024-03-21T10:09:03.342 INFO:tasks.workunit.client.0.smithi153.stdout:5/804: creat d4/d5/df/d67/d86/d96/deb/d56/d105/f10e x:0 0 0 2024-03-21T10:09:03.345 INFO:tasks.workunit.client.0.smithi153.stdout:3/883: rmdir dd/d1e/d45/d105 39 2024-03-21T10:09:03.346 INFO:tasks.workunit.client.1.smithi174.stdout:1/737: read d4/d5/d6c/fc0 [788476,89363] 0 2024-03-21T10:09:03.349 INFO:tasks.workunit.client.0.smithi153.stdout:4/844: dwrite d11/fb0 [0,4194304] 0 2024-03-21T10:09:03.355 INFO:tasks.workunit.client.0.smithi153.stdout:8/786: write d5/d9/d43/dd/d2d/f66 [2669807,67851] 0 2024-03-21T10:09:03.355 INFO:tasks.workunit.client.0.smithi153.stdout:6/983: symlink d1f/d93/dd8/d59/dfc/d135/l15e 0 2024-03-21T10:09:03.358 INFO:tasks.workunit.client.1.smithi174.stdout:6/498: write d0/d3/d5/d3b/f55 [1113965,112308] 0 2024-03-21T10:09:03.358 INFO:tasks.workunit.client.0.smithi153.stdout:8/787: stat d5/d9/d43/dd/d20/d26/d7a/f81 0 2024-03-21T10:09:03.360 INFO:tasks.workunit.client.0.smithi153.stdout:2/871: truncate dd/d15/d3c/fcf 203585 0 2024-03-21T10:09:03.363 INFO:tasks.workunit.client.0.smithi153.stdout:3/884: rmdir dd/d1e/d2f/d35/dd0/d109 39 2024-03-21T10:09:03.366 INFO:tasks.workunit.client.0.smithi153.stdout:7/812: creat d4/d11/d56/d10c/f111 x:0 0 0 2024-03-21T10:09:03.367 INFO:tasks.workunit.client.1.smithi174.stdout:9/674: dread d2/d54/f5b [0,4194304] 0 2024-03-21T10:09:03.367 INFO:tasks.workunit.client.0.smithi153.stdout:8/788: dread d5/d9/d43/dd/d20/d26/d47/d9a/db9/fcb [0,4194304] 0 2024-03-21T10:09:03.369 INFO:tasks.workunit.client.0.smithi153.stdout:7/813: write d4/d2a/fdc [742283,27258] 0 2024-03-21T10:09:03.372 INFO:tasks.workunit.client.0.smithi153.stdout:4/845: creat d11/d2d/da3/dac/dce/f107 x:0 0 0 2024-03-21T10:09:03.374 INFO:tasks.workunit.client.1.smithi174.stdout:9/675: fsync d2/d11/d1d/d4b/d72/fe0 0 2024-03-21T10:09:03.375 INFO:tasks.workunit.client.0.smithi153.stdout:7/814: dread - d4/d14/d16/fdb zero size 2024-03-21T10:09:03.378 INFO:tasks.workunit.client.0.smithi153.stdout:6/984: creat d1f/d2b/d10a/d40/d4f/f15f x:0 0 0 2024-03-21T10:09:03.379 INFO:tasks.workunit.client.0.smithi153.stdout:7/815: truncate d4/d19/fc5 674906 0 2024-03-21T10:09:03.380 INFO:tasks.workunit.client.0.smithi153.stdout:6/985: chown d1f/da0/la3 28987 1 2024-03-21T10:09:03.381 INFO:tasks.workunit.client.0.smithi153.stdout:0/745: dread d4/d16/d27/f5c [0,4194304] 0 2024-03-21T10:09:03.383 INFO:tasks.workunit.client.1.smithi174.stdout:9/676: chown d2/d24/d2e/d7e/c9d 15 1 2024-03-21T10:09:03.383 INFO:tasks.workunit.client.1.smithi174.stdout:2/626: write d5/d25/d27/d2d/d45/d5b/ff [3975041,113336] 0 2024-03-21T10:09:03.384 INFO:tasks.workunit.client.0.smithi153.stdout:7/816: truncate d4/d14/fa4 1516310 0 2024-03-21T10:09:03.385 INFO:tasks.workunit.client.0.smithi153.stdout:9/867: rename d1/d43/d44/d46/d4d/fb7 to d1/d43/d44/d46/d9e/d24/f11b 0 2024-03-21T10:09:03.387 INFO:tasks.workunit.client.0.smithi153.stdout:2/872: fdatasync dd/d15/d18/d2c/d4b/d56/d63/d8d/d9b/dc3/df5/dae/d50/d59/fe9 0 2024-03-21T10:09:03.388 INFO:tasks.workunit.client.0.smithi153.stdout:4/846: symlink d11/d2d/d6b/d9c/l108 0 2024-03-21T10:09:03.390 INFO:tasks.workunit.client.1.smithi174.stdout:7/554: dread d0/d4/f42 [0,4194304] 0 2024-03-21T10:09:03.391 INFO:tasks.workunit.client.0.smithi153.stdout:2/873: chown dd/d15/d18/d2c/d33/d98/dc7/ded 1287636 1 2024-03-21T10:09:03.391 INFO:tasks.workunit.client.1.smithi174.stdout:0/631: dread d9/d35/d4a/f5e [0,4194304] 0 2024-03-21T10:09:03.392 INFO:tasks.workunit.client.1.smithi174.stdout:5/702: dread d1/ff [4194304,4194304] 0 2024-03-21T10:09:03.393 INFO:tasks.workunit.client.1.smithi174.stdout:4/755: truncate d4/d21/fd6 3691711 0 2024-03-21T10:09:03.394 INFO:tasks.workunit.client.1.smithi174.stdout:8/604: truncate d9/dc6/f4f 3833655 0 2024-03-21T10:09:03.395 INFO:tasks.workunit.client.1.smithi174.stdout:9/677: write d2/d24/d2e/fc4 [1798541,65139] 0 2024-03-21T10:09:03.396 INFO:tasks.workunit.client.0.smithi153.stdout:0/746: rename d4/d16/d19/d58/da1 to d4/d16/d19/d58/de9 0 2024-03-21T10:09:03.397 INFO:tasks.workunit.client.1.smithi174.stdout:5/703: write d1/d16/d2e/d87/fcf [919100,85128] 0 2024-03-21T10:09:03.398 INFO:tasks.workunit.client.1.smithi174.stdout:6/499: truncate d0/d3/d5/d3b/f85 3255357 0 2024-03-21T10:09:03.398 INFO:tasks.workunit.client.1.smithi174.stdout:0/632: readlink d9/d21/d23/da5/lad 0 2024-03-21T10:09:03.399 INFO:tasks.workunit.client.0.smithi153.stdout:8/789: rename d5/d9/d43/d7b/dc4/da7/dfa to d5/d9/d43/d7b/daa/d102 0 2024-03-21T10:09:03.400 INFO:tasks.workunit.client.1.smithi174.stdout:2/627: mknod d5/d25/d27/d2e/d4f/d65/d77/cd4 0 2024-03-21T10:09:03.405 INFO:tasks.workunit.client.0.smithi153.stdout:9/868: creat d1/d43/d44/d46/d100/f11c x:0 0 0 2024-03-21T10:09:03.405 INFO:tasks.workunit.client.1.smithi174.stdout:5/704: chown d1/d16/d2e/l79 5525 1 2024-03-21T10:09:03.409 INFO:tasks.workunit.client.0.smithi153.stdout:6/986: creat d1f/d93/df7/f160 x:0 0 0 2024-03-21T10:09:03.409 INFO:tasks.workunit.client.0.smithi153.stdout:0/747: unlink d4/d16/d19/d95/cc9 0 2024-03-21T10:09:03.411 INFO:tasks.workunit.client.0.smithi153.stdout:6/987: chown d1f/d2b/d10a/d40/d5f/f14a 1 1 2024-03-21T10:09:03.412 INFO:tasks.workunit.client.1.smithi174.stdout:4/756: dread d4/d21/d24/d28/d79/fbb [0,4194304] 0 2024-03-21T10:09:03.412 INFO:tasks.workunit.client.1.smithi174.stdout:9/678: chown d2/d11/d1d/d2a/da7/fb3 98967 1 2024-03-21T10:09:03.412 INFO:tasks.workunit.client.1.smithi174.stdout:6/500: write d0/d3/d5/d21/f6c [4026341,97448] 0 2024-03-21T10:09:03.413 INFO:tasks.workunit.client.0.smithi153.stdout:4/847: rename d11/d18/d6d/dd6/ddd/f101 to d11/d2d/d57/ded/f109 0 2024-03-21T10:09:03.419 INFO:tasks.workunit.client.1.smithi174.stdout:7/555: dwrite d0/d4/d10/d60/da2/fb9 [0,4194304] 0 2024-03-21T10:09:03.419 INFO:tasks.workunit.client.1.smithi174.stdout:4/757: stat d4/d21/d24/d28/de9/d3a/c4f 0 2024-03-21T10:09:03.421 INFO:tasks.workunit.client.1.smithi174.stdout:6/501: chown d0/d3/d2a/f8d 59142455 1 2024-03-21T10:09:03.422 INFO:tasks.workunit.client.1.smithi174.stdout:0/633: rename d9/fde to d9/d35/d4a/d67/d89/d99/db2/dbd/fe1 0 2024-03-21T10:09:03.423 INFO:tasks.workunit.client.0.smithi153.stdout:9/869: rename d1/d4e/c101 to d1/d4/d5f/df3/c11d 0 2024-03-21T10:09:03.425 INFO:tasks.workunit.client.1.smithi174.stdout:2/628: creat d5/d25/d27/d2e/d4f/d65/d77/dcf/fd5 x:0 0 0 2024-03-21T10:09:03.425 INFO:tasks.workunit.client.0.smithi153.stdout:4/848: fsync d11/d2d/f38 0 2024-03-21T10:09:03.426 INFO:tasks.workunit.client.1.smithi174.stdout:1/738: dread d4/d2d/fe1 [0,4194304] 0 2024-03-21T10:09:03.429 INFO:tasks.workunit.client.0.smithi153.stdout:6/988: unlink d1f/d2b/d10a/d40/d4f/d66/cf6 0 2024-03-21T10:09:03.430 INFO:tasks.workunit.client.0.smithi153.stdout:9/870: rmdir d1/d43/d44 39 2024-03-21T10:09:03.432 INFO:tasks.workunit.client.1.smithi174.stdout:7/556: creat d0/d4/d38/fbf x:0 0 0 2024-03-21T10:09:03.432 INFO:tasks.workunit.client.1.smithi174.stdout:8/605: write d9/d2a/d39/d76/d19/d27/d60/fb5 [3198163,78526] 0 2024-03-21T10:09:03.433 INFO:tasks.workunit.client.0.smithi153.stdout:3/885: write dd/f27 [3380696,18670] 0 2024-03-21T10:09:03.433 INFO:tasks.workunit.client.0.smithi153.stdout:6/989: chown d1f/d2b/c54 387783 1 2024-03-21T10:09:03.434 INFO:tasks.workunit.client.1.smithi174.stdout:7/557: chown d0/d4/l20 0 1 2024-03-21T10:09:03.436 INFO:tasks.workunit.client.0.smithi153.stdout:4/849: mkdir d11/d2d/da3/dac/dce/d10a 0 2024-03-21T10:09:03.437 INFO:tasks.workunit.client.1.smithi174.stdout:4/758: mkdir d4/d21/d24/d28/de9/d6a/d8f/d9c/dfd 0 2024-03-21T10:09:03.437 INFO:tasks.workunit.client.1.smithi174.stdout:5/705: dwrite d1/d2f/fae [0,4194304] 0 2024-03-21T10:09:03.439 INFO:tasks.workunit.client.0.smithi153.stdout:6/990: rmdir d1f/d2b/d10a/d40/d51/dbd 39 2024-03-21T10:09:03.443 INFO:tasks.workunit.client.1.smithi174.stdout:1/739: mknod d4/d7/d68/d8e/c10a 0 2024-03-21T10:09:03.443 INFO:tasks.workunit.client.0.smithi153.stdout:9/871: write d1/d3b/df8/dfa/f115 [322839,15777] 0 2024-03-21T10:09:03.443 INFO:tasks.workunit.client.0.smithi153.stdout:4/850: creat d11/d18/d7e/f10b x:0 0 0 2024-03-21T10:09:03.445 INFO:tasks.workunit.client.1.smithi174.stdout:2/629: dwrite d5/d25/d27/d2e/d90/fc3 [0,4194304] 0 2024-03-21T10:09:03.445 INFO:tasks.workunit.client.0.smithi153.stdout:0/748: dwrite d4/d16/d19/d58/d9a/fb4 [0,4194304] 0 2024-03-21T10:09:03.449 INFO:tasks.workunit.client.0.smithi153.stdout:3/886: mknod dd/d1e/d3d/d11b/c135 0 2024-03-21T10:09:03.451 INFO:tasks.workunit.client.0.smithi153.stdout:5/805: dwrite d4/d8/f26 [0,4194304] 0 2024-03-21T10:09:03.453 INFO:tasks.workunit.client.1.smithi174.stdout:7/558: mkdir d0/d4/d10/d44/d81/d9a/dc0 0 2024-03-21T10:09:03.454 INFO:tasks.workunit.client.1.smithi174.stdout:6/502: mknod d0/d3/d2a/d56/d5e/d91/c9a 0 2024-03-21T10:09:03.458 INFO:tasks.workunit.client.1.smithi174.stdout:9/679: rename d2/d24/d2e/d7e to d2/d11/d1d/d2a/df2 0 2024-03-21T10:09:03.460 INFO:tasks.workunit.client.1.smithi174.stdout:1/740: creat d4/d7/d15/d53/d78/f10b x:0 0 0 2024-03-21T10:09:03.460 INFO:tasks.workunit.client.1.smithi174.stdout:4/759: mknod d4/d21/d24/d28/d38/de5/cfe 0 2024-03-21T10:09:03.460 INFO:tasks.workunit.client.0.smithi153.stdout:9/872: read d1/d43/d44/d46/d9e/d24/d41/f6a [170402,63606] 0 2024-03-21T10:09:03.463 INFO:tasks.workunit.client.0.smithi153.stdout:0/749: rename d4/d16/d19/l2f to d4/d16/d19/d30/d6c/d98/d5e/dbe/lea 0 2024-03-21T10:09:03.464 INFO:tasks.workunit.client.1.smithi174.stdout:5/706: symlink d1/d49/d4f/de4/lff 0 2024-03-21T10:09:03.464 INFO:tasks.workunit.client.1.smithi174.stdout:4/760: write d4/f20 [102066,19918] 0 2024-03-21T10:09:03.464 INFO:tasks.workunit.client.1.smithi174.stdout:6/503: unlink d0/d3/d5/df/d16/f67 0 2024-03-21T10:09:03.464 INFO:tasks.workunit.client.0.smithi153.stdout:7/817: dwrite d4/d14/d2f/d78/f3a [4194304,4194304] 0 2024-03-21T10:09:03.466 INFO:tasks.workunit.client.1.smithi174.stdout:0/634: getdents d9/d1c 0 2024-03-21T10:09:03.470 INFO:tasks.workunit.client.1.smithi174.stdout:8/606: rename d9/d2a/d39/d76/d19/d27/l90 to d9/d2a/d39/d76/lc8 0 2024-03-21T10:09:03.471 INFO:tasks.workunit.client.1.smithi174.stdout:1/741: creat d4/d7/d32/d7d/f10c x:0 0 0 2024-03-21T10:09:03.471 INFO:tasks.workunit.client.1.smithi174.stdout:3/646: sync 2024-03-21T10:09:03.474 INFO:tasks.workunit.client.1.smithi174.stdout:9/680: creat d2/d11/d1d/d8b/ff3 x:0 0 0 2024-03-21T10:09:03.474 INFO:tasks.workunit.client.0.smithi153.stdout:6/991: dread d1f/d93/dd8/f32 [0,4194304] 0 2024-03-21T10:09:03.477 INFO:tasks.workunit.client.0.smithi153.stdout:5/806: creat d4/d8/d5a/d5d/dd9/d44/dc7/de2/f10f x:0 0 0 2024-03-21T10:09:03.479 INFO:tasks.workunit.client.1.smithi174.stdout:1/742: dread - d4/d7/d70/fe8 zero size 2024-03-21T10:09:03.480 INFO:tasks.workunit.client.0.smithi153.stdout:2/874: dwrite dd/faf [0,4194304] 0 2024-03-21T10:09:03.483 INFO:tasks.workunit.client.1.smithi174.stdout:0/635: readlink d9/d21/d31/d3c/l4d 0 2024-03-21T10:09:03.484 INFO:tasks.workunit.client.0.smithi153.stdout:7/818: rename d4/d2a/fa3 to d4/d2a/d3b/d81/df6/f112 0 2024-03-21T10:09:03.486 INFO:tasks.workunit.client.1.smithi174.stdout:6/504: unlink d0/d3/d5/d3b/f51 0 2024-03-21T10:09:03.487 INFO:tasks.workunit.client.0.smithi153.stdout:8/790: dwrite d5/d9/d43/d7b/daa/fb5 [0,4194304] 0 2024-03-21T10:09:03.487 INFO:tasks.workunit.client.1.smithi174.stdout:4/761: truncate d4/d21/f27 3978154 0 2024-03-21T10:09:03.488 INFO:tasks.workunit.client.0.smithi153.stdout:4/851: getdents d11/d71/d89/dca 0 2024-03-21T10:09:03.488 INFO:tasks.workunit.client.0.smithi153.stdout:3/887: truncate dd/d1e/d2f/d35/d68/f96 486470 0 2024-03-21T10:09:03.493 INFO:tasks.workunit.client.0.smithi153.stdout:2/875: truncate fc 807314 0 2024-03-21T10:09:03.499 INFO:tasks.workunit.client.1.smithi174.stdout:2/630: rename d5/d25/d27/d2e to d5/d25/d27/d2d/d45/dae/dd6 0 2024-03-21T10:09:03.499 INFO:tasks.workunit.client.0.smithi153.stdout:8/791: read - d5/d9/f4b zero size 2024-03-21T10:09:03.500 INFO:tasks.workunit.client.1.smithi174.stdout:9/681: mknod d2/d11/d1d/d4b/cf4 0 2024-03-21T10:09:03.501 INFO:tasks.workunit.client.1.smithi174.stdout:1/743: mknod d4/d7/d15/d53/c10d 0 2024-03-21T10:09:03.502 INFO:tasks.workunit.client.1.smithi174.stdout:4/762: dread - d4/d21/d24/d28/de9/d3a/d67/d76/fdb zero size 2024-03-21T10:09:03.503 INFO:tasks.workunit.client.0.smithi153.stdout:7/819: dread d4/d2a/d40/d50/f9b [0,4194304] 0 2024-03-21T10:09:03.503 INFO:tasks.workunit.client.1.smithi174.stdout:7/559: dwrite d0/d4/d10/d79/f83 [0,4194304] 0 2024-03-21T10:09:03.505 INFO:tasks.workunit.client.1.smithi174.stdout:1/744: read d4/d2d/fae [191684,84143] 0 2024-03-21T10:09:03.506 INFO:tasks.workunit.client.0.smithi153.stdout:5/807: dwrite d4/d8/d5a/d5d/dd9/f3f [0,4194304] 0 2024-03-21T10:09:03.506 INFO:tasks.workunit.client.1.smithi174.stdout:8/607: mkdir d9/d53/dc9 0 2024-03-21T10:09:03.507 INFO:tasks.workunit.client.0.smithi153.stdout:4/852: creat d11/d2d/d57/f10c x:0 0 0 2024-03-21T10:09:03.507 INFO:tasks.workunit.client.0.smithi153.stdout:6/992: truncate d1f/d2b/d10a/d40/d51/f71 4136503 0 2024-03-21T10:09:03.508 INFO:tasks.workunit.client.0.smithi153.stdout:3/888: rename dd/d1e/d3d/d94/fdb to dd/d41/d80/dbb/d112/f136 0 2024-03-21T10:09:03.509 INFO:tasks.workunit.client.1.smithi174.stdout:4/763: readlink d4/d21/d24/d28/de9/d3a/l45 0 2024-03-21T10:09:03.510 INFO:tasks.workunit.client.1.smithi174.stdout:3/647: dread d0/d12/d14/d1a/d27/d3c/d73/db7/d7d/f8b [0,4194304] 0 2024-03-21T10:09:03.511 INFO:tasks.workunit.client.0.smithi153.stdout:7/820: stat d4/d2a/d3b/f103 0 2024-03-21T10:09:03.514 INFO:tasks.workunit.client.0.smithi153.stdout:1/683: sync 2024-03-21T10:09:03.514 INFO:tasks.workunit.client.0.smithi153.stdout:9/873: link d1/d4/f75 d1/d43/d44/d46/d9e/f11e 0 2024-03-21T10:09:03.516 INFO:tasks.workunit.client.0.smithi153.stdout:3/889: stat dd/d1e/d3d/d53/dae/d126/f12b 0 2024-03-21T10:09:03.516 INFO:tasks.workunit.client.1.smithi174.stdout:2/631: creat d5/d25/d27/d2d/d45/dae/dd6/d7b/d81/fd7 x:0 0 0 2024-03-21T10:09:03.518 INFO:tasks.workunit.client.0.smithi153.stdout:3/890: read - dd/d41/d80/f103 zero size 2024-03-21T10:09:03.522 INFO:tasks.workunit.client.0.smithi153.stdout:3/891: write dd/d1e/d2f/f36 [3710937,80850] 0 2024-03-21T10:09:03.526 INFO:tasks.workunit.client.0.smithi153.stdout:4/853: unlink d11/d18/d7e/dc5/lcc 0 2024-03-21T10:09:03.528 INFO:tasks.workunit.client.0.smithi153.stdout:7/821: creat d4/d19/d4d/dd5/f113 x:0 0 0 2024-03-21T10:09:03.533 INFO:tasks.workunit.client.0.smithi153.stdout:7/822: chown d4/d2a/d3b/d60 36351532 1 2024-03-21T10:09:03.533 INFO:tasks.workunit.client.0.smithi153.stdout:1/684: unlink d6/d32/cd1 0 2024-03-21T10:09:03.539 INFO:tasks.workunit.client.0.smithi153.stdout:3/892: dread - dd/d1e/ff0 zero size 2024-03-21T10:09:03.539 INFO:tasks.workunit.client.0.smithi153.stdout:6/993: dwrite d1f/d2b/d10a/f3c [0,4194304] 0 2024-03-21T10:09:03.544 INFO:tasks.workunit.client.0.smithi153.stdout:0/750: truncate d4/d16/d27/f79 2173513 0 2024-03-21T10:09:03.545 INFO:tasks.workunit.client.1.smithi174.stdout:0/636: dwrite d9/d35/d4a/d67/d89/d99/db2/dbd/fe1 [0,4194304] 0 2024-03-21T10:09:03.547 INFO:tasks.workunit.client.1.smithi174.stdout:6/505: dwrite d0/d3/d20/f7f [0,4194304] 0 2024-03-21T10:09:03.548 INFO:tasks.workunit.client.1.smithi174.stdout:3/648: dwrite d0/d12/d14/d1a/d5c/f8e [0,4194304] 0 2024-03-21T10:09:03.552 INFO:tasks.workunit.client.0.smithi153.stdout:7/823: dread d4/d2a/d3b/d60/ff1 [0,4194304] 0 2024-03-21T10:09:03.557 INFO:tasks.workunit.client.0.smithi153.stdout:3/893: fsync dd/d1e/d2f/d35/d68/d75/ddd/fbd 0 2024-03-21T10:09:03.557 INFO:tasks.workunit.client.0.smithi153.stdout:7/824: read d4/d14/fa4 [816115,103673] 0 2024-03-21T10:09:03.558 INFO:tasks.workunit.client.1.smithi174.stdout:0/637: symlink d9/le2 0 2024-03-21T10:09:03.558 INFO:tasks.workunit.client.0.smithi153.stdout:8/792: getdents d5/d9/d43/d7b/daa/d102 0 2024-03-21T10:09:03.570 INFO:tasks.workunit.client.0.smithi153.stdout:4/854: rename d11/d18/d50/d65/l8e to d11/d2d/d6b/d9c/l10d 0 2024-03-21T10:09:03.575 INFO:tasks.workunit.client.1.smithi174.stdout:7/560: dwrite d0/d4/d10/d44/d81/d9a/fa3 [0,4194304] 0 2024-03-21T10:09:03.577 INFO:tasks.workunit.client.0.smithi153.stdout:4/855: read - d11/d5e/f7d zero size 2024-03-21T10:09:03.580 INFO:tasks.workunit.client.1.smithi174.stdout:5/707: dwrite d1/d16/d1c/f34 [4194304,4194304] 0 2024-03-21T10:09:03.582 INFO:tasks.workunit.client.0.smithi153.stdout:0/751: rename d4/f37 to d4/db/feb 0 2024-03-21T10:09:03.585 INFO:tasks.workunit.client.0.smithi153.stdout:6/994: link d1f/d2b/d10a/d40/d5f/f6d d1f/d2b/d10a/d40/d51/f161 0 2024-03-21T10:09:03.586 INFO:tasks.workunit.client.0.smithi153.stdout:4/856: symlink d11/d71/l10e 0 2024-03-21T10:09:03.588 INFO:tasks.workunit.client.1.smithi174.stdout:1/745: dwrite d4/d7/d33/d43/d77/d87/d92/ff4 [0,4194304] 0 2024-03-21T10:09:03.593 INFO:tasks.workunit.client.0.smithi153.stdout:4/857: readlink d11/d2d/d3d/lc4 0 2024-03-21T10:09:03.596 INFO:tasks.workunit.client.1.smithi174.stdout:1/746: read d4/d5/f72 [1460274,53065] 0 2024-03-21T10:09:03.597 INFO:tasks.workunit.client.1.smithi174.stdout:7/561: dread d0/f52 [0,4194304] 0 2024-03-21T10:09:03.598 INFO:tasks.workunit.client.1.smithi174.stdout:4/764: write d4/d21/d24/d28/de9/d3a/d67/d76/fc0 [3502663,98904] 0 2024-03-21T10:09:03.598 INFO:tasks.workunit.client.1.smithi174.stdout:8/608: dread d9/f30 [0,4194304] 0 2024-03-21T10:09:03.600 INFO:tasks.workunit.client.1.smithi174.stdout:9/682: dwrite d2/d11/f2c [0,4194304] 0 2024-03-21T10:09:03.602 INFO:tasks.workunit.client.0.smithi153.stdout:4/858: chown d11/d2d/d3d/lc4 255884 1 2024-03-21T10:09:03.603 INFO:tasks.workunit.client.1.smithi174.stdout:4/765: chown d4/d21/d24/d28/d38/f54 1 1 2024-03-21T10:09:03.610 INFO:tasks.workunit.client.1.smithi174.stdout:7/562: unlink d0/d53/lbb 0 2024-03-21T10:09:03.611 INFO:tasks.workunit.client.0.smithi153.stdout:9/874: write d1/d43/d44/d46/d4d/f74 [1737098,19882] 0 2024-03-21T10:09:03.613 INFO:tasks.workunit.client.1.smithi174.stdout:8/609: write d9/d2a/d39/d76/d19/d27/d79/fa6 [333575,93343] 0 2024-03-21T10:09:03.614 INFO:tasks.workunit.client.0.smithi153.stdout:2/876: dwrite dd/d15/d18/d3a/fdb [0,4194304] 0 2024-03-21T10:09:03.616 INFO:tasks.workunit.client.1.smithi174.stdout:7/563: stat d0/d4/d10/d1b/l84 0 2024-03-21T10:09:03.616 INFO:tasks.workunit.client.1.smithi174.stdout:1/747: mkdir d4/d5/d6c/d10e 0 2024-03-21T10:09:03.617 INFO:tasks.workunit.client.0.smithi153.stdout:5/808: truncate d4/d8/f11 833424 0 2024-03-21T10:09:03.621 INFO:tasks.workunit.client.1.smithi174.stdout:9/683: rename d2/d11/fdf to d2/d11/d1d/d4b/ff5 0 2024-03-21T10:09:03.622 INFO:tasks.workunit.client.1.smithi174.stdout:3/649: dwrite d0/d12/d14/d1a/d44/d65/f6b [0,4194304] 0 2024-03-21T10:09:03.623 INFO:tasks.workunit.client.0.smithi153.stdout:1/685: write d6/d3a/d3c/fa7 [1299417,128833] 0 2024-03-21T10:09:03.630 INFO:tasks.workunit.client.1.smithi174.stdout:8/610: creat d9/d2a/d4d/fca x:0 0 0 2024-03-21T10:09:03.633 INFO:tasks.workunit.client.0.smithi153.stdout:5/809: mkdir d4/d5/df/d5e/d110 0 2024-03-21T10:09:03.633 INFO:tasks.workunit.client.1.smithi174.stdout:4/766: truncate d4/d21/f49 1468476 0 2024-03-21T10:09:03.633 INFO:tasks.workunit.client.1.smithi174.stdout:6/506: dwrite d0/d3/f5a [0,4194304] 0 2024-03-21T10:09:03.637 INFO:tasks.workunit.client.0.smithi153.stdout:3/894: truncate dd/d1e/d6b/f6c 1210285 0 2024-03-21T10:09:03.637 INFO:tasks.workunit.client.1.smithi174.stdout:7/564: creat d0/d4/d10/d60/da2/fc1 x:0 0 0 2024-03-21T10:09:03.637 INFO:tasks.workunit.client.0.smithi153.stdout:3/895: fdatasync dd/f1f 0 2024-03-21T10:09:03.639 INFO:tasks.workunit.client.1.smithi174.stdout:9/684: mkdir d2/d24/d2e/df6 0 2024-03-21T10:09:03.641 INFO:tasks.workunit.client.1.smithi174.stdout:7/565: readlink d0/d4/d10/d60/lb1 0 2024-03-21T10:09:03.642 INFO:tasks.workunit.client.0.smithi153.stdout:1/686: creat d6/d10/d35/d5b/fde x:0 0 0 2024-03-21T10:09:03.643 INFO:tasks.workunit.client.1.smithi174.stdout:3/650: truncate d0/d12/d14/d1a/d27/f47 359296 0 2024-03-21T10:09:03.644 INFO:tasks.workunit.client.1.smithi174.stdout:4/767: write d4/d21/d24/d28/de9/d6a/d8f/f94 [4449900,117879] 0 2024-03-21T10:09:03.644 INFO:tasks.workunit.client.0.smithi153.stdout:5/810: symlink d4/d8/d5a/d5d/dd9/d44/de1/dfb/l111 0 2024-03-21T10:09:03.647 INFO:tasks.workunit.client.1.smithi174.stdout:6/507: rename d0/d3/d5/d6/d42/c6e to d0/d3/d20/d6d/c9b 0 2024-03-21T10:09:03.648 INFO:tasks.workunit.client.1.smithi174.stdout:1/748: dwrite d4/d7/f90 [0,4194304] 0 2024-03-21T10:09:03.650 INFO:tasks.workunit.client.0.smithi153.stdout:5/811: chown d4/d5/d16/f8a 22409466 1 2024-03-21T10:09:03.650 INFO:tasks.workunit.client.1.smithi174.stdout:7/566: read - d0/d4/d10/d60/da2/fc1 zero size 2024-03-21T10:09:03.652 INFO:tasks.workunit.client.1.smithi174.stdout:5/708: dwrite d1/d16/d2e/d8d/d99/faf [0,4194304] 0 2024-03-21T10:09:03.654 INFO:tasks.workunit.client.1.smithi174.stdout:8/611: symlink d9/d53/lcb 0 2024-03-21T10:09:03.657 INFO:tasks.workunit.client.1.smithi174.stdout:8/612: stat d9/d2a/d39/d58/fb8 0 2024-03-21T10:09:03.658 INFO:tasks.workunit.client.1.smithi174.stdout:6/508: dread - d0/d3/d2a/d8f/f97 zero size 2024-03-21T10:09:03.659 INFO:tasks.workunit.client.1.smithi174.stdout:7/567: creat d0/d4/d10/d1b/d86/fc2 x:0 0 0 2024-03-21T10:09:03.659 INFO:tasks.workunit.client.0.smithi153.stdout:7/825: dwrite d4/d11/f61 [0,4194304] 0 2024-03-21T10:09:03.659 INFO:tasks.workunit.client.0.smithi153.stdout:2/877: link dd/d15/d3c/fe6 dd/d15/d18/d3a/f133 0 2024-03-21T10:09:03.661 INFO:tasks.workunit.client.0.smithi153.stdout:8/793: dwrite d5/d9/d43/dd/d20/d26/d24/f32 [0,4194304] 0 2024-03-21T10:09:03.663 INFO:tasks.workunit.client.1.smithi174.stdout:5/709: mknod d1/d49/c100 0 2024-03-21T10:09:03.663 INFO:tasks.workunit.client.1.smithi174.stdout:7/568: chown d0/d4/d10/d1b/f4e 106142481 1 2024-03-21T10:09:03.664 INFO:tasks.workunit.client.0.smithi153.stdout:9/875: getdents d1/d43/d44/d46/d9e/d61 0 2024-03-21T10:09:03.664 INFO:tasks.workunit.client.0.smithi153.stdout:2/878: stat dd/d15/d18/d3a/d3d/d125 0 2024-03-21T10:09:03.665 INFO:tasks.workunit.client.1.smithi174.stdout:2/632: sync 2024-03-21T10:09:03.665 INFO:tasks.workunit.client.1.smithi174.stdout:0/638: sync 2024-03-21T10:09:03.667 INFO:tasks.workunit.client.1.smithi174.stdout:8/613: mknod d9/d2a/d39/d76/d19/d27/d60/ccc 0 2024-03-21T10:09:03.669 INFO:tasks.workunit.client.1.smithi174.stdout:5/710: mkdir d1/d101 0 2024-03-21T10:09:03.670 INFO:tasks.workunit.client.1.smithi174.stdout:8/614: fsync d9/d53/fbc 0 2024-03-21T10:09:03.672 INFO:tasks.workunit.client.1.smithi174.stdout:2/633: mkdir d5/d25/d27/d2d/d45/d5b/d2f/d4b/dd8 0 2024-03-21T10:09:03.672 INFO:tasks.workunit.client.1.smithi174.stdout:3/651: getdents d0/d12/d14/d1a/d44/d65 0 2024-03-21T10:09:03.673 INFO:tasks.workunit.client.1.smithi174.stdout:6/509: read d0/d3/d5/df/d16/f1a [230406,88404] 0 2024-03-21T10:09:03.676 INFO:tasks.workunit.client.1.smithi174.stdout:7/569: rename d0/d4/d10/d60/f8a to d0/d4/d10/d1b/d86/fc3 0 2024-03-21T10:09:03.678 INFO:tasks.workunit.client.1.smithi174.stdout:8/615: unlink d9/d53/d5b/f77 0 2024-03-21T10:09:03.680 INFO:tasks.workunit.client.0.smithi153.stdout:7/826: symlink d4/d11/d56/d10c/l114 0 2024-03-21T10:09:03.681 INFO:tasks.workunit.client.0.smithi153.stdout:3/896: fsync dd/d1e/d2f/d35/d68/ff6 0 2024-03-21T10:09:03.681 INFO:tasks.workunit.client.0.smithi153.stdout:4/859: write d11/d2d/f92 [831550,50687] 0 2024-03-21T10:09:03.682 INFO:tasks.workunit.client.0.smithi153.stdout:7/827: write d4/d2a/d40/d50/f7b [2366726,64462] 0 2024-03-21T10:09:03.683 INFO:tasks.workunit.client.0.smithi153.stdout:4/860: truncate d11/db1/fd5 827746 0 2024-03-21T10:09:03.685 INFO:tasks.workunit.client.1.smithi174.stdout:1/749: truncate d4/d5/f72 1970304 0 2024-03-21T10:09:03.686 INFO:tasks.workunit.client.1.smithi174.stdout:2/634: creat d5/d25/d27/d2d/d45/dae/fd9 x:0 0 0 2024-03-21T10:09:03.686 INFO:tasks.workunit.client.1.smithi174.stdout:1/750: chown d4/d7/d32/d7d 2237424 1 2024-03-21T10:09:03.689 INFO:tasks.workunit.client.1.smithi174.stdout:1/751: chown d4/d7/d32/d86/f107 0 1 2024-03-21T10:09:03.690 INFO:tasks.workunit.client.1.smithi174.stdout:6/510: rename d0/d3/f5a to d0/d3/d5/d21/f9c 0 2024-03-21T10:09:03.690 INFO:tasks.workunit.client.0.smithi153.stdout:7/828: truncate d4/d14/d2f/ffe 896106 0 2024-03-21T10:09:03.694 INFO:tasks.workunit.client.0.smithi153.stdout:2/879: rmdir dd/d15/d18/d2c/d4b/d56 39 2024-03-21T10:09:03.695 INFO:tasks.workunit.client.0.smithi153.stdout:1/687: creat d6/d3a/fdf x:0 0 0 2024-03-21T10:09:03.695 INFO:tasks.workunit.client.0.smithi153.stdout:0/752: dwrite d4/d16/d19/d30/d6c/d98/d5e/f70 [0,4194304] 0 2024-03-21T10:09:03.697 INFO:tasks.workunit.client.0.smithi153.stdout:9/876: mknod d1/d43/d44/d46/d9e/d24/d41/dbe/dcc/c11f 0 2024-03-21T10:09:03.699 INFO:tasks.workunit.client.0.smithi153.stdout:6/995: dwrite d1f/d2b/d10a/d40/d51/f71 [0,4194304] 0 2024-03-21T10:09:03.701 INFO:tasks.workunit.client.1.smithi174.stdout:1/752: chown d4/d7/d70/cf0 0 1 2024-03-21T10:09:03.701 INFO:tasks.workunit.client.0.smithi153.stdout:7/829: mkdir d4/d11/d56/d115 0 2024-03-21T10:09:03.703 INFO:tasks.workunit.client.0.smithi153.stdout:1/688: fsync d6/f9b 0 2024-03-21T10:09:03.705 INFO:tasks.workunit.client.1.smithi174.stdout:9/685: write d2/d11/d1d/d2a/df2/f8c [323032,104702] 0 2024-03-21T10:09:03.705 INFO:tasks.workunit.client.0.smithi153.stdout:4/861: read - d11/d2d/da3/dac/fbf zero size 2024-03-21T10:09:03.706 INFO:tasks.workunit.client.1.smithi174.stdout:7/570: rename d0/d4/d10/d44/d81/f87 to d0/d4/d10/d1b/d86/fc4 0 2024-03-21T10:09:03.706 INFO:tasks.workunit.client.0.smithi153.stdout:0/753: write d4/d16/d19/fa2 [7781910,69196] 0 2024-03-21T10:09:03.708 INFO:tasks.workunit.client.1.smithi174.stdout:8/616: mknod d9/d53/ccd 0 2024-03-21T10:09:03.709 INFO:tasks.workunit.client.1.smithi174.stdout:4/768: truncate d4/f3e 2413946 0 2024-03-21T10:09:03.711 INFO:tasks.workunit.client.1.smithi174.stdout:7/571: truncate d0/d4/d38/fb7 562656 0 2024-03-21T10:09:03.712 INFO:tasks.workunit.client.1.smithi174.stdout:8/617: stat d9/d2a/d39/d76/lc8 0 2024-03-21T10:09:03.712 INFO:tasks.workunit.client.0.smithi153.stdout:4/862: stat d11/d2d/da3/dac/lf6 0 2024-03-21T10:09:03.713 INFO:tasks.workunit.client.1.smithi174.stdout:4/769: creat d4/d21/fff x:0 0 0 2024-03-21T10:09:03.716 INFO:tasks.workunit.client.1.smithi174.stdout:1/753: creat d4/d7/d68/f10f x:0 0 0 2024-03-21T10:09:03.717 INFO:tasks.workunit.client.0.smithi153.stdout:8/794: dwrite d5/d9/d43/dd/d2d/d45/fc0 [0,4194304] 0 2024-03-21T10:09:03.717 INFO:tasks.workunit.client.0.smithi153.stdout:0/754: mkdir d4/d16/d19/dad/dec 0 2024-03-21T10:09:03.720 INFO:tasks.workunit.client.1.smithi174.stdout:7/572: symlink d0/d4/d10/d44/lc5 0 2024-03-21T10:09:03.721 INFO:tasks.workunit.client.1.smithi174.stdout:6/511: dwrite d0/d3/d2a/f2f [0,4194304] 0 2024-03-21T10:09:03.722 INFO:tasks.workunit.client.0.smithi153.stdout:1/689: dread d6/d10/da0/fa1 [0,4194304] 0 2024-03-21T10:09:03.727 INFO:tasks.workunit.client.1.smithi174.stdout:1/754: chown d4/d5/d6c/lec 901 1 2024-03-21T10:09:03.727 INFO:tasks.workunit.client.1.smithi174.stdout:0/639: write d9/d21/f26 [1865829,92719] 0 2024-03-21T10:09:03.732 INFO:tasks.workunit.client.1.smithi174.stdout:9/686: dwrite d2/d11/f30 [0,4194304] 0 2024-03-21T10:09:03.732 INFO:tasks.workunit.client.0.smithi153.stdout:7/830: dwrite d4/d14/d2f/d78/f35 [0,4194304] 0 2024-03-21T10:09:03.732 INFO:tasks.workunit.client.0.smithi153.stdout:1/690: readlink d6/d13/l24 0 2024-03-21T10:09:03.732 INFO:tasks.workunit.client.0.smithi153.stdout:2/880: rename dd/f90 to dd/d15/d18/d2c/d4b/d56/d63/f134 0 2024-03-21T10:09:03.736 INFO:tasks.workunit.client.1.smithi174.stdout:6/512: read d0/d3/d5/d6/f6a [2340269,16675] 0 2024-03-21T10:09:03.737 INFO:tasks.workunit.client.0.smithi153.stdout:8/795: creat d5/d9/d43/dd/ddd/f103 x:0 0 0 2024-03-21T10:09:03.740 INFO:tasks.workunit.client.0.smithi153.stdout:0/755: rename d4/db/c5d to d4/d16/d9f/ced 0 2024-03-21T10:09:03.743 INFO:tasks.workunit.client.0.smithi153.stdout:2/881: fdatasync dd/d15/d18/d2c/d33/f43 0 2024-03-21T10:09:03.743 INFO:tasks.workunit.client.0.smithi153.stdout:9/877: read d1/d3b/daa/fea [2560737,18607] 0 2024-03-21T10:09:03.746 INFO:tasks.workunit.client.1.smithi174.stdout:7/573: dread d0/d4/d38/f67 [0,4194304] 0 2024-03-21T10:09:03.748 INFO:tasks.workunit.client.1.smithi174.stdout:4/770: truncate d4/d21/d24/d28/f4e 535268 0 2024-03-21T10:09:03.750 INFO:tasks.workunit.client.1.smithi174.stdout:7/574: fdatasync d0/d4/d10/d79/f9f 0 2024-03-21T10:09:03.750 INFO:tasks.workunit.client.0.smithi153.stdout:8/796: fsync d5/d9/d43/f16 0 2024-03-21T10:09:03.751 INFO:tasks.workunit.client.1.smithi174.stdout:4/771: fsync d4/d19/f58 0 2024-03-21T10:09:03.753 INFO:tasks.workunit.client.1.smithi174.stdout:4/772: stat d4/d19/f2d 0 2024-03-21T10:09:03.753 INFO:tasks.workunit.client.0.smithi153.stdout:8/797: fdatasync d5/d9/d43/d7b/fe4 0 2024-03-21T10:09:03.757 INFO:tasks.workunit.client.0.smithi153.stdout:2/882: creat dd/d15/d18/d3a/d3d/ddc/f135 x:0 0 0 2024-03-21T10:09:03.758 INFO:tasks.workunit.client.1.smithi174.stdout:2/635: write d5/d25/d27/d2d/d45/d5b/d2f/d66/fb9 [1520319,107102] 0 2024-03-21T10:09:03.766 INFO:tasks.workunit.client.1.smithi174.stdout:9/687: rmdir d2/d24/d2e/de6 39 2024-03-21T10:09:03.766 INFO:tasks.workunit.client.1.smithi174.stdout:6/513: readlink d0/d3/d5/df/l14 0 2024-03-21T10:09:03.768 INFO:tasks.workunit.client.0.smithi153.stdout:1/691: creat d6/d10/d35/da5/d4f/d9d/dc7/dcc/fe0 x:0 0 0 2024-03-21T10:09:03.768 INFO:tasks.workunit.client.1.smithi174.stdout:7/575: rename d0/d4/d10/da4/daf to d0/d4/d38/da9/dc6 0 2024-03-21T10:09:03.770 INFO:tasks.workunit.client.1.smithi174.stdout:2/636: creat d5/d25/d27/d2d/d45/d5b/d55/fda x:0 0 0 2024-03-21T10:09:03.774 INFO:tasks.workunit.client.1.smithi174.stdout:7/576: write d0/d4/d10/f19 [814124,396] 0 2024-03-21T10:09:03.776 INFO:tasks.workunit.client.0.smithi153.stdout:2/883: mknod dd/d15/d18/d2c/c136 0 2024-03-21T10:09:03.779 INFO:tasks.workunit.client.0.smithi153.stdout:0/756: mkdir d4/d16/d27/dc5/dcd/de8/dee 0 2024-03-21T10:09:03.783 INFO:tasks.workunit.client.0.smithi153.stdout:1/692: chown d6/d13/d75/c56 6172 1 2024-03-21T10:09:03.786 INFO:tasks.workunit.client.1.smithi174.stdout:8/618: truncate d9/d2a/f3e 3872916 0 2024-03-21T10:09:03.788 INFO:tasks.workunit.client.1.smithi174.stdout:5/711: getdents d1/d49/d93 0 2024-03-21T10:09:03.790 INFO:tasks.workunit.client.1.smithi174.stdout:4/773: creat d4/d21/d24/d28/de9/d3a/f100 x:0 0 0 2024-03-21T10:09:03.791 INFO:tasks.workunit.client.0.smithi153.stdout:1/693: write d6/d3a/d3c/fa7 [822911,14055] 0 2024-03-21T10:09:03.791 INFO:tasks.workunit.client.0.smithi153.stdout:7/831: write d4/d2a/dc6/dfb/d9a/dd2/db9/fbf [2887588,115327] 0 2024-03-21T10:09:03.793 INFO:tasks.workunit.client.1.smithi174.stdout:0/640: write d9/d21/d23/da5/f6f [935994,88188] 0 2024-03-21T10:09:03.794 INFO:tasks.workunit.client.1.smithi174.stdout:8/619: read d9/d53/f95 [727808,63949] 0 2024-03-21T10:09:03.795 INFO:tasks.workunit.client.0.smithi153.stdout:6/996: dread d1f/d2b/d10a/d40/d51/d98/fcd [0,4194304] 0 2024-03-21T10:09:03.802 INFO:tasks.workunit.client.0.smithi153.stdout:9/878: link d1/d43/d44/d46/d9e/d4b/f73 d1/f120 0 2024-03-21T10:09:03.803 INFO:tasks.workunit.client.1.smithi174.stdout:7/577: getdents d0/d4/d10/d44 0 2024-03-21T10:09:03.803 INFO:tasks.workunit.client.1.smithi174.stdout:8/620: mknod d9/d2a/d39/d76/d19/d27/d60/cce 0 2024-03-21T10:09:03.804 INFO:tasks.workunit.client.1.smithi174.stdout:9/688: getdents d2/d11/d1d/d5e 0 2024-03-21T10:09:03.804 INFO:tasks.workunit.client.0.smithi153.stdout:2/884: dread dd/d15/d3c/da7/fe5 [0,4194304] 0 2024-03-21T10:09:03.806 INFO:tasks.workunit.client.1.smithi174.stdout:5/712: link d1/d16/d2e/d66/d8b/f48 d1/d49/f102 0 2024-03-21T10:09:03.807 INFO:tasks.workunit.client.1.smithi174.stdout:9/689: chown d2/d11/d1d/dab/ccf 2362 1 2024-03-21T10:09:03.808 INFO:tasks.workunit.client.0.smithi153.stdout:7/832: rmdir d4/d19/d4d 39 2024-03-21T10:09:03.809 INFO:tasks.workunit.client.1.smithi174.stdout:4/774: rename d4/d21/d24/d28/de9/d3a/d77/ld3 to d4/d21/d24/d28/l101 0 2024-03-21T10:09:03.809 INFO:tasks.workunit.client.1.smithi174.stdout:5/713: dread - d1/d16/d2e/d8d/d99/ffa zero size 2024-03-21T10:09:03.810 INFO:tasks.workunit.client.0.smithi153.stdout:2/885: readlink dd/d15/d18/d2c/d4b/d56/d63/d8d/d9b/dc3/df5/l115 0 2024-03-21T10:09:03.811 INFO:tasks.workunit.client.0.smithi153.stdout:4/863: dread d11/d2d/d3d/dc7/f106 [0,4194304] 0 2024-03-21T10:09:03.811 INFO:tasks.workunit.client.0.smithi153.stdout:0/757: fdatasync d4/d16/d27/f90 0 2024-03-21T10:09:03.813 INFO:tasks.workunit.client.0.smithi153.stdout:0/758: stat d4 0 2024-03-21T10:09:03.813 INFO:tasks.workunit.client.1.smithi174.stdout:0/641: rename d9/d21/d31/d5c/d92/d98/db7/dd4 to d9/d35/d4a/d67/d89/d99/de3 0 2024-03-21T10:09:03.814 INFO:tasks.workunit.client.0.smithi153.stdout:6/997: rmdir d1f/d2b/d10a/d40/d51 39 2024-03-21T10:09:03.815 INFO:tasks.workunit.client.0.smithi153.stdout:4/864: chown d11/d2d/d6b/d9c/ldc 256829694 1 2024-03-21T10:09:03.818 INFO:tasks.workunit.client.1.smithi174.stdout:5/714: mkdir d1/d49/d4f/d51/d8f/d103 0 2024-03-21T10:09:03.819 INFO:tasks.workunit.client.0.smithi153.stdout:1/694: dread d6/d13/d75/f57 [0,4194304] 0 2024-03-21T10:09:03.820 INFO:tasks.workunit.client.0.smithi153.stdout:7/833: stat d4/d14/dd0/d83/c8a 0 2024-03-21T10:09:03.822 INFO:tasks.workunit.client.1.smithi174.stdout:1/755: dwrite d4/d7/d33/d43/d77/d87/d92/ff2 [0,4194304] 0 2024-03-21T10:09:03.822 INFO:tasks.workunit.client.0.smithi153.stdout:7/834: write d4/d2a/d3b/d81/f90 [1094546,52462] 0 2024-03-21T10:09:03.823 INFO:tasks.workunit.client.1.smithi174.stdout:6/514: write d0/d3/d5/d6/d42/f71 [1490769,38269] 0 2024-03-21T10:09:03.823 INFO:tasks.workunit.client.0.smithi153.stdout:8/798: truncate d5/d9/d43/dd/d2d/d45/d88/fbe 907934 0 2024-03-21T10:09:03.825 INFO:tasks.workunit.client.0.smithi153.stdout:7/835: fdatasync d4/d2a/f2d 0 2024-03-21T10:09:03.826 INFO:tasks.workunit.client.1.smithi174.stdout:6/515: chown d0/d3/d5/d21/l30 399160439 1 2024-03-21T10:09:03.829 INFO:tasks.workunit.client.1.smithi174.stdout:5/715: creat d1/d49/d4f/d51/d8f/f104 x:0 0 0 2024-03-21T10:09:03.832 INFO:tasks.workunit.client.1.smithi174.stdout:6/516: mknod d0/d3/d5/df/d16/c9d 0 2024-03-21T10:09:03.833 INFO:tasks.workunit.client.1.smithi174.stdout:1/756: truncate d4/d7/d68/d8e/db8/fd8 839010 0 2024-03-21T10:09:03.833 INFO:tasks.workunit.client.0.smithi153.stdout:0/759: unlink d4/d16/d19/d30/d6c/d98/d5e/db2/ld6 0 2024-03-21T10:09:03.834 INFO:tasks.workunit.client.0.smithi153.stdout:7/836: creat d4/d2a/dc6/dfb/f116 x:0 0 0 2024-03-21T10:09:03.834 INFO:tasks.workunit.client.1.smithi174.stdout:7/578: rename d0/lc to d0/d4/lc7 0 2024-03-21T10:09:03.838 INFO:tasks.workunit.client.0.smithi153.stdout:2/886: getdents dd/d15/d18/d2c/d33/d98/dc7/d107 0 2024-03-21T10:09:03.838 INFO:tasks.workunit.client.0.smithi153.stdout:4/865: getdents d11/d2d/d6b 0 2024-03-21T10:09:03.843 INFO:tasks.workunit.client.1.smithi174.stdout:8/621: write d9/d53/f6c [4192703,16661] 0 2024-03-21T10:09:03.843 INFO:tasks.workunit.client.0.smithi153.stdout:6/998: symlink d1f/d2b/d10a/d40/d150/l162 0 2024-03-21T10:09:03.843 INFO:tasks.workunit.client.0.smithi153.stdout:9/879: dread d1/d43/d44/d46/d9e/d4b/f73 [0,4194304] 0 2024-03-21T10:09:03.844 INFO:tasks.workunit.client.0.smithi153.stdout:7/837: creat d4/d14/d16/f117 x:0 0 0 2024-03-21T10:09:03.844 INFO:tasks.workunit.client.0.smithi153.stdout:8/799: fsync d5/d9/d43/dd/d2d/d45/d88/f46 0 2024-03-21T10:09:03.848 INFO:tasks.workunit.client.1.smithi174.stdout:1/757: dread d4/d7/d32/d86/fd2 [0,4194304] 0 2024-03-21T10:09:03.850 INFO:tasks.workunit.client.0.smithi153.stdout:4/866: fdatasync d11/d18/d54/f68 0 2024-03-21T10:09:03.852 INFO:tasks.workunit.client.0.smithi153.stdout:4/867: truncate d11/ff4 137771 0 2024-03-21T10:09:03.853 INFO:tasks.workunit.client.1.smithi174.stdout:7/579: rename d0/d4/d38/l50 to d0/d53/d5f/lc8 0 2024-03-21T10:09:03.858 INFO:tasks.workunit.client.1.smithi174.stdout:9/690: dwrite d2/d11/f36 [0,4194304] 0 2024-03-21T10:09:03.861 INFO:tasks.workunit.client.1.smithi174.stdout:7/580: chown d0/d4/d10/d44/d81/d9a/fa3 1504844 1 2024-03-21T10:09:03.861 INFO:tasks.workunit.client.0.smithi153.stdout:1/695: dwrite d6/d3a/f53 [4194304,4194304] 0 2024-03-21T10:09:03.864 INFO:tasks.workunit.client.0.smithi153.stdout:7/838: chown d4/d19/d4d/dd5/fe2 42700618 1 2024-03-21T10:09:03.865 INFO:tasks.workunit.client.0.smithi153.stdout:5/812: sync 2024-03-21T10:09:03.865 INFO:tasks.workunit.client.0.smithi153.stdout:3/897: sync 2024-03-21T10:09:03.865 INFO:tasks.workunit.client.1.smithi174.stdout:3/652: sync 2024-03-21T10:09:03.866 INFO:tasks.workunit.client.0.smithi153.stdout:0/760: getdents d4/d4c/d83/d22/d61 0 2024-03-21T10:09:03.866 INFO:tasks.workunit.client.0.smithi153.stdout:8/800: mkdir d5/d9/d43/dd/d2d/d45/d104 0 2024-03-21T10:09:03.866 INFO:tasks.workunit.client.1.smithi174.stdout:9/691: creat d2/d11/d1d/d8b/ff7 x:0 0 0 2024-03-21T10:09:03.868 INFO:tasks.workunit.client.0.smithi153.stdout:4/868: unlink d11/d2d/f38 0 2024-03-21T10:09:03.870 INFO:tasks.workunit.client.1.smithi174.stdout:8/622: dwrite f5 [4194304,4194304] 0 2024-03-21T10:09:03.873 INFO:tasks.workunit.client.1.smithi174.stdout:6/517: dwrite d0/d3/d5/df/d66/f84 [0,4194304] 0 2024-03-21T10:09:03.874 INFO:tasks.workunit.client.1.smithi174.stdout:7/581: rmdir d0/d4/d10/da4 0 2024-03-21T10:09:03.874 INFO:tasks.workunit.client.1.smithi174.stdout:3/653: read - d0/d12/d14/d4c/d7a/d9e/fbd zero size 2024-03-21T10:09:03.874 INFO:tasks.workunit.client.1.smithi174.stdout:0/642: dread d9/f19 [4194304,4194304] 0 2024-03-21T10:09:03.877 INFO:tasks.workunit.client.0.smithi153.stdout:3/898: mkdir dd/d1e/d6b/d137 0 2024-03-21T10:09:03.879 INFO:tasks.workunit.client.1.smithi174.stdout:9/692: chown d2/d11/d1d/d4b/d72/fee 16 1 2024-03-21T10:09:03.880 INFO:tasks.workunit.client.0.smithi153.stdout:4/869: unlink d11/d2d/d3d/lc4 0 2024-03-21T10:09:03.881 INFO:tasks.workunit.client.0.smithi153.stdout:6/999: getdents d1f/de0/de1/df2/d10f 0 2024-03-21T10:09:03.882 INFO:tasks.workunit.client.0.smithi153.stdout:1/696: mkdir d6/d13/d8e/dbb/de1 0 2024-03-21T10:09:03.886 INFO:tasks.workunit.client.1.smithi174.stdout:5/716: dread d1/d16/d2e/d66/f6c [0,4194304] 0 2024-03-21T10:09:03.887 INFO:tasks.workunit.client.0.smithi153.stdout:9/880: dwrite d1/d4/d5f/df3/f10c [0,4194304] 0 2024-03-21T10:09:03.888 INFO:tasks.workunit.client.1.smithi174.stdout:9/693: chown d2/d11/d1d/d4b/f6e 3 1 2024-03-21T10:09:03.891 INFO:tasks.workunit.client.1.smithi174.stdout:3/654: creat d0/d12/d14/d98/fd6 x:0 0 0 2024-03-21T10:09:03.896 INFO:tasks.workunit.client.1.smithi174.stdout:9/694: stat d2/d11/d1d/d5e/c9f 0 2024-03-21T10:09:03.896 INFO:tasks.workunit.client.1.smithi174.stdout:6/518: truncate d0/d3/d5/d6/f38 333478 0 2024-03-21T10:09:03.898 INFO:tasks.workunit.client.1.smithi174.stdout:1/758: dwrite d4/d7/d33/d43/d77/d87/fb4 [0,4194304] 0 2024-03-21T10:09:03.899 INFO:tasks.workunit.client.0.smithi153.stdout:8/801: mknod d5/d9/d43/dd/d20/d26/d47/d69/c105 0 2024-03-21T10:09:03.900 INFO:tasks.workunit.client.1.smithi174.stdout:7/582: rename d0/d4/d10/f21 to d0/d53/d5f/fc9 0 2024-03-21T10:09:03.903 INFO:tasks.workunit.client.1.smithi174.stdout:5/717: creat d1/d5c/d96/f105 x:0 0 0 2024-03-21T10:09:03.903 INFO:tasks.workunit.client.1.smithi174.stdout:9/695: write d2/d11/d7d/fde [1030498,90907] 0 2024-03-21T10:09:03.905 INFO:tasks.workunit.client.0.smithi153.stdout:3/899: mkdir dd/d1e/d2f/d35/daf/dcc/d58/d138 0 2024-03-21T10:09:03.906 INFO:tasks.workunit.client.0.smithi153.stdout:8/802: stat d5/d9/d43/dd/d20/d26/d24/d52/d55/f71 0 2024-03-21T10:09:03.910 INFO:tasks.workunit.client.0.smithi153.stdout:9/881: mkdir d1/d43/d44/dc3/d121 0 2024-03-21T10:09:03.910 INFO:tasks.workunit.client.1.smithi174.stdout:9/696: rmdir d2/d54 39 2024-03-21T10:09:03.911 INFO:tasks.workunit.client.0.smithi153.stdout:3/900: dread - dd/d1e/ff0 zero size 2024-03-21T10:09:03.912 INFO:tasks.workunit.client.1.smithi174.stdout:8/623: write d9/d2a/d39/d58/f7b [106025,86833] 0 2024-03-21T10:09:03.913 INFO:tasks.workunit.client.0.smithi153.stdout:9/882: chown d1/d43/d44/f25 3 1 2024-03-21T10:09:03.913 INFO:tasks.workunit.client.0.smithi153.stdout:2/887: dread dd/d15/d5e/fbb [0,4194304] 0 2024-03-21T10:09:03.913 INFO:tasks.workunit.client.1.smithi174.stdout:1/759: unlink d4/d5/d6c/fd1 0 2024-03-21T10:09:03.914 INFO:tasks.workunit.client.1.smithi174.stdout:9/697: stat d2/d11/d1d/d8b/fa2 0 2024-03-21T10:09:03.916 INFO:tasks.workunit.client.0.smithi153.stdout:4/870: mknod d11/d2d/d57/c10f 0 2024-03-21T10:09:03.916 INFO:tasks.workunit.client.1.smithi174.stdout:0/643: dwrite d9/d21/d23/d80/fae [0,4194304] 0 2024-03-21T10:09:03.916 INFO:tasks.workunit.client.1.smithi174.stdout:5/718: creat d1/d5c/f106 x:0 0 0 2024-03-21T10:09:03.917 INFO:tasks.workunit.client.1.smithi174.stdout:8/624: fdatasync d9/dc6/fac 0 2024-03-21T10:09:03.919 INFO:tasks.workunit.client.0.smithi153.stdout:1/697: dwrite d6/d3a/fb0 [0,4194304] 0 2024-03-21T10:09:03.920 INFO:tasks.workunit.client.1.smithi174.stdout:7/583: fdatasync d0/d4/d10/d1b/d86/fc4 0 2024-03-21T10:09:03.920 INFO:tasks.workunit.client.0.smithi153.stdout:7/839: getdents d4/d11/da1 0 2024-03-21T10:09:03.922 INFO:tasks.workunit.client.1.smithi174.stdout:8/625: readlink d9/d2a/d39/d76/l22 0 2024-03-21T10:09:03.922 INFO:tasks.workunit.client.1.smithi174.stdout:1/760: write d4/d7/f2c [3913373,61969] 0 2024-03-21T10:09:03.923 INFO:tasks.workunit.client.1.smithi174.stdout:5/719: chown d1/d5c/d73/de3/cfb 954937 1 2024-03-21T10:09:03.929 INFO:tasks.workunit.client.0.smithi153.stdout:8/803: chown d5/d9/d43/dd/cbf 295 1 2024-03-21T10:09:03.930 INFO:tasks.workunit.client.1.smithi174.stdout:9/698: creat d2/d11/d60/ff8 x:0 0 0 2024-03-21T10:09:03.931 INFO:tasks.workunit.client.1.smithi174.stdout:3/655: link d0/d12/d14/d1a/d27/d3c/f61 d0/d12/d14/d1a/d27/d3c/d73/fd7 0 2024-03-21T10:09:03.932 INFO:tasks.workunit.client.0.smithi153.stdout:0/761: getdents d4/d16/d19/d30/d6c/d98/d5e/dbe 0 2024-03-21T10:09:03.933 INFO:tasks.workunit.client.1.smithi174.stdout:9/699: readlink d2/d11/d60/db2/ld4 0 2024-03-21T10:09:03.935 INFO:tasks.workunit.client.0.smithi153.stdout:7/840: readlink d4/d19/d4d/dd5/laf 0 2024-03-21T10:09:03.937 INFO:tasks.workunit.client.1.smithi174.stdout:2/637: sync 2024-03-21T10:09:03.949 INFO:tasks.workunit.client.0.smithi153.stdout:5/813: dwrite d4/d5/df/d67/d86/fac [0,4194304] 0 2024-03-21T10:09:03.952 INFO:tasks.workunit.client.0.smithi153.stdout:9/883: truncate d1/d43/d44/d46/d9e/d24/d41/f5b 2052066 0 2024-03-21T10:09:03.952 INFO:tasks.workunit.client.1.smithi174.stdout:7/584: dwrite d0/d4/d10/d79/f83 [4194304,4194304] 0 2024-03-21T10:09:03.953 INFO:tasks.workunit.client.0.smithi153.stdout:5/814: write d4/d5/f52 [2541100,63739] 0 2024-03-21T10:09:03.955 INFO:tasks.workunit.client.1.smithi174.stdout:3/656: dwrite d0/d12/d14/d1a/d27/d3c/f72 [0,4194304] 0 2024-03-21T10:09:03.956 INFO:tasks.workunit.client.0.smithi153.stdout:9/884: read d1/d43/d44/f5d [3676299,4093] 0 2024-03-21T10:09:03.959 INFO:tasks.workunit.client.1.smithi174.stdout:3/657: chown d0/d12/d14/d1a/d44/d56/fa0 9 1 2024-03-21T10:09:03.960 INFO:tasks.workunit.client.0.smithi153.stdout:9/885: write d1/d4/d5f/fc8 [2871449,38846] 0 2024-03-21T10:09:03.961 INFO:tasks.workunit.client.0.smithi153.stdout:0/762: mknod d4/d16/d19/d58/d9a/cef 0 2024-03-21T10:09:03.966 INFO:tasks.workunit.client.1.smithi174.stdout:1/761: dread d4/d7/d15/d4f/f69 [0,4194304] 0 2024-03-21T10:09:03.966 INFO:tasks.workunit.client.1.smithi174.stdout:6/519: rmdir d0/d3/d5/df/d66/d8e 0 2024-03-21T10:09:03.967 INFO:tasks.workunit.client.0.smithi153.stdout:7/841: mkdir d4/d14/d2f/d118 0 2024-03-21T10:09:03.973 INFO:tasks.workunit.client.1.smithi174.stdout:4/775: sync 2024-03-21T10:09:03.976 INFO:tasks.workunit.client.0.smithi153.stdout:5/815: dread - d4/d5/df/d67/d86/fda zero size 2024-03-21T10:09:03.977 INFO:tasks.workunit.client.1.smithi174.stdout:6/520: dread - d0/d3/d20/d3d/f76 zero size 2024-03-21T10:09:03.979 INFO:tasks.workunit.client.0.smithi153.stdout:3/901: write dd/d1e/fb0 [959600,57796] 0 2024-03-21T10:09:03.980 INFO:tasks.workunit.client.1.smithi174.stdout:2/638: symlink d5/d25/d27/d2d/d45/d5b/d2f/d4b/da7/ldb 0 2024-03-21T10:09:03.980 INFO:tasks.workunit.client.1.smithi174.stdout:7/585: dread - d0/f98 zero size 2024-03-21T10:09:03.981 INFO:tasks.workunit.client.0.smithi153.stdout:4/871: write d11/d18/d50/d65/ff8 [92471,36105] 0 2024-03-21T10:09:03.982 INFO:tasks.workunit.client.1.smithi174.stdout:9/700: creat d2/d24/d2e/ff9 x:0 0 0 2024-03-21T10:09:03.984 INFO:tasks.workunit.client.1.smithi174.stdout:5/720: getdents d1/d49/d4f/d51 0 2024-03-21T10:09:03.986 INFO:tasks.workunit.client.1.smithi174.stdout:4/776: fsync d4/f3e 0 2024-03-21T10:09:03.989 INFO:tasks.workunit.client.0.smithi153.stdout:3/902: chown dd/d1e/d2f/d35 2 1 2024-03-21T10:09:03.989 INFO:tasks.workunit.client.0.smithi153.stdout:5/816: symlink d4/d5/d9e/l112 0 2024-03-21T10:09:03.989 INFO:tasks.workunit.client.0.smithi153.stdout:8/804: dread d5/d9/d43/dd/d20/d26/d8b/f62 [0,4194304] 0 2024-03-21T10:09:03.990 INFO:tasks.workunit.client.0.smithi153.stdout:7/842: read d4/d2a/f2d [4044555,12262] 0 2024-03-21T10:09:03.991 INFO:tasks.workunit.client.1.smithi174.stdout:9/701: dread - d2/d11/f90 zero size 2024-03-21T10:09:03.992 INFO:tasks.workunit.client.0.smithi153.stdout:9/886: symlink d1/d3b/daa/l122 0 2024-03-21T10:09:03.997 INFO:tasks.workunit.client.1.smithi174.stdout:3/658: dwrite d0/d12/d14/d1a/d5c/f8e [0,4194304] 0 2024-03-21T10:09:03.998 INFO:tasks.workunit.client.1.smithi174.stdout:2/639: creat d5/d25/d27/d2d/d45/dae/dd6/d90/fdc x:0 0 0 2024-03-21T10:09:03.998 INFO:tasks.workunit.client.0.smithi153.stdout:5/817: creat d4/d8/d5c/d90/f113 x:0 0 0 2024-03-21T10:09:03.999 INFO:tasks.workunit.client.0.smithi153.stdout:1/698: dwrite d6/d10/d35/da5/f7d [0,4194304] 0 2024-03-21T10:09:04.003 INFO:tasks.workunit.client.0.smithi153.stdout:2/888: dwrite dd/d15/d18/d2c/d4b/d56/d63/d8d/d9b/dc3/df5/dae/d50/fcc [0,4194304] 0 2024-03-21T10:09:04.004 INFO:tasks.workunit.client.0.smithi153.stdout:4/872: symlink d11/l110 0 2024-03-21T10:09:04.005 INFO:tasks.workunit.client.1.smithi174.stdout:4/777: symlink d4/d21/d24/d28/d38/l102 0 2024-03-21T10:09:04.008 INFO:tasks.workunit.client.1.smithi174.stdout:0/644: dwrite d9/d21/d31/d3c/d72/fb1 [0,4194304] 0 2024-03-21T10:09:04.009 INFO:tasks.workunit.client.0.smithi153.stdout:1/699: write d6/d3a/f53 [8146139,31833] 0 2024-03-21T10:09:04.010 INFO:tasks.workunit.client.0.smithi153.stdout:0/763: dread d4/d4c/d83/f35 [0,4194304] 0 2024-03-21T10:09:04.012 INFO:tasks.workunit.client.1.smithi174.stdout:2/640: dread - d5/d25/d27/d2d/d45/d5b/d2f/d4b/fcb zero size 2024-03-21T10:09:04.012 INFO:tasks.workunit.client.1.smithi174.stdout:8/626: dread d9/dc6/fac [0,4194304] 0 2024-03-21T10:09:04.013 INFO:tasks.workunit.client.0.smithi153.stdout:5/818: chown d4/d5/df/d67/d86/d96/deb/d56/da5/fc4 3146432 1 2024-03-21T10:09:04.013 INFO:tasks.workunit.client.0.smithi153.stdout:1/700: fsync d6/d32/f37 0 2024-03-21T10:09:04.013 INFO:tasks.workunit.client.0.smithi153.stdout:9/887: mknod d1/d43/d44/d46/d9e/d2e/c123 0 2024-03-21T10:09:04.014 INFO:tasks.workunit.client.1.smithi174.stdout:7/586: dread d0/d4/f43 [0,4194304] 0 2024-03-21T10:09:04.016 INFO:tasks.workunit.client.1.smithi174.stdout:2/641: write d5/d25/d27/d2d/d45/d5b/d63/f48 [4661405,92865] 0 2024-03-21T10:09:04.017 INFO:tasks.workunit.client.0.smithi153.stdout:7/843: symlink d4/d2a/d3b/d81/l119 0 2024-03-21T10:09:04.019 INFO:tasks.workunit.client.0.smithi153.stdout:3/903: rmdir dd/d1e/d3d/d94/de2/d11e 0 2024-03-21T10:09:04.021 INFO:tasks.workunit.client.1.smithi174.stdout:3/659: creat d0/d12/d46/fd8 x:0 0 0 2024-03-21T10:09:04.024 INFO:tasks.workunit.client.1.smithi174.stdout:5/721: rename d1/d5c/d96/ld3 to d1/d49/d4f/d51/d8f/d103/l107 0 2024-03-21T10:09:04.024 INFO:tasks.workunit.client.0.smithi153.stdout:5/819: fdatasync d4/d5/df/d67/da0/db3/fd2 0 2024-03-21T10:09:04.026 INFO:tasks.workunit.client.1.smithi174.stdout:6/521: getdents d0/d3/d5/df 0 2024-03-21T10:09:04.028 INFO:tasks.workunit.client.1.smithi174.stdout:5/722: write d1/d49/fdd [1016114,75038] 0 2024-03-21T10:09:04.030 INFO:tasks.workunit.client.0.smithi153.stdout:1/701: symlink d6/d10/d35/d5b/le2 0 2024-03-21T10:09:04.032 INFO:tasks.workunit.client.0.smithi153.stdout:5/820: write d4/d5/df/f102 [663650,33950] 0 2024-03-21T10:09:04.035 INFO:tasks.workunit.client.0.smithi153.stdout:8/805: dwrite d5/d9/d43/d7b/daa/fb5 [0,4194304] 0 2024-03-21T10:09:04.035 INFO:tasks.workunit.client.0.smithi153.stdout:1/702: write d6/f36 [4282504,41918] 0 2024-03-21T10:09:04.036 INFO:tasks.workunit.client.0.smithi153.stdout:3/904: write dd/d1e/d3d/ff7 [917103,4589] 0 2024-03-21T10:09:04.038 INFO:tasks.workunit.client.1.smithi174.stdout:7/587: creat d0/d4/d38/da9/dc6/fca x:0 0 0 2024-03-21T10:09:04.038 INFO:tasks.workunit.client.1.smithi174.stdout:2/642: dread d5/d25/d27/d2d/d45/d5b/d2f/f9d [0,4194304] 0 2024-03-21T10:09:04.039 INFO:tasks.workunit.client.1.smithi174.stdout:8/627: symlink d9/d2a/d39/d76/d19/d27/d60/lcf 0 2024-03-21T10:09:04.042 INFO:tasks.workunit.client.0.smithi153.stdout:4/873: symlink d11/d2d/da3/d103/l111 0 2024-03-21T10:09:04.042 INFO:tasks.workunit.client.1.smithi174.stdout:7/588: write d0/d4/d38/fbf [254265,86286] 0 2024-03-21T10:09:04.043 INFO:tasks.workunit.client.1.smithi174.stdout:0/645: dread d9/fc [0,4194304] 0 2024-03-21T10:09:04.045 INFO:tasks.workunit.client.1.smithi174.stdout:7/589: write d0/d53/f8d [472298,64111] 0 2024-03-21T10:09:04.048 INFO:tasks.workunit.client.0.smithi153.stdout:2/889: creat dd/d15/d18/d2c/d4b/d56/d63/d8d/d9b/dc3/df5/dae/d50/d59/f137 x:0 0 0 2024-03-21T10:09:04.048 INFO:tasks.workunit.client.0.smithi153.stdout:1/703: creat d6/d10/d2c/fe3 x:0 0 0 2024-03-21T10:09:04.048 INFO:tasks.workunit.client.0.smithi153.stdout:5/821: read d4/d8/d5a/d5d/dd9/d44/de1/dfb/d108/f84 [5154378,101209] 0 2024-03-21T10:09:04.050 INFO:tasks.workunit.client.1.smithi174.stdout:2/643: creat d5/d25/d27/d2d/d45/dae/dd6/d90/fdd x:0 0 0 2024-03-21T10:09:04.052 INFO:tasks.workunit.client.0.smithi153.stdout:3/905: creat dd/d41/d80/dbb/f139 x:0 0 0 2024-03-21T10:09:04.054 INFO:tasks.workunit.client.1.smithi174.stdout:0/646: truncate d9/d21/f2b 885279 0 2024-03-21T10:09:04.056 INFO:tasks.workunit.client.1.smithi174.stdout:6/522: creat d0/d3/d5/df/d66/d87/f9e x:0 0 0 2024-03-21T10:09:04.059 INFO:tasks.workunit.client.0.smithi153.stdout:0/764: dwrite d4/d16/d19/d30/d6c/d98/d5e/db2/fc6 [0,4194304] 0 2024-03-21T10:09:04.060 INFO:tasks.workunit.client.1.smithi174.stdout:4/778: getdents d4/d19 0 2024-03-21T10:09:04.065 INFO:tasks.workunit.client.0.smithi153.stdout:5/822: chown d4/d5/df/d67/d86/d96/deb/d56/d105/f10e 793 1 2024-03-21T10:09:04.069 INFO:tasks.workunit.client.0.smithi153.stdout:7/844: dread d4/d2a/dc6/dfb/f2c [0,4194304] 0 2024-03-21T10:09:04.070 INFO:tasks.workunit.client.1.smithi174.stdout:6/523: creat d0/d3/d20/d3d/d81/f9f x:0 0 0 2024-03-21T10:09:04.073 INFO:tasks.workunit.client.1.smithi174.stdout:6/524: dread - d0/d3/d5/df/d66/f96 zero size 2024-03-21T10:09:04.074 INFO:tasks.workunit.client.0.smithi153.stdout:4/874: truncate d11/d18/f49 519033 0 2024-03-21T10:09:04.075 INFO:tasks.workunit.client.0.smithi153.stdout:3/906: creat dd/d1e/d2f/d35/daf/dcc/d58/d138/f13a x:0 0 0 2024-03-21T10:09:04.076 INFO:tasks.workunit.client.1.smithi174.stdout:0/647: dread d9/d21/f26 [0,4194304] 0 2024-03-21T10:09:04.078 INFO:tasks.workunit.client.0.smithi153.stdout:1/704: rename d6/d10/d35/da5/d4f/d9d/cbd to d6/d10/da0/ce4 0 2024-03-21T10:09:04.078 INFO:tasks.workunit.client.0.smithi153.stdout:4/875: chown d11/d5e/f7d 5112166 1 2024-03-21T10:09:04.080 INFO:tasks.workunit.client.0.smithi153.stdout:8/806: dwrite d5/d9/d43/dd/d20/d26/d47/d69/f86 [0,4194304] 0 2024-03-21T10:09:04.081 INFO:tasks.workunit.client.0.smithi153.stdout:1/705: chown d6/d13/d75/l78 45141 1 2024-03-21T10:09:04.082 INFO:tasks.workunit.client.0.smithi153.stdout:2/890: fsync dd/d15/d18/d2c/d4b/d56/d63/fdf 0 2024-03-21T10:09:04.083 INFO:tasks.workunit.client.1.smithi174.stdout:9/702: dread d2/d11/d1d/f41 [0,4194304] 0 2024-03-21T10:09:04.084 INFO:tasks.workunit.client.1.smithi174.stdout:1/762: dwrite d4/d5/f101 [0,4194304] 0 2024-03-21T10:09:04.084 INFO:tasks.workunit.client.0.smithi153.stdout:3/907: mkdir dd/d1e/d3d/dc9/d13b 0 2024-03-21T10:09:04.085 INFO:tasks.workunit.client.1.smithi174.stdout:0/648: creat d9/d21/d31/d5c/d86/fe4 x:0 0 0 2024-03-21T10:09:04.086 INFO:tasks.workunit.client.0.smithi153.stdout:4/876: write d11/d18/d50/d65/ff8 [3599897,99133] 0 2024-03-21T10:09:04.091 INFO:tasks.workunit.client.0.smithi153.stdout:3/908: write dd/d1e/d3d/d94/de2/f100 [3862264,23857] 0 2024-03-21T10:09:04.092 INFO:tasks.workunit.client.1.smithi174.stdout:0/649: fsync d9/f6d 0 2024-03-21T10:09:04.095 INFO:tasks.workunit.client.0.smithi153.stdout:9/888: dwrite d1/d4/f9c [0,4194304] 0 2024-03-21T10:09:04.096 INFO:tasks.workunit.client.0.smithi153.stdout:7/845: creat d4/d14/d16/f11a x:0 0 0 2024-03-21T10:09:04.097 INFO:tasks.workunit.client.1.smithi174.stdout:7/590: write d0/d53/d5f/fc9 [2575507,51156] 0 2024-03-21T10:09:04.097 INFO:tasks.workunit.client.0.smithi153.stdout:7/846: chown d4/d11/da1 1 1 2024-03-21T10:09:04.097 INFO:tasks.workunit.client.1.smithi174.stdout:9/703: truncate d2/d11/d1d/d2a/df2/f95 51725 0 2024-03-21T10:09:04.098 INFO:tasks.workunit.client.1.smithi174.stdout:2/644: dread d5/d25/d27/d2d/d45/d5b/f22 [0,4194304] 0 2024-03-21T10:09:04.100 INFO:tasks.workunit.client.1.smithi174.stdout:6/525: dwrite d0/d3/d5/df/d16/f27 [0,4194304] 0 2024-03-21T10:09:04.101 INFO:tasks.workunit.client.1.smithi174.stdout:7/591: readlink d0/d4/d10/l93 0 2024-03-21T10:09:04.101 INFO:tasks.workunit.client.1.smithi174.stdout:1/763: link d4/d67/f99 d4/f110 0 2024-03-21T10:09:04.101 INFO:tasks.workunit.client.1.smithi174.stdout:0/650: creat d9/d21/d31/d5c/dd5/fe5 x:0 0 0 2024-03-21T10:09:04.103 INFO:tasks.workunit.client.0.smithi153.stdout:8/807: rmdir d5/d9/d43/d7b/dc4/da7 39 2024-03-21T10:09:04.103 INFO:tasks.workunit.client.0.smithi153.stdout:2/891: fsync dd/d15/d5e/f8f 0 2024-03-21T10:09:04.105 INFO:tasks.workunit.client.1.smithi174.stdout:2/645: chown d5/d25/d27/d2d/d45/d5b/c1a 447 1 2024-03-21T10:09:04.110 INFO:tasks.workunit.client.1.smithi174.stdout:4/779: dread d4/d21/d24/d28/de9/d3a/d67/d76/fbd [4194304,4194304] 0 2024-03-21T10:09:04.110 INFO:tasks.workunit.client.1.smithi174.stdout:5/723: dread d1/d16/d2e/d66/d8b/f36 [0,4194304] 0 2024-03-21T10:09:04.111 INFO:tasks.workunit.client.0.smithi153.stdout:2/892: write dd/d15/f40 [2981527,26580] 0 2024-03-21T10:09:04.114 INFO:tasks.workunit.client.1.smithi174.stdout:4/780: readlink d4/l2a 0 2024-03-21T10:09:04.115 INFO:tasks.workunit.client.1.smithi174.stdout:0/651: dread d9/d21/d31/f34 [0,4194304] 0 2024-03-21T10:09:04.118 INFO:tasks.workunit.client.1.smithi174.stdout:2/646: chown d5/d25/d27/d2d/d45/dae/dd6/d4f/d65/d77/cd4 41934 1 2024-03-21T10:09:04.119 INFO:tasks.workunit.client.1.smithi174.stdout:7/592: creat d0/d4/d10/d44/d81/d9a/fcb x:0 0 0 2024-03-21T10:09:04.119 INFO:tasks.workunit.client.1.smithi174.stdout:6/526: truncate d0/d3/d5/df/d16/f2b 3956720 0 2024-03-21T10:09:04.121 INFO:tasks.workunit.client.1.smithi174.stdout:5/724: readlink d1/d49/d4f/l59 0 2024-03-21T10:09:04.122 INFO:tasks.workunit.client.0.smithi153.stdout:2/893: rmdir dd/d15/d18/d3a/d3d 39 2024-03-21T10:09:04.123 INFO:tasks.workunit.client.1.smithi174.stdout:4/781: rmdir d4/d21/d24/d28/de9/d6a/d8f/d9c 39 2024-03-21T10:09:04.125 INFO:tasks.workunit.client.1.smithi174.stdout:7/593: unlink d0/f27 0 2024-03-21T10:09:04.125 INFO:tasks.workunit.client.0.smithi153.stdout:1/706: rename d6/d10/d35/cd7 to d6/d3a/ce5 0 2024-03-21T10:09:04.132 INFO:tasks.workunit.client.0.smithi153.stdout:7/847: creat d4/d14/d2f/d118/f11b x:0 0 0 2024-03-21T10:09:04.133 INFO:tasks.workunit.client.1.smithi174.stdout:6/527: mknod d0/d3/d5/d21/ca0 0 2024-03-21T10:09:04.134 INFO:tasks.workunit.client.0.smithi153.stdout:1/707: write d6/d10/d35/da5/d4f/d9d/dc7/dcc/f68 [5692705,48569] 0 2024-03-21T10:09:04.136 INFO:tasks.workunit.client.0.smithi153.stdout:9/889: link d1/d43/d44/fff d1/d43/d44/dc3/f124 0 2024-03-21T10:09:04.139 INFO:tasks.workunit.client.0.smithi153.stdout:1/708: fsync d6/d10/d35/d5b/d92/fc0 0 2024-03-21T10:09:04.141 INFO:tasks.workunit.client.0.smithi153.stdout:0/765: dread d4/d16/d19/d30/d6c/d98/d5e/f78 [0,4194304] 0 2024-03-21T10:09:04.142 INFO:tasks.workunit.client.1.smithi174.stdout:5/725: truncate d1/d2f/dc1/fe8 537764 0 2024-03-21T10:09:04.142 INFO:tasks.workunit.client.1.smithi174.stdout:9/704: dwrite d2/d11/d1d/f33 [0,4194304] 0 2024-03-21T10:09:04.144 INFO:tasks.workunit.client.1.smithi174.stdout:7/594: link d0/d4/d10/d1b/d86/l97 d0/d53/d5f/d65/lcc 0 2024-03-21T10:09:04.146 INFO:tasks.workunit.client.0.smithi153.stdout:9/890: dread d1/d43/d44/d46/d9e/d24/d41/f6a [0,4194304] 0 2024-03-21T10:09:04.148 INFO:tasks.workunit.client.1.smithi174.stdout:9/705: stat d2/d54/fea 0 2024-03-21T10:09:04.148 INFO:tasks.workunit.client.0.smithi153.stdout:7/848: mknod d4/d11/da1/c11c 0 2024-03-21T10:09:04.151 INFO:tasks.workunit.client.1.smithi174.stdout:6/528: mknod d0/d3/d2a/d56/d5e/d91/ca1 0 2024-03-21T10:09:04.151 INFO:tasks.workunit.client.1.smithi174.stdout:5/726: write d1/f86 [302509,40645] 0 2024-03-21T10:09:04.151 INFO:tasks.workunit.client.1.smithi174.stdout:1/764: dwrite d4/d2d/f57 [0,4194304] 0 2024-03-21T10:09:04.153 INFO:tasks.workunit.client.1.smithi174.stdout:6/529: write d0/d3/d2a/d56/d5e/f98 [371883,23439] 0 2024-03-21T10:09:04.154 INFO:tasks.workunit.client.1.smithi174.stdout:7/595: rmdir d0/d53/d5f 39 2024-03-21T10:09:04.156 INFO:tasks.workunit.client.0.smithi153.stdout:4/877: write d11/f78 [446052,51358] 0 2024-03-21T10:09:04.156 INFO:tasks.workunit.client.0.smithi153.stdout:5/823: link d4/d8/d5a/d5d/dd9/f43 d4/d8/d5a/d5d/dd9/f114 0 2024-03-21T10:09:04.157 INFO:tasks.workunit.client.1.smithi174.stdout:2/647: link d5/d25/d27/d2d/d45/d5b/d2f/d66/fb9 d5/d25/d27/d2d/d45/dae/dd6/d4f/da0/fde 0 2024-03-21T10:09:04.158 INFO:tasks.workunit.client.1.smithi174.stdout:9/706: chown d2/d24/c26 239 1 2024-03-21T10:09:04.158 INFO:tasks.workunit.client.1.smithi174.stdout:5/727: readlink d1/l14 0 2024-03-21T10:09:04.160 INFO:tasks.workunit.client.0.smithi153.stdout:2/894: mknod dd/d15/d18/d3a/c138 0 2024-03-21T10:09:04.161 INFO:tasks.workunit.client.1.smithi174.stdout:9/707: write d2/d11/d1d/d2a/da7/fe9 [792368,61303] 0 2024-03-21T10:09:04.163 INFO:tasks.workunit.client.1.smithi174.stdout:1/765: read d4/d7/d15/d53/f54 [2520996,50210] 0 2024-03-21T10:09:04.164 INFO:tasks.workunit.client.0.smithi153.stdout:2/895: chown dd/d15/d18/d2c/d33/d98/dc7/d107 22987 1 2024-03-21T10:09:04.165 INFO:tasks.workunit.client.0.smithi153.stdout:9/891: rename d1/d4e/fa6 to d1/d43/d44/d46/d9e/d2e/f125 0 2024-03-21T10:09:04.168 INFO:tasks.workunit.client.1.smithi174.stdout:4/782: dwrite d4/d21/d24/d28/de9/d6a/dab/f99 [0,4194304] 0 2024-03-21T10:09:04.170 INFO:tasks.workunit.client.1.smithi174.stdout:5/728: symlink d1/d5c/d73/de3/l108 0 2024-03-21T10:09:04.171 INFO:tasks.workunit.client.0.smithi153.stdout:8/808: dread d5/f2f [4194304,4194304] 0 2024-03-21T10:09:04.172 INFO:tasks.workunit.client.0.smithi153.stdout:0/766: creat d4/d16/d19/dad/dec/ff0 x:0 0 0 2024-03-21T10:09:04.172 INFO:tasks.workunit.client.0.smithi153.stdout:3/909: dread dd/d1e/d2f/d35/daf/dcc/d58/fee [0,4194304] 0 2024-03-21T10:09:04.173 INFO:tasks.workunit.client.1.smithi174.stdout:6/530: link d0/d3/d5/d21/f6c d0/d3/d2a/fa2 0 2024-03-21T10:09:04.175 INFO:tasks.workunit.client.1.smithi174.stdout:9/708: write d2/d11/d1d/d8b/ff7 [721387,6369] 0 2024-03-21T10:09:04.175 INFO:tasks.workunit.client.1.smithi174.stdout:6/531: stat d0/d3 0 2024-03-21T10:09:04.178 INFO:tasks.workunit.client.0.smithi153.stdout:7/849: rename d4/d14/d2f/f5e to d4/d2a/f11d 0 2024-03-21T10:09:04.178 INFO:tasks.workunit.client.0.smithi153.stdout:4/878: rename d11 to d11/d18/d6d/d112 22 2024-03-21T10:09:04.180 INFO:tasks.workunit.client.1.smithi174.stdout:5/729: write d1/d49/d4f/f57 [4899347,40196] 0 2024-03-21T10:09:04.183 INFO:tasks.workunit.client.0.smithi153.stdout:7/850: write d4/d14/d2f/d78/f6f [1527755,103580] 0 2024-03-21T10:09:04.183 INFO:tasks.workunit.client.0.smithi153.stdout:2/896: chown dd/d15/d18/d3a/d3d/f131 63368769 1 2024-03-21T10:09:04.184 INFO:tasks.workunit.client.1.smithi174.stdout:0/652: dread d9/d21/d23/f91 [0,4194304] 0 2024-03-21T10:09:04.184 INFO:tasks.workunit.client.0.smithi153.stdout:8/809: mknod d5/d9/c106 0 2024-03-21T10:09:04.186 INFO:tasks.workunit.client.1.smithi174.stdout:9/709: truncate d2/d11/d1d/d2a/da7/fd1 900795 0 2024-03-21T10:09:04.187 INFO:tasks.workunit.client.0.smithi153.stdout:0/767: mkdir d4/d4c/d83/d22/d61/d77/df1 0 2024-03-21T10:09:04.188 INFO:tasks.workunit.client.1.smithi174.stdout:6/532: symlink d0/d3/d2a/la3 0 2024-03-21T10:09:04.190 INFO:tasks.workunit.client.0.smithi153.stdout:0/768: fsync d4/d16/d19/fc3 0 2024-03-21T10:09:04.192 INFO:tasks.workunit.client.0.smithi153.stdout:7/851: write d4/d2a/d3b/d81/f90 [1683843,106483] 0 2024-03-21T10:09:04.192 INFO:tasks.workunit.client.0.smithi153.stdout:2/897: read - dd/d15/d18/d2c/d4b/d56/d63/d8d/d9b/dc3/f132 zero size 2024-03-21T10:09:04.192 INFO:tasks.workunit.client.0.smithi153.stdout:4/879: fsync d11/d71/d89/fb9 0 2024-03-21T10:09:04.193 INFO:tasks.workunit.client.1.smithi174.stdout:5/730: unlink d1/d16/d1c/c52 0 2024-03-21T10:09:04.195 INFO:tasks.workunit.client.0.smithi153.stdout:3/910: mknod dd/d1e/d3d/d94/d73/d7f/c13c 0 2024-03-21T10:09:04.195 INFO:tasks.workunit.client.1.smithi174.stdout:7/596: dwrite d0/d4/f12 [0,4194304] 0 2024-03-21T10:09:04.199 INFO:tasks.workunit.client.0.smithi153.stdout:3/911: stat dd/d1e/d2f/d35/d111 0 2024-03-21T10:09:04.201 INFO:tasks.workunit.client.1.smithi174.stdout:0/653: fsync d9/d1c/f20 0 2024-03-21T10:09:04.205 INFO:tasks.workunit.client.1.smithi174.stdout:1/766: write d4/d7/d68/d7b/f8c [1619958,32659] 0 2024-03-21T10:09:04.207 INFO:tasks.workunit.client.0.smithi153.stdout:1/709: truncate f3 3797971 0 2024-03-21T10:09:04.207 INFO:tasks.workunit.client.1.smithi174.stdout:5/731: symlink d1/d2f/d37/d9b/l109 0 2024-03-21T10:09:04.210 INFO:tasks.workunit.client.1.smithi174.stdout:0/654: write d9/d21/d75/fc7 [630132,117855] 0 2024-03-21T10:09:04.210 INFO:tasks.workunit.client.1.smithi174.stdout:8/628: sync 2024-03-21T10:09:04.210 INFO:tasks.workunit.client.1.smithi174.stdout:7/597: readlink d0/d53/d5f/l7d 0 2024-03-21T10:09:04.215 INFO:tasks.workunit.client.0.smithi153.stdout:7/852: unlink d4/d2a/d40/fee 0 2024-03-21T10:09:04.215 INFO:tasks.workunit.client.1.smithi174.stdout:9/710: rename d2/f9 to d2/d11/d60/db2/ffa 0 2024-03-21T10:09:04.219 INFO:tasks.workunit.client.1.smithi174.stdout:2/648: truncate d5/d25/d27/d2d/d45/f6e 2930827 0 2024-03-21T10:09:04.222 INFO:tasks.workunit.client.0.smithi153.stdout:0/769: chown d4/d16/d19/d30/da7/lae 831188007 1 2024-03-21T10:09:04.222 INFO:tasks.workunit.client.1.smithi174.stdout:2/649: chown d5/d25/d27/d2d/d45/d5b/d55 11066 1 2024-03-21T10:09:04.224 INFO:tasks.workunit.client.0.smithi153.stdout:4/880: symlink d11/d9d/dc3/l113 0 2024-03-21T10:09:04.229 INFO:tasks.workunit.client.0.smithi153.stdout:9/892: dread d1/d43/d44/d46/d100/fd0 [0,4194304] 0 2024-03-21T10:09:04.230 INFO:tasks.workunit.client.0.smithi153.stdout:8/810: symlink d5/d9/d43/dd/d20/dfb/l107 0 2024-03-21T10:09:04.230 INFO:tasks.workunit.client.1.smithi174.stdout:8/629: chown d9/d2a/d39/l46 1556210 1 2024-03-21T10:09:04.232 INFO:tasks.workunit.client.1.smithi174.stdout:4/783: dread d4/d21/f8c [0,4194304] 0 2024-03-21T10:09:04.233 INFO:tasks.workunit.client.0.smithi153.stdout:3/912: fdatasync dd/d1e/d2f/d35/daf/dcc/d58/fee 0 2024-03-21T10:09:04.234 INFO:tasks.workunit.client.1.smithi174.stdout:8/630: write d9/d2a/d39/d76/f5a [4978674,100255] 0 2024-03-21T10:09:04.234 INFO:tasks.workunit.client.1.smithi174.stdout:2/650: creat d5/d25/d27/d2d/d45/dae/dd6/d4f/d65/d77/fdf x:0 0 0 2024-03-21T10:09:04.235 INFO:tasks.workunit.client.0.smithi153.stdout:3/913: fdatasync dd/d1e/d2f/d35/f7b 0 2024-03-21T10:09:04.236 INFO:tasks.workunit.client.1.smithi174.stdout:4/784: chown d4/d21/d24/d28/de9/d3a/d67 1 1 2024-03-21T10:09:04.237 INFO:tasks.workunit.client.1.smithi174.stdout:5/732: rmdir d1/d2f/d37 39 2024-03-21T10:09:04.237 INFO:tasks.workunit.client.0.smithi153.stdout:5/824: dwrite d4/d8/f6c [0,4194304] 0 2024-03-21T10:09:04.238 INFO:tasks.workunit.client.1.smithi174.stdout:0/655: dread d9/d35/d4a/d67/d89/fd3 [0,4194304] 0 2024-03-21T10:09:04.239 INFO:tasks.workunit.client.0.smithi153.stdout:2/898: creat dd/d15/d18/d2c/d4b/d56/d63/d8d/d9b/dc3/df5/dae/d50/d58/d79/f139 x:0 0 0 2024-03-21T10:09:04.241 INFO:tasks.workunit.client.1.smithi174.stdout:8/631: write d9/d2a/d39/d76/d19/d27/d60/fa8 [1667219,12662] 0 2024-03-21T10:09:04.242 INFO:tasks.workunit.client.0.smithi153.stdout:5/825: write d4/d8/d5a/d5d/dd9/d44/dc7/fe8 [392763,14816] 0 2024-03-21T10:09:04.243 INFO:tasks.workunit.client.1.smithi174.stdout:2/651: rmdir d5/d25 39 2024-03-21T10:09:04.245 INFO:tasks.workunit.client.0.smithi153.stdout:7/853: creat d4/d14/d16/f11e x:0 0 0 2024-03-21T10:09:04.245 INFO:tasks.workunit.client.0.smithi153.stdout:4/881: creat d11/d2d/d3d/dc7/dda/f114 x:0 0 0 2024-03-21T10:09:04.246 INFO:tasks.workunit.client.1.smithi174.stdout:8/632: write d9/d53/fbc [887910,129041] 0 2024-03-21T10:09:04.248 INFO:tasks.workunit.client.0.smithi153.stdout:8/811: symlink d5/d9/d43/dd/l108 0 2024-03-21T10:09:04.248 INFO:tasks.workunit.client.0.smithi153.stdout:2/899: stat dd/d15/d18/d2c/d4b/d56/d63/d8d/d9b/dc3/df5/dae/d50/d59/f137 0 2024-03-21T10:09:04.249 INFO:tasks.workunit.client.1.smithi174.stdout:5/733: fsync d1/d49/f102 0 2024-03-21T10:09:04.249 INFO:tasks.workunit.client.1.smithi174.stdout:1/767: link d4/d7/d33/d43/d77/d87/l103 d4/d7/d15/d53/d78/l111 0 2024-03-21T10:09:04.251 INFO:tasks.workunit.client.1.smithi174.stdout:6/533: dread d0/d3/d5/f57 [0,4194304] 0 2024-03-21T10:09:04.251 INFO:tasks.workunit.client.0.smithi153.stdout:2/900: fsync dd/d15/d18/d3a/fdb 0 2024-03-21T10:09:04.251 INFO:tasks.workunit.client.0.smithi153.stdout:7/854: chown d4/d2a/d3b/cb3 627 1 2024-03-21T10:09:04.252 INFO:tasks.workunit.client.1.smithi174.stdout:9/711: rename d2/d11/d60/l61 to d2/d11/d1d/lfb 0 2024-03-21T10:09:04.252 INFO:tasks.workunit.client.1.smithi174.stdout:4/785: mkdir d4/d21/d24/d28/de9/d3a/dd2/d103 0 2024-03-21T10:09:04.253 INFO:tasks.workunit.client.0.smithi153.stdout:3/914: creat dd/d1e/d2f/d35/d68/d75/ddd/f13d x:0 0 0 2024-03-21T10:09:04.255 INFO:tasks.workunit.client.0.smithi153.stdout:7/855: fsync d4/d2a/d3b/f103 0 2024-03-21T10:09:04.255 INFO:tasks.workunit.client.1.smithi174.stdout:6/534: stat d0/d3/d5/d21/l30 0 2024-03-21T10:09:04.256 INFO:tasks.workunit.client.1.smithi174.stdout:8/633: dread d9/d2a/d39/d76/d19/d27/d79/fa6 [0,4194304] 0 2024-03-21T10:09:04.258 INFO:tasks.workunit.client.1.smithi174.stdout:5/734: symlink d1/d49/d4f/l10a 0 2024-03-21T10:09:04.262 INFO:tasks.workunit.client.0.smithi153.stdout:1/710: truncate d6/d10/d35/da5/d4f/d9d/dc7/dcc/faf 8079456 0 2024-03-21T10:09:04.262 INFO:tasks.workunit.client.1.smithi174.stdout:0/656: dread d9/f65 [0,4194304] 0 2024-03-21T10:09:04.266 INFO:tasks.workunit.client.1.smithi174.stdout:1/768: rename d4/d7/d15/l46 to d4/d7/d68/d8e/db8/l112 0 2024-03-21T10:09:04.266 INFO:tasks.workunit.client.0.smithi153.stdout:4/882: mkdir d11/d18/d50/d65/de3/d115 0 2024-03-21T10:09:04.267 INFO:tasks.workunit.client.0.smithi153.stdout:5/826: read d4/d8/d5a/d5d/dd9/d44/de1/dfb/d108/fc6 [3303380,30161] 0 2024-03-21T10:09:04.270 INFO:tasks.workunit.client.1.smithi174.stdout:3/660: sync 2024-03-21T10:09:04.277 INFO:tasks.workunit.client.1.smithi174.stdout:7/598: dwrite d0/d4/d10/d1b/f25 [0,4194304] 0 2024-03-21T10:09:04.278 INFO:tasks.workunit.client.0.smithi153.stdout:7/856: rename d4/d2a/dc6/dfb/d9a/dd2/db9/fe3 to d4/d2a/dc6/dfb/f11f 0 2024-03-21T10:09:04.283 INFO:tasks.workunit.client.1.smithi174.stdout:9/712: write d2/d11/d1d/d2a/da7/ff1 [515373,3280] 0 2024-03-21T10:09:04.283 INFO:tasks.workunit.client.1.smithi174.stdout:0/657: dread - d9/d21/fa9 zero size 2024-03-21T10:09:04.283 INFO:tasks.workunit.client.1.smithi174.stdout:8/634: rename d9/d2a/d39/d58/ca4 to d9/d2a/d39/d76/d19/d27/d60/cd0 0 2024-03-21T10:09:04.283 INFO:tasks.workunit.client.0.smithi153.stdout:7/857: write d4/d14/d2f/d78/f35 [876810,17122] 0 2024-03-21T10:09:04.285 INFO:tasks.workunit.client.0.smithi153.stdout:4/883: mkdir d11/d18/d50/d116 0 2024-03-21T10:09:04.286 INFO:tasks.workunit.client.1.smithi174.stdout:6/535: getdents d0/d3/d20 0 2024-03-21T10:09:04.286 INFO:tasks.workunit.client.0.smithi153.stdout:2/901: dread dd/d15/d18/d3a/f133 [0,4194304] 0 2024-03-21T10:09:04.287 INFO:tasks.workunit.client.1.smithi174.stdout:8/635: chown d9/d2a/d4d/f6d 13485462 1 2024-03-21T10:09:04.289 INFO:tasks.workunit.client.0.smithi153.stdout:7/858: read d4/d14/d2f/d78/f35 [3194793,26601] 0 2024-03-21T10:09:04.289 INFO:tasks.workunit.client.0.smithi153.stdout:8/812: unlink d5/d9/d43/dd/d2d/d45/d88/fbe 0 2024-03-21T10:09:04.290 INFO:tasks.workunit.client.0.smithi153.stdout:5/827: dread d4/d8/d5a/d5d/dd9/d38/f64 [0,4194304] 0 2024-03-21T10:09:04.291 INFO:tasks.workunit.client.0.smithi153.stdout:8/813: stat d5/d9/d43/f77 0 2024-03-21T10:09:04.293 INFO:tasks.workunit.client.1.smithi174.stdout:2/652: link d5/d25/d27/d2d/d45/d5b/f29 d5/d25/d27/d2d/d45/dae/dd6/d7b/d81/da1/fe0 0 2024-03-21T10:09:04.293 INFO:tasks.workunit.client.1.smithi174.stdout:2/653: stat d5/d25/l5a 0 2024-03-21T10:09:04.295 INFO:tasks.workunit.client.1.smithi174.stdout:1/769: getdents d4/d7/d32/d86 0 2024-03-21T10:09:04.296 INFO:tasks.workunit.client.1.smithi174.stdout:6/536: truncate d0/d3/d5/d6/d42/f82 1045078 0 2024-03-21T10:09:04.301 INFO:tasks.workunit.client.0.smithi153.stdout:1/711: mkdir d6/de6 0 2024-03-21T10:09:04.303 INFO:tasks.workunit.client.1.smithi174.stdout:3/661: dwrite d0/f7 [0,4194304] 0 2024-03-21T10:09:04.304 INFO:tasks.workunit.client.1.smithi174.stdout:4/786: rename d4/d21/d24/f4d to d4/d21/d24/d28/de9/d3a/d7a/d9d/f104 0 2024-03-21T10:09:04.305 INFO:tasks.workunit.client.1.smithi174.stdout:1/770: creat d4/d7/d68/d89/f113 x:0 0 0 2024-03-21T10:09:04.306 INFO:tasks.workunit.client.1.smithi174.stdout:0/658: getdents d9/d21/d81 0 2024-03-21T10:09:04.308 INFO:tasks.workunit.client.0.smithi153.stdout:4/884: mknod d11/d2d/d3d/dc7/c117 0 2024-03-21T10:09:04.309 INFO:tasks.workunit.client.0.smithi153.stdout:1/712: write d6/fd [2044371,29976] 0 2024-03-21T10:09:04.311 INFO:tasks.workunit.client.0.smithi153.stdout:8/814: symlink d5/d9/d43/dd/d20/d26/d47/d9a/db9/l109 0 2024-03-21T10:09:04.311 INFO:tasks.workunit.client.1.smithi174.stdout:4/787: chown d4/d21/d24/d28/d38/de5 949039 1 2024-03-21T10:09:04.312 INFO:tasks.workunit.client.1.smithi174.stdout:6/537: creat d0/d3/d2a/d56/d5e/d91/fa4 x:0 0 0 2024-03-21T10:09:04.314 INFO:tasks.workunit.client.0.smithi153.stdout:5/828: creat d4/d8/d5a/d5d/dd9/d38/f115 x:0 0 0 2024-03-21T10:09:04.316 INFO:tasks.workunit.client.1.smithi174.stdout:5/735: dwrite d1/d16/d2e/d66/d8b/d85/fee [4194304,4194304] 0 2024-03-21T10:09:04.316 INFO:tasks.workunit.client.0.smithi153.stdout:0/770: truncate d4/d16/d19/d30/d6c/d98/d5e/db2/fc6 71178 0 2024-03-21T10:09:04.319 INFO:tasks.workunit.client.1.smithi174.stdout:7/599: rename d0/d4/d38/fb7 to d0/d53/d5f/d65/fcd 0 2024-03-21T10:09:04.319 INFO:tasks.workunit.client.1.smithi174.stdout:9/713: dwrite d2/d11/d1d/f41 [0,4194304] 0 2024-03-21T10:09:04.322 INFO:tasks.workunit.client.0.smithi153.stdout:5/829: chown d4/d5/df/d67/d86/d96/deb 374 1 2024-03-21T10:09:04.326 INFO:tasks.workunit.client.0.smithi153.stdout:5/830: dread - d4/d8/d5a/d5d/fef zero size 2024-03-21T10:09:04.329 INFO:tasks.workunit.client.1.smithi174.stdout:3/662: chown d0/d12/d14/d1a/d27/c50 335975975 1 2024-03-21T10:09:04.329 INFO:tasks.workunit.client.1.smithi174.stdout:5/736: fdatasync d1/d49/d93/d9c/fb2 0 2024-03-21T10:09:04.330 INFO:tasks.workunit.client.0.smithi153.stdout:3/915: write dd/d1e/d3d/fe7 [393201,127773] 0 2024-03-21T10:09:04.333 INFO:tasks.workunit.client.1.smithi174.stdout:1/771: creat d4/d7/d33/d43/d77/d87/d92/f114 x:0 0 0 2024-03-21T10:09:04.334 INFO:tasks.workunit.client.0.smithi153.stdout:7/859: dwrite d4/d19/fc1 [0,4194304] 0 2024-03-21T10:09:04.334 INFO:tasks.workunit.client.0.smithi153.stdout:9/893: dwrite d1/d4/f75 [0,4194304] 0 2024-03-21T10:09:04.335 INFO:tasks.workunit.client.1.smithi174.stdout:3/663: chown d0/d12/d14/d1a/d44/d65/f6b 18355 1 2024-03-21T10:09:04.336 INFO:tasks.workunit.client.1.smithi174.stdout:4/788: chown d4/d21/d24/d28/de9/d6a/d8f/d9c 0 1 2024-03-21T10:09:04.338 INFO:tasks.workunit.client.1.smithi174.stdout:6/538: mkdir d0/d3/d20/d3d/d81/da5 0 2024-03-21T10:09:04.342 INFO:tasks.workunit.client.1.smithi174.stdout:8/636: rename d9/d2a/d39/d76/d19/d27/d79 to d9/dc6/dd1 0 2024-03-21T10:09:04.352 INFO:tasks.workunit.client.1.smithi174.stdout:2/654: write d5/d25/d27/d2d/d45/dae/dd6/d7b/f8e [2896121,88752] 0 2024-03-21T10:09:04.356 INFO:tasks.workunit.client.1.smithi174.stdout:5/737: dwrite d1/d16/d2e/d66/d8b/d85/f90 [0,4194304] 0 2024-03-21T10:09:04.356 INFO:tasks.workunit.client.1.smithi174.stdout:9/714: readlink d2/d11/d1d/dab/dc1/l9e 0 2024-03-21T10:09:04.357 INFO:tasks.workunit.client.0.smithi153.stdout:0/771: dwrite d4/d16/d19/d30/d6c/fcf [0,4194304] 0 2024-03-21T10:09:04.364 INFO:tasks.workunit.client.1.smithi174.stdout:3/664: fsync d0/f13 0 2024-03-21T10:09:04.374 INFO:tasks.workunit.client.1.smithi174.stdout:7/600: dwrite d0/d4/d10/d44/d81/d9a/fa3 [0,4194304] 0 2024-03-21T10:09:04.375 INFO:tasks.workunit.client.0.smithi153.stdout:2/902: symlink dd/d15/d18/d2c/d33/d98/dc7/de1/dfe/l13a 0 2024-03-21T10:09:04.377 INFO:tasks.workunit.client.1.smithi174.stdout:1/772: rmdir d4/d7/d68/d7b/de0 39 2024-03-21T10:09:04.380 INFO:tasks.workunit.client.1.smithi174.stdout:8/637: readlink d9/d2a/l93 0 2024-03-21T10:09:04.383 INFO:tasks.workunit.client.0.smithi153.stdout:9/894: creat d1/d3b/df8/f126 x:0 0 0 2024-03-21T10:09:04.385 INFO:tasks.workunit.client.1.smithi174.stdout:9/715: fdatasync d2/d54/f92 0 2024-03-21T10:09:04.385 INFO:tasks.workunit.client.1.smithi174.stdout:5/738: mkdir d1/d49/d4f/d51/d8f/d103/d10b 0 2024-03-21T10:09:04.386 INFO:tasks.workunit.client.0.smithi153.stdout:0/772: creat d4/d16/d19/d30/d6c/d98/d5e/dbe/ff2 x:0 0 0 2024-03-21T10:09:04.387 INFO:tasks.workunit.client.1.smithi174.stdout:5/739: readlink d1/d49/d4f/de4/lff 0 2024-03-21T10:09:04.388 INFO:tasks.workunit.client.1.smithi174.stdout:7/601: readlink d0/d4/d10/d1b/l84 0 2024-03-21T10:09:04.388 INFO:tasks.workunit.client.0.smithi153.stdout:1/713: mkdir d6/d13/d8e/dbb/de1/de7 0 2024-03-21T10:09:04.390 INFO:tasks.workunit.client.1.smithi174.stdout:5/740: readlink d1/d16/d1c/l55 0 2024-03-21T10:09:04.392 INFO:tasks.workunit.client.1.smithi174.stdout:2/655: getdents d5/d25/d27/d2d/d45/d5b/d2f/d4b/dd8 0 2024-03-21T10:09:04.392 INFO:tasks.workunit.client.1.smithi174.stdout:8/638: creat d9/fd2 x:0 0 0 2024-03-21T10:09:04.392 INFO:tasks.workunit.client.0.smithi153.stdout:3/916: mknod dd/d1e/d2f/d35/daf/dcc/c13e 0 2024-03-21T10:09:04.394 INFO:tasks.workunit.client.0.smithi153.stdout:1/714: chown d6/d10/d2c/f47 815696 1 2024-03-21T10:09:04.394 INFO:tasks.workunit.client.0.smithi153.stdout:7/860: dread d4/d2a/fdc [0,4194304] 0 2024-03-21T10:09:04.395 INFO:tasks.workunit.client.0.smithi153.stdout:9/895: creat d1/d43/d44/d46/d9e/d2e/d67/f127 x:0 0 0 2024-03-21T10:09:04.397 INFO:tasks.workunit.client.1.smithi174.stdout:4/789: dread d4/d21/d24/d28/de9/d6a/d8f/f94 [0,4194304] 0 2024-03-21T10:09:04.398 INFO:tasks.workunit.client.1.smithi174.stdout:9/716: creat d2/d11/d1d/dab/ffc x:0 0 0 2024-03-21T10:09:04.399 INFO:tasks.workunit.client.0.smithi153.stdout:2/903: unlink dd/d15/d3c/fcf 0 2024-03-21T10:09:04.399 INFO:tasks.workunit.client.0.smithi153.stdout:7/861: rmdir d4/d14/d73 39 2024-03-21T10:09:04.400 INFO:tasks.workunit.client.1.smithi174.stdout:7/602: write d0/f26 [2348080,106281] 0 2024-03-21T10:09:04.400 INFO:tasks.workunit.client.0.smithi153.stdout:9/896: chown d1/d43/d44/d46/d100/fd0 103 1 2024-03-21T10:09:04.401 INFO:tasks.workunit.client.0.smithi153.stdout:1/715: mknod d6/d10/d35/d5b/d92/ce8 0 2024-03-21T10:09:04.403 INFO:tasks.workunit.client.1.smithi174.stdout:1/773: mknod d4/d7/d68/c115 0 2024-03-21T10:09:04.405 INFO:tasks.workunit.client.0.smithi153.stdout:7/862: readlink d4/d11/l3f 0 2024-03-21T10:09:04.405 INFO:tasks.workunit.client.0.smithi153.stdout:3/917: creat dd/d1e/d2f/d35/d68/d75/ddd/f13f x:0 0 0 2024-03-21T10:09:04.406 INFO:tasks.workunit.client.1.smithi174.stdout:9/717: unlink d2/d24/d2e/l78 0 2024-03-21T10:09:04.407 INFO:tasks.workunit.client.1.smithi174.stdout:1/774: write d4/d7/d32/fee [295092,66140] 0 2024-03-21T10:09:04.408 INFO:tasks.workunit.client.0.smithi153.stdout:2/904: unlink dd/d15/d18/d3a/d3d/f131 0 2024-03-21T10:09:04.410 INFO:tasks.workunit.client.1.smithi174.stdout:7/603: symlink d0/d4/d10/d44/lce 0 2024-03-21T10:09:04.410 INFO:tasks.workunit.client.0.smithi153.stdout:9/897: mknod d1/d43/d44/d46/d9e/c128 0 2024-03-21T10:09:04.413 INFO:tasks.workunit.client.0.smithi153.stdout:9/898: stat d1/d4/d5f/df3/ff4 0 2024-03-21T10:09:04.415 INFO:tasks.workunit.client.0.smithi153.stdout:2/905: chown dd/d15/d18/d2c/d4b/d56/d63/d8d/d9b/dde 572 1 2024-03-21T10:09:04.415 INFO:tasks.workunit.client.0.smithi153.stdout:3/918: fsync dd/d1e/d3d/d94/db8/f104 0 2024-03-21T10:09:04.416 INFO:tasks.workunit.client.0.smithi153.stdout:1/716: fsync d6/d13/f71 0 2024-03-21T10:09:04.417 INFO:tasks.workunit.client.1.smithi174.stdout:5/741: dwrite d1/d16/f40 [0,4194304] 0 2024-03-21T10:09:04.423 INFO:tasks.workunit.client.1.smithi174.stdout:1/775: symlink d4/d7/d70/l116 0 2024-03-21T10:09:04.424 INFO:tasks.workunit.client.0.smithi153.stdout:2/906: chown dd/d15/d18/d2c/d4b/d56/d63/d8d/d9b/dc3/df5/dae/f77 6108 1 2024-03-21T10:09:04.425 INFO:tasks.workunit.client.0.smithi153.stdout:9/899: mkdir d1/d43/d44/d46/d129 0 2024-03-21T10:09:04.426 INFO:tasks.workunit.client.1.smithi174.stdout:1/776: write d4/d7/f90 [3123198,61466] 0 2024-03-21T10:09:04.432 INFO:tasks.workunit.client.0.smithi153.stdout:3/919: symlink dd/d1e/d3d/dc9/d13b/l140 0 2024-03-21T10:09:04.433 INFO:tasks.workunit.client.1.smithi174.stdout:7/604: symlink d0/d4/d10/lcf 0 2024-03-21T10:09:04.435 INFO:tasks.workunit.client.1.smithi174.stdout:1/777: chown d4/d7/d33/d43/l3f 48927 1 2024-03-21T10:09:04.439 INFO:tasks.workunit.client.1.smithi174.stdout:0/659: dwrite d9/d21/d31/f34 [0,4194304] 0 2024-03-21T10:09:04.441 INFO:tasks.workunit.client.1.smithi174.stdout:1/778: chown d4/d7/d15/d53/d78/f10b 14201 1 2024-03-21T10:09:04.447 INFO:tasks.workunit.client.0.smithi153.stdout:2/907: dread dd/d15/d5e/fbb [0,4194304] 0 2024-03-21T10:09:04.448 INFO:tasks.workunit.client.1.smithi174.stdout:4/790: dread d4/d21/d24/d28/de9/d3a/d7a/d9d/fc7 [0,4194304] 0 2024-03-21T10:09:04.449 INFO:tasks.workunit.client.1.smithi174.stdout:1/779: fsync d4/d2d/fe1 0 2024-03-21T10:09:04.452 INFO:tasks.workunit.client.1.smithi174.stdout:7/605: creat d0/d4/d10/d79/da7/fd0 x:0 0 0 2024-03-21T10:09:04.454 INFO:tasks.workunit.client.1.smithi174.stdout:1/780: dread - d4/d7/d68/d8e/db8/fd5 zero size 2024-03-21T10:09:04.454 INFO:tasks.workunit.client.1.smithi174.stdout:6/539: write d0/d3/d5/d6/f44 [713211,96087] 0 2024-03-21T10:09:04.454 INFO:tasks.workunit.client.1.smithi174.stdout:0/660: getdents d9/d35 0 2024-03-21T10:09:04.457 INFO:tasks.workunit.client.1.smithi174.stdout:7/606: chown d0/c5d 1879 1 2024-03-21T10:09:04.460 INFO:tasks.workunit.client.1.smithi174.stdout:0/661: dread - d9/d21/d31/d5c/d86/fbf zero size 2024-03-21T10:09:04.461 INFO:tasks.workunit.client.0.smithi153.stdout:8/815: dwrite d5/d9/d43/d7b/daa/dca/fd7 [0,4194304] 0 2024-03-21T10:09:04.462 INFO:tasks.workunit.client.1.smithi174.stdout:1/781: creat d4/d2d/f117 x:0 0 0 2024-03-21T10:09:04.469 INFO:tasks.workunit.client.0.smithi153.stdout:2/908: getdents dd/d15/d18/d2c/d4b/d56/d63/d8d/d9b/dc3/df5 0 2024-03-21T10:09:04.470 INFO:tasks.workunit.client.1.smithi174.stdout:5/742: dwrite d1/d5c/d73/de3/fde [0,4194304] 0 2024-03-21T10:09:04.472 INFO:tasks.workunit.client.1.smithi174.stdout:0/662: creat d9/d35/dd2/fe6 x:0 0 0 2024-03-21T10:09:04.473 INFO:tasks.workunit.client.0.smithi153.stdout:5/831: dread d4/d5/df/d67/d86/d96/deb/d56/fca [0,4194304] 0 2024-03-21T10:09:04.474 INFO:tasks.workunit.client.1.smithi174.stdout:2/656: dread d5/d25/d27/d2d/d45/d5b/f23 [4194304,4194304] 0 2024-03-21T10:09:04.476 INFO:tasks.workunit.client.1.smithi174.stdout:6/540: creat d0/d3/d20/d3d/fa6 x:0 0 0 2024-03-21T10:09:04.480 INFO:tasks.workunit.client.0.smithi153.stdout:4/885: dwrite d11/f76 [0,4194304] 0 2024-03-21T10:09:04.481 INFO:tasks.workunit.client.1.smithi174.stdout:0/663: fdatasync d9/d35/d4a/f6a 0 2024-03-21T10:09:04.481 INFO:tasks.workunit.client.1.smithi174.stdout:5/743: write d1/d49/d4f/d51/d8f/fb6 [981384,109701] 0 2024-03-21T10:09:04.483 INFO:tasks.workunit.client.0.smithi153.stdout:7/863: write d4/d11/f7f [1053593,27655] 0 2024-03-21T10:09:04.484 INFO:tasks.workunit.client.1.smithi174.stdout:8/639: truncate d9/d2a/d39/d76/f87 1882353 0 2024-03-21T10:09:04.485 INFO:tasks.workunit.client.0.smithi153.stdout:1/717: write d6/d32/f5a [673223,108667] 0 2024-03-21T10:09:04.488 INFO:tasks.workunit.client.0.smithi153.stdout:3/920: write dd/f22 [676692,68372] 0 2024-03-21T10:09:04.489 INFO:tasks.workunit.client.0.smithi153.stdout:9/900: write d1/d4e/fe9 [857737,127032] 0 2024-03-21T10:09:04.489 INFO:tasks.workunit.client.0.smithi153.stdout:2/909: write dd/d15/d18/d2c/d4b/d56/d63/d8d/d9b/dc3/df5/dae/d50/d58/d79/f139 [350806,22375] 0 2024-03-21T10:09:04.490 INFO:tasks.workunit.client.0.smithi153.stdout:0/773: dwrite d4/db/fd4 [0,4194304] 0 2024-03-21T10:09:04.492 INFO:tasks.workunit.client.0.smithi153.stdout:7/864: chown d4/d2a/d3b/d60/lc2 5555762 1 2024-03-21T10:09:04.493 INFO:tasks.workunit.client.1.smithi174.stdout:5/744: symlink d1/d16/d2e/d66/l10c 0 2024-03-21T10:09:04.493 INFO:tasks.workunit.client.1.smithi174.stdout:9/718: dwrite d2/d11/d1d/d2a/df2/faf [0,4194304] 0 2024-03-21T10:09:04.493 INFO:tasks.workunit.client.0.smithi153.stdout:4/886: fsync d11/fa6 0 2024-03-21T10:09:04.496 INFO:tasks.workunit.client.1.smithi174.stdout:1/782: rename d4/d7/d68/d8e/db8/cdb to d4/c118 0 2024-03-21T10:09:04.497 INFO:tasks.workunit.client.0.smithi153.stdout:3/921: chown dd/d1e/d2f/d35/dd0/d109 1579 1 2024-03-21T10:09:04.497 INFO:tasks.workunit.client.1.smithi174.stdout:0/664: creat d9/d21/fe7 x:0 0 0 2024-03-21T10:09:04.498 INFO:tasks.workunit.client.0.smithi153.stdout:9/901: symlink d1/d43/d44/d46/d100/l12a 0 2024-03-21T10:09:04.499 INFO:tasks.workunit.client.0.smithi153.stdout:0/774: chown d4/d4c/d83/f35 6 1 2024-03-21T10:09:04.499 INFO:tasks.workunit.client.1.smithi174.stdout:8/640: rmdir d9/d2a/d4d/db4 39 2024-03-21T10:09:04.500 INFO:tasks.workunit.client.1.smithi174.stdout:1/783: dread d4/d7/d15/d4f/f69 [0,4194304] 0 2024-03-21T10:09:04.504 INFO:tasks.workunit.client.1.smithi174.stdout:0/665: truncate d9/d21/d31/d5c/dd5/fe5 623791 0 2024-03-21T10:09:04.504 INFO:tasks.workunit.client.1.smithi174.stdout:7/607: link d0/d4/d10/d44/c5e d0/d4/d10/d79/cd1 0 2024-03-21T10:09:04.508 INFO:tasks.workunit.client.0.smithi153.stdout:4/887: symlink d11/d18/d50/dfa/l118 0 2024-03-21T10:09:04.509 INFO:tasks.workunit.client.0.smithi153.stdout:9/902: mkdir d1/d108/d12b 0 2024-03-21T10:09:04.510 INFO:tasks.workunit.client.0.smithi153.stdout:8/816: write d5/d9/d43/d7b/daa/fb8 [2565605,96894] 0 2024-03-21T10:09:04.511 INFO:tasks.workunit.client.0.smithi153.stdout:4/888: stat d11/d18/f82 0 2024-03-21T10:09:04.511 INFO:tasks.workunit.client.1.smithi174.stdout:5/745: truncate d1/f11 4474910 0 2024-03-21T10:09:04.511 INFO:tasks.workunit.client.1.smithi174.stdout:4/791: truncate d4/d21/d24/d28/de9/d6a/dab/da8/fc3 3441169 0 2024-03-21T10:09:04.513 INFO:tasks.workunit.client.1.smithi174.stdout:8/641: mknod d9/d2a/d39/d76/d19/d27/d60/daa/dab/cd3 0 2024-03-21T10:09:04.515 INFO:tasks.workunit.client.1.smithi174.stdout:1/784: truncate d4/d7/d15/f1f 119376 0 2024-03-21T10:09:04.516 INFO:tasks.workunit.client.0.smithi153.stdout:1/718: mkdir d6/d13/d8e/dbb/de1/de7/de9 0 2024-03-21T10:09:04.517 INFO:tasks.workunit.client.0.smithi153.stdout:3/922: mkdir dd/d1e/d45/d141 0 2024-03-21T10:09:04.519 INFO:tasks.workunit.client.1.smithi174.stdout:5/746: write d1/d5c/f7a [43566,80413] 0 2024-03-21T10:09:04.519 INFO:tasks.workunit.client.1.smithi174.stdout:6/541: dwrite d0/d3/d5/d6/d42/f69 [0,4194304] 0 2024-03-21T10:09:04.520 INFO:tasks.workunit.client.0.smithi153.stdout:3/923: chown dd/d1e/d6b/l84 240300 1 2024-03-21T10:09:04.523 INFO:tasks.workunit.client.1.smithi174.stdout:7/608: rename d0/d4/d10/d79/da7 to d0/d4/d10/d1b/d86/dd2 0 2024-03-21T10:09:04.523 INFO:tasks.workunit.client.1.smithi174.stdout:8/642: creat d9/d53/d5b/fd4 x:0 0 0 2024-03-21T10:09:04.523 INFO:tasks.workunit.client.0.smithi153.stdout:9/903: rmdir d1/d43/d44/d46/d9e/d24 39 2024-03-21T10:09:04.525 INFO:tasks.workunit.client.0.smithi153.stdout:7/865: dread d4/d14/d2f/d78/f35 [0,4194304] 0 2024-03-21T10:09:04.525 INFO:tasks.workunit.client.1.smithi174.stdout:1/785: truncate d4/d2d/f5e 5079730 0 2024-03-21T10:09:04.526 INFO:tasks.workunit.client.0.smithi153.stdout:9/904: dread - d1/d8a/fd8 zero size 2024-03-21T10:09:04.527 INFO:tasks.workunit.client.0.smithi153.stdout:1/719: fdatasync d6/d10/d35/da5/d4f/d9d/dc7/dcc/f6a 0 2024-03-21T10:09:04.527 INFO:tasks.workunit.client.1.smithi174.stdout:1/786: fsync d4/d67/fde 0 2024-03-21T10:09:04.529 INFO:tasks.workunit.client.0.smithi153.stdout:3/924: rmdir dd/d1e/d2f/d35/dd0 39 2024-03-21T10:09:04.529 INFO:tasks.workunit.client.1.smithi174.stdout:6/542: fdatasync d0/d3/d5/df/d16/f88 0 2024-03-21T10:09:04.530 INFO:tasks.workunit.client.1.smithi174.stdout:0/666: dread d9/d21/d31/f34 [0,4194304] 0 2024-03-21T10:09:04.532 INFO:tasks.workunit.client.1.smithi174.stdout:7/609: rmdir d0/d4/d10/d44/d81 39 2024-03-21T10:09:04.534 INFO:tasks.workunit.client.1.smithi174.stdout:4/792: link d4/c8b d4/d21/d24/d28/de9/d6a/dab/da8/c105 0 2024-03-21T10:09:04.537 INFO:tasks.workunit.client.1.smithi174.stdout:8/643: read d9/d2a/d39/d76/d19/d27/d60/fa8 [1510900,50676] 0 2024-03-21T10:09:04.537 INFO:tasks.workunit.client.1.smithi174.stdout:0/667: unlink d9/d35/d4a/d67/d89/fd3 0 2024-03-21T10:09:04.539 INFO:tasks.workunit.client.1.smithi174.stdout:1/787: creat d4/d5/d6c/d10e/f119 x:0 0 0 2024-03-21T10:09:04.540 INFO:tasks.workunit.client.1.smithi174.stdout:8/644: chown d9/d2a/d39/l4b 14 1 2024-03-21T10:09:04.541 INFO:tasks.workunit.client.0.smithi153.stdout:3/925: truncate dd/d41/dab/f11c 437548 0 2024-03-21T10:09:04.542 INFO:tasks.workunit.client.1.smithi174.stdout:1/788: fsync d4/d7/d68/f10f 0 2024-03-21T10:09:04.545 INFO:tasks.workunit.client.1.smithi174.stdout:1/789: chown d4/d2d/cb0 4229 1 2024-03-21T10:09:04.546 INFO:tasks.workunit.client.0.smithi153.stdout:3/926: chown dd/d1e/d2f/d35/l40 0 1 2024-03-21T10:09:04.547 INFO:tasks.workunit.client.0.smithi153.stdout:5/832: dwrite d4/d5/f7 [0,4194304] 0 2024-03-21T10:09:04.549 INFO:tasks.workunit.client.0.smithi153.stdout:2/910: write dd/d15/d18/d2c/d4b/f66 [116075,56046] 0 2024-03-21T10:09:04.549 INFO:tasks.workunit.client.0.smithi153.stdout:9/905: mkdir d1/d43/d44/d46/d9e/d24/d12c 0 2024-03-21T10:09:04.549 INFO:tasks.workunit.client.1.smithi174.stdout:6/543: truncate d0/d3/d5/df/d16/f17 3983753 0 2024-03-21T10:09:04.552 INFO:tasks.workunit.client.1.smithi174.stdout:0/668: mknod d9/d21/d31/d3c/d72/ce8 0 2024-03-21T10:09:04.553 INFO:tasks.workunit.client.0.smithi153.stdout:3/927: stat dd/d1e/d3d/d94/d73/d7f 0 2024-03-21T10:09:04.555 INFO:tasks.workunit.client.1.smithi174.stdout:0/669: chown d9/d21/d31/d5c/f7c 11 1 2024-03-21T10:09:04.556 INFO:tasks.workunit.client.1.smithi174.stdout:0/670: chown d9/d21/d31/d3c/d72 70466479 1 2024-03-21T10:09:04.557 INFO:tasks.workunit.client.1.smithi174.stdout:4/793: symlink d4/d21/d24/l106 0 2024-03-21T10:09:04.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:04 smithi153 ceph-mon[29117]: pgmap v18: 65 pgs: 65 active+clean; 2.9 GiB data, 8.4 GiB used, 528 GiB / 536 GiB avail; 73 MiB/s rd, 231 MiB/s wr, 439 op/s 2024-03-21T10:09:04.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:04 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:09:04.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:04 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:09:04.558 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:04 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:09:04.558 INFO:tasks.workunit.client.0.smithi153.stdout:3/928: stat dd/d1e/l28 0 2024-03-21T10:09:04.559 INFO:tasks.workunit.client.1.smithi174.stdout:9/719: dwrite d2/d11/d60/db2/ffa [4194304,4194304] 0 2024-03-21T10:09:04.560 INFO:tasks.workunit.client.1.smithi174.stdout:1/790: mknod d4/d5/d6c/d10e/c11a 0 2024-03-21T10:09:04.562 INFO:tasks.workunit.client.1.smithi174.stdout:4/794: fdatasync d4/d21/d24/d28/d38/f5f 0 2024-03-21T10:09:04.562 INFO:tasks.workunit.client.1.smithi174.stdout:6/544: creat d0/d3/d2a/d56/d5e/fa7 x:0 0 0 2024-03-21T10:09:04.563 INFO:tasks.workunit.client.0.smithi153.stdout:1/720: dread d6/d3a/d3c/d50/f67 [0,4194304] 0 2024-03-21T10:09:04.564 INFO:tasks.workunit.client.1.smithi174.stdout:4/795: fdatasync d4/d19/fcd 0 2024-03-21T10:09:04.564 INFO:tasks.workunit.client.0.smithi153.stdout:5/833: creat d4/d8/d5a/d92/d9f/f116 x:0 0 0 2024-03-21T10:09:04.566 INFO:tasks.workunit.client.1.smithi174.stdout:7/610: getdents d0/d4/d38/da9 0 2024-03-21T10:09:04.567 INFO:tasks.workunit.client.0.smithi153.stdout:5/834: stat d4/d5/df/d67/da0/db3/f8c 0 2024-03-21T10:09:04.570 INFO:tasks.workunit.client.1.smithi174.stdout:6/545: unlink d0/d3/d5/d6/d42/f69 0 2024-03-21T10:09:04.570 INFO:tasks.workunit.client.0.smithi153.stdout:7/866: dread d4/d14/dd0/d83/fdf [0,4194304] 0 2024-03-21T10:09:04.571 INFO:tasks.workunit.client.1.smithi174.stdout:0/671: link d9/d21/d31/d5c/d86/fbf d9/d21/d81/fe9 0 2024-03-21T10:09:04.574 INFO:tasks.workunit.client.0.smithi153.stdout:1/721: fsync d6/d10/d35/da5/d4f/d9d/dc7/dcc/f6a 0 2024-03-21T10:09:04.576 INFO:tasks.workunit.client.1.smithi174.stdout:7/611: creat d0/d53/d5f/fd3 x:0 0 0 2024-03-21T10:09:04.577 INFO:tasks.workunit.client.1.smithi174.stdout:6/546: mkdir d0/d3/da8 0 2024-03-21T10:09:04.578 INFO:tasks.workunit.client.1.smithi174.stdout:4/796: fdatasync d4/d21/d24/d28/de9/d3a/d7a/d9d/f104 0 2024-03-21T10:09:04.583 INFO:tasks.workunit.client.0.smithi153.stdout:1/722: dread - d6/d10/d2c/fe3 zero size 2024-03-21T10:09:04.583 INFO:tasks.workunit.client.0.smithi153.stdout:9/906: truncate d1/d43/d44/d46/d9e/d24/d41/fa3 268259 0 2024-03-21T10:09:04.585 INFO:tasks.workunit.client.1.smithi174.stdout:0/672: rename d9/d35/dd2/fc5 to d9/d21/d31/d9f/d61/fea 0 2024-03-21T10:09:04.585 INFO:tasks.workunit.client.1.smithi174.stdout:6/547: mkdir d0/d3/d5/df/d16/da9 0 2024-03-21T10:09:04.586 INFO:tasks.workunit.client.1.smithi174.stdout:0/673: chown d9/d1c/f1e 446 1 2024-03-21T10:09:04.589 INFO:tasks.workunit.client.0.smithi153.stdout:0/775: dread d4/db/f29 [0,4194304] 0 2024-03-21T10:09:04.592 INFO:tasks.workunit.client.1.smithi174.stdout:0/674: chown d9/d21/d31/c6b 185303 1 2024-03-21T10:09:04.592 INFO:tasks.workunit.client.1.smithi174.stdout:4/797: creat d4/d21/d24/d28/de9/d3a/d7a/d9d/da3/f107 x:0 0 0 2024-03-21T10:09:04.595 INFO:tasks.workunit.client.1.smithi174.stdout:4/798: mkdir d4/d21/d24/d28/d79/d108 0 2024-03-21T10:09:04.596 INFO:tasks.workunit.client.1.smithi174.stdout:0/675: mkdir d9/d21/d31/d3c/deb 0 2024-03-21T10:09:04.598 INFO:tasks.workunit.client.0.smithi153.stdout:5/835: fsync d4/d5/df/d67/d86/d96/deb/d56/d105/f10b 0 2024-03-21T10:09:04.600 INFO:tasks.workunit.client.0.smithi153.stdout:7/867: dwrite d4/d11/d56/db1/fba [0,4194304] 0 2024-03-21T10:09:04.602 INFO:tasks.workunit.client.0.smithi153.stdout:1/723: mkdir d6/d10/d35/da5/dea 0 2024-03-21T10:09:04.604 INFO:tasks.workunit.client.1.smithi174.stdout:0/676: symlink d9/d21/d31/d9f/d79/d82/dbb/lec 0 2024-03-21T10:09:04.605 INFO:tasks.workunit.client.1.smithi174.stdout:9/720: write d2/d11/f90 [902485,43470] 0 2024-03-21T10:09:04.609 INFO:tasks.workunit.client.0.smithi153.stdout:5/836: fdatasync d4/d8/d5a/d5d/f55 0 2024-03-21T10:09:04.610 INFO:tasks.workunit.client.1.smithi174.stdout:9/721: chown d2/d11/f2c 439964 1 2024-03-21T10:09:04.613 INFO:tasks.workunit.client.1.smithi174.stdout:9/722: readlink d2/d11/d1d/d4b/l71 0 2024-03-21T10:09:04.613 INFO:tasks.workunit.client.0.smithi153.stdout:2/911: dwrite dd/d15/d18/d3a/f133 [0,4194304] 0 2024-03-21T10:09:04.614 INFO:tasks.workunit.client.1.smithi174.stdout:0/677: rename l1 to d9/d21/d31/d5c/led 0 2024-03-21T10:09:04.614 INFO:tasks.workunit.client.0.smithi153.stdout:3/929: write dd/d1e/d2f/d35/dd0/d109/f121 [515671,16642] 0 2024-03-21T10:09:04.614 INFO:tasks.workunit.client.0.smithi153.stdout:5/837: write d4/d8/d5a/d5d/dd9/f3f [2015007,55930] 0 2024-03-21T10:09:04.616 INFO:tasks.workunit.client.1.smithi174.stdout:1/791: dwrite d4/d7/d15/d4f/f83 [0,4194304] 0 2024-03-21T10:09:04.616 INFO:tasks.workunit.client.0.smithi153.stdout:4/889: dread d11/d18/f61 [0,4194304] 0 2024-03-21T10:09:04.618 INFO:tasks.workunit.client.0.smithi153.stdout:2/912: stat dd/d15/d18 0 2024-03-21T10:09:04.619 INFO:tasks.workunit.client.0.smithi153.stdout:5/838: write d4/ff8 [461000,52128] 0 2024-03-21T10:09:04.623 INFO:tasks.workunit.client.0.smithi153.stdout:7/868: mkdir d4/d14/d120 0 2024-03-21T10:09:04.624 INFO:tasks.workunit.client.0.smithi153.stdout:3/930: truncate dd/d1e/d2f/f36 4402026 0 2024-03-21T10:09:04.625 INFO:tasks.workunit.client.1.smithi174.stdout:1/792: dread - d4/d67/ff9 zero size 2024-03-21T10:09:04.627 INFO:tasks.workunit.client.1.smithi174.stdout:2/657: dread d5/d25/f7e [0,4194304] 0 2024-03-21T10:09:04.627 INFO:tasks.workunit.client.0.smithi153.stdout:1/724: rename d6/d10/d35/da5/l99 to d6/d10/d35/da5/d4f/leb 0 2024-03-21T10:09:04.630 INFO:tasks.workunit.client.0.smithi153.stdout:3/931: readlink dd/d1e/d2f/d35/l76 0 2024-03-21T10:09:04.632 INFO:tasks.workunit.client.1.smithi174.stdout:7/612: dwrite d0/d4/d10/d1b/f3e [0,4194304] 0 2024-03-21T10:09:04.635 INFO:tasks.workunit.client.0.smithi153.stdout:5/839: creat d4/d8/d5a/d92/f117 x:0 0 0 2024-03-21T10:09:04.640 INFO:tasks.workunit.client.0.smithi153.stdout:3/932: unlink lb 0 2024-03-21T10:09:04.640 INFO:tasks.workunit.client.1.smithi174.stdout:0/678: dread d9/d21/d31/d3c/d72/fb1 [0,4194304] 0 2024-03-21T10:09:04.640 INFO:tasks.workunit.client.1.smithi174.stdout:7/613: fdatasync d0/f52 0 2024-03-21T10:09:04.641 INFO:tasks.workunit.client.0.smithi153.stdout:5/840: creat d4/d8/d5a/d92/d9f/f118 x:0 0 0 2024-03-21T10:09:04.644 INFO:tasks.workunit.client.1.smithi174.stdout:6/548: dread d0/d3/d5/f1c [0,4194304] 0 2024-03-21T10:09:04.647 INFO:tasks.workunit.client.0.smithi153.stdout:2/913: rename dd/d15/cad to dd/d15/d18/d2c/d4b/d56/d63/d8d/d9b/dc3/df5/dae/d50/d59/c13b 0 2024-03-21T10:09:04.654 INFO:tasks.workunit.client.0.smithi153.stdout:9/907: dwrite d1/d4/d5f/fe6 [0,4194304] 0 2024-03-21T10:09:04.654 INFO:tasks.workunit.client.0.smithi153.stdout:3/933: fdatasync dd/d41/d80/dbb/fc1 0 2024-03-21T10:09:04.654 INFO:tasks.workunit.client.1.smithi174.stdout:9/723: dwrite d2/d24/f74 [0,4194304] 0 2024-03-21T10:09:04.654 INFO:tasks.workunit.client.1.smithi174.stdout:1/793: dwrite d4/d5/d6c/fea [0,4194304] 0 2024-03-21T10:09:04.654 INFO:tasks.workunit.client.1.smithi174.stdout:0/679: rmdir d9/d21/d31/d3c/deb 0 2024-03-21T10:09:04.655 INFO:tasks.workunit.client.1.smithi174.stdout:2/658: truncate d5/d25/d27/d2d/d45/d5b/d2f/d4b/f78 1553031 0 2024-03-21T10:09:04.655 INFO:tasks.workunit.client.0.smithi153.stdout:5/841: chown d4/d8/d5a/d92/d9f/cd3 748133 1 2024-03-21T10:09:04.660 INFO:tasks.workunit.client.1.smithi174.stdout:3/665: sync 2024-03-21T10:09:04.663 INFO:tasks.workunit.client.0.smithi153.stdout:7/869: rename d4/d2a/d40/d50 to d4/d19/d4d/d10e/d121 0 2024-03-21T10:09:04.667 INFO:tasks.workunit.client.0.smithi153.stdout:1/725: dwrite d6/d32/f5a [0,4194304] 0 2024-03-21T10:09:04.667 INFO:tasks.workunit.client.1.smithi174.stdout:2/659: mkdir d5/d25/d27/d2d/d45/d5b/d63/de1 0 2024-03-21T10:09:04.669 INFO:tasks.workunit.client.0.smithi153.stdout:2/914: fdatasync dd/d15/d18/d2c/d4b/fce 0 2024-03-21T10:09:04.669 INFO:tasks.workunit.client.0.smithi153.stdout:1/726: chown d6/d13/f2b 5681 1 2024-03-21T10:09:04.672 INFO:tasks.workunit.client.0.smithi153.stdout:3/934: symlink dd/d1e/d2f/d35/dd0/d109/l142 0 2024-03-21T10:09:04.674 INFO:tasks.workunit.client.0.smithi153.stdout:7/870: chown d4/d2a/d40/f4c 510879 1 2024-03-21T10:09:04.677 INFO:tasks.workunit.client.0.smithi153.stdout:1/727: creat d6/d13/d8e/fec x:0 0 0 2024-03-21T10:09:04.678 INFO:tasks.workunit.client.1.smithi174.stdout:3/666: mkdir d0/d12/d14/d1a/d44/d94/dd9 0 2024-03-21T10:09:04.682 INFO:tasks.workunit.client.1.smithi174.stdout:7/614: dwrite d0/d4/d10/d44/f8e [0,4194304] 0 2024-03-21T10:09:04.683 INFO:tasks.workunit.client.0.smithi153.stdout:2/915: rmdir dd/d15/d18/d3a/d3d 39 2024-03-21T10:09:04.686 INFO:tasks.workunit.client.1.smithi174.stdout:9/724: dread d2/f8e [0,4194304] 0 2024-03-21T10:09:04.686 INFO:tasks.workunit.client.1.smithi174.stdout:9/725: chown d2/f84 1378407 1 2024-03-21T10:09:04.690 INFO:tasks.workunit.client.0.smithi153.stdout:4/890: write d11/d18/d7e/ffb [560598,77846] 0 2024-03-21T10:09:04.691 INFO:tasks.workunit.client.1.smithi174.stdout:1/794: creat d4/d7/d28/d59/d98/f11b x:0 0 0 2024-03-21T10:09:04.692 INFO:tasks.workunit.client.0.smithi153.stdout:9/908: link d1/d4/lfd d1/d43/d44/d46/d9e/d2e/d8f/l12d 0 2024-03-21T10:09:04.697 INFO:tasks.workunit.client.0.smithi153.stdout:7/871: mknod d4/d2a/d100/c122 0 2024-03-21T10:09:04.701 INFO:tasks.workunit.client.0.smithi153.stdout:5/842: getdents d4/d8/d5a/d5d/dd9/d44 0 2024-03-21T10:09:04.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:04 smithi174 ceph-mon[37496]: pgmap v18: 65 pgs: 65 active+clean; 2.9 GiB data, 8.4 GiB used, 528 GiB / 536 GiB avail; 73 MiB/s rd, 231 MiB/s wr, 439 op/s 2024-03-21T10:09:04.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:04 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:09:04.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:04 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:09:04.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:04 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:09:04.703 INFO:tasks.workunit.client.1.smithi174.stdout:1/795: rename d4/d7/d33/d43/ff5 to d4/d5/f11c 0 2024-03-21T10:09:04.704 INFO:tasks.workunit.client.1.smithi174.stdout:0/680: getdents d9/d35/d4a/d67 0 2024-03-21T10:09:04.705 INFO:tasks.workunit.client.1.smithi174.stdout:1/796: chown d4/d7/d15/d4f 41912 1 2024-03-21T10:09:04.707 INFO:tasks.workunit.client.0.smithi153.stdout:4/891: read d11/d18/f4b [7147954,9985] 0 2024-03-21T10:09:04.707 INFO:tasks.workunit.client.1.smithi174.stdout:7/615: rmdir d0/db6 0 2024-03-21T10:09:04.708 INFO:tasks.workunit.client.0.smithi153.stdout:9/909: fdatasync d1/d43/d44/fff 0 2024-03-21T10:09:04.708 INFO:tasks.workunit.client.1.smithi174.stdout:7/616: stat d0/d53/f80 0 2024-03-21T10:09:04.716 INFO:tasks.workunit.client.0.smithi153.stdout:3/935: link dd/c11 dd/d1e/d3d/dc9/d13b/c143 0 2024-03-21T10:09:04.718 INFO:tasks.workunit.client.0.smithi153.stdout:4/892: mkdir d11/d18/d6d/dd6/ddd/d119 0 2024-03-21T10:09:04.720 INFO:tasks.workunit.client.0.smithi153.stdout:2/916: creat dd/d15/f13c x:0 0 0 2024-03-21T10:09:04.722 INFO:tasks.workunit.client.0.smithi153.stdout:5/843: readlink d4/d8/d5a/d5d/dd9/d44/de1/dfb/d108/ld5 0 2024-03-21T10:09:04.723 INFO:tasks.workunit.client.0.smithi153.stdout:1/728: link d6/d10/d35/d5b/fad d6/d3a/d3c/d50/da8/fed 0 2024-03-21T10:09:04.726 INFO:tasks.workunit.client.1.smithi174.stdout:0/681: read d9/d35/d4a/d67/fdd [2411211,62470] 0 2024-03-21T10:09:04.727 INFO:tasks.workunit.client.0.smithi153.stdout:7/872: rename d4/d2a/dc6/dfb/lcd to d4/d19/d4d/l123 0 2024-03-21T10:09:04.727 INFO:tasks.workunit.client.0.smithi153.stdout:4/893: symlink d11/d2d/d3d/da1/l11a 0 2024-03-21T10:09:04.727 INFO:tasks.workunit.client.0.smithi153.stdout:7/873: write d4/d2a/d3b/d60/f7c [1913225,2217] 0 2024-03-21T10:09:04.727 INFO:tasks.workunit.client.0.smithi153.stdout:5/844: dread - d4/d5/f85 zero size 2024-03-21T10:09:04.729 INFO:tasks.workunit.client.1.smithi174.stdout:0/682: read - d9/d35/d4a/f9e zero size 2024-03-21T10:09:04.730 INFO:tasks.workunit.client.1.smithi174.stdout:0/683: stat d9/d1c/f7d 0 2024-03-21T10:09:04.731 INFO:tasks.workunit.client.0.smithi153.stdout:9/910: getdents d1/d43/d44/d46/d9e 0 2024-03-21T10:09:04.731 INFO:tasks.workunit.client.1.smithi174.stdout:9/726: dread d2/d24/fd5 [0,4194304] 0 2024-03-21T10:09:04.733 INFO:tasks.workunit.client.0.smithi153.stdout:5/845: rmdir d4/d8/d5a/d5d 39 2024-03-21T10:09:04.733 INFO:tasks.workunit.client.1.smithi174.stdout:0/684: write d9/d21/d31/d5c/dd5/fd6 [982243,45098] 0 2024-03-21T10:09:04.734 INFO:tasks.workunit.client.1.smithi174.stdout:9/727: creat d2/d11/d60/db2/ffd x:0 0 0 2024-03-21T10:09:04.736 INFO:tasks.workunit.client.0.smithi153.stdout:4/894: creat d11/d2d/da3/d103/f11b x:0 0 0 2024-03-21T10:09:04.736 INFO:tasks.workunit.client.1.smithi174.stdout:0/685: creat d9/d21/d75/da1/fee x:0 0 0 2024-03-21T10:09:04.736 INFO:tasks.workunit.client.0.smithi153.stdout:7/874: mkdir d4/d2a/d100/d124 0 2024-03-21T10:09:04.737 INFO:tasks.workunit.client.1.smithi174.stdout:0/686: stat d9/d21/d81/lb3 0 2024-03-21T10:09:04.737 INFO:tasks.workunit.client.1.smithi174.stdout:9/728: read - d2/d11/d1d/d4b/d72/fbb zero size 2024-03-21T10:09:04.737 INFO:tasks.workunit.client.0.smithi153.stdout:5/846: truncate d4/d8/d5c/d90/f109 1674982 0 2024-03-21T10:09:04.738 INFO:tasks.workunit.client.0.smithi153.stdout:2/917: mkdir dd/d15/d18/d3a/d3d/d125/d13d 0 2024-03-21T10:09:04.740 INFO:tasks.workunit.client.1.smithi174.stdout:9/729: creat d2/d24/d2e/ffe x:0 0 0 2024-03-21T10:09:04.740 INFO:tasks.workunit.client.0.smithi153.stdout:9/911: creat d1/d3b/df8/f12e x:0 0 0 2024-03-21T10:09:04.741 INFO:tasks.workunit.client.1.smithi174.stdout:0/687: link d9/d21/d31/d3c/f83 d9/d21/d81/fef 0 2024-03-21T10:09:04.743 INFO:tasks.workunit.client.0.smithi153.stdout:4/895: chown d11/d2d/f36 4898 1 2024-03-21T10:09:04.744 INFO:tasks.workunit.client.1.smithi174.stdout:0/688: symlink d9/d1c/d76/d87/lf0 0 2024-03-21T10:09:04.745 INFO:tasks.workunit.client.0.smithi153.stdout:4/896: chown d11/d2d/d6b/caa 190606106 1 2024-03-21T10:09:04.745 INFO:tasks.workunit.client.0.smithi153.stdout:9/912: rmdir d1/d3b/daa/df9 39 2024-03-21T10:09:04.747 INFO:tasks.workunit.client.1.smithi174.stdout:0/689: stat d9/d21/d31/d5c/d92/fdb 0 2024-03-21T10:09:04.749 INFO:tasks.workunit.client.0.smithi153.stdout:5/847: chown d4/d5/df/d67/da0/db3/f74 5840 1 2024-03-21T10:09:04.750 INFO:tasks.workunit.client.0.smithi153.stdout:2/918: fsync dd/d15/d18/d2c/d4b/d56/d63/d8d/d9b/dc3/fd5 0 2024-03-21T10:09:04.751 INFO:tasks.workunit.client.0.smithi153.stdout:4/897: truncate d11/d2d/d6b/d9c/fe1 1356196 0 2024-03-21T10:09:04.757 INFO:tasks.workunit.client.0.smithi153.stdout:5/848: write d4/d8/d5c/d90/f113 [984035,42210] 0 2024-03-21T10:09:04.757 INFO:tasks.workunit.client.1.smithi174.stdout:1/797: getdents d4/d5 0 2024-03-21T10:09:04.762 INFO:tasks.workunit.client.1.smithi174.stdout:9/730: dwrite d2/d11/d1d/d4b/ff5 [0,4194304] 0 2024-03-21T10:09:04.764 INFO:tasks.workunit.client.1.smithi174.stdout:1/798: write d4/d7/d15/f39 [1124406,70865] 0 2024-03-21T10:09:04.767 INFO:tasks.workunit.client.1.smithi174.stdout:3/667: dwrite d0/d12/d14/d1a/f34 [0,4194304] 0 2024-03-21T10:09:04.770 INFO:tasks.workunit.client.0.smithi153.stdout:7/875: dwrite d4/d19/fb6 [0,4194304] 0 2024-03-21T10:09:04.770 INFO:tasks.workunit.client.0.smithi153.stdout:9/913: write d1/d4/d5f/df3/f10c [4711062,10935] 0 2024-03-21T10:09:04.776 INFO:tasks.workunit.client.0.smithi153.stdout:2/919: symlink dd/d15/d18/d2c/d4b/d56/d63/d8d/d9b/dc3/df5/dae/l13e 0 2024-03-21T10:09:04.776 INFO:tasks.workunit.client.0.smithi153.stdout:5/849: creat d4/d5/df/d67/d86/d96/deb/dc1/f119 x:0 0 0 2024-03-21T10:09:04.778 INFO:tasks.workunit.client.0.smithi153.stdout:5/850: fdatasync d4/d8/f26 0 2024-03-21T10:09:04.779 INFO:tasks.workunit.client.1.smithi174.stdout:3/668: rmdir d0/d12/d14/d1a/d44/d65/d7e/dac 39 2024-03-21T10:09:04.782 INFO:tasks.workunit.client.1.smithi174.stdout:7/617: dwrite d0/f88 [0,4194304] 0 2024-03-21T10:09:04.782 INFO:tasks.workunit.client.1.smithi174.stdout:5/747: sync 2024-03-21T10:09:04.784 INFO:tasks.workunit.client.0.smithi153.stdout:3/936: truncate dd/d1e/d3d/d94/f115 992612 0 2024-03-21T10:09:04.787 INFO:tasks.workunit.client.0.smithi153.stdout:7/876: link d4/d2a/f2d d4/d2a/d3b/d81/df6/f125 0 2024-03-21T10:09:04.789 INFO:tasks.workunit.client.0.smithi153.stdout:9/914: write d1/f97 [1257245,116311] 0 2024-03-21T10:09:04.789 INFO:tasks.workunit.client.1.smithi174.stdout:7/618: symlink d0/d4/d38/da9/ld4 0 2024-03-21T10:09:04.791 INFO:tasks.workunit.client.0.smithi153.stdout:5/851: rename d4/cee to d4/d5/df/d67/d86/d96/deb/df0/d10a/c11a 0 2024-03-21T10:09:04.794 INFO:tasks.workunit.client.0.smithi153.stdout:7/877: symlink d4/d19/d4d/l126 0 2024-03-21T10:09:04.797 INFO:tasks.workunit.client.0.smithi153.stdout:9/915: rename d1/cd to d1/d3b/df8/c12f 0 2024-03-21T10:09:04.799 INFO:tasks.workunit.client.0.smithi153.stdout:3/937: mkdir dd/d1e/d6b/d137/d144 0 2024-03-21T10:09:04.800 INFO:tasks.workunit.client.1.smithi174.stdout:3/669: dread d0/d12/f17 [0,4194304] 0 2024-03-21T10:09:04.801 INFO:tasks.workunit.client.0.smithi153.stdout:5/852: chown d4/d8/d5a/d5d/dd9/d38/c7f 516051555 1 2024-03-21T10:09:04.801 INFO:tasks.workunit.client.0.smithi153.stdout:7/878: unlink d4/d14/d2f/d78/f58 0 2024-03-21T10:09:04.805 INFO:tasks.workunit.client.0.smithi153.stdout:9/916: chown d1/d43/d44/d46/d9e/d2e/d67/ld1 482 1 2024-03-21T10:09:04.807 INFO:tasks.workunit.client.0.smithi153.stdout:5/853: creat d4/d8/d5a/d5d/dd9/d44/de1/dfb/f11b x:0 0 0 2024-03-21T10:09:04.808 INFO:tasks.workunit.client.1.smithi174.stdout:3/670: symlink d0/d12/d14/d1a/d44/d94/dd9/lda 0 2024-03-21T10:09:04.809 INFO:tasks.workunit.client.0.smithi153.stdout:9/917: read d1/d4/fc [4127060,115958] 0 2024-03-21T10:09:04.810 INFO:tasks.workunit.client.1.smithi174.stdout:2/660: dwrite d5/d25/d27/d2d/d45/f6e [0,4194304] 0 2024-03-21T10:09:04.812 INFO:tasks.workunit.client.1.smithi174.stdout:3/671: dread - d0/d12/d14/d1a/d44/d65/d7e/fd5 zero size 2024-03-21T10:09:04.813 INFO:tasks.workunit.client.0.smithi153.stdout:7/879: mknod d4/d14/d2f/c127 0 2024-03-21T10:09:04.815 INFO:tasks.workunit.client.1.smithi174.stdout:1/799: dwrite d4/d7/d33/d43/f47 [4194304,4194304] 0 2024-03-21T10:09:04.819 INFO:tasks.workunit.client.1.smithi174.stdout:2/661: write d5/d25/d27/d2d/d45/d5b/d63/f48 [5455841,125691] 0 2024-03-21T10:09:04.820 INFO:tasks.workunit.client.0.smithi153.stdout:5/854: read d4/d5/f1a [1687735,53621] 0 2024-03-21T10:09:04.820 INFO:tasks.workunit.client.1.smithi174.stdout:1/800: chown d4/d7/d15/d53/d78/fab 201445 1 2024-03-21T10:09:04.823 INFO:tasks.workunit.client.1.smithi174.stdout:0/690: dread d9/d1c/f7d [4194304,4194304] 0 2024-03-21T10:09:04.827 INFO:tasks.workunit.client.0.smithi153.stdout:7/880: rename d4/d2a/dc6/dfb/d9a/dd2/db9/fbf to d4/d14/d2f/d78/f128 0 2024-03-21T10:09:04.827 INFO:tasks.workunit.client.1.smithi174.stdout:0/691: chown d9/d21/d31/d3c/l4d 421168980 1 2024-03-21T10:09:04.831 INFO:tasks.workunit.client.1.smithi174.stdout:0/692: fsync d9/d21/d31/d9f/d61/fea 0 2024-03-21T10:09:04.832 INFO:tasks.workunit.client.0.smithi153.stdout:5/855: rmdir d4/d5/df/d5e/d110 0 2024-03-21T10:09:04.832 INFO:tasks.workunit.client.1.smithi174.stdout:9/731: write d2/d11/d1d/f79 [503168,3365] 0 2024-03-21T10:09:04.834 INFO:tasks.workunit.client.1.smithi174.stdout:1/801: creat d4/d7/d28/f11d x:0 0 0 2024-03-21T10:09:04.835 INFO:tasks.workunit.client.0.smithi153.stdout:4/898: dread d11/d18/f21 [0,4194304] 0 2024-03-21T10:09:04.838 INFO:tasks.workunit.client.0.smithi153.stdout:5/856: fsync d4/d5/d16/f19 0 2024-03-21T10:09:04.842 INFO:tasks.workunit.client.1.smithi174.stdout:1/802: mkdir d4/d7/d28/d59/d98/d11e 0 2024-03-21T10:09:04.844 INFO:tasks.workunit.client.1.smithi174.stdout:9/732: rename d2/d11/d1d/d5e/fc3 to d2/d24/d2e/df6/fff 0 2024-03-21T10:09:04.844 INFO:tasks.workunit.client.1.smithi174.stdout:3/672: dread d0/d12/d14/d1a/d44/d94/f9d [0,4194304] 0 2024-03-21T10:09:04.847 INFO:tasks.workunit.client.1.smithi174.stdout:1/803: mknod d4/d7/d32/d7d/df1/c11f 0 2024-03-21T10:09:04.849 INFO:tasks.workunit.client.0.smithi153.stdout:5/857: mknod d4/d5/df/d67/c11c 0 2024-03-21T10:09:04.850 INFO:tasks.workunit.client.0.smithi153.stdout:7/881: rename d4/d2a/d40/cce to d4/d19/d4d/c129 0 2024-03-21T10:09:04.851 INFO:tasks.workunit.client.1.smithi174.stdout:1/804: fsync d4/d7/d68/d89/fc1 0 2024-03-21T10:09:04.852 INFO:tasks.workunit.client.1.smithi174.stdout:5/748: dwrite d1/d16/d2e/d66/d8b/d85/f8e [0,4194304] 0 2024-03-21T10:09:04.853 INFO:tasks.workunit.client.1.smithi174.stdout:6/549: sync 2024-03-21T10:09:04.853 INFO:tasks.workunit.client.1.smithi174.stdout:4/799: sync 2024-03-21T10:09:04.854 INFO:tasks.workunit.client.1.smithi174.stdout:8/645: sync 2024-03-21T10:09:04.854 INFO:tasks.workunit.client.1.smithi174.stdout:9/733: creat d2/d24/d2e/f100 x:0 0 0 2024-03-21T10:09:04.854 INFO:tasks.workunit.client.0.smithi153.stdout:3/938: write dd/f48 [4468743,63105] 0 2024-03-21T10:09:04.856 INFO:tasks.workunit.client.0.smithi153.stdout:9/918: dwrite d1/d43/d44/d46/d9e/d24/fb4 [0,4194304] 0 2024-03-21T10:09:04.856 INFO:tasks.workunit.client.1.smithi174.stdout:1/805: dread - d4/d7/d68/f10f zero size 2024-03-21T10:09:04.861 INFO:tasks.workunit.client.1.smithi174.stdout:4/800: mknod d4/d19/c109 0 2024-03-21T10:09:04.861 INFO:tasks.workunit.client.1.smithi174.stdout:4/801: chown d4/d21/d24/d28/de9/d3a 25 1 2024-03-21T10:09:04.862 INFO:tasks.workunit.client.0.smithi153.stdout:9/919: read d1/d4/d5f/fce [4130405,126349] 0 2024-03-21T10:09:04.862 INFO:tasks.workunit.client.1.smithi174.stdout:7/619: dread d0/d53/f80 [0,4194304] 0 2024-03-21T10:09:04.863 INFO:tasks.workunit.client.1.smithi174.stdout:2/662: write d5/d25/d27/f39 [3929328,99031] 0 2024-03-21T10:09:04.863 INFO:tasks.workunit.client.1.smithi174.stdout:5/749: rmdir d1/d16/d2e 39 2024-03-21T10:09:04.866 INFO:tasks.workunit.client.1.smithi174.stdout:6/550: fsync d0/d3/d5/d21/f6c 0 2024-03-21T10:09:04.870 INFO:tasks.workunit.client.0.smithi153.stdout:5/858: mknod d4/d8/d5a/c11d 0 2024-03-21T10:09:04.874 INFO:tasks.workunit.client.0.smithi153.stdout:2/920: dread dd/d15/d18/d2c/d4b/d56/d63/d8d/f110 [4194304,4194304] 0 2024-03-21T10:09:04.875 INFO:tasks.workunit.client.1.smithi174.stdout:4/802: mkdir d4/d21/d24/d28/de9/d3a/dd2/d10a 0 2024-03-21T10:09:04.876 INFO:tasks.workunit.client.1.smithi174.stdout:1/806: fdatasync d4/d7/d15/f1f 0 2024-03-21T10:09:04.879 INFO:tasks.workunit.client.1.smithi174.stdout:0/693: write d9/f19 [4558009,95852] 0 2024-03-21T10:09:04.883 INFO:tasks.workunit.client.1.smithi174.stdout:1/807: fsync d4/d7/d33/d43/d77/d87/d92/f114 0 2024-03-21T10:09:04.888 INFO:tasks.workunit.client.1.smithi174.stdout:3/673: write d0/d12/d14/d4c/f79 [1138341,28575] 0 2024-03-21T10:09:04.893 INFO:tasks.workunit.client.1.smithi174.stdout:4/803: creat d4/d21/d24/d28/de9/d6a/dab/f10b x:0 0 0 2024-03-21T10:09:04.895 INFO:tasks.workunit.client.1.smithi174.stdout:7/620: creat d0/d53/fd5 x:0 0 0 2024-03-21T10:09:04.896 INFO:tasks.workunit.client.1.smithi174.stdout:0/694: chown d9/d21/c2e 280241812 1 2024-03-21T10:09:04.902 INFO:tasks.workunit.client.1.smithi174.stdout:1/808: mkdir d4/d7/d32/d7d/d120 0 2024-03-21T10:09:04.902 INFO:tasks.workunit.client.1.smithi174.stdout:5/750: link d1/ce d1/d49/c10d 0 2024-03-21T10:09:04.903 INFO:tasks.workunit.client.1.smithi174.stdout:8/646: dwrite f1 [0,4194304] 0 2024-03-21T10:09:04.904 INFO:tasks.workunit.client.1.smithi174.stdout:4/804: rmdir d4/d21/d24/d28/de9/d6a 39 2024-03-21T10:09:04.904 INFO:tasks.workunit.client.0.smithi153.stdout:9/920: dwrite d1/d43/d44/f33 [4194304,4194304] 0 2024-03-21T10:09:04.905 INFO:tasks.workunit.client.1.smithi174.stdout:5/751: dread - d1/d2f/dc1/ffc zero size 2024-03-21T10:09:04.908 INFO:tasks.workunit.client.0.smithi153.stdout:2/921: dwrite dd/d15/d3c/fe6 [4194304,4194304] 0 2024-03-21T10:09:04.909 INFO:tasks.workunit.client.0.smithi153.stdout:5/859: dwrite d4/d5/df/d67/d86/d96/deb/d56/d105/f10e [0,4194304] 0 2024-03-21T10:09:04.910 INFO:tasks.workunit.client.1.smithi174.stdout:6/551: link d0/d3/d20/d3d/c75 d0/d3/d5/caa 0 2024-03-21T10:09:04.912 INFO:tasks.workunit.client.1.smithi174.stdout:1/809: symlink d4/d7/d33/d43/d51/ded/l121 0 2024-03-21T10:09:04.917 INFO:tasks.workunit.client.0.smithi153.stdout:4/899: dread d11/f24 [0,4194304] 0 2024-03-21T10:09:04.917 INFO:tasks.workunit.client.1.smithi174.stdout:2/663: dwrite d5/d25/d27/f40 [0,4194304] 0 2024-03-21T10:09:04.917 INFO:tasks.workunit.client.1.smithi174.stdout:7/621: creat d0/d4/fd6 x:0 0 0 2024-03-21T10:09:04.919 INFO:tasks.workunit.client.0.smithi153.stdout:2/922: chown dd/d15/d18/d3a/d3d/ddc/f135 4740691 1 2024-03-21T10:09:04.923 INFO:tasks.workunit.client.1.smithi174.stdout:5/752: mknod d1/d49/d93/c10e 0 2024-03-21T10:09:04.924 INFO:tasks.workunit.client.1.smithi174.stdout:0/695: creat d9/d21/d31/d3c/ff1 x:0 0 0 2024-03-21T10:09:04.926 INFO:tasks.workunit.client.1.smithi174.stdout:6/552: mknod d0/d3/d2a/d56/cab 0 2024-03-21T10:09:04.940 INFO:tasks.workunit.client.0.smithi153.stdout:4/900: read f1 [5877356,31576] 0 2024-03-21T10:09:04.944 INFO:tasks.workunit.client.1.smithi174.stdout:8/647: link d9/d2a/d39/fad d9/d53/d5b/fd5 0 2024-03-21T10:09:04.944 INFO:tasks.workunit.client.1.smithi174.stdout:6/553: write d0/d3/d5/f45 [3026188,20204] 0 2024-03-21T10:09:04.945 INFO:tasks.workunit.client.1.smithi174.stdout:1/810: rename d4/d7/d68/d7b/d8b to d4/d7/d33/d43/d77/d122 0 2024-03-21T10:09:04.948 INFO:tasks.workunit.client.1.smithi174.stdout:3/674: write d0/d12/d14/f80 [1152736,83997] 0 2024-03-21T10:09:04.949 INFO:tasks.workunit.client.0.smithi153.stdout:7/882: dwrite d4/d2a/f2d [4194304,4194304] 0 2024-03-21T10:09:04.949 INFO:tasks.workunit.client.1.smithi174.stdout:1/811: stat d4/d7/d32/d7d/c94 0 2024-03-21T10:09:04.949 INFO:tasks.workunit.client.1.smithi174.stdout:6/554: chown d0/d3/d5/d6 75509017 1 2024-03-21T10:09:04.950 INFO:tasks.workunit.client.0.smithi153.stdout:4/901: fsync d11/d2d/f4a 0 2024-03-21T10:09:04.952 INFO:tasks.workunit.client.1.smithi174.stdout:9/734: dread d2/d11/d60/f77 [0,4194304] 0 2024-03-21T10:09:04.954 INFO:tasks.workunit.client.0.smithi153.stdout:7/883: dread - d4/f6c zero size 2024-03-21T10:09:04.960 INFO:tasks.workunit.client.0.smithi153.stdout:3/939: dwrite dd/d1e/d3d/d94/d73/d7f/f12e [0,4194304] 0 2024-03-21T10:09:04.960 INFO:tasks.workunit.client.1.smithi174.stdout:9/735: chown d2/d54/f5c 0 1 2024-03-21T10:09:04.968 INFO:tasks.workunit.client.0.smithi153.stdout:9/921: dwrite d1/d43/d44/d46/d9e/f31 [0,4194304] 0 2024-03-21T10:09:04.969 INFO:tasks.workunit.client.1.smithi174.stdout:6/555: dwrite d0/d3/d2a/d56/f95 [0,4194304] 0 2024-03-21T10:09:04.982 INFO:tasks.workunit.client.1.smithi174.stdout:5/753: truncate d1/d16/f60 527492 0 2024-03-21T10:09:04.983 INFO:tasks.workunit.client.0.smithi153.stdout:3/940: unlink dd/d1e/d2f/d35/d68/f96 0 2024-03-21T10:09:04.984 INFO:tasks.workunit.client.0.smithi153.stdout:8/817: sync 2024-03-21T10:09:04.986 INFO:tasks.workunit.client.1.smithi174.stdout:7/622: creat d0/d4/d10/d1b/fd7 x:0 0 0 2024-03-21T10:09:04.987 INFO:tasks.workunit.client.1.smithi174.stdout:8/648: mknod d9/dc6/cd6 0 2024-03-21T10:09:04.987 INFO:tasks.workunit.client.1.smithi174.stdout:0/696: truncate d9/d21/f6e 2022723 0 2024-03-21T10:09:04.989 INFO:tasks.workunit.client.1.smithi174.stdout:2/664: rename d5/d25/d27/d2d/d45/d5b/d55/d75/f8d to d5/d25/fe2 0 2024-03-21T10:09:04.991 INFO:tasks.workunit.client.1.smithi174.stdout:3/675: mknod d0/d12/d14/d4c/d7a/cdb 0 2024-03-21T10:09:04.995 INFO:tasks.workunit.client.1.smithi174.stdout:7/623: mknod d0/d4/d38/da9/cd8 0 2024-03-21T10:09:04.995 INFO:tasks.workunit.client.1.smithi174.stdout:9/736: creat d2/d24/d2e/d63/f101 x:0 0 0 2024-03-21T10:09:04.996 INFO:tasks.workunit.client.0.smithi153.stdout:8/818: chown d5/d9/d43/dd/d2d/d45/d88/f4c 110 1 2024-03-21T10:09:04.997 INFO:tasks.workunit.client.0.smithi153.stdout:2/923: dwrite dd/d15/d5e/f8f [0,4194304] 0 2024-03-21T10:09:04.998 INFO:tasks.workunit.client.1.smithi174.stdout:8/649: creat d9/d2a/d39/d76/d19/fd7 x:0 0 0 2024-03-21T10:09:04.999 INFO:tasks.workunit.client.1.smithi174.stdout:5/754: rename d1/d49/fdd to d1/d49/d4f/d51/d8f/d103/f10f 0 2024-03-21T10:09:05.006 INFO:tasks.workunit.client.0.smithi153.stdout:7/884: dread d4/d19/d4d/dd5/fe2 [0,4194304] 0 2024-03-21T10:09:05.008 INFO:tasks.workunit.client.1.smithi174.stdout:6/556: dwrite d0/d3/d5/df/d66/f96 [0,4194304] 0 2024-03-21T10:09:05.009 INFO:tasks.workunit.client.0.smithi153.stdout:5/860: dwrite d4/d5/df/d67/d6a/f77 [0,4194304] 0 2024-03-21T10:09:05.011 INFO:tasks.workunit.client.1.smithi174.stdout:0/697: fdatasync d9/d21/d31/d3c/d72/f85 0 2024-03-21T10:09:05.011 INFO:tasks.workunit.client.0.smithi153.stdout:9/922: truncate d1/d3b/daa/fea 377065 0 2024-03-21T10:09:05.012 INFO:tasks.workunit.client.0.smithi153.stdout:7/885: fsync d4/d14/d2f/d78/f6f 0 2024-03-21T10:09:05.013 INFO:tasks.workunit.client.0.smithi153.stdout:5/861: readlink d4/d8/d5a/d92/lff 0 2024-03-21T10:09:05.014 INFO:tasks.workunit.client.1.smithi174.stdout:8/650: write d9/d2a/d4d/db4/fb6 [415414,25537] 0 2024-03-21T10:09:05.014 INFO:tasks.workunit.client.1.smithi174.stdout:9/737: fsync d2/d11/d1d/d2a/fc9 0 2024-03-21T10:09:05.022 INFO:tasks.workunit.client.1.smithi174.stdout:1/812: dread d4/d7/d33/d43/d77/d87/d92/ff4 [0,4194304] 0 2024-03-21T10:09:05.023 INFO:tasks.workunit.client.1.smithi174.stdout:8/651: truncate d9/d53/f6c 5031821 0 2024-03-21T10:09:05.030 INFO:tasks.workunit.client.1.smithi174.stdout:7/624: dwrite d0/d4/f23 [4194304,4194304] 0 2024-03-21T10:09:05.030 INFO:tasks.workunit.client.1.smithi174.stdout:5/755: dread d1/d5c/d73/fa5 [0,4194304] 0 2024-03-21T10:09:05.032 INFO:tasks.workunit.client.1.smithi174.stdout:6/557: dread d0/d3/d2a/d56/f95 [0,4194304] 0 2024-03-21T10:09:05.035 INFO:tasks.workunit.client.1.smithi174.stdout:6/558: read - d0/d3/d2a/f8d zero size 2024-03-21T10:09:05.042 INFO:tasks.workunit.client.1.smithi174.stdout:7/625: truncate d0/d4/d10/d1b/d86/dd2/fd0 788831 0 2024-03-21T10:09:05.047 INFO:tasks.workunit.client.1.smithi174.stdout:6/559: truncate d0/d3/d5/df/d66/d87/f9e 443838 0 2024-03-21T10:09:05.050 INFO:tasks.workunit.client.1.smithi174.stdout:8/652: dwrite f5 [4194304,4194304] 0 2024-03-21T10:09:05.050 INFO:tasks.workunit.client.1.smithi174.stdout:1/813: dread d4/d7/d15/d53/f54 [0,4194304] 0 2024-03-21T10:09:05.053 INFO:tasks.workunit.client.1.smithi174.stdout:5/756: truncate d1/fe1 894922 0 2024-03-21T10:09:05.054 INFO:tasks.workunit.client.1.smithi174.stdout:7/626: creat d0/d4/d38/fd9 x:0 0 0 2024-03-21T10:09:05.057 INFO:tasks.workunit.client.1.smithi174.stdout:7/627: chown d0/d53/fd5 5 1 2024-03-21T10:09:05.058 INFO:tasks.workunit.client.1.smithi174.stdout:8/653: truncate d9/d53/d5b/fd4 513824 0 2024-03-21T10:09:05.058 INFO:tasks.workunit.client.1.smithi174.stdout:1/814: mkdir d4/d7/d33/d43/d77/d87/d123 0 2024-03-21T10:09:05.059 INFO:tasks.workunit.client.1.smithi174.stdout:7/628: fdatasync d0/d4/d38/fd9 0 2024-03-21T10:09:05.064 INFO:tasks.workunit.client.1.smithi174.stdout:0/698: getdents d9/d35/d4a/d67/d89/d99 0 2024-03-21T10:09:05.065 INFO:tasks.workunit.client.1.smithi174.stdout:7/629: dread - d0/d4/d38/da9/dc6/fca zero size 2024-03-21T10:09:05.066 INFO:tasks.workunit.client.1.smithi174.stdout:6/560: dwrite d0/d3/d2a/d8f/f97 [0,4194304] 0 2024-03-21T10:09:05.075 INFO:tasks.workunit.client.1.smithi174.stdout:8/654: dread d9/d2a/d4d/db4/fb6 [0,4194304] 0 2024-03-21T10:09:05.076 INFO:tasks.workunit.client.1.smithi174.stdout:1/815: creat d4/d7/d68/f124 x:0 0 0 2024-03-21T10:09:05.079 INFO:tasks.workunit.client.1.smithi174.stdout:6/561: fdatasync d0/d3/d5/d6/f4a 0 2024-03-21T10:09:05.082 INFO:tasks.workunit.client.1.smithi174.stdout:8/655: creat d9/d2a/d39/d58/dbb/fd8 x:0 0 0 2024-03-21T10:09:05.085 INFO:tasks.workunit.client.1.smithi174.stdout:8/656: write d9/d2a/d39/d76/d19/fd7 [278277,75786] 0 2024-03-21T10:09:05.085 INFO:tasks.workunit.client.1.smithi174.stdout:1/816: write d4/d7/d15/d4f/f69 [576493,48748] 0 2024-03-21T10:09:05.090 INFO:tasks.workunit.client.1.smithi174.stdout:1/817: truncate d4/d7/d32/da2/ffc 988011 0 2024-03-21T10:09:05.092 INFO:tasks.workunit.client.1.smithi174.stdout:8/657: creat d9/d2a/d39/d58/fd9 x:0 0 0 2024-03-21T10:09:05.098 INFO:tasks.workunit.client.1.smithi174.stdout:4/805: truncate d4/d21/d24/d28/de9/d6a/dab/f99 3733484 0 2024-03-21T10:09:05.104 INFO:tasks.workunit.client.1.smithi174.stdout:3/676: truncate d0/f70 2922053 0 2024-03-21T10:09:05.106 INFO:tasks.workunit.client.1.smithi174.stdout:5/757: dread d1/f22 [0,4194304] 0 2024-03-21T10:09:05.107 INFO:tasks.workunit.client.1.smithi174.stdout:4/806: rename d4/fbf to d4/d21/d24/d28/de9/f10c 0 2024-03-21T10:09:05.109 INFO:tasks.workunit.client.1.smithi174.stdout:3/677: write d0/d12/d14/d1a/d27/f2c [538719,72993] 0 2024-03-21T10:09:05.113 INFO:tasks.workunit.client.1.smithi174.stdout:1/818: dwrite d4/d7/d32/d7d/f10c [0,4194304] 0 2024-03-21T10:09:05.117 INFO:tasks.workunit.client.1.smithi174.stdout:4/807: chown d4/d21/d24/d28/de9/d3a/d7a 1370 1 2024-03-21T10:09:05.123 INFO:tasks.workunit.client.1.smithi174.stdout:7/630: write d0/d4/f22 [1603976,111883] 0 2024-03-21T10:09:05.123 INFO:tasks.workunit.client.1.smithi174.stdout:0/699: write d9/d21/f2b [277581,33663] 0 2024-03-21T10:09:05.124 INFO:tasks.workunit.client.1.smithi174.stdout:2/665: write d5/d25/d27/d2d/d45/dae/dd6/d4f/d65/f92 [8459,23748] 0 2024-03-21T10:09:05.124 INFO:tasks.workunit.client.1.smithi174.stdout:6/562: write d0/d3/d5/d21/f6c [4958141,14740] 0 2024-03-21T10:09:05.125 INFO:tasks.workunit.client.1.smithi174.stdout:1/819: dread d4/d7/d15/f1f [0,4194304] 0 2024-03-21T10:09:05.127 INFO:tasks.workunit.client.1.smithi174.stdout:4/808: chown d4/d21/d24/d28/d79/cc6 5176 1 2024-03-21T10:09:05.130 INFO:tasks.workunit.client.1.smithi174.stdout:9/738: dwrite d2/d11/d1d/d4b/f6e [0,4194304] 0 2024-03-21T10:09:05.131 INFO:tasks.workunit.client.1.smithi174.stdout:3/678: unlink d0/d12/d14/d1a/d27/d3c/l99 0 2024-03-21T10:09:05.133 INFO:tasks.workunit.client.1.smithi174.stdout:5/758: write d1/f11 [3336656,30298] 0 2024-03-21T10:09:05.133 INFO:tasks.workunit.client.1.smithi174.stdout:0/700: fdatasync d9/d35/dd2/fe6 0 2024-03-21T10:09:05.136 INFO:tasks.workunit.client.1.smithi174.stdout:1/820: chown d4/d7/d33/d43/d77/d87/d123 14174 1 2024-03-21T10:09:05.140 INFO:tasks.workunit.client.1.smithi174.stdout:8/658: dread d9/d53/f95 [0,4194304] 0 2024-03-21T10:09:05.141 INFO:tasks.workunit.client.0.smithi153.stdout:3/941: getdents dd/d1e/d2f 0 2024-03-21T10:09:05.143 INFO:tasks.workunit.client.0.smithi153.stdout:4/902: dwrite d11/d2d/d6b/f8a [4194304,4194304] 0 2024-03-21T10:09:05.146 INFO:tasks.workunit.client.1.smithi174.stdout:7/631: rename d0/d53 to d0/d4/d10/d79/db2/dda 0 2024-03-21T10:09:05.147 INFO:tasks.workunit.client.1.smithi174.stdout:9/739: rmdir d2/d24/d2e/df6 39 2024-03-21T10:09:05.147 INFO:tasks.workunit.client.1.smithi174.stdout:3/679: dread - d0/f91 zero size 2024-03-21T10:09:05.148 INFO:tasks.workunit.client.1.smithi174.stdout:6/563: creat d0/d3/d5/df/fac x:0 0 0 2024-03-21T10:09:05.152 INFO:tasks.workunit.client.0.smithi153.stdout:7/886: rename d4/d14/d2f/fbe to d4/d2a/d40/df8/f12a 0 2024-03-21T10:09:05.153 INFO:tasks.workunit.client.0.smithi153.stdout:7/887: chown d4/l9c 14943 1 2024-03-21T10:09:05.154 INFO:tasks.workunit.client.1.smithi174.stdout:7/632: truncate d0/d4/f7e 4027425 0 2024-03-21T10:09:05.154 INFO:tasks.workunit.client.1.smithi174.stdout:1/821: creat d4/d7/d33/d43/d77/d87/d92/f125 x:0 0 0 2024-03-21T10:09:05.158 INFO:tasks.workunit.client.0.smithi153.stdout:9/923: dwrite d1/d43/d44/d46/d9e/d24/d41/f109 [0,4194304] 0 2024-03-21T10:09:05.159 INFO:tasks.workunit.client.0.smithi153.stdout:7/888: write d4/d14/d2f/d78/f29 [2324142,89890] 0 2024-03-21T10:09:05.161 INFO:tasks.workunit.client.1.smithi174.stdout:6/564: mknod d0/d3/da8/cad 0 2024-03-21T10:09:05.162 INFO:tasks.workunit.client.1.smithi174.stdout:3/680: symlink d0/d12/d14/d67/dc8/ldc 0 2024-03-21T10:09:05.163 INFO:tasks.workunit.client.0.smithi153.stdout:7/889: stat d4/d14/ce0 0 2024-03-21T10:09:05.168 INFO:tasks.workunit.client.0.smithi153.stdout:9/924: rename d1/d4e/d68 to d1/d43/d44/d46/d4d/d88/d130 0 2024-03-21T10:09:05.168 INFO:tasks.workunit.client.0.smithi153.stdout:5/862: write d4/d5/d16/f19 [5654500,40044] 0 2024-03-21T10:09:05.171 INFO:tasks.workunit.client.0.smithi153.stdout:3/942: dwrite dd/f15 [0,4194304] 0 2024-03-21T10:09:05.172 INFO:tasks.workunit.client.1.smithi174.stdout:7/633: mknod d0/d4/d10/cdb 0 2024-03-21T10:09:05.174 INFO:tasks.workunit.client.1.smithi174.stdout:8/659: dwrite d9/d2a/d39/fa0 [0,4194304] 0 2024-03-21T10:09:05.178 INFO:tasks.workunit.client.0.smithi153.stdout:4/903: write d11/feb [958937,121498] 0 2024-03-21T10:09:05.178 INFO:tasks.workunit.client.0.smithi153.stdout:3/943: dread - dd/d1e/d3d/d94/d73/ff5 zero size 2024-03-21T10:09:05.180 INFO:tasks.workunit.client.0.smithi153.stdout:9/925: truncate d1/d43/d44/d46/d9e/d2e/d67/f127 206407 0 2024-03-21T10:09:05.181 INFO:tasks.workunit.client.1.smithi174.stdout:2/666: truncate d5/d25/d27/d2d/d45/dae/dd6/d90/fc3 2630374 0 2024-03-21T10:09:05.185 INFO:tasks.workunit.client.1.smithi174.stdout:1/822: rmdir d4/d7/d33/d43/d77/d87/dfb 0 2024-03-21T10:09:05.186 INFO:tasks.workunit.client.1.smithi174.stdout:4/809: dwrite d4/d19/fa4 [0,4194304] 0 2024-03-21T10:09:05.186 INFO:tasks.workunit.client.1.smithi174.stdout:6/565: link d0/d3/d5/d6/f4a d0/d3/d5/df/fae 0 2024-03-21T10:09:05.187 INFO:tasks.workunit.client.0.smithi153.stdout:2/924: dread dd/d15/d18/d2c/d4b/d56/d63/f80 [0,4194304] 0 2024-03-21T10:09:05.189 INFO:tasks.workunit.client.1.smithi174.stdout:2/667: fsync d5/d25/fa4 0 2024-03-21T10:09:05.190 INFO:tasks.workunit.client.0.smithi153.stdout:5/863: dread d4/d5/f7 [0,4194304] 0 2024-03-21T10:09:05.191 INFO:tasks.workunit.client.1.smithi174.stdout:0/701: dwrite d9/d35/d4a/d67/fdd [0,4194304] 0 2024-03-21T10:09:05.203 INFO:tasks.workunit.client.1.smithi174.stdout:3/681: dwrite d0/f7 [0,4194304] 0 2024-03-21T10:09:05.204 INFO:tasks.workunit.client.1.smithi174.stdout:7/634: read d0/d4/f12 [1839494,18248] 0 2024-03-21T10:09:05.204 INFO:tasks.workunit.client.1.smithi174.stdout:9/740: truncate d2/d11/f36 633043 0 2024-03-21T10:09:05.207 INFO:tasks.workunit.client.1.smithi174.stdout:9/741: write d2/d24/f74 [3817694,399] 0 2024-03-21T10:09:05.210 INFO:tasks.workunit.client.1.smithi174.stdout:3/682: write d0/d12/d14/f80 [1848050,61525] 0 2024-03-21T10:09:05.211 INFO:tasks.workunit.client.1.smithi174.stdout:9/742: dread - d2/d24/d2e/d63/f101 zero size 2024-03-21T10:09:05.213 INFO:tasks.workunit.client.1.smithi174.stdout:5/759: dread d1/d49/f102 [0,4194304] 0 2024-03-21T10:09:05.213 INFO:tasks.workunit.client.0.smithi153.stdout:9/926: rmdir d1/d3b/df8 39 2024-03-21T10:09:05.214 INFO:tasks.workunit.client.1.smithi174.stdout:8/660: rmdir d9/d2a/d39/d76/d19/d27/d60 39 2024-03-21T10:09:05.217 INFO:tasks.workunit.client.0.smithi153.stdout:4/904: rename d11/f16 to d11/d18/d50/d65/de3/f11c 0 2024-03-21T10:09:05.218 INFO:tasks.workunit.client.0.smithi153.stdout:5/864: truncate d4/d8/d5a/d5d/dd9/d44/de1/dfb/d108/fc6 4637090 0 2024-03-21T10:09:05.219 INFO:tasks.workunit.client.0.smithi153.stdout:0/776: sync 2024-03-21T10:09:05.219 INFO:tasks.workunit.client.0.smithi153.stdout:1/729: sync 2024-03-21T10:09:05.223 INFO:tasks.workunit.client.1.smithi174.stdout:0/702: symlink d9/d35/d4a/d67/d89/d99/lf2 0 2024-03-21T10:09:05.224 INFO:tasks.workunit.client.0.smithi153.stdout:9/927: creat d1/d43/d44/d46/d9e/d61/f131 x:0 0 0 2024-03-21T10:09:05.227 INFO:tasks.workunit.client.0.smithi153.stdout:5/865: unlink d4/d5/df/d67/d86/d96/deb/d56/da5/fc4 0 2024-03-21T10:09:05.233 INFO:tasks.workunit.client.0.smithi153.stdout:9/928: rename d1/d4/fd6 to d1/d43/d44/d46/d9e/d24/f132 0 2024-03-21T10:09:05.237 INFO:tasks.workunit.client.0.smithi153.stdout:9/929: rmdir d1/d43/d44/d46/d4d/d88 39 2024-03-21T10:09:05.238 INFO:tasks.workunit.client.1.smithi174.stdout:3/683: dwrite d0/d12/d14/d1a/d27/d3c/d73/fbb [0,4194304] 0 2024-03-21T10:09:05.241 INFO:tasks.workunit.client.1.smithi174.stdout:6/566: write d0/d3/d5/df/fae [6896605,105098] 0 2024-03-21T10:09:05.246 INFO:tasks.workunit.client.0.smithi153.stdout:9/930: rename d1/d43/d44/d46/d9e/d24/d41/f47 to d1/d43/d44/d46/d9e/d61/f133 0 2024-03-21T10:09:05.247 INFO:tasks.workunit.client.1.smithi174.stdout:3/684: readlink d0/d12/d14/d1a/d27/d3c/l58 0 2024-03-21T10:09:05.250 INFO:tasks.workunit.client.1.smithi174.stdout:0/703: creat d9/d21/d31/d5c/d86/ff3 x:0 0 0 2024-03-21T10:09:05.252 INFO:tasks.workunit.client.0.smithi153.stdout:9/931: mkdir d1/d3b/df8/dfa/d134 0 2024-03-21T10:09:05.256 INFO:tasks.workunit.client.0.smithi153.stdout:9/932: chown d1/d43/d44/d46/d9e/d24/d41/l10d 2 1 2024-03-21T10:09:05.257 INFO:tasks.workunit.client.0.smithi153.stdout:9/933: chown d1/d43/c4a 22 1 2024-03-21T10:09:05.264 INFO:tasks.workunit.client.0.smithi153.stdout:9/934: truncate d1/d43/d44/d46/f1d 3206884 0 2024-03-21T10:09:05.266 INFO:tasks.workunit.client.1.smithi174.stdout:1/823: dwrite d4/d2d/fce [0,4194304] 0 2024-03-21T10:09:05.266 INFO:tasks.workunit.client.1.smithi174.stdout:6/567: mkdir d0/d3/d2a/daf 0 2024-03-21T10:09:05.266 INFO:tasks.workunit.client.0.smithi153.stdout:3/944: dread dd/d1e/d2f/d35/f7b [0,4194304] 0 2024-03-21T10:09:05.268 INFO:tasks.workunit.client.0.smithi153.stdout:0/777: write d4/d16/d19/d58/f9c [2364290,81991] 0 2024-03-21T10:09:05.270 INFO:tasks.workunit.client.1.smithi174.stdout:2/668: dwrite d5/d25/d27/f80 [0,4194304] 0 2024-03-21T10:09:05.271 INFO:tasks.workunit.client.1.smithi174.stdout:5/760: getdents d1/d5c/d73 0 2024-03-21T10:09:05.274 INFO:tasks.workunit.client.1.smithi174.stdout:0/704: creat d9/ff4 x:0 0 0 2024-03-21T10:09:05.276 INFO:tasks.workunit.client.1.smithi174.stdout:3/685: read d0/d12/f2d [4051925,2881] 0 2024-03-21T10:09:05.278 INFO:tasks.workunit.client.0.smithi153.stdout:9/935: write d1/d4/d5f/df3/f114 [253117,78206] 0 2024-03-21T10:09:05.279 INFO:tasks.workunit.client.1.smithi174.stdout:8/661: dwrite d9/d53/f78 [0,4194304] 0 2024-03-21T10:09:05.279 INFO:tasks.workunit.client.1.smithi174.stdout:4/810: write d4/d21/d24/d28/de9/d6a/dab/f99 [179904,116010] 0 2024-03-21T10:09:05.283 INFO:tasks.workunit.client.0.smithi153.stdout:2/925: dwrite dd/d15/d18/f7a [0,4194304] 0 2024-03-21T10:09:05.284 INFO:tasks.workunit.client.0.smithi153.stdout:9/936: chown d1/d43/d44/d46/d9e/d24/fa1 16016556 1 2024-03-21T10:09:05.286 INFO:tasks.workunit.client.1.smithi174.stdout:6/568: symlink d0/d3/d5/df/d16/lb0 0 2024-03-21T10:09:05.287 INFO:tasks.workunit.client.1.smithi174.stdout:9/743: dwrite d2/d11/d1d/d2a/da7/fb3 [0,4194304] 0 2024-03-21T10:09:05.288 INFO:tasks.workunit.client.1.smithi174.stdout:3/686: chown d0/d12/d14/d1a/d44/l82 240219498 1 2024-03-21T10:09:05.288 INFO:tasks.workunit.client.1.smithi174.stdout:5/761: fsync d1/d16/d2e/d66/d8b/d85/f90 0 2024-03-21T10:09:05.289 INFO:tasks.workunit.client.0.smithi153.stdout:5/866: dwrite d4/d5/df/d67/d86/d96/deb/d56/f62 [0,4194304] 0 2024-03-21T10:09:05.290 INFO:tasks.workunit.client.1.smithi174.stdout:0/705: rmdir d9/d21/d75/da1 39 2024-03-21T10:09:05.290 INFO:tasks.workunit.client.1.smithi174.stdout:6/569: stat d0/d3/d2a/daf 0 2024-03-21T10:09:05.292 INFO:tasks.workunit.client.1.smithi174.stdout:1/824: dread d4/d7/d32/d86/fd2 [0,4194304] 0 2024-03-21T10:09:05.293 INFO:tasks.workunit.client.0.smithi153.stdout:4/905: dwrite d11/d18/d54/dab/fd2 [0,4194304] 0 2024-03-21T10:09:05.293 INFO:tasks.workunit.client.0.smithi153.stdout:0/778: rename d4/d16/d19/dad/dec to d4/d16/d19/d30/d6c/d98/df3 0 2024-03-21T10:09:05.293 INFO:tasks.workunit.client.1.smithi174.stdout:7/635: dwrite d0/d4/d10/fbd [4194304,4194304] 0 2024-03-21T10:09:05.295 INFO:tasks.workunit.client.0.smithi153.stdout:0/779: stat d4/d16/d19/d30/d6c/d98/d5e/f70 0 2024-03-21T10:09:05.297 INFO:tasks.workunit.client.0.smithi153.stdout:9/937: fdatasync d1/d43/d44/d46/d9e/d2e/f62 0 2024-03-21T10:09:05.298 INFO:tasks.workunit.client.1.smithi174.stdout:3/687: write d0/d12/d14/f80 [1258866,33830] 0 2024-03-21T10:09:05.305 INFO:tasks.workunit.client.0.smithi153.stdout:0/780: mkdir d4/d16/d19/d30/d6c/df4 0 2024-03-21T10:09:05.306 INFO:tasks.workunit.client.1.smithi174.stdout:0/706: write d9/d35/d4a/d67/d89/d99/db2/dbd/fe1 [244513,58459] 0 2024-03-21T10:09:05.312 INFO:tasks.workunit.client.1.smithi174.stdout:6/570: rmdir d0/d3/d5/d6/d42 39 2024-03-21T10:09:05.312 INFO:tasks.workunit.client.1.smithi174.stdout:3/688: chown d0/d12/d14/d1a/d5c/f8e 2731 1 2024-03-21T10:09:05.317 INFO:tasks.workunit.client.1.smithi174.stdout:5/762: symlink d1/d5c/d73/de3/l110 0 2024-03-21T10:09:05.325 INFO:tasks.workunit.client.0.smithi153.stdout:5/867: dwrite d4/d5/df/d67/d86/d96/deb/d56/fca [4194304,4194304] 0 2024-03-21T10:09:05.326 INFO:tasks.workunit.client.1.smithi174.stdout:4/811: rename d4/d21/d24/cf0 to d4/d21/d24/d28/de9/d3a/d67/d76/d82/c10d 0 2024-03-21T10:09:05.338 INFO:tasks.workunit.client.1.smithi174.stdout:0/707: dwrite d9/d21/d31/d5c/d92/d98/fc4 [4194304,4194304] 0 2024-03-21T10:09:05.342 INFO:tasks.workunit.client.0.smithi153.stdout:0/781: truncate d4/d16/d19/d58/d9a/fc8 695468 0 2024-03-21T10:09:05.344 INFO:tasks.workunit.client.0.smithi153.stdout:9/938: link d1/d43/d44/fff d1/d43/f135 0 2024-03-21T10:09:05.345 INFO:tasks.workunit.client.1.smithi174.stdout:6/571: chown d0/d3/d5/d6/f38 264351150 1 2024-03-21T10:09:05.346 INFO:tasks.workunit.client.0.smithi153.stdout:9/939: unlink d1/d3b/f66 0 2024-03-21T10:09:05.349 INFO:tasks.workunit.client.0.smithi153.stdout:0/782: link d4/d4c/f7a d4/d16/d19/d30/d6c/d98/d5e/db2/ff5 0 2024-03-21T10:09:05.352 INFO:tasks.workunit.client.0.smithi153.stdout:9/940: read d1/d43/d44/f3e [3435786,75614] 0 2024-03-21T10:09:05.353 INFO:tasks.workunit.client.0.smithi153.stdout:0/783: rename d4/d4c/d83/f35 to d4/dd0/ff6 0 2024-03-21T10:09:05.355 INFO:tasks.workunit.client.1.smithi174.stdout:3/689: mknod d0/d12/d14/d67/dc8/cdd 0 2024-03-21T10:09:05.358 INFO:tasks.workunit.client.1.smithi174.stdout:7/636: creat d0/d4/d10/d60/da2/fdc x:0 0 0 2024-03-21T10:09:05.361 INFO:tasks.workunit.client.1.smithi174.stdout:8/662: link f8 d9/d2a/d39/d76/d19/d27/d60/daa/dab/dc4/fda 0 2024-03-21T10:09:05.362 INFO:tasks.workunit.client.0.smithi153.stdout:2/926: write dd/d15/d18/d2c/d33/d98/fb5 [678304,28718] 0 2024-03-21T10:09:05.367 INFO:tasks.workunit.client.1.smithi174.stdout:6/572: creat d0/d3/d5/d3b/fb1 x:0 0 0 2024-03-21T10:09:05.367 INFO:tasks.workunit.client.1.smithi174.stdout:3/690: write d0/d12/d14/d4c/d7a/d9e/fbd [198851,60339] 0 2024-03-21T10:09:05.368 INFO:tasks.workunit.client.0.smithi153.stdout:3/945: dread dd/d1e/d2f/d35/d68/fc6 [0,4194304] 0 2024-03-21T10:09:05.371 INFO:tasks.workunit.client.0.smithi153.stdout:3/946: fdatasync dd/d41/f64 0 2024-03-21T10:09:05.372 INFO:tasks.workunit.client.1.smithi174.stdout:7/637: rmdir d0/d9e 39 2024-03-21T10:09:05.373 INFO:tasks.workunit.client.1.smithi174.stdout:9/744: rename d2/f17 to d2/f102 0 2024-03-21T10:09:05.376 INFO:tasks.workunit.client.0.smithi153.stdout:3/947: fdatasync dd/d1e/d2f/d35/daf/dcc/fa5 0 2024-03-21T10:09:05.381 INFO:tasks.workunit.client.1.smithi174.stdout:0/708: creat d9/d21/d31/d3c/ff5 x:0 0 0 2024-03-21T10:09:05.381 INFO:tasks.workunit.client.0.smithi153.stdout:3/948: rename dd/c55 to dd/d1e/d6b/c145 0 2024-03-21T10:09:05.382 INFO:tasks.workunit.client.0.smithi153.stdout:2/927: dwrite dd/d15/d18/d2c/d4b/d56/d63/d8d/d9b/dc3/f132 [0,4194304] 0 2024-03-21T10:09:05.383 INFO:tasks.workunit.client.1.smithi174.stdout:9/745: write d2/d11/d1d/d2a/da7/fe9 [1658114,11133] 0 2024-03-21T10:09:05.384 INFO:tasks.workunit.client.1.smithi174.stdout:4/812: getdents d4/d19 0 2024-03-21T10:09:05.384 INFO:tasks.workunit.client.1.smithi174.stdout:1/825: dread d4/d7/f90 [0,4194304] 0 2024-03-21T10:09:05.385 INFO:tasks.workunit.client.0.smithi153.stdout:3/949: chown dd/d1e/d2f/d35/daf/dcc/dd6 204779 1 2024-03-21T10:09:05.386 INFO:tasks.workunit.client.1.smithi174.stdout:7/638: dread d0/d4/d10/d79/db2/dda/d5f/d65/fcd [0,4194304] 0 2024-03-21T10:09:05.390 INFO:tasks.workunit.client.1.smithi174.stdout:9/746: chown d2/d11/d1d/d2a 1840 1 2024-03-21T10:09:05.391 INFO:tasks.workunit.client.1.smithi174.stdout:6/573: dread d0/d3/d2a/d56/d5e/f98 [0,4194304] 0 2024-03-21T10:09:05.391 INFO:tasks.workunit.client.1.smithi174.stdout:5/763: rename d1/d16/d1c/d81/f8a to d1/d2f/f111 0 2024-03-21T10:09:05.392 INFO:tasks.workunit.client.0.smithi153.stdout:4/906: dwrite d11/d18/f4b [4194304,4194304] 0 2024-03-21T10:09:05.393 INFO:tasks.workunit.client.1.smithi174.stdout:1/826: write d4/d5/f105 [829912,92506] 0 2024-03-21T10:09:05.394 INFO:tasks.workunit.client.0.smithi153.stdout:3/950: unlink dd/f13 0 2024-03-21T10:09:05.396 INFO:tasks.workunit.client.1.smithi174.stdout:7/639: read - d0/d4/d10/d1b/fd7 zero size 2024-03-21T10:09:05.397 INFO:tasks.workunit.client.1.smithi174.stdout:2/669: truncate d5/d25/d27/d2d/d45/dae/dd6/d7b/f99 2927779 0 2024-03-21T10:09:05.398 INFO:tasks.workunit.client.0.smithi153.stdout:8/819: sync 2024-03-21T10:09:05.399 INFO:tasks.workunit.client.1.smithi174.stdout:8/663: dwrite d9/d2a/d4d/fca [0,4194304] 0 2024-03-21T10:09:05.400 INFO:tasks.workunit.client.0.smithi153.stdout:8/820: readlink d5/d9/d43/dd/d20/df3/lf8 0 2024-03-21T10:09:05.401 INFO:tasks.workunit.client.1.smithi174.stdout:1/827: write d4/d7/d15/f39 [447286,4572] 0 2024-03-21T10:09:05.404 INFO:tasks.workunit.client.0.smithi153.stdout:3/951: mknod dd/d1e/d3d/d11b/c146 0 2024-03-21T10:09:05.406 INFO:tasks.workunit.client.1.smithi174.stdout:1/828: read d4/d7/d33/d43/d77/d87/d92/ff4 [881168,20577] 0 2024-03-21T10:09:05.408 INFO:tasks.workunit.client.1.smithi174.stdout:4/813: rename d4/c44 to d4/d19/c10e 0 2024-03-21T10:09:05.410 INFO:tasks.workunit.client.1.smithi174.stdout:1/829: fsync d4/d7/f2c 0 2024-03-21T10:09:05.411 INFO:tasks.workunit.client.1.smithi174.stdout:9/747: mknod d2/d24/d2e/d63/c103 0 2024-03-21T10:09:05.412 INFO:tasks.workunit.client.0.smithi153.stdout:3/952: mkdir dd/d1e/d3d/d53/d147 0 2024-03-21T10:09:05.414 INFO:tasks.workunit.client.1.smithi174.stdout:7/640: dread - d0/d4/d38/da9/dc6/fca zero size 2024-03-21T10:09:05.416 INFO:tasks.workunit.client.1.smithi174.stdout:6/574: creat d0/d3/d20/d6d/fb2 x:0 0 0 2024-03-21T10:09:05.417 INFO:tasks.workunit.client.0.smithi153.stdout:3/953: rename dd/d1e/d2f/f36 to dd/d1e/d2f/d35/daf/f148 0 2024-03-21T10:09:05.418 INFO:tasks.workunit.client.1.smithi174.stdout:5/764: dread d1/d5c/d73/fa5 [0,4194304] 0 2024-03-21T10:09:05.418 INFO:tasks.workunit.client.1.smithi174.stdout:2/670: write d5/d25/d27/d2d/d45/f6e [4823252,92063] 0 2024-03-21T10:09:05.424 INFO:tasks.workunit.client.1.smithi174.stdout:2/671: chown d5/l5e 1864221470 1 2024-03-21T10:09:05.424 INFO:tasks.workunit.client.1.smithi174.stdout:3/691: getdents d0/d12/d14/d1a/d27/d3c/d73/db4 0 2024-03-21T10:09:05.424 INFO:tasks.workunit.client.0.smithi153.stdout:5/868: write d4/d5/d16/f8a [73797,5206] 0 2024-03-21T10:09:05.428 INFO:tasks.workunit.client.1.smithi174.stdout:2/672: chown d5/d25/d27/d2d/d45/d5b/d2f/d66/db5 270701278 1 2024-03-21T10:09:05.428 INFO:tasks.workunit.client.1.smithi174.stdout:4/814: write d4/d21/d24/d28/de9/d6a/d8f/f94 [3044194,65554] 0 2024-03-21T10:09:05.428 INFO:tasks.workunit.client.0.smithi153.stdout:9/941: write d1/d43/d44/d46/d9e/d61/f87 [29539,100227] 0 2024-03-21T10:09:05.433 INFO:tasks.workunit.client.0.smithi153.stdout:0/784: creat d4/d4c/d83/d22/d61/d77/ff7 x:0 0 0 2024-03-21T10:09:05.435 INFO:tasks.workunit.client.0.smithi153.stdout:9/942: symlink d1/d43/d44/d46/d9e/l136 0 2024-03-21T10:09:05.439 INFO:tasks.workunit.client.0.smithi153.stdout:3/954: dwrite dd/d41/dab/f11c [0,4194304] 0 2024-03-21T10:09:05.443 INFO:tasks.workunit.client.0.smithi153.stdout:0/785: mkdir d4/d16/d19/d30/d6c/df4/df8 0 2024-03-21T10:09:05.443 INFO:tasks.workunit.client.0.smithi153.stdout:8/821: write d5/d9/d43/dd/d20/d26/d24/fa3 [984532,63386] 0 2024-03-21T10:09:05.443 INFO:tasks.workunit.client.1.smithi174.stdout:0/709: dread d9/d1c/d76/d87/dcd/fcf [0,4194304] 0 2024-03-21T10:09:05.445 INFO:tasks.workunit.client.1.smithi174.stdout:1/830: dwrite d4/d5/d6c/fc0 [0,4194304] 0 2024-03-21T10:09:05.447 INFO:tasks.workunit.client.1.smithi174.stdout:0/710: write d9/d21/d75/fc7 [416872,121695] 0 2024-03-21T10:09:05.448 INFO:tasks.workunit.client.0.smithi153.stdout:9/943: rename d1/d4/d5f/df3/ff4 to d1/d108/f137 0 2024-03-21T10:09:05.454 INFO:tasks.workunit.client.0.smithi153.stdout:3/955: creat dd/d1e/d3d/d53/dae/d126/f149 x:0 0 0 2024-03-21T10:09:05.458 INFO:tasks.workunit.client.0.smithi153.stdout:0/786: unlink d4/l7c 0 2024-03-21T10:09:05.463 INFO:tasks.workunit.client.0.smithi153.stdout:9/944: getdents d1/d3b 0 2024-03-21T10:09:05.463 INFO:tasks.workunit.client.0.smithi153.stdout:2/928: dwrite dd/d15/fab [0,4194304] 0 2024-03-21T10:09:05.465 INFO:tasks.workunit.client.0.smithi153.stdout:4/907: dwrite d11/f6f [0,4194304] 0 2024-03-21T10:09:05.466 INFO:tasks.workunit.client.0.smithi153.stdout:0/787: readlink d4/d16/d19/d30/da7/laf 0 2024-03-21T10:09:05.472 INFO:tasks.workunit.client.0.smithi153.stdout:2/929: fsync dd/d15/f13c 0 2024-03-21T10:09:05.472 INFO:tasks.workunit.client.0.smithi153.stdout:5/869: dread d4/d8/d5a/d5d/dd9/d44/de1/dfb/d108/fc6 [0,4194304] 0 2024-03-21T10:09:05.479 INFO:tasks.workunit.client.0.smithi153.stdout:2/930: unlink dd/d15/fab 0 2024-03-21T10:09:05.480 INFO:tasks.workunit.client.0.smithi153.stdout:8/822: dread d5/d9/d43/d7b/f98 [0,4194304] 0 2024-03-21T10:09:05.486 INFO:tasks.workunit.client.0.smithi153.stdout:0/788: truncate d4/d16/d19/d30/d6c/d98/d5e/f75 1636262 0 2024-03-21T10:09:05.489 INFO:tasks.workunit.client.0.smithi153.stdout:3/956: mkdir dd/d1e/d2f/d35/d14a 0 2024-03-21T10:09:05.489 INFO:tasks.workunit.client.0.smithi153.stdout:8/823: dread - d5/d9/d43/dd/d20/d26/d24/d52/d55/f90 zero size 2024-03-21T10:09:05.491 INFO:tasks.workunit.client.1.smithi174.stdout:6/575: dread d0/d3/d2a/d56/d5e/f73 [0,4194304] 0 2024-03-21T10:09:05.494 INFO:tasks.workunit.client.1.smithi174.stdout:6/576: chown d0/d3/d2a/d8f/f97 143 1 2024-03-21T10:09:05.495 INFO:tasks.workunit.client.0.smithi153.stdout:2/931: truncate dd/d15/d18/d2c/d4b/d56/d63/d8d/d9b/dc3/df5/dae/f103 55076 0 2024-03-21T10:09:05.495 INFO:tasks.workunit.client.0.smithi153.stdout:8/824: chown d5/d9/f9b 0 1 2024-03-21T10:09:05.498 INFO:tasks.workunit.client.0.smithi153.stdout:0/789: rename d4/d16/d19/d30/d6c/d98/d5b/c67 to d4/d16/d19/d30/d6c/de3/cf9 0 2024-03-21T10:09:05.499 INFO:tasks.workunit.client.0.smithi153.stdout:3/957: truncate dd/d1e/d2f/d35/f7b 4632167 0 2024-03-21T10:09:05.507 INFO:tasks.workunit.client.0.smithi153.stdout:7/890: sync 2024-03-21T10:09:05.507 INFO:tasks.workunit.client.0.smithi153.stdout:1/730: sync 2024-03-21T10:09:05.508 INFO:tasks.workunit.client.0.smithi153.stdout:9/945: dwrite d1/d43/d44/dc3/fd4 [0,4194304] 0 2024-03-21T10:09:05.514 INFO:tasks.workunit.client.0.smithi153.stdout:8/825: mknod d5/d9/d43/d7b/daa/d102/c10a 0 2024-03-21T10:09:05.515 INFO:tasks.workunit.client.0.smithi153.stdout:0/790: creat d4/d16/d19/d30/d6c/d98/d5e/db2/ffa x:0 0 0 2024-03-21T10:09:05.515 INFO:tasks.workunit.client.0.smithi153.stdout:4/908: write d11/d2d/d57/f88 [445931,93410] 0 2024-03-21T10:09:05.516 INFO:tasks.workunit.client.0.smithi153.stdout:1/731: dread - d6/d13/d75/f9e zero size 2024-03-21T10:09:05.516 INFO:tasks.workunit.client.1.smithi174.stdout:2/673: creat d5/d25/d27/d2d/d45/d5b/d2f/d68/d9e/fe3 x:0 0 0 2024-03-21T10:09:05.519 INFO:tasks.workunit.client.1.smithi174.stdout:2/674: stat d5/d25/d27/d2d/d45/dae/dd6/l37 0 2024-03-21T10:09:05.524 INFO:tasks.workunit.client.0.smithi153.stdout:3/958: dread dd/d1e/d45/f4d [0,4194304] 0 2024-03-21T10:09:05.529 INFO:tasks.workunit.client.0.smithi153.stdout:9/946: creat d1/d8a/f138 x:0 0 0 2024-03-21T10:09:05.531 INFO:tasks.workunit.client.1.smithi174.stdout:6/577: stat d0/d3/d5/d6/d42/l80 0 2024-03-21T10:09:05.532 INFO:tasks.workunit.client.1.smithi174.stdout:3/692: symlink d0/d12/d14/d1a/d44/d65/d7e/dac/lde 0 2024-03-21T10:09:05.533 INFO:tasks.workunit.client.0.smithi153.stdout:1/732: fdatasync d6/f9 0 2024-03-21T10:09:05.537 INFO:tasks.workunit.client.1.smithi174.stdout:8/664: truncate d9/d53/f6c 1091795 0 2024-03-21T10:09:05.540 INFO:tasks.workunit.client.1.smithi174.stdout:8/665: write d9/d2a/d39/d58/fb8 [897432,10098] 0 2024-03-21T10:09:05.542 INFO:tasks.workunit.client.1.smithi174.stdout:2/675: write d5/d25/f7e [1442850,119190] 0 2024-03-21T10:09:05.547 INFO:tasks.workunit.client.1.smithi174.stdout:6/578: creat d0/d3/d2a/d56/d5e/d91/fb3 x:0 0 0 2024-03-21T10:09:05.550 INFO:tasks.workunit.client.0.smithi153.stdout:5/870: dread d4/d8/d5a/d5d/dd9/d44/de1/dfb/d108/f84 [0,4194304] 0 2024-03-21T10:09:05.552 INFO:tasks.workunit.client.0.smithi153.stdout:3/959: symlink dd/d1e/d3d/l14b 0 2024-03-21T10:09:05.557 INFO:tasks.workunit.client.1.smithi174.stdout:2/676: mknod d5/d25/d27/d2d/d45/d5b/d2f/d4b/d76/ce4 0 2024-03-21T10:09:05.558 INFO:tasks.workunit.client.0.smithi153.stdout:7/891: rename d4/d11/f7f to d4/f12b 0 2024-03-21T10:09:05.559 INFO:tasks.workunit.client.0.smithi153.stdout:9/947: creat d1/d4/d5f/df3/d11a/f139 x:0 0 0 2024-03-21T10:09:05.560 INFO:tasks.workunit.client.1.smithi174.stdout:6/579: creat d0/d3/d2a/fb4 x:0 0 0 2024-03-21T10:09:05.560 INFO:tasks.workunit.client.0.smithi153.stdout:0/791: link d4/c14 d4/db/cfb 0 2024-03-21T10:09:05.561 INFO:tasks.workunit.client.0.smithi153.stdout:1/733: creat d6/d10/d35/da5/dea/fee x:0 0 0 2024-03-21T10:09:05.563 INFO:tasks.workunit.client.0.smithi153.stdout:2/932: truncate dd/d15/d18/d2c/d4b/d56/d63/d8d/d9b/dc3/df5/dae/f100 465020 0 2024-03-21T10:09:05.564 INFO:tasks.workunit.client.1.smithi174.stdout:2/677: creat d5/d25/d27/d2d/d45/d5b/d2f/d4b/fe5 x:0 0 0 2024-03-21T10:09:05.565 INFO:tasks.workunit.client.0.smithi153.stdout:5/871: rename d4/d5/d16/f8a to d4/d5/df/d67/d86/d96/deb/d56/da5/f11e 0 2024-03-21T10:09:05.566 INFO:tasks.workunit.client.1.smithi174.stdout:6/580: write d0/d3/d2a/d56/d5e/f73 [1726948,65719] 0 2024-03-21T10:09:05.567 INFO:tasks.workunit.client.0.smithi153.stdout:3/960: creat dd/d1e/d2f/d35/d111/f14c x:0 0 0 2024-03-21T10:09:05.567 INFO:tasks.workunit.client.0.smithi153.stdout:4/909: dread d11/d2d/d6b/f8a [0,4194304] 0 2024-03-21T10:09:05.571 INFO:tasks.workunit.client.0.smithi153.stdout:0/792: creat d4/d16/d19/d58/d9a/ffc x:0 0 0 2024-03-21T10:09:05.572 INFO:tasks.workunit.client.1.smithi174.stdout:8/666: dwrite d9/dc6/f8f [0,4194304] 0 2024-03-21T10:09:05.573 INFO:tasks.workunit.client.0.smithi153.stdout:3/961: write dd/d1e/d2f/d35/daf/f120 [30175,6553] 0 2024-03-21T10:09:05.573 INFO:tasks.workunit.client.0.smithi153.stdout:5/872: read d4/d5/fd4 [2513263,77443] 0 2024-03-21T10:09:05.574 INFO:tasks.workunit.client.1.smithi174.stdout:0/711: dread d9/d35/d4a/f5e [0,4194304] 0 2024-03-21T10:09:05.575 INFO:tasks.workunit.client.1.smithi174.stdout:2/678: creat d5/d25/d27/d2d/d45/d5b/d2f/d68/d9e/fe6 x:0 0 0 2024-03-21T10:09:05.578 INFO:tasks.workunit.client.1.smithi174.stdout:6/581: rmdir d0/d3/d5/df/d66/d87 39 2024-03-21T10:09:05.579 INFO:tasks.workunit.client.0.smithi153.stdout:4/910: mknod d11/d2d/da3/d103/c11d 0 2024-03-21T10:09:05.579 INFO:tasks.workunit.client.0.smithi153.stdout:3/962: rename dd to dd/d1e/d3d/d53/dca/d14d 22 2024-03-21T10:09:05.579 INFO:tasks.workunit.client.0.smithi153.stdout:9/948: chown d1/d43/d44/d46/d9e/d24/d41/dbe/dcc/fb5 5148 1 2024-03-21T10:09:05.583 INFO:tasks.workunit.client.1.smithi174.stdout:2/679: read d5/d25/d27/d2d/d45/dae/dd6/d4f/d65/fa2 [5132344,82176] 0 2024-03-21T10:09:05.584 INFO:tasks.workunit.client.0.smithi153.stdout:5/873: symlink d4/d5/d16/l11f 0 2024-03-21T10:09:05.584 INFO:tasks.workunit.client.0.smithi153.stdout:3/963: chown dd/d1e/d2f/d35/d68/d75/l8f 15 1 2024-03-21T10:09:05.584 INFO:tasks.workunit.client.0.smithi153.stdout:8/826: dread d5/d1b/f59 [0,4194304] 0 2024-03-21T10:09:05.584 INFO:tasks.workunit.client.0.smithi153.stdout:9/949: read d1/d4/d5f/f102 [2250367,72838] 0 2024-03-21T10:09:05.586 INFO:tasks.workunit.client.1.smithi174.stdout:6/582: creat d0/d3/d2a/d8f/fb5 x:0 0 0 2024-03-21T10:09:05.587 INFO:tasks.workunit.client.1.smithi174.stdout:6/583: fdatasync d0/f37 0 2024-03-21T10:09:05.591 INFO:tasks.workunit.client.0.smithi153.stdout:8/827: write d5/d9/d43/d7b/daa/fb8 [1985911,24848] 0 2024-03-21T10:09:05.597 INFO:tasks.workunit.client.0.smithi153.stdout:1/734: write d6/d32/f5a [4154430,42804] 0 2024-03-21T10:09:05.597 INFO:tasks.workunit.client.0.smithi153.stdout:7/892: write d4/d19/d4d/f95 [12466,30487] 0 2024-03-21T10:09:05.601 INFO:tasks.workunit.client.0.smithi153.stdout:5/874: truncate d4/d5/df/d67/d86/fda 90016 0 2024-03-21T10:09:05.602 INFO:tasks.workunit.client.1.smithi174.stdout:1/831: creat d4/d7/d15/d53/d78/f126 x:0 0 0 2024-03-21T10:09:05.603 INFO:tasks.workunit.client.0.smithi153.stdout:7/893: fsync d4/d11/ff2 0 2024-03-21T10:09:05.604 INFO:tasks.workunit.client.1.smithi174.stdout:1/832: chown d4/d5/fc9 3381 1 2024-03-21T10:09:05.604 INFO:tasks.workunit.client.0.smithi153.stdout:9/950: creat d1/d43/d44/d46/d9e/d24/d41/f13a x:0 0 0 2024-03-21T10:09:05.606 INFO:tasks.workunit.client.0.smithi153.stdout:5/875: readlink d4/d8/d5a/d5d/lb9 0 2024-03-21T10:09:05.609 INFO:tasks.workunit.client.0.smithi153.stdout:7/894: readlink d4/d11/l94 0 2024-03-21T10:09:05.609 INFO:tasks.workunit.client.1.smithi174.stdout:1/833: symlink d4/d7/d68/d7b/de0/l127 0 2024-03-21T10:09:05.612 INFO:tasks.workunit.client.1.smithi174.stdout:1/834: fdatasync d4/d5/d6c/f73 0 2024-03-21T10:09:05.615 INFO:tasks.workunit.client.0.smithi153.stdout:0/793: write d4/d16/d19/d30/d6c/d98/d5e/f99 [30714,7691] 0 2024-03-21T10:09:05.617 INFO:tasks.workunit.client.0.smithi153.stdout:2/933: dread dd/d15/d18/d2c/d33/f43 [0,4194304] 0 2024-03-21T10:09:05.617 INFO:tasks.workunit.client.0.smithi153.stdout:3/964: dwrite dd/d41/dab/f11c [4194304,4194304] 0 2024-03-21T10:09:05.620 INFO:tasks.workunit.client.1.smithi174.stdout:4/815: write d4/d21/d24/d28/de9/d6a/dab/da8/fc3 [3626338,59649] 0 2024-03-21T10:09:05.621 INFO:tasks.workunit.client.0.smithi153.stdout:2/934: stat dd/d15/d5e/fb8 0 2024-03-21T10:09:05.622 INFO:tasks.workunit.client.1.smithi174.stdout:5/765: creat d1/d49/f112 x:0 0 0 2024-03-21T10:09:05.624 INFO:tasks.workunit.client.0.smithi153.stdout:1/735: rename d6/d10/d35/d5b to d6/d10/d35/da5/d4f/d9d/db6/def 0 2024-03-21T10:09:05.626 INFO:tasks.workunit.client.1.smithi174.stdout:7/641: rename d0/d4/d10/d44/c61 to d0/d4/d10/d79/db2/dda/cdd 0 2024-03-21T10:09:05.626 INFO:tasks.workunit.client.0.smithi153.stdout:5/876: truncate d4/d5/df/d67/d86/d96/deb/dc1/f119 360146 0 2024-03-21T10:09:05.627 INFO:tasks.workunit.client.1.smithi174.stdout:6/584: rmdir d0/d3/d2a/d56/d5e/d91 39 2024-03-21T10:09:05.629 INFO:tasks.workunit.client.0.smithi153.stdout:8/828: write d5/d9/d43/dd/d20/d26/d8b/f4f [533384,103040] 0 2024-03-21T10:09:05.629 INFO:tasks.workunit.client.0.smithi153.stdout:1/736: readlink d6/d10/l14 0 2024-03-21T10:09:05.631 INFO:tasks.workunit.client.0.smithi153.stdout:4/911: dwrite d11/d2d/f36 [4194304,4194304] 0 2024-03-21T10:09:05.631 INFO:tasks.workunit.client.0.smithi153.stdout:3/965: chown dd/d1e/l23 102 1 2024-03-21T10:09:05.631 INFO:tasks.workunit.client.0.smithi153.stdout:0/794: creat d4/d16/d19/d30/d6c/df4/ffd x:0 0 0 2024-03-21T10:09:05.633 INFO:tasks.workunit.client.1.smithi174.stdout:4/816: mknod d4/d21/d24/d28/c10f 0 2024-03-21T10:09:05.634 INFO:tasks.workunit.client.0.smithi153.stdout:2/935: symlink dd/d15/d18/d2c/l13f 0 2024-03-21T10:09:05.635 INFO:tasks.workunit.client.0.smithi153.stdout:3/966: fdatasync dd/d1e/d2f/d35/daf/dcc/f119 0 2024-03-21T10:09:05.639 INFO:tasks.workunit.client.1.smithi174.stdout:4/817: creat d4/f110 x:0 0 0 2024-03-21T10:09:05.643 INFO:tasks.workunit.client.1.smithi174.stdout:7/642: creat d0/d4/d10/d44/d81/fde x:0 0 0 2024-03-21T10:09:05.643 INFO:tasks.workunit.client.1.smithi174.stdout:4/818: chown d4/f110 1490758 1 2024-03-21T10:09:05.644 INFO:tasks.workunit.client.1.smithi174.stdout:8/667: write d9/d2a/d39/d76/d19/d27/d60/daa/dab/dc4/fda [1101266,22276] 0 2024-03-21T10:09:05.645 INFO:tasks.workunit.client.1.smithi174.stdout:3/693: creat d0/d12/fdf x:0 0 0 2024-03-21T10:09:05.647 INFO:tasks.workunit.client.1.smithi174.stdout:8/668: stat d9/d2a/d39/f47 0 2024-03-21T10:09:05.648 INFO:tasks.workunit.client.1.smithi174.stdout:6/585: dwrite d0/d3/d5/d6/f2d [0,4194304] 0 2024-03-21T10:09:05.651 INFO:tasks.workunit.client.1.smithi174.stdout:4/819: mknod d4/d21/d24/d28/de9/d3a/d67/c111 0 2024-03-21T10:09:05.651 INFO:tasks.workunit.client.0.smithi153.stdout:0/795: fdatasync d4/d16/d19/d30/d6c/d98/d5b/fa9 0 2024-03-21T10:09:05.651 INFO:tasks.workunit.client.1.smithi174.stdout:0/712: rename d9/d21/d5f/dd9 to d9/d21/df6 0 2024-03-21T10:09:05.653 INFO:tasks.workunit.client.0.smithi153.stdout:4/912: rmdir d11/d18/d50/d65/de3/d115 0 2024-03-21T10:09:05.653 INFO:tasks.workunit.client.1.smithi174.stdout:8/669: chown d9/d2a/d39/d76/d19/cbe 73 1 2024-03-21T10:09:05.654 INFO:tasks.workunit.client.1.smithi174.stdout:4/820: readlink d4/d21/d24/d28/de9/d3a/d67/l70 0 2024-03-21T10:09:05.654 INFO:tasks.workunit.client.1.smithi174.stdout:0/713: read d9/d35/d4a/f5e [1144026,46088] 0 2024-03-21T10:09:05.657 INFO:tasks.workunit.client.1.smithi174.stdout:3/694: fdatasync d0/d12/d14/d4c/d7a/d9e/fb8 0 2024-03-21T10:09:05.659 INFO:tasks.workunit.client.0.smithi153.stdout:5/877: dread d4/d5/df/d5e/f6e [0,4194304] 0 2024-03-21T10:09:05.661 INFO:tasks.workunit.client.0.smithi153.stdout:0/796: rename d4/l9 to d4/d4c/lfe 0 2024-03-21T10:09:05.661 INFO:tasks.workunit.client.1.smithi174.stdout:7/643: rename d0/fa to d0/d4/d10/d44/d81/d9a/fdf 0 2024-03-21T10:09:05.662 INFO:tasks.workunit.client.0.smithi153.stdout:9/951: dwrite d1/d43/d44/f3e [4194304,4194304] 0 2024-03-21T10:09:05.664 INFO:tasks.workunit.client.1.smithi174.stdout:5/766: rmdir d1/dca 39 2024-03-21T10:09:05.667 INFO:tasks.workunit.client.0.smithi153.stdout:8/829: dwrite d5/d9/d43/dd/d2d/f66 [0,4194304] 0 2024-03-21T10:09:05.669 INFO:tasks.workunit.client.1.smithi174.stdout:3/695: creat d0/d12/d14/d1a/d27/d4f/fe0 x:0 0 0 2024-03-21T10:09:05.671 INFO:tasks.workunit.client.1.smithi174.stdout:4/821: truncate d4/d21/f49 78745 0 2024-03-21T10:09:05.675 INFO:tasks.workunit.client.0.smithi153.stdout:0/797: fdatasync d4/db/f96 0 2024-03-21T10:09:05.675 INFO:tasks.workunit.client.0.smithi153.stdout:1/737: write d6/d10/d35/da5/d4f/f58 [2971327,98433] 0 2024-03-21T10:09:05.678 INFO:tasks.workunit.client.1.smithi174.stdout:2/680: symlink d5/d25/d27/d2d/le7 0 2024-03-21T10:09:05.680 INFO:tasks.workunit.client.1.smithi174.stdout:0/714: mknod d9/d21/d75/da1/cf7 0 2024-03-21T10:09:05.680 INFO:tasks.workunit.client.0.smithi153.stdout:7/895: dread d4/d2a/f63 [0,4194304] 0 2024-03-21T10:09:05.681 INFO:tasks.workunit.client.0.smithi153.stdout:9/952: truncate d1/d4/fe8 58526 0 2024-03-21T10:09:05.682 INFO:tasks.workunit.client.0.smithi153.stdout:7/896: write d4/d19/d4d/dd5/df7/f10f [545811,12053] 0 2024-03-21T10:09:05.683 INFO:tasks.workunit.client.0.smithi153.stdout:2/936: getdents dd/d15 0 2024-03-21T10:09:05.691 INFO:tasks.workunit.client.0.smithi153.stdout:4/913: creat d11/d18/d50/dfa/dff/f11e x:0 0 0 2024-03-21T10:09:05.696 INFO:tasks.workunit.client.0.smithi153.stdout:1/738: mkdir d6/d10/d35/da5/d4f/df0 0 2024-03-21T10:09:05.697 INFO:tasks.workunit.client.1.smithi174.stdout:2/681: mkdir d5/d25/d27/d2d/d45/d5b/d2f/d68/de8 0 2024-03-21T10:09:05.697 INFO:tasks.workunit.client.1.smithi174.stdout:1/835: dwrite d4/d5/d6c/fa7 [0,4194304] 0 2024-03-21T10:09:05.702 INFO:tasks.workunit.client.0.smithi153.stdout:4/914: write d11/d2d/d6b/d9c/fe1 [329965,120927] 0 2024-03-21T10:09:05.704 INFO:tasks.workunit.client.1.smithi174.stdout:6/586: link d0/d3/d20/d6d/c9b d0/d3/d5/df/d66/cb6 0 2024-03-21T10:09:05.705 INFO:tasks.workunit.client.0.smithi153.stdout:3/967: truncate dd/d1e/d2f/d35/f7b 525298 0 2024-03-21T10:09:05.706 INFO:tasks.workunit.client.0.smithi153.stdout:9/953: rename d1/d43/d44/d46/d4d/d88 to d1/d43/d44/d46/d13b 0 2024-03-21T10:09:05.707 INFO:tasks.workunit.client.1.smithi174.stdout:5/767: mkdir d1/d113 0 2024-03-21T10:09:05.707 INFO:tasks.workunit.client.0.smithi153.stdout:0/798: dwrite d4/d16/d19/d58/d9a/ffc [0,4194304] 0 2024-03-21T10:09:05.707 INFO:tasks.workunit.client.0.smithi153.stdout:2/937: truncate dd/d15/d18/d2c/d4b/d56/d63/d8d/d9b/dc3/df5/dae/ff4 885301 0 2024-03-21T10:09:05.711 INFO:tasks.workunit.client.1.smithi174.stdout:1/836: mknod d4/d7/d33/d43/d51/c128 0 2024-03-21T10:09:05.712 INFO:tasks.workunit.client.1.smithi174.stdout:8/670: dread d9/dc6/f94 [0,4194304] 0 2024-03-21T10:09:05.714 INFO:tasks.workunit.client.1.smithi174.stdout:2/682: creat d5/d25/d27/d2d/d45/d5b/d2f/d68/de8/fe9 x:0 0 0 2024-03-21T10:09:05.714 INFO:tasks.workunit.client.0.smithi153.stdout:5/878: dread d4/d8/f26 [0,4194304] 0 2024-03-21T10:09:05.718 INFO:tasks.workunit.client.1.smithi174.stdout:8/671: fsync d9/f12 0 2024-03-21T10:09:05.718 INFO:tasks.workunit.client.1.smithi174.stdout:6/587: mkdir d0/d3/d5/df/d16/da9/db7 0 2024-03-21T10:09:05.718 INFO:tasks.workunit.client.1.smithi174.stdout:4/822: getdents d4/d21/d24/d28/de9/d3a/d67/d76 0 2024-03-21T10:09:05.718 INFO:tasks.workunit.client.1.smithi174.stdout:1/837: creat d4/d7/d15/f129 x:0 0 0 2024-03-21T10:09:05.721 INFO:tasks.workunit.client.0.smithi153.stdout:7/897: rename d4/d2a/dc6/dfb/d9a/db7/l10d to d4/d14/dd0/d83/l12c 0 2024-03-21T10:09:05.723 INFO:tasks.workunit.client.1.smithi174.stdout:6/588: mknod d0/d3/d5/d6/d42/cb8 0 2024-03-21T10:09:05.725 INFO:tasks.workunit.client.0.smithi153.stdout:8/830: dwrite d5/fc9 [0,4194304] 0 2024-03-21T10:09:05.725 INFO:tasks.workunit.client.1.smithi174.stdout:8/672: rmdir d9/dc6 39 2024-03-21T10:09:05.728 INFO:tasks.workunit.client.1.smithi174.stdout:2/683: mkdir d5/d25/d27/d2d/d45/d5b/d2f/d68/d9e/dea 0 2024-03-21T10:09:05.729 INFO:tasks.workunit.client.0.smithi153.stdout:7/898: mknod d4/d2a/d3b/d81/c12d 0 2024-03-21T10:09:05.732 INFO:tasks.workunit.client.0.smithi153.stdout:7/899: chown d4/d2a/f3e 2 1 2024-03-21T10:09:05.735 INFO:tasks.workunit.client.0.smithi153.stdout:7/900: chown d4/d14/d2f/c5c 123551436 1 2024-03-21T10:09:05.735 INFO:tasks.workunit.client.1.smithi174.stdout:3/696: dread d0/f70 [0,4194304] 0 2024-03-21T10:09:05.738 INFO:tasks.workunit.client.1.smithi174.stdout:7/644: dread d0/d4/d10/d79/db2/dda/f63 [0,4194304] 0 2024-03-21T10:09:05.738 INFO:tasks.workunit.client.0.smithi153.stdout:7/901: dread - d4/d2a/dc6/dfb/f116 zero size 2024-03-21T10:09:05.746 INFO:tasks.workunit.client.0.smithi153.stdout:4/915: dwrite d11/d2d/d57/ded/f109 [0,4194304] 0 2024-03-21T10:09:05.748 INFO:tasks.workunit.client.0.smithi153.stdout:4/916: stat d11/d5e/lcd 0 2024-03-21T10:09:05.748 INFO:tasks.workunit.client.1.smithi174.stdout:7/645: read d0/d4/f23 [6394768,111793] 0 2024-03-21T10:09:05.752 INFO:tasks.workunit.client.0.smithi153.stdout:4/917: write d11/d2d/d6b/d9c/fe1 [1045521,81883] 0 2024-03-21T10:09:05.758 INFO:tasks.workunit.client.1.smithi174.stdout:8/673: link d9/dc6/f94 d9/d2a/d39/d58/dbb/dc3/fdb 0 2024-03-21T10:09:05.760 INFO:tasks.workunit.client.1.smithi174.stdout:6/589: dwrite d0/d3/d20/f7f [4194304,4194304] 0 2024-03-21T10:09:05.761 INFO:tasks.workunit.client.1.smithi174.stdout:8/674: chown d9/d2a/d39/f47 1143781881 1 2024-03-21T10:09:05.768 INFO:tasks.workunit.client.1.smithi174.stdout:7/646: mknod d0/d4/d10/d44/d81/d9a/ce0 0 2024-03-21T10:09:05.770 INFO:tasks.workunit.client.1.smithi174.stdout:8/675: write d9/fae [3894240,107038] 0 2024-03-21T10:09:05.770 INFO:tasks.workunit.client.1.smithi174.stdout:1/838: dwrite d4/d7/d28/f11d [0,4194304] 0 2024-03-21T10:09:05.773 INFO:tasks.workunit.client.1.smithi174.stdout:4/823: write d4/d21/d24/d28/f39 [2060877,62862] 0 2024-03-21T10:09:05.775 INFO:tasks.workunit.client.0.smithi153.stdout:5/879: truncate d4/d8/f2d 4884852 0 2024-03-21T10:09:05.776 INFO:tasks.workunit.client.1.smithi174.stdout:6/590: creat d0/d3/d20/d6d/fb9 x:0 0 0 2024-03-21T10:09:05.778 INFO:tasks.workunit.client.0.smithi153.stdout:5/880: chown d4/d8/d5c 61599 1 2024-03-21T10:09:05.781 INFO:tasks.workunit.client.0.smithi153.stdout:4/918: dwrite d11/d2d/da3/d103/f11b [0,4194304] 0 2024-03-21T10:09:05.783 INFO:tasks.workunit.client.1.smithi174.stdout:1/839: rename d4/d7/d68/d89/fb3 to d4/d2d/f12a 0 2024-03-21T10:09:05.784 INFO:tasks.workunit.client.1.smithi174.stdout:2/684: dwrite d5/d25/fbd [0,4194304] 0 2024-03-21T10:09:05.786 INFO:tasks.workunit.client.0.smithi153.stdout:7/902: dread d4/d19/d4d/d10e/d121/fa5 [0,4194304] 0 2024-03-21T10:09:05.790 INFO:tasks.workunit.client.0.smithi153.stdout:7/903: chown d4/d14/d2f/d78/de6 0 1 2024-03-21T10:09:05.793 INFO:tasks.workunit.client.1.smithi174.stdout:1/840: chown d4/d2d/f5e 213266 1 2024-03-21T10:09:05.802 INFO:tasks.workunit.client.1.smithi174.stdout:1/841: write d4/d5/d6c/d10e/f119 [806738,71518] 0 2024-03-21T10:09:05.805 INFO:tasks.workunit.client.1.smithi174.stdout:4/824: link d4/d21/f7b d4/d21/d24/d28/de9/d6a/d8f/d9c/dfd/f112 0 2024-03-21T10:09:05.807 INFO:tasks.workunit.client.1.smithi174.stdout:2/685: rename d5/d25/d27/d2d/d45/dae/dd6/d4f/d65/d77/cb7 to d5/d25/d27/d2d/d45/d5b/d63/ceb 0 2024-03-21T10:09:05.812 INFO:tasks.workunit.client.1.smithi174.stdout:3/697: dread d0/d12/d14/d67/dc8/fc5 [0,4194304] 0 2024-03-21T10:09:05.812 INFO:tasks.workunit.client.1.smithi174.stdout:1/842: getdents d4/d7/d68/d89/dc6 0 2024-03-21T10:09:05.813 INFO:tasks.workunit.client.1.smithi174.stdout:1/843: dread - d4/d67/ff9 zero size 2024-03-21T10:09:05.818 INFO:tasks.workunit.client.1.smithi174.stdout:3/698: write d0/d12/d14/d1a/d27/d7f/fc4 [777644,110719] 0 2024-03-21T10:09:05.821 INFO:tasks.workunit.client.0.smithi153.stdout:9/954: getdents d1/d43/d44/d46/d9e/d24/d41 0 2024-03-21T10:09:05.821 INFO:tasks.workunit.client.1.smithi174.stdout:4/825: getdents d4/d21/d24/d28/de9/d6a/d8f/d9c 0 2024-03-21T10:09:05.822 INFO:tasks.workunit.client.0.smithi153.stdout:0/799: mkdir d4/d16/d19/d30/d6c/df4/df8/dff 0 2024-03-21T10:09:05.823 INFO:tasks.workunit.client.0.smithi153.stdout:1/739: write d6/d32/f37 [7852455,122103] 0 2024-03-21T10:09:05.824 INFO:tasks.workunit.client.0.smithi153.stdout:8/831: rmdir d5/d9/d43/d7b/daa 39 2024-03-21T10:09:05.825 INFO:tasks.workunit.client.1.smithi174.stdout:7/647: write d0/d4/d10/d79/db2/dda/f80 [449866,94252] 0 2024-03-21T10:09:05.826 INFO:tasks.workunit.client.1.smithi174.stdout:5/768: write d1/d2f/fc9 [263598,51531] 0 2024-03-21T10:09:05.827 INFO:tasks.workunit.client.1.smithi174.stdout:3/699: symlink d0/d12/d14/d1a/d44/d56/le1 0 2024-03-21T10:09:05.834 INFO:tasks.workunit.client.1.smithi174.stdout:6/591: read d0/d3/d5/d3b/f85 [851203,4668] 0 2024-03-21T10:09:05.835 INFO:tasks.workunit.client.0.smithi153.stdout:4/919: symlink d11/d18/d54/dab/l11f 0 2024-03-21T10:09:05.836 INFO:tasks.workunit.client.1.smithi174.stdout:3/700: rmdir d0/d12/d46/d59 39 2024-03-21T10:09:05.839 INFO:tasks.workunit.client.1.smithi174.stdout:4/826: creat d4/d21/d24/d28/de9/d3a/f113 x:0 0 0 2024-03-21T10:09:05.843 INFO:tasks.workunit.client.0.smithi153.stdout:2/938: dwrite dd/d15/d18/d2c/d33/d98/dc7/ded/ff3 [0,4194304] 0 2024-03-21T10:09:05.844 INFO:tasks.workunit.client.1.smithi174.stdout:7/648: link d0/d4/f43 d0/d4/d10/d44/fe1 0 2024-03-21T10:09:05.844 INFO:tasks.workunit.client.1.smithi174.stdout:1/844: dwrite d4/d7/d68/d8e/db8/fd8 [0,4194304] 0 2024-03-21T10:09:05.845 INFO:tasks.workunit.client.0.smithi153.stdout:9/955: dread d1/d43/d44/d46/d9e/d24/d41/f109 [0,4194304] 0 2024-03-21T10:09:05.846 INFO:tasks.workunit.client.0.smithi153.stdout:0/800: dwrite d4/d16/d9f/fbb [0,4194304] 0 2024-03-21T10:09:05.847 INFO:tasks.workunit.client.1.smithi174.stdout:8/676: dwrite d9/d53/f9d [0,4194304] 0 2024-03-21T10:09:05.850 INFO:tasks.workunit.client.1.smithi174.stdout:3/701: symlink d0/d12/d14/d67/da1/le2 0 2024-03-21T10:09:05.851 INFO:tasks.workunit.client.1.smithi174.stdout:1/845: chown d4/d5/c23 785545 1 2024-03-21T10:09:05.852 INFO:tasks.workunit.client.1.smithi174.stdout:6/592: link d0/d3/d5/df/d16/f1a d0/d3/d20/fba 0 2024-03-21T10:09:05.855 INFO:tasks.workunit.client.1.smithi174.stdout:9/748: sync 2024-03-21T10:09:05.856 INFO:tasks.workunit.client.0.smithi153.stdout:1/740: mknod d6/d10/d35/da5/d4f/d9d/dc7/dcc/d80/cf1 0 2024-03-21T10:09:05.862 INFO:tasks.workunit.client.0.smithi153.stdout:8/832: symlink d5/d9/d43/dd/d20/d26/d47/d9a/l10b 0 2024-03-21T10:09:05.862 INFO:tasks.workunit.client.1.smithi174.stdout:1/846: fdatasync d4/d7/d33/d43/d77/d122/fe2 0 2024-03-21T10:09:05.862 INFO:tasks.workunit.client.1.smithi174.stdout:3/702: getdents d0/d12/d14/d98 0 2024-03-21T10:09:05.864 INFO:tasks.workunit.client.1.smithi174.stdout:6/593: read - d0/d3/d5/d6/f70 zero size 2024-03-21T10:09:05.865 INFO:tasks.workunit.client.0.smithi153.stdout:4/920: truncate d11/d2d/d3d/da1/fb5 169721 0 2024-03-21T10:09:05.868 INFO:tasks.workunit.client.1.smithi174.stdout:9/749: rename d2/d24/d2e/d63/f101 to d2/d11/d60/f104 0 2024-03-21T10:09:05.868 INFO:tasks.workunit.client.0.smithi153.stdout:2/939: creat dd/d15/d18/d2c/dd7/f140 x:0 0 0 2024-03-21T10:09:05.871 INFO:tasks.workunit.client.1.smithi174.stdout:6/594: chown d0/d3/d5/d3b/c77 57796365 1 2024-03-21T10:09:05.871 INFO:tasks.workunit.client.1.smithi174.stdout:2/686: dread d5/d25/f7e [0,4194304] 0 2024-03-21T10:09:05.871 INFO:tasks.workunit.client.1.smithi174.stdout:3/703: fdatasync d0/d12/d14/d1a/f23 0 2024-03-21T10:09:05.873 INFO:tasks.workunit.client.0.smithi153.stdout:0/801: mkdir d4/d16/d19/dad/d100 0 2024-03-21T10:09:05.882 INFO:tasks.workunit.client.0.smithi153.stdout:9/956: creat d1/d43/d44/d46/d9e/d24/d12c/f13c x:0 0 0 2024-03-21T10:09:05.883 INFO:tasks.workunit.client.0.smithi153.stdout:2/940: rmdir dd/d15/d18/d2c/d4b 39 2024-03-21T10:09:05.885 INFO:tasks.workunit.client.1.smithi174.stdout:1/847: creat d4/d7/d33/d43/d77/d87/d123/f12b x:0 0 0 2024-03-21T10:09:05.885 INFO:tasks.workunit.client.1.smithi174.stdout:2/687: rename d5/d25/d27/d2d/d45/d5b/f6f to d5/d25/d27/d2d/d45/d5b/d2f/d68/de8/fec 0 2024-03-21T10:09:05.893 INFO:tasks.workunit.client.1.smithi174.stdout:4/827: write d4/d21/f5b [237224,94512] 0 2024-03-21T10:09:05.895 INFO:tasks.workunit.client.0.smithi153.stdout:9/957: link d1/c15 d1/d43/d44/d46/d9e/d2e/d67/c13d 0 2024-03-21T10:09:05.898 INFO:tasks.workunit.client.1.smithi174.stdout:0/715: sync 2024-03-21T10:09:05.899 INFO:tasks.workunit.client.1.smithi174.stdout:8/677: dwrite d9/dc6/fa5 [0,4194304] 0 2024-03-21T10:09:05.904 INFO:tasks.workunit.client.1.smithi174.stdout:5/769: dwrite d1/f71 [0,4194304] 0 2024-03-21T10:09:05.905 INFO:tasks.workunit.client.1.smithi174.stdout:7/649: dwrite d0/d4/d10/d1b/d86/fc4 [0,4194304] 0 2024-03-21T10:09:05.909 INFO:tasks.workunit.client.0.smithi153.stdout:2/941: dwrite dd/d15/d18/d2c/d4b/d56/d63/d8d/d9b/dc3/df5/dae/d50/f121 [0,4194304] 0 2024-03-21T10:09:05.915 INFO:tasks.workunit.client.1.smithi174.stdout:4/828: creat d4/d21/d24/d28/de9/d3a/f114 x:0 0 0 2024-03-21T10:09:05.916 INFO:tasks.workunit.client.1.smithi174.stdout:0/716: mknod d9/d21/d31/d9f/d79/cf8 0 2024-03-21T10:09:05.920 INFO:tasks.workunit.client.1.smithi174.stdout:0/717: write d9/d21/d31/fc6 [279167,69006] 0 2024-03-21T10:09:05.922 INFO:tasks.workunit.client.1.smithi174.stdout:8/678: truncate d9/d53/d7e/f8e 2683143 0 2024-03-21T10:09:05.922 INFO:tasks.workunit.client.1.smithi174.stdout:4/829: chown d4/d21/d24/d28/de9/d3a/d7a/d9d/fc7 1077758224 1 2024-03-21T10:09:05.927 INFO:tasks.workunit.client.1.smithi174.stdout:2/688: dwrite d5/d25/d27/d2d/d45/dae/dd6/d7b/d81/fba [0,4194304] 0 2024-03-21T10:09:05.929 INFO:tasks.workunit.client.0.smithi153.stdout:9/958: symlink d1/d43/d72/l13e 0 2024-03-21T10:09:05.932 INFO:tasks.workunit.client.1.smithi174.stdout:3/704: write d0/d12/d14/d1a/f1e [770983,115185] 0 2024-03-21T10:09:05.932 INFO:tasks.workunit.client.1.smithi174.stdout:6/595: write d0/d3/d5/df/f2e [1898491,117168] 0 2024-03-21T10:09:05.932 INFO:tasks.workunit.client.0.smithi153.stdout:9/959: stat d1/d43/d44/d46/d100/lcf 0 2024-03-21T10:09:05.932 INFO:tasks.workunit.client.0.smithi153.stdout:2/942: dwrite dd/d15/d18/d2c/d4b/d56/d63/d8d/d9b/dc3/df5/dae/d50/f121 [0,4194304] 0 2024-03-21T10:09:05.940 INFO:tasks.workunit.client.1.smithi174.stdout:0/718: mknod d9/d35/cf9 0 2024-03-21T10:09:05.940 INFO:tasks.workunit.client.0.smithi153.stdout:2/943: read dd/d15/d18/d2c/d33/d98/dc7/d107/f10f [440843,70464] 0 2024-03-21T10:09:05.941 INFO:tasks.workunit.client.1.smithi174.stdout:4/830: chown d4/lb 231346 1 2024-03-21T10:09:05.944 INFO:tasks.workunit.client.0.smithi153.stdout:2/944: fdatasync dd/d15/f40 0 2024-03-21T10:09:05.952 INFO:tasks.workunit.client.0.smithi153.stdout:5/881: rename d4/d5/df/d67/d86/d96/lc8 to d4/d5/df/d67/d86/d96/deb/df0/dfa/l120 0 2024-03-21T10:09:05.955 INFO:tasks.workunit.client.0.smithi153.stdout:5/882: dread - d4/d8/d5a/d5d/dd9/d44/ff9 zero size 2024-03-21T10:09:05.958 INFO:tasks.workunit.client.1.smithi174.stdout:2/689: creat d5/d25/d27/d2d/d45/d5b/d2f/d68/de8/fed x:0 0 0 2024-03-21T10:09:05.960 INFO:tasks.workunit.client.1.smithi174.stdout:9/750: dwrite d2/d11/d1d/d8b/fa2 [0,4194304] 0 2024-03-21T10:09:05.962 INFO:tasks.workunit.client.0.smithi153.stdout:5/883: unlink d4/d8/d5a/d5d/dd9/f100 0 2024-03-21T10:09:05.964 INFO:tasks.workunit.client.0.smithi153.stdout:0/802: mkdir d4/d16/d19/d30/d101 0 2024-03-21T10:09:05.965 INFO:tasks.workunit.client.1.smithi174.stdout:0/719: chown d9/d21/d31/f34 1 1 2024-03-21T10:09:05.965 INFO:tasks.workunit.client.0.smithi153.stdout:2/945: dread dd/d15/d18/d2c/d4b/d56/d63/d8d/d9b/dc3/f132 [0,4194304] 0 2024-03-21T10:09:05.969 INFO:tasks.workunit.client.0.smithi153.stdout:0/803: rmdir d4/d16/d27/dc5/dcd 39 2024-03-21T10:09:05.970 INFO:tasks.workunit.client.1.smithi174.stdout:1/848: dread d4/d5/f18 [4194304,4194304] 0 2024-03-21T10:09:05.970 INFO:tasks.workunit.client.1.smithi174.stdout:8/679: dwrite f1 [0,4194304] 0 2024-03-21T10:09:05.972 INFO:tasks.workunit.client.0.smithi153.stdout:4/921: write d11/d2d/d3d/f67 [4419338,88449] 0 2024-03-21T10:09:05.975 INFO:tasks.workunit.client.1.smithi174.stdout:6/596: dread d0/d3/d5/d3b/f94 [0,4194304] 0 2024-03-21T10:09:05.975 INFO:tasks.workunit.client.1.smithi174.stdout:1/849: chown d4/d7/f2c 385666589 1 2024-03-21T10:09:05.977 INFO:tasks.workunit.client.1.smithi174.stdout:5/770: rename d1/d49/d93/d9c/cd6 to d1/d16/d2e/d66/c114 0 2024-03-21T10:09:05.978 INFO:tasks.workunit.client.1.smithi174.stdout:6/597: chown d0/d3/d5/d3b/fb1 956635157 1 2024-03-21T10:09:05.984 INFO:tasks.workunit.client.1.smithi174.stdout:3/705: fsync d0/d12/d14/d67/dc8/fc5 0 2024-03-21T10:09:05.985 INFO:tasks.workunit.client.1.smithi174.stdout:7/650: truncate d0/d4/d10/d1b/f3e 708518 0 2024-03-21T10:09:05.985 INFO:tasks.workunit.client.0.smithi153.stdout:1/741: dwrite d6/d10/f73 [0,4194304] 0 2024-03-21T10:09:05.987 INFO:tasks.workunit.client.1.smithi174.stdout:4/831: creat d4/d21/d24/d28/de9/d3a/d67/f115 x:0 0 0 2024-03-21T10:09:05.988 INFO:tasks.workunit.client.1.smithi174.stdout:0/720: mknod d9/d21/d23/cfa 0 2024-03-21T10:09:05.989 INFO:tasks.workunit.client.0.smithi153.stdout:8/833: dwrite d5/d9/f9b [0,4194304] 0 2024-03-21T10:09:05.990 INFO:tasks.workunit.client.1.smithi174.stdout:8/680: creat d9/dc6/fdc x:0 0 0 2024-03-21T10:09:05.993 INFO:tasks.workunit.client.1.smithi174.stdout:5/771: creat d1/d16/d2e/d66/f115 x:0 0 0 2024-03-21T10:09:05.996 INFO:tasks.workunit.client.1.smithi174.stdout:6/598: write d0/d3/d2a/d56/d5e/d91/fa4 [468427,96109] 0 2024-03-21T10:09:05.997 INFO:tasks.workunit.client.0.smithi153.stdout:7/904: creat d4/d2a/dc6/dfb/f12e x:0 0 0 2024-03-21T10:09:05.997 INFO:tasks.workunit.client.0.smithi153.stdout:9/960: write d1/d43/d44/d46/d9e/f3d [2870576,32564] 0 2024-03-21T10:09:05.999 INFO:tasks.workunit.client.0.smithi153.stdout:4/922: link d11/d2d/d6b/d9c/ld3 d11/d71/l120 0 2024-03-21T10:09:06.003 INFO:tasks.workunit.client.1.smithi174.stdout:3/706: creat d0/d12/d14/d1a/d44/d56/d76/fe3 x:0 0 0 2024-03-21T10:09:06.003 INFO:tasks.workunit.client.0.smithi153.stdout:5/884: write d4/d5/df/d67/d86/fd7 [1263388,126544] 0 2024-03-21T10:09:06.004 INFO:tasks.workunit.client.0.smithi153.stdout:1/742: creat d6/d10/d35/da5/d4f/d9d/dc7/dcc/d80/ff2 x:0 0 0 2024-03-21T10:09:06.007 INFO:tasks.workunit.client.0.smithi153.stdout:7/905: creat d4/d19/d4d/f12f x:0 0 0 2024-03-21T10:09:06.010 INFO:tasks.workunit.client.1.smithi174.stdout:0/721: creat d9/d21/d31/d9f/d79/ffb x:0 0 0 2024-03-21T10:09:06.011 INFO:tasks.workunit.client.1.smithi174.stdout:2/690: link d5/caf d5/d25/d27/d2d/d45/dae/dd6/d4f/d65/d77/cee 0 2024-03-21T10:09:06.012 INFO:tasks.workunit.client.1.smithi174.stdout:9/751: rename d2/d24/d2e/l85 to d2/d11/d1d/l105 0 2024-03-21T10:09:06.013 INFO:tasks.workunit.client.0.smithi153.stdout:1/743: dread - d6/f90 zero size 2024-03-21T10:09:06.015 INFO:tasks.workunit.client.1.smithi174.stdout:0/722: symlink d9/d21/df6/lfc 0 2024-03-21T10:09:06.016 INFO:tasks.workunit.client.1.smithi174.stdout:6/599: symlink d0/d3/lbb 0 2024-03-21T10:09:06.016 INFO:tasks.workunit.client.1.smithi174.stdout:3/707: truncate d0/d12/d14/d1a/d27/d3c/d73/db7/f1b 4872136 0 2024-03-21T10:09:06.017 INFO:tasks.workunit.client.0.smithi153.stdout:2/946: dwrite dd/d15/d18/d2c/d4b/d56/d63/d8d/d9b/dc3/df5/dae/d50/d58/d79/f139 [0,4194304] 0 2024-03-21T10:09:06.019 INFO:tasks.workunit.client.1.smithi174.stdout:6/600: write d0/d3/d20/d3d/fa6 [684139,109612] 0 2024-03-21T10:09:06.020 INFO:tasks.workunit.client.0.smithi153.stdout:4/923: rename d11/f63 to d11/d18/d6d/f121 0 2024-03-21T10:09:06.020 INFO:tasks.workunit.client.0.smithi153.stdout:1/744: truncate d6/fc 581421 0 2024-03-21T10:09:06.021 INFO:tasks.workunit.client.0.smithi153.stdout:7/906: symlink d4/d2a/d100/d124/l130 0 2024-03-21T10:09:06.023 INFO:tasks.workunit.client.0.smithi153.stdout:2/947: stat dd/d15/d18/d2c/d33/d98 0 2024-03-21T10:09:06.023 INFO:tasks.workunit.client.0.smithi153.stdout:4/924: unlink d11/d2d/d3d/f99 0 2024-03-21T10:09:06.024 INFO:tasks.workunit.client.1.smithi174.stdout:8/681: rename d9/d2a/d39/d76/f8b to d9/d2a/d4d/db4/fdd 0 2024-03-21T10:09:06.026 INFO:tasks.workunit.client.0.smithi153.stdout:0/804: dwrite d4/db/fa5 [0,4194304] 0 2024-03-21T10:09:06.026 INFO:tasks.workunit.client.1.smithi174.stdout:7/651: getdents d0/d4/d10 0 2024-03-21T10:09:06.029 INFO:tasks.workunit.client.1.smithi174.stdout:4/832: getdents d4/d21/d24/d28/de9/d6a/d8f/d9c/dfd 0 2024-03-21T10:09:06.034 INFO:tasks.workunit.client.0.smithi153.stdout:0/805: chown d4/d4c/d83/d22/c4a 174 1 2024-03-21T10:09:06.035 INFO:tasks.workunit.client.1.smithi174.stdout:9/752: mknod d2/d11/d1d/d2a/c106 0 2024-03-21T10:09:06.036 INFO:tasks.workunit.client.0.smithi153.stdout:4/925: mkdir d11/d18/d6d/d122 0 2024-03-21T10:09:06.040 INFO:tasks.workunit.client.0.smithi153.stdout:2/948: creat dd/d15/d3c/da7/f141 x:0 0 0 2024-03-21T10:09:06.040 INFO:tasks.workunit.client.0.smithi153.stdout:5/885: rename d4/d8/c29 to d4/d8/d5a/c121 0 2024-03-21T10:09:06.043 INFO:tasks.workunit.client.0.smithi153.stdout:7/907: getdents d4/d2a 0 2024-03-21T10:09:06.044 INFO:tasks.workunit.client.0.smithi153.stdout:3/968: sync 2024-03-21T10:09:06.045 INFO:tasks.workunit.client.1.smithi174.stdout:4/833: creat d4/d21/d24/f116 x:0 0 0 2024-03-21T10:09:06.045 INFO:tasks.workunit.client.1.smithi174.stdout:7/652: getdents d0/d4/d10/d44/d81 0 2024-03-21T10:09:06.046 INFO:tasks.workunit.client.0.smithi153.stdout:0/806: chown d4/d16/d19/d30/d6c/d98/d5e/f78 234571 1 2024-03-21T10:09:06.048 INFO:tasks.workunit.client.0.smithi153.stdout:3/969: fsync dd/d1e/d2f/d35/f5a 0 2024-03-21T10:09:06.050 INFO:tasks.workunit.client.1.smithi174.stdout:9/753: rename d2/d24/fe1 to d2/d11/d1d/d2a/f107 0 2024-03-21T10:09:06.052 INFO:tasks.workunit.client.0.smithi153.stdout:2/949: mknod dd/d15/d18/d2c/d33/d98/dc7/d107/c142 0 2024-03-21T10:09:06.053 INFO:tasks.workunit.client.1.smithi174.stdout:9/754: stat d2/f45 0 2024-03-21T10:09:06.054 INFO:tasks.workunit.client.1.smithi174.stdout:7/653: rename d0/d4/d10/d60/f77 to d0/d4/d38/da9/fe2 0 2024-03-21T10:09:06.055 INFO:tasks.workunit.client.0.smithi153.stdout:7/908: creat d4/d19/d4d/d10e/f131 x:0 0 0 2024-03-21T10:09:06.057 INFO:tasks.workunit.client.1.smithi174.stdout:9/755: creat d2/d11/d60/f108 x:0 0 0 2024-03-21T10:09:06.058 INFO:tasks.workunit.client.1.smithi174.stdout:7/654: write d0/d4/d10/d44/d81/d9a/fa3 [3971625,114581] 0 2024-03-21T10:09:06.059 INFO:tasks.workunit.client.0.smithi153.stdout:7/909: chown d4/d19/d4d/d10e/d121/f7b 0 1 2024-03-21T10:09:06.061 INFO:tasks.workunit.client.0.smithi153.stdout:3/970: write dd/d41/d80/dbb/d112/f136 [4467217,96502] 0 2024-03-21T10:09:06.064 INFO:tasks.workunit.client.1.smithi174.stdout:7/655: mkdir d0/d4/de3 0 2024-03-21T10:09:06.066 INFO:tasks.workunit.client.1.smithi174.stdout:9/756: creat d2/d24/d2e/f109 x:0 0 0 2024-03-21T10:09:06.067 INFO:tasks.workunit.client.1.smithi174.stdout:3/708: dread d0/f13 [0,4194304] 0 2024-03-21T10:09:06.068 INFO:tasks.workunit.client.0.smithi153.stdout:5/886: rename d4/d8/d5a/d5d/dd9/d44/dc7/de2 to d4/d8/d5a/d5d/dd9/d44/de1/dfb/d108/d122 0 2024-03-21T10:09:06.070 INFO:tasks.workunit.client.1.smithi174.stdout:7/656: mknod d0/d4/d10/d44/d81/ce4 0 2024-03-21T10:09:06.071 INFO:tasks.workunit.client.1.smithi174.stdout:9/757: mkdir d2/d10a 0 2024-03-21T10:09:06.075 INFO:tasks.workunit.client.1.smithi174.stdout:9/758: rmdir d2/d24/d2e/dac 39 2024-03-21T10:09:06.075 INFO:tasks.workunit.client.1.smithi174.stdout:1/850: dread d4/d2d/fe1 [0,4194304] 0 2024-03-21T10:09:06.077 INFO:tasks.workunit.client.0.smithi153.stdout:2/950: mkdir dd/d15/d18/d2c/d4b/d56/d63/d8d/d143 0 2024-03-21T10:09:06.078 INFO:tasks.workunit.client.1.smithi174.stdout:2/691: dread d5/d25/d27/d2d/d45/dae/dd6/d85/dad/fcd [0,4194304] 0 2024-03-21T10:09:06.078 INFO:tasks.workunit.client.1.smithi174.stdout:8/682: dread d9/dc6/dd1/fba [0,4194304] 0 2024-03-21T10:09:06.078 INFO:tasks.workunit.client.1.smithi174.stdout:7/657: truncate d0/d4/d10/d44/d81/fde 555640 0 2024-03-21T10:09:06.079 INFO:tasks.workunit.client.0.smithi153.stdout:9/961: dread d1/d43/d44/d46/d9e/d24/d41/dbe/dcc/fa4 [0,4194304] 0 2024-03-21T10:09:06.080 INFO:tasks.workunit.client.1.smithi174.stdout:7/658: stat d0/d4/d10/d44/d81/d9a 0 2024-03-21T10:09:06.084 INFO:tasks.workunit.client.1.smithi174.stdout:3/709: link d0/d12/d14/d1a/d27/d3c/d73/db7/d7d/f8b d0/fe4 0 2024-03-21T10:09:06.085 INFO:tasks.workunit.client.1.smithi174.stdout:7/659: read d0/d4/d10/d60/da2/fb9 [335721,1951] 0 2024-03-21T10:09:06.085 INFO:tasks.workunit.client.1.smithi174.stdout:1/851: mkdir d4/d7/d33/d12c 0 2024-03-21T10:09:06.086 INFO:tasks.workunit.client.0.smithi153.stdout:8/834: dread d5/d9/d43/f2b [0,4194304] 0 2024-03-21T10:09:06.087 INFO:tasks.workunit.client.0.smithi153.stdout:0/807: dwrite d4/d4c/d83/d22/d61/fe0 [0,4194304] 0 2024-03-21T10:09:06.089 INFO:tasks.workunit.client.0.smithi153.stdout:5/887: mknod d4/d5/df/d67/d86/d96/deb/d56/d105/d10d/c123 0 2024-03-21T10:09:06.093 INFO:tasks.workunit.client.0.smithi153.stdout:8/835: stat d5/d9/d43/dd/d20/d26/d24/d52/d55/f60 0 2024-03-21T10:09:06.095 INFO:tasks.workunit.client.0.smithi153.stdout:5/888: read d4/d5/df/d5e/f6e [367317,45816] 0 2024-03-21T10:09:06.095 INFO:tasks.workunit.client.0.smithi153.stdout:9/962: rmdir d1/d43/d44/d46/d9e/d2e/d8f 39 2024-03-21T10:09:06.097 INFO:tasks.workunit.client.0.smithi153.stdout:8/836: stat d5/d9/d43/dd/d2d/d45/f4a 0 2024-03-21T10:09:06.099 INFO:tasks.workunit.client.1.smithi174.stdout:2/692: mkdir d5/d25/d27/d2d/d45/dae/dd6/d4f/da0/def 0 2024-03-21T10:09:06.103 INFO:tasks.workunit.client.1.smithi174.stdout:3/710: creat d0/d12/d14/d1a/d27/d3c/d73/db7/d7d/fe5 x:0 0 0 2024-03-21T10:09:06.103 INFO:tasks.workunit.client.0.smithi153.stdout:1/745: dread d6/d3a/f53 [4194304,4194304] 0 2024-03-21T10:09:06.106 INFO:tasks.workunit.client.0.smithi153.stdout:5/889: mknod d4/d5/df/d67/d86/d96/deb/df0/dfa/c124 0 2024-03-21T10:09:06.109 INFO:tasks.workunit.client.1.smithi174.stdout:8/683: creat d9/d2a/d39/d58/dbb/dc3/fde x:0 0 0 2024-03-21T10:09:06.109 INFO:tasks.workunit.client.1.smithi174.stdout:1/852: creat d4/d7/d28/dc5/f12d x:0 0 0 2024-03-21T10:09:06.110 INFO:tasks.workunit.client.1.smithi174.stdout:1/853: stat d4/d5/c10 0 2024-03-21T10:09:06.112 INFO:tasks.workunit.client.1.smithi174.stdout:9/759: dread d2/d11/d1d/fa4 [0,4194304] 0 2024-03-21T10:09:06.113 INFO:tasks.workunit.client.0.smithi153.stdout:1/746: fsync d6/d3a/d3c/d50/fcd 0 2024-03-21T10:09:06.114 INFO:tasks.workunit.client.1.smithi174.stdout:7/660: rename d0/d4/d38/f91 to d0/d4/d10/d79/fe5 0 2024-03-21T10:09:06.115 INFO:tasks.workunit.client.0.smithi153.stdout:5/890: mkdir d4/d5/df/d67/d86/d96/deb/df0/dfa/d125 0 2024-03-21T10:09:06.116 INFO:tasks.workunit.client.1.smithi174.stdout:8/684: creat d9/d2a/d39/d58/fdf x:0 0 0 2024-03-21T10:09:06.117 INFO:tasks.workunit.client.1.smithi174.stdout:8/685: stat d9/d53/d7e/faf 0 2024-03-21T10:09:06.118 INFO:tasks.workunit.client.0.smithi153.stdout:3/971: truncate dd/f15 1412865 0 2024-03-21T10:09:06.120 INFO:tasks.workunit.client.1.smithi174.stdout:8/686: readlink d9/d2a/d39/d76/l97 0 2024-03-21T10:09:06.122 INFO:tasks.workunit.client.0.smithi153.stdout:8/837: write d5/d9/d43/dd/d2d/d45/d88/f73 [999216,38528] 0 2024-03-21T10:09:06.123 INFO:tasks.workunit.client.1.smithi174.stdout:3/711: dwrite d0/d12/d14/d67/dc8/fc5 [0,4194304] 0 2024-03-21T10:09:06.124 INFO:tasks.workunit.client.0.smithi153.stdout:5/891: creat d4/d5/df/d67/d86/d96/deb/df0/d10a/f126 x:0 0 0 2024-03-21T10:09:06.124 INFO:tasks.workunit.client.1.smithi174.stdout:0/723: truncate d9/d35/d4a/d67/fdd 2148114 0 2024-03-21T10:09:06.127 INFO:tasks.workunit.client.0.smithi153.stdout:3/972: read dd/d1e/d2f/d35/d68/fc6 [102655,44127] 0 2024-03-21T10:09:06.129 INFO:tasks.workunit.client.1.smithi174.stdout:7/661: creat d0/d4/de3/fe6 x:0 0 0 2024-03-21T10:09:06.130 INFO:tasks.workunit.client.1.smithi174.stdout:9/760: rename d2/d54 to d2/d10b 0 2024-03-21T10:09:06.131 INFO:tasks.workunit.client.1.smithi174.stdout:0/724: truncate d9/d21/d31/d5c/d86/fbf 340986 0 2024-03-21T10:09:06.132 INFO:tasks.workunit.client.0.smithi153.stdout:7/910: dwrite d4/d14/d2f/d78/f35 [0,4194304] 0 2024-03-21T10:09:06.134 INFO:tasks.workunit.client.1.smithi174.stdout:7/662: truncate d0/d4/f22 2083932 0 2024-03-21T10:09:06.134 INFO:tasks.workunit.client.0.smithi153.stdout:7/911: stat d4/d14/ccc 0 2024-03-21T10:09:06.134 INFO:tasks.workunit.client.1.smithi174.stdout:9/761: write d2/d11/d60/ff8 [981557,65846] 0 2024-03-21T10:09:06.135 INFO:tasks.workunit.client.1.smithi174.stdout:0/725: stat d9/d21/d23/da5/ca4 0 2024-03-21T10:09:06.137 INFO:tasks.workunit.client.1.smithi174.stdout:3/712: dread d0/d12/d14/f97 [0,4194304] 0 2024-03-21T10:09:06.137 INFO:tasks.workunit.client.0.smithi153.stdout:2/951: dwrite dd/d15/d18/d2c/d4b/fd0 [0,4194304] 0 2024-03-21T10:09:06.138 INFO:tasks.workunit.client.0.smithi153.stdout:7/912: chown d4/d19/d4d/l126 141508020 1 2024-03-21T10:09:06.139 INFO:tasks.workunit.client.1.smithi174.stdout:0/726: dread - d9/d21/d31/d9f/d79/fa0 zero size 2024-03-21T10:09:06.141 INFO:tasks.workunit.client.0.smithi153.stdout:0/808: dread d4/db/f29 [0,4194304] 0 2024-03-21T10:09:06.142 INFO:tasks.workunit.client.1.smithi174.stdout:6/601: dwrite d0/d3/d5/df/d16/f88 [0,4194304] 0 2024-03-21T10:09:06.143 INFO:tasks.workunit.client.0.smithi153.stdout:8/838: symlink d5/d9/d43/dd/d20/d26/d47/d9a/l10c 0 2024-03-21T10:09:06.146 INFO:tasks.workunit.client.0.smithi153.stdout:7/913: symlink d4/d14/d110/l132 0 2024-03-21T10:09:06.148 INFO:tasks.workunit.client.0.smithi153.stdout:0/809: fdatasync d4/d16/d19/d30/d6c/d98/d5e/f78 0 2024-03-21T10:09:06.149 INFO:tasks.workunit.client.0.smithi153.stdout:1/747: dread d6/fb [0,4194304] 0 2024-03-21T10:09:06.150 INFO:tasks.workunit.client.0.smithi153.stdout:8/839: truncate d5/d9/d43/dd/d20/d26/d24/d52/d55/f60 224571 0 2024-03-21T10:09:06.152 INFO:tasks.workunit.client.0.smithi153.stdout:7/914: mkdir d4/d2a/dc6/dfb/d9a/dd2/d133 0 2024-03-21T10:09:06.152 INFO:tasks.workunit.client.0.smithi153.stdout:0/810: symlink d4/d4c/d83/d22/d61/d77/l102 0 2024-03-21T10:09:06.153 INFO:tasks.workunit.client.0.smithi153.stdout:8/840: chown d5/d9/d43/dd/d2d/d45/d88/cb4 3 1 2024-03-21T10:09:06.154 INFO:tasks.workunit.client.1.smithi174.stdout:9/762: mkdir d2/d11/d1d/d10c 0 2024-03-21T10:09:06.155 INFO:tasks.workunit.client.1.smithi174.stdout:2/693: write d5/d25/d27/d2d/d45/dae/fac [74082,56601] 0 2024-03-21T10:09:06.155 INFO:tasks.workunit.client.0.smithi153.stdout:8/841: chown d5/d9/d43/d7b/fe4 2334306 1 2024-03-21T10:09:06.156 INFO:tasks.workunit.client.0.smithi153.stdout:0/811: creat d4/f103 x:0 0 0 2024-03-21T10:09:06.160 INFO:tasks.workunit.client.0.smithi153.stdout:8/842: write d5/d9/d43/dd/d2d/d45/d88/f76 [1436900,86186] 0 2024-03-21T10:09:06.161 INFO:tasks.workunit.client.1.smithi174.stdout:9/763: write d2/d24/d2e/f100 [994803,75371] 0 2024-03-21T10:09:06.167 INFO:tasks.workunit.client.1.smithi174.stdout:1/854: truncate d4/d5/d6c/fc0 2082427 0 2024-03-21T10:09:06.167 INFO:tasks.workunit.client.1.smithi174.stdout:2/694: mknod d5/d25/d27/d2d/d45/dae/dd6/d90/cf0 0 2024-03-21T10:09:06.168 INFO:tasks.workunit.client.1.smithi174.stdout:9/764: fdatasync d2/d11/d60/f104 0 2024-03-21T10:09:06.170 INFO:tasks.workunit.client.0.smithi153.stdout:0/812: getdents d4/d16/d19/d30/d6c/d98/d5e 0 2024-03-21T10:09:06.175 INFO:tasks.workunit.client.1.smithi174.stdout:9/765: symlink d2/d24/d2e/df6/l10d 0 2024-03-21T10:09:06.178 INFO:tasks.workunit.client.0.smithi153.stdout:3/973: write dd/d1e/d3d/d53/fa3 [2209422,100109] 0 2024-03-21T10:09:06.180 INFO:tasks.workunit.client.0.smithi153.stdout:3/974: readlink dd/d1e/d3d/d94/l83 0 2024-03-21T10:09:06.180 INFO:tasks.workunit.client.1.smithi174.stdout:7/663: write d0/d4/d10/d1b/d86/f92 [486862,39344] 0 2024-03-21T10:09:06.180 INFO:tasks.workunit.client.1.smithi174.stdout:6/602: write d0/d3/d5/d3b/f85 [3869972,52820] 0 2024-03-21T10:09:06.182 INFO:tasks.workunit.client.0.smithi153.stdout:3/975: chown dd/d1e/d2f/d35/daf/dcc/lcb 21 1 2024-03-21T10:09:06.186 INFO:tasks.workunit.client.1.smithi174.stdout:6/603: write d0/d3/d5/df/fac [1029152,71026] 0 2024-03-21T10:09:06.186 INFO:tasks.workunit.client.0.smithi153.stdout:3/976: getdents dd/d1e/d6b/d137/d144 0 2024-03-21T10:09:06.191 INFO:tasks.workunit.client.1.smithi174.stdout:9/766: write d2/d11/d1d/d4b/d72/fbb [53472,123963] 0 2024-03-21T10:09:06.191 INFO:tasks.workunit.client.0.smithi153.stdout:3/977: mknod dd/d1e/d2f/d35/daf/dcc/d58/d138/c14e 0 2024-03-21T10:09:06.195 INFO:tasks.workunit.client.0.smithi153.stdout:3/978: chown dd/d1e/d3d/d53/dae 968144205 1 2024-03-21T10:09:06.196 INFO:tasks.workunit.client.1.smithi174.stdout:6/604: symlink d0/d3/d5/df/lbc 0 2024-03-21T10:09:06.196 INFO:tasks.workunit.client.0.smithi153.stdout:2/952: dread dd/d15/d5e/db3/fb9 [0,4194304] 0 2024-03-21T10:09:06.198 INFO:tasks.workunit.client.0.smithi153.stdout:7/915: dwrite d4/d2a/dc6/dfb/d9a/dd2/fc0 [0,4194304] 0 2024-03-21T10:09:06.198 INFO:tasks.workunit.client.0.smithi153.stdout:3/979: unlink dd/f27 0 2024-03-21T10:09:06.198 INFO:tasks.workunit.client.0.smithi153.stdout:2/953: read dd/d15/d5e/db3/fb9 [1575528,41434] 0 2024-03-21T10:09:06.202 INFO:tasks.workunit.client.0.smithi153.stdout:7/916: mkdir d4/d2a/d40/df8/d134 0 2024-03-21T10:09:06.202 INFO:tasks.workunit.client.1.smithi174.stdout:3/713: dwrite d0/d12/d14/d1a/d44/d65/d7e/dac/fc0 [0,4194304] 0 2024-03-21T10:09:06.205 INFO:tasks.workunit.client.0.smithi153.stdout:7/917: symlink d4/d11/d56/d10c/l135 0 2024-03-21T10:09:06.206 INFO:tasks.workunit.client.0.smithi153.stdout:3/980: link dd/c1d dd/d1e/d3d/c14f 0 2024-03-21T10:09:06.208 INFO:tasks.workunit.client.1.smithi174.stdout:9/767: creat d2/d11/f10e x:0 0 0 2024-03-21T10:09:06.208 INFO:tasks.workunit.client.0.smithi153.stdout:7/918: mkdir d4/d14/d136 0 2024-03-21T10:09:06.208 INFO:tasks.workunit.client.0.smithi153.stdout:3/981: dread - dd/d1e/d3d/dc9/f118 zero size 2024-03-21T10:09:06.210 INFO:tasks.workunit.client.1.smithi174.stdout:0/727: dread d9/d21/d31/f42 [0,4194304] 0 2024-03-21T10:09:06.211 INFO:tasks.workunit.client.1.smithi174.stdout:6/605: getdents d0 0 2024-03-21T10:09:06.213 INFO:tasks.workunit.client.1.smithi174.stdout:2/695: dwrite d5/d25/d27/d2d/d45/dae/dd6/d85/dad/fb3 [0,4194304] 0 2024-03-21T10:09:06.213 INFO:tasks.workunit.client.0.smithi153.stdout:7/919: link d4/d2a/d3b/d60/le8 d4/d19/d4d/l137 0 2024-03-21T10:09:06.218 INFO:tasks.workunit.client.0.smithi153.stdout:7/920: symlink d4/d19/d4d/dd5/df7/l138 0 2024-03-21T10:09:06.219 INFO:tasks.workunit.client.1.smithi174.stdout:3/714: creat d0/d12/d14/d67/fe6 x:0 0 0 2024-03-21T10:09:06.222 INFO:tasks.workunit.client.0.smithi153.stdout:7/921: creat d4/d2a/d40/df8/f139 x:0 0 0 2024-03-21T10:09:06.223 INFO:tasks.workunit.client.0.smithi153.stdout:7/922: readlink d4/l65 0 2024-03-21T10:09:06.224 INFO:tasks.workunit.client.1.smithi174.stdout:6/606: read d0/d3/d5/d3b/f55 [3682706,33850] 0 2024-03-21T10:09:06.224 INFO:tasks.workunit.client.1.smithi174.stdout:0/728: mkdir d9/d35/d4a/d67/d89/dfd 0 2024-03-21T10:09:06.225 INFO:tasks.workunit.client.0.smithi153.stdout:3/982: dwrite dd/d1e/d3d/d11b/f123 [0,4194304] 0 2024-03-21T10:09:06.227 INFO:tasks.workunit.client.0.smithi153.stdout:7/923: creat d4/d2a/d100/d124/f13a x:0 0 0 2024-03-21T10:09:06.227 INFO:tasks.workunit.client.1.smithi174.stdout:9/768: symlink d2/d11/d1d/d2a/df2/l10f 0 2024-03-21T10:09:06.231 INFO:tasks.workunit.client.1.smithi174.stdout:7/664: dwrite d0/f11 [0,4194304] 0 2024-03-21T10:09:06.233 INFO:tasks.workunit.client.0.smithi153.stdout:7/924: creat d4/d19/d102/f13b x:0 0 0 2024-03-21T10:09:06.233 INFO:tasks.workunit.client.1.smithi174.stdout:9/769: readlink d2/l2b 0 2024-03-21T10:09:06.236 INFO:tasks.workunit.client.0.smithi153.stdout:3/983: rename dd/d1e/d2f/f90 to dd/d1e/d2f/d35/daf/f150 0 2024-03-21T10:09:06.236 INFO:tasks.workunit.client.1.smithi174.stdout:0/729: creat d9/d21/d31/d5c/d92/ffe x:0 0 0 2024-03-21T10:09:06.236 INFO:tasks.workunit.client.0.smithi153.stdout:7/925: dread - d4/d2a/dc6/dfb/f116 zero size 2024-03-21T10:09:06.237 INFO:tasks.workunit.client.1.smithi174.stdout:6/607: unlink d0/d3/d2a/d56/f95 0 2024-03-21T10:09:06.237 INFO:tasks.workunit.client.0.smithi153.stdout:3/984: chown l9 1876861 1 2024-03-21T10:09:06.242 INFO:tasks.workunit.client.1.smithi174.stdout:2/696: rename d5/d25/d27/d2d/d45/d5b/d2f/d68/d9e to d5/d25/d27/d2d/d45/dae/dd6/d4f/df1 0 2024-03-21T10:09:06.245 INFO:tasks.workunit.client.0.smithi153.stdout:7/926: mkdir d4/d14/dd0/d83/d13c 0 2024-03-21T10:09:06.246 INFO:tasks.workunit.client.1.smithi174.stdout:9/770: rename d2/d11/fb8 to d2/d11/d7d/f110 0 2024-03-21T10:09:06.249 INFO:tasks.workunit.client.1.smithi174.stdout:1/855: dread d4/d5/d6c/fcb [4194304,4194304] 0 2024-03-21T10:09:06.249 INFO:tasks.workunit.client.1.smithi174.stdout:2/697: write d5/d25/d27/d2d/d45/dae/dd6/d7b/d81/fd7 [902117,59451] 0 2024-03-21T10:09:06.249 INFO:tasks.workunit.client.1.smithi174.stdout:3/715: getdents d0/d12/d14/d67/dc8 0 2024-03-21T10:09:06.250 INFO:tasks.workunit.client.1.smithi174.stdout:9/771: rmdir d2/d11/d1d/d4b 39 2024-03-21T10:09:06.251 INFO:tasks.workunit.client.0.smithi153.stdout:7/927: creat d4/d2a/dc6/dfb/d9a/dd2/f13d x:0 0 0 2024-03-21T10:09:06.251 INFO:tasks.workunit.client.1.smithi174.stdout:7/665: link d0/d4/d10/l93 d0/d4/d10/d44/le7 0 2024-03-21T10:09:06.253 INFO:tasks.workunit.client.0.smithi153.stdout:7/928: chown d4/d2a/d100/c122 82 1 2024-03-21T10:09:06.256 INFO:tasks.workunit.client.1.smithi174.stdout:1/856: creat d4/d7/d32/d7d/f12e x:0 0 0 2024-03-21T10:09:06.257 INFO:tasks.workunit.client.1.smithi174.stdout:7/666: read d0/d4/f42 [539291,26974] 0 2024-03-21T10:09:06.258 INFO:tasks.workunit.client.0.smithi153.stdout:2/954: write dd/d15/d18/d2c/d4b/fce [270236,41248] 0 2024-03-21T10:09:06.262 INFO:tasks.workunit.client.1.smithi174.stdout:6/608: write d0/d3/d5/f5c [1070982,81758] 0 2024-03-21T10:09:06.269 INFO:tasks.workunit.client.0.smithi153.stdout:7/929: write d4/d14/d16/f39 [4856898,46001] 0 2024-03-21T10:09:06.270 INFO:tasks.workunit.client.1.smithi174.stdout:7/667: dread - d0/d4/d10/d1b/fd7 zero size 2024-03-21T10:09:06.273 INFO:tasks.workunit.client.0.smithi153.stdout:7/930: chown d4/d14/c106 26385 1 2024-03-21T10:09:06.274 INFO:tasks.workunit.client.0.smithi153.stdout:7/931: stat d4/d2a/d3b/d60/lc2 0 2024-03-21T10:09:06.275 INFO:tasks.workunit.client.1.smithi174.stdout:0/730: dwrite d9/d21/d31/d5c/d92/fdb [0,4194304] 0 2024-03-21T10:09:06.277 INFO:tasks.workunit.client.0.smithi153.stdout:3/985: dwrite dd/f12 [0,4194304] 0 2024-03-21T10:09:06.277 INFO:tasks.workunit.client.0.smithi153.stdout:7/932: write d4/d19/d4d/dd5/df7/f10f [1192319,62231] 0 2024-03-21T10:09:06.278 INFO:tasks.workunit.client.1.smithi174.stdout:1/857: rename d4/d7/d15/d53/d78/fab to d4/d7/d15/d4f/f12f 0 2024-03-21T10:09:06.279 INFO:tasks.workunit.client.1.smithi174.stdout:3/716: truncate d0/d12/d14/d1a/f34 1533061 0 2024-03-21T10:09:06.282 INFO:tasks.workunit.client.1.smithi174.stdout:9/772: dwrite d2/d24/d2e/fb1 [0,4194304] 0 2024-03-21T10:09:06.283 INFO:tasks.workunit.client.0.smithi153.stdout:7/933: getdents d4/d2a/d40/df8 0 2024-03-21T10:09:06.285 INFO:tasks.workunit.client.1.smithi174.stdout:6/609: mknod d0/d3/d5/d6/cbd 0 2024-03-21T10:09:06.287 INFO:tasks.workunit.client.0.smithi153.stdout:7/934: creat d4/d19/f13e x:0 0 0 2024-03-21T10:09:06.288 INFO:tasks.workunit.client.1.smithi174.stdout:2/698: dwrite d5/d25/d27/d2d/d45/dae/dd6/d7b/d81/fd7 [0,4194304] 0 2024-03-21T10:09:06.298 INFO:tasks.workunit.client.1.smithi174.stdout:0/731: link d9/d1c/d76/d87/dcd/cd7 d9/d21/d23/da5/cff 0 2024-03-21T10:09:06.303 INFO:tasks.workunit.client.1.smithi174.stdout:5/772: sync 2024-03-21T10:09:06.303 INFO:tasks.workunit.client.1.smithi174.stdout:6/610: dread d0/d3/d20/d3d/fa6 [0,4194304] 0 2024-03-21T10:09:06.307 INFO:tasks.workunit.client.0.smithi153.stdout:7/935: write d4/d2a/d3b/d60/ff1 [442626,114652] 0 2024-03-21T10:09:06.308 INFO:tasks.workunit.client.1.smithi174.stdout:2/699: read d5/d25/d27/d2d/f41 [12020,84449] 0 2024-03-21T10:09:06.308 INFO:tasks.workunit.client.0.smithi153.stdout:7/936: chown d4/d14/d2f/f45 741 1 2024-03-21T10:09:06.313 INFO:tasks.workunit.client.0.smithi153.stdout:7/937: rmdir d4/d2a/d40/df8/d134 0 2024-03-21T10:09:06.323 INFO:tasks.workunit.client.1.smithi174.stdout:1/858: dwrite d4/d7/d33/d43/f42 [0,4194304] 0 2024-03-21T10:09:06.323 INFO:tasks.workunit.client.1.smithi174.stdout:5/773: mknod d1/d2f/d37/d9b/c116 0 2024-03-21T10:09:06.330 INFO:tasks.workunit.client.1.smithi174.stdout:6/611: unlink d0/d3/d2a/f8d 0 2024-03-21T10:09:06.331 INFO:tasks.workunit.client.1.smithi174.stdout:2/700: symlink d5/d25/d27/d2d/d45/dae/dd6/d7b/d81/lf2 0 2024-03-21T10:09:06.332 INFO:tasks.workunit.client.1.smithi174.stdout:9/773: dread d2/d11/d1d/d2a/df2/faf [0,4194304] 0 2024-03-21T10:09:06.334 INFO:tasks.workunit.client.1.smithi174.stdout:3/717: write d0/d12/d46/d59/f95 [336946,98483] 0 2024-03-21T10:09:06.335 INFO:tasks.workunit.client.1.smithi174.stdout:2/701: chown d5/d25/d27/d2d/d45/d5b/d63/l50 1 1 2024-03-21T10:09:06.335 INFO:tasks.workunit.client.1.smithi174.stdout:1/859: fdatasync d4/d7/d33/d43/d77/f9a 0 2024-03-21T10:09:06.337 INFO:tasks.workunit.client.0.smithi153.stdout:7/938: fsync d4/d19/d4d/d10e/d121/fca 0 2024-03-21T10:09:06.341 INFO:tasks.workunit.client.1.smithi174.stdout:9/774: read d2/f1e [1580772,47195] 0 2024-03-21T10:09:06.346 INFO:tasks.workunit.client.1.smithi174.stdout:1/860: creat d4/d2d/f130 x:0 0 0 2024-03-21T10:09:06.346 INFO:tasks.workunit.client.1.smithi174.stdout:2/702: mknod d5/d25/d27/d2d/d45/d5b/d2f/d4b/da7/cf3 0 2024-03-21T10:09:06.349 INFO:tasks.workunit.client.1.smithi174.stdout:6/612: creat d0/d3/d5/fbe x:0 0 0 2024-03-21T10:09:06.350 INFO:tasks.workunit.client.1.smithi174.stdout:9/775: truncate d2/d11/d1d/d4b/d72/fee 964611 0 2024-03-21T10:09:06.351 INFO:tasks.workunit.client.1.smithi174.stdout:8/687: sync 2024-03-21T10:09:06.351 INFO:tasks.workunit.client.1.smithi174.stdout:4/834: sync 2024-03-21T10:09:06.351 INFO:tasks.workunit.client.0.smithi153.stdout:7/939: dwrite d4/d11/f72 [0,4194304] 0 2024-03-21T10:09:06.353 INFO:tasks.workunit.client.1.smithi174.stdout:6/613: chown d0/d3/d5/d21/l4c 1424 1 2024-03-21T10:09:06.354 INFO:tasks.workunit.client.0.smithi153.stdout:7/940: mknod d4/d2a/dc6/c13f 0 2024-03-21T10:09:06.355 INFO:tasks.workunit.client.1.smithi174.stdout:5/774: rename d1/fef to d1/d2f/f117 0 2024-03-21T10:09:06.356 INFO:tasks.workunit.client.0.smithi153.stdout:7/941: dread - d4/d19/d4d/dd5/f113 zero size 2024-03-21T10:09:06.357 INFO:tasks.workunit.client.0.smithi153.stdout:7/942: fdatasync d4/d19/fc1 0 2024-03-21T10:09:06.357 INFO:tasks.workunit.client.1.smithi174.stdout:6/614: chown d0/d3/d5/df/f22 17 1 2024-03-21T10:09:06.360 INFO:tasks.workunit.client.0.smithi153.stdout:7/943: stat d4/d2a/d3b/d60/de1/cfd 0 2024-03-21T10:09:06.360 INFO:tasks.workunit.client.1.smithi174.stdout:7/668: dread d0/d4/d10/d79/db2/dda/d5f/d65/f6c [0,4194304] 0 2024-03-21T10:09:06.365 INFO:tasks.workunit.client.1.smithi174.stdout:1/861: creat d4/d7/d28/f131 x:0 0 0 2024-03-21T10:09:06.367 INFO:tasks.workunit.client.0.smithi153.stdout:7/944: dread d4/d14/fa4 [0,4194304] 0 2024-03-21T10:09:06.369 INFO:tasks.workunit.client.1.smithi174.stdout:8/688: link d9/d2a/d4d/db2/fc7 d9/d2a/d39/d76/d19/d27/d60/daa/dab/fe0 0 2024-03-21T10:09:06.369 INFO:tasks.workunit.client.0.smithi153.stdout:7/945: symlink d4/d19/d4d/dd5/l140 0 2024-03-21T10:09:06.370 INFO:tasks.workunit.client.1.smithi174.stdout:2/703: getdents d5/d25/d27/d2d/d45/d5b/d2f/d68/de8 0 2024-03-21T10:09:06.371 INFO:tasks.workunit.client.1.smithi174.stdout:9/776: getdents d2/d11/d1d/d4b/d72 0 2024-03-21T10:09:06.372 INFO:tasks.workunit.client.0.smithi153.stdout:7/946: write d4/d2a/d40/f4c [6097717,122044] 0 2024-03-21T10:09:06.375 INFO:tasks.workunit.client.1.smithi174.stdout:6/615: link d0/d3/d5/df/fae d0/d3/d2a/fbf 0 2024-03-21T10:09:06.377 INFO:tasks.workunit.client.1.smithi174.stdout:7/669: fsync d0/d4/d10/d1b/f3e 0 2024-03-21T10:09:06.378 INFO:tasks.workunit.client.1.smithi174.stdout:6/616: dread - d0/d3/d5/d3b/fb1 zero size 2024-03-21T10:09:06.392 INFO:tasks.workunit.client.1.smithi174.stdout:9/777: creat d2/d24/d2e/de6/f111 x:0 0 0 2024-03-21T10:09:06.396 INFO:tasks.workunit.client.1.smithi174.stdout:3/718: write d0/d12/d14/d1a/f23 [2194069,111096] 0 2024-03-21T10:09:06.397 INFO:tasks.workunit.client.1.smithi174.stdout:4/835: write d4/d21/d24/d28/f5e [1581795,50497] 0 2024-03-21T10:09:06.399 INFO:tasks.workunit.client.1.smithi174.stdout:1/862: write d4/d7/d33/d43/d77/fc2 [840903,58130] 0 2024-03-21T10:09:06.402 INFO:tasks.workunit.client.1.smithi174.stdout:2/704: dwrite d5/d25/d27/d2d/d45/d5b/d55/fbf [4194304,4194304] 0 2024-03-21T10:09:06.403 INFO:tasks.workunit.client.1.smithi174.stdout:8/689: write d9/d53/f6c [2135017,114570] 0 2024-03-21T10:09:06.407 INFO:tasks.workunit.client.1.smithi174.stdout:9/778: truncate d2/d11/f2c 4572597 0 2024-03-21T10:09:06.409 INFO:tasks.workunit.client.1.smithi174.stdout:3/719: chown d0/d12/d14/d1a/d27/d3c/f81 228 1 2024-03-21T10:09:06.410 INFO:tasks.workunit.client.1.smithi174.stdout:0/732: dwrite d9/d21/d31/d5c/d86/fbf [0,4194304] 0 2024-03-21T10:09:06.410 INFO:tasks.workunit.client.1.smithi174.stdout:5/775: fsync d1/d16/d2e/d66/f89 0 2024-03-21T10:09:06.416 INFO:tasks.workunit.client.1.smithi174.stdout:9/779: mkdir d2/d10b/d112 0 2024-03-21T10:09:06.419 INFO:tasks.workunit.client.1.smithi174.stdout:7/670: dwrite d0/d4/d10/d1b/d86/dd2/fd0 [0,4194304] 0 2024-03-21T10:09:06.421 INFO:tasks.workunit.client.1.smithi174.stdout:1/863: rename d4/d67 to d4/d7/d132 0 2024-03-21T10:09:06.428 INFO:tasks.workunit.client.1.smithi174.stdout:6/617: dwrite d0/d3/d2a/fbf [4194304,4194304] 0 2024-03-21T10:09:06.428 INFO:tasks.workunit.client.1.smithi174.stdout:4/836: symlink d4/d21/d24/d28/de9/d3a/dd2/l117 0 2024-03-21T10:09:06.429 INFO:tasks.workunit.client.1.smithi174.stdout:9/780: write d2/d11/d1d/d2a/da7/ff1 [978809,111927] 0 2024-03-21T10:09:06.430 INFO:tasks.workunit.client.1.smithi174.stdout:0/733: creat d9/d21/d31/d5c/d92/f100 x:0 0 0 2024-03-21T10:09:06.438 INFO:tasks.workunit.client.1.smithi174.stdout:8/690: mkdir d9/d2a/d39/de1 0 2024-03-21T10:09:06.441 INFO:tasks.workunit.client.1.smithi174.stdout:8/691: fsync d9/d2a/d4d/fca 0 2024-03-21T10:09:06.441 INFO:tasks.workunit.client.1.smithi174.stdout:6/618: read d0/d3/d5/d6/f38 [176842,100861] 0 2024-03-21T10:09:06.441 INFO:tasks.workunit.client.1.smithi174.stdout:3/720: dread d0/d12/d14/d1a/f3f [0,4194304] 0 2024-03-21T10:09:06.444 INFO:tasks.workunit.client.1.smithi174.stdout:7/671: creat d0/d4/d10/d79/db2/dda/d5f/d65/fe8 x:0 0 0 2024-03-21T10:09:06.449 INFO:tasks.workunit.client.1.smithi174.stdout:8/692: chown d9/d2a/d39/d76/c35 1084457 1 2024-03-21T10:09:06.449 INFO:tasks.workunit.client.1.smithi174.stdout:9/781: unlink d2/d11/d60/f77 0 2024-03-21T10:09:06.450 INFO:tasks.workunit.client.1.smithi174.stdout:8/693: chown d9/d2a 2066194 1 2024-03-21T10:09:06.451 INFO:tasks.workunit.client.1.smithi174.stdout:2/705: write d5/d25/d27/d2d/d45/dae/dd6/d4f/f6d [708400,41395] 0 2024-03-21T10:09:06.453 INFO:tasks.workunit.client.1.smithi174.stdout:2/706: stat d5/d25/d27/d2d/d45/lca 0 2024-03-21T10:09:06.458 INFO:tasks.workunit.client.1.smithi174.stdout:5/776: mkdir d1/d49/d4f/d51/d8f/d118 0 2024-03-21T10:09:06.463 INFO:tasks.workunit.client.1.smithi174.stdout:1/864: rename d4/d7/d68/d7b/c108 to d4/d7/d33/d43/d77/d122/ddc/c133 0 2024-03-21T10:09:06.464 INFO:tasks.workunit.client.1.smithi174.stdout:4/837: rmdir d4/d21/d24 39 2024-03-21T10:09:06.465 INFO:tasks.workunit.client.1.smithi174.stdout:8/694: chown d9/dc6/f4f 0 1 2024-03-21T10:09:06.469 INFO:tasks.workunit.client.1.smithi174.stdout:2/707: fsync d5/d25/d27/d2d/d45/d5b/d2f/d4b/d76/f97 0 2024-03-21T10:09:06.469 INFO:tasks.workunit.client.1.smithi174.stdout:4/838: read d4/d19/f23 [332061,113358] 0 2024-03-21T10:09:06.469 INFO:tasks.workunit.client.1.smithi174.stdout:0/734: dwrite d9/d21/d31/d5c/dd5/fd6 [0,4194304] 0 2024-03-21T10:09:06.470 INFO:tasks.workunit.client.1.smithi174.stdout:5/777: rmdir d1/d16/d2e/d87 39 2024-03-21T10:09:06.470 INFO:tasks.workunit.client.1.smithi174.stdout:7/672: dwrite d0/d4/d10/d79/db2/dda/d5f/d65/fb5 [0,4194304] 0 2024-03-21T10:09:06.474 INFO:tasks.workunit.client.1.smithi174.stdout:6/619: rename d0/d3/d5/d6/f4a to d0/d3/d20/d3d/d81/fc0 0 2024-03-21T10:09:06.477 INFO:tasks.workunit.client.1.smithi174.stdout:8/695: unlink d9/d53/d5b/fd5 0 2024-03-21T10:09:06.480 INFO:tasks.workunit.client.1.smithi174.stdout:3/721: truncate d0/d12/f17 1764550 0 2024-03-21T10:09:06.482 INFO:tasks.workunit.client.1.smithi174.stdout:3/722: chown d0/d12/d14/fa5 14926958 1 2024-03-21T10:09:06.491 INFO:tasks.workunit.client.1.smithi174.stdout:2/708: creat d5/d25/d27/d2d/d45/d5b/d55/d75/ff4 x:0 0 0 2024-03-21T10:09:06.492 INFO:tasks.workunit.client.1.smithi174.stdout:1/865: write d4/d7/d68/d8e/db8/fbb [446502,66181] 0 2024-03-21T10:09:06.498 INFO:tasks.workunit.client.1.smithi174.stdout:7/673: mknod d0/d9e/ce9 0 2024-03-21T10:09:06.504 INFO:tasks.workunit.client.1.smithi174.stdout:3/723: rename d0/d12/d14/d1a/d27/d3c/d73 to d0/d12/d14/da9/de7 0 2024-03-21T10:09:06.504 INFO:tasks.workunit.client.1.smithi174.stdout:9/782: link d2/d11/d1d/lb0 d2/d24/d2e/l113 0 2024-03-21T10:09:06.506 INFO:tasks.workunit.client.1.smithi174.stdout:7/674: stat d0/d4/d10/d44/d81/d9a/fa3 0 2024-03-21T10:09:06.507 INFO:tasks.workunit.client.1.smithi174.stdout:2/709: mkdir d5/d25/d27/d2d/d79/df5 0 2024-03-21T10:09:06.508 INFO:tasks.workunit.client.1.smithi174.stdout:3/724: stat d0/f91 0 2024-03-21T10:09:06.510 INFO:tasks.workunit.client.1.smithi174.stdout:5/778: dwrite d1/d16/d2e/fe9 [0,4194304] 0 2024-03-21T10:09:06.517 INFO:tasks.workunit.client.1.smithi174.stdout:4/839: dwrite d4/d21/d24/d28/de9/d3a/d7a/d9d/da3/f107 [0,4194304] 0 2024-03-21T10:09:06.517 INFO:tasks.workunit.client.1.smithi174.stdout:0/735: creat d9/d21/d31/d9f/f101 x:0 0 0 2024-03-21T10:09:06.521 INFO:tasks.workunit.client.1.smithi174.stdout:6/620: link d0/d3/d2a/d56/d5e/fa7 d0/d3/d20/d3d/fc1 0 2024-03-21T10:09:06.521 INFO:tasks.workunit.client.1.smithi174.stdout:7/675: mkdir d0/d4/d10/d1b/d86/dea 0 2024-03-21T10:09:06.535 INFO:tasks.workunit.client.1.smithi174.stdout:7/676: mknod d0/d9e/ceb 0 2024-03-21T10:09:06.535 INFO:tasks.workunit.client.1.smithi174.stdout:8/696: dread d9/d2a/d39/d76/d19/d27/d60/fb5 [0,4194304] 0 2024-03-21T10:09:06.538 INFO:tasks.workunit.client.1.smithi174.stdout:7/677: chown d0/f52 1147748178 1 2024-03-21T10:09:06.543 INFO:tasks.workunit.client.1.smithi174.stdout:1/866: write d4/d7/d15/d4f/f61 [278037,74843] 0 2024-03-21T10:09:06.548 INFO:tasks.workunit.client.1.smithi174.stdout:1/867: chown d4/d2d/fe1 909503 1 2024-03-21T10:09:06.551 INFO:tasks.workunit.client.1.smithi174.stdout:7/678: mkdir d0/d4/d38/dec 0 2024-03-21T10:09:06.552 INFO:tasks.workunit.client.1.smithi174.stdout:8/697: rename d9/d2a/d39/d76/d19/cbf to d9/d2a/d39/d76/d19/d27/d60/daa/ce2 0 2024-03-21T10:09:06.552 INFO:tasks.workunit.client.1.smithi174.stdout:6/621: dwrite d0/d3/d5/df/fac [0,4194304] 0 2024-03-21T10:09:06.553 INFO:tasks.workunit.client.1.smithi174.stdout:4/840: link d4/d19/f74 d4/d21/d24/d28/de9/d3a/d77/dcf/f118 0 2024-03-21T10:09:06.553 INFO:tasks.workunit.client.1.smithi174.stdout:9/783: write d2/d11/d7d/fbd [925283,17321] 0 2024-03-21T10:09:06.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:06 smithi153 ceph-mon[29117]: pgmap v19: 65 pgs: 65 active+clean; 3.4 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 121 MiB/s rd, 333 MiB/s wr, 673 op/s 2024-03-21T10:09:06.557 INFO:tasks.workunit.client.1.smithi174.stdout:7/679: dread - d0/d4/d38/da9/dc6/fca zero size 2024-03-21T10:09:06.558 INFO:tasks.workunit.client.1.smithi174.stdout:5/779: getdents d1/d16/d2e/d8d/d99 0 2024-03-21T10:09:06.559 INFO:tasks.workunit.client.1.smithi174.stdout:4/841: dread - d4/d21/d24/d28/fed zero size 2024-03-21T10:09:06.560 INFO:tasks.workunit.client.1.smithi174.stdout:2/710: dwrite d5/d25/d27/d2d/d45/d5b/fb6 [0,4194304] 0 2024-03-21T10:09:06.569 INFO:tasks.workunit.client.1.smithi174.stdout:6/622: fsync d0/d3/d5/f1c 0 2024-03-21T10:09:06.572 INFO:tasks.workunit.client.1.smithi174.stdout:3/725: dread d0/f7 [0,4194304] 0 2024-03-21T10:09:06.578 INFO:tasks.workunit.client.1.smithi174.stdout:9/784: unlink d2/d11/d1d/d4b/fa8 0 2024-03-21T10:09:06.585 INFO:tasks.workunit.client.1.smithi174.stdout:7/680: read - d0/d4/d38/f70 zero size 2024-03-21T10:09:06.587 INFO:tasks.workunit.client.1.smithi174.stdout:1/868: mkdir d4/d7/d33/d43/d51/da4/d134 0 2024-03-21T10:09:06.588 INFO:tasks.workunit.client.1.smithi174.stdout:0/736: dwrite d9/d21/d31/f4c [0,4194304] 0 2024-03-21T10:09:06.589 INFO:tasks.workunit.client.1.smithi174.stdout:8/698: mknod d9/d2a/d4d/ce3 0 2024-03-21T10:09:06.590 INFO:tasks.workunit.client.1.smithi174.stdout:2/711: symlink d5/d25/d27/d2d/d45/dae/dd6/d4f/d65/d77/lf6 0 2024-03-21T10:09:06.592 INFO:tasks.workunit.client.1.smithi174.stdout:5/780: write d1/d49/d93/d9c/fce [9323676,36990] 0 2024-03-21T10:09:06.592 INFO:tasks.workunit.client.1.smithi174.stdout:6/623: fdatasync d0/d3/d20/d3d/fc1 0 2024-03-21T10:09:06.596 INFO:tasks.workunit.client.1.smithi174.stdout:3/726: unlink d0/d12/d14/d1a/d27/d4f/fc1 0 2024-03-21T10:09:06.597 INFO:tasks.workunit.client.1.smithi174.stdout:2/712: chown d5/d25/d27/d2d/d45/dae/dd6/d4f/d65/d77/dcf/fd5 32 1 2024-03-21T10:09:06.599 INFO:tasks.workunit.client.1.smithi174.stdout:4/842: symlink d4/d21/d24/d28/de9/d6a/d8f/l119 0 2024-03-21T10:09:06.600 INFO:tasks.workunit.client.1.smithi174.stdout:7/681: creat d0/d4/d10/d44/d81/d9a/fed x:0 0 0 2024-03-21T10:09:06.607 INFO:tasks.workunit.client.1.smithi174.stdout:8/699: creat d9/dc6/fe4 x:0 0 0 2024-03-21T10:09:06.612 INFO:tasks.workunit.client.1.smithi174.stdout:6/624: mkdir d0/d3/d5/df/d16/da9/db7/dc2 0 2024-03-21T10:09:06.612 INFO:tasks.workunit.client.1.smithi174.stdout:5/781: read d1/f2c [2226641,23851] 0 2024-03-21T10:09:06.616 INFO:tasks.workunit.client.1.smithi174.stdout:8/700: write d9/d2a/d4d/db4/fb6 [134046,5963] 0 2024-03-21T10:09:06.616 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:06 smithi174 ceph-mon[37496]: pgmap v19: 65 pgs: 65 active+clean; 3.4 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 121 MiB/s rd, 333 MiB/s wr, 673 op/s 2024-03-21T10:09:06.619 INFO:tasks.workunit.client.1.smithi174.stdout:1/869: write d4/d7/d15/d4f/f82 [1200988,112921] 0 2024-03-21T10:09:06.622 INFO:tasks.workunit.client.1.smithi174.stdout:9/785: dread d2/d24/d2e/f31 [0,4194304] 0 2024-03-21T10:09:06.623 INFO:tasks.workunit.client.1.smithi174.stdout:3/727: write d0/f70 [3017701,55581] 0 2024-03-21T10:09:06.624 INFO:tasks.workunit.client.1.smithi174.stdout:5/782: creat d1/d49/d93/f119 x:0 0 0 2024-03-21T10:09:06.624 INFO:tasks.workunit.client.1.smithi174.stdout:6/625: symlink d0/d3/d5/d6/d42/lc3 0 2024-03-21T10:09:06.625 INFO:tasks.workunit.client.1.smithi174.stdout:1/870: read - d4/d7/d28/f131 zero size 2024-03-21T10:09:06.627 INFO:tasks.workunit.client.1.smithi174.stdout:8/701: mknod d9/d53/dc9/ce5 0 2024-03-21T10:09:06.633 INFO:tasks.workunit.client.1.smithi174.stdout:0/737: dwrite d9/d21/d75/da1/fa8 [0,4194304] 0 2024-03-21T10:09:06.634 INFO:tasks.workunit.client.0.smithi153.stdout:4/926: sync 2024-03-21T10:09:06.635 INFO:tasks.workunit.client.1.smithi174.stdout:8/702: read - d9/d2a/d39/d58/dbb/dc3/fde zero size 2024-03-21T10:09:06.639 INFO:tasks.workunit.client.0.smithi153.stdout:4/927: rmdir d11/d18/d50/d65 39 2024-03-21T10:09:06.640 INFO:tasks.workunit.client.1.smithi174.stdout:1/871: unlink d4/d7/d68/d89/f113 0 2024-03-21T10:09:06.644 INFO:tasks.workunit.client.1.smithi174.stdout:4/843: dwrite d4/d21/d24/d28/de9/d3a/d67/d76/fc0 [0,4194304] 0 2024-03-21T10:09:06.651 INFO:tasks.workunit.client.1.smithi174.stdout:4/844: truncate d4/d21/fff 754696 0 2024-03-21T10:09:06.655 INFO:tasks.workunit.client.1.smithi174.stdout:2/713: dread d5/d25/d27/f3b [4194304,4194304] 0 2024-03-21T10:09:06.656 INFO:tasks.workunit.client.1.smithi174.stdout:4/845: readlink d4/d21/d24/d28/lcb 0 2024-03-21T10:09:06.660 INFO:tasks.workunit.client.1.smithi174.stdout:4/846: stat d4/d21/d24/d28/de9/d6a/dd1 0 2024-03-21T10:09:06.660 INFO:tasks.workunit.client.1.smithi174.stdout:3/728: dwrite d0/fa [4194304,4194304] 0 2024-03-21T10:09:06.662 INFO:tasks.workunit.client.1.smithi174.stdout:0/738: dwrite d9/d21/fe7 [0,4194304] 0 2024-03-21T10:09:06.670 INFO:tasks.workunit.client.1.smithi174.stdout:9/786: write d2/f45 [1058217,73924] 0 2024-03-21T10:09:06.677 INFO:tasks.workunit.client.1.smithi174.stdout:4/847: creat d4/d21/d24/d28/de9/d3a/d77/dcf/f11a x:0 0 0 2024-03-21T10:09:06.677 INFO:tasks.workunit.client.1.smithi174.stdout:9/787: readlink d2/d11/l94 0 2024-03-21T10:09:06.686 INFO:tasks.workunit.client.1.smithi174.stdout:6/626: write d0/d3/d5/d3b/f61 [291053,40619] 0 2024-03-21T10:09:06.692 INFO:tasks.workunit.client.1.smithi174.stdout:6/627: creat d0/d3/d5/d3b/fc4 x:0 0 0 2024-03-21T10:09:06.692 INFO:tasks.workunit.client.1.smithi174.stdout:8/703: dread f8 [0,4194304] 0 2024-03-21T10:09:06.693 INFO:tasks.workunit.client.1.smithi174.stdout:3/729: dwrite d0/d12/d14/d98/fd6 [0,4194304] 0 2024-03-21T10:09:06.694 INFO:tasks.workunit.client.1.smithi174.stdout:6/628: fsync d0/d3/d5/d6/f13 0 2024-03-21T10:09:06.695 INFO:tasks.workunit.client.1.smithi174.stdout:8/704: chown d9/d2a/d39/d76/c35 248 1 2024-03-21T10:09:06.697 INFO:tasks.workunit.client.1.smithi174.stdout:6/629: dread - d0/d3/d5/d3b/fc4 zero size 2024-03-21T10:09:06.699 INFO:tasks.workunit.client.1.smithi174.stdout:8/705: mkdir d9/d53/dc9/de6 0 2024-03-21T10:09:06.701 INFO:tasks.workunit.client.1.smithi174.stdout:6/630: fsync d0/d3/d5/df/d16/f34 0 2024-03-21T10:09:06.703 INFO:tasks.workunit.client.1.smithi174.stdout:8/706: creat d9/d2a/d39/d58/dbb/dc3/fe7 x:0 0 0 2024-03-21T10:09:06.704 INFO:tasks.workunit.client.1.smithi174.stdout:6/631: creat d0/d3/d2a/fc5 x:0 0 0 2024-03-21T10:09:06.706 INFO:tasks.workunit.client.1.smithi174.stdout:8/707: creat d9/d53/fe8 x:0 0 0 2024-03-21T10:09:06.722 INFO:tasks.workunit.client.1.smithi174.stdout:0/739: dread d9/d21/d75/fc7 [0,4194304] 0 2024-03-21T10:09:06.723 INFO:tasks.workunit.client.1.smithi174.stdout:6/632: dwrite d0/d3/d5/df/d16/f27 [0,4194304] 0 2024-03-21T10:09:06.727 INFO:tasks.workunit.client.1.smithi174.stdout:6/633: write d0/d3/d5/d3b/f61 [361913,41198] 0 2024-03-21T10:09:06.734 INFO:tasks.workunit.client.1.smithi174.stdout:0/740: link d9/d21/c43 d9/d21/d31/d5c/d92/d98/db7/c102 0 2024-03-21T10:09:06.736 INFO:tasks.workunit.client.1.smithi174.stdout:0/741: write d9/d35/dd2/fe6 [425170,94260] 0 2024-03-21T10:09:06.742 INFO:tasks.workunit.client.1.smithi174.stdout:1/872: dread d4/d2d/f57 [0,4194304] 0 2024-03-21T10:09:06.753 INFO:tasks.workunit.client.1.smithi174.stdout:6/634: dwrite d0/d3/d5/df/d16/f1a [4194304,4194304] 0 2024-03-21T10:09:06.754 INFO:tasks.workunit.client.1.smithi174.stdout:4/848: write d4/d21/d24/d28/d38/f90 [4934543,31690] 0 2024-03-21T10:09:06.756 INFO:tasks.workunit.client.1.smithi174.stdout:3/730: write d0/d12/d14/da9/de7/fd7 [3717293,62507] 0 2024-03-21T10:09:06.758 INFO:tasks.workunit.client.1.smithi174.stdout:4/849: chown d4/d21/d24/d28/d79/cc6 2779494 1 2024-03-21T10:09:06.761 INFO:tasks.workunit.client.1.smithi174.stdout:6/635: symlink d0/d3/d5/d6/lc6 0 2024-03-21T10:09:06.761 INFO:tasks.workunit.client.1.smithi174.stdout:3/731: getdents d0/d12/d14/d67/dc8 0 2024-03-21T10:09:06.762 INFO:tasks.workunit.client.1.smithi174.stdout:6/636: readlink d0/d3/d5/df/l7b 0 2024-03-21T10:09:06.767 INFO:tasks.workunit.client.1.smithi174.stdout:4/850: creat d4/d21/d24/d28/de9/d3a/d67/d76/f11b x:0 0 0 2024-03-21T10:09:06.773 INFO:tasks.workunit.client.1.smithi174.stdout:6/637: rename d0/d3/d5/d3b/f85 to d0/d3/d5/df/d16/da9/db7/dc2/fc7 0 2024-03-21T10:09:06.775 INFO:tasks.workunit.client.1.smithi174.stdout:8/708: dwrite d9/dc6/f94 [0,4194304] 0 2024-03-21T10:09:06.782 INFO:tasks.workunit.client.1.smithi174.stdout:0/742: dwrite d9/d35/d4a/d67/d89/fc2 [0,4194304] 0 2024-03-21T10:09:06.785 INFO:tasks.workunit.client.0.smithi153.stdout:1/748: sync 2024-03-21T10:09:06.785 INFO:tasks.workunit.client.0.smithi153.stdout:9/963: sync 2024-03-21T10:09:06.785 INFO:tasks.workunit.client.0.smithi153.stdout:5/892: sync 2024-03-21T10:09:06.786 INFO:tasks.workunit.client.1.smithi174.stdout:6/638: creat d0/d3/d20/d3d/d81/da5/fc8 x:0 0 0 2024-03-21T10:09:06.787 INFO:tasks.workunit.client.1.smithi174.stdout:4/851: getdents d4/d21/d24/d28/d38 0 2024-03-21T10:09:06.790 INFO:tasks.workunit.client.1.smithi174.stdout:4/852: fdatasync d4/f8 0 2024-03-21T10:09:06.791 INFO:tasks.workunit.client.1.smithi174.stdout:0/743: stat d9/d21/d23/da5/cff 0 2024-03-21T10:09:06.795 INFO:tasks.workunit.client.0.smithi153.stdout:9/964: mknod d1/d3b/daa/c13f 0 2024-03-21T10:09:06.795 INFO:tasks.workunit.client.1.smithi174.stdout:4/853: dread - d4/d21/d24/d28/de9/d3a/f100 zero size 2024-03-21T10:09:06.797 INFO:tasks.workunit.client.1.smithi174.stdout:6/639: unlink d0/d3/d5/f3c 0 2024-03-21T10:09:06.800 INFO:tasks.workunit.client.1.smithi174.stdout:0/744: mkdir d9/d21/d31/d5c/d92/d98/db7/ddc/d103 0 2024-03-21T10:09:06.800 INFO:tasks.workunit.client.1.smithi174.stdout:4/854: fdatasync d4/d21/d24/ffc 0 2024-03-21T10:09:06.800 INFO:tasks.workunit.client.0.smithi153.stdout:5/893: creat d4/d8/d5a/d5d/dd9/d44/f127 x:0 0 0 2024-03-21T10:09:06.801 INFO:tasks.workunit.client.1.smithi174.stdout:3/732: dwrite d0/d12/d14/d1a/d27/f47 [0,4194304] 0 2024-03-21T10:09:06.802 INFO:tasks.workunit.client.1.smithi174.stdout:4/855: stat d4/d21/d24/d28/de9/d3a/d7a 0 2024-03-21T10:09:06.805 INFO:tasks.workunit.client.1.smithi174.stdout:6/640: rename d0/d3/d20/f23 to d0/d3/d20/d3d/d81/da5/fc9 0 2024-03-21T10:09:06.805 INFO:tasks.workunit.client.0.smithi153.stdout:1/749: creat d6/d10/d35/da5/d4f/d9d/db6/def/ff3 x:0 0 0 2024-03-21T10:09:06.806 INFO:tasks.workunit.client.1.smithi174.stdout:4/856: symlink d4/d19/l11c 0 2024-03-21T10:09:06.806 INFO:tasks.workunit.client.0.smithi153.stdout:9/965: creat d1/d43/d72/d9d/f140 x:0 0 0 2024-03-21T10:09:06.811 INFO:tasks.workunit.client.1.smithi174.stdout:0/745: rename d9/d21/d31/d3c to d9/d35/dd2/d104 0 2024-03-21T10:09:06.812 INFO:tasks.workunit.client.0.smithi153.stdout:1/750: creat d6/d10/d2c/ff4 x:0 0 0 2024-03-21T10:09:06.813 INFO:tasks.workunit.client.1.smithi174.stdout:8/709: write d9/d2a/d39/d76/d19/d27/f38 [1890175,29916] 0 2024-03-21T10:09:06.813 INFO:tasks.workunit.client.0.smithi153.stdout:9/966: mknod d1/d43/d44/d46/d9e/d2e/d67/c141 0 2024-03-21T10:09:06.814 INFO:tasks.workunit.client.0.smithi153.stdout:1/751: chown d6/d32/l49 3 1 2024-03-21T10:09:06.816 INFO:tasks.workunit.client.1.smithi174.stdout:4/857: creat d4/d19/dd5/f11d x:0 0 0 2024-03-21T10:09:06.816 INFO:tasks.workunit.client.1.smithi174.stdout:3/733: dread d0/d12/d14/d1a/d27/d7f/fc4 [0,4194304] 0 2024-03-21T10:09:06.816 INFO:tasks.workunit.client.1.smithi174.stdout:6/641: mknod d0/d3/d5/df/d66/d87/cca 0 2024-03-21T10:09:06.817 INFO:tasks.workunit.client.1.smithi174.stdout:0/746: fsync d9/d1c/f7d 0 2024-03-21T10:09:06.817 INFO:tasks.workunit.client.0.smithi153.stdout:9/967: rmdir d1/d43/d44/d46/d9e/d24/d41 39 2024-03-21T10:09:06.818 INFO:tasks.workunit.client.1.smithi174.stdout:8/710: mkdir d9/d2a/d39/d76/d19/de9 0 2024-03-21T10:09:06.820 INFO:tasks.workunit.client.1.smithi174.stdout:0/747: mkdir d9/d35/dd2/d105 0 2024-03-21T10:09:06.822 INFO:tasks.workunit.client.1.smithi174.stdout:6/642: symlink d0/d3/d5/df/d66/d87/lcb 0 2024-03-21T10:09:06.825 INFO:tasks.workunit.client.0.smithi153.stdout:1/752: rename d6/d3a/d3c to d6/d13/d8e/dbb/de1/de7/de9/df5 0 2024-03-21T10:09:06.826 INFO:tasks.workunit.client.1.smithi174.stdout:6/643: read - d0/d3/d20/d3d/d81/da5/fc8 zero size 2024-03-21T10:09:06.827 INFO:tasks.workunit.client.0.smithi153.stdout:9/968: stat d1/d43/d44/d46/d4d/d5c/cf5 0 2024-03-21T10:09:06.830 INFO:tasks.workunit.client.0.smithi153.stdout:5/894: dwrite d4/d8/d5a/d5d/dd9/d44/f127 [0,4194304] 0 2024-03-21T10:09:06.831 INFO:tasks.workunit.client.1.smithi174.stdout:6/644: mknod d0/d3/da8/ccc 0 2024-03-21T10:09:06.831 INFO:tasks.workunit.client.0.smithi153.stdout:1/753: rename d6/d3a/fdf to d6/de6/ff6 0 2024-03-21T10:09:06.831 INFO:tasks.workunit.client.0.smithi153.stdout:5/895: stat d4/d5/f7 0 2024-03-21T10:09:06.837 INFO:tasks.workunit.client.0.smithi153.stdout:1/754: chown d6/d10/d35/da5/d4f/d9d/dc7/dcc 744 1 2024-03-21T10:09:06.837 INFO:tasks.workunit.client.0.smithi153.stdout:9/969: symlink d1/d43/d44/l142 0 2024-03-21T10:09:06.837 INFO:tasks.workunit.client.1.smithi174.stdout:3/734: dwrite d0/d12/d14/d1a/d44/d65/d7e/dac/fc0 [0,4194304] 0 2024-03-21T10:09:06.839 INFO:tasks.workunit.client.1.smithi174.stdout:8/711: write d9/d53/d7e/f8e [2221911,12449] 0 2024-03-21T10:09:06.840 INFO:tasks.workunit.client.0.smithi153.stdout:5/896: rename d4/d5/df/d5e to d4/d8/d5a/d5d/dd9/d2f/d128 0 2024-03-21T10:09:06.843 INFO:tasks.workunit.client.0.smithi153.stdout:0/813: sync 2024-03-21T10:09:06.843 INFO:tasks.workunit.client.0.smithi153.stdout:7/947: sync 2024-03-21T10:09:06.843 INFO:tasks.workunit.client.0.smithi153.stdout:2/955: sync 2024-03-21T10:09:06.843 INFO:tasks.workunit.client.1.smithi174.stdout:6/645: write d0/d3/d20/d3d/fc1 [840247,105722] 0 2024-03-21T10:09:06.843 INFO:tasks.workunit.client.1.smithi174.stdout:8/712: readlink d9/d53/d5b/l71 0 2024-03-21T10:09:06.844 INFO:tasks.workunit.client.0.smithi153.stdout:3/986: sync 2024-03-21T10:09:06.844 INFO:tasks.workunit.client.1.smithi174.stdout:3/735: creat d0/d12/d14/db9/fe8 x:0 0 0 2024-03-21T10:09:06.845 INFO:tasks.workunit.client.0.smithi153.stdout:2/956: fsync dd/faf 0 2024-03-21T10:09:06.845 INFO:tasks.workunit.client.0.smithi153.stdout:4/928: sync 2024-03-21T10:09:06.845 INFO:tasks.workunit.client.0.smithi153.stdout:8/843: sync 2024-03-21T10:09:06.853 INFO:tasks.workunit.client.0.smithi153.stdout:0/814: unlink d4/c5 0 2024-03-21T10:09:06.854 INFO:tasks.workunit.client.1.smithi174.stdout:6/646: rename d0/d3/d5/df/d66/d87/cca to d0/d3/d20/d6d/ccd 0 2024-03-21T10:09:06.854 INFO:tasks.workunit.client.1.smithi174.stdout:0/748: dwrite d9/d35/d4a/f9e [0,4194304] 0 2024-03-21T10:09:06.855 INFO:tasks.workunit.client.1.smithi174.stdout:8/713: mkdir d9/d53/dc9/de6/dea 0 2024-03-21T10:09:06.856 INFO:tasks.workunit.client.0.smithi153.stdout:5/897: fdatasync d4/d5/f57 0 2024-03-21T10:09:06.857 INFO:tasks.workunit.client.0.smithi153.stdout:5/898: write d4/d5/df/d67/d86/fd7 [528522,85332] 0 2024-03-21T10:09:06.859 INFO:tasks.workunit.client.1.smithi174.stdout:0/749: dread d9/d21/d75/fc7 [0,4194304] 0 2024-03-21T10:09:06.861 INFO:tasks.workunit.client.0.smithi153.stdout:2/957: rmdir dd/d15/d18/d3a/d3d/d125 39 2024-03-21T10:09:06.862 INFO:tasks.workunit.client.0.smithi153.stdout:3/987: mkdir dd/d1e/d45/d141/d151 0 2024-03-21T10:09:06.863 INFO:tasks.workunit.client.0.smithi153.stdout:8/844: chown d5/d9/d43/dd/d20/d26/d8b/lf9 1306 1 2024-03-21T10:09:06.864 INFO:tasks.workunit.client.1.smithi174.stdout:0/750: creat d9/d1c/d76/d87/f106 x:0 0 0 2024-03-21T10:09:06.865 INFO:tasks.workunit.client.1.smithi174.stdout:6/647: link d0/d3/d5/c50 d0/d3/d2a/d56/d5e/cce 0 2024-03-21T10:09:06.865 INFO:tasks.workunit.client.0.smithi153.stdout:5/899: chown d4/d5/df/d67/da0/cf2 1030535 1 2024-03-21T10:09:06.869 INFO:tasks.workunit.client.0.smithi153.stdout:0/815: fsync d4/d16/d19/d30/d6c/d98/d5b/fa9 0 2024-03-21T10:09:06.870 INFO:tasks.workunit.client.0.smithi153.stdout:3/988: chown dd/d1e/d3d/l14b 0 1 2024-03-21T10:09:06.872 INFO:tasks.workunit.client.1.smithi174.stdout:3/736: dwrite d0/d12/f54 [4194304,4194304] 0 2024-03-21T10:09:06.876 INFO:tasks.workunit.client.1.smithi174.stdout:8/714: dwrite d9/d2a/d39/d58/fdf [0,4194304] 0 2024-03-21T10:09:06.877 INFO:tasks.workunit.client.0.smithi153.stdout:9/970: write d1/d4/d5f/fce [5134813,85626] 0 2024-03-21T10:09:06.877 INFO:tasks.workunit.client.0.smithi153.stdout:5/900: stat d4/d5/df/d67/d86/d96/deb/df0 0 2024-03-21T10:09:06.879 INFO:tasks.workunit.client.0.smithi153.stdout:4/929: dwrite d11/f6f [0,4194304] 0 2024-03-21T10:09:06.879 INFO:tasks.workunit.client.0.smithi153.stdout:1/755: write d6/d13/d8e/dbb/de1/de7/de9/df5/d50/fcd [981321,121686] 0 2024-03-21T10:09:06.881 INFO:tasks.workunit.client.0.smithi153.stdout:5/901: stat d4/d8/d5a/d5d/dd9/d2f 0 2024-03-21T10:09:06.883 INFO:tasks.workunit.client.0.smithi153.stdout:0/816: creat d4/d16/d19/d30/f104 x:0 0 0 2024-03-21T10:09:06.885 INFO:tasks.workunit.client.0.smithi153.stdout:3/989: unlink dd/f62 0 2024-03-21T10:09:06.886 INFO:tasks.workunit.client.1.smithi174.stdout:6/648: dread d0/d3/d5/df/d16/f27 [0,4194304] 0 2024-03-21T10:09:06.886 INFO:tasks.workunit.client.0.smithi153.stdout:8/845: rmdir d5/d9/d43/dd/d20/d26/d24/d52/d55 39 2024-03-21T10:09:06.893 INFO:tasks.workunit.client.1.smithi174.stdout:6/649: chown d0/d3/d5/df/f4e 1 1 2024-03-21T10:09:06.894 INFO:tasks.workunit.client.0.smithi153.stdout:4/930: mkdir d11/db1/d123 0 2024-03-21T10:09:06.896 INFO:tasks.workunit.client.1.smithi174.stdout:0/751: dwrite d9/d21/d31/d5c/d92/fdb [0,4194304] 0 2024-03-21T10:09:06.898 INFO:tasks.workunit.client.0.smithi153.stdout:1/756: mknod d6/d10/d35/da5/d4f/d9d/db6/def/cf7 0 2024-03-21T10:09:06.898 INFO:tasks.workunit.client.1.smithi174.stdout:0/752: readlink d9/d21/d23/da5/lad 0 2024-03-21T10:09:06.902 INFO:tasks.workunit.client.0.smithi153.stdout:8/846: unlink d5/d9/l65 0 2024-03-21T10:09:06.905 INFO:tasks.workunit.client.1.smithi174.stdout:8/715: dwrite d9/d53/d7e/faf [0,4194304] 0 2024-03-21T10:09:06.905 INFO:tasks.workunit.client.0.smithi153.stdout:7/948: truncate d4/d14/d16/fed 2067562 0 2024-03-21T10:09:06.905 INFO:tasks.workunit.client.0.smithi153.stdout:0/817: rmdir d4/d16/d27/dc5/dcd/de8 39 2024-03-21T10:09:06.911 INFO:tasks.workunit.client.0.smithi153.stdout:3/990: dwrite dd/f22 [0,4194304] 0 2024-03-21T10:09:06.912 INFO:tasks.workunit.client.0.smithi153.stdout:9/971: rename d1/d43/d44/d46/d9e/d4b/f73 to d1/d4/d5f/df3/d11a/f143 0 2024-03-21T10:09:06.916 INFO:tasks.workunit.client.0.smithi153.stdout:2/958: truncate dd/d15/d18/d2c/d33/f7d 2660327 0 2024-03-21T10:09:06.926 INFO:tasks.workunit.client.0.smithi153.stdout:8/847: rename d5/d9/d43/dd/d20/d26/d24/f82 to d5/d9/d43/dd/ddd/f10d 0 2024-03-21T10:09:06.935 INFO:tasks.workunit.client.0.smithi153.stdout:7/949: creat d4/d19/f141 x:0 0 0 2024-03-21T10:09:06.950 INFO:tasks.workunit.client.0.smithi153.stdout:3/991: creat dd/d1e/d2f/d35/daf/dcc/dd6/f152 x:0 0 0 2024-03-21T10:09:06.951 INFO:tasks.workunit.client.0.smithi153.stdout:7/950: dwrite d4/d14/dd0/d83/f89 [0,4194304] 0 2024-03-21T10:09:06.959 INFO:tasks.workunit.client.0.smithi153.stdout:2/959: creat dd/d15/d18/d2c/d4b/d56/d63/d8d/d9b/dc3/df5/dae/d50/d59/d5d/f144 x:0 0 0 2024-03-21T10:09:06.967 INFO:tasks.workunit.client.0.smithi153.stdout:5/902: dwrite d4/d8/d5c/d90/d95/f99 [0,4194304] 0 2024-03-21T10:09:06.967 INFO:tasks.workunit.client.0.smithi153.stdout:1/757: write d6/d10/da0/fa1 [415736,42197] 0 2024-03-21T10:09:06.971 INFO:tasks.workunit.client.0.smithi153.stdout:1/758: fsync d6/d10/fc1 0 2024-03-21T10:09:06.979 INFO:tasks.workunit.client.0.smithi153.stdout:3/992: link dd/d1e/d2f/d35/dd0/f11d dd/d1e/d3d/d11b/f153 0 2024-03-21T10:09:06.981 INFO:tasks.workunit.client.0.smithi153.stdout:8/848: write d5/d1b/f37 [473137,72508] 0 2024-03-21T10:09:06.982 INFO:tasks.workunit.client.0.smithi153.stdout:5/903: rmdir d4/d5/df/d67 39 2024-03-21T10:09:06.982 INFO:tasks.workunit.client.0.smithi153.stdout:0/818: dwrite d4/db/f29 [4194304,4194304] 0 2024-03-21T10:09:06.984 INFO:tasks.workunit.client.0.smithi153.stdout:2/960: read dd/d15/d18/d2c/d4b/d56/d63/d8d/d9b/dc3/df5/dae/f77 [3125114,57787] 0 2024-03-21T10:09:06.990 INFO:tasks.workunit.client.0.smithi153.stdout:2/961: write dd/d15/d18/d2c/d33/d98/fb5 [285535,103180] 0 2024-03-21T10:09:06.996 INFO:tasks.workunit.client.1.smithi174.stdout:3/737: rename d0/d12/d14/d4c/d7a/cd0 to d0/d12/d14/da9/de7/db7/d7d/ce9 0 2024-03-21T10:09:06.997 INFO:tasks.workunit.client.0.smithi153.stdout:9/972: dwrite d1/d43/d44/d46/d9e/d24/d41/f109 [0,4194304] 0 2024-03-21T10:09:06.998 INFO:tasks.workunit.client.1.smithi174.stdout:6/650: unlink d0/d3/d2a/d56/d5e/d91/fa4 0 2024-03-21T10:09:06.999 INFO:tasks.workunit.client.0.smithi153.stdout:7/951: dread d4/f25 [0,4194304] 0 2024-03-21T10:09:07.000 INFO:tasks.workunit.client.1.smithi174.stdout:0/753: symlink d9/d21/d5f/l107 0 2024-03-21T10:09:07.001 INFO:tasks.workunit.client.0.smithi153.stdout:7/952: chown d4/d14/d110/l132 14636 1 2024-03-21T10:09:07.001 INFO:tasks.workunit.client.1.smithi174.stdout:8/716: mknod d9/d2a/d39/d58/dbb/ceb 0 2024-03-21T10:09:07.008 INFO:tasks.workunit.client.1.smithi174.stdout:8/717: read d9/d2a/d4d/fca [1736400,83803] 0 2024-03-21T10:09:07.009 INFO:tasks.workunit.client.1.smithi174.stdout:8/718: chown d9/d53/d5b/c74 2997301 1 2024-03-21T10:09:07.010 INFO:tasks.workunit.client.1.smithi174.stdout:8/719: fdatasync d9/f12 0 2024-03-21T10:09:07.015 INFO:tasks.workunit.client.0.smithi153.stdout:2/962: rmdir dd/d15/d18/d2c/d4b/d56/d63/d8d/d9b/dc3 39 2024-03-21T10:09:07.018 INFO:tasks.workunit.client.0.smithi153.stdout:3/993: fsync dd/d1e/d2f/f3b 0 2024-03-21T10:09:07.020 INFO:tasks.workunit.client.1.smithi174.stdout:8/720: dread d9/dc6/dd1/fa6 [0,4194304] 0 2024-03-21T10:09:07.021 INFO:tasks.workunit.client.0.smithi153.stdout:8/849: dwrite d5/d9/d43/d7b/daa/fb8 [0,4194304] 0 2024-03-21T10:09:07.022 INFO:tasks.workunit.client.0.smithi153.stdout:1/759: write d6/d13/d8e/dbb/de1/de7/de9/df5/f8d [3351656,10426] 0 2024-03-21T10:09:07.024 INFO:tasks.workunit.client.1.smithi174.stdout:3/738: mknod d0/cea 0 2024-03-21T10:09:07.026 INFO:tasks.workunit.client.0.smithi153.stdout:0/819: creat d4/d4c/d83/d22/d61/d77/df1/f105 x:0 0 0 2024-03-21T10:09:07.027 INFO:tasks.workunit.client.0.smithi153.stdout:8/850: dread - d5/d9/d43/dd/d20/d26/d8b/ffd zero size 2024-03-21T10:09:07.028 INFO:tasks.workunit.client.0.smithi153.stdout:2/963: rmdir dd/d15/d18/d3a/d3d/ddc 39 2024-03-21T10:09:07.031 INFO:tasks.workunit.client.0.smithi153.stdout:9/973: mkdir d1/d43/d44/d144 0 2024-03-21T10:09:07.033 INFO:tasks.workunit.client.0.smithi153.stdout:5/904: mknod d4/d5/df/d67/d86/d96/deb/c129 0 2024-03-21T10:09:07.036 INFO:tasks.workunit.client.0.smithi153.stdout:3/994: symlink dd/d1e/d2f/l154 0 2024-03-21T10:09:07.038 INFO:tasks.workunit.client.1.smithi174.stdout:8/721: dwrite d9/d2a/d4d/db4/fdd [0,4194304] 0 2024-03-21T10:09:07.040 INFO:tasks.workunit.client.1.smithi174.stdout:8/722: stat d9/d2a/d39/d58/fdf 0 2024-03-21T10:09:07.041 INFO:tasks.workunit.client.0.smithi153.stdout:1/760: creat d6/d10/d35/da5/ff8 x:0 0 0 2024-03-21T10:09:07.041 INFO:tasks.workunit.client.1.smithi174.stdout:8/723: dread - d9/d2a/d39/d76/fc1 zero size 2024-03-21T10:09:07.042 INFO:tasks.workunit.client.0.smithi153.stdout:5/905: mknod d4/d8/d5a/d92/d9f/c12a 0 2024-03-21T10:09:07.043 INFO:tasks.workunit.client.0.smithi153.stdout:9/974: creat d1/f145 x:0 0 0 2024-03-21T10:09:07.044 INFO:tasks.workunit.client.0.smithi153.stdout:2/964: truncate dd/d15/d18/d2c/d4b/d56/d63/d8d/d9b/fa5 1007588 0 2024-03-21T10:09:07.046 INFO:tasks.workunit.client.0.smithi153.stdout:7/953: write d4/d2a/d3b/f98 [91137,56473] 0 2024-03-21T10:09:07.048 INFO:tasks.workunit.client.1.smithi174.stdout:7/682: sync 2024-03-21T10:09:07.050 INFO:tasks.workunit.client.1.smithi174.stdout:8/724: creat d9/d2a/fec x:0 0 0 2024-03-21T10:09:07.050 INFO:tasks.workunit.client.0.smithi153.stdout:1/761: mknod d6/d13/cf9 0 2024-03-21T10:09:07.051 INFO:tasks.workunit.client.0.smithi153.stdout:0/820: write d4/d16/d19/d30/d6c/d98/d5e/db2/fce [464815,91282] 0 2024-03-21T10:09:07.053 INFO:tasks.workunit.client.1.smithi174.stdout:7/683: creat d0/d4/d10/d60/da2/fee x:0 0 0 2024-03-21T10:09:07.054 INFO:tasks.workunit.client.0.smithi153.stdout:7/954: creat d4/d11/d56/db1/f142 x:0 0 0 2024-03-21T10:09:07.056 INFO:tasks.workunit.client.0.smithi153.stdout:2/965: write dd/d15/d18/d3a/d3d/f74 [5223212,26521] 0 2024-03-21T10:09:07.057 INFO:tasks.workunit.client.1.smithi174.stdout:7/684: write d0/d4/d10/d79/db2/dda/d5f/d65/fe8 [432832,56813] 0 2024-03-21T10:09:07.058 INFO:tasks.workunit.client.0.smithi153.stdout:0/821: mknod d4/d16/d19/d30/d6c/c106 0 2024-03-21T10:09:07.060 INFO:tasks.workunit.client.1.smithi174.stdout:7/685: stat d0/d4/d10/d44/f72 0 2024-03-21T10:09:07.061 INFO:tasks.workunit.client.1.smithi174.stdout:6/651: write d0/d3/d5/df/f4e [697357,27251] 0 2024-03-21T10:09:07.064 INFO:tasks.workunit.client.1.smithi174.stdout:3/739: write d0/f13 [1217825,65913] 0 2024-03-21T10:09:07.064 INFO:tasks.workunit.client.1.smithi174.stdout:8/725: write d9/d53/f6f [4112225,13022] 0 2024-03-21T10:09:07.064 INFO:tasks.workunit.client.0.smithi153.stdout:0/822: fsync d4/d16/d19/d58/fbf 0 2024-03-21T10:09:07.067 INFO:tasks.workunit.client.0.smithi153.stdout:7/955: rename d4/d2a/d40/ceb to d4/d19/d4d/dd5/c143 0 2024-03-21T10:09:07.067 INFO:tasks.workunit.client.0.smithi153.stdout:3/995: dwrite dd/f2c [0,4194304] 0 2024-03-21T10:09:07.069 INFO:tasks.workunit.client.0.smithi153.stdout:2/966: fsync dd/d15/d18/d2c/d4b/d56/f7b 0 2024-03-21T10:09:07.071 INFO:tasks.workunit.client.0.smithi153.stdout:7/956: rmdir d4/d2a 39 2024-03-21T10:09:07.072 INFO:tasks.workunit.client.1.smithi174.stdout:8/726: mknod d9/d2a/d39/d76/d19/d27/d60/daa/dab/ced 0 2024-03-21T10:09:07.072 INFO:tasks.workunit.client.0.smithi153.stdout:3/996: write dd/d1e/d2f/d35/d68/ff6 [4989648,9952] 0 2024-03-21T10:09:07.075 INFO:tasks.workunit.client.1.smithi174.stdout:0/754: dread d9/d1c/f1e [0,4194304] 0 2024-03-21T10:09:07.077 INFO:tasks.workunit.client.0.smithi153.stdout:3/997: read - dd/d1e/d3d/d53/dae/d126/f149 zero size 2024-03-21T10:09:07.078 INFO:tasks.workunit.client.1.smithi174.stdout:7/686: link d0/d4/d10/d1b/d86/l97 d0/d4/d10/d1b/d86/dea/lef 0 2024-03-21T10:09:07.078 INFO:tasks.workunit.client.0.smithi153.stdout:7/957: fsync d4/d2a/dc6/dfb/fe9 0 2024-03-21T10:09:07.081 INFO:tasks.workunit.client.0.smithi153.stdout:7/958: dread - d4/d2a/dc6/dfb/fe9 zero size 2024-03-21T10:09:07.082 INFO:tasks.workunit.client.1.smithi174.stdout:7/687: fdatasync d0/d4/d10/d44/d81/d9a/fcb 0 2024-03-21T10:09:07.083 INFO:tasks.workunit.client.0.smithi153.stdout:9/975: dread d1/d4e/fe9 [0,4194304] 0 2024-03-21T10:09:07.085 INFO:tasks.workunit.client.0.smithi153.stdout:7/959: chown d4/d19/d4d/d10e/d121/f7b 497462 1 2024-03-21T10:09:07.086 INFO:tasks.workunit.client.1.smithi174.stdout:6/652: dwrite d0/d3/d2a/d56/d5e/d91/fb3 [0,4194304] 0 2024-03-21T10:09:07.088 INFO:tasks.workunit.client.0.smithi153.stdout:9/976: chown d1/d3b/df8/dfa 226 1 2024-03-21T10:09:07.089 INFO:tasks.workunit.client.0.smithi153.stdout:7/960: chown d4/d2a/d3b/d81/c12d 22091 1 2024-03-21T10:09:07.090 INFO:tasks.workunit.client.0.smithi153.stdout:8/851: dread d5/d9/d43/dd/d20/d26/d24/fa3 [0,4194304] 0 2024-03-21T10:09:07.092 INFO:tasks.workunit.client.0.smithi153.stdout:2/967: rename dd/d15/d18/d2c/d4b/d56/d63/d8d/d9b/dc3/df5/dae/d50/d59/f137 to dd/d15/d18/dca/f145 0 2024-03-21T10:09:07.095 INFO:tasks.workunit.client.0.smithi153.stdout:0/823: write d4/f6 [1879657,56438] 0 2024-03-21T10:09:07.099 INFO:tasks.workunit.client.0.smithi153.stdout:2/968: stat dd/d15/d18/d2c/d4b/d56/d63/d8d/d9b/dc3/df5/dae/d50/d59/d5d/fef 0 2024-03-21T10:09:07.101 INFO:tasks.workunit.client.0.smithi153.stdout:2/969: write dd/d15/d18/d2c/d4b/fce [179463,90532] 0 2024-03-21T10:09:07.105 INFO:tasks.workunit.client.0.smithi153.stdout:5/906: dread d4/d5/f7 [0,4194304] 0 2024-03-21T10:09:07.106 INFO:tasks.workunit.client.0.smithi153.stdout:3/998: truncate dd/d41/d80/dbb/fc1 875430 0 2024-03-21T10:09:07.106 INFO:tasks.workunit.client.1.smithi174.stdout:8/727: write d9/d2a/d4d/db2/fc7 [152125,124077] 0 2024-03-21T10:09:07.107 INFO:tasks.workunit.client.1.smithi174.stdout:3/740: dwrite d0/f5f [0,4194304] 0 2024-03-21T10:09:07.109 INFO:tasks.workunit.client.1.smithi174.stdout:8/728: fsync d9/dc6/fac 0 2024-03-21T10:09:07.109 INFO:tasks.workunit.client.0.smithi153.stdout:5/907: dread - d4/d5/df/d67/d86/ff4 zero size 2024-03-21T10:09:07.109 INFO:tasks.workunit.client.0.smithi153.stdout:9/977: creat d1/d4/d5f/f146 x:0 0 0 2024-03-21T10:09:07.110 INFO:tasks.workunit.client.1.smithi174.stdout:3/741: chown d0/d12/d14/d1a/d27/d3c/l84 1703107 1 2024-03-21T10:09:07.112 INFO:tasks.workunit.client.0.smithi153.stdout:8/852: creat d5/f10e x:0 0 0 2024-03-21T10:09:07.113 INFO:tasks.workunit.client.0.smithi153.stdout:2/970: mkdir dd/d15/d18/d2c/d4b/d56/d63/d8d/d9b/dc3/df5/dae/d50/d58/d79/d146 0 2024-03-21T10:09:07.113 INFO:tasks.workunit.client.1.smithi174.stdout:6/653: symlink d0/d3/d20/lcf 0 2024-03-21T10:09:07.114 INFO:tasks.workunit.client.0.smithi153.stdout:8/853: chown d5/d1b/lff 0 1 2024-03-21T10:09:07.115 INFO:tasks.workunit.client.1.smithi174.stdout:3/742: write d0/d12/d14/d1a/f34 [1263006,79490] 0 2024-03-21T10:09:07.118 INFO:tasks.workunit.client.0.smithi153.stdout:3/999: mknod dd/d1e/d3d/d53/dca/c155 0 2024-03-21T10:09:07.118 INFO:tasks.workunit.client.0.smithi153.stdout:5/908: creat d4/d8/d5a/d92/d9f/f12b x:0 0 0 2024-03-21T10:09:07.119 INFO:tasks.workunit.client.1.smithi174.stdout:6/654: fdatasync d0/d3/d20/d3d/d81/da5/fc8 0 2024-03-21T10:09:07.119 INFO:tasks.workunit.client.0.smithi153.stdout:0/824: dread d4/db/fa5 [0,4194304] 0 2024-03-21T10:09:07.119 INFO:tasks.workunit.client.1.smithi174.stdout:8/729: rename d9/d2a/d39/d76/d19/d27/d60/daa/ce2 to d9/d53/d5b/cee 0 2024-03-21T10:09:07.121 INFO:tasks.workunit.client.1.smithi174.stdout:3/743: mknod d0/d12/d14/d1a/ceb 0 2024-03-21T10:09:07.122 INFO:tasks.workunit.client.1.smithi174.stdout:8/730: stat d9/d2a/d39/d76/fa2 0 2024-03-21T10:09:07.122 INFO:tasks.workunit.client.1.smithi174.stdout:6/655: readlink d0/d3/d5/df/d66/d87/lcb 0 2024-03-21T10:09:07.122 INFO:tasks.workunit.client.0.smithi153.stdout:2/971: readlink dd/d15/d18/d2c/d4b/d56/d63/d8d/d9b/dc3/df5/dae/d50/d58/d79/lcd 0 2024-03-21T10:09:07.125 INFO:tasks.workunit.client.1.smithi174.stdout:7/688: dwrite d0/d4/d38/f66 [0,4194304] 0 2024-03-21T10:09:07.125 INFO:tasks.workunit.client.1.smithi174.stdout:3/744: mknod d0/d12/d46/d59/cec 0 2024-03-21T10:09:07.127 INFO:tasks.workunit.client.0.smithi153.stdout:0/825: chown d4/d4c/d83/d22/f7d 115139 1 2024-03-21T10:09:07.127 INFO:tasks.workunit.client.0.smithi153.stdout:7/961: truncate d4/d14/dd0/d83/fdf 537906 0 2024-03-21T10:09:07.128 INFO:tasks.workunit.client.0.smithi153.stdout:8/854: dread d5/d9/d43/dd/d2d/d45/fc1 [0,4194304] 0 2024-03-21T10:09:07.129 INFO:tasks.workunit.client.1.smithi174.stdout:3/745: write d0/d12/d14/d1a/d27/f47 [2058437,59805] 0 2024-03-21T10:09:07.133 INFO:tasks.workunit.client.0.smithi153.stdout:0/826: write d4/d16/d27/f90 [1796955,89708] 0 2024-03-21T10:09:07.134 INFO:tasks.workunit.client.1.smithi174.stdout:8/731: rename d9/d2a/d4d/l59 to d9/d2a/d39/d58/dbb/lef 0 2024-03-21T10:09:07.134 INFO:tasks.workunit.client.1.smithi174.stdout:7/689: mkdir d0/d4/d10/d1b/d86/df0 0 2024-03-21T10:09:07.134 INFO:tasks.workunit.client.0.smithi153.stdout:5/909: rmdir d4/d5/df/d67/d86/d96/deb/df0/dfa 39 2024-03-21T10:09:07.138 INFO:tasks.workunit.client.0.smithi153.stdout:9/978: write d1/d43/d44/d46/d13b/d130/fb3 [579477,48622] 0 2024-03-21T10:09:07.139 INFO:tasks.workunit.client.1.smithi174.stdout:8/732: creat d9/d53/d5b/ff0 x:0 0 0 2024-03-21T10:09:07.141 INFO:tasks.workunit.client.0.smithi153.stdout:0/827: chown d4/d4c/d83/l2a 2723493 1 2024-03-21T10:09:07.141 INFO:tasks.workunit.client.1.smithi174.stdout:3/746: rename d0/f2 to d0/d12/d14/da9/de7/db7/fed 0 2024-03-21T10:09:07.144 INFO:tasks.workunit.client.0.smithi153.stdout:8/855: dread d5/d9/d43/dd/d20/d26/d47/d9a/db9/fcb [0,4194304] 0 2024-03-21T10:09:07.147 INFO:tasks.workunit.client.1.smithi174.stdout:7/690: creat d0/d4/d10/d44/d81/d9a/ff1 x:0 0 0 2024-03-21T10:09:07.147 INFO:tasks.workunit.client.1.smithi174.stdout:8/733: mkdir d9/dc6/df1 0 2024-03-21T10:09:07.154 INFO:tasks.workunit.client.0.smithi153.stdout:9/979: truncate d1/d43/d44/fff 4338884 0 2024-03-21T10:09:07.155 INFO:tasks.workunit.client.1.smithi174.stdout:7/691: rmdir d0/d4/d10/d1b/d86 39 2024-03-21T10:09:07.156 INFO:tasks.workunit.client.0.smithi153.stdout:9/980: fsync d1/d4/d5f/df3/d11a/f139 0 2024-03-21T10:09:07.159 INFO:tasks.workunit.client.1.smithi174.stdout:8/734: unlink d9/d2a/d39/d76/d19/d27/d60/daa/dab/cd3 0 2024-03-21T10:09:07.159 INFO:tasks.workunit.client.0.smithi153.stdout:9/981: read - d1/d3b/df8/f12e zero size 2024-03-21T10:09:07.160 INFO:tasks.workunit.client.0.smithi153.stdout:8/856: write d5/f2f [9326304,99364] 0 2024-03-21T10:09:07.161 INFO:tasks.workunit.client.1.smithi174.stdout:3/747: dwrite d0/d12/d14/db9/fe8 [0,4194304] 0 2024-03-21T10:09:07.162 INFO:tasks.workunit.client.1.smithi174.stdout:6/656: dwrite d0/d3/d5/d6/f38 [0,4194304] 0 2024-03-21T10:09:07.165 INFO:tasks.workunit.client.1.smithi174.stdout:8/735: readlink d9/d2a/d39/d76/l57 0 2024-03-21T10:09:07.166 INFO:tasks.workunit.client.1.smithi174.stdout:6/657: chown d0/d3/d5/f57 107 1 2024-03-21T10:09:07.167 INFO:tasks.workunit.client.0.smithi153.stdout:7/962: dread d4/d2a/f5b [0,4194304] 0 2024-03-21T10:09:07.169 INFO:tasks.workunit.client.0.smithi153.stdout:4/931: sync 2024-03-21T10:09:07.170 INFO:tasks.workunit.client.0.smithi153.stdout:8/857: mkdir d5/d9/d43/dd/d20/d26/d47/d9a/d10f 0 2024-03-21T10:09:07.172 INFO:tasks.workunit.client.1.smithi174.stdout:7/692: rename d0/d4/d10/d79/db2/dda/c90 to d0/d4/d10/d1b/d86/dd2/cf2 0 2024-03-21T10:09:07.174 INFO:tasks.workunit.client.0.smithi153.stdout:2/972: dread fb [0,4194304] 0 2024-03-21T10:09:07.177 INFO:tasks.workunit.client.1.smithi174.stdout:6/658: rmdir d0/d3/d5 39 2024-03-21T10:09:07.178 INFO:tasks.workunit.client.1.smithi174.stdout:7/693: dread - d0/d4/d10/d1b/f6f zero size 2024-03-21T10:09:07.179 INFO:tasks.workunit.client.1.smithi174.stdout:8/736: read d9/f1a [3193113,29861] 0 2024-03-21T10:09:07.180 INFO:tasks.workunit.client.0.smithi153.stdout:7/963: creat d4/d19/d4d/f144 x:0 0 0 2024-03-21T10:09:07.182 INFO:tasks.workunit.client.1.smithi174.stdout:8/737: chown d9/dc6 300 1 2024-03-21T10:09:07.185 INFO:tasks.workunit.client.0.smithi153.stdout:4/932: rmdir d11/d18/d6d/dd6/ddd 39 2024-03-21T10:09:07.185 INFO:tasks.workunit.client.1.smithi174.stdout:6/659: creat d0/d3/d5/df/d16/da9/fd0 x:0 0 0 2024-03-21T10:09:07.186 INFO:tasks.workunit.client.0.smithi153.stdout:1/762: sync 2024-03-21T10:09:07.186 INFO:tasks.workunit.client.1.smithi174.stdout:8/738: read d9/f30 [891708,4636] 0 2024-03-21T10:09:07.187 INFO:tasks.workunit.client.0.smithi153.stdout:7/964: mkdir d4/d2a/dc6/dfb/d9a/dd2/db9/d145 0 2024-03-21T10:09:07.190 INFO:tasks.workunit.client.0.smithi153.stdout:8/858: mkdir d5/d9/d43/d7b/d110 0 2024-03-21T10:09:07.191 INFO:tasks.workunit.client.0.smithi153.stdout:5/910: dread d4/d5/df/f58 [0,4194304] 0 2024-03-21T10:09:07.192 INFO:tasks.workunit.client.1.smithi174.stdout:6/660: write d0/d3/d5/df/f2e [1400789,97916] 0 2024-03-21T10:09:07.193 INFO:tasks.workunit.client.1.smithi174.stdout:3/748: dwrite d0/d12/d14/d1a/d27/d3c/f5a [0,4194304] 0 2024-03-21T10:09:07.193 INFO:tasks.workunit.client.0.smithi153.stdout:4/933: creat d11/d18/d7e/dc5/f124 x:0 0 0 2024-03-21T10:09:07.194 INFO:tasks.workunit.client.0.smithi153.stdout:0/828: dread d4/d16/d19/d58/d9a/fb4 [0,4194304] 0 2024-03-21T10:09:07.196 INFO:tasks.workunit.client.1.smithi174.stdout:3/749: stat d0/d12/d14/d67/dc8/ldc 0 2024-03-21T10:09:07.197 INFO:tasks.workunit.client.0.smithi153.stdout:2/973: rename dd/d15/d18/d2c/d4b/d56/d63/d8d/d9b/dc3/df5/dae/d50/d58/l68 to dd/d15/d18/d3a/d10d/l147 0 2024-03-21T10:09:07.198 INFO:tasks.workunit.client.1.smithi174.stdout:7/694: dwrite d0/f11 [0,4194304] 0 2024-03-21T10:09:07.200 INFO:tasks.workunit.client.0.smithi153.stdout:8/859: write d5/d9/d43/dd/ff5 [295864,43975] 0 2024-03-21T10:09:07.202 INFO:tasks.workunit.client.1.smithi174.stdout:3/750: mkdir d0/d12/d14/d4c/d7a/d9e/dee 0 2024-03-21T10:09:07.202 INFO:tasks.workunit.client.0.smithi153.stdout:7/965: rmdir d4/d19/d4d/dd5/df7 39 2024-03-21T10:09:07.204 INFO:tasks.workunit.client.0.smithi153.stdout:9/982: dwrite d1/d43/d44/d46/d4d/d5c/ff6 [0,4194304] 0 2024-03-21T10:09:07.205 INFO:tasks.workunit.client.0.smithi153.stdout:5/911: rmdir d4/d5 39 2024-03-21T10:09:07.205 INFO:tasks.workunit.client.1.smithi174.stdout:3/751: chown d0/d12/d14/da9/de7/db7/l3e 949297 1 2024-03-21T10:09:07.208 INFO:tasks.workunit.client.1.smithi174.stdout:1/873: sync 2024-03-21T10:09:07.209 INFO:tasks.workunit.client.1.smithi174.stdout:4/858: sync 2024-03-21T10:09:07.209 INFO:tasks.workunit.client.1.smithi174.stdout:2/714: sync 2024-03-21T10:09:07.209 INFO:tasks.workunit.client.1.smithi174.stdout:5/783: sync 2024-03-21T10:09:07.209 INFO:tasks.workunit.client.1.smithi174.stdout:9/788: sync 2024-03-21T10:09:07.209 INFO:tasks.workunit.client.1.smithi174.stdout:8/739: dwrite d9/d2a/d4d/f50 [0,4194304] 0 2024-03-21T10:09:07.214 INFO:tasks.workunit.client.1.smithi174.stdout:8/740: readlink d9/d2a/d39/l4b 0 2024-03-21T10:09:07.214 INFO:tasks.workunit.client.1.smithi174.stdout:9/789: read d2/f45 [3677378,19661] 0 2024-03-21T10:09:07.216 INFO:tasks.workunit.client.0.smithi153.stdout:2/974: symlink dd/d15/d18/d2c/d33/d98/dc7/de1/dfe/l148 0 2024-03-21T10:09:07.216 INFO:tasks.workunit.client.0.smithi153.stdout:8/860: write d5/d9/d43/dd/d20/d26/d8b/f4f [1037522,19585] 0 2024-03-21T10:09:07.218 INFO:tasks.workunit.client.1.smithi174.stdout:1/874: creat d4/d7/d15/f135 x:0 0 0 2024-03-21T10:09:07.219 INFO:tasks.workunit.client.1.smithi174.stdout:7/695: write d0/d4/d10/d79/fe5 [1240214,103283] 0 2024-03-21T10:09:07.223 INFO:tasks.workunit.client.0.smithi153.stdout:1/763: dread d6/d10/f73 [0,4194304] 0 2024-03-21T10:09:07.223 INFO:tasks.workunit.client.1.smithi174.stdout:3/752: truncate d0/d12/d14/da9/de7/db7/d7d/fe5 178615 0 2024-03-21T10:09:07.224 INFO:tasks.workunit.client.0.smithi153.stdout:7/966: read d4/d14/dd0/d83/fdf [114644,59543] 0 2024-03-21T10:09:07.224 INFO:tasks.workunit.client.0.smithi153.stdout:4/934: dread d11/d2d/d3d/f90 [0,4194304] 0 2024-03-21T10:09:07.226 INFO:tasks.workunit.client.1.smithi174.stdout:9/790: write d2/d11/d1d/d4b/d72/fee [457551,84263] 0 2024-03-21T10:09:07.228 INFO:tasks.workunit.client.1.smithi174.stdout:1/875: write d4/d5/f105 [818977,78687] 0 2024-03-21T10:09:07.229 INFO:tasks.workunit.client.1.smithi174.stdout:2/715: creat d5/d25/d27/d2d/d45/dae/dd6/d4f/d65/d77/ff7 x:0 0 0 2024-03-21T10:09:07.229 INFO:tasks.workunit.client.0.smithi153.stdout:2/975: fdatasync dd/d15/d5e/f8f 0 2024-03-21T10:09:07.229 INFO:tasks.workunit.client.1.smithi174.stdout:5/784: rename d1/d16/c28 to d1/d49/d4f/de4/c11a 0 2024-03-21T10:09:07.230 INFO:tasks.workunit.client.0.smithi153.stdout:0/829: creat d4/d16/d19/dad/d100/f107 x:0 0 0 2024-03-21T10:09:07.231 INFO:tasks.workunit.client.0.smithi153.stdout:7/967: fsync d4/d14/d16/f11a 0 2024-03-21T10:09:07.232 INFO:tasks.workunit.client.1.smithi174.stdout:8/741: mkdir d9/d2a/d39/d76/d19/d27/d60/daa/df2 0 2024-03-21T10:09:07.232 INFO:tasks.workunit.client.0.smithi153.stdout:1/764: dread - d6/d13/d8e/fda zero size 2024-03-21T10:09:07.233 INFO:tasks.workunit.client.1.smithi174.stdout:9/791: fdatasync d2/d11/d1d/d4b/f56 0 2024-03-21T10:09:07.236 INFO:tasks.workunit.client.1.smithi174.stdout:5/785: chown d1/d5c 597 1 2024-03-21T10:09:07.237 INFO:tasks.workunit.client.0.smithi153.stdout:4/935: mknod d11/d2d/d3d/da1/c125 0 2024-03-21T10:09:07.238 INFO:tasks.workunit.client.1.smithi174.stdout:3/753: rmdir d0/d12/d14/d4c/d7a/d9e 39 2024-03-21T10:09:07.238 INFO:tasks.workunit.client.1.smithi174.stdout:4/859: getdents d4/d21/d24/d28/de9/d3a/d7a/d9d/da3 0 2024-03-21T10:09:07.240 INFO:tasks.workunit.client.1.smithi174.stdout:1/876: dread d4/d7/d32/da2/ffc [0,4194304] 0 2024-03-21T10:09:07.242 INFO:tasks.workunit.client.0.smithi153.stdout:7/968: write d4/d14/d16/f11a [908465,73660] 0 2024-03-21T10:09:07.244 INFO:tasks.workunit.client.0.smithi153.stdout:4/936: write d11/d18/d6d/f121 [2486264,86632] 0 2024-03-21T10:09:07.245 INFO:tasks.workunit.client.1.smithi174.stdout:5/786: mknod d1/d49/d93/d9c/c11b 0 2024-03-21T10:09:07.249 INFO:tasks.workunit.client.0.smithi153.stdout:7/969: chown d4/d2a/dc6/dfb/d9a/db7/f96 5639 1 2024-03-21T10:09:07.250 INFO:tasks.workunit.client.0.smithi153.stdout:1/765: rename d6/d13/l24 to d6/d10/d35/da5/d4f/d9d/dc7/lfa 0 2024-03-21T10:09:07.251 INFO:tasks.workunit.client.1.smithi174.stdout:4/860: symlink d4/d21/d24/d28/de9/d6a/d8f/d9c/dfd/l11e 0 2024-03-21T10:09:07.251 INFO:tasks.workunit.client.1.smithi174.stdout:6/661: dread d0/d3/d2a/d56/d5e/f73 [0,4194304] 0 2024-03-21T10:09:07.254 INFO:tasks.workunit.client.1.smithi174.stdout:4/861: readlink d4/l17 0 2024-03-21T10:09:07.255 INFO:tasks.workunit.client.1.smithi174.stdout:2/716: rename d5/d25/d27/d2d/d45/d5b/d2f/d68/de8/fec to d5/d25/d27/d2d/d45/d5b/d2f/d4b/d76/ff8 0 2024-03-21T10:09:07.259 INFO:tasks.workunit.client.0.smithi153.stdout:5/912: dwrite d4/d5/df/d67/d86/d96/deb/df0/d10a/f126 [0,4194304] 0 2024-03-21T10:09:07.262 INFO:tasks.workunit.client.1.smithi174.stdout:1/877: link d4/d7/d15/f135 d4/d7/d68/d7b/de0/de4/f136 0 2024-03-21T10:09:07.263 INFO:tasks.workunit.client.1.smithi174.stdout:3/754: dread d0/d12/d14/f80 [0,4194304] 0 2024-03-21T10:09:07.264 INFO:tasks.workunit.client.0.smithi153.stdout:1/766: unlink d6/d10/d35/da5/f4b 0 2024-03-21T10:09:07.264 INFO:tasks.workunit.client.1.smithi174.stdout:6/662: creat d0/d3/d5/df/d16/fd1 x:0 0 0 2024-03-21T10:09:07.266 INFO:tasks.workunit.client.1.smithi174.stdout:4/862: mkdir d4/d21/d24/d28/de9/d3a/d7a/d9d/d11f 0 2024-03-21T10:09:07.267 INFO:tasks.workunit.client.1.smithi174.stdout:0/755: sync 2024-03-21T10:09:07.267 INFO:tasks.workunit.client.1.smithi174.stdout:9/792: dwrite d2/d24/d2e/df6/fff [0,4194304] 0 2024-03-21T10:09:07.267 INFO:tasks.workunit.client.0.smithi153.stdout:0/830: dread d4/d16/d19/d58/d9a/ffc [0,4194304] 0 2024-03-21T10:09:07.270 INFO:tasks.workunit.client.0.smithi153.stdout:2/976: dread dd/d15/d3c/f69 [0,4194304] 0 2024-03-21T10:09:07.271 INFO:tasks.workunit.client.1.smithi174.stdout:9/793: write d2/d11/d1d/f96 [8151828,55486] 0 2024-03-21T10:09:07.271 INFO:tasks.workunit.client.1.smithi174.stdout:1/878: creat d4/d7/d68/d89/dc6/f137 x:0 0 0 2024-03-21T10:09:07.271 INFO:tasks.workunit.client.1.smithi174.stdout:2/717: mkdir d5/d25/d27/d2d/d45/d5b/d2f/d4b/da7/df9 0 2024-03-21T10:09:07.276 INFO:tasks.workunit.client.0.smithi153.stdout:7/970: getdents d4/d2a/dc6/dfb/d9a/dd2/d133 0 2024-03-21T10:09:07.276 INFO:tasks.workunit.client.1.smithi174.stdout:8/742: dwrite d9/d2a/d39/d76/f20 [0,4194304] 0 2024-03-21T10:09:07.278 INFO:tasks.workunit.client.1.smithi174.stdout:3/755: unlink d0/d12/d14/d1a/d27/d3c/l84 0 2024-03-21T10:09:07.279 INFO:tasks.workunit.client.0.smithi153.stdout:9/983: dwrite d1/d43/d44/d46/d9e/d24/d41/dbe/dcc/fa4 [0,4194304] 0 2024-03-21T10:09:07.281 INFO:tasks.workunit.client.1.smithi174.stdout:8/743: fsync d9/d2a/d39/d76/d19/d27/d60/daa/fb9 0 2024-03-21T10:09:07.283 INFO:tasks.workunit.client.0.smithi153.stdout:5/913: write d4/d5/df/d67/d86/d96/deb/d56/fca [4229678,115954] 0 2024-03-21T10:09:07.283 INFO:tasks.workunit.client.0.smithi153.stdout:4/937: dwrite d11/d2d/fd0 [4194304,4194304] 0 2024-03-21T10:09:07.284 INFO:tasks.workunit.client.1.smithi174.stdout:8/744: chown d9/d2a/d39/d58/fdf 128346896 1 2024-03-21T10:09:07.287 INFO:tasks.workunit.client.0.smithi153.stdout:1/767: mknod d6/d13/d8e/dbb/de1/de7/de9/df5/d50/da8/cfb 0 2024-03-21T10:09:07.288 INFO:tasks.workunit.client.1.smithi174.stdout:8/745: chown d9/d2a/d4d/ce3 666 1 2024-03-21T10:09:07.289 INFO:tasks.workunit.client.1.smithi174.stdout:9/794: symlink d2/d11/d1d/dab/dc1/l114 0 2024-03-21T10:09:07.289 INFO:tasks.workunit.client.0.smithi153.stdout:2/977: chown dd/d15/d18/d3a/d3d/ddc 1 1 2024-03-21T10:09:07.290 INFO:tasks.workunit.client.1.smithi174.stdout:4/863: truncate d4/d21/f49 152260 0 2024-03-21T10:09:07.292 INFO:tasks.workunit.client.1.smithi174.stdout:5/787: fsync d1/fe1 0 2024-03-21T10:09:07.292 INFO:tasks.workunit.client.1.smithi174.stdout:6/663: dread d0/d3/d20/d3d/fc1 [0,4194304] 0 2024-03-21T10:09:07.293 INFO:tasks.workunit.client.1.smithi174.stdout:2/718: mknod d5/d25/d27/d2d/d45/dae/dd6/d90/da9/cfa 0 2024-03-21T10:09:07.296 INFO:tasks.workunit.client.1.smithi174.stdout:6/664: fdatasync d0/d3/d5/d3b/fc4 0 2024-03-21T10:09:07.297 INFO:tasks.workunit.client.0.smithi153.stdout:1/768: write d6/d10/d2c/f47 [1506455,64942] 0 2024-03-21T10:09:07.299 INFO:tasks.workunit.client.1.smithi174.stdout:2/719: write d5/d25/d27/d2d/d45/dae/fd9 [881013,58142] 0 2024-03-21T10:09:07.299 INFO:tasks.workunit.client.1.smithi174.stdout:0/756: mknod d9/da2/c108 0 2024-03-21T10:09:07.300 INFO:tasks.workunit.client.0.smithi153.stdout:4/938: fdatasync d11/d2d/f5a 0 2024-03-21T10:09:07.302 INFO:tasks.workunit.client.1.smithi174.stdout:7/696: dwrite d0/d4/f8 [0,4194304] 0 2024-03-21T10:09:07.302 INFO:tasks.workunit.client.0.smithi153.stdout:8/861: dwrite d5/d9/d43/dd/d20/d26/d24/d52/f72 [0,4194304] 0 2024-03-21T10:09:07.305 INFO:tasks.workunit.client.1.smithi174.stdout:4/864: dread - d4/d21/d24/d28/de9/d3a/dd2/fe6 zero size 2024-03-21T10:09:07.309 INFO:tasks.workunit.client.1.smithi174.stdout:6/665: rename d0/d3/d20/d3d/f59 to d0/d3/da8/fd2 0 2024-03-21T10:09:07.310 INFO:tasks.workunit.client.1.smithi174.stdout:9/795: mkdir d2/d10b/d112/d115 0 2024-03-21T10:09:07.312 INFO:tasks.workunit.client.0.smithi153.stdout:2/978: read dd/f2e [2443444,31177] 0 2024-03-21T10:09:07.312 INFO:tasks.workunit.client.0.smithi153.stdout:9/984: write d1/d43/d44/d46/d9e/d24/f11b [1654229,93987] 0 2024-03-21T10:09:07.313 INFO:tasks.workunit.client.1.smithi174.stdout:4/865: fsync d4/d21/d24/d28/de9/d3a/d67/d76/fc0 0 2024-03-21T10:09:07.316 INFO:tasks.workunit.client.0.smithi153.stdout:4/939: mknod d11/d2d/da3/dac/c126 0 2024-03-21T10:09:07.318 INFO:tasks.workunit.client.1.smithi174.stdout:0/757: mknod d9/d21/d31/d5c/d92/d98/db7/ddc/c109 0 2024-03-21T10:09:07.318 INFO:tasks.workunit.client.1.smithi174.stdout:1/879: link d4/d5/c10 d4/d7/d33/d43/c138 0 2024-03-21T10:09:07.320 INFO:tasks.workunit.client.1.smithi174.stdout:6/666: chown d0/d3/d5/d6/d42/f58 1 1 2024-03-21T10:09:07.322 INFO:tasks.workunit.client.1.smithi174.stdout:4/866: write d4/d21/d24/d28/de9/d3a/d7a/d9d/da3/f107 [4974963,2169] 0 2024-03-21T10:09:07.327 INFO:tasks.workunit.client.0.smithi153.stdout:1/769: mkdir d6/d13/d75/dfc 0 2024-03-21T10:09:07.328 INFO:tasks.workunit.client.1.smithi174.stdout:0/758: creat d9/dd8/f10a x:0 0 0 2024-03-21T10:09:07.329 INFO:tasks.workunit.client.0.smithi153.stdout:8/862: fdatasync d5/d9/d43/dd/d2d/f31 0 2024-03-21T10:09:07.329 INFO:tasks.workunit.client.1.smithi174.stdout:1/880: creat d4/d7/d33/d43/d77/d87/f139 x:0 0 0 2024-03-21T10:09:07.332 INFO:tasks.workunit.client.1.smithi174.stdout:6/667: creat d0/d3/d5/df/fd3 x:0 0 0 2024-03-21T10:09:07.334 INFO:tasks.workunit.client.1.smithi174.stdout:4/867: mkdir d4/d21/d24/d28/de9/d3a/d7a/d9d/d11f/d120 0 2024-03-21T10:09:07.339 INFO:tasks.workunit.client.1.smithi174.stdout:9/796: dwrite d2/d11/f10e [0,4194304] 0 2024-03-21T10:09:07.344 INFO:tasks.workunit.client.0.smithi153.stdout:5/914: dread d4/d5/df/d67/da0/db3/f8c [0,4194304] 0 2024-03-21T10:09:07.344 INFO:tasks.workunit.client.1.smithi174.stdout:5/788: dread d1/d2f/fae [4194304,4194304] 0 2024-03-21T10:09:07.348 INFO:tasks.workunit.client.0.smithi153.stdout:7/971: dwrite d4/d2a/d3b/d81/df6/f112 [4194304,4194304] 0 2024-03-21T10:09:07.348 INFO:tasks.workunit.client.1.smithi174.stdout:5/789: mknod d1/d113/c11c 0 2024-03-21T10:09:07.357 INFO:tasks.workunit.client.1.smithi174.stdout:5/790: dread - d1/d2f/fd0 zero size 2024-03-21T10:09:07.362 INFO:tasks.workunit.client.1.smithi174.stdout:9/797: creat d2/d11/d1d/d5e/f116 x:0 0 0 2024-03-21T10:09:07.362 INFO:tasks.workunit.client.1.smithi174.stdout:0/759: dwrite d9/d35/dd2/d104/fb0 [4194304,4194304] 0 2024-03-21T10:09:07.362 INFO:tasks.workunit.client.1.smithi174.stdout:1/881: dwrite d4/d7/d15/d53/f54 [0,4194304] 0 2024-03-21T10:09:07.363 INFO:tasks.workunit.client.1.smithi174.stdout:1/882: chown d4/d7/d68/d7b/de0/de4/le6 724443 1 2024-03-21T10:09:07.364 INFO:tasks.workunit.client.1.smithi174.stdout:0/760: stat d9/d35/dd2/d104/l4d 0 2024-03-21T10:09:07.365 INFO:tasks.workunit.client.1.smithi174.stdout:4/868: dwrite d4/f8 [0,4194304] 0 2024-03-21T10:09:07.370 INFO:tasks.workunit.client.1.smithi174.stdout:9/798: chown d2/d24/fef 78381 1 2024-03-21T10:09:07.372 INFO:tasks.workunit.client.1.smithi174.stdout:3/756: truncate d0/d12/d14/d1a/d44/d65/f6b 1714045 0 2024-03-21T10:09:07.374 INFO:tasks.workunit.client.0.smithi153.stdout:5/915: dread d4/d8/d5c/d90/f113 [0,4194304] 0 2024-03-21T10:09:07.378 INFO:tasks.workunit.client.1.smithi174.stdout:0/761: fdatasync d9/d21/d75/da1/fb8 0 2024-03-21T10:09:07.379 INFO:tasks.workunit.client.0.smithi153.stdout:4/940: dread fd [0,4194304] 0 2024-03-21T10:09:07.386 INFO:tasks.workunit.client.1.smithi174.stdout:9/799: symlink d2/d11/d7d/l117 0 2024-03-21T10:09:07.389 INFO:tasks.workunit.client.1.smithi174.stdout:0/762: chown d9/d21/d31/d5c/d92/d98/db7 164886 1 2024-03-21T10:09:07.392 INFO:tasks.workunit.client.1.smithi174.stdout:0/763: chown d9/d1c/d76/d87/faa 755403 1 2024-03-21T10:09:07.392 INFO:tasks.workunit.client.0.smithi153.stdout:0/831: dread d4/f10 [0,4194304] 0 2024-03-21T10:09:07.396 INFO:tasks.workunit.client.1.smithi174.stdout:4/869: unlink d4/d21/d24/d28/de9/d3a/d7a/d9d/ca6 0 2024-03-21T10:09:07.396 INFO:tasks.workunit.client.1.smithi174.stdout:7/697: dread d0/d4/f22 [0,4194304] 0 2024-03-21T10:09:07.404 INFO:tasks.workunit.client.1.smithi174.stdout:4/870: readlink d4/d21/d24/d28/ld8 0 2024-03-21T10:09:07.408 INFO:tasks.workunit.client.1.smithi174.stdout:9/800: creat d2/d11/f118 x:0 0 0 2024-03-21T10:09:07.410 INFO:tasks.workunit.client.0.smithi153.stdout:1/770: symlink d6/d10/d35/lfd 0 2024-03-21T10:09:07.413 INFO:tasks.workunit.client.1.smithi174.stdout:1/883: dwrite d4/d7/d33/d43/d77/d87/d123/f12b [0,4194304] 0 2024-03-21T10:09:07.416 INFO:tasks.workunit.client.0.smithi153.stdout:7/972: creat d4/d19/d102/f146 x:0 0 0 2024-03-21T10:09:07.416 INFO:tasks.workunit.client.0.smithi153.stdout:5/916: creat d4/d8/d5a/d5d/dd9/d2f/f12c x:0 0 0 2024-03-21T10:09:07.416 INFO:tasks.workunit.client.0.smithi153.stdout:4/941: fdatasync d11/d2d/da3/dac/fc2 0 2024-03-21T10:09:07.417 INFO:tasks.workunit.client.0.smithi153.stdout:0/832: creat d4/d4c/d83/d22/d61/f108 x:0 0 0 2024-03-21T10:09:07.421 INFO:tasks.workunit.client.1.smithi174.stdout:1/884: stat d4/d7/d15/d53/d78/ca6 0 2024-03-21T10:09:07.421 INFO:tasks.workunit.client.1.smithi174.stdout:2/720: dwrite d5/d25/d27/d2d/d45/d5b/d2f/d4b/d76/ff8 [4194304,4194304] 0 2024-03-21T10:09:07.421 INFO:tasks.workunit.client.0.smithi153.stdout:8/863: creat d5/d9/d43/d7b/dc4/da7/f111 x:0 0 0 2024-03-21T10:09:07.421 INFO:tasks.workunit.client.1.smithi174.stdout:5/791: write d1/d16/d2e/d66/d8b/d85/fcb [55958,101377] 0 2024-03-21T10:09:07.422 INFO:tasks.workunit.client.0.smithi153.stdout:1/771: rename d6/d13/f45 to d6/d10/da0/ffe 0 2024-03-21T10:09:07.422 INFO:tasks.workunit.client.1.smithi174.stdout:4/871: dread - d4/d21/d24/d28/de9/d6a/d8f/d9c/fb2 zero size 2024-03-21T10:09:07.426 INFO:tasks.workunit.client.1.smithi174.stdout:9/801: unlink d2/d11/d1d/l105 0 2024-03-21T10:09:07.429 INFO:tasks.workunit.client.0.smithi153.stdout:1/772: write d6/d10/d35/da5/d4f/d9d/db6/def/ff3 [555893,91487] 0 2024-03-21T10:09:07.430 INFO:tasks.workunit.client.0.smithi153.stdout:5/917: dread - d4/d8/fbd zero size 2024-03-21T10:09:07.430 INFO:tasks.workunit.client.0.smithi153.stdout:4/942: symlink d11/d2d/d57/ded/l127 0 2024-03-21T10:09:07.430 INFO:tasks.workunit.client.1.smithi174.stdout:8/746: dread d9/d53/f5d [0,4194304] 0 2024-03-21T10:09:07.437 INFO:tasks.workunit.client.1.smithi174.stdout:9/802: chown d2/d11/d1d/dab/dc1/lae 8509 1 2024-03-21T10:09:07.438 INFO:tasks.workunit.client.0.smithi153.stdout:9/985: link d1/d43/d44/d46/d9e/d2e/l69 d1/d43/d44/d46/d9e/l147 0 2024-03-21T10:09:07.439 INFO:tasks.workunit.client.1.smithi174.stdout:2/721: rmdir d5/d25/d27/d2d/d45/dae/dd6/d90 39 2024-03-21T10:09:07.442 INFO:tasks.workunit.client.1.smithi174.stdout:2/722: readlink d5/l5e 0 2024-03-21T10:09:07.443 INFO:tasks.workunit.client.0.smithi153.stdout:1/773: creat d6/d10/d35/da5/d4f/d9d/db6/def/d92/fff x:0 0 0 2024-03-21T10:09:07.443 INFO:tasks.workunit.client.0.smithi153.stdout:5/918: truncate d4/d8/d5a/d5d/dd9/d2f/f12c 194191 0 2024-03-21T10:09:07.444 INFO:tasks.workunit.client.1.smithi174.stdout:4/872: mknod d4/d21/d24/d28/de9/d3a/dd2/d103/c121 0 2024-03-21T10:09:07.446 INFO:tasks.workunit.client.0.smithi153.stdout:0/833: dwrite d4/d4c/d83/d22/d61/d77/df1/f105 [0,4194304] 0 2024-03-21T10:09:07.449 INFO:tasks.workunit.client.1.smithi174.stdout:6/668: dread d0/d3/d2a/d8f/f97 [0,4194304] 0 2024-03-21T10:09:07.453 INFO:tasks.workunit.client.0.smithi153.stdout:4/943: symlink d11/d18/d50/d65/de3/dec/l128 0 2024-03-21T10:09:07.455 INFO:tasks.workunit.client.0.smithi153.stdout:9/986: creat d1/d43/d44/d46/f148 x:0 0 0 2024-03-21T10:09:07.457 INFO:tasks.workunit.client.1.smithi174.stdout:3/757: dwrite d0/d12/d14/f3a [0,4194304] 0 2024-03-21T10:09:07.458 INFO:tasks.workunit.client.0.smithi153.stdout:9/987: chown d1/d43/d44/d46/d13b/d130 0 1 2024-03-21T10:09:07.458 INFO:tasks.workunit.client.0.smithi153.stdout:5/919: chown d4/d8/d5a/d5d/dd9/d44/de1/dfb/d108/fc6 465 1 2024-03-21T10:09:07.458 INFO:tasks.workunit.client.0.smithi153.stdout:1/774: unlink d6/d10/d35/da5/d4f/d9d/db6/def/le2 0 2024-03-21T10:09:07.462 INFO:tasks.workunit.client.0.smithi153.stdout:0/834: symlink d4/d4c/d83/d22/d61/d77/df1/l109 0 2024-03-21T10:09:07.463 INFO:tasks.workunit.client.1.smithi174.stdout:9/803: rename d2/d11/d1d/d4b/ff5 to d2/d11/d60/db2/f119 0 2024-03-21T10:09:07.464 INFO:tasks.workunit.client.1.smithi174.stdout:0/764: dwrite d9/d35/dd2/d104/d72/f85 [0,4194304] 0 2024-03-21T10:09:07.465 INFO:tasks.workunit.client.1.smithi174.stdout:4/873: unlink d4/d21/d24/d28/de9/d3a/d67/d76/d82/fe4 0 2024-03-21T10:09:07.467 INFO:tasks.workunit.client.0.smithi153.stdout:5/920: mkdir d4/d8/d5a/d5d/d12d 0 2024-03-21T10:09:07.467 INFO:tasks.workunit.client.0.smithi153.stdout:0/835: mknod d4/db/c10a 0 2024-03-21T10:09:07.468 INFO:tasks.workunit.client.0.smithi153.stdout:1/775: truncate f3 1783940 0 2024-03-21T10:09:07.468 INFO:tasks.workunit.client.0.smithi153.stdout:4/944: read d11/d9d/dc3/ff7 [1744566,41513] 0 2024-03-21T10:09:07.469 INFO:tasks.workunit.client.0.smithi153.stdout:9/988: getdents d1/d43/d44/d46/d129 0 2024-03-21T10:09:07.472 INFO:tasks.workunit.client.1.smithi174.stdout:2/723: truncate d5/d25/d27/d2d/d45/d5b/d2f/d4b/d76/f97 402385 0 2024-03-21T10:09:07.473 INFO:tasks.workunit.client.1.smithi174.stdout:5/792: dread d1/d16/d2e/d66/d8b/f33 [0,4194304] 0 2024-03-21T10:09:07.474 INFO:tasks.workunit.client.0.smithi153.stdout:4/945: creat d11/d2d/d6b/f129 x:0 0 0 2024-03-21T10:09:07.477 INFO:tasks.workunit.client.1.smithi174.stdout:9/804: creat d2/d24/d2e/de6/f11a x:0 0 0 2024-03-21T10:09:07.480 INFO:tasks.workunit.client.0.smithi153.stdout:1/776: chown d6/d10/d35/da5/d4f/leb 13151744 1 2024-03-21T10:09:07.480 INFO:tasks.workunit.client.0.smithi153.stdout:5/921: truncate d4/d8/d5a/d5d/dd9/f2a 1292285 0 2024-03-21T10:09:07.482 INFO:tasks.workunit.client.1.smithi174.stdout:8/747: dwrite d9/d2a/d39/d76/d19/d27/d60/fa8 [0,4194304] 0 2024-03-21T10:09:07.487 INFO:tasks.workunit.client.1.smithi174.stdout:2/724: fsync d5/d25/d27/d2d/d45/dae/dd6/d7b/f99 0 2024-03-21T10:09:07.488 INFO:tasks.workunit.client.0.smithi153.stdout:4/946: rename d11/d18/f55 to d11/d18/d50/dfa/f12a 0 2024-03-21T10:09:07.491 INFO:tasks.workunit.client.1.smithi174.stdout:6/669: dwrite d0/d3/d2a/d8f/f97 [0,4194304] 0 2024-03-21T10:09:07.492 INFO:tasks.workunit.client.0.smithi153.stdout:5/922: mkdir d4/d8/d5a/d5d/dd9/d44/d12e 0 2024-03-21T10:09:07.492 INFO:tasks.workunit.client.0.smithi153.stdout:0/836: dwrite d4/d16/d19/d30/d6c/d98/d5e/f99 [0,4194304] 0 2024-03-21T10:09:07.494 INFO:tasks.workunit.client.0.smithi153.stdout:0/837: readlink d4/l7 0 2024-03-21T10:09:07.497 INFO:tasks.workunit.client.0.smithi153.stdout:1/777: creat d6/d10/d35/da5/d4f/d9d/db6/def/f100 x:0 0 0 2024-03-21T10:09:07.498 INFO:tasks.workunit.client.0.smithi153.stdout:5/923: fdatasync d4/d5/df/f58 0 2024-03-21T10:09:07.501 INFO:tasks.workunit.client.1.smithi174.stdout:6/670: write d0/d3/d20/f7f [7269142,57999] 0 2024-03-21T10:09:07.503 INFO:tasks.workunit.client.1.smithi174.stdout:4/874: write d4/fa [2562423,3334] 0 2024-03-21T10:09:07.503 INFO:tasks.workunit.client.0.smithi153.stdout:5/924: chown d4/ff8 3764 1 2024-03-21T10:09:07.504 INFO:tasks.workunit.client.1.smithi174.stdout:2/725: symlink d5/d25/d27/d2d/d45/d5b/d2f/d68/de8/lfb 0 2024-03-21T10:09:07.507 INFO:tasks.workunit.client.1.smithi174.stdout:9/805: rmdir d2 39 2024-03-21T10:09:07.509 INFO:tasks.workunit.client.0.smithi153.stdout:0/838: symlink d4/d16/d19/l10b 0 2024-03-21T10:09:07.515 INFO:tasks.workunit.client.1.smithi174.stdout:0/765: dwrite d9/d35/dd2/d104/f5b [0,4194304] 0 2024-03-21T10:09:07.519 INFO:tasks.workunit.client.0.smithi153.stdout:7/973: dwrite d4/d2a/dc6/dfb/f2c [0,4194304] 0 2024-03-21T10:09:07.521 INFO:tasks.workunit.client.1.smithi174.stdout:0/766: chown d9/d21/d31/l3a 684600104 1 2024-03-21T10:09:07.521 INFO:tasks.workunit.client.1.smithi174.stdout:3/758: truncate d0/d12/d14/da9/de7/fbb 1126236 0 2024-03-21T10:09:07.524 INFO:tasks.workunit.client.1.smithi174.stdout:8/748: dread d9/d2a/d39/fa0 [0,4194304] 0 2024-03-21T10:09:07.525 INFO:tasks.workunit.client.1.smithi174.stdout:6/671: link d0/d3/d5/d6/f13 d0/d3/d5/fd4 0 2024-03-21T10:09:07.525 INFO:tasks.workunit.client.0.smithi153.stdout:0/839: symlink d4/d16/d19/d30/d6c/d98/l10c 0 2024-03-21T10:09:07.527 INFO:tasks.workunit.client.0.smithi153.stdout:8/864: dwrite d5/d9/d43/dd/d20/d26/d24/fd6 [0,4194304] 0 2024-03-21T10:09:07.529 INFO:tasks.workunit.client.0.smithi153.stdout:5/925: creat d4/d8/d5a/d5d/dd9/f12f x:0 0 0 2024-03-21T10:09:07.534 INFO:tasks.workunit.client.1.smithi174.stdout:5/793: dwrite d1/d16/d1c/d81/f95 [0,4194304] 0 2024-03-21T10:09:07.534 INFO:tasks.workunit.client.0.smithi153.stdout:7/974: mknod d4/d2a/dc6/c147 0 2024-03-21T10:09:07.537 INFO:tasks.workunit.client.1.smithi174.stdout:3/759: creat d0/d12/d14/da9/de7/db7/d7d/fef x:0 0 0 2024-03-21T10:09:07.539 INFO:tasks.workunit.client.0.smithi153.stdout:8/865: rename d5/d9/d43/dd/d20/d26/d24 to d5/d9/d43/dd/d20/d26/d47/d9a/db9/d112 0 2024-03-21T10:09:07.540 INFO:tasks.workunit.client.0.smithi153.stdout:9/989: truncate d1/d43/d44/d46/d9e/f11e 1207401 0 2024-03-21T10:09:07.546 INFO:tasks.workunit.client.1.smithi174.stdout:3/760: rmdir d0/d12/d14/d1a/d27/d3c 39 2024-03-21T10:09:07.548 INFO:tasks.workunit.client.0.smithi153.stdout:0/840: read d4/d16/d27/f79 [81508,81161] 0 2024-03-21T10:09:07.549 INFO:tasks.workunit.client.0.smithi153.stdout:8/866: unlink d5/d9/fee 0 2024-03-21T10:09:07.553 INFO:tasks.workunit.client.0.smithi153.stdout:4/947: dread d11/d18/f28 [0,4194304] 0 2024-03-21T10:09:07.556 INFO:tasks.workunit.client.1.smithi174.stdout:2/726: write d5/d25/d27/d2d/d45/dae/dd6/d4f/da0/fde [606156,29637] 0 2024-03-21T10:09:07.556 INFO:tasks.workunit.client.1.smithi174.stdout:0/767: write d9/d1c/d76/d87/faa [834400,78497] 0 2024-03-21T10:09:07.558 INFO:tasks.workunit.client.0.smithi153.stdout:7/975: creat d4/d11/d56/d115/f148 x:0 0 0 2024-03-21T10:09:07.559 INFO:tasks.workunit.client.1.smithi174.stdout:0/768: stat d9 0 2024-03-21T10:09:07.559 INFO:tasks.workunit.client.1.smithi174.stdout:3/761: symlink d0/d12/d14/d1a/d5c/lf0 0 2024-03-21T10:09:07.560 INFO:tasks.workunit.client.0.smithi153.stdout:0/841: rename d4/d16/d19/d30/d6c/fcf to d4/d16/d19/d58/f10d 0 2024-03-21T10:09:07.562 INFO:tasks.workunit.client.0.smithi153.stdout:1/778: dwrite d6/fd9 [0,4194304] 0 2024-03-21T10:09:07.563 INFO:tasks.workunit.client.0.smithi153.stdout:8/867: mkdir d5/d9/d43/dd/d2d/d45/d88/de0/d113 0 2024-03-21T10:09:07.566 INFO:tasks.workunit.client.0.smithi153.stdout:2/979: sync 2024-03-21T10:09:07.569 INFO:tasks.workunit.client.1.smithi174.stdout:7/698: sync 2024-03-21T10:09:07.569 INFO:tasks.workunit.client.1.smithi174.stdout:5/794: dwrite d1/d2f/dc1/fe8 [0,4194304] 0 2024-03-21T10:09:07.569 INFO:tasks.workunit.client.1.smithi174.stdout:6/672: dwrite d0/d3/d20/d6d/fb2 [0,4194304] 0 2024-03-21T10:09:07.572 INFO:tasks.workunit.client.1.smithi174.stdout:2/727: stat d5/d25/d27/d2d/d45/dae/dd6/d85/f9a 0 2024-03-21T10:09:07.573 INFO:tasks.workunit.client.0.smithi153.stdout:7/976: creat d4/d2a/d100/f149 x:0 0 0 2024-03-21T10:09:07.573 INFO:tasks.workunit.client.1.smithi174.stdout:0/769: fdatasync d9/d21/f37 0 2024-03-21T10:09:07.574 INFO:tasks.workunit.client.0.smithi153.stdout:5/926: write d4/d5/fd4 [2746752,49600] 0 2024-03-21T10:09:07.575 INFO:tasks.workunit.client.0.smithi153.stdout:9/990: creat d1/d43/d44/f149 x:0 0 0 2024-03-21T10:09:07.576 INFO:tasks.workunit.client.0.smithi153.stdout:2/980: rmdir dd/d15/d18/d2c/d33/d98/dc7 39 2024-03-21T10:09:07.578 INFO:tasks.workunit.client.1.smithi174.stdout:7/699: read - d0/d4/fd6 zero size 2024-03-21T10:09:07.580 INFO:tasks.workunit.client.0.smithi153.stdout:9/991: chown d1/d43/d44/d46/d9e/l136 3513 1 2024-03-21T10:09:07.581 INFO:tasks.workunit.client.0.smithi153.stdout:5/927: stat d4/d8/d5a/d5d/dd9/d44/f127 0 2024-03-21T10:09:07.582 INFO:tasks.workunit.client.1.smithi174.stdout:8/749: dwrite d9/dc6/f3b [8388608,4194304] 0 2024-03-21T10:09:07.585 INFO:tasks.workunit.client.0.smithi153.stdout:0/842: rename d4/d16/d19/d30/d6c/d98/d5e/db2/fc6 to d4/d16/d19/d58/de9/f10e 0 2024-03-21T10:09:07.589 INFO:tasks.workunit.client.0.smithi153.stdout:5/928: fsync d4/d5/df/d67/d6a/f77 0 2024-03-21T10:09:07.590 INFO:tasks.workunit.client.1.smithi174.stdout:5/795: creat d1/d2f/d37/d9b/f11d x:0 0 0 2024-03-21T10:09:07.590 INFO:tasks.workunit.client.1.smithi174.stdout:4/875: dread d4/d19/f2d [0,4194304] 0 2024-03-21T10:09:07.590 INFO:tasks.workunit.client.1.smithi174.stdout:6/673: creat d0/d3/d5/df/fd5 x:0 0 0 2024-03-21T10:09:07.593 INFO:tasks.workunit.client.1.smithi174.stdout:2/728: dread - d5/d25/d27/d2d/d45/dae/dd6/d4f/df1/fe3 zero size 2024-03-21T10:09:07.594 INFO:tasks.workunit.client.1.smithi174.stdout:3/762: mknod d0/d12/d14/d1a/cf1 0 2024-03-21T10:09:07.594 INFO:tasks.workunit.client.0.smithi153.stdout:9/992: stat d1/d4/d5f/df3/d11a/f143 0 2024-03-21T10:09:07.597 INFO:tasks.workunit.client.0.smithi153.stdout:1/779: dwrite d6/d10/d35/da5/d4f/d9d/dc7/dcc/faf [0,4194304] 0 2024-03-21T10:09:07.597 INFO:tasks.workunit.client.0.smithi153.stdout:8/868: dread d5/d9/d43/f16 [0,4194304] 0 2024-03-21T10:09:07.600 INFO:tasks.workunit.client.1.smithi174.stdout:9/806: dread d2/d24/d2e/fc4 [0,4194304] 0 2024-03-21T10:09:07.600 INFO:tasks.workunit.client.1.smithi174.stdout:6/674: unlink d0/d3/d5/df/l14 0 2024-03-21T10:09:07.602 INFO:tasks.workunit.client.1.smithi174.stdout:4/876: creat d4/d21/d24/d28/de9/d3a/d7a/d9d/d11f/f122 x:0 0 0 2024-03-21T10:09:07.607 INFO:tasks.workunit.client.1.smithi174.stdout:5/796: mkdir d1/d2f/d37/df5/d11e 0 2024-03-21T10:09:07.607 INFO:tasks.workunit.client.1.smithi174.stdout:8/750: write d9/d2a/d4d/fc5 [261987,124476] 0 2024-03-21T10:09:07.607 INFO:tasks.workunit.client.1.smithi174.stdout:2/729: chown d5/d25/d27/d2d/d45/dae/dd6/d4f/d65/d77/lf6 0 1 2024-03-21T10:09:07.607 INFO:tasks.workunit.client.1.smithi174.stdout:6/675: dread - d0/d3/d5/df/fd3 zero size 2024-03-21T10:09:07.612 INFO:tasks.workunit.client.1.smithi174.stdout:4/877: mkdir d4/d21/d24/d28/de9/d3a/dd2/d10a/d123 0 2024-03-21T10:09:07.612 INFO:tasks.workunit.client.0.smithi153.stdout:8/869: rmdir d5/d9/d43/d7b/daa/d102 39 2024-03-21T10:09:07.612 INFO:tasks.workunit.client.1.smithi174.stdout:8/751: write d9/f12 [3991636,119899] 0 2024-03-21T10:09:07.616 INFO:tasks.workunit.client.1.smithi174.stdout:8/752: write d9/dc6/fe4 [70715,27711] 0 2024-03-21T10:09:07.618 INFO:tasks.workunit.client.0.smithi153.stdout:4/948: dwrite d11/d2d/d6b/db6/fa9 [4194304,4194304] 0 2024-03-21T10:09:07.619 INFO:tasks.workunit.client.0.smithi153.stdout:2/981: dwrite dd/faf [0,4194304] 0 2024-03-21T10:09:07.620 INFO:tasks.workunit.client.1.smithi174.stdout:8/753: readlink d9/d2a/d39/d76/d19/d27/d60/lcf 0 2024-03-21T10:09:07.625 INFO:tasks.workunit.client.1.smithi174.stdout:4/878: symlink d4/d21/d24/d28/de9/d3a/d7a/d9d/d11f/d120/l124 0 2024-03-21T10:09:07.626 INFO:tasks.workunit.client.0.smithi153.stdout:2/982: read dd/d15/d18/d3a/f133 [813113,36829] 0 2024-03-21T10:09:07.626 INFO:tasks.workunit.client.0.smithi153.stdout:9/993: truncate d1/d43/d44/d46/d9e/d2e/f76 2087199 0 2024-03-21T10:09:07.626 INFO:tasks.workunit.client.0.smithi153.stdout:8/870: symlink d5/d9/d43/d7b/dc4/da7/l114 0 2024-03-21T10:09:07.629 INFO:tasks.workunit.client.1.smithi174.stdout:4/879: fsync d4/d21/d24/d28/de9/d3a/d67/d76/fc0 0 2024-03-21T10:09:07.629 INFO:tasks.workunit.client.0.smithi153.stdout:4/949: creat d11/d2d/d57/f12b x:0 0 0 2024-03-21T10:09:07.632 INFO:tasks.workunit.client.0.smithi153.stdout:9/994: mkdir d1/d4/d14a 0 2024-03-21T10:09:07.633 INFO:tasks.workunit.client.0.smithi153.stdout:8/871: creat d5/d9/d43/dd/d20/df3/f115 x:0 0 0 2024-03-21T10:09:07.639 INFO:tasks.workunit.client.0.smithi153.stdout:9/995: chown d1/d43/d44/d46/d9e/d2e/d8f/la7 205988 1 2024-03-21T10:09:07.643 INFO:tasks.workunit.client.0.smithi153.stdout:8/872: truncate d5/d9/d43/dd/d2d/d45/fba 523456 0 2024-03-21T10:09:07.646 INFO:tasks.workunit.client.1.smithi174.stdout:6/676: dwrite d0/d3/d2a/d56/d5e/f98 [0,4194304] 0 2024-03-21T10:09:07.650 INFO:tasks.workunit.client.0.smithi153.stdout:2/983: read dd/d15/d18/d2c/d33/f43 [3048419,44249] 0 2024-03-21T10:09:07.650 INFO:tasks.workunit.client.0.smithi153.stdout:4/950: read d11/d2d/d3d/dc7/f106 [1242504,92807] 0 2024-03-21T10:09:07.650 INFO:tasks.workunit.client.0.smithi153.stdout:4/951: stat d11/fa6 0 2024-03-21T10:09:07.650 INFO:tasks.workunit.client.0.smithi153.stdout:9/996: rmdir d1/d108/d12b 0 2024-03-21T10:09:07.652 INFO:tasks.workunit.client.0.smithi153.stdout:0/843: write d4/d16/d19/d30/d6c/d98/d5e/db2/fd9 [36772,36937] 0 2024-03-21T10:09:07.654 INFO:tasks.workunit.client.0.smithi153.stdout:1/780: dread d6/d3a/f53 [0,4194304] 0 2024-03-21T10:09:07.654 INFO:tasks.workunit.client.0.smithi153.stdout:7/977: dwrite d4/d14/d73/fb8 [0,4194304] 0 2024-03-21T10:09:07.655 INFO:tasks.workunit.client.1.smithi174.stdout:6/677: truncate d0/d3/d5/d3b/f46 3089898 0 2024-03-21T10:09:07.655 INFO:tasks.workunit.client.1.smithi174.stdout:0/770: dread d9/d35/d4a/f5e [0,4194304] 0 2024-03-21T10:09:07.665 INFO:tasks.workunit.client.1.smithi174.stdout:3/763: truncate d0/d12/d14/f80 1663236 0 2024-03-21T10:09:07.667 INFO:tasks.workunit.client.0.smithi153.stdout:0/844: chown d4/d16/d19/d30/f41 1 1 2024-03-21T10:09:07.668 INFO:tasks.workunit.client.1.smithi174.stdout:0/771: truncate d9/d35/d4a/d67/d89/fa7 675464 0 2024-03-21T10:09:07.671 INFO:tasks.workunit.client.0.smithi153.stdout:7/978: fsync d4/d14/d16/f11a 0 2024-03-21T10:09:07.673 INFO:tasks.workunit.client.1.smithi174.stdout:2/730: write d5/d25/d27/d2d/d45/d5b/d2f/d4b/d76/f97 [62054,38904] 0 2024-03-21T10:09:07.674 INFO:tasks.workunit.client.0.smithi153.stdout:0/845: chown d4/d4c/d83/d22/d61/faa 0 1 2024-03-21T10:09:07.679 INFO:tasks.workunit.client.0.smithi153.stdout:5/929: dread d4/d8/d5a/d5d/dd9/f43 [0,4194304] 0 2024-03-21T10:09:07.679 INFO:tasks.workunit.client.0.smithi153.stdout:4/952: getdents d11/d18/d50/dfa/dff 0 2024-03-21T10:09:07.679 INFO:tasks.workunit.client.0.smithi153.stdout:1/781: read f3 [1071162,103812] 0 2024-03-21T10:09:07.680 INFO:tasks.workunit.client.1.smithi174.stdout:7/700: dwrite d0/d4/d10/d1b/f3b [0,4194304] 0 2024-03-21T10:09:07.680 INFO:tasks.workunit.client.0.smithi153.stdout:2/984: dread dd/d15/d18/d2c/d33/d98/fb5 [0,4194304] 0 2024-03-21T10:09:07.680 INFO:tasks.workunit.client.1.smithi174.stdout:0/772: creat d9/d35/d4a/d67/d89/dfd/f10b x:0 0 0 2024-03-21T10:09:07.681 INFO:tasks.workunit.client.1.smithi174.stdout:5/797: dread d1/d16/d2e/d66/d8b/f48 [0,4194304] 0 2024-03-21T10:09:07.685 INFO:tasks.workunit.client.1.smithi174.stdout:4/880: write d4/d19/f98 [370225,26113] 0 2024-03-21T10:09:07.686 INFO:tasks.workunit.client.0.smithi153.stdout:4/953: creat d11/d2d/d3d/f12c x:0 0 0 2024-03-21T10:09:07.688 INFO:tasks.workunit.client.0.smithi153.stdout:2/985: symlink dd/d15/d3c/l149 0 2024-03-21T10:09:07.690 INFO:tasks.workunit.client.1.smithi174.stdout:2/731: getdents d5/d25/d27/d2d/d45/d5b/d2f/d66 0 2024-03-21T10:09:07.691 INFO:tasks.workunit.client.0.smithi153.stdout:5/930: dread - d4/d8/d5a/d92/ffd zero size 2024-03-21T10:09:07.691 INFO:tasks.workunit.client.0.smithi153.stdout:1/782: rmdir d6/d10/d35/da5/d4f/d9d/dc7 39 2024-03-21T10:09:07.692 INFO:tasks.workunit.client.1.smithi174.stdout:5/798: symlink d1/d16/d1c/d81/l11f 0 2024-03-21T10:09:07.694 INFO:tasks.workunit.client.0.smithi153.stdout:4/954: write d11/d2d/d57/ded/f109 [4039547,121597] 0 2024-03-21T10:09:07.695 INFO:tasks.workunit.client.1.smithi174.stdout:8/754: dwrite d9/d2a/d39/d76/f70 [0,4194304] 0 2024-03-21T10:09:07.695 INFO:tasks.workunit.client.0.smithi153.stdout:2/986: stat dd/d15/d18/d2c/d4b/c54 0 2024-03-21T10:09:07.695 INFO:tasks.workunit.client.0.smithi153.stdout:1/783: fsync d6/d13/f3f 0 2024-03-21T10:09:07.697 INFO:tasks.workunit.client.1.smithi174.stdout:8/755: fdatasync f5 0 2024-03-21T10:09:07.701 INFO:tasks.workunit.client.1.smithi174.stdout:4/881: chown d4/d21/d24/d28/l101 216069 1 2024-03-21T10:09:07.703 INFO:tasks.workunit.client.0.smithi153.stdout:7/979: dwrite d4/d2a/dc6/dfb/f12e [0,4194304] 0 2024-03-21T10:09:07.703 INFO:tasks.workunit.client.0.smithi153.stdout:0/846: dread d4/d16/d19/d30/d6c/f86 [0,4194304] 0 2024-03-21T10:09:07.711 INFO:tasks.workunit.client.1.smithi174.stdout:2/732: creat d5/d25/d27/d2d/d45/dae/dd6/d4f/da0/def/ffc x:0 0 0 2024-03-21T10:09:07.711 INFO:tasks.workunit.client.0.smithi153.stdout:1/784: unlink d6/d10/d35/da5/d4f/d9d/db6/def/lcf 0 2024-03-21T10:09:07.713 INFO:tasks.workunit.client.1.smithi174.stdout:4/882: rename d4/d21/le7 to d4/d21/d24/d28/de9/d3a/d67/d76/l125 0 2024-03-21T10:09:07.714 INFO:tasks.workunit.client.0.smithi153.stdout:2/987: mkdir dd/d15/d18/d2c/d33/d98/dc7/ded/d14a 0 2024-03-21T10:09:07.716 INFO:tasks.workunit.client.1.smithi174.stdout:2/733: readlink d5/d25/d27/d2d/d45/d5b/d2f/d4b/da7/ldb 0 2024-03-21T10:09:07.718 INFO:tasks.workunit.client.0.smithi153.stdout:7/980: truncate d4/d19/d4d/d10e/d121/f9f 1624801 0 2024-03-21T10:09:07.720 INFO:tasks.workunit.client.0.smithi153.stdout:2/988: creat dd/d15/d3c/f14b x:0 0 0 2024-03-21T10:09:07.721 INFO:tasks.workunit.client.0.smithi153.stdout:7/981: write d4/d14/d16/f117 [897314,6973] 0 2024-03-21T10:09:07.723 INFO:tasks.workunit.client.1.smithi174.stdout:4/883: mknod d4/d21/d24/d28/d38/c126 0 2024-03-21T10:09:07.725 INFO:tasks.workunit.client.0.smithi153.stdout:0/847: rename d4/d4c/d83/c2d to d4/d16/d19/d30/d6c/d98/c10f 0 2024-03-21T10:09:07.728 INFO:tasks.workunit.client.0.smithi153.stdout:7/982: mknod d4/d2a/d3b/d81/c14a 0 2024-03-21T10:09:07.729 INFO:tasks.workunit.client.1.smithi174.stdout:8/756: getdents d9/d53/dc9 0 2024-03-21T10:09:07.730 INFO:tasks.workunit.client.0.smithi153.stdout:0/848: chown d4/d4c/lfe 862828 1 2024-03-21T10:09:07.731 INFO:tasks.workunit.client.1.smithi174.stdout:1/885: sync 2024-03-21T10:09:07.737 INFO:tasks.workunit.client.0.smithi153.stdout:7/983: rename d4/d2a/dc6/dfb/d9a/db7/cb4 to d4/d19/d4d/dd5/df7/d108/c14b 0 2024-03-21T10:09:07.739 INFO:tasks.workunit.client.0.smithi153.stdout:7/984: dread - d4/d19/ff5 zero size 2024-03-21T10:09:07.739 INFO:tasks.workunit.client.0.smithi153.stdout:2/989: dread fc [0,4194304] 0 2024-03-21T10:09:07.740 INFO:tasks.workunit.client.0.smithi153.stdout:5/931: dread d4/d5/fd4 [0,4194304] 0 2024-03-21T10:09:07.741 INFO:tasks.workunit.client.1.smithi174.stdout:1/886: chown d4/d7/d68/d7b/de0/l102 16055 1 2024-03-21T10:09:07.741 INFO:tasks.workunit.client.1.smithi174.stdout:3/764: dwrite d0/d12/d14/d1a/fbc [0,4194304] 0 2024-03-21T10:09:07.743 INFO:tasks.workunit.client.1.smithi174.stdout:2/734: creat d5/d25/d27/d2d/d45/ffd x:0 0 0 2024-03-21T10:09:07.744 INFO:tasks.workunit.client.1.smithi174.stdout:7/701: dread d0/f88 [0,4194304] 0 2024-03-21T10:09:07.745 INFO:tasks.workunit.client.0.smithi153.stdout:4/955: truncate d11/d2d/d6b/f8a 133331 0 2024-03-21T10:09:07.746 INFO:tasks.workunit.client.1.smithi174.stdout:5/799: write d1/d16/d2e/d66/f89 [3425851,41618] 0 2024-03-21T10:09:07.747 INFO:tasks.workunit.client.0.smithi153.stdout:5/932: read d4/d5/df/d67/d86/d96/deb/d56/da5/f11e [3001,14355] 0 2024-03-21T10:09:07.750 INFO:tasks.workunit.client.0.smithi153.stdout:4/956: chown c5 4147 1 2024-03-21T10:09:07.750 INFO:tasks.workunit.client.1.smithi174.stdout:1/887: chown d4/d7/d32/d86/fca 106742752 1 2024-03-21T10:09:07.750 INFO:tasks.workunit.client.1.smithi174.stdout:0/773: dwrite d9/f65 [0,4194304] 0 2024-03-21T10:09:07.752 INFO:tasks.workunit.client.0.smithi153.stdout:0/849: stat d4/d4c/d83/d22/d61/le2 0 2024-03-21T10:09:07.754 INFO:tasks.workunit.client.1.smithi174.stdout:0/774: read d9/f6d [591945,62242] 0 2024-03-21T10:09:07.755 INFO:tasks.workunit.client.0.smithi153.stdout:5/933: mknod d4/d8/d5a/d5d/dd9/d44/de1/c130 0 2024-03-21T10:09:07.756 INFO:tasks.workunit.client.1.smithi174.stdout:7/702: symlink d0/d9e/lf3 0 2024-03-21T10:09:07.757 INFO:tasks.workunit.client.0.smithi153.stdout:1/785: dread d6/d10/d35/da5/d4f/f58 [0,4194304] 0 2024-03-21T10:09:07.757 INFO:tasks.workunit.client.1.smithi174.stdout:2/735: stat d5/d25/d27/d2d/d45/dae/dd6/d4f/d65/d77/dcf/fd5 0 2024-03-21T10:09:07.758 INFO:tasks.workunit.client.1.smithi174.stdout:4/884: dwrite d4/d19/fa4 [0,4194304] 0 2024-03-21T10:09:07.760 INFO:tasks.workunit.client.0.smithi153.stdout:5/934: read d4/d8/d5c/d90/d95/f99 [4061207,92845] 0 2024-03-21T10:09:07.764 INFO:tasks.workunit.client.0.smithi153.stdout:4/957: creat d11/f12d x:0 0 0 2024-03-21T10:09:07.770 INFO:tasks.workunit.client.1.smithi174.stdout:5/800: creat d1/d49/d4f/d51/d8f/f120 x:0 0 0 2024-03-21T10:09:07.771 INFO:tasks.workunit.client.0.smithi153.stdout:1/786: mkdir d6/d10/d35/da5/dea/d101 0 2024-03-21T10:09:07.771 INFO:tasks.workunit.client.1.smithi174.stdout:8/757: getdents d9/d2a/d39/d58 0 2024-03-21T10:09:07.771 INFO:tasks.workunit.client.0.smithi153.stdout:5/935: truncate d4/d5/df/d67/d6a/fc3 3468002 0 2024-03-21T10:09:07.774 INFO:tasks.workunit.client.1.smithi174.stdout:7/703: mkdir d0/d4/d10/d1b/d86/dea/df4 0 2024-03-21T10:09:07.774 INFO:tasks.workunit.client.0.smithi153.stdout:2/990: dread dd/d15/d18/fc8 [0,4194304] 0 2024-03-21T10:09:07.778 INFO:tasks.workunit.client.0.smithi153.stdout:4/958: link d11/d18/f61 d11/d18/d50/d65/de3/f12e 0 2024-03-21T10:09:07.778 INFO:tasks.workunit.client.1.smithi174.stdout:7/704: chown d0/d4/d38/f67 102871 1 2024-03-21T10:09:07.778 INFO:tasks.workunit.client.1.smithi174.stdout:3/765: rename d0/d12/d14/d1a/d5c/lf0 to d0/lf2 0 2024-03-21T10:09:07.782 INFO:tasks.workunit.client.0.smithi153.stdout:2/991: write dd/d15/d18/d2c/d33/d98/fb5 [173757,105926] 0 2024-03-21T10:09:07.783 INFO:tasks.workunit.client.0.smithi153.stdout:5/936: creat d4/d5/df/d67/d86/d96/deb/d56/da5/f131 x:0 0 0 2024-03-21T10:09:07.784 INFO:tasks.workunit.client.0.smithi153.stdout:0/850: dread d4/d16/d27/f90 [0,4194304] 0 2024-03-21T10:09:07.784 INFO:tasks.workunit.client.1.smithi174.stdout:8/758: write d9/d53/f95 [744792,113938] 0 2024-03-21T10:09:07.791 INFO:tasks.workunit.client.0.smithi153.stdout:5/937: creat d4/d8/d5a/d5d/d12d/f132 x:0 0 0 2024-03-21T10:09:07.792 INFO:tasks.workunit.client.1.smithi174.stdout:1/888: dwrite d4/d7/d33/d43/d77/d87/f139 [0,4194304] 0 2024-03-21T10:09:07.794 INFO:tasks.workunit.client.1.smithi174.stdout:2/736: dwrite d5/d25/d27/d2d/d45/dae/fac [0,4194304] 0 2024-03-21T10:09:07.796 INFO:tasks.workunit.client.1.smithi174.stdout:8/759: mknod d9/d2a/d39/d76/cf3 0 2024-03-21T10:09:07.796 INFO:tasks.workunit.client.0.smithi153.stdout:0/851: creat d4/d16/d27/dc5/f110 x:0 0 0 2024-03-21T10:09:07.800 INFO:tasks.workunit.client.0.smithi153.stdout:5/938: link d4/d8/d5a/d5d/dd9/f12f d4/d8/d5a/d5d/dd9/d44/f133 0 2024-03-21T10:09:07.801 INFO:tasks.workunit.client.1.smithi174.stdout:5/801: dwrite d1/d49/d93/d9c/feb [0,4194304] 0 2024-03-21T10:09:07.801 INFO:tasks.workunit.client.1.smithi174.stdout:7/705: link d0/f11 d0/d4/d10/d44/d81/d9a/ff5 0 2024-03-21T10:09:07.801 INFO:tasks.workunit.client.1.smithi174.stdout:6/678: sync 2024-03-21T10:09:07.801 INFO:tasks.workunit.client.1.smithi174.stdout:9/807: sync 2024-03-21T10:09:07.802 INFO:tasks.workunit.client.0.smithi153.stdout:4/959: dwrite d11/d9d/fbd [0,4194304] 0 2024-03-21T10:09:07.802 INFO:tasks.workunit.client.0.smithi153.stdout:0/852: symlink d4/l111 0 2024-03-21T10:09:07.802 INFO:tasks.workunit.client.1.smithi174.stdout:0/775: truncate d9/d35/dd2/d104/d72/fb1 3771071 0 2024-03-21T10:09:07.806 INFO:tasks.workunit.client.0.smithi153.stdout:0/853: stat d4/db/c33 0 2024-03-21T10:09:07.807 INFO:tasks.workunit.client.1.smithi174.stdout:2/737: stat d5/d25/d27/d2d/d45/dae 0 2024-03-21T10:09:07.808 INFO:tasks.workunit.client.0.smithi153.stdout:5/939: symlink d4/d5/df/d67/d86/d96/deb/l134 0 2024-03-21T10:09:07.808 INFO:tasks.workunit.client.0.smithi153.stdout:4/960: truncate d11/d18/f95 930737 0 2024-03-21T10:09:07.808 INFO:tasks.workunit.client.0.smithi153.stdout:5/940: stat d4/d8/f1d 0 2024-03-21T10:09:07.811 INFO:tasks.workunit.client.0.smithi153.stdout:0/854: truncate d4/d16/d19/d30/d6c/df4/ffd 264652 0 2024-03-21T10:09:07.812 INFO:tasks.workunit.client.1.smithi174.stdout:0/776: chown d9/d21/d5f/l107 52860093 1 2024-03-21T10:09:07.813 INFO:tasks.workunit.client.1.smithi174.stdout:4/885: truncate d4/f20 704658 0 2024-03-21T10:09:07.813 INFO:tasks.workunit.client.0.smithi153.stdout:5/941: mkdir d4/d5/df/d135 0 2024-03-21T10:09:07.815 INFO:tasks.workunit.client.0.smithi153.stdout:4/961: link d11/d18/d50/d65/de3/f12e d11/d2d/da3/dac/f12f 0 2024-03-21T10:09:07.816 INFO:tasks.workunit.client.1.smithi174.stdout:8/760: symlink d9/d2a/d39/d76/d19/lf4 0 2024-03-21T10:09:07.817 INFO:tasks.workunit.client.1.smithi174.stdout:2/738: read d5/d25/fa4 [350113,24896] 0 2024-03-21T10:09:07.818 INFO:tasks.workunit.client.0.smithi153.stdout:5/942: chown d4/d5/df/d67/da0/db3/c2b 500 1 2024-03-21T10:09:07.819 INFO:tasks.workunit.client.1.smithi174.stdout:7/706: fdatasync d0/d4/d10/d44/f72 0 2024-03-21T10:09:07.820 INFO:tasks.workunit.client.0.smithi153.stdout:0/855: unlink d4/d16/d19/d30/d6c/d98/d5e/db2/cde 0 2024-03-21T10:09:07.822 INFO:tasks.workunit.client.0.smithi153.stdout:4/962: creat d11/d18/d6d/f130 x:0 0 0 2024-03-21T10:09:07.826 INFO:tasks.workunit.client.0.smithi153.stdout:1/787: write d6/d10/d35/da5/f63 [1195520,113163] 0 2024-03-21T10:09:07.826 INFO:tasks.workunit.client.0.smithi153.stdout:5/943: read d4/d8/d5a/f51 [1199603,15047] 0 2024-03-21T10:09:07.827 INFO:tasks.workunit.client.0.smithi153.stdout:4/963: truncate d11/d18/fdb 4206685 0 2024-03-21T10:09:07.827 INFO:tasks.workunit.client.1.smithi174.stdout:0/777: mkdir d9/d35/d4a/d8a/d10c 0 2024-03-21T10:09:07.828 INFO:tasks.workunit.client.0.smithi153.stdout:2/992: write dd/d15/d18/d2c/d33/f9d [3164817,29997] 0 2024-03-21T10:09:07.828 INFO:tasks.workunit.client.1.smithi174.stdout:9/808: mknod d2/d11/d60/db2/c11b 0 2024-03-21T10:09:07.830 INFO:tasks.workunit.client.1.smithi174.stdout:0/778: fsync d9/f6d 0 2024-03-21T10:09:07.830 INFO:tasks.workunit.client.0.smithi153.stdout:1/788: truncate d6/f2f 3943077 0 2024-03-21T10:09:07.830 INFO:tasks.workunit.client.0.smithi153.stdout:5/944: symlink d4/d5/d9e/l136 0 2024-03-21T10:09:07.831 INFO:tasks.workunit.client.0.smithi153.stdout:2/993: dread - dd/d15/d3c/f106 zero size 2024-03-21T10:09:07.832 INFO:tasks.workunit.client.1.smithi174.stdout:3/766: dread d0/d12/d14/da9/de7/fbb [0,4194304] 0 2024-03-21T10:09:07.833 INFO:tasks.workunit.client.1.smithi174.stdout:9/809: write d2/d11/d1d/d4b/d72/fe0 [897194,49428] 0 2024-03-21T10:09:07.834 INFO:tasks.workunit.client.0.smithi153.stdout:5/945: creat d4/d8/d5a/d92/d9f/f137 x:0 0 0 2024-03-21T10:09:07.835 INFO:tasks.workunit.client.0.smithi153.stdout:1/789: fdatasync d6/d13/d8e/dbb/de1/de7/de9/df5/d50/da8/fed 0 2024-03-21T10:09:07.839 INFO:tasks.workunit.client.0.smithi153.stdout:1/790: write d6/d3a/fb0 [3400629,35705] 0 2024-03-21T10:09:07.840 INFO:tasks.workunit.client.1.smithi174.stdout:6/679: dwrite d0/d3/d5/d6/f44 [0,4194304] 0 2024-03-21T10:09:07.840 INFO:tasks.workunit.client.0.smithi153.stdout:5/946: truncate d4/d8/d5a/d92/d9f/f118 1021546 0 2024-03-21T10:09:07.843 INFO:tasks.workunit.client.0.smithi153.stdout:2/994: link dd/d15/d18/d2c/d4b/d56/d63/d8d/d9b/dc3/df5/c96 dd/d15/d18/d2c/d33/d98/dc7/de1/dfe/c14c 0 2024-03-21T10:09:07.846 INFO:tasks.workunit.client.1.smithi174.stdout:3/767: symlink d0/d12/lf3 0 2024-03-21T10:09:07.847 INFO:tasks.workunit.client.0.smithi153.stdout:2/995: write dd/d15/d18/d2c/d33/f9d [560641,114662] 0 2024-03-21T10:09:07.849 INFO:tasks.workunit.client.1.smithi174.stdout:5/802: dwrite d1/d49/d93/d9c/fb2 [0,4194304] 0 2024-03-21T10:09:07.849 INFO:tasks.workunit.client.1.smithi174.stdout:6/680: creat d0/d3/d20/d3d/fd6 x:0 0 0 2024-03-21T10:09:07.850 INFO:tasks.workunit.client.1.smithi174.stdout:3/768: chown d0/d12/d14/d67/da1/lad 95 1 2024-03-21T10:09:07.855 INFO:tasks.workunit.client.1.smithi174.stdout:5/803: creat d1/d2f/dc1/f121 x:0 0 0 2024-03-21T10:09:07.857 INFO:tasks.workunit.client.0.smithi153.stdout:2/996: chown dd/d15/d18/d2c/d4b/d56/d63/d8d/d9b/dc3/df5/dae/d50/d58/c12a 39 1 2024-03-21T10:09:07.857 INFO:tasks.workunit.client.1.smithi174.stdout:5/804: fsync d1/d5c/d96/f105 0 2024-03-21T10:09:07.861 INFO:tasks.workunit.client.1.smithi174.stdout:8/761: rmdir d9/d2a/d39 39 2024-03-21T10:09:07.861 INFO:tasks.workunit.client.1.smithi174.stdout:5/805: write d1/d16/d2e/d66/f7d [1025724,37952] 0 2024-03-21T10:09:07.862 INFO:tasks.workunit.client.1.smithi174.stdout:6/681: rename d0/d3/d5/df/lbc to d0/d3/d5/df/d16/da9/ld7 0 2024-03-21T10:09:07.865 INFO:tasks.workunit.client.1.smithi174.stdout:3/769: mkdir d0/d12/df4 0 2024-03-21T10:09:07.870 INFO:tasks.workunit.client.1.smithi174.stdout:8/762: chown d9/d2a/d39/d76/d19/d27/d60/daa 191220 1 2024-03-21T10:09:07.871 INFO:tasks.workunit.client.1.smithi174.stdout:8/763: chown d9/d2a/d39/de1 319 1 2024-03-21T10:09:07.872 INFO:tasks.workunit.client.0.smithi153.stdout:2/997: getdents dd/d15/d18/d2c/d4b/d56/d63 0 2024-03-21T10:09:07.872 INFO:tasks.workunit.client.1.smithi174.stdout:6/682: creat d0/d3/d5/df/d66/d87/fd8 x:0 0 0 2024-03-21T10:09:07.874 INFO:tasks.workunit.client.1.smithi174.stdout:3/770: creat d0/d12/d14/d1a/d44/d65/ff5 x:0 0 0 2024-03-21T10:09:07.874 INFO:tasks.workunit.client.0.smithi153.stdout:2/998: chown dd/d15/d18/d2c/d33/f88 17641140 1 2024-03-21T10:09:07.877 INFO:tasks.workunit.client.1.smithi174.stdout:5/806: rename d1/f71 to d1/d5c/d96/f122 0 2024-03-21T10:09:07.878 INFO:tasks.workunit.client.1.smithi174.stdout:4/886: write d4/d21/d24/d28/de9/d6a/f72 [3999176,40986] 0 2024-03-21T10:09:07.880 INFO:tasks.workunit.client.1.smithi174.stdout:3/771: write d0/d12/d14/d1a/f1e [360753,52210] 0 2024-03-21T10:09:07.884 INFO:tasks.workunit.client.1.smithi174.stdout:0/779: write d9/d35/d4a/d67/fdd [754890,58164] 0 2024-03-21T10:09:07.884 INFO:tasks.workunit.client.1.smithi174.stdout:6/683: truncate d0/d3/d20/d3d/d81/f9f 803722 0 2024-03-21T10:09:07.886 INFO:tasks.workunit.client.1.smithi174.stdout:1/889: dwrite d4/d7/d28/f30 [4194304,4194304] 0 2024-03-21T10:09:07.888 INFO:tasks.workunit.client.0.smithi153.stdout:5/947: dwrite d4/d8/d5a/d5d/dd9/d44/de1/dfb/d108/f87 [0,4194304] 0 2024-03-21T10:09:07.892 INFO:tasks.workunit.client.1.smithi174.stdout:2/739: dwrite d5/d25/d27/d2d/d45/d5b/d2f/d66/f93 [0,4194304] 0 2024-03-21T10:09:07.893 INFO:tasks.workunit.client.1.smithi174.stdout:1/890: write d4/d7/d15/f39 [1220986,66903] 0 2024-03-21T10:09:07.896 INFO:tasks.workunit.client.0.smithi153.stdout:5/948: mknod d4/d8/d5a/d5d/dd9/d2f/d128/d69/c138 0 2024-03-21T10:09:07.898 INFO:tasks.workunit.client.0.smithi153.stdout:5/949: chown d4/d8/d5a/f51 497 1 2024-03-21T10:09:07.899 INFO:tasks.workunit.client.1.smithi174.stdout:3/772: fdatasync d0/d12/d14/da9/de7/fbb 0 2024-03-21T10:09:07.902 INFO:tasks.workunit.client.1.smithi174.stdout:6/684: creat d0/d3/d5/df/fd9 x:0 0 0 2024-03-21T10:09:07.903 INFO:tasks.workunit.client.0.smithi153.stdout:2/999: dwrite dd/d15/d18/d2c/d4b/fce [0,4194304] 0 2024-03-21T10:09:07.911 INFO:tasks.workunit.client.1.smithi174.stdout:8/764: write d9/dc6/f4f [2039242,56392] 0 2024-03-21T10:09:07.919 INFO:tasks.workunit.client.0.smithi153.stdout:5/950: dread d4/d8/d5a/d5d/dd9/d44/f127 [0,4194304] 0 2024-03-21T10:09:07.919 INFO:tasks.workunit.client.1.smithi174.stdout:3/773: rmdir d0/d12/d14/d67/dc8 39 2024-03-21T10:09:07.924 INFO:tasks.workunit.client.1.smithi174.stdout:1/891: mkdir d4/d7/d33/d43/d51/da4/d134/d13a 0 2024-03-21T10:09:07.924 INFO:tasks.workunit.client.1.smithi174.stdout:4/887: creat d4/d21/f127 x:0 0 0 2024-03-21T10:09:07.925 INFO:tasks.workunit.client.1.smithi174.stdout:8/765: unlink d9/d2a/d39/d76/d19/d27/d60/l86 0 2024-03-21T10:09:07.925 INFO:tasks.workunit.client.0.smithi153.stdout:1/791: dread d6/d13/d8e/dbb/de1/de7/de9/df5/d50/f7f [0,4194304] 0 2024-03-21T10:09:07.929 INFO:tasks.workunit.client.0.smithi153.stdout:1/792: creat d6/d10/d35/da5/d4f/f102 x:0 0 0 2024-03-21T10:09:07.930 INFO:tasks.workunit.client.1.smithi174.stdout:3/774: unlink d0/d12/d14/da9/de7/db7/fed 0 2024-03-21T10:09:07.931 INFO:tasks.workunit.client.1.smithi174.stdout:4/888: dread - d4/d21/d24/d28/de9/d3a/d7a/d9d/d11f/f122 zero size 2024-03-21T10:09:07.931 INFO:tasks.workunit.client.0.smithi153.stdout:1/793: mkdir d6/d10/d5c/d103 0 2024-03-21T10:09:07.932 INFO:tasks.workunit.client.1.smithi174.stdout:9/810: dread d2/d24/fd5 [0,4194304] 0 2024-03-21T10:09:07.932 INFO:tasks.workunit.client.1.smithi174.stdout:1/892: mkdir d4/d7/d33/d43/d77/d87/d13b 0 2024-03-21T10:09:07.934 INFO:tasks.workunit.client.1.smithi174.stdout:6/685: dread d0/d3/d5/d6/f13 [8388608,4194304] 0 2024-03-21T10:09:07.934 INFO:tasks.workunit.client.0.smithi153.stdout:1/794: mknod d6/d10/da0/c104 0 2024-03-21T10:09:07.935 INFO:tasks.workunit.client.0.smithi153.stdout:5/951: dwrite d4/d5/df/d67/d86/d96/deb/d56/f62 [0,4194304] 0 2024-03-21T10:09:07.937 INFO:tasks.workunit.client.1.smithi174.stdout:8/766: symlink d9/d53/dc9/de6/lf5 0 2024-03-21T10:09:07.937 INFO:tasks.workunit.client.1.smithi174.stdout:7/707: dread d0/d4/fd [0,4194304] 0 2024-03-21T10:09:07.939 INFO:tasks.workunit.client.0.smithi153.stdout:1/795: fsync d6/f16 0 2024-03-21T10:09:07.939 INFO:tasks.workunit.client.1.smithi174.stdout:9/811: rmdir d2/d24 39 2024-03-21T10:09:07.942 INFO:tasks.workunit.client.1.smithi174.stdout:1/893: creat d4/d7/d33/d43/d77/d87/d92/f13c x:0 0 0 2024-03-21T10:09:07.942 INFO:tasks.workunit.client.0.smithi153.stdout:5/952: rename d4/d8/d5a/d5d/dd9/ff7 to d4/d8/d5a/d5d/d12d/f139 0 2024-03-21T10:09:07.944 INFO:tasks.workunit.client.0.smithi153.stdout:1/796: write d6/d10/d35/da5/d4f/d9d/dc7/dcc/f6a [140008,65593] 0 2024-03-21T10:09:07.947 INFO:tasks.workunit.client.0.smithi153.stdout:5/953: creat d4/d8/d5a/d5d/dd9/d2f/d128/d69/f13a x:0 0 0 2024-03-21T10:09:07.948 INFO:tasks.workunit.client.1.smithi174.stdout:3/775: write d0/d12/d14/d67/dc8/fd1 [2862694,42484] 0 2024-03-21T10:09:07.949 INFO:tasks.workunit.client.1.smithi174.stdout:7/708: chown d0/l96 0 1 2024-03-21T10:09:07.949 INFO:tasks.workunit.client.0.smithi153.stdout:1/797: unlink d6/d3a/ce5 0 2024-03-21T10:09:07.949 INFO:tasks.workunit.client.1.smithi174.stdout:6/686: mkdir d0/d3/d5/df/d16/da9/db7/dc2/dda 0 2024-03-21T10:09:07.950 INFO:tasks.workunit.client.1.smithi174.stdout:1/894: mknod d4/d5/d6c/d10e/c13d 0 2024-03-21T10:09:07.952 INFO:tasks.workunit.client.0.smithi153.stdout:5/954: creat d4/d8/d5a/d5d/dd9/d38/dfe/f13b x:0 0 0 2024-03-21T10:09:07.952 INFO:tasks.workunit.client.0.smithi153.stdout:1/798: rmdir d6/d13/d8e/dbb 39 2024-03-21T10:09:07.953 INFO:tasks.workunit.client.1.smithi174.stdout:0/780: dread d9/d21/d75/da1/fa8 [0,4194304] 0 2024-03-21T10:09:07.955 INFO:tasks.workunit.client.0.smithi153.stdout:5/955: write d4/d5/df/d67/d86/d96/deb/d56/d105/f10e [295660,109153] 0 2024-03-21T10:09:07.957 INFO:tasks.workunit.client.0.smithi153.stdout:5/956: stat d4/d8/d5a/c121 0 2024-03-21T10:09:07.959 INFO:tasks.workunit.client.1.smithi174.stdout:4/889: getdents d4/d21 0 2024-03-21T10:09:07.959 INFO:tasks.workunit.client.0.smithi153.stdout:5/957: fdatasync d4/d5/df/d67/d6a/f77 0 2024-03-21T10:09:07.961 INFO:tasks.workunit.client.0.smithi153.stdout:1/799: write d6/d13/f72 [381687,82793] 0 2024-03-21T10:09:07.963 INFO:tasks.workunit.client.0.smithi153.stdout:5/958: creat d4/d5/df/d67/d86/d96/f13c x:0 0 0 2024-03-21T10:09:07.964 INFO:tasks.workunit.client.0.smithi153.stdout:5/959: chown d4/d5/d9e 2986786 1 2024-03-21T10:09:07.964 INFO:tasks.workunit.client.0.smithi153.stdout:1/800: mknod d6/d10/d35/da5/dea/c105 0 2024-03-21T10:09:07.968 INFO:tasks.workunit.client.0.smithi153.stdout:1/801: mkdir d6/d10/da0/d106 0 2024-03-21T10:09:07.974 INFO:tasks.workunit.client.0.smithi153.stdout:1/802: creat d6/d10/d35/da5/d4f/d9d/dc7/dcc/d80/f107 x:0 0 0 2024-03-21T10:09:07.974 INFO:tasks.workunit.client.1.smithi174.stdout:8/767: link d9/d2a/c72 d9/d2a/d39/d76/d19/d27/d60/daa/cf6 0 2024-03-21T10:09:07.974 INFO:tasks.workunit.client.1.smithi174.stdout:0/781: creat d9/d21/d31/d9f/d79/d82/f10d x:0 0 0 2024-03-21T10:09:07.974 INFO:tasks.workunit.client.1.smithi174.stdout:1/895: truncate d4/d5/d6c/fc0 2457058 0 2024-03-21T10:09:07.975 INFO:tasks.workunit.client.1.smithi174.stdout:8/768: readlink d9/d53/d7e/l99 0 2024-03-21T10:09:07.976 INFO:tasks.workunit.client.0.smithi153.stdout:5/960: dread d4/d5/df/d67/d86/d96/deb/df0/d10a/f126 [0,4194304] 0 2024-03-21T10:09:07.977 INFO:tasks.workunit.client.1.smithi174.stdout:0/782: write d9/d21/d23/da5/f6f [1609122,6996] 0 2024-03-21T10:09:07.978 INFO:tasks.workunit.client.1.smithi174.stdout:9/812: truncate d2/d11/d1d/d2a/da7/ff1 243755 0 2024-03-21T10:09:07.978 INFO:tasks.workunit.client.1.smithi174.stdout:1/896: write d4/d2d/fce [5114613,65440] 0 2024-03-21T10:09:07.986 INFO:tasks.workunit.client.1.smithi174.stdout:7/709: dwrite d0/f88 [4194304,4194304] 0 2024-03-21T10:09:07.988 INFO:tasks.workunit.client.0.smithi153.stdout:5/961: dread d4/d8/d5a/d5d/dd9/d44/de1/dfb/d108/f87 [0,4194304] 0 2024-03-21T10:09:07.989 INFO:tasks.workunit.client.0.smithi153.stdout:5/962: read - d4/d8/d5a/d5d/fef zero size 2024-03-21T10:09:07.990 INFO:tasks.workunit.client.1.smithi174.stdout:6/687: dwrite d0/d3/d5/d21/f9c [0,4194304] 0 2024-03-21T10:09:07.990 INFO:tasks.workunit.client.0.smithi153.stdout:1/803: dread d6/d10/f97 [0,4194304] 0 2024-03-21T10:09:07.993 INFO:tasks.workunit.client.0.smithi153.stdout:1/804: readlink d6/d10/d35/da5/d4f/lbf 0 2024-03-21T10:09:07.994 INFO:tasks.workunit.client.0.smithi153.stdout:5/963: creat d4/d5/df/d135/f13d x:0 0 0 2024-03-21T10:09:07.997 INFO:tasks.workunit.client.0.smithi153.stdout:1/805: dread - d6/d10/d35/da5/d4f/d9d/dc7/dcc/d80/ff2 zero size 2024-03-21T10:09:07.997 INFO:tasks.workunit.client.0.smithi153.stdout:5/964: stat d4/d5/df/d67/d86/d96/d104 0 2024-03-21T10:09:07.998 INFO:tasks.workunit.client.1.smithi174.stdout:3/776: dread d0/d12/d14/d1a/d27/d3c/f72 [0,4194304] 0 2024-03-21T10:09:08.004 INFO:tasks.workunit.client.0.smithi153.stdout:5/965: getdents d4/d5/df/d67/da0 0 2024-03-21T10:09:08.008 INFO:tasks.workunit.client.0.smithi153.stdout:5/966: symlink d4/d8/d5a/d5d/dd9/d38/l13e 0 2024-03-21T10:09:08.010 INFO:tasks.workunit.client.1.smithi174.stdout:9/813: chown d2/d24/d2e/fb1 220 1 2024-03-21T10:09:08.011 INFO:tasks.workunit.client.0.smithi153.stdout:5/967: mkdir d4/d8/d5a/d92/d13f 0 2024-03-21T10:09:08.011 INFO:tasks.workunit.client.1.smithi174.stdout:7/710: unlink d0/d4/f8 0 2024-03-21T10:09:08.012 INFO:tasks.workunit.client.1.smithi174.stdout:6/688: mkdir d0/d3/d5/df/d16/da9/ddb 0 2024-03-21T10:09:08.017 INFO:tasks.workunit.client.1.smithi174.stdout:8/769: mkdir d9/d2a/d39/de1/df7 0 2024-03-21T10:09:08.021 INFO:tasks.workunit.client.1.smithi174.stdout:1/897: link d4/d7/d32/d7d/f12e d4/d7/d33/d43/d77/d87/f13e 0 2024-03-21T10:09:08.024 INFO:tasks.workunit.client.1.smithi174.stdout:8/770: write d9/d2a/d4d/f50 [3136613,47449] 0 2024-03-21T10:09:08.025 INFO:tasks.workunit.client.0.smithi153.stdout:5/968: dwrite d4/d8/d5a/d92/d9f/f12b [0,4194304] 0 2024-03-21T10:09:08.029 INFO:tasks.workunit.client.0.smithi153.stdout:5/969: stat d4/d8/d5a/d5d/dd9/d38/l13e 0 2024-03-21T10:09:08.032 INFO:tasks.workunit.client.0.smithi153.stdout:5/970: read d4/d5/df/d67/d86/d96/deb/d56/d105/f10b [604110,116679] 0 2024-03-21T10:09:08.033 INFO:tasks.workunit.client.1.smithi174.stdout:9/814: unlink d2/d24/lb7 0 2024-03-21T10:09:08.035 INFO:tasks.workunit.client.0.smithi153.stdout:1/806: dread d6/d10/d2c/f7e [0,4194304] 0 2024-03-21T10:09:08.035 INFO:tasks.workunit.client.1.smithi174.stdout:0/783: getdents d9/d1c/d76/d87/dcd 0 2024-03-21T10:09:08.036 INFO:tasks.workunit.client.0.smithi153.stdout:5/971: creat d4/d5/df/d67/d86/d96/deb/d56/d105/f140 x:0 0 0 2024-03-21T10:09:08.037 INFO:tasks.workunit.client.1.smithi174.stdout:6/689: dwrite d0/d3/d5/d6/f2d [0,4194304] 0 2024-03-21T10:09:08.040 INFO:tasks.workunit.client.0.smithi153.stdout:1/807: rmdir d6/d13/d8e/dbb/de1/de7/de9/df5 39 2024-03-21T10:09:08.042 INFO:tasks.workunit.client.0.smithi153.stdout:5/972: creat d4/d5/df/d67/d86/d96/deb/df0/f141 x:0 0 0 2024-03-21T10:09:08.043 INFO:tasks.workunit.client.1.smithi174.stdout:2/740: sync 2024-03-21T10:09:08.043 INFO:tasks.workunit.client.1.smithi174.stdout:5/807: sync 2024-03-21T10:09:08.044 INFO:tasks.workunit.client.0.smithi153.stdout:1/808: symlink d6/d10/l108 0 2024-03-21T10:09:08.044 INFO:tasks.workunit.client.0.smithi153.stdout:5/973: chown d4/d8/d5a/d5d/dd9/d44/ff9 18660263 1 2024-03-21T10:09:08.046 INFO:tasks.workunit.client.1.smithi174.stdout:7/711: rename d0/l8b to d0/d4/d10/d44/d81/d9a/dc0/lf6 0 2024-03-21T10:09:08.049 INFO:tasks.workunit.client.1.smithi174.stdout:0/784: mkdir d9/d35/d4a/d67/d89/d99/d10e 0 2024-03-21T10:09:08.052 INFO:tasks.workunit.client.1.smithi174.stdout:8/771: symlink d9/d2a/d39/d76/d19/de9/lf8 0 2024-03-21T10:09:08.053 INFO:tasks.workunit.client.0.smithi153.stdout:1/809: rename d6/d13/d8e/dbb/de1/de7/de9/df5/d50/l5d to d6/d10/l109 0 2024-03-21T10:09:08.055 INFO:tasks.workunit.client.1.smithi174.stdout:4/890: dread d4/d21/d24/d28/de9/d3a/f3d [0,4194304] 0 2024-03-21T10:09:08.055 INFO:tasks.workunit.client.0.smithi153.stdout:1/810: write d6/d13/d75/f9e [765059,37773] 0 2024-03-21T10:09:08.057 INFO:tasks.workunit.client.1.smithi174.stdout:4/891: dread - d4/d21/d24/d28/de9/d3a/f100 zero size 2024-03-21T10:09:08.058 INFO:tasks.workunit.client.0.smithi153.stdout:1/811: write d6/d10/d2c/fe3 [701416,129793] 0 2024-03-21T10:09:08.059 INFO:tasks.workunit.client.0.smithi153.stdout:5/974: write d4/d8/f11 [53136,100898] 0 2024-03-21T10:09:08.060 INFO:tasks.workunit.client.1.smithi174.stdout:2/741: mknod d5/d25/d27/d2d/d45/dae/dd6/d4f/cfe 0 2024-03-21T10:09:08.064 INFO:tasks.workunit.client.1.smithi174.stdout:0/785: creat d9/d35/d4a/d67/d89/f10f x:0 0 0 2024-03-21T10:09:08.064 INFO:tasks.workunit.client.0.smithi153.stdout:5/975: creat d4/d8/d5a/d5d/dd9/f142 x:0 0 0 2024-03-21T10:09:08.065 INFO:tasks.workunit.client.1.smithi174.stdout:5/808: read d1/f86 [1195029,64277] 0 2024-03-21T10:09:08.065 INFO:tasks.workunit.client.1.smithi174.stdout:4/892: rmdir d4/d21/d24/d28/de9/d3a/dd2/d103 39 2024-03-21T10:09:08.069 INFO:tasks.workunit.client.1.smithi174.stdout:8/772: mknod d9/d2a/d39/d76/d19/d27/d60/daa/df2/cf9 0 2024-03-21T10:09:08.073 INFO:tasks.workunit.client.1.smithi174.stdout:1/898: write d4/d7/d28/d59/ffa [662409,16195] 0 2024-03-21T10:09:08.073 INFO:tasks.workunit.client.0.smithi153.stdout:1/812: dwrite d6/d10/d35/da5/d4f/d9d/fd2 [0,4194304] 0 2024-03-21T10:09:08.077 INFO:tasks.workunit.client.0.smithi153.stdout:1/813: dread - d6/d10/d35/da5/d4f/d9d/db6/def/d92/fc0 zero size 2024-03-21T10:09:08.080 INFO:tasks.workunit.client.1.smithi174.stdout:3/777: dwrite d0/fb [0,4194304] 0 2024-03-21T10:09:08.081 INFO:tasks.workunit.client.0.smithi153.stdout:1/814: creat d6/d10/d35/da5/d4f/d9d/db6/f10a x:0 0 0 2024-03-21T10:09:08.082 INFO:tasks.workunit.client.0.smithi153.stdout:5/976: dwrite d4/d5/f1a [0,4194304] 0 2024-03-21T10:09:08.083 INFO:tasks.workunit.client.1.smithi174.stdout:9/815: write d2/d24/f88 [3843739,69607] 0 2024-03-21T10:09:08.084 INFO:tasks.workunit.client.0.smithi153.stdout:5/977: chown d4/d8/d5c/lf6 0 1 2024-03-21T10:09:08.084 INFO:tasks.workunit.client.1.smithi174.stdout:5/809: truncate d1/d49/f102 103238 0 2024-03-21T10:09:08.090 INFO:tasks.workunit.client.1.smithi174.stdout:1/899: creat d4/d7/d70/f13f x:0 0 0 2024-03-21T10:09:08.091 INFO:tasks.workunit.client.1.smithi174.stdout:4/893: mknod d4/d21/d24/d28/de9/d3a/d7a/c128 0 2024-03-21T10:09:08.092 INFO:tasks.workunit.client.0.smithi153.stdout:1/815: dread d6/f52 [0,4194304] 0 2024-03-21T10:09:08.096 INFO:tasks.workunit.client.1.smithi174.stdout:3/778: symlink d0/d12/d14/d1a/d44/d65/d7e/dac/lf6 0 2024-03-21T10:09:08.097 INFO:tasks.workunit.client.1.smithi174.stdout:6/690: dwrite d0/d3/d5/d6/f2d [4194304,4194304] 0 2024-03-21T10:09:08.099 INFO:tasks.workunit.client.1.smithi174.stdout:7/712: dwrite d0/f11 [0,4194304] 0 2024-03-21T10:09:08.107 INFO:tasks.workunit.client.0.smithi153.stdout:9/997: sync 2024-03-21T10:09:08.107 INFO:tasks.workunit.client.0.smithi153.stdout:0/856: sync 2024-03-21T10:09:08.108 INFO:tasks.workunit.client.0.smithi153.stdout:8/873: sync 2024-03-21T10:09:08.108 INFO:tasks.workunit.client.0.smithi153.stdout:4/964: sync 2024-03-21T10:09:08.108 INFO:tasks.workunit.client.0.smithi153.stdout:7/985: sync 2024-03-21T10:09:08.108 INFO:tasks.workunit.client.1.smithi174.stdout:6/691: fsync d0/d3/d5/d3b/fb1 0 2024-03-21T10:09:08.108 INFO:tasks.workunit.client.1.smithi174.stdout:3/779: mknod d0/d12/d14/d1a/d5c/cf7 0 2024-03-21T10:09:08.111 INFO:tasks.workunit.client.1.smithi174.stdout:2/742: dread d5/d25/d27/d2d/d45/dae/dd6/d4f/d65/f92 [0,4194304] 0 2024-03-21T10:09:08.114 INFO:tasks.workunit.client.1.smithi174.stdout:0/786: dwrite d9/f13 [0,4194304] 0 2024-03-21T10:09:08.115 INFO:tasks.workunit.client.0.smithi153.stdout:7/986: unlink d4/d11/d56/d10c/f111 0 2024-03-21T10:09:08.115 INFO:tasks.workunit.client.1.smithi174.stdout:6/692: fsync d0/d3/d5/df/fae 0 2024-03-21T10:09:08.118 INFO:tasks.workunit.client.1.smithi174.stdout:1/900: symlink d4/d7/d68/d7b/l140 0 2024-03-21T10:09:08.119 INFO:tasks.workunit.client.1.smithi174.stdout:5/810: link d1/d2f/d37/ff4 d1/f123 0 2024-03-21T10:09:08.120 INFO:tasks.workunit.client.1.smithi174.stdout:2/743: fdatasync d5/d25/d27/d2d/d45/dae/dd6/d4f/df1/fe3 0 2024-03-21T10:09:08.120 INFO:tasks.workunit.client.0.smithi153.stdout:0/857: rename d4/d16/d19/d58/f10d to d4/d16/d19/d30/d6c/d98/d5e/f112 0 2024-03-21T10:09:08.121 INFO:tasks.workunit.client.1.smithi174.stdout:1/901: read - d4/d5/f11c zero size 2024-03-21T10:09:08.122 INFO:tasks.workunit.client.0.smithi153.stdout:7/987: mknod d4/d14/d2f/d78/c14c 0 2024-03-21T10:09:08.123 INFO:tasks.workunit.client.1.smithi174.stdout:2/744: chown d5/d25/d27/d2d/d45/d5b/d2f/d68/f96 7 1 2024-03-21T10:09:08.123 INFO:tasks.workunit.client.1.smithi174.stdout:6/693: creat d0/d3/d20/d3d/d81/da5/fdc x:0 0 0 2024-03-21T10:09:08.124 INFO:tasks.workunit.client.0.smithi153.stdout:5/978: dread d4/d8/d5a/d5d/dd9/d2f/d128/f6e [0,4194304] 0 2024-03-21T10:09:08.124 INFO:tasks.workunit.client.1.smithi174.stdout:1/902: creat d4/d7/d15/d4f/f141 x:0 0 0 2024-03-21T10:09:08.128 INFO:tasks.workunit.client.0.smithi153.stdout:5/979: chown d4/d8/d5a/d5d/dd9/d44/dc7 0 1 2024-03-21T10:09:08.130 INFO:tasks.workunit.client.1.smithi174.stdout:7/713: dread d0/d4/d10/d44/fe1 [0,4194304] 0 2024-03-21T10:09:08.131 INFO:tasks.workunit.client.1.smithi174.stdout:2/745: dread - d5/d25/d27/d2d/d45/dae/dd6/d85/f9a zero size 2024-03-21T10:09:08.136 INFO:tasks.workunit.client.1.smithi174.stdout:6/694: creat d0/d3/d5/df/d16/da9/db7/dc2/fdd x:0 0 0 2024-03-21T10:09:08.142 INFO:tasks.workunit.client.0.smithi153.stdout:4/965: dwrite d11/f78 [0,4194304] 0 2024-03-21T10:09:08.146 INFO:tasks.workunit.client.1.smithi174.stdout:2/746: mkdir d5/d25/d27/d2d/d45/d5b/d2f/d4b/dd8/dff 0 2024-03-21T10:09:08.146 INFO:tasks.workunit.client.1.smithi174.stdout:7/714: symlink d0/lf7 0 2024-03-21T10:09:08.152 INFO:tasks.workunit.client.0.smithi153.stdout:1/816: fsync d6/d10/d35/da5/d4f/d9d/db6/f10a 0 2024-03-21T10:09:08.153 INFO:tasks.workunit.client.1.smithi174.stdout:4/894: dwrite d4/d19/f41 [0,4194304] 0 2024-03-21T10:09:08.156 INFO:tasks.workunit.client.1.smithi174.stdout:6/695: creat d0/d3/d20/d3d/fde x:0 0 0 2024-03-21T10:09:08.157 INFO:tasks.workunit.client.0.smithi153.stdout:1/817: creat d6/d10/d35/da5/d4f/d9d/dc7/dcc/d80/f10b x:0 0 0 2024-03-21T10:09:08.158 INFO:tasks.workunit.client.0.smithi153.stdout:9/998: write d1/d43/d44/d46/d4d/d5c/fa9 [777696,80544] 0 2024-03-21T10:09:08.161 INFO:tasks.workunit.client.1.smithi174.stdout:3/780: write d0/d12/d46/f4b [229982,63589] 0 2024-03-21T10:09:08.166 INFO:tasks.workunit.client.0.smithi153.stdout:8/874: dwrite d5/d9/f8d [0,4194304] 0 2024-03-21T10:09:08.169 INFO:tasks.workunit.client.0.smithi153.stdout:8/875: readlink d5/d9/d43/dd/l108 0 2024-03-21T10:09:08.174 INFO:tasks.workunit.client.1.smithi174.stdout:2/747: dread d5/f14 [0,4194304] 0 2024-03-21T10:09:08.175 INFO:tasks.workunit.client.0.smithi153.stdout:9/999: dwrite d1/d43/d44/d46/d9e/f31 [4194304,4194304] 0 2024-03-21T10:09:08.176 INFO:tasks.workunit.client.1.smithi174.stdout:8/773: dread d9/d2a/d39/d76/d19/f7d [0,4194304] 0 2024-03-21T10:09:08.183 INFO:tasks.workunit.client.1.smithi174.stdout:7/715: dread d0/d4/d10/d1b/f3b [0,4194304] 0 2024-03-21T10:09:08.184 INFO:tasks.workunit.client.1.smithi174.stdout:1/903: dwrite d4/d7/ff [0,4194304] 0 2024-03-21T10:09:08.185 INFO:tasks.workunit.client.0.smithi153.stdout:8/876: read d5/d9/d43/d7b/dc4/da7/fc6 [1905860,60240] 0 2024-03-21T10:09:08.187 INFO:tasks.workunit.client.0.smithi153.stdout:1/818: dread d6/d13/d8e/dbb/de1/de7/de9/df5/fa7 [0,4194304] 0 2024-03-21T10:09:08.189 INFO:tasks.workunit.client.0.smithi153.stdout:1/819: dread - d6/d10/fc1 zero size 2024-03-21T10:09:08.192 INFO:tasks.workunit.client.0.smithi153.stdout:8/877: creat d5/d9/d43/d7b/daa/f116 x:0 0 0 2024-03-21T10:09:08.193 INFO:tasks.workunit.client.1.smithi174.stdout:3/781: dwrite d0/d12/d14/d1a/f34 [0,4194304] 0 2024-03-21T10:09:08.194 INFO:tasks.workunit.client.0.smithi153.stdout:1/820: unlink d6/d32/fdb 0 2024-03-21T10:09:08.197 INFO:tasks.workunit.client.0.smithi153.stdout:8/878: mknod d5/d9/d43/dd/d20/dfb/c117 0 2024-03-21T10:09:08.199 INFO:tasks.workunit.client.1.smithi174.stdout:2/748: fdatasync d5/d25/d27/d2d/d45/d5b/d2f/d66/fbb 0 2024-03-21T10:09:08.201 INFO:tasks.workunit.client.0.smithi153.stdout:5/980: dwrite d4/d8/d5a/d5d/f55 [4194304,4194304] 0 2024-03-21T10:09:08.205 INFO:tasks.workunit.client.1.smithi174.stdout:7/716: mkdir d0/d4/d10/d1b/d86/dea/df4/df8 0 2024-03-21T10:09:08.205 INFO:tasks.workunit.client.1.smithi174.stdout:4/895: dread d4/d21/d24/d28/f4e [0,4194304] 0 2024-03-21T10:09:08.207 INFO:tasks.workunit.client.1.smithi174.stdout:1/904: mkdir d4/d7/d32/d142 0 2024-03-21T10:09:08.207 INFO:tasks.workunit.client.0.smithi153.stdout:1/821: symlink d6/d32/ddd/l10c 0 2024-03-21T10:09:08.208 INFO:tasks.workunit.client.0.smithi153.stdout:8/879: rename d5/d9/d43/d7b/f98 to d5/d9/d43/dd/f118 0 2024-03-21T10:09:08.210 INFO:tasks.workunit.client.0.smithi153.stdout:4/966: write d11/f96 [4139274,100905] 0 2024-03-21T10:09:08.211 INFO:tasks.workunit.client.1.smithi174.stdout:0/787: dread d9/d21/d31/f4c [0,4194304] 0 2024-03-21T10:09:08.212 INFO:tasks.workunit.client.0.smithi153.stdout:5/981: creat d4/d5/df/d67/d86/f143 x:0 0 0 2024-03-21T10:09:08.214 INFO:tasks.workunit.client.1.smithi174.stdout:7/717: symlink d0/d4/d10/d60/lf9 0 2024-03-21T10:09:08.214 INFO:tasks.workunit.client.1.smithi174.stdout:1/905: readlink d4/d7/d68/d7b/l140 0 2024-03-21T10:09:08.215 INFO:tasks.workunit.client.1.smithi174.stdout:2/749: creat d5/d25/d27/d2d/d79/df5/f100 x:0 0 0 2024-03-21T10:09:08.216 INFO:tasks.workunit.client.0.smithi153.stdout:1/822: creat d6/d13/d8e/dbb/de1/de7/de9/df5/d50/d51/f10d x:0 0 0 2024-03-21T10:09:08.218 INFO:tasks.workunit.client.0.smithi153.stdout:8/880: rename d5/f10e to d5/d9/d43/dd/ddd/f119 0 2024-03-21T10:09:08.219 INFO:tasks.workunit.client.1.smithi174.stdout:4/896: rmdir d4/d21/d24/d28/de9/d3a/d7a/d9d/da3 39 2024-03-21T10:09:08.221 INFO:tasks.workunit.client.1.smithi174.stdout:0/788: rmdir d9/d35/d4a/d67/d89/d99 39 2024-03-21T10:09:08.221 INFO:tasks.workunit.client.0.smithi153.stdout:5/982: symlink d4/d5/df/l144 0 2024-03-21T10:09:08.224 INFO:tasks.workunit.client.1.smithi174.stdout:6/696: dwrite d0/d3/d5/f1c [0,4194304] 0 2024-03-21T10:09:08.224 INFO:tasks.workunit.client.1.smithi174.stdout:2/750: chown d5/d25/d27/d2d/db4/dbc 1899492499 1 2024-03-21T10:09:08.224 INFO:tasks.workunit.client.1.smithi174.stdout:3/782: creat d0/d12/d14/d1a/d27/ff8 x:0 0 0 2024-03-21T10:09:08.227 INFO:tasks.workunit.client.0.smithi153.stdout:1/823: unlink d6/d10/d35/da5/d4f/d9d/fd2 0 2024-03-21T10:09:08.231 INFO:tasks.workunit.client.0.smithi153.stdout:0/858: sync 2024-03-21T10:09:08.232 INFO:tasks.workunit.client.0.smithi153.stdout:7/988: dread d4/d14/dd0/d83/f89 [0,4194304] 0 2024-03-21T10:09:08.235 INFO:tasks.workunit.client.1.smithi174.stdout:6/697: mknod d0/d3/d20/d6d/cdf 0 2024-03-21T10:09:08.235 INFO:tasks.workunit.client.0.smithi153.stdout:5/983: rename d4/c2e to d4/d5/df/d67/d86/d96/deb/df0/c145 0 2024-03-21T10:09:08.237 INFO:tasks.workunit.client.0.smithi153.stdout:5/984: stat d4/d5/fd4 0 2024-03-21T10:09:08.238 INFO:tasks.workunit.client.1.smithi174.stdout:3/783: read - d0/d12/d14/d1a/d5c/f8d zero size 2024-03-21T10:09:08.243 INFO:tasks.workunit.client.1.smithi174.stdout:2/751: fdatasync d5/d25/fe2 0 2024-03-21T10:09:08.244 INFO:tasks.workunit.client.1.smithi174.stdout:0/789: creat d9/d21/d31/d9f/f110 x:0 0 0 2024-03-21T10:09:08.246 INFO:tasks.workunit.client.1.smithi174.stdout:6/698: creat d0/d3/d5/d21/fe0 x:0 0 0 2024-03-21T10:09:08.247 INFO:tasks.workunit.client.0.smithi153.stdout:7/989: rename d4/d2a/dc6/dfb/c75 to d4/d2a/d3b/d81/c14d 0 2024-03-21T10:09:08.247 INFO:tasks.workunit.client.1.smithi174.stdout:1/906: dwrite d4/d7/ff [0,4194304] 0 2024-03-21T10:09:08.251 INFO:tasks.workunit.client.1.smithi174.stdout:9/816: sync 2024-03-21T10:09:08.253 INFO:tasks.workunit.client.0.smithi153.stdout:4/967: dread d11/d18/d50/d65/de3/f11c [0,4194304] 0 2024-03-21T10:09:08.256 INFO:tasks.workunit.client.0.smithi153.stdout:4/968: write d11/d2d/da3/d103/f11b [268348,33285] 0 2024-03-21T10:09:08.258 INFO:tasks.workunit.client.1.smithi174.stdout:0/790: mkdir d9/d21/d31/d9f/d61/d111 0 2024-03-21T10:09:08.258 INFO:tasks.workunit.client.1.smithi174.stdout:7/718: write d0/d4/d10/d79/db2/dda/d5f/f71 [3372258,71268] 0 2024-03-21T10:09:08.258 INFO:tasks.workunit.client.1.smithi174.stdout:6/699: write d0/d3/d5/df/d16/da9/db7/dc2/fdd [759739,88762] 0 2024-03-21T10:09:08.260 INFO:tasks.workunit.client.0.smithi153.stdout:4/969: mkdir d11/d18/d6d/d122/d131 0 2024-03-21T10:09:08.263 INFO:tasks.workunit.client.1.smithi174.stdout:8/774: dread d9/dc6/fa5 [0,4194304] 0 2024-03-21T10:09:08.269 INFO:tasks.workunit.client.1.smithi174.stdout:3/784: readlink d0/d12/d14/d1a/d27/d3c/l58 0 2024-03-21T10:09:08.271 INFO:tasks.workunit.client.1.smithi174.stdout:2/752: truncate d5/d25/d27/d2d/d45/dae/dd6/d7b/d81/da1/fe0 760540 0 2024-03-21T10:09:08.275 INFO:tasks.workunit.client.0.smithi153.stdout:8/881: write d5/d9/d43/dd/d20/d26/d47/d9a/fda [995072,105502] 0 2024-03-21T10:09:08.275 INFO:tasks.workunit.client.1.smithi174.stdout:4/897: write d4/d21/f27 [646318,35118] 0 2024-03-21T10:09:08.278 INFO:tasks.workunit.client.0.smithi153.stdout:0/859: write d4/d4c/d83/f20 [2064974,9008] 0 2024-03-21T10:09:08.282 INFO:tasks.workunit.client.0.smithi153.stdout:8/882: rename d5/d9/d43/f77 to d5/d9/d43/d7b/daa/dca/f11a 0 2024-03-21T10:09:08.283 INFO:tasks.workunit.client.0.smithi153.stdout:1/824: truncate d6/d13/d8e/dbb/de1/de7/de9/df5/d50/d51/f5f 627685 0 2024-03-21T10:09:08.284 INFO:tasks.workunit.client.0.smithi153.stdout:0/860: symlink d4/d16/d19/d30/d6c/d98/d5e/l113 0 2024-03-21T10:09:08.285 INFO:tasks.workunit.client.0.smithi153.stdout:1/825: chown d6/d13/d8e/dbb 14788275 1 2024-03-21T10:09:08.287 INFO:tasks.workunit.client.0.smithi153.stdout:7/990: truncate d4/d2a/dc6/dfb/f2c 2739456 0 2024-03-21T10:09:08.294 INFO:tasks.workunit.client.1.smithi174.stdout:8/775: creat d9/d2a/d39/d76/d19/d27/d60/daa/df2/ffa x:0 0 0 2024-03-21T10:09:08.294 INFO:tasks.workunit.client.1.smithi174.stdout:1/907: dwrite d4/d7/d15/d53/d78/f126 [0,4194304] 0 2024-03-21T10:09:08.297 INFO:tasks.workunit.client.0.smithi153.stdout:1/826: creat d6/d13/d8e/dbb/de1/de7/de9/f10e x:0 0 0 2024-03-21T10:09:08.299 INFO:tasks.workunit.client.1.smithi174.stdout:0/791: dwrite d9/ff4 [0,4194304] 0 2024-03-21T10:09:08.301 INFO:tasks.workunit.client.1.smithi174.stdout:8/776: truncate d9/dc6/fdc 153680 0 2024-03-21T10:09:08.302 INFO:tasks.workunit.client.0.smithi153.stdout:7/991: truncate d4/d2a/d3b/d60/ff4 967016 0 2024-03-21T10:09:08.302 INFO:tasks.workunit.client.0.smithi153.stdout:1/827: chown d6/d10/d35/da5/d4f 218 1 2024-03-21T10:09:08.304 INFO:tasks.workunit.client.1.smithi174.stdout:3/785: getdents d0/d12/d14/d1a/d44 0 2024-03-21T10:09:08.305 INFO:tasks.workunit.client.0.smithi153.stdout:1/828: write f5 [2273309,111362] 0 2024-03-21T10:09:08.307 INFO:tasks.workunit.client.0.smithi153.stdout:7/992: write d4/d14/d73/fb8 [2677645,92013] 0 2024-03-21T10:09:08.309 INFO:tasks.workunit.client.0.smithi153.stdout:4/970: dwrite d11/d18/f41 [8388608,4194304] 0 2024-03-21T10:09:08.314 INFO:tasks.workunit.client.1.smithi174.stdout:6/700: dwrite d0/d3/d5/df/f22 [0,4194304] 0 2024-03-21T10:09:08.315 INFO:tasks.workunit.client.1.smithi174.stdout:8/777: dread - d9/d2a/d39/d58/dbb/fd8 zero size 2024-03-21T10:09:08.316 INFO:tasks.workunit.client.1.smithi174.stdout:4/898: dwrite d4/d21/fd6 [4194304,4194304] 0 2024-03-21T10:09:08.316 INFO:tasks.workunit.client.1.smithi174.stdout:0/792: mknod d9/d21/d75/da1/c112 0 2024-03-21T10:09:08.318 INFO:tasks.workunit.client.1.smithi174.stdout:5/811: sync 2024-03-21T10:09:08.319 INFO:tasks.workunit.client.1.smithi174.stdout:6/701: readlink d0/d3/d5/df/d16/lb0 0 2024-03-21T10:09:08.319 INFO:tasks.workunit.client.0.smithi153.stdout:7/993: fsync d4/d2a/f3e 0 2024-03-21T10:09:08.319 INFO:tasks.workunit.client.0.smithi153.stdout:1/829: mkdir d6/d10f 0 2024-03-21T10:09:08.323 INFO:tasks.workunit.client.1.smithi174.stdout:9/817: dwrite d2/d11/d7d/f110 [0,4194304] 0 2024-03-21T10:09:08.325 INFO:tasks.workunit.client.0.smithi153.stdout:8/883: dwrite d5/d9/d43/dd/d20/d26/d8b/fa4 [0,4194304] 0 2024-03-21T10:09:08.326 INFO:tasks.workunit.client.1.smithi174.stdout:0/793: mkdir d9/d21/d31/d5c/dd5/d113 0 2024-03-21T10:09:08.326 INFO:tasks.workunit.client.1.smithi174.stdout:4/899: truncate d4/f3e 1574701 0 2024-03-21T10:09:08.327 INFO:tasks.workunit.client.1.smithi174.stdout:6/702: write d0/d3/d5/df/fae [6633800,18264] 0 2024-03-21T10:09:08.329 INFO:tasks.workunit.client.1.smithi174.stdout:9/818: fsync d2/d11/d60/f108 0 2024-03-21T10:09:08.330 INFO:tasks.workunit.client.1.smithi174.stdout:5/812: chown d1/d16/d2e/d66/ff2 1 1 2024-03-21T10:09:08.336 INFO:tasks.workunit.client.1.smithi174.stdout:2/753: dwrite d5/f14 [8388608,4194304] 0 2024-03-21T10:09:08.338 INFO:tasks.workunit.client.1.smithi174.stdout:8/778: creat d9/d53/ffb x:0 0 0 2024-03-21T10:09:08.351 INFO:tasks.workunit.client.1.smithi174.stdout:2/754: creat d5/d25/d27/d2d/d45/dae/dd6/d85/dad/f101 x:0 0 0 2024-03-21T10:09:08.353 INFO:tasks.workunit.client.0.smithi153.stdout:0/861: dread d4/d16/d19/d30/d6c/d98/d5e/db2/ff5 [0,4194304] 0 2024-03-21T10:09:08.353 INFO:tasks.workunit.client.1.smithi174.stdout:8/779: rename d9/d2a/d39/d76/d19/d27/d60/daa/df2/cf9 to d9/d2a/d39/de1/df7/cfc 0 2024-03-21T10:09:08.355 INFO:tasks.workunit.client.1.smithi174.stdout:5/813: getdents d1/d49 0 2024-03-21T10:09:08.356 INFO:tasks.workunit.client.1.smithi174.stdout:5/814: stat d1/d16/f40 0 2024-03-21T10:09:08.359 INFO:tasks.workunit.client.1.smithi174.stdout:9/819: dwrite d2/d11/f75 [0,4194304] 0 2024-03-21T10:09:08.359 INFO:tasks.workunit.client.1.smithi174.stdout:8/780: mknod d9/d2a/d39/d76/cfd 0 2024-03-21T10:09:08.359 INFO:tasks.workunit.client.0.smithi153.stdout:4/971: dwrite d11/feb [0,4194304] 0 2024-03-21T10:09:08.364 INFO:tasks.workunit.client.1.smithi174.stdout:5/815: creat d1/d101/f124 x:0 0 0 2024-03-21T10:09:08.367 INFO:tasks.workunit.client.1.smithi174.stdout:8/781: symlink d9/d2a/d39/lfe 0 2024-03-21T10:09:08.370 INFO:tasks.workunit.client.0.smithi153.stdout:1/830: dwrite d6/d13/d8e/dbb/de1/de7/de9/df5/d50/d51/f10d [0,4194304] 0 2024-03-21T10:09:08.370 INFO:tasks.workunit.client.1.smithi174.stdout:5/816: read d1/d49/d4f/d51/d8f/fb6 [768925,94867] 0 2024-03-21T10:09:08.371 INFO:tasks.workunit.client.1.smithi174.stdout:9/820: mknod d2/d24/d2e/dac/c11c 0 2024-03-21T10:09:08.372 INFO:tasks.workunit.client.1.smithi174.stdout:8/782: creat d9/d2a/d39/d58/dbb/dc3/fff x:0 0 0 2024-03-21T10:09:08.372 INFO:tasks.workunit.client.1.smithi174.stdout:9/821: readlink d2/d11/d1d/d8b/la0 0 2024-03-21T10:09:08.375 INFO:tasks.workunit.client.1.smithi174.stdout:8/783: write d9/d53/f78 [4707875,31684] 0 2024-03-21T10:09:08.375 INFO:tasks.workunit.client.1.smithi174.stdout:9/822: dread - d2/d11/d60/f104 zero size 2024-03-21T10:09:08.376 INFO:tasks.workunit.client.1.smithi174.stdout:5/817: rmdir d1/d5c/d73/de3 39 2024-03-21T10:09:08.382 INFO:tasks.workunit.client.1.smithi174.stdout:8/784: write d9/d2a/fec [100382,45490] 0 2024-03-21T10:09:08.387 INFO:tasks.workunit.client.1.smithi174.stdout:1/908: truncate d4/d2d/f76 3905192 0 2024-03-21T10:09:08.387 INFO:tasks.workunit.client.1.smithi174.stdout:0/794: dread d9/d1c/d76/d87/faa [0,4194304] 0 2024-03-21T10:09:08.387 INFO:tasks.workunit.client.1.smithi174.stdout:5/818: dread - d1/d2f/d37/fe7 zero size 2024-03-21T10:09:08.387 INFO:tasks.workunit.client.0.smithi153.stdout:1/831: dwrite d6/fd9 [0,4194304] 0 2024-03-21T10:09:08.394 INFO:tasks.workunit.client.1.smithi174.stdout:4/900: write d4/d21/d24/d28/d38/f54 [855791,126377] 0 2024-03-21T10:09:08.398 INFO:tasks.workunit.client.1.smithi174.stdout:8/785: creat d9/d2a/d39/d58/f100 x:0 0 0 2024-03-21T10:09:08.401 INFO:tasks.workunit.client.1.smithi174.stdout:3/786: dwrite d0/f91 [0,4194304] 0 2024-03-21T10:09:08.407 INFO:tasks.workunit.client.0.smithi153.stdout:7/994: rename d4/d14/d2f/d78/f3a to d4/d19/d4d/d10e/f14e 0 2024-03-21T10:09:08.409 INFO:tasks.workunit.client.0.smithi153.stdout:8/884: unlink d5/d9/d43/f35 0 2024-03-21T10:09:08.412 INFO:tasks.workunit.client.1.smithi174.stdout:2/755: truncate d5/d25/d27/f80 1590936 0 2024-03-21T10:09:08.412 INFO:tasks.workunit.client.1.smithi174.stdout:4/901: creat d4/d21/d24/d28/de9/d3a/d7a/d9d/d11f/f129 x:0 0 0 2024-03-21T10:09:08.421 INFO:tasks.workunit.client.1.smithi174.stdout:7/719: dread d0/d4/d10/d79/db2/dda/d5f/f71 [0,4194304] 0 2024-03-21T10:09:08.422 INFO:tasks.workunit.client.0.smithi153.stdout:7/995: rename d4/d2a/d3b/d81 to d4/d2a/d3b/d14f 0 2024-03-21T10:09:08.426 INFO:tasks.workunit.client.1.smithi174.stdout:0/795: getdents d9/d21/d31/d5c/d92/d98 0 2024-03-21T10:09:08.427 INFO:tasks.workunit.client.1.smithi174.stdout:0/796: dread - d9/d21/d31/d9f/f110 zero size 2024-03-21T10:09:08.432 INFO:tasks.workunit.client.0.smithi153.stdout:5/985: sync 2024-03-21T10:09:08.433 INFO:tasks.workunit.client.1.smithi174.stdout:6/703: dread d0/d3/d5/df/fac [0,4194304] 0 2024-03-21T10:09:08.434 INFO:tasks.workunit.client.1.smithi174.stdout:5/819: rename d1/d2f/d37/d9b/laa to d1/d2f/d37/l125 0 2024-03-21T10:09:08.435 INFO:tasks.workunit.client.0.smithi153.stdout:5/986: symlink d4/d5/df/d67/d6a/l146 0 2024-03-21T10:09:08.438 INFO:tasks.workunit.client.0.smithi153.stdout:5/987: fdatasync d4/d5/df/d67/d86/d96/deb/d56/d105/f140 0 2024-03-21T10:09:08.439 INFO:tasks.workunit.client.1.smithi174.stdout:7/720: creat d0/d4/d10/d1b/d86/dea/ffa x:0 0 0 2024-03-21T10:09:08.439 INFO:tasks.workunit.client.0.smithi153.stdout:5/988: stat d4/d5/df/d67/d6a/f77 0 2024-03-21T10:09:08.440 INFO:tasks.workunit.client.1.smithi174.stdout:0/797: creat d9/d1c/d76/d87/dcd/f114 x:0 0 0 2024-03-21T10:09:08.440 INFO:tasks.workunit.client.0.smithi153.stdout:0/862: dread d4/d16/d19/d95/fb5 [0,4194304] 0 2024-03-21T10:09:08.443 INFO:tasks.workunit.client.1.smithi174.stdout:0/798: stat l8 0 2024-03-21T10:09:08.445 INFO:tasks.workunit.client.1.smithi174.stdout:4/902: rename d4/d21/d24/d28/d79/d108 to d4/d21/d24/d28/de9/d12a 0 2024-03-21T10:09:08.447 INFO:tasks.workunit.client.1.smithi174.stdout:7/721: fdatasync d0/d4/d10/d1b/d86/dea/ffa 0 2024-03-21T10:09:08.448 INFO:tasks.workunit.client.0.smithi153.stdout:0/863: rmdir d4 39 2024-03-21T10:09:08.450 INFO:tasks.workunit.client.0.smithi153.stdout:5/989: symlink d4/d8/d5a/d5d/dd9/d38/l147 0 2024-03-21T10:09:08.450 INFO:tasks.workunit.client.1.smithi174.stdout:9/823: dwrite d2/d11/d1d/d4b/f56 [4194304,4194304] 0 2024-03-21T10:09:08.451 INFO:tasks.workunit.client.0.smithi153.stdout:4/972: dread d11/d18/f64 [0,4194304] 0 2024-03-21T10:09:08.453 INFO:tasks.workunit.client.1.smithi174.stdout:6/704: rename d0/d3/d5/df/d16/c9d to d0/d3/d5/df/d16/da9/ce1 0 2024-03-21T10:09:08.453 INFO:tasks.workunit.client.1.smithi174.stdout:0/799: stat d9/d35/dd2/d104/ff5 0 2024-03-21T10:09:08.453 INFO:tasks.workunit.client.1.smithi174.stdout:3/787: write d0/d12/d14/da9/de7/db7/f38 [5169192,70167] 0 2024-03-21T10:09:08.455 INFO:tasks.workunit.client.1.smithi174.stdout:2/756: dwrite d5/d25/d27/d2d/d45/dae/dd6/d4f/da0/fde [0,4194304] 0 2024-03-21T10:09:08.461 INFO:tasks.workunit.client.1.smithi174.stdout:5/820: link d1/d49/d4f/ca0 d1/d2f/d37/c126 0 2024-03-21T10:09:08.461 INFO:tasks.workunit.client.1.smithi174.stdout:3/788: chown d0/d12/d14/da9/de7/ca4 264 1 2024-03-21T10:09:08.465 INFO:tasks.workunit.client.1.smithi174.stdout:2/757: rmdir d5/d25 39 2024-03-21T10:09:08.468 INFO:tasks.workunit.client.1.smithi174.stdout:0/800: creat d9/d1c/d76/f115 x:0 0 0 2024-03-21T10:09:08.469 INFO:tasks.workunit.client.1.smithi174.stdout:8/786: truncate d9/d2a/d39/fa0 3764395 0 2024-03-21T10:09:08.471 INFO:tasks.workunit.client.1.smithi174.stdout:6/705: write d0/d3/d2a/d56/d5e/fa7 [1081067,108227] 0 2024-03-21T10:09:08.478 INFO:tasks.workunit.client.1.smithi174.stdout:6/706: chown d0/d3/d5/d21/l4c 34709296 1 2024-03-21T10:09:08.478 INFO:tasks.workunit.client.1.smithi174.stdout:0/801: rmdir d9/da2 39 2024-03-21T10:09:08.482 INFO:tasks.workunit.client.1.smithi174.stdout:2/758: creat d5/d25/d27/d2d/d45/d5b/d2f/d66/f102 x:0 0 0 2024-03-21T10:09:08.482 INFO:tasks.workunit.client.1.smithi174.stdout:1/909: dread d4/d2d/f12a [0,4194304] 0 2024-03-21T10:09:08.483 INFO:tasks.workunit.client.1.smithi174.stdout:6/707: read d0/d3/d5/d6/d42/f82 [128819,42074] 0 2024-03-21T10:09:08.483 INFO:tasks.workunit.client.1.smithi174.stdout:9/824: dwrite d2/d24/f74 [0,4194304] 0 2024-03-21T10:09:08.485 INFO:tasks.workunit.client.0.smithi153.stdout:4/973: dread d11/f20 [0,4194304] 0 2024-03-21T10:09:08.487 INFO:tasks.workunit.client.1.smithi174.stdout:3/789: rename d0/d12/d14/d1a/d27/d3c to d0/d12/d46/df9 0 2024-03-21T10:09:08.487 INFO:tasks.workunit.client.0.smithi153.stdout:8/885: write d5/d9/d43/dd/d20/d26/d7a/f81 [5194276,28013] 0 2024-03-21T10:09:08.488 INFO:tasks.workunit.client.1.smithi174.stdout:9/825: chown d2/d24/d2e/dac/lf0 9 1 2024-03-21T10:09:08.488 INFO:tasks.workunit.client.0.smithi153.stdout:1/832: truncate f4 511291 0 2024-03-21T10:09:08.489 INFO:tasks.workunit.client.0.smithi153.stdout:4/974: creat d11/d71/d89/dca/f132 x:0 0 0 2024-03-21T10:09:08.490 INFO:tasks.workunit.client.0.smithi153.stdout:8/886: readlink d5/d9/d43/dd/d20/d26/d47/l78 0 2024-03-21T10:09:08.491 INFO:tasks.workunit.client.0.smithi153.stdout:7/996: write d4/d14/d16/fed [2760049,31466] 0 2024-03-21T10:09:08.492 INFO:tasks.workunit.client.0.smithi153.stdout:4/975: read d11/f24 [2104091,47801] 0 2024-03-21T10:09:08.493 INFO:tasks.workunit.client.0.smithi153.stdout:0/864: write d4/d4c/d83/d22/f7d [4276407,99172] 0 2024-03-21T10:09:08.493 INFO:tasks.workunit.client.0.smithi153.stdout:5/990: truncate d4/d8/f11 256137 0 2024-03-21T10:09:08.496 INFO:tasks.workunit.client.1.smithi174.stdout:2/759: mkdir d5/d25/d103 0 2024-03-21T10:09:08.497 INFO:tasks.workunit.client.0.smithi153.stdout:7/997: write d4/d14/d16/f39 [3672907,108246] 0 2024-03-21T10:09:08.499 INFO:tasks.workunit.client.0.smithi153.stdout:4/976: dread - d11/d2d/d3d/ffe zero size 2024-03-21T10:09:08.501 INFO:tasks.workunit.client.0.smithi153.stdout:1/833: dread d6/d13/d8e/dbb/de1/de7/de9/df5/fa7 [0,4194304] 0 2024-03-21T10:09:08.502 INFO:tasks.workunit.client.0.smithi153.stdout:7/998: chown d4/d14/dd0/c80 235295262 1 2024-03-21T10:09:08.504 INFO:tasks.workunit.client.0.smithi153.stdout:0/865: rename d4/d4c/d83/d22/c4a to d4/d16/d19/d30/d101/c114 0 2024-03-21T10:09:08.504 INFO:tasks.workunit.client.1.smithi174.stdout:4/903: write d4/d19/f65 [552251,4281] 0 2024-03-21T10:09:08.505 INFO:tasks.workunit.client.0.smithi153.stdout:7/999: write d4/d2a/d3b/f98 [3817138,66517] 0 2024-03-21T10:09:08.506 INFO:tasks.workunit.client.1.smithi174.stdout:6/708: creat d0/d3/d5/df/d66/d87/fe2 x:0 0 0 2024-03-21T10:09:08.507 INFO:tasks.workunit.client.0.smithi153.stdout:8/887: creat d5/d9/d43/d7b/dc4/f11b x:0 0 0 2024-03-21T10:09:08.508 INFO:tasks.workunit.client.0.smithi153.stdout:5/991: mkdir d4/d5/df/d67/d86/d96/deb/df0/d148 0 2024-03-21T10:09:08.509 INFO:tasks.workunit.client.1.smithi174.stdout:3/790: stat d0/d12/d14/da9/de7/db7/d7d/ce9 0 2024-03-21T10:09:08.509 INFO:tasks.workunit.client.1.smithi174.stdout:8/787: dwrite d9/d2a/d39/d76/d19/d27/d60/daa/dab/fe0 [0,4194304] 0 2024-03-21T10:09:08.510 INFO:tasks.workunit.client.0.smithi153.stdout:5/992: stat d4/d5/df/d67/d86/d96 0 2024-03-21T10:09:08.510 INFO:tasks.workunit.client.0.smithi153.stdout:0/866: readlink d4/d16/l59 0 2024-03-21T10:09:08.511 INFO:tasks.workunit.client.1.smithi174.stdout:5/821: write d1/d2f/f4c [1124823,116465] 0 2024-03-21T10:09:08.511 INFO:tasks.workunit.client.0.smithi153.stdout:1/834: mknod d6/d10/d35/da5/d4f/d9d/db6/def/d92/c110 0 2024-03-21T10:09:08.513 INFO:tasks.workunit.client.0.smithi153.stdout:5/993: dread - d4/d8/d5a/d92/d9f/f116 zero size 2024-03-21T10:09:08.514 INFO:tasks.workunit.client.0.smithi153.stdout:1/835: chown d6/d10/da0/fb7 43119 1 2024-03-21T10:09:08.515 INFO:tasks.workunit.client.0.smithi153.stdout:4/977: link d11/db1/lf5 d11/d2d/d6b/l133 0 2024-03-21T10:09:08.515 INFO:tasks.workunit.client.1.smithi174.stdout:8/788: write d9/d53/ffb [290189,127636] 0 2024-03-21T10:09:08.516 INFO:tasks.workunit.client.1.smithi174.stdout:1/910: dwrite d4/d7/d68/d8e/db8/fd8 [0,4194304] 0 2024-03-21T10:09:08.517 INFO:tasks.workunit.client.1.smithi174.stdout:3/791: write d0/d12/d46/df9/f5a [4699548,61347] 0 2024-03-21T10:09:08.518 INFO:tasks.workunit.client.1.smithi174.stdout:0/802: mkdir d9/d21/d31/d9f/d61/db9/d116 0 2024-03-21T10:09:08.520 INFO:tasks.workunit.client.0.smithi153.stdout:5/994: write d4/d8/d5a/d92/d9f/f12b [3590960,123737] 0 2024-03-21T10:09:08.521 INFO:tasks.workunit.client.0.smithi153.stdout:0/867: readlink d4/d4c/d83/l2a 0 2024-03-21T10:09:08.522 INFO:tasks.workunit.client.1.smithi174.stdout:0/803: readlink d9/d21/df6/lfc 0 2024-03-21T10:09:08.523 INFO:tasks.workunit.client.1.smithi174.stdout:7/722: dwrite d0/d4/d10/d79/db2/dda/f8d [0,4194304] 0 2024-03-21T10:09:08.526 INFO:tasks.workunit.client.1.smithi174.stdout:4/904: mknod d4/d21/d24/d28/de9/d3a/dd2/d10a/c12b 0 2024-03-21T10:09:08.528 INFO:tasks.workunit.client.0.smithi153.stdout:5/995: mkdir d4/d5/df/d67/d86/dbf/d149 0 2024-03-21T10:09:08.530 INFO:tasks.workunit.client.1.smithi174.stdout:5/822: stat d1/d5c/d73/de3/l108 0 2024-03-21T10:09:08.530 INFO:tasks.workunit.client.1.smithi174.stdout:6/709: fsync d0/d3/d5/d6/f13 0 2024-03-21T10:09:08.531 INFO:tasks.workunit.client.0.smithi153.stdout:8/888: dwrite d5/d9/d43/dd/d20/d26/d47/d9a/db9/d112/fd6 [0,4194304] 0 2024-03-21T10:09:08.531 INFO:tasks.workunit.client.0.smithi153.stdout:1/836: fsync d6/d13/d75/f83 0 2024-03-21T10:09:08.534 INFO:tasks.workunit.client.1.smithi174.stdout:4/905: write d4/d21/d24/d28/de9/d3a/d7a/d9d/fc2 [4367641,84880] 0 2024-03-21T10:09:08.534 INFO:tasks.workunit.client.1.smithi174.stdout:5/823: chown d1/d5c/d73/de3/df3 11550722 1 2024-03-21T10:09:08.543 INFO:tasks.workunit.client.1.smithi174.stdout:2/760: unlink d5/d25/d27/d2d/d45/d5b/c2b 0 2024-03-21T10:09:08.544 INFO:tasks.workunit.client.0.smithi153.stdout:1/837: dread - d6/d13/d8e/dbb/de1/de7/de9/df5/d50/d51/fd5 zero size 2024-03-21T10:09:08.549 INFO:tasks.workunit.client.0.smithi153.stdout:8/889: creat d5/d9/d43/dd/d20/d26/d47/d9a/f11c x:0 0 0 2024-03-21T10:09:08.554 INFO:tasks.workunit.client.1.smithi174.stdout:6/710: creat d0/d3/d5/df/d16/da9/db7/dc2/dda/fe3 x:0 0 0 2024-03-21T10:09:08.555 INFO:tasks.workunit.client.1.smithi174.stdout:0/804: creat d9/f117 x:0 0 0 2024-03-21T10:09:08.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:08 smithi153 ceph-mon[29117]: pgmap v20: 65 pgs: 65 active+clean; 3.4 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 87 MiB/s rd, 218 MiB/s wr, 456 op/s 2024-03-21T10:09:08.557 INFO:tasks.workunit.client.1.smithi174.stdout:9/826: dwrite d2/d11/d1d/d2a/f107 [0,4194304] 0 2024-03-21T10:09:08.557 INFO:tasks.workunit.client.0.smithi153.stdout:1/838: mkdir d6/d10/d35/da5/d111 0 2024-03-21T10:09:08.557 INFO:tasks.workunit.client.1.smithi174.stdout:8/789: getdents d9/d2a 0 2024-03-21T10:09:08.558 INFO:tasks.workunit.client.0.smithi153.stdout:8/890: mknod d5/d9/d43/d7b/daa/c11d 0 2024-03-21T10:09:08.560 INFO:tasks.workunit.client.1.smithi174.stdout:8/790: stat d9/d2a/d39/d58/dbb/dc3/fdb 0 2024-03-21T10:09:08.560 INFO:tasks.workunit.client.0.smithi153.stdout:1/839: rename d6/f52 to d6/d10/da0/f112 0 2024-03-21T10:09:08.561 INFO:tasks.workunit.client.0.smithi153.stdout:8/891: fdatasync d5/fe1 0 2024-03-21T10:09:08.563 INFO:tasks.workunit.client.1.smithi174.stdout:1/911: dread d4/d7/f2c [4194304,4194304] 0 2024-03-21T10:09:08.565 INFO:tasks.workunit.client.1.smithi174.stdout:7/723: rename d0/d4/d10/d44/d81/f82 to d0/d4/ffb 0 2024-03-21T10:09:08.566 INFO:tasks.workunit.client.0.smithi153.stdout:8/892: unlink d5/d9/d43/dd/cbf 0 2024-03-21T10:09:08.568 INFO:tasks.workunit.client.1.smithi174.stdout:0/805: rmdir d9/d35/d4a 39 2024-03-21T10:09:08.570 INFO:tasks.workunit.client.1.smithi174.stdout:1/912: write d4/d7/d68/d8e/db8/fd5 [603115,61656] 0 2024-03-21T10:09:08.573 INFO:tasks.workunit.client.1.smithi174.stdout:7/724: readlink d0/d4/d38/da9/ld4 0 2024-03-21T10:09:08.573 INFO:tasks.workunit.client.1.smithi174.stdout:3/792: getdents d0/d12/d14/d1a/d27/d4f/dbe 0 2024-03-21T10:09:08.573 INFO:tasks.workunit.client.1.smithi174.stdout:1/913: chown d4/d5/d6c/lec 72 1 2024-03-21T10:09:08.576 INFO:tasks.workunit.client.1.smithi174.stdout:1/914: readlink d4/d7/d68/d7b/de0/l127 0 2024-03-21T10:09:08.576 INFO:tasks.workunit.client.1.smithi174.stdout:7/725: chown d0/d4/d10/d44 215446096 1 2024-03-21T10:09:08.577 INFO:tasks.workunit.client.0.smithi153.stdout:5/996: truncate d4/d5/df/d67/d86/d96/deb/df0/d10a/f126 3373619 0 2024-03-21T10:09:08.581 INFO:tasks.workunit.client.1.smithi174.stdout:8/791: rename d9/l1d to d9/d2a/d39/de1/df7/l101 0 2024-03-21T10:09:08.583 INFO:tasks.workunit.client.0.smithi153.stdout:0/868: dread d4/db/f13 [0,4194304] 0 2024-03-21T10:09:08.584 INFO:tasks.workunit.client.1.smithi174.stdout:8/792: write d9/dc6/f3b [209596,24442] 0 2024-03-21T10:09:08.590 INFO:tasks.workunit.client.1.smithi174.stdout:8/793: dread - d9/d2a/d39/d58/dbb/dc3/fff zero size 2024-03-21T10:09:08.590 INFO:tasks.workunit.client.1.smithi174.stdout:5/824: write d1/f2c [1237298,115573] 0 2024-03-21T10:09:08.591 INFO:tasks.workunit.client.1.smithi174.stdout:2/761: getdents d5/d25/d27/d2d/d45 0 2024-03-21T10:09:08.591 INFO:tasks.workunit.client.1.smithi174.stdout:3/793: truncate d0/f33 1026272 0 2024-03-21T10:09:08.591 INFO:tasks.workunit.client.1.smithi174.stdout:1/915: dread d4/d7/d32/fee [0,4194304] 0 2024-03-21T10:09:08.598 INFO:tasks.workunit.client.0.smithi153.stdout:0/869: dread d4/d4c/d83/d22/f73 [4194304,4194304] 0 2024-03-21T10:09:08.598 INFO:tasks.workunit.client.0.smithi153.stdout:5/997: dwrite d4/d5/fd4 [0,4194304] 0 2024-03-21T10:09:08.598 INFO:tasks.workunit.client.1.smithi174.stdout:1/916: write d4/d7/d33/d43/d77/fc2 [1100087,98423] 0 2024-03-21T10:09:08.601 INFO:tasks.workunit.client.1.smithi174.stdout:1/917: dread - d4/d7/d70/fe8 zero size 2024-03-21T10:09:08.602 INFO:tasks.workunit.client.0.smithi153.stdout:0/870: mkdir d4/d16/d19/d30/d6c/d98/df3/d115 0 2024-03-21T10:09:08.603 INFO:tasks.workunit.client.0.smithi153.stdout:1/840: dread d6/f9 [0,4194304] 0 2024-03-21T10:09:08.605 INFO:tasks.workunit.client.1.smithi174.stdout:6/711: dread d0/d3/d5/df/d66/f96 [0,4194304] 0 2024-03-21T10:09:08.605 INFO:tasks.workunit.client.0.smithi153.stdout:5/998: unlink d4/d8/d5a/d5d/dd9/d38/c7f 0 2024-03-21T10:09:08.606 INFO:tasks.workunit.client.1.smithi174.stdout:7/726: truncate d0/d4/d10/d44/d81/d9a/ff5 2238416 0 2024-03-21T10:09:08.606 INFO:tasks.workunit.client.0.smithi153.stdout:5/999: stat d4/d8/d5c/l9c 0 2024-03-21T10:09:08.607 INFO:tasks.workunit.client.0.smithi153.stdout:1/841: truncate d6/d10/d35/da5/dea/fee 753179 0 2024-03-21T10:09:08.608 INFO:tasks.workunit.client.1.smithi174.stdout:5/825: unlink d1/f4a 0 2024-03-21T10:09:08.608 INFO:tasks.workunit.client.1.smithi174.stdout:6/712: write d0/d3/d2a/d56/d5e/fa7 [1810981,86735] 0 2024-03-21T10:09:08.609 INFO:tasks.workunit.client.1.smithi174.stdout:9/827: dread d2/d11/f10e [0,4194304] 0 2024-03-21T10:09:08.610 INFO:tasks.workunit.client.0.smithi153.stdout:8/893: dread d5/d9/d43/dd/d20/d26/d8b/f4e [0,4194304] 0 2024-03-21T10:09:08.612 INFO:tasks.workunit.client.1.smithi174.stdout:8/794: rename d9/d2a/d39/d76/d19/d27/d60/lcf to d9/d2a/d39/de1/df7/l102 0 2024-03-21T10:09:08.614 INFO:tasks.workunit.client.0.smithi153.stdout:1/842: write d6/d10/d35/da5/d4f/f58 [4233813,94480] 0 2024-03-21T10:09:08.614 INFO:tasks.workunit.client.1.smithi174.stdout:2/762: dread - d5/d25/d27/d2d/d45/dae/dd6/d4f/d65/fc7 zero size 2024-03-21T10:09:08.615 INFO:tasks.workunit.client.0.smithi153.stdout:8/894: mknod d5/d9/d43/dd/d20/d26/d47/df4/c11e 0 2024-03-21T10:09:08.616 INFO:tasks.workunit.client.1.smithi174.stdout:0/806: getdents d9/d21/d23/da5 0 2024-03-21T10:09:08.618 INFO:tasks.workunit.client.1.smithi174.stdout:7/727: rmdir d0/d4/d10/d1b/d86/dd2 39 2024-03-21T10:09:08.619 INFO:tasks.workunit.client.1.smithi174.stdout:7/728: chown d0/d4/d38 3539129 1 2024-03-21T10:09:08.620 INFO:tasks.workunit.client.1.smithi174.stdout:6/713: unlink d0/d3/d5/d3b/c77 0 2024-03-21T10:09:08.621 INFO:tasks.workunit.client.0.smithi153.stdout:0/871: dwrite d4/d16/d19/d30/f41 [0,4194304] 0 2024-03-21T10:09:08.623 INFO:tasks.workunit.client.1.smithi174.stdout:9/828: rmdir d2/d10b 39 2024-03-21T10:09:08.623 INFO:tasks.workunit.client.1.smithi174.stdout:8/795: mkdir d9/d2a/d103 0 2024-03-21T10:09:08.624 INFO:tasks.workunit.client.0.smithi153.stdout:0/872: truncate d4/d16/d19/fc3 672026 0 2024-03-21T10:09:08.628 INFO:tasks.workunit.client.1.smithi174.stdout:2/763: mknod d5/d25/d103/c104 0 2024-03-21T10:09:08.628 INFO:tasks.workunit.client.0.smithi153.stdout:0/873: chown d4/d16/d19/d30/d101/c114 50302 1 2024-03-21T10:09:08.628 INFO:tasks.workunit.client.1.smithi174.stdout:1/918: link d4/d7/d68/d7b/l140 d4/d7/d68/d89/l143 0 2024-03-21T10:09:08.631 INFO:tasks.workunit.client.1.smithi174.stdout:7/729: getdents d0/d4/d10/d79/db2 0 2024-03-21T10:09:08.632 INFO:tasks.workunit.client.1.smithi174.stdout:3/794: dread d0/d12/d14/d98/fd6 [0,4194304] 0 2024-03-21T10:09:08.632 INFO:tasks.workunit.client.0.smithi153.stdout:0/874: truncate d4/d4c/d83/d22/f47 1239167 0 2024-03-21T10:09:08.633 INFO:tasks.workunit.client.1.smithi174.stdout:2/764: truncate d5/d25/d27/d2d/d45/ffd 166612 0 2024-03-21T10:09:08.636 INFO:tasks.workunit.client.0.smithi153.stdout:8/895: write d5/d9/d43/dd/d2d/f31 [1565057,47106] 0 2024-03-21T10:09:08.638 INFO:tasks.workunit.client.1.smithi174.stdout:1/919: getdents d4/d7/d33/d43/d51/da4/d134/d13a 0 2024-03-21T10:09:08.639 INFO:tasks.workunit.client.0.smithi153.stdout:0/875: dread - d4/d16/d19/d30/d6c/d98/d5e/dbe/fdf zero size 2024-03-21T10:09:08.640 INFO:tasks.workunit.client.1.smithi174.stdout:3/795: write d0/d12/d14/d1a/d44/d56/fa2 [56651,86894] 0 2024-03-21T10:09:08.640 INFO:tasks.workunit.client.1.smithi174.stdout:7/730: chown d0/d4/d10/d1b/d86/l97 101 1 2024-03-21T10:09:08.642 INFO:tasks.workunit.client.0.smithi153.stdout:8/896: read d5/d9/d43/dd/d20/d26/d8b/f62 [188415,93149] 0 2024-03-21T10:09:08.642 INFO:tasks.workunit.client.1.smithi174.stdout:2/765: creat d5/d25/d27/d2d/d45/d5b/d2f/d4b/d76/f105 x:0 0 0 2024-03-21T10:09:08.646 INFO:tasks.workunit.client.1.smithi174.stdout:6/714: dwrite d0/d3/d20/d3d/d81/da5/fc9 [0,4194304] 0 2024-03-21T10:09:08.646 INFO:tasks.workunit.client.1.smithi174.stdout:1/920: creat d4/d7/d28/d59/f144 x:0 0 0 2024-03-21T10:09:08.651 INFO:tasks.workunit.client.1.smithi174.stdout:3/796: fdatasync d0/d12/d14/f97 0 2024-03-21T10:09:08.653 INFO:tasks.workunit.client.1.smithi174.stdout:0/807: write d9/d21/d31/d5c/f7c [735950,107837] 0 2024-03-21T10:09:08.662 INFO:tasks.workunit.client.0.smithi153.stdout:0/876: dwrite d4/d4c/d83/d22/d61/d77/fb1 [0,4194304] 0 2024-03-21T10:09:08.663 INFO:tasks.workunit.client.1.smithi174.stdout:7/731: unlink d0/d4/d10/d44/d81/d9a/fdf 0 2024-03-21T10:09:08.664 INFO:tasks.workunit.client.0.smithi153.stdout:0/877: fdatasync d4/d16/d19/d58/fbf 0 2024-03-21T10:09:08.666 INFO:tasks.workunit.client.1.smithi174.stdout:2/766: write d5/d25/fe2 [1286657,30784] 0 2024-03-21T10:09:08.670 INFO:tasks.workunit.client.0.smithi153.stdout:0/878: link d4/d4c/d83/f20 d4/d16/d19/dad/d100/f116 0 2024-03-21T10:09:08.671 INFO:tasks.workunit.client.1.smithi174.stdout:0/808: chown d9/d35/d4a/d67/d89/d99/db2 21 1 2024-03-21T10:09:08.673 INFO:tasks.workunit.client.0.smithi153.stdout:0/879: write d4/db/fa5 [3558318,13024] 0 2024-03-21T10:09:08.675 INFO:tasks.workunit.client.1.smithi174.stdout:8/796: dwrite d9/d2a/d4d/db4/fb6 [0,4194304] 0 2024-03-21T10:09:08.678 INFO:tasks.workunit.client.1.smithi174.stdout:7/732: rename d0/d4/l15 to d0/lfc 0 2024-03-21T10:09:08.678 INFO:tasks.workunit.client.1.smithi174.stdout:2/767: mkdir d5/d25/d27/d2d/d45/dae/dd6/d4f/d65/d77/d106 0 2024-03-21T10:09:08.678 INFO:tasks.workunit.client.1.smithi174.stdout:0/809: symlink d9/dd8/l118 0 2024-03-21T10:09:08.679 INFO:tasks.workunit.client.1.smithi174.stdout:8/797: chown d9/d53/ffb 5793459 1 2024-03-21T10:09:08.681 INFO:tasks.workunit.client.1.smithi174.stdout:2/768: write d5/f11 [8942996,113977] 0 2024-03-21T10:09:08.682 INFO:tasks.workunit.client.0.smithi153.stdout:0/880: link d4/d16/d19/d30/d6c/d98/d5e/db2/ffa d4/d16/d19/d58/d9a/f117 0 2024-03-21T10:09:08.682 INFO:tasks.workunit.client.1.smithi174.stdout:1/921: truncate d4/d7/d33/d43/d77/d87/f139 2194855 0 2024-03-21T10:09:08.683 INFO:tasks.workunit.client.0.smithi153.stdout:8/897: dwrite d5/d9/d43/dd/d20/d26/d47/d9a/db9/d112/d52/f54 [0,4194304] 0 2024-03-21T10:09:08.683 INFO:tasks.workunit.client.1.smithi174.stdout:0/810: chown d9/d35/d4a/d67/d89/fc2 2234011 1 2024-03-21T10:09:08.685 INFO:tasks.workunit.client.0.smithi153.stdout:0/881: fdatasync d4/d16/d19/d30/d6c/d98/f80 0 2024-03-21T10:09:08.688 INFO:tasks.workunit.client.0.smithi153.stdout:0/882: chown d4/d16/d19/d30/d6c/d98/df3/ff0 108717 1 2024-03-21T10:09:08.688 INFO:tasks.workunit.client.1.smithi174.stdout:0/811: stat d9/d21/f7f 0 2024-03-21T10:09:08.689 INFO:tasks.workunit.client.1.smithi174.stdout:8/798: symlink d9/d2a/d4d/db2/l104 0 2024-03-21T10:09:08.689 INFO:tasks.workunit.client.0.smithi153.stdout:8/898: mkdir d5/d9/d43/dd/d20/d11f 0 2024-03-21T10:09:08.692 INFO:tasks.workunit.client.0.smithi153.stdout:8/899: read d5/d9/d43/d7b/daa/fb8 [2968367,28530] 0 2024-03-21T10:09:08.693 INFO:tasks.workunit.client.1.smithi174.stdout:8/799: write d9/d2a/d39/d76/f87 [3431751,110857] 0 2024-03-21T10:09:08.697 INFO:tasks.workunit.client.1.smithi174.stdout:8/800: fdatasync d9/d53/ffb 0 2024-03-21T10:09:08.697 INFO:tasks.workunit.client.1.smithi174.stdout:1/922: rename d4/d7/d15/d4f/f82 to d4/d7/d32/da2/da8/f145 0 2024-03-21T10:09:08.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:08 smithi174 ceph-mon[37496]: pgmap v20: 65 pgs: 65 active+clean; 3.4 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 87 MiB/s rd, 218 MiB/s wr, 456 op/s 2024-03-21T10:09:08.701 INFO:tasks.workunit.client.1.smithi174.stdout:1/923: dread - d4/d7/d15/f129 zero size 2024-03-21T10:09:08.704 INFO:tasks.workunit.client.1.smithi174.stdout:8/801: rmdir d9/d2a/d39/d58/dbb/dc3 39 2024-03-21T10:09:08.708 INFO:tasks.workunit.client.1.smithi174.stdout:1/924: creat d4/d7/d33/d43/d77/d87/f146 x:0 0 0 2024-03-21T10:09:08.709 INFO:tasks.workunit.client.1.smithi174.stdout:8/802: chown d9/d2a/d39/l4b 6977 1 2024-03-21T10:09:08.715 INFO:tasks.workunit.client.1.smithi174.stdout:7/733: dwrite d0/f88 [0,4194304] 0 2024-03-21T10:09:08.720 INFO:tasks.workunit.client.1.smithi174.stdout:0/812: dwrite d9/d21/f26 [0,4194304] 0 2024-03-21T10:09:08.725 INFO:tasks.workunit.client.1.smithi174.stdout:2/769: dwrite d5/d25/d27/d2d/d45/dae/dd6/d4f/d65/d77/dcf/fd5 [0,4194304] 0 2024-03-21T10:09:08.730 INFO:tasks.workunit.client.1.smithi174.stdout:1/925: link d4/d5/f101 d4/d7/d28/dc5/f147 0 2024-03-21T10:09:08.733 INFO:tasks.workunit.client.1.smithi174.stdout:0/813: truncate d9/d21/d23/f91 594650 0 2024-03-21T10:09:08.735 INFO:tasks.workunit.client.1.smithi174.stdout:7/734: dread d0/f88 [4194304,4194304] 0 2024-03-21T10:09:08.735 INFO:tasks.workunit.client.1.smithi174.stdout:1/926: write d4/d7/d28/f11d [1824136,61150] 0 2024-03-21T10:09:08.735 INFO:tasks.workunit.client.1.smithi174.stdout:2/770: fdatasync d5/d25/d27/d2d/d45/d5b/f23 0 2024-03-21T10:09:08.739 INFO:tasks.workunit.client.0.smithi153.stdout:8/900: dread d5/d9/d43/fa0 [0,4194304] 0 2024-03-21T10:09:08.742 INFO:tasks.workunit.client.1.smithi174.stdout:7/735: write d0/d4/f7e [1054930,57311] 0 2024-03-21T10:09:08.742 INFO:tasks.workunit.client.1.smithi174.stdout:0/814: creat d9/d21/d31/d5c/d92/d98/f119 x:0 0 0 2024-03-21T10:09:08.744 INFO:tasks.workunit.client.1.smithi174.stdout:8/803: dwrite d9/dc6/f4f [0,4194304] 0 2024-03-21T10:09:08.745 INFO:tasks.workunit.client.1.smithi174.stdout:1/927: chown d4/d7/d33/d43/d77/d87/d92/le5 159571 1 2024-03-21T10:09:08.746 INFO:tasks.workunit.client.1.smithi174.stdout:2/771: mknod d5/d25/d27/d2d/db4/dbc/c107 0 2024-03-21T10:09:08.747 INFO:tasks.workunit.client.1.smithi174.stdout:0/815: creat d9/d21/d31/d5c/dd5/f11a x:0 0 0 2024-03-21T10:09:08.750 INFO:tasks.workunit.client.1.smithi174.stdout:1/928: mknod d4/d7/d32/d7d/df1/c148 0 2024-03-21T10:09:08.751 INFO:tasks.workunit.client.0.smithi153.stdout:8/901: getdents d5/d9/d43/d7b/dc4/da7 0 2024-03-21T10:09:08.752 INFO:tasks.workunit.client.1.smithi174.stdout:8/804: truncate d9/d2a/d4d/f6d 2647338 0 2024-03-21T10:09:08.754 INFO:tasks.workunit.client.1.smithi174.stdout:0/816: mknod d9/d21/d31/d9f/d79/d82/dbb/c11b 0 2024-03-21T10:09:08.755 INFO:tasks.workunit.client.0.smithi153.stdout:8/902: readlink d5/d9/d43/dd/d20/d26/l61 0 2024-03-21T10:09:08.756 INFO:tasks.workunit.client.1.smithi174.stdout:7/736: rename d0/d4/d10/d1b/l84 to d0/lfd 0 2024-03-21T10:09:08.757 INFO:tasks.workunit.client.1.smithi174.stdout:0/817: chown d9/da2/c108 50809374 1 2024-03-21T10:09:08.758 INFO:tasks.workunit.client.1.smithi174.stdout:8/805: mknod d9/d2a/d39/c105 0 2024-03-21T10:09:08.758 INFO:tasks.workunit.client.1.smithi174.stdout:8/806: stat d9/dc6/dd1 0 2024-03-21T10:09:08.759 INFO:tasks.workunit.client.1.smithi174.stdout:1/929: getdents d4/d7/d33 0 2024-03-21T10:09:08.760 INFO:tasks.workunit.client.1.smithi174.stdout:7/737: mkdir d0/d4/d10/d60/dfe 0 2024-03-21T10:09:08.761 INFO:tasks.workunit.client.0.smithi153.stdout:8/903: rmdir d5/d9/d43/dd/d2d/d45/d88 39 2024-03-21T10:09:08.763 INFO:tasks.workunit.client.1.smithi174.stdout:1/930: mknod d4/d7/d32/da2/c149 0 2024-03-21T10:09:08.764 INFO:tasks.workunit.client.0.smithi153.stdout:8/904: creat d5/d1b/f120 x:0 0 0 2024-03-21T10:09:08.764 INFO:tasks.workunit.client.1.smithi174.stdout:7/738: mknod d0/d4/d10/d79/cff 0 2024-03-21T10:09:08.766 INFO:tasks.workunit.client.1.smithi174.stdout:0/818: rename d9/d35/d4a/d8a/c93 to d9/d35/d4a/d67/d89/c11c 0 2024-03-21T10:09:08.766 INFO:tasks.workunit.client.0.smithi153.stdout:8/905: creat d5/f121 x:0 0 0 2024-03-21T10:09:08.768 INFO:tasks.workunit.client.1.smithi174.stdout:1/931: mkdir d4/d7/d68/d7b/d14a 0 2024-03-21T10:09:08.768 INFO:tasks.workunit.client.0.smithi153.stdout:8/906: chown d5/d9/d43/d7b/ce7 27 1 2024-03-21T10:09:08.769 INFO:tasks.workunit.client.1.smithi174.stdout:8/807: rename d9/d2a/d39/d76/d19/c9e to d9/d53/dc9/de6/dea/c106 0 2024-03-21T10:09:08.770 INFO:tasks.workunit.client.0.smithi153.stdout:8/907: write d5/d9/d43/d7b/dc4/da7/f111 [286831,17649] 0 2024-03-21T10:09:08.771 INFO:tasks.workunit.client.1.smithi174.stdout:8/808: chown d9/d2a/d39/d76/d19/cbe 2003317 1 2024-03-21T10:09:08.772 INFO:tasks.workunit.client.1.smithi174.stdout:0/819: creat d9/d35/d4a/d67/d89/d99/db2/f11d x:0 0 0 2024-03-21T10:09:08.772 INFO:tasks.workunit.client.1.smithi174.stdout:7/739: creat d0/f100 x:0 0 0 2024-03-21T10:09:08.772 INFO:tasks.workunit.client.0.smithi153.stdout:8/908: write d5/d9/d43/dd/d20/d26/d8b/f4f [431605,3934] 0 2024-03-21T10:09:08.773 INFO:tasks.workunit.client.1.smithi174.stdout:1/932: fsync d4/d7/d32/f3c 0 2024-03-21T10:09:08.775 INFO:tasks.workunit.client.1.smithi174.stdout:8/809: symlink d9/d2a/d39/d58/l107 0 2024-03-21T10:09:08.776 INFO:tasks.workunit.client.1.smithi174.stdout:7/740: write d0/f100 [904130,116592] 0 2024-03-21T10:09:08.785 INFO:tasks.workunit.client.1.smithi174.stdout:8/810: creat d9/f108 x:0 0 0 2024-03-21T10:09:08.786 INFO:tasks.workunit.client.1.smithi174.stdout:1/933: rename d4/d7/d33/d43/c138 to d4/d7/d33/d43/d51/da4/d134/c14b 0 2024-03-21T10:09:08.788 INFO:tasks.workunit.client.1.smithi174.stdout:1/934: dread - d4/d2d/f130 zero size 2024-03-21T10:09:08.789 INFO:tasks.workunit.client.1.smithi174.stdout:7/741: link d0/d4/d10/d44/c5e d0/d4/d10/d1b/d86/dea/df4/c101 0 2024-03-21T10:09:08.792 INFO:tasks.workunit.client.1.smithi174.stdout:8/811: creat d9/d53/dc9/de6/f109 x:0 0 0 2024-03-21T10:09:08.795 INFO:tasks.workunit.client.1.smithi174.stdout:8/812: readlink d9/d2a/d39/d76/d19/l9b 0 2024-03-21T10:09:08.797 INFO:tasks.workunit.client.1.smithi174.stdout:7/742: unlink d0/d4/d10/d79/db2/dda/d5f/d65/fb0 0 2024-03-21T10:09:08.798 INFO:tasks.workunit.client.1.smithi174.stdout:7/743: chown d0/d4/c54 13 1 2024-03-21T10:09:08.799 INFO:tasks.workunit.client.1.smithi174.stdout:7/744: stat d0/c5d 0 2024-03-21T10:09:08.802 INFO:tasks.workunit.client.1.smithi174.stdout:8/813: mkdir d9/d53/d5b/d10a 0 2024-03-21T10:09:08.803 INFO:tasks.workunit.client.1.smithi174.stdout:2/772: truncate d5/d25/d27/d2d/d45/d5b/d2f/d66/fb9 4175039 0 2024-03-21T10:09:08.804 INFO:tasks.workunit.client.1.smithi174.stdout:7/745: symlink d0/d4/d38/da9/l102 0 2024-03-21T10:09:08.808 INFO:tasks.workunit.client.1.smithi174.stdout:7/746: mkdir d0/d4/d38/d103 0 2024-03-21T10:09:08.812 INFO:tasks.workunit.client.1.smithi174.stdout:2/773: rename d5/d25/d27/d2d/d45/d5b/d2f/d66/f102 to d5/d25/d27/d2d/d45/f108 0 2024-03-21T10:09:08.815 INFO:tasks.workunit.client.1.smithi174.stdout:2/774: mkdir d5/d25/d109 0 2024-03-21T10:09:08.817 INFO:tasks.workunit.client.1.smithi174.stdout:7/747: creat d0/d4/d10/d1b/f104 x:0 0 0 2024-03-21T10:09:08.817 INFO:tasks.workunit.client.1.smithi174.stdout:8/814: dwrite d9/f1a [4194304,4194304] 0 2024-03-21T10:09:08.820 INFO:tasks.workunit.client.1.smithi174.stdout:7/748: chown d0/d4/d10/d44/d81/d9a/ff1 14023 1 2024-03-21T10:09:08.823 INFO:tasks.workunit.client.1.smithi174.stdout:1/935: write d4/d5/d6c/fc0 [1179109,16227] 0 2024-03-21T10:09:08.824 INFO:tasks.workunit.client.1.smithi174.stdout:8/815: mknod d9/d53/d7e/c10b 0 2024-03-21T10:09:08.829 INFO:tasks.workunit.client.1.smithi174.stdout:1/936: write d4/d7/d68/f124 [875458,21225] 0 2024-03-21T10:09:08.831 INFO:tasks.workunit.client.1.smithi174.stdout:7/749: mknod d0/d4/d10/d60/da2/c105 0 2024-03-21T10:09:08.831 INFO:tasks.workunit.client.1.smithi174.stdout:0/820: dwrite d9/fc [4194304,4194304] 0 2024-03-21T10:09:08.839 INFO:tasks.workunit.client.1.smithi174.stdout:7/750: unlink d0/d4/f43 0 2024-03-21T10:09:08.839 INFO:tasks.workunit.client.1.smithi174.stdout:8/816: read d9/d2a/d39/d58/dbb/dc3/fdb [1120708,119052] 0 2024-03-21T10:09:08.843 INFO:tasks.workunit.client.1.smithi174.stdout:8/817: rename d9/d2a to d9/d2a/d39/d76/d19/d27/d60/daa/d10c 22 2024-03-21T10:09:08.845 INFO:tasks.workunit.client.1.smithi174.stdout:1/937: getdents d4/d7/d28/d59/d98/d11e 0 2024-03-21T10:09:08.845 INFO:tasks.workunit.client.1.smithi174.stdout:2/775: write d5/d25/d27/d2d/d45/dae/dd6/d4f/d65/d77/ff7 [889522,93948] 0 2024-03-21T10:09:08.847 INFO:tasks.workunit.client.1.smithi174.stdout:7/751: readlink d0/d4/d10/d1b/d86/l97 0 2024-03-21T10:09:08.848 INFO:tasks.workunit.client.1.smithi174.stdout:8/818: fsync d9/d2a/d39/d76/d19/d27/f38 0 2024-03-21T10:09:08.849 INFO:tasks.workunit.client.1.smithi174.stdout:0/821: write d9/f1f [1267814,96665] 0 2024-03-21T10:09:08.851 INFO:tasks.workunit.client.1.smithi174.stdout:7/752: creat d0/d4/d38/f106 x:0 0 0 2024-03-21T10:09:08.852 INFO:tasks.workunit.client.1.smithi174.stdout:0/822: fdatasync d9/d21/d31/f40 0 2024-03-21T10:09:08.855 INFO:tasks.workunit.client.1.smithi174.stdout:0/823: write d9/d21/d31/d5c/d92/d98/fc4 [3656644,15216] 0 2024-03-21T10:09:08.873 INFO:tasks.workunit.client.1.smithi174.stdout:0/824: dread d9/f65 [0,4194304] 0 2024-03-21T10:09:08.874 INFO:tasks.workunit.client.1.smithi174.stdout:0/825: chown d9/d35/d4a/d67/d89/d99/db2/dbd 20047454 1 2024-03-21T10:09:08.888 INFO:tasks.workunit.client.1.smithi174.stdout:6/715: sync 2024-03-21T10:09:08.888 INFO:tasks.workunit.client.1.smithi174.stdout:5/826: sync 2024-03-21T10:09:08.889 INFO:tasks.workunit.client.1.smithi174.stdout:9/829: sync 2024-03-21T10:09:08.889 INFO:tasks.workunit.client.1.smithi174.stdout:4/906: sync 2024-03-21T10:09:08.889 INFO:tasks.workunit.client.1.smithi174.stdout:8/819: rmdir d9/d2a/d39/d76/d19/d27/d60/daa/dab 39 2024-03-21T10:09:08.890 INFO:tasks.workunit.client.1.smithi174.stdout:1/938: link d4/d5/f72 d4/d7/d28/d59/f14c 0 2024-03-21T10:09:08.893 INFO:tasks.workunit.client.1.smithi174.stdout:1/939: fsync d4/d7/d15/d4f/f83 0 2024-03-21T10:09:08.895 INFO:tasks.workunit.client.1.smithi174.stdout:0/826: chown d9/d21/d23/f91 412 1 2024-03-21T10:09:08.896 INFO:tasks.workunit.client.1.smithi174.stdout:9/830: readlink d2/d11/d1d/lb0 0 2024-03-21T10:09:08.899 INFO:tasks.workunit.client.1.smithi174.stdout:5/827: rename d1/d16/d2e/d66/d8b/fb7 to d1/d49/d4f/d51/d8f/f127 0 2024-03-21T10:09:08.905 INFO:tasks.workunit.client.1.smithi174.stdout:8/820: write d9/d53/d5b/fc0 [356297,63279] 0 2024-03-21T10:09:08.909 INFO:tasks.workunit.client.1.smithi174.stdout:5/828: unlink d1/d5c/d73/de3/lf7 0 2024-03-21T10:09:08.910 INFO:tasks.workunit.client.1.smithi174.stdout:7/753: truncate d0/d4/d10/d79/db2/dda/d5f/d65/fac 2334578 0 2024-03-21T10:09:08.911 INFO:tasks.workunit.client.1.smithi174.stdout:6/716: truncate d0/d3/d20/d6d/fb2 328109 0 2024-03-21T10:09:08.913 INFO:tasks.workunit.client.1.smithi174.stdout:9/831: mknod d2/d10b/d112/c11d 0 2024-03-21T10:09:08.916 INFO:tasks.workunit.client.1.smithi174.stdout:6/717: write d0/d3/d2a/fbf [288294,8567] 0 2024-03-21T10:09:08.916 INFO:tasks.workunit.client.1.smithi174.stdout:8/821: truncate d9/d2a/d39/d76/d19/fd7 647539 0 2024-03-21T10:09:08.917 INFO:tasks.workunit.client.1.smithi174.stdout:5/829: unlink d1/d16/d1c/d81/fa6 0 2024-03-21T10:09:08.919 INFO:tasks.workunit.client.1.smithi174.stdout:4/907: dwrite d4/d21/d24/d28/f69 [0,4194304] 0 2024-03-21T10:09:08.921 INFO:tasks.workunit.client.1.smithi174.stdout:9/832: fsync d2/d11/fe3 0 2024-03-21T10:09:08.921 INFO:tasks.workunit.client.1.smithi174.stdout:0/827: getdents d9/d21/d31/d5c/d92/d98/db7 0 2024-03-21T10:09:08.921 INFO:tasks.workunit.client.1.smithi174.stdout:8/822: write d9/fd2 [896272,115511] 0 2024-03-21T10:09:08.923 INFO:tasks.workunit.client.1.smithi174.stdout:7/754: creat d0/d4/f107 x:0 0 0 2024-03-21T10:09:08.926 INFO:tasks.workunit.client.1.smithi174.stdout:1/940: dwrite d4/d2d/f12a [0,4194304] 0 2024-03-21T10:09:08.927 INFO:tasks.workunit.client.1.smithi174.stdout:6/718: creat d0/d3/d2a/d56/fe4 x:0 0 0 2024-03-21T10:09:08.931 INFO:tasks.workunit.client.1.smithi174.stdout:5/830: symlink d1/d49/d4f/d51/l128 0 2024-03-21T10:09:08.933 INFO:tasks.workunit.client.1.smithi174.stdout:6/719: truncate d0/f37 630410 0 2024-03-21T10:09:08.936 INFO:tasks.workunit.client.1.smithi174.stdout:7/755: readlink d0/d4/d10/d44/l55 0 2024-03-21T10:09:08.943 INFO:tasks.workunit.client.1.smithi174.stdout:4/908: creat d4/d21/d24/d28/de9/d3a/d7a/d9d/d11f/f12c x:0 0 0 2024-03-21T10:09:08.946 INFO:tasks.workunit.client.1.smithi174.stdout:1/941: rename d4/d7/d33/d43/d51/c9f to d4/d7/d32/d142/c14d 0 2024-03-21T10:09:08.947 INFO:tasks.workunit.client.1.smithi174.stdout:0/828: symlink d9/d35/l11e 0 2024-03-21T10:09:08.947 INFO:tasks.workunit.client.1.smithi174.stdout:1/942: chown d4/d7/d70/cf0 32217789 1 2024-03-21T10:09:08.948 INFO:tasks.workunit.client.1.smithi174.stdout:5/831: mknod d1/d16/d2e/d66/d8b/c129 0 2024-03-21T10:09:08.949 INFO:tasks.workunit.client.0.smithi153.stdout:1/843: sync 2024-03-21T10:09:08.949 INFO:tasks.workunit.client.0.smithi153.stdout:4/978: sync 2024-03-21T10:09:08.949 INFO:tasks.workunit.client.0.smithi153.stdout:8/909: sync 2024-03-21T10:09:08.949 INFO:tasks.workunit.client.0.smithi153.stdout:0/883: sync 2024-03-21T10:09:08.949 INFO:tasks.workunit.client.1.smithi174.stdout:0/829: write d9/f117 [75644,56090] 0 2024-03-21T10:09:08.953 INFO:tasks.workunit.client.1.smithi174.stdout:8/823: mknod d9/d2a/d39/d76/d19/d27/d60/daa/c10d 0 2024-03-21T10:09:08.954 INFO:tasks.workunit.client.1.smithi174.stdout:5/832: truncate d1/d5c/d96/f122 4779591 0 2024-03-21T10:09:08.956 INFO:tasks.workunit.client.0.smithi153.stdout:4/979: creat d11/d9d/dc3/f134 x:0 0 0 2024-03-21T10:09:08.956 INFO:tasks.workunit.client.1.smithi174.stdout:4/909: rmdir d4/d21/d24/d28/de9 39 2024-03-21T10:09:08.958 INFO:tasks.workunit.client.0.smithi153.stdout:4/980: chown d11/d2d/c72 224489 1 2024-03-21T10:09:08.959 INFO:tasks.workunit.client.0.smithi153.stdout:1/844: read - d6/d13/d8e/dbb/de1/de7/de9/df5/d50/d51/fd5 zero size 2024-03-21T10:09:08.960 INFO:tasks.workunit.client.1.smithi174.stdout:4/910: dread - d4/f110 zero size 2024-03-21T10:09:08.960 INFO:tasks.workunit.client.1.smithi174.stdout:6/720: dread d0/d3/d5/d6/d42/f71 [0,4194304] 0 2024-03-21T10:09:08.961 INFO:tasks.workunit.client.1.smithi174.stdout:8/824: symlink d9/d53/d5b/l10e 0 2024-03-21T10:09:08.963 INFO:tasks.workunit.client.0.smithi153.stdout:1/845: dread - d6/d10/d35/da5/d4f/d9d/db6/def/d92/fc0 zero size 2024-03-21T10:09:08.964 INFO:tasks.workunit.client.1.smithi174.stdout:9/833: getdents d2/d11/d1d/d2a 0 2024-03-21T10:09:08.964 INFO:tasks.workunit.client.1.smithi174.stdout:0/830: symlink d9/d35/d4a/d8a/d10c/l11f 0 2024-03-21T10:09:08.967 INFO:tasks.workunit.client.0.smithi153.stdout:1/846: chown d6/d10/d35/da5/f7d 12076 1 2024-03-21T10:09:08.967 INFO:tasks.workunit.client.1.smithi174.stdout:9/834: write d2/d11/d1d/d4b/fbf [3138721,47430] 0 2024-03-21T10:09:08.968 INFO:tasks.workunit.client.1.smithi174.stdout:5/833: mknod d1/d16/d1c/c12a 0 2024-03-21T10:09:08.969 INFO:tasks.workunit.client.1.smithi174.stdout:1/943: creat d4/d7/d68/d7b/f14e x:0 0 0 2024-03-21T10:09:08.971 INFO:tasks.workunit.client.1.smithi174.stdout:0/831: symlink d9/d21/df6/l120 0 2024-03-21T10:09:08.971 INFO:tasks.workunit.client.0.smithi153.stdout:1/847: write d6/d10/d35/da5/d4f/d9d/dc7/dcc/d80/f10b [391253,43486] 0 2024-03-21T10:09:08.978 INFO:tasks.workunit.client.1.smithi174.stdout:8/825: fdatasync d9/d2a/d39/d76/fa2 0 2024-03-21T10:09:08.978 INFO:tasks.workunit.client.0.smithi153.stdout:8/910: dwrite d5/d9/d43/dd/d2d/d45/d88/fd2 [0,4194304] 0 2024-03-21T10:09:08.978 INFO:tasks.workunit.client.0.smithi153.stdout:1/848: creat d6/d10/d35/da5/f113 x:0 0 0 2024-03-21T10:09:08.979 INFO:tasks.workunit.client.1.smithi174.stdout:7/756: rename d0/d4/d10/l93 to d0/l108 0 2024-03-21T10:09:08.979 INFO:tasks.workunit.client.1.smithi174.stdout:5/834: read - d1/d101/f124 zero size 2024-03-21T10:09:08.982 INFO:tasks.workunit.client.0.smithi153.stdout:1/849: chown d6/d10/d35/da5/d4f/d9d/db6/def/d92/c110 614405342 1 2024-03-21T10:09:08.987 INFO:tasks.workunit.client.0.smithi153.stdout:1/850: creat d6/d10f/f114 x:0 0 0 2024-03-21T10:09:08.987 INFO:tasks.workunit.client.0.smithi153.stdout:0/884: dread d4/db/feb [0,4194304] 0 2024-03-21T10:09:08.987 INFO:tasks.workunit.client.1.smithi174.stdout:3/797: sync 2024-03-21T10:09:08.988 INFO:tasks.workunit.client.0.smithi153.stdout:1/851: chown d6/d13/f72 295795 1 2024-03-21T10:09:08.989 INFO:tasks.workunit.client.1.smithi174.stdout:6/721: dwrite d0/d3/d5/df/d66/f96 [0,4194304] 0 2024-03-21T10:09:08.990 INFO:tasks.workunit.client.0.smithi153.stdout:0/885: rmdir d4 39 2024-03-21T10:09:08.994 INFO:tasks.workunit.client.1.smithi174.stdout:3/798: chown d0/d12/d14/db9/fe8 932 1 2024-03-21T10:09:08.998 INFO:tasks.workunit.client.1.smithi174.stdout:3/799: write d0/d12/f2d [4640935,1057] 0 2024-03-21T10:09:08.998 INFO:tasks.workunit.client.0.smithi153.stdout:1/852: creat d6/d13/d75/f115 x:0 0 0 2024-03-21T10:09:08.999 INFO:tasks.workunit.client.0.smithi153.stdout:0/886: chown d4/d16/d19/d30/d6c/d98/l10c 254211661 1 2024-03-21T10:09:09.000 INFO:tasks.workunit.client.0.smithi153.stdout:0/887: creat d4/d16/d19/d30/d6c/d98/df3/f118 x:0 0 0 2024-03-21T10:09:09.001 INFO:tasks.workunit.client.1.smithi174.stdout:9/835: dwrite d2/d11/f75 [0,4194304] 0 2024-03-21T10:09:09.004 INFO:tasks.workunit.client.0.smithi153.stdout:8/911: write d5/d9/d43/dd/d20/d26/d47/d9a/db9/d112/d52/d55/ff2 [2557478,38850] 0 2024-03-21T10:09:09.004 INFO:tasks.workunit.client.0.smithi153.stdout:4/981: dread d11/d2d/f9e [0,4194304] 0 2024-03-21T10:09:09.011 INFO:tasks.workunit.client.0.smithi153.stdout:4/982: mknod d11/d2d/d6b/d9c/c135 0 2024-03-21T10:09:09.015 INFO:tasks.workunit.client.1.smithi174.stdout:4/911: mknod d4/d21/d24/d28/de9/d3a/c12d 0 2024-03-21T10:09:09.016 INFO:tasks.workunit.client.1.smithi174.stdout:1/944: rmdir d4/d7/d68/d8e 39 2024-03-21T10:09:09.017 INFO:tasks.workunit.client.1.smithi174.stdout:4/912: fsync d4/d21/fd6 0 2024-03-21T10:09:09.017 INFO:tasks.workunit.client.0.smithi153.stdout:1/853: dwrite d6/f9 [0,4194304] 0 2024-03-21T10:09:09.018 INFO:tasks.workunit.client.1.smithi174.stdout:5/835: creat d1/d16/d2e/d66/d8b/f12b x:0 0 0 2024-03-21T10:09:09.020 INFO:tasks.workunit.client.1.smithi174.stdout:7/757: creat d0/d4/d10/d1b/d86/dea/df4/f109 x:0 0 0 2024-03-21T10:09:09.022 INFO:tasks.workunit.client.0.smithi153.stdout:8/912: rename d5/d9/d43/dd/ff5 to d5/d9/d43/dd/d2d/d45/d88/de0/f122 0 2024-03-21T10:09:09.023 INFO:tasks.workunit.client.0.smithi153.stdout:4/983: mknod d11/c136 0 2024-03-21T10:09:09.025 INFO:tasks.workunit.client.0.smithi153.stdout:0/888: dwrite d4/f103 [0,4194304] 0 2024-03-21T10:09:09.025 INFO:tasks.workunit.client.1.smithi174.stdout:6/722: write d0/d3/d5/df/d66/f84 [3296634,100269] 0 2024-03-21T10:09:09.026 INFO:tasks.workunit.client.0.smithi153.stdout:1/854: mknod d6/d32/c116 0 2024-03-21T10:09:09.028 INFO:tasks.workunit.client.1.smithi174.stdout:4/913: read d4/d21/d24/d28/de9/d3a/d7a/d9d/f104 [383912,61748] 0 2024-03-21T10:09:09.030 INFO:tasks.workunit.client.0.smithi153.stdout:0/889: fdatasync d4/d16/d27/f79 0 2024-03-21T10:09:09.030 INFO:tasks.workunit.client.0.smithi153.stdout:8/913: symlink d5/d9/d43/dd/l123 0 2024-03-21T10:09:09.030 INFO:tasks.workunit.client.1.smithi174.stdout:1/945: rename d4/d7/d15/d53/d78 to d4/d7/d33/d14f 0 2024-03-21T10:09:09.032 INFO:tasks.workunit.client.1.smithi174.stdout:9/836: symlink d2/d10b/d112/d115/l11e 0 2024-03-21T10:09:09.032 INFO:tasks.workunit.client.0.smithi153.stdout:1/855: creat d6/d13/d8e/dbb/de1/de7/de9/df5/d50/f117 x:0 0 0 2024-03-21T10:09:09.033 INFO:tasks.workunit.client.0.smithi153.stdout:0/890: fdatasync d4/f10 0 2024-03-21T10:09:09.036 INFO:tasks.workunit.client.0.smithi153.stdout:1/856: truncate d6/f9 4696053 0 2024-03-21T10:09:09.040 INFO:tasks.workunit.client.0.smithi153.stdout:8/914: creat d5/d9/d43/dd/d20/d26/d8b/d80/f124 x:0 0 0 2024-03-21T10:09:09.041 INFO:tasks.workunit.client.1.smithi174.stdout:9/837: creat d2/d10b/f11f x:0 0 0 2024-03-21T10:09:09.042 INFO:tasks.workunit.client.0.smithi153.stdout:8/915: readlink d5/d1b/lff 0 2024-03-21T10:09:09.043 INFO:tasks.workunit.client.0.smithi153.stdout:1/857: rename d6/d10f/f114 to d6/d13/d75/dfc/f118 0 2024-03-21T10:09:09.044 INFO:tasks.workunit.client.0.smithi153.stdout:8/916: truncate d5/f121 527810 0 2024-03-21T10:09:09.046 INFO:tasks.workunit.client.0.smithi153.stdout:8/917: chown d5/d9/d43/f16 8987821 1 2024-03-21T10:09:09.050 INFO:tasks.workunit.client.0.smithi153.stdout:1/858: mkdir d6/d13/d8e/dbb/de1/de7/de9/d119 0 2024-03-21T10:09:09.052 INFO:tasks.workunit.client.0.smithi153.stdout:1/859: fdatasync d6/fb 0 2024-03-21T10:09:09.054 INFO:tasks.workunit.client.1.smithi174.stdout:1/946: dwrite d4/d2d/f130 [0,4194304] 0 2024-03-21T10:09:09.054 INFO:tasks.workunit.client.1.smithi174.stdout:0/832: dread d9/d21/d31/f42 [0,4194304] 0 2024-03-21T10:09:09.058 INFO:tasks.workunit.client.1.smithi174.stdout:7/758: rename d0/d4/d38/f66 to d0/d4/d10/d1b/d86/f10a 0 2024-03-21T10:09:09.060 INFO:tasks.workunit.client.1.smithi174.stdout:1/947: chown d4/d7/d33/d43/d77/d87/f13e 830676739 1 2024-03-21T10:09:09.062 INFO:tasks.workunit.client.1.smithi174.stdout:1/948: dread - d4/d7/d28/d59/f144 zero size 2024-03-21T10:09:09.070 INFO:tasks.workunit.client.1.smithi174.stdout:1/949: fsync d4/d5/f11c 0 2024-03-21T10:09:09.071 INFO:tasks.workunit.client.0.smithi153.stdout:8/918: truncate d5/d9/fa8 3508254 0 2024-03-21T10:09:09.073 INFO:tasks.workunit.client.0.smithi153.stdout:8/919: dread - d5/d9/d43/dd/d2d/d45/fab zero size 2024-03-21T10:09:09.076 INFO:tasks.workunit.client.1.smithi174.stdout:0/833: link d9/d35/d4a/d8a/d10c/l11f d9/d35/dd2/d104/l121 0 2024-03-21T10:09:09.077 INFO:tasks.workunit.client.0.smithi153.stdout:8/920: unlink d5/d9/d43/dd/d2d/d45/d88/f6e 0 2024-03-21T10:09:09.081 INFO:tasks.workunit.client.1.smithi174.stdout:1/950: mknod d4/d7/d32/d7d/d120/c150 0 2024-03-21T10:09:09.081 INFO:tasks.workunit.client.0.smithi153.stdout:8/921: symlink d5/d9/d43/dd/d2d/d45/d88/l125 0 2024-03-21T10:09:09.084 INFO:tasks.workunit.client.0.smithi153.stdout:0/891: dread d4/d16/d19/d58/f9c [0,4194304] 0 2024-03-21T10:09:09.085 INFO:tasks.workunit.client.0.smithi153.stdout:8/922: symlink d5/d9/d43/d7b/dc4/l126 0 2024-03-21T10:09:09.085 INFO:tasks.workunit.client.1.smithi174.stdout:1/951: creat d4/d7/d32/f151 x:0 0 0 2024-03-21T10:09:09.088 INFO:tasks.workunit.client.0.smithi153.stdout:0/892: truncate d4/d16/d19/d95/fb5 2165800 0 2024-03-21T10:09:09.090 INFO:tasks.workunit.client.1.smithi174.stdout:1/952: write d4/d7/d68/d7b/f8c [87684,102200] 0 2024-03-21T10:09:09.091 INFO:tasks.workunit.client.0.smithi153.stdout:0/893: stat d4/d16/d19/d30/d6c/d98/d5e/f99 0 2024-03-21T10:09:09.094 INFO:tasks.workunit.client.1.smithi174.stdout:2/776: sync 2024-03-21T10:09:09.095 INFO:tasks.workunit.client.0.smithi153.stdout:0/894: read - d4/d4c/d83/fd5 zero size 2024-03-21T10:09:09.095 INFO:tasks.workunit.client.0.smithi153.stdout:8/923: write d5/d9/d43/dd/d20/d26/d8b/faf [999329,47674] 0 2024-03-21T10:09:09.096 INFO:tasks.workunit.client.1.smithi174.stdout:7/759: dread d0/f26 [0,4194304] 0 2024-03-21T10:09:09.097 INFO:tasks.workunit.client.0.smithi153.stdout:0/895: symlink d4/d4c/l119 0 2024-03-21T10:09:09.098 INFO:tasks.workunit.client.0.smithi153.stdout:8/924: truncate d5/d1b/fd1 1305582 0 2024-03-21T10:09:09.100 INFO:tasks.workunit.client.1.smithi174.stdout:9/838: truncate d2/d11/d1d/d4b/fbf 601867 0 2024-03-21T10:09:09.101 INFO:tasks.workunit.client.1.smithi174.stdout:1/953: truncate d4/d5/f101 3738677 0 2024-03-21T10:09:09.102 INFO:tasks.workunit.client.0.smithi153.stdout:8/925: mknod d5/d9/d43/dd/d20/d26/d47/d9a/db9/c127 0 2024-03-21T10:09:09.102 INFO:tasks.workunit.client.1.smithi174.stdout:7/760: symlink d0/d4/de3/l10b 0 2024-03-21T10:09:09.104 INFO:tasks.workunit.client.0.smithi153.stdout:8/926: symlink d5/d9/d43/dd/d20/d99/l128 0 2024-03-21T10:09:09.105 INFO:tasks.workunit.client.1.smithi174.stdout:4/914: dread d4/d21/f8c [0,4194304] 0 2024-03-21T10:09:09.107 INFO:tasks.workunit.client.1.smithi174.stdout:1/954: creat d4/d7/d32/d7d/f152 x:0 0 0 2024-03-21T10:09:09.107 INFO:tasks.workunit.client.1.smithi174.stdout:6/723: dread d0/d3/d2a/d8f/f97 [0,4194304] 0 2024-03-21T10:09:09.108 INFO:tasks.workunit.client.0.smithi153.stdout:8/927: link d5/d1b/f59 d5/d9/d43/dd/d20/d26/d47/d69/f129 0 2024-03-21T10:09:09.109 INFO:tasks.workunit.client.1.smithi174.stdout:2/777: write d5/d25/d27/d2d/d45/d5b/d55/fbf [1460311,4039] 0 2024-03-21T10:09:09.115 INFO:tasks.workunit.client.1.smithi174.stdout:5/836: dwrite d1/d2f/d37/d9b/ff0 [0,4194304] 0 2024-03-21T10:09:09.115 INFO:tasks.workunit.client.1.smithi174.stdout:1/955: chown d4/d5/d6c/fc0 40 1 2024-03-21T10:09:09.116 INFO:tasks.workunit.client.1.smithi174.stdout:3/800: dwrite d0/d12/d14/d4c/d7a/d9e/fb8 [0,4194304] 0 2024-03-21T10:09:09.122 INFO:tasks.workunit.client.0.smithi153.stdout:8/928: dwrite d5/d9/d43/dd/d20/d26/d8b/f4f [0,4194304] 0 2024-03-21T10:09:09.128 INFO:tasks.workunit.client.0.smithi153.stdout:8/929: creat d5/d9/d43/dd/d2d/d45/d88/de0/d113/f12a x:0 0 0 2024-03-21T10:09:09.132 INFO:tasks.workunit.client.1.smithi174.stdout:1/956: symlink d4/d7/d15/d4f/l153 0 2024-03-21T10:09:09.133 INFO:tasks.workunit.client.0.smithi153.stdout:8/930: read - d5/d9/d43/dd/d2d/d45/d88/de0/f101 zero size 2024-03-21T10:09:09.142 INFO:tasks.workunit.client.1.smithi174.stdout:5/837: rename d1/d49/d4f/f57 to d1/d5c/d96/f12c 0 2024-03-21T10:09:09.143 INFO:tasks.workunit.client.1.smithi174.stdout:3/801: mknod d0/d12/cfa 0 2024-03-21T10:09:09.146 INFO:tasks.workunit.client.1.smithi174.stdout:7/761: getdents d0/d4/d10/d60/da2 0 2024-03-21T10:09:09.146 INFO:tasks.workunit.client.1.smithi174.stdout:6/724: dwrite d0/d3/d2a/d56/d5e/f73 [0,4194304] 0 2024-03-21T10:09:09.146 INFO:tasks.workunit.client.1.smithi174.stdout:1/957: mknod d4/d7/d28/d59/d98/c154 0 2024-03-21T10:09:09.149 INFO:tasks.workunit.client.1.smithi174.stdout:6/725: truncate d0/d3/d5/df/f4e 1311824 0 2024-03-21T10:09:09.152 INFO:tasks.workunit.client.1.smithi174.stdout:9/839: link d2/d24/d2e/l51 d2/d11/d1d/d4b/d72/l120 0 2024-03-21T10:09:09.156 INFO:tasks.workunit.client.1.smithi174.stdout:6/726: write d0/d3/d2a/d56/fe4 [571792,101171] 0 2024-03-21T10:09:09.157 INFO:tasks.workunit.client.1.smithi174.stdout:7/762: creat d0/d4/d10/d60/da2/f10c x:0 0 0 2024-03-21T10:09:09.157 INFO:tasks.workunit.client.1.smithi174.stdout:2/778: link d5/c34 d5/d25/d27/d2d/db4/c10a 0 2024-03-21T10:09:09.159 INFO:tasks.workunit.client.1.smithi174.stdout:4/915: write d4/d21/d24/d28/de9/d6a/dd1/fc9 [1618227,92191] 0 2024-03-21T10:09:09.160 INFO:tasks.workunit.client.1.smithi174.stdout:9/840: mkdir d2/d11/d60/d121 0 2024-03-21T10:09:09.161 INFO:tasks.workunit.client.0.smithi153.stdout:8/931: truncate d5/d9/d43/dd/d20/d26/d47/d69/f86 4673184 0 2024-03-21T10:09:09.163 INFO:tasks.workunit.client.1.smithi174.stdout:7/763: stat d0/d4/d10/d79/db2/dda/d5f/d65/c7b 0 2024-03-21T10:09:09.163 INFO:tasks.workunit.client.1.smithi174.stdout:1/958: truncate d4/d7/d28/d59/f14c 1220987 0 2024-03-21T10:09:09.166 INFO:tasks.workunit.client.1.smithi174.stdout:2/779: chown d5/d25/d27/d2d/d45/d5b/f60 1237264 1 2024-03-21T10:09:09.167 INFO:tasks.workunit.client.0.smithi153.stdout:8/932: symlink d5/d9/d43/dd/d20/d26/d8b/d80/l12b 0 2024-03-21T10:09:09.168 INFO:tasks.workunit.client.1.smithi174.stdout:6/727: creat d0/d3/d2a/d56/d5e/d91/fe5 x:0 0 0 2024-03-21T10:09:09.171 INFO:tasks.workunit.client.0.smithi153.stdout:8/933: mkdir d5/d9/d43/dd/d20/d26/d47/d12c 0 2024-03-21T10:09:09.174 INFO:tasks.workunit.client.0.smithi153.stdout:8/934: mkdir d5/d9/d43/d7b/dc4/dc2/d12d 0 2024-03-21T10:09:09.175 INFO:tasks.workunit.client.1.smithi174.stdout:6/728: chown d0/d3/d5/d3b/f94 2693842 1 2024-03-21T10:09:09.176 INFO:tasks.workunit.client.1.smithi174.stdout:7/764: write d0/d4/d10/d79/db2/dda/d5f/d65/fcd [476584,32361] 0 2024-03-21T10:09:09.179 INFO:tasks.workunit.client.0.smithi153.stdout:8/935: symlink d5/d9/d43/dd/d20/d26/d47/l12e 0 2024-03-21T10:09:09.180 INFO:tasks.workunit.client.1.smithi174.stdout:4/916: creat d4/d21/d24/d28/de9/d3a/dd2/d103/f12e x:0 0 0 2024-03-21T10:09:09.180 INFO:tasks.workunit.client.1.smithi174.stdout:1/959: truncate d4/d7/d15/d4f/f83 2964220 0 2024-03-21T10:09:09.181 INFO:tasks.workunit.client.0.smithi153.stdout:8/936: chown d5/d9/d43/dd/d20/d26/d8b/d80/l12b 122417 1 2024-03-21T10:09:09.181 INFO:tasks.workunit.client.1.smithi174.stdout:6/729: write d0/d3/d5/df/fd9 [525034,38238] 0 2024-03-21T10:09:09.185 INFO:tasks.workunit.client.0.smithi153.stdout:8/937: truncate d5/d9/d43/dd/d20/d26/d8b/ff6 878755 0 2024-03-21T10:09:09.188 INFO:tasks.workunit.client.1.smithi174.stdout:1/960: chown d4/d7/d33/d43/d51/da4 5719129 1 2024-03-21T10:09:09.189 INFO:tasks.workunit.client.0.smithi153.stdout:8/938: mknod d5/d9/d43/dd/d20/d26/d8b/d80/c12f 0 2024-03-21T10:09:09.190 INFO:tasks.workunit.client.1.smithi174.stdout:7/765: rename d0/f52 to d0/d4/d10/d1b/d86/dea/f10d 0 2024-03-21T10:09:09.190 INFO:tasks.workunit.client.1.smithi174.stdout:9/841: dwrite d2/d24/d2e/f100 [0,4194304] 0 2024-03-21T10:09:09.191 INFO:tasks.workunit.client.1.smithi174.stdout:3/802: dread d0/d12/d14/d1a/d44/d65/f6b [0,4194304] 0 2024-03-21T10:09:09.191 INFO:tasks.workunit.client.1.smithi174.stdout:2/780: write d5/d25/d27/d2d/d45/d5b/d2f/f9d [185834,70381] 0 2024-03-21T10:09:09.191 INFO:tasks.workunit.client.1.smithi174.stdout:4/917: dread - d4/d21/d24/d28/de9/d3a/d7a/d9d/d11f/f12c zero size 2024-03-21T10:09:09.192 INFO:tasks.workunit.client.1.smithi174.stdout:1/961: stat d4/d7/d32/d7d/d120/c150 0 2024-03-21T10:09:09.194 INFO:tasks.workunit.client.0.smithi153.stdout:8/939: link d5/f121 d5/d9/d43/f130 0 2024-03-21T10:09:09.197 INFO:tasks.workunit.client.1.smithi174.stdout:1/962: stat d4/d7/d15/d53/f54 0 2024-03-21T10:09:09.197 INFO:tasks.workunit.client.1.smithi174.stdout:2/781: write d5/d25/f7e [1530763,20832] 0 2024-03-21T10:09:09.202 INFO:tasks.workunit.client.1.smithi174.stdout:3/803: rmdir d0/d12/d14/d1a/d44/d57 39 2024-03-21T10:09:09.205 INFO:tasks.workunit.client.1.smithi174.stdout:6/730: link d0/l5b d0/d3/d5/d21/le6 0 2024-03-21T10:09:09.207 INFO:tasks.workunit.client.1.smithi174.stdout:4/918: symlink d4/d21/d24/d28/l12f 0 2024-03-21T10:09:09.208 INFO:tasks.workunit.client.1.smithi174.stdout:1/963: mknod d4/d7/d132/c155 0 2024-03-21T10:09:09.208 INFO:tasks.workunit.client.1.smithi174.stdout:5/838: dread d1/d49/d93/d9c/fce [0,4194304] 0 2024-03-21T10:09:09.210 INFO:tasks.workunit.client.1.smithi174.stdout:8/826: sync 2024-03-21T10:09:09.211 INFO:tasks.workunit.client.0.smithi153.stdout:8/940: dwrite d5/d9/d43/dd/d2d/d45/d88/de0/d113/f12a [0,4194304] 0 2024-03-21T10:09:09.212 INFO:tasks.workunit.client.1.smithi174.stdout:7/766: getdents d0/d4/d10/d60/da2 0 2024-03-21T10:09:09.214 INFO:tasks.workunit.client.1.smithi174.stdout:3/804: chown d0/d12/d14/d1a/d27/d4f/d64/l71 35143 1 2024-03-21T10:09:09.214 INFO:tasks.workunit.client.0.smithi153.stdout:8/941: rmdir d5/d9/d43/dd/d20/d26/d47/df4 39 2024-03-21T10:09:09.216 INFO:tasks.workunit.client.1.smithi174.stdout:2/782: rename d5/d25/d27/d2d/d45/d5b/d2f/cc8 to d5/d25/d27/d2d/d45/dae/dd6/d7b/d81/da1/c10b 0 2024-03-21T10:09:09.222 INFO:tasks.workunit.client.1.smithi174.stdout:6/731: creat d0/d3/d2a/d56/d5e/d91/fe7 x:0 0 0 2024-03-21T10:09:09.225 INFO:tasks.workunit.client.1.smithi174.stdout:1/964: truncate d4/d7/d15/d4f/f141 776147 0 2024-03-21T10:09:09.227 INFO:tasks.workunit.client.1.smithi174.stdout:8/827: mkdir d9/d2a/d39/d58/dbb/d10f 0 2024-03-21T10:09:09.228 INFO:tasks.workunit.client.1.smithi174.stdout:9/842: dwrite d2/d11/d60/db2/ffa [0,4194304] 0 2024-03-21T10:09:09.231 INFO:tasks.workunit.client.1.smithi174.stdout:9/843: chown d2/d11/d1d/f33 95744423 1 2024-03-21T10:09:09.244 INFO:tasks.workunit.client.1.smithi174.stdout:3/805: rmdir d0/d12/d14/d1a/d44/d65/d7e 39 2024-03-21T10:09:09.247 INFO:tasks.workunit.client.1.smithi174.stdout:3/806: fdatasync d0/d12/d14/da9/de7/fd7 0 2024-03-21T10:09:09.249 INFO:tasks.workunit.client.1.smithi174.stdout:1/965: dread d4/d7/d68/d8e/db8/fd8 [0,4194304] 0 2024-03-21T10:09:09.250 INFO:tasks.workunit.client.1.smithi174.stdout:1/966: chown d4/d7/d32/d7d/df1 373 1 2024-03-21T10:09:09.250 INFO:tasks.workunit.client.1.smithi174.stdout:2/783: truncate d5/d25/d27/d2d/d45/dae/dd6/d85/f9a 316425 0 2024-03-21T10:09:09.257 INFO:tasks.workunit.client.1.smithi174.stdout:9/844: creat d2/d11/d1d/d10c/f122 x:0 0 0 2024-03-21T10:09:09.257 INFO:tasks.workunit.client.1.smithi174.stdout:7/767: write d0/d4/fd [2912210,103224] 0 2024-03-21T10:09:09.257 INFO:tasks.workunit.client.1.smithi174.stdout:3/807: creat d0/d12/d14/db9/ffb x:0 0 0 2024-03-21T10:09:09.257 INFO:tasks.workunit.client.1.smithi174.stdout:5/839: write d1/d16/d2e/d66/d8b/f36 [2275853,84373] 0 2024-03-21T10:09:09.259 INFO:tasks.workunit.client.1.smithi174.stdout:1/967: symlink d4/d7/d33/d14f/l156 0 2024-03-21T10:09:09.263 INFO:tasks.workunit.client.1.smithi174.stdout:9/845: write d2/d24/d2e/f109 [907958,126352] 0 2024-03-21T10:09:09.265 INFO:tasks.workunit.client.1.smithi174.stdout:9/846: fdatasync d2/d11/d1d/f79 0 2024-03-21T10:09:09.266 INFO:tasks.workunit.client.1.smithi174.stdout:8/828: rename d9/d2a/d39/d76/f5a to d9/d2a/d39/d76/d19/d27/d60/daa/f110 0 2024-03-21T10:09:09.267 INFO:tasks.workunit.client.1.smithi174.stdout:7/768: creat d0/d4/d10/d79/f10e x:0 0 0 2024-03-21T10:09:09.271 INFO:tasks.workunit.client.1.smithi174.stdout:2/784: creat d5/d25/d27/d2d/d45/dae/dd6/d90/da9/f10c x:0 0 0 2024-03-21T10:09:09.274 INFO:tasks.workunit.client.1.smithi174.stdout:4/919: dread d4/f8 [0,4194304] 0 2024-03-21T10:09:09.275 INFO:tasks.workunit.client.1.smithi174.stdout:1/968: mknod d4/d7/d33/d43/c157 0 2024-03-21T10:09:09.276 INFO:tasks.workunit.client.1.smithi174.stdout:5/840: symlink d1/d49/d4f/d51/d8f/d103/l12d 0 2024-03-21T10:09:09.277 INFO:tasks.workunit.client.1.smithi174.stdout:2/785: fsync d5/d25/d27/d2d/d45/d5b/d2f/d4b/d76/f97 0 2024-03-21T10:09:09.279 INFO:tasks.workunit.client.1.smithi174.stdout:5/841: readlink d1/d16/d2e/d66/lba 0 2024-03-21T10:09:09.280 INFO:tasks.workunit.client.1.smithi174.stdout:7/769: dread d0/d4/d10/d79/db2/dda/d5f/d65/fcd [0,4194304] 0 2024-03-21T10:09:09.282 INFO:tasks.workunit.client.1.smithi174.stdout:5/842: write d1/d49/d93/d9c/fb2 [2558000,92781] 0 2024-03-21T10:09:09.284 INFO:tasks.workunit.client.1.smithi174.stdout:3/808: rename d0/d12/d46/df9/f5a to d0/d12/d14/d4c/d7a/ffc 0 2024-03-21T10:09:09.286 INFO:tasks.workunit.client.1.smithi174.stdout:7/770: creat d0/d4/d10/f10f x:0 0 0 2024-03-21T10:09:09.290 INFO:tasks.workunit.client.1.smithi174.stdout:2/786: fdatasync d5/d25/d27/d2d/d45/d5b/d2f/d4b/fbe 0 2024-03-21T10:09:09.293 INFO:tasks.workunit.client.1.smithi174.stdout:5/843: symlink d1/d49/d93/l12e 0 2024-03-21T10:09:09.298 INFO:tasks.workunit.client.1.smithi174.stdout:7/771: creat d0/d4/d38/d103/f110 x:0 0 0 2024-03-21T10:09:09.300 INFO:tasks.workunit.client.1.smithi174.stdout:3/809: dread d0/f33 [0,4194304] 0 2024-03-21T10:09:09.300 INFO:tasks.workunit.client.1.smithi174.stdout:1/969: dwrite d4/d7/d28/d59/f144 [0,4194304] 0 2024-03-21T10:09:09.304 INFO:tasks.workunit.client.1.smithi174.stdout:3/810: dread - d0/d12/d14/d1a/d27/ff8 zero size 2024-03-21T10:09:09.306 INFO:tasks.workunit.client.1.smithi174.stdout:9/847: truncate d2/d11/d7d/f110 1513383 0 2024-03-21T10:09:09.307 INFO:tasks.workunit.client.1.smithi174.stdout:7/772: fdatasync d0/d4/d10/d1b/f6f 0 2024-03-21T10:09:09.308 INFO:tasks.workunit.client.1.smithi174.stdout:1/970: fdatasync d4/d7/d15/f135 0 2024-03-21T10:09:09.317 INFO:tasks.workunit.client.0.smithi153.stdout:1/860: sync 2024-03-21T10:09:09.317 INFO:tasks.workunit.client.0.smithi153.stdout:4/984: sync 2024-03-21T10:09:09.317 INFO:tasks.workunit.client.0.smithi153.stdout:0/896: sync 2024-03-21T10:09:09.318 INFO:tasks.workunit.client.1.smithi174.stdout:3/811: truncate d0/d12/d14/d1a/d27/d4f/d64/fd2 546094 0 2024-03-21T10:09:09.320 INFO:tasks.workunit.client.0.smithi153.stdout:1/861: fsync d6/d10/fc1 0 2024-03-21T10:09:09.320 INFO:tasks.workunit.client.1.smithi174.stdout:8/829: dread d9/d53/d7e/faf [0,4194304] 0 2024-03-21T10:09:09.323 INFO:tasks.workunit.client.0.smithi153.stdout:1/862: dread - d6/d13/f71 zero size 2024-03-21T10:09:09.324 INFO:tasks.workunit.client.1.smithi174.stdout:1/971: fdatasync d4/d7/d15/f9b 0 2024-03-21T10:09:09.324 INFO:tasks.workunit.client.1.smithi174.stdout:4/920: dwrite d4/d21/f7b [0,4194304] 0 2024-03-21T10:09:09.324 INFO:tasks.workunit.client.1.smithi174.stdout:7/773: creat d0/d4/d10/d1b/d86/dea/df4/f111 x:0 0 0 2024-03-21T10:09:09.325 INFO:tasks.workunit.client.0.smithi153.stdout:0/897: mkdir d4/d16/d19/d30/d6c/d98/d5b/d11a 0 2024-03-21T10:09:09.326 INFO:tasks.workunit.client.1.smithi174.stdout:0/834: sync 2024-03-21T10:09:09.327 INFO:tasks.workunit.client.1.smithi174.stdout:2/787: truncate d5/f49 906227 0 2024-03-21T10:09:09.328 INFO:tasks.workunit.client.1.smithi174.stdout:7/774: fdatasync d0/d4/d10/d1b/f104 0 2024-03-21T10:09:09.330 INFO:tasks.workunit.client.1.smithi174.stdout:8/830: creat d9/d53/dc9/f111 x:0 0 0 2024-03-21T10:09:09.331 INFO:tasks.workunit.client.1.smithi174.stdout:0/835: dread - d9/d1c/d76/d87/dcd/f114 zero size 2024-03-21T10:09:09.333 INFO:tasks.workunit.client.1.smithi174.stdout:7/775: fdatasync d0/d4/f42 0 2024-03-21T10:09:09.333 INFO:tasks.workunit.client.0.smithi153.stdout:1/863: creat d6/d10/d35/da5/d111/f11a x:0 0 0 2024-03-21T10:09:09.334 INFO:tasks.workunit.client.1.smithi174.stdout:0/836: readlink d9/d21/df6/l120 0 2024-03-21T10:09:09.335 INFO:tasks.workunit.client.1.smithi174.stdout:5/844: truncate d1/d49/d93/d9c/fce 8926037 0 2024-03-21T10:09:09.335 INFO:tasks.workunit.client.1.smithi174.stdout:8/831: write d9/d2a/d39/d58/dbb/fd8 [730551,121479] 0 2024-03-21T10:09:09.336 INFO:tasks.workunit.client.1.smithi174.stdout:4/921: truncate d4/d19/f23 1489487 0 2024-03-21T10:09:09.337 INFO:tasks.workunit.client.0.smithi153.stdout:0/898: link d4/d16/d19/d30/d6c/d98/d5e/l113 d4/d16/d19/dad/l11b 0 2024-03-21T10:09:09.344 INFO:tasks.workunit.client.1.smithi174.stdout:7/776: mknod d0/d4/d10/d79/db2/dda/d5f/d65/c112 0 2024-03-21T10:09:09.345 INFO:tasks.workunit.client.0.smithi153.stdout:1/864: creat d6/d3a/f11b x:0 0 0 2024-03-21T10:09:09.346 INFO:tasks.workunit.client.1.smithi174.stdout:2/788: read d5/d25/d27/f3b [4436666,127512] 0 2024-03-21T10:09:09.350 INFO:tasks.workunit.client.1.smithi174.stdout:4/922: symlink d4/d21/d24/d28/de9/d6a/d8f/d9c/l130 0 2024-03-21T10:09:09.355 INFO:tasks.workunit.client.0.smithi153.stdout:4/985: dwrite d11/d18/d7e/d93/fb8 [0,4194304] 0 2024-03-21T10:09:09.357 INFO:tasks.workunit.client.1.smithi174.stdout:9/848: dwrite d2/d11/d1d/f41 [0,4194304] 0 2024-03-21T10:09:09.357 INFO:tasks.workunit.client.1.smithi174.stdout:0/837: dwrite d9/d21/f2b [0,4194304] 0 2024-03-21T10:09:09.357 INFO:tasks.workunit.client.0.smithi153.stdout:4/986: read - d11/d9d/dc3/f134 zero size 2024-03-21T10:09:09.358 INFO:tasks.workunit.client.1.smithi174.stdout:5/845: getdents d1/d16/d2e/d8d/d99 0 2024-03-21T10:09:09.362 INFO:tasks.workunit.client.1.smithi174.stdout:5/846: readlink d1/d2f/d37/d9b/l109 0 2024-03-21T10:09:09.363 INFO:tasks.workunit.client.0.smithi153.stdout:4/987: write d11/d71/d89/dca/f132 [45229,5737] 0 2024-03-21T10:09:09.363 INFO:tasks.workunit.client.1.smithi174.stdout:9/849: readlink d2/d11/d1d/d2a/l3d 0 2024-03-21T10:09:09.363 INFO:tasks.workunit.client.1.smithi174.stdout:2/789: symlink d5/d25/d27/d2d/d45/d5b/d2f/d4b/da7/l10d 0 2024-03-21T10:09:09.364 INFO:tasks.workunit.client.1.smithi174.stdout:7/777: mknod d0/d4/d10/d1b/d86/df0/c113 0 2024-03-21T10:09:09.366 INFO:tasks.workunit.client.1.smithi174.stdout:9/850: chown d2/d11/d1d/d2a/l3d 9444797 1 2024-03-21T10:09:09.368 INFO:tasks.workunit.client.1.smithi174.stdout:0/838: unlink d9/d21/d31/d5c/d86/fbf 0 2024-03-21T10:09:09.374 INFO:tasks.workunit.client.1.smithi174.stdout:9/851: symlink d2/d11/d1d/d10c/l123 0 2024-03-21T10:09:09.379 INFO:tasks.workunit.client.1.smithi174.stdout:3/812: dread d0/d12/d14/da9/de7/fd7 [0,4194304] 0 2024-03-21T10:09:09.381 INFO:tasks.workunit.client.1.smithi174.stdout:9/852: mkdir d2/d11/d1d/dab/dc1/d124 0 2024-03-21T10:09:09.391 INFO:tasks.workunit.client.0.smithi153.stdout:8/942: sync 2024-03-21T10:09:09.399 INFO:tasks.workunit.client.0.smithi153.stdout:0/899: mkdir d4/d16/d19/d58/de9/d11c 0 2024-03-21T10:09:09.401 INFO:tasks.workunit.client.0.smithi153.stdout:4/988: creat d11/db1/f137 x:0 0 0 2024-03-21T10:09:09.402 INFO:tasks.workunit.client.0.smithi153.stdout:1/865: symlink d6/d10/d35/da5/d4f/df0/l11c 0 2024-03-21T10:09:09.406 INFO:tasks.workunit.client.0.smithi153.stdout:4/989: truncate d11/d18/f64 3134940 0 2024-03-21T10:09:09.408 INFO:tasks.workunit.client.0.smithi153.stdout:1/866: rename d6/d10/d35/ld4 to d6/d10/d35/da5/d4f/d9d/dc7/l11d 0 2024-03-21T10:09:09.409 INFO:tasks.workunit.client.0.smithi153.stdout:4/990: creat d11/d18/d6d/f138 x:0 0 0 2024-03-21T10:09:09.411 INFO:tasks.workunit.client.0.smithi153.stdout:1/867: dread - d6/d13/d8e/dbb/de1/de7/de9/df5/f9c zero size 2024-03-21T10:09:09.412 INFO:tasks.workunit.client.0.smithi153.stdout:1/868: readlink d6/lb1 0 2024-03-21T10:09:09.416 INFO:tasks.workunit.client.1.smithi174.stdout:3/813: chown d0/d12/d14/d1a/d27/d7f/cb1 495315 1 2024-03-21T10:09:09.416 INFO:tasks.workunit.client.1.smithi174.stdout:4/923: write d4/f46 [1624596,123758] 0 2024-03-21T10:09:09.421 INFO:tasks.workunit.client.1.smithi174.stdout:8/832: dwrite d9/d2a/d39/d76/fa2 [0,4194304] 0 2024-03-21T10:09:09.427 INFO:tasks.workunit.client.1.smithi174.stdout:7/778: dwrite d0/d4/d10/d79/db2/dda/d5f/d65/f6c [4194304,4194304] 0 2024-03-21T10:09:09.433 INFO:tasks.workunit.client.1.smithi174.stdout:7/779: dread - d0/d4/d10/d60/da2/f10c zero size 2024-03-21T10:09:09.437 INFO:tasks.workunit.client.1.smithi174.stdout:4/924: mknod d4/d21/d24/d28/de9/d3a/d7a/d9d/c131 0 2024-03-21T10:09:09.439 INFO:tasks.workunit.client.1.smithi174.stdout:3/814: truncate d0/d12/d46/df9/f61 2929890 0 2024-03-21T10:09:09.441 INFO:tasks.workunit.client.1.smithi174.stdout:5/847: truncate d1/d2f/d37/ff4 7253944 0 2024-03-21T10:09:09.442 INFO:tasks.workunit.client.1.smithi174.stdout:3/815: fdatasync d0/f70 0 2024-03-21T10:09:09.444 INFO:tasks.workunit.client.1.smithi174.stdout:9/853: dwrite d2/d11/d1d/d8b/f9a [0,4194304] 0 2024-03-21T10:09:09.445 INFO:tasks.workunit.client.1.smithi174.stdout:7/780: unlink d0/d4/d10/d1b/d86/dea/ffa 0 2024-03-21T10:09:09.447 INFO:tasks.workunit.client.1.smithi174.stdout:5/848: chown d1/c21 108308 1 2024-03-21T10:09:09.447 INFO:tasks.workunit.client.0.smithi153.stdout:1/869: write d6/d13/f2d [2034614,17800] 0 2024-03-21T10:09:09.449 INFO:tasks.workunit.client.1.smithi174.stdout:3/816: symlink d0/d12/d14/d98/lfd 0 2024-03-21T10:09:09.451 INFO:tasks.workunit.client.1.smithi174.stdout:8/833: dread d9/d2a/d39/d76/d19/d27/f41 [0,4194304] 0 2024-03-21T10:09:09.451 INFO:tasks.workunit.client.1.smithi174.stdout:9/854: dread - d2/d11/d60/f108 zero size 2024-03-21T10:09:09.453 INFO:tasks.workunit.client.0.smithi153.stdout:1/870: creat d6/d3a/f11e x:0 0 0 2024-03-21T10:09:09.456 INFO:tasks.workunit.client.0.smithi153.stdout:4/991: dwrite d11/d18/d50/d65/de3/f12e [4194304,4194304] 0 2024-03-21T10:09:09.456 INFO:tasks.workunit.client.1.smithi174.stdout:8/834: truncate d9/d53/d5b/fc0 642304 0 2024-03-21T10:09:09.458 INFO:tasks.workunit.client.1.smithi174.stdout:9/855: mkdir d2/d11/d1d/d4b/d72/d125 0 2024-03-21T10:09:09.460 INFO:tasks.workunit.client.0.smithi153.stdout:1/871: mkdir d6/d13/d8e/dbb/de1/de7/de9/df5/d11f 0 2024-03-21T10:09:09.463 INFO:tasks.workunit.client.1.smithi174.stdout:8/835: unlink d9/d2a/d39/d76/d19/l9b 0 2024-03-21T10:09:09.465 INFO:tasks.workunit.client.1.smithi174.stdout:9/856: symlink d2/d11/d1d/d4b/l126 0 2024-03-21T10:09:09.466 INFO:tasks.workunit.client.1.smithi174.stdout:0/839: dread d9/d21/d23/f2d [0,4194304] 0 2024-03-21T10:09:09.466 INFO:tasks.workunit.client.0.smithi153.stdout:4/992: rename d11/d18/d50/dfa/dff to d11/d9d/d139 0 2024-03-21T10:09:09.473 INFO:tasks.workunit.client.1.smithi174.stdout:4/925: dwrite d4/d21/d24/d28/de9/d6a/dab/f10b [0,4194304] 0 2024-03-21T10:09:09.476 INFO:tasks.workunit.client.1.smithi174.stdout:9/857: creat d2/d24/d2e/dac/f127 x:0 0 0 2024-03-21T10:09:09.477 INFO:tasks.workunit.client.0.smithi153.stdout:1/872: dread d6/d13/f7c [0,4194304] 0 2024-03-21T10:09:09.477 INFO:tasks.workunit.client.1.smithi174.stdout:8/836: symlink d9/d53/l112 0 2024-03-21T10:09:09.478 INFO:tasks.workunit.client.1.smithi174.stdout:7/781: write d0/d4/f33 [6253997,112064] 0 2024-03-21T10:09:09.479 INFO:tasks.workunit.client.1.smithi174.stdout:5/849: write d1/d5c/fe6 [599756,125559] 0 2024-03-21T10:09:09.484 INFO:tasks.workunit.client.1.smithi174.stdout:0/840: link d9/d21/d31/c8e d9/d35/d4a/d67/d89/d99/c122 0 2024-03-21T10:09:09.488 INFO:tasks.workunit.client.1.smithi174.stdout:9/858: creat d2/f128 x:0 0 0 2024-03-21T10:09:09.488 INFO:tasks.workunit.client.1.smithi174.stdout:5/850: write d1/d16/d2e/d66/f89 [954913,72263] 0 2024-03-21T10:09:09.489 INFO:tasks.workunit.client.1.smithi174.stdout:8/837: write d9/d2a/d39/d76/d19/d27/d60/daa/f110 [2152055,96654] 0 2024-03-21T10:09:09.490 INFO:tasks.workunit.client.1.smithi174.stdout:7/782: unlink d0/d4/d10/d1b/d86/dea/df4/f109 0 2024-03-21T10:09:09.493 INFO:tasks.workunit.client.0.smithi153.stdout:4/993: truncate d11/d18/d50/d65/de3/dec/ff9 2319235 0 2024-03-21T10:09:09.494 INFO:tasks.workunit.client.1.smithi174.stdout:6/732: sync 2024-03-21T10:09:09.497 INFO:tasks.workunit.client.0.smithi153.stdout:4/994: rename d11/d18/d7e/cf3 to d11/d9d/d139/c13a 0 2024-03-21T10:09:09.497 INFO:tasks.workunit.client.1.smithi174.stdout:3/817: dwrite d0/d12/d14/d1a/f3d [4194304,4194304] 0 2024-03-21T10:09:09.499 INFO:tasks.workunit.client.1.smithi174.stdout:6/733: dread - d0/d3/d5/d6/f70 zero size 2024-03-21T10:09:09.499 INFO:tasks.workunit.client.1.smithi174.stdout:0/841: symlink d9/d21/d31/d5c/dd5/d113/l123 0 2024-03-21T10:09:09.500 INFO:tasks.workunit.client.0.smithi153.stdout:4/995: getdents d11/d2d/da3/dac 0 2024-03-21T10:09:09.503 INFO:tasks.workunit.client.1.smithi174.stdout:7/783: truncate d0/d4/d10/d79/db2/dda/f63 394163 0 2024-03-21T10:09:09.505 INFO:tasks.workunit.client.0.smithi153.stdout:4/996: creat d11/d2d/d6b/db6/f13b x:0 0 0 2024-03-21T10:09:09.506 INFO:tasks.workunit.client.1.smithi174.stdout:0/842: chown d9/d35/d4a/d67/d89/f10f 8361230 1 2024-03-21T10:09:09.508 INFO:tasks.workunit.client.0.smithi153.stdout:4/997: creat d11/d2d/d3d/da1/f13c x:0 0 0 2024-03-21T10:09:09.511 INFO:tasks.workunit.client.0.smithi153.stdout:4/998: write d11/d2d/d6b/d9c/fe1 [887667,128161] 0 2024-03-21T10:09:09.513 INFO:tasks.workunit.client.0.smithi153.stdout:4/999: creat d11/db1/f13d x:0 0 0 2024-03-21T10:09:09.513 INFO:tasks.workunit.client.1.smithi174.stdout:0/843: creat d9/d35/d4a/d67/d89/d99/db2/f124 x:0 0 0 2024-03-21T10:09:09.515 INFO:tasks.workunit.client.1.smithi174.stdout:5/851: dwrite d1/f2c [0,4194304] 0 2024-03-21T10:09:09.518 INFO:tasks.workunit.client.1.smithi174.stdout:9/859: getdents d2 0 2024-03-21T10:09:09.519 INFO:tasks.workunit.client.1.smithi174.stdout:7/784: creat d0/d4/d10/d1b/d86/dd2/f114 x:0 0 0 2024-03-21T10:09:09.525 INFO:tasks.workunit.client.1.smithi174.stdout:8/838: write d9/d2a/d39/f75 [1485950,114162] 0 2024-03-21T10:09:09.529 INFO:tasks.workunit.client.1.smithi174.stdout:5/852: symlink d1/dca/l12f 0 2024-03-21T10:09:09.531 INFO:tasks.workunit.client.1.smithi174.stdout:9/860: link d2/d24/d2e/f109 d2/d10b/d112/f129 0 2024-03-21T10:09:09.534 INFO:tasks.workunit.client.1.smithi174.stdout:3/818: dwrite d0/d12/d14/da9/de7/db7/d7d/fef [0,4194304] 0 2024-03-21T10:09:09.588 INFO:tasks.workunit.client.1.smithi174.stdout:5/853: mkdir d1/d130 0 2024-03-21T10:09:09.588 INFO:tasks.workunit.client.1.smithi174.stdout:3/819: write d0/f91 [3712832,72230] 0 2024-03-21T10:09:09.590 INFO:tasks.workunit.client.1.smithi174.stdout:6/734: dread d0/d3/d5/df/d16/da9/db7/dc2/fc7 [0,4194304] 0 2024-03-21T10:09:09.593 INFO:tasks.workunit.client.1.smithi174.stdout:9/861: rename d2/d11/d7d/fbd to d2/d10a/f12a 0 2024-03-21T10:09:09.593 INFO:tasks.workunit.client.1.smithi174.stdout:0/844: dwrite d9/f19 [0,4194304] 0 2024-03-21T10:09:09.595 INFO:tasks.workunit.client.1.smithi174.stdout:8/839: mknod d9/d2a/d39/d76/d19/d27/d60/daa/dab/dc4/c113 0 2024-03-21T10:09:09.599 INFO:tasks.workunit.client.1.smithi174.stdout:9/862: creat d2/d11/d1d/d10c/f12b x:0 0 0 2024-03-21T10:09:09.600 INFO:tasks.workunit.client.1.smithi174.stdout:0/845: read - d9/d35/dd2/d104/ff5 zero size 2024-03-21T10:09:09.610 INFO:tasks.workunit.client.1.smithi174.stdout:6/735: dread d0/f37 [0,4194304] 0 2024-03-21T10:09:09.610 INFO:tasks.workunit.client.1.smithi174.stdout:9/863: truncate d2/d24/d2e/f67 2277660 0 2024-03-21T10:09:09.615 INFO:tasks.workunit.client.1.smithi174.stdout:9/864: symlink d2/d10a/l12c 0 2024-03-21T10:09:09.617 INFO:tasks.workunit.client.1.smithi174.stdout:5/854: dread d1/d16/d2e/d66/f7d [0,4194304] 0 2024-03-21T10:09:09.617 INFO:tasks.workunit.client.1.smithi174.stdout:6/736: mknod d0/d3/d5/df/d16/da9/db7/dc2/dda/ce8 0 2024-03-21T10:09:09.621 INFO:tasks.workunit.client.1.smithi174.stdout:6/737: chown d0/d3/d5/df/d66 0 1 2024-03-21T10:09:09.621 INFO:tasks.workunit.client.1.smithi174.stdout:9/865: symlink d2/d11/d60/d121/l12d 0 2024-03-21T10:09:09.625 INFO:tasks.workunit.client.1.smithi174.stdout:0/846: dwrite d9/d21/f2b [0,4194304] 0 2024-03-21T10:09:09.626 INFO:tasks.workunit.client.1.smithi174.stdout:8/840: dwrite d9/d2a/d39/d58/dbb/dc3/fff [0,4194304] 0 2024-03-21T10:09:09.634 INFO:tasks.workunit.client.1.smithi174.stdout:6/738: mkdir d0/d3/d2a/d56/d5e/d91/de9 0 2024-03-21T10:09:09.634 INFO:tasks.workunit.client.1.smithi174.stdout:8/841: write d9/fae [3140929,12284] 0 2024-03-21T10:09:09.636 INFO:tasks.workunit.client.1.smithi174.stdout:1/972: sync 2024-03-21T10:09:09.638 INFO:tasks.workunit.client.1.smithi174.stdout:1/973: readlink d4/d7/d28/d59/la0 0 2024-03-21T10:09:09.642 INFO:tasks.workunit.client.1.smithi174.stdout:9/866: rename d2/d11/d1d/d8b/ff3 to d2/d11/f12e 0 2024-03-21T10:09:09.643 INFO:tasks.workunit.client.1.smithi174.stdout:3/820: dread d0/d12/d14/d1a/f34 [0,4194304] 0 2024-03-21T10:09:09.647 INFO:tasks.workunit.client.1.smithi174.stdout:0/847: link d9/dd8/f10a d9/da2/f125 0 2024-03-21T10:09:09.648 INFO:tasks.workunit.client.1.smithi174.stdout:9/867: rmdir d2/d24 39 2024-03-21T10:09:09.651 INFO:tasks.workunit.client.1.smithi174.stdout:0/848: write d9/d21/d31/d5c/dd5/fd6 [3534036,65354] 0 2024-03-21T10:09:09.653 INFO:tasks.workunit.client.1.smithi174.stdout:5/855: dwrite d1/d16/d2e/d66/fbe [0,4194304] 0 2024-03-21T10:09:09.654 INFO:tasks.workunit.client.1.smithi174.stdout:1/974: link d4/d7/d68/d89/fc1 d4/d5/d6c/f158 0 2024-03-21T10:09:09.660 INFO:tasks.workunit.client.1.smithi174.stdout:6/739: write d0/d3/d5/df/d16/f8a [131201,80673] 0 2024-03-21T10:09:09.661 INFO:tasks.workunit.client.1.smithi174.stdout:0/849: creat d9/d21/d31/d5c/d92/d98/f126 x:0 0 0 2024-03-21T10:09:09.664 INFO:tasks.workunit.client.1.smithi174.stdout:1/975: creat d4/d7/d33/d43/d77/d87/f159 x:0 0 0 2024-03-21T10:09:09.668 INFO:tasks.workunit.client.1.smithi174.stdout:5/856: rename d1/d2f/d37/l43 to d1/d113/l131 0 2024-03-21T10:09:09.675 INFO:tasks.workunit.client.1.smithi174.stdout:6/740: creat d0/d3/d2a/daf/fea x:0 0 0 2024-03-21T10:09:09.678 INFO:tasks.workunit.client.1.smithi174.stdout:3/821: dread d0/d12/d14/d1a/d27/d7f/fc4 [0,4194304] 0 2024-03-21T10:09:09.679 INFO:tasks.workunit.client.1.smithi174.stdout:0/850: truncate d9/d35/dd2/d104/d72/fb1 3806329 0 2024-03-21T10:09:09.680 INFO:tasks.workunit.client.1.smithi174.stdout:6/741: write d0/d3/d20/d3d/d81/da5/fc8 [10439,99625] 0 2024-03-21T10:09:09.681 INFO:tasks.workunit.client.0.smithi153.stdout:8/943: sync 2024-03-21T10:09:09.684 INFO:tasks.workunit.client.1.smithi174.stdout:3/822: stat d0/d12/d14/d1a/d44/d94/dd9/lda 0 2024-03-21T10:09:09.686 INFO:tasks.workunit.client.1.smithi174.stdout:0/851: write d9/d1c/d76/d87/dcd/f114 [1008941,16591] 0 2024-03-21T10:09:09.688 INFO:tasks.workunit.client.1.smithi174.stdout:6/742: rename d0/d3/d20/d3d/fde to d0/feb 0 2024-03-21T10:09:09.689 INFO:tasks.workunit.client.1.smithi174.stdout:3/823: chown d0/d12/d14/da9/de7/db7/l4a 697134 1 2024-03-21T10:09:09.691 INFO:tasks.workunit.client.0.smithi153.stdout:8/944: rename d5/d9/d43/d7b/daa/d102/l68 to d5/d9/d43/d7b/dc4/da7/l131 0 2024-03-21T10:09:09.693 INFO:tasks.workunit.client.1.smithi174.stdout:9/868: truncate d2/d11/d60/db2/ffa 6013305 0 2024-03-21T10:09:09.693 INFO:tasks.workunit.client.1.smithi174.stdout:5/857: dwrite d1/d5c/d96/f105 [0,4194304] 0 2024-03-21T10:09:09.693 INFO:tasks.workunit.client.0.smithi153.stdout:8/945: stat d5/d9/d43/dd/d2d/c97 0 2024-03-21T10:09:09.694 INFO:tasks.workunit.client.1.smithi174.stdout:0/852: symlink d9/d35/dd2/d104/l127 0 2024-03-21T10:09:09.698 INFO:tasks.workunit.client.1.smithi174.stdout:5/858: chown d1/d49/d93/d9c 25903078 1 2024-03-21T10:09:09.698 INFO:tasks.workunit.client.1.smithi174.stdout:0/853: chown d9/d21/d23/da5/ca4 2913468 1 2024-03-21T10:09:09.700 INFO:tasks.workunit.client.1.smithi174.stdout:9/869: creat d2/d11/d1d/d8b/f12f x:0 0 0 2024-03-21T10:09:09.703 INFO:tasks.workunit.client.1.smithi174.stdout:0/854: write d9/d21/f26 [132515,6902] 0 2024-03-21T10:09:09.703 INFO:tasks.workunit.client.0.smithi153.stdout:8/946: rename d5/d9/d43/dd/d20/dfb to d5/d9/d43/dd/d20/d26/d47/d9a/d132 0 2024-03-21T10:09:09.705 INFO:tasks.workunit.client.1.smithi174.stdout:1/976: dwrite d4/d7/d15/f38 [0,4194304] 0 2024-03-21T10:09:09.708 INFO:tasks.workunit.client.1.smithi174.stdout:9/870: read d2/d24/d2e/f31 [3767857,90332] 0 2024-03-21T10:09:09.709 INFO:tasks.workunit.client.1.smithi174.stdout:1/977: symlink d4/d5/d6c/l15a 0 2024-03-21T10:09:09.715 INFO:tasks.workunit.client.1.smithi174.stdout:0/855: getdents d9/d35/dd2 0 2024-03-21T10:09:09.716 INFO:tasks.workunit.client.1.smithi174.stdout:9/871: dread - d2/d11/d1d/d5e/f116 zero size 2024-03-21T10:09:09.719 INFO:tasks.workunit.client.1.smithi174.stdout:7/785: sync 2024-03-21T10:09:09.719 INFO:tasks.workunit.client.1.smithi174.stdout:4/926: sync 2024-03-21T10:09:09.719 INFO:tasks.workunit.client.1.smithi174.stdout:6/743: dwrite d0/d3/d5/df/d66/f96 [0,4194304] 0 2024-03-21T10:09:09.719 INFO:tasks.workunit.client.1.smithi174.stdout:2/790: sync 2024-03-21T10:09:09.720 INFO:tasks.workunit.client.0.smithi153.stdout:8/947: dread d5/d9/d43/dd/d20/d26/d8b/d80/fa1 [0,4194304] 0 2024-03-21T10:09:09.724 INFO:tasks.workunit.client.1.smithi174.stdout:9/872: mknod d2/d24/d2e/de6/c130 0 2024-03-21T10:09:09.726 INFO:tasks.workunit.client.1.smithi174.stdout:2/791: write d5/d25/d27/d2d/d45/dae/dd6/d7b/d81/fba [1567035,80573] 0 2024-03-21T10:09:09.735 INFO:tasks.workunit.client.0.smithi153.stdout:1/873: sync 2024-03-21T10:09:09.735 INFO:tasks.workunit.client.0.smithi153.stdout:0/900: sync 2024-03-21T10:09:09.739 INFO:tasks.workunit.client.1.smithi174.stdout:8/842: sync 2024-03-21T10:09:09.739 INFO:tasks.workunit.client.1.smithi174.stdout:9/873: creat d2/d11/d60/f131 x:0 0 0 2024-03-21T10:09:09.742 INFO:tasks.workunit.client.1.smithi174.stdout:4/927: link d4/d21/c40 d4/d19/dd5/c132 0 2024-03-21T10:09:09.742 INFO:tasks.workunit.client.1.smithi174.stdout:6/744: creat d0/d3/d5/df/d16/da9/db7/fec x:0 0 0 2024-03-21T10:09:09.744 INFO:tasks.workunit.client.0.smithi153.stdout:0/901: truncate d4/d16/d19/d30/d6c/f9b 21659 0 2024-03-21T10:09:09.745 INFO:tasks.workunit.client.1.smithi174.stdout:1/978: dread d4/d7/d132/f99 [0,4194304] 0 2024-03-21T10:09:09.745 INFO:tasks.workunit.client.1.smithi174.stdout:7/786: link d0/d4/d10/d44/f72 d0/d4/d38/dec/f115 0 2024-03-21T10:09:09.747 INFO:tasks.workunit.client.1.smithi174.stdout:2/792: chown d5/d25/d27/d2d/d45/d5b/d2f/d66/fb9 40906 1 2024-03-21T10:09:09.747 INFO:tasks.workunit.client.0.smithi153.stdout:0/902: chown d4/d16/d19/d30/d6c/d98/df3/ff0 227 1 2024-03-21T10:09:09.747 INFO:tasks.workunit.client.1.smithi174.stdout:9/874: rename d2/d11/ld3 to d2/d11/d60/d121/l132 0 2024-03-21T10:09:09.748 INFO:tasks.workunit.client.1.smithi174.stdout:3/824: truncate d0/d12/d14/d1a/d44/d65/d7e/dac/fc0 2385101 0 2024-03-21T10:09:09.750 INFO:tasks.workunit.client.1.smithi174.stdout:1/979: dread - d4/d7/d132/ff9 zero size 2024-03-21T10:09:09.752 INFO:tasks.workunit.client.0.smithi153.stdout:0/903: mkdir d4/d11d 0 2024-03-21T10:09:09.753 INFO:tasks.workunit.client.1.smithi174.stdout:4/928: mknod d4/d21/d24/d28/de9/d6a/dac/c133 0 2024-03-21T10:09:09.754 INFO:tasks.workunit.client.1.smithi174.stdout:7/787: chown d0/lfc 11497 1 2024-03-21T10:09:09.755 INFO:tasks.workunit.client.1.smithi174.stdout:0/856: write d9/d1c/f7d [8504611,105909] 0 2024-03-21T10:09:09.757 INFO:tasks.workunit.client.1.smithi174.stdout:9/875: creat d2/d11/d1d/dab/dc1/f133 x:0 0 0 2024-03-21T10:09:09.760 INFO:tasks.workunit.client.0.smithi153.stdout:1/874: dwrite d6/d10/d35/da5/d4f/d9d/db6/def/f100 [0,4194304] 0 2024-03-21T10:09:09.761 INFO:tasks.workunit.client.1.smithi174.stdout:4/929: fsync d4/d21/d24/d28/f5e 0 2024-03-21T10:09:09.770 INFO:tasks.workunit.client.1.smithi174.stdout:5/859: dwrite d1/d49/d4f/d51/d8f/f104 [0,4194304] 0 2024-03-21T10:09:09.770 INFO:tasks.workunit.client.1.smithi174.stdout:9/876: unlink d2/d24/f74 0 2024-03-21T10:09:09.771 INFO:tasks.workunit.client.0.smithi153.stdout:8/948: dwrite d5/d9/d43/dd/d20/d26/d8b/d80/fa1 [0,4194304] 0 2024-03-21T10:09:09.772 INFO:tasks.workunit.client.1.smithi174.stdout:8/843: write d9/d2a/d39/d76/d19/d27/f41 [1318633,26699] 0 2024-03-21T10:09:09.773 INFO:tasks.workunit.client.0.smithi153.stdout:1/875: link d6/d10/l108 d6/d13/l120 0 2024-03-21T10:09:09.778 INFO:tasks.workunit.client.0.smithi153.stdout:0/904: dread d4/d4c/d83/d22/f47 [0,4194304] 0 2024-03-21T10:09:09.778 INFO:tasks.workunit.client.1.smithi174.stdout:8/844: fsync d9/dc6/fdc 0 2024-03-21T10:09:09.779 INFO:tasks.workunit.client.0.smithi153.stdout:1/876: creat d6/d10/d35/da5/d4f/d9d/dc7/dcc/f121 x:0 0 0 2024-03-21T10:09:09.779 INFO:tasks.workunit.client.1.smithi174.stdout:0/857: creat d9/d21/d31/d5c/d92/f128 x:0 0 0 2024-03-21T10:09:09.780 INFO:tasks.workunit.client.0.smithi153.stdout:1/877: stat d6/d3a 0 2024-03-21T10:09:09.781 INFO:tasks.workunit.client.1.smithi174.stdout:1/980: creat d4/d7/d28/d59/d98/d11e/f15b x:0 0 0 2024-03-21T10:09:09.782 INFO:tasks.workunit.client.0.smithi153.stdout:8/949: link d5/d9/d43/dd/ddd/ldf d5/d9/d43/dd/d2d/d45/d88/d85/ddc/l133 0 2024-03-21T10:09:09.787 INFO:tasks.workunit.client.0.smithi153.stdout:1/878: creat d6/d13/f122 x:0 0 0 2024-03-21T10:09:09.789 INFO:tasks.workunit.client.1.smithi174.stdout:8/845: fsync d9/d2a/d39/d76/f87 0 2024-03-21T10:09:09.791 INFO:tasks.workunit.client.0.smithi153.stdout:1/879: mkdir d6/d10/da0/d106/d123 0 2024-03-21T10:09:09.793 INFO:tasks.workunit.client.1.smithi174.stdout:9/877: creat d2/d11/d60/f134 x:0 0 0 2024-03-21T10:09:09.793 INFO:tasks.workunit.client.1.smithi174.stdout:3/825: dwrite d0/d12/d14/d1a/f1e [0,4194304] 0 2024-03-21T10:09:09.795 INFO:tasks.workunit.client.1.smithi174.stdout:1/981: dread - d4/d7/d33/d43/d77/d87/d92/f114 zero size 2024-03-21T10:09:09.799 INFO:tasks.workunit.client.1.smithi174.stdout:0/858: mkdir d9/d35/d4a/d8a/d129 0 2024-03-21T10:09:09.801 INFO:tasks.workunit.client.1.smithi174.stdout:2/793: dwrite d5/d25/d27/d2d/d45/dae/dd6/d4f/df1/fe3 [0,4194304] 0 2024-03-21T10:09:09.805 INFO:tasks.workunit.client.1.smithi174.stdout:3/826: chown d0/d12/d14/d67/da1/le2 997351 1 2024-03-21T10:09:09.808 INFO:tasks.workunit.client.1.smithi174.stdout:6/745: dwrite d0/d3/d5/df/d16/f17 [0,4194304] 0 2024-03-21T10:09:09.813 INFO:tasks.workunit.client.1.smithi174.stdout:9/878: mknod d2/d11/d1d/d4b/d72/c135 0 2024-03-21T10:09:09.814 INFO:tasks.workunit.client.1.smithi174.stdout:1/982: fsync d4/d7/feb 0 2024-03-21T10:09:09.814 INFO:tasks.workunit.client.1.smithi174.stdout:6/746: dread - d0/d3/d2a/d56/d5e/d91/fe7 zero size 2024-03-21T10:09:09.816 INFO:tasks.workunit.client.1.smithi174.stdout:6/747: dread - d0/d3/d5/fbe zero size 2024-03-21T10:09:09.819 INFO:tasks.workunit.client.1.smithi174.stdout:6/748: chown d0/d3/d5/d21 17 1 2024-03-21T10:09:09.822 INFO:tasks.workunit.client.0.smithi153.stdout:8/950: write d5/d9/d43/dd/d20/d26/d47/d9a/db9/d112/f28 [104953,128995] 0 2024-03-21T10:09:09.822 INFO:tasks.workunit.client.1.smithi174.stdout:2/794: creat d5/d25/d27/d2d/db4/dbc/f10e x:0 0 0 2024-03-21T10:09:09.824 INFO:tasks.workunit.client.0.smithi153.stdout:1/880: write d6/d32/f9a [25796,113281] 0 2024-03-21T10:09:09.824 INFO:tasks.workunit.client.1.smithi174.stdout:7/788: dread d0/d4/f33 [0,4194304] 0 2024-03-21T10:09:09.829 INFO:tasks.workunit.client.0.smithi153.stdout:1/881: creat d6/d10/d35/da5/d111/f124 x:0 0 0 2024-03-21T10:09:09.832 INFO:tasks.workunit.client.0.smithi153.stdout:1/882: dread - d6/de6/ff6 zero size 2024-03-21T10:09:09.833 INFO:tasks.workunit.client.1.smithi174.stdout:4/930: getdents d4/d21/d24/d28/de9/d3a/d67/d76 0 2024-03-21T10:09:09.833 INFO:tasks.workunit.client.1.smithi174.stdout:1/983: read d4/d7/f8f [1783106,86163] 0 2024-03-21T10:09:09.836 INFO:tasks.workunit.client.0.smithi153.stdout:0/905: dwrite d4/d16/d19/d30/f9d [0,4194304] 0 2024-03-21T10:09:09.836 INFO:tasks.workunit.client.0.smithi153.stdout:1/883: mknod d6/d10/d2c/c125 0 2024-03-21T10:09:09.840 INFO:tasks.workunit.client.0.smithi153.stdout:1/884: stat d6/d13/d8e/dbb/de1/de7/de9/df5/d50/d51 0 2024-03-21T10:09:09.840 INFO:tasks.workunit.client.1.smithi174.stdout:6/749: mkdir d0/d3/d5/df/d66/d87/ded 0 2024-03-21T10:09:09.843 INFO:tasks.workunit.client.0.smithi153.stdout:0/906: getdents d4/d16/d27/dc5/dcd/de8/dee 0 2024-03-21T10:09:09.843 INFO:tasks.workunit.client.1.smithi174.stdout:7/789: mknod d0/d4/d10/d79/c116 0 2024-03-21T10:09:09.845 INFO:tasks.workunit.client.1.smithi174.stdout:2/795: symlink d5/d25/d27/l10f 0 2024-03-21T10:09:09.845 INFO:tasks.workunit.client.1.smithi174.stdout:3/827: mknod d0/d12/d14/d1a/cfe 0 2024-03-21T10:09:09.850 INFO:tasks.workunit.client.1.smithi174.stdout:1/984: creat d4/d7/d32/d86/f15c x:0 0 0 2024-03-21T10:09:09.850 INFO:tasks.workunit.client.1.smithi174.stdout:8/846: dread d9/fd2 [0,4194304] 0 2024-03-21T10:09:09.852 INFO:tasks.workunit.client.1.smithi174.stdout:9/879: creat d2/d24/d2e/f136 x:0 0 0 2024-03-21T10:09:09.853 INFO:tasks.workunit.client.1.smithi174.stdout:4/931: fdatasync d4/d21/d24/d28/de9/d3a/d7a/d9d/fc7 0 2024-03-21T10:09:09.855 INFO:tasks.workunit.client.1.smithi174.stdout:7/790: mknod d0/d4/d10/d1b/d86/df0/c117 0 2024-03-21T10:09:09.856 INFO:tasks.workunit.client.1.smithi174.stdout:5/860: dread d1/d16/d2e/d66/f89 [0,4194304] 0 2024-03-21T10:09:09.856 INFO:tasks.workunit.client.1.smithi174.stdout:2/796: symlink d5/d25/d27/d2d/d45/dae/dd6/d90/da9/l110 0 2024-03-21T10:09:09.858 INFO:tasks.workunit.client.0.smithi153.stdout:1/885: write d6/f16 [3058318,95193] 0 2024-03-21T10:09:09.863 INFO:tasks.workunit.client.1.smithi174.stdout:4/932: write d4/d21/d24/f116 [84234,37810] 0 2024-03-21T10:09:09.864 INFO:tasks.workunit.client.1.smithi174.stdout:3/828: truncate d0/d12/d14/d4c/f7b 128675 0 2024-03-21T10:09:09.866 INFO:tasks.workunit.client.0.smithi153.stdout:0/907: dwrite d4/d16/fc1 [0,4194304] 0 2024-03-21T10:09:09.867 INFO:tasks.workunit.client.1.smithi174.stdout:6/750: dread d0/d3/d2a/d8f/f97 [0,4194304] 0 2024-03-21T10:09:09.870 INFO:tasks.workunit.client.0.smithi153.stdout:0/908: creat d4/d11d/f11e x:0 0 0 2024-03-21T10:09:09.872 INFO:tasks.workunit.client.1.smithi174.stdout:1/985: dread d4/d7/d68/f124 [0,4194304] 0 2024-03-21T10:09:09.872 INFO:tasks.workunit.client.1.smithi174.stdout:9/880: read d2/d10b/f5c [3357569,10940] 0 2024-03-21T10:09:09.874 INFO:tasks.workunit.client.1.smithi174.stdout:8/847: rename d9/d2a/d4d/db4/fb6 to d9/d2a/d39/de1/df7/f114 0 2024-03-21T10:09:09.874 INFO:tasks.workunit.client.0.smithi153.stdout:0/909: symlink d4/d16/d19/d30/d6c/de3/l11f 0 2024-03-21T10:09:09.877 INFO:tasks.workunit.client.0.smithi153.stdout:0/910: mknod d4/d16/d19/d95/c120 0 2024-03-21T10:09:09.878 INFO:tasks.workunit.client.1.smithi174.stdout:2/797: truncate d5/d25/d27/d2d/d45/d5b/d2f/d4b/f78 5594650 0 2024-03-21T10:09:09.878 INFO:tasks.workunit.client.1.smithi174.stdout:7/791: creat d0/d4/d38/dec/f118 x:0 0 0 2024-03-21T10:09:09.880 INFO:tasks.workunit.client.1.smithi174.stdout:4/933: mknod d4/d21/d24/d28/de9/d3a/dd2/d10a/c134 0 2024-03-21T10:09:09.884 INFO:tasks.workunit.client.1.smithi174.stdout:9/881: symlink d2/d11/d1d/d4b/l137 0 2024-03-21T10:09:09.885 INFO:tasks.workunit.client.1.smithi174.stdout:0/859: dwrite d9/d35/dd2/d104/d72/fb1 [0,4194304] 0 2024-03-21T10:09:09.890 INFO:tasks.workunit.client.1.smithi174.stdout:1/986: mknod d4/d7/d32/d7d/df1/c15d 0 2024-03-21T10:09:09.890 INFO:tasks.workunit.client.1.smithi174.stdout:0/860: chown d9/d1c/c4e 1659746815 1 2024-03-21T10:09:09.890 INFO:tasks.workunit.client.1.smithi174.stdout:2/798: chown c0 60216341 1 2024-03-21T10:09:09.890 INFO:tasks.workunit.client.1.smithi174.stdout:8/848: write d9/d2a/d39/d76/d19/d27/f38 [2123840,126296] 0 2024-03-21T10:09:09.896 INFO:tasks.workunit.client.0.smithi153.stdout:0/911: write d4/d16/d19/d30/d6c/d98/d5b/fa9 [3043246,7743] 0 2024-03-21T10:09:09.897 INFO:tasks.workunit.client.1.smithi174.stdout:5/861: write d1/d16/d2e/d87/fcf [295047,130868] 0 2024-03-21T10:09:09.898 INFO:tasks.workunit.client.1.smithi174.stdout:1/987: write d4/d7/d15/d53/f54 [657766,12037] 0 2024-03-21T10:09:09.898 INFO:tasks.workunit.client.1.smithi174.stdout:8/849: symlink d9/d2a/d39/d58/dbb/dc3/l115 0 2024-03-21T10:09:09.899 INFO:tasks.workunit.client.0.smithi153.stdout:0/912: dread - d4/d16/d27/dc5/f110 zero size 2024-03-21T10:09:09.902 INFO:tasks.workunit.client.1.smithi174.stdout:6/751: link d0/d3/d5/df/fd3 d0/fee 0 2024-03-21T10:09:09.906 INFO:tasks.workunit.client.0.smithi153.stdout:1/886: dread d6/d10/d35/da5/d4f/d9d/dc7/dcc/faf [0,4194304] 0 2024-03-21T10:09:09.911 INFO:tasks.workunit.client.0.smithi153.stdout:1/887: creat d6/d10/d35/da5/d4f/d9d/dc7/dcc/f126 x:0 0 0 2024-03-21T10:09:09.912 INFO:tasks.workunit.client.1.smithi174.stdout:3/829: dwrite d0/d12/d14/da9/de7/db7/f78 [0,4194304] 0 2024-03-21T10:09:09.912 INFO:tasks.workunit.client.1.smithi174.stdout:1/988: write d4/d7/d33/d43/f47 [871500,3495] 0 2024-03-21T10:09:09.912 INFO:tasks.workunit.client.0.smithi153.stdout:1/888: chown d6/d13/d8e/fda 52 1 2024-03-21T10:09:09.913 INFO:tasks.workunit.client.1.smithi174.stdout:2/799: link d5/d25/d27/d2d/d45/d5b/d2f/d4b/d76/f105 d5/d25/d27/d2d/d45/dae/dd6/d4f/d65/d77/f111 0 2024-03-21T10:09:09.916 INFO:tasks.workunit.client.0.smithi153.stdout:1/889: rename d6/d32/f5a to d6/de6/f127 0 2024-03-21T10:09:09.917 INFO:tasks.workunit.client.1.smithi174.stdout:0/861: link d9/d35/f3e d9/d21/d31/d9f/f12a 0 2024-03-21T10:09:09.919 INFO:tasks.workunit.client.1.smithi174.stdout:8/850: getdents d9/dc6/df1 0 2024-03-21T10:09:09.921 INFO:tasks.workunit.client.0.smithi153.stdout:1/890: creat d6/d13/f128 x:0 0 0 2024-03-21T10:09:09.921 INFO:tasks.workunit.client.1.smithi174.stdout:6/752: dread - d0/d3/d5/df/d16/da9/fd0 zero size 2024-03-21T10:09:09.932 INFO:tasks.workunit.client.0.smithi153.stdout:1/891: rename d6/d10/d35/da5/d4f/d9d/dc7/dcc/d80/c84 to d6/d13/d75/c129 0 2024-03-21T10:09:09.933 INFO:tasks.workunit.client.1.smithi174.stdout:2/800: unlink d5/d25/d27/d2d/d45/d5b/d2f/d66/fb2 0 2024-03-21T10:09:09.935 INFO:tasks.workunit.client.0.smithi153.stdout:1/892: symlink d6/d13/d75/l12a 0 2024-03-21T10:09:09.936 INFO:tasks.workunit.client.1.smithi174.stdout:3/830: write d0/d12/f49 [3185322,64751] 0 2024-03-21T10:09:09.937 INFO:tasks.workunit.client.0.smithi153.stdout:1/893: chown d6/f16 923 1 2024-03-21T10:09:09.937 INFO:tasks.workunit.client.0.smithi153.stdout:0/913: dread d4/db/f29 [0,4194304] 0 2024-03-21T10:09:09.939 INFO:tasks.workunit.client.0.smithi153.stdout:1/894: chown d6/d10/d35/da5/d4f/l7a 22271383 1 2024-03-21T10:09:09.941 INFO:tasks.workunit.client.0.smithi153.stdout:0/914: chown d4/d16/cb6 812888493 1 2024-03-21T10:09:09.942 INFO:tasks.workunit.client.0.smithi153.stdout:1/895: creat d6/d10/da0/f12b x:0 0 0 2024-03-21T10:09:09.942 INFO:tasks.workunit.client.1.smithi174.stdout:0/862: creat d9/d21/d31/d9f/f12b x:0 0 0 2024-03-21T10:09:09.943 INFO:tasks.workunit.client.0.smithi153.stdout:1/896: write d6/d13/f122 [882054,18321] 0 2024-03-21T10:09:09.945 INFO:tasks.workunit.client.0.smithi153.stdout:1/897: chown d6/d32/f74 406 1 2024-03-21T10:09:09.945 INFO:tasks.workunit.client.0.smithi153.stdout:0/915: mkdir d4/d16/d19/d30/d6c/d121 0 2024-03-21T10:09:09.946 INFO:tasks.workunit.client.1.smithi174.stdout:9/882: dwrite d2/d11/f4a [0,4194304] 0 2024-03-21T10:09:09.946 INFO:tasks.workunit.client.0.smithi153.stdout:1/898: chown d6/d10/da0/d106/d123 39 1 2024-03-21T10:09:09.947 INFO:tasks.workunit.client.1.smithi174.stdout:6/753: link d0/d3/d5/df/d66/d87/fe2 d0/fef 0 2024-03-21T10:09:09.948 INFO:tasks.workunit.client.0.smithi153.stdout:1/899: truncate d6/d10/fc1 601470 0 2024-03-21T10:09:09.950 INFO:tasks.workunit.client.0.smithi153.stdout:0/916: creat d4/d16/d19/d30/d101/f122 x:0 0 0 2024-03-21T10:09:09.950 INFO:tasks.workunit.client.1.smithi174.stdout:7/792: dread d0/d4/d10/d1b/f4e [0,4194304] 0 2024-03-21T10:09:09.950 INFO:tasks.workunit.client.1.smithi174.stdout:1/989: dwrite d4/d7/d15/f129 [0,4194304] 0 2024-03-21T10:09:09.951 INFO:tasks.workunit.client.1.smithi174.stdout:3/831: stat d0/d12/d14/d1a/d27/c63 0 2024-03-21T10:09:09.951 INFO:tasks.workunit.client.1.smithi174.stdout:8/851: dread d9/d2a/d4d/f50 [0,4194304] 0 2024-03-21T10:09:09.951 INFO:tasks.workunit.client.0.smithi153.stdout:1/900: write d6/d13/d8e/dbb/de1/de7/de9/f10e [46923,58069] 0 2024-03-21T10:09:09.952 INFO:tasks.workunit.client.0.smithi153.stdout:0/917: mknod d4/d4c/d83/d22/d61/c123 0 2024-03-21T10:09:09.953 INFO:tasks.workunit.client.1.smithi174.stdout:2/801: creat d5/d25/d27/d2d/d45/d5b/d2f/f112 x:0 0 0 2024-03-21T10:09:09.954 INFO:tasks.workunit.client.0.smithi153.stdout:0/918: read d4/d4c/d83/d22/f47 [1087108,16882] 0 2024-03-21T10:09:09.955 INFO:tasks.workunit.client.0.smithi153.stdout:1/901: symlink d6/d13/d8e/dbb/de1/de7/l12c 0 2024-03-21T10:09:09.957 INFO:tasks.workunit.client.1.smithi174.stdout:5/862: dread d1/d16/d2e/d66/f6c [0,4194304] 0 2024-03-21T10:09:09.961 INFO:tasks.workunit.client.1.smithi174.stdout:0/863: creat d9/d35/d4a/d67/d89/d99/db2/f12c x:0 0 0 2024-03-21T10:09:09.961 INFO:tasks.workunit.client.1.smithi174.stdout:6/754: chown d0/d3/da8/fd2 2234 1 2024-03-21T10:09:09.964 INFO:tasks.workunit.client.0.smithi153.stdout:1/902: symlink d6/d10/d35/da5/l12d 0 2024-03-21T10:09:09.965 INFO:tasks.workunit.client.1.smithi174.stdout:9/883: symlink d2/d11/d1d/d2a/df2/l138 0 2024-03-21T10:09:09.967 INFO:tasks.workunit.client.1.smithi174.stdout:3/832: unlink d0/d12/d14/d1a/d44/d65/d7e/fd5 0 2024-03-21T10:09:09.972 INFO:tasks.workunit.client.0.smithi153.stdout:1/903: creat d6/d13/d8e/dbb/de1/de7/de9/df5/d50/f12e x:0 0 0 2024-03-21T10:09:09.976 INFO:tasks.workunit.client.1.smithi174.stdout:6/755: mkdir d0/d3/d2a/d56/d5e/d91/df0 0 2024-03-21T10:09:09.976 INFO:tasks.workunit.client.1.smithi174.stdout:1/990: dread d4/d7/d15/d4f/f61 [0,4194304] 0 2024-03-21T10:09:09.977 INFO:tasks.workunit.client.0.smithi153.stdout:8/951: sync 2024-03-21T10:09:09.981 INFO:tasks.workunit.client.0.smithi153.stdout:1/904: rename d6/d13/d75/l12a to d6/d13/l12f 0 2024-03-21T10:09:09.982 INFO:tasks.workunit.client.0.smithi153.stdout:8/952: getdents d5/d9/dfe 0 2024-03-21T10:09:09.985 INFO:tasks.workunit.client.1.smithi174.stdout:8/852: symlink d9/d2a/d4d/l116 0 2024-03-21T10:09:09.986 INFO:tasks.workunit.client.1.smithi174.stdout:7/793: write d0/d4/d10/f62 [1455469,39816] 0 2024-03-21T10:09:09.987 INFO:tasks.workunit.client.0.smithi153.stdout:1/905: fsync d6/d13/d8e/dbb/de1/de7/de9/df5/d50/f67 0 2024-03-21T10:09:09.988 INFO:tasks.workunit.client.1.smithi174.stdout:5/863: write d1/d2f/d37/fe7 [781734,28944] 0 2024-03-21T10:09:09.989 INFO:tasks.workunit.client.1.smithi174.stdout:1/991: stat d4/d7/d32/d7d/c94 0 2024-03-21T10:09:09.989 INFO:tasks.workunit.client.0.smithi153.stdout:0/919: dwrite d4/d16/d19/d30/d6c/d98/d5e/dbe/fdf [0,4194304] 0 2024-03-21T10:09:09.990 INFO:tasks.workunit.client.1.smithi174.stdout:2/802: symlink d5/d25/d27/d2d/d45/dae/dd6/d4f/d65/d77/d106/l113 0 2024-03-21T10:09:09.990 INFO:tasks.workunit.client.0.smithi153.stdout:1/906: chown d6/d10/d2c/c44 55 1 2024-03-21T10:09:09.992 INFO:tasks.workunit.client.0.smithi153.stdout:1/907: read - d6/d13/d8e/fec zero size 2024-03-21T10:09:09.993 INFO:tasks.workunit.client.1.smithi174.stdout:7/794: stat d0/d4/d10/d1b/d86/fc2 0 2024-03-21T10:09:09.993 INFO:tasks.workunit.client.0.smithi153.stdout:8/953: truncate d5/d9/d43/dd/d20/d26/d47/d9a/db9/d112/d52/d55/f71 5051743 0 2024-03-21T10:09:09.995 INFO:tasks.workunit.client.1.smithi174.stdout:0/864: dwrite d9/fc [4194304,4194304] 0 2024-03-21T10:09:09.996 INFO:tasks.workunit.client.1.smithi174.stdout:0/865: fdatasync d9/d21/d31/fc6 0 2024-03-21T10:09:09.999 INFO:tasks.workunit.client.0.smithi153.stdout:8/954: mknod d5/d9/d43/dd/d2d/d45/d88/de0/c134 0 2024-03-21T10:09:10.000 INFO:tasks.workunit.client.0.smithi153.stdout:1/908: truncate d6/d13/d8e/dbb/de1/de7/de9/df5/d50/d51/fa6 365342 0 2024-03-21T10:09:10.001 INFO:tasks.workunit.client.0.smithi153.stdout:8/955: dread - d5/d9/d43/dd/d20/d26/d8b/ffd zero size 2024-03-21T10:09:10.005 INFO:tasks.workunit.client.0.smithi153.stdout:1/909: mkdir d6/d10/d5c/d103/d130 0 2024-03-21T10:09:10.006 INFO:tasks.workunit.client.0.smithi153.stdout:0/920: write d4/d16/d19/d30/d6c/d98/df3/ff0 [744899,99059] 0 2024-03-21T10:09:10.008 INFO:tasks.workunit.client.1.smithi174.stdout:6/756: creat d0/d3/d20/d6d/ff1 x:0 0 0 2024-03-21T10:09:10.008 INFO:tasks.workunit.client.0.smithi153.stdout:8/956: unlink d5/d9/d43/dd/d2d/d45/d88/d85/fe2 0 2024-03-21T10:09:10.009 INFO:tasks.workunit.client.1.smithi174.stdout:3/833: mkdir d0/d12/d14/d1a/d27/dff 0 2024-03-21T10:09:10.010 INFO:tasks.workunit.client.1.smithi174.stdout:8/853: mkdir d9/d53/dc9/de6/dea/d117 0 2024-03-21T10:09:10.010 INFO:tasks.workunit.client.0.smithi153.stdout:1/910: chown d6/d10/d35/da5/d4f/d9d/db6/def/c7b 1633994190 1 2024-03-21T10:09:10.012 INFO:tasks.workunit.client.1.smithi174.stdout:1/992: fdatasync d4/d2d/fb9 0 2024-03-21T10:09:10.012 INFO:tasks.workunit.client.0.smithi153.stdout:0/921: creat d4/d4c/d83/d22/d61/d77/f124 x:0 0 0 2024-03-21T10:09:10.013 INFO:tasks.workunit.client.1.smithi174.stdout:3/834: chown d0/d12/d14/d1a/d44/d65/d7e 722628 1 2024-03-21T10:09:10.013 INFO:tasks.workunit.client.1.smithi174.stdout:5/864: symlink d1/d16/d2e/d8d/l132 0 2024-03-21T10:09:10.015 INFO:tasks.workunit.client.0.smithi153.stdout:1/911: creat d6/d10/d35/da5/d111/f131 x:0 0 0 2024-03-21T10:09:10.015 INFO:tasks.workunit.client.1.smithi174.stdout:1/993: chown d4/d7/d70/fe3 4 1 2024-03-21T10:09:10.016 INFO:tasks.workunit.client.1.smithi174.stdout:2/803: creat d5/d25/d27/d2d/d45/d5b/d63/f114 x:0 0 0 2024-03-21T10:09:10.017 INFO:tasks.workunit.client.1.smithi174.stdout:0/866: dwrite d9/d35/d4a/f5e [0,4194304] 0 2024-03-21T10:09:10.018 INFO:tasks.workunit.client.0.smithi153.stdout:0/922: mknod d4/d16/d19/d30/d6c/d98/d5e/dbe/c125 0 2024-03-21T10:09:10.022 INFO:tasks.workunit.client.0.smithi153.stdout:1/912: creat d6/d10/d35/da5/d4f/df0/f132 x:0 0 0 2024-03-21T10:09:10.022 INFO:tasks.workunit.client.0.smithi153.stdout:8/957: dread d5/d9/f19 [0,4194304] 0 2024-03-21T10:09:10.023 INFO:tasks.workunit.client.1.smithi174.stdout:0/867: write d9/d21/d31/d5c/d92/f100 [160689,97] 0 2024-03-21T10:09:10.025 INFO:tasks.workunit.client.0.smithi153.stdout:8/958: dread - d5/d9/d43/dd/d2d/d45/f4a zero size 2024-03-21T10:09:10.028 INFO:tasks.workunit.client.0.smithi153.stdout:8/959: readlink d5/d9/d43/d7b/dc4/l126 0 2024-03-21T10:09:10.028 INFO:tasks.workunit.client.0.smithi153.stdout:1/913: dread d6/d13/d8e/dbb/de1/de7/de9/df5/d50/d51/f8f [0,4194304] 0 2024-03-21T10:09:10.028 INFO:tasks.workunit.client.0.smithi153.stdout:0/923: dread d4/f6 [4194304,4194304] 0 2024-03-21T10:09:10.029 INFO:tasks.workunit.client.1.smithi174.stdout:7/795: mknod d0/d4/d10/d79/db2/dda/d5f/daa/c119 0 2024-03-21T10:09:10.029 INFO:tasks.workunit.client.0.smithi153.stdout:8/960: stat d5/d9/d43/dd/ddd/fc7 0 2024-03-21T10:09:10.030 INFO:tasks.workunit.client.1.smithi174.stdout:1/994: rename d4/d7/d28/d59/f144 to d4/d7/d33/d43/d77/d122/f15e 0 2024-03-21T10:09:10.032 INFO:tasks.workunit.client.1.smithi174.stdout:2/804: mknod d5/d25/d27/d2d/d45/d5b/d2f/d4b/d76/c115 0 2024-03-21T10:09:10.033 INFO:tasks.workunit.client.1.smithi174.stdout:0/868: write d9/d35/d4a/d67/d89/f10f [265490,89072] 0 2024-03-21T10:09:10.033 INFO:tasks.workunit.client.0.smithi153.stdout:0/924: chown d4/d16/d19/d30/le7 1600 1 2024-03-21T10:09:10.033 INFO:tasks.workunit.client.0.smithi153.stdout:8/961: chown d5/d9/d43/dd/d20/df3/f115 3 1 2024-03-21T10:09:10.037 INFO:tasks.workunit.client.1.smithi174.stdout:4/934: sync 2024-03-21T10:09:10.040 INFO:tasks.workunit.client.0.smithi153.stdout:0/925: dread d4/d16/d27/f76 [4194304,4194304] 0 2024-03-21T10:09:10.043 INFO:tasks.workunit.client.1.smithi174.stdout:4/935: read d4/d21/d24/ffc [936115,68461] 0 2024-03-21T10:09:10.044 INFO:tasks.workunit.client.1.smithi174.stdout:3/835: dwrite d0/f5f [0,4194304] 0 2024-03-21T10:09:10.050 INFO:tasks.workunit.client.1.smithi174.stdout:0/869: rename d9/d35/dd2/fbe to d9/d21/d23/f12d 0 2024-03-21T10:09:10.056 INFO:tasks.workunit.client.1.smithi174.stdout:4/936: symlink d4/d21/l135 0 2024-03-21T10:09:10.059 INFO:tasks.workunit.client.0.smithi153.stdout:8/962: write d5/fe1 [350540,47217] 0 2024-03-21T10:09:10.061 INFO:tasks.workunit.client.0.smithi153.stdout:0/926: write d4/d4c/d83/d22/d61/fbd [280372,101987] 0 2024-03-21T10:09:10.064 INFO:tasks.workunit.client.1.smithi174.stdout:0/870: read d9/d21/d31/d9f/f12a [8715590,77818] 0 2024-03-21T10:09:10.065 INFO:tasks.workunit.client.1.smithi174.stdout:9/884: truncate d2/d24/d2e/f31 935297 0 2024-03-21T10:09:10.065 INFO:tasks.workunit.client.1.smithi174.stdout:4/937: chown d4/d19/lb4 3683 1 2024-03-21T10:09:10.066 INFO:tasks.workunit.client.1.smithi174.stdout:8/854: dread d9/f30 [0,4194304] 0 2024-03-21T10:09:10.066 INFO:tasks.workunit.client.1.smithi174.stdout:0/871: fdatasync d9/f1f 0 2024-03-21T10:09:10.068 INFO:tasks.workunit.client.1.smithi174.stdout:7/796: dwrite d0/d4/d10/d79/db2/dda/f80 [4194304,4194304] 0 2024-03-21T10:09:10.068 INFO:tasks.workunit.client.0.smithi153.stdout:0/927: rename d4/d16/d19/dad/d100/f107 to d4/d16/d19/d30/d6c/de3/f126 0 2024-03-21T10:09:10.071 INFO:tasks.workunit.client.1.smithi174.stdout:7/797: chown d0/d4/d10/d79/db2/dda/d5f/d65/f6c 53910250 1 2024-03-21T10:09:10.074 INFO:tasks.workunit.client.1.smithi174.stdout:8/855: rmdir d9/d53/d7e 39 2024-03-21T10:09:10.078 INFO:tasks.workunit.client.1.smithi174.stdout:0/872: mkdir d9/d21/d23/da5/ddf/d12e 0 2024-03-21T10:09:10.080 INFO:tasks.workunit.client.0.smithi153.stdout:8/963: write d5/d9/d43/d7b/fa2 [4272646,115265] 0 2024-03-21T10:09:10.081 INFO:tasks.workunit.client.0.smithi153.stdout:1/914: dread d6/d3a/fb0 [0,4194304] 0 2024-03-21T10:09:10.084 INFO:tasks.workunit.client.1.smithi174.stdout:4/938: mknod d4/d21/d24/d28/de9/d3a/d77/dcf/c136 0 2024-03-21T10:09:10.089 INFO:tasks.workunit.client.0.smithi153.stdout:8/964: dread - d5/d9/d43/dd/d20/d26/d47/d9a/f11c zero size 2024-03-21T10:09:10.089 INFO:tasks.workunit.client.1.smithi174.stdout:4/939: mkdir d4/d21/d24/d28/de9/d12a/d137 0 2024-03-21T10:09:10.089 INFO:tasks.workunit.client.0.smithi153.stdout:1/915: truncate d6/d10/d35/da5/d4f/d9d/db6/def/f100 5151145 0 2024-03-21T10:09:10.090 INFO:tasks.workunit.client.1.smithi174.stdout:5/865: write d1/d2f/f111 [1103520,120987] 0 2024-03-21T10:09:10.090 INFO:tasks.workunit.client.1.smithi174.stdout:6/757: truncate d0/d3/d20/d3d/d81/da5/fc9 1451376 0 2024-03-21T10:09:10.093 INFO:tasks.workunit.client.0.smithi153.stdout:0/928: dwrite d4/d16/d19/d30/d6c/d98/d5e/f78 [0,4194304] 0 2024-03-21T10:09:10.099 INFO:tasks.workunit.client.1.smithi174.stdout:3/836: write d0/d12/d14/f97 [1787436,25966] 0 2024-03-21T10:09:10.100 INFO:tasks.workunit.client.0.smithi153.stdout:0/929: truncate d4/d16/d19/d58/f9c 4320160 0 2024-03-21T10:09:10.100 INFO:tasks.workunit.client.1.smithi174.stdout:0/873: rename d9/d1c/d76/d87/dcd/f114 to d9/d21/d81/f12f 0 2024-03-21T10:09:10.101 INFO:tasks.workunit.client.1.smithi174.stdout:4/940: chown d4/d21/d24/d28/de9/d6a/dd1/fc9 76 1 2024-03-21T10:09:10.102 INFO:tasks.workunit.client.0.smithi153.stdout:8/965: rename d5/d9/d43/dd/d20/d26/d8b/d80/fa1 to d5/d9/d43/dd/d2d/d45/d88/d85/f135 0 2024-03-21T10:09:10.103 INFO:tasks.workunit.client.0.smithi153.stdout:0/930: chown d4/d4c/d83/d22/c52 1077663 1 2024-03-21T10:09:10.105 INFO:tasks.workunit.client.0.smithi153.stdout:1/916: write d6/d10/d35/da5/d4f/d9d/db6/def/d92/fa4 [991692,53363] 0 2024-03-21T10:09:10.106 INFO:tasks.workunit.client.0.smithi153.stdout:8/966: stat d5/d9/d43/dd/d20/d26/d7a/l95 0 2024-03-21T10:09:10.108 INFO:tasks.workunit.client.0.smithi153.stdout:0/931: dread - d4/d4c/d83/d22/d61/d77/fe6 zero size 2024-03-21T10:09:10.109 INFO:tasks.workunit.client.1.smithi174.stdout:7/798: dwrite d0/d4/d10/d79/db2/dda/d5f/f71 [4194304,4194304] 0 2024-03-21T10:09:10.111 INFO:tasks.workunit.client.0.smithi153.stdout:8/967: mknod d5/d9/d43/dd/d2d/d45/c136 0 2024-03-21T10:09:10.114 INFO:tasks.workunit.client.1.smithi174.stdout:8/856: truncate d9/f30 694756 0 2024-03-21T10:09:10.119 INFO:tasks.workunit.client.0.smithi153.stdout:0/932: dread d4/d16/d19/d30/d6c/d98/d5e/db2/fd9 [0,4194304] 0 2024-03-21T10:09:10.123 INFO:tasks.workunit.client.0.smithi153.stdout:0/933: symlink d4/d16/d27/dc5/dcd/de8/l127 0 2024-03-21T10:09:10.124 INFO:tasks.workunit.client.1.smithi174.stdout:0/874: mkdir d9/d21/d31/d5c/d92/d98/d130 0 2024-03-21T10:09:10.126 INFO:tasks.workunit.client.0.smithi153.stdout:0/934: fsync d4/d16/d19/dad/fca 0 2024-03-21T10:09:10.128 INFO:tasks.workunit.client.1.smithi174.stdout:2/805: dwrite d5/d25/d27/d2d/d45/dae/dd6/d85/dad/fcd [0,4194304] 0 2024-03-21T10:09:10.129 INFO:tasks.workunit.client.0.smithi153.stdout:0/935: rmdir d4/d16/d19 39 2024-03-21T10:09:10.132 INFO:tasks.workunit.client.1.smithi174.stdout:5/866: write d1/d16/f1a [1007637,85068] 0 2024-03-21T10:09:10.135 INFO:tasks.workunit.client.1.smithi174.stdout:1/995: dwrite d4/d7/f21 [0,4194304] 0 2024-03-21T10:09:10.137 INFO:tasks.workunit.client.1.smithi174.stdout:9/885: dwrite d2/f8e [0,4194304] 0 2024-03-21T10:09:10.137 INFO:tasks.workunit.client.0.smithi153.stdout:0/936: symlink d4/d4c/d83/d22/d61/d77/df1/l128 0 2024-03-21T10:09:10.138 INFO:tasks.workunit.client.1.smithi174.stdout:7/799: creat d0/d4/d38/f11a x:0 0 0 2024-03-21T10:09:10.141 INFO:tasks.workunit.client.1.smithi174.stdout:5/867: dread - d1/d49/f53 zero size 2024-03-21T10:09:10.142 INFO:tasks.workunit.client.0.smithi153.stdout:0/937: creat d4/d16/d19/d30/d6c/d98/d5b/d11a/f129 x:0 0 0 2024-03-21T10:09:10.144 INFO:tasks.workunit.client.1.smithi174.stdout:3/837: unlink d0/d12/d14/d1a/d27/f66 0 2024-03-21T10:09:10.145 INFO:tasks.workunit.client.1.smithi174.stdout:1/996: truncate d4/d2d/f12a 4574126 0 2024-03-21T10:09:10.145 INFO:tasks.workunit.client.1.smithi174.stdout:7/800: dread - d0/d4/d38/d103/f110 zero size 2024-03-21T10:09:10.145 INFO:tasks.workunit.client.0.smithi153.stdout:0/938: mknod d4/d16/d19/d30/c12a 0 2024-03-21T10:09:10.146 INFO:tasks.workunit.client.1.smithi174.stdout:9/886: chown d2/d11/d1d/d2a/da7/ld6 10 1 2024-03-21T10:09:10.150 INFO:tasks.workunit.client.0.smithi153.stdout:8/968: write d5/d9/d43/dd/d2d/d45/d88/f4c [858122,85125] 0 2024-03-21T10:09:10.153 INFO:tasks.workunit.client.1.smithi174.stdout:0/875: creat d9/d21/d31/d9f/d79/d82/f131 x:0 0 0 2024-03-21T10:09:10.153 INFO:tasks.workunit.client.1.smithi174.stdout:1/997: write d4/d7/d70/fe3 [4116972,7853] 0 2024-03-21T10:09:10.155 INFO:tasks.workunit.client.1.smithi174.stdout:8/857: creat d9/d53/dc9/de6/dea/d117/f118 x:0 0 0 2024-03-21T10:09:10.155 INFO:tasks.workunit.client.0.smithi153.stdout:8/969: creat d5/d9/d43/d7b/daa/dca/f137 x:0 0 0 2024-03-21T10:09:10.158 INFO:tasks.workunit.client.0.smithi153.stdout:1/917: dwrite d6/f9b [0,4194304] 0 2024-03-21T10:09:10.159 INFO:tasks.workunit.client.1.smithi174.stdout:1/998: dread - d4/d7/d70/fe8 zero size 2024-03-21T10:09:10.159 INFO:tasks.workunit.client.1.smithi174.stdout:4/941: dwrite d4/d21/d24/ff1 [0,4194304] 0 2024-03-21T10:09:10.162 INFO:tasks.workunit.client.1.smithi174.stdout:5/868: creat d1/d49/d4f/d51/d8f/d103/f133 x:0 0 0 2024-03-21T10:09:10.163 INFO:tasks.workunit.client.0.smithi153.stdout:8/970: symlink d5/d9/d43/d7b/dc4/dc2/d12d/l138 0 2024-03-21T10:09:10.163 INFO:tasks.workunit.client.1.smithi174.stdout:8/858: dread - d9/d2a/d39/d58/dbb/dc3/fe7 zero size 2024-03-21T10:09:10.163 INFO:tasks.workunit.client.1.smithi174.stdout:0/876: mkdir d9/d1c/d132 0 2024-03-21T10:09:10.164 INFO:tasks.workunit.client.0.smithi153.stdout:1/918: symlink d6/d10/d35/da5/d4f/d9d/db6/def/l133 0 2024-03-21T10:09:10.165 INFO:tasks.workunit.client.0.smithi153.stdout:0/939: dwrite d4/d16/fc1 [0,4194304] 0 2024-03-21T10:09:10.167 INFO:tasks.workunit.client.1.smithi174.stdout:8/859: fsync d9/d53/d5b/fc0 0 2024-03-21T10:09:10.169 INFO:tasks.workunit.client.0.smithi153.stdout:8/971: fdatasync d5/d9/d43/dd/d20/d26/f23 0 2024-03-21T10:09:10.175 INFO:tasks.workunit.client.1.smithi174.stdout:7/801: rename d0/d4/d10/d60 to d0/d11b 0 2024-03-21T10:09:10.175 INFO:tasks.workunit.client.0.smithi153.stdout:0/940: truncate d4/d16/d19/d30/f104 297936 0 2024-03-21T10:09:10.176 INFO:tasks.workunit.client.0.smithi153.stdout:1/919: truncate d6/d10/d35/da5/d4f/d9d/db6/def/fde 678169 0 2024-03-21T10:09:10.181 INFO:tasks.workunit.client.1.smithi174.stdout:6/758: dread d0/d3/d5/d6/f2d [0,4194304] 0 2024-03-21T10:09:10.183 INFO:tasks.workunit.client.0.smithi153.stdout:0/941: mkdir d4/d16/d19/d30/d6c/d98/d12b 0 2024-03-21T10:09:10.183 INFO:tasks.workunit.client.1.smithi174.stdout:6/759: chown d0/d3/d5/d6/d42/f71 884349 1 2024-03-21T10:09:10.186 INFO:tasks.workunit.client.0.smithi153.stdout:1/920: mknod d6/d13/d8e/dbb/de1/de7/de9/c134 0 2024-03-21T10:09:10.188 INFO:tasks.workunit.client.1.smithi174.stdout:6/760: write d0/d3/d5/f1c [17091,44123] 0 2024-03-21T10:09:10.189 INFO:tasks.workunit.client.0.smithi153.stdout:8/972: link d5/d9/fa8 d5/d9/d43/dd/d2d/d45/d88/de0/f139 0 2024-03-21T10:09:10.193 INFO:tasks.workunit.client.1.smithi174.stdout:4/942: rmdir d4/d21/d24/d28/de9/d3a/d7a/d9d/d11f 39 2024-03-21T10:09:10.194 INFO:tasks.workunit.client.0.smithi153.stdout:8/973: unlink d5/d9/f9b 0 2024-03-21T10:09:10.197 INFO:tasks.workunit.client.0.smithi153.stdout:1/921: rename d6/d10/d35/l3d to d6/d10/da0/d106/d123/l135 0 2024-03-21T10:09:10.199 INFO:tasks.workunit.client.1.smithi174.stdout:9/887: getdents d2/d10b 0 2024-03-21T10:09:10.200 INFO:tasks.workunit.client.0.smithi153.stdout:8/974: fsync d5/d9/d43/dd/d20/d26/f23 0 2024-03-21T10:09:10.200 INFO:tasks.workunit.client.1.smithi174.stdout:0/877: mkdir d9/d21/d31/d5c/d92/d98/d133 0 2024-03-21T10:09:10.202 INFO:tasks.workunit.client.1.smithi174.stdout:7/802: mkdir d0/d4/d10/d1b/d86/df0/d11c 0 2024-03-21T10:09:10.202 INFO:tasks.workunit.client.1.smithi174.stdout:6/761: dread d0/d3/d20/d3d/d81/f9f [0,4194304] 0 2024-03-21T10:09:10.203 INFO:tasks.workunit.client.1.smithi174.stdout:5/869: creat d1/d2f/d37/df5/d11e/f134 x:0 0 0 2024-03-21T10:09:10.205 INFO:tasks.workunit.client.0.smithi153.stdout:1/922: truncate d6/d10/d35/da5/d4f/df0/f132 613196 0 2024-03-21T10:09:10.207 INFO:tasks.workunit.client.0.smithi153.stdout:8/975: chown d5/d9/d43/dd/d20/d26/d8b/d80/f124 0 1 2024-03-21T10:09:10.210 INFO:tasks.workunit.client.1.smithi174.stdout:0/878: creat d9/d35/d4a/d67/d89/dfd/f134 x:0 0 0 2024-03-21T10:09:10.214 INFO:tasks.workunit.client.0.smithi153.stdout:8/976: creat d5/d9/d43/dd/d20/d11f/f13a x:0 0 0 2024-03-21T10:09:10.215 INFO:tasks.workunit.client.1.smithi174.stdout:6/762: readlink d0/d3/d5/df/d16/da9/ld7 0 2024-03-21T10:09:10.216 INFO:tasks.workunit.client.0.smithi153.stdout:0/942: dwrite d4/d16/d27/f76 [4194304,4194304] 0 2024-03-21T10:09:10.219 INFO:tasks.workunit.client.0.smithi153.stdout:1/923: rename d6/d13/d8e/dbb/de1/de7/de9/df5/d50/da8/fed to d6/d13/d75/dfc/f136 0 2024-03-21T10:09:10.220 INFO:tasks.workunit.client.1.smithi174.stdout:0/879: fsync d9/fc1 0 2024-03-21T10:09:10.221 INFO:tasks.workunit.client.0.smithi153.stdout:8/977: creat d5/d9/d43/dd/d20/d26/d47/d9a/d132/f13b x:0 0 0 2024-03-21T10:09:10.226 INFO:tasks.workunit.client.1.smithi174.stdout:6/763: rmdir d0/d3/d5/df/d16/da9/db7/dc2/dda 39 2024-03-21T10:09:10.227 INFO:tasks.workunit.client.1.smithi174.stdout:0/880: chown d9/d21/d31/d5c/d92/d98/db7/ddc 3820 1 2024-03-21T10:09:10.227 INFO:tasks.workunit.client.0.smithi153.stdout:1/924: stat d6/d13/d8e/dbb/de1/de7 0 2024-03-21T10:09:10.227 INFO:tasks.workunit.client.0.smithi153.stdout:8/978: fsync d5/d9/d43/dd/d20/d26/d47/d9a/db9/d112/d52/d55/ff2 0 2024-03-21T10:09:10.227 INFO:tasks.workunit.client.0.smithi153.stdout:0/943: fsync d4/d4c/d83/d22/d61/d77/fe6 0 2024-03-21T10:09:10.229 INFO:tasks.workunit.client.1.smithi174.stdout:6/764: symlink d0/d3/d20/d3d/lf2 0 2024-03-21T10:09:10.234 INFO:tasks.workunit.client.1.smithi174.stdout:0/881: truncate d9/d21/d31/d9f/d79/fa0 611931 0 2024-03-21T10:09:10.235 INFO:tasks.workunit.client.0.smithi153.stdout:0/944: write d4/d16/d19/d30/d6c/d98/d5e/db2/ff5 [4739115,87395] 0 2024-03-21T10:09:10.236 INFO:tasks.workunit.client.1.smithi174.stdout:5/870: dread d1/d16/f60 [0,4194304] 0 2024-03-21T10:09:10.240 INFO:tasks.workunit.client.1.smithi174.stdout:3/838: truncate d0/d12/d14/da9/de7/f8a 5377 0 2024-03-21T10:09:10.242 INFO:tasks.workunit.client.0.smithi153.stdout:0/945: unlink d4/d16/d19/d30/da7/la8 0 2024-03-21T10:09:10.245 INFO:tasks.workunit.client.1.smithi174.stdout:6/765: symlink d0/d3/d5/lf3 0 2024-03-21T10:09:10.247 INFO:tasks.workunit.client.0.smithi153.stdout:0/946: mknod d4/d16/d19/d58/de9/c12c 0 2024-03-21T10:09:10.253 INFO:tasks.workunit.client.1.smithi174.stdout:3/839: creat d0/d12/d14/d67/dc8/f100 x:0 0 0 2024-03-21T10:09:10.256 INFO:tasks.workunit.client.1.smithi174.stdout:0/882: read d9/d21/d81/fef [2507281,21605] 0 2024-03-21T10:09:10.261 INFO:tasks.workunit.client.1.smithi174.stdout:6/766: rmdir d0/d3/d5/df/d16/da9/ddb 0 2024-03-21T10:09:10.262 INFO:tasks.workunit.client.1.smithi174.stdout:2/806: dread d5/f26 [0,4194304] 0 2024-03-21T10:09:10.265 INFO:tasks.workunit.client.0.smithi153.stdout:1/925: dwrite d6/d13/d8e/dbb/de1/de7/de9/df5/d50/d51/fa6 [0,4194304] 0 2024-03-21T10:09:10.266 INFO:tasks.workunit.client.1.smithi174.stdout:1/999: dread d4/f19 [0,4194304] 0 2024-03-21T10:09:10.266 INFO:tasks.workunit.client.0.smithi153.stdout:0/947: dread d4/d16/d19/d30/f41 [0,4194304] 0 2024-03-21T10:09:10.268 INFO:tasks.workunit.client.0.smithi153.stdout:8/979: dwrite d5/d9/d43/dd/d20/d26/d47/d9a/db9/d112/d52/d55/f90 [0,4194304] 0 2024-03-21T10:09:10.271 INFO:tasks.workunit.client.0.smithi153.stdout:0/948: readlink d4/d16/d19/d30/da7/lae 0 2024-03-21T10:09:10.271 INFO:tasks.workunit.client.1.smithi174.stdout:0/883: chown d9/d21/d31/d5c/d92/d98/f126 224854 1 2024-03-21T10:09:10.271 INFO:tasks.workunit.client.1.smithi174.stdout:5/871: dwrite d1/d5c/d96/f105 [4194304,4194304] 0 2024-03-21T10:09:10.274 INFO:tasks.workunit.client.1.smithi174.stdout:7/803: dwrite d0/d11b/f85 [0,4194304] 0 2024-03-21T10:09:10.275 INFO:tasks.workunit.client.0.smithi153.stdout:8/980: write d5/d9/d43/d7b/daa/f116 [917971,31594] 0 2024-03-21T10:09:10.279 INFO:tasks.workunit.client.1.smithi174.stdout:9/888: dread d2/d24/d2e/fb1 [0,4194304] 0 2024-03-21T10:09:10.280 INFO:tasks.workunit.client.0.smithi153.stdout:1/926: mkdir d6/d10/d35/da5/d4f/d9d/dc7/dcc/d137 0 2024-03-21T10:09:10.285 INFO:tasks.workunit.client.1.smithi174.stdout:5/872: creat d1/d5c/f135 x:0 0 0 2024-03-21T10:09:10.287 INFO:tasks.workunit.client.0.smithi153.stdout:1/927: truncate f4 1487238 0 2024-03-21T10:09:10.289 INFO:tasks.workunit.client.1.smithi174.stdout:8/860: dread d9/d2a/d4d/db2/fc7 [0,4194304] 0 2024-03-21T10:09:10.293 INFO:tasks.workunit.client.0.smithi153.stdout:0/949: dread d4/d16/fc1 [0,4194304] 0 2024-03-21T10:09:10.295 INFO:tasks.workunit.client.0.smithi153.stdout:1/928: creat d6/d13/d8e/dbb/de1/de7/de9/d119/f138 x:0 0 0 2024-03-21T10:09:10.296 INFO:tasks.workunit.client.1.smithi174.stdout:0/884: dread d9/d35/f46 [0,4194304] 0 2024-03-21T10:09:10.300 INFO:tasks.workunit.client.0.smithi153.stdout:8/981: write d5/d9/d43/f8c [1072502,18886] 0 2024-03-21T10:09:10.301 INFO:tasks.workunit.client.1.smithi174.stdout:6/767: dwrite d0/d3/d5/f1c [0,4194304] 0 2024-03-21T10:09:10.301 INFO:tasks.workunit.client.1.smithi174.stdout:3/840: dwrite d0/d12/d14/d67/dc8/fd1 [0,4194304] 0 2024-03-21T10:09:10.301 INFO:tasks.workunit.client.0.smithi153.stdout:1/929: getdents d6/d10f 0 2024-03-21T10:09:10.305 INFO:tasks.workunit.client.0.smithi153.stdout:8/982: creat d5/d9/d43/dd/d20/d11f/f13c x:0 0 0 2024-03-21T10:09:10.305 INFO:tasks.workunit.client.1.smithi174.stdout:6/768: fsync d0/d3/d2a/d56/d5e/fa7 0 2024-03-21T10:09:10.306 INFO:tasks.workunit.client.1.smithi174.stdout:5/873: unlink d1/d16/f5f 0 2024-03-21T10:09:10.309 INFO:tasks.workunit.client.0.smithi153.stdout:1/930: dread d6/d10/d35/da5/d4f/d9d/dc7/dcc/fd8 [0,4194304] 0 2024-03-21T10:09:10.310 INFO:tasks.workunit.client.0.smithi153.stdout:8/983: truncate d5/d9/d43/dd/d20/d26/d8b/f4e 5355196 0 2024-03-21T10:09:10.310 INFO:tasks.workunit.client.1.smithi174.stdout:9/889: write d2/d24/d2e/d63/fad [915622,38016] 0 2024-03-21T10:09:10.314 INFO:tasks.workunit.client.0.smithi153.stdout:1/931: stat d6/d13/d8e/dbb/de1/de7/de9/df5/d50/l6b 0 2024-03-21T10:09:10.314 INFO:tasks.workunit.client.1.smithi174.stdout:8/861: write d9/dc6/fa5 [2671799,125466] 0 2024-03-21T10:09:10.322 INFO:tasks.workunit.client.1.smithi174.stdout:0/885: creat d9/d21/d23/d80/f135 x:0 0 0 2024-03-21T10:09:10.324 INFO:tasks.workunit.client.0.smithi153.stdout:1/932: readlink d6/d10/d35/da5/d4f/d9d/db6/def/d92/lc9 0 2024-03-21T10:09:10.324 INFO:tasks.workunit.client.0.smithi153.stdout:0/950: dwrite d4/d4c/d83/f20 [4194304,4194304] 0 2024-03-21T10:09:10.329 INFO:tasks.workunit.client.1.smithi174.stdout:2/807: dwrite d5/d25/d27/f80 [0,4194304] 0 2024-03-21T10:09:10.329 INFO:tasks.workunit.client.1.smithi174.stdout:9/890: symlink d2/d10a/l139 0 2024-03-21T10:09:10.330 INFO:tasks.workunit.client.1.smithi174.stdout:8/862: mknod d9/d2a/d39/d76/c119 0 2024-03-21T10:09:10.331 INFO:tasks.workunit.client.0.smithi153.stdout:1/933: fsync d6/d13/d8e/dbb/de1/de7/de9/df5/d50/f7f 0 2024-03-21T10:09:10.332 INFO:tasks.workunit.client.1.smithi174.stdout:7/804: dwrite d0/d4/d10/d1b/d86/dea/f10d [0,4194304] 0 2024-03-21T10:09:10.332 INFO:tasks.workunit.client.1.smithi174.stdout:3/841: getdents d0/d12/d14/da9/de7 0 2024-03-21T10:09:10.336 INFO:tasks.workunit.client.0.smithi153.stdout:0/951: symlink d4/d16/d19/d58/de9/d11c/l12d 0 2024-03-21T10:09:10.337 INFO:tasks.workunit.client.1.smithi174.stdout:0/886: creat d9/d21/d81/f136 x:0 0 0 2024-03-21T10:09:10.347 INFO:tasks.workunit.client.1.smithi174.stdout:7/805: truncate d0/d4/d10/d79/db2/dda/d5f/d65/fcd 626793 0 2024-03-21T10:09:10.347 INFO:tasks.workunit.client.1.smithi174.stdout:2/808: symlink d5/d25/d27/d2d/db4/l116 0 2024-03-21T10:09:10.348 INFO:tasks.workunit.client.1.smithi174.stdout:8/863: symlink d9/d2a/d39/d76/d19/d27/d60/daa/dab/l11a 0 2024-03-21T10:09:10.349 INFO:tasks.workunit.client.1.smithi174.stdout:0/887: rmdir d9/d1c/d76/d87 39 2024-03-21T10:09:10.353 INFO:tasks.workunit.client.1.smithi174.stdout:7/806: write d0/d4/d10/d44/d81/f8f [9176973,25002] 0 2024-03-21T10:09:10.364 INFO:tasks.workunit.client.1.smithi174.stdout:0/888: link d9/d21/d31/fc6 d9/d35/d4a/d67/d89/f137 0 2024-03-21T10:09:10.371 INFO:tasks.workunit.client.1.smithi174.stdout:0/889: mkdir d9/d21/d31/d5c/d92/d138 0 2024-03-21T10:09:10.376 INFO:tasks.workunit.client.1.smithi174.stdout:0/890: creat d9/d21/d5f/f139 x:0 0 0 2024-03-21T10:09:10.378 INFO:tasks.workunit.client.1.smithi174.stdout:2/809: dwrite d5/f14 [12582912,4194304] 0 2024-03-21T10:09:10.379 INFO:tasks.workunit.client.0.smithi153.stdout:8/984: write d5/d9/d43/f2b [4735450,107744] 0 2024-03-21T10:09:10.383 INFO:tasks.workunit.client.1.smithi174.stdout:2/810: stat d5/d25/d27/d2d/le7 0 2024-03-21T10:09:10.383 INFO:tasks.workunit.client.1.smithi174.stdout:2/811: read d5/f26 [1793860,80611] 0 2024-03-21T10:09:10.384 INFO:tasks.workunit.client.0.smithi153.stdout:8/985: fsync d5/d9/d43/dd/d20/d26/d7a/f81 0 2024-03-21T10:09:10.386 INFO:tasks.workunit.client.1.smithi174.stdout:0/891: truncate d9/d35/dd2/d104/f83 4518635 0 2024-03-21T10:09:10.390 INFO:tasks.workunit.client.1.smithi174.stdout:2/812: rmdir d5/d25/d27/d2d/d45/dae/dd6/d90/da9 39 2024-03-21T10:09:10.393 INFO:tasks.workunit.client.1.smithi174.stdout:3/842: dread d0/d12/d14/d1a/d44/d65/d7e/dac/fc0 [0,4194304] 0 2024-03-21T10:09:10.393 INFO:tasks.workunit.client.1.smithi174.stdout:0/892: dread - d9/d21/d31/d9f/d79/fa6 zero size 2024-03-21T10:09:10.393 INFO:tasks.workunit.client.1.smithi174.stdout:5/874: write d1/fd4 [300194,69047] 0 2024-03-21T10:09:10.394 INFO:tasks.workunit.client.1.smithi174.stdout:9/891: dread d2/d24/f59 [0,4194304] 0 2024-03-21T10:09:10.395 INFO:tasks.workunit.client.1.smithi174.stdout:2/813: fdatasync d5/d25/d27/d2d/d45/d5b/f22 0 2024-03-21T10:09:10.397 INFO:tasks.workunit.client.1.smithi174.stdout:9/892: dread - d2/d11/d1d/d2a/fc9 zero size 2024-03-21T10:09:10.401 INFO:tasks.workunit.client.0.smithi153.stdout:1/934: truncate d6/fd9 1579602 0 2024-03-21T10:09:10.405 INFO:tasks.workunit.client.1.smithi174.stdout:0/893: mknod d9/d35/d4a/d67/d89/dfd/c13a 0 2024-03-21T10:09:10.414 INFO:tasks.workunit.client.1.smithi174.stdout:2/814: creat d5/d25/d27/d2d/d45/d5b/d2f/d68/de8/f117 x:0 0 0 2024-03-21T10:09:10.415 INFO:tasks.workunit.client.1.smithi174.stdout:5/875: creat d1/f136 x:0 0 0 2024-03-21T10:09:10.419 INFO:tasks.workunit.client.1.smithi174.stdout:8/864: dwrite d9/f7c [0,4194304] 0 2024-03-21T10:09:10.420 INFO:tasks.workunit.client.0.smithi153.stdout:8/986: creat d5/d9/d43/d7b/daa/f13d x:0 0 0 2024-03-21T10:09:10.421 INFO:tasks.workunit.client.1.smithi174.stdout:7/807: dwrite d0/d4/d10/f19 [0,4194304] 0 2024-03-21T10:09:10.424 INFO:tasks.workunit.client.1.smithi174.stdout:5/876: rmdir d1/d49/d4f/d51 39 2024-03-21T10:09:10.426 INFO:tasks.workunit.client.1.smithi174.stdout:3/843: link d0/d12/d14/d1a/d44/d56/l69 d0/d12/d14/d1a/d27/d4f/l101 0 2024-03-21T10:09:10.427 INFO:tasks.workunit.client.0.smithi153.stdout:8/987: rmdir d5/d9/d43/dd/d20 39 2024-03-21T10:09:10.428 INFO:tasks.workunit.client.1.smithi174.stdout:8/865: mknod d9/d2a/d39/de1/c11b 0 2024-03-21T10:09:10.430 INFO:tasks.workunit.client.1.smithi174.stdout:3/844: truncate d0/d12/fdf 731583 0 2024-03-21T10:09:10.430 INFO:tasks.workunit.client.1.smithi174.stdout:0/894: dwrite d9/d21/d31/d5c/d86/ff3 [0,4194304] 0 2024-03-21T10:09:10.434 INFO:tasks.workunit.client.1.smithi174.stdout:8/866: chown d9/d53/dc9/de6 447217663 1 2024-03-21T10:09:10.435 INFO:tasks.workunit.client.1.smithi174.stdout:7/808: creat d0/d11b/dfe/f11d x:0 0 0 2024-03-21T10:09:10.435 INFO:tasks.workunit.client.1.smithi174.stdout:2/815: creat d5/d25/d27/d2d/d45/dae/dd6/d90/f118 x:0 0 0 2024-03-21T10:09:10.436 INFO:tasks.workunit.client.1.smithi174.stdout:5/877: mkdir d1/d2f/d37/d137 0 2024-03-21T10:09:10.436 INFO:tasks.workunit.client.0.smithi153.stdout:8/988: truncate d5/d9/d43/dd/d20/d26/d47/d9a/f11c 392807 0 2024-03-21T10:09:10.436 INFO:tasks.workunit.client.0.smithi153.stdout:0/952: dread d4/d16/d19/d30/d6c/d98/d5e/f70 [0,4194304] 0 2024-03-21T10:09:10.442 INFO:tasks.workunit.client.1.smithi174.stdout:4/943: sync 2024-03-21T10:09:10.445 INFO:tasks.workunit.client.1.smithi174.stdout:9/893: dwrite d2/d11/d60/fb6 [0,4194304] 0 2024-03-21T10:09:10.457 INFO:tasks.workunit.client.0.smithi153.stdout:1/935: write d6/d13/d8e/dbb/de1/de7/de9/df5/d50/f77 [2292217,32676] 0 2024-03-21T10:09:10.469 INFO:tasks.workunit.client.0.smithi153.stdout:1/936: creat d6/de6/f139 x:0 0 0 2024-03-21T10:09:10.476 INFO:tasks.workunit.client.0.smithi153.stdout:1/937: mkdir d6/d10/d35/da5/d4f/d9d/db6/def/d13a 0 2024-03-21T10:09:10.485 INFO:tasks.workunit.client.0.smithi153.stdout:8/989: dwrite d5/d9/d43/d7b/daa/dca/fd7 [0,4194304] 0 2024-03-21T10:09:10.486 INFO:tasks.workunit.client.0.smithi153.stdout:1/938: dread d6/d10/f98 [0,4194304] 0 2024-03-21T10:09:10.490 INFO:tasks.workunit.client.0.smithi153.stdout:0/953: write d4/d16/d19/d30/d6c/d98/f80 [2592318,110598] 0 2024-03-21T10:09:10.491 INFO:tasks.workunit.client.0.smithi153.stdout:1/939: stat d6/d10/d35/da5/d4f/d9d/db6/def/d92/lc9 0 2024-03-21T10:09:10.501 INFO:tasks.workunit.client.0.smithi153.stdout:1/940: dread - d6/d10/da0/f12b zero size 2024-03-21T10:09:10.504 INFO:tasks.workunit.client.0.smithi153.stdout:1/941: readlink d6/d32/l49 0 2024-03-21T10:09:10.511 INFO:tasks.workunit.client.0.smithi153.stdout:1/942: truncate d6/d10/d35/da5/d4f/d9d/dc7/dcc/f6a 969282 0 2024-03-21T10:09:10.514 INFO:tasks.workunit.client.0.smithi153.stdout:1/943: mknod d6/d3a/c13b 0 2024-03-21T10:09:10.526 INFO:tasks.workunit.client.1.smithi174.stdout:4/944: mknod d4/d21/d24/d28/de9/d6a/dd1/c138 0 2024-03-21T10:09:10.529 INFO:tasks.workunit.client.1.smithi174.stdout:6/769: sync 2024-03-21T10:09:10.530 INFO:tasks.workunit.client.1.smithi174.stdout:5/878: link d1/d2f/d37/d9b/ff0 d1/d2f/d37/df5/d11e/f138 0 2024-03-21T10:09:10.531 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:10 smithi153 ceph-mon[29117]: pgmap v21: 65 pgs: 65 active+clean; 3.4 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 87 MiB/s rd, 218 MiB/s wr, 456 op/s 2024-03-21T10:09:10.532 INFO:tasks.workunit.client.1.smithi174.stdout:2/816: creat d5/d25/d27/d2d/d45/dae/dd6/d4f/df1/dea/f119 x:0 0 0 2024-03-21T10:09:10.538 INFO:tasks.workunit.client.1.smithi174.stdout:0/895: write d9/d21/d31/d9f/f12a [8563551,64145] 0 2024-03-21T10:09:10.542 INFO:tasks.workunit.client.1.smithi174.stdout:2/817: creat d5/d25/d27/d2d/db4/f11a x:0 0 0 2024-03-21T10:09:10.544 INFO:tasks.workunit.client.1.smithi174.stdout:3/845: truncate d0/d12/f2d 160244 0 2024-03-21T10:09:10.548 INFO:tasks.workunit.client.1.smithi174.stdout:2/818: chown d5/d25/d27/d2d/d45/d5b/ff 15054 1 2024-03-21T10:09:10.548 INFO:tasks.workunit.client.0.smithi153.stdout:8/990: dread d5/d9/d43/d7b/daa/fb8 [0,4194304] 0 2024-03-21T10:09:10.549 INFO:tasks.workunit.client.0.smithi153.stdout:1/944: dwrite f3 [0,4194304] 0 2024-03-21T10:09:10.557 INFO:tasks.workunit.client.1.smithi174.stdout:2/819: creat d5/d25/d27/d2d/d45/d5b/d55/d75/f11b x:0 0 0 2024-03-21T10:09:10.558 INFO:tasks.workunit.client.0.smithi153.stdout:8/991: mkdir d5/d9/d43/dd/d2d/d45/d13e 0 2024-03-21T10:09:10.560 INFO:tasks.workunit.client.1.smithi174.stdout:5/879: dwrite d1/d49/d4f/d51/d8f/f104 [0,4194304] 0 2024-03-21T10:09:10.565 INFO:tasks.workunit.client.0.smithi153.stdout:8/992: rmdir d5/d9/d43/dd/d20/d26/d47/d69 39 2024-03-21T10:09:10.567 INFO:tasks.workunit.client.1.smithi174.stdout:2/820: creat d5/d25/d27/f11c x:0 0 0 2024-03-21T10:09:10.568 INFO:tasks.workunit.client.0.smithi153.stdout:8/993: creat d5/d9/d43/dd/d2d/d45/d88/de0/d113/f13f x:0 0 0 2024-03-21T10:09:10.575 INFO:tasks.workunit.client.0.smithi153.stdout:8/994: readlink d5/d9/d43/dd/d20/d26/d47/d9a/db9/d112/d52/le9 0 2024-03-21T10:09:10.580 INFO:tasks.workunit.client.0.smithi153.stdout:8/995: mknod d5/d9/d43/dd/d20/d26/d47/d9a/d132/c140 0 2024-03-21T10:09:10.582 INFO:tasks.workunit.client.1.smithi174.stdout:5/880: dwrite d1/d2f/f111 [0,4194304] 0 2024-03-21T10:09:10.585 INFO:tasks.workunit.client.0.smithi153.stdout:1/945: dwrite d6/d10/d35/da5/f42 [0,4194304] 0 2024-03-21T10:09:10.586 INFO:tasks.workunit.client.1.smithi174.stdout:5/881: chown d1/d49/d4f/d51/d8f/d103/f133 0 1 2024-03-21T10:09:10.591 INFO:tasks.workunit.client.0.smithi153.stdout:1/946: fsync d6/d10/d35/da5/d4f/d9d/db6/def/fde 0 2024-03-21T10:09:10.593 INFO:tasks.workunit.client.1.smithi174.stdout:5/882: write d1/d49/d4f/d51/d8f/fb6 [1670265,106059] 0 2024-03-21T10:09:10.594 INFO:tasks.workunit.client.0.smithi153.stdout:1/947: chown d6/d10/d35/da5/d4f/l89 1 1 2024-03-21T10:09:10.603 INFO:tasks.workunit.client.1.smithi174.stdout:5/883: link d1/d2f/d37/f5d d1/d16/d2e/d8d/f139 0 2024-03-21T10:09:10.603 INFO:tasks.workunit.client.0.smithi153.stdout:1/948: rmdir d6/d13/d75 39 2024-03-21T10:09:10.609 INFO:tasks.workunit.client.0.smithi153.stdout:8/996: write d5/d9/d43/d7b/daa/dca/f11a [955704,107856] 0 2024-03-21T10:09:10.629 INFO:tasks.workunit.client.1.smithi174.stdout:6/770: dwrite d0/fef [0,4194304] 0 2024-03-21T10:09:10.632 INFO:tasks.workunit.client.0.smithi153.stdout:1/949: write d6/d13/d8e/dbb/de1/de7/de9/df5/d50/d51/f5f [1537244,68613] 0 2024-03-21T10:09:10.638 INFO:tasks.workunit.client.0.smithi153.stdout:1/950: read - d6/f90 zero size 2024-03-21T10:09:10.638 INFO:tasks.workunit.client.0.smithi153.stdout:1/951: symlink d6/d32/daa/l13c 0 2024-03-21T10:09:10.638 INFO:tasks.workunit.client.1.smithi174.stdout:9/894: rename d2/fce to d2/d11/d60/db2/f13a 0 2024-03-21T10:09:10.640 INFO:tasks.workunit.client.1.smithi174.stdout:9/895: dread - d2/d24/d2e/dac/f127 zero size 2024-03-21T10:09:10.642 INFO:tasks.workunit.client.0.smithi153.stdout:1/952: write d6/de6/f139 [63639,106038] 0 2024-03-21T10:09:10.645 INFO:tasks.workunit.client.0.smithi153.stdout:8/997: dwrite d5/d9/d43/dd/d20/d26/d8b/f62 [0,4194304] 0 2024-03-21T10:09:10.650 INFO:tasks.workunit.client.0.smithi153.stdout:1/953: creat d6/d13/d8e/dbb/f13d x:0 0 0 2024-03-21T10:09:10.652 INFO:tasks.workunit.client.1.smithi174.stdout:8/867: rename d9/d2a/d39/d76/d19/de9 to d9/d2a/d39/d58/dbb/d11c 0 2024-03-21T10:09:10.653 INFO:tasks.workunit.client.0.smithi153.stdout:1/954: readlink d6/d10/d35/da5/d4f/lbf 0 2024-03-21T10:09:10.654 INFO:tasks.workunit.client.0.smithi153.stdout:1/955: chown d6/f16 7073295 1 2024-03-21T10:09:10.658 INFO:tasks.workunit.client.0.smithi153.stdout:8/998: link d5/d9/d43/dd/l123 d5/d9/d43/dd/d20/d26/d47/l141 0 2024-03-21T10:09:10.661 INFO:tasks.workunit.client.1.smithi174.stdout:7/809: rename d0/d4/d10/d79/db2/dda/d5f/c69 to d0/d4/d10/d44/d81/d9a/dc0/c11e 0 2024-03-21T10:09:10.666 INFO:tasks.workunit.client.1.smithi174.stdout:6/771: dread d0/d3/d5/df/f2e [0,4194304] 0 2024-03-21T10:09:10.667 INFO:tasks.workunit.client.1.smithi174.stdout:4/945: rename d4/d21/d24/f116 to d4/d21/d24/d28/de9/d3a/d7a/d9d/d11f/f139 0 2024-03-21T10:09:10.672 INFO:tasks.workunit.client.1.smithi174.stdout:5/884: mknod d1/d5c/c13a 0 2024-03-21T10:09:10.673 INFO:tasks.workunit.client.1.smithi174.stdout:5/885: stat d1/d16/d2e/d66/f89 0 2024-03-21T10:09:10.674 INFO:tasks.workunit.client.1.smithi174.stdout:9/896: rename d2/d10b/d112/f129 to d2/d10a/f13b 0 2024-03-21T10:09:10.678 INFO:tasks.workunit.client.0.smithi153.stdout:1/956: dwrite d6/d10/d35/da5/d4f/d9d/db6/def/d92/fc0 [0,4194304] 0 2024-03-21T10:09:10.679 INFO:tasks.workunit.client.1.smithi174.stdout:7/810: creat d0/d4/d10/d44/f11f x:0 0 0 2024-03-21T10:09:10.684 INFO:tasks.workunit.client.1.smithi174.stdout:2/821: creat d5/d25/d27/d2d/d45/d5b/f11d x:0 0 0 2024-03-21T10:09:10.685 INFO:tasks.workunit.client.0.smithi153.stdout:1/957: getdents d6/d13/d8e/dbb/de1/de7/de9 0 2024-03-21T10:09:10.686 INFO:tasks.workunit.client.1.smithi174.stdout:5/886: creat d1/d130/f13b x:0 0 0 2024-03-21T10:09:10.687 INFO:tasks.workunit.client.1.smithi174.stdout:7/811: creat d0/d4/de3/f120 x:0 0 0 2024-03-21T10:09:10.689 INFO:tasks.workunit.client.0.smithi153.stdout:1/958: mkdir d6/d13/d8e/dbb/de1/de7/d13e 0 2024-03-21T10:09:10.690 INFO:tasks.workunit.client.0.smithi153.stdout:8/999: truncate d5/d9/d43/dd/d20/d26/d8b/fa4 1149960 0 2024-03-21T10:09:10.690 INFO:tasks.workunit.client.1.smithi174.stdout:2/822: chown d5/d25/d27/d2d/d45/dae/fac 3 1 2024-03-21T10:09:10.692 INFO:tasks.workunit.client.1.smithi174.stdout:5/887: write d1/d2f/dc1/fe8 [2523153,56398] 0 2024-03-21T10:09:10.692 INFO:tasks.workunit.client.1.smithi174.stdout:9/897: unlink d2/d24/d2e/d63/c66 0 2024-03-21T10:09:10.695 INFO:tasks.workunit.client.0.smithi153.stdout:1/959: write d6/d10/d35/da5/d4f/d9d/dc7/dcc/fd8 [915962,95399] 0 2024-03-21T10:09:10.695 INFO:tasks.workunit.client.1.smithi174.stdout:9/898: write d2/d11/d1d/d10c/f12b [637467,43153] 0 2024-03-21T10:09:10.696 INFO:tasks.workunit.client.1.smithi174.stdout:4/946: creat d4/d21/d24/d28/de9/d3a/dd2/f13a x:0 0 0 2024-03-21T10:09:10.700 INFO:tasks.workunit.client.1.smithi174.stdout:4/947: dread - d4/d21/d24/d28/de9/d3a/d7a/d9d/d11f/f12c zero size 2024-03-21T10:09:10.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:10 smithi174 ceph-mon[37496]: pgmap v21: 65 pgs: 65 active+clean; 3.4 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 87 MiB/s rd, 218 MiB/s wr, 456 op/s 2024-03-21T10:09:10.701 INFO:tasks.workunit.client.1.smithi174.stdout:5/888: creat d1/d5c/d73/de3/f13c x:0 0 0 2024-03-21T10:09:10.701 INFO:tasks.workunit.client.0.smithi153.stdout:1/960: creat d6/d10/da0/d106/f13f x:0 0 0 2024-03-21T10:09:10.703 INFO:tasks.workunit.client.1.smithi174.stdout:6/772: link d0/d3/d20/d3d/d81/fc0 d0/d3/d2a/d56/d5e/d91/de9/ff4 0 2024-03-21T10:09:10.705 INFO:tasks.workunit.client.1.smithi174.stdout:9/899: fdatasync d2/d24/fe7 0 2024-03-21T10:09:10.705 INFO:tasks.workunit.client.0.smithi153.stdout:1/961: dread d6/d13/d75/f83 [0,4194304] 0 2024-03-21T10:09:10.707 INFO:tasks.workunit.client.1.smithi174.stdout:7/812: mkdir d0/d4/d121 0 2024-03-21T10:09:10.708 INFO:tasks.workunit.client.1.smithi174.stdout:4/948: symlink d4/d19/l13b 0 2024-03-21T10:09:10.709 INFO:tasks.workunit.client.1.smithi174.stdout:6/773: truncate d0/d3/d20/d3d/d81/da5/fc8 332490 0 2024-03-21T10:09:10.709 INFO:tasks.workunit.client.1.smithi174.stdout:2/823: mkdir d5/d25/d27/d2d/d45/dae/dd6/d11e 0 2024-03-21T10:09:10.710 INFO:tasks.workunit.client.1.smithi174.stdout:5/889: creat d1/d49/d4f/d51/d8f/d103/f13d x:0 0 0 2024-03-21T10:09:10.711 INFO:tasks.workunit.client.1.smithi174.stdout:4/949: chown d4/d21/d24/d28/de9/d3a/d7a/d9d/c131 1783226814 1 2024-03-21T10:09:10.717 INFO:tasks.workunit.client.1.smithi174.stdout:6/774: mkdir d0/df5 0 2024-03-21T10:09:10.718 INFO:tasks.workunit.client.1.smithi174.stdout:7/813: symlink d0/d4/d10/d79/l122 0 2024-03-21T10:09:10.720 INFO:tasks.workunit.client.0.smithi153.stdout:1/962: dwrite d6/d10/d35/da5/d4f/d9d/dc7/dcc/d80/f10b [0,4194304] 0 2024-03-21T10:09:10.723 INFO:tasks.workunit.client.1.smithi174.stdout:2/824: creat d5/d25/d27/d2d/d45/dae/dd6/d4f/d65/d77/d106/f11f x:0 0 0 2024-03-21T10:09:10.724 INFO:tasks.workunit.client.1.smithi174.stdout:3/846: dwrite d0/d12/f2d [0,4194304] 0 2024-03-21T10:09:10.726 INFO:tasks.workunit.client.0.smithi153.stdout:1/963: getdents d6/d13/d8e 0 2024-03-21T10:09:10.731 INFO:tasks.workunit.client.1.smithi174.stdout:5/890: creat d1/d49/d4f/d51/f13e x:0 0 0 2024-03-21T10:09:10.740 INFO:tasks.workunit.client.1.smithi174.stdout:6/775: dwrite d0/fef [0,4194304] 0 2024-03-21T10:09:10.745 INFO:tasks.workunit.client.1.smithi174.stdout:7/814: rename d0/d11b/da2 to d0/d4/d10/d1b/d86/dea/df4/d123 0 2024-03-21T10:09:10.749 INFO:tasks.workunit.client.1.smithi174.stdout:0/896: sync 2024-03-21T10:09:10.752 INFO:tasks.workunit.client.1.smithi174.stdout:0/897: write d9/ff4 [4096156,6752] 0 2024-03-21T10:09:10.753 INFO:tasks.workunit.client.1.smithi174.stdout:6/776: chown d0/d3/d5/d3b/l47 2978 1 2024-03-21T10:09:10.756 INFO:tasks.workunit.client.1.smithi174.stdout:7/815: dread - d0/d4/d10/d1b/d86/f9d zero size 2024-03-21T10:09:10.762 INFO:tasks.workunit.client.1.smithi174.stdout:2/825: link d5/d25/d27/d2d/d45/dae/dd6/d90/da9/f10c d5/d25/d27/d2d/d45/d5b/d2f/d66/db5/f120 0 2024-03-21T10:09:10.763 INFO:tasks.workunit.client.1.smithi174.stdout:3/847: creat d0/d12/f102 x:0 0 0 2024-03-21T10:09:10.765 INFO:tasks.workunit.client.1.smithi174.stdout:6/777: symlink d0/d3/d2a/d56/d5e/lf6 0 2024-03-21T10:09:10.771 INFO:tasks.workunit.client.1.smithi174.stdout:6/778: chown d0/d3/d5/df/fd5 4134979 1 2024-03-21T10:09:10.773 INFO:tasks.workunit.client.1.smithi174.stdout:3/848: mkdir d0/d12/d14/d98/d103 0 2024-03-21T10:09:10.779 INFO:tasks.workunit.client.1.smithi174.stdout:6/779: write d0/d3/d2a/d56/d5e/d91/fb3 [5095069,81028] 0 2024-03-21T10:09:10.784 INFO:tasks.workunit.client.1.smithi174.stdout:3/849: chown d0/d12/d14/d1a/d44/cc3 1742 1 2024-03-21T10:09:10.789 INFO:tasks.workunit.client.1.smithi174.stdout:4/950: dwrite d4/d21/d24/d28/fed [0,4194304] 0 2024-03-21T10:09:10.797 INFO:tasks.workunit.client.1.smithi174.stdout:4/951: mkdir d4/d19/dd5/d13c 0 2024-03-21T10:09:10.801 INFO:tasks.workunit.client.1.smithi174.stdout:0/898: write d9/d21/d31/fc6 [2386902,107704] 0 2024-03-21T10:09:10.803 INFO:tasks.workunit.client.1.smithi174.stdout:9/900: dwrite d2/d11/d60/db2/f13a [0,4194304] 0 2024-03-21T10:09:10.804 INFO:tasks.workunit.client.1.smithi174.stdout:4/952: truncate d4/d19/dd5/f11d 858475 0 2024-03-21T10:09:10.807 INFO:tasks.workunit.client.1.smithi174.stdout:6/780: dwrite d0/d3/d5/d6/d42/f71 [0,4194304] 0 2024-03-21T10:09:10.810 INFO:tasks.workunit.client.1.smithi174.stdout:2/826: truncate d5/d25/d27/d2d/d45/dae/dd6/d4f/d65/d77/ff7 118450 0 2024-03-21T10:09:10.820 INFO:tasks.workunit.client.1.smithi174.stdout:9/901: creat d2/d24/d2e/d63/f13c x:0 0 0 2024-03-21T10:09:10.827 INFO:tasks.workunit.client.1.smithi174.stdout:6/781: creat d0/d3/d20/ff7 x:0 0 0 2024-03-21T10:09:10.829 INFO:tasks.workunit.client.1.smithi174.stdout:4/953: write d4/d21/d24/d28/de9/d3a/d7a/d9d/da3/f107 [789293,95443] 0 2024-03-21T10:09:10.834 INFO:tasks.workunit.client.1.smithi174.stdout:9/902: fsync d2/d11/d1d/d2a/fc9 0 2024-03-21T10:09:10.839 INFO:tasks.workunit.client.1.smithi174.stdout:9/903: truncate d2/d11/d1d/d10c/f12b 845321 0 2024-03-21T10:09:10.843 INFO:tasks.workunit.client.1.smithi174.stdout:6/782: write d0/d3/d5/df/d16/da9/db7/dc2/dda/fe3 [764742,50626] 0 2024-03-21T10:09:10.846 INFO:tasks.workunit.client.1.smithi174.stdout:4/954: creat d4/d21/d24/d28/de9/d3a/f13d x:0 0 0 2024-03-21T10:09:10.847 INFO:tasks.workunit.client.1.smithi174.stdout:9/904: mknod d2/d11/c13d 0 2024-03-21T10:09:10.849 INFO:tasks.workunit.client.1.smithi174.stdout:9/905: stat d2/d11/d1d/d5e 0 2024-03-21T10:09:10.850 INFO:tasks.workunit.client.1.smithi174.stdout:3/850: dread d0/d12/d46/df9/fce [4194304,4194304] 0 2024-03-21T10:09:10.852 INFO:tasks.workunit.client.1.smithi174.stdout:9/906: write d2/d24/d2e/d63/f13c [501423,102666] 0 2024-03-21T10:09:10.854 INFO:tasks.workunit.client.1.smithi174.stdout:9/907: chown d2/d11/d1d/d4b/f56 0 1 2024-03-21T10:09:10.855 INFO:tasks.workunit.client.0.smithi153.stdout:0/954: sync 2024-03-21T10:09:10.856 INFO:tasks.workunit.client.1.smithi174.stdout:3/851: unlink d0/d12/d14/f62 0 2024-03-21T10:09:10.858 INFO:tasks.workunit.client.1.smithi174.stdout:6/783: truncate d0/d3/d20/d3d/d81/fc0 622853 0 2024-03-21T10:09:10.864 INFO:tasks.workunit.client.1.smithi174.stdout:3/852: mkdir d0/d12/d14/d67/da1/da7/d104 0 2024-03-21T10:09:10.864 INFO:tasks.workunit.client.1.smithi174.stdout:8/868: sync 2024-03-21T10:09:10.864 INFO:tasks.workunit.client.1.smithi174.stdout:5/891: sync 2024-03-21T10:09:10.864 INFO:tasks.workunit.client.1.smithi174.stdout:7/816: sync 2024-03-21T10:09:10.868 INFO:tasks.workunit.client.1.smithi174.stdout:0/899: dwrite d9/d21/d75/da1/fee [0,4194304] 0 2024-03-21T10:09:10.868 INFO:tasks.workunit.client.1.smithi174.stdout:6/784: mknod d0/d3/d5/d3b/cf8 0 2024-03-21T10:09:10.872 INFO:tasks.workunit.client.1.smithi174.stdout:0/900: fsync d9/f6d 0 2024-03-21T10:09:10.878 INFO:tasks.workunit.client.1.smithi174.stdout:2/827: dwrite d5/d25/d27/f2a [0,4194304] 0 2024-03-21T10:09:10.881 INFO:tasks.workunit.client.0.smithi153.stdout:0/955: dwrite d4/d4c/d83/d22/d61/d77/fb1 [0,4194304] 0 2024-03-21T10:09:10.886 INFO:tasks.workunit.client.0.smithi153.stdout:0/956: rmdir d4/d4c 39 2024-03-21T10:09:10.888 INFO:tasks.workunit.client.1.smithi174.stdout:0/901: creat d9/d35/d4a/d67/d89/d99/db2/dbd/f13b x:0 0 0 2024-03-21T10:09:10.890 INFO:tasks.workunit.client.0.smithi153.stdout:0/957: write d4/d16/d19/d30/d6c/d98/df3/ff0 [77229,60348] 0 2024-03-21T10:09:10.891 INFO:tasks.workunit.client.1.smithi174.stdout:9/908: getdents d2/d11/d60/db2 0 2024-03-21T10:09:10.894 INFO:tasks.workunit.client.0.smithi153.stdout:0/958: mknod d4/d16/d19/d58/d9a/c12e 0 2024-03-21T10:09:10.897 INFO:tasks.workunit.client.1.smithi174.stdout:4/955: dread d4/d19/f97 [0,4194304] 0 2024-03-21T10:09:10.898 INFO:tasks.workunit.client.0.smithi153.stdout:0/959: mkdir d4/d16/d27/d12f 0 2024-03-21T10:09:10.900 INFO:tasks.workunit.client.1.smithi174.stdout:7/817: symlink d0/d4/l124 0 2024-03-21T10:09:10.902 INFO:tasks.workunit.client.0.smithi153.stdout:0/960: chown d4/d4c/d83/d22/d61/d77/fb1 14989 1 2024-03-21T10:09:10.903 INFO:tasks.workunit.client.1.smithi174.stdout:4/956: chown d4/d21/d24/d28/d38/l102 18950325 1 2024-03-21T10:09:10.903 INFO:tasks.workunit.client.1.smithi174.stdout:4/957: readlink d4/d21/d24/ldf 0 2024-03-21T10:09:10.904 INFO:tasks.workunit.client.0.smithi153.stdout:0/961: write d4/f28 [7911629,24737] 0 2024-03-21T10:09:10.907 INFO:tasks.workunit.client.1.smithi174.stdout:9/909: creat d2/d11/d1d/d2a/da7/f13e x:0 0 0 2024-03-21T10:09:10.908 INFO:tasks.workunit.client.1.smithi174.stdout:7/818: symlink d0/d4/d10/d44/d81/l125 0 2024-03-21T10:09:10.910 INFO:tasks.workunit.client.1.smithi174.stdout:7/819: readlink d0/d4/d10/l8c 0 2024-03-21T10:09:10.910 INFO:tasks.workunit.client.1.smithi174.stdout:8/869: creat d9/d2a/d39/d58/dbb/f11d x:0 0 0 2024-03-21T10:09:10.911 INFO:tasks.workunit.client.1.smithi174.stdout:3/853: creat d0/d12/d14/d1a/d44/f105 x:0 0 0 2024-03-21T10:09:10.913 INFO:tasks.workunit.client.1.smithi174.stdout:8/870: fdatasync d9/d2a/d39/d58/f100 0 2024-03-21T10:09:10.913 INFO:tasks.workunit.client.1.smithi174.stdout:4/958: creat d4/d21/d24/d28/de9/d3a/d77/dcf/f13e x:0 0 0 2024-03-21T10:09:10.914 INFO:tasks.workunit.client.1.smithi174.stdout:3/854: stat d0/d12/d14/d1a/d44/d65/d7e/dac/fc0 0 2024-03-21T10:09:10.918 INFO:tasks.workunit.client.1.smithi174.stdout:7/820: mkdir d0/d4/d10/d79/db2/dda/d126 0 2024-03-21T10:09:10.938 INFO:tasks.workunit.client.0.smithi153.stdout:1/964: rename d6/d10/d35/lfd to d6/d13/d75/dfc/l140 0 2024-03-21T10:09:10.942 INFO:tasks.workunit.client.1.smithi174.stdout:3/855: mknod d0/d12/d14/d1a/d44/d57/c106 0 2024-03-21T10:09:10.943 INFO:tasks.workunit.client.0.smithi153.stdout:1/965: rmdir d6/d10/da0/d106 39 2024-03-21T10:09:10.948 INFO:tasks.workunit.client.1.smithi174.stdout:8/871: link d9/d2a/d4d/ce3 d9/d53/d7e/c11e 0 2024-03-21T10:09:10.948 INFO:tasks.workunit.client.1.smithi174.stdout:4/959: dread d4/f20 [0,4194304] 0 2024-03-21T10:09:10.952 INFO:tasks.workunit.client.1.smithi174.stdout:7/821: write d0/d4/d10/d1b/f3b [4638144,1822] 0 2024-03-21T10:09:10.956 INFO:tasks.workunit.client.0.smithi153.stdout:1/966: rmdir d6/d13/d8e/dbb/de1 39 2024-03-21T10:09:10.958 INFO:tasks.workunit.client.1.smithi174.stdout:5/892: dwrite d1/d16/d2e/d8d/d99/faf [0,4194304] 0 2024-03-21T10:09:10.958 INFO:tasks.workunit.client.1.smithi174.stdout:2/828: dwrite d5/d25/d27/f4e [4194304,4194304] 0 2024-03-21T10:09:10.962 INFO:tasks.workunit.client.0.smithi153.stdout:1/967: creat d6/d13/d8e/dbb/de1/de7/de9/d119/f141 x:0 0 0 2024-03-21T10:09:10.963 INFO:tasks.workunit.client.0.smithi153.stdout:1/968: stat d6/d10/f97 0 2024-03-21T10:09:10.965 INFO:tasks.workunit.client.1.smithi174.stdout:0/902: dread d9/d35/dd2/d104/d72/f85 [0,4194304] 0 2024-03-21T10:09:10.965 INFO:tasks.workunit.client.1.smithi174.stdout:4/960: mknod d4/d21/d24/d28/de9/c13f 0 2024-03-21T10:09:10.965 INFO:tasks.workunit.client.1.smithi174.stdout:7/822: creat d0/d4/f127 x:0 0 0 2024-03-21T10:09:10.966 INFO:tasks.workunit.client.0.smithi153.stdout:1/969: chown d6/d13/d8e/dbb/de1/de7/de9/df5/d50/d51 866 1 2024-03-21T10:09:10.966 INFO:tasks.workunit.client.0.smithi153.stdout:0/962: dwrite d4/d4c/d83/d22/f94 [0,4194304] 0 2024-03-21T10:09:10.971 INFO:tasks.workunit.client.1.smithi174.stdout:7/823: rmdir d0/d4/d10/d44 39 2024-03-21T10:09:10.974 INFO:tasks.workunit.client.1.smithi174.stdout:6/785: truncate d0/d3/d2a/d56/d5e/d91/de9/ff4 316889 0 2024-03-21T10:09:10.977 INFO:tasks.workunit.client.1.smithi174.stdout:2/829: dread - d5/d25/d27/d2d/d45/dae/dd6/d90/da9/f10c zero size 2024-03-21T10:09:10.978 INFO:tasks.workunit.client.0.smithi153.stdout:0/963: mknod d4/d16/d27/dc5/dcd/de8/dee/c130 0 2024-03-21T10:09:10.983 INFO:tasks.workunit.client.1.smithi174.stdout:3/856: dwrite d0/d12/d14/f4e [0,4194304] 0 2024-03-21T10:09:10.983 INFO:tasks.workunit.client.1.smithi174.stdout:5/893: rename d1/d16/d2e/d66/d8b/f33 to d1/d49/d4f/d51/d8f/d118/f13f 0 2024-03-21T10:09:10.992 INFO:tasks.workunit.client.1.smithi174.stdout:8/872: dwrite d9/d2a/d39/d76/d19/d27/d60/daa/fb9 [0,4194304] 0 2024-03-21T10:09:10.993 INFO:tasks.workunit.client.1.smithi174.stdout:4/961: mkdir d4/d21/d24/d28/de9/d3a/d77/dcf/d140 0 2024-03-21T10:09:10.996 INFO:tasks.workunit.client.0.smithi153.stdout:1/970: dwrite d6/fc [0,4194304] 0 2024-03-21T10:09:10.996 INFO:tasks.workunit.client.1.smithi174.stdout:6/786: creat d0/d3/d20/d6d/ff9 x:0 0 0 2024-03-21T10:09:11.002 INFO:tasks.workunit.client.0.smithi153.stdout:0/964: truncate d4/d4c/f7a 4416882 0 2024-03-21T10:09:11.007 INFO:tasks.workunit.client.1.smithi174.stdout:0/903: write d9/d35/dd2/d104/d72/f7e [4034083,95631] 0 2024-03-21T10:09:11.008 INFO:tasks.workunit.client.1.smithi174.stdout:9/910: sync 2024-03-21T10:09:11.009 INFO:tasks.workunit.client.1.smithi174.stdout:0/904: chown d9/f6d 7351660 1 2024-03-21T10:09:11.010 INFO:tasks.workunit.client.0.smithi153.stdout:0/965: rmdir d4/d4c/d83/d22/d61/d77 39 2024-03-21T10:09:11.014 INFO:tasks.workunit.client.0.smithi153.stdout:1/971: dread d6/d13/d8e/dbb/de1/de7/de9/df5/d50/f67 [0,4194304] 0 2024-03-21T10:09:11.025 INFO:tasks.workunit.client.0.smithi153.stdout:0/966: link d4/db/c3a d4/d16/d19/d30/d6c/d98/d5e/dbe/c131 0 2024-03-21T10:09:11.029 INFO:tasks.workunit.client.0.smithi153.stdout:0/967: creat d4/d4c/d83/d22/d61/f132 x:0 0 0 2024-03-21T10:09:11.039 INFO:tasks.workunit.client.1.smithi174.stdout:5/894: creat d1/dca/f140 x:0 0 0 2024-03-21T10:09:11.040 INFO:tasks.workunit.client.1.smithi174.stdout:2/830: fsync d5/d25/d27/d2d/d45/f6e 0 2024-03-21T10:09:11.040 INFO:tasks.workunit.client.0.smithi153.stdout:0/968: dwrite d4/f28 [0,4194304] 0 2024-03-21T10:09:11.041 INFO:tasks.workunit.client.1.smithi174.stdout:7/824: truncate d0/d4/d10/d79/f83 2698229 0 2024-03-21T10:09:11.042 INFO:tasks.workunit.client.1.smithi174.stdout:3/857: creat d0/d12/d14/da9/de7/db7/f107 x:0 0 0 2024-03-21T10:09:11.043 INFO:tasks.workunit.client.1.smithi174.stdout:8/873: mknod d9/d2a/d4d/db4/c11f 0 2024-03-21T10:09:11.047 INFO:tasks.workunit.client.1.smithi174.stdout:3/858: read d0/fe4 [1077515,107169] 0 2024-03-21T10:09:11.047 INFO:tasks.workunit.client.1.smithi174.stdout:5/895: creat d1/d49/f141 x:0 0 0 2024-03-21T10:09:11.051 INFO:tasks.workunit.client.1.smithi174.stdout:6/787: symlink d0/d3/d5/lfa 0 2024-03-21T10:09:11.051 INFO:tasks.workunit.client.1.smithi174.stdout:7/825: creat d0/d4/d38/f128 x:0 0 0 2024-03-21T10:09:11.053 INFO:tasks.workunit.client.1.smithi174.stdout:0/905: mkdir d9/d21/d23/da5/ddf/d12e/d13c 0 2024-03-21T10:09:11.056 INFO:tasks.workunit.client.1.smithi174.stdout:4/962: creat d4/d21/d24/d28/f141 x:0 0 0 2024-03-21T10:09:11.057 INFO:tasks.workunit.client.1.smithi174.stdout:5/896: mkdir d1/d16/d2e/d66/d8b/d85/d142 0 2024-03-21T10:09:11.059 INFO:tasks.workunit.client.1.smithi174.stdout:2/831: mkdir d5/d25/d109/d121 0 2024-03-21T10:09:11.062 INFO:tasks.workunit.client.1.smithi174.stdout:8/874: mknod d9/d53/d5b/d10a/c120 0 2024-03-21T10:09:11.063 INFO:tasks.workunit.client.1.smithi174.stdout:6/788: chown d0/d3/d5/d21/le6 23106 1 2024-03-21T10:09:11.064 INFO:tasks.workunit.client.1.smithi174.stdout:7/826: creat d0/d4/d10/d1b/d86/df0/f129 x:0 0 0 2024-03-21T10:09:11.065 INFO:tasks.workunit.client.1.smithi174.stdout:9/911: getdents d2/d11/d1d/d8b 0 2024-03-21T10:09:11.066 INFO:tasks.workunit.client.1.smithi174.stdout:6/789: stat d0/d3/d5/df/d16/da9/db7/dc2/dda/ce8 0 2024-03-21T10:09:11.067 INFO:tasks.workunit.client.1.smithi174.stdout:5/897: creat d1/d16/d1c/d81/f143 x:0 0 0 2024-03-21T10:09:11.068 INFO:tasks.workunit.client.1.smithi174.stdout:4/963: symlink d4/d21/d24/d28/de9/d3a/dd2/d103/l142 0 2024-03-21T10:09:11.070 INFO:tasks.workunit.client.1.smithi174.stdout:5/898: write d1/d16/d2e/d8d/d99/faf [7181621,4532] 0 2024-03-21T10:09:11.073 INFO:tasks.workunit.client.1.smithi174.stdout:9/912: rmdir d2/d24/d2e/df6 39 2024-03-21T10:09:11.079 INFO:tasks.workunit.client.1.smithi174.stdout:0/906: getdents d9/d35/d4a/d67 0 2024-03-21T10:09:11.082 INFO:tasks.workunit.client.1.smithi174.stdout:0/907: write d9/d1c/f20 [6757874,128638] 0 2024-03-21T10:09:11.082 INFO:tasks.workunit.client.1.smithi174.stdout:9/913: mknod d2/d10b/d112/d115/c13f 0 2024-03-21T10:09:11.088 INFO:tasks.workunit.client.1.smithi174.stdout:8/875: link d9/d2a/d39/d76/d19/d27/f64 d9/d2a/d39/d76/d19/d27/d60/daa/df2/f121 0 2024-03-21T10:09:11.088 INFO:tasks.workunit.client.1.smithi174.stdout:9/914: truncate d2/d24/d2e/f109 1976195 0 2024-03-21T10:09:11.095 INFO:tasks.workunit.client.1.smithi174.stdout:5/899: dwrite d1/d16/d2e/d66/fbe [0,4194304] 0 2024-03-21T10:09:11.098 INFO:tasks.workunit.client.1.smithi174.stdout:4/964: link d4/d21/d24/d28/d79/lb3 d4/d21/d24/d28/de9/d3a/dd2/d103/l143 0 2024-03-21T10:09:11.101 INFO:tasks.workunit.client.1.smithi174.stdout:7/827: dread d0/f11 [0,4194304] 0 2024-03-21T10:09:11.101 INFO:tasks.workunit.client.1.smithi174.stdout:9/915: mkdir d2/d24/d140 0 2024-03-21T10:09:11.103 INFO:tasks.workunit.client.1.smithi174.stdout:8/876: creat d9/d53/f122 x:0 0 0 2024-03-21T10:09:11.105 INFO:tasks.workunit.client.1.smithi174.stdout:5/900: chown d1/d5c/d73/de3/fc3 151737410 1 2024-03-21T10:09:11.106 INFO:tasks.workunit.client.1.smithi174.stdout:6/790: dread d0/d3/d2a/d56/d5e/f98 [0,4194304] 0 2024-03-21T10:09:11.108 INFO:tasks.workunit.client.1.smithi174.stdout:4/965: mkdir d4/d21/d24/d28/de9/d3a/d7a/d9d/d11f/d120/d144 0 2024-03-21T10:09:11.108 INFO:tasks.workunit.client.1.smithi174.stdout:9/916: creat d2/d11/d1d/d2a/f141 x:0 0 0 2024-03-21T10:09:11.108 INFO:tasks.workunit.client.1.smithi174.stdout:8/877: dread - d9/d2a/d39/d58/f100 zero size 2024-03-21T10:09:11.109 INFO:tasks.workunit.client.1.smithi174.stdout:5/901: fdatasync d1/d2f/d37/fe7 0 2024-03-21T10:09:11.109 INFO:tasks.workunit.client.1.smithi174.stdout:0/908: dwrite d9/d21/d31/d5c/d92/fdb [0,4194304] 0 2024-03-21T10:09:11.113 INFO:tasks.workunit.client.1.smithi174.stdout:8/878: fdatasync d9/d53/dc9/f111 0 2024-03-21T10:09:11.113 INFO:tasks.workunit.client.1.smithi174.stdout:5/902: dread - d1/d49/f141 zero size 2024-03-21T10:09:11.115 INFO:tasks.workunit.client.1.smithi174.stdout:9/917: symlink d2/d11/d1d/dab/l142 0 2024-03-21T10:09:11.117 INFO:tasks.workunit.client.1.smithi174.stdout:7/828: write d0/d4/d10/d1b/d86/dea/df4/f111 [40613,74910] 0 2024-03-21T10:09:11.117 INFO:tasks.workunit.client.1.smithi174.stdout:0/909: unlink d9/d21/c90 0 2024-03-21T10:09:11.120 INFO:tasks.workunit.client.1.smithi174.stdout:6/791: symlink d0/d3/d2a/d8f/lfb 0 2024-03-21T10:09:11.120 INFO:tasks.workunit.client.1.smithi174.stdout:3/859: dwrite d0/d12/d14/d1a/fc2 [0,4194304] 0 2024-03-21T10:09:11.121 INFO:tasks.workunit.client.1.smithi174.stdout:5/903: chown d1/d49/d93/lb8 1807272 1 2024-03-21T10:09:11.123 INFO:tasks.workunit.client.1.smithi174.stdout:7/829: mkdir d0/d9e/d12a 0 2024-03-21T10:09:11.126 INFO:tasks.workunit.client.1.smithi174.stdout:5/904: read - d1/dca/f140 zero size 2024-03-21T10:09:11.130 INFO:tasks.workunit.client.1.smithi174.stdout:3/860: read - d0/d12/d14/db9/ffb zero size 2024-03-21T10:09:11.131 INFO:tasks.workunit.client.1.smithi174.stdout:9/918: mkdir d2/d24/d143 0 2024-03-21T10:09:11.134 INFO:tasks.workunit.client.1.smithi174.stdout:5/905: write d1/d49/d4f/d51/d8f/d103/f133 [168904,114805] 0 2024-03-21T10:09:11.136 INFO:tasks.workunit.client.1.smithi174.stdout:3/861: write d0/fa [2245832,85531] 0 2024-03-21T10:09:11.136 INFO:tasks.workunit.client.1.smithi174.stdout:8/879: symlink d9/d2a/d39/d76/d19/d27/d60/daa/l123 0 2024-03-21T10:09:11.138 INFO:tasks.workunit.client.1.smithi174.stdout:6/792: symlink d0/d3/d2a/d56/d5e/d91/df0/lfc 0 2024-03-21T10:09:11.140 INFO:tasks.workunit.client.1.smithi174.stdout:9/919: dread - d2/d10b/fea zero size 2024-03-21T10:09:11.144 INFO:tasks.workunit.client.1.smithi174.stdout:5/906: fsync d1/d2f/dc1/fe8 0 2024-03-21T10:09:11.147 INFO:tasks.workunit.client.1.smithi174.stdout:5/907: dread - d1/d101/f124 zero size 2024-03-21T10:09:11.147 INFO:tasks.workunit.client.1.smithi174.stdout:8/880: rmdir d9/dc6 39 2024-03-21T10:09:11.152 INFO:tasks.workunit.client.1.smithi174.stdout:8/881: write d9/d2a/d39/d76/d19/d27/f38 [2200954,117107] 0 2024-03-21T10:09:11.177 INFO:tasks.workunit.client.1.smithi174.stdout:4/966: dread d4/d21/d24/d28/de9/d6a/d8f/f94 [0,4194304] 0 2024-03-21T10:09:11.182 INFO:tasks.workunit.client.1.smithi174.stdout:0/910: dread d9/d21/d31/f4c [0,4194304] 0 2024-03-21T10:09:11.182 INFO:tasks.workunit.client.1.smithi174.stdout:8/882: creat d9/d2a/d39/d58/dbb/f124 x:0 0 0 2024-03-21T10:09:11.187 INFO:tasks.workunit.client.1.smithi174.stdout:2/832: dwrite d5/d25/d27/d2d/d45/dae/dd6/d90/fc3 [0,4194304] 0 2024-03-21T10:09:11.189 INFO:tasks.workunit.client.1.smithi174.stdout:0/911: chown d9/d35/d4a/d8a 2040 1 2024-03-21T10:09:11.191 INFO:tasks.workunit.client.1.smithi174.stdout:9/920: dread d2/d11/d1d/f96 [4194304,4194304] 0 2024-03-21T10:09:11.196 INFO:tasks.workunit.client.1.smithi174.stdout:6/793: write d0/d3/d5/df/d16/f34 [1067258,31065] 0 2024-03-21T10:09:11.209 INFO:tasks.workunit.client.1.smithi174.stdout:9/921: creat d2/d11/d60/db2/f144 x:0 0 0 2024-03-21T10:09:11.210 INFO:tasks.workunit.client.1.smithi174.stdout:5/908: dread f0 [0,4194304] 0 2024-03-21T10:09:11.211 INFO:tasks.workunit.client.1.smithi174.stdout:7/830: dwrite d0/d4/f12 [0,4194304] 0 2024-03-21T10:09:11.215 INFO:tasks.workunit.client.1.smithi174.stdout:2/833: write d5/d25/d27/d2d/d45/d5b/f23 [9204082,50858] 0 2024-03-21T10:09:11.224 INFO:tasks.workunit.client.1.smithi174.stdout:2/834: fdatasync d5/d25/d27/d2d/d45/d5b/d2f/f112 0 2024-03-21T10:09:11.227 INFO:tasks.workunit.client.1.smithi174.stdout:6/794: rename d0/d3/d5/fd4 to d0/d3/d20/d3d/ffd 0 2024-03-21T10:09:11.227 INFO:tasks.workunit.client.1.smithi174.stdout:3/862: dwrite d0/d12/d14/d1a/d44/d94/f9d [0,4194304] 0 2024-03-21T10:09:11.234 INFO:tasks.workunit.client.1.smithi174.stdout:8/883: dread f5 [0,4194304] 0 2024-03-21T10:09:11.236 INFO:tasks.workunit.client.1.smithi174.stdout:4/967: dwrite d4/d21/fc8 [4194304,4194304] 0 2024-03-21T10:09:11.238 INFO:tasks.workunit.client.1.smithi174.stdout:3/863: chown d0/d12/d14/d1a/d44/d56/d76/cb5 22196109 1 2024-03-21T10:09:11.243 INFO:tasks.workunit.client.1.smithi174.stdout:3/864: write d0/fb [5169819,65410] 0 2024-03-21T10:09:11.243 INFO:tasks.workunit.client.1.smithi174.stdout:7/831: dread - d0/d4/d10/d1b/d86/fc2 zero size 2024-03-21T10:09:11.243 INFO:tasks.workunit.client.1.smithi174.stdout:9/922: write d2/d11/d60/db2/ffd [25475,101729] 0 2024-03-21T10:09:11.250 INFO:tasks.workunit.client.1.smithi174.stdout:5/909: dread d1/d16/d2e/d66/d8b/d85/fcb [0,4194304] 0 2024-03-21T10:09:11.257 INFO:tasks.workunit.client.1.smithi174.stdout:5/910: chown d1/d2f/d37/d9b/ff0 530694099 1 2024-03-21T10:09:11.260 INFO:tasks.workunit.client.1.smithi174.stdout:9/923: readlink d2/l2b 0 2024-03-21T10:09:11.265 INFO:tasks.workunit.client.1.smithi174.stdout:7/832: dread d0/f100 [0,4194304] 0 2024-03-21T10:09:11.265 INFO:tasks.workunit.client.1.smithi174.stdout:4/968: fsync d4/d19/f23 0 2024-03-21T10:09:11.269 INFO:tasks.workunit.client.1.smithi174.stdout:5/911: mkdir d1/d2f/dc1/d144 0 2024-03-21T10:09:11.269 INFO:tasks.workunit.client.1.smithi174.stdout:6/795: dwrite d0/d3/d5/f45 [0,4194304] 0 2024-03-21T10:09:11.270 INFO:tasks.workunit.client.1.smithi174.stdout:3/865: symlink d0/d12/d14/d1a/d27/l108 0 2024-03-21T10:09:11.274 INFO:tasks.workunit.client.1.smithi174.stdout:9/924: fdatasync d2/d24/d2e/d63/f6c 0 2024-03-21T10:09:11.281 INFO:tasks.workunit.client.1.smithi174.stdout:2/835: link d5/d25/d27/d2d/d45/dae/dd6/d4f/d65/d77/fdf d5/d25/d27/d2d/d45/dae/dd6/d85/dad/f122 0 2024-03-21T10:09:11.282 INFO:tasks.workunit.client.1.smithi174.stdout:9/925: read d2/d24/f88 [2121890,39171] 0 2024-03-21T10:09:11.296 INFO:tasks.workunit.client.1.smithi174.stdout:8/884: getdents d9/d2a/d39/d76/d19/d27/d60/daa 0 2024-03-21T10:09:11.300 INFO:tasks.workunit.client.1.smithi174.stdout:4/969: symlink d4/d21/d24/d28/de9/d3a/d77/dcf/d140/l145 0 2024-03-21T10:09:11.304 INFO:tasks.workunit.client.1.smithi174.stdout:3/866: dwrite d0/d12/d14/da9/de7/db7/d7d/fef [0,4194304] 0 2024-03-21T10:09:11.304 INFO:tasks.workunit.client.1.smithi174.stdout:5/912: creat d1/d2f/d37/d137/f145 x:0 0 0 2024-03-21T10:09:11.307 INFO:tasks.workunit.client.1.smithi174.stdout:2/836: mkdir d5/d25/d27/d2d/d45/dae/dd6/d90/da9/d123 0 2024-03-21T10:09:11.313 INFO:tasks.workunit.client.1.smithi174.stdout:4/970: write d4/d21/d24/d28/de9/d3a/d7a/d9d/d11f/f122 [941285,79693] 0 2024-03-21T10:09:11.316 INFO:tasks.workunit.client.1.smithi174.stdout:7/833: dwrite d0/d4/d10/d1b/d86/dd2/f114 [0,4194304] 0 2024-03-21T10:09:11.320 INFO:tasks.workunit.client.1.smithi174.stdout:6/796: dwrite d0/d3/d20/d6d/fb9 [0,4194304] 0 2024-03-21T10:09:11.321 INFO:tasks.workunit.client.1.smithi174.stdout:5/913: mknod d1/d130/c146 0 2024-03-21T10:09:11.323 INFO:tasks.workunit.client.1.smithi174.stdout:2/837: unlink d5/d25/d27/f3c 0 2024-03-21T10:09:11.324 INFO:tasks.workunit.client.1.smithi174.stdout:3/867: rename d0/d12/d46/df9/f72 to d0/d12/d14/d1a/d44/d94/dd9/f109 0 2024-03-21T10:09:11.327 INFO:tasks.workunit.client.1.smithi174.stdout:7/834: fdatasync d0/d4/d38/da9/dc6/fca 0 2024-03-21T10:09:11.328 INFO:tasks.workunit.client.1.smithi174.stdout:6/797: mkdir d0/d3/d5/d6/dfe 0 2024-03-21T10:09:11.328 INFO:tasks.workunit.client.1.smithi174.stdout:3/868: read - d0/d12/d14/d1a/d27/ff8 zero size 2024-03-21T10:09:11.338 INFO:tasks.workunit.client.1.smithi174.stdout:4/971: symlink d4/d21/d24/d28/de9/d3a/dd2/l146 0 2024-03-21T10:09:11.339 INFO:tasks.workunit.client.1.smithi174.stdout:2/838: readlink d5/d25/d27/d2d/d45/dae/dd6/l89 0 2024-03-21T10:09:11.345 INFO:tasks.workunit.client.1.smithi174.stdout:6/798: rmdir d0/d3/d5/df 39 2024-03-21T10:09:11.348 INFO:tasks.workunit.client.1.smithi174.stdout:8/885: getdents d9/dc6/dd1 0 2024-03-21T10:09:11.351 INFO:tasks.workunit.client.1.smithi174.stdout:5/914: dread d1/d16/d2e/d87/fcf [0,4194304] 0 2024-03-21T10:09:11.352 INFO:tasks.workunit.client.1.smithi174.stdout:7/835: creat d0/d4/d121/f12b x:0 0 0 2024-03-21T10:09:11.352 INFO:tasks.workunit.client.1.smithi174.stdout:6/799: unlink d0/d3/d20/f7f 0 2024-03-21T10:09:11.355 INFO:tasks.workunit.client.1.smithi174.stdout:2/839: write d5/d25/d27/d2d/d45/d5b/d63/f114 [541516,27533] 0 2024-03-21T10:09:11.355 INFO:tasks.workunit.client.1.smithi174.stdout:8/886: chown d9/d2a/la1 1099555812 1 2024-03-21T10:09:11.355 INFO:tasks.workunit.client.1.smithi174.stdout:9/926: dread d2/d11/d1d/d2a/da7/fe9 [0,4194304] 0 2024-03-21T10:09:11.358 INFO:tasks.workunit.client.1.smithi174.stdout:3/869: dwrite d0/d12/f49 [0,4194304] 0 2024-03-21T10:09:11.366 INFO:tasks.workunit.client.1.smithi174.stdout:2/840: unlink d5/d25/d27/f11c 0 2024-03-21T10:09:11.367 INFO:tasks.workunit.client.1.smithi174.stdout:6/800: mknod d0/d3/d5/df/cff 0 2024-03-21T10:09:11.367 INFO:tasks.workunit.client.1.smithi174.stdout:3/870: mknod d0/d12/d14/d98/c10a 0 2024-03-21T10:09:11.367 INFO:tasks.workunit.client.1.smithi174.stdout:9/927: chown d2/d11/d1d/d2a/c3b 15410 1 2024-03-21T10:09:11.374 INFO:tasks.workunit.client.0.smithi153.stdout:0/969: sync 2024-03-21T10:09:11.374 INFO:tasks.workunit.client.0.smithi153.stdout:1/972: sync 2024-03-21T10:09:11.375 INFO:tasks.workunit.client.1.smithi174.stdout:2/841: creat d5/d25/d27/d2d/d45/dae/dd6/d4f/da0/def/f124 x:0 0 0 2024-03-21T10:09:11.376 INFO:tasks.workunit.client.0.smithi153.stdout:0/970: stat d4/db/f8b 0 2024-03-21T10:09:11.376 INFO:tasks.workunit.client.1.smithi174.stdout:9/928: creat d2/d10b/d112/d115/f145 x:0 0 0 2024-03-21T10:09:11.378 INFO:tasks.workunit.client.1.smithi174.stdout:8/887: link d9/d2a/d39/d76/c98 d9/dc6/dd1/c125 0 2024-03-21T10:09:11.379 INFO:tasks.workunit.client.0.smithi153.stdout:1/973: dread - d6/d10/d35/da5/d4f/d9d/db6/f10a zero size 2024-03-21T10:09:11.382 INFO:tasks.workunit.client.1.smithi174.stdout:9/929: truncate d2/d11/f90 977680 0 2024-03-21T10:09:11.385 INFO:tasks.workunit.client.1.smithi174.stdout:6/801: creat d0/d3/d2a/d56/d5e/f100 x:0 0 0 2024-03-21T10:09:11.386 INFO:tasks.workunit.client.1.smithi174.stdout:9/930: rmdir d2/d11/d1d 39 2024-03-21T10:09:11.391 INFO:tasks.workunit.client.1.smithi174.stdout:6/802: truncate d0/d3/d5/df/d16/da9/fd0 711929 0 2024-03-21T10:09:11.391 INFO:tasks.workunit.client.1.smithi174.stdout:4/972: dread d4/d21/d24/ff1 [0,4194304] 0 2024-03-21T10:09:11.392 INFO:tasks.workunit.client.1.smithi174.stdout:9/931: unlink d2/d24/d2e/f109 0 2024-03-21T10:09:11.395 INFO:tasks.workunit.client.1.smithi174.stdout:4/973: dread - d4/d21/d24/d28/de9/d3a/d67/d76/fdb zero size 2024-03-21T10:09:11.396 INFO:tasks.workunit.client.1.smithi174.stdout:3/871: write d0/d12/f17 [1258312,85221] 0 2024-03-21T10:09:11.398 INFO:tasks.workunit.client.1.smithi174.stdout:8/888: dwrite d9/d2a/d39/de1/df7/f114 [0,4194304] 0 2024-03-21T10:09:11.403 INFO:tasks.workunit.client.0.smithi153.stdout:0/971: write d4/d16/d19/d30/f41 [1672892,53612] 0 2024-03-21T10:09:11.409 INFO:tasks.workunit.client.1.smithi174.stdout:3/872: rmdir d0/d12/d14/d1a/d44/d94 39 2024-03-21T10:09:11.415 INFO:tasks.workunit.client.1.smithi174.stdout:3/873: creat d0/d12/d14/da9/de7/db7/f10b x:0 0 0 2024-03-21T10:09:11.417 INFO:tasks.workunit.client.1.smithi174.stdout:9/932: dwrite d2/d10a/f13b [0,4194304] 0 2024-03-21T10:09:11.423 INFO:tasks.workunit.client.1.smithi174.stdout:4/974: dwrite d4/d21/d24/d28/de9/d3a/f113 [0,4194304] 0 2024-03-21T10:09:11.426 INFO:tasks.workunit.client.1.smithi174.stdout:3/874: symlink d0/d12/d14/d1a/d44/d65/l10c 0 2024-03-21T10:09:11.427 INFO:tasks.workunit.client.1.smithi174.stdout:9/933: mknod d2/d11/d1d/dab/dc1/d124/c146 0 2024-03-21T10:09:11.429 INFO:tasks.workunit.client.0.smithi153.stdout:0/972: dread d4/f10 [0,4194304] 0 2024-03-21T10:09:11.434 INFO:tasks.workunit.client.0.smithi153.stdout:0/973: mknod d4/d16/d19/c133 0 2024-03-21T10:09:11.435 INFO:tasks.workunit.client.1.smithi174.stdout:9/934: symlink d2/d11/d1d/d2a/df2/l147 0 2024-03-21T10:09:11.435 INFO:tasks.workunit.client.1.smithi174.stdout:3/875: link d0/d12/d14/d67/dc8/cdd d0/d12/d14/da9/de7/db4/c10d 0 2024-03-21T10:09:11.437 INFO:tasks.workunit.client.1.smithi174.stdout:9/935: readlink d2/d11/d1d/dab/dc1/lae 0 2024-03-21T10:09:11.438 INFO:tasks.workunit.client.1.smithi174.stdout:0/912: sync 2024-03-21T10:09:11.439 INFO:tasks.workunit.client.1.smithi174.stdout:2/842: dread d5/d25/d27/d2d/d45/dae/dd6/d7b/d81/fd7 [0,4194304] 0 2024-03-21T10:09:11.442 INFO:tasks.workunit.client.1.smithi174.stdout:3/876: truncate d0/d12/d46/d59/fb2 799696 0 2024-03-21T10:09:11.442 INFO:tasks.workunit.client.1.smithi174.stdout:4/975: dwrite d4/fa [0,4194304] 0 2024-03-21T10:09:11.442 INFO:tasks.workunit.client.1.smithi174.stdout:3/877: chown d0/d12/lf3 38 1 2024-03-21T10:09:11.451 INFO:tasks.workunit.client.1.smithi174.stdout:0/913: creat d9/d21/d31/d5c/d86/f13d x:0 0 0 2024-03-21T10:09:11.452 INFO:tasks.workunit.client.1.smithi174.stdout:3/878: rename d0/cea to d0/d12/d14/da9/de7/db4/c10e 0 2024-03-21T10:09:11.455 INFO:tasks.workunit.client.1.smithi174.stdout:0/914: chown d9/d21/d31/d5c/led 65681114 1 2024-03-21T10:09:11.456 INFO:tasks.workunit.client.1.smithi174.stdout:2/843: rename d5/d25/d27/d2d/db4/f11a to d5/d25/d27/d2d/d45/dae/f125 0 2024-03-21T10:09:11.460 INFO:tasks.workunit.client.1.smithi174.stdout:0/915: creat d9/d21/d31/d9f/d79/d82/dbb/f13e x:0 0 0 2024-03-21T10:09:11.461 INFO:tasks.workunit.client.1.smithi174.stdout:3/879: truncate d0/d12/d14/d1a/d44/d65/f6b 2072183 0 2024-03-21T10:09:11.464 INFO:tasks.workunit.client.1.smithi174.stdout:2/844: rename d5/d25/d27/f80 to d5/d25/d27/d2d/d45/dae/dd6/d4f/d65/d77/dcf/f126 0 2024-03-21T10:09:11.464 INFO:tasks.workunit.client.1.smithi174.stdout:0/916: chown d9/d21/d31/d5c/d92/d98/db7/ddc/c109 236 1 2024-03-21T10:09:11.468 INFO:tasks.workunit.client.1.smithi174.stdout:0/917: creat d9/d21/d31/d5c/d86/f13f x:0 0 0 2024-03-21T10:09:11.470 INFO:tasks.workunit.client.1.smithi174.stdout:2/845: symlink d5/d25/d27/d2d/d45/d5b/d2f/d4b/d76/l127 0 2024-03-21T10:09:11.472 INFO:tasks.workunit.client.1.smithi174.stdout:4/976: write d4/f3e [1907127,100833] 0 2024-03-21T10:09:11.479 INFO:tasks.workunit.client.1.smithi174.stdout:2/846: link d5/d25/d27/d2d/d45/dae/dd6/d90/f118 d5/d25/d27/d2d/d45/d5b/d63/de1/f128 0 2024-03-21T10:09:11.481 INFO:tasks.workunit.client.1.smithi174.stdout:2/847: dread - d5/d25/d27/d2d/d45/dae/dd6/d4f/d65/fc7 zero size 2024-03-21T10:09:11.488 INFO:tasks.workunit.client.0.smithi153.stdout:0/974: dwrite d4/d16/d19/d58/de9/f10e [0,4194304] 0 2024-03-21T10:09:11.489 INFO:tasks.workunit.client.0.smithi153.stdout:0/975: chown d4 51 1 2024-03-21T10:09:11.490 INFO:tasks.workunit.client.1.smithi174.stdout:3/880: dread d0/d12/d14/da9/de7/db7/f38 [0,4194304] 0 2024-03-21T10:09:11.491 INFO:tasks.workunit.client.0.smithi153.stdout:0/976: write d4/d16/d19/fc3 [1308503,47774] 0 2024-03-21T10:09:11.492 INFO:tasks.workunit.client.1.smithi174.stdout:3/881: chown d0/d12/d14/f4e 766 1 2024-03-21T10:09:11.493 INFO:tasks.workunit.client.1.smithi174.stdout:4/977: dread d4/d21/d24/d28/de9/d3a/d7a/d9d/fc7 [0,4194304] 0 2024-03-21T10:09:11.494 INFO:tasks.workunit.client.0.smithi153.stdout:0/977: symlink d4/d16/d19/d95/l134 0 2024-03-21T10:09:11.496 INFO:tasks.workunit.client.1.smithi174.stdout:2/848: dread d5/d25/d27/f8b [0,4194304] 0 2024-03-21T10:09:11.498 INFO:tasks.workunit.client.1.smithi174.stdout:3/882: creat d0/d12/d14/d1a/d44/d65/f10f x:0 0 0 2024-03-21T10:09:11.501 INFO:tasks.workunit.client.1.smithi174.stdout:3/883: truncate d0/d12/d14/d1a/f40 311782 0 2024-03-21T10:09:11.503 INFO:tasks.workunit.client.1.smithi174.stdout:3/884: chown d0/d12/d14/d1a/d27/d7f/fcc 2819 1 2024-03-21T10:09:11.509 INFO:tasks.workunit.client.0.smithi153.stdout:0/978: dread d4/d16/d19/d30/d6c/d98/d5e/dbe/fdf [0,4194304] 0 2024-03-21T10:09:11.512 INFO:tasks.workunit.client.0.smithi153.stdout:0/979: creat d4/d4c/f135 x:0 0 0 2024-03-21T10:09:11.513 INFO:tasks.workunit.client.0.smithi153.stdout:0/980: chown d4/d16/d27/f79 1435164 1 2024-03-21T10:09:11.513 INFO:tasks.workunit.client.1.smithi174.stdout:2/849: write d5/d25/d27/d2d/d45/d5b/d55/fda [990351,63443] 0 2024-03-21T10:09:11.514 INFO:tasks.workunit.client.1.smithi174.stdout:2/850: readlink d5/d25/l5a 0 2024-03-21T10:09:11.515 INFO:tasks.workunit.client.1.smithi174.stdout:3/885: dwrite d0/f7 [0,4194304] 0 2024-03-21T10:09:11.517 INFO:tasks.workunit.client.0.smithi153.stdout:0/981: truncate d4/d16/d19/d30/d6c/d98/df3/ff0 1456707 0 2024-03-21T10:09:11.522 INFO:tasks.workunit.client.0.smithi153.stdout:0/982: creat d4/d16/d19/d95/f136 x:0 0 0 2024-03-21T10:09:11.524 INFO:tasks.workunit.client.1.smithi174.stdout:3/886: dread d0/d12/d14/d1a/d27/d4f/d64/fd2 [0,4194304] 0 2024-03-21T10:09:11.525 INFO:tasks.workunit.client.1.smithi174.stdout:2/851: write d5/f26 [992617,91919] 0 2024-03-21T10:09:11.526 INFO:tasks.workunit.client.0.smithi153.stdout:0/983: mknod d4/d16/d19/d30/d6c/df4/c137 0 2024-03-21T10:09:11.529 INFO:tasks.workunit.client.0.smithi153.stdout:0/984: mkdir d4/d16/d19/d30/d138 0 2024-03-21T10:09:11.530 INFO:tasks.workunit.client.1.smithi174.stdout:3/887: getdents d0 0 2024-03-21T10:09:11.531 INFO:tasks.workunit.client.0.smithi153.stdout:0/985: read d4/f103 [3701962,74189] 0 2024-03-21T10:09:11.538 INFO:tasks.workunit.client.1.smithi174.stdout:4/978: dread d4/d19/fa4 [0,4194304] 0 2024-03-21T10:09:11.544 INFO:tasks.workunit.client.1.smithi174.stdout:2/852: dwrite d5/d25/d27/d2d/d45/dae/dd6/d85/dad/f101 [0,4194304] 0 2024-03-21T10:09:11.551 INFO:tasks.workunit.client.1.smithi174.stdout:3/888: dwrite d0/d12/d14/d1a/d5c/f8e [0,4194304] 0 2024-03-21T10:09:11.555 INFO:tasks.workunit.client.1.smithi174.stdout:3/889: write d0/d12/f49 [3775448,122878] 0 2024-03-21T10:09:11.558 INFO:tasks.workunit.client.1.smithi174.stdout:3/890: fdatasync d0/f5f 0 2024-03-21T10:09:11.559 INFO:tasks.workunit.client.0.smithi153.stdout:0/986: dread d4/d4c/d83/d22/d61/fe0 [0,4194304] 0 2024-03-21T10:09:11.564 INFO:tasks.workunit.client.1.smithi174.stdout:4/979: dwrite d4/d21/d24/d28/f4e [0,4194304] 0 2024-03-21T10:09:11.570 INFO:tasks.workunit.client.1.smithi174.stdout:2/853: dwrite d5/d25/f7e [0,4194304] 0 2024-03-21T10:09:11.577 INFO:tasks.workunit.client.1.smithi174.stdout:3/891: getdents d0/d12/d14/d1a/d44/d65/d7e/dac 0 2024-03-21T10:09:11.579 INFO:tasks.workunit.client.1.smithi174.stdout:3/892: truncate d0/d12/d14/d67/fe6 323000 0 2024-03-21T10:09:11.586 INFO:tasks.workunit.client.0.smithi153.stdout:0/987: dwrite d4/d16/d27/f79 [0,4194304] 0 2024-03-21T10:09:11.589 INFO:tasks.workunit.client.1.smithi174.stdout:3/893: dread d0/fa [0,4194304] 0 2024-03-21T10:09:11.600 INFO:tasks.workunit.client.1.smithi174.stdout:2/854: write d5/d25/d27/f3b [8439452,49400] 0 2024-03-21T10:09:11.600 INFO:tasks.workunit.client.0.smithi153.stdout:0/988: dread d4/d16/d19/d30/d6c/d98/d5e/dbe/fdf [0,4194304] 0 2024-03-21T10:09:11.604 INFO:tasks.workunit.client.0.smithi153.stdout:0/989: symlink d4/d16/d19/d58/l139 0 2024-03-21T10:09:11.606 INFO:tasks.workunit.client.1.smithi174.stdout:2/855: creat d5/d25/d27/d2d/d45/dae/dd6/d4f/d65/d77/dcf/f129 x:0 0 0 2024-03-21T10:09:11.607 INFO:tasks.workunit.client.0.smithi153.stdout:0/990: write d4/db/f29 [8993525,63645] 0 2024-03-21T10:09:11.607 INFO:tasks.workunit.client.1.smithi174.stdout:4/980: dwrite d4/d21/d24/d28/de9/d3a/d67/d76/fdb [0,4194304] 0 2024-03-21T10:09:11.609 INFO:tasks.workunit.client.0.smithi153.stdout:1/974: sync 2024-03-21T10:09:11.610 INFO:tasks.workunit.client.1.smithi174.stdout:2/856: stat d5/d25/d27/d2d/d45/dae/dd6/d7b/f99 0 2024-03-21T10:09:11.613 INFO:tasks.workunit.client.0.smithi153.stdout:0/991: symlink d4/d4c/d83/d22/d61/d77/l13a 0 2024-03-21T10:09:11.613 INFO:tasks.workunit.client.1.smithi174.stdout:2/857: creat d5/d25/d27/d2d/d45/d5b/d55/f12a x:0 0 0 2024-03-21T10:09:11.618 INFO:tasks.workunit.client.1.smithi174.stdout:2/858: mkdir d5/d25/d27/d2d/d45/d5b/d2f/d66/db5/d12b 0 2024-03-21T10:09:11.619 INFO:tasks.workunit.client.0.smithi153.stdout:1/975: dread d6/d10/d35/da5/d4f/d9d/db6/def/ff3 [0,4194304] 0 2024-03-21T10:09:11.620 INFO:tasks.workunit.client.1.smithi174.stdout:2/859: creat d5/d25/d109/f12c x:0 0 0 2024-03-21T10:09:11.621 INFO:tasks.workunit.client.0.smithi153.stdout:1/976: readlink d6/d10/d35/da5/l12d 0 2024-03-21T10:09:11.626 INFO:tasks.workunit.client.1.smithi174.stdout:2/860: mkdir d5/d25/d27/d2d/d45/dae/dd6/d4f/d65/d12d 0 2024-03-21T10:09:11.626 INFO:tasks.workunit.client.1.smithi174.stdout:4/981: dwrite d4/d21/d24/d28/f81 [0,4194304] 0 2024-03-21T10:09:11.626 INFO:tasks.workunit.client.0.smithi153.stdout:1/977: readlink d6/d10/d35/da5/d4f/d9d/db6/def/l133 0 2024-03-21T10:09:11.629 INFO:tasks.workunit.client.0.smithi153.stdout:1/978: mknod d6/d10/da0/c142 0 2024-03-21T10:09:11.630 INFO:tasks.workunit.client.1.smithi174.stdout:2/861: write d5/d25/d27/d2d/d45/d5b/d2f/d68/de8/f117 [769637,1161] 0 2024-03-21T10:09:11.634 INFO:tasks.workunit.client.1.smithi174.stdout:2/862: mknod d5/d25/d27/d2d/d79/c12e 0 2024-03-21T10:09:11.641 INFO:tasks.workunit.client.0.smithi153.stdout:0/992: dwrite d4/d16/d19/dad/fca [0,4194304] 0 2024-03-21T10:09:11.641 INFO:tasks.workunit.client.1.smithi174.stdout:4/982: write d4/d19/f74 [4899135,37183] 0 2024-03-21T10:09:11.647 INFO:tasks.workunit.client.1.smithi174.stdout:4/983: fsync d4/d21/d24/d28/de9/d6a/d8f/d9c/dfd/f112 0 2024-03-21T10:09:11.648 INFO:tasks.workunit.client.0.smithi153.stdout:0/993: getdents d4/d16/d19/dad/d100 0 2024-03-21T10:09:11.653 INFO:tasks.workunit.client.1.smithi174.stdout:2/863: dwrite d5/d25/d109/f12c [0,4194304] 0 2024-03-21T10:09:11.653 INFO:tasks.workunit.client.0.smithi153.stdout:0/994: creat d4/d16/d19/d30/d6c/df4/f13b x:0 0 0 2024-03-21T10:09:11.658 INFO:tasks.workunit.client.0.smithi153.stdout:0/995: symlink d4/d11d/l13c 0 2024-03-21T10:09:11.661 INFO:tasks.workunit.client.0.smithi153.stdout:0/996: truncate d4/d16/d19/dad/fca 4650044 0 2024-03-21T10:09:11.662 INFO:tasks.workunit.client.0.smithi153.stdout:1/979: dread d6/fb [0,4194304] 0 2024-03-21T10:09:11.663 INFO:tasks.workunit.client.0.smithi153.stdout:0/997: stat d4/d16/d19/d30/d6c/d98/d5e/f75 0 2024-03-21T10:09:11.668 INFO:tasks.workunit.client.0.smithi153.stdout:1/980: creat d6/d13/d8e/dbb/de1/de7/f143 x:0 0 0 2024-03-21T10:09:11.684 INFO:tasks.workunit.client.1.smithi174.stdout:4/984: dread d4/d21/d24/fec [0,4194304] 0 2024-03-21T10:09:11.695 INFO:tasks.workunit.client.1.smithi174.stdout:4/985: dread d4/f8 [0,4194304] 0 2024-03-21T10:09:11.698 INFO:tasks.workunit.client.0.smithi153.stdout:1/981: dread d6/f2f [0,4194304] 0 2024-03-21T10:09:11.702 INFO:tasks.workunit.client.0.smithi153.stdout:1/982: creat d6/d13/d8e/dbb/de1/de7/de9/df5/d11f/f144 x:0 0 0 2024-03-21T10:09:11.703 INFO:tasks.workunit.client.1.smithi174.stdout:4/986: truncate d4/d21/f8c 3737773 0 2024-03-21T10:09:11.704 INFO:tasks.workunit.client.0.smithi153.stdout:1/983: mkdir d6/d13/d8e/d145 0 2024-03-21T10:09:11.707 INFO:tasks.workunit.client.0.smithi153.stdout:1/984: getdents d6/d10f 0 2024-03-21T10:09:11.708 INFO:tasks.workunit.client.1.smithi174.stdout:4/987: rename d4/d21/d24/d28/de9/d3a/d7a/d9d/f104 to d4/f147 0 2024-03-21T10:09:11.710 INFO:tasks.workunit.client.1.smithi174.stdout:4/988: readlink d4/d21/lf6 0 2024-03-21T10:09:11.723 INFO:tasks.workunit.client.0.smithi153.stdout:1/985: write d6/d13/d8e/dbb/de1/de7/de9/df5/f64 [4363719,115045] 0 2024-03-21T10:09:11.726 INFO:tasks.workunit.client.0.smithi153.stdout:1/986: rmdir d6/d10/d35 39 2024-03-21T10:09:11.726 INFO:tasks.workunit.client.1.smithi174.stdout:6/803: sync 2024-03-21T10:09:11.726 INFO:tasks.workunit.client.1.smithi174.stdout:8/889: sync 2024-03-21T10:09:11.726 INFO:tasks.workunit.client.1.smithi174.stdout:5/915: sync 2024-03-21T10:09:11.726 INFO:tasks.workunit.client.1.smithi174.stdout:7/836: sync 2024-03-21T10:09:11.729 INFO:tasks.workunit.client.1.smithi174.stdout:6/804: write d0/d3/d5/f1c [463327,20512] 0 2024-03-21T10:09:11.732 INFO:tasks.workunit.client.1.smithi174.stdout:8/890: mkdir d9/d2a/d39/d58/d126 0 2024-03-21T10:09:11.732 INFO:tasks.workunit.client.1.smithi174.stdout:4/989: dread d4/d21/f27 [0,4194304] 0 2024-03-21T10:09:11.733 INFO:tasks.workunit.client.1.smithi174.stdout:7/837: fdatasync d0/d4/d38/f67 0 2024-03-21T10:09:11.734 INFO:tasks.workunit.client.0.smithi153.stdout:1/987: rename d6/d13/d8e/dbb/de1/de7/de9/df5/d50/da8 to d6/d10/d35/da5/d4f/d9d/db6/d146 0 2024-03-21T10:09:11.735 INFO:tasks.workunit.client.1.smithi174.stdout:6/805: rmdir d0/d3/d2a/d56/d5e/d91 39 2024-03-21T10:09:11.738 INFO:tasks.workunit.client.1.smithi174.stdout:6/806: fdatasync d0/d3/d20/d6d/fb9 0 2024-03-21T10:09:11.738 INFO:tasks.workunit.client.0.smithi153.stdout:1/988: creat d6/d10/d35/da5/d4f/df0/f147 x:0 0 0 2024-03-21T10:09:11.739 INFO:tasks.workunit.client.1.smithi174.stdout:8/891: fdatasync d9/dc6/dd1/fa6 0 2024-03-21T10:09:11.746 INFO:tasks.workunit.client.1.smithi174.stdout:6/807: mknod d0/d3/d2a/d56/d5e/d91/df0/c101 0 2024-03-21T10:09:11.748 INFO:tasks.workunit.client.1.smithi174.stdout:5/916: dwrite d1/d2f/dc1/fe8 [4194304,4194304] 0 2024-03-21T10:09:11.751 INFO:tasks.workunit.client.1.smithi174.stdout:7/838: dread d0/d4/f22 [0,4194304] 0 2024-03-21T10:09:11.752 INFO:tasks.workunit.client.1.smithi174.stdout:5/917: creat d1/d5c/d96/f147 x:0 0 0 2024-03-21T10:09:11.754 INFO:tasks.workunit.client.1.smithi174.stdout:8/892: getdents d9 0 2024-03-21T10:09:11.755 INFO:tasks.workunit.client.1.smithi174.stdout:6/808: read d0/d3/d20/d3d/d81/da5/fc9 [181539,49079] 0 2024-03-21T10:09:11.763 INFO:tasks.workunit.client.1.smithi174.stdout:7/839: mknod d0/d4/d10/d1b/d86/dea/c12c 0 2024-03-21T10:09:11.764 INFO:tasks.workunit.client.1.smithi174.stdout:4/990: dwrite d4/d21/d24/d28/de9/d3a/d67/d76/fbd [4194304,4194304] 0 2024-03-21T10:09:11.776 INFO:tasks.workunit.client.1.smithi174.stdout:6/809: truncate d0/d3/d2a/d56/d5e/fa7 1891001 0 2024-03-21T10:09:11.777 INFO:tasks.workunit.client.1.smithi174.stdout:7/840: mknod d0/d4/d10/d44/d81/c12d 0 2024-03-21T10:09:11.781 INFO:tasks.workunit.client.1.smithi174.stdout:5/918: dwrite d1/d16/d2e/d66/f6f [0,4194304] 0 2024-03-21T10:09:11.789 INFO:tasks.workunit.client.1.smithi174.stdout:5/919: mkdir d1/d16/d1c/d148 0 2024-03-21T10:09:11.793 INFO:tasks.workunit.client.1.smithi174.stdout:6/810: write d0/d3/da8/fd2 [2880895,23492] 0 2024-03-21T10:09:11.796 INFO:tasks.workunit.client.1.smithi174.stdout:8/893: dwrite d9/d2a/d39/f47 [4194304,4194304] 0 2024-03-21T10:09:11.796 INFO:tasks.workunit.client.1.smithi174.stdout:7/841: dread d0/d4/ffb [0,4194304] 0 2024-03-21T10:09:11.800 INFO:tasks.workunit.client.1.smithi174.stdout:4/991: dwrite d4/d21/d24/d28/de9/d6a/d8f/d9c/fb2 [0,4194304] 0 2024-03-21T10:09:11.801 INFO:tasks.workunit.client.1.smithi174.stdout:7/842: fdatasync d0/f88 0 2024-03-21T10:09:11.809 INFO:tasks.workunit.client.1.smithi174.stdout:8/894: mknod d9/d2a/d39/d58/c127 0 2024-03-21T10:09:11.809 INFO:tasks.workunit.client.1.smithi174.stdout:4/992: rename d4/d21/d24/fea to d4/d19/f148 0 2024-03-21T10:09:11.811 INFO:tasks.workunit.client.1.smithi174.stdout:7/843: symlink d0/d9e/d12a/l12e 0 2024-03-21T10:09:11.815 INFO:tasks.workunit.client.1.smithi174.stdout:7/844: readlink d0/d4/d10/d79/db2/dda/d5f/l7d 0 2024-03-21T10:09:11.820 INFO:tasks.workunit.client.1.smithi174.stdout:8/895: creat d9/d2a/d39/d58/dbb/f128 x:0 0 0 2024-03-21T10:09:11.821 INFO:tasks.workunit.client.1.smithi174.stdout:6/811: dwrite d0/d3/d20/d3d/ffd [4194304,4194304] 0 2024-03-21T10:09:11.823 INFO:tasks.workunit.client.1.smithi174.stdout:6/812: stat d0/d3/d5/d3b/f61 0 2024-03-21T10:09:11.829 INFO:tasks.workunit.client.1.smithi174.stdout:8/896: truncate d9/d2a/d39/d76/d19/d27/f64 990052 0 2024-03-21T10:09:11.833 INFO:tasks.workunit.client.1.smithi174.stdout:4/993: dwrite d4/d21/fff [0,4194304] 0 2024-03-21T10:09:11.840 INFO:tasks.workunit.client.1.smithi174.stdout:8/897: rename d9/c48 to d9/d2a/d39/d76/d19/d27/c129 0 2024-03-21T10:09:11.841 INFO:tasks.workunit.client.1.smithi174.stdout:4/994: read - d4/d21/d24/d28/de9/d6a/dab/ff3 zero size 2024-03-21T10:09:11.852 INFO:tasks.workunit.client.1.smithi174.stdout:6/813: dwrite d0/d3/d2a/d56/d5e/fa7 [0,4194304] 0 2024-03-21T10:09:11.854 INFO:tasks.workunit.client.1.smithi174.stdout:6/814: chown d0/d3/d5/d21/le6 179677 1 2024-03-21T10:09:11.856 INFO:tasks.workunit.client.1.smithi174.stdout:4/995: dread d4/d21/d24/ff1 [0,4194304] 0 2024-03-21T10:09:11.856 INFO:tasks.workunit.client.1.smithi174.stdout:6/815: fsync d0/d3/d5/fbe 0 2024-03-21T10:09:11.856 INFO:tasks.workunit.client.0.smithi153.stdout:0/998: sync 2024-03-21T10:09:11.857 INFO:tasks.workunit.client.1.smithi174.stdout:6/816: readlink d0/d3/d5/d3b/l62 0 2024-03-21T10:09:11.860 INFO:tasks.workunit.client.0.smithi153.stdout:0/999: write d4/d16/d19/d58/d9a/fb4 [2253026,60146] 0 2024-03-21T10:09:11.862 INFO:tasks.workunit.client.1.smithi174.stdout:8/898: getdents d9/d2a/d39/d76/d19 0 2024-03-21T10:09:11.862 INFO:tasks.workunit.client.1.smithi174.stdout:4/996: write d4/f15 [3612415,65314] 0 2024-03-21T10:09:11.865 INFO:tasks.workunit.client.1.smithi174.stdout:8/899: chown d9/d2a/d39/d76/d19/lf4 63688404 1 2024-03-21T10:09:11.867 INFO:tasks.workunit.client.1.smithi174.stdout:4/997: fsync d4/d21/d24/d28/de9/d3a/d77/dcf/f118 0 2024-03-21T10:09:11.868 INFO:tasks.workunit.client.1.smithi174.stdout:9/936: sync 2024-03-21T10:09:11.869 INFO:tasks.workunit.client.1.smithi174.stdout:0/918: sync 2024-03-21T10:09:11.871 INFO:tasks.workunit.client.1.smithi174.stdout:8/900: read d9/d2a/d39/d76/d19/d27/d60/f9f [2137,115890] 0 2024-03-21T10:09:11.874 INFO:tasks.workunit.client.1.smithi174.stdout:9/937: creat d2/d11/f148 x:0 0 0 2024-03-21T10:09:11.875 INFO:tasks.workunit.client.1.smithi174.stdout:4/998: rename d4/d21/lf6 to d4/d21/d24/d28/de9/d6a/dac/l149 0 2024-03-21T10:09:11.877 INFO:tasks.workunit.client.1.smithi174.stdout:6/817: dwrite d0/d3/d2a/d56/d5e/d91/fe7 [0,4194304] 0 2024-03-21T10:09:11.890 INFO:tasks.workunit.client.1.smithi174.stdout:0/919: dwrite d9/f6d [0,4194304] 0 2024-03-21T10:09:11.894 INFO:tasks.workunit.client.1.smithi174.stdout:0/920: write d9/d21/d31/d5c/d92/ffe [243900,79550] 0 2024-03-21T10:09:11.896 INFO:tasks.workunit.client.1.smithi174.stdout:4/999: dwrite d4/d21/d24/d28/f39 [4194304,4194304] 0 2024-03-21T10:09:11.915 INFO:tasks.workunit.client.1.smithi174.stdout:0/921: dwrite d9/d21/d81/fe9 [0,4194304] 0 2024-03-21T10:09:11.920 INFO:tasks.workunit.client.1.smithi174.stdout:0/922: mkdir d9/d21/d81/d140 0 2024-03-21T10:09:11.925 INFO:tasks.workunit.client.1.smithi174.stdout:0/923: mkdir d9/d35/d4a/d67/d89/d99/d141 0 2024-03-21T10:09:11.927 INFO:tasks.workunit.client.1.smithi174.stdout:0/924: fsync d9/d21/d31/d5c/d92/f128 0 2024-03-21T10:09:11.928 INFO:tasks.workunit.client.0.smithi153.stdout:1/989: sync 2024-03-21T10:09:11.929 INFO:tasks.workunit.client.1.smithi174.stdout:0/925: mknod d9/d21/d23/da5/c142 0 2024-03-21T10:09:11.931 INFO:tasks.workunit.client.1.smithi174.stdout:0/926: mkdir d9/d21/d143 0 2024-03-21T10:09:11.935 INFO:tasks.workunit.client.1.smithi174.stdout:0/927: mkdir d9/d21/d31/d9f/d61/db9/d144 0 2024-03-21T10:09:11.941 INFO:tasks.workunit.client.0.smithi153.stdout:1/990: getdents d6/d13/d8e/dbb/de1/de7/de9/df5/d50 0 2024-03-21T10:09:11.943 INFO:tasks.workunit.client.1.smithi174.stdout:0/928: dread d9/f15 [0,4194304] 0 2024-03-21T10:09:11.963 INFO:tasks.workunit.client.0.smithi153.stdout:1/991: dwrite d6/d10/f97 [0,4194304] 0 2024-03-21T10:09:11.969 INFO:tasks.workunit.client.0.smithi153.stdout:1/992: symlink d6/d10/l148 0 2024-03-21T10:09:11.974 INFO:tasks.workunit.client.1.smithi174.stdout:2/864: sync 2024-03-21T10:09:11.974 INFO:tasks.workunit.client.1.smithi174.stdout:3/894: sync 2024-03-21T10:09:11.976 INFO:tasks.workunit.client.1.smithi174.stdout:3/895: chown d0/fa 281126 1 2024-03-21T10:09:11.979 INFO:tasks.workunit.client.1.smithi174.stdout:2/865: fsync d5/d25/d27/d2d/d45/d5b/d2f/d68/de8/fe9 0 2024-03-21T10:09:11.979 INFO:tasks.workunit.client.0.smithi153.stdout:1/993: symlink d6/d10/da0/d106/l149 0 2024-03-21T10:09:11.982 INFO:tasks.workunit.client.1.smithi174.stdout:2/866: stat d5/d25/d27/d2d/d45/dae/dd6/d4f/da0/def/f124 0 2024-03-21T10:09:11.983 INFO:tasks.workunit.client.0.smithi153.stdout:1/994: chown d6/d10/d35/da5/d4f/f102 26 1 2024-03-21T10:09:11.990 INFO:tasks.workunit.client.1.smithi174.stdout:2/867: getdents d5/d25/d27/d2d/d45/dae/dd6/d7b/d81/da1 0 2024-03-21T10:09:12.002 INFO:tasks.workunit.client.1.smithi174.stdout:3/896: dwrite d0/d12/d14/d1a/d44/d56/fa2 [0,4194304] 0 2024-03-21T10:09:12.015 INFO:tasks.workunit.client.1.smithi174.stdout:3/897: dread d0/d12/d14/d1a/d44/d65/d7e/dac/fc0 [0,4194304] 0 2024-03-21T10:09:12.055 INFO:tasks.workunit.client.1.smithi174.stdout:2/868: write d5/d25/d27/d2d/d45/d5b/d2f/d4b/fcb [518243,43193] 0 2024-03-21T10:09:12.055 INFO:tasks.workunit.client.1.smithi174.stdout:2/869: stat d5/l1b 0 2024-03-21T10:09:12.060 INFO:tasks.workunit.client.1.smithi174.stdout:2/870: creat d5/d25/d27/d2d/d45/dae/dd6/d85/f12f x:0 0 0 2024-03-21T10:09:12.061 INFO:tasks.workunit.client.1.smithi174.stdout:3/898: dread d0/d12/d14/da9/de7/db7/f31 [0,4194304] 0 2024-03-21T10:09:12.066 INFO:tasks.workunit.client.1.smithi174.stdout:3/899: mkdir d0/d12/d14/d1a/d27/dff/d110 0 2024-03-21T10:09:12.068 INFO:tasks.workunit.client.1.smithi174.stdout:3/900: mkdir d0/d12/d46/d111 0 2024-03-21T10:09:12.069 INFO:tasks.workunit.client.1.smithi174.stdout:2/871: write d5/d25/d27/d2d/d45/dae/dd6/d85/dad/f122 [518172,122976] 0 2024-03-21T10:09:12.070 INFO:tasks.workunit.client.1.smithi174.stdout:3/901: dread - d0/d12/d14/d1a/d27/ff8 zero size 2024-03-21T10:09:12.082 INFO:tasks.workunit.client.1.smithi174.stdout:5/920: sync 2024-03-21T10:09:12.082 INFO:tasks.workunit.client.1.smithi174.stdout:8/901: sync 2024-03-21T10:09:12.082 INFO:tasks.workunit.client.1.smithi174.stdout:7/845: sync 2024-03-21T10:09:12.084 INFO:tasks.workunit.client.1.smithi174.stdout:8/902: read d9/d2a/d39/f47 [4310679,38254] 0 2024-03-21T10:09:12.085 INFO:tasks.workunit.client.1.smithi174.stdout:7/846: dread - d0/d4/d10/d44/d81/d9a/f9b zero size 2024-03-21T10:09:12.088 INFO:tasks.workunit.client.1.smithi174.stdout:7/847: fsync d0/d4/d10/d1b/d86/fc3 0 2024-03-21T10:09:12.090 INFO:tasks.workunit.client.1.smithi174.stdout:2/872: dread d5/d25/d27/d2d/d45/d5b/d2f/d66/f93 [0,4194304] 0 2024-03-21T10:09:12.090 INFO:tasks.workunit.client.1.smithi174.stdout:7/848: write d0/d4/d10/d1b/f3b [834410,364] 0 2024-03-21T10:09:12.094 INFO:tasks.workunit.client.1.smithi174.stdout:7/849: unlink d0/d4/d10/d1b/f4e 0 2024-03-21T10:09:12.095 INFO:tasks.workunit.client.1.smithi174.stdout:2/873: mknod d5/d25/d27/d2d/d45/d5b/d2f/d4b/c130 0 2024-03-21T10:09:12.102 INFO:tasks.workunit.client.0.smithi153.stdout:1/995: sync 2024-03-21T10:09:12.112 INFO:tasks.workunit.client.0.smithi153.stdout:1/996: unlink d6/d10/d35/da5/d4f/d9d/dc7/dcc/d80/ld6 0 2024-03-21T10:09:12.113 INFO:tasks.workunit.client.0.smithi153.stdout:1/997: chown d6/fb 928149 1 2024-03-21T10:09:12.116 INFO:tasks.workunit.client.0.smithi153.stdout:1/998: mkdir d6/d32/d14a 0 2024-03-21T10:09:12.121 INFO:tasks.workunit.client.0.smithi153.stdout:1/999: mknod d6/d32/d14a/c14b 0 2024-03-21T10:09:12.132 INFO:tasks.workunit.client.0.smithi153.stderr:+ rm -rf -- ./tmp.2EStReJkf8 2024-03-21T10:09:12.153 INFO:tasks.workunit.client.1.smithi174.stdout:7/850: mknod d0/d4/d10/d1b/d86/dea/df4/c12f 0 2024-03-21T10:09:12.159 INFO:tasks.workunit.client.1.smithi174.stdout:3/902: dread d0/d12/d14/da9/de7/db7/f1b [0,4194304] 0 2024-03-21T10:09:12.164 INFO:tasks.workunit.client.1.smithi174.stdout:7/851: mkdir d0/d4/d10/d79/db2/dda/d5f/d130 0 2024-03-21T10:09:12.168 INFO:tasks.workunit.client.1.smithi174.stdout:3/903: symlink d0/d12/d14/d1a/d44/d65/d7e/l112 0 2024-03-21T10:09:12.174 INFO:tasks.workunit.client.1.smithi174.stdout:3/904: chown d0/d12/d46/f6f 30245272 1 2024-03-21T10:09:12.176 INFO:tasks.workunit.client.1.smithi174.stdout:7/852: rename d0/f57 to d0/d4/d10/d79/db2/dda/d5f/f131 0 2024-03-21T10:09:12.179 INFO:tasks.workunit.client.1.smithi174.stdout:7/853: chown d0/d4/fd 483327888 1 2024-03-21T10:09:12.180 INFO:tasks.workunit.client.1.smithi174.stdout:3/905: stat d0/d12/d14/d1a/d44/d56/l69 0 2024-03-21T10:09:12.181 INFO:tasks.workunit.client.1.smithi174.stdout:2/874: dwrite d5/f49 [0,4194304] 0 2024-03-21T10:09:12.192 INFO:tasks.workunit.client.1.smithi174.stdout:2/875: creat d5/d25/d27/d2d/d45/d5b/d63/de1/f131 x:0 0 0 2024-03-21T10:09:12.196 INFO:tasks.workunit.client.1.smithi174.stdout:2/876: chown d5/d25/d27/d2d/d45/d5b/f22 9 1 2024-03-21T10:09:12.198 INFO:tasks.workunit.client.1.smithi174.stdout:3/906: rename d0/d12/d14/da9/de7/fd7 to d0/d12/d14/d1a/d44/d65/f113 0 2024-03-21T10:09:12.202 INFO:tasks.workunit.client.1.smithi174.stdout:2/877: creat d5/d25/d27/d2d/d45/d5b/d2f/d66/db5/d12b/f132 x:0 0 0 2024-03-21T10:09:12.202 INFO:tasks.workunit.client.1.smithi174.stdout:3/907: creat d0/d12/d14/d4c/f114 x:0 0 0 2024-03-21T10:09:12.205 INFO:tasks.workunit.client.1.smithi174.stdout:9/938: sync 2024-03-21T10:09:12.206 INFO:tasks.workunit.client.1.smithi174.stdout:0/929: sync 2024-03-21T10:09:12.206 INFO:tasks.workunit.client.1.smithi174.stdout:6/818: sync 2024-03-21T10:09:12.207 INFO:tasks.workunit.client.1.smithi174.stdout:3/908: unlink d0/d12/d14/d4c/f79 0 2024-03-21T10:09:12.207 INFO:tasks.workunit.client.1.smithi174.stdout:9/939: fsync d2/d11/d60/f108 0 2024-03-21T10:09:12.209 INFO:tasks.workunit.client.1.smithi174.stdout:2/878: symlink d5/d25/d27/d2d/d45/d5b/d2f/d4b/dd8/dff/l133 0 2024-03-21T10:09:12.210 INFO:tasks.workunit.client.1.smithi174.stdout:3/909: readlink d0/d12/d14/d1a/d44/d65/l10c 0 2024-03-21T10:09:12.211 INFO:tasks.workunit.client.1.smithi174.stdout:6/819: symlink d0/d3/d2a/d56/d5e/d91/df0/l102 0 2024-03-21T10:09:12.213 INFO:tasks.workunit.client.1.smithi174.stdout:0/930: symlink d9/d21/d23/da5/ddf/l145 0 2024-03-21T10:09:12.213 INFO:tasks.workunit.client.1.smithi174.stdout:3/910: stat d0/d12/d14/da9/de7/db7/f31 0 2024-03-21T10:09:12.216 INFO:tasks.workunit.client.1.smithi174.stdout:6/820: truncate d0/d3/d20/d6d/ff1 619939 0 2024-03-21T10:09:12.216 INFO:tasks.workunit.client.1.smithi174.stdout:3/911: fsync d0/d12/d14/d1a/d44/d65/f10f 0 2024-03-21T10:09:12.219 INFO:tasks.workunit.client.1.smithi174.stdout:6/821: readlink d0/d3/d20/lcf 0 2024-03-21T10:09:12.221 INFO:tasks.workunit.client.1.smithi174.stdout:0/931: rename d9/d21/d31/d5c/d92/f100 to d9/d21/d31/d9f/d61/db9/d144/f146 0 2024-03-21T10:09:12.222 INFO:tasks.workunit.client.1.smithi174.stdout:3/912: creat d0/d12/d14/db9/f115 x:0 0 0 2024-03-21T10:09:12.224 INFO:tasks.workunit.client.1.smithi174.stdout:2/879: getdents d5/d25/d27/d2d/d45/dae/dd6/d4f/d65/d77/dcf 0 2024-03-21T10:09:12.224 INFO:tasks.workunit.client.1.smithi174.stdout:0/932: chown d9/d21/d31/d9f/d79/d82/dbb/f13e 1495 1 2024-03-21T10:09:12.225 INFO:tasks.workunit.client.1.smithi174.stdout:2/880: stat d5/d25/fbd 0 2024-03-21T10:09:12.228 INFO:tasks.workunit.client.1.smithi174.stdout:0/933: truncate d9/d21/d31/d5c/d92/fdb 4430394 0 2024-03-21T10:09:12.228 INFO:tasks.workunit.client.1.smithi174.stdout:6/822: mknod d0/d3/d5/d6/dfe/c103 0 2024-03-21T10:09:12.231 INFO:tasks.workunit.client.1.smithi174.stdout:0/934: stat d9/d1c/d76/d87/faa 0 2024-03-21T10:09:12.232 INFO:tasks.workunit.client.1.smithi174.stdout:6/823: mkdir d0/d3/d2a/d56/d5e/d91/d104 0 2024-03-21T10:09:12.232 INFO:tasks.workunit.client.1.smithi174.stdout:2/881: truncate d5/d25/d27/d2d/d45/dae/dd6/d85/dad/fb3 1747301 0 2024-03-21T10:09:12.237 INFO:tasks.workunit.client.1.smithi174.stdout:2/882: unlink d5/d25/l4c 0 2024-03-21T10:09:12.238 INFO:tasks.workunit.client.1.smithi174.stdout:9/940: dwrite d2/d11/f36 [0,4194304] 0 2024-03-21T10:09:12.240 INFO:tasks.workunit.client.1.smithi174.stdout:9/941: readlink d2/d11/d1d/dab/dc1/lae 0 2024-03-21T10:09:12.245 INFO:tasks.workunit.client.1.smithi174.stdout:0/935: write d9/dd8/f10a [1020299,37823] 0 2024-03-21T10:09:12.247 INFO:tasks.workunit.client.1.smithi174.stdout:9/942: getdents d2 0 2024-03-21T10:09:12.249 INFO:tasks.workunit.client.1.smithi174.stdout:6/824: truncate d0/d3/d5/df/f2e 2198519 0 2024-03-21T10:09:12.249 INFO:tasks.workunit.client.1.smithi174.stdout:2/883: write d5/d25/d27/d2d/d45/d5b/d2f/d68/f96 [475102,10084] 0 2024-03-21T10:09:12.251 INFO:tasks.workunit.client.1.smithi174.stdout:0/936: rmdir d9/d21/d31/d9f/d79/d82/dbb 39 2024-03-21T10:09:12.252 INFO:tasks.workunit.client.1.smithi174.stdout:9/943: rmdir d2/d11/d1d/d4b/d72 39 2024-03-21T10:09:12.253 INFO:tasks.workunit.client.1.smithi174.stdout:0/937: chown d9/d1c/d76/c94 12787957 1 2024-03-21T10:09:12.254 INFO:tasks.workunit.client.1.smithi174.stdout:6/825: symlink d0/d3/d2a/d56/d5e/d91/l105 0 2024-03-21T10:09:12.255 INFO:tasks.workunit.client.1.smithi174.stdout:9/944: chown d2/d11/d7d/fde 0 1 2024-03-21T10:09:12.259 INFO:tasks.workunit.client.1.smithi174.stdout:0/938: truncate d9/da2/f125 1930898 0 2024-03-21T10:09:12.260 INFO:tasks.workunit.client.1.smithi174.stdout:6/826: write d0/d3/d2a/d56/d5e/f98 [2159782,13609] 0 2024-03-21T10:09:12.264 INFO:tasks.workunit.client.1.smithi174.stdout:6/827: write d0/d3/d2a/d56/d5e/f100 [413799,125457] 0 2024-03-21T10:09:12.264 INFO:tasks.workunit.client.1.smithi174.stdout:9/945: truncate d2/d11/d1d/d4b/d72/fe0 1044916 0 2024-03-21T10:09:12.265 INFO:tasks.workunit.client.1.smithi174.stdout:0/939: mknod d9/d21/d23/da5/ddf/d12e/d13c/c147 0 2024-03-21T10:09:12.269 INFO:tasks.workunit.client.1.smithi174.stdout:0/940: fdatasync d9/d35/f46 0 2024-03-21T10:09:12.274 INFO:tasks.workunit.client.1.smithi174.stdout:0/941: stat d9/d1c/c24 0 2024-03-21T10:09:12.274 INFO:tasks.workunit.client.1.smithi174.stdout:9/946: unlink d2/d11/d1d/d2a/df2/l98 0 2024-03-21T10:09:12.275 INFO:tasks.workunit.client.1.smithi174.stdout:6/828: getdents d0/d3 0 2024-03-21T10:09:12.276 INFO:tasks.workunit.client.1.smithi174.stdout:2/884: dwrite d5/d25/d27/d2d/d45/d5b/d2f/d4b/d76/f7f [0,4194304] 0 2024-03-21T10:09:12.280 INFO:tasks.workunit.client.1.smithi174.stdout:6/829: mkdir d0/d3/d2a/daf/d106 0 2024-03-21T10:09:12.281 INFO:tasks.workunit.client.1.smithi174.stdout:2/885: chown d5/d25/d27/d2d/d45/dae/dd6/d7b/d81/cc0 37918 1 2024-03-21T10:09:12.283 INFO:tasks.workunit.client.1.smithi174.stdout:0/942: write d9/d21/d31/d5c/dd5/fe5 [322641,70840] 0 2024-03-21T10:09:12.284 INFO:tasks.workunit.client.1.smithi174.stdout:9/947: write d2/d24/d2e/ffe [302848,78972] 0 2024-03-21T10:09:12.285 INFO:tasks.workunit.client.1.smithi174.stdout:2/886: creat d5/d25/d27/d2d/d45/dae/dd6/d4f/da0/def/f134 x:0 0 0 2024-03-21T10:09:12.287 INFO:tasks.workunit.client.1.smithi174.stdout:9/948: fsync d2/d24/fe7 0 2024-03-21T10:09:12.289 INFO:tasks.workunit.client.1.smithi174.stdout:0/943: fsync d9/d21/f37 0 2024-03-21T10:09:12.292 INFO:tasks.workunit.client.1.smithi174.stdout:2/887: fdatasync d5/d25/d27/d2d/d45/dae/dd6/d4f/f6d 0 2024-03-21T10:09:12.292 INFO:tasks.workunit.client.1.smithi174.stdout:9/949: write d2/f3c [1418246,35287] 0 2024-03-21T10:09:12.336 INFO:tasks.workunit.client.1.smithi174.stdout:5/921: sync 2024-03-21T10:09:12.350 INFO:tasks.workunit.client.1.smithi174.stdout:0/944: mkdir d9/d21/d31/d5c/d86/d148 0 2024-03-21T10:09:12.354 INFO:tasks.workunit.client.1.smithi174.stdout:9/950: stat d2/d11/d1d/d4b/caa 0 2024-03-21T10:09:12.357 INFO:tasks.workunit.client.1.smithi174.stdout:2/888: mknod d5/d25/d27/d2d/d45/d5b/d2f/d68/de8/c135 0 2024-03-21T10:09:12.357 INFO:tasks.workunit.client.1.smithi174.stdout:0/945: creat d9/d35/dd2/f149 x:0 0 0 2024-03-21T10:09:12.360 INFO:tasks.workunit.client.1.smithi174.stdout:2/889: chown d5/d25/d27/d2d/d45/dae/dd6/d4f/df1 642520303 1 2024-03-21T10:09:12.360 INFO:tasks.workunit.client.1.smithi174.stdout:0/946: creat d9/d21/f14a x:0 0 0 2024-03-21T10:09:12.364 INFO:tasks.workunit.client.1.smithi174.stdout:2/890: creat d5/d25/d27/d2d/d45/dae/dd6/d4f/d65/d77/d106/f136 x:0 0 0 2024-03-21T10:09:12.369 INFO:tasks.workunit.client.1.smithi174.stdout:5/922: dwrite d1/fe1 [0,4194304] 0 2024-03-21T10:09:12.370 INFO:tasks.workunit.client.1.smithi174.stdout:5/923: fsync d1/fd4 0 2024-03-21T10:09:12.371 INFO:tasks.workunit.client.1.smithi174.stdout:2/891: truncate d5/d25/d27/d2d/d45/d5b/d55/f98 3770151 0 2024-03-21T10:09:12.376 INFO:tasks.workunit.client.1.smithi174.stdout:2/892: write d5/d25/d27/d2d/d45/dae/dd6/d85/f12f [351509,33295] 0 2024-03-21T10:09:12.382 INFO:tasks.workunit.client.1.smithi174.stdout:5/924: read d1/d16/d2e/d66/f7d [363002,110415] 0 2024-03-21T10:09:12.402 INFO:tasks.workunit.client.1.smithi174.stdout:2/893: dread d5/d25/d27/d2d/d45/dae/dd6/d7b/d81/fba [0,4194304] 0 2024-03-21T10:09:12.405 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:12.403+0000 7fca85a00700 1 -- 172.21.15.153:0/3340784063 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca80074dc0 msgr2=0x7fca80073220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:09:12.405 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:12.403+0000 7fca85a00700 1 --2- 172.21.15.153:0/3340784063 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca80074dc0 0x7fca80073220 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fca70009910 tx=0x7fca70009c20 comp rx=0 tx=0).stop 2024-03-21T10:09:12.405 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:12.403+0000 7fca85a00700 1 -- 172.21.15.153:0/3340784063 shutdown_connections 2024-03-21T10:09:12.406 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:12.403+0000 7fca85a00700 1 --2- 172.21.15.153:0/3340784063 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fca800737f0 0x7fca80073c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:09:12.406 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:12.403+0000 7fca85a00700 1 --2- 172.21.15.153:0/3340784063 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca80074dc0 0x7fca80073220 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:09:12.406 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:12.403+0000 7fca85a00700 1 -- 172.21.15.153:0/3340784063 >> 172.21.15.153:0/3340784063 conn(0x7fca800fc110 msgr2=0x7fca800fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:09:12.406 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:12.403+0000 7fca85a00700 1 -- 172.21.15.153:0/3340784063 shutdown_connections 2024-03-21T10:09:12.406 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:12.403+0000 7fca85a00700 1 -- 172.21.15.153:0/3340784063 wait complete. 2024-03-21T10:09:12.406 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:12 smithi153 ceph-mon[29117]: pgmap v22: 65 pgs: 65 active+clean; 4.0 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 129 MiB/s rd, 295 MiB/s wr, 668 op/s 2024-03-21T10:09:12.408 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:12.404+0000 7fca85a00700 1 Processor -- start 2024-03-21T10:09:12.408 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:12.405+0000 7fca85a00700 1 -- start start 2024-03-21T10:09:12.408 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:12.405+0000 7fca85a00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fca800737f0 0x7fca80071d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:09:12.408 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:12.405+0000 7fca85a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca80074dc0 0x7fca800722e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:09:12.408 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:12.405+0000 7fca85a00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca80072920 con 0x7fca80074dc0 2024-03-21T10:09:12.408 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:12.405+0000 7fca85a00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca80194fb0 con 0x7fca800737f0 2024-03-21T10:09:12.408 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:12.405+0000 7fca7e000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca80074dc0 0x7fca800722e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:09:12.408 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:12.406+0000 7fca7e000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca80074dc0 0x7fca800722e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:52186/0 (socket says 172.21.15.153:52186) 2024-03-21T10:09:12.409 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:12.406+0000 7fca7e000700 1 -- 172.21.15.153:0/1941418309 learned_addr learned my addr 172.21.15.153:0/1941418309 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:09:12.409 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:12.406+0000 7fca7ea00700 1 --2- 172.21.15.153:0/1941418309 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fca800737f0 0x7fca80071d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:09:12.409 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:12.406+0000 7fca7e000700 1 -- 172.21.15.153:0/1941418309 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fca800737f0 msgr2=0x7fca80071d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:09:12.409 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:12.406+0000 7fca7e000700 1 --2- 172.21.15.153:0/1941418309 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fca800737f0 0x7fca80071d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:09:12.409 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:12.406+0000 7fca7e000700 1 -- 172.21.15.153:0/1941418309 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca700095f0 con 0x7fca80074dc0 2024-03-21T10:09:12.409 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:12.407+0000 7fca7e000700 1 --2- 172.21.15.153:0/1941418309 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca80074dc0 0x7fca800722e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fca6c00d900 tx=0x7fca6c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:09:12.409 INFO:tasks.workunit.client.1.smithi174.stdout:2/894: creat d5/d25/d27/d2d/d45/dae/dd6/d4f/d65/f137 x:0 0 0 2024-03-21T10:09:12.410 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:12.407+0000 7fca77400700 1 -- 172.21.15.153:0/1941418309 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fca6c00b920 con 0x7fca80074dc0 2024-03-21T10:09:12.410 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:12.407+0000 7fca77400700 1 -- 172.21.15.153:0/1941418309 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fca6c010460 con 0x7fca80074dc0 2024-03-21T10:09:12.410 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:12.407+0000 7fca77400700 1 -- 172.21.15.153:0/1941418309 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fca6c00f590 con 0x7fca80074dc0 2024-03-21T10:09:12.410 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:12.407+0000 7fca85a00700 1 -- 172.21.15.153:0/1941418309 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca80195290 con 0x7fca80074dc0 2024-03-21T10:09:12.410 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:12.407+0000 7fca85a00700 1 -- 172.21.15.153:0/1941418309 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca801957b0 con 0x7fca80074dc0 2024-03-21T10:09:12.410 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:12.409+0000 7fca85a00700 1 -- 172.21.15.153:0/1941418309 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca8010b920 con 0x7fca80074dc0 2024-03-21T10:09:12.411 INFO:tasks.workunit.client.1.smithi174.stdout:2/895: chown d5/d25/d27/d2d/d79/c12e 0 1 2024-03-21T10:09:12.411 INFO:tasks.workunit.client.1.smithi174.stdout:2/896: chown d5/d25/d27/d2d/d45/d5b/f11d 1 1 2024-03-21T10:09:12.412 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:12.410+0000 7fca77400700 1 -- 172.21.15.153:0/1941418309 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 29) v1 ==== 92174+0+0 (secure 0 0 0) 0x7fca6c0105d0 con 0x7fca80074dc0 2024-03-21T10:09:12.416 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:12.411+0000 7fca77400700 1 --2- 172.21.15.153:0/1941418309 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7fca6406e210 0x7fca640706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:09:12.416 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:12.411+0000 7fca77400700 1 -- 172.21.15.153:0/1941418309 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fca6c08dfe0 con 0x7fca80074dc0 2024-03-21T10:09:12.416 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:12.412+0000 7fca7ea00700 1 --2- 172.21.15.153:0/1941418309 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7fca6406e210 0x7fca640706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:09:12.416 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:12.413+0000 7fca7ea00700 1 --2- 172.21.15.153:0/1941418309 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7fca6406e210 0x7fca640706d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fca70009fd0 tx=0x7fca7000b560 comp rx=0 tx=0).ready entity=mgr.14666 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:09:12.420 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:12.419+0000 7fca77400700 1 -- 172.21.15.153:0/1941418309 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185454 (secure 0 0 0) 0x7fca6c0574e0 con 0x7fca80074dc0 2024-03-21T10:09:12.433 INFO:tasks.workunit.client.1.smithi174.stdout:2/897: dwrite d5/d25/d27/d2d/d45/d5b/d55/f98 [0,4194304] 0 2024-03-21T10:09:12.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:12 smithi174 ceph-mon[37496]: pgmap v22: 65 pgs: 65 active+clean; 4.0 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 129 MiB/s rd, 295 MiB/s wr, 668 op/s 2024-03-21T10:09:12.487 INFO:tasks.workunit.client.1.smithi174.stdout:7/854: sync 2024-03-21T10:09:12.487 INFO:tasks.workunit.client.1.smithi174.stdout:3/913: sync 2024-03-21T10:09:12.487 INFO:tasks.workunit.client.1.smithi174.stdout:8/903: sync 2024-03-21T10:09:12.501 INFO:tasks.workunit.client.1.smithi174.stdout:2/898: fsync d5/d25/d27/d2d/d45/d5b/fb6 0 2024-03-21T10:09:12.502 INFO:tasks.workunit.client.1.smithi174.stdout:3/914: dread - d0/d12/d14/db9/f115 zero size 2024-03-21T10:09:12.504 INFO:tasks.workunit.client.1.smithi174.stdout:7/855: creat d0/d9e/f132 x:0 0 0 2024-03-21T10:09:12.512 INFO:tasks.workunit.client.1.smithi174.stdout:8/904: unlink d9/d2a/d39/d76/d19/d27/d60/daa/dab/fe0 0 2024-03-21T10:09:12.515 INFO:tasks.workunit.client.1.smithi174.stdout:7/856: rmdir d0/d4/d10/d44/d81/d9a/dc0 39 2024-03-21T10:09:12.516 INFO:tasks.workunit.client.1.smithi174.stdout:3/915: mknod d0/d12/d14/d1a/d44/d56/c116 0 2024-03-21T10:09:12.518 INFO:tasks.workunit.client.1.smithi174.stdout:8/905: rmdir d9/d2a/d39/de1 39 2024-03-21T10:09:12.519 INFO:tasks.workunit.client.1.smithi174.stdout:7/857: read d0/f11 [1007577,60939] 0 2024-03-21T10:09:12.522 INFO:tasks.workunit.client.1.smithi174.stdout:8/906: chown d9/fa9 0 1 2024-03-21T10:09:12.527 INFO:tasks.workunit.client.1.smithi174.stdout:8/907: dread - d9/d53/fe8 zero size 2024-03-21T10:09:12.530 INFO:tasks.workunit.client.1.smithi174.stdout:7/858: truncate d0/d4/d38/dec/f115 78982 0 2024-03-21T10:09:12.536 INFO:tasks.workunit.client.1.smithi174.stdout:2/899: dwrite d5/d25/d27/f39 [0,4194304] 0 2024-03-21T10:09:12.538 INFO:tasks.workunit.client.1.smithi174.stdout:7/859: creat d0/d4/f133 x:0 0 0 2024-03-21T10:09:12.539 INFO:tasks.workunit.client.1.smithi174.stdout:3/916: dread d0/d12/f2d [0,4194304] 0 2024-03-21T10:09:12.542 INFO:tasks.workunit.client.1.smithi174.stdout:7/860: write d0/d4/f33 [1035229,54464] 0 2024-03-21T10:09:12.548 INFO:tasks.workunit.client.1.smithi174.stdout:8/908: write d9/d2a/d39/fa0 [824597,106746] 0 2024-03-21T10:09:12.550 INFO:tasks.workunit.client.1.smithi174.stdout:3/917: write d0/fe4 [4402711,48583] 0 2024-03-21T10:09:12.551 INFO:tasks.workunit.client.1.smithi174.stdout:2/900: write d5/d25/d27/d2d/d45/d5b/d63/de1/f128 [687831,92973] 0 2024-03-21T10:09:12.552 INFO:tasks.workunit.client.1.smithi174.stdout:8/909: fdatasync d9/dc6/fac 0 2024-03-21T10:09:12.558 INFO:tasks.workunit.client.1.smithi174.stdout:8/910: readlink d9/d2a/d39/l46 0 2024-03-21T10:09:12.566 INFO:tasks.workunit.client.1.smithi174.stdout:8/911: rename d9/d53/d5b/c74 to d9/d53/dc9/de6/c12a 0 2024-03-21T10:09:12.568 INFO:tasks.workunit.client.1.smithi174.stdout:2/901: getdents d5/d25/d27/d2d/d45/d5b/d2f/d4b 0 2024-03-21T10:09:12.573 INFO:tasks.workunit.client.1.smithi174.stdout:2/902: read d5/d25/d27/d2d/d45/d5b/d63/f48 [5349910,13001] 0 2024-03-21T10:09:12.575 INFO:tasks.workunit.client.1.smithi174.stdout:3/918: dwrite d0/d12/d14/d67/dc8/fc5 [0,4194304] 0 2024-03-21T10:09:12.577 INFO:tasks.workunit.client.1.smithi174.stdout:8/912: link d9/d2a/d39/d76/l22 d9/d2a/d39/d76/d19/d27/d60/daa/df2/l12b 0 2024-03-21T10:09:12.580 INFO:tasks.workunit.client.1.smithi174.stdout:7/861: dread d0/d4/d10/d1b/d86/fc4 [0,4194304] 0 2024-03-21T10:09:12.582 INFO:tasks.workunit.client.1.smithi174.stdout:8/913: symlink d9/d2a/d39/d76/d19/d27/d60/daa/l12c 0 2024-03-21T10:09:12.582 INFO:tasks.workunit.client.1.smithi174.stdout:3/919: chown d0/d12/d14/d1a/d44/d94/db6/lae 31884060 1 2024-03-21T10:09:12.586 INFO:tasks.workunit.client.1.smithi174.stdout:7/862: creat d0/d4/d38/d103/f134 x:0 0 0 2024-03-21T10:09:12.587 INFO:tasks.workunit.client.1.smithi174.stdout:3/920: write d0/d12/d14/da9/de7/db7/f107 [969814,74018] 0 2024-03-21T10:09:12.588 INFO:tasks.workunit.client.1.smithi174.stdout:7/863: dread - d0/d4/d38/f70 zero size 2024-03-21T10:09:12.598 INFO:tasks.workunit.client.1.smithi174.stdout:2/903: dwrite d5/d25/d27/d2d/d45/d5b/d2f/d4b/fbe [0,4194304] 0 2024-03-21T10:09:12.600 INFO:tasks.workunit.client.1.smithi174.stdout:2/904: stat d5/l1b 0 2024-03-21T10:09:12.600 INFO:tasks.workunit.client.1.smithi174.stdout:7/864: write d0/d4/d38/f70 [127274,18492] 0 2024-03-21T10:09:12.602 INFO:tasks.workunit.client.1.smithi174.stdout:2/905: dread - d5/d25/d27/d74/fcc zero size 2024-03-21T10:09:12.606 INFO:tasks.workunit.client.1.smithi174.stdout:7/865: mkdir d0/d135 0 2024-03-21T10:09:12.607 INFO:tasks.workunit.client.1.smithi174.stdout:2/906: write d5/d25/d27/d2d/d45/dae/dd6/d4f/d65/d77/dcf/fd5 [1686745,13308] 0 2024-03-21T10:09:12.608 INFO:tasks.workunit.client.1.smithi174.stdout:6/830: sync 2024-03-21T10:09:12.612 INFO:tasks.workunit.client.1.smithi174.stdout:6/831: symlink d0/d3/d2a/l107 0 2024-03-21T10:09:12.612 INFO:tasks.workunit.client.1.smithi174.stdout:7/866: write d0/d4/ffb [1082670,41421] 0 2024-03-21T10:09:12.615 INFO:tasks.workunit.client.1.smithi174.stdout:2/907: write d5/d25/d27/d2d/d45/dae/dd6/d85/dad/f101 [3540660,28128] 0 2024-03-21T10:09:12.616 INFO:tasks.workunit.client.1.smithi174.stdout:8/914: truncate d9/d2a/d39/d76/d19/d27/d60/daa/f110 583475 0 2024-03-21T10:09:12.618 INFO:tasks.workunit.client.1.smithi174.stdout:2/908: fdatasync d5/d25/d27/f40 0 2024-03-21T10:09:12.619 INFO:tasks.workunit.client.1.smithi174.stdout:3/921: dwrite d0/d12/d14/d1a/d44/d65/d7e/dac/fc0 [0,4194304] 0 2024-03-21T10:09:12.621 INFO:tasks.workunit.client.1.smithi174.stdout:8/915: unlink d9/d53/f78 0 2024-03-21T10:09:12.624 INFO:tasks.workunit.client.1.smithi174.stdout:2/909: chown d5/d25/d27/d2d/d45/d5b/d55/fbf 35360914 1 2024-03-21T10:09:12.629 INFO:tasks.workunit.client.1.smithi174.stdout:7/867: rename d0/d4/d38/dec/f115 to d0/d4/d10/d79/db2/dda/d5f/d130/f136 0 2024-03-21T10:09:12.633 INFO:tasks.workunit.client.1.smithi174.stdout:6/832: rename d0/d3/d2a/d56/fe4 to d0/d3/d2a/f108 0 2024-03-21T10:09:12.635 INFO:tasks.workunit.client.1.smithi174.stdout:6/833: dread - d0/d3/d5/df/d16/da9/db7/fec zero size 2024-03-21T10:09:12.636 INFO:tasks.workunit.client.1.smithi174.stdout:3/922: rename d0/d12/d14/da9/de7/db7/l4a to d0/d12/d14/d4c/d7a/d9e/l117 0 2024-03-21T10:09:12.638 INFO:tasks.workunit.client.1.smithi174.stdout:7/868: getdents d0/d4/d10/d44/d81/d9a 0 2024-03-21T10:09:12.639 INFO:tasks.workunit.client.1.smithi174.stdout:6/834: mknod d0/d3/d2a/d56/c109 0 2024-03-21T10:09:12.640 INFO:tasks.workunit.client.1.smithi174.stdout:7/869: write d0/d4/f23 [5996571,124919] 0 2024-03-21T10:09:12.644 INFO:tasks.workunit.client.1.smithi174.stdout:3/923: getdents d0/d12/d14/d1a/d44 0 2024-03-21T10:09:12.644 INFO:tasks.workunit.client.1.smithi174.stdout:6/835: write d0/feb [852777,39965] 0 2024-03-21T10:09:12.646 INFO:tasks.workunit.client.1.smithi174.stdout:7/870: mkdir d0/d4/d10/d1b/d86/dea/df4/d137 0 2024-03-21T10:09:12.648 INFO:tasks.workunit.client.1.smithi174.stdout:8/916: write d9/f30 [136411,102835] 0 2024-03-21T10:09:12.649 INFO:tasks.workunit.client.1.smithi174.stdout:7/871: chown d0/d4/d10/d44/f11f 32 1 2024-03-21T10:09:12.650 INFO:tasks.workunit.client.1.smithi174.stdout:3/924: symlink d0/d12/d46/df9/l118 0 2024-03-21T10:09:12.653 INFO:tasks.workunit.client.1.smithi174.stdout:7/872: symlink d0/d11b/dfe/l138 0 2024-03-21T10:09:12.656 INFO:tasks.workunit.client.1.smithi174.stdout:3/925: creat d0/d12/d14/d1a/d44/d94/dd9/f119 x:0 0 0 2024-03-21T10:09:12.660 INFO:tasks.workunit.client.1.smithi174.stdout:3/926: chown d0/d12/d14/d4c/d7a/ffc 1858380262 1 2024-03-21T10:09:12.661 INFO:tasks.workunit.client.1.smithi174.stdout:8/917: getdents d9/d53/dc9/de6/dea/d117 0 2024-03-21T10:09:12.664 INFO:tasks.workunit.client.1.smithi174.stdout:7/873: link d0/d4/d10/d79/db2/dda/d5f/fd3 d0/f139 0 2024-03-21T10:09:12.665 INFO:tasks.workunit.client.1.smithi174.stdout:8/918: mkdir d9/d2a/d12d 0 2024-03-21T10:09:12.666 INFO:tasks.workunit.client.1.smithi174.stdout:7/874: dread - d0/d4/d38/f128 zero size 2024-03-21T10:09:12.669 INFO:tasks.workunit.client.1.smithi174.stdout:3/927: mkdir d0/d12/d14/d1a/d27/dff/d110/d11a 0 2024-03-21T10:09:12.669 INFO:tasks.workunit.client.1.smithi174.stdout:6/836: dwrite d0/fee [0,4194304] 0 2024-03-21T10:09:12.675 INFO:tasks.workunit.client.1.smithi174.stdout:3/928: rmdir d0/d12/d14/d1a/d44/d94/dd9 39 2024-03-21T10:09:12.682 INFO:tasks.workunit.client.1.smithi174.stdout:6/837: rename d0/d3/d2a to d0/d3/d5/d6/d42/d10a 0 2024-03-21T10:09:12.704 INFO:tasks.workunit.client.1.smithi174.stdout:6/838: creat d0/d3/d20/d6d/f10b x:0 0 0 2024-03-21T10:09:12.709 INFO:tasks.workunit.client.1.smithi174.stdout:8/919: dwrite d9/d2a/d39/d76/f20 [0,4194304] 0 2024-03-21T10:09:12.710 INFO:tasks.workunit.client.1.smithi174.stdout:7/875: dwrite d0/d4/d10/d1b/d86/dea/df4/d123/fb9 [4194304,4194304] 0 2024-03-21T10:09:12.713 INFO:tasks.workunit.client.1.smithi174.stdout:6/839: write d0/d3/d5/df/d16/f17 [3574565,26752] 0 2024-03-21T10:09:12.713 INFO:tasks.workunit.client.1.smithi174.stdout:3/929: dwrite d0/d12/f54 [4194304,4194304] 0 2024-03-21T10:09:12.729 INFO:tasks.workunit.client.1.smithi174.stdout:3/930: link d0/d12/d14/d1a/d44/d94/l96 d0/d12/d14/d67/da1/da7/l11b 0 2024-03-21T10:09:12.732 INFO:tasks.workunit.client.1.smithi174.stdout:6/840: getdents d0/d3/d5/d6/d42 0 2024-03-21T10:09:12.732 INFO:tasks.workunit.client.1.smithi174.stdout:8/920: link d9/d2a/d39/d76/d19/d27/f64 d9/d53/d5b/d10a/f12e 0 2024-03-21T10:09:12.735 INFO:tasks.workunit.client.1.smithi174.stdout:3/931: symlink d0/d12/d46/d59/l11c 0 2024-03-21T10:09:12.735 INFO:tasks.workunit.client.1.smithi174.stdout:6/841: stat d0/d3/d20/d3d/fa6 0 2024-03-21T10:09:12.737 INFO:tasks.workunit.client.1.smithi174.stdout:0/947: sync 2024-03-21T10:09:12.737 INFO:tasks.workunit.client.1.smithi174.stdout:5/925: sync 2024-03-21T10:09:12.737 INFO:tasks.workunit.client.1.smithi174.stdout:9/951: sync 2024-03-21T10:09:12.744 INFO:tasks.workunit.client.1.smithi174.stdout:3/932: chown d0/d12/d14/d1a/d44/d94/dd9/f119 3 1 2024-03-21T10:09:12.744 INFO:tasks.workunit.client.1.smithi174.stdout:7/876: dwrite d0/f98 [0,4194304] 0 2024-03-21T10:09:12.746 INFO:tasks.workunit.client.1.smithi174.stdout:7/877: chown d0/d4/d10/d1b/fd7 2552 1 2024-03-21T10:09:12.748 INFO:tasks.workunit.client.1.smithi174.stdout:0/948: unlink d9/d21/d31/d5c/d92/d98/db7/ddc/c109 0 2024-03-21T10:09:12.749 INFO:tasks.workunit.client.1.smithi174.stdout:6/842: read d0/d3/da8/fd2 [1696390,108645] 0 2024-03-21T10:09:12.750 INFO:tasks.workunit.client.1.smithi174.stdout:9/952: rename d2/d24/f59 to d2/d11/d1d/dab/dc1/f149 0 2024-03-21T10:09:12.756 INFO:tasks.workunit.client.1.smithi174.stdout:3/933: mkdir d0/d12/d14/d1a/d27/dff/d110/d11a/d11d 0 2024-03-21T10:09:12.757 INFO:tasks.workunit.client.1.smithi174.stdout:0/949: mknod d9/d21/d31/d5c/d92/d98/d130/c14b 0 2024-03-21T10:09:12.760 INFO:tasks.workunit.client.1.smithi174.stdout:6/843: rename d0/d3/da8 to d0/d10c 0 2024-03-21T10:09:12.761 INFO:tasks.workunit.client.1.smithi174.stdout:3/934: symlink d0/d12/d46/d59/l11e 0 2024-03-21T10:09:12.764 INFO:tasks.workunit.client.1.smithi174.stdout:3/935: mkdir d0/d12/d14/da9/de7/d11f 0 2024-03-21T10:09:12.766 INFO:tasks.workunit.client.1.smithi174.stdout:3/936: chown d0/d12/d14/d1a/cf1 7487 1 2024-03-21T10:09:12.767 INFO:tasks.workunit.client.1.smithi174.stdout:9/953: dwrite d2/d11/d1d/d2a/da7/f13e [0,4194304] 0 2024-03-21T10:09:12.772 INFO:tasks.workunit.client.1.smithi174.stdout:9/954: symlink d2/d11/d1d/dab/l14a 0 2024-03-21T10:09:12.772 INFO:tasks.workunit.client.1.smithi174.stdout:8/921: dread d9/d2a/d39/d76/d19/d27/d60/daa/f110 [0,4194304] 0 2024-03-21T10:09:12.774 INFO:tasks.workunit.client.1.smithi174.stdout:3/937: symlink d0/d12/d14/d4c/d7a/d9e/dee/l120 0 2024-03-21T10:09:12.775 INFO:tasks.workunit.client.1.smithi174.stdout:7/878: write d0/d4/d10/d1b/d86/fc4 [3151696,41222] 0 2024-03-21T10:09:12.782 INFO:tasks.workunit.client.1.smithi174.stdout:7/879: stat d0/d4/d10/d1b/d86/dea/df4/d123/c105 0 2024-03-21T10:09:12.784 INFO:tasks.workunit.client.1.smithi174.stdout:3/938: mkdir d0/d12/d14/d1a/d27/dff/d110/d11a/d121 0 2024-03-21T10:09:12.786 INFO:tasks.workunit.client.1.smithi174.stdout:6/844: mkdir d0/d3/d5/df/d66/d87/ded/d10d 0 2024-03-21T10:09:12.788 INFO:tasks.workunit.client.1.smithi174.stdout:7/880: mknod d0/d4/d10/c13a 0 2024-03-21T10:09:12.791 INFO:tasks.workunit.client.1.smithi174.stdout:5/926: dwrite d1/d49/d4f/d51/f56 [8388608,4194304] 0 2024-03-21T10:09:12.791 INFO:tasks.workunit.client.1.smithi174.stdout:3/939: creat d0/d12/d14/d1a/d44/f122 x:0 0 0 2024-03-21T10:09:12.793 INFO:tasks.workunit.client.1.smithi174.stdout:7/881: dread - d0/d4/d10/d1b/f104 zero size 2024-03-21T10:09:12.796 INFO:tasks.workunit.client.1.smithi174.stdout:7/882: chown d0/d4/d10/d1b/d86 181004453 1 2024-03-21T10:09:12.798 INFO:tasks.workunit.client.1.smithi174.stdout:9/955: dwrite d2/d11/d1d/d8b/f12f [0,4194304] 0 2024-03-21T10:09:12.803 INFO:tasks.workunit.client.1.smithi174.stdout:7/883: write d0/f100 [567754,77937] 0 2024-03-21T10:09:12.804 INFO:tasks.workunit.client.1.smithi174.stdout:9/956: chown d2/d11/d60/db2/f119 10814169 1 2024-03-21T10:09:12.808 INFO:tasks.workunit.client.1.smithi174.stdout:6/845: dread d0/d3/d20/d3d/ffd [4194304,4194304] 0 2024-03-21T10:09:12.808 INFO:tasks.workunit.client.1.smithi174.stdout:9/957: mkdir d2/d10b/d14b 0 2024-03-21T10:09:12.821 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:12.819+0000 7fca85a00700 1 -- 172.21.15.153:0/1941418309 --> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fca801080b0 con 0x7fca6406e210 2024-03-21T10:09:12.822 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:12.821+0000 7fca77400700 1 -- 172.21.15.153:0/1941418309 <== mgr.14666 v2:172.21.15.153:6800/2524145044 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+329 (secure 0 0 0) 0x7fca801080b0 con 0x7fca6406e210 2024-03-21T10:09:12.825 INFO:tasks.workunit.client.1.smithi174.stdout:6/846: truncate d0/d3/d5/df/d16/da9/db7/dc2/fc7 410738 0 2024-03-21T10:09:12.826 INFO:tasks.workunit.client.1.smithi174.stdout:9/958: truncate d2/d11/d1d/d4b/fbf 3950198 0 2024-03-21T10:09:12.827 INFO:tasks.workunit.client.1.smithi174.stdout:7/884: dwrite d0/d4/d10/d1b/d86/f92 [0,4194304] 0 2024-03-21T10:09:12.828 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:12.826+0000 7fca85a00700 1 -- 172.21.15.153:0/1941418309 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7fca6406e210 msgr2=0x7fca640706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:09:12.828 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:12.826+0000 7fca85a00700 1 --2- 172.21.15.153:0/1941418309 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7fca6406e210 0x7fca640706d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fca70009fd0 tx=0x7fca7000b560 comp rx=0 tx=0).stop 2024-03-21T10:09:12.828 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:12.826+0000 7fca85a00700 1 -- 172.21.15.153:0/1941418309 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca80074dc0 msgr2=0x7fca800722e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:09:12.828 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:12.826+0000 7fca85a00700 1 --2- 172.21.15.153:0/1941418309 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca80074dc0 0x7fca800722e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fca6c00d900 tx=0x7fca6c00dc10 comp rx=0 tx=0).stop 2024-03-21T10:09:12.828 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:12.826+0000 7fca85a00700 1 -- 172.21.15.153:0/1941418309 shutdown_connections 2024-03-21T10:09:12.828 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:12.826+0000 7fca85a00700 1 --2- 172.21.15.153:0/1941418309 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7fca6406e210 0x7fca640706d0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:09:12.828 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:12.826+0000 7fca85a00700 1 --2- 172.21.15.153:0/1941418309 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fca800737f0 0x7fca80071d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:09:12.828 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:12.826+0000 7fca85a00700 1 --2- 172.21.15.153:0/1941418309 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca80074dc0 0x7fca800722e0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:09:12.828 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:12.826+0000 7fca85a00700 1 -- 172.21.15.153:0/1941418309 >> 172.21.15.153:0/1941418309 conn(0x7fca800fc110 msgr2=0x7fca80106990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:09:12.828 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:12.826+0000 7fca85a00700 1 -- 172.21.15.153:0/1941418309 shutdown_connections 2024-03-21T10:09:12.828 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:12.826+0000 7fca85a00700 1 -- 172.21.15.153:0/1941418309 wait complete. 2024-03-21T10:09:12.831 INFO:tasks.workunit.client.1.smithi174.stdout:9/959: truncate d2/d11/d60/db2/ffd 696011 0 2024-03-21T10:09:12.832 INFO:tasks.workunit.client.1.smithi174.stdout:7/885: fdatasync d0/d4/d10/d1b/fa6 0 2024-03-21T10:09:12.832 INFO:tasks.workunit.client.1.smithi174.stdout:6/847: dread d0/d3/d5/df/d16/da9/fd0 [0,4194304] 0 2024-03-21T10:09:12.834 INFO:tasks.workunit.client.1.smithi174.stdout:9/960: truncate d2/d10b/f11f 229801 0 2024-03-21T10:09:12.834 INFO:tasks.workunit.client.1.smithi174.stdout:5/927: dread d1/f2c [0,4194304] 0 2024-03-21T10:09:12.836 INFO:tasks.workunit.client.1.smithi174.stdout:7/886: chown d0/d4/d10/d44/f73 0 1 2024-03-21T10:09:12.840 INFO:tasks.workunit.client.1.smithi174.stdout:9/961: rmdir d2/d11 39 2024-03-21T10:09:12.843 INFO:tasks.workunit.client.1.smithi174.stdout:5/928: symlink d1/d2f/d37/d9b/l149 0 2024-03-21T10:09:12.851 INFO:teuthology.orchestra.run.smithi153.stdout:true 2024-03-21T10:09:12.851 INFO:tasks.workunit.client.1.smithi174.stdout:2/910: sync 2024-03-21T10:09:12.852 INFO:tasks.workunit.client.1.smithi174.stdout:7/887: write d0/d4/d10/d1b/d86/dd2/fd0 [4612795,68936] 0 2024-03-21T10:09:12.854 INFO:tasks.workunit.client.1.smithi174.stdout:9/962: stat d2/d11/d1d/d4b/l126 0 2024-03-21T10:09:12.855 INFO:tasks.workunit.client.1.smithi174.stdout:2/911: write d5/d25/d27/f4e [1925688,114346] 0 2024-03-21T10:09:12.855 INFO:tasks.workunit.client.1.smithi174.stdout:6/848: dread d0/d3/d5/df/d16/f65 [0,4194304] 0 2024-03-21T10:09:12.856 INFO:tasks.workunit.client.1.smithi174.stdout:9/963: chown d2/d11/d7d 1941060 1 2024-03-21T10:09:12.858 INFO:tasks.workunit.client.1.smithi174.stdout:7/888: symlink d0/d4/d10/l13b 0 2024-03-21T10:09:12.861 INFO:tasks.workunit.client.1.smithi174.stdout:6/849: read d0/d3/d20/d3d/fa6 [487662,17778] 0 2024-03-21T10:09:12.861 INFO:tasks.workunit.client.1.smithi174.stdout:7/889: fdatasync d0/d4/d10/d79/fe5 0 2024-03-21T10:09:12.863 INFO:tasks.workunit.client.1.smithi174.stdout:9/964: mknod d2/d11/d1d/d4b/d72/c14c 0 2024-03-21T10:09:12.863 INFO:tasks.workunit.client.1.smithi174.stdout:7/890: fdatasync d0/d4/f133 0 2024-03-21T10:09:12.864 INFO:tasks.workunit.client.1.smithi174.stdout:6/850: chown d0/d3/d20/d3d 25337682 1 2024-03-21T10:09:12.868 INFO:tasks.workunit.client.1.smithi174.stdout:5/929: dwrite d1/d16/d2e/d66/f7d [0,4194304] 0 2024-03-21T10:09:12.869 INFO:tasks.workunit.client.1.smithi174.stdout:2/912: rename d5/d25/d27/d2d/d45/dae/dd6/d90/da9/cfa to d5/d25/d27/d2d/d45/d5b/d2f/d66/c138 0 2024-03-21T10:09:12.871 INFO:tasks.workunit.client.1.smithi174.stdout:9/965: mknod d2/d24/d2e/dac/c14d 0 2024-03-21T10:09:12.875 INFO:tasks.workunit.client.1.smithi174.stdout:5/930: mkdir d1/d113/d14a 0 2024-03-21T10:09:12.876 INFO:tasks.workunit.client.1.smithi174.stdout:7/891: mknod d0/d4/d10/d1b/d86/dea/df4/df8/c13c 0 2024-03-21T10:09:12.877 INFO:tasks.workunit.client.1.smithi174.stdout:6/851: rename d0/d3/d5/df/d16/da9/ce1 to d0/df5/c10e 0 2024-03-21T10:09:12.878 INFO:tasks.workunit.client.1.smithi174.stdout:2/913: mkdir d5/d25/d109/d121/d139 0 2024-03-21T10:09:12.879 INFO:tasks.workunit.client.1.smithi174.stdout:9/966: read d2/d11/f2c [3550848,118862] 0 2024-03-21T10:09:12.879 INFO:tasks.workunit.client.1.smithi174.stdout:7/892: write d0/d4/d38/da9/dc6/fca [593140,63275] 0 2024-03-21T10:09:12.882 INFO:tasks.workunit.client.1.smithi174.stdout:6/852: write d0/fee [403817,104688] 0 2024-03-21T10:09:12.890 INFO:tasks.workunit.client.1.smithi174.stdout:2/914: dread d5/d25/d27/d2d/d45/dae/dd6/d4f/d65/d77/ff7 [0,4194304] 0 2024-03-21T10:09:12.892 INFO:tasks.workunit.client.1.smithi174.stdout:5/931: write d1/d2f/d37/f5d [4114755,55176] 0 2024-03-21T10:09:12.892 INFO:tasks.workunit.client.1.smithi174.stdout:9/967: write d2/d11/d1d/d8b/fa2 [4890361,119934] 0 2024-03-21T10:09:12.894 INFO:tasks.workunit.client.1.smithi174.stdout:2/915: mknod d5/d25/d27/d2d/d45/d5b/d63/c13a 0 2024-03-21T10:09:12.899 INFO:tasks.workunit.client.1.smithi174.stdout:2/916: creat d5/d25/d27/d2d/d79/f13b x:0 0 0 2024-03-21T10:09:12.901 INFO:tasks.workunit.client.1.smithi174.stdout:7/893: dwrite d0/d4/d10/d1b/f25 [0,4194304] 0 2024-03-21T10:09:12.906 INFO:tasks.workunit.client.1.smithi174.stdout:7/894: chown d0/d4/d10/d1b/d86/dd2/cf2 1133239 1 2024-03-21T10:09:12.907 INFO:tasks.workunit.client.1.smithi174.stdout:9/968: truncate d2/d11/d60/db2/f119 885540 0 2024-03-21T10:09:12.908 INFO:tasks.workunit.client.1.smithi174.stdout:2/917: dread d5/d25/d27/d2d/d45/d5b/d63/f114 [0,4194304] 0 2024-03-21T10:09:12.909 INFO:tasks.workunit.client.1.smithi174.stdout:7/895: mknod d0/d11b/dfe/c13d 0 2024-03-21T10:09:12.913 INFO:tasks.workunit.client.1.smithi174.stdout:7/896: fsync d0/f11 0 2024-03-21T10:09:12.913 INFO:tasks.workunit.client.1.smithi174.stdout:5/932: dwrite d1/d5c/f7a [0,4194304] 0 2024-03-21T10:09:12.915 INFO:tasks.workunit.client.1.smithi174.stdout:2/918: rmdir d5/d25/d27/d2d/d45/dae/dd6/d4f/df1 39 2024-03-21T10:09:12.915 INFO:tasks.workunit.client.1.smithi174.stdout:9/969: truncate d2/d11/d60/ff8 1075190 0 2024-03-21T10:09:12.918 INFO:tasks.workunit.client.1.smithi174.stdout:7/897: write d0/d4/d10/d1b/d86/fc4 [3944040,100498] 0 2024-03-21T10:09:12.922 INFO:tasks.workunit.client.1.smithi174.stdout:7/898: write d0/d4/d10/d1b/d86/fc4 [4822595,26958] 0 2024-03-21T10:09:12.927 INFO:tasks.workunit.client.1.smithi174.stdout:5/933: mkdir d1/d2f/dc1/d14b 0 2024-03-21T10:09:12.929 INFO:tasks.workunit.client.1.smithi174.stdout:9/970: chown d2/d24/d2e/l5f 474504115 1 2024-03-21T10:09:12.929 INFO:tasks.workunit.client.1.smithi174.stdout:2/919: dread - d5/d25/d27/d2d/d45/dae/dd6/d90/fdd zero size 2024-03-21T10:09:12.933 INFO:tasks.workunit.client.1.smithi174.stdout:7/899: symlink d0/d4/d38/da9/l13e 0 2024-03-21T10:09:12.934 INFO:tasks.workunit.client.1.smithi174.stdout:5/934: creat d1/d49/d4f/d51/d8f/d103/f14c x:0 0 0 2024-03-21T10:09:12.935 INFO:tasks.workunit.client.1.smithi174.stdout:2/920: symlink d5/d25/d27/d2d/d45/d5b/d2f/d66/l13c 0 2024-03-21T10:09:12.936 INFO:tasks.workunit.client.1.smithi174.stdout:9/971: creat d2/d10b/d112/d115/f14e x:0 0 0 2024-03-21T10:09:12.936 INFO:tasks.workunit.client.1.smithi174.stdout:5/935: readlink d1/d16/d2e/l38 0 2024-03-21T10:09:12.938 INFO:tasks.workunit.client.1.smithi174.stdout:7/900: rename d0/f100 to d0/d4/d10/d79/f13f 0 2024-03-21T10:09:12.940 INFO:tasks.workunit.client.1.smithi174.stdout:5/936: truncate d1/d16/f60 837767 0 2024-03-21T10:09:12.944 INFO:tasks.workunit.client.1.smithi174.stdout:2/921: getdents d5/d25/d27/d2d/d45 0 2024-03-21T10:09:12.945 INFO:tasks.workunit.client.1.smithi174.stdout:5/937: mknod d1/d2f/dc1/d144/c14d 0 2024-03-21T10:09:12.949 INFO:tasks.workunit.client.1.smithi174.stdout:5/938: fdatasync d1/d49/d4f/d51/d8f/f127 0 2024-03-21T10:09:12.953 INFO:tasks.workunit.client.1.smithi174.stdout:9/972: write d2/d11/d1d/d2a/df2/f8c [1429373,68795] 0 2024-03-21T10:09:12.956 INFO:tasks.workunit.client.1.smithi174.stdout:7/901: dwrite d0/d4/d10/f19 [0,4194304] 0 2024-03-21T10:09:12.958 INFO:tasks.workunit.client.1.smithi174.stdout:3/940: sync 2024-03-21T10:09:12.958 INFO:tasks.workunit.client.1.smithi174.stdout:8/922: sync 2024-03-21T10:09:12.959 INFO:tasks.workunit.client.1.smithi174.stdout:0/950: sync 2024-03-21T10:09:12.969 INFO:tasks.workunit.client.1.smithi174.stdout:3/941: rmdir d0/d12/d14/d1a/d5c 39 2024-03-21T10:09:12.970 INFO:tasks.workunit.client.1.smithi174.stdout:7/902: read d0/d4/d10/d1b/d86/dea/f10d [3741773,102825] 0 2024-03-21T10:09:12.975 INFO:tasks.workunit.client.1.smithi174.stdout:8/923: unlink d9/d2a/d39/d76/d19/d27/d60/ccc 0 2024-03-21T10:09:12.975 INFO:tasks.workunit.client.1.smithi174.stdout:2/922: dwrite d5/d25/d27/d2d/d45/dae/dd6/d4f/da0/def/f134 [0,4194304] 0 2024-03-21T10:09:12.976 INFO:tasks.workunit.client.1.smithi174.stdout:6/853: sync 2024-03-21T10:09:12.985 INFO:tasks.workunit.client.1.smithi174.stdout:5/939: dwrite f0 [4194304,4194304] 0 2024-03-21T10:09:12.991 INFO:tasks.workunit.client.1.smithi174.stdout:5/940: chown d1/d16/d2e/fe9 80 1 2024-03-21T10:09:12.995 INFO:tasks.workunit.client.1.smithi174.stdout:9/973: dwrite d2/d24/d2e/f67 [0,4194304] 0 2024-03-21T10:09:12.998 INFO:tasks.workunit.client.1.smithi174.stdout:6/854: write d0/d3/d5/d6/d42/d10a/d56/d5e/f98 [4253283,70122] 0 2024-03-21T10:09:13.017 INFO:tasks.workunit.client.1.smithi174.stdout:0/951: dwrite d9/d35/dd2/f149 [0,4194304] 0 2024-03-21T10:09:13.020 INFO:tasks.workunit.client.1.smithi174.stdout:3/942: dwrite d0/d12/d46/df9/fa3 [0,4194304] 0 2024-03-21T10:09:13.031 INFO:tasks.workunit.client.1.smithi174.stdout:7/903: dwrite d0/d4/d38/fbf [0,4194304] 0 2024-03-21T10:09:13.042 INFO:tasks.workunit.client.1.smithi174.stdout:8/924: dwrite d9/f8c [0,4194304] 0 2024-03-21T10:09:13.045 INFO:tasks.workunit.client.1.smithi174.stdout:8/925: fsync d9/d2a/d39/fa0 0 2024-03-21T10:09:13.050 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:13.048+0000 7fbdc3c00700 1 -- 172.21.15.153:0/4016558040 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbdbc1016e0 msgr2=0x7fbdbc103ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:09:13.050 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:13.048+0000 7fbdc3c00700 1 --2- 172.21.15.153:0/4016558040 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbdbc1016e0 0x7fbdbc103ad0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fbda8009870 tx=0x7fbda8009b80 comp rx=0 tx=0).stop 2024-03-21T10:09:13.051 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:13.049+0000 7fbdc3c00700 1 -- 172.21.15.153:0/4016558040 shutdown_connections 2024-03-21T10:09:13.051 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:13.049+0000 7fbdc3c00700 1 --2- 172.21.15.153:0/4016558040 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbdbc104010 0x7fbdbc106400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:09:13.051 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:13.049+0000 7fbdc3c00700 1 --2- 172.21.15.153:0/4016558040 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbdbc1016e0 0x7fbdbc103ad0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:09:13.052 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:13.049+0000 7fbdc3c00700 1 -- 172.21.15.153:0/4016558040 >> 172.21.15.153:0/4016558040 conn(0x7fbdbc0fb030 msgr2=0x7fbdbc0fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:09:13.052 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:13.049+0000 7fbdc3c00700 1 -- 172.21.15.153:0/4016558040 shutdown_connections 2024-03-21T10:09:13.052 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:13.049+0000 7fbdc3c00700 1 -- 172.21.15.153:0/4016558040 wait complete. 2024-03-21T10:09:13.053 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:13.051+0000 7fbdc3c00700 1 Processor -- start 2024-03-21T10:09:13.053 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:13.051+0000 7fbdc3c00700 1 -- start start 2024-03-21T10:09:13.053 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:13.051+0000 7fbdc3c00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbdbc1016e0 0x7fbdbc19c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:09:13.054 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:13.051+0000 7fbdc3c00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbdbc104010 0x7fbdbc19ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:09:13.054 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:13.051+0000 7fbdc3c00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdbc19d480 con 0x7fbdbc104010 2024-03-21T10:09:13.054 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:13.051+0000 7fbdc3c00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdbc19d5c0 con 0x7fbdbc1016e0 2024-03-21T10:09:13.054 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:13.052+0000 7fbdbbe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbdbc104010 0x7fbdbc19ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:09:13.054 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:13.052+0000 7fbdbbe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbdbc104010 0x7fbdbc19ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42088/0 (socket says 172.21.15.153:42088) 2024-03-21T10:09:13.054 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:13.052+0000 7fbdbbe00700 1 -- 172.21.15.153:0/1278120695 learned_addr learned my addr 172.21.15.153:0/1278120695 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:09:13.055 INFO:tasks.workunit.client.1.smithi174.stdout:2/923: creat d5/d25/d27/d2d/d45/dae/dd6/d85/f13d x:0 0 0 2024-03-21T10:09:13.056 INFO:tasks.workunit.client.1.smithi174.stdout:5/941: creat d1/d5c/d96/f14e x:0 0 0 2024-03-21T10:09:13.056 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:13.053+0000 7fbdbbe00700 1 -- 172.21.15.153:0/1278120695 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbdbc1016e0 msgr2=0x7fbdbc19c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:09:13.056 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:13.053+0000 7fbdbbe00700 1 --2- 172.21.15.153:0/1278120695 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbdbc1016e0 0x7fbdbc19c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:09:13.056 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:13.053+0000 7fbdbbe00700 1 -- 172.21.15.153:0/1278120695 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbda8009520 con 0x7fbdbc104010 2024-03-21T10:09:13.057 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:13.053+0000 7fbdbbe00700 1 --2- 172.21.15.153:0/1278120695 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbdbc104010 0x7fbdbc19ce60 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fbdb400c0c0 tx=0x7fbdb400c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:09:13.057 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:13.054+0000 7fbdb9600700 1 -- 172.21.15.153:0/1278120695 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbdb4018070 con 0x7fbdbc104010 2024-03-21T10:09:13.057 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:13.054+0000 7fbdb9600700 1 -- 172.21.15.153:0/1278120695 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fbdb400f040 con 0x7fbdbc104010 2024-03-21T10:09:13.057 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:13.054+0000 7fbdc3c00700 1 -- 172.21.15.153:0/1278120695 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdbc1a2010 con 0x7fbdbc104010 2024-03-21T10:09:13.057 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:13.054+0000 7fbdc3c00700 1 -- 172.21.15.153:0/1278120695 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdbc1a25e0 con 0x7fbdbc104010 2024-03-21T10:09:13.057 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:13.055+0000 7fbdb9600700 1 -- 172.21.15.153:0/1278120695 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbdb400e020 con 0x7fbdbc104010 2024-03-21T10:09:13.058 INFO:tasks.workunit.client.1.smithi174.stdout:0/952: rename d9/d21/c55 to d9/d35/d4a/d8a/d10c/c14c 0 2024-03-21T10:09:13.058 INFO:tasks.workunit.client.1.smithi174.stdout:7/904: readlink d0/d4/d10/d44/le7 0 2024-03-21T10:09:13.059 INFO:tasks.workunit.client.1.smithi174.stdout:7/905: fsync d0/d4/f23 0 2024-03-21T10:09:13.059 INFO:tasks.workunit.client.1.smithi174.stdout:8/926: mkdir d9/d2a/d39/d76/d19/d12f 0 2024-03-21T10:09:13.060 INFO:tasks.workunit.client.1.smithi174.stdout:2/924: fdatasync d5/d25/d27/d2d/d45/d5b/d55/f98 0 2024-03-21T10:09:13.062 INFO:tasks.workunit.client.1.smithi174.stdout:9/974: mknod d2/d11/d1d/d5e/c14f 0 2024-03-21T10:09:13.062 INFO:tasks.workunit.client.1.smithi174.stdout:3/943: symlink d0/d12/d14/d1a/d27/d4f/dbe/l123 0 2024-03-21T10:09:13.062 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:13.059+0000 7fbdc3c00700 1 -- 172.21.15.153:0/1278120695 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd9c005320 con 0x7fbdbc104010 2024-03-21T10:09:13.063 INFO:tasks.workunit.client.1.smithi174.stdout:5/942: rmdir d1/d2f/d37 39 2024-03-21T10:09:13.063 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:13.061+0000 7fbdb9600700 1 -- 172.21.15.153:0/1278120695 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 29) v1 ==== 92174+0+0 (secure 0 0 0) 0x7fbdb4007930 con 0x7fbdbc104010 2024-03-21T10:09:13.064 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:13.063+0000 7fbdb9600700 1 --2- 172.21.15.153:0/1278120695 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7fbda006e210 0x7fbda00706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:09:13.064 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:13.063+0000 7fbdb9600700 1 -- 172.21.15.153:0/1278120695 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fbdb408e200 con 0x7fbdbc104010 2024-03-21T10:09:13.065 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:13.064+0000 7fbdc1200700 1 --2- 172.21.15.153:0/1278120695 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7fbda006e210 0x7fbda00706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:09:13.066 INFO:tasks.workunit.client.1.smithi174.stdout:2/925: write d5/d25/d27/d2d/d45/d5b/d2f/f9d [679486,66265] 0 2024-03-21T10:09:13.066 INFO:tasks.workunit.client.1.smithi174.stdout:7/906: mknod d0/d4/d10/d79/c140 0 2024-03-21T10:09:13.066 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:13.065+0000 7fbdc1200700 1 --2- 172.21.15.153:0/1278120695 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7fbda006e210 0x7fbda00706d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fbda800b650 tx=0x7fbda8009ed0 comp rx=0 tx=0).ready entity=mgr.14666 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:09:13.072 INFO:tasks.workunit.client.1.smithi174.stdout:7/907: dread d0/d4/d38/f70 [0,4194304] 0 2024-03-21T10:09:13.072 INFO:tasks.workunit.client.1.smithi174.stdout:5/943: symlink d1/d16/d2e/d66/d8b/d85/l14f 0 2024-03-21T10:09:13.073 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:13.072+0000 7fbdb9600700 1 -- 172.21.15.153:0/1278120695 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185454 (secure 0 0 0) 0x7fbdb405b1d0 con 0x7fbdbc104010 2024-03-21T10:09:13.075 INFO:tasks.workunit.client.1.smithi174.stdout:2/926: dread d5/d25/d27/d2d/f41 [0,4194304] 0 2024-03-21T10:09:13.079 INFO:tasks.workunit.client.1.smithi174.stdout:5/944: mkdir d1/d16/d2e/d8d/d150 0 2024-03-21T10:09:13.081 INFO:tasks.workunit.client.1.smithi174.stdout:5/945: chown d1/d16/d1c/d81/f143 147063170 1 2024-03-21T10:09:13.082 INFO:tasks.workunit.client.1.smithi174.stdout:9/975: creat d2/d11/d1d/d2a/f150 x:0 0 0 2024-03-21T10:09:13.084 INFO:tasks.workunit.client.1.smithi174.stdout:8/927: getdents d9/d53/d5b 0 2024-03-21T10:09:13.091 INFO:tasks.workunit.client.1.smithi174.stdout:5/946: creat d1/d16/d2e/d66/f151 x:0 0 0 2024-03-21T10:09:13.092 INFO:tasks.workunit.client.1.smithi174.stdout:3/944: link d0/d12/d14/d1a/d44/d65/d7e/dac/fc0 d0/d12/d14/da9/de7/d11f/f124 0 2024-03-21T10:09:13.098 INFO:tasks.workunit.client.1.smithi174.stdout:8/928: fdatasync d9/d2a/d39/d76/d19/d27/d60/daa/dab/dc4/fda 0 2024-03-21T10:09:13.100 INFO:tasks.workunit.client.1.smithi174.stdout:5/947: fsync d1/f29 0 2024-03-21T10:09:13.101 INFO:tasks.workunit.client.1.smithi174.stdout:2/927: getdents d5/d25/d27/d2d/d45/d5b/d2f/d4b/dd8/dff 0 2024-03-21T10:09:13.105 INFO:tasks.workunit.client.1.smithi174.stdout:3/945: dread d0/d12/d14/da9/de7/db7/f38 [4194304,4194304] 0 2024-03-21T10:09:13.105 INFO:tasks.workunit.client.1.smithi174.stdout:8/929: symlink d9/d2a/d4d/db4/l130 0 2024-03-21T10:09:13.107 INFO:tasks.workunit.client.1.smithi174.stdout:8/930: readlink d9/d2a/d4d/db2/lbd 0 2024-03-21T10:09:13.107 INFO:tasks.workunit.client.1.smithi174.stdout:2/928: mknod d5/d25/d27/d2d/d45/dae/dd6/d7b/d81/c13e 0 2024-03-21T10:09:13.109 INFO:tasks.workunit.client.1.smithi174.stdout:3/946: mknod d0/d12/d46/d111/c125 0 2024-03-21T10:09:13.109 INFO:tasks.workunit.client.1.smithi174.stdout:8/931: chown d9/d2a/d39/d76/d19/d27/d60/daa 1 1 2024-03-21T10:09:13.112 INFO:tasks.workunit.client.1.smithi174.stdout:2/929: symlink d5/d25/d27/d2d/d45/d5b/d2f/d4b/dd8/dff/l13f 0 2024-03-21T10:09:13.112 INFO:tasks.workunit.client.1.smithi174.stdout:8/932: chown d9/d2a/d39/d76/d19/cbe 696349130 1 2024-03-21T10:09:13.114 INFO:tasks.workunit.client.1.smithi174.stdout:3/947: mknod d0/d12/d14/da9/de7/db7/c126 0 2024-03-21T10:09:13.123 INFO:tasks.workunit.client.1.smithi174.stdout:5/948: dwrite d1/d5c/d73/de3/fc3 [0,4194304] 0 2024-03-21T10:09:13.131 INFO:tasks.workunit.client.1.smithi174.stdout:5/949: unlink d1/d49/d4f/d51/d8f/cb4 0 2024-03-21T10:09:13.136 INFO:tasks.workunit.client.1.smithi174.stdout:7/908: truncate d0/f26 5949973 0 2024-03-21T10:09:13.142 INFO:tasks.workunit.client.1.smithi174.stdout:6/855: dwrite d0/d3/d5/df/d16/da9/fd0 [0,4194304] 0 2024-03-21T10:09:13.143 INFO:tasks.workunit.client.1.smithi174.stdout:7/909: symlink d0/l141 0 2024-03-21T10:09:13.146 INFO:tasks.workunit.client.1.smithi174.stdout:0/953: dwrite d9/f65 [0,4194304] 0 2024-03-21T10:09:13.149 INFO:tasks.workunit.client.1.smithi174.stdout:2/930: dread d5/d25/d27/f3b [4194304,4194304] 0 2024-03-21T10:09:13.153 INFO:tasks.workunit.client.1.smithi174.stdout:7/910: unlink d0/d4/d10/d1b/f25 0 2024-03-21T10:09:13.160 INFO:tasks.workunit.client.1.smithi174.stdout:9/976: dwrite d2/d11/d1d/d2a/da7/fe9 [0,4194304] 0 2024-03-21T10:09:13.167 INFO:tasks.workunit.client.1.smithi174.stdout:5/950: truncate d1/f2c 2785931 0 2024-03-21T10:09:13.168 INFO:tasks.workunit.client.1.smithi174.stdout:2/931: rename d5/d25/d27/d2d/d45/d5b/d55/f69 to d5/d25/d109/f140 0 2024-03-21T10:09:13.174 INFO:tasks.workunit.client.1.smithi174.stdout:9/977: fdatasync d2/d11/d1d/d2a/da7/f13e 0 2024-03-21T10:09:13.174 INFO:tasks.workunit.client.1.smithi174.stdout:6/856: creat d0/d3/d20/d3d/f10f x:0 0 0 2024-03-21T10:09:13.174 INFO:tasks.workunit.client.1.smithi174.stdout:0/954: read d9/d35/dd2/d104/f49 [1815229,37772] 0 2024-03-21T10:09:13.175 INFO:tasks.workunit.client.1.smithi174.stdout:0/955: stat d9/fc1 0 2024-03-21T10:09:13.176 INFO:tasks.workunit.client.1.smithi174.stdout:8/933: dwrite d9/d2a/d39/d58/fb8 [0,4194304] 0 2024-03-21T10:09:13.178 INFO:tasks.workunit.client.1.smithi174.stdout:7/911: rename d0/d11b/dfe/c13d to d0/d9e/c142 0 2024-03-21T10:09:13.179 INFO:tasks.workunit.client.1.smithi174.stdout:0/956: fsync d9/d21/d31/d5c/d92/fdb 0 2024-03-21T10:09:13.180 INFO:tasks.workunit.client.1.smithi174.stdout:5/951: creat d1/d113/d14a/f152 x:0 0 0 2024-03-21T10:09:13.182 INFO:tasks.workunit.client.1.smithi174.stdout:6/857: fsync d0/d3/d5/d21/f6c 0 2024-03-21T10:09:13.184 INFO:tasks.workunit.client.1.smithi174.stdout:8/934: fsync d9/d53/d5b/fd4 0 2024-03-21T10:09:13.186 INFO:tasks.workunit.client.1.smithi174.stdout:6/858: chown d0/d3/d20/d3d 2064256073 1 2024-03-21T10:09:13.189 INFO:tasks.workunit.client.1.smithi174.stdout:6/859: chown d0/d3/d5/lf3 25 1 2024-03-21T10:09:13.193 INFO:tasks.workunit.client.1.smithi174.stdout:5/952: creat d1/d2f/d37/df5/f153 x:0 0 0 2024-03-21T10:09:13.194 INFO:tasks.workunit.client.1.smithi174.stdout:5/953: readlink d1/d2f/l72 0 2024-03-21T10:09:13.197 INFO:tasks.workunit.client.1.smithi174.stdout:8/935: mknod d9/dc6/df1/c131 0 2024-03-21T10:09:13.198 INFO:tasks.workunit.client.1.smithi174.stdout:7/912: dread d0/d4/d38/fbf [0,4194304] 0 2024-03-21T10:09:13.199 INFO:tasks.workunit.client.1.smithi174.stdout:6/860: chown d0/d3/d5/df/d66/cb6 3369 1 2024-03-21T10:09:13.201 INFO:tasks.workunit.client.1.smithi174.stdout:6/861: dread - d0/d3/d5/fbe zero size 2024-03-21T10:09:13.202 INFO:tasks.workunit.client.1.smithi174.stdout:7/913: dread - d0/d4/d10/f10f zero size 2024-03-21T10:09:13.204 INFO:tasks.workunit.client.1.smithi174.stdout:0/957: write d9/d35/dd2/d104/f83 [3481938,7881] 0 2024-03-21T10:09:13.210 INFO:tasks.workunit.client.1.smithi174.stdout:2/932: dwrite d5/d25/d27/d2d/d45/d5b/d2f/d68/f7a [0,4194304] 0 2024-03-21T10:09:13.210 INFO:tasks.workunit.client.1.smithi174.stdout:6/862: unlink d0/d3/d5/d6/d42/d10a/d8f/f97 0 2024-03-21T10:09:13.214 INFO:tasks.workunit.client.1.smithi174.stdout:7/914: creat d0/d4/d10/d1b/f143 x:0 0 0 2024-03-21T10:09:13.220 INFO:tasks.workunit.client.1.smithi174.stdout:5/954: dwrite d1/d16/d2e/d66/f89 [0,4194304] 0 2024-03-21T10:09:13.222 INFO:tasks.workunit.client.1.smithi174.stdout:7/915: creat d0/d4/d38/da9/dc6/f144 x:0 0 0 2024-03-21T10:09:13.225 INFO:tasks.workunit.client.1.smithi174.stdout:5/955: mkdir d1/d154 0 2024-03-21T10:09:13.227 INFO:tasks.workunit.client.1.smithi174.stdout:8/936: dwrite f8 [4194304,4194304] 0 2024-03-21T10:09:13.241 INFO:tasks.workunit.client.1.smithi174.stdout:0/958: dwrite d9/d35/dd2/d104/f83 [0,4194304] 0 2024-03-21T10:09:13.242 INFO:tasks.workunit.client.1.smithi174.stdout:8/937: read - d9/d2a/d39/d58/dbb/dc3/fde zero size 2024-03-21T10:09:13.242 INFO:tasks.workunit.client.1.smithi174.stdout:6/863: dwrite d0/d3/d5/d6/f2d [4194304,4194304] 0 2024-03-21T10:09:13.246 INFO:tasks.workunit.client.1.smithi174.stdout:0/959: fsync d9/d21/d31/d5c/dd5/fe5 0 2024-03-21T10:09:13.247 INFO:tasks.workunit.client.1.smithi174.stdout:6/864: chown d0/d3/d20/d3d/d81 33375652 1 2024-03-21T10:09:13.258 INFO:tasks.workunit.client.1.smithi174.stdout:6/865: unlink d0/d3/d5/d6/d42/d10a/d56/d5e/d91/c9a 0 2024-03-21T10:09:13.259 INFO:tasks.workunit.client.1.smithi174.stdout:7/916: dwrite d0/d4/f127 [0,4194304] 0 2024-03-21T10:09:13.260 INFO:tasks.workunit.client.1.smithi174.stdout:5/956: write d1/d16/d2e/d66/d8b/f48 [113165,35318] 0 2024-03-21T10:09:13.268 INFO:tasks.workunit.client.1.smithi174.stdout:6/866: mknod d0/d3/d5/d6/d42/d10a/d56/d5e/d91/df0/c110 0 2024-03-21T10:09:13.269 INFO:tasks.workunit.client.1.smithi174.stdout:2/933: dread d5/d25/d27/d74/f95 [0,4194304] 0 2024-03-21T10:09:13.273 INFO:tasks.workunit.client.1.smithi174.stdout:8/938: dwrite d9/d53/d5b/d10a/f12e [0,4194304] 0 2024-03-21T10:09:13.273 INFO:tasks.workunit.client.1.smithi174.stdout:5/957: getdents d1/d5c/d73 0 2024-03-21T10:09:13.275 INFO:tasks.workunit.client.1.smithi174.stdout:2/934: stat d5/d25/d27/d2d/d45/d5b/d63/f61 0 2024-03-21T10:09:13.282 INFO:tasks.workunit.client.1.smithi174.stdout:2/935: chown d5/d25/d109 43 1 2024-03-21T10:09:13.286 INFO:tasks.workunit.client.1.smithi174.stdout:2/936: fdatasync d5/d25/d27/d2d/d45/d5b/d2f/d4b/d76/f97 0 2024-03-21T10:09:13.294 INFO:tasks.workunit.client.1.smithi174.stdout:0/960: dwrite d9/d35/dd2/d104/f5b [4194304,4194304] 0 2024-03-21T10:09:13.307 INFO:tasks.workunit.client.1.smithi174.stdout:7/917: dwrite d0/d4/d10/d44/d81/fde [0,4194304] 0 2024-03-21T10:09:13.309 INFO:tasks.workunit.client.1.smithi174.stdout:5/958: dwrite f0 [0,4194304] 0 2024-03-21T10:09:13.310 INFO:tasks.workunit.client.1.smithi174.stdout:2/937: mkdir d5/d25/d27/d2d/d45/dae/dd6/d4f/d65/d141 0 2024-03-21T10:09:13.321 INFO:tasks.workunit.client.1.smithi174.stdout:7/918: mknod d0/d4/d38/dec/c145 0 2024-03-21T10:09:13.323 INFO:tasks.workunit.client.1.smithi174.stdout:8/939: dwrite d9/d2a/d39/d58/dbb/f128 [0,4194304] 0 2024-03-21T10:09:13.328 INFO:tasks.workunit.client.1.smithi174.stdout:2/938: symlink d5/d25/d27/d2d/d79/l142 0 2024-03-21T10:09:13.328 INFO:tasks.workunit.client.1.smithi174.stdout:6/867: dwrite d0/d3/d5/df/fd5 [0,4194304] 0 2024-03-21T10:09:13.332 INFO:tasks.workunit.client.1.smithi174.stdout:5/959: rename d1/d49/lbc to d1/d49/d93/d9c/l155 0 2024-03-21T10:09:13.337 INFO:tasks.workunit.client.1.smithi174.stdout:2/939: write d5/f14 [12347396,91349] 0 2024-03-21T10:09:13.340 INFO:tasks.workunit.client.1.smithi174.stdout:7/919: creat d0/d4/d10/d44/d81/d9a/f146 x:0 0 0 2024-03-21T10:09:13.340 INFO:tasks.workunit.client.1.smithi174.stdout:0/961: dwrite d9/f65 [0,4194304] 0 2024-03-21T10:09:13.349 INFO:tasks.workunit.client.1.smithi174.stdout:2/940: write d5/d25/d27/d2d/d45/d5b/d63/f114 [643454,90193] 0 2024-03-21T10:09:13.351 INFO:tasks.workunit.client.1.smithi174.stdout:0/962: mknod d9/d21/d81/c14d 0 2024-03-21T10:09:13.356 INFO:tasks.workunit.client.1.smithi174.stdout:0/963: write d9/d1c/d76/d87/faa [1683577,7661] 0 2024-03-21T10:09:13.356 INFO:tasks.workunit.client.1.smithi174.stdout:2/941: mkdir d5/d25/d27/d2d/d45/d5b/d2f/d4b/dd8/dff/d143 0 2024-03-21T10:09:13.359 INFO:tasks.workunit.client.1.smithi174.stdout:0/964: rmdir d9/d21/d75/da1 39 2024-03-21T10:09:13.364 INFO:tasks.workunit.client.1.smithi174.stdout:2/942: mkdir d5/d25/d27/d2d/d45/d5b/d2f/d66/db5/d144 0 2024-03-21T10:09:13.365 INFO:tasks.workunit.client.1.smithi174.stdout:0/965: write d9/d21/d31/d9f/d79/d82/dbb/f13e [856884,18488] 0 2024-03-21T10:09:13.367 INFO:tasks.workunit.client.1.smithi174.stdout:2/943: readlink d5/d25/d27/d2d/d45/d5b/d2f/d66/l13c 0 2024-03-21T10:09:13.374 INFO:tasks.workunit.client.1.smithi174.stdout:6/868: dread d0/d3/d5/d6/d42/d10a/f2f [0,4194304] 0 2024-03-21T10:09:13.377 INFO:tasks.workunit.client.1.smithi174.stdout:2/944: unlink d5/d25/d27/d2d/d45/d5b/d2f/d4b/c82 0 2024-03-21T10:09:13.383 INFO:tasks.workunit.client.1.smithi174.stdout:8/940: truncate d9/fae 3326089 0 2024-03-21T10:09:13.384 INFO:tasks.workunit.client.1.smithi174.stdout:2/945: write d5/d25/d27/d2d/d45/d5b/d2f/d4b/d76/f7f [2080173,3791] 0 2024-03-21T10:09:13.385 INFO:tasks.workunit.client.1.smithi174.stdout:7/920: stat d0/d4/d10/d44/d81/d9a/ff1 0 2024-03-21T10:09:13.388 INFO:tasks.workunit.client.1.smithi174.stdout:8/941: write d9/dc6/fa5 [3454887,7647] 0 2024-03-21T10:09:13.389 INFO:tasks.workunit.client.1.smithi174.stdout:0/966: dwrite d9/d21/d31/d5c/d86/f13d [0,4194304] 0 2024-03-21T10:09:13.391 INFO:tasks.workunit.client.1.smithi174.stdout:8/942: write d9/d2a/d39/f75 [2617784,122093] 0 2024-03-21T10:09:13.395 INFO:tasks.workunit.client.1.smithi174.stdout:7/921: symlink d0/d4/d38/d103/l147 0 2024-03-21T10:09:13.397 INFO:tasks.workunit.client.1.smithi174.stdout:5/960: dwrite d1/d2f/d37/ff4 [4194304,4194304] 0 2024-03-21T10:09:13.400 INFO:tasks.workunit.client.1.smithi174.stdout:6/869: truncate d0/d3/d5/d6/f6a 2293812 0 2024-03-21T10:09:13.402 INFO:tasks.workunit.client.1.smithi174.stdout:2/946: creat d5/d25/d27/d2d/d45/d5b/d2f/d4b/f145 x:0 0 0 2024-03-21T10:09:13.405 INFO:tasks.workunit.client.1.smithi174.stdout:0/967: creat d9/d35/dd2/f14e x:0 0 0 2024-03-21T10:09:13.407 INFO:tasks.workunit.client.1.smithi174.stdout:0/968: readlink d9/d35/dd2/d104/le0 0 2024-03-21T10:09:13.408 INFO:tasks.workunit.client.1.smithi174.stdout:8/943: rename d9/d2a/d39/d76/d19/d27/c73 to d9/d53/d7e/c132 0 2024-03-21T10:09:13.412 INFO:tasks.workunit.client.1.smithi174.stdout:6/870: creat d0/d3/d5/d6/d42/d10a/d56/f111 x:0 0 0 2024-03-21T10:09:13.412 INFO:tasks.workunit.client.1.smithi174.stdout:5/961: stat d1/d49/d4f/ce2 0 2024-03-21T10:09:13.414 INFO:tasks.workunit.client.1.smithi174.stdout:7/922: symlink d0/d4/d10/d44/d81/d9a/dc0/l148 0 2024-03-21T10:09:13.414 INFO:tasks.workunit.client.1.smithi174.stdout:2/947: symlink d5/d25/d27/d2d/d45/d5b/d55/d75/l146 0 2024-03-21T10:09:13.416 INFO:tasks.workunit.client.1.smithi174.stdout:0/969: creat d9/d21/d31/d5c/d92/d98/f14f x:0 0 0 2024-03-21T10:09:13.417 INFO:tasks.workunit.client.1.smithi174.stdout:6/871: read - d0/d3/d5/d6/d42/d10a/daf/fea zero size 2024-03-21T10:09:13.418 INFO:tasks.workunit.client.1.smithi174.stdout:3/948: sync 2024-03-21T10:09:13.419 INFO:tasks.workunit.client.1.smithi174.stdout:9/978: sync 2024-03-21T10:09:13.427 INFO:tasks.workunit.client.1.smithi174.stdout:2/948: mkdir d5/d25/d27/d74/d147 0 2024-03-21T10:09:13.428 INFO:tasks.workunit.client.1.smithi174.stdout:8/944: dread d9/d2a/d39/d58/dbb/fd8 [0,4194304] 0 2024-03-21T10:09:13.432 INFO:tasks.workunit.client.1.smithi174.stdout:9/979: read d2/d11/d7d/fde [372087,44154] 0 2024-03-21T10:09:13.433 INFO:tasks.workunit.client.1.smithi174.stdout:3/949: mkdir d0/d12/d14/db9/d127 0 2024-03-21T10:09:13.437 INFO:tasks.workunit.client.1.smithi174.stdout:2/949: mkdir d5/d25/d27/d2d/d45/dae/dd6/d85/dad/d148 0 2024-03-21T10:09:13.439 INFO:tasks.workunit.client.1.smithi174.stdout:9/980: chown d2/d11/d1d/d8b/fc8 2 1 2024-03-21T10:09:13.442 INFO:tasks.workunit.client.1.smithi174.stdout:3/950: unlink d0/d12/d14/d98/lfd 0 2024-03-21T10:09:13.444 INFO:tasks.workunit.client.1.smithi174.stdout:8/945: mkdir d9/d2a/d39/d76/d19/d27/d60/daa/dab/dc4/d133 0 2024-03-21T10:09:13.444 INFO:tasks.workunit.client.1.smithi174.stdout:6/872: dwrite d0/fee [0,4194304] 0 2024-03-21T10:09:13.448 INFO:tasks.workunit.client.1.smithi174.stdout:3/951: chown d0/d12/d14/d1a/d44/d65/f10f 4533133 1 2024-03-21T10:09:13.450 INFO:tasks.workunit.client.1.smithi174.stdout:0/970: truncate d9/d21/d75/da1/fee 2737090 0 2024-03-21T10:09:13.453 INFO:tasks.workunit.client.1.smithi174.stdout:2/950: creat d5/d25/d27/d2d/d45/dae/dd6/d4f/da0/def/f149 x:0 0 0 2024-03-21T10:09:13.455 INFO:tasks.workunit.client.1.smithi174.stdout:6/873: rmdir d0/d3/d20/d3d/d81 39 2024-03-21T10:09:13.463 INFO:tasks.workunit.client.1.smithi174.stdout:0/971: creat d9/d35/d4a/d8a/d10c/f150 x:0 0 0 2024-03-21T10:09:13.463 INFO:tasks.workunit.client.1.smithi174.stdout:5/962: dwrite d1/d2f/dc1/f121 [0,4194304] 0 2024-03-21T10:09:13.463 INFO:tasks.workunit.client.1.smithi174.stdout:7/923: dread d0/d4/f42 [0,4194304] 0 2024-03-21T10:09:13.463 INFO:tasks.workunit.client.1.smithi174.stdout:2/951: read d5/d25/d27/d2d/d45/f6e [3742198,28795] 0 2024-03-21T10:09:13.467 INFO:tasks.workunit.client.1.smithi174.stdout:0/972: symlink d9/d1c/d76/d87/dcd/l151 0 2024-03-21T10:09:13.470 INFO:tasks.workunit.client.1.smithi174.stdout:7/924: rename d0 to d0/d4/d10/d149 22 2024-03-21T10:09:13.471 INFO:tasks.workunit.client.1.smithi174.stdout:0/973: readlink d9/d21/d31/d5c/led 0 2024-03-21T10:09:13.475 INFO:tasks.workunit.client.1.smithi174.stdout:2/952: read d5/d25/d27/d2d/d45/d5b/d2f/f9d [281214,91212] 0 2024-03-21T10:09:13.476 INFO:tasks.workunit.client.1.smithi174.stdout:7/925: write d0/d4/d10/d44/d81/f8f [4413003,51336] 0 2024-03-21T10:09:13.479 INFO:tasks.workunit.client.1.smithi174.stdout:8/946: dwrite d9/d2a/fec [0,4194304] 0 2024-03-21T10:09:13.480 INFO:tasks.workunit.client.1.smithi174.stdout:3/952: rename d0/d12/d14/d1a/d44/d56/d76/ld3 to d0/d12/d14/d1a/d27/dff/d110/d11a/d11d/l128 0 2024-03-21T10:09:13.481 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:13.478+0000 7fbdc3c00700 1 -- 172.21.15.153:0/1278120695 --> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbd9c000bf0 con 0x7fbda006e210 2024-03-21T10:09:13.482 INFO:tasks.workunit.client.1.smithi174.stdout:3/953: fsync d0/d12/fdf 0 2024-03-21T10:09:13.483 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:13.482+0000 7fbdb9600700 1 -- 172.21.15.153:0/1278120695 <== mgr.14666 v2:172.21.15.153:6800/2524145044 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+329 (secure 0 0 0) 0x7fbd9c000bf0 con 0x7fbda006e210 2024-03-21T10:09:13.485 INFO:tasks.workunit.client.1.smithi174.stdout:7/926: fdatasync d0/d4/d38/da9/dc6/f144 0 2024-03-21T10:09:13.486 INFO:tasks.workunit.client.1.smithi174.stdout:0/974: creat d9/d21/d31/d5c/d92/d98/db7/f152 x:0 0 0 2024-03-21T10:09:13.486 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:13.485+0000 7fbdc3c00700 1 -- 172.21.15.153:0/1278120695 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7fbda006e210 msgr2=0x7fbda00706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:09:13.486 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:13.485+0000 7fbdc3c00700 1 --2- 172.21.15.153:0/1278120695 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7fbda006e210 0x7fbda00706d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fbda800b650 tx=0x7fbda8009ed0 comp rx=0 tx=0).stop 2024-03-21T10:09:13.486 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:13.485+0000 7fbdc3c00700 1 -- 172.21.15.153:0/1278120695 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbdbc104010 msgr2=0x7fbdbc19ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:09:13.486 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:13.485+0000 7fbdc3c00700 1 --2- 172.21.15.153:0/1278120695 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbdbc104010 0x7fbdbc19ce60 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fbdb400c0c0 tx=0x7fbdb400c480 comp rx=0 tx=0).stop 2024-03-21T10:09:13.487 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:13.485+0000 7fbdc3c00700 1 -- 172.21.15.153:0/1278120695 shutdown_connections 2024-03-21T10:09:13.487 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:13.485+0000 7fbdc3c00700 1 --2- 172.21.15.153:0/1278120695 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7fbda006e210 0x7fbda00706d0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:09:13.487 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:13.485+0000 7fbdc3c00700 1 --2- 172.21.15.153:0/1278120695 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbdbc1016e0 0x7fbdbc19c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:09:13.487 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:13.485+0000 7fbdc3c00700 1 --2- 172.21.15.153:0/1278120695 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbdbc104010 0x7fbdbc19ce60 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:09:13.487 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:13.485+0000 7fbdc3c00700 1 -- 172.21.15.153:0/1278120695 >> 172.21.15.153:0/1278120695 conn(0x7fbdbc0fb030 msgr2=0x7fbdbc0fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:09:13.487 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:13.485+0000 7fbdc3c00700 1 -- 172.21.15.153:0/1278120695 shutdown_connections 2024-03-21T10:09:13.487 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:13.486+0000 7fbdc3c00700 1 -- 172.21.15.153:0/1278120695 wait complete. 2024-03-21T10:09:13.492 INFO:tasks.workunit.client.1.smithi174.stdout:9/981: dread d2/f3c [0,4194304] 0 2024-03-21T10:09:13.498 INFO:tasks.workunit.client.1.smithi174.stdout:9/982: chown d2/d11/d7d/cb5 420 1 2024-03-21T10:09:13.499 INFO:tasks.workunit.client.1.smithi174.stdout:2/953: creat d5/d25/d27/d2d/d45/d5b/d2f/d4b/dd8/dff/f14a x:0 0 0 2024-03-21T10:09:13.503 INFO:tasks.workunit.client.1.smithi174.stdout:0/975: fsync d9/d21/d31/d9f/d79/ffb 0 2024-03-21T10:09:13.505 INFO:tasks.workunit.client.1.smithi174.stdout:9/983: creat d2/d24/d2e/de6/f151 x:0 0 0 2024-03-21T10:09:13.507 INFO:tasks.workunit.client.1.smithi174.stdout:3/954: rmdir d0/d12 39 2024-03-21T10:09:13.509 INFO:tasks.workunit.client.1.smithi174.stdout:2/954: chown d5/d25/d27/d2d/d45/dae/dd6/d85/dad/fb3 583 1 2024-03-21T10:09:13.513 INFO:tasks.workunit.client.1.smithi174.stdout:5/963: dread d1/d2f/d37/df5/d11e/f138 [0,4194304] 0 2024-03-21T10:09:13.514 INFO:tasks.workunit.client.1.smithi174.stdout:8/947: write d9/d53/fe8 [422163,130325] 0 2024-03-21T10:09:13.517 INFO:tasks.workunit.client.1.smithi174.stdout:8/948: readlink d9/d53/l68 0 2024-03-21T10:09:13.528 INFO:tasks.workunit.client.1.smithi174.stdout:5/964: getdents d1/d49/d93 0 2024-03-21T10:09:13.531 INFO:tasks.workunit.client.1.smithi174.stdout:9/984: dwrite d2/d10b/f92 [0,4194304] 0 2024-03-21T10:09:13.531 INFO:tasks.workunit.client.1.smithi174.stdout:8/949: getdents d9/d2a/d4d/db4 0 2024-03-21T10:09:13.535 INFO:tasks.workunit.client.1.smithi174.stdout:7/927: dread d0/d4/d10/d1b/d86/dd2/f114 [0,4194304] 0 2024-03-21T10:09:13.540 INFO:tasks.workunit.client.1.smithi174.stdout:5/965: fsync d1/d49/d4f/d51/d8f/d103/f133 0 2024-03-21T10:09:13.541 INFO:tasks.workunit.client.1.smithi174.stdout:9/985: rmdir d2/d11/d7d 39 2024-03-21T10:09:13.542 INFO:tasks.workunit.client.1.smithi174.stdout:8/950: mknod d9/d53/dc9/de6/dea/d117/c134 0 2024-03-21T10:09:13.544 INFO:tasks.workunit.client.1.smithi174.stdout:0/976: dwrite d9/d21/f37 [0,4194304] 0 2024-03-21T10:09:13.552 INFO:tasks.workunit.client.1.smithi174.stdout:3/955: dread d0/fa [0,4194304] 0 2024-03-21T10:09:13.556 INFO:tasks.workunit.client.1.smithi174.stdout:2/955: dwrite d5/d25/d27/d74/f95 [0,4194304] 0 2024-03-21T10:09:13.558 INFO:tasks.workunit.client.1.smithi174.stdout:7/928: creat d0/d4/d10/d79/db2/dda/d5f/d130/f14a x:0 0 0 2024-03-21T10:09:13.565 INFO:tasks.workunit.client.1.smithi174.stdout:8/951: link d9/d2a/d39/fa0 d9/d2a/d39/d76/d19/d27/d60/daa/f135 0 2024-03-21T10:09:13.568 INFO:tasks.workunit.client.1.smithi174.stdout:2/956: symlink d5/d25/d27/d2d/d45/d5b/d2f/d68/l14b 0 2024-03-21T10:09:13.573 INFO:tasks.workunit.client.1.smithi174.stdout:5/966: dwrite d1/d2f/f117 [0,4194304] 0 2024-03-21T10:09:13.574 INFO:tasks.workunit.client.1.smithi174.stdout:7/929: truncate d0/d4/d38/da9/dc6/f144 847944 0 2024-03-21T10:09:13.578 INFO:tasks.workunit.client.1.smithi174.stdout:0/977: dwrite d9/d21/d81/f136 [0,4194304] 0 2024-03-21T10:09:13.579 INFO:tasks.workunit.client.1.smithi174.stdout:8/952: chown d9/d2a/d39/d76/d19/d27/d60/daa/df2/ffa 3 1 2024-03-21T10:09:13.592 INFO:tasks.workunit.client.1.smithi174.stdout:3/956: dwrite d0/d12/d14/d1a/d44/d65/f10f [0,4194304] 0 2024-03-21T10:09:13.592 INFO:tasks.workunit.client.1.smithi174.stdout:2/957: symlink d5/d25/d27/d2d/d45/dae/dd6/d4f/d65/d77/l14c 0 2024-03-21T10:09:13.600 INFO:tasks.workunit.client.1.smithi174.stdout:3/957: mknod d0/d12/d14/d1a/d27/d7f/c129 0 2024-03-21T10:09:13.602 INFO:tasks.workunit.client.1.smithi174.stdout:5/967: dread d1/ff [8388608,4194304] 0 2024-03-21T10:09:13.607 INFO:tasks.workunit.client.1.smithi174.stdout:9/986: dread d2/d24/d2e/dac/fcc [0,4194304] 0 2024-03-21T10:09:13.607 INFO:tasks.workunit.client.1.smithi174.stdout:5/968: creat d1/d2f/d37/df5/f156 x:0 0 0 2024-03-21T10:09:13.609 INFO:tasks.workunit.client.1.smithi174.stdout:9/987: dread - d2/d11/fe3 zero size 2024-03-21T10:09:13.611 INFO:tasks.workunit.client.1.smithi174.stdout:3/958: rmdir d0/d12/d14/d1a/d27/dff/d110/d11a/d121 0 2024-03-21T10:09:13.612 INFO:tasks.workunit.client.1.smithi174.stdout:5/969: truncate d1/f86 406823 0 2024-03-21T10:09:13.612 INFO:tasks.workunit.client.1.smithi174.stdout:7/930: write d0/d4/d10/d79/db2/dda/d5f/d65/fe8 [16491,65504] 0 2024-03-21T10:09:13.616 INFO:tasks.workunit.client.1.smithi174.stdout:0/978: truncate d9/d1c/d76/d87/faa 47699 0 2024-03-21T10:09:13.622 INFO:tasks.workunit.client.1.smithi174.stdout:5/970: read d1/d16/d1c/f34 [1050359,70647] 0 2024-03-21T10:09:13.622 INFO:tasks.workunit.client.1.smithi174.stdout:8/953: dread d9/fae [0,4194304] 0 2024-03-21T10:09:13.623 INFO:tasks.workunit.client.1.smithi174.stdout:3/959: rename d0/d12/d46/d111 to d0/d12/d14/d67/d12a 0 2024-03-21T10:09:13.624 INFO:tasks.workunit.client.1.smithi174.stdout:8/954: write d9/d2a/d39/f75 [1793254,130766] 0 2024-03-21T10:09:13.632 INFO:tasks.workunit.client.1.smithi174.stdout:5/971: mknod d1/d2f/dc1/d144/c157 0 2024-03-21T10:09:13.635 INFO:tasks.workunit.client.1.smithi174.stdout:8/955: mkdir d9/dc6/df1/d136 0 2024-03-21T10:09:13.639 INFO:tasks.workunit.client.1.smithi174.stdout:5/972: chown d1/d49/d4f/d51/d8f/d103/f14c 29 1 2024-03-21T10:09:13.642 INFO:tasks.workunit.client.1.smithi174.stdout:2/958: dread d5/d25/d27/d2d/d45/dae/dd6/d4f/d65/f92 [0,4194304] 0 2024-03-21T10:09:13.643 INFO:tasks.workunit.client.1.smithi174.stdout:3/960: rename d0/f33 to d0/d12/d14/d1a/d44/d65/d7e/f12b 0 2024-03-21T10:09:13.644 INFO:tasks.workunit.client.1.smithi174.stdout:6/874: sync 2024-03-21T10:09:13.649 INFO:tasks.workunit.client.1.smithi174.stdout:6/875: readlink d0/d3/d20/d3d/lf2 0 2024-03-21T10:09:13.650 INFO:tasks.workunit.client.1.smithi174.stdout:9/988: dwrite d2/d24/d2e/ff9 [0,4194304] 0 2024-03-21T10:09:13.652 INFO:tasks.workunit.client.1.smithi174.stdout:5/973: chown d1/d49/d4f/d51/d8f/d103/l107 0 1 2024-03-21T10:09:13.660 INFO:tasks.workunit.client.1.smithi174.stdout:3/961: rename d0/d12/d14/d4c/f114 to d0/d12/d14/da9/f12c 0 2024-03-21T10:09:13.662 INFO:tasks.workunit.client.1.smithi174.stdout:6/876: symlink d0/d3/d5/d6/l112 0 2024-03-21T10:09:13.663 INFO:tasks.workunit.client.1.smithi174.stdout:7/931: dwrite d0/d4/d10/d79/db2/dda/f63 [0,4194304] 0 2024-03-21T10:09:13.663 INFO:tasks.workunit.client.1.smithi174.stdout:0/979: dread d9/d1c/f20 [4194304,4194304] 0 2024-03-21T10:09:13.667 INFO:tasks.workunit.client.1.smithi174.stdout:5/974: mknod d1/d49/d93/d9c/c158 0 2024-03-21T10:09:13.668 INFO:tasks.workunit.client.1.smithi174.stdout:2/959: truncate d5/d25/d27/d2d/d45/dae/dd6/d4f/d65/d77/f111 698014 0 2024-03-21T10:09:13.672 INFO:tasks.workunit.client.1.smithi174.stdout:2/960: write d5/d25/d27/d2d/d79/df5/f100 [560998,58604] 0 2024-03-21T10:09:13.674 INFO:tasks.workunit.client.1.smithi174.stdout:5/975: read d1/d16/d2e/d66/d8b/f36 [2722448,75929] 0 2024-03-21T10:09:13.674 INFO:tasks.workunit.client.1.smithi174.stdout:9/989: link d2/d11/d1d/d4b/c89 d2/d11/d1d/d2a/df2/c152 0 2024-03-21T10:09:13.677 INFO:tasks.workunit.client.1.smithi174.stdout:7/932: creat d0/d4/d10/d1b/d86/dd2/f14b x:0 0 0 2024-03-21T10:09:13.678 INFO:tasks.workunit.client.1.smithi174.stdout:3/962: truncate d0/d12/d14/da9/de7/db7/d7d/fe5 52670 0 2024-03-21T10:09:13.679 INFO:tasks.workunit.client.1.smithi174.stdout:9/990: dread - d2/d10b/fea zero size 2024-03-21T10:09:13.680 INFO:tasks.workunit.client.1.smithi174.stdout:7/933: chown d0/d4/fd 15087 1 2024-03-21T10:09:13.681 INFO:tasks.workunit.client.1.smithi174.stdout:3/963: fsync d0/d12/d14/db9/f115 0 2024-03-21T10:09:13.683 INFO:tasks.workunit.client.1.smithi174.stdout:7/934: chown d0/d4/d10/d44/d81/d9a/dc0 138957761 1 2024-03-21T10:09:13.688 INFO:tasks.workunit.client.1.smithi174.stdout:3/964: creat d0/d12/d14/d1a/d27/d4f/f12d x:0 0 0 2024-03-21T10:09:13.689 INFO:tasks.workunit.client.1.smithi174.stdout:0/980: dread d9/d21/d23/da5/f6f [0,4194304] 0 2024-03-21T10:09:13.691 INFO:tasks.workunit.client.1.smithi174.stdout:2/961: creat d5/d25/d27/d2d/d45/dae/dd6/d7b/d81/da1/f14d x:0 0 0 2024-03-21T10:09:13.691 INFO:tasks.workunit.client.1.smithi174.stdout:8/956: dread d9/d2a/d39/d76/d19/d27/d60/daa/dab/dc4/fda [0,4194304] 0 2024-03-21T10:09:13.695 INFO:tasks.workunit.client.1.smithi174.stdout:0/981: mknod d9/d21/d23/d80/c153 0 2024-03-21T10:09:13.698 INFO:tasks.workunit.client.1.smithi174.stdout:8/957: fdatasync d9/d2a/d39/d76/f70 0 2024-03-21T10:09:13.702 INFO:tasks.workunit.client.1.smithi174.stdout:0/982: dread d9/d21/d75/da1/fb8 [0,4194304] 0 2024-03-21T10:09:13.702 INFO:tasks.workunit.client.1.smithi174.stdout:8/958: stat d9/d2a/d39/d76/f20 0 2024-03-21T10:09:13.703 INFO:tasks.workunit.client.1.smithi174.stdout:2/962: rmdir d5/d25/d27/d2d/d45/d5b 39 2024-03-21T10:09:13.706 INFO:tasks.workunit.client.1.smithi174.stdout:6/877: truncate d0/d3/d20/d3d/ffd 7781570 0 2024-03-21T10:09:13.707 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:13.705+0000 7f1d69600700 1 -- 172.21.15.153:0/3458419623 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1d64101db0 msgr2=0x7f1d64102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:09:13.707 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:13.705+0000 7f1d69600700 1 --2- 172.21.15.153:0/3458419623 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1d64101db0 0x7f1d64102210 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f1d50009910 tx=0x7f1d50009c20 comp rx=0 tx=0).stop 2024-03-21T10:09:13.707 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:13.706+0000 7f1d69600700 1 -- 172.21.15.153:0/3458419623 shutdown_connections 2024-03-21T10:09:13.707 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:13.706+0000 7f1d69600700 1 --2- 172.21.15.153:0/3458419623 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1d64101db0 0x7f1d64102210 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:09:13.707 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:13.706+0000 7f1d69600700 1 --2- 172.21.15.153:0/3458419623 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1d64100bb0 0x7f1d64100fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:09:13.707 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:13.706+0000 7f1d69600700 1 -- 172.21.15.153:0/3458419623 >> 172.21.15.153:0/3458419623 conn(0x7f1d640fc110 msgr2=0x7f1d640fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:09:13.707 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:13.706+0000 7f1d69600700 1 -- 172.21.15.153:0/3458419623 shutdown_connections 2024-03-21T10:09:13.708 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:13.706+0000 7f1d69600700 1 -- 172.21.15.153:0/3458419623 wait complete. 2024-03-21T10:09:13.708 INFO:tasks.workunit.client.1.smithi174.stdout:9/991: write d2/d11/d1d/fa4 [474626,43183] 0 2024-03-21T10:09:13.708 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:13.707+0000 7f1d69600700 1 Processor -- start 2024-03-21T10:09:13.708 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:13.707+0000 7f1d69600700 1 -- start start 2024-03-21T10:09:13.710 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:13.708+0000 7f1d69600700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1d64100bb0 0x7f1d6419edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:09:13.710 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:13.708+0000 7f1d69600700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1d64101db0 0x7f1d6419f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:09:13.710 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:13.708+0000 7f1d69600700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d6419f9f0 con 0x7f1d64100bb0 2024-03-21T10:09:13.710 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:13.708+0000 7f1d69600700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d6419fb60 con 0x7f1d64101db0 2024-03-21T10:09:13.710 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:13.708+0000 7f1d62a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1d64100bb0 0x7f1d6419edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:09:13.710 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:13.708+0000 7f1d62a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1d64100bb0 0x7f1d6419edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42106/0 (socket says 172.21.15.153:42106) 2024-03-21T10:09:13.710 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:13.708+0000 7f1d62a00700 1 -- 172.21.15.153:0/2696647580 learned_addr learned my addr 172.21.15.153:0/2696647580 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:09:13.711 INFO:tasks.workunit.client.1.smithi174.stdout:9/992: chown d2/d11/d1d/d2a/df2 2911170 1 2024-03-21T10:09:13.712 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:13.709+0000 7f1d62000700 1 --2- 172.21.15.153:0/2696647580 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1d64101db0 0x7f1d6419f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:09:13.712 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:13.709+0000 7f1d62a00700 1 -- 172.21.15.153:0/2696647580 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1d64101db0 msgr2=0x7f1d6419f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:09:13.712 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:13.709+0000 7f1d62a00700 1 --2- 172.21.15.153:0/2696647580 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1d64101db0 0x7f1d6419f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:09:13.712 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:13.709+0000 7f1d62a00700 1 -- 172.21.15.153:0/2696647580 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d500095f0 con 0x7f1d64100bb0 2024-03-21T10:09:13.712 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:13.709+0000 7f1d62000700 1 --2- 172.21.15.153:0/2696647580 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1d64101db0 0x7f1d6419f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:09:13.712 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:13.710+0000 7f1d62a00700 1 --2- 172.21.15.153:0/2696647580 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1d64100bb0 0x7f1d6419edb0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f1d5400ecb0 tx=0x7f1d54009ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:09:13.713 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:13.710+0000 7f1d5b400700 1 -- 172.21.15.153:0/2696647580 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1d54009790 con 0x7f1d64100bb0 2024-03-21T10:09:13.713 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:13.710+0000 7f1d5b400700 1 -- 172.21.15.153:0/2696647580 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f1d5400c8e0 con 0x7f1d64100bb0 2024-03-21T10:09:13.713 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:13.710+0000 7f1d5b400700 1 -- 172.21.15.153:0/2696647580 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1d54003d70 con 0x7f1d64100bb0 2024-03-21T10:09:13.713 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:13.710+0000 7f1d69600700 1 -- 172.21.15.153:0/2696647580 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d641a4280 con 0x7f1d64100bb0 2024-03-21T10:09:13.713 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:13.710+0000 7f1d69600700 1 -- 172.21.15.153:0/2696647580 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d641a47d0 con 0x7f1d64100bb0 2024-03-21T10:09:13.714 INFO:tasks.workunit.client.1.smithi174.stdout:0/983: creat d9/d21/d31/d5c/d92/d98/db7/ddc/d103/f154 x:0 0 0 2024-03-21T10:09:13.715 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:13.713+0000 7f1d5b400700 1 -- 172.21.15.153:0/2696647580 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 29) v1 ==== 92174+0+0 (secure 0 0 0) 0x7f1d54018430 con 0x7f1d64100bb0 2024-03-21T10:09:13.716 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:13.714+0000 7f1d69600700 1 -- 172.21.15.153:0/2696647580 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d6406b040 con 0x7f1d64100bb0 2024-03-21T10:09:13.716 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:13.715+0000 7f1d5b400700 1 --2- 172.21.15.153:0/2696647580 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f1d48072680 0x7f1d48074b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:09:13.718 INFO:tasks.workunit.client.1.smithi174.stdout:8/959: link d9/d2a/d39/d76/d19/d27/f64 d9/d2a/d39/d76/d19/d12f/f137 0 2024-03-21T10:09:13.719 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:13.715+0000 7f1d5b400700 1 -- 172.21.15.153:0/2696647580 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f1d54014070 con 0x7f1d64100bb0 2024-03-21T10:09:13.719 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:13.716+0000 7f1d62000700 1 --2- 172.21.15.153:0/2696647580 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f1d48072680 0x7f1d48074b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:09:13.720 INFO:tasks.workunit.client.1.smithi174.stdout:3/965: getdents d0/d12/d14/d1a/d44/d57 0 2024-03-21T10:09:13.720 INFO:tasks.workunit.client.1.smithi174.stdout:9/993: fsync d2/d11/d1d/d5e/f116 0 2024-03-21T10:09:13.720 INFO:tasks.workunit.client.1.smithi174.stdout:7/935: dwrite d0/d4/d38/fd9 [0,4194304] 0 2024-03-21T10:09:13.720 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:13.718+0000 7f1d62000700 1 --2- 172.21.15.153:0/2696647580 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f1d48072680 0x7f1d48074b40 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1d50009fd0 tx=0x7f1d50005940 comp rx=0 tx=0).ready entity=mgr.14666 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:09:13.725 INFO:tasks.workunit.client.1.smithi174.stdout:2/963: dwrite d5/d25/d27/f2a [0,4194304] 0 2024-03-21T10:09:13.725 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:13.722+0000 7f1d5b400700 1 -- 172.21.15.153:0/2696647580 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185454 (secure 0 0 0) 0x7f1d540578e0 con 0x7f1d64100bb0 2024-03-21T10:09:13.726 INFO:tasks.workunit.client.1.smithi174.stdout:9/994: creat d2/d11/d60/db2/f153 x:0 0 0 2024-03-21T10:09:13.729 INFO:tasks.workunit.client.1.smithi174.stdout:0/984: dread d9/d35/dd2/fe6 [0,4194304] 0 2024-03-21T10:09:13.733 INFO:tasks.workunit.client.1.smithi174.stdout:7/936: symlink d0/d4/d10/d79/db2/dda/d5f/d130/l14c 0 2024-03-21T10:09:13.735 INFO:tasks.workunit.client.1.smithi174.stdout:8/960: rename d9/d53/c92 to d9/d2a/c138 0 2024-03-21T10:09:13.744 INFO:tasks.workunit.client.1.smithi174.stdout:7/937: rename d0/d11b/dfe/l138 to d0/d4/d121/l14d 0 2024-03-21T10:09:13.746 INFO:tasks.workunit.client.1.smithi174.stdout:7/938: read - d0/d4/d38/f128 zero size 2024-03-21T10:09:13.746 INFO:tasks.workunit.client.1.smithi174.stdout:3/966: getdents d0/d12/d14/d1a/d27/dff 0 2024-03-21T10:09:13.749 INFO:tasks.workunit.client.1.smithi174.stdout:6/878: dread d0/d10c/fd2 [0,4194304] 0 2024-03-21T10:09:13.750 INFO:tasks.workunit.client.1.smithi174.stdout:3/967: chown d0/l9 1 1 2024-03-21T10:09:13.751 INFO:tasks.workunit.client.1.smithi174.stdout:8/961: rename d9/d53/f6f to d9/d2a/d39/d76/d19/d27/d60/daa/dab/dc4/f139 0 2024-03-21T10:09:13.751 INFO:tasks.workunit.client.1.smithi174.stdout:0/985: link d9/d21/d31/d9f/c69 d9/d21/d31/d5c/c155 0 2024-03-21T10:09:13.751 INFO:tasks.workunit.client.1.smithi174.stdout:9/995: write d2/d10a/f12a [1333582,72622] 0 2024-03-21T10:09:13.753 INFO:tasks.workunit.client.1.smithi174.stdout:7/939: creat d0/d4/d10/d79/db2/f14e x:0 0 0 2024-03-21T10:09:13.754 INFO:tasks.workunit.client.1.smithi174.stdout:2/964: symlink d5/d25/d27/d2d/d45/dae/dd6/d85/dad/d148/l14e 0 2024-03-21T10:09:13.757 INFO:tasks.workunit.client.1.smithi174.stdout:3/968: read d0/d12/d14/d4c/d7a/ffc [95011,14768] 0 2024-03-21T10:09:13.757 INFO:tasks.workunit.client.1.smithi174.stdout:8/962: readlink d9/d2a/d39/d76/d19/d27/l4c 0 2024-03-21T10:09:13.759 INFO:tasks.workunit.client.1.smithi174.stdout:7/940: chown d0/d4/d10/f62 2563438 1 2024-03-21T10:09:13.759 INFO:tasks.workunit.client.1.smithi174.stdout:6/879: mkdir d0/d3/d20/d3d/d81/da5/d113 0 2024-03-21T10:09:13.763 INFO:tasks.workunit.client.1.smithi174.stdout:8/963: fsync d9/d53/fbc 0 2024-03-21T10:09:13.766 INFO:tasks.workunit.client.1.smithi174.stdout:3/969: unlink d0/d12/d14/d67/dc8/cdd 0 2024-03-21T10:09:13.769 INFO:tasks.workunit.client.1.smithi174.stdout:8/964: stat d9/d2a/d39/d58/fd9 0 2024-03-21T10:09:13.770 INFO:tasks.workunit.client.1.smithi174.stdout:6/880: mknod d0/d3/d5/d6/d42/d10a/d8f/c114 0 2024-03-21T10:09:13.774 INFO:tasks.workunit.client.1.smithi174.stdout:9/996: dwrite d2/d24/d2e/ffe [0,4194304] 0 2024-03-21T10:09:13.776 INFO:tasks.workunit.client.1.smithi174.stdout:9/997: readlink d2/d11/d60/db2/ld4 0 2024-03-21T10:09:13.776 INFO:tasks.workunit.client.1.smithi174.stdout:8/965: symlink d9/d2a/d103/l13a 0 2024-03-21T10:09:13.778 INFO:tasks.workunit.client.1.smithi174.stdout:6/881: truncate d0/d3/d5/d6/f70 927328 0 2024-03-21T10:09:13.781 INFO:tasks.workunit.client.1.smithi174.stdout:9/998: rmdir d2/d24/d2e/de6 39 2024-03-21T10:09:13.786 INFO:tasks.workunit.client.1.smithi174.stdout:8/966: symlink d9/d2a/d39/l13b 0 2024-03-21T10:09:13.786 INFO:tasks.workunit.client.1.smithi174.stdout:9/999: stat d2/d11/d1d/f33 0 2024-03-21T10:09:13.786 INFO:tasks.workunit.client.1.smithi174.stdout:6/882: mknod d0/d3/d5/df/d16/da9/db7/c115 0 2024-03-21T10:09:13.795 INFO:tasks.workunit.client.1.smithi174.stdout:2/965: dwrite d5/d25/d27/d2d/d45/d5b/d2f/f9d [0,4194304] 0 2024-03-21T10:09:13.795 INFO:tasks.workunit.client.1.smithi174.stdout:0/986: dwrite d9/f33 [0,4194304] 0 2024-03-21T10:09:13.801 INFO:tasks.workunit.client.1.smithi174.stdout:8/967: creat d9/d53/dc9/de6/dea/d117/f13c x:0 0 0 2024-03-21T10:09:13.806 INFO:tasks.workunit.client.1.smithi174.stdout:0/987: write d9/ff4 [2405909,35139] 0 2024-03-21T10:09:13.810 INFO:tasks.workunit.client.1.smithi174.stdout:0/988: fdatasync d9/da2/f125 0 2024-03-21T10:09:13.815 INFO:tasks.workunit.client.1.smithi174.stdout:2/966: creat d5/d25/d27/d2d/d45/f14f x:0 0 0 2024-03-21T10:09:13.816 INFO:tasks.workunit.client.1.smithi174.stdout:3/970: dwrite d0/d12/d14/da9/de7/db7/f1f [4194304,4194304] 0 2024-03-21T10:09:13.816 INFO:tasks.workunit.client.1.smithi174.stdout:8/968: creat d9/d2a/d39/d76/d19/d27/d60/daa/dab/dc4/f13d x:0 0 0 2024-03-21T10:09:13.819 INFO:tasks.workunit.client.1.smithi174.stdout:0/989: write d9/d21/d31/d5c/d86/ff3 [1549428,23187] 0 2024-03-21T10:09:13.822 INFO:tasks.workunit.client.1.smithi174.stdout:6/883: dwrite d0/d3/d5/df/d16/f34 [0,4194304] 0 2024-03-21T10:09:13.823 INFO:tasks.workunit.client.1.smithi174.stdout:0/990: mkdir d9/dd8/d156 0 2024-03-21T10:09:13.835 INFO:tasks.workunit.client.1.smithi174.stdout:3/971: creat d0/d12/d14/d67/da1/da7/f12e x:0 0 0 2024-03-21T10:09:13.835 INFO:tasks.workunit.client.1.smithi174.stdout:6/884: truncate d0/d3/d5/f1c 4422365 0 2024-03-21T10:09:13.837 INFO:tasks.workunit.client.1.smithi174.stdout:0/991: creat d9/d21/d23/da5/ddf/d12e/f157 x:0 0 0 2024-03-21T10:09:13.839 INFO:tasks.workunit.client.1.smithi174.stdout:6/885: rmdir d0/df5 39 2024-03-21T10:09:13.842 INFO:tasks.workunit.client.1.smithi174.stdout:0/992: truncate d9/d21/d31/d5c/d92/d98/db7/f152 230307 0 2024-03-21T10:09:13.845 INFO:tasks.workunit.client.1.smithi174.stdout:0/993: readlink d9/d21/d31/d9f/d79/d82/dbb/lec 0 2024-03-21T10:09:13.846 INFO:tasks.workunit.client.1.smithi174.stdout:3/972: rename d0/d12/d14/d1a/d27/dff/d110/d11a/d11d to d0/d12/d14/d1a/d44/d12f 0 2024-03-21T10:09:13.847 INFO:tasks.workunit.client.1.smithi174.stdout:0/994: rmdir d9/d1c/d76 39 2024-03-21T10:09:13.849 INFO:tasks.workunit.client.1.smithi174.stdout:3/973: rename d0/d12/d14/d1a to d0/d12/d14/d1a/d27/dff/d110/d130 22 2024-03-21T10:09:13.854 INFO:tasks.workunit.client.1.smithi174.stdout:2/967: dwrite d5/d25/d27/d2d/d45/dae/dd6/d85/dad/fb3 [0,4194304] 0 2024-03-21T10:09:13.858 INFO:tasks.workunit.client.1.smithi174.stdout:3/974: write d0/d12/d14/d67/da1/da7/f12e [264800,77699] 0 2024-03-21T10:09:13.864 INFO:tasks.workunit.client.1.smithi174.stdout:8/969: dread d9/dc6/f8f [0,4194304] 0 2024-03-21T10:09:13.864 INFO:tasks.workunit.client.1.smithi174.stdout:3/975: mkdir d0/d12/d14/d1a/d27/d4f/d131 0 2024-03-21T10:09:13.866 INFO:tasks.workunit.client.1.smithi174.stdout:2/968: mkdir d5/d25/d27/d2d/d45/d5b/d2f/d66/db5/d144/d150 0 2024-03-21T10:09:13.868 INFO:tasks.workunit.client.1.smithi174.stdout:8/970: creat d9/dc6/f13e x:0 0 0 2024-03-21T10:09:13.872 INFO:tasks.workunit.client.1.smithi174.stdout:3/976: creat d0/d12/d14/d1a/d44/d94/dd9/f132 x:0 0 0 2024-03-21T10:09:13.872 INFO:tasks.workunit.client.1.smithi174.stdout:8/971: chown d9/d2a/d39/d76/d19/d27/f41 2472 1 2024-03-21T10:09:13.872 INFO:tasks.workunit.client.1.smithi174.stdout:2/969: getdents d5/d25/d27/d2d/db4 0 2024-03-21T10:09:13.876 INFO:tasks.workunit.client.1.smithi174.stdout:8/972: rmdir d9/d53/dc9/de6 39 2024-03-21T10:09:13.880 INFO:tasks.workunit.client.1.smithi174.stdout:2/970: write d5/d25/d27/d2d/d45/dae/f125 [958071,89331] 0 2024-03-21T10:09:13.882 INFO:tasks.workunit.client.1.smithi174.stdout:6/886: dread d0/d3/d5/d6/d42/d10a/d56/d5e/d91/fe7 [0,4194304] 0 2024-03-21T10:09:13.883 INFO:tasks.workunit.client.1.smithi174.stdout:8/973: creat d9/d2a/f13f x:0 0 0 2024-03-21T10:09:13.885 INFO:tasks.workunit.client.1.smithi174.stdout:8/974: fsync d9/d53/fbc 0 2024-03-21T10:09:13.886 INFO:tasks.workunit.client.1.smithi174.stdout:0/995: dread d9/f1d [4194304,4194304] 0 2024-03-21T10:09:13.888 INFO:tasks.workunit.client.1.smithi174.stdout:3/977: write d0/d12/d14/d4c/d7a/d9e/fbd [164989,38039] 0 2024-03-21T10:09:13.890 INFO:tasks.workunit.client.1.smithi174.stdout:3/978: chown d0/d12/d14/d1a/d27/d7f 24 1 2024-03-21T10:09:13.891 INFO:tasks.workunit.client.1.smithi174.stdout:2/971: rename d5/d25/d27/d2d/d45/d5b/d2f/d4b/c130 to d5/d25/d27/d2d/d45/dae/dd6/d4f/d65/d77/c151 0 2024-03-21T10:09:13.892 INFO:tasks.workunit.client.1.smithi174.stdout:6/887: truncate d0/d3/d5/d6/d42/d10a/d56/d5e/f86 887628 0 2024-03-21T10:09:13.893 INFO:tasks.workunit.client.1.smithi174.stdout:2/972: stat d5/d25/d27/d2d/d45/d5b/d2f/l5c 0 2024-03-21T10:09:13.895 INFO:tasks.workunit.client.1.smithi174.stdout:3/979: write d0/d12/d14/d1a/d44/d94/dd9/f109 [2442439,84068] 0 2024-03-21T10:09:13.895 INFO:tasks.workunit.client.1.smithi174.stdout:8/975: creat d9/d2a/d39/f140 x:0 0 0 2024-03-21T10:09:13.900 INFO:tasks.workunit.client.1.smithi174.stdout:2/973: fsync d5/d25/d27/d2d/d45/d5b/d63/f61 0 2024-03-21T10:09:13.907 INFO:tasks.workunit.client.1.smithi174.stdout:5/976: sync 2024-03-21T10:09:13.907 INFO:tasks.workunit.client.1.smithi174.stdout:7/941: sync 2024-03-21T10:09:13.907 INFO:tasks.workunit.client.1.smithi174.stdout:8/976: creat d9/d2a/d39/de1/f141 x:0 0 0 2024-03-21T10:09:13.907 INFO:tasks.workunit.client.1.smithi174.stdout:0/996: dwrite d9/fc1 [4194304,4194304] 0 2024-03-21T10:09:13.912 INFO:tasks.workunit.client.1.smithi174.stdout:2/974: fsync d5/d25/d27/d2d/d45/d5b/f1c 0 2024-03-21T10:09:13.917 INFO:tasks.workunit.client.1.smithi174.stdout:8/977: write d9/d2a/d39/d76/d19/d27/d60/fb5 [4473877,70839] 0 2024-03-21T10:09:13.917 INFO:tasks.workunit.client.1.smithi174.stdout:5/977: mkdir d1/d16/d1c/d148/d159 0 2024-03-21T10:09:13.919 INFO:tasks.workunit.client.1.smithi174.stdout:0/997: rename d9/d35/l8b to d9/d21/d23/da5/ddf/d12e/l158 0 2024-03-21T10:09:13.920 INFO:tasks.workunit.client.1.smithi174.stdout:5/978: chown d1/d16/d1c/f6d 60 1 2024-03-21T10:09:13.921 INFO:tasks.workunit.client.1.smithi174.stdout:3/980: dwrite d0/f13 [0,4194304] 0 2024-03-21T10:09:13.923 INFO:tasks.workunit.client.1.smithi174.stdout:8/978: write d9/dc6/f13e [495737,55454] 0 2024-03-21T10:09:13.925 INFO:tasks.workunit.client.1.smithi174.stdout:6/888: dread d0/d3/d5/d21/f9c [0,4194304] 0 2024-03-21T10:09:13.927 INFO:tasks.workunit.client.1.smithi174.stdout:5/979: unlink d1/d16/d1c/f34 0 2024-03-21T10:09:13.930 INFO:tasks.workunit.client.1.smithi174.stdout:8/979: truncate d9/fae 3500446 0 2024-03-21T10:09:13.935 INFO:tasks.workunit.client.1.smithi174.stdout:2/975: getdents d5/d25/d27/d2d/d45/dae/dd6/d85 0 2024-03-21T10:09:13.935 INFO:tasks.workunit.client.1.smithi174.stdout:3/981: mkdir d0/d12/d14/d4c/d7a/d9e/dee/d133 0 2024-03-21T10:09:13.939 INFO:tasks.workunit.client.1.smithi174.stdout:0/998: link d9/d35/dd2/f14e d9/d21/d31/d5c/d92/d138/f159 0 2024-03-21T10:09:13.941 INFO:tasks.workunit.client.1.smithi174.stdout:0/999: dread - d9/d21/d5f/f139 zero size 2024-03-21T10:09:13.942 INFO:tasks.workunit.client.1.smithi174.stdout:7/942: dwrite d0/d4/d10/d79/db2/dda/d5f/d65/fb5 [0,4194304] 0 2024-03-21T10:09:13.942 INFO:tasks.workunit.client.1.smithi174.stdout:8/980: write d9/d2a/d39/d76/d19/d27/d60/daa/dab/dc4/fda [4907999,124747] 0 2024-03-21T10:09:13.943 INFO:tasks.workunit.client.1.smithi174.stdout:3/982: rmdir d0/d12/d14/d1a/d27/d4f 39 2024-03-21T10:09:13.948 INFO:tasks.workunit.client.1.smithi174.stdout:6/889: creat d0/d3/d20/d3d/f116 x:0 0 0 2024-03-21T10:09:13.948 INFO:tasks.workunit.client.1.smithi174.stdout:2/976: symlink d5/d25/d27/d2d/d45/l152 0 2024-03-21T10:09:13.950 INFO:tasks.workunit.client.1.smithi174.stdout:8/981: symlink d9/d2a/d39/d58/dbb/d10f/l142 0 2024-03-21T10:09:13.951 INFO:tasks.workunit.client.1.smithi174.stdout:7/943: write d0/d4/d38/da9/fe2 [227180,122972] 0 2024-03-21T10:09:13.953 INFO:tasks.workunit.client.1.smithi174.stdout:6/890: write d0/d3/d20/d6d/ff9 [175458,67441] 0 2024-03-21T10:09:13.954 INFO:tasks.workunit.client.1.smithi174.stdout:3/983: creat d0/d12/d14/d67/f134 x:0 0 0 2024-03-21T10:09:13.960 INFO:tasks.workunit.client.1.smithi174.stdout:5/980: truncate d1/d49/d4f/d51/f56 6545357 0 2024-03-21T10:09:13.960 INFO:tasks.workunit.client.1.smithi174.stdout:7/944: mkdir d0/d4/d10/d79/d14f 0 2024-03-21T10:09:13.963 INFO:tasks.workunit.client.1.smithi174.stdout:7/945: write d0/d4/d121/f12b [891958,124728] 0 2024-03-21T10:09:13.965 INFO:tasks.workunit.client.1.smithi174.stdout:5/981: unlink d1/d16/le0 0 2024-03-21T10:09:13.966 INFO:tasks.workunit.client.1.smithi174.stdout:2/977: creat d5/d25/d27/d2d/d45/dae/dd6/d90/da9/d123/f153 x:0 0 0 2024-03-21T10:09:13.966 INFO:tasks.workunit.client.1.smithi174.stdout:8/982: mknod d9/d2a/d39/d76/d19/d27/d60/daa/dab/dc4/d133/c143 0 2024-03-21T10:09:13.970 INFO:tasks.workunit.client.1.smithi174.stdout:5/982: truncate d1/d5c/d73/de3/f13c 738582 0 2024-03-21T10:09:13.970 INFO:tasks.workunit.client.1.smithi174.stdout:3/984: getdents d0/d12/d14/d4c/d7a/d9e/dee 0 2024-03-21T10:09:13.970 INFO:tasks.workunit.client.1.smithi174.stdout:7/946: truncate d0/d4/d10/d1b/d86/dea/df4/d123/f10c 730923 0 2024-03-21T10:09:13.971 INFO:tasks.workunit.client.1.smithi174.stdout:7/947: fdatasync d0/d4/d10/d79/f10e 0 2024-03-21T10:09:13.974 INFO:tasks.workunit.client.1.smithi174.stdout:2/978: chown d5/d25/d27/d2d/d45/d5b/d2f/c6b 49140 1 2024-03-21T10:09:13.975 INFO:tasks.workunit.client.1.smithi174.stdout:6/891: rename d0/df5/c10e to d0/d3/d5/df/d66/c117 0 2024-03-21T10:09:13.976 INFO:tasks.workunit.client.1.smithi174.stdout:7/948: symlink d0/d4/d10/d44/d81/d9a/dc0/l150 0 2024-03-21T10:09:13.976 INFO:tasks.workunit.client.1.smithi174.stdout:2/979: fsync d5/d25/d27/d2d/d45/d5b/d63/de1/f128 0 2024-03-21T10:09:13.978 INFO:tasks.workunit.client.1.smithi174.stdout:3/985: write d0/d12/d14/d1a/d27/f2c [3332431,52787] 0 2024-03-21T10:09:13.979 INFO:tasks.workunit.client.1.smithi174.stdout:8/983: rename d9/d2a/d39/de1/c11b to d9/d2a/d39/d76/d19/d12f/c144 0 2024-03-21T10:09:13.981 INFO:tasks.workunit.client.1.smithi174.stdout:7/949: unlink d0/d4/d38/da9/fe2 0 2024-03-21T10:09:14.020 INFO:tasks.workunit.client.1.smithi174.stdout:3/986: readlink d0/d12/d46/lba 0 2024-03-21T10:09:14.285 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:14.095+0000 7f1d69600700 1 -- 172.21.15.153:0/2696647580 --> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f1d64106700 con 0x7f1d48072680 2024-03-21T10:09:14.285 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:14.113+0000 7f1d5b400700 1 -- 172.21.15.153:0/2696647580 <== mgr.14666 v2:172.21.15.153:6800/2524145044 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f1d64106700 con 0x7f1d48072680 2024-03-21T10:09:14.286 INFO:teuthology.orchestra.run.smithi153.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-21T10:09:14.286 INFO:teuthology.orchestra.run.smithi153.stdout:alertmanager.smithi153 smithi153 *:9093,9094 running (9m) 13s ago 11m 13.8M - 0.25.0 c8568f914cd2 2cfcc225c7e6 2024-03-21T10:09:14.286 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi153 smithi153 running (12m) 13s ago 12m 7894k - 18.2.1 5be31c24972a 5b61994396a6 2024-03-21T10:09:14.287 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi174 smithi174 running (10m) 34s ago 10m 7814k - 18.2.1 5be31c24972a 4f7a5fc3ac28 2024-03-21T10:09:14.287 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi153 smithi153 running (12m) 13s ago 12m 6665k - 18.2.1 5be31c24972a 90a8b8ea03d1 2024-03-21T10:09:14.287 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi174 smithi174 running (10m) 34s ago 10m 6665k - 18.2.1 5be31c24972a a4c929e4dc77 2024-03-21T10:09:14.287 INFO:teuthology.orchestra.run.smithi153.stdout:grafana.smithi153 smithi153 *:3000 running (9m) 13s ago 11m 74.6M - 9.4.7 954c08fa6188 3af5800ecfef 2024-03-21T10:09:14.287 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.iuyluj smithi153 running (4m) 13s ago 4m 140M - 18.2.1 5be31c24972a 02f667cee979 2024-03-21T10:09:14.287 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.zipckr smithi153 running (4m) 13s ago 4m 138M - 18.2.1 5be31c24972a 68a853929f8f 2024-03-21T10:09:14.287 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.nhozag smithi174 running (4m) 34s ago 4m 119M - 18.2.1 5be31c24972a 2fee4000308f 2024-03-21T10:09:14.287 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.yyimst smithi174 running (4m) 34s ago 4m 117M - 18.2.1 5be31c24972a d78a729220d1 2024-03-21T10:09:14.287 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi153.tluxni smithi153 *:8443,9283,8765 running (62s) 13s ago 13m 476M - 19.0.0-2231-g2edfee68 237e43c7a65f 7ffee0e55382 2024-03-21T10:09:14.287 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi174.qpcwat smithi174 *:8443,9283,8765 running (116s) 34s ago 10m 99.0M - 19.0.0-2231-g2edfee68 237e43c7a65f 2f7934875e11 2024-03-21T10:09:14.287 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi153 smithi153 running (13m) 13s ago 13m 51.4M 2048M 18.2.1 5be31c24972a 813963a95d69 2024-03-21T10:09:14.288 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi174 smithi174 running (10m) 34s ago 10m 40.9M 2048M 18.2.1 5be31c24972a 99204923eec6 2024-03-21T10:09:14.288 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi153 smithi153 *:9100 running (11m) 13s ago 11m 7650k - 1.5.0 0da6a335fe13 babbc29a0eaf 2024-03-21T10:09:14.288 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi174 smithi174 *:9100 running (10m) 34s ago 10m 8166k - 1.5.0 0da6a335fe13 86841435042a 2024-03-21T10:09:14.288 INFO:teuthology.orchestra.run.smithi153.stdout:osd.0 smithi153 running (9m) 13s ago 9m 102M 1217M 18.2.1 5be31c24972a 28912a2319b4 2024-03-21T10:09:14.288 INFO:teuthology.orchestra.run.smithi153.stdout:osd.1 smithi153 running (8m) 13s ago 8m 111M 1217M 18.2.1 5be31c24972a ca60681e2314 2024-03-21T10:09:14.288 INFO:teuthology.orchestra.run.smithi153.stdout:osd.2 smithi153 running (8m) 13s ago 8m 103M 1217M 18.2.1 5be31c24972a 8db799f8962a 2024-03-21T10:09:14.288 INFO:teuthology.orchestra.run.smithi153.stdout:osd.3 smithi174 running (7m) 34s ago 7m 132M 2241M 18.2.1 5be31c24972a dafc89e0658d 2024-03-21T10:09:14.288 INFO:teuthology.orchestra.run.smithi153.stdout:osd.4 smithi174 running (6m) 34s ago 6m 118M 2241M 18.2.1 5be31c24972a c4cd15701f05 2024-03-21T10:09:14.288 INFO:teuthology.orchestra.run.smithi153.stdout:osd.5 smithi174 running (6m) 34s ago 6m 111M 2241M 18.2.1 5be31c24972a 3b69886ce22e 2024-03-21T10:09:14.288 INFO:teuthology.orchestra.run.smithi153.stdout:prometheus.smithi153 smithi153 *:9095 running (72s) 13s ago 10m 45.7M - 2.43.0 a07b618ecd1d 08adc18937d9 2024-03-21T10:09:14.290 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:14.117+0000 7f1d69600700 1 -- 172.21.15.153:0/2696647580 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f1d48072680 msgr2=0x7f1d48074b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:09:14.290 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:14.117+0000 7f1d69600700 1 --2- 172.21.15.153:0/2696647580 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f1d48072680 0x7f1d48074b40 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1d50009fd0 tx=0x7f1d50005940 comp rx=0 tx=0).stop 2024-03-21T10:09:14.290 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:14.117+0000 7f1d69600700 1 -- 172.21.15.153:0/2696647580 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1d64100bb0 msgr2=0x7f1d6419edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:09:14.291 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:14.117+0000 7f1d69600700 1 --2- 172.21.15.153:0/2696647580 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1d64100bb0 0x7f1d6419edb0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f1d5400ecb0 tx=0x7f1d54009ea0 comp rx=0 tx=0).stop 2024-03-21T10:09:14.291 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:14.118+0000 7f1d69600700 1 -- 172.21.15.153:0/2696647580 shutdown_connections 2024-03-21T10:09:14.291 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:14.118+0000 7f1d69600700 1 --2- 172.21.15.153:0/2696647580 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f1d48072680 0x7f1d48074b40 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:09:14.291 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:14.118+0000 7f1d69600700 1 --2- 172.21.15.153:0/2696647580 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1d64100bb0 0x7f1d6419edb0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:09:14.291 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:14.118+0000 7f1d69600700 1 --2- 172.21.15.153:0/2696647580 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1d64101db0 0x7f1d6419f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:09:14.291 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:14.118+0000 7f1d69600700 1 -- 172.21.15.153:0/2696647580 >> 172.21.15.153:0/2696647580 conn(0x7f1d640fc110 msgr2=0x7f1d64104fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:09:14.291 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:14.118+0000 7f1d69600700 1 -- 172.21.15.153:0/2696647580 shutdown_connections 2024-03-21T10:09:14.291 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:14.118+0000 7f1d69600700 1 -- 172.21.15.153:0/2696647580 wait complete. 2024-03-21T10:09:14.300 INFO:tasks.workunit.client.1.smithi174.stdout:6/892: write d0/d3/d5/d3b/f94 [1175581,36594] 0 2024-03-21T10:09:14.301 INFO:tasks.workunit.client.1.smithi174.stdout:2/980: read - d5/d25/d27/d2d/d45/dae/dd6/d4f/df1/fa6 zero size 2024-03-21T10:09:14.301 INFO:tasks.workunit.client.1.smithi174.stdout:3/987: dread d0/d12/d14/da9/de7/db7/d7d/fef [0,4194304] 0 2024-03-21T10:09:14.301 INFO:tasks.workunit.client.1.smithi174.stdout:6/893: write d0/d3/d5/d6/d42/d10a/f108 [314805,61251] 0 2024-03-21T10:09:14.301 INFO:tasks.workunit.client.1.smithi174.stdout:7/950: fsync d0/d4/d10/d44/d81/d9a/fed 0 2024-03-21T10:09:14.301 INFO:tasks.workunit.client.1.smithi174.stdout:3/988: symlink d0/d12/d14/d4c/l135 0 2024-03-21T10:09:14.301 INFO:tasks.workunit.client.1.smithi174.stdout:3/989: write d0/d12/d14/d1a/f34 [4738664,53910] 0 2024-03-21T10:09:14.301 INFO:tasks.workunit.client.1.smithi174.stdout:6/894: rename d0/d10c/cad to d0/d3/d5/df/c118 0 2024-03-21T10:09:14.301 INFO:tasks.workunit.client.1.smithi174.stdout:8/984: link d9/d2a/f51 d9/d53/dc9/de6/dea/d117/f145 0 2024-03-21T10:09:14.301 INFO:tasks.workunit.client.1.smithi174.stdout:7/951: mkdir d0/d4/d10/d1b/d86/df0/d11c/d151 0 2024-03-21T10:09:14.301 INFO:tasks.workunit.client.1.smithi174.stdout:6/895: rmdir d0/d3/d5/df/d16/da9/db7/dc2 39 2024-03-21T10:09:14.301 INFO:tasks.workunit.client.1.smithi174.stdout:8/985: chown d9/d2a/d39/d76/d19/d27/d60/la7 1708351 1 2024-03-21T10:09:14.301 INFO:tasks.workunit.client.1.smithi174.stdout:7/952: mkdir d0/d4/d38/d152 0 2024-03-21T10:09:14.301 INFO:tasks.workunit.client.1.smithi174.stdout:6/896: chown d0/d3/d5/df/d16/da9/db7/c115 123559187 1 2024-03-21T10:09:14.302 INFO:tasks.workunit.client.1.smithi174.stdout:3/990: creat d0/d12/d14/d1a/d5c/f136 x:0 0 0 2024-03-21T10:09:14.302 INFO:tasks.workunit.client.1.smithi174.stdout:8/986: read d9/d2a/d39/d76/d19/d27/f41 [2211836,3421] 0 2024-03-21T10:09:14.302 INFO:tasks.workunit.client.1.smithi174.stdout:6/897: creat d0/d10c/f119 x:0 0 0 2024-03-21T10:09:14.302 INFO:tasks.workunit.client.1.smithi174.stdout:3/991: symlink d0/d12/d14/d1a/d27/d7f/l137 0 2024-03-21T10:09:14.302 INFO:tasks.workunit.client.1.smithi174.stdout:6/898: stat d0/d3/d5/df/d66/d87 0 2024-03-21T10:09:14.302 INFO:tasks.workunit.client.1.smithi174.stdout:8/987: symlink d9/dc6/df1/d136/l146 0 2024-03-21T10:09:14.302 INFO:tasks.workunit.client.1.smithi174.stdout:8/988: unlink d9/d53/d7e/l99 0 2024-03-21T10:09:14.302 INFO:tasks.workunit.client.1.smithi174.stdout:6/899: truncate d0/d3/d5/d6/d42/d10a/daf/fea 703790 0 2024-03-21T10:09:14.302 INFO:tasks.workunit.client.1.smithi174.stdout:7/953: dwrite d0/d4/f42 [0,4194304] 0 2024-03-21T10:09:14.302 INFO:tasks.workunit.client.1.smithi174.stdout:3/992: dread d0/d12/d14/f80 [0,4194304] 0 2024-03-21T10:09:14.302 INFO:tasks.workunit.client.1.smithi174.stdout:8/989: mkdir d9/d2a/d39/d58/dbb/dc3/d147 0 2024-03-21T10:09:14.302 INFO:tasks.workunit.client.1.smithi174.stdout:8/990: write d9/d2a/d39/de1/f141 [705990,111284] 0 2024-03-21T10:09:14.302 INFO:tasks.workunit.client.1.smithi174.stdout:8/991: symlink d9/d53/dc9/de6/dea/l148 0 2024-03-21T10:09:14.302 INFO:tasks.workunit.client.1.smithi174.stdout:7/954: link d0/d4/d10/d79/db2/dda/l78 d0/d11b/dfe/l153 0 2024-03-21T10:09:14.303 INFO:tasks.workunit.client.1.smithi174.stdout:7/955: write d0/d4/d10/d44/d81/fde [1381881,65818] 0 2024-03-21T10:09:14.303 INFO:tasks.workunit.client.1.smithi174.stdout:3/993: dread d0/d12/d14/f97 [0,4194304] 0 2024-03-21T10:09:14.303 INFO:tasks.workunit.client.1.smithi174.stdout:8/992: dread d9/dc6/fe4 [0,4194304] 0 2024-03-21T10:09:14.303 INFO:tasks.workunit.client.1.smithi174.stdout:2/981: write d5/d25/d27/d2d/f41 [118523,12681] 0 2024-03-21T10:09:14.303 INFO:tasks.workunit.client.1.smithi174.stdout:7/956: creat d0/d4/d38/da9/dc6/f154 x:0 0 0 2024-03-21T10:09:14.303 INFO:tasks.workunit.client.1.smithi174.stdout:2/982: write d5/d25/d27/d2d/d45/dae/dd6/d85/dad/f122 [278983,92423] 0 2024-03-21T10:09:14.303 INFO:tasks.workunit.client.1.smithi174.stdout:8/993: write d9/d2a/d4d/f50 [282176,21429] 0 2024-03-21T10:09:14.303 INFO:tasks.workunit.client.1.smithi174.stdout:7/957: write d0/d4/d10/d79/f10e [861091,21977] 0 2024-03-21T10:09:14.303 INFO:tasks.workunit.client.1.smithi174.stdout:7/958: dread - d0/d4/d10/d44/d81/d9a/fed zero size 2024-03-21T10:09:14.303 INFO:tasks.workunit.client.1.smithi174.stdout:2/983: fdatasync d5/d25/d27/d2d/d45/d5b/d2f/d68/de8/fe9 0 2024-03-21T10:09:14.303 INFO:tasks.workunit.client.1.smithi174.stdout:3/994: link d0/l9 d0/d12/d14/d4c/l138 0 2024-03-21T10:09:14.303 INFO:tasks.workunit.client.1.smithi174.stdout:7/959: write d0/d4/d10/d44/f11f [166295,20128] 0 2024-03-21T10:09:14.303 INFO:tasks.workunit.client.1.smithi174.stdout:5/983: sync 2024-03-21T10:09:14.303 INFO:tasks.workunit.client.1.smithi174.stdout:6/900: write d0/d3/d5/d3b/f55 [108261,28476] 0 2024-03-21T10:09:14.304 INFO:tasks.workunit.client.1.smithi174.stdout:5/984: rmdir d1/d2f 39 2024-03-21T10:09:14.304 INFO:tasks.workunit.client.1.smithi174.stdout:6/901: creat d0/df5/f11a x:0 0 0 2024-03-21T10:09:14.304 INFO:tasks.workunit.client.1.smithi174.stdout:5/985: mknod d1/d5c/d73/c15a 0 2024-03-21T10:09:14.304 INFO:tasks.workunit.client.1.smithi174.stdout:3/995: fsync d0/d12/d14/d1a/d44/d65/d7e/f12b 0 2024-03-21T10:09:14.304 INFO:tasks.workunit.client.1.smithi174.stdout:7/960: write d0/f11 [985406,6097] 0 2024-03-21T10:09:14.304 INFO:tasks.workunit.client.1.smithi174.stdout:2/984: getdents d5/d25/d27/d2d/d45/d5b/d55 0 2024-03-21T10:09:14.304 INFO:tasks.workunit.client.1.smithi174.stdout:3/996: chown d0/d12/d14/d1a/d44/d65/l10c 6238829 1 2024-03-21T10:09:14.304 INFO:tasks.workunit.client.1.smithi174.stdout:7/961: symlink d0/d4/d10/d1b/d86/df0/d11c/d151/l155 0 2024-03-21T10:09:14.304 INFO:tasks.workunit.client.1.smithi174.stdout:6/902: truncate d0/d3/d5/d6/d42/d10a/d56/d5e/f98 2660995 0 2024-03-21T10:09:14.304 INFO:tasks.workunit.client.1.smithi174.stdout:3/997: mknod d0/d12/d14/d4c/d7a/c139 0 2024-03-21T10:09:14.304 INFO:tasks.workunit.client.1.smithi174.stdout:8/994: dread d9/f1a [4194304,4194304] 0 2024-03-21T10:09:14.304 INFO:tasks.workunit.client.1.smithi174.stdout:6/903: mknod d0/c11b 0 2024-03-21T10:09:14.304 INFO:tasks.workunit.client.1.smithi174.stdout:7/962: chown d0/d4/d10/d44/d81/d9a/lb4 1243 1 2024-03-21T10:09:14.304 INFO:tasks.workunit.client.1.smithi174.stdout:5/986: dwrite d1/d49/d4f/d51/d8f/d103/f10f [0,4194304] 0 2024-03-21T10:09:14.304 INFO:tasks.workunit.client.1.smithi174.stdout:6/904: chown d0/d3/d20/d3d/c75 13 1 2024-03-21T10:09:14.305 INFO:tasks.workunit.client.1.smithi174.stdout:8/995: creat d9/d53/dc9/de6/dea/d117/f149 x:0 0 0 2024-03-21T10:09:14.305 INFO:tasks.workunit.client.1.smithi174.stdout:5/987: chown d1/d5c/f135 1814 1 2024-03-21T10:09:14.305 INFO:tasks.workunit.client.1.smithi174.stdout:5/988: chown d1/d16/d2e/d8d 0 1 2024-03-21T10:09:14.305 INFO:tasks.workunit.client.1.smithi174.stdout:3/998: link d0/d12/d14/d1a/d5c/l89 d0/d12/d14/d1a/d44/d56/l13a 0 2024-03-21T10:09:14.305 INFO:tasks.workunit.client.1.smithi174.stdout:2/985: dwrite d5/d25/d27/d2d/d45/dae/dd6/d85/dad/fb0 [0,4194304] 0 2024-03-21T10:09:14.305 INFO:tasks.workunit.client.1.smithi174.stdout:7/963: rename d0/d4/l20 to d0/d4/d10/d79/db2/dda/l156 0 2024-03-21T10:09:14.305 INFO:tasks.workunit.client.1.smithi174.stdout:8/996: truncate d9/f108 764574 0 2024-03-21T10:09:14.305 INFO:tasks.workunit.client.1.smithi174.stdout:3/999: mkdir d0/d12/d14/d67/da1/da7/d13b 0 2024-03-21T10:09:14.305 INFO:tasks.workunit.client.1.smithi174.stdout:7/964: rename d0/d9e/lf3 to d0/d4/d10/d79/db2/l157 0 2024-03-21T10:09:14.305 INFO:tasks.workunit.client.1.smithi174.stdout:8/997: write d9/d2a/d39/d76/d19/d27/d60/daa/fb9 [4511210,2002] 0 2024-03-21T10:09:14.305 INFO:tasks.workunit.client.1.smithi174.stdout:2/986: getdents d5/d25/d27/d2d/d45/d5b/d2f/d4b/d76 0 2024-03-21T10:09:14.305 INFO:tasks.workunit.client.1.smithi174.stdout:8/998: write d9/d2a/d39/d76/d19/d27/d60/daa/dab/dc4/f13d [217900,100233] 0 2024-03-21T10:09:14.305 INFO:tasks.workunit.client.1.smithi174.stdout:7/965: mkdir d0/d4/d10/d44/d81/d9a/d158 0 2024-03-21T10:09:14.305 INFO:tasks.workunit.client.1.smithi174.stdout:6/905: dwrite d0/d3/d5/d6/d42/d10a/f2f [4194304,4194304] 0 2024-03-21T10:09:14.305 INFO:tasks.workunit.client.1.smithi174.stdout:8/999: fdatasync d9/d2a/d39/d76/f6a 0 2024-03-21T10:09:14.305 INFO:tasks.workunit.client.1.smithi174.stdout:2/987: write d5/d25/d27/d2d/d45/dae/dd6/d4f/d65/d77/dcf/f129 [628694,105751] 0 2024-03-21T10:09:14.306 INFO:tasks.workunit.client.1.smithi174.stdout:6/906: rmdir d0/d3/d5/d6/dfe 39 2024-03-21T10:09:14.306 INFO:tasks.workunit.client.1.smithi174.stdout:6/907: dread d0/d3/d5/df/d16/f8a [0,4194304] 0 2024-03-21T10:09:14.306 INFO:tasks.workunit.client.1.smithi174.stdout:7/966: truncate d0/d4/d10/d1b/d86/dd2/f114 3087251 0 2024-03-21T10:09:14.306 INFO:tasks.workunit.client.1.smithi174.stdout:6/908: read d0/d3/d5/d6/d42/d10a/d56/d5e/f98 [72642,37041] 0 2024-03-21T10:09:14.306 INFO:tasks.workunit.client.1.smithi174.stdout:6/909: read d0/d3/d5/d21/f9c [3274677,68507] 0 2024-03-21T10:09:14.306 INFO:tasks.workunit.client.1.smithi174.stdout:7/967: write d0/d4/d10/d79/db2/dda/d5f/d65/fac [2754063,66842] 0 2024-03-21T10:09:14.306 INFO:tasks.workunit.client.1.smithi174.stdout:7/968: getdents d0/d4/d10/d44/d81/d9a/dc0 0 2024-03-21T10:09:14.306 INFO:tasks.workunit.client.1.smithi174.stdout:2/988: dread d5/d25/d27/d2d/d45/dae/dd6/d4f/d65/d77/dcf/fd5 [0,4194304] 0 2024-03-21T10:09:14.306 INFO:tasks.workunit.client.1.smithi174.stdout:2/989: read d5/d25/d27/d2d/d45/d5b/d2f/d4b/d76/f105 [445817,86208] 0 2024-03-21T10:09:14.306 INFO:tasks.workunit.client.1.smithi174.stdout:6/910: dwrite d0/d3/d5/df/d16/da9/db7/dc2/dda/fe3 [0,4194304] 0 2024-03-21T10:09:14.306 INFO:tasks.workunit.client.1.smithi174.stdout:6/911: creat d0/d3/d5/d6/d42/d10a/d56/d5e/d91/de9/f11c x:0 0 0 2024-03-21T10:09:14.306 INFO:tasks.workunit.client.1.smithi174.stdout:2/990: link d5/d25/d27/d2d/d45/lca d5/d25/d27/d2d/d45/d5b/d2f/l154 0 2024-03-21T10:09:14.306 INFO:tasks.workunit.client.1.smithi174.stdout:2/991: getdents d5/d25/d27 0 2024-03-21T10:09:14.306 INFO:tasks.workunit.client.1.smithi174.stdout:6/912: dread d0/d3/d5/f57 [0,4194304] 0 2024-03-21T10:09:14.307 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:14.304+0000 7f545f200700 1 -- 172.21.15.153:0/1020519491 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5460101470 msgr2=0x7f5460103860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:09:14.307 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:14.304+0000 7f545f200700 1 --2- 172.21.15.153:0/1020519491 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5460101470 0x7f5460103860 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f5450009910 tx=0x7f5450009c20 comp rx=0 tx=0).stop 2024-03-21T10:09:14.308 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:14.304+0000 7f545f200700 1 -- 172.21.15.153:0/1020519491 shutdown_connections 2024-03-21T10:09:14.308 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:14.304+0000 7f545f200700 1 --2- 172.21.15.153:0/1020519491 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5460103da0 0x7f5460106190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:09:14.308 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:14.304+0000 7f545f200700 1 --2- 172.21.15.153:0/1020519491 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5460101470 0x7f5460103860 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:09:14.308 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:14.304+0000 7f545f200700 1 -- 172.21.15.153:0/1020519491 >> 172.21.15.153:0/1020519491 conn(0x7f54600fae40 msgr2=0x7f54600fd2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:09:14.308 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:14.304+0000 7f545f200700 1 -- 172.21.15.153:0/1020519491 shutdown_connections 2024-03-21T10:09:14.308 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:14.304+0000 7f545f200700 1 -- 172.21.15.153:0/1020519491 wait complete. 2024-03-21T10:09:14.308 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:14.305+0000 7f545f200700 1 Processor -- start 2024-03-21T10:09:14.308 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:14.305+0000 7f545f200700 1 -- start start 2024-03-21T10:09:14.309 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:14.306+0000 7f545f200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5460101470 0x7f54601a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:09:14.309 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:14.306+0000 7f545f200700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5460103da0 0x7f54601a10a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:09:14.309 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:14.306+0000 7f545f200700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54601a16c0 con 0x7f5460101470 2024-03-21T10:09:14.309 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:14.306+0000 7f545f200700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54601a1800 con 0x7f5460103da0 2024-03-21T10:09:14.309 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:14.306+0000 7f545de00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5460101470 0x7f54601a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:09:14.309 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:14.306+0000 7f545de00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5460101470 0x7f54601a0b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42136/0 (socket says 172.21.15.153:42136) 2024-03-21T10:09:14.309 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:14.306+0000 7f545de00700 1 -- 172.21.15.153:0/482951478 learned_addr learned my addr 172.21.15.153:0/482951478 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:09:14.309 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:14.306+0000 7f545d400700 1 --2- 172.21.15.153:0/482951478 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5460103da0 0x7f54601a10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:09:14.310 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:14.307+0000 7f545de00700 1 -- 172.21.15.153:0/482951478 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5460103da0 msgr2=0x7f54601a10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:09:14.310 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:14.307+0000 7f545de00700 1 --2- 172.21.15.153:0/482951478 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5460103da0 0x7f54601a10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:09:14.310 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:14.307+0000 7f545de00700 1 -- 172.21.15.153:0/482951478 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54500095f0 con 0x7f5460101470 2024-03-21T10:09:14.310 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:14.308+0000 7f545de00700 1 --2- 172.21.15.153:0/482951478 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5460101470 0x7f54601a0b60 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f545000b5c0 tx=0x7f545000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:09:14.314 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:14.309+0000 7f5456a00700 1 -- 172.21.15.153:0/482951478 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f545001d070 con 0x7f5460101470 2024-03-21T10:09:14.315 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:14.309+0000 7f5456a00700 1 -- 172.21.15.153:0/482951478 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f545000f460 con 0x7f5460101470 2024-03-21T10:09:14.315 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:14.309+0000 7f5456a00700 1 -- 172.21.15.153:0/482951478 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5450021640 con 0x7f5460101470 2024-03-21T10:09:14.315 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:14.309+0000 7f545f200700 1 -- 172.21.15.153:0/482951478 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54601a6250 con 0x7f5460101470 2024-03-21T10:09:14.316 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:14.310+0000 7f545f200700 1 -- 172.21.15.153:0/482951478 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54601a6740 con 0x7f5460101470 2024-03-21T10:09:14.316 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:14.311+0000 7f545f200700 1 -- 172.21.15.153:0/482951478 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54600fca50 con 0x7f5460101470 2024-03-21T10:09:14.316 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:14.314+0000 7f5456a00700 1 -- 172.21.15.153:0/482951478 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 29) v1 ==== 92174+0+0 (secure 0 0 0) 0x7f5450003a60 con 0x7f5460101470 2024-03-21T10:09:14.318 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:14.316+0000 7f5456a00700 1 --2- 172.21.15.153:0/482951478 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f544406e260 0x7f5444070720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:09:14.318 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:14.316+0000 7f5456a00700 1 -- 172.21.15.153:0/482951478 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f545008f1a0 con 0x7f5460101470 2024-03-21T10:09:14.318 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:14.316+0000 7f545d400700 1 --2- 172.21.15.153:0/482951478 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f544406e260 0x7f5444070720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:09:14.319 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:14.318+0000 7f545d400700 1 --2- 172.21.15.153:0/482951478 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f544406e260 0x7f5444070720 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f544c005950 tx=0x7f544c0058e0 comp rx=0 tx=0).ready entity=mgr.14666 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:09:14.323 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:14.322+0000 7f5456a00700 1 -- 172.21.15.153:0/482951478 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185454 (secure 0 0 0) 0x7f545005c0f0 con 0x7f5460101470 2024-03-21T10:09:14.343 INFO:tasks.workunit.client.1.smithi174.stdout:5/989: sync 2024-03-21T10:09:14.372 INFO:tasks.workunit.client.1.smithi174.stdout:5/990: dwrite d1/d5c/d73/fa5 [0,4194304] 0 2024-03-21T10:09:14.385 INFO:tasks.workunit.client.1.smithi174.stdout:5/991: write d1/d2f/d37/df5/d11e/f138 [2419390,77006] 0 2024-03-21T10:09:14.405 INFO:tasks.workunit.client.1.smithi174.stdout:7/969: sync 2024-03-21T10:09:14.408 INFO:tasks.workunit.client.1.smithi174.stdout:7/970: chown d0/d4/d10/d1b/d86/dea 4643533 1 2024-03-21T10:09:14.417 INFO:tasks.workunit.client.1.smithi174.stdout:7/971: creat d0/d4/d10/d1b/d86/dd2/f159 x:0 0 0 2024-03-21T10:09:14.418 INFO:tasks.workunit.client.1.smithi174.stdout:5/992: dwrite d1/d16/f40 [0,4194304] 0 2024-03-21T10:09:14.424 INFO:tasks.workunit.client.1.smithi174.stdout:7/972: truncate d0/d4/d10/f94 996090 0 2024-03-21T10:09:14.425 INFO:tasks.workunit.client.1.smithi174.stdout:5/993: write d1/d49/f102 [926577,11434] 0 2024-03-21T10:09:14.434 INFO:tasks.workunit.client.1.smithi174.stdout:5/994: truncate d1/d49/d4f/d51/f56 2039831 0 2024-03-21T10:09:14.437 INFO:tasks.workunit.client.1.smithi174.stdout:5/995: write d1/d5c/f135 [795974,2454] 0 2024-03-21T10:09:14.445 INFO:tasks.workunit.client.1.smithi174.stdout:5/996: getdents d1/d2f/d37 0 2024-03-21T10:09:14.447 INFO:tasks.workunit.client.1.smithi174.stdout:5/997: creat d1/d130/f15b x:0 0 0 2024-03-21T10:09:14.451 INFO:tasks.workunit.client.1.smithi174.stdout:7/973: dread d0/d4/d10/d79/f10e [0,4194304] 0 2024-03-21T10:09:14.454 INFO:tasks.workunit.client.1.smithi174.stdout:5/998: symlink d1/d16/d2e/d66/d8b/d85/d142/l15c 0 2024-03-21T10:09:14.457 INFO:tasks.workunit.client.1.smithi174.stdout:5/999: fdatasync d1/d49/d93/f119 0 2024-03-21T10:09:14.461 INFO:tasks.workunit.client.1.smithi174.stdout:6/913: sync 2024-03-21T10:09:14.463 INFO:tasks.workunit.client.1.smithi174.stdout:2/992: sync 2024-03-21T10:09:14.465 INFO:tasks.workunit.client.1.smithi174.stdout:6/914: read - d0/d3/d5/d6/d42/f8b zero size 2024-03-21T10:09:14.474 INFO:tasks.workunit.client.1.smithi174.stdout:6/915: mknod d0/d3/d5/d6/d42/d10a/d56/d5e/d91/df0/c11d 0 2024-03-21T10:09:14.476 INFO:tasks.workunit.client.1.smithi174.stdout:2/993: getdents d5/d25/d27/d2d/d45/d5b/d2f/d4b/da7 0 2024-03-21T10:09:14.477 INFO:tasks.workunit.client.1.smithi174.stdout:6/916: rmdir d0/d3/d5/df 39 2024-03-21T10:09:14.479 INFO:tasks.workunit.client.1.smithi174.stdout:7/974: dread d0/d4/d10/f94 [0,4194304] 0 2024-03-21T10:09:14.480 INFO:tasks.workunit.client.1.smithi174.stdout:2/994: creat d5/d25/d27/d2d/d45/d5b/d55/d75/f155 x:0 0 0 2024-03-21T10:09:14.482 INFO:tasks.workunit.client.1.smithi174.stdout:7/975: symlink d0/d9e/l15a 0 2024-03-21T10:09:14.484 INFO:tasks.workunit.client.1.smithi174.stdout:2/995: fsync d5/d25/d27/d2d/d45/d5b/f60 0 2024-03-21T10:09:14.489 INFO:tasks.workunit.client.1.smithi174.stdout:2/996: read d5/d25/d27/f4e [7469379,22739] 0 2024-03-21T10:09:14.493 INFO:tasks.workunit.client.1.smithi174.stdout:7/976: truncate d0/d4/d10/d79/f10e 646676 0 2024-03-21T10:09:14.495 INFO:tasks.workunit.client.1.smithi174.stdout:2/997: dread d5/d25/d27/d2d/d45/d5b/d2f/d4b/d76/f105 [0,4194304] 0 2024-03-21T10:09:14.496 INFO:tasks.workunit.client.1.smithi174.stdout:6/917: dwrite d0/d3/d5/d6/d42/d10a/d8f/fb5 [0,4194304] 0 2024-03-21T10:09:14.503 INFO:tasks.workunit.client.1.smithi174.stdout:7/977: creat d0/d4/d10/d1b/d86/dea/df4/df8/f15b x:0 0 0 2024-03-21T10:09:14.504 INFO:tasks.workunit.client.1.smithi174.stdout:7/978: dread - d0/d4/f133 zero size 2024-03-21T10:09:14.505 INFO:tasks.workunit.client.1.smithi174.stdout:2/998: creat d5/f156 x:0 0 0 2024-03-21T10:09:14.506 INFO:tasks.workunit.client.1.smithi174.stdout:7/979: chown d0/lf7 179 1 2024-03-21T10:09:14.508 INFO:tasks.workunit.client.1.smithi174.stdout:7/980: fdatasync d0/d4/d38/f11a 0 2024-03-21T10:09:14.511 INFO:tasks.workunit.client.1.smithi174.stdout:2/999: dread d5/d25/d27/d2d/d45/d5b/d2f/d4b/d76/f105 [0,4194304] 0 2024-03-21T10:09:14.512 INFO:tasks.workunit.client.1.smithi174.stdout:6/918: dwrite d0/d3/d5/f45 [0,4194304] 0 2024-03-21T10:09:14.512 INFO:tasks.workunit.client.1.smithi174.stdout:7/981: creat d0/d4/d10/d1b/d86/dea/df4/df8/f15c x:0 0 0 2024-03-21T10:09:14.518 INFO:tasks.workunit.client.1.smithi174.stdout:7/982: chown d0/d4/d10/d1b/d86/dd2/f114 133 1 2024-03-21T10:09:14.519 INFO:tasks.workunit.client.1.smithi174.stdout:7/983: chown d0/d4/fa8 110 1 2024-03-21T10:09:14.525 INFO:tasks.workunit.client.1.smithi174.stdout:6/919: write d0/d3/d5/d6/d42/d10a/fb4 [768479,77398] 0 2024-03-21T10:09:14.530 INFO:tasks.workunit.client.1.smithi174.stdout:6/920: unlink d0/d3/d5/d6/d42/l5d 0 2024-03-21T10:09:14.535 INFO:tasks.workunit.client.1.smithi174.stdout:6/921: creat d0/d3/d5/d6/d42/d10a/d56/d5e/f11e x:0 0 0 2024-03-21T10:09:14.536 INFO:tasks.workunit.client.1.smithi174.stdout:7/984: dwrite d0/d4/d10/d44/d81/f8f [8388608,4194304] 0 2024-03-21T10:09:14.542 INFO:tasks.workunit.client.1.smithi174.stdout:7/985: rename d0/d4/d38/fd9 to d0/d4/d10/d79/db2/dda/d5f/daa/f15d 0 2024-03-21T10:09:14.546 INFO:tasks.workunit.client.1.smithi174.stdout:6/922: rmdir d0/d3/d5/df/d66/d87/ded/d10d 0 2024-03-21T10:09:14.550 INFO:tasks.workunit.client.1.smithi174.stdout:6/923: dread d0/d3/d20/d6d/ff1 [0,4194304] 0 2024-03-21T10:09:14.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:14 smithi153 ceph-mon[29117]: from='client.14714 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:09:14.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:14 smithi153 ceph-mon[29117]: pgmap v23: 65 pgs: 65 active+clean; 4.0 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 96 MiB/s rd, 201 MiB/s wr, 472 op/s 2024-03-21T10:09:14.565 INFO:tasks.workunit.client.1.smithi174.stdout:7/986: dwrite d0/d4/f7e [0,4194304] 0 2024-03-21T10:09:14.566 INFO:tasks.workunit.client.1.smithi174.stdout:6/924: dwrite d0/d3/d20/fba [4194304,4194304] 0 2024-03-21T10:09:14.569 INFO:tasks.workunit.client.1.smithi174.stdout:7/987: creat d0/d4/d10/d79/db2/dda/d5f/daa/f15e x:0 0 0 2024-03-21T10:09:14.569 INFO:tasks.workunit.client.1.smithi174.stdout:6/925: rmdir d0/d3/d5/df 39 2024-03-21T10:09:14.582 INFO:tasks.workunit.client.1.smithi174.stdout:7/988: dwrite d0/d4/d10/d79/db2/dda/f63 [0,4194304] 0 2024-03-21T10:09:14.590 INFO:tasks.workunit.client.1.smithi174.stdout:6/926: write d0/d3/d5/d6/d42/d10a/fbf [476523,54962] 0 2024-03-21T10:09:14.601 INFO:tasks.workunit.client.1.smithi174.stdout:7/989: dwrite d0/d4/d10/d1b/d86/f10a [0,4194304] 0 2024-03-21T10:09:14.608 INFO:tasks.workunit.client.1.smithi174.stdout:7/990: symlink d0/d4/d10/d79/db2/dda/d5f/d130/l15f 0 2024-03-21T10:09:14.608 INFO:tasks.workunit.client.1.smithi174.stdout:7/991: readlink d0/d4/d38/da9/l13e 0 2024-03-21T10:09:14.620 INFO:tasks.workunit.client.1.smithi174.stdout:7/992: dwrite d0/d4/f33 [0,4194304] 0 2024-03-21T10:09:14.625 INFO:tasks.workunit.client.1.smithi174.stdout:7/993: mknod d0/d135/c160 0 2024-03-21T10:09:14.628 INFO:tasks.workunit.client.1.smithi174.stdout:7/994: fsync d0/d4/d10/d79/db2/dda/d5f/d65/fb5 0 2024-03-21T10:09:14.631 INFO:tasks.workunit.client.1.smithi174.stdout:7/995: symlink d0/d4/d10/d44/d81/d9a/d158/l161 0 2024-03-21T10:09:14.634 INFO:tasks.workunit.client.1.smithi174.stdout:7/996: dread - d0/d4/d10/d1b/d86/dea/df4/d123/fc1 zero size 2024-03-21T10:09:14.635 INFO:tasks.workunit.client.1.smithi174.stdout:7/997: stat d0/d4/d10/d79/db2/dda/f63 0 2024-03-21T10:09:14.648 INFO:tasks.workunit.client.1.smithi174.stdout:7/998: dwrite d0/d4/d10/d79/db2/dda/d5f/d130/f14a [0,4194304] 0 2024-03-21T10:09:14.652 INFO:tasks.workunit.client.1.smithi174.stdout:7/999: mkdir d0/d4/d121/d162 0 2024-03-21T10:09:14.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:14 smithi174 ceph-mon[37496]: from='client.14714 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:09:14.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:14 smithi174 ceph-mon[37496]: pgmap v23: 65 pgs: 65 active+clean; 4.0 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 96 MiB/s rd, 201 MiB/s wr, 472 op/s 2024-03-21T10:09:14.726 INFO:tasks.workunit.client.1.smithi174.stdout:6/927: sync 2024-03-21T10:09:14.732 INFO:tasks.workunit.client.1.smithi174.stdout:6/928: mknod d0/d3/d5/d21/c11f 0 2024-03-21T10:09:14.737 INFO:tasks.workunit.client.1.smithi174.stdout:6/929: truncate d0/d3/d5/fbe 551724 0 2024-03-21T10:09:14.810 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:14.809+0000 7f545f200700 1 -- 172.21.15.153:0/482951478 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f546002ced0 con 0x7f5460101470 2024-03-21T10:09:14.811 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:14.809+0000 7f5456a00700 1 -- 172.21.15.153:0/482951478 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7f545005bc80 con 0x7f5460101470 2024-03-21T10:09:14.811 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:09:14.811 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": { 2024-03-21T10:09:14.811 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-03-21T10:09:14.811 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:09:14.811 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": { 2024-03-21T10:09:14.811 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:09:14.811 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:09:14.811 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": { 2024-03-21T10:09:14.812 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-03-21T10:09:14.812 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:09:14.812 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": { 2024-03-21T10:09:14.812 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-03-21T10:09:14.812 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:09:14.812 INFO:teuthology.orchestra.run.smithi153.stdout: "overall": { 2024-03-21T10:09:14.812 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 12, 2024-03-21T10:09:14.812 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:09:14.812 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-03-21T10:09:14.812 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:09:14.814 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:14.813+0000 7f545f200700 1 -- 172.21.15.153:0/482951478 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f544406e260 msgr2=0x7f5444070720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:09:14.814 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:14.813+0000 7f545f200700 1 --2- 172.21.15.153:0/482951478 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f544406e260 0x7f5444070720 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f544c005950 tx=0x7f544c0058e0 comp rx=0 tx=0).stop 2024-03-21T10:09:14.814 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:14.813+0000 7f545f200700 1 -- 172.21.15.153:0/482951478 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5460101470 msgr2=0x7f54601a0b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:09:14.814 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:14.813+0000 7f545f200700 1 --2- 172.21.15.153:0/482951478 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5460101470 0x7f54601a0b60 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f545000b5c0 tx=0x7f545000be20 comp rx=0 tx=0).stop 2024-03-21T10:09:14.815 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:14.813+0000 7f545f200700 1 -- 172.21.15.153:0/482951478 shutdown_connections 2024-03-21T10:09:14.815 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:14.813+0000 7f545f200700 1 --2- 172.21.15.153:0/482951478 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f544406e260 0x7f5444070720 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:09:14.815 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:14.813+0000 7f545f200700 1 --2- 172.21.15.153:0/482951478 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5460101470 0x7f54601a0b60 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:09:14.815 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:14.813+0000 7f545f200700 1 --2- 172.21.15.153:0/482951478 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5460103da0 0x7f54601a10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:09:14.815 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:14.813+0000 7f545f200700 1 -- 172.21.15.153:0/482951478 >> 172.21.15.153:0/482951478 conn(0x7f54600fae40 msgr2=0x7f54600ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:09:14.815 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:14.813+0000 7f545f200700 1 -- 172.21.15.153:0/482951478 shutdown_connections 2024-03-21T10:09:14.815 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:14.814+0000 7f545f200700 1 -- 172.21.15.153:0/482951478 wait complete. 2024-03-21T10:09:14.824 INFO:tasks.workunit.client.1.smithi174.stdout:6/930: sync 2024-03-21T10:09:14.839 INFO:tasks.workunit.client.1.smithi174.stdout:6/931: write d0/d3/d5/df/fac [295135,108790] 0 2024-03-21T10:09:14.947 INFO:tasks.workunit.client.1.smithi174.stdout:6/932: sync 2024-03-21T10:09:14.951 INFO:tasks.workunit.client.1.smithi174.stdout:6/933: creat d0/d3/d5/df/d66/f120 x:0 0 0 2024-03-21T10:09:14.953 INFO:tasks.workunit.client.1.smithi174.stdout:6/934: fdatasync d0/d3/d5/f45 0 2024-03-21T10:09:14.999 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:14.997+0000 7fbaa6800700 1 -- 172.21.15.153:0/2953654579 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbaa0101db0 msgr2=0x7fbaa0102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:09:14.999 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:14.997+0000 7fbaa6800700 1 --2- 172.21.15.153:0/2953654579 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbaa0101db0 0x7fbaa0102210 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fba8c009910 tx=0x7fba8c009c20 comp rx=0 tx=0).stop 2024-03-21T10:09:14.999 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:14.997+0000 7fbaa6800700 1 -- 172.21.15.153:0/2953654579 shutdown_connections 2024-03-21T10:09:14.999 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:14.997+0000 7fbaa6800700 1 --2- 172.21.15.153:0/2953654579 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbaa0101db0 0x7fbaa0102210 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:09:14.999 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:14.997+0000 7fbaa6800700 1 --2- 172.21.15.153:0/2953654579 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbaa0100bb0 0x7fbaa0100fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:09:14.999 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:14.997+0000 7fbaa6800700 1 -- 172.21.15.153:0/2953654579 >> 172.21.15.153:0/2953654579 conn(0x7fbaa00fc110 msgr2=0x7fbaa00fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:09:14.999 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:14.997+0000 7fbaa6800700 1 -- 172.21.15.153:0/2953654579 shutdown_connections 2024-03-21T10:09:14.999 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:14.997+0000 7fbaa6800700 1 -- 172.21.15.153:0/2953654579 wait complete. 2024-03-21T10:09:15.001 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:14.998+0000 7fbaa6800700 1 Processor -- start 2024-03-21T10:09:15.001 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:14.998+0000 7fbaa6800700 1 -- start start 2024-03-21T10:09:15.001 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:14.999+0000 7fbaa6800700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbaa0100bb0 0x7fbaa0198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:09:15.001 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:14.999+0000 7fbaa6800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbaa0101db0 0x7fbaa0198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:09:15.001 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:14.999+0000 7fbaa6800700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaa01990b0 con 0x7fbaa0101db0 2024-03-21T10:09:15.001 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:14.999+0000 7fbaa6800700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaa01991f0 con 0x7fbaa0100bb0 2024-03-21T10:09:15.001 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:14.999+0000 7fba9ea00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbaa0101db0 0x7fbaa0198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:09:15.001 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:14.999+0000 7fba9ea00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbaa0101db0 0x7fbaa0198a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42158/0 (socket says 172.21.15.153:42158) 2024-03-21T10:09:15.001 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:14.999+0000 7fba9ea00700 1 -- 172.21.15.153:0/562806086 learned_addr learned my addr 172.21.15.153:0/562806086 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:09:15.001 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:14.999+0000 7fba9f400700 1 --2- 172.21.15.153:0/562806086 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbaa0100bb0 0x7fbaa0198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:09:15.001 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:14.999+0000 7fba9ea00700 1 -- 172.21.15.153:0/562806086 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbaa0100bb0 msgr2=0x7fbaa0198550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:09:15.001 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:14.999+0000 7fba9ea00700 1 --2- 172.21.15.153:0/562806086 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbaa0100bb0 0x7fbaa0198550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:09:15.002 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:14.999+0000 7fba9ea00700 1 -- 172.21.15.153:0/562806086 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba8c0095f0 con 0x7fbaa0101db0 2024-03-21T10:09:15.002 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:15.000+0000 7fba9ea00700 1 --2- 172.21.15.153:0/562806086 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbaa0101db0 0x7fbaa0198a90 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fba8c0092e0 tx=0x7fba8c00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:09:15.002 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:15.000+0000 7fba97e00700 1 -- 172.21.15.153:0/562806086 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fba8c01d070 con 0x7fbaa0101db0 2024-03-21T10:09:15.002 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:15.000+0000 7fba97e00700 1 -- 172.21.15.153:0/562806086 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fba8c005dc0 con 0x7fbaa0101db0 2024-03-21T10:09:15.003 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:15.000+0000 7fba97e00700 1 -- 172.21.15.153:0/562806086 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fba8c017630 con 0x7fbaa0101db0 2024-03-21T10:09:15.003 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:15.000+0000 7fbaa6800700 1 -- 172.21.15.153:0/562806086 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbaa019dc40 con 0x7fbaa0101db0 2024-03-21T10:09:15.003 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:15.001+0000 7fbaa6800700 1 -- 172.21.15.153:0/562806086 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbaa019e130 con 0x7fbaa0101db0 2024-03-21T10:09:15.004 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:15.003+0000 7fba97e00700 1 -- 172.21.15.153:0/562806086 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 29) v1 ==== 92174+0+0 (secure 0 0 0) 0x7fba8c003680 con 0x7fbaa0101db0 2024-03-21T10:09:15.004 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:15.003+0000 7fbaa6800700 1 -- 172.21.15.153:0/562806086 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbaa0105e90 con 0x7fbaa0101db0 2024-03-21T10:09:15.005 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:15.003+0000 7fba97e00700 1 --2- 172.21.15.153:0/562806086 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7fba8406e210 0x7fba840706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:09:15.005 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:15.004+0000 7fba97e00700 1 -- 172.21.15.153:0/562806086 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fba8c08f510 con 0x7fbaa0101db0 2024-03-21T10:09:15.006 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:15.004+0000 7fba9f400700 1 --2- 172.21.15.153:0/562806086 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7fba8406e210 0x7fba840706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:09:15.006 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:15.005+0000 7fba9f400700 1 --2- 172.21.15.153:0/562806086 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7fba8406e210 0x7fba840706d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fbaa0101c10 tx=0x7fba90009260 comp rx=0 tx=0).ready entity=mgr.14666 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:09:15.011 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:15.010+0000 7fba97e00700 1 -- 172.21.15.153:0/562806086 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185454 (secure 0 0 0) 0x7fba8c05c4e0 con 0x7fbaa0101db0 2024-03-21T10:09:15.048 INFO:tasks.workunit.client.1.smithi174.stdout:6/935: sync 2024-03-21T10:09:15.056 INFO:tasks.workunit.client.1.smithi174.stdout:6/936: symlink d0/d3/d5/d3b/l121 0 2024-03-21T10:09:15.059 INFO:tasks.workunit.client.1.smithi174.stdout:6/937: stat d0/d3/d5/d6/d42/d10a/d56/d5e/c6f 0 2024-03-21T10:09:15.063 INFO:tasks.workunit.client.1.smithi174.stdout:6/938: getdents d0/d3/d20/d3d 0 2024-03-21T10:09:15.067 INFO:tasks.workunit.client.1.smithi174.stdout:6/939: mknod d0/d3/d20/d3d/d81/da5/c122 0 2024-03-21T10:09:15.069 INFO:tasks.workunit.client.1.smithi174.stdout:6/940: creat d0/d10c/f123 x:0 0 0 2024-03-21T10:09:15.073 INFO:tasks.workunit.client.1.smithi174.stdout:6/941: fsync d0/d3/d5/d6/d42/d10a/d56/d5e/f98 0 2024-03-21T10:09:15.082 INFO:tasks.workunit.client.1.smithi174.stdout:6/942: link d0/d3/d5/df/d16/l83 d0/d3/d5/d6/d42/d10a/d56/d5e/d91/de9/l124 0 2024-03-21T10:09:15.098 INFO:tasks.workunit.client.1.smithi174.stdout:6/943: dwrite d0/d3/d5/d3b/f55 [0,4194304] 0 2024-03-21T10:09:15.101 INFO:tasks.workunit.client.1.smithi174.stdout:6/944: truncate d0/d3/d20/d3d/f116 517139 0 2024-03-21T10:09:15.241 INFO:tasks.workunit.client.1.smithi174.stdout:6/945: sync 2024-03-21T10:09:15.257 INFO:tasks.workunit.client.1.smithi174.stdout:6/946: dwrite d0/d3/d5/d6/d42/d10a/fa2 [0,4194304] 0 2024-03-21T10:09:15.260 INFO:tasks.workunit.client.1.smithi174.stdout:6/947: unlink d0/df5/f11a 0 2024-03-21T10:09:15.276 INFO:tasks.workunit.client.1.smithi174.stdout:6/948: dwrite d0/d3/d5/df/d16/f17 [0,4194304] 0 2024-03-21T10:09:15.285 INFO:tasks.workunit.client.1.smithi174.stdout:6/949: creat d0/d3/d5/d6/dfe/f125 x:0 0 0 2024-03-21T10:09:15.304 INFO:tasks.workunit.client.1.smithi174.stdout:6/950: dread d0/d3/d5/f45 [0,4194304] 0 2024-03-21T10:09:15.374 INFO:tasks.workunit.client.1.smithi174.stdout:6/951: sync 2024-03-21T10:09:15.375 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:15.373+0000 7fbaa6800700 1 -- 172.21.15.153:0/562806086 --> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbaa00611d0 con 0x7fba8406e210 2024-03-21T10:09:15.382 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:15.378+0000 7fba97e00700 1 -- 172.21.15.153:0/562806086 <== mgr.14666 v2:172.21.15.153:6800/2524145044 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+329 (secure 0 0 0) 0x7fbaa00611d0 con 0x7fba8406e210 2024-03-21T10:09:15.382 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:09:15.382 INFO:teuthology.orchestra.run.smithi153.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232", 2024-03-21T10:09:15.382 INFO:teuthology.orchestra.run.smithi153.stdout: "in_progress": true, 2024-03-21T10:09:15.382 INFO:teuthology.orchestra.run.smithi153.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-03-21T10:09:15.382 INFO:teuthology.orchestra.run.smithi153.stdout: "services_complete": [ 2024-03-21T10:09:15.382 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr" 2024-03-21T10:09:15.382 INFO:teuthology.orchestra.run.smithi153.stdout: ], 2024-03-21T10:09:15.382 INFO:teuthology.orchestra.run.smithi153.stdout: "progress": "2/2 daemons upgraded", 2024-03-21T10:09:15.382 INFO:teuthology.orchestra.run.smithi153.stdout: "message": "", 2024-03-21T10:09:15.382 INFO:teuthology.orchestra.run.smithi153.stdout: "is_paused": false 2024-03-21T10:09:15.383 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:09:15.385 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:15.383+0000 7fbaa6800700 1 -- 172.21.15.153:0/562806086 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7fba8406e210 msgr2=0x7fba840706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:09:15.385 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:15.383+0000 7fbaa6800700 1 --2- 172.21.15.153:0/562806086 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7fba8406e210 0x7fba840706d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fbaa0101c10 tx=0x7fba90009260 comp rx=0 tx=0).stop 2024-03-21T10:09:15.385 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:15.383+0000 7fbaa6800700 1 -- 172.21.15.153:0/562806086 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbaa0101db0 msgr2=0x7fbaa0198a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:09:15.385 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:15.383+0000 7fbaa6800700 1 --2- 172.21.15.153:0/562806086 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbaa0101db0 0x7fbaa0198a90 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fba8c0092e0 tx=0x7fba8c00f690 comp rx=0 tx=0).stop 2024-03-21T10:09:15.385 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:15.384+0000 7fbaa6800700 1 -- 172.21.15.153:0/562806086 shutdown_connections 2024-03-21T10:09:15.385 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:15.384+0000 7fbaa6800700 1 --2- 172.21.15.153:0/562806086 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7fba8406e210 0x7fba840706d0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:09:15.385 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:15.384+0000 7fbaa6800700 1 --2- 172.21.15.153:0/562806086 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbaa0100bb0 0x7fbaa0198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:09:15.386 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:15.384+0000 7fbaa6800700 1 --2- 172.21.15.153:0/562806086 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbaa0101db0 0x7fbaa0198a90 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:09:15.386 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:15.384+0000 7fbaa6800700 1 -- 172.21.15.153:0/562806086 >> 172.21.15.153:0/562806086 conn(0x7fbaa00fc110 msgr2=0x7fbaa0102fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:09:15.386 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:15.384+0000 7fbaa6800700 1 -- 172.21.15.153:0/562806086 shutdown_connections 2024-03-21T10:09:15.386 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:15.384+0000 7fbaa6800700 1 -- 172.21.15.153:0/562806086 wait complete. 2024-03-21T10:09:15.391 INFO:tasks.workunit.client.1.smithi174.stdout:6/952: dwrite d0/d3/d5/d6/d42/d10a/fb4 [0,4194304] 0 2024-03-21T10:09:15.393 INFO:tasks.workunit.client.1.smithi174.stdout:6/953: write d0/d3/d20/fba [8609863,81644] 0 2024-03-21T10:09:15.413 INFO:tasks.workunit.client.1.smithi174.stdout:6/954: dwrite d0/d3/d5/d6/d42/d10a/d56/d5e/d91/fe7 [4194304,4194304] 0 2024-03-21T10:09:15.416 INFO:tasks.workunit.client.1.smithi174.stdout:6/955: write d0/d3/d5/d6/d42/d10a/fb4 [2880518,58177] 0 2024-03-21T10:09:15.423 INFO:tasks.workunit.client.1.smithi174.stdout:6/956: unlink d0/c2 0 2024-03-21T10:09:15.424 INFO:tasks.workunit.client.1.smithi174.stdout:6/957: chown d0/d3/d5/d3b 247828 1 2024-03-21T10:09:15.425 INFO:tasks.workunit.client.1.smithi174.stdout:6/958: stat d0/d3/d5/f57 0 2024-03-21T10:09:15.431 INFO:tasks.workunit.client.1.smithi174.stdout:6/959: truncate d0/d3/d20/d3d/d81/da5/fc8 1349207 0 2024-03-21T10:09:15.432 INFO:tasks.workunit.client.1.smithi174.stdout:6/960: chown d0/d3/d5/d6 3 1 2024-03-21T10:09:15.433 INFO:tasks.workunit.client.1.smithi174.stdout:6/961: chown d0/d3/d5/d6/d42/f82 1167861 1 2024-03-21T10:09:15.438 INFO:tasks.workunit.client.1.smithi174.stdout:6/962: link d0/d3/d20/d3d/fd6 d0/d3/d20/d6d/f126 0 2024-03-21T10:09:15.440 INFO:tasks.workunit.client.1.smithi174.stdout:6/963: dread - d0/d3/d20/d3d/f10f zero size 2024-03-21T10:09:15.443 INFO:tasks.workunit.client.1.smithi174.stdout:6/964: truncate d0/d3/d20/d3d/d81/da5/fc9 1168308 0 2024-03-21T10:09:15.446 INFO:tasks.workunit.client.1.smithi174.stdout:6/965: creat d0/d3/d5/df/d16/da9/f127 x:0 0 0 2024-03-21T10:09:15.507 INFO:tasks.workunit.client.1.smithi174.stdout:6/966: truncate d0/d3/d5/d6/f6a 2566607 0 2024-03-21T10:09:15.518 INFO:tasks.workunit.client.1.smithi174.stdout:6/967: truncate d0/d3/d20/d3d/fd6 89143 0 2024-03-21T10:09:15.519 INFO:tasks.workunit.client.1.smithi174.stdout:6/968: chown d0/d3/d20/d6d 7535155 1 2024-03-21T10:09:15.527 INFO:tasks.workunit.client.1.smithi174.stdout:6/969: symlink d0/d3/d5/d6/d42/d10a/d56/d5e/d91/df0/l128 0 2024-03-21T10:09:15.550 INFO:tasks.workunit.client.1.smithi174.stdout:6/970: symlink d0/d3/d20/d3d/d81/l129 0 2024-03-21T10:09:15.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:15 smithi153 ceph-mon[29117]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:09:15.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:15 smithi153 ceph-mon[29117]: from='client.14722 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:09:15.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:15 smithi153 ceph-mon[29117]: from='client.? 172.21.15.153:0/482951478' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:09:15.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:15 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:09:15.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:15 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:09:15.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:15 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:09:15.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:15 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:09:15.565 INFO:tasks.workunit.client.1.smithi174.stdout:6/971: dwrite d0/d3/d5/df/d16/f27 [0,4194304] 0 2024-03-21T10:09:15.573 INFO:tasks.workunit.client.1.smithi174.stdout:6/972: mknod d0/d3/d20/c12a 0 2024-03-21T10:09:15.577 INFO:tasks.workunit.client.1.smithi174.stdout:6/973: mkdir d0/d3/d5/df/d66/d87/d12b 0 2024-03-21T10:09:15.583 INFO:tasks.workunit.client.1.smithi174.stdout:6/974: creat d0/d3/d5/df/d16/da9/db7/dc2/dda/f12c x:0 0 0 2024-03-21T10:09:15.589 INFO:tasks.workunit.client.1.smithi174.stdout:6/975: mknod d0/d3/d20/d3d/d81/da5/c12d 0 2024-03-21T10:09:15.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:15 smithi174 ceph-mon[37496]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:09:15.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:15 smithi174 ceph-mon[37496]: from='client.14722 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:09:15.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:15 smithi174 ceph-mon[37496]: from='client.? 172.21.15.153:0/482951478' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:09:15.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:15 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:09:15.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:15 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:09:15.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:15 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:09:15.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:15 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:09:15.761 INFO:tasks.workunit.client.1.smithi174.stdout:6/976: sync 2024-03-21T10:09:15.766 INFO:tasks.workunit.client.1.smithi174.stdout:6/977: mknod d0/d3/d5/d6/d42/c12e 0 2024-03-21T10:09:15.793 INFO:tasks.workunit.client.1.smithi174.stdout:6/978: dwrite d0/d3/d5/d6/d42/f82 [0,4194304] 0 2024-03-21T10:09:15.803 INFO:tasks.workunit.client.1.smithi174.stdout:6/979: chown d0/d3/d5/d6/d42/d10a/d56/d5e/d91/fe5 4 1 2024-03-21T10:09:15.813 INFO:tasks.workunit.client.1.smithi174.stdout:6/980: getdents d0/d3/d5/d6/d42/d10a/d8f 0 2024-03-21T10:09:15.836 INFO:tasks.workunit.client.1.smithi174.stdout:6/981: dwrite d0/d3/d5/d6/f44 [0,4194304] 0 2024-03-21T10:09:15.844 INFO:tasks.workunit.client.1.smithi174.stdout:6/982: write d0/d3/d20/d6d/f126 [568286,52464] 0 2024-03-21T10:09:15.861 INFO:tasks.workunit.client.1.smithi174.stdout:6/983: dwrite d0/d3/d5/d6/f2d [4194304,4194304] 0 2024-03-21T10:09:15.867 INFO:tasks.workunit.client.1.smithi174.stdout:6/984: read d0/d3/d5/fe [2651778,79646] 0 2024-03-21T10:09:15.869 INFO:tasks.workunit.client.1.smithi174.stdout:6/985: write d0/d3/d5/d6/dfe/f125 [862873,13063] 0 2024-03-21T10:09:15.871 INFO:tasks.workunit.client.1.smithi174.stdout:6/986: chown d0/d3/d5/d21/c11f 726858 1 2024-03-21T10:09:15.886 INFO:tasks.workunit.client.1.smithi174.stdout:6/987: dwrite d0/d3/d5/df/d16/f17 [4194304,4194304] 0 2024-03-21T10:09:15.897 INFO:tasks.workunit.client.1.smithi174.stdout:6/988: dread d0/d3/d5/d6/d42/f82 [0,4194304] 0 2024-03-21T10:09:15.904 INFO:tasks.workunit.client.1.smithi174.stdout:6/989: rename d0/d3/d20/d6d/f10b to d0/d3/d5/df/d16/da9/db7/dc2/dda/f12f 0 2024-03-21T10:09:15.920 INFO:tasks.workunit.client.1.smithi174.stdout:6/990: dwrite d0/d3/d5/fe [0,4194304] 0 2024-03-21T10:09:15.937 INFO:tasks.workunit.client.1.smithi174.stdout:6/991: dwrite d0/d3/d5/d6/dfe/f125 [0,4194304] 0 2024-03-21T10:09:15.942 INFO:tasks.workunit.client.1.smithi174.stdout:6/992: creat d0/d10c/f130 x:0 0 0 2024-03-21T10:09:15.946 INFO:tasks.workunit.client.1.smithi174.stdout:6/993: write d0/d3/d5/d6/d42/d10a/d56/d5e/fa7 [3221651,41540] 0 2024-03-21T10:09:15.952 INFO:tasks.workunit.client.1.smithi174.stdout:6/994: truncate d0/d3/d5/d6/d42/f8b 98211 0 2024-03-21T10:09:15.956 INFO:tasks.workunit.client.1.smithi174.stdout:6/995: symlink d0/d3/d20/d3d/d81/l131 0 2024-03-21T10:09:15.961 INFO:tasks.workunit.client.1.smithi174.stdout:6/996: getdents d0/d3/d5/df/d16 0 2024-03-21T10:09:15.970 INFO:tasks.workunit.client.1.smithi174.stdout:6/997: getdents d0/d3/d20/d3d/d81 0 2024-03-21T10:09:15.973 INFO:tasks.workunit.client.1.smithi174.stdout:6/998: mknod d0/d3/d5/df/d66/d87/ded/c132 0 2024-03-21T10:09:15.979 INFO:tasks.workunit.client.1.smithi174.stdout:6/999: write d0/d3/d20/d6d/fb2 [447278,17846] 0 2024-03-21T10:09:15.988 INFO:tasks.workunit.client.1.smithi174.stderr:+ rm -rf -- ./tmp.pIui9Xc5jq 2024-03-21T10:09:16.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:16 smithi174 ceph-mon[37496]: Updating smithi153:/etc/ceph/ceph.conf 2024-03-21T10:09:16.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:16 smithi174 ceph-mon[37496]: Updating smithi174:/etc/ceph/ceph.conf 2024-03-21T10:09:16.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:16 smithi174 ceph-mon[37496]: pgmap v24: 65 pgs: 65 active+clean; 4.3 GiB data, 13 GiB used, 524 GiB / 536 GiB avail; 126 MiB/s rd, 270 MiB/s wr, 650 op/s 2024-03-21T10:09:16.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:16 smithi174 ceph-mon[37496]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:09:16.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:16 smithi153 ceph-mon[29117]: Updating smithi153:/etc/ceph/ceph.conf 2024-03-21T10:09:16.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:16 smithi153 ceph-mon[29117]: Updating smithi174:/etc/ceph/ceph.conf 2024-03-21T10:09:16.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:16 smithi153 ceph-mon[29117]: pgmap v24: 65 pgs: 65 active+clean; 4.3 GiB data, 13 GiB used, 524 GiB / 536 GiB avail; 126 MiB/s rd, 270 MiB/s wr, 650 op/s 2024-03-21T10:09:16.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:16 smithi153 ceph-mon[29117]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:09:17.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:17 smithi174 ceph-mon[37496]: Updating smithi153:/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T10:09:17.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:17 smithi174 ceph-mon[37496]: Updating smithi174:/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T10:09:17.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:17 smithi174 ceph-mon[37496]: Updating smithi153:/etc/ceph/ceph.client.admin.keyring 2024-03-21T10:09:17.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:17 smithi174 ceph-mon[37496]: Updating smithi174:/etc/ceph/ceph.client.admin.keyring 2024-03-21T10:09:17.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:17 smithi153 ceph-mon[29117]: Updating smithi153:/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T10:09:17.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:17 smithi153 ceph-mon[29117]: Updating smithi174:/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T10:09:17.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:17 smithi153 ceph-mon[29117]: Updating smithi153:/etc/ceph/ceph.client.admin.keyring 2024-03-21T10:09:17.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:17 smithi153 ceph-mon[29117]: Updating smithi174:/etc/ceph/ceph.client.admin.keyring 2024-03-21T10:09:18.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:18 smithi174 ceph-mon[37496]: Updating smithi153:/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-21T10:09:18.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:18 smithi174 ceph-mon[37496]: Updating smithi174:/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-21T10:09:18.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:18 smithi174 ceph-mon[37496]: pgmap v25: 65 pgs: 65 active+clean; 4.3 GiB data, 13 GiB used, 524 GiB / 536 GiB avail; 78 MiB/s rd, 168 MiB/s wr, 416 op/s 2024-03-21T10:09:18.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:18 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:09:18.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:18 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:09:18.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:18 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:09:18.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:18 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:09:18.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:18 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:09:18.703 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:18 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:09:18.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:18 smithi153 ceph-mon[29117]: Updating smithi153:/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-21T10:09:18.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:18 smithi153 ceph-mon[29117]: Updating smithi174:/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-21T10:09:18.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:18 smithi153 ceph-mon[29117]: pgmap v25: 65 pgs: 65 active+clean; 4.3 GiB data, 13 GiB used, 524 GiB / 536 GiB avail; 78 MiB/s rd, 168 MiB/s wr, 416 op/s 2024-03-21T10:09:18.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:18 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:09:18.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:18 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:09:18.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:18 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:09:18.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:18 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:09:18.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:18 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:09:18.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:18 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:09:19.547 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:19 smithi153 ceph-mon[29117]: Reconfiguring prometheus.smithi153 (dependencies changed)... 2024-03-21T10:09:19.547 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:19 smithi153 ceph-mon[29117]: Reconfiguring daemon prometheus.smithi153 on smithi153 2024-03-21T10:09:19.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:19 smithi174 ceph-mon[37496]: Reconfiguring prometheus.smithi153 (dependencies changed)... 2024-03-21T10:09:19.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:19 smithi174 ceph-mon[37496]: Reconfiguring daemon prometheus.smithi153 on smithi153 2024-03-21T10:09:20.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:20 smithi174 ceph-mon[37496]: pgmap v26: 65 pgs: 65 active+clean; 4.3 GiB data, 13 GiB used, 524 GiB / 536 GiB avail; 78 MiB/s rd, 168 MiB/s wr, 416 op/s 2024-03-21T10:09:20.780 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:20 smithi153 ceph-mon[29117]: pgmap v26: 65 pgs: 65 active+clean; 4.3 GiB data, 13 GiB used, 524 GiB / 536 GiB avail; 78 MiB/s rd, 168 MiB/s wr, 416 op/s 2024-03-21T10:09:22.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:22 smithi153 ceph-mon[29117]: pgmap v27: 65 pgs: 65 active+clean; 3.6 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 79 MiB/s rd, 177 MiB/s wr, 505 op/s 2024-03-21T10:09:22.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:22 smithi174 ceph-mon[37496]: pgmap v27: 65 pgs: 65 active+clean; 3.6 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 79 MiB/s rd, 177 MiB/s wr, 505 op/s 2024-03-21T10:09:24.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:24 smithi153 ceph-mon[29117]: pgmap v28: 65 pgs: 65 active+clean; 3.6 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 32 MiB/s rd, 78 MiB/s wr, 267 op/s 2024-03-21T10:09:24.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:24 smithi174 ceph-mon[37496]: pgmap v28: 65 pgs: 65 active+clean; 3.6 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 32 MiB/s rd, 78 MiB/s wr, 267 op/s 2024-03-21T10:09:26.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:26 smithi153 ceph-mon[29117]: pgmap v29: 65 pgs: 65 active+clean; 2.7 GiB data, 8.2 GiB used, 528 GiB / 536 GiB avail; 33 MiB/s rd, 79 MiB/s wr, 353 op/s 2024-03-21T10:09:26.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:26 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:09:26.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:26 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:09:26.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:26 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-21T10:09:26.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:26 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:09:26.813 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:26 smithi174 ceph-mon[37496]: pgmap v29: 65 pgs: 65 active+clean; 2.7 GiB data, 8.2 GiB used, 528 GiB / 536 GiB avail; 33 MiB/s rd, 79 MiB/s wr, 353 op/s 2024-03-21T10:09:26.813 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:26 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:09:26.813 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:26 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:09:26.813 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:26 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-21T10:09:26.813 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:26 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:09:27.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:27 smithi153 ceph-mon[29117]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-21T10:09:27.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:27 smithi174 ceph-mon[37496]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-21T10:09:28.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:28 smithi153 ceph-mon[29117]: pgmap v30: 65 pgs: 65 active+clean; 2.7 GiB data, 8.2 GiB used, 528 GiB / 536 GiB avail; 2.0 MiB/s rd, 9.3 MiB/s wr, 174 op/s 2024-03-21T10:09:28.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:28 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi174.qpcwat", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-21T10:09:28.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:28 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-21T10:09:28.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:28 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:09:28.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:28 smithi174 ceph-mon[37496]: pgmap v30: 65 pgs: 65 active+clean; 2.7 GiB data, 8.2 GiB used, 528 GiB / 536 GiB avail; 2.0 MiB/s rd, 9.3 MiB/s wr, 174 op/s 2024-03-21T10:09:28.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:28 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi174.qpcwat", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-21T10:09:28.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:28 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-21T10:09:28.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:28 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:09:29.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:29 smithi174 ceph-mon[37496]: Upgrade: Updating mgr.smithi174.qpcwat 2024-03-21T10:09:29.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:29 smithi174 ceph-mon[37496]: Deploying daemon mgr.smithi174.qpcwat on smithi174 2024-03-21T10:09:29.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:29 smithi174 ceph-mon[37496]: pgmap v31: 65 pgs: 65 active+clean; 2.7 GiB data, 8.2 GiB used, 528 GiB / 536 GiB avail; 2.0 MiB/s rd, 9.3 MiB/s wr, 174 op/s 2024-03-21T10:09:29.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:29 smithi153 ceph-mon[29117]: Upgrade: Updating mgr.smithi174.qpcwat 2024-03-21T10:09:29.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:29 smithi153 ceph-mon[29117]: Deploying daemon mgr.smithi174.qpcwat on smithi174 2024-03-21T10:09:29.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:29 smithi153 ceph-mon[29117]: pgmap v31: 65 pgs: 65 active+clean; 2.7 GiB data, 8.2 GiB used, 528 GiB / 536 GiB avail; 2.0 MiB/s rd, 9.3 MiB/s wr, 174 op/s 2024-03-21T10:09:29.810 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-21T10:09:29.810 DEBUG:teuthology.orchestra.run.smithi153:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-21T10:09:32.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:32 smithi174 ceph-mon[37496]: pgmap v32: 65 pgs: 65 active+clean; 1.5 GiB data, 4.9 GiB used, 532 GiB / 536 GiB avail; 2.6 MiB/s rd, 9.8 MiB/s wr, 253 op/s 2024-03-21T10:09:32.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:32 smithi153 ceph-mon[29117]: pgmap v32: 65 pgs: 65 active+clean; 1.5 GiB data, 4.9 GiB used, 532 GiB / 536 GiB avail; 2.6 MiB/s rd, 9.8 MiB/s wr, 253 op/s 2024-03-21T10:09:33.619 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-21T10:09:33.619 DEBUG:teuthology.orchestra.run.smithi174:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1/tmp 2024-03-21T10:09:34.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:34 smithi174 ceph-mon[37496]: pgmap v33: 65 pgs: 65 active+clean; 1.5 GiB data, 4.9 GiB used, 532 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 165 op/s 2024-03-21T10:09:34.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:34 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:09:34.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:34 smithi153 ceph-mon[29117]: pgmap v33: 65 pgs: 65 active+clean; 1.5 GiB data, 4.9 GiB used, 532 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 165 op/s 2024-03-21T10:09:34.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:34 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:09:35.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:35 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:09:35.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:35 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:09:35.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:35 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:09:35.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:35 smithi174 ceph-mon[37496]: pgmap v34: 65 pgs: 65 active+clean; 489 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 243 op/s 2024-03-21T10:09:36.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:35 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:09:36.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:35 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:09:36.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:35 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:09:36.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:35 smithi153 ceph-mon[29117]: pgmap v34: 65 pgs: 65 active+clean; 489 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 243 op/s 2024-03-21T10:09:38.304 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:38 smithi174 ceph-mon[37496]: pgmap v35: 65 pgs: 65 active+clean; 489 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 157 op/s 2024-03-21T10:09:38.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:38 smithi153 ceph-mon[29117]: pgmap v35: 65 pgs: 65 active+clean; 489 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 157 op/s 2024-03-21T10:09:40.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:40 smithi153 ceph-mon[29117]: pgmap v36: 65 pgs: 65 active+clean; 489 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 157 op/s 2024-03-21T10:09:40.568 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:40 smithi174 ceph-mon[37496]: pgmap v36: 65 pgs: 65 active+clean; 489 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 157 op/s 2024-03-21T10:09:42.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:42 smithi174 ceph-mon[37496]: pgmap v37: 65 pgs: 65 active+clean; 293 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 213 op/s 2024-03-21T10:09:42.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:42 smithi153 ceph-mon[29117]: pgmap v37: 65 pgs: 65 active+clean; 293 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 213 op/s 2024-03-21T10:09:44.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:44 smithi174 ceph-mon[37496]: pgmap v38: 65 pgs: 65 active+clean; 293 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 134 op/s 2024-03-21T10:09:44.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:44 smithi153 ceph-mon[29117]: pgmap v38: 65 pgs: 65 active+clean; 293 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 134 op/s 2024-03-21T10:09:45.605 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:45.604+0000 7ff928600700 1 -- 172.21.15.153:0/2186198102 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff9200ff430 msgr2=0x7ff920101860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:09:45.605 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:45.604+0000 7ff928600700 1 --2- 172.21.15.153:0/2186198102 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff9200ff430 0x7ff920101860 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7ff914009910 tx=0x7ff914009c20 comp rx=0 tx=0).stop 2024-03-21T10:09:45.608 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:45.605+0000 7ff928600700 1 -- 172.21.15.153:0/2186198102 shutdown_connections 2024-03-21T10:09:45.608 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:45.605+0000 7ff928600700 1 --2- 172.21.15.153:0/2186198102 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff920101da0 0x7ff920104230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:09:45.608 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:45.605+0000 7ff928600700 1 --2- 172.21.15.153:0/2186198102 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff9200ff430 0x7ff920101860 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:09:45.608 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:45.605+0000 7ff928600700 1 -- 172.21.15.153:0/2186198102 >> 172.21.15.153:0/2186198102 conn(0x7ff9200fb030 msgr2=0x7ff9200fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:09:45.609 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:45.605+0000 7ff928600700 1 -- 172.21.15.153:0/2186198102 shutdown_connections 2024-03-21T10:09:45.609 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:45.605+0000 7ff928600700 1 -- 172.21.15.153:0/2186198102 wait complete. 2024-03-21T10:09:45.610 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:45.607+0000 7ff928600700 1 Processor -- start 2024-03-21T10:09:45.611 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:45.607+0000 7ff928600700 1 -- start start 2024-03-21T10:09:45.611 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:45.607+0000 7ff928600700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff9200ff430 0x7ff92019c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:09:45.611 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:45.607+0000 7ff928600700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff920101da0 0x7ff92019ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:09:45.611 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:45.607+0000 7ff928600700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff92019d410 con 0x7ff9200ff430 2024-03-21T10:09:45.611 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:45.607+0000 7ff928600700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff92019d550 con 0x7ff920101da0 2024-03-21T10:09:45.611 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:45.608+0000 7ff925c00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff9200ff430 0x7ff92019c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:09:45.611 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:45.608+0000 7ff925200700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff920101da0 0x7ff92019ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:09:45.612 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:45.608+0000 7ff925200700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff920101da0 0x7ff92019ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.153:44736/0 (socket says 172.21.15.153:44736) 2024-03-21T10:09:45.612 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:45.608+0000 7ff925200700 1 -- 172.21.15.153:0/4197563584 learned_addr learned my addr 172.21.15.153:0/4197563584 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:09:45.612 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:45.609+0000 7ff925200700 1 -- 172.21.15.153:0/4197563584 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff9200ff430 msgr2=0x7ff92019c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:09:45.612 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:45.609+0000 7ff925200700 1 --2- 172.21.15.153:0/4197563584 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff9200ff430 0x7ff92019c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:09:45.612 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:45.609+0000 7ff925200700 1 -- 172.21.15.153:0/4197563584 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9140095f0 con 0x7ff920101da0 2024-03-21T10:09:45.612 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:45.610+0000 7ff925200700 1 --2- 172.21.15.153:0/4197563584 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff920101da0 0x7ff92019ce80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff90c00c930 tx=0x7ff90c00ccf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:09:45.618 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:45.611+0000 7ff91e000700 1 -- 172.21.15.153:0/4197563584 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff90c007ad0 con 0x7ff920101da0 2024-03-21T10:09:45.618 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:45.611+0000 7ff91e000700 1 -- 172.21.15.153:0/4197563584 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff90c007c30 con 0x7ff920101da0 2024-03-21T10:09:45.618 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:45.611+0000 7ff928600700 1 -- 172.21.15.153:0/4197563584 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9201a2010 con 0x7ff920101da0 2024-03-21T10:09:45.618 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:45.611+0000 7ff91e000700 1 -- 172.21.15.153:0/4197563584 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff90c018630 con 0x7ff920101da0 2024-03-21T10:09:45.618 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:45.611+0000 7ff928600700 1 -- 172.21.15.153:0/4197563584 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9201a2530 con 0x7ff920101da0 2024-03-21T10:09:45.619 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:45.612+0000 7ff928600700 1 -- 172.21.15.153:0/4197563584 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff900005320 con 0x7ff920101da0 2024-03-21T10:09:45.620 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:45.615+0000 7ff91e000700 1 -- 172.21.15.153:0/4197563584 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 29) v1 ==== 92174+0+0 (secure 0 0 0) 0x7ff90c018850 con 0x7ff920101da0 2024-03-21T10:09:45.621 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:45.616+0000 7ff91e000700 1 --2- 172.21.15.153:0/4197563584 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7ff90806e260 0x7ff908070720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:09:45.621 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:45.616+0000 7ff91e000700 1 -- 172.21.15.153:0/4197563584 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7ff90c08e1d0 con 0x7ff920101da0 2024-03-21T10:09:45.621 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:45.617+0000 7ff925c00700 1 --2- 172.21.15.153:0/4197563584 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7ff90806e260 0x7ff908070720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:09:45.621 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:45.618+0000 7ff925c00700 1 --2- 172.21.15.153:0/4197563584 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7ff90806e260 0x7ff908070720 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ff914000c00 tx=0x7ff914005fb0 comp rx=0 tx=0).ready entity=mgr.14666 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:09:45.628 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:45.627+0000 7ff91e000700 1 -- 172.21.15.153:0/4197563584 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185454 (secure 0 0 0) 0x7ff90c05b1a0 con 0x7ff920101da0 2024-03-21T10:09:45.985 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:45.984+0000 7ff928600700 1 -- 172.21.15.153:0/4197563584 --> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff900000bf0 con 0x7ff90806e260 2024-03-21T10:09:45.988 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:45.987+0000 7ff91e000700 1 -- 172.21.15.153:0/4197563584 <== mgr.14666 v2:172.21.15.153:6800/2524145044 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+360 (secure 0 0 0) 0x7ff900000bf0 con 0x7ff90806e260 2024-03-21T10:09:45.996 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:45.994+0000 7ff928600700 1 -- 172.21.15.153:0/4197563584 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7ff90806e260 msgr2=0x7ff908070720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:09:45.996 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:45.994+0000 7ff928600700 1 --2- 172.21.15.153:0/4197563584 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7ff90806e260 0x7ff908070720 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ff914000c00 tx=0x7ff914005fb0 comp rx=0 tx=0).stop 2024-03-21T10:09:45.996 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:45.994+0000 7ff928600700 1 -- 172.21.15.153:0/4197563584 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff920101da0 msgr2=0x7ff92019ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:09:45.996 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:45.994+0000 7ff928600700 1 --2- 172.21.15.153:0/4197563584 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff920101da0 0x7ff92019ce80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff90c00c930 tx=0x7ff90c00ccf0 comp rx=0 tx=0).stop 2024-03-21T10:09:45.997 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:45.995+0000 7ff928600700 1 -- 172.21.15.153:0/4197563584 shutdown_connections 2024-03-21T10:09:45.997 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:45.995+0000 7ff928600700 1 --2- 172.21.15.153:0/4197563584 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7ff90806e260 0x7ff908070720 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:09:45.997 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:45.995+0000 7ff928600700 1 --2- 172.21.15.153:0/4197563584 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff9200ff430 0x7ff92019c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:09:45.997 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:45.995+0000 7ff928600700 1 --2- 172.21.15.153:0/4197563584 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff920101da0 0x7ff92019ce80 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:09:45.997 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:45.995+0000 7ff928600700 1 -- 172.21.15.153:0/4197563584 >> 172.21.15.153:0/4197563584 conn(0x7ff9200fb030 msgr2=0x7ff9200fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:09:45.997 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:45.995+0000 7ff928600700 1 -- 172.21.15.153:0/4197563584 shutdown_connections 2024-03-21T10:09:45.998 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:45.995+0000 7ff928600700 1 -- 172.21.15.153:0/4197563584 wait complete. 2024-03-21T10:09:46.021 INFO:teuthology.orchestra.run.smithi153.stdout:true 2024-03-21T10:09:46.228 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:46.228+0000 7f6cf2600700 1 -- 172.21.15.153:0/2438250239 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6cec0ff430 msgr2=0x7f6cec101860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:09:46.229 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:46.228+0000 7f6cf2600700 1 --2- 172.21.15.153:0/2438250239 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6cec0ff430 0x7f6cec101860 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc009910 tx=0x7f6cdc009c20 comp rx=0 tx=0).stop 2024-03-21T10:09:46.229 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:46.228+0000 7f6cf2600700 1 -- 172.21.15.153:0/2438250239 shutdown_connections 2024-03-21T10:09:46.229 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:46.228+0000 7f6cf2600700 1 --2- 172.21.15.153:0/2438250239 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6cec101da0 0x7f6cec104230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:09:46.229 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:46.228+0000 7f6cf2600700 1 --2- 172.21.15.153:0/2438250239 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6cec0ff430 0x7f6cec101860 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:09:46.229 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:46.228+0000 7f6cf2600700 1 -- 172.21.15.153:0/2438250239 >> 172.21.15.153:0/2438250239 conn(0x7f6cec0fb030 msgr2=0x7f6cec0fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:09:46.230 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:46.229+0000 7f6cf2600700 1 -- 172.21.15.153:0/2438250239 shutdown_connections 2024-03-21T10:09:46.230 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:46.229+0000 7f6cf2600700 1 -- 172.21.15.153:0/2438250239 wait complete. 2024-03-21T10:09:46.231 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:46.231+0000 7f6cf2600700 1 Processor -- start 2024-03-21T10:09:46.232 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:46.231+0000 7f6cf2600700 1 -- start start 2024-03-21T10:09:46.232 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:46.231+0000 7f6cf2600700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6cec0ff430 0x7f6cec194110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:09:46.232 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:46.231+0000 7f6cf2600700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6cec101da0 0x7f6cec194650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:09:46.232 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:46.231+0000 7f6cf2600700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cec194c70 con 0x7f6cec101da0 2024-03-21T10:09:46.232 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:46.231+0000 7f6cf2600700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cec194db0 con 0x7f6cec0ff430 2024-03-21T10:09:46.233 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:46.232+0000 7f6ceb400700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6cec0ff430 0x7f6cec194110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:09:46.233 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:46.232+0000 7f6ceb400700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6cec0ff430 0x7f6cec194110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.153:44752/0 (socket says 172.21.15.153:44752) 2024-03-21T10:09:46.233 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:46.232+0000 7f6ceb400700 1 -- 172.21.15.153:0/2061762089 learned_addr learned my addr 172.21.15.153:0/2061762089 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:09:46.233 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:46.232+0000 7f6ceaa00700 1 --2- 172.21.15.153:0/2061762089 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6cec101da0 0x7f6cec194650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:09:46.234 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:46.232+0000 7f6ceb400700 1 -- 172.21.15.153:0/2061762089 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6cec101da0 msgr2=0x7f6cec194650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:09:46.234 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:46.232+0000 7f6ceb400700 1 --2- 172.21.15.153:0/2061762089 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6cec101da0 0x7f6cec194650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:09:46.234 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:46.233+0000 7f6ceb400700 1 -- 172.21.15.153:0/2061762089 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cd801a530 con 0x7f6cec0ff430 2024-03-21T10:09:46.234 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:46.233+0000 7f6ceb400700 1 --2- 172.21.15.153:0/2061762089 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6cec0ff430 0x7f6cec194110 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc0092e0 tx=0x7f6cdc00f740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:09:46.234 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:46.233+0000 7f6ce3e00700 1 -- 172.21.15.153:0/2061762089 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6cdc01d070 con 0x7f6cec0ff430 2024-03-21T10:09:46.234 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:46.234+0000 7f6ce3e00700 1 -- 172.21.15.153:0/2061762089 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f6cdc005dc0 con 0x7f6cec0ff430 2024-03-21T10:09:46.236 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:46.234+0000 7f6ce3e00700 1 -- 172.21.15.153:0/2061762089 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6cdc017610 con 0x7f6cec0ff430 2024-03-21T10:09:46.236 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:46.235+0000 7f6cf2600700 1 -- 172.21.15.153:0/2061762089 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cdc0095f0 con 0x7f6cec0ff430 2024-03-21T10:09:46.236 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:46.236+0000 7f6cf2600700 1 -- 172.21.15.153:0/2061762089 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cec199ae0 con 0x7f6cec0ff430 2024-03-21T10:09:46.237 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:46.237+0000 7f6cf2600700 1 -- 172.21.15.153:0/2061762089 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cec18e300 con 0x7f6cec0ff430 2024-03-21T10:09:46.243 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:46.242+0000 7f6ce3e00700 1 -- 172.21.15.153:0/2061762089 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 29) v1 ==== 92174+0+0 (secure 0 0 0) 0x7f6cdc00fba0 con 0x7f6cec0ff430 2024-03-21T10:09:46.243 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:46.243+0000 7f6ce3e00700 1 --2- 172.21.15.153:0/2061762089 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f6cd006e210 0x7f6cd00706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:09:46.244 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:46.243+0000 7f6ce3e00700 1 -- 172.21.15.153:0/2061762089 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f6cdc08f920 con 0x7f6cec0ff430 2024-03-21T10:09:46.244 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:46.243+0000 7f6ceaa00700 1 --2- 172.21.15.153:0/2061762089 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f6cd006e210 0x7f6cd00706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:09:46.245 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:46.244+0000 7f6ceaa00700 1 --2- 172.21.15.153:0/2061762089 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f6cd006e210 0x7f6cd00706d0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f6cd801acb0 tx=0x7f6cd801a270 comp rx=0 tx=0).ready entity=mgr.14666 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:09:46.250 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:46.250+0000 7f6ce3e00700 1 -- 172.21.15.153:0/2061762089 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185454 (secure 0 0 0) 0x7f6cdc05c930 con 0x7f6cec0ff430 2024-03-21T10:09:46.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:46 smithi153 ceph-mon[29117]: pgmap v39: 65 pgs: 65 active+clean; 301 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 184 op/s 2024-03-21T10:09:46.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:46 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:09:46.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:46 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:09:46.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:46 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:09:46.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:46 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:09:46.593 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:46 smithi174 ceph-mon[37496]: pgmap v39: 65 pgs: 65 active+clean; 301 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 184 op/s 2024-03-21T10:09:46.594 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:46 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:09:46.594 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:46 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:09:46.594 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:46 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:09:46.594 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:46 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:09:46.608 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:46.607+0000 7f6cf2600700 1 -- 172.21.15.153:0/2061762089 --> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6cec0611d0 con 0x7f6cd006e210 2024-03-21T10:09:46.614 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:46.614+0000 7f6ce3e00700 1 -- 172.21.15.153:0/2061762089 <== mgr.14666 v2:172.21.15.153:6800/2524145044 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+360 (secure 0 0 0) 0x7f6cec0611d0 con 0x7f6cd006e210 2024-03-21T10:09:46.617 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:46.616+0000 7f6cf2600700 1 -- 172.21.15.153:0/2061762089 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f6cd006e210 msgr2=0x7f6cd00706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:09:46.617 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:46.616+0000 7f6cf2600700 1 --2- 172.21.15.153:0/2061762089 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f6cd006e210 0x7f6cd00706d0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f6cd801acb0 tx=0x7f6cd801a270 comp rx=0 tx=0).stop 2024-03-21T10:09:46.617 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:46.616+0000 7f6cf2600700 1 -- 172.21.15.153:0/2061762089 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6cec0ff430 msgr2=0x7f6cec194110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:09:46.617 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:46.617+0000 7f6cf2600700 1 --2- 172.21.15.153:0/2061762089 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6cec0ff430 0x7f6cec194110 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc0092e0 tx=0x7f6cdc00f740 comp rx=0 tx=0).stop 2024-03-21T10:09:46.617 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:46.617+0000 7f6cf2600700 1 -- 172.21.15.153:0/2061762089 shutdown_connections 2024-03-21T10:09:46.617 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:46.617+0000 7f6cf2600700 1 --2- 172.21.15.153:0/2061762089 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f6cd006e210 0x7f6cd00706d0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:09:46.617 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:46.617+0000 7f6cf2600700 1 --2- 172.21.15.153:0/2061762089 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6cec0ff430 0x7f6cec194110 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:09:46.618 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:46.617+0000 7f6cf2600700 1 --2- 172.21.15.153:0/2061762089 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6cec101da0 0x7f6cec194650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:09:46.618 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:46.617+0000 7f6cf2600700 1 -- 172.21.15.153:0/2061762089 >> 172.21.15.153:0/2061762089 conn(0x7f6cec0fb030 msgr2=0x7f6cec0fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:09:46.618 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:46.617+0000 7f6cf2600700 1 -- 172.21.15.153:0/2061762089 shutdown_connections 2024-03-21T10:09:46.618 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:46.618+0000 7f6cf2600700 1 -- 172.21.15.153:0/2061762089 wait complete. 2024-03-21T10:09:46.810 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:46.809+0000 7f4bcb000700 1 -- 172.21.15.153:0/488365373 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4bc4103f70 msgr2=0x7f4bc4106360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:09:46.810 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:46.809+0000 7f4bcb000700 1 --2- 172.21.15.153:0/488365373 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4bc4103f70 0x7f4bc4106360 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4bb0009910 tx=0x7f4bb0009c20 comp rx=0 tx=0).stop 2024-03-21T10:09:46.810 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:46.809+0000 7f4bcb000700 1 -- 172.21.15.153:0/488365373 shutdown_connections 2024-03-21T10:09:46.810 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:46.809+0000 7f4bcb000700 1 --2- 172.21.15.153:0/488365373 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4bc4103f70 0x7f4bc4106360 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:09:46.810 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:46.809+0000 7f4bcb000700 1 --2- 172.21.15.153:0/488365373 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4bc4101640 0x7f4bc4103a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:09:46.810 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:46.809+0000 7f4bcb000700 1 -- 172.21.15.153:0/488365373 >> 172.21.15.153:0/488365373 conn(0x7f4bc40fb030 msgr2=0x7f4bc40fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:09:46.811 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:46.809+0000 7f4bcb000700 1 -- 172.21.15.153:0/488365373 shutdown_connections 2024-03-21T10:09:46.811 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:46.810+0000 7f4bcb000700 1 -- 172.21.15.153:0/488365373 wait complete. 2024-03-21T10:09:46.812 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:46.811+0000 7f4bcb000700 1 Processor -- start 2024-03-21T10:09:46.812 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:46.811+0000 7f4bcb000700 1 -- start start 2024-03-21T10:09:46.812 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:46.811+0000 7f4bcb000700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4bc4101640 0x7f4bc41984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:09:46.812 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:46.811+0000 7f4bcb000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4bc4103f70 0x7f4bc4198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:09:46.812 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:46.811+0000 7f4bcb000700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc4199040 con 0x7f4bc4103f70 2024-03-21T10:09:46.812 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:46.811+0000 7f4bcb000700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc4199180 con 0x7f4bc4101640 2024-03-21T10:09:46.812 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:46.811+0000 7f4bc3400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4bc4103f70 0x7f4bc4198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:09:46.813 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:46.812+0000 7f4bc3e00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4bc4101640 0x7f4bc41984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:09:46.813 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:46.812+0000 7f4bc3e00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4bc4101640 0x7f4bc41984e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.153:44768/0 (socket says 172.21.15.153:44768) 2024-03-21T10:09:46.813 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:46.812+0000 7f4bc3400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4bc4103f70 0x7f4bc4198a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:52758/0 (socket says 172.21.15.153:52758) 2024-03-21T10:09:46.813 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:46.812+0000 7f4bc3e00700 1 -- 172.21.15.153:0/2559142082 learned_addr learned my addr 172.21.15.153:0/2559142082 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:09:46.814 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:46.812+0000 7f4bc3400700 1 -- 172.21.15.153:0/2559142082 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4bc4101640 msgr2=0x7f4bc41984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:09:46.814 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:46.812+0000 7f4bc3400700 1 --2- 172.21.15.153:0/2559142082 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4bc4101640 0x7f4bc41984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:09:46.814 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:46.812+0000 7f4bc3400700 1 -- 172.21.15.153:0/2559142082 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bb4009520 con 0x7f4bc4103f70 2024-03-21T10:09:46.814 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:46.812+0000 7f4bc3400700 1 --2- 172.21.15.153:0/2559142082 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4bc4103f70 0x7f4bc4198a20 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f4bb0009910 tx=0x7f4bb000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:09:46.814 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:46.813+0000 7f4bc0c00700 1 -- 172.21.15.153:0/2559142082 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4bb001d070 con 0x7f4bc4103f70 2024-03-21T10:09:46.814 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:46.813+0000 7f4bc0c00700 1 -- 172.21.15.153:0/2559142082 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f4bb0005dc0 con 0x7f4bc4103f70 2024-03-21T10:09:46.814 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:46.813+0000 7f4bc0c00700 1 -- 172.21.15.153:0/2559142082 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4bb0017400 con 0x7f4bc4103f70 2024-03-21T10:09:46.814 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:46.813+0000 7f4bcb000700 1 -- 172.21.15.153:0/2559142082 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bb00095f0 con 0x7f4bc4103f70 2024-03-21T10:09:46.814 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:46.813+0000 7f4bcb000700 1 -- 172.21.15.153:0/2559142082 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bc419dee0 con 0x7f4bc4103f70 2024-03-21T10:09:46.815 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:46.814+0000 7f4bcb000700 1 -- 172.21.15.153:0/2559142082 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bc40fcc20 con 0x7f4bc4103f70 2024-03-21T10:09:46.816 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:46.816+0000 7f4bc0c00700 1 -- 172.21.15.153:0/2559142082 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 29) v1 ==== 92174+0+0 (secure 0 0 0) 0x7f4bb0003810 con 0x7f4bc4103f70 2024-03-21T10:09:46.818 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:46.817+0000 7f4bc0c00700 1 --2- 172.21.15.153:0/2559142082 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f4ba806e210 0x7f4ba80706d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:09:46.819 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:46.817+0000 7f4bc0c00700 1 -- 172.21.15.153:0/2559142082 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f4bb008f500 con 0x7f4bc4103f70 2024-03-21T10:09:46.819 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:46.818+0000 7f4bc3e00700 1 --2- 172.21.15.153:0/2559142082 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f4ba806e210 0x7f4ba80706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:09:46.819 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:46.818+0000 7f4bc3e00700 1 --2- 172.21.15.153:0/2559142082 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f4ba806e210 0x7f4ba80706d0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f4bb4009ef0 tx=0x7f4bb4009260 comp rx=0 tx=0).ready entity=mgr.14666 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:09:46.824 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:46.824+0000 7f4bc0c00700 1 -- 172.21.15.153:0/2559142082 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185454 (secure 0 0 0) 0x7f4bb005c450 con 0x7f4bc4103f70 2024-03-21T10:09:47.170 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:47.170+0000 7f4bcb000700 1 -- 172.21.15.153:0/2559142082 --> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f4bc40611d0 con 0x7f4ba806e210 2024-03-21T10:09:47.189 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:47.188+0000 7f4bc0c00700 1 -- 172.21.15.153:0/2559142082 <== mgr.14666 v2:172.21.15.153:6800/2524145044 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f4bc40611d0 con 0x7f4ba806e210 2024-03-21T10:09:47.195 INFO:teuthology.orchestra.run.smithi153.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-21T10:09:47.195 INFO:teuthology.orchestra.run.smithi153.stdout:alertmanager.smithi153 smithi153 *:9093,9094 running (10m) 1s ago 12m 13.4M - 0.25.0 c8568f914cd2 2cfcc225c7e6 2024-03-21T10:09:47.195 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi153 smithi153 running (12m) 1s ago 12m 7972k - 18.2.1 5be31c24972a 5b61994396a6 2024-03-21T10:09:47.195 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi174 smithi174 running (11m) 1s ago 11m 7894k - 18.2.1 5be31c24972a 4f7a5fc3ac28 2024-03-21T10:09:47.195 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi153 smithi153 running (12m) 1s ago 12m 6665k - 18.2.1 5be31c24972a 90a8b8ea03d1 2024-03-21T10:09:47.195 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi174 smithi174 running (11m) 1s ago 11m 6665k - 18.2.1 5be31c24972a a4c929e4dc77 2024-03-21T10:09:47.195 INFO:teuthology.orchestra.run.smithi153.stdout:grafana.smithi153 smithi153 *:3000 running (10m) 1s ago 11m 74.6M - 9.4.7 954c08fa6188 3af5800ecfef 2024-03-21T10:09:47.195 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.iuyluj smithi153 running (5m) 1s ago 5m 172M - 18.2.1 5be31c24972a 02f667cee979 2024-03-21T10:09:47.195 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.zipckr smithi153 running (5m) 1s ago 5m 163M - 18.2.1 5be31c24972a 68a853929f8f 2024-03-21T10:09:47.195 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.nhozag smithi174 running (5m) 1s ago 5m 151M - 18.2.1 5be31c24972a 2fee4000308f 2024-03-21T10:09:47.196 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.yyimst smithi174 running (5m) 1s ago 5m 162M - 18.2.1 5be31c24972a d78a729220d1 2024-03-21T10:09:47.196 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi153.tluxni smithi153 *:8443,9283,8765 running (95s) 1s ago 14m 494M - 19.0.0-2231-g2edfee68 237e43c7a65f 7ffee0e55382 2024-03-21T10:09:47.196 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi174.qpcwat smithi174 *:8443,9283,8765 running (13s) 1s ago 10m 64.9M - 19.0.0-2231-g2edfee68 237e43c7a65f 748968f216ab 2024-03-21T10:09:47.196 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi153 smithi153 running (14m) 1s ago 14m 54.0M 2048M 18.2.1 5be31c24972a 813963a95d69 2024-03-21T10:09:47.196 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi174 smithi174 running (10m) 1s ago 10m 41.9M 2048M 18.2.1 5be31c24972a 99204923eec6 2024-03-21T10:09:47.196 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi153 smithi153 *:9100 running (12m) 1s ago 12m 8162k - 1.5.0 0da6a335fe13 babbc29a0eaf 2024-03-21T10:09:47.196 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi174 smithi174 *:9100 running (10m) 1s ago 10m 7654k - 1.5.0 0da6a335fe13 86841435042a 2024-03-21T10:09:47.196 INFO:teuthology.orchestra.run.smithi153.stdout:osd.0 smithi153 running (9m) 1s ago 9m 178M 1217M 18.2.1 5be31c24972a 28912a2319b4 2024-03-21T10:09:47.196 INFO:teuthology.orchestra.run.smithi153.stdout:osd.1 smithi153 running (9m) 1s ago 9m 179M 1217M 18.2.1 5be31c24972a ca60681e2314 2024-03-21T10:09:47.196 INFO:teuthology.orchestra.run.smithi153.stdout:osd.2 smithi153 running (8m) 1s ago 8m 141M 1217M 18.2.1 5be31c24972a 8db799f8962a 2024-03-21T10:09:47.196 INFO:teuthology.orchestra.run.smithi153.stdout:osd.3 smithi174 running (8m) 1s ago 8m 194M 2241M 18.2.1 5be31c24972a dafc89e0658d 2024-03-21T10:09:47.196 INFO:teuthology.orchestra.run.smithi153.stdout:osd.4 smithi174 running (7m) 1s ago 7m 185M 2241M 18.2.1 5be31c24972a c4cd15701f05 2024-03-21T10:09:47.196 INFO:teuthology.orchestra.run.smithi153.stdout:osd.5 smithi174 running (6m) 1s ago 6m 179M 2241M 18.2.1 5be31c24972a 3b69886ce22e 2024-03-21T10:09:47.196 INFO:teuthology.orchestra.run.smithi153.stdout:prometheus.smithi153 smithi153 *:9095 running (22s) 1s ago 11m 48.1M - 2.43.0 a07b618ecd1d 9fe52bbde6e1 2024-03-21T10:09:47.197 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:47.194+0000 7f4bcb000700 1 -- 172.21.15.153:0/2559142082 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f4ba806e210 msgr2=0x7f4ba80706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:09:47.197 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:47.194+0000 7f4bcb000700 1 --2- 172.21.15.153:0/2559142082 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f4ba806e210 0x7f4ba80706d0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f4bb4009ef0 tx=0x7f4bb4009260 comp rx=0 tx=0).stop 2024-03-21T10:09:47.198 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:47.194+0000 7f4bcb000700 1 -- 172.21.15.153:0/2559142082 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4bc4103f70 msgr2=0x7f4bc4198a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:09:47.198 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:47.194+0000 7f4bcb000700 1 --2- 172.21.15.153:0/2559142082 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4bc4103f70 0x7f4bc4198a20 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f4bb0009910 tx=0x7f4bb000f690 comp rx=0 tx=0).stop 2024-03-21T10:09:47.198 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:47.195+0000 7f4bcb000700 1 -- 172.21.15.153:0/2559142082 shutdown_connections 2024-03-21T10:09:47.198 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:47.195+0000 7f4bcb000700 1 --2- 172.21.15.153:0/2559142082 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f4ba806e210 0x7f4ba80706d0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:09:47.198 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:47.195+0000 7f4bcb000700 1 --2- 172.21.15.153:0/2559142082 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4bc4101640 0x7f4bc41984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:09:47.198 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:47.195+0000 7f4bcb000700 1 --2- 172.21.15.153:0/2559142082 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4bc4103f70 0x7f4bc4198a20 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:09:47.198 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:47.195+0000 7f4bcb000700 1 -- 172.21.15.153:0/2559142082 >> 172.21.15.153:0/2559142082 conn(0x7f4bc40fb030 msgr2=0x7f4bc40ffcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:09:47.198 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:47.195+0000 7f4bcb000700 1 -- 172.21.15.153:0/2559142082 shutdown_connections 2024-03-21T10:09:47.198 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:47.195+0000 7f4bcb000700 1 -- 172.21.15.153:0/2559142082 wait complete. 2024-03-21T10:09:47.372 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:47.372+0000 7f5aa8a00700 1 -- 172.21.15.153:0/2209026909 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5aa4074dc0 msgr2=0x7f5aa4073220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:09:47.373 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:47.372+0000 7f5aa8a00700 1 --2- 172.21.15.153:0/2209026909 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5aa4074dc0 0x7f5aa4073220 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f5a94009910 tx=0x7f5a94009c20 comp rx=0 tx=0).stop 2024-03-21T10:09:47.373 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:47.372+0000 7f5aa8a00700 1 -- 172.21.15.153:0/2209026909 shutdown_connections 2024-03-21T10:09:47.373 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:47.372+0000 7f5aa8a00700 1 --2- 172.21.15.153:0/2209026909 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5aa40737f0 0x7f5aa4073c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:09:47.373 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:47.372+0000 7f5aa8a00700 1 --2- 172.21.15.153:0/2209026909 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5aa4074dc0 0x7f5aa4073220 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:09:47.373 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:47.372+0000 7f5aa8a00700 1 -- 172.21.15.153:0/2209026909 >> 172.21.15.153:0/2209026909 conn(0x7f5aa40fc110 msgr2=0x7f5aa40fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:09:47.373 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:47.372+0000 7f5aa8a00700 1 -- 172.21.15.153:0/2209026909 shutdown_connections 2024-03-21T10:09:47.373 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:47.372+0000 7f5aa8a00700 1 -- 172.21.15.153:0/2209026909 wait complete. 2024-03-21T10:09:47.374 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:47.373+0000 7f5aa8a00700 1 Processor -- start 2024-03-21T10:09:47.374 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:47.374+0000 7f5aa8a00700 1 -- start start 2024-03-21T10:09:47.375 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:47.374+0000 7f5aa8a00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5aa40737f0 0x7f5aa4198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:09:47.375 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:47.374+0000 7f5aa8a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5aa4074dc0 0x7f5aa4198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:09:47.375 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:47.374+0000 7f5aa8a00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa41990d0 con 0x7f5aa4074dc0 2024-03-21T10:09:47.375 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:47.374+0000 7f5aa8a00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa4199210 con 0x7f5aa40737f0 2024-03-21T10:09:47.375 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:47.374+0000 7f5aa1400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5aa4074dc0 0x7f5aa4198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:09:47.375 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:47.374+0000 7f5aa1400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5aa4074dc0 0x7f5aa4198ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:52774/0 (socket says 172.21.15.153:52774) 2024-03-21T10:09:47.375 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:47.374+0000 7f5aa1400700 1 -- 172.21.15.153:0/4248503302 learned_addr learned my addr 172.21.15.153:0/4248503302 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:09:47.375 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:47.374+0000 7f5aa1e00700 1 --2- 172.21.15.153:0/4248503302 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5aa40737f0 0x7f5aa4198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:09:47.376 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:47.375+0000 7f5aa1400700 1 -- 172.21.15.153:0/4248503302 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5aa40737f0 msgr2=0x7f5aa4198570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:09:47.376 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:47.375+0000 7f5aa1400700 1 --2- 172.21.15.153:0/4248503302 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5aa40737f0 0x7f5aa4198570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:09:47.376 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:47.375+0000 7f5aa1400700 1 -- 172.21.15.153:0/4248503302 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a940095f0 con 0x7f5aa4074dc0 2024-03-21T10:09:47.376 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:47.375+0000 7f5aa1400700 1 --2- 172.21.15.153:0/4248503302 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5aa4074dc0 0x7f5aa4198ab0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f5a9000d900 tx=0x7f5a9000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:09:47.376 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:47.375+0000 7f5a9aa00700 1 -- 172.21.15.153:0/4248503302 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5a900041f0 con 0x7f5aa4074dc0 2024-03-21T10:09:47.376 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:47.375+0000 7f5a9aa00700 1 -- 172.21.15.153:0/4248503302 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f5a90004350 con 0x7f5aa4074dc0 2024-03-21T10:09:47.376 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:47.375+0000 7f5a9aa00700 1 -- 172.21.15.153:0/4248503302 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5a90003d20 con 0x7f5aa4074dc0 2024-03-21T10:09:47.377 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:47.375+0000 7f5aa8a00700 1 -- 172.21.15.153:0/4248503302 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5aa419dcc0 con 0x7f5aa4074dc0 2024-03-21T10:09:47.377 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:47.376+0000 7f5aa8a00700 1 -- 172.21.15.153:0/4248503302 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5aa419e210 con 0x7f5aa4074dc0 2024-03-21T10:09:47.377 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:47.377+0000 7f5aa8a00700 1 -- 172.21.15.153:0/4248503302 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5aa410b920 con 0x7f5aa4074dc0 2024-03-21T10:09:47.381 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:47.379+0000 7f5a9aa00700 1 -- 172.21.15.153:0/4248503302 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 30) v1 ==== 92174+0+0 (secure 0 0 0) 0x7f5a90009730 con 0x7f5aa4074dc0 2024-03-21T10:09:47.381 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:47.380+0000 7f5a9aa00700 1 --2- 172.21.15.153:0/4248503302 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f5a8806e260 0x7f5a88070720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:09:47.381 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:47.381+0000 7f5a9aa00700 1 -- 172.21.15.153:0/4248503302 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f5a9008df70 con 0x7f5aa4074dc0 2024-03-21T10:09:47.381 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:47.381+0000 7f5aa1e00700 1 --2- 172.21.15.153:0/4248503302 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f5a8806e260 0x7f5a88070720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:09:47.382 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:47.382+0000 7f5aa1e00700 1 --2- 172.21.15.153:0/4248503302 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f5a8806e260 0x7f5a88070720 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f5a94006010 tx=0x7f5a9400b560 comp rx=0 tx=0).ready entity=mgr.14666 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:09:47.386 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:47.386+0000 7f5a9aa00700 1 -- 172.21.15.153:0/4248503302 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185454 (secure 0 0 0) 0x7f5a9005af40 con 0x7f5aa4074dc0 2024-03-21T10:09:47.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:47 smithi153 ceph-mon[29117]: from='client.24483 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:09:47.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:47 smithi153 ceph-mon[29117]: Standby manager daemon smithi174.qpcwat restarted 2024-03-21T10:09:47.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:47 smithi153 ceph-mon[29117]: Standby manager daemon smithi174.qpcwat started 2024-03-21T10:09:47.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:47 smithi153 ceph-mon[29117]: from='mgr.? 172.21.15.174:0/3543106391' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi174.qpcwat/crt"}]: dispatch 2024-03-21T10:09:47.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:47 smithi153 ceph-mon[29117]: from='mgr.? 172.21.15.174:0/3543106391' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-21T10:09:47.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:47 smithi153 ceph-mon[29117]: from='mgr.? 172.21.15.174:0/3543106391' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi174.qpcwat/key"}]: dispatch 2024-03-21T10:09:47.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:47 smithi153 ceph-mon[29117]: from='mgr.? 172.21.15.174:0/3543106391' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-21T10:09:47.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:47 smithi174 ceph-mon[37496]: from='client.24483 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:09:47.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:47 smithi174 ceph-mon[37496]: Standby manager daemon smithi174.qpcwat restarted 2024-03-21T10:09:47.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:47 smithi174 ceph-mon[37496]: Standby manager daemon smithi174.qpcwat started 2024-03-21T10:09:47.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:47 smithi174 ceph-mon[37496]: from='mgr.? 172.21.15.174:0/3543106391' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi174.qpcwat/crt"}]: dispatch 2024-03-21T10:09:47.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:47 smithi174 ceph-mon[37496]: from='mgr.? 172.21.15.174:0/3543106391' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-21T10:09:47.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:47 smithi174 ceph-mon[37496]: from='mgr.? 172.21.15.174:0/3543106391' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi174.qpcwat/key"}]: dispatch 2024-03-21T10:09:47.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:47 smithi174 ceph-mon[37496]: from='mgr.? 172.21.15.174:0/3543106391' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-21T10:09:47.785 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:47.784+0000 7f5aa8a00700 1 -- 172.21.15.153:0/4248503302 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f5aa410bb10 con 0x7f5aa4074dc0 2024-03-21T10:09:47.787 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:09:47.787 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": { 2024-03-21T10:09:47.787 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-03-21T10:09:47.787 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:09:47.787 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": { 2024-03-21T10:09:47.787 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:09:47.787 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:09:47.787 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": { 2024-03-21T10:09:47.787 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-03-21T10:09:47.787 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:09:47.787 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": { 2024-03-21T10:09:47.788 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-03-21T10:09:47.788 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:09:47.788 INFO:teuthology.orchestra.run.smithi153.stdout: "overall": { 2024-03-21T10:09:47.788 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 12, 2024-03-21T10:09:47.788 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:09:47.788 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-03-21T10:09:47.788 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:09:47.788 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:47.785+0000 7f5a9aa00700 1 -- 172.21.15.153:0/4248503302 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7f5a90003e80 con 0x7f5aa4074dc0 2024-03-21T10:09:47.789 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:47.789+0000 7f5aa8a00700 1 -- 172.21.15.153:0/4248503302 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f5a8806e260 msgr2=0x7f5a88070720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:09:47.789 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:47.789+0000 7f5aa8a00700 1 --2- 172.21.15.153:0/4248503302 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f5a8806e260 0x7f5a88070720 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f5a94006010 tx=0x7f5a9400b560 comp rx=0 tx=0).stop 2024-03-21T10:09:47.789 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:47.789+0000 7f5aa8a00700 1 -- 172.21.15.153:0/4248503302 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5aa4074dc0 msgr2=0x7f5aa4198ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:09:47.789 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:47.789+0000 7f5aa8a00700 1 --2- 172.21.15.153:0/4248503302 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5aa4074dc0 0x7f5aa4198ab0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f5a9000d900 tx=0x7f5a9000dc10 comp rx=0 tx=0).stop 2024-03-21T10:09:47.789 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:47.789+0000 7f5aa8a00700 1 -- 172.21.15.153:0/4248503302 shutdown_connections 2024-03-21T10:09:47.789 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:47.789+0000 7f5aa8a00700 1 --2- 172.21.15.153:0/4248503302 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f5a8806e260 0x7f5a88070720 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:09:47.790 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:47.789+0000 7f5aa8a00700 1 --2- 172.21.15.153:0/4248503302 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5aa40737f0 0x7f5aa4198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:09:47.790 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:47.789+0000 7f5aa8a00700 1 --2- 172.21.15.153:0/4248503302 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5aa4074dc0 0x7f5aa4198ab0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:09:47.790 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:47.789+0000 7f5aa8a00700 1 -- 172.21.15.153:0/4248503302 >> 172.21.15.153:0/4248503302 conn(0x7f5aa40fc110 msgr2=0x7f5aa4106990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:09:47.790 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:47.789+0000 7f5aa8a00700 1 -- 172.21.15.153:0/4248503302 shutdown_connections 2024-03-21T10:09:47.790 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:47.789+0000 7f5aa8a00700 1 -- 172.21.15.153:0/4248503302 wait complete. 2024-03-21T10:09:47.968 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:47.966+0000 7f9914400700 1 -- 172.21.15.153:0/3032410399 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f990c101640 msgr2=0x7f990c103a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:09:47.968 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:47.966+0000 7f9914400700 1 --2- 172.21.15.153:0/3032410399 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f990c101640 0x7f990c103a30 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f9900009910 tx=0x7f9900009c20 comp rx=0 tx=0).stop 2024-03-21T10:09:47.968 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:47.966+0000 7f9914400700 1 -- 172.21.15.153:0/3032410399 shutdown_connections 2024-03-21T10:09:47.968 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:47.966+0000 7f9914400700 1 --2- 172.21.15.153:0/3032410399 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f990c103f70 0x7f990c106360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:09:47.968 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:47.966+0000 7f9914400700 1 --2- 172.21.15.153:0/3032410399 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f990c101640 0x7f990c103a30 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:09:47.969 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:47.966+0000 7f9914400700 1 -- 172.21.15.153:0/3032410399 >> 172.21.15.153:0/3032410399 conn(0x7f990c0fb030 msgr2=0x7f990c0fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:09:47.969 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:47.966+0000 7f9914400700 1 -- 172.21.15.153:0/3032410399 shutdown_connections 2024-03-21T10:09:47.969 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:47.966+0000 7f9914400700 1 -- 172.21.15.153:0/3032410399 wait complete. 2024-03-21T10:09:47.969 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:47.968+0000 7f9914400700 1 Processor -- start 2024-03-21T10:09:47.969 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:47.968+0000 7f9914400700 1 -- start start 2024-03-21T10:09:47.970 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:47.968+0000 7f9914400700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f990c101640 0x7f990c198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:09:47.970 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:47.968+0000 7f9914400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f990c103f70 0x7f990c198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:09:47.970 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:47.968+0000 7f9914400700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f990c199070 con 0x7f990c103f70 2024-03-21T10:09:47.970 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:47.968+0000 7f9914400700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f990c1991b0 con 0x7f990c101640 2024-03-21T10:09:47.970 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:47.968+0000 7f9911000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f990c103f70 0x7f990c198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:09:47.970 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:47.969+0000 7f9911000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f990c103f70 0x7f990c198a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:52780/0 (socket says 172.21.15.153:52780) 2024-03-21T10:09:47.970 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:47.969+0000 7f9911000700 1 -- 172.21.15.153:0/2785604772 learned_addr learned my addr 172.21.15.153:0/2785604772 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:09:47.971 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:47.969+0000 7f9911a00700 1 --2- 172.21.15.153:0/2785604772 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f990c101640 0x7f990c198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:09:47.971 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:47.969+0000 7f9911000700 1 -- 172.21.15.153:0/2785604772 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f990c101640 msgr2=0x7f990c198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:09:47.971 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:47.969+0000 7f9911000700 1 --2- 172.21.15.153:0/2785604772 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f990c101640 0x7f990c198510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:09:47.971 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:47.969+0000 7f9911000700 1 -- 172.21.15.153:0/2785604772 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99000095f0 con 0x7f990c103f70 2024-03-21T10:09:47.971 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:47.969+0000 7f9911000700 1 --2- 172.21.15.153:0/2785604772 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f990c103f70 0x7f990c198a50 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f98f800d8d0 tx=0x7f98f800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:09:47.971 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:47.970+0000 7f990a000700 1 -- 172.21.15.153:0/2785604772 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f98f8009940 con 0x7f990c103f70 2024-03-21T10:09:47.971 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:47.970+0000 7f9914400700 1 -- 172.21.15.153:0/2785604772 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f990c19dc60 con 0x7f990c103f70 2024-03-21T10:09:47.972 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:47.970+0000 7f990a000700 1 -- 172.21.15.153:0/2785604772 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f98f8010460 con 0x7f990c103f70 2024-03-21T10:09:47.972 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:47.970+0000 7f990a000700 1 -- 172.21.15.153:0/2785604772 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f98f800f5f0 con 0x7f990c103f70 2024-03-21T10:09:47.972 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:47.970+0000 7f9914400700 1 -- 172.21.15.153:0/2785604772 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f990c19e1b0 con 0x7f990c103f70 2024-03-21T10:09:47.972 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:47.971+0000 7f9914400700 1 -- 172.21.15.153:0/2785604772 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f990c0fcc20 con 0x7f990c103f70 2024-03-21T10:09:47.973 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:47.972+0000 7f990a000700 1 -- 172.21.15.153:0/2785604772 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 30) v1 ==== 92174+0+0 (secure 0 0 0) 0x7f98f80105d0 con 0x7f990c103f70 2024-03-21T10:09:47.974 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:47.973+0000 7f990a000700 1 --2- 172.21.15.153:0/2785604772 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f98f406e220 0x7f98f40706e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:09:47.974 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:47.973+0000 7f990a000700 1 -- 172.21.15.153:0/2785604772 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f98f808e070 con 0x7f990c103f70 2024-03-21T10:09:47.975 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:47.973+0000 7f9911a00700 1 --2- 172.21.15.153:0/2785604772 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f98f406e220 0x7f98f40706e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:09:47.975 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:47.974+0000 7f9911a00700 1 --2- 172.21.15.153:0/2785604772 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f98f406e220 0x7f98f40706e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f9900005b40 tx=0x7f9900005a90 comp rx=0 tx=0).ready entity=mgr.14666 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:09:47.981 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:47.979+0000 7f990a000700 1 -- 172.21.15.153:0/2785604772 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185454 (secure 0 0 0) 0x7f98f805b040 con 0x7f990c103f70 2024-03-21T10:09:48.323 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:48.322+0000 7f9914400700 1 -- 172.21.15.153:0/2785604772 --> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f990c0611d0 con 0x7f98f406e220 2024-03-21T10:09:48.325 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:48.323+0000 7f990a000700 1 -- 172.21.15.153:0/2785604772 <== mgr.14666 v2:172.21.15.153:6800/2524145044 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+360 (secure 0 0 0) 0x7f990c0611d0 con 0x7f98f406e220 2024-03-21T10:09:48.325 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:09:48.325 INFO:teuthology.orchestra.run.smithi153.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232", 2024-03-21T10:09:48.325 INFO:teuthology.orchestra.run.smithi153.stdout: "in_progress": true, 2024-03-21T10:09:48.325 INFO:teuthology.orchestra.run.smithi153.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-03-21T10:09:48.325 INFO:teuthology.orchestra.run.smithi153.stdout: "services_complete": [ 2024-03-21T10:09:48.325 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr" 2024-03-21T10:09:48.325 INFO:teuthology.orchestra.run.smithi153.stdout: ], 2024-03-21T10:09:48.325 INFO:teuthology.orchestra.run.smithi153.stdout: "progress": "2/2 daemons upgraded", 2024-03-21T10:09:48.325 INFO:teuthology.orchestra.run.smithi153.stdout: "message": "Currently upgrading mgr daemons", 2024-03-21T10:09:48.325 INFO:teuthology.orchestra.run.smithi153.stdout: "is_paused": false 2024-03-21T10:09:48.326 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:09:48.327 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:48.326+0000 7f9914400700 1 -- 172.21.15.153:0/2785604772 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f98f406e220 msgr2=0x7f98f40706e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:09:48.328 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:48.326+0000 7f9914400700 1 --2- 172.21.15.153:0/2785604772 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f98f406e220 0x7f98f40706e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f9900005b40 tx=0x7f9900005a90 comp rx=0 tx=0).stop 2024-03-21T10:09:48.328 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:48.326+0000 7f9914400700 1 -- 172.21.15.153:0/2785604772 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f990c103f70 msgr2=0x7f990c198a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:09:48.328 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:48.326+0000 7f9914400700 1 --2- 172.21.15.153:0/2785604772 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f990c103f70 0x7f990c198a50 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f98f800d8d0 tx=0x7f98f800dc90 comp rx=0 tx=0).stop 2024-03-21T10:09:48.328 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:48.326+0000 7f9914400700 1 -- 172.21.15.153:0/2785604772 shutdown_connections 2024-03-21T10:09:48.328 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:48.326+0000 7f9914400700 1 --2- 172.21.15.153:0/2785604772 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f98f406e220 0x7f98f40706e0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:09:48.328 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:48.326+0000 7f9914400700 1 --2- 172.21.15.153:0/2785604772 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f990c101640 0x7f990c198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:09:48.328 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:48.326+0000 7f9914400700 1 --2- 172.21.15.153:0/2785604772 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f990c103f70 0x7f990c198a50 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:09:48.329 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:48.326+0000 7f9914400700 1 -- 172.21.15.153:0/2785604772 >> 172.21.15.153:0/2785604772 conn(0x7f990c0fb030 msgr2=0x7f990c104be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:09:48.329 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:48.327+0000 7f9914400700 1 -- 172.21.15.153:0/2785604772 shutdown_connections 2024-03-21T10:09:48.329 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:09:48.327+0000 7f9914400700 1 -- 172.21.15.153:0/2785604772 wait complete. 2024-03-21T10:09:48.650 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:48 smithi174 ceph-mon[37496]: from='client.24487 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:09:48.650 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:48 smithi174 ceph-mon[37496]: from='client.14740 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:09:48.650 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:48 smithi174 ceph-mon[37496]: pgmap v40: 65 pgs: 65 active+clean; 301 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 106 op/s 2024-03-21T10:09:48.650 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:48 smithi174 ceph-mon[37496]: mgrmap e30: smithi153.tluxni(active, since 74s), standbys: smithi174.qpcwat 2024-03-21T10:09:48.651 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:48 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:09:48.651 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:48 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:09:48.651 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:48 smithi174 ceph-mon[37496]: from='client.? 172.21.15.153:0/4248503302' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:09:48.651 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:48 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:09:48.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:48 smithi153 ceph-mon[29117]: from='client.24487 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:09:48.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:48 smithi153 ceph-mon[29117]: from='client.14740 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:09:48.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:48 smithi153 ceph-mon[29117]: pgmap v40: 65 pgs: 65 active+clean; 301 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 106 op/s 2024-03-21T10:09:48.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:48 smithi153 ceph-mon[29117]: mgrmap e30: smithi153.tluxni(active, since 74s), standbys: smithi174.qpcwat 2024-03-21T10:09:48.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:48 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:09:48.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:48 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:09:48.808 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:48 smithi153 ceph-mon[29117]: from='client.? 172.21.15.153:0/4248503302' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:09:48.808 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:48 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:09:49.664 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:49 smithi174 ceph-mon[37496]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:09:49.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:49 smithi153 ceph-mon[29117]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:09:50.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:50 smithi174 ceph-mon[37496]: pgmap v41: 65 pgs: 65 active+clean; 301 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 106 op/s 2024-03-21T10:09:50.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:50 smithi153 ceph-mon[29117]: pgmap v41: 65 pgs: 65 active+clean; 301 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 106 op/s 2024-03-21T10:09:52.609 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:52 smithi153 ceph-mon[29117]: pgmap v42: 65 pgs: 65 active+clean; 310 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 155 op/s 2024-03-21T10:09:52.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:52 smithi174 ceph-mon[37496]: pgmap v42: 65 pgs: 65 active+clean; 310 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 155 op/s 2024-03-21T10:09:54.663 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:54 smithi174 ceph-mon[37496]: pgmap v43: 65 pgs: 65 active+clean; 310 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 99 op/s 2024-03-21T10:09:54.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:54 smithi153 ceph-mon[29117]: pgmap v43: 65 pgs: 65 active+clean; 310 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 99 op/s 2024-03-21T10:09:56.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:55 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:09:56.058 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:55 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:09:56.058 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:55 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:09:56.058 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:55 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:09:56.058 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:55 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:09:56.058 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:55 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:09:56.058 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:55 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:09:56.058 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:55 smithi153 ceph-mon[29117]: Upgrade: Setting container_image for all mgr 2024-03-21T10:09:56.058 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:55 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:09:56.058 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:55 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi153.tluxni"}]: dispatch 2024-03-21T10:09:56.058 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:55 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi153.tluxni"}]': finished 2024-03-21T10:09:56.058 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:55 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi174.qpcwat"}]: dispatch 2024-03-21T10:09:56.059 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:55 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi174.qpcwat"}]': finished 2024-03-21T10:09:56.059 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:55 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:09:56.059 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:55 smithi153 ceph-mon[29117]: Upgrade: Setting container_image for all rgw 2024-03-21T10:09:56.059 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:55 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:09:56.059 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:55 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:09:56.059 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:55 smithi153 ceph-mon[29117]: Upgrade: Setting container_image for all rbd-mirror 2024-03-21T10:09:56.059 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:55 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:09:56.059 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:55 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:09:56.059 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:55 smithi153 ceph-mon[29117]: Upgrade: Setting container_image for all cephfs-mirror 2024-03-21T10:09:56.059 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:55 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:09:56.059 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:55 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:09:56.059 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:55 smithi153 ceph-mon[29117]: Upgrade: Setting container_image for all iscsi 2024-03-21T10:09:56.060 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:55 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:09:56.060 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:55 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:09:56.060 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:55 smithi153 ceph-mon[29117]: Upgrade: Setting container_image for all nfs 2024-03-21T10:09:56.060 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:55 smithi153 ceph-mon[29117]: pgmap v44: 65 pgs: 65 active+clean; 317 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 154 op/s 2024-03-21T10:09:56.060 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:55 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:09:56.060 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:55 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:09:56.060 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:55 smithi153 ceph-mon[29117]: Upgrade: Setting container_image for all nvmeof 2024-03-21T10:09:56.060 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:55 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:09:56.203 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:55 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:09:56.203 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:55 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:09:56.203 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:55 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:09:56.203 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:55 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:09:56.203 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:55 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:09:56.203 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:55 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:09:56.203 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:55 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:09:56.203 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:55 smithi174 ceph-mon[37496]: Upgrade: Setting container_image for all mgr 2024-03-21T10:09:56.203 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:55 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:09:56.203 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:55 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi153.tluxni"}]: dispatch 2024-03-21T10:09:56.203 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:55 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi153.tluxni"}]': finished 2024-03-21T10:09:56.204 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:55 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi174.qpcwat"}]: dispatch 2024-03-21T10:09:56.204 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:55 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi174.qpcwat"}]': finished 2024-03-21T10:09:56.204 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:55 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:09:56.204 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:55 smithi174 ceph-mon[37496]: Upgrade: Setting container_image for all rgw 2024-03-21T10:09:56.204 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:55 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:09:56.204 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:55 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:09:56.204 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:55 smithi174 ceph-mon[37496]: Upgrade: Setting container_image for all rbd-mirror 2024-03-21T10:09:56.204 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:55 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:09:56.204 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:55 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:09:56.204 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:55 smithi174 ceph-mon[37496]: Upgrade: Setting container_image for all cephfs-mirror 2024-03-21T10:09:56.204 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:55 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:09:56.205 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:55 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:09:56.205 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:55 smithi174 ceph-mon[37496]: Upgrade: Setting container_image for all iscsi 2024-03-21T10:09:56.205 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:55 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:09:56.205 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:55 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:09:56.205 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:55 smithi174 ceph-mon[37496]: Upgrade: Setting container_image for all nfs 2024-03-21T10:09:56.205 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:55 smithi174 ceph-mon[37496]: pgmap v44: 65 pgs: 65 active+clean; 317 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 154 op/s 2024-03-21T10:09:56.205 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:55 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:09:56.205 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:55 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:09:56.205 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:55 smithi174 ceph-mon[37496]: Upgrade: Setting container_image for all nvmeof 2024-03-21T10:09:56.205 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:55 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:09:58.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:58 smithi153 ceph-mon[29117]: pgmap v45: 65 pgs: 65 active+clean; 317 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 104 op/s 2024-03-21T10:09:58.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:58 smithi174 ceph-mon[37496]: pgmap v45: 65 pgs: 65 active+clean; 317 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 104 op/s 2024-03-21T10:09:59.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:59 smithi153 ceph-mon[29117]: Upgrade: Updating node-exporter.smithi153 (1/2) 2024-03-21T10:09:59.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:09:59 smithi153 ceph-mon[29117]: Deploying daemon node-exporter.smithi153 on smithi153 2024-03-21T10:09:59.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:59 smithi174 ceph-mon[37496]: Upgrade: Updating node-exporter.smithi153 (1/2) 2024-03-21T10:09:59.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:09:59 smithi174 ceph-mon[37496]: Deploying daemon node-exporter.smithi153 on smithi153 2024-03-21T10:10:00.446 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:10:00 smithi174 ceph-mon[37496]: pgmap v46: 65 pgs: 65 active+clean; 317 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 104 op/s 2024-03-21T10:10:00.446 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:10:00 smithi174 ceph-mon[37496]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-03-21T10:10:00.446 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:10:00 smithi174 ceph-mon[37496]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-21T10:10:00.446 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:10:00 smithi174 ceph-mon[37496]: fs cephfs has deprecated feature inline_data enabled. 2024-03-21T10:10:00.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:10:00 smithi153 ceph-mon[29117]: pgmap v46: 65 pgs: 65 active+clean; 317 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 104 op/s 2024-03-21T10:10:00.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:10:00 smithi153 ceph-mon[29117]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-03-21T10:10:00.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:10:00 smithi153 ceph-mon[29117]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-21T10:10:00.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:10:00 smithi153 ceph-mon[29117]: fs cephfs has deprecated feature inline_data enabled. 2024-03-21T10:10:02.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:10:02 smithi153 ceph-mon[29117]: pgmap v47: 65 pgs: 65 active+clean; 314 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 145 op/s 2024-03-21T10:10:02.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:10:02 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:10:02.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:10:02 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:10:02.597 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:10:02 smithi174 ceph-mon[37496]: pgmap v47: 65 pgs: 65 active+clean; 314 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 145 op/s 2024-03-21T10:10:02.598 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:10:02 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:10:02.598 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:10:02 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:10:04.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:10:04 smithi174 ceph-mon[37496]: pgmap v48: 65 pgs: 65 active+clean; 314 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 96 op/s 2024-03-21T10:10:04.453 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:10:04 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:10:04.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:10:04 smithi153 ceph-mon[29117]: pgmap v48: 65 pgs: 65 active+clean; 314 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 96 op/s 2024-03-21T10:10:04.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:10:04 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:10:05.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:10:05 smithi153 ceph-mon[29117]: mgrmap e31: smithi153.tluxni(active, since 91s), standbys: smithi174.qpcwat 2024-03-21T10:10:05.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:10:05 smithi174 ceph-mon[37496]: mgrmap e31: smithi153.tluxni(active, since 91s), standbys: smithi174.qpcwat 2024-03-21T10:10:06.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:10:06 smithi153 ceph-mon[29117]: Upgrade: Updating node-exporter.smithi174 (2/2) 2024-03-21T10:10:06.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:10:06 smithi153 ceph-mon[29117]: Deploying daemon node-exporter.smithi174 on smithi174 2024-03-21T10:10:06.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:10:06 smithi153 ceph-mon[29117]: pgmap v49: 65 pgs: 65 active+clean; 307 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 160 op/s 2024-03-21T10:10:06.659 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:10:06 smithi174 ceph-mon[37496]: Upgrade: Updating node-exporter.smithi174 (2/2) 2024-03-21T10:10:06.659 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:10:06 smithi174 ceph-mon[37496]: Deploying daemon node-exporter.smithi174 on smithi174 2024-03-21T10:10:06.659 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:10:06 smithi174 ceph-mon[37496]: pgmap v49: 65 pgs: 65 active+clean; 307 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 160 op/s 2024-03-21T10:10:08.545 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:10:08 smithi174 ceph-mon[37496]: pgmap v50: 65 pgs: 65 active+clean; 307 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 104 op/s 2024-03-21T10:10:08.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:10:08 smithi153 ceph-mon[29117]: pgmap v50: 65 pgs: 65 active+clean; 307 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 104 op/s 2024-03-21T10:10:10.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:10:09 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:10:10.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:10:09 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:10:10.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:10:09 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:10:10.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:10:09 smithi174 ceph-mon[37496]: pgmap v51: 65 pgs: 65 active+clean; 307 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 104 op/s 2024-03-21T10:10:10.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:10:09 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:10:10.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:10:09 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:10:10.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:10:09 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:10:10.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:10:09 smithi153 ceph-mon[29117]: pgmap v51: 65 pgs: 65 active+clean; 307 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 104 op/s 2024-03-21T10:10:12.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:10:12 smithi153 ceph-mon[29117]: pgmap v52: 65 pgs: 65 active+clean; 312 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 155 op/s 2024-03-21T10:10:12.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:10:12 smithi174 ceph-mon[37496]: pgmap v52: 65 pgs: 65 active+clean; 312 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 155 op/s 2024-03-21T10:10:14.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:10:14 smithi153 ceph-mon[29117]: pgmap v53: 65 pgs: 65 active+clean; 312 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 114 op/s 2024-03-21T10:10:14.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:10:14 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:10:14.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:10:14 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:10:14.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:10:14 smithi174 ceph-mon[37496]: pgmap v53: 65 pgs: 65 active+clean; 312 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 114 op/s 2024-03-21T10:10:14.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:10:14 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:10:14.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:10:14 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:10:16.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:10:16 smithi153 ceph-mon[29117]: pgmap v54: 65 pgs: 65 active+clean; 318 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 179 op/s 2024-03-21T10:10:16.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:10:16 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:10:16.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:10:16 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:10:16.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:10:16 smithi174 ceph-mon[37496]: pgmap v54: 65 pgs: 65 active+clean; 318 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 179 op/s 2024-03-21T10:10:16.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:10:16 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:10:16.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:10:16 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:10:18.529 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:10:18 smithi153 ceph-mon[29117]: pgmap v55: 65 pgs: 65 active+clean; 318 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 115 op/s 2024-03-21T10:10:18.530 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:18.526+0000 7f077d000700 1 -- 172.21.15.153:0/2281794194 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0778100bb0 msgr2=0x7f0778100fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:10:18.530 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:18.527+0000 7f077d000700 1 --2- 172.21.15.153:0/2281794194 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0778100bb0 0x7f0778100fd0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f077000b0a0 tx=0x7f077000b3b0 comp rx=0 tx=0).stop 2024-03-21T10:10:18.530 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:18.527+0000 7f077d000700 1 -- 172.21.15.153:0/2281794194 shutdown_connections 2024-03-21T10:10:18.530 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:18.527+0000 7f077d000700 1 --2- 172.21.15.153:0/2281794194 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0778101db0 0x7f0778102210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:10:18.530 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:18.528+0000 7f077d000700 1 --2- 172.21.15.153:0/2281794194 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0778100bb0 0x7f0778100fd0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:10:18.530 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:18.528+0000 7f077d000700 1 -- 172.21.15.153:0/2281794194 >> 172.21.15.153:0/2281794194 conn(0x7f07780fc110 msgr2=0x7f07780fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:10:18.530 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:18.529+0000 7f077d000700 1 -- 172.21.15.153:0/2281794194 shutdown_connections 2024-03-21T10:10:18.532 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:18.529+0000 7f077d000700 1 -- 172.21.15.153:0/2281794194 wait complete. 2024-03-21T10:10:18.532 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:18.530+0000 7f077d000700 1 Processor -- start 2024-03-21T10:10:18.532 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:18.531+0000 7f077d000700 1 -- start start 2024-03-21T10:10:18.534 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:18.532+0000 7f077d000700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0778100bb0 0x7f0778071de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:10:18.534 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:18.532+0000 7f077d000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0778101db0 0x7f0778072320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:10:18.534 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:18.532+0000 7f077d000700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0778072940 con 0x7f0778101db0 2024-03-21T10:10:18.534 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:18.532+0000 7f077d000700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0778072a80 con 0x7f0778100bb0 2024-03-21T10:10:18.535 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:18.533+0000 7f0775e00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0778100bb0 0x7f0778071de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:10:18.536 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:18.533+0000 7f0775400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0778101db0 0x7f0778072320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:10:18.536 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:18.533+0000 7f0775e00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0778100bb0 0x7f0778071de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.153:44408/0 (socket says 172.21.15.153:44408) 2024-03-21T10:10:18.536 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:18.533+0000 7f0775e00700 1 -- 172.21.15.153:0/2179637578 learned_addr learned my addr 172.21.15.153:0/2179637578 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:10:18.536 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:18.534+0000 7f0775e00700 1 -- 172.21.15.153:0/2179637578 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0778101db0 msgr2=0x7f0778072320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:10:18.537 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:18.535+0000 7f0775e00700 1 --2- 172.21.15.153:0/2179637578 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0778101db0 0x7f0778072320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:10:18.537 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:18.535+0000 7f0775e00700 1 -- 172.21.15.153:0/2179637578 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0770009d00 con 0x7f0778100bb0 2024-03-21T10:10:18.537 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:18.535+0000 7f0775e00700 1 --2- 172.21.15.153:0/2179637578 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0778100bb0 0x7f0778071de0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0770009fd0 tx=0x7f07700096e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:10:18.537 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:18.535+0000 7f0775400700 1 --2- 172.21.15.153:0/2179637578 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0778101db0 0x7f0778072320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:10:18.539 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:18.537+0000 7f076ea00700 1 -- 172.21.15.153:0/2179637578 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f077000e040 con 0x7f0778100bb0 2024-03-21T10:10:18.539 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:18.537+0000 7f076ea00700 1 -- 172.21.15.153:0/2179637578 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0770003de0 con 0x7f0778100bb0 2024-03-21T10:10:18.539 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:18.537+0000 7f076ea00700 1 -- 172.21.15.153:0/2179637578 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f077001d910 con 0x7f0778100bb0 2024-03-21T10:10:18.540 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:18.537+0000 7f077d000700 1 -- 172.21.15.153:0/2179637578 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07780734b0 con 0x7f0778100bb0 2024-03-21T10:10:18.540 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:18.538+0000 7f077d000700 1 -- 172.21.15.153:0/2179637578 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0778073a20 con 0x7f0778100bb0 2024-03-21T10:10:18.545 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:18.542+0000 7f077d000700 1 -- 172.21.15.153:0/2179637578 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0778109710 con 0x7f0778100bb0 2024-03-21T10:10:18.545 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:18.544+0000 7f076ea00700 1 -- 172.21.15.153:0/2179637578 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 31) v1 ==== 92072+0+0 (secure 0 0 0) 0x7f07700048a0 con 0x7f0778100bb0 2024-03-21T10:10:18.547 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:18.545+0000 7f076ea00700 1 --2- 172.21.15.153:0/2179637578 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f075c06e200 0x7f075c0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:10:18.547 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:18.546+0000 7f076ea00700 1 -- 172.21.15.153:0/2179637578 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f077008f710 con 0x7f0778100bb0 2024-03-21T10:10:18.548 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:18.546+0000 7f0775400700 1 --2- 172.21.15.153:0/2179637578 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f075c06e200 0x7f075c0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:10:18.549 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:18.548+0000 7f0775400700 1 --2- 172.21.15.153:0/2179637578 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f075c06e200 0x7f075c0706c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0764005950 tx=0x7f076400b410 comp rx=0 tx=0).ready entity=mgr.14666 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:10:18.562 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:18.559+0000 7f076ea00700 1 -- 172.21.15.153:0/2179637578 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185454 (secure 0 0 0) 0x7f077005c980 con 0x7f0778100bb0 2024-03-21T10:10:18.591 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:10:18 smithi174 ceph-mon[37496]: pgmap v55: 65 pgs: 65 active+clean; 318 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 115 op/s 2024-03-21T10:10:18.907 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:18.905+0000 7f077d000700 1 -- 172.21.15.153:0/2179637578 --> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0778106700 con 0x7f075c06e200 2024-03-21T10:10:18.909 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:18.907+0000 7f076ea00700 1 -- 172.21.15.153:0/2179637578 <== mgr.14666 v2:172.21.15.153:6800/2524145044 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+370 (secure 0 0 0) 0x7f0778106700 con 0x7f075c06e200 2024-03-21T10:10:18.913 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:18.911+0000 7f077d000700 1 -- 172.21.15.153:0/2179637578 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f075c06e200 msgr2=0x7f075c0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:10:18.913 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:18.911+0000 7f077d000700 1 --2- 172.21.15.153:0/2179637578 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f075c06e200 0x7f075c0706c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0764005950 tx=0x7f076400b410 comp rx=0 tx=0).stop 2024-03-21T10:10:18.913 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:18.911+0000 7f077d000700 1 -- 172.21.15.153:0/2179637578 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0778100bb0 msgr2=0x7f0778071de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:10:18.913 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:18.911+0000 7f077d000700 1 --2- 172.21.15.153:0/2179637578 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0778100bb0 0x7f0778071de0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0770009fd0 tx=0x7f07700096e0 comp rx=0 tx=0).stop 2024-03-21T10:10:18.913 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:18.911+0000 7f077d000700 1 -- 172.21.15.153:0/2179637578 shutdown_connections 2024-03-21T10:10:18.913 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:18.911+0000 7f077d000700 1 --2- 172.21.15.153:0/2179637578 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f075c06e200 0x7f075c0706c0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:10:18.913 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:18.911+0000 7f077d000700 1 --2- 172.21.15.153:0/2179637578 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0778100bb0 0x7f0778071de0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:10:18.913 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:18.911+0000 7f077d000700 1 --2- 172.21.15.153:0/2179637578 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0778101db0 0x7f0778072320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:10:18.913 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:18.911+0000 7f077d000700 1 -- 172.21.15.153:0/2179637578 >> 172.21.15.153:0/2179637578 conn(0x7f07780fc110 msgr2=0x7f0778104fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:10:18.914 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:18.911+0000 7f077d000700 1 -- 172.21.15.153:0/2179637578 shutdown_connections 2024-03-21T10:10:18.914 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:18.911+0000 7f077d000700 1 -- 172.21.15.153:0/2179637578 wait complete. 2024-03-21T10:10:18.929 INFO:teuthology.orchestra.run.smithi153.stdout:true 2024-03-21T10:10:19.097 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:19.095+0000 7fd74e200700 1 -- 172.21.15.153:0/2101527158 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd748074dc0 msgr2=0x7fd748073220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:10:19.097 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:19.095+0000 7fd74e200700 1 --2- 172.21.15.153:0/2101527158 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd748074dc0 0x7fd748073220 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fd738009910 tx=0x7fd738009c20 comp rx=0 tx=0).stop 2024-03-21T10:10:19.097 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:19.096+0000 7fd74e200700 1 -- 172.21.15.153:0/2101527158 shutdown_connections 2024-03-21T10:10:19.098 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:19.096+0000 7fd74e200700 1 --2- 172.21.15.153:0/2101527158 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd7480737f0 0x7fd748073c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:10:19.098 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:19.096+0000 7fd74e200700 1 --2- 172.21.15.153:0/2101527158 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd748074dc0 0x7fd748073220 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:10:19.098 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:19.096+0000 7fd74e200700 1 -- 172.21.15.153:0/2101527158 >> 172.21.15.153:0/2101527158 conn(0x7fd7480fc110 msgr2=0x7fd7480fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:10:19.098 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:19.096+0000 7fd74e200700 1 -- 172.21.15.153:0/2101527158 shutdown_connections 2024-03-21T10:10:19.098 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:19.096+0000 7fd74e200700 1 -- 172.21.15.153:0/2101527158 wait complete. 2024-03-21T10:10:19.098 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:19.097+0000 7fd74e200700 1 Processor -- start 2024-03-21T10:10:19.099 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:19.097+0000 7fd74e200700 1 -- start start 2024-03-21T10:10:19.099 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:19.098+0000 7fd74e200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd7480737f0 0x7fd748198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:10:19.099 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:19.098+0000 7fd74e200700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd748074dc0 0x7fd748198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:10:19.099 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:19.098+0000 7fd74e200700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7481990d0 con 0x7fd7480737f0 2024-03-21T10:10:19.099 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:19.098+0000 7fd74e200700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd748199210 con 0x7fd748074dc0 2024-03-21T10:10:19.100 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:19.098+0000 7fd747400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd7480737f0 0x7fd748198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:10:19.100 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:19.098+0000 7fd747400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd7480737f0 0x7fd748198570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41202/0 (socket says 172.21.15.153:41202) 2024-03-21T10:10:19.100 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:19.098+0000 7fd747400700 1 -- 172.21.15.153:0/2121977275 learned_addr learned my addr 172.21.15.153:0/2121977275 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:10:19.100 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:19.098+0000 7fd746a00700 1 --2- 172.21.15.153:0/2121977275 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd748074dc0 0x7fd748198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:10:19.100 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:19.099+0000 7fd747400700 1 -- 172.21.15.153:0/2121977275 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd748074dc0 msgr2=0x7fd748198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:10:19.100 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:19.099+0000 7fd747400700 1 --2- 172.21.15.153:0/2121977275 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd748074dc0 0x7fd748198ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:10:19.100 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:19.099+0000 7fd747400700 1 -- 172.21.15.153:0/2121977275 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7380095f0 con 0x7fd7480737f0 2024-03-21T10:10:19.101 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:19.099+0000 7fd747400700 1 --2- 172.21.15.153:0/2121977275 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd7480737f0 0x7fd748198570 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fd73800bd70 tx=0x7fd73800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:10:19.101 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:19.099+0000 7fd73fe00700 1 -- 172.21.15.153:0/2121977275 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd73801d070 con 0x7fd7480737f0 2024-03-21T10:10:19.101 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:19.100+0000 7fd73fe00700 1 -- 172.21.15.153:0/2121977275 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fd73800f460 con 0x7fd7480737f0 2024-03-21T10:10:19.101 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:19.100+0000 7fd74e200700 1 -- 172.21.15.153:0/2121977275 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd74819dc60 con 0x7fd7480737f0 2024-03-21T10:10:19.101 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:19.100+0000 7fd73fe00700 1 -- 172.21.15.153:0/2121977275 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd738021620 con 0x7fd7480737f0 2024-03-21T10:10:19.103 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:19.100+0000 7fd74e200700 1 -- 172.21.15.153:0/2121977275 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd74819e150 con 0x7fd7480737f0 2024-03-21T10:10:19.103 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:19.101+0000 7fd74e200700 1 -- 172.21.15.153:0/2121977275 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd74810b920 con 0x7fd7480737f0 2024-03-21T10:10:19.104 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:19.102+0000 7fd73fe00700 1 -- 172.21.15.153:0/2121977275 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 31) v1 ==== 92072+0+0 (secure 0 0 0) 0x7fd73800fa40 con 0x7fd7480737f0 2024-03-21T10:10:19.105 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:19.103+0000 7fd73fe00700 1 --2- 172.21.15.153:0/2121977275 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7fd72c06e0e0 0x7fd72c0705a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:10:19.105 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:19.104+0000 7fd73fe00700 1 -- 172.21.15.153:0/2121977275 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fd73808f0d0 con 0x7fd7480737f0 2024-03-21T10:10:19.106 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:19.105+0000 7fd746a00700 1 --2- 172.21.15.153:0/2121977275 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7fd72c06e0e0 0x7fd72c0705a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:10:19.107 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:19.106+0000 7fd746a00700 1 --2- 172.21.15.153:0/2121977275 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7fd72c06e0e0 0x7fd72c0705a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd734005950 tx=0x7fd7340058e0 comp rx=0 tx=0).ready entity=mgr.14666 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:10:19.111 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:19.109+0000 7fd73fe00700 1 -- 172.21.15.153:0/2121977275 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185454 (secure 0 0 0) 0x7fd73805c360 con 0x7fd7480737f0 2024-03-21T10:10:19.452 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:19.451+0000 7fd74e200700 1 -- 172.21.15.153:0/2121977275 --> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd7481080b0 con 0x7fd72c06e0e0 2024-03-21T10:10:19.454 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:19.453+0000 7fd73fe00700 1 -- 172.21.15.153:0/2121977275 <== mgr.14666 v2:172.21.15.153:6800/2524145044 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+370 (secure 0 0 0) 0x7fd7481080b0 con 0x7fd72c06e0e0 2024-03-21T10:10:19.457 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:19.456+0000 7fd74e200700 1 -- 172.21.15.153:0/2121977275 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7fd72c06e0e0 msgr2=0x7fd72c0705a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:10:19.457 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:19.456+0000 7fd74e200700 1 --2- 172.21.15.153:0/2121977275 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7fd72c06e0e0 0x7fd72c0705a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd734005950 tx=0x7fd7340058e0 comp rx=0 tx=0).stop 2024-03-21T10:10:19.458 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:19.456+0000 7fd74e200700 1 -- 172.21.15.153:0/2121977275 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd7480737f0 msgr2=0x7fd748198570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:10:19.458 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:19.456+0000 7fd74e200700 1 --2- 172.21.15.153:0/2121977275 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd7480737f0 0x7fd748198570 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fd73800bd70 tx=0x7fd73800be50 comp rx=0 tx=0).stop 2024-03-21T10:10:19.458 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:19.456+0000 7fd74e200700 1 -- 172.21.15.153:0/2121977275 shutdown_connections 2024-03-21T10:10:19.458 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:19.456+0000 7fd74e200700 1 --2- 172.21.15.153:0/2121977275 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7fd72c06e0e0 0x7fd72c0705a0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:10:19.458 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:19.456+0000 7fd74e200700 1 --2- 172.21.15.153:0/2121977275 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd7480737f0 0x7fd748198570 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:10:19.458 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:19.456+0000 7fd74e200700 1 --2- 172.21.15.153:0/2121977275 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd748074dc0 0x7fd748198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:10:19.458 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:19.456+0000 7fd74e200700 1 -- 172.21.15.153:0/2121977275 >> 172.21.15.153:0/2121977275 conn(0x7fd7480fc110 msgr2=0x7fd748106990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:10:19.458 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:19.456+0000 7fd74e200700 1 -- 172.21.15.153:0/2121977275 shutdown_connections 2024-03-21T10:10:19.458 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:19.456+0000 7fd74e200700 1 -- 172.21.15.153:0/2121977275 wait complete. 2024-03-21T10:10:19.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:10:19 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:10:19.692 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:19.690+0000 7f15f2600700 1 -- 172.21.15.153:0/1088786153 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f15ec107e20 msgr2=0x7f15ec10a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:10:19.692 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:19.690+0000 7f15f2600700 1 --2- 172.21.15.153:0/1088786153 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f15ec107e20 0x7f15ec10a210 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f15dc009500 tx=0x7f15dc009810 comp rx=0 tx=0).stop 2024-03-21T10:10:19.693 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:19.691+0000 7f15f2600700 1 -- 172.21.15.153:0/1088786153 shutdown_connections 2024-03-21T10:10:19.693 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:19.691+0000 7f15f2600700 1 --2- 172.21.15.153:0/1088786153 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f15ec10a7e0 0x7f15ec10cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:10:19.693 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:19.691+0000 7f15f2600700 1 --2- 172.21.15.153:0/1088786153 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f15ec107e20 0x7f15ec10a210 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:10:19.693 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:19.691+0000 7f15f2600700 1 -- 172.21.15.153:0/1088786153 >> 172.21.15.153:0/1088786153 conn(0x7f15ec06d9f0 msgr2=0x7f15ec06fe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:10:19.693 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:19.692+0000 7f15f2600700 1 -- 172.21.15.153:0/1088786153 shutdown_connections 2024-03-21T10:10:19.694 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:19.692+0000 7f15f2600700 1 -- 172.21.15.153:0/1088786153 wait complete. 2024-03-21T10:10:19.696 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:19.694+0000 7f15f2600700 1 Processor -- start 2024-03-21T10:10:19.696 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:19.695+0000 7f15f2600700 1 -- start start 2024-03-21T10:10:19.696 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:19.695+0000 7f15f2600700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f15ec107e20 0x7f15ec117bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:10:19.697 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:19.695+0000 7f15f2600700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f15ec10a7e0 0x7f15ec118130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:10:19.697 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:19.695+0000 7f15f2600700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15ec118750 con 0x7f15ec107e20 2024-03-21T10:10:19.698 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:19.695+0000 7f15f2600700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15ec118890 con 0x7f15ec10a7e0 2024-03-21T10:10:19.698 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:19.696+0000 7f15eb400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f15ec107e20 0x7f15ec117bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:10:19.698 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:19.696+0000 7f15eb400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f15ec107e20 0x7f15ec117bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41220/0 (socket says 172.21.15.153:41220) 2024-03-21T10:10:19.698 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:19.696+0000 7f15eb400700 1 -- 172.21.15.153:0/3446659532 learned_addr learned my addr 172.21.15.153:0/3446659532 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:10:19.699 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:19.697+0000 7f15eaa00700 1 --2- 172.21.15.153:0/3446659532 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f15ec10a7e0 0x7f15ec118130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:10:19.699 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:19.697+0000 7f15eb400700 1 -- 172.21.15.153:0/3446659532 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f15ec10a7e0 msgr2=0x7f15ec118130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:10:19.700 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:19.697+0000 7f15eb400700 1 --2- 172.21.15.153:0/3446659532 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f15ec10a7e0 0x7f15ec118130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:10:19.700 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:19.697+0000 7f15eb400700 1 -- 172.21.15.153:0/3446659532 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15dc0091e0 con 0x7f15ec107e20 2024-03-21T10:10:19.700 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:19.697+0000 7f15eb400700 1 --2- 172.21.15.153:0/3446659532 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f15ec107e20 0x7f15ec117bf0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f15dc004bd0 tx=0x7f15dc004cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:10:19.700 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:19.698+0000 7f15e3e00700 1 -- 172.21.15.153:0/3446659532 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f15dc00caf0 con 0x7f15ec107e20 2024-03-21T10:10:19.700 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:19.698+0000 7f15f2600700 1 -- 172.21.15.153:0/3446659532 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15ec11b2d0 con 0x7f15ec107e20 2024-03-21T10:10:19.701 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:19.698+0000 7f15e3e00700 1 -- 172.21.15.153:0/3446659532 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f15dc00cc50 con 0x7f15ec107e20 2024-03-21T10:10:19.701 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:19.698+0000 7f15e3e00700 1 -- 172.21.15.153:0/3446659532 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f15dc016660 con 0x7f15ec107e20 2024-03-21T10:10:19.701 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:19.698+0000 7f15f2600700 1 -- 172.21.15.153:0/3446659532 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15ec11b840 con 0x7f15ec107e20 2024-03-21T10:10:19.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:10:19 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:10:19.703 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:19.700+0000 7f15f2600700 1 -- 172.21.15.153:0/3446659532 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15ec10f5f0 con 0x7f15ec107e20 2024-03-21T10:10:19.705 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:19.704+0000 7f15e3e00700 1 -- 172.21.15.153:0/3446659532 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 31) v1 ==== 92072+0+0 (secure 0 0 0) 0x7f15dc00e450 con 0x7f15ec107e20 2024-03-21T10:10:19.706 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:19.705+0000 7f15e3e00700 1 --2- 172.21.15.153:0/3446659532 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f15d006e130 0x7f15d00705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:10:19.707 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:19.705+0000 7f15e3e00700 1 -- 172.21.15.153:0/3446659532 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f15dc012070 con 0x7f15ec107e20 2024-03-21T10:10:19.707 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:19.706+0000 7f15eaa00700 1 --2- 172.21.15.153:0/3446659532 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f15d006e130 0x7f15d00705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:10:19.708 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:19.707+0000 7f15eaa00700 1 --2- 172.21.15.153:0/3446659532 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f15d006e130 0x7f15d00705f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f15d8009bf0 tx=0x7f15d8009260 comp rx=0 tx=0).ready entity=mgr.14666 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:10:19.715 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:19.713+0000 7f15e3e00700 1 -- 172.21.15.153:0/3446659532 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185454 (secure 0 0 0) 0x7f15dc05cd00 con 0x7f15ec107e20 2024-03-21T10:10:20.044 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:20.042+0000 7f15f2600700 1 -- 172.21.15.153:0/3446659532 --> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f15ec0611d0 con 0x7f15d006e130 2024-03-21T10:10:20.056 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:20.054+0000 7f15e3e00700 1 -- 172.21.15.153:0/3446659532 <== mgr.14666 v2:172.21.15.153:6800/2524145044 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f15ec0611d0 con 0x7f15d006e130 2024-03-21T10:10:20.056 INFO:teuthology.orchestra.run.smithi153.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-21T10:10:20.056 INFO:teuthology.orchestra.run.smithi153.stdout:alertmanager.smithi153 smithi153 *:9093,9094 running (10m) 34s ago 12m 13.4M - 0.25.0 c8568f914cd2 2cfcc225c7e6 2024-03-21T10:10:20.056 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi153 smithi153 running (13m) 34s ago 13m 7972k - 18.2.1 5be31c24972a 5b61994396a6 2024-03-21T10:10:20.057 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi174 smithi174 running (11m) 6s ago 11m 7940k - 18.2.1 5be31c24972a 4f7a5fc3ac28 2024-03-21T10:10:20.057 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi153 smithi153 running (13m) 34s ago 13m 6665k - 18.2.1 5be31c24972a 90a8b8ea03d1 2024-03-21T10:10:20.057 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi174 smithi174 running (11m) 6s ago 11m 6665k - 18.2.1 5be31c24972a a4c929e4dc77 2024-03-21T10:10:20.057 INFO:teuthology.orchestra.run.smithi153.stdout:grafana.smithi153 smithi153 *:3000 running (10m) 34s ago 12m 74.6M - 9.4.7 954c08fa6188 3af5800ecfef 2024-03-21T10:10:20.057 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.iuyluj smithi153 running (5m) 34s ago 5m 172M - 18.2.1 5be31c24972a 02f667cee979 2024-03-21T10:10:20.057 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.zipckr smithi153 running (5m) 34s ago 5m 163M - 18.2.1 5be31c24972a 68a853929f8f 2024-03-21T10:10:20.057 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.nhozag smithi174 running (5m) 6s ago 5m 128M - 18.2.1 5be31c24972a 2fee4000308f 2024-03-21T10:10:20.057 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.yyimst smithi174 running (5m) 6s ago 5m 155M - 18.2.1 5be31c24972a d78a729220d1 2024-03-21T10:10:20.057 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi153.tluxni smithi153 *:8443,9283,8765 running (2m) 34s ago 14m 494M - 19.0.0-2231-g2edfee68 237e43c7a65f 7ffee0e55382 2024-03-21T10:10:20.057 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi174.qpcwat smithi174 *:8443,9283,8765 running (45s) 6s ago 11m 447M - 19.0.0-2231-g2edfee68 237e43c7a65f 748968f216ab 2024-03-21T10:10:20.057 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi153 smithi153 running (14m) 34s ago 15m 54.0M 2048M 18.2.1 5be31c24972a 813963a95d69 2024-03-21T10:10:20.057 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi174 smithi174 running (11m) 6s ago 11m 45.3M 2048M 18.2.1 5be31c24972a 99204923eec6 2024-03-21T10:10:20.057 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi153 smithi153 *:9100 starting - - - - 2024-03-21T10:10:20.057 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi174 smithi174 *:9100 running (11s) 6s ago 11m 6765k - 1.5.0 0da6a335fe13 7c4ecad790fa 2024-03-21T10:10:20.057 INFO:teuthology.orchestra.run.smithi153.stdout:osd.0 smithi153 running (10m) 34s ago 10m 178M 1217M 18.2.1 5be31c24972a 28912a2319b4 2024-03-21T10:10:20.058 INFO:teuthology.orchestra.run.smithi153.stdout:osd.1 smithi153 running (9m) 34s ago 9m 179M 1217M 18.2.1 5be31c24972a ca60681e2314 2024-03-21T10:10:20.058 INFO:teuthology.orchestra.run.smithi153.stdout:osd.2 smithi153 running (9m) 34s ago 9m 141M 1217M 18.2.1 5be31c24972a 8db799f8962a 2024-03-21T10:10:20.058 INFO:teuthology.orchestra.run.smithi153.stdout:osd.3 smithi174 running (8m) 6s ago 8m 213M 2241M 18.2.1 5be31c24972a dafc89e0658d 2024-03-21T10:10:20.058 INFO:teuthology.orchestra.run.smithi153.stdout:osd.4 smithi174 running (8m) 6s ago 8m 201M 2241M 18.2.1 5be31c24972a c4cd15701f05 2024-03-21T10:10:20.058 INFO:teuthology.orchestra.run.smithi153.stdout:osd.5 smithi174 running (7m) 6s ago 7m 187M 2241M 18.2.1 5be31c24972a 3b69886ce22e 2024-03-21T10:10:20.058 INFO:teuthology.orchestra.run.smithi153.stdout:prometheus.smithi153 smithi153 *:9095 running (55s) 34s ago 12m 48.1M - 2.43.0 a07b618ecd1d 9fe52bbde6e1 2024-03-21T10:10:20.059 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:20.057+0000 7f15f2600700 1 -- 172.21.15.153:0/3446659532 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f15d006e130 msgr2=0x7f15d00705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:10:20.059 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:20.057+0000 7f15f2600700 1 --2- 172.21.15.153:0/3446659532 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f15d006e130 0x7f15d00705f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f15d8009bf0 tx=0x7f15d8009260 comp rx=0 tx=0).stop 2024-03-21T10:10:20.059 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:20.057+0000 7f15f2600700 1 -- 172.21.15.153:0/3446659532 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f15ec107e20 msgr2=0x7f15ec117bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:10:20.059 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:20.057+0000 7f15f2600700 1 --2- 172.21.15.153:0/3446659532 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f15ec107e20 0x7f15ec117bf0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f15dc004bd0 tx=0x7f15dc004cb0 comp rx=0 tx=0).stop 2024-03-21T10:10:20.059 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:20.057+0000 7f15f2600700 1 -- 172.21.15.153:0/3446659532 shutdown_connections 2024-03-21T10:10:20.059 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:20.057+0000 7f15f2600700 1 --2- 172.21.15.153:0/3446659532 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f15d006e130 0x7f15d00705f0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:10:20.059 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:20.057+0000 7f15f2600700 1 --2- 172.21.15.153:0/3446659532 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f15ec107e20 0x7f15ec117bf0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:10:20.059 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:20.057+0000 7f15f2600700 1 --2- 172.21.15.153:0/3446659532 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f15ec10a7e0 0x7f15ec118130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:10:20.060 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:20.057+0000 7f15f2600700 1 -- 172.21.15.153:0/3446659532 >> 172.21.15.153:0/3446659532 conn(0x7f15ec06d9f0 msgr2=0x7f15ec06fe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:10:20.060 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:20.058+0000 7f15f2600700 1 -- 172.21.15.153:0/3446659532 shutdown_connections 2024-03-21T10:10:20.060 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:20.058+0000 7f15f2600700 1 -- 172.21.15.153:0/3446659532 wait complete. 2024-03-21T10:10:20.228 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:20.226+0000 7f6336000700 1 -- 172.21.15.153:0/312025875 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6330103e30 msgr2=0x7f6330104290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:10:20.228 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:20.226+0000 7f6336000700 1 --2- 172.21.15.153:0/312025875 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6330103e30 0x7f6330104290 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f631c009910 tx=0x7f631c009c20 comp rx=0 tx=0).stop 2024-03-21T10:10:20.228 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:20.226+0000 7f6336000700 1 -- 172.21.15.153:0/312025875 shutdown_connections 2024-03-21T10:10:20.228 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:20.226+0000 7f6336000700 1 --2- 172.21.15.153:0/312025875 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6330103e30 0x7f6330104290 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:10:20.229 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:20.226+0000 7f6336000700 1 --2- 172.21.15.153:0/312025875 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6330102c30 0x7f6330103050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:10:20.229 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:20.226+0000 7f6336000700 1 -- 172.21.15.153:0/312025875 >> 172.21.15.153:0/312025875 conn(0x7f63300fe1b0 msgr2=0x7f6330100610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:10:20.229 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:20.226+0000 7f6336000700 1 -- 172.21.15.153:0/312025875 shutdown_connections 2024-03-21T10:10:20.229 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:20.226+0000 7f6336000700 1 -- 172.21.15.153:0/312025875 wait complete. 2024-03-21T10:10:20.230 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:20.227+0000 7f6336000700 1 Processor -- start 2024-03-21T10:10:20.230 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:20.228+0000 7f6336000700 1 -- start start 2024-03-21T10:10:20.230 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:20.228+0000 7f6336000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6330102c30 0x7f63301983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:10:20.230 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:20.228+0000 7f6336000700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6330103e30 0x7f6330198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:10:20.230 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:20.228+0000 7f6336000700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6330198f20 con 0x7f6330102c30 2024-03-21T10:10:20.230 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:20.228+0000 7f6336000700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6330199060 con 0x7f6330103e30 2024-03-21T10:10:20.230 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:20.228+0000 7f6334c00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6330102c30 0x7f63301983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:10:20.231 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:20.228+0000 7f6334c00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6330102c30 0x7f63301983c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41238/0 (socket says 172.21.15.153:41238) 2024-03-21T10:10:20.231 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:20.228+0000 7f6334c00700 1 -- 172.21.15.153:0/2650183057 learned_addr learned my addr 172.21.15.153:0/2650183057 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:10:20.231 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:20.228+0000 7f632fe00700 1 --2- 172.21.15.153:0/2650183057 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6330103e30 0x7f6330198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:10:20.231 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:20.229+0000 7f6334c00700 1 -- 172.21.15.153:0/2650183057 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6330103e30 msgr2=0x7f6330198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:10:20.231 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:20.229+0000 7f6334c00700 1 --2- 172.21.15.153:0/2650183057 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6330103e30 0x7f6330198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:10:20.232 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:20.229+0000 7f6334c00700 1 -- 172.21.15.153:0/2650183057 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f631c0095f0 con 0x7f6330102c30 2024-03-21T10:10:20.232 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:20.229+0000 7f6334c00700 1 --2- 172.21.15.153:0/2650183057 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6330102c30 0x7f63301983c0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f632400e8f0 tx=0x7f632400ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:10:20.232 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:20.230+0000 7f632d600700 1 -- 172.21.15.153:0/2650183057 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6324009790 con 0x7f6330102c30 2024-03-21T10:10:20.232 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:20.230+0000 7f6336000700 1 -- 172.21.15.153:0/2650183057 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f633019db10 con 0x7f6330102c30 2024-03-21T10:10:20.232 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:20.230+0000 7f632d600700 1 -- 172.21.15.153:0/2650183057 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f632400c810 con 0x7f6330102c30 2024-03-21T10:10:20.232 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:20.230+0000 7f6336000700 1 -- 172.21.15.153:0/2650183057 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f633019e060 con 0x7f6330102c30 2024-03-21T10:10:20.233 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:20.230+0000 7f632d600700 1 -- 172.21.15.153:0/2650183057 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6324003d20 con 0x7f6330102c30 2024-03-21T10:10:20.233 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:20.231+0000 7f6336000700 1 -- 172.21.15.153:0/2650183057 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6310005320 con 0x7f6330102c30 2024-03-21T10:10:20.234 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:20.232+0000 7f632d600700 1 -- 172.21.15.153:0/2650183057 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 31) v1 ==== 92072+0+0 (secure 0 0 0) 0x7f63240041f0 con 0x7f6330102c30 2024-03-21T10:10:20.235 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:20.233+0000 7f632d600700 1 --2- 172.21.15.153:0/2650183057 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f631806e0e0 0x7f63180705a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:10:20.235 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:20.234+0000 7f632d600700 1 -- 172.21.15.153:0/2650183057 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f6324014070 con 0x7f6330102c30 2024-03-21T10:10:20.236 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:20.235+0000 7f632fe00700 1 --2- 172.21.15.153:0/2650183057 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f631806e0e0 0x7f63180705a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:10:20.237 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:20.236+0000 7f632fe00700 1 --2- 172.21.15.153:0/2650183057 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f631806e0e0 0x7f63180705a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f631c009fd0 tx=0x7f631c005fd0 comp rx=0 tx=0).ready entity=mgr.14666 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:10:20.240 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:20.239+0000 7f632d600700 1 -- 172.21.15.153:0/2650183057 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185454 (secure 0 0 0) 0x7f632405b2e0 con 0x7f6330102c30 2024-03-21T10:10:20.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:10:20 smithi153 ceph-mon[29117]: from='client.24503 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:10:20.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:10:20 smithi153 ceph-mon[29117]: pgmap v56: 65 pgs: 65 active+clean; 318 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 115 op/s 2024-03-21T10:10:20.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:10:20 smithi153 ceph-mon[29117]: from='client.14756 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:10:20.597 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:20.595+0000 7f6336000700 1 -- 172.21.15.153:0/2650183057 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f6310006200 con 0x7f6330102c30 2024-03-21T10:10:20.600 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:10:20.601 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": { 2024-03-21T10:10:20.601 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-03-21T10:10:20.601 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:10:20.601 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": { 2024-03-21T10:10:20.601 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:10:20.601 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:10:20.601 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": { 2024-03-21T10:10:20.601 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-03-21T10:10:20.601 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:10:20.601 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": { 2024-03-21T10:10:20.602 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-03-21T10:10:20.602 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:10:20.602 INFO:teuthology.orchestra.run.smithi153.stdout: "overall": { 2024-03-21T10:10:20.602 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 12, 2024-03-21T10:10:20.602 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:10:20.602 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-03-21T10:10:20.602 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:10:20.602 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:20.597+0000 7f632d600700 1 -- 172.21.15.153:0/2650183057 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7f632405ae70 con 0x7f6330102c30 2024-03-21T10:10:20.603 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:20.601+0000 7f6336000700 1 -- 172.21.15.153:0/2650183057 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f631806e0e0 msgr2=0x7f63180705a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:10:20.603 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:20.601+0000 7f6336000700 1 --2- 172.21.15.153:0/2650183057 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f631806e0e0 0x7f63180705a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f631c009fd0 tx=0x7f631c005fd0 comp rx=0 tx=0).stop 2024-03-21T10:10:20.603 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:20.601+0000 7f6336000700 1 -- 172.21.15.153:0/2650183057 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6330102c30 msgr2=0x7f63301983c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:10:20.603 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:20.601+0000 7f6336000700 1 --2- 172.21.15.153:0/2650183057 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6330102c30 0x7f63301983c0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f632400e8f0 tx=0x7f632400ec00 comp rx=0 tx=0).stop 2024-03-21T10:10:20.603 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:20.602+0000 7f6336000700 1 -- 172.21.15.153:0/2650183057 shutdown_connections 2024-03-21T10:10:20.603 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:20.602+0000 7f6336000700 1 --2- 172.21.15.153:0/2650183057 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f631806e0e0 0x7f63180705a0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:10:20.604 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:20.602+0000 7f6336000700 1 --2- 172.21.15.153:0/2650183057 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6330102c30 0x7f63301983c0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:10:20.604 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:20.602+0000 7f6336000700 1 --2- 172.21.15.153:0/2650183057 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6330103e30 0x7f6330198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:10:20.604 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:20.602+0000 7f6336000700 1 -- 172.21.15.153:0/2650183057 >> 172.21.15.153:0/2650183057 conn(0x7f63300fe1b0 msgr2=0x7f6330107060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:10:20.604 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:20.602+0000 7f6336000700 1 -- 172.21.15.153:0/2650183057 shutdown_connections 2024-03-21T10:10:20.604 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:20.602+0000 7f6336000700 1 -- 172.21.15.153:0/2650183057 wait complete. 2024-03-21T10:10:20.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:10:20 smithi174 ceph-mon[37496]: from='client.24503 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:10:20.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:10:20 smithi174 ceph-mon[37496]: pgmap v56: 65 pgs: 65 active+clean; 318 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 115 op/s 2024-03-21T10:10:20.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:10:20 smithi174 ceph-mon[37496]: from='client.14756 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:10:20.813 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:20.811+0000 7f25e4e00700 1 -- 172.21.15.153:0/4166309661 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f25e0103f70 msgr2=0x7f25e0106360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:10:20.814 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:20.811+0000 7f25e4e00700 1 --2- 172.21.15.153:0/4166309661 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f25e0103f70 0x7f25e0106360 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f25cc009910 tx=0x7f25cc009c20 comp rx=0 tx=0).stop 2024-03-21T10:10:20.814 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:20.812+0000 7f25e4e00700 1 -- 172.21.15.153:0/4166309661 shutdown_connections 2024-03-21T10:10:20.814 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:20.812+0000 7f25e4e00700 1 --2- 172.21.15.153:0/4166309661 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f25e0103f70 0x7f25e0106360 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:10:20.814 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:20.812+0000 7f25e4e00700 1 --2- 172.21.15.153:0/4166309661 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f25e0101640 0x7f25e0103a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:10:20.814 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:20.812+0000 7f25e4e00700 1 -- 172.21.15.153:0/4166309661 >> 172.21.15.153:0/4166309661 conn(0x7f25e00fb030 msgr2=0x7f25e00fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:10:20.814 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:20.812+0000 7f25e4e00700 1 -- 172.21.15.153:0/4166309661 shutdown_connections 2024-03-21T10:10:20.814 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:20.812+0000 7f25e4e00700 1 -- 172.21.15.153:0/4166309661 wait complete. 2024-03-21T10:10:20.815 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:20.813+0000 7f25e4e00700 1 Processor -- start 2024-03-21T10:10:20.815 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:20.813+0000 7f25e4e00700 1 -- start start 2024-03-21T10:10:20.815 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:20.814+0000 7f25e4e00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f25e0101640 0x7f25e0196330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:10:20.815 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:20.814+0000 7f25e4e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f25e0103f70 0x7f25e0196870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:10:20.815 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:20.814+0000 7f25e4e00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25e0196e90 con 0x7f25e0103f70 2024-03-21T10:10:20.815 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:20.814+0000 7f25e4e00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25e0196fd0 con 0x7f25e0101640 2024-03-21T10:10:20.816 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:20.814+0000 7f25dd400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f25e0103f70 0x7f25e0196870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:10:20.816 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:20.814+0000 7f25dd400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f25e0103f70 0x7f25e0196870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41252/0 (socket says 172.21.15.153:41252) 2024-03-21T10:10:20.817 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:20.814+0000 7f25dd400700 1 -- 172.21.15.153:0/923643087 learned_addr learned my addr 172.21.15.153:0/923643087 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:10:20.817 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:20.814+0000 7f25dde00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f25e0101640 0x7f25e0196330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:10:20.817 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:20.815+0000 7f25dd400700 1 -- 172.21.15.153:0/923643087 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f25e0101640 msgr2=0x7f25e0196330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:10:20.817 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:20.815+0000 7f25dd400700 1 --2- 172.21.15.153:0/923643087 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f25e0101640 0x7f25e0196330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:10:20.817 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:20.815+0000 7f25dd400700 1 -- 172.21.15.153:0/923643087 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25cc0095f0 con 0x7f25e0103f70 2024-03-21T10:10:20.817 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:20.815+0000 7f25dde00700 1 --2- 172.21.15.153:0/923643087 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f25e0101640 0x7f25e0196330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:10:20.817 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:20.815+0000 7f25dd400700 1 --2- 172.21.15.153:0/923643087 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f25e0103f70 0x7f25e0196870 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f25cc0092e0 tx=0x7f25cc00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:10:20.818 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:20.815+0000 7f25d6a00700 1 -- 172.21.15.153:0/923643087 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f25cc01d070 con 0x7f25e0103f70 2024-03-21T10:10:20.818 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:20.815+0000 7f25d6a00700 1 -- 172.21.15.153:0/923643087 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f25cc005dc0 con 0x7f25e0103f70 2024-03-21T10:10:20.818 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:20.815+0000 7f25d6a00700 1 -- 172.21.15.153:0/923643087 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f25cc017630 con 0x7f25e0103f70 2024-03-21T10:10:20.818 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:20.815+0000 7f25e4e00700 1 -- 172.21.15.153:0/923643087 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25e00ff430 con 0x7f25e0103f70 2024-03-21T10:10:20.818 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:20.816+0000 7f25e4e00700 1 -- 172.21.15.153:0/923643087 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25e00ff920 con 0x7f25e0103f70 2024-03-21T10:10:20.819 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:20.818+0000 7f25e4e00700 1 -- 172.21.15.153:0/923643087 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25e0190520 con 0x7f25e0103f70 2024-03-21T10:10:20.820 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:20.819+0000 7f25d6a00700 1 -- 172.21.15.153:0/923643087 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 31) v1 ==== 92072+0+0 (secure 0 0 0) 0x7f25cc003680 con 0x7f25e0103f70 2024-03-21T10:10:20.821 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:20.820+0000 7f25d6a00700 1 --2- 172.21.15.153:0/923643087 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f25c406e0e0 0x7f25c40705a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:10:20.821 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:20.820+0000 7f25d6a00700 1 -- 172.21.15.153:0/923643087 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f25cc08f1e0 con 0x7f25e0103f70 2024-03-21T10:10:20.822 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:20.820+0000 7f25dde00700 1 --2- 172.21.15.153:0/923643087 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f25c406e0e0 0x7f25c40705a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:10:20.822 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:20.821+0000 7f25dde00700 1 --2- 172.21.15.153:0/923643087 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f25c406e0e0 0x7f25c40705a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f25d0011490 tx=0x7f25d0009260 comp rx=0 tx=0).ready entity=mgr.14666 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:10:20.828 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:20.827+0000 7f25d6a00700 1 -- 172.21.15.153:0/923643087 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185454 (secure 0 0 0) 0x7f25cc05c470 con 0x7f25e0103f70 2024-03-21T10:10:21.142 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:21.140+0000 7f25e4e00700 1 -- 172.21.15.153:0/923643087 --> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f25e0100ec0 con 0x7f25c406e0e0 2024-03-21T10:10:21.144 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:21.142+0000 7f25d6a00700 1 -- 172.21.15.153:0/923643087 <== mgr.14666 v2:172.21.15.153:6800/2524145044 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+370 (secure 0 0 0) 0x7f25e0100ec0 con 0x7f25c406e0e0 2024-03-21T10:10:21.144 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:10:21.144 INFO:teuthology.orchestra.run.smithi153.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232", 2024-03-21T10:10:21.144 INFO:teuthology.orchestra.run.smithi153.stdout: "in_progress": true, 2024-03-21T10:10:21.144 INFO:teuthology.orchestra.run.smithi153.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-03-21T10:10:21.144 INFO:teuthology.orchestra.run.smithi153.stdout: "services_complete": [ 2024-03-21T10:10:21.144 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr" 2024-03-21T10:10:21.145 INFO:teuthology.orchestra.run.smithi153.stdout: ], 2024-03-21T10:10:21.145 INFO:teuthology.orchestra.run.smithi153.stdout: "progress": "2/2 daemons upgraded", 2024-03-21T10:10:21.145 INFO:teuthology.orchestra.run.smithi153.stdout: "message": "Currently upgrading node-exporter daemons", 2024-03-21T10:10:21.145 INFO:teuthology.orchestra.run.smithi153.stdout: "is_paused": false 2024-03-21T10:10:21.145 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:10:21.148 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:21.147+0000 7f25e4e00700 1 -- 172.21.15.153:0/923643087 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f25c406e0e0 msgr2=0x7f25c40705a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:10:21.148 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:21.147+0000 7f25e4e00700 1 --2- 172.21.15.153:0/923643087 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f25c406e0e0 0x7f25c40705a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f25d0011490 tx=0x7f25d0009260 comp rx=0 tx=0).stop 2024-03-21T10:10:21.149 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:21.147+0000 7f25e4e00700 1 -- 172.21.15.153:0/923643087 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f25e0103f70 msgr2=0x7f25e0196870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:10:21.149 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:21.147+0000 7f25e4e00700 1 --2- 172.21.15.153:0/923643087 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f25e0103f70 0x7f25e0196870 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f25cc0092e0 tx=0x7f25cc00f690 comp rx=0 tx=0).stop 2024-03-21T10:10:21.149 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:21.147+0000 7f25e4e00700 1 -- 172.21.15.153:0/923643087 shutdown_connections 2024-03-21T10:10:21.149 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:21.147+0000 7f25e4e00700 1 --2- 172.21.15.153:0/923643087 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f25c406e0e0 0x7f25c40705a0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:10:21.149 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:21.147+0000 7f25e4e00700 1 --2- 172.21.15.153:0/923643087 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f25e0101640 0x7f25e0196330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:10:21.149 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:21.147+0000 7f25e4e00700 1 --2- 172.21.15.153:0/923643087 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f25e0103f70 0x7f25e0196870 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:10:21.149 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:21.147+0000 7f25e4e00700 1 -- 172.21.15.153:0/923643087 >> 172.21.15.153:0/923643087 conn(0x7f25e00fb030 msgr2=0x7f25e00fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:10:21.149 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:21.147+0000 7f25e4e00700 1 -- 172.21.15.153:0/923643087 shutdown_connections 2024-03-21T10:10:21.149 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:21.147+0000 7f25e4e00700 1 -- 172.21.15.153:0/923643087 wait complete. 2024-03-21T10:10:21.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:10:21 smithi153 ceph-mon[29117]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:10:21.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:10:21 smithi153 ceph-mon[29117]: from='client.? 172.21.15.153:0/2650183057' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:10:21.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:10:21 smithi174 ceph-mon[37496]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:10:21.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:10:21 smithi174 ceph-mon[37496]: from='client.? 172.21.15.153:0/2650183057' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:10:22.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:10:22 smithi153 ceph-mon[29117]: from='client.14768 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:10:22.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:10:22 smithi153 ceph-mon[29117]: pgmap v57: 65 pgs: 65 active+clean; 322 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 166 op/s 2024-03-21T10:10:22.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:10:22 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:10:22.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:10:22 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:10:22.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:10:22 smithi174 ceph-mon[37496]: from='client.14768 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:10:22.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:10:22 smithi174 ceph-mon[37496]: pgmap v57: 65 pgs: 65 active+clean; 322 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 166 op/s 2024-03-21T10:10:22.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:10:22 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:10:22.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:10:22 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:10:24.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:10:24 smithi153 ceph-mon[29117]: pgmap v58: 65 pgs: 65 active+clean; 322 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 116 op/s 2024-03-21T10:10:24.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:10:24 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:10:24.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:10:24 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:10:24.597 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:10:24 smithi174 ceph-mon[37496]: pgmap v58: 65 pgs: 65 active+clean; 322 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 116 op/s 2024-03-21T10:10:24.597 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:10:24 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:10:24.597 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:10:24 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:10:26.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:10:26 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:10:26.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:10:26 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:10:26.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:10:26 smithi153 ceph-mon[29117]: pgmap v59: 65 pgs: 65 active+clean; 325 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 177 op/s 2024-03-21T10:10:26.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:10:26 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:10:26.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:10:26 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:10:26.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:10:26 smithi174 ceph-mon[37496]: pgmap v59: 65 pgs: 65 active+clean; 325 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 177 op/s 2024-03-21T10:10:28.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:10:28 smithi153 ceph-mon[29117]: pgmap v60: 65 pgs: 65 active+clean; 325 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 112 op/s 2024-03-21T10:10:28.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:10:28 smithi174 ceph-mon[37496]: pgmap v60: 65 pgs: 65 active+clean; 325 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 112 op/s 2024-03-21T10:10:30.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:10:30 smithi174 ceph-mon[37496]: pgmap v61: 65 pgs: 65 active+clean; 325 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 112 op/s 2024-03-21T10:10:30.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:10:30 smithi153 ceph-mon[29117]: pgmap v61: 65 pgs: 65 active+clean; 325 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 112 op/s 2024-03-21T10:10:32.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:10:32 smithi153 ceph-mon[29117]: pgmap v62: 65 pgs: 65 active+clean; 329 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 179 op/s 2024-03-21T10:10:32.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:10:32 smithi174 ceph-mon[37496]: pgmap v62: 65 pgs: 65 active+clean; 329 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 179 op/s 2024-03-21T10:10:33.649 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:10:33 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:10:33.649 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:10:33 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:10:33.649 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:10:33 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:10:33.649 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:10:33 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:10:33.649 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:10:33 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:10:33.649 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:10:33 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:10:33.649 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:10:33 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:10:33.649 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:10:33 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:10:33.649 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:10:33 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:10:33.650 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:10:33 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:10:33.650 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:10:33 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:10:33.650 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:10:33 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:10:33.650 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:10:33 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:10:33.650 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:10:33 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:10:33.650 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:10:33 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:10:33.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:10:33 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:10:33.703 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:10:33 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:10:33.703 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:10:33 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:10:33.703 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:10:33 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:10:33.703 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:10:33 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:10:33.703 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:10:33 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:10:33.703 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:10:33 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:10:33.703 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:10:33 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:10:33.703 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:10:33 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:10:33.703 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:10:33 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:10:33.703 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:10:33 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:10:33.703 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:10:33 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:10:33.703 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:10:33 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:10:33.704 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:10:33 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:10:33.704 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:10:33 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:10:34.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:10:34 smithi174 ceph-mon[37496]: pgmap v63: 65 pgs: 65 active+clean; 329 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 127 op/s 2024-03-21T10:10:34.710 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:10:34 smithi153 ceph-mon[29117]: pgmap v63: 65 pgs: 65 active+clean; 329 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 127 op/s 2024-03-21T10:10:36.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:10:36 smithi153 ceph-mon[29117]: Upgrade: Updating prometheus.smithi153 2024-03-21T10:10:36.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:10:36 smithi153 ceph-mon[29117]: pgmap v64: 65 pgs: 65 active+clean; 325 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 214 op/s 2024-03-21T10:10:36.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:10:36 smithi153 ceph-mon[29117]: Deploying daemon prometheus.smithi153 on smithi153 2024-03-21T10:10:36.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:10:36 smithi174 ceph-mon[37496]: Upgrade: Updating prometheus.smithi153 2024-03-21T10:10:36.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:10:36 smithi174 ceph-mon[37496]: pgmap v64: 65 pgs: 65 active+clean; 325 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 214 op/s 2024-03-21T10:10:36.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:10:36 smithi174 ceph-mon[37496]: Deploying daemon prometheus.smithi153 on smithi153 2024-03-21T10:10:38.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:10:38 smithi153 ceph-mon[29117]: pgmap v65: 65 pgs: 65 active+clean; 325 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 153 op/s 2024-03-21T10:10:38.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:10:38 smithi174 ceph-mon[37496]: pgmap v65: 65 pgs: 65 active+clean; 325 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 153 op/s 2024-03-21T10:10:40.529 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:10:40 smithi153 ceph-mon[29117]: pgmap v66: 65 pgs: 65 active+clean; 325 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 153 op/s 2024-03-21T10:10:40.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:10:40 smithi174 ceph-mon[37496]: pgmap v66: 65 pgs: 65 active+clean; 325 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 153 op/s 2024-03-21T10:10:42.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:10:42 smithi174 ceph-mon[37496]: pgmap v67: 65 pgs: 65 active+clean; 324 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 240 op/s 2024-03-21T10:10:42.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:10:42 smithi153 ceph-mon[29117]: pgmap v67: 65 pgs: 65 active+clean; 324 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 240 op/s 2024-03-21T10:10:44.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:10:44 smithi174 ceph-mon[37496]: pgmap v68: 65 pgs: 65 active+clean; 324 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 910 KiB/s rd, 984 KiB/s wr, 173 op/s 2024-03-21T10:10:44.708 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:10:44 smithi153 ceph-mon[29117]: pgmap v68: 65 pgs: 65 active+clean; 324 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 910 KiB/s rd, 984 KiB/s wr, 173 op/s 2024-03-21T10:10:45.268 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.0... 2024-03-21T10:10:45.269 DEBUG:teuthology.orchestra.run.smithi153:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-03-21T10:10:46.438 DEBUG:teuthology.parallel:result is None 2024-03-21T10:10:46.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:10:46 smithi153 ceph-mon[29117]: pgmap v69: 65 pgs: 65 active+clean; 324 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 258 op/s 2024-03-21T10:10:46.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:10:46 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:10:46.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:10:46 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:10:46.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:10:46 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:10:46.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:10:46 smithi174 ceph-mon[37496]: pgmap v69: 65 pgs: 65 active+clean; 324 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 258 op/s 2024-03-21T10:10:46.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:10:46 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:10:46.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:10:46 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:10:46.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:10:46 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:10:48.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:10:48 smithi153 ceph-mon[29117]: pgmap v70: 65 pgs: 65 active+clean; 324 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 817 KiB/s rd, 837 KiB/s wr, 171 op/s 2024-03-21T10:10:48.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:10:48 smithi174 ceph-mon[37496]: pgmap v70: 65 pgs: 65 active+clean; 324 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 817 KiB/s rd, 837 KiB/s wr, 171 op/s 2024-03-21T10:10:49.441 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:10:49 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:10:49.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:10:49 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:10:50.362 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:10:50 smithi153 ceph-mon[29117]: pgmap v71: 65 pgs: 65 active+clean; 324 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 817 KiB/s rd, 837 KiB/s wr, 171 op/s 2024-03-21T10:10:50.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:10:50 smithi174 ceph-mon[37496]: pgmap v71: 65 pgs: 65 active+clean; 324 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 817 KiB/s rd, 837 KiB/s wr, 171 op/s 2024-03-21T10:10:51.322 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:51.320+0000 7f23d9800700 1 -- 172.21.15.153:0/3527143102 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f23d4100bb0 msgr2=0x7f23d4100fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:10:51.323 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:51.320+0000 7f23d9800700 1 --2- 172.21.15.153:0/3527143102 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f23d4100bb0 0x7f23d4100fd0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f23c4009910 tx=0x7f23c4009c20 comp rx=0 tx=0).stop 2024-03-21T10:10:51.323 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:51.320+0000 7f23d9800700 1 -- 172.21.15.153:0/3527143102 shutdown_connections 2024-03-21T10:10:51.323 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:51.320+0000 7f23d9800700 1 --2- 172.21.15.153:0/3527143102 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f23d4101db0 0x7f23d4102210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:10:51.323 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:51.320+0000 7f23d9800700 1 --2- 172.21.15.153:0/3527143102 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f23d4100bb0 0x7f23d4100fd0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:10:51.323 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:51.320+0000 7f23d9800700 1 -- 172.21.15.153:0/3527143102 >> 172.21.15.153:0/3527143102 conn(0x7f23d40fc110 msgr2=0x7f23d40fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:10:51.323 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:51.321+0000 7f23d9800700 1 -- 172.21.15.153:0/3527143102 shutdown_connections 2024-03-21T10:10:51.323 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:51.321+0000 7f23d9800700 1 -- 172.21.15.153:0/3527143102 wait complete. 2024-03-21T10:10:51.324 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:51.322+0000 7f23d9800700 1 Processor -- start 2024-03-21T10:10:51.324 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:51.322+0000 7f23d9800700 1 -- start start 2024-03-21T10:10:51.324 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:51.322+0000 7f23d9800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f23d4100bb0 0x7f23d4198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:10:51.324 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:51.322+0000 7f23d9800700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f23d4101db0 0x7f23d4198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:10:51.324 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:51.322+0000 7f23d9800700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23d41990f0 con 0x7f23d4100bb0 2024-03-21T10:10:51.324 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:51.322+0000 7f23d9800700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23d4199230 con 0x7f23d4101db0 2024-03-21T10:10:51.325 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:51.323+0000 7f23d2a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f23d4100bb0 0x7f23d4198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:10:51.325 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:51.323+0000 7f23d2a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f23d4100bb0 0x7f23d4198590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:32856/0 (socket says 172.21.15.153:32856) 2024-03-21T10:10:51.325 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:51.323+0000 7f23d2a00700 1 -- 172.21.15.153:0/1606066482 learned_addr learned my addr 172.21.15.153:0/1606066482 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:10:51.325 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:51.323+0000 7f23d2000700 1 --2- 172.21.15.153:0/1606066482 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f23d4101db0 0x7f23d4198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:10:51.325 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:51.323+0000 7f23d2a00700 1 -- 172.21.15.153:0/1606066482 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f23d4101db0 msgr2=0x7f23d4198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:10:51.325 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:51.323+0000 7f23d2a00700 1 --2- 172.21.15.153:0/1606066482 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f23d4101db0 0x7f23d4198ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:10:51.325 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:51.323+0000 7f23d2a00700 1 -- 172.21.15.153:0/1606066482 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23c40095f0 con 0x7f23d4100bb0 2024-03-21T10:10:51.325 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:51.323+0000 7f23d2a00700 1 --2- 172.21.15.153:0/1606066482 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f23d4100bb0 0x7f23d4198590 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f23c400bd70 tx=0x7f23c400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:10:51.326 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:51.324+0000 7f23cb400700 1 -- 172.21.15.153:0/1606066482 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f23c401d070 con 0x7f23d4100bb0 2024-03-21T10:10:51.326 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:51.324+0000 7f23cb400700 1 -- 172.21.15.153:0/1606066482 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f23c400f460 con 0x7f23d4100bb0 2024-03-21T10:10:51.326 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:51.324+0000 7f23d9800700 1 -- 172.21.15.153:0/1606066482 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23d419dc80 con 0x7f23d4100bb0 2024-03-21T10:10:51.326 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:51.324+0000 7f23cb400700 1 -- 172.21.15.153:0/1606066482 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f23c4021640 con 0x7f23d4100bb0 2024-03-21T10:10:51.326 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:51.324+0000 7f23d9800700 1 -- 172.21.15.153:0/1606066482 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23d419e170 con 0x7f23d4100bb0 2024-03-21T10:10:51.327 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:51.325+0000 7f23d9800700 1 -- 172.21.15.153:0/1606066482 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23d4105e90 con 0x7f23d4100bb0 2024-03-21T10:10:51.327 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:51.326+0000 7f23cb400700 1 -- 172.21.15.153:0/1606066482 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 31) v1 ==== 92072+0+0 (secure 0 0 0) 0x7f23c400fab0 con 0x7f23d4100bb0 2024-03-21T10:10:51.328 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:51.327+0000 7f23cb400700 1 --2- 172.21.15.153:0/1606066482 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f23b806e0e0 0x7f23b80705a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:10:51.330 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:51.327+0000 7f23cb400700 1 -- 172.21.15.153:0/1606066482 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f23c408f2c0 con 0x7f23d4100bb0 2024-03-21T10:10:51.331 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:51.329+0000 7f23d2000700 1 --2- 172.21.15.153:0/1606066482 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f23b806e0e0 0x7f23b80705a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:10:51.331 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:51.330+0000 7f23d2000700 1 --2- 172.21.15.153:0/1606066482 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f23b806e0e0 0x7f23b80705a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f23c0005950 tx=0x7f23c0005cf0 comp rx=0 tx=0).ready entity=mgr.14666 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:10:51.336 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:51.335+0000 7f23cb400700 1 -- 172.21.15.153:0/1606066482 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185454 (secure 0 0 0) 0x7f23c405c4d0 con 0x7f23d4100bb0 2024-03-21T10:10:51.598 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:51.596+0000 7f23d9800700 1 -- 172.21.15.153:0/1606066482 --> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f23d40611d0 con 0x7f23b806e0e0 2024-03-21T10:10:51.599 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:51.598+0000 7f23cb400700 1 -- 172.21.15.153:0/1606066482 <== mgr.14666 v2:172.21.15.153:6800/2524145044 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+367 (secure 0 0 0) 0x7f23d40611d0 con 0x7f23b806e0e0 2024-03-21T10:10:51.604 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:51.603+0000 7f23d9800700 1 -- 172.21.15.153:0/1606066482 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f23b806e0e0 msgr2=0x7f23b80705a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:10:51.604 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:51.603+0000 7f23d9800700 1 --2- 172.21.15.153:0/1606066482 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f23b806e0e0 0x7f23b80705a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f23c0005950 tx=0x7f23c0005cf0 comp rx=0 tx=0).stop 2024-03-21T10:10:51.604 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:51.603+0000 7f23d9800700 1 -- 172.21.15.153:0/1606066482 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f23d4100bb0 msgr2=0x7f23d4198590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:10:51.604 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:51.603+0000 7f23d9800700 1 --2- 172.21.15.153:0/1606066482 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f23d4100bb0 0x7f23d4198590 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f23c400bd70 tx=0x7f23c400be50 comp rx=0 tx=0).stop 2024-03-21T10:10:51.605 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:51.603+0000 7f23d9800700 1 -- 172.21.15.153:0/1606066482 shutdown_connections 2024-03-21T10:10:51.605 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:51.603+0000 7f23d9800700 1 --2- 172.21.15.153:0/1606066482 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f23b806e0e0 0x7f23b80705a0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:10:51.605 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:51.603+0000 7f23d9800700 1 --2- 172.21.15.153:0/1606066482 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f23d4100bb0 0x7f23d4198590 secure :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f23c400bd70 tx=0x7f23c400be50 comp rx=0 tx=0).stop 2024-03-21T10:10:51.605 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:51.603+0000 7f23d9800700 1 --2- 172.21.15.153:0/1606066482 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f23d4101db0 0x7f23d4198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:10:51.605 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:51.603+0000 7f23d9800700 1 -- 172.21.15.153:0/1606066482 >> 172.21.15.153:0/1606066482 conn(0x7f23d40fc110 msgr2=0x7f23d4102fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:10:51.605 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:51.604+0000 7f23d9800700 1 -- 172.21.15.153:0/1606066482 shutdown_connections 2024-03-21T10:10:51.606 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:51.604+0000 7f23d9800700 1 -- 172.21.15.153:0/1606066482 wait complete. 2024-03-21T10:10:51.631 INFO:teuthology.orchestra.run.smithi153.stdout:true 2024-03-21T10:10:51.783 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:51.781+0000 7f7c53400700 1 -- 172.21.15.153:0/1837036166 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7c4c101db0 msgr2=0x7f7c4c102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:10:51.784 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:51.781+0000 7f7c53400700 1 --2- 172.21.15.153:0/1837036166 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7c4c101db0 0x7f7c4c102210 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f7c38009910 tx=0x7f7c38009c20 comp rx=0 tx=0).stop 2024-03-21T10:10:51.784 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:51.782+0000 7f7c53400700 1 -- 172.21.15.153:0/1837036166 shutdown_connections 2024-03-21T10:10:51.784 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:51.782+0000 7f7c53400700 1 --2- 172.21.15.153:0/1837036166 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7c4c101db0 0x7f7c4c102210 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:10:51.784 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:51.782+0000 7f7c53400700 1 --2- 172.21.15.153:0/1837036166 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7c4c100bb0 0x7f7c4c100fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:10:51.784 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:51.782+0000 7f7c53400700 1 -- 172.21.15.153:0/1837036166 >> 172.21.15.153:0/1837036166 conn(0x7f7c4c0fc110 msgr2=0x7f7c4c0fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:10:51.784 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:51.782+0000 7f7c53400700 1 -- 172.21.15.153:0/1837036166 shutdown_connections 2024-03-21T10:10:51.784 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:51.782+0000 7f7c53400700 1 -- 172.21.15.153:0/1837036166 wait complete. 2024-03-21T10:10:51.785 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:51.783+0000 7f7c53400700 1 Processor -- start 2024-03-21T10:10:51.785 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:51.783+0000 7f7c53400700 1 -- start start 2024-03-21T10:10:51.785 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:51.784+0000 7f7c53400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7c4c100bb0 0x7f7c4c194160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:10:51.785 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:51.784+0000 7f7c53400700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7c4c101db0 0x7f7c4c1946a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:10:51.785 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:51.784+0000 7f7c53400700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c4c194cc0 con 0x7f7c4c100bb0 2024-03-21T10:10:51.785 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:51.784+0000 7f7c53400700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c4c194e00 con 0x7f7c4c101db0 2024-03-21T10:10:51.786 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:51.784+0000 7f7c50a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7c4c100bb0 0x7f7c4c194160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:10:51.786 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:51.784+0000 7f7c50a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7c4c100bb0 0x7f7c4c194160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:32870/0 (socket says 172.21.15.153:32870) 2024-03-21T10:10:51.786 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:51.784+0000 7f7c50a00700 1 -- 172.21.15.153:0/3128223215 learned_addr learned my addr 172.21.15.153:0/3128223215 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:10:51.787 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:51.785+0000 7f7c50a00700 1 -- 172.21.15.153:0/3128223215 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7c4c101db0 msgr2=0x7f7c4c1946a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:10:51.787 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:51.785+0000 7f7c50a00700 1 --2- 172.21.15.153:0/3128223215 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7c4c101db0 0x7f7c4c1946a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:10:51.787 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:51.785+0000 7f7c50a00700 1 -- 172.21.15.153:0/3128223215 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c380095f0 con 0x7f7c4c100bb0 2024-03-21T10:10:51.787 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:51.785+0000 7f7c50a00700 1 --2- 172.21.15.153:0/3128223215 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7c4c100bb0 0x7f7c4c194160 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f7c4000d900 tx=0x7f7c4000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:10:51.787 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:51.785+0000 7f7c49600700 1 -- 172.21.15.153:0/3128223215 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c400041f0 con 0x7f7c4c100bb0 2024-03-21T10:10:51.787 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:51.786+0000 7f7c49600700 1 -- 172.21.15.153:0/3128223215 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f7c40004350 con 0x7f7c4c100bb0 2024-03-21T10:10:51.788 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:51.786+0000 7f7c49600700 1 -- 172.21.15.153:0/3128223215 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c40003d70 con 0x7f7c4c100bb0 2024-03-21T10:10:51.788 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:51.786+0000 7f7c53400700 1 -- 172.21.15.153:0/3128223215 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c4c1998b0 con 0x7f7c4c100bb0 2024-03-21T10:10:51.788 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:51.786+0000 7f7c53400700 1 -- 172.21.15.153:0/3128223215 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c4c199e00 con 0x7f7c4c100bb0 2024-03-21T10:10:51.789 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:51.787+0000 7f7c53400700 1 -- 172.21.15.153:0/3128223215 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c4c107510 con 0x7f7c4c100bb0 2024-03-21T10:10:51.789 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:51.788+0000 7f7c49600700 1 -- 172.21.15.153:0/3128223215 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 31) v1 ==== 92072+0+0 (secure 0 0 0) 0x7f7c40009730 con 0x7f7c4c100bb0 2024-03-21T10:10:51.791 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:51.789+0000 7f7c49600700 1 --2- 172.21.15.153:0/3128223215 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f7c340724f0 0x7f7c340749b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:10:51.791 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:51.789+0000 7f7c49600700 1 -- 172.21.15.153:0/3128223215 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f7c4008ddd0 con 0x7f7c4c100bb0 2024-03-21T10:10:51.791 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:51.790+0000 7f7c4be00700 1 --2- 172.21.15.153:0/3128223215 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f7c340724f0 0x7f7c340749b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:10:51.792 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:51.790+0000 7f7c4be00700 1 --2- 172.21.15.153:0/3128223215 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f7c340724f0 0x7f7c340749b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7c3800b5c0 tx=0x7f7c38005fd0 comp rx=0 tx=0).ready entity=mgr.14666 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:10:51.795 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:51.794+0000 7f7c49600700 1 -- 172.21.15.153:0/3128223215 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185454 (secure 0 0 0) 0x7f7c400c2480 con 0x7f7c4c100bb0 2024-03-21T10:10:52.096 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:52.094+0000 7f7c53400700 1 -- 172.21.15.153:0/3128223215 --> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7c4c106700 con 0x7f7c340724f0 2024-03-21T10:10:52.097 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:52.096+0000 7f7c49600700 1 -- 172.21.15.153:0/3128223215 <== mgr.14666 v2:172.21.15.153:6800/2524145044 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+367 (secure 0 0 0) 0x7f7c4c106700 con 0x7f7c340724f0 2024-03-21T10:10:52.100 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:52.098+0000 7f7c53400700 1 -- 172.21.15.153:0/3128223215 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f7c340724f0 msgr2=0x7f7c340749b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:10:52.100 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:52.098+0000 7f7c53400700 1 --2- 172.21.15.153:0/3128223215 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f7c340724f0 0x7f7c340749b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7c3800b5c0 tx=0x7f7c38005fd0 comp rx=0 tx=0).stop 2024-03-21T10:10:52.100 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:52.098+0000 7f7c53400700 1 -- 172.21.15.153:0/3128223215 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7c4c100bb0 msgr2=0x7f7c4c194160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:10:52.101 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:52.099+0000 7f7c53400700 1 --2- 172.21.15.153:0/3128223215 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7c4c100bb0 0x7f7c4c194160 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f7c4000d900 tx=0x7f7c4000dcc0 comp rx=0 tx=0).stop 2024-03-21T10:10:52.101 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:52.099+0000 7f7c53400700 1 -- 172.21.15.153:0/3128223215 shutdown_connections 2024-03-21T10:10:52.101 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:52.099+0000 7f7c53400700 1 --2- 172.21.15.153:0/3128223215 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f7c340724f0 0x7f7c340749b0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:10:52.101 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:52.099+0000 7f7c53400700 1 --2- 172.21.15.153:0/3128223215 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7c4c100bb0 0x7f7c4c194160 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:10:52.101 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:52.099+0000 7f7c53400700 1 --2- 172.21.15.153:0/3128223215 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7c4c101db0 0x7f7c4c1946a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:10:52.101 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:52.099+0000 7f7c53400700 1 -- 172.21.15.153:0/3128223215 >> 172.21.15.153:0/3128223215 conn(0x7f7c4c0fc110 msgr2=0x7f7c4c104fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:10:52.101 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:52.099+0000 7f7c53400700 1 -- 172.21.15.153:0/3128223215 shutdown_connections 2024-03-21T10:10:52.101 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:52.099+0000 7f7c53400700 1 -- 172.21.15.153:0/3128223215 wait complete. 2024-03-21T10:10:52.262 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:52.260+0000 7fd16c800700 1 -- 172.21.15.153:0/3388094903 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd164108400 msgr2=0x7fd164108860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:10:52.262 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:52.260+0000 7fd16c800700 1 --2- 172.21.15.153:0/3388094903 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd164108400 0x7fd164108860 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fd154009910 tx=0x7fd154009c20 comp rx=0 tx=0).stop 2024-03-21T10:10:52.262 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:52.260+0000 7fd16c800700 1 -- 172.21.15.153:0/3388094903 shutdown_connections 2024-03-21T10:10:52.262 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:52.260+0000 7fd16c800700 1 --2- 172.21.15.153:0/3388094903 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd164108400 0x7fd164108860 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:10:52.262 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:52.260+0000 7fd16c800700 1 --2- 172.21.15.153:0/3388094903 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd164107200 0x7fd164107620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:10:52.262 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:52.260+0000 7fd16c800700 1 -- 172.21.15.153:0/3388094903 >> 172.21.15.153:0/3388094903 conn(0x7fd1640761b0 msgr2=0x7fd164078610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:10:52.262 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:52.260+0000 7fd16c800700 1 -- 172.21.15.153:0/3388094903 shutdown_connections 2024-03-21T10:10:52.262 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:52.260+0000 7fd16c800700 1 -- 172.21.15.153:0/3388094903 wait complete. 2024-03-21T10:10:52.264 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:52.261+0000 7fd16c800700 1 Processor -- start 2024-03-21T10:10:52.264 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:52.261+0000 7fd16c800700 1 -- start start 2024-03-21T10:10:52.264 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:52.261+0000 7fd16c800700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd164107200 0x7fd16419ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:10:52.264 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:52.261+0000 7fd16c800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd164108400 0x7fd16419cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:10:52.264 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:52.261+0000 7fd16c800700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd16419d560 con 0x7fd164108400 2024-03-21T10:10:52.264 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:52.261+0000 7fd16c800700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd16419d6a0 con 0x7fd164107200 2024-03-21T10:10:52.264 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:52.262+0000 7fd169400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd164108400 0x7fd16419cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:10:52.264 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:52.262+0000 7fd169400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd164108400 0x7fd16419cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:32886/0 (socket says 172.21.15.153:32886) 2024-03-21T10:10:52.264 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:52.262+0000 7fd169400700 1 -- 172.21.15.153:0/2786261702 learned_addr learned my addr 172.21.15.153:0/2786261702 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:10:52.264 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:52.262+0000 7fd169e00700 1 --2- 172.21.15.153:0/2786261702 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd164107200 0x7fd16419ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:10:52.264 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:52.262+0000 7fd169400700 1 -- 172.21.15.153:0/2786261702 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd164107200 msgr2=0x7fd16419ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:10:52.264 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:52.262+0000 7fd169400700 1 --2- 172.21.15.153:0/2786261702 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd164107200 0x7fd16419ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:10:52.265 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:52.262+0000 7fd169400700 1 -- 172.21.15.153:0/2786261702 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1540095f0 con 0x7fd164108400 2024-03-21T10:10:52.265 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:52.262+0000 7fd169400700 1 --2- 172.21.15.153:0/2786261702 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd164108400 0x7fd16419cf40 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fd154005b40 tx=0x7fd15400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:10:52.265 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:52.263+0000 7fd15ea00700 1 -- 172.21.15.153:0/2786261702 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd15401d070 con 0x7fd164108400 2024-03-21T10:10:52.265 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:52.263+0000 7fd15ea00700 1 -- 172.21.15.153:0/2786261702 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fd154022470 con 0x7fd164108400 2024-03-21T10:10:52.265 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:52.263+0000 7fd16c800700 1 -- 172.21.15.153:0/2786261702 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1641a20f0 con 0x7fd164108400 2024-03-21T10:10:52.265 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:52.263+0000 7fd15ea00700 1 -- 172.21.15.153:0/2786261702 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd15400f690 con 0x7fd164108400 2024-03-21T10:10:52.266 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:52.263+0000 7fd16c800700 1 -- 172.21.15.153:0/2786261702 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1641a2660 con 0x7fd164108400 2024-03-21T10:10:52.266 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:52.264+0000 7fd16c800700 1 -- 172.21.15.153:0/2786261702 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd16410fe00 con 0x7fd164108400 2024-03-21T10:10:52.266 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:52.265+0000 7fd15ea00700 1 -- 172.21.15.153:0/2786261702 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 31) v1 ==== 92072+0+0 (secure 0 0 0) 0x7fd1540225e0 con 0x7fd164108400 2024-03-21T10:10:52.268 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:52.266+0000 7fd15ea00700 1 --2- 172.21.15.153:0/2786261702 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7fd14c06e0e0 0x7fd14c0705a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:10:52.268 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:52.266+0000 7fd15ea00700 1 -- 172.21.15.153:0/2786261702 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fd15408f6d0 con 0x7fd164108400 2024-03-21T10:10:52.268 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:52.267+0000 7fd169e00700 1 --2- 172.21.15.153:0/2786261702 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7fd14c06e0e0 0x7fd14c0705a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:10:52.268 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:52.267+0000 7fd169e00700 1 --2- 172.21.15.153:0/2786261702 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7fd14c06e0e0 0x7fd14c0705a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd158006fd0 tx=0x7fd158008040 comp rx=0 tx=0).ready entity=mgr.14666 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:10:52.273 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:52.271+0000 7fd15ea00700 1 -- 172.21.15.153:0/2786261702 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185454 (secure 0 0 0) 0x7fd15405c960 con 0x7fd164108400 2024-03-21T10:10:52.314 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:10:52 smithi153 ceph-mon[29117]: pgmap v72: 65 pgs: 65 active+clean; 320 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 235 op/s 2024-03-21T10:10:52.536 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:52.535+0000 7fd16c800700 1 -- 172.21.15.153:0/2786261702 --> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fd1641a2d60 con 0x7fd14c06e0e0 2024-03-21T10:10:52.550 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:52.549+0000 7fd15ea00700 1 -- 172.21.15.153:0/2786261702 <== mgr.14666 v2:172.21.15.153:6800/2524145044 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fd1641a2d60 con 0x7fd14c06e0e0 2024-03-21T10:10:52.551 INFO:teuthology.orchestra.run.smithi153.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-21T10:10:52.551 INFO:teuthology.orchestra.run.smithi153.stdout:alertmanager.smithi153 smithi153 *:9093,9094 running (11m) 30s ago 13m 13.4M - 0.25.0 c8568f914cd2 2cfcc225c7e6 2024-03-21T10:10:52.551 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi153 smithi153 running (13m) 30s ago 13m 8010k - 18.2.1 5be31c24972a 5b61994396a6 2024-03-21T10:10:52.551 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi174 smithi174 running (12m) 38s ago 12m 7940k - 18.2.1 5be31c24972a 4f7a5fc3ac28 2024-03-21T10:10:52.551 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi153 smithi153 running (13m) 30s ago 13m 6665k - 18.2.1 5be31c24972a 90a8b8ea03d1 2024-03-21T10:10:52.551 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi174 smithi174 running (12m) 38s ago 12m 6665k - 18.2.1 5be31c24972a a4c929e4dc77 2024-03-21T10:10:52.551 INFO:teuthology.orchestra.run.smithi153.stdout:grafana.smithi153 smithi153 *:3000 running (11m) 30s ago 12m 74.8M - 9.4.7 954c08fa6188 3af5800ecfef 2024-03-21T10:10:52.551 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.iuyluj smithi153 running (6m) 30s ago 6m 153M - 18.2.1 5be31c24972a 02f667cee979 2024-03-21T10:10:52.551 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.zipckr smithi153 running (6m) 30s ago 6m 132M - 18.2.1 5be31c24972a 68a853929f8f 2024-03-21T10:10:52.551 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.nhozag smithi174 running (6m) 38s ago 6m 128M - 18.2.1 5be31c24972a 2fee4000308f 2024-03-21T10:10:52.552 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.yyimst smithi174 running (6m) 38s ago 6m 155M - 18.2.1 5be31c24972a d78a729220d1 2024-03-21T10:10:52.552 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi153.tluxni smithi153 *:8443,9283,8765 running (2m) 30s ago 15m 496M - 19.0.0-2231-g2edfee68 237e43c7a65f 7ffee0e55382 2024-03-21T10:10:52.552 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi174.qpcwat smithi174 *:8443,9283,8765 running (78s) 38s ago 11m 447M - 19.0.0-2231-g2edfee68 237e43c7a65f 748968f216ab 2024-03-21T10:10:52.552 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi153 smithi153 running (15m) 30s ago 15m 55.2M 2048M 18.2.1 5be31c24972a 813963a95d69 2024-03-21T10:10:52.552 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi174 smithi174 running (11m) 38s ago 11m 45.3M 2048M 18.2.1 5be31c24972a 99204923eec6 2024-03-21T10:10:52.552 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi153 smithi153 *:9100 running (50s) 30s ago 13m 2805k - 1.5.0 0da6a335fe13 bb835edc0e0b 2024-03-21T10:10:52.552 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi174 smithi174 *:9100 running (44s) 38s ago 11m 6765k - 1.5.0 0da6a335fe13 7c4ecad790fa 2024-03-21T10:10:52.552 INFO:teuthology.orchestra.run.smithi153.stdout:osd.0 smithi153 running (10m) 30s ago 10m 191M 1217M 18.2.1 5be31c24972a 28912a2319b4 2024-03-21T10:10:52.552 INFO:teuthology.orchestra.run.smithi153.stdout:osd.1 smithi153 running (10m) 30s ago 10m 194M 1217M 18.2.1 5be31c24972a ca60681e2314 2024-03-21T10:10:52.552 INFO:teuthology.orchestra.run.smithi153.stdout:osd.2 smithi153 running (9m) 30s ago 9m 158M 1217M 18.2.1 5be31c24972a 8db799f8962a 2024-03-21T10:10:52.552 INFO:teuthology.orchestra.run.smithi153.stdout:osd.3 smithi174 running (9m) 38s ago 9m 213M 2241M 18.2.1 5be31c24972a dafc89e0658d 2024-03-21T10:10:52.552 INFO:teuthology.orchestra.run.smithi153.stdout:osd.4 smithi174 running (8m) 38s ago 8m 201M 2241M 18.2.1 5be31c24972a c4cd15701f05 2024-03-21T10:10:52.552 INFO:teuthology.orchestra.run.smithi153.stdout:osd.5 smithi174 running (8m) 38s ago 8m 187M 2241M 18.2.1 5be31c24972a 3b69886ce22e 2024-03-21T10:10:52.553 INFO:teuthology.orchestra.run.smithi153.stdout:prometheus.smithi153 smithi153 *:9095 starting - - - - 2024-03-21T10:10:52.553 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:52.552+0000 7fd16c800700 1 -- 172.21.15.153:0/2786261702 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7fd14c06e0e0 msgr2=0x7fd14c0705a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:10:52.553 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:52.552+0000 7fd16c800700 1 --2- 172.21.15.153:0/2786261702 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7fd14c06e0e0 0x7fd14c0705a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd158006fd0 tx=0x7fd158008040 comp rx=0 tx=0).stop 2024-03-21T10:10:52.553 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:52.552+0000 7fd16c800700 1 -- 172.21.15.153:0/2786261702 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd164108400 msgr2=0x7fd16419cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:10:52.554 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:52.552+0000 7fd16c800700 1 --2- 172.21.15.153:0/2786261702 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd164108400 0x7fd16419cf40 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fd154005b40 tx=0x7fd15400be20 comp rx=0 tx=0).stop 2024-03-21T10:10:52.554 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:52.552+0000 7fd16c800700 1 -- 172.21.15.153:0/2786261702 shutdown_connections 2024-03-21T10:10:52.554 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:52.552+0000 7fd16c800700 1 --2- 172.21.15.153:0/2786261702 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7fd14c06e0e0 0x7fd14c0705a0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:10:52.554 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:52.552+0000 7fd16c800700 1 --2- 172.21.15.153:0/2786261702 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd164107200 0x7fd16419ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:10:52.554 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:52.552+0000 7fd16c800700 1 --2- 172.21.15.153:0/2786261702 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd164108400 0x7fd16419cf40 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:10:52.554 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:52.552+0000 7fd16c800700 1 -- 172.21.15.153:0/2786261702 >> 172.21.15.153:0/2786261702 conn(0x7fd1640761b0 msgr2=0x7fd16410b630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:10:52.554 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:52.552+0000 7fd16c800700 1 -- 172.21.15.153:0/2786261702 shutdown_connections 2024-03-21T10:10:52.555 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:52.552+0000 7fd16c800700 1 -- 172.21.15.153:0/2786261702 wait complete. 2024-03-21T10:10:52.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:10:52 smithi174 ceph-mon[37496]: pgmap v72: 65 pgs: 65 active+clean; 320 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 235 op/s 2024-03-21T10:10:52.707 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:52.705+0000 7f09a1000700 1 -- 172.21.15.153:0/3594203421 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f099c103f70 msgr2=0x7f099c1043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:10:52.707 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:52.705+0000 7f09a1000700 1 --2- 172.21.15.153:0/3594203421 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f099c103f70 0x7f099c1043f0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f0988009910 tx=0x7f0988009c20 comp rx=0 tx=0).stop 2024-03-21T10:10:52.707 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:52.705+0000 7f09a1000700 1 -- 172.21.15.153:0/3594203421 shutdown_connections 2024-03-21T10:10:52.707 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:52.705+0000 7f09a1000700 1 --2- 172.21.15.153:0/3594203421 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f099c103f70 0x7f099c1043f0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:10:52.707 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:52.705+0000 7f09a1000700 1 --2- 172.21.15.153:0/3594203421 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f099c102e10 0x7f099c103230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:10:52.707 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:52.705+0000 7f09a1000700 1 -- 172.21.15.153:0/3594203421 >> 172.21.15.153:0/3594203421 conn(0x7f099c0fe3d0 msgr2=0x7f099c1007f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:10:52.707 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:52.705+0000 7f09a1000700 1 -- 172.21.15.153:0/3594203421 shutdown_connections 2024-03-21T10:10:52.707 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:52.705+0000 7f09a1000700 1 -- 172.21.15.153:0/3594203421 wait complete. 2024-03-21T10:10:52.709 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:52.706+0000 7f09a1000700 1 Processor -- start 2024-03-21T10:10:52.709 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:52.706+0000 7f09a1000700 1 -- start start 2024-03-21T10:10:52.709 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:52.707+0000 7f09a1000700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f099c102e10 0x7f099c198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:10:52.709 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:52.707+0000 7f09a1000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f099c103f70 0x7f099c198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:10:52.709 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:52.707+0000 7f09a1000700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f099c1990a0 con 0x7f099c103f70 2024-03-21T10:10:52.709 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:52.707+0000 7f09a1000700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f099c1991e0 con 0x7f099c102e10 2024-03-21T10:10:52.710 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:52.707+0000 7f0999400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f099c103f70 0x7f099c198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:10:52.710 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:52.707+0000 7f0999400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f099c103f70 0x7f099c198a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:32912/0 (socket says 172.21.15.153:32912) 2024-03-21T10:10:52.710 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:52.707+0000 7f0999400700 1 -- 172.21.15.153:0/1390575366 learned_addr learned my addr 172.21.15.153:0/1390575366 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:10:52.710 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:52.707+0000 7f0999400700 1 -- 172.21.15.153:0/1390575366 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f099c102e10 msgr2=0x7f099c198540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:10:52.710 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:52.707+0000 7f0999e00700 1 --2- 172.21.15.153:0/1390575366 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f099c102e10 0x7f099c198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:10:52.710 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:52.707+0000 7f0999400700 1 --2- 172.21.15.153:0/1390575366 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f099c102e10 0x7f099c198540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:10:52.710 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:52.707+0000 7f0999400700 1 -- 172.21.15.153:0/1390575366 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09880095f0 con 0x7f099c103f70 2024-03-21T10:10:52.710 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:52.708+0000 7f0999400700 1 --2- 172.21.15.153:0/1390575366 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f099c103f70 0x7f099c198a80 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f0988005b40 tx=0x7f098800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:10:52.711 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:52.708+0000 7f0992a00700 1 -- 172.21.15.153:0/1390575366 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f098801d070 con 0x7f099c103f70 2024-03-21T10:10:52.711 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:52.708+0000 7f0992a00700 1 -- 172.21.15.153:0/1390575366 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f098800f460 con 0x7f099c103f70 2024-03-21T10:10:52.711 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:52.708+0000 7f09a1000700 1 -- 172.21.15.153:0/1390575366 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f099c19dc30 con 0x7f099c103f70 2024-03-21T10:10:52.711 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:52.708+0000 7f0992a00700 1 -- 172.21.15.153:0/1390575366 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0988021640 con 0x7f099c103f70 2024-03-21T10:10:52.712 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:52.708+0000 7f09a1000700 1 -- 172.21.15.153:0/1390575366 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f099c19e120 con 0x7f099c103f70 2024-03-21T10:10:52.712 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:52.709+0000 7f09a1000700 1 -- 172.21.15.153:0/1390575366 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f099c10b8f0 con 0x7f099c103f70 2024-03-21T10:10:52.712 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:52.710+0000 7f0992a00700 1 -- 172.21.15.153:0/1390575366 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 31) v1 ==== 92072+0+0 (secure 0 0 0) 0x7f098800fa90 con 0x7f099c103f70 2024-03-21T10:10:52.713 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:52.711+0000 7f0992a00700 1 --2- 172.21.15.153:0/1390575366 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f098006e0e0 0x7f09800705a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:10:52.713 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:52.711+0000 7f0992a00700 1 -- 172.21.15.153:0/1390575366 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f098808f1e0 con 0x7f099c103f70 2024-03-21T10:10:52.713 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:52.712+0000 7f0999e00700 1 --2- 172.21.15.153:0/1390575366 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f098006e0e0 0x7f09800705a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:10:52.713 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:52.712+0000 7f0999e00700 1 --2- 172.21.15.153:0/1390575366 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f098006e0e0 0x7f09800705a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f098c005950 tx=0x7f098c0058e0 comp rx=0 tx=0).ready entity=mgr.14666 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:10:52.717 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:52.716+0000 7f0992a00700 1 -- 172.21.15.153:0/1390575366 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185454 (secure 0 0 0) 0x7f098805c3f0 con 0x7f099c103f70 2024-03-21T10:10:53.051 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:53.048+0000 7f09a1000700 1 -- 172.21.15.153:0/1390575366 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f099c10bb10 con 0x7f099c103f70 2024-03-21T10:10:53.052 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:53.050+0000 7f0992a00700 1 -- 172.21.15.153:0/1390575366 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7f098805bf80 con 0x7f099c103f70 2024-03-21T10:10:53.053 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:10:53.053 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": { 2024-03-21T10:10:53.053 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-03-21T10:10:53.053 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:10:53.053 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": { 2024-03-21T10:10:53.053 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:10:53.053 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:10:53.053 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": { 2024-03-21T10:10:53.054 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-03-21T10:10:53.054 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:10:53.054 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": { 2024-03-21T10:10:53.054 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-03-21T10:10:53.054 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:10:53.054 INFO:teuthology.orchestra.run.smithi153.stdout: "overall": { 2024-03-21T10:10:53.054 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 12, 2024-03-21T10:10:53.054 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:10:53.054 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-03-21T10:10:53.054 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:10:53.058 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:53.055+0000 7f09a1000700 1 -- 172.21.15.153:0/1390575366 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f098006e0e0 msgr2=0x7f09800705a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:10:53.058 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:53.055+0000 7f09a1000700 1 --2- 172.21.15.153:0/1390575366 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f098006e0e0 0x7f09800705a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f098c005950 tx=0x7f098c0058e0 comp rx=0 tx=0).stop 2024-03-21T10:10:53.058 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:53.055+0000 7f09a1000700 1 -- 172.21.15.153:0/1390575366 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f099c103f70 msgr2=0x7f099c198a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:10:53.058 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:53.055+0000 7f09a1000700 1 --2- 172.21.15.153:0/1390575366 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f099c103f70 0x7f099c198a80 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f0988005b40 tx=0x7f098800be20 comp rx=0 tx=0).stop 2024-03-21T10:10:53.058 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:53.055+0000 7f09a1000700 1 -- 172.21.15.153:0/1390575366 shutdown_connections 2024-03-21T10:10:53.059 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:53.055+0000 7f09a1000700 1 --2- 172.21.15.153:0/1390575366 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f098006e0e0 0x7f09800705a0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:10:53.059 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:53.055+0000 7f09a1000700 1 --2- 172.21.15.153:0/1390575366 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f099c102e10 0x7f099c198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:10:53.059 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:53.055+0000 7f09a1000700 1 --2- 172.21.15.153:0/1390575366 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f099c103f70 0x7f099c198a80 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:10:53.059 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:53.055+0000 7f09a1000700 1 -- 172.21.15.153:0/1390575366 >> 172.21.15.153:0/1390575366 conn(0x7f099c0fe3d0 msgr2=0x7f099c107230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:10:53.059 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:53.056+0000 7f09a1000700 1 -- 172.21.15.153:0/1390575366 shutdown_connections 2024-03-21T10:10:53.059 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:53.056+0000 7f09a1000700 1 -- 172.21.15.153:0/1390575366 wait complete. 2024-03-21T10:10:53.252 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:53.250+0000 7f321f200700 1 -- 172.21.15.153:0/3263255876 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3218074dc0 msgr2=0x7f3218073220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:10:53.252 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:53.250+0000 7f321f200700 1 --2- 172.21.15.153:0/3263255876 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3218074dc0 0x7f3218073220 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f3208009910 tx=0x7f3208009c20 comp rx=0 tx=0).stop 2024-03-21T10:10:53.252 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:53.250+0000 7f321f200700 1 -- 172.21.15.153:0/3263255876 shutdown_connections 2024-03-21T10:10:53.253 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:53.250+0000 7f321f200700 1 --2- 172.21.15.153:0/3263255876 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f32180737f0 0x7f3218073c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:10:53.253 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:53.250+0000 7f321f200700 1 --2- 172.21.15.153:0/3263255876 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3218074dc0 0x7f3218073220 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:10:53.253 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:53.250+0000 7f321f200700 1 -- 172.21.15.153:0/3263255876 >> 172.21.15.153:0/3263255876 conn(0x7f32180fc110 msgr2=0x7f32180fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:10:53.253 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:53.251+0000 7f321f200700 1 -- 172.21.15.153:0/3263255876 shutdown_connections 2024-03-21T10:10:53.253 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:53.251+0000 7f321f200700 1 -- 172.21.15.153:0/3263255876 wait complete. 2024-03-21T10:10:53.253 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:53.252+0000 7f321f200700 1 Processor -- start 2024-03-21T10:10:53.253 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:53.252+0000 7f321f200700 1 -- start start 2024-03-21T10:10:53.254 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:53.252+0000 7f321f200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32180737f0 0x7f3218198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:10:53.254 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:53.252+0000 7f321f200700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3218074dc0 0x7f3218198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:10:53.254 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:53.252+0000 7f321f200700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32181990d0 con 0x7f32180737f0 2024-03-21T10:10:53.254 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:53.252+0000 7f321f200700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3218199210 con 0x7f3218074dc0 2024-03-21T10:10:53.255 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:53.253+0000 7f3217e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32180737f0 0x7f3218198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:10:53.255 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:53.253+0000 7f3217e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32180737f0 0x7f3218198570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:53944/0 (socket says 172.21.15.153:53944) 2024-03-21T10:10:53.255 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:53.253+0000 7f3217e00700 1 -- 172.21.15.153:0/645486645 learned_addr learned my addr 172.21.15.153:0/645486645 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:10:53.255 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:53.253+0000 7f3217400700 1 --2- 172.21.15.153:0/645486645 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3218074dc0 0x7f3218198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:10:53.255 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:53.253+0000 7f3217e00700 1 -- 172.21.15.153:0/645486645 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3218074dc0 msgr2=0x7f3218198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:10:53.255 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:53.253+0000 7f3217e00700 1 --2- 172.21.15.153:0/645486645 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3218074dc0 0x7f3218198ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:10:53.255 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:53.253+0000 7f3217e00700 1 -- 172.21.15.153:0/645486645 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32080095f0 con 0x7f32180737f0 2024-03-21T10:10:53.256 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:53.254+0000 7f3217e00700 1 --2- 172.21.15.153:0/645486645 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32180737f0 0x7f3218198570 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f320800bda0 tx=0x7f320800be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:10:53.258 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:53.256+0000 7f3214c00700 1 -- 172.21.15.153:0/645486645 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f320801d070 con 0x7f32180737f0 2024-03-21T10:10:53.259 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:53.256+0000 7f3214c00700 1 -- 172.21.15.153:0/645486645 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f3208004c50 con 0x7f32180737f0 2024-03-21T10:10:53.259 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:53.256+0000 7f3214c00700 1 -- 172.21.15.153:0/645486645 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f320800f690 con 0x7f32180737f0 2024-03-21T10:10:53.259 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:53.256+0000 7f321f200700 1 -- 172.21.15.153:0/645486645 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f321819dc60 con 0x7f32180737f0 2024-03-21T10:10:53.259 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:53.256+0000 7f321f200700 1 -- 172.21.15.153:0/645486645 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f321819e150 con 0x7f32180737f0 2024-03-21T10:10:53.259 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:53.258+0000 7f320e000700 1 -- 172.21.15.153:0/645486645 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31f00052f0 con 0x7f32180737f0 2024-03-21T10:10:53.260 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:53.258+0000 7f3214c00700 1 -- 172.21.15.153:0/645486645 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 31) v1 ==== 92072+0+0 (secure 0 0 0) 0x7f3208022a20 con 0x7f32180737f0 2024-03-21T10:10:53.261 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:53.259+0000 7f3214c00700 1 --2- 172.21.15.153:0/645486645 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f31fc06e0e0 0x7f31fc0705a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:10:53.261 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:53.260+0000 7f3214c00700 1 -- 172.21.15.153:0/645486645 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f320808f550 con 0x7f32180737f0 2024-03-21T10:10:53.261 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:53.260+0000 7f3217400700 1 --2- 172.21.15.153:0/645486645 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f31fc06e0e0 0x7f31fc0705a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:10:53.262 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:53.261+0000 7f3217400700 1 --2- 172.21.15.153:0/645486645 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f31fc06e0e0 0x7f31fc0705a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3204005fd0 tx=0x7f3204005dd0 comp rx=0 tx=0).ready entity=mgr.14666 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:10:53.266 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:53.264+0000 7f3214c00700 1 -- 172.21.15.153:0/645486645 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185454 (secure 0 0 0) 0x7f320805c7f0 con 0x7f32180737f0 2024-03-21T10:10:53.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:10:53 smithi153 ceph-mon[29117]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:10:53.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:10:53 smithi153 ceph-mon[29117]: from='client.14776 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:10:53.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:10:53 smithi153 ceph-mon[29117]: from='client.? 172.21.15.153:0/1390575366' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:10:53.538 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:53.536+0000 7f320e000700 1 -- 172.21.15.153:0/645486645 --> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f31f0000bc0 con 0x7f31fc06e0e0 2024-03-21T10:10:53.538 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:53.537+0000 7f3214c00700 1 -- 172.21.15.153:0/645486645 <== mgr.14666 v2:172.21.15.153:6800/2524145044 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+367 (secure 0 0 0) 0x7f31f0000bc0 con 0x7f31fc06e0e0 2024-03-21T10:10:53.539 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:10:53.539 INFO:teuthology.orchestra.run.smithi153.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232", 2024-03-21T10:10:53.539 INFO:teuthology.orchestra.run.smithi153.stdout: "in_progress": true, 2024-03-21T10:10:53.539 INFO:teuthology.orchestra.run.smithi153.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-03-21T10:10:53.539 INFO:teuthology.orchestra.run.smithi153.stdout: "services_complete": [ 2024-03-21T10:10:53.539 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr" 2024-03-21T10:10:53.539 INFO:teuthology.orchestra.run.smithi153.stdout: ], 2024-03-21T10:10:53.539 INFO:teuthology.orchestra.run.smithi153.stdout: "progress": "2/2 daemons upgraded", 2024-03-21T10:10:53.539 INFO:teuthology.orchestra.run.smithi153.stdout: "message": "Currently upgrading prometheus daemons", 2024-03-21T10:10:53.539 INFO:teuthology.orchestra.run.smithi153.stdout: "is_paused": false 2024-03-21T10:10:53.540 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:10:53.542 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:53.540+0000 7f320e000700 1 -- 172.21.15.153:0/645486645 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f31fc06e0e0 msgr2=0x7f31fc0705a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:10:53.542 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:53.540+0000 7f320e000700 1 --2- 172.21.15.153:0/645486645 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f31fc06e0e0 0x7f31fc0705a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3204005fd0 tx=0x7f3204005dd0 comp rx=0 tx=0).stop 2024-03-21T10:10:53.542 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:53.540+0000 7f320e000700 1 -- 172.21.15.153:0/645486645 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32180737f0 msgr2=0x7f3218198570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:10:53.542 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:53.540+0000 7f320e000700 1 --2- 172.21.15.153:0/645486645 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32180737f0 0x7f3218198570 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f320800bda0 tx=0x7f320800be80 comp rx=0 tx=0).stop 2024-03-21T10:10:53.542 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:53.540+0000 7f320e000700 1 -- 172.21.15.153:0/645486645 shutdown_connections 2024-03-21T10:10:53.542 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:53.540+0000 7f320e000700 1 --2- 172.21.15.153:0/645486645 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f31fc06e0e0 0x7f31fc0705a0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:10:53.542 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:53.540+0000 7f320e000700 1 --2- 172.21.15.153:0/645486645 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32180737f0 0x7f3218198570 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:10:53.542 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:53.540+0000 7f320e000700 1 --2- 172.21.15.153:0/645486645 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3218074dc0 0x7f3218198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:10:53.542 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:53.540+0000 7f320e000700 1 -- 172.21.15.153:0/645486645 >> 172.21.15.153:0/645486645 conn(0x7f32180fc110 msgr2=0x7f3218106990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:10:53.542 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:53.540+0000 7f320e000700 1 -- 172.21.15.153:0/645486645 shutdown_connections 2024-03-21T10:10:53.543 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:10:53.540+0000 7f320e000700 1 -- 172.21.15.153:0/645486645 wait complete. 2024-03-21T10:10:53.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:10:53 smithi174 ceph-mon[37496]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:10:53.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:10:53 smithi174 ceph-mon[37496]: from='client.14776 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:10:53.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:10:53 smithi174 ceph-mon[37496]: from='client.? 172.21.15.153:0/1390575366' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:10:54.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:10:54 smithi174 ceph-mon[37496]: from='client.14780 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:10:54.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:10:54 smithi174 ceph-mon[37496]: pgmap v73: 65 pgs: 65 active+clean; 320 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 703 KiB/s rd, 713 KiB/s wr, 149 op/s 2024-03-21T10:10:54.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:10:54 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:10:54.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:10:54 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:10:54.705 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:10:54 smithi153 ceph-mon[29117]: from='client.14780 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:10:54.705 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:10:54 smithi153 ceph-mon[29117]: pgmap v73: 65 pgs: 65 active+clean; 320 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 703 KiB/s rd, 713 KiB/s wr, 149 op/s 2024-03-21T10:10:54.705 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:10:54 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:10:54.705 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:10:54 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:10:55.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:10:55 smithi174 ceph-mon[37496]: from='client.14788 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:10:55.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:10:55 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:10:55.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:10:55 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:10:55.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:10:55 smithi153 ceph-mon[29117]: from='client.14788 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:10:55.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:10:55 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:10:55.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:10:55 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:10:56.161 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.1... 2024-03-21T10:10:56.162 DEBUG:teuthology.orchestra.run.smithi174:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.1 /home/ubuntu/cephtest/clone.client.1 2024-03-21T10:10:56.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:10:56 smithi174 ceph-mon[37496]: pgmap v74: 65 pgs: 65 active+clean; 320 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 931 KiB/s rd, 954 KiB/s wr, 188 op/s 2024-03-21T10:10:56.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:10:56 smithi153 ceph-mon[29117]: pgmap v74: 65 pgs: 65 active+clean; 320 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 931 KiB/s rd, 954 KiB/s wr, 188 op/s 2024-03-21T10:10:57.194 DEBUG:teuthology.parallel:result is None 2024-03-21T10:10:57.194 DEBUG:teuthology.orchestra.run.smithi153:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-03-21T10:10:57.295 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-03-21T10:10:57.295 DEBUG:teuthology.orchestra.run.smithi174:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1 2024-03-21T10:10:57.444 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.1/client.1 2024-03-21T10:10:57.444 DEBUG:teuthology.parallel:result is None 2024-03-21T10:10:58.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:10:58 smithi174 ceph-mon[37496]: pgmap v75: 65 pgs: 65 active+clean; 320 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 496 KiB/s rd, 517 KiB/s wr, 103 op/s 2024-03-21T10:10:58.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:10:58 smithi153 ceph-mon[29117]: pgmap v75: 65 pgs: 65 active+clean; 320 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 496 KiB/s rd, 517 KiB/s wr, 103 op/s 2024-03-21T10:11:00.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:00 smithi174 ceph-mon[37496]: pgmap v76: 65 pgs: 65 active+clean; 320 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 496 KiB/s rd, 517 KiB/s wr, 103 op/s 2024-03-21T10:11:00.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:00 smithi153 ceph-mon[29117]: pgmap v76: 65 pgs: 65 active+clean; 320 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 496 KiB/s rd, 517 KiB/s wr, 103 op/s 2024-03-21T10:11:01.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:01 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:11:01.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:01 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:11:01.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:01 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:11:01.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:01 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:11:01.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:01 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:11:01.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:01 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-21T10:11:01.703 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:01 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:11:01.703 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:01 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:11:01.703 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:01 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:11:01.703 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:01 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:11:01.703 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:01 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:11:01.703 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:01 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:11:01.703 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:01 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:11:01.703 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:01 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:11:01.703 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:01 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:11:01.703 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:01 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:11:01.738 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:01 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:11:01.738 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:01 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:11:01.739 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:01 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:11:01.739 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:01 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:11:01.739 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:01 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:11:01.739 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:01 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-21T10:11:01.739 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:01 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:11:01.739 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:01 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:11:01.739 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:01 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:11:01.739 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:01 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:11:01.739 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:01 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:11:01.739 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:01 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:11:01.739 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:01 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:11:01.739 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:01 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:11:01.739 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:01 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:11:01.739 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:01 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:11:02.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:02 smithi153 ceph-mon[29117]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-21T10:11:02.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:02 smithi153 ceph-mon[29117]: pgmap v77: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 630 KiB/s rd, 628 KiB/s wr, 127 op/s 2024-03-21T10:11:02.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:02 smithi174 ceph-mon[37496]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-21T10:11:02.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:02 smithi174 ceph-mon[37496]: pgmap v77: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 630 KiB/s rd, 628 KiB/s wr, 127 op/s 2024-03-21T10:11:03.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:03 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:11:03.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:03 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:11:04.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:04 smithi153 ceph-mon[29117]: Upgrade: Updating alertmanager.smithi153 2024-03-21T10:11:04.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:04 smithi153 ceph-mon[29117]: Deploying daemon alertmanager.smithi153 on smithi153 2024-03-21T10:11:04.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:04 smithi153 ceph-mon[29117]: pgmap v78: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 362 KiB/s rd, 352 KiB/s wr, 63 op/s 2024-03-21T10:11:04.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:04 smithi174 ceph-mon[37496]: Upgrade: Updating alertmanager.smithi153 2024-03-21T10:11:04.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:04 smithi174 ceph-mon[37496]: Deploying daemon alertmanager.smithi153 on smithi153 2024-03-21T10:11:04.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:04 smithi174 ceph-mon[37496]: pgmap v78: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 362 KiB/s rd, 352 KiB/s wr, 63 op/s 2024-03-21T10:11:06.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:06 smithi153 ceph-mon[29117]: pgmap v79: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 363 KiB/s rd, 353 KiB/s wr, 66 op/s 2024-03-21T10:11:06.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:06 smithi174 ceph-mon[37496]: pgmap v79: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 363 KiB/s rd, 353 KiB/s wr, 66 op/s 2024-03-21T10:11:07.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:07 smithi153 ceph-mon[29117]: pgmap v80: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 136 KiB/s rd, 112 KiB/s wr, 26 op/s 2024-03-21T10:11:07.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:07 smithi174 ceph-mon[37496]: pgmap v80: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 136 KiB/s rd, 112 KiB/s wr, 26 op/s 2024-03-21T10:11:10.519 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:10 smithi153 ceph-mon[29117]: pgmap v81: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 136 KiB/s rd, 112 KiB/s wr, 26 op/s 2024-03-21T10:11:10.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:10 smithi174 ceph-mon[37496]: pgmap v81: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 136 KiB/s rd, 112 KiB/s wr, 26 op/s 2024-03-21T10:11:11.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:11 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:11:11.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:11 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:11:11.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:11 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:11:11.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:11 smithi174 ceph-mon[37496]: pgmap v82: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 137 KiB/s rd, 112 KiB/s wr, 28 op/s 2024-03-21T10:11:12.051 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:11 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:11:12.051 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:11 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:11:12.051 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:11 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:11:12.051 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:11 smithi153 ceph-mon[29117]: pgmap v82: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 137 KiB/s rd, 112 KiB/s wr, 28 op/s 2024-03-21T10:11:14.540 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:14 smithi174 ceph-mon[37496]: pgmap v83: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 767 B/s wr, 4 op/s 2024-03-21T10:11:14.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:14 smithi153 ceph-mon[29117]: pgmap v83: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 767 B/s wr, 4 op/s 2024-03-21T10:11:16.558 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:16 smithi153 ceph-mon[29117]: pgmap v84: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 3.1 KiB/s rd, 767 B/s wr, 6 op/s 2024-03-21T10:11:16.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:16 smithi174 ceph-mon[37496]: pgmap v84: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 3.1 KiB/s rd, 767 B/s wr, 6 op/s 2024-03-21T10:11:18.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:18 smithi153 ceph-mon[29117]: pgmap v85: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 170 B/s wr, 3 op/s 2024-03-21T10:11:18.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:18 smithi174 ceph-mon[37496]: pgmap v85: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 170 B/s wr, 3 op/s 2024-03-21T10:11:19.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:19 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:11:19.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:19 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:11:19.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:19 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:11:19.696 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:19 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:11:19.696 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:19 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:11:19.696 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:19 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:11:20.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:20 smithi174 ceph-mon[37496]: pgmap v86: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 170 B/s wr, 3 op/s 2024-03-21T10:11:20.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:20 smithi153 ceph-mon[29117]: pgmap v86: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 170 B/s wr, 3 op/s 2024-03-21T10:11:21.680 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:21 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:11:21.680 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:21 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:11:21.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:21 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:11:21.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:21 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:11:22.649 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:22 smithi153 ceph-mon[29117]: pgmap v87: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2024-03-21T10:11:22.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:22 smithi174 ceph-mon[37496]: pgmap v87: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2024-03-21T10:11:23.712 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:23.710+0000 7fa187c00700 1 -- 172.21.15.153:0/3622328630 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa180101db0 msgr2=0x7fa180102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:11:23.712 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:23.710+0000 7fa187c00700 1 --2- 172.21.15.153:0/3622328630 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa180101db0 0x7fa180102210 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fa16c009910 tx=0x7fa16c009c20 comp rx=0 tx=0).stop 2024-03-21T10:11:23.712 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:23.710+0000 7fa187c00700 1 -- 172.21.15.153:0/3622328630 shutdown_connections 2024-03-21T10:11:23.712 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:23.710+0000 7fa187c00700 1 --2- 172.21.15.153:0/3622328630 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa180101db0 0x7fa180102210 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:11:23.712 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:23.710+0000 7fa187c00700 1 --2- 172.21.15.153:0/3622328630 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa180100bb0 0x7fa180100fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:11:23.713 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:23.710+0000 7fa187c00700 1 -- 172.21.15.153:0/3622328630 >> 172.21.15.153:0/3622328630 conn(0x7fa1800fc110 msgr2=0x7fa1800fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:11:23.713 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:23.710+0000 7fa187c00700 1 -- 172.21.15.153:0/3622328630 shutdown_connections 2024-03-21T10:11:23.713 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:23.710+0000 7fa187c00700 1 -- 172.21.15.153:0/3622328630 wait complete. 2024-03-21T10:11:23.713 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:23.711+0000 7fa187c00700 1 Processor -- start 2024-03-21T10:11:23.714 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:23.712+0000 7fa187c00700 1 -- start start 2024-03-21T10:11:23.714 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:23.712+0000 7fa187c00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa180100bb0 0x7fa180074b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:11:23.714 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:23.712+0000 7fa187c00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa180101db0 0x7fa180073180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:11:23.714 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:23.712+0000 7fa187c00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1800736c0 con 0x7fa180100bb0 2024-03-21T10:11:23.714 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:23.712+0000 7fa187c00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa180073800 con 0x7fa180101db0 2024-03-21T10:11:23.715 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:23.712+0000 7fa185200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa180100bb0 0x7fa180074b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:11:23.715 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:23.713+0000 7fa185200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa180100bb0 0x7fa180074b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:48928/0 (socket says 172.21.15.153:48928) 2024-03-21T10:11:23.715 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:23.713+0000 7fa185200700 1 -- 172.21.15.153:0/3269760165 learned_addr learned my addr 172.21.15.153:0/3269760165 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:11:23.715 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:23.713+0000 7fa17fe00700 1 --2- 172.21.15.153:0/3269760165 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa180101db0 0x7fa180073180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:11:23.715 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:23.713+0000 7fa185200700 1 -- 172.21.15.153:0/3269760165 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa180101db0 msgr2=0x7fa180073180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:11:23.715 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:23.713+0000 7fa185200700 1 --2- 172.21.15.153:0/3269760165 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa180101db0 0x7fa180073180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:11:23.715 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:23.713+0000 7fa185200700 1 -- 172.21.15.153:0/3269760165 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa16c0095f0 con 0x7fa180100bb0 2024-03-21T10:11:23.715 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:23.714+0000 7fa185200700 1 --2- 172.21.15.153:0/3269760165 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa180100bb0 0x7fa180074b30 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fa17400c8f0 tx=0x7fa17400cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:11:23.716 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:23.714+0000 7fa17d600700 1 -- 172.21.15.153:0/3269760165 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa1740078e0 con 0x7fa180100bb0 2024-03-21T10:11:23.716 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:23.714+0000 7fa17d600700 1 -- 172.21.15.153:0/3269760165 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa17400f450 con 0x7fa180100bb0 2024-03-21T10:11:23.716 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:23.714+0000 7fa187c00700 1 -- 172.21.15.153:0/3269760165 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa180073ae0 con 0x7fa180100bb0 2024-03-21T10:11:23.716 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:23.714+0000 7fa17d600700 1 -- 172.21.15.153:0/3269760165 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa174018610 con 0x7fa180100bb0 2024-03-21T10:11:23.716 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:23.714+0000 7fa187c00700 1 -- 172.21.15.153:0/3269760165 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa18019e3c0 con 0x7fa180100bb0 2024-03-21T10:11:23.717 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:23.715+0000 7fa187c00700 1 -- 172.21.15.153:0/3269760165 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa180109710 con 0x7fa180100bb0 2024-03-21T10:11:23.718 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:23.716+0000 7fa17d600700 1 -- 172.21.15.153:0/3269760165 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 31) v1 ==== 92072+0+0 (secure 0 0 0) 0x7fa17400f5c0 con 0x7fa180100bb0 2024-03-21T10:11:23.719 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:23.717+0000 7fa17d600700 1 --2- 172.21.15.153:0/3269760165 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7fa16806e0e0 0x7fa1680705a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:11:23.719 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:23.717+0000 7fa17d600700 1 -- 172.21.15.153:0/3269760165 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fa17408dcf0 con 0x7fa180100bb0 2024-03-21T10:11:23.719 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:23.718+0000 7fa17fe00700 1 --2- 172.21.15.153:0/3269760165 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7fa16806e0e0 0x7fa1680705a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:11:23.720 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:23.719+0000 7fa17fe00700 1 --2- 172.21.15.153:0/3269760165 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7fa16806e0e0 0x7fa1680705a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa16c006010 tx=0x7fa16c00b560 comp rx=0 tx=0).ready entity=mgr.14666 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:11:23.724 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:23.723+0000 7fa17d600700 1 -- 172.21.15.153:0/3269760165 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185454 (secure 0 0 0) 0x7fa17405aee0 con 0x7fa180100bb0 2024-03-21T10:11:24.009 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:24.007+0000 7fa187c00700 1 -- 172.21.15.153:0/3269760165 --> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa180106700 con 0x7fa16806e0e0 2024-03-21T10:11:24.010 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:24.009+0000 7fa17d600700 1 -- 172.21.15.153:0/3269760165 <== mgr.14666 v2:172.21.15.153:6800/2524145044 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+369 (secure 0 0 0) 0x7fa180106700 con 0x7fa16806e0e0 2024-03-21T10:11:24.013 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:24.012+0000 7fa187c00700 1 -- 172.21.15.153:0/3269760165 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7fa16806e0e0 msgr2=0x7fa1680705a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:11:24.014 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:24.012+0000 7fa187c00700 1 --2- 172.21.15.153:0/3269760165 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7fa16806e0e0 0x7fa1680705a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa16c006010 tx=0x7fa16c00b560 comp rx=0 tx=0).stop 2024-03-21T10:11:24.014 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:24.012+0000 7fa187c00700 1 -- 172.21.15.153:0/3269760165 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa180100bb0 msgr2=0x7fa180074b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:11:24.014 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:24.012+0000 7fa187c00700 1 --2- 172.21.15.153:0/3269760165 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa180100bb0 0x7fa180074b30 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fa17400c8f0 tx=0x7fa17400cc00 comp rx=0 tx=0).stop 2024-03-21T10:11:24.014 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:24.012+0000 7fa187c00700 1 -- 172.21.15.153:0/3269760165 shutdown_connections 2024-03-21T10:11:24.014 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:24.012+0000 7fa187c00700 1 --2- 172.21.15.153:0/3269760165 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7fa16806e0e0 0x7fa1680705a0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:11:24.014 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:24.012+0000 7fa187c00700 1 --2- 172.21.15.153:0/3269760165 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa180100bb0 0x7fa180074b30 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:11:24.014 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:24.012+0000 7fa187c00700 1 --2- 172.21.15.153:0/3269760165 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa180101db0 0x7fa180073180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:11:24.014 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:24.012+0000 7fa187c00700 1 -- 172.21.15.153:0/3269760165 >> 172.21.15.153:0/3269760165 conn(0x7fa1800fc110 msgr2=0x7fa180104fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:11:24.014 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:24.012+0000 7fa187c00700 1 -- 172.21.15.153:0/3269760165 shutdown_connections 2024-03-21T10:11:24.014 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:24.012+0000 7fa187c00700 1 -- 172.21.15.153:0/3269760165 wait complete. 2024-03-21T10:11:24.030 INFO:teuthology.orchestra.run.smithi153.stdout:true 2024-03-21T10:11:24.178 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:24.176+0000 7fa38b800700 1 -- 172.21.15.153:0/1126710647 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa384100bb0 msgr2=0x7fa384100fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:11:24.178 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:24.176+0000 7fa38b800700 1 --2- 172.21.15.153:0/1126710647 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa384100bb0 0x7fa384100fd0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fa378009910 tx=0x7fa378009c20 comp rx=0 tx=0).stop 2024-03-21T10:11:24.179 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:24.176+0000 7fa38b800700 1 -- 172.21.15.153:0/1126710647 shutdown_connections 2024-03-21T10:11:24.179 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:24.176+0000 7fa38b800700 1 --2- 172.21.15.153:0/1126710647 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa384101db0 0x7fa384102210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:11:24.179 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:24.176+0000 7fa38b800700 1 --2- 172.21.15.153:0/1126710647 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa384100bb0 0x7fa384100fd0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:11:24.179 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:24.176+0000 7fa38b800700 1 -- 172.21.15.153:0/1126710647 >> 172.21.15.153:0/1126710647 conn(0x7fa3840fc110 msgr2=0x7fa3840fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:11:24.179 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:24.177+0000 7fa38b800700 1 -- 172.21.15.153:0/1126710647 shutdown_connections 2024-03-21T10:11:24.179 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:24.177+0000 7fa38b800700 1 -- 172.21.15.153:0/1126710647 wait complete. 2024-03-21T10:11:24.179 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:24.178+0000 7fa38b800700 1 Processor -- start 2024-03-21T10:11:24.180 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:24.178+0000 7fa38b800700 1 -- start start 2024-03-21T10:11:24.180 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:24.179+0000 7fa38b800700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa384101db0 0x7fa3841987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:11:24.180 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:24.179+0000 7fa38b800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa384198d20 0x7fa38419dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:11:24.180 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:24.179+0000 7fa38b800700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa384199230 con 0x7fa384198d20 2024-03-21T10:11:24.180 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:24.179+0000 7fa38b800700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3841993a0 con 0x7fa384101db0 2024-03-21T10:11:24.181 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:24.179+0000 7fa383e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa384198d20 0x7fa38419dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:11:24.181 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:24.179+0000 7fa383e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa384198d20 0x7fa38419dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:48958/0 (socket says 172.21.15.153:48958) 2024-03-21T10:11:24.182 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:24.179+0000 7fa383e00700 1 -- 172.21.15.153:0/3368251580 learned_addr learned my addr 172.21.15.153:0/3368251580 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:11:24.182 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:24.179+0000 7fa388e00700 1 --2- 172.21.15.153:0/3368251580 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa384101db0 0x7fa3841987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:11:24.182 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:24.180+0000 7fa383e00700 1 -- 172.21.15.153:0/3368251580 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa384101db0 msgr2=0x7fa3841987e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:11:24.182 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:24.180+0000 7fa383e00700 1 --2- 172.21.15.153:0/3368251580 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa384101db0 0x7fa3841987e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:11:24.182 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:24.180+0000 7fa383e00700 1 -- 172.21.15.153:0/3368251580 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3780095f0 con 0x7fa384198d20 2024-03-21T10:11:24.182 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:24.180+0000 7fa383e00700 1 --2- 172.21.15.153:0/3368251580 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa384198d20 0x7fa38419dd90 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fa37000d8d0 tx=0x7fa37000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:11:24.183 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:24.180+0000 7fa381600700 1 -- 172.21.15.153:0/3368251580 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa370009940 con 0x7fa384198d20 2024-03-21T10:11:24.183 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:24.180+0000 7fa381600700 1 -- 172.21.15.153:0/3368251580 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa370010460 con 0x7fa384198d20 2024-03-21T10:11:24.183 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:24.181+0000 7fa38b800700 1 -- 172.21.15.153:0/3368251580 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa38419e330 con 0x7fa384198d20 2024-03-21T10:11:24.183 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:24.181+0000 7fa381600700 1 -- 172.21.15.153:0/3368251580 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa37000f5f0 con 0x7fa384198d20 2024-03-21T10:11:24.183 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:24.181+0000 7fa38b800700 1 -- 172.21.15.153:0/3368251580 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa38419e880 con 0x7fa384198d20 2024-03-21T10:11:24.184 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:24.182+0000 7fa38b800700 1 -- 172.21.15.153:0/3368251580 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa384105e90 con 0x7fa384198d20 2024-03-21T10:11:24.185 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:24.183+0000 7fa381600700 1 -- 172.21.15.153:0/3368251580 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 31) v1 ==== 92072+0+0 (secure 0 0 0) 0x7fa3700105d0 con 0x7fa384198d20 2024-03-21T10:11:24.186 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:24.184+0000 7fa381600700 1 --2- 172.21.15.153:0/3368251580 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7fa36c06e0e0 0x7fa36c0705a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:11:24.187 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:24.184+0000 7fa381600700 1 -- 172.21.15.153:0/3368251580 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fa37008dde0 con 0x7fa384198d20 2024-03-21T10:11:24.187 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:24.185+0000 7fa388e00700 1 --2- 172.21.15.153:0/3368251580 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7fa36c06e0e0 0x7fa36c0705a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:11:24.187 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:24.186+0000 7fa388e00700 1 --2- 172.21.15.153:0/3368251580 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7fa36c06e0e0 0x7fa36c0705a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa378000c00 tx=0x7fa37800b540 comp rx=0 tx=0).ready entity=mgr.14666 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:11:24.192 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:24.190+0000 7fa381600700 1 -- 172.21.15.153:0/3368251580 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185454 (secure 0 0 0) 0x7fa37005afd0 con 0x7fa384198d20 2024-03-21T10:11:24.550 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:24.547+0000 7fa38b800700 1 -- 172.21.15.153:0/3368251580 --> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa3840611d0 con 0x7fa36c06e0e0 2024-03-21T10:11:24.551 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:24.549+0000 7fa381600700 1 -- 172.21.15.153:0/3368251580 <== mgr.14666 v2:172.21.15.153:6800/2524145044 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+369 (secure 0 0 0) 0x7fa3840611d0 con 0x7fa36c06e0e0 2024-03-21T10:11:24.554 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:24.552+0000 7fa38b800700 1 -- 172.21.15.153:0/3368251580 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7fa36c06e0e0 msgr2=0x7fa36c0705a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:11:24.555 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:24.552+0000 7fa38b800700 1 --2- 172.21.15.153:0/3368251580 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7fa36c06e0e0 0x7fa36c0705a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa378000c00 tx=0x7fa37800b540 comp rx=0 tx=0).stop 2024-03-21T10:11:24.555 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:24.552+0000 7fa38b800700 1 -- 172.21.15.153:0/3368251580 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa384198d20 msgr2=0x7fa38419dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:11:24.555 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:24.552+0000 7fa38b800700 1 --2- 172.21.15.153:0/3368251580 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa384198d20 0x7fa38419dd90 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fa37000d8d0 tx=0x7fa37000dc90 comp rx=0 tx=0).stop 2024-03-21T10:11:24.555 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:24.553+0000 7fa38b800700 1 -- 172.21.15.153:0/3368251580 shutdown_connections 2024-03-21T10:11:24.555 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:24.553+0000 7fa38b800700 1 --2- 172.21.15.153:0/3368251580 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7fa36c06e0e0 0x7fa36c0705a0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:11:24.555 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:24.553+0000 7fa38b800700 1 --2- 172.21.15.153:0/3368251580 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa384101db0 0x7fa3841987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:11:24.555 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:24.553+0000 7fa38b800700 1 --2- 172.21.15.153:0/3368251580 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa384198d20 0x7fa38419dd90 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:11:24.555 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:24.553+0000 7fa38b800700 1 -- 172.21.15.153:0/3368251580 >> 172.21.15.153:0/3368251580 conn(0x7fa3840fc110 msgr2=0x7fa384102fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:11:24.555 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:24.553+0000 7fa38b800700 1 -- 172.21.15.153:0/3368251580 shutdown_connections 2024-03-21T10:11:24.555 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:24.553+0000 7fa38b800700 1 -- 172.21.15.153:0/3368251580 wait complete. 2024-03-21T10:11:24.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:24 smithi174 ceph-mon[37496]: pgmap v88: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-21T10:11:24.709 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:24 smithi153 ceph-mon[29117]: pgmap v88: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-21T10:11:24.715 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:24.714+0000 7f39b4800700 1 -- 172.21.15.153:0/1601004028 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f39ac0ff490 msgr2=0x7f39ac0ff8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:11:24.716 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:24.714+0000 7f39b4800700 1 --2- 172.21.15.153:0/1601004028 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f39ac0ff490 0x7f39ac0ff8b0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f39a0009910 tx=0x7f39a0009c20 comp rx=0 tx=0).stop 2024-03-21T10:11:24.716 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:24.714+0000 7f39b4800700 1 -- 172.21.15.153:0/1601004028 shutdown_connections 2024-03-21T10:11:24.716 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:24.714+0000 7f39b4800700 1 --2- 172.21.15.153:0/1601004028 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f39ac0ffdf0 0x7f39ac100270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:11:24.716 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:24.714+0000 7f39b4800700 1 --2- 172.21.15.153:0/1601004028 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f39ac0ff490 0x7f39ac0ff8b0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:11:24.716 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:24.714+0000 7f39b4800700 1 -- 172.21.15.153:0/1601004028 >> 172.21.15.153:0/1601004028 conn(0x7f39ac0fb030 msgr2=0x7f39ac0fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:11:24.716 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:24.714+0000 7f39b4800700 1 -- 172.21.15.153:0/1601004028 shutdown_connections 2024-03-21T10:11:24.716 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:24.714+0000 7f39b4800700 1 -- 172.21.15.153:0/1601004028 wait complete. 2024-03-21T10:11:24.716 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:24.715+0000 7f39b4800700 1 Processor -- start 2024-03-21T10:11:24.716 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:24.715+0000 7f39b4800700 1 -- start start 2024-03-21T10:11:24.717 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:24.716+0000 7f39b4800700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f39ac0ff490 0x7f39ac198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:11:24.717 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:24.716+0000 7f39b4800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f39ac0ffdf0 0x7f39ac198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:11:24.717 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:24.716+0000 7f39b4800700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39ac199080 con 0x7f39ac0ffdf0 2024-03-21T10:11:24.717 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:24.716+0000 7f39b4800700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39ac1991c0 con 0x7f39ac0ff490 2024-03-21T10:11:24.718 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:24.716+0000 7f39b1400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f39ac0ffdf0 0x7f39ac198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:11:24.718 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:24.716+0000 7f39b1400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f39ac0ffdf0 0x7f39ac198a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:48978/0 (socket says 172.21.15.153:48978) 2024-03-21T10:11:24.719 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:24.716+0000 7f39b1400700 1 -- 172.21.15.153:0/2460307668 learned_addr learned my addr 172.21.15.153:0/2460307668 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:11:24.719 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:24.716+0000 7f39b1e00700 1 --2- 172.21.15.153:0/2460307668 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f39ac0ff490 0x7f39ac198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:11:24.719 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:24.716+0000 7f39b1400700 1 -- 172.21.15.153:0/2460307668 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f39ac0ff490 msgr2=0x7f39ac198520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:11:24.719 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:24.716+0000 7f39b1400700 1 --2- 172.21.15.153:0/2460307668 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f39ac0ff490 0x7f39ac198520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:11:24.719 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:24.716+0000 7f39b1400700 1 -- 172.21.15.153:0/2460307668 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39a00095f0 con 0x7f39ac0ffdf0 2024-03-21T10:11:24.719 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:24.716+0000 7f39b1e00700 1 --2- 172.21.15.153:0/2460307668 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f39ac0ff490 0x7f39ac198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:11:24.719 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:24.717+0000 7f39b1400700 1 --2- 172.21.15.153:0/2460307668 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f39ac0ffdf0 0x7f39ac198a60 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f399c00d8d0 tx=0x7f399c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:11:24.719 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:24.717+0000 7f39a6a00700 1 -- 172.21.15.153:0/2460307668 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f399c009940 con 0x7f39ac0ffdf0 2024-03-21T10:11:24.719 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:24.717+0000 7f39a6a00700 1 -- 172.21.15.153:0/2460307668 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f399c010460 con 0x7f39ac0ffdf0 2024-03-21T10:11:24.719 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:24.717+0000 7f39b4800700 1 -- 172.21.15.153:0/2460307668 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39ac19dc70 con 0x7f39ac0ffdf0 2024-03-21T10:11:24.719 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:24.717+0000 7f39a6a00700 1 -- 172.21.15.153:0/2460307668 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f399c00f5f0 con 0x7f39ac0ffdf0 2024-03-21T10:11:24.719 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:24.717+0000 7f39b4800700 1 -- 172.21.15.153:0/2460307668 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39ac19e1c0 con 0x7f39ac0ffdf0 2024-03-21T10:11:24.720 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:24.718+0000 7f39b4800700 1 -- 172.21.15.153:0/2460307668 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39ac10b910 con 0x7f39ac0ffdf0 2024-03-21T10:11:24.721 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:24.720+0000 7f39a6a00700 1 -- 172.21.15.153:0/2460307668 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 31) v1 ==== 92072+0+0 (secure 0 0 0) 0x7f399c0105d0 con 0x7f39ac0ffdf0 2024-03-21T10:11:24.723 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:24.721+0000 7f39a6a00700 1 --2- 172.21.15.153:0/2460307668 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f399406e0e0 0x7f39940705a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:11:24.723 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:24.721+0000 7f39a6a00700 1 -- 172.21.15.153:0/2460307668 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f399c08dde0 con 0x7f39ac0ffdf0 2024-03-21T10:11:24.723 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:24.722+0000 7f39b1e00700 1 --2- 172.21.15.153:0/2460307668 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f399406e0e0 0x7f39940705a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:11:24.724 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:24.722+0000 7f39b1e00700 1 --2- 172.21.15.153:0/2460307668 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f399406e0e0 0x7f39940705a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f39a0006010 tx=0x7f39a000b540 comp rx=0 tx=0).ready entity=mgr.14666 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:11:24.727 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:24.726+0000 7f39a6a00700 1 -- 172.21.15.153:0/2460307668 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185454 (secure 0 0 0) 0x7f399c05afd0 con 0x7f39ac0ffdf0 2024-03-21T10:11:24.963 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:24.961+0000 7f39b4800700 1 -- 172.21.15.153:0/2460307668 --> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f39ac1091e0 con 0x7f399406e0e0 2024-03-21T10:11:24.993 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:24.991+0000 7f39a6a00700 1 -- 172.21.15.153:0/2460307668 <== mgr.14666 v2:172.21.15.153:6800/2524145044 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f39ac1091e0 con 0x7f399406e0e0 2024-03-21T10:11:24.993 INFO:teuthology.orchestra.run.smithi153.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-21T10:11:24.993 INFO:teuthology.orchestra.run.smithi153.stdout:alertmanager.smithi153 smithi153 *:9093,9094 running (15s) 6s ago 13m 13.7M - 0.25.0 c8568f914cd2 9ed3f2200b5f 2024-03-21T10:11:24.994 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi153 smithi153 running (14m) 6s ago 14m 8150k - 18.2.1 5be31c24972a 5b61994396a6 2024-03-21T10:11:24.994 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi174 smithi174 running (12m) 71s ago 12m 7940k - 18.2.1 5be31c24972a 4f7a5fc3ac28 2024-03-21T10:11:24.994 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi153 smithi153 running (14m) 6s ago 14m 6665k - 18.2.1 5be31c24972a 90a8b8ea03d1 2024-03-21T10:11:24.994 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi174 smithi174 running (12m) 71s ago 12m 6665k - 18.2.1 5be31c24972a a4c929e4dc77 2024-03-21T10:11:24.994 INFO:teuthology.orchestra.run.smithi153.stdout:grafana.smithi153 smithi153 *:3000 running (11m) 6s ago 13m 74.2M - 9.4.7 954c08fa6188 3af5800ecfef 2024-03-21T10:11:24.994 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.iuyluj smithi153 running (6m) 6s ago 6m 129M - 18.2.1 5be31c24972a 02f667cee979 2024-03-21T10:11:24.994 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.zipckr smithi153 running (6m) 6s ago 6m 58.6M - 18.2.1 5be31c24972a 68a853929f8f 2024-03-21T10:11:24.994 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.nhozag smithi174 running (6m) 71s ago 6m 128M - 18.2.1 5be31c24972a 2fee4000308f 2024-03-21T10:11:24.994 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.yyimst smithi174 running (6m) 71s ago 6m 155M - 18.2.1 5be31c24972a d78a729220d1 2024-03-21T10:11:24.994 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi153.tluxni smithi153 *:8443,9283,8765 running (3m) 6s ago 15m 504M - 19.0.0-2231-g2edfee68 237e43c7a65f 7ffee0e55382 2024-03-21T10:11:24.994 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi174.qpcwat smithi174 *:8443,9283,8765 running (110s) 71s ago 12m 447M - 19.0.0-2231-g2edfee68 237e43c7a65f 748968f216ab 2024-03-21T10:11:24.994 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi153 smithi153 running (15m) 6s ago 16m 56.4M 2048M 18.2.1 5be31c24972a 813963a95d69 2024-03-21T10:11:24.994 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi174 smithi174 running (12m) 71s ago 12m 45.3M 2048M 18.2.1 5be31c24972a 99204923eec6 2024-03-21T10:11:24.994 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi153 smithi153 *:9100 running (83s) 6s ago 14m 7524k - 1.5.0 0da6a335fe13 bb835edc0e0b 2024-03-21T10:11:24.995 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi174 smithi174 *:9100 running (76s) 71s ago 12m 6765k - 1.5.0 0da6a335fe13 7c4ecad790fa 2024-03-21T10:11:24.995 INFO:teuthology.orchestra.run.smithi153.stdout:osd.0 smithi153 running (11m) 6s ago 11m 215M 1217M 18.2.1 5be31c24972a 28912a2319b4 2024-03-21T10:11:24.995 INFO:teuthology.orchestra.run.smithi153.stdout:osd.1 smithi153 running (10m) 6s ago 10m 213M 1217M 18.2.1 5be31c24972a ca60681e2314 2024-03-21T10:11:24.995 INFO:teuthology.orchestra.run.smithi153.stdout:osd.2 smithi153 running (10m) 6s ago 10m 171M 1217M 18.2.1 5be31c24972a 8db799f8962a 2024-03-21T10:11:24.995 INFO:teuthology.orchestra.run.smithi153.stdout:osd.3 smithi174 running (9m) 71s ago 9m 213M 2241M 18.2.1 5be31c24972a dafc89e0658d 2024-03-21T10:11:24.995 INFO:teuthology.orchestra.run.smithi153.stdout:osd.4 smithi174 running (9m) 71s ago 9m 201M 2241M 18.2.1 5be31c24972a c4cd15701f05 2024-03-21T10:11:24.995 INFO:teuthology.orchestra.run.smithi153.stdout:osd.5 smithi174 running (8m) 71s ago 8m 187M 2241M 18.2.1 5be31c24972a 3b69886ce22e 2024-03-21T10:11:24.995 INFO:teuthology.orchestra.run.smithi153.stdout:prometheus.smithi153 smithi153 *:9095 running (40s) 6s ago 13m 57.8M - 2.43.0 a07b618ecd1d 7d95561e50c2 2024-03-21T10:11:24.997 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:24.995+0000 7f39b4800700 1 -- 172.21.15.153:0/2460307668 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f399406e0e0 msgr2=0x7f39940705a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:11:24.997 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:24.995+0000 7f39b4800700 1 --2- 172.21.15.153:0/2460307668 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f399406e0e0 0x7f39940705a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f39a0006010 tx=0x7f39a000b540 comp rx=0 tx=0).stop 2024-03-21T10:11:24.997 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:24.995+0000 7f39b4800700 1 -- 172.21.15.153:0/2460307668 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f39ac0ffdf0 msgr2=0x7f39ac198a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:11:24.997 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:24.995+0000 7f39b4800700 1 --2- 172.21.15.153:0/2460307668 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f39ac0ffdf0 0x7f39ac198a60 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f399c00d8d0 tx=0x7f399c00dc90 comp rx=0 tx=0).stop 2024-03-21T10:11:24.997 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:24.995+0000 7f39b4800700 1 -- 172.21.15.153:0/2460307668 shutdown_connections 2024-03-21T10:11:24.997 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:24.995+0000 7f39b4800700 1 --2- 172.21.15.153:0/2460307668 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f399406e0e0 0x7f39940705a0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:11:24.997 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:24.995+0000 7f39b4800700 1 --2- 172.21.15.153:0/2460307668 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f39ac0ff490 0x7f39ac198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:11:24.997 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:24.995+0000 7f39b4800700 1 --2- 172.21.15.153:0/2460307668 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f39ac0ffdf0 0x7f39ac198a60 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:11:24.997 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:24.995+0000 7f39b4800700 1 -- 172.21.15.153:0/2460307668 >> 172.21.15.153:0/2460307668 conn(0x7f39ac0fb030 msgr2=0x7f39ac107ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:11:24.997 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:24.995+0000 7f39b4800700 1 -- 172.21.15.153:0/2460307668 shutdown_connections 2024-03-21T10:11:24.997 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:24.995+0000 7f39b4800700 1 -- 172.21.15.153:0/2460307668 wait complete. 2024-03-21T10:11:25.153 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:25.151+0000 7f3638400700 1 -- 172.21.15.153:0/1610321276 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3630101db0 msgr2=0x7f3630102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:11:25.153 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:25.151+0000 7f3638400700 1 --2- 172.21.15.153:0/1610321276 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3630101db0 0x7f3630102210 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f361c009910 tx=0x7f361c009c20 comp rx=0 tx=0).stop 2024-03-21T10:11:25.153 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:25.151+0000 7f3638400700 1 -- 172.21.15.153:0/1610321276 shutdown_connections 2024-03-21T10:11:25.153 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:25.151+0000 7f3638400700 1 --2- 172.21.15.153:0/1610321276 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3630101db0 0x7f3630102210 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:11:25.153 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:25.151+0000 7f3638400700 1 --2- 172.21.15.153:0/1610321276 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3630100bb0 0x7f3630100fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:11:25.153 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:25.151+0000 7f3638400700 1 -- 172.21.15.153:0/1610321276 >> 172.21.15.153:0/1610321276 conn(0x7f36300fc110 msgr2=0x7f36300fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:11:25.153 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:25.151+0000 7f3638400700 1 -- 172.21.15.153:0/1610321276 shutdown_connections 2024-03-21T10:11:25.153 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:25.151+0000 7f3638400700 1 -- 172.21.15.153:0/1610321276 wait complete. 2024-03-21T10:11:25.154 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:25.152+0000 7f3638400700 1 Processor -- start 2024-03-21T10:11:25.154 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:25.152+0000 7f3638400700 1 -- start start 2024-03-21T10:11:25.154 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:25.152+0000 7f3638400700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3630100bb0 0x7f3630198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:11:25.155 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:25.153+0000 7f3638400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3630101db0 0x7f3630198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:11:25.155 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:25.153+0000 7f3638400700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36301990b0 con 0x7f3630101db0 2024-03-21T10:11:25.155 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:25.153+0000 7f3638400700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36301991f0 con 0x7f3630100bb0 2024-03-21T10:11:25.155 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:25.153+0000 7f3635000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3630101db0 0x7f3630198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:11:25.155 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:25.153+0000 7f3635000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3630101db0 0x7f3630198a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:48982/0 (socket says 172.21.15.153:48982) 2024-03-21T10:11:25.155 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:25.153+0000 7f3635000700 1 -- 172.21.15.153:0/2171154332 learned_addr learned my addr 172.21.15.153:0/2171154332 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:11:25.156 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:25.153+0000 7f3635a00700 1 --2- 172.21.15.153:0/2171154332 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3630100bb0 0x7f3630198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:11:25.156 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:25.153+0000 7f3635000700 1 -- 172.21.15.153:0/2171154332 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3630100bb0 msgr2=0x7f3630198550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:11:25.156 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:25.153+0000 7f3635000700 1 --2- 172.21.15.153:0/2171154332 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3630100bb0 0x7f3630198550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:11:25.156 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:25.153+0000 7f3635000700 1 -- 172.21.15.153:0/2171154332 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f361c0095f0 con 0x7f3630101db0 2024-03-21T10:11:25.156 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:25.153+0000 7f3635a00700 1 --2- 172.21.15.153:0/2171154332 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3630100bb0 0x7f3630198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:11:25.156 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:25.154+0000 7f3635000700 1 --2- 172.21.15.153:0/2171154332 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3630101db0 0x7f3630198a90 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f361c005b40 tx=0x7f361c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:11:25.156 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:25.154+0000 7f362e000700 1 -- 172.21.15.153:0/2171154332 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f361c01d070 con 0x7f3630101db0 2024-03-21T10:11:25.156 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:25.154+0000 7f362e000700 1 -- 172.21.15.153:0/2171154332 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f361c022470 con 0x7f3630101db0 2024-03-21T10:11:25.156 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:25.154+0000 7f362e000700 1 -- 172.21.15.153:0/2171154332 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f361c00f690 con 0x7f3630101db0 2024-03-21T10:11:25.156 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:25.154+0000 7f3638400700 1 -- 172.21.15.153:0/2171154332 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f363019dc40 con 0x7f3630101db0 2024-03-21T10:11:25.156 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:25.154+0000 7f3638400700 1 -- 172.21.15.153:0/2171154332 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f363019e0e0 con 0x7f3630101db0 2024-03-21T10:11:25.157 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:25.155+0000 7f3638400700 1 -- 172.21.15.153:0/2171154332 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3630105e90 con 0x7f3630101db0 2024-03-21T10:11:25.159 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:25.158+0000 7f362e000700 1 -- 172.21.15.153:0/2171154332 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 31) v1 ==== 92072+0+0 (secure 0 0 0) 0x7f361c0225e0 con 0x7f3630101db0 2024-03-21T10:11:25.160 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:25.159+0000 7f362e000700 1 --2- 172.21.15.153:0/2171154332 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f361806e0e0 0x7f36180705a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:11:25.160 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:25.159+0000 7f362e000700 1 -- 172.21.15.153:0/2171154332 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f361c08fbd0 con 0x7f3630101db0 2024-03-21T10:11:25.160 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:25.159+0000 7f3635a00700 1 --2- 172.21.15.153:0/2171154332 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f361806e0e0 0x7f36180705a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:11:25.161 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:25.160+0000 7f3635a00700 1 --2- 172.21.15.153:0/2171154332 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f361806e0e0 0x7f36180705a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3624005fd0 tx=0x7f3624005dc0 comp rx=0 tx=0).ready entity=mgr.14666 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:11:25.163 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:25.162+0000 7f362e000700 1 -- 172.21.15.153:0/2171154332 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185454 (secure 0 0 0) 0x7f361c08fe60 con 0x7f3630101db0 2024-03-21T10:11:25.481 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:25.479+0000 7f3638400700 1 -- 172.21.15.153:0/2171154332 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f363002cc70 con 0x7f3630101db0 2024-03-21T10:11:25.482 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:25 smithi153 ceph-mon[29117]: from='client.14792 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:11:25.482 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:25 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:11:25.482 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:25 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:11:25.482 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:25 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:11:25.482 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:25 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:11:25.482 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:25 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:11:25.482 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:25.480+0000 7f362e000700 1 -- 172.21.15.153:0/2171154332 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7f361c0312c0 con 0x7f3630101db0 2024-03-21T10:11:25.483 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:11:25.483 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": { 2024-03-21T10:11:25.483 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-03-21T10:11:25.483 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:11:25.483 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": { 2024-03-21T10:11:25.483 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:11:25.483 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:11:25.483 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": { 2024-03-21T10:11:25.483 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-03-21T10:11:25.483 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:11:25.483 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": { 2024-03-21T10:11:25.483 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-03-21T10:11:25.483 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:11:25.484 INFO:teuthology.orchestra.run.smithi153.stdout: "overall": { 2024-03-21T10:11:25.484 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 12, 2024-03-21T10:11:25.484 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:11:25.484 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-03-21T10:11:25.484 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:11:25.485 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:25.483+0000 7f3638400700 1 -- 172.21.15.153:0/2171154332 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f361806e0e0 msgr2=0x7f36180705a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:11:25.485 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:25.483+0000 7f3638400700 1 --2- 172.21.15.153:0/2171154332 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f361806e0e0 0x7f36180705a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3624005fd0 tx=0x7f3624005dc0 comp rx=0 tx=0).stop 2024-03-21T10:11:25.485 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:25.483+0000 7f3638400700 1 -- 172.21.15.153:0/2171154332 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3630101db0 msgr2=0x7f3630198a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:11:25.485 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:25.483+0000 7f3638400700 1 --2- 172.21.15.153:0/2171154332 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3630101db0 0x7f3630198a90 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f361c005b40 tx=0x7f361c00be20 comp rx=0 tx=0).stop 2024-03-21T10:11:25.485 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:25.483+0000 7f3638400700 1 -- 172.21.15.153:0/2171154332 shutdown_connections 2024-03-21T10:11:25.485 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:25.483+0000 7f3638400700 1 --2- 172.21.15.153:0/2171154332 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f361806e0e0 0x7f36180705a0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:11:25.485 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:25.483+0000 7f3638400700 1 --2- 172.21.15.153:0/2171154332 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3630100bb0 0x7f3630198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:11:25.485 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:25.483+0000 7f3638400700 1 --2- 172.21.15.153:0/2171154332 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3630101db0 0x7f3630198a90 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:11:25.486 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:25.483+0000 7f3638400700 1 -- 172.21.15.153:0/2171154332 >> 172.21.15.153:0/2171154332 conn(0x7f36300fc110 msgr2=0x7f3630102fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:11:25.486 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:25.483+0000 7f3638400700 1 -- 172.21.15.153:0/2171154332 shutdown_connections 2024-03-21T10:11:25.486 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:25.483+0000 7f3638400700 1 -- 172.21.15.153:0/2171154332 wait complete. 2024-03-21T10:11:25.639 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:25.637+0000 7fa216600700 1 -- 172.21.15.153:0/3995380487 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa210102df0 msgr2=0x7fa210103210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:11:25.639 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:25.637+0000 7fa216600700 1 --2- 172.21.15.153:0/3995380487 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa210102df0 0x7fa210103210 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fa200009910 tx=0x7fa200009c20 comp rx=0 tx=0).stop 2024-03-21T10:11:25.639 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:25.637+0000 7fa216600700 1 -- 172.21.15.153:0/3995380487 shutdown_connections 2024-03-21T10:11:25.639 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:25.637+0000 7fa216600700 1 --2- 172.21.15.153:0/3995380487 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa210103ff0 0x7fa210104450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:11:25.639 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:25.637+0000 7fa216600700 1 --2- 172.21.15.153:0/3995380487 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa210102df0 0x7fa210103210 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:11:25.639 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:25.637+0000 7fa216600700 1 -- 172.21.15.153:0/3995380487 >> 172.21.15.153:0/3995380487 conn(0x7fa2100fe390 msgr2=0x7fa2101007d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:11:25.639 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:25.637+0000 7fa216600700 1 -- 172.21.15.153:0/3995380487 shutdown_connections 2024-03-21T10:11:25.640 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:25.638+0000 7fa216600700 1 -- 172.21.15.153:0/3995380487 wait complete. 2024-03-21T10:11:25.640 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:25.639+0000 7fa216600700 1 Processor -- start 2024-03-21T10:11:25.640 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:25.639+0000 7fa216600700 1 -- start start 2024-03-21T10:11:25.640 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:25.639+0000 7fa216600700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa210102df0 0x7fa210198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:11:25.641 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:25.639+0000 7fa216600700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa210103ff0 0x7fa210198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:11:25.641 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:25.639+0000 7fa216600700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2101991a0 con 0x7fa210103ff0 2024-03-21T10:11:25.641 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:25.639+0000 7fa216600700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2101992e0 con 0x7fa210102df0 2024-03-21T10:11:25.642 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:25.640+0000 7fa20ea00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa210103ff0 0x7fa210198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:11:25.642 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:25.640+0000 7fa20ea00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa210103ff0 0x7fa210198b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:49002/0 (socket says 172.21.15.153:49002) 2024-03-21T10:11:25.642 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:25.640+0000 7fa20ea00700 1 -- 172.21.15.153:0/1942544559 learned_addr learned my addr 172.21.15.153:0/1942544559 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:11:25.642 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:25.640+0000 7fa20f400700 1 --2- 172.21.15.153:0/1942544559 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa210102df0 0x7fa210198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:11:25.642 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:25.640+0000 7fa20ea00700 1 -- 172.21.15.153:0/1942544559 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa210102df0 msgr2=0x7fa210198640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:11:25.642 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:25.640+0000 7fa20ea00700 1 --2- 172.21.15.153:0/1942544559 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa210102df0 0x7fa210198640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:11:25.642 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:25.640+0000 7fa20ea00700 1 -- 172.21.15.153:0/1942544559 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2000095f0 con 0x7fa210103ff0 2024-03-21T10:11:25.643 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:25.640+0000 7fa20f400700 1 --2- 172.21.15.153:0/1942544559 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa210102df0 0x7fa210198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:11:25.643 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:25.640+0000 7fa20ea00700 1 --2- 172.21.15.153:0/1942544559 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa210103ff0 0x7fa210198b80 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc00b4b0 tx=0x7fa1fc00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:11:25.643 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:25.641+0000 7fa207e00700 1 -- 172.21.15.153:0/1942544559 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa1fc011800 con 0x7fa210103ff0 2024-03-21T10:11:25.643 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:25.641+0000 7fa207e00700 1 -- 172.21.15.153:0/1942544559 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa1fc011e10 con 0x7fa210103ff0 2024-03-21T10:11:25.643 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:25.641+0000 7fa207e00700 1 -- 172.21.15.153:0/1942544559 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa1fc00f2e0 con 0x7fa210103ff0 2024-03-21T10:11:25.643 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:25.641+0000 7fa216600700 1 -- 172.21.15.153:0/1942544559 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa21019dd90 con 0x7fa210103ff0 2024-03-21T10:11:25.643 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:25.641+0000 7fa216600700 1 -- 172.21.15.153:0/1942544559 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa21019e330 con 0x7fa210103ff0 2024-03-21T10:11:25.644 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:25.642+0000 7fa216600700 1 -- 172.21.15.153:0/1942544559 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa21010b9d0 con 0x7fa210103ff0 2024-03-21T10:11:25.645 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:25.644+0000 7fa207e00700 1 -- 172.21.15.153:0/1942544559 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 31) v1 ==== 92072+0+0 (secure 0 0 0) 0x7fa1fc010380 con 0x7fa210103ff0 2024-03-21T10:11:25.647 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:25.645+0000 7fa207e00700 1 --2- 172.21.15.153:0/1942544559 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7fa1f406e0e0 0x7fa1f40705a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:11:25.647 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:25.646+0000 7fa207e00700 1 -- 172.21.15.153:0/1942544559 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fa1fc08d960 con 0x7fa210103ff0 2024-03-21T10:11:25.647 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:25.646+0000 7fa20f400700 1 --2- 172.21.15.153:0/1942544559 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7fa1f406e0e0 0x7fa1f40705a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:11:25.648 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:25.647+0000 7fa20f400700 1 --2- 172.21.15.153:0/1942544559 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7fa1f406e0e0 0x7fa1f40705a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa200006010 tx=0x7fa20000b540 comp rx=0 tx=0).ready entity=mgr.14666 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:11:25.651 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:25.650+0000 7fa207e00700 1 -- 172.21.15.153:0/1942544559 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185454 (secure 0 0 0) 0x7fa1fc05abf0 con 0x7fa210103ff0 2024-03-21T10:11:25.913 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:25.912+0000 7fa216600700 1 -- 172.21.15.153:0/1942544559 --> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa210108940 con 0x7fa1f406e0e0 2024-03-21T10:11:25.914 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:25.914+0000 7fa207e00700 1 -- 172.21.15.153:0/1942544559 <== mgr.14666 v2:172.21.15.153:6800/2524145044 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+364 (secure 0 0 0) 0x7fa210108940 con 0x7fa1f406e0e0 2024-03-21T10:11:25.914 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:11:25.915 INFO:teuthology.orchestra.run.smithi153.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232", 2024-03-21T10:11:25.915 INFO:teuthology.orchestra.run.smithi153.stdout: "in_progress": true, 2024-03-21T10:11:25.915 INFO:teuthology.orchestra.run.smithi153.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-03-21T10:11:25.915 INFO:teuthology.orchestra.run.smithi153.stdout: "services_complete": [ 2024-03-21T10:11:25.915 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr" 2024-03-21T10:11:25.915 INFO:teuthology.orchestra.run.smithi153.stdout: ], 2024-03-21T10:11:25.915 INFO:teuthology.orchestra.run.smithi153.stdout: "progress": "2/2 daemons upgraded", 2024-03-21T10:11:25.915 INFO:teuthology.orchestra.run.smithi153.stdout: "message": "Currently upgrading grafana daemons", 2024-03-21T10:11:25.915 INFO:teuthology.orchestra.run.smithi153.stdout: "is_paused": false 2024-03-21T10:11:25.915 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:11:25.917 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:25.916+0000 7fa216600700 1 -- 172.21.15.153:0/1942544559 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7fa1f406e0e0 msgr2=0x7fa1f40705a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:11:25.918 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:25.916+0000 7fa216600700 1 --2- 172.21.15.153:0/1942544559 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7fa1f406e0e0 0x7fa1f40705a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa200006010 tx=0x7fa20000b540 comp rx=0 tx=0).stop 2024-03-21T10:11:25.918 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:25.916+0000 7fa216600700 1 -- 172.21.15.153:0/1942544559 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa210103ff0 msgr2=0x7fa210198b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:11:25.918 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:25.916+0000 7fa216600700 1 --2- 172.21.15.153:0/1942544559 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa210103ff0 0x7fa210198b80 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc00b4b0 tx=0x7fa1fc00b870 comp rx=0 tx=0).stop 2024-03-21T10:11:25.918 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:25.917+0000 7fa216600700 1 -- 172.21.15.153:0/1942544559 shutdown_connections 2024-03-21T10:11:25.918 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:25.917+0000 7fa216600700 1 --2- 172.21.15.153:0/1942544559 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7fa1f406e0e0 0x7fa1f40705a0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:11:25.918 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:25.917+0000 7fa216600700 1 --2- 172.21.15.153:0/1942544559 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa210102df0 0x7fa210198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:11:25.918 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:25.917+0000 7fa216600700 1 --2- 172.21.15.153:0/1942544559 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa210103ff0 0x7fa210198b80 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:11:25.918 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:25.917+0000 7fa216600700 1 -- 172.21.15.153:0/1942544559 >> 172.21.15.153:0/1942544559 conn(0x7fa2100fe390 msgr2=0x7fa210107220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:11:25.918 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:25.917+0000 7fa216600700 1 -- 172.21.15.153:0/1942544559 shutdown_connections 2024-03-21T10:11:25.918 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:25.917+0000 7fa216600700 1 -- 172.21.15.153:0/1942544559 wait complete. 2024-03-21T10:11:25.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:25 smithi174 ceph-mon[37496]: from='client.14792 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:11:25.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:25 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:11:25.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:25 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:11:25.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:25 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:11:25.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:25 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:11:25.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:25 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:11:26.734 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:26 smithi153 ceph-mon[29117]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:11:26.734 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:26 smithi153 ceph-mon[29117]: from='client.14800 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:11:26.735 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:26 smithi153 ceph-mon[29117]: pgmap v89: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-21T10:11:26.735 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:26 smithi153 ceph-mon[29117]: from='client.? 172.21.15.153:0/2171154332' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:11:26.735 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:26 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-21T10:11:26.735 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:26 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:11:26.735 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:26 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:11:26.735 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:26 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:11:26.735 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:26 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:11:26.735 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:26 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:11:26.735 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:26 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:11:26.735 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:26 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:11:26.735 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:26 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:11:26.735 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:26 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:11:26.735 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:26 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:11:26.735 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:26 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:11:26.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:26 smithi174 ceph-mon[37496]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:11:26.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:26 smithi174 ceph-mon[37496]: from='client.14800 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:11:26.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:26 smithi174 ceph-mon[37496]: pgmap v89: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-21T10:11:26.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:26 smithi174 ceph-mon[37496]: from='client.? 172.21.15.153:0/2171154332' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:11:26.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:26 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-21T10:11:26.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:26 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:11:26.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:26 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:11:26.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:26 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:11:26.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:26 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:11:26.953 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:26 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:11:26.953 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:26 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:11:26.953 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:26 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:11:26.953 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:26 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:11:26.953 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:26 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:11:26.953 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:26 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:11:26.953 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:26 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:11:27.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:27 smithi153 ceph-mon[29117]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-21T10:11:27.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:27 smithi153 ceph-mon[29117]: from='client.14808 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:11:27.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:27 smithi174 ceph-mon[37496]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-21T10:11:27.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:27 smithi174 ceph-mon[37496]: from='client.14808 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:11:28.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:28 smithi153 ceph-mon[29117]: pgmap v90: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-21T10:11:28.853 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:28 smithi174 ceph-mon[37496]: pgmap v90: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-21T10:11:29.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:29 smithi153 ceph-mon[29117]: Upgrade: Updating grafana.smithi153 2024-03-21T10:11:29.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:29 smithi153 ceph-mon[29117]: Deploying daemon grafana.smithi153 on smithi153 2024-03-21T10:11:29.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:29 smithi174 ceph-mon[37496]: Upgrade: Updating grafana.smithi153 2024-03-21T10:11:29.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:29 smithi174 ceph-mon[37496]: Deploying daemon grafana.smithi153 on smithi153 2024-03-21T10:11:30.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:30 smithi153 ceph-mon[29117]: pgmap v91: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-21T10:11:30.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:30 smithi174 ceph-mon[37496]: pgmap v91: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-21T10:11:32.649 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:32 smithi153 ceph-mon[29117]: pgmap v92: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-21T10:11:32.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:32 smithi174 ceph-mon[37496]: pgmap v92: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-21T10:11:33.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:33 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:11:33.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:33 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:11:34.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:34 smithi153 ceph-mon[29117]: pgmap v93: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:11:34.955 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:34 smithi174 ceph-mon[37496]: pgmap v93: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:11:36.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:36 smithi153 ceph-mon[29117]: pgmap v94: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:11:36.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:36 smithi174 ceph-mon[37496]: pgmap v94: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:11:37.779 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:37 smithi153 ceph-mon[29117]: fsmap cephfs:2 {0=cephfs.smithi153.iuyluj=up:active,1=cephfs.smithi174.yyimst=up:active} 2 up:standby-replay 2024-03-21T10:11:37.779 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:37 smithi153 ceph-mon[29117]: pgmap v95: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:11:37.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:37 smithi174 ceph-mon[37496]: fsmap cephfs:2 {0=cephfs.smithi153.iuyluj=up:active,1=cephfs.smithi174.yyimst=up:active} 2 up:standby-replay 2024-03-21T10:11:37.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:37 smithi174 ceph-mon[37496]: pgmap v95: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:11:40.540 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:40 smithi153 ceph-mon[29117]: pgmap v96: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:11:40.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:40 smithi174 ceph-mon[37496]: pgmap v96: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:11:42.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:42 smithi153 ceph-mon[29117]: pgmap v97: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:11:42.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:42 smithi174 ceph-mon[37496]: pgmap v97: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:11:44.143 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:43 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:11:44.143 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:43 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:11:44.143 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:43 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:11:44.143 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:43 smithi153 ceph-mon[29117]: pgmap v98: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:11:44.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:43 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:11:44.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:43 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:11:44.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:43 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:11:44.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:43 smithi174 ceph-mon[37496]: pgmap v98: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:11:46.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:46 smithi153 ceph-mon[29117]: pgmap v99: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:11:46.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:46 smithi174 ceph-mon[37496]: pgmap v99: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:11:48.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:48 smithi153 ceph-mon[29117]: pgmap v100: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:11:48.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:48 smithi174 ceph-mon[37496]: pgmap v100: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:11:49.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:49 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:11:49.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:49 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:11:50.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:50 smithi153 ceph-mon[29117]: pgmap v101: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:11:50.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:50 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:11:50.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:50 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:11:50.634 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:50 smithi174 ceph-mon[37496]: pgmap v101: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:11:50.634 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:50 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:11:50.634 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:50 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:11:52.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:52 smithi153 ceph-mon[29117]: pgmap v102: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:11:52.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:52 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:11:52.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:52 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:11:52.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:52 smithi174 ceph-mon[37496]: pgmap v102: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:11:52.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:52 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:11:52.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:52 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:11:54.472 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:54 smithi153 ceph-mon[29117]: pgmap v103: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:11:54.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:54 smithi174 ceph-mon[37496]: pgmap v103: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:11:56.118 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.114+0000 7f62b2400700 1 -- 172.21.15.153:0/4068461870 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62ac074d40 msgr2=0x7f62ac0731a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:11:56.118 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.114+0000 7f62b2400700 1 --2- 172.21.15.153:0/4068461870 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62ac074d40 0x7f62ac0731a0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f629c009910 tx=0x7f629c009c20 comp rx=0 tx=0).stop 2024-03-21T10:11:56.118 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.114+0000 7f62b2400700 1 -- 172.21.15.153:0/4068461870 shutdown_connections 2024-03-21T10:11:56.118 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.114+0000 7f62b2400700 1 --2- 172.21.15.153:0/4068461870 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f62ac073770 0x7f62ac073bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:11:56.118 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.114+0000 7f62b2400700 1 --2- 172.21.15.153:0/4068461870 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62ac074d40 0x7f62ac0731a0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:11:56.118 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.114+0000 7f62b2400700 1 -- 172.21.15.153:0/4068461870 >> 172.21.15.153:0/4068461870 conn(0x7f62ac0fc110 msgr2=0x7f62ac0fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:11:56.118 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.114+0000 7f62b2400700 1 -- 172.21.15.153:0/4068461870 shutdown_connections 2024-03-21T10:11:56.118 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.114+0000 7f62b2400700 1 -- 172.21.15.153:0/4068461870 wait complete. 2024-03-21T10:11:56.118 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.115+0000 7f62b2400700 1 Processor -- start 2024-03-21T10:11:56.118 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.115+0000 7f62b2400700 1 -- start start 2024-03-21T10:11:56.118 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.116+0000 7f62b2400700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f62ac073770 0x7f62ac198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:11:56.118 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.116+0000 7f62b2400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62ac074d40 0x7f62ac198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:11:56.119 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.116+0000 7f62b2400700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62ac1990b0 con 0x7f62ac074d40 2024-03-21T10:11:56.119 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.116+0000 7f62b2400700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62ac1991f0 con 0x7f62ac073770 2024-03-21T10:11:56.119 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.116+0000 7f62aaa00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62ac074d40 0x7f62ac198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:11:56.119 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.116+0000 7f62aaa00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62ac074d40 0x7f62ac198a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:50184/0 (socket says 172.21.15.153:50184) 2024-03-21T10:11:56.119 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.116+0000 7f62aaa00700 1 -- 172.21.15.153:0/3882433219 learned_addr learned my addr 172.21.15.153:0/3882433219 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:11:56.120 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.116+0000 7f62ab400700 1 --2- 172.21.15.153:0/3882433219 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f62ac073770 0x7f62ac198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:11:56.120 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.117+0000 7f62aaa00700 1 -- 172.21.15.153:0/3882433219 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f62ac073770 msgr2=0x7f62ac198550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:11:56.120 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.117+0000 7f62aaa00700 1 --2- 172.21.15.153:0/3882433219 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f62ac073770 0x7f62ac198550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:11:56.120 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.117+0000 7f62aaa00700 1 -- 172.21.15.153:0/3882433219 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f629c0095f0 con 0x7f62ac074d40 2024-03-21T10:11:56.120 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.117+0000 7f62aaa00700 1 --2- 172.21.15.153:0/3882433219 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62ac074d40 0x7f62ac198a90 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f629800d8d0 tx=0x7f629800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:11:56.120 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.117+0000 7f62a3e00700 1 -- 172.21.15.153:0/3882433219 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f62980098a0 con 0x7f62ac074d40 2024-03-21T10:11:56.120 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.118+0000 7f62b2400700 1 -- 172.21.15.153:0/3882433219 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62ac19dca0 con 0x7f62ac074d40 2024-03-21T10:11:56.120 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.118+0000 7f62b2400700 1 -- 172.21.15.153:0/3882433219 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62ac19e1f0 con 0x7f62ac074d40 2024-03-21T10:11:56.120 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.118+0000 7f62a3e00700 1 -- 172.21.15.153:0/3882433219 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f6298010460 con 0x7f62ac074d40 2024-03-21T10:11:56.120 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.118+0000 7f62a3e00700 1 -- 172.21.15.153:0/3882433219 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f629800f5f0 con 0x7f62ac074d40 2024-03-21T10:11:56.121 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.119+0000 7f62b2400700 1 -- 172.21.15.153:0/3882433219 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f628c005320 con 0x7f62ac074d40 2024-03-21T10:11:56.121 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.120+0000 7f62a3e00700 1 -- 172.21.15.153:0/3882433219 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 31) v1 ==== 92072+0+0 (secure 0 0 0) 0x7f6298009a00 con 0x7f62ac074d40 2024-03-21T10:11:56.122 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.121+0000 7f62a3e00700 1 --2- 172.21.15.153:0/3882433219 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f629006e0e0 0x7f62900705a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:11:56.123 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.121+0000 7f62a3e00700 1 -- 172.21.15.153:0/3882433219 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f629808dbf0 con 0x7f62ac074d40 2024-03-21T10:11:56.123 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.122+0000 7f62ab400700 1 --2- 172.21.15.153:0/3882433219 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f629006e0e0 0x7f62900705a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:11:56.124 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.123+0000 7f62ab400700 1 --2- 172.21.15.153:0/3882433219 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f629006e0e0 0x7f62900705a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f629c006010 tx=0x7f629c00b560 comp rx=0 tx=0).ready entity=mgr.14666 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:11:56.128 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.127+0000 7f62a3e00700 1 -- 172.21.15.153:0/3882433219 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185454 (secure 0 0 0) 0x7f629805ade0 con 0x7f62ac074d40 2024-03-21T10:11:56.388 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:56 smithi153 ceph-mon[29117]: pgmap v104: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:11:56.391 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.389+0000 7f62b2400700 1 -- 172.21.15.153:0/3882433219 --> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f628c000bf0 con 0x7f629006e0e0 2024-03-21T10:11:56.392 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.391+0000 7f62a3e00700 1 -- 172.21.15.153:0/3882433219 <== mgr.14666 v2:172.21.15.153:6800/2524145044 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+364 (secure 0 0 0) 0x7f628c000bf0 con 0x7f629006e0e0 2024-03-21T10:11:56.395 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.393+0000 7f62b2400700 1 -- 172.21.15.153:0/3882433219 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f629006e0e0 msgr2=0x7f62900705a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:11:56.395 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.393+0000 7f62b2400700 1 --2- 172.21.15.153:0/3882433219 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f629006e0e0 0x7f62900705a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f629c006010 tx=0x7f629c00b560 comp rx=0 tx=0).stop 2024-03-21T10:11:56.395 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.393+0000 7f62b2400700 1 -- 172.21.15.153:0/3882433219 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62ac074d40 msgr2=0x7f62ac198a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:11:56.395 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.393+0000 7f62b2400700 1 --2- 172.21.15.153:0/3882433219 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62ac074d40 0x7f62ac198a90 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f629800d8d0 tx=0x7f629800dbe0 comp rx=0 tx=0).stop 2024-03-21T10:11:56.395 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.393+0000 7f62b2400700 1 -- 172.21.15.153:0/3882433219 shutdown_connections 2024-03-21T10:11:56.395 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.393+0000 7f62b2400700 1 --2- 172.21.15.153:0/3882433219 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f629006e0e0 0x7f62900705a0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:11:56.395 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.393+0000 7f62b2400700 1 --2- 172.21.15.153:0/3882433219 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f62ac073770 0x7f62ac198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:11:56.395 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.393+0000 7f62b2400700 1 --2- 172.21.15.153:0/3882433219 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62ac074d40 0x7f62ac198a90 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:11:56.395 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.393+0000 7f62b2400700 1 -- 172.21.15.153:0/3882433219 >> 172.21.15.153:0/3882433219 conn(0x7f62ac0fc110 msgr2=0x7f62ac106a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:11:56.395 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.393+0000 7f62b2400700 1 -- 172.21.15.153:0/3882433219 shutdown_connections 2024-03-21T10:11:56.396 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.394+0000 7f62b2400700 1 -- 172.21.15.153:0/3882433219 wait complete. 2024-03-21T10:11:56.411 INFO:teuthology.orchestra.run.smithi153.stdout:true 2024-03-21T10:11:56.550 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.548+0000 7efddd200700 1 -- 172.21.15.153:0/2427053146 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efdd8101c00 msgr2=0x7efdd8102060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:11:56.550 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.548+0000 7efddd200700 1 --2- 172.21.15.153:0/2427053146 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efdd8101c00 0x7efdd8102060 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7efdc0009910 tx=0x7efdc0009c20 comp rx=0 tx=0).stop 2024-03-21T10:11:56.550 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.548+0000 7efddd200700 1 -- 172.21.15.153:0/2427053146 shutdown_connections 2024-03-21T10:11:56.550 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.548+0000 7efddd200700 1 --2- 172.21.15.153:0/2427053146 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efdd8101c00 0x7efdd8102060 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:11:56.551 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.548+0000 7efddd200700 1 --2- 172.21.15.153:0/2427053146 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efdd8100a00 0x7efdd8100e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:11:56.551 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.548+0000 7efddd200700 1 -- 172.21.15.153:0/2427053146 >> 172.21.15.153:0/2427053146 conn(0x7efdd80fbf60 msgr2=0x7efdd80fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:11:56.551 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.549+0000 7efddd200700 1 -- 172.21.15.153:0/2427053146 shutdown_connections 2024-03-21T10:11:56.551 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.549+0000 7efddd200700 1 -- 172.21.15.153:0/2427053146 wait complete. 2024-03-21T10:11:56.552 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.550+0000 7efddd200700 1 Processor -- start 2024-03-21T10:11:56.552 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.550+0000 7efddd200700 1 -- start start 2024-03-21T10:11:56.552 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.550+0000 7efddd200700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efdd8100a00 0x7efdd8198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:11:56.552 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.550+0000 7efddd200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efdd8101c00 0x7efdd81988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:11:56.552 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.550+0000 7efddd200700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdd8198ed0 con 0x7efdd8101c00 2024-03-21T10:11:56.552 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.550+0000 7efddd200700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdd8199010 con 0x7efdd8100a00 2024-03-21T10:11:56.552 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.551+0000 7efdcfe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efdd8101c00 0x7efdd81988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:11:56.553 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.551+0000 7efdcfe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efdd8101c00 0x7efdd81988b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:50200/0 (socket says 172.21.15.153:50200) 2024-03-21T10:11:56.553 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.551+0000 7efdcfe00700 1 -- 172.21.15.153:0/3842545129 learned_addr learned my addr 172.21.15.153:0/3842545129 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:11:56.554 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.551+0000 7efdd7400700 1 --2- 172.21.15.153:0/3842545129 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efdd8100a00 0x7efdd8198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:11:56.554 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.551+0000 7efdcfe00700 1 -- 172.21.15.153:0/3842545129 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efdd8100a00 msgr2=0x7efdd8198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:11:56.554 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.551+0000 7efdcfe00700 1 --2- 172.21.15.153:0/3842545129 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efdd8100a00 0x7efdd8198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:11:56.554 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.551+0000 7efdcfe00700 1 -- 172.21.15.153:0/3842545129 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdc00095f0 con 0x7efdd8101c00 2024-03-21T10:11:56.554 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.551+0000 7efdd7400700 1 --2- 172.21.15.153:0/3842545129 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efdd8100a00 0x7efdd8198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:11:56.554 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.552+0000 7efdcfe00700 1 --2- 172.21.15.153:0/3842545129 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efdd8101c00 0x7efdd81988b0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7efdc000b5c0 tx=0x7efdc000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:11:56.554 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.552+0000 7efdd4c00700 1 -- 172.21.15.153:0/3842545129 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efdc001d070 con 0x7efdd8101c00 2024-03-21T10:11:56.554 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.552+0000 7efdd4c00700 1 -- 172.21.15.153:0/3842545129 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7efdc000f460 con 0x7efdd8101c00 2024-03-21T10:11:56.554 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.552+0000 7efdd4c00700 1 -- 172.21.15.153:0/3842545129 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efdc0021640 con 0x7efdd8101c00 2024-03-21T10:11:56.554 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.552+0000 7efddd200700 1 -- 172.21.15.153:0/3842545129 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdd819da60 con 0x7efdd8101c00 2024-03-21T10:11:56.554 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.552+0000 7efddd200700 1 -- 172.21.15.153:0/3842545129 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdd819df50 con 0x7efdd8101c00 2024-03-21T10:11:56.555 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.554+0000 7efddd200700 1 -- 172.21.15.153:0/3842545129 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdd8105ce0 con 0x7efdd8101c00 2024-03-21T10:11:56.556 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.555+0000 7efdd4c00700 1 -- 172.21.15.153:0/3842545129 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 31) v1 ==== 92072+0+0 (secure 0 0 0) 0x7efdc000f5d0 con 0x7efdd8101c00 2024-03-21T10:11:56.557 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.556+0000 7efdd4c00700 1 --2- 172.21.15.153:0/3842545129 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7efdbc06e0e0 0x7efdbc0705a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:11:56.557 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.556+0000 7efdd4c00700 1 -- 172.21.15.153:0/3842545129 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7efdc008f250 con 0x7efdd8101c00 2024-03-21T10:11:56.558 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.556+0000 7efdd7400700 1 --2- 172.21.15.153:0/3842545129 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7efdbc06e0e0 0x7efdbc0705a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:11:56.558 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.557+0000 7efdd7400700 1 --2- 172.21.15.153:0/3842545129 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7efdbc06e0e0 0x7efdbc0705a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7efdc8005fd0 tx=0x7efdc8005f00 comp rx=0 tx=0).ready entity=mgr.14666 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:11:56.562 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.561+0000 7efdd4c00700 1 -- 172.21.15.153:0/3842545129 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185454 (secure 0 0 0) 0x7efdc005c460 con 0x7efdd8101c00 2024-03-21T10:11:56.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:56 smithi174 ceph-mon[37496]: pgmap v104: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:11:56.805 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.803+0000 7efddd200700 1 -- 172.21.15.153:0/3842545129 --> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7efdd80611d0 con 0x7efdbc06e0e0 2024-03-21T10:11:56.807 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.805+0000 7efdd4c00700 1 -- 172.21.15.153:0/3842545129 <== mgr.14666 v2:172.21.15.153:6800/2524145044 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+364 (secure 0 0 0) 0x7efdd80611d0 con 0x7efdbc06e0e0 2024-03-21T10:11:56.810 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.808+0000 7efddd200700 1 -- 172.21.15.153:0/3842545129 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7efdbc06e0e0 msgr2=0x7efdbc0705a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:11:56.810 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.808+0000 7efddd200700 1 --2- 172.21.15.153:0/3842545129 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7efdbc06e0e0 0x7efdbc0705a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7efdc8005fd0 tx=0x7efdc8005f00 comp rx=0 tx=0).stop 2024-03-21T10:11:56.810 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.808+0000 7efddd200700 1 -- 172.21.15.153:0/3842545129 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efdd8101c00 msgr2=0x7efdd81988b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:11:56.810 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.808+0000 7efddd200700 1 --2- 172.21.15.153:0/3842545129 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efdd8101c00 0x7efdd81988b0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7efdc000b5c0 tx=0x7efdc000be20 comp rx=0 tx=0).stop 2024-03-21T10:11:56.810 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.808+0000 7efddd200700 1 -- 172.21.15.153:0/3842545129 shutdown_connections 2024-03-21T10:11:56.810 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.808+0000 7efddd200700 1 --2- 172.21.15.153:0/3842545129 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7efdbc06e0e0 0x7efdbc0705a0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:11:56.810 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.808+0000 7efddd200700 1 --2- 172.21.15.153:0/3842545129 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efdd8100a00 0x7efdd8198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:11:56.810 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.808+0000 7efddd200700 1 --2- 172.21.15.153:0/3842545129 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efdd8101c00 0x7efdd81988b0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:11:56.810 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.808+0000 7efddd200700 1 -- 172.21.15.153:0/3842545129 >> 172.21.15.153:0/3842545129 conn(0x7efdd80fbf60 msgr2=0x7efdd8102e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:11:56.810 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.808+0000 7efddd200700 1 -- 172.21.15.153:0/3842545129 shutdown_connections 2024-03-21T10:11:56.810 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.808+0000 7efddd200700 1 -- 172.21.15.153:0/3842545129 wait complete. 2024-03-21T10:11:56.953 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.951+0000 7ff523e00700 1 -- 172.21.15.153:0/3624788203 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff51c108400 msgr2=0x7ff51c108860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:11:56.953 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.951+0000 7ff523e00700 1 --2- 172.21.15.153:0/3624788203 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff51c108400 0x7ff51c108860 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7ff508009910 tx=0x7ff508009c20 comp rx=0 tx=0).stop 2024-03-21T10:11:56.953 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.951+0000 7ff523e00700 1 -- 172.21.15.153:0/3624788203 shutdown_connections 2024-03-21T10:11:56.953 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.951+0000 7ff523e00700 1 --2- 172.21.15.153:0/3624788203 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff51c108400 0x7ff51c108860 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:11:56.953 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.951+0000 7ff523e00700 1 --2- 172.21.15.153:0/3624788203 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff51c107200 0x7ff51c107620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:11:56.953 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.951+0000 7ff523e00700 1 -- 172.21.15.153:0/3624788203 >> 172.21.15.153:0/3624788203 conn(0x7ff51c0761b0 msgr2=0x7ff51c078610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:11:56.953 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.951+0000 7ff523e00700 1 -- 172.21.15.153:0/3624788203 shutdown_connections 2024-03-21T10:11:56.953 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.951+0000 7ff523e00700 1 -- 172.21.15.153:0/3624788203 wait complete. 2024-03-21T10:11:56.955 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.952+0000 7ff523e00700 1 Processor -- start 2024-03-21T10:11:56.955 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.952+0000 7ff523e00700 1 -- start start 2024-03-21T10:11:56.955 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.952+0000 7ff523e00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff51c107200 0x7ff51c19c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:11:56.955 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.952+0000 7ff523e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff51c108400 0x7ff51c19cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:11:56.955 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.952+0000 7ff523e00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff51c19d4c0 con 0x7ff51c108400 2024-03-21T10:11:56.955 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.952+0000 7ff523e00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff51c19d600 con 0x7ff51c107200 2024-03-21T10:11:56.955 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.953+0000 7ff520a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff51c108400 0x7ff51c19cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:11:56.955 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.953+0000 7ff520a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff51c108400 0x7ff51c19cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:50218/0 (socket says 172.21.15.153:50218) 2024-03-21T10:11:56.955 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.953+0000 7ff520a00700 1 -- 172.21.15.153:0/4234253489 learned_addr learned my addr 172.21.15.153:0/4234253489 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:11:56.955 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.953+0000 7ff521400700 1 --2- 172.21.15.153:0/4234253489 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff51c107200 0x7ff51c19c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:11:56.955 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.953+0000 7ff520a00700 1 -- 172.21.15.153:0/4234253489 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff51c107200 msgr2=0x7ff51c19c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:11:56.956 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.953+0000 7ff520a00700 1 --2- 172.21.15.153:0/4234253489 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff51c107200 0x7ff51c19c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:11:56.956 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.953+0000 7ff520a00700 1 -- 172.21.15.153:0/4234253489 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5080095f0 con 0x7ff51c108400 2024-03-21T10:11:56.956 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.953+0000 7ff521400700 1 --2- 172.21.15.153:0/4234253489 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff51c107200 0x7ff51c19c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:11:56.956 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.953+0000 7ff520a00700 1 --2- 172.21.15.153:0/4234253489 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff51c108400 0x7ff51c19cea0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7ff508005b40 tx=0x7ff50800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:11:56.956 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.954+0000 7ff51a000700 1 -- 172.21.15.153:0/4234253489 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff50801d070 con 0x7ff51c108400 2024-03-21T10:11:56.957 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.954+0000 7ff51a000700 1 -- 172.21.15.153:0/4234253489 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff50800f460 con 0x7ff51c108400 2024-03-21T10:11:56.957 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.954+0000 7ff51a000700 1 -- 172.21.15.153:0/4234253489 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff508021640 con 0x7ff51c108400 2024-03-21T10:11:56.957 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.954+0000 7ff523e00700 1 -- 172.21.15.153:0/4234253489 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff51c1a2050 con 0x7ff51c108400 2024-03-21T10:11:56.957 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.954+0000 7ff523e00700 1 -- 172.21.15.153:0/4234253489 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff51c1a2540 con 0x7ff51c108400 2024-03-21T10:11:56.957 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.955+0000 7ff523e00700 1 -- 172.21.15.153:0/4234253489 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff51c10fd60 con 0x7ff51c108400 2024-03-21T10:11:56.957 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.956+0000 7ff51a000700 1 -- 172.21.15.153:0/4234253489 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 31) v1 ==== 92072+0+0 (secure 0 0 0) 0x7ff50800fa90 con 0x7ff51c108400 2024-03-21T10:11:56.958 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.957+0000 7ff51a000700 1 --2- 172.21.15.153:0/4234253489 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7ff50406e0e0 0x7ff5040705a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:11:56.959 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.957+0000 7ff51a000700 1 -- 172.21.15.153:0/4234253489 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7ff50808f1b0 con 0x7ff51c108400 2024-03-21T10:11:56.959 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.957+0000 7ff521400700 1 --2- 172.21.15.153:0/4234253489 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7ff50406e0e0 0x7ff5040705a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:11:56.959 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.958+0000 7ff521400700 1 --2- 172.21.15.153:0/4234253489 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7ff50406e0e0 0x7ff5040705a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff510005fd0 tx=0x7ff510005f00 comp rx=0 tx=0).ready entity=mgr.14666 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:11:56.963 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:56.962+0000 7ff51a000700 1 -- 172.21.15.153:0/4234253489 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185454 (secure 0 0 0) 0x7ff50805c3a0 con 0x7ff51c108400 2024-03-21T10:11:57.202 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:57.200+0000 7ff523e00700 1 -- 172.21.15.153:0/4234253489 --> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7ff51c10cd50 con 0x7ff50406e0e0 2024-03-21T10:11:57.219 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:57.218+0000 7ff51a000700 1 -- 172.21.15.153:0/4234253489 <== mgr.14666 v2:172.21.15.153:6800/2524145044 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7ff51c10cd50 con 0x7ff50406e0e0 2024-03-21T10:11:57.220 INFO:teuthology.orchestra.run.smithi153.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-21T10:11:57.220 INFO:teuthology.orchestra.run.smithi153.stdout:alertmanager.smithi153 smithi153 *:9093,9094 running (47s) 7s ago 14m 13.7M - 0.25.0 c8568f914cd2 9ed3f2200b5f 2024-03-21T10:11:57.220 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi153 smithi153 running (14m) 7s ago 14m 8196k - 18.2.1 5be31c24972a 5b61994396a6 2024-03-21T10:11:57.220 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi174 smithi174 running (13m) 103s ago 13m 7940k - 18.2.1 5be31c24972a 4f7a5fc3ac28 2024-03-21T10:11:57.220 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi153 smithi153 running (14m) 7s ago 14m 6665k - 18.2.1 5be31c24972a 90a8b8ea03d1 2024-03-21T10:11:57.220 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi174 smithi174 running (13m) 103s ago 13m 6665k - 18.2.1 5be31c24972a a4c929e4dc77 2024-03-21T10:11:57.220 INFO:teuthology.orchestra.run.smithi153.stdout:grafana.smithi153 smithi153 *:3000 running (14s) 7s ago 13m 50.6M - 9.4.12 2bacad6d85d8 ee24026ab786 2024-03-21T10:11:57.220 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.iuyluj smithi153 running (7m) 7s ago 7m 129M - 18.2.1 5be31c24972a 02f667cee979 2024-03-21T10:11:57.220 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.zipckr smithi153 running (7m) 7s ago 7m 58.6M - 18.2.1 5be31c24972a 68a853929f8f 2024-03-21T10:11:57.221 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.nhozag smithi174 running (7m) 103s ago 7m 128M - 18.2.1 5be31c24972a 2fee4000308f 2024-03-21T10:11:57.221 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.yyimst smithi174 running (7m) 103s ago 7m 155M - 18.2.1 5be31c24972a d78a729220d1 2024-03-21T10:11:57.221 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi153.tluxni smithi153 *:8443,9283,8765 running (3m) 7s ago 16m 511M - 19.0.0-2231-g2edfee68 237e43c7a65f 7ffee0e55382 2024-03-21T10:11:57.221 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi174.qpcwat smithi174 *:8443,9283,8765 running (2m) 103s ago 12m 447M - 19.0.0-2231-g2edfee68 237e43c7a65f 748968f216ab 2024-03-21T10:11:57.221 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi153 smithi153 running (16m) 7s ago 16m 57.2M 2048M 18.2.1 5be31c24972a 813963a95d69 2024-03-21T10:11:57.221 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi174 smithi174 running (12m) 103s ago 12m 45.3M 2048M 18.2.1 5be31c24972a 99204923eec6 2024-03-21T10:11:57.221 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi153 smithi153 *:9100 running (115s) 7s ago 14m 7394k - 1.5.0 0da6a335fe13 bb835edc0e0b 2024-03-21T10:11:57.221 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi174 smithi174 *:9100 running (108s) 103s ago 13m 6765k - 1.5.0 0da6a335fe13 7c4ecad790fa 2024-03-21T10:11:57.221 INFO:teuthology.orchestra.run.smithi153.stdout:osd.0 smithi153 running (11m) 7s ago 11m 215M 1217M 18.2.1 5be31c24972a 28912a2319b4 2024-03-21T10:11:57.221 INFO:teuthology.orchestra.run.smithi153.stdout:osd.1 smithi153 running (11m) 7s ago 11m 214M 1217M 18.2.1 5be31c24972a ca60681e2314 2024-03-21T10:11:57.221 INFO:teuthology.orchestra.run.smithi153.stdout:osd.2 smithi153 running (10m) 7s ago 10m 172M 1217M 18.2.1 5be31c24972a 8db799f8962a 2024-03-21T10:11:57.221 INFO:teuthology.orchestra.run.smithi153.stdout:osd.3 smithi174 running (10m) 103s ago 10m 213M 2241M 18.2.1 5be31c24972a dafc89e0658d 2024-03-21T10:11:57.221 INFO:teuthology.orchestra.run.smithi153.stdout:osd.4 smithi174 running (9m) 103s ago 9m 201M 2241M 18.2.1 5be31c24972a c4cd15701f05 2024-03-21T10:11:57.222 INFO:teuthology.orchestra.run.smithi153.stdout:osd.5 smithi174 running (9m) 103s ago 9m 187M 2241M 18.2.1 5be31c24972a 3b69886ce22e 2024-03-21T10:11:57.222 INFO:teuthology.orchestra.run.smithi153.stdout:prometheus.smithi153 smithi153 *:9095 running (72s) 7s ago 13m 59.7M - 2.43.0 a07b618ecd1d 7d95561e50c2 2024-03-21T10:11:57.223 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:57.222+0000 7ff523e00700 1 -- 172.21.15.153:0/4234253489 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7ff50406e0e0 msgr2=0x7ff5040705a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:11:57.223 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:57.222+0000 7ff523e00700 1 --2- 172.21.15.153:0/4234253489 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7ff50406e0e0 0x7ff5040705a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff510005fd0 tx=0x7ff510005f00 comp rx=0 tx=0).stop 2024-03-21T10:11:57.223 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:57.222+0000 7ff523e00700 1 -- 172.21.15.153:0/4234253489 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff51c108400 msgr2=0x7ff51c19cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:11:57.224 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:57.222+0000 7ff523e00700 1 --2- 172.21.15.153:0/4234253489 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff51c108400 0x7ff51c19cea0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7ff508005b40 tx=0x7ff50800be20 comp rx=0 tx=0).stop 2024-03-21T10:11:57.224 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:57.222+0000 7ff523e00700 1 -- 172.21.15.153:0/4234253489 shutdown_connections 2024-03-21T10:11:57.224 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:57.222+0000 7ff523e00700 1 --2- 172.21.15.153:0/4234253489 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7ff50406e0e0 0x7ff5040705a0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:11:57.224 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:57.222+0000 7ff523e00700 1 --2- 172.21.15.153:0/4234253489 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff51c107200 0x7ff51c19c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:11:57.224 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:57.222+0000 7ff523e00700 1 --2- 172.21.15.153:0/4234253489 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff51c108400 0x7ff51c19cea0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:11:57.224 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:57.222+0000 7ff523e00700 1 -- 172.21.15.153:0/4234253489 >> 172.21.15.153:0/4234253489 conn(0x7ff51c0761b0 msgr2=0x7ff51c10b630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:11:57.224 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:57.222+0000 7ff523e00700 1 -- 172.21.15.153:0/4234253489 shutdown_connections 2024-03-21T10:11:57.224 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:57.222+0000 7ff523e00700 1 -- 172.21.15.153:0/4234253489 wait complete. 2024-03-21T10:11:57.384 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:57.382+0000 7f3d92200700 1 -- 172.21.15.153:0/500425986 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d8c102dd0 msgr2=0x7f3d8c1031f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:11:57.384 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:57.382+0000 7f3d92200700 1 --2- 172.21.15.153:0/500425986 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d8c102dd0 0x7f3d8c1031f0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c009910 tx=0x7f3d7c009c20 comp rx=0 tx=0).stop 2024-03-21T10:11:57.384 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:57.382+0000 7f3d92200700 1 -- 172.21.15.153:0/500425986 shutdown_connections 2024-03-21T10:11:57.384 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:57.382+0000 7f3d92200700 1 --2- 172.21.15.153:0/500425986 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3d8c103fd0 0x7f3d8c104430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:11:57.384 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:57.382+0000 7f3d92200700 1 --2- 172.21.15.153:0/500425986 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d8c102dd0 0x7f3d8c1031f0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:11:57.384 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:57.382+0000 7f3d92200700 1 -- 172.21.15.153:0/500425986 >> 172.21.15.153:0/500425986 conn(0x7f3d8c0fe390 msgr2=0x7f3d8c1007b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:11:57.385 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:57 smithi153 ceph-mon[29117]: from='client.14812 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:11:57.385 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:57.382+0000 7f3d92200700 1 -- 172.21.15.153:0/500425986 shutdown_connections 2024-03-21T10:11:57.386 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:57.382+0000 7f3d92200700 1 -- 172.21.15.153:0/500425986 wait complete. 2024-03-21T10:11:57.386 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:57.384+0000 7f3d92200700 1 Processor -- start 2024-03-21T10:11:57.386 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:57.384+0000 7f3d92200700 1 -- start start 2024-03-21T10:11:57.386 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:57.384+0000 7f3d92200700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3d8c102dd0 0x7f3d8c198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:11:57.386 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:57.384+0000 7f3d92200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d8c103fd0 0x7f3d8c198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:11:57.386 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:57.384+0000 7f3d92200700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d8c1990c0 con 0x7f3d8c103fd0 2024-03-21T10:11:57.386 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:57.384+0000 7f3d92200700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d8c199200 con 0x7f3d8c102dd0 2024-03-21T10:11:57.387 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:57.385+0000 7f3d8aa00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d8c103fd0 0x7f3d8c198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:11:57.387 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:57.385+0000 7f3d8aa00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d8c103fd0 0x7f3d8c198aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:50234/0 (socket says 172.21.15.153:50234) 2024-03-21T10:11:57.387 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:57.385+0000 7f3d8aa00700 1 -- 172.21.15.153:0/1258101778 learned_addr learned my addr 172.21.15.153:0/1258101778 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:11:57.387 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:57.385+0000 7f3d8b400700 1 --2- 172.21.15.153:0/1258101778 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3d8c102dd0 0x7f3d8c198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:11:57.387 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:57.385+0000 7f3d8aa00700 1 -- 172.21.15.153:0/1258101778 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3d8c102dd0 msgr2=0x7f3d8c198560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:11:57.387 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:57.385+0000 7f3d8aa00700 1 --2- 172.21.15.153:0/1258101778 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3d8c102dd0 0x7f3d8c198560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:11:57.387 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:57.385+0000 7f3d8aa00700 1 -- 172.21.15.153:0/1258101778 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d7c0095f0 con 0x7f3d8c103fd0 2024-03-21T10:11:57.388 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:57.385+0000 7f3d8aa00700 1 --2- 172.21.15.153:0/1258101778 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d8c103fd0 0x7f3d8c198aa0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f3d7800c8a0 tx=0x7f3d7800cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:11:57.388 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:57.386+0000 7f3d83e00700 1 -- 172.21.15.153:0/1258101778 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3d780078e0 con 0x7f3d8c103fd0 2024-03-21T10:11:57.388 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:57.386+0000 7f3d92200700 1 -- 172.21.15.153:0/1258101778 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d8c19dcb0 con 0x7f3d8c103fd0 2024-03-21T10:11:57.388 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:57.386+0000 7f3d92200700 1 -- 172.21.15.153:0/1258101778 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d8c19e200 con 0x7f3d8c103fd0 2024-03-21T10:11:57.388 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:57.386+0000 7f3d83e00700 1 -- 172.21.15.153:0/1258101778 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f3d7800f450 con 0x7f3d8c103fd0 2024-03-21T10:11:57.388 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:57.386+0000 7f3d83e00700 1 -- 172.21.15.153:0/1258101778 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3d7800e5e0 con 0x7f3d8c103fd0 2024-03-21T10:11:57.389 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:57.387+0000 7f3d92200700 1 -- 172.21.15.153:0/1258101778 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d6c005320 con 0x7f3d8c103fd0 2024-03-21T10:11:57.390 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:57.389+0000 7f3d83e00700 1 -- 172.21.15.153:0/1258101778 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 31) v1 ==== 92072+0+0 (secure 0 0 0) 0x7f3d78007a40 con 0x7f3d8c103fd0 2024-03-21T10:11:57.391 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:57.390+0000 7f3d83e00700 1 --2- 172.21.15.153:0/1258101778 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f3d7006e0e0 0x7f3d700705a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:11:57.391 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:57.390+0000 7f3d83e00700 1 -- 172.21.15.153:0/1258101778 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f3d7808de00 con 0x7f3d8c103fd0 2024-03-21T10:11:57.391 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:57.390+0000 7f3d8b400700 1 --2- 172.21.15.153:0/1258101778 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f3d7006e0e0 0x7f3d700705a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:11:57.392 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:57.391+0000 7f3d8b400700 1 --2- 172.21.15.153:0/1258101778 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f3d7006e0e0 0x7f3d700705a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c006010 tx=0x7f3d7c005280 comp rx=0 tx=0).ready entity=mgr.14666 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:11:57.396 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:57.395+0000 7f3d83e00700 1 -- 172.21.15.153:0/1258101778 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185454 (secure 0 0 0) 0x7f3d7805af70 con 0x7f3d8c103fd0 2024-03-21T10:11:57.656 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:57.654+0000 7f3d83e00700 1 -- 172.21.15.153:0/1258101778 <== mon.0 v2:172.21.15.153:3300/0 7 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3d7805af70 con 0x7f3d8c103fd0 2024-03-21T10:11:57.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:57 smithi174 ceph-mon[37496]: from='client.14812 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:11:57.729 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:57.727+0000 7f3d92200700 1 -- 172.21.15.153:0/1258101778 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f3d6c006200 con 0x7f3d8c103fd0 2024-03-21T10:11:57.733 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:57.731+0000 7f3d83e00700 1 -- 172.21.15.153:0/1258101778 <== mon.0 v2:172.21.15.153:3300/0 8 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7f3d7805ab00 con 0x7f3d8c103fd0 2024-03-21T10:11:57.733 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:11:57.733 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": { 2024-03-21T10:11:57.733 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-03-21T10:11:57.734 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:11:57.734 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": { 2024-03-21T10:11:57.734 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:11:57.734 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:11:57.734 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": { 2024-03-21T10:11:57.734 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-03-21T10:11:57.734 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:11:57.734 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": { 2024-03-21T10:11:57.734 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-03-21T10:11:57.734 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:11:57.734 INFO:teuthology.orchestra.run.smithi153.stdout: "overall": { 2024-03-21T10:11:57.734 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 12, 2024-03-21T10:11:57.734 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:11:57.735 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-03-21T10:11:57.735 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:11:57.736 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:57.735+0000 7f3d92200700 1 -- 172.21.15.153:0/1258101778 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f3d7006e0e0 msgr2=0x7f3d700705a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:11:57.736 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:57.735+0000 7f3d92200700 1 --2- 172.21.15.153:0/1258101778 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f3d7006e0e0 0x7f3d700705a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c006010 tx=0x7f3d7c005280 comp rx=0 tx=0).stop 2024-03-21T10:11:57.737 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:57.735+0000 7f3d92200700 1 -- 172.21.15.153:0/1258101778 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d8c103fd0 msgr2=0x7f3d8c198aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:11:57.737 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:57.735+0000 7f3d92200700 1 --2- 172.21.15.153:0/1258101778 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d8c103fd0 0x7f3d8c198aa0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f3d7800c8a0 tx=0x7f3d7800cbb0 comp rx=0 tx=0).stop 2024-03-21T10:11:57.737 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:57.735+0000 7f3d92200700 1 -- 172.21.15.153:0/1258101778 shutdown_connections 2024-03-21T10:11:57.737 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:57.735+0000 7f3d92200700 1 --2- 172.21.15.153:0/1258101778 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f3d7006e0e0 0x7f3d700705a0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:11:57.737 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:57.735+0000 7f3d92200700 1 --2- 172.21.15.153:0/1258101778 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3d8c102dd0 0x7f3d8c198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:11:57.737 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:57.735+0000 7f3d92200700 1 --2- 172.21.15.153:0/1258101778 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3d8c103fd0 0x7f3d8c198aa0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:11:57.738 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:57.735+0000 7f3d92200700 1 -- 172.21.15.153:0/1258101778 >> 172.21.15.153:0/1258101778 conn(0x7f3d8c0fe390 msgr2=0x7f3d8c107200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:11:57.738 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:57.735+0000 7f3d92200700 1 -- 172.21.15.153:0/1258101778 shutdown_connections 2024-03-21T10:11:57.738 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:57.735+0000 7f3d92200700 1 -- 172.21.15.153:0/1258101778 wait complete. 2024-03-21T10:11:57.895 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:57.893+0000 7f96d5600700 1 -- 172.21.15.153:0/2275735698 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f96d0100bb0 msgr2=0x7f96d0100fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:11:57.895 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:57.893+0000 7f96d5600700 1 --2- 172.21.15.153:0/2275735698 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f96d0100bb0 0x7f96d0100fd0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f96c0009910 tx=0x7f96c0009c20 comp rx=0 tx=0).stop 2024-03-21T10:11:57.895 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:57.893+0000 7f96d5600700 1 -- 172.21.15.153:0/2275735698 shutdown_connections 2024-03-21T10:11:57.895 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:57.893+0000 7f96d5600700 1 --2- 172.21.15.153:0/2275735698 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f96d0101db0 0x7f96d0102210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:11:57.895 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:57.893+0000 7f96d5600700 1 --2- 172.21.15.153:0/2275735698 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f96d0100bb0 0x7f96d0100fd0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:11:57.895 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:57.893+0000 7f96d5600700 1 -- 172.21.15.153:0/2275735698 >> 172.21.15.153:0/2275735698 conn(0x7f96d00fc110 msgr2=0x7f96d00fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:11:57.895 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:57.893+0000 7f96d5600700 1 -- 172.21.15.153:0/2275735698 shutdown_connections 2024-03-21T10:11:57.895 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:57.893+0000 7f96d5600700 1 -- 172.21.15.153:0/2275735698 wait complete. 2024-03-21T10:11:57.896 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:57.894+0000 7f96d5600700 1 Processor -- start 2024-03-21T10:11:57.896 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:57.894+0000 7f96d5600700 1 -- start start 2024-03-21T10:11:57.896 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:57.895+0000 7f96d5600700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f96d0100bb0 0x7f96d0196380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:11:57.896 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:57.895+0000 7f96d5600700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f96d0101db0 0x7f96d01968c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:11:57.896 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:57.895+0000 7f96d5600700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96d0196ee0 con 0x7f96d0101db0 2024-03-21T10:11:57.896 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:57.895+0000 7f96d5600700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96d0197020 con 0x7f96d0100bb0 2024-03-21T10:11:57.897 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:57.895+0000 7f96ce000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f96d0101db0 0x7f96d01968c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:11:57.898 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:57.895+0000 7f96ce000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f96d0101db0 0x7f96d01968c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:50240/0 (socket says 172.21.15.153:50240) 2024-03-21T10:11:57.898 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:57.895+0000 7f96ce000700 1 -- 172.21.15.153:0/1518509768 learned_addr learned my addr 172.21.15.153:0/1518509768 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:11:57.898 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:57.895+0000 7f96cea00700 1 --2- 172.21.15.153:0/1518509768 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f96d0100bb0 0x7f96d0196380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:11:57.898 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:57.895+0000 7f96ce000700 1 -- 172.21.15.153:0/1518509768 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f96d0100bb0 msgr2=0x7f96d0196380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:11:57.898 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:57.895+0000 7f96ce000700 1 --2- 172.21.15.153:0/1518509768 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f96d0100bb0 0x7f96d0196380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:11:57.898 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:57.895+0000 7f96ce000700 1 -- 172.21.15.153:0/1518509768 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96c00095f0 con 0x7f96d0101db0 2024-03-21T10:11:57.898 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:57.896+0000 7f96ce000700 1 --2- 172.21.15.153:0/1518509768 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f96d0101db0 0x7f96d01968c0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f96bc00d8d0 tx=0x7f96bc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:11:57.898 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:57.896+0000 7f96c7400700 1 -- 172.21.15.153:0/1518509768 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f96bc009940 con 0x7f96d0101db0 2024-03-21T10:11:57.898 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:57.896+0000 7f96c7400700 1 -- 172.21.15.153:0/1518509768 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f96bc010460 con 0x7f96d0101db0 2024-03-21T10:11:57.898 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:57.896+0000 7f96d5600700 1 -- 172.21.15.153:0/1518509768 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96d019bad0 con 0x7f96d0101db0 2024-03-21T10:11:57.898 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:57.896+0000 7f96c7400700 1 -- 172.21.15.153:0/1518509768 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f96bc00f5f0 con 0x7f96d0101db0 2024-03-21T10:11:57.898 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:57.896+0000 7f96d5600700 1 -- 172.21.15.153:0/1518509768 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96d019c0a0 con 0x7f96d0101db0 2024-03-21T10:11:57.899 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:57.897+0000 7f96d5600700 1 -- 172.21.15.153:0/1518509768 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96d0109710 con 0x7f96d0101db0 2024-03-21T10:11:57.900 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:57.898+0000 7f96c7400700 1 -- 172.21.15.153:0/1518509768 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 31) v1 ==== 92072+0+0 (secure 0 0 0) 0x7f96bc0105d0 con 0x7f96d0101db0 2024-03-21T10:11:57.901 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:57.899+0000 7f96c7400700 1 --2- 172.21.15.153:0/1518509768 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f96b406e0e0 0x7f96b40705a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:11:57.901 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:57.899+0000 7f96c7400700 1 -- 172.21.15.153:0/1518509768 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f96bc08dd10 con 0x7f96d0101db0 2024-03-21T10:11:57.901 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:57.900+0000 7f96cea00700 1 --2- 172.21.15.153:0/1518509768 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f96b406e0e0 0x7f96b40705a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:11:57.901 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:57.900+0000 7f96cea00700 1 --2- 172.21.15.153:0/1518509768 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f96b406e0e0 0x7f96b40705a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f96c0006010 tx=0x7f96c0005940 comp rx=0 tx=0).ready entity=mgr.14666 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:11:57.905 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:57.904+0000 7f96c7400700 1 -- 172.21.15.153:0/1518509768 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185454 (secure 0 0 0) 0x7f96bc05afa0 con 0x7f96d0101db0 2024-03-21T10:11:58.173 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:58.171+0000 7f96d5600700 1 -- 172.21.15.153:0/1518509768 --> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f96d0106700 con 0x7f96b406e0e0 2024-03-21T10:11:58.174 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:58.172+0000 7f96c7400700 1 -- 172.21.15.153:0/1518509768 <== mgr.14666 v2:172.21.15.153:6800/2524145044 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7f96d0106700 con 0x7f96b406e0e0 2024-03-21T10:11:58.174 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:11:58.174 INFO:teuthology.orchestra.run.smithi153.stdout: "target_image": null, 2024-03-21T10:11:58.174 INFO:teuthology.orchestra.run.smithi153.stdout: "in_progress": false, 2024-03-21T10:11:58.175 INFO:teuthology.orchestra.run.smithi153.stdout: "which": "", 2024-03-21T10:11:58.175 INFO:teuthology.orchestra.run.smithi153.stdout: "services_complete": [], 2024-03-21T10:11:58.175 INFO:teuthology.orchestra.run.smithi153.stdout: "progress": null, 2024-03-21T10:11:58.175 INFO:teuthology.orchestra.run.smithi153.stdout: "message": "", 2024-03-21T10:11:58.175 INFO:teuthology.orchestra.run.smithi153.stdout: "is_paused": false 2024-03-21T10:11:58.175 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:11:58.177 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:58.175+0000 7f96d5600700 1 -- 172.21.15.153:0/1518509768 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f96b406e0e0 msgr2=0x7f96b40705a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:11:58.177 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:58.175+0000 7f96d5600700 1 --2- 172.21.15.153:0/1518509768 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f96b406e0e0 0x7f96b40705a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f96c0006010 tx=0x7f96c0005940 comp rx=0 tx=0).stop 2024-03-21T10:11:58.177 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:58.175+0000 7f96d5600700 1 -- 172.21.15.153:0/1518509768 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f96d0101db0 msgr2=0x7f96d01968c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:11:58.177 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:58.175+0000 7f96d5600700 1 --2- 172.21.15.153:0/1518509768 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f96d0101db0 0x7f96d01968c0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f96bc00d8d0 tx=0x7f96bc00dc90 comp rx=0 tx=0).stop 2024-03-21T10:11:58.177 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:58.175+0000 7f96d5600700 1 -- 172.21.15.153:0/1518509768 shutdown_connections 2024-03-21T10:11:58.177 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:58.175+0000 7f96d5600700 1 --2- 172.21.15.153:0/1518509768 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f96b406e0e0 0x7f96b40705a0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:11:58.177 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:58.175+0000 7f96d5600700 1 --2- 172.21.15.153:0/1518509768 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f96d0100bb0 0x7f96d0196380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:11:58.177 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:58.175+0000 7f96d5600700 1 --2- 172.21.15.153:0/1518509768 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f96d0101db0 0x7f96d01968c0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:11:58.177 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:58.175+0000 7f96d5600700 1 -- 172.21.15.153:0/1518509768 >> 172.21.15.153:0/1518509768 conn(0x7f96d00fc110 msgr2=0x7f96d0104fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:11:58.177 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:58.175+0000 7f96d5600700 1 -- 172.21.15.153:0/1518509768 shutdown_connections 2024-03-21T10:11:58.177 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:11:58.176+0000 7f96d5600700 1 -- 172.21.15.153:0/1518509768 wait complete. 2024-03-21T10:11:58.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:58 smithi153 ceph-mon[29117]: from='client.14816 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:11:58.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:58 smithi153 ceph-mon[29117]: from='client.14820 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:11:58.558 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:58 smithi153 ceph-mon[29117]: pgmap v105: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:11:58.558 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:58 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:11:58.558 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:58 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:11:58.558 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:58 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:11:58.558 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:58 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:11:58.558 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:58 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:11:58.558 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:58 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-21T10:11:58.558 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:58 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:11:58.558 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:58 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:11:58.558 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:58 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:11:58.558 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:58 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:11:58.559 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:58 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:11:58.559 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:58 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:11:58.559 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:58 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:11:58.559 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:58 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:11:58.559 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:58 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:11:58.559 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:58 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:11:58.559 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:58 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:11:58.559 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:58 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:11:58.559 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:58 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:11:58.559 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:58 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:11:58.559 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:58 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:11:58.559 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:58 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-03-21T10:11:58.559 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:58 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-03-21T10:11:58.560 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:58 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-21T10:11:58.560 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:58 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-03-21T10:11:58.560 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:58 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-03-21T10:11:58.560 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:58 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-03-21T10:11:58.560 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:58 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-03-21T10:11:58.560 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:58 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-03-21T10:11:58.560 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:58 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-03-21T10:11:58.560 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:58 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-03-21T10:11:58.561 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:58 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-03-21T10:11:58.561 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:58 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-21T10:11:58.561 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:58 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-03-21T10:11:58.561 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:58 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-03-21T10:11:58.561 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:58 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-03-21T10:11:58.561 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:58 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-03-21T10:11:58.561 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:58 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-03-21T10:11:58.561 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:58 smithi153 ceph-mon[29117]: from='client.? 172.21.15.153:0/1258101778' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:11:58.561 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:58 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-03-21T10:11:58.561 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:58 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-03-21T10:11:58.561 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:58 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-21T10:11:58.561 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:58 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-21T10:11:58.561 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:58 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-21T10:11:58.561 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:58 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-21T10:11:58.562 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:58 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-21T10:11:58.562 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:58 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-21T10:11:58.562 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:58 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-03-21T10:11:58.562 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:58 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-03-21T10:11:58.562 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:58 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:11:58.562 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:58 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:11:58.562 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:58 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:11:58.562 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:58 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:11:58.562 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:58 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:11:58.562 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:58 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:11:58.562 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:58 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:11:58.562 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:58 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:11:58.605 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:58 smithi174 ceph-mon[37496]: from='client.14816 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:11:58.605 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:58 smithi174 ceph-mon[37496]: from='client.14820 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:11:58.605 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:58 smithi174 ceph-mon[37496]: pgmap v105: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:11:58.605 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:58 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:11:58.605 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:58 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:11:58.606 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:58 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:11:58.606 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:58 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:11:58.606 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:58 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:11:58.606 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:58 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-21T10:11:58.606 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:58 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:11:58.606 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:58 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:11:58.606 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:58 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:11:58.606 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:58 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:11:58.606 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:58 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:11:58.606 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:58 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:11:58.606 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:58 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:11:58.606 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:58 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:11:58.606 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:58 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:11:58.607 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:58 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:11:58.607 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:58 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:11:58.607 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:58 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:11:58.607 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:58 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:11:58.607 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:58 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:11:58.607 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:58 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:11:58.607 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:58 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-03-21T10:11:58.607 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:58 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-03-21T10:11:58.607 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:58 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-21T10:11:58.608 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:58 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-03-21T10:11:58.608 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:58 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-03-21T10:11:58.608 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:58 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-03-21T10:11:58.608 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:58 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-03-21T10:11:58.608 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:58 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-03-21T10:11:58.608 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:58 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-03-21T10:11:58.608 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:58 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-03-21T10:11:58.608 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:58 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-03-21T10:11:58.608 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:58 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-21T10:11:58.608 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:58 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-03-21T10:11:58.608 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:58 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-03-21T10:11:58.608 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:58 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-03-21T10:11:58.608 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:58 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-03-21T10:11:58.609 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:58 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-03-21T10:11:58.609 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:58 smithi174 ceph-mon[37496]: from='client.? 172.21.15.153:0/1258101778' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:11:58.609 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:58 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-03-21T10:11:58.609 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:58 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-03-21T10:11:58.609 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:58 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-21T10:11:58.609 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:58 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-21T10:11:58.609 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:58 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-21T10:11:58.609 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:58 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-21T10:11:58.609 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:58 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-21T10:11:58.609 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:58 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-21T10:11:58.609 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:58 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-03-21T10:11:58.609 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:58 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-03-21T10:11:58.609 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:58 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:11:58.610 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:58 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:11:58.610 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:58 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:11:58.610 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:58 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:11:58.610 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:58 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:11:58.610 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:58 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:11:58.610 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:58 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:11:58.610 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:58 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:11:59.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:59 smithi174 ceph-mon[37496]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-21T10:11:59.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:59 smithi174 ceph-mon[37496]: Upgrade: Finalizing container_image settings 2024-03-21T10:11:59.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:59 smithi174 ceph-mon[37496]: Upgrade: Complete! 2024-03-21T10:11:59.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:59 smithi174 ceph-mon[37496]: from='client.14828 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:11:59.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:11:59 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:11:59.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:59 smithi153 ceph-mon[29117]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-21T10:11:59.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:59 smithi153 ceph-mon[29117]: Upgrade: Finalizing container_image settings 2024-03-21T10:11:59.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:59 smithi153 ceph-mon[29117]: Upgrade: Complete! 2024-03-21T10:11:59.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:59 smithi153 ceph-mon[29117]: from='client.14828 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:11:59.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:11:59 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:12:00.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:12:00 smithi174 ceph-mon[37496]: pgmap v106: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:12:00.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:12:00 smithi153 ceph-mon[29117]: pgmap v106: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:12:02.649 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:12:02 smithi153 ceph-mon[29117]: pgmap v107: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:12:02.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:12:02 smithi174 ceph-mon[37496]: pgmap v107: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:12:03.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:12:03 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:12:03.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:12:03 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:12:04.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:12:04 smithi174 ceph-mon[37496]: pgmap v108: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:12:04.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:12:04 smithi153 ceph-mon[29117]: pgmap v108: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:12:06.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:12:06 smithi174 ceph-mon[37496]: pgmap v109: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:12:06.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:12:06 smithi153 ceph-mon[29117]: pgmap v109: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:12:08.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:12:08 smithi153 ceph-mon[29117]: pgmap v110: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:12:08.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:12:08 smithi174 ceph-mon[37496]: pgmap v110: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:12:10.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:12:10 smithi174 ceph-mon[37496]: pgmap v111: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:12:10.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:12:10 smithi153 ceph-mon[29117]: pgmap v111: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:12:12.649 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:12:12 smithi153 ceph-mon[29117]: pgmap v112: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:12:12.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:12:12 smithi174 ceph-mon[37496]: pgmap v112: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:12:14.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:12:14 smithi174 ceph-mon[37496]: pgmap v113: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:12:14.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:12:14 smithi153 ceph-mon[29117]: pgmap v113: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:12:16.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:12:16 smithi174 ceph-mon[37496]: pgmap v114: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:12:16.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:12:16 smithi153 ceph-mon[29117]: pgmap v114: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:12:18.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:12:18 smithi174 ceph-mon[37496]: pgmap v115: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:12:18.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:12:18 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:12:18.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:12:18 smithi153 ceph-mon[29117]: pgmap v115: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:12:18.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:12:18 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:12:20.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:12:20 smithi174 ceph-mon[37496]: pgmap v116: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:12:20.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:12:20 smithi153 ceph-mon[29117]: pgmap v116: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:12:22.609 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:12:22 smithi153 ceph-mon[29117]: pgmap v117: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:12:22.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:12:22 smithi174 ceph-mon[37496]: pgmap v117: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:12:24.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:12:24 smithi153 ceph-mon[29117]: pgmap v118: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:12:24.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:12:24 smithi174 ceph-mon[37496]: pgmap v118: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:12:26.728 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:12:26 smithi174 ceph-mon[37496]: pgmap v119: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:12:26.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:12:26 smithi153 ceph-mon[29117]: pgmap v119: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:12:28.349 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:28.348+0000 7fb611800700 1 -- 172.21.15.153:0/4169429743 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb60c101640 msgr2=0x7fb60c103a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:12:28.349 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:28.348+0000 7fb611800700 1 --2- 172.21.15.153:0/4169429743 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb60c101640 0x7fb60c103a30 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc009910 tx=0x7fb5fc009c20 comp rx=0 tx=0).stop 2024-03-21T10:12:28.349 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:28.348+0000 7fb611800700 1 -- 172.21.15.153:0/4169429743 shutdown_connections 2024-03-21T10:12:28.349 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:28.348+0000 7fb611800700 1 --2- 172.21.15.153:0/4169429743 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb60c103f70 0x7fb60c106360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:12:28.349 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:28.348+0000 7fb611800700 1 --2- 172.21.15.153:0/4169429743 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb60c101640 0x7fb60c103a30 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:12:28.349 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:28.348+0000 7fb611800700 1 -- 172.21.15.153:0/4169429743 >> 172.21.15.153:0/4169429743 conn(0x7fb60c0fb030 msgr2=0x7fb60c0fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:12:28.350 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:28.349+0000 7fb611800700 1 -- 172.21.15.153:0/4169429743 shutdown_connections 2024-03-21T10:12:28.350 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:28.349+0000 7fb611800700 1 -- 172.21.15.153:0/4169429743 wait complete. 2024-03-21T10:12:28.355 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:28.350+0000 7fb611800700 1 Processor -- start 2024-03-21T10:12:28.355 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:28.350+0000 7fb611800700 1 -- start start 2024-03-21T10:12:28.355 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:28.350+0000 7fb611800700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb60c101640 0x7fb60c198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:12:28.355 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:28.350+0000 7fb611800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb60c103f70 0x7fb60c198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:12:28.356 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:28.350+0000 7fb611800700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb60c199070 con 0x7fb60c103f70 2024-03-21T10:12:28.356 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:28.350+0000 7fb611800700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb60c1991b0 con 0x7fb60c101640 2024-03-21T10:12:28.356 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:28.350+0000 7fb60a000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb60c103f70 0x7fb60c198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:12:28.356 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:28.350+0000 7fb60a000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb60c103f70 0x7fb60c198a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:50096/0 (socket says 172.21.15.153:50096) 2024-03-21T10:12:28.356 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:28.350+0000 7fb60a000700 1 -- 172.21.15.153:0/2709707148 learned_addr learned my addr 172.21.15.153:0/2709707148 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:12:28.356 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:28.351+0000 7fb60aa00700 1 --2- 172.21.15.153:0/2709707148 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb60c101640 0x7fb60c198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:12:28.356 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:28.351+0000 7fb60a000700 1 -- 172.21.15.153:0/2709707148 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb60c101640 msgr2=0x7fb60c198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:12:28.356 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:28.351+0000 7fb60a000700 1 --2- 172.21.15.153:0/2709707148 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb60c101640 0x7fb60c198510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:12:28.356 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:28.351+0000 7fb60a000700 1 -- 172.21.15.153:0/2709707148 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5fc0095f0 con 0x7fb60c103f70 2024-03-21T10:12:28.356 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:28.351+0000 7fb60aa00700 1 --2- 172.21.15.153:0/2709707148 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb60c101640 0x7fb60c198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:12:28.356 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:28.351+0000 7fb60a000700 1 --2- 172.21.15.153:0/2709707148 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb60c103f70 0x7fb60c198a50 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fb5f800d8d0 tx=0x7fb5f800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:12:28.357 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:28.351+0000 7fb603400700 1 -- 172.21.15.153:0/2709707148 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb5f8009940 con 0x7fb60c103f70 2024-03-21T10:12:28.357 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:28.351+0000 7fb603400700 1 -- 172.21.15.153:0/2709707148 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb5f8010460 con 0x7fb60c103f70 2024-03-21T10:12:28.357 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:28.351+0000 7fb603400700 1 -- 172.21.15.153:0/2709707148 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb5f800f5f0 con 0x7fb60c103f70 2024-03-21T10:12:28.357 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:28.351+0000 7fb611800700 1 -- 172.21.15.153:0/2709707148 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb60c19dc60 con 0x7fb60c103f70 2024-03-21T10:12:28.357 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:28.352+0000 7fb611800700 1 -- 172.21.15.153:0/2709707148 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb60c19e230 con 0x7fb60c103f70 2024-03-21T10:12:28.357 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:28.353+0000 7fb611800700 1 -- 172.21.15.153:0/2709707148 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb60c0fcc20 con 0x7fb60c103f70 2024-03-21T10:12:28.357 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:28.354+0000 7fb603400700 1 -- 172.21.15.153:0/2709707148 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 31) v1 ==== 92072+0+0 (secure 0 0 0) 0x7fb5f80105d0 con 0x7fb60c103f70 2024-03-21T10:12:28.357 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:28.355+0000 7fb603400700 1 --2- 172.21.15.153:0/2709707148 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7fb5f006e0e0 0x7fb5f00705a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:12:28.357 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:28.355+0000 7fb603400700 1 -- 172.21.15.153:0/2709707148 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fb5f808ddb0 con 0x7fb60c103f70 2024-03-21T10:12:28.358 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:28.355+0000 7fb60aa00700 1 --2- 172.21.15.153:0/2709707148 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7fb5f006e0e0 0x7fb5f00705a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:12:28.358 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:28.356+0000 7fb60aa00700 1 --2- 172.21.15.153:0/2709707148 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7fb5f006e0e0 0x7fb5f00705a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc006010 tx=0x7fb5fc00b540 comp rx=0 tx=0).ready entity=mgr.14666 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:12:28.360 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:28.360+0000 7fb603400700 1 -- 172.21.15.153:0/2709707148 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185454 (secure 0 0 0) 0x7fb5f805afa0 con 0x7fb60c103f70 2024-03-21T10:12:28.603 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:28.601+0000 7fb611800700 1 -- 172.21.15.153:0/2709707148 --> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb60c0611d0 con 0x7fb5f006e0e0 2024-03-21T10:12:28.603 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:12:28 smithi153 ceph-mon[29117]: pgmap v120: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:12:28.604 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:28.603+0000 7fb603400700 1 -- 172.21.15.153:0/2709707148 <== mgr.14666 v2:172.21.15.153:6800/2524145044 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7fb60c0611d0 con 0x7fb5f006e0e0 2024-03-21T10:12:28.609 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:28.608+0000 7fb611800700 1 -- 172.21.15.153:0/2709707148 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7fb5f006e0e0 msgr2=0x7fb5f00705a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:12:28.609 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:28.608+0000 7fb611800700 1 --2- 172.21.15.153:0/2709707148 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7fb5f006e0e0 0x7fb5f00705a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc006010 tx=0x7fb5fc00b540 comp rx=0 tx=0).stop 2024-03-21T10:12:28.609 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:28.608+0000 7fb611800700 1 -- 172.21.15.153:0/2709707148 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb60c103f70 msgr2=0x7fb60c198a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:12:28.609 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:28.608+0000 7fb611800700 1 --2- 172.21.15.153:0/2709707148 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb60c103f70 0x7fb60c198a50 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fb5f800d8d0 tx=0x7fb5f800dc90 comp rx=0 tx=0).stop 2024-03-21T10:12:28.609 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:28.608+0000 7fb611800700 1 -- 172.21.15.153:0/2709707148 shutdown_connections 2024-03-21T10:12:28.609 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:28.608+0000 7fb611800700 1 --2- 172.21.15.153:0/2709707148 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7fb5f006e0e0 0x7fb5f00705a0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:12:28.609 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:28.608+0000 7fb611800700 1 --2- 172.21.15.153:0/2709707148 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb60c101640 0x7fb60c198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:12:28.609 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:28.608+0000 7fb611800700 1 --2- 172.21.15.153:0/2709707148 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb60c103f70 0x7fb60c198a50 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:12:28.609 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:28.608+0000 7fb611800700 1 -- 172.21.15.153:0/2709707148 >> 172.21.15.153:0/2709707148 conn(0x7fb60c0fb030 msgr2=0x7fb60c104be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:12:28.609 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:28.608+0000 7fb611800700 1 -- 172.21.15.153:0/2709707148 shutdown_connections 2024-03-21T10:12:28.609 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:28.608+0000 7fb611800700 1 -- 172.21.15.153:0/2709707148 wait complete. 2024-03-21T10:12:28.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:12:28 smithi174 ceph-mon[37496]: pgmap v120: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:12:29.397 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 070ed3e4-e769-11ee-95cd-87774f69a715 -e sha1=2edfee68ad7243bed422a049b6f5f58619300f11 -- bash -c 'ceph versions | jq -e '"'"'.mgr | length == 1'"'"'' 2024-03-21T10:12:29.801 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi153/config 2024-03-21T10:12:30.775 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:12:30 smithi153 ceph-mon[29117]: from='client.14832 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:12:30.775 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:12:30 smithi153 ceph-mon[29117]: pgmap v121: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:12:30.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:12:30 smithi174 ceph-mon[37496]: from='client.14832 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:12:30.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:12:30 smithi174 ceph-mon[37496]: pgmap v121: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:12:31.933 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:31.932+0000 7faf4ee00700 1 -- 172.21.15.153:0/3822030682 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faf48068670 msgr2=0x7faf48068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:12:31.934 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:31.932+0000 7faf4ee00700 1 --2- 172.21.15.153:0/3822030682 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faf48068670 0x7faf48068a50 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7faf38009910 tx=0x7faf38009c20 comp rx=0 tx=0).stop 2024-03-21T10:12:31.934 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:31.932+0000 7faf4ee00700 1 -- 172.21.15.153:0/3822030682 shutdown_connections 2024-03-21T10:12:31.934 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:31.932+0000 7faf4ee00700 1 --2- 172.21.15.153:0/3822030682 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faf48069020 0x7faf48105790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:12:31.934 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:31.932+0000 7faf4ee00700 1 --2- 172.21.15.153:0/3822030682 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faf48068670 0x7faf48068a50 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:12:31.934 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:31.932+0000 7faf4ee00700 1 -- 172.21.15.153:0/3822030682 >> 172.21.15.153:0/3822030682 conn(0x7faf480755a0 msgr2=0x7faf480759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:12:31.934 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:31.932+0000 7faf4ee00700 1 -- 172.21.15.153:0/3822030682 shutdown_connections 2024-03-21T10:12:31.934 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:31.932+0000 7faf4ee00700 1 -- 172.21.15.153:0/3822030682 wait complete. 2024-03-21T10:12:31.936 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:31.933+0000 7faf4ee00700 1 Processor -- start 2024-03-21T10:12:31.936 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:31.934+0000 7faf4ee00700 1 -- start start 2024-03-21T10:12:31.936 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:31.934+0000 7faf4ee00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faf48068670 0x7faf4819e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:12:31.936 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:31.934+0000 7faf4ee00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faf48069020 0x7faf4819ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:12:31.936 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:31.934+0000 7faf4ee00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf4819f250 con 0x7faf48069020 2024-03-21T10:12:31.936 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:31.934+0000 7faf4ee00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf48198790 con 0x7faf48068670 2024-03-21T10:12:31.936 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:31.934+0000 7faf3fe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faf48069020 0x7faf4819ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:12:31.936 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:31.934+0000 7faf3fe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faf48069020 0x7faf4819ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:50104/0 (socket says 172.21.15.153:50104) 2024-03-21T10:12:31.936 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:31.934+0000 7faf3fe00700 1 -- 172.21.15.153:0/3532818087 learned_addr learned my addr 172.21.15.153:0/3532818087 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:12:31.936 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:31.935+0000 7faf47e00700 1 --2- 172.21.15.153:0/3532818087 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faf48068670 0x7faf4819e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:12:31.937 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:31.935+0000 7faf3fe00700 1 -- 172.21.15.153:0/3532818087 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faf48068670 msgr2=0x7faf4819e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:12:31.937 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:31.935+0000 7faf3fe00700 1 --2- 172.21.15.153:0/3532818087 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faf48068670 0x7faf4819e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:12:31.937 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:31.935+0000 7faf3fe00700 1 -- 172.21.15.153:0/3532818087 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf380095f0 con 0x7faf48069020 2024-03-21T10:12:31.937 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:31.935+0000 7faf3fe00700 1 --2- 172.21.15.153:0/3532818087 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faf48069020 0x7faf4819ec50 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7faf3000c960 tx=0x7faf3000cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:12:31.937 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:31.935+0000 7faf45600700 1 -- 172.21.15.153:0/3532818087 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faf30007a70 con 0x7faf48069020 2024-03-21T10:12:31.937 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:31.935+0000 7faf45600700 1 -- 172.21.15.153:0/3532818087 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faf30007bd0 con 0x7faf48069020 2024-03-21T10:12:31.937 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:31.935+0000 7faf4ee00700 1 -- 172.21.15.153:0/3532818087 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf48198a70 con 0x7faf48069020 2024-03-21T10:12:31.937 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:31.935+0000 7faf45600700 1 -- 172.21.15.153:0/3532818087 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faf300186c0 con 0x7faf48069020 2024-03-21T10:12:31.937 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:31.935+0000 7faf4ee00700 1 -- 172.21.15.153:0/3532818087 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf48198fc0 con 0x7faf48069020 2024-03-21T10:12:31.938 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:31.937+0000 7faf4ee00700 1 -- 172.21.15.153:0/3532818087 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf4804ea90 con 0x7faf48069020 2024-03-21T10:12:31.938 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:31.938+0000 7faf45600700 1 -- 172.21.15.153:0/3532818087 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 31) v1 ==== 92072+0+0 (secure 0 0 0) 0x7faf3000faa0 con 0x7faf48069020 2024-03-21T10:12:31.939 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:31.939+0000 7faf45600700 1 --2- 172.21.15.153:0/3532818087 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7faf2c06e0e0 0x7faf2c0705a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:12:31.939 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:31.939+0000 7faf45600700 1 -- 172.21.15.153:0/3532818087 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7faf3008df20 con 0x7faf48069020 2024-03-21T10:12:31.939 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:31.939+0000 7faf47e00700 1 --2- 172.21.15.153:0/3532818087 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7faf2c06e0e0 0x7faf2c0705a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:12:31.940 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:31.940+0000 7faf47e00700 1 --2- 172.21.15.153:0/3532818087 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7faf2c06e0e0 0x7faf2c0705a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7faf38006010 tx=0x7faf3800b540 comp rx=0 tx=0).ready entity=mgr.14666 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:12:31.944 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:31.944+0000 7faf45600700 1 -- 172.21.15.153:0/3532818087 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185454 (secure 0 0 0) 0x7faf3005ae00 con 0x7faf48069020 2024-03-21T10:12:32.264 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:32.263+0000 7faf4ee00700 1 -- 172.21.15.153:0/3532818087 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7faf481992e0 con 0x7faf48069020 2024-03-21T10:12:32.265 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:32.264+0000 7faf45600700 1 -- 172.21.15.153:0/3532818087 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7faf3005ac20 con 0x7faf48069020 2024-03-21T10:12:32.268 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:32.267+0000 7faf4ee00700 1 -- 172.21.15.153:0/3532818087 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7faf2c06e0e0 msgr2=0x7faf2c0705a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:12:32.268 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:32.267+0000 7faf4ee00700 1 --2- 172.21.15.153:0/3532818087 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7faf2c06e0e0 0x7faf2c0705a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7faf38006010 tx=0x7faf3800b540 comp rx=0 tx=0).stop 2024-03-21T10:12:32.268 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:32.267+0000 7faf4ee00700 1 -- 172.21.15.153:0/3532818087 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faf48069020 msgr2=0x7faf4819ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:12:32.268 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:32.267+0000 7faf4ee00700 1 --2- 172.21.15.153:0/3532818087 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faf48069020 0x7faf4819ec50 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7faf3000c960 tx=0x7faf3000cd20 comp rx=0 tx=0).stop 2024-03-21T10:12:32.268 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:32.267+0000 7faf4ee00700 1 -- 172.21.15.153:0/3532818087 shutdown_connections 2024-03-21T10:12:32.268 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:32.267+0000 7faf4ee00700 1 --2- 172.21.15.153:0/3532818087 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7faf2c06e0e0 0x7faf2c0705a0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:12:32.268 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:32.267+0000 7faf4ee00700 1 --2- 172.21.15.153:0/3532818087 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faf48068670 0x7faf4819e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:12:32.268 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:32.267+0000 7faf4ee00700 1 --2- 172.21.15.153:0/3532818087 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faf48069020 0x7faf4819ec50 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:12:32.268 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:32.267+0000 7faf4ee00700 1 -- 172.21.15.153:0/3532818087 >> 172.21.15.153:0/3532818087 conn(0x7faf480755a0 msgr2=0x7faf48102760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:12:32.269 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:32.268+0000 7faf4ee00700 1 -- 172.21.15.153:0/3532818087 shutdown_connections 2024-03-21T10:12:32.269 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:32.268+0000 7faf4ee00700 1 -- 172.21.15.153:0/3532818087 wait complete. 2024-03-21T10:12:32.284 INFO:teuthology.orchestra.run.smithi153.stdout:true 2024-03-21T10:12:32.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:12:32 smithi153 ceph-mon[29117]: pgmap v122: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:12:32.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:12:32 smithi153 ceph-mon[29117]: from='client.? 172.21.15.153:0/3532818087' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:12:32.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:12:32 smithi174 ceph-mon[37496]: pgmap v122: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:12:32.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:12:32 smithi174 ceph-mon[37496]: from='client.? 172.21.15.153:0/3532818087' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:12:33.016 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 070ed3e4-e769-11ee-95cd-87774f69a715 -e sha1=2edfee68ad7243bed422a049b6f5f58619300f11 -- bash -c 'ceph versions | jq -e '"'"'.mgr | keys'"'"' | grep $sha1' 2024-03-21T10:12:33.424 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi153/config 2024-03-21T10:12:33.741 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:12:33 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:12:33.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:12:33 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:12:34.622 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:12:34 smithi153 ceph-mon[29117]: pgmap v123: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:12:34.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:12:34 smithi174 ceph-mon[37496]: pgmap v123: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:12:35.426 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:35.425+0000 7f133ca00700 1 -- 172.21.15.153:0/3776169479 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f13381039a0 msgr2=0x7f13381079f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:12:35.427 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:35.425+0000 7f133ca00700 1 --2- 172.21.15.153:0/3776169479 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f13381039a0 0x7f13381079f0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f1320009910 tx=0x7f1320009c20 comp rx=0 tx=0).stop 2024-03-21T10:12:35.427 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:35.426+0000 7f133ca00700 1 -- 172.21.15.153:0/3776169479 shutdown_connections 2024-03-21T10:12:35.427 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:35.426+0000 7f133ca00700 1 --2- 172.21.15.153:0/3776169479 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f13381039a0 0x7f13381079f0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:12:35.427 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:35.426+0000 7f133ca00700 1 --2- 172.21.15.153:0/3776169479 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1338102ff0 0x7f13381033d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:12:35.427 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:35.426+0000 7f133ca00700 1 -- 172.21.15.153:0/3776169479 >> 172.21.15.153:0/3776169479 conn(0x7f13380fe880 msgr2=0x7f1338100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:12:35.427 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:35.426+0000 7f133ca00700 1 -- 172.21.15.153:0/3776169479 shutdown_connections 2024-03-21T10:12:35.427 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:35.426+0000 7f133ca00700 1 -- 172.21.15.153:0/3776169479 wait complete. 2024-03-21T10:12:35.428 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:35.427+0000 7f133ca00700 1 Processor -- start 2024-03-21T10:12:35.428 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:35.427+0000 7f133ca00700 1 -- start start 2024-03-21T10:12:35.428 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:35.428+0000 7f133ca00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1338102ff0 0x7f133819e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:12:35.428 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:35.428+0000 7f133ca00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f13381039a0 0x7f133819ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:12:35.429 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:35.428+0000 7f133ca00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f133819f420 con 0x7f1338102ff0 2024-03-21T10:12:35.429 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:35.428+0000 7f133ca00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13381988d0 con 0x7f13381039a0 2024-03-21T10:12:35.429 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:35.428+0000 7f1335e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1338102ff0 0x7f133819e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:12:35.429 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:35.428+0000 7f1335e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1338102ff0 0x7f133819e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:47690/0 (socket says 172.21.15.153:47690) 2024-03-21T10:12:35.429 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:35.428+0000 7f1335e00700 1 -- 172.21.15.153:0/1713698794 learned_addr learned my addr 172.21.15.153:0/1713698794 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:12:35.430 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:35.428+0000 7f132d400700 1 --2- 172.21.15.153:0/1713698794 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f13381039a0 0x7f133819ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:12:35.430 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:35.429+0000 7f1335e00700 1 -- 172.21.15.153:0/1713698794 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f13381039a0 msgr2=0x7f133819ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:12:35.430 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:35.429+0000 7f1335e00700 1 --2- 172.21.15.153:0/1713698794 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f13381039a0 0x7f133819ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:12:35.430 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:35.429+0000 7f1335e00700 1 -- 172.21.15.153:0/1713698794 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13200095f0 con 0x7f1338102ff0 2024-03-21T10:12:35.430 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:35.429+0000 7f132d400700 1 --2- 172.21.15.153:0/1713698794 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f13381039a0 0x7f133819ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:12:35.430 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:35.429+0000 7f1335e00700 1 --2- 172.21.15.153:0/1713698794 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1338102ff0 0x7f133819e850 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f132800ba50 tx=0x7f132800be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:12:35.431 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:35.429+0000 7f132f400700 1 -- 172.21.15.153:0/1713698794 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f132800ca00 con 0x7f1338102ff0 2024-03-21T10:12:35.431 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:35.429+0000 7f132f400700 1 -- 172.21.15.153:0/1713698794 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f132800cb60 con 0x7f1338102ff0 2024-03-21T10:12:35.431 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:35.429+0000 7f133ca00700 1 -- 172.21.15.153:0/1713698794 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1338198bb0 con 0x7f1338102ff0 2024-03-21T10:12:35.431 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:35.429+0000 7f132f400700 1 -- 172.21.15.153:0/1713698794 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f13280075a0 con 0x7f1338102ff0 2024-03-21T10:12:35.431 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:35.429+0000 7f133ca00700 1 -- 172.21.15.153:0/1713698794 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1338199100 con 0x7f1338102ff0 2024-03-21T10:12:35.431 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:35.431+0000 7f133ca00700 1 -- 172.21.15.153:0/1713698794 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f133810b340 con 0x7f1338102ff0 2024-03-21T10:12:35.432 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:35.432+0000 7f132f400700 1 -- 172.21.15.153:0/1713698794 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 31) v1 ==== 92072+0+0 (secure 0 0 0) 0x7f132800ccd0 con 0x7f1338102ff0 2024-03-21T10:12:35.433 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:35.432+0000 7f132f400700 1 --2- 172.21.15.153:0/1713698794 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f131c06e0e0 0x7f131c0705a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:12:35.433 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:35.433+0000 7f132f400700 1 -- 172.21.15.153:0/1713698794 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f1328006e40 con 0x7f1338102ff0 2024-03-21T10:12:35.433 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:35.433+0000 7f132d400700 1 --2- 172.21.15.153:0/1713698794 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f131c06e0e0 0x7f131c0705a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:12:35.434 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:35.434+0000 7f132d400700 1 --2- 172.21.15.153:0/1713698794 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f131c06e0e0 0x7f131c0705a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1320006010 tx=0x7f132000b540 comp rx=0 tx=0).ready entity=mgr.14666 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:12:35.438 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:35.438+0000 7f132f400700 1 -- 172.21.15.153:0/1713698794 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185454 (secure 0 0 0) 0x7f132805a8b0 con 0x7f1338102ff0 2024-03-21T10:12:35.756 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:35.755+0000 7f133ca00700 1 -- 172.21.15.153:0/1713698794 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f13381993d0 con 0x7f1338102ff0 2024-03-21T10:12:35.757 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:35.756+0000 7f132f400700 1 -- 172.21.15.153:0/1713698794 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7f132805a440 con 0x7f1338102ff0 2024-03-21T10:12:35.760 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:35.759+0000 7f133ca00700 1 -- 172.21.15.153:0/1713698794 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f131c06e0e0 msgr2=0x7f131c0705a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:12:35.760 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:35.759+0000 7f133ca00700 1 --2- 172.21.15.153:0/1713698794 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f131c06e0e0 0x7f131c0705a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1320006010 tx=0x7f132000b540 comp rx=0 tx=0).stop 2024-03-21T10:12:35.760 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:35.759+0000 7f133ca00700 1 -- 172.21.15.153:0/1713698794 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1338102ff0 msgr2=0x7f133819e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:12:35.761 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:35.759+0000 7f133ca00700 1 --2- 172.21.15.153:0/1713698794 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1338102ff0 0x7f133819e850 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f132800ba50 tx=0x7f132800be10 comp rx=0 tx=0).stop 2024-03-21T10:12:35.761 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:35.759+0000 7f133ca00700 1 -- 172.21.15.153:0/1713698794 shutdown_connections 2024-03-21T10:12:35.761 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:35.759+0000 7f133ca00700 1 --2- 172.21.15.153:0/1713698794 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f131c06e0e0 0x7f131c0705a0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:12:35.761 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:35.759+0000 7f133ca00700 1 --2- 172.21.15.153:0/1713698794 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1338102ff0 0x7f133819e850 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:12:35.761 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:35.759+0000 7f133ca00700 1 --2- 172.21.15.153:0/1713698794 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f13381039a0 0x7f133819ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:12:35.761 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:35.759+0000 7f133ca00700 1 -- 172.21.15.153:0/1713698794 >> 172.21.15.153:0/1713698794 conn(0x7f13380fe880 msgr2=0x7f1338100000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:12:35.761 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:35.760+0000 7f133ca00700 1 -- 172.21.15.153:0/1713698794 shutdown_connections 2024-03-21T10:12:35.761 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:35.760+0000 7f133ca00700 1 -- 172.21.15.153:0/1713698794 wait complete. 2024-03-21T10:12:35.777 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)" 2024-03-21T10:12:36.510 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 070ed3e4-e769-11ee-95cd-87774f69a715 -e sha1=2edfee68ad7243bed422a049b6f5f58619300f11 -- bash -c 'ceph versions | jq -e '"'"'.overall | length == 2'"'"'' 2024-03-21T10:12:36.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:12:36 smithi153 ceph-mon[29117]: pgmap v124: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:12:36.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:12:36 smithi153 ceph-mon[29117]: from='client.? 172.21.15.153:0/1713698794' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:12:36.918 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi153/config 2024-03-21T10:12:36.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:12:36 smithi174 ceph-mon[37496]: pgmap v124: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:12:36.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:12:36 smithi174 ceph-mon[37496]: from='client.? 172.21.15.153:0/1713698794' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:12:38.707 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:12:38 smithi153 ceph-mon[29117]: pgmap v125: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:12:38.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:12:38 smithi174 ceph-mon[37496]: pgmap v125: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:12:39.231 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:39.230+0000 7fd5c2c00700 1 -- 172.21.15.153:0/3246190752 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd5bc102ff0 msgr2=0x7fd5bc1033d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:12:39.232 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:39.230+0000 7fd5c2c00700 1 --2- 172.21.15.153:0/3246190752 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd5bc102ff0 0x7fd5bc1033d0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac009910 tx=0x7fd5ac009c20 comp rx=0 tx=0).stop 2024-03-21T10:12:39.232 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:39.230+0000 7fd5c2c00700 1 -- 172.21.15.153:0/3246190752 shutdown_connections 2024-03-21T10:12:39.232 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:39.230+0000 7fd5c2c00700 1 --2- 172.21.15.153:0/3246190752 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd5bc1039a0 0x7fd5bc1079f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:12:39.232 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:39.230+0000 7fd5c2c00700 1 --2- 172.21.15.153:0/3246190752 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd5bc102ff0 0x7fd5bc1033d0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:12:39.232 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:39.230+0000 7fd5c2c00700 1 -- 172.21.15.153:0/3246190752 >> 172.21.15.153:0/3246190752 conn(0x7fd5bc0fe880 msgr2=0x7fd5bc100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:12:39.232 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:39.230+0000 7fd5c2c00700 1 -- 172.21.15.153:0/3246190752 shutdown_connections 2024-03-21T10:12:39.232 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:39.230+0000 7fd5c2c00700 1 -- 172.21.15.153:0/3246190752 wait complete. 2024-03-21T10:12:39.233 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:39.231+0000 7fd5c2c00700 1 Processor -- start 2024-03-21T10:12:39.233 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:39.231+0000 7fd5c2c00700 1 -- start start 2024-03-21T10:12:39.234 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:39.232+0000 7fd5c2c00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd5bc102ff0 0x7fd5bc198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:12:39.234 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:39.232+0000 7fd5c2c00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd5bc1039a0 0x7fd5bc198eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:12:39.234 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:39.232+0000 7fd5c2c00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5bc1994b0 con 0x7fd5bc1039a0 2024-03-21T10:12:39.234 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:39.232+0000 7fd5c2c00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5bc199620 con 0x7fd5bc102ff0 2024-03-21T10:12:39.234 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:39.232+0000 7fd5bb400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd5bc1039a0 0x7fd5bc198eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:12:39.234 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:39.232+0000 7fd5bb400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd5bc1039a0 0x7fd5bc198eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:47702/0 (socket says 172.21.15.153:47702) 2024-03-21T10:12:39.234 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:39.232+0000 7fd5bb400700 1 -- 172.21.15.153:0/4279430004 learned_addr learned my addr 172.21.15.153:0/4279430004 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:12:39.234 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:39.233+0000 7fd5bbe00700 1 --2- 172.21.15.153:0/4279430004 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd5bc102ff0 0x7fd5bc198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:12:39.235 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:39.233+0000 7fd5bb400700 1 -- 172.21.15.153:0/4279430004 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd5bc102ff0 msgr2=0x7fd5bc198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:12:39.235 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:39.233+0000 7fd5bb400700 1 --2- 172.21.15.153:0/4279430004 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd5bc102ff0 0x7fd5bc198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:12:39.235 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:39.233+0000 7fd5bb400700 1 -- 172.21.15.153:0/4279430004 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5ac0095f0 con 0x7fd5bc1039a0 2024-03-21T10:12:39.235 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:39.233+0000 7fd5bb400700 1 --2- 172.21.15.153:0/4279430004 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd5bc1039a0 0x7fd5bc198eb0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fd5a800c960 tx=0x7fd5a800cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:12:39.235 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:39.233+0000 7fd5b8c00700 1 -- 172.21.15.153:0/4279430004 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd5a8007a70 con 0x7fd5bc1039a0 2024-03-21T10:12:39.235 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:39.234+0000 7fd5b8c00700 1 -- 172.21.15.153:0/4279430004 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd5a8007bd0 con 0x7fd5bc1039a0 2024-03-21T10:12:39.236 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:39.234+0000 7fd5c2c00700 1 -- 172.21.15.153:0/4279430004 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5bc19d4a0 con 0x7fd5bc1039a0 2024-03-21T10:12:39.236 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:39.234+0000 7fd5b8c00700 1 -- 172.21.15.153:0/4279430004 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd5a80186c0 con 0x7fd5bc1039a0 2024-03-21T10:12:39.236 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:39.234+0000 7fd5c2c00700 1 -- 172.21.15.153:0/4279430004 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5bc19d9f0 con 0x7fd5bc1039a0 2024-03-21T10:12:39.236 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:39.235+0000 7fd5c2c00700 1 -- 172.21.15.153:0/4279430004 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5bc10b390 con 0x7fd5bc1039a0 2024-03-21T10:12:39.237 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:39.236+0000 7fd5b8c00700 1 -- 172.21.15.153:0/4279430004 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 31) v1 ==== 92072+0+0 (secure 0 0 0) 0x7fd5a800faa0 con 0x7fd5bc1039a0 2024-03-21T10:12:39.238 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:39.237+0000 7fd5b8c00700 1 --2- 172.21.15.153:0/4279430004 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7fd5a006e0e0 0x7fd5a00705a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:12:39.238 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:39.237+0000 7fd5b8c00700 1 -- 172.21.15.153:0/4279430004 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fd5a808df20 con 0x7fd5bc1039a0 2024-03-21T10:12:39.239 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:39.237+0000 7fd5bbe00700 1 --2- 172.21.15.153:0/4279430004 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7fd5a006e0e0 0x7fd5a00705a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:12:39.239 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:39.238+0000 7fd5bbe00700 1 --2- 172.21.15.153:0/4279430004 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7fd5a006e0e0 0x7fd5a00705a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac006010 tx=0x7fd5ac01a040 comp rx=0 tx=0).ready entity=mgr.14666 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:12:39.243 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:39.242+0000 7fd5b8c00700 1 -- 172.21.15.153:0/4279430004 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185454 (secure 0 0 0) 0x7fd5a805ae00 con 0x7fd5bc1039a0 2024-03-21T10:12:39.559 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:39.556+0000 7fd5c2c00700 1 -- 172.21.15.153:0/4279430004 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fd5bc02cd30 con 0x7fd5bc1039a0 2024-03-21T10:12:39.559 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:39.558+0000 7fd5b8c00700 1 -- 172.21.15.153:0/4279430004 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7fd5a805ac20 con 0x7fd5bc1039a0 2024-03-21T10:12:39.562 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:39.561+0000 7fd5c2c00700 1 -- 172.21.15.153:0/4279430004 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7fd5a006e0e0 msgr2=0x7fd5a00705a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:12:39.563 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:39.561+0000 7fd5c2c00700 1 --2- 172.21.15.153:0/4279430004 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7fd5a006e0e0 0x7fd5a00705a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac006010 tx=0x7fd5ac01a040 comp rx=0 tx=0).stop 2024-03-21T10:12:39.563 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:39.561+0000 7fd5c2c00700 1 -- 172.21.15.153:0/4279430004 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd5bc1039a0 msgr2=0x7fd5bc198eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:12:39.563 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:39.561+0000 7fd5c2c00700 1 --2- 172.21.15.153:0/4279430004 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd5bc1039a0 0x7fd5bc198eb0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fd5a800c960 tx=0x7fd5a800cd20 comp rx=0 tx=0).stop 2024-03-21T10:12:39.563 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:39.561+0000 7fd5c2c00700 1 -- 172.21.15.153:0/4279430004 shutdown_connections 2024-03-21T10:12:39.563 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:39.561+0000 7fd5c2c00700 1 --2- 172.21.15.153:0/4279430004 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7fd5a006e0e0 0x7fd5a00705a0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:12:39.563 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:39.561+0000 7fd5c2c00700 1 --2- 172.21.15.153:0/4279430004 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd5bc102ff0 0x7fd5bc198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:12:39.563 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:39.561+0000 7fd5c2c00700 1 --2- 172.21.15.153:0/4279430004 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd5bc1039a0 0x7fd5bc198eb0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:12:39.563 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:39.561+0000 7fd5c2c00700 1 -- 172.21.15.153:0/4279430004 >> 172.21.15.153:0/4279430004 conn(0x7fd5bc0fe880 msgr2=0x7fd5bc0fff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:12:39.564 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:39.561+0000 7fd5c2c00700 1 -- 172.21.15.153:0/4279430004 shutdown_connections 2024-03-21T10:12:39.564 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:39.561+0000 7fd5c2c00700 1 -- 172.21.15.153:0/4279430004 wait complete. 2024-03-21T10:12:39.578 INFO:teuthology.orchestra.run.smithi153.stdout:true 2024-03-21T10:12:40.307 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 070ed3e4-e769-11ee-95cd-87774f69a715 -e sha1=2edfee68ad7243bed422a049b6f5f58619300f11 -- bash -c 'ceph orch upgrade check quay.ceph.io/ceph-ci/ceph:$sha1 | jq -e '"'"'.up_to_date | length == 2'"'"'' 2024-03-21T10:12:40.726 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi153/config 2024-03-21T10:12:40.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:12:40 smithi153 ceph-mon[29117]: pgmap v126: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:12:40.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:12:40 smithi153 ceph-mon[29117]: from='client.? 172.21.15.153:0/4279430004' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:12:40.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:12:40 smithi174 ceph-mon[37496]: pgmap v126: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:12:40.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:12:40 smithi174 ceph-mon[37496]: from='client.? 172.21.15.153:0/4279430004' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:12:42.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:12:42 smithi153 ceph-mon[29117]: pgmap v127: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:12:42.790 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:42.787+0000 7f4dcca00700 1 -- 172.21.15.153:0/63367062 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4dc8105b50 msgr2=0x7f4dc8068410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:12:42.790 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:42.787+0000 7f4dcca00700 1 --2- 172.21.15.153:0/63367062 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4dc8105b50 0x7f4dc8068410 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f4db800b0a0 tx=0x7f4db800b3b0 comp rx=0 tx=0).stop 2024-03-21T10:12:42.790 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:42.788+0000 7f4dcca00700 1 -- 172.21.15.153:0/63367062 shutdown_connections 2024-03-21T10:12:42.790 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:42.788+0000 7f4dcca00700 1 --2- 172.21.15.153:0/63367062 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4dc8068950 0x7f4dc8068dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:12:42.790 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:42.788+0000 7f4dcca00700 1 --2- 172.21.15.153:0/63367062 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4dc8105b50 0x7f4dc8068410 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:12:42.790 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:42.788+0000 7f4dcca00700 1 -- 172.21.15.153:0/63367062 >> 172.21.15.153:0/63367062 conn(0x7f4dc80755a0 msgr2=0x7f4dc80759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:12:42.790 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:42.788+0000 7f4dcca00700 1 -- 172.21.15.153:0/63367062 shutdown_connections 2024-03-21T10:12:42.790 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:42.788+0000 7f4dcca00700 1 -- 172.21.15.153:0/63367062 wait complete. 2024-03-21T10:12:42.791 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:42.789+0000 7f4dcca00700 1 Processor -- start 2024-03-21T10:12:42.791 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:42.789+0000 7f4dcca00700 1 -- start start 2024-03-21T10:12:42.792 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:42.789+0000 7f4dcca00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4dc8068950 0x7f4dc819ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:12:42.792 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:42.789+0000 7f4dcca00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4dc8105b50 0x7f4dc819d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:12:42.792 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:42.790+0000 7f4dcca00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dc819da90 con 0x7f4dc8105b50 2024-03-21T10:12:42.792 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:42.790+0000 7f4dcca00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dc81a1820 con 0x7f4dc8068950 2024-03-21T10:12:42.792 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:42.790+0000 7f4dc5400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4dc8105b50 0x7f4dc819d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:12:42.792 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:42.790+0000 7f4dc5400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4dc8105b50 0x7f4dc819d3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:47732/0 (socket says 172.21.15.153:47732) 2024-03-21T10:12:42.792 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:42.790+0000 7f4dc5400700 1 -- 172.21.15.153:0/3170101325 learned_addr learned my addr 172.21.15.153:0/3170101325 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:12:42.792 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:42.790+0000 7f4dc5e00700 1 --2- 172.21.15.153:0/3170101325 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4dc8068950 0x7f4dc819ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:12:42.793 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:42.790+0000 7f4dc5400700 1 -- 172.21.15.153:0/3170101325 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4dc8068950 msgr2=0x7f4dc819ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:12:42.793 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:42.790+0000 7f4dc5400700 1 --2- 172.21.15.153:0/3170101325 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4dc8068950 0x7f4dc819ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:12:42.793 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:42.790+0000 7f4dc5400700 1 -- 172.21.15.153:0/3170101325 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4db8009d00 con 0x7f4dc8105b50 2024-03-21T10:12:42.793 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:42.791+0000 7f4dc5400700 1 --2- 172.21.15.153:0/3170101325 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4dc8105b50 0x7f4dc819d3b0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f4db400b4b0 tx=0x7f4db400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:12:42.793 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:42.791+0000 7f4dbea00700 1 -- 172.21.15.153:0/3170101325 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4db4011800 con 0x7f4dc8105b50 2024-03-21T10:12:42.793 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:42.791+0000 7f4dbea00700 1 -- 172.21.15.153:0/3170101325 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4db4011e20 con 0x7f4dc8105b50 2024-03-21T10:12:42.794 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:42.791+0000 7f4dcca00700 1 -- 172.21.15.153:0/3170101325 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4dc81a1b00 con 0x7f4dc8105b50 2024-03-21T10:12:42.794 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:42.791+0000 7f4dbea00700 1 -- 172.21.15.153:0/3170101325 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4db400f2e0 con 0x7f4dc8105b50 2024-03-21T10:12:42.794 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:42.791+0000 7f4dcca00700 1 -- 172.21.15.153:0/3170101325 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dc81a2050 con 0x7f4dc8105b50 2024-03-21T10:12:42.794 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:42.792+0000 7f4dcca00700 1 -- 172.21.15.153:0/3170101325 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4dc81101a0 con 0x7f4dc8105b50 2024-03-21T10:12:42.795 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:42.793+0000 7f4dbea00700 1 -- 172.21.15.153:0/3170101325 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 31) v1 ==== 92072+0+0 (secure 0 0 0) 0x7f4db4010380 con 0x7f4dc8105b50 2024-03-21T10:12:42.796 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:42.794+0000 7f4dbea00700 1 --2- 172.21.15.153:0/3170101325 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f4dac06e0e0 0x7f4dac0705a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:12:42.796 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:42.794+0000 7f4dbea00700 1 -- 172.21.15.153:0/3170101325 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f4db408d970 con 0x7f4dc8105b50 2024-03-21T10:12:42.796 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:42.795+0000 7f4dc5e00700 1 --2- 172.21.15.153:0/3170101325 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f4dac06e0e0 0x7f4dac0705a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:12:42.797 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:42.795+0000 7f4dc5e00700 1 --2- 172.21.15.153:0/3170101325 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f4dac06e0e0 0x7f4dac0705a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4db8000f80 tx=0x7f4db800b7f0 comp rx=0 tx=0).ready entity=mgr.14666 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:12:42.801 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:42.799+0000 7f4dbea00700 1 -- 172.21.15.153:0/3170101325 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185454 (secure 0 0 0) 0x7f4db4057180 con 0x7f4dc8105b50 2024-03-21T10:12:42.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:12:42 smithi174 ceph-mon[37496]: pgmap v127: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:12:43.044 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:43.042+0000 7f4dcca00700 1 -- 172.21.15.153:0/3170101325 --> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] -- mgr_command(tid 0: {"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11", "target": ["mon-mgr", ""]}) v1 -- 0x7f4dc81a23e0 con 0x7f4dac06e0e0 2024-03-21T10:12:44.694 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:12:44 smithi153 ceph-mon[29117]: from='client.14848 -' entity='client.admin' cmd=[{"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:12:44.695 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:12:44 smithi153 ceph-mon[29117]: pgmap v128: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:12:44.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:12:44 smithi174 ceph-mon[37496]: from='client.14848 -' entity='client.admin' cmd=[{"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:12:44.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:12:44 smithi174 ceph-mon[37496]: pgmap v128: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:12:45.800 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:12:45 smithi153 ceph-mon[29117]: pgmap v129: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:12:45.922 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:45.919+0000 7f4dbea00700 1 -- 172.21.15.153:0/3170101325 <== mgr.14666 v2:172.21.15.153:6800/2524145044 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+5391 (secure 0 0 0) 0x7f4dc81a23e0 con 0x7f4dac06e0e0 2024-03-21T10:12:45.925 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:45.923+0000 7f4dcca00700 1 -- 172.21.15.153:0/3170101325 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f4dac06e0e0 msgr2=0x7f4dac0705a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:12:45.926 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:45.923+0000 7f4dcca00700 1 --2- 172.21.15.153:0/3170101325 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f4dac06e0e0 0x7f4dac0705a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4db8000f80 tx=0x7f4db800b7f0 comp rx=0 tx=0).stop 2024-03-21T10:12:45.926 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:45.924+0000 7f4dcca00700 1 -- 172.21.15.153:0/3170101325 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4dc8105b50 msgr2=0x7f4dc819d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:12:45.926 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:45.924+0000 7f4dcca00700 1 --2- 172.21.15.153:0/3170101325 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4dc8105b50 0x7f4dc819d3b0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f4db400b4b0 tx=0x7f4db400b870 comp rx=0 tx=0).stop 2024-03-21T10:12:45.926 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:45.924+0000 7f4dcca00700 1 -- 172.21.15.153:0/3170101325 shutdown_connections 2024-03-21T10:12:45.926 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:45.924+0000 7f4dcca00700 1 --2- 172.21.15.153:0/3170101325 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f4dac06e0e0 0x7f4dac0705a0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:12:45.926 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:45.924+0000 7f4dcca00700 1 --2- 172.21.15.153:0/3170101325 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4dc8068950 0x7f4dc819ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:12:45.926 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:45.924+0000 7f4dcca00700 1 --2- 172.21.15.153:0/3170101325 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4dc8105b50 0x7f4dc819d3b0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:12:45.926 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:45.924+0000 7f4dcca00700 1 -- 172.21.15.153:0/3170101325 >> 172.21.15.153:0/3170101325 conn(0x7f4dc80755a0 msgr2=0x7f4dc81015b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:12:45.926 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:45.924+0000 7f4dcca00700 1 -- 172.21.15.153:0/3170101325 shutdown_connections 2024-03-21T10:12:45.926 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:45.924+0000 7f4dcca00700 1 -- 172.21.15.153:0/3170101325 wait complete. 2024-03-21T10:12:45.944 INFO:teuthology.orchestra.run.smithi153.stdout:true 2024-03-21T10:12:45.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:12:45 smithi174 ceph-mon[37496]: pgmap v129: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:12:46.723 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 070ed3e4-e769-11ee-95cd-87774f69a715 -e sha1=2edfee68ad7243bed422a049b6f5f58619300f11 -- bash -c 'ceph orch ps' 2024-03-21T10:12:47.150 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi153/config 2024-03-21T10:12:48.382 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:12:48 smithi153 ceph-mon[29117]: pgmap v130: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:12:48.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:12:48 smithi174 ceph-mon[37496]: pgmap v130: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:12:49.347 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:49.258+0000 7f1fea400700 1 -- 172.21.15.153:0/2616423550 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1fe41071d0 msgr2=0x7f1fe4107650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:12:49.348 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:49.258+0000 7f1fea400700 1 --2- 172.21.15.153:0/2616423550 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1fe41071d0 0x7f1fe4107650 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc009910 tx=0x7f1fcc009c20 comp rx=0 tx=0).stop 2024-03-21T10:12:49.348 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:49.258+0000 7f1fea400700 1 -- 172.21.15.153:0/2616423550 shutdown_connections 2024-03-21T10:12:49.348 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:49.258+0000 7f1fea400700 1 --2- 172.21.15.153:0/2616423550 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1fe41071d0 0x7f1fe4107650 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:12:49.348 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:49.258+0000 7f1fea400700 1 --2- 172.21.15.153:0/2616423550 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1fe410d280 0x7f1fe410d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:12:49.348 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:49.258+0000 7f1fea400700 1 -- 172.21.15.153:0/2616423550 >> 172.21.15.153:0/2616423550 conn(0x7f1fe40769e0 msgr2=0x7f1fe4078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:12:49.348 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:49.258+0000 7f1fea400700 1 -- 172.21.15.153:0/2616423550 shutdown_connections 2024-03-21T10:12:49.348 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:49.258+0000 7f1fea400700 1 -- 172.21.15.153:0/2616423550 wait complete. 2024-03-21T10:12:49.348 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:49.259+0000 7f1fea400700 1 Processor -- start 2024-03-21T10:12:49.348 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:49.260+0000 7f1fea400700 1 -- start start 2024-03-21T10:12:49.348 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:49.260+0000 7f1fea400700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1fe41071d0 0x7f1fe419ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:12:49.348 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:49.260+0000 7f1fea400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1fe410d280 0x7f1fe419d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:12:49.348 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:49.260+0000 7f1fea400700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe419d880 con 0x7f1fe410d280 2024-03-21T10:12:49.349 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:49.260+0000 7f1fea400700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe41a05b0 con 0x7f1fe41071d0 2024-03-21T10:12:49.349 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:49.260+0000 7f1fdbe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1fe410d280 0x7f1fe419d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:12:49.349 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:49.261+0000 7f1fdbe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1fe410d280 0x7f1fe419d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:44362/0 (socket says 172.21.15.153:44362) 2024-03-21T10:12:49.349 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:49.261+0000 7f1fdbe00700 1 -- 172.21.15.153:0/1017910132 learned_addr learned my addr 172.21.15.153:0/1017910132 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:12:49.349 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:49.261+0000 7f1fe3400700 1 --2- 172.21.15.153:0/1017910132 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1fe41071d0 0x7f1fe419ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:12:49.349 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:49.261+0000 7f1fdbe00700 1 -- 172.21.15.153:0/1017910132 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1fe41071d0 msgr2=0x7f1fe419ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:12:49.349 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:49.261+0000 7f1fdbe00700 1 --2- 172.21.15.153:0/1017910132 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1fe41071d0 0x7f1fe419ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:12:49.349 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:49.261+0000 7f1fdbe00700 1 -- 172.21.15.153:0/1017910132 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fcc0095f0 con 0x7f1fe410d280 2024-03-21T10:12:49.349 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:49.261+0000 7f1fdbe00700 1 --2- 172.21.15.153:0/1017910132 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1fe410d280 0x7f1fe419d1f0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc0092e0 tx=0x7f1fcc00f460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:12:49.349 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:49.262+0000 7f1fe0c00700 1 -- 172.21.15.153:0/1017910132 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1fcc01d070 con 0x7f1fe410d280 2024-03-21T10:12:49.349 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:49.262+0000 7f1fe0c00700 1 -- 172.21.15.153:0/1017910132 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1fcc0052a0 con 0x7f1fe410d280 2024-03-21T10:12:49.349 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:49.262+0000 7f1fe0c00700 1 -- 172.21.15.153:0/1017910132 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1fcc017630 con 0x7f1fe410d280 2024-03-21T10:12:49.349 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:49.262+0000 7f1fea400700 1 -- 172.21.15.153:0/1017910132 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fe41a0830 con 0x7f1fe410d280 2024-03-21T10:12:49.350 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:49.262+0000 7f1fea400700 1 -- 172.21.15.153:0/1017910132 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fe41a0da0 con 0x7f1fe410d280 2024-03-21T10:12:49.350 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:49.263+0000 7f1fea400700 1 -- 172.21.15.153:0/1017910132 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fe410f6c0 con 0x7f1fe410d280 2024-03-21T10:12:49.350 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:49.264+0000 7f1fe0c00700 1 -- 172.21.15.153:0/1017910132 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 31) v1 ==== 92072+0+0 (secure 0 0 0) 0x7f1fcc0178a0 con 0x7f1fe410d280 2024-03-21T10:12:49.350 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:49.265+0000 7f1fe0c00700 1 --2- 172.21.15.153:0/1017910132 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f1fc806e0e0 0x7f1fc80705a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:12:49.350 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:49.265+0000 7f1fe0c00700 1 -- 172.21.15.153:0/1017910132 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f1fcc08f1d0 con 0x7f1fe410d280 2024-03-21T10:12:49.350 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:49.265+0000 7f1fe3400700 1 --2- 172.21.15.153:0/1017910132 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f1fc806e0e0 0x7f1fc80705a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:12:49.350 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:49.266+0000 7f1fe3400700 1 --2- 172.21.15.153:0/1017910132 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f1fc806e0e0 0x7f1fc80705a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1fd4009bf0 tx=0x7f1fd4009260 comp rx=0 tx=0).ready entity=mgr.14666 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:12:49.350 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:49.270+0000 7f1fe0c00700 1 -- 172.21.15.153:0/1017910132 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185454 (secure 0 0 0) 0x7f1fcc058890 con 0x7f1fe410d280 2024-03-21T10:12:49.350 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:12:49 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:12:49.509 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:49.506+0000 7f1fea400700 1 -- 172.21.15.153:0/1017910132 --> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f1fe41a10d0 con 0x7f1fc806e0e0 2024-03-21T10:12:49.539 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:49.537+0000 7f1fe0c00700 1 -- 172.21.15.153:0/1017910132 <== mgr.14666 v2:172.21.15.153:6800/2524145044 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f1fe41a10d0 con 0x7f1fc806e0e0 2024-03-21T10:12:49.539 INFO:teuthology.orchestra.run.smithi153.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-21T10:12:49.540 INFO:teuthology.orchestra.run.smithi153.stdout:alertmanager.smithi153 smithi153 *:9093,9094 running (99s) 59s ago 15m 13.7M - 0.25.0 c8568f914cd2 9ed3f2200b5f 2024-03-21T10:12:49.540 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi153 smithi153 running (15m) 59s ago 15m 8196k - 18.2.1 5be31c24972a 5b61994396a6 2024-03-21T10:12:49.540 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi174 smithi174 running (14m) 2m ago 14m 7940k - 18.2.1 5be31c24972a 4f7a5fc3ac28 2024-03-21T10:12:49.540 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi153 smithi153 running (15m) 59s ago 15m 6665k - 18.2.1 5be31c24972a 90a8b8ea03d1 2024-03-21T10:12:49.540 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi174 smithi174 running (14m) 2m ago 14m 6665k - 18.2.1 5be31c24972a a4c929e4dc77 2024-03-21T10:12:49.540 INFO:teuthology.orchestra.run.smithi153.stdout:grafana.smithi153 smithi153 *:3000 running (67s) 59s ago 14m 50.6M - 9.4.12 2bacad6d85d8 ee24026ab786 2024-03-21T10:12:49.540 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.iuyluj smithi153 running (8m) 59s ago 8m 129M - 18.2.1 5be31c24972a 02f667cee979 2024-03-21T10:12:49.540 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.zipckr smithi153 running (8m) 59s ago 8m 58.6M - 18.2.1 5be31c24972a 68a853929f8f 2024-03-21T10:12:49.540 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.nhozag smithi174 running (8m) 2m ago 8m 128M - 18.2.1 5be31c24972a 2fee4000308f 2024-03-21T10:12:49.540 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.yyimst smithi174 running (8m) 2m ago 8m 155M - 18.2.1 5be31c24972a d78a729220d1 2024-03-21T10:12:49.540 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi153.tluxni smithi153 *:8443,9283,8765 running (4m) 59s ago 17m 511M - 19.0.0-2231-g2edfee68 237e43c7a65f 7ffee0e55382 2024-03-21T10:12:49.540 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi174.qpcwat smithi174 *:8443,9283,8765 running (3m) 2m ago 13m 447M - 19.0.0-2231-g2edfee68 237e43c7a65f 748968f216ab 2024-03-21T10:12:49.541 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi153 smithi153 running (17m) 59s ago 17m 57.2M 2048M 18.2.1 5be31c24972a 813963a95d69 2024-03-21T10:12:49.541 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi174 smithi174 running (13m) 2m ago 13m 45.3M 2048M 18.2.1 5be31c24972a 99204923eec6 2024-03-21T10:12:49.541 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi153 smithi153 *:9100 running (2m) 59s ago 15m 7394k - 1.5.0 0da6a335fe13 bb835edc0e0b 2024-03-21T10:12:49.541 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi174 smithi174 *:9100 running (2m) 2m ago 13m 6765k - 1.5.0 0da6a335fe13 7c4ecad790fa 2024-03-21T10:12:49.541 INFO:teuthology.orchestra.run.smithi153.stdout:osd.0 smithi153 running (12m) 59s ago 12m 215M 1217M 18.2.1 5be31c24972a 28912a2319b4 2024-03-21T10:12:49.541 INFO:teuthology.orchestra.run.smithi153.stdout:osd.1 smithi153 running (12m) 59s ago 12m 214M 1217M 18.2.1 5be31c24972a ca60681e2314 2024-03-21T10:12:49.541 INFO:teuthology.orchestra.run.smithi153.stdout:osd.2 smithi153 running (11m) 59s ago 11m 172M 1217M 18.2.1 5be31c24972a 8db799f8962a 2024-03-21T10:12:49.541 INFO:teuthology.orchestra.run.smithi153.stdout:osd.3 smithi174 running (11m) 2m ago 11m 213M 2241M 18.2.1 5be31c24972a dafc89e0658d 2024-03-21T10:12:49.541 INFO:teuthology.orchestra.run.smithi153.stdout:osd.4 smithi174 running (10m) 2m ago 10m 201M 2241M 18.2.1 5be31c24972a c4cd15701f05 2024-03-21T10:12:49.541 INFO:teuthology.orchestra.run.smithi153.stdout:osd.5 smithi174 running (10m) 2m ago 10m 187M 2241M 18.2.1 5be31c24972a 3b69886ce22e 2024-03-21T10:12:49.541 INFO:teuthology.orchestra.run.smithi153.stdout:prometheus.smithi153 smithi153 *:9095 running (2m) 59s ago 14m 59.7M - 2.43.0 a07b618ecd1d 7d95561e50c2 2024-03-21T10:12:49.542 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:49.540+0000 7f1fea400700 1 -- 172.21.15.153:0/1017910132 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f1fc806e0e0 msgr2=0x7f1fc80705a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:12:49.543 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:49.540+0000 7f1fea400700 1 --2- 172.21.15.153:0/1017910132 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f1fc806e0e0 0x7f1fc80705a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1fd4009bf0 tx=0x7f1fd4009260 comp rx=0 tx=0).stop 2024-03-21T10:12:49.543 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:49.540+0000 7f1fea400700 1 -- 172.21.15.153:0/1017910132 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1fe410d280 msgr2=0x7f1fe419d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:12:49.543 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:49.540+0000 7f1fea400700 1 --2- 172.21.15.153:0/1017910132 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1fe410d280 0x7f1fe419d1f0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc0092e0 tx=0x7f1fcc00f460 comp rx=0 tx=0).stop 2024-03-21T10:12:49.543 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:49.541+0000 7f1fea400700 1 -- 172.21.15.153:0/1017910132 shutdown_connections 2024-03-21T10:12:49.543 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:49.541+0000 7f1fea400700 1 --2- 172.21.15.153:0/1017910132 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f1fc806e0e0 0x7f1fc80705a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:12:49.543 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:49.541+0000 7f1fea400700 1 --2- 172.21.15.153:0/1017910132 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1fe41071d0 0x7f1fe419ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:12:49.543 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:49.541+0000 7f1fea400700 1 --2- 172.21.15.153:0/1017910132 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1fe410d280 0x7f1fe419d1f0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:12:49.543 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:49.541+0000 7f1fea400700 1 -- 172.21.15.153:0/1017910132 >> 172.21.15.153:0/1017910132 conn(0x7f1fe40769e0 msgr2=0x7f1fe4078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:12:49.543 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:49.541+0000 7f1fea400700 1 -- 172.21.15.153:0/1017910132 shutdown_connections 2024-03-21T10:12:49.543 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:49.541+0000 7f1fea400700 1 -- 172.21.15.153:0/1017910132 wait complete. 2024-03-21T10:12:49.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:12:49 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:12:50.346 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-03-21T10:12:50.347 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi153.front.sepia.ceph.com 2024-03-21T10:12:50.347 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 070ed3e4-e769-11ee-95cd-87774f69a715 -e sha1=2edfee68ad7243bed422a049b6f5f58619300f11 -- bash -c 'ceph config set mgr mgr/orchestrator/fail_fs true' 2024-03-21T10:12:50.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:12:50 smithi153 ceph-mon[29117]: pgmap v131: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:12:50.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:12:50 smithi153 ceph-mon[29117]: from='client.14852 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:12:50.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:12:50 smithi174 ceph-mon[37496]: pgmap v131: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:12:50.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:12:50 smithi174 ceph-mon[37496]: from='client.14852 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:12:50.771 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi153/config 2024-03-21T10:12:52.454 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:12:52 smithi153 ceph-mon[29117]: pgmap v132: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:12:52.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:12:52 smithi174 ceph-mon[37496]: pgmap v132: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:12:52.942 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:52.940+0000 7f84cc600700 1 -- 172.21.15.153:0/859958478 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f84c410d280 msgr2=0x7f84c410d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:12:52.942 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:52.940+0000 7f84cc600700 1 --2- 172.21.15.153:0/859958478 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f84c410d280 0x7f84c410d660 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f84b8009910 tx=0x7f84b8009c20 comp rx=0 tx=0).stop 2024-03-21T10:12:52.942 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:52.940+0000 7f84cc600700 1 -- 172.21.15.153:0/859958478 shutdown_connections 2024-03-21T10:12:52.942 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:52.940+0000 7f84cc600700 1 --2- 172.21.15.153:0/859958478 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f84c41071d0 0x7f84c4107650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:12:52.942 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:52.940+0000 7f84cc600700 1 --2- 172.21.15.153:0/859958478 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f84c410d280 0x7f84c410d660 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:12:52.942 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:52.940+0000 7f84cc600700 1 -- 172.21.15.153:0/859958478 >> 172.21.15.153:0/859958478 conn(0x7f84c40769e0 msgr2=0x7f84c4078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:12:52.942 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:52.940+0000 7f84cc600700 1 -- 172.21.15.153:0/859958478 shutdown_connections 2024-03-21T10:12:52.943 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:52.940+0000 7f84cc600700 1 -- 172.21.15.153:0/859958478 wait complete. 2024-03-21T10:12:52.944 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:52.941+0000 7f84cc600700 1 Processor -- start 2024-03-21T10:12:52.944 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:52.941+0000 7f84cc600700 1 -- start start 2024-03-21T10:12:52.944 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:52.941+0000 7f84cc600700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f84c41071d0 0x7f84c419ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:12:52.944 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:52.941+0000 7f84cc600700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f84c410d280 0x7f84c419d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:12:52.944 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:52.941+0000 7f84cc600700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c419d810 con 0x7f84c410d280 2024-03-21T10:12:52.944 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:52.941+0000 7f84cc600700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c419d980 con 0x7f84c41071d0 2024-03-21T10:12:52.944 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:52.942+0000 7f84c9200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f84c410d280 0x7f84c419d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:12:52.944 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:52.942+0000 7f84c9200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f84c410d280 0x7f84c419d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:43854/0 (socket says 172.21.15.153:43854) 2024-03-21T10:12:52.944 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:52.942+0000 7f84c9200700 1 -- 172.21.15.153:0/569224654 learned_addr learned my addr 172.21.15.153:0/569224654 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:12:52.944 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:52.942+0000 7f84c9c00700 1 --2- 172.21.15.153:0/569224654 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f84c41071d0 0x7f84c419ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:12:52.944 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:52.942+0000 7f84c9200700 1 -- 172.21.15.153:0/569224654 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f84c41071d0 msgr2=0x7f84c419ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:12:52.945 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:52.942+0000 7f84c9200700 1 --2- 172.21.15.153:0/569224654 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f84c41071d0 0x7f84c419ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:12:52.945 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:52.942+0000 7f84c9200700 1 -- 172.21.15.153:0/569224654 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84b80095f0 con 0x7f84c410d280 2024-03-21T10:12:52.945 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:52.943+0000 7f84c9200700 1 --2- 172.21.15.153:0/569224654 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f84c410d280 0x7f84c419d210 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f84b000c930 tx=0x7f84b000ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:12:52.945 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:52.943+0000 7f84c2000700 1 -- 172.21.15.153:0/569224654 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f84b0007ad0 con 0x7f84c410d280 2024-03-21T10:12:52.945 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:52.943+0000 7f84c2000700 1 -- 172.21.15.153:0/569224654 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f84b0007c30 con 0x7f84c410d280 2024-03-21T10:12:52.946 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:52.943+0000 7f84c2000700 1 -- 172.21.15.153:0/569224654 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f84b0018610 con 0x7f84c410d280 2024-03-21T10:12:52.946 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:52.943+0000 7f84cc600700 1 -- 172.21.15.153:0/569224654 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84c41a0720 con 0x7f84c410d280 2024-03-21T10:12:52.946 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:52.943+0000 7f84cc600700 1 -- 172.21.15.153:0/569224654 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84c41a0cf0 con 0x7f84c410d280 2024-03-21T10:12:52.946 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:52.944+0000 7f84cc600700 1 -- 172.21.15.153:0/569224654 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84c404ea90 con 0x7f84c410d280 2024-03-21T10:12:52.947 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:52.945+0000 7f84c2000700 1 -- 172.21.15.153:0/569224654 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 31) v1 ==== 92072+0+0 (secure 0 0 0) 0x7f84b000f450 con 0x7f84c410d280 2024-03-21T10:12:52.948 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:52.946+0000 7f84c2000700 1 --2- 172.21.15.153:0/569224654 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f84ac06e0e0 0x7f84ac0705a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:12:52.948 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:52.946+0000 7f84c2000700 1 -- 172.21.15.153:0/569224654 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f84b008dff0 con 0x7f84c410d280 2024-03-21T10:12:52.949 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:52.947+0000 7f84c9c00700 1 --2- 172.21.15.153:0/569224654 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f84ac06e0e0 0x7f84ac0705a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:12:52.949 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:52.947+0000 7f84c9c00700 1 --2- 172.21.15.153:0/569224654 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f84ac06e0e0 0x7f84ac0705a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f84b800b5c0 tx=0x7f84b801a040 comp rx=0 tx=0).ready entity=mgr.14666 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:12:52.952 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:52.951+0000 7f84c2000700 1 -- 172.21.15.153:0/569224654 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185454 (secure 0 0 0) 0x7f84b00c2480 con 0x7f84c410d280 2024-03-21T10:12:53.174 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:53.172+0000 7f84cc600700 1 -- 172.21.15.153:0/569224654 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command([{prefix=config set, name=mgr/orchestrator/fail_fs}] v 0) v1 -- 0x7f84c41a1170 con 0x7f84c410d280 2024-03-21T10:12:53.181 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:53.179+0000 7f84c2000700 1 -- 172.21.15.153:0/569224654 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/orchestrator/fail_fs}]=0 v44)=0 v44) v1 ==== 125+0+0 (secure 0 0 0) 0x7f84b00576b0 con 0x7f84c410d280 2024-03-21T10:12:53.184 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:53.183+0000 7f84cc600700 1 -- 172.21.15.153:0/569224654 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f84ac06e0e0 msgr2=0x7f84ac0705a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:12:53.185 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:53.183+0000 7f84cc600700 1 --2- 172.21.15.153:0/569224654 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f84ac06e0e0 0x7f84ac0705a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f84b800b5c0 tx=0x7f84b801a040 comp rx=0 tx=0).stop 2024-03-21T10:12:53.185 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:53.183+0000 7f84cc600700 1 -- 172.21.15.153:0/569224654 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f84c410d280 msgr2=0x7f84c419d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:12:53.185 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:53.183+0000 7f84cc600700 1 --2- 172.21.15.153:0/569224654 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f84c410d280 0x7f84c419d210 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f84b000c930 tx=0x7f84b000ccf0 comp rx=0 tx=0).stop 2024-03-21T10:12:53.185 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:53.183+0000 7f84cc600700 1 -- 172.21.15.153:0/569224654 shutdown_connections 2024-03-21T10:12:53.185 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:53.183+0000 7f84cc600700 1 --2- 172.21.15.153:0/569224654 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f84ac06e0e0 0x7f84ac0705a0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:12:53.185 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:53.183+0000 7f84cc600700 1 --2- 172.21.15.153:0/569224654 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f84c41071d0 0x7f84c419ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:12:53.186 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:53.183+0000 7f84cc600700 1 --2- 172.21.15.153:0/569224654 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f84c410d280 0x7f84c419d210 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:12:53.186 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:53.183+0000 7f84cc600700 1 -- 172.21.15.153:0/569224654 >> 172.21.15.153:0/569224654 conn(0x7f84c40769e0 msgr2=0x7f84c4078320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:12:53.186 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:53.183+0000 7f84cc600700 1 -- 172.21.15.153:0/569224654 shutdown_connections 2024-03-21T10:12:53.186 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:53.183+0000 7f84cc600700 1 -- 172.21.15.153:0/569224654 wait complete. 2024-03-21T10:12:53.973 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-03-21T10:12:53.973 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi153.front.sepia.ceph.com 2024-03-21T10:12:53.973 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 070ed3e4-e769-11ee-95cd-87774f69a715 -e sha1=2edfee68ad7243bed422a049b6f5f58619300f11 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force' 2024-03-21T10:12:54.388 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi153/config 2024-03-21T10:12:54.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:12:54 smithi174 ceph-mon[37496]: from='client.? 172.21.15.153:0/569224654' entity='client.admin' 2024-03-21T10:12:54.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:12:54 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:12:54.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:12:54 smithi174 ceph-mon[37496]: pgmap v133: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:12:54.536 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:12:54 smithi153 ceph-mon[29117]: from='client.? 172.21.15.153:0/569224654' entity='client.admin' 2024-03-21T10:12:54.536 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:12:54 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:12:54.536 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:12:54 smithi153 ceph-mon[29117]: pgmap v133: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:12:55.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:12:55 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:12:55.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:12:55 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:12:55.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:12:55 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:12:55.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:12:55 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:12:55.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:12:55 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:12:55.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:12:55 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:12:56.589 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:56.586+0000 7f4cdd400700 1 -- 172.21.15.153:0/2587801750 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4cd81069a0 msgr2=0x7f4cd8106d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:12:56.589 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:56.586+0000 7f4cdd400700 1 --2- 172.21.15.153:0/2587801750 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4cd81069a0 0x7f4cd8106d80 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f4cc8009910 tx=0x7f4cc8009c20 comp rx=0 tx=0).stop 2024-03-21T10:12:56.589 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:56.586+0000 7f4cdd400700 1 -- 172.21.15.153:0/2587801750 shutdown_connections 2024-03-21T10:12:56.589 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:56.586+0000 7f4cdd400700 1 --2- 172.21.15.153:0/2587801750 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4cd81009a0 0x7f4cd8100e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:12:56.589 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:56.586+0000 7f4cdd400700 1 --2- 172.21.15.153:0/2587801750 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4cd81069a0 0x7f4cd8106d80 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:12:56.589 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:56.586+0000 7f4cdd400700 1 -- 172.21.15.153:0/2587801750 >> 172.21.15.153:0/2587801750 conn(0x7f4cd80fc4d0 msgr2=0x7f4cd80fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:12:56.589 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:56.586+0000 7f4cdd400700 1 -- 172.21.15.153:0/2587801750 shutdown_connections 2024-03-21T10:12:56.589 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:56.587+0000 7f4cdd400700 1 -- 172.21.15.153:0/2587801750 wait complete. 2024-03-21T10:12:56.590 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:56.588+0000 7f4cdd400700 1 Processor -- start 2024-03-21T10:12:56.590 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:56.588+0000 7f4cdd400700 1 -- start start 2024-03-21T10:12:56.590 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:56.589+0000 7f4cdd400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4cd81009a0 0x7f4cd81965e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:12:56.590 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:56.589+0000 7f4cdd400700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4cd81069a0 0x7f4cd8196b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:12:56.590 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:56.589+0000 7f4cdd400700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cd8197200 con 0x7f4cd81009a0 2024-03-21T10:12:56.590 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:56.589+0000 7f4cdd400700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cd819af90 con 0x7f4cd81069a0 2024-03-21T10:12:56.592 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:56.589+0000 7f4cd7e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4cd81009a0 0x7f4cd81965e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:12:56.592 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:56.589+0000 7f4cd7e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4cd81009a0 0x7f4cd81965e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:43864/0 (socket says 172.21.15.153:43864) 2024-03-21T10:12:56.592 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:56.589+0000 7f4cd7e00700 1 -- 172.21.15.153:0/1662071348 learned_addr learned my addr 172.21.15.153:0/1662071348 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:12:56.592 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:56.589+0000 7f4ccf400700 1 --2- 172.21.15.153:0/1662071348 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4cd81069a0 0x7f4cd8196b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:12:56.592 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:56.589+0000 7f4cd7e00700 1 -- 172.21.15.153:0/1662071348 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4cd81069a0 msgr2=0x7f4cd8196b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:12:56.592 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:56.589+0000 7f4cd7e00700 1 --2- 172.21.15.153:0/1662071348 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4cd81069a0 0x7f4cd8196b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:12:56.592 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:56.589+0000 7f4cd7e00700 1 -- 172.21.15.153:0/1662071348 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cc80095f0 con 0x7f4cd81009a0 2024-03-21T10:12:56.592 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:56.590+0000 7f4cd7e00700 1 --2- 172.21.15.153:0/1662071348 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4cd81009a0 0x7f4cd81965e0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f4cc8006010 tx=0x7f4cc800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:12:56.592 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:56.590+0000 7f4cd5600700 1 -- 172.21.15.153:0/1662071348 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4cc801d070 con 0x7f4cd81009a0 2024-03-21T10:12:56.592 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:56.590+0000 7f4cd5600700 1 -- 172.21.15.153:0/1662071348 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4cc800f460 con 0x7f4cd81009a0 2024-03-21T10:12:56.592 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:56.590+0000 7f4cd5600700 1 -- 172.21.15.153:0/1662071348 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4cc8017630 con 0x7f4cd81009a0 2024-03-21T10:12:56.592 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:56.590+0000 7f4cdd400700 1 -- 172.21.15.153:0/1662071348 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cd819b210 con 0x7f4cd81009a0 2024-03-21T10:12:56.592 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:56.590+0000 7f4cdd400700 1 -- 172.21.15.153:0/1662071348 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cd819b6b0 con 0x7f4cd81009a0 2024-03-21T10:12:56.593 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:56.591+0000 7f4cdd400700 1 -- 172.21.15.153:0/1662071348 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cd8108fa0 con 0x7f4cd81009a0 2024-03-21T10:12:56.594 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:56.592+0000 7f4cd5600700 1 -- 172.21.15.153:0/1662071348 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 31) v1 ==== 92072+0+0 (secure 0 0 0) 0x7f4cc800f5d0 con 0x7f4cd81009a0 2024-03-21T10:12:56.595 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:56.593+0000 7f4cd5600700 1 --2- 172.21.15.153:0/1662071348 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f4cbc06e0e0 0x7f4cbc0705a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:12:56.595 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:56.593+0000 7f4cd5600700 1 -- 172.21.15.153:0/1662071348 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f4cc808f2f0 con 0x7f4cd81009a0 2024-03-21T10:12:56.595 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:56.594+0000 7f4ccf400700 1 --2- 172.21.15.153:0/1662071348 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f4cbc06e0e0 0x7f4cbc0705a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:12:56.596 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:56.595+0000 7f4ccf400700 1 --2- 172.21.15.153:0/1662071348 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f4cbc06e0e0 0x7f4cbc0705a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4cc0005950 tx=0x7f4cc00058e0 comp rx=0 tx=0).ready entity=mgr.14666 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:12:56.600 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:56.598+0000 7f4cd5600700 1 -- 172.21.15.153:0/1662071348 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185454 (secure 0 0 0) 0x7f4cc80589b0 con 0x7f4cd81009a0 2024-03-21T10:12:56.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:12:56 smithi174 ceph-mon[37496]: pgmap v134: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:12:56.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:12:56 smithi153 ceph-mon[29117]: pgmap v134: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:12:56.822 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:56.820+0000 7f4cdd400700 1 -- 172.21.15.153:0/1662071348 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}] v 0) v1 -- 0x7f4cd804f2e0 con 0x7f4cd81009a0 2024-03-21T10:12:56.823 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:56.821+0000 7f4cd5600700 1 -- 172.21.15.153:0/1662071348 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}]=0 v44)=0 v44) v1 ==== 155+0+0 (secure 0 0 0) 0x7f4cc805c070 con 0x7f4cd81009a0 2024-03-21T10:12:56.826 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:56.824+0000 7f4cdd400700 1 -- 172.21.15.153:0/1662071348 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f4cbc06e0e0 msgr2=0x7f4cbc0705a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:12:56.826 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:56.824+0000 7f4cdd400700 1 --2- 172.21.15.153:0/1662071348 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f4cbc06e0e0 0x7f4cbc0705a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4cc0005950 tx=0x7f4cc00058e0 comp rx=0 tx=0).stop 2024-03-21T10:12:56.826 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:56.824+0000 7f4cdd400700 1 -- 172.21.15.153:0/1662071348 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4cd81009a0 msgr2=0x7f4cd81965e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:12:56.826 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:56.824+0000 7f4cdd400700 1 --2- 172.21.15.153:0/1662071348 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4cd81009a0 0x7f4cd81965e0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f4cc8006010 tx=0x7f4cc800be20 comp rx=0 tx=0).stop 2024-03-21T10:12:56.826 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:56.824+0000 7f4cdd400700 1 -- 172.21.15.153:0/1662071348 shutdown_connections 2024-03-21T10:12:56.827 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:56.824+0000 7f4cdd400700 1 --2- 172.21.15.153:0/1662071348 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f4cbc06e0e0 0x7f4cbc0705a0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:12:56.827 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:56.824+0000 7f4cdd400700 1 --2- 172.21.15.153:0/1662071348 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4cd81009a0 0x7f4cd81965e0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:12:56.827 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:56.824+0000 7f4cdd400700 1 --2- 172.21.15.153:0/1662071348 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4cd81069a0 0x7f4cd8196b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:12:56.827 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:56.824+0000 7f4cdd400700 1 -- 172.21.15.153:0/1662071348 >> 172.21.15.153:0/1662071348 conn(0x7f4cd80fc4d0 msgr2=0x7f4cd80fce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:12:56.827 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:56.825+0000 7f4cdd400700 1 -- 172.21.15.153:0/1662071348 shutdown_connections 2024-03-21T10:12:56.827 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:12:56.825+0000 7f4cdd400700 1 -- 172.21.15.153:0/1662071348 wait complete. 2024-03-21T10:12:57.740 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 070ed3e4-e769-11ee-95cd-87774f69a715 -e sha1=2edfee68ad7243bed422a049b6f5f58619300f11 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force' 2024-03-21T10:12:58.165 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi153/config 2024-03-21T10:12:58.627 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:12:58 smithi153 ceph-mon[29117]: pgmap v135: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:12:58.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:12:58 smithi174 ceph-mon[37496]: pgmap v135: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:13:00.460 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:00.458+0000 7f2363800700 1 -- 172.21.15.153:0/1939552474 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f235c103000 msgr2=0x7f235c1033e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:13:00.460 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:00.458+0000 7f2363800700 1 --2- 172.21.15.153:0/1939552474 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f235c103000 0x7f235c1033e0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f2350009910 tx=0x7f2350009c20 comp rx=0 tx=0).stop 2024-03-21T10:13:00.460 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:00.458+0000 7f2363800700 1 -- 172.21.15.153:0/1939552474 shutdown_connections 2024-03-21T10:13:00.460 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:00.458+0000 7f2363800700 1 --2- 172.21.15.153:0/1939552474 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f235c1039b0 0x7f235c107a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:00.460 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:00.458+0000 7f2363800700 1 --2- 172.21.15.153:0/1939552474 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f235c103000 0x7f235c1033e0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:00.461 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:00.458+0000 7f2363800700 1 -- 172.21.15.153:0/1939552474 >> 172.21.15.153:0/1939552474 conn(0x7f235c0fe8d0 msgr2=0x7f235c100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:13:00.461 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:00.458+0000 7f2363800700 1 -- 172.21.15.153:0/1939552474 shutdown_connections 2024-03-21T10:13:00.461 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:00.458+0000 7f2363800700 1 -- 172.21.15.153:0/1939552474 wait complete. 2024-03-21T10:13:00.462 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:00.459+0000 7f2363800700 1 Processor -- start 2024-03-21T10:13:00.462 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:00.459+0000 7f2363800700 1 -- start start 2024-03-21T10:13:00.462 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:00.460+0000 7f2363800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f235c103000 0x7f235c19e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:13:00.462 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:00.460+0000 7f2363800700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f235c1039b0 0x7f235c19ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:13:00.462 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:00.460+0000 7f2363800700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f235c19f3c0 con 0x7f235c103000 2024-03-21T10:13:00.463 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:00.460+0000 7f2363800700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f235c198870 con 0x7f235c1039b0 2024-03-21T10:13:00.463 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:00.460+0000 7f2360e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f235c103000 0x7f235c19e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:13:00.463 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:00.460+0000 7f2360e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f235c103000 0x7f235c19e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:43890/0 (socket says 172.21.15.153:43890) 2024-03-21T10:13:00.463 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:00.460+0000 7f2360e00700 1 -- 172.21.15.153:0/1592802704 learned_addr learned my addr 172.21.15.153:0/1592802704 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:13:00.463 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:00.461+0000 7f235be00700 1 --2- 172.21.15.153:0/1592802704 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f235c1039b0 0x7f235c19ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:13:00.463 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:00.461+0000 7f2360e00700 1 -- 172.21.15.153:0/1592802704 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f235c1039b0 msgr2=0x7f235c19ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:13:00.463 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:00.461+0000 7f2360e00700 1 --2- 172.21.15.153:0/1592802704 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f235c1039b0 0x7f235c19ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:00.463 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:00.461+0000 7f2360e00700 1 -- 172.21.15.153:0/1592802704 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23500095f0 con 0x7f235c103000 2024-03-21T10:13:00.464 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:00.461+0000 7f2360e00700 1 --2- 172.21.15.153:0/1592802704 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f235c103000 0x7f235c19e7f0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f235000b5c0 tx=0x7f235000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:13:00.464 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:00.461+0000 7f2359600700 1 -- 172.21.15.153:0/1592802704 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f235001d070 con 0x7f235c103000 2024-03-21T10:13:00.464 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:00.461+0000 7f2359600700 1 -- 172.21.15.153:0/1592802704 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2350005e00 con 0x7f235c103000 2024-03-21T10:13:00.464 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:00.461+0000 7f2363800700 1 -- 172.21.15.153:0/1592802704 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f235c198af0 con 0x7f235c103000 2024-03-21T10:13:00.464 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:00.461+0000 7f2359600700 1 -- 172.21.15.153:0/1592802704 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2350017400 con 0x7f235c103000 2024-03-21T10:13:00.464 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:00.462+0000 7f2363800700 1 -- 172.21.15.153:0/1592802704 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f235c198fe0 con 0x7f235c103000 2024-03-21T10:13:00.465 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:00.463+0000 7f2363800700 1 -- 172.21.15.153:0/1592802704 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f235c10b3a0 con 0x7f235c103000 2024-03-21T10:13:00.465 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:00.464+0000 7f2359600700 1 -- 172.21.15.153:0/1592802704 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 31) v1 ==== 92072+0+0 (secure 0 0 0) 0x7f235000f460 con 0x7f235c103000 2024-03-21T10:13:00.467 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:00.465+0000 7f2359600700 1 --2- 172.21.15.153:0/1592802704 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f234406e0e0 0x7f23440705a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:13:00.467 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:00.465+0000 7f2359600700 1 -- 172.21.15.153:0/1592802704 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f235008f400 con 0x7f235c103000 2024-03-21T10:13:00.467 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:00.465+0000 7f235be00700 1 --2- 172.21.15.153:0/1592802704 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f234406e0e0 0x7f23440705a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:13:00.467 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:00.466+0000 7f235be00700 1 --2- 172.21.15.153:0/1592802704 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f234406e0e0 0x7f23440705a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2348005950 tx=0x7f23480058e0 comp rx=0 tx=0).ready entity=mgr.14666 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:13:00.472 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:00.470+0000 7f2359600700 1 -- 172.21.15.153:0/1592802704 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185454 (secure 0 0 0) 0x7f235005c570 con 0x7f235c103000 2024-03-21T10:13:00.701 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:00.699+0000 7f2363800700 1 -- 172.21.15.153:0/1592802704 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}] v 0) v1 -- 0x7f235c04ea90 con 0x7f235c103000 2024-03-21T10:13:00.701 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:00 smithi153 ceph-mon[29117]: pgmap v136: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:13:00.702 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:00.700+0000 7f2359600700 1 -- 172.21.15.153:0/1592802704 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}]=0 v44)=0 v44) v1 ==== 163+0+0 (secure 0 0 0) 0x7f235005c100 con 0x7f235c103000 2024-03-21T10:13:00.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:00 smithi174 ceph-mon[37496]: pgmap v136: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:13:00.704 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:00.703+0000 7f2363800700 1 -- 172.21.15.153:0/1592802704 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f234406e0e0 msgr2=0x7f23440705a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:13:00.705 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:00.703+0000 7f2363800700 1 --2- 172.21.15.153:0/1592802704 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f234406e0e0 0x7f23440705a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2348005950 tx=0x7f23480058e0 comp rx=0 tx=0).stop 2024-03-21T10:13:00.705 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:00.703+0000 7f2363800700 1 -- 172.21.15.153:0/1592802704 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f235c103000 msgr2=0x7f235c19e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:13:00.705 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:00.703+0000 7f2363800700 1 --2- 172.21.15.153:0/1592802704 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f235c103000 0x7f235c19e7f0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f235000b5c0 tx=0x7f235000be20 comp rx=0 tx=0).stop 2024-03-21T10:13:00.705 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:00.703+0000 7f2363800700 1 -- 172.21.15.153:0/1592802704 shutdown_connections 2024-03-21T10:13:00.705 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:00.703+0000 7f2363800700 1 --2- 172.21.15.153:0/1592802704 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f234406e0e0 0x7f23440705a0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:00.705 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:00.703+0000 7f2363800700 1 --2- 172.21.15.153:0/1592802704 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f235c103000 0x7f235c19e7f0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:00.705 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:00.703+0000 7f2363800700 1 --2- 172.21.15.153:0/1592802704 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f235c1039b0 0x7f235c19ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:00.705 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:00.703+0000 7f2363800700 1 -- 172.21.15.153:0/1592802704 >> 172.21.15.153:0/1592802704 conn(0x7f235c0fe8d0 msgr2=0x7f235c0ffee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:13:00.705 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:00.703+0000 7f2363800700 1 -- 172.21.15.153:0/1592802704 shutdown_connections 2024-03-21T10:13:00.705 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:00.703+0000 7f2363800700 1 -- 172.21.15.153:0/1592802704 wait complete. 2024-03-21T10:13:01.668 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 070ed3e4-e769-11ee-95cd-87774f69a715 -e sha1=2edfee68ad7243bed422a049b6f5f58619300f11 -- bash -c 'ceph config set global log_to_journald false --force' 2024-03-21T10:13:02.090 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi153/config 2024-03-21T10:13:02.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:02 smithi153 ceph-mon[29117]: pgmap v137: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:13:02.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:02 smithi153 ceph-mon[29117]: fsmap cephfs:2 {0=cephfs.smithi153.iuyluj=up:active,1=cephfs.smithi174.yyimst=up:active} 2 up:standby-replay 2024-03-21T10:13:02.769 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:02 smithi174 ceph-mon[37496]: pgmap v137: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:13:02.770 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:02 smithi174 ceph-mon[37496]: fsmap cephfs:2 {0=cephfs.smithi153.iuyluj=up:active,1=cephfs.smithi174.yyimst=up:active} 2 up:standby-replay 2024-03-21T10:13:03.712 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:03 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:13:03.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:03 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:13:04.280 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:04.277+0000 7f298ba00700 1 -- 172.21.15.153:0/2807016246 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2984103000 msgr2=0x7f29841033e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:13:04.280 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:04.277+0000 7f298ba00700 1 --2- 172.21.15.153:0/2807016246 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2984103000 0x7f29841033e0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f2978009910 tx=0x7f2978009c20 comp rx=0 tx=0).stop 2024-03-21T10:13:04.280 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:04.278+0000 7f298ba00700 1 -- 172.21.15.153:0/2807016246 shutdown_connections 2024-03-21T10:13:04.280 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:04.278+0000 7f298ba00700 1 --2- 172.21.15.153:0/2807016246 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f29841039b0 0x7f2984107a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:04.280 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:04.278+0000 7f298ba00700 1 --2- 172.21.15.153:0/2807016246 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2984103000 0x7f29841033e0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:04.280 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:04.278+0000 7f298ba00700 1 -- 172.21.15.153:0/2807016246 >> 172.21.15.153:0/2807016246 conn(0x7f29840fe8d0 msgr2=0x7f2984100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:13:04.280 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:04.278+0000 7f298ba00700 1 -- 172.21.15.153:0/2807016246 shutdown_connections 2024-03-21T10:13:04.280 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:04.278+0000 7f298ba00700 1 -- 172.21.15.153:0/2807016246 wait complete. 2024-03-21T10:13:04.281 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:04.279+0000 7f298ba00700 1 Processor -- start 2024-03-21T10:13:04.281 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:04.279+0000 7f298ba00700 1 -- start start 2024-03-21T10:13:04.281 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:04.279+0000 7f298ba00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2984103000 0x7f298419e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:13:04.281 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:04.279+0000 7f298ba00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f29841039b0 0x7f298419ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:13:04.281 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:04.279+0000 7f298ba00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f298419f4d0 con 0x7f2984103000 2024-03-21T10:13:04.281 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:04.279+0000 7f298ba00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2984198980 con 0x7f29841039b0 2024-03-21T10:13:04.282 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:04.280+0000 7f2989000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2984103000 0x7f298419e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:13:04.282 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:04.280+0000 7f2989000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2984103000 0x7f298419e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41846/0 (socket says 172.21.15.153:41846) 2024-03-21T10:13:04.282 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:04.280+0000 7f2989000700 1 -- 172.21.15.153:0/3503496772 learned_addr learned my addr 172.21.15.153:0/3503496772 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:13:04.282 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:04.280+0000 7f2983e00700 1 --2- 172.21.15.153:0/3503496772 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f29841039b0 0x7f298419ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:13:04.282 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:04.280+0000 7f2989000700 1 -- 172.21.15.153:0/3503496772 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f29841039b0 msgr2=0x7f298419ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:13:04.283 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:04.280+0000 7f2989000700 1 --2- 172.21.15.153:0/3503496772 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f29841039b0 0x7f298419ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:04.283 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:04.280+0000 7f2989000700 1 -- 172.21.15.153:0/3503496772 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29780095f0 con 0x7f2984103000 2024-03-21T10:13:04.283 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:04.280+0000 7f2983e00700 1 --2- 172.21.15.153:0/3503496772 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f29841039b0 0x7f298419ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:13:04.283 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:04.281+0000 7f2989000700 1 --2- 172.21.15.153:0/3503496772 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2984103000 0x7f298419e900 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f297800b5c0 tx=0x7f297800be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:13:04.283 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:04.281+0000 7f2981600700 1 -- 172.21.15.153:0/3503496772 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f297801d070 con 0x7f2984103000 2024-03-21T10:13:04.283 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:04.281+0000 7f2981600700 1 -- 172.21.15.153:0/3503496772 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f297800f460 con 0x7f2984103000 2024-03-21T10:13:04.283 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:04.281+0000 7f2981600700 1 -- 172.21.15.153:0/3503496772 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2978017610 con 0x7f2984103000 2024-03-21T10:13:04.283 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:04.281+0000 7f298ba00700 1 -- 172.21.15.153:0/3503496772 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2984198c00 con 0x7f2984103000 2024-03-21T10:13:04.284 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:04.281+0000 7f298ba00700 1 -- 172.21.15.153:0/3503496772 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29841990f0 con 0x7f2984103000 2024-03-21T10:13:04.284 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:04.283+0000 7f298ba00700 1 -- 172.21.15.153:0/3503496772 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f298410b3a0 con 0x7f2984103000 2024-03-21T10:13:04.285 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:04.284+0000 7f2981600700 1 -- 172.21.15.153:0/3503496772 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 31) v1 ==== 92072+0+0 (secure 0 0 0) 0x7f297800f5d0 con 0x7f2984103000 2024-03-21T10:13:04.286 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:04.285+0000 7f2981600700 1 --2- 172.21.15.153:0/3503496772 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f296c06e0e0 0x7f296c0705a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:13:04.286 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:04.285+0000 7f2981600700 1 -- 172.21.15.153:0/3503496772 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f297808f460 con 0x7f2984103000 2024-03-21T10:13:04.287 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:04.285+0000 7f2983e00700 1 --2- 172.21.15.153:0/3503496772 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f296c06e0e0 0x7f296c0705a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:13:04.287 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:04.286+0000 7f2983e00700 1 --2- 172.21.15.153:0/3503496772 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f296c06e0e0 0x7f296c0705a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f29840ff140 tx=0x7f2970008040 comp rx=0 tx=0).ready entity=mgr.14666 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:13:04.291 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:04.290+0000 7f2981600700 1 -- 172.21.15.153:0/3503496772 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185454 (secure 0 0 0) 0x7f297805c650 con 0x7f2984103000 2024-03-21T10:13:04.518 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:04.515+0000 7f298ba00700 1 -- 172.21.15.153:0/3503496772 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command([{prefix=config set, name=log_to_journald}] v 0) v1 -- 0x7f298404ea90 con 0x7f2984103000 2024-03-21T10:13:04.518 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:04.516+0000 7f2981600700 1 -- 172.21.15.153:0/3503496772 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{prefix=config set, name=log_to_journald}]=0 v44)=0 v44) v1 ==== 135+0+0 (secure 0 0 0) 0x7f297805c1e0 con 0x7f2984103000 2024-03-21T10:13:04.521 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:04.519+0000 7f298ba00700 1 -- 172.21.15.153:0/3503496772 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f296c06e0e0 msgr2=0x7f296c0705a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:13:04.521 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:04.519+0000 7f298ba00700 1 --2- 172.21.15.153:0/3503496772 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f296c06e0e0 0x7f296c0705a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f29840ff140 tx=0x7f2970008040 comp rx=0 tx=0).stop 2024-03-21T10:13:04.521 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:04.519+0000 7f298ba00700 1 -- 172.21.15.153:0/3503496772 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2984103000 msgr2=0x7f298419e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:13:04.521 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:04.519+0000 7f298ba00700 1 --2- 172.21.15.153:0/3503496772 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2984103000 0x7f298419e900 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f297800b5c0 tx=0x7f297800be00 comp rx=0 tx=0).stop 2024-03-21T10:13:04.521 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:04.519+0000 7f298ba00700 1 -- 172.21.15.153:0/3503496772 shutdown_connections 2024-03-21T10:13:04.521 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:04.519+0000 7f298ba00700 1 --2- 172.21.15.153:0/3503496772 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f296c06e0e0 0x7f296c0705a0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:04.521 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:04.519+0000 7f298ba00700 1 --2- 172.21.15.153:0/3503496772 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2984103000 0x7f298419e900 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:04.521 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:04.519+0000 7f298ba00700 1 --2- 172.21.15.153:0/3503496772 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f29841039b0 0x7f298419ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:04.521 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:04.519+0000 7f298ba00700 1 -- 172.21.15.153:0/3503496772 >> 172.21.15.153:0/3503496772 conn(0x7f29840fe8d0 msgr2=0x7f29840fff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:13:04.521 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:04.519+0000 7f298ba00700 1 -- 172.21.15.153:0/3503496772 shutdown_connections 2024-03-21T10:13:04.521 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:04.519+0000 7f298ba00700 1 -- 172.21.15.153:0/3503496772 wait complete. 2024-03-21T10:13:04.539 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:04 smithi153 ceph-mon[29117]: pgmap v138: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:13:04.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:04 smithi174 ceph-mon[37496]: pgmap v138: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:13:05.450 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 070ed3e4-e769-11ee-95cd-87774f69a715 -e sha1=2edfee68ad7243bed422a049b6f5f58619300f11 -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1' 2024-03-21T10:13:05.854 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi153/config 2024-03-21T10:13:06.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:06 smithi153 ceph-mon[29117]: pgmap v139: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:13:06.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:06 smithi174 ceph-mon[37496]: pgmap v139: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:13:08.006 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:08.004+0000 7f1392a00700 1 -- 172.21.15.153:0/3260434641 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f138c102fe0 msgr2=0x7f138c1033c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:13:08.006 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:08.004+0000 7f1392a00700 1 --2- 172.21.15.153:0/3260434641 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f138c102fe0 0x7f138c1033c0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f137c009910 tx=0x7f137c009c20 comp rx=0 tx=0).stop 2024-03-21T10:13:08.006 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:08.004+0000 7f1392a00700 1 -- 172.21.15.153:0/3260434641 shutdown_connections 2024-03-21T10:13:08.006 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:08.004+0000 7f1392a00700 1 --2- 172.21.15.153:0/3260434641 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f138c103990 0x7f138c1079e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:08.006 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:08.004+0000 7f1392a00700 1 --2- 172.21.15.153:0/3260434641 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f138c102fe0 0x7f138c1033c0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:08.007 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:08.004+0000 7f1392a00700 1 -- 172.21.15.153:0/3260434641 >> 172.21.15.153:0/3260434641 conn(0x7f138c0fe8d0 msgr2=0x7f138c100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:13:08.007 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:08.004+0000 7f1392a00700 1 -- 172.21.15.153:0/3260434641 shutdown_connections 2024-03-21T10:13:08.007 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:08.004+0000 7f1392a00700 1 -- 172.21.15.153:0/3260434641 wait complete. 2024-03-21T10:13:08.007 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:08.006+0000 7f1392a00700 1 Processor -- start 2024-03-21T10:13:08.007 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:08.006+0000 7f1392a00700 1 -- start start 2024-03-21T10:13:08.008 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:08.006+0000 7f1392a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f138c102fe0 0x7f138c1989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:13:08.009 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:08.006+0000 7f1392a00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f138c103990 0x7f138c198ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:13:08.009 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:08.006+0000 7f1392a00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f138c1994e0 con 0x7f138c102fe0 2024-03-21T10:13:08.009 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:08.006+0000 7f1392a00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f138c199650 con 0x7f138c103990 2024-03-21T10:13:08.009 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:08.006+0000 7f138be00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f138c102fe0 0x7f138c1989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:13:08.009 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:08.007+0000 7f138be00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f138c102fe0 0x7f138c1989a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41858/0 (socket says 172.21.15.153:41858) 2024-03-21T10:13:08.009 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:08.007+0000 7f138be00700 1 -- 172.21.15.153:0/160143567 learned_addr learned my addr 172.21.15.153:0/160143567 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:13:08.009 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:08.007+0000 7f1383400700 1 --2- 172.21.15.153:0/160143567 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f138c103990 0x7f138c198ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:13:08.009 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:08.007+0000 7f138be00700 1 -- 172.21.15.153:0/160143567 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f138c103990 msgr2=0x7f138c198ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:13:08.009 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:08.007+0000 7f138be00700 1 --2- 172.21.15.153:0/160143567 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f138c103990 0x7f138c198ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:08.009 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:08.007+0000 7f138be00700 1 -- 172.21.15.153:0/160143567 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f137c0095f0 con 0x7f138c102fe0 2024-03-21T10:13:08.009 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:08.007+0000 7f138be00700 1 --2- 172.21.15.153:0/160143567 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f138c102fe0 0x7f138c1989a0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f137c00b5c0 tx=0x7f137c00be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:13:08.010 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:08.007+0000 7f1389600700 1 -- 172.21.15.153:0/160143567 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f137c01d070 con 0x7f138c102fe0 2024-03-21T10:13:08.010 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:08.008+0000 7f1389600700 1 -- 172.21.15.153:0/160143567 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f137c00f460 con 0x7f138c102fe0 2024-03-21T10:13:08.010 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:08.008+0000 7f1392a00700 1 -- 172.21.15.153:0/160143567 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f138c19d470 con 0x7f138c102fe0 2024-03-21T10:13:08.010 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:08.008+0000 7f1389600700 1 -- 172.21.15.153:0/160143567 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f137c017610 con 0x7f138c102fe0 2024-03-21T10:13:08.010 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:08.008+0000 7f1392a00700 1 -- 172.21.15.153:0/160143567 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f138c19d960 con 0x7f138c102fe0 2024-03-21T10:13:08.011 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:08.009+0000 7f1392a00700 1 -- 172.21.15.153:0/160143567 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f138c04ea90 con 0x7f138c102fe0 2024-03-21T10:13:08.011 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:08.010+0000 7f1389600700 1 -- 172.21.15.153:0/160143567 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 31) v1 ==== 92072+0+0 (secure 0 0 0) 0x7f137c00f5d0 con 0x7f138c102fe0 2024-03-21T10:13:08.013 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:08.011+0000 7f1389600700 1 --2- 172.21.15.153:0/160143567 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f137006e0e0 0x7f13700705a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:13:08.013 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:08.011+0000 7f1389600700 1 -- 172.21.15.153:0/160143567 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f137c08f4e0 con 0x7f138c102fe0 2024-03-21T10:13:08.013 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:08.011+0000 7f1383400700 1 --2- 172.21.15.153:0/160143567 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f137006e0e0 0x7f13700705a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:13:08.013 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:08.012+0000 7f1383400700 1 --2- 172.21.15.153:0/160143567 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f137006e0e0 0x7f13700705a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f138c199fc0 tx=0x7f1374009040 comp rx=0 tx=0).ready entity=mgr.14666 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:13:08.018 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:08.017+0000 7f1389600700 1 -- 172.21.15.153:0/160143567 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185454 (secure 0 0 0) 0x7f137c05c650 con 0x7f138c102fe0 2024-03-21T10:13:08.272 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:08.270+0000 7f1392a00700 1 -- 172.21.15.153:0/160143567 --> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] -- mgr_command(tid 0: {"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11", "target": ["mon-mgr", ""]}) v1 -- 0x7f138c10b070 con 0x7f137006e0e0 2024-03-21T10:13:08.280 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:08.278+0000 7f1389600700 1 -- 172.21.15.153:0/160143567 <== mgr.14666 v2:172.21.15.153:6800/2524145044 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+89 (secure 0 0 0) 0x7f138c10b070 con 0x7f137006e0e0 2024-03-21T10:13:08.280 INFO:teuthology.orchestra.run.smithi153.stdout:Initiating upgrade to quay.ceph.io/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 2024-03-21T10:13:08.283 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:08.281+0000 7f1392a00700 1 -- 172.21.15.153:0/160143567 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f137006e0e0 msgr2=0x7f13700705a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:13:08.283 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:08.281+0000 7f1392a00700 1 --2- 172.21.15.153:0/160143567 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f137006e0e0 0x7f13700705a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f138c199fc0 tx=0x7f1374009040 comp rx=0 tx=0).stop 2024-03-21T10:13:08.283 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:08.281+0000 7f1392a00700 1 -- 172.21.15.153:0/160143567 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f138c102fe0 msgr2=0x7f138c1989a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:13:08.283 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:08.281+0000 7f1392a00700 1 --2- 172.21.15.153:0/160143567 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f138c102fe0 0x7f138c1989a0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f137c00b5c0 tx=0x7f137c00be00 comp rx=0 tx=0).stop 2024-03-21T10:13:08.283 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:08.281+0000 7f1392a00700 1 -- 172.21.15.153:0/160143567 shutdown_connections 2024-03-21T10:13:08.283 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:08.281+0000 7f1392a00700 1 --2- 172.21.15.153:0/160143567 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f137006e0e0 0x7f13700705a0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:08.283 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:08.281+0000 7f1392a00700 1 --2- 172.21.15.153:0/160143567 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f138c102fe0 0x7f138c1989a0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:08.283 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:08.281+0000 7f1392a00700 1 --2- 172.21.15.153:0/160143567 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f138c103990 0x7f138c198ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:08.283 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:08.281+0000 7f1392a00700 1 -- 172.21.15.153:0/160143567 >> 172.21.15.153:0/160143567 conn(0x7f138c0fe8d0 msgr2=0x7f138c0fff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:13:08.284 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:08.281+0000 7f1392a00700 1 -- 172.21.15.153:0/160143567 shutdown_connections 2024-03-21T10:13:08.284 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:08.281+0000 7f1392a00700 1 -- 172.21.15.153:0/160143567 wait complete. 2024-03-21T10:13:08.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:08 smithi153 ceph-mon[29117]: pgmap v140: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:13:08.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:08 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:13:08.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:08 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:13:08.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:08 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:13:08.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:08 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:13:08.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:08 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:13:08.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:08 smithi174 ceph-mon[37496]: pgmap v140: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:13:08.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:08 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:13:08.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:08 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:13:08.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:08 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:13:08.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:08 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:13:08.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:08 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:13:09.202 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-03-21T10:13:09.203 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi153.front.sepia.ceph.com 2024-03-21T10:13:09.203 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 070ed3e4-e769-11ee-95cd-87774f69a715 -e sha1=2edfee68ad7243bed422a049b6f5f58619300f11 -- bash -c 'while ceph orch upgrade status | jq '"'"'.in_progress'"'"' | grep true && ! ceph orch upgrade status | jq '"'"'.message'"'"' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done' 2024-03-21T10:13:09.621 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi153/config 2024-03-21T10:13:09.764 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:09 smithi153 ceph-mon[29117]: from='client.14872 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:13:09.764 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:09 smithi153 ceph-mon[29117]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 2024-03-21T10:13:09.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:09 smithi174 ceph-mon[37496]: from='client.14872 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:13:09.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:09 smithi174 ceph-mon[37496]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 2024-03-21T10:13:10.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:10 smithi174 ceph-mon[37496]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 2024-03-21T10:13:10.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:10 smithi174 ceph-mon[37496]: pgmap v141: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:13:10.707 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:10 smithi153 ceph-mon[29117]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 2024-03-21T10:13:10.707 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:10 smithi153 ceph-mon[29117]: pgmap v141: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:13:12.583 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:12.530+0000 7fec16a00700 1 -- 172.21.15.153:0/277290261 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec100731c0 msgr2=0x7fec100735a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:13:12.583 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:12.530+0000 7fec16a00700 1 --2- 172.21.15.153:0/277290261 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec100731c0 0x7fec100735a0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fec00009910 tx=0x7fec00009c20 comp rx=0 tx=0).stop 2024-03-21T10:13:12.584 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:12.531+0000 7fec16a00700 1 -- 172.21.15.153:0/277290261 shutdown_connections 2024-03-21T10:13:12.584 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:12.531+0000 7fec16a00700 1 --2- 172.21.15.153:0/277290261 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fec10073ae0 0x7fec1010ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:12.584 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:12.531+0000 7fec16a00700 1 --2- 172.21.15.153:0/277290261 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec100731c0 0x7fec100735a0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:12.584 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:12.531+0000 7fec16a00700 1 -- 172.21.15.153:0/277290261 >> 172.21.15.153:0/277290261 conn(0x7fec100fc630 msgr2=0x7fec100fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:13:12.584 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:12.531+0000 7fec16a00700 1 -- 172.21.15.153:0/277290261 shutdown_connections 2024-03-21T10:13:12.584 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:12.531+0000 7fec16a00700 1 -- 172.21.15.153:0/277290261 wait complete. 2024-03-21T10:13:12.584 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:12.532+0000 7fec16a00700 1 Processor -- start 2024-03-21T10:13:12.584 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:12.532+0000 7fec16a00700 1 -- start start 2024-03-21T10:13:12.584 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:12.532+0000 7fec16a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec100731c0 0x7fec1019e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:13:12.584 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:12.532+0000 7fec16a00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fec10073ae0 0x7fec1019eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:13:12.585 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:12.532+0000 7fec16a00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec1019f430 con 0x7fec100731c0 2024-03-21T10:13:12.585 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:12.532+0000 7fec16a00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec101988e0 con 0x7fec10073ae0 2024-03-21T10:13:12.585 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:12.533+0000 7fec0fe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec100731c0 0x7fec1019e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:13:12.585 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:12.533+0000 7fec0fe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec100731c0 0x7fec1019e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41882/0 (socket says 172.21.15.153:41882) 2024-03-21T10:13:12.585 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:12.533+0000 7fec0fe00700 1 -- 172.21.15.153:0/3206706800 learned_addr learned my addr 172.21.15.153:0/3206706800 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:13:12.586 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:12.533+0000 7fec0fe00700 1 -- 172.21.15.153:0/3206706800 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fec10073ae0 msgr2=0x7fec1019eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:13:12.586 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:12.533+0000 7fec0fe00700 1 --2- 172.21.15.153:0/3206706800 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fec10073ae0 0x7fec1019eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:12.586 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:12.533+0000 7fec0fe00700 1 -- 172.21.15.153:0/3206706800 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec000095f0 con 0x7fec100731c0 2024-03-21T10:13:12.586 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:12.533+0000 7fec0fe00700 1 --2- 172.21.15.153:0/3206706800 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec100731c0 0x7fec1019e860 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fec000092e0 tx=0x7fec0000be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:13:12.586 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:12.534+0000 7fec0cc00700 1 -- 172.21.15.153:0/3206706800 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fec0001d070 con 0x7fec100731c0 2024-03-21T10:13:12.586 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:12.534+0000 7fec0cc00700 1 -- 172.21.15.153:0/3206706800 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fec0000f460 con 0x7fec100731c0 2024-03-21T10:13:12.586 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:12.534+0000 7fec16a00700 1 -- 172.21.15.153:0/3206706800 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec10198b60 con 0x7fec100731c0 2024-03-21T10:13:12.586 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:12.534+0000 7fec0cc00700 1 -- 172.21.15.153:0/3206706800 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fec00017630 con 0x7fec100731c0 2024-03-21T10:13:12.586 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:12.534+0000 7fec16a00700 1 -- 172.21.15.153:0/3206706800 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec10199050 con 0x7fec100731c0 2024-03-21T10:13:12.586 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:12.535+0000 7fec16a00700 1 -- 172.21.15.153:0/3206706800 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec1010a540 con 0x7fec100731c0 2024-03-21T10:13:12.586 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:12.536+0000 7fec0cc00700 1 -- 172.21.15.153:0/3206706800 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 31) v1 ==== 92072+0+0 (secure 0 0 0) 0x7fec0000f5d0 con 0x7fec100731c0 2024-03-21T10:13:12.587 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:12.537+0000 7fec0cc00700 1 --2- 172.21.15.153:0/3206706800 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7febf406e0e0 0x7febf40705a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:13:12.587 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:12.537+0000 7fec0cc00700 1 -- 172.21.15.153:0/3206706800 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fec0008f360 con 0x7fec100731c0 2024-03-21T10:13:12.587 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:12.537+0000 7fec0f400700 1 --2- 172.21.15.153:0/3206706800 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7febf406e0e0 0x7febf40705a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:13:12.587 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:12.538+0000 7fec0f400700 1 --2- 172.21.15.153:0/3206706800 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7febf406e0e0 0x7febf40705a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fec100fddc0 tx=0x7febfc006d20 comp rx=0 tx=0).ready entity=mgr.14666 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:13:12.587 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:12.542+0000 7fec0cc00700 1 -- 172.21.15.153:0/3206706800 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185454 (secure 0 0 0) 0x7fec0005c5f0 con 0x7fec100731c0 2024-03-21T10:13:12.587 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:12 smithi153 ceph-mon[29117]: pgmap v142: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:13:12.790 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:12.788+0000 7fec16a00700 1 -- 172.21.15.153:0/3206706800 --> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fec1002cce0 con 0x7febf406e0e0 2024-03-21T10:13:12.792 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:12.790+0000 7fec0cc00700 1 -- 172.21.15.153:0/3206706800 <== mgr.14666 v2:172.21.15.153:6800/2524145044 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7fec1002cce0 con 0x7febf406e0e0 2024-03-21T10:13:12.795 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:12.793+0000 7fec16a00700 1 -- 172.21.15.153:0/3206706800 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7febf406e0e0 msgr2=0x7febf40705a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:13:12.795 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:12.793+0000 7fec16a00700 1 --2- 172.21.15.153:0/3206706800 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7febf406e0e0 0x7febf40705a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fec100fddc0 tx=0x7febfc006d20 comp rx=0 tx=0).stop 2024-03-21T10:13:12.795 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:12.793+0000 7fec16a00700 1 -- 172.21.15.153:0/3206706800 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec100731c0 msgr2=0x7fec1019e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:13:12.795 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:12.793+0000 7fec16a00700 1 --2- 172.21.15.153:0/3206706800 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec100731c0 0x7fec1019e860 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fec000092e0 tx=0x7fec0000be00 comp rx=0 tx=0).stop 2024-03-21T10:13:12.795 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:12.793+0000 7fec16a00700 1 -- 172.21.15.153:0/3206706800 shutdown_connections 2024-03-21T10:13:12.795 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:12.793+0000 7fec16a00700 1 --2- 172.21.15.153:0/3206706800 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7febf406e0e0 0x7febf40705a0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:12.795 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:12.793+0000 7fec16a00700 1 --2- 172.21.15.153:0/3206706800 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec100731c0 0x7fec1019e860 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:12.795 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:12.793+0000 7fec16a00700 1 --2- 172.21.15.153:0/3206706800 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fec10073ae0 0x7fec1019eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:12.796 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:12.793+0000 7fec16a00700 1 -- 172.21.15.153:0/3206706800 >> 172.21.15.153:0/3206706800 conn(0x7fec100fc630 msgr2=0x7fec10107680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:13:12.796 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:12.793+0000 7fec16a00700 1 -- 172.21.15.153:0/3206706800 shutdown_connections 2024-03-21T10:13:12.796 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:12.793+0000 7fec16a00700 1 -- 172.21.15.153:0/3206706800 wait complete. 2024-03-21T10:13:12.811 INFO:teuthology.orchestra.run.smithi153.stdout:true 2024-03-21T10:13:12.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:12 smithi174 ceph-mon[37496]: pgmap v142: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:13:12.953 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:12.951+0000 7efcde600700 1 -- 172.21.15.153:0/208072197 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efcd8069020 msgr2=0x7efcd8105790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:13:12.953 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:12.951+0000 7efcde600700 1 --2- 172.21.15.153:0/208072197 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efcd8069020 0x7efcd8105790 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7efcc4009910 tx=0x7efcc4009c20 comp rx=0 tx=0).stop 2024-03-21T10:13:12.953 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:12.951+0000 7efcde600700 1 -- 172.21.15.153:0/208072197 shutdown_connections 2024-03-21T10:13:12.953 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:12.951+0000 7efcde600700 1 --2- 172.21.15.153:0/208072197 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efcd8069020 0x7efcd8105790 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:12.953 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:12.951+0000 7efcde600700 1 --2- 172.21.15.153:0/208072197 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efcd8068670 0x7efcd8068a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:12.954 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:12.951+0000 7efcde600700 1 -- 172.21.15.153:0/208072197 >> 172.21.15.153:0/208072197 conn(0x7efcd80755a0 msgr2=0x7efcd80759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:13:12.954 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:12.951+0000 7efcde600700 1 -- 172.21.15.153:0/208072197 shutdown_connections 2024-03-21T10:13:12.954 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:12.951+0000 7efcde600700 1 -- 172.21.15.153:0/208072197 wait complete. 2024-03-21T10:13:12.954 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:12.953+0000 7efcde600700 1 Processor -- start 2024-03-21T10:13:12.955 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:12.953+0000 7efcde600700 1 -- start start 2024-03-21T10:13:12.955 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:12.953+0000 7efcde600700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efcd8068670 0x7efcd819e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:13:12.955 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:12.953+0000 7efcde600700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efcd8069020 0x7efcd819ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:13:12.955 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:12.953+0000 7efcde600700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcd819f3b0 con 0x7efcd8068670 2024-03-21T10:13:12.955 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:12.953+0000 7efcde600700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcd8198860 con 0x7efcd8069020 2024-03-21T10:13:12.956 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:12.953+0000 7efcd7400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efcd8068670 0x7efcd819e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:13:12.956 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:12.953+0000 7efcd7400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efcd8068670 0x7efcd819e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:44748/0 (socket says 172.21.15.153:44748) 2024-03-21T10:13:12.956 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:12.953+0000 7efcd7400700 1 -- 172.21.15.153:0/1499819984 learned_addr learned my addr 172.21.15.153:0/1499819984 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:13:12.956 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:12.954+0000 7efcd6a00700 1 --2- 172.21.15.153:0/1499819984 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efcd8069020 0x7efcd819ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:13:12.957 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:12.954+0000 7efcd7400700 1 -- 172.21.15.153:0/1499819984 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efcd8069020 msgr2=0x7efcd819ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:13:12.957 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:12.954+0000 7efcd7400700 1 --2- 172.21.15.153:0/1499819984 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efcd8069020 0x7efcd819ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:12.957 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:12.954+0000 7efcd7400700 1 -- 172.21.15.153:0/1499819984 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcc40095f0 con 0x7efcd8068670 2024-03-21T10:13:12.957 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:12.954+0000 7efcd7400700 1 --2- 172.21.15.153:0/1499819984 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efcd8068670 0x7efcd819e7e0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7efcc800d8d0 tx=0x7efcc800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:13:12.957 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:12.955+0000 7efccfe00700 1 -- 172.21.15.153:0/1499819984 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efcc8009940 con 0x7efcd8068670 2024-03-21T10:13:12.957 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:12.955+0000 7efccfe00700 1 -- 172.21.15.153:0/1499819984 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efcc8010460 con 0x7efcd8068670 2024-03-21T10:13:12.957 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:12.955+0000 7efccfe00700 1 -- 172.21.15.153:0/1499819984 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efcc800f5f0 con 0x7efcd8068670 2024-03-21T10:13:12.957 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:12.955+0000 7efcde600700 1 -- 172.21.15.153:0/1499819984 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcd8198b40 con 0x7efcd8068670 2024-03-21T10:13:12.957 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:12.955+0000 7efcde600700 1 -- 172.21.15.153:0/1499819984 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcd8199090 con 0x7efcd8068670 2024-03-21T10:13:12.958 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:12.956+0000 7efcde600700 1 -- 172.21.15.153:0/1499819984 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcd804ea90 con 0x7efcd8068670 2024-03-21T10:13:12.958 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:12.957+0000 7efccfe00700 1 -- 172.21.15.153:0/1499819984 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 31) v1 ==== 92072+0+0 (secure 0 0 0) 0x7efcc80105d0 con 0x7efcd8068670 2024-03-21T10:13:12.959 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:12.958+0000 7efccfe00700 1 --2- 172.21.15.153:0/1499819984 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7efcbc06e0e0 0x7efcbc0705a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:13:12.960 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:12.958+0000 7efccfe00700 1 -- 172.21.15.153:0/1499819984 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7efcc808de60 con 0x7efcd8068670 2024-03-21T10:13:12.960 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:12.958+0000 7efcd6a00700 1 --2- 172.21.15.153:0/1499819984 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7efcbc06e0e0 0x7efcbc0705a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:13:12.960 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:12.959+0000 7efcd6a00700 1 --2- 172.21.15.153:0/1499819984 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7efcbc06e0e0 0x7efcbc0705a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7efcc4006010 tx=0x7efcc4005960 comp rx=0 tx=0).ready entity=mgr.14666 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:13:12.964 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:12.963+0000 7efccfe00700 1 -- 172.21.15.153:0/1499819984 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185454 (secure 0 0 0) 0x7efcc805b070 con 0x7efcd8068670 2024-03-21T10:13:13.223 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:13.221+0000 7efcde600700 1 -- 172.21.15.153:0/1499819984 --> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7efcd8199430 con 0x7efcbc06e0e0 2024-03-21T10:13:13.226 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:13.224+0000 7efccfe00700 1 -- 172.21.15.153:0/1499819984 <== mgr.14666 v2:172.21.15.153:6800/2524145044 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7efcd8199430 con 0x7efcbc06e0e0 2024-03-21T10:13:13.230 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:13.228+0000 7efcde600700 1 -- 172.21.15.153:0/1499819984 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7efcbc06e0e0 msgr2=0x7efcbc0705a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:13:13.230 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:13.228+0000 7efcde600700 1 --2- 172.21.15.153:0/1499819984 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7efcbc06e0e0 0x7efcbc0705a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7efcc4006010 tx=0x7efcc4005960 comp rx=0 tx=0).stop 2024-03-21T10:13:13.230 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:13.228+0000 7efcde600700 1 -- 172.21.15.153:0/1499819984 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efcd8068670 msgr2=0x7efcd819e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:13:13.230 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:13.228+0000 7efcde600700 1 --2- 172.21.15.153:0/1499819984 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efcd8068670 0x7efcd819e7e0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7efcc800d8d0 tx=0x7efcc800dc90 comp rx=0 tx=0).stop 2024-03-21T10:13:13.230 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:13.228+0000 7efcde600700 1 -- 172.21.15.153:0/1499819984 shutdown_connections 2024-03-21T10:13:13.230 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:13.228+0000 7efcde600700 1 --2- 172.21.15.153:0/1499819984 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7efcbc06e0e0 0x7efcbc0705a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:13.230 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:13.228+0000 7efcde600700 1 --2- 172.21.15.153:0/1499819984 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efcd8068670 0x7efcd819e7e0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:13.231 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:13.228+0000 7efcde600700 1 --2- 172.21.15.153:0/1499819984 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efcd8069020 0x7efcd819ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:13.231 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:13.228+0000 7efcde600700 1 -- 172.21.15.153:0/1499819984 >> 172.21.15.153:0/1499819984 conn(0x7efcd80755a0 msgr2=0x7efcd8102760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:13:13.231 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:13.228+0000 7efcde600700 1 -- 172.21.15.153:0/1499819984 shutdown_connections 2024-03-21T10:13:13.231 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:13.228+0000 7efcde600700 1 -- 172.21.15.153:0/1499819984 wait complete. 2024-03-21T10:13:13.395 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:13.393+0000 7f93e9c00700 1 -- 172.21.15.153:0/2291179609 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f93e4105a40 msgr2=0x7f93e4105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:13:13.395 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:13.393+0000 7f93e9c00700 1 --2- 172.21.15.153:0/2291179609 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f93e4105a40 0x7f93e4105e20 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f93d4009910 tx=0x7f93d4009c20 comp rx=0 tx=0).stop 2024-03-21T10:13:13.396 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:13.393+0000 7f93e9c00700 1 -- 172.21.15.153:0/2291179609 shutdown_connections 2024-03-21T10:13:13.396 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:13.393+0000 7f93e9c00700 1 --2- 172.21.15.153:0/2291179609 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f93e40ffa40 0x7f93e40ffec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:13.396 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:13.393+0000 7f93e9c00700 1 --2- 172.21.15.153:0/2291179609 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f93e4105a40 0x7f93e4105e20 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:13.396 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:13.393+0000 7f93e9c00700 1 -- 172.21.15.153:0/2291179609 >> 172.21.15.153:0/2291179609 conn(0x7f93e40fb570 msgr2=0x7f93e40fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:13:13.396 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:13.394+0000 7f93e9c00700 1 -- 172.21.15.153:0/2291179609 shutdown_connections 2024-03-21T10:13:13.396 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:13.394+0000 7f93e9c00700 1 -- 172.21.15.153:0/2291179609 wait complete. 2024-03-21T10:13:13.397 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:13.395+0000 7f93e9c00700 1 Processor -- start 2024-03-21T10:13:13.397 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:13.395+0000 7f93e9c00700 1 -- start start 2024-03-21T10:13:13.397 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:13.395+0000 7f93e9c00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f93e40ffa40 0x7f93e4198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:13:13.397 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:13.396+0000 7f93e9c00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f93e4105a40 0x7f93e4198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:13:13.397 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:13.396+0000 7f93e9c00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93e4199530 con 0x7f93e4105a40 2024-03-21T10:13:13.397 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:13.396+0000 7f93e9c00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93e419d340 con 0x7f93e40ffa40 2024-03-21T10:13:13.399 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:13.396+0000 7f93e2000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f93e4105a40 0x7f93e4198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:13:13.399 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:13.396+0000 7f93e2000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f93e4105a40 0x7f93e4198ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:44772/0 (socket says 172.21.15.153:44772) 2024-03-21T10:13:13.399 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:13.396+0000 7f93e2000700 1 -- 172.21.15.153:0/330550578 learned_addr learned my addr 172.21.15.153:0/330550578 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:13:13.399 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:13.396+0000 7f93e2a00700 1 --2- 172.21.15.153:0/330550578 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f93e40ffa40 0x7f93e4198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:13:13.399 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:13.396+0000 7f93e2000700 1 -- 172.21.15.153:0/330550578 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f93e40ffa40 msgr2=0x7f93e4198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:13:13.399 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:13.396+0000 7f93e2000700 1 --2- 172.21.15.153:0/330550578 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f93e40ffa40 0x7f93e4198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:13.399 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:13.396+0000 7f93e2000700 1 -- 172.21.15.153:0/330550578 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93d40095f0 con 0x7f93e4105a40 2024-03-21T10:13:13.399 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:13.396+0000 7f93e2a00700 1 --2- 172.21.15.153:0/330550578 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f93e40ffa40 0x7f93e4198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:13:13.400 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:13.397+0000 7f93e2000700 1 --2- 172.21.15.153:0/330550578 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f93e4105a40 0x7f93e4198ea0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f93d000d8d0 tx=0x7f93d000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:13:13.400 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:13.397+0000 7f93db400700 1 -- 172.21.15.153:0/330550578 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f93d0009940 con 0x7f93e4105a40 2024-03-21T10:13:13.400 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:13.397+0000 7f93db400700 1 -- 172.21.15.153:0/330550578 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f93d0010460 con 0x7f93e4105a40 2024-03-21T10:13:13.400 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:13.397+0000 7f93db400700 1 -- 172.21.15.153:0/330550578 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f93d000f5f0 con 0x7f93e4105a40 2024-03-21T10:13:13.400 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:13.397+0000 7f93e9c00700 1 -- 172.21.15.153:0/330550578 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93e419d620 con 0x7f93e4105a40 2024-03-21T10:13:13.400 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:13.397+0000 7f93e9c00700 1 -- 172.21.15.153:0/330550578 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93e419db70 con 0x7f93e4105a40 2024-03-21T10:13:13.400 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:13.398+0000 7f93e9c00700 1 -- 172.21.15.153:0/330550578 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93e404ea90 con 0x7f93e4105a40 2024-03-21T10:13:13.401 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:13.400+0000 7f93db400700 1 -- 172.21.15.153:0/330550578 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 31) v1 ==== 92072+0+0 (secure 0 0 0) 0x7f93d00105d0 con 0x7f93e4105a40 2024-03-21T10:13:13.402 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:13.401+0000 7f93db400700 1 --2- 172.21.15.153:0/330550578 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f93c806e0e0 0x7f93c80705a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:13:13.402 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:13.401+0000 7f93db400700 1 -- 172.21.15.153:0/330550578 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f93d008ddf0 con 0x7f93e4105a40 2024-03-21T10:13:13.403 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:13.401+0000 7f93e2a00700 1 --2- 172.21.15.153:0/330550578 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f93c806e0e0 0x7f93c80705a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:13:13.403 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:13.402+0000 7f93e2a00700 1 --2- 172.21.15.153:0/330550578 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f93c806e0e0 0x7f93c80705a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f93d400b5c0 tx=0x7f93d40053a0 comp rx=0 tx=0).ready entity=mgr.14666 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:13:13.407 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:13.406+0000 7f93db400700 1 -- 172.21.15.153:0/330550578 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185454 (secure 0 0 0) 0x7f93d005afe0 con 0x7f93e4105a40 2024-03-21T10:13:13.651 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:13.649+0000 7f93e9c00700 1 -- 172.21.15.153:0/330550578 --> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f93e419df90 con 0x7f93c806e0e0 2024-03-21T10:13:13.662 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:13.661+0000 7f93db400700 1 -- 172.21.15.153:0/330550578 <== mgr.14666 v2:172.21.15.153:6800/2524145044 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f93e419df90 con 0x7f93c806e0e0 2024-03-21T10:13:13.663 INFO:teuthology.orchestra.run.smithi153.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-21T10:13:13.663 INFO:teuthology.orchestra.run.smithi153.stdout:alertmanager.smithi153 smithi153 *:9093,9094 running (2m) 83s ago 15m 13.7M - 0.25.0 c8568f914cd2 9ed3f2200b5f 2024-03-21T10:13:13.663 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi153 smithi153 running (16m) 83s ago 16m 8196k - 18.2.1 5be31c24972a 5b61994396a6 2024-03-21T10:13:13.663 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi174 smithi174 running (14m) 3m ago 14m 7940k - 18.2.1 5be31c24972a 4f7a5fc3ac28 2024-03-21T10:13:13.663 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi153 smithi153 running (16m) 83s ago 16m 6665k - 18.2.1 5be31c24972a 90a8b8ea03d1 2024-03-21T10:13:13.663 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi174 smithi174 running (14m) 3m ago 14m 6665k - 18.2.1 5be31c24972a a4c929e4dc77 2024-03-21T10:13:13.664 INFO:teuthology.orchestra.run.smithi153.stdout:grafana.smithi153 smithi153 *:3000 running (91s) 83s ago 15m 50.6M - 9.4.12 2bacad6d85d8 ee24026ab786 2024-03-21T10:13:13.664 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.iuyluj smithi153 running (8m) 83s ago 8m 129M - 18.2.1 5be31c24972a 02f667cee979 2024-03-21T10:13:13.664 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.zipckr smithi153 running (8m) 83s ago 8m 58.6M - 18.2.1 5be31c24972a 68a853929f8f 2024-03-21T10:13:13.664 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.nhozag smithi174 running (8m) 3m ago 8m 128M - 18.2.1 5be31c24972a 2fee4000308f 2024-03-21T10:13:13.664 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.yyimst smithi174 running (8m) 3m ago 8m 155M - 18.2.1 5be31c24972a d78a729220d1 2024-03-21T10:13:13.664 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi153.tluxni smithi153 *:8443,9283,8765 running (5m) 83s ago 17m 511M - 19.0.0-2231-g2edfee68 237e43c7a65f 7ffee0e55382 2024-03-21T10:13:13.664 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi174.qpcwat smithi174 *:8443,9283,8765 running (3m) 3m ago 14m 447M - 19.0.0-2231-g2edfee68 237e43c7a65f 748968f216ab 2024-03-21T10:13:13.664 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi153 smithi153 running (17m) 83s ago 17m 57.2M 2048M 18.2.1 5be31c24972a 813963a95d69 2024-03-21T10:13:13.664 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi174 smithi174 running (14m) 3m ago 14m 45.3M 2048M 18.2.1 5be31c24972a 99204923eec6 2024-03-21T10:13:13.664 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi153 smithi153 *:9100 running (3m) 83s ago 15m 7394k - 1.5.0 0da6a335fe13 bb835edc0e0b 2024-03-21T10:13:13.664 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi174 smithi174 *:9100 running (3m) 3m ago 14m 6765k - 1.5.0 0da6a335fe13 7c4ecad790fa 2024-03-21T10:13:13.664 INFO:teuthology.orchestra.run.smithi153.stdout:osd.0 smithi153 running (13m) 83s ago 13m 215M 1217M 18.2.1 5be31c24972a 28912a2319b4 2024-03-21T10:13:13.664 INFO:teuthology.orchestra.run.smithi153.stdout:osd.1 smithi153 running (12m) 83s ago 12m 214M 1217M 18.2.1 5be31c24972a ca60681e2314 2024-03-21T10:13:13.664 INFO:teuthology.orchestra.run.smithi153.stdout:osd.2 smithi153 running (12m) 83s ago 12m 172M 1217M 18.2.1 5be31c24972a 8db799f8962a 2024-03-21T10:13:13.664 INFO:teuthology.orchestra.run.smithi153.stdout:osd.3 smithi174 running (11m) 3m ago 11m 213M 2241M 18.2.1 5be31c24972a dafc89e0658d 2024-03-21T10:13:13.665 INFO:teuthology.orchestra.run.smithi153.stdout:osd.4 smithi174 running (10m) 3m ago 10m 201M 2241M 18.2.1 5be31c24972a c4cd15701f05 2024-03-21T10:13:13.665 INFO:teuthology.orchestra.run.smithi153.stdout:osd.5 smithi174 running (10m) 3m ago 10m 187M 2241M 18.2.1 5be31c24972a 3b69886ce22e 2024-03-21T10:13:13.665 INFO:teuthology.orchestra.run.smithi153.stdout:prometheus.smithi153 smithi153 *:9095 running (2m) 83s ago 14m 59.7M - 2.43.0 a07b618ecd1d 7d95561e50c2 2024-03-21T10:13:13.666 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:13.664+0000 7f93e9c00700 1 -- 172.21.15.153:0/330550578 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f93c806e0e0 msgr2=0x7f93c80705a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:13:13.666 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:13.664+0000 7f93e9c00700 1 --2- 172.21.15.153:0/330550578 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f93c806e0e0 0x7f93c80705a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f93d400b5c0 tx=0x7f93d40053a0 comp rx=0 tx=0).stop 2024-03-21T10:13:13.666 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:13.664+0000 7f93e9c00700 1 -- 172.21.15.153:0/330550578 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f93e4105a40 msgr2=0x7f93e4198ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:13:13.667 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:13.664+0000 7f93e9c00700 1 --2- 172.21.15.153:0/330550578 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f93e4105a40 0x7f93e4198ea0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f93d000d8d0 tx=0x7f93d000dc90 comp rx=0 tx=0).stop 2024-03-21T10:13:13.667 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:13.664+0000 7f93e9c00700 1 -- 172.21.15.153:0/330550578 shutdown_connections 2024-03-21T10:13:13.667 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:13.664+0000 7f93e9c00700 1 --2- 172.21.15.153:0/330550578 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f93c806e0e0 0x7f93c80705a0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:13.667 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:13.664+0000 7f93e9c00700 1 --2- 172.21.15.153:0/330550578 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f93e40ffa40 0x7f93e4198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:13.667 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:13.664+0000 7f93e9c00700 1 --2- 172.21.15.153:0/330550578 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f93e4105a40 0x7f93e4198ea0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:13.667 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:13.664+0000 7f93e9c00700 1 -- 172.21.15.153:0/330550578 >> 172.21.15.153:0/330550578 conn(0x7f93e40fb570 msgr2=0x7f93e4068de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:13:13.667 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:13.665+0000 7f93e9c00700 1 -- 172.21.15.153:0/330550578 shutdown_connections 2024-03-21T10:13:13.667 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:13.665+0000 7f93e9c00700 1 -- 172.21.15.153:0/330550578 wait complete. 2024-03-21T10:13:13.833 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:13.827+0000 7fe0e6800700 1 -- 172.21.15.153:0/3083085565 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe0e0102e10 msgr2=0x7fe0e01031f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:13:13.833 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:13.827+0000 7fe0e6800700 1 --2- 172.21.15.153:0/3083085565 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe0e0102e10 0x7fe0e01031f0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fe0d4009910 tx=0x7fe0d4009c20 comp rx=0 tx=0).stop 2024-03-21T10:13:13.833 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:13.827+0000 7fe0e6800700 1 -- 172.21.15.153:0/3083085565 shutdown_connections 2024-03-21T10:13:13.833 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:13.827+0000 7fe0e6800700 1 --2- 172.21.15.153:0/3083085565 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe0e01037c0 0x7fe0e0107810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:13.833 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:13.827+0000 7fe0e6800700 1 --2- 172.21.15.153:0/3083085565 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe0e0102e10 0x7fe0e01031f0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:13.834 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:13.827+0000 7fe0e6800700 1 -- 172.21.15.153:0/3083085565 >> 172.21.15.153:0/3083085565 conn(0x7fe0e00fe6c0 msgr2=0x7fe0e0100ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:13:13.834 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:13.827+0000 7fe0e6800700 1 -- 172.21.15.153:0/3083085565 shutdown_connections 2024-03-21T10:13:13.834 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:13.827+0000 7fe0e6800700 1 -- 172.21.15.153:0/3083085565 wait complete. 2024-03-21T10:13:13.834 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:13.828+0000 7fe0e6800700 1 Processor -- start 2024-03-21T10:13:13.834 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:13.828+0000 7fe0e6800700 1 -- start start 2024-03-21T10:13:13.835 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:13.828+0000 7fe0e6800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe0e0102e10 0x7fe0e0198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:13:13.835 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:13.828+0000 7fe0e6800700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe0e01037c0 0x7fe0e0198ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:13:13.835 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:13.829+0000 7fe0e6800700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0e0199550 con 0x7fe0e0102e10 2024-03-21T10:13:13.835 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:13.829+0000 7fe0e6800700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0e019d360 con 0x7fe0e01037c0 2024-03-21T10:13:13.835 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:13.829+0000 7fe0e5400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe0e0102e10 0x7fe0e0198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:13:13.835 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:13.829+0000 7fe0e5400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe0e0102e10 0x7fe0e0198980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:44798/0 (socket says 172.21.15.153:44798) 2024-03-21T10:13:13.836 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:13.829+0000 7fe0e5400700 1 -- 172.21.15.153:0/3022462556 learned_addr learned my addr 172.21.15.153:0/3022462556 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:13:13.836 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:13.829+0000 7fe0e4a00700 1 --2- 172.21.15.153:0/3022462556 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe0e01037c0 0x7fe0e0198ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:13:13.836 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:13.829+0000 7fe0e5400700 1 -- 172.21.15.153:0/3022462556 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe0e01037c0 msgr2=0x7fe0e0198ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:13:13.836 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:13.829+0000 7fe0e5400700 1 --2- 172.21.15.153:0/3022462556 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe0e01037c0 0x7fe0e0198ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:13.836 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:13.829+0000 7fe0e5400700 1 -- 172.21.15.153:0/3022462556 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0d40095f0 con 0x7fe0e0102e10 2024-03-21T10:13:13.837 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:13.829+0000 7fe0e4a00700 1 --2- 172.21.15.153:0/3022462556 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe0e01037c0 0x7fe0e0198ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:13:13.837 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:13.830+0000 7fe0e5400700 1 --2- 172.21.15.153:0/3022462556 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe0e0102e10 0x7fe0e0198980 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fe0d400b5c0 tx=0x7fe0d400be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:13:13.837 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:13.830+0000 7fe0de000700 1 -- 172.21.15.153:0/3022462556 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe0d401d070 con 0x7fe0e0102e10 2024-03-21T10:13:13.837 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:13.830+0000 7fe0de000700 1 -- 172.21.15.153:0/3022462556 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe0d400f460 con 0x7fe0e0102e10 2024-03-21T10:13:13.838 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:13.830+0000 7fe0e6800700 1 -- 172.21.15.153:0/3022462556 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0e019d5e0 con 0x7fe0e0102e10 2024-03-21T10:13:13.838 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:13.830+0000 7fe0de000700 1 -- 172.21.15.153:0/3022462556 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe0d4017610 con 0x7fe0e0102e10 2024-03-21T10:13:13.838 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:13.830+0000 7fe0e6800700 1 -- 172.21.15.153:0/3022462556 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0e019dad0 con 0x7fe0e0102e10 2024-03-21T10:13:13.840 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:13.831+0000 7fe0e6800700 1 -- 172.21.15.153:0/3022462556 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0e010b1b0 con 0x7fe0e0102e10 2024-03-21T10:13:13.840 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:13.832+0000 7fe0de000700 1 -- 172.21.15.153:0/3022462556 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 31) v1 ==== 92072+0+0 (secure 0 0 0) 0x7fe0d400f5d0 con 0x7fe0e0102e10 2024-03-21T10:13:13.840 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:13.833+0000 7fe0de000700 1 --2- 172.21.15.153:0/3022462556 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7fe0c806e0e0 0x7fe0c80705a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:13:13.841 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:13.833+0000 7fe0de000700 1 -- 172.21.15.153:0/3022462556 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fe0d408f460 con 0x7fe0e0102e10 2024-03-21T10:13:13.841 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:13.834+0000 7fe0e4a00700 1 --2- 172.21.15.153:0/3022462556 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7fe0c806e0e0 0x7fe0c80705a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:13:13.841 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:13.834+0000 7fe0e4a00700 1 --2- 172.21.15.153:0/3022462556 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7fe0c806e0e0 0x7fe0c80705a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe0e0199fa0 tx=0x7fe0cc008040 comp rx=0 tx=0).ready entity=mgr.14666 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:13:13.841 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:13.838+0000 7fe0de000700 1 -- 172.21.15.153:0/3022462556 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185454 (secure 0 0 0) 0x7fe0d405c650 con 0x7fe0e0102e10 2024-03-21T10:13:14.166 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:14.164+0000 7fe0e6800700 1 -- 172.21.15.153:0/3022462556 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fe0e0199de0 con 0x7fe0e0102e10 2024-03-21T10:13:14.167 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:13 smithi153 ceph-mon[29117]: from='client.14876 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:13:14.167 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:13 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:13:14.167 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:13 smithi153 ceph-mon[29117]: Upgrade: Target is version 19.0.0-2231-g2edfee68 (squid) 2024-03-21T10:13:14.167 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:13 smithi153 ceph-mon[29117]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, digests ['quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232'] 2024-03-21T10:13:14.167 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:13 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:13:14.167 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:13 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:13:14.167 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:13 smithi153 ceph-mon[29117]: Upgrade: Setting container_image for all mgr 2024-03-21T10:13:14.167 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:13 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:13:14.167 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:13 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-03-21T10:13:14.167 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:13 smithi153 ceph-mon[29117]: from='client.14880 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:13:14.167 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:13 smithi153 ceph-mon[29117]: pgmap v143: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:13:14.168 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:14.165+0000 7fe0de000700 1 -- 172.21.15.153:0/3022462556 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7fe0d405c1e0 con 0x7fe0e0102e10 2024-03-21T10:13:14.168 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:13:14.168 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": { 2024-03-21T10:13:14.168 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-03-21T10:13:14.168 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:13:14.168 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": { 2024-03-21T10:13:14.168 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:13:14.168 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:13:14.168 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": { 2024-03-21T10:13:14.168 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-03-21T10:13:14.169 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:13:14.169 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": { 2024-03-21T10:13:14.169 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-03-21T10:13:14.169 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:13:14.169 INFO:teuthology.orchestra.run.smithi153.stdout: "overall": { 2024-03-21T10:13:14.169 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 12, 2024-03-21T10:13:14.169 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:13:14.169 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-03-21T10:13:14.169 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:13:14.169 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:14.168+0000 7fe0e6800700 1 -- 172.21.15.153:0/3022462556 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7fe0c806e0e0 msgr2=0x7fe0c80705a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:13:14.169 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:14.168+0000 7fe0e6800700 1 --2- 172.21.15.153:0/3022462556 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7fe0c806e0e0 0x7fe0c80705a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe0e0199fa0 tx=0x7fe0cc008040 comp rx=0 tx=0).stop 2024-03-21T10:13:14.170 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:14.168+0000 7fe0e6800700 1 -- 172.21.15.153:0/3022462556 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe0e0102e10 msgr2=0x7fe0e0198980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:13:14.170 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:14.168+0000 7fe0e6800700 1 --2- 172.21.15.153:0/3022462556 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe0e0102e10 0x7fe0e0198980 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fe0d400b5c0 tx=0x7fe0d400be00 comp rx=0 tx=0).stop 2024-03-21T10:13:14.170 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:14.168+0000 7fe0e6800700 1 -- 172.21.15.153:0/3022462556 shutdown_connections 2024-03-21T10:13:14.170 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:14.168+0000 7fe0e6800700 1 --2- 172.21.15.153:0/3022462556 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7fe0c806e0e0 0x7fe0c80705a0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:14.170 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:14.168+0000 7fe0e6800700 1 --2- 172.21.15.153:0/3022462556 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe0e0102e10 0x7fe0e0198980 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:14.170 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:14.168+0000 7fe0e6800700 1 --2- 172.21.15.153:0/3022462556 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe0e01037c0 0x7fe0e0198ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:14.171 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:14.168+0000 7fe0e6800700 1 -- 172.21.15.153:0/3022462556 >> 172.21.15.153:0/3022462556 conn(0x7fe0e00fe6c0 msgr2=0x7fe0e00ffda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:13:14.171 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:14.168+0000 7fe0e6800700 1 -- 172.21.15.153:0/3022462556 shutdown_connections 2024-03-21T10:13:14.171 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:14.168+0000 7fe0e6800700 1 -- 172.21.15.153:0/3022462556 wait complete. 2024-03-21T10:13:14.324 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:14.322+0000 7f200ba00700 1 -- 172.21.15.153:0/1208742738 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f20040ffbd0 msgr2=0x7f2004100050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:13:14.324 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:14.322+0000 7f200ba00700 1 --2- 172.21.15.153:0/1208742738 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f20040ffbd0 0x7f2004100050 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f1ff0009910 tx=0x7f1ff0009c20 comp rx=0 tx=0).stop 2024-03-21T10:13:14.325 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:14.322+0000 7f200ba00700 1 -- 172.21.15.153:0/1208742738 shutdown_connections 2024-03-21T10:13:14.325 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:14.322+0000 7f200ba00700 1 --2- 172.21.15.153:0/1208742738 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f20040ffbd0 0x7f2004100050 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:14.325 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:14.322+0000 7f200ba00700 1 --2- 172.21.15.153:0/1208742738 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2004105a40 0x7f2004105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:14.325 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:14.322+0000 7f200ba00700 1 -- 172.21.15.153:0/1208742738 >> 172.21.15.153:0/1208742738 conn(0x7f20040755a0 msgr2=0x7f20040759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:13:14.325 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:14.323+0000 7f200ba00700 1 -- 172.21.15.153:0/1208742738 shutdown_connections 2024-03-21T10:13:14.325 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:14.323+0000 7f200ba00700 1 -- 172.21.15.153:0/1208742738 wait complete. 2024-03-21T10:13:14.325 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:14.324+0000 7f200ba00700 1 Processor -- start 2024-03-21T10:13:14.326 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:14.324+0000 7f200ba00700 1 -- start start 2024-03-21T10:13:14.327 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:14.325+0000 7f200ba00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f20040ffbd0 0x7f200419c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:13:14.327 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:14.325+0000 7f200ba00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2004105a40 0x7f200419ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:13:14.327 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:14.325+0000 7f200ba00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f200419d040 con 0x7f2004105a40 2024-03-21T10:13:14.327 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:14.325+0000 7f200ba00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2004196580 con 0x7f20040ffbd0 2024-03-21T10:13:14.327 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:14.325+0000 7f2003e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2004105a40 0x7f200419ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:13:14.327 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:14.325+0000 7f2003e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2004105a40 0x7f200419ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:44812/0 (socket says 172.21.15.153:44812) 2024-03-21T10:13:14.327 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:14.325+0000 7f2003e00700 1 -- 172.21.15.153:0/471093540 learned_addr learned my addr 172.21.15.153:0/471093540 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:13:14.327 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:14.326+0000 7f2009000700 1 --2- 172.21.15.153:0/471093540 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f20040ffbd0 0x7f200419c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:13:14.328 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:14.326+0000 7f2003e00700 1 -- 172.21.15.153:0/471093540 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f20040ffbd0 msgr2=0x7f200419c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:13:14.328 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:14.326+0000 7f2003e00700 1 --2- 172.21.15.153:0/471093540 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f20040ffbd0 0x7f200419c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:14.328 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:14.326+0000 7f2003e00700 1 -- 172.21.15.153:0/471093540 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ff00095f0 con 0x7f2004105a40 2024-03-21T10:13:14.329 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:14.326+0000 7f2003e00700 1 --2- 172.21.15.153:0/471093540 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2004105a40 0x7f200419ca40 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f1ff000b5c0 tx=0x7f1ff000be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:13:14.329 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:14.326+0000 7f2001600700 1 -- 172.21.15.153:0/471093540 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1ff001d070 con 0x7f2004105a40 2024-03-21T10:13:14.329 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:14.326+0000 7f2001600700 1 -- 172.21.15.153:0/471093540 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1ff000f460 con 0x7f2004105a40 2024-03-21T10:13:14.329 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:14.326+0000 7f2001600700 1 -- 172.21.15.153:0/471093540 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1ff0017610 con 0x7f2004105a40 2024-03-21T10:13:14.329 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:14.326+0000 7f200ba00700 1 -- 172.21.15.153:0/471093540 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2004196800 con 0x7f2004105a40 2024-03-21T10:13:14.329 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:14.327+0000 7f200ba00700 1 -- 172.21.15.153:0/471093540 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2004196cf0 con 0x7f2004105a40 2024-03-21T10:13:14.329 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:14.328+0000 7f200ba00700 1 -- 172.21.15.153:0/471093540 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2004108fe0 con 0x7f2004105a40 2024-03-21T10:13:14.330 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:14.329+0000 7f2001600700 1 -- 172.21.15.153:0/471093540 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 31) v1 ==== 92072+0+0 (secure 0 0 0) 0x7f1ff000f5d0 con 0x7f2004105a40 2024-03-21T10:13:14.331 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:14.330+0000 7f2001600700 1 --2- 172.21.15.153:0/471093540 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f1fec06e0e0 0x7f1fec0705a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:13:14.331 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:14.330+0000 7f2001600700 1 -- 172.21.15.153:0/471093540 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f1ff008f3c0 con 0x7f2004105a40 2024-03-21T10:13:14.331 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:14.330+0000 7f2009000700 1 --2- 172.21.15.153:0/471093540 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f1fec06e0e0 0x7f1fec0705a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:13:14.332 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:14.331+0000 7f2009000700 1 --2- 172.21.15.153:0/471093540 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f1fec06e0e0 0x7f1fec0705a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1ff8005ea0 tx=0x7f1ff8005e30 comp rx=0 tx=0).ready entity=mgr.14666 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:13:14.336 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:14.335+0000 7f2001600700 1 -- 172.21.15.153:0/471093540 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185454 (secure 0 0 0) 0x7f1ff005c650 con 0x7f2004105a40 2024-03-21T10:13:14.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:13 smithi174 ceph-mon[37496]: from='client.14876 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:13:14.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:13 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:13:14.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:13 smithi174 ceph-mon[37496]: Upgrade: Target is version 19.0.0-2231-g2edfee68 (squid) 2024-03-21T10:13:14.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:13 smithi174 ceph-mon[37496]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, digests ['quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232'] 2024-03-21T10:13:14.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:13 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:13:14.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:13 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:13:14.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:13 smithi174 ceph-mon[37496]: Upgrade: Setting container_image for all mgr 2024-03-21T10:13:14.453 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:13 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:13:14.453 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:13 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-03-21T10:13:14.453 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:13 smithi174 ceph-mon[37496]: from='client.14880 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:13:14.453 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:13 smithi174 ceph-mon[37496]: pgmap v143: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:13:14.608 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:14.607+0000 7f200ba00700 1 -- 172.21.15.153:0/471093540 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f2004197020 con 0x7f2004105a40 2024-03-21T10:13:14.609 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:14.607+0000 7f2001600700 1 -- 172.21.15.153:0/471093540 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 21 v21) v1 ==== 76+0+1850 (secure 0 0 0) 0x7f1ff005c1e0 con 0x7f2004105a40 2024-03-21T10:13:14.610 INFO:teuthology.orchestra.run.smithi153.stdout:e21 2024-03-21T10:13:14.610 INFO:teuthology.orchestra.run.smithi153.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-21T10:13:14.610 INFO:teuthology.orchestra.run.smithi153.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:13:14.610 INFO:teuthology.orchestra.run.smithi153.stdout:legacy client fscid: 1 2024-03-21T10:13:14.610 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:13:14.610 INFO:teuthology.orchestra.run.smithi153.stdout:Filesystem 'cephfs' (1) 2024-03-21T10:13:14.610 INFO:teuthology.orchestra.run.smithi153.stdout:fs_name cephfs 2024-03-21T10:13:14.610 INFO:teuthology.orchestra.run.smithi153.stdout:epoch 21 2024-03-21T10:13:14.610 INFO:teuthology.orchestra.run.smithi153.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-21T10:13:14.611 INFO:teuthology.orchestra.run.smithi153.stdout:created 2024-03-21T10:04:27.086576+0000 2024-03-21T10:13:14.611 INFO:teuthology.orchestra.run.smithi153.stdout:modified 2024-03-21T10:13:01.455594+0000 2024-03-21T10:13:14.611 INFO:teuthology.orchestra.run.smithi153.stdout:tableserver 0 2024-03-21T10:13:14.611 INFO:teuthology.orchestra.run.smithi153.stdout:root 0 2024-03-21T10:13:14.611 INFO:teuthology.orchestra.run.smithi153.stdout:session_timeout 60 2024-03-21T10:13:14.611 INFO:teuthology.orchestra.run.smithi153.stdout:session_autoclose 300 2024-03-21T10:13:14.611 INFO:teuthology.orchestra.run.smithi153.stdout:max_file_size 1099511627776 2024-03-21T10:13:14.611 INFO:teuthology.orchestra.run.smithi153.stdout:required_client_features {} 2024-03-21T10:13:14.611 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure 0 2024-03-21T10:13:14.611 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure_osd_epoch 47 2024-03-21T10:13:14.611 INFO:teuthology.orchestra.run.smithi153.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:13:14.611 INFO:teuthology.orchestra.run.smithi153.stdout:max_mds 2 2024-03-21T10:13:14.611 INFO:teuthology.orchestra.run.smithi153.stdout:in 0,1 2024-03-21T10:13:14.611 INFO:teuthology.orchestra.run.smithi153.stdout:up {0=14484,1=24273} 2024-03-21T10:13:14.611 INFO:teuthology.orchestra.run.smithi153.stdout:failed 2024-03-21T10:13:14.612 INFO:teuthology.orchestra.run.smithi153.stdout:damaged 2024-03-21T10:13:14.612 INFO:teuthology.orchestra.run.smithi153.stdout:stopped 2024-03-21T10:13:14.612 INFO:teuthology.orchestra.run.smithi153.stdout:data_pools [3] 2024-03-21T10:13:14.612 INFO:teuthology.orchestra.run.smithi153.stdout:metadata_pool 2 2024-03-21T10:13:14.612 INFO:teuthology.orchestra.run.smithi153.stdout:inline_data enabled 2024-03-21T10:13:14.612 INFO:teuthology.orchestra.run.smithi153.stdout:balancer 2024-03-21T10:13:14.612 INFO:teuthology.orchestra.run.smithi153.stdout:bal_rank_mask -1 2024-03-21T10:13:14.612 INFO:teuthology.orchestra.run.smithi153.stdout:standby_count_wanted 1 2024-03-21T10:13:14.612 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.iuyluj{0:14484} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.153:6828/866138999,v1:172.21.15.153:6829/866138999] compat {c=[1],r=[1],i=[7ff]}] 2024-03-21T10:13:14.612 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.zipckr{0:14494} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.153:6826/4033622615,v1:172.21.15.153:6827/4033622615] compat {c=[1],r=[1],i=[7ff]}] 2024-03-21T10:13:14.612 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.yyimst{1:24273} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.174:6824/1776947618,v1:172.21.15.174:6825/1776947618] compat {c=[1],r=[1],i=[7ff]}] 2024-03-21T10:13:14.612 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.nhozag{1:24285} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.174:6826/806095873,v1:172.21.15.174:6827/806095873] compat {c=[1],r=[1],i=[7ff]}] 2024-03-21T10:13:14.612 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:13:14.612 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:13:14.613 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:14.610+0000 7f200ba00700 1 -- 172.21.15.153:0/471093540 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f1fec06e0e0 msgr2=0x7f1fec0705a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:13:14.613 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:14.610+0000 7f200ba00700 1 --2- 172.21.15.153:0/471093540 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f1fec06e0e0 0x7f1fec0705a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1ff8005ea0 tx=0x7f1ff8005e30 comp rx=0 tx=0).stop 2024-03-21T10:13:14.613 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:14.610+0000 7f200ba00700 1 -- 172.21.15.153:0/471093540 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2004105a40 msgr2=0x7f200419ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:13:14.613 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:14.610+0000 7f200ba00700 1 --2- 172.21.15.153:0/471093540 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2004105a40 0x7f200419ca40 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f1ff000b5c0 tx=0x7f1ff000be00 comp rx=0 tx=0).stop 2024-03-21T10:13:14.614 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:14.610+0000 7f200ba00700 1 -- 172.21.15.153:0/471093540 shutdown_connections 2024-03-21T10:13:14.614 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:14.610+0000 7f200ba00700 1 --2- 172.21.15.153:0/471093540 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f1fec06e0e0 0x7f1fec0705a0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:14.614 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:14.610+0000 7f200ba00700 1 --2- 172.21.15.153:0/471093540 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f20040ffbd0 0x7f200419c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:14.614 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:14.610+0000 7f200ba00700 1 --2- 172.21.15.153:0/471093540 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2004105a40 0x7f200419ca40 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:14.614 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:14.610+0000 7f200ba00700 1 -- 172.21.15.153:0/471093540 >> 172.21.15.153:0/471093540 conn(0x7f20040755a0 msgr2=0x7f2004068650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:13:14.614 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:14.610+0000 7f200ba00700 1 -- 172.21.15.153:0/471093540 shutdown_connections 2024-03-21T10:13:14.614 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:14.610+0000 7f200ba00700 1 -- 172.21.15.153:0/471093540 wait complete. 2024-03-21T10:13:14.614 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 21 2024-03-21T10:13:14.763 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:14.761+0000 7f86d9200700 1 -- 172.21.15.153:0/1944475471 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f86d40752c0 msgr2=0x7f86d40756a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:13:14.763 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:14.761+0000 7f86d9200700 1 --2- 172.21.15.153:0/1944475471 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f86d40752c0 0x7f86d40756a0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f86c4009910 tx=0x7f86c4009c20 comp rx=0 tx=0).stop 2024-03-21T10:13:14.763 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:14.761+0000 7f86d9200700 1 -- 172.21.15.153:0/1944475471 shutdown_connections 2024-03-21T10:13:14.763 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:14.761+0000 7f86d9200700 1 --2- 172.21.15.153:0/1944475471 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f86d4075be0 0x7f86d41114f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:14.764 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:14.761+0000 7f86d9200700 1 --2- 172.21.15.153:0/1944475471 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f86d40752c0 0x7f86d40756a0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:14.764 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:14.761+0000 7f86d9200700 1 -- 172.21.15.153:0/1944475471 >> 172.21.15.153:0/1944475471 conn(0x7f86d40fe8a0 msgr2=0x7f86d4100cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:13:14.764 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:14.761+0000 7f86d9200700 1 -- 172.21.15.153:0/1944475471 shutdown_connections 2024-03-21T10:13:14.764 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:14.761+0000 7f86d9200700 1 -- 172.21.15.153:0/1944475471 wait complete. 2024-03-21T10:13:14.764 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:14.763+0000 7f86d9200700 1 Processor -- start 2024-03-21T10:13:14.764 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:14.763+0000 7f86d9200700 1 -- start start 2024-03-21T10:13:14.765 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:14.764+0000 7f86d9200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f86d40752c0 0x7f86d419cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:13:14.765 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:14.764+0000 7f86d9200700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f86d4075be0 0x7f86d419d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:13:14.765 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:14.764+0000 7f86d9200700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86d419da10 con 0x7f86d40752c0 2024-03-21T10:13:14.765 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:14.764+0000 7f86d9200700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86d41a17a0 con 0x7f86d4075be0 2024-03-21T10:13:14.766 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:14.764+0000 7f86d1e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f86d40752c0 0x7f86d419cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:13:14.766 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:14.764+0000 7f86d1e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f86d40752c0 0x7f86d419cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:44826/0 (socket says 172.21.15.153:44826) 2024-03-21T10:13:14.766 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:14.764+0000 7f86d1e00700 1 -- 172.21.15.153:0/2644974136 learned_addr learned my addr 172.21.15.153:0/2644974136 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:13:14.767 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:14.764+0000 7f86d1e00700 1 -- 172.21.15.153:0/2644974136 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f86d4075be0 msgr2=0x7f86d419d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:13:14.767 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:14.764+0000 7f86d1400700 1 --2- 172.21.15.153:0/2644974136 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f86d4075be0 0x7f86d419d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:13:14.767 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:14.764+0000 7f86d1e00700 1 --2- 172.21.15.153:0/2644974136 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f86d4075be0 0x7f86d419d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:14.767 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:14.764+0000 7f86d1e00700 1 -- 172.21.15.153:0/2644974136 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86c40095f0 con 0x7f86d40752c0 2024-03-21T10:13:14.767 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:14.764+0000 7f86d1400700 1 --2- 172.21.15.153:0/2644974136 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f86d4075be0 0x7f86d419d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:13:14.767 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:14.765+0000 7f86d1e00700 1 --2- 172.21.15.153:0/2644974136 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f86d40752c0 0x7f86d419cdf0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f86c4005b40 tx=0x7f86c40056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:13:14.767 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:14.765+0000 7f86caa00700 1 -- 172.21.15.153:0/2644974136 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f86c401d070 con 0x7f86d40752c0 2024-03-21T10:13:14.767 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:14.765+0000 7f86caa00700 1 -- 172.21.15.153:0/2644974136 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f86c400f460 con 0x7f86d40752c0 2024-03-21T10:13:14.767 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:14.765+0000 7f86caa00700 1 -- 172.21.15.153:0/2644974136 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f86c4021640 con 0x7f86d40752c0 2024-03-21T10:13:14.767 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:14.765+0000 7f86d9200700 1 -- 172.21.15.153:0/2644974136 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86d41a1a80 con 0x7f86d40752c0 2024-03-21T10:13:14.767 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:14.765+0000 7f86d9200700 1 -- 172.21.15.153:0/2644974136 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86d41a1fd0 con 0x7f86d40752c0 2024-03-21T10:13:14.768 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:14.766+0000 7f86d9200700 1 -- 172.21.15.153:0/2644974136 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86d410ec60 con 0x7f86d40752c0 2024-03-21T10:13:14.769 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:14.767+0000 7f86caa00700 1 -- 172.21.15.153:0/2644974136 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 31) v1 ==== 92072+0+0 (secure 0 0 0) 0x7f86c400f5d0 con 0x7f86d40752c0 2024-03-21T10:13:14.770 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:14.768+0000 7f86caa00700 1 --2- 172.21.15.153:0/2644974136 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f86b806e0e0 0x7f86b80705a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:13:14.770 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:14.768+0000 7f86caa00700 1 -- 172.21.15.153:0/2644974136 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f86c408f170 con 0x7f86d40752c0 2024-03-21T10:13:14.770 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:14.769+0000 7f86d1400700 1 --2- 172.21.15.153:0/2644974136 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f86b806e0e0 0x7f86b80705a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:13:14.771 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:14.769+0000 7f86d1400700 1 --2- 172.21.15.153:0/2644974136 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f86b806e0e0 0x7f86b80705a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f86c0005fd0 tx=0x7f86c0005dc0 comp rx=0 tx=0).ready entity=mgr.14666 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:13:14.774 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:14.773+0000 7f86caa00700 1 -- 172.21.15.153:0/2644974136 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185454 (secure 0 0 0) 0x7f86c4058830 con 0x7f86d40752c0 2024-03-21T10:13:15.015 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:15.013+0000 7f86d9200700 1 -- 172.21.15.153:0/2644974136 --> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f86d41a2300 con 0x7f86b806e0e0 2024-03-21T10:13:15.017 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:15.015+0000 7f86caa00700 1 -- 172.21.15.153:0/2644974136 <== mgr.14666 v2:172.21.15.153:6800/2524145044 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7f86d41a2300 con 0x7f86b806e0e0 2024-03-21T10:13:15.017 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:13:15.017 INFO:teuthology.orchestra.run.smithi153.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232", 2024-03-21T10:13:15.017 INFO:teuthology.orchestra.run.smithi153.stdout: "in_progress": true, 2024-03-21T10:13:15.017 INFO:teuthology.orchestra.run.smithi153.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-21T10:13:15.017 INFO:teuthology.orchestra.run.smithi153.stdout: "services_complete": [ 2024-03-21T10:13:15.017 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr" 2024-03-21T10:13:15.017 INFO:teuthology.orchestra.run.smithi153.stdout: ], 2024-03-21T10:13:15.018 INFO:teuthology.orchestra.run.smithi153.stdout: "progress": "2/23 daemons upgraded", 2024-03-21T10:13:15.018 INFO:teuthology.orchestra.run.smithi153.stdout: "message": "Currently upgrading mon daemons", 2024-03-21T10:13:15.018 INFO:teuthology.orchestra.run.smithi153.stdout: "is_paused": false 2024-03-21T10:13:15.018 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:13:15.020 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:15.018+0000 7f86d9200700 1 -- 172.21.15.153:0/2644974136 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f86b806e0e0 msgr2=0x7f86b80705a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:13:15.020 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:15.018+0000 7f86d9200700 1 --2- 172.21.15.153:0/2644974136 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f86b806e0e0 0x7f86b80705a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f86c0005fd0 tx=0x7f86c0005dc0 comp rx=0 tx=0).stop 2024-03-21T10:13:15.020 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:15.018+0000 7f86d9200700 1 -- 172.21.15.153:0/2644974136 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f86d40752c0 msgr2=0x7f86d419cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:13:15.020 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:15.018+0000 7f86d9200700 1 --2- 172.21.15.153:0/2644974136 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f86d40752c0 0x7f86d419cdf0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f86c4005b40 tx=0x7f86c40056a0 comp rx=0 tx=0).stop 2024-03-21T10:13:15.020 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:15.018+0000 7f86d9200700 1 -- 172.21.15.153:0/2644974136 shutdown_connections 2024-03-21T10:13:15.020 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:15.018+0000 7f86d9200700 1 --2- 172.21.15.153:0/2644974136 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7f86b806e0e0 0x7f86b80705a0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:15.020 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:15.018+0000 7f86d9200700 1 --2- 172.21.15.153:0/2644974136 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f86d40752c0 0x7f86d419cdf0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:15.020 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:15.018+0000 7f86d9200700 1 --2- 172.21.15.153:0/2644974136 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f86d4075be0 0x7f86d419d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:15.020 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:15.018+0000 7f86d9200700 1 -- 172.21.15.153:0/2644974136 >> 172.21.15.153:0/2644974136 conn(0x7f86d40fe8a0 msgr2=0x7f86d40ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:13:15.020 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:15.018+0000 7f86d9200700 1 -- 172.21.15.153:0/2644974136 shutdown_connections 2024-03-21T10:13:15.021 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:15.018+0000 7f86d9200700 1 -- 172.21.15.153:0/2644974136 wait complete. 2024-03-21T10:13:15.182 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:15.180+0000 7fd19fa00700 1 -- 172.21.15.153:0/147608518 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd1981026a0 msgr2=0x7fd19810ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:13:15.182 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:15.180+0000 7fd19fa00700 1 --2- 172.21.15.153:0/147608518 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd1981026a0 0x7fd19810ab90 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fd184009940 tx=0x7fd184009c50 comp rx=0 tx=0).stop 2024-03-21T10:13:15.183 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:14 smithi153 ceph-mon[29117]: from='client.14884 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:13:15.183 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:14 smithi153 ceph-mon[29117]: from='client.? 172.21.15.153:0/3022462556' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:13:15.183 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:14 smithi153 ceph-mon[29117]: from='client.? 172.21.15.153:0/471093540' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:13:15.183 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:15.181+0000 7fd19fa00700 1 -- 172.21.15.153:0/147608518 shutdown_connections 2024-03-21T10:13:15.183 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:15.181+0000 7fd19fa00700 1 --2- 172.21.15.153:0/147608518 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd1981026a0 0x7fd19810ab90 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:15.184 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:15.181+0000 7fd19fa00700 1 --2- 172.21.15.153:0/147608518 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd198101d80 0x7fd198102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:15.184 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:15.181+0000 7fd19fa00700 1 -- 172.21.15.153:0/147608518 >> 172.21.15.153:0/147608518 conn(0x7fd1980fb570 msgr2=0x7fd1980fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:13:15.184 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:15.181+0000 7fd19fa00700 1 -- 172.21.15.153:0/147608518 shutdown_connections 2024-03-21T10:13:15.184 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:15.181+0000 7fd19fa00700 1 -- 172.21.15.153:0/147608518 wait complete. 2024-03-21T10:13:15.184 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:15.182+0000 7fd19fa00700 1 Processor -- start 2024-03-21T10:13:15.184 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:15.183+0000 7fd19fa00700 1 -- start start 2024-03-21T10:13:15.185 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:15.183+0000 7fd19fa00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd198101d80 0x7fd1980ff9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:13:15.185 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:15.183+0000 7fd19fa00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd1981026a0 0x7fd1980fff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:13:15.185 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:15.183+0000 7fd19fa00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd198100460 con 0x7fd1981026a0 2024-03-21T10:13:15.185 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:15.183+0000 7fd19fa00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1981005a0 con 0x7fd198101d80 2024-03-21T10:13:15.185 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:15.184+0000 7fd197e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd1981026a0 0x7fd1980fff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:13:15.186 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:15.184+0000 7fd197e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd1981026a0 0x7fd1980fff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:44854/0 (socket says 172.21.15.153:44854) 2024-03-21T10:13:15.186 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:15.184+0000 7fd197e00700 1 -- 172.21.15.153:0/2627761604 learned_addr learned my addr 172.21.15.153:0/2627761604 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:13:15.186 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:15.184+0000 7fd19d000700 1 --2- 172.21.15.153:0/2627761604 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd198101d80 0x7fd1980ff9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:13:15.186 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:15.184+0000 7fd197e00700 1 -- 172.21.15.153:0/2627761604 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd198101d80 msgr2=0x7fd1980ff9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:13:15.186 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:15.184+0000 7fd197e00700 1 --2- 172.21.15.153:0/2627761604 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd198101d80 0x7fd1980ff9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:15.186 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:15.184+0000 7fd197e00700 1 -- 172.21.15.153:0/2627761604 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1840095f0 con 0x7fd1981026a0 2024-03-21T10:13:15.186 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:15.184+0000 7fd19d000700 1 --2- 172.21.15.153:0/2627761604 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd198101d80 0x7fd1980ff9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:13:15.186 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:15.185+0000 7fd197e00700 1 --2- 172.21.15.153:0/2627761604 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd1981026a0 0x7fd1980fff20 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fd184005b40 tx=0x7fd1840056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:13:15.187 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:15.185+0000 7fd195600700 1 -- 172.21.15.153:0/2627761604 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd18401d070 con 0x7fd1981026a0 2024-03-21T10:13:15.187 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:15.185+0000 7fd195600700 1 -- 172.21.15.153:0/2627761604 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd18400f460 con 0x7fd1981026a0 2024-03-21T10:13:15.187 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:15.185+0000 7fd19fa00700 1 -- 172.21.15.153:0/2627761604 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd198100820 con 0x7fd1981026a0 2024-03-21T10:13:15.187 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:15.185+0000 7fd195600700 1 -- 172.21.15.153:0/2627761604 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd184021640 con 0x7fd1981026a0 2024-03-21T10:13:15.187 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:15.185+0000 7fd19fa00700 1 -- 172.21.15.153:0/2627761604 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1981a2db0 con 0x7fd1981026a0 2024-03-21T10:13:15.188 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:15.187+0000 7fd19fa00700 1 -- 172.21.15.153:0/2627761604 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd198108290 con 0x7fd1981026a0 2024-03-21T10:13:15.189 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:15.187+0000 7fd195600700 1 -- 172.21.15.153:0/2627761604 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 31) v1 ==== 92072+0+0 (secure 0 0 0) 0x7fd18400fa60 con 0x7fd1981026a0 2024-03-21T10:13:15.190 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:15.189+0000 7fd195600700 1 --2- 172.21.15.153:0/2627761604 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7fd18006e0e0 0x7fd1800705a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:13:15.190 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:15.189+0000 7fd195600700 1 -- 172.21.15.153:0/2627761604 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fd18408f120 con 0x7fd1981026a0 2024-03-21T10:13:15.191 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:15.189+0000 7fd19d000700 1 --2- 172.21.15.153:0/2627761604 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7fd18006e0e0 0x7fd1800705a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:13:15.191 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:15.190+0000 7fd19d000700 1 --2- 172.21.15.153:0/2627761604 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7fd18006e0e0 0x7fd1800705a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd18c0099d0 tx=0x7fd18c008040 comp rx=0 tx=0).ready entity=mgr.14666 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:13:15.196 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:15.194+0000 7fd195600700 1 -- 172.21.15.153:0/2627761604 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185454 (secure 0 0 0) 0x7fd1840587e0 con 0x7fd1981026a0 2024-03-21T10:13:15.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:14 smithi174 ceph-mon[37496]: from='client.14884 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:13:15.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:14 smithi174 ceph-mon[37496]: from='client.? 172.21.15.153:0/3022462556' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:13:15.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:14 smithi174 ceph-mon[37496]: from='client.? 172.21.15.153:0/471093540' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:13:15.517 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:15.515+0000 7fd19fa00700 1 -- 172.21.15.153:0/2627761604 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fd19804ea90 con 0x7fd1981026a0 2024-03-21T10:13:15.518 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:15.516+0000 7fd195600700 1 -- 172.21.15.153:0/2627761604 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7fd18405bea0 con 0x7fd1981026a0 2024-03-21T10:13:15.518 INFO:teuthology.orchestra.run.smithi153.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-03-21T10:13:15.519 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-21T10:13:15.519 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-21T10:13:15.522 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:15.520+0000 7fd19fa00700 1 -- 172.21.15.153:0/2627761604 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7fd18006e0e0 msgr2=0x7fd1800705a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:13:15.522 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:15.520+0000 7fd19fa00700 1 --2- 172.21.15.153:0/2627761604 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7fd18006e0e0 0x7fd1800705a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd18c0099d0 tx=0x7fd18c008040 comp rx=0 tx=0).stop 2024-03-21T10:13:15.522 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:15.520+0000 7fd19fa00700 1 -- 172.21.15.153:0/2627761604 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd1981026a0 msgr2=0x7fd1980fff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:13:15.522 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:15.520+0000 7fd19fa00700 1 --2- 172.21.15.153:0/2627761604 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd1981026a0 0x7fd1980fff20 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fd184005b40 tx=0x7fd1840056a0 comp rx=0 tx=0).stop 2024-03-21T10:13:15.522 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:15.520+0000 7fd19fa00700 1 -- 172.21.15.153:0/2627761604 shutdown_connections 2024-03-21T10:13:15.522 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:15.520+0000 7fd19fa00700 1 --2- 172.21.15.153:0/2627761604 >> [v2:172.21.15.153:6800/2524145044,v1:172.21.15.153:6801/2524145044] conn(0x7fd18006e0e0 0x7fd1800705a0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:15.522 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:15.520+0000 7fd19fa00700 1 --2- 172.21.15.153:0/2627761604 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd198101d80 0x7fd1980ff9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:15.522 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:15.520+0000 7fd19fa00700 1 --2- 172.21.15.153:0/2627761604 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd1981026a0 0x7fd1980fff20 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:15.522 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:15.520+0000 7fd19fa00700 1 -- 172.21.15.153:0/2627761604 >> 172.21.15.153:0/2627761604 conn(0x7fd1980fb570 msgr2=0x7fd1981053d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:13:15.522 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:15.520+0000 7fd19fa00700 1 -- 172.21.15.153:0/2627761604 shutdown_connections 2024-03-21T10:13:15.523 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:15.520+0000 7fd19fa00700 1 -- 172.21.15.153:0/2627761604 wait complete. 2024-03-21T10:13:16.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:16 smithi153 ceph-mon[29117]: from='client.14896 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:13:16.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:16 smithi153 ceph-mon[29117]: Upgrade: Updating mon.smithi153 2024-03-21T10:13:16.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:16 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:13:16.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:16 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T10:13:16.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:16 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-21T10:13:16.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:16 smithi153 ceph-mon[29117]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:13:16.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:16 smithi153 ceph-mon[29117]: Deploying daemon mon.smithi153 on smithi153 2024-03-21T10:13:16.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:16 smithi153 ceph-mon[29117]: pgmap v144: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:13:16.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:16 smithi153 ceph-mon[29117]: from='client.? 172.21.15.153:0/2627761604' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:13:16.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:16 smithi174 ceph-mon[37496]: from='client.14896 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:13:16.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:16 smithi174 ceph-mon[37496]: Upgrade: Updating mon.smithi153 2024-03-21T10:13:16.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:16 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' 2024-03-21T10:13:16.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:16 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T10:13:16.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:16 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-21T10:13:16.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:16 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:13:16.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:16 smithi174 ceph-mon[37496]: Deploying daemon mon.smithi153 on smithi153 2024-03-21T10:13:16.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:16 smithi174 ceph-mon[37496]: pgmap v144: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:13:16.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:16 smithi174 ceph-mon[37496]: from='client.? 172.21.15.153:0/2627761604' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:13:18.271 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:17 smithi153 systemd[1]: Stopping Ceph mon.smithi153 for 070ed3e4-e769-11ee-95cd-87774f69a715... 2024-03-21T10:13:18.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:18 smithi153 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-mon-smithi153[29113]: 2024-03-21T10:13:18.269+0000 7fc70c000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi153 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T10:13:18.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:18 smithi153 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-mon-smithi153[29113]: 2024-03-21T10:13:18.269+0000 7fc70c000700 -1 mon.smithi153@0(leader) e2 *** Got Signal Terminated *** 2024-03-21T10:13:18.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:18 smithi153 podman[113563]: 2024-03-21 10:13:18.331013765 +0000 UTC m=+0.216265140 container died 813963a95d695aa24cfc165fa64d61eefb808093dabd0ae5739a25830dddc77d (image=quay.io/ceph/ceph:v18.2.1, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-mon-smithi153, GIT_BRANCH=HEAD, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=-18.2.1, org.label-schema.build-date=20240222, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.29.1, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True) 2024-03-21T10:13:18.979 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:18 smithi153 podman[113563]: 2024-03-21 10:13:18.777655785 +0000 UTC m=+0.662907157 container cleanup 813963a95d695aa24cfc165fa64d61eefb808093dabd0ae5739a25830dddc77d (image=quay.io/ceph/ceph:v18.2.1, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-mon-smithi153, RELEASE=HEAD, org.label-schema.license=GPLv2, GIT_REPO=https://github.com/ceph/ceph-container.git, io.buildah.version=1.29.1, org.label-schema.build-date=20240222, CEPH_POINT_RELEASE=-18.2.1, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS) 2024-03-21T10:13:18.979 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:18 smithi153 bash[113563]: ceph-070ed3e4-e769-11ee-95cd-87774f69a715-mon-smithi153 2024-03-21T10:13:19.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:18 smithi153 podman[113577]: 2024-03-21 10:13:18.978398262 +0000 UTC m=+0.638576862 container remove 813963a95d695aa24cfc165fa64d61eefb808093dabd0ae5739a25830dddc77d (image=quay.io/ceph/ceph:v18.2.1, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-mon-smithi153, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=HEAD, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=-18.2.1, GIT_CLEAN=True, io.buildah.version=1.29.1) 2024-03-21T10:13:19.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:19 smithi153 systemd[1]: ceph-070ed3e4-e769-11ee-95cd-87774f69a715@mon.smithi153.service: Deactivated successfully. 2024-03-21T10:13:19.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:19 smithi153 systemd[1]: Stopped Ceph mon.smithi153 for 070ed3e4-e769-11ee-95cd-87774f69a715. 2024-03-21T10:13:19.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:19 smithi153 systemd[1]: ceph-070ed3e4-e769-11ee-95cd-87774f69a715@mon.smithi153.service: Consumed 32.459s CPU time. 2024-03-21T10:13:20.181 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:19 smithi153 systemd[1]: Starting Ceph mon.smithi153 for 070ed3e4-e769-11ee-95cd-87774f69a715... 2024-03-21T10:13:20.434 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 podman[113715]: 2024-03-21 10:13:20.088021552 +0000 UTC m=+0.034552347 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T10:13:20.434 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 podman[113715]: 2024-03-21 10:13:20.229627137 +0000 UTC m=+0.176157956 container create dd8038303dc317e8543d4a848b533df3ed09720587219ba01a349d97247df207 (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-mon-smithi153, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, ceph=True, maintainer=Guillaume Abrioux ) 2024-03-21T10:13:20.809 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 podman[113715]: 2024-03-21 10:13:20.516853085 +0000 UTC m=+0.463383948 container init dd8038303dc317e8543d4a848b533df3ed09720587219ba01a349d97247df207 (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-mon-smithi153, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319) 2024-03-21T10:13:20.867 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 podman[113715]: 2024-03-21 10:13:20.527837287 +0000 UTC m=+0.474368076 container start dd8038303dc317e8543d4a848b533df3ed09720587219ba01a349d97247df207 (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-mon-smithi153, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-03-21T10:13:20.868 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: set uid:gid to 167:167 (ceph:ceph) 2024-03-21T10:13:20.868 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev), process ceph-mon, pid 2 2024-03-21T10:13:20.868 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: pidfile_write: ignore empty --pid-file 2024-03-21T10:13:20.868 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: load: jerasure load: lrc 2024-03-21T10:13:20.868 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: RocksDB version: 7.9.2 2024-03-21T10:13:20.868 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Git sha 0 2024-03-21T10:13:20.868 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Compile date 2024-03-20 06:23:48 2024-03-21T10:13:20.868 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: DB SUMMARY 2024-03-21T10:13:20.868 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: DB Session ID: WVYYT207RWKGAPMT5OXJ 2024-03-21T10:13:20.868 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: CURRENT file: CURRENT 2024-03-21T10:13:20.868 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: IDENTITY file: IDENTITY 2024-03-21T10:13:20.868 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: MANIFEST file: MANIFEST-000015 size: 1746 Bytes 2024-03-21T10:13:20.868 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi153/store.db dir, Total Num: 1, files: 000035.sst 2024-03-21T10:13:20.869 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi153/store.db: 000033.log size: 596893 ; 2024-03-21T10:13:20.869 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.error_if_exists: 0 2024-03-21T10:13:20.869 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.create_if_missing: 0 2024-03-21T10:13:20.869 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.paranoid_checks: 1 2024-03-21T10:13:20.869 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-21T10:13:20.869 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-21T10:13:20.869 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-21T10:13:20.869 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.env: 0x55ea9cec1b20 2024-03-21T10:13:20.869 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.fs: PosixFileSystem 2024-03-21T10:13:20.869 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.info_log: 0x55ea9eb97840 2024-03-21T10:13:20.869 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.max_file_opening_threads: 16 2024-03-21T10:13:20.871 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.statistics: (nil) 2024-03-21T10:13:20.871 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.use_fsync: 0 2024-03-21T10:13:20.871 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.max_log_file_size: 0 2024-03-21T10:13:20.872 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-21T10:13:20.872 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-21T10:13:20.872 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.keep_log_file_num: 1000 2024-03-21T10:13:20.872 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.recycle_log_file_num: 0 2024-03-21T10:13:20.872 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.allow_fallocate: 1 2024-03-21T10:13:20.872 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.allow_mmap_reads: 0 2024-03-21T10:13:20.872 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.allow_mmap_writes: 0 2024-03-21T10:13:20.872 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.use_direct_reads: 0 2024-03-21T10:13:20.872 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-21T10:13:20.872 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.create_missing_column_families: 0 2024-03-21T10:13:20.872 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.db_log_dir: 2024-03-21T10:13:20.873 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.wal_dir: 2024-03-21T10:13:20.873 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-21T10:13:20.873 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-21T10:13:20.873 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-21T10:13:20.873 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-21T10:13:20.873 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-21T10:13:20.873 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-21T10:13:20.873 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.advise_random_on_open: 1 2024-03-21T10:13:20.873 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.db_write_buffer_size: 0 2024-03-21T10:13:20.873 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.write_buffer_manager: 0x55ea9eb6a1e0 2024-03-21T10:13:20.873 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-21T10:13:20.874 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-21T10:13:20.874 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-21T10:13:20.874 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.rate_limiter: (nil) 2024-03-21T10:13:20.874 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-21T10:13:20.874 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.wal_recovery_mode: 2 2024-03-21T10:13:20.874 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.enable_thread_tracking: 0 2024-03-21T10:13:20.874 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.enable_pipelined_write: 0 2024-03-21T10:13:20.874 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.unordered_write: 0 2024-03-21T10:13:20.874 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-21T10:13:20.874 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-21T10:13:20.874 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-21T10:13:20.874 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-21T10:13:20.874 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.row_cache: None 2024-03-21T10:13:20.874 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.wal_filter: None 2024-03-21T10:13:20.875 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-21T10:13:20.875 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.allow_ingest_behind: 0 2024-03-21T10:13:20.875 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.two_write_queues: 0 2024-03-21T10:13:20.875 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.manual_wal_flush: 0 2024-03-21T10:13:20.875 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.wal_compression: 0 2024-03-21T10:13:20.875 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.atomic_flush: 0 2024-03-21T10:13:20.875 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-21T10:13:20.875 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-21T10:13:20.875 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-21T10:13:20.875 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.log_readahead_size: 0 2024-03-21T10:13:20.875 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-21T10:13:20.875 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.best_efforts_recovery: 0 2024-03-21T10:13:20.875 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-21T10:13:20.876 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-21T10:13:20.876 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.allow_data_in_errors: 0 2024-03-21T10:13:20.876 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.db_host_id: __hostname__ 2024-03-21T10:13:20.876 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-21T10:13:20.876 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.max_background_jobs: 2 2024-03-21T10:13:20.876 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.max_background_compactions: -1 2024-03-21T10:13:20.876 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.max_subcompactions: 1 2024-03-21T10:13:20.876 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-21T10:13:20.876 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-21T10:13:20.876 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-21T10:13:20.876 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.max_total_wal_size: 0 2024-03-21T10:13:20.876 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-21T10:13:20.876 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-21T10:13:20.876 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-21T10:13:20.877 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-21T10:13:20.877 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.max_open_files: -1 2024-03-21T10:13:20.877 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.bytes_per_sync: 0 2024-03-21T10:13:20.877 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-21T10:13:20.877 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-21T10:13:20.877 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.compaction_readahead_size: 0 2024-03-21T10:13:20.877 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.max_background_flushes: -1 2024-03-21T10:13:20.877 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Compression algorithms supported: 2024-03-21T10:13:20.877 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-21T10:13:20.877 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: kZSTD supported: 0 2024-03-21T10:13:20.877 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: kXpressCompression supported: 0 2024-03-21T10:13:20.877 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: kLZ4HCCompression supported: 1 2024-03-21T10:13:20.877 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: kZlibCompression supported: 1 2024-03-21T10:13:20.878 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: kSnappyCompression supported: 1 2024-03-21T10:13:20.878 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: kLZ4Compression supported: 1 2024-03-21T10:13:20.878 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: kBZip2Compression supported: 0 2024-03-21T10:13:20.878 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-21T10:13:20.878 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-21T10:13:20.878 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi153/store.db/MANIFEST-000015 2024-03-21T10:13:20.878 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-21T10:13:20.878 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-21T10:13:20.878 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.merge_operator: 2024-03-21T10:13:20.878 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.compaction_filter: None 2024-03-21T10:13:20.878 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.compaction_filter_factory: None 2024-03-21T10:13:20.878 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.sst_partitioner_factory: None 2024-03-21T10:13:20.879 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-21T10:13:20.879 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-21T10:13:20.879 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55ea9eb97940) 2024-03-21T10:13:20.879 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: cache_index_and_filter_blocks: 1 2024-03-21T10:13:20.879 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-21T10:13:20.879 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-21T10:13:20.879 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: pin_top_level_index_and_filter: 1 2024-03-21T10:13:20.879 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: index_type: 0 2024-03-21T10:13:20.879 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: data_block_index_type: 0 2024-03-21T10:13:20.879 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: index_shortening: 1 2024-03-21T10:13:20.879 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-21T10:13:20.879 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: checksum: 4 2024-03-21T10:13:20.879 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: no_block_cache: 0 2024-03-21T10:13:20.879 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: block_cache: 0x55ea9eb9d090 2024-03-21T10:13:20.879 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: block_cache_name: BinnedLRUCache 2024-03-21T10:13:20.880 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: block_cache_options: 2024-03-21T10:13:20.880 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: capacity : 536870912 2024-03-21T10:13:20.880 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: num_shard_bits : 4 2024-03-21T10:13:20.880 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: strict_capacity_limit : 0 2024-03-21T10:13:20.880 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: high_pri_pool_ratio: 0.000 2024-03-21T10:13:20.880 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: block_cache_compressed: (nil) 2024-03-21T10:13:20.880 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: persistent_cache: (nil) 2024-03-21T10:13:20.880 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: block_size: 4096 2024-03-21T10:13:20.880 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: block_size_deviation: 10 2024-03-21T10:13:20.880 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: block_restart_interval: 16 2024-03-21T10:13:20.880 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: index_block_restart_interval: 1 2024-03-21T10:13:20.880 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: metadata_block_size: 4096 2024-03-21T10:13:20.881 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: partition_filters: 0 2024-03-21T10:13:20.881 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: use_delta_encoding: 1 2024-03-21T10:13:20.881 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: filter_policy: bloomfilter 2024-03-21T10:13:20.881 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: whole_key_filtering: 1 2024-03-21T10:13:20.881 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: verify_compression: 0 2024-03-21T10:13:20.881 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: read_amp_bytes_per_bit: 0 2024-03-21T10:13:20.881 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: format_version: 5 2024-03-21T10:13:20.881 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: enable_index_compression: 1 2024-03-21T10:13:20.881 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: block_align: 0 2024-03-21T10:13:20.881 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: max_auto_readahead_size: 262144 2024-03-21T10:13:20.881 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: prepopulate_block_cache: 0 2024-03-21T10:13:20.881 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: initial_auto_readahead_size: 8192 2024-03-21T10:13:20.881 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: num_file_reads_for_auto_readahead: 2 2024-03-21T10:13:20.881 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.write_buffer_size: 33554432 2024-03-21T10:13:20.882 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.max_write_buffer_number: 2 2024-03-21T10:13:20.882 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.compression: NoCompression 2024-03-21T10:13:20.882 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.bottommost_compression: Disabled 2024-03-21T10:13:20.882 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.prefix_extractor: nullptr 2024-03-21T10:13:20.882 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-21T10:13:20.882 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.num_levels: 7 2024-03-21T10:13:20.882 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-21T10:13:20.882 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-21T10:13:20.882 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-21T10:13:20.882 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-21T10:13:20.882 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-21T10:13:20.882 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-21T10:13:20.883 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-21T10:13:20.883 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-21T10:13:20.883 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-21T10:13:20.883 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-21T10:13:20.883 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-21T10:13:20.883 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-21T10:13:20.883 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-21T10:13:20.883 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.compression_opts.level: 32767 2024-03-21T10:13:20.883 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.compression_opts.strategy: 0 2024-03-21T10:13:20.883 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-21T10:13:20.883 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-21T10:13:20.883 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-21T10:13:20.883 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-21T10:13:20.884 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.compression_opts.enabled: false 2024-03-21T10:13:20.884 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-21T10:13:20.884 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-21T10:13:20.884 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-21T10:13:20.884 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-21T10:13:20.884 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.target_file_size_base: 67108864 2024-03-21T10:13:20.884 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-21T10:13:20.884 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-21T10:13:20.884 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-21T10:13:20.884 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-21T10:13:20.884 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-21T10:13:20.884 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-21T10:13:20.884 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-21T10:13:20.884 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-21T10:13:20.885 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-21T10:13:20.885 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-21T10:13:20.885 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-21T10:13:20.885 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-21T10:13:20.885 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-21T10:13:20.885 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-21T10:13:20.885 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.arena_block_size: 1048576 2024-03-21T10:13:20.885 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-21T10:13:20.885 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-21T10:13:20.885 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.disable_auto_compactions: 0 2024-03-21T10:13:20.885 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-21T10:13:20.885 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-21T10:13:20.886 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-21T10:13:20.886 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-21T10:13:20.886 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-21T10:13:20.886 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-21T10:13:20.886 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-21T10:13:20.886 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-21T10:13:20.886 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-21T10:13:20.886 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-21T10:13:20.886 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.table_properties_collectors: 2024-03-21T10:13:20.886 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.inplace_update_support: 0 2024-03-21T10:13:20.886 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-21T10:13:20.886 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-21T10:13:20.886 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-21T10:13:20.887 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-21T10:13:20.887 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.bloom_locality: 0 2024-03-21T10:13:20.887 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.max_successive_merges: 0 2024-03-21T10:13:20.887 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-21T10:13:20.887 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.paranoid_file_checks: 0 2024-03-21T10:13:20.887 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.force_consistency_checks: 1 2024-03-21T10:13:20.887 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.report_bg_io_stats: 0 2024-03-21T10:13:20.887 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.ttl: 2592000 2024-03-21T10:13:20.887 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-21T10:13:20.887 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-21T10:13:20.887 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-21T10:13:20.887 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.enable_blob_files: false 2024-03-21T10:13:20.888 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.min_blob_size: 0 2024-03-21T10:13:20.888 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.blob_file_size: 268435456 2024-03-21T10:13:20.888 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-21T10:13:20.888 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-21T10:13:20.888 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-21T10:13:20.888 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-21T10:13:20.888 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-21T10:13:20.888 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.blob_file_starting_level: 0 2024-03-21T10:13:20.888 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-21T10:13:20.888 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-03-21T10:13:20.889 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi153/store.db/MANIFEST-000015 succeeded,manifest_file_number is 15, next_file_number is 37, last_sequence is 16598, log_number is 33,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 33 2024-03-21T10:13:20.889 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 33 2024-03-21T10:13:20.889 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: cbb17578-3ee9-4747-bb14-d021fbdfda98 2024-03-21T10:13:20.889 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711016000586824, "job": 1, "event": "recovery_started", "wal_files": [33]} 2024-03-21T10:13:20.889 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #33 mode 2 2024-03-21T10:13:20.889 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711016000595594, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 38, "file_size": 579659, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 16599, "largest_seqno": 17577, "table_properties": {"data_size": 575264, "index_size": 2357, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 1029, "raw_key_size": 9321, "raw_average_key_size": 23, "raw_value_size": 566935, "raw_average_value_size": 1399, "num_data_blocks": 107, "num_entries": 405, "num_filter_entries": 405, "num_deletions": 1, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1711016000, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "cbb17578-3ee9-4747-bb14-d021fbdfda98", "db_session_id": "WVYYT207RWKGAPMT5OXJ", "orig_file_number": 38, "seqno_to_time_mapping": "N/A"}} 2024-03-21T10:13:20.889 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711016000595834, "job": 1, "event": "recovery_finished"} 2024-03-21T10:13:20.890 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: [db/version_set.cc:5047] Creating manifest 40 2024-03-21T10:13:20.890 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-03-21T10:13:20.890 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi153/store.db/000033.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-21T10:13:20.890 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55ea9ec8e000 2024-03-21T10:13:20.890 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: DB pointer 0x55ea9ec7a000 2024-03-21T10:13:20.890 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-21T10:13:20.891 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-21T10:13:20.891 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: ** DB Stats ** 2024-03-21T10:13:20.891 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-21T10:13:20.891 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-21T10:13:20.891 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-21T10:13:20.892 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-21T10:13:20.892 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-21T10:13:20.892 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-21T10:13:20.892 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-21T10:13:20.892 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: 2024-03-21T10:13:20.892 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: ** Compaction Stats [default] ** 2024-03-21T10:13:20.892 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-21T10:13:20.892 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-21T10:13:20.893 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: L0 1/0 566.07 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 110.1 0.01 0.00 1 0.005 0 0 0.0 0.0 2024-03-21T10:13:20.893 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: L6 1/0 7.99 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-03-21T10:13:20.893 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: Sum 2/0 8.54 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 110.1 0.01 0.00 1 0.005 0 0 0.0 0.0 2024-03-21T10:13:20.893 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 110.1 0.01 0.00 1 0.005 0 0 0.0 0.0 2024-03-21T10:13:20.893 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: 2024-03-21T10:13:20.893 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: ** Compaction Stats [default] ** 2024-03-21T10:13:20.893 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-21T10:13:20.893 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-21T10:13:20.893 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 110.1 0.01 0.00 1 0.005 0 0 0.0 0.0 2024-03-21T10:13:20.893 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: 2024-03-21T10:13:20.893 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-21T10:13:20.893 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: 2024-03-21T10:13:20.894 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-21T10:13:20.894 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: Flush(GB): cumulative 0.001, interval 0.001 2024-03-21T10:13:20.894 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-21T10:13:20.894 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-21T10:13:20.894 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-21T10:13:20.894 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-21T10:13:20.894 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: Cumulative compaction: 0.00 GB write, 37.11 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-21T10:13:20.894 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: Interval compaction: 0.00 GB write, 37.11 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-21T10:13:20.894 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-21T10:13:20.894 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: Block cache BinnedLRUCache@0x55ea9eb9d090#2 capacity: 512.00 MB usage: 3.73 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 2.4e-05 secs_since: 0 2024-03-21T10:13:20.894 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,1.16 KB,0.000220537%) IndexBlock(1,2.58 KB,0.000491738%) Misc(1,0.00 KB,0%) 2024-03-21T10:13:20.894 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: 2024-03-21T10:13:20.894 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-21T10:13:20.895 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: starting mon.smithi153 rank 0 at public addrs [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] at bind addrs [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi153 fsid 070ed3e4-e769-11ee-95cd-87774f69a715 2024-03-21T10:13:20.895 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: mon.smithi153@-1(???) e2 preinit fsid 070ed3e4-e769-11ee-95cd-87774f69a715 2024-03-21T10:13:20.895 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: mon.smithi153@-1(???).mds e21 new map 2024-03-21T10:13:20.895 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: mon.smithi153@-1(???).mds e21 print_map 2024-03-21T10:13:20.895 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: e21 2024-03-21T10:13:20.895 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-21T10:13:20.895 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:13:20.895 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: legacy client fscid: 1 2024-03-21T10:13:20.895 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: 2024-03-21T10:13:20.895 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: Filesystem 'cephfs' (1) 2024-03-21T10:13:20.895 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: fs_name cephfs 2024-03-21T10:13:20.895 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: epoch 21 2024-03-21T10:13:20.895 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-21T10:13:20.895 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: created 2024-03-21T10:04:27.086576+0000 2024-03-21T10:13:20.895 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: modified 2024-03-21T10:13:01.455594+0000 2024-03-21T10:13:20.896 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: tableserver 0 2024-03-21T10:13:20.896 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: root 0 2024-03-21T10:13:20.896 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: session_timeout 60 2024-03-21T10:13:20.896 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: session_autoclose 300 2024-03-21T10:13:20.896 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: max_file_size 1099511627776 2024-03-21T10:13:20.896 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: max_xattr_size 65536 2024-03-21T10:13:20.896 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: required_client_features {} 2024-03-21T10:13:20.896 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: last_failure 0 2024-03-21T10:13:20.896 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: last_failure_osd_epoch 47 2024-03-21T10:13:20.896 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:13:20.896 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: max_mds 2 2024-03-21T10:13:20.896 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: in 0,1 2024-03-21T10:13:20.896 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: up {0=14484,1=24273} 2024-03-21T10:13:20.896 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: failed 2024-03-21T10:13:20.896 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: damaged 2024-03-21T10:13:20.897 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: stopped 2024-03-21T10:13:20.897 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: data_pools [3] 2024-03-21T10:13:20.897 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: metadata_pool 2 2024-03-21T10:13:20.897 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: inline_data enabled 2024-03-21T10:13:20.897 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: balancer 2024-03-21T10:13:20.897 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: bal_rank_mask -1 2024-03-21T10:13:20.897 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: standby_count_wanted 1 2024-03-21T10:13:20.897 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: qdb_cluster leader: 0 members: 2024-03-21T10:13:20.897 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: [mds.cephfs.smithi153.iuyluj{0:14484} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.153:6828/866138999,v1:172.21.15.153:6829/866138999] compat {c=[1],r=[1],i=[7ff]}] 2024-03-21T10:13:20.897 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: [mds.cephfs.smithi153.zipckr{0:14494} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.153:6826/4033622615,v1:172.21.15.153:6827/4033622615] compat {c=[1],r=[1],i=[7ff]}] 2024-03-21T10:13:20.897 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: [mds.cephfs.smithi174.yyimst{1:24273} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.174:6824/1776947618,v1:172.21.15.174:6825/1776947618] compat {c=[1],r=[1],i=[7ff]}] 2024-03-21T10:13:20.897 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: [mds.cephfs.smithi174.nhozag{1:24285} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.174:6826/806095873,v1:172.21.15.174:6827/806095873] compat {c=[1],r=[1],i=[7ff]}] 2024-03-21T10:13:20.897 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: 2024-03-21T10:13:20.897 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout: 2024-03-21T10:13:20.897 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: mon.smithi153@-1(???).osd e49 crush map has features 3314933000852226048, adjusting msgr requires 2024-03-21T10:13:20.897 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: mon.smithi153@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-03-21T10:13:20.898 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: mon.smithi153@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-03-21T10:13:20.898 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: mon.smithi153@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-03-21T10:13:20.898 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 ceph-mon[113733]: mon.smithi153@-1(???).paxosservice(auth 1..24) refresh upgraded, format 0 -> 3 2024-03-21T10:13:20.898 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 bash[113715]: dd8038303dc317e8543d4a848b533df3ed09720587219ba01a349d97247df207 2024-03-21T10:13:20.898 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:20 smithi153 systemd[1]: Started Ceph mon.smithi153 for 070ed3e4-e769-11ee-95cd-87774f69a715. 2024-03-21T10:13:22.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:22 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mon metadata", "id": "smithi153"}]: dispatch 2024-03-21T10:13:22.453 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:22 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mon metadata", "id": "smithi174"}]: dispatch 2024-03-21T10:13:22.453 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:22 smithi174 ceph-mon[37496]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:13:22.453 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:22 smithi174 ceph-mon[37496]: pgmap v146: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:13:22.453 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:22 smithi174 ceph-mon[37496]: pgmap v147: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:13:22.453 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:22 smithi174 ceph-mon[37496]: mon.smithi153 calling monitor election 2024-03-21T10:13:22.453 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:22 smithi174 ceph-mon[37496]: mon.smithi153 is new leader, mons smithi153,smithi174 in quorum (ranks 0,1) 2024-03-21T10:13:22.453 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:22 smithi174 ceph-mon[37496]: monmap e2: 2 mons at {smithi153=[v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0],smithi174=[v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-21T10:13:22.453 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:22 smithi174 ceph-mon[37496]: fsmap cephfs:2 {0=cephfs.smithi153.iuyluj=up:active,1=cephfs.smithi174.yyimst=up:active} 2 up:standby-replay 2024-03-21T10:13:22.453 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:22 smithi174 ceph-mon[37496]: osdmap e49: 6 total, 6 up, 6 in 2024-03-21T10:13:22.453 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:22 smithi174 ceph-mon[37496]: mgrmap e31: smithi153.tluxni(active, since 4m), standbys: smithi174.qpcwat 2024-03-21T10:13:22.453 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:22 smithi174 ceph-mon[37496]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-03-21T10:13:22.453 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:22 smithi174 ceph-mon[37496]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-21T10:13:22.454 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:22 smithi174 ceph-mon[37496]: fs cephfs has deprecated feature inline_data enabled. 2024-03-21T10:13:22.454 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:22 smithi174 ceph-mon[37496]: mgrmap e32: smithi153.tluxni(active, since 4m), standbys: smithi174.qpcwat 2024-03-21T10:13:22.454 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:22 smithi174 ceph-mon[37496]: from='mgr.14666 ' entity='' 2024-03-21T10:13:22.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:22 smithi153 ceph-mon[113733]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mon metadata", "id": "smithi153"}]: dispatch 2024-03-21T10:13:22.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:22 smithi153 ceph-mon[113733]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mon metadata", "id": "smithi174"}]: dispatch 2024-03-21T10:13:22.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:22 smithi153 ceph-mon[113733]: from='mgr.14666 172.21.15.153:0/3084674997' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:13:22.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:22 smithi153 ceph-mon[113733]: pgmap v146: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:13:22.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:22 smithi153 ceph-mon[113733]: pgmap v147: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:13:22.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:22 smithi153 ceph-mon[113733]: mon.smithi153 calling monitor election 2024-03-21T10:13:22.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:22 smithi153 ceph-mon[113733]: mon.smithi153 is new leader, mons smithi153,smithi174 in quorum (ranks 0,1) 2024-03-21T10:13:22.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:22 smithi153 ceph-mon[113733]: monmap e2: 2 mons at {smithi153=[v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0],smithi174=[v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-21T10:13:22.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:22 smithi153 ceph-mon[113733]: fsmap cephfs:2 {0=cephfs.smithi153.iuyluj=up:active,1=cephfs.smithi174.yyimst=up:active} 2 up:standby-replay 2024-03-21T10:13:22.558 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:22 smithi153 ceph-mon[113733]: osdmap e49: 6 total, 6 up, 6 in 2024-03-21T10:13:22.558 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:22 smithi153 ceph-mon[113733]: mgrmap e31: smithi153.tluxni(active, since 4m), standbys: smithi174.qpcwat 2024-03-21T10:13:22.558 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:22 smithi153 ceph-mon[113733]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-03-21T10:13:22.558 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:22 smithi153 ceph-mon[113733]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-21T10:13:22.558 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:22 smithi153 ceph-mon[113733]: fs cephfs has deprecated feature inline_data enabled. 2024-03-21T10:13:22.558 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:22 smithi153 ceph-mon[113733]: mgrmap e32: smithi153.tluxni(active, since 4m), standbys: smithi174.qpcwat 2024-03-21T10:13:22.558 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:22 smithi153 ceph-mon[113733]: from='mgr.14666 ' entity='' 2024-03-21T10:13:31.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:30 smithi174 ceph-mon[37496]: Standby manager daemon smithi174.qpcwat restarted 2024-03-21T10:13:31.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:30 smithi174 ceph-mon[37496]: Standby manager daemon smithi174.qpcwat started 2024-03-21T10:13:31.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:30 smithi174 ceph-mon[37496]: from='mgr.? 172.21.15.174:0/2734583943' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi174.qpcwat/crt"}]: dispatch 2024-03-21T10:13:31.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:30 smithi174 ceph-mon[37496]: from='mgr.? 172.21.15.174:0/2734583943' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-21T10:13:31.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:30 smithi174 ceph-mon[37496]: from='mgr.? 172.21.15.174:0/2734583943' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi174.qpcwat/key"}]: dispatch 2024-03-21T10:13:31.203 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:30 smithi174 ceph-mon[37496]: from='mgr.? 172.21.15.174:0/2734583943' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-21T10:13:31.264 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:30 smithi153 ceph-mon[113733]: Standby manager daemon smithi174.qpcwat restarted 2024-03-21T10:13:31.264 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:30 smithi153 ceph-mon[113733]: Standby manager daemon smithi174.qpcwat started 2024-03-21T10:13:31.265 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:30 smithi153 ceph-mon[113733]: from='mgr.? 172.21.15.174:0/2734583943' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi174.qpcwat/crt"}]: dispatch 2024-03-21T10:13:31.265 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:30 smithi153 ceph-mon[113733]: from='mgr.? 172.21.15.174:0/2734583943' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-21T10:13:31.265 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:30 smithi153 ceph-mon[113733]: from='mgr.? 172.21.15.174:0/2734583943' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi174.qpcwat/key"}]: dispatch 2024-03-21T10:13:31.265 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:30 smithi153 ceph-mon[113733]: from='mgr.? 172.21.15.174:0/2734583943' entity='mgr.smithi174.qpcwat' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-21T10:13:32.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:31 smithi153 ceph-mon[113733]: mgrmap e33: smithi153.tluxni(active, since 4m), standbys: smithi174.qpcwat 2024-03-21T10:13:32.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:31 smithi153 ceph-mon[113733]: Active manager daemon smithi153.tluxni restarted 2024-03-21T10:13:32.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:31 smithi153 ceph-mon[113733]: Activating manager daemon smithi153.tluxni 2024-03-21T10:13:32.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:31 smithi153 ceph-mon[113733]: osdmap e50: 6 total, 6 up, 6 in 2024-03-21T10:13:32.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:31 smithi153 ceph-mon[113733]: mgrmap e34: smithi153.tluxni(active, starting, since 0.0168643s), standbys: smithi174.qpcwat 2024-03-21T10:13:32.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:31 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mon metadata", "id": "smithi153"}]: dispatch 2024-03-21T10:13:32.308 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:31 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mon metadata", "id": "smithi174"}]: dispatch 2024-03-21T10:13:32.308 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:31 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi153.iuyluj"}]: dispatch 2024-03-21T10:13:32.308 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:31 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi153.zipckr"}]: dispatch 2024-03-21T10:13:32.308 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:31 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi174.yyimst"}]: dispatch 2024-03-21T10:13:32.308 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:31 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi174.nhozag"}]: dispatch 2024-03-21T10:13:32.308 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:31 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mgr metadata", "who": "smithi153.tluxni", "id": "smithi153.tluxni"}]: dispatch 2024-03-21T10:13:32.308 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:31 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mgr metadata", "who": "smithi174.qpcwat", "id": "smithi174.qpcwat"}]: dispatch 2024-03-21T10:13:32.308 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:31 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T10:13:32.308 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:31 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T10:13:32.308 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:31 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T10:13:32.308 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:31 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T10:13:32.308 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:31 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T10:13:32.308 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:31 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T10:13:32.308 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:31 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-21T10:13:32.308 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:31 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-21T10:13:32.309 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:31 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-21T10:13:32.309 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:31 smithi153 ceph-mon[113733]: Manager daemon smithi153.tluxni is now available 2024-03-21T10:13:32.309 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:31 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:13:32.309 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:31 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:13:32.309 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:31 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi153.tluxni/mirror_snapshot_schedule"}]: dispatch 2024-03-21T10:13:32.309 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:31 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi153.tluxni/trash_purge_schedule"}]: dispatch 2024-03-21T10:13:32.453 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:31 smithi174 ceph-mon[37496]: mgrmap e33: smithi153.tluxni(active, since 4m), standbys: smithi174.qpcwat 2024-03-21T10:13:32.453 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:31 smithi174 ceph-mon[37496]: Active manager daemon smithi153.tluxni restarted 2024-03-21T10:13:32.453 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:31 smithi174 ceph-mon[37496]: Activating manager daemon smithi153.tluxni 2024-03-21T10:13:32.453 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:31 smithi174 ceph-mon[37496]: osdmap e50: 6 total, 6 up, 6 in 2024-03-21T10:13:32.453 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:31 smithi174 ceph-mon[37496]: mgrmap e34: smithi153.tluxni(active, starting, since 0.0168643s), standbys: smithi174.qpcwat 2024-03-21T10:13:32.453 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:31 smithi174 ceph-mon[37496]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mon metadata", "id": "smithi153"}]: dispatch 2024-03-21T10:13:32.453 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:31 smithi174 ceph-mon[37496]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mon metadata", "id": "smithi174"}]: dispatch 2024-03-21T10:13:32.453 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:31 smithi174 ceph-mon[37496]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi153.iuyluj"}]: dispatch 2024-03-21T10:13:32.453 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:31 smithi174 ceph-mon[37496]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi153.zipckr"}]: dispatch 2024-03-21T10:13:32.453 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:31 smithi174 ceph-mon[37496]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi174.yyimst"}]: dispatch 2024-03-21T10:13:32.453 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:31 smithi174 ceph-mon[37496]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi174.nhozag"}]: dispatch 2024-03-21T10:13:32.454 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:31 smithi174 ceph-mon[37496]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mgr metadata", "who": "smithi153.tluxni", "id": "smithi153.tluxni"}]: dispatch 2024-03-21T10:13:32.454 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:31 smithi174 ceph-mon[37496]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mgr metadata", "who": "smithi174.qpcwat", "id": "smithi174.qpcwat"}]: dispatch 2024-03-21T10:13:32.454 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:31 smithi174 ceph-mon[37496]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T10:13:32.454 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:31 smithi174 ceph-mon[37496]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T10:13:32.454 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:31 smithi174 ceph-mon[37496]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T10:13:32.454 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:31 smithi174 ceph-mon[37496]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T10:13:32.454 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:31 smithi174 ceph-mon[37496]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T10:13:32.454 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:31 smithi174 ceph-mon[37496]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T10:13:32.454 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:31 smithi174 ceph-mon[37496]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-21T10:13:32.454 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:31 smithi174 ceph-mon[37496]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-21T10:13:32.454 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:31 smithi174 ceph-mon[37496]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-21T10:13:32.454 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:31 smithi174 ceph-mon[37496]: Manager daemon smithi153.tluxni is now available 2024-03-21T10:13:32.454 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:31 smithi174 ceph-mon[37496]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:13:32.455 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:31 smithi174 ceph-mon[37496]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:13:32.455 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:31 smithi174 ceph-mon[37496]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi153.tluxni/mirror_snapshot_schedule"}]: dispatch 2024-03-21T10:13:32.455 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:31 smithi174 ceph-mon[37496]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi153.tluxni/trash_purge_schedule"}]: dispatch 2024-03-21T10:13:33.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:33 smithi174 ceph-mon[37496]: mgrmap e35: smithi153.tluxni(active, since 1.02595s), standbys: smithi174.qpcwat 2024-03-21T10:13:33.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:33 smithi174 ceph-mon[37496]: pgmap v3: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:13:33.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:33 smithi153 ceph-mon[113733]: mgrmap e35: smithi153.tluxni(active, since 1.02595s), standbys: smithi174.qpcwat 2024-03-21T10:13:33.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:33 smithi153 ceph-mon[113733]: pgmap v3: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:13:34.281 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:34 smithi153 ceph-mon[113733]: [21/Mar/2024:10:13:32] ENGINE Bus STARTING 2024-03-21T10:13:34.281 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:34 smithi153 ceph-mon[113733]: [21/Mar/2024:10:13:33] ENGINE Serving on http://172.21.15.153:8765 2024-03-21T10:13:34.281 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:34 smithi153 ceph-mon[113733]: pgmap v4: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:13:34.281 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:34 smithi153 ceph-mon[113733]: mgrmap e36: smithi153.tluxni(active, since 2s), standbys: smithi174.qpcwat 2024-03-21T10:13:34.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:34 smithi174 ceph-mon[37496]: [21/Mar/2024:10:13:32] ENGINE Bus STARTING 2024-03-21T10:13:34.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:34 smithi174 ceph-mon[37496]: [21/Mar/2024:10:13:33] ENGINE Serving on http://172.21.15.153:8765 2024-03-21T10:13:34.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:34 smithi174 ceph-mon[37496]: pgmap v4: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:13:34.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:34 smithi174 ceph-mon[37496]: mgrmap e36: smithi153.tluxni(active, since 2s), standbys: smithi174.qpcwat 2024-03-21T10:13:35.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:35 smithi174 ceph-mon[37496]: [21/Mar/2024:10:13:33] ENGINE Serving on https://172.21.15.153:7150 2024-03-21T10:13:35.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:35 smithi174 ceph-mon[37496]: [21/Mar/2024:10:13:33] ENGINE Bus STARTED 2024-03-21T10:13:35.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:35 smithi153 ceph-mon[113733]: [21/Mar/2024:10:13:33] ENGINE Serving on https://172.21.15.153:7150 2024-03-21T10:13:35.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:35 smithi153 ceph-mon[113733]: [21/Mar/2024:10:13:33] ENGINE Bus STARTED 2024-03-21T10:13:36.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:36 smithi174 ceph-mon[37496]: pgmap v5: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:13:36.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:36 smithi174 ceph-mon[37496]: mgrmap e37: smithi153.tluxni(active, since 4s), standbys: smithi174.qpcwat 2024-03-21T10:13:36.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:36 smithi174 ceph-mon[37496]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:13:36.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:36 smithi174 ceph-mon[37496]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:13:36.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:36 smithi153 ceph-mon[113733]: pgmap v5: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:13:36.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:36 smithi153 ceph-mon[113733]: mgrmap e37: smithi153.tluxni(active, since 4s), standbys: smithi174.qpcwat 2024-03-21T10:13:36.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:36 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:13:36.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:36 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:13:38.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:38 smithi174 ceph-mon[37496]: pgmap v6: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:13:38.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:38 smithi174 ceph-mon[37496]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:13:38.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:38 smithi174 ceph-mon[37496]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:13:38.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:38 smithi153 ceph-mon[113733]: pgmap v6: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:13:38.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:38 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:13:38.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:38 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:13:40.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:39 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:13:40.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:39 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:13:40.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:39 smithi153 ceph-mon[113733]: pgmap v7: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 31 KiB/s rd, 0 B/s wr, 19 op/s 2024-03-21T10:13:40.122 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:39 smithi174 ceph-mon[37496]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:13:40.122 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:39 smithi174 ceph-mon[37496]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:13:40.122 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:39 smithi174 ceph-mon[37496]: pgmap v7: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 31 KiB/s rd, 0 B/s wr, 19 op/s 2024-03-21T10:13:42.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:41 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:13:42.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:41 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:13:42.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:41 smithi153 ceph-mon[113733]: pgmap v8: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 24 KiB/s rd, 0 B/s wr, 15 op/s 2024-03-21T10:13:42.137 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:41 smithi174 ceph-mon[37496]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:13:42.138 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:41 smithi174 ceph-mon[37496]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:13:42.138 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:41 smithi174 ceph-mon[37496]: pgmap v8: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 24 KiB/s rd, 0 B/s wr, 15 op/s 2024-03-21T10:13:44.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:44 smithi174 ceph-mon[37496]: pgmap v9: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 21 KiB/s rd, 0 B/s wr, 14 op/s 2024-03-21T10:13:44.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:44 smithi174 ceph-mon[37496]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:13:44.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:44 smithi174 ceph-mon[37496]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:13:44.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:44 smithi153 ceph-mon[113733]: pgmap v9: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 21 KiB/s rd, 0 B/s wr, 14 op/s 2024-03-21T10:13:44.558 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:44 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:13:44.558 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:44 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:13:45.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:45 smithi174 ceph-mon[37496]: Detected new or changed devices on smithi174 2024-03-21T10:13:45.476 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:45 smithi153 ceph-mon[113733]: Detected new or changed devices on smithi174 2024-03-21T10:13:45.689 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:45.687+0000 7fd175e00700 1 -- 172.21.15.153:0/57277130 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd1701017c0 msgr2=0x7fd170105810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:13:45.690 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:45.687+0000 7fd175e00700 1 --2- 172.21.15.153:0/57277130 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd1701017c0 0x7fd170105810 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd15c009910 tx=0x7fd15c009c20 comp rx=0 tx=0).stop 2024-03-21T10:13:45.690 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:45.687+0000 7fd175e00700 1 -- 172.21.15.153:0/57277130 shutdown_connections 2024-03-21T10:13:45.690 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:45.687+0000 7fd175e00700 1 --2- 172.21.15.153:0/57277130 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd1701017c0 0x7fd170105810 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:45.690 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:45.687+0000 7fd175e00700 1 --2- 172.21.15.153:0/57277130 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd170100e10 0x7fd1701011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:45.690 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:45.687+0000 7fd175e00700 1 -- 172.21.15.153:0/57277130 >> 172.21.15.153:0/57277130 conn(0x7fd1700fc6c0 msgr2=0x7fd1700feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:13:45.690 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:45.688+0000 7fd175e00700 1 -- 172.21.15.153:0/57277130 shutdown_connections 2024-03-21T10:13:45.690 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:45.688+0000 7fd175e00700 1 -- 172.21.15.153:0/57277130 wait complete. 2024-03-21T10:13:45.691 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:45.689+0000 7fd175e00700 1 Processor -- start 2024-03-21T10:13:45.691 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:45.689+0000 7fd175e00700 1 -- start start 2024-03-21T10:13:45.691 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:45.689+0000 7fd175e00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd170100e10 0x7fd17019e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:13:45.691 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:45.689+0000 7fd175e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd1701017c0 0x7fd17019ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:13:45.691 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:45.689+0000 7fd175e00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd17019f3c0 con 0x7fd1701017c0 2024-03-21T10:13:45.691 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:45.689+0000 7fd175e00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd170198870 con 0x7fd170100e10 2024-03-21T10:13:45.692 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:45.690+0000 7fd16e000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd1701017c0 0x7fd17019ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:13:45.692 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:45.690+0000 7fd16e000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd1701017c0 0x7fd17019ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:40274/0 (socket says 172.21.15.153:40274) 2024-03-21T10:13:45.692 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:45.690+0000 7fd16e000700 1 -- 172.21.15.153:0/1100481189 learned_addr learned my addr 172.21.15.153:0/1100481189 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:13:45.692 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:45.690+0000 7fd16ea00700 1 --2- 172.21.15.153:0/1100481189 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd170100e10 0x7fd17019e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:13:45.692 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:45.690+0000 7fd16e000700 1 -- 172.21.15.153:0/1100481189 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd170100e10 msgr2=0x7fd17019e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:13:45.692 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:45.690+0000 7fd16e000700 1 --2- 172.21.15.153:0/1100481189 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd170100e10 0x7fd17019e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:45.692 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:45.690+0000 7fd16e000700 1 -- 172.21.15.153:0/1100481189 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd15c0095f0 con 0x7fd1701017c0 2024-03-21T10:13:45.693 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:45.691+0000 7fd16e000700 1 --2- 172.21.15.153:0/1100481189 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd1701017c0 0x7fd17019ed30 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd15c00bda0 tx=0x7fd15c00be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:13:45.693 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:45.691+0000 7fd174a00700 1 -- 172.21.15.153:0/1100481189 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd15c01d070 con 0x7fd1701017c0 2024-03-21T10:13:45.693 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:45.691+0000 7fd175e00700 1 -- 172.21.15.153:0/1100481189 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd170198af0 con 0x7fd1701017c0 2024-03-21T10:13:45.693 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:45.691+0000 7fd174a00700 1 -- 172.21.15.153:0/1100481189 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd15c00f460 con 0x7fd1701017c0 2024-03-21T10:13:45.693 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:45.691+0000 7fd174a00700 1 -- 172.21.15.153:0/1100481189 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd15c017610 con 0x7fd1701017c0 2024-03-21T10:13:45.693 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:45.691+0000 7fd175e00700 1 -- 172.21.15.153:0/1100481189 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd170198fe0 con 0x7fd1701017c0 2024-03-21T10:13:45.694 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:45.692+0000 7fd175e00700 1 -- 172.21.15.153:0/1100481189 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd170104f90 con 0x7fd1701017c0 2024-03-21T10:13:45.695 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:45.693+0000 7fd174a00700 1 -- 172.21.15.153:0/1100481189 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 37) v1 ==== 92321+0+0 (secure 0 0 0) 0x7fd15c00fa40 con 0x7fd1701017c0 2024-03-21T10:13:45.696 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:45.694+0000 7fd174a00700 1 --2- 172.21.15.153:0/1100481189 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd15406e590 0x7fd154070a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:13:45.696 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:45.694+0000 7fd174a00700 1 -- 172.21.15.153:0/1100481189 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fd15c08faf0 con 0x7fd1701017c0 2024-03-21T10:13:45.696 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:45.695+0000 7fd16ea00700 1 --2- 172.21.15.153:0/1100481189 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd15406e590 0x7fd154070a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:13:45.697 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:45.695+0000 7fd16ea00700 1 --2- 172.21.15.153:0/1100481189 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd15406e590 0x7fd154070a50 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd160005950 tx=0x7fd1600058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:13:45.702 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:45.700+0000 7fd174a00700 1 -- 172.21.15.153:0/1100481189 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd15c05c5c0 con 0x7fd1701017c0 2024-03-21T10:13:45.958 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:45.956+0000 7fd175e00700 1 -- 172.21.15.153:0/1100481189 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd1701992c0 con 0x7fd15406e590 2024-03-21T10:13:45.960 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:45.959+0000 7fd174a00700 1 -- 172.21.15.153:0/1100481189 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7fd1701992c0 con 0x7fd15406e590 2024-03-21T10:13:45.966 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:45.961+0000 7fd175e00700 1 -- 172.21.15.153:0/1100481189 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd15406e590 msgr2=0x7fd154070a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:13:45.966 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:45.961+0000 7fd175e00700 1 --2- 172.21.15.153:0/1100481189 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd15406e590 0x7fd154070a50 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd160005950 tx=0x7fd1600058e0 comp rx=0 tx=0).stop 2024-03-21T10:13:45.966 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:45.961+0000 7fd175e00700 1 -- 172.21.15.153:0/1100481189 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd1701017c0 msgr2=0x7fd17019ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:13:45.966 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:45.961+0000 7fd175e00700 1 --2- 172.21.15.153:0/1100481189 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd1701017c0 0x7fd17019ed30 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd15c00bda0 tx=0x7fd15c00be80 comp rx=0 tx=0).stop 2024-03-21T10:13:45.966 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:45.962+0000 7fd175e00700 1 -- 172.21.15.153:0/1100481189 shutdown_connections 2024-03-21T10:13:45.966 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:45.962+0000 7fd175e00700 1 --2- 172.21.15.153:0/1100481189 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd170100e10 0x7fd17019e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:45.966 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:45.962+0000 7fd175e00700 1 --2- 172.21.15.153:0/1100481189 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd15406e590 0x7fd154070a50 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:45.966 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:45.962+0000 7fd175e00700 1 --2- 172.21.15.153:0/1100481189 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd1701017c0 0x7fd17019ed30 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:45.966 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:45.962+0000 7fd175e00700 1 -- 172.21.15.153:0/1100481189 >> 172.21.15.153:0/1100481189 conn(0x7fd1700fc6c0 msgr2=0x7fd1700fdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:13:45.967 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:45.962+0000 7fd175e00700 1 -- 172.21.15.153:0/1100481189 shutdown_connections 2024-03-21T10:13:45.967 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:45.962+0000 7fd175e00700 1 -- 172.21.15.153:0/1100481189 wait complete. 2024-03-21T10:13:45.979 INFO:teuthology.orchestra.run.smithi153.stdout:true 2024-03-21T10:13:46.115 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.114+0000 7fb2e0400700 1 -- 172.21.15.153:0/2347474517 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb2d8101750 msgr2=0x7fb2d81057a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:13:46.116 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.114+0000 7fb2e0400700 1 --2- 172.21.15.153:0/2347474517 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb2d8101750 0x7fb2d81057a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fb2c4009910 tx=0x7fb2c4009c20 comp rx=0 tx=0).stop 2024-03-21T10:13:46.116 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.114+0000 7fb2e0400700 1 -- 172.21.15.153:0/2347474517 shutdown_connections 2024-03-21T10:13:46.116 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.114+0000 7fb2e0400700 1 --2- 172.21.15.153:0/2347474517 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb2d8101750 0x7fb2d81057a0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:46.116 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.114+0000 7fb2e0400700 1 --2- 172.21.15.153:0/2347474517 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb2d8100da0 0x7fb2d8101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:46.116 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.114+0000 7fb2e0400700 1 -- 172.21.15.153:0/2347474517 >> 172.21.15.153:0/2347474517 conn(0x7fb2d80fc630 msgr2=0x7fb2d80fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:13:46.116 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.114+0000 7fb2e0400700 1 -- 172.21.15.153:0/2347474517 shutdown_connections 2024-03-21T10:13:46.116 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.114+0000 7fb2e0400700 1 -- 172.21.15.153:0/2347474517 wait complete. 2024-03-21T10:13:46.116 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.115+0000 7fb2e0400700 1 Processor -- start 2024-03-21T10:13:46.116 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.115+0000 7fb2e0400700 1 -- start start 2024-03-21T10:13:46.117 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.115+0000 7fb2e0400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb2d8100da0 0x7fb2d819c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:13:46.117 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.115+0000 7fb2e0400700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb2d8101750 0x7fb2d819ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:13:46.117 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.115+0000 7fb2e0400700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d819d040 con 0x7fb2d8100da0 2024-03-21T10:13:46.117 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.115+0000 7fb2e0400700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d8196580 con 0x7fb2d8101750 2024-03-21T10:13:46.117 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.116+0000 7fb2dda00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb2d8100da0 0x7fb2d819c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:13:46.117 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.116+0000 7fb2dda00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb2d8100da0 0x7fb2d819c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:40290/0 (socket says 172.21.15.153:40290) 2024-03-21T10:13:46.117 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.116+0000 7fb2dda00700 1 -- 172.21.15.153:0/919842783 learned_addr learned my addr 172.21.15.153:0/919842783 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:13:46.118 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.116+0000 7fb2dd000700 1 --2- 172.21.15.153:0/919842783 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb2d8101750 0x7fb2d819ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:13:46.118 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.116+0000 7fb2dda00700 1 -- 172.21.15.153:0/919842783 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb2d8101750 msgr2=0x7fb2d819ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:13:46.118 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.116+0000 7fb2dda00700 1 --2- 172.21.15.153:0/919842783 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb2d8101750 0x7fb2d819ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:46.119 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.116+0000 7fb2dda00700 1 -- 172.21.15.153:0/919842783 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2c40095f0 con 0x7fb2d8100da0 2024-03-21T10:13:46.119 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.116+0000 7fb2dda00700 1 --2- 172.21.15.153:0/919842783 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb2d8100da0 0x7fb2d819c500 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc00c930 tx=0x7fb2cc00ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:13:46.119 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.117+0000 7fb2d6000700 1 -- 172.21.15.153:0/919842783 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb2cc007ad0 con 0x7fb2d8100da0 2024-03-21T10:13:46.119 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.117+0000 7fb2d6000700 1 -- 172.21.15.153:0/919842783 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb2cc007c30 con 0x7fb2d8100da0 2024-03-21T10:13:46.119 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.117+0000 7fb2d6000700 1 -- 172.21.15.153:0/919842783 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb2cc018610 con 0x7fb2d8100da0 2024-03-21T10:13:46.119 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.117+0000 7fb2e0400700 1 -- 172.21.15.153:0/919842783 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2d8196860 con 0x7fb2d8100da0 2024-03-21T10:13:46.119 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.117+0000 7fb2e0400700 1 -- 172.21.15.153:0/919842783 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2d8196d10 con 0x7fb2d8100da0 2024-03-21T10:13:46.119 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.118+0000 7fb2e0400700 1 -- 172.21.15.153:0/919842783 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2d81091a0 con 0x7fb2d8100da0 2024-03-21T10:13:46.121 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.119+0000 7fb2d6000700 1 -- 172.21.15.153:0/919842783 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 37) v1 ==== 92321+0+0 (secure 0 0 0) 0x7fb2cc00f450 con 0x7fb2d8100da0 2024-03-21T10:13:46.122 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.120+0000 7fb2d6000700 1 --2- 172.21.15.153:0/919842783 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb2c006e590 0x7fb2c0070a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:13:46.122 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.120+0000 7fb2d6000700 1 -- 172.21.15.153:0/919842783 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fb2cc08e800 con 0x7fb2d8100da0 2024-03-21T10:13:46.122 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.121+0000 7fb2dd000700 1 --2- 172.21.15.153:0/919842783 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb2c006e590 0x7fb2c0070a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:13:46.123 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.121+0000 7fb2dd000700 1 --2- 172.21.15.153:0/919842783 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb2c006e590 0x7fb2c0070a50 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb2c4006010 tx=0x7fb2c400b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:13:46.127 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.125+0000 7fb2d6000700 1 -- 172.21.15.153:0/919842783 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb2cc05b370 con 0x7fb2d8100da0 2024-03-21T10:13:46.374 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.372+0000 7fb2e0400700 1 -- 172.21.15.153:0/919842783 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb2d8197010 con 0x7fb2c006e590 2024-03-21T10:13:46.377 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.376+0000 7fb2d6000700 1 -- 172.21.15.153:0/919842783 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7fb2d8197010 con 0x7fb2c006e590 2024-03-21T10:13:46.380 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.378+0000 7fb2e0400700 1 -- 172.21.15.153:0/919842783 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb2c006e590 msgr2=0x7fb2c0070a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:13:46.380 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.378+0000 7fb2e0400700 1 --2- 172.21.15.153:0/919842783 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb2c006e590 0x7fb2c0070a50 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb2c4006010 tx=0x7fb2c400b540 comp rx=0 tx=0).stop 2024-03-21T10:13:46.380 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.378+0000 7fb2e0400700 1 -- 172.21.15.153:0/919842783 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb2d8100da0 msgr2=0x7fb2d819c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:13:46.380 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.378+0000 7fb2e0400700 1 --2- 172.21.15.153:0/919842783 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb2d8100da0 0x7fb2d819c500 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc00c930 tx=0x7fb2cc00ccf0 comp rx=0 tx=0).stop 2024-03-21T10:13:46.381 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.379+0000 7fb2e0400700 1 -- 172.21.15.153:0/919842783 shutdown_connections 2024-03-21T10:13:46.381 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.379+0000 7fb2e0400700 1 --2- 172.21.15.153:0/919842783 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb2c006e590 0x7fb2c0070a50 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:46.381 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.379+0000 7fb2e0400700 1 --2- 172.21.15.153:0/919842783 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb2d8100da0 0x7fb2d819c500 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:46.381 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.379+0000 7fb2e0400700 1 --2- 172.21.15.153:0/919842783 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb2d8101750 0x7fb2d819ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:46.381 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.379+0000 7fb2e0400700 1 -- 172.21.15.153:0/919842783 >> 172.21.15.153:0/919842783 conn(0x7fb2d80fc630 msgr2=0x7fb2d80fea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:13:46.381 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.379+0000 7fb2e0400700 1 -- 172.21.15.153:0/919842783 shutdown_connections 2024-03-21T10:13:46.381 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.379+0000 7fb2e0400700 1 -- 172.21.15.153:0/919842783 wait complete. 2024-03-21T10:13:46.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:46 smithi174 ceph-mon[37496]: pgmap v10: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 19 KiB/s rd, 0 B/s wr, 12 op/s 2024-03-21T10:13:46.477 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:46 smithi153 ceph-mon[113733]: pgmap v10: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 19 KiB/s rd, 0 B/s wr, 12 op/s 2024-03-21T10:13:46.543 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.541+0000 7f95a1e00700 1 -- 172.21.15.153:0/2425984823 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f959c101750 msgr2=0x7f959c1057a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:13:46.543 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.541+0000 7f95a1e00700 1 --2- 172.21.15.153:0/2425984823 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f959c101750 0x7f959c1057a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f9588009910 tx=0x7f9588009c20 comp rx=0 tx=0).stop 2024-03-21T10:13:46.543 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.541+0000 7f95a1e00700 1 -- 172.21.15.153:0/2425984823 shutdown_connections 2024-03-21T10:13:46.543 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.541+0000 7f95a1e00700 1 --2- 172.21.15.153:0/2425984823 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f959c101750 0x7f959c1057a0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:46.543 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.541+0000 7f95a1e00700 1 --2- 172.21.15.153:0/2425984823 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f959c100da0 0x7f959c101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:46.543 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.541+0000 7f95a1e00700 1 -- 172.21.15.153:0/2425984823 >> 172.21.15.153:0/2425984823 conn(0x7f959c0fc630 msgr2=0x7f959c0fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:13:46.544 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.541+0000 7f95a1e00700 1 -- 172.21.15.153:0/2425984823 shutdown_connections 2024-03-21T10:13:46.544 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.542+0000 7f95a1e00700 1 -- 172.21.15.153:0/2425984823 wait complete. 2024-03-21T10:13:46.544 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.542+0000 7f95a1e00700 1 Processor -- start 2024-03-21T10:13:46.544 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.543+0000 7f95a1e00700 1 -- start start 2024-03-21T10:13:46.545 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.543+0000 7f95a1e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f959c100da0 0x7f959c198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:13:46.545 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.543+0000 7f95a1e00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f959c101750 0x7f959c198e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:13:46.545 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.543+0000 7f95a1e00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f959c199520 con 0x7f959c100da0 2024-03-21T10:13:46.545 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.543+0000 7f95a1e00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f959c19d330 con 0x7f959c101750 2024-03-21T10:13:46.545 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.543+0000 7f959aa00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f959c100da0 0x7f959c198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:13:46.546 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.544+0000 7f959aa00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f959c100da0 0x7f959c198950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:40310/0 (socket says 172.21.15.153:40310) 2024-03-21T10:13:46.546 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.544+0000 7f959aa00700 1 -- 172.21.15.153:0/2336517735 learned_addr learned my addr 172.21.15.153:0/2336517735 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:13:46.546 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.544+0000 7f959a000700 1 --2- 172.21.15.153:0/2336517735 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f959c101750 0x7f959c198e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:13:46.546 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.544+0000 7f959aa00700 1 -- 172.21.15.153:0/2336517735 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f959c101750 msgr2=0x7f959c198e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:13:46.546 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.544+0000 7f959aa00700 1 --2- 172.21.15.153:0/2336517735 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f959c101750 0x7f959c198e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:46.546 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.544+0000 7f959aa00700 1 -- 172.21.15.153:0/2336517735 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95880095f0 con 0x7f959c100da0 2024-03-21T10:13:46.546 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.544+0000 7f959aa00700 1 --2- 172.21.15.153:0/2336517735 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f959c100da0 0x7f959c198950 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f958c00e920 tx=0x7f958c00ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:13:46.547 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.544+0000 7f95a0a00700 1 -- 172.21.15.153:0/2336517735 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f958c00c9d0 con 0x7f959c100da0 2024-03-21T10:13:46.547 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.545+0000 7f95a0a00700 1 -- 172.21.15.153:0/2336517735 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f958c010430 con 0x7f959c100da0 2024-03-21T10:13:46.547 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.545+0000 7f95a0a00700 1 -- 172.21.15.153:0/2336517735 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f958c018630 con 0x7f959c100da0 2024-03-21T10:13:46.547 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.545+0000 7f95a1e00700 1 -- 172.21.15.153:0/2336517735 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f959c19d610 con 0x7f959c100da0 2024-03-21T10:13:46.547 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.545+0000 7f95a1e00700 1 -- 172.21.15.153:0/2336517735 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f959c19db30 con 0x7f959c100da0 2024-03-21T10:13:46.547 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.546+0000 7f95a1e00700 1 -- 172.21.15.153:0/2336517735 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f959c104f20 con 0x7f959c100da0 2024-03-21T10:13:46.548 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.547+0000 7f95a0a00700 1 -- 172.21.15.153:0/2336517735 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 37) v1 ==== 92321+0+0 (secure 0 0 0) 0x7f958c0105a0 con 0x7f959c100da0 2024-03-21T10:13:46.549 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.548+0000 7f95a0a00700 1 --2- 172.21.15.153:0/2336517735 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f958006e590 0x7f9580070a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:13:46.549 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.548+0000 7f95a0a00700 1 -- 172.21.15.153:0/2336517735 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f958c014070 con 0x7f959c100da0 2024-03-21T10:13:46.550 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.548+0000 7f959a000700 1 --2- 172.21.15.153:0/2336517735 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f958006e590 0x7f9580070a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:13:46.550 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.549+0000 7f959a000700 1 --2- 172.21.15.153:0/2336517735 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f958006e590 0x7f9580070a50 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f9588005b40 tx=0x7f9588005a90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:13:46.554 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.553+0000 7f95a0a00700 1 -- 172.21.15.153:0/2336517735 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f958c053740 con 0x7f959c100da0 2024-03-21T10:13:46.786 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.784+0000 7f95a1e00700 1 -- 172.21.15.153:0/2336517735 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f959c19dd70 con 0x7f958006e590 2024-03-21T10:13:46.800 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.799+0000 7f95a0a00700 1 -- 172.21.15.153:0/2336517735 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f959c19dd70 con 0x7f958006e590 2024-03-21T10:13:46.800 INFO:teuthology.orchestra.run.smithi153.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-21T10:13:46.800 INFO:teuthology.orchestra.run.smithi153.stdout:alertmanager.smithi153 smithi153 *:9093,9094 running (2m) 8s ago 16m 14.5M - 0.25.0 c8568f914cd2 9ed3f2200b5f 2024-03-21T10:13:46.801 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi153 smithi153 running (16m) 8s ago 16m 8334k - 18.2.1 5be31c24972a 5b61994396a6 2024-03-21T10:13:46.801 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi174 smithi174 running (15m) 10s ago 15m 8232k - 18.2.1 5be31c24972a 4f7a5fc3ac28 2024-03-21T10:13:46.801 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi153 smithi153 running (16m) 8s ago 16m 6665k - 18.2.1 5be31c24972a 90a8b8ea03d1 2024-03-21T10:13:46.801 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi174 smithi174 running (15m) 10s ago 15m 6665k - 18.2.1 5be31c24972a a4c929e4dc77 2024-03-21T10:13:46.801 INFO:teuthology.orchestra.run.smithi153.stdout:grafana.smithi153 smithi153 *:3000 running (2m) 8s ago 15m 75.6M - 9.4.12 2bacad6d85d8 ee24026ab786 2024-03-21T10:13:46.801 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.iuyluj smithi153 running (9m) 8s ago 9m 128M - 18.2.1 5be31c24972a 02f667cee979 2024-03-21T10:13:46.801 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.zipckr smithi153 running (9m) 8s ago 9m 58.8M - 18.2.1 5be31c24972a 68a853929f8f 2024-03-21T10:13:46.801 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.nhozag smithi174 running (9m) 10s ago 9m 57.2M - 18.2.1 5be31c24972a 2fee4000308f 2024-03-21T10:13:46.801 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.yyimst smithi174 running (9m) 10s ago 9m 118M - 18.2.1 5be31c24972a d78a729220d1 2024-03-21T10:13:46.801 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi153.tluxni smithi153 *:8443,9283,8765 running (5m) 8s ago 18m 475M - 19.0.0-2231-g2edfee68 237e43c7a65f 7ffee0e55382 2024-03-21T10:13:46.801 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi174.qpcwat smithi174 *:8443,9283,8765 running (4m) 10s ago 14m 447M - 19.0.0-2231-g2edfee68 237e43c7a65f 748968f216ab 2024-03-21T10:13:46.801 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi153 smithi153 running (26s) 8s ago 18m 29.0M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f dd8038303dc3 2024-03-21T10:13:46.801 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi174 smithi174 running (14m) 10s ago 14m 48.5M 2048M 18.2.1 5be31c24972a 99204923eec6 2024-03-21T10:13:46.801 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi153 smithi153 *:9100 running (3m) 8s ago 16m 7814k - 1.5.0 0da6a335fe13 bb835edc0e0b 2024-03-21T10:13:46.801 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi174 smithi174 *:9100 running (3m) 10s ago 14m 7750k - 1.5.0 0da6a335fe13 7c4ecad790fa 2024-03-21T10:13:46.802 INFO:teuthology.orchestra.run.smithi153.stdout:osd.0 smithi153 running (13m) 8s ago 13m 216M 1217M 18.2.1 5be31c24972a 28912a2319b4 2024-03-21T10:13:46.802 INFO:teuthology.orchestra.run.smithi153.stdout:osd.1 smithi153 running (13m) 8s ago 13m 215M 1217M 18.2.1 5be31c24972a ca60681e2314 2024-03-21T10:13:46.802 INFO:teuthology.orchestra.run.smithi153.stdout:osd.2 smithi153 running (12m) 8s ago 12m 173M 1217M 18.2.1 5be31c24972a 8db799f8962a 2024-03-21T10:13:46.802 INFO:teuthology.orchestra.run.smithi153.stdout:osd.3 smithi174 running (12m) 10s ago 12m 233M 2241M 18.2.1 5be31c24972a dafc89e0658d 2024-03-21T10:13:46.802 INFO:teuthology.orchestra.run.smithi153.stdout:osd.4 smithi174 running (11m) 10s ago 11m 224M 2241M 18.2.1 5be31c24972a c4cd15701f05 2024-03-21T10:13:46.802 INFO:teuthology.orchestra.run.smithi153.stdout:osd.5 smithi174 running (10m) 10s ago 10m 202M 2241M 18.2.1 5be31c24972a 3b69886ce22e 2024-03-21T10:13:46.802 INFO:teuthology.orchestra.run.smithi153.stdout:prometheus.smithi153 smithi153 *:9095 running (3m) 8s ago 15m 67.9M - 2.43.0 a07b618ecd1d 7d95561e50c2 2024-03-21T10:13:46.805 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.803+0000 7f95a1e00700 1 -- 172.21.15.153:0/2336517735 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f958006e590 msgr2=0x7f9580070a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:13:46.805 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.803+0000 7f95a1e00700 1 --2- 172.21.15.153:0/2336517735 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f958006e590 0x7f9580070a50 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f9588005b40 tx=0x7f9588005a90 comp rx=0 tx=0).stop 2024-03-21T10:13:46.805 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.803+0000 7f95a1e00700 1 -- 172.21.15.153:0/2336517735 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f959c100da0 msgr2=0x7f959c198950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:13:46.805 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.803+0000 7f95a1e00700 1 --2- 172.21.15.153:0/2336517735 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f959c100da0 0x7f959c198950 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f958c00e920 tx=0x7f958c00ec30 comp rx=0 tx=0).stop 2024-03-21T10:13:46.805 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.803+0000 7f95a1e00700 1 -- 172.21.15.153:0/2336517735 shutdown_connections 2024-03-21T10:13:46.805 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.803+0000 7f95a1e00700 1 --2- 172.21.15.153:0/2336517735 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f958006e590 0x7f9580070a50 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:46.806 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.803+0000 7f95a1e00700 1 --2- 172.21.15.153:0/2336517735 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f959c100da0 0x7f959c198950 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:46.806 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.803+0000 7f95a1e00700 1 --2- 172.21.15.153:0/2336517735 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f959c101750 0x7f959c198e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:46.806 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.803+0000 7f95a1e00700 1 -- 172.21.15.153:0/2336517735 >> 172.21.15.153:0/2336517735 conn(0x7f959c0fc630 msgr2=0x7f959c0fe9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:13:46.806 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.804+0000 7f95a1e00700 1 -- 172.21.15.153:0/2336517735 shutdown_connections 2024-03-21T10:13:46.806 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.804+0000 7f95a1e00700 1 -- 172.21.15.153:0/2336517735 wait complete. 2024-03-21T10:13:46.952 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.950+0000 7ff3cbe00700 1 -- 172.21.15.153:0/1021869694 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff3cc1037c0 msgr2=0x7ff3cc107810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:13:46.953 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.950+0000 7ff3cbe00700 1 --2- 172.21.15.153:0/1021869694 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff3cc1037c0 0x7ff3cc107810 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff3b4009910 tx=0x7ff3b4009c20 comp rx=0 tx=0).stop 2024-03-21T10:13:46.953 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.951+0000 7ff3cbe00700 1 -- 172.21.15.153:0/1021869694 shutdown_connections 2024-03-21T10:13:46.953 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.951+0000 7ff3cbe00700 1 --2- 172.21.15.153:0/1021869694 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff3cc1037c0 0x7ff3cc107810 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:46.953 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.951+0000 7ff3cbe00700 1 --2- 172.21.15.153:0/1021869694 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff3cc102e10 0x7ff3cc1031f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:46.953 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.951+0000 7ff3cbe00700 1 -- 172.21.15.153:0/1021869694 >> 172.21.15.153:0/1021869694 conn(0x7ff3cc0fe6c0 msgr2=0x7ff3cc100ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:13:46.953 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.951+0000 7ff3cbe00700 1 -- 172.21.15.153:0/1021869694 shutdown_connections 2024-03-21T10:13:46.953 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.951+0000 7ff3cbe00700 1 -- 172.21.15.153:0/1021869694 wait complete. 2024-03-21T10:13:46.954 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.952+0000 7ff3cbe00700 1 Processor -- start 2024-03-21T10:13:46.954 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.952+0000 7ff3cbe00700 1 -- start start 2024-03-21T10:13:46.954 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.952+0000 7ff3cbe00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff3cc102e10 0x7ff3cc19e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:13:46.954 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.952+0000 7ff3cbe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff3cc1037c0 0x7ff3cc19ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:13:46.954 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.952+0000 7ff3cbe00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3cc19f120 con 0x7ff3cc1037c0 2024-03-21T10:13:46.954 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.952+0000 7ff3cbe00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3cc1985d0 con 0x7ff3cc102e10 2024-03-21T10:13:46.955 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.953+0000 7ff3c3e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff3cc1037c0 0x7ff3cc19ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:13:46.955 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.953+0000 7ff3c3e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff3cc1037c0 0x7ff3cc19ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:40328/0 (socket says 172.21.15.153:40328) 2024-03-21T10:13:46.955 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.953+0000 7ff3caa00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff3cc102e10 0x7ff3cc19e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:13:46.955 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.953+0000 7ff3c3e00700 1 -- 172.21.15.153:0/4005055651 learned_addr learned my addr 172.21.15.153:0/4005055651 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:13:46.956 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.953+0000 7ff3c3e00700 1 -- 172.21.15.153:0/4005055651 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff3cc102e10 msgr2=0x7ff3cc19e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:13:46.956 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.953+0000 7ff3c3e00700 1 --2- 172.21.15.153:0/4005055651 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff3cc102e10 0x7ff3cc19e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:46.956 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.953+0000 7ff3c3e00700 1 -- 172.21.15.153:0/4005055651 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3b40095f0 con 0x7ff3cc1037c0 2024-03-21T10:13:46.956 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.954+0000 7ff3c3e00700 1 --2- 172.21.15.153:0/4005055651 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff3cc1037c0 0x7ff3cc19ea90 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff3b400bd70 tx=0x7ff3b400bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:13:46.956 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.954+0000 7ff3c3400700 1 -- 172.21.15.153:0/4005055651 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff3b401d070 con 0x7ff3cc1037c0 2024-03-21T10:13:46.956 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.954+0000 7ff3c3400700 1 -- 172.21.15.153:0/4005055651 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff3b4005dc0 con 0x7ff3cc1037c0 2024-03-21T10:13:46.956 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.954+0000 7ff3c3400700 1 -- 172.21.15.153:0/4005055651 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff3b4017400 con 0x7ff3cc1037c0 2024-03-21T10:13:46.956 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.954+0000 7ff3cbe00700 1 -- 172.21.15.153:0/4005055651 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3cc198850 con 0x7ff3cc1037c0 2024-03-21T10:13:46.956 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.954+0000 7ff3cbe00700 1 -- 172.21.15.153:0/4005055651 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3cc198d40 con 0x7ff3cc1037c0 2024-03-21T10:13:46.957 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.955+0000 7ff3cbe00700 1 -- 172.21.15.153:0/4005055651 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3cc10b210 con 0x7ff3cc1037c0 2024-03-21T10:13:46.958 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.956+0000 7ff3c3400700 1 -- 172.21.15.153:0/4005055651 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 37) v1 ==== 92321+0+0 (secure 0 0 0) 0x7ff3b400f460 con 0x7ff3cc1037c0 2024-03-21T10:13:46.959 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.957+0000 7ff3c3400700 1 --2- 172.21.15.153:0/4005055651 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff3b006e590 0x7ff3b0070a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:13:46.959 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.957+0000 7ff3c3400700 1 -- 172.21.15.153:0/4005055651 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7ff3b408f9e0 con 0x7ff3cc1037c0 2024-03-21T10:13:46.959 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.957+0000 7ff3caa00700 1 --2- 172.21.15.153:0/4005055651 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff3b006e590 0x7ff3b0070a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:13:46.959 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.958+0000 7ff3caa00700 1 --2- 172.21.15.153:0/4005055651 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff3b006e590 0x7ff3b0070a50 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc005950 tx=0x7ff3bc00b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:13:46.964 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:46.963+0000 7ff3c3400700 1 -- 172.21.15.153:0/4005055651 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff3b405c530 con 0x7ff3cc1037c0 2024-03-21T10:13:47.280 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:47.279+0000 7ff3cbe00700 1 -- 172.21.15.153:0/4005055651 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff3cc02cca0 con 0x7ff3cc1037c0 2024-03-21T10:13:47.281 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:47.279+0000 7ff3c3400700 1 -- 172.21.15.153:0/4005055651 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+779 (secure 0 0 0) 0x7ff3b405c0c0 con 0x7ff3cc1037c0 2024-03-21T10:13:47.281 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:13:47.281 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": { 2024-03-21T10:13:47.281 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-03-21T10:13:47.281 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 1 2024-03-21T10:13:47.281 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:13:47.281 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": { 2024-03-21T10:13:47.282 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:13:47.282 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:13:47.282 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": { 2024-03-21T10:13:47.282 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-03-21T10:13:47.282 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:13:47.282 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": { 2024-03-21T10:13:47.282 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-03-21T10:13:47.282 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:13:47.282 INFO:teuthology.orchestra.run.smithi153.stdout: "overall": { 2024-03-21T10:13:47.282 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 11, 2024-03-21T10:13:47.282 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 3 2024-03-21T10:13:47.282 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-03-21T10:13:47.282 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:13:47.283 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:47.282+0000 7ff3cbe00700 1 -- 172.21.15.153:0/4005055651 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff3b006e590 msgr2=0x7ff3b0070a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:13:47.283 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:47.282+0000 7ff3cbe00700 1 --2- 172.21.15.153:0/4005055651 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff3b006e590 0x7ff3b0070a50 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc005950 tx=0x7ff3bc00b410 comp rx=0 tx=0).stop 2024-03-21T10:13:47.284 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:47.282+0000 7ff3cbe00700 1 -- 172.21.15.153:0/4005055651 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff3cc1037c0 msgr2=0x7ff3cc19ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:13:47.284 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:47.282+0000 7ff3cbe00700 1 --2- 172.21.15.153:0/4005055651 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff3cc1037c0 0x7ff3cc19ea90 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff3b400bd70 tx=0x7ff3b400bda0 comp rx=0 tx=0).stop 2024-03-21T10:13:47.284 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:47.282+0000 7ff3cbe00700 1 -- 172.21.15.153:0/4005055651 shutdown_connections 2024-03-21T10:13:47.284 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:47.282+0000 7ff3cbe00700 1 --2- 172.21.15.153:0/4005055651 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff3cc102e10 0x7ff3cc19e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:47.284 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:47.282+0000 7ff3cbe00700 1 --2- 172.21.15.153:0/4005055651 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff3b006e590 0x7ff3b0070a50 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:47.284 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:47.282+0000 7ff3cbe00700 1 --2- 172.21.15.153:0/4005055651 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff3cc1037c0 0x7ff3cc19ea90 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:47.284 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:47.282+0000 7ff3cbe00700 1 -- 172.21.15.153:0/4005055651 >> 172.21.15.153:0/4005055651 conn(0x7ff3cc0fe6c0 msgr2=0x7ff3cc100ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:13:47.285 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:47.282+0000 7ff3cbe00700 1 -- 172.21.15.153:0/4005055651 shutdown_connections 2024-03-21T10:13:47.285 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:47.282+0000 7ff3cbe00700 1 -- 172.21.15.153:0/4005055651 wait complete. 2024-03-21T10:13:47.434 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:47.432+0000 7f915f400700 1 -- 172.21.15.153:0/835453343 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9158102630 msgr2=0x7f915810ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:13:47.435 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:47.432+0000 7f915f400700 1 --2- 172.21.15.153:0/835453343 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9158102630 0x7f915810ab20 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f9144009910 tx=0x7f9144009c20 comp rx=0 tx=0).stop 2024-03-21T10:13:47.435 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:47.433+0000 7f915f400700 1 -- 172.21.15.153:0/835453343 shutdown_connections 2024-03-21T10:13:47.435 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:47.433+0000 7f915f400700 1 --2- 172.21.15.153:0/835453343 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9158102630 0x7f915810ab20 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:47.435 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:47.433+0000 7f915f400700 1 --2- 172.21.15.153:0/835453343 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9158101d10 0x7f91581020f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:47.435 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:47.433+0000 7f915f400700 1 -- 172.21.15.153:0/835453343 >> 172.21.15.153:0/835453343 conn(0x7f91580fb540 msgr2=0x7f91580fd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:13:47.435 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:47.433+0000 7f915f400700 1 -- 172.21.15.153:0/835453343 shutdown_connections 2024-03-21T10:13:47.435 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:47.433+0000 7f915f400700 1 -- 172.21.15.153:0/835453343 wait complete. 2024-03-21T10:13:47.435 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:47.434+0000 7f915f400700 1 Processor -- start 2024-03-21T10:13:47.435 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:47.434+0000 7f915f400700 1 -- start start 2024-03-21T10:13:47.436 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:47.435+0000 7f915f400700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9158101d10 0x7f915819c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:13:47.436 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:47.435+0000 7f915f400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9158102630 0x7f915819caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:13:47.436 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:47.435+0000 7f915f400700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f915819d130 con 0x7f9158102630 2024-03-21T10:13:47.436 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:47.435+0000 7f915f400700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91581965e0 con 0x7f9158101d10 2024-03-21T10:13:47.437 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:47.435+0000 7f9157e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9158102630 0x7f915819caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:13:47.437 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:47.435+0000 7f9157e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9158102630 0x7f915819caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:40338/0 (socket says 172.21.15.153:40338) 2024-03-21T10:13:47.437 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:47.435+0000 7f9157e00700 1 -- 172.21.15.153:0/3995134315 learned_addr learned my addr 172.21.15.153:0/3995134315 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:13:47.437 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:47.435+0000 7f915ca00700 1 --2- 172.21.15.153:0/3995134315 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9158101d10 0x7f915819c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:13:47.437 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:47.435+0000 7f915ca00700 1 -- 172.21.15.153:0/3995134315 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9158102630 msgr2=0x7f915819caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:13:47.437 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:47.435+0000 7f915ca00700 1 --2- 172.21.15.153:0/3995134315 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9158102630 0x7f915819caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:47.437 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:47.435+0000 7f915ca00700 1 -- 172.21.15.153:0/3995134315 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91440095f0 con 0x7f9158101d10 2024-03-21T10:13:47.437 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:47.436+0000 7f9157e00700 1 --2- 172.21.15.153:0/3995134315 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9158102630 0x7f915819caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:13:47.438 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:47.436+0000 7f915ca00700 1 --2- 172.21.15.153:0/3995134315 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9158101d10 0x7f915819c560 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f914c00e970 tx=0x7f914c00ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:13:47.438 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:47.436+0000 7f9155600700 1 -- 172.21.15.153:0/3995134315 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f914c00c9d0 con 0x7f9158101d10 2024-03-21T10:13:47.438 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:47.436+0000 7f915f400700 1 -- 172.21.15.153:0/3995134315 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91581968c0 con 0x7f9158101d10 2024-03-21T10:13:47.438 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:47.436+0000 7f9155600700 1 -- 172.21.15.153:0/3995134315 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f914c010430 con 0x7f9158101d10 2024-03-21T10:13:47.438 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:47.436+0000 7f9155600700 1 -- 172.21.15.153:0/3995134315 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f914c018630 con 0x7f9158101d10 2024-03-21T10:13:47.439 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:47.437+0000 7f915f400700 1 -- 172.21.15.153:0/3995134315 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9158196e10 con 0x7f9158101d10 2024-03-21T10:13:47.439 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:47.438+0000 7f915f400700 1 -- 172.21.15.153:0/3995134315 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9158108220 con 0x7f9158101d10 2024-03-21T10:13:47.440 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:47.439+0000 7f9155600700 1 -- 172.21.15.153:0/3995134315 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 37) v1 ==== 92313+0+0 (secure 0 0 0) 0x7f914c0105a0 con 0x7f9158101d10 2024-03-21T10:13:47.441 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:47 smithi153 ceph-mon[113733]: from='client.34130 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:13:47.441 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:47 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:13:47.441 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:47 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:13:47.441 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:47 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:13:47.441 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:47 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:13:47.441 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:47 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:13:47.442 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:47.440+0000 7f9155600700 1 --2- 172.21.15.153:0/3995134315 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f914006e590 0x7f9140070a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:13:47.442 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:47.440+0000 7f9155600700 1 -- 172.21.15.153:0/3995134315 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f914c014070 con 0x7f9158101d10 2024-03-21T10:13:47.442 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:47.441+0000 7f9157e00700 1 --2- 172.21.15.153:0/3995134315 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f914006e590 0x7f9140070a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:13:47.443 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:47.441+0000 7f9157e00700 1 --2- 172.21.15.153:0/3995134315 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f914006e590 0x7f9140070a50 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f914400b760 tx=0x7f9144005a90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:13:47.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:47 smithi174 ceph-mon[37496]: from='client.34130 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:13:47.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:47 smithi174 ceph-mon[37496]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:13:47.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:47 smithi174 ceph-mon[37496]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:13:47.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:47 smithi174 ceph-mon[37496]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:13:47.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:47 smithi174 ceph-mon[37496]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:13:47.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:47 smithi174 ceph-mon[37496]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:13:47.455 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:47.454+0000 7f9155600700 1 -- 172.21.15.153:0/3995134315 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f914c05b860 con 0x7f9158101d10 2024-03-21T10:13:47.723 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:47.721+0000 7f915f400700 1 -- 172.21.15.153:0/3995134315 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f91581970d0 con 0x7f9158101d10 2024-03-21T10:13:47.723 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:47.722+0000 7f9155600700 1 -- 172.21.15.153:0/3995134315 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 21 v21) v1 ==== 76+0+1850 (secure 0 0 0) 0x7f914c05b3f0 con 0x7f9158101d10 2024-03-21T10:13:47.724 INFO:teuthology.orchestra.run.smithi153.stdout:e21 2024-03-21T10:13:47.724 INFO:teuthology.orchestra.run.smithi153.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-21T10:13:47.724 INFO:teuthology.orchestra.run.smithi153.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:13:47.724 INFO:teuthology.orchestra.run.smithi153.stdout:legacy client fscid: 1 2024-03-21T10:13:47.724 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:13:47.724 INFO:teuthology.orchestra.run.smithi153.stdout:Filesystem 'cephfs' (1) 2024-03-21T10:13:47.724 INFO:teuthology.orchestra.run.smithi153.stdout:fs_name cephfs 2024-03-21T10:13:47.724 INFO:teuthology.orchestra.run.smithi153.stdout:epoch 21 2024-03-21T10:13:47.724 INFO:teuthology.orchestra.run.smithi153.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-21T10:13:47.725 INFO:teuthology.orchestra.run.smithi153.stdout:created 2024-03-21T10:04:27.086576+0000 2024-03-21T10:13:47.725 INFO:teuthology.orchestra.run.smithi153.stdout:modified 2024-03-21T10:13:01.455594+0000 2024-03-21T10:13:47.725 INFO:teuthology.orchestra.run.smithi153.stdout:tableserver 0 2024-03-21T10:13:47.725 INFO:teuthology.orchestra.run.smithi153.stdout:root 0 2024-03-21T10:13:47.725 INFO:teuthology.orchestra.run.smithi153.stdout:session_timeout 60 2024-03-21T10:13:47.725 INFO:teuthology.orchestra.run.smithi153.stdout:session_autoclose 300 2024-03-21T10:13:47.725 INFO:teuthology.orchestra.run.smithi153.stdout:max_file_size 1099511627776 2024-03-21T10:13:47.725 INFO:teuthology.orchestra.run.smithi153.stdout:required_client_features {} 2024-03-21T10:13:47.725 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure 0 2024-03-21T10:13:47.725 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure_osd_epoch 47 2024-03-21T10:13:47.725 INFO:teuthology.orchestra.run.smithi153.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:13:47.725 INFO:teuthology.orchestra.run.smithi153.stdout:max_mds 2 2024-03-21T10:13:47.725 INFO:teuthology.orchestra.run.smithi153.stdout:in 0,1 2024-03-21T10:13:47.725 INFO:teuthology.orchestra.run.smithi153.stdout:up {0=14484,1=24273} 2024-03-21T10:13:47.726 INFO:teuthology.orchestra.run.smithi153.stdout:failed 2024-03-21T10:13:47.726 INFO:teuthology.orchestra.run.smithi153.stdout:damaged 2024-03-21T10:13:47.726 INFO:teuthology.orchestra.run.smithi153.stdout:stopped 2024-03-21T10:13:47.726 INFO:teuthology.orchestra.run.smithi153.stdout:data_pools [3] 2024-03-21T10:13:47.726 INFO:teuthology.orchestra.run.smithi153.stdout:metadata_pool 2 2024-03-21T10:13:47.726 INFO:teuthology.orchestra.run.smithi153.stdout:inline_data enabled 2024-03-21T10:13:47.726 INFO:teuthology.orchestra.run.smithi153.stdout:balancer 2024-03-21T10:13:47.726 INFO:teuthology.orchestra.run.smithi153.stdout:bal_rank_mask -1 2024-03-21T10:13:47.726 INFO:teuthology.orchestra.run.smithi153.stdout:standby_count_wanted 1 2024-03-21T10:13:47.726 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.iuyluj{0:14484} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.153:6828/866138999,v1:172.21.15.153:6829/866138999] compat {c=[1],r=[1],i=[7ff]}] 2024-03-21T10:13:47.726 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.zipckr{0:14494} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.153:6826/4033622615,v1:172.21.15.153:6827/4033622615] compat {c=[1],r=[1],i=[7ff]}] 2024-03-21T10:13:47.726 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.yyimst{1:24273} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.174:6824/1776947618,v1:172.21.15.174:6825/1776947618] compat {c=[1],r=[1],i=[7ff]}] 2024-03-21T10:13:47.726 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.nhozag{1:24285} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.174:6826/806095873,v1:172.21.15.174:6827/806095873] compat {c=[1],r=[1],i=[7ff]}] 2024-03-21T10:13:47.726 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:13:47.727 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:13:47.727 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:47.725+0000 7f915f400700 1 -- 172.21.15.153:0/3995134315 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f914006e590 msgr2=0x7f9140070a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:13:47.727 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:47.725+0000 7f915f400700 1 --2- 172.21.15.153:0/3995134315 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f914006e590 0x7f9140070a50 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f914400b760 tx=0x7f9144005a90 comp rx=0 tx=0).stop 2024-03-21T10:13:47.728 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:47.725+0000 7f915f400700 1 -- 172.21.15.153:0/3995134315 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9158101d10 msgr2=0x7f915819c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:13:47.728 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:47.725+0000 7f915f400700 1 --2- 172.21.15.153:0/3995134315 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9158101d10 0x7f915819c560 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f914c00e970 tx=0x7f914c00ec80 comp rx=0 tx=0).stop 2024-03-21T10:13:47.728 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:47.725+0000 7f915f400700 1 -- 172.21.15.153:0/3995134315 shutdown_connections 2024-03-21T10:13:47.728 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:47.725+0000 7f915f400700 1 --2- 172.21.15.153:0/3995134315 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9158101d10 0x7f915819c560 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:47.728 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:47.725+0000 7f915f400700 1 --2- 172.21.15.153:0/3995134315 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f914006e590 0x7f9140070a50 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:47.728 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:47.725+0000 7f915f400700 1 --2- 172.21.15.153:0/3995134315 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9158102630 0x7f915819caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:47.728 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:47.725+0000 7f915f400700 1 -- 172.21.15.153:0/3995134315 >> 172.21.15.153:0/3995134315 conn(0x7f91580fb540 msgr2=0x7f9158105360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:13:47.728 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:47.725+0000 7f915f400700 1 -- 172.21.15.153:0/3995134315 shutdown_connections 2024-03-21T10:13:47.728 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:47.725+0000 7f915f400700 1 -- 172.21.15.153:0/3995134315 wait complete. 2024-03-21T10:13:47.728 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 21 2024-03-21T10:13:47.876 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:47.874+0000 7f007c600700 1 -- 172.21.15.153:0/2009269512 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f00741039a0 msgr2=0x7f00741079f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:13:47.876 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:47.874+0000 7f007c600700 1 --2- 172.21.15.153:0/2009269512 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f00741039a0 0x7f00741079f0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f0060009910 tx=0x7f0060009c20 comp rx=0 tx=0).stop 2024-03-21T10:13:47.876 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:47.874+0000 7f007c600700 1 -- 172.21.15.153:0/2009269512 shutdown_connections 2024-03-21T10:13:47.876 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:47.874+0000 7f007c600700 1 --2- 172.21.15.153:0/2009269512 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f00741039a0 0x7f00741079f0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:47.877 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:47.874+0000 7f007c600700 1 --2- 172.21.15.153:0/2009269512 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0074102ff0 0x7f00741033d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:47.877 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:47.874+0000 7f007c600700 1 -- 172.21.15.153:0/2009269512 >> 172.21.15.153:0/2009269512 conn(0x7f00740fe880 msgr2=0x7f0074100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:13:47.877 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:47.874+0000 7f007c600700 1 -- 172.21.15.153:0/2009269512 shutdown_connections 2024-03-21T10:13:47.877 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:47.875+0000 7f007c600700 1 -- 172.21.15.153:0/2009269512 wait complete. 2024-03-21T10:13:47.878 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:47.875+0000 7f007c600700 1 Processor -- start 2024-03-21T10:13:47.878 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:47.876+0000 7f007c600700 1 -- start start 2024-03-21T10:13:47.878 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:47.876+0000 7f007c600700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0074102ff0 0x7f00741989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:13:47.878 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:47.876+0000 7f007c600700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f00741039a0 0x7f0074198ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:13:47.878 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:47.876+0000 7f007c600700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0074199570 con 0x7f0074102ff0 2024-03-21T10:13:47.878 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:47.876+0000 7f007c600700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f007419d380 con 0x7f00741039a0 2024-03-21T10:13:47.878 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:47.876+0000 7f0079c00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0074102ff0 0x7f00741989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:13:47.878 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:47.876+0000 7f0079c00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0074102ff0 0x7f00741989a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:40362/0 (socket says 172.21.15.153:40362) 2024-03-21T10:13:47.878 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:47.876+0000 7f0079c00700 1 -- 172.21.15.153:0/3034271285 learned_addr learned my addr 172.21.15.153:0/3034271285 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:13:47.879 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:47.877+0000 7f0079200700 1 --2- 172.21.15.153:0/3034271285 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f00741039a0 0x7f0074198ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:13:47.879 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:47.877+0000 7f0079c00700 1 -- 172.21.15.153:0/3034271285 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f00741039a0 msgr2=0x7f0074198ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:13:47.880 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:47.877+0000 7f0079c00700 1 --2- 172.21.15.153:0/3034271285 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f00741039a0 0x7f0074198ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:47.880 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:47.877+0000 7f0079c00700 1 -- 172.21.15.153:0/3034271285 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00600095f0 con 0x7f0074102ff0 2024-03-21T10:13:47.880 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:47.877+0000 7f0079c00700 1 --2- 172.21.15.153:0/3034271285 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0074102ff0 0x7f00741989a0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f00680060b0 tx=0x7f006800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:13:47.880 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:47.877+0000 7f0072000700 1 -- 172.21.15.153:0/3034271285 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f006800dda0 con 0x7f0074102ff0 2024-03-21T10:13:47.880 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:47.878+0000 7f007c600700 1 -- 172.21.15.153:0/3034271285 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f007419d660 con 0x7f0074102ff0 2024-03-21T10:13:47.880 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:47.878+0000 7f0072000700 1 -- 172.21.15.153:0/3034271285 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f006800f040 con 0x7f0074102ff0 2024-03-21T10:13:47.880 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:47.878+0000 7f0072000700 1 -- 172.21.15.153:0/3034271285 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f00680146f0 con 0x7f0074102ff0 2024-03-21T10:13:47.880 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:47.878+0000 7f007c600700 1 -- 172.21.15.153:0/3034271285 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f007419db80 con 0x7f0074102ff0 2024-03-21T10:13:47.880 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:47.879+0000 7f007c600700 1 -- 172.21.15.153:0/3034271285 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f007410b3f0 con 0x7f0074102ff0 2024-03-21T10:13:47.881 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:47.880+0000 7f0072000700 1 -- 172.21.15.153:0/3034271285 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 37) v1 ==== 92321+0+0 (secure 0 0 0) 0x7f0068015400 con 0x7f0074102ff0 2024-03-21T10:13:47.882 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:47.881+0000 7f0072000700 1 --2- 172.21.15.153:0/3034271285 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f005c06e590 0x7f005c070a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:13:47.882 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:47.881+0000 7f0072000700 1 -- 172.21.15.153:0/3034271285 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f006808dfa0 con 0x7f0074102ff0 2024-03-21T10:13:47.883 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:47.881+0000 7f0079200700 1 --2- 172.21.15.153:0/3034271285 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f005c06e590 0x7f005c070a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:13:47.883 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:47.882+0000 7f0079200700 1 --2- 172.21.15.153:0/3034271285 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f005c06e590 0x7f005c070a50 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f0060006010 tx=0x7f006001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:13:47.887 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:47.886+0000 7f0072000700 1 -- 172.21.15.153:0/3034271285 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f006805abc0 con 0x7f0074102ff0 2024-03-21T10:13:48.150 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:48.148+0000 7f007c600700 1 -- 172.21.15.153:0/3034271285 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f007419ddd0 con 0x7f005c06e590 2024-03-21T10:13:48.151 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:48.150+0000 7f0072000700 1 -- 172.21.15.153:0/3034271285 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7f007419ddd0 con 0x7f005c06e590 2024-03-21T10:13:48.152 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:13:48.152 INFO:teuthology.orchestra.run.smithi153.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232", 2024-03-21T10:13:48.152 INFO:teuthology.orchestra.run.smithi153.stdout: "in_progress": true, 2024-03-21T10:13:48.152 INFO:teuthology.orchestra.run.smithi153.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-21T10:13:48.152 INFO:teuthology.orchestra.run.smithi153.stdout: "services_complete": [ 2024-03-21T10:13:48.152 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr" 2024-03-21T10:13:48.152 INFO:teuthology.orchestra.run.smithi153.stdout: ], 2024-03-21T10:13:48.152 INFO:teuthology.orchestra.run.smithi153.stdout: "progress": "3/23 daemons upgraded", 2024-03-21T10:13:48.152 INFO:teuthology.orchestra.run.smithi153.stdout: "message": "", 2024-03-21T10:13:48.152 INFO:teuthology.orchestra.run.smithi153.stdout: "is_paused": false 2024-03-21T10:13:48.152 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:13:48.154 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:48.152+0000 7f007c600700 1 -- 172.21.15.153:0/3034271285 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f005c06e590 msgr2=0x7f005c070a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:13:48.154 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:48.152+0000 7f007c600700 1 --2- 172.21.15.153:0/3034271285 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f005c06e590 0x7f005c070a50 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f0060006010 tx=0x7f006001a040 comp rx=0 tx=0).stop 2024-03-21T10:13:48.154 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:48.152+0000 7f007c600700 1 -- 172.21.15.153:0/3034271285 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0074102ff0 msgr2=0x7f00741989a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:13:48.154 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:48.152+0000 7f007c600700 1 --2- 172.21.15.153:0/3034271285 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0074102ff0 0x7f00741989a0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f00680060b0 tx=0x7f006800d3b0 comp rx=0 tx=0).stop 2024-03-21T10:13:48.154 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:48.152+0000 7f007c600700 1 -- 172.21.15.153:0/3034271285 shutdown_connections 2024-03-21T10:13:48.155 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:48.152+0000 7f007c600700 1 --2- 172.21.15.153:0/3034271285 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f005c06e590 0x7f005c070a50 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:48.155 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:48.152+0000 7f007c600700 1 --2- 172.21.15.153:0/3034271285 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0074102ff0 0x7f00741989a0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:48.155 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:48.152+0000 7f007c600700 1 --2- 172.21.15.153:0/3034271285 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f00741039a0 0x7f0074198ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:48.155 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:48.152+0000 7f007c600700 1 -- 172.21.15.153:0/3034271285 >> 172.21.15.153:0/3034271285 conn(0x7f00740fe880 msgr2=0x7f0074100c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:13:48.155 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:48.152+0000 7f007c600700 1 -- 172.21.15.153:0/3034271285 shutdown_connections 2024-03-21T10:13:48.155 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:48.153+0000 7f007c600700 1 -- 172.21.15.153:0/3034271285 wait complete. 2024-03-21T10:13:48.303 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:48.301+0000 7f9023800700 1 -- 172.21.15.153:0/583924156 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f901c101770 msgr2=0x7f901c1057c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:13:48.304 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:48.301+0000 7f9023800700 1 --2- 172.21.15.153:0/583924156 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f901c101770 0x7f901c1057c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f9008009910 tx=0x7f9008009c20 comp rx=0 tx=0).stop 2024-03-21T10:13:48.304 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:48.302+0000 7f9023800700 1 -- 172.21.15.153:0/583924156 shutdown_connections 2024-03-21T10:13:48.304 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:48.302+0000 7f9023800700 1 --2- 172.21.15.153:0/583924156 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f901c101770 0x7f901c1057c0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:48.304 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:48.302+0000 7f9023800700 1 --2- 172.21.15.153:0/583924156 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f901c100dc0 0x7f901c1011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:48.304 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:48.302+0000 7f9023800700 1 -- 172.21.15.153:0/583924156 >> 172.21.15.153:0/583924156 conn(0x7f901c0fc650 msgr2=0x7f901c0fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:13:48.304 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:48.302+0000 7f9023800700 1 -- 172.21.15.153:0/583924156 shutdown_connections 2024-03-21T10:13:48.304 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:48.302+0000 7f9023800700 1 -- 172.21.15.153:0/583924156 wait complete. 2024-03-21T10:13:48.305 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:48.303+0000 7f9023800700 1 Processor -- start 2024-03-21T10:13:48.305 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:48.303+0000 7f9023800700 1 -- start start 2024-03-21T10:13:48.306 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:48.303+0000 7f9023800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f901c100dc0 0x7f901c073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:13:48.306 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:48.303+0000 7f9023800700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f901c101770 0x7f901c0735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:13:48.306 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:48.303+0000 7f9023800700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f901c073b10 con 0x7f901c100dc0 2024-03-21T10:13:48.306 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:48.303+0000 7f9023800700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f901c073c50 con 0x7f901c101770 2024-03-21T10:13:48.306 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:48.304+0000 7f9020e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f901c100dc0 0x7f901c073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:13:48.306 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:48.304+0000 7f9020e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f901c100dc0 0x7f901c073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:40386/0 (socket says 172.21.15.153:40386) 2024-03-21T10:13:48.306 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:48.304+0000 7f9020e00700 1 -- 172.21.15.153:0/1642882712 learned_addr learned my addr 172.21.15.153:0/1642882712 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:13:48.306 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:48.304+0000 7f901be00700 1 --2- 172.21.15.153:0/1642882712 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f901c101770 0x7f901c0735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:13:48.306 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:48.304+0000 7f9020e00700 1 -- 172.21.15.153:0/1642882712 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f901c101770 msgr2=0x7f901c0735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:13:48.306 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:48.304+0000 7f9020e00700 1 --2- 172.21.15.153:0/1642882712 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f901c101770 0x7f901c0735d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:48.306 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:48.304+0000 7f9020e00700 1 -- 172.21.15.153:0/1642882712 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90080095f0 con 0x7f901c100dc0 2024-03-21T10:13:48.307 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:48.304+0000 7f9020e00700 1 --2- 172.21.15.153:0/1642882712 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f901c100dc0 0x7f901c073090 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f901000d8d0 tx=0x7f901000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:13:48.307 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:48.305+0000 7f9019600700 1 -- 172.21.15.153:0/1642882712 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f90100098a0 con 0x7f901c100dc0 2024-03-21T10:13:48.307 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:48.305+0000 7f9019600700 1 -- 172.21.15.153:0/1642882712 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9010010460 con 0x7f901c100dc0 2024-03-21T10:13:48.307 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:48.305+0000 7f9019600700 1 -- 172.21.15.153:0/1642882712 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f901000f5f0 con 0x7f901c100dc0 2024-03-21T10:13:48.307 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:48.305+0000 7f9023800700 1 -- 172.21.15.153:0/1642882712 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f901c1a2940 con 0x7f901c100dc0 2024-03-21T10:13:48.307 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:48.305+0000 7f9023800700 1 -- 172.21.15.153:0/1642882712 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f901c1a2d50 con 0x7f901c100dc0 2024-03-21T10:13:48.308 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:48.306+0000 7f9023800700 1 -- 172.21.15.153:0/1642882712 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f901c1091c0 con 0x7f901c100dc0 2024-03-21T10:13:48.309 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:48.307+0000 7f9019600700 1 -- 172.21.15.153:0/1642882712 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 37) v1 ==== 92321+0+0 (secure 0 0 0) 0x7f9010009a00 con 0x7f901c100dc0 2024-03-21T10:13:48.310 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:48.309+0000 7f9019600700 1 --2- 172.21.15.153:0/1642882712 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f900406e590 0x7f9004070a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:13:48.310 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:48.309+0000 7f9019600700 1 -- 172.21.15.153:0/1642882712 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f901008e480 con 0x7f901c100dc0 2024-03-21T10:13:48.311 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:48.309+0000 7f901be00700 1 --2- 172.21.15.153:0/1642882712 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f900406e590 0x7f9004070a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:13:48.312 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:48.310+0000 7f901be00700 1 --2- 172.21.15.153:0/1642882712 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f900406e590 0x7f9004070a50 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f901c074880 tx=0x7f900801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:13:48.315 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:48.314+0000 7f9019600700 1 -- 172.21.15.153:0/1642882712 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f90100574a0 con 0x7f901c100dc0 2024-03-21T10:13:48.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:48 smithi174 ceph-mon[37496]: Detected new or changed devices on smithi153 2024-03-21T10:13:48.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:48 smithi174 ceph-mon[37496]: Updating smithi153:/etc/ceph/ceph.conf 2024-03-21T10:13:48.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:48 smithi174 ceph-mon[37496]: Updating smithi174:/etc/ceph/ceph.conf 2024-03-21T10:13:48.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:48 smithi174 ceph-mon[37496]: from='client.34134 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:13:48.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:48 smithi174 ceph-mon[37496]: from='client.34138 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:13:48.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:48 smithi174 ceph-mon[37496]: Updating smithi153:/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T10:13:48.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:48 smithi174 ceph-mon[37496]: Updating smithi174:/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T10:13:48.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:48 smithi174 ceph-mon[37496]: pgmap v11: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 19 KiB/s rd, 0 B/s wr, 12 op/s 2024-03-21T10:13:48.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:48 smithi174 ceph-mon[37496]: from='client.? 172.21.15.153:0/4005055651' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:13:48.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:48 smithi174 ceph-mon[37496]: from='client.? 172.21.15.153:0/3995134315' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:13:48.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:48 smithi153 ceph-mon[113733]: Detected new or changed devices on smithi153 2024-03-21T10:13:48.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:48 smithi153 ceph-mon[113733]: Updating smithi153:/etc/ceph/ceph.conf 2024-03-21T10:13:48.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:48 smithi153 ceph-mon[113733]: Updating smithi174:/etc/ceph/ceph.conf 2024-03-21T10:13:48.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:48 smithi153 ceph-mon[113733]: from='client.34134 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:13:48.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:48 smithi153 ceph-mon[113733]: from='client.34138 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:13:48.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:48 smithi153 ceph-mon[113733]: Updating smithi153:/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T10:13:48.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:48 smithi153 ceph-mon[113733]: Updating smithi174:/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T10:13:48.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:48 smithi153 ceph-mon[113733]: pgmap v11: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 19 KiB/s rd, 0 B/s wr, 12 op/s 2024-03-21T10:13:48.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:48 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/4005055651' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:13:48.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:48 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/3995134315' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:13:48.670 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:48.668+0000 7f9023800700 1 -- 172.21.15.153:0/1642882712 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f901c073e10 con 0x7f901c100dc0 2024-03-21T10:13:48.671 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:48.669+0000 7f9019600700 1 -- 172.21.15.153:0/1642882712 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f901005ab60 con 0x7f901c100dc0 2024-03-21T10:13:48.671 INFO:teuthology.orchestra.run.smithi153.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-03-21T10:13:48.671 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-21T10:13:48.671 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-21T10:13:48.673 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:48.671+0000 7f9023800700 1 -- 172.21.15.153:0/1642882712 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f900406e590 msgr2=0x7f9004070a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:13:48.673 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:48.671+0000 7f9023800700 1 --2- 172.21.15.153:0/1642882712 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f900406e590 0x7f9004070a50 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f901c074880 tx=0x7f900801a040 comp rx=0 tx=0).stop 2024-03-21T10:13:48.673 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:48.671+0000 7f9023800700 1 -- 172.21.15.153:0/1642882712 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f901c100dc0 msgr2=0x7f901c073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:13:48.674 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:48.671+0000 7f9023800700 1 --2- 172.21.15.153:0/1642882712 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f901c100dc0 0x7f901c073090 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f901000d8d0 tx=0x7f901000dbe0 comp rx=0 tx=0).stop 2024-03-21T10:13:48.674 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:48.672+0000 7f9023800700 1 -- 172.21.15.153:0/1642882712 shutdown_connections 2024-03-21T10:13:48.674 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:48.672+0000 7f9023800700 1 --2- 172.21.15.153:0/1642882712 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f900406e590 0x7f9004070a50 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:48.674 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:48.672+0000 7f9023800700 1 --2- 172.21.15.153:0/1642882712 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f901c100dc0 0x7f901c073090 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:48.674 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:48.672+0000 7f9023800700 1 --2- 172.21.15.153:0/1642882712 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f901c101770 0x7f901c0735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:13:48.674 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:48.672+0000 7f9023800700 1 -- 172.21.15.153:0/1642882712 >> 172.21.15.153:0/1642882712 conn(0x7f901c0fc650 msgr2=0x7f901c0fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:13:48.674 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:48.672+0000 7f9023800700 1 -- 172.21.15.153:0/1642882712 shutdown_connections 2024-03-21T10:13:48.674 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:13:48.672+0000 7f9023800700 1 -- 172.21.15.153:0/1642882712 wait complete. 2024-03-21T10:13:49.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:49 smithi174 ceph-mon[37496]: Updating smithi153:/etc/ceph/ceph.client.admin.keyring 2024-03-21T10:13:49.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:49 smithi174 ceph-mon[37496]: Updating smithi174:/etc/ceph/ceph.client.admin.keyring 2024-03-21T10:13:49.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:49 smithi174 ceph-mon[37496]: Updating smithi153:/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-21T10:13:49.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:49 smithi174 ceph-mon[37496]: from='client.34150 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:13:49.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:49 smithi174 ceph-mon[37496]: from='client.? 172.21.15.153:0/1642882712' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:13:49.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:49 smithi174 ceph-mon[37496]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:13:49.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:49 smithi174 ceph-mon[37496]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:13:49.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:49 smithi153 ceph-mon[113733]: Updating smithi153:/etc/ceph/ceph.client.admin.keyring 2024-03-21T10:13:49.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:49 smithi153 ceph-mon[113733]: Updating smithi174:/etc/ceph/ceph.client.admin.keyring 2024-03-21T10:13:49.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:49 smithi153 ceph-mon[113733]: Updating smithi153:/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-21T10:13:49.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:49 smithi153 ceph-mon[113733]: from='client.34150 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:13:49.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:49 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/1642882712' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:13:49.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:49 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:13:49.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:49 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:13:50.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:50 smithi174 ceph-mon[37496]: Updating smithi174:/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-21T10:13:50.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:50 smithi174 ceph-mon[37496]: pgmap v12: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 20 KiB/s rd, 0 B/s wr, 14 op/s 2024-03-21T10:13:50.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:50 smithi174 ceph-mon[37496]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:13:50.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:50 smithi174 ceph-mon[37496]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:13:50.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:50 smithi174 ceph-mon[37496]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:13:50.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:50 smithi174 ceph-mon[37496]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:13:50.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:50 smithi174 ceph-mon[37496]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:13:50.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:50 smithi174 ceph-mon[37496]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-03-21T10:13:50.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:50 smithi153 ceph-mon[113733]: Updating smithi174:/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-21T10:13:50.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:50 smithi153 ceph-mon[113733]: pgmap v12: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 20 KiB/s rd, 0 B/s wr, 14 op/s 2024-03-21T10:13:50.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:50 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:13:50.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:50 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:13:50.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:50 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:13:50.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:50 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:13:50.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:50 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:13:50.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:50 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-03-21T10:13:52.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:52 smithi153 ceph-mon[113733]: pgmap v13: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:13:52.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:52 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:13:52.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:52 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T10:13:52.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:52 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-21T10:13:52.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:52 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:13:52.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:52 smithi174 ceph-mon[37496]: pgmap v13: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:13:52.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:52 smithi174 ceph-mon[37496]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:13:52.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:52 smithi174 ceph-mon[37496]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T10:13:52.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:52 smithi174 ceph-mon[37496]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-21T10:13:52.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:52 smithi174 ceph-mon[37496]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:13:53.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:53 smithi174 ceph-mon[37496]: Upgrade: Updating mon.smithi174 2024-03-21T10:13:53.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:53 smithi174 ceph-mon[37496]: Deploying daemon mon.smithi174 on smithi174 2024-03-21T10:13:53.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:53 smithi153 ceph-mon[113733]: Upgrade: Updating mon.smithi174 2024-03-21T10:13:53.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:53 smithi153 ceph-mon[113733]: Deploying daemon mon.smithi174 on smithi174 2024-03-21T10:13:54.412 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:54 smithi174 ceph-mon[37496]: pgmap v14: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:13:54.558 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:54 smithi153 ceph-mon[113733]: pgmap v14: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:13:54.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:54 smithi174 systemd[1]: Stopping Ceph mon.smithi174 for 070ed3e4-e769-11ee-95cd-87774f69a715... 2024-03-21T10:13:55.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:54 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-mon-smithi174[37482]: 2024-03-21T10:13:54.839+0000 7f7186e00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi174 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T10:13:55.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:54 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-mon-smithi174[37482]: 2024-03-21T10:13:54.839+0000 7f7186e00700 -1 mon.smithi174@1(peon) e2 *** Got Signal Terminated *** 2024-03-21T10:13:55.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:55 smithi174 podman[87845]: 2024-03-21 10:13:55.087492205 +0000 UTC m=+0.370522461 container died 99204923eec612ae776aff9c666751603c4144d04fd61bf839168efafed8d00d (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-mon-smithi174, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=-18.2.1, io.buildah.version=1.29.1) 2024-03-21T10:13:55.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:55 smithi174 podman[87845]: 2024-03-21 10:13:55.549911972 +0000 UTC m=+0.832942221 container cleanup 99204923eec612ae776aff9c666751603c4144d04fd61bf839168efafed8d00d (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-mon-smithi174, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=HEAD, io.buildah.version=1.29.1, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=https://github.com/ceph/ceph-container.git, CEPH_POINT_RELEASE=-18.2.1, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-03-21T10:13:55.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:55 smithi174 bash[87845]: ceph-070ed3e4-e769-11ee-95cd-87774f69a715-mon-smithi174 2024-03-21T10:13:56.161 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:55 smithi174 podman[87859]: 2024-03-21 10:13:55.731193928 +0000 UTC m=+0.621857973 container remove 99204923eec612ae776aff9c666751603c4144d04fd61bf839168efafed8d00d (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-mon-smithi174, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , io.buildah.version=1.29.1, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=HEAD, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=-18.2.1, org.label-schema.vendor=CentOS) 2024-03-21T10:13:56.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:56 smithi174 systemd[1]: ceph-070ed3e4-e769-11ee-95cd-87774f69a715@mon.smithi174.service: Deactivated successfully. 2024-03-21T10:13:56.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:56 smithi174 systemd[1]: Stopped Ceph mon.smithi174 for 070ed3e4-e769-11ee-95cd-87774f69a715. 2024-03-21T10:13:56.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:56 smithi174 systemd[1]: ceph-070ed3e4-e769-11ee-95cd-87774f69a715@mon.smithi174.service: Consumed 19.706s CPU time. 2024-03-21T10:13:56.933 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:56 smithi174 systemd[1]: Starting Ceph mon.smithi174 for 070ed3e4-e769-11ee-95cd-87774f69a715... 2024-03-21T10:13:57.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:56 smithi174 podman[87986]: 2024-03-21 10:13:56.842019065 +0000 UTC m=+0.038628223 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T10:13:57.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:56 smithi174 podman[87986]: 2024-03-21 10:13:56.951781112 +0000 UTC m=+0.148390302 container create 29d78a788f2cf394f70c4b9c11f4bf74d6f46c36e05cb7f0a0a8a17bc9718a44 (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-mon-smithi174, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True) 2024-03-21T10:13:57.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 podman[87986]: 2024-03-21 10:13:57.213493426 +0000 UTC m=+0.410102623 container init 29d78a788f2cf394f70c4b9c11f4bf74d6f46c36e05cb7f0a0a8a17bc9718a44 (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-mon-smithi174, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_BRANCH=HEAD) 2024-03-21T10:13:57.703 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 podman[87986]: 2024-03-21 10:13:57.224083897 +0000 UTC m=+0.420693113 container start 29d78a788f2cf394f70c4b9c11f4bf74d6f46c36e05cb7f0a0a8a17bc9718a44 (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-mon-smithi174, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-03-21T10:13:57.703 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: set uid:gid to 167:167 (ceph:ceph) 2024-03-21T10:13:57.703 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev), process ceph-mon, pid 2 2024-03-21T10:13:57.703 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: pidfile_write: ignore empty --pid-file 2024-03-21T10:13:57.703 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: load: jerasure load: lrc 2024-03-21T10:13:57.703 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: RocksDB version: 7.9.2 2024-03-21T10:13:57.703 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Git sha 0 2024-03-21T10:13:57.703 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Compile date 2024-03-20 06:23:48 2024-03-21T10:13:57.703 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: DB SUMMARY 2024-03-21T10:13:57.703 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: DB Session ID: S16UBQACJONHXIJC6QYL 2024-03-21T10:13:57.703 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: CURRENT file: CURRENT 2024-03-21T10:13:57.703 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: IDENTITY file: IDENTITY 2024-03-21T10:13:57.704 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: MANIFEST file: MANIFEST-000010 size: 1640 Bytes 2024-03-21T10:13:57.704 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi174/store.db dir, Total Num: 1, files: 000030.sst 2024-03-21T10:13:57.704 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi174/store.db: 000028.log size: 3960382 ; 2024-03-21T10:13:57.704 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.error_if_exists: 0 2024-03-21T10:13:57.704 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.create_if_missing: 0 2024-03-21T10:13:57.704 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.paranoid_checks: 1 2024-03-21T10:13:57.704 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-21T10:13:57.704 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-21T10:13:57.704 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-21T10:13:57.704 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.env: 0x55dd6752db20 2024-03-21T10:13:57.704 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.fs: PosixFileSystem 2024-03-21T10:13:57.704 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.info_log: 0x55dd68f45840 2024-03-21T10:13:57.704 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.max_file_opening_threads: 16 2024-03-21T10:13:57.705 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.statistics: (nil) 2024-03-21T10:13:57.705 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.use_fsync: 0 2024-03-21T10:13:57.705 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.max_log_file_size: 0 2024-03-21T10:13:57.705 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-21T10:13:57.705 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-21T10:13:57.705 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.keep_log_file_num: 1000 2024-03-21T10:13:57.705 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.recycle_log_file_num: 0 2024-03-21T10:13:57.705 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.allow_fallocate: 1 2024-03-21T10:13:57.705 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.allow_mmap_reads: 0 2024-03-21T10:13:57.705 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.allow_mmap_writes: 0 2024-03-21T10:13:57.705 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.use_direct_reads: 0 2024-03-21T10:13:57.706 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-21T10:13:57.706 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.create_missing_column_families: 0 2024-03-21T10:13:57.706 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.db_log_dir: 2024-03-21T10:13:57.706 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.wal_dir: 2024-03-21T10:13:57.706 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-21T10:13:57.706 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-21T10:13:57.706 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-21T10:13:57.706 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-21T10:13:57.706 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-21T10:13:57.706 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-21T10:13:57.706 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.advise_random_on_open: 1 2024-03-21T10:13:57.706 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.db_write_buffer_size: 0 2024-03-21T10:13:57.706 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.write_buffer_manager: 0x55dd68f181e0 2024-03-21T10:13:57.706 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-21T10:13:57.707 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-21T10:13:57.707 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-21T10:13:57.707 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.rate_limiter: (nil) 2024-03-21T10:13:57.707 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-21T10:13:57.707 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.wal_recovery_mode: 2 2024-03-21T10:13:57.707 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.enable_thread_tracking: 0 2024-03-21T10:13:57.707 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.enable_pipelined_write: 0 2024-03-21T10:13:57.707 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.unordered_write: 0 2024-03-21T10:13:57.707 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-21T10:13:57.707 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-21T10:13:57.707 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-21T10:13:57.707 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-21T10:13:57.707 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.row_cache: None 2024-03-21T10:13:57.708 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.wal_filter: None 2024-03-21T10:13:57.708 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-21T10:13:57.708 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.allow_ingest_behind: 0 2024-03-21T10:13:57.708 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.two_write_queues: 0 2024-03-21T10:13:57.709 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.manual_wal_flush: 0 2024-03-21T10:13:57.709 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.wal_compression: 0 2024-03-21T10:13:57.709 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.atomic_flush: 0 2024-03-21T10:13:57.709 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-21T10:13:57.709 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-21T10:13:57.709 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-21T10:13:57.709 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.log_readahead_size: 0 2024-03-21T10:13:57.709 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-21T10:13:57.709 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.best_efforts_recovery: 0 2024-03-21T10:13:57.709 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-21T10:13:57.709 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-21T10:13:57.709 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.allow_data_in_errors: 0 2024-03-21T10:13:57.710 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.db_host_id: __hostname__ 2024-03-21T10:13:57.710 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-21T10:13:57.710 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.max_background_jobs: 2 2024-03-21T10:13:57.710 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.max_background_compactions: -1 2024-03-21T10:13:57.712 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.max_subcompactions: 1 2024-03-21T10:13:57.712 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-21T10:13:57.712 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-21T10:13:57.712 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-21T10:13:57.712 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.max_total_wal_size: 0 2024-03-21T10:13:57.712 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-21T10:13:57.712 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-21T10:13:57.712 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-21T10:13:57.712 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-21T10:13:57.712 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.max_open_files: -1 2024-03-21T10:13:57.712 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.bytes_per_sync: 0 2024-03-21T10:13:57.713 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-21T10:13:57.713 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-21T10:13:57.713 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.compaction_readahead_size: 0 2024-03-21T10:13:57.713 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.max_background_flushes: -1 2024-03-21T10:13:57.713 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Compression algorithms supported: 2024-03-21T10:13:57.713 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-21T10:13:57.713 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: kZSTD supported: 0 2024-03-21T10:13:57.713 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: kXpressCompression supported: 0 2024-03-21T10:13:57.713 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: kLZ4HCCompression supported: 1 2024-03-21T10:13:57.713 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: kZlibCompression supported: 1 2024-03-21T10:13:57.713 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: kSnappyCompression supported: 1 2024-03-21T10:13:57.713 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: kLZ4Compression supported: 1 2024-03-21T10:13:57.714 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: kBZip2Compression supported: 0 2024-03-21T10:13:57.714 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-21T10:13:57.714 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-21T10:13:57.714 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi174/store.db/MANIFEST-000010 2024-03-21T10:13:57.714 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-21T10:13:57.714 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-21T10:13:57.714 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.merge_operator: 2024-03-21T10:13:57.714 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.compaction_filter: None 2024-03-21T10:13:57.714 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.compaction_filter_factory: None 2024-03-21T10:13:57.714 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.sst_partitioner_factory: None 2024-03-21T10:13:57.714 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-21T10:13:57.714 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-21T10:13:57.714 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55dd68f45940) 2024-03-21T10:13:57.715 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: cache_index_and_filter_blocks: 1 2024-03-21T10:13:57.715 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-21T10:13:57.715 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-21T10:13:57.715 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: pin_top_level_index_and_filter: 1 2024-03-21T10:13:57.715 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: index_type: 0 2024-03-21T10:13:57.715 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: data_block_index_type: 0 2024-03-21T10:13:57.715 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: index_shortening: 1 2024-03-21T10:13:57.715 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-21T10:13:57.715 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: checksum: 4 2024-03-21T10:13:57.715 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: no_block_cache: 0 2024-03-21T10:13:57.715 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: block_cache: 0x55dd68f4b090 2024-03-21T10:13:57.715 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: block_cache_name: BinnedLRUCache 2024-03-21T10:13:57.715 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: block_cache_options: 2024-03-21T10:13:57.715 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: capacity : 536870912 2024-03-21T10:13:57.715 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: num_shard_bits : 4 2024-03-21T10:13:57.716 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: strict_capacity_limit : 0 2024-03-21T10:13:57.716 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: high_pri_pool_ratio: 0.000 2024-03-21T10:13:57.716 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: block_cache_compressed: (nil) 2024-03-21T10:13:57.716 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: persistent_cache: (nil) 2024-03-21T10:13:57.716 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: block_size: 4096 2024-03-21T10:13:57.716 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: block_size_deviation: 10 2024-03-21T10:13:57.716 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: block_restart_interval: 16 2024-03-21T10:13:57.716 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: index_block_restart_interval: 1 2024-03-21T10:13:57.716 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: metadata_block_size: 4096 2024-03-21T10:13:57.716 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: partition_filters: 0 2024-03-21T10:13:57.716 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: use_delta_encoding: 1 2024-03-21T10:13:57.716 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: filter_policy: bloomfilter 2024-03-21T10:13:57.716 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: whole_key_filtering: 1 2024-03-21T10:13:57.716 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: verify_compression: 0 2024-03-21T10:13:57.716 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: read_amp_bytes_per_bit: 0 2024-03-21T10:13:57.717 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: format_version: 5 2024-03-21T10:13:57.717 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: enable_index_compression: 1 2024-03-21T10:13:57.717 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: block_align: 0 2024-03-21T10:13:57.717 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: max_auto_readahead_size: 262144 2024-03-21T10:13:57.717 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: prepopulate_block_cache: 0 2024-03-21T10:13:57.717 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: initial_auto_readahead_size: 8192 2024-03-21T10:13:57.717 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: num_file_reads_for_auto_readahead: 2 2024-03-21T10:13:57.717 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.write_buffer_size: 33554432 2024-03-21T10:13:57.717 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.max_write_buffer_number: 2 2024-03-21T10:13:57.717 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.compression: NoCompression 2024-03-21T10:13:57.717 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.bottommost_compression: Disabled 2024-03-21T10:13:57.717 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.prefix_extractor: nullptr 2024-03-21T10:13:57.718 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-21T10:13:57.718 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.num_levels: 7 2024-03-21T10:13:57.718 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-21T10:13:57.718 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-21T10:13:57.718 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-21T10:13:57.718 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-21T10:13:57.718 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-21T10:13:57.718 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-21T10:13:57.718 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-21T10:13:57.718 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-21T10:13:57.718 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-21T10:13:57.718 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-21T10:13:57.718 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-21T10:13:57.719 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-21T10:13:57.719 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-21T10:13:57.719 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.compression_opts.level: 32767 2024-03-21T10:13:57.719 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.compression_opts.strategy: 0 2024-03-21T10:13:57.719 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-21T10:13:57.719 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-21T10:13:57.719 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-21T10:13:57.719 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-21T10:13:57.719 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.compression_opts.enabled: false 2024-03-21T10:13:57.719 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-21T10:13:57.719 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-21T10:13:57.719 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-21T10:13:57.719 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-21T10:13:57.719 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.target_file_size_base: 67108864 2024-03-21T10:13:57.720 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-21T10:13:57.720 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-21T10:13:57.720 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-21T10:13:57.720 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-21T10:13:57.720 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-21T10:13:57.721 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-21T10:13:57.721 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-21T10:13:57.721 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-21T10:13:57.721 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-21T10:13:57.721 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-21T10:13:57.721 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-21T10:13:57.721 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-21T10:13:57.721 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-21T10:13:57.722 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-21T10:13:57.722 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.arena_block_size: 1048576 2024-03-21T10:13:57.722 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-21T10:13:57.722 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-21T10:13:57.722 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.disable_auto_compactions: 0 2024-03-21T10:13:57.722 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-21T10:13:57.722 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-21T10:13:57.722 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-21T10:13:57.722 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-21T10:13:57.722 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-21T10:13:57.722 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-21T10:13:57.722 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-21T10:13:57.722 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-21T10:13:57.723 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-21T10:13:57.723 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-21T10:13:57.723 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.table_properties_collectors: 2024-03-21T10:13:57.723 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.inplace_update_support: 0 2024-03-21T10:13:57.723 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-21T10:13:57.723 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-21T10:13:57.723 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-21T10:13:57.723 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-21T10:13:57.723 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.bloom_locality: 0 2024-03-21T10:13:57.723 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.max_successive_merges: 0 2024-03-21T10:13:57.723 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-21T10:13:57.723 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.paranoid_file_checks: 0 2024-03-21T10:13:57.724 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.force_consistency_checks: 1 2024-03-21T10:13:57.724 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.report_bg_io_stats: 0 2024-03-21T10:13:57.724 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.ttl: 2592000 2024-03-21T10:13:57.724 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-21T10:13:57.724 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-21T10:13:57.724 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-21T10:13:57.724 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.enable_blob_files: false 2024-03-21T10:13:57.724 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.min_blob_size: 0 2024-03-21T10:13:57.724 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.blob_file_size: 268435456 2024-03-21T10:13:57.724 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-21T10:13:57.724 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-21T10:13:57.724 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-21T10:13:57.724 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-21T10:13:57.724 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-21T10:13:57.725 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.blob_file_starting_level: 0 2024-03-21T10:13:57.725 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-21T10:13:57.725 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-03-21T10:13:57.725 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi174/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 32, last_sequence is 16883, log_number is 28,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 28 2024-03-21T10:13:57.725 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 28 2024-03-21T10:13:57.725 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 6b684d55-665e-4251-a5ca-36c284b8aed1 2024-03-21T10:13:57.725 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711016037279929, "job": 1, "event": "recovery_started", "wal_files": [28]} 2024-03-21T10:13:57.725 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #28 mode 2 2024-03-21T10:13:57.725 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711016037296151, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 33, "file_size": 2528193, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 16888, "largest_seqno": 18563, "table_properties": {"data_size": 2521357, "index_size": 4153, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 1669, "raw_key_size": 15331, "raw_average_key_size": 23, "raw_value_size": 2507458, "raw_average_value_size": 3799, "num_data_blocks": 187, "num_entries": 660, "num_filter_entries": 660, "num_deletions": 1, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1711016037, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "6b684d55-665e-4251-a5ca-36c284b8aed1", "db_session_id": "S16UBQACJONHXIJC6QYL", "orig_file_number": 33, "seqno_to_time_mapping": "N/A"}} 2024-03-21T10:13:57.725 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711016037296343, "job": 1, "event": "recovery_finished"} 2024-03-21T10:13:57.725 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: [db/version_set.cc:5047] Creating manifest 35 2024-03-21T10:13:57.725 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-03-21T10:13:57.726 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi174/store.db/000028.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-21T10:13:57.726 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55dd6903c000 2024-03-21T10:13:57.726 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: DB pointer 0x55dd69028000 2024-03-21T10:13:57.726 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-21T10:13:57.726 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-21T10:13:57.726 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: ** DB Stats ** 2024-03-21T10:13:57.726 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-21T10:13:57.726 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-21T10:13:57.726 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-21T10:13:57.726 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-21T10:13:57.726 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-21T10:13:57.726 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-21T10:13:57.727 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-21T10:13:57.727 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: 2024-03-21T10:13:57.727 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: ** Compaction Stats [default] ** 2024-03-21T10:13:57.727 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-21T10:13:57.727 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-21T10:13:57.727 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: L0 1/0 2.41 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 457.9 0.01 0.00 1 0.005 0 0 0.0 0.0 2024-03-21T10:13:57.727 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: L6 1/0 7.99 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-03-21T10:13:57.727 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: Sum 2/0 10.40 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 457.9 0.01 0.00 1 0.005 0 0 0.0 0.0 2024-03-21T10:13:57.727 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 457.9 0.01 0.00 1 0.005 0 0 0.0 0.0 2024-03-21T10:13:57.727 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: 2024-03-21T10:13:57.727 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: ** Compaction Stats [default] ** 2024-03-21T10:13:57.727 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-21T10:13:57.727 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-21T10:13:57.728 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 457.9 0.01 0.00 1 0.005 0 0 0.0 0.0 2024-03-21T10:13:57.728 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: 2024-03-21T10:13:57.728 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-21T10:13:57.728 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: 2024-03-21T10:13:57.728 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-21T10:13:57.728 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: Flush(GB): cumulative 0.002, interval 0.002 2024-03-21T10:13:57.728 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-21T10:13:57.728 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-21T10:13:57.728 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-21T10:13:57.728 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-21T10:13:57.728 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: Cumulative compaction: 0.00 GB write, 109.88 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-21T10:13:57.728 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: Interval compaction: 0.00 GB write, 109.88 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-21T10:13:57.728 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-21T10:13:57.728 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: Block cache BinnedLRUCache@0x55dd68f4b090#2 capacity: 512.00 MB usage: 6.36 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.7e-05 secs_since: 0 2024-03-21T10:13:57.728 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,1.78 KB,0.000339746%) IndexBlock(1,4.58 KB,0.000873208%) Misc(1,0.00 KB,0%) 2024-03-21T10:13:57.729 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: 2024-03-21T10:13:57.729 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-21T10:13:57.729 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: starting mon.smithi174 rank 1 at public addrs [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] at bind addrs [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi174 fsid 070ed3e4-e769-11ee-95cd-87774f69a715 2024-03-21T10:13:57.729 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: mon.smithi174@-1(???) e2 preinit fsid 070ed3e4-e769-11ee-95cd-87774f69a715 2024-03-21T10:13:57.729 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: mon.smithi174@-1(???).mds e21 new map 2024-03-21T10:13:57.729 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: mon.smithi174@-1(???).mds e21 print_map 2024-03-21T10:13:57.729 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: e21 2024-03-21T10:13:57.729 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-21T10:13:57.729 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:13:57.729 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: legacy client fscid: 1 2024-03-21T10:13:57.729 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: 2024-03-21T10:13:57.729 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: Filesystem 'cephfs' (1) 2024-03-21T10:13:57.729 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: fs_name cephfs 2024-03-21T10:13:57.730 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: epoch 21 2024-03-21T10:13:57.730 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-21T10:13:57.730 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: created 2024-03-21T10:04:27.086576+0000 2024-03-21T10:13:57.730 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: modified 2024-03-21T10:13:01.455594+0000 2024-03-21T10:13:57.730 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: tableserver 0 2024-03-21T10:13:57.730 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: root 0 2024-03-21T10:13:57.730 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: session_timeout 60 2024-03-21T10:13:57.730 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: session_autoclose 300 2024-03-21T10:13:57.730 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: max_file_size 1099511627776 2024-03-21T10:13:57.730 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: max_xattr_size 65536 2024-03-21T10:13:57.730 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: required_client_features {} 2024-03-21T10:13:57.730 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: last_failure 0 2024-03-21T10:13:57.730 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: last_failure_osd_epoch 47 2024-03-21T10:13:57.731 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:13:57.731 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: max_mds 2 2024-03-21T10:13:57.731 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: in 0,1 2024-03-21T10:13:57.731 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: up {0=14484,1=24273} 2024-03-21T10:13:57.731 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: failed 2024-03-21T10:13:57.731 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: damaged 2024-03-21T10:13:57.731 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: stopped 2024-03-21T10:13:57.731 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: data_pools [3] 2024-03-21T10:13:57.731 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: metadata_pool 2 2024-03-21T10:13:57.731 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: inline_data enabled 2024-03-21T10:13:57.731 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: balancer 2024-03-21T10:13:57.731 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: bal_rank_mask -1 2024-03-21T10:13:57.731 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: standby_count_wanted 1 2024-03-21T10:13:57.731 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: qdb_cluster leader: 0 members: 2024-03-21T10:13:57.732 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: [mds.cephfs.smithi153.iuyluj{0:14484} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.153:6828/866138999,v1:172.21.15.153:6829/866138999] compat {c=[1],r=[1],i=[7ff]}] 2024-03-21T10:13:57.732 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: [mds.cephfs.smithi153.zipckr{0:14494} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.153:6826/4033622615,v1:172.21.15.153:6827/4033622615] compat {c=[1],r=[1],i=[7ff]}] 2024-03-21T10:13:57.732 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: [mds.cephfs.smithi174.yyimst{1:24273} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.174:6824/1776947618,v1:172.21.15.174:6825/1776947618] compat {c=[1],r=[1],i=[7ff]}] 2024-03-21T10:13:57.732 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: [mds.cephfs.smithi174.nhozag{1:24285} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.174:6826/806095873,v1:172.21.15.174:6827/806095873] compat {c=[1],r=[1],i=[7ff]}] 2024-03-21T10:13:57.732 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: 2024-03-21T10:13:57.732 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout: 2024-03-21T10:13:57.732 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: mon.smithi174@-1(???).osd e50 crush map has features 3314933000852226048, adjusting msgr requires 2024-03-21T10:13:57.732 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: mon.smithi174@-1(???).osd e50 crush map has features 288514051259236352, adjusting msgr requires 2024-03-21T10:13:57.732 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: mon.smithi174@-1(???).osd e50 crush map has features 288514051259236352, adjusting msgr requires 2024-03-21T10:13:57.732 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: mon.smithi174@-1(???).osd e50 crush map has features 288514051259236352, adjusting msgr requires 2024-03-21T10:13:57.732 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 ceph-mon[88004]: mon.smithi174@-1(???).paxosservice(auth 1..25) refresh upgraded, format 0 -> 3 2024-03-21T10:13:57.732 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 bash[87986]: 29d78a788f2cf394f70c4b9c11f4bf74d6f46c36e05cb7f0a0a8a17bc9718a44 2024-03-21T10:13:57.732 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:57 smithi174 systemd[1]: Started Ceph mon.smithi174 for 070ed3e4-e769-11ee-95cd-87774f69a715. 2024-03-21T10:13:59.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:59 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mon metadata", "id": "smithi153"}]: dispatch 2024-03-21T10:13:59.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:59 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mon metadata", "id": "smithi174"}]: dispatch 2024-03-21T10:13:59.453 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:59 smithi174 ceph-mon[88004]: mon.smithi153 calling monitor election 2024-03-21T10:13:59.453 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:59 smithi174 ceph-mon[88004]: mon.smithi174 calling monitor election 2024-03-21T10:13:59.453 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:59 smithi174 ceph-mon[88004]: mon.smithi153 is new leader, mons smithi153,smithi174 in quorum (ranks 0,1) 2024-03-21T10:13:59.453 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:59 smithi174 ceph-mon[88004]: monmap e3: 2 mons at {smithi153=[v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0],smithi174=[v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-21T10:13:59.453 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:59 smithi174 ceph-mon[88004]: fsmap cephfs:2 {0=cephfs.smithi153.iuyluj=up:active,1=cephfs.smithi174.yyimst=up:active} 2 up:standby-replay 2024-03-21T10:13:59.453 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:59 smithi174 ceph-mon[88004]: osdmap e50: 6 total, 6 up, 6 in 2024-03-21T10:13:59.453 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:59 smithi174 ceph-mon[88004]: mgrmap e37: smithi153.tluxni(active, since 27s), standbys: smithi174.qpcwat 2024-03-21T10:13:59.453 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:59 smithi174 ceph-mon[88004]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-03-21T10:13:59.453 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:59 smithi174 ceph-mon[88004]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-21T10:13:59.453 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:59 smithi174 ceph-mon[88004]: fs cephfs has deprecated feature inline_data enabled. 2024-03-21T10:13:59.453 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:59 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:13:59.453 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:59 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:13:59.453 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:13:59 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:13:59.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:59 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mon metadata", "id": "smithi153"}]: dispatch 2024-03-21T10:13:59.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:59 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mon metadata", "id": "smithi174"}]: dispatch 2024-03-21T10:13:59.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:59 smithi153 ceph-mon[113733]: mon.smithi153 calling monitor election 2024-03-21T10:13:59.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:59 smithi153 ceph-mon[113733]: mon.smithi174 calling monitor election 2024-03-21T10:13:59.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:59 smithi153 ceph-mon[113733]: mon.smithi153 is new leader, mons smithi153,smithi174 in quorum (ranks 0,1) 2024-03-21T10:13:59.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:59 smithi153 ceph-mon[113733]: monmap e3: 2 mons at {smithi153=[v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0],smithi174=[v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-21T10:13:59.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:59 smithi153 ceph-mon[113733]: fsmap cephfs:2 {0=cephfs.smithi153.iuyluj=up:active,1=cephfs.smithi174.yyimst=up:active} 2 up:standby-replay 2024-03-21T10:13:59.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:59 smithi153 ceph-mon[113733]: osdmap e50: 6 total, 6 up, 6 in 2024-03-21T10:13:59.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:59 smithi153 ceph-mon[113733]: mgrmap e37: smithi153.tluxni(active, since 27s), standbys: smithi174.qpcwat 2024-03-21T10:13:59.558 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:59 smithi153 ceph-mon[113733]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-03-21T10:13:59.558 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:59 smithi153 ceph-mon[113733]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-21T10:13:59.558 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:59 smithi153 ceph-mon[113733]: fs cephfs has deprecated feature inline_data enabled. 2024-03-21T10:13:59.558 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:59 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:13:59.558 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:59 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:13:59.558 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:13:59 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:14:00.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:00 smithi174 ceph-mon[88004]: pgmap v17: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:14:00.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:00 smithi153 ceph-mon[113733]: pgmap v17: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:14:02.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:02 smithi174 ceph-mon[88004]: pgmap v18: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:14:02.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:02 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:14:02.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:02 smithi153 ceph-mon[113733]: pgmap v18: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:14:02.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:02 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:14:03.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:03 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:14:03.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:03 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:14:03.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:03 smithi174 ceph-mon[88004]: pgmap v19: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:14:04.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:03 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:14:04.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:03 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:14:04.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:03 smithi153 ceph-mon[113733]: pgmap v19: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:14:05.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:05 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:14:05.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:05 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:14:05.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:05 smithi174 ceph-mon[88004]: pgmap v20: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:14:06.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:05 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:14:06.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:05 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:14:06.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:05 smithi153 ceph-mon[113733]: pgmap v20: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:14:08.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:08 smithi174 ceph-mon[88004]: pgmap v21: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:14:08.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:08 smithi153 ceph-mon[113733]: pgmap v21: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:14:10.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:10 smithi174 ceph-mon[88004]: pgmap v22: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:14:10.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:10 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:14:10.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:10 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:14:10.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:10 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:14:10.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:10 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:14:10.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:10 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:14:10.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:10 smithi153 ceph-mon[113733]: pgmap v22: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:14:10.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:10 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:14:10.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:10 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:14:10.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:10 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:14:10.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:10 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:14:10.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:10 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:14:11.460 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:11 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T10:14:11.460 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:11 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-21T10:14:11.460 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:11 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:14:11.467 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:11 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T10:14:11.467 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:11 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-21T10:14:11.468 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:11 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:14:12.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:12 smithi174 ceph-mon[88004]: Reconfiguring mon.smithi153 (monmap changed)... 2024-03-21T10:14:12.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:12 smithi174 ceph-mon[88004]: Reconfiguring daemon mon.smithi153 on smithi153 2024-03-21T10:14:12.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:12 smithi174 ceph-mon[88004]: pgmap v23: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:14:12.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:12 smithi153 ceph-mon[113733]: Reconfiguring mon.smithi153 (monmap changed)... 2024-03-21T10:14:12.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:12 smithi153 ceph-mon[113733]: Reconfiguring daemon mon.smithi153 on smithi153 2024-03-21T10:14:12.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:12 smithi153 ceph-mon[113733]: pgmap v23: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:14:14.003 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:13 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:14:14.003 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:13 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:14:14.003 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:13 smithi153 ceph-mon[113733]: Reconfiguring mgr.smithi153.tluxni (monmap changed)... 2024-03-21T10:14:14.004 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:13 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi153.tluxni", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-21T10:14:14.004 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:13 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-21T10:14:14.004 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:13 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:14:14.004 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:13 smithi153 ceph-mon[113733]: Reconfiguring daemon mgr.smithi153.tluxni on smithi153 2024-03-21T10:14:14.004 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:13 smithi153 ceph-mon[113733]: pgmap v24: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:14:14.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:13 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:14:14.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:13 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:14:14.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:13 smithi174 ceph-mon[88004]: Reconfiguring mgr.smithi153.tluxni (monmap changed)... 2024-03-21T10:14:14.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:13 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi153.tluxni", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-21T10:14:14.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:13 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-21T10:14:14.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:13 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:14:14.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:13 smithi174 ceph-mon[88004]: Reconfiguring daemon mgr.smithi153.tluxni on smithi153 2024-03-21T10:14:14.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:13 smithi174 ceph-mon[88004]: pgmap v24: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:14:16.363 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:16 smithi153 ceph-mon[113733]: pgmap v25: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:14:16.363 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:16 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:14:16.363 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:16 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:14:16.363 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:16 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi153", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-21T10:14:16.363 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:16 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi153", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-21T10:14:16.364 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:16 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi153", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-03-21T10:14:16.364 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:16 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.ceph-exporter.smithi153"}]: dispatch 2024-03-21T10:14:16.364 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:16 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:14:16.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:16 smithi174 ceph-mon[88004]: pgmap v25: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:14:16.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:16 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:14:16.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:16 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:14:16.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:16 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi153", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-21T10:14:16.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:16 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi153", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-21T10:14:16.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:16 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi153", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-03-21T10:14:16.453 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:16 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.ceph-exporter.smithi153"}]: dispatch 2024-03-21T10:14:16.453 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:16 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:14:17.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:17 smithi153 ceph-mon[113733]: Reconfiguring ceph-exporter.smithi153 (monmap changed)... 2024-03-21T10:14:17.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:17 smithi153 ceph-mon[113733]: Unable to update caps for client.ceph-exporter.smithi153 2024-03-21T10:14:17.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:17 smithi153 ceph-mon[113733]: Reconfiguring daemon ceph-exporter.smithi153 on smithi153 2024-03-21T10:14:17.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:17 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:14:17.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:17 smithi174 ceph-mon[88004]: Reconfiguring ceph-exporter.smithi153 (monmap changed)... 2024-03-21T10:14:17.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:17 smithi174 ceph-mon[88004]: Unable to update caps for client.ceph-exporter.smithi153 2024-03-21T10:14:17.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:17 smithi174 ceph-mon[88004]: Reconfiguring daemon ceph-exporter.smithi153 on smithi153 2024-03-21T10:14:17.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:17 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:14:18.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:18 smithi153 ceph-mon[113733]: pgmap v26: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:14:18.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:18 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:14:18.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:18 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:14:18.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:18 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi153", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-21T10:14:18.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:18 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:14:18.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:18 smithi174 ceph-mon[88004]: pgmap v26: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:14:18.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:18 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:14:18.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:18 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:14:18.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:18 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi153", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-21T10:14:18.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:18 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:14:18.837 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:18.836+0000 7f7681c00700 1 -- 172.21.15.153:0/3286096847 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f767c102fe0 msgr2=0x7f767c1033c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:14:18.837 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:18.836+0000 7f7681c00700 1 --2- 172.21.15.153:0/3286096847 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f767c102fe0 0x7f767c1033c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f766c009910 tx=0x7f766c009c20 comp rx=0 tx=0).stop 2024-03-21T10:14:18.837 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:18.836+0000 7f7681c00700 1 -- 172.21.15.153:0/3286096847 shutdown_connections 2024-03-21T10:14:18.837 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:18.836+0000 7f7681c00700 1 --2- 172.21.15.153:0/3286096847 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f767c103990 0x7f767c1079e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:14:18.837 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:18.836+0000 7f7681c00700 1 --2- 172.21.15.153:0/3286096847 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f767c102fe0 0x7f767c1033c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:14:18.837 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:18.836+0000 7f7681c00700 1 -- 172.21.15.153:0/3286096847 >> 172.21.15.153:0/3286096847 conn(0x7f767c0fe8d0 msgr2=0x7f767c100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:14:18.837 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:18.836+0000 7f7681c00700 1 -- 172.21.15.153:0/3286096847 shutdown_connections 2024-03-21T10:14:18.838 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:18.836+0000 7f7681c00700 1 -- 172.21.15.153:0/3286096847 wait complete. 2024-03-21T10:14:18.839 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:18.837+0000 7f7681c00700 1 Processor -- start 2024-03-21T10:14:18.839 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:18.837+0000 7f7681c00700 1 -- start start 2024-03-21T10:14:18.839 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:18.837+0000 7f7681c00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f767c102fe0 0x7f767c19e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:14:18.839 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:18.837+0000 7f7681c00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f767c103990 0x7f767c19ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:14:18.839 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:18.837+0000 7f7681c00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f767c19f3d0 con 0x7f767c103990 2024-03-21T10:14:18.839 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:18.837+0000 7f7681c00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f767c198880 con 0x7f767c102fe0 2024-03-21T10:14:18.839 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:18.838+0000 7f767a000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f767c103990 0x7f767c19ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:14:18.839 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:18.838+0000 7f767a000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f767c103990 0x7f767c19ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:46164/0 (socket says 172.21.15.153:46164) 2024-03-21T10:14:18.839 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:18.838+0000 7f767a000700 1 -- 172.21.15.153:0/3285347765 learned_addr learned my addr 172.21.15.153:0/3285347765 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:14:18.840 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:18.838+0000 7f767aa00700 1 --2- 172.21.15.153:0/3285347765 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f767c102fe0 0x7f767c19e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:14:18.840 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:18.838+0000 7f767a000700 1 -- 172.21.15.153:0/3285347765 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f767c102fe0 msgr2=0x7f767c19e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:14:18.840 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:18.838+0000 7f767a000700 1 --2- 172.21.15.153:0/3285347765 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f767c102fe0 0x7f767c19e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:14:18.840 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:18.838+0000 7f767a000700 1 -- 172.21.15.153:0/3285347765 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f766c0095f0 con 0x7f767c103990 2024-03-21T10:14:18.840 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:18.839+0000 7f767a000700 1 --2- 172.21.15.153:0/3285347765 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f767c103990 0x7f767c19ed40 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f766800b480 tx=0x7f766800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:14:18.841 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:18.839+0000 7f7673400700 1 -- 172.21.15.153:0/3285347765 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f76680118c0 con 0x7f767c103990 2024-03-21T10:14:18.841 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:18.839+0000 7f7673400700 1 -- 172.21.15.153:0/3285347765 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7668010380 con 0x7f767c103990 2024-03-21T10:14:18.841 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:18.839+0000 7f7681c00700 1 -- 172.21.15.153:0/3285347765 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f767c198b60 con 0x7f767c103990 2024-03-21T10:14:18.841 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:18.839+0000 7f7673400700 1 -- 172.21.15.153:0/3285347765 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f766800f4f0 con 0x7f767c103990 2024-03-21T10:14:18.841 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:18.839+0000 7f7681c00700 1 -- 172.21.15.153:0/3285347765 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f767c1990b0 con 0x7f767c103990 2024-03-21T10:14:18.841 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:18.840+0000 7f7681c00700 1 -- 172.21.15.153:0/3285347765 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f767c10b380 con 0x7f767c103990 2024-03-21T10:14:18.841 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:18.841+0000 7f7673400700 1 -- 172.21.15.153:0/3285347765 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 37) v1 ==== 92321+0+0 (secure 0 0 0) 0x7f76680104f0 con 0x7f767c103990 2024-03-21T10:14:18.843 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:18.842+0000 7f7673400700 1 --2- 172.21.15.153:0/3285347765 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f766006e590 0x7f7660070a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:14:18.843 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:18.842+0000 7f7673400700 1 -- 172.21.15.153:0/3285347765 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f766808e100 con 0x7f767c103990 2024-03-21T10:14:18.843 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:18.842+0000 7f767aa00700 1 --2- 172.21.15.153:0/3285347765 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f766006e590 0x7f7660070a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:14:18.843 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:18.843+0000 7f767aa00700 1 --2- 172.21.15.153:0/3285347765 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f766006e590 0x7f7660070a50 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f766c006010 tx=0x7f766c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:14:18.848 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:18.848+0000 7f7673400700 1 -- 172.21.15.153:0/3285347765 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f766805ad60 con 0x7f767c103990 2024-03-21T10:14:19.102 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:19.101+0000 7f7681c00700 1 -- 172.21.15.153:0/3285347765 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f767c1993f0 con 0x7f766006e590 2024-03-21T10:14:19.103 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:19.103+0000 7f7673400700 1 -- 172.21.15.153:0/3285347765 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7f767c1993f0 con 0x7f766006e590 2024-03-21T10:14:19.106 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:19.106+0000 7f7681c00700 1 -- 172.21.15.153:0/3285347765 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f766006e590 msgr2=0x7f7660070a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:14:19.106 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:19.106+0000 7f7681c00700 1 --2- 172.21.15.153:0/3285347765 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f766006e590 0x7f7660070a50 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f766c006010 tx=0x7f766c01a040 comp rx=0 tx=0).stop 2024-03-21T10:14:19.106 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:19.106+0000 7f7681c00700 1 -- 172.21.15.153:0/3285347765 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f767c103990 msgr2=0x7f767c19ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:14:19.106 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:19.106+0000 7f7681c00700 1 --2- 172.21.15.153:0/3285347765 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f767c103990 0x7f767c19ed40 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f766800b480 tx=0x7f766800b840 comp rx=0 tx=0).stop 2024-03-21T10:14:19.107 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:19.106+0000 7f7681c00700 1 -- 172.21.15.153:0/3285347765 shutdown_connections 2024-03-21T10:14:19.107 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:19.106+0000 7f7681c00700 1 --2- 172.21.15.153:0/3285347765 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f767c102fe0 0x7f767c19e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:14:19.107 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:19.106+0000 7f7681c00700 1 --2- 172.21.15.153:0/3285347765 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f766006e590 0x7f7660070a50 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:14:19.107 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:19.106+0000 7f7681c00700 1 --2- 172.21.15.153:0/3285347765 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f767c103990 0x7f767c19ed40 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:14:19.107 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:19.106+0000 7f7681c00700 1 -- 172.21.15.153:0/3285347765 >> 172.21.15.153:0/3285347765 conn(0x7f767c0fe8d0 msgr2=0x7f767c0fff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:14:19.107 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:19.106+0000 7f7681c00700 1 -- 172.21.15.153:0/3285347765 shutdown_connections 2024-03-21T10:14:19.108 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:19.106+0000 7f7681c00700 1 -- 172.21.15.153:0/3285347765 wait complete. 2024-03-21T10:14:19.125 INFO:teuthology.orchestra.run.smithi153.stdout:true 2024-03-21T10:14:19.267 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:19.266+0000 7f0ec6a00700 1 -- 172.21.15.153:0/795665425 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0ec0101750 msgr2=0x7f0ec01057a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:14:19.267 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:19.266+0000 7f0ec6a00700 1 --2- 172.21.15.153:0/795665425 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0ec0101750 0x7f0ec01057a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0ea8009910 tx=0x7f0ea8009c20 comp rx=0 tx=0).stop 2024-03-21T10:14:19.267 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:19.266+0000 7f0ec6a00700 1 -- 172.21.15.153:0/795665425 shutdown_connections 2024-03-21T10:14:19.267 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:19.266+0000 7f0ec6a00700 1 --2- 172.21.15.153:0/795665425 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0ec0101750 0x7f0ec01057a0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:14:19.267 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:19.266+0000 7f0ec6a00700 1 --2- 172.21.15.153:0/795665425 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0ec0100da0 0x7f0ec0101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:14:19.268 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:19.266+0000 7f0ec6a00700 1 -- 172.21.15.153:0/795665425 >> 172.21.15.153:0/795665425 conn(0x7f0ec00fc630 msgr2=0x7f0ec00fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:14:19.268 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:19.266+0000 7f0ec6a00700 1 -- 172.21.15.153:0/795665425 shutdown_connections 2024-03-21T10:14:19.268 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:19.266+0000 7f0ec6a00700 1 -- 172.21.15.153:0/795665425 wait complete. 2024-03-21T10:14:19.271 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:19.267+0000 7f0ec6a00700 1 Processor -- start 2024-03-21T10:14:19.271 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:19.267+0000 7f0ec6a00700 1 -- start start 2024-03-21T10:14:19.271 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:19.268+0000 7f0ec6a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0ec0100da0 0x7f0ec019c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:14:19.271 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:19.268+0000 7f0ec6a00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0ec0101750 0x7f0ec019cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:14:19.271 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:19.268+0000 7f0ec6a00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec019d270 con 0x7f0ec0100da0 2024-03-21T10:14:19.271 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:19.268+0000 7f0ec6a00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec0196720 con 0x7f0ec0101750 2024-03-21T10:14:19.271 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:19.268+0000 7f0ebfe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0ec0100da0 0x7f0ec019c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:14:19.271 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:19.268+0000 7f0ebfe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0ec0100da0 0x7f0ec019c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:46194/0 (socket says 172.21.15.153:46194) 2024-03-21T10:14:19.271 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:19.268+0000 7f0ebfe00700 1 -- 172.21.15.153:0/511312720 learned_addr learned my addr 172.21.15.153:0/511312720 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:14:19.271 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:19.268+0000 7f0eb7e00700 1 --2- 172.21.15.153:0/511312720 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0ec0101750 0x7f0ec019cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:14:19.271 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:19.268+0000 7f0ebfe00700 1 -- 172.21.15.153:0/511312720 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0ec0101750 msgr2=0x7f0ec019cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:14:19.271 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:19.268+0000 7f0ebfe00700 1 --2- 172.21.15.153:0/511312720 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0ec0101750 0x7f0ec019cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:14:19.271 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:19.269+0000 7f0ebfe00700 1 -- 172.21.15.153:0/511312720 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ea80095f0 con 0x7f0ec0100da0 2024-03-21T10:14:19.272 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:19.269+0000 7f0ebfe00700 1 --2- 172.21.15.153:0/511312720 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0ec0100da0 0x7f0ec019c6a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0eb000ba50 tx=0x7f0eb000be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:14:19.272 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:19.269+0000 7f0ebd600700 1 -- 172.21.15.153:0/511312720 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0eb000ca00 con 0x7f0ec0100da0 2024-03-21T10:14:19.272 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:19.269+0000 7f0ebd600700 1 -- 172.21.15.153:0/511312720 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0eb000cb60 con 0x7f0ec0100da0 2024-03-21T10:14:19.272 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:19.269+0000 7f0ebd600700 1 -- 172.21.15.153:0/511312720 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0eb00075a0 con 0x7f0ec0100da0 2024-03-21T10:14:19.272 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:19.269+0000 7f0ec6a00700 1 -- 172.21.15.153:0/511312720 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ec0196a00 con 0x7f0ec0100da0 2024-03-21T10:14:19.272 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:19.269+0000 7f0ec6a00700 1 -- 172.21.15.153:0/511312720 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ec0196f00 con 0x7f0ec0100da0 2024-03-21T10:14:19.272 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:19.270+0000 7f0ec6a00700 1 -- 172.21.15.153:0/511312720 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ec01090f0 con 0x7f0ec0100da0 2024-03-21T10:14:19.272 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:19.272+0000 7f0ebd600700 1 -- 172.21.15.153:0/511312720 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 37) v1 ==== 92321+0+0 (secure 0 0 0) 0x7f0eb000ccd0 con 0x7f0ec0100da0 2024-03-21T10:14:19.273 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:19.272+0000 7f0ebd600700 1 --2- 172.21.15.153:0/511312720 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f0ea406e590 0x7f0ea4070a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:14:19.273 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:19.273+0000 7f0ebd600700 1 -- 172.21.15.153:0/511312720 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f0eb0006e40 con 0x7f0ec0100da0 2024-03-21T10:14:19.273 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:19.273+0000 7f0eb7e00700 1 --2- 172.21.15.153:0/511312720 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f0ea406e590 0x7f0ea4070a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:14:19.274 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:19.274+0000 7f0eb7e00700 1 --2- 172.21.15.153:0/511312720 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f0ea406e590 0x7f0ea4070a50 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f0ec0197b90 tx=0x7f0ea801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:14:19.278 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:19.278+0000 7f0ebd600700 1 -- 172.21.15.153:0/511312720 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0eb005a9b0 con 0x7f0ec0100da0 2024-03-21T10:14:19.407 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:19 smithi153 ceph-mon[113733]: Reconfiguring crash.smithi153 (monmap changed)... 2024-03-21T10:14:19.408 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:19 smithi153 ceph-mon[113733]: Reconfiguring daemon crash.smithi153 on smithi153 2024-03-21T10:14:19.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:19 smithi174 ceph-mon[88004]: Reconfiguring crash.smithi153 (monmap changed)... 2024-03-21T10:14:19.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:19 smithi174 ceph-mon[88004]: Reconfiguring daemon crash.smithi153 on smithi153 2024-03-21T10:14:19.523 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:19.523+0000 7f0ec6a00700 1 -- 172.21.15.153:0/511312720 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0ec0197290 con 0x7f0ea406e590 2024-03-21T10:14:19.526 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:19.526+0000 7f0ebd600700 1 -- 172.21.15.153:0/511312720 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7f0ec0197290 con 0x7f0ea406e590 2024-03-21T10:14:19.529 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:19.528+0000 7f0ec6a00700 1 -- 172.21.15.153:0/511312720 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f0ea406e590 msgr2=0x7f0ea4070a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:14:19.529 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:19.528+0000 7f0ec6a00700 1 --2- 172.21.15.153:0/511312720 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f0ea406e590 0x7f0ea4070a50 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f0ec0197b90 tx=0x7f0ea801a040 comp rx=0 tx=0).stop 2024-03-21T10:14:19.530 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:19.528+0000 7f0ec6a00700 1 -- 172.21.15.153:0/511312720 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0ec0100da0 msgr2=0x7f0ec019c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:14:19.530 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:19.528+0000 7f0ec6a00700 1 --2- 172.21.15.153:0/511312720 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0ec0100da0 0x7f0ec019c6a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0eb000ba50 tx=0x7f0eb000be10 comp rx=0 tx=0).stop 2024-03-21T10:14:19.530 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:19.529+0000 7f0ec6a00700 1 -- 172.21.15.153:0/511312720 shutdown_connections 2024-03-21T10:14:19.530 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:19.529+0000 7f0ec6a00700 1 --2- 172.21.15.153:0/511312720 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f0ea406e590 0x7f0ea4070a50 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:14:19.530 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:19.529+0000 7f0ec6a00700 1 --2- 172.21.15.153:0/511312720 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0ec0100da0 0x7f0ec019c6a0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:14:19.530 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:19.529+0000 7f0ec6a00700 1 --2- 172.21.15.153:0/511312720 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0ec0101750 0x7f0ec019cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:14:19.530 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:19.529+0000 7f0ec6a00700 1 -- 172.21.15.153:0/511312720 >> 172.21.15.153:0/511312720 conn(0x7f0ec00fc630 msgr2=0x7f0ec00fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:14:19.530 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:19.529+0000 7f0ec6a00700 1 -- 172.21.15.153:0/511312720 shutdown_connections 2024-03-21T10:14:19.530 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:19.529+0000 7f0ec6a00700 1 -- 172.21.15.153:0/511312720 wait complete. 2024-03-21T10:14:19.687 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:19.686+0000 7f6868800700 1 -- 172.21.15.153:0/1464445929 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6860073ae0 msgr2=0x7f686010ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:14:19.687 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:19.686+0000 7f6868800700 1 --2- 172.21.15.153:0/1464445929 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6860073ae0 0x7f686010ce40 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6850009910 tx=0x7f6850009c20 comp rx=0 tx=0).stop 2024-03-21T10:14:19.687 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:19.686+0000 7f6868800700 1 -- 172.21.15.153:0/1464445929 shutdown_connections 2024-03-21T10:14:19.687 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:19.686+0000 7f6868800700 1 --2- 172.21.15.153:0/1464445929 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6860073ae0 0x7f686010ce40 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:14:19.687 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:19.686+0000 7f6868800700 1 --2- 172.21.15.153:0/1464445929 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f68600731c0 0x7f68600735a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:14:19.687 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:19.686+0000 7f6868800700 1 -- 172.21.15.153:0/1464445929 >> 172.21.15.153:0/1464445929 conn(0x7f68600fc630 msgr2=0x7f68600fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:14:19.688 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:19.686+0000 7f6868800700 1 -- 172.21.15.153:0/1464445929 shutdown_connections 2024-03-21T10:14:19.688 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:19.686+0000 7f6868800700 1 -- 172.21.15.153:0/1464445929 wait complete. 2024-03-21T10:14:19.688 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:19.688+0000 7f6868800700 1 Processor -- start 2024-03-21T10:14:19.688 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:19.688+0000 7f6868800700 1 -- start start 2024-03-21T10:14:19.689 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:19.688+0000 7f6868800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f68600731c0 0x7f6860198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:14:19.689 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:19.688+0000 7f6868800700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6860073ae0 0x7f6860198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:14:19.689 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:19.688+0000 7f6868800700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6860199530 con 0x7f68600731c0 2024-03-21T10:14:19.689 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:19.688+0000 7f6868800700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f686019d340 con 0x7f6860073ae0 2024-03-21T10:14:19.689 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:19.688+0000 7f6865e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f68600731c0 0x7f6860198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:14:19.689 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:19.688+0000 7f6865e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f68600731c0 0x7f6860198960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:46212/0 (socket says 172.21.15.153:46212) 2024-03-21T10:14:19.690 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:19.688+0000 7f6865e00700 1 -- 172.21.15.153:0/3094041289 learned_addr learned my addr 172.21.15.153:0/3094041289 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:14:19.690 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:19.689+0000 7f6865400700 1 --2- 172.21.15.153:0/3094041289 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6860073ae0 0x7f6860198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:14:19.690 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:19.689+0000 7f6865e00700 1 -- 172.21.15.153:0/3094041289 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6860073ae0 msgr2=0x7f6860198ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:14:19.690 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:19.689+0000 7f6865e00700 1 --2- 172.21.15.153:0/3094041289 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6860073ae0 0x7f6860198ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:14:19.690 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:19.689+0000 7f6865e00700 1 -- 172.21.15.153:0/3094041289 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68500095f0 con 0x7f68600731c0 2024-03-21T10:14:19.691 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:19.689+0000 7f6865e00700 1 --2- 172.21.15.153:0/3094041289 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f68600731c0 0x7f6860198960 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f685400d8d0 tx=0x7f685400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:14:19.693 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:19.692+0000 7f685aa00700 1 -- 172.21.15.153:0/3094041289 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6854009940 con 0x7f68600731c0 2024-03-21T10:14:19.693 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:19.692+0000 7f6868800700 1 -- 172.21.15.153:0/3094041289 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f686019d620 con 0x7f68600731c0 2024-03-21T10:14:19.693 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:19.692+0000 7f685aa00700 1 -- 172.21.15.153:0/3094041289 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6854010460 con 0x7f68600731c0 2024-03-21T10:14:19.693 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:19.692+0000 7f685aa00700 1 -- 172.21.15.153:0/3094041289 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f685400f5f0 con 0x7f68600731c0 2024-03-21T10:14:19.693 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:19.692+0000 7f6868800700 1 -- 172.21.15.153:0/3094041289 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f686019db70 con 0x7f68600731c0 2024-03-21T10:14:19.694 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:19.694+0000 7f6868800700 1 -- 172.21.15.153:0/3094041289 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f686010a540 con 0x7f68600731c0 2024-03-21T10:14:19.695 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:19.695+0000 7f685aa00700 1 -- 172.21.15.153:0/3094041289 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 37) v1 ==== 92321+0+0 (secure 0 0 0) 0x7f68540105d0 con 0x7f68600731c0 2024-03-21T10:14:19.696 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:19.696+0000 7f685aa00700 1 --2- 172.21.15.153:0/3094041289 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f684806e590 0x7f6848070a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:14:19.696 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:19.696+0000 7f685aa00700 1 -- 172.21.15.153:0/3094041289 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f685408e6a0 con 0x7f68600731c0 2024-03-21T10:14:19.697 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:19.696+0000 7f6865400700 1 --2- 172.21.15.153:0/3094041289 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f684806e590 0x7f6848070a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:14:19.697 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:19.697+0000 7f6865400700 1 --2- 172.21.15.153:0/3094041289 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f684806e590 0x7f6848070a50 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f6860199f80 tx=0x7f685001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:14:19.701 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:19.701+0000 7f685aa00700 1 -- 172.21.15.153:0/3094041289 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f685405b170 con 0x7f68600731c0 2024-03-21T10:14:19.937 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:19.936+0000 7f6868800700 1 -- 172.21.15.153:0/3094041289 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f686019de00 con 0x7f684806e590 2024-03-21T10:14:19.949 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:19.948+0000 7f685aa00700 1 -- 172.21.15.153:0/3094041289 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f686019de00 con 0x7f684806e590 2024-03-21T10:14:19.949 INFO:teuthology.orchestra.run.smithi153.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-21T10:14:19.949 INFO:teuthology.orchestra.run.smithi153.stdout:alertmanager.smithi153 smithi153 *:9093,9094 running (3m) 41s ago 16m 14.5M - 0.25.0 c8568f914cd2 9ed3f2200b5f 2024-03-21T10:14:19.949 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi153 smithi153 running (17m) 41s ago 17m 8334k - 18.2.1 5be31c24972a 5b61994396a6 2024-03-21T10:14:19.949 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi174 smithi174 running (15m) 17s ago 15m 8266k - 18.2.1 5be31c24972a 4f7a5fc3ac28 2024-03-21T10:14:19.949 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi153 smithi153 running (17m) 41s ago 17m 6665k - 18.2.1 5be31c24972a 90a8b8ea03d1 2024-03-21T10:14:19.949 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi174 smithi174 running (15m) 17s ago 15m 6665k - 18.2.1 5be31c24972a a4c929e4dc77 2024-03-21T10:14:19.949 INFO:teuthology.orchestra.run.smithi153.stdout:grafana.smithi153 smithi153 *:3000 running (2m) 41s ago 16m 75.6M - 9.4.12 2bacad6d85d8 ee24026ab786 2024-03-21T10:14:19.949 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.iuyluj smithi153 running (9m) 41s ago 9m 128M - 18.2.1 5be31c24972a 02f667cee979 2024-03-21T10:14:19.950 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.zipckr smithi153 running (9m) 41s ago 9m 58.8M - 18.2.1 5be31c24972a 68a853929f8f 2024-03-21T10:14:19.950 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.nhozag smithi174 running (9m) 17s ago 9m 57.5M - 18.2.1 5be31c24972a 2fee4000308f 2024-03-21T10:14:19.950 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.yyimst smithi174 running (9m) 17s ago 9m 117M - 18.2.1 5be31c24972a d78a729220d1 2024-03-21T10:14:19.950 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi153.tluxni smithi153 *:8443,9283,8765 running (6m) 41s ago 18m 475M - 19.0.0-2231-g2edfee68 237e43c7a65f 7ffee0e55382 2024-03-21T10:14:19.950 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi174.qpcwat smithi174 *:8443,9283,8765 running (4m) 17s ago 15m 448M - 19.0.0-2231-g2edfee68 237e43c7a65f 748968f216ab 2024-03-21T10:14:19.950 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi153 smithi153 running (59s) 41s ago 19m 29.0M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f dd8038303dc3 2024-03-21T10:14:19.950 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi174 smithi174 running (23s) 17s ago 15m 20.4M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f 29d78a788f2c 2024-03-21T10:14:19.950 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi153 smithi153 *:9100 running (4m) 41s ago 17m 7814k - 1.5.0 0da6a335fe13 bb835edc0e0b 2024-03-21T10:14:19.950 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi174 smithi174 *:9100 running (4m) 17s ago 15m 7532k - 1.5.0 0da6a335fe13 7c4ecad790fa 2024-03-21T10:14:19.950 INFO:teuthology.orchestra.run.smithi153.stdout:osd.0 smithi153 running (14m) 41s ago 14m 216M 1217M 18.2.1 5be31c24972a 28912a2319b4 2024-03-21T10:14:19.950 INFO:teuthology.orchestra.run.smithi153.stdout:osd.1 smithi153 running (13m) 41s ago 13m 215M 1217M 18.2.1 5be31c24972a ca60681e2314 2024-03-21T10:14:19.951 INFO:teuthology.orchestra.run.smithi153.stdout:osd.2 smithi153 running (13m) 41s ago 13m 173M 1217M 18.2.1 5be31c24972a 8db799f8962a 2024-03-21T10:14:19.951 INFO:teuthology.orchestra.run.smithi153.stdout:osd.3 smithi174 running (12m) 17s ago 12m 232M 2241M 18.2.1 5be31c24972a dafc89e0658d 2024-03-21T10:14:19.951 INFO:teuthology.orchestra.run.smithi153.stdout:osd.4 smithi174 running (12m) 17s ago 12m 224M 2241M 18.2.1 5be31c24972a c4cd15701f05 2024-03-21T10:14:19.951 INFO:teuthology.orchestra.run.smithi153.stdout:osd.5 smithi174 running (11m) 17s ago 11m 202M 2241M 18.2.1 5be31c24972a 3b69886ce22e 2024-03-21T10:14:19.951 INFO:teuthology.orchestra.run.smithi153.stdout:prometheus.smithi153 smithi153 *:9095 running (3m) 41s ago 16m 67.9M - 2.43.0 a07b618ecd1d 7d95561e50c2 2024-03-21T10:14:19.952 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:19.952+0000 7f6868800700 1 -- 172.21.15.153:0/3094041289 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f684806e590 msgr2=0x7f6848070a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:14:19.953 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:19.952+0000 7f6868800700 1 --2- 172.21.15.153:0/3094041289 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f684806e590 0x7f6848070a50 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f6860199f80 tx=0x7f685001a040 comp rx=0 tx=0).stop 2024-03-21T10:14:19.953 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:19.952+0000 7f6868800700 1 -- 172.21.15.153:0/3094041289 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f68600731c0 msgr2=0x7f6860198960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:14:19.953 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:19.952+0000 7f6868800700 1 --2- 172.21.15.153:0/3094041289 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f68600731c0 0x7f6860198960 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f685400d8d0 tx=0x7f685400dc90 comp rx=0 tx=0).stop 2024-03-21T10:14:19.953 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:19.952+0000 7f6868800700 1 -- 172.21.15.153:0/3094041289 shutdown_connections 2024-03-21T10:14:19.953 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:19.952+0000 7f6868800700 1 --2- 172.21.15.153:0/3094041289 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f684806e590 0x7f6848070a50 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:14:19.953 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:19.952+0000 7f6868800700 1 --2- 172.21.15.153:0/3094041289 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f68600731c0 0x7f6860198960 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:14:19.954 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:19.952+0000 7f6868800700 1 --2- 172.21.15.153:0/3094041289 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6860073ae0 0x7f6860198ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:14:19.954 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:19.952+0000 7f6868800700 1 -- 172.21.15.153:0/3094041289 >> 172.21.15.153:0/3094041289 conn(0x7f68600fc630 msgr2=0x7f6860107680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:14:19.954 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:19.953+0000 7f6868800700 1 -- 172.21.15.153:0/3094041289 shutdown_connections 2024-03-21T10:14:19.954 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:19.953+0000 7f6868800700 1 -- 172.21.15.153:0/3094041289 wait complete. 2024-03-21T10:14:20.116 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:20.115+0000 7f8dd2e00700 1 -- 172.21.15.153:0/3524185691 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8dcc100da0 msgr2=0x7f8dcc101180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:14:20.116 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:20.115+0000 7f8dd2e00700 1 --2- 172.21.15.153:0/3524185691 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8dcc100da0 0x7f8dcc101180 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8dbc009910 tx=0x7f8dbc009c20 comp rx=0 tx=0).stop 2024-03-21T10:14:20.116 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:20.115+0000 7f8dd2e00700 1 -- 172.21.15.153:0/3524185691 shutdown_connections 2024-03-21T10:14:20.116 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:20.115+0000 7f8dd2e00700 1 --2- 172.21.15.153:0/3524185691 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8dcc101750 0x7f8dcc1057a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:14:20.116 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:20.115+0000 7f8dd2e00700 1 --2- 172.21.15.153:0/3524185691 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8dcc100da0 0x7f8dcc101180 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:14:20.116 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:20.115+0000 7f8dd2e00700 1 -- 172.21.15.153:0/3524185691 >> 172.21.15.153:0/3524185691 conn(0x7f8dcc0fc630 msgr2=0x7f8dcc0fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:14:20.116 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:20.115+0000 7f8dd2e00700 1 -- 172.21.15.153:0/3524185691 shutdown_connections 2024-03-21T10:14:20.117 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:20.115+0000 7f8dd2e00700 1 -- 172.21.15.153:0/3524185691 wait complete. 2024-03-21T10:14:20.117 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:20.116+0000 7f8dd2e00700 1 Processor -- start 2024-03-21T10:14:20.117 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:20.117+0000 7f8dd2e00700 1 -- start start 2024-03-21T10:14:20.118 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:20.117+0000 7f8dd2e00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8dcc100da0 0x7f8dcc073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:14:20.118 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:20.117+0000 7f8dd2e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8dcc101750 0x7f8dcc0735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:14:20.118 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:20.117+0000 7f8dd2e00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dcc073b10 con 0x7f8dcc101750 2024-03-21T10:14:20.118 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:20.117+0000 7f8dd2e00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dcc073c50 con 0x7f8dcc100da0 2024-03-21T10:14:20.119 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:20.117+0000 7f8dcb400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8dcc101750 0x7f8dcc0735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:14:20.119 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:20.117+0000 7f8dcb400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8dcc101750 0x7f8dcc0735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:46244/0 (socket says 172.21.15.153:46244) 2024-03-21T10:14:20.119 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:20.117+0000 7f8dcb400700 1 -- 172.21.15.153:0/930924834 learned_addr learned my addr 172.21.15.153:0/930924834 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:14:20.119 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:20.118+0000 7f8dcbe00700 1 --2- 172.21.15.153:0/930924834 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8dcc100da0 0x7f8dcc073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:14:20.119 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:20.118+0000 7f8dcb400700 1 -- 172.21.15.153:0/930924834 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8dcc100da0 msgr2=0x7f8dcc073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:14:20.120 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:20.118+0000 7f8dcb400700 1 --2- 172.21.15.153:0/930924834 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8dcc100da0 0x7f8dcc073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:14:20.120 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:20.118+0000 7f8dcb400700 1 -- 172.21.15.153:0/930924834 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dbc0095f0 con 0x7f8dcc101750 2024-03-21T10:14:20.120 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:20.118+0000 7f8dcb400700 1 --2- 172.21.15.153:0/930924834 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8dcc101750 0x7f8dcc0735d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8db800d900 tx=0x7f8db800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:14:20.120 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:20.118+0000 7f8dc8c00700 1 -- 172.21.15.153:0/930924834 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8db8009900 con 0x7f8dcc101750 2024-03-21T10:14:20.120 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:20.118+0000 7f8dc8c00700 1 -- 172.21.15.153:0/930924834 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8db8010460 con 0x7f8dcc101750 2024-03-21T10:14:20.120 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:20.118+0000 7f8dc8c00700 1 -- 172.21.15.153:0/930924834 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8db800f5f0 con 0x7f8dcc101750 2024-03-21T10:14:20.120 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:20.118+0000 7f8dd2e00700 1 -- 172.21.15.153:0/930924834 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dcc1a29a0 con 0x7f8dcc101750 2024-03-21T10:14:20.120 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:20.119+0000 7f8dd2e00700 1 -- 172.21.15.153:0/930924834 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dcc1a2db0 con 0x7f8dcc101750 2024-03-21T10:14:20.121 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:20.120+0000 7f8dd2e00700 1 -- 172.21.15.153:0/930924834 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8dcc109140 con 0x7f8dcc101750 2024-03-21T10:14:20.121 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:20.121+0000 7f8dc8c00700 1 -- 172.21.15.153:0/930924834 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 37) v1 ==== 92321+0+0 (secure 0 0 0) 0x7f8db80105d0 con 0x7f8dcc101750 2024-03-21T10:14:20.122 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:20.122+0000 7f8dc8c00700 1 --2- 172.21.15.153:0/930924834 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f8db006e590 0x7f8db0070a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:14:20.122 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:20.122+0000 7f8dc8c00700 1 -- 172.21.15.153:0/930924834 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f8db808e3b0 con 0x7f8dcc101750 2024-03-21T10:14:20.123 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:20.122+0000 7f8dcbe00700 1 --2- 172.21.15.153:0/930924834 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f8db006e590 0x7f8db0070a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:14:20.123 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:20.123+0000 7f8dcbe00700 1 --2- 172.21.15.153:0/930924834 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f8db006e590 0x7f8db0070a50 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f8dbc006010 tx=0x7f8dbc01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:14:20.131 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:20.131+0000 7f8dc8c00700 1 -- 172.21.15.153:0/930924834 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f8db8057470 con 0x7f8dcc101750 2024-03-21T10:14:20.199 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:20 smithi153 ceph-mon[113733]: from='client.34158 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:14:20.200 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:20 smithi153 ceph-mon[113733]: pgmap v27: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:14:20.200 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:20 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:14:20.200 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:20 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:14:20.200 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:20 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-21T10:14:20.200 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:20 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:14:20.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:20 smithi174 ceph-mon[88004]: from='client.34158 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:14:20.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:20 smithi174 ceph-mon[88004]: pgmap v27: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:14:20.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:20 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:14:20.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:20 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:14:20.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:20 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-21T10:14:20.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:20 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:14:20.474 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:20.473+0000 7f8dd2e00700 1 -- 172.21.15.153:0/930924834 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f8dcc04ea90 con 0x7f8dcc101750 2024-03-21T10:14:20.475 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:20.474+0000 7f8dc8c00700 1 -- 172.21.15.153:0/930924834 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+688 (secure 0 0 0) 0x7f8db805ab30 con 0x7f8dcc101750 2024-03-21T10:14:20.475 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:14:20.476 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": { 2024-03-21T10:14:20.476 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:14:20.476 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:14:20.476 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": { 2024-03-21T10:14:20.476 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:14:20.476 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:14:20.476 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": { 2024-03-21T10:14:20.476 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-03-21T10:14:20.476 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:14:20.476 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": { 2024-03-21T10:14:20.476 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-03-21T10:14:20.476 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:14:20.476 INFO:teuthology.orchestra.run.smithi153.stdout: "overall": { 2024-03-21T10:14:20.476 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 10, 2024-03-21T10:14:20.477 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 4 2024-03-21T10:14:20.477 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-03-21T10:14:20.477 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:14:20.478 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:20.477+0000 7f8dd2e00700 1 -- 172.21.15.153:0/930924834 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f8db006e590 msgr2=0x7f8db0070a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:14:20.478 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:20.477+0000 7f8dd2e00700 1 --2- 172.21.15.153:0/930924834 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f8db006e590 0x7f8db0070a50 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f8dbc006010 tx=0x7f8dbc01a040 comp rx=0 tx=0).stop 2024-03-21T10:14:20.478 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:20.477+0000 7f8dd2e00700 1 -- 172.21.15.153:0/930924834 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8dcc101750 msgr2=0x7f8dcc0735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:14:20.478 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:20.477+0000 7f8dd2e00700 1 --2- 172.21.15.153:0/930924834 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8dcc101750 0x7f8dcc0735d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8db800d900 tx=0x7f8db800dcc0 comp rx=0 tx=0).stop 2024-03-21T10:14:20.478 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:20.477+0000 7f8dd2e00700 1 -- 172.21.15.153:0/930924834 shutdown_connections 2024-03-21T10:14:20.478 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:20.477+0000 7f8dd2e00700 1 --2- 172.21.15.153:0/930924834 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8dcc100da0 0x7f8dcc073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:14:20.479 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:20.477+0000 7f8dd2e00700 1 --2- 172.21.15.153:0/930924834 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f8db006e590 0x7f8db0070a50 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:14:20.479 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:20.477+0000 7f8dd2e00700 1 --2- 172.21.15.153:0/930924834 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8dcc101750 0x7f8dcc0735d0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:14:20.479 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:20.477+0000 7f8dd2e00700 1 -- 172.21.15.153:0/930924834 >> 172.21.15.153:0/930924834 conn(0x7f8dcc0fc630 msgr2=0x7f8dcc0fdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:14:20.479 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:20.478+0000 7f8dd2e00700 1 -- 172.21.15.153:0/930924834 shutdown_connections 2024-03-21T10:14:20.479 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:20.478+0000 7f8dd2e00700 1 -- 172.21.15.153:0/930924834 wait complete. 2024-03-21T10:14:20.631 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:20.629+0000 7f6d6d800700 1 -- 172.21.15.153:0/1327513348 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6d68102ff0 msgr2=0x7f6d681033d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:14:20.631 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:20.629+0000 7f6d6d800700 1 --2- 172.21.15.153:0/1327513348 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6d68102ff0 0x7f6d681033d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6d58009910 tx=0x7f6d58009c20 comp rx=0 tx=0).stop 2024-03-21T10:14:20.631 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:20.629+0000 7f6d6d800700 1 -- 172.21.15.153:0/1327513348 shutdown_connections 2024-03-21T10:14:20.631 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:20.629+0000 7f6d6d800700 1 --2- 172.21.15.153:0/1327513348 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6d681039a0 0x7f6d681079f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:14:20.631 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:20.629+0000 7f6d6d800700 1 --2- 172.21.15.153:0/1327513348 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6d68102ff0 0x7f6d681033d0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:14:20.631 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:20.629+0000 7f6d6d800700 1 -- 172.21.15.153:0/1327513348 >> 172.21.15.153:0/1327513348 conn(0x7f6d680fe880 msgr2=0x7f6d68100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:14:20.632 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:20.629+0000 7f6d6d800700 1 -- 172.21.15.153:0/1327513348 shutdown_connections 2024-03-21T10:14:20.632 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:20.629+0000 7f6d6d800700 1 -- 172.21.15.153:0/1327513348 wait complete. 2024-03-21T10:14:20.632 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:20.630+0000 7f6d6d800700 1 Processor -- start 2024-03-21T10:14:20.634 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:20.630+0000 7f6d6d800700 1 -- start start 2024-03-21T10:14:20.634 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:20.631+0000 7f6d6d800700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6d68102ff0 0x7f6d680752a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:14:20.635 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:20.631+0000 7f6d6d800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6d681039a0 0x7f6d680757e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:14:20.635 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:20.631+0000 7f6d6d800700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d68079430 con 0x7f6d681039a0 2024-03-21T10:14:20.635 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:20.631+0000 7f6d6d800700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d68075d20 con 0x7f6d68102ff0 2024-03-21T10:14:20.635 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:20.631+0000 7f6d66000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6d681039a0 0x7f6d680757e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:14:20.636 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:20.631+0000 7f6d66000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6d681039a0 0x7f6d680757e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:46266/0 (socket says 172.21.15.153:46266) 2024-03-21T10:14:20.636 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:20.631+0000 7f6d66000700 1 -- 172.21.15.153:0/501006586 learned_addr learned my addr 172.21.15.153:0/501006586 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:14:20.636 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:20.632+0000 7f6d66a00700 1 --2- 172.21.15.153:0/501006586 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6d68102ff0 0x7f6d680752a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:14:20.636 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:20.632+0000 7f6d66000700 1 -- 172.21.15.153:0/501006586 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6d68102ff0 msgr2=0x7f6d680752a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:14:20.636 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:20.632+0000 7f6d66000700 1 --2- 172.21.15.153:0/501006586 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6d68102ff0 0x7f6d680752a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:14:20.636 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:20.632+0000 7f6d66000700 1 -- 172.21.15.153:0/501006586 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d580095f0 con 0x7f6d681039a0 2024-03-21T10:14:20.636 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:20.632+0000 7f6d66000700 1 --2- 172.21.15.153:0/501006586 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6d681039a0 0x7f6d680757e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6d5400b480 tx=0x7f6d5400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:14:20.636 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:20.632+0000 7f6d5f400700 1 -- 172.21.15.153:0/501006586 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6d540118c0 con 0x7f6d681039a0 2024-03-21T10:14:20.637 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:20.632+0000 7f6d5f400700 1 -- 172.21.15.153:0/501006586 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6d54010380 con 0x7f6d681039a0 2024-03-21T10:14:20.637 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:20.632+0000 7f6d6d800700 1 -- 172.21.15.153:0/501006586 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d68076000 con 0x7f6d681039a0 2024-03-21T10:14:20.637 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:20.632+0000 7f6d5f400700 1 -- 172.21.15.153:0/501006586 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6d5400f4f0 con 0x7f6d681039a0 2024-03-21T10:14:20.637 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:20.632+0000 7f6d6d800700 1 -- 172.21.15.153:0/501006586 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d681a7280 con 0x7f6d681039a0 2024-03-21T10:14:20.637 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:20.633+0000 7f6d6d800700 1 -- 172.21.15.153:0/501006586 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d6810b390 con 0x7f6d681039a0 2024-03-21T10:14:20.638 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:20.634+0000 7f6d5f400700 1 -- 172.21.15.153:0/501006586 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 37) v1 ==== 92321+0+0 (secure 0 0 0) 0x7f6d540104f0 con 0x7f6d681039a0 2024-03-21T10:14:20.639 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:20.635+0000 7f6d5f400700 1 --2- 172.21.15.153:0/501006586 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f6d4c06e590 0x7f6d4c070a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:14:20.639 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:20.635+0000 7f6d5f400700 1 -- 172.21.15.153:0/501006586 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f6d5408e0b0 con 0x7f6d681039a0 2024-03-21T10:14:20.639 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:20.636+0000 7f6d66a00700 1 --2- 172.21.15.153:0/501006586 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f6d4c06e590 0x7f6d4c070a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:14:20.639 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:20.636+0000 7f6d66a00700 1 --2- 172.21.15.153:0/501006586 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f6d4c06e590 0x7f6d4c070a50 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f6d58006010 tx=0x7f6d5801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:14:20.641 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:20.641+0000 7f6d5f400700 1 -- 172.21.15.153:0/501006586 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f6d54057200 con 0x7f6d681039a0 2024-03-21T10:14:20.914 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:20.914+0000 7f6d6d800700 1 -- 172.21.15.153:0/501006586 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f6d6804ea90 con 0x7f6d681039a0 2024-03-21T10:14:20.915 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:20.914+0000 7f6d5f400700 1 -- 172.21.15.153:0/501006586 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 21 v21) v1 ==== 76+0+1903 (secure 0 0 0) 0x7f6d5405a8c0 con 0x7f6d681039a0 2024-03-21T10:14:20.915 INFO:teuthology.orchestra.run.smithi153.stdout:e21 2024-03-21T10:14:20.915 INFO:teuthology.orchestra.run.smithi153.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-21T10:14:20.915 INFO:teuthology.orchestra.run.smithi153.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:14:20.915 INFO:teuthology.orchestra.run.smithi153.stdout:legacy client fscid: 1 2024-03-21T10:14:20.915 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:14:20.916 INFO:teuthology.orchestra.run.smithi153.stdout:Filesystem 'cephfs' (1) 2024-03-21T10:14:20.916 INFO:teuthology.orchestra.run.smithi153.stdout:fs_name cephfs 2024-03-21T10:14:20.916 INFO:teuthology.orchestra.run.smithi153.stdout:epoch 21 2024-03-21T10:14:20.916 INFO:teuthology.orchestra.run.smithi153.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-21T10:14:20.916 INFO:teuthology.orchestra.run.smithi153.stdout:created 2024-03-21T10:04:27.086576+0000 2024-03-21T10:14:20.916 INFO:teuthology.orchestra.run.smithi153.stdout:modified 2024-03-21T10:13:01.455594+0000 2024-03-21T10:14:20.916 INFO:teuthology.orchestra.run.smithi153.stdout:tableserver 0 2024-03-21T10:14:20.916 INFO:teuthology.orchestra.run.smithi153.stdout:root 0 2024-03-21T10:14:20.916 INFO:teuthology.orchestra.run.smithi153.stdout:session_timeout 60 2024-03-21T10:14:20.916 INFO:teuthology.orchestra.run.smithi153.stdout:session_autoclose 300 2024-03-21T10:14:20.916 INFO:teuthology.orchestra.run.smithi153.stdout:max_file_size 1099511627776 2024-03-21T10:14:20.916 INFO:teuthology.orchestra.run.smithi153.stdout:max_xattr_size 65536 2024-03-21T10:14:20.916 INFO:teuthology.orchestra.run.smithi153.stdout:required_client_features {} 2024-03-21T10:14:20.916 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure 0 2024-03-21T10:14:20.916 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure_osd_epoch 47 2024-03-21T10:14:20.917 INFO:teuthology.orchestra.run.smithi153.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:14:20.917 INFO:teuthology.orchestra.run.smithi153.stdout:max_mds 2 2024-03-21T10:14:20.917 INFO:teuthology.orchestra.run.smithi153.stdout:in 0,1 2024-03-21T10:14:20.917 INFO:teuthology.orchestra.run.smithi153.stdout:up {0=14484,1=24273} 2024-03-21T10:14:20.917 INFO:teuthology.orchestra.run.smithi153.stdout:failed 2024-03-21T10:14:20.917 INFO:teuthology.orchestra.run.smithi153.stdout:damaged 2024-03-21T10:14:20.917 INFO:teuthology.orchestra.run.smithi153.stdout:stopped 2024-03-21T10:14:20.917 INFO:teuthology.orchestra.run.smithi153.stdout:data_pools [3] 2024-03-21T10:14:20.917 INFO:teuthology.orchestra.run.smithi153.stdout:metadata_pool 2 2024-03-21T10:14:20.917 INFO:teuthology.orchestra.run.smithi153.stdout:inline_data enabled 2024-03-21T10:14:20.917 INFO:teuthology.orchestra.run.smithi153.stdout:balancer 2024-03-21T10:14:20.917 INFO:teuthology.orchestra.run.smithi153.stdout:bal_rank_mask -1 2024-03-21T10:14:20.917 INFO:teuthology.orchestra.run.smithi153.stdout:standby_count_wanted 1 2024-03-21T10:14:20.917 INFO:teuthology.orchestra.run.smithi153.stdout:qdb_cluster leader: 0 members: 2024-03-21T10:14:20.917 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.iuyluj{0:14484} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.153:6828/866138999,v1:172.21.15.153:6829/866138999] compat {c=[1],r=[1],i=[7ff]}] 2024-03-21T10:14:20.918 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.zipckr{0:14494} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.153:6826/4033622615,v1:172.21.15.153:6827/4033622615] compat {c=[1],r=[1],i=[7ff]}] 2024-03-21T10:14:20.918 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.yyimst{1:24273} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.174:6824/1776947618,v1:172.21.15.174:6825/1776947618] compat {c=[1],r=[1],i=[7ff]}] 2024-03-21T10:14:20.918 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.nhozag{1:24285} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.174:6826/806095873,v1:172.21.15.174:6827/806095873] compat {c=[1],r=[1],i=[7ff]}] 2024-03-21T10:14:20.918 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:14:20.918 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:14:20.919 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:20.917+0000 7f6d6d800700 1 -- 172.21.15.153:0/501006586 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f6d4c06e590 msgr2=0x7f6d4c070a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:14:20.919 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:20.917+0000 7f6d6d800700 1 --2- 172.21.15.153:0/501006586 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f6d4c06e590 0x7f6d4c070a50 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f6d58006010 tx=0x7f6d5801a040 comp rx=0 tx=0).stop 2024-03-21T10:14:20.919 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:20.917+0000 7f6d6d800700 1 -- 172.21.15.153:0/501006586 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6d681039a0 msgr2=0x7f6d680757e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:14:20.919 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:20.917+0000 7f6d6d800700 1 --2- 172.21.15.153:0/501006586 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6d681039a0 0x7f6d680757e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6d5400b480 tx=0x7f6d5400b840 comp rx=0 tx=0).stop 2024-03-21T10:14:20.919 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:20.917+0000 7f6d6d800700 1 -- 172.21.15.153:0/501006586 shutdown_connections 2024-03-21T10:14:20.919 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:20.917+0000 7f6d6d800700 1 --2- 172.21.15.153:0/501006586 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6d68102ff0 0x7f6d680752a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:14:20.919 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:20.917+0000 7f6d6d800700 1 --2- 172.21.15.153:0/501006586 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f6d4c06e590 0x7f6d4c070a50 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:14:20.919 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:20.917+0000 7f6d6d800700 1 --2- 172.21.15.153:0/501006586 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6d681039a0 0x7f6d680757e0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:14:20.919 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:20.917+0000 7f6d6d800700 1 -- 172.21.15.153:0/501006586 >> 172.21.15.153:0/501006586 conn(0x7f6d680fe880 msgr2=0x7f6d68100000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:14:20.919 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:20.917+0000 7f6d6d800700 1 -- 172.21.15.153:0/501006586 shutdown_connections 2024-03-21T10:14:20.919 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:20.918+0000 7f6d6d800700 1 -- 172.21.15.153:0/501006586 wait complete. 2024-03-21T10:14:20.920 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 21 2024-03-21T10:14:21.064 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:21.063+0000 7f3c2c000700 1 -- 172.21.15.153:0/670410897 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c24101750 msgr2=0x7f3c241057a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:14:21.064 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:21.063+0000 7f3c2c000700 1 --2- 172.21.15.153:0/670410897 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c24101750 0x7f3c241057a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3c10009910 tx=0x7f3c10009c20 comp rx=0 tx=0).stop 2024-03-21T10:14:21.064 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:21.063+0000 7f3c2c000700 1 -- 172.21.15.153:0/670410897 shutdown_connections 2024-03-21T10:14:21.064 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:21.063+0000 7f3c2c000700 1 --2- 172.21.15.153:0/670410897 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c24101750 0x7f3c241057a0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:14:21.065 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:21.063+0000 7f3c2c000700 1 --2- 172.21.15.153:0/670410897 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c24100da0 0x7f3c24101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:14:21.065 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:21.063+0000 7f3c2c000700 1 -- 172.21.15.153:0/670410897 >> 172.21.15.153:0/670410897 conn(0x7f3c240fc630 msgr2=0x7f3c240fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:14:21.065 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:21.063+0000 7f3c2c000700 1 -- 172.21.15.153:0/670410897 shutdown_connections 2024-03-21T10:14:21.065 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:21.063+0000 7f3c2c000700 1 -- 172.21.15.153:0/670410897 wait complete. 2024-03-21T10:14:21.066 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:21.064+0000 7f3c2c000700 1 Processor -- start 2024-03-21T10:14:21.066 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:21.064+0000 7f3c2c000700 1 -- start start 2024-03-21T10:14:21.066 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:21.065+0000 7f3c2c000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c24100da0 0x7f3c2419c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:14:21.066 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:21.065+0000 7f3c2c000700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c24101750 0x7f3c2419ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:14:21.067 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:21.065+0000 7f3c2c000700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c2419d0d0 con 0x7f3c24100da0 2024-03-21T10:14:21.067 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:21.065+0000 7f3c2c000700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c24196580 con 0x7f3c24101750 2024-03-21T10:14:21.067 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:21.065+0000 7f3c29600700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c24100da0 0x7f3c2419c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:14:21.067 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:21.065+0000 7f3c29600700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c24100da0 0x7f3c2419c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:46296/0 (socket says 172.21.15.153:46296) 2024-03-21T10:14:21.067 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:21.065+0000 7f3c29600700 1 -- 172.21.15.153:0/2833681112 learned_addr learned my addr 172.21.15.153:0/2833681112 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:14:21.067 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:21.065+0000 7f3c28c00700 1 --2- 172.21.15.153:0/2833681112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c24101750 0x7f3c2419ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:14:21.067 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:21.065+0000 7f3c29600700 1 -- 172.21.15.153:0/2833681112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c24101750 msgr2=0x7f3c2419ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:14:21.067 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:21.065+0000 7f3c29600700 1 --2- 172.21.15.153:0/2833681112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c24101750 0x7f3c2419ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:14:21.067 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:21.065+0000 7f3c29600700 1 -- 172.21.15.153:0/2833681112 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c100095f0 con 0x7f3c24100da0 2024-03-21T10:14:21.067 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:21.066+0000 7f3c29600700 1 --2- 172.21.15.153:0/2833681112 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c24100da0 0x7f3c2419c500 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3c1800c8a0 tx=0x7f3c1800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:14:21.068 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:21.066+0000 7f3c22000700 1 -- 172.21.15.153:0/2833681112 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3c1800cea0 con 0x7f3c24100da0 2024-03-21T10:14:21.068 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:21.066+0000 7f3c22000700 1 -- 172.21.15.153:0/2833681112 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3c18004830 con 0x7f3c24100da0 2024-03-21T10:14:21.068 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:21.066+0000 7f3c22000700 1 -- 172.21.15.153:0/2833681112 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3c180055b0 con 0x7f3c24100da0 2024-03-21T10:14:21.068 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:21.066+0000 7f3c2c000700 1 -- 172.21.15.153:0/2833681112 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c24196860 con 0x7f3c24100da0 2024-03-21T10:14:21.068 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:21.066+0000 7f3c2c000700 1 -- 172.21.15.153:0/2833681112 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c24196db0 con 0x7f3c24100da0 2024-03-21T10:14:21.068 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:21.068+0000 7f3c2c000700 1 -- 172.21.15.153:0/2833681112 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c241091a0 con 0x7f3c24100da0 2024-03-21T10:14:21.069 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:21.068+0000 7f3c22000700 1 -- 172.21.15.153:0/2833681112 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 37) v1 ==== 92321+0+0 (secure 0 0 0) 0x7f3c180049a0 con 0x7f3c24100da0 2024-03-21T10:14:21.070 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:21.069+0000 7f3c22000700 1 --2- 172.21.15.153:0/2833681112 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f3c0c06e590 0x7f3c0c070a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:14:21.070 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:21.070+0000 7f3c22000700 1 -- 172.21.15.153:0/2833681112 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f3c1808e1a0 con 0x7f3c24100da0 2024-03-21T10:14:21.070 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:21.070+0000 7f3c28c00700 1 --2- 172.21.15.153:0/2833681112 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f3c0c06e590 0x7f3c0c070a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:14:21.071 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:21.070+0000 7f3c28c00700 1 --2- 172.21.15.153:0/2833681112 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f3c0c06e590 0x7f3c0c070a50 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f3c240fce50 tx=0x7f3c1001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:14:21.075 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:21.074+0000 7f3c22000700 1 -- 172.21.15.153:0/2833681112 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f3c1805ad90 con 0x7f3c24100da0 2024-03-21T10:14:21.312 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:21.311+0000 7f3c2c000700 1 -- 172.21.15.153:0/2833681112 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3c24197100 con 0x7f3c0c06e590 2024-03-21T10:14:21.314 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:21.313+0000 7f3c22000700 1 -- 172.21.15.153:0/2833681112 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7f3c24197100 con 0x7f3c0c06e590 2024-03-21T10:14:21.314 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:21 smithi153 ceph-mon[113733]: from='client.34162 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:14:21.314 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:21 smithi153 ceph-mon[113733]: Reconfiguring osd.0 (monmap changed)... 2024-03-21T10:14:21.314 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:21 smithi153 ceph-mon[113733]: Reconfiguring daemon osd.0 on smithi153 2024-03-21T10:14:21.314 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:21 smithi153 ceph-mon[113733]: from='client.34166 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:14:21.314 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:21 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/930924834' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:14:21.315 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:21 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/501006586' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:14:21.315 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:14:21.315 INFO:teuthology.orchestra.run.smithi153.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232", 2024-03-21T10:14:21.315 INFO:teuthology.orchestra.run.smithi153.stdout: "in_progress": true, 2024-03-21T10:14:21.315 INFO:teuthology.orchestra.run.smithi153.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-21T10:14:21.315 INFO:teuthology.orchestra.run.smithi153.stdout: "services_complete": [ 2024-03-21T10:14:21.315 INFO:teuthology.orchestra.run.smithi153.stdout: "mon", 2024-03-21T10:14:21.315 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr" 2024-03-21T10:14:21.315 INFO:teuthology.orchestra.run.smithi153.stdout: ], 2024-03-21T10:14:21.315 INFO:teuthology.orchestra.run.smithi153.stdout: "progress": "4/23 daemons upgraded", 2024-03-21T10:14:21.315 INFO:teuthology.orchestra.run.smithi153.stdout: "message": "Currently upgrading mon daemons", 2024-03-21T10:14:21.315 INFO:teuthology.orchestra.run.smithi153.stdout: "is_paused": false 2024-03-21T10:14:21.316 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:14:21.316 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:21.316+0000 7f3c2c000700 1 -- 172.21.15.153:0/2833681112 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f3c0c06e590 msgr2=0x7f3c0c070a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:14:21.316 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:21.316+0000 7f3c2c000700 1 --2- 172.21.15.153:0/2833681112 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f3c0c06e590 0x7f3c0c070a50 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f3c240fce50 tx=0x7f3c1001a040 comp rx=0 tx=0).stop 2024-03-21T10:14:21.316 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:21.316+0000 7f3c2c000700 1 -- 172.21.15.153:0/2833681112 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c24100da0 msgr2=0x7f3c2419c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:14:21.317 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:21.316+0000 7f3c2c000700 1 --2- 172.21.15.153:0/2833681112 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c24100da0 0x7f3c2419c500 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3c1800c8a0 tx=0x7f3c1800cc60 comp rx=0 tx=0).stop 2024-03-21T10:14:21.317 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:21.316+0000 7f3c2c000700 1 -- 172.21.15.153:0/2833681112 shutdown_connections 2024-03-21T10:14:21.317 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:21.316+0000 7f3c2c000700 1 --2- 172.21.15.153:0/2833681112 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f3c0c06e590 0x7f3c0c070a50 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:14:21.317 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:21.316+0000 7f3c2c000700 1 --2- 172.21.15.153:0/2833681112 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c24100da0 0x7f3c2419c500 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:14:21.317 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:21.316+0000 7f3c2c000700 1 --2- 172.21.15.153:0/2833681112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c24101750 0x7f3c2419ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:14:21.317 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:21.316+0000 7f3c2c000700 1 -- 172.21.15.153:0/2833681112 >> 172.21.15.153:0/2833681112 conn(0x7f3c240fc630 msgr2=0x7f3c240fea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:14:21.317 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:21.316+0000 7f3c2c000700 1 -- 172.21.15.153:0/2833681112 shutdown_connections 2024-03-21T10:14:21.317 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:21.316+0000 7f3c2c000700 1 -- 172.21.15.153:0/2833681112 wait complete. 2024-03-21T10:14:21.460 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:21.459+0000 7fd9cb800700 1 -- 172.21.15.153:0/69776147 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd9c4100dc0 msgr2=0x7fd9c41011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:14:21.460 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:21.459+0000 7fd9cb800700 1 --2- 172.21.15.153:0/69776147 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd9c4100dc0 0x7fd9c41011a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd9b8009910 tx=0x7fd9b8009c20 comp rx=0 tx=0).stop 2024-03-21T10:14:21.460 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:21.459+0000 7fd9cb800700 1 -- 172.21.15.153:0/69776147 shutdown_connections 2024-03-21T10:14:21.460 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:21.459+0000 7fd9cb800700 1 --2- 172.21.15.153:0/69776147 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd9c4101770 0x7fd9c41057c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:14:21.460 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:21.459+0000 7fd9cb800700 1 --2- 172.21.15.153:0/69776147 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd9c4100dc0 0x7fd9c41011a0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:14:21.460 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:21.459+0000 7fd9cb800700 1 -- 172.21.15.153:0/69776147 >> 172.21.15.153:0/69776147 conn(0x7fd9c40fc650 msgr2=0x7fd9c40fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:14:21.460 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:21.459+0000 7fd9cb800700 1 -- 172.21.15.153:0/69776147 shutdown_connections 2024-03-21T10:14:21.460 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:21.459+0000 7fd9cb800700 1 -- 172.21.15.153:0/69776147 wait complete. 2024-03-21T10:14:21.461 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:21.460+0000 7fd9cb800700 1 Processor -- start 2024-03-21T10:14:21.461 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:21.460+0000 7fd9cb800700 1 -- start start 2024-03-21T10:14:21.461 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:21.460+0000 7fd9cb800700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd9c4100dc0 0x7fd9c4073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:14:21.461 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:21.460+0000 7fd9cb800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd9c4101770 0x7fd9c40735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:14:21.461 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:21.460+0000 7fd9cb800700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c4073b10 con 0x7fd9c4101770 2024-03-21T10:14:21.461 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:21.460+0000 7fd9cb800700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c4073c50 con 0x7fd9c4100dc0 2024-03-21T10:14:21.461 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:21.461+0000 7fd9c3e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd9c4101770 0x7fd9c40735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:14:21.462 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:21.461+0000 7fd9c3e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd9c4101770 0x7fd9c40735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:46322/0 (socket says 172.21.15.153:46322) 2024-03-21T10:14:21.462 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:21.461+0000 7fd9c3e00700 1 -- 172.21.15.153:0/2604025398 learned_addr learned my addr 172.21.15.153:0/2604025398 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:14:21.462 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:21.461+0000 7fd9c3e00700 1 -- 172.21.15.153:0/2604025398 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd9c4100dc0 msgr2=0x7fd9c4073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:14:21.463 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:21.461+0000 7fd9c8e00700 1 --2- 172.21.15.153:0/2604025398 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd9c4100dc0 0x7fd9c4073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:14:21.463 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:21.461+0000 7fd9c3e00700 1 --2- 172.21.15.153:0/2604025398 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd9c4100dc0 0x7fd9c4073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:14:21.463 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:21.461+0000 7fd9c3e00700 1 -- 172.21.15.153:0/2604025398 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9b80095f0 con 0x7fd9c4101770 2024-03-21T10:14:21.463 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:21.461+0000 7fd9c3e00700 1 --2- 172.21.15.153:0/2604025398 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd9c4101770 0x7fd9c40735d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd9b000b480 tx=0x7fd9b000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:14:21.463 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:21.462+0000 7fd9c1600700 1 -- 172.21.15.153:0/2604025398 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd9b00118c0 con 0x7fd9c4101770 2024-03-21T10:14:21.463 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:21.462+0000 7fd9cb800700 1 -- 172.21.15.153:0/2604025398 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9c41a2a10 con 0x7fd9c4101770 2024-03-21T10:14:21.463 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:21.462+0000 7fd9c1600700 1 -- 172.21.15.153:0/2604025398 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd9b0010380 con 0x7fd9c4101770 2024-03-21T10:14:21.463 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:21.462+0000 7fd9cb800700 1 -- 172.21.15.153:0/2604025398 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9c41a2e20 con 0x7fd9c4101770 2024-03-21T10:14:21.463 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:21.462+0000 7fd9c1600700 1 -- 172.21.15.153:0/2604025398 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd9b001a5d0 con 0x7fd9c4101770 2024-03-21T10:14:21.464 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:21.463+0000 7fd9cb800700 1 -- 172.21.15.153:0/2604025398 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9c4109160 con 0x7fd9c4101770 2024-03-21T10:14:21.464 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:21.464+0000 7fd9c1600700 1 -- 172.21.15.153:0/2604025398 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 37) v1 ==== 92321+0+0 (secure 0 0 0) 0x7fd9b00104f0 con 0x7fd9c4101770 2024-03-21T10:14:21.465 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:21.465+0000 7fd9c1600700 1 --2- 172.21.15.153:0/2604025398 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd9ac06e5a0 0x7fd9ac070a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:14:21.465 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:21.465+0000 7fd9c1600700 1 -- 172.21.15.153:0/2604025398 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fd9b008e0d0 con 0x7fd9c4101770 2024-03-21T10:14:21.466 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:21.465+0000 7fd9c8e00700 1 --2- 172.21.15.153:0/2604025398 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd9ac06e5a0 0x7fd9ac070a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:14:21.466 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:21.466+0000 7fd9c8e00700 1 --2- 172.21.15.153:0/2604025398 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd9ac06e5a0 0x7fd9ac070a60 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fd9b800b5c0 tx=0x7fd9b8005960 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:14:21.470 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:21.470+0000 7fd9c1600700 1 -- 172.21.15.153:0/2604025398 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd9b0057220 con 0x7fd9c4101770 2024-03-21T10:14:21.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:21 smithi174 ceph-mon[88004]: from='client.34162 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:14:21.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:21 smithi174 ceph-mon[88004]: Reconfiguring osd.0 (monmap changed)... 2024-03-21T10:14:21.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:21 smithi174 ceph-mon[88004]: Reconfiguring daemon osd.0 on smithi153 2024-03-21T10:14:21.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:21 smithi174 ceph-mon[88004]: from='client.34166 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:14:21.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:21 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/930924834' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:14:21.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:21 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/501006586' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:14:21.769 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:21.768+0000 7fd9cb800700 1 -- 172.21.15.153:0/2604025398 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fd9c404ea90 con 0x7fd9c4101770 2024-03-21T10:14:21.769 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:21.769+0000 7fd9c1600700 1 -- 172.21.15.153:0/2604025398 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7fd9b005a8e0 con 0x7fd9c4101770 2024-03-21T10:14:21.770 INFO:teuthology.orchestra.run.smithi153.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-03-21T10:14:21.770 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-21T10:14:21.770 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-21T10:14:21.772 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:21.772+0000 7fd9cb800700 1 -- 172.21.15.153:0/2604025398 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd9ac06e5a0 msgr2=0x7fd9ac070a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:14:21.772 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:21.772+0000 7fd9cb800700 1 --2- 172.21.15.153:0/2604025398 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd9ac06e5a0 0x7fd9ac070a60 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fd9b800b5c0 tx=0x7fd9b8005960 comp rx=0 tx=0).stop 2024-03-21T10:14:21.772 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:21.772+0000 7fd9cb800700 1 -- 172.21.15.153:0/2604025398 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd9c4101770 msgr2=0x7fd9c40735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:14:21.772 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:21.772+0000 7fd9cb800700 1 --2- 172.21.15.153:0/2604025398 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd9c4101770 0x7fd9c40735d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd9b000b480 tx=0x7fd9b000b840 comp rx=0 tx=0).stop 2024-03-21T10:14:21.773 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:21.772+0000 7fd9cb800700 1 -- 172.21.15.153:0/2604025398 shutdown_connections 2024-03-21T10:14:21.773 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:21.772+0000 7fd9cb800700 1 --2- 172.21.15.153:0/2604025398 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd9c4100dc0 0x7fd9c4073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:14:21.773 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:21.772+0000 7fd9cb800700 1 --2- 172.21.15.153:0/2604025398 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd9ac06e5a0 0x7fd9ac070a60 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:14:21.773 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:21.772+0000 7fd9cb800700 1 --2- 172.21.15.153:0/2604025398 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd9c4101770 0x7fd9c40735d0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:14:21.774 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:21.772+0000 7fd9cb800700 1 -- 172.21.15.153:0/2604025398 >> 172.21.15.153:0/2604025398 conn(0x7fd9c40fc650 msgr2=0x7fd9c40fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:14:21.774 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:21.772+0000 7fd9cb800700 1 -- 172.21.15.153:0/2604025398 shutdown_connections 2024-03-21T10:14:21.774 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:21.773+0000 7fd9cb800700 1 -- 172.21.15.153:0/2604025398 wait complete. 2024-03-21T10:14:22.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:22 smithi153 ceph-mon[113733]: pgmap v28: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:14:22.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:22 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/2604025398' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:14:22.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:22 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:14:22.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:22 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:14:22.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:22 smithi174 ceph-mon[88004]: pgmap v28: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:14:22.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:22 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/2604025398' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:14:22.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:22 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:14:22.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:22 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:14:23.425 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:23 smithi153 ceph-mon[113733]: from='client.34178 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:14:23.426 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:23 smithi153 ceph-mon[113733]: Reconfiguring osd.1 (monmap changed)... 2024-03-21T10:14:23.426 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:23 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-21T10:14:23.426 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:23 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:14:23.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:23 smithi174 ceph-mon[88004]: from='client.34178 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:14:23.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:23 smithi174 ceph-mon[88004]: Reconfiguring osd.1 (monmap changed)... 2024-03-21T10:14:23.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:23 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-21T10:14:23.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:23 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:14:24.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:24 smithi153 ceph-mon[113733]: Reconfiguring daemon osd.1 on smithi153 2024-03-21T10:14:24.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:24 smithi153 ceph-mon[113733]: pgmap v29: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:14:24.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:24 smithi174 ceph-mon[88004]: Reconfiguring daemon osd.1 on smithi153 2024-03-21T10:14:24.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:24 smithi174 ceph-mon[88004]: pgmap v29: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:14:25.950 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:25 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:14:25.950 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:25 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:14:25.950 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:25 smithi153 ceph-mon[113733]: Reconfiguring osd.2 (monmap changed)... 2024-03-21T10:14:25.950 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:25 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-21T10:14:25.950 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:25 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:14:25.951 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:25 smithi153 ceph-mon[113733]: Reconfiguring daemon osd.2 on smithi153 2024-03-21T10:14:25.951 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:25 smithi153 ceph-mon[113733]: pgmap v30: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:14:26.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:25 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:14:26.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:25 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:14:26.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:25 smithi174 ceph-mon[88004]: Reconfiguring osd.2 (monmap changed)... 2024-03-21T10:14:26.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:25 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-21T10:14:26.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:25 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:14:26.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:25 smithi174 ceph-mon[88004]: Reconfiguring daemon osd.2 on smithi153 2024-03-21T10:14:26.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:25 smithi174 ceph-mon[88004]: pgmap v30: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:14:28.448 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:28 smithi153 ceph-mon[113733]: pgmap v31: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:14:28.448 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:28 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:14:28.449 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:28 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:14:28.449 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:28 smithi153 ceph-mon[113733]: Reconfiguring mds.cephfs.smithi153.zipckr (monmap changed)... 2024-03-21T10:14:28.449 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:28 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi153.zipckr", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T10:14:28.449 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:28 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:14:28.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:28 smithi174 ceph-mon[88004]: pgmap v31: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:14:28.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:28 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:14:28.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:28 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:14:28.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:28 smithi174 ceph-mon[88004]: Reconfiguring mds.cephfs.smithi153.zipckr (monmap changed)... 2024-03-21T10:14:28.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:28 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi153.zipckr", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T10:14:28.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:28 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:14:29.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:29 smithi153 ceph-mon[113733]: Reconfiguring daemon mds.cephfs.smithi153.zipckr on smithi153 2024-03-21T10:14:29.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:29 smithi174 ceph-mon[88004]: Reconfiguring daemon mds.cephfs.smithi153.zipckr on smithi153 2024-03-21T10:14:30.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:30 smithi174 ceph-mon[88004]: pgmap v32: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:14:30.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:30 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:14:30.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:30 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:14:30.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:30 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi153.iuyluj", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T10:14:30.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:30 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:14:30.453 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:30 smithi153 ceph-mon[113733]: pgmap v32: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:14:30.453 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:30 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:14:30.453 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:30 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:14:30.453 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:30 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi153.iuyluj", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T10:14:30.453 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:30 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:14:31.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:31 smithi174 ceph-mon[88004]: Reconfiguring mds.cephfs.smithi153.iuyluj (monmap changed)... 2024-03-21T10:14:31.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:31 smithi174 ceph-mon[88004]: Reconfiguring daemon mds.cephfs.smithi153.iuyluj on smithi153 2024-03-21T10:14:31.516 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:31 smithi153 ceph-mon[113733]: Reconfiguring mds.cephfs.smithi153.iuyluj (monmap changed)... 2024-03-21T10:14:31.516 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:31 smithi153 ceph-mon[113733]: Reconfiguring daemon mds.cephfs.smithi153.iuyluj on smithi153 2024-03-21T10:14:32.368 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:32 smithi153 ceph-mon[113733]: pgmap v33: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:14:32.368 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:32 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:14:32.369 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:32 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:14:32.369 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:32 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:14:32.369 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:32 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi174", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-21T10:14:32.369 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:32 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi174", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-21T10:14:32.369 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:32 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi174", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-03-21T10:14:32.369 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:32 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.ceph-exporter.smithi174"}]: dispatch 2024-03-21T10:14:32.369 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:32 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:14:32.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:32 smithi174 ceph-mon[88004]: pgmap v33: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:14:32.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:32 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:14:32.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:32 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:14:32.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:32 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:14:32.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:32 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi174", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-21T10:14:32.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:32 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi174", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-21T10:14:32.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:32 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi174", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-03-21T10:14:32.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:32 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.ceph-exporter.smithi174"}]: dispatch 2024-03-21T10:14:32.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:32 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:14:33.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:33 smithi174 ceph-mon[88004]: Reconfiguring ceph-exporter.smithi174 (monmap changed)... 2024-03-21T10:14:33.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:33 smithi174 ceph-mon[88004]: Unable to update caps for client.ceph-exporter.smithi174 2024-03-21T10:14:33.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:33 smithi174 ceph-mon[88004]: Reconfiguring daemon ceph-exporter.smithi174 on smithi174 2024-03-21T10:14:33.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:33 smithi153 ceph-mon[113733]: Reconfiguring ceph-exporter.smithi174 (monmap changed)... 2024-03-21T10:14:33.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:33 smithi153 ceph-mon[113733]: Unable to update caps for client.ceph-exporter.smithi174 2024-03-21T10:14:33.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:33 smithi153 ceph-mon[113733]: Reconfiguring daemon ceph-exporter.smithi174 on smithi174 2024-03-21T10:14:34.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:34 smithi153 ceph-mon[113733]: pgmap v34: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:14:34.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:34 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:14:34.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:34 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:14:34.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:34 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi174", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-21T10:14:34.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:34 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:14:34.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:34 smithi174 ceph-mon[88004]: pgmap v34: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:14:34.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:34 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:14:34.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:34 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:14:34.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:34 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi174", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-21T10:14:34.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:34 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:14:35.521 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:35 smithi174 ceph-mon[88004]: Reconfiguring crash.smithi174 (monmap changed)... 2024-03-21T10:14:35.521 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:35 smithi174 ceph-mon[88004]: Reconfiguring daemon crash.smithi174 on smithi174 2024-03-21T10:14:35.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:35 smithi153 ceph-mon[113733]: Reconfiguring crash.smithi174 (monmap changed)... 2024-03-21T10:14:35.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:35 smithi153 ceph-mon[113733]: Reconfiguring daemon crash.smithi174 on smithi174 2024-03-21T10:14:36.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:36 smithi174 ceph-mon[88004]: pgmap v35: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:14:36.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:36 smithi153 ceph-mon[113733]: pgmap v35: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:14:37.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:37 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:14:37.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:37 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:14:37.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:37 smithi174 ceph-mon[88004]: Reconfiguring mgr.smithi174.qpcwat (monmap changed)... 2024-03-21T10:14:37.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:37 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi174.qpcwat", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-21T10:14:37.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:37 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-21T10:14:37.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:37 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:14:37.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:37 smithi174 ceph-mon[88004]: Reconfiguring daemon mgr.smithi174.qpcwat on smithi174 2024-03-21T10:14:37.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:37 smithi174 ceph-mon[88004]: pgmap v36: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:14:37.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:37 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:14:37.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:37 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:14:37.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:37 smithi153 ceph-mon[113733]: Reconfiguring mgr.smithi174.qpcwat (monmap changed)... 2024-03-21T10:14:37.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:37 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi174.qpcwat", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-21T10:14:37.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:37 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-21T10:14:37.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:37 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:14:37.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:37 smithi153 ceph-mon[113733]: Reconfiguring daemon mgr.smithi174.qpcwat on smithi174 2024-03-21T10:14:37.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:37 smithi153 ceph-mon[113733]: pgmap v36: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:14:39.492 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:39 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:14:39.492 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:39 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:14:39.492 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:39 smithi174 ceph-mon[88004]: Reconfiguring mon.smithi174 (monmap changed)... 2024-03-21T10:14:39.492 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:39 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T10:14:39.492 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:39 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-21T10:14:39.493 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:39 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:14:39.493 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:39 smithi174 ceph-mon[88004]: Reconfiguring daemon mon.smithi174 on smithi174 2024-03-21T10:14:39.493 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:39 smithi174 ceph-mon[88004]: pgmap v37: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:14:39.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:39 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:14:39.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:39 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:14:39.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:39 smithi153 ceph-mon[113733]: Reconfiguring mon.smithi174 (monmap changed)... 2024-03-21T10:14:39.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:39 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T10:14:39.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:39 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-21T10:14:39.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:39 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:14:39.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:39 smithi153 ceph-mon[113733]: Reconfiguring daemon mon.smithi174 on smithi174 2024-03-21T10:14:39.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:39 smithi153 ceph-mon[113733]: pgmap v37: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:14:41.643 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:41 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:14:41.643 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:41 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:14:41.643 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:41 smithi174 ceph-mon[88004]: Reconfiguring osd.3 (monmap changed)... 2024-03-21T10:14:41.643 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:41 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-21T10:14:41.643 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:41 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:14:41.643 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:41 smithi174 ceph-mon[88004]: Reconfiguring daemon osd.3 on smithi174 2024-03-21T10:14:41.643 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:41 smithi174 ceph-mon[88004]: pgmap v38: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:14:41.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:41 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:14:41.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:41 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:14:41.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:41 smithi153 ceph-mon[113733]: Reconfiguring osd.3 (monmap changed)... 2024-03-21T10:14:41.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:41 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-21T10:14:41.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:41 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:14:41.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:41 smithi153 ceph-mon[113733]: Reconfiguring daemon osd.3 on smithi174 2024-03-21T10:14:41.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:41 smithi153 ceph-mon[113733]: pgmap v38: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:14:43.883 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:43 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:14:43.883 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:43 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:14:43.884 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:43 smithi174 ceph-mon[88004]: Reconfiguring osd.4 (monmap changed)... 2024-03-21T10:14:43.884 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:43 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-21T10:14:43.884 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:43 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:14:43.884 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:43 smithi174 ceph-mon[88004]: Reconfiguring daemon osd.4 on smithi174 2024-03-21T10:14:43.884 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:43 smithi174 ceph-mon[88004]: pgmap v39: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:14:44.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:43 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:14:44.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:43 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:14:44.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:43 smithi153 ceph-mon[113733]: Reconfiguring osd.4 (monmap changed)... 2024-03-21T10:14:44.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:43 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-21T10:14:44.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:43 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:14:44.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:43 smithi153 ceph-mon[113733]: Reconfiguring daemon osd.4 on smithi174 2024-03-21T10:14:44.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:43 smithi153 ceph-mon[113733]: pgmap v39: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:14:46.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:45 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:14:46.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:45 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:14:46.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:45 smithi174 ceph-mon[88004]: Reconfiguring osd.5 (monmap changed)... 2024-03-21T10:14:46.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:45 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-21T10:14:46.203 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:45 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:14:46.203 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:45 smithi174 ceph-mon[88004]: Reconfiguring daemon osd.5 on smithi174 2024-03-21T10:14:46.203 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:45 smithi174 ceph-mon[88004]: pgmap v40: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:14:46.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:45 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:14:46.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:45 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:14:46.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:45 smithi153 ceph-mon[113733]: Reconfiguring osd.5 (monmap changed)... 2024-03-21T10:14:46.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:45 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-21T10:14:46.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:45 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:14:46.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:45 smithi153 ceph-mon[113733]: Reconfiguring daemon osd.5 on smithi174 2024-03-21T10:14:46.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:45 smithi153 ceph-mon[113733]: pgmap v40: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:14:47.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:46 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:14:47.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:46 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:14:48.231 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:47 smithi174 ceph-mon[88004]: pgmap v41: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:14:48.231 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:47 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:14:48.231 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:47 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:14:48.232 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:47 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi174.yyimst", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T10:14:48.232 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:47 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:14:48.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:47 smithi153 ceph-mon[113733]: pgmap v41: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:14:48.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:47 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:14:48.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:47 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:14:48.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:47 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi174.yyimst", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T10:14:48.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:47 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:14:49.243 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:48 smithi174 ceph-mon[88004]: Reconfiguring mds.cephfs.smithi174.yyimst (monmap changed)... 2024-03-21T10:14:49.244 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:48 smithi174 ceph-mon[88004]: Reconfiguring daemon mds.cephfs.smithi174.yyimst on smithi174 2024-03-21T10:14:49.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:48 smithi153 ceph-mon[113733]: Reconfiguring mds.cephfs.smithi174.yyimst (monmap changed)... 2024-03-21T10:14:49.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:48 smithi153 ceph-mon[113733]: Reconfiguring daemon mds.cephfs.smithi174.yyimst on smithi174 2024-03-21T10:14:50.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:49 smithi153 ceph-mon[113733]: pgmap v42: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:14:50.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:49 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:14:50.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:49 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:14:50.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:49 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi174.nhozag", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T10:14:50.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:49 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:14:50.395 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:49 smithi174 ceph-mon[88004]: pgmap v42: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:14:50.395 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:49 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:14:50.395 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:49 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:14:50.395 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:49 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi174.nhozag", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T10:14:50.395 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:49 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:14:51.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:50 smithi153 ceph-mon[113733]: Reconfiguring mds.cephfs.smithi174.nhozag (monmap changed)... 2024-03-21T10:14:51.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:50 smithi153 ceph-mon[113733]: Reconfiguring daemon mds.cephfs.smithi174.nhozag on smithi174 2024-03-21T10:14:51.407 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:50 smithi174 ceph-mon[88004]: Reconfiguring mds.cephfs.smithi174.nhozag (monmap changed)... 2024-03-21T10:14:51.407 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:50 smithi174 ceph-mon[88004]: Reconfiguring daemon mds.cephfs.smithi174.nhozag on smithi174 2024-03-21T10:14:51.943 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:51.941+0000 7fd6e3800700 1 -- 172.21.15.153:0/4099116660 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd6dc0731c0 msgr2=0x7fd6dc0735a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:14:51.943 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:51.941+0000 7fd6e3800700 1 --2- 172.21.15.153:0/4099116660 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd6dc0731c0 0x7fd6dc0735a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd6d0009910 tx=0x7fd6d0009c20 comp rx=0 tx=0).stop 2024-03-21T10:14:51.943 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:51.941+0000 7fd6e3800700 1 -- 172.21.15.153:0/4099116660 shutdown_connections 2024-03-21T10:14:51.943 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:51.941+0000 7fd6e3800700 1 --2- 172.21.15.153:0/4099116660 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd6dc073ae0 0x7fd6dc10ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:14:51.943 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:51.941+0000 7fd6e3800700 1 --2- 172.21.15.153:0/4099116660 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd6dc0731c0 0x7fd6dc0735a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:14:51.944 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:51.941+0000 7fd6e3800700 1 -- 172.21.15.153:0/4099116660 >> 172.21.15.153:0/4099116660 conn(0x7fd6dc0fc630 msgr2=0x7fd6dc0fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:14:51.944 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:51.941+0000 7fd6e3800700 1 -- 172.21.15.153:0/4099116660 shutdown_connections 2024-03-21T10:14:51.944 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:51.941+0000 7fd6e3800700 1 -- 172.21.15.153:0/4099116660 wait complete. 2024-03-21T10:14:51.945 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:51.942+0000 7fd6e3800700 1 Processor -- start 2024-03-21T10:14:51.945 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:51.942+0000 7fd6e3800700 1 -- start start 2024-03-21T10:14:51.945 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:51.943+0000 7fd6e3800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd6dc0731c0 0x7fd6dc198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:14:51.945 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:51.943+0000 7fd6e3800700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd6dc073ae0 0x7fd6dc198e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:14:51.946 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:51.943+0000 7fd6e3800700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6dc199520 con 0x7fd6dc0731c0 2024-03-21T10:14:51.946 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:51.943+0000 7fd6e3800700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6dc19d330 con 0x7fd6dc073ae0 2024-03-21T10:14:51.946 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:51.943+0000 7fd6e0e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd6dc0731c0 0x7fd6dc198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:14:51.946 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:51.943+0000 7fd6e0e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd6dc0731c0 0x7fd6dc198950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:58418/0 (socket says 172.21.15.153:58418) 2024-03-21T10:14:51.946 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:51.943+0000 7fd6e0e00700 1 -- 172.21.15.153:0/2802662047 learned_addr learned my addr 172.21.15.153:0/2802662047 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:14:51.946 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:51.943+0000 7fd6dbe00700 1 --2- 172.21.15.153:0/2802662047 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd6dc073ae0 0x7fd6dc198e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:14:51.946 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:51.943+0000 7fd6e0e00700 1 -- 172.21.15.153:0/2802662047 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd6dc073ae0 msgr2=0x7fd6dc198e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:14:51.946 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:51.943+0000 7fd6e0e00700 1 --2- 172.21.15.153:0/2802662047 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd6dc073ae0 0x7fd6dc198e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:14:51.946 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:51.944+0000 7fd6e0e00700 1 -- 172.21.15.153:0/2802662047 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6d00095f0 con 0x7fd6dc0731c0 2024-03-21T10:14:51.946 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:51.944+0000 7fd6e0e00700 1 --2- 172.21.15.153:0/2802662047 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd6dc0731c0 0x7fd6dc198950 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd6d000bd70 tx=0x7fd6d000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:14:51.947 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:51.944+0000 7fd6d9600700 1 -- 172.21.15.153:0/2802662047 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd6d001d070 con 0x7fd6dc0731c0 2024-03-21T10:14:51.947 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:51.944+0000 7fd6d9600700 1 -- 172.21.15.153:0/2802662047 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd6d000f460 con 0x7fd6dc0731c0 2024-03-21T10:14:51.947 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:51.944+0000 7fd6d9600700 1 -- 172.21.15.153:0/2802662047 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd6d0017630 con 0x7fd6dc0731c0 2024-03-21T10:14:51.947 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:51.944+0000 7fd6e3800700 1 -- 172.21.15.153:0/2802662047 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6dc19d5b0 con 0x7fd6dc0731c0 2024-03-21T10:14:51.947 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:51.944+0000 7fd6e3800700 1 -- 172.21.15.153:0/2802662047 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6dc19daa0 con 0x7fd6dc0731c0 2024-03-21T10:14:51.947 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:51.946+0000 7fd6e3800700 1 -- 172.21.15.153:0/2802662047 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6dc10a540 con 0x7fd6dc0731c0 2024-03-21T10:14:51.950 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:51.948+0000 7fd6d9600700 1 -- 172.21.15.153:0/2802662047 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 37) v1 ==== 92321+0+0 (secure 0 0 0) 0x7fd6d0003a60 con 0x7fd6dc0731c0 2024-03-21T10:14:51.951 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:51.949+0000 7fd6d9600700 1 --2- 172.21.15.153:0/2802662047 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd6c406e5a0 0x7fd6c4070a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:14:51.951 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:51.949+0000 7fd6d9600700 1 -- 172.21.15.153:0/2802662047 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fd6d008fca0 con 0x7fd6dc0731c0 2024-03-21T10:14:51.951 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:51.950+0000 7fd6dbe00700 1 --2- 172.21.15.153:0/2802662047 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd6c406e5a0 0x7fd6c4070a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:14:51.952 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:51.951+0000 7fd6dbe00700 1 --2- 172.21.15.153:0/2802662047 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd6c406e5a0 0x7fd6c4070a60 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd6c8005950 tx=0x7fd6c80058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:14:51.954 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:51.953+0000 7fd6d9600700 1 -- 172.21.15.153:0/2802662047 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd6d005c930 con 0x7fd6dc0731c0 2024-03-21T10:14:52.208 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:52.206+0000 7fd6e3800700 1 -- 172.21.15.153:0/2802662047 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd6dc19dd80 con 0x7fd6c406e5a0 2024-03-21T10:14:52.208 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:51 smithi153 ceph-mon[113733]: pgmap v43: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:14:52.208 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:51 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:14:52.208 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:51 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:14:52.208 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:51 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:14:52.208 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:51 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:14:52.209 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:51 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:14:52.209 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:51 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:14:52.209 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:51 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi153"}]: dispatch 2024-03-21T10:14:52.209 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:51 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi153"}]': finished 2024-03-21T10:14:52.209 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:51 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi174"}]: dispatch 2024-03-21T10:14:52.209 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:51 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi174"}]': finished 2024-03-21T10:14:52.211 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:52.209+0000 7fd6d9600700 1 -- 172.21.15.153:0/2802662047 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+385 (secure 0 0 0) 0x7fd6dc19dd80 con 0x7fd6c406e5a0 2024-03-21T10:14:52.215 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:52.213+0000 7fd6e3800700 1 -- 172.21.15.153:0/2802662047 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd6c406e5a0 msgr2=0x7fd6c4070a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:14:52.215 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:52.213+0000 7fd6e3800700 1 --2- 172.21.15.153:0/2802662047 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd6c406e5a0 0x7fd6c4070a60 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd6c8005950 tx=0x7fd6c80058e0 comp rx=0 tx=0).stop 2024-03-21T10:14:52.215 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:52.213+0000 7fd6e3800700 1 -- 172.21.15.153:0/2802662047 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd6dc0731c0 msgr2=0x7fd6dc198950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:14:52.215 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:52.213+0000 7fd6e3800700 1 --2- 172.21.15.153:0/2802662047 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd6dc0731c0 0x7fd6dc198950 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd6d000bd70 tx=0x7fd6d000be50 comp rx=0 tx=0).stop 2024-03-21T10:14:52.215 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:52.213+0000 7fd6e3800700 1 -- 172.21.15.153:0/2802662047 shutdown_connections 2024-03-21T10:14:52.216 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:52.214+0000 7fd6e3800700 1 --2- 172.21.15.153:0/2802662047 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd6c406e5a0 0x7fd6c4070a60 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:14:52.216 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:52.214+0000 7fd6e3800700 1 --2- 172.21.15.153:0/2802662047 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd6dc0731c0 0x7fd6dc198950 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:14:52.216 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:52.214+0000 7fd6e3800700 1 --2- 172.21.15.153:0/2802662047 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd6dc073ae0 0x7fd6dc198e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:14:52.216 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:52.214+0000 7fd6e3800700 1 -- 172.21.15.153:0/2802662047 >> 172.21.15.153:0/2802662047 conn(0x7fd6dc0fc630 msgr2=0x7fd6dc107680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:14:52.216 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:52.214+0000 7fd6e3800700 1 -- 172.21.15.153:0/2802662047 shutdown_connections 2024-03-21T10:14:52.216 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:52.214+0000 7fd6e3800700 1 -- 172.21.15.153:0/2802662047 wait complete. 2024-03-21T10:14:52.242 INFO:teuthology.orchestra.run.smithi153.stdout:true 2024-03-21T10:14:52.390 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:52.388+0000 7f4e43000700 1 -- 172.21.15.153:0/444101395 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e3c073a50 msgr2=0x7f4e3c111610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:14:52.391 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:52.388+0000 7f4e43000700 1 --2- 172.21.15.153:0/444101395 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e3c073a50 0x7f4e3c111610 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4e28009910 tx=0x7f4e28009c20 comp rx=0 tx=0).stop 2024-03-21T10:14:52.391 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:52.388+0000 7f4e43000700 1 -- 172.21.15.153:0/444101395 shutdown_connections 2024-03-21T10:14:52.391 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:52.388+0000 7f4e43000700 1 --2- 172.21.15.153:0/444101395 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e3c073a50 0x7f4e3c111610 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:14:52.391 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:52.388+0000 7f4e43000700 1 --2- 172.21.15.153:0/444101395 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4e3c073130 0x7f4e3c073510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:14:52.391 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:52.388+0000 7f4e43000700 1 -- 172.21.15.153:0/444101395 >> 172.21.15.153:0/444101395 conn(0x7f4e3c0fc630 msgr2=0x7f4e3c0fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:14:52.391 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:52.389+0000 7f4e43000700 1 -- 172.21.15.153:0/444101395 shutdown_connections 2024-03-21T10:14:52.391 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:52.389+0000 7f4e43000700 1 -- 172.21.15.153:0/444101395 wait complete. 2024-03-21T10:14:52.391 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:52.390+0000 7f4e43000700 1 Processor -- start 2024-03-21T10:14:52.392 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:52.390+0000 7f4e43000700 1 -- start start 2024-03-21T10:14:52.392 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:52.391+0000 7f4e43000700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4e3c073130 0x7f4e3c19cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:14:52.392 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:52.391+0000 7f4e43000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e3c073a50 0x7f4e3c19d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:14:52.393 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:52.391+0000 7f4e43000700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e3c19d9e0 con 0x7f4e3c073a50 2024-03-21T10:14:52.393 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:52.391+0000 7f4e43000700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e3c1a1770 con 0x7f4e3c073130 2024-03-21T10:14:52.393 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:52.391+0000 7f4e3b400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e3c073a50 0x7f4e3c19d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:14:52.393 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:52.391+0000 7f4e3b400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e3c073a50 0x7f4e3c19d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:58430/0 (socket says 172.21.15.153:58430) 2024-03-21T10:14:52.393 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:52.391+0000 7f4e3b400700 1 -- 172.21.15.153:0/3985280446 learned_addr learned my addr 172.21.15.153:0/3985280446 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:14:52.393 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:52.391+0000 7f4e3be00700 1 --2- 172.21.15.153:0/3985280446 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4e3c073130 0x7f4e3c19cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:14:52.394 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:52.392+0000 7f4e3b400700 1 -- 172.21.15.153:0/3985280446 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4e3c073130 msgr2=0x7f4e3c19cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:14:52.394 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:52.392+0000 7f4e3b400700 1 --2- 172.21.15.153:0/3985280446 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4e3c073130 0x7f4e3c19cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:14:52.394 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:52.392+0000 7f4e3b400700 1 -- 172.21.15.153:0/3985280446 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e280095f0 con 0x7f4e3c073a50 2024-03-21T10:14:52.394 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:52.392+0000 7f4e3b400700 1 --2- 172.21.15.153:0/3985280446 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e3c073a50 0x7f4e3c19d300 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4e28006010 tx=0x7f4e2800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:14:52.395 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:52.393+0000 7f4e38c00700 1 -- 172.21.15.153:0/3985280446 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4e2801d070 con 0x7f4e3c073a50 2024-03-21T10:14:52.395 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:52.393+0000 7f4e38c00700 1 -- 172.21.15.153:0/3985280446 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4e2800f460 con 0x7f4e3c073a50 2024-03-21T10:14:52.395 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:52.393+0000 7f4e43000700 1 -- 172.21.15.153:0/3985280446 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e3c1a1a50 con 0x7f4e3c073a50 2024-03-21T10:14:52.395 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:52.393+0000 7f4e38c00700 1 -- 172.21.15.153:0/3985280446 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4e28017610 con 0x7f4e3c073a50 2024-03-21T10:14:52.395 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:52.393+0000 7f4e43000700 1 -- 172.21.15.153:0/3985280446 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e3c1a1fa0 con 0x7f4e3c073a50 2024-03-21T10:14:52.395 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:52.394+0000 7f4e43000700 1 -- 172.21.15.153:0/3985280446 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e3c10ed90 con 0x7f4e3c073a50 2024-03-21T10:14:52.397 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:52.396+0000 7f4e38c00700 1 -- 172.21.15.153:0/3985280446 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 37) v1 ==== 92321+0+0 (secure 0 0 0) 0x7f4e2800f5d0 con 0x7f4e3c073a50 2024-03-21T10:14:52.399 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:52.397+0000 7f4e38c00700 1 --2- 172.21.15.153:0/3985280446 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f4e2006e590 0x7f4e20070a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:14:52.399 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:52.397+0000 7f4e38c00700 1 -- 172.21.15.153:0/3985280446 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f4e2808fbf0 con 0x7f4e3c073a50 2024-03-21T10:14:52.399 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:52.398+0000 7f4e3be00700 1 --2- 172.21.15.153:0/3985280446 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f4e2006e590 0x7f4e20070a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:14:52.400 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:52.398+0000 7f4e3be00700 1 --2- 172.21.15.153:0/3985280446 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f4e2006e590 0x7f4e20070a50 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c009a10 tx=0x7f4e2c009190 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:14:52.402 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:52.400+0000 7f4e38c00700 1 -- 172.21.15.153:0/3985280446 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4e28058d50 con 0x7f4e3c073a50 2024-03-21T10:14:52.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:51 smithi174 ceph-mon[88004]: pgmap v43: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:14:52.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:51 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:14:52.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:51 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:14:52.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:51 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:14:52.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:51 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:14:52.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:51 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:14:52.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:51 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:14:52.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:51 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi153"}]: dispatch 2024-03-21T10:14:52.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:51 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi153"}]': finished 2024-03-21T10:14:52.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:51 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi174"}]: dispatch 2024-03-21T10:14:52.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:51 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi174"}]': finished 2024-03-21T10:14:52.651 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:52.649+0000 7f4e43000700 1 -- 172.21.15.153:0/3985280446 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4e3c1a2340 con 0x7f4e2006e590 2024-03-21T10:14:52.652 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:52.651+0000 7f4e38c00700 1 -- 172.21.15.153:0/3985280446 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+385 (secure 0 0 0) 0x7f4e3c1a2340 con 0x7f4e2006e590 2024-03-21T10:14:52.655 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:52.653+0000 7f4e43000700 1 -- 172.21.15.153:0/3985280446 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f4e2006e590 msgr2=0x7f4e20070a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:14:52.656 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:52.653+0000 7f4e43000700 1 --2- 172.21.15.153:0/3985280446 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f4e2006e590 0x7f4e20070a50 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c009a10 tx=0x7f4e2c009190 comp rx=0 tx=0).stop 2024-03-21T10:14:52.656 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:52.653+0000 7f4e43000700 1 -- 172.21.15.153:0/3985280446 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e3c073a50 msgr2=0x7f4e3c19d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:14:52.656 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:52.653+0000 7f4e43000700 1 --2- 172.21.15.153:0/3985280446 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e3c073a50 0x7f4e3c19d300 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4e28006010 tx=0x7f4e2800bd70 comp rx=0 tx=0).stop 2024-03-21T10:14:52.656 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:52.654+0000 7f4e43000700 1 -- 172.21.15.153:0/3985280446 shutdown_connections 2024-03-21T10:14:52.656 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:52.654+0000 7f4e43000700 1 --2- 172.21.15.153:0/3985280446 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4e3c073130 0x7f4e3c19cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:14:52.656 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:52.654+0000 7f4e43000700 1 --2- 172.21.15.153:0/3985280446 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f4e2006e590 0x7f4e20070a50 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:14:52.656 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:52.654+0000 7f4e43000700 1 --2- 172.21.15.153:0/3985280446 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e3c073a50 0x7f4e3c19d300 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:14:52.656 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:52.654+0000 7f4e43000700 1 -- 172.21.15.153:0/3985280446 >> 172.21.15.153:0/3985280446 conn(0x7f4e3c0fc630 msgr2=0x7f4e3c103120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:14:52.656 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:52.654+0000 7f4e43000700 1 -- 172.21.15.153:0/3985280446 shutdown_connections 2024-03-21T10:14:52.656 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:52.654+0000 7f4e43000700 1 -- 172.21.15.153:0/3985280446 wait complete. 2024-03-21T10:14:52.811 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:52.809+0000 7f8f93a00700 1 -- 172.21.15.153:0/872196581 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f8c069020 msgr2=0x7f8f8c105790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:14:52.812 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:52.809+0000 7f8f93a00700 1 --2- 172.21.15.153:0/872196581 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f8c069020 0x7f8f8c105790 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8f78009910 tx=0x7f8f78009c20 comp rx=0 tx=0).stop 2024-03-21T10:14:52.812 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:52.809+0000 7f8f93a00700 1 -- 172.21.15.153:0/872196581 shutdown_connections 2024-03-21T10:14:52.812 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:52.810+0000 7f8f93a00700 1 --2- 172.21.15.153:0/872196581 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f8c069020 0x7f8f8c105790 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:14:52.812 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:52.810+0000 7f8f93a00700 1 --2- 172.21.15.153:0/872196581 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8f8c068670 0x7f8f8c068a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:14:52.812 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:52.810+0000 7f8f93a00700 1 -- 172.21.15.153:0/872196581 >> 172.21.15.153:0/872196581 conn(0x7f8f8c0755a0 msgr2=0x7f8f8c0759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:14:52.812 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:52.810+0000 7f8f93a00700 1 -- 172.21.15.153:0/872196581 shutdown_connections 2024-03-21T10:14:52.812 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:52.810+0000 7f8f93a00700 1 -- 172.21.15.153:0/872196581 wait complete. 2024-03-21T10:14:52.813 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:52.811+0000 7f8f93a00700 1 Processor -- start 2024-03-21T10:14:52.813 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:52.811+0000 7f8f93a00700 1 -- start start 2024-03-21T10:14:52.813 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:52.812+0000 7f8f93a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f8c068670 0x7f8f8c196700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:14:52.813 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:52.812+0000 7f8f93a00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8f8c069020 0x7f8f8c196c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:14:52.813 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:52.812+0000 7f8f93a00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f8c1972d0 con 0x7f8f8c068670 2024-03-21T10:14:52.814 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:52.812+0000 7f8f93a00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f8c19a000 con 0x7f8f8c069020 2024-03-21T10:14:52.814 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:52.812+0000 7f8f91000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f8c068670 0x7f8f8c196700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:14:52.815 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:52.812+0000 7f8f91000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f8c068670 0x7f8f8c196700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:58440/0 (socket says 172.21.15.153:58440) 2024-03-21T10:14:52.815 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:52.812+0000 7f8f91000700 1 -- 172.21.15.153:0/150924011 learned_addr learned my addr 172.21.15.153:0/150924011 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:14:52.815 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:52.813+0000 7f8f8be00700 1 --2- 172.21.15.153:0/150924011 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8f8c069020 0x7f8f8c196c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:14:52.815 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:52.813+0000 7f8f91000700 1 -- 172.21.15.153:0/150924011 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8f8c069020 msgr2=0x7f8f8c196c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:14:52.815 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:52.813+0000 7f8f91000700 1 --2- 172.21.15.153:0/150924011 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8f8c069020 0x7f8f8c196c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:14:52.815 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:52.813+0000 7f8f91000700 1 -- 172.21.15.153:0/150924011 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f780095f0 con 0x7f8f8c068670 2024-03-21T10:14:52.815 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:52.813+0000 7f8f91000700 1 --2- 172.21.15.153:0/150924011 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f8c068670 0x7f8f8c196700 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8f8000c8a0 tx=0x7f8f8000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:14:52.816 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:52.813+0000 7f8f89600700 1 -- 172.21.15.153:0/150924011 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8f8000cea0 con 0x7f8f8c068670 2024-03-21T10:14:52.816 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:52.813+0000 7f8f89600700 1 -- 172.21.15.153:0/150924011 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8f80004830 con 0x7f8f8c068670 2024-03-21T10:14:52.816 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:52.813+0000 7f8f93a00700 1 -- 172.21.15.153:0/150924011 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f8c19a2e0 con 0x7f8f8c068670 2024-03-21T10:14:52.816 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:52.814+0000 7f8f89600700 1 -- 172.21.15.153:0/150924011 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8f800055b0 con 0x7f8f8c068670 2024-03-21T10:14:52.816 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:52.814+0000 7f8f93a00700 1 -- 172.21.15.153:0/150924011 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f8c19a800 con 0x7f8f8c068670 2024-03-21T10:14:52.816 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:52.815+0000 7f8f93a00700 1 -- 172.21.15.153:0/150924011 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f8c1090a0 con 0x7f8f8c068670 2024-03-21T10:14:52.817 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:52.816+0000 7f8f89600700 1 -- 172.21.15.153:0/150924011 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 37) v1 ==== 92321+0+0 (secure 0 0 0) 0x7f8f800049a0 con 0x7f8f8c068670 2024-03-21T10:14:52.818 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:52.817+0000 7f8f89600700 1 --2- 172.21.15.153:0/150924011 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f8f7406e5a0 0x7f8f74070a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:14:52.818 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:52.817+0000 7f8f89600700 1 -- 172.21.15.153:0/150924011 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f8f8008e360 con 0x7f8f8c068670 2024-03-21T10:14:52.819 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:52.817+0000 7f8f8be00700 1 --2- 172.21.15.153:0/150924011 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f8f7406e5a0 0x7f8f74070a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:14:52.819 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:52.818+0000 7f8f8be00700 1 --2- 172.21.15.153:0/150924011 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f8f7406e5a0 0x7f8f74070a60 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c197e20 tx=0x7f8f7801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:14:52.823 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:52.822+0000 7f8f89600700 1 -- 172.21.15.153:0/150924011 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f8f80057300 con 0x7f8f8c068670 2024-03-21T10:14:53.065 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:53.063+0000 7f8f93a00700 1 -- 172.21.15.153:0/150924011 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f8f8c19aa80 con 0x7f8f7406e5a0 2024-03-21T10:14:53.070 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:53 smithi153 ceph-mon[113733]: Upgrade: Setting container_image for all mon 2024-03-21T10:14:53.070 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:53 smithi153 ceph-mon[113733]: from='client.34186 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:14:53.076 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:53.075+0000 7f8f89600700 1 -- 172.21.15.153:0/150924011 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f8f8c19aa80 con 0x7f8f7406e5a0 2024-03-21T10:14:53.077 INFO:teuthology.orchestra.run.smithi153.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-21T10:14:53.077 INFO:teuthology.orchestra.run.smithi153.stdout:alertmanager.smithi153 smithi153 *:9093,9094 running (3m) 74s ago 17m 14.5M - 0.25.0 c8568f914cd2 9ed3f2200b5f 2024-03-21T10:14:53.077 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi153 smithi153 running (17m) 74s ago 17m 8334k - 18.2.1 5be31c24972a 5b61994396a6 2024-03-21T10:14:53.077 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi174 smithi174 running (16m) 50s ago 16m 8266k - 18.2.1 5be31c24972a 4f7a5fc3ac28 2024-03-21T10:14:53.077 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi153 smithi153 running (17m) 74s ago 17m 6665k - 18.2.1 5be31c24972a 90a8b8ea03d1 2024-03-21T10:14:53.077 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi174 smithi174 running (16m) 50s ago 16m 6665k - 18.2.1 5be31c24972a a4c929e4dc77 2024-03-21T10:14:53.077 INFO:teuthology.orchestra.run.smithi153.stdout:grafana.smithi153 smithi153 *:3000 running (3m) 74s ago 16m 75.6M - 9.4.12 2bacad6d85d8 ee24026ab786 2024-03-21T10:14:53.077 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.iuyluj smithi153 running (10m) 74s ago 10m 128M - 18.2.1 5be31c24972a 02f667cee979 2024-03-21T10:14:53.078 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.zipckr smithi153 running (10m) 74s ago 10m 58.8M - 18.2.1 5be31c24972a 68a853929f8f 2024-03-21T10:14:53.078 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.nhozag smithi174 running (10m) 50s ago 10m 57.5M - 18.2.1 5be31c24972a 2fee4000308f 2024-03-21T10:14:53.078 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.yyimst smithi174 running (10m) 50s ago 10m 117M - 18.2.1 5be31c24972a d78a729220d1 2024-03-21T10:14:53.078 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi153.tluxni smithi153 *:8443,9283,8765 running (6m) 74s ago 19m 475M - 19.0.0-2231-g2edfee68 237e43c7a65f 7ffee0e55382 2024-03-21T10:14:53.078 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi174.qpcwat smithi174 *:8443,9283,8765 running (5m) 50s ago 15m 448M - 19.0.0-2231-g2edfee68 237e43c7a65f 748968f216ab 2024-03-21T10:14:53.078 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi153 smithi153 running (92s) 74s ago 19m 29.0M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f dd8038303dc3 2024-03-21T10:14:53.078 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi174 smithi174 running (56s) 50s ago 15m 20.4M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f 29d78a788f2c 2024-03-21T10:14:53.078 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi153 smithi153 *:9100 running (4m) 74s ago 17m 7814k - 1.5.0 0da6a335fe13 bb835edc0e0b 2024-03-21T10:14:53.078 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi174 smithi174 *:9100 running (4m) 50s ago 16m 7532k - 1.5.0 0da6a335fe13 7c4ecad790fa 2024-03-21T10:14:53.078 INFO:teuthology.orchestra.run.smithi153.stdout:osd.0 smithi153 running (14m) 74s ago 14m 216M 1217M 18.2.1 5be31c24972a 28912a2319b4 2024-03-21T10:14:53.078 INFO:teuthology.orchestra.run.smithi153.stdout:osd.1 smithi153 running (14m) 74s ago 14m 215M 1217M 18.2.1 5be31c24972a ca60681e2314 2024-03-21T10:14:53.078 INFO:teuthology.orchestra.run.smithi153.stdout:osd.2 smithi153 running (13m) 74s ago 13m 173M 1217M 18.2.1 5be31c24972a 8db799f8962a 2024-03-21T10:14:53.078 INFO:teuthology.orchestra.run.smithi153.stdout:osd.3 smithi174 running (13m) 50s ago 13m 232M 2241M 18.2.1 5be31c24972a dafc89e0658d 2024-03-21T10:14:53.079 INFO:teuthology.orchestra.run.smithi153.stdout:osd.4 smithi174 running (12m) 50s ago 12m 224M 2241M 18.2.1 5be31c24972a c4cd15701f05 2024-03-21T10:14:53.079 INFO:teuthology.orchestra.run.smithi153.stdout:osd.5 smithi174 running (12m) 50s ago 12m 202M 2241M 18.2.1 5be31c24972a 3b69886ce22e 2024-03-21T10:14:53.079 INFO:teuthology.orchestra.run.smithi153.stdout:prometheus.smithi153 smithi153 *:9095 running (4m) 74s ago 16m 67.9M - 2.43.0 a07b618ecd1d 7d95561e50c2 2024-03-21T10:14:53.081 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:53.079+0000 7f8f93a00700 1 -- 172.21.15.153:0/150924011 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f8f7406e5a0 msgr2=0x7f8f74070a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:14:53.081 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:53.079+0000 7f8f93a00700 1 --2- 172.21.15.153:0/150924011 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f8f7406e5a0 0x7f8f74070a60 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c197e20 tx=0x7f8f7801a040 comp rx=0 tx=0).stop 2024-03-21T10:14:53.081 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:53.079+0000 7f8f93a00700 1 -- 172.21.15.153:0/150924011 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f8c068670 msgr2=0x7f8f8c196700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:14:53.082 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:53.079+0000 7f8f93a00700 1 --2- 172.21.15.153:0/150924011 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f8c068670 0x7f8f8c196700 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8f8000c8a0 tx=0x7f8f8000cc60 comp rx=0 tx=0).stop 2024-03-21T10:14:53.082 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:53.079+0000 7f8f93a00700 1 -- 172.21.15.153:0/150924011 shutdown_connections 2024-03-21T10:14:53.082 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:53.079+0000 7f8f93a00700 1 --2- 172.21.15.153:0/150924011 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f8f7406e5a0 0x7f8f74070a60 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:14:53.082 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:53.080+0000 7f8f93a00700 1 --2- 172.21.15.153:0/150924011 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f8c068670 0x7f8f8c196700 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:14:53.082 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:53.080+0000 7f8f93a00700 1 --2- 172.21.15.153:0/150924011 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8f8c069020 0x7f8f8c196c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:14:53.082 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:53.080+0000 7f8f93a00700 1 -- 172.21.15.153:0/150924011 >> 172.21.15.153:0/150924011 conn(0x7f8f8c0755a0 msgr2=0x7f8f8c105000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:14:53.082 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:53.080+0000 7f8f93a00700 1 -- 172.21.15.153:0/150924011 shutdown_connections 2024-03-21T10:14:53.082 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:53.080+0000 7f8f93a00700 1 -- 172.21.15.153:0/150924011 wait complete. 2024-03-21T10:14:53.236 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:53.233+0000 7f2629400700 1 -- 172.21.15.153:0/123841530 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2624101750 msgr2=0x7f26241057a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:14:53.236 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:53.233+0000 7f2629400700 1 --2- 172.21.15.153:0/123841530 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2624101750 0x7f26241057a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f260c009910 tx=0x7f260c009c20 comp rx=0 tx=0).stop 2024-03-21T10:14:53.236 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:53.233+0000 7f2629400700 1 -- 172.21.15.153:0/123841530 shutdown_connections 2024-03-21T10:14:53.236 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:53.233+0000 7f2629400700 1 --2- 172.21.15.153:0/123841530 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2624101750 0x7f26241057a0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:14:53.236 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:53.233+0000 7f2629400700 1 --2- 172.21.15.153:0/123841530 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2624100da0 0x7f2624101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:14:53.236 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:53.233+0000 7f2629400700 1 -- 172.21.15.153:0/123841530 >> 172.21.15.153:0/123841530 conn(0x7f26240fc630 msgr2=0x7f26240fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:14:53.237 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:53.233+0000 7f2629400700 1 -- 172.21.15.153:0/123841530 shutdown_connections 2024-03-21T10:14:53.237 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:53.233+0000 7f2629400700 1 -- 172.21.15.153:0/123841530 wait complete. 2024-03-21T10:14:53.237 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:53.234+0000 7f2629400700 1 Processor -- start 2024-03-21T10:14:53.237 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:53.234+0000 7f2629400700 1 -- start start 2024-03-21T10:14:53.239 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:53.234+0000 7f2629400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2624100da0 0x7f262419c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:14:53.239 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:53.234+0000 7f2629400700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2624101750 0x7f262419cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:14:53.239 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:53.234+0000 7f2629400700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f262419d270 con 0x7f2624100da0 2024-03-21T10:14:53.239 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:53.234+0000 7f2629400700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2624196720 con 0x7f2624101750 2024-03-21T10:14:53.239 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:53.235+0000 7f2622800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2624100da0 0x7f262419c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:14:53.239 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:53.235+0000 7f2622800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2624100da0 0x7f262419c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39354/0 (socket says 172.21.15.153:39354) 2024-03-21T10:14:53.239 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:53.235+0000 7f2622800700 1 -- 172.21.15.153:0/904456884 learned_addr learned my addr 172.21.15.153:0/904456884 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:14:53.239 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:53.235+0000 7f261be00700 1 --2- 172.21.15.153:0/904456884 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2624101750 0x7f262419cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:14:53.239 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:53.235+0000 7f2622800700 1 -- 172.21.15.153:0/904456884 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2624101750 msgr2=0x7f262419cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:14:53.240 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:53.235+0000 7f2622800700 1 --2- 172.21.15.153:0/904456884 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2624101750 0x7f262419cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:14:53.240 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:53.235+0000 7f2622800700 1 -- 172.21.15.153:0/904456884 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f260c0095f0 con 0x7f2624100da0 2024-03-21T10:14:53.240 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:53.236+0000 7f2622800700 1 --2- 172.21.15.153:0/904456884 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2624100da0 0x7f262419c6a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f261400b4b0 tx=0x7f261400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:14:53.240 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:53.236+0000 7f261b400700 1 -- 172.21.15.153:0/904456884 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2614011800 con 0x7f2624100da0 2024-03-21T10:14:53.240 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:53.236+0000 7f261b400700 1 -- 172.21.15.153:0/904456884 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2614011e20 con 0x7f2624100da0 2024-03-21T10:14:53.240 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:53.236+0000 7f2629400700 1 -- 172.21.15.153:0/904456884 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2624196a00 con 0x7f2624100da0 2024-03-21T10:14:53.240 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:53.236+0000 7f261b400700 1 -- 172.21.15.153:0/904456884 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f261400f2e0 con 0x7f2624100da0 2024-03-21T10:14:53.240 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:53.236+0000 7f2629400700 1 -- 172.21.15.153:0/904456884 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2624196f50 con 0x7f2624100da0 2024-03-21T10:14:53.240 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:53.237+0000 7f2629400700 1 -- 172.21.15.153:0/904456884 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26241090d0 con 0x7f2624100da0 2024-03-21T10:14:53.241 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:53.238+0000 7f261b400700 1 -- 172.21.15.153:0/904456884 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 37) v1 ==== 92321+0+0 (secure 0 0 0) 0x7f2614010380 con 0x7f2624100da0 2024-03-21T10:14:53.241 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:53.239+0000 7f261b400700 1 --2- 172.21.15.153:0/904456884 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f260806e590 0x7f2608070a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:14:53.241 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:53.239+0000 7f261b400700 1 -- 172.21.15.153:0/904456884 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f261408e100 con 0x7f2624100da0 2024-03-21T10:14:53.241 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:53.239+0000 7f261be00700 1 --2- 172.21.15.153:0/904456884 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f260806e590 0x7f2608070a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:14:53.241 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:53.240+0000 7f261be00700 1 --2- 172.21.15.153:0/904456884 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f260806e590 0x7f2608070a50 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2624197b90 tx=0x7f260c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:14:53.245 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:53.244+0000 7f261b400700 1 -- 172.21.15.153:0/904456884 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f261405ad80 con 0x7f2624100da0 2024-03-21T10:14:53.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:53 smithi174 ceph-mon[88004]: Upgrade: Setting container_image for all mon 2024-03-21T10:14:53.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:53 smithi174 ceph-mon[88004]: from='client.34186 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:14:53.553 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:53.551+0000 7f2629400700 1 -- 172.21.15.153:0/904456884 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f2624197290 con 0x7f2624100da0 2024-03-21T10:14:53.554 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:53.553+0000 7f261b400700 1 -- 172.21.15.153:0/904456884 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+688 (secure 0 0 0) 0x7f261405a910 con 0x7f2624100da0 2024-03-21T10:14:53.554 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:14:53.555 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": { 2024-03-21T10:14:53.555 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:14:53.555 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:14:53.555 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": { 2024-03-21T10:14:53.555 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:14:53.555 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:14:53.555 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": { 2024-03-21T10:14:53.555 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-03-21T10:14:53.555 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:14:53.555 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": { 2024-03-21T10:14:53.555 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-03-21T10:14:53.555 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:14:53.555 INFO:teuthology.orchestra.run.smithi153.stdout: "overall": { 2024-03-21T10:14:53.555 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 10, 2024-03-21T10:14:53.555 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 4 2024-03-21T10:14:53.556 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-03-21T10:14:53.556 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:14:53.557 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:53.555+0000 7f2629400700 1 -- 172.21.15.153:0/904456884 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f260806e590 msgr2=0x7f2608070a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:14:53.557 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:53.555+0000 7f2629400700 1 --2- 172.21.15.153:0/904456884 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f260806e590 0x7f2608070a50 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2624197b90 tx=0x7f260c01a040 comp rx=0 tx=0).stop 2024-03-21T10:14:53.557 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:53.555+0000 7f2629400700 1 -- 172.21.15.153:0/904456884 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2624100da0 msgr2=0x7f262419c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:14:53.558 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:53.555+0000 7f2629400700 1 --2- 172.21.15.153:0/904456884 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2624100da0 0x7f262419c6a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f261400b4b0 tx=0x7f261400b870 comp rx=0 tx=0).stop 2024-03-21T10:14:53.558 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:53.556+0000 7f2629400700 1 -- 172.21.15.153:0/904456884 shutdown_connections 2024-03-21T10:14:53.558 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:53.556+0000 7f2629400700 1 --2- 172.21.15.153:0/904456884 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f260806e590 0x7f2608070a50 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:14:53.558 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:53.556+0000 7f2629400700 1 --2- 172.21.15.153:0/904456884 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2624100da0 0x7f262419c6a0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:14:53.558 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:53.556+0000 7f2629400700 1 --2- 172.21.15.153:0/904456884 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2624101750 0x7f262419cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:14:53.558 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:53.556+0000 7f2629400700 1 -- 172.21.15.153:0/904456884 >> 172.21.15.153:0/904456884 conn(0x7f26240fc630 msgr2=0x7f26240fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:14:53.558 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:53.556+0000 7f2629400700 1 -- 172.21.15.153:0/904456884 shutdown_connections 2024-03-21T10:14:53.558 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:53.556+0000 7f2629400700 1 -- 172.21.15.153:0/904456884 wait complete. 2024-03-21T10:14:53.709 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:53.708+0000 7f2736800700 1 -- 172.21.15.153:0/2062886753 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2730100b50 msgr2=0x7f2730100fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:14:53.710 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:53.708+0000 7f2736800700 1 --2- 172.21.15.153:0/2062886753 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2730100b50 0x7f2730100fd0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f271c009910 tx=0x7f271c009c20 comp rx=0 tx=0).stop 2024-03-21T10:14:53.710 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:53.708+0000 7f2736800700 1 -- 172.21.15.153:0/2062886753 shutdown_connections 2024-03-21T10:14:53.710 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:53.708+0000 7f2736800700 1 --2- 172.21.15.153:0/2062886753 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2730100b50 0x7f2730100fd0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:14:53.710 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:53.708+0000 7f2736800700 1 --2- 172.21.15.153:0/2062886753 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2730106c00 0x7f2730106fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:14:53.710 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:53.708+0000 7f2736800700 1 -- 172.21.15.153:0/2062886753 >> 172.21.15.153:0/2062886753 conn(0x7f27300fc640 msgr2=0x7f27300fea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:14:53.710 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:53.708+0000 7f2736800700 1 -- 172.21.15.153:0/2062886753 shutdown_connections 2024-03-21T10:14:53.710 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:53.708+0000 7f2736800700 1 -- 172.21.15.153:0/2062886753 wait complete. 2024-03-21T10:14:53.711 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:53.709+0000 7f2736800700 1 Processor -- start 2024-03-21T10:14:53.711 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:53.709+0000 7f2736800700 1 -- start start 2024-03-21T10:14:53.711 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:53.710+0000 7f2736800700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2730100b50 0x7f273019e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:14:53.712 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:53.710+0000 7f2736800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2730106c00 0x7f273019eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:14:53.712 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:53.710+0000 7f2736800700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f273019f230 con 0x7f2730106c00 2024-03-21T10:14:53.712 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:53.710+0000 7f2736800700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27301986e0 con 0x7f2730100b50 2024-03-21T10:14:53.712 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:53.710+0000 7f272ea00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2730106c00 0x7f273019eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:14:53.712 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:53.710+0000 7f272ea00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2730106c00 0x7f273019eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39362/0 (socket says 172.21.15.153:39362) 2024-03-21T10:14:53.712 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:53.710+0000 7f272ea00700 1 -- 172.21.15.153:0/731803126 learned_addr learned my addr 172.21.15.153:0/731803126 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:14:53.713 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:53.710+0000 7f272f400700 1 --2- 172.21.15.153:0/731803126 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2730100b50 0x7f273019e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:14:53.713 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:53.711+0000 7f272ea00700 1 -- 172.21.15.153:0/731803126 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2730100b50 msgr2=0x7f273019e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:14:53.713 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:53.711+0000 7f272ea00700 1 --2- 172.21.15.153:0/731803126 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2730100b50 0x7f273019e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:14:53.713 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:53.711+0000 7f272ea00700 1 -- 172.21.15.153:0/731803126 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f271c0095f0 con 0x7f2730106c00 2024-03-21T10:14:53.713 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:53.711+0000 7f272ea00700 1 --2- 172.21.15.153:0/731803126 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2730106c00 0x7f273019eba0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f271c006010 tx=0x7f271c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:14:53.713 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:53.711+0000 7f2727e00700 1 -- 172.21.15.153:0/731803126 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f271c01d070 con 0x7f2730106c00 2024-03-21T10:14:53.713 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:53.711+0000 7f2727e00700 1 -- 172.21.15.153:0/731803126 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f271c00f460 con 0x7f2730106c00 2024-03-21T10:14:53.713 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:53.711+0000 7f2736800700 1 -- 172.21.15.153:0/731803126 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2730198960 con 0x7f2730106c00 2024-03-21T10:14:53.714 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:53.711+0000 7f2736800700 1 -- 172.21.15.153:0/731803126 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2730198ed0 con 0x7f2730106c00 2024-03-21T10:14:53.714 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:53.712+0000 7f2727e00700 1 -- 172.21.15.153:0/731803126 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f271c017630 con 0x7f2730106c00 2024-03-21T10:14:53.714 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:53.713+0000 7f2736800700 1 -- 172.21.15.153:0/731803126 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f273004ea90 con 0x7f2730106c00 2024-03-21T10:14:53.715 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:53.714+0000 7f2727e00700 1 -- 172.21.15.153:0/731803126 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 37) v1 ==== 92321+0+0 (secure 0 0 0) 0x7f271c00f5d0 con 0x7f2730106c00 2024-03-21T10:14:53.716 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:53.715+0000 7f2727e00700 1 --2- 172.21.15.153:0/731803126 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f271406e5a0 0x7f2714070a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:14:53.716 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:53.715+0000 7f2727e00700 1 -- 172.21.15.153:0/731803126 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f271c08faf0 con 0x7f2730106c00 2024-03-21T10:14:53.717 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:53.715+0000 7f272f400700 1 --2- 172.21.15.153:0/731803126 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f271406e5a0 0x7f2714070a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:14:53.717 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:53.716+0000 7f272f400700 1 --2- 172.21.15.153:0/731803126 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f271406e5a0 0x7f2714070a60 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2720005950 tx=0x7f27200058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:14:53.721 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:53.719+0000 7f2727e00700 1 -- 172.21.15.153:0/731803126 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f271c05c5c0 con 0x7f2730106c00 2024-03-21T10:14:53.986 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:53.984+0000 7f2736800700 1 -- 172.21.15.153:0/731803126 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f2730199240 con 0x7f2730106c00 2024-03-21T10:14:53.986 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:53.984+0000 7f2727e00700 1 -- 172.21.15.153:0/731803126 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 21 v21) v1 ==== 76+0+1903 (secure 0 0 0) 0x7f271c05c150 con 0x7f2730106c00 2024-03-21T10:14:53.987 INFO:teuthology.orchestra.run.smithi153.stdout:e21 2024-03-21T10:14:53.987 INFO:teuthology.orchestra.run.smithi153.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-21T10:14:53.987 INFO:teuthology.orchestra.run.smithi153.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:14:53.987 INFO:teuthology.orchestra.run.smithi153.stdout:legacy client fscid: 1 2024-03-21T10:14:53.987 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:14:53.987 INFO:teuthology.orchestra.run.smithi153.stdout:Filesystem 'cephfs' (1) 2024-03-21T10:14:53.987 INFO:teuthology.orchestra.run.smithi153.stdout:fs_name cephfs 2024-03-21T10:14:53.987 INFO:teuthology.orchestra.run.smithi153.stdout:epoch 21 2024-03-21T10:14:53.987 INFO:teuthology.orchestra.run.smithi153.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-21T10:14:53.987 INFO:teuthology.orchestra.run.smithi153.stdout:created 2024-03-21T10:04:27.086576+0000 2024-03-21T10:14:53.987 INFO:teuthology.orchestra.run.smithi153.stdout:modified 2024-03-21T10:13:01.455594+0000 2024-03-21T10:14:53.987 INFO:teuthology.orchestra.run.smithi153.stdout:tableserver 0 2024-03-21T10:14:53.988 INFO:teuthology.orchestra.run.smithi153.stdout:root 0 2024-03-21T10:14:53.988 INFO:teuthology.orchestra.run.smithi153.stdout:session_timeout 60 2024-03-21T10:14:53.988 INFO:teuthology.orchestra.run.smithi153.stdout:session_autoclose 300 2024-03-21T10:14:53.988 INFO:teuthology.orchestra.run.smithi153.stdout:max_file_size 1099511627776 2024-03-21T10:14:53.988 INFO:teuthology.orchestra.run.smithi153.stdout:max_xattr_size 65536 2024-03-21T10:14:53.988 INFO:teuthology.orchestra.run.smithi153.stdout:required_client_features {} 2024-03-21T10:14:53.988 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure 0 2024-03-21T10:14:53.988 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure_osd_epoch 47 2024-03-21T10:14:53.988 INFO:teuthology.orchestra.run.smithi153.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:14:53.988 INFO:teuthology.orchestra.run.smithi153.stdout:max_mds 2 2024-03-21T10:14:53.988 INFO:teuthology.orchestra.run.smithi153.stdout:in 0,1 2024-03-21T10:14:53.988 INFO:teuthology.orchestra.run.smithi153.stdout:up {0=14484,1=24273} 2024-03-21T10:14:53.988 INFO:teuthology.orchestra.run.smithi153.stdout:failed 2024-03-21T10:14:53.988 INFO:teuthology.orchestra.run.smithi153.stdout:damaged 2024-03-21T10:14:53.989 INFO:teuthology.orchestra.run.smithi153.stdout:stopped 2024-03-21T10:14:53.989 INFO:teuthology.orchestra.run.smithi153.stdout:data_pools [3] 2024-03-21T10:14:53.989 INFO:teuthology.orchestra.run.smithi153.stdout:metadata_pool 2 2024-03-21T10:14:53.989 INFO:teuthology.orchestra.run.smithi153.stdout:inline_data enabled 2024-03-21T10:14:53.989 INFO:teuthology.orchestra.run.smithi153.stdout:balancer 2024-03-21T10:14:53.989 INFO:teuthology.orchestra.run.smithi153.stdout:bal_rank_mask -1 2024-03-21T10:14:53.989 INFO:teuthology.orchestra.run.smithi153.stdout:standby_count_wanted 1 2024-03-21T10:14:53.989 INFO:teuthology.orchestra.run.smithi153.stdout:qdb_cluster leader: 0 members: 2024-03-21T10:14:53.989 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.iuyluj{0:14484} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.153:6828/866138999,v1:172.21.15.153:6829/866138999] compat {c=[1],r=[1],i=[7ff]}] 2024-03-21T10:14:53.989 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.zipckr{0:14494} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.153:6826/4033622615,v1:172.21.15.153:6827/4033622615] compat {c=[1],r=[1],i=[7ff]}] 2024-03-21T10:14:53.989 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.yyimst{1:24273} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.174:6824/1776947618,v1:172.21.15.174:6825/1776947618] compat {c=[1],r=[1],i=[7ff]}] 2024-03-21T10:14:53.989 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.nhozag{1:24285} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.174:6826/806095873,v1:172.21.15.174:6827/806095873] compat {c=[1],r=[1],i=[7ff]}] 2024-03-21T10:14:53.989 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:14:53.989 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:14:53.990 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:53.987+0000 7f2736800700 1 -- 172.21.15.153:0/731803126 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f271406e5a0 msgr2=0x7f2714070a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:14:53.990 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:53.987+0000 7f2736800700 1 --2- 172.21.15.153:0/731803126 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f271406e5a0 0x7f2714070a60 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2720005950 tx=0x7f27200058e0 comp rx=0 tx=0).stop 2024-03-21T10:14:53.990 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:53.987+0000 7f2736800700 1 -- 172.21.15.153:0/731803126 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2730106c00 msgr2=0x7f273019eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:14:53.990 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:53.987+0000 7f2736800700 1 --2- 172.21.15.153:0/731803126 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2730106c00 0x7f273019eba0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f271c006010 tx=0x7f271c00be20 comp rx=0 tx=0).stop 2024-03-21T10:14:53.991 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:53.987+0000 7f2736800700 1 -- 172.21.15.153:0/731803126 shutdown_connections 2024-03-21T10:14:53.991 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:53.988+0000 7f2736800700 1 --2- 172.21.15.153:0/731803126 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2730100b50 0x7f273019e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:14:53.991 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:53.988+0000 7f2736800700 1 --2- 172.21.15.153:0/731803126 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f271406e5a0 0x7f2714070a60 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:14:53.991 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:53.988+0000 7f2736800700 1 --2- 172.21.15.153:0/731803126 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2730106c00 0x7f273019eba0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:14:53.991 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:53.988+0000 7f2736800700 1 -- 172.21.15.153:0/731803126 >> 172.21.15.153:0/731803126 conn(0x7f27300fc640 msgr2=0x7f27300fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:14:53.991 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:53.988+0000 7f2736800700 1 -- 172.21.15.153:0/731803126 shutdown_connections 2024-03-21T10:14:53.991 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:53.988+0000 7f2736800700 1 -- 172.21.15.153:0/731803126 wait complete. 2024-03-21T10:14:53.991 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 21 2024-03-21T10:14:54.136 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:54.134+0000 7f0b70600700 1 -- 172.21.15.153:0/2907870008 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0b68100da0 msgr2=0x7f0b68101180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:14:54.136 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:54.134+0000 7f0b70600700 1 --2- 172.21.15.153:0/2907870008 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0b68100da0 0x7f0b68101180 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c009910 tx=0x7f0b5c009c20 comp rx=0 tx=0).stop 2024-03-21T10:14:54.136 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:54.134+0000 7f0b70600700 1 -- 172.21.15.153:0/2907870008 shutdown_connections 2024-03-21T10:14:54.136 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:54.134+0000 7f0b70600700 1 --2- 172.21.15.153:0/2907870008 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b68101750 0x7f0b681057a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:14:54.136 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:54.134+0000 7f0b70600700 1 --2- 172.21.15.153:0/2907870008 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0b68100da0 0x7f0b68101180 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:14:54.136 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:54.134+0000 7f0b70600700 1 -- 172.21.15.153:0/2907870008 >> 172.21.15.153:0/2907870008 conn(0x7f0b680fc630 msgr2=0x7f0b680fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:14:54.136 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:54.134+0000 7f0b70600700 1 -- 172.21.15.153:0/2907870008 shutdown_connections 2024-03-21T10:14:54.137 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:54.134+0000 7f0b70600700 1 -- 172.21.15.153:0/2907870008 wait complete. 2024-03-21T10:14:54.137 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:54.135+0000 7f0b70600700 1 Processor -- start 2024-03-21T10:14:54.137 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:54.135+0000 7f0b70600700 1 -- start start 2024-03-21T10:14:54.137 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:54.136+0000 7f0b70600700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0b68100da0 0x7f0b6819e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:14:54.137 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:54.136+0000 7f0b70600700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b68101750 0x7f0b6819ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:14:54.137 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:54.136+0000 7f0b70600700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b6819f400 con 0x7f0b68100da0 2024-03-21T10:14:54.138 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:54.136+0000 7f0b70600700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b68198920 con 0x7f0b68101750 2024-03-21T10:14:54.138 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:54.136+0000 7f0b6dc00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0b68100da0 0x7f0b6819e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:14:54.138 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:54.136+0000 7f0b6dc00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0b68100da0 0x7f0b6819e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39370/0 (socket says 172.21.15.153:39370) 2024-03-21T10:14:54.139 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:54.136+0000 7f0b6dc00700 1 -- 172.21.15.153:0/2574107891 learned_addr learned my addr 172.21.15.153:0/2574107891 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:14:54.139 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:54.136+0000 7f0b6d200700 1 --2- 172.21.15.153:0/2574107891 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b68101750 0x7f0b6819ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:14:54.139 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:54.137+0000 7f0b6dc00700 1 -- 172.21.15.153:0/2574107891 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b68101750 msgr2=0x7f0b6819ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:14:54.139 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:54.137+0000 7f0b6dc00700 1 --2- 172.21.15.153:0/2574107891 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b68101750 0x7f0b6819ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:14:54.139 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:54.137+0000 7f0b6dc00700 1 -- 172.21.15.153:0/2574107891 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b5c0095f0 con 0x7f0b68100da0 2024-03-21T10:14:54.139 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:54.137+0000 7f0b6d200700 1 --2- 172.21.15.153:0/2574107891 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b68101750 0x7f0b6819ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:14:54.139 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:54.137+0000 7f0b6dc00700 1 --2- 172.21.15.153:0/2574107891 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0b68100da0 0x7f0b6819e8a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c00b5c0 tx=0x7f0b5c00be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:14:54.139 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:54.137+0000 7f0b66000700 1 -- 172.21.15.153:0/2574107891 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0b5c01d070 con 0x7f0b68100da0 2024-03-21T10:14:54.140 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:54.137+0000 7f0b66000700 1 -- 172.21.15.153:0/2574107891 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0b5c00f460 con 0x7f0b68100da0 2024-03-21T10:14:54.140 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:54.137+0000 7f0b70600700 1 -- 172.21.15.153:0/2574107891 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b68198ba0 con 0x7f0b68100da0 2024-03-21T10:14:54.140 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:54.137+0000 7f0b66000700 1 -- 172.21.15.153:0/2574107891 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0b5c017610 con 0x7f0b68100da0 2024-03-21T10:14:54.140 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:54.137+0000 7f0b70600700 1 -- 172.21.15.153:0/2574107891 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b68199090 con 0x7f0b68100da0 2024-03-21T10:14:54.140 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:54.139+0000 7f0b70600700 1 -- 172.21.15.153:0/2574107891 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b68104f20 con 0x7f0b68100da0 2024-03-21T10:14:54.141 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:54.140+0000 7f0b66000700 1 -- 172.21.15.153:0/2574107891 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 37) v1 ==== 92321+0+0 (secure 0 0 0) 0x7f0b5c00f5d0 con 0x7f0b68100da0 2024-03-21T10:14:54.142 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:54.140+0000 7f0b66000700 1 --2- 172.21.15.153:0/2574107891 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f0b5006e590 0x7f0b50070a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:14:54.142 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:54.141+0000 7f0b66000700 1 -- 172.21.15.153:0/2574107891 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f0b5c08fc80 con 0x7f0b68100da0 2024-03-21T10:14:54.142 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:54.141+0000 7f0b6d200700 1 --2- 172.21.15.153:0/2574107891 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f0b5006e590 0x7f0b50070a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:14:54.143 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:54.141+0000 7f0b6d200700 1 --2- 172.21.15.153:0/2574107891 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f0b5006e590 0x7f0b50070a50 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0b6819a080 tx=0x7f0b54008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:14:54.147 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:54.145+0000 7f0b66000700 1 -- 172.21.15.153:0/2574107891 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0b5c05c750 con 0x7f0b68100da0 2024-03-21T10:14:54.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:54 smithi153 ceph-mon[113733]: from='client.34190 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:14:54.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:54 smithi153 ceph-mon[113733]: from='client.34194 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:14:54.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:54 smithi153 ceph-mon[113733]: pgmap v44: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:14:54.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:54 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/904456884' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:14:54.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:54 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:14:54.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:54 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi153", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-21T10:14:54.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:54 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:14:54.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:54 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/731803126' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:14:54.384 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:54.382+0000 7f0b70600700 1 -- 172.21.15.153:0/2574107891 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0b68199370 con 0x7f0b5006e590 2024-03-21T10:14:54.386 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:54.384+0000 7f0b66000700 1 -- 172.21.15.153:0/2574107891 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+385 (secure 0 0 0) 0x7f0b68199370 con 0x7f0b5006e590 2024-03-21T10:14:54.386 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:14:54.386 INFO:teuthology.orchestra.run.smithi153.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232", 2024-03-21T10:14:54.386 INFO:teuthology.orchestra.run.smithi153.stdout: "in_progress": true, 2024-03-21T10:14:54.386 INFO:teuthology.orchestra.run.smithi153.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-21T10:14:54.386 INFO:teuthology.orchestra.run.smithi153.stdout: "services_complete": [ 2024-03-21T10:14:54.387 INFO:teuthology.orchestra.run.smithi153.stdout: "mon", 2024-03-21T10:14:54.387 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr" 2024-03-21T10:14:54.387 INFO:teuthology.orchestra.run.smithi153.stdout: ], 2024-03-21T10:14:54.387 INFO:teuthology.orchestra.run.smithi153.stdout: "progress": "4/23 daemons upgraded", 2024-03-21T10:14:54.387 INFO:teuthology.orchestra.run.smithi153.stdout: "message": "Currently upgrading crash daemons", 2024-03-21T10:14:54.387 INFO:teuthology.orchestra.run.smithi153.stdout: "is_paused": false 2024-03-21T10:14:54.387 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:14:54.389 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:54.387+0000 7f0b70600700 1 -- 172.21.15.153:0/2574107891 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f0b5006e590 msgr2=0x7f0b50070a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:14:54.389 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:54.387+0000 7f0b70600700 1 --2- 172.21.15.153:0/2574107891 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f0b5006e590 0x7f0b50070a50 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0b6819a080 tx=0x7f0b54008040 comp rx=0 tx=0).stop 2024-03-21T10:14:54.389 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:54.387+0000 7f0b70600700 1 -- 172.21.15.153:0/2574107891 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0b68100da0 msgr2=0x7f0b6819e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:14:54.390 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:54.387+0000 7f0b70600700 1 --2- 172.21.15.153:0/2574107891 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0b68100da0 0x7f0b6819e8a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c00b5c0 tx=0x7f0b5c00be00 comp rx=0 tx=0).stop 2024-03-21T10:14:54.390 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:54.388+0000 7f0b70600700 1 -- 172.21.15.153:0/2574107891 shutdown_connections 2024-03-21T10:14:54.390 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:54.388+0000 7f0b70600700 1 --2- 172.21.15.153:0/2574107891 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f0b5006e590 0x7f0b50070a50 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:14:54.390 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:54.388+0000 7f0b70600700 1 --2- 172.21.15.153:0/2574107891 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0b68100da0 0x7f0b6819e8a0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:14:54.390 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:54.388+0000 7f0b70600700 1 --2- 172.21.15.153:0/2574107891 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b68101750 0x7f0b6819ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:14:54.390 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:54.388+0000 7f0b70600700 1 -- 172.21.15.153:0/2574107891 >> 172.21.15.153:0/2574107891 conn(0x7f0b680fc630 msgr2=0x7f0b680fdcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:14:54.390 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:54.388+0000 7f0b70600700 1 -- 172.21.15.153:0/2574107891 shutdown_connections 2024-03-21T10:14:54.390 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:54.388+0000 7f0b70600700 1 -- 172.21.15.153:0/2574107891 wait complete. 2024-03-21T10:14:54.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:54 smithi174 ceph-mon[88004]: from='client.34190 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:14:54.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:54 smithi174 ceph-mon[88004]: from='client.34194 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:14:54.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:54 smithi174 ceph-mon[88004]: pgmap v44: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:14:54.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:54 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/904456884' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:14:54.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:54 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:14:54.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:54 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi153", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-21T10:14:54.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:54 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:14:54.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:54 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/731803126' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:14:54.542 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:54.540+0000 7f65b0800700 1 -- 172.21.15.153:0/791344943 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f65a8101750 msgr2=0x7f65a81057a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:14:54.542 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:54.540+0000 7f65b0800700 1 --2- 172.21.15.153:0/791344943 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f65a8101750 0x7f65a81057a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6598009910 tx=0x7f6598009c20 comp rx=0 tx=0).stop 2024-03-21T10:14:54.542 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:54.540+0000 7f65b0800700 1 -- 172.21.15.153:0/791344943 shutdown_connections 2024-03-21T10:14:54.542 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:54.540+0000 7f65b0800700 1 --2- 172.21.15.153:0/791344943 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f65a8101750 0x7f65a81057a0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:14:54.542 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:54.540+0000 7f65b0800700 1 --2- 172.21.15.153:0/791344943 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f65a8100da0 0x7f65a8101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:14:54.542 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:54.540+0000 7f65b0800700 1 -- 172.21.15.153:0/791344943 >> 172.21.15.153:0/791344943 conn(0x7f65a80fc630 msgr2=0x7f65a80fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:14:54.543 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:54.540+0000 7f65b0800700 1 -- 172.21.15.153:0/791344943 shutdown_connections 2024-03-21T10:14:54.543 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:54.540+0000 7f65b0800700 1 -- 172.21.15.153:0/791344943 wait complete. 2024-03-21T10:14:54.544 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:54.541+0000 7f65b0800700 1 Processor -- start 2024-03-21T10:14:54.544 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:54.541+0000 7f65b0800700 1 -- start start 2024-03-21T10:14:54.544 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:54.542+0000 7f65b0800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f65a8100da0 0x7f65a8073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:14:54.544 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:54.542+0000 7f65b0800700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f65a8101750 0x7f65a80735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:14:54.544 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:54.542+0000 7f65b0800700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a8073b10 con 0x7f65a8100da0 2024-03-21T10:14:54.544 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:54.542+0000 7f65b0800700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a8073c50 con 0x7f65a8101750 2024-03-21T10:14:54.544 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:54.542+0000 7f65ade00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f65a8100da0 0x7f65a8073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:14:54.544 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:54.542+0000 7f65ade00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f65a8100da0 0x7f65a8073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39390/0 (socket says 172.21.15.153:39390) 2024-03-21T10:14:54.544 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:54.542+0000 7f65ade00700 1 -- 172.21.15.153:0/239905196 learned_addr learned my addr 172.21.15.153:0/239905196 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:14:54.544 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:54.542+0000 7f65ad400700 1 --2- 172.21.15.153:0/239905196 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f65a8101750 0x7f65a80735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:14:54.545 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:54.542+0000 7f65ade00700 1 -- 172.21.15.153:0/239905196 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f65a8101750 msgr2=0x7f65a80735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:14:54.545 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:54.542+0000 7f65ade00700 1 --2- 172.21.15.153:0/239905196 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f65a8101750 0x7f65a80735d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:14:54.545 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:54.542+0000 7f65ade00700 1 -- 172.21.15.153:0/239905196 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65980095f0 con 0x7f65a8100da0 2024-03-21T10:14:54.545 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:54.543+0000 7f65ade00700 1 --2- 172.21.15.153:0/239905196 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f65a8100da0 0x7f65a8073090 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f659c00c8a0 tx=0x7f659c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:14:54.545 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:54.543+0000 7f65a2a00700 1 -- 172.21.15.153:0/239905196 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f659c00cea0 con 0x7f65a8100da0 2024-03-21T10:14:54.546 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:54.543+0000 7f65a2a00700 1 -- 172.21.15.153:0/239905196 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f659c004830 con 0x7f65a8100da0 2024-03-21T10:14:54.546 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:54.543+0000 7f65a2a00700 1 -- 172.21.15.153:0/239905196 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f659c0055b0 con 0x7f65a8100da0 2024-03-21T10:14:54.546 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:54.543+0000 7f65b0800700 1 -- 172.21.15.153:0/239905196 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65a81a2900 con 0x7f65a8100da0 2024-03-21T10:14:54.546 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:54.543+0000 7f65b0800700 1 -- 172.21.15.153:0/239905196 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65a81a2c60 con 0x7f65a8100da0 2024-03-21T10:14:54.546 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:54.545+0000 7f65b0800700 1 -- 172.21.15.153:0/239905196 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65a81091a0 con 0x7f65a8100da0 2024-03-21T10:14:54.548 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:54.547+0000 7f65a2a00700 1 -- 172.21.15.153:0/239905196 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 37) v1 ==== 92321+0+0 (secure 0 0 0) 0x7f659c0049a0 con 0x7f65a8100da0 2024-03-21T10:14:54.549 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:54.548+0000 7f65a2a00700 1 --2- 172.21.15.153:0/239905196 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f659006e590 0x7f6590070a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:14:54.549 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:54.548+0000 7f65a2a00700 1 -- 172.21.15.153:0/239905196 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f659c08e340 con 0x7f65a8100da0 2024-03-21T10:14:54.550 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:54.548+0000 7f65ad400700 1 --2- 172.21.15.153:0/239905196 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f659006e590 0x7f6590070a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:14:54.550 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:54.549+0000 7f65ad400700 1 --2- 172.21.15.153:0/239905196 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f659006e590 0x7f6590070a50 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f65a8074810 tx=0x7f659801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:14:54.555 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:54.554+0000 7f65a2a00700 1 -- 172.21.15.153:0/239905196 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f659c0572e0 con 0x7f65a8100da0 2024-03-21T10:14:54.859 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:54.857+0000 7f65b0800700 1 -- 172.21.15.153:0/239905196 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f65a804ea90 con 0x7f65a8100da0 2024-03-21T10:14:54.860 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:54.858+0000 7f65a2a00700 1 -- 172.21.15.153:0/239905196 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f659c004c50 con 0x7f65a8100da0 2024-03-21T10:14:54.860 INFO:teuthology.orchestra.run.smithi153.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-03-21T10:14:54.860 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-21T10:14:54.860 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-21T10:14:54.862 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:54.860+0000 7f65b0800700 1 -- 172.21.15.153:0/239905196 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f659006e590 msgr2=0x7f6590070a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:14:54.863 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:54.860+0000 7f65b0800700 1 --2- 172.21.15.153:0/239905196 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f659006e590 0x7f6590070a50 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f65a8074810 tx=0x7f659801a040 comp rx=0 tx=0).stop 2024-03-21T10:14:54.863 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:54.860+0000 7f65b0800700 1 -- 172.21.15.153:0/239905196 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f65a8100da0 msgr2=0x7f65a8073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:14:54.863 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:54.860+0000 7f65b0800700 1 --2- 172.21.15.153:0/239905196 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f65a8100da0 0x7f65a8073090 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f659c00c8a0 tx=0x7f659c00cc60 comp rx=0 tx=0).stop 2024-03-21T10:14:54.863 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:54.861+0000 7f65b0800700 1 -- 172.21.15.153:0/239905196 shutdown_connections 2024-03-21T10:14:54.863 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:54.861+0000 7f65b0800700 1 --2- 172.21.15.153:0/239905196 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f659006e590 0x7f6590070a50 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:14:54.863 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:54.861+0000 7f65b0800700 1 --2- 172.21.15.153:0/239905196 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f65a8100da0 0x7f65a8073090 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:14:54.863 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:54.861+0000 7f65b0800700 1 --2- 172.21.15.153:0/239905196 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f65a8101750 0x7f65a80735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:14:54.863 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:54.861+0000 7f65b0800700 1 -- 172.21.15.153:0/239905196 >> 172.21.15.153:0/239905196 conn(0x7f65a80fc630 msgr2=0x7f65a80fea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:14:54.863 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:54.861+0000 7f65b0800700 1 -- 172.21.15.153:0/239905196 shutdown_connections 2024-03-21T10:14:54.863 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:14:54.861+0000 7f65b0800700 1 -- 172.21.15.153:0/239905196 wait complete. 2024-03-21T10:14:55.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:55 smithi153 ceph-mon[113733]: Upgrade: Updating crash.smithi153 (1/2) 2024-03-21T10:14:55.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:55 smithi153 ceph-mon[113733]: Deploying daemon crash.smithi153 on smithi153 2024-03-21T10:14:55.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:55 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/239905196' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:14:55.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:55 smithi174 ceph-mon[88004]: Upgrade: Updating crash.smithi153 (1/2) 2024-03-21T10:14:55.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:55 smithi174 ceph-mon[88004]: Deploying daemon crash.smithi153 on smithi153 2024-03-21T10:14:55.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:55 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/239905196' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:14:56.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:56 smithi153 ceph-mon[113733]: from='client.34206 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:14:56.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:56 smithi153 ceph-mon[113733]: pgmap v45: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:14:56.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:56 smithi174 ceph-mon[88004]: from='client.34206 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:14:56.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:56 smithi174 ceph-mon[88004]: pgmap v45: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:14:58.460 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:14:58 smithi153 ceph-mon[113733]: pgmap v46: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:14:58.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:14:58 smithi174 ceph-mon[88004]: pgmap v46: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:15:00.503 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:00 smithi174 ceph-mon[88004]: pgmap v47: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:15:00.504 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:00 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:15:00.504 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:00 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:15:00.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:00 smithi153 ceph-mon[113733]: pgmap v47: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:15:00.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:00 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:15:00.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:00 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:15:01.416 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:01 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:15:01.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:01 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:15:02.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:02 smithi153 ceph-mon[113733]: pgmap v48: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:15:02.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:02 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:15:02.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:02 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi174", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-21T10:15:02.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:02 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:15:02.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:02 smithi174 ceph-mon[88004]: pgmap v48: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:15:02.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:02 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:15:02.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:02 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi174", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-21T10:15:02.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:02 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:15:03.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:03 smithi174 ceph-mon[88004]: Upgrade: Updating crash.smithi174 (2/2) 2024-03-21T10:15:03.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:03 smithi174 ceph-mon[88004]: Deploying daemon crash.smithi174 on smithi174 2024-03-21T10:15:03.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:03 smithi153 ceph-mon[113733]: Upgrade: Updating crash.smithi174 (2/2) 2024-03-21T10:15:03.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:03 smithi153 ceph-mon[113733]: Deploying daemon crash.smithi174 on smithi174 2024-03-21T10:15:04.434 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:04 smithi174 ceph-mon[88004]: pgmap v49: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.7 KiB/s rd, 511 B/s wr, 5 op/s 2024-03-21T10:15:04.434 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:04 smithi174 ceph-mon[88004]: mgrmap e38: smithi153.tluxni(active, since 92s), standbys: smithi174.qpcwat 2024-03-21T10:15:04.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:04 smithi153 ceph-mon[113733]: pgmap v49: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.7 KiB/s rd, 511 B/s wr, 5 op/s 2024-03-21T10:15:04.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:04 smithi153 ceph-mon[113733]: mgrmap e38: smithi153.tluxni(active, since 92s), standbys: smithi174.qpcwat 2024-03-21T10:15:05.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:05 smithi174 ceph-mon[88004]: pgmap v50: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 511 B/s wr, 3 op/s 2024-03-21T10:15:05.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:05 smithi153 ceph-mon[113733]: pgmap v50: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 511 B/s wr, 3 op/s 2024-03-21T10:15:08.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:08 smithi174 ceph-mon[88004]: pgmap v51: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 511 B/s wr, 4 op/s 2024-03-21T10:15:08.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:08 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:15:08.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:08 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:15:08.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:08 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:15:08.453 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:08 smithi153 ceph-mon[113733]: pgmap v51: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 511 B/s wr, 4 op/s 2024-03-21T10:15:08.453 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:08 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:15:08.454 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:08 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:15:08.454 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:08 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:15:10.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:10 smithi174 ceph-mon[88004]: pgmap v52: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 511 B/s wr, 5 op/s 2024-03-21T10:15:10.492 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:10 smithi153 ceph-mon[113733]: pgmap v52: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 511 B/s wr, 5 op/s 2024-03-21T10:15:12.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:12 smithi174 ceph-mon[88004]: pgmap v53: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 511 B/s wr, 3 op/s 2024-03-21T10:15:12.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:12 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:15:12.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:12 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:15:12.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:12 smithi153 ceph-mon[113733]: pgmap v53: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 511 B/s wr, 3 op/s 2024-03-21T10:15:12.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:12 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:15:12.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:12 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:15:14.548 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:14 smithi174 ceph-mon[88004]: pgmap v54: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.6 KiB/s rd, 511 B/s wr, 5 op/s 2024-03-21T10:15:14.548 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:14 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:15:14.548 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:14 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:15:14.548 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:14 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:15:14.548 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:14 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:15:14.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:14 smithi153 ceph-mon[113733]: pgmap v54: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.6 KiB/s rd, 511 B/s wr, 5 op/s 2024-03-21T10:15:14.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:14 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:15:14.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:14 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:15:14.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:14 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:15:14.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:14 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:15:16.330 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:16 smithi174 ceph-mon[88004]: pgmap v55: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-21T10:15:16.330 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:16 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:15:16.330 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:16 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:15:16.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:16 smithi153 ceph-mon[113733]: pgmap v55: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-21T10:15:16.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:16 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:15:16.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:16 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:15:17.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:17 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:15:17.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:17 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:15:18.394 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:18 smithi174 ceph-mon[88004]: pgmap v56: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-21T10:15:18.554 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:18 smithi153 ceph-mon[113733]: pgmap v56: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-21T10:15:20.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:19 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:15:20.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:19 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:15:20.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:19 smithi174 ceph-mon[88004]: pgmap v57: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 4 op/s 2024-03-21T10:15:20.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:19 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:15:20.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:19 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:15:20.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:19 smithi153 ceph-mon[113733]: pgmap v57: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 4 op/s 2024-03-21T10:15:21.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:21 smithi153 ceph-mon[113733]: pgmap v58: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-21T10:15:21.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:21 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:15:21.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:21 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:15:21.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:21 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:15:21.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:21 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:15:21.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:21 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:15:21.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:21 smithi174 ceph-mon[88004]: pgmap v58: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-21T10:15:21.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:21 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:15:21.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:21 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:15:21.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:21 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:15:21.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:21 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:15:21.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:21 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:15:22.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:22 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:15:22.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:22 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:15:22.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:22 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:15:22.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:22 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:15:22.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:22 smithi174 ceph-mon[88004]: Upgrade: Setting container_image for all crash 2024-03-21T10:15:22.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:22 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:15:22.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:22 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi153"}]: dispatch 2024-03-21T10:15:22.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:22 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi153"}]': finished 2024-03-21T10:15:22.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:22 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi174"}]: dispatch 2024-03-21T10:15:22.953 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:22 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi174"}]': finished 2024-03-21T10:15:22.953 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:22 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-03-21T10:15:22.953 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:22 smithi174 ceph-mon[88004]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-03-21T10:15:22.953 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:22 smithi174 ceph-mon[88004]: Upgrade: osd.0 is safe to restart 2024-03-21T10:15:23.020 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:22 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:15:23.020 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:22 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:15:23.020 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:22 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:15:23.020 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:22 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:15:23.020 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:22 smithi153 ceph-mon[113733]: Upgrade: Setting container_image for all crash 2024-03-21T10:15:23.020 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:22 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:15:23.020 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:22 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi153"}]: dispatch 2024-03-21T10:15:23.020 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:22 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi153"}]': finished 2024-03-21T10:15:23.020 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:22 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi174"}]: dispatch 2024-03-21T10:15:23.020 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:22 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi174"}]': finished 2024-03-21T10:15:23.020 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:22 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-03-21T10:15:23.021 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:22 smithi153 ceph-mon[113733]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-03-21T10:15:23.021 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:22 smithi153 ceph-mon[113733]: Upgrade: osd.0 is safe to restart 2024-03-21T10:15:23.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:23 smithi174 ceph-mon[88004]: pgmap v59: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-21T10:15:24.034 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:23 smithi153 ceph-mon[113733]: pgmap v59: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-21T10:15:25.016 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.015+0000 7f6764c00700 1 -- 172.21.15.153:0/3508570125 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6760101750 msgr2=0x7f67601057a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:15:25.016 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.015+0000 7f6764c00700 1 --2- 172.21.15.153:0/3508570125 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6760101750 0x7f67601057a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6748009910 tx=0x7f6748009c20 comp rx=0 tx=0).stop 2024-03-21T10:15:25.016 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.015+0000 7f6764c00700 1 -- 172.21.15.153:0/3508570125 shutdown_connections 2024-03-21T10:15:25.016 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.015+0000 7f6764c00700 1 --2- 172.21.15.153:0/3508570125 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6760101750 0x7f67601057a0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:15:25.016 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.015+0000 7f6764c00700 1 --2- 172.21.15.153:0/3508570125 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6760100da0 0x7f6760101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:15:25.016 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.015+0000 7f6764c00700 1 -- 172.21.15.153:0/3508570125 >> 172.21.15.153:0/3508570125 conn(0x7f67600fc630 msgr2=0x7f67600fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:15:25.016 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.015+0000 7f6764c00700 1 -- 172.21.15.153:0/3508570125 shutdown_connections 2024-03-21T10:15:25.016 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.015+0000 7f6764c00700 1 -- 172.21.15.153:0/3508570125 wait complete. 2024-03-21T10:15:25.017 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.016+0000 7f6764c00700 1 Processor -- start 2024-03-21T10:15:25.017 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.016+0000 7f6764c00700 1 -- start start 2024-03-21T10:15:25.017 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.017+0000 7f6764c00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6760100da0 0x7f6760196730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:15:25.018 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.017+0000 7f6764c00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6760101750 0x7f6760196c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:15:25.018 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.017+0000 7f6764c00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6760197300 con 0x7f6760101750 2024-03-21T10:15:25.018 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.017+0000 7f6764c00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6760199fc0 con 0x7f6760100da0 2024-03-21T10:15:25.018 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.017+0000 7f6755400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6760101750 0x7f6760196c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:15:25.018 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.017+0000 7f6755400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6760101750 0x7f6760196c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:52272/0 (socket says 172.21.15.153:52272) 2024-03-21T10:15:25.018 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.017+0000 7f6755400700 1 -- 172.21.15.153:0/3601822156 learned_addr learned my addr 172.21.15.153:0/3601822156 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:15:25.019 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.017+0000 7f6755400700 1 -- 172.21.15.153:0/3601822156 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6760100da0 msgr2=0x7f6760196730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:15:25.019 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.017+0000 7f675de00700 1 --2- 172.21.15.153:0/3601822156 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6760100da0 0x7f6760196730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:15:25.019 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.017+0000 7f6755400700 1 --2- 172.21.15.153:0/3601822156 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6760100da0 0x7f6760196730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:15:25.019 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.017+0000 7f6755400700 1 -- 172.21.15.153:0/3601822156 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67480095f0 con 0x7f6760101750 2024-03-21T10:15:25.019 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.017+0000 7f675de00700 1 --2- 172.21.15.153:0/3601822156 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6760100da0 0x7f6760196730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:15:25.019 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.018+0000 7f6755400700 1 --2- 172.21.15.153:0/3601822156 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6760101750 0x7f6760196c70 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f674800bda0 tx=0x7f674800be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:15:25.019 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.018+0000 7f6757400700 1 -- 172.21.15.153:0/3601822156 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f674801d070 con 0x7f6760101750 2024-03-21T10:15:25.019 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.018+0000 7f6757400700 1 -- 172.21.15.153:0/3601822156 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6748004fb0 con 0x7f6760101750 2024-03-21T10:15:25.020 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.018+0000 7f6757400700 1 -- 172.21.15.153:0/3601822156 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f674800f460 con 0x7f6760101750 2024-03-21T10:15:25.020 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.018+0000 7f6764c00700 1 -- 172.21.15.153:0/3601822156 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f676019a240 con 0x7f6760101750 2024-03-21T10:15:25.020 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.018+0000 7f6764c00700 1 -- 172.21.15.153:0/3601822156 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f676019a730 con 0x7f6760101750 2024-03-21T10:15:25.020 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.019+0000 7f6764c00700 1 -- 172.21.15.153:0/3601822156 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67601091a0 con 0x7f6760101750 2024-03-21T10:15:25.021 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.020+0000 7f6757400700 1 -- 172.21.15.153:0/3601822156 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f6748022800 con 0x7f6760101750 2024-03-21T10:15:25.021 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.021+0000 7f6757400700 1 --2- 172.21.15.153:0/3601822156 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f674406e460 0x7f6744070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:15:25.022 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.021+0000 7f6757400700 1 -- 172.21.15.153:0/3601822156 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f674808f880 con 0x7f6760101750 2024-03-21T10:15:25.022 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.021+0000 7f675de00700 1 --2- 172.21.15.153:0/3601822156 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f674406e460 0x7f6744070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:15:25.022 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.022+0000 7f675de00700 1 --2- 172.21.15.153:0/3601822156 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f674406e460 0x7f6744070920 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6750005fd0 tx=0x7f6750005dc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:15:25.026 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.026+0000 7f6757400700 1 -- 172.21.15.153:0/3601822156 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f6748058c90 con 0x7f6760101750 2024-03-21T10:15:25.238 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:25 smithi153 ceph-mon[113733]: Upgrade: Updating osd.0 2024-03-21T10:15:25.238 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:25 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:15:25.238 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:25 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-21T10:15:25.238 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:25 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:15:25.238 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:25 smithi153 ceph-mon[113733]: Deploying daemon osd.0 on smithi153 2024-03-21T10:15:25.306 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.304+0000 7f6764c00700 1 -- 172.21.15.153:0/3601822156 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f676019aa60 con 0x7f674406e460 2024-03-21T10:15:25.307 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.306+0000 7f6757400700 1 -- 172.21.15.153:0/3601822156 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f676019aa60 con 0x7f674406e460 2024-03-21T10:15:25.310 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.308+0000 7f6764c00700 1 -- 172.21.15.153:0/3601822156 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f674406e460 msgr2=0x7f6744070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:15:25.310 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.308+0000 7f6764c00700 1 --2- 172.21.15.153:0/3601822156 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f674406e460 0x7f6744070920 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6750005fd0 tx=0x7f6750005dc0 comp rx=0 tx=0).stop 2024-03-21T10:15:25.310 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.308+0000 7f6764c00700 1 -- 172.21.15.153:0/3601822156 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6760101750 msgr2=0x7f6760196c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:15:25.310 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.308+0000 7f6764c00700 1 --2- 172.21.15.153:0/3601822156 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6760101750 0x7f6760196c70 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f674800bda0 tx=0x7f674800be80 comp rx=0 tx=0).stop 2024-03-21T10:15:25.310 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.308+0000 7f6764c00700 1 -- 172.21.15.153:0/3601822156 shutdown_connections 2024-03-21T10:15:25.310 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.308+0000 7f6764c00700 1 --2- 172.21.15.153:0/3601822156 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6760100da0 0x7f6760196730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:15:25.311 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.308+0000 7f6764c00700 1 --2- 172.21.15.153:0/3601822156 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f674406e460 0x7f6744070920 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:15:25.311 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.308+0000 7f6764c00700 1 --2- 172.21.15.153:0/3601822156 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6760101750 0x7f6760196c70 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:15:25.311 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.308+0000 7f6764c00700 1 -- 172.21.15.153:0/3601822156 >> 172.21.15.153:0/3601822156 conn(0x7f67600fc630 msgr2=0x7f67600fea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:15:25.311 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.309+0000 7f6764c00700 1 -- 172.21.15.153:0/3601822156 shutdown_connections 2024-03-21T10:15:25.311 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.309+0000 7f6764c00700 1 -- 172.21.15.153:0/3601822156 wait complete. 2024-03-21T10:15:25.325 INFO:teuthology.orchestra.run.smithi153.stdout:true 2024-03-21T10:15:25.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:25 smithi174 ceph-mon[88004]: Upgrade: Updating osd.0 2024-03-21T10:15:25.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:25 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:15:25.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:25 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-21T10:15:25.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:25 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:15:25.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:25 smithi174 ceph-mon[88004]: Deploying daemon osd.0 on smithi153 2024-03-21T10:15:25.459 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.457+0000 7f96cba00700 1 -- 172.21.15.153:0/3291355081 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f96c4073a50 msgr2=0x7f96c4111610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:15:25.459 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.457+0000 7f96cba00700 1 --2- 172.21.15.153:0/3291355081 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f96c4073a50 0x7f96c4111610 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f96b0009910 tx=0x7f96b0009c20 comp rx=0 tx=0).stop 2024-03-21T10:15:25.459 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.457+0000 7f96cba00700 1 -- 172.21.15.153:0/3291355081 shutdown_connections 2024-03-21T10:15:25.459 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.457+0000 7f96cba00700 1 --2- 172.21.15.153:0/3291355081 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f96c4073a50 0x7f96c4111610 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:15:25.459 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.457+0000 7f96cba00700 1 --2- 172.21.15.153:0/3291355081 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f96c4073130 0x7f96c4073510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:15:25.459 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.457+0000 7f96cba00700 1 -- 172.21.15.153:0/3291355081 >> 172.21.15.153:0/3291355081 conn(0x7f96c40fc630 msgr2=0x7f96c40fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:15:25.459 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.457+0000 7f96cba00700 1 -- 172.21.15.153:0/3291355081 shutdown_connections 2024-03-21T10:15:25.459 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.457+0000 7f96cba00700 1 -- 172.21.15.153:0/3291355081 wait complete. 2024-03-21T10:15:25.460 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.458+0000 7f96cba00700 1 Processor -- start 2024-03-21T10:15:25.460 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.458+0000 7f96cba00700 1 -- start start 2024-03-21T10:15:25.461 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.459+0000 7f96cba00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f96c4073130 0x7f96c419cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:15:25.461 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.459+0000 7f96cba00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f96c4073a50 0x7f96c419d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:15:25.461 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.459+0000 7f96cba00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96c419d930 con 0x7f96c4073a50 2024-03-21T10:15:25.461 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.459+0000 7f96cba00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96c41a1650 con 0x7f96c4073130 2024-03-21T10:15:25.461 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.459+0000 7f96bbe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f96c4073a50 0x7f96c419d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:15:25.461 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.459+0000 7f96bbe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f96c4073a50 0x7f96c419d250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:52286/0 (socket says 172.21.15.153:52286) 2024-03-21T10:15:25.461 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.459+0000 7f96bbe00700 1 -- 172.21.15.153:0/269737147 learned_addr learned my addr 172.21.15.153:0/269737147 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:15:25.462 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.459+0000 7f96c9000700 1 --2- 172.21.15.153:0/269737147 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f96c4073130 0x7f96c419cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:15:25.462 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.459+0000 7f96bbe00700 1 -- 172.21.15.153:0/269737147 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f96c4073130 msgr2=0x7f96c419cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:15:25.462 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.459+0000 7f96bbe00700 1 --2- 172.21.15.153:0/269737147 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f96c4073130 0x7f96c419cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:15:25.462 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.459+0000 7f96bbe00700 1 -- 172.21.15.153:0/269737147 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96b00095f0 con 0x7f96c4073a50 2024-03-21T10:15:25.462 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.459+0000 7f96c9000700 1 --2- 172.21.15.153:0/269737147 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f96c4073130 0x7f96c419cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:15:25.462 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.460+0000 7f96bbe00700 1 --2- 172.21.15.153:0/269737147 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f96c4073a50 0x7f96c419d250 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f96b00092e0 tx=0x7f96b000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:15:25.462 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.460+0000 7f96c2000700 1 -- 172.21.15.153:0/269737147 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f96b001d070 con 0x7f96c4073a50 2024-03-21T10:15:25.463 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.460+0000 7f96c2000700 1 -- 172.21.15.153:0/269737147 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f96b0005e00 con 0x7f96c4073a50 2024-03-21T10:15:25.463 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.460+0000 7f96c2000700 1 -- 172.21.15.153:0/269737147 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f96b0017400 con 0x7f96c4073a50 2024-03-21T10:15:25.463 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.460+0000 7f96cba00700 1 -- 172.21.15.153:0/269737147 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96c41a1930 con 0x7f96c4073a50 2024-03-21T10:15:25.463 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.460+0000 7f96cba00700 1 -- 172.21.15.153:0/269737147 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96c41a1e80 con 0x7f96c4073a50 2024-03-21T10:15:25.463 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.461+0000 7f96cba00700 1 -- 172.21.15.153:0/269737147 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96c410ed90 con 0x7f96c4073a50 2024-03-21T10:15:25.463 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.462+0000 7f96c2000700 1 -- 172.21.15.153:0/269737147 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f96b000f460 con 0x7f96c4073a50 2024-03-21T10:15:25.465 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.463+0000 7f96c2000700 1 --2- 172.21.15.153:0/269737147 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f96a806e460 0x7f96a8070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:15:25.465 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.463+0000 7f96c2000700 1 -- 172.21.15.153:0/269737147 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f96b008f7e0 con 0x7f96c4073a50 2024-03-21T10:15:25.465 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.463+0000 7f96c9000700 1 --2- 172.21.15.153:0/269737147 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f96a806e460 0x7f96a8070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:15:25.465 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.464+0000 7f96c9000700 1 --2- 172.21.15.153:0/269737147 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f96a806e460 0x7f96a8070920 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f96b4005fd0 tx=0x7f96b4005ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:15:25.469 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.468+0000 7f96c2000700 1 -- 172.21.15.153:0/269737147 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f96b0058a20 con 0x7f96c4073a50 2024-03-21T10:15:25.710 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.708+0000 7f96cba00700 1 -- 172.21.15.153:0/269737147 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f96c41a2250 con 0x7f96a806e460 2024-03-21T10:15:25.711 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.710+0000 7f96c2000700 1 -- 172.21.15.153:0/269737147 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f96c41a2250 con 0x7f96a806e460 2024-03-21T10:15:25.715 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.713+0000 7f96cba00700 1 -- 172.21.15.153:0/269737147 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f96a806e460 msgr2=0x7f96a8070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:15:25.715 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.713+0000 7f96cba00700 1 --2- 172.21.15.153:0/269737147 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f96a806e460 0x7f96a8070920 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f96b4005fd0 tx=0x7f96b4005ee0 comp rx=0 tx=0).stop 2024-03-21T10:15:25.715 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.713+0000 7f96cba00700 1 -- 172.21.15.153:0/269737147 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f96c4073a50 msgr2=0x7f96c419d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:15:25.715 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.713+0000 7f96cba00700 1 --2- 172.21.15.153:0/269737147 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f96c4073a50 0x7f96c419d250 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f96b00092e0 tx=0x7f96b000be20 comp rx=0 tx=0).stop 2024-03-21T10:15:25.715 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.713+0000 7f96cba00700 1 -- 172.21.15.153:0/269737147 shutdown_connections 2024-03-21T10:15:25.715 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.713+0000 7f96cba00700 1 --2- 172.21.15.153:0/269737147 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f96c4073130 0x7f96c419cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:15:25.715 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.713+0000 7f96cba00700 1 --2- 172.21.15.153:0/269737147 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f96a806e460 0x7f96a8070920 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:15:25.715 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.713+0000 7f96cba00700 1 --2- 172.21.15.153:0/269737147 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f96c4073a50 0x7f96c419d250 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:15:25.715 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.713+0000 7f96cba00700 1 -- 172.21.15.153:0/269737147 >> 172.21.15.153:0/269737147 conn(0x7f96c40fc630 msgr2=0x7f96c4103120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:15:25.716 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.713+0000 7f96cba00700 1 -- 172.21.15.153:0/269737147 shutdown_connections 2024-03-21T10:15:25.716 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.713+0000 7f96cba00700 1 -- 172.21.15.153:0/269737147 wait complete. 2024-03-21T10:15:25.869 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.867+0000 7f6c9a600700 1 -- 172.21.15.153:0/4064176403 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6c94100da0 msgr2=0x7f6c94101180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:15:25.869 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.867+0000 7f6c9a600700 1 --2- 172.21.15.153:0/4064176403 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6c94100da0 0x7f6c94101180 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6c8400b0a0 tx=0x7f6c8400b3b0 comp rx=0 tx=0).stop 2024-03-21T10:15:25.869 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.867+0000 7f6c9a600700 1 -- 172.21.15.153:0/4064176403 shutdown_connections 2024-03-21T10:15:25.869 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.867+0000 7f6c9a600700 1 --2- 172.21.15.153:0/4064176403 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6c94101750 0x7f6c941057a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:15:25.869 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.867+0000 7f6c9a600700 1 --2- 172.21.15.153:0/4064176403 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6c94100da0 0x7f6c94101180 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:15:25.869 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.867+0000 7f6c9a600700 1 -- 172.21.15.153:0/4064176403 >> 172.21.15.153:0/4064176403 conn(0x7f6c940fc630 msgr2=0x7f6c940fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:15:25.869 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.867+0000 7f6c9a600700 1 -- 172.21.15.153:0/4064176403 shutdown_connections 2024-03-21T10:15:25.869 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.867+0000 7f6c9a600700 1 -- 172.21.15.153:0/4064176403 wait complete. 2024-03-21T10:15:25.870 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.868+0000 7f6c9a600700 1 Processor -- start 2024-03-21T10:15:25.870 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.868+0000 7f6c9a600700 1 -- start start 2024-03-21T10:15:25.871 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.868+0000 7f6c9a600700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6c94100da0 0x7f6c94073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:15:25.871 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.868+0000 7f6c9a600700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6c94101750 0x7f6c940735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:15:25.871 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.868+0000 7f6c9a600700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c94073b10 con 0x7f6c94100da0 2024-03-21T10:15:25.871 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.868+0000 7f6c9a600700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c94073c50 con 0x7f6c94101750 2024-03-21T10:15:25.871 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.869+0000 7f6c93400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6c94100da0 0x7f6c94073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:15:25.871 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.869+0000 7f6c93400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6c94100da0 0x7f6c94073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:52308/0 (socket says 172.21.15.153:52308) 2024-03-21T10:15:25.871 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.869+0000 7f6c93400700 1 -- 172.21.15.153:0/3057683678 learned_addr learned my addr 172.21.15.153:0/3057683678 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:15:25.871 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.869+0000 7f6c93400700 1 -- 172.21.15.153:0/3057683678 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6c94101750 msgr2=0x7f6c940735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:15:25.871 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.869+0000 7f6c93400700 1 --2- 172.21.15.153:0/3057683678 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6c94101750 0x7f6c940735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:15:25.871 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.869+0000 7f6c93400700 1 -- 172.21.15.153:0/3057683678 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c84009d00 con 0x7f6c94100da0 2024-03-21T10:15:25.872 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.869+0000 7f6c93400700 1 --2- 172.21.15.153:0/3057683678 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6c94100da0 0x7f6c94073090 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6c8400b070 tx=0x7f6c8400cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:15:25.872 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.870+0000 7f6c8be00700 1 -- 172.21.15.153:0/3057683678 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c8400e040 con 0x7f6c94100da0 2024-03-21T10:15:25.872 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.870+0000 7f6c8be00700 1 -- 172.21.15.153:0/3057683678 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6c84003de0 con 0x7f6c94100da0 2024-03-21T10:15:25.872 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.870+0000 7f6c8be00700 1 -- 172.21.15.153:0/3057683678 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c8401d8e0 con 0x7f6c94100da0 2024-03-21T10:15:25.872 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.870+0000 7f6c9a600700 1 -- 172.21.15.153:0/3057683678 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c94073ed0 con 0x7f6c94100da0 2024-03-21T10:15:25.872 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.870+0000 7f6c9a600700 1 -- 172.21.15.153:0/3057683678 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c941a2cb0 con 0x7f6c94100da0 2024-03-21T10:15:25.873 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.871+0000 7f6c9a600700 1 -- 172.21.15.153:0/3057683678 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c94109140 con 0x7f6c94100da0 2024-03-21T10:15:25.873 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.872+0000 7f6c8be00700 1 -- 172.21.15.153:0/3057683678 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f6c84004040 con 0x7f6c94100da0 2024-03-21T10:15:25.874 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.873+0000 7f6c8be00700 1 --2- 172.21.15.153:0/3057683678 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f6c7806e460 0x7f6c78070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:15:25.875 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.873+0000 7f6c8be00700 1 -- 172.21.15.153:0/3057683678 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f6c8408fac0 con 0x7f6c94100da0 2024-03-21T10:15:25.875 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.873+0000 7f6c92a00700 1 --2- 172.21.15.153:0/3057683678 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f6c7806e460 0x7f6c78070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:15:25.875 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.874+0000 7f6c92a00700 1 --2- 172.21.15.153:0/3057683678 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f6c7806e460 0x7f6c78070920 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6c80005950 tx=0x7f6c800058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:15:25.880 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:25.878+0000 7f6c8be00700 1 -- 172.21.15.153:0/3057683678 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f6c84058da0 con 0x7f6c94100da0 2024-03-21T10:15:26.102 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:26.100+0000 7f6c9a600700 1 -- 172.21.15.153:0/3057683678 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f6c9402cd10 con 0x7f6c7806e460 2024-03-21T10:15:26.124 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:26.122+0000 7f6c8be00700 1 -- 172.21.15.153:0/3057683678 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f6c9402cd10 con 0x7f6c7806e460 2024-03-21T10:15:26.124 INFO:teuthology.orchestra.run.smithi153.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-21T10:15:26.125 INFO:teuthology.orchestra.run.smithi153.stdout:alertmanager.smithi153 smithi153 *:9093,9094 running (4m) 12s ago 17m 14.7M - 0.25.0 c8568f914cd2 9ed3f2200b5f 2024-03-21T10:15:26.125 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi153 smithi153 running (18m) 12s ago 18m 8468k - 18.2.1 5be31c24972a 5b61994396a6 2024-03-21T10:15:26.125 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi174 smithi174 running (16m) 14s ago 16m 8363k - 18.2.1 5be31c24972a 4f7a5fc3ac28 2024-03-21T10:15:26.125 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi153 smithi153 running (27s) 12s ago 18m 6656k - 19.0.0-2231-g2edfee68 237e43c7a65f 32c21104d5b0 2024-03-21T10:15:26.125 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi174 smithi174 running (19s) 14s ago 16m 6660k - 19.0.0-2231-g2edfee68 237e43c7a65f a04e48967ab6 2024-03-21T10:15:26.125 INFO:teuthology.orchestra.run.smithi153.stdout:grafana.smithi153 smithi153 *:3000 running (3m) 12s ago 17m 73.8M - 9.4.12 2bacad6d85d8 ee24026ab786 2024-03-21T10:15:26.125 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.iuyluj smithi153 running (10m) 12s ago 10m 127M - 18.2.1 5be31c24972a 02f667cee979 2024-03-21T10:15:26.125 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.zipckr smithi153 running (10m) 12s ago 10m 59.0M - 18.2.1 5be31c24972a 68a853929f8f 2024-03-21T10:15:26.125 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.nhozag smithi174 running (10m) 14s ago 10m 57.5M - 18.2.1 5be31c24972a 2fee4000308f 2024-03-21T10:15:26.125 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.yyimst smithi174 running (10m) 14s ago 10m 117M - 18.2.1 5be31c24972a d78a729220d1 2024-03-21T10:15:26.125 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi153.tluxni smithi153 *:8443,9283,8765 running (7m) 12s ago 19m 497M - 19.0.0-2231-g2edfee68 237e43c7a65f 7ffee0e55382 2024-03-21T10:15:26.125 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi174.qpcwat smithi174 *:8443,9283,8765 running (5m) 14s ago 16m 450M - 19.0.0-2231-g2edfee68 237e43c7a65f 748968f216ab 2024-03-21T10:15:26.125 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi153 smithi153 running (2m) 12s ago 20m 44.3M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f dd8038303dc3 2024-03-21T10:15:26.125 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi174 smithi174 running (89s) 14s ago 16m 36.6M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f 29d78a788f2c 2024-03-21T10:15:26.126 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi153 smithi153 *:9100 running (5m) 12s ago 18m 8174k - 1.5.0 0da6a335fe13 bb835edc0e0b 2024-03-21T10:15:26.126 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi174 smithi174 *:9100 running (5m) 14s ago 16m 7600k - 1.5.0 0da6a335fe13 7c4ecad790fa 2024-03-21T10:15:26.126 INFO:teuthology.orchestra.run.smithi153.stdout:osd.0 smithi153 running (15m) 12s ago 15m 215M 1217M 18.2.1 5be31c24972a 28912a2319b4 2024-03-21T10:15:26.126 INFO:teuthology.orchestra.run.smithi153.stdout:osd.1 smithi153 running (14m) 12s ago 14m 215M 1217M 18.2.1 5be31c24972a ca60681e2314 2024-03-21T10:15:26.126 INFO:teuthology.orchestra.run.smithi153.stdout:osd.2 smithi153 running (14m) 12s ago 14m 174M 1217M 18.2.1 5be31c24972a 8db799f8962a 2024-03-21T10:15:26.126 INFO:teuthology.orchestra.run.smithi153.stdout:osd.3 smithi174 running (13m) 14s ago 13m 234M 2241M 18.2.1 5be31c24972a dafc89e0658d 2024-03-21T10:15:26.126 INFO:teuthology.orchestra.run.smithi153.stdout:osd.4 smithi174 running (13m) 14s ago 13m 224M 2241M 18.2.1 5be31c24972a c4cd15701f05 2024-03-21T10:15:26.126 INFO:teuthology.orchestra.run.smithi153.stdout:osd.5 smithi174 running (12m) 14s ago 12m 203M 2241M 18.2.1 5be31c24972a 3b69886ce22e 2024-03-21T10:15:26.126 INFO:teuthology.orchestra.run.smithi153.stdout:prometheus.smithi153 smithi153 *:9095 running (4m) 12s ago 17m 68.9M - 2.43.0 a07b618ecd1d 7d95561e50c2 2024-03-21T10:15:26.127 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:26.125+0000 7f6c9a600700 1 -- 172.21.15.153:0/3057683678 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f6c7806e460 msgr2=0x7f6c78070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:15:26.127 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:26.125+0000 7f6c9a600700 1 --2- 172.21.15.153:0/3057683678 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f6c7806e460 0x7f6c78070920 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6c80005950 tx=0x7f6c800058e0 comp rx=0 tx=0).stop 2024-03-21T10:15:26.127 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:26.125+0000 7f6c9a600700 1 -- 172.21.15.153:0/3057683678 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6c94100da0 msgr2=0x7f6c94073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:15:26.127 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:26.125+0000 7f6c9a600700 1 --2- 172.21.15.153:0/3057683678 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6c94100da0 0x7f6c94073090 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6c8400b070 tx=0x7f6c8400cdd0 comp rx=0 tx=0).stop 2024-03-21T10:15:26.127 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:26.125+0000 7f6c9a600700 1 -- 172.21.15.153:0/3057683678 shutdown_connections 2024-03-21T10:15:26.127 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:26.125+0000 7f6c9a600700 1 --2- 172.21.15.153:0/3057683678 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f6c7806e460 0x7f6c78070920 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:15:26.127 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:26.125+0000 7f6c9a600700 1 --2- 172.21.15.153:0/3057683678 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6c94100da0 0x7f6c94073090 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:15:26.127 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:26.125+0000 7f6c9a600700 1 --2- 172.21.15.153:0/3057683678 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6c94101750 0x7f6c940735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:15:26.127 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:26.125+0000 7f6c9a600700 1 -- 172.21.15.153:0/3057683678 >> 172.21.15.153:0/3057683678 conn(0x7f6c940fc630 msgr2=0x7f6c940fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:15:26.127 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:26.125+0000 7f6c9a600700 1 -- 172.21.15.153:0/3057683678 shutdown_connections 2024-03-21T10:15:26.127 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:26.125+0000 7f6c9a600700 1 -- 172.21.15.153:0/3057683678 wait complete. 2024-03-21T10:15:26.276 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:26 smithi153 ceph-mon[113733]: pgmap v60: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:15:26.277 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:26.274+0000 7f72a8600700 1 -- 172.21.15.153:0/262825416 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72a0101770 msgr2=0x7f72a01057c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:15:26.277 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:26.274+0000 7f72a8600700 1 --2- 172.21.15.153:0/262825416 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72a0101770 0x7f72a01057c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f728c009910 tx=0x7f728c009c20 comp rx=0 tx=0).stop 2024-03-21T10:15:26.277 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:26.274+0000 7f72a8600700 1 -- 172.21.15.153:0/262825416 shutdown_connections 2024-03-21T10:15:26.277 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:26.274+0000 7f72a8600700 1 --2- 172.21.15.153:0/262825416 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72a0101770 0x7f72a01057c0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:15:26.277 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:26.274+0000 7f72a8600700 1 --2- 172.21.15.153:0/262825416 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f72a0100dc0 0x7f72a01011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:15:26.277 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:26.274+0000 7f72a8600700 1 -- 172.21.15.153:0/262825416 >> 172.21.15.153:0/262825416 conn(0x7f72a00fc670 msgr2=0x7f72a00fea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:15:26.277 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:26.274+0000 7f72a8600700 1 -- 172.21.15.153:0/262825416 shutdown_connections 2024-03-21T10:15:26.277 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:26.274+0000 7f72a8600700 1 -- 172.21.15.153:0/262825416 wait complete. 2024-03-21T10:15:26.278 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:26.275+0000 7f72a8600700 1 Processor -- start 2024-03-21T10:15:26.278 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:26.276+0000 7f72a8600700 1 -- start start 2024-03-21T10:15:26.278 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:26.276+0000 7f72a8600700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f72a0100dc0 0x7f72a0073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:15:26.278 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:26.276+0000 7f72a8600700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72a0101770 0x7f72a00735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:15:26.278 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:26.276+0000 7f72a8600700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72a0073b10 con 0x7f72a0101770 2024-03-21T10:15:26.278 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:26.276+0000 7f72a8600700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72a0073c50 con 0x7f72a0100dc0 2024-03-21T10:15:26.278 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:26.276+0000 7f72a5200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72a0101770 0x7f72a00735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:15:26.278 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:26.276+0000 7f72a5200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72a0101770 0x7f72a00735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:52326/0 (socket says 172.21.15.153:52326) 2024-03-21T10:15:26.279 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:26.277+0000 7f72a5200700 1 -- 172.21.15.153:0/724294149 learned_addr learned my addr 172.21.15.153:0/724294149 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:15:26.279 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:26.277+0000 7f72a5c00700 1 --2- 172.21.15.153:0/724294149 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f72a0100dc0 0x7f72a0073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:15:26.279 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:26.277+0000 7f72a5200700 1 -- 172.21.15.153:0/724294149 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f72a0100dc0 msgr2=0x7f72a0073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:15:26.280 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:26.277+0000 7f72a5200700 1 --2- 172.21.15.153:0/724294149 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f72a0100dc0 0x7f72a0073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:15:26.280 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:26.277+0000 7f72a5200700 1 -- 172.21.15.153:0/724294149 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f728c0095f0 con 0x7f72a0101770 2024-03-21T10:15:26.280 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:26.277+0000 7f72a5200700 1 --2- 172.21.15.153:0/724294149 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72a0101770 0x7f72a00735d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f728c00bda0 tx=0x7f728c00be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:15:26.280 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:26.278+0000 7f729e000700 1 -- 172.21.15.153:0/724294149 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f728c01d070 con 0x7f72a0101770 2024-03-21T10:15:26.280 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:26.278+0000 7f729e000700 1 -- 172.21.15.153:0/724294149 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f728c00f460 con 0x7f72a0101770 2024-03-21T10:15:26.280 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:26.278+0000 7f72a8600700 1 -- 172.21.15.153:0/724294149 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72a0073ed0 con 0x7f72a0101770 2024-03-21T10:15:26.280 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:26.278+0000 7f729e000700 1 -- 172.21.15.153:0/724294149 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f728c017610 con 0x7f72a0101770 2024-03-21T10:15:26.280 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:26.278+0000 7f72a8600700 1 -- 172.21.15.153:0/724294149 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72a01a2cb0 con 0x7f72a0101770 2024-03-21T10:15:26.281 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:26.279+0000 7f72a8600700 1 -- 172.21.15.153:0/724294149 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72a01091c0 con 0x7f72a0101770 2024-03-21T10:15:26.282 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:26.280+0000 7f729e000700 1 -- 172.21.15.153:0/724294149 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f728c00fa40 con 0x7f72a0101770 2024-03-21T10:15:26.284 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:26.282+0000 7f729e000700 1 --2- 172.21.15.153:0/724294149 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f728806e460 0x7f7288070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:15:26.284 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:26.282+0000 7f729e000700 1 -- 172.21.15.153:0/724294149 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f728c08f860 con 0x7f72a0101770 2024-03-21T10:15:26.284 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:26.282+0000 7f72a5c00700 1 --2- 172.21.15.153:0/724294149 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f728806e460 0x7f7288070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:15:26.285 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:26.283+0000 7f72a5c00700 1 --2- 172.21.15.153:0/724294149 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f728806e460 0x7f7288070920 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7294005950 tx=0x7f72940058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:15:26.288 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:26.286+0000 7f729e000700 1 -- 172.21.15.153:0/724294149 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f728c058a20 con 0x7f72a0101770 2024-03-21T10:15:26.610 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:26.606+0000 7f72a8600700 1 -- 172.21.15.153:0/724294149 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f72a004ea90 con 0x7f72a0101770 2024-03-21T10:15:26.610 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:26.607+0000 7f729e000700 1 -- 172.21.15.153:0/724294149 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+688 (secure 0 0 0) 0x7f728c05c0e0 con 0x7f72a0101770 2024-03-21T10:15:26.610 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:15:26.610 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": { 2024-03-21T10:15:26.610 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:15:26.610 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:15:26.610 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": { 2024-03-21T10:15:26.610 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:15:26.611 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:15:26.611 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": { 2024-03-21T10:15:26.611 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-03-21T10:15:26.611 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:15:26.611 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": { 2024-03-21T10:15:26.611 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-03-21T10:15:26.611 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:15:26.611 INFO:teuthology.orchestra.run.smithi153.stdout: "overall": { 2024-03-21T10:15:26.611 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 10, 2024-03-21T10:15:26.611 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 4 2024-03-21T10:15:26.611 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-03-21T10:15:26.612 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:15:26.612 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:26.610+0000 7f72a8600700 1 -- 172.21.15.153:0/724294149 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f728806e460 msgr2=0x7f7288070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:15:26.613 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:26.610+0000 7f72a8600700 1 --2- 172.21.15.153:0/724294149 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f728806e460 0x7f7288070920 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7294005950 tx=0x7f72940058e0 comp rx=0 tx=0).stop 2024-03-21T10:15:26.613 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:26.610+0000 7f72a8600700 1 -- 172.21.15.153:0/724294149 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72a0101770 msgr2=0x7f72a00735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:15:26.613 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:26.610+0000 7f72a8600700 1 --2- 172.21.15.153:0/724294149 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72a0101770 0x7f72a00735d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f728c00bda0 tx=0x7f728c00be80 comp rx=0 tx=0).stop 2024-03-21T10:15:26.613 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:26.610+0000 7f72a8600700 1 -- 172.21.15.153:0/724294149 shutdown_connections 2024-03-21T10:15:26.613 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:26.610+0000 7f72a8600700 1 --2- 172.21.15.153:0/724294149 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f72a0100dc0 0x7f72a0073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:15:26.613 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:26.610+0000 7f72a8600700 1 --2- 172.21.15.153:0/724294149 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f728806e460 0x7f7288070920 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:15:26.613 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:26.610+0000 7f72a8600700 1 --2- 172.21.15.153:0/724294149 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72a0101770 0x7f72a00735d0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:15:26.613 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:26.610+0000 7f72a8600700 1 -- 172.21.15.153:0/724294149 >> 172.21.15.153:0/724294149 conn(0x7f72a00fc670 msgr2=0x7f72a00fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:15:26.613 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:26.610+0000 7f72a8600700 1 -- 172.21.15.153:0/724294149 shutdown_connections 2024-03-21T10:15:26.613 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:26.610+0000 7f72a8600700 1 -- 172.21.15.153:0/724294149 wait complete. 2024-03-21T10:15:26.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:26 smithi174 ceph-mon[88004]: pgmap v60: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:15:26.760 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:26.758+0000 7fc119c00700 1 -- 172.21.15.153:0/3092883578 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc1141069a0 msgr2=0x7fc114106d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:15:26.760 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:26.758+0000 7fc119c00700 1 --2- 172.21.15.153:0/3092883578 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc1141069a0 0x7fc114106d80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc104009910 tx=0x7fc104009c20 comp rx=0 tx=0).stop 2024-03-21T10:15:26.760 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:26.758+0000 7fc119c00700 1 -- 172.21.15.153:0/3092883578 shutdown_connections 2024-03-21T10:15:26.760 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:26.758+0000 7fc119c00700 1 --2- 172.21.15.153:0/3092883578 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc1141009a0 0x7fc114100e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:15:26.761 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:26.758+0000 7fc119c00700 1 --2- 172.21.15.153:0/3092883578 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc1141069a0 0x7fc114106d80 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:15:26.761 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:26.758+0000 7fc119c00700 1 -- 172.21.15.153:0/3092883578 >> 172.21.15.153:0/3092883578 conn(0x7fc1140fc490 msgr2=0x7fc1140fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:15:26.761 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:26.759+0000 7fc119c00700 1 -- 172.21.15.153:0/3092883578 shutdown_connections 2024-03-21T10:15:26.761 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:26.759+0000 7fc119c00700 1 -- 172.21.15.153:0/3092883578 wait complete. 2024-03-21T10:15:26.762 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:26.759+0000 7fc119c00700 1 Processor -- start 2024-03-21T10:15:26.762 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:26.760+0000 7fc119c00700 1 -- start start 2024-03-21T10:15:26.762 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:26.760+0000 7fc119c00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc1141009a0 0x7fc114073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:15:26.762 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:26.760+0000 7fc119c00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc1141069a0 0x7fc1140735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:15:26.762 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:26.760+0000 7fc119c00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc114073b10 con 0x7fc1141009a0 2024-03-21T10:15:26.762 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:26.760+0000 7fc119c00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc114073c50 con 0x7fc1141069a0 2024-03-21T10:15:26.762 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:26.760+0000 7fc113e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc1141009a0 0x7fc114073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:15:26.762 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:26.760+0000 7fc113e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc1141009a0 0x7fc114073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:52350/0 (socket says 172.21.15.153:52350) 2024-03-21T10:15:26.762 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:26.760+0000 7fc113e00700 1 -- 172.21.15.153:0/332341648 learned_addr learned my addr 172.21.15.153:0/332341648 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:15:26.763 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:26.761+0000 7fc113e00700 1 -- 172.21.15.153:0/332341648 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc1141069a0 msgr2=0x7fc1140735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:15:26.763 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:26.761+0000 7fc113400700 1 --2- 172.21.15.153:0/332341648 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc1141069a0 0x7fc1140735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:15:26.763 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:26.761+0000 7fc113e00700 1 --2- 172.21.15.153:0/332341648 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc1141069a0 0x7fc1140735d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:15:26.763 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:26.761+0000 7fc113e00700 1 -- 172.21.15.153:0/332341648 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1040095f0 con 0x7fc1141009a0 2024-03-21T10:15:26.764 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:26.761+0000 7fc113400700 1 --2- 172.21.15.153:0/332341648 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc1141069a0 0x7fc1140735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:15:26.764 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:26.761+0000 7fc113e00700 1 --2- 172.21.15.153:0/332341648 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc1141009a0 0x7fc114073090 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc104006010 tx=0x7fc10400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:15:26.764 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:26.761+0000 7fc110c00700 1 -- 172.21.15.153:0/332341648 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc10401d070 con 0x7fc1141009a0 2024-03-21T10:15:26.764 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:26.761+0000 7fc110c00700 1 -- 172.21.15.153:0/332341648 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc104004fb0 con 0x7fc1141009a0 2024-03-21T10:15:26.764 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:26.761+0000 7fc119c00700 1 -- 172.21.15.153:0/332341648 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc114073ed0 con 0x7fc1141009a0 2024-03-21T10:15:26.764 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:26.761+0000 7fc110c00700 1 -- 172.21.15.153:0/332341648 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc104022ea0 con 0x7fc1141009a0 2024-03-21T10:15:26.764 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:26.762+0000 7fc119c00700 1 -- 172.21.15.153:0/332341648 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1141a2b60 con 0x7fc1141009a0 2024-03-21T10:15:26.764 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:26.763+0000 7fc119c00700 1 -- 172.21.15.153:0/332341648 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc114108fa0 con 0x7fc1141009a0 2024-03-21T10:15:26.765 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:26.764+0000 7fc110c00700 1 -- 172.21.15.153:0/332341648 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fc104022470 con 0x7fc1141009a0 2024-03-21T10:15:26.766 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:26.764+0000 7fc110c00700 1 --2- 172.21.15.153:0/332341648 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fc0f806e460 0x7fc0f8070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:15:26.766 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:26.764+0000 7fc110c00700 1 -- 172.21.15.153:0/332341648 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fc10408f8e0 con 0x7fc1141009a0 2024-03-21T10:15:26.766 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:26.765+0000 7fc113400700 1 --2- 172.21.15.153:0/332341648 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fc0f806e460 0x7fc0f8070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:15:26.767 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:26.765+0000 7fc113400700 1 --2- 172.21.15.153:0/332341648 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fc0f806e460 0x7fc0f8070920 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc114074880 tx=0x7fc100008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:15:26.770 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:26.769+0000 7fc110c00700 1 -- 172.21.15.153:0/332341648 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc104058bd0 con 0x7fc1141009a0 2024-03-21T10:15:27.032 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:27.030+0000 7fc119c00700 1 -- 172.21.15.153:0/332341648 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fc11404f2e0 con 0x7fc1141009a0 2024-03-21T10:15:27.033 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:27.031+0000 7fc110c00700 1 -- 172.21.15.153:0/332341648 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 21 v21) v1 ==== 76+0+1903 (secure 0 0 0) 0x7fc104027780 con 0x7fc1141009a0 2024-03-21T10:15:27.033 INFO:teuthology.orchestra.run.smithi153.stdout:e21 2024-03-21T10:15:27.033 INFO:teuthology.orchestra.run.smithi153.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-21T10:15:27.033 INFO:teuthology.orchestra.run.smithi153.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:15:27.034 INFO:teuthology.orchestra.run.smithi153.stdout:legacy client fscid: 1 2024-03-21T10:15:27.034 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:15:27.034 INFO:teuthology.orchestra.run.smithi153.stdout:Filesystem 'cephfs' (1) 2024-03-21T10:15:27.034 INFO:teuthology.orchestra.run.smithi153.stdout:fs_name cephfs 2024-03-21T10:15:27.034 INFO:teuthology.orchestra.run.smithi153.stdout:epoch 21 2024-03-21T10:15:27.034 INFO:teuthology.orchestra.run.smithi153.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-21T10:15:27.034 INFO:teuthology.orchestra.run.smithi153.stdout:created 2024-03-21T10:04:27.086576+0000 2024-03-21T10:15:27.034 INFO:teuthology.orchestra.run.smithi153.stdout:modified 2024-03-21T10:13:01.455594+0000 2024-03-21T10:15:27.034 INFO:teuthology.orchestra.run.smithi153.stdout:tableserver 0 2024-03-21T10:15:27.034 INFO:teuthology.orchestra.run.smithi153.stdout:root 0 2024-03-21T10:15:27.034 INFO:teuthology.orchestra.run.smithi153.stdout:session_timeout 60 2024-03-21T10:15:27.034 INFO:teuthology.orchestra.run.smithi153.stdout:session_autoclose 300 2024-03-21T10:15:27.035 INFO:teuthology.orchestra.run.smithi153.stdout:max_file_size 1099511627776 2024-03-21T10:15:27.035 INFO:teuthology.orchestra.run.smithi153.stdout:max_xattr_size 65536 2024-03-21T10:15:27.035 INFO:teuthology.orchestra.run.smithi153.stdout:required_client_features {} 2024-03-21T10:15:27.035 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure 0 2024-03-21T10:15:27.035 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure_osd_epoch 47 2024-03-21T10:15:27.035 INFO:teuthology.orchestra.run.smithi153.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:15:27.035 INFO:teuthology.orchestra.run.smithi153.stdout:max_mds 2 2024-03-21T10:15:27.035 INFO:teuthology.orchestra.run.smithi153.stdout:in 0,1 2024-03-21T10:15:27.035 INFO:teuthology.orchestra.run.smithi153.stdout:up {0=14484,1=24273} 2024-03-21T10:15:27.035 INFO:teuthology.orchestra.run.smithi153.stdout:failed 2024-03-21T10:15:27.035 INFO:teuthology.orchestra.run.smithi153.stdout:damaged 2024-03-21T10:15:27.035 INFO:teuthology.orchestra.run.smithi153.stdout:stopped 2024-03-21T10:15:27.035 INFO:teuthology.orchestra.run.smithi153.stdout:data_pools [3] 2024-03-21T10:15:27.035 INFO:teuthology.orchestra.run.smithi153.stdout:metadata_pool 2 2024-03-21T10:15:27.035 INFO:teuthology.orchestra.run.smithi153.stdout:inline_data enabled 2024-03-21T10:15:27.036 INFO:teuthology.orchestra.run.smithi153.stdout:balancer 2024-03-21T10:15:27.036 INFO:teuthology.orchestra.run.smithi153.stdout:bal_rank_mask -1 2024-03-21T10:15:27.036 INFO:teuthology.orchestra.run.smithi153.stdout:standby_count_wanted 1 2024-03-21T10:15:27.036 INFO:teuthology.orchestra.run.smithi153.stdout:qdb_cluster leader: 0 members: 2024-03-21T10:15:27.036 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.iuyluj{0:14484} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.153:6828/866138999,v1:172.21.15.153:6829/866138999] compat {c=[1],r=[1],i=[7ff]}] 2024-03-21T10:15:27.036 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.zipckr{0:14494} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.153:6826/4033622615,v1:172.21.15.153:6827/4033622615] compat {c=[1],r=[1],i=[7ff]}] 2024-03-21T10:15:27.036 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.yyimst{1:24273} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.174:6824/1776947618,v1:172.21.15.174:6825/1776947618] compat {c=[1],r=[1],i=[7ff]}] 2024-03-21T10:15:27.036 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.nhozag{1:24285} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.174:6826/806095873,v1:172.21.15.174:6827/806095873] compat {c=[1],r=[1],i=[7ff]}] 2024-03-21T10:15:27.036 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:15:27.036 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:15:27.037 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:27.034+0000 7fc119c00700 1 -- 172.21.15.153:0/332341648 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fc0f806e460 msgr2=0x7fc0f8070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:15:27.037 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:27.034+0000 7fc119c00700 1 --2- 172.21.15.153:0/332341648 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fc0f806e460 0x7fc0f8070920 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc114074880 tx=0x7fc100008040 comp rx=0 tx=0).stop 2024-03-21T10:15:27.037 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:27.034+0000 7fc119c00700 1 -- 172.21.15.153:0/332341648 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc1141009a0 msgr2=0x7fc114073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:15:27.037 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:27.034+0000 7fc119c00700 1 --2- 172.21.15.153:0/332341648 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc1141009a0 0x7fc114073090 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc104006010 tx=0x7fc10400be20 comp rx=0 tx=0).stop 2024-03-21T10:15:27.037 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:27.034+0000 7fc119c00700 1 -- 172.21.15.153:0/332341648 shutdown_connections 2024-03-21T10:15:27.037 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:27.034+0000 7fc119c00700 1 --2- 172.21.15.153:0/332341648 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fc0f806e460 0x7fc0f8070920 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:15:27.037 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:27.034+0000 7fc119c00700 1 --2- 172.21.15.153:0/332341648 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc1141009a0 0x7fc114073090 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:15:27.038 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:27.034+0000 7fc119c00700 1 --2- 172.21.15.153:0/332341648 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc1141069a0 0x7fc1140735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:15:27.038 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:27.034+0000 7fc119c00700 1 -- 172.21.15.153:0/332341648 >> 172.21.15.153:0/332341648 conn(0x7fc1140fc490 msgr2=0x7fc1140fcee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:15:27.038 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:27.034+0000 7fc119c00700 1 -- 172.21.15.153:0/332341648 shutdown_connections 2024-03-21T10:15:27.038 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:27.034+0000 7fc119c00700 1 -- 172.21.15.153:0/332341648 wait complete. 2024-03-21T10:15:27.038 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 21 2024-03-21T10:15:27.195 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:27.193+0000 7f4eeae00700 1 -- 172.21.15.153:0/3792334335 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4ee4068670 msgr2=0x7f4ee4068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:15:27.195 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:27.193+0000 7f4eeae00700 1 --2- 172.21.15.153:0/3792334335 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4ee4068670 0x7f4ee4068a50 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4ed4009910 tx=0x7f4ed4009c20 comp rx=0 tx=0).stop 2024-03-21T10:15:27.196 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:27.193+0000 7f4eeae00700 1 -- 172.21.15.153:0/3792334335 shutdown_connections 2024-03-21T10:15:27.196 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:27.193+0000 7f4eeae00700 1 --2- 172.21.15.153:0/3792334335 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4ee4069020 0x7f4ee4105790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:15:27.196 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:27.193+0000 7f4eeae00700 1 --2- 172.21.15.153:0/3792334335 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4ee4068670 0x7f4ee4068a50 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:15:27.196 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:27.193+0000 7f4eeae00700 1 -- 172.21.15.153:0/3792334335 >> 172.21.15.153:0/3792334335 conn(0x7f4ee40755a0 msgr2=0x7f4ee40759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:15:27.196 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:27.194+0000 7f4eeae00700 1 -- 172.21.15.153:0/3792334335 shutdown_connections 2024-03-21T10:15:27.196 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:27.194+0000 7f4eeae00700 1 -- 172.21.15.153:0/3792334335 wait complete. 2024-03-21T10:15:27.196 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:27.195+0000 7f4eeae00700 1 Processor -- start 2024-03-21T10:15:27.197 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:27.195+0000 7f4eeae00700 1 -- start start 2024-03-21T10:15:27.197 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:27.195+0000 7f4eeae00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4ee4068670 0x7f4ee419e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:15:27.197 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:27.195+0000 7f4eeae00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4ee4069020 0x7f4ee419ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:15:27.197 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:27.195+0000 7f4eeae00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee419f2e0 con 0x7f4ee4068670 2024-03-21T10:15:27.197 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:27.195+0000 7f4eeae00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee4198790 con 0x7f4ee4069020 2024-03-21T10:15:27.197 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:27.195+0000 7f4ee3e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4ee4068670 0x7f4ee419e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:15:27.198 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:27.196+0000 7f4ee3e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4ee4068670 0x7f4ee419e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:52356/0 (socket says 172.21.15.153:52356) 2024-03-21T10:15:27.198 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:27.196+0000 7f4ee3e00700 1 -- 172.21.15.153:0/3773780607 learned_addr learned my addr 172.21.15.153:0/3773780607 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:15:27.198 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:27.196+0000 7f4ee3400700 1 --2- 172.21.15.153:0/3773780607 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4ee4069020 0x7f4ee419ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:15:27.198 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:27.196+0000 7f4ee3e00700 1 -- 172.21.15.153:0/3773780607 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4ee4069020 msgr2=0x7f4ee419ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:15:27.198 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:27.196+0000 7f4ee3e00700 1 --2- 172.21.15.153:0/3773780607 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4ee4069020 0x7f4ee419ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:15:27.198 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:27.196+0000 7f4ee3e00700 1 -- 172.21.15.153:0/3773780607 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ed40095f0 con 0x7f4ee4068670 2024-03-21T10:15:27.199 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:27.196+0000 7f4ee3e00700 1 --2- 172.21.15.153:0/3773780607 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4ee4068670 0x7f4ee419e710 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4ed40092e0 tx=0x7f4ed400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:15:27.199 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:27.197+0000 7f4ee0c00700 1 -- 172.21.15.153:0/3773780607 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ed401d070 con 0x7f4ee4068670 2024-03-21T10:15:27.199 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:27.197+0000 7f4eeae00700 1 -- 172.21.15.153:0/3773780607 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ee4198a10 con 0x7f4ee4068670 2024-03-21T10:15:27.199 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:27.197+0000 7f4ee0c00700 1 -- 172.21.15.153:0/3773780607 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4ed4005e00 con 0x7f4ee4068670 2024-03-21T10:15:27.199 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:27.197+0000 7f4ee0c00700 1 -- 172.21.15.153:0/3773780607 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ed4017400 con 0x7f4ee4068670 2024-03-21T10:15:27.199 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:27.197+0000 7f4eeae00700 1 -- 172.21.15.153:0/3773780607 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ee4198f00 con 0x7f4ee4068670 2024-03-21T10:15:27.199 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:27.198+0000 7f4eeae00700 1 -- 172.21.15.153:0/3773780607 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ee404ea90 con 0x7f4ee4068670 2024-03-21T10:15:27.201 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:27.199+0000 7f4ee0c00700 1 -- 172.21.15.153:0/3773780607 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f4ed400f460 con 0x7f4ee4068670 2024-03-21T10:15:27.202 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:27.200+0000 7f4ee0c00700 1 --2- 172.21.15.153:0/3773780607 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f4ec806e460 0x7f4ec8070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:15:27.202 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:27.200+0000 7f4ee0c00700 1 -- 172.21.15.153:0/3773780607 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f4ed408f810 con 0x7f4ee4068670 2024-03-21T10:15:27.202 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:27.200+0000 7f4ee3400700 1 --2- 172.21.15.153:0/3773780607 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f4ec806e460 0x7f4ec8070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:15:27.202 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:27.201+0000 7f4ee3400700 1 --2- 172.21.15.153:0/3773780607 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f4ec806e460 0x7f4ec8070920 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4ed0005950 tx=0x7f4ed00058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:15:27.207 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:27.206+0000 7f4ee0c00700 1 -- 172.21.15.153:0/3773780607 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4ed405c580 con 0x7f4ee4068670 2024-03-21T10:15:27.454 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:27.452+0000 7f4eeae00700 1 -- 172.21.15.153:0/3773780607 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4ee41993a0 con 0x7f4ec806e460 2024-03-21T10:15:27.454 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:27 smithi153 ceph-mon[113733]: from='client.34214 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:15:27.454 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:27 smithi153 ceph-mon[113733]: from='client.34218 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:15:27.454 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:27 smithi153 ceph-mon[113733]: from='client.34222 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:15:27.454 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:27 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/724294149' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:15:27.454 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:27 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/332341648' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:15:27.455 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:27.454+0000 7f4ee0c00700 1 -- 172.21.15.153:0/3773780607 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f4ee41993a0 con 0x7f4ec806e460 2024-03-21T10:15:27.456 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:15:27.456 INFO:teuthology.orchestra.run.smithi153.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232", 2024-03-21T10:15:27.456 INFO:teuthology.orchestra.run.smithi153.stdout: "in_progress": true, 2024-03-21T10:15:27.456 INFO:teuthology.orchestra.run.smithi153.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-21T10:15:27.456 INFO:teuthology.orchestra.run.smithi153.stdout: "services_complete": [ 2024-03-21T10:15:27.456 INFO:teuthology.orchestra.run.smithi153.stdout: "crash", 2024-03-21T10:15:27.456 INFO:teuthology.orchestra.run.smithi153.stdout: "mon", 2024-03-21T10:15:27.456 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr" 2024-03-21T10:15:27.456 INFO:teuthology.orchestra.run.smithi153.stdout: ], 2024-03-21T10:15:27.457 INFO:teuthology.orchestra.run.smithi153.stdout: "progress": "6/23 daemons upgraded", 2024-03-21T10:15:27.457 INFO:teuthology.orchestra.run.smithi153.stdout: "message": "Currently upgrading osd daemons", 2024-03-21T10:15:27.457 INFO:teuthology.orchestra.run.smithi153.stdout: "is_paused": false 2024-03-21T10:15:27.457 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:15:27.459 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:27.456+0000 7f4eeae00700 1 -- 172.21.15.153:0/3773780607 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f4ec806e460 msgr2=0x7f4ec8070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:15:27.459 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:27.456+0000 7f4eeae00700 1 --2- 172.21.15.153:0/3773780607 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f4ec806e460 0x7f4ec8070920 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4ed0005950 tx=0x7f4ed00058e0 comp rx=0 tx=0).stop 2024-03-21T10:15:27.459 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:27.456+0000 7f4eeae00700 1 -- 172.21.15.153:0/3773780607 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4ee4068670 msgr2=0x7f4ee419e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:15:27.459 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:27.456+0000 7f4eeae00700 1 --2- 172.21.15.153:0/3773780607 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4ee4068670 0x7f4ee419e710 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4ed40092e0 tx=0x7f4ed400be20 comp rx=0 tx=0).stop 2024-03-21T10:15:27.459 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:27.457+0000 7f4eeae00700 1 -- 172.21.15.153:0/3773780607 shutdown_connections 2024-03-21T10:15:27.459 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:27.457+0000 7f4eeae00700 1 --2- 172.21.15.153:0/3773780607 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f4ec806e460 0x7f4ec8070920 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:15:27.459 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:27.457+0000 7f4eeae00700 1 --2- 172.21.15.153:0/3773780607 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4ee4068670 0x7f4ee419e710 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:15:27.459 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:27.457+0000 7f4eeae00700 1 --2- 172.21.15.153:0/3773780607 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4ee4069020 0x7f4ee419ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:15:27.459 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:27.457+0000 7f4eeae00700 1 -- 172.21.15.153:0/3773780607 >> 172.21.15.153:0/3773780607 conn(0x7f4ee40755a0 msgr2=0x7f4ee4102760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:15:27.459 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:27.457+0000 7f4eeae00700 1 -- 172.21.15.153:0/3773780607 shutdown_connections 2024-03-21T10:15:27.459 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:27.457+0000 7f4eeae00700 1 -- 172.21.15.153:0/3773780607 wait complete. 2024-03-21T10:15:27.613 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:27.611+0000 7f8bf0e00700 1 -- 172.21.15.153:0/1754345323 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8bec069020 msgr2=0x7f8bec105790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:15:27.614 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:27.611+0000 7f8bf0e00700 1 --2- 172.21.15.153:0/1754345323 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8bec069020 0x7f8bec105790 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8bd8009910 tx=0x7f8bd8009c20 comp rx=0 tx=0).stop 2024-03-21T10:15:27.614 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:27.612+0000 7f8bf0e00700 1 -- 172.21.15.153:0/1754345323 shutdown_connections 2024-03-21T10:15:27.614 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:27.612+0000 7f8bf0e00700 1 --2- 172.21.15.153:0/1754345323 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8bec069020 0x7f8bec105790 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:15:27.614 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:27.612+0000 7f8bf0e00700 1 --2- 172.21.15.153:0/1754345323 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8bec068670 0x7f8bec068a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:15:27.614 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:27.612+0000 7f8bf0e00700 1 -- 172.21.15.153:0/1754345323 >> 172.21.15.153:0/1754345323 conn(0x7f8bec0755a0 msgr2=0x7f8bec0759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:15:27.614 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:27.612+0000 7f8bf0e00700 1 -- 172.21.15.153:0/1754345323 shutdown_connections 2024-03-21T10:15:27.614 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:27.612+0000 7f8bf0e00700 1 -- 172.21.15.153:0/1754345323 wait complete. 2024-03-21T10:15:27.615 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:27.613+0000 7f8bf0e00700 1 Processor -- start 2024-03-21T10:15:27.615 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:27.613+0000 7f8bf0e00700 1 -- start start 2024-03-21T10:15:27.615 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:27.613+0000 7f8bf0e00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8bec068670 0x7f8bec19e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:15:27.615 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:27.613+0000 7f8bf0e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8bec069020 0x7f8bec19ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:15:27.615 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:27.613+0000 7f8bf0e00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bec19f3f0 con 0x7f8bec069020 2024-03-21T10:15:27.615 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:27.613+0000 7f8bf0e00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bec1988a0 con 0x7f8bec068670 2024-03-21T10:15:27.616 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:27.614+0000 7f8be9400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8bec069020 0x7f8bec19ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:15:27.616 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:27.614+0000 7f8be9400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8bec069020 0x7f8bec19ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:52380/0 (socket says 172.21.15.153:52380) 2024-03-21T10:15:27.616 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:27.614+0000 7f8be9400700 1 -- 172.21.15.153:0/1058994418 learned_addr learned my addr 172.21.15.153:0/1058994418 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:15:27.616 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:27.614+0000 7f8be9e00700 1 --2- 172.21.15.153:0/1058994418 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8bec068670 0x7f8bec19e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:15:27.616 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:27.614+0000 7f8be9400700 1 -- 172.21.15.153:0/1058994418 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8bec068670 msgr2=0x7f8bec19e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:15:27.616 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:27.614+0000 7f8be9400700 1 --2- 172.21.15.153:0/1058994418 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8bec068670 0x7f8bec19e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:15:27.617 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:27.614+0000 7f8be9400700 1 -- 172.21.15.153:0/1058994418 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bd80095f0 con 0x7f8bec069020 2024-03-21T10:15:27.617 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:27.615+0000 7f8be9400700 1 --2- 172.21.15.153:0/1058994418 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8bec069020 0x7f8bec19ed60 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8bd800bda0 tx=0x7f8bd800be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:15:27.617 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:27.615+0000 7f8be2a00700 1 -- 172.21.15.153:0/1058994418 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8bd801d070 con 0x7f8bec069020 2024-03-21T10:15:27.617 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:27.615+0000 7f8bf0e00700 1 -- 172.21.15.153:0/1058994418 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bec198b20 con 0x7f8bec069020 2024-03-21T10:15:27.617 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:27.615+0000 7f8be2a00700 1 -- 172.21.15.153:0/1058994418 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8bd8004fb0 con 0x7f8bec069020 2024-03-21T10:15:27.617 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:27.615+0000 7f8bf0e00700 1 -- 172.21.15.153:0/1058994418 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bec199010 con 0x7f8bec069020 2024-03-21T10:15:27.618 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:27.616+0000 7f8be2a00700 1 -- 172.21.15.153:0/1058994418 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8bd800f460 con 0x7f8bec069020 2024-03-21T10:15:27.618 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:27.617+0000 7f8bf0e00700 1 -- 172.21.15.153:0/1058994418 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bcc005320 con 0x7f8bec069020 2024-03-21T10:15:27.619 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:27.618+0000 7f8be2a00700 1 -- 172.21.15.153:0/1058994418 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f8bd8004af0 con 0x7f8bec069020 2024-03-21T10:15:27.620 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:27.619+0000 7f8be2a00700 1 --2- 172.21.15.153:0/1058994418 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f8bd006e460 0x7f8bd0070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:15:27.620 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:27.619+0000 7f8be2a00700 1 -- 172.21.15.153:0/1058994418 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f8bd808f7a0 con 0x7f8bec069020 2024-03-21T10:15:27.620 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:27.619+0000 7f8be9e00700 1 --2- 172.21.15.153:0/1058994418 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f8bd006e460 0x7f8bd0070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:15:27.621 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:27.620+0000 7f8be9e00700 1 --2- 172.21.15.153:0/1058994418 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f8bd006e460 0x7f8bd0070920 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc006fd0 tx=0x7f8bdc008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:15:27.629 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:27.627+0000 7f8be2a00700 1 -- 172.21.15.153:0/1058994418 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f8bd805c540 con 0x7f8bec069020 2024-03-21T10:15:27.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:27 smithi174 ceph-mon[88004]: from='client.34214 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:15:27.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:27 smithi174 ceph-mon[88004]: from='client.34218 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:15:27.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:27 smithi174 ceph-mon[88004]: from='client.34222 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:15:27.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:27 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/724294149' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:15:27.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:27 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/332341648' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:15:27.922 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:27.920+0000 7f8bf0e00700 1 -- 172.21.15.153:0/1058994418 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f8bcc0059f0 con 0x7f8bec069020 2024-03-21T10:15:27.923 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:27.921+0000 7f8be2a00700 1 -- 172.21.15.153:0/1058994418 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f8bd805c0d0 con 0x7f8bec069020 2024-03-21T10:15:27.923 INFO:teuthology.orchestra.run.smithi153.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-03-21T10:15:27.923 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-21T10:15:27.923 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-21T10:15:27.926 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:27.924+0000 7f8bf0e00700 1 -- 172.21.15.153:0/1058994418 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f8bd006e460 msgr2=0x7f8bd0070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:15:27.926 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:27.924+0000 7f8bf0e00700 1 --2- 172.21.15.153:0/1058994418 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f8bd006e460 0x7f8bd0070920 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc006fd0 tx=0x7f8bdc008040 comp rx=0 tx=0).stop 2024-03-21T10:15:27.926 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:27.924+0000 7f8bf0e00700 1 -- 172.21.15.153:0/1058994418 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8bec069020 msgr2=0x7f8bec19ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:15:27.926 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:27.924+0000 7f8bf0e00700 1 --2- 172.21.15.153:0/1058994418 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8bec069020 0x7f8bec19ed60 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8bd800bda0 tx=0x7f8bd800be80 comp rx=0 tx=0).stop 2024-03-21T10:15:27.926 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:27.924+0000 7f8bf0e00700 1 -- 172.21.15.153:0/1058994418 shutdown_connections 2024-03-21T10:15:27.926 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:27.924+0000 7f8bf0e00700 1 --2- 172.21.15.153:0/1058994418 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8bec068670 0x7f8bec19e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:15:27.926 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:27.924+0000 7f8bf0e00700 1 --2- 172.21.15.153:0/1058994418 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f8bd006e460 0x7f8bd0070920 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:15:27.927 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:27.924+0000 7f8bf0e00700 1 --2- 172.21.15.153:0/1058994418 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8bec069020 0x7f8bec19ed60 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:15:27.927 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:27.924+0000 7f8bf0e00700 1 -- 172.21.15.153:0/1058994418 >> 172.21.15.153:0/1058994418 conn(0x7f8bec0755a0 msgr2=0x7f8bec1027c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:15:27.927 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:27.924+0000 7f8bf0e00700 1 -- 172.21.15.153:0/1058994418 shutdown_connections 2024-03-21T10:15:27.927 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:27.924+0000 7f8bf0e00700 1 -- 172.21.15.153:0/1058994418 wait complete. 2024-03-21T10:15:28.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:28 smithi153 ceph-mon[113733]: pgmap v61: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:15:28.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:28 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/1058994418' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:15:28.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:28 smithi174 ceph-mon[88004]: pgmap v61: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:15:28.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:28 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/1058994418' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:15:29.237 INFO:journalctl@ceph.osd.0.smithi153.stdout:Mar 21 10:15:28 smithi153 systemd[1]: Stopping Ceph osd.0 for 070ed3e4-e769-11ee-95cd-87774f69a715... 2024-03-21T10:15:29.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:29 smithi153 ceph-mon[113733]: from='client.34234 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:15:29.557 INFO:journalctl@ceph.osd.0.smithi153.stdout:Mar 21 10:15:29 smithi153 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-0[48092]: 2024-03-21T10:15:29.333+0000 7f061c000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T10:15:29.557 INFO:journalctl@ceph.osd.0.smithi153.stdout:Mar 21 10:15:29 smithi153 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-0[48092]: 2024-03-21T10:15:29.333+0000 7f061c000700 -1 osd.0 50 *** Got signal Terminated *** 2024-03-21T10:15:29.557 INFO:journalctl@ceph.osd.0.smithi153.stdout:Mar 21 10:15:29 smithi153 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-0[48092]: 2024-03-21T10:15:29.333+0000 7f061c000700 -1 osd.0 50 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-21T10:15:29.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:29 smithi174 ceph-mon[88004]: from='client.34234 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:15:30.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:30 smithi153 ceph-mon[113733]: pgmap v62: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:15:30.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:30 smithi153 ceph-mon[113733]: osd.0 marked itself down and dead 2024-03-21T10:15:30.603 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:30 smithi174 ceph-mon[88004]: pgmap v62: 65 pgs: 65 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:15:30.603 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:30 smithi174 ceph-mon[88004]: osd.0 marked itself down and dead 2024-03-21T10:15:31.057 INFO:journalctl@ceph.osd.0.smithi153.stdout:Mar 21 10:15:30 smithi153 podman[124879]: 2024-03-21 10:15:30.794943892 +0000 UTC m=+1.624843514 container died 28912a2319b47fcba9535f95a314be34744c17e3bc2d12d1457b25df0e340b5e (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-0, GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.29.1, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=-18.2.1, RELEASE=HEAD, ceph=True, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-03-21T10:15:31.351 INFO:journalctl@ceph.osd.0.smithi153.stdout:Mar 21 10:15:31 smithi153 podman[124879]: 2024-03-21 10:15:31.344186009 +0000 UTC m=+2.174085656 container cleanup 28912a2319b47fcba9535f95a314be34744c17e3bc2d12d1457b25df0e340b5e (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-0, org.label-schema.vendor=CentOS, RELEASE=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.29.1, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=-18.2.1, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-03-21T10:15:31.352 INFO:journalctl@ceph.osd.0.smithi153.stdout:Mar 21 10:15:31 smithi153 bash[124879]: ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-0 2024-03-21T10:15:31.352 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:31 smithi153 ceph-mon[113733]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-21T10:15:31.352 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:31 smithi153 ceph-mon[113733]: osdmap e51: 6 total, 5 up, 6 in 2024-03-21T10:15:31.352 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:31 smithi153 ceph-mon[113733]: pgmap v64: 65 pgs: 9 stale+active+clean, 56 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-03-21T10:15:31.352 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:31 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [3, 2]}]: dispatch 2024-03-21T10:15:31.352 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:31 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [3, 0]}]: dispatch 2024-03-21T10:15:31.352 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:31 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [3, 0]}]: dispatch 2024-03-21T10:15:31.352 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:31 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.7", "id": [1, 0]}]: dispatch 2024-03-21T10:15:31.352 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:31 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.b", "id": [1, 2, 4, 5]}]: dispatch 2024-03-21T10:15:31.353 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:31 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.15", "id": [3, 2]}]: dispatch 2024-03-21T10:15:31.353 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:31 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.19", "id": [3, 5]}]: dispatch 2024-03-21T10:15:31.353 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:31 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1c", "id": [4, 0]}]: dispatch 2024-03-21T10:15:31.353 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:31 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:15:31.353 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:31 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:15:31.583 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:31 smithi174 ceph-mon[88004]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-21T10:15:31.583 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:31 smithi174 ceph-mon[88004]: osdmap e51: 6 total, 5 up, 6 in 2024-03-21T10:15:31.583 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:31 smithi174 ceph-mon[88004]: pgmap v64: 65 pgs: 9 stale+active+clean, 56 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-03-21T10:15:31.583 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:31 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [3, 2]}]: dispatch 2024-03-21T10:15:31.583 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:31 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [3, 0]}]: dispatch 2024-03-21T10:15:31.583 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:31 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [3, 0]}]: dispatch 2024-03-21T10:15:31.583 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:31 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.7", "id": [1, 0]}]: dispatch 2024-03-21T10:15:31.583 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:31 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.b", "id": [1, 2, 4, 5]}]: dispatch 2024-03-21T10:15:31.583 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:31 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.15", "id": [3, 2]}]: dispatch 2024-03-21T10:15:31.583 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:31 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.19", "id": [3, 5]}]: dispatch 2024-03-21T10:15:31.584 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:31 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1c", "id": [4, 0]}]: dispatch 2024-03-21T10:15:31.584 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:31 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:15:31.584 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:31 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:15:32.056 INFO:journalctl@ceph.osd.0.smithi153.stdout:Mar 21 10:15:31 smithi153 podman[124899]: 2024-03-21 10:15:31.604054923 +0000 UTC m=+0.798220337 container remove 28912a2319b47fcba9535f95a314be34744c17e3bc2d12d1457b25df0e340b5e (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-0, io.buildah.version=1.29.1, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=-18.2.1, GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=HEAD) 2024-03-21T10:15:32.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:32 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [3, 2]}]': finished 2024-03-21T10:15:32.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:32 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [3, 0]}]': finished 2024-03-21T10:15:32.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:32 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [3, 0]}]': finished 2024-03-21T10:15:32.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:32 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.7", "id": [1, 0]}]': finished 2024-03-21T10:15:32.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:32 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.b", "id": [1, 2, 4, 5]}]': finished 2024-03-21T10:15:32.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:32 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.15", "id": [3, 2]}]': finished 2024-03-21T10:15:32.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:32 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.19", "id": [3, 5]}]': finished 2024-03-21T10:15:32.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:32 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1c", "id": [4, 0]}]': finished 2024-03-21T10:15:32.558 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:32 smithi153 ceph-mon[113733]: osdmap e52: 6 total, 5 up, 6 in 2024-03-21T10:15:32.558 INFO:journalctl@ceph.osd.0.smithi153.stdout:Mar 21 10:15:32 smithi153 podman[124985]: 2024-03-21 10:15:32.314987843 +0000 UTC m=+0.036016067 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T10:15:32.558 INFO:journalctl@ceph.osd.0.smithi153.stdout:Mar 21 10:15:32 smithi153 podman[124985]: 2024-03-21 10:15:32.45500942 +0000 UTC m=+0.176037675 container create 9e1c208383575bf55fe330095d2ae9540b96b5a4419f87d747be52d29f07eecd (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-0-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-03-21T10:15:32.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:32 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [3, 2]}]': finished 2024-03-21T10:15:32.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:32 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [3, 0]}]': finished 2024-03-21T10:15:32.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:32 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [3, 0]}]': finished 2024-03-21T10:15:32.703 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:32 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.7", "id": [1, 0]}]': finished 2024-03-21T10:15:32.703 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:32 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.b", "id": [1, 2, 4, 5]}]': finished 2024-03-21T10:15:32.703 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:32 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.15", "id": [3, 2]}]': finished 2024-03-21T10:15:32.703 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:32 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.19", "id": [3, 5]}]': finished 2024-03-21T10:15:32.703 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:32 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1c", "id": [4, 0]}]': finished 2024-03-21T10:15:32.703 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:32 smithi174 ceph-mon[88004]: osdmap e52: 6 total, 5 up, 6 in 2024-03-21T10:15:33.056 INFO:journalctl@ceph.osd.0.smithi153.stdout:Mar 21 10:15:32 smithi153 podman[124985]: 2024-03-21 10:15:32.733782909 +0000 UTC m=+0.454811171 container init 9e1c208383575bf55fe330095d2ae9540b96b5a4419f87d747be52d29f07eecd (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-0-deactivate, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0) 2024-03-21T10:15:33.057 INFO:journalctl@ceph.osd.0.smithi153.stdout:Mar 21 10:15:32 smithi153 podman[124985]: 2024-03-21 10:15:32.745504537 +0000 UTC m=+0.466532819 container start 9e1c208383575bf55fe330095d2ae9540b96b5a4419f87d747be52d29f07eecd (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-0-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=) 2024-03-21T10:15:33.057 INFO:journalctl@ceph.osd.0.smithi153.stdout:Mar 21 10:15:32 smithi153 podman[124985]: 2024-03-21 10:15:32.832432532 +0000 UTC m=+0.553460773 container attach 9e1c208383575bf55fe330095d2ae9540b96b5a4419f87d747be52d29f07eecd (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-0-deactivate, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, ceph=True) 2024-03-21T10:15:33.553 INFO:journalctl@ceph.osd.0.smithi153.stdout:Mar 21 10:15:33 smithi153 podman[124985]: 2024-03-21 10:15:33.190837979 +0000 UTC m=+0.911866251 container died 9e1c208383575bf55fe330095d2ae9540b96b5a4419f87d747be52d29f07eecd (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-0-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, ceph=True) 2024-03-21T10:15:33.554 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:33 smithi153 ceph-mon[113733]: osdmap e53: 6 total, 5 up, 6 in 2024-03-21T10:15:33.554 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:33 smithi153 ceph-mon[113733]: pgmap v67: 65 pgs: 4 unknown, 4 peering, 18 active+undersized+degraded, 13 active+undersized, 26 active+clean; 293 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 66/348 objects degraded (18.966%) 2024-03-21T10:15:33.554 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:33 smithi153 ceph-mon[113733]: osdmap e54: 6 total, 5 up, 6 in 2024-03-21T10:15:33.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:33 smithi174 ceph-mon[88004]: osdmap e53: 6 total, 5 up, 6 in 2024-03-21T10:15:33.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:33 smithi174 ceph-mon[88004]: pgmap v67: 65 pgs: 4 unknown, 4 peering, 18 active+undersized+degraded, 13 active+undersized, 26 active+clean; 293 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 66/348 objects degraded (18.966%) 2024-03-21T10:15:33.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:33 smithi174 ceph-mon[88004]: osdmap e54: 6 total, 5 up, 6 in 2024-03-21T10:15:34.298 INFO:journalctl@ceph.osd.0.smithi153.stdout:Mar 21 10:15:33 smithi153 podman[125010]: 2024-03-21 10:15:33.911846305 +0000 UTC m=+0.702780347 container remove 9e1c208383575bf55fe330095d2ae9540b96b5a4419f87d747be52d29f07eecd (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-0-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.schema-version=1.0) 2024-03-21T10:15:34.298 INFO:journalctl@ceph.osd.0.smithi153.stdout:Mar 21 10:15:33 smithi153 systemd[1]: ceph-070ed3e4-e769-11ee-95cd-87774f69a715@osd.0.service: Deactivated successfully. 2024-03-21T10:15:34.298 INFO:journalctl@ceph.osd.0.smithi153.stdout:Mar 21 10:15:33 smithi153 systemd[1]: Stopped Ceph osd.0 for 070ed3e4-e769-11ee-95cd-87774f69a715. 2024-03-21T10:15:34.298 INFO:journalctl@ceph.osd.0.smithi153.stdout:Mar 21 10:15:33 smithi153 systemd[1]: ceph-070ed3e4-e769-11ee-95cd-87774f69a715@osd.0.service: Consumed 1min 37.195s CPU time. 2024-03-21T10:15:34.556 INFO:journalctl@ceph.osd.0.smithi153.stdout:Mar 21 10:15:34 smithi153 systemd[1]: Starting Ceph osd.0 for 070ed3e4-e769-11ee-95cd-87774f69a715... 2024-03-21T10:15:34.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:34 smithi153 ceph-mon[113733]: Health check failed: Degraded data redundancy: 66/348 objects degraded (18.966%), 18 pgs degraded (PG_DEGRADED) 2024-03-21T10:15:34.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:34 smithi174 ceph-mon[88004]: Health check failed: Degraded data redundancy: 66/348 objects degraded (18.966%), 18 pgs degraded (PG_DEGRADED) 2024-03-21T10:15:35.054 INFO:journalctl@ceph.osd.0.smithi153.stdout:Mar 21 10:15:34 smithi153 podman[125108]: 2024-03-21 10:15:34.616866732 +0000 UTC m=+0.032970733 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T10:15:35.055 INFO:journalctl@ceph.osd.0.smithi153.stdout:Mar 21 10:15:34 smithi153 podman[125108]: 2024-03-21 10:15:34.721243613 +0000 UTC m=+0.137347647 container create 84d207a7607bc6c4163db5855e851b120378e674bd0feed7eb97ea55a5b2aa0e (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-0-activate, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux ) 2024-03-21T10:15:35.307 INFO:journalctl@ceph.osd.0.smithi153.stdout:Mar 21 10:15:35 smithi153 podman[125108]: 2024-03-21 10:15:35.051739453 +0000 UTC m=+0.467843527 container init 84d207a7607bc6c4163db5855e851b120378e674bd0feed7eb97ea55a5b2aa0e (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-0-activate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240319, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6) 2024-03-21T10:15:35.307 INFO:journalctl@ceph.osd.0.smithi153.stdout:Mar 21 10:15:35 smithi153 podman[125108]: 2024-03-21 10:15:35.066390546 +0000 UTC m=+0.482494607 container start 84d207a7607bc6c4163db5855e851b120378e674bd0feed7eb97ea55a5b2aa0e (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-0-activate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-21T10:15:35.307 INFO:journalctl@ceph.osd.0.smithi153.stdout:Mar 21 10:15:35 smithi153 podman[125108]: 2024-03-21 10:15:35.170390431 +0000 UTC m=+0.586494437 container attach 84d207a7607bc6c4163db5855e851b120378e674bd0feed7eb97ea55a5b2aa0e (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-0-activate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, ceph=True) 2024-03-21T10:15:35.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:35 smithi174 ceph-mon[88004]: osdmap e55: 6 total, 5 up, 6 in 2024-03-21T10:15:35.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:35 smithi174 ceph-mon[88004]: pgmap v70: 65 pgs: 4 unknown, 4 peering, 18 active+undersized+degraded, 13 active+undersized, 26 active+clean; 293 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 66/348 objects degraded (18.966%) 2024-03-21T10:15:35.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:35 smithi153 ceph-mon[113733]: osdmap e55: 6 total, 5 up, 6 in 2024-03-21T10:15:35.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:35 smithi153 ceph-mon[113733]: pgmap v70: 65 pgs: 4 unknown, 4 peering, 18 active+undersized+degraded, 13 active+undersized, 26 active+clean; 293 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 66/348 objects degraded (18.966%) 2024-03-21T10:15:36.308 INFO:journalctl@ceph.osd.0.smithi153.stdout:Mar 21 10:15:35 smithi153 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-0-activate[125123]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-03-21T10:15:36.308 INFO:journalctl@ceph.osd.0.smithi153.stdout:Mar 21 10:15:35 smithi153 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-0-activate[125123]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-03-21T10:15:36.308 INFO:journalctl@ceph.osd.0.smithi153.stdout:Mar 21 10:15:35 smithi153 bash[125108]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-03-21T10:15:36.308 INFO:journalctl@ceph.osd.0.smithi153.stdout:Mar 21 10:15:35 smithi153 bash[125108]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-03-21T10:15:36.309 INFO:journalctl@ceph.osd.0.smithi153.stdout:Mar 21 10:15:35 smithi153 bash[125108]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-03-21T10:15:36.309 INFO:journalctl@ceph.osd.0.smithi153.stdout:Mar 21 10:15:35 smithi153 bash[125108]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-03-21T10:15:36.309 INFO:journalctl@ceph.osd.0.smithi153.stdout:Mar 21 10:15:35 smithi153 bash[125108]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-03-21T10:15:36.309 INFO:journalctl@ceph.osd.0.smithi153.stdout:Mar 21 10:15:35 smithi153 bash[125108]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-03-21T10:15:36.309 INFO:journalctl@ceph.osd.0.smithi153.stdout:Mar 21 10:15:35 smithi153 bash[125108]: --> ceph-volume raw activate successful for osd ID: 0 2024-03-21T10:15:36.309 INFO:journalctl@ceph.osd.0.smithi153.stdout:Mar 21 10:15:35 smithi153 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-0-activate[125123]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-03-21T10:15:36.309 INFO:journalctl@ceph.osd.0.smithi153.stdout:Mar 21 10:15:35 smithi153 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-0-activate[125123]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-03-21T10:15:36.309 INFO:journalctl@ceph.osd.0.smithi153.stdout:Mar 21 10:15:35 smithi153 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-0-activate[125123]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-03-21T10:15:36.309 INFO:journalctl@ceph.osd.0.smithi153.stdout:Mar 21 10:15:35 smithi153 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-0-activate[125123]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-03-21T10:15:36.309 INFO:journalctl@ceph.osd.0.smithi153.stdout:Mar 21 10:15:35 smithi153 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-0-activate[125123]: --> ceph-volume raw activate successful for osd ID: 0 2024-03-21T10:15:36.309 INFO:journalctl@ceph.osd.0.smithi153.stdout:Mar 21 10:15:36 smithi153 podman[125108]: 2024-03-21 10:15:36.020564151 +0000 UTC m=+1.436668191 container died 84d207a7607bc6c4163db5855e851b120378e674bd0feed7eb97ea55a5b2aa0e (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-0-activate, org.label-schema.build-date=20240319, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , ceph=True) 2024-03-21T10:15:36.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:36 smithi174 ceph-mon[88004]: osdmap e56: 6 total, 5 up, 6 in 2024-03-21T10:15:36.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:36 smithi174 ceph-mon[88004]: osdmap e57: 6 total, 5 up, 6 in 2024-03-21T10:15:36.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:36 smithi153 ceph-mon[113733]: osdmap e56: 6 total, 5 up, 6 in 2024-03-21T10:15:36.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:36 smithi153 ceph-mon[113733]: osdmap e57: 6 total, 5 up, 6 in 2024-03-21T10:15:37.155 INFO:journalctl@ceph.osd.0.smithi153.stdout:Mar 21 10:15:36 smithi153 podman[125257]: 2024-03-21 10:15:36.811925887 +0000 UTC m=+0.774455778 container remove 84d207a7607bc6c4163db5855e851b120378e674bd0feed7eb97ea55a5b2aa0e (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-0-activate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-21T10:15:37.426 INFO:journalctl@ceph.osd.0.smithi153.stdout:Mar 21 10:15:37 smithi153 podman[125320]: 2024-03-21 10:15:37.061018471 +0000 UTC m=+0.035068583 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T10:15:37.426 INFO:journalctl@ceph.osd.0.smithi153.stdout:Mar 21 10:15:37 smithi153 podman[125320]: 2024-03-21 10:15:37.187999046 +0000 UTC m=+0.162049182 container create 507900fd4791712a0c96135fc03a723571112a9198e19c14d19724a63134ee19 (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-0, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-03-21T10:15:37.426 INFO:journalctl@ceph.osd.0.smithi153.stdout:Mar 21 10:15:37 smithi153 podman[125320]: 2024-03-21 10:15:37.424470002 +0000 UTC m=+0.398520144 container init 507900fd4791712a0c96135fc03a723571112a9198e19c14d19724a63134ee19 (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-0, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6) 2024-03-21T10:15:37.679 INFO:journalctl@ceph.osd.0.smithi153.stdout:Mar 21 10:15:37 smithi153 podman[125320]: 2024-03-21 10:15:37.437127446 +0000 UTC m=+0.411177555 container start 507900fd4791712a0c96135fc03a723571112a9198e19c14d19724a63134ee19 (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-0, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-03-21T10:15:37.679 INFO:journalctl@ceph.osd.0.smithi153.stdout:Mar 21 10:15:37 smithi153 ceph-osd[125339]: -- 172.21.15.153:0/1441258633 <== mon.0 v2:172.21.15.153:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x561790605680 con 0x5617905cec00 2024-03-21T10:15:37.679 INFO:journalctl@ceph.osd.0.smithi153.stdout:Mar 21 10:15:37 smithi153 bash[125320]: 507900fd4791712a0c96135fc03a723571112a9198e19c14d19724a63134ee19 2024-03-21T10:15:37.679 INFO:journalctl@ceph.osd.0.smithi153.stdout:Mar 21 10:15:37 smithi153 systemd[1]: Started Ceph osd.0 for 070ed3e4-e769-11ee-95cd-87774f69a715. 2024-03-21T10:15:37.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:37 smithi174 ceph-mon[88004]: osdmap e58: 6 total, 5 up, 6 in 2024-03-21T10:15:37.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:37 smithi174 ceph-mon[88004]: pgmap v74: 65 pgs: 1 remapped+peering, 27 active+clean, 13 active+undersized, 18 active+undersized+degraded, 6 active+clean+remapped; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 63 MiB/s rd, 167 MiB/s wr, 823 op/s; 66/366 objects degraded (18.033%); 18/366 objects misplaced (4.918%); 6.2 MiB/s, 111 objects/s recovering 2024-03-21T10:15:38.010 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:37 smithi153 ceph-mon[113733]: osdmap e58: 6 total, 5 up, 6 in 2024-03-21T10:15:38.010 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:37 smithi153 ceph-mon[113733]: pgmap v74: 65 pgs: 1 remapped+peering, 27 active+clean, 13 active+undersized, 18 active+undersized+degraded, 6 active+clean+remapped; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 63 MiB/s rd, 167 MiB/s wr, 823 op/s; 66/366 objects degraded (18.033%); 18/366 objects misplaced (4.918%); 6.2 MiB/s, 111 objects/s recovering 2024-03-21T10:15:38.306 INFO:journalctl@ceph.osd.0.smithi153.stdout:Mar 21 10:15:38 smithi153 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-0[125335]: 2024-03-21T10:15:38.007+0000 7f151222b7c0 -1 Falling back to public interface 2024-03-21T10:15:38.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:38 smithi174 ceph-mon[88004]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-03-21T10:15:38.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:38 smithi174 ceph-mon[88004]: osdmap e59: 6 total, 5 up, 6 in 2024-03-21T10:15:38.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:38 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:15:38.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:38 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:15:38.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:38 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:15:39.029 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:38 smithi153 ceph-mon[113733]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-03-21T10:15:39.029 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:38 smithi153 ceph-mon[113733]: osdmap e59: 6 total, 5 up, 6 in 2024-03-21T10:15:39.029 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:38 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:15:39.029 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:38 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:15:39.029 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:38 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:15:39.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:39 smithi174 ceph-mon[88004]: pgmap v76: 65 pgs: 1 remapped+peering, 27 active+clean, 13 active+undersized, 18 active+undersized+degraded, 6 active+clean+remapped; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 62 MiB/s rd, 163 MiB/s wr, 800 op/s; 66/366 objects degraded (18.033%); 18/366 objects misplaced (4.918%); 6.0 MiB/s, 107 objects/s recovering 2024-03-21T10:15:40.029 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:39 smithi153 ceph-mon[113733]: pgmap v76: 65 pgs: 1 remapped+peering, 27 active+clean, 13 active+undersized, 18 active+undersized+degraded, 6 active+clean+remapped; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 62 MiB/s rd, 163 MiB/s wr, 800 op/s; 66/366 objects degraded (18.033%); 18/366 objects misplaced (4.918%); 6.0 MiB/s, 107 objects/s recovering 2024-03-21T10:15:40.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:40 smithi174 ceph-mon[88004]: Health check update: Degraded data redundancy: 66/366 objects degraded (18.033%), 18 pgs degraded (PG_DEGRADED) 2024-03-21T10:15:41.000 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:40 smithi153 ceph-mon[113733]: Health check update: Degraded data redundancy: 66/366 objects degraded (18.033%), 18 pgs degraded (PG_DEGRADED) 2024-03-21T10:15:41.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:41 smithi174 ceph-mon[88004]: pgmap v77: 65 pgs: 1 remapped+peering, 27 active+clean, 13 active+undersized, 18 active+undersized+degraded, 6 active+clean+remapped; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 42 MiB/s rd, 111 MiB/s wr, 548 op/s; 66/366 objects degraded (18.033%); 18/366 objects misplaced (4.918%); 4.1 MiB/s, 73 objects/s recovering 2024-03-21T10:15:42.051 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:41 smithi153 ceph-mon[113733]: pgmap v77: 65 pgs: 1 remapped+peering, 27 active+clean, 13 active+undersized, 18 active+undersized+degraded, 6 active+clean+remapped; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 42 MiB/s rd, 111 MiB/s wr, 548 op/s; 66/366 objects degraded (18.033%); 18/366 objects misplaced (4.918%); 4.1 MiB/s, 73 objects/s recovering 2024-03-21T10:15:42.920 INFO:journalctl@ceph.osd.0.smithi153.stdout:Mar 21 10:15:42 smithi153 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-0[125335]: 2024-03-21T10:15:42.821+0000 7f151222b7c0 -1 osd.0 0 read_superblock omap replica is missing. 2024-03-21T10:15:43.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:43 smithi153 ceph-mon[113733]: pgmap v78: 65 pgs: 27 active+clean, 13 active+undersized, 18 active+undersized+degraded, 7 active+clean+remapped; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 33 MiB/s rd, 87 MiB/s wr, 429 op/s; 66/366 objects degraded (18.033%); 18/366 objects misplaced (4.918%); 3.2 MiB/s, 57 objects/s recovering 2024-03-21T10:15:43.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:43 smithi153 ceph-mon[113733]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-03-21T10:15:43.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:43 smithi153 ceph-mon[113733]: from='osd.0 [v2:172.21.15.153:6802/2525147935,v1:172.21.15.153:6803/2525147935]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-21T10:15:43.557 INFO:journalctl@ceph.osd.0.smithi153.stdout:Mar 21 10:15:43 smithi153 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-0[125335]: 2024-03-21T10:15:43.257+0000 7f151222b7c0 -1 osd.0 50 log_to_monitors true 2024-03-21T10:15:43.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:43 smithi174 ceph-mon[88004]: pgmap v78: 65 pgs: 27 active+clean, 13 active+undersized, 18 active+undersized+degraded, 7 active+clean+remapped; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 33 MiB/s rd, 87 MiB/s wr, 429 op/s; 66/366 objects degraded (18.033%); 18/366 objects misplaced (4.918%); 3.2 MiB/s, 57 objects/s recovering 2024-03-21T10:15:43.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:43 smithi174 ceph-mon[88004]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-03-21T10:15:43.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:43 smithi174 ceph-mon[88004]: from='osd.0 [v2:172.21.15.153:6802/2525147935,v1:172.21.15.153:6803/2525147935]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-21T10:15:45.556 INFO:journalctl@ceph.osd.0.smithi153.stdout:Mar 21 10:15:45 smithi153 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-0[125335]: 2024-03-21T10:15:45.325+0000 7f14fc600700 -1 osd.0 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-21T10:15:45.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:45 smithi153 ceph-mon[113733]: from='osd.0 [v2:172.21.15.153:6802/2525147935,v1:172.21.15.153:6803/2525147935]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-21T10:15:45.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:45 smithi153 ceph-mon[113733]: osdmap e60: 6 total, 5 up, 6 in 2024-03-21T10:15:45.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:45 smithi153 ceph-mon[113733]: from='osd.0 [v2:172.21.15.153:6802/2525147935,v1:172.21.15.153:6803/2525147935]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-03-21T10:15:45.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:45 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:15:45.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:45 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:15:45.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:45 smithi174 ceph-mon[88004]: from='osd.0 [v2:172.21.15.153:6802/2525147935,v1:172.21.15.153:6803/2525147935]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-21T10:15:45.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:45 smithi174 ceph-mon[88004]: osdmap e60: 6 total, 5 up, 6 in 2024-03-21T10:15:45.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:45 smithi174 ceph-mon[88004]: from='osd.0 [v2:172.21.15.153:6802/2525147935,v1:172.21.15.153:6803/2525147935]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-03-21T10:15:45.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:45 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:15:45.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:45 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:15:46.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:46 smithi153 ceph-mon[113733]: pgmap v80: 65 pgs: 27 active+clean, 13 active+undersized, 18 active+undersized+degraded, 7 active+clean+remapped; 317 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s; 66/366 objects degraded (18.033%); 18/366 objects misplaced (4.918%) 2024-03-21T10:15:46.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:46 smithi153 ceph-mon[113733]: from='osd.0 [v2:172.21.15.153:6802/2525147935,v1:172.21.15.153:6803/2525147935]' entity='osd.0' 2024-03-21T10:15:46.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:46 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:15:46.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:46 smithi174 ceph-mon[88004]: pgmap v80: 65 pgs: 27 active+clean, 13 active+undersized, 18 active+undersized+degraded, 7 active+clean+remapped; 317 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s; 66/366 objects degraded (18.033%); 18/366 objects misplaced (4.918%) 2024-03-21T10:15:46.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:46 smithi174 ceph-mon[88004]: from='osd.0 [v2:172.21.15.153:6802/2525147935,v1:172.21.15.153:6803/2525147935]' entity='osd.0' 2024-03-21T10:15:46.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:46 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:15:47.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:47 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:15:47.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:47 smithi174 ceph-mon[88004]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-21T10:15:47.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:47 smithi174 ceph-mon[88004]: osd.0 [v2:172.21.15.153:6802/2525147935,v1:172.21.15.153:6803/2525147935] boot 2024-03-21T10:15:47.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:47 smithi174 ceph-mon[88004]: osdmap e61: 6 total, 6 up, 6 in 2024-03-21T10:15:47.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:47 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T10:15:47.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:47 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:15:47.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:47 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:15:47.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:47 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:15:47.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:47 smithi153 ceph-mon[113733]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-21T10:15:47.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:47 smithi153 ceph-mon[113733]: osd.0 [v2:172.21.15.153:6802/2525147935,v1:172.21.15.153:6803/2525147935] boot 2024-03-21T10:15:47.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:47 smithi153 ceph-mon[113733]: osdmap e61: 6 total, 6 up, 6 in 2024-03-21T10:15:47.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:47 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T10:15:47.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:47 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:15:47.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:47 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:15:48.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:48 smithi153 ceph-mon[113733]: pgmap v82: 65 pgs: 7 remapped+peering, 22 peering, 27 active+clean, 6 active+undersized, 3 active+undersized+degraded; 317 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s; 14/366 objects degraded (3.825%) 2024-03-21T10:15:48.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:48 smithi153 ceph-mon[113733]: osdmap e62: 6 total, 6 up, 6 in 2024-03-21T10:15:48.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:48 smithi153 ceph-mon[113733]: Health check update: Degraded data redundancy: 14/366 objects degraded (3.825%), 3 pgs degraded (PG_DEGRADED) 2024-03-21T10:15:48.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:48 smithi174 ceph-mon[88004]: pgmap v82: 65 pgs: 7 remapped+peering, 22 peering, 27 active+clean, 6 active+undersized, 3 active+undersized+degraded; 317 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s; 14/366 objects degraded (3.825%) 2024-03-21T10:15:48.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:48 smithi174 ceph-mon[88004]: osdmap e62: 6 total, 6 up, 6 in 2024-03-21T10:15:48.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:48 smithi174 ceph-mon[88004]: Health check update: Degraded data redundancy: 14/366 objects degraded (3.825%), 3 pgs degraded (PG_DEGRADED) 2024-03-21T10:15:49.678 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:49 smithi153 ceph-mon[113733]: osdmap e63: 6 total, 6 up, 6 in 2024-03-21T10:15:49.678 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:49 smithi153 ceph-mon[113733]: pgmap v85: 65 pgs: 7 remapped+peering, 22 peering, 27 active+clean, 6 active+undersized, 3 active+undersized+degraded; 317 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.4 KiB/s rd, 2 op/s; 14/366 objects degraded (3.825%) 2024-03-21T10:15:49.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:49 smithi174 ceph-mon[88004]: osdmap e63: 6 total, 6 up, 6 in 2024-03-21T10:15:49.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:49 smithi174 ceph-mon[88004]: pgmap v85: 65 pgs: 7 remapped+peering, 22 peering, 27 active+clean, 6 active+undersized, 3 active+undersized+degraded; 317 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.4 KiB/s rd, 2 op/s; 14/366 objects degraded (3.825%) 2024-03-21T10:15:50.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:50 smithi174 ceph-mon[88004]: osdmap e64: 6 total, 6 up, 6 in 2024-03-21T10:15:50.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:50 smithi153 ceph-mon[113733]: osdmap e64: 6 total, 6 up, 6 in 2024-03-21T10:15:51.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:51 smithi174 ceph-mon[88004]: osdmap e65: 6 total, 6 up, 6 in 2024-03-21T10:15:51.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:51 smithi174 ceph-mon[88004]: pgmap v88: 65 pgs: 7 remapped+peering, 22 peering, 36 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:15:51.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:51 smithi174 ceph-mon[88004]: osdmap e66: 6 total, 6 up, 6 in 2024-03-21T10:15:51.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:51 smithi153 ceph-mon[113733]: osdmap e65: 6 total, 6 up, 6 in 2024-03-21T10:15:51.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:51 smithi153 ceph-mon[113733]: pgmap v88: 65 pgs: 7 remapped+peering, 22 peering, 36 active+clean; 317 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:15:51.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:51 smithi153 ceph-mon[113733]: osdmap e66: 6 total, 6 up, 6 in 2024-03-21T10:15:52.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:52 smithi153 ceph-mon[113733]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 14/366 objects degraded (3.825%), 3 pgs degraded) 2024-03-21T10:15:52.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:52 smithi174 ceph-mon[88004]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 14/366 objects degraded (3.825%), 3 pgs degraded) 2024-03-21T10:15:54.058 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:53 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:15:54.058 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:53 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:15:54.058 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:53 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:15:54.058 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:53 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:15:54.059 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:53 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:15:54.059 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:53 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:15:54.059 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:53 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:15:54.059 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:53 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:15:54.059 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:53 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:15:54.059 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:53 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-21T10:15:54.059 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:53 smithi153 ceph-mon[113733]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-21T10:15:54.059 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:53 smithi153 ceph-mon[113733]: Upgrade: unsafe to stop osd(s) at this time (15 PGs are or would become offline) 2024-03-21T10:15:54.059 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:53 smithi153 ceph-mon[113733]: pgmap v90: 65 pgs: 1 remapped+peering, 64 active+clean; 317 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 847 B/s rd, 2 op/s; 2.5 MiB/s, 84 objects/s recovering 2024-03-21T10:15:54.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:53 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:15:54.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:53 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:15:54.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:53 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:15:54.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:53 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:15:54.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:53 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:15:54.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:53 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:15:54.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:53 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:15:54.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:53 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:15:54.203 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:53 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:15:54.203 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:53 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-21T10:15:54.203 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:53 smithi174 ceph-mon[88004]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-21T10:15:54.203 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:53 smithi174 ceph-mon[88004]: Upgrade: unsafe to stop osd(s) at this time (15 PGs are or would become offline) 2024-03-21T10:15:54.203 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:53 smithi174 ceph-mon[88004]: pgmap v90: 65 pgs: 1 remapped+peering, 64 active+clean; 317 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 847 B/s rd, 2 op/s; 2.5 MiB/s, 84 objects/s recovering 2024-03-21T10:15:56.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:56 smithi153 ceph-mon[113733]: pgmap v91: 65 pgs: 65 active+clean; 317 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 2 op/s; 4.7 MiB/s, 69 objects/s recovering 2024-03-21T10:15:56.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:56 smithi174 ceph-mon[88004]: pgmap v91: 65 pgs: 65 active+clean; 317 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 2 op/s; 4.7 MiB/s, 69 objects/s recovering 2024-03-21T10:15:58.096 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.091+0000 7fafb8600700 1 -- 172.21.15.153:0/3433248868 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fafb00731c0 msgr2=0x7fafb00735a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:15:58.096 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.091+0000 7fafb8600700 1 --2- 172.21.15.153:0/3433248868 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fafb00731c0 0x7fafb00735a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fafa400b0a0 tx=0x7fafa400b3b0 comp rx=0 tx=0).stop 2024-03-21T10:15:58.096 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.092+0000 7fafb8600700 1 -- 172.21.15.153:0/3433248868 shutdown_connections 2024-03-21T10:15:58.096 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.092+0000 7fafb8600700 1 --2- 172.21.15.153:0/3433248868 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fafb0073ae0 0x7fafb010ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:15:58.096 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.092+0000 7fafb8600700 1 --2- 172.21.15.153:0/3433248868 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fafb00731c0 0x7fafb00735a0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:15:58.096 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.092+0000 7fafb8600700 1 -- 172.21.15.153:0/3433248868 >> 172.21.15.153:0/3433248868 conn(0x7fafb00fc670 msgr2=0x7fafb00fea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:15:58.096 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.092+0000 7fafb8600700 1 -- 172.21.15.153:0/3433248868 shutdown_connections 2024-03-21T10:15:58.096 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.092+0000 7fafb8600700 1 -- 172.21.15.153:0/3433248868 wait complete. 2024-03-21T10:15:58.096 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.093+0000 7fafb8600700 1 Processor -- start 2024-03-21T10:15:58.096 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.093+0000 7fafb8600700 1 -- start start 2024-03-21T10:15:58.096 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.093+0000 7fafb8600700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fafb00731c0 0x7fafb0198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:15:58.097 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.093+0000 7fafb8600700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fafb0073ae0 0x7fafb0198ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:15:58.097 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.093+0000 7fafb8600700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafb01994c0 con 0x7fafb00731c0 2024-03-21T10:15:58.097 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.093+0000 7fafb8600700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafb0199630 con 0x7fafb0073ae0 2024-03-21T10:15:58.097 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.094+0000 7fafb5c00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fafb00731c0 0x7fafb0198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:15:58.097 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.094+0000 7fafb5c00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fafb00731c0 0x7fafb0198980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:36010/0 (socket says 172.21.15.153:36010) 2024-03-21T10:15:58.097 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.094+0000 7fafb5c00700 1 -- 172.21.15.153:0/2128559923 learned_addr learned my addr 172.21.15.153:0/2128559923 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:15:58.097 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.094+0000 7fafb5c00700 1 -- 172.21.15.153:0/2128559923 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fafb0073ae0 msgr2=0x7fafb0198ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:15:58.097 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.094+0000 7fafb5c00700 1 --2- 172.21.15.153:0/2128559923 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fafb0073ae0 0x7fafb0198ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:15:58.097 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.094+0000 7fafb5c00700 1 -- 172.21.15.153:0/2128559923 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafa4009d00 con 0x7fafb00731c0 2024-03-21T10:15:58.097 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.094+0000 7fafb5c00700 1 --2- 172.21.15.153:0/2128559923 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fafb00731c0 0x7fafb0198980 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fafa400b070 tx=0x7fafa4003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:15:58.098 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.095+0000 7fafae000700 1 -- 172.21.15.153:0/2128559923 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fafa400e040 con 0x7fafb00731c0 2024-03-21T10:15:58.098 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.095+0000 7fafae000700 1 -- 172.21.15.153:0/2128559923 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fafa4007c00 con 0x7fafb00731c0 2024-03-21T10:15:58.098 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.095+0000 7fafb8600700 1 -- 172.21.15.153:0/2128559923 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafb019d450 con 0x7fafb00731c0 2024-03-21T10:15:58.098 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.095+0000 7fafae000700 1 -- 172.21.15.153:0/2128559923 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fafa401d740 con 0x7fafb00731c0 2024-03-21T10:15:58.099 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.095+0000 7fafb8600700 1 -- 172.21.15.153:0/2128559923 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafb019d940 con 0x7fafb00731c0 2024-03-21T10:15:58.099 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.096+0000 7fafb8600700 1 -- 172.21.15.153:0/2128559923 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafb004ea90 con 0x7fafb00731c0 2024-03-21T10:15:58.099 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.097+0000 7fafae000700 1 -- 172.21.15.153:0/2128559923 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fafa4004040 con 0x7fafb00731c0 2024-03-21T10:15:58.100 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.098+0000 7fafae000700 1 --2- 172.21.15.153:0/2128559923 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7faf9806e460 0x7faf98070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:15:58.100 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.098+0000 7fafae000700 1 -- 172.21.15.153:0/2128559923 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6654+0+0 (secure 0 0 0) 0x7fafa408fd70 con 0x7fafb00731c0 2024-03-21T10:15:58.100 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.098+0000 7fafb5200700 1 --2- 172.21.15.153:0/2128559923 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7faf9806e460 0x7faf98070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:15:58.100 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.099+0000 7fafb5200700 1 --2- 172.21.15.153:0/2128559923 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7faf9806e460 0x7faf98070920 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7faf9c005fd0 tx=0x7faf9c005d00 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:15:58.104 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.103+0000 7fafae000700 1 -- 172.21.15.153:0/2128559923 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fafa405c970 con 0x7fafb00731c0 2024-03-21T10:15:58.347 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.345+0000 7fafb8600700 1 -- 172.21.15.153:0/2128559923 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fafb010a250 con 0x7faf9806e460 2024-03-21T10:15:58.348 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:15:58 smithi153 ceph-mon[113733]: pgmap v92: 65 pgs: 65 active+clean; 317 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 4 op/s; 3.6 MiB/s, 53 objects/s recovering 2024-03-21T10:15:58.349 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.348+0000 7fafae000700 1 -- 172.21.15.153:0/2128559923 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fafb010a250 con 0x7faf9806e460 2024-03-21T10:15:58.353 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.351+0000 7fafb8600700 1 -- 172.21.15.153:0/2128559923 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7faf9806e460 msgr2=0x7faf98070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:15:58.353 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.351+0000 7fafb8600700 1 --2- 172.21.15.153:0/2128559923 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7faf9806e460 0x7faf98070920 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7faf9c005fd0 tx=0x7faf9c005d00 comp rx=0 tx=0).stop 2024-03-21T10:15:58.353 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.351+0000 7fafb8600700 1 -- 172.21.15.153:0/2128559923 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fafb00731c0 msgr2=0x7fafb0198980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:15:58.353 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.351+0000 7fafb8600700 1 --2- 172.21.15.153:0/2128559923 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fafb00731c0 0x7fafb0198980 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fafa400b070 tx=0x7fafa4003ce0 comp rx=0 tx=0).stop 2024-03-21T10:15:58.353 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.351+0000 7fafb8600700 1 -- 172.21.15.153:0/2128559923 shutdown_connections 2024-03-21T10:15:58.353 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.351+0000 7fafb8600700 1 --2- 172.21.15.153:0/2128559923 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7faf9806e460 0x7faf98070920 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:15:58.353 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.351+0000 7fafb8600700 1 --2- 172.21.15.153:0/2128559923 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fafb00731c0 0x7fafb0198980 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:15:58.353 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.351+0000 7fafb8600700 1 --2- 172.21.15.153:0/2128559923 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fafb0073ae0 0x7fafb0198ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:15:58.353 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.351+0000 7fafb8600700 1 -- 172.21.15.153:0/2128559923 >> 172.21.15.153:0/2128559923 conn(0x7fafb00fc670 msgr2=0x7fafb01076a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:15:58.353 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.351+0000 7fafb8600700 1 -- 172.21.15.153:0/2128559923 shutdown_connections 2024-03-21T10:15:58.353 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.351+0000 7fafb8600700 1 -- 172.21.15.153:0/2128559923 wait complete. 2024-03-21T10:15:58.369 INFO:teuthology.orchestra.run.smithi153.stdout:true 2024-03-21T10:15:58.515 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.512+0000 7f9c7a600700 1 -- 172.21.15.153:0/1421517659 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c74101750 msgr2=0x7f9c741057a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:15:58.515 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.512+0000 7f9c7a600700 1 --2- 172.21.15.153:0/1421517659 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c74101750 0x7f9c741057a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c009910 tx=0x7f9c5c009c20 comp rx=0 tx=0).stop 2024-03-21T10:15:58.515 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.513+0000 7f9c7a600700 1 -- 172.21.15.153:0/1421517659 shutdown_connections 2024-03-21T10:15:58.515 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.513+0000 7f9c7a600700 1 --2- 172.21.15.153:0/1421517659 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c74101750 0x7f9c741057a0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:15:58.515 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.513+0000 7f9c7a600700 1 --2- 172.21.15.153:0/1421517659 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9c74100da0 0x7f9c74101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:15:58.515 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.513+0000 7f9c7a600700 1 -- 172.21.15.153:0/1421517659 >> 172.21.15.153:0/1421517659 conn(0x7f9c740fc630 msgr2=0x7f9c740fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:15:58.515 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.513+0000 7f9c7a600700 1 -- 172.21.15.153:0/1421517659 shutdown_connections 2024-03-21T10:15:58.515 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.513+0000 7f9c7a600700 1 -- 172.21.15.153:0/1421517659 wait complete. 2024-03-21T10:15:58.516 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.514+0000 7f9c7a600700 1 Processor -- start 2024-03-21T10:15:58.516 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.514+0000 7f9c7a600700 1 -- start start 2024-03-21T10:15:58.516 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.514+0000 7f9c7a600700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9c74100da0 0x7f9c7419e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:15:58.516 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.514+0000 7f9c7a600700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c74101750 0x7f9c7419ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:15:58.516 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.514+0000 7f9c7a600700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c7419f350 con 0x7f9c74101750 2024-03-21T10:15:58.517 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.514+0000 7f9c7a600700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c74198800 con 0x7f9c74100da0 2024-03-21T10:15:58.517 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.515+0000 7f9c6aa00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c74101750 0x7f9c7419ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:15:58.517 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.515+0000 7f9c6aa00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c74101750 0x7f9c7419ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:36024/0 (socket says 172.21.15.153:36024) 2024-03-21T10:15:58.517 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.515+0000 7f9c6aa00700 1 -- 172.21.15.153:0/412467030 learned_addr learned my addr 172.21.15.153:0/412467030 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:15:58.518 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.515+0000 7f9c73400700 1 --2- 172.21.15.153:0/412467030 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9c74100da0 0x7f9c7419e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:15:58.518 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.515+0000 7f9c6aa00700 1 -- 172.21.15.153:0/412467030 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9c74100da0 msgr2=0x7f9c7419e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:15:58.518 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.515+0000 7f9c6aa00700 1 --2- 172.21.15.153:0/412467030 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9c74100da0 0x7f9c7419e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:15:58.518 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.515+0000 7f9c6aa00700 1 -- 172.21.15.153:0/412467030 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c5c0095f0 con 0x7f9c74101750 2024-03-21T10:15:58.518 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.515+0000 7f9c73400700 1 --2- 172.21.15.153:0/412467030 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9c74100da0 0x7f9c7419e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:15:58.518 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.515+0000 7f9c6aa00700 1 --2- 172.21.15.153:0/412467030 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c74101750 0x7f9c7419ecc0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c00bd70 tx=0x7f9c5c00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:15:58.518 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.516+0000 7f9c70c00700 1 -- 172.21.15.153:0/412467030 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9c5c01d070 con 0x7f9c74101750 2024-03-21T10:15:58.518 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.516+0000 7f9c70c00700 1 -- 172.21.15.153:0/412467030 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9c5c005dc0 con 0x7f9c74101750 2024-03-21T10:15:58.519 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.516+0000 7f9c7a600700 1 -- 172.21.15.153:0/412467030 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c74198a80 con 0x7f9c74101750 2024-03-21T10:15:58.519 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.516+0000 7f9c70c00700 1 -- 172.21.15.153:0/412467030 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9c5c017400 con 0x7f9c74101750 2024-03-21T10:15:58.519 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.516+0000 7f9c7a600700 1 -- 172.21.15.153:0/412467030 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c74198f70 con 0x7f9c74101750 2024-03-21T10:15:58.519 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.517+0000 7f9c7a600700 1 -- 172.21.15.153:0/412467030 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c74104f20 con 0x7f9c74101750 2024-03-21T10:15:58.519 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.518+0000 7f9c70c00700 1 -- 172.21.15.153:0/412467030 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f9c5c00f7f0 con 0x7f9c74101750 2024-03-21T10:15:58.521 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.519+0000 7f9c70c00700 1 --2- 172.21.15.153:0/412467030 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f9c5806e460 0x7f9c58070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:15:58.521 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.520+0000 7f9c70c00700 1 -- 172.21.15.153:0/412467030 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6654+0+0 (secure 0 0 0) 0x7f9c5c08f9b0 con 0x7f9c74101750 2024-03-21T10:15:58.521 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.520+0000 7f9c73400700 1 --2- 172.21.15.153:0/412467030 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f9c5806e460 0x7f9c58070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:15:58.522 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.520+0000 7f9c73400700 1 --2- 172.21.15.153:0/412467030 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f9c5806e460 0x7f9c58070920 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9c64005fd0 tx=0x7f9c64005ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:15:58.526 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.524+0000 7f9c70c00700 1 -- 172.21.15.153:0/412467030 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f9c5c05c5b0 con 0x7f9c74101750 2024-03-21T10:15:58.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:15:58 smithi174 ceph-mon[88004]: pgmap v92: 65 pgs: 65 active+clean; 317 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 4 op/s; 3.6 MiB/s, 53 objects/s recovering 2024-03-21T10:15:58.751 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.749+0000 7f9c7a600700 1 -- 172.21.15.153:0/412467030 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9c7402cd30 con 0x7f9c5806e460 2024-03-21T10:15:58.752 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.751+0000 7f9c70c00700 1 -- 172.21.15.153:0/412467030 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f9c7402cd30 con 0x7f9c5806e460 2024-03-21T10:15:58.755 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.754+0000 7f9c7a600700 1 -- 172.21.15.153:0/412467030 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f9c5806e460 msgr2=0x7f9c58070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:15:58.756 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.754+0000 7f9c7a600700 1 --2- 172.21.15.153:0/412467030 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f9c5806e460 0x7f9c58070920 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9c64005fd0 tx=0x7f9c64005ee0 comp rx=0 tx=0).stop 2024-03-21T10:15:58.756 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.754+0000 7f9c7a600700 1 -- 172.21.15.153:0/412467030 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c74101750 msgr2=0x7f9c7419ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:15:58.756 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.754+0000 7f9c7a600700 1 --2- 172.21.15.153:0/412467030 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c74101750 0x7f9c7419ecc0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c00bd70 tx=0x7f9c5c00be50 comp rx=0 tx=0).stop 2024-03-21T10:15:58.756 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.754+0000 7f9c7a600700 1 -- 172.21.15.153:0/412467030 shutdown_connections 2024-03-21T10:15:58.756 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.754+0000 7f9c7a600700 1 --2- 172.21.15.153:0/412467030 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9c74100da0 0x7f9c7419e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:15:58.756 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.754+0000 7f9c7a600700 1 --2- 172.21.15.153:0/412467030 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f9c5806e460 0x7f9c58070920 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:15:58.757 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.754+0000 7f9c7a600700 1 --2- 172.21.15.153:0/412467030 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c74101750 0x7f9c7419ecc0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:15:58.757 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.754+0000 7f9c7a600700 1 -- 172.21.15.153:0/412467030 >> 172.21.15.153:0/412467030 conn(0x7f9c740fc630 msgr2=0x7f9c740fe9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:15:58.757 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.754+0000 7f9c7a600700 1 -- 172.21.15.153:0/412467030 shutdown_connections 2024-03-21T10:15:58.757 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.755+0000 7f9c7a600700 1 -- 172.21.15.153:0/412467030 wait complete. 2024-03-21T10:15:58.909 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.907+0000 7f236fc00700 1 -- 172.21.15.153:0/2727076299 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f23681071d0 msgr2=0x7f2368107650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:15:58.909 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.907+0000 7f236fc00700 1 --2- 172.21.15.153:0/2727076299 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f23681071d0 0x7f2368107650 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2354009910 tx=0x7f2354009c20 comp rx=0 tx=0).stop 2024-03-21T10:15:58.909 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.907+0000 7f236fc00700 1 -- 172.21.15.153:0/2727076299 shutdown_connections 2024-03-21T10:15:58.909 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.907+0000 7f236fc00700 1 --2- 172.21.15.153:0/2727076299 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f23681071d0 0x7f2368107650 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:15:58.909 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.907+0000 7f236fc00700 1 --2- 172.21.15.153:0/2727076299 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f236810d280 0x7f236810d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:15:58.910 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.907+0000 7f236fc00700 1 -- 172.21.15.153:0/2727076299 >> 172.21.15.153:0/2727076299 conn(0x7f23680769e0 msgr2=0x7f2368078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:15:58.910 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.907+0000 7f236fc00700 1 -- 172.21.15.153:0/2727076299 shutdown_connections 2024-03-21T10:15:58.910 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.907+0000 7f236fc00700 1 -- 172.21.15.153:0/2727076299 wait complete. 2024-03-21T10:15:58.910 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.909+0000 7f236fc00700 1 Processor -- start 2024-03-21T10:15:58.910 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.909+0000 7f236fc00700 1 -- start start 2024-03-21T10:15:58.911 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.909+0000 7f236fc00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f23681071d0 0x7f236819ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:15:58.911 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.909+0000 7f236fc00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f236810d280 0x7f236819d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:15:58.912 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.909+0000 7f236fc00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f236819d880 con 0x7f236810d280 2024-03-21T10:15:58.912 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.909+0000 7f236fc00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23681a05b0 con 0x7f23681071d0 2024-03-21T10:15:58.912 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.909+0000 7f2367e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f236810d280 0x7f236819d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:15:58.912 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.910+0000 7f2367e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f236810d280 0x7f236819d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:36048/0 (socket says 172.21.15.153:36048) 2024-03-21T10:15:58.912 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.910+0000 7f2367e00700 1 -- 172.21.15.153:0/2512541190 learned_addr learned my addr 172.21.15.153:0/2512541190 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:15:58.912 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.910+0000 7f236d200700 1 --2- 172.21.15.153:0/2512541190 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f23681071d0 0x7f236819ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:15:58.912 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.910+0000 7f2367e00700 1 -- 172.21.15.153:0/2512541190 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f23681071d0 msgr2=0x7f236819ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:15:58.912 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.910+0000 7f2367e00700 1 --2- 172.21.15.153:0/2512541190 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f23681071d0 0x7f236819ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:15:58.912 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.910+0000 7f2367e00700 1 -- 172.21.15.153:0/2512541190 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23540095f0 con 0x7f236810d280 2024-03-21T10:15:58.913 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.910+0000 7f2367e00700 1 --2- 172.21.15.153:0/2512541190 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f236810d280 0x7f236819d1f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2354006010 tx=0x7f235400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:15:58.913 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.911+0000 7f2365600700 1 -- 172.21.15.153:0/2512541190 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f235401d070 con 0x7f236810d280 2024-03-21T10:15:58.913 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.911+0000 7f2365600700 1 -- 172.21.15.153:0/2512541190 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f235400f460 con 0x7f236810d280 2024-03-21T10:15:58.913 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.911+0000 7f236fc00700 1 -- 172.21.15.153:0/2512541190 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23681a0830 con 0x7f236810d280 2024-03-21T10:15:58.913 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.911+0000 7f2365600700 1 -- 172.21.15.153:0/2512541190 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2354017630 con 0x7f236810d280 2024-03-21T10:15:58.913 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.911+0000 7f236fc00700 1 -- 172.21.15.153:0/2512541190 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23681a0da0 con 0x7f236810d280 2024-03-21T10:15:58.914 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.912+0000 7f236fc00700 1 -- 172.21.15.153:0/2512541190 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f236810f6c0 con 0x7f236810d280 2024-03-21T10:15:58.914 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.913+0000 7f2365600700 1 -- 172.21.15.153:0/2512541190 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f235400f5d0 con 0x7f236810d280 2024-03-21T10:15:58.915 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.914+0000 7f2365600700 1 --2- 172.21.15.153:0/2512541190 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f235006e460 0x7f2350070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:15:58.916 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.914+0000 7f2365600700 1 -- 172.21.15.153:0/2512541190 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6654+0+0 (secure 0 0 0) 0x7f235408f8d0 con 0x7f236810d280 2024-03-21T10:15:58.916 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.914+0000 7f236d200700 1 --2- 172.21.15.153:0/2512541190 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f235006e460 0x7f2350070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:15:58.916 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.915+0000 7f236d200700 1 --2- 172.21.15.153:0/2512541190 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f235006e460 0x7f2350070920 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f235c005950 tx=0x7f235c0058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:15:58.920 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:58.918+0000 7f2365600700 1 -- 172.21.15.153:0/2512541190 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2354058a40 con 0x7f236810d280 2024-03-21T10:15:59.140 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:59.138+0000 7f236fc00700 1 -- 172.21.15.153:0/2512541190 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f236802cce0 con 0x7f235006e460 2024-03-21T10:15:59.170 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:59.168+0000 7f2365600700 1 -- 172.21.15.153:0/2512541190 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f236802cce0 con 0x7f235006e460 2024-03-21T10:15:59.170 INFO:teuthology.orchestra.run.smithi153.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-21T10:15:59.171 INFO:teuthology.orchestra.run.smithi153.stdout:alertmanager.smithi153 smithi153 *:9093,9094 running (4m) 14s ago 18m 14.1M - 0.25.0 c8568f914cd2 9ed3f2200b5f 2024-03-21T10:15:59.171 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi153 smithi153 running (18m) 14s ago 18m 8506k - 18.2.1 5be31c24972a 5b61994396a6 2024-03-21T10:15:59.171 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi174 smithi174 running (17m) 47s ago 17m 8363k - 18.2.1 5be31c24972a 4f7a5fc3ac28 2024-03-21T10:15:59.171 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi153 smithi153 running (60s) 14s ago 18m 6656k - 19.0.0-2231-g2edfee68 237e43c7a65f 32c21104d5b0 2024-03-21T10:15:59.171 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi174 smithi174 running (52s) 47s ago 17m 6660k - 19.0.0-2231-g2edfee68 237e43c7a65f a04e48967ab6 2024-03-21T10:15:59.171 INFO:teuthology.orchestra.run.smithi153.stdout:grafana.smithi153 smithi153 *:3000 running (4m) 14s ago 18m 75.2M - 9.4.12 2bacad6d85d8 ee24026ab786 2024-03-21T10:15:59.171 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.iuyluj smithi153 running (11m) 14s ago 11m 128M - 18.2.1 5be31c24972a 02f667cee979 2024-03-21T10:15:59.171 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.zipckr smithi153 running (11m) 14s ago 11m 58.6M - 18.2.1 5be31c24972a 68a853929f8f 2024-03-21T10:15:59.171 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.nhozag smithi174 running (11m) 47s ago 11m 57.5M - 18.2.1 5be31c24972a 2fee4000308f 2024-03-21T10:15:59.171 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.yyimst smithi174 running (11m) 47s ago 11m 117M - 18.2.1 5be31c24972a d78a729220d1 2024-03-21T10:15:59.171 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi153.tluxni smithi153 *:8443,9283,8765 running (7m) 14s ago 20m 499M - 19.0.0-2231-g2edfee68 237e43c7a65f 7ffee0e55382 2024-03-21T10:15:59.172 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi174.qpcwat smithi174 *:8443,9283,8765 running (6m) 47s ago 17m 450M - 19.0.0-2231-g2edfee68 237e43c7a65f 748968f216ab 2024-03-21T10:15:59.172 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi153 smithi153 running (2m) 14s ago 20m 45.4M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f dd8038303dc3 2024-03-21T10:15:59.172 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi174 smithi174 running (2m) 47s ago 16m 36.6M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f 29d78a788f2c 2024-03-21T10:15:59.172 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi153 smithi153 *:9100 running (5m) 14s ago 18m 7936k - 1.5.0 0da6a335fe13 bb835edc0e0b 2024-03-21T10:15:59.172 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi174 smithi174 *:9100 running (5m) 47s ago 17m 7600k - 1.5.0 0da6a335fe13 7c4ecad790fa 2024-03-21T10:15:59.172 INFO:teuthology.orchestra.run.smithi153.stdout:osd.0 smithi153 running (22s) 14s ago 15m 11.1M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 507900fd4791 2024-03-21T10:15:59.172 INFO:teuthology.orchestra.run.smithi153.stdout:osd.1 smithi153 running (15m) 14s ago 15m 229M 1217M 18.2.1 5be31c24972a ca60681e2314 2024-03-21T10:15:59.172 INFO:teuthology.orchestra.run.smithi153.stdout:osd.2 smithi153 running (14m) 14s ago 14m 185M 1217M 18.2.1 5be31c24972a 8db799f8962a 2024-03-21T10:15:59.172 INFO:teuthology.orchestra.run.smithi153.stdout:osd.3 smithi174 running (14m) 47s ago 14m 234M 2241M 18.2.1 5be31c24972a dafc89e0658d 2024-03-21T10:15:59.172 INFO:teuthology.orchestra.run.smithi153.stdout:osd.4 smithi174 running (13m) 47s ago 13m 224M 2241M 18.2.1 5be31c24972a c4cd15701f05 2024-03-21T10:15:59.172 INFO:teuthology.orchestra.run.smithi153.stdout:osd.5 smithi174 running (13m) 47s ago 13m 203M 2241M 18.2.1 5be31c24972a 3b69886ce22e 2024-03-21T10:15:59.172 INFO:teuthology.orchestra.run.smithi153.stdout:prometheus.smithi153 smithi153 *:9095 running (5m) 14s ago 17m 71.2M - 2.43.0 a07b618ecd1d 7d95561e50c2 2024-03-21T10:15:59.173 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:59.171+0000 7f236fc00700 1 -- 172.21.15.153:0/2512541190 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f235006e460 msgr2=0x7f2350070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:15:59.173 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:59.171+0000 7f236fc00700 1 --2- 172.21.15.153:0/2512541190 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f235006e460 0x7f2350070920 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f235c005950 tx=0x7f235c0058e0 comp rx=0 tx=0).stop 2024-03-21T10:15:59.173 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:59.171+0000 7f236fc00700 1 -- 172.21.15.153:0/2512541190 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f236810d280 msgr2=0x7f236819d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:15:59.174 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:59.171+0000 7f236fc00700 1 --2- 172.21.15.153:0/2512541190 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f236810d280 0x7f236819d1f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2354006010 tx=0x7f235400be20 comp rx=0 tx=0).stop 2024-03-21T10:15:59.174 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:59.172+0000 7f236fc00700 1 -- 172.21.15.153:0/2512541190 shutdown_connections 2024-03-21T10:15:59.174 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:59.172+0000 7f236fc00700 1 --2- 172.21.15.153:0/2512541190 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f23681071d0 0x7f236819ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:15:59.174 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:59.172+0000 7f236fc00700 1 --2- 172.21.15.153:0/2512541190 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f235006e460 0x7f2350070920 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:15:59.174 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:59.172+0000 7f236fc00700 1 --2- 172.21.15.153:0/2512541190 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f236810d280 0x7f236819d1f0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:15:59.174 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:59.172+0000 7f236fc00700 1 -- 172.21.15.153:0/2512541190 >> 172.21.15.153:0/2512541190 conn(0x7f23680769e0 msgr2=0x7f2368078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:15:59.174 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:59.172+0000 7f236fc00700 1 -- 172.21.15.153:0/2512541190 shutdown_connections 2024-03-21T10:15:59.174 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:59.172+0000 7f236fc00700 1 -- 172.21.15.153:0/2512541190 wait complete. 2024-03-21T10:15:59.328 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:59.326+0000 7fbf94a00700 1 -- 172.21.15.153:0/3733774128 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbf901039a0 msgr2=0x7fbf901079f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:15:59.329 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:59.326+0000 7fbf94a00700 1 --2- 172.21.15.153:0/3733774128 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbf901039a0 0x7fbf901079f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fbf78009910 tx=0x7fbf78009c20 comp rx=0 tx=0).stop 2024-03-21T10:15:59.329 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:59.326+0000 7fbf94a00700 1 -- 172.21.15.153:0/3733774128 shutdown_connections 2024-03-21T10:15:59.329 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:59.326+0000 7fbf94a00700 1 --2- 172.21.15.153:0/3733774128 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbf901039a0 0x7fbf901079f0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:15:59.329 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:59.326+0000 7fbf94a00700 1 --2- 172.21.15.153:0/3733774128 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbf90102ff0 0x7fbf901033d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:15:59.329 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:59.326+0000 7fbf94a00700 1 -- 172.21.15.153:0/3733774128 >> 172.21.15.153:0/3733774128 conn(0x7fbf900fe880 msgr2=0x7fbf90100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:15:59.329 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:59.326+0000 7fbf94a00700 1 -- 172.21.15.153:0/3733774128 shutdown_connections 2024-03-21T10:15:59.329 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:59.326+0000 7fbf94a00700 1 -- 172.21.15.153:0/3733774128 wait complete. 2024-03-21T10:15:59.331 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:59.327+0000 7fbf94a00700 1 Processor -- start 2024-03-21T10:15:59.331 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:59.328+0000 7fbf94a00700 1 -- start start 2024-03-21T10:15:59.331 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:59.328+0000 7fbf94a00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbf90102ff0 0x7fbf901989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:15:59.331 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:59.328+0000 7fbf94a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbf901039a0 0x7fbf90198ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:15:59.331 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:59.328+0000 7fbf94a00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf90199570 con 0x7fbf901039a0 2024-03-21T10:15:59.331 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:59.328+0000 7fbf94a00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf9019d380 con 0x7fbf90102ff0 2024-03-21T10:15:59.331 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:59.328+0000 7fbf85400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbf901039a0 0x7fbf90198ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:15:59.331 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:59.328+0000 7fbf85400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbf901039a0 0x7fbf90198ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:36058/0 (socket says 172.21.15.153:36058) 2024-03-21T10:15:59.331 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:59.328+0000 7fbf85400700 1 -- 172.21.15.153:0/4131018455 learned_addr learned my addr 172.21.15.153:0/4131018455 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:15:59.332 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:59.329+0000 7fbf8de00700 1 --2- 172.21.15.153:0/4131018455 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbf90102ff0 0x7fbf901989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:15:59.332 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:59.329+0000 7fbf85400700 1 -- 172.21.15.153:0/4131018455 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbf90102ff0 msgr2=0x7fbf901989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:15:59.332 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:59.329+0000 7fbf85400700 1 --2- 172.21.15.153:0/4131018455 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbf90102ff0 0x7fbf901989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:15:59.332 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:59.329+0000 7fbf85400700 1 -- 172.21.15.153:0/4131018455 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf780095f0 con 0x7fbf901039a0 2024-03-21T10:15:59.332 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:59.329+0000 7fbf85400700 1 --2- 172.21.15.153:0/4131018455 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbf901039a0 0x7fbf90198ee0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fbf7800bda0 tx=0x7fbf7800be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:15:59.332 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:59.329+0000 7fbf87400700 1 -- 172.21.15.153:0/4131018455 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbf7801d070 con 0x7fbf901039a0 2024-03-21T10:15:59.332 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:59.329+0000 7fbf87400700 1 -- 172.21.15.153:0/4131018455 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbf7800f460 con 0x7fbf901039a0 2024-03-21T10:15:59.332 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:59.329+0000 7fbf87400700 1 -- 172.21.15.153:0/4131018455 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbf78017610 con 0x7fbf901039a0 2024-03-21T10:15:59.332 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:59.329+0000 7fbf94a00700 1 -- 172.21.15.153:0/4131018455 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf9019d600 con 0x7fbf901039a0 2024-03-21T10:15:59.333 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:59.330+0000 7fbf94a00700 1 -- 172.21.15.153:0/4131018455 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf9019daf0 con 0x7fbf901039a0 2024-03-21T10:15:59.333 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:59.331+0000 7fbf94a00700 1 -- 172.21.15.153:0/4131018455 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf9010b3f0 con 0x7fbf901039a0 2024-03-21T10:15:59.334 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:59.333+0000 7fbf87400700 1 -- 172.21.15.153:0/4131018455 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fbf7800fa40 con 0x7fbf901039a0 2024-03-21T10:15:59.335 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:59.334+0000 7fbf87400700 1 --2- 172.21.15.153:0/4131018455 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fbf7406e460 0x7fbf74070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:15:59.335 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:59.334+0000 7fbf87400700 1 -- 172.21.15.153:0/4131018455 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6654+0+0 (secure 0 0 0) 0x7fbf7808fa60 con 0x7fbf901039a0 2024-03-21T10:15:59.336 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:59.334+0000 7fbf8de00700 1 --2- 172.21.15.153:0/4131018455 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fbf7406e460 0x7fbf74070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:15:59.336 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:59.335+0000 7fbf8de00700 1 --2- 172.21.15.153:0/4131018455 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fbf7406e460 0x7fbf74070920 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbf80005950 tx=0x7fbf800058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:15:59.339 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:59.337+0000 7fbf87400700 1 -- 172.21.15.153:0/4131018455 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fbf7805c760 con 0x7fbf901039a0 2024-03-21T10:15:59.651 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:59.649+0000 7fbf94a00700 1 -- 172.21.15.153:0/4131018455 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fbf9004ea90 con 0x7fbf901039a0 2024-03-21T10:15:59.651 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:59.650+0000 7fbf87400700 1 -- 172.21.15.153:0/4131018455 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+791 (secure 0 0 0) 0x7fbf7805c2f0 con 0x7fbf901039a0 2024-03-21T10:15:59.652 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:15:59.652 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": { 2024-03-21T10:15:59.652 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:15:59.652 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:15:59.653 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": { 2024-03-21T10:15:59.653 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:15:59.653 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:15:59.653 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": { 2024-03-21T10:15:59.653 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 5, 2024-03-21T10:15:59.653 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 1 2024-03-21T10:15:59.653 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:15:59.653 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": { 2024-03-21T10:15:59.653 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-03-21T10:15:59.653 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:15:59.653 INFO:teuthology.orchestra.run.smithi153.stdout: "overall": { 2024-03-21T10:15:59.653 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 9, 2024-03-21T10:15:59.653 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 5 2024-03-21T10:15:59.653 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-03-21T10:15:59.653 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:15:59.654 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:59.652+0000 7fbf94a00700 1 -- 172.21.15.153:0/4131018455 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fbf7406e460 msgr2=0x7fbf74070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:15:59.654 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:59.652+0000 7fbf94a00700 1 --2- 172.21.15.153:0/4131018455 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fbf7406e460 0x7fbf74070920 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbf80005950 tx=0x7fbf800058e0 comp rx=0 tx=0).stop 2024-03-21T10:15:59.654 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:59.652+0000 7fbf94a00700 1 -- 172.21.15.153:0/4131018455 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbf901039a0 msgr2=0x7fbf90198ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:15:59.655 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:59.652+0000 7fbf94a00700 1 --2- 172.21.15.153:0/4131018455 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbf901039a0 0x7fbf90198ee0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fbf7800bda0 tx=0x7fbf7800be80 comp rx=0 tx=0).stop 2024-03-21T10:15:59.655 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:59.652+0000 7fbf94a00700 1 -- 172.21.15.153:0/4131018455 shutdown_connections 2024-03-21T10:15:59.655 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:59.652+0000 7fbf94a00700 1 --2- 172.21.15.153:0/4131018455 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbf90102ff0 0x7fbf901989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:15:59.655 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:59.652+0000 7fbf94a00700 1 --2- 172.21.15.153:0/4131018455 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fbf7406e460 0x7fbf74070920 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:15:59.655 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:59.652+0000 7fbf94a00700 1 --2- 172.21.15.153:0/4131018455 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbf901039a0 0x7fbf90198ee0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:15:59.655 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:59.652+0000 7fbf94a00700 1 -- 172.21.15.153:0/4131018455 >> 172.21.15.153:0/4131018455 conn(0x7fbf900fe880 msgr2=0x7fbf90100c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:15:59.655 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:59.652+0000 7fbf94a00700 1 -- 172.21.15.153:0/4131018455 shutdown_connections 2024-03-21T10:15:59.655 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:59.653+0000 7fbf94a00700 1 -- 172.21.15.153:0/4131018455 wait complete. 2024-03-21T10:15:59.817 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:59.808+0000 7f9284e00700 1 -- 172.21.15.153:0/2143140844 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f92801071d0 msgr2=0x7f9280107650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:15:59.818 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:59.808+0000 7f9284e00700 1 --2- 172.21.15.153:0/2143140844 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f92801071d0 0x7f9280107650 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9268009910 tx=0x7f9268009c20 comp rx=0 tx=0).stop 2024-03-21T10:15:59.818 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:59.808+0000 7f9284e00700 1 -- 172.21.15.153:0/2143140844 shutdown_connections 2024-03-21T10:15:59.818 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:59.808+0000 7f9284e00700 1 --2- 172.21.15.153:0/2143140844 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f92801071d0 0x7f9280107650 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:15:59.818 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:59.808+0000 7f9284e00700 1 --2- 172.21.15.153:0/2143140844 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f928010d280 0x7f928010d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:15:59.818 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:59.808+0000 7f9284e00700 1 -- 172.21.15.153:0/2143140844 >> 172.21.15.153:0/2143140844 conn(0x7f92800769e0 msgr2=0x7f9280078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:15:59.819 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:59.808+0000 7f9284e00700 1 -- 172.21.15.153:0/2143140844 shutdown_connections 2024-03-21T10:15:59.819 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:59.808+0000 7f9284e00700 1 -- 172.21.15.153:0/2143140844 wait complete. 2024-03-21T10:15:59.819 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:59.809+0000 7f9284e00700 1 Processor -- start 2024-03-21T10:15:59.820 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:59.809+0000 7f9284e00700 1 -- start start 2024-03-21T10:15:59.820 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:59.809+0000 7f9284e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f92801071d0 0x7f928019ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:15:59.820 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:59.809+0000 7f9284e00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f928010d280 0x7f928019d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:15:59.820 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:59.810+0000 7f9284e00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f928019d7f0 con 0x7f92801071d0 2024-03-21T10:15:59.821 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:59.810+0000 7f9284e00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f928019d960 con 0x7f928010d280 2024-03-21T10:15:59.821 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:59.810+0000 7f927de00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f92801071d0 0x7f928019ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:15:59.821 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:59.810+0000 7f927de00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f92801071d0 0x7f928019ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:36072/0 (socket says 172.21.15.153:36072) 2024-03-21T10:15:59.821 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:59.810+0000 7f927de00700 1 -- 172.21.15.153:0/824423743 learned_addr learned my addr 172.21.15.153:0/824423743 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:15:59.821 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:59.810+0000 7f927d400700 1 --2- 172.21.15.153:0/824423743 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f928010d280 0x7f928019d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:15:59.821 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:59.810+0000 7f927de00700 1 -- 172.21.15.153:0/824423743 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f928010d280 msgr2=0x7f928019d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:15:59.822 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:59.810+0000 7f927de00700 1 --2- 172.21.15.153:0/824423743 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f928010d280 0x7f928019d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:15:59.822 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:59.810+0000 7f927de00700 1 -- 172.21.15.153:0/824423743 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92680095f0 con 0x7f92801071d0 2024-03-21T10:15:59.822 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:59.811+0000 7f927de00700 1 --2- 172.21.15.153:0/824423743 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f92801071d0 0x7f928019ccb0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f927000c930 tx=0x7f927000ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:15:59.822 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:59.811+0000 7f9276a00700 1 -- 172.21.15.153:0/824423743 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9270007ad0 con 0x7f92801071d0 2024-03-21T10:15:59.823 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:59.811+0000 7f9276a00700 1 -- 172.21.15.153:0/824423743 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9270007c30 con 0x7f92801071d0 2024-03-21T10:15:59.823 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:59.811+0000 7f9284e00700 1 -- 172.21.15.153:0/824423743 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92801a0700 con 0x7f92801071d0 2024-03-21T10:15:59.823 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:59.811+0000 7f9276a00700 1 -- 172.21.15.153:0/824423743 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9270018610 con 0x7f92801071d0 2024-03-21T10:15:59.823 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:59.811+0000 7f9284e00700 1 -- 172.21.15.153:0/824423743 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92801a0cd0 con 0x7f92801071d0 2024-03-21T10:15:59.824 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:59.812+0000 7f9284e00700 1 -- 172.21.15.153:0/824423743 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f928010f6c0 con 0x7f92801071d0 2024-03-21T10:15:59.825 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:59.813+0000 7f9276a00700 1 -- 172.21.15.153:0/824423743 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f927000f450 con 0x7f92801071d0 2024-03-21T10:15:59.826 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:59.814+0000 7f9276a00700 1 --2- 172.21.15.153:0/824423743 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f926406e460 0x7f9264070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:15:59.826 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:59.814+0000 7f9276a00700 1 -- 172.21.15.153:0/824423743 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6654+0+0 (secure 0 0 0) 0x7f927008e680 con 0x7f92801071d0 2024-03-21T10:15:59.826 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:59.815+0000 7f927d400700 1 --2- 172.21.15.153:0/824423743 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f926406e460 0x7f9264070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:15:59.826 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:59.815+0000 7f927d400700 1 --2- 172.21.15.153:0/824423743 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f926406e460 0x7f9264070920 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9268000c00 tx=0x7f9268005fd0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:15:59.827 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:15:59.819+0000 7f9276a00700 1 -- 172.21.15.153:0/824423743 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f92700577f0 con 0x7f92801071d0 2024-03-21T10:16:00.076 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:00.074+0000 7f9284e00700 1 -- 172.21.15.153:0/824423743 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f928004ea90 con 0x7f92801071d0 2024-03-21T10:16:00.077 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:00.075+0000 7f9276a00700 1 -- 172.21.15.153:0/824423743 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 21 v21) v1 ==== 76+0+1903 (secure 0 0 0) 0x7f927005aeb0 con 0x7f92801071d0 2024-03-21T10:16:00.078 INFO:teuthology.orchestra.run.smithi153.stdout:e21 2024-03-21T10:16:00.078 INFO:teuthology.orchestra.run.smithi153.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-21T10:16:00.078 INFO:teuthology.orchestra.run.smithi153.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:16:00.078 INFO:teuthology.orchestra.run.smithi153.stdout:legacy client fscid: 1 2024-03-21T10:16:00.079 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:16:00.079 INFO:teuthology.orchestra.run.smithi153.stdout:Filesystem 'cephfs' (1) 2024-03-21T10:16:00.079 INFO:teuthology.orchestra.run.smithi153.stdout:fs_name cephfs 2024-03-21T10:16:00.079 INFO:teuthology.orchestra.run.smithi153.stdout:epoch 21 2024-03-21T10:16:00.079 INFO:teuthology.orchestra.run.smithi153.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-21T10:16:00.079 INFO:teuthology.orchestra.run.smithi153.stdout:created 2024-03-21T10:04:27.086576+0000 2024-03-21T10:16:00.079 INFO:teuthology.orchestra.run.smithi153.stdout:modified 2024-03-21T10:13:01.455594+0000 2024-03-21T10:16:00.079 INFO:teuthology.orchestra.run.smithi153.stdout:tableserver 0 2024-03-21T10:16:00.079 INFO:teuthology.orchestra.run.smithi153.stdout:root 0 2024-03-21T10:16:00.079 INFO:teuthology.orchestra.run.smithi153.stdout:session_timeout 60 2024-03-21T10:16:00.079 INFO:teuthology.orchestra.run.smithi153.stdout:session_autoclose 300 2024-03-21T10:16:00.079 INFO:teuthology.orchestra.run.smithi153.stdout:max_file_size 1099511627776 2024-03-21T10:16:00.079 INFO:teuthology.orchestra.run.smithi153.stdout:max_xattr_size 65536 2024-03-21T10:16:00.079 INFO:teuthology.orchestra.run.smithi153.stdout:required_client_features {} 2024-03-21T10:16:00.080 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure 0 2024-03-21T10:16:00.080 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure_osd_epoch 47 2024-03-21T10:16:00.080 INFO:teuthology.orchestra.run.smithi153.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:16:00.080 INFO:teuthology.orchestra.run.smithi153.stdout:max_mds 2 2024-03-21T10:16:00.080 INFO:teuthology.orchestra.run.smithi153.stdout:in 0,1 2024-03-21T10:16:00.080 INFO:teuthology.orchestra.run.smithi153.stdout:up {0=14484,1=24273} 2024-03-21T10:16:00.080 INFO:teuthology.orchestra.run.smithi153.stdout:failed 2024-03-21T10:16:00.080 INFO:teuthology.orchestra.run.smithi153.stdout:damaged 2024-03-21T10:16:00.080 INFO:teuthology.orchestra.run.smithi153.stdout:stopped 2024-03-21T10:16:00.080 INFO:teuthology.orchestra.run.smithi153.stdout:data_pools [3] 2024-03-21T10:16:00.080 INFO:teuthology.orchestra.run.smithi153.stdout:metadata_pool 2 2024-03-21T10:16:00.080 INFO:teuthology.orchestra.run.smithi153.stdout:inline_data enabled 2024-03-21T10:16:00.080 INFO:teuthology.orchestra.run.smithi153.stdout:balancer 2024-03-21T10:16:00.080 INFO:teuthology.orchestra.run.smithi153.stdout:bal_rank_mask -1 2024-03-21T10:16:00.081 INFO:teuthology.orchestra.run.smithi153.stdout:standby_count_wanted 1 2024-03-21T10:16:00.081 INFO:teuthology.orchestra.run.smithi153.stdout:qdb_cluster leader: 0 members: 2024-03-21T10:16:00.081 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.iuyluj{0:14484} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.153:6828/866138999,v1:172.21.15.153:6829/866138999] compat {c=[1],r=[1],i=[7ff]}] 2024-03-21T10:16:00.081 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.zipckr{0:14494} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.153:6826/4033622615,v1:172.21.15.153:6827/4033622615] compat {c=[1],r=[1],i=[7ff]}] 2024-03-21T10:16:00.081 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.yyimst{1:24273} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.174:6824/1776947618,v1:172.21.15.174:6825/1776947618] compat {c=[1],r=[1],i=[7ff]}] 2024-03-21T10:16:00.081 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.nhozag{1:24285} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.174:6826/806095873,v1:172.21.15.174:6827/806095873] compat {c=[1],r=[1],i=[7ff]}] 2024-03-21T10:16:00.081 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:16:00.081 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:16:00.082 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:00.078+0000 7f9284e00700 1 -- 172.21.15.153:0/824423743 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f926406e460 msgr2=0x7f9264070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:16:00.082 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:00.078+0000 7f9284e00700 1 --2- 172.21.15.153:0/824423743 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f926406e460 0x7f9264070920 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9268000c00 tx=0x7f9268005fd0 comp rx=0 tx=0).stop 2024-03-21T10:16:00.083 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:00.078+0000 7f9284e00700 1 -- 172.21.15.153:0/824423743 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f92801071d0 msgr2=0x7f928019ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:16:00.083 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:00.078+0000 7f9284e00700 1 --2- 172.21.15.153:0/824423743 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f92801071d0 0x7f928019ccb0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f927000c930 tx=0x7f927000ccf0 comp rx=0 tx=0).stop 2024-03-21T10:16:00.083 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:00.078+0000 7f9284e00700 1 -- 172.21.15.153:0/824423743 shutdown_connections 2024-03-21T10:16:00.083 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:00.078+0000 7f9284e00700 1 --2- 172.21.15.153:0/824423743 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f926406e460 0x7f9264070920 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:16:00.083 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:00.079+0000 7f9284e00700 1 --2- 172.21.15.153:0/824423743 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f92801071d0 0x7f928019ccb0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:16:00.083 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:00.079+0000 7f9284e00700 1 --2- 172.21.15.153:0/824423743 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f928010d280 0x7f928019d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:16:00.083 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:00.079+0000 7f9284e00700 1 -- 172.21.15.153:0/824423743 >> 172.21.15.153:0/824423743 conn(0x7f92800769e0 msgr2=0x7f9280078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:16:00.083 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:00.079+0000 7f9284e00700 1 -- 172.21.15.153:0/824423743 shutdown_connections 2024-03-21T10:16:00.083 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:00.079+0000 7f9284e00700 1 -- 172.21.15.153:0/824423743 wait complete. 2024-03-21T10:16:00.083 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 21 2024-03-21T10:16:00.237 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:00.235+0000 7f99b3000700 1 -- 172.21.15.153:0/3625917413 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f99ac102fe0 msgr2=0x7f99ac1033c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:16:00.237 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:00.235+0000 7f99b3000700 1 --2- 172.21.15.153:0/3625917413 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f99ac102fe0 0x7f99ac1033c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f999c009910 tx=0x7f999c009c20 comp rx=0 tx=0).stop 2024-03-21T10:16:00.237 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:00.235+0000 7f99b3000700 1 -- 172.21.15.153:0/3625917413 shutdown_connections 2024-03-21T10:16:00.237 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:00.235+0000 7f99b3000700 1 --2- 172.21.15.153:0/3625917413 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f99ac103990 0x7f99ac1079e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:16:00.237 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:00.235+0000 7f99b3000700 1 --2- 172.21.15.153:0/3625917413 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f99ac102fe0 0x7f99ac1033c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:16:00.237 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:00.235+0000 7f99b3000700 1 -- 172.21.15.153:0/3625917413 >> 172.21.15.153:0/3625917413 conn(0x7f99ac0fe8d0 msgr2=0x7f99ac100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:16:00.237 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:00.235+0000 7f99b3000700 1 -- 172.21.15.153:0/3625917413 shutdown_connections 2024-03-21T10:16:00.237 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:00.235+0000 7f99b3000700 1 -- 172.21.15.153:0/3625917413 wait complete. 2024-03-21T10:16:00.238 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:00.236+0000 7f99b3000700 1 Processor -- start 2024-03-21T10:16:00.238 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:00.236+0000 7f99b3000700 1 -- start start 2024-03-21T10:16:00.238 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:00.237+0000 7f99b3000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f99ac102fe0 0x7f99ac19e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:16:00.238 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:00.237+0000 7f99b3000700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f99ac103990 0x7f99ac19ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:16:00.239 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:00.237+0000 7f99b3000700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99ac19f3d0 con 0x7f99ac102fe0 2024-03-21T10:16:00.239 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:00.237+0000 7f99b3000700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99ac198880 con 0x7f99ac103990 2024-03-21T10:16:00.239 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:00.237+0000 7f99abe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f99ac102fe0 0x7f99ac19e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:16:00.239 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:00.237+0000 7f99abe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f99ac102fe0 0x7f99ac19e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:36092/0 (socket says 172.21.15.153:36092) 2024-03-21T10:16:00.239 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:00.237+0000 7f99abe00700 1 -- 172.21.15.153:0/3176812881 learned_addr learned my addr 172.21.15.153:0/3176812881 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:16:00.239 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:00.237+0000 7f99abe00700 1 -- 172.21.15.153:0/3176812881 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f99ac103990 msgr2=0x7f99ac19ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:16:00.239 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:00.237+0000 7f99abe00700 1 --2- 172.21.15.153:0/3176812881 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f99ac103990 0x7f99ac19ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:16:00.240 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:00.237+0000 7f99abe00700 1 -- 172.21.15.153:0/3176812881 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f999c0095f0 con 0x7f99ac102fe0 2024-03-21T10:16:00.240 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:00.238+0000 7f99abe00700 1 --2- 172.21.15.153:0/3176812881 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f99ac102fe0 0x7f99ac19e800 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f999c006010 tx=0x7f999c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:16:00.241 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:00.239+0000 7f99a9600700 1 -- 172.21.15.153:0/3176812881 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f999c01d070 con 0x7f99ac102fe0 2024-03-21T10:16:00.241 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:00.239+0000 7f99a9600700 1 -- 172.21.15.153:0/3176812881 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f999c00f460 con 0x7f99ac102fe0 2024-03-21T10:16:00.241 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:00.239+0000 7f99a9600700 1 -- 172.21.15.153:0/3176812881 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f999c0175f0 con 0x7f99ac102fe0 2024-03-21T10:16:00.241 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:00.239+0000 7f99b3000700 1 -- 172.21.15.153:0/3176812881 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99ac198b00 con 0x7f99ac102fe0 2024-03-21T10:16:00.241 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:00.239+0000 7f99b3000700 1 -- 172.21.15.153:0/3176812881 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99ac198fa0 con 0x7f99ac102fe0 2024-03-21T10:16:00.242 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:00.240+0000 7f99b3000700 1 -- 172.21.15.153:0/3176812881 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99ac10b380 con 0x7f99ac102fe0 2024-03-21T10:16:00.243 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:00.241+0000 7f99a9600700 1 -- 172.21.15.153:0/3176812881 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f999c00f5d0 con 0x7f99ac102fe0 2024-03-21T10:16:00.244 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:00.242+0000 7f99a9600700 1 --2- 172.21.15.153:0/3176812881 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f999006e460 0x7f9990070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:16:00.244 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:00.242+0000 7f99a9600700 1 -- 172.21.15.153:0/3176812881 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6654+0+0 (secure 0 0 0) 0x7f999c08fa30 con 0x7f99ac102fe0 2024-03-21T10:16:00.244 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:00.243+0000 7f99a3e00700 1 --2- 172.21.15.153:0/3176812881 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f999006e460 0x7f9990070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:16:00.244 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:00.243+0000 7f99a3e00700 1 --2- 172.21.15.153:0/3176812881 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f999006e460 0x7f9990070920 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f99ac0ff140 tx=0x7f9994006d20 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:16:00.249 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:00.247+0000 7f99a9600700 1 -- 172.21.15.153:0/3176812881 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f999c05c630 con 0x7f99ac102fe0 2024-03-21T10:16:00.488 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:00.486+0000 7f99b3000700 1 -- 172.21.15.153:0/3176812881 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f99ac02cd40 con 0x7f999006e460 2024-03-21T10:16:00.489 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:00 smithi153 ceph-mon[113733]: from='client.34246 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:16:00.489 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:00 smithi153 ceph-mon[113733]: from='client.34250 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:16:00.489 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:00 smithi153 ceph-mon[113733]: from='client.34254 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:16:00.489 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:00 smithi153 ceph-mon[113733]: pgmap v93: 65 pgs: 65 active+clean; 317 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s; 3.2 MiB/s, 47 objects/s recovering 2024-03-21T10:16:00.490 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:00 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/4131018455' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:16:00.490 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:00 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/824423743' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:16:00.490 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:16:00.491 INFO:teuthology.orchestra.run.smithi153.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232", 2024-03-21T10:16:00.491 INFO:teuthology.orchestra.run.smithi153.stdout: "in_progress": true, 2024-03-21T10:16:00.491 INFO:teuthology.orchestra.run.smithi153.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-21T10:16:00.491 INFO:teuthology.orchestra.run.smithi153.stdout: "services_complete": [ 2024-03-21T10:16:00.491 INFO:teuthology.orchestra.run.smithi153.stdout: "crash", 2024-03-21T10:16:00.491 INFO:teuthology.orchestra.run.smithi153.stdout: "mon", 2024-03-21T10:16:00.492 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr" 2024-03-21T10:16:00.492 INFO:teuthology.orchestra.run.smithi153.stdout: ], 2024-03-21T10:16:00.492 INFO:teuthology.orchestra.run.smithi153.stdout: "progress": "7/23 daemons upgraded", 2024-03-21T10:16:00.492 INFO:teuthology.orchestra.run.smithi153.stdout: "message": "Currently upgrading osd daemons", 2024-03-21T10:16:00.492 INFO:teuthology.orchestra.run.smithi153.stdout: "is_paused": false 2024-03-21T10:16:00.492 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:16:00.494 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:00.488+0000 7f99a9600700 1 -- 172.21.15.153:0/3176812881 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f99ac02cd40 con 0x7f999006e460 2024-03-21T10:16:00.494 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:00.491+0000 7f99b3000700 1 -- 172.21.15.153:0/3176812881 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f999006e460 msgr2=0x7f9990070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:16:00.494 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:00.491+0000 7f99b3000700 1 --2- 172.21.15.153:0/3176812881 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f999006e460 0x7f9990070920 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f99ac0ff140 tx=0x7f9994006d20 comp rx=0 tx=0).stop 2024-03-21T10:16:00.494 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:00.491+0000 7f99b3000700 1 -- 172.21.15.153:0/3176812881 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f99ac102fe0 msgr2=0x7f99ac19e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:16:00.495 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:00.491+0000 7f99b3000700 1 --2- 172.21.15.153:0/3176812881 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f99ac102fe0 0x7f99ac19e800 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f999c006010 tx=0x7f999c00be20 comp rx=0 tx=0).stop 2024-03-21T10:16:00.495 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:00.491+0000 7f99b3000700 1 -- 172.21.15.153:0/3176812881 shutdown_connections 2024-03-21T10:16:00.495 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:00.491+0000 7f99b3000700 1 --2- 172.21.15.153:0/3176812881 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f999006e460 0x7f9990070920 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:16:00.495 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:00.491+0000 7f99b3000700 1 --2- 172.21.15.153:0/3176812881 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f99ac102fe0 0x7f99ac19e800 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:16:00.495 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:00.491+0000 7f99b3000700 1 --2- 172.21.15.153:0/3176812881 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f99ac103990 0x7f99ac19ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:16:00.495 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:00.491+0000 7f99b3000700 1 -- 172.21.15.153:0/3176812881 >> 172.21.15.153:0/3176812881 conn(0x7f99ac0fe8d0 msgr2=0x7f99ac0fff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:16:00.496 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:00.491+0000 7f99b3000700 1 -- 172.21.15.153:0/3176812881 shutdown_connections 2024-03-21T10:16:00.496 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:00.492+0000 7f99b3000700 1 -- 172.21.15.153:0/3176812881 wait complete. 2024-03-21T10:16:00.505 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:00 smithi174 ceph-mon[88004]: from='client.34246 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:16:00.505 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:00 smithi174 ceph-mon[88004]: from='client.34250 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:16:00.505 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:00 smithi174 ceph-mon[88004]: from='client.34254 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:16:00.505 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:00 smithi174 ceph-mon[88004]: pgmap v93: 65 pgs: 65 active+clean; 317 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s; 3.2 MiB/s, 47 objects/s recovering 2024-03-21T10:16:00.505 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:00 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/4131018455' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:16:00.505 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:00 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/824423743' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:16:00.649 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:00.646+0000 7f2b32c00700 1 -- 172.21.15.153:0/1696259578 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2b2c1071d0 msgr2=0x7f2b2c107650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:16:00.649 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:00.646+0000 7f2b32c00700 1 --2- 172.21.15.153:0/1696259578 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2b2c1071d0 0x7f2b2c107650 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2b18009910 tx=0x7f2b18009c20 comp rx=0 tx=0).stop 2024-03-21T10:16:00.649 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:00.646+0000 7f2b32c00700 1 -- 172.21.15.153:0/1696259578 shutdown_connections 2024-03-21T10:16:00.649 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:00.646+0000 7f2b32c00700 1 --2- 172.21.15.153:0/1696259578 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2b2c1071d0 0x7f2b2c107650 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:16:00.649 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:00.646+0000 7f2b32c00700 1 --2- 172.21.15.153:0/1696259578 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b2c10d280 0x7f2b2c10d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:16:00.649 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:00.646+0000 7f2b32c00700 1 -- 172.21.15.153:0/1696259578 >> 172.21.15.153:0/1696259578 conn(0x7f2b2c0769e0 msgr2=0x7f2b2c078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:16:00.650 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:00.646+0000 7f2b32c00700 1 -- 172.21.15.153:0/1696259578 shutdown_connections 2024-03-21T10:16:00.650 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:00.646+0000 7f2b32c00700 1 -- 172.21.15.153:0/1696259578 wait complete. 2024-03-21T10:16:00.650 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:00.647+0000 7f2b32c00700 1 Processor -- start 2024-03-21T10:16:00.650 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:00.647+0000 7f2b32c00700 1 -- start start 2024-03-21T10:16:00.653 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:00.648+0000 7f2b32c00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b2c1071d0 0x7f2b2c19ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:16:00.653 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:00.648+0000 7f2b32c00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2b2c10d280 0x7f2b2c19d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:16:00.653 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:00.648+0000 7f2b32c00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b2c19d880 con 0x7f2b2c10d280 2024-03-21T10:16:00.653 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:00.648+0000 7f2b32c00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b2c1a05b0 con 0x7f2b2c1071d0 2024-03-21T10:16:00.653 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:00.648+0000 7f2b2b400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2b2c10d280 0x7f2b2c19d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:16:00.653 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:00.648+0000 7f2b2b400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2b2c10d280 0x7f2b2c19d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:36120/0 (socket says 172.21.15.153:36120) 2024-03-21T10:16:00.654 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:00.648+0000 7f2b2b400700 1 -- 172.21.15.153:0/1394667261 learned_addr learned my addr 172.21.15.153:0/1394667261 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:16:00.654 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:00.648+0000 7f2b2be00700 1 --2- 172.21.15.153:0/1394667261 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b2c1071d0 0x7f2b2c19ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:16:00.654 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:00.648+0000 7f2b2b400700 1 -- 172.21.15.153:0/1394667261 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b2c1071d0 msgr2=0x7f2b2c19ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:16:00.654 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:00.648+0000 7f2b2b400700 1 --2- 172.21.15.153:0/1394667261 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b2c1071d0 0x7f2b2c19ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:16:00.654 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:00.648+0000 7f2b2b400700 1 -- 172.21.15.153:0/1394667261 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b180095f0 con 0x7f2b2c10d280 2024-03-21T10:16:00.654 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:00.649+0000 7f2b2b400700 1 --2- 172.21.15.153:0/1394667261 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2b2c10d280 0x7f2b2c19d1f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f2b1800b5c0 tx=0x7f2b1800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:16:00.655 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:00.649+0000 7f2b28c00700 1 -- 172.21.15.153:0/1394667261 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2b1801d070 con 0x7f2b2c10d280 2024-03-21T10:16:00.655 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:00.649+0000 7f2b28c00700 1 -- 172.21.15.153:0/1394667261 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2b18005e00 con 0x7f2b2c10d280 2024-03-21T10:16:00.655 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:00.649+0000 7f2b32c00700 1 -- 172.21.15.153:0/1394667261 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b2c1a0830 con 0x7f2b2c10d280 2024-03-21T10:16:00.655 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:00.649+0000 7f2b28c00700 1 -- 172.21.15.153:0/1394667261 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2b18017400 con 0x7f2b2c10d280 2024-03-21T10:16:00.655 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:00.649+0000 7f2b32c00700 1 -- 172.21.15.153:0/1394667261 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b2c1a0da0 con 0x7f2b2c10d280 2024-03-21T10:16:00.655 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:00.650+0000 7f2b32c00700 1 -- 172.21.15.153:0/1394667261 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b2c10f6c0 con 0x7f2b2c10d280 2024-03-21T10:16:00.655 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:00.651+0000 7f2b28c00700 1 -- 172.21.15.153:0/1394667261 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f2b1800f460 con 0x7f2b2c10d280 2024-03-21T10:16:00.656 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:00.652+0000 7f2b28c00700 1 --2- 172.21.15.153:0/1394667261 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f2b1006e460 0x7f2b10070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:16:00.657 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:00.652+0000 7f2b28c00700 1 -- 172.21.15.153:0/1394667261 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6654+0+0 (secure 0 0 0) 0x7f2b1808fa00 con 0x7f2b2c10d280 2024-03-21T10:16:00.657 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:00.653+0000 7f2b2be00700 1 --2- 172.21.15.153:0/1394667261 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f2b1006e460 0x7f2b10070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:16:00.657 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:00.653+0000 7f2b2be00700 1 --2- 172.21.15.153:0/1394667261 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f2b1006e460 0x7f2b10070920 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c005950 tx=0x7f2b1c0058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:16:00.658 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:00.657+0000 7f2b28c00700 1 -- 172.21.15.153:0/1394667261 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2b18058ad0 con 0x7f2b2c10d280 2024-03-21T10:16:00.963 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:00.961+0000 7f2b32c00700 1 -- 172.21.15.153:0/1394667261 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f2b2c04ea90 con 0x7f2b2c10d280 2024-03-21T10:16:00.964 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:00.962+0000 7f2b28c00700 1 -- 172.21.15.153:0/1394667261 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f2b1805c190 con 0x7f2b2c10d280 2024-03-21T10:16:00.964 INFO:teuthology.orchestra.run.smithi153.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-03-21T10:16:00.964 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-21T10:16:00.965 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-21T10:16:00.967 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:00.965+0000 7f2b32c00700 1 -- 172.21.15.153:0/1394667261 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f2b1006e460 msgr2=0x7f2b10070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:16:00.967 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:00.965+0000 7f2b32c00700 1 --2- 172.21.15.153:0/1394667261 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f2b1006e460 0x7f2b10070920 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c005950 tx=0x7f2b1c0058e0 comp rx=0 tx=0).stop 2024-03-21T10:16:00.967 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:00.965+0000 7f2b32c00700 1 -- 172.21.15.153:0/1394667261 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2b2c10d280 msgr2=0x7f2b2c19d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:16:00.968 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:00.965+0000 7f2b32c00700 1 --2- 172.21.15.153:0/1394667261 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2b2c10d280 0x7f2b2c19d1f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f2b1800b5c0 tx=0x7f2b1800be20 comp rx=0 tx=0).stop 2024-03-21T10:16:00.968 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:00.965+0000 7f2b32c00700 1 -- 172.21.15.153:0/1394667261 shutdown_connections 2024-03-21T10:16:00.968 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:00.965+0000 7f2b32c00700 1 --2- 172.21.15.153:0/1394667261 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b2c1071d0 0x7f2b2c19ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:16:00.968 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:00.965+0000 7f2b32c00700 1 --2- 172.21.15.153:0/1394667261 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f2b1006e460 0x7f2b10070920 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:16:00.968 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:00.965+0000 7f2b32c00700 1 --2- 172.21.15.153:0/1394667261 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2b2c10d280 0x7f2b2c19d1f0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:16:00.968 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:00.965+0000 7f2b32c00700 1 -- 172.21.15.153:0/1394667261 >> 172.21.15.153:0/1394667261 conn(0x7f2b2c0769e0 msgr2=0x7f2b2c078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:16:00.969 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:00.965+0000 7f2b32c00700 1 -- 172.21.15.153:0/1394667261 shutdown_connections 2024-03-21T10:16:00.969 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:00.965+0000 7f2b32c00700 1 -- 172.21.15.153:0/1394667261 wait complete. 2024-03-21T10:16:01.532 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:01 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/1394667261' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:16:01.532 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:01 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:16:01.532 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:01 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:16:01.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:01 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/1394667261' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:16:01.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:01 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:16:01.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:01 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:16:02.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:02 smithi174 ceph-mon[88004]: from='client.34266 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:16:02.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:02 smithi174 ceph-mon[88004]: pgmap v94: 65 pgs: 65 active+clean; 317 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s; 2.8 MiB/s, 41 objects/s recovering 2024-03-21T10:16:02.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:02 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:16:02.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:02 smithi153 ceph-mon[113733]: from='client.34266 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:16:02.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:02 smithi153 ceph-mon[113733]: pgmap v94: 65 pgs: 65 active+clean; 317 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s; 2.8 MiB/s, 41 objects/s recovering 2024-03-21T10:16:02.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:02 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:16:03.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:03 smithi174 ceph-mon[88004]: pgmap v95: 65 pgs: 65 active+clean; 317 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 4 op/s; 2.4 MiB/s, 35 objects/s recovering 2024-03-21T10:16:03.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:03 smithi153 ceph-mon[113733]: pgmap v95: 65 pgs: 65 active+clean; 317 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 4 op/s; 2.4 MiB/s, 35 objects/s recovering 2024-03-21T10:16:06.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:06 smithi153 ceph-mon[113733]: pgmap v96: 65 pgs: 65 active+clean; 317 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s; 1.3 MiB/s, 0 objects/s recovering 2024-03-21T10:16:06.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:06 smithi174 ceph-mon[88004]: pgmap v96: 65 pgs: 65 active+clean; 317 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s; 1.3 MiB/s, 0 objects/s recovering 2024-03-21T10:16:08.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:08 smithi153 ceph-mon[113733]: pgmap v97: 65 pgs: 65 active+clean; 317 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:16:08.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:08 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-21T10:16:08.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:08 smithi174 ceph-mon[88004]: pgmap v97: 65 pgs: 65 active+clean; 317 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:16:08.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:08 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-21T10:16:09.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:09 smithi153 ceph-mon[113733]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-21T10:16:09.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:09 smithi153 ceph-mon[113733]: Upgrade: osd.1 is safe to restart 2024-03-21T10:16:09.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:09 smithi174 ceph-mon[88004]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-21T10:16:09.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:09 smithi174 ceph-mon[88004]: Upgrade: osd.1 is safe to restart 2024-03-21T10:16:10.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:10 smithi153 ceph-mon[113733]: pgmap v98: 65 pgs: 65 active+clean; 317 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:16:10.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:10 smithi174 ceph-mon[88004]: pgmap v98: 65 pgs: 65 active+clean; 317 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:16:11.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:11 smithi153 ceph-mon[113733]: Upgrade: Updating osd.1 2024-03-21T10:16:11.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:11 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:16:11.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:11 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-21T10:16:11.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:11 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:16:11.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:11 smithi153 ceph-mon[113733]: Deploying daemon osd.1 on smithi153 2024-03-21T10:16:11.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:11 smithi153 ceph-mon[113733]: pgmap v99: 65 pgs: 65 active+clean; 317 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:16:11.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:11 smithi174 ceph-mon[88004]: Upgrade: Updating osd.1 2024-03-21T10:16:11.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:11 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:16:11.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:11 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-21T10:16:11.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:11 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:16:11.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:11 smithi174 ceph-mon[88004]: Deploying daemon osd.1 on smithi153 2024-03-21T10:16:11.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:11 smithi174 ceph-mon[88004]: pgmap v99: 65 pgs: 65 active+clean; 317 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:16:14.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:14 smithi153 ceph-mon[113733]: pgmap v100: 65 pgs: 65 active+clean; 317 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:16:14.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:14 smithi174 ceph-mon[88004]: pgmap v100: 65 pgs: 65 active+clean; 317 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:16:15.306 INFO:journalctl@ceph.osd.1.smithi153.stdout:Mar 21 10:16:15 smithi153 systemd[1]: Stopping Ceph osd.1 for 070ed3e4-e769-11ee-95cd-87774f69a715... 2024-03-21T10:16:15.807 INFO:journalctl@ceph.osd.1.smithi153.stdout:Mar 21 10:16:15 smithi153 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-1[55561]: 2024-03-21T10:16:15.466+0000 7f963d400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T10:16:15.807 INFO:journalctl@ceph.osd.1.smithi153.stdout:Mar 21 10:16:15 smithi153 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-1[55561]: 2024-03-21T10:16:15.466+0000 7f963d400700 -1 osd.1 66 *** Got signal Terminated *** 2024-03-21T10:16:15.807 INFO:journalctl@ceph.osd.1.smithi153.stdout:Mar 21 10:16:15 smithi153 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-1[55561]: 2024-03-21T10:16:15.466+0000 7f963d400700 -1 osd.1 66 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-21T10:16:16.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:16 smithi153 ceph-mon[113733]: pgmap v101: 65 pgs: 65 active+clean; 317 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:16:16.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:16 smithi153 ceph-mon[113733]: osd.1 marked itself down and dead 2024-03-21T10:16:16.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:16 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:16:16.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:16 smithi174 ceph-mon[88004]: pgmap v101: 65 pgs: 65 active+clean; 317 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:16:16.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:16 smithi174 ceph-mon[88004]: osd.1 marked itself down and dead 2024-03-21T10:16:16.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:16 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:16:16.978 INFO:journalctl@ceph.osd.1.smithi153.stdout:Mar 21 10:16:16 smithi153 podman[130753]: 2024-03-21 10:16:16.670483136 +0000 UTC m=+1.360442353 container died ca60681e23149f65b1ebb22ad05ddf924d26b91d2bc02d9ff190e5f5a6ab404e (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-1, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=HEAD, io.buildah.version=1.29.1, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=https://github.com/ceph/ceph-container.git, CEPH_POINT_RELEASE=-18.2.1, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True) 2024-03-21T10:16:17.277 INFO:journalctl@ceph.osd.1.smithi153.stdout:Mar 21 10:16:17 smithi153 podman[130753]: 2024-03-21 10:16:17.093047215 +0000 UTC m=+1.783006425 container cleanup ca60681e23149f65b1ebb22ad05ddf924d26b91d2bc02d9ff190e5f5a6ab404e (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-1, ceph=True, GIT_CLEAN=True, RELEASE=HEAD, GIT_REPO=https://github.com/ceph/ceph-container.git, io.buildah.version=1.29.1, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240222, CEPH_POINT_RELEASE=-18.2.1, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2) 2024-03-21T10:16:17.277 INFO:journalctl@ceph.osd.1.smithi153.stdout:Mar 21 10:16:17 smithi153 bash[130753]: ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-1 2024-03-21T10:16:17.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:17 smithi153 ceph-mon[113733]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-21T10:16:17.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:17 smithi153 ceph-mon[113733]: osdmap e67: 6 total, 5 up, 6 in 2024-03-21T10:16:17.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:17 smithi153 ceph-mon[113733]: pgmap v103: 65 pgs: 28 peering, 2 stale+active+clean, 35 active+clean; 317 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 2 op/s 2024-03-21T10:16:17.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:17 smithi153 ceph-mon[113733]: osdmap e68: 6 total, 5 up, 6 in 2024-03-21T10:16:17.557 INFO:journalctl@ceph.osd.1.smithi153.stdout:Mar 21 10:16:17 smithi153 podman[130770]: 2024-03-21 10:16:17.369648978 +0000 UTC m=+0.683493134 container remove ca60681e23149f65b1ebb22ad05ddf924d26b91d2bc02d9ff190e5f5a6ab404e (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-1, GIT_BRANCH=HEAD, GIT_REPO=https://github.com/ceph/ceph-container.git, io.buildah.version=1.29.1, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=-18.2.1, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, RELEASE=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-03-21T10:16:17.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:17 smithi174 ceph-mon[88004]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-21T10:16:17.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:17 smithi174 ceph-mon[88004]: osdmap e67: 6 total, 5 up, 6 in 2024-03-21T10:16:17.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:17 smithi174 ceph-mon[88004]: pgmap v103: 65 pgs: 28 peering, 2 stale+active+clean, 35 active+clean; 317 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 2 op/s 2024-03-21T10:16:17.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:17 smithi174 ceph-mon[88004]: osdmap e68: 6 total, 5 up, 6 in 2024-03-21T10:16:18.534 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:18 smithi153 ceph-mon[113733]: Health check failed: Reduced data availability: 9 pgs peering (PG_AVAILABILITY) 2024-03-21T10:16:18.535 INFO:journalctl@ceph.osd.1.smithi153.stdout:Mar 21 10:16:18 smithi153 podman[130858]: 2024-03-21 10:16:18.119322624 +0000 UTC m=+0.035372123 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T10:16:18.535 INFO:journalctl@ceph.osd.1.smithi153.stdout:Mar 21 10:16:18 smithi153 podman[130858]: 2024-03-21 10:16:18.22882911 +0000 UTC m=+0.144878643 container create 771ad7ae4e9e1a8b2088b2db3d5ed13bcd47d81d524205dd0f71e24b9769d8a4 (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-1-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-21T10:16:18.535 INFO:journalctl@ceph.osd.1.smithi153.stdout:Mar 21 10:16:18 smithi153 podman[130858]: 2024-03-21 10:16:18.533466316 +0000 UTC m=+0.449515852 container init 771ad7ae4e9e1a8b2088b2db3d5ed13bcd47d81d524205dd0f71e24b9769d8a4 (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-1-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-21T10:16:18.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:18 smithi174 ceph-mon[88004]: Health check failed: Reduced data availability: 9 pgs peering (PG_AVAILABILITY) 2024-03-21T10:16:18.806 INFO:journalctl@ceph.osd.1.smithi153.stdout:Mar 21 10:16:18 smithi153 podman[130858]: 2024-03-21 10:16:18.54503534 +0000 UTC m=+0.461084849 container start 771ad7ae4e9e1a8b2088b2db3d5ed13bcd47d81d524205dd0f71e24b9769d8a4 (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-1-deactivate, org.label-schema.build-date=20240319, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-03-21T10:16:18.807 INFO:journalctl@ceph.osd.1.smithi153.stdout:Mar 21 10:16:18 smithi153 podman[130858]: 2024-03-21 10:16:18.623205085 +0000 UTC m=+0.539254647 container attach 771ad7ae4e9e1a8b2088b2db3d5ed13bcd47d81d524205dd0f71e24b9769d8a4 (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-1-deactivate, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-03-21T10:16:19.290 INFO:journalctl@ceph.osd.1.smithi153.stdout:Mar 21 10:16:18 smithi153 podman[130858]: 2024-03-21 10:16:18.982134878 +0000 UTC m=+0.898184377 container died 771ad7ae4e9e1a8b2088b2db3d5ed13bcd47d81d524205dd0f71e24b9769d8a4 (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-1-deactivate, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.schema-version=1.0) 2024-03-21T10:16:19.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:19 smithi153 ceph-mon[113733]: pgmap v105: 65 pgs: 28 peering, 2 stale+active+clean, 35 active+clean; 317 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 2 op/s 2024-03-21T10:16:19.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:19 smithi174 ceph-mon[88004]: pgmap v105: 65 pgs: 28 peering, 2 stale+active+clean, 35 active+clean; 317 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 2 op/s 2024-03-21T10:16:20.057 INFO:journalctl@ceph.osd.1.smithi153.stdout:Mar 21 10:16:19 smithi153 podman[130884]: 2024-03-21 10:16:19.785536197 +0000 UTC m=+0.786988199 container remove 771ad7ae4e9e1a8b2088b2db3d5ed13bcd47d81d524205dd0f71e24b9769d8a4 (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-1-deactivate, ceph=True, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-03-21T10:16:20.057 INFO:journalctl@ceph.osd.1.smithi153.stdout:Mar 21 10:16:19 smithi153 systemd[1]: ceph-070ed3e4-e769-11ee-95cd-87774f69a715@osd.1.service: Deactivated successfully. 2024-03-21T10:16:20.057 INFO:journalctl@ceph.osd.1.smithi153.stdout:Mar 21 10:16:19 smithi153 systemd[1]: Stopped Ceph osd.1 for 070ed3e4-e769-11ee-95cd-87774f69a715. 2024-03-21T10:16:20.057 INFO:journalctl@ceph.osd.1.smithi153.stdout:Mar 21 10:16:19 smithi153 systemd[1]: ceph-070ed3e4-e769-11ee-95cd-87774f69a715@osd.1.service: Consumed 1min 45.983s CPU time. 2024-03-21T10:16:20.555 INFO:journalctl@ceph.osd.1.smithi153.stdout:Mar 21 10:16:20 smithi153 systemd[1]: Starting Ceph osd.1 for 070ed3e4-e769-11ee-95cd-87774f69a715... 2024-03-21T10:16:20.807 INFO:journalctl@ceph.osd.1.smithi153.stdout:Mar 21 10:16:20 smithi153 podman[130985]: 2024-03-21 10:16:20.459054191 +0000 UTC m=+0.035614575 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T10:16:20.807 INFO:journalctl@ceph.osd.1.smithi153.stdout:Mar 21 10:16:20 smithi153 podman[130985]: 2024-03-21 10:16:20.569882632 +0000 UTC m=+0.146443052 container create 532a8f1bc9426f9cf3be19641317aef710d7944df3129213221890e0552b5c1d (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-1-activate, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319) 2024-03-21T10:16:21.283 INFO:journalctl@ceph.osd.1.smithi153.stdout:Mar 21 10:16:20 smithi153 podman[130985]: 2024-03-21 10:16:20.849249595 +0000 UTC m=+0.425810088 container init 532a8f1bc9426f9cf3be19641317aef710d7944df3129213221890e0552b5c1d (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-1-activate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-03-21T10:16:21.283 INFO:journalctl@ceph.osd.1.smithi153.stdout:Mar 21 10:16:20 smithi153 podman[130985]: 2024-03-21 10:16:20.859811375 +0000 UTC m=+0.436371829 container start 532a8f1bc9426f9cf3be19641317aef710d7944df3129213221890e0552b5c1d (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-1-activate, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS) 2024-03-21T10:16:21.283 INFO:journalctl@ceph.osd.1.smithi153.stdout:Mar 21 10:16:20 smithi153 podman[130985]: 2024-03-21 10:16:20.939393833 +0000 UTC m=+0.515954297 container attach 532a8f1bc9426f9cf3be19641317aef710d7944df3129213221890e0552b5c1d (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-1-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-03-21T10:16:21.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:21 smithi153 ceph-mon[113733]: pgmap v106: 65 pgs: 2 active+undersized, 2 active+undersized+degraded, 28 peering, 33 active+clean; 317 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 9/366 objects degraded (2.459%) 2024-03-21T10:16:21.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:21 smithi153 ceph-mon[113733]: Health check failed: Degraded data redundancy: 9/366 objects degraded (2.459%), 2 pgs degraded (PG_DEGRADED) 2024-03-21T10:16:21.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:21 smithi174 ceph-mon[88004]: pgmap v106: 65 pgs: 2 active+undersized, 2 active+undersized+degraded, 28 peering, 33 active+clean; 317 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 9/366 objects degraded (2.459%) 2024-03-21T10:16:21.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:21 smithi174 ceph-mon[88004]: Health check failed: Degraded data redundancy: 9/366 objects degraded (2.459%), 2 pgs degraded (PG_DEGRADED) 2024-03-21T10:16:22.058 INFO:journalctl@ceph.osd.1.smithi153.stdout:Mar 21 10:16:21 smithi153 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-1-activate[130999]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-03-21T10:16:22.058 INFO:journalctl@ceph.osd.1.smithi153.stdout:Mar 21 10:16:21 smithi153 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-1-activate[130999]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-03-21T10:16:22.058 INFO:journalctl@ceph.osd.1.smithi153.stdout:Mar 21 10:16:21 smithi153 bash[130985]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-03-21T10:16:22.059 INFO:journalctl@ceph.osd.1.smithi153.stdout:Mar 21 10:16:21 smithi153 bash[130985]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-03-21T10:16:22.059 INFO:journalctl@ceph.osd.1.smithi153.stdout:Mar 21 10:16:21 smithi153 bash[130985]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-03-21T10:16:22.059 INFO:journalctl@ceph.osd.1.smithi153.stdout:Mar 21 10:16:21 smithi153 bash[130985]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-03-21T10:16:22.059 INFO:journalctl@ceph.osd.1.smithi153.stdout:Mar 21 10:16:21 smithi153 bash[130985]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-03-21T10:16:22.059 INFO:journalctl@ceph.osd.1.smithi153.stdout:Mar 21 10:16:21 smithi153 bash[130985]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-03-21T10:16:22.059 INFO:journalctl@ceph.osd.1.smithi153.stdout:Mar 21 10:16:21 smithi153 bash[130985]: --> ceph-volume raw activate successful for osd ID: 1 2024-03-21T10:16:22.059 INFO:journalctl@ceph.osd.1.smithi153.stdout:Mar 21 10:16:21 smithi153 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-1-activate[130999]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-03-21T10:16:22.059 INFO:journalctl@ceph.osd.1.smithi153.stdout:Mar 21 10:16:21 smithi153 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-1-activate[130999]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-03-21T10:16:22.059 INFO:journalctl@ceph.osd.1.smithi153.stdout:Mar 21 10:16:21 smithi153 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-1-activate[130999]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-03-21T10:16:22.059 INFO:journalctl@ceph.osd.1.smithi153.stdout:Mar 21 10:16:21 smithi153 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-1-activate[130999]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-03-21T10:16:22.059 INFO:journalctl@ceph.osd.1.smithi153.stdout:Mar 21 10:16:21 smithi153 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-1-activate[130999]: --> ceph-volume raw activate successful for osd ID: 1 2024-03-21T10:16:22.059 INFO:journalctl@ceph.osd.1.smithi153.stdout:Mar 21 10:16:21 smithi153 podman[130985]: 2024-03-21 10:16:21.747598411 +0000 UTC m=+1.324158874 container died 532a8f1bc9426f9cf3be19641317aef710d7944df3129213221890e0552b5c1d (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-1-activate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux ) 2024-03-21T10:16:22.951 INFO:journalctl@ceph.osd.1.smithi153.stdout:Mar 21 10:16:22 smithi153 podman[131134]: 2024-03-21 10:16:22.593619855 +0000 UTC m=+0.826515367 container remove 532a8f1bc9426f9cf3be19641317aef710d7944df3129213221890e0552b5c1d (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-1-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6) 2024-03-21T10:16:23.284 INFO:journalctl@ceph.osd.1.smithi153.stdout:Mar 21 10:16:22 smithi153 podman[131197]: 2024-03-21 10:16:22.85742942 +0000 UTC m=+0.038524302 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T10:16:23.284 INFO:journalctl@ceph.osd.1.smithi153.stdout:Mar 21 10:16:23 smithi153 podman[131197]: 2024-03-21 10:16:23.002641019 +0000 UTC m=+0.183735960 container create 0d646452e9c1c131b93daee5f73a85f0a3086b9c3c5f0d9b4725770fd3516727 (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=) 2024-03-21T10:16:23.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:23 smithi174 ceph-mon[88004]: pgmap v107: 65 pgs: 16 active+undersized, 16 active+undersized+degraded, 33 active+clean; 317 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 58/366 objects degraded (15.847%) 2024-03-21T10:16:23.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:23 smithi174 ceph-mon[88004]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 9 pgs peering) 2024-03-21T10:16:23.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:23 smithi153 ceph-mon[113733]: pgmap v107: 65 pgs: 16 active+undersized, 16 active+undersized+degraded, 33 active+clean; 317 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 58/366 objects degraded (15.847%) 2024-03-21T10:16:23.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:23 smithi153 ceph-mon[113733]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 9 pgs peering) 2024-03-21T10:16:23.557 INFO:journalctl@ceph.osd.1.smithi153.stdout:Mar 21 10:16:23 smithi153 podman[131197]: 2024-03-21 10:16:23.323126554 +0000 UTC m=+0.504221505 container init 0d646452e9c1c131b93daee5f73a85f0a3086b9c3c5f0d9b4725770fd3516727 (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-1, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, CEPH_POINT_RELEASE=) 2024-03-21T10:16:23.557 INFO:journalctl@ceph.osd.1.smithi153.stdout:Mar 21 10:16:23 smithi153 podman[131197]: 2024-03-21 10:16:23.333089373 +0000 UTC m=+0.514184270 container start 0d646452e9c1c131b93daee5f73a85f0a3086b9c3c5f0d9b4725770fd3516727 (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True) 2024-03-21T10:16:23.557 INFO:journalctl@ceph.osd.1.smithi153.stdout:Mar 21 10:16:23 smithi153 ceph-osd[131216]: -- 172.21.15.153:0/2187758762 <== mon.0 v2:172.21.15.153:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x561fb4f4d680 con 0x561fb4f17000 2024-03-21T10:16:23.557 INFO:journalctl@ceph.osd.1.smithi153.stdout:Mar 21 10:16:23 smithi153 bash[131197]: 0d646452e9c1c131b93daee5f73a85f0a3086b9c3c5f0d9b4725770fd3516727 2024-03-21T10:16:23.557 INFO:journalctl@ceph.osd.1.smithi153.stdout:Mar 21 10:16:23 smithi153 systemd[1]: Started Ceph osd.1 for 070ed3e4-e769-11ee-95cd-87774f69a715. 2024-03-21T10:16:24.146 INFO:journalctl@ceph.osd.1.smithi153.stdout:Mar 21 10:16:23 smithi153 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-1[131211]: 2024-03-21T10:16:23.893+0000 7f9ebafce7c0 -1 Falling back to public interface 2024-03-21T10:16:24.781 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:24 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:16:24.781 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:24 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:16:24.781 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:24 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:16:24.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:24 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:16:24.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:24 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:16:24.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:24 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:16:25.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:25 smithi153 ceph-mon[113733]: pgmap v108: 65 pgs: 16 active+undersized, 16 active+undersized+degraded, 33 active+clean; 317 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 58/366 objects degraded (15.847%) 2024-03-21T10:16:25.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:25 smithi174 ceph-mon[88004]: pgmap v108: 65 pgs: 16 active+undersized, 16 active+undersized+degraded, 33 active+clean; 317 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 58/366 objects degraded (15.847%) 2024-03-21T10:16:28.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:28 smithi153 ceph-mon[113733]: pgmap v109: 65 pgs: 16 active+undersized, 16 active+undersized+degraded, 33 active+clean; 317 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 58/366 objects degraded (15.847%) 2024-03-21T10:16:28.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:28 smithi174 ceph-mon[88004]: pgmap v109: 65 pgs: 16 active+undersized, 16 active+undersized+degraded, 33 active+clean; 317 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 58/366 objects degraded (15.847%) 2024-03-21T10:16:29.495 INFO:journalctl@ceph.osd.1.smithi153.stdout:Mar 21 10:16:29 smithi153 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-1[131211]: 2024-03-21T10:16:29.065+0000 7f9ebafce7c0 -1 osd.1 0 read_superblock omap replica is missing. 2024-03-21T10:16:29.496 INFO:journalctl@ceph.osd.1.smithi153.stdout:Mar 21 10:16:29 smithi153 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-1[131211]: 2024-03-21T10:16:29.493+0000 7f9ebafce7c0 -1 osd.1 66 log_to_monitors true 2024-03-21T10:16:30.504 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:30 smithi174 ceph-mon[88004]: pgmap v110: 65 pgs: 16 active+undersized, 16 active+undersized+degraded, 33 active+clean; 317 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 58/366 objects degraded (15.847%) 2024-03-21T10:16:30.633 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:30 smithi174 ceph-mon[88004]: from='osd.1 [v2:172.21.15.153:6810/1529835318,v1:172.21.15.153:6811/1529835318]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-21T10:16:30.634 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:30 smithi153 ceph-mon[113733]: pgmap v110: 65 pgs: 16 active+undersized, 16 active+undersized+degraded, 33 active+clean; 317 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 58/366 objects degraded (15.847%) 2024-03-21T10:16:30.634 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:30 smithi153 ceph-mon[113733]: from='osd.1 [v2:172.21.15.153:6810/1529835318,v1:172.21.15.153:6811/1529835318]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-21T10:16:31.173 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:31.170+0000 7f882a800700 1 -- 172.21.15.153:0/1705680108 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8824068410 msgr2=0x7f88240687f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:16:31.173 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:31.170+0000 7f882a800700 1 --2- 172.21.15.153:0/1705680108 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8824068410 0x7f88240687f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f8818009910 tx=0x7f8818009c20 comp rx=0 tx=0).stop 2024-03-21T10:16:31.173 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:31.170+0000 7f882a800700 1 -- 172.21.15.153:0/1705680108 shutdown_connections 2024-03-21T10:16:31.173 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:31.170+0000 7f882a800700 1 --2- 172.21.15.153:0/1705680108 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8824068d30 0x7f882410d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:16:31.173 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:31.170+0000 7f882a800700 1 --2- 172.21.15.153:0/1705680108 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8824068410 0x7f88240687f0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:16:31.173 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:31.170+0000 7f882a800700 1 -- 172.21.15.153:0/1705680108 >> 172.21.15.153:0/1705680108 conn(0x7f8824075330 msgr2=0x7f8824075740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:16:31.173 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:31.170+0000 7f882a800700 1 -- 172.21.15.153:0/1705680108 shutdown_connections 2024-03-21T10:16:31.173 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:31.171+0000 7f882a800700 1 -- 172.21.15.153:0/1705680108 wait complete. 2024-03-21T10:16:31.174 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:31.172+0000 7f882a800700 1 Processor -- start 2024-03-21T10:16:31.175 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:31.172+0000 7f882a800700 1 -- start start 2024-03-21T10:16:31.175 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:31.173+0000 7f882a800700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8824068410 0x7f882419e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:16:31.175 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:31.173+0000 7f882a800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8824068d30 0x7f882419eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:16:31.175 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:31.173+0000 7f882a800700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f882419f180 con 0x7f8824068d30 2024-03-21T10:16:31.175 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:31.173+0000 7f882a800700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8824198630 con 0x7f8824068410 2024-03-21T10:16:31.176 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:31.173+0000 7f8828a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8824068d30 0x7f882419eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:16:31.176 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:31.173+0000 7f8828a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8824068d30 0x7f882419eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:33436/0 (socket says 172.21.15.153:33436) 2024-03-21T10:16:31.176 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:31.173+0000 7f8828a00700 1 -- 172.21.15.153:0/2691109054 learned_addr learned my addr 172.21.15.153:0/2691109054 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:16:31.176 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:31.173+0000 7f8829400700 1 --2- 172.21.15.153:0/2691109054 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8824068410 0x7f882419e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:16:31.176 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:31.174+0000 7f8828a00700 1 -- 172.21.15.153:0/2691109054 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8824068410 msgr2=0x7f882419e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:16:31.176 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:31.174+0000 7f8828a00700 1 --2- 172.21.15.153:0/2691109054 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8824068410 0x7f882419e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:16:31.176 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:31.174+0000 7f8828a00700 1 -- 172.21.15.153:0/2691109054 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88180095f0 con 0x7f8824068d30 2024-03-21T10:16:31.176 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:31.174+0000 7f8828a00700 1 --2- 172.21.15.153:0/2691109054 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8824068d30 0x7f882419eaf0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f881000dc40 tx=0x7f881000df50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:16:31.177 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:31.175+0000 7f8822000700 1 -- 172.21.15.153:0/2691109054 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8810009900 con 0x7f8824068d30 2024-03-21T10:16:31.177 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:31.175+0000 7f8822000700 1 -- 172.21.15.153:0/2691109054 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8810010460 con 0x7f8824068d30 2024-03-21T10:16:31.177 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:31.175+0000 7f8822000700 1 -- 172.21.15.153:0/2691109054 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f881000f5f0 con 0x7f8824068d30 2024-03-21T10:16:31.177 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:31.175+0000 7f882a800700 1 -- 172.21.15.153:0/2691109054 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8824198910 con 0x7f8824068d30 2024-03-21T10:16:31.177 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:31.175+0000 7f882a800700 1 -- 172.21.15.153:0/2691109054 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8824198e60 con 0x7f8824068d30 2024-03-21T10:16:31.178 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:31.177+0000 7f882a800700 1 -- 172.21.15.153:0/2691109054 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f882410a7f0 con 0x7f8824068d30 2024-03-21T10:16:31.182 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:31.180+0000 7f8822000700 1 -- 172.21.15.153:0/2691109054 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f88100105d0 con 0x7f8824068d30 2024-03-21T10:16:31.183 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:31.182+0000 7f8822000700 1 --2- 172.21.15.153:0/2691109054 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f880c06e4b0 0x7f880c070970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:16:31.184 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:31.182+0000 7f8822000700 1 -- 172.21.15.153:0/2691109054 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6654+0+0 (secure 0 0 0) 0x7f881008e950 con 0x7f8824068d30 2024-03-21T10:16:31.184 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:31.182+0000 7f8829400700 1 --2- 172.21.15.153:0/2691109054 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f880c06e4b0 0x7f880c070970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:16:31.185 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:31.183+0000 7f8829400700 1 --2- 172.21.15.153:0/2691109054 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f880c06e4b0 0x7f880c070970 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8818006010 tx=0x7f881801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:16:31.187 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:31.185+0000 7f8822000700 1 -- 172.21.15.153:0/2691109054 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f881005b650 con 0x7f8824068d30 2024-03-21T10:16:31.448 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:31.446+0000 7f882a800700 1 -- 172.21.15.153:0/2691109054 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8824199140 con 0x7f880c06e4b0 2024-03-21T10:16:31.448 INFO:journalctl@ceph.osd.1.smithi153.stdout:Mar 21 10:16:31 smithi153 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-1[131211]: 2024-03-21T10:16:31.319+0000 7f9ea5400700 -1 osd.1 66 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-21T10:16:31.449 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:31 smithi153 ceph-mon[113733]: from='osd.1 [v2:172.21.15.153:6810/1529835318,v1:172.21.15.153:6811/1529835318]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-21T10:16:31.449 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:31 smithi153 ceph-mon[113733]: osdmap e69: 6 total, 5 up, 6 in 2024-03-21T10:16:31.449 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:31 smithi153 ceph-mon[113733]: from='osd.1 [v2:172.21.15.153:6810/1529835318,v1:172.21.15.153:6811/1529835318]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-03-21T10:16:31.449 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:31 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:16:31.449 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:31 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:16:31.449 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:31 smithi153 ceph-mon[113733]: Health check update: Degraded data redundancy: 58/366 objects degraded (15.847%), 16 pgs degraded (PG_DEGRADED) 2024-03-21T10:16:31.449 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:31 smithi153 ceph-mon[113733]: pgmap v112: 65 pgs: 16 active+undersized, 16 active+undersized+degraded, 33 active+clean; 317 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s; 58/366 objects degraded (15.847%) 2024-03-21T10:16:31.449 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:31 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:16:31.449 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:31 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:16:31.450 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:31.449+0000 7f8822000700 1 -- 172.21.15.153:0/2691109054 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f8824199140 con 0x7f880c06e4b0 2024-03-21T10:16:31.453 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:31.452+0000 7f882a800700 1 -- 172.21.15.153:0/2691109054 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f880c06e4b0 msgr2=0x7f880c070970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:16:31.453 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:31.452+0000 7f882a800700 1 --2- 172.21.15.153:0/2691109054 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f880c06e4b0 0x7f880c070970 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8818006010 tx=0x7f881801a040 comp rx=0 tx=0).stop 2024-03-21T10:16:31.453 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:31.452+0000 7f882a800700 1 -- 172.21.15.153:0/2691109054 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8824068d30 msgr2=0x7f882419eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:16:31.454 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:31.452+0000 7f882a800700 1 --2- 172.21.15.153:0/2691109054 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8824068d30 0x7f882419eaf0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f881000dc40 tx=0x7f881000df50 comp rx=0 tx=0).stop 2024-03-21T10:16:31.454 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:31.452+0000 7f882a800700 1 -- 172.21.15.153:0/2691109054 shutdown_connections 2024-03-21T10:16:31.454 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:31.452+0000 7f882a800700 1 --2- 172.21.15.153:0/2691109054 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8824068410 0x7f882419e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:16:31.454 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:31.452+0000 7f882a800700 1 --2- 172.21.15.153:0/2691109054 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f880c06e4b0 0x7f880c070970 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:16:31.454 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:31.452+0000 7f882a800700 1 --2- 172.21.15.153:0/2691109054 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8824068d30 0x7f882419eaf0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:16:31.454 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:31.452+0000 7f882a800700 1 -- 172.21.15.153:0/2691109054 >> 172.21.15.153:0/2691109054 conn(0x7f8824075330 msgr2=0x7f88240fe540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:16:31.455 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:31.452+0000 7f882a800700 1 -- 172.21.15.153:0/2691109054 shutdown_connections 2024-03-21T10:16:31.455 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:31.452+0000 7f882a800700 1 -- 172.21.15.153:0/2691109054 wait complete. 2024-03-21T10:16:31.468 INFO:teuthology.orchestra.run.smithi153.stdout:true 2024-03-21T10:16:31.604 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:31.602+0000 7fadf3800700 1 -- 172.21.15.153:0/371915408 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fadec1002b0 msgr2=0x7fadec10cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:16:31.605 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:31.602+0000 7fadf3800700 1 --2- 172.21.15.153:0/371915408 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fadec1002b0 0x7fadec10cf70 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fadd8009910 tx=0x7fadd8009c20 comp rx=0 tx=0).stop 2024-03-21T10:16:31.605 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:31.602+0000 7fadf3800700 1 -- 172.21.15.153:0/371915408 shutdown_connections 2024-03-21T10:16:31.605 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:31.602+0000 7fadf3800700 1 --2- 172.21.15.153:0/371915408 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fadec1002b0 0x7fadec10cf70 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:16:31.605 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:31.602+0000 7fadf3800700 1 --2- 172.21.15.153:0/371915408 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fadec0ff900 0x7fadec0ffce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:16:31.605 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:31.602+0000 7fadf3800700 1 -- 172.21.15.153:0/371915408 >> 172.21.15.153:0/371915408 conn(0x7fadec0fb540 msgr2=0x7fadec0fd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:16:31.605 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:31.603+0000 7fadf3800700 1 -- 172.21.15.153:0/371915408 shutdown_connections 2024-03-21T10:16:31.605 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:31.603+0000 7fadf3800700 1 -- 172.21.15.153:0/371915408 wait complete. 2024-03-21T10:16:31.606 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:31.604+0000 7fadf3800700 1 Processor -- start 2024-03-21T10:16:31.606 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:31.604+0000 7fadf3800700 1 -- start start 2024-03-21T10:16:31.606 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:31.605+0000 7fadf3800700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fadec0ff900 0x7fadec19e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:16:31.606 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:31.605+0000 7fadf3800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fadec1002b0 0x7fadec19ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:16:31.606 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:31.605+0000 7fadf3800700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadec19f3b0 con 0x7fadec1002b0 2024-03-21T10:16:31.606 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:31.605+0000 7fadf3800700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadec198860 con 0x7fadec0ff900 2024-03-21T10:16:31.607 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:31.605+0000 7fadebe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fadec1002b0 0x7fadec19ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:16:31.607 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:31.605+0000 7fadebe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fadec1002b0 0x7fadec19ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:33450/0 (socket says 172.21.15.153:33450) 2024-03-21T10:16:31.608 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:31.605+0000 7fadebe00700 1 -- 172.21.15.153:0/2437574377 learned_addr learned my addr 172.21.15.153:0/2437574377 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:16:31.608 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:31.605+0000 7fadf0e00700 1 --2- 172.21.15.153:0/2437574377 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fadec0ff900 0x7fadec19e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:16:31.608 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:31.606+0000 7fadebe00700 1 -- 172.21.15.153:0/2437574377 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fadec0ff900 msgr2=0x7fadec19e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:16:31.608 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:31.606+0000 7fadebe00700 1 --2- 172.21.15.153:0/2437574377 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fadec0ff900 0x7fadec19e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:16:31.608 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:31.606+0000 7fadebe00700 1 -- 172.21.15.153:0/2437574377 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadd80095f0 con 0x7fadec1002b0 2024-03-21T10:16:31.608 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:31.606+0000 7fadf0e00700 1 --2- 172.21.15.153:0/2437574377 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fadec0ff900 0x7fadec19e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:16:31.608 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:31.606+0000 7fadebe00700 1 --2- 172.21.15.153:0/2437574377 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fadec1002b0 0x7fadec19ed20 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fadd8005b40 tx=0x7fadd80056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:16:31.609 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:31.606+0000 7fade9600700 1 -- 172.21.15.153:0/2437574377 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fadd801d070 con 0x7fadec1002b0 2024-03-21T10:16:31.609 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:31.606+0000 7fade9600700 1 -- 172.21.15.153:0/2437574377 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fadd800f460 con 0x7fadec1002b0 2024-03-21T10:16:31.609 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:31.606+0000 7fadf3800700 1 -- 172.21.15.153:0/2437574377 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadec198b40 con 0x7fadec1002b0 2024-03-21T10:16:31.609 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:31.606+0000 7fade9600700 1 -- 172.21.15.153:0/2437574377 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fadd8021640 con 0x7fadec1002b0 2024-03-21T10:16:31.609 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:31.607+0000 7fadf3800700 1 -- 172.21.15.153:0/2437574377 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadec199090 con 0x7fadec1002b0 2024-03-21T10:16:31.609 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:31.608+0000 7fadf3800700 1 -- 172.21.15.153:0/2437574377 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadec10a6e0 con 0x7fadec1002b0 2024-03-21T10:16:31.610 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:31.609+0000 7fade9600700 1 -- 172.21.15.153:0/2437574377 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fadd800f5d0 con 0x7fadec1002b0 2024-03-21T10:16:31.612 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:31.611+0000 7fade9600700 1 --2- 172.21.15.153:0/2437574377 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fadd406e460 0x7fadd4070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:16:31.612 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:31.611+0000 7fade9600700 1 -- 172.21.15.153:0/2437574377 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6654+0+0 (secure 0 0 0) 0x7fadd808fc00 con 0x7fadec1002b0 2024-03-21T10:16:31.613 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:31.611+0000 7fadf0e00700 1 --2- 172.21.15.153:0/2437574377 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fadd406e460 0x7fadd4070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:16:31.613 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:31.612+0000 7fadf0e00700 1 --2- 172.21.15.153:0/2437574377 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fadd406e460 0x7fadd4070920 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fade0009fa0 tx=0x7fade0009570 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:16:31.615 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:31.614+0000 7fade9600700 1 -- 172.21.15.153:0/2437574377 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fadd805c8c0 con 0x7fadec1002b0 2024-03-21T10:16:31.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:31 smithi174 ceph-mon[88004]: from='osd.1 [v2:172.21.15.153:6810/1529835318,v1:172.21.15.153:6811/1529835318]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-21T10:16:31.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:31 smithi174 ceph-mon[88004]: osdmap e69: 6 total, 5 up, 6 in 2024-03-21T10:16:31.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:31 smithi174 ceph-mon[88004]: from='osd.1 [v2:172.21.15.153:6810/1529835318,v1:172.21.15.153:6811/1529835318]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-03-21T10:16:31.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:31 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:16:31.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:31 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:16:31.703 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:31 smithi174 ceph-mon[88004]: Health check update: Degraded data redundancy: 58/366 objects degraded (15.847%), 16 pgs degraded (PG_DEGRADED) 2024-03-21T10:16:31.703 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:31 smithi174 ceph-mon[88004]: pgmap v112: 65 pgs: 16 active+undersized, 16 active+undersized+degraded, 33 active+clean; 317 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s; 58/366 objects degraded (15.847%) 2024-03-21T10:16:31.703 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:31 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:16:31.703 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:31 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:16:31.883 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:31.880+0000 7fadf3800700 1 -- 172.21.15.153:0/2437574377 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fadec199380 con 0x7fadd406e460 2024-03-21T10:16:31.884 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:31.883+0000 7fade9600700 1 -- 172.21.15.153:0/2437574377 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fadec199380 con 0x7fadd406e460 2024-03-21T10:16:31.887 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:31.885+0000 7fadf3800700 1 -- 172.21.15.153:0/2437574377 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fadd406e460 msgr2=0x7fadd4070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:16:31.887 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:31.885+0000 7fadf3800700 1 --2- 172.21.15.153:0/2437574377 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fadd406e460 0x7fadd4070920 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fade0009fa0 tx=0x7fade0009570 comp rx=0 tx=0).stop 2024-03-21T10:16:31.887 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:31.885+0000 7fadf3800700 1 -- 172.21.15.153:0/2437574377 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fadec1002b0 msgr2=0x7fadec19ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:16:31.887 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:31.885+0000 7fadf3800700 1 --2- 172.21.15.153:0/2437574377 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fadec1002b0 0x7fadec19ed20 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fadd8005b40 tx=0x7fadd80056a0 comp rx=0 tx=0).stop 2024-03-21T10:16:31.887 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:31.885+0000 7fadf3800700 1 -- 172.21.15.153:0/2437574377 shutdown_connections 2024-03-21T10:16:31.887 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:31.885+0000 7fadf3800700 1 --2- 172.21.15.153:0/2437574377 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fadec0ff900 0x7fadec19e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:16:31.887 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:31.885+0000 7fadf3800700 1 --2- 172.21.15.153:0/2437574377 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fadd406e460 0x7fadd4070920 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:16:31.887 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:31.885+0000 7fadf3800700 1 --2- 172.21.15.153:0/2437574377 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fadec1002b0 0x7fadec19ed20 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:16:31.887 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:31.885+0000 7fadf3800700 1 -- 172.21.15.153:0/2437574377 >> 172.21.15.153:0/2437574377 conn(0x7fadec0fb540 msgr2=0x7fadec0fcbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:16:31.888 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:31.886+0000 7fadf3800700 1 -- 172.21.15.153:0/2437574377 shutdown_connections 2024-03-21T10:16:31.888 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:31.886+0000 7fadf3800700 1 -- 172.21.15.153:0/2437574377 wait complete. 2024-03-21T10:16:32.045 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.042+0000 7f95d3800700 1 -- 172.21.15.153:0/2669999470 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f95cc100da0 msgr2=0x7f95cc101180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:16:32.045 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.042+0000 7f95d3800700 1 --2- 172.21.15.153:0/2669999470 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f95cc100da0 0x7f95cc101180 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f95c0009910 tx=0x7f95c0009c20 comp rx=0 tx=0).stop 2024-03-21T10:16:32.045 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.042+0000 7f95d3800700 1 -- 172.21.15.153:0/2669999470 shutdown_connections 2024-03-21T10:16:32.045 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.042+0000 7f95d3800700 1 --2- 172.21.15.153:0/2669999470 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f95cc101750 0x7f95cc1057a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:16:32.045 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.042+0000 7f95d3800700 1 --2- 172.21.15.153:0/2669999470 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f95cc100da0 0x7f95cc101180 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:16:32.045 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.042+0000 7f95d3800700 1 -- 172.21.15.153:0/2669999470 >> 172.21.15.153:0/2669999470 conn(0x7f95cc0fc630 msgr2=0x7f95cc0fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:16:32.045 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.043+0000 7f95d3800700 1 -- 172.21.15.153:0/2669999470 shutdown_connections 2024-03-21T10:16:32.045 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.043+0000 7f95d3800700 1 -- 172.21.15.153:0/2669999470 wait complete. 2024-03-21T10:16:32.048 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.044+0000 7f95d3800700 1 Processor -- start 2024-03-21T10:16:32.048 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.046+0000 7f95d3800700 1 -- start start 2024-03-21T10:16:32.049 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.047+0000 7f95d3800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f95cc100da0 0x7f95cc199440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:16:32.049 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.047+0000 7f95d3800700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f95cc101750 0x7f95cc1999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:16:32.049 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.047+0000 7f95d3800700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95cc19f1d0 con 0x7f95cc100da0 2024-03-21T10:16:32.049 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.047+0000 7f95d3800700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95cc19f340 con 0x7f95cc101750 2024-03-21T10:16:32.049 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.048+0000 7f95cbe00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f95cc101750 0x7f95cc1999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:16:32.049 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.048+0000 7f95cbe00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f95cc101750 0x7f95cc1999a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.153:35094/0 (socket says 172.21.15.153:35094) 2024-03-21T10:16:32.049 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.048+0000 7f95cbe00700 1 -- 172.21.15.153:0/550370047 learned_addr learned my addr 172.21.15.153:0/550370047 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:16:32.050 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.048+0000 7f95cbe00700 1 -- 172.21.15.153:0/550370047 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f95cc100da0 msgr2=0x7f95cc199440 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-21T10:16:32.050 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.048+0000 7f95cbe00700 1 --2- 172.21.15.153:0/550370047 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f95cc100da0 0x7f95cc199440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:16:32.050 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.048+0000 7f95cbe00700 1 -- 172.21.15.153:0/550370047 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95c00095f0 con 0x7f95cc101750 2024-03-21T10:16:32.050 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.049+0000 7f95cbe00700 1 --2- 172.21.15.153:0/550370047 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f95cc101750 0x7f95cc1999a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f95b800b480 tx=0x7f95b800b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:16:32.051 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.049+0000 7f95c9600700 1 -- 172.21.15.153:0/550370047 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f95b80118c0 con 0x7f95cc101750 2024-03-21T10:16:32.051 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.049+0000 7f95d3800700 1 -- 172.21.15.153:0/550370047 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95cc19f620 con 0x7f95cc101750 2024-03-21T10:16:32.052 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.049+0000 7f95d3800700 1 -- 172.21.15.153:0/550370047 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95cc19fb70 con 0x7f95cc101750 2024-03-21T10:16:32.052 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.050+0000 7f95c9600700 1 -- 172.21.15.153:0/550370047 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f95b801a400 con 0x7f95cc101750 2024-03-21T10:16:32.052 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.050+0000 7f95c9600700 1 -- 172.21.15.153:0/550370047 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f95b80195c0 con 0x7f95cc101750 2024-03-21T10:16:32.052 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.051+0000 7f95d3800700 1 -- 172.21.15.153:0/550370047 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95ac005320 con 0x7f95cc101750 2024-03-21T10:16:32.054 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.053+0000 7f95c9600700 1 -- 172.21.15.153:0/550370047 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f95b8011a20 con 0x7f95cc101750 2024-03-21T10:16:32.056 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.054+0000 7f95c9600700 1 --2- 172.21.15.153:0/550370047 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f95b406e460 0x7f95b4070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:16:32.056 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.054+0000 7f95c9600700 1 -- 172.21.15.153:0/550370047 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6654+0+0 (secure 0 0 0) 0x7f95b808e350 con 0x7f95cc101750 2024-03-21T10:16:32.056 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.054+0000 7f95d0e00700 1 --2- 172.21.15.153:0/550370047 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f95b406e460 0x7f95b4070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:16:32.056 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.055+0000 7f95d0e00700 1 --2- 172.21.15.153:0/550370047 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f95b406e460 0x7f95b4070920 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f95c0000c00 tx=0x7f95c0005fb0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:16:32.070 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.068+0000 7f95c9600700 1 -- 172.21.15.153:0/550370047 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f95b80574a0 con 0x7f95cc101750 2024-03-21T10:16:32.295 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.293+0000 7f95d3800700 1 -- 172.21.15.153:0/550370047 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f95ac000bf0 con 0x7f95b406e460 2024-03-21T10:16:32.318 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.317+0000 7f95c9600700 1 -- 172.21.15.153:0/550370047 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f95ac000bf0 con 0x7f95b406e460 2024-03-21T10:16:32.319 INFO:teuthology.orchestra.run.smithi153.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-21T10:16:32.319 INFO:teuthology.orchestra.run.smithi153.stdout:alertmanager.smithi153 smithi153 *:9093,9094 running (5m) 1s ago 19m 14.3M - 0.25.0 c8568f914cd2 9ed3f2200b5f 2024-03-21T10:16:32.319 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi153 smithi153 running (19m) 1s ago 19m 8757k - 18.2.1 5be31c24972a 5b61994396a6 2024-03-21T10:16:32.319 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi174 smithi174 running (17m) 80s ago 17m 8363k - 18.2.1 5be31c24972a 4f7a5fc3ac28 2024-03-21T10:16:32.320 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi153 smithi153 running (93s) 1s ago 19m 6656k - 19.0.0-2231-g2edfee68 237e43c7a65f 32c21104d5b0 2024-03-21T10:16:32.320 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi174 smithi174 running (85s) 80s ago 17m 6660k - 19.0.0-2231-g2edfee68 237e43c7a65f a04e48967ab6 2024-03-21T10:16:32.320 INFO:teuthology.orchestra.run.smithi153.stdout:grafana.smithi153 smithi153 *:3000 running (4m) 1s ago 18m 75.2M - 9.4.12 2bacad6d85d8 ee24026ab786 2024-03-21T10:16:32.320 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.iuyluj smithi153 running (11m) 1s ago 11m 128M - 18.2.1 5be31c24972a 02f667cee979 2024-03-21T10:16:32.320 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.zipckr smithi153 running (12m) 1s ago 12m 58.6M - 18.2.1 5be31c24972a 68a853929f8f 2024-03-21T10:16:32.320 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.nhozag smithi174 running (11m) 80s ago 11m 57.5M - 18.2.1 5be31c24972a 2fee4000308f 2024-03-21T10:16:32.320 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.yyimst smithi174 running (11m) 80s ago 11m 117M - 18.2.1 5be31c24972a d78a729220d1 2024-03-21T10:16:32.320 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi153.tluxni smithi153 *:8443,9283,8765 running (8m) 1s ago 20m 504M - 19.0.0-2231-g2edfee68 237e43c7a65f 7ffee0e55382 2024-03-21T10:16:32.320 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi174.qpcwat smithi174 *:8443,9283,8765 running (6m) 80s ago 17m 450M - 19.0.0-2231-g2edfee68 237e43c7a65f 748968f216ab 2024-03-21T10:16:32.320 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi153 smithi153 running (3m) 1s ago 21m 47.2M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f dd8038303dc3 2024-03-21T10:16:32.320 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi174 smithi174 running (2m) 80s ago 17m 36.6M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f 29d78a788f2c 2024-03-21T10:16:32.320 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi153 smithi153 *:9100 running (6m) 1s ago 19m 7894k - 1.5.0 0da6a335fe13 bb835edc0e0b 2024-03-21T10:16:32.320 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi174 smithi174 *:9100 running (6m) 80s ago 17m 7600k - 1.5.0 0da6a335fe13 7c4ecad790fa 2024-03-21T10:16:32.320 INFO:teuthology.orchestra.run.smithi153.stdout:osd.0 smithi153 running (55s) 1s ago 16m 120M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 507900fd4791 2024-03-21T10:16:32.321 INFO:teuthology.orchestra.run.smithi153.stdout:osd.1 smithi153 running (9s) 1s ago 15m 11.2M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 0d646452e9c1 2024-03-21T10:16:32.321 INFO:teuthology.orchestra.run.smithi153.stdout:osd.2 smithi153 running (15m) 1s ago 15m 185M 1217M 18.2.1 5be31c24972a 8db799f8962a 2024-03-21T10:16:32.321 INFO:teuthology.orchestra.run.smithi153.stdout:osd.3 smithi174 running (14m) 80s ago 14m 234M 2241M 18.2.1 5be31c24972a dafc89e0658d 2024-03-21T10:16:32.321 INFO:teuthology.orchestra.run.smithi153.stdout:osd.4 smithi174 running (14m) 80s ago 14m 224M 2241M 18.2.1 5be31c24972a c4cd15701f05 2024-03-21T10:16:32.321 INFO:teuthology.orchestra.run.smithi153.stdout:osd.5 smithi174 running (13m) 80s ago 13m 203M 2241M 18.2.1 5be31c24972a 3b69886ce22e 2024-03-21T10:16:32.321 INFO:teuthology.orchestra.run.smithi153.stdout:prometheus.smithi153 smithi153 *:9095 running (5m) 1s ago 18m 71.8M - 2.43.0 a07b618ecd1d 7d95561e50c2 2024-03-21T10:16:32.321 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.320+0000 7f95d3800700 1 -- 172.21.15.153:0/550370047 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f95b406e460 msgr2=0x7f95b4070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:16:32.321 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.320+0000 7f95d3800700 1 --2- 172.21.15.153:0/550370047 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f95b406e460 0x7f95b4070920 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f95c0000c00 tx=0x7f95c0005fb0 comp rx=0 tx=0).stop 2024-03-21T10:16:32.322 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.320+0000 7f95d3800700 1 -- 172.21.15.153:0/550370047 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f95cc101750 msgr2=0x7f95cc1999a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:16:32.322 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.320+0000 7f95d3800700 1 --2- 172.21.15.153:0/550370047 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f95cc101750 0x7f95cc1999a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f95b800b480 tx=0x7f95b800b840 comp rx=0 tx=0).stop 2024-03-21T10:16:32.322 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.320+0000 7f95d3800700 1 -- 172.21.15.153:0/550370047 shutdown_connections 2024-03-21T10:16:32.322 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.320+0000 7f95d3800700 1 --2- 172.21.15.153:0/550370047 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f95b406e460 0x7f95b4070920 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:16:32.322 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.320+0000 7f95d3800700 1 --2- 172.21.15.153:0/550370047 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f95cc100da0 0x7f95cc199440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:16:32.322 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.320+0000 7f95d3800700 1 --2- 172.21.15.153:0/550370047 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f95cc101750 0x7f95cc1999a0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:16:32.323 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.320+0000 7f95d3800700 1 -- 172.21.15.153:0/550370047 >> 172.21.15.153:0/550370047 conn(0x7f95cc0fc630 msgr2=0x7f95cc0fdcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:16:32.323 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.320+0000 7f95d3800700 1 -- 172.21.15.153:0/550370047 shutdown_connections 2024-03-21T10:16:32.323 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.320+0000 7f95d3800700 1 -- 172.21.15.153:0/550370047 wait complete. 2024-03-21T10:16:32.485 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.483+0000 7f0f0a200700 1 -- 172.21.15.153:0/1931810993 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0f04106970 msgr2=0x7f0f04106d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:16:32.486 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.483+0000 7f0f0a200700 1 --2- 172.21.15.153:0/1931810993 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0f04106970 0x7f0f04106d50 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0ef8009910 tx=0x7f0ef8009c20 comp rx=0 tx=0).stop 2024-03-21T10:16:32.486 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.483+0000 7f0f0a200700 1 -- 172.21.15.153:0/1931810993 shutdown_connections 2024-03-21T10:16:32.486 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.483+0000 7f0f0a200700 1 --2- 172.21.15.153:0/1931810993 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0f04100970 0x7f0f04100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:16:32.486 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.483+0000 7f0f0a200700 1 --2- 172.21.15.153:0/1931810993 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0f04106970 0x7f0f04106d50 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:16:32.486 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.483+0000 7f0f0a200700 1 -- 172.21.15.153:0/1931810993 >> 172.21.15.153:0/1931810993 conn(0x7f0f040fc440 msgr2=0x7f0f040fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:16:32.486 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.484+0000 7f0f0a200700 1 -- 172.21.15.153:0/1931810993 shutdown_connections 2024-03-21T10:16:32.486 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.484+0000 7f0f0a200700 1 -- 172.21.15.153:0/1931810993 wait complete. 2024-03-21T10:16:32.487 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.485+0000 7f0f0a200700 1 Processor -- start 2024-03-21T10:16:32.487 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.485+0000 7f0f0a200700 1 -- start start 2024-03-21T10:16:32.487 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.485+0000 7f0f0a200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0f04100970 0x7f0f04196650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:16:32.487 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.485+0000 7f0f0a200700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0f04106970 0x7f0f04196b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:16:32.487 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.485+0000 7f0f0a200700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f04197270 con 0x7f0f04100970 2024-03-21T10:16:32.487 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.485+0000 7f0f0a200700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f0419b000 con 0x7f0f04106970 2024-03-21T10:16:32.487 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.485+0000 7f0f08e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0f04100970 0x7f0f04196650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:16:32.487 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.486+0000 7f0f08e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0f04100970 0x7f0f04196650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:33470/0 (socket says 172.21.15.153:33470) 2024-03-21T10:16:32.487 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.486+0000 7f0f08e00700 1 -- 172.21.15.153:0/3261718673 learned_addr learned my addr 172.21.15.153:0/3261718673 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:16:32.488 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.486+0000 7f0f03e00700 1 --2- 172.21.15.153:0/3261718673 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0f04106970 0x7f0f04196b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:16:32.488 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.486+0000 7f0f08e00700 1 -- 172.21.15.153:0/3261718673 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0f04106970 msgr2=0x7f0f04196b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:16:32.489 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.486+0000 7f0f08e00700 1 --2- 172.21.15.153:0/3261718673 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0f04106970 0x7f0f04196b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:16:32.489 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.486+0000 7f0f08e00700 1 -- 172.21.15.153:0/3261718673 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ef80095f0 con 0x7f0f04100970 2024-03-21T10:16:32.489 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.486+0000 7f0f08e00700 1 --2- 172.21.15.153:0/3261718673 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0f04100970 0x7f0f04196650 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0ef8006010 tx=0x7f0ef800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:16:32.489 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.487+0000 7f0f01600700 1 -- 172.21.15.153:0/3261718673 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ef801d070 con 0x7f0f04100970 2024-03-21T10:16:32.489 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.487+0000 7f0f01600700 1 -- 172.21.15.153:0/3261718673 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0ef8004fb0 con 0x7f0f04100970 2024-03-21T10:16:32.489 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.487+0000 7f0f01600700 1 -- 172.21.15.153:0/3261718673 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ef8022ea0 con 0x7f0f04100970 2024-03-21T10:16:32.489 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.487+0000 7f0f0a200700 1 -- 172.21.15.153:0/3261718673 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f0419b280 con 0x7f0f04100970 2024-03-21T10:16:32.489 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.487+0000 7f0f0a200700 1 -- 172.21.15.153:0/3261718673 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f0419b770 con 0x7f0f04100970 2024-03-21T10:16:32.489 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.488+0000 7f0f0a200700 1 -- 172.21.15.153:0/3261718673 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f04108f70 con 0x7f0f04100970 2024-03-21T10:16:32.490 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.489+0000 7f0f01600700 1 -- 172.21.15.153:0/3261718673 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f0ef8022470 con 0x7f0f04100970 2024-03-21T10:16:32.492 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.490+0000 7f0f01600700 1 --2- 172.21.15.153:0/3261718673 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f0eec06e460 0x7f0eec070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:16:32.492 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.490+0000 7f0f01600700 1 -- 172.21.15.153:0/3261718673 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6654+0+0 (secure 0 0 0) 0x7f0ef808f9a0 con 0x7f0f04100970 2024-03-21T10:16:32.492 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.491+0000 7f0f03e00700 1 --2- 172.21.15.153:0/3261718673 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f0eec06e460 0x7f0eec070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:16:32.493 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.491+0000 7f0f03e00700 1 --2- 172.21.15.153:0/3261718673 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f0eec06e460 0x7f0eec070920 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0ef0006fd0 tx=0x7f0ef0008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:16:32.497 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.496+0000 7f0f01600700 1 -- 172.21.15.153:0/3261718673 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0ef8058ba0 con 0x7f0f04100970 2024-03-21T10:16:32.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:32 smithi153 ceph-mon[113733]: from='osd.1 [v2:172.21.15.153:6810/1529835318,v1:172.21.15.153:6811/1529835318]' entity='osd.1' 2024-03-21T10:16:32.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:32 smithi153 ceph-mon[113733]: from='client.34278 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:16:32.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:32 smithi153 ceph-mon[113733]: from='client.34282 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:16:32.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:32 smithi174 ceph-mon[88004]: from='osd.1 [v2:172.21.15.153:6810/1529835318,v1:172.21.15.153:6811/1529835318]' entity='osd.1' 2024-03-21T10:16:32.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:32 smithi174 ceph-mon[88004]: from='client.34278 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:16:32.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:32 smithi174 ceph-mon[88004]: from='client.34282 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:16:32.816 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.815+0000 7f0f0a200700 1 -- 172.21.15.153:0/3261718673 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f0f0404f2e0 con 0x7f0f04100970 2024-03-21T10:16:32.817 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.815+0000 7f0f01600700 1 -- 172.21.15.153:0/3261718673 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+791 (secure 0 0 0) 0x7f0ef8027330 con 0x7f0f04100970 2024-03-21T10:16:32.817 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:16:32.817 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": { 2024-03-21T10:16:32.817 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:16:32.817 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:16:32.818 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": { 2024-03-21T10:16:32.818 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:16:32.818 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:16:32.818 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": { 2024-03-21T10:16:32.818 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4, 2024-03-21T10:16:32.818 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:16:32.818 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:16:32.818 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": { 2024-03-21T10:16:32.818 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-03-21T10:16:32.818 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:16:32.818 INFO:teuthology.orchestra.run.smithi153.stdout: "overall": { 2024-03-21T10:16:32.818 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 8, 2024-03-21T10:16:32.818 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 6 2024-03-21T10:16:32.818 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-03-21T10:16:32.819 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:16:32.819 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.818+0000 7f0f0a200700 1 -- 172.21.15.153:0/3261718673 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f0eec06e460 msgr2=0x7f0eec070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:16:32.819 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.818+0000 7f0f0a200700 1 --2- 172.21.15.153:0/3261718673 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f0eec06e460 0x7f0eec070920 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0ef0006fd0 tx=0x7f0ef0008040 comp rx=0 tx=0).stop 2024-03-21T10:16:32.820 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.818+0000 7f0f0a200700 1 -- 172.21.15.153:0/3261718673 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0f04100970 msgr2=0x7f0f04196650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:16:32.820 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.818+0000 7f0f0a200700 1 --2- 172.21.15.153:0/3261718673 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0f04100970 0x7f0f04196650 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0ef8006010 tx=0x7f0ef800be20 comp rx=0 tx=0).stop 2024-03-21T10:16:32.820 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.818+0000 7f0f0a200700 1 -- 172.21.15.153:0/3261718673 shutdown_connections 2024-03-21T10:16:32.820 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.818+0000 7f0f0a200700 1 --2- 172.21.15.153:0/3261718673 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f0eec06e460 0x7f0eec070920 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:16:32.820 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.818+0000 7f0f0a200700 1 --2- 172.21.15.153:0/3261718673 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0f04100970 0x7f0f04196650 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:16:32.820 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.818+0000 7f0f0a200700 1 --2- 172.21.15.153:0/3261718673 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0f04106970 0x7f0f04196b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:16:32.820 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.818+0000 7f0f0a200700 1 -- 172.21.15.153:0/3261718673 >> 172.21.15.153:0/3261718673 conn(0x7f0f040fc440 msgr2=0x7f0f040fcec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:16:32.820 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.818+0000 7f0f0a200700 1 -- 172.21.15.153:0/3261718673 shutdown_connections 2024-03-21T10:16:32.820 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.818+0000 7f0f0a200700 1 -- 172.21.15.153:0/3261718673 wait complete. 2024-03-21T10:16:32.975 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.973+0000 7f5efda00700 1 -- 172.21.15.153:0/2471902491 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5ef8068670 msgr2=0x7f5ef8068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:16:32.975 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.973+0000 7f5efda00700 1 --2- 172.21.15.153:0/2471902491 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5ef8068670 0x7f5ef8068a50 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5ee8009910 tx=0x7f5ee8009c20 comp rx=0 tx=0).stop 2024-03-21T10:16:32.975 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.973+0000 7f5efda00700 1 -- 172.21.15.153:0/2471902491 shutdown_connections 2024-03-21T10:16:32.976 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.973+0000 7f5efda00700 1 --2- 172.21.15.153:0/2471902491 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5ef8069020 0x7f5ef8105790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:16:32.976 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.973+0000 7f5efda00700 1 --2- 172.21.15.153:0/2471902491 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5ef8068670 0x7f5ef8068a50 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:16:32.976 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.973+0000 7f5efda00700 1 -- 172.21.15.153:0/2471902491 >> 172.21.15.153:0/2471902491 conn(0x7f5ef80755a0 msgr2=0x7f5ef80759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:16:32.976 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.974+0000 7f5efda00700 1 -- 172.21.15.153:0/2471902491 shutdown_connections 2024-03-21T10:16:32.976 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.974+0000 7f5efda00700 1 -- 172.21.15.153:0/2471902491 wait complete. 2024-03-21T10:16:32.976 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.975+0000 7f5efda00700 1 Processor -- start 2024-03-21T10:16:32.976 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.975+0000 7f5efda00700 1 -- start start 2024-03-21T10:16:32.977 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.975+0000 7f5efda00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5ef8068670 0x7f5ef819e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:16:32.978 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.975+0000 7f5efda00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5ef8069020 0x7f5ef819ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:16:32.978 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.975+0000 7f5efda00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef819f2e0 con 0x7f5ef8068670 2024-03-21T10:16:32.978 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.975+0000 7f5efda00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef8198790 con 0x7f5ef8069020 2024-03-21T10:16:32.978 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.976+0000 7f5ef6a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5ef8068670 0x7f5ef819e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:16:32.978 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.976+0000 7f5ef6a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5ef8068670 0x7f5ef819e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:46136/0 (socket says 172.21.15.153:46136) 2024-03-21T10:16:32.978 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.976+0000 7f5ef6a00700 1 -- 172.21.15.153:0/1793367211 learned_addr learned my addr 172.21.15.153:0/1793367211 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:16:32.978 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.976+0000 7f5ef6000700 1 --2- 172.21.15.153:0/1793367211 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5ef8069020 0x7f5ef819ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:16:32.978 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.976+0000 7f5ef6a00700 1 -- 172.21.15.153:0/1793367211 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5ef8069020 msgr2=0x7f5ef819ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:16:32.978 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.976+0000 7f5ef6a00700 1 --2- 172.21.15.153:0/1793367211 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5ef8069020 0x7f5ef819ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:16:32.978 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.976+0000 7f5ef6a00700 1 -- 172.21.15.153:0/1793367211 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ee80095f0 con 0x7f5ef8068670 2024-03-21T10:16:32.979 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.977+0000 7f5ef6a00700 1 --2- 172.21.15.153:0/1793367211 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5ef8068670 0x7f5ef819e710 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5ee8006010 tx=0x7f5ee800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:16:32.979 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.977+0000 7f5eef400700 1 -- 172.21.15.153:0/1793367211 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5ee801d070 con 0x7f5ef8068670 2024-03-21T10:16:32.979 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.977+0000 7f5eef400700 1 -- 172.21.15.153:0/1793367211 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5ee8004fb0 con 0x7f5ef8068670 2024-03-21T10:16:32.979 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.977+0000 7f5efda00700 1 -- 172.21.15.153:0/1793367211 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ef8198a10 con 0x7f5ef8068670 2024-03-21T10:16:32.979 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.977+0000 7f5eef400700 1 -- 172.21.15.153:0/1793367211 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5ee8022ea0 con 0x7f5ef8068670 2024-03-21T10:16:32.980 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.977+0000 7f5efda00700 1 -- 172.21.15.153:0/1793367211 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ef8198f00 con 0x7f5ef8068670 2024-03-21T10:16:32.980 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.978+0000 7f5efda00700 1 -- 172.21.15.153:0/1793367211 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ef804ea90 con 0x7f5ef8068670 2024-03-21T10:16:32.981 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.980+0000 7f5eef400700 1 -- 172.21.15.153:0/1793367211 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f5ee8022470 con 0x7f5ef8068670 2024-03-21T10:16:32.982 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.980+0000 7f5eef400700 1 --2- 172.21.15.153:0/1793367211 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f5edc06e460 0x7f5edc070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:16:32.982 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.981+0000 7f5eef400700 1 -- 172.21.15.153:0/1793367211 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6654+0+0 (secure 0 0 0) 0x7f5ee808fe70 con 0x7f5ef8068670 2024-03-21T10:16:32.982 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.981+0000 7f5ef6000700 1 --2- 172.21.15.153:0/1793367211 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f5edc06e460 0x7f5edc070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:16:32.983 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.982+0000 7f5ef6000700 1 --2- 172.21.15.153:0/1793367211 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f5edc06e460 0x7f5edc070920 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5ee4005fd0 tx=0x7f5ee400a560 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:16:32.986 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:32.985+0000 7f5eef400700 1 -- 172.21.15.153:0/1793367211 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f5ee805cc40 con 0x7f5ef8068670 2024-03-21T10:16:33.248 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:33.246+0000 7f5efda00700 1 -- 172.21.15.153:0/1793367211 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f5ef8199ca0 con 0x7f5ef8068670 2024-03-21T10:16:33.248 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:33.246+0000 7f5eef400700 1 -- 172.21.15.153:0/1793367211 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 21 v21) v1 ==== 76+0+1903 (secure 0 0 0) 0x7f5ee8027070 con 0x7f5ef8068670 2024-03-21T10:16:33.248 INFO:teuthology.orchestra.run.smithi153.stdout:e21 2024-03-21T10:16:33.248 INFO:teuthology.orchestra.run.smithi153.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-21T10:16:33.248 INFO:teuthology.orchestra.run.smithi153.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:16:33.248 INFO:teuthology.orchestra.run.smithi153.stdout:legacy client fscid: 1 2024-03-21T10:16:33.248 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:16:33.249 INFO:teuthology.orchestra.run.smithi153.stdout:Filesystem 'cephfs' (1) 2024-03-21T10:16:33.249 INFO:teuthology.orchestra.run.smithi153.stdout:fs_name cephfs 2024-03-21T10:16:33.249 INFO:teuthology.orchestra.run.smithi153.stdout:epoch 21 2024-03-21T10:16:33.249 INFO:teuthology.orchestra.run.smithi153.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-21T10:16:33.249 INFO:teuthology.orchestra.run.smithi153.stdout:created 2024-03-21T10:04:27.086576+0000 2024-03-21T10:16:33.249 INFO:teuthology.orchestra.run.smithi153.stdout:modified 2024-03-21T10:13:01.455594+0000 2024-03-21T10:16:33.249 INFO:teuthology.orchestra.run.smithi153.stdout:tableserver 0 2024-03-21T10:16:33.249 INFO:teuthology.orchestra.run.smithi153.stdout:root 0 2024-03-21T10:16:33.249 INFO:teuthology.orchestra.run.smithi153.stdout:session_timeout 60 2024-03-21T10:16:33.249 INFO:teuthology.orchestra.run.smithi153.stdout:session_autoclose 300 2024-03-21T10:16:33.249 INFO:teuthology.orchestra.run.smithi153.stdout:max_file_size 1099511627776 2024-03-21T10:16:33.249 INFO:teuthology.orchestra.run.smithi153.stdout:max_xattr_size 65536 2024-03-21T10:16:33.249 INFO:teuthology.orchestra.run.smithi153.stdout:required_client_features {} 2024-03-21T10:16:33.249 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure 0 2024-03-21T10:16:33.249 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure_osd_epoch 47 2024-03-21T10:16:33.250 INFO:teuthology.orchestra.run.smithi153.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:16:33.250 INFO:teuthology.orchestra.run.smithi153.stdout:max_mds 2 2024-03-21T10:16:33.250 INFO:teuthology.orchestra.run.smithi153.stdout:in 0,1 2024-03-21T10:16:33.250 INFO:teuthology.orchestra.run.smithi153.stdout:up {0=14484,1=24273} 2024-03-21T10:16:33.250 INFO:teuthology.orchestra.run.smithi153.stdout:failed 2024-03-21T10:16:33.250 INFO:teuthology.orchestra.run.smithi153.stdout:damaged 2024-03-21T10:16:33.250 INFO:teuthology.orchestra.run.smithi153.stdout:stopped 2024-03-21T10:16:33.250 INFO:teuthology.orchestra.run.smithi153.stdout:data_pools [3] 2024-03-21T10:16:33.250 INFO:teuthology.orchestra.run.smithi153.stdout:metadata_pool 2 2024-03-21T10:16:33.250 INFO:teuthology.orchestra.run.smithi153.stdout:inline_data enabled 2024-03-21T10:16:33.251 INFO:teuthology.orchestra.run.smithi153.stdout:balancer 2024-03-21T10:16:33.251 INFO:teuthology.orchestra.run.smithi153.stdout:bal_rank_mask -1 2024-03-21T10:16:33.251 INFO:teuthology.orchestra.run.smithi153.stdout:standby_count_wanted 1 2024-03-21T10:16:33.251 INFO:teuthology.orchestra.run.smithi153.stdout:qdb_cluster leader: 0 members: 2024-03-21T10:16:33.251 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.iuyluj{0:14484} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.153:6828/866138999,v1:172.21.15.153:6829/866138999] compat {c=[1],r=[1],i=[7ff]}] 2024-03-21T10:16:33.251 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.zipckr{0:14494} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.153:6826/4033622615,v1:172.21.15.153:6827/4033622615] compat {c=[1],r=[1],i=[7ff]}] 2024-03-21T10:16:33.251 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.yyimst{1:24273} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.174:6824/1776947618,v1:172.21.15.174:6825/1776947618] compat {c=[1],r=[1],i=[7ff]}] 2024-03-21T10:16:33.251 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.nhozag{1:24285} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.174:6826/806095873,v1:172.21.15.174:6827/806095873] compat {c=[1],r=[1],i=[7ff]}] 2024-03-21T10:16:33.251 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:16:33.251 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:16:33.252 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:33.249+0000 7f5efda00700 1 -- 172.21.15.153:0/1793367211 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f5edc06e460 msgr2=0x7f5edc070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:16:33.252 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:33.249+0000 7f5efda00700 1 --2- 172.21.15.153:0/1793367211 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f5edc06e460 0x7f5edc070920 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5ee4005fd0 tx=0x7f5ee400a560 comp rx=0 tx=0).stop 2024-03-21T10:16:33.252 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:33.249+0000 7f5efda00700 1 -- 172.21.15.153:0/1793367211 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5ef8068670 msgr2=0x7f5ef819e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:16:33.253 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:33.249+0000 7f5efda00700 1 --2- 172.21.15.153:0/1793367211 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5ef8068670 0x7f5ef819e710 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5ee8006010 tx=0x7f5ee800be20 comp rx=0 tx=0).stop 2024-03-21T10:16:33.253 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:33.249+0000 7f5efda00700 1 -- 172.21.15.153:0/1793367211 shutdown_connections 2024-03-21T10:16:33.253 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:33.249+0000 7f5efda00700 1 --2- 172.21.15.153:0/1793367211 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f5edc06e460 0x7f5edc070920 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:16:33.253 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:33.249+0000 7f5efda00700 1 --2- 172.21.15.153:0/1793367211 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5ef8068670 0x7f5ef819e710 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:16:33.253 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:33.249+0000 7f5efda00700 1 --2- 172.21.15.153:0/1793367211 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5ef8069020 0x7f5ef819ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:16:33.253 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:33.249+0000 7f5efda00700 1 -- 172.21.15.153:0/1793367211 >> 172.21.15.153:0/1793367211 conn(0x7f5ef80755a0 msgr2=0x7f5ef8102760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:16:33.253 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:33.250+0000 7f5efda00700 1 -- 172.21.15.153:0/1793367211 shutdown_connections 2024-03-21T10:16:33.253 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:33.250+0000 7f5efda00700 1 -- 172.21.15.153:0/1793367211 wait complete. 2024-03-21T10:16:33.253 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 21 2024-03-21T10:16:33.419 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:33.417+0000 7fcb2ae00700 1 -- 172.21.15.153:0/893085301 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb24068d30 msgr2=0x7fcb2410d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:16:33.420 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:33.417+0000 7fcb2ae00700 1 --2- 172.21.15.153:0/893085301 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb24068d30 0x7fcb2410d240 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fcb10009910 tx=0x7fcb10009c20 comp rx=0 tx=0).stop 2024-03-21T10:16:33.420 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:33.418+0000 7fcb2ae00700 1 -- 172.21.15.153:0/893085301 shutdown_connections 2024-03-21T10:16:33.420 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:33.418+0000 7fcb2ae00700 1 --2- 172.21.15.153:0/893085301 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb24068d30 0x7fcb2410d240 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:16:33.420 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:33.418+0000 7fcb2ae00700 1 --2- 172.21.15.153:0/893085301 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcb24068410 0x7fcb240687f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:16:33.420 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:33.418+0000 7fcb2ae00700 1 -- 172.21.15.153:0/893085301 >> 172.21.15.153:0/893085301 conn(0x7fcb240755a0 msgr2=0x7fcb240759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:16:33.420 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:33.418+0000 7fcb2ae00700 1 -- 172.21.15.153:0/893085301 shutdown_connections 2024-03-21T10:16:33.420 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:33.418+0000 7fcb2ae00700 1 -- 172.21.15.153:0/893085301 wait complete. 2024-03-21T10:16:33.421 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:33.419+0000 7fcb2ae00700 1 Processor -- start 2024-03-21T10:16:33.421 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:33.419+0000 7fcb2ae00700 1 -- start start 2024-03-21T10:16:33.422 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:33.420+0000 7fcb2ae00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcb24068410 0x7fcb2419e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:16:33.422 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:33.420+0000 7fcb2ae00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb24068d30 0x7fcb2419ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:16:33.422 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:33.420+0000 7fcb2ae00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb2419f2b0 con 0x7fcb24068d30 2024-03-21T10:16:33.422 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:33.420+0000 7fcb2ae00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb24198760 con 0x7fcb24068410 2024-03-21T10:16:33.423 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:33.420+0000 7fcb23400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb24068d30 0x7fcb2419ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:16:33.423 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:33.420+0000 7fcb23400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb24068d30 0x7fcb2419ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:46144/0 (socket says 172.21.15.153:46144) 2024-03-21T10:16:33.423 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:33.420+0000 7fcb23400700 1 -- 172.21.15.153:0/2986252510 learned_addr learned my addr 172.21.15.153:0/2986252510 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:16:33.423 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:33.420+0000 7fcb23e00700 1 --2- 172.21.15.153:0/2986252510 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcb24068410 0x7fcb2419e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:16:33.423 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:33.420+0000 7fcb23400700 1 -- 172.21.15.153:0/2986252510 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcb24068410 msgr2=0x7fcb2419e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:16:33.423 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:33.420+0000 7fcb23400700 1 --2- 172.21.15.153:0/2986252510 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcb24068410 0x7fcb2419e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:16:33.423 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:33.420+0000 7fcb23400700 1 -- 172.21.15.153:0/2986252510 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb100095f0 con 0x7fcb24068d30 2024-03-21T10:16:33.423 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:33.421+0000 7fcb23400700 1 --2- 172.21.15.153:0/2986252510 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb24068d30 0x7fcb2419ec20 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fcb1000b5c0 tx=0x7fcb1000be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:16:33.424 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:33.421+0000 7fcb20c00700 1 -- 172.21.15.153:0/2986252510 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcb1001d070 con 0x7fcb24068d30 2024-03-21T10:16:33.424 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:33.421+0000 7fcb2ae00700 1 -- 172.21.15.153:0/2986252510 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb24198a40 con 0x7fcb24068d30 2024-03-21T10:16:33.424 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:33.421+0000 7fcb20c00700 1 -- 172.21.15.153:0/2986252510 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcb1000f460 con 0x7fcb24068d30 2024-03-21T10:16:33.424 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:33.421+0000 7fcb20c00700 1 -- 172.21.15.153:0/2986252510 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcb10017630 con 0x7fcb24068d30 2024-03-21T10:16:33.424 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:33.421+0000 7fcb2ae00700 1 -- 172.21.15.153:0/2986252510 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb24198f90 con 0x7fcb24068d30 2024-03-21T10:16:33.424 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:33.423+0000 7fcb2ae00700 1 -- 172.21.15.153:0/2986252510 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb2410a9b0 con 0x7fcb24068d30 2024-03-21T10:16:33.426 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:33.425+0000 7fcb20c00700 1 -- 172.21.15.153:0/2986252510 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fcb1000f5d0 con 0x7fcb24068d30 2024-03-21T10:16:33.428 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:33.426+0000 7fcb20c00700 1 --2- 172.21.15.153:0/2986252510 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fcb0806e460 0x7fcb08070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:16:33.428 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:33.426+0000 7fcb20c00700 1 -- 172.21.15.153:0/2986252510 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6654+0+0 (secure 0 0 0) 0x7fcb1008fc60 con 0x7fcb24068d30 2024-03-21T10:16:33.428 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:33.426+0000 7fcb23e00700 1 --2- 172.21.15.153:0/2986252510 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fcb0806e460 0x7fcb08070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:16:33.428 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:33.427+0000 7fcb23e00700 1 --2- 172.21.15.153:0/2986252510 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fcb0806e460 0x7fcb08070920 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fcb14005fd0 tx=0x7fcb14005dd0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:16:33.431 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:33.429+0000 7fcb20c00700 1 -- 172.21.15.153:0/2986252510 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fcb1005c8e0 con 0x7fcb24068d30 2024-03-21T10:16:33.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:33 smithi153 ceph-mon[113733]: from='client.44167 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:16:33.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:33 smithi153 ceph-mon[113733]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-21T10:16:33.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:33 smithi153 ceph-mon[113733]: osd.1 [v2:172.21.15.153:6810/1529835318,v1:172.21.15.153:6811/1529835318] boot 2024-03-21T10:16:33.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:33 smithi153 ceph-mon[113733]: osdmap e70: 6 total, 6 up, 6 in 2024-03-21T10:16:33.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:33 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T10:16:33.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:33 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:16:33.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:33 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:16:33.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:33 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/3261718673' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:16:33.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:33 smithi153 ceph-mon[113733]: pgmap v114: 65 pgs: 16 active+undersized, 16 active+undersized+degraded, 33 active+clean; 317 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s; 58/366 objects degraded (15.847%) 2024-03-21T10:16:33.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:33 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/1793367211' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:16:33.678 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:33.677+0000 7fcb2ae00700 1 -- 172.21.15.153:0/2986252510 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcb24199270 con 0x7fcb0806e460 2024-03-21T10:16:33.681 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:33.679+0000 7fcb20c00700 1 -- 172.21.15.153:0/2986252510 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fcb24199270 con 0x7fcb0806e460 2024-03-21T10:16:33.681 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:16:33.681 INFO:teuthology.orchestra.run.smithi153.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232", 2024-03-21T10:16:33.681 INFO:teuthology.orchestra.run.smithi153.stdout: "in_progress": true, 2024-03-21T10:16:33.681 INFO:teuthology.orchestra.run.smithi153.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-21T10:16:33.682 INFO:teuthology.orchestra.run.smithi153.stdout: "services_complete": [ 2024-03-21T10:16:33.682 INFO:teuthology.orchestra.run.smithi153.stdout: "crash", 2024-03-21T10:16:33.682 INFO:teuthology.orchestra.run.smithi153.stdout: "mon", 2024-03-21T10:16:33.682 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr" 2024-03-21T10:16:33.682 INFO:teuthology.orchestra.run.smithi153.stdout: ], 2024-03-21T10:16:33.682 INFO:teuthology.orchestra.run.smithi153.stdout: "progress": "8/23 daemons upgraded", 2024-03-21T10:16:33.682 INFO:teuthology.orchestra.run.smithi153.stdout: "message": "Currently upgrading osd daemons", 2024-03-21T10:16:33.682 INFO:teuthology.orchestra.run.smithi153.stdout: "is_paused": false 2024-03-21T10:16:33.682 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:16:33.684 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:33.682+0000 7fcb2ae00700 1 -- 172.21.15.153:0/2986252510 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fcb0806e460 msgr2=0x7fcb08070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:16:33.684 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:33.682+0000 7fcb2ae00700 1 --2- 172.21.15.153:0/2986252510 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fcb0806e460 0x7fcb08070920 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fcb14005fd0 tx=0x7fcb14005dd0 comp rx=0 tx=0).stop 2024-03-21T10:16:33.684 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:33.682+0000 7fcb2ae00700 1 -- 172.21.15.153:0/2986252510 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb24068d30 msgr2=0x7fcb2419ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:16:33.684 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:33.682+0000 7fcb2ae00700 1 --2- 172.21.15.153:0/2986252510 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb24068d30 0x7fcb2419ec20 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fcb1000b5c0 tx=0x7fcb1000be00 comp rx=0 tx=0).stop 2024-03-21T10:16:33.684 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:33.682+0000 7fcb2ae00700 1 -- 172.21.15.153:0/2986252510 shutdown_connections 2024-03-21T10:16:33.684 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:33.682+0000 7fcb2ae00700 1 --2- 172.21.15.153:0/2986252510 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcb24068410 0x7fcb2419e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:16:33.684 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:33.682+0000 7fcb2ae00700 1 --2- 172.21.15.153:0/2986252510 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fcb0806e460 0x7fcb08070920 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:16:33.684 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:33.682+0000 7fcb2ae00700 1 --2- 172.21.15.153:0/2986252510 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb24068d30 0x7fcb2419ec20 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:16:33.684 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:33.682+0000 7fcb2ae00700 1 -- 172.21.15.153:0/2986252510 >> 172.21.15.153:0/2986252510 conn(0x7fcb240755a0 msgr2=0x7fcb240fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:16:33.684 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:33.682+0000 7fcb2ae00700 1 -- 172.21.15.153:0/2986252510 shutdown_connections 2024-03-21T10:16:33.684 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:33.682+0000 7fcb2ae00700 1 -- 172.21.15.153:0/2986252510 wait complete. 2024-03-21T10:16:33.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:33 smithi174 ceph-mon[88004]: from='client.44167 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:16:33.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:33 smithi174 ceph-mon[88004]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-21T10:16:33.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:33 smithi174 ceph-mon[88004]: osd.1 [v2:172.21.15.153:6810/1529835318,v1:172.21.15.153:6811/1529835318] boot 2024-03-21T10:16:33.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:33 smithi174 ceph-mon[88004]: osdmap e70: 6 total, 6 up, 6 in 2024-03-21T10:16:33.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:33 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T10:16:33.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:33 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:16:33.703 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:33 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:16:33.703 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:33 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/3261718673' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:16:33.703 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:33 smithi174 ceph-mon[88004]: pgmap v114: 65 pgs: 16 active+undersized, 16 active+undersized+degraded, 33 active+clean; 317 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s; 58/366 objects degraded (15.847%) 2024-03-21T10:16:33.703 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:33 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/1793367211' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:16:33.835 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:33.833+0000 7f3fdc600700 1 -- 172.21.15.153:0/2064715419 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3fd4103980 msgr2=0x7f3fd41079d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:16:33.835 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:33.833+0000 7f3fdc600700 1 --2- 172.21.15.153:0/2064715419 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3fd4103980 0x7f3fd41079d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f3fc0009910 tx=0x7f3fc0009c20 comp rx=0 tx=0).stop 2024-03-21T10:16:33.835 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:33.833+0000 7f3fdc600700 1 -- 172.21.15.153:0/2064715419 shutdown_connections 2024-03-21T10:16:33.835 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:33.833+0000 7f3fdc600700 1 --2- 172.21.15.153:0/2064715419 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3fd4103980 0x7f3fd41079d0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:16:33.835 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:33.833+0000 7f3fdc600700 1 --2- 172.21.15.153:0/2064715419 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3fd4102fd0 0x7f3fd41033b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:16:33.835 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:33.833+0000 7f3fdc600700 1 -- 172.21.15.153:0/2064715419 >> 172.21.15.153:0/2064715419 conn(0x7f3fd40fe840 msgr2=0x7f3fd4100c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:16:33.835 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:33.833+0000 7f3fdc600700 1 -- 172.21.15.153:0/2064715419 shutdown_connections 2024-03-21T10:16:33.835 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:33.833+0000 7f3fdc600700 1 -- 172.21.15.153:0/2064715419 wait complete. 2024-03-21T10:16:33.837 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:33.834+0000 7f3fdc600700 1 Processor -- start 2024-03-21T10:16:33.837 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:33.834+0000 7f3fdc600700 1 -- start start 2024-03-21T10:16:33.837 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:33.834+0000 7f3fdc600700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3fd4102fd0 0x7f3fd4198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:16:33.837 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:33.834+0000 7f3fdc600700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3fd4103980 0x7f3fd4198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:16:33.837 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:33.835+0000 7f3fdc600700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd4199530 con 0x7f3fd4103980 2024-03-21T10:16:33.837 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:33.835+0000 7f3fdc600700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd419d340 con 0x7f3fd4102fd0 2024-03-21T10:16:33.837 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:33.835+0000 7f3fd9200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3fd4103980 0x7f3fd4198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:16:33.837 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:33.835+0000 7f3fd9200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3fd4103980 0x7f3fd4198ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:46158/0 (socket says 172.21.15.153:46158) 2024-03-21T10:16:33.838 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:33.835+0000 7f3fd9200700 1 -- 172.21.15.153:0/3991087210 learned_addr learned my addr 172.21.15.153:0/3991087210 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:16:33.838 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:33.835+0000 7f3fd9200700 1 -- 172.21.15.153:0/3991087210 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3fd4102fd0 msgr2=0x7f3fd4198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:16:33.838 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:33.835+0000 7f3fd9c00700 1 --2- 172.21.15.153:0/3991087210 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3fd4102fd0 0x7f3fd4198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:16:33.838 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:33.835+0000 7f3fd9200700 1 --2- 172.21.15.153:0/3991087210 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3fd4102fd0 0x7f3fd4198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:16:33.838 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:33.835+0000 7f3fd9200700 1 -- 172.21.15.153:0/3991087210 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fc00095f0 con 0x7f3fd4103980 2024-03-21T10:16:33.838 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:33.835+0000 7f3fd9c00700 1 --2- 172.21.15.153:0/3991087210 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3fd4102fd0 0x7f3fd4198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:16:33.838 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:33.835+0000 7f3fd9200700 1 --2- 172.21.15.153:0/3991087210 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3fd4103980 0x7f3fd4198ea0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f3fc000bda0 tx=0x7f3fc000be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:16:33.839 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:33.836+0000 7f3fd2000700 1 -- 172.21.15.153:0/3991087210 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3fc001d070 con 0x7f3fd4103980 2024-03-21T10:16:33.839 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:33.836+0000 7f3fd2000700 1 -- 172.21.15.153:0/3991087210 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3fc0004fb0 con 0x7f3fd4103980 2024-03-21T10:16:33.839 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:33.836+0000 7f3fd2000700 1 -- 172.21.15.153:0/3991087210 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3fc000f480 con 0x7f3fd4103980 2024-03-21T10:16:33.839 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:33.836+0000 7f3fdc600700 1 -- 172.21.15.153:0/3991087210 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fd419d5c0 con 0x7f3fd4103980 2024-03-21T10:16:33.839 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:33.836+0000 7f3fdc600700 1 -- 172.21.15.153:0/3991087210 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fd419dab0 con 0x7f3fd4103980 2024-03-21T10:16:33.839 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:33.837+0000 7f3fdc600700 1 -- 172.21.15.153:0/3991087210 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fd410b2e0 con 0x7f3fd4103980 2024-03-21T10:16:33.840 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:33.838+0000 7f3fd2000700 1 -- 172.21.15.153:0/3991087210 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f3fc0022870 con 0x7f3fd4103980 2024-03-21T10:16:33.841 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:33.839+0000 7f3fd2000700 1 --2- 172.21.15.153:0/3991087210 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f3fbc06e460 0x7f3fbc070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:16:33.841 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:33.839+0000 7f3fd2000700 1 -- 172.21.15.153:0/3991087210 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6654+0+0 (secure 0 0 0) 0x7f3fc008f970 con 0x7f3fd4103980 2024-03-21T10:16:33.841 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:33.839+0000 7f3fd9c00700 1 --2- 172.21.15.153:0/3991087210 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f3fbc06e460 0x7f3fbc070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:16:33.842 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:33.840+0000 7f3fd9c00700 1 --2- 172.21.15.153:0/3991087210 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f3fbc06e460 0x7f3fbc070920 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3fc8005fd0 tx=0x7f3fc8005dc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:16:33.845 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:33.844+0000 7f3fd2000700 1 -- 172.21.15.153:0/3991087210 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f3fc005c6a0 con 0x7f3fd4103980 2024-03-21T10:16:34.155 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:34.153+0000 7f3fdc600700 1 -- 172.21.15.153:0/3991087210 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f3fd404ea90 con 0x7f3fd4103980 2024-03-21T10:16:34.155 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:34.154+0000 7f3fd2000700 1 -- 172.21.15.153:0/3991087210 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+1261 (secure 0 0 0) 0x7f3fc005c230 con 0x7f3fd4103980 2024-03-21T10:16:34.156 INFO:teuthology.orchestra.run.smithi153.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data; Degraded data redundancy: 58/366 objects degraded (15.847%), 16 pgs degraded 2024-03-21T10:16:34.156 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-21T10:16:34.156 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-21T10:16:34.156 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 58/366 objects degraded (15.847%), 16 pgs degraded 2024-03-21T10:16:34.156 INFO:teuthology.orchestra.run.smithi153.stdout: pg 1.0 is active+undersized+degraded, acting [3,0] 2024-03-21T10:16:34.156 INFO:teuthology.orchestra.run.smithi153.stdout: pg 2.0 is active+undersized+degraded, acting [2,0] 2024-03-21T10:16:34.156 INFO:teuthology.orchestra.run.smithi153.stdout: pg 2.1 is active+undersized+degraded, acting [2,0] 2024-03-21T10:16:34.156 INFO:teuthology.orchestra.run.smithi153.stdout: pg 2.2 is active+undersized+degraded, acting [5,0] 2024-03-21T10:16:34.156 INFO:teuthology.orchestra.run.smithi153.stdout: pg 2.3 is active+undersized+degraded, acting [5,2] 2024-03-21T10:16:34.157 INFO:teuthology.orchestra.run.smithi153.stdout: pg 2.4 is active+undersized+degraded, acting [0,4] 2024-03-21T10:16:34.157 INFO:teuthology.orchestra.run.smithi153.stdout: pg 2.6 is active+undersized+degraded, acting [3,4] 2024-03-21T10:16:34.157 INFO:teuthology.orchestra.run.smithi153.stdout: pg 2.9 is active+undersized+degraded, acting [4,0] 2024-03-21T10:16:34.157 INFO:teuthology.orchestra.run.smithi153.stdout: pg 2.a is active+undersized+degraded, acting [4,3] 2024-03-21T10:16:34.157 INFO:teuthology.orchestra.run.smithi153.stdout: pg 2.d is active+undersized+degraded, acting [3,2] 2024-03-21T10:16:34.157 INFO:teuthology.orchestra.run.smithi153.stdout: pg 2.10 is active+undersized+degraded, acting [2,0] 2024-03-21T10:16:34.157 INFO:teuthology.orchestra.run.smithi153.stdout: pg 2.11 is active+undersized+degraded, acting [4,3] 2024-03-21T10:16:34.157 INFO:teuthology.orchestra.run.smithi153.stdout: pg 2.12 is active+undersized+degraded, acting [3,0] 2024-03-21T10:16:34.157 INFO:teuthology.orchestra.run.smithi153.stdout: pg 2.15 is active+undersized+degraded, acting [3,0] 2024-03-21T10:16:34.157 INFO:teuthology.orchestra.run.smithi153.stdout: pg 2.17 is active+undersized+degraded, acting [5,2] 2024-03-21T10:16:34.157 INFO:teuthology.orchestra.run.smithi153.stdout: pg 2.1b is active+undersized+degraded, acting [0,5] 2024-03-21T10:16:34.159 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:34.157+0000 7f3fdc600700 1 -- 172.21.15.153:0/3991087210 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f3fbc06e460 msgr2=0x7f3fbc070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:16:34.159 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:34.157+0000 7f3fdc600700 1 --2- 172.21.15.153:0/3991087210 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f3fbc06e460 0x7f3fbc070920 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3fc8005fd0 tx=0x7f3fc8005dc0 comp rx=0 tx=0).stop 2024-03-21T10:16:34.159 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:34.157+0000 7f3fdc600700 1 -- 172.21.15.153:0/3991087210 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3fd4103980 msgr2=0x7f3fd4198ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:16:34.159 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:34.157+0000 7f3fdc600700 1 --2- 172.21.15.153:0/3991087210 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3fd4103980 0x7f3fd4198ea0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f3fc000bda0 tx=0x7f3fc000be80 comp rx=0 tx=0).stop 2024-03-21T10:16:34.159 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:34.157+0000 7f3fdc600700 1 -- 172.21.15.153:0/3991087210 shutdown_connections 2024-03-21T10:16:34.159 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:34.157+0000 7f3fdc600700 1 --2- 172.21.15.153:0/3991087210 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3fd4102fd0 0x7f3fd4198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:16:34.160 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:34.157+0000 7f3fdc600700 1 --2- 172.21.15.153:0/3991087210 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f3fbc06e460 0x7f3fbc070920 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:16:34.160 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:34.157+0000 7f3fdc600700 1 --2- 172.21.15.153:0/3991087210 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3fd4103980 0x7f3fd4198ea0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:16:34.160 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:34.157+0000 7f3fdc600700 1 -- 172.21.15.153:0/3991087210 >> 172.21.15.153:0/3991087210 conn(0x7f3fd40fe840 msgr2=0x7f3fd4107240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:16:34.160 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:34.157+0000 7f3fdc600700 1 -- 172.21.15.153:0/3991087210 shutdown_connections 2024-03-21T10:16:34.160 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:16:34.157+0000 7f3fdc600700 1 -- 172.21.15.153:0/3991087210 wait complete. 2024-03-21T10:16:34.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:34 smithi174 ceph-mon[88004]: osdmap e71: 6 total, 6 up, 6 in 2024-03-21T10:16:34.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:34 smithi174 ceph-mon[88004]: from='client.34296 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:16:34.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:34 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/3991087210' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:16:34.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:34 smithi153 ceph-mon[113733]: osdmap e71: 6 total, 6 up, 6 in 2024-03-21T10:16:34.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:34 smithi153 ceph-mon[113733]: from='client.34296 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:16:34.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:34 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/3991087210' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:16:35.426 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:35 smithi153 ceph-mon[113733]: pgmap v116: 65 pgs: 14 active+undersized, 16 active+undersized+degraded, 35 active+clean; 317 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.3 KiB/s rd, 3 op/s; 58/366 objects degraded (15.847%) 2024-03-21T10:16:35.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:35 smithi174 ceph-mon[88004]: pgmap v116: 65 pgs: 14 active+undersized, 16 active+undersized+degraded, 35 active+clean; 317 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.3 KiB/s rd, 3 op/s; 58/366 objects degraded (15.847%) 2024-03-21T10:16:37.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:37 smithi153 ceph-mon[113733]: pgmap v117: 65 pgs: 65 active+clean; 317 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 2 op/s 2024-03-21T10:16:37.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:37 smithi153 ceph-mon[113733]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 58/366 objects degraded (15.847%), 16 pgs degraded) 2024-03-21T10:16:37.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:37 smithi174 ceph-mon[88004]: pgmap v117: 65 pgs: 65 active+clean; 317 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 2 op/s 2024-03-21T10:16:37.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:37 smithi174 ceph-mon[88004]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 58/366 objects degraded (15.847%), 16 pgs degraded) 2024-03-21T10:16:39.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:39 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:16:39.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:39 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:16:39.703 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:39 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:16:39.703 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:39 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:16:39.703 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:39 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:16:39.703 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:39 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:16:39.703 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:39 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:16:39.703 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:39 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:16:39.703 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:39 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:16:39.703 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:39 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-21T10:16:39.703 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:39 smithi174 ceph-mon[88004]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-21T10:16:39.703 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:39 smithi174 ceph-mon[88004]: Upgrade: osd.2 is safe to restart 2024-03-21T10:16:39.703 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:39 smithi174 ceph-mon[88004]: pgmap v118: 65 pgs: 65 active+clean; 317 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.4 KiB/s rd, 2 op/s 2024-03-21T10:16:39.760 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:39 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:16:39.760 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:39 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:16:39.760 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:39 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:16:39.760 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:39 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:16:39.760 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:39 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:16:39.761 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:39 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:16:39.761 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:39 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:16:39.761 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:39 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:16:39.761 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:39 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:16:39.761 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:39 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-21T10:16:39.761 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:39 smithi153 ceph-mon[113733]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-21T10:16:39.761 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:39 smithi153 ceph-mon[113733]: Upgrade: osd.2 is safe to restart 2024-03-21T10:16:39.761 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:39 smithi153 ceph-mon[113733]: pgmap v118: 65 pgs: 65 active+clean; 317 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.4 KiB/s rd, 2 op/s 2024-03-21T10:16:42.436 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:42 smithi153 ceph-mon[113733]: pgmap v119: 65 pgs: 65 active+clean; 317 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 3 op/s 2024-03-21T10:16:42.436 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:42 smithi153 ceph-mon[113733]: Upgrade: Updating osd.2 2024-03-21T10:16:42.436 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:42 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:16:42.437 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:42 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-21T10:16:42.437 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:42 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:16:42.437 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:42 smithi153 ceph-mon[113733]: Deploying daemon osd.2 on smithi153 2024-03-21T10:16:42.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:42 smithi174 ceph-mon[88004]: pgmap v119: 65 pgs: 65 active+clean; 317 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 3 op/s 2024-03-21T10:16:42.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:42 smithi174 ceph-mon[88004]: Upgrade: Updating osd.2 2024-03-21T10:16:42.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:42 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:16:42.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:42 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-21T10:16:42.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:42 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:16:42.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:42 smithi174 ceph-mon[88004]: Deploying daemon osd.2 on smithi153 2024-03-21T10:16:44.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:44 smithi153 ceph-mon[113733]: pgmap v120: 65 pgs: 65 active+clean; 317 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-21T10:16:44.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:44 smithi174 ceph-mon[88004]: pgmap v120: 65 pgs: 65 active+clean; 317 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-21T10:16:46.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:46 smithi153 ceph-mon[113733]: pgmap v121: 65 pgs: 65 active+clean; 317 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 3 op/s 2024-03-21T10:16:46.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:46 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:16:46.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:46 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:16:46.557 INFO:journalctl@ceph.osd.2.smithi153.stdout:Mar 21 10:16:46 smithi153 systemd[1]: Stopping Ceph osd.2 for 070ed3e4-e769-11ee-95cd-87774f69a715... 2024-03-21T10:16:46.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:46 smithi174 ceph-mon[88004]: pgmap v121: 65 pgs: 65 active+clean; 317 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 3 op/s 2024-03-21T10:16:46.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:46 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:16:46.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:46 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:16:47.057 INFO:journalctl@ceph.osd.2.smithi153.stdout:Mar 21 10:16:46 smithi153 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-2[63123]: 2024-03-21T10:16:46.555+0000 7fe234c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T10:16:47.057 INFO:journalctl@ceph.osd.2.smithi153.stdout:Mar 21 10:16:46 smithi153 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-2[63123]: 2024-03-21T10:16:46.555+0000 7fe234c00700 -1 osd.2 71 *** Got signal Terminated *** 2024-03-21T10:16:47.057 INFO:journalctl@ceph.osd.2.smithi153.stdout:Mar 21 10:16:46 smithi153 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-2[63123]: 2024-03-21T10:16:46.555+0000 7fe234c00700 -1 osd.2 71 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-21T10:16:47.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:47 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:16:47.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:47 smithi153 ceph-mon[113733]: osd.2 marked itself down and dead 2024-03-21T10:16:47.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:47 smithi153 ceph-mon[113733]: pgmap v122: 65 pgs: 65 active+clean; 317 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 3 op/s 2024-03-21T10:16:47.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:47 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:16:47.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:47 smithi174 ceph-mon[88004]: osd.2 marked itself down and dead 2024-03-21T10:16:47.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:47 smithi174 ceph-mon[88004]: pgmap v122: 65 pgs: 65 active+clean; 317 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 3 op/s 2024-03-21T10:16:48.233 INFO:journalctl@ceph.osd.2.smithi153.stdout:Mar 21 10:16:47 smithi153 podman[136470]: 2024-03-21 10:16:47.94559822 +0000 UTC m=+1.539346774 container died 8db799f8962a3bdd9ad08f3dae3d065d40a9f6f5a33423bb61c50bf40a6cf68c (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-2, GIT_BRANCH=HEAD, io.buildah.version=1.29.1, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=-18.2.1, RELEASE=HEAD, ceph=True) 2024-03-21T10:16:48.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:48 smithi153 ceph-mon[113733]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-21T10:16:48.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:48 smithi153 ceph-mon[113733]: osdmap e72: 6 total, 5 up, 6 in 2024-03-21T10:16:48.557 INFO:journalctl@ceph.osd.2.smithi153.stdout:Mar 21 10:16:48 smithi153 podman[136470]: 2024-03-21 10:16:48.373906544 +0000 UTC m=+1.967655105 container cleanup 8db799f8962a3bdd9ad08f3dae3d065d40a9f6f5a33423bb61c50bf40a6cf68c (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=HEAD, ceph=True, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=-18.2.1, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=https://github.com/ceph/ceph-container.git, io.buildah.version=1.29.1, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-03-21T10:16:48.557 INFO:journalctl@ceph.osd.2.smithi153.stdout:Mar 21 10:16:48 smithi153 bash[136470]: ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-2 2024-03-21T10:16:48.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:48 smithi174 ceph-mon[88004]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-21T10:16:48.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:48 smithi174 ceph-mon[88004]: osdmap e72: 6 total, 5 up, 6 in 2024-03-21T10:16:49.056 INFO:journalctl@ceph.osd.2.smithi153.stdout:Mar 21 10:16:48 smithi153 podman[136490]: 2024-03-21 10:16:48.575085867 +0000 UTC m=+0.619984220 container remove 8db799f8962a3bdd9ad08f3dae3d065d40a9f6f5a33423bb61c50bf40a6cf68c (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=https://github.com/ceph/ceph-container.git, CEPH_POINT_RELEASE=-18.2.1, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.29.1, RELEASE=HEAD, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-03-21T10:16:49.636 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:49 smithi153 ceph-mon[113733]: osdmap e73: 6 total, 5 up, 6 in 2024-03-21T10:16:49.637 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:49 smithi153 ceph-mon[113733]: pgmap v125: 65 pgs: 12 stale+active+clean, 53 active+clean; 317 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s 2024-03-21T10:16:49.637 INFO:journalctl@ceph.osd.2.smithi153.stdout:Mar 21 10:16:49 smithi153 podman[136580]: 2024-03-21 10:16:49.347400368 +0000 UTC m=+0.032243536 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T10:16:49.637 INFO:journalctl@ceph.osd.2.smithi153.stdout:Mar 21 10:16:49 smithi153 podman[136580]: 2024-03-21 10:16:49.476013274 +0000 UTC m=+0.160856479 container create f57a30fd860c12923fd7b41926e0b3f3447c1f1cbd0a3116b637c48edacbd9f7 (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-2-deactivate, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-03-21T10:16:49.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:49 smithi174 ceph-mon[88004]: osdmap e73: 6 total, 5 up, 6 in 2024-03-21T10:16:49.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:49 smithi174 ceph-mon[88004]: pgmap v125: 65 pgs: 12 stale+active+clean, 53 active+clean; 317 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s 2024-03-21T10:16:50.057 INFO:journalctl@ceph.osd.2.smithi153.stdout:Mar 21 10:16:49 smithi153 podman[136580]: 2024-03-21 10:16:49.762922465 +0000 UTC m=+0.447765676 container init f57a30fd860c12923fd7b41926e0b3f3447c1f1cbd0a3116b637c48edacbd9f7 (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6) 2024-03-21T10:16:50.057 INFO:journalctl@ceph.osd.2.smithi153.stdout:Mar 21 10:16:49 smithi153 podman[136580]: 2024-03-21 10:16:49.774017147 +0000 UTC m=+0.458860358 container start f57a30fd860c12923fd7b41926e0b3f3447c1f1cbd0a3116b637c48edacbd9f7 (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-2-deactivate, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-21T10:16:50.057 INFO:journalctl@ceph.osd.2.smithi153.stdout:Mar 21 10:16:49 smithi153 podman[136580]: 2024-03-21 10:16:49.853787965 +0000 UTC m=+0.538631176 container attach f57a30fd860c12923fd7b41926e0b3f3447c1f1cbd0a3116b637c48edacbd9f7 (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-2-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-21T10:16:50.557 INFO:journalctl@ceph.osd.2.smithi153.stdout:Mar 21 10:16:50 smithi153 podman[136580]: 2024-03-21 10:16:50.209212501 +0000 UTC m=+0.894055742 container died f57a30fd860c12923fd7b41926e0b3f3447c1f1cbd0a3116b637c48edacbd9f7 (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-2-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, CEPH_POINT_RELEASE=) 2024-03-21T10:16:51.295 INFO:journalctl@ceph.osd.2.smithi153.stdout:Mar 21 10:16:51 smithi153 podman[136606]: 2024-03-21 10:16:51.040320074 +0000 UTC m=+0.809784535 container remove f57a30fd860c12923fd7b41926e0b3f3447c1f1cbd0a3116b637c48edacbd9f7 (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-2-deactivate, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-03-21T10:16:51.295 INFO:journalctl@ceph.osd.2.smithi153.stdout:Mar 21 10:16:51 smithi153 systemd[1]: ceph-070ed3e4-e769-11ee-95cd-87774f69a715@osd.2.service: Deactivated successfully. 2024-03-21T10:16:51.295 INFO:journalctl@ceph.osd.2.smithi153.stdout:Mar 21 10:16:51 smithi153 systemd[1]: Stopped Ceph osd.2 for 070ed3e4-e769-11ee-95cd-87774f69a715. 2024-03-21T10:16:51.295 INFO:journalctl@ceph.osd.2.smithi153.stdout:Mar 21 10:16:51 smithi153 systemd[1]: ceph-070ed3e4-e769-11ee-95cd-87774f69a715@osd.2.service: Consumed 1min 23.695s CPU time. 2024-03-21T10:16:51.556 INFO:journalctl@ceph.osd.2.smithi153.stdout:Mar 21 10:16:51 smithi153 systemd[1]: Starting Ceph osd.2 for 070ed3e4-e769-11ee-95cd-87774f69a715... 2024-03-21T10:16:51.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:51 smithi153 ceph-mon[113733]: pgmap v126: 65 pgs: 10 active+undersized, 8 active+undersized+degraded, 2 stale+active+clean, 45 active+clean; 317 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 34/366 objects degraded (9.290%) 2024-03-21T10:16:51.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:51 smithi153 ceph-mon[113733]: Health check failed: Degraded data redundancy: 34/366 objects degraded (9.290%), 8 pgs degraded (PG_DEGRADED) 2024-03-21T10:16:51.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:51 smithi174 ceph-mon[88004]: pgmap v126: 65 pgs: 10 active+undersized, 8 active+undersized+degraded, 2 stale+active+clean, 45 active+clean; 317 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 34/366 objects degraded (9.290%) 2024-03-21T10:16:51.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:51 smithi174 ceph-mon[88004]: Health check failed: Degraded data redundancy: 34/366 objects degraded (9.290%), 8 pgs degraded (PG_DEGRADED) 2024-03-21T10:16:52.198 INFO:journalctl@ceph.osd.2.smithi153.stdout:Mar 21 10:16:51 smithi153 podman[136710]: 2024-03-21 10:16:51.758778378 +0000 UTC m=+0.045207650 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T10:16:52.198 INFO:journalctl@ceph.osd.2.smithi153.stdout:Mar 21 10:16:51 smithi153 podman[136710]: 2024-03-21 10:16:51.883057993 +0000 UTC m=+0.169487271 container create f3da751cb8c1e355535fafa2a2443a153e206520c122aae2e0b7b2b5ec642970 (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-2-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.build-date=20240319) 2024-03-21T10:16:52.198 INFO:journalctl@ceph.osd.2.smithi153.stdout:Mar 21 10:16:52 smithi153 podman[136710]: 2024-03-21 10:16:52.195947559 +0000 UTC m=+0.482376847 container init f3da751cb8c1e355535fafa2a2443a153e206520c122aae2e0b7b2b5ec642970 (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-2-activate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T10:16:52.556 INFO:journalctl@ceph.osd.2.smithi153.stdout:Mar 21 10:16:52 smithi153 podman[136710]: 2024-03-21 10:16:52.209905021 +0000 UTC m=+0.496334266 container start f3da751cb8c1e355535fafa2a2443a153e206520c122aae2e0b7b2b5ec642970 (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-2-activate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.build-date=20240319, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-03-21T10:16:52.556 INFO:journalctl@ceph.osd.2.smithi153.stdout:Mar 21 10:16:52 smithi153 podman[136710]: 2024-03-21 10:16:52.29432008 +0000 UTC m=+0.580749362 container attach f3da751cb8c1e355535fafa2a2443a153e206520c122aae2e0b7b2b5ec642970 (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-2-activate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-03-21T10:16:53.497 INFO:journalctl@ceph.osd.2.smithi153.stdout:Mar 21 10:16:53 smithi153 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-2-activate[136725]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-03-21T10:16:53.498 INFO:journalctl@ceph.osd.2.smithi153.stdout:Mar 21 10:16:53 smithi153 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-2-activate[136725]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-03-21T10:16:53.498 INFO:journalctl@ceph.osd.2.smithi153.stdout:Mar 21 10:16:53 smithi153 bash[136710]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-03-21T10:16:53.498 INFO:journalctl@ceph.osd.2.smithi153.stdout:Mar 21 10:16:53 smithi153 bash[136710]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-03-21T10:16:53.498 INFO:journalctl@ceph.osd.2.smithi153.stdout:Mar 21 10:16:53 smithi153 bash[136710]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-03-21T10:16:53.498 INFO:journalctl@ceph.osd.2.smithi153.stdout:Mar 21 10:16:53 smithi153 bash[136710]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-03-21T10:16:53.498 INFO:journalctl@ceph.osd.2.smithi153.stdout:Mar 21 10:16:53 smithi153 bash[136710]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-03-21T10:16:53.498 INFO:journalctl@ceph.osd.2.smithi153.stdout:Mar 21 10:16:53 smithi153 bash[136710]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-03-21T10:16:53.498 INFO:journalctl@ceph.osd.2.smithi153.stdout:Mar 21 10:16:53 smithi153 bash[136710]: --> ceph-volume raw activate successful for osd ID: 2 2024-03-21T10:16:53.498 INFO:journalctl@ceph.osd.2.smithi153.stdout:Mar 21 10:16:53 smithi153 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-2-activate[136725]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-03-21T10:16:53.498 INFO:journalctl@ceph.osd.2.smithi153.stdout:Mar 21 10:16:53 smithi153 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-2-activate[136725]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-03-21T10:16:53.498 INFO:journalctl@ceph.osd.2.smithi153.stdout:Mar 21 10:16:53 smithi153 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-2-activate[136725]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-03-21T10:16:53.499 INFO:journalctl@ceph.osd.2.smithi153.stdout:Mar 21 10:16:53 smithi153 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-2-activate[136725]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-03-21T10:16:53.499 INFO:journalctl@ceph.osd.2.smithi153.stdout:Mar 21 10:16:53 smithi153 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-2-activate[136725]: --> ceph-volume raw activate successful for osd ID: 2 2024-03-21T10:16:53.499 INFO:journalctl@ceph.osd.2.smithi153.stdout:Mar 21 10:16:53 smithi153 podman[136710]: 2024-03-21 10:16:53.127191892 +0000 UTC m=+1.413621193 container died f3da751cb8c1e355535fafa2a2443a153e206520c122aae2e0b7b2b5ec642970 (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-2-activate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319) 2024-03-21T10:16:54.246 INFO:journalctl@ceph.osd.2.smithi153.stdout:Mar 21 10:16:53 smithi153 podman[136864]: 2024-03-21 10:16:53.906370826 +0000 UTC m=+0.761473194 container remove f3da751cb8c1e355535fafa2a2443a153e206520c122aae2e0b7b2b5ec642970 (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-2-activate, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-03-21T10:16:54.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:54 smithi174 ceph-mon[88004]: pgmap v127: 65 pgs: 17 active+undersized, 13 active+undersized+degraded, 35 active+clean; 317 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 51/366 objects degraded (13.934%) 2024-03-21T10:16:54.521 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:54 smithi153 ceph-mon[113733]: pgmap v127: 65 pgs: 17 active+undersized, 13 active+undersized+degraded, 35 active+clean; 317 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 51/366 objects degraded (13.934%) 2024-03-21T10:16:54.522 INFO:journalctl@ceph.osd.2.smithi153.stdout:Mar 21 10:16:54 smithi153 podman[136924]: 2024-03-21 10:16:54.152944359 +0000 UTC m=+0.038394614 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T10:16:54.522 INFO:journalctl@ceph.osd.2.smithi153.stdout:Mar 21 10:16:54 smithi153 podman[136924]: 2024-03-21 10:16:54.273771143 +0000 UTC m=+0.159221388 container create 1326f2afdfcd42fc768f7dc44b6f9626fe9db845493e08c37364fb131f7054c1 (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-03-21T10:16:54.522 INFO:journalctl@ceph.osd.2.smithi153.stdout:Mar 21 10:16:54 smithi153 podman[136924]: 2024-03-21 10:16:54.519166843 +0000 UTC m=+0.404617131 container init 1326f2afdfcd42fc768f7dc44b6f9626fe9db845493e08c37364fb131f7054c1 (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_CLEAN=True) 2024-03-21T10:16:54.807 INFO:journalctl@ceph.osd.2.smithi153.stdout:Mar 21 10:16:54 smithi153 podman[136924]: 2024-03-21 10:16:54.532082263 +0000 UTC m=+0.417532490 container start 1326f2afdfcd42fc768f7dc44b6f9626fe9db845493e08c37364fb131f7054c1 (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-2, ceph=True, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-03-21T10:16:54.807 INFO:journalctl@ceph.osd.2.smithi153.stdout:Mar 21 10:16:54 smithi153 ceph-osd[136943]: -- 172.21.15.153:0/4116654699 <== mon.0 v2:172.21.15.153:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x559ad7b27680 con 0x559ad7af1000 2024-03-21T10:16:54.807 INFO:journalctl@ceph.osd.2.smithi153.stdout:Mar 21 10:16:54 smithi153 bash[136924]: 1326f2afdfcd42fc768f7dc44b6f9626fe9db845493e08c37364fb131f7054c1 2024-03-21T10:16:54.807 INFO:journalctl@ceph.osd.2.smithi153.stdout:Mar 21 10:16:54 smithi153 systemd[1]: Started Ceph osd.2 for 070ed3e4-e769-11ee-95cd-87774f69a715. 2024-03-21T10:16:55.556 INFO:journalctl@ceph.osd.2.smithi153.stdout:Mar 21 10:16:55 smithi153 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-2[136939]: 2024-03-21T10:16:55.095+0000 7fa5829157c0 -1 Falling back to public interface 2024-03-21T10:16:56.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:55 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:16:56.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:55 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:16:56.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:55 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:16:56.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:55 smithi153 ceph-mon[113733]: pgmap v128: 65 pgs: 17 active+undersized, 13 active+undersized+degraded, 35 active+clean; 317 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 51/366 objects degraded (13.934%) 2024-03-21T10:16:56.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:55 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:16:56.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:55 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:16:56.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:55 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:16:56.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:55 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:16:56.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:55 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:16:56.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:55 smithi174 ceph-mon[88004]: pgmap v128: 65 pgs: 17 active+undersized, 13 active+undersized+degraded, 35 active+clean; 317 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 51/366 objects degraded (13.934%) 2024-03-21T10:16:56.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:55 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:16:56.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:55 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:16:58.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:58 smithi153 ceph-mon[113733]: pgmap v129: 65 pgs: 17 active+undersized, 13 active+undersized+degraded, 35 active+clean; 317 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 51/366 objects degraded (13.934%) 2024-03-21T10:16:58.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:58 smithi174 ceph-mon[88004]: pgmap v129: 65 pgs: 17 active+undersized, 13 active+undersized+degraded, 35 active+clean; 317 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 51/366 objects degraded (13.934%) 2024-03-21T10:16:59.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:16:59 smithi174 ceph-mon[88004]: pgmap v130: 65 pgs: 17 active+undersized, 13 active+undersized+degraded, 35 active+clean; 317 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 51/366 objects degraded (13.934%) 2024-03-21T10:16:59.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:16:59 smithi153 ceph-mon[113733]: pgmap v130: 65 pgs: 17 active+undersized, 13 active+undersized+degraded, 35 active+clean; 317 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 51/366 objects degraded (13.934%) 2024-03-21T10:16:59.942 INFO:journalctl@ceph.osd.2.smithi153.stdout:Mar 21 10:16:59 smithi153 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-2[136939]: 2024-03-21T10:16:59.667+0000 7fa5829157c0 -1 osd.2 0 read_superblock omap replica is missing. 2024-03-21T10:17:00.306 INFO:journalctl@ceph.osd.2.smithi153.stdout:Mar 21 10:17:00 smithi153 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-2[136939]: 2024-03-21T10:17:00.056+0000 7fa5829157c0 -1 osd.2 71 log_to_monitors true 2024-03-21T10:17:00.687 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:00 smithi153 ceph-mon[113733]: from='osd.2 [v2:172.21.15.153:6818/292763575,v1:172.21.15.153:6819/292763575]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-21T10:17:00.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:00 smithi174 ceph-mon[88004]: from='osd.2 [v2:172.21.15.153:6818/292763575,v1:172.21.15.153:6819/292763575]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-21T10:17:01.686 INFO:journalctl@ceph.osd.2.smithi153.stdout:Mar 21 10:17:01 smithi153 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-2[136939]: 2024-03-21T10:17:01.422+0000 7fa56cc00700 -1 osd.2 71 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-21T10:17:01.687 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:01 smithi153 ceph-mon[113733]: from='osd.2 [v2:172.21.15.153:6818/292763575,v1:172.21.15.153:6819/292763575]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-21T10:17:01.687 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:01 smithi153 ceph-mon[113733]: osdmap e74: 6 total, 5 up, 6 in 2024-03-21T10:17:01.687 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:01 smithi153 ceph-mon[113733]: from='osd.2 [v2:172.21.15.153:6818/292763575,v1:172.21.15.153:6819/292763575]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-03-21T10:17:01.687 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:01 smithi153 ceph-mon[113733]: Health check update: Degraded data redundancy: 51/366 objects degraded (13.934%), 13 pgs degraded (PG_DEGRADED) 2024-03-21T10:17:01.687 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:01 smithi153 ceph-mon[113733]: pgmap v132: 65 pgs: 17 active+undersized, 13 active+undersized+degraded, 35 active+clean; 317 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 3 op/s; 51/366 objects degraded (13.934%) 2024-03-21T10:17:01.687 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:01 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:17:01.687 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:01 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:17:01.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:01 smithi174 ceph-mon[88004]: from='osd.2 [v2:172.21.15.153:6818/292763575,v1:172.21.15.153:6819/292763575]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-21T10:17:01.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:01 smithi174 ceph-mon[88004]: osdmap e74: 6 total, 5 up, 6 in 2024-03-21T10:17:01.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:01 smithi174 ceph-mon[88004]: from='osd.2 [v2:172.21.15.153:6818/292763575,v1:172.21.15.153:6819/292763575]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi153", "root=default"]}]: dispatch 2024-03-21T10:17:01.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:01 smithi174 ceph-mon[88004]: Health check update: Degraded data redundancy: 51/366 objects degraded (13.934%), 13 pgs degraded (PG_DEGRADED) 2024-03-21T10:17:01.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:01 smithi174 ceph-mon[88004]: pgmap v132: 65 pgs: 17 active+undersized, 13 active+undersized+degraded, 35 active+clean; 317 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 3 op/s; 51/366 objects degraded (13.934%) 2024-03-21T10:17:01.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:01 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:17:01.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:01 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:17:02.649 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:02 smithi153 ceph-mon[113733]: from='osd.2 [v2:172.21.15.153:6818/292763575,v1:172.21.15.153:6819/292763575]' entity='osd.2' 2024-03-21T10:17:02.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:02 smithi174 ceph-mon[88004]: from='osd.2 [v2:172.21.15.153:6818/292763575,v1:172.21.15.153:6819/292763575]' entity='osd.2' 2024-03-21T10:17:03.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:03 smithi153 ceph-mon[113733]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-21T10:17:03.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:03 smithi153 ceph-mon[113733]: osd.2 [v2:172.21.15.153:6818/292763575,v1:172.21.15.153:6819/292763575] boot 2024-03-21T10:17:03.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:03 smithi153 ceph-mon[113733]: osdmap e75: 6 total, 6 up, 6 in 2024-03-21T10:17:03.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:03 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T10:17:03.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:03 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:17:03.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:03 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:17:03.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:03 smithi153 ceph-mon[113733]: pgmap v134: 65 pgs: 17 active+undersized, 13 active+undersized+degraded, 35 active+clean; 317 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 4 op/s; 51/366 objects degraded (13.934%) 2024-03-21T10:17:03.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:03 smithi153 ceph-mon[113733]: osdmap e76: 6 total, 6 up, 6 in 2024-03-21T10:17:03.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:03 smithi174 ceph-mon[88004]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-21T10:17:03.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:03 smithi174 ceph-mon[88004]: osd.2 [v2:172.21.15.153:6818/292763575,v1:172.21.15.153:6819/292763575] boot 2024-03-21T10:17:03.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:03 smithi174 ceph-mon[88004]: osdmap e75: 6 total, 6 up, 6 in 2024-03-21T10:17:03.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:03 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T10:17:03.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:03 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:17:03.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:03 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:17:03.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:03 smithi174 ceph-mon[88004]: pgmap v134: 65 pgs: 17 active+undersized, 13 active+undersized+degraded, 35 active+clean; 317 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 4 op/s; 51/366 objects degraded (13.934%) 2024-03-21T10:17:03.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:03 smithi174 ceph-mon[88004]: osdmap e76: 6 total, 6 up, 6 in 2024-03-21T10:17:04.334 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:04.332+0000 7f11ece00700 1 -- 172.21.15.153:0/1873467862 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f11e8069020 msgr2=0x7f11e8105790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:17:04.334 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:04.332+0000 7f11ece00700 1 --2- 172.21.15.153:0/1873467862 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f11e8069020 0x7f11e8105790 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f11d0009910 tx=0x7f11d0009c20 comp rx=0 tx=0).stop 2024-03-21T10:17:04.334 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:04.332+0000 7f11ece00700 1 -- 172.21.15.153:0/1873467862 shutdown_connections 2024-03-21T10:17:04.334 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:04.332+0000 7f11ece00700 1 --2- 172.21.15.153:0/1873467862 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f11e8069020 0x7f11e8105790 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:17:04.334 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:04.332+0000 7f11ece00700 1 --2- 172.21.15.153:0/1873467862 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f11e8068670 0x7f11e8068a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:17:04.334 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:04.332+0000 7f11ece00700 1 -- 172.21.15.153:0/1873467862 >> 172.21.15.153:0/1873467862 conn(0x7f11e80755a0 msgr2=0x7f11e80759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:17:04.334 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:04.332+0000 7f11ece00700 1 -- 172.21.15.153:0/1873467862 shutdown_connections 2024-03-21T10:17:04.334 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:04.333+0000 7f11ece00700 1 -- 172.21.15.153:0/1873467862 wait complete. 2024-03-21T10:17:04.335 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:04.334+0000 7f11ece00700 1 Processor -- start 2024-03-21T10:17:04.336 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:04.334+0000 7f11ece00700 1 -- start start 2024-03-21T10:17:04.336 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:04.334+0000 7f11ece00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f11e8068670 0x7f11e819e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:17:04.336 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:04.334+0000 7f11ece00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f11e8069020 0x7f11e819ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:17:04.336 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:04.334+0000 7f11ece00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11e819f3f0 con 0x7f11e8069020 2024-03-21T10:17:04.336 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:04.334+0000 7f11ece00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11e81988a0 con 0x7f11e8068670 2024-03-21T10:17:04.336 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:04.335+0000 7f11dfe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f11e8069020 0x7f11e819ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:17:04.336 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:04.335+0000 7f11dfe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f11e8069020 0x7f11e819ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:52252/0 (socket says 172.21.15.153:52252) 2024-03-21T10:17:04.336 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:04.335+0000 7f11dfe00700 1 -- 172.21.15.153:0/2623653969 learned_addr learned my addr 172.21.15.153:0/2623653969 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:17:04.336 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:04.335+0000 7f11e5e00700 1 --2- 172.21.15.153:0/2623653969 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f11e8068670 0x7f11e819e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:17:04.337 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:04.335+0000 7f11dfe00700 1 -- 172.21.15.153:0/2623653969 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f11e8068670 msgr2=0x7f11e819e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:17:04.337 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:04.335+0000 7f11dfe00700 1 --2- 172.21.15.153:0/2623653969 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f11e8068670 0x7f11e819e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:17:04.337 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:04.335+0000 7f11dfe00700 1 -- 172.21.15.153:0/2623653969 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11d00095f0 con 0x7f11e8069020 2024-03-21T10:17:04.337 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:04.336+0000 7f11dfe00700 1 --2- 172.21.15.153:0/2623653969 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f11e8069020 0x7f11e819ed60 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f11d00092e0 tx=0x7f11d000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:17:04.337 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:04.336+0000 7f11dea00700 1 -- 172.21.15.153:0/2623653969 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f11d001d070 con 0x7f11e8069020 2024-03-21T10:17:04.337 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:04.336+0000 7f11dea00700 1 -- 172.21.15.153:0/2623653969 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f11d0005dc0 con 0x7f11e8069020 2024-03-21T10:17:04.337 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:04.336+0000 7f11dea00700 1 -- 172.21.15.153:0/2623653969 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f11d0017420 con 0x7f11e8069020 2024-03-21T10:17:04.337 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:04.336+0000 7f11ece00700 1 -- 172.21.15.153:0/2623653969 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11e8198b20 con 0x7f11e8069020 2024-03-21T10:17:04.338 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:04.336+0000 7f11ece00700 1 -- 172.21.15.153:0/2623653969 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11e8199010 con 0x7f11e8069020 2024-03-21T10:17:04.338 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:04.337+0000 7f11ece00700 1 -- 172.21.15.153:0/2623653969 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11e804ea90 con 0x7f11e8069020 2024-03-21T10:17:04.339 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:04.339+0000 7f11dea00700 1 -- 172.21.15.153:0/2623653969 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f11d000f7f0 con 0x7f11e8069020 2024-03-21T10:17:04.341 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:04.340+0000 7f11dea00700 1 --2- 172.21.15.153:0/2623653969 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f11cc06e460 0x7f11cc070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:17:04.341 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:04.340+0000 7f11dea00700 1 -- 172.21.15.153:0/2623653969 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6654+0+0 (secure 0 0 0) 0x7f11d008fa70 con 0x7f11e8069020 2024-03-21T10:17:04.341 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:04.340+0000 7f11e5e00700 1 --2- 172.21.15.153:0/2623653969 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f11cc06e460 0x7f11cc070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:17:04.342 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:04.341+0000 7f11e5e00700 1 --2- 172.21.15.153:0/2623653969 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f11cc06e460 0x7f11cc070920 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f11d8006fd0 tx=0x7f11d8008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:17:04.344 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:04.344+0000 7f11dea00700 1 -- 172.21.15.153:0/2623653969 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f11d005c830 con 0x7f11e8069020 2024-03-21T10:17:04.590 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:04.589+0000 7f11ece00700 1 -- 172.21.15.153:0/2623653969 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f11e8199500 con 0x7f11cc06e460 2024-03-21T10:17:04.592 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:04.591+0000 7f11dea00700 1 -- 172.21.15.153:0/2623653969 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f11e8199500 con 0x7f11cc06e460 2024-03-21T10:17:04.595 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:04.594+0000 7f11ece00700 1 -- 172.21.15.153:0/2623653969 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f11cc06e460 msgr2=0x7f11cc070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:17:04.595 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:04.594+0000 7f11ece00700 1 --2- 172.21.15.153:0/2623653969 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f11cc06e460 0x7f11cc070920 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f11d8006fd0 tx=0x7f11d8008040 comp rx=0 tx=0).stop 2024-03-21T10:17:04.595 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:04.594+0000 7f11ece00700 1 -- 172.21.15.153:0/2623653969 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f11e8069020 msgr2=0x7f11e819ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:17:04.595 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:04.594+0000 7f11ece00700 1 --2- 172.21.15.153:0/2623653969 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f11e8069020 0x7f11e819ed60 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f11d00092e0 tx=0x7f11d000be20 comp rx=0 tx=0).stop 2024-03-21T10:17:04.595 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:04.594+0000 7f11ece00700 1 -- 172.21.15.153:0/2623653969 shutdown_connections 2024-03-21T10:17:04.595 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:04.594+0000 7f11ece00700 1 --2- 172.21.15.153:0/2623653969 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f11e8068670 0x7f11e819e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:17:04.595 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:04.594+0000 7f11ece00700 1 --2- 172.21.15.153:0/2623653969 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f11cc06e460 0x7f11cc070920 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:17:04.595 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:04.594+0000 7f11ece00700 1 --2- 172.21.15.153:0/2623653969 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f11e8069020 0x7f11e819ed60 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:17:04.595 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:04.594+0000 7f11ece00700 1 -- 172.21.15.153:0/2623653969 >> 172.21.15.153:0/2623653969 conn(0x7f11e80755a0 msgr2=0x7f11e81027c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:17:04.596 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:04.594+0000 7f11ece00700 1 -- 172.21.15.153:0/2623653969 shutdown_connections 2024-03-21T10:17:04.596 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:04.594+0000 7f11ece00700 1 -- 172.21.15.153:0/2623653969 wait complete. 2024-03-21T10:17:04.611 INFO:teuthology.orchestra.run.smithi153.stdout:true 2024-03-21T10:17:04.755 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:04.753+0000 7fb91dc00700 1 -- 172.21.15.153:0/3802509117 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb918068670 msgr2=0x7fb918068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:17:04.755 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:04.753+0000 7fb91dc00700 1 --2- 172.21.15.153:0/3802509117 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb918068670 0x7fb918068a50 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb908009910 tx=0x7fb908009c20 comp rx=0 tx=0).stop 2024-03-21T10:17:04.755 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:04.754+0000 7fb91dc00700 1 -- 172.21.15.153:0/3802509117 shutdown_connections 2024-03-21T10:17:04.755 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:04.754+0000 7fb91dc00700 1 --2- 172.21.15.153:0/3802509117 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb918069020 0x7fb918105780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:17:04.755 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:04.754+0000 7fb91dc00700 1 --2- 172.21.15.153:0/3802509117 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb918068670 0x7fb918068a50 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:17:04.755 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:04.754+0000 7fb91dc00700 1 -- 172.21.15.153:0/3802509117 >> 172.21.15.153:0/3802509117 conn(0x7fb9180754f0 msgr2=0x7fb918075900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:17:04.755 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:04.754+0000 7fb91dc00700 1 -- 172.21.15.153:0/3802509117 shutdown_connections 2024-03-21T10:17:04.755 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:04.754+0000 7fb91dc00700 1 -- 172.21.15.153:0/3802509117 wait complete. 2024-03-21T10:17:04.756 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:04.755+0000 7fb91dc00700 1 Processor -- start 2024-03-21T10:17:04.756 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:04.756+0000 7fb91dc00700 1 -- start start 2024-03-21T10:17:04.757 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:04.756+0000 7fb91dc00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb918068670 0x7fb91819c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:17:04.757 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:04.756+0000 7fb91dc00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb918069020 0x7fb91819cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:17:04.757 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:04.756+0000 7fb91dc00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb91819d170 con 0x7fb918068670 2024-03-21T10:17:04.757 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:04.756+0000 7fb91dc00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb918196620 con 0x7fb918069020 2024-03-21T10:17:04.758 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:04.756+0000 7fb916a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb918068670 0x7fb91819c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:17:04.758 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:04.756+0000 7fb916a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb918068670 0x7fb91819c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:52256/0 (socket says 172.21.15.153:52256) 2024-03-21T10:17:04.758 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:04.756+0000 7fb916a00700 1 -- 172.21.15.153:0/881866629 learned_addr learned my addr 172.21.15.153:0/881866629 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:17:04.758 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:04.757+0000 7fb916000700 1 --2- 172.21.15.153:0/881866629 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb918069020 0x7fb91819cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:17:04.758 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:04.757+0000 7fb916a00700 1 -- 172.21.15.153:0/881866629 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb918069020 msgr2=0x7fb91819cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:17:04.758 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:04.757+0000 7fb916a00700 1 --2- 172.21.15.153:0/881866629 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb918069020 0x7fb91819cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:17:04.758 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:04.757+0000 7fb916a00700 1 -- 172.21.15.153:0/881866629 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9080095f0 con 0x7fb918068670 2024-03-21T10:17:04.758 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:04.757+0000 7fb916a00700 1 --2- 172.21.15.153:0/881866629 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb918068670 0x7fb91819c5a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fb90800b5c0 tx=0x7fb90800be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:17:04.759 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:04.758+0000 7fb90f400700 1 -- 172.21.15.153:0/881866629 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb90801d070 con 0x7fb918068670 2024-03-21T10:17:04.759 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:04.758+0000 7fb90f400700 1 -- 172.21.15.153:0/881866629 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb90800f460 con 0x7fb918068670 2024-03-21T10:17:04.759 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:04.758+0000 7fb91dc00700 1 -- 172.21.15.153:0/881866629 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9181968a0 con 0x7fb918068670 2024-03-21T10:17:04.759 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:04.758+0000 7fb90f400700 1 -- 172.21.15.153:0/881866629 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb908017610 con 0x7fb918068670 2024-03-21T10:17:04.759 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:04.758+0000 7fb91dc00700 1 -- 172.21.15.153:0/881866629 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb918196d90 con 0x7fb918068670 2024-03-21T10:17:04.759 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:04.759+0000 7fb91dc00700 1 -- 172.21.15.153:0/881866629 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb918109120 con 0x7fb918068670 2024-03-21T10:17:04.760 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:04.760+0000 7fb90f400700 1 -- 172.21.15.153:0/881866629 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fb90800f5d0 con 0x7fb918068670 2024-03-21T10:17:04.761 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:04.761+0000 7fb90f400700 1 --2- 172.21.15.153:0/881866629 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb8fc06e460 0x7fb8fc070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:17:04.761 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:04.761+0000 7fb90f400700 1 -- 172.21.15.153:0/881866629 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6654+0+0 (secure 0 0 0) 0x7fb90808fa60 con 0x7fb918068670 2024-03-21T10:17:04.762 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:04.761+0000 7fb916000700 1 --2- 172.21.15.153:0/881866629 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb8fc06e460 0x7fb8fc070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:17:04.762 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:04.762+0000 7fb916000700 1 --2- 172.21.15.153:0/881866629 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb8fc06e460 0x7fb8fc070920 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb904006fd0 tx=0x7fb904008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:17:04.766 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:04.766+0000 7fb90f400700 1 -- 172.21.15.153:0/881866629 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb90805c6e0 con 0x7fb918068670 2024-03-21T10:17:05.012 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:05.011+0000 7fb91dc00700 1 -- 172.21.15.153:0/881866629 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb9181970c0 con 0x7fb8fc06e460 2024-03-21T10:17:05.013 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:05.013+0000 7fb90f400700 1 -- 172.21.15.153:0/881866629 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fb9181970c0 con 0x7fb8fc06e460 2024-03-21T10:17:05.016 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:05.015+0000 7fb91dc00700 1 -- 172.21.15.153:0/881866629 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb8fc06e460 msgr2=0x7fb8fc070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:17:05.016 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:05.015+0000 7fb91dc00700 1 --2- 172.21.15.153:0/881866629 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb8fc06e460 0x7fb8fc070920 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb904006fd0 tx=0x7fb904008040 comp rx=0 tx=0).stop 2024-03-21T10:17:05.016 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:05.015+0000 7fb91dc00700 1 -- 172.21.15.153:0/881866629 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb918068670 msgr2=0x7fb91819c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:17:05.016 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:05.015+0000 7fb91dc00700 1 --2- 172.21.15.153:0/881866629 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb918068670 0x7fb91819c5a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fb90800b5c0 tx=0x7fb90800be00 comp rx=0 tx=0).stop 2024-03-21T10:17:05.016 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:05.015+0000 7fb91dc00700 1 -- 172.21.15.153:0/881866629 shutdown_connections 2024-03-21T10:17:05.016 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:05.015+0000 7fb91dc00700 1 --2- 172.21.15.153:0/881866629 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb8fc06e460 0x7fb8fc070920 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:17:05.016 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:05.015+0000 7fb91dc00700 1 --2- 172.21.15.153:0/881866629 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb918068670 0x7fb91819c5a0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:17:05.017 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:05.015+0000 7fb91dc00700 1 --2- 172.21.15.153:0/881866629 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb918069020 0x7fb91819cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:17:05.017 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:05.015+0000 7fb91dc00700 1 -- 172.21.15.153:0/881866629 >> 172.21.15.153:0/881866629 conn(0x7fb9180754f0 msgr2=0x7fb9180fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:17:05.017 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:05.015+0000 7fb91dc00700 1 -- 172.21.15.153:0/881866629 shutdown_connections 2024-03-21T10:17:05.017 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:05.016+0000 7fb91dc00700 1 -- 172.21.15.153:0/881866629 wait complete. 2024-03-21T10:17:05.172 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:05.171+0000 7fabe5000700 1 -- 172.21.15.153:0/3058184288 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fabe0073ae0 msgr2=0x7fabe010ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:17:05.172 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:05.171+0000 7fabe5000700 1 --2- 172.21.15.153:0/3058184288 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fabe0073ae0 0x7fabe010ce40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fabcc009910 tx=0x7fabcc009c20 comp rx=0 tx=0).stop 2024-03-21T10:17:05.172 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:05.171+0000 7fabe5000700 1 -- 172.21.15.153:0/3058184288 shutdown_connections 2024-03-21T10:17:05.172 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:05.171+0000 7fabe5000700 1 --2- 172.21.15.153:0/3058184288 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fabe0073ae0 0x7fabe010ce40 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:17:05.172 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:05.171+0000 7fabe5000700 1 --2- 172.21.15.153:0/3058184288 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fabe00731c0 0x7fabe00735a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:17:05.172 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:05.171+0000 7fabe5000700 1 -- 172.21.15.153:0/3058184288 >> 172.21.15.153:0/3058184288 conn(0x7fabe00fc630 msgr2=0x7fabe00fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:17:05.172 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:05.171+0000 7fabe5000700 1 -- 172.21.15.153:0/3058184288 shutdown_connections 2024-03-21T10:17:05.172 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:05.171+0000 7fabe5000700 1 -- 172.21.15.153:0/3058184288 wait complete. 2024-03-21T10:17:05.173 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:05.172+0000 7fabe5000700 1 Processor -- start 2024-03-21T10:17:05.173 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:05.173+0000 7fabe5000700 1 -- start start 2024-03-21T10:17:05.174 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:05.173+0000 7fabe5000700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fabe00731c0 0x7fabe0198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:17:05.174 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:05.173+0000 7fabe5000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fabe0073ae0 0x7fabe0198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:17:05.174 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:05.173+0000 7fabe5000700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabe01994a0 con 0x7fabe0073ae0 2024-03-21T10:17:05.174 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:05.173+0000 7fabe5000700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabe0199610 con 0x7fabe00731c0 2024-03-21T10:17:05.175 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:05.174+0000 7fabdd400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fabe0073ae0 0x7fabe0198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:17:05.175 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:05.174+0000 7fabdd400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fabe0073ae0 0x7fabe0198ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:52272/0 (socket says 172.21.15.153:52272) 2024-03-21T10:17:05.175 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:05.174+0000 7fabdd400700 1 -- 172.21.15.153:0/3768442195 learned_addr learned my addr 172.21.15.153:0/3768442195 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:17:05.175 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:05.174+0000 7fabdde00700 1 --2- 172.21.15.153:0/3768442195 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fabe00731c0 0x7fabe0198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:17:05.175 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:05.174+0000 7fabdd400700 1 -- 172.21.15.153:0/3768442195 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fabe00731c0 msgr2=0x7fabe0198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:17:05.175 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:05.174+0000 7fabdd400700 1 --2- 172.21.15.153:0/3768442195 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fabe00731c0 0x7fabe0198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:17:05.175 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:05.174+0000 7fabdd400700 1 -- 172.21.15.153:0/3768442195 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabcc0095f0 con 0x7fabe0073ae0 2024-03-21T10:17:05.175 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:05.174+0000 7fabdd400700 1 --2- 172.21.15.153:0/3768442195 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fabe0073ae0 0x7fabe0198ea0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fabcc0092e0 tx=0x7fabcc00be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:17:05.176 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:05.175+0000 7fabd6a00700 1 -- 172.21.15.153:0/3768442195 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fabcc01d070 con 0x7fabe0073ae0 2024-03-21T10:17:05.176 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:05.175+0000 7fabd6a00700 1 -- 172.21.15.153:0/3768442195 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fabcc00f460 con 0x7fabe0073ae0 2024-03-21T10:17:05.176 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:05.175+0000 7fabe5000700 1 -- 172.21.15.153:0/3768442195 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabe019d430 con 0x7fabe0073ae0 2024-03-21T10:17:05.176 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:05.175+0000 7fabd6a00700 1 -- 172.21.15.153:0/3768442195 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fabcc017630 con 0x7fabe0073ae0 2024-03-21T10:17:05.176 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:05.175+0000 7fabe5000700 1 -- 172.21.15.153:0/3768442195 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabe019d920 con 0x7fabe0073ae0 2024-03-21T10:17:05.176 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:05.176+0000 7fabe5000700 1 -- 172.21.15.153:0/3768442195 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabe004ea90 con 0x7fabe0073ae0 2024-03-21T10:17:05.177 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:05.177+0000 7fabd6a00700 1 -- 172.21.15.153:0/3768442195 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fabcc00f5d0 con 0x7fabe0073ae0 2024-03-21T10:17:05.179 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:05.178+0000 7fabd6a00700 1 --2- 172.21.15.153:0/3768442195 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fabc406e460 0x7fabc4070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:17:05.179 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:05.178+0000 7fabd6a00700 1 -- 172.21.15.153:0/3768442195 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6654+0+0 (secure 0 0 0) 0x7fabcc08fa20 con 0x7fabe0073ae0 2024-03-21T10:17:05.179 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:05.179+0000 7fabdde00700 1 --2- 172.21.15.153:0/3768442195 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fabc406e460 0x7fabc4070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:17:05.180 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:05.179+0000 7fabdde00700 1 --2- 172.21.15.153:0/3768442195 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fabc406e460 0x7fabc4070920 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fabd0005ea0 tx=0x7fabd0005e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:17:05.184 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:05.183+0000 7fabd6a00700 1 -- 172.21.15.153:0/3768442195 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fabcc05c6c0 con 0x7fabe0073ae0 2024-03-21T10:17:05.412 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:05.412+0000 7fabe5000700 1 -- 172.21.15.153:0/3768442195 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fabe010a1e0 con 0x7fabc406e460 2024-03-21T10:17:05.445 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:05.444+0000 7fabd6a00700 1 -- 172.21.15.153:0/3768442195 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7fabe010a1e0 con 0x7fabc406e460 2024-03-21T10:17:05.446 INFO:teuthology.orchestra.run.smithi153.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-21T10:17:05.446 INFO:teuthology.orchestra.run.smithi153.stdout:alertmanager.smithi153 smithi153 *:9093,9094 running (5m) 2s ago 19m 14.5M - 0.25.0 c8568f914cd2 9ed3f2200b5f 2024-03-21T10:17:05.446 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi153 smithi153 running (20m) 2s ago 19m 8988k - 18.2.1 5be31c24972a 5b61994396a6 2024-03-21T10:17:05.446 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi174 smithi174 running (18m) 113s ago 18m 8363k - 18.2.1 5be31c24972a 4f7a5fc3ac28 2024-03-21T10:17:05.446 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi153 smithi153 running (2m) 2s ago 19m 6656k - 19.0.0-2231-g2edfee68 237e43c7a65f 32c21104d5b0 2024-03-21T10:17:05.446 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi174 smithi174 running (118s) 113s ago 18m 6660k - 19.0.0-2231-g2edfee68 237e43c7a65f a04e48967ab6 2024-03-21T10:17:05.446 INFO:teuthology.orchestra.run.smithi153.stdout:grafana.smithi153 smithi153 *:3000 running (5m) 2s ago 19m 75.4M - 9.4.12 2bacad6d85d8 ee24026ab786 2024-03-21T10:17:05.446 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.iuyluj smithi153 running (12m) 2s ago 12m 128M - 18.2.1 5be31c24972a 02f667cee979 2024-03-21T10:17:05.446 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.zipckr smithi153 running (12m) 2s ago 12m 58.8M - 18.2.1 5be31c24972a 68a853929f8f 2024-03-21T10:17:05.446 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.nhozag smithi174 running (12m) 113s ago 12m 57.5M - 18.2.1 5be31c24972a 2fee4000308f 2024-03-21T10:17:05.446 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.yyimst smithi174 running (12m) 113s ago 12m 117M - 18.2.1 5be31c24972a d78a729220d1 2024-03-21T10:17:05.446 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi153.tluxni smithi153 *:8443,9283,8765 running (8m) 2s ago 21m 505M - 19.0.0-2231-g2edfee68 237e43c7a65f 7ffee0e55382 2024-03-21T10:17:05.447 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi174.qpcwat smithi174 *:8443,9283,8765 running (7m) 113s ago 18m 450M - 19.0.0-2231-g2edfee68 237e43c7a65f 748968f216ab 2024-03-21T10:17:05.447 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi153 smithi153 running (3m) 2s ago 21m 48.2M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f dd8038303dc3 2024-03-21T10:17:05.447 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi174 smithi174 running (3m) 113s ago 18m 36.6M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f 29d78a788f2c 2024-03-21T10:17:05.447 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi153 smithi153 *:9100 running (7m) 2s ago 19m 8284k - 1.5.0 0da6a335fe13 bb835edc0e0b 2024-03-21T10:17:05.447 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi174 smithi174 *:9100 running (6m) 113s ago 18m 7600k - 1.5.0 0da6a335fe13 7c4ecad790fa 2024-03-21T10:17:05.447 INFO:teuthology.orchestra.run.smithi153.stdout:osd.0 smithi153 running (88s) 2s ago 17m 119M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 507900fd4791 2024-03-21T10:17:05.447 INFO:teuthology.orchestra.run.smithi153.stdout:osd.1 smithi153 running (42s) 2s ago 16m 86.6M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 0d646452e9c1 2024-03-21T10:17:05.447 INFO:teuthology.orchestra.run.smithi153.stdout:osd.2 smithi153 running (11s) 2s ago 15m 12.6M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 1326f2afdfcd 2024-03-21T10:17:05.447 INFO:teuthology.orchestra.run.smithi153.stdout:osd.3 smithi174 running (15m) 113s ago 15m 234M 2241M 18.2.1 5be31c24972a dafc89e0658d 2024-03-21T10:17:05.447 INFO:teuthology.orchestra.run.smithi153.stdout:osd.4 smithi174 running (14m) 113s ago 14m 224M 2241M 18.2.1 5be31c24972a c4cd15701f05 2024-03-21T10:17:05.447 INFO:teuthology.orchestra.run.smithi153.stdout:osd.5 smithi174 running (14m) 113s ago 14m 203M 2241M 18.2.1 5be31c24972a 3b69886ce22e 2024-03-21T10:17:05.447 INFO:teuthology.orchestra.run.smithi153.stdout:prometheus.smithi153 smithi153 *:9095 running (6m) 2s ago 18m 72.2M - 2.43.0 a07b618ecd1d 7d95561e50c2 2024-03-21T10:17:05.448 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:05.447+0000 7fabe5000700 1 -- 172.21.15.153:0/3768442195 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fabc406e460 msgr2=0x7fabc4070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:17:05.448 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:05.447+0000 7fabe5000700 1 --2- 172.21.15.153:0/3768442195 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fabc406e460 0x7fabc4070920 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fabd0005ea0 tx=0x7fabd0005e30 comp rx=0 tx=0).stop 2024-03-21T10:17:05.448 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:05.447+0000 7fabe5000700 1 -- 172.21.15.153:0/3768442195 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fabe0073ae0 msgr2=0x7fabe0198ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:17:05.449 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:05.447+0000 7fabe5000700 1 --2- 172.21.15.153:0/3768442195 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fabe0073ae0 0x7fabe0198ea0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fabcc0092e0 tx=0x7fabcc00be00 comp rx=0 tx=0).stop 2024-03-21T10:17:05.449 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:05.447+0000 7fabe5000700 1 -- 172.21.15.153:0/3768442195 shutdown_connections 2024-03-21T10:17:05.449 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:05.447+0000 7fabe5000700 1 --2- 172.21.15.153:0/3768442195 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fabe00731c0 0x7fabe0198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:17:05.449 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:05.447+0000 7fabe5000700 1 --2- 172.21.15.153:0/3768442195 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fabc406e460 0x7fabc4070920 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:17:05.449 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:05.447+0000 7fabe5000700 1 --2- 172.21.15.153:0/3768442195 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fabe0073ae0 0x7fabe0198ea0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:17:05.449 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:05.447+0000 7fabe5000700 1 -- 172.21.15.153:0/3768442195 >> 172.21.15.153:0/3768442195 conn(0x7fabe00fc630 msgr2=0x7fabe0107680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:17:05.449 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:05.448+0000 7fabe5000700 1 -- 172.21.15.153:0/3768442195 shutdown_connections 2024-03-21T10:17:05.449 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:05.448+0000 7fabe5000700 1 -- 172.21.15.153:0/3768442195 wait complete. 2024-03-21T10:17:05.610 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:05.609+0000 7f8b67a00700 1 -- 172.21.15.153:0/4203504029 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8b60069020 msgr2=0x7f8b60105790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:17:05.610 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:05.609+0000 7f8b67a00700 1 --2- 172.21.15.153:0/4203504029 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8b60069020 0x7f8b60105790 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c009910 tx=0x7f8b4c009c20 comp rx=0 tx=0).stop 2024-03-21T10:17:05.610 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:05.609+0000 7f8b67a00700 1 -- 172.21.15.153:0/4203504029 shutdown_connections 2024-03-21T10:17:05.610 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:05.609+0000 7f8b67a00700 1 --2- 172.21.15.153:0/4203504029 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8b60069020 0x7f8b60105790 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:17:05.610 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:05.609+0000 7f8b67a00700 1 --2- 172.21.15.153:0/4203504029 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8b60068670 0x7f8b60068a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:17:05.610 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:05.609+0000 7f8b67a00700 1 -- 172.21.15.153:0/4203504029 >> 172.21.15.153:0/4203504029 conn(0x7f8b600755a0 msgr2=0x7f8b600759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:17:05.611 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:05.609+0000 7f8b67a00700 1 -- 172.21.15.153:0/4203504029 shutdown_connections 2024-03-21T10:17:05.611 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:05.609+0000 7f8b67a00700 1 -- 172.21.15.153:0/4203504029 wait complete. 2024-03-21T10:17:05.612 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:05.610+0000 7f8b67a00700 1 Processor -- start 2024-03-21T10:17:05.612 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:05.610+0000 7f8b67a00700 1 -- start start 2024-03-21T10:17:05.612 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:05.611+0000 7f8b67a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8b60068670 0x7f8b60196700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:17:05.612 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:05.611+0000 7f8b67a00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8b60069020 0x7f8b60196c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:17:05.612 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:05.611+0000 7f8b67a00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b601972d0 con 0x7f8b60068670 2024-03-21T10:17:05.612 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:05.611+0000 7f8b67a00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b6019a000 con 0x7f8b60069020 2024-03-21T10:17:05.612 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:05.611+0000 7f8b65000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8b60068670 0x7f8b60196700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:17:05.612 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:05.611+0000 7f8b65000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8b60068670 0x7f8b60196700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:52296/0 (socket says 172.21.15.153:52296) 2024-03-21T10:17:05.612 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:05.611+0000 7f8b65000700 1 -- 172.21.15.153:0/822381317 learned_addr learned my addr 172.21.15.153:0/822381317 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:17:05.612 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:05.611+0000 7f8b5fe00700 1 --2- 172.21.15.153:0/822381317 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8b60069020 0x7f8b60196c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:17:05.613 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:05.612+0000 7f8b65000700 1 -- 172.21.15.153:0/822381317 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8b60069020 msgr2=0x7f8b60196c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:17:05.613 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:05.612+0000 7f8b65000700 1 --2- 172.21.15.153:0/822381317 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8b60069020 0x7f8b60196c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:17:05.613 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:05.612+0000 7f8b65000700 1 -- 172.21.15.153:0/822381317 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b4c0095f0 con 0x7f8b60068670 2024-03-21T10:17:05.614 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:05.612+0000 7f8b65000700 1 --2- 172.21.15.153:0/822381317 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8b60068670 0x7f8b60196700 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8b5400c8a0 tx=0x7f8b5400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:17:05.614 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:05.612+0000 7f8b5d600700 1 -- 172.21.15.153:0/822381317 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8b5400cea0 con 0x7f8b60068670 2024-03-21T10:17:05.614 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:05.612+0000 7f8b5d600700 1 -- 172.21.15.153:0/822381317 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8b54004830 con 0x7f8b60068670 2024-03-21T10:17:05.614 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:05.612+0000 7f8b5d600700 1 -- 172.21.15.153:0/822381317 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8b540055b0 con 0x7f8b60068670 2024-03-21T10:17:05.614 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:05.612+0000 7f8b67a00700 1 -- 172.21.15.153:0/822381317 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b6019a2e0 con 0x7f8b60068670 2024-03-21T10:17:05.614 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:05.612+0000 7f8b67a00700 1 -- 172.21.15.153:0/822381317 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b6019a7b0 con 0x7f8b60068670 2024-03-21T10:17:05.614 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:05.614+0000 7f8b67a00700 1 -- 172.21.15.153:0/822381317 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b601090a0 con 0x7f8b60068670 2024-03-21T10:17:05.615 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:05.615+0000 7f8b5d600700 1 -- 172.21.15.153:0/822381317 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f8b540049a0 con 0x7f8b60068670 2024-03-21T10:17:05.616 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:05.616+0000 7f8b5d600700 1 --2- 172.21.15.153:0/822381317 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f8b4806e460 0x7f8b48070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:17:05.616 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:05.616+0000 7f8b5d600700 1 -- 172.21.15.153:0/822381317 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6654+0+0 (secure 0 0 0) 0x7f8b5408e1a0 con 0x7f8b60068670 2024-03-21T10:17:05.617 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:05.616+0000 7f8b5fe00700 1 --2- 172.21.15.153:0/822381317 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f8b4806e460 0x7f8b48070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:17:05.617 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:05.617+0000 7f8b5fe00700 1 --2- 172.21.15.153:0/822381317 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f8b4806e460 0x7f8b48070920 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8b60197e20 tx=0x7f8b4c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:17:05.621 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:05.621+0000 7f8b5d600700 1 -- 172.21.15.153:0/822381317 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f8b54057270 con 0x7f8b60068670 2024-03-21T10:17:05.948 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:05.946+0000 7f8b67a00700 1 -- 172.21.15.153:0/822381317 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f8b6004ea90 con 0x7f8b60068670 2024-03-21T10:17:05.948 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:05.947+0000 7f8b5d600700 1 -- 172.21.15.153:0/822381317 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+791 (secure 0 0 0) 0x7f8b54004c50 con 0x7f8b60068670 2024-03-21T10:17:05.948 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:17:05.948 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": { 2024-03-21T10:17:05.949 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:17:05.949 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:17:05.949 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": { 2024-03-21T10:17:05.949 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:17:05.949 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:17:05.949 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": { 2024-03-21T10:17:05.949 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 3, 2024-03-21T10:17:05.949 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 3 2024-03-21T10:17:05.949 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:17:05.949 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": { 2024-03-21T10:17:05.949 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-03-21T10:17:05.949 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:17:05.949 INFO:teuthology.orchestra.run.smithi153.stdout: "overall": { 2024-03-21T10:17:05.949 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 7, 2024-03-21T10:17:05.950 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 7 2024-03-21T10:17:05.950 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-03-21T10:17:05.950 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:17:05.952 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:05.951+0000 7f8b67a00700 1 -- 172.21.15.153:0/822381317 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f8b4806e460 msgr2=0x7f8b48070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:17:05.952 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:05.951+0000 7f8b67a00700 1 --2- 172.21.15.153:0/822381317 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f8b4806e460 0x7f8b48070920 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8b60197e20 tx=0x7f8b4c01a040 comp rx=0 tx=0).stop 2024-03-21T10:17:05.952 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:05.951+0000 7f8b67a00700 1 -- 172.21.15.153:0/822381317 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8b60068670 msgr2=0x7f8b60196700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:17:05.953 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:05.951+0000 7f8b67a00700 1 --2- 172.21.15.153:0/822381317 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8b60068670 0x7f8b60196700 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8b5400c8a0 tx=0x7f8b5400cc60 comp rx=0 tx=0).stop 2024-03-21T10:17:05.953 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:05.952+0000 7f8b67a00700 1 -- 172.21.15.153:0/822381317 shutdown_connections 2024-03-21T10:17:05.953 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:05.952+0000 7f8b67a00700 1 --2- 172.21.15.153:0/822381317 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f8b4806e460 0x7f8b48070920 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:17:05.953 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:05.952+0000 7f8b67a00700 1 --2- 172.21.15.153:0/822381317 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8b60068670 0x7f8b60196700 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:17:05.953 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:05.952+0000 7f8b67a00700 1 --2- 172.21.15.153:0/822381317 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8b60069020 0x7f8b60196c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:17:05.953 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:05.952+0000 7f8b67a00700 1 -- 172.21.15.153:0/822381317 >> 172.21.15.153:0/822381317 conn(0x7f8b600755a0 msgr2=0x7f8b60105000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:17:05.953 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:05.952+0000 7f8b67a00700 1 -- 172.21.15.153:0/822381317 shutdown_connections 2024-03-21T10:17:05.953 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:05.952+0000 7f8b67a00700 1 -- 172.21.15.153:0/822381317 wait complete. 2024-03-21T10:17:06.119 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.118+0000 7f480ca00700 1 -- 172.21.15.153:0/3175128521 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4808069020 msgr2=0x7f4808105790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:17:06.119 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.118+0000 7f480ca00700 1 --2- 172.21.15.153:0/3175128521 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4808069020 0x7f4808105790 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f47f4009910 tx=0x7f47f4009c20 comp rx=0 tx=0).stop 2024-03-21T10:17:06.119 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.118+0000 7f480ca00700 1 -- 172.21.15.153:0/3175128521 shutdown_connections 2024-03-21T10:17:06.119 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.118+0000 7f480ca00700 1 --2- 172.21.15.153:0/3175128521 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4808069020 0x7f4808105790 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:17:06.119 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.118+0000 7f480ca00700 1 --2- 172.21.15.153:0/3175128521 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4808068670 0x7f4808068a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:17:06.119 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.118+0000 7f480ca00700 1 -- 172.21.15.153:0/3175128521 >> 172.21.15.153:0/3175128521 conn(0x7f48080755a0 msgr2=0x7f48080759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:17:06.120 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:05 smithi153 ceph-mon[113733]: from='client.34308 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:17:06.120 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:05 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:17:06.120 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:05 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:17:06.120 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:05 smithi153 ceph-mon[113733]: from='client.34312 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:17:06.120 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:05 smithi153 ceph-mon[113733]: pgmap v136: 65 pgs: 14 active+undersized, 11 active+undersized+degraded, 40 active+clean; 317 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s; 41/366 objects degraded (11.202%) 2024-03-21T10:17:06.121 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.119+0000 7f480ca00700 1 -- 172.21.15.153:0/3175128521 shutdown_connections 2024-03-21T10:17:06.121 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.119+0000 7f480ca00700 1 -- 172.21.15.153:0/3175128521 wait complete. 2024-03-21T10:17:06.121 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.120+0000 7f480ca00700 1 Processor -- start 2024-03-21T10:17:06.122 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.120+0000 7f480ca00700 1 -- start start 2024-03-21T10:17:06.122 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.120+0000 7f480ca00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4808068670 0x7f4808196700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:17:06.122 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.120+0000 7f480ca00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4808069020 0x7f4808196c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:17:06.122 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.120+0000 7f480ca00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48081972d0 con 0x7f4808068670 2024-03-21T10:17:06.122 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.120+0000 7f480ca00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f480819a000 con 0x7f4808069020 2024-03-21T10:17:06.122 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.121+0000 7f4805e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4808068670 0x7f4808196700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:17:06.122 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.121+0000 7f4805e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4808068670 0x7f4808196700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:52306/0 (socket says 172.21.15.153:52306) 2024-03-21T10:17:06.122 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.121+0000 7f4805e00700 1 -- 172.21.15.153:0/1689563890 learned_addr learned my addr 172.21.15.153:0/1689563890 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:17:06.122 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.121+0000 7f4805e00700 1 -- 172.21.15.153:0/1689563890 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4808069020 msgr2=0x7f4808196c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:17:06.122 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.121+0000 7f4805e00700 1 --2- 172.21.15.153:0/1689563890 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4808069020 0x7f4808196c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:17:06.122 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.121+0000 7f4805e00700 1 -- 172.21.15.153:0/1689563890 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47f40095f0 con 0x7f4808068670 2024-03-21T10:17:06.123 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.121+0000 7f4805e00700 1 --2- 172.21.15.153:0/1689563890 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4808068670 0x7f4808196700 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f47f800d8d0 tx=0x7f47f800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:17:06.123 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.121+0000 7f47fea00700 1 -- 172.21.15.153:0/1689563890 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f47f8009940 con 0x7f4808068670 2024-03-21T10:17:06.124 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.122+0000 7f47fea00700 1 -- 172.21.15.153:0/1689563890 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f47f8010460 con 0x7f4808068670 2024-03-21T10:17:06.124 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.122+0000 7f47fea00700 1 -- 172.21.15.153:0/1689563890 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f47f800f5f0 con 0x7f4808068670 2024-03-21T10:17:06.124 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.122+0000 7f480ca00700 1 -- 172.21.15.153:0/1689563890 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f480819a2e0 con 0x7f4808068670 2024-03-21T10:17:06.124 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.122+0000 7f480ca00700 1 -- 172.21.15.153:0/1689563890 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f480819a830 con 0x7f4808068670 2024-03-21T10:17:06.124 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.123+0000 7f480ca00700 1 -- 172.21.15.153:0/1689563890 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48081090a0 con 0x7f4808068670 2024-03-21T10:17:06.124 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.124+0000 7f47fea00700 1 -- 172.21.15.153:0/1689563890 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f47f80105d0 con 0x7f4808068670 2024-03-21T10:17:06.125 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.125+0000 7f47fea00700 1 --2- 172.21.15.153:0/1689563890 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f47ec06e460 0x7f47ec070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:17:06.125 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.125+0000 7f47fea00700 1 -- 172.21.15.153:0/1689563890 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6654+0+0 (secure 0 0 0) 0x7f47f808e460 con 0x7f4808068670 2024-03-21T10:17:06.125 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.125+0000 7f4805400700 1 --2- 172.21.15.153:0/1689563890 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f47ec06e460 0x7f47ec070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:17:06.126 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.125+0000 7f4805400700 1 --2- 172.21.15.153:0/1689563890 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f47ec06e460 0x7f47ec070920 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f47f4006010 tx=0x7f47f4005960 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:17:06.130 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.130+0000 7f47fea00700 1 -- 172.21.15.153:0/1689563890 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f47f80575d0 con 0x7f4808068670 2024-03-21T10:17:06.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:05 smithi174 ceph-mon[88004]: from='client.34308 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:17:06.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:05 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:17:06.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:05 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:17:06.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:05 smithi174 ceph-mon[88004]: from='client.34312 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:17:06.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:05 smithi174 ceph-mon[88004]: pgmap v136: 65 pgs: 14 active+undersized, 11 active+undersized+degraded, 40 active+clean; 317 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s; 41/366 objects degraded (11.202%) 2024-03-21T10:17:06.400 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.399+0000 7f480ca00700 1 -- 172.21.15.153:0/1689563890 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f480804ea90 con 0x7f4808068670 2024-03-21T10:17:06.401 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.400+0000 7f47fea00700 1 -- 172.21.15.153:0/1689563890 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 21 v21) v1 ==== 76+0+1903 (secure 0 0 0) 0x7f47f805ac90 con 0x7f4808068670 2024-03-21T10:17:06.401 INFO:teuthology.orchestra.run.smithi153.stdout:e21 2024-03-21T10:17:06.401 INFO:teuthology.orchestra.run.smithi153.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-21T10:17:06.402 INFO:teuthology.orchestra.run.smithi153.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:17:06.402 INFO:teuthology.orchestra.run.smithi153.stdout:legacy client fscid: 1 2024-03-21T10:17:06.402 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:17:06.402 INFO:teuthology.orchestra.run.smithi153.stdout:Filesystem 'cephfs' (1) 2024-03-21T10:17:06.402 INFO:teuthology.orchestra.run.smithi153.stdout:fs_name cephfs 2024-03-21T10:17:06.402 INFO:teuthology.orchestra.run.smithi153.stdout:epoch 21 2024-03-21T10:17:06.402 INFO:teuthology.orchestra.run.smithi153.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-21T10:17:06.402 INFO:teuthology.orchestra.run.smithi153.stdout:created 2024-03-21T10:04:27.086576+0000 2024-03-21T10:17:06.402 INFO:teuthology.orchestra.run.smithi153.stdout:modified 2024-03-21T10:13:01.455594+0000 2024-03-21T10:17:06.402 INFO:teuthology.orchestra.run.smithi153.stdout:tableserver 0 2024-03-21T10:17:06.402 INFO:teuthology.orchestra.run.smithi153.stdout:root 0 2024-03-21T10:17:06.402 INFO:teuthology.orchestra.run.smithi153.stdout:session_timeout 60 2024-03-21T10:17:06.402 INFO:teuthology.orchestra.run.smithi153.stdout:session_autoclose 300 2024-03-21T10:17:06.402 INFO:teuthology.orchestra.run.smithi153.stdout:max_file_size 1099511627776 2024-03-21T10:17:06.403 INFO:teuthology.orchestra.run.smithi153.stdout:max_xattr_size 65536 2024-03-21T10:17:06.403 INFO:teuthology.orchestra.run.smithi153.stdout:required_client_features {} 2024-03-21T10:17:06.403 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure 0 2024-03-21T10:17:06.403 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure_osd_epoch 47 2024-03-21T10:17:06.403 INFO:teuthology.orchestra.run.smithi153.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:17:06.403 INFO:teuthology.orchestra.run.smithi153.stdout:max_mds 2 2024-03-21T10:17:06.403 INFO:teuthology.orchestra.run.smithi153.stdout:in 0,1 2024-03-21T10:17:06.403 INFO:teuthology.orchestra.run.smithi153.stdout:up {0=14484,1=24273} 2024-03-21T10:17:06.403 INFO:teuthology.orchestra.run.smithi153.stdout:failed 2024-03-21T10:17:06.403 INFO:teuthology.orchestra.run.smithi153.stdout:damaged 2024-03-21T10:17:06.403 INFO:teuthology.orchestra.run.smithi153.stdout:stopped 2024-03-21T10:17:06.403 INFO:teuthology.orchestra.run.smithi153.stdout:data_pools [3] 2024-03-21T10:17:06.403 INFO:teuthology.orchestra.run.smithi153.stdout:metadata_pool 2 2024-03-21T10:17:06.403 INFO:teuthology.orchestra.run.smithi153.stdout:inline_data enabled 2024-03-21T10:17:06.403 INFO:teuthology.orchestra.run.smithi153.stdout:balancer 2024-03-21T10:17:06.404 INFO:teuthology.orchestra.run.smithi153.stdout:bal_rank_mask -1 2024-03-21T10:17:06.404 INFO:teuthology.orchestra.run.smithi153.stdout:standby_count_wanted 1 2024-03-21T10:17:06.404 INFO:teuthology.orchestra.run.smithi153.stdout:qdb_cluster leader: 0 members: 2024-03-21T10:17:06.404 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.iuyluj{0:14484} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.153:6828/866138999,v1:172.21.15.153:6829/866138999] compat {c=[1],r=[1],i=[7ff]}] 2024-03-21T10:17:06.404 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.zipckr{0:14494} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.153:6826/4033622615,v1:172.21.15.153:6827/4033622615] compat {c=[1],r=[1],i=[7ff]}] 2024-03-21T10:17:06.404 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.yyimst{1:24273} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.174:6824/1776947618,v1:172.21.15.174:6825/1776947618] compat {c=[1],r=[1],i=[7ff]}] 2024-03-21T10:17:06.404 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.nhozag{1:24285} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.174:6826/806095873,v1:172.21.15.174:6827/806095873] compat {c=[1],r=[1],i=[7ff]}] 2024-03-21T10:17:06.404 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:17:06.404 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:17:06.405 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.403+0000 7f480ca00700 1 -- 172.21.15.153:0/1689563890 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f47ec06e460 msgr2=0x7f47ec070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:17:06.405 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.403+0000 7f480ca00700 1 --2- 172.21.15.153:0/1689563890 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f47ec06e460 0x7f47ec070920 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f47f4006010 tx=0x7f47f4005960 comp rx=0 tx=0).stop 2024-03-21T10:17:06.405 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.403+0000 7f480ca00700 1 -- 172.21.15.153:0/1689563890 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4808068670 msgr2=0x7f4808196700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:17:06.405 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.403+0000 7f480ca00700 1 --2- 172.21.15.153:0/1689563890 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4808068670 0x7f4808196700 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f47f800d8d0 tx=0x7f47f800dc90 comp rx=0 tx=0).stop 2024-03-21T10:17:06.405 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.403+0000 7f480ca00700 1 -- 172.21.15.153:0/1689563890 shutdown_connections 2024-03-21T10:17:06.405 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.403+0000 7f480ca00700 1 --2- 172.21.15.153:0/1689563890 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f47ec06e460 0x7f47ec070920 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:17:06.405 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.403+0000 7f480ca00700 1 --2- 172.21.15.153:0/1689563890 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4808068670 0x7f4808196700 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:17:06.405 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.403+0000 7f480ca00700 1 --2- 172.21.15.153:0/1689563890 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4808069020 0x7f4808196c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:17:06.406 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.403+0000 7f480ca00700 1 -- 172.21.15.153:0/1689563890 >> 172.21.15.153:0/1689563890 conn(0x7f48080755a0 msgr2=0x7f4808105000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:17:06.406 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.403+0000 7f480ca00700 1 -- 172.21.15.153:0/1689563890 shutdown_connections 2024-03-21T10:17:06.406 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.404+0000 7f480ca00700 1 -- 172.21.15.153:0/1689563890 wait complete. 2024-03-21T10:17:06.406 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 21 2024-03-21T10:17:06.561 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.560+0000 7f28f8000700 1 -- 172.21.15.153:0/3190755595 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f28f010d280 msgr2=0x7f28f010d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:17:06.561 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.560+0000 7f28f8000700 1 --2- 172.21.15.153:0/3190755595 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f28f010d280 0x7f28f010d660 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f28e4009910 tx=0x7f28e4009c20 comp rx=0 tx=0).stop 2024-03-21T10:17:06.562 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.560+0000 7f28f8000700 1 -- 172.21.15.153:0/3190755595 shutdown_connections 2024-03-21T10:17:06.562 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.560+0000 7f28f8000700 1 --2- 172.21.15.153:0/3190755595 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f28f01071d0 0x7f28f0107650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:17:06.562 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.560+0000 7f28f8000700 1 --2- 172.21.15.153:0/3190755595 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f28f010d280 0x7f28f010d660 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:17:06.562 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.560+0000 7f28f8000700 1 -- 172.21.15.153:0/3190755595 >> 172.21.15.153:0/3190755595 conn(0x7f28f0076990 msgr2=0x7f28f0078db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:17:06.562 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.560+0000 7f28f8000700 1 -- 172.21.15.153:0/3190755595 shutdown_connections 2024-03-21T10:17:06.562 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.560+0000 7f28f8000700 1 -- 172.21.15.153:0/3190755595 wait complete. 2024-03-21T10:17:06.563 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.561+0000 7f28f8000700 1 Processor -- start 2024-03-21T10:17:06.564 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.561+0000 7f28f8000700 1 -- start start 2024-03-21T10:17:06.564 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.562+0000 7f28f8000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f28f01071d0 0x7f28f019cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:17:06.564 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.562+0000 7f28f8000700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f28f010d280 0x7f28f019d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:17:06.564 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.562+0000 7f28f8000700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28f019d9f0 con 0x7f28f01071d0 2024-03-21T10:17:06.564 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.562+0000 7f28f8000700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28f01a1780 con 0x7f28f010d280 2024-03-21T10:17:06.564 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.562+0000 7f28f5600700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f28f01071d0 0x7f28f019cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:17:06.564 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.562+0000 7f28f5600700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f28f01071d0 0x7f28f019cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:52332/0 (socket says 172.21.15.153:52332) 2024-03-21T10:17:06.564 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.562+0000 7f28f5600700 1 -- 172.21.15.153:0/3971828421 learned_addr learned my addr 172.21.15.153:0/3971828421 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:17:06.564 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.562+0000 7f28f4c00700 1 --2- 172.21.15.153:0/3971828421 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f28f010d280 0x7f28f019d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:17:06.564 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.562+0000 7f28f5600700 1 -- 172.21.15.153:0/3971828421 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f28f010d280 msgr2=0x7f28f019d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:17:06.564 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.562+0000 7f28f5600700 1 --2- 172.21.15.153:0/3971828421 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f28f010d280 0x7f28f019d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:17:06.564 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.562+0000 7f28f5600700 1 -- 172.21.15.153:0/3971828421 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28e40095f0 con 0x7f28f01071d0 2024-03-21T10:17:06.565 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.563+0000 7f28f5600700 1 --2- 172.21.15.153:0/3971828421 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f28f01071d0 0x7f28f019cdd0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f28e400b5c0 tx=0x7f28e400be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:17:06.565 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.563+0000 7f28ee000700 1 -- 172.21.15.153:0/3971828421 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f28e401d070 con 0x7f28f01071d0 2024-03-21T10:17:06.565 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.563+0000 7f28ee000700 1 -- 172.21.15.153:0/3971828421 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f28e400f460 con 0x7f28f01071d0 2024-03-21T10:17:06.565 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.563+0000 7f28ee000700 1 -- 172.21.15.153:0/3971828421 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f28e4017610 con 0x7f28f01071d0 2024-03-21T10:17:06.565 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.563+0000 7f28f8000700 1 -- 172.21.15.153:0/3971828421 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28f01a1a00 con 0x7f28f01071d0 2024-03-21T10:17:06.565 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.563+0000 7f28f8000700 1 -- 172.21.15.153:0/3971828421 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28f01a1f20 con 0x7f28f01071d0 2024-03-21T10:17:06.565 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.565+0000 7f28f8000700 1 -- 172.21.15.153:0/3971828421 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28f004ea90 con 0x7f28f01071d0 2024-03-21T10:17:06.566 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.566+0000 7f28ee000700 1 -- 172.21.15.153:0/3971828421 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f28e400f5d0 con 0x7f28f01071d0 2024-03-21T10:17:06.567 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.567+0000 7f28ee000700 1 --2- 172.21.15.153:0/3971828421 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f28d806e460 0x7f28d8070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:17:06.567 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.567+0000 7f28ee000700 1 -- 172.21.15.153:0/3971828421 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6654+0+0 (secure 0 0 0) 0x7f28e408fad0 con 0x7f28f01071d0 2024-03-21T10:17:06.568 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.567+0000 7f28f4c00700 1 --2- 172.21.15.153:0/3971828421 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f28d806e460 0x7f28d8070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:17:06.568 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.568+0000 7f28f4c00700 1 --2- 172.21.15.153:0/3971828421 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f28d806e460 0x7f28d8070920 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f28dc006fd0 tx=0x7f28dc008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:17:06.573 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.573+0000 7f28ee000700 1 -- 172.21.15.153:0/3971828421 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f28e4058ba0 con 0x7f28f01071d0 2024-03-21T10:17:06.812 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.811+0000 7f28f8000700 1 -- 172.21.15.153:0/3971828421 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f28f01a2450 con 0x7f28d806e460 2024-03-21T10:17:06.814 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.813+0000 7f28ee000700 1 -- 172.21.15.153:0/3971828421 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f28f01a2450 con 0x7f28d806e460 2024-03-21T10:17:06.814 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:17:06.814 INFO:teuthology.orchestra.run.smithi153.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232", 2024-03-21T10:17:06.814 INFO:teuthology.orchestra.run.smithi153.stdout: "in_progress": true, 2024-03-21T10:17:06.814 INFO:teuthology.orchestra.run.smithi153.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-21T10:17:06.814 INFO:teuthology.orchestra.run.smithi153.stdout: "services_complete": [ 2024-03-21T10:17:06.814 INFO:teuthology.orchestra.run.smithi153.stdout: "crash", 2024-03-21T10:17:06.815 INFO:teuthology.orchestra.run.smithi153.stdout: "mon", 2024-03-21T10:17:06.815 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr" 2024-03-21T10:17:06.815 INFO:teuthology.orchestra.run.smithi153.stdout: ], 2024-03-21T10:17:06.815 INFO:teuthology.orchestra.run.smithi153.stdout: "progress": "9/23 daemons upgraded", 2024-03-21T10:17:06.815 INFO:teuthology.orchestra.run.smithi153.stdout: "message": "Currently upgrading osd daemons", 2024-03-21T10:17:06.815 INFO:teuthology.orchestra.run.smithi153.stdout: "is_paused": false 2024-03-21T10:17:06.815 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:17:06.816 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.816+0000 7f28f8000700 1 -- 172.21.15.153:0/3971828421 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f28d806e460 msgr2=0x7f28d8070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:17:06.816 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.816+0000 7f28f8000700 1 --2- 172.21.15.153:0/3971828421 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f28d806e460 0x7f28d8070920 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f28dc006fd0 tx=0x7f28dc008040 comp rx=0 tx=0).stop 2024-03-21T10:17:06.816 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.816+0000 7f28f8000700 1 -- 172.21.15.153:0/3971828421 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f28f01071d0 msgr2=0x7f28f019cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:17:06.817 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.816+0000 7f28f8000700 1 --2- 172.21.15.153:0/3971828421 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f28f01071d0 0x7f28f019cdd0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f28e400b5c0 tx=0x7f28e400be00 comp rx=0 tx=0).stop 2024-03-21T10:17:06.817 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.816+0000 7f28f8000700 1 -- 172.21.15.153:0/3971828421 shutdown_connections 2024-03-21T10:17:06.817 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.816+0000 7f28f8000700 1 --2- 172.21.15.153:0/3971828421 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f28d806e460 0x7f28d8070920 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:17:06.817 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.816+0000 7f28f8000700 1 --2- 172.21.15.153:0/3971828421 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f28f01071d0 0x7f28f019cdd0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:17:06.817 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.816+0000 7f28f8000700 1 --2- 172.21.15.153:0/3971828421 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f28f010d280 0x7f28f019d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:17:06.818 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.816+0000 7f28f8000700 1 -- 172.21.15.153:0/3971828421 >> 172.21.15.153:0/3971828421 conn(0x7f28f0076990 msgr2=0x7f28f0078430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:17:06.818 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.816+0000 7f28f8000700 1 -- 172.21.15.153:0/3971828421 shutdown_connections 2024-03-21T10:17:06.818 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.816+0000 7f28f8000700 1 -- 172.21.15.153:0/3971828421 wait complete. 2024-03-21T10:17:06.964 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.963+0000 7fcab8200700 1 -- 172.21.15.153:0/2201329240 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcab0100da0 msgr2=0x7fcab0101180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:17:06.965 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.963+0000 7fcab8200700 1 --2- 172.21.15.153:0/2201329240 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcab0100da0 0x7fcab0101180 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fcaa4009910 tx=0x7fcaa4009c20 comp rx=0 tx=0).stop 2024-03-21T10:17:06.965 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.963+0000 7fcab8200700 1 -- 172.21.15.153:0/2201329240 shutdown_connections 2024-03-21T10:17:06.965 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.963+0000 7fcab8200700 1 --2- 172.21.15.153:0/2201329240 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcab0101750 0x7fcab01057a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:17:06.965 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.963+0000 7fcab8200700 1 --2- 172.21.15.153:0/2201329240 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcab0100da0 0x7fcab0101180 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:17:06.965 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.963+0000 7fcab8200700 1 -- 172.21.15.153:0/2201329240 >> 172.21.15.153:0/2201329240 conn(0x7fcab00fc650 msgr2=0x7fcab00fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:17:06.965 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.963+0000 7fcab8200700 1 -- 172.21.15.153:0/2201329240 shutdown_connections 2024-03-21T10:17:06.965 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.963+0000 7fcab8200700 1 -- 172.21.15.153:0/2201329240 wait complete. 2024-03-21T10:17:06.966 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.964+0000 7fcab8200700 1 Processor -- start 2024-03-21T10:17:06.966 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.965+0000 7fcab8200700 1 -- start start 2024-03-21T10:17:06.966 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.965+0000 7fcab8200700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcab0100da0 0x7fcab019c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:17:06.967 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.965+0000 7fcab8200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcab0101750 0x7fcab019ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:17:06.967 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.965+0000 7fcab8200700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcab019d0f0 con 0x7fcab0101750 2024-03-21T10:17:06.967 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.965+0000 7fcab8200700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcab01965a0 con 0x7fcab0100da0 2024-03-21T10:17:06.967 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.965+0000 7fcab4e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcab0101750 0x7fcab019ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:17:06.967 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.965+0000 7fcab4e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcab0101750 0x7fcab019ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:52354/0 (socket says 172.21.15.153:52354) 2024-03-21T10:17:06.967 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.965+0000 7fcab4e00700 1 -- 172.21.15.153:0/2440265099 learned_addr learned my addr 172.21.15.153:0/2440265099 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:17:06.967 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.965+0000 7fcab5800700 1 --2- 172.21.15.153:0/2440265099 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcab0100da0 0x7fcab019c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:17:06.967 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.966+0000 7fcab4e00700 1 -- 172.21.15.153:0/2440265099 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcab0100da0 msgr2=0x7fcab019c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:17:06.967 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.966+0000 7fcab4e00700 1 --2- 172.21.15.153:0/2440265099 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcab0100da0 0x7fcab019c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:17:06.967 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.966+0000 7fcab4e00700 1 -- 172.21.15.153:0/2440265099 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaa40095f0 con 0x7fcab0101750 2024-03-21T10:17:06.967 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.966+0000 7fcab4e00700 1 --2- 172.21.15.153:0/2440265099 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcab0101750 0x7fcab019ca60 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fca9c00c8f0 tx=0x7fca9c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:17:06.968 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.966+0000 7fcaae000700 1 -- 172.21.15.153:0/2440265099 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fca9c007a50 con 0x7fcab0101750 2024-03-21T10:17:06.968 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.966+0000 7fcaae000700 1 -- 172.21.15.153:0/2440265099 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fca9c007bb0 con 0x7fcab0101750 2024-03-21T10:17:06.968 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.966+0000 7fcab8200700 1 -- 172.21.15.153:0/2440265099 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcab0196880 con 0x7fcab0101750 2024-03-21T10:17:06.968 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.966+0000 7fcaae000700 1 -- 172.21.15.153:0/2440265099 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fca9c018630 con 0x7fcab0101750 2024-03-21T10:17:06.968 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.966+0000 7fcab8200700 1 -- 172.21.15.153:0/2440265099 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcab0196dd0 con 0x7fcab0101750 2024-03-21T10:17:06.968 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.967+0000 7fcab8200700 1 -- 172.21.15.153:0/2440265099 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcab0109140 con 0x7fcab0101750 2024-03-21T10:17:06.969 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.968+0000 7fcaae000700 1 -- 172.21.15.153:0/2440265099 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fca9c007d20 con 0x7fcab0101750 2024-03-21T10:17:06.969 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:06 smithi153 ceph-mon[113733]: from='client.34316 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:17:06.969 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:06 smithi153 ceph-mon[113733]: Health check update: Degraded data redundancy: 41/366 objects degraded (11.202%), 11 pgs degraded (PG_DEGRADED) 2024-03-21T10:17:06.969 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:06 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/822381317' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:17:06.969 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:06 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/1689563890' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:17:06.970 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.969+0000 7fcaae000700 1 --2- 172.21.15.153:0/2440265099 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fca9806e460 0x7fca98070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:17:06.970 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.969+0000 7fcaae000700 1 -- 172.21.15.153:0/2440265099 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6654+0+0 (secure 0 0 0) 0x7fca9c08e3a0 con 0x7fcab0101750 2024-03-21T10:17:06.970 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.970+0000 7fcab5800700 1 --2- 172.21.15.153:0/2440265099 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fca9806e460 0x7fca98070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:17:06.971 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.970+0000 7fcab5800700 1 --2- 172.21.15.153:0/2440265099 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fca9806e460 0x7fca98070920 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fcaa400b5c0 tx=0x7fcaa401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:17:06.975 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:06.975+0000 7fcaae000700 1 -- 172.21.15.153:0/2440265099 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fca9c05b020 con 0x7fcab0101750 2024-03-21T10:17:07.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:06 smithi174 ceph-mon[88004]: from='client.34316 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:17:07.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:06 smithi174 ceph-mon[88004]: Health check update: Degraded data redundancy: 41/366 objects degraded (11.202%), 11 pgs degraded (PG_DEGRADED) 2024-03-21T10:17:07.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:06 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/822381317' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:17:07.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:06 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/1689563890' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:17:07.287 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:07.286+0000 7fcab8200700 1 -- 172.21.15.153:0/2440265099 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fcab01970d0 con 0x7fcab0101750 2024-03-21T10:17:07.287 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:07.287+0000 7fcaae000700 1 -- 172.21.15.153:0/2440265099 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+985 (secure 0 0 0) 0x7fca9c05abb0 con 0x7fcab0101750 2024-03-21T10:17:07.288 INFO:teuthology.orchestra.run.smithi153.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data; Degraded data redundancy: 41/366 objects degraded (11.202%), 11 pgs degraded 2024-03-21T10:17:07.288 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-21T10:17:07.288 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-21T10:17:07.288 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 41/366 objects degraded (11.202%), 11 pgs degraded 2024-03-21T10:17:07.288 INFO:teuthology.orchestra.run.smithi153.stdout: pg 2.0 is active+undersized+degraded, acting [1,0] 2024-03-21T10:17:07.288 INFO:teuthology.orchestra.run.smithi153.stdout: pg 2.1 is active+undersized+degraded, acting [1,0] 2024-03-21T10:17:07.288 INFO:teuthology.orchestra.run.smithi153.stdout: pg 2.3 is active+undersized+degraded, acting [5,1] 2024-03-21T10:17:07.288 INFO:teuthology.orchestra.run.smithi153.stdout: pg 2.7 is active+undersized+degraded, acting [3,4] 2024-03-21T10:17:07.288 INFO:teuthology.orchestra.run.smithi153.stdout: pg 2.d is active+undersized+degraded, acting [1,3] 2024-03-21T10:17:07.288 INFO:teuthology.orchestra.run.smithi153.stdout: pg 2.e is active+undersized+degraded, acting [0,3] 2024-03-21T10:17:07.288 INFO:teuthology.orchestra.run.smithi153.stdout: pg 2.10 is active+undersized+degraded, acting [1,0] 2024-03-21T10:17:07.288 INFO:teuthology.orchestra.run.smithi153.stdout: pg 2.16 is active+undersized+degraded, acting [5,0] 2024-03-21T10:17:07.289 INFO:teuthology.orchestra.run.smithi153.stdout: pg 2.17 is active+undersized+degraded, acting [5,1] 2024-03-21T10:17:07.289 INFO:teuthology.orchestra.run.smithi153.stdout: pg 2.1c is active+undersized+degraded, acting [4,5] 2024-03-21T10:17:07.289 INFO:teuthology.orchestra.run.smithi153.stdout: pg 2.1e is active+undersized+degraded, acting [0,5] 2024-03-21T10:17:07.290 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:07.290+0000 7fcab8200700 1 -- 172.21.15.153:0/2440265099 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fca9806e460 msgr2=0x7fca98070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:17:07.291 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:07.290+0000 7fcab8200700 1 --2- 172.21.15.153:0/2440265099 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fca9806e460 0x7fca98070920 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fcaa400b5c0 tx=0x7fcaa401a040 comp rx=0 tx=0).stop 2024-03-21T10:17:07.291 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:07.290+0000 7fcab8200700 1 -- 172.21.15.153:0/2440265099 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcab0101750 msgr2=0x7fcab019ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:17:07.291 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:07.290+0000 7fcab8200700 1 --2- 172.21.15.153:0/2440265099 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcab0101750 0x7fcab019ca60 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fca9c00c8f0 tx=0x7fca9c00ccb0 comp rx=0 tx=0).stop 2024-03-21T10:17:07.291 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:07.290+0000 7fcab8200700 1 -- 172.21.15.153:0/2440265099 shutdown_connections 2024-03-21T10:17:07.291 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:07.290+0000 7fcab8200700 1 --2- 172.21.15.153:0/2440265099 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcab0100da0 0x7fcab019c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:17:07.291 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:07.290+0000 7fcab8200700 1 --2- 172.21.15.153:0/2440265099 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fca9806e460 0x7fca98070920 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:17:07.291 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:07.290+0000 7fcab8200700 1 --2- 172.21.15.153:0/2440265099 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcab0101750 0x7fcab019ca60 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:17:07.291 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:07.290+0000 7fcab8200700 1 -- 172.21.15.153:0/2440265099 >> 172.21.15.153:0/2440265099 conn(0x7fcab00fc650 msgr2=0x7fcab00fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:17:07.291 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:07.290+0000 7fcab8200700 1 -- 172.21.15.153:0/2440265099 shutdown_connections 2024-03-21T10:17:07.292 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:07.290+0000 7fcab8200700 1 -- 172.21.15.153:0/2440265099 wait complete. 2024-03-21T10:17:08.028 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:07 smithi153 ceph-mon[113733]: from='client.34328 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:17:08.028 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:07 smithi153 ceph-mon[113733]: pgmap v137: 65 pgs: 65 active+clean; 317 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:17:08.029 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:07 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/2440265099' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:17:08.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:07 smithi174 ceph-mon[88004]: from='client.34328 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:17:08.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:07 smithi174 ceph-mon[88004]: pgmap v137: 65 pgs: 65 active+clean; 317 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:17:08.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:07 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/2440265099' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:17:09.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:08 smithi174 ceph-mon[88004]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 41/366 objects degraded (11.202%), 11 pgs degraded) 2024-03-21T10:17:09.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:08 smithi153 ceph-mon[113733]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 41/366 objects degraded (11.202%), 11 pgs degraded) 2024-03-21T10:17:10.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:09 smithi174 ceph-mon[88004]: pgmap v138: 65 pgs: 65 active+clean; 317 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 2 op/s 2024-03-21T10:17:10.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:09 smithi153 ceph-mon[113733]: pgmap v138: 65 pgs: 65 active+clean; 317 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 2 op/s 2024-03-21T10:17:11.888 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:11 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:17:11.888 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:11 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:17:11.888 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:11 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:17:11.888 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:11 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:17:11.888 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:11 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:17:11.888 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:11 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:17:11.889 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:11 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:17:11.889 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:11 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:17:11.889 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:11 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:17:11.889 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:11 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-21T10:17:11.889 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:11 smithi174 ceph-mon[88004]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-21T10:17:11.889 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:11 smithi174 ceph-mon[88004]: Upgrade: osd.3 is safe to restart 2024-03-21T10:17:11.889 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:11 smithi174 ceph-mon[88004]: pgmap v139: 65 pgs: 65 active+clean; 317 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.4 KiB/s rd, 3 op/s 2024-03-21T10:17:12.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:11 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:17:12.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:11 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:17:12.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:11 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:17:12.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:11 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:17:12.058 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:11 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:17:12.058 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:11 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:17:12.058 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:11 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:17:12.058 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:11 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:17:12.058 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:11 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:17:12.058 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:11 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-21T10:17:12.058 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:11 smithi153 ceph-mon[113733]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-21T10:17:12.058 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:11 smithi153 ceph-mon[113733]: Upgrade: osd.3 is safe to restart 2024-03-21T10:17:12.058 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:11 smithi153 ceph-mon[113733]: pgmap v139: 65 pgs: 65 active+clean; 317 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.4 KiB/s rd, 3 op/s 2024-03-21T10:17:14.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:14 smithi174 ceph-mon[88004]: pgmap v140: 65 pgs: 65 active+clean; 317 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:17:14.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:14 smithi174 ceph-mon[88004]: Upgrade: Updating osd.3 2024-03-21T10:17:14.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:14 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:17:14.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:14 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-21T10:17:14.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:14 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:17:14.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:14 smithi153 ceph-mon[113733]: pgmap v140: 65 pgs: 65 active+clean; 317 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:17:14.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:14 smithi153 ceph-mon[113733]: Upgrade: Updating osd.3 2024-03-21T10:17:14.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:14 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:17:14.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:14 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-21T10:17:14.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:14 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:17:15.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:15 smithi153 ceph-mon[113733]: Deploying daemon osd.3 on smithi174 2024-03-21T10:17:15.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:15 smithi174 ceph-mon[88004]: Deploying daemon osd.3 on smithi174 2024-03-21T10:17:16.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:16 smithi174 ceph-mon[88004]: pgmap v141: 65 pgs: 65 active+clean; 317 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s 2024-03-21T10:17:16.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:16 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:17:16.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:16 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:17:16.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:16 smithi153 ceph-mon[113733]: pgmap v141: 65 pgs: 65 active+clean; 317 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s 2024-03-21T10:17:16.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:16 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:17:16.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:16 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:17:17.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:17 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:17:17.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:17 smithi153 ceph-mon[113733]: pgmap v142: 65 pgs: 65 active+clean; 317 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 4 op/s 2024-03-21T10:17:17.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:17 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:17:17.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:17 smithi174 ceph-mon[88004]: pgmap v142: 65 pgs: 65 active+clean; 317 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 4 op/s 2024-03-21T10:17:18.452 INFO:journalctl@ceph.osd.3.smithi174.stdout:Mar 21 10:17:18 smithi174 systemd[1]: Stopping Ceph osd.3 for 070ed3e4-e769-11ee-95cd-87774f69a715... 2024-03-21T10:17:18.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:18 smithi153 ceph-mon[113733]: osd.3 marked itself down and dead 2024-03-21T10:17:18.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:18 smithi153 ceph-mon[113733]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-21T10:17:18.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:18 smithi174 ceph-mon[88004]: osd.3 marked itself down and dead 2024-03-21T10:17:18.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:18 smithi174 ceph-mon[88004]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-21T10:17:18.952 INFO:journalctl@ceph.osd.3.smithi174.stdout:Mar 21 10:17:18 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-3[43457]: 2024-03-21T10:17:18.482+0000 7f352ca00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T10:17:18.952 INFO:journalctl@ceph.osd.3.smithi174.stdout:Mar 21 10:17:18 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-3[43457]: 2024-03-21T10:17:18.482+0000 7f352ca00700 -1 osd.3 76 *** Got signal Terminated *** 2024-03-21T10:17:18.952 INFO:journalctl@ceph.osd.3.smithi174.stdout:Mar 21 10:17:18 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-3[43457]: 2024-03-21T10:17:18.482+0000 7f352ca00700 -1 osd.3 76 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-21T10:17:19.452 INFO:journalctl@ceph.osd.3.smithi174.stdout:Mar 21 10:17:19 smithi174 podman[96681]: 2024-03-21 10:17:19.195654174 +0000 UTC m=+0.849736208 container died dafc89e0658d1306a8be7e36f14103aeb2c839644c3f0f3f1b006a33a1f9c37c (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-3, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, RELEASE=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=https://github.com/ceph/ceph-container.git, CEPH_POINT_RELEASE=-18.2.1, maintainer=Guillaume Abrioux , io.buildah.version=1.29.1, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True) 2024-03-21T10:17:19.754 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:19 smithi174 ceph-mon[88004]: osdmap e77: 6 total, 5 up, 6 in 2024-03-21T10:17:19.754 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:19 smithi174 ceph-mon[88004]: pgmap v144: 65 pgs: 13 stale+active+clean, 52 active+clean; 317 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-03-21T10:17:19.754 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:19 smithi174 ceph-mon[88004]: osdmap e78: 6 total, 5 up, 6 in 2024-03-21T10:17:19.754 INFO:journalctl@ceph.osd.3.smithi174.stdout:Mar 21 10:17:19 smithi174 podman[96681]: 2024-03-21 10:17:19.584278166 +0000 UTC m=+1.238360204 container cleanup dafc89e0658d1306a8be7e36f14103aeb2c839644c3f0f3f1b006a33a1f9c37c (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-3, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=HEAD, GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=-18.2.1, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.29.1, org.label-schema.schema-version=1.0) 2024-03-21T10:17:19.754 INFO:journalctl@ceph.osd.3.smithi174.stdout:Mar 21 10:17:19 smithi174 bash[96681]: ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-3 2024-03-21T10:17:20.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:19 smithi153 ceph-mon[113733]: osdmap e77: 6 total, 5 up, 6 in 2024-03-21T10:17:20.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:19 smithi153 ceph-mon[113733]: pgmap v144: 65 pgs: 13 stale+active+clean, 52 active+clean; 317 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-03-21T10:17:20.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:19 smithi153 ceph-mon[113733]: osdmap e78: 6 total, 5 up, 6 in 2024-03-21T10:17:20.201 INFO:journalctl@ceph.osd.3.smithi174.stdout:Mar 21 10:17:19 smithi174 podman[96700]: 2024-03-21 10:17:19.752283931 +0000 UTC m=+0.542411809 container remove dafc89e0658d1306a8be7e36f14103aeb2c839644c3f0f3f1b006a33a1f9c37c (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-3, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=https://github.com/ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, io.buildah.version=1.29.1, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=-18.2.1, org.label-schema.schema-version=1.0, RELEASE=HEAD, org.label-schema.build-date=20240222) 2024-03-21T10:17:20.777 INFO:journalctl@ceph.osd.3.smithi174.stdout:Mar 21 10:17:20 smithi174 podman[96789]: 2024-03-21 10:17:20.37014304 +0000 UTC m=+0.031723207 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T10:17:20.777 INFO:journalctl@ceph.osd.3.smithi174.stdout:Mar 21 10:17:20 smithi174 podman[96789]: 2024-03-21 10:17:20.494921392 +0000 UTC m=+0.156501613 container create d0aea2ba6321f16a65584161ce4088dcacd3d9354d86fa60d77c8256348d3153 (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-3-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_BRANCH=HEAD) 2024-03-21T10:17:20.777 INFO:journalctl@ceph.osd.3.smithi174.stdout:Mar 21 10:17:20 smithi174 podman[96789]: 2024-03-21 10:17:20.775255965 +0000 UTC m=+0.436836195 container init d0aea2ba6321f16a65584161ce4088dcacd3d9354d86fa60d77c8256348d3153 (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-3-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T10:17:21.201 INFO:journalctl@ceph.osd.3.smithi174.stdout:Mar 21 10:17:20 smithi174 podman[96789]: 2024-03-21 10:17:20.788080983 +0000 UTC m=+0.449661220 container start d0aea2ba6321f16a65584161ce4088dcacd3d9354d86fa60d77c8256348d3153 (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-3-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-03-21T10:17:21.201 INFO:journalctl@ceph.osd.3.smithi174.stdout:Mar 21 10:17:20 smithi174 podman[96789]: 2024-03-21 10:17:20.864039454 +0000 UTC m=+0.525619665 container attach d0aea2ba6321f16a65584161ce4088dcacd3d9354d86fa60d77c8256348d3153 (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-3-deactivate, org.label-schema.build-date=20240319, ceph=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T10:17:21.598 INFO:journalctl@ceph.osd.3.smithi174.stdout:Mar 21 10:17:21 smithi174 podman[96789]: 2024-03-21 10:17:21.215967704 +0000 UTC m=+0.877547915 container died d0aea2ba6321f16a65584161ce4088dcacd3d9354d86fa60d77c8256348d3153 (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-3-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6) 2024-03-21T10:17:21.893 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:21 smithi174 ceph-mon[88004]: pgmap v146: 65 pgs: 5 active+undersized, 6 active+undersized+degraded, 6 peering, 9 stale+active+clean, 39 active+clean; 317 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 17/366 objects degraded (4.645%) 2024-03-21T10:17:22.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:21 smithi153 ceph-mon[113733]: pgmap v146: 65 pgs: 5 active+undersized, 6 active+undersized+degraded, 6 peering, 9 stale+active+clean, 39 active+clean; 317 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 17/366 objects degraded (4.645%) 2024-03-21T10:17:22.202 INFO:journalctl@ceph.osd.3.smithi174.stdout:Mar 21 10:17:21 smithi174 podman[96815]: 2024-03-21 10:17:21.892161646 +0000 UTC m=+0.660555231 container remove d0aea2ba6321f16a65584161ce4088dcacd3d9354d86fa60d77c8256348d3153 (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-3-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True) 2024-03-21T10:17:22.202 INFO:journalctl@ceph.osd.3.smithi174.stdout:Mar 21 10:17:21 smithi174 systemd[1]: ceph-070ed3e4-e769-11ee-95cd-87774f69a715@osd.3.service: Deactivated successfully. 2024-03-21T10:17:22.202 INFO:journalctl@ceph.osd.3.smithi174.stdout:Mar 21 10:17:21 smithi174 systemd[1]: Stopped Ceph osd.3 for 070ed3e4-e769-11ee-95cd-87774f69a715. 2024-03-21T10:17:22.202 INFO:journalctl@ceph.osd.3.smithi174.stdout:Mar 21 10:17:21 smithi174 systemd[1]: ceph-070ed3e4-e769-11ee-95cd-87774f69a715@osd.3.service: Consumed 1min 55.992s CPU time. 2024-03-21T10:17:22.677 INFO:journalctl@ceph.osd.3.smithi174.stdout:Mar 21 10:17:22 smithi174 systemd[1]: Starting Ceph osd.3 for 070ed3e4-e769-11ee-95cd-87774f69a715... 2024-03-21T10:17:22.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:22 smithi174 ceph-mon[88004]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-03-21T10:17:22.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:22 smithi174 ceph-mon[88004]: Health check failed: Degraded data redundancy: 17/366 objects degraded (4.645%), 6 pgs degraded (PG_DEGRADED) 2024-03-21T10:17:22.952 INFO:journalctl@ceph.osd.3.smithi174.stdout:Mar 21 10:17:22 smithi174 podman[96914]: 2024-03-21 10:17:22.581001858 +0000 UTC m=+0.032515069 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T10:17:22.952 INFO:journalctl@ceph.osd.3.smithi174.stdout:Mar 21 10:17:22 smithi174 podman[96914]: 2024-03-21 10:17:22.684753808 +0000 UTC m=+0.136267043 container create 46367b9c7e10ee0a1811c11309e3135dc4a858610cfe51bd5e6378d01fdcac21 (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-3-activate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, ceph=True, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T10:17:23.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:22 smithi153 ceph-mon[113733]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-03-21T10:17:23.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:22 smithi153 ceph-mon[113733]: Health check failed: Degraded data redundancy: 17/366 objects degraded (4.645%), 6 pgs degraded (PG_DEGRADED) 2024-03-21T10:17:23.452 INFO:journalctl@ceph.osd.3.smithi174.stdout:Mar 21 10:17:22 smithi174 podman[96914]: 2024-03-21 10:17:22.968452497 +0000 UTC m=+0.419965785 container init 46367b9c7e10ee0a1811c11309e3135dc4a858610cfe51bd5e6378d01fdcac21 (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-3-activate, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-03-21T10:17:23.452 INFO:journalctl@ceph.osd.3.smithi174.stdout:Mar 21 10:17:22 smithi174 podman[96914]: 2024-03-21 10:17:22.980082946 +0000 UTC m=+0.431596214 container start 46367b9c7e10ee0a1811c11309e3135dc4a858610cfe51bd5e6378d01fdcac21 (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-3-activate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, io.buildah.version=1.33.5) 2024-03-21T10:17:23.452 INFO:journalctl@ceph.osd.3.smithi174.stdout:Mar 21 10:17:23 smithi174 podman[96914]: 2024-03-21 10:17:23.062209439 +0000 UTC m=+0.513722718 container attach 46367b9c7e10ee0a1811c11309e3135dc4a858610cfe51bd5e6378d01fdcac21 (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-3-activate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-03-21T10:17:24.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:23 smithi153 ceph-mon[113733]: pgmap v147: 65 pgs: 14 active+undersized, 13 active+undersized+degraded, 6 peering, 32 active+clean; 317 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 47/366 objects degraded (12.842%) 2024-03-21T10:17:24.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:23 smithi174 ceph-mon[88004]: pgmap v147: 65 pgs: 14 active+undersized, 13 active+undersized+degraded, 6 peering, 32 active+clean; 317 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 47/366 objects degraded (12.842%) 2024-03-21T10:17:24.202 INFO:journalctl@ceph.osd.3.smithi174.stdout:Mar 21 10:17:23 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-3-activate[96929]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-03-21T10:17:24.202 INFO:journalctl@ceph.osd.3.smithi174.stdout:Mar 21 10:17:23 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-3-activate[96929]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-03-21T10:17:24.202 INFO:journalctl@ceph.osd.3.smithi174.stdout:Mar 21 10:17:23 smithi174 bash[96914]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-03-21T10:17:24.202 INFO:journalctl@ceph.osd.3.smithi174.stdout:Mar 21 10:17:23 smithi174 bash[96914]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-03-21T10:17:24.203 INFO:journalctl@ceph.osd.3.smithi174.stdout:Mar 21 10:17:23 smithi174 bash[96914]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-03-21T10:17:24.203 INFO:journalctl@ceph.osd.3.smithi174.stdout:Mar 21 10:17:23 smithi174 bash[96914]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-03-21T10:17:24.203 INFO:journalctl@ceph.osd.3.smithi174.stdout:Mar 21 10:17:23 smithi174 bash[96914]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3/block 2024-03-21T10:17:24.203 INFO:journalctl@ceph.osd.3.smithi174.stdout:Mar 21 10:17:23 smithi174 bash[96914]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-03-21T10:17:24.203 INFO:journalctl@ceph.osd.3.smithi174.stdout:Mar 21 10:17:23 smithi174 bash[96914]: --> ceph-volume raw activate successful for osd ID: 3 2024-03-21T10:17:24.203 INFO:journalctl@ceph.osd.3.smithi174.stdout:Mar 21 10:17:23 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-3-activate[96929]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-03-21T10:17:24.203 INFO:journalctl@ceph.osd.3.smithi174.stdout:Mar 21 10:17:23 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-3-activate[96929]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-03-21T10:17:24.203 INFO:journalctl@ceph.osd.3.smithi174.stdout:Mar 21 10:17:23 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-3-activate[96929]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3/block 2024-03-21T10:17:24.203 INFO:journalctl@ceph.osd.3.smithi174.stdout:Mar 21 10:17:23 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-3-activate[96929]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-03-21T10:17:24.203 INFO:journalctl@ceph.osd.3.smithi174.stdout:Mar 21 10:17:23 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-3-activate[96929]: --> ceph-volume raw activate successful for osd ID: 3 2024-03-21T10:17:24.203 INFO:journalctl@ceph.osd.3.smithi174.stdout:Mar 21 10:17:23 smithi174 podman[96914]: 2024-03-21 10:17:23.894620947 +0000 UTC m=+1.346134215 container died 46367b9c7e10ee0a1811c11309e3135dc4a858610cfe51bd5e6378d01fdcac21 (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-3-activate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-03-21T10:17:24.907 INFO:journalctl@ceph.osd.3.smithi174.stdout:Mar 21 10:17:24 smithi174 podman[97065]: 2024-03-21 10:17:24.615776771 +0000 UTC m=+0.701068461 container remove 46367b9c7e10ee0a1811c11309e3135dc4a858610cfe51bd5e6378d01fdcac21 (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-3-activate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD) 2024-03-21T10:17:25.188 INFO:journalctl@ceph.osd.3.smithi174.stdout:Mar 21 10:17:24 smithi174 podman[97126]: 2024-03-21 10:17:24.880143054 +0000 UTC m=+0.039021245 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T10:17:25.188 INFO:journalctl@ceph.osd.3.smithi174.stdout:Mar 21 10:17:25 smithi174 podman[97126]: 2024-03-21 10:17:25.01683888 +0000 UTC m=+0.175717109 container create 70aafac71ad5172e74975729b2b146172e4465077c25e200138804e94bb98d64 (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-3, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-03-21T10:17:25.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:25 smithi174 ceph-mon[88004]: pgmap v148: 65 pgs: 17 active+undersized, 16 active+undersized+degraded, 32 active+clean; 317 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 55/366 objects degraded (15.027%) 2024-03-21T10:17:25.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:25 smithi174 ceph-mon[88004]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-03-21T10:17:25.701 INFO:journalctl@ceph.osd.3.smithi174.stdout:Mar 21 10:17:25 smithi174 podman[97126]: 2024-03-21 10:17:25.311964871 +0000 UTC m=+0.470843076 container init 70aafac71ad5172e74975729b2b146172e4465077c25e200138804e94bb98d64 (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-3, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-03-21T10:17:25.702 INFO:journalctl@ceph.osd.3.smithi174.stdout:Mar 21 10:17:25 smithi174 podman[97126]: 2024-03-21 10:17:25.323119182 +0000 UTC m=+0.481997363 container start 70aafac71ad5172e74975729b2b146172e4465077c25e200138804e94bb98d64 (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-3, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-03-21T10:17:25.702 INFO:journalctl@ceph.osd.3.smithi174.stdout:Mar 21 10:17:25 smithi174 ceph-osd[97145]: -- 172.21.15.174:0/1169151060 <== mon.1 v2:172.21.15.174:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x556af8063680 con 0x556af802cc00 2024-03-21T10:17:25.702 INFO:journalctl@ceph.osd.3.smithi174.stdout:Mar 21 10:17:25 smithi174 bash[97126]: 70aafac71ad5172e74975729b2b146172e4465077c25e200138804e94bb98d64 2024-03-21T10:17:25.702 INFO:journalctl@ceph.osd.3.smithi174.stdout:Mar 21 10:17:25 smithi174 systemd[1]: Started Ceph osd.3 for 070ed3e4-e769-11ee-95cd-87774f69a715. 2024-03-21T10:17:25.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:25 smithi153 ceph-mon[113733]: pgmap v148: 65 pgs: 17 active+undersized, 16 active+undersized+degraded, 32 active+clean; 317 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 55/366 objects degraded (15.027%) 2024-03-21T10:17:25.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:25 smithi153 ceph-mon[113733]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-03-21T10:17:26.201 INFO:journalctl@ceph.osd.3.smithi174.stdout:Mar 21 10:17:25 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-3[97141]: 2024-03-21T10:17:25.891+0000 7f27592ac7c0 -1 Falling back to public interface 2024-03-21T10:17:26.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:26 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:17:26.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:26 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:17:26.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:26 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:17:26.866 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:26 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:17:26.866 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:26 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:17:26.866 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:26 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:17:27.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:27 smithi174 ceph-mon[88004]: pgmap v149: 65 pgs: 17 active+undersized, 16 active+undersized+degraded, 32 active+clean; 317 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 55/366 objects degraded (15.027%) 2024-03-21T10:17:28.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:27 smithi153 ceph-mon[113733]: pgmap v149: 65 pgs: 17 active+undersized, 16 active+undersized+degraded, 32 active+clean; 317 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 55/366 objects degraded (15.027%) 2024-03-21T10:17:29.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:29 smithi174 ceph-mon[88004]: pgmap v150: 65 pgs: 17 active+undersized, 16 active+undersized+degraded, 32 active+clean; 317 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 55/366 objects degraded (15.027%) 2024-03-21T10:17:29.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:29 smithi153 ceph-mon[113733]: pgmap v150: 65 pgs: 17 active+undersized, 16 active+undersized+degraded, 32 active+clean; 317 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 55/366 objects degraded (15.027%) 2024-03-21T10:17:31.104 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:30 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:17:31.104 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:30 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:17:31.104 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:30 smithi174 ceph-mon[88004]: Health check update: Degraded data redundancy: 55/366 objects degraded (15.027%), 16 pgs degraded (PG_DEGRADED) 2024-03-21T10:17:31.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:30 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:17:31.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:30 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:17:31.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:30 smithi153 ceph-mon[113733]: Health check update: Degraded data redundancy: 55/366 objects degraded (15.027%), 16 pgs degraded (PG_DEGRADED) 2024-03-21T10:17:31.451 INFO:journalctl@ceph.osd.3.smithi174.stdout:Mar 21 10:17:31 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-3[97141]: 2024-03-21T10:17:31.101+0000 7f27592ac7c0 -1 osd.3 0 read_superblock omap replica is missing. 2024-03-21T10:17:31.951 INFO:journalctl@ceph.osd.3.smithi174.stdout:Mar 21 10:17:31 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-3[97141]: 2024-03-21T10:17:31.475+0000 7f27592ac7c0 -1 osd.3 76 log_to_monitors true 2024-03-21T10:17:32.452 INFO:journalctl@ceph.osd.3.smithi174.stdout:Mar 21 10:17:32 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-3[97141]: 2024-03-21T10:17:32.260+0000 7f2748600700 -1 osd.3 76 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-21T10:17:32.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:32 smithi174 ceph-mon[88004]: pgmap v151: 65 pgs: 17 active+undersized, 16 active+undersized+degraded, 32 active+clean; 317 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 55/366 objects degraded (15.027%) 2024-03-21T10:17:32.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:32 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:17:32.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:32 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:17:32.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:32 smithi174 ceph-mon[88004]: from='osd.3 [v2:172.21.15.174:6800/1830370519,v1:172.21.15.174:6801/1830370519]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-21T10:17:32.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:32 smithi174 ceph-mon[88004]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-21T10:17:32.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:32 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:17:32.453 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:32 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:17:32.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:32 smithi153 ceph-mon[113733]: pgmap v151: 65 pgs: 17 active+undersized, 16 active+undersized+degraded, 32 active+clean; 317 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 55/366 objects degraded (15.027%) 2024-03-21T10:17:32.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:32 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:17:32.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:32 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:17:32.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:32 smithi153 ceph-mon[113733]: from='osd.3 [v2:172.21.15.174:6800/1830370519,v1:172.21.15.174:6801/1830370519]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-21T10:17:32.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:32 smithi153 ceph-mon[113733]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-21T10:17:32.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:32 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:17:32.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:32 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:17:33.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:33 smithi174 ceph-mon[88004]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-21T10:17:33.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:33 smithi174 ceph-mon[88004]: osdmap e79: 6 total, 5 up, 6 in 2024-03-21T10:17:33.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:33 smithi174 ceph-mon[88004]: from='osd.3 [v2:172.21.15.174:6800/1830370519,v1:172.21.15.174:6801/1830370519]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-03-21T10:17:33.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:33 smithi174 ceph-mon[88004]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-03-21T10:17:33.558 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:33 smithi153 ceph-mon[113733]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-21T10:17:33.559 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:33 smithi153 ceph-mon[113733]: osdmap e79: 6 total, 5 up, 6 in 2024-03-21T10:17:33.559 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:33 smithi153 ceph-mon[113733]: from='osd.3 [v2:172.21.15.174:6800/1830370519,v1:172.21.15.174:6801/1830370519]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-03-21T10:17:33.559 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:33 smithi153 ceph-mon[113733]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-03-21T10:17:34.417 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:34 smithi174 ceph-mon[88004]: pgmap v153: 65 pgs: 17 active+undersized, 16 active+undersized+degraded, 32 active+clean; 317 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s; 55/366 objects degraded (15.027%) 2024-03-21T10:17:34.417 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:34 smithi174 ceph-mon[88004]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-21T10:17:34.417 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:34 smithi174 ceph-mon[88004]: osd.3 [v2:172.21.15.174:6800/1830370519,v1:172.21.15.174:6801/1830370519] boot 2024-03-21T10:17:34.417 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:34 smithi174 ceph-mon[88004]: osdmap e80: 6 total, 6 up, 6 in 2024-03-21T10:17:34.417 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:34 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T10:17:34.417 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:34 smithi174 ceph-mon[88004]: osdmap e81: 6 total, 6 up, 6 in 2024-03-21T10:17:34.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:34 smithi153 ceph-mon[113733]: pgmap v153: 65 pgs: 17 active+undersized, 16 active+undersized+degraded, 32 active+clean; 317 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s; 55/366 objects degraded (15.027%) 2024-03-21T10:17:34.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:34 smithi153 ceph-mon[113733]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-21T10:17:34.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:34 smithi153 ceph-mon[113733]: osd.3 [v2:172.21.15.174:6800/1830370519,v1:172.21.15.174:6801/1830370519] boot 2024-03-21T10:17:34.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:34 smithi153 ceph-mon[113733]: osdmap e80: 6 total, 6 up, 6 in 2024-03-21T10:17:34.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:34 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T10:17:34.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:34 smithi153 ceph-mon[113733]: osdmap e81: 6 total, 6 up, 6 in 2024-03-21T10:17:35.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:35 smithi174 ceph-mon[88004]: pgmap v156: 65 pgs: 1 activating, 1 active, 14 active+undersized, 15 active+undersized+degraded, 34 active+clean; 317 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s; 51/366 objects degraded (13.934%) 2024-03-21T10:17:35.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:35 smithi153 ceph-mon[113733]: pgmap v156: 65 pgs: 1 activating, 1 active, 14 active+undersized, 15 active+undersized+degraded, 34 active+clean; 317 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s; 51/366 objects degraded (13.934%) 2024-03-21T10:17:36.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:36 smithi153 ceph-mon[113733]: Health check update: Degraded data redundancy: 51/366 objects degraded (13.934%), 15 pgs degraded (PG_DEGRADED) 2024-03-21T10:17:36.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:36 smithi174 ceph-mon[88004]: Health check update: Degraded data redundancy: 51/366 objects degraded (13.934%), 15 pgs degraded (PG_DEGRADED) 2024-03-21T10:17:37.466 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:37.461+0000 7f1fb5000700 1 -- 172.21.15.153:0/140402109 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1fb00ffbd0 msgr2=0x7f1fb0100050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:17:37.467 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:37.461+0000 7f1fb5000700 1 --2- 172.21.15.153:0/140402109 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1fb00ffbd0 0x7f1fb0100050 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c009910 tx=0x7f1f9c009c20 comp rx=0 tx=0).stop 2024-03-21T10:17:37.467 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:37.461+0000 7f1fb5000700 1 -- 172.21.15.153:0/140402109 shutdown_connections 2024-03-21T10:17:37.467 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:37.461+0000 7f1fb5000700 1 --2- 172.21.15.153:0/140402109 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1fb00ffbd0 0x7f1fb0100050 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:17:37.467 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:37.461+0000 7f1fb5000700 1 --2- 172.21.15.153:0/140402109 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1fb0105a40 0x7f1fb0105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:17:37.467 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:37.461+0000 7f1fb5000700 1 -- 172.21.15.153:0/140402109 >> 172.21.15.153:0/140402109 conn(0x7f1fb00755a0 msgr2=0x7f1fb00759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:17:37.467 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:37.461+0000 7f1fb5000700 1 -- 172.21.15.153:0/140402109 shutdown_connections 2024-03-21T10:17:37.467 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:37.461+0000 7f1fb5000700 1 -- 172.21.15.153:0/140402109 wait complete. 2024-03-21T10:17:37.467 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:37.462+0000 7f1fb5000700 1 Processor -- start 2024-03-21T10:17:37.468 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:37.463+0000 7f1fb5000700 1 -- start start 2024-03-21T10:17:37.468 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:37.463+0000 7f1fb5000700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1fb00ffbd0 0x7f1fb019e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:17:37.468 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:37.463+0000 7f1fb5000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1fb0105a40 0x7f1fb019ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:17:37.468 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:37.463+0000 7f1fb5000700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb019f3f0 con 0x7f1fb0105a40 2024-03-21T10:17:37.468 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:37.463+0000 7f1fb5000700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb01988a0 con 0x7f1fb00ffbd0 2024-03-21T10:17:37.468 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:37.463+0000 7f1fad400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1fb0105a40 0x7f1fb019ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:17:37.468 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:37.463+0000 7f1fad400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1fb0105a40 0x7f1fb019ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:49638/0 (socket says 172.21.15.153:49638) 2024-03-21T10:17:37.468 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:37.463+0000 7f1fad400700 1 -- 172.21.15.153:0/492488154 learned_addr learned my addr 172.21.15.153:0/492488154 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:17:37.469 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:37.464+0000 7f1fade00700 1 --2- 172.21.15.153:0/492488154 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1fb00ffbd0 0x7f1fb019e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:17:37.469 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:37.464+0000 7f1fad400700 1 -- 172.21.15.153:0/492488154 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1fb00ffbd0 msgr2=0x7f1fb019e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:17:37.469 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:37.464+0000 7f1fad400700 1 --2- 172.21.15.153:0/492488154 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1fb00ffbd0 0x7f1fb019e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:17:37.469 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:37.464+0000 7f1fad400700 1 -- 172.21.15.153:0/492488154 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f9c0095f0 con 0x7f1fb0105a40 2024-03-21T10:17:37.471 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:37.464+0000 7f1fad400700 1 --2- 172.21.15.153:0/492488154 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1fb0105a40 0x7f1fb019ed60 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c006010 tx=0x7f1f9c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:17:37.471 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:37.465+0000 7f1fa6a00700 1 -- 172.21.15.153:0/492488154 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f9c01d070 con 0x7f1fb0105a40 2024-03-21T10:17:37.471 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:37.465+0000 7f1fa6a00700 1 -- 172.21.15.153:0/492488154 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1f9c004fb0 con 0x7f1fb0105a40 2024-03-21T10:17:37.471 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:37.465+0000 7f1fb5000700 1 -- 172.21.15.153:0/492488154 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fb0198b20 con 0x7f1fb0105a40 2024-03-21T10:17:37.471 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:37.465+0000 7f1fa6a00700 1 -- 172.21.15.153:0/492488154 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f9c00f690 con 0x7f1fb0105a40 2024-03-21T10:17:37.472 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:37.465+0000 7f1fb5000700 1 -- 172.21.15.153:0/492488154 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fb0199010 con 0x7f1fb0105a40 2024-03-21T10:17:37.472 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:37.466+0000 7f1fb5000700 1 -- 172.21.15.153:0/492488154 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fb004ea90 con 0x7f1fb0105a40 2024-03-21T10:17:37.472 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:37.467+0000 7f1fa6a00700 1 -- 172.21.15.153:0/492488154 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f1f9c0228a0 con 0x7f1fb0105a40 2024-03-21T10:17:37.472 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:37.468+0000 7f1fa6a00700 1 --2- 172.21.15.153:0/492488154 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f1f9406e460 0x7f1f94070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:17:37.472 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:37.468+0000 7f1fa6a00700 1 -- 172.21.15.153:0/492488154 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6654+0+0 (secure 0 0 0) 0x7f1f9c08fc80 con 0x7f1fb0105a40 2024-03-21T10:17:37.473 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:37.469+0000 7f1fade00700 1 --2- 172.21.15.153:0/492488154 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f1f9406e460 0x7f1f94070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:17:37.473 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:37.469+0000 7f1fade00700 1 --2- 172.21.15.153:0/492488154 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f1f9406e460 0x7f1f94070920 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1fa0006fd0 tx=0x7f1fa0008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:17:37.475 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:37.473+0000 7f1fa6a00700 1 -- 172.21.15.153:0/492488154 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f1f9c05c900 con 0x7f1fb0105a40 2024-03-21T10:17:37.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:37 smithi153 ceph-mon[113733]: pgmap v157: 65 pgs: 1 activating, 1 active, 5 active+undersized, 8 active+undersized+degraded, 50 active+clean; 317 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 3 op/s; 30/366 objects degraded (8.197%) 2024-03-21T10:17:37.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:37 smithi174 ceph-mon[88004]: pgmap v157: 65 pgs: 1 activating, 1 active, 5 active+undersized, 8 active+undersized+degraded, 50 active+clean; 317 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 3 op/s; 30/366 objects degraded (8.197%) 2024-03-21T10:17:37.721 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:37.719+0000 7f1fb5000700 1 -- 172.21.15.153:0/492488154 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1fb0199540 con 0x7f1f9406e460 2024-03-21T10:17:37.724 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:37.722+0000 7f1fa6a00700 1 -- 172.21.15.153:0/492488154 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f1fb0199540 con 0x7f1f9406e460 2024-03-21T10:17:37.727 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:37.725+0000 7f1fb5000700 1 -- 172.21.15.153:0/492488154 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f1f9406e460 msgr2=0x7f1f94070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:17:37.727 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:37.725+0000 7f1fb5000700 1 --2- 172.21.15.153:0/492488154 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f1f9406e460 0x7f1f94070920 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1fa0006fd0 tx=0x7f1fa0008040 comp rx=0 tx=0).stop 2024-03-21T10:17:37.727 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:37.725+0000 7f1fb5000700 1 -- 172.21.15.153:0/492488154 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1fb0105a40 msgr2=0x7f1fb019ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:17:37.727 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:37.725+0000 7f1fb5000700 1 --2- 172.21.15.153:0/492488154 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1fb0105a40 0x7f1fb019ed60 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c006010 tx=0x7f1f9c00be20 comp rx=0 tx=0).stop 2024-03-21T10:17:37.728 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:37.725+0000 7f1fb5000700 1 -- 172.21.15.153:0/492488154 shutdown_connections 2024-03-21T10:17:37.728 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:37.725+0000 7f1fb5000700 1 --2- 172.21.15.153:0/492488154 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1fb00ffbd0 0x7f1fb019e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:17:37.728 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:37.725+0000 7f1fb5000700 1 --2- 172.21.15.153:0/492488154 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f1f9406e460 0x7f1f94070920 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:17:37.729 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:37.725+0000 7f1fb5000700 1 --2- 172.21.15.153:0/492488154 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1fb0105a40 0x7f1fb019ed60 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:17:37.729 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:37.725+0000 7f1fb5000700 1 -- 172.21.15.153:0/492488154 >> 172.21.15.153:0/492488154 conn(0x7f1fb00755a0 msgr2=0x7f1fb00684b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:17:37.729 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:37.726+0000 7f1fb5000700 1 -- 172.21.15.153:0/492488154 shutdown_connections 2024-03-21T10:17:37.729 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:37.726+0000 7f1fb5000700 1 -- 172.21.15.153:0/492488154 wait complete. 2024-03-21T10:17:37.744 INFO:teuthology.orchestra.run.smithi153.stdout:true 2024-03-21T10:17:37.889 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:37.887+0000 7fa4db800700 1 -- 172.21.15.153:0/2686321073 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa4d410d280 msgr2=0x7fa4d410d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:17:37.889 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:37.887+0000 7fa4db800700 1 --2- 172.21.15.153:0/2686321073 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa4d410d280 0x7fa4d410d660 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa4c8009910 tx=0x7fa4c8009c20 comp rx=0 tx=0).stop 2024-03-21T10:17:37.889 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:37.887+0000 7fa4db800700 1 -- 172.21.15.153:0/2686321073 shutdown_connections 2024-03-21T10:17:37.889 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:37.887+0000 7fa4db800700 1 --2- 172.21.15.153:0/2686321073 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa4d41071d0 0x7fa4d4107650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:17:37.889 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:37.887+0000 7fa4db800700 1 --2- 172.21.15.153:0/2686321073 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa4d410d280 0x7fa4d410d660 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:17:37.889 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:37.887+0000 7fa4db800700 1 -- 172.21.15.153:0/2686321073 >> 172.21.15.153:0/2686321073 conn(0x7fa4d40769e0 msgr2=0x7fa4d4078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:17:37.889 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:37.887+0000 7fa4db800700 1 -- 172.21.15.153:0/2686321073 shutdown_connections 2024-03-21T10:17:37.889 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:37.887+0000 7fa4db800700 1 -- 172.21.15.153:0/2686321073 wait complete. 2024-03-21T10:17:37.891 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:37.888+0000 7fa4db800700 1 Processor -- start 2024-03-21T10:17:37.891 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:37.888+0000 7fa4db800700 1 -- start start 2024-03-21T10:17:37.891 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:37.888+0000 7fa4db800700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa4d41071d0 0x7fa4d419ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:17:37.891 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:37.889+0000 7fa4db800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa4d410d280 0x7fa4d419d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:17:37.891 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:37.889+0000 7fa4db800700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4d419d8a0 con 0x7fa4d410d280 2024-03-21T10:17:37.891 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:37.889+0000 7fa4db800700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4d41a05d0 con 0x7fa4d41071d0 2024-03-21T10:17:37.891 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:37.889+0000 7fa4d3e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa4d410d280 0x7fa4d419d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:17:37.891 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:37.889+0000 7fa4d3e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa4d410d280 0x7fa4d419d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:49650/0 (socket says 172.21.15.153:49650) 2024-03-21T10:17:37.891 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:37.889+0000 7fa4d3e00700 1 -- 172.21.15.153:0/1324259564 learned_addr learned my addr 172.21.15.153:0/1324259564 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:17:37.892 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:37.889+0000 7fa4d8e00700 1 --2- 172.21.15.153:0/1324259564 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa4d41071d0 0x7fa4d419ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:17:37.892 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:37.889+0000 7fa4d3e00700 1 -- 172.21.15.153:0/1324259564 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa4d41071d0 msgr2=0x7fa4d419ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:17:37.892 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:37.889+0000 7fa4d3e00700 1 --2- 172.21.15.153:0/1324259564 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa4d41071d0 0x7fa4d419ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:17:37.892 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:37.889+0000 7fa4d3e00700 1 -- 172.21.15.153:0/1324259564 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4c80095f0 con 0x7fa4d410d280 2024-03-21T10:17:37.892 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:37.890+0000 7fa4d3e00700 1 --2- 172.21.15.153:0/1324259564 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa4d410d280 0x7fa4d419d210 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa4c000d8d0 tx=0x7fa4c000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:17:37.893 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:37.890+0000 7fa4d1600700 1 -- 172.21.15.153:0/1324259564 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa4c0009940 con 0x7fa4d410d280 2024-03-21T10:17:37.893 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:37.890+0000 7fa4d1600700 1 -- 172.21.15.153:0/1324259564 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa4c0010460 con 0x7fa4d410d280 2024-03-21T10:17:37.893 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:37.890+0000 7fa4db800700 1 -- 172.21.15.153:0/1324259564 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4d41a08b0 con 0x7fa4d410d280 2024-03-21T10:17:37.893 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:37.890+0000 7fa4d1600700 1 -- 172.21.15.153:0/1324259564 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa4c000f5f0 con 0x7fa4d410d280 2024-03-21T10:17:37.893 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:37.890+0000 7fa4db800700 1 -- 172.21.15.153:0/1324259564 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4d41a0e80 con 0x7fa4d410d280 2024-03-21T10:17:37.893 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:37.891+0000 7fa4db800700 1 -- 172.21.15.153:0/1324259564 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4d404ea90 con 0x7fa4d410d280 2024-03-21T10:17:37.894 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:37.892+0000 7fa4d1600700 1 -- 172.21.15.153:0/1324259564 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fa4c00105d0 con 0x7fa4d410d280 2024-03-21T10:17:37.895 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:37.893+0000 7fa4d1600700 1 --2- 172.21.15.153:0/1324259564 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa4bc06e460 0x7fa4bc070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:17:37.895 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:37.893+0000 7fa4d1600700 1 -- 172.21.15.153:0/1324259564 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6654+0+0 (secure 0 0 0) 0x7fa4c008e450 con 0x7fa4d410d280 2024-03-21T10:17:37.895 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:37.894+0000 7fa4d8e00700 1 --2- 172.21.15.153:0/1324259564 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa4bc06e460 0x7fa4bc070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:17:37.896 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:37.894+0000 7fa4d8e00700 1 --2- 172.21.15.153:0/1324259564 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa4bc06e460 0x7fa4bc070920 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa4c8006010 tx=0x7fa4c801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:17:37.900 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:37.898+0000 7fa4d1600700 1 -- 172.21.15.153:0/1324259564 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa4c00575c0 con 0x7fa4d410d280 2024-03-21T10:17:38.141 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:38.139+0000 7fa4db800700 1 -- 172.21.15.153:0/1324259564 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa4d41a1290 con 0x7fa4bc06e460 2024-03-21T10:17:38.143 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:38.141+0000 7fa4d1600700 1 -- 172.21.15.153:0/1324259564 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7fa4d41a1290 con 0x7fa4bc06e460 2024-03-21T10:17:38.146 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:38.144+0000 7fa4db800700 1 -- 172.21.15.153:0/1324259564 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa4bc06e460 msgr2=0x7fa4bc070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:17:38.146 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:38.144+0000 7fa4db800700 1 --2- 172.21.15.153:0/1324259564 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa4bc06e460 0x7fa4bc070920 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa4c8006010 tx=0x7fa4c801a040 comp rx=0 tx=0).stop 2024-03-21T10:17:38.146 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:38.144+0000 7fa4db800700 1 -- 172.21.15.153:0/1324259564 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa4d410d280 msgr2=0x7fa4d419d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:17:38.146 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:38.144+0000 7fa4db800700 1 --2- 172.21.15.153:0/1324259564 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa4d410d280 0x7fa4d419d210 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa4c000d8d0 tx=0x7fa4c000dc90 comp rx=0 tx=0).stop 2024-03-21T10:17:38.146 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:38.144+0000 7fa4db800700 1 -- 172.21.15.153:0/1324259564 shutdown_connections 2024-03-21T10:17:38.146 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:38.144+0000 7fa4db800700 1 --2- 172.21.15.153:0/1324259564 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa4d41071d0 0x7fa4d419ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:17:38.146 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:38.144+0000 7fa4db800700 1 --2- 172.21.15.153:0/1324259564 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa4bc06e460 0x7fa4bc070920 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:17:38.146 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:38.144+0000 7fa4db800700 1 --2- 172.21.15.153:0/1324259564 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa4d410d280 0x7fa4d419d210 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:17:38.146 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:38.144+0000 7fa4db800700 1 -- 172.21.15.153:0/1324259564 >> 172.21.15.153:0/1324259564 conn(0x7fa4d40769e0 msgr2=0x7fa4d4078320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:17:38.147 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:38.144+0000 7fa4db800700 1 -- 172.21.15.153:0/1324259564 shutdown_connections 2024-03-21T10:17:38.147 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:38.144+0000 7fa4db800700 1 -- 172.21.15.153:0/1324259564 wait complete. 2024-03-21T10:17:38.293 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:38.291+0000 7ff742c00700 1 -- 172.21.15.153:0/150570480 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff73c1069c0 msgr2=0x7ff73c106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:17:38.293 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:38.291+0000 7ff742c00700 1 --2- 172.21.15.153:0/150570480 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff73c1069c0 0x7ff73c106da0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff730009910 tx=0x7ff730009c20 comp rx=0 tx=0).stop 2024-03-21T10:17:38.293 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:38.291+0000 7ff742c00700 1 -- 172.21.15.153:0/150570480 shutdown_connections 2024-03-21T10:17:38.293 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:38.291+0000 7ff742c00700 1 --2- 172.21.15.153:0/150570480 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff73c1009c0 0x7ff73c100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:17:38.293 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:38.291+0000 7ff742c00700 1 --2- 172.21.15.153:0/150570480 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff73c1069c0 0x7ff73c106da0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:17:38.293 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:38.291+0000 7ff742c00700 1 -- 172.21.15.153:0/150570480 >> 172.21.15.153:0/150570480 conn(0x7ff73c0fc4f0 msgr2=0x7ff73c0fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:17:38.293 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:38.291+0000 7ff742c00700 1 -- 172.21.15.153:0/150570480 shutdown_connections 2024-03-21T10:17:38.293 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:38.291+0000 7ff742c00700 1 -- 172.21.15.153:0/150570480 wait complete. 2024-03-21T10:17:38.294 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:38.292+0000 7ff742c00700 1 Processor -- start 2024-03-21T10:17:38.294 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:38.292+0000 7ff742c00700 1 -- start start 2024-03-21T10:17:38.295 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:38.293+0000 7ff742c00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff73c1009c0 0x7ff73c19e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:17:38.295 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:38.293+0000 7ff742c00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff73c1069c0 0x7ff73c19eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:17:38.295 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:38.293+0000 7ff742c00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff73c198640 con 0x7ff73c1069c0 2024-03-21T10:17:38.295 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:38.293+0000 7ff742c00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff73c1987b0 con 0x7ff73c1009c0 2024-03-21T10:17:38.295 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:38.293+0000 7ff740e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff73c1069c0 0x7ff73c19eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:17:38.295 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:38.293+0000 7ff740e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff73c1069c0 0x7ff73c19eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:49662/0 (socket says 172.21.15.153:49662) 2024-03-21T10:17:38.295 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:38.293+0000 7ff740e00700 1 -- 172.21.15.153:0/3215658308 learned_addr learned my addr 172.21.15.153:0/3215658308 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:17:38.296 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:38.294+0000 7ff741800700 1 --2- 172.21.15.153:0/3215658308 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff73c1009c0 0x7ff73c19e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:17:38.296 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:38.294+0000 7ff740e00700 1 -- 172.21.15.153:0/3215658308 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff73c1009c0 msgr2=0x7ff73c19e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:17:38.296 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:38.294+0000 7ff740e00700 1 --2- 172.21.15.153:0/3215658308 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff73c1009c0 0x7ff73c19e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:17:38.297 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:38.294+0000 7ff740e00700 1 -- 172.21.15.153:0/3215658308 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7300095f0 con 0x7ff73c1069c0 2024-03-21T10:17:38.297 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:38.294+0000 7ff740e00700 1 --2- 172.21.15.153:0/3215658308 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff73c1069c0 0x7ff73c19eb90 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff72800d900 tx=0x7ff72800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:17:38.297 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:38.294+0000 7ff73a000700 1 -- 172.21.15.153:0/3215658308 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff728004a00 con 0x7ff73c1069c0 2024-03-21T10:17:38.297 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:38.294+0000 7ff73a000700 1 -- 172.21.15.153:0/3215658308 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff7280054e0 con 0x7ff73c1069c0 2024-03-21T10:17:38.297 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:38.294+0000 7ff73a000700 1 -- 172.21.15.153:0/3215658308 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff728009d60 con 0x7ff73c1069c0 2024-03-21T10:17:38.297 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:38.294+0000 7ff742c00700 1 -- 172.21.15.153:0/3215658308 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff73c198a90 con 0x7ff73c1069c0 2024-03-21T10:17:38.297 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:38.294+0000 7ff742c00700 1 -- 172.21.15.153:0/3215658308 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff73c198f90 con 0x7ff73c1069c0 2024-03-21T10:17:38.297 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:38.296+0000 7ff742c00700 1 -- 172.21.15.153:0/3215658308 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff73c04f2e0 con 0x7ff73c1069c0 2024-03-21T10:17:38.298 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:38.297+0000 7ff73a000700 1 -- 172.21.15.153:0/3215658308 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7ff728005020 con 0x7ff73c1069c0 2024-03-21T10:17:38.299 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:38.297+0000 7ff73a000700 1 --2- 172.21.15.153:0/3215658308 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff72406e460 0x7ff724070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:17:38.299 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:38.298+0000 7ff73a000700 1 -- 172.21.15.153:0/3215658308 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6654+0+0 (secure 0 0 0) 0x7ff72808e6d0 con 0x7ff73c1069c0 2024-03-21T10:17:38.299 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:38.298+0000 7ff741800700 1 --2- 172.21.15.153:0/3215658308 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff72406e460 0x7ff724070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:17:38.300 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:38.299+0000 7ff741800700 1 --2- 172.21.15.153:0/3215658308 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff72406e460 0x7ff724070920 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff73000b5c0 tx=0x7ff73001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:17:38.304 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:38.303+0000 7ff73a000700 1 -- 172.21.15.153:0/3215658308 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff7280577a0 con 0x7ff73c1069c0 2024-03-21T10:17:38.539 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:38.537+0000 7ff742c00700 1 -- 172.21.15.153:0/3215658308 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7ff73c1992e0 con 0x7ff72406e460 2024-03-21T10:17:38.539 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:38 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:17:38.540 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:38 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:17:38.540 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:38 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:17:38.540 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:38 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:17:38.540 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:38 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:17:38.540 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:38 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:17:38.540 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:38 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:17:38.540 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:38 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:17:38.540 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:38 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:17:38.540 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:38 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-21T10:17:38.540 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:38 smithi153 ceph-mon[113733]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-21T10:17:38.540 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:38 smithi153 ceph-mon[113733]: Upgrade: unsafe to stop osd(s) at this time (9 PGs are or would become offline) 2024-03-21T10:17:38.540 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:38 smithi153 ceph-mon[113733]: from='client.34338 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:17:38.540 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:38 smithi153 ceph-mon[113733]: from='client.34342 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:17:38.558 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:38.556+0000 7ff73a000700 1 -- 172.21.15.153:0/3215658308 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7ff73c1992e0 con 0x7ff72406e460 2024-03-21T10:17:38.558 INFO:teuthology.orchestra.run.smithi153.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-21T10:17:38.558 INFO:teuthology.orchestra.run.smithi153.stdout:alertmanager.smithi153 smithi153 *:9093,9094 running (6m) 35s ago 20m 14.5M - 0.25.0 c8568f914cd2 9ed3f2200b5f 2024-03-21T10:17:38.558 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi153 smithi153 running (20m) 35s ago 20m 8988k - 18.2.1 5be31c24972a 5b61994396a6 2024-03-21T10:17:38.558 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi174 smithi174 running (18m) 8s ago 18m 8539k - 18.2.1 5be31c24972a 4f7a5fc3ac28 2024-03-21T10:17:38.559 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi153 smithi153 running (2m) 35s ago 20m 6656k - 19.0.0-2231-g2edfee68 237e43c7a65f 32c21104d5b0 2024-03-21T10:17:38.559 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi174 smithi174 running (2m) 8s ago 18m 6660k - 19.0.0-2231-g2edfee68 237e43c7a65f a04e48967ab6 2024-03-21T10:17:38.559 INFO:teuthology.orchestra.run.smithi153.stdout:grafana.smithi153 smithi153 *:3000 running (5m) 35s ago 19m 75.4M - 9.4.12 2bacad6d85d8 ee24026ab786 2024-03-21T10:17:38.559 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.iuyluj smithi153 running (12m) 35s ago 12m 128M - 18.2.1 5be31c24972a 02f667cee979 2024-03-21T10:17:38.559 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.zipckr smithi153 running (13m) 35s ago 13m 58.8M - 18.2.1 5be31c24972a 68a853929f8f 2024-03-21T10:17:38.559 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.nhozag smithi174 running (12m) 8s ago 12m 57.5M - 18.2.1 5be31c24972a 2fee4000308f 2024-03-21T10:17:38.559 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.yyimst smithi174 running (13m) 8s ago 13m 117M - 18.2.1 5be31c24972a d78a729220d1 2024-03-21T10:17:38.559 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi153.tluxni smithi153 *:8443,9283,8765 running (9m) 35s ago 22m 505M - 19.0.0-2231-g2edfee68 237e43c7a65f 7ffee0e55382 2024-03-21T10:17:38.559 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi174.qpcwat smithi174 *:8443,9283,8765 running (8m) 8s ago 18m 451M - 19.0.0-2231-g2edfee68 237e43c7a65f 748968f216ab 2024-03-21T10:17:38.559 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi153 smithi153 running (4m) 35s ago 22m 48.2M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f dd8038303dc3 2024-03-21T10:17:38.559 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi174 smithi174 running (3m) 8s ago 18m 41.9M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f 29d78a788f2c 2024-03-21T10:17:38.559 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi153 smithi153 *:9100 running (7m) 35s ago 20m 8284k - 1.5.0 0da6a335fe13 bb835edc0e0b 2024-03-21T10:17:38.559 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi174 smithi174 *:9100 running (7m) 8s ago 18m 7834k - 1.5.0 0da6a335fe13 7c4ecad790fa 2024-03-21T10:17:38.559 INFO:teuthology.orchestra.run.smithi153.stdout:osd.0 smithi153 running (2m) 35s ago 17m 119M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 507900fd4791 2024-03-21T10:17:38.559 INFO:teuthology.orchestra.run.smithi153.stdout:osd.1 smithi153 running (75s) 35s ago 17m 86.6M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 0d646452e9c1 2024-03-21T10:17:38.560 INFO:teuthology.orchestra.run.smithi153.stdout:osd.2 smithi153 running (44s) 35s ago 16m 12.6M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 1326f2afdfcd 2024-03-21T10:17:38.560 INFO:teuthology.orchestra.run.smithi153.stdout:osd.3 smithi174 running (13s) 8s ago 15m 11.1M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 70aafac71ad5 2024-03-21T10:17:38.560 INFO:teuthology.orchestra.run.smithi153.stdout:osd.4 smithi174 running (15m) 8s ago 15m 230M 2241M 18.2.1 5be31c24972a c4cd15701f05 2024-03-21T10:17:38.560 INFO:teuthology.orchestra.run.smithi153.stdout:osd.5 smithi174 running (14m) 8s ago 14m 213M 2241M 18.2.1 5be31c24972a 3b69886ce22e 2024-03-21T10:17:38.560 INFO:teuthology.orchestra.run.smithi153.stdout:prometheus.smithi153 smithi153 *:9095 running (6m) 35s ago 19m 72.2M - 2.43.0 a07b618ecd1d 7d95561e50c2 2024-03-21T10:17:38.561 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:38.559+0000 7ff742c00700 1 -- 172.21.15.153:0/3215658308 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff72406e460 msgr2=0x7ff724070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:17:38.561 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:38.559+0000 7ff742c00700 1 --2- 172.21.15.153:0/3215658308 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff72406e460 0x7ff724070920 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff73000b5c0 tx=0x7ff73001a040 comp rx=0 tx=0).stop 2024-03-21T10:17:38.561 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:38.559+0000 7ff742c00700 1 -- 172.21.15.153:0/3215658308 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff73c1069c0 msgr2=0x7ff73c19eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:17:38.561 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:38.559+0000 7ff742c00700 1 --2- 172.21.15.153:0/3215658308 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff73c1069c0 0x7ff73c19eb90 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff72800d900 tx=0x7ff72800dcc0 comp rx=0 tx=0).stop 2024-03-21T10:17:38.561 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:38.559+0000 7ff742c00700 1 -- 172.21.15.153:0/3215658308 shutdown_connections 2024-03-21T10:17:38.561 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:38.559+0000 7ff742c00700 1 --2- 172.21.15.153:0/3215658308 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff73c1009c0 0x7ff73c19e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:17:38.562 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:38.559+0000 7ff742c00700 1 --2- 172.21.15.153:0/3215658308 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff72406e460 0x7ff724070920 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:17:38.562 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:38.559+0000 7ff742c00700 1 --2- 172.21.15.153:0/3215658308 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff73c1069c0 0x7ff73c19eb90 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:17:38.562 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:38.559+0000 7ff742c00700 1 -- 172.21.15.153:0/3215658308 >> 172.21.15.153:0/3215658308 conn(0x7ff73c0fc4f0 msgr2=0x7ff73c072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:17:38.562 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:38.559+0000 7ff742c00700 1 -- 172.21.15.153:0/3215658308 shutdown_connections 2024-03-21T10:17:38.562 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:38.560+0000 7ff742c00700 1 -- 172.21.15.153:0/3215658308 wait complete. 2024-03-21T10:17:38.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:38 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:17:38.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:38 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:17:38.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:38 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:17:38.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:38 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:17:38.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:38 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:17:38.703 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:38 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:17:38.703 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:38 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:17:38.703 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:38 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:17:38.703 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:38 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:17:38.703 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:38 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-21T10:17:38.703 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:38 smithi174 ceph-mon[88004]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-21T10:17:38.703 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:38 smithi174 ceph-mon[88004]: Upgrade: unsafe to stop osd(s) at this time (9 PGs are or would become offline) 2024-03-21T10:17:38.703 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:38 smithi174 ceph-mon[88004]: from='client.34338 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:17:38.703 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:38 smithi174 ceph-mon[88004]: from='client.34342 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:17:38.705 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:38.700+0000 7f73b4a00700 1 -- 172.21.15.153:0/1899416604 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f73b01039b0 msgr2=0x7f73b0107a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:17:38.705 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:38.700+0000 7f73b4a00700 1 --2- 172.21.15.153:0/1899416604 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f73b01039b0 0x7f73b0107a00 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f7398009910 tx=0x7f7398009c20 comp rx=0 tx=0).stop 2024-03-21T10:17:38.705 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:38.700+0000 7f73b4a00700 1 -- 172.21.15.153:0/1899416604 shutdown_connections 2024-03-21T10:17:38.705 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:38.700+0000 7f73b4a00700 1 --2- 172.21.15.153:0/1899416604 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f73b01039b0 0x7f73b0107a00 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:17:38.705 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:38.700+0000 7f73b4a00700 1 --2- 172.21.15.153:0/1899416604 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f73b0103000 0x7f73b01033e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:17:38.705 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:38.700+0000 7f73b4a00700 1 -- 172.21.15.153:0/1899416604 >> 172.21.15.153:0/1899416604 conn(0x7f73b00fe8d0 msgr2=0x7f73b0100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:17:38.706 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:38.700+0000 7f73b4a00700 1 -- 172.21.15.153:0/1899416604 shutdown_connections 2024-03-21T10:17:38.706 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:38.700+0000 7f73b4a00700 1 -- 172.21.15.153:0/1899416604 wait complete. 2024-03-21T10:17:38.706 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:38.701+0000 7f73b4a00700 1 Processor -- start 2024-03-21T10:17:38.706 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:38.701+0000 7f73b4a00700 1 -- start start 2024-03-21T10:17:38.706 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:38.702+0000 7f73b4a00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f73b0103000 0x7f73b019e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:17:38.706 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:38.702+0000 7f73b4a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f73b01039b0 0x7f73b019ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:17:38.706 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:38.702+0000 7f73b4a00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73b019f490 con 0x7f73b01039b0 2024-03-21T10:17:38.706 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:38.702+0000 7f73b4a00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73b0198940 con 0x7f73b0103000 2024-03-21T10:17:38.706 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:38.702+0000 7f73a7e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f73b01039b0 0x7f73b019ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:17:38.706 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:38.702+0000 7f73a7e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f73b01039b0 0x7f73b019ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:49672/0 (socket says 172.21.15.153:49672) 2024-03-21T10:17:38.706 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:38.702+0000 7f73a7e00700 1 -- 172.21.15.153:0/1104631026 learned_addr learned my addr 172.21.15.153:0/1104631026 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:17:38.706 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:38.702+0000 7f73ade00700 1 --2- 172.21.15.153:0/1104631026 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f73b0103000 0x7f73b019e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:17:38.706 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:38.702+0000 7f73a7e00700 1 -- 172.21.15.153:0/1104631026 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f73b0103000 msgr2=0x7f73b019e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:17:38.707 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:38.702+0000 7f73a7e00700 1 --2- 172.21.15.153:0/1104631026 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f73b0103000 0x7f73b019e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:17:38.707 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:38.702+0000 7f73a7e00700 1 -- 172.21.15.153:0/1104631026 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73980095f0 con 0x7f73b01039b0 2024-03-21T10:17:38.707 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:38.703+0000 7f73a7e00700 1 --2- 172.21.15.153:0/1104631026 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f73b01039b0 0x7f73b019ee00 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f739800bda0 tx=0x7f739800be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:17:38.707 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:38.703+0000 7f73a6a00700 1 -- 172.21.15.153:0/1104631026 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f739801d070 con 0x7f73b01039b0 2024-03-21T10:17:38.707 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:38.703+0000 7f73a6a00700 1 -- 172.21.15.153:0/1104631026 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7398004fb0 con 0x7f73b01039b0 2024-03-21T10:17:38.707 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:38.703+0000 7f73a6a00700 1 -- 172.21.15.153:0/1104631026 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f739800f460 con 0x7f73b01039b0 2024-03-21T10:17:38.707 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:38.703+0000 7f73b4a00700 1 -- 172.21.15.153:0/1104631026 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73b0198bc0 con 0x7f73b01039b0 2024-03-21T10:17:38.707 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:38.703+0000 7f73b4a00700 1 -- 172.21.15.153:0/1104631026 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73b01990b0 con 0x7f73b01039b0 2024-03-21T10:17:38.707 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:38.704+0000 7f73b4a00700 1 -- 172.21.15.153:0/1104631026 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73b010b350 con 0x7f73b01039b0 2024-03-21T10:17:38.708 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:38.705+0000 7f73a6a00700 1 -- 172.21.15.153:0/1104631026 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f7398022800 con 0x7f73b01039b0 2024-03-21T10:17:38.708 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:38.707+0000 7f73a6a00700 1 --2- 172.21.15.153:0/1104631026 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f739406e460 0x7f7394070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:17:38.708 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:38.707+0000 7f73a6a00700 1 -- 172.21.15.153:0/1104631026 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6654+0+0 (secure 0 0 0) 0x7f739808fa40 con 0x7f73b01039b0 2024-03-21T10:17:38.708 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:38.707+0000 7f73ade00700 1 --2- 172.21.15.153:0/1104631026 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f739406e460 0x7f7394070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:17:38.709 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:38.708+0000 7f73ade00700 1 --2- 172.21.15.153:0/1104631026 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f739406e460 0x7f7394070920 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f73a0006fd0 tx=0x7f73a0008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:17:38.713 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:38.711+0000 7f73a6a00700 1 -- 172.21.15.153:0/1104631026 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f739805c810 con 0x7f73b01039b0 2024-03-21T10:17:39.021 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:39.019+0000 7f73b4a00700 1 -- 172.21.15.153:0/1104631026 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f73b004ea90 con 0x7f73b01039b0 2024-03-21T10:17:39.022 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:39.020+0000 7f73a6a00700 1 -- 172.21.15.153:0/1104631026 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+791 (secure 0 0 0) 0x7f739805c3a0 con 0x7f73b01039b0 2024-03-21T10:17:39.023 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:17:39.023 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": { 2024-03-21T10:17:39.023 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:17:39.023 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:17:39.023 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": { 2024-03-21T10:17:39.023 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:17:39.023 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:17:39.023 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": { 2024-03-21T10:17:39.023 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2, 2024-03-21T10:17:39.023 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 4 2024-03-21T10:17:39.023 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:17:39.023 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": { 2024-03-21T10:17:39.023 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-03-21T10:17:39.023 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:17:39.024 INFO:teuthology.orchestra.run.smithi153.stdout: "overall": { 2024-03-21T10:17:39.024 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6, 2024-03-21T10:17:39.024 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 8 2024-03-21T10:17:39.024 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-03-21T10:17:39.024 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:17:39.025 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:39.023+0000 7f73b4a00700 1 -- 172.21.15.153:0/1104631026 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f739406e460 msgr2=0x7f7394070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:17:39.025 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:39.023+0000 7f73b4a00700 1 --2- 172.21.15.153:0/1104631026 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f739406e460 0x7f7394070920 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f73a0006fd0 tx=0x7f73a0008040 comp rx=0 tx=0).stop 2024-03-21T10:17:39.025 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:39.023+0000 7f73b4a00700 1 -- 172.21.15.153:0/1104631026 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f73b01039b0 msgr2=0x7f73b019ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:17:39.025 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:39.023+0000 7f73b4a00700 1 --2- 172.21.15.153:0/1104631026 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f73b01039b0 0x7f73b019ee00 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f739800bda0 tx=0x7f739800be80 comp rx=0 tx=0).stop 2024-03-21T10:17:39.025 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:39.023+0000 7f73b4a00700 1 -- 172.21.15.153:0/1104631026 shutdown_connections 2024-03-21T10:17:39.025 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:39.023+0000 7f73b4a00700 1 --2- 172.21.15.153:0/1104631026 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f73b0103000 0x7f73b019e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:17:39.025 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:39.023+0000 7f73b4a00700 1 --2- 172.21.15.153:0/1104631026 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f739406e460 0x7f7394070920 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:17:39.025 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:39.023+0000 7f73b4a00700 1 --2- 172.21.15.153:0/1104631026 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f73b01039b0 0x7f73b019ee00 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:17:39.026 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:39.023+0000 7f73b4a00700 1 -- 172.21.15.153:0/1104631026 >> 172.21.15.153:0/1104631026 conn(0x7f73b00fe8d0 msgr2=0x7f73b00fffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:17:39.026 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:39.023+0000 7f73b4a00700 1 -- 172.21.15.153:0/1104631026 shutdown_connections 2024-03-21T10:17:39.026 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:39.024+0000 7f73b4a00700 1 -- 172.21.15.153:0/1104631026 wait complete. 2024-03-21T10:17:39.178 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:39.176+0000 7f72e3c00700 1 -- 172.21.15.153:0/1167296731 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72dc101d10 msgr2=0x7f72dc1020f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:17:39.178 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:39.176+0000 7f72e3c00700 1 --2- 172.21.15.153:0/1167296731 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72dc101d10 0x7f72dc1020f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f72d0009910 tx=0x7f72d0009c20 comp rx=0 tx=0).stop 2024-03-21T10:17:39.178 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:39.176+0000 7f72e3c00700 1 -- 172.21.15.153:0/1167296731 shutdown_connections 2024-03-21T10:17:39.178 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:39.176+0000 7f72e3c00700 1 --2- 172.21.15.153:0/1167296731 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f72dc102630 0x7f72dc10ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:17:39.178 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:39.176+0000 7f72e3c00700 1 --2- 172.21.15.153:0/1167296731 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72dc101d10 0x7f72dc1020f0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:17:39.178 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:39.176+0000 7f72e3c00700 1 -- 172.21.15.153:0/1167296731 >> 172.21.15.153:0/1167296731 conn(0x7f72dc0fb540 msgr2=0x7f72dc0fd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:17:39.179 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:39.176+0000 7f72e3c00700 1 -- 172.21.15.153:0/1167296731 shutdown_connections 2024-03-21T10:17:39.179 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:39.176+0000 7f72e3c00700 1 -- 172.21.15.153:0/1167296731 wait complete. 2024-03-21T10:17:39.180 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:39.177+0000 7f72e3c00700 1 Processor -- start 2024-03-21T10:17:39.180 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:39.177+0000 7f72e3c00700 1 -- start start 2024-03-21T10:17:39.180 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:39.178+0000 7f72e3c00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f72dc101d10 0x7f72dc0ff970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:17:39.180 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:39.178+0000 7f72e3c00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72dc102630 0x7f72dc0ffeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:17:39.180 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:39.178+0000 7f72e3c00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72dc1003f0 con 0x7f72dc102630 2024-03-21T10:17:39.180 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:39.178+0000 7f72e3c00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72dc100530 con 0x7f72dc101d10 2024-03-21T10:17:39.180 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:39.178+0000 7f72dbe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72dc102630 0x7f72dc0ffeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:17:39.180 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:39.178+0000 7f72dbe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72dc102630 0x7f72dc0ffeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:49690/0 (socket says 172.21.15.153:49690) 2024-03-21T10:17:39.180 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:39.178+0000 7f72dbe00700 1 -- 172.21.15.153:0/4275279380 learned_addr learned my addr 172.21.15.153:0/4275279380 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:17:39.180 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:39.178+0000 7f72e1200700 1 --2- 172.21.15.153:0/4275279380 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f72dc101d10 0x7f72dc0ff970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:17:39.181 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:39.178+0000 7f72dbe00700 1 -- 172.21.15.153:0/4275279380 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f72dc101d10 msgr2=0x7f72dc0ff970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:17:39.181 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:39.178+0000 7f72dbe00700 1 --2- 172.21.15.153:0/4275279380 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f72dc101d10 0x7f72dc0ff970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:17:39.181 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:39.178+0000 7f72dbe00700 1 -- 172.21.15.153:0/4275279380 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72d00095f0 con 0x7f72dc102630 2024-03-21T10:17:39.181 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:39.179+0000 7f72e1200700 1 --2- 172.21.15.153:0/4275279380 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f72dc101d10 0x7f72dc0ff970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:17:39.181 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:39.179+0000 7f72dbe00700 1 --2- 172.21.15.153:0/4275279380 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72dc102630 0x7f72dc0ffeb0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f72c800d8d0 tx=0x7f72c800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:17:39.182 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:39.179+0000 7f72d9600700 1 -- 172.21.15.153:0/4275279380 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f72c80041f0 con 0x7f72dc102630 2024-03-21T10:17:39.182 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:39.179+0000 7f72d9600700 1 -- 172.21.15.153:0/4275279380 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f72c8004350 con 0x7f72dc102630 2024-03-21T10:17:39.182 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:39.179+0000 7f72e3c00700 1 -- 172.21.15.153:0/4275279380 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72dc1a2910 con 0x7f72dc102630 2024-03-21T10:17:39.182 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:39.179+0000 7f72d9600700 1 -- 172.21.15.153:0/4275279380 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f72c8003d60 con 0x7f72dc102630 2024-03-21T10:17:39.182 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:39.179+0000 7f72e3c00700 1 -- 172.21.15.153:0/4275279380 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72dc1a2d20 con 0x7f72dc102630 2024-03-21T10:17:39.182 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:39.180+0000 7f72e3c00700 1 -- 172.21.15.153:0/4275279380 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72dc108220 con 0x7f72dc102630 2024-03-21T10:17:39.183 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:39.181+0000 7f72d9600700 1 -- 172.21.15.153:0/4275279380 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f72c8009730 con 0x7f72dc102630 2024-03-21T10:17:39.184 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:39.182+0000 7f72d9600700 1 --2- 172.21.15.153:0/4275279380 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f72c406e460 0x7f72c4070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:17:39.184 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:39.182+0000 7f72d9600700 1 -- 172.21.15.153:0/4275279380 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6654+0+0 (secure 0 0 0) 0x7f72c808dfc0 con 0x7f72dc102630 2024-03-21T10:17:39.184 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:39.182+0000 7f72e1200700 1 --2- 172.21.15.153:0/4275279380 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f72c406e460 0x7f72c4070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:17:39.184 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:39.183+0000 7f72e1200700 1 --2- 172.21.15.153:0/4275279380 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f72c406e460 0x7f72c4070920 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f72d0006010 tx=0x7f72d000b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:17:39.188 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:39.187+0000 7f72d9600700 1 -- 172.21.15.153:0/4275279380 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f72c80571b0 con 0x7f72dc102630 2024-03-21T10:17:39.449 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:39.447+0000 7f72e3c00700 1 -- 172.21.15.153:0/4275279380 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f72dc04ea90 con 0x7f72dc102630 2024-03-21T10:17:39.450 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:39.448+0000 7f72d9600700 1 -- 172.21.15.153:0/4275279380 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 21 v21) v1 ==== 76+0+1903 (secure 0 0 0) 0x7f72c8017780 con 0x7f72dc102630 2024-03-21T10:17:39.451 INFO:teuthology.orchestra.run.smithi153.stdout:e21 2024-03-21T10:17:39.451 INFO:teuthology.orchestra.run.smithi153.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-21T10:17:39.451 INFO:teuthology.orchestra.run.smithi153.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:17:39.451 INFO:teuthology.orchestra.run.smithi153.stdout:legacy client fscid: 1 2024-03-21T10:17:39.451 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:17:39.451 INFO:teuthology.orchestra.run.smithi153.stdout:Filesystem 'cephfs' (1) 2024-03-21T10:17:39.451 INFO:teuthology.orchestra.run.smithi153.stdout:fs_name cephfs 2024-03-21T10:17:39.451 INFO:teuthology.orchestra.run.smithi153.stdout:epoch 21 2024-03-21T10:17:39.451 INFO:teuthology.orchestra.run.smithi153.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-21T10:17:39.451 INFO:teuthology.orchestra.run.smithi153.stdout:created 2024-03-21T10:04:27.086576+0000 2024-03-21T10:17:39.451 INFO:teuthology.orchestra.run.smithi153.stdout:modified 2024-03-21T10:13:01.455594+0000 2024-03-21T10:17:39.451 INFO:teuthology.orchestra.run.smithi153.stdout:tableserver 0 2024-03-21T10:17:39.451 INFO:teuthology.orchestra.run.smithi153.stdout:root 0 2024-03-21T10:17:39.452 INFO:teuthology.orchestra.run.smithi153.stdout:session_timeout 60 2024-03-21T10:17:39.452 INFO:teuthology.orchestra.run.smithi153.stdout:session_autoclose 300 2024-03-21T10:17:39.452 INFO:teuthology.orchestra.run.smithi153.stdout:max_file_size 1099511627776 2024-03-21T10:17:39.452 INFO:teuthology.orchestra.run.smithi153.stdout:max_xattr_size 65536 2024-03-21T10:17:39.452 INFO:teuthology.orchestra.run.smithi153.stdout:required_client_features {} 2024-03-21T10:17:39.452 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure 0 2024-03-21T10:17:39.452 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure_osd_epoch 47 2024-03-21T10:17:39.452 INFO:teuthology.orchestra.run.smithi153.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:17:39.452 INFO:teuthology.orchestra.run.smithi153.stdout:max_mds 2 2024-03-21T10:17:39.452 INFO:teuthology.orchestra.run.smithi153.stdout:in 0,1 2024-03-21T10:17:39.452 INFO:teuthology.orchestra.run.smithi153.stdout:up {0=14484,1=24273} 2024-03-21T10:17:39.452 INFO:teuthology.orchestra.run.smithi153.stdout:failed 2024-03-21T10:17:39.452 INFO:teuthology.orchestra.run.smithi153.stdout:damaged 2024-03-21T10:17:39.452 INFO:teuthology.orchestra.run.smithi153.stdout:stopped 2024-03-21T10:17:39.452 INFO:teuthology.orchestra.run.smithi153.stdout:data_pools [3] 2024-03-21T10:17:39.453 INFO:teuthology.orchestra.run.smithi153.stdout:metadata_pool 2 2024-03-21T10:17:39.453 INFO:teuthology.orchestra.run.smithi153.stdout:inline_data enabled 2024-03-21T10:17:39.453 INFO:teuthology.orchestra.run.smithi153.stdout:balancer 2024-03-21T10:17:39.453 INFO:teuthology.orchestra.run.smithi153.stdout:bal_rank_mask -1 2024-03-21T10:17:39.453 INFO:teuthology.orchestra.run.smithi153.stdout:standby_count_wanted 1 2024-03-21T10:17:39.453 INFO:teuthology.orchestra.run.smithi153.stdout:qdb_cluster leader: 0 members: 2024-03-21T10:17:39.453 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.iuyluj{0:14484} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.153:6828/866138999,v1:172.21.15.153:6829/866138999] compat {c=[1],r=[1],i=[7ff]}] 2024-03-21T10:17:39.453 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.zipckr{0:14494} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.153:6826/4033622615,v1:172.21.15.153:6827/4033622615] compat {c=[1],r=[1],i=[7ff]}] 2024-03-21T10:17:39.453 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.yyimst{1:24273} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.174:6824/1776947618,v1:172.21.15.174:6825/1776947618] compat {c=[1],r=[1],i=[7ff]}] 2024-03-21T10:17:39.453 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.nhozag{1:24285} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.174:6826/806095873,v1:172.21.15.174:6827/806095873] compat {c=[1],r=[1],i=[7ff]}] 2024-03-21T10:17:39.453 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:17:39.453 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:17:39.454 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:39.451+0000 7f72e3c00700 1 -- 172.21.15.153:0/4275279380 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f72c406e460 msgr2=0x7f72c4070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:17:39.454 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:39.451+0000 7f72e3c00700 1 --2- 172.21.15.153:0/4275279380 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f72c406e460 0x7f72c4070920 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f72d0006010 tx=0x7f72d000b540 comp rx=0 tx=0).stop 2024-03-21T10:17:39.454 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:39.451+0000 7f72e3c00700 1 -- 172.21.15.153:0/4275279380 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72dc102630 msgr2=0x7f72dc0ffeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:17:39.454 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:39.451+0000 7f72e3c00700 1 --2- 172.21.15.153:0/4275279380 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72dc102630 0x7f72dc0ffeb0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f72c800d8d0 tx=0x7f72c800dc90 comp rx=0 tx=0).stop 2024-03-21T10:17:39.454 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:39.451+0000 7f72e3c00700 1 -- 172.21.15.153:0/4275279380 shutdown_connections 2024-03-21T10:17:39.454 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:39.451+0000 7f72e3c00700 1 --2- 172.21.15.153:0/4275279380 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f72dc101d10 0x7f72dc0ff970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:17:39.454 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:39.451+0000 7f72e3c00700 1 --2- 172.21.15.153:0/4275279380 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f72c406e460 0x7f72c4070920 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:17:39.455 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:39.451+0000 7f72e3c00700 1 --2- 172.21.15.153:0/4275279380 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72dc102630 0x7f72dc0ffeb0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:17:39.455 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:39.451+0000 7f72e3c00700 1 -- 172.21.15.153:0/4275279380 >> 172.21.15.153:0/4275279380 conn(0x7f72dc0fb540 msgr2=0x7f72dc105360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:17:39.455 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:39.451+0000 7f72e3c00700 1 -- 172.21.15.153:0/4275279380 shutdown_connections 2024-03-21T10:17:39.455 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:39.451+0000 7f72e3c00700 1 -- 172.21.15.153:0/4275279380 wait complete. 2024-03-21T10:17:39.455 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 21 2024-03-21T10:17:39.608 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:39.605+0000 7fcec6800700 1 -- 172.21.15.153:0/2766520627 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcec00fee00 msgr2=0x7fcec00ff1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:17:39.608 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:39.605+0000 7fcec6800700 1 --2- 172.21.15.153:0/2766520627 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcec00fee00 0x7fcec00ff1e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fceb4009910 tx=0x7fceb4009c20 comp rx=0 tx=0).stop 2024-03-21T10:17:39.608 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:39.606+0000 7fcec6800700 1 -- 172.21.15.153:0/2766520627 shutdown_connections 2024-03-21T10:17:39.608 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:39.606+0000 7fcec6800700 1 --2- 172.21.15.153:0/2766520627 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcec00ff7b0 0x7fcec0103800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:17:39.608 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:39.606+0000 7fcec6800700 1 --2- 172.21.15.153:0/2766520627 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcec00fee00 0x7fcec00ff1e0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:17:39.608 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:39.606+0000 7fcec6800700 1 -- 172.21.15.153:0/2766520627 >> 172.21.15.153:0/2766520627 conn(0x7fcec00fa690 msgr2=0x7fcec00fcab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:17:39.608 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:39.606+0000 7fcec6800700 1 -- 172.21.15.153:0/2766520627 shutdown_connections 2024-03-21T10:17:39.608 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:39.606+0000 7fcec6800700 1 -- 172.21.15.153:0/2766520627 wait complete. 2024-03-21T10:17:39.609 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:39.607+0000 7fcec6800700 1 Processor -- start 2024-03-21T10:17:39.609 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:39.607+0000 7fcec6800700 1 -- start start 2024-03-21T10:17:39.609 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:39.607+0000 7fcec6800700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcec00fee00 0x7fcec01008a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:17:39.609 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:39.607+0000 7fcec6800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcec00ff7b0 0x7fcec0100de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:17:39.610 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:39.607+0000 7fcec6800700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcec0101320 con 0x7fcec00ff7b0 2024-03-21T10:17:39.610 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:39.608+0000 7fcec6800700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcec0101460 con 0x7fcec00fee00 2024-03-21T10:17:39.610 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:39.608+0000 7fcec4a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcec00ff7b0 0x7fcec0100de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:17:39.610 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:39.608+0000 7fcec4a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcec00ff7b0 0x7fcec0100de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:49714/0 (socket says 172.21.15.153:49714) 2024-03-21T10:17:39.610 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:39.608+0000 7fcec4a00700 1 -- 172.21.15.153:0/3080142429 learned_addr learned my addr 172.21.15.153:0/3080142429 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:17:39.611 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:39.608+0000 7fcec5400700 1 --2- 172.21.15.153:0/3080142429 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcec00fee00 0x7fcec01008a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:17:39.611 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:39.608+0000 7fcec4a00700 1 -- 172.21.15.153:0/3080142429 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcec00fee00 msgr2=0x7fcec01008a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:17:39.611 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:39.608+0000 7fcec4a00700 1 --2- 172.21.15.153:0/3080142429 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcec00fee00 0x7fcec01008a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:17:39.611 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:39.608+0000 7fcec4a00700 1 -- 172.21.15.153:0/3080142429 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fceb40095f0 con 0x7fcec00ff7b0 2024-03-21T10:17:39.611 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:39.609+0000 7fcec4a00700 1 --2- 172.21.15.153:0/3080142429 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcec00ff7b0 0x7fcec0100de0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fceac00d900 tx=0x7fceac00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:17:39.611 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:39.609+0000 7fcebe000700 1 -- 172.21.15.153:0/3080142429 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fceac009900 con 0x7fcec00ff7b0 2024-03-21T10:17:39.611 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:39.609+0000 7fcebe000700 1 -- 172.21.15.153:0/3080142429 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fceac010460 con 0x7fcec00ff7b0 2024-03-21T10:17:39.611 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:39.609+0000 7fcec6800700 1 -- 172.21.15.153:0/3080142429 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcec01a2740 con 0x7fcec00ff7b0 2024-03-21T10:17:39.611 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:39.609+0000 7fcebe000700 1 -- 172.21.15.153:0/3080142429 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fceac00f5f0 con 0x7fcec00ff7b0 2024-03-21T10:17:39.611 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:39.609+0000 7fcec6800700 1 -- 172.21.15.153:0/3080142429 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcec01a2b50 con 0x7fcec00ff7b0 2024-03-21T10:17:39.612 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:39.610+0000 7fcec6800700 1 -- 172.21.15.153:0/3080142429 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcec01091b0 con 0x7fcec00ff7b0 2024-03-21T10:17:39.612 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:39.611+0000 7fcebe000700 1 -- 172.21.15.153:0/3080142429 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fceac0105d0 con 0x7fcec00ff7b0 2024-03-21T10:17:39.614 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:39.612+0000 7fcebe000700 1 --2- 172.21.15.153:0/3080142429 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fcea806e460 0x7fcea8070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:17:39.614 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:39.612+0000 7fcebe000700 1 -- 172.21.15.153:0/3080142429 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6654+0+0 (secure 0 0 0) 0x7fceac08e2c0 con 0x7fcec00ff7b0 2024-03-21T10:17:39.614 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:39.612+0000 7fcec5400700 1 --2- 172.21.15.153:0/3080142429 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fcea806e460 0x7fcea8070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:17:39.614 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:39.613+0000 7fcec5400700 1 --2- 172.21.15.153:0/3080142429 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fcea806e460 0x7fcea8070920 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fceb4006010 tx=0x7fceb401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:17:39.618 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:39.617+0000 7fcebe000700 1 -- 172.21.15.153:0/3080142429 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fceac057430 con 0x7fcec00ff7b0 2024-03-21T10:17:39.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:39 smithi174 ceph-mon[88004]: from='client.34346 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:17:39.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:39 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/1104631026' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:17:39.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:39 smithi174 ceph-mon[88004]: pgmap v158: 65 pgs: 1 activating, 1 active, 63 active+clean; 317 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.4 KiB/s rd, 2 op/s 2024-03-21T10:17:39.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:39 smithi153 ceph-mon[113733]: from='client.34346 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:17:39.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:39 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/1104631026' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:17:39.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:39 smithi153 ceph-mon[113733]: pgmap v158: 65 pgs: 1 activating, 1 active, 63 active+clean; 317 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.4 KiB/s rd, 2 op/s 2024-03-21T10:17:39.863 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:39.861+0000 7fcec6800700 1 -- 172.21.15.153:0/3080142429 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcec002cd80 con 0x7fcea806e460 2024-03-21T10:17:39.865 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:39.863+0000 7fcebe000700 1 -- 172.21.15.153:0/3080142429 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7fcec002cd80 con 0x7fcea806e460 2024-03-21T10:17:39.865 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:17:39.865 INFO:teuthology.orchestra.run.smithi153.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232", 2024-03-21T10:17:39.865 INFO:teuthology.orchestra.run.smithi153.stdout: "in_progress": true, 2024-03-21T10:17:39.865 INFO:teuthology.orchestra.run.smithi153.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-21T10:17:39.865 INFO:teuthology.orchestra.run.smithi153.stdout: "services_complete": [ 2024-03-21T10:17:39.865 INFO:teuthology.orchestra.run.smithi153.stdout: "crash", 2024-03-21T10:17:39.865 INFO:teuthology.orchestra.run.smithi153.stdout: "mon", 2024-03-21T10:17:39.865 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr" 2024-03-21T10:17:39.865 INFO:teuthology.orchestra.run.smithi153.stdout: ], 2024-03-21T10:17:39.865 INFO:teuthology.orchestra.run.smithi153.stdout: "progress": "10/23 daemons upgraded", 2024-03-21T10:17:39.866 INFO:teuthology.orchestra.run.smithi153.stdout: "message": "Currently upgrading osd daemons", 2024-03-21T10:17:39.866 INFO:teuthology.orchestra.run.smithi153.stdout: "is_paused": false 2024-03-21T10:17:39.866 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:17:39.868 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:39.866+0000 7fcec6800700 1 -- 172.21.15.153:0/3080142429 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fcea806e460 msgr2=0x7fcea8070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:17:39.868 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:39.866+0000 7fcec6800700 1 --2- 172.21.15.153:0/3080142429 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fcea806e460 0x7fcea8070920 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fceb4006010 tx=0x7fceb401a040 comp rx=0 tx=0).stop 2024-03-21T10:17:39.868 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:39.866+0000 7fcec6800700 1 -- 172.21.15.153:0/3080142429 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcec00ff7b0 msgr2=0x7fcec0100de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:17:39.868 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:39.866+0000 7fcec6800700 1 --2- 172.21.15.153:0/3080142429 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcec00ff7b0 0x7fcec0100de0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fceac00d900 tx=0x7fceac00dcc0 comp rx=0 tx=0).stop 2024-03-21T10:17:39.868 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:39.866+0000 7fcec6800700 1 -- 172.21.15.153:0/3080142429 shutdown_connections 2024-03-21T10:17:39.868 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:39.866+0000 7fcec6800700 1 --2- 172.21.15.153:0/3080142429 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcec00fee00 0x7fcec01008a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:17:39.868 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:39.866+0000 7fcec6800700 1 --2- 172.21.15.153:0/3080142429 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fcea806e460 0x7fcea8070920 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:17:39.868 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:39.866+0000 7fcec6800700 1 --2- 172.21.15.153:0/3080142429 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcec00ff7b0 0x7fcec0100de0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:17:39.868 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:39.866+0000 7fcec6800700 1 -- 172.21.15.153:0/3080142429 >> 172.21.15.153:0/3080142429 conn(0x7fcec00fa690 msgr2=0x7fcec00fba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:17:39.868 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:39.866+0000 7fcec6800700 1 -- 172.21.15.153:0/3080142429 shutdown_connections 2024-03-21T10:17:39.868 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:39.866+0000 7fcec6800700 1 -- 172.21.15.153:0/3080142429 wait complete. 2024-03-21T10:17:40.024 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:40.022+0000 7f0e16e00700 1 -- 172.21.15.153:0/1034670716 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e10075be0 msgr2=0x7f0e101114c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:17:40.024 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:40.022+0000 7f0e16e00700 1 --2- 172.21.15.153:0/1034670716 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e10075be0 0x7f0e101114c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0dfc009910 tx=0x7f0dfc009c20 comp rx=0 tx=0).stop 2024-03-21T10:17:40.024 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:40.022+0000 7f0e16e00700 1 -- 172.21.15.153:0/1034670716 shutdown_connections 2024-03-21T10:17:40.024 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:40.022+0000 7f0e16e00700 1 --2- 172.21.15.153:0/1034670716 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e10075be0 0x7f0e101114c0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:17:40.024 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:40.022+0000 7f0e16e00700 1 --2- 172.21.15.153:0/1034670716 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0e100752c0 0x7f0e100756a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:17:40.025 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:40.022+0000 7f0e16e00700 1 -- 172.21.15.153:0/1034670716 >> 172.21.15.153:0/1034670716 conn(0x7f0e100fe870 msgr2=0x7f0e10100c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:17:40.025 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:40.022+0000 7f0e16e00700 1 -- 172.21.15.153:0/1034670716 shutdown_connections 2024-03-21T10:17:40.025 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:40.022+0000 7f0e16e00700 1 -- 172.21.15.153:0/1034670716 wait complete. 2024-03-21T10:17:40.027 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:40.023+0000 7f0e16e00700 1 Processor -- start 2024-03-21T10:17:40.027 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:40.023+0000 7f0e16e00700 1 -- start start 2024-03-21T10:17:40.027 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:40.024+0000 7f0e16e00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0e100752c0 0x7f0e1019ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:17:40.027 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:40.024+0000 7f0e16e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e10075be0 0x7f0e1019d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:17:40.027 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:40.024+0000 7f0e16e00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e1019d800 con 0x7f0e10075be0 2024-03-21T10:17:40.027 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:40.024+0000 7f0e16e00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e1019d970 con 0x7f0e100752c0 2024-03-21T10:17:40.027 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:40.024+0000 7f0e0f400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e10075be0 0x7f0e1019d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:17:40.027 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:40.024+0000 7f0e0f400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e10075be0 0x7f0e1019d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:49728/0 (socket says 172.21.15.153:49728) 2024-03-21T10:17:40.027 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:40.024+0000 7f0e0f400700 1 -- 172.21.15.153:0/323134473 learned_addr learned my addr 172.21.15.153:0/323134473 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:17:40.027 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:40.024+0000 7f0e0fe00700 1 --2- 172.21.15.153:0/323134473 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0e100752c0 0x7f0e1019ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:17:40.027 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:40.025+0000 7f0e0f400700 1 -- 172.21.15.153:0/323134473 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0e100752c0 msgr2=0x7f0e1019ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:17:40.027 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:40.025+0000 7f0e0f400700 1 --2- 172.21.15.153:0/323134473 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0e100752c0 0x7f0e1019ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:17:40.028 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:40.025+0000 7f0e0f400700 1 -- 172.21.15.153:0/323134473 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dfc0095f0 con 0x7f0e10075be0 2024-03-21T10:17:40.028 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:40.025+0000 7f0e0f400700 1 --2- 172.21.15.153:0/323134473 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e10075be0 0x7f0e1019d200 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0dfc006010 tx=0x7f0dfc0056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:17:40.028 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:40.025+0000 7f0e0cc00700 1 -- 172.21.15.153:0/323134473 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0dfc01d070 con 0x7f0e10075be0 2024-03-21T10:17:40.028 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:40.025+0000 7f0e0cc00700 1 -- 172.21.15.153:0/323134473 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0dfc00f460 con 0x7f0e10075be0 2024-03-21T10:17:40.028 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:40.025+0000 7f0e16e00700 1 -- 172.21.15.153:0/323134473 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e101a0710 con 0x7f0e10075be0 2024-03-21T10:17:40.028 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:40.025+0000 7f0e0cc00700 1 -- 172.21.15.153:0/323134473 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0dfc021640 con 0x7f0e10075be0 2024-03-21T10:17:40.028 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:40.025+0000 7f0e16e00700 1 -- 172.21.15.153:0/323134473 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e101a0c60 con 0x7f0e10075be0 2024-03-21T10:17:40.029 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:40.027+0000 7f0e16e00700 1 -- 172.21.15.153:0/323134473 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e1010ec30 con 0x7f0e10075be0 2024-03-21T10:17:40.029 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:40.027+0000 7f0e0cc00700 1 -- 172.21.15.153:0/323134473 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f0dfc00f5d0 con 0x7f0e10075be0 2024-03-21T10:17:40.030 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:40.028+0000 7f0e0cc00700 1 --2- 172.21.15.153:0/323134473 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f0df406e460 0x7f0df4070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:17:40.030 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:40.028+0000 7f0e0cc00700 1 -- 172.21.15.153:0/323134473 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6654+0+0 (secure 0 0 0) 0x7f0dfc08f6e0 con 0x7f0e10075be0 2024-03-21T10:17:40.030 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:40.029+0000 7f0e0fe00700 1 --2- 172.21.15.153:0/323134473 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f0df406e460 0x7f0df4070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:17:40.031 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:40.029+0000 7f0e0fe00700 1 --2- 172.21.15.153:0/323134473 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f0df406e460 0x7f0df4070920 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0e00005ea0 tx=0x7f0e00005e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:17:40.034 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:40.033+0000 7f0e0cc00700 1 -- 172.21.15.153:0/323134473 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0dfc058850 con 0x7f0e10075be0 2024-03-21T10:17:40.322 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:40.321+0000 7f0e16e00700 1 -- 172.21.15.153:0/323134473 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f0e1002cd30 con 0x7f0e10075be0 2024-03-21T10:17:40.323 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:40.322+0000 7f0e0cc00700 1 -- 172.21.15.153:0/323134473 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f0dfc05bf10 con 0x7f0e10075be0 2024-03-21T10:17:40.323 INFO:teuthology.orchestra.run.smithi153.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-03-21T10:17:40.324 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-21T10:17:40.324 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-21T10:17:40.326 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:40.324+0000 7f0e16e00700 1 -- 172.21.15.153:0/323134473 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f0df406e460 msgr2=0x7f0df4070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:17:40.327 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:40.324+0000 7f0e16e00700 1 --2- 172.21.15.153:0/323134473 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f0df406e460 0x7f0df4070920 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0e00005ea0 tx=0x7f0e00005e30 comp rx=0 tx=0).stop 2024-03-21T10:17:40.327 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:40.324+0000 7f0e16e00700 1 -- 172.21.15.153:0/323134473 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e10075be0 msgr2=0x7f0e1019d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:17:40.327 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:40.324+0000 7f0e16e00700 1 --2- 172.21.15.153:0/323134473 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e10075be0 0x7f0e1019d200 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0dfc006010 tx=0x7f0dfc0056a0 comp rx=0 tx=0).stop 2024-03-21T10:17:40.327 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:40.325+0000 7f0e16e00700 1 -- 172.21.15.153:0/323134473 shutdown_connections 2024-03-21T10:17:40.327 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:40.325+0000 7f0e16e00700 1 --2- 172.21.15.153:0/323134473 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0e100752c0 0x7f0e1019ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:17:40.327 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:40.325+0000 7f0e16e00700 1 --2- 172.21.15.153:0/323134473 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f0df406e460 0x7f0df4070920 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:17:40.327 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:40.325+0000 7f0e16e00700 1 --2- 172.21.15.153:0/323134473 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e10075be0 0x7f0e1019d200 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:17:40.327 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:40.325+0000 7f0e16e00700 1 -- 172.21.15.153:0/323134473 >> 172.21.15.153:0/323134473 conn(0x7f0e100fe870 msgr2=0x7f0e100ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:17:40.327 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:40.325+0000 7f0e16e00700 1 -- 172.21.15.153:0/323134473 shutdown_connections 2024-03-21T10:17:40.327 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:17:40.325+0000 7f0e16e00700 1 -- 172.21.15.153:0/323134473 wait complete. 2024-03-21T10:17:40.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:40 smithi174 ceph-mon[88004]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 30/366 objects degraded (8.197%), 8 pgs degraded) 2024-03-21T10:17:40.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:40 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/4275279380' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:17:40.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:40 smithi174 ceph-mon[88004]: from='client.34358 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:17:40.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:40 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/323134473' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:17:40.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:40 smithi153 ceph-mon[113733]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 30/366 objects degraded (8.197%), 8 pgs degraded) 2024-03-21T10:17:40.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:40 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/4275279380' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:17:40.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:40 smithi153 ceph-mon[113733]: from='client.34358 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:17:40.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:40 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/323134473' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:17:41.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:41 smithi174 ceph-mon[88004]: pgmap v159: 65 pgs: 65 active+clean; 317 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-21T10:17:41.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:41 smithi153 ceph-mon[113733]: pgmap v159: 65 pgs: 65 active+clean; 317 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-21T10:17:43.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:43 smithi174 ceph-mon[88004]: pgmap v160: 65 pgs: 65 active+clean; 317 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s 2024-03-21T10:17:43.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:43 smithi153 ceph-mon[113733]: pgmap v160: 65 pgs: 65 active+clean; 317 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s 2024-03-21T10:17:45.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:45 smithi174 ceph-mon[88004]: pgmap v161: 65 pgs: 65 active+clean; 317 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 3 op/s 2024-03-21T10:17:45.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:45 smithi153 ceph-mon[113733]: pgmap v161: 65 pgs: 65 active+clean; 317 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 3 op/s 2024-03-21T10:17:47.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:47 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:17:47.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:47 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:17:47.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:47 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:17:47.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:47 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:17:47.703 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:47 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:17:47.703 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:47 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:17:48.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:48 smithi153 ceph-mon[113733]: pgmap v162: 65 pgs: 65 active+clean; 317 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 4 op/s 2024-03-21T10:17:48.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:48 smithi174 ceph-mon[88004]: pgmap v162: 65 pgs: 65 active+clean; 317 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 4 op/s 2024-03-21T10:17:50.551 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:50 smithi174 ceph-mon[88004]: pgmap v163: 65 pgs: 65 active+clean; 317 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-21T10:17:50.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:50 smithi153 ceph-mon[113733]: pgmap v163: 65 pgs: 65 active+clean; 317 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-21T10:17:52.511 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:52 smithi153 ceph-mon[113733]: pgmap v164: 65 pgs: 65 active+clean; 317 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 4 op/s 2024-03-21T10:17:52.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:52 smithi174 ceph-mon[88004]: pgmap v164: 65 pgs: 65 active+clean; 317 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 4 op/s 2024-03-21T10:17:53.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:53 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-21T10:17:53.585 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:53 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-21T10:17:54.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:54 smithi153 ceph-mon[113733]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-21T10:17:54.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:54 smithi153 ceph-mon[113733]: Upgrade: osd.4 is safe to restart 2024-03-21T10:17:54.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:54 smithi153 ceph-mon[113733]: pgmap v165: 65 pgs: 65 active+clean; 317 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:17:54.595 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:54 smithi174 ceph-mon[88004]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-21T10:17:54.595 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:54 smithi174 ceph-mon[88004]: Upgrade: osd.4 is safe to restart 2024-03-21T10:17:54.595 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:54 smithi174 ceph-mon[88004]: pgmap v165: 65 pgs: 65 active+clean; 317 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:17:55.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:55 smithi174 ceph-mon[88004]: Upgrade: Updating osd.4 2024-03-21T10:17:55.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:55 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:17:55.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:55 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-21T10:17:55.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:55 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:17:55.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:55 smithi174 ceph-mon[88004]: Deploying daemon osd.4 on smithi174 2024-03-21T10:17:55.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:55 smithi174 ceph-mon[88004]: pgmap v166: 65 pgs: 65 active+clean; 317 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:17:56.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:55 smithi153 ceph-mon[113733]: Upgrade: Updating osd.4 2024-03-21T10:17:56.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:55 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:17:56.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:55 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-21T10:17:56.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:55 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:17:56.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:55 smithi153 ceph-mon[113733]: Deploying daemon osd.4 on smithi174 2024-03-21T10:17:56.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:55 smithi153 ceph-mon[113733]: pgmap v166: 65 pgs: 65 active+clean; 317 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:17:57.576 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:57 smithi174 ceph-mon[88004]: pgmap v167: 65 pgs: 65 active+clean; 317 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:17:57.685 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:57 smithi153 ceph-mon[113733]: pgmap v167: 65 pgs: 65 active+clean; 317 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:17:59.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:17:59 smithi174 ceph-mon[88004]: pgmap v168: 65 pgs: 65 active+clean; 317 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:17:59.702 INFO:journalctl@ceph.osd.4.smithi174.stdout:Mar 21 10:17:59 smithi174 systemd[1]: Stopping Ceph osd.4 for 070ed3e4-e769-11ee-95cd-87774f69a715... 2024-03-21T10:17:59.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:17:59 smithi153 ceph-mon[113733]: pgmap v168: 65 pgs: 65 active+clean; 317 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:18:00.203 INFO:journalctl@ceph.osd.4.smithi174.stdout:Mar 21 10:17:59 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-4[50194]: 2024-03-21T10:17:59.780+0000 7fc755a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T10:18:00.203 INFO:journalctl@ceph.osd.4.smithi174.stdout:Mar 21 10:17:59 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-4[50194]: 2024-03-21T10:17:59.780+0000 7fc755a00700 -1 osd.4 81 *** Got signal Terminated *** 2024-03-21T10:18:00.203 INFO:journalctl@ceph.osd.4.smithi174.stdout:Mar 21 10:17:59 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-4[50194]: 2024-03-21T10:17:59.780+0000 7fc755a00700 -1 osd.4 81 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-21T10:18:00.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:00 smithi174 ceph-mon[88004]: osd.4 marked itself down and dead 2024-03-21T10:18:00.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:00 smithi153 ceph-mon[113733]: osd.4 marked itself down and dead 2024-03-21T10:18:01.202 INFO:journalctl@ceph.osd.4.smithi174.stdout:Mar 21 10:18:01 smithi174 podman[101592]: 2024-03-21 10:18:01.04594009 +0000 UTC m=+1.376751764 container died c4cd15701f056d25e76064bcde742daf61a8650dd7c51bd2382f4829192fca5b (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-4, GIT_CLEAN=True, GIT_REPO=https://github.com/ceph/ceph-container.git, RELEASE=HEAD, io.buildah.version=1.29.1, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=-18.2.1, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T10:18:01.492 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:01 smithi174 ceph-mon[88004]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-21T10:18:01.492 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:01 smithi174 ceph-mon[88004]: osdmap e82: 6 total, 5 up, 6 in 2024-03-21T10:18:01.492 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:01 smithi174 ceph-mon[88004]: pgmap v170: 65 pgs: 3 peering, 5 stale+active+clean, 57 active+clean; 317 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-03-21T10:18:01.492 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:01 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:18:01.492 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:01 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:18:01.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:01 smithi153 ceph-mon[113733]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-21T10:18:01.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:01 smithi153 ceph-mon[113733]: osdmap e82: 6 total, 5 up, 6 in 2024-03-21T10:18:01.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:01 smithi153 ceph-mon[113733]: pgmap v170: 65 pgs: 3 peering, 5 stale+active+clean, 57 active+clean; 317 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-03-21T10:18:01.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:01 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:18:01.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:01 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:18:01.951 INFO:journalctl@ceph.osd.4.smithi174.stdout:Mar 21 10:18:01 smithi174 podman[101592]: 2024-03-21 10:18:01.490856513 +0000 UTC m=+1.821668147 container cleanup c4cd15701f056d25e76064bcde742daf61a8650dd7c51bd2382f4829192fca5b (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-4, GIT_CLEAN=True, RELEASE=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=-18.2.1, ceph=True, io.buildah.version=1.29.1, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T10:18:01.952 INFO:journalctl@ceph.osd.4.smithi174.stdout:Mar 21 10:18:01 smithi174 bash[101592]: ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-4 2024-03-21T10:18:01.952 INFO:journalctl@ceph.osd.4.smithi174.stdout:Mar 21 10:18:01 smithi174 podman[101611]: 2024-03-21 10:18:01.683848253 +0000 UTC m=+0.629675799 container remove c4cd15701f056d25e76064bcde742daf61a8650dd7c51bd2382f4829192fca5b (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-4, CEPH_POINT_RELEASE=-18.2.1, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.29.1, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=HEAD, ceph=True, GIT_BRANCH=HEAD) 2024-03-21T10:18:02.650 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:02 smithi153 ceph-mon[113733]: osdmap e83: 6 total, 5 up, 6 in 2024-03-21T10:18:02.650 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:02 smithi153 ceph-mon[113733]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-03-21T10:18:02.690 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:02 smithi174 ceph-mon[88004]: osdmap e83: 6 total, 5 up, 6 in 2024-03-21T10:18:02.690 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:02 smithi174 ceph-mon[88004]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-03-21T10:18:02.691 INFO:journalctl@ceph.osd.4.smithi174.stdout:Mar 21 10:18:02 smithi174 podman[101701]: 2024-03-21 10:18:02.430060554 +0000 UTC m=+0.040623195 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T10:18:02.691 INFO:journalctl@ceph.osd.4.smithi174.stdout:Mar 21 10:18:02 smithi174 podman[101701]: 2024-03-21 10:18:02.543267981 +0000 UTC m=+0.153830646 container create 53d37a88664a57c0adcd354008658418997348c87c1387948e07d50062cbe8e3 (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-4-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True) 2024-03-21T10:18:02.952 INFO:journalctl@ceph.osd.4.smithi174.stdout:Mar 21 10:18:02 smithi174 podman[101701]: 2024-03-21 10:18:02.780068728 +0000 UTC m=+0.390631395 container init 53d37a88664a57c0adcd354008658418997348c87c1387948e07d50062cbe8e3 (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-4-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-21T10:18:02.960 INFO:journalctl@ceph.osd.4.smithi174.stdout:Mar 21 10:18:02 smithi174 podman[101701]: 2024-03-21 10:18:02.79174765 +0000 UTC m=+0.402310317 container start 53d37a88664a57c0adcd354008658418997348c87c1387948e07d50062cbe8e3 (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=) 2024-03-21T10:18:02.960 INFO:journalctl@ceph.osd.4.smithi174.stdout:Mar 21 10:18:02 smithi174 podman[101701]: 2024-03-21 10:18:02.870545131 +0000 UTC m=+0.481107842 container attach 53d37a88664a57c0adcd354008658418997348c87c1387948e07d50062cbe8e3 (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-4-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-03-21T10:18:03.524 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:03 smithi174 ceph-mon[88004]: pgmap v172: 65 pgs: 6 active+undersized, 7 active+undersized+degraded, 3 peering, 3 stale+active+clean, 46 active+clean; 317 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 3 op/s; 25/366 objects degraded (6.831%) 2024-03-21T10:18:03.524 INFO:journalctl@ceph.osd.4.smithi174.stdout:Mar 21 10:18:03 smithi174 podman[101701]: 2024-03-21 10:18:03.221905541 +0000 UTC m=+0.832468208 container died 53d37a88664a57c0adcd354008658418997348c87c1387948e07d50062cbe8e3 (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-4-deactivate, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T10:18:03.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:03 smithi153 ceph-mon[113733]: pgmap v172: 65 pgs: 6 active+undersized, 7 active+undersized+degraded, 3 peering, 3 stale+active+clean, 46 active+clean; 317 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 3 op/s; 25/366 objects degraded (6.831%) 2024-03-21T10:18:04.186 INFO:journalctl@ceph.osd.4.smithi174.stdout:Mar 21 10:18:03 smithi174 podman[101726]: 2024-03-21 10:18:03.781899753 +0000 UTC m=+0.542849987 container remove 53d37a88664a57c0adcd354008658418997348c87c1387948e07d50062cbe8e3 (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-4-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T10:18:04.186 INFO:journalctl@ceph.osd.4.smithi174.stdout:Mar 21 10:18:03 smithi174 systemd[1]: ceph-070ed3e4-e769-11ee-95cd-87774f69a715@osd.4.service: Deactivated successfully. 2024-03-21T10:18:04.187 INFO:journalctl@ceph.osd.4.smithi174.stdout:Mar 21 10:18:03 smithi174 systemd[1]: Stopped Ceph osd.4 for 070ed3e4-e769-11ee-95cd-87774f69a715. 2024-03-21T10:18:04.187 INFO:journalctl@ceph.osd.4.smithi174.stdout:Mar 21 10:18:03 smithi174 systemd[1]: ceph-070ed3e4-e769-11ee-95cd-87774f69a715@osd.4.service: Consumed 1min 43.628s CPU time. 2024-03-21T10:18:04.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:04 smithi174 ceph-mon[88004]: Health check failed: Degraded data redundancy: 25/366 objects degraded (6.831%), 7 pgs degraded (PG_DEGRADED) 2024-03-21T10:18:04.452 INFO:journalctl@ceph.osd.4.smithi174.stdout:Mar 21 10:18:04 smithi174 systemd[1]: Starting Ceph osd.4 for 070ed3e4-e769-11ee-95cd-87774f69a715... 2024-03-21T10:18:04.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:04 smithi153 ceph-mon[113733]: Health check failed: Degraded data redundancy: 25/366 objects degraded (6.831%), 7 pgs degraded (PG_DEGRADED) 2024-03-21T10:18:04.882 INFO:journalctl@ceph.osd.4.smithi174.stdout:Mar 21 10:18:04 smithi174 podman[101827]: 2024-03-21 10:18:04.448769087 +0000 UTC m=+0.032153940 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T10:18:04.882 INFO:journalctl@ceph.osd.4.smithi174.stdout:Mar 21 10:18:04 smithi174 podman[101827]: 2024-03-21 10:18:04.591530779 +0000 UTC m=+0.174915657 container create 20bcf9394d789f25e4ec8dcd4ff18ef45c86434edbccb5b683f828d220dd3336 (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-4-activate, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, ceph=True) 2024-03-21T10:18:04.882 INFO:journalctl@ceph.osd.4.smithi174.stdout:Mar 21 10:18:04 smithi174 podman[101827]: 2024-03-21 10:18:04.880949873 +0000 UTC m=+0.464334813 container init 20bcf9394d789f25e4ec8dcd4ff18ef45c86434edbccb5b683f828d220dd3336 (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-4-activate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, GIT_CLEAN=True, ceph=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-21T10:18:05.202 INFO:journalctl@ceph.osd.4.smithi174.stdout:Mar 21 10:18:04 smithi174 podman[101827]: 2024-03-21 10:18:04.89221942 +0000 UTC m=+0.475604317 container start 20bcf9394d789f25e4ec8dcd4ff18ef45c86434edbccb5b683f828d220dd3336 (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-4-activate, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-03-21T10:18:05.202 INFO:journalctl@ceph.osd.4.smithi174.stdout:Mar 21 10:18:04 smithi174 podman[101827]: 2024-03-21 10:18:04.969081315 +0000 UTC m=+0.552466190 container attach 20bcf9394d789f25e4ec8dcd4ff18ef45c86434edbccb5b683f828d220dd3336 (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-4-activate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS) 2024-03-21T10:18:05.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:05 smithi174 ceph-mon[88004]: pgmap v173: 65 pgs: 10 active+undersized, 12 active+undersized+degraded, 3 peering, 1 stale+active+clean, 39 active+clean; 317 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 49/366 objects degraded (13.388%) 2024-03-21T10:18:05.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:05 smithi153 ceph-mon[113733]: pgmap v173: 65 pgs: 10 active+undersized, 12 active+undersized+degraded, 3 peering, 1 stale+active+clean, 39 active+clean; 317 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 49/366 objects degraded (13.388%) 2024-03-21T10:18:06.052 INFO:journalctl@ceph.osd.4.smithi174.stdout:Mar 21 10:18:05 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-4-activate[101842]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-03-21T10:18:06.052 INFO:journalctl@ceph.osd.4.smithi174.stdout:Mar 21 10:18:05 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-4-activate[101842]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-03-21T10:18:06.052 INFO:journalctl@ceph.osd.4.smithi174.stdout:Mar 21 10:18:05 smithi174 bash[101827]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-03-21T10:18:06.052 INFO:journalctl@ceph.osd.4.smithi174.stdout:Mar 21 10:18:05 smithi174 bash[101827]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-03-21T10:18:06.052 INFO:journalctl@ceph.osd.4.smithi174.stdout:Mar 21 10:18:05 smithi174 bash[101827]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-03-21T10:18:06.052 INFO:journalctl@ceph.osd.4.smithi174.stdout:Mar 21 10:18:05 smithi174 bash[101827]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-03-21T10:18:06.052 INFO:journalctl@ceph.osd.4.smithi174.stdout:Mar 21 10:18:05 smithi174 bash[101827]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-4/block 2024-03-21T10:18:06.053 INFO:journalctl@ceph.osd.4.smithi174.stdout:Mar 21 10:18:05 smithi174 bash[101827]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-03-21T10:18:06.053 INFO:journalctl@ceph.osd.4.smithi174.stdout:Mar 21 10:18:05 smithi174 bash[101827]: --> ceph-volume raw activate successful for osd ID: 4 2024-03-21T10:18:06.053 INFO:journalctl@ceph.osd.4.smithi174.stdout:Mar 21 10:18:05 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-4-activate[101842]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-03-21T10:18:06.053 INFO:journalctl@ceph.osd.4.smithi174.stdout:Mar 21 10:18:05 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-4-activate[101842]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-03-21T10:18:06.053 INFO:journalctl@ceph.osd.4.smithi174.stdout:Mar 21 10:18:05 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-4-activate[101842]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-4/block 2024-03-21T10:18:06.053 INFO:journalctl@ceph.osd.4.smithi174.stdout:Mar 21 10:18:05 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-4-activate[101842]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-03-21T10:18:06.053 INFO:journalctl@ceph.osd.4.smithi174.stdout:Mar 21 10:18:05 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-4-activate[101842]: --> ceph-volume raw activate successful for osd ID: 4 2024-03-21T10:18:06.053 INFO:journalctl@ceph.osd.4.smithi174.stdout:Mar 21 10:18:05 smithi174 podman[101827]: 2024-03-21 10:18:05.814341477 +0000 UTC m=+1.397726367 container died 20bcf9394d789f25e4ec8dcd4ff18ef45c86434edbccb5b683f828d220dd3336 (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-4-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5) 2024-03-21T10:18:06.843 INFO:journalctl@ceph.osd.4.smithi174.stdout:Mar 21 10:18:06 smithi174 podman[101962]: 2024-03-21 10:18:06.489542071 +0000 UTC m=+0.655719238 container remove 20bcf9394d789f25e4ec8dcd4ff18ef45c86434edbccb5b683f828d220dd3336 (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-4-activate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_CLEAN=True) 2024-03-21T10:18:07.122 INFO:journalctl@ceph.osd.4.smithi174.stdout:Mar 21 10:18:06 smithi174 podman[102023]: 2024-03-21 10:18:06.74739383 +0000 UTC m=+0.036091303 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T10:18:07.122 INFO:journalctl@ceph.osd.4.smithi174.stdout:Mar 21 10:18:06 smithi174 podman[102023]: 2024-03-21 10:18:06.865387662 +0000 UTC m=+0.154085181 container create 0d746ce6e79d8411fd4fa57061dc9fed9bbd3ac66fd24a7d190a00b47059c9e9 (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-4, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, ceph=True, org.label-schema.vendor=CentOS) 2024-03-21T10:18:07.122 INFO:journalctl@ceph.osd.4.smithi174.stdout:Mar 21 10:18:07 smithi174 podman[102023]: 2024-03-21 10:18:07.120639337 +0000 UTC m=+0.409336833 container init 0d746ce6e79d8411fd4fa57061dc9fed9bbd3ac66fd24a7d190a00b47059c9e9 (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-4, org.label-schema.build-date=20240319, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-03-21T10:18:07.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:07 smithi174 ceph-mon[88004]: pgmap v174: 65 pgs: 16 active+undersized, 16 active+undersized+degraded, 33 active+clean; 317 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 63/366 objects degraded (17.213%) 2024-03-21T10:18:07.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:07 smithi174 ceph-mon[88004]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-03-21T10:18:07.452 INFO:journalctl@ceph.osd.4.smithi174.stdout:Mar 21 10:18:07 smithi174 podman[102023]: 2024-03-21 10:18:07.13113715 +0000 UTC m=+0.419834672 container start 0d746ce6e79d8411fd4fa57061dc9fed9bbd3ac66fd24a7d190a00b47059c9e9 (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-4, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-21T10:18:07.452 INFO:journalctl@ceph.osd.4.smithi174.stdout:Mar 21 10:18:07 smithi174 ceph-osd[102042]: -- 172.21.15.174:0/3228839265 <== mon.1 v2:172.21.15.174:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5653f171f680 con 0x5653f16e8c00 2024-03-21T10:18:07.452 INFO:journalctl@ceph.osd.4.smithi174.stdout:Mar 21 10:18:07 smithi174 bash[102023]: 0d746ce6e79d8411fd4fa57061dc9fed9bbd3ac66fd24a7d190a00b47059c9e9 2024-03-21T10:18:07.452 INFO:journalctl@ceph.osd.4.smithi174.stdout:Mar 21 10:18:07 smithi174 systemd[1]: Started Ceph osd.4 for 070ed3e4-e769-11ee-95cd-87774f69a715. 2024-03-21T10:18:07.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:07 smithi153 ceph-mon[113733]: pgmap v174: 65 pgs: 16 active+undersized, 16 active+undersized+degraded, 33 active+clean; 317 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 63/366 objects degraded (17.213%) 2024-03-21T10:18:07.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:07 smithi153 ceph-mon[113733]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-03-21T10:18:08.201 INFO:journalctl@ceph.osd.4.smithi174.stdout:Mar 21 10:18:07 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-4[102038]: 2024-03-21T10:18:07.713+0000 7f19c7b2b7c0 -1 Falling back to public interface 2024-03-21T10:18:08.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:08 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:18:08.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:08 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:18:08.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:08 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:18:08.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:08 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:18:08.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:08 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:18:08.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:08 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:18:09.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:09 smithi174 ceph-mon[88004]: pgmap v175: 65 pgs: 16 active+undersized, 16 active+undersized+degraded, 33 active+clean; 317 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 63/366 objects degraded (17.213%) 2024-03-21T10:18:09.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:09 smithi153 ceph-mon[113733]: pgmap v175: 65 pgs: 16 active+undersized, 16 active+undersized+degraded, 33 active+clean; 317 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 63/366 objects degraded (17.213%) 2024-03-21T10:18:10.499 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:10.497+0000 7f4fbfe00700 1 -- 172.21.15.153:0/2480087988 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4fb81039b0 msgr2=0x7f4fb8107a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:18:10.499 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:10.497+0000 7f4fbfe00700 1 --2- 172.21.15.153:0/2480087988 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4fb81039b0 0x7f4fb8107a00 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4fa4009910 tx=0x7f4fa4009c20 comp rx=0 tx=0).stop 2024-03-21T10:18:10.499 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:10.497+0000 7f4fbfe00700 1 -- 172.21.15.153:0/2480087988 shutdown_connections 2024-03-21T10:18:10.499 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:10.497+0000 7f4fbfe00700 1 --2- 172.21.15.153:0/2480087988 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4fb81039b0 0x7f4fb8107a00 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:18:10.499 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:10.497+0000 7f4fbfe00700 1 --2- 172.21.15.153:0/2480087988 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4fb8103000 0x7f4fb81033e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:18:10.499 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:10.497+0000 7f4fbfe00700 1 -- 172.21.15.153:0/2480087988 >> 172.21.15.153:0/2480087988 conn(0x7f4fb80fe8d0 msgr2=0x7f4fb8100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:18:10.499 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:10.497+0000 7f4fbfe00700 1 -- 172.21.15.153:0/2480087988 shutdown_connections 2024-03-21T10:18:10.499 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:10.497+0000 7f4fbfe00700 1 -- 172.21.15.153:0/2480087988 wait complete. 2024-03-21T10:18:10.501 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:10.498+0000 7f4fbfe00700 1 Processor -- start 2024-03-21T10:18:10.501 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:10.499+0000 7f4fbfe00700 1 -- start start 2024-03-21T10:18:10.501 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:10.499+0000 7f4fbfe00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4fb8103000 0x7f4fb80752a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:18:10.501 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:10.499+0000 7f4fbfe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4fb81039b0 0x7f4fb80757e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:18:10.501 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:10.499+0000 7f4fbfe00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb8079430 con 0x7f4fb81039b0 2024-03-21T10:18:10.501 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:10.499+0000 7f4fbfe00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb8075d20 con 0x7f4fb8103000 2024-03-21T10:18:10.501 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:10.499+0000 7f4fbca00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4fb81039b0 0x7f4fb80757e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:18:10.501 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:10.499+0000 7f4fbca00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4fb81039b0 0x7f4fb80757e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:36954/0 (socket says 172.21.15.153:36954) 2024-03-21T10:18:10.501 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:10.499+0000 7f4fbca00700 1 -- 172.21.15.153:0/1575310380 learned_addr learned my addr 172.21.15.153:0/1575310380 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:18:10.501 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:10.499+0000 7f4fbd400700 1 --2- 172.21.15.153:0/1575310380 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4fb8103000 0x7f4fb80752a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:18:10.501 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:10.500+0000 7f4fbca00700 1 -- 172.21.15.153:0/1575310380 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4fb8103000 msgr2=0x7f4fb80752a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:18:10.501 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:10.500+0000 7f4fbca00700 1 --2- 172.21.15.153:0/1575310380 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4fb8103000 0x7f4fb80752a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:18:10.502 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:10.500+0000 7f4fbca00700 1 -- 172.21.15.153:0/1575310380 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fa40095f0 con 0x7f4fb81039b0 2024-03-21T10:18:10.502 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:10.500+0000 7f4fbca00700 1 --2- 172.21.15.153:0/1575310380 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4fb81039b0 0x7f4fb80757e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4fa400bd70 tx=0x7f4fa400bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:18:10.502 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:10.501+0000 7f4fb6000700 1 -- 172.21.15.153:0/1575310380 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4fa401d070 con 0x7f4fb81039b0 2024-03-21T10:18:10.502 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:10.501+0000 7f4fb6000700 1 -- 172.21.15.153:0/1575310380 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4fa4005dc0 con 0x7f4fb81039b0 2024-03-21T10:18:10.502 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:10.501+0000 7f4fb6000700 1 -- 172.21.15.153:0/1575310380 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4fa4017400 con 0x7f4fb81039b0 2024-03-21T10:18:10.503 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:10.501+0000 7f4fbfe00700 1 -- 172.21.15.153:0/1575310380 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fb8075fa0 con 0x7f4fb81039b0 2024-03-21T10:18:10.503 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:10.501+0000 7f4fbfe00700 1 -- 172.21.15.153:0/1575310380 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fb81a7170 con 0x7f4fb81039b0 2024-03-21T10:18:10.503 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:10.502+0000 7f4fbfe00700 1 -- 172.21.15.153:0/1575310380 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fb810b400 con 0x7f4fb81039b0 2024-03-21T10:18:10.503 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:10.503+0000 7f4fb6000700 1 -- 172.21.15.153:0/1575310380 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f4fa400f460 con 0x7f4fb81039b0 2024-03-21T10:18:10.505 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:10.504+0000 7f4fb6000700 1 --2- 172.21.15.153:0/1575310380 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f4fa006e460 0x7f4fa0070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:18:10.505 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:10.504+0000 7f4fb6000700 1 -- 172.21.15.153:0/1575310380 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6654+0+0 (secure 0 0 0) 0x7f4fa408f920 con 0x7f4fb81039b0 2024-03-21T10:18:10.505 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:10.504+0000 7f4fbd400700 1 --2- 172.21.15.153:0/1575310380 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f4fa006e460 0x7f4fa0070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:18:10.506 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:10.505+0000 7f4fbd400700 1 --2- 172.21.15.153:0/1575310380 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f4fa006e460 0x7f4fa0070920 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4fac005950 tx=0x7f4fac00b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:18:10.509 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:10.509+0000 7f4fb6000700 1 -- 172.21.15.153:0/1575310380 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4fa40589f0 con 0x7f4fb81039b0 2024-03-21T10:18:10.742 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:10.741+0000 7f4fbfe00700 1 -- 172.21.15.153:0/1575310380 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4fb80611d0 con 0x7f4fa006e460 2024-03-21T10:18:10.745 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:10.744+0000 7f4fb6000700 1 -- 172.21.15.153:0/1575310380 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f4fb80611d0 con 0x7f4fa006e460 2024-03-21T10:18:10.748 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:10.747+0000 7f4fbfe00700 1 -- 172.21.15.153:0/1575310380 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f4fa006e460 msgr2=0x7f4fa0070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:18:10.748 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:10.747+0000 7f4fbfe00700 1 --2- 172.21.15.153:0/1575310380 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f4fa006e460 0x7f4fa0070920 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4fac005950 tx=0x7f4fac00b410 comp rx=0 tx=0).stop 2024-03-21T10:18:10.748 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:10.747+0000 7f4fbfe00700 1 -- 172.21.15.153:0/1575310380 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4fb81039b0 msgr2=0x7f4fb80757e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:18:10.748 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:10.747+0000 7f4fbfe00700 1 --2- 172.21.15.153:0/1575310380 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4fb81039b0 0x7f4fb80757e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4fa400bd70 tx=0x7f4fa400bda0 comp rx=0 tx=0).stop 2024-03-21T10:18:10.749 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:10.747+0000 7f4fbfe00700 1 -- 172.21.15.153:0/1575310380 shutdown_connections 2024-03-21T10:18:10.749 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:10.747+0000 7f4fbfe00700 1 --2- 172.21.15.153:0/1575310380 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4fb8103000 0x7f4fb80752a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:18:10.749 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:10.747+0000 7f4fbfe00700 1 --2- 172.21.15.153:0/1575310380 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f4fa006e460 0x7f4fa0070920 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:18:10.749 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:10.747+0000 7f4fbfe00700 1 --2- 172.21.15.153:0/1575310380 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4fb81039b0 0x7f4fb80757e0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:18:10.749 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:10.747+0000 7f4fbfe00700 1 -- 172.21.15.153:0/1575310380 >> 172.21.15.153:0/1575310380 conn(0x7f4fb80fe8d0 msgr2=0x7f4fb8100cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:18:10.749 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:10.747+0000 7f4fbfe00700 1 -- 172.21.15.153:0/1575310380 shutdown_connections 2024-03-21T10:18:10.749 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:10.747+0000 7f4fbfe00700 1 -- 172.21.15.153:0/1575310380 wait complete. 2024-03-21T10:18:10.763 INFO:teuthology.orchestra.run.smithi153.stdout:true 2024-03-21T10:18:10.905 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:10.903+0000 7fac51e00700 1 -- 172.21.15.153:0/4185526584 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fac4c0730b0 msgr2=0x7fac4c073490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:18:10.905 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:10.903+0000 7fac51e00700 1 --2- 172.21.15.153:0/4185526584 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fac4c0730b0 0x7fac4c073490 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fac3c009910 tx=0x7fac3c009c20 comp rx=0 tx=0).stop 2024-03-21T10:18:10.905 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:10.903+0000 7fac51e00700 1 -- 172.21.15.153:0/4185526584 shutdown_connections 2024-03-21T10:18:10.905 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:10.903+0000 7fac51e00700 1 --2- 172.21.15.153:0/4185526584 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fac4c0739d0 0x7fac4c10cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:18:10.905 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:10.903+0000 7fac51e00700 1 --2- 172.21.15.153:0/4185526584 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fac4c0730b0 0x7fac4c073490 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:18:10.905 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:10.904+0000 7fac51e00700 1 -- 172.21.15.153:0/4185526584 >> 172.21.15.153:0/4185526584 conn(0x7fac4c0fc650 msgr2=0x7fac4c0fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:18:10.905 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:10.904+0000 7fac51e00700 1 -- 172.21.15.153:0/4185526584 shutdown_connections 2024-03-21T10:18:10.905 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:10.904+0000 7fac51e00700 1 -- 172.21.15.153:0/4185526584 wait complete. 2024-03-21T10:18:10.906 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:10.905+0000 7fac51e00700 1 Processor -- start 2024-03-21T10:18:10.906 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:10.905+0000 7fac51e00700 1 -- start start 2024-03-21T10:18:10.907 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:10.906+0000 7fac51e00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fac4c0730b0 0x7fac4c19e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:18:10.907 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:10.906+0000 7fac51e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fac4c0739d0 0x7fac4c19ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:18:10.907 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:10.906+0000 7fac51e00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac4c19f2a0 con 0x7fac4c0739d0 2024-03-21T10:18:10.907 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:10.906+0000 7fac51e00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac4c1987a0 con 0x7fac4c0730b0 2024-03-21T10:18:10.907 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:10.906+0000 7fac4a000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fac4c0739d0 0x7fac4c19ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:18:10.907 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:10.906+0000 7fac4a000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fac4c0739d0 0x7fac4c19ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:36976/0 (socket says 172.21.15.153:36976) 2024-03-21T10:18:10.907 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:10.906+0000 7fac4a000700 1 -- 172.21.15.153:0/1557176849 learned_addr learned my addr 172.21.15.153:0/1557176849 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:18:10.907 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:10.906+0000 7fac4aa00700 1 --2- 172.21.15.153:0/1557176849 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fac4c0730b0 0x7fac4c19e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:18:10.908 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:10.906+0000 7fac4a000700 1 -- 172.21.15.153:0/1557176849 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fac4c0730b0 msgr2=0x7fac4c19e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:18:10.908 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:10.906+0000 7fac4a000700 1 --2- 172.21.15.153:0/1557176849 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fac4c0730b0 0x7fac4c19e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:18:10.908 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:10.907+0000 7fac4a000700 1 -- 172.21.15.153:0/1557176849 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac3c0095f0 con 0x7fac4c0739d0 2024-03-21T10:18:10.908 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:10.907+0000 7fac4a000700 1 --2- 172.21.15.153:0/1557176849 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fac4c0739d0 0x7fac4c19ec10 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fac3800d8d0 tx=0x7fac3800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:18:10.909 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:10.907+0000 7fac50a00700 1 -- 172.21.15.153:0/1557176849 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fac38009940 con 0x7fac4c0739d0 2024-03-21T10:18:10.909 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:10.907+0000 7fac50a00700 1 -- 172.21.15.153:0/1557176849 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fac38010460 con 0x7fac4c0739d0 2024-03-21T10:18:10.909 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:10.907+0000 7fac51e00700 1 -- 172.21.15.153:0/1557176849 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac4c198a80 con 0x7fac4c0739d0 2024-03-21T10:18:10.909 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:10.907+0000 7fac50a00700 1 -- 172.21.15.153:0/1557176849 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fac3800f5f0 con 0x7fac4c0739d0 2024-03-21T10:18:10.909 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:10.907+0000 7fac51e00700 1 -- 172.21.15.153:0/1557176849 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac4c198fd0 con 0x7fac4c0739d0 2024-03-21T10:18:10.909 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:10.909+0000 7fac51e00700 1 -- 172.21.15.153:0/1557176849 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac4c10a5e0 con 0x7fac4c0739d0 2024-03-21T10:18:10.910 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:10.910+0000 7fac50a00700 1 -- 172.21.15.153:0/1557176849 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fac380105d0 con 0x7fac4c0739d0 2024-03-21T10:18:10.912 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:10.911+0000 7fac50a00700 1 --2- 172.21.15.153:0/1557176849 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fac3006e460 0x7fac30070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:18:10.912 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:10.911+0000 7fac50a00700 1 -- 172.21.15.153:0/1557176849 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6654+0+0 (secure 0 0 0) 0x7fac3808e500 con 0x7fac4c0739d0 2024-03-21T10:18:10.912 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:10.911+0000 7fac4aa00700 1 --2- 172.21.15.153:0/1557176849 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fac3006e460 0x7fac30070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:18:10.913 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:10.912+0000 7fac4aa00700 1 --2- 172.21.15.153:0/1557176849 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fac3006e460 0x7fac30070920 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fac3c006010 tx=0x7fac3c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:18:10.917 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:10.916+0000 7fac50a00700 1 -- 172.21.15.153:0/1557176849 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fac3805b100 con 0x7fac4c0739d0 2024-03-21T10:18:11.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:10 smithi153 ceph-mon[113733]: Health check update: Degraded data redundancy: 63/366 objects degraded (17.213%), 16 pgs degraded (PG_DEGRADED) 2024-03-21T10:18:11.147 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:11.146+0000 7fac51e00700 1 -- 172.21.15.153:0/1557176849 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fac4c199260 con 0x7fac3006e460 2024-03-21T10:18:11.149 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:11.148+0000 7fac50a00700 1 -- 172.21.15.153:0/1557176849 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7fac4c199260 con 0x7fac3006e460 2024-03-21T10:18:11.154 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:11.153+0000 7fac51e00700 1 -- 172.21.15.153:0/1557176849 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fac3006e460 msgr2=0x7fac30070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:18:11.154 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:11.153+0000 7fac51e00700 1 --2- 172.21.15.153:0/1557176849 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fac3006e460 0x7fac30070920 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fac3c006010 tx=0x7fac3c01a040 comp rx=0 tx=0).stop 2024-03-21T10:18:11.154 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:11.153+0000 7fac51e00700 1 -- 172.21.15.153:0/1557176849 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fac4c0739d0 msgr2=0x7fac4c19ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:18:11.154 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:11.153+0000 7fac51e00700 1 --2- 172.21.15.153:0/1557176849 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fac4c0739d0 0x7fac4c19ec10 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fac3800d8d0 tx=0x7fac3800dc90 comp rx=0 tx=0).stop 2024-03-21T10:18:11.154 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:11.153+0000 7fac51e00700 1 -- 172.21.15.153:0/1557176849 shutdown_connections 2024-03-21T10:18:11.154 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:11.153+0000 7fac51e00700 1 --2- 172.21.15.153:0/1557176849 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fac4c0730b0 0x7fac4c19e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:18:11.154 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:11.153+0000 7fac51e00700 1 --2- 172.21.15.153:0/1557176849 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fac3006e460 0x7fac30070920 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:18:11.155 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:11.153+0000 7fac51e00700 1 --2- 172.21.15.153:0/1557176849 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fac4c0739d0 0x7fac4c19ec10 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:18:11.155 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:11.153+0000 7fac51e00700 1 -- 172.21.15.153:0/1557176849 >> 172.21.15.153:0/1557176849 conn(0x7fac4c0fc650 msgr2=0x7fac4c107720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:18:11.155 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:11.153+0000 7fac51e00700 1 -- 172.21.15.153:0/1557176849 shutdown_connections 2024-03-21T10:18:11.155 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:11.153+0000 7fac51e00700 1 -- 172.21.15.153:0/1557176849 wait complete. 2024-03-21T10:18:11.191 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:10 smithi174 ceph-mon[88004]: Health check update: Degraded data redundancy: 63/366 objects degraded (17.213%), 16 pgs degraded (PG_DEGRADED) 2024-03-21T10:18:11.309 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:11.308+0000 7f189be00700 1 -- 172.21.15.153:0/1715941077 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1894101c60 msgr2=0x7f1894102040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:18:11.309 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:11.308+0000 7f189be00700 1 --2- 172.21.15.153:0/1715941077 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1894101c60 0x7f1894102040 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1888009910 tx=0x7f1888009c20 comp rx=0 tx=0).stop 2024-03-21T10:18:11.310 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:11.308+0000 7f189be00700 1 -- 172.21.15.153:0/1715941077 shutdown_connections 2024-03-21T10:18:11.310 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:11.308+0000 7f189be00700 1 --2- 172.21.15.153:0/1715941077 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1894102610 0x7f189410ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:18:11.310 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:11.308+0000 7f189be00700 1 --2- 172.21.15.153:0/1715941077 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1894101c60 0x7f1894102040 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:18:11.310 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:11.308+0000 7f189be00700 1 -- 172.21.15.153:0/1715941077 >> 172.21.15.153:0/1715941077 conn(0x7f18940fb540 msgr2=0x7f18940fd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:18:11.310 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:11.308+0000 7f189be00700 1 -- 172.21.15.153:0/1715941077 shutdown_connections 2024-03-21T10:18:11.310 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:11.308+0000 7f189be00700 1 -- 172.21.15.153:0/1715941077 wait complete. 2024-03-21T10:18:11.311 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:11.309+0000 7f189be00700 1 Processor -- start 2024-03-21T10:18:11.312 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:11.309+0000 7f189be00700 1 -- start start 2024-03-21T10:18:11.312 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:11.310+0000 7f189be00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1894101c60 0x7f18940ff950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:18:11.312 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:11.310+0000 7f189be00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1894102610 0x7f18940ffe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:18:11.312 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:11.310+0000 7f189be00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18941003d0 con 0x7f1894101c60 2024-03-21T10:18:11.312 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:11.310+0000 7f189be00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1894100510 con 0x7f1894102610 2024-03-21T10:18:11.312 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:11.310+0000 7f1899400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1894101c60 0x7f18940ff950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:18:11.312 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:11.310+0000 7f1899400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1894101c60 0x7f18940ff950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:36998/0 (socket says 172.21.15.153:36998) 2024-03-21T10:18:11.313 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:11.310+0000 7f1899400700 1 -- 172.21.15.153:0/3643444431 learned_addr learned my addr 172.21.15.153:0/3643444431 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:18:11.313 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:11.311+0000 7f1899400700 1 -- 172.21.15.153:0/3643444431 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1894102610 msgr2=0x7f18940ffe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:18:11.313 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:11.311+0000 7f1898a00700 1 --2- 172.21.15.153:0/3643444431 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1894102610 0x7f18940ffe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:18:11.313 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:11.311+0000 7f1899400700 1 --2- 172.21.15.153:0/3643444431 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1894102610 0x7f18940ffe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:18:11.313 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:11.311+0000 7f1899400700 1 -- 172.21.15.153:0/3643444431 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18880095f0 con 0x7f1894101c60 2024-03-21T10:18:11.313 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:11.311+0000 7f1899400700 1 --2- 172.21.15.153:0/3643444431 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1894101c60 0x7f18940ff950 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1888006010 tx=0x7f188800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:18:11.314 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:11.311+0000 7f1892000700 1 -- 172.21.15.153:0/3643444431 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f188801d070 con 0x7f1894101c60 2024-03-21T10:18:11.314 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:11.311+0000 7f1892000700 1 -- 172.21.15.153:0/3643444431 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f188800f460 con 0x7f1894101c60 2024-03-21T10:18:11.314 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:11.311+0000 7f189be00700 1 -- 172.21.15.153:0/3643444431 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1894100790 con 0x7f1894101c60 2024-03-21T10:18:11.314 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:11.311+0000 7f1892000700 1 -- 172.21.15.153:0/3643444431 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1888017630 con 0x7f1894101c60 2024-03-21T10:18:11.314 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:11.311+0000 7f189be00700 1 -- 172.21.15.153:0/3643444431 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18941a2c10 con 0x7f1894101c60 2024-03-21T10:18:11.314 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:11.312+0000 7f189be00700 1 -- 172.21.15.153:0/3643444431 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1894108200 con 0x7f1894101c60 2024-03-21T10:18:11.315 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:11.313+0000 7f1892000700 1 -- 172.21.15.153:0/3643444431 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f188800f5d0 con 0x7f1894101c60 2024-03-21T10:18:11.315 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:11.314+0000 7f1892000700 1 --2- 172.21.15.153:0/3643444431 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f187c06e460 0x7f187c070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:18:11.315 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:11.314+0000 7f1892000700 1 -- 172.21.15.153:0/3643444431 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6654+0+0 (secure 0 0 0) 0x7f188808f970 con 0x7f1894101c60 2024-03-21T10:18:11.315 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:11.315+0000 7f1898a00700 1 --2- 172.21.15.153:0/3643444431 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f187c06e460 0x7f187c070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:18:11.316 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:11.315+0000 7f1898a00700 1 --2- 172.21.15.153:0/3643444431 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f187c06e460 0x7f187c070920 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1880005950 tx=0x7f18800058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:18:11.320 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:11.319+0000 7f1892000700 1 -- 172.21.15.153:0/3643444431 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f1888058a40 con 0x7f1894101c60 2024-03-21T10:18:11.537 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:11.535+0000 7f189be00700 1 -- 172.21.15.153:0/3643444431 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f189402cd80 con 0x7f187c06e460 2024-03-21T10:18:11.550 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:11.549+0000 7f1892000700 1 -- 172.21.15.153:0/3643444431 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f189402cd80 con 0x7f187c06e460 2024-03-21T10:18:11.550 INFO:teuthology.orchestra.run.smithi153.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-21T10:18:11.550 INFO:teuthology.orchestra.run.smithi153.stdout:alertmanager.smithi153 smithi153 *:9093,9094 running (7m) 68s ago 20m 14.5M - 0.25.0 c8568f914cd2 9ed3f2200b5f 2024-03-21T10:18:11.550 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi153 smithi153 running (21m) 68s ago 21m 8988k - 18.2.1 5be31c24972a 5b61994396a6 2024-03-21T10:18:11.550 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi174 smithi174 running (19m) 41s ago 19m 8539k - 18.2.1 5be31c24972a 4f7a5fc3ac28 2024-03-21T10:18:11.550 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi153 smithi153 running (3m) 68s ago 21m 6656k - 19.0.0-2231-g2edfee68 237e43c7a65f 32c21104d5b0 2024-03-21T10:18:11.550 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi174 smithi174 running (3m) 41s ago 19m 6660k - 19.0.0-2231-g2edfee68 237e43c7a65f a04e48967ab6 2024-03-21T10:18:11.550 INFO:teuthology.orchestra.run.smithi153.stdout:grafana.smithi153 smithi153 *:3000 running (6m) 68s ago 20m 75.4M - 9.4.12 2bacad6d85d8 ee24026ab786 2024-03-21T10:18:11.551 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.iuyluj smithi153 running (13m) 68s ago 13m 128M - 18.2.1 5be31c24972a 02f667cee979 2024-03-21T10:18:11.551 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.zipckr smithi153 running (13m) 68s ago 13m 58.8M - 18.2.1 5be31c24972a 68a853929f8f 2024-03-21T10:18:11.551 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.nhozag smithi174 running (13m) 41s ago 13m 57.5M - 18.2.1 5be31c24972a 2fee4000308f 2024-03-21T10:18:11.551 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.yyimst smithi174 running (13m) 41s ago 13m 117M - 18.2.1 5be31c24972a d78a729220d1 2024-03-21T10:18:11.551 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi153.tluxni smithi153 *:8443,9283,8765 running (9m) 68s ago 22m 505M - 19.0.0-2231-g2edfee68 237e43c7a65f 7ffee0e55382 2024-03-21T10:18:11.551 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi174.qpcwat smithi174 *:8443,9283,8765 running (8m) 41s ago 19m 451M - 19.0.0-2231-g2edfee68 237e43c7a65f 748968f216ab 2024-03-21T10:18:11.551 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi153 smithi153 running (4m) 68s ago 22m 48.2M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f dd8038303dc3 2024-03-21T10:18:11.551 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi174 smithi174 running (4m) 41s ago 19m 41.9M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f 29d78a788f2c 2024-03-21T10:18:11.551 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi153 smithi153 *:9100 running (8m) 68s ago 20m 8284k - 1.5.0 0da6a335fe13 bb835edc0e0b 2024-03-21T10:18:11.551 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi174 smithi174 *:9100 running (8m) 41s ago 19m 7834k - 1.5.0 0da6a335fe13 7c4ecad790fa 2024-03-21T10:18:11.551 INFO:teuthology.orchestra.run.smithi153.stdout:osd.0 smithi153 running (2m) 68s ago 18m 119M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 507900fd4791 2024-03-21T10:18:11.551 INFO:teuthology.orchestra.run.smithi153.stdout:osd.1 smithi153 running (108s) 68s ago 17m 86.6M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 0d646452e9c1 2024-03-21T10:18:11.552 INFO:teuthology.orchestra.run.smithi153.stdout:osd.2 smithi153 running (77s) 68s ago 17m 12.6M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 1326f2afdfcd 2024-03-21T10:18:11.552 INFO:teuthology.orchestra.run.smithi153.stdout:osd.3 smithi174 running (46s) 41s ago 16m 11.1M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 70aafac71ad5 2024-03-21T10:18:11.552 INFO:teuthology.orchestra.run.smithi153.stdout:osd.4 smithi174 starting - - - 2241M 2024-03-21T10:18:11.552 INFO:teuthology.orchestra.run.smithi153.stdout:osd.5 smithi174 running (15m) 41s ago 15m 213M 2241M 18.2.1 5be31c24972a 3b69886ce22e 2024-03-21T10:18:11.552 INFO:teuthology.orchestra.run.smithi153.stdout:prometheus.smithi153 smithi153 *:9095 running (7m) 68s ago 19m 72.2M - 2.43.0 a07b618ecd1d 7d95561e50c2 2024-03-21T10:18:11.553 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:11.551+0000 7f189be00700 1 -- 172.21.15.153:0/3643444431 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f187c06e460 msgr2=0x7f187c070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:18:11.553 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:11.551+0000 7f189be00700 1 --2- 172.21.15.153:0/3643444431 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f187c06e460 0x7f187c070920 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1880005950 tx=0x7f18800058e0 comp rx=0 tx=0).stop 2024-03-21T10:18:11.553 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:11.551+0000 7f189be00700 1 -- 172.21.15.153:0/3643444431 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1894101c60 msgr2=0x7f18940ff950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:18:11.553 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:11.551+0000 7f189be00700 1 --2- 172.21.15.153:0/3643444431 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1894101c60 0x7f18940ff950 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1888006010 tx=0x7f188800be20 comp rx=0 tx=0).stop 2024-03-21T10:18:11.553 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:11.551+0000 7f189be00700 1 -- 172.21.15.153:0/3643444431 shutdown_connections 2024-03-21T10:18:11.553 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:11.551+0000 7f189be00700 1 --2- 172.21.15.153:0/3643444431 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f187c06e460 0x7f187c070920 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:18:11.554 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:11.551+0000 7f189be00700 1 --2- 172.21.15.153:0/3643444431 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1894101c60 0x7f18940ff950 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:18:11.554 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:11.551+0000 7f189be00700 1 --2- 172.21.15.153:0/3643444431 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1894102610 0x7f18940ffe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:18:11.554 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:11.551+0000 7f189be00700 1 -- 172.21.15.153:0/3643444431 >> 172.21.15.153:0/3643444431 conn(0x7f18940fb540 msgr2=0x7f1894105340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:18:11.554 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:11.552+0000 7f189be00700 1 -- 172.21.15.153:0/3643444431 shutdown_connections 2024-03-21T10:18:11.554 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:11.552+0000 7f189be00700 1 -- 172.21.15.153:0/3643444431 wait complete. 2024-03-21T10:18:11.728 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:11.722+0000 7f7423800700 1 -- 172.21.15.153:0/1854031239 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f741c100dc0 msgr2=0x7f741c1011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:18:11.728 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:11.722+0000 7f7423800700 1 --2- 172.21.15.153:0/1854031239 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f741c100dc0 0x7f741c1011a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f7410009910 tx=0x7f7410009c20 comp rx=0 tx=0).stop 2024-03-21T10:18:11.728 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:11.722+0000 7f7423800700 1 -- 172.21.15.153:0/1854031239 shutdown_connections 2024-03-21T10:18:11.728 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:11.722+0000 7f7423800700 1 --2- 172.21.15.153:0/1854031239 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f741c101770 0x7f741c1057c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:18:11.728 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:11.723+0000 7f7423800700 1 --2- 172.21.15.153:0/1854031239 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f741c100dc0 0x7f741c1011a0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:18:11.728 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:11.723+0000 7f7423800700 1 -- 172.21.15.153:0/1854031239 >> 172.21.15.153:0/1854031239 conn(0x7f741c0fc650 msgr2=0x7f741c0fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:18:11.728 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:11.723+0000 7f7423800700 1 -- 172.21.15.153:0/1854031239 shutdown_connections 2024-03-21T10:18:11.728 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:11.723+0000 7f7423800700 1 -- 172.21.15.153:0/1854031239 wait complete. 2024-03-21T10:18:11.729 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:11.724+0000 7f7423800700 1 Processor -- start 2024-03-21T10:18:11.729 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:11.724+0000 7f7423800700 1 -- start start 2024-03-21T10:18:11.729 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:11.724+0000 7f7423800700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f741c100dc0 0x7f741c19c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:18:11.729 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:11.724+0000 7f7423800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f741c101770 0x7f741c19ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:18:11.729 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:11.724+0000 7f7423800700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f741c19d0d0 con 0x7f741c101770 2024-03-21T10:18:11.729 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:11.724+0000 7f7423800700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f741c196580 con 0x7f741c100dc0 2024-03-21T10:18:11.729 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:11.724+0000 7f741be00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f741c101770 0x7f741c19ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:18:11.729 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:11.724+0000 7f741be00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f741c101770 0x7f741c19ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:37016/0 (socket says 172.21.15.153:37016) 2024-03-21T10:18:11.729 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:11.724+0000 7f741be00700 1 -- 172.21.15.153:0/3038720287 learned_addr learned my addr 172.21.15.153:0/3038720287 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:18:11.729 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:11.725+0000 7f7420e00700 1 --2- 172.21.15.153:0/3038720287 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f741c100dc0 0x7f741c19c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:18:11.729 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:11.725+0000 7f741be00700 1 -- 172.21.15.153:0/3038720287 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f741c100dc0 msgr2=0x7f741c19c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:18:11.729 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:11.725+0000 7f741be00700 1 --2- 172.21.15.153:0/3038720287 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f741c100dc0 0x7f741c19c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:18:11.730 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:11.725+0000 7f741be00700 1 -- 172.21.15.153:0/3038720287 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74100095f0 con 0x7f741c101770 2024-03-21T10:18:11.730 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:11.725+0000 7f7420e00700 1 --2- 172.21.15.153:0/3038720287 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f741c100dc0 0x7f741c19c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:18:11.730 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:11.725+0000 7f741be00700 1 --2- 172.21.15.153:0/3038720287 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f741c101770 0x7f741c19ca40 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f740800d900 tx=0x7f740800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:18:11.730 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:11.725+0000 7f7419600700 1 -- 172.21.15.153:0/3038720287 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7408009900 con 0x7f741c101770 2024-03-21T10:18:11.730 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:11.725+0000 7f7423800700 1 -- 172.21.15.153:0/3038720287 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f741c196860 con 0x7f741c101770 2024-03-21T10:18:11.730 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:11.726+0000 7f7419600700 1 -- 172.21.15.153:0/3038720287 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7408010460 con 0x7f741c101770 2024-03-21T10:18:11.730 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:11.726+0000 7f7419600700 1 -- 172.21.15.153:0/3038720287 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f740800f5f0 con 0x7f741c101770 2024-03-21T10:18:11.730 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:11.726+0000 7f7423800700 1 -- 172.21.15.153:0/3038720287 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f741c196db0 con 0x7f741c101770 2024-03-21T10:18:11.730 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:11.727+0000 7f7423800700 1 -- 172.21.15.153:0/3038720287 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f741c109160 con 0x7f741c101770 2024-03-21T10:18:11.731 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:11.728+0000 7f7419600700 1 -- 172.21.15.153:0/3038720287 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f74080105d0 con 0x7f741c101770 2024-03-21T10:18:11.731 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:11.729+0000 7f7419600700 1 --2- 172.21.15.153:0/3038720287 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f740406e460 0x7f7404070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:18:11.731 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:11.729+0000 7f7419600700 1 -- 172.21.15.153:0/3038720287 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6654+0+0 (secure 0 0 0) 0x7f740808e360 con 0x7f741c101770 2024-03-21T10:18:11.731 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:11.729+0000 7f7420e00700 1 --2- 172.21.15.153:0/3038720287 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f740406e460 0x7f7404070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:18:11.731 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:11.729+0000 7f7420e00700 1 --2- 172.21.15.153:0/3038720287 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f740406e460 0x7f7404070920 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f741000b5c0 tx=0x7f741001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:18:11.734 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:11.733+0000 7f7419600700 1 -- 172.21.15.153:0/3038720287 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f740805af60 con 0x7f741c101770 2024-03-21T10:18:11.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:11 smithi153 ceph-mon[113733]: from='client.34368 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:18:11.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:11 smithi153 ceph-mon[113733]: from='client.34372 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:18:11.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:11 smithi153 ceph-mon[113733]: pgmap v176: 65 pgs: 16 active+undersized, 16 active+undersized+degraded, 33 active+clean; 317 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 63/366 objects degraded (17.213%) 2024-03-21T10:18:11.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:11 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:18:11.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:11 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:18:12.029 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:12.028+0000 7f7423800700 1 -- 172.21.15.153:0/3038720287 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f741c04ea90 con 0x7f741c101770 2024-03-21T10:18:12.030 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:12.029+0000 7f7419600700 1 -- 172.21.15.153:0/3038720287 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+791 (secure 0 0 0) 0x7f740805aaf0 con 0x7f741c101770 2024-03-21T10:18:12.030 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:18:12.030 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": { 2024-03-21T10:18:12.030 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:18:12.030 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:18:12.030 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": { 2024-03-21T10:18:12.030 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:18:12.031 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:18:12.031 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": { 2024-03-21T10:18:12.031 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-03-21T10:18:12.031 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 4 2024-03-21T10:18:12.031 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:18:12.031 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": { 2024-03-21T10:18:12.031 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-03-21T10:18:12.031 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:18:12.031 INFO:teuthology.orchestra.run.smithi153.stdout: "overall": { 2024-03-21T10:18:12.031 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 5, 2024-03-21T10:18:12.031 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 8 2024-03-21T10:18:12.031 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-03-21T10:18:12.031 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:18:12.033 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:12.032+0000 7f7423800700 1 -- 172.21.15.153:0/3038720287 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f740406e460 msgr2=0x7f7404070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:18:12.033 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:12.032+0000 7f7423800700 1 --2- 172.21.15.153:0/3038720287 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f740406e460 0x7f7404070920 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f741000b5c0 tx=0x7f741001a040 comp rx=0 tx=0).stop 2024-03-21T10:18:12.033 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:12.032+0000 7f7423800700 1 -- 172.21.15.153:0/3038720287 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f741c101770 msgr2=0x7f741c19ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:18:12.033 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:12.032+0000 7f7423800700 1 --2- 172.21.15.153:0/3038720287 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f741c101770 0x7f741c19ca40 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f740800d900 tx=0x7f740800dcc0 comp rx=0 tx=0).stop 2024-03-21T10:18:12.033 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:12.032+0000 7f7423800700 1 -- 172.21.15.153:0/3038720287 shutdown_connections 2024-03-21T10:18:12.033 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:12.032+0000 7f7423800700 1 --2- 172.21.15.153:0/3038720287 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f741c100dc0 0x7f741c19c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:18:12.033 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:12.032+0000 7f7423800700 1 --2- 172.21.15.153:0/3038720287 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f740406e460 0x7f7404070920 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:18:12.033 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:12.032+0000 7f7423800700 1 --2- 172.21.15.153:0/3038720287 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f741c101770 0x7f741c19ca40 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:18:12.033 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:12.032+0000 7f7423800700 1 -- 172.21.15.153:0/3038720287 >> 172.21.15.153:0/3038720287 conn(0x7f741c0fc650 msgr2=0x7f741c0fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:18:12.033 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:12.032+0000 7f7423800700 1 -- 172.21.15.153:0/3038720287 shutdown_connections 2024-03-21T10:18:12.033 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:12.032+0000 7f7423800700 1 -- 172.21.15.153:0/3038720287 wait complete. 2024-03-21T10:18:12.186 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:12.185+0000 7f036fc00700 1 -- 172.21.15.153:0/3633881414 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0368103000 msgr2=0x7f03681033e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:18:12.186 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:12.185+0000 7f036fc00700 1 --2- 172.21.15.153:0/3633881414 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0368103000 0x7f03681033e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0358009910 tx=0x7f0358009c20 comp rx=0 tx=0).stop 2024-03-21T10:18:12.186 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:12.185+0000 7f036fc00700 1 -- 172.21.15.153:0/3633881414 shutdown_connections 2024-03-21T10:18:12.187 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:12.185+0000 7f036fc00700 1 --2- 172.21.15.153:0/3633881414 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f03681039b0 0x7f0368107a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:18:12.187 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:12.185+0000 7f036fc00700 1 --2- 172.21.15.153:0/3633881414 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0368103000 0x7f03681033e0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:18:12.187 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:12.185+0000 7f036fc00700 1 -- 172.21.15.153:0/3633881414 >> 172.21.15.153:0/3633881414 conn(0x7f03680fe8d0 msgr2=0x7f0368100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:18:12.187 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:12.185+0000 7f036fc00700 1 -- 172.21.15.153:0/3633881414 shutdown_connections 2024-03-21T10:18:12.187 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:12.185+0000 7f036fc00700 1 -- 172.21.15.153:0/3633881414 wait complete. 2024-03-21T10:18:12.188 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:12.186+0000 7f036fc00700 1 Processor -- start 2024-03-21T10:18:12.188 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:12.186+0000 7f036fc00700 1 -- start start 2024-03-21T10:18:12.188 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:12.187+0000 7f036fc00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0368103000 0x7f036819e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:18:12.188 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:12.187+0000 7f036fc00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f03681039b0 0x7f036819ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:18:12.189 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:12.187+0000 7f036fc00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f036819f3c0 con 0x7f0368103000 2024-03-21T10:18:12.189 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:12.187+0000 7f036fc00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0368198870 con 0x7f03681039b0 2024-03-21T10:18:12.189 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:12.187+0000 7f036d200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0368103000 0x7f036819e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:18:12.189 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:12.187+0000 7f036d200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0368103000 0x7f036819e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:37040/0 (socket says 172.21.15.153:37040) 2024-03-21T10:18:12.189 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:12.187+0000 7f036d200700 1 -- 172.21.15.153:0/285190745 learned_addr learned my addr 172.21.15.153:0/285190745 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:18:12.189 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:12.187+0000 7f035fe00700 1 --2- 172.21.15.153:0/285190745 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f03681039b0 0x7f036819ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:18:12.189 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:12.187+0000 7f036d200700 1 -- 172.21.15.153:0/285190745 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f03681039b0 msgr2=0x7f036819ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:18:12.189 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:12.187+0000 7f036d200700 1 --2- 172.21.15.153:0/285190745 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f03681039b0 0x7f036819ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:18:12.189 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:12.187+0000 7f036d200700 1 -- 172.21.15.153:0/285190745 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03580095f0 con 0x7f0368103000 2024-03-21T10:18:12.189 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:12.188+0000 7f036d200700 1 --2- 172.21.15.153:0/285190745 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0368103000 0x7f036819e7f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0358006010 tx=0x7f035800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:18:12.190 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:12.188+0000 7f0366000700 1 -- 172.21.15.153:0/285190745 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f035801d070 con 0x7f0368103000 2024-03-21T10:18:12.190 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:12.188+0000 7f0366000700 1 -- 172.21.15.153:0/285190745 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f035800f460 con 0x7f0368103000 2024-03-21T10:18:12.190 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:12.188+0000 7f036fc00700 1 -- 172.21.15.153:0/285190745 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0368198af0 con 0x7f0368103000 2024-03-21T10:18:12.190 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:12.188+0000 7f0366000700 1 -- 172.21.15.153:0/285190745 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0358017630 con 0x7f0368103000 2024-03-21T10:18:12.190 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:12.188+0000 7f036fc00700 1 -- 172.21.15.153:0/285190745 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0368198fe0 con 0x7f0368103000 2024-03-21T10:18:12.190 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:12.190+0000 7f036fc00700 1 -- 172.21.15.153:0/285190745 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f036810b3a0 con 0x7f0368103000 2024-03-21T10:18:12.193 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:12.190+0000 7f0366000700 1 -- 172.21.15.153:0/285190745 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f035800f5d0 con 0x7f0368103000 2024-03-21T10:18:12.194 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:12.192+0000 7f0366000700 1 --2- 172.21.15.153:0/285190745 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f034c06e460 0x7f034c070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:18:12.194 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:12.192+0000 7f0366000700 1 -- 172.21.15.153:0/285190745 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6654+0+0 (secure 0 0 0) 0x7f035808f900 con 0x7f0368103000 2024-03-21T10:18:12.194 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:12.192+0000 7f035fe00700 1 --2- 172.21.15.153:0/285190745 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f034c06e460 0x7f034c070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:18:12.194 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:12.193+0000 7f035fe00700 1 --2- 172.21.15.153:0/285190745 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f034c06e460 0x7f034c070920 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0350005950 tx=0x7f03500058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:18:12.197 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:12.197+0000 7f0366000700 1 -- 172.21.15.153:0/285190745 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f035805c5a0 con 0x7f0368103000 2024-03-21T10:18:12.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:11 smithi174 ceph-mon[88004]: from='client.34368 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:18:12.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:11 smithi174 ceph-mon[88004]: from='client.34372 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:18:12.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:11 smithi174 ceph-mon[88004]: pgmap v176: 65 pgs: 16 active+undersized, 16 active+undersized+degraded, 33 active+clean; 317 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 63/366 objects degraded (17.213%) 2024-03-21T10:18:12.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:11 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:18:12.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:11 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:18:12.448 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:12.447+0000 7f036fc00700 1 -- 172.21.15.153:0/285190745 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f036810ad80 con 0x7f0368103000 2024-03-21T10:18:12.449 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:12.448+0000 7f0366000700 1 -- 172.21.15.153:0/285190745 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 21 v21) v1 ==== 76+0+1903 (secure 0 0 0) 0x7f035805c130 con 0x7f0368103000 2024-03-21T10:18:12.449 INFO:teuthology.orchestra.run.smithi153.stdout:e21 2024-03-21T10:18:12.449 INFO:teuthology.orchestra.run.smithi153.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-21T10:18:12.449 INFO:teuthology.orchestra.run.smithi153.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:18:12.449 INFO:teuthology.orchestra.run.smithi153.stdout:legacy client fscid: 1 2024-03-21T10:18:12.449 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:18:12.450 INFO:teuthology.orchestra.run.smithi153.stdout:Filesystem 'cephfs' (1) 2024-03-21T10:18:12.450 INFO:teuthology.orchestra.run.smithi153.stdout:fs_name cephfs 2024-03-21T10:18:12.450 INFO:teuthology.orchestra.run.smithi153.stdout:epoch 21 2024-03-21T10:18:12.450 INFO:teuthology.orchestra.run.smithi153.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-21T10:18:12.450 INFO:teuthology.orchestra.run.smithi153.stdout:created 2024-03-21T10:04:27.086576+0000 2024-03-21T10:18:12.450 INFO:teuthology.orchestra.run.smithi153.stdout:modified 2024-03-21T10:13:01.455594+0000 2024-03-21T10:18:12.450 INFO:teuthology.orchestra.run.smithi153.stdout:tableserver 0 2024-03-21T10:18:12.450 INFO:teuthology.orchestra.run.smithi153.stdout:root 0 2024-03-21T10:18:12.450 INFO:teuthology.orchestra.run.smithi153.stdout:session_timeout 60 2024-03-21T10:18:12.450 INFO:teuthology.orchestra.run.smithi153.stdout:session_autoclose 300 2024-03-21T10:18:12.450 INFO:teuthology.orchestra.run.smithi153.stdout:max_file_size 1099511627776 2024-03-21T10:18:12.450 INFO:teuthology.orchestra.run.smithi153.stdout:max_xattr_size 65536 2024-03-21T10:18:12.450 INFO:teuthology.orchestra.run.smithi153.stdout:required_client_features {} 2024-03-21T10:18:12.450 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure 0 2024-03-21T10:18:12.451 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure_osd_epoch 47 2024-03-21T10:18:12.451 INFO:teuthology.orchestra.run.smithi153.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:18:12.451 INFO:teuthology.orchestra.run.smithi153.stdout:max_mds 2 2024-03-21T10:18:12.451 INFO:teuthology.orchestra.run.smithi153.stdout:in 0,1 2024-03-21T10:18:12.451 INFO:teuthology.orchestra.run.smithi153.stdout:up {0=14484,1=24273} 2024-03-21T10:18:12.451 INFO:teuthology.orchestra.run.smithi153.stdout:failed 2024-03-21T10:18:12.451 INFO:teuthology.orchestra.run.smithi153.stdout:damaged 2024-03-21T10:18:12.451 INFO:teuthology.orchestra.run.smithi153.stdout:stopped 2024-03-21T10:18:12.451 INFO:teuthology.orchestra.run.smithi153.stdout:data_pools [3] 2024-03-21T10:18:12.451 INFO:teuthology.orchestra.run.smithi153.stdout:metadata_pool 2 2024-03-21T10:18:12.451 INFO:teuthology.orchestra.run.smithi153.stdout:inline_data enabled 2024-03-21T10:18:12.451 INFO:teuthology.orchestra.run.smithi153.stdout:balancer 2024-03-21T10:18:12.451 INFO:teuthology.orchestra.run.smithi153.stdout:bal_rank_mask -1 2024-03-21T10:18:12.451 INFO:teuthology.orchestra.run.smithi153.stdout:standby_count_wanted 1 2024-03-21T10:18:12.452 INFO:teuthology.orchestra.run.smithi153.stdout:qdb_cluster leader: 0 members: 2024-03-21T10:18:12.452 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.iuyluj{0:14484} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.153:6828/866138999,v1:172.21.15.153:6829/866138999] compat {c=[1],r=[1],i=[7ff]}] 2024-03-21T10:18:12.452 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.zipckr{0:14494} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.153:6826/4033622615,v1:172.21.15.153:6827/4033622615] compat {c=[1],r=[1],i=[7ff]}] 2024-03-21T10:18:12.452 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.yyimst{1:24273} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.174:6824/1776947618,v1:172.21.15.174:6825/1776947618] compat {c=[1],r=[1],i=[7ff]}] 2024-03-21T10:18:12.452 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.nhozag{1:24285} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.174:6826/806095873,v1:172.21.15.174:6827/806095873] compat {c=[1],r=[1],i=[7ff]}] 2024-03-21T10:18:12.452 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:18:12.452 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:18:12.453 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:12.451+0000 7f036fc00700 1 -- 172.21.15.153:0/285190745 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f034c06e460 msgr2=0x7f034c070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:18:12.453 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:12.451+0000 7f036fc00700 1 --2- 172.21.15.153:0/285190745 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f034c06e460 0x7f034c070920 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0350005950 tx=0x7f03500058e0 comp rx=0 tx=0).stop 2024-03-21T10:18:12.453 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:12.451+0000 7f036fc00700 1 -- 172.21.15.153:0/285190745 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0368103000 msgr2=0x7f036819e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:18:12.453 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:12.451+0000 7f036fc00700 1 --2- 172.21.15.153:0/285190745 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0368103000 0x7f036819e7f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0358006010 tx=0x7f035800be20 comp rx=0 tx=0).stop 2024-03-21T10:18:12.453 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:12.451+0000 7f036fc00700 1 -- 172.21.15.153:0/285190745 shutdown_connections 2024-03-21T10:18:12.453 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:12.451+0000 7f036fc00700 1 --2- 172.21.15.153:0/285190745 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f034c06e460 0x7f034c070920 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:18:12.453 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:12.451+0000 7f036fc00700 1 --2- 172.21.15.153:0/285190745 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0368103000 0x7f036819e7f0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:18:12.454 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:12.451+0000 7f036fc00700 1 --2- 172.21.15.153:0/285190745 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f03681039b0 0x7f036819ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:18:12.454 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:12.451+0000 7f036fc00700 1 -- 172.21.15.153:0/285190745 >> 172.21.15.153:0/285190745 conn(0x7f03680fe8d0 msgr2=0x7f03680ffee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:18:12.454 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:12.451+0000 7f036fc00700 1 -- 172.21.15.153:0/285190745 shutdown_connections 2024-03-21T10:18:12.454 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:12.451+0000 7f036fc00700 1 -- 172.21.15.153:0/285190745 wait complete. 2024-03-21T10:18:12.454 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 21 2024-03-21T10:18:12.605 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:12.604+0000 7f2880c00700 1 -- 172.21.15.153:0/1312771242 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f287c1071d0 msgr2=0x7f287c107650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:18:12.605 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:12.604+0000 7f2880c00700 1 --2- 172.21.15.153:0/1312771242 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f287c1071d0 0x7f287c107650 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2868009910 tx=0x7f2868009c20 comp rx=0 tx=0).stop 2024-03-21T10:18:12.605 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:12.604+0000 7f2880c00700 1 -- 172.21.15.153:0/1312771242 shutdown_connections 2024-03-21T10:18:12.605 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:12.604+0000 7f2880c00700 1 --2- 172.21.15.153:0/1312771242 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f287c1071d0 0x7f287c107650 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:18:12.605 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:12.604+0000 7f2880c00700 1 --2- 172.21.15.153:0/1312771242 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f287c10d280 0x7f287c10d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:18:12.605 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:12.604+0000 7f2880c00700 1 -- 172.21.15.153:0/1312771242 >> 172.21.15.153:0/1312771242 conn(0x7f287c0769e0 msgr2=0x7f287c078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:18:12.605 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:12.604+0000 7f2880c00700 1 -- 172.21.15.153:0/1312771242 shutdown_connections 2024-03-21T10:18:12.606 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:12.604+0000 7f2880c00700 1 -- 172.21.15.153:0/1312771242 wait complete. 2024-03-21T10:18:12.606 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:12.606+0000 7f2880c00700 1 Processor -- start 2024-03-21T10:18:12.606 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:12.606+0000 7f2880c00700 1 -- start start 2024-03-21T10:18:12.607 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:12.606+0000 7f2880c00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f287c1071d0 0x7f287c19cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:18:12.607 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:12.606+0000 7f2880c00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f287c10d280 0x7f287c19d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:18:12.607 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:12.606+0000 7f2880c00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f287c19d9b0 con 0x7f287c10d280 2024-03-21T10:18:12.607 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:12.606+0000 7f2880c00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f287c1a1740 con 0x7f287c1071d0 2024-03-21T10:18:12.608 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:12.607+0000 7f2879400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f287c10d280 0x7f287c19d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:18:12.608 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:12.607+0000 7f2879400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f287c10d280 0x7f287c19d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:37062/0 (socket says 172.21.15.153:37062) 2024-03-21T10:18:12.608 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:12.607+0000 7f2879400700 1 -- 172.21.15.153:0/3797982238 learned_addr learned my addr 172.21.15.153:0/3797982238 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:18:12.608 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:12.607+0000 7f2879e00700 1 --2- 172.21.15.153:0/3797982238 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f287c1071d0 0x7f287c19cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:18:12.609 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:12.607+0000 7f2879400700 1 -- 172.21.15.153:0/3797982238 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f287c1071d0 msgr2=0x7f287c19cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:18:12.609 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:12.607+0000 7f2879400700 1 --2- 172.21.15.153:0/3797982238 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f287c1071d0 0x7f287c19cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:18:12.609 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:12.607+0000 7f2879400700 1 -- 172.21.15.153:0/3797982238 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28680095f0 con 0x7f287c10d280 2024-03-21T10:18:12.609 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:12.607+0000 7f2879400700 1 --2- 172.21.15.153:0/3797982238 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f287c10d280 0x7f287c19d2d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f286800b5c0 tx=0x7f286800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:18:12.609 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:12.608+0000 7f2872a00700 1 -- 172.21.15.153:0/3797982238 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f286801d070 con 0x7f287c10d280 2024-03-21T10:18:12.609 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:12.608+0000 7f2872a00700 1 -- 172.21.15.153:0/3797982238 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2868005e00 con 0x7f287c10d280 2024-03-21T10:18:12.609 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:12.608+0000 7f2872a00700 1 -- 172.21.15.153:0/3797982238 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2868017400 con 0x7f287c10d280 2024-03-21T10:18:12.609 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:12.608+0000 7f2880c00700 1 -- 172.21.15.153:0/3797982238 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f287c1a19c0 con 0x7f287c10d280 2024-03-21T10:18:12.609 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:12.608+0000 7f2880c00700 1 -- 172.21.15.153:0/3797982238 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f287c1a1eb0 con 0x7f287c10d280 2024-03-21T10:18:12.609 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:12.609+0000 7f2880c00700 1 -- 172.21.15.153:0/3797982238 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f287c04ea90 con 0x7f287c10d280 2024-03-21T10:18:12.610 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:12.610+0000 7f2872a00700 1 -- 172.21.15.153:0/3797982238 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f286800f460 con 0x7f287c10d280 2024-03-21T10:18:12.612 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:12.611+0000 7f2872a00700 1 --2- 172.21.15.153:0/3797982238 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f286006e460 0x7f2860070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:18:12.612 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:12.611+0000 7f2872a00700 1 -- 172.21.15.153:0/3797982238 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6654+0+0 (secure 0 0 0) 0x7f286808f9c0 con 0x7f287c10d280 2024-03-21T10:18:12.612 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:12.611+0000 7f2879e00700 1 --2- 172.21.15.153:0/3797982238 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f286006e460 0x7f2860070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:18:12.612 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:12.612+0000 7f2879e00700 1 --2- 172.21.15.153:0/3797982238 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f286006e460 0x7f2860070920 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f286c005950 tx=0x7f286c0058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:18:12.616 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:12.616+0000 7f2872a00700 1 -- 172.21.15.153:0/3797982238 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2868058a90 con 0x7f287c10d280 2024-03-21T10:18:12.878 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:12.877+0000 7f2880c00700 1 -- 172.21.15.153:0/3797982238 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f287c1a2450 con 0x7f286006e460 2024-03-21T10:18:12.878 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:12 smithi153 ceph-mon[113733]: from='client.34376 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:18:12.878 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:12 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/3038720287' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:18:12.878 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:12 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/285190745' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:18:12.879 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:12.879+0000 7f2872a00700 1 -- 172.21.15.153:0/3797982238 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f287c1a2450 con 0x7f286006e460 2024-03-21T10:18:12.880 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:18:12.880 INFO:teuthology.orchestra.run.smithi153.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232", 2024-03-21T10:18:12.880 INFO:teuthology.orchestra.run.smithi153.stdout: "in_progress": true, 2024-03-21T10:18:12.880 INFO:teuthology.orchestra.run.smithi153.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-21T10:18:12.880 INFO:teuthology.orchestra.run.smithi153.stdout: "services_complete": [ 2024-03-21T10:18:12.880 INFO:teuthology.orchestra.run.smithi153.stdout: "crash", 2024-03-21T10:18:12.880 INFO:teuthology.orchestra.run.smithi153.stdout: "mon", 2024-03-21T10:18:12.880 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr" 2024-03-21T10:18:12.880 INFO:teuthology.orchestra.run.smithi153.stdout: ], 2024-03-21T10:18:12.880 INFO:teuthology.orchestra.run.smithi153.stdout: "progress": "11/23 daemons upgraded", 2024-03-21T10:18:12.880 INFO:teuthology.orchestra.run.smithi153.stdout: "message": "Currently upgrading osd daemons", 2024-03-21T10:18:12.881 INFO:teuthology.orchestra.run.smithi153.stdout: "is_paused": false 2024-03-21T10:18:12.881 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:18:12.882 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:12.881+0000 7f2880c00700 1 -- 172.21.15.153:0/3797982238 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f286006e460 msgr2=0x7f2860070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:18:12.883 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:12.881+0000 7f2880c00700 1 --2- 172.21.15.153:0/3797982238 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f286006e460 0x7f2860070920 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f286c005950 tx=0x7f286c0058e0 comp rx=0 tx=0).stop 2024-03-21T10:18:12.883 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:12.881+0000 7f2880c00700 1 -- 172.21.15.153:0/3797982238 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f287c10d280 msgr2=0x7f287c19d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:18:12.883 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:12.882+0000 7f2880c00700 1 --2- 172.21.15.153:0/3797982238 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f287c10d280 0x7f287c19d2d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f286800b5c0 tx=0x7f286800be20 comp rx=0 tx=0).stop 2024-03-21T10:18:12.883 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:12.882+0000 7f2880c00700 1 -- 172.21.15.153:0/3797982238 shutdown_connections 2024-03-21T10:18:12.883 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:12.882+0000 7f2880c00700 1 --2- 172.21.15.153:0/3797982238 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f287c1071d0 0x7f287c19cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:18:12.883 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:12.882+0000 7f2880c00700 1 --2- 172.21.15.153:0/3797982238 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f286006e460 0x7f2860070920 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:18:12.883 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:12.882+0000 7f2880c00700 1 --2- 172.21.15.153:0/3797982238 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f287c10d280 0x7f287c19d2d0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:18:12.883 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:12.882+0000 7f2880c00700 1 -- 172.21.15.153:0/3797982238 >> 172.21.15.153:0/3797982238 conn(0x7f287c0769e0 msgr2=0x7f287c078480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:18:12.883 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:12.882+0000 7f2880c00700 1 -- 172.21.15.153:0/3797982238 shutdown_connections 2024-03-21T10:18:12.883 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:12.882+0000 7f2880c00700 1 -- 172.21.15.153:0/3797982238 wait complete. 2024-03-21T10:18:13.034 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:13.033+0000 7fb037400700 1 -- 172.21.15.153:0/3303001418 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb030103000 msgr2=0x7fb0301033e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:18:13.034 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:13.033+0000 7fb037400700 1 --2- 172.21.15.153:0/3303001418 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb030103000 0x7fb0301033e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb024009910 tx=0x7fb024009c20 comp rx=0 tx=0).stop 2024-03-21T10:18:13.034 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:13.033+0000 7fb037400700 1 -- 172.21.15.153:0/3303001418 shutdown_connections 2024-03-21T10:18:13.034 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:13.033+0000 7fb037400700 1 --2- 172.21.15.153:0/3303001418 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0301039b0 0x7fb030107a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:18:13.034 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:13.033+0000 7fb037400700 1 --2- 172.21.15.153:0/3303001418 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb030103000 0x7fb0301033e0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:18:13.034 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:13.033+0000 7fb037400700 1 -- 172.21.15.153:0/3303001418 >> 172.21.15.153:0/3303001418 conn(0x7fb0300fe8d0 msgr2=0x7fb030100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:18:13.034 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:13.033+0000 7fb037400700 1 -- 172.21.15.153:0/3303001418 shutdown_connections 2024-03-21T10:18:13.034 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:13.033+0000 7fb037400700 1 -- 172.21.15.153:0/3303001418 wait complete. 2024-03-21T10:18:13.036 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:13.034+0000 7fb037400700 1 Processor -- start 2024-03-21T10:18:13.036 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:13.034+0000 7fb037400700 1 -- start start 2024-03-21T10:18:13.036 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:13.034+0000 7fb037400700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb030103000 0x7fb03019e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:18:13.036 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:13.034+0000 7fb037400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb0301039b0 0x7fb03019ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:18:13.036 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:13.034+0000 7fb037400700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb03019f3c0 con 0x7fb0301039b0 2024-03-21T10:18:13.036 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:13.034+0000 7fb037400700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb030198870 con 0x7fb030103000 2024-03-21T10:18:13.036 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:13.035+0000 7fb02fe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb0301039b0 0x7fb03019ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:18:13.036 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:13.035+0000 7fb034a00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb030103000 0x7fb03019e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:18:13.036 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:13.035+0000 7fb034a00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb030103000 0x7fb03019e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.153:51904/0 (socket says 172.21.15.153:51904) 2024-03-21T10:18:13.036 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:13.035+0000 7fb02fe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb0301039b0 0x7fb03019ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:44820/0 (socket says 172.21.15.153:44820) 2024-03-21T10:18:13.036 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:13.035+0000 7fb034a00700 1 -- 172.21.15.153:0/1177299262 learned_addr learned my addr 172.21.15.153:0/1177299262 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:18:13.036 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:13.035+0000 7fb02fe00700 1 -- 172.21.15.153:0/1177299262 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb030103000 msgr2=0x7fb03019e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:18:13.036 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:13.035+0000 7fb02fe00700 1 --2- 172.21.15.153:0/1177299262 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb030103000 0x7fb03019e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:18:13.036 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:13.035+0000 7fb02fe00700 1 -- 172.21.15.153:0/1177299262 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0240095f0 con 0x7fb0301039b0 2024-03-21T10:18:13.037 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:13.035+0000 7fb02fe00700 1 --2- 172.21.15.153:0/1177299262 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb0301039b0 0x7fb03019ed30 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb01c00e940 tx=0x7fb01c00ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:18:13.037 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:13.036+0000 7fb02d600700 1 -- 172.21.15.153:0/1177299262 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb01c00c990 con 0x7fb0301039b0 2024-03-21T10:18:13.037 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:13.036+0000 7fb02d600700 1 -- 172.21.15.153:0/1177299262 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb01c010430 con 0x7fb0301039b0 2024-03-21T10:18:13.037 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:13.036+0000 7fb02d600700 1 -- 172.21.15.153:0/1177299262 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb01c018650 con 0x7fb0301039b0 2024-03-21T10:18:13.037 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:13.036+0000 7fb037400700 1 -- 172.21.15.153:0/1177299262 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb030198b50 con 0x7fb0301039b0 2024-03-21T10:18:13.038 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:13.036+0000 7fb037400700 1 -- 172.21.15.153:0/1177299262 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0301990a0 con 0x7fb0301039b0 2024-03-21T10:18:13.038 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:13.037+0000 7fb037400700 1 -- 172.21.15.153:0/1177299262 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb03010b3a0 con 0x7fb0301039b0 2024-03-21T10:18:13.038 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:13.038+0000 7fb02d600700 1 -- 172.21.15.153:0/1177299262 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fb01c0105a0 con 0x7fb0301039b0 2024-03-21T10:18:13.040 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:13.039+0000 7fb02d600700 1 --2- 172.21.15.153:0/1177299262 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb01806e460 0x7fb018070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:18:13.040 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:13.039+0000 7fb02d600700 1 -- 172.21.15.153:0/1177299262 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6654+0+0 (secure 0 0 0) 0x7fb01c014070 con 0x7fb0301039b0 2024-03-21T10:18:13.040 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:13.039+0000 7fb034a00700 1 --2- 172.21.15.153:0/1177299262 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb01806e460 0x7fb018070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:18:13.040 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:13.040+0000 7fb034a00700 1 --2- 172.21.15.153:0/1177299262 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb01806e460 0x7fb018070920 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb02400b760 tx=0x7fb024005a90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:18:13.044 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:13.043+0000 7fb02d600700 1 -- 172.21.15.153:0/1177299262 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb01c053740 con 0x7fb0301039b0 2024-03-21T10:18:13.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:12 smithi174 ceph-mon[88004]: from='client.34376 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:18:13.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:12 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/3038720287' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:18:13.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:12 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/285190745' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:18:13.202 INFO:journalctl@ceph.osd.4.smithi174.stdout:Mar 21 10:18:12 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-4[102038]: 2024-03-21T10:18:12.910+0000 7f19c7b2b7c0 -1 osd.4 0 read_superblock omap replica is missing. 2024-03-21T10:18:13.353 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:13.352+0000 7fb037400700 1 -- 172.21.15.153:0/1177299262 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fb03004ea90 con 0x7fb0301039b0 2024-03-21T10:18:13.354 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:13.353+0000 7fb02d600700 1 -- 172.21.15.153:0/1177299262 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+1352 (secure 0 0 0) 0x7fb01c05b310 con 0x7fb0301039b0 2024-03-21T10:18:13.354 INFO:teuthology.orchestra.run.smithi153.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data; 1 osds down; Degraded data redundancy: 63/366 objects degraded (17.213%), 16 pgs degraded 2024-03-21T10:18:13.354 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-21T10:18:13.354 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-21T10:18:13.354 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] OSD_DOWN: 1 osds down 2024-03-21T10:18:13.354 INFO:teuthology.orchestra.run.smithi153.stdout: osd.4 (root=default,host=smithi174) is down 2024-03-21T10:18:13.355 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 63/366 objects degraded (17.213%), 16 pgs degraded 2024-03-21T10:18:13.355 INFO:teuthology.orchestra.run.smithi153.stdout: pg 2.4 is active+undersized+degraded, acting [1,0] 2024-03-21T10:18:13.355 INFO:teuthology.orchestra.run.smithi153.stdout: pg 2.5 is active+undersized+degraded, acting [3,0] 2024-03-21T10:18:13.355 INFO:teuthology.orchestra.run.smithi153.stdout: pg 2.6 is active+undersized+degraded, acting [1,3] 2024-03-21T10:18:13.355 INFO:teuthology.orchestra.run.smithi153.stdout: pg 2.7 is active+undersized+degraded, acting [3,2] 2024-03-21T10:18:13.355 INFO:teuthology.orchestra.run.smithi153.stdout: pg 2.9 is active+undersized+degraded, acting [1,0] 2024-03-21T10:18:13.355 INFO:teuthology.orchestra.run.smithi153.stdout: pg 2.a is active+undersized+degraded, acting [1,3] 2024-03-21T10:18:13.355 INFO:teuthology.orchestra.run.smithi153.stdout: pg 2.b is active+undersized+degraded, acting [3,5] 2024-03-21T10:18:13.355 INFO:teuthology.orchestra.run.smithi153.stdout: pg 2.f is active+undersized+degraded, acting [0,5] 2024-03-21T10:18:13.355 INFO:teuthology.orchestra.run.smithi153.stdout: pg 2.11 is active+undersized+degraded, acting [3,1] 2024-03-21T10:18:13.355 INFO:teuthology.orchestra.run.smithi153.stdout: pg 2.13 is active+undersized+degraded, acting [0,2] 2024-03-21T10:18:13.355 INFO:teuthology.orchestra.run.smithi153.stdout: pg 2.14 is active+undersized+degraded, acting [0,5] 2024-03-21T10:18:13.356 INFO:teuthology.orchestra.run.smithi153.stdout: pg 2.18 is active+undersized+degraded, acting [5,3] 2024-03-21T10:18:13.356 INFO:teuthology.orchestra.run.smithi153.stdout: pg 2.19 is active+undersized+degraded, acting [0,2] 2024-03-21T10:18:13.356 INFO:teuthology.orchestra.run.smithi153.stdout: pg 2.1a is active+undersized+degraded, acting [3,5] 2024-03-21T10:18:13.356 INFO:teuthology.orchestra.run.smithi153.stdout: pg 2.1c is active+undersized+degraded, acting [5,2] 2024-03-21T10:18:13.356 INFO:teuthology.orchestra.run.smithi153.stdout: pg 2.1f is active+undersized+degraded, acting [0,3] 2024-03-21T10:18:13.357 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:13.356+0000 7fb037400700 1 -- 172.21.15.153:0/1177299262 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb01806e460 msgr2=0x7fb018070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:18:13.357 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:13.356+0000 7fb037400700 1 --2- 172.21.15.153:0/1177299262 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb01806e460 0x7fb018070920 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb02400b760 tx=0x7fb024005a90 comp rx=0 tx=0).stop 2024-03-21T10:18:13.357 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:13.356+0000 7fb037400700 1 -- 172.21.15.153:0/1177299262 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb0301039b0 msgr2=0x7fb03019ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:18:13.357 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:13.356+0000 7fb037400700 1 --2- 172.21.15.153:0/1177299262 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb0301039b0 0x7fb03019ed30 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb01c00e940 tx=0x7fb01c00ec50 comp rx=0 tx=0).stop 2024-03-21T10:18:13.357 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:13.356+0000 7fb037400700 1 -- 172.21.15.153:0/1177299262 shutdown_connections 2024-03-21T10:18:13.357 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:13.356+0000 7fb037400700 1 --2- 172.21.15.153:0/1177299262 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb030103000 0x7fb03019e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:18:13.357 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:13.356+0000 7fb037400700 1 --2- 172.21.15.153:0/1177299262 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb01806e460 0x7fb018070920 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:18:13.357 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:13.356+0000 7fb037400700 1 --2- 172.21.15.153:0/1177299262 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb0301039b0 0x7fb03019ed30 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:18:13.357 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:13.356+0000 7fb037400700 1 -- 172.21.15.153:0/1177299262 >> 172.21.15.153:0/1177299262 conn(0x7fb0300fe8d0 msgr2=0x7fb0300ffee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:18:13.358 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:13.356+0000 7fb037400700 1 -- 172.21.15.153:0/1177299262 shutdown_connections 2024-03-21T10:18:13.358 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:13.356+0000 7fb037400700 1 -- 172.21.15.153:0/1177299262 wait complete. 2024-03-21T10:18:13.701 INFO:journalctl@ceph.osd.4.smithi174.stdout:Mar 21 10:18:13 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-4[102038]: 2024-03-21T10:18:13.341+0000 7f19c7b2b7c0 -1 osd.4 81 log_to_monitors true 2024-03-21T10:18:14.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:13 smithi153 ceph-mon[113733]: from='client.34388 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:18:14.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:13 smithi153 ceph-mon[113733]: pgmap v177: 65 pgs: 16 active+undersized, 16 active+undersized+degraded, 33 active+clean; 317 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 63/366 objects degraded (17.213%) 2024-03-21T10:18:14.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:13 smithi153 ceph-mon[113733]: from='osd.4 [v2:172.21.15.174:6808/1143789116,v1:172.21.15.174:6809/1143789116]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-21T10:18:14.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:13 smithi153 ceph-mon[113733]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-21T10:18:14.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:13 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/1177299262' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:18:14.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:13 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:18:14.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:13 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:18:14.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:13 smithi174 ceph-mon[88004]: from='client.34388 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:18:14.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:13 smithi174 ceph-mon[88004]: pgmap v177: 65 pgs: 16 active+undersized, 16 active+undersized+degraded, 33 active+clean; 317 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 63/366 objects degraded (17.213%) 2024-03-21T10:18:14.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:13 smithi174 ceph-mon[88004]: from='osd.4 [v2:172.21.15.174:6808/1143789116,v1:172.21.15.174:6809/1143789116]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-21T10:18:14.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:13 smithi174 ceph-mon[88004]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-21T10:18:14.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:13 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/1177299262' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:18:14.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:13 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:18:14.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:13 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:18:14.865 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:14 smithi174 ceph-mon[88004]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-21T10:18:14.865 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:14 smithi174 ceph-mon[88004]: from='osd.4 [v2:172.21.15.174:6808/1143789116,v1:172.21.15.174:6809/1143789116]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-03-21T10:18:14.865 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:14 smithi174 ceph-mon[88004]: osdmap e84: 6 total, 5 up, 6 in 2024-03-21T10:18:14.865 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:14 smithi174 ceph-mon[88004]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-03-21T10:18:14.866 INFO:journalctl@ceph.osd.4.smithi174.stdout:Mar 21 10:18:14 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-4[102038]: 2024-03-21T10:18:14.863+0000 7f19b1e00700 -1 osd.4 81 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-21T10:18:15.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:14 smithi153 ceph-mon[113733]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-21T10:18:15.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:14 smithi153 ceph-mon[113733]: from='osd.4 [v2:172.21.15.174:6808/1143789116,v1:172.21.15.174:6809/1143789116]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-03-21T10:18:15.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:14 smithi153 ceph-mon[113733]: osdmap e84: 6 total, 5 up, 6 in 2024-03-21T10:18:15.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:14 smithi153 ceph-mon[113733]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-03-21T10:18:15.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:15 smithi174 ceph-mon[88004]: from='osd.4 ' entity='osd.4' 2024-03-21T10:18:15.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:15 smithi174 ceph-mon[88004]: pgmap v179: 65 pgs: 16 active+undersized, 16 active+undersized+degraded, 33 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 2 op/s; 63/366 objects degraded (17.213%) 2024-03-21T10:18:16.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:15 smithi153 ceph-mon[113733]: from='osd.4 ' entity='osd.4' 2024-03-21T10:18:16.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:15 smithi153 ceph-mon[113733]: pgmap v179: 65 pgs: 16 active+undersized, 16 active+undersized+degraded, 33 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 2 op/s; 63/366 objects degraded (17.213%) 2024-03-21T10:18:17.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:16 smithi174 ceph-mon[88004]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-21T10:18:17.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:16 smithi174 ceph-mon[88004]: osd.4 [v2:172.21.15.174:6808/1143789116,v1:172.21.15.174:6809/1143789116] boot 2024-03-21T10:18:17.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:16 smithi174 ceph-mon[88004]: osdmap e85: 6 total, 6 up, 6 in 2024-03-21T10:18:17.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:16 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T10:18:17.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:16 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:18:17.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:16 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:18:17.308 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:16 smithi153 ceph-mon[113733]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-21T10:18:17.308 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:16 smithi153 ceph-mon[113733]: osd.4 [v2:172.21.15.174:6808/1143789116,v1:172.21.15.174:6809/1143789116] boot 2024-03-21T10:18:17.308 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:16 smithi153 ceph-mon[113733]: osdmap e85: 6 total, 6 up, 6 in 2024-03-21T10:18:17.309 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:16 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T10:18:17.309 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:16 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:18:17.309 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:16 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:18:18.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:17 smithi174 ceph-mon[88004]: osdmap e86: 6 total, 6 up, 6 in 2024-03-21T10:18:18.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:17 smithi174 ceph-mon[88004]: pgmap v182: 65 pgs: 14 active+undersized, 15 active+undersized+degraded, 36 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s; 60/366 objects degraded (16.393%) 2024-03-21T10:18:18.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:17 smithi153 ceph-mon[113733]: osdmap e86: 6 total, 6 up, 6 in 2024-03-21T10:18:18.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:17 smithi153 ceph-mon[113733]: pgmap v182: 65 pgs: 14 active+undersized, 15 active+undersized+degraded, 36 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s; 60/366 objects degraded (16.393%) 2024-03-21T10:18:19.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:18 smithi174 ceph-mon[88004]: Health check update: Degraded data redundancy: 60/366 objects degraded (16.393%), 15 pgs degraded (PG_DEGRADED) 2024-03-21T10:18:19.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:18 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:18:19.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:18 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:18:19.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:18 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:18:19.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:18 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:18:19.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:18 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:18:19.224 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:18 smithi153 ceph-mon[113733]: Health check update: Degraded data redundancy: 60/366 objects degraded (16.393%), 15 pgs degraded (PG_DEGRADED) 2024-03-21T10:18:19.224 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:18 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:18:19.224 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:18 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:18:19.224 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:18 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:18:19.224 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:18 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:18:19.224 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:18 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:18:20.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:19 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:18:20.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:19 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:18:20.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:19 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:18:20.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:19 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:18:20.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:19 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-21T10:18:20.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:19 smithi174 ceph-mon[88004]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-21T10:18:20.203 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:19 smithi174 ceph-mon[88004]: pgmap v183: 65 pgs: 7 active+undersized, 8 active+undersized+degraded, 50 active+clean; 317 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s; 36/366 objects degraded (9.836%) 2024-03-21T10:18:20.203 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:19 smithi174 ceph-mon[88004]: Upgrade: unsafe to stop osd(s) at this time (8 PGs are or would become offline) 2024-03-21T10:18:20.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:19 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:18:20.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:19 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:18:20.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:19 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:18:20.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:19 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:18:20.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:19 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-21T10:18:20.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:19 smithi153 ceph-mon[113733]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-21T10:18:20.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:19 smithi153 ceph-mon[113733]: pgmap v183: 65 pgs: 7 active+undersized, 8 active+undersized+degraded, 50 active+clean; 317 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s; 36/366 objects degraded (9.836%) 2024-03-21T10:18:20.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:19 smithi153 ceph-mon[113733]: Upgrade: unsafe to stop osd(s) at this time (8 PGs are or would become offline) 2024-03-21T10:18:21.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:21 smithi174 ceph-mon[88004]: pgmap v184: 65 pgs: 65 active+clean; 317 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:18:21.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:21 smithi174 ceph-mon[88004]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 36/366 objects degraded (9.836%), 8 pgs degraded) 2024-03-21T10:18:21.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:21 smithi153 ceph-mon[113733]: pgmap v184: 65 pgs: 65 active+clean; 317 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:18:21.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:21 smithi153 ceph-mon[113733]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 36/366 objects degraded (9.836%), 8 pgs degraded) 2024-03-21T10:18:23.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:23 smithi174 ceph-mon[88004]: pgmap v185: 65 pgs: 65 active+clean; 317 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-21T10:18:23.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:23 smithi153 ceph-mon[113733]: pgmap v185: 65 pgs: 65 active+clean; 317 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-21T10:18:25.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:25 smithi174 ceph-mon[88004]: pgmap v186: 65 pgs: 65 active+clean; 317 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 2 op/s 2024-03-21T10:18:25.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:25 smithi153 ceph-mon[113733]: pgmap v186: 65 pgs: 65 active+clean; 317 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 2 op/s 2024-03-21T10:18:27.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:27 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:18:27.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:27 smithi174 ceph-mon[88004]: pgmap v187: 65 pgs: 65 active+clean; 317 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 3 op/s 2024-03-21T10:18:27.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:27 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:18:27.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:27 smithi153 ceph-mon[113733]: pgmap v187: 65 pgs: 65 active+clean; 317 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 3 op/s 2024-03-21T10:18:29.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:29 smithi174 ceph-mon[88004]: pgmap v188: 65 pgs: 65 active+clean; 317 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 2 op/s 2024-03-21T10:18:29.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:29 smithi153 ceph-mon[113733]: pgmap v188: 65 pgs: 65 active+clean; 317 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 2 op/s 2024-03-21T10:18:32.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:32 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:18:32.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:32 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:18:32.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:32 smithi153 ceph-mon[113733]: pgmap v189: 65 pgs: 65 active+clean; 317 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 4 op/s 2024-03-21T10:18:32.568 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:32 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:18:32.568 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:32 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:18:32.568 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:32 smithi174 ceph-mon[88004]: pgmap v189: 65 pgs: 65 active+clean; 317 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 4 op/s 2024-03-21T10:18:33.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:33 smithi174 ceph-mon[88004]: pgmap v190: 65 pgs: 65 active+clean; 317 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:18:33.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:33 smithi153 ceph-mon[113733]: pgmap v190: 65 pgs: 65 active+clean; 317 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:18:34.706 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:34 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-21T10:18:34.706 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:34 smithi174 ceph-mon[88004]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-21T10:18:34.706 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:34 smithi174 ceph-mon[88004]: Upgrade: osd.5 is safe to restart 2024-03-21T10:18:34.808 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:34 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-21T10:18:34.808 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:34 smithi153 ceph-mon[113733]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-21T10:18:34.808 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:34 smithi153 ceph-mon[113733]: Upgrade: osd.5 is safe to restart 2024-03-21T10:18:35.421 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:35 smithi174 ceph-mon[88004]: pgmap v191: 65 pgs: 65 active+clean; 317 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:18:35.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:35 smithi153 ceph-mon[113733]: pgmap v191: 65 pgs: 65 active+clean; 317 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:18:37.527 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:37 smithi174 ceph-mon[88004]: Upgrade: Updating osd.5 2024-03-21T10:18:37.527 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:37 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:18:37.527 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:37 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-21T10:18:37.527 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:37 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:18:37.528 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:37 smithi174 ceph-mon[88004]: Deploying daemon osd.5 on smithi174 2024-03-21T10:18:37.528 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:37 smithi174 ceph-mon[88004]: pgmap v192: 65 pgs: 65 active+clean; 317 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:18:37.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:37 smithi153 ceph-mon[113733]: Upgrade: Updating osd.5 2024-03-21T10:18:37.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:37 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:18:37.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:37 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-21T10:18:37.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:37 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:18:37.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:37 smithi153 ceph-mon[113733]: Deploying daemon osd.5 on smithi174 2024-03-21T10:18:37.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:37 smithi153 ceph-mon[113733]: pgmap v192: 65 pgs: 65 active+clean; 317 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:18:39.437 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:39 smithi174 ceph-mon[88004]: pgmap v193: 65 pgs: 65 active+clean; 317 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:18:39.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:39 smithi153 ceph-mon[113733]: pgmap v193: 65 pgs: 65 active+clean; 317 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:18:41.327 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 21 10:18:41 smithi174 systemd[1]: Stopping Ceph osd.5 for 070ed3e4-e769-11ee-95cd-87774f69a715... 2024-03-21T10:18:41.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:41 smithi174 ceph-mon[88004]: pgmap v194: 65 pgs: 65 active+clean; 317 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:18:41.702 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 21 10:18:41 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-5[56774]: 2024-03-21T10:18:41.478+0000 7fb12b800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T10:18:41.702 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 21 10:18:41 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-5[56774]: 2024-03-21T10:18:41.478+0000 7fb12b800700 -1 osd.5 86 *** Got signal Terminated *** 2024-03-21T10:18:41.702 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 21 10:18:41 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-5[56774]: 2024-03-21T10:18:41.478+0000 7fb12b800700 -1 osd.5 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-21T10:18:41.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:41 smithi153 ceph-mon[113733]: pgmap v194: 65 pgs: 65 active+clean; 317 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:18:42.650 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:42 smithi153 ceph-mon[113733]: osd.5 marked itself down and dead 2024-03-21T10:18:42.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:42 smithi174 ceph-mon[88004]: osd.5 marked itself down and dead 2024-03-21T10:18:43.201 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 21 10:18:42 smithi174 podman[106898]: 2024-03-21 10:18:42.88809311 +0000 UTC m=+1.564923495 container died 3b69886ce22e25f6848737b9a87d487792de5526e106e117add74795ca4568cc (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.29.1, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=-18.2.1) 2024-03-21T10:18:43.529 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:43.527+0000 7f555f200700 1 -- 172.21.15.153:0/1982220839 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5558075be0 msgr2=0x7f55581114c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:18:43.529 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:43.527+0000 7f555f200700 1 --2- 172.21.15.153:0/1982220839 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5558075be0 0x7f55581114c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5544009910 tx=0x7f5544009c20 comp rx=0 tx=0).stop 2024-03-21T10:18:43.529 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:43.527+0000 7f555f200700 1 -- 172.21.15.153:0/1982220839 shutdown_connections 2024-03-21T10:18:43.530 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:43.527+0000 7f555f200700 1 --2- 172.21.15.153:0/1982220839 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5558075be0 0x7f55581114c0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:18:43.530 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:43.527+0000 7f555f200700 1 --2- 172.21.15.153:0/1982220839 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f55580752c0 0x7f55580756a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:18:43.530 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:43.527+0000 7f555f200700 1 -- 172.21.15.153:0/1982220839 >> 172.21.15.153:0/1982220839 conn(0x7f55580fe870 msgr2=0x7f5558100c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:18:43.530 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:43.527+0000 7f555f200700 1 -- 172.21.15.153:0/1982220839 shutdown_connections 2024-03-21T10:18:43.530 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:43.527+0000 7f555f200700 1 -- 172.21.15.153:0/1982220839 wait complete. 2024-03-21T10:18:43.531 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:43.528+0000 7f555f200700 1 Processor -- start 2024-03-21T10:18:43.531 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:43.528+0000 7f555f200700 1 -- start start 2024-03-21T10:18:43.531 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:43.529+0000 7f555f200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f55580752c0 0x7f555819cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:18:43.532 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:43.529+0000 7f555f200700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5558075be0 0x7f555819d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:18:43.532 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:43.529+0000 7f555f200700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f555819d870 con 0x7f55580752c0 2024-03-21T10:18:43.532 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:43.529+0000 7f555f200700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55581a05a0 con 0x7f5558075be0 2024-03-21T10:18:43.532 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:43.529+0000 7f5557e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f55580752c0 0x7f555819cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:18:43.532 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:43.529+0000 7f5557e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f55580752c0 0x7f555819cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:44608/0 (socket says 172.21.15.153:44608) 2024-03-21T10:18:43.532 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:43.529+0000 7f5557e00700 1 -- 172.21.15.153:0/2439175449 learned_addr learned my addr 172.21.15.153:0/2439175449 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:18:43.532 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:43.529+0000 7f5557400700 1 --2- 172.21.15.153:0/2439175449 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5558075be0 0x7f555819d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:18:43.532 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:43.529+0000 7f5557e00700 1 -- 172.21.15.153:0/2439175449 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5558075be0 msgr2=0x7f555819d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:18:43.532 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:43.529+0000 7f5557e00700 1 --2- 172.21.15.153:0/2439175449 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5558075be0 0x7f555819d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:18:43.532 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:43.530+0000 7f5557e00700 1 -- 172.21.15.153:0/2439175449 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55440095f0 con 0x7f55580752c0 2024-03-21T10:18:43.532 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:43.530+0000 7f5557400700 1 --2- 172.21.15.153:0/2439175449 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5558075be0 0x7f555819d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:18:43.532 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:43.530+0000 7f5557e00700 1 --2- 172.21.15.153:0/2439175449 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f55580752c0 0x7f555819cca0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f554800d8d0 tx=0x7f554800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:18:43.533 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:43.530+0000 7f5554c00700 1 -- 172.21.15.153:0/2439175449 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5548009940 con 0x7f55580752c0 2024-03-21T10:18:43.533 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:43.530+0000 7f555f200700 1 -- 172.21.15.153:0/2439175449 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55581a0880 con 0x7f55580752c0 2024-03-21T10:18:43.533 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:43.530+0000 7f5554c00700 1 -- 172.21.15.153:0/2439175449 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5548010460 con 0x7f55580752c0 2024-03-21T10:18:43.533 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:43.530+0000 7f5554c00700 1 -- 172.21.15.153:0/2439175449 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f554800f5f0 con 0x7f55580752c0 2024-03-21T10:18:43.534 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:43.530+0000 7f555f200700 1 -- 172.21.15.153:0/2439175449 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55581a0dd0 con 0x7f55580752c0 2024-03-21T10:18:43.534 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:43.532+0000 7f555f200700 1 -- 172.21.15.153:0/2439175449 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f555810ec30 con 0x7f55580752c0 2024-03-21T10:18:43.534 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:43.533+0000 7f5554c00700 1 -- 172.21.15.153:0/2439175449 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f55480105d0 con 0x7f55580752c0 2024-03-21T10:18:43.535 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:43.533+0000 7f5554c00700 1 --2- 172.21.15.153:0/2439175449 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f553c06e460 0x7f553c070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:18:43.535 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:43.534+0000 7f5554c00700 1 -- 172.21.15.153:0/2439175449 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6654+0+0 (secure 0 0 0) 0x7f554808e460 con 0x7f55580752c0 2024-03-21T10:18:43.535 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:43.534+0000 7f5557400700 1 --2- 172.21.15.153:0/2439175449 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f553c06e460 0x7f553c070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:18:43.536 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:43.534+0000 7f5557400700 1 --2- 172.21.15.153:0/2439175449 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f553c06e460 0x7f553c070920 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f555819e280 tx=0x7f5544005dc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:18:43.540 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:43.539+0000 7f5554c00700 1 -- 172.21.15.153:0/2439175449 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f55480575d0 con 0x7f55580752c0 2024-03-21T10:18:43.684 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:43 smithi174 ceph-mon[88004]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-21T10:18:43.684 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:43 smithi174 ceph-mon[88004]: Health check failed: all OSDs are running squid or later but require_osd_release < squid (OSD_UPGRADE_FINISHED) 2024-03-21T10:18:43.684 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:43 smithi174 ceph-mon[88004]: osdmap e87: 6 total, 5 up, 6 in 2024-03-21T10:18:43.684 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:43 smithi174 ceph-mon[88004]: pgmap v196: 65 pgs: 13 stale+active+clean, 52 active+clean; 317 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-03-21T10:18:43.684 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 21 10:18:43 smithi174 podman[106898]: 2024-03-21 10:18:43.422726088 +0000 UTC m=+2.099556494 container cleanup 3b69886ce22e25f6848737b9a87d487792de5526e106e117add74795ca4568cc (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-5, RELEASE=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, GIT_REPO=https://github.com/ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=-18.2.1, org.label-schema.schema-version=1.0, io.buildah.version=1.29.1, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240222) 2024-03-21T10:18:43.684 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 21 10:18:43 smithi174 bash[106898]: ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-5 2024-03-21T10:18:43.783 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:43.781+0000 7f555f200700 1 -- 172.21.15.153:0/2439175449 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f55581a0ff0 con 0x7f553c06e460 2024-03-21T10:18:43.783 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:43 smithi153 ceph-mon[113733]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-21T10:18:43.783 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:43 smithi153 ceph-mon[113733]: Health check failed: all OSDs are running squid or later but require_osd_release < squid (OSD_UPGRADE_FINISHED) 2024-03-21T10:18:43.784 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:43 smithi153 ceph-mon[113733]: osdmap e87: 6 total, 5 up, 6 in 2024-03-21T10:18:43.784 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:43 smithi153 ceph-mon[113733]: pgmap v196: 65 pgs: 13 stale+active+clean, 52 active+clean; 317 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-03-21T10:18:43.785 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:43.784+0000 7f5554c00700 1 -- 172.21.15.153:0/2439175449 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f55581a0ff0 con 0x7f553c06e460 2024-03-21T10:18:43.789 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:43.787+0000 7f555f200700 1 -- 172.21.15.153:0/2439175449 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f553c06e460 msgr2=0x7f553c070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:18:43.789 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:43.787+0000 7f555f200700 1 --2- 172.21.15.153:0/2439175449 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f553c06e460 0x7f553c070920 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f555819e280 tx=0x7f5544005dc0 comp rx=0 tx=0).stop 2024-03-21T10:18:43.789 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:43.787+0000 7f555f200700 1 -- 172.21.15.153:0/2439175449 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f55580752c0 msgr2=0x7f555819cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:18:43.789 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:43.787+0000 7f555f200700 1 --2- 172.21.15.153:0/2439175449 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f55580752c0 0x7f555819cca0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f554800d8d0 tx=0x7f554800dc90 comp rx=0 tx=0).stop 2024-03-21T10:18:43.789 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:43.787+0000 7f555f200700 1 -- 172.21.15.153:0/2439175449 shutdown_connections 2024-03-21T10:18:43.790 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:43.787+0000 7f555f200700 1 --2- 172.21.15.153:0/2439175449 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f553c06e460 0x7f553c070920 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:18:43.790 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:43.787+0000 7f555f200700 1 --2- 172.21.15.153:0/2439175449 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f55580752c0 0x7f555819cca0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:18:43.790 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:43.787+0000 7f555f200700 1 --2- 172.21.15.153:0/2439175449 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5558075be0 0x7f555819d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:18:43.790 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:43.787+0000 7f555f200700 1 -- 172.21.15.153:0/2439175449 >> 172.21.15.153:0/2439175449 conn(0x7f55580fe870 msgr2=0x7f55580ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:18:43.790 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:43.788+0000 7f555f200700 1 -- 172.21.15.153:0/2439175449 shutdown_connections 2024-03-21T10:18:43.790 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:43.788+0000 7f555f200700 1 -- 172.21.15.153:0/2439175449 wait complete. 2024-03-21T10:18:43.807 INFO:teuthology.orchestra.run.smithi153.stdout:true 2024-03-21T10:18:43.948 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:43.946+0000 7fde83600700 1 -- 172.21.15.153:0/3922270160 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde7c1071d0 msgr2=0x7fde7c107650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:18:43.948 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:43.946+0000 7fde83600700 1 --2- 172.21.15.153:0/3922270160 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde7c1071d0 0x7fde7c107650 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fde68009910 tx=0x7fde68009c20 comp rx=0 tx=0).stop 2024-03-21T10:18:43.949 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:43.946+0000 7fde83600700 1 -- 172.21.15.153:0/3922270160 shutdown_connections 2024-03-21T10:18:43.949 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:43.946+0000 7fde83600700 1 --2- 172.21.15.153:0/3922270160 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde7c1071d0 0x7fde7c107650 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:18:43.949 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:43.946+0000 7fde83600700 1 --2- 172.21.15.153:0/3922270160 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fde7c10d280 0x7fde7c10d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:18:43.949 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:43.946+0000 7fde83600700 1 -- 172.21.15.153:0/3922270160 >> 172.21.15.153:0/3922270160 conn(0x7fde7c0769e0 msgr2=0x7fde7c078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:18:43.949 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:43.946+0000 7fde83600700 1 -- 172.21.15.153:0/3922270160 shutdown_connections 2024-03-21T10:18:43.949 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:43.946+0000 7fde83600700 1 -- 172.21.15.153:0/3922270160 wait complete. 2024-03-21T10:18:43.950 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:43.947+0000 7fde83600700 1 Processor -- start 2024-03-21T10:18:43.950 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:43.948+0000 7fde83600700 1 -- start start 2024-03-21T10:18:43.950 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:43.948+0000 7fde83600700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde7c1071d0 0x7fde7c19ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:18:43.950 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:43.948+0000 7fde83600700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fde7c10d280 0x7fde7c19d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:18:43.950 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:43.948+0000 7fde83600700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde7c19d880 con 0x7fde7c1071d0 2024-03-21T10:18:43.950 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:43.948+0000 7fde83600700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde7c1a05b0 con 0x7fde7c10d280 2024-03-21T10:18:43.950 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:43.948+0000 7fde80c00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde7c1071d0 0x7fde7c19ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:18:43.950 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:43.948+0000 7fde80c00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde7c1071d0 0x7fde7c19ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:44640/0 (socket says 172.21.15.153:44640) 2024-03-21T10:18:43.950 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:43.948+0000 7fde80c00700 1 -- 172.21.15.153:0/4161655704 learned_addr learned my addr 172.21.15.153:0/4161655704 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:18:43.951 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:43.948+0000 7fde7be00700 1 --2- 172.21.15.153:0/4161655704 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fde7c10d280 0x7fde7c19d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:18:43.951 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 21 10:18:43 smithi174 podman[106918]: 2024-03-21 10:18:43.68215352 +0000 UTC m=+0.779828421 container remove 3b69886ce22e25f6848737b9a87d487792de5526e106e117add74795ca4568cc (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-5, RELEASE=HEAD, org.label-schema.build-date=20240222, io.buildah.version=1.29.1, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=-18.2.1, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=https://github.com/ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-03-21T10:18:43.952 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:43.949+0000 7fde80c00700 1 -- 172.21.15.153:0/4161655704 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fde7c10d280 msgr2=0x7fde7c19d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:18:43.952 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:43.949+0000 7fde80c00700 1 --2- 172.21.15.153:0/4161655704 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fde7c10d280 0x7fde7c19d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:18:43.952 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:43.949+0000 7fde80c00700 1 -- 172.21.15.153:0/4161655704 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde680095f0 con 0x7fde7c1071d0 2024-03-21T10:18:43.952 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:43.949+0000 7fde80c00700 1 --2- 172.21.15.153:0/4161655704 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde7c1071d0 0x7fde7c19ccb0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fde7000d8d0 tx=0x7fde7000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:18:43.952 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:43.949+0000 7fde79600700 1 -- 172.21.15.153:0/4161655704 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fde700098a0 con 0x7fde7c1071d0 2024-03-21T10:18:43.952 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:43.949+0000 7fde79600700 1 -- 172.21.15.153:0/4161655704 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fde70010460 con 0x7fde7c1071d0 2024-03-21T10:18:43.952 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:43.949+0000 7fde83600700 1 -- 172.21.15.153:0/4161655704 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde7c1a0890 con 0x7fde7c1071d0 2024-03-21T10:18:43.952 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:43.949+0000 7fde79600700 1 -- 172.21.15.153:0/4161655704 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fde7000f5f0 con 0x7fde7c1071d0 2024-03-21T10:18:43.952 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:43.950+0000 7fde83600700 1 -- 172.21.15.153:0/4161655704 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde7c1a0e60 con 0x7fde7c1071d0 2024-03-21T10:18:43.953 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:43.951+0000 7fde83600700 1 -- 172.21.15.153:0/4161655704 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde7c10f6c0 con 0x7fde7c1071d0 2024-03-21T10:18:43.953 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:43.952+0000 7fde79600700 1 -- 172.21.15.153:0/4161655704 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fde70009a00 con 0x7fde7c1071d0 2024-03-21T10:18:43.958 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:43.953+0000 7fde79600700 1 --2- 172.21.15.153:0/4161655704 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fde6406e460 0x7fde64070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:18:43.958 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:43.953+0000 7fde79600700 1 -- 172.21.15.153:0/4161655704 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6654+0+0 (secure 0 0 0) 0x7fde7008e240 con 0x7fde7c1071d0 2024-03-21T10:18:43.958 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:43.953+0000 7fde7be00700 1 --2- 172.21.15.153:0/4161655704 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fde6406e460 0x7fde64070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:18:43.958 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:43.954+0000 7fde7be00700 1 --2- 172.21.15.153:0/4161655704 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fde6406e460 0x7fde64070920 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fde7c19e290 tx=0x7fde6801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:18:43.959 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:43.958+0000 7fde79600700 1 -- 172.21.15.153:0/4161655704 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fde70057430 con 0x7fde7c1071d0 2024-03-21T10:18:44.208 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:44.206+0000 7fde83600700 1 -- 172.21.15.153:0/4161655704 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fde7c1a10d0 con 0x7fde6406e460 2024-03-21T10:18:44.210 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:44.208+0000 7fde79600700 1 -- 172.21.15.153:0/4161655704 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7fde7c1a10d0 con 0x7fde6406e460 2024-03-21T10:18:44.213 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:44.210+0000 7fde83600700 1 -- 172.21.15.153:0/4161655704 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fde6406e460 msgr2=0x7fde64070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:18:44.213 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:44.211+0000 7fde83600700 1 --2- 172.21.15.153:0/4161655704 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fde6406e460 0x7fde64070920 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fde7c19e290 tx=0x7fde6801a040 comp rx=0 tx=0).stop 2024-03-21T10:18:44.213 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:44.211+0000 7fde83600700 1 -- 172.21.15.153:0/4161655704 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde7c1071d0 msgr2=0x7fde7c19ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:18:44.213 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:44.211+0000 7fde83600700 1 --2- 172.21.15.153:0/4161655704 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde7c1071d0 0x7fde7c19ccb0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fde7000d8d0 tx=0x7fde7000dbe0 comp rx=0 tx=0).stop 2024-03-21T10:18:44.213 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:44.211+0000 7fde83600700 1 -- 172.21.15.153:0/4161655704 shutdown_connections 2024-03-21T10:18:44.213 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:44.211+0000 7fde83600700 1 --2- 172.21.15.153:0/4161655704 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fde6406e460 0x7fde64070920 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:18:44.213 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:44.211+0000 7fde83600700 1 --2- 172.21.15.153:0/4161655704 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde7c1071d0 0x7fde7c19ccb0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:18:44.213 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:44.211+0000 7fde83600700 1 --2- 172.21.15.153:0/4161655704 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fde7c10d280 0x7fde7c19d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:18:44.213 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:44.211+0000 7fde83600700 1 -- 172.21.15.153:0/4161655704 >> 172.21.15.153:0/4161655704 conn(0x7fde7c0769e0 msgr2=0x7fde7c078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:18:44.213 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:44.211+0000 7fde83600700 1 -- 172.21.15.153:0/4161655704 shutdown_connections 2024-03-21T10:18:44.213 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:44.211+0000 7fde83600700 1 -- 172.21.15.153:0/4161655704 wait complete. 2024-03-21T10:18:44.358 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:44.356+0000 7f3bc9a00700 1 -- 172.21.15.153:0/2655747690 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3bc4101d40 msgr2=0x7f3bc4102120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:18:44.358 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:44.356+0000 7f3bc9a00700 1 --2- 172.21.15.153:0/2655747690 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3bc4101d40 0x7f3bc4102120 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3bb4009910 tx=0x7f3bb4009c20 comp rx=0 tx=0).stop 2024-03-21T10:18:44.358 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:44.356+0000 7f3bc9a00700 1 -- 172.21.15.153:0/2655747690 shutdown_connections 2024-03-21T10:18:44.358 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:44.356+0000 7f3bc9a00700 1 --2- 172.21.15.153:0/2655747690 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3bc4102660 0x7f3bc410ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:18:44.358 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:44.356+0000 7f3bc9a00700 1 --2- 172.21.15.153:0/2655747690 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3bc4101d40 0x7f3bc4102120 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:18:44.359 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:44.356+0000 7f3bc9a00700 1 -- 172.21.15.153:0/2655747690 >> 172.21.15.153:0/2655747690 conn(0x7f3bc40fb570 msgr2=0x7f3bc40fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:18:44.359 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:44.356+0000 7f3bc9a00700 1 -- 172.21.15.153:0/2655747690 shutdown_connections 2024-03-21T10:18:44.359 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:44.356+0000 7f3bc9a00700 1 -- 172.21.15.153:0/2655747690 wait complete. 2024-03-21T10:18:44.359 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:44.358+0000 7f3bc9a00700 1 Processor -- start 2024-03-21T10:18:44.359 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:44.358+0000 7f3bc9a00700 1 -- start start 2024-03-21T10:18:44.360 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:44.358+0000 7f3bc9a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3bc4101d40 0x7f3bc419e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:18:44.360 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:44.358+0000 7f3bc9a00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3bc4102660 0x7f3bc419ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:18:44.360 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:44.358+0000 7f3bc9a00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc419f3a0 con 0x7f3bc4101d40 2024-03-21T10:18:44.360 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:44.358+0000 7f3bc9a00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc4198850 con 0x7f3bc4102660 2024-03-21T10:18:44.361 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:44.359+0000 7f3bc2a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3bc4101d40 0x7f3bc419e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:18:44.361 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:44.359+0000 7f3bc2a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3bc4101d40 0x7f3bc419e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:44670/0 (socket says 172.21.15.153:44670) 2024-03-21T10:18:44.361 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:44.359+0000 7f3bc2a00700 1 -- 172.21.15.153:0/3515614118 learned_addr learned my addr 172.21.15.153:0/3515614118 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:18:44.362 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:44.359+0000 7f3bc2000700 1 --2- 172.21.15.153:0/3515614118 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3bc4102660 0x7f3bc419ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:18:44.362 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:44.359+0000 7f3bc2a00700 1 -- 172.21.15.153:0/3515614118 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3bc4102660 msgr2=0x7f3bc419ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:18:44.362 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:44.359+0000 7f3bc2a00700 1 --2- 172.21.15.153:0/3515614118 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3bc4102660 0x7f3bc419ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:18:44.362 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:44.359+0000 7f3bc2a00700 1 -- 172.21.15.153:0/3515614118 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bb40095f0 con 0x7f3bc4101d40 2024-03-21T10:18:44.362 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:44.360+0000 7f3bc2a00700 1 --2- 172.21.15.153:0/3515614118 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3bc4101d40 0x7f3bc419e7d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3bb40092e0 tx=0x7f3bb400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:18:44.362 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:44.360+0000 7f3bbb400700 1 -- 172.21.15.153:0/3515614118 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3bb401d070 con 0x7f3bc4101d40 2024-03-21T10:18:44.362 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:44.360+0000 7f3bbb400700 1 -- 172.21.15.153:0/3515614118 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3bb4005e00 con 0x7f3bc4101d40 2024-03-21T10:18:44.362 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:44.360+0000 7f3bbb400700 1 -- 172.21.15.153:0/3515614118 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3bb4017400 con 0x7f3bc4101d40 2024-03-21T10:18:44.362 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:44.360+0000 7f3bc9a00700 1 -- 172.21.15.153:0/3515614118 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bc4198ad0 con 0x7f3bc4101d40 2024-03-21T10:18:44.363 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:44.360+0000 7f3bc9a00700 1 -- 172.21.15.153:0/3515614118 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bc4198fc0 con 0x7f3bc4101d40 2024-03-21T10:18:44.363 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:44.361+0000 7f3bc9a00700 1 -- 172.21.15.153:0/3515614118 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bc410a6b0 con 0x7f3bc4101d40 2024-03-21T10:18:44.364 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:44.362+0000 7f3bbb400700 1 -- 172.21.15.153:0/3515614118 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f3bb400f6f0 con 0x7f3bc4101d40 2024-03-21T10:18:44.365 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:44.363+0000 7f3bbb400700 1 --2- 172.21.15.153:0/3515614118 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f3ba806e460 0x7f3ba8070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:18:44.365 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:44.363+0000 7f3bbb400700 1 -- 172.21.15.153:0/3515614118 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6654+0+0 (secure 0 0 0) 0x7f3bb408f890 con 0x7f3bc4101d40 2024-03-21T10:18:44.365 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:44.363+0000 7f3bc2000700 1 --2- 172.21.15.153:0/3515614118 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f3ba806e460 0x7f3ba8070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:18:44.366 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:44.364+0000 7f3bc2000700 1 --2- 172.21.15.153:0/3515614118 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f3ba806e460 0x7f3ba8070920 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3bc40fcce0 tx=0x7f3bb000b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:18:44.370 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:44.368+0000 7f3bbb400700 1 -- 172.21.15.153:0/3515614118 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f3bb405c510 con 0x7f3bc4101d40 2024-03-21T10:18:44.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:44 smithi153 ceph-mon[113733]: osdmap e88: 6 total, 5 up, 6 in 2024-03-21T10:18:44.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:44 smithi153 ceph-mon[113733]: from='client.34398 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:18:44.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:44 smithi153 ceph-mon[113733]: from='client.34402 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:18:44.608 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:44.606+0000 7f3bc9a00700 1 -- 172.21.15.153:0/3515614118 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f3bc4199230 con 0x7f3ba806e460 2024-03-21T10:18:44.640 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:44.638+0000 7f3bbb400700 1 -- 172.21.15.153:0/3515614118 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f3bc4199230 con 0x7f3ba806e460 2024-03-21T10:18:44.640 INFO:teuthology.orchestra.run.smithi153.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-21T10:18:44.641 INFO:teuthology.orchestra.run.smithi153.stdout:alertmanager.smithi153 smithi153 *:9093,9094 running (7m) 101s ago 21m 14.5M - 0.25.0 c8568f914cd2 9ed3f2200b5f 2024-03-21T10:18:44.641 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi153 smithi153 running (21m) 101s ago 21m 8988k - 18.2.1 5be31c24972a 5b61994396a6 2024-03-21T10:18:44.641 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi174 smithi174 running (20m) 32s ago 20m 8724k - 18.2.1 5be31c24972a 4f7a5fc3ac28 2024-03-21T10:18:44.641 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi153 smithi153 running (3m) 101s ago 21m 6656k - 19.0.0-2231-g2edfee68 237e43c7a65f 32c21104d5b0 2024-03-21T10:18:44.641 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi174 smithi174 running (3m) 32s ago 19m 6660k - 19.0.0-2231-g2edfee68 237e43c7a65f a04e48967ab6 2024-03-21T10:18:44.641 INFO:teuthology.orchestra.run.smithi153.stdout:grafana.smithi153 smithi153 *:3000 running (7m) 101s ago 20m 75.4M - 9.4.12 2bacad6d85d8 ee24026ab786 2024-03-21T10:18:44.641 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.iuyluj smithi153 running (14m) 101s ago 14m 128M - 18.2.1 5be31c24972a 02f667cee979 2024-03-21T10:18:44.641 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.zipckr smithi153 running (14m) 101s ago 14m 58.8M - 18.2.1 5be31c24972a 68a853929f8f 2024-03-21T10:18:44.641 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.nhozag smithi174 running (14m) 32s ago 14m 57.5M - 18.2.1 5be31c24972a 2fee4000308f 2024-03-21T10:18:44.641 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.yyimst smithi174 running (14m) 32s ago 14m 117M - 18.2.1 5be31c24972a d78a729220d1 2024-03-21T10:18:44.641 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi153.tluxni smithi153 *:8443,9283,8765 running (10m) 101s ago 23m 505M - 19.0.0-2231-g2edfee68 237e43c7a65f 7ffee0e55382 2024-03-21T10:18:44.641 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi174.qpcwat smithi174 *:8443,9283,8765 running (9m) 32s ago 19m 451M - 19.0.0-2231-g2edfee68 237e43c7a65f 748968f216ab 2024-03-21T10:18:44.641 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi153 smithi153 running (5m) 101s ago 23m 48.2M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f dd8038303dc3 2024-03-21T10:18:44.642 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi174 smithi174 running (4m) 32s ago 19m 39.5M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f 29d78a788f2c 2024-03-21T10:18:44.642 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi153 smithi153 *:9100 running (8m) 101s ago 21m 8284k - 1.5.0 0da6a335fe13 bb835edc0e0b 2024-03-21T10:18:44.642 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi174 smithi174 *:9100 running (8m) 32s ago 19m 7914k - 1.5.0 0da6a335fe13 7c4ecad790fa 2024-03-21T10:18:44.642 INFO:teuthology.orchestra.run.smithi153.stdout:osd.0 smithi153 running (3m) 101s ago 18m 119M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 507900fd4791 2024-03-21T10:18:44.642 INFO:teuthology.orchestra.run.smithi153.stdout:osd.1 smithi153 running (2m) 101s ago 18m 86.6M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 0d646452e9c1 2024-03-21T10:18:44.642 INFO:teuthology.orchestra.run.smithi153.stdout:osd.2 smithi153 running (110s) 101s ago 17m 12.6M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 1326f2afdfcd 2024-03-21T10:18:44.642 INFO:teuthology.orchestra.run.smithi153.stdout:osd.3 smithi174 running (79s) 32s ago 17m 82.1M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 70aafac71ad5 2024-03-21T10:18:44.642 INFO:teuthology.orchestra.run.smithi153.stdout:osd.4 smithi174 running (37s) 32s ago 16m 11.1M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 0d746ce6e79d 2024-03-21T10:18:44.642 INFO:teuthology.orchestra.run.smithi153.stdout:osd.5 smithi174 running (15m) 32s ago 15m 213M 2241M 18.2.1 5be31c24972a 3b69886ce22e 2024-03-21T10:18:44.642 INFO:teuthology.orchestra.run.smithi153.stdout:prometheus.smithi153 smithi153 *:9095 running (7m) 101s ago 20m 72.2M - 2.43.0 a07b618ecd1d 7d95561e50c2 2024-03-21T10:18:44.643 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:44.641+0000 7f3bc9a00700 1 -- 172.21.15.153:0/3515614118 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f3ba806e460 msgr2=0x7f3ba8070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:18:44.643 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:44.641+0000 7f3bc9a00700 1 --2- 172.21.15.153:0/3515614118 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f3ba806e460 0x7f3ba8070920 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3bc40fcce0 tx=0x7f3bb000b410 comp rx=0 tx=0).stop 2024-03-21T10:18:44.644 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:44.641+0000 7f3bc9a00700 1 -- 172.21.15.153:0/3515614118 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3bc4101d40 msgr2=0x7f3bc419e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:18:44.644 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:44.641+0000 7f3bc9a00700 1 --2- 172.21.15.153:0/3515614118 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3bc4101d40 0x7f3bc419e7d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3bb40092e0 tx=0x7f3bb400bd70 comp rx=0 tx=0).stop 2024-03-21T10:18:44.644 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:44.641+0000 7f3bc9a00700 1 -- 172.21.15.153:0/3515614118 shutdown_connections 2024-03-21T10:18:44.644 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:44.641+0000 7f3bc9a00700 1 --2- 172.21.15.153:0/3515614118 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f3ba806e460 0x7f3ba8070920 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:18:44.644 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:44.641+0000 7f3bc9a00700 1 --2- 172.21.15.153:0/3515614118 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3bc4101d40 0x7f3bc419e7d0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:18:44.644 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:44.641+0000 7f3bc9a00700 1 --2- 172.21.15.153:0/3515614118 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3bc4102660 0x7f3bc419ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:18:44.644 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:44.641+0000 7f3bc9a00700 1 -- 172.21.15.153:0/3515614118 >> 172.21.15.153:0/3515614118 conn(0x7f3bc40fb570 msgr2=0x7f3bc4100120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:18:44.644 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:44.642+0000 7f3bc9a00700 1 -- 172.21.15.153:0/3515614118 shutdown_connections 2024-03-21T10:18:44.644 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:44.642+0000 7f3bc9a00700 1 -- 172.21.15.153:0/3515614118 wait complete. 2024-03-21T10:18:44.663 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:44 smithi174 ceph-mon[88004]: osdmap e88: 6 total, 5 up, 6 in 2024-03-21T10:18:44.663 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:44 smithi174 ceph-mon[88004]: from='client.34398 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:18:44.663 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:44 smithi174 ceph-mon[88004]: from='client.34402 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:18:44.663 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 21 10:18:44 smithi174 podman[107010]: 2024-03-21 10:18:44.361108391 +0000 UTC m=+0.032507518 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T10:18:44.664 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 21 10:18:44 smithi174 podman[107010]: 2024-03-21 10:18:44.500444293 +0000 UTC m=+0.171843446 container create 2c8bcc0183df5f6475174b935c6affdf257ca7239974ddeb51b64349bbaae658 (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-5-deactivate, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True) 2024-03-21T10:18:44.795 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:44.793+0000 7fec56600700 1 -- 172.21.15.153:0/3704212588 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec501039b0 msgr2=0x7fec50107a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:18:44.796 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:44.793+0000 7fec56600700 1 --2- 172.21.15.153:0/3704212588 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec501039b0 0x7fec50107a00 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fec3c009910 tx=0x7fec3c009c20 comp rx=0 tx=0).stop 2024-03-21T10:18:44.796 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:44.793+0000 7fec56600700 1 -- 172.21.15.153:0/3704212588 shutdown_connections 2024-03-21T10:18:44.796 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:44.793+0000 7fec56600700 1 --2- 172.21.15.153:0/3704212588 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec501039b0 0x7fec50107a00 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:18:44.796 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:44.793+0000 7fec56600700 1 --2- 172.21.15.153:0/3704212588 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fec50103000 0x7fec501033e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:18:44.796 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:44.793+0000 7fec56600700 1 -- 172.21.15.153:0/3704212588 >> 172.21.15.153:0/3704212588 conn(0x7fec500fe8d0 msgr2=0x7fec50100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:18:44.796 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:44.793+0000 7fec56600700 1 -- 172.21.15.153:0/3704212588 shutdown_connections 2024-03-21T10:18:44.796 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:44.793+0000 7fec56600700 1 -- 172.21.15.153:0/3704212588 wait complete. 2024-03-21T10:18:44.796 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:44.794+0000 7fec56600700 1 Processor -- start 2024-03-21T10:18:44.796 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:44.794+0000 7fec56600700 1 -- start start 2024-03-21T10:18:44.798 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:44.794+0000 7fec56600700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fec50103000 0x7fec5019e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:18:44.799 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:44.794+0000 7fec56600700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec501039b0 0x7fec5019ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:18:44.799 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:44.794+0000 7fec56600700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec5019f490 con 0x7fec501039b0 2024-03-21T10:18:44.799 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:44.794+0000 7fec56600700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec50198940 con 0x7fec50103000 2024-03-21T10:18:44.799 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:44.795+0000 7fec4ea00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec501039b0 0x7fec5019ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:18:44.799 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:44.795+0000 7fec4ea00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec501039b0 0x7fec5019ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:44678/0 (socket says 172.21.15.153:44678) 2024-03-21T10:18:44.799 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:44.795+0000 7fec4ea00700 1 -- 172.21.15.153:0/3651920590 learned_addr learned my addr 172.21.15.153:0/3651920590 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:18:44.799 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:44.795+0000 7fec4f400700 1 --2- 172.21.15.153:0/3651920590 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fec50103000 0x7fec5019e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:18:44.799 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:44.795+0000 7fec4ea00700 1 -- 172.21.15.153:0/3651920590 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fec50103000 msgr2=0x7fec5019e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:18:44.799 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:44.795+0000 7fec4ea00700 1 --2- 172.21.15.153:0/3651920590 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fec50103000 0x7fec5019e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:18:44.799 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:44.795+0000 7fec4ea00700 1 -- 172.21.15.153:0/3651920590 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec3c0095f0 con 0x7fec501039b0 2024-03-21T10:18:44.799 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:44.795+0000 7fec4ea00700 1 --2- 172.21.15.153:0/3651920590 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec501039b0 0x7fec5019ee00 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fec3c00bda0 tx=0x7fec3c00be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:18:44.800 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:44.796+0000 7fec47e00700 1 -- 172.21.15.153:0/3651920590 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fec3c01d070 con 0x7fec501039b0 2024-03-21T10:18:44.800 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:44.796+0000 7fec47e00700 1 -- 172.21.15.153:0/3651920590 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fec3c004fb0 con 0x7fec501039b0 2024-03-21T10:18:44.800 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:44.796+0000 7fec47e00700 1 -- 172.21.15.153:0/3651920590 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fec3c00f460 con 0x7fec501039b0 2024-03-21T10:18:44.800 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:44.796+0000 7fec56600700 1 -- 172.21.15.153:0/3651920590 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec50198bc0 con 0x7fec501039b0 2024-03-21T10:18:44.800 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:44.796+0000 7fec56600700 1 -- 172.21.15.153:0/3651920590 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec501990b0 con 0x7fec501039b0 2024-03-21T10:18:44.800 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:44.797+0000 7fec56600700 1 -- 172.21.15.153:0/3651920590 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec5010b350 con 0x7fec501039b0 2024-03-21T10:18:44.801 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:44.798+0000 7fec47e00700 1 -- 172.21.15.153:0/3651920590 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fec3c022800 con 0x7fec501039b0 2024-03-21T10:18:44.801 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:44.799+0000 7fec47e00700 1 --2- 172.21.15.153:0/3651920590 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fec3406e460 0x7fec34070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:18:44.801 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:44.799+0000 7fec47e00700 1 -- 172.21.15.153:0/3651920590 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6654+0+0 (secure 0 0 0) 0x7fec3c08f970 con 0x7fec501039b0 2024-03-21T10:18:44.801 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:44.799+0000 7fec4f400700 1 --2- 172.21.15.153:0/3651920590 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fec3406e460 0x7fec34070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:18:44.802 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:44.800+0000 7fec4f400700 1 --2- 172.21.15.153:0/3651920590 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fec3406e460 0x7fec34070920 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fec40006fd0 tx=0x7fec40008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:18:44.805 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:44.804+0000 7fec47e00700 1 -- 172.21.15.153:0/3651920590 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fec3c05c7c0 con 0x7fec501039b0 2024-03-21T10:18:44.952 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 21 10:18:44 smithi174 podman[107010]: 2024-03-21 10:18:44.770043845 +0000 UTC m=+0.441443041 container init 2c8bcc0183df5f6475174b935c6affdf257ca7239974ddeb51b64349bbaae658 (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6) 2024-03-21T10:18:44.952 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 21 10:18:44 smithi174 podman[107010]: 2024-03-21 10:18:44.780807707 +0000 UTC m=+0.452206867 container start 2c8bcc0183df5f6475174b935c6affdf257ca7239974ddeb51b64349bbaae658 (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-5-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-03-21T10:18:44.952 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 21 10:18:44 smithi174 podman[107010]: 2024-03-21 10:18:44.877273331 +0000 UTC m=+0.548672551 container attach 2c8bcc0183df5f6475174b935c6affdf257ca7239974ddeb51b64349bbaae658 (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-5-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-03-21T10:18:45.106 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:45.104+0000 7fec56600700 1 -- 172.21.15.153:0/3651920590 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fec5004ea90 con 0x7fec501039b0 2024-03-21T10:18:45.107 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:45.105+0000 7fec47e00700 1 -- 172.21.15.153:0/3651920590 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+700 (secure 0 0 0) 0x7fec3c05c350 con 0x7fec501039b0 2024-03-21T10:18:45.107 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:18:45.107 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": { 2024-03-21T10:18:45.107 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:18:45.107 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:18:45.108 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": { 2024-03-21T10:18:45.108 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:18:45.108 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:18:45.108 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": { 2024-03-21T10:18:45.108 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 5 2024-03-21T10:18:45.108 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:18:45.108 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": { 2024-03-21T10:18:45.108 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-03-21T10:18:45.108 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:18:45.108 INFO:teuthology.orchestra.run.smithi153.stdout: "overall": { 2024-03-21T10:18:45.108 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4, 2024-03-21T10:18:45.108 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 9 2024-03-21T10:18:45.108 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-03-21T10:18:45.108 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:18:45.110 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:45.108+0000 7fec56600700 1 -- 172.21.15.153:0/3651920590 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fec3406e460 msgr2=0x7fec34070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:18:45.110 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:45.108+0000 7fec56600700 1 --2- 172.21.15.153:0/3651920590 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fec3406e460 0x7fec34070920 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fec40006fd0 tx=0x7fec40008040 comp rx=0 tx=0).stop 2024-03-21T10:18:45.110 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:45.108+0000 7fec56600700 1 -- 172.21.15.153:0/3651920590 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec501039b0 msgr2=0x7fec5019ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:18:45.110 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:45.108+0000 7fec56600700 1 --2- 172.21.15.153:0/3651920590 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec501039b0 0x7fec5019ee00 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fec3c00bda0 tx=0x7fec3c00be80 comp rx=0 tx=0).stop 2024-03-21T10:18:45.111 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:45.108+0000 7fec56600700 1 -- 172.21.15.153:0/3651920590 shutdown_connections 2024-03-21T10:18:45.111 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:45.108+0000 7fec56600700 1 --2- 172.21.15.153:0/3651920590 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fec50103000 0x7fec5019e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:18:45.111 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:45.108+0000 7fec56600700 1 --2- 172.21.15.153:0/3651920590 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fec3406e460 0x7fec34070920 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:18:45.111 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:45.108+0000 7fec56600700 1 --2- 172.21.15.153:0/3651920590 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec501039b0 0x7fec5019ee00 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:18:45.111 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:45.108+0000 7fec56600700 1 -- 172.21.15.153:0/3651920590 >> 172.21.15.153:0/3651920590 conn(0x7fec500fe8d0 msgr2=0x7fec500fffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:18:45.111 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:45.109+0000 7fec56600700 1 -- 172.21.15.153:0/3651920590 shutdown_connections 2024-03-21T10:18:45.111 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:45.109+0000 7fec56600700 1 -- 172.21.15.153:0/3651920590 wait complete. 2024-03-21T10:18:45.268 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:45.265+0000 7f1642c00700 1 -- 172.21.15.153:0/1294078771 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f163c101da0 msgr2=0x7f163c102180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:18:45.268 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:45.265+0000 7f1642c00700 1 --2- 172.21.15.153:0/1294078771 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f163c101da0 0x7f163c102180 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f162c009910 tx=0x7f162c009c20 comp rx=0 tx=0).stop 2024-03-21T10:18:45.268 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:45.266+0000 7f1642c00700 1 -- 172.21.15.153:0/1294078771 shutdown_connections 2024-03-21T10:18:45.268 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:45.266+0000 7f1642c00700 1 --2- 172.21.15.153:0/1294078771 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f163c1026c0 0x7f163c10abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:18:45.268 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:45.266+0000 7f1642c00700 1 --2- 172.21.15.153:0/1294078771 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f163c101da0 0x7f163c102180 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:18:45.268 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:45.266+0000 7f1642c00700 1 -- 172.21.15.153:0/1294078771 >> 172.21.15.153:0/1294078771 conn(0x7f163c0fb570 msgr2=0x7f163c0fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:18:45.268 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:45.266+0000 7f1642c00700 1 -- 172.21.15.153:0/1294078771 shutdown_connections 2024-03-21T10:18:45.268 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:45.266+0000 7f1642c00700 1 -- 172.21.15.153:0/1294078771 wait complete. 2024-03-21T10:18:45.270 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:45.267+0000 7f1642c00700 1 Processor -- start 2024-03-21T10:18:45.270 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:45.267+0000 7f1642c00700 1 -- start start 2024-03-21T10:18:45.270 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:45.267+0000 7f1642c00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f163c101da0 0x7f163c0ffa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:18:45.270 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:45.267+0000 7f1642c00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f163c1026c0 0x7f163c0fff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:18:45.270 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:45.267+0000 7f1642c00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f163c100480 con 0x7f163c101da0 2024-03-21T10:18:45.270 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:45.267+0000 7f1642c00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f163c1005c0 con 0x7f163c1026c0 2024-03-21T10:18:45.270 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:45.267+0000 7f163be00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f163c101da0 0x7f163c0ffa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:18:45.270 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:45.267+0000 7f163be00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f163c101da0 0x7f163c0ffa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:44696/0 (socket says 172.21.15.153:44696) 2024-03-21T10:18:45.270 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:45.267+0000 7f163be00700 1 -- 172.21.15.153:0/4094735331 learned_addr learned my addr 172.21.15.153:0/4094735331 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:18:45.270 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:45.268+0000 7f163b400700 1 --2- 172.21.15.153:0/4094735331 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f163c1026c0 0x7f163c0fff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:18:45.270 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:45.268+0000 7f163be00700 1 -- 172.21.15.153:0/4094735331 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f163c1026c0 msgr2=0x7f163c0fff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:18:45.270 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:45.268+0000 7f163be00700 1 --2- 172.21.15.153:0/4094735331 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f163c1026c0 0x7f163c0fff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:18:45.271 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:45.268+0000 7f163be00700 1 -- 172.21.15.153:0/4094735331 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f162c0095f0 con 0x7f163c101da0 2024-03-21T10:18:45.271 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:45.268+0000 7f163be00700 1 --2- 172.21.15.153:0/4094735331 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f163c101da0 0x7f163c0ffa00 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f162c0092e0 tx=0x7f162c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:18:45.271 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:45.268+0000 7f1638c00700 1 -- 172.21.15.153:0/4094735331 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f162c01d070 con 0x7f163c101da0 2024-03-21T10:18:45.271 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:45.269+0000 7f1642c00700 1 -- 172.21.15.153:0/4094735331 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f163c100840 con 0x7f163c101da0 2024-03-21T10:18:45.272 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:45.268+0000 7f1638c00700 1 -- 172.21.15.153:0/4094735331 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f162c00f460 con 0x7f163c101da0 2024-03-21T10:18:45.272 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:45.269+0000 7f1638c00700 1 -- 172.21.15.153:0/4094735331 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f162c005e20 con 0x7f163c101da0 2024-03-21T10:18:45.272 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:45.269+0000 7f1642c00700 1 -- 172.21.15.153:0/4094735331 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f163c1a2c40 con 0x7f163c101da0 2024-03-21T10:18:45.272 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:45.270+0000 7f1642c00700 1 -- 172.21.15.153:0/4094735331 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f163c1082b0 con 0x7f163c101da0 2024-03-21T10:18:45.272 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:45.271+0000 7f1638c00700 1 -- 172.21.15.153:0/4094735331 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f162c00f5d0 con 0x7f163c101da0 2024-03-21T10:18:45.273 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:45.272+0000 7f1638c00700 1 --2- 172.21.15.153:0/4094735331 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f162006e460 0x7f1620070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:18:45.274 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:45.272+0000 7f1638c00700 1 -- 172.21.15.153:0/4094735331 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6654+0+0 (secure 0 0 0) 0x7f162c08f890 con 0x7f163c101da0 2024-03-21T10:18:45.274 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:45.272+0000 7f163b400700 1 --2- 172.21.15.153:0/4094735331 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f162006e460 0x7f1620070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:18:45.274 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:45.273+0000 7f163b400700 1 --2- 172.21.15.153:0/4094735331 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f162006e460 0x7f1620070920 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f163c1011f0 tx=0x7f162800b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:18:45.278 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:45.276+0000 7f1638c00700 1 -- 172.21.15.153:0/4094735331 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f162c058b70 con 0x7f163c101da0 2024-03-21T10:18:45.546 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:45.545+0000 7f1642c00700 1 -- 172.21.15.153:0/4094735331 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f163c04ea90 con 0x7f163c101da0 2024-03-21T10:18:45.547 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:45.546+0000 7f1638c00700 1 -- 172.21.15.153:0/4094735331 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 21 v21) v1 ==== 76+0+1903 (secure 0 0 0) 0x7f162c05c230 con 0x7f163c101da0 2024-03-21T10:18:45.548 INFO:teuthology.orchestra.run.smithi153.stdout:e21 2024-03-21T10:18:45.548 INFO:teuthology.orchestra.run.smithi153.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-21T10:18:45.548 INFO:teuthology.orchestra.run.smithi153.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:18:45.548 INFO:teuthology.orchestra.run.smithi153.stdout:legacy client fscid: 1 2024-03-21T10:18:45.548 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:18:45.549 INFO:teuthology.orchestra.run.smithi153.stdout:Filesystem 'cephfs' (1) 2024-03-21T10:18:45.549 INFO:teuthology.orchestra.run.smithi153.stdout:fs_name cephfs 2024-03-21T10:18:45.549 INFO:teuthology.orchestra.run.smithi153.stdout:epoch 21 2024-03-21T10:18:45.549 INFO:teuthology.orchestra.run.smithi153.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-21T10:18:45.549 INFO:teuthology.orchestra.run.smithi153.stdout:created 2024-03-21T10:04:27.086576+0000 2024-03-21T10:18:45.549 INFO:teuthology.orchestra.run.smithi153.stdout:modified 2024-03-21T10:13:01.455594+0000 2024-03-21T10:18:45.549 INFO:teuthology.orchestra.run.smithi153.stdout:tableserver 0 2024-03-21T10:18:45.549 INFO:teuthology.orchestra.run.smithi153.stdout:root 0 2024-03-21T10:18:45.549 INFO:teuthology.orchestra.run.smithi153.stdout:session_timeout 60 2024-03-21T10:18:45.549 INFO:teuthology.orchestra.run.smithi153.stdout:session_autoclose 300 2024-03-21T10:18:45.549 INFO:teuthology.orchestra.run.smithi153.stdout:max_file_size 1099511627776 2024-03-21T10:18:45.549 INFO:teuthology.orchestra.run.smithi153.stdout:max_xattr_size 65536 2024-03-21T10:18:45.549 INFO:teuthology.orchestra.run.smithi153.stdout:required_client_features {} 2024-03-21T10:18:45.549 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure 0 2024-03-21T10:18:45.550 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure_osd_epoch 47 2024-03-21T10:18:45.550 INFO:teuthology.orchestra.run.smithi153.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:18:45.550 INFO:teuthology.orchestra.run.smithi153.stdout:max_mds 2 2024-03-21T10:18:45.550 INFO:teuthology.orchestra.run.smithi153.stdout:in 0,1 2024-03-21T10:18:45.550 INFO:teuthology.orchestra.run.smithi153.stdout:up {0=14484,1=24273} 2024-03-21T10:18:45.550 INFO:teuthology.orchestra.run.smithi153.stdout:failed 2024-03-21T10:18:45.550 INFO:teuthology.orchestra.run.smithi153.stdout:damaged 2024-03-21T10:18:45.550 INFO:teuthology.orchestra.run.smithi153.stdout:stopped 2024-03-21T10:18:45.550 INFO:teuthology.orchestra.run.smithi153.stdout:data_pools [3] 2024-03-21T10:18:45.550 INFO:teuthology.orchestra.run.smithi153.stdout:metadata_pool 2 2024-03-21T10:18:45.550 INFO:teuthology.orchestra.run.smithi153.stdout:inline_data enabled 2024-03-21T10:18:45.550 INFO:teuthology.orchestra.run.smithi153.stdout:balancer 2024-03-21T10:18:45.550 INFO:teuthology.orchestra.run.smithi153.stdout:bal_rank_mask -1 2024-03-21T10:18:45.550 INFO:teuthology.orchestra.run.smithi153.stdout:standby_count_wanted 1 2024-03-21T10:18:45.551 INFO:teuthology.orchestra.run.smithi153.stdout:qdb_cluster leader: 0 members: 2024-03-21T10:18:45.551 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.iuyluj{0:14484} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.153:6828/866138999,v1:172.21.15.153:6829/866138999] compat {c=[1],r=[1],i=[7ff]}] 2024-03-21T10:18:45.551 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.zipckr{0:14494} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.153:6826/4033622615,v1:172.21.15.153:6827/4033622615] compat {c=[1],r=[1],i=[7ff]}] 2024-03-21T10:18:45.551 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.yyimst{1:24273} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.174:6824/1776947618,v1:172.21.15.174:6825/1776947618] compat {c=[1],r=[1],i=[7ff]}] 2024-03-21T10:18:45.551 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.nhozag{1:24285} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.174:6826/806095873,v1:172.21.15.174:6827/806095873] compat {c=[1],r=[1],i=[7ff]}] 2024-03-21T10:18:45.551 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:18:45.551 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:18:45.552 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:45.548+0000 7f1642c00700 1 -- 172.21.15.153:0/4094735331 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f162006e460 msgr2=0x7f1620070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:18:45.552 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:45.548+0000 7f1642c00700 1 --2- 172.21.15.153:0/4094735331 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f162006e460 0x7f1620070920 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f163c1011f0 tx=0x7f162800b410 comp rx=0 tx=0).stop 2024-03-21T10:18:45.552 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:45.548+0000 7f1642c00700 1 -- 172.21.15.153:0/4094735331 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f163c101da0 msgr2=0x7f163c0ffa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:18:45.552 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:45.548+0000 7f1642c00700 1 --2- 172.21.15.153:0/4094735331 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f163c101da0 0x7f163c0ffa00 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f162c0092e0 tx=0x7f162c00bd70 comp rx=0 tx=0).stop 2024-03-21T10:18:45.552 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:45.549+0000 7f1642c00700 1 -- 172.21.15.153:0/4094735331 shutdown_connections 2024-03-21T10:18:45.552 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:45.549+0000 7f1642c00700 1 --2- 172.21.15.153:0/4094735331 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f162006e460 0x7f1620070920 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:18:45.552 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:45.549+0000 7f1642c00700 1 --2- 172.21.15.153:0/4094735331 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f163c101da0 0x7f163c0ffa00 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:18:45.552 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:45.549+0000 7f1642c00700 1 --2- 172.21.15.153:0/4094735331 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f163c1026c0 0x7f163c0fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:18:45.553 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:45.549+0000 7f1642c00700 1 -- 172.21.15.153:0/4094735331 >> 172.21.15.153:0/4094735331 conn(0x7f163c0fb570 msgr2=0x7f163c1053f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:18:45.553 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:45.549+0000 7f1642c00700 1 -- 172.21.15.153:0/4094735331 shutdown_connections 2024-03-21T10:18:45.553 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:45.549+0000 7f1642c00700 1 -- 172.21.15.153:0/4094735331 wait complete. 2024-03-21T10:18:45.553 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 21 2024-03-21T10:18:45.578 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 21 10:18:45 smithi174 podman[107037]: 2024-03-21 10:18:45.257889411 +0000 UTC m=+0.031264886 container died 2c8bcc0183df5f6475174b935c6affdf257ca7239974ddeb51b64349bbaae658 (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-5-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-03-21T10:18:45.578 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:45 smithi174 ceph-mon[88004]: from='client.34406 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:18:45.579 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:45 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/3651920590' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:18:45.579 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:45 smithi174 ceph-mon[88004]: pgmap v198: 65 pgs: 7 active+undersized, 5 active+undersized+degraded, 6 stale+active+clean, 47 active+clean; 317 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 19/366 objects degraded (5.191%) 2024-03-21T10:18:45.702 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:45.700+0000 7fe139800700 1 -- 172.21.15.153:0/53173430 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe134103000 msgr2=0x7fe1341033e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:18:45.702 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:45.700+0000 7fe139800700 1 --2- 172.21.15.153:0/53173430 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe134103000 0x7fe1341033e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe124009910 tx=0x7fe124009c20 comp rx=0 tx=0).stop 2024-03-21T10:18:45.702 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:45.700+0000 7fe139800700 1 -- 172.21.15.153:0/53173430 shutdown_connections 2024-03-21T10:18:45.702 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:45.700+0000 7fe139800700 1 --2- 172.21.15.153:0/53173430 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe1341039b0 0x7fe134107a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:18:45.702 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:45.700+0000 7fe139800700 1 --2- 172.21.15.153:0/53173430 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe134103000 0x7fe1341033e0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:18:45.702 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:45.700+0000 7fe139800700 1 -- 172.21.15.153:0/53173430 >> 172.21.15.153:0/53173430 conn(0x7fe1340fe8d0 msgr2=0x7fe134100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:18:45.703 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:45 smithi153 ceph-mon[113733]: from='client.34406 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:18:45.703 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:45 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/3651920590' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:18:45.703 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:45 smithi153 ceph-mon[113733]: pgmap v198: 65 pgs: 7 active+undersized, 5 active+undersized+degraded, 6 stale+active+clean, 47 active+clean; 317 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 19/366 objects degraded (5.191%) 2024-03-21T10:18:45.704 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:45.700+0000 7fe139800700 1 -- 172.21.15.153:0/53173430 shutdown_connections 2024-03-21T10:18:45.704 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:45.700+0000 7fe139800700 1 -- 172.21.15.153:0/53173430 wait complete. 2024-03-21T10:18:45.704 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:45.701+0000 7fe139800700 1 Processor -- start 2024-03-21T10:18:45.704 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:45.701+0000 7fe139800700 1 -- start start 2024-03-21T10:18:45.704 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:45.702+0000 7fe139800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe134103000 0x7fe13419e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:18:45.705 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:45.702+0000 7fe139800700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe1341039b0 0x7fe13419ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:18:45.705 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:45.702+0000 7fe139800700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe13419f330 con 0x7fe134103000 2024-03-21T10:18:45.705 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:45.702+0000 7fe139800700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe134198870 con 0x7fe1341039b0 2024-03-21T10:18:45.705 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:45.702+0000 7fe132a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe134103000 0x7fe13419e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:18:45.705 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:45.702+0000 7fe132a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe134103000 0x7fe13419e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:44714/0 (socket says 172.21.15.153:44714) 2024-03-21T10:18:45.705 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:45.702+0000 7fe132a00700 1 -- 172.21.15.153:0/4143222753 learned_addr learned my addr 172.21.15.153:0/4143222753 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:18:45.705 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:45.702+0000 7fe132a00700 1 -- 172.21.15.153:0/4143222753 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe1341039b0 msgr2=0x7fe13419ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:18:45.705 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:45.702+0000 7fe132a00700 1 --2- 172.21.15.153:0/4143222753 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe1341039b0 0x7fe13419ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:18:45.705 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:45.702+0000 7fe132a00700 1 -- 172.21.15.153:0/4143222753 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1240095f0 con 0x7fe134103000 2024-03-21T10:18:45.705 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:45.703+0000 7fe132a00700 1 --2- 172.21.15.153:0/4143222753 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe134103000 0x7fe13419e7f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe12400b5c0 tx=0x7fe12400be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:18:45.706 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:45.703+0000 7fe12b400700 1 -- 172.21.15.153:0/4143222753 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe12401d070 con 0x7fe134103000 2024-03-21T10:18:45.706 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:45.703+0000 7fe12b400700 1 -- 172.21.15.153:0/4143222753 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe12400f460 con 0x7fe134103000 2024-03-21T10:18:45.706 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:45.703+0000 7fe139800700 1 -- 172.21.15.153:0/4143222753 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe134198af0 con 0x7fe134103000 2024-03-21T10:18:45.706 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:45.703+0000 7fe12b400700 1 -- 172.21.15.153:0/4143222753 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe124017610 con 0x7fe134103000 2024-03-21T10:18:45.706 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:45.703+0000 7fe139800700 1 -- 172.21.15.153:0/4143222753 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe134198fe0 con 0x7fe134103000 2024-03-21T10:18:45.706 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:45.704+0000 7fe139800700 1 -- 172.21.15.153:0/4143222753 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe13410b3a0 con 0x7fe134103000 2024-03-21T10:18:45.707 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:45.706+0000 7fe12b400700 1 -- 172.21.15.153:0/4143222753 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fe12400f5d0 con 0x7fe134103000 2024-03-21T10:18:45.709 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:45.707+0000 7fe12b400700 1 --2- 172.21.15.153:0/4143222753 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe11806e460 0x7fe118070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:18:45.709 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:45.707+0000 7fe12b400700 1 -- 172.21.15.153:0/4143222753 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6654+0+0 (secure 0 0 0) 0x7fe12408fc30 con 0x7fe134103000 2024-03-21T10:18:45.709 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:45.707+0000 7fe132000700 1 --2- 172.21.15.153:0/4143222753 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe11806e460 0x7fe118070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:18:45.709 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:45.708+0000 7fe132000700 1 --2- 172.21.15.153:0/4143222753 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe11806e460 0x7fe118070920 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe120005fd0 tx=0x7fe120005d00 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:18:45.712 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:45.711+0000 7fe12b400700 1 -- 172.21.15.153:0/4143222753 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe12405c950 con 0x7fe134103000 2024-03-21T10:18:45.938 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:45.935+0000 7fe139800700 1 -- 172.21.15.153:0/4143222753 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe134199200 con 0x7fe11806e460 2024-03-21T10:18:45.938 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:45.937+0000 7fe12b400700 1 -- 172.21.15.153:0/4143222753 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7fe134199200 con 0x7fe11806e460 2024-03-21T10:18:45.939 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:18:45.939 INFO:teuthology.orchestra.run.smithi153.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232", 2024-03-21T10:18:45.939 INFO:teuthology.orchestra.run.smithi153.stdout: "in_progress": true, 2024-03-21T10:18:45.939 INFO:teuthology.orchestra.run.smithi153.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-21T10:18:45.939 INFO:teuthology.orchestra.run.smithi153.stdout: "services_complete": [ 2024-03-21T10:18:45.939 INFO:teuthology.orchestra.run.smithi153.stdout: "crash", 2024-03-21T10:18:45.939 INFO:teuthology.orchestra.run.smithi153.stdout: "mon", 2024-03-21T10:18:45.939 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr" 2024-03-21T10:18:45.939 INFO:teuthology.orchestra.run.smithi153.stdout: ], 2024-03-21T10:18:45.939 INFO:teuthology.orchestra.run.smithi153.stdout: "progress": "11/23 daemons upgraded", 2024-03-21T10:18:45.939 INFO:teuthology.orchestra.run.smithi153.stdout: "message": "Currently upgrading osd daemons", 2024-03-21T10:18:45.939 INFO:teuthology.orchestra.run.smithi153.stdout: "is_paused": false 2024-03-21T10:18:45.939 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:18:45.942 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:45.940+0000 7fe139800700 1 -- 172.21.15.153:0/4143222753 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe11806e460 msgr2=0x7fe118070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:18:45.942 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:45.940+0000 7fe139800700 1 --2- 172.21.15.153:0/4143222753 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe11806e460 0x7fe118070920 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe120005fd0 tx=0x7fe120005d00 comp rx=0 tx=0).stop 2024-03-21T10:18:45.942 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:45.940+0000 7fe139800700 1 -- 172.21.15.153:0/4143222753 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe134103000 msgr2=0x7fe13419e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:18:45.942 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:45.940+0000 7fe139800700 1 --2- 172.21.15.153:0/4143222753 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe134103000 0x7fe13419e7f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe12400b5c0 tx=0x7fe12400be00 comp rx=0 tx=0).stop 2024-03-21T10:18:45.942 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:45.940+0000 7fe139800700 1 -- 172.21.15.153:0/4143222753 shutdown_connections 2024-03-21T10:18:45.942 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:45.940+0000 7fe139800700 1 --2- 172.21.15.153:0/4143222753 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe11806e460 0x7fe118070920 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:18:45.942 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:45.940+0000 7fe139800700 1 --2- 172.21.15.153:0/4143222753 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe134103000 0x7fe13419e7f0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:18:45.942 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:45.940+0000 7fe139800700 1 --2- 172.21.15.153:0/4143222753 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe1341039b0 0x7fe13419ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:18:45.942 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:45.940+0000 7fe139800700 1 -- 172.21.15.153:0/4143222753 >> 172.21.15.153:0/4143222753 conn(0x7fe1340fe8d0 msgr2=0x7fe1340ffee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:18:45.942 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:45.940+0000 7fe139800700 1 -- 172.21.15.153:0/4143222753 shutdown_connections 2024-03-21T10:18:45.943 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:45.940+0000 7fe139800700 1 -- 172.21.15.153:0/4143222753 wait complete. 2024-03-21T10:18:46.098 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:46.096+0000 7f12b1000700 1 -- 172.21.15.153:0/1916484287 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f12ac10d280 msgr2=0x7f12ac10d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:18:46.098 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:46.096+0000 7f12b1000700 1 --2- 172.21.15.153:0/1916484287 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f12ac10d280 0x7f12ac10d660 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f129c009910 tx=0x7f129c009c20 comp rx=0 tx=0).stop 2024-03-21T10:18:46.098 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:46.096+0000 7f12b1000700 1 -- 172.21.15.153:0/1916484287 shutdown_connections 2024-03-21T10:18:46.098 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:46.096+0000 7f12b1000700 1 --2- 172.21.15.153:0/1916484287 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f12ac1071d0 0x7f12ac107650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:18:46.098 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:46.096+0000 7f12b1000700 1 --2- 172.21.15.153:0/1916484287 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f12ac10d280 0x7f12ac10d660 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:18:46.098 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:46.096+0000 7f12b1000700 1 -- 172.21.15.153:0/1916484287 >> 172.21.15.153:0/1916484287 conn(0x7f12ac0769e0 msgr2=0x7f12ac078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:18:46.098 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:46.096+0000 7f12b1000700 1 -- 172.21.15.153:0/1916484287 shutdown_connections 2024-03-21T10:18:46.098 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:46.096+0000 7f12b1000700 1 -- 172.21.15.153:0/1916484287 wait complete. 2024-03-21T10:18:46.100 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:46.097+0000 7f12b1000700 1 Processor -- start 2024-03-21T10:18:46.100 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:46.097+0000 7f12b1000700 1 -- start start 2024-03-21T10:18:46.100 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:46.098+0000 7f12b1000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f12ac1071d0 0x7f12ac19ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:18:46.100 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:46.098+0000 7f12b1000700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f12ac10d280 0x7f12ac19d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:18:46.100 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:46.098+0000 7f12b1000700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12ac19da30 con 0x7f12ac1071d0 2024-03-21T10:18:46.100 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:46.098+0000 7f12b1000700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12ac1a1770 con 0x7f12ac10d280 2024-03-21T10:18:46.100 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:46.098+0000 7f12a9e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f12ac1071d0 0x7f12ac19ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:18:46.100 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:46.098+0000 7f12a9e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f12ac1071d0 0x7f12ac19ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:44736/0 (socket says 172.21.15.153:44736) 2024-03-21T10:18:46.100 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:46.098+0000 7f12a9e00700 1 -- 172.21.15.153:0/3541507387 learned_addr learned my addr 172.21.15.153:0/3541507387 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:18:46.100 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:46.098+0000 7f12a9400700 1 --2- 172.21.15.153:0/3541507387 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f12ac10d280 0x7f12ac19d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:18:46.100 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:46.098+0000 7f12a9e00700 1 -- 172.21.15.153:0/3541507387 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f12ac10d280 msgr2=0x7f12ac19d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:18:46.101 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:46.098+0000 7f12a9e00700 1 --2- 172.21.15.153:0/3541507387 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f12ac10d280 0x7f12ac19d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:18:46.101 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:46.098+0000 7f12a9e00700 1 -- 172.21.15.153:0/3541507387 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f129c0095f0 con 0x7f12ac1071d0 2024-03-21T10:18:46.101 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:46.099+0000 7f12a9e00700 1 --2- 172.21.15.153:0/3541507387 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f12ac1071d0 0x7f12ac19ce10 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f129c006010 tx=0x7f129c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:18:46.101 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:46.099+0000 7f12a2a00700 1 -- 172.21.15.153:0/3541507387 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f129c01d070 con 0x7f12ac1071d0 2024-03-21T10:18:46.101 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:46.099+0000 7f12a2a00700 1 -- 172.21.15.153:0/3541507387 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f129c004fb0 con 0x7f12ac1071d0 2024-03-21T10:18:46.101 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:46.099+0000 7f12b1000700 1 -- 172.21.15.153:0/3541507387 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12ac1a19f0 con 0x7f12ac1071d0 2024-03-21T10:18:46.102 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:46.099+0000 7f12a2a00700 1 -- 172.21.15.153:0/3541507387 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f129c022ea0 con 0x7f12ac1071d0 2024-03-21T10:18:46.102 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:46.099+0000 7f12b1000700 1 -- 172.21.15.153:0/3541507387 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12ac1a1f60 con 0x7f12ac1071d0 2024-03-21T10:18:46.102 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:46.100+0000 7f12b1000700 1 -- 172.21.15.153:0/3541507387 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12ac10f6f0 con 0x7f12ac1071d0 2024-03-21T10:18:46.103 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:46.101+0000 7f12a2a00700 1 -- 172.21.15.153:0/3541507387 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f129c022470 con 0x7f12ac1071d0 2024-03-21T10:18:46.104 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:46.102+0000 7f12a2a00700 1 --2- 172.21.15.153:0/3541507387 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f129006e460 0x7f1290070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:18:46.104 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:46.102+0000 7f12a2a00700 1 -- 172.21.15.153:0/3541507387 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6654+0+0 (secure 0 0 0) 0x7f129c08fa70 con 0x7f12ac1071d0 2024-03-21T10:18:46.104 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:46.103+0000 7f12a9400700 1 --2- 172.21.15.153:0/3541507387 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f129006e460 0x7f1290070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:18:46.105 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:46.103+0000 7f12a9400700 1 --2- 172.21.15.153:0/3541507387 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f129006e460 0x7f1290070920 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1298006fd0 tx=0x7f1298008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:18:46.109 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:46.107+0000 7f12a2a00700 1 -- 172.21.15.153:0/3541507387 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f129c058bf0 con 0x7f12ac1071d0 2024-03-21T10:18:46.202 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 21 10:18:45 smithi174 podman[107037]: 2024-03-21 10:18:45.880757356 +0000 UTC m=+0.654132901 container remove 2c8bcc0183df5f6475174b935c6affdf257ca7239974ddeb51b64349bbaae658 (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-5-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-03-21T10:18:46.202 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 21 10:18:45 smithi174 systemd[1]: ceph-070ed3e4-e769-11ee-95cd-87774f69a715@osd.5.service: Deactivated successfully. 2024-03-21T10:18:46.202 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 21 10:18:45 smithi174 systemd[1]: Stopped Ceph osd.5 for 070ed3e4-e769-11ee-95cd-87774f69a715. 2024-03-21T10:18:46.202 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 21 10:18:45 smithi174 systemd[1]: ceph-070ed3e4-e769-11ee-95cd-87774f69a715@osd.5.service: Consumed 1min 43.512s CPU time. 2024-03-21T10:18:46.431 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:46.430+0000 7f12b1000700 1 -- 172.21.15.153:0/3541507387 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f12ac04ea90 con 0x7f12ac1071d0 2024-03-21T10:18:46.432 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:46.431+0000 7f12a2a00700 1 -- 172.21.15.153:0/3541507387 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+976 (secure 0 0 0) 0x7f129c022720 con 0x7f12ac1071d0 2024-03-21T10:18:46.433 INFO:teuthology.orchestra.run.smithi153.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data; 1 osds down; all OSDs are running squid or later but require_osd_release < squid; Degraded data redundancy: 19/366 objects degraded (5.191%), 5 pgs degraded 2024-03-21T10:18:46.433 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-21T10:18:46.433 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-21T10:18:46.433 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] OSD_DOWN: 1 osds down 2024-03-21T10:18:46.433 INFO:teuthology.orchestra.run.smithi153.stdout: osd.5 (root=default,host=smithi174) is down 2024-03-21T10:18:46.433 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] OSD_UPGRADE_FINISHED: all OSDs are running squid or later but require_osd_release < squid 2024-03-21T10:18:46.433 INFO:teuthology.orchestra.run.smithi153.stdout: all OSDs are running squid or later but require_osd_release < squid 2024-03-21T10:18:46.433 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 19/366 objects degraded (5.191%), 5 pgs degraded 2024-03-21T10:18:46.433 INFO:teuthology.orchestra.run.smithi153.stdout: pg 2.f is active+undersized+degraded, acting [4,0] 2024-03-21T10:18:46.433 INFO:teuthology.orchestra.run.smithi153.stdout: pg 2.14 is active+undersized+degraded, acting [0,4] 2024-03-21T10:18:46.433 INFO:teuthology.orchestra.run.smithi153.stdout: pg 2.16 is active+undersized+degraded, acting [0,2] 2024-03-21T10:18:46.433 INFO:teuthology.orchestra.run.smithi153.stdout: pg 2.18 is active+undersized+degraded, acting [4,3] 2024-03-21T10:18:46.433 INFO:teuthology.orchestra.run.smithi153.stdout: pg 2.1c is active+undersized+degraded, acting [4,2] 2024-03-21T10:18:46.435 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:46.434+0000 7f12b1000700 1 -- 172.21.15.153:0/3541507387 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f129006e460 msgr2=0x7f1290070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:18:46.436 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:46.434+0000 7f12b1000700 1 --2- 172.21.15.153:0/3541507387 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f129006e460 0x7f1290070920 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1298006fd0 tx=0x7f1298008040 comp rx=0 tx=0).stop 2024-03-21T10:18:46.436 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:46.434+0000 7f12b1000700 1 -- 172.21.15.153:0/3541507387 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f12ac1071d0 msgr2=0x7f12ac19ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:18:46.436 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:46.434+0000 7f12b1000700 1 --2- 172.21.15.153:0/3541507387 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f12ac1071d0 0x7f12ac19ce10 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f129c006010 tx=0x7f129c00be20 comp rx=0 tx=0).stop 2024-03-21T10:18:46.436 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:46.434+0000 7f12b1000700 1 -- 172.21.15.153:0/3541507387 shutdown_connections 2024-03-21T10:18:46.436 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:46.434+0000 7f12b1000700 1 --2- 172.21.15.153:0/3541507387 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f129006e460 0x7f1290070920 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:18:46.436 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:46.434+0000 7f12b1000700 1 --2- 172.21.15.153:0/3541507387 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f12ac1071d0 0x7f12ac19ce10 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:18:46.436 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:46.434+0000 7f12b1000700 1 --2- 172.21.15.153:0/3541507387 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f12ac10d280 0x7f12ac19d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:18:46.437 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:46.434+0000 7f12b1000700 1 -- 172.21.15.153:0/3541507387 >> 172.21.15.153:0/3541507387 conn(0x7f12ac0769e0 msgr2=0x7f12ac0785c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:18:46.437 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:46.434+0000 7f12b1000700 1 -- 172.21.15.153:0/3541507387 shutdown_connections 2024-03-21T10:18:46.437 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:18:46.434+0000 7f12b1000700 1 -- 172.21.15.153:0/3541507387 wait complete. 2024-03-21T10:18:46.641 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 21 10:18:46 smithi174 systemd[1]: Starting Ceph osd.5 for 070ed3e4-e769-11ee-95cd-87774f69a715... 2024-03-21T10:18:46.641 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:46 smithi174 ceph-mon[88004]: Health check failed: Degraded data redundancy: 19/366 objects degraded (5.191%), 5 pgs degraded (PG_DEGRADED) 2024-03-21T10:18:46.642 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:46 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/4094735331' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:18:46.642 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:46 smithi174 ceph-mon[88004]: from='client.34418 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:18:46.642 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:46 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:18:46.642 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:46 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:18:46.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:46 smithi153 ceph-mon[113733]: Health check failed: Degraded data redundancy: 19/366 objects degraded (5.191%), 5 pgs degraded (PG_DEGRADED) 2024-03-21T10:18:46.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:46 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/4094735331' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:18:46.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:46 smithi153 ceph-mon[113733]: from='client.34418 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:18:46.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:46 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:18:46.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:46 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:18:46.920 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 21 10:18:46 smithi174 podman[107139]: 2024-03-21 10:18:46.639887529 +0000 UTC m=+0.122878190 container create 86f375693e50798ead8b9b647b50245c917fb2d74c14fb71d083705e4d94e031 (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-5-activate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True) 2024-03-21T10:18:46.920 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 21 10:18:46 smithi174 podman[107139]: 2024-03-21 10:18:46.552788466 +0000 UTC m=+0.035779129 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T10:18:46.920 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 21 10:18:46 smithi174 podman[107139]: 2024-03-21 10:18:46.918606784 +0000 UTC m=+0.401597488 container init 86f375693e50798ead8b9b647b50245c917fb2d74c14fb71d083705e4d94e031 (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-5-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-03-21T10:18:47.201 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 21 10:18:46 smithi174 podman[107139]: 2024-03-21 10:18:46.929961562 +0000 UTC m=+0.412952257 container start 86f375693e50798ead8b9b647b50245c917fb2d74c14fb71d083705e4d94e031 (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-5-activate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6) 2024-03-21T10:18:47.202 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 21 10:18:47 smithi174 podman[107139]: 2024-03-21 10:18:47.008942772 +0000 UTC m=+0.491933417 container attach 86f375693e50798ead8b9b647b50245c917fb2d74c14fb71d083705e4d94e031 (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-5-activate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, io.buildah.version=1.33.5, ceph=True) 2024-03-21T10:18:47.557 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:47 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/3541507387' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:18:47.558 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:47 smithi174 ceph-mon[88004]: pgmap v199: 65 pgs: 12 active+undersized, 10 active+undersized+degraded, 2 stale+active+clean, 41 active+clean; 317 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 35/366 objects degraded (9.563%) 2024-03-21T10:18:47.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:47 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/3541507387' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:18:47.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:47 smithi153 ceph-mon[113733]: pgmap v199: 65 pgs: 12 active+undersized, 10 active+undersized+degraded, 2 stale+active+clean, 41 active+clean; 317 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 35/366 objects degraded (9.563%) 2024-03-21T10:18:47.836 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 21 10:18:47 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-5-activate[107154]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-03-21T10:18:47.836 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 21 10:18:47 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-5-activate[107154]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-03-21T10:18:47.836 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 21 10:18:47 smithi174 bash[107139]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-03-21T10:18:47.836 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 21 10:18:47 smithi174 bash[107139]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-03-21T10:18:47.836 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 21 10:18:47 smithi174 bash[107139]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-03-21T10:18:47.836 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 21 10:18:47 smithi174 bash[107139]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-03-21T10:18:47.836 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 21 10:18:47 smithi174 bash[107139]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5/block 2024-03-21T10:18:47.836 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 21 10:18:47 smithi174 bash[107139]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-03-21T10:18:47.836 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 21 10:18:47 smithi174 bash[107139]: --> ceph-volume raw activate successful for osd ID: 5 2024-03-21T10:18:47.836 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 21 10:18:47 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-5-activate[107154]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-03-21T10:18:47.836 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 21 10:18:47 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-5-activate[107154]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-03-21T10:18:47.837 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 21 10:18:47 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-5-activate[107154]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5/block 2024-03-21T10:18:47.837 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 21 10:18:47 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-5-activate[107154]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-03-21T10:18:47.837 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 21 10:18:47 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-5-activate[107154]: --> ceph-volume raw activate successful for osd ID: 5 2024-03-21T10:18:47.837 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 21 10:18:47 smithi174 podman[107139]: 2024-03-21 10:18:47.832252016 +0000 UTC m=+1.315242658 container died 86f375693e50798ead8b9b647b50245c917fb2d74c14fb71d083705e4d94e031 (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-5-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2) 2024-03-21T10:18:48.951 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 21 10:18:48 smithi174 podman[107286]: 2024-03-21 10:18:48.595748574 +0000 UTC m=+0.743110924 container remove 86f375693e50798ead8b9b647b50245c917fb2d74c14fb71d083705e4d94e031 (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-5-activate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319) 2024-03-21T10:18:49.251 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 21 10:18:48 smithi174 podman[107349]: 2024-03-21 10:18:48.856487958 +0000 UTC m=+0.031051470 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T10:18:49.251 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 21 10:18:48 smithi174 podman[107349]: 2024-03-21 10:18:48.963512732 +0000 UTC m=+0.138076291 container create 53e5f35b8cac50c4e87ebf5975a1afe9781dec032450aaa403128bc38f3c0a4d (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-5, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-03-21T10:18:49.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:49 smithi174 ceph-mon[88004]: pgmap v200: 65 pgs: 16 active+undersized, 14 active+undersized+degraded, 35 active+clean; 317 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 55/366 objects degraded (15.027%) 2024-03-21T10:18:49.702 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 21 10:18:49 smithi174 podman[107349]: 2024-03-21 10:18:49.250493973 +0000 UTC m=+0.425057528 container init 53e5f35b8cac50c4e87ebf5975a1afe9781dec032450aaa403128bc38f3c0a4d (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T10:18:49.702 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 21 10:18:49 smithi174 podman[107349]: 2024-03-21 10:18:49.26209714 +0000 UTC m=+0.436660657 container start 53e5f35b8cac50c4e87ebf5975a1afe9781dec032450aaa403128bc38f3c0a4d (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-03-21T10:18:49.702 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 21 10:18:49 smithi174 ceph-osd[107367]: -- 172.21.15.174:0/1896962443 <== mon.1 v2:172.21.15.174:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55d2ddfd3680 con 0x55d2ddf9d000 2024-03-21T10:18:49.702 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 21 10:18:49 smithi174 bash[107349]: 53e5f35b8cac50c4e87ebf5975a1afe9781dec032450aaa403128bc38f3c0a4d 2024-03-21T10:18:49.702 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 21 10:18:49 smithi174 systemd[1]: Started Ceph osd.5 for 070ed3e4-e769-11ee-95cd-87774f69a715. 2024-03-21T10:18:49.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:49 smithi153 ceph-mon[113733]: pgmap v200: 65 pgs: 16 active+undersized, 14 active+undersized+degraded, 35 active+clean; 317 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 55/366 objects degraded (15.027%) 2024-03-21T10:18:50.201 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 21 10:18:49 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-5[107363]: 2024-03-21T10:18:49.826+0000 7f6dfeecf7c0 -1 Falling back to public interface 2024-03-21T10:18:50.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:50 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:18:50.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:50 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:18:50.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:50 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:18:50.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:50 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:18:50.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:50 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:18:50.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:50 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:18:50.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:50 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:18:50.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:50 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:18:50.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:50 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:18:50.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:50 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:18:51.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:51 smithi153 ceph-mon[113733]: Health check update: Degraded data redundancy: 55/366 objects degraded (15.027%), 14 pgs degraded (PG_DEGRADED) 2024-03-21T10:18:51.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:51 smithi153 ceph-mon[113733]: pgmap v201: 65 pgs: 16 active+undersized, 14 active+undersized+degraded, 35 active+clean; 317 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 55/366 objects degraded (15.027%) 2024-03-21T10:18:51.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:51 smithi174 ceph-mon[88004]: Health check update: Degraded data redundancy: 55/366 objects degraded (15.027%), 14 pgs degraded (PG_DEGRADED) 2024-03-21T10:18:51.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:51 smithi174 ceph-mon[88004]: pgmap v201: 65 pgs: 16 active+undersized, 14 active+undersized+degraded, 35 active+clean; 317 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 55/366 objects degraded (15.027%) 2024-03-21T10:18:53.659 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:53 smithi174 ceph-mon[88004]: pgmap v202: 65 pgs: 16 active+undersized, 14 active+undersized+degraded, 35 active+clean; 317 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 55/366 objects degraded (15.027%) 2024-03-21T10:18:53.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:53 smithi153 ceph-mon[113733]: pgmap v202: 65 pgs: 16 active+undersized, 14 active+undersized+degraded, 35 active+clean; 317 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 55/366 objects degraded (15.027%) 2024-03-21T10:18:54.701 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 21 10:18:54 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-5[107363]: 2024-03-21T10:18:54.391+0000 7f6dfeecf7c0 -1 osd.5 0 read_superblock omap replica is missing. 2024-03-21T10:18:55.201 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 21 10:18:54 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-5[107363]: 2024-03-21T10:18:54.756+0000 7f6dfeecf7c0 -1 osd.5 86 log_to_monitors true 2024-03-21T10:18:56.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:55 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:18:56.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:55 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:18:56.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:55 smithi153 ceph-mon[113733]: from='osd.5 [v2:172.21.15.174:6816/385015973,v1:172.21.15.174:6817/385015973]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-21T10:18:56.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:55 smithi153 ceph-mon[113733]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-21T10:18:56.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:55 smithi153 ceph-mon[113733]: pgmap v203: 65 pgs: 16 active+undersized, 14 active+undersized+degraded, 35 active+clean; 317 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 55/366 objects degraded (15.027%) 2024-03-21T10:18:56.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:55 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:18:56.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:55 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:18:56.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:55 smithi174 ceph-mon[88004]: from='osd.5 [v2:172.21.15.174:6816/385015973,v1:172.21.15.174:6817/385015973]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-21T10:18:56.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:55 smithi174 ceph-mon[88004]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-21T10:18:56.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:55 smithi174 ceph-mon[88004]: pgmap v203: 65 pgs: 16 active+undersized, 14 active+undersized+degraded, 35 active+clean; 317 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 55/366 objects degraded (15.027%) 2024-03-21T10:18:57.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:56 smithi153 ceph-mon[113733]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-21T10:18:57.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:56 smithi153 ceph-mon[113733]: osdmap e89: 6 total, 5 up, 6 in 2024-03-21T10:18:57.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:56 smithi153 ceph-mon[113733]: from='osd.5 [v2:172.21.15.174:6816/385015973,v1:172.21.15.174:6817/385015973]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-03-21T10:18:57.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:56 smithi153 ceph-mon[113733]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-03-21T10:18:57.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:56 smithi174 ceph-mon[88004]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-21T10:18:57.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:56 smithi174 ceph-mon[88004]: osdmap e89: 6 total, 5 up, 6 in 2024-03-21T10:18:57.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:56 smithi174 ceph-mon[88004]: from='osd.5 [v2:172.21.15.174:6816/385015973,v1:172.21.15.174:6817/385015973]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-03-21T10:18:57.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:56 smithi174 ceph-mon[88004]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-03-21T10:18:57.202 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 21 10:18:56 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-5[107363]: 2024-03-21T10:18:56.861+0000 7f6de9200700 -1 osd.5 86 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-21T10:18:58.156 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:57 smithi174 ceph-mon[88004]: from='osd.5 ' entity='osd.5' 2024-03-21T10:18:58.156 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:57 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:18:58.156 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:57 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:18:58.156 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:57 smithi174 ceph-mon[88004]: pgmap v205: 65 pgs: 16 active+undersized, 14 active+undersized+degraded, 35 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s; 55/366 objects degraded (15.027%) 2024-03-21T10:18:58.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:57 smithi153 ceph-mon[113733]: from='osd.5 ' entity='osd.5' 2024-03-21T10:18:58.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:57 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:18:58.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:57 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:18:58.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:57 smithi153 ceph-mon[113733]: pgmap v205: 65 pgs: 16 active+undersized, 14 active+undersized+degraded, 35 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s; 55/366 objects degraded (15.027%) 2024-03-21T10:18:59.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:58 smithi174 ceph-mon[88004]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-21T10:18:59.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:58 smithi174 ceph-mon[88004]: osd.5 [v2:172.21.15.174:6816/385015973,v1:172.21.15.174:6817/385015973] boot 2024-03-21T10:18:59.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:58 smithi174 ceph-mon[88004]: osdmap e90: 6 total, 6 up, 6 in 2024-03-21T10:18:59.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:58 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T10:18:59.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:58 smithi174 ceph-mon[88004]: osdmap e91: 6 total, 6 up, 6 in 2024-03-21T10:18:59.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:58 smithi153 ceph-mon[113733]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-21T10:18:59.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:58 smithi153 ceph-mon[113733]: osd.5 [v2:172.21.15.174:6816/385015973,v1:172.21.15.174:6817/385015973] boot 2024-03-21T10:18:59.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:58 smithi153 ceph-mon[113733]: osdmap e90: 6 total, 6 up, 6 in 2024-03-21T10:18:59.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:58 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T10:18:59.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:58 smithi153 ceph-mon[113733]: osdmap e91: 6 total, 6 up, 6 in 2024-03-21T10:19:00.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:18:59 smithi153 ceph-mon[113733]: pgmap v208: 65 pgs: 3 peering, 15 active+undersized, 12 active+undersized+degraded, 35 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s; 51/366 objects degraded (13.934%) 2024-03-21T10:19:00.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:18:59 smithi174 ceph-mon[88004]: pgmap v208: 65 pgs: 3 peering, 15 active+undersized, 12 active+undersized+degraded, 35 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s; 51/366 objects degraded (13.934%) 2024-03-21T10:19:01.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:00 smithi174 ceph-mon[88004]: Health check update: Degraded data redundancy: 51/366 objects degraded (13.934%), 12 pgs degraded (PG_DEGRADED) 2024-03-21T10:19:01.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:00 smithi153 ceph-mon[113733]: Health check update: Degraded data redundancy: 51/366 objects degraded (13.934%), 12 pgs degraded (PG_DEGRADED) 2024-03-21T10:19:02.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:02 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:19:02.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:02 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:19:02.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:02 smithi174 ceph-mon[88004]: pgmap v209: 65 pgs: 3 peering, 2 active+undersized, 4 active+undersized+degraded, 56 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 2 op/s; 20/366 objects degraded (5.464%) 2024-03-21T10:19:02.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:02 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:19:02.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:02 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:19:02.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:02 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:19:02.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:02 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:19:02.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:02 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:19:02.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:02 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:19:02.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:02 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:19:02.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:02 smithi153 ceph-mon[113733]: pgmap v209: 65 pgs: 3 peering, 2 active+undersized, 4 active+undersized+degraded, 56 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 2 op/s; 20/366 objects degraded (5.464%) 2024-03-21T10:19:02.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:02 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:19:02.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:02 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:19:02.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:02 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:19:02.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:02 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:19:02.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:02 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:19:03.506 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:03 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:19:03.507 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:03 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:19:03.507 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:03 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:19:03.507 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:03 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:19:03.507 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:03 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:19:03.507 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:03 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:19:03.507 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:03 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-03-21T10:19:03.507 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:03 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-03-21T10:19:03.507 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:03 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-03-21T10:19:03.507 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:03 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-03-21T10:19:03.507 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:03 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-03-21T10:19:03.507 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:03 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-03-21T10:19:03.507 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:03 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-03-21T10:19:03.508 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:03 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-03-21T10:19:03.508 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:03 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-03-21T10:19:03.508 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:03 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-03-21T10:19:03.508 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:03 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-03-21T10:19:03.508 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:03 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-03-21T10:19:03.508 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:03 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd require-osd-release", "release": "squid"}]: dispatch 2024-03-21T10:19:03.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:03 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:19:03.558 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:03 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:19:03.558 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:03 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:19:03.558 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:03 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:19:03.558 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:03 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:19:03.558 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:03 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:19:03.558 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:03 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-03-21T10:19:03.558 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:03 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-03-21T10:19:03.558 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:03 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-03-21T10:19:03.558 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:03 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-03-21T10:19:03.558 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:03 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-03-21T10:19:03.558 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:03 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-03-21T10:19:03.558 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:03 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-03-21T10:19:03.558 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:03 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-03-21T10:19:03.559 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:03 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-03-21T10:19:03.559 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:03 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-03-21T10:19:03.559 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:03 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-03-21T10:19:03.559 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:03 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-03-21T10:19:03.559 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:03 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd require-osd-release", "release": "squid"}]: dispatch 2024-03-21T10:19:04.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:04 smithi174 ceph-mon[88004]: Upgrade: Setting container_image for all osd 2024-03-21T10:19:04.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:04 smithi174 ceph-mon[88004]: Upgrade: Setting require_osd_release to 19 squid 2024-03-21T10:19:04.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:04 smithi174 ceph-mon[88004]: pgmap v210: 65 pgs: 3 peering, 62 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s 2024-03-21T10:19:04.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:04 smithi174 ceph-mon[88004]: Health check cleared: OSD_UPGRADE_FINISHED (was: all OSDs are running squid or later but require_osd_release < squid) 2024-03-21T10:19:04.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:04 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "osd require-osd-release", "release": "squid"}]': finished 2024-03-21T10:19:04.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:04 smithi174 ceph-mon[88004]: osdmap e92: 6 total, 6 up, 6 in 2024-03-21T10:19:04.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:04 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:19:04.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:04 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]: dispatch 2024-03-21T10:19:04.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:04 smithi153 ceph-mon[113733]: Upgrade: Setting container_image for all osd 2024-03-21T10:19:04.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:04 smithi153 ceph-mon[113733]: Upgrade: Setting require_osd_release to 19 squid 2024-03-21T10:19:04.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:04 smithi153 ceph-mon[113733]: pgmap v210: 65 pgs: 3 peering, 62 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s 2024-03-21T10:19:04.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:04 smithi153 ceph-mon[113733]: Health check cleared: OSD_UPGRADE_FINISHED (was: all OSDs are running squid or later but require_osd_release < squid) 2024-03-21T10:19:04.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:04 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "osd require-osd-release", "release": "squid"}]': finished 2024-03-21T10:19:04.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:04 smithi153 ceph-mon[113733]: osdmap e92: 6 total, 6 up, 6 in 2024-03-21T10:19:04.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:04 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:19:04.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:04 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]: dispatch 2024-03-21T10:19:05.175 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:05 smithi153 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-mon-smithi153[113729]: 2024-03-21T10:19:05.022+0000 7f2166c00700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-21T10:19:05.453 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:05 smithi174 ceph-mon[88004]: Upgrade: Disabling standby-replay for filesystem cephfs 2024-03-21T10:19:05.453 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:05 smithi174 ceph-mon[88004]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 20/366 objects degraded (5.464%), 4 pgs degraded) 2024-03-21T10:19:05.453 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:05 smithi174 ceph-mon[88004]: osdmap e93: 6 total, 6 up, 6 in 2024-03-21T10:19:05.453 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:05 smithi174 ceph-mon[88004]: Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY) 2024-03-21T10:19:05.453 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:05 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]': finished 2024-03-21T10:19:05.453 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:05 smithi174 ceph-mon[88004]: fsmap cephfs:2 {0=cephfs.smithi153.iuyluj=up:active,1=cephfs.smithi174.yyimst=up:active} 2024-03-21T10:19:05.453 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:05 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:19:05.453 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:05 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:19:05.453 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:05 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:19:05.453 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:05 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:19:05.453 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:05 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:19:05.453 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:05 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:19:05.453 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:05 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:19:05.454 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:05 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:19:05.454 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:05 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:19:05.454 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:05 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-21T10:19:05.454 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:05 smithi174 ceph-mon[88004]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-21T10:19:05.454 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:05 smithi174 ceph-mon[88004]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-21T10:19:05.454 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:05 smithi174 ceph-mon[88004]: Health check cleared: MDS_INSUFFICIENT_STANDBY (was: insufficient standby MDS daemons available) 2024-03-21T10:19:05.454 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:05 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-21T10:19:05.454 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:05 smithi174 ceph-mon[88004]: osdmap e94: 6 total, 6 up, 6 in 2024-03-21T10:19:05.454 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:05 smithi174 ceph-mon[88004]: mds.? [v2:172.21.15.174:6826/3967223227,v1:172.21.15.174:6827/3967223227] up:boot 2024-03-21T10:19:05.454 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:05 smithi174 ceph-mon[88004]: mds.? [v2:172.21.15.153:6826/2010306761,v1:172.21.15.153:6827/2010306761] up:boot 2024-03-21T10:19:05.454 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:05 smithi174 ceph-mon[88004]: fsmap cephfs:0/2 2 up:standby, 2 failed 2024-03-21T10:19:05.454 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:05 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi153.zipckr"}]: dispatch 2024-03-21T10:19:05.454 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:05 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi174.nhozag"}]: dispatch 2024-03-21T10:19:05.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:05 smithi153 ceph-mon[113733]: Upgrade: Disabling standby-replay for filesystem cephfs 2024-03-21T10:19:05.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:05 smithi153 ceph-mon[113733]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 20/366 objects degraded (5.464%), 4 pgs degraded) 2024-03-21T10:19:05.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:05 smithi153 ceph-mon[113733]: osdmap e93: 6 total, 6 up, 6 in 2024-03-21T10:19:05.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:05 smithi153 ceph-mon[113733]: Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY) 2024-03-21T10:19:05.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:05 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]': finished 2024-03-21T10:19:05.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:05 smithi153 ceph-mon[113733]: fsmap cephfs:2 {0=cephfs.smithi153.iuyluj=up:active,1=cephfs.smithi174.yyimst=up:active} 2024-03-21T10:19:05.558 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:05 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:19:05.558 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:05 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:19:05.558 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:05 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:19:05.558 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:05 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:19:05.558 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:05 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:19:05.558 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:05 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:19:05.558 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:05 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:19:05.558 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:05 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:19:05.558 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:05 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:19:05.558 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:05 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-21T10:19:05.558 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:05 smithi153 ceph-mon[113733]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-21T10:19:05.558 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:05 smithi153 ceph-mon[113733]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-21T10:19:05.558 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:05 smithi153 ceph-mon[113733]: Health check cleared: MDS_INSUFFICIENT_STANDBY (was: insufficient standby MDS daemons available) 2024-03-21T10:19:05.558 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:05 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-21T10:19:05.559 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:05 smithi153 ceph-mon[113733]: osdmap e94: 6 total, 6 up, 6 in 2024-03-21T10:19:05.559 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:05 smithi153 ceph-mon[113733]: mds.? [v2:172.21.15.174:6826/3967223227,v1:172.21.15.174:6827/3967223227] up:boot 2024-03-21T10:19:05.559 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:05 smithi153 ceph-mon[113733]: mds.? [v2:172.21.15.153:6826/2010306761,v1:172.21.15.153:6827/2010306761] up:boot 2024-03-21T10:19:05.559 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:05 smithi153 ceph-mon[113733]: fsmap cephfs:0/2 2 up:standby, 2 failed 2024-03-21T10:19:05.559 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:05 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi153.zipckr"}]: dispatch 2024-03-21T10:19:05.559 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:05 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi174.nhozag"}]: dispatch 2024-03-21T10:19:06.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:06 smithi153 ceph-mon[113733]: Upgrade: failing fs cephfs for rapid multi-rank mds upgrade 2024-03-21T10:19:06.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:06 smithi153 ceph-mon[113733]: pgmap v214: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s 2024-03-21T10:19:06.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:06 smithi153 ceph-mon[113733]: mds.? [v2:172.21.15.174:6824/787559196,v1:172.21.15.174:6825/787559196] up:boot 2024-03-21T10:19:06.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:06 smithi153 ceph-mon[113733]: mds.? [v2:172.21.15.153:6828/147840850,v1:172.21.15.153:6829/147840850] up:boot 2024-03-21T10:19:06.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:06 smithi153 ceph-mon[113733]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-03-21T10:19:06.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:06 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi153.iuyluj"}]: dispatch 2024-03-21T10:19:06.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:06 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi174.yyimst"}]: dispatch 2024-03-21T10:19:06.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:06 smithi174 ceph-mon[88004]: Upgrade: failing fs cephfs for rapid multi-rank mds upgrade 2024-03-21T10:19:06.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:06 smithi174 ceph-mon[88004]: pgmap v214: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s 2024-03-21T10:19:06.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:06 smithi174 ceph-mon[88004]: mds.? [v2:172.21.15.174:6824/787559196,v1:172.21.15.174:6825/787559196] up:boot 2024-03-21T10:19:06.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:06 smithi174 ceph-mon[88004]: mds.? [v2:172.21.15.153:6828/147840850,v1:172.21.15.153:6829/147840850] up:boot 2024-03-21T10:19:06.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:06 smithi174 ceph-mon[88004]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-03-21T10:19:06.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:06 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi153.iuyluj"}]: dispatch 2024-03-21T10:19:06.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:06 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi174.yyimst"}]: dispatch 2024-03-21T10:19:08.395 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:08 smithi153 ceph-mon[113733]: Upgrade: Updating mds.cephfs.smithi153.zipckr 2024-03-21T10:19:08.395 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:08 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:19:08.395 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:08 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi153.zipckr", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T10:19:08.395 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:08 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:19:08.396 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:08 smithi153 ceph-mon[113733]: Deploying daemon mds.cephfs.smithi153.zipckr on smithi153 2024-03-21T10:19:08.396 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:08 smithi153 ceph-mon[113733]: pgmap v215: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 2 op/s 2024-03-21T10:19:08.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:08 smithi174 ceph-mon[88004]: Upgrade: Updating mds.cephfs.smithi153.zipckr 2024-03-21T10:19:08.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:08 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:19:08.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:08 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi153.zipckr", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T10:19:08.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:08 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:19:08.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:08 smithi174 ceph-mon[88004]: Deploying daemon mds.cephfs.smithi153.zipckr on smithi153 2024-03-21T10:19:08.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:08 smithi174 ceph-mon[88004]: pgmap v215: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 2 op/s 2024-03-21T10:19:09.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:09 smithi153 ceph-mon[113733]: pgmap v216: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 2 op/s 2024-03-21T10:19:09.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:09 smithi174 ceph-mon[88004]: pgmap v216: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 2 op/s 2024-03-21T10:19:11.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:11 smithi174 ceph-mon[88004]: osdmap e95: 6 total, 6 up, 6 in 2024-03-21T10:19:11.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:11 smithi174 ceph-mon[88004]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-03-21T10:19:11.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:11 smithi174 ceph-mon[88004]: pgmap v218: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 2 op/s 2024-03-21T10:19:11.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:11 smithi153 ceph-mon[113733]: osdmap e95: 6 total, 6 up, 6 in 2024-03-21T10:19:11.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:11 smithi153 ceph-mon[113733]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-03-21T10:19:11.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:11 smithi153 ceph-mon[113733]: pgmap v218: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 2 op/s 2024-03-21T10:19:12.649 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:12 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:19:12.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:12 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:19:13.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:13 smithi174 ceph-mon[88004]: pgmap v219: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 498 B/s rd, 1 op/s 2024-03-21T10:19:13.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:13 smithi153 ceph-mon[113733]: pgmap v219: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 498 B/s rd, 1 op/s 2024-03-21T10:19:15.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:15 smithi174 ceph-mon[88004]: pgmap v220: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 409 B/s rd, 1 op/s 2024-03-21T10:19:15.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:15 smithi153 ceph-mon[113733]: pgmap v220: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 409 B/s rd, 1 op/s 2024-03-21T10:19:16.611 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:16.608+0000 7fdc8b600700 1 -- 172.21.15.153:0/1561063234 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdc84100da0 msgr2=0x7fdc84101180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:19:16.611 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:16.608+0000 7fdc8b600700 1 --2- 172.21.15.153:0/1561063234 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdc84100da0 0x7fdc84101180 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fdc7800b0a0 tx=0x7fdc7800b3b0 comp rx=0 tx=0).stop 2024-03-21T10:19:16.611 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:16.608+0000 7fdc8b600700 1 -- 172.21.15.153:0/1561063234 shutdown_connections 2024-03-21T10:19:16.611 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:16.608+0000 7fdc8b600700 1 --2- 172.21.15.153:0/1561063234 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdc84101750 0x7fdc841057a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:19:16.612 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:16.608+0000 7fdc8b600700 1 --2- 172.21.15.153:0/1561063234 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdc84100da0 0x7fdc84101180 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:19:16.612 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:16.608+0000 7fdc8b600700 1 -- 172.21.15.153:0/1561063234 >> 172.21.15.153:0/1561063234 conn(0x7fdc840fc630 msgr2=0x7fdc840fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:19:16.612 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:16.608+0000 7fdc8b600700 1 -- 172.21.15.153:0/1561063234 shutdown_connections 2024-03-21T10:19:16.612 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:16.608+0000 7fdc8b600700 1 -- 172.21.15.153:0/1561063234 wait complete. 2024-03-21T10:19:16.612 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:16.609+0000 7fdc8b600700 1 Processor -- start 2024-03-21T10:19:16.612 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:16.609+0000 7fdc8b600700 1 -- start start 2024-03-21T10:19:16.612 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:16.610+0000 7fdc8b600700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdc84100da0 0x7fdc8419e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:19:16.612 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:16.610+0000 7fdc8b600700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdc84101750 0x7fdc8419ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:19:16.612 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:16.610+0000 7fdc8b600700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc8419f470 con 0x7fdc84100da0 2024-03-21T10:19:16.612 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:16.610+0000 7fdc8b600700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc84198920 con 0x7fdc84101750 2024-03-21T10:19:16.613 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:16.610+0000 7fdc88c00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdc84100da0 0x7fdc8419e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:19:16.614 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:16.610+0000 7fdc88c00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdc84100da0 0x7fdc8419e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:47166/0 (socket says 172.21.15.153:47166) 2024-03-21T10:19:16.614 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:16.610+0000 7fdc88c00700 1 -- 172.21.15.153:0/3462555647 learned_addr learned my addr 172.21.15.153:0/3462555647 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:19:16.614 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:16.610+0000 7fdc83e00700 1 --2- 172.21.15.153:0/3462555647 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdc84101750 0x7fdc8419ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:19:16.614 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:16.610+0000 7fdc88c00700 1 -- 172.21.15.153:0/3462555647 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdc84101750 msgr2=0x7fdc8419ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:19:16.614 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:16.610+0000 7fdc88c00700 1 --2- 172.21.15.153:0/3462555647 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdc84101750 0x7fdc8419ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:19:16.614 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:16.611+0000 7fdc88c00700 1 -- 172.21.15.153:0/3462555647 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc78009d00 con 0x7fdc84100da0 2024-03-21T10:19:16.614 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:16.611+0000 7fdc83e00700 1 --2- 172.21.15.153:0/3462555647 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdc84101750 0x7fdc8419ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:19:16.614 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:16.611+0000 7fdc88c00700 1 --2- 172.21.15.153:0/3462555647 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdc84100da0 0x7fdc8419e8a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fdc7800b070 tx=0x7fdc7800cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:19:16.614 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:16.611+0000 7fdc81600700 1 -- 172.21.15.153:0/3462555647 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdc7800e040 con 0x7fdc84100da0 2024-03-21T10:19:16.614 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:16.611+0000 7fdc81600700 1 -- 172.21.15.153:0/3462555647 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdc78003e60 con 0x7fdc84100da0 2024-03-21T10:19:16.614 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:16.611+0000 7fdc8b600700 1 -- 172.21.15.153:0/3462555647 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc84198ba0 con 0x7fdc84100da0 2024-03-21T10:19:16.614 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:16.611+0000 7fdc81600700 1 -- 172.21.15.153:0/3462555647 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdc7801d910 con 0x7fdc84100da0 2024-03-21T10:19:16.615 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:16.611+0000 7fdc8b600700 1 -- 172.21.15.153:0/3462555647 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc84199090 con 0x7fdc84100da0 2024-03-21T10:19:16.615 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:16.612+0000 7fdc8b600700 1 -- 172.21.15.153:0/3462555647 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc84104f20 con 0x7fdc84100da0 2024-03-21T10:19:16.615 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:16.613+0000 7fdc81600700 1 -- 172.21.15.153:0/3462555647 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fdc78004820 con 0x7fdc84100da0 2024-03-21T10:19:16.616 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:16.614+0000 7fdc81600700 1 --2- 172.21.15.153:0/3462555647 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fdc6c06e460 0x7fdc6c070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:19:16.616 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:16.614+0000 7fdc81600700 1 -- 172.21.15.153:0/3462555647 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fdc7808ff10 con 0x7fdc84100da0 2024-03-21T10:19:16.616 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:16.615+0000 7fdc83e00700 1 --2- 172.21.15.153:0/3462555647 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fdc6c06e460 0x7fdc6c070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:19:16.617 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:16.615+0000 7fdc83e00700 1 --2- 172.21.15.153:0/3462555647 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fdc6c06e460 0x7fdc6c070920 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fdc840fce10 tx=0x7fdc70008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:19:16.621 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:16.619+0000 7fdc81600700 1 -- 172.21.15.153:0/3462555647 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fdc7805ca60 con 0x7fdc84100da0 2024-03-21T10:19:16.766 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:16 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:19:16.767 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:16 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:19:16.767 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:16 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:19:16.767 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:16 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:19:16.767 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:16 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:19:16.767 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:16 smithi153 ceph-mon[113733]: mds.? [v2:172.21.15.153:6826/1103846295,v1:172.21.15.153:6827/1103846295] up:boot 2024-03-21T10:19:16.767 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:16 smithi153 ceph-mon[113733]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-03-21T10:19:16.767 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:16 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi153.zipckr"}]: dispatch 2024-03-21T10:19:16.867 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:16.865+0000 7fdc8b600700 1 -- 172.21.15.153:0/3462555647 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdc841992b0 con 0x7fdc6c06e460 2024-03-21T10:19:16.869 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:16.867+0000 7fdc81600700 1 -- 172.21.15.153:0/3462555647 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fdc841992b0 con 0x7fdc6c06e460 2024-03-21T10:19:16.872 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:16.870+0000 7fdc8b600700 1 -- 172.21.15.153:0/3462555647 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fdc6c06e460 msgr2=0x7fdc6c070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:19:16.872 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:16.870+0000 7fdc8b600700 1 --2- 172.21.15.153:0/3462555647 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fdc6c06e460 0x7fdc6c070920 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fdc840fce10 tx=0x7fdc70008040 comp rx=0 tx=0).stop 2024-03-21T10:19:16.872 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:16.870+0000 7fdc8b600700 1 -- 172.21.15.153:0/3462555647 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdc84100da0 msgr2=0x7fdc8419e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:19:16.872 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:16.870+0000 7fdc8b600700 1 --2- 172.21.15.153:0/3462555647 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdc84100da0 0x7fdc8419e8a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fdc7800b070 tx=0x7fdc7800cdd0 comp rx=0 tx=0).stop 2024-03-21T10:19:16.872 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:16.870+0000 7fdc8b600700 1 -- 172.21.15.153:0/3462555647 shutdown_connections 2024-03-21T10:19:16.873 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:16.870+0000 7fdc8b600700 1 --2- 172.21.15.153:0/3462555647 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fdc6c06e460 0x7fdc6c070920 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:19:16.873 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:16.870+0000 7fdc8b600700 1 --2- 172.21.15.153:0/3462555647 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdc84100da0 0x7fdc8419e8a0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:19:16.873 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:16.870+0000 7fdc8b600700 1 --2- 172.21.15.153:0/3462555647 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdc84101750 0x7fdc8419ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:19:16.873 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:16.870+0000 7fdc8b600700 1 -- 172.21.15.153:0/3462555647 >> 172.21.15.153:0/3462555647 conn(0x7fdc840fc630 msgr2=0x7fdc840fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:19:16.873 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:16.870+0000 7fdc8b600700 1 -- 172.21.15.153:0/3462555647 shutdown_connections 2024-03-21T10:19:16.873 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:16.870+0000 7fdc8b600700 1 -- 172.21.15.153:0/3462555647 wait complete. 2024-03-21T10:19:16.888 INFO:teuthology.orchestra.run.smithi153.stdout:true 2024-03-21T10:19:16.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:16 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:19:16.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:16 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:19:16.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:16 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:19:16.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:16 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:19:16.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:16 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:19:16.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:16 smithi174 ceph-mon[88004]: mds.? [v2:172.21.15.153:6826/1103846295,v1:172.21.15.153:6827/1103846295] up:boot 2024-03-21T10:19:16.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:16 smithi174 ceph-mon[88004]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-03-21T10:19:16.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:16 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi153.zipckr"}]: dispatch 2024-03-21T10:19:17.035 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.030+0000 7f1d45000700 1 -- 172.21.15.153:0/3252389382 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1d40100da0 msgr2=0x7f1d40101180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:19:17.035 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.030+0000 7f1d45000700 1 --2- 172.21.15.153:0/3252389382 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1d40100da0 0x7f1d40101180 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1d3000b0a0 tx=0x7f1d3000b3b0 comp rx=0 tx=0).stop 2024-03-21T10:19:17.035 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.030+0000 7f1d45000700 1 -- 172.21.15.153:0/3252389382 shutdown_connections 2024-03-21T10:19:17.035 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.030+0000 7f1d45000700 1 --2- 172.21.15.153:0/3252389382 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1d40101750 0x7f1d401057a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:19:17.035 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.030+0000 7f1d45000700 1 --2- 172.21.15.153:0/3252389382 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1d40100da0 0x7f1d40101180 secure :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1d3000b0a0 tx=0x7f1d3000b3b0 comp rx=0 tx=0).stop 2024-03-21T10:19:17.035 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.030+0000 7f1d45000700 1 -- 172.21.15.153:0/3252389382 >> 172.21.15.153:0/3252389382 conn(0x7f1d400fc630 msgr2=0x7f1d400fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:19:17.036 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.033+0000 7f1d45000700 1 -- 172.21.15.153:0/3252389382 shutdown_connections 2024-03-21T10:19:17.036 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.033+0000 7f1d45000700 1 -- 172.21.15.153:0/3252389382 wait complete. 2024-03-21T10:19:17.037 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.034+0000 7f1d45000700 1 Processor -- start 2024-03-21T10:19:17.037 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.034+0000 7f1d45000700 1 -- start start 2024-03-21T10:19:17.037 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.035+0000 7f1d45000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1d40101750 0x7f1d4006dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:19:17.037 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.035+0000 7f1d45000700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1d4006e150 0x7f1d4006e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:19:17.037 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.035+0000 7f1d45000700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d40072c10 con 0x7f1d40101750 2024-03-21T10:19:17.037 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.035+0000 7f1d45000700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d40072d80 con 0x7f1d4006e150 2024-03-21T10:19:17.038 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.036+0000 7f1d3de00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1d40101750 0x7f1d4006dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:19:17.038 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.036+0000 7f1d3de00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1d40101750 0x7f1d4006dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:47188/0 (socket says 172.21.15.153:47188) 2024-03-21T10:19:17.038 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.036+0000 7f1d3de00700 1 -- 172.21.15.153:0/764897329 learned_addr learned my addr 172.21.15.153:0/764897329 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:19:17.038 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.036+0000 7f1d35400700 1 --2- 172.21.15.153:0/764897329 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1d4006e150 0x7f1d4006e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:19:17.038 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.036+0000 7f1d3de00700 1 -- 172.21.15.153:0/764897329 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1d4006e150 msgr2=0x7f1d4006e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:19:17.038 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.036+0000 7f1d3de00700 1 --2- 172.21.15.153:0/764897329 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1d4006e150 0x7f1d4006e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:19:17.039 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.036+0000 7f1d3de00700 1 -- 172.21.15.153:0/764897329 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d30009d00 con 0x7f1d40101750 2024-03-21T10:19:17.039 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.037+0000 7f1d3de00700 1 --2- 172.21.15.153:0/764897329 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1d40101750 0x7f1d4006dc10 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1d30009fd0 tx=0x7f1d3000bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:19:17.039 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.037+0000 7f1d37400700 1 -- 172.21.15.153:0/764897329 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1d3000e040 con 0x7f1d40101750 2024-03-21T10:19:17.039 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.037+0000 7f1d37400700 1 -- 172.21.15.153:0/764897329 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1d30003de0 con 0x7f1d40101750 2024-03-21T10:19:17.039 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.037+0000 7f1d45000700 1 -- 172.21.15.153:0/764897329 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d401a9410 con 0x7f1d40101750 2024-03-21T10:19:17.040 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.037+0000 7f1d37400700 1 -- 172.21.15.153:0/764897329 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1d3001d860 con 0x7f1d40101750 2024-03-21T10:19:17.040 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.037+0000 7f1d45000700 1 -- 172.21.15.153:0/764897329 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d401a98b0 con 0x7f1d40101750 2024-03-21T10:19:17.040 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.039+0000 7f1d45000700 1 -- 172.21.15.153:0/764897329 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d40109140 con 0x7f1d40101750 2024-03-21T10:19:17.041 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.039+0000 7f1d37400700 1 -- 172.21.15.153:0/764897329 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f1d30019070 con 0x7f1d40101750 2024-03-21T10:19:17.042 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.041+0000 7f1d37400700 1 --2- 172.21.15.153:0/764897329 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f1d2406e460 0x7f1d24070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:19:17.043 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.041+0000 7f1d37400700 1 -- 172.21.15.153:0/764897329 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f1d3008f8e0 con 0x7f1d40101750 2024-03-21T10:19:17.043 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.041+0000 7f1d35400700 1 --2- 172.21.15.153:0/764897329 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f1d2406e460 0x7f1d24070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:19:17.044 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.042+0000 7f1d35400700 1 --2- 172.21.15.153:0/764897329 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f1d2406e460 0x7f1d24070920 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1d28005ea0 tx=0x7f1d28005e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:19:17.048 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.046+0000 7f1d37400700 1 -- 172.21.15.153:0/764897329 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f1d30058a20 con 0x7f1d40101750 2024-03-21T10:19:17.301 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.299+0000 7f1d45000700 1 -- 172.21.15.153:0/764897329 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1d400611d0 con 0x7f1d2406e460 2024-03-21T10:19:17.302 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.301+0000 7f1d37400700 1 -- 172.21.15.153:0/764897329 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f1d400611d0 con 0x7f1d2406e460 2024-03-21T10:19:17.306 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.304+0000 7f1d45000700 1 -- 172.21.15.153:0/764897329 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f1d2406e460 msgr2=0x7f1d24070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:19:17.306 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.304+0000 7f1d45000700 1 --2- 172.21.15.153:0/764897329 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f1d2406e460 0x7f1d24070920 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1d28005ea0 tx=0x7f1d28005e30 comp rx=0 tx=0).stop 2024-03-21T10:19:17.306 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.304+0000 7f1d45000700 1 -- 172.21.15.153:0/764897329 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1d40101750 msgr2=0x7f1d4006dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:19:17.306 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.304+0000 7f1d45000700 1 --2- 172.21.15.153:0/764897329 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1d40101750 0x7f1d4006dc10 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1d30009fd0 tx=0x7f1d3000bdc0 comp rx=0 tx=0).stop 2024-03-21T10:19:17.306 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.304+0000 7f1d45000700 1 -- 172.21.15.153:0/764897329 shutdown_connections 2024-03-21T10:19:17.306 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.304+0000 7f1d45000700 1 --2- 172.21.15.153:0/764897329 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f1d2406e460 0x7f1d24070920 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:19:17.306 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.304+0000 7f1d45000700 1 --2- 172.21.15.153:0/764897329 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1d40101750 0x7f1d4006dc10 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:19:17.307 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.304+0000 7f1d45000700 1 --2- 172.21.15.153:0/764897329 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1d4006e150 0x7f1d4006e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:19:17.307 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.304+0000 7f1d45000700 1 -- 172.21.15.153:0/764897329 >> 172.21.15.153:0/764897329 conn(0x7f1d400fc630 msgr2=0x7f1d400fddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:19:17.307 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.304+0000 7f1d45000700 1 -- 172.21.15.153:0/764897329 shutdown_connections 2024-03-21T10:19:17.307 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.304+0000 7f1d45000700 1 -- 172.21.15.153:0/764897329 wait complete. 2024-03-21T10:19:17.465 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.462+0000 7f2d79c00700 1 -- 172.21.15.153:0/4162375786 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2d74100da0 msgr2=0x7f2d74101180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:19:17.465 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.462+0000 7f2d79c00700 1 --2- 172.21.15.153:0/4162375786 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2d74100da0 0x7f2d74101180 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2d64009910 tx=0x7f2d64009c20 comp rx=0 tx=0).stop 2024-03-21T10:19:17.465 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.463+0000 7f2d79c00700 1 -- 172.21.15.153:0/4162375786 shutdown_connections 2024-03-21T10:19:17.465 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.463+0000 7f2d79c00700 1 --2- 172.21.15.153:0/4162375786 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d74101750 0x7f2d741057a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:19:17.465 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.463+0000 7f2d79c00700 1 --2- 172.21.15.153:0/4162375786 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2d74100da0 0x7f2d74101180 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:19:17.465 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.463+0000 7f2d79c00700 1 -- 172.21.15.153:0/4162375786 >> 172.21.15.153:0/4162375786 conn(0x7f2d740fc630 msgr2=0x7f2d740fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:19:17.465 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.463+0000 7f2d79c00700 1 -- 172.21.15.153:0/4162375786 shutdown_connections 2024-03-21T10:19:17.465 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.463+0000 7f2d79c00700 1 -- 172.21.15.153:0/4162375786 wait complete. 2024-03-21T10:19:17.467 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.464+0000 7f2d79c00700 1 Processor -- start 2024-03-21T10:19:17.467 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.464+0000 7f2d79c00700 1 -- start start 2024-03-21T10:19:17.467 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.464+0000 7f2d79c00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d74100da0 0x7f2d7419e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:19:17.467 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.464+0000 7f2d79c00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2d74101750 0x7f2d7419ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:19:17.468 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.464+0000 7f2d79c00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d7419f420 con 0x7f2d74101750 2024-03-21T10:19:17.468 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.464+0000 7f2d79c00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d741988d0 con 0x7f2d74100da0 2024-03-21T10:19:17.468 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.465+0000 7f2d72000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2d74101750 0x7f2d7419ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:19:17.468 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.465+0000 7f2d72000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2d74101750 0x7f2d7419ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:47210/0 (socket says 172.21.15.153:47210) 2024-03-21T10:19:17.468 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.465+0000 7f2d72000700 1 -- 172.21.15.153:0/3887097464 learned_addr learned my addr 172.21.15.153:0/3887097464 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:19:17.468 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.465+0000 7f2d72000700 1 -- 172.21.15.153:0/3887097464 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d74100da0 msgr2=0x7f2d7419e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:19:17.468 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.465+0000 7f2d72000700 1 --2- 172.21.15.153:0/3887097464 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d74100da0 0x7f2d7419e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:19:17.468 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.465+0000 7f2d72000700 1 -- 172.21.15.153:0/3887097464 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d640095f0 con 0x7f2d74101750 2024-03-21T10:19:17.468 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.465+0000 7f2d72000700 1 --2- 172.21.15.153:0/3887097464 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2d74101750 0x7f2d7419ed90 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2d6000b480 tx=0x7f2d6000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:19:17.468 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.466+0000 7f2d6b400700 1 -- 172.21.15.153:0/3887097464 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2d600118c0 con 0x7f2d74101750 2024-03-21T10:19:17.468 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.466+0000 7f2d6b400700 1 -- 172.21.15.153:0/3887097464 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2d60010380 con 0x7f2d74101750 2024-03-21T10:19:17.468 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.466+0000 7f2d79c00700 1 -- 172.21.15.153:0/3887097464 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d74198bb0 con 0x7f2d74101750 2024-03-21T10:19:17.468 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.466+0000 7f2d6b400700 1 -- 172.21.15.153:0/3887097464 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2d6000f4f0 con 0x7f2d74101750 2024-03-21T10:19:17.469 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.466+0000 7f2d79c00700 1 -- 172.21.15.153:0/3887097464 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d74199100 con 0x7f2d74101750 2024-03-21T10:19:17.469 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.467+0000 7f2d79c00700 1 -- 172.21.15.153:0/3887097464 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d74104f20 con 0x7f2d74101750 2024-03-21T10:19:17.470 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.468+0000 7f2d6b400700 1 -- 172.21.15.153:0/3887097464 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f2d600104f0 con 0x7f2d74101750 2024-03-21T10:19:17.471 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.469+0000 7f2d6b400700 1 --2- 172.21.15.153:0/3887097464 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f2d5806e460 0x7f2d58070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:19:17.471 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.469+0000 7f2d6b400700 1 -- 172.21.15.153:0/3887097464 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f2d6008e120 con 0x7f2d74101750 2024-03-21T10:19:17.472 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.470+0000 7f2d72a00700 1 --2- 172.21.15.153:0/3887097464 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f2d5806e460 0x7f2d58070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:19:17.472 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.471+0000 7f2d72a00700 1 --2- 172.21.15.153:0/3887097464 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f2d5806e460 0x7f2d58070920 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2d6400b5c0 tx=0x7f2d640058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:19:17.478 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.476+0000 7f2d6b400700 1 -- 172.21.15.153:0/3887097464 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2d6005acf0 con 0x7f2d74101750 2024-03-21T10:19:17.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:17 smithi153 ceph-mon[113733]: from='client.34446 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:19:17.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:17 smithi153 ceph-mon[113733]: pgmap v221: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:19:17.713 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.711+0000 7f2d79c00700 1 -- 172.21.15.153:0/3887097464 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f2d74199490 con 0x7f2d5806e460 2024-03-21T10:19:17.743 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.742+0000 7f2d6b400700 1 -- 172.21.15.153:0/3887097464 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f2d74199490 con 0x7f2d5806e460 2024-03-21T10:19:17.744 INFO:teuthology.orchestra.run.smithi153.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-21T10:19:17.744 INFO:teuthology.orchestra.run.smithi153.stdout:alertmanager.smithi153 smithi153 *:9093,9094 running (8m) 2m ago 21m 14.5M - 0.25.0 c8568f914cd2 9ed3f2200b5f 2024-03-21T10:19:17.744 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi153 smithi153 running (22m) 2m ago 22m 8988k - 18.2.1 5be31c24972a 5b61994396a6 2024-03-21T10:19:17.744 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi174 smithi174 running (20m) 23s ago 20m 9026k - 18.2.1 5be31c24972a 4f7a5fc3ac28 2024-03-21T10:19:17.744 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi153 smithi153 running (4m) 2m ago 22m 6656k - 19.0.0-2231-g2edfee68 237e43c7a65f 32c21104d5b0 2024-03-21T10:19:17.744 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi174 smithi174 running (4m) 23s ago 20m 6660k - 19.0.0-2231-g2edfee68 237e43c7a65f a04e48967ab6 2024-03-21T10:19:17.744 INFO:teuthology.orchestra.run.smithi153.stdout:grafana.smithi153 smithi153 *:3000 running (7m) 2m ago 21m 75.4M - 9.4.12 2bacad6d85d8 ee24026ab786 2024-03-21T10:19:17.745 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.iuyluj smithi153 running (14m) 2m ago 14m 128M - 18.2.1 5be31c24972a 02f667cee979 2024-03-21T10:19:17.745 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.zipckr smithi153 starting - - - - 2024-03-21T10:19:17.745 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.nhozag smithi174 running (14m) 23s ago 14m 57.5M - 18.2.1 5be31c24972a 2fee4000308f 2024-03-21T10:19:17.745 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.yyimst smithi174 running (14m) 23s ago 14m 117M - 18.2.1 5be31c24972a d78a729220d1 2024-03-21T10:19:17.745 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi153.tluxni smithi153 *:8443,9283,8765 running (11m) 2m ago 23m 505M - 19.0.0-2231-g2edfee68 237e43c7a65f 7ffee0e55382 2024-03-21T10:19:17.745 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi174.qpcwat smithi174 *:8443,9283,8765 running (9m) 23s ago 20m 452M - 19.0.0-2231-g2edfee68 237e43c7a65f 748968f216ab 2024-03-21T10:19:17.745 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi153 smithi153 running (5m) 2m ago 23m 48.2M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f dd8038303dc3 2024-03-21T10:19:17.745 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi174 smithi174 running (5m) 23s ago 20m 40.9M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f 29d78a788f2c 2024-03-21T10:19:17.745 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi153 smithi153 *:9100 running (9m) 2m ago 22m 8284k - 1.5.0 0da6a335fe13 bb835edc0e0b 2024-03-21T10:19:17.745 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi174 smithi174 *:9100 running (9m) 23s ago 20m 8200k - 1.5.0 0da6a335fe13 7c4ecad790fa 2024-03-21T10:19:17.745 INFO:teuthology.orchestra.run.smithi153.stdout:osd.0 smithi153 running (3m) 2m ago 19m 119M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 507900fd4791 2024-03-21T10:19:17.745 INFO:teuthology.orchestra.run.smithi153.stdout:osd.1 smithi153 running (2m) 2m ago 18m 86.6M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 0d646452e9c1 2024-03-21T10:19:17.745 INFO:teuthology.orchestra.run.smithi153.stdout:osd.2 smithi153 running (2m) 2m ago 18m 12.6M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 1326f2afdfcd 2024-03-21T10:19:17.745 INFO:teuthology.orchestra.run.smithi153.stdout:osd.3 smithi174 running (112s) 23s ago 17m 82.8M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 70aafac71ad5 2024-03-21T10:19:17.746 INFO:teuthology.orchestra.run.smithi153.stdout:osd.4 smithi174 running (70s) 23s ago 17m 91.0M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 0d746ce6e79d 2024-03-21T10:19:17.746 INFO:teuthology.orchestra.run.smithi153.stdout:osd.5 smithi174 running (28s) 23s ago 16m 11.2M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 53e5f35b8cac 2024-03-21T10:19:17.746 INFO:teuthology.orchestra.run.smithi153.stdout:prometheus.smithi153 smithi153 *:9095 running (8m) 2m ago 21m 72.2M - 2.43.0 a07b618ecd1d 7d95561e50c2 2024-03-21T10:19:17.747 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.745+0000 7f2d79c00700 1 -- 172.21.15.153:0/3887097464 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f2d5806e460 msgr2=0x7f2d58070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:19:17.747 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.745+0000 7f2d79c00700 1 --2- 172.21.15.153:0/3887097464 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f2d5806e460 0x7f2d58070920 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2d6400b5c0 tx=0x7f2d640058e0 comp rx=0 tx=0).stop 2024-03-21T10:19:17.747 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.745+0000 7f2d79c00700 1 -- 172.21.15.153:0/3887097464 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2d74101750 msgr2=0x7f2d7419ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:19:17.747 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.745+0000 7f2d79c00700 1 --2- 172.21.15.153:0/3887097464 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2d74101750 0x7f2d7419ed90 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2d6000b480 tx=0x7f2d6000b840 comp rx=0 tx=0).stop 2024-03-21T10:19:17.747 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.745+0000 7f2d79c00700 1 -- 172.21.15.153:0/3887097464 shutdown_connections 2024-03-21T10:19:17.747 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.745+0000 7f2d79c00700 1 --2- 172.21.15.153:0/3887097464 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d74100da0 0x7f2d7419e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:19:17.747 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.745+0000 7f2d79c00700 1 --2- 172.21.15.153:0/3887097464 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f2d5806e460 0x7f2d58070920 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:19:17.748 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.745+0000 7f2d79c00700 1 --2- 172.21.15.153:0/3887097464 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2d74101750 0x7f2d7419ed90 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:19:17.748 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.745+0000 7f2d79c00700 1 -- 172.21.15.153:0/3887097464 >> 172.21.15.153:0/3887097464 conn(0x7f2d740fc630 msgr2=0x7f2d740fdd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:19:17.748 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.745+0000 7f2d79c00700 1 -- 172.21.15.153:0/3887097464 shutdown_connections 2024-03-21T10:19:17.748 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.746+0000 7f2d79c00700 1 -- 172.21.15.153:0/3887097464 wait complete. 2024-03-21T10:19:17.902 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.899+0000 7f71eda00700 1 -- 172.21.15.153:0/3443650290 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f71e8101750 msgr2=0x7f71e81057a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:19:17.902 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.899+0000 7f71eda00700 1 --2- 172.21.15.153:0/3443650290 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f71e8101750 0x7f71e81057a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f71d4009910 tx=0x7f71d4009c20 comp rx=0 tx=0).stop 2024-03-21T10:19:17.902 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.899+0000 7f71eda00700 1 -- 172.21.15.153:0/3443650290 shutdown_connections 2024-03-21T10:19:17.902 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.899+0000 7f71eda00700 1 --2- 172.21.15.153:0/3443650290 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f71e8101750 0x7f71e81057a0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:19:17.902 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.899+0000 7f71eda00700 1 --2- 172.21.15.153:0/3443650290 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f71e8100da0 0x7f71e8101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:19:17.902 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.899+0000 7f71eda00700 1 -- 172.21.15.153:0/3443650290 >> 172.21.15.153:0/3443650290 conn(0x7f71e80fc630 msgr2=0x7f71e80fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:19:17.902 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.900+0000 7f71eda00700 1 -- 172.21.15.153:0/3443650290 shutdown_connections 2024-03-21T10:19:17.902 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.900+0000 7f71eda00700 1 -- 172.21.15.153:0/3443650290 wait complete. 2024-03-21T10:19:17.903 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.901+0000 7f71eda00700 1 Processor -- start 2024-03-21T10:19:17.903 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.901+0000 7f71eda00700 1 -- start start 2024-03-21T10:19:17.903 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.901+0000 7f71eda00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f71e8100da0 0x7f71e819e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:19:17.904 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.901+0000 7f71eda00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f71e8101750 0x7f71e819ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:19:17.904 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.901+0000 7f71eda00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71e819f240 con 0x7f71e8101750 2024-03-21T10:19:17.904 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.901+0000 7f71eda00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71e8198780 con 0x7f71e8100da0 2024-03-21T10:19:17.904 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.902+0000 7f71e6000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f71e8101750 0x7f71e819ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:19:17.904 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.902+0000 7f71e6000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f71e8101750 0x7f71e819ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:47228/0 (socket says 172.21.15.153:47228) 2024-03-21T10:19:17.904 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.902+0000 7f71e6000700 1 -- 172.21.15.153:0/1365055814 learned_addr learned my addr 172.21.15.153:0/1365055814 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:19:17.904 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.902+0000 7f71e6a00700 1 --2- 172.21.15.153:0/1365055814 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f71e8100da0 0x7f71e819e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:19:17.905 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.902+0000 7f71e6000700 1 -- 172.21.15.153:0/1365055814 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f71e8100da0 msgr2=0x7f71e819e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:19:17.905 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.902+0000 7f71e6000700 1 --2- 172.21.15.153:0/1365055814 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f71e8100da0 0x7f71e819e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:19:17.905 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.902+0000 7f71e6000700 1 -- 172.21.15.153:0/1365055814 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71d40095f0 con 0x7f71e8101750 2024-03-21T10:19:17.905 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.902+0000 7f71e6000700 1 --2- 172.21.15.153:0/1365055814 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f71e8101750 0x7f71e819ec40 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f71d400bda0 tx=0x7f71d400be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:19:17.905 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.903+0000 7f71df400700 1 -- 172.21.15.153:0/1365055814 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f71d401d070 con 0x7f71e8101750 2024-03-21T10:19:17.905 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.903+0000 7f71df400700 1 -- 172.21.15.153:0/1365055814 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f71d4004fb0 con 0x7f71e8101750 2024-03-21T10:19:17.905 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.903+0000 7f71eda00700 1 -- 172.21.15.153:0/1365055814 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71e8198a00 con 0x7f71e8101750 2024-03-21T10:19:17.905 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.903+0000 7f71df400700 1 -- 172.21.15.153:0/1365055814 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f71d400f460 con 0x7f71e8101750 2024-03-21T10:19:17.905 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.903+0000 7f71eda00700 1 -- 172.21.15.153:0/1365055814 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71e8198ef0 con 0x7f71e8101750 2024-03-21T10:19:17.906 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.904+0000 7f71eda00700 1 -- 172.21.15.153:0/1365055814 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71e8104f20 con 0x7f71e8101750 2024-03-21T10:19:17.907 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.905+0000 7f71df400700 1 -- 172.21.15.153:0/1365055814 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f71d4022800 con 0x7f71e8101750 2024-03-21T10:19:17.908 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.906+0000 7f71df400700 1 --2- 172.21.15.153:0/1365055814 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f71cc06e460 0x7f71cc070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:19:17.908 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.906+0000 7f71df400700 1 -- 172.21.15.153:0/1365055814 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f71d408fa20 con 0x7f71e8101750 2024-03-21T10:19:17.908 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.906+0000 7f71e6a00700 1 --2- 172.21.15.153:0/1365055814 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f71cc06e460 0x7f71cc070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:19:17.909 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.907+0000 7f71e6a00700 1 --2- 172.21.15.153:0/1365055814 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f71cc06e460 0x7f71cc070920 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f71d8005ea0 tx=0x7f71d8005e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:19:17.913 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:17.911+0000 7f71df400700 1 -- 172.21.15.153:0/1365055814 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f71d405c7c0 con 0x7f71e8101750 2024-03-21T10:19:17.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:17 smithi174 ceph-mon[88004]: from='client.34446 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:19:17.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:17 smithi174 ceph-mon[88004]: pgmap v221: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:19:18.231 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:18.229+0000 7f71eda00700 1 -- 172.21.15.153:0/1365055814 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f71e8199d60 con 0x7f71e8101750 2024-03-21T10:19:18.232 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:18.230+0000 7f71df400700 1 -- 172.21.15.153:0/1365055814 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+805 (secure 0 0 0) 0x7f71d405c350 con 0x7f71e8101750 2024-03-21T10:19:18.232 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:19:18.233 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": { 2024-03-21T10:19:18.233 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:19:18.233 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:19:18.233 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": { 2024-03-21T10:19:18.233 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:19:18.233 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:19:18.233 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": { 2024-03-21T10:19:18.233 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 6 2024-03-21T10:19:18.233 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:19:18.233 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": { 2024-03-21T10:19:18.233 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 3, 2024-03-21T10:19:18.233 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 1 2024-03-21T10:19:18.233 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:19:18.233 INFO:teuthology.orchestra.run.smithi153.stdout: "overall": { 2024-03-21T10:19:18.234 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 3, 2024-03-21T10:19:18.234 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 11 2024-03-21T10:19:18.234 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-03-21T10:19:18.234 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:19:18.235 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:18.233+0000 7f71eda00700 1 -- 172.21.15.153:0/1365055814 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f71cc06e460 msgr2=0x7f71cc070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:19:18.235 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:18.233+0000 7f71eda00700 1 --2- 172.21.15.153:0/1365055814 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f71cc06e460 0x7f71cc070920 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f71d8005ea0 tx=0x7f71d8005e30 comp rx=0 tx=0).stop 2024-03-21T10:19:18.235 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:18.233+0000 7f71eda00700 1 -- 172.21.15.153:0/1365055814 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f71e8101750 msgr2=0x7f71e819ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:19:18.235 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:18.233+0000 7f71eda00700 1 --2- 172.21.15.153:0/1365055814 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f71e8101750 0x7f71e819ec40 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f71d400bda0 tx=0x7f71d400be80 comp rx=0 tx=0).stop 2024-03-21T10:19:18.235 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:18.234+0000 7f71eda00700 1 -- 172.21.15.153:0/1365055814 shutdown_connections 2024-03-21T10:19:18.235 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:18.234+0000 7f71eda00700 1 --2- 172.21.15.153:0/1365055814 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f71e8100da0 0x7f71e819e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:19:18.235 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:18.234+0000 7f71eda00700 1 --2- 172.21.15.153:0/1365055814 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f71cc06e460 0x7f71cc070920 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:19:18.236 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:18.234+0000 7f71eda00700 1 --2- 172.21.15.153:0/1365055814 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f71e8101750 0x7f71e819ec40 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:19:18.236 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:18.234+0000 7f71eda00700 1 -- 172.21.15.153:0/1365055814 >> 172.21.15.153:0/1365055814 conn(0x7f71e80fc630 msgr2=0x7f71e80fea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:19:18.236 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:18.234+0000 7f71eda00700 1 -- 172.21.15.153:0/1365055814 shutdown_connections 2024-03-21T10:19:18.236 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:18.234+0000 7f71eda00700 1 -- 172.21.15.153:0/1365055814 wait complete. 2024-03-21T10:19:18.395 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:18.393+0000 7f74c7800700 1 -- 172.21.15.153:0/4222116168 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f74c0100da0 msgr2=0x7f74c0101180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:19:18.395 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:18.393+0000 7f74c7800700 1 --2- 172.21.15.153:0/4222116168 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f74c0100da0 0x7f74c0101180 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f74b4009910 tx=0x7f74b4009c20 comp rx=0 tx=0).stop 2024-03-21T10:19:18.395 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:18.393+0000 7f74c7800700 1 -- 172.21.15.153:0/4222116168 shutdown_connections 2024-03-21T10:19:18.395 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:18.393+0000 7f74c7800700 1 --2- 172.21.15.153:0/4222116168 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f74c0101750 0x7f74c01057a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:19:18.395 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:18.393+0000 7f74c7800700 1 --2- 172.21.15.153:0/4222116168 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f74c0100da0 0x7f74c0101180 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:19:18.395 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:18.393+0000 7f74c7800700 1 -- 172.21.15.153:0/4222116168 >> 172.21.15.153:0/4222116168 conn(0x7f74c00fc630 msgr2=0x7f74c00fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:19:18.395 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:18.393+0000 7f74c7800700 1 -- 172.21.15.153:0/4222116168 shutdown_connections 2024-03-21T10:19:18.395 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:18.393+0000 7f74c7800700 1 -- 172.21.15.153:0/4222116168 wait complete. 2024-03-21T10:19:18.396 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:18.394+0000 7f74c7800700 1 Processor -- start 2024-03-21T10:19:18.396 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:18.394+0000 7f74c7800700 1 -- start start 2024-03-21T10:19:18.397 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:18.395+0000 7f74c7800700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f74c0100da0 0x7f74c019e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:19:18.397 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:18.395+0000 7f74c7800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f74c0101750 0x7f74c019ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:19:18.397 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:18.395+0000 7f74c7800700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74c019f420 con 0x7f74c0101750 2024-03-21T10:19:18.397 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:18.395+0000 7f74c7800700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74c01988d0 con 0x7f74c0100da0 2024-03-21T10:19:18.397 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:18.395+0000 7f74bfe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f74c0101750 0x7f74c019ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:19:18.397 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:18.395+0000 7f74bfe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f74c0101750 0x7f74c019ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:47258/0 (socket says 172.21.15.153:47258) 2024-03-21T10:19:18.398 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:18.395+0000 7f74bfe00700 1 -- 172.21.15.153:0/1280922992 learned_addr learned my addr 172.21.15.153:0/1280922992 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:19:18.398 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:18.395+0000 7f74c4e00700 1 --2- 172.21.15.153:0/1280922992 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f74c0100da0 0x7f74c019e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:19:18.398 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:18.396+0000 7f74bfe00700 1 -- 172.21.15.153:0/1280922992 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f74c0100da0 msgr2=0x7f74c019e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:19:18.398 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:18.396+0000 7f74bfe00700 1 --2- 172.21.15.153:0/1280922992 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f74c0100da0 0x7f74c019e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:19:18.398 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:18.396+0000 7f74bfe00700 1 -- 172.21.15.153:0/1280922992 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74b40095f0 con 0x7f74c0101750 2024-03-21T10:19:18.399 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:18.396+0000 7f74bfe00700 1 --2- 172.21.15.153:0/1280922992 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f74c0101750 0x7f74c019ed90 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f74ac00b480 tx=0x7f74ac00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:19:18.399 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:18.396+0000 7f74bd600700 1 -- 172.21.15.153:0/1280922992 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f74ac0118c0 con 0x7f74c0101750 2024-03-21T10:19:18.399 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:18.396+0000 7f74bd600700 1 -- 172.21.15.153:0/1280922992 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f74ac010380 con 0x7f74c0101750 2024-03-21T10:19:18.399 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:18.396+0000 7f74bd600700 1 -- 172.21.15.153:0/1280922992 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f74ac00f4f0 con 0x7f74c0101750 2024-03-21T10:19:18.399 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:18.396+0000 7f74c7800700 1 -- 172.21.15.153:0/1280922992 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74c0198bb0 con 0x7f74c0101750 2024-03-21T10:19:18.399 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:18.397+0000 7f74c7800700 1 -- 172.21.15.153:0/1280922992 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74c0199100 con 0x7f74c0101750 2024-03-21T10:19:18.399 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:18.398+0000 7f74c7800700 1 -- 172.21.15.153:0/1280922992 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74c0104f20 con 0x7f74c0101750 2024-03-21T10:19:18.400 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:18.399+0000 7f74bd600700 1 -- 172.21.15.153:0/1280922992 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f74ac0104f0 con 0x7f74c0101750 2024-03-21T10:19:18.402 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:18.400+0000 7f74bd600700 1 --2- 172.21.15.153:0/1280922992 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f74a806e460 0x7f74a8070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:19:18.402 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:18.400+0000 7f74bd600700 1 -- 172.21.15.153:0/1280922992 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f74ac08e080 con 0x7f74c0101750 2024-03-21T10:19:18.402 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:18.400+0000 7f74c4e00700 1 --2- 172.21.15.153:0/1280922992 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f74a806e460 0x7f74a8070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:19:18.403 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:18.401+0000 7f74c4e00700 1 --2- 172.21.15.153:0/1280922992 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f74a806e460 0x7f74a8070920 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f74b4006010 tx=0x7f74b401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:19:18.406 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:18.405+0000 7f74bd600700 1 -- 172.21.15.153:0/1280922992 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f74ac05acf0 con 0x7f74c0101750 2024-03-21T10:19:18.671 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:18.669+0000 7f74c7800700 1 -- 172.21.15.153:0/1280922992 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f74c004ea90 con 0x7f74c0101750 2024-03-21T10:19:18.672 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:18.670+0000 7f74bd600700 1 -- 172.21.15.153:0/1280922992 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 26 v26) v1 ==== 76+0+1872 (secure 0 0 0) 0x7f74ac05a880 con 0x7f74c0101750 2024-03-21T10:19:18.673 INFO:teuthology.orchestra.run.smithi153.stdout:e26 2024-03-21T10:19:18.673 INFO:teuthology.orchestra.run.smithi153.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-21T10:19:18.673 INFO:teuthology.orchestra.run.smithi153.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:19:18.673 INFO:teuthology.orchestra.run.smithi153.stdout:legacy client fscid: 1 2024-03-21T10:19:18.674 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:19:18.674 INFO:teuthology.orchestra.run.smithi153.stdout:Filesystem 'cephfs' (1) 2024-03-21T10:19:18.674 INFO:teuthology.orchestra.run.smithi153.stdout:fs_name cephfs 2024-03-21T10:19:18.674 INFO:teuthology.orchestra.run.smithi153.stdout:epoch 23 2024-03-21T10:19:18.674 INFO:teuthology.orchestra.run.smithi153.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-21T10:19:18.674 INFO:teuthology.orchestra.run.smithi153.stdout:created 2024-03-21T10:04:27.086576+0000 2024-03-21T10:19:18.674 INFO:teuthology.orchestra.run.smithi153.stdout:modified 2024-03-21T10:19:05.022092+0000 2024-03-21T10:19:18.674 INFO:teuthology.orchestra.run.smithi153.stdout:tableserver 0 2024-03-21T10:19:18.674 INFO:teuthology.orchestra.run.smithi153.stdout:root 0 2024-03-21T10:19:18.674 INFO:teuthology.orchestra.run.smithi153.stdout:session_timeout 60 2024-03-21T10:19:18.674 INFO:teuthology.orchestra.run.smithi153.stdout:session_autoclose 300 2024-03-21T10:19:18.674 INFO:teuthology.orchestra.run.smithi153.stdout:max_file_size 1099511627776 2024-03-21T10:19:18.674 INFO:teuthology.orchestra.run.smithi153.stdout:max_xattr_size 65536 2024-03-21T10:19:18.675 INFO:teuthology.orchestra.run.smithi153.stdout:required_client_features {} 2024-03-21T10:19:18.675 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure 0 2024-03-21T10:19:18.675 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure_osd_epoch 94 2024-03-21T10:19:18.675 INFO:teuthology.orchestra.run.smithi153.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:19:18.675 INFO:teuthology.orchestra.run.smithi153.stdout:max_mds 2 2024-03-21T10:19:18.675 INFO:teuthology.orchestra.run.smithi153.stdout:in 0,1 2024-03-21T10:19:18.675 INFO:teuthology.orchestra.run.smithi153.stdout:up {} 2024-03-21T10:19:18.675 INFO:teuthology.orchestra.run.smithi153.stdout:failed 0,1 2024-03-21T10:19:18.675 INFO:teuthology.orchestra.run.smithi153.stdout:damaged 2024-03-21T10:19:18.675 INFO:teuthology.orchestra.run.smithi153.stdout:stopped 2024-03-21T10:19:18.675 INFO:teuthology.orchestra.run.smithi153.stdout:data_pools [3] 2024-03-21T10:19:18.675 INFO:teuthology.orchestra.run.smithi153.stdout:metadata_pool 2 2024-03-21T10:19:18.675 INFO:teuthology.orchestra.run.smithi153.stdout:inline_data enabled 2024-03-21T10:19:18.676 INFO:teuthology.orchestra.run.smithi153.stdout:balancer 2024-03-21T10:19:18.676 INFO:teuthology.orchestra.run.smithi153.stdout:bal_rank_mask -1 2024-03-21T10:19:18.676 INFO:teuthology.orchestra.run.smithi153.stdout:standby_count_wanted 1 2024-03-21T10:19:18.676 INFO:teuthology.orchestra.run.smithi153.stdout:qdb_cluster leader: 0 members: 2024-03-21T10:19:18.676 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:19:18.676 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:19:18.676 INFO:teuthology.orchestra.run.smithi153.stdout:Standby daemons: 2024-03-21T10:19:18.676 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:19:18.676 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.iuyluj{-1:34438} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6828/147840850,v1:172.21.15.153:6829/147840850] compat {c=[1],r=[1],i=[7ff]}] 2024-03-21T10:19:18.676 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.zipckr{-1:34442} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6826/1103846295,v1:172.21.15.153:6827/1103846295] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:19:18.676 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.nhozag{-1:44269} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.174:6826/3967223227,v1:172.21.15.174:6827/3967223227] compat {c=[1],r=[1],i=[7ff]}] 2024-03-21T10:19:18.676 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.yyimst{-1:44277} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.174:6824/787559196,v1:172.21.15.174:6825/787559196] compat {c=[1],r=[1],i=[7ff]}] 2024-03-21T10:19:18.677 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:18.673+0000 7f74c7800700 1 -- 172.21.15.153:0/1280922992 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f74a806e460 msgr2=0x7f74a8070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:19:18.677 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:18.673+0000 7f74c7800700 1 --2- 172.21.15.153:0/1280922992 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f74a806e460 0x7f74a8070920 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f74b4006010 tx=0x7f74b401a040 comp rx=0 tx=0).stop 2024-03-21T10:19:18.677 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:18.673+0000 7f74c7800700 1 -- 172.21.15.153:0/1280922992 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f74c0101750 msgr2=0x7f74c019ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:19:18.678 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:18.673+0000 7f74c7800700 1 --2- 172.21.15.153:0/1280922992 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f74c0101750 0x7f74c019ed90 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f74ac00b480 tx=0x7f74ac00b840 comp rx=0 tx=0).stop 2024-03-21T10:19:18.678 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:18.674+0000 7f74c7800700 1 -- 172.21.15.153:0/1280922992 shutdown_connections 2024-03-21T10:19:18.678 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:18.674+0000 7f74c7800700 1 --2- 172.21.15.153:0/1280922992 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f74c0100da0 0x7f74c019e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:19:18.678 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:18.674+0000 7f74c7800700 1 --2- 172.21.15.153:0/1280922992 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f74a806e460 0x7f74a8070920 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:19:18.678 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:18.674+0000 7f74c7800700 1 --2- 172.21.15.153:0/1280922992 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f74c0101750 0x7f74c019ed90 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:19:18.678 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:18.674+0000 7f74c7800700 1 -- 172.21.15.153:0/1280922992 >> 172.21.15.153:0/1280922992 conn(0x7f74c00fc630 msgr2=0x7f74c00fdd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:19:18.678 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:18.674+0000 7f74c7800700 1 -- 172.21.15.153:0/1280922992 shutdown_connections 2024-03-21T10:19:18.678 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:18.674+0000 7f74c7800700 1 -- 172.21.15.153:0/1280922992 wait complete. 2024-03-21T10:19:18.678 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 26 2024-03-21T10:19:18.774 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:18 smithi153 ceph-mon[113733]: from='client.34450 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:19:18.774 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:18 smithi153 ceph-mon[113733]: from='client.34454 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:19:18.774 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:18 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/1365055814' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:19:18.818 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:18.816+0000 7f4678a00700 1 -- 172.21.15.153:0/210344248 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f46740731c0 msgr2=0x7f46740735a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:19:18.818 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:18.816+0000 7f4678a00700 1 --2- 172.21.15.153:0/210344248 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f46740731c0 0x7f46740735a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f466400b0a0 tx=0x7f466400b3b0 comp rx=0 tx=0).stop 2024-03-21T10:19:18.818 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:18.816+0000 7f4678a00700 1 -- 172.21.15.153:0/210344248 shutdown_connections 2024-03-21T10:19:18.818 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:18.816+0000 7f4678a00700 1 --2- 172.21.15.153:0/210344248 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4674073ae0 0x7f467410ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:19:18.818 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:18.816+0000 7f4678a00700 1 --2- 172.21.15.153:0/210344248 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f46740731c0 0x7f46740735a0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:19:18.818 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:18.816+0000 7f4678a00700 1 -- 172.21.15.153:0/210344248 >> 172.21.15.153:0/210344248 conn(0x7f46740fc630 msgr2=0x7f46740fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:19:18.818 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:18.816+0000 7f4678a00700 1 -- 172.21.15.153:0/210344248 shutdown_connections 2024-03-21T10:19:18.819 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:18.816+0000 7f4678a00700 1 -- 172.21.15.153:0/210344248 wait complete. 2024-03-21T10:19:18.819 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:18.817+0000 7f4678a00700 1 Processor -- start 2024-03-21T10:19:18.819 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:18.818+0000 7f4678a00700 1 -- start start 2024-03-21T10:19:18.820 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:18.818+0000 7f4678a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f46740731c0 0x7f46741989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:19:18.820 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:18.818+0000 7f4678a00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4674073ae0 0x7f4674198f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:19:18.820 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:18.818+0000 7f4678a00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46741995a0 con 0x7f46740731c0 2024-03-21T10:19:18.820 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:18.818+0000 7f4678a00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f467419d360 con 0x7f4674073ae0 2024-03-21T10:19:18.821 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:18.818+0000 7f4671e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f46740731c0 0x7f46741989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:19:18.821 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:18.818+0000 7f4671e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f46740731c0 0x7f46741989d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:47276/0 (socket says 172.21.15.153:47276) 2024-03-21T10:19:18.821 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:18.818+0000 7f4671e00700 1 -- 172.21.15.153:0/1255908196 learned_addr learned my addr 172.21.15.153:0/1255908196 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:19:18.821 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:18.819+0000 7f466be00700 1 --2- 172.21.15.153:0/1255908196 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4674073ae0 0x7f4674198f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:19:18.821 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:18.819+0000 7f4671e00700 1 -- 172.21.15.153:0/1255908196 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4674073ae0 msgr2=0x7f4674198f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:19:18.821 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:18.819+0000 7f4671e00700 1 --2- 172.21.15.153:0/1255908196 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4674073ae0 0x7f4674198f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:19:18.821 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:18.819+0000 7f4671e00700 1 -- 172.21.15.153:0/1255908196 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4664009d00 con 0x7f46740731c0 2024-03-21T10:19:18.822 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:18.819+0000 7f4671e00700 1 --2- 172.21.15.153:0/1255908196 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f46740731c0 0x7f46741989d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f466400b070 tx=0x7f4664003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:19:18.822 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:18.819+0000 7f466aa00700 1 -- 172.21.15.153:0/1255908196 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f466400e040 con 0x7f46740731c0 2024-03-21T10:19:18.822 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:18.819+0000 7f466aa00700 1 -- 172.21.15.153:0/1255908196 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f46640096e0 con 0x7f46740731c0 2024-03-21T10:19:18.822 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:18.819+0000 7f4678a00700 1 -- 172.21.15.153:0/1255908196 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f467419d5e0 con 0x7f46740731c0 2024-03-21T10:19:18.822 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:18.820+0000 7f466aa00700 1 -- 172.21.15.153:0/1255908196 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f466401d6b0 con 0x7f46740731c0 2024-03-21T10:19:18.822 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:18.820+0000 7f4678a00700 1 -- 172.21.15.153:0/1255908196 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f467419dad0 con 0x7f46740731c0 2024-03-21T10:19:18.823 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:18.821+0000 7f4678a00700 1 -- 172.21.15.153:0/1255908196 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f467410a540 con 0x7f46740731c0 2024-03-21T10:19:18.823 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:18.822+0000 7f466aa00700 1 -- 172.21.15.153:0/1255908196 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f4664004590 con 0x7f46740731c0 2024-03-21T10:19:18.824 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:18.823+0000 7f466aa00700 1 --2- 172.21.15.153:0/1255908196 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f465806e460 0x7f4658070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:19:18.825 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:18.823+0000 7f466aa00700 1 -- 172.21.15.153:0/1255908196 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f466408fcf0 con 0x7f46740731c0 2024-03-21T10:19:18.825 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:18.823+0000 7f466be00700 1 --2- 172.21.15.153:0/1255908196 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f465806e460 0x7f4658070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:19:18.825 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:18.824+0000 7f466be00700 1 --2- 172.21.15.153:0/1255908196 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f465806e460 0x7f4658070920 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f465c005950 tx=0x7f465c0058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:19:18.829 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:18.827+0000 7f466aa00700 1 -- 172.21.15.153:0/1255908196 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f466405c840 con 0x7f46740731c0 2024-03-21T10:19:18.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:18 smithi174 ceph-mon[88004]: from='client.34450 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:19:18.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:18 smithi174 ceph-mon[88004]: from='client.34454 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:19:18.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:18 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/1365055814' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:19:19.062 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:19.060+0000 7f4678a00700 1 -- 172.21.15.153:0/1255908196 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f467419ddb0 con 0x7f465806e460 2024-03-21T10:19:19.063 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:19.062+0000 7f466aa00700 1 -- 172.21.15.153:0/1255908196 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f467419ddb0 con 0x7f465806e460 2024-03-21T10:19:19.063 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:19:19.063 INFO:teuthology.orchestra.run.smithi153.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232", 2024-03-21T10:19:19.064 INFO:teuthology.orchestra.run.smithi153.stdout: "in_progress": true, 2024-03-21T10:19:19.064 INFO:teuthology.orchestra.run.smithi153.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-21T10:19:19.064 INFO:teuthology.orchestra.run.smithi153.stdout: "services_complete": [ 2024-03-21T10:19:19.064 INFO:teuthology.orchestra.run.smithi153.stdout: "crash", 2024-03-21T10:19:19.064 INFO:teuthology.orchestra.run.smithi153.stdout: "mon", 2024-03-21T10:19:19.064 INFO:teuthology.orchestra.run.smithi153.stdout: "osd", 2024-03-21T10:19:19.064 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr" 2024-03-21T10:19:19.064 INFO:teuthology.orchestra.run.smithi153.stdout: ], 2024-03-21T10:19:19.064 INFO:teuthology.orchestra.run.smithi153.stdout: "progress": "12/23 daemons upgraded", 2024-03-21T10:19:19.064 INFO:teuthology.orchestra.run.smithi153.stdout: "message": "Currently upgrading mds daemons", 2024-03-21T10:19:19.064 INFO:teuthology.orchestra.run.smithi153.stdout: "is_paused": false 2024-03-21T10:19:19.064 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:19:19.066 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:19.064+0000 7f4678a00700 1 -- 172.21.15.153:0/1255908196 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f465806e460 msgr2=0x7f4658070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:19:19.066 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:19.064+0000 7f4678a00700 1 --2- 172.21.15.153:0/1255908196 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f465806e460 0x7f4658070920 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f465c005950 tx=0x7f465c0058e0 comp rx=0 tx=0).stop 2024-03-21T10:19:19.066 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:19.064+0000 7f4678a00700 1 -- 172.21.15.153:0/1255908196 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f46740731c0 msgr2=0x7f46741989d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:19:19.066 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:19.064+0000 7f4678a00700 1 --2- 172.21.15.153:0/1255908196 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f46740731c0 0x7f46741989d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f466400b070 tx=0x7f4664003ce0 comp rx=0 tx=0).stop 2024-03-21T10:19:19.066 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:19.064+0000 7f4678a00700 1 -- 172.21.15.153:0/1255908196 shutdown_connections 2024-03-21T10:19:19.067 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:19.064+0000 7f4678a00700 1 --2- 172.21.15.153:0/1255908196 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f465806e460 0x7f4658070920 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:19:19.067 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:19.064+0000 7f4678a00700 1 --2- 172.21.15.153:0/1255908196 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f46740731c0 0x7f46741989d0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:19:19.067 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:19.064+0000 7f4678a00700 1 --2- 172.21.15.153:0/1255908196 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4674073ae0 0x7f4674198f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:19:19.067 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:19.064+0000 7f4678a00700 1 -- 172.21.15.153:0/1255908196 >> 172.21.15.153:0/1255908196 conn(0x7f46740fc630 msgr2=0x7f4674107680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:19:19.067 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:19.065+0000 7f4678a00700 1 -- 172.21.15.153:0/1255908196 shutdown_connections 2024-03-21T10:19:19.067 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:19.065+0000 7f4678a00700 1 -- 172.21.15.153:0/1255908196 wait complete. 2024-03-21T10:19:19.219 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:19.217+0000 7f5786a00700 1 -- 172.21.15.153:0/17270225 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f578010d280 msgr2=0x7f578010d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:19:19.219 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:19.217+0000 7f5786a00700 1 --2- 172.21.15.153:0/17270225 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f578010d280 0x7f578010d660 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5770009910 tx=0x7f5770009c20 comp rx=0 tx=0).stop 2024-03-21T10:19:19.219 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:19.217+0000 7f5786a00700 1 -- 172.21.15.153:0/17270225 shutdown_connections 2024-03-21T10:19:19.219 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:19.217+0000 7f5786a00700 1 --2- 172.21.15.153:0/17270225 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f57801071d0 0x7f5780107650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:19:19.219 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:19.217+0000 7f5786a00700 1 --2- 172.21.15.153:0/17270225 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f578010d280 0x7f578010d660 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:19:19.220 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:19.217+0000 7f5786a00700 1 -- 172.21.15.153:0/17270225 >> 172.21.15.153:0/17270225 conn(0x7f57800769e0 msgr2=0x7f5780078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:19:19.220 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:19.217+0000 7f5786a00700 1 -- 172.21.15.153:0/17270225 shutdown_connections 2024-03-21T10:19:19.220 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:19.217+0000 7f5786a00700 1 -- 172.21.15.153:0/17270225 wait complete. 2024-03-21T10:19:19.222 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:19.218+0000 7f5786a00700 1 Processor -- start 2024-03-21T10:19:19.222 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:19.218+0000 7f5786a00700 1 -- start start 2024-03-21T10:19:19.222 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:19.218+0000 7f5786a00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f57801071d0 0x7f578019ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:19:19.222 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:19.219+0000 7f5786a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f578010d280 0x7f578019d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:19:19.222 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:19.219+0000 7f5786a00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f578019d8a0 con 0x7f578010d280 2024-03-21T10:19:19.222 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:19.219+0000 7f5786a00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57801a05d0 con 0x7f57801071d0 2024-03-21T10:19:19.222 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:19.219+0000 7f577f400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f578010d280 0x7f578019d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:19:19.222 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:19.219+0000 7f577f400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f578010d280 0x7f578019d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:47304/0 (socket says 172.21.15.153:47304) 2024-03-21T10:19:19.223 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:19.219+0000 7f577f400700 1 -- 172.21.15.153:0/2940876854 learned_addr learned my addr 172.21.15.153:0/2940876854 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:19:19.223 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:19.219+0000 7f577fe00700 1 --2- 172.21.15.153:0/2940876854 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f57801071d0 0x7f578019ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:19:19.223 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:19.219+0000 7f577f400700 1 -- 172.21.15.153:0/2940876854 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f57801071d0 msgr2=0x7f578019ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:19:19.223 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:19.219+0000 7f577f400700 1 --2- 172.21.15.153:0/2940876854 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f57801071d0 0x7f578019ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:19:19.223 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:19.219+0000 7f577f400700 1 -- 172.21.15.153:0/2940876854 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57700095f0 con 0x7f578010d280 2024-03-21T10:19:19.223 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:19.220+0000 7f577f400700 1 --2- 172.21.15.153:0/2940876854 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f578010d280 0x7f578019d210 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f576c00d8d0 tx=0x7f576c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:19:19.223 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:19.220+0000 7f577cc00700 1 -- 172.21.15.153:0/2940876854 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f576c009940 con 0x7f578010d280 2024-03-21T10:19:19.223 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:19.220+0000 7f577cc00700 1 -- 172.21.15.153:0/2940876854 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f576c010460 con 0x7f578010d280 2024-03-21T10:19:19.223 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:19.220+0000 7f5786a00700 1 -- 172.21.15.153:0/2940876854 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57801a08b0 con 0x7f578010d280 2024-03-21T10:19:19.223 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:19.220+0000 7f577cc00700 1 -- 172.21.15.153:0/2940876854 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f576c00f5f0 con 0x7f578010d280 2024-03-21T10:19:19.223 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:19.220+0000 7f5786a00700 1 -- 172.21.15.153:0/2940876854 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57801a0e80 con 0x7f578010d280 2024-03-21T10:19:19.224 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:19.221+0000 7f5786a00700 1 -- 172.21.15.153:0/2940876854 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f578004ea90 con 0x7f578010d280 2024-03-21T10:19:19.224 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:19.222+0000 7f577cc00700 1 -- 172.21.15.153:0/2940876854 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f576c0105d0 con 0x7f578010d280 2024-03-21T10:19:19.225 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:19.223+0000 7f577cc00700 1 --2- 172.21.15.153:0/2940876854 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f576406e460 0x7f5764070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:19:19.225 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:19.223+0000 7f577cc00700 1 -- 172.21.15.153:0/2940876854 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f576c08e3e0 con 0x7f578010d280 2024-03-21T10:19:19.225 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:19.224+0000 7f577fe00700 1 --2- 172.21.15.153:0/2940876854 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f576406e460 0x7f5764070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:19:19.226 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:19.224+0000 7f577fe00700 1 --2- 172.21.15.153:0/2940876854 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f576406e460 0x7f5764070920 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5770006010 tx=0x7f577001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:19:19.230 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:19.228+0000 7f577cc00700 1 -- 172.21.15.153:0/2940876854 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f576c057520 con 0x7f578010d280 2024-03-21T10:19:19.545 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:19.543+0000 7f5786a00700 1 -- 172.21.15.153:0/2940876854 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f57801a11f0 con 0x7f578010d280 2024-03-21T10:19:19.545 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:19.543+0000 7f577cc00700 1 -- 172.21.15.153:0/2940876854 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f576c05abe0 con 0x7f578010d280 2024-03-21T10:19:19.545 INFO:teuthology.orchestra.run.smithi153.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-03-21T10:19:19.546 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-21T10:19:19.546 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is degraded 2024-03-21T10:19:19.546 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-21T10:19:19.546 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-21T10:19:19.546 INFO:teuthology.orchestra.run.smithi153.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-21T10:19:19.546 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-21T10:19:19.548 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:19.546+0000 7f5786a00700 1 -- 172.21.15.153:0/2940876854 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f576406e460 msgr2=0x7f5764070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:19:19.549 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:19.546+0000 7f5786a00700 1 --2- 172.21.15.153:0/2940876854 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f576406e460 0x7f5764070920 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5770006010 tx=0x7f577001a040 comp rx=0 tx=0).stop 2024-03-21T10:19:19.549 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:19.546+0000 7f5786a00700 1 -- 172.21.15.153:0/2940876854 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f578010d280 msgr2=0x7f578019d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:19:19.549 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:19.546+0000 7f5786a00700 1 --2- 172.21.15.153:0/2940876854 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f578010d280 0x7f578019d210 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f576c00d8d0 tx=0x7f576c00dc90 comp rx=0 tx=0).stop 2024-03-21T10:19:19.549 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:19.546+0000 7f5786a00700 1 -- 172.21.15.153:0/2940876854 shutdown_connections 2024-03-21T10:19:19.549 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:19.546+0000 7f5786a00700 1 --2- 172.21.15.153:0/2940876854 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f57801071d0 0x7f578019ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:19:19.549 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:19.547+0000 7f5786a00700 1 --2- 172.21.15.153:0/2940876854 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f576406e460 0x7f5764070920 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:19:19.549 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:19.547+0000 7f5786a00700 1 --2- 172.21.15.153:0/2940876854 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f578010d280 0x7f578019d210 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:19:19.549 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:19.547+0000 7f5786a00700 1 -- 172.21.15.153:0/2940876854 >> 172.21.15.153:0/2940876854 conn(0x7f57800769e0 msgr2=0x7f5780078320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:19:19.549 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:19.547+0000 7f5786a00700 1 -- 172.21.15.153:0/2940876854 shutdown_connections 2024-03-21T10:19:19.549 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:19.547+0000 7f5786a00700 1 -- 172.21.15.153:0/2940876854 wait complete. 2024-03-21T10:19:19.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:19 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/1280922992' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:19:19.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:19 smithi153 ceph-mon[113733]: from='client.34466 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:19:19.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:19 smithi153 ceph-mon[113733]: pgmap v222: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:19:19.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:19 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/1280922992' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:19:19.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:19 smithi174 ceph-mon[88004]: from='client.34466 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:19:19.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:19 smithi174 ceph-mon[88004]: pgmap v222: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:19:20.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:20 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/2940876854' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:19:20.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:20 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/2940876854' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:19:21.780 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:21 smithi153 ceph-mon[113733]: pgmap v223: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:19:21.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:21 smithi174 ceph-mon[88004]: pgmap v223: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:19:23.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:23 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:19:23.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:23 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:19:23.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:23 smithi153 ceph-mon[113733]: pgmap v224: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:19:23.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:23 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:19:23.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:23 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:19:23.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:23 smithi174 ceph-mon[88004]: pgmap v224: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:19:25.780 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:25 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:19:25.781 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:25 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:19:25.781 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:25 smithi153 ceph-mon[113733]: pgmap v225: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:19:25.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:25 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:19:25.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:25 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:19:25.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:25 smithi174 ceph-mon[88004]: pgmap v225: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:19:27.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:27 smithi174 ceph-mon[88004]: pgmap v226: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:19:27.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:27 smithi153 ceph-mon[113733]: pgmap v226: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:19:29.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:29 smithi153 ceph-mon[113733]: pgmap v227: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:19:29.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:29 smithi174 ceph-mon[88004]: pgmap v227: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:19:31.362 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:31 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:19:31.362 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:31 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:19:31.362 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:31 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:19:31.362 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:31 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:19:31.362 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:31 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:19:31.362 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:31 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:19:31.362 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:31 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:19:31.362 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:31 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:19:31.362 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:31 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:19:31.362 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:31 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:19:31.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:31 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:19:31.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:31 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:19:31.453 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:31 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:19:31.453 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:31 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:19:31.453 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:31 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:19:31.453 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:31 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:19:31.453 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:31 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:19:31.453 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:31 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:19:31.453 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:31 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:19:31.453 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:31 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:19:32.355 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:32 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:19:32.355 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:32 smithi174 ceph-mon[88004]: pgmap v228: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:19:32.399 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:32 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:19:32.399 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:32 smithi153 ceph-mon[113733]: pgmap v228: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:19:33.914 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:33 smithi153 ceph-mon[113733]: Upgrade: Updating mds.cephfs.smithi153.iuyluj 2024-03-21T10:19:33.915 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:33 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:19:33.915 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:33 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi153.iuyluj", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T10:19:33.915 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:33 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:19:33.915 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:33 smithi153 ceph-mon[113733]: Deploying daemon mds.cephfs.smithi153.iuyluj on smithi153 2024-03-21T10:19:33.915 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:33 smithi153 ceph-mon[113733]: pgmap v229: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:19:33.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:33 smithi174 ceph-mon[88004]: Upgrade: Updating mds.cephfs.smithi153.iuyluj 2024-03-21T10:19:33.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:33 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:19:33.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:33 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi153.iuyluj", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T10:19:33.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:33 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:19:33.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:33 smithi174 ceph-mon[88004]: Deploying daemon mds.cephfs.smithi153.iuyluj on smithi153 2024-03-21T10:19:33.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:33 smithi174 ceph-mon[88004]: pgmap v229: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:19:35.635 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:35 smithi153 ceph-mon[113733]: pgmap v230: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:19:35.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:35 smithi174 ceph-mon[88004]: pgmap v230: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:19:37.555 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:37 smithi153 ceph-mon[113733]: osdmap e96: 6 total, 6 up, 6 in 2024-03-21T10:19:37.555 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:37 smithi153 ceph-mon[113733]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-03-21T10:19:37.555 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:37 smithi153 ceph-mon[113733]: pgmap v232: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:19:37.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:37 smithi174 ceph-mon[88004]: osdmap e96: 6 total, 6 up, 6 in 2024-03-21T10:19:37.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:37 smithi174 ceph-mon[88004]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-03-21T10:19:37.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:37 smithi174 ceph-mon[88004]: pgmap v232: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:19:39.604 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:39 smithi153 ceph-mon[113733]: pgmap v233: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:19:39.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:39 smithi174 ceph-mon[88004]: pgmap v233: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:19:40.886 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:40 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:19:40.887 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:40 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:19:40.887 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:40 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:19:40.887 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:40 smithi153 ceph-mon[113733]: mds.? [v2:172.21.15.153:6828/1738630832,v1:172.21.15.153:6829/1738630832] up:boot 2024-03-21T10:19:40.887 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:40 smithi153 ceph-mon[113733]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-03-21T10:19:40.887 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:40 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi153.iuyluj"}]: dispatch 2024-03-21T10:19:40.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:40 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:19:40.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:40 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:19:40.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:40 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:19:40.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:40 smithi174 ceph-mon[88004]: mds.? [v2:172.21.15.153:6828/1738630832,v1:172.21.15.153:6829/1738630832] up:boot 2024-03-21T10:19:40.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:40 smithi174 ceph-mon[88004]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-03-21T10:19:40.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:40 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi153.iuyluj"}]: dispatch 2024-03-21T10:19:41.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:41 smithi174 ceph-mon[88004]: pgmap v234: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:19:42.024 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:41 smithi153 ceph-mon[113733]: pgmap v234: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:19:43.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:43 smithi174 ceph-mon[88004]: pgmap v235: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:19:43.724 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:43 smithi153 ceph-mon[113733]: pgmap v235: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:19:45.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:45 smithi174 ceph-mon[88004]: pgmap v236: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:19:45.709 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:45 smithi153 ceph-mon[113733]: pgmap v236: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:19:46.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:46 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:19:46.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:46 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:19:47.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:47 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:19:47.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:47 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:19:47.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:47 smithi174 ceph-mon[88004]: pgmap v237: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:19:48.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:47 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:19:48.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:47 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:19:48.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:47 smithi153 ceph-mon[113733]: pgmap v237: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:19:49.714 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:49.711+0000 7ff764600700 1 -- 172.21.15.153:0/901228341 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff75c102620 msgr2=0x7ff75c10ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:19:49.714 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:49.711+0000 7ff764600700 1 --2- 172.21.15.153:0/901228341 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff75c102620 0x7ff75c10ab10 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff748009910 tx=0x7ff748009c20 comp rx=0 tx=0).stop 2024-03-21T10:19:49.714 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:49.711+0000 7ff764600700 1 -- 172.21.15.153:0/901228341 shutdown_connections 2024-03-21T10:19:49.714 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:49.711+0000 7ff764600700 1 --2- 172.21.15.153:0/901228341 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff75c102620 0x7ff75c10ab10 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:19:49.714 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:49.711+0000 7ff764600700 1 --2- 172.21.15.153:0/901228341 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff75c101d00 0x7ff75c1020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:19:49.714 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:49.711+0000 7ff764600700 1 -- 172.21.15.153:0/901228341 >> 172.21.15.153:0/901228341 conn(0x7ff75c0fb570 msgr2=0x7ff75c0fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:19:49.714 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:49.711+0000 7ff764600700 1 -- 172.21.15.153:0/901228341 shutdown_connections 2024-03-21T10:19:49.714 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:49.711+0000 7ff764600700 1 -- 172.21.15.153:0/901228341 wait complete. 2024-03-21T10:19:49.715 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:49.713+0000 7ff764600700 1 Processor -- start 2024-03-21T10:19:49.715 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:49.713+0000 7ff764600700 1 -- start start 2024-03-21T10:19:49.716 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:49.713+0000 7ff764600700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff75c101d00 0x7ff75c19c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:19:49.716 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:49.714+0000 7ff764600700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff75c102620 0x7ff75c19ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:19:49.716 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:49.714+0000 7ff764600700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff75c19d100 con 0x7ff75c102620 2024-03-21T10:19:49.716 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:49.714+0000 7ff764600700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff75c1965b0 con 0x7ff75c101d00 2024-03-21T10:19:49.716 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:49.714+0000 7ff761200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff75c102620 0x7ff75c19ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:19:49.716 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:49.714+0000 7ff761200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff75c102620 0x7ff75c19ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:37856/0 (socket says 172.21.15.153:37856) 2024-03-21T10:19:49.716 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:49.714+0000 7ff761200700 1 -- 172.21.15.153:0/1419361566 learned_addr learned my addr 172.21.15.153:0/1419361566 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:19:49.717 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:49.714+0000 7ff761c00700 1 --2- 172.21.15.153:0/1419361566 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff75c101d00 0x7ff75c19c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:19:49.717 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:49.714+0000 7ff761200700 1 -- 172.21.15.153:0/1419361566 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff75c101d00 msgr2=0x7ff75c19c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:19:49.717 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:49.714+0000 7ff761200700 1 --2- 172.21.15.153:0/1419361566 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff75c101d00 0x7ff75c19c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:19:49.717 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:49.714+0000 7ff761200700 1 -- 172.21.15.153:0/1419361566 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7480095f0 con 0x7ff75c102620 2024-03-21T10:19:49.717 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:49.715+0000 7ff761200700 1 --2- 172.21.15.153:0/1419361566 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff75c102620 0x7ff75c19ca70 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff748005f50 tx=0x7ff74800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:19:49.717 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:49.715+0000 7ff75a000700 1 -- 172.21.15.153:0/1419361566 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff74801d070 con 0x7ff75c102620 2024-03-21T10:19:49.717 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:49.715+0000 7ff75a000700 1 -- 172.21.15.153:0/1419361566 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff74800f460 con 0x7ff75c102620 2024-03-21T10:19:49.717 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:49.715+0000 7ff764600700 1 -- 172.21.15.153:0/1419361566 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff75c196830 con 0x7ff75c102620 2024-03-21T10:19:49.717 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:49.715+0000 7ff75a000700 1 -- 172.21.15.153:0/1419361566 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff748017630 con 0x7ff75c102620 2024-03-21T10:19:49.718 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:49.715+0000 7ff764600700 1 -- 172.21.15.153:0/1419361566 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff75c196d20 con 0x7ff75c102620 2024-03-21T10:19:49.718 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:49.716+0000 7ff764600700 1 -- 172.21.15.153:0/1419361566 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff75c108210 con 0x7ff75c102620 2024-03-21T10:19:49.719 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:49.717+0000 7ff75a000700 1 -- 172.21.15.153:0/1419361566 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7ff74800f5d0 con 0x7ff75c102620 2024-03-21T10:19:49.720 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:49.718+0000 7ff75a000700 1 --2- 172.21.15.153:0/1419361566 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff74406e460 0x7ff744070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:19:49.720 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:49.718+0000 7ff75a000700 1 -- 172.21.15.153:0/1419361566 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6826+0+0 (secure 0 0 0) 0x7ff74808fb10 con 0x7ff75c102620 2024-03-21T10:19:49.720 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:49.719+0000 7ff761c00700 1 --2- 172.21.15.153:0/1419361566 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff74406e460 0x7ff744070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:19:49.721 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:49.719+0000 7ff761c00700 1 --2- 172.21.15.153:0/1419361566 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff74406e460 0x7ff744070920 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff750005950 tx=0x7ff750005eb0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:19:49.725 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:49.724+0000 7ff75a000700 1 -- 172.21.15.153:0/1419361566 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff74805c660 con 0x7ff75c102620 2024-03-21T10:19:49.978 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:49.975+0000 7ff764600700 1 -- 172.21.15.153:0/1419361566 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff75c02cd00 con 0x7ff74406e460 2024-03-21T10:19:49.978 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:49 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:19:49.978 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:49 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:19:49.978 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:49 smithi153 ceph-mon[113733]: pgmap v238: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:19:49.979 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:49.978+0000 7ff75a000700 1 -- 172.21.15.153:0/1419361566 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7ff75c02cd00 con 0x7ff74406e460 2024-03-21T10:19:49.982 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:49.980+0000 7ff764600700 1 -- 172.21.15.153:0/1419361566 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff74406e460 msgr2=0x7ff744070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:19:49.982 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:49.980+0000 7ff764600700 1 --2- 172.21.15.153:0/1419361566 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff74406e460 0x7ff744070920 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff750005950 tx=0x7ff750005eb0 comp rx=0 tx=0).stop 2024-03-21T10:19:49.982 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:49.980+0000 7ff764600700 1 -- 172.21.15.153:0/1419361566 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff75c102620 msgr2=0x7ff75c19ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:19:49.982 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:49.980+0000 7ff764600700 1 --2- 172.21.15.153:0/1419361566 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff75c102620 0x7ff75c19ca70 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff748005f50 tx=0x7ff74800be20 comp rx=0 tx=0).stop 2024-03-21T10:19:49.983 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:49.981+0000 7ff764600700 1 -- 172.21.15.153:0/1419361566 shutdown_connections 2024-03-21T10:19:49.983 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:49.981+0000 7ff764600700 1 --2- 172.21.15.153:0/1419361566 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff75c101d00 0x7ff75c19c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:19:49.983 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:49.981+0000 7ff764600700 1 --2- 172.21.15.153:0/1419361566 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff74406e460 0x7ff744070920 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:19:49.983 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:49.981+0000 7ff764600700 1 --2- 172.21.15.153:0/1419361566 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff75c102620 0x7ff75c19ca70 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:19:49.983 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:49.981+0000 7ff764600700 1 -- 172.21.15.153:0/1419361566 >> 172.21.15.153:0/1419361566 conn(0x7ff75c0fb570 msgr2=0x7ff75c105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:19:49.983 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:49.981+0000 7ff764600700 1 -- 172.21.15.153:0/1419361566 shutdown_connections 2024-03-21T10:19:49.983 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:49.981+0000 7ff764600700 1 -- 172.21.15.153:0/1419361566 wait complete. 2024-03-21T10:19:49.999 INFO:teuthology.orchestra.run.smithi153.stdout:true 2024-03-21T10:19:50.144 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:50.141+0000 7f9b4a000700 1 -- 172.21.15.153:0/375370510 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9b44103000 msgr2=0x7f9b441033e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:19:50.144 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:50.141+0000 7f9b4a000700 1 --2- 172.21.15.153:0/375370510 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9b44103000 0x7f9b441033e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9b34009910 tx=0x7f9b34009c20 comp rx=0 tx=0).stop 2024-03-21T10:19:50.144 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:50.142+0000 7f9b4a000700 1 -- 172.21.15.153:0/375370510 shutdown_connections 2024-03-21T10:19:50.144 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:50.142+0000 7f9b4a000700 1 --2- 172.21.15.153:0/375370510 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9b441039b0 0x7f9b44107a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:19:50.144 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:50.142+0000 7f9b4a000700 1 --2- 172.21.15.153:0/375370510 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9b44103000 0x7f9b441033e0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:19:50.144 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:50.142+0000 7f9b4a000700 1 -- 172.21.15.153:0/375370510 >> 172.21.15.153:0/375370510 conn(0x7f9b440fe8d0 msgr2=0x7f9b44100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:19:50.144 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:50.142+0000 7f9b4a000700 1 -- 172.21.15.153:0/375370510 shutdown_connections 2024-03-21T10:19:50.145 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:50.142+0000 7f9b4a000700 1 -- 172.21.15.153:0/375370510 wait complete. 2024-03-21T10:19:50.147 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:50.143+0000 7f9b4a000700 1 Processor -- start 2024-03-21T10:19:50.147 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:50.143+0000 7f9b4a000700 1 -- start start 2024-03-21T10:19:50.147 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:50.143+0000 7f9b4a000700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9b44103000 0x7f9b4419e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:19:50.147 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:50.143+0000 7f9b4a000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9b441039b0 0x7f9b4419ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:19:50.147 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:50.143+0000 7f9b4a000700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b4419f330 con 0x7f9b441039b0 2024-03-21T10:19:50.147 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:50.143+0000 7f9b4a000700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b44198870 con 0x7f9b44103000 2024-03-21T10:19:50.147 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:50.143+0000 7f9b42a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9b441039b0 0x7f9b4419ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:19:50.147 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:50.144+0000 7f9b42a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9b441039b0 0x7f9b4419ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:37868/0 (socket says 172.21.15.153:37868) 2024-03-21T10:19:50.147 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:50.144+0000 7f9b42a00700 1 -- 172.21.15.153:0/2900425872 learned_addr learned my addr 172.21.15.153:0/2900425872 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:19:50.147 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:50.144+0000 7f9b43400700 1 --2- 172.21.15.153:0/2900425872 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9b44103000 0x7f9b4419e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:19:50.148 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:50.144+0000 7f9b42a00700 1 -- 172.21.15.153:0/2900425872 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9b44103000 msgr2=0x7f9b4419e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:19:50.148 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:50.144+0000 7f9b42a00700 1 --2- 172.21.15.153:0/2900425872 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9b44103000 0x7f9b4419e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:19:50.148 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:50.144+0000 7f9b42a00700 1 -- 172.21.15.153:0/2900425872 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b340095f0 con 0x7f9b441039b0 2024-03-21T10:19:50.148 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:50.144+0000 7f9b43400700 1 --2- 172.21.15.153:0/2900425872 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9b44103000 0x7f9b4419e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:19:50.148 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:50.144+0000 7f9b42a00700 1 --2- 172.21.15.153:0/2900425872 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9b441039b0 0x7f9b4419ed30 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9b3000c960 tx=0x7f9b3000cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:19:50.148 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:50.145+0000 7f9b3be00700 1 -- 172.21.15.153:0/2900425872 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9b30007a70 con 0x7f9b441039b0 2024-03-21T10:19:50.148 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:50.145+0000 7f9b3be00700 1 -- 172.21.15.153:0/2900425872 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9b30007bd0 con 0x7f9b441039b0 2024-03-21T10:19:50.148 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:50.145+0000 7f9b3be00700 1 -- 172.21.15.153:0/2900425872 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9b300186c0 con 0x7f9b441039b0 2024-03-21T10:19:50.148 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:50.145+0000 7f9b4a000700 1 -- 172.21.15.153:0/2900425872 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b44198b50 con 0x7f9b441039b0 2024-03-21T10:19:50.148 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:50.145+0000 7f9b4a000700 1 -- 172.21.15.153:0/2900425872 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b441990a0 con 0x7f9b441039b0 2024-03-21T10:19:50.149 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:50.146+0000 7f9b4a000700 1 -- 172.21.15.153:0/2900425872 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b4410b3a0 con 0x7f9b441039b0 2024-03-21T10:19:50.149 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:50.147+0000 7f9b3be00700 1 -- 172.21.15.153:0/2900425872 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f9b3000faa0 con 0x7f9b441039b0 2024-03-21T10:19:50.150 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:50.148+0000 7f9b3be00700 1 --2- 172.21.15.153:0/2900425872 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f9b2806e460 0x7f9b28070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:19:50.150 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:50.148+0000 7f9b3be00700 1 -- 172.21.15.153:0/2900425872 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f9b3008e520 con 0x7f9b441039b0 2024-03-21T10:19:50.150 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:50.148+0000 7f9b43400700 1 --2- 172.21.15.153:0/2900425872 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f9b2806e460 0x7f9b28070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:19:50.151 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:50.149+0000 7f9b43400700 1 --2- 172.21.15.153:0/2900425872 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f9b2806e460 0x7f9b28070920 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9b34006010 tx=0x7f9b3400b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:19:50.155 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:50.153+0000 7f9b3be00700 1 -- 172.21.15.153:0/2900425872 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f9b3005ae60 con 0x7f9b441039b0 2024-03-21T10:19:50.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:49 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:19:50.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:49 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:19:50.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:49 smithi174 ceph-mon[88004]: pgmap v238: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:19:50.414 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:50.412+0000 7f9b4a000700 1 -- 172.21.15.153:0/2900425872 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9b44199370 con 0x7f9b2806e460 2024-03-21T10:19:50.416 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:50.414+0000 7f9b3be00700 1 -- 172.21.15.153:0/2900425872 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f9b44199370 con 0x7f9b2806e460 2024-03-21T10:19:50.419 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:50.417+0000 7f9b4a000700 1 -- 172.21.15.153:0/2900425872 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f9b2806e460 msgr2=0x7f9b28070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:19:50.419 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:50.417+0000 7f9b4a000700 1 --2- 172.21.15.153:0/2900425872 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f9b2806e460 0x7f9b28070920 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9b34006010 tx=0x7f9b3400b540 comp rx=0 tx=0).stop 2024-03-21T10:19:50.419 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:50.417+0000 7f9b4a000700 1 -- 172.21.15.153:0/2900425872 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9b441039b0 msgr2=0x7f9b4419ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:19:50.419 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:50.417+0000 7f9b4a000700 1 --2- 172.21.15.153:0/2900425872 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9b441039b0 0x7f9b4419ed30 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9b3000c960 tx=0x7f9b3000cd20 comp rx=0 tx=0).stop 2024-03-21T10:19:50.420 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:50.417+0000 7f9b4a000700 1 -- 172.21.15.153:0/2900425872 shutdown_connections 2024-03-21T10:19:50.420 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:50.417+0000 7f9b4a000700 1 --2- 172.21.15.153:0/2900425872 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9b44103000 0x7f9b4419e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:19:50.420 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:50.417+0000 7f9b4a000700 1 --2- 172.21.15.153:0/2900425872 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f9b2806e460 0x7f9b28070920 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:19:50.420 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:50.417+0000 7f9b4a000700 1 --2- 172.21.15.153:0/2900425872 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9b441039b0 0x7f9b4419ed30 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:19:50.420 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:50.417+0000 7f9b4a000700 1 -- 172.21.15.153:0/2900425872 >> 172.21.15.153:0/2900425872 conn(0x7f9b440fe8d0 msgr2=0x7f9b440ffee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:19:50.420 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:50.418+0000 7f9b4a000700 1 -- 172.21.15.153:0/2900425872 shutdown_connections 2024-03-21T10:19:50.420 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:50.418+0000 7f9b4a000700 1 -- 172.21.15.153:0/2900425872 wait complete. 2024-03-21T10:19:50.575 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:50.573+0000 7f44b2200700 1 -- 172.21.15.153:0/3409429880 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f44ac0730b0 msgr2=0x7f44ac073490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:19:50.576 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:50.573+0000 7f44b2200700 1 --2- 172.21.15.153:0/3409429880 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f44ac0730b0 0x7f44ac073490 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f449c009910 tx=0x7f449c009c20 comp rx=0 tx=0).stop 2024-03-21T10:19:50.576 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:50.573+0000 7f44b2200700 1 -- 172.21.15.153:0/3409429880 shutdown_connections 2024-03-21T10:19:50.576 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:50.573+0000 7f44b2200700 1 --2- 172.21.15.153:0/3409429880 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f44ac0739d0 0x7f44ac10cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:19:50.576 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:50.573+0000 7f44b2200700 1 --2- 172.21.15.153:0/3409429880 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f44ac0730b0 0x7f44ac073490 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:19:50.576 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:50.573+0000 7f44b2200700 1 -- 172.21.15.153:0/3409429880 >> 172.21.15.153:0/3409429880 conn(0x7f44ac0fc650 msgr2=0x7f44ac0fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:19:50.576 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:50.574+0000 7f44b2200700 1 -- 172.21.15.153:0/3409429880 shutdown_connections 2024-03-21T10:19:50.576 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:50.574+0000 7f44b2200700 1 -- 172.21.15.153:0/3409429880 wait complete. 2024-03-21T10:19:50.577 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:50.575+0000 7f44b2200700 1 Processor -- start 2024-03-21T10:19:50.577 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:50.575+0000 7f44b2200700 1 -- start start 2024-03-21T10:19:50.577 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:50.575+0000 7f44b2200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f44ac0730b0 0x7f44ac19e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:19:50.577 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:50.575+0000 7f44b2200700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f44ac0739d0 0x7f44ac19ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:19:50.577 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:50.575+0000 7f44b2200700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44ac19f3d0 con 0x7f44ac0730b0 2024-03-21T10:19:50.577 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:50.575+0000 7f44b2200700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44ac198880 con 0x7f44ac0739d0 2024-03-21T10:19:50.578 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:50.576+0000 7f44ab400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f44ac0730b0 0x7f44ac19e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:19:50.578 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:50.576+0000 7f44ab400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f44ac0730b0 0x7f44ac19e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:37886/0 (socket says 172.21.15.153:37886) 2024-03-21T10:19:50.578 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:50.576+0000 7f44ab400700 1 -- 172.21.15.153:0/1004083541 learned_addr learned my addr 172.21.15.153:0/1004083541 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:19:50.578 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:50.576+0000 7f44ab400700 1 -- 172.21.15.153:0/1004083541 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f44ac0739d0 msgr2=0x7f44ac19ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:19:50.578 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:50.576+0000 7f44ab400700 1 --2- 172.21.15.153:0/1004083541 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f44ac0739d0 0x7f44ac19ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:19:50.578 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:50.576+0000 7f44ab400700 1 -- 172.21.15.153:0/1004083541 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f449c0095f0 con 0x7f44ac0730b0 2024-03-21T10:19:50.578 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:50.576+0000 7f44ab400700 1 --2- 172.21.15.153:0/1004083541 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f44ac0730b0 0x7f44ac19e800 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f449c006010 tx=0x7f449c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:19:50.579 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:50.577+0000 7f44a3e00700 1 -- 172.21.15.153:0/1004083541 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f449c01d070 con 0x7f44ac0730b0 2024-03-21T10:19:50.579 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:50.577+0000 7f44a3e00700 1 -- 172.21.15.153:0/1004083541 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f449c004fb0 con 0x7f44ac0730b0 2024-03-21T10:19:50.579 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:50.577+0000 7f44b2200700 1 -- 172.21.15.153:0/1004083541 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44ac198b00 con 0x7f44ac0730b0 2024-03-21T10:19:50.579 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:50.577+0000 7f44a3e00700 1 -- 172.21.15.153:0/1004083541 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f449c022ea0 con 0x7f44ac0730b0 2024-03-21T10:19:50.579 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:50.577+0000 7f44b2200700 1 -- 172.21.15.153:0/1004083541 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44ac198ff0 con 0x7f44ac0730b0 2024-03-21T10:19:50.579 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:50.578+0000 7f44b2200700 1 -- 172.21.15.153:0/1004083541 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44ac10a5e0 con 0x7f44ac0730b0 2024-03-21T10:19:50.580 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:50.579+0000 7f44a3e00700 1 -- 172.21.15.153:0/1004083541 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f449c022470 con 0x7f44ac0730b0 2024-03-21T10:19:50.581 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:50.580+0000 7f44a3e00700 1 --2- 172.21.15.153:0/1004083541 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f449006e460 0x7f4490070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:19:50.581 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:50.580+0000 7f44a3e00700 1 -- 172.21.15.153:0/1004083541 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f449c08fa70 con 0x7f44ac0730b0 2024-03-21T10:19:50.581 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:50.580+0000 7f44aaa00700 1 --2- 172.21.15.153:0/1004083541 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f449006e460 0x7f4490070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:19:50.582 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:50.581+0000 7f44aaa00700 1 --2- 172.21.15.153:0/1004083541 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f449006e460 0x7f4490070920 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f44ac0fdd80 tx=0x7f4498006d20 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:19:50.586 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:50.584+0000 7f44a3e00700 1 -- 172.21.15.153:0/1004083541 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f449c05c810 con 0x7f44ac0730b0 2024-03-21T10:19:50.819 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:50.817+0000 7f44b2200700 1 -- 172.21.15.153:0/1004083541 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f44ac199320 con 0x7f449006e460 2024-03-21T10:19:50.851 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:50.849+0000 7f44a3e00700 1 -- 172.21.15.153:0/1004083541 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f44ac199320 con 0x7f449006e460 2024-03-21T10:19:50.852 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:50 smithi153 ceph-mon[113733]: from='client.34478 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:19:50.852 INFO:teuthology.orchestra.run.smithi153.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-21T10:19:50.852 INFO:teuthology.orchestra.run.smithi153.stdout:alertmanager.smithi153 smithi153 *:9093,9094 running (8m) 4s ago 22m 14.7M - 0.25.0 c8568f914cd2 9ed3f2200b5f 2024-03-21T10:19:50.852 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi153 smithi153 running (22m) 4s ago 22m 9424k - 18.2.1 5be31c24972a 5b61994396a6 2024-03-21T10:19:50.852 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi174 smithi174 running (21m) 56s ago 21m 9026k - 18.2.1 5be31c24972a 4f7a5fc3ac28 2024-03-21T10:19:50.852 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi153 smithi153 running (4m) 4s ago 22m 6656k - 19.0.0-2231-g2edfee68 237e43c7a65f 32c21104d5b0 2024-03-21T10:19:50.853 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi174 smithi174 running (4m) 56s ago 21m 6660k - 19.0.0-2231-g2edfee68 237e43c7a65f a04e48967ab6 2024-03-21T10:19:50.853 INFO:teuthology.orchestra.run.smithi153.stdout:grafana.smithi153 smithi153 *:3000 running (8m) 4s ago 21m 73.6M - 9.4.12 2bacad6d85d8 ee24026ab786 2024-03-21T10:19:50.853 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.iuyluj smithi153 running (11s) 4s ago 15m 10.9M - 19.0.0-2231-g2edfee68 237e43c7a65f 87aff8a5de70 2024-03-21T10:19:50.853 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.zipckr smithi153 running (36s) 4s ago 15m 11.8M - 19.0.0-2231-g2edfee68 237e43c7a65f ccc262bd5dd3 2024-03-21T10:19:50.853 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.nhozag smithi174 running (15m) 56s ago 15m 57.5M - 18.2.1 5be31c24972a 2fee4000308f 2024-03-21T10:19:50.853 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.yyimst smithi174 running (15m) 56s ago 15m 117M - 18.2.1 5be31c24972a d78a729220d1 2024-03-21T10:19:50.853 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi153.tluxni smithi153 *:8443,9283,8765 running (11m) 4s ago 24m 509M - 19.0.0-2231-g2edfee68 237e43c7a65f 7ffee0e55382 2024-03-21T10:19:50.853 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi174.qpcwat smithi174 *:8443,9283,8765 running (10m) 56s ago 20m 452M - 19.0.0-2231-g2edfee68 237e43c7a65f 748968f216ab 2024-03-21T10:19:50.853 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi153 smithi153 running (6m) 4s ago 24m 55.0M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f dd8038303dc3 2024-03-21T10:19:50.853 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi174 smithi174 running (5m) 56s ago 20m 40.9M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f 29d78a788f2c 2024-03-21T10:19:50.853 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi153 smithi153 *:9100 running (9m) 4s ago 22m 7889k - 1.5.0 0da6a335fe13 bb835edc0e0b 2024-03-21T10:19:50.853 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi174 smithi174 *:9100 running (9m) 56s ago 20m 8200k - 1.5.0 0da6a335fe13 7c4ecad790fa 2024-03-21T10:19:50.853 INFO:teuthology.orchestra.run.smithi153.stdout:osd.0 smithi153 running (4m) 4s ago 19m 116M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 507900fd4791 2024-03-21T10:19:50.853 INFO:teuthology.orchestra.run.smithi153.stdout:osd.1 smithi153 running (3m) 4s ago 19m 89.2M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 0d646452e9c1 2024-03-21T10:19:50.854 INFO:teuthology.orchestra.run.smithi153.stdout:osd.2 smithi153 running (2m) 4s ago 18m 87.4M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 1326f2afdfcd 2024-03-21T10:19:50.854 INFO:teuthology.orchestra.run.smithi153.stdout:osd.3 smithi174 running (2m) 56s ago 18m 82.8M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 70aafac71ad5 2024-03-21T10:19:50.854 INFO:teuthology.orchestra.run.smithi153.stdout:osd.4 smithi174 running (104s) 56s ago 17m 91.0M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 0d746ce6e79d 2024-03-21T10:19:50.854 INFO:teuthology.orchestra.run.smithi153.stdout:osd.5 smithi174 running (61s) 56s ago 17m 11.2M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 53e5f35b8cac 2024-03-21T10:19:50.854 INFO:teuthology.orchestra.run.smithi153.stdout:prometheus.smithi153 smithi153 *:9095 running (9m) 4s ago 21m 73.5M - 2.43.0 a07b618ecd1d 7d95561e50c2 2024-03-21T10:19:50.855 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:50.852+0000 7f44b2200700 1 -- 172.21.15.153:0/1004083541 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f449006e460 msgr2=0x7f4490070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:19:50.855 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:50.852+0000 7f44b2200700 1 --2- 172.21.15.153:0/1004083541 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f449006e460 0x7f4490070920 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f44ac0fdd80 tx=0x7f4498006d20 comp rx=0 tx=0).stop 2024-03-21T10:19:50.855 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:50.853+0000 7f44b2200700 1 -- 172.21.15.153:0/1004083541 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f44ac0730b0 msgr2=0x7f44ac19e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:19:50.855 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:50.853+0000 7f44b2200700 1 --2- 172.21.15.153:0/1004083541 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f44ac0730b0 0x7f44ac19e800 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f449c006010 tx=0x7f449c00be20 comp rx=0 tx=0).stop 2024-03-21T10:19:50.855 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:50.853+0000 7f44b2200700 1 -- 172.21.15.153:0/1004083541 shutdown_connections 2024-03-21T10:19:50.855 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:50.853+0000 7f44b2200700 1 --2- 172.21.15.153:0/1004083541 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f449006e460 0x7f4490070920 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:19:50.855 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:50.853+0000 7f44b2200700 1 --2- 172.21.15.153:0/1004083541 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f44ac0730b0 0x7f44ac19e800 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:19:50.855 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:50.853+0000 7f44b2200700 1 --2- 172.21.15.153:0/1004083541 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f44ac0739d0 0x7f44ac19ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:19:50.855 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:50.853+0000 7f44b2200700 1 -- 172.21.15.153:0/1004083541 >> 172.21.15.153:0/1004083541 conn(0x7f44ac0fc650 msgr2=0x7f44ac107720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:19:50.856 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:50.853+0000 7f44b2200700 1 -- 172.21.15.153:0/1004083541 shutdown_connections 2024-03-21T10:19:50.856 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:50.853+0000 7f44b2200700 1 -- 172.21.15.153:0/1004083541 wait complete. 2024-03-21T10:19:51.024 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.022+0000 7f942c800700 1 -- 172.21.15.153:0/748398672 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f94241071d0 msgr2=0x7f9424107650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:19:51.025 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.022+0000 7f942c800700 1 --2- 172.21.15.153:0/748398672 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f94241071d0 0x7f9424107650 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9414009910 tx=0x7f9414009c20 comp rx=0 tx=0).stop 2024-03-21T10:19:51.025 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.022+0000 7f942c800700 1 -- 172.21.15.153:0/748398672 shutdown_connections 2024-03-21T10:19:51.025 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.022+0000 7f942c800700 1 --2- 172.21.15.153:0/748398672 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f94241071d0 0x7f9424107650 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:19:51.025 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.022+0000 7f942c800700 1 --2- 172.21.15.153:0/748398672 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f942410d280 0x7f942410d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:19:51.025 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.022+0000 7f942c800700 1 -- 172.21.15.153:0/748398672 >> 172.21.15.153:0/748398672 conn(0x7f94240769e0 msgr2=0x7f9424078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:19:51.025 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.022+0000 7f942c800700 1 -- 172.21.15.153:0/748398672 shutdown_connections 2024-03-21T10:19:51.025 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.022+0000 7f942c800700 1 -- 172.21.15.153:0/748398672 wait complete. 2024-03-21T10:19:51.027 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.023+0000 7f942c800700 1 Processor -- start 2024-03-21T10:19:51.027 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.024+0000 7f942c800700 1 -- start start 2024-03-21T10:19:51.027 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.024+0000 7f942c800700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f94241071d0 0x7f942419cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:19:51.028 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.024+0000 7f942c800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f942410d280 0x7f942419d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:19:51.028 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.024+0000 7f942c800700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f942419d9b0 con 0x7f942410d280 2024-03-21T10:19:51.028 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.024+0000 7f942c800700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94241a1740 con 0x7f94241071d0 2024-03-21T10:19:51.028 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.024+0000 7f9429400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f942410d280 0x7f942419d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:19:51.028 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.024+0000 7f9429400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f942410d280 0x7f942419d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:37910/0 (socket says 172.21.15.153:37910) 2024-03-21T10:19:51.028 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.024+0000 7f9429400700 1 -- 172.21.15.153:0/2476613570 learned_addr learned my addr 172.21.15.153:0/2476613570 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:19:51.028 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.025+0000 7f9429e00700 1 --2- 172.21.15.153:0/2476613570 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f94241071d0 0x7f942419cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:19:51.028 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.025+0000 7f9429400700 1 -- 172.21.15.153:0/2476613570 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f94241071d0 msgr2=0x7f942419cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:19:51.028 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.025+0000 7f9429400700 1 --2- 172.21.15.153:0/2476613570 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f94241071d0 0x7f942419cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:19:51.028 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.025+0000 7f9429400700 1 -- 172.21.15.153:0/2476613570 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94140095f0 con 0x7f942410d280 2024-03-21T10:19:51.028 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.025+0000 7f9429e00700 1 --2- 172.21.15.153:0/2476613570 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f94241071d0 0x7f942419cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:19:51.028 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.025+0000 7f9429400700 1 --2- 172.21.15.153:0/2476613570 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f942410d280 0x7f942419d2d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f941400b5c0 tx=0x7f941400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:19:51.028 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.025+0000 7f941ea00700 1 -- 172.21.15.153:0/2476613570 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f941401d070 con 0x7f942410d280 2024-03-21T10:19:51.029 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.025+0000 7f942c800700 1 -- 172.21.15.153:0/2476613570 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94241a19c0 con 0x7f942410d280 2024-03-21T10:19:51.029 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.025+0000 7f941ea00700 1 -- 172.21.15.153:0/2476613570 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9414005e00 con 0x7f942410d280 2024-03-21T10:19:51.029 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.025+0000 7f942c800700 1 -- 172.21.15.153:0/2476613570 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94241a1eb0 con 0x7f942410d280 2024-03-21T10:19:51.029 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.025+0000 7f941ea00700 1 -- 172.21.15.153:0/2476613570 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9414017400 con 0x7f942410d280 2024-03-21T10:19:51.029 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.026+0000 7f942c800700 1 -- 172.21.15.153:0/2476613570 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f942404ea90 con 0x7f942410d280 2024-03-21T10:19:51.029 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.028+0000 7f941ea00700 1 -- 172.21.15.153:0/2476613570 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f941400f460 con 0x7f942410d280 2024-03-21T10:19:51.030 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.028+0000 7f941ea00700 1 --2- 172.21.15.153:0/2476613570 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f940c06e460 0x7f940c070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:19:51.030 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.029+0000 7f941ea00700 1 -- 172.21.15.153:0/2476613570 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f941408fa70 con 0x7f942410d280 2024-03-21T10:19:51.030 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.029+0000 7f9429e00700 1 --2- 172.21.15.153:0/2476613570 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f940c06e460 0x7f940c070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:19:51.033 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.029+0000 7f9429e00700 1 --2- 172.21.15.153:0/2476613570 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f940c06e460 0x7f940c070920 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f9418005fd0 tx=0x7f9418005ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:19:51.036 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.034+0000 7f941ea00700 1 -- 172.21.15.153:0/2476613570 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f9414058a90 con 0x7f942410d280 2024-03-21T10:19:51.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:50 smithi174 ceph-mon[88004]: from='client.34478 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:19:51.366 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.364+0000 7f942c800700 1 -- 172.21.15.153:0/2476613570 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f942419e0f0 con 0x7f942410d280 2024-03-21T10:19:51.367 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.365+0000 7f941ea00700 1 -- 172.21.15.153:0/2476613570 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+805 (secure 0 0 0) 0x7f941405c150 con 0x7f942410d280 2024-03-21T10:19:51.367 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:19:51.367 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": { 2024-03-21T10:19:51.367 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:19:51.367 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:19:51.367 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": { 2024-03-21T10:19:51.367 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:19:51.368 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:19:51.368 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": { 2024-03-21T10:19:51.368 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 6 2024-03-21T10:19:51.368 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:19:51.368 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": { 2024-03-21T10:19:51.368 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2, 2024-03-21T10:19:51.368 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:19:51.368 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:19:51.368 INFO:teuthology.orchestra.run.smithi153.stdout: "overall": { 2024-03-21T10:19:51.368 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2, 2024-03-21T10:19:51.368 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 12 2024-03-21T10:19:51.368 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-03-21T10:19:51.368 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:19:51.369 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.368+0000 7f942c800700 1 -- 172.21.15.153:0/2476613570 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f940c06e460 msgr2=0x7f940c070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:19:51.369 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.368+0000 7f942c800700 1 --2- 172.21.15.153:0/2476613570 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f940c06e460 0x7f940c070920 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f9418005fd0 tx=0x7f9418005ee0 comp rx=0 tx=0).stop 2024-03-21T10:19:51.369 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.368+0000 7f942c800700 1 -- 172.21.15.153:0/2476613570 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f942410d280 msgr2=0x7f942419d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:19:51.370 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.368+0000 7f942c800700 1 --2- 172.21.15.153:0/2476613570 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f942410d280 0x7f942419d2d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f941400b5c0 tx=0x7f941400be20 comp rx=0 tx=0).stop 2024-03-21T10:19:51.370 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.368+0000 7f942c800700 1 -- 172.21.15.153:0/2476613570 shutdown_connections 2024-03-21T10:19:51.370 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.368+0000 7f942c800700 1 --2- 172.21.15.153:0/2476613570 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f94241071d0 0x7f942419cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:19:51.370 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.368+0000 7f942c800700 1 --2- 172.21.15.153:0/2476613570 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f940c06e460 0x7f940c070920 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:19:51.370 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.368+0000 7f942c800700 1 --2- 172.21.15.153:0/2476613570 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f942410d280 0x7f942419d2d0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:19:51.370 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.368+0000 7f942c800700 1 -- 172.21.15.153:0/2476613570 >> 172.21.15.153:0/2476613570 conn(0x7f94240769e0 msgr2=0x7f9424078480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:19:51.371 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.368+0000 7f942c800700 1 -- 172.21.15.153:0/2476613570 shutdown_connections 2024-03-21T10:19:51.371 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.368+0000 7f942c800700 1 -- 172.21.15.153:0/2476613570 wait complete. 2024-03-21T10:19:51.526 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.524+0000 7fc75cc00700 1 -- 172.21.15.153:0/246531319 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc7581039a0 msgr2=0x7fc7581079f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:19:51.526 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.524+0000 7fc75cc00700 1 --2- 172.21.15.153:0/246531319 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc7581039a0 0x7fc7581079f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc744009910 tx=0x7fc744009c20 comp rx=0 tx=0).stop 2024-03-21T10:19:51.526 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.524+0000 7fc75cc00700 1 -- 172.21.15.153:0/246531319 shutdown_connections 2024-03-21T10:19:51.526 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.524+0000 7fc75cc00700 1 --2- 172.21.15.153:0/246531319 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc7581039a0 0x7fc7581079f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:19:51.526 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.524+0000 7fc75cc00700 1 --2- 172.21.15.153:0/246531319 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc758102ff0 0x7fc7581033d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:19:51.527 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.524+0000 7fc75cc00700 1 -- 172.21.15.153:0/246531319 >> 172.21.15.153:0/246531319 conn(0x7fc7580fe880 msgr2=0x7fc758100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:19:51.527 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.524+0000 7fc75cc00700 1 -- 172.21.15.153:0/246531319 shutdown_connections 2024-03-21T10:19:51.527 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.524+0000 7fc75cc00700 1 -- 172.21.15.153:0/246531319 wait complete. 2024-03-21T10:19:51.527 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.526+0000 7fc75cc00700 1 Processor -- start 2024-03-21T10:19:51.527 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.526+0000 7fc75cc00700 1 -- start start 2024-03-21T10:19:51.528 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.527+0000 7fc75cc00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc758102ff0 0x7fc7581989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:19:51.528 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.527+0000 7fc75cc00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc7581039a0 0x7fc758198ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:19:51.528 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.527+0000 7fc75cc00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc758199570 con 0x7fc7581039a0 2024-03-21T10:19:51.528 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.527+0000 7fc75cc00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc75819d380 con 0x7fc758102ff0 2024-03-21T10:19:51.529 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.527+0000 7fc755400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc7581039a0 0x7fc758198ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:19:51.529 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.527+0000 7fc755400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc7581039a0 0x7fc758198ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:37918/0 (socket says 172.21.15.153:37918) 2024-03-21T10:19:51.529 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.527+0000 7fc755400700 1 -- 172.21.15.153:0/4082994197 learned_addr learned my addr 172.21.15.153:0/4082994197 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:19:51.529 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.527+0000 7fc755e00700 1 --2- 172.21.15.153:0/4082994197 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc758102ff0 0x7fc7581989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:19:51.530 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.528+0000 7fc755400700 1 -- 172.21.15.153:0/4082994197 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc758102ff0 msgr2=0x7fc7581989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:19:51.530 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.528+0000 7fc755400700 1 --2- 172.21.15.153:0/4082994197 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc758102ff0 0x7fc7581989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:19:51.530 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.528+0000 7fc755400700 1 -- 172.21.15.153:0/4082994197 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7440095f0 con 0x7fc7581039a0 2024-03-21T10:19:51.530 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.528+0000 7fc755400700 1 --2- 172.21.15.153:0/4082994197 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc7581039a0 0x7fc758198ee0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc74400bda0 tx=0x7fc74400bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:19:51.531 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.528+0000 7fc74ea00700 1 -- 172.21.15.153:0/4082994197 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc74401d070 con 0x7fc7581039a0 2024-03-21T10:19:51.531 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.528+0000 7fc74ea00700 1 -- 172.21.15.153:0/4082994197 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc74400f460 con 0x7fc7581039a0 2024-03-21T10:19:51.531 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.528+0000 7fc74ea00700 1 -- 172.21.15.153:0/4082994197 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc744017630 con 0x7fc7581039a0 2024-03-21T10:19:51.531 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.528+0000 7fc75cc00700 1 -- 172.21.15.153:0/4082994197 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc75819d600 con 0x7fc7581039a0 2024-03-21T10:19:51.532 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.529+0000 7fc75cc00700 1 -- 172.21.15.153:0/4082994197 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc75819daf0 con 0x7fc7581039a0 2024-03-21T10:19:51.532 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.530+0000 7fc75cc00700 1 -- 172.21.15.153:0/4082994197 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc75810b3f0 con 0x7fc7581039a0 2024-03-21T10:19:51.532 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.531+0000 7fc74ea00700 1 -- 172.21.15.153:0/4082994197 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fc74400fab0 con 0x7fc7581039a0 2024-03-21T10:19:51.534 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.532+0000 7fc74ea00700 1 --2- 172.21.15.153:0/4082994197 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fc73c06e460 0x7fc73c070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:19:51.534 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.533+0000 7fc74ea00700 1 -- 172.21.15.153:0/4082994197 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fc74408f980 con 0x7fc7581039a0 2024-03-21T10:19:51.534 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.533+0000 7fc755e00700 1 --2- 172.21.15.153:0/4082994197 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fc73c06e460 0x7fc73c070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:19:51.535 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.534+0000 7fc755e00700 1 --2- 172.21.15.153:0/4082994197 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fc73c06e460 0x7fc73c070920 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc748005950 tx=0x7fc74800a300 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:19:51.538 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.537+0000 7fc74ea00700 1 -- 172.21.15.153:0/4082994197 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc74405c550 con 0x7fc7581039a0 2024-03-21T10:19:51.808 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.807+0000 7fc75cc00700 1 -- 172.21.15.153:0/4082994197 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fc75804ea90 con 0x7fc7581039a0 2024-03-21T10:19:51.809 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.808+0000 7fc74ea00700 1 -- 172.21.15.153:0/4082994197 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) v1 ==== 76+0+1874 (secure 0 0 0) 0x7fc74405c0e0 con 0x7fc7581039a0 2024-03-21T10:19:51.809 INFO:teuthology.orchestra.run.smithi153.stdout:e28 2024-03-21T10:19:51.809 INFO:teuthology.orchestra.run.smithi153.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-21T10:19:51.809 INFO:teuthology.orchestra.run.smithi153.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:19:51.809 INFO:teuthology.orchestra.run.smithi153.stdout:legacy client fscid: 1 2024-03-21T10:19:51.809 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:19:51.809 INFO:teuthology.orchestra.run.smithi153.stdout:Filesystem 'cephfs' (1) 2024-03-21T10:19:51.810 INFO:teuthology.orchestra.run.smithi153.stdout:fs_name cephfs 2024-03-21T10:19:51.810 INFO:teuthology.orchestra.run.smithi153.stdout:epoch 23 2024-03-21T10:19:51.810 INFO:teuthology.orchestra.run.smithi153.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-21T10:19:51.810 INFO:teuthology.orchestra.run.smithi153.stdout:created 2024-03-21T10:04:27.086576+0000 2024-03-21T10:19:51.810 INFO:teuthology.orchestra.run.smithi153.stdout:modified 2024-03-21T10:19:05.022092+0000 2024-03-21T10:19:51.810 INFO:teuthology.orchestra.run.smithi153.stdout:tableserver 0 2024-03-21T10:19:51.810 INFO:teuthology.orchestra.run.smithi153.stdout:root 0 2024-03-21T10:19:51.810 INFO:teuthology.orchestra.run.smithi153.stdout:session_timeout 60 2024-03-21T10:19:51.810 INFO:teuthology.orchestra.run.smithi153.stdout:session_autoclose 300 2024-03-21T10:19:51.810 INFO:teuthology.orchestra.run.smithi153.stdout:max_file_size 1099511627776 2024-03-21T10:19:51.810 INFO:teuthology.orchestra.run.smithi153.stdout:max_xattr_size 65536 2024-03-21T10:19:51.810 INFO:teuthology.orchestra.run.smithi153.stdout:required_client_features {} 2024-03-21T10:19:51.811 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure 0 2024-03-21T10:19:51.811 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure_osd_epoch 94 2024-03-21T10:19:51.811 INFO:teuthology.orchestra.run.smithi153.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:19:51.811 INFO:teuthology.orchestra.run.smithi153.stdout:max_mds 2 2024-03-21T10:19:51.811 INFO:teuthology.orchestra.run.smithi153.stdout:in 0,1 2024-03-21T10:19:51.811 INFO:teuthology.orchestra.run.smithi153.stdout:up {} 2024-03-21T10:19:51.811 INFO:teuthology.orchestra.run.smithi153.stdout:failed 0,1 2024-03-21T10:19:51.811 INFO:teuthology.orchestra.run.smithi153.stdout:damaged 2024-03-21T10:19:51.811 INFO:teuthology.orchestra.run.smithi153.stdout:stopped 2024-03-21T10:19:51.811 INFO:teuthology.orchestra.run.smithi153.stdout:data_pools [3] 2024-03-21T10:19:51.811 INFO:teuthology.orchestra.run.smithi153.stdout:metadata_pool 2 2024-03-21T10:19:51.811 INFO:teuthology.orchestra.run.smithi153.stdout:inline_data enabled 2024-03-21T10:19:51.811 INFO:teuthology.orchestra.run.smithi153.stdout:balancer 2024-03-21T10:19:51.812 INFO:teuthology.orchestra.run.smithi153.stdout:bal_rank_mask -1 2024-03-21T10:19:51.812 INFO:teuthology.orchestra.run.smithi153.stdout:standby_count_wanted 1 2024-03-21T10:19:51.812 INFO:teuthology.orchestra.run.smithi153.stdout:qdb_cluster leader: 0 members: 2024-03-21T10:19:51.812 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:19:51.812 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:19:51.812 INFO:teuthology.orchestra.run.smithi153.stdout:Standby daemons: 2024-03-21T10:19:51.812 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:19:51.812 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.zipckr{-1:34442} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6826/1103846295,v1:172.21.15.153:6827/1103846295] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:19:51.812 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.iuyluj{-1:34474} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6828/1738630832,v1:172.21.15.153:6829/1738630832] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:19:51.812 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.nhozag{-1:44269} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.174:6826/3967223227,v1:172.21.15.174:6827/3967223227] compat {c=[1],r=[1],i=[7ff]}] 2024-03-21T10:19:51.812 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.yyimst{-1:44277} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.174:6824/787559196,v1:172.21.15.174:6825/787559196] compat {c=[1],r=[1],i=[7ff]}] 2024-03-21T10:19:51.813 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.812+0000 7fc75cc00700 1 -- 172.21.15.153:0/4082994197 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fc73c06e460 msgr2=0x7fc73c070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:19:51.813 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.812+0000 7fc75cc00700 1 --2- 172.21.15.153:0/4082994197 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fc73c06e460 0x7fc73c070920 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc748005950 tx=0x7fc74800a300 comp rx=0 tx=0).stop 2024-03-21T10:19:51.813 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.812+0000 7fc75cc00700 1 -- 172.21.15.153:0/4082994197 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc7581039a0 msgr2=0x7fc758198ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:19:51.813 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.812+0000 7fc75cc00700 1 --2- 172.21.15.153:0/4082994197 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc7581039a0 0x7fc758198ee0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc74400bda0 tx=0x7fc74400bdd0 comp rx=0 tx=0).stop 2024-03-21T10:19:51.814 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.813+0000 7fc75cc00700 1 -- 172.21.15.153:0/4082994197 shutdown_connections 2024-03-21T10:19:51.814 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.813+0000 7fc75cc00700 1 --2- 172.21.15.153:0/4082994197 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc758102ff0 0x7fc7581989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:19:51.814 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.813+0000 7fc75cc00700 1 --2- 172.21.15.153:0/4082994197 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fc73c06e460 0x7fc73c070920 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:19:51.814 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.813+0000 7fc75cc00700 1 --2- 172.21.15.153:0/4082994197 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc7581039a0 0x7fc758198ee0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:19:51.814 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.813+0000 7fc75cc00700 1 -- 172.21.15.153:0/4082994197 >> 172.21.15.153:0/4082994197 conn(0x7fc7580fe880 msgr2=0x7fc758100c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:19:51.814 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.813+0000 7fc75cc00700 1 -- 172.21.15.153:0/4082994197 shutdown_connections 2024-03-21T10:19:51.814 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.813+0000 7fc75cc00700 1 -- 172.21.15.153:0/4082994197 wait complete. 2024-03-21T10:19:51.815 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 28 2024-03-21T10:19:51.971 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.970+0000 7fd498200700 1 -- 172.21.15.153:0/3320363505 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd490073130 msgr2=0x7fd490073510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:19:51.972 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.970+0000 7fd498200700 1 --2- 172.21.15.153:0/3320363505 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd490073130 0x7fd490073510 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd484009910 tx=0x7fd484009c20 comp rx=0 tx=0).stop 2024-03-21T10:19:51.972 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.970+0000 7fd498200700 1 -- 172.21.15.153:0/3320363505 shutdown_connections 2024-03-21T10:19:51.972 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.970+0000 7fd498200700 1 --2- 172.21.15.153:0/3320363505 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd490073a50 0x7fd490111610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:19:51.972 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.970+0000 7fd498200700 1 --2- 172.21.15.153:0/3320363505 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd490073130 0x7fd490073510 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:19:51.972 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.970+0000 7fd498200700 1 -- 172.21.15.153:0/3320363505 >> 172.21.15.153:0/3320363505 conn(0x7fd4900fc630 msgr2=0x7fd4900fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:19:51.972 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.971+0000 7fd498200700 1 -- 172.21.15.153:0/3320363505 shutdown_connections 2024-03-21T10:19:51.972 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.971+0000 7fd498200700 1 -- 172.21.15.153:0/3320363505 wait complete. 2024-03-21T10:19:51.973 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:51 smithi153 ceph-mon[113733]: from='client.34482 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:19:51.973 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:51 smithi153 ceph-mon[113733]: from='client.34486 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:19:51.973 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:51 smithi153 ceph-mon[113733]: pgmap v239: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:19:51.973 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:51 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/2476613570' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:19:51.973 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.972+0000 7fd498200700 1 Processor -- start 2024-03-21T10:19:51.973 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.972+0000 7fd498200700 1 -- start start 2024-03-21T10:19:51.974 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.973+0000 7fd498200700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd490073130 0x7fd49019cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:19:51.974 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.973+0000 7fd498200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd490073a50 0x7fd49019d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:19:51.974 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.973+0000 7fd498200700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd49019d9e0 con 0x7fd490073a50 2024-03-21T10:19:51.974 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.973+0000 7fd498200700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4901a1770 con 0x7fd490073130 2024-03-21T10:19:51.975 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.973+0000 7fd494e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd490073a50 0x7fd49019d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:19:51.975 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.973+0000 7fd494e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd490073a50 0x7fd49019d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:37946/0 (socket says 172.21.15.153:37946) 2024-03-21T10:19:51.975 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.973+0000 7fd494e00700 1 -- 172.21.15.153:0/979249693 learned_addr learned my addr 172.21.15.153:0/979249693 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:19:51.975 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.973+0000 7fd495800700 1 --2- 172.21.15.153:0/979249693 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd490073130 0x7fd49019cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:19:51.975 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.974+0000 7fd494e00700 1 -- 172.21.15.153:0/979249693 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd490073130 msgr2=0x7fd49019cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:19:51.975 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.974+0000 7fd494e00700 1 --2- 172.21.15.153:0/979249693 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd490073130 0x7fd49019cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:19:51.975 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.974+0000 7fd494e00700 1 -- 172.21.15.153:0/979249693 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4840095f0 con 0x7fd490073a50 2024-03-21T10:19:51.975 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.974+0000 7fd494e00700 1 --2- 172.21.15.153:0/979249693 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd490073a50 0x7fd49019d300 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd47c00b4b0 tx=0x7fd47c00b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:19:51.976 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.974+0000 7fd48e000700 1 -- 172.21.15.153:0/979249693 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd47c011800 con 0x7fd490073a50 2024-03-21T10:19:51.976 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.975+0000 7fd48e000700 1 -- 172.21.15.153:0/979249693 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd47c011e20 con 0x7fd490073a50 2024-03-21T10:19:51.976 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.975+0000 7fd48e000700 1 -- 172.21.15.153:0/979249693 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd47c00f2e0 con 0x7fd490073a50 2024-03-21T10:19:51.976 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.975+0000 7fd498200700 1 -- 172.21.15.153:0/979249693 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4901a1a50 con 0x7fd490073a50 2024-03-21T10:19:51.976 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.975+0000 7fd498200700 1 -- 172.21.15.153:0/979249693 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4901a1fa0 con 0x7fd490073a50 2024-03-21T10:19:51.977 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.976+0000 7fd498200700 1 -- 172.21.15.153:0/979249693 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd49010ed90 con 0x7fd490073a50 2024-03-21T10:19:51.978 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.977+0000 7fd48e000700 1 -- 172.21.15.153:0/979249693 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fd47c010380 con 0x7fd490073a50 2024-03-21T10:19:51.979 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.979+0000 7fd48e000700 1 --2- 172.21.15.153:0/979249693 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd47806e460 0x7fd478070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:19:51.980 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.979+0000 7fd48e000700 1 -- 172.21.15.153:0/979249693 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fd47c08dfb0 con 0x7fd490073a50 2024-03-21T10:19:51.980 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.979+0000 7fd495800700 1 --2- 172.21.15.153:0/979249693 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd47806e460 0x7fd478070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:19:51.981 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.980+0000 7fd495800700 1 --2- 172.21.15.153:0/979249693 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd47806e460 0x7fd478070920 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd484005f50 tx=0x7fd48401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:19:51.984 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:51.984+0000 7fd48e000700 1 -- 172.21.15.153:0/979249693 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd47c0c2740 con 0x7fd490073a50 2024-03-21T10:19:52.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:51 smithi174 ceph-mon[88004]: from='client.34482 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:19:52.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:51 smithi174 ceph-mon[88004]: from='client.34486 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:19:52.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:51 smithi174 ceph-mon[88004]: pgmap v239: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:19:52.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:51 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/2476613570' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:19:52.229 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:52.227+0000 7fd498200700 1 -- 172.21.15.153:0/979249693 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd4901a2280 con 0x7fd47806e460 2024-03-21T10:19:52.230 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:52.230+0000 7fd48e000700 1 -- 172.21.15.153:0/979249693 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fd4901a2280 con 0x7fd47806e460 2024-03-21T10:19:52.231 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:19:52.231 INFO:teuthology.orchestra.run.smithi153.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232", 2024-03-21T10:19:52.231 INFO:teuthology.orchestra.run.smithi153.stdout: "in_progress": true, 2024-03-21T10:19:52.231 INFO:teuthology.orchestra.run.smithi153.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-21T10:19:52.231 INFO:teuthology.orchestra.run.smithi153.stdout: "services_complete": [ 2024-03-21T10:19:52.231 INFO:teuthology.orchestra.run.smithi153.stdout: "crash", 2024-03-21T10:19:52.231 INFO:teuthology.orchestra.run.smithi153.stdout: "mon", 2024-03-21T10:19:52.231 INFO:teuthology.orchestra.run.smithi153.stdout: "osd", 2024-03-21T10:19:52.231 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr" 2024-03-21T10:19:52.231 INFO:teuthology.orchestra.run.smithi153.stdout: ], 2024-03-21T10:19:52.231 INFO:teuthology.orchestra.run.smithi153.stdout: "progress": "14/23 daemons upgraded", 2024-03-21T10:19:52.231 INFO:teuthology.orchestra.run.smithi153.stdout: "message": "Currently upgrading mds daemons", 2024-03-21T10:19:52.232 INFO:teuthology.orchestra.run.smithi153.stdout: "is_paused": false 2024-03-21T10:19:52.232 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:19:52.235 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:52.233+0000 7fd498200700 1 -- 172.21.15.153:0/979249693 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd47806e460 msgr2=0x7fd478070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:19:52.235 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:52.233+0000 7fd498200700 1 --2- 172.21.15.153:0/979249693 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd47806e460 0x7fd478070920 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd484005f50 tx=0x7fd48401a040 comp rx=0 tx=0).stop 2024-03-21T10:19:52.235 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:52.233+0000 7fd498200700 1 -- 172.21.15.153:0/979249693 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd490073a50 msgr2=0x7fd49019d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:19:52.235 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:52.233+0000 7fd498200700 1 --2- 172.21.15.153:0/979249693 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd490073a50 0x7fd49019d300 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd47c00b4b0 tx=0x7fd47c00b7c0 comp rx=0 tx=0).stop 2024-03-21T10:19:52.235 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:52.234+0000 7fd498200700 1 -- 172.21.15.153:0/979249693 shutdown_connections 2024-03-21T10:19:52.235 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:52.234+0000 7fd498200700 1 --2- 172.21.15.153:0/979249693 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd490073130 0x7fd49019cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:19:52.235 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:52.234+0000 7fd498200700 1 --2- 172.21.15.153:0/979249693 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd47806e460 0x7fd478070920 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:19:52.235 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:52.234+0000 7fd498200700 1 --2- 172.21.15.153:0/979249693 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd490073a50 0x7fd49019d300 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:19:52.235 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:52.234+0000 7fd498200700 1 -- 172.21.15.153:0/979249693 >> 172.21.15.153:0/979249693 conn(0x7fd4900fc630 msgr2=0x7fd490103120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:19:52.235 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:52.234+0000 7fd498200700 1 -- 172.21.15.153:0/979249693 shutdown_connections 2024-03-21T10:19:52.235 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:52.234+0000 7fd498200700 1 -- 172.21.15.153:0/979249693 wait complete. 2024-03-21T10:19:52.388 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:52.387+0000 7f4b40400700 1 -- 172.21.15.153:0/1635796425 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4b38068670 msgr2=0x7f4b38068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:19:52.389 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:52.387+0000 7f4b40400700 1 --2- 172.21.15.153:0/1635796425 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4b38068670 0x7f4b38068a50 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c009910 tx=0x7f4b2c009c20 comp rx=0 tx=0).stop 2024-03-21T10:19:52.389 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:52.387+0000 7f4b40400700 1 -- 172.21.15.153:0/1635796425 shutdown_connections 2024-03-21T10:19:52.389 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:52.387+0000 7f4b40400700 1 --2- 172.21.15.153:0/1635796425 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4b38069020 0x7f4b38105790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:19:52.389 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:52.387+0000 7f4b40400700 1 --2- 172.21.15.153:0/1635796425 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4b38068670 0x7f4b38068a50 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:19:52.389 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:52.387+0000 7f4b40400700 1 -- 172.21.15.153:0/1635796425 >> 172.21.15.153:0/1635796425 conn(0x7f4b380755a0 msgr2=0x7f4b380759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:19:52.389 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:52.387+0000 7f4b40400700 1 -- 172.21.15.153:0/1635796425 shutdown_connections 2024-03-21T10:19:52.389 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:52.387+0000 7f4b40400700 1 -- 172.21.15.153:0/1635796425 wait complete. 2024-03-21T10:19:52.390 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:52.388+0000 7f4b40400700 1 Processor -- start 2024-03-21T10:19:52.391 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:52.388+0000 7f4b40400700 1 -- start start 2024-03-21T10:19:52.391 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:52.389+0000 7f4b40400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4b38068670 0x7f4b3819e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:19:52.391 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:52.389+0000 7f4b40400700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4b38069020 0x7f4b3819ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:19:52.391 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:52.389+0000 7f4b40400700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b3819f370 con 0x7f4b38068670 2024-03-21T10:19:52.391 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:52.389+0000 7f4b40400700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b38198820 con 0x7f4b38069020 2024-03-21T10:19:52.391 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:52.389+0000 7f4b3da00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4b38068670 0x7f4b3819e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:19:52.391 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:52.389+0000 7f4b3da00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4b38068670 0x7f4b3819e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:37964/0 (socket says 172.21.15.153:37964) 2024-03-21T10:19:52.391 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:52.389+0000 7f4b3da00700 1 -- 172.21.15.153:0/432836690 learned_addr learned my addr 172.21.15.153:0/432836690 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:19:52.391 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:52.389+0000 7f4b3d000700 1 --2- 172.21.15.153:0/432836690 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4b38069020 0x7f4b3819ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:19:52.391 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:52.390+0000 7f4b3da00700 1 -- 172.21.15.153:0/432836690 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4b38069020 msgr2=0x7f4b3819ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:19:52.391 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:52.390+0000 7f4b3da00700 1 --2- 172.21.15.153:0/432836690 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4b38069020 0x7f4b3819ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:19:52.391 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:52.390+0000 7f4b3da00700 1 -- 172.21.15.153:0/432836690 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b2c0095f0 con 0x7f4b38068670 2024-03-21T10:19:52.392 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:52.390+0000 7f4b3d000700 1 --2- 172.21.15.153:0/432836690 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4b38069020 0x7f4b3819ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:19:52.392 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:52.390+0000 7f4b3da00700 1 --2- 172.21.15.153:0/432836690 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4b38068670 0x7f4b3819e7a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c0092e0 tx=0x7f4b2c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:19:52.392 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:52.390+0000 7f4b36000700 1 -- 172.21.15.153:0/432836690 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b2c01d070 con 0x7f4b38068670 2024-03-21T10:19:52.392 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:52.390+0000 7f4b36000700 1 -- 172.21.15.153:0/432836690 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4b2c005e00 con 0x7f4b38068670 2024-03-21T10:19:52.393 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:52.390+0000 7f4b36000700 1 -- 172.21.15.153:0/432836690 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b2c017400 con 0x7f4b38068670 2024-03-21T10:19:52.393 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:52.390+0000 7f4b40400700 1 -- 172.21.15.153:0/432836690 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b38198aa0 con 0x7f4b38068670 2024-03-21T10:19:52.393 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:52.391+0000 7f4b40400700 1 -- 172.21.15.153:0/432836690 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b38198f90 con 0x7f4b38068670 2024-03-21T10:19:52.393 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:52.392+0000 7f4b40400700 1 -- 172.21.15.153:0/432836690 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b3804ea90 con 0x7f4b38068670 2024-03-21T10:19:52.395 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:52.394+0000 7f4b36000700 1 -- 172.21.15.153:0/432836690 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f4b2c00f460 con 0x7f4b38068670 2024-03-21T10:19:52.396 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:52.396+0000 7f4b36000700 1 --2- 172.21.15.153:0/432836690 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f4b2006e460 0x7f4b20070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:19:52.397 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:52.396+0000 7f4b36000700 1 -- 172.21.15.153:0/432836690 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f4b2c08fc00 con 0x7f4b38068670 2024-03-21T10:19:52.397 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:52.396+0000 7f4b3d000700 1 --2- 172.21.15.153:0/432836690 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f4b2006e460 0x7f4b20070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:19:52.397 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:52.397+0000 7f4b3d000700 1 --2- 172.21.15.153:0/432836690 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f4b2006e460 0x7f4b20070920 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4b381076e0 tx=0x7f4b2400b2c0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:19:52.399 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:52.399+0000 7f4b36000700 1 -- 172.21.15.153:0/432836690 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4b2c05c850 con 0x7f4b38068670 2024-03-21T10:19:52.749 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:52.748+0000 7f4b40400700 1 -- 172.21.15.153:0/432836690 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f4b38199d30 con 0x7f4b38068670 2024-03-21T10:19:52.749 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:52.748+0000 7f4b36000700 1 -- 172.21.15.153:0/432836690 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f4b2c05c3e0 con 0x7f4b38068670 2024-03-21T10:19:52.750 INFO:teuthology.orchestra.run.smithi153.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-03-21T10:19:52.750 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-21T10:19:52.750 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is degraded 2024-03-21T10:19:52.750 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-21T10:19:52.750 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-21T10:19:52.750 INFO:teuthology.orchestra.run.smithi153.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-21T10:19:52.750 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-21T10:19:52.753 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:52.752+0000 7f4b40400700 1 -- 172.21.15.153:0/432836690 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f4b2006e460 msgr2=0x7f4b20070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:19:52.753 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:52.752+0000 7f4b40400700 1 --2- 172.21.15.153:0/432836690 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f4b2006e460 0x7f4b20070920 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4b381076e0 tx=0x7f4b2400b2c0 comp rx=0 tx=0).stop 2024-03-21T10:19:52.753 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:52.752+0000 7f4b40400700 1 -- 172.21.15.153:0/432836690 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4b38068670 msgr2=0x7f4b3819e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:19:52.753 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:52.752+0000 7f4b40400700 1 --2- 172.21.15.153:0/432836690 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4b38068670 0x7f4b3819e7a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c0092e0 tx=0x7f4b2c00be20 comp rx=0 tx=0).stop 2024-03-21T10:19:52.753 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:52.752+0000 7f4b40400700 1 -- 172.21.15.153:0/432836690 shutdown_connections 2024-03-21T10:19:52.753 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:52.752+0000 7f4b40400700 1 --2- 172.21.15.153:0/432836690 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f4b2006e460 0x7f4b20070920 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:19:52.753 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:52.752+0000 7f4b40400700 1 --2- 172.21.15.153:0/432836690 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4b38068670 0x7f4b3819e7a0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:19:52.753 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:52.752+0000 7f4b40400700 1 --2- 172.21.15.153:0/432836690 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4b38069020 0x7f4b3819ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:19:52.753 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:52.752+0000 7f4b40400700 1 -- 172.21.15.153:0/432836690 >> 172.21.15.153:0/432836690 conn(0x7f4b380755a0 msgr2=0x7f4b38102760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:19:52.753 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:52.752+0000 7f4b40400700 1 -- 172.21.15.153:0/432836690 shutdown_connections 2024-03-21T10:19:52.754 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:19:52.752+0000 7f4b40400700 1 -- 172.21.15.153:0/432836690 wait complete. 2024-03-21T10:19:52.885 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:52 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/4082994197' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:19:52.885 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:52 smithi153 ceph-mon[113733]: from='client.34498 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:19:52.885 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:52 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/432836690' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:19:53.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:52 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/4082994197' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:19:53.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:52 smithi174 ceph-mon[88004]: from='client.34498 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:19:53.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:52 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/432836690' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:19:54.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:53 smithi153 ceph-mon[113733]: pgmap v240: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:19:54.196 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:53 smithi174 ceph-mon[88004]: pgmap v240: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:19:55.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:55 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:19:55.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:55 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:19:55.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:55 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:19:55.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:55 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:19:55.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:55 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:19:55.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:55 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:19:55.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:55 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:19:55.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:55 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:19:55.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:55 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:19:55.308 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:55 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:19:55.402 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:55 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:19:55.402 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:55 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:19:55.403 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:55 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:19:55.403 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:55 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:19:55.403 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:55 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:19:55.403 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:55 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:19:55.403 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:55 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:19:55.403 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:55 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:19:55.403 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:55 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:19:55.403 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:55 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:19:56.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:56 smithi174 ceph-mon[88004]: pgmap v241: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:19:56.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:56 smithi153 ceph-mon[113733]: pgmap v241: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:19:57.788 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:57 smithi174 ceph-mon[88004]: Upgrade: Updating mds.cephfs.smithi174.yyimst 2024-03-21T10:19:57.788 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:57 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:19:57.788 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:57 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi174.yyimst", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T10:19:57.788 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:57 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:19:57.788 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:57 smithi174 ceph-mon[88004]: Deploying daemon mds.cephfs.smithi174.yyimst on smithi174 2024-03-21T10:19:57.788 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:57 smithi174 ceph-mon[88004]: pgmap v242: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:19:58.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:57 smithi153 ceph-mon[113733]: Upgrade: Updating mds.cephfs.smithi174.yyimst 2024-03-21T10:19:58.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:57 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:19:58.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:57 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi174.yyimst", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T10:19:58.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:57 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:19:58.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:57 smithi153 ceph-mon[113733]: Deploying daemon mds.cephfs.smithi174.yyimst on smithi174 2024-03-21T10:19:58.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:57 smithi153 ceph-mon[113733]: pgmap v242: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:19:59.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:19:59 smithi174 ceph-mon[88004]: pgmap v243: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:19:59.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:19:59 smithi153 ceph-mon[113733]: pgmap v243: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:20:00.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:20:00 smithi153 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-mon-smithi153[113729]: 2024-03-21T10:19:59.999+0000 7f2169e00700 -1 log_channel(cluster) log [ERR] : Health detail: HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-03-21T10:20:00.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:20:00 smithi153 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-mon-smithi153[113729]: 2024-03-21T10:20:00.000+0000 7f2169e00700 -1 log_channel(cluster) log [ERR] : [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-21T10:20:00.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:20:00 smithi153 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-mon-smithi153[113729]: 2024-03-21T10:20:00.000+0000 7f2169e00700 -1 log_channel(cluster) log [ERR] : fs cephfs is degraded 2024-03-21T10:20:00.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:20:00 smithi153 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-mon-smithi153[113729]: 2024-03-21T10:20:00.000+0000 7f2169e00700 -1 log_channel(cluster) log [ERR] : [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-21T10:20:00.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:20:00 smithi153 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-mon-smithi153[113729]: 2024-03-21T10:20:00.000+0000 7f2169e00700 -1 log_channel(cluster) log [ERR] : fs cephfs has deprecated feature inline_data enabled. 2024-03-21T10:20:00.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:20:00 smithi153 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-mon-smithi153[113729]: 2024-03-21T10:20:00.000+0000 7f2169e00700 -1 log_channel(cluster) log [ERR] : [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-21T10:20:00.308 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:20:00 smithi153 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-mon-smithi153[113729]: 2024-03-21T10:20:00.000+0000 7f2169e00700 -1 log_channel(cluster) log [ERR] : fs cephfs is offline because no MDS is active for it. 2024-03-21T10:20:00.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:20:00 smithi174 ceph-mon[88004]: Health detail: HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-03-21T10:20:00.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:20:00 smithi174 ceph-mon[88004]: [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-21T10:20:00.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:20:00 smithi174 ceph-mon[88004]: fs cephfs is degraded 2024-03-21T10:20:00.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:20:00 smithi174 ceph-mon[88004]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-21T10:20:00.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:20:00 smithi174 ceph-mon[88004]: fs cephfs has deprecated feature inline_data enabled. 2024-03-21T10:20:00.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:20:00 smithi174 ceph-mon[88004]: [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-21T10:20:00.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:20:00 smithi174 ceph-mon[88004]: fs cephfs is offline because no MDS is active for it. 2024-03-21T10:20:00.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:20:00 smithi153 ceph-mon[113733]: Health detail: HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-03-21T10:20:00.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:20:00 smithi153 ceph-mon[113733]: [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-21T10:20:00.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:20:00 smithi153 ceph-mon[113733]: fs cephfs is degraded 2024-03-21T10:20:00.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:20:00 smithi153 ceph-mon[113733]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-21T10:20:00.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:20:00 smithi153 ceph-mon[113733]: fs cephfs has deprecated feature inline_data enabled. 2024-03-21T10:20:00.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:20:00 smithi153 ceph-mon[113733]: [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-21T10:20:00.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:20:00 smithi153 ceph-mon[113733]: fs cephfs is offline because no MDS is active for it. 2024-03-21T10:20:01.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:20:01 smithi153 ceph-mon[113733]: osdmap e97: 6 total, 6 up, 6 in 2024-03-21T10:20:01.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:20:01 smithi153 ceph-mon[113733]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-03-21T10:20:01.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:20:01 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi174.yyimst"}]: dispatch 2024-03-21T10:20:01.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:20:01 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:20:01.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:20:01 smithi153 ceph-mon[113733]: pgmap v245: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:20:01.692 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:20:01 smithi174 ceph-mon[88004]: osdmap e97: 6 total, 6 up, 6 in 2024-03-21T10:20:01.692 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:20:01 smithi174 ceph-mon[88004]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-03-21T10:20:01.692 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:20:01 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi174.yyimst"}]: dispatch 2024-03-21T10:20:01.692 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:20:01 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:20:01.693 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:20:01 smithi174 ceph-mon[88004]: pgmap v245: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:20:03.558 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:20:03 smithi153 ceph-mon[113733]: pgmap v246: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:20:03.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:20:03 smithi174 ceph-mon[88004]: pgmap v246: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:20:04.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:20:04 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:20:04.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:20:04 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:20:04.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:20:04 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:20:04.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:20:04 smithi174 ceph-mon[88004]: mds.? [v2:172.21.15.174:6824/4127786797,v1:172.21.15.174:6825/4127786797] up:boot 2024-03-21T10:20:04.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:20:04 smithi174 ceph-mon[88004]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-03-21T10:20:04.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:20:04 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi174.yyimst"}]: dispatch 2024-03-21T10:20:05.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:20:04 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:20:05.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:20:04 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:20:05.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:20:04 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:20:05.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:20:04 smithi153 ceph-mon[113733]: mds.? [v2:172.21.15.174:6824/4127786797,v1:172.21.15.174:6825/4127786797] up:boot 2024-03-21T10:20:05.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:20:04 smithi153 ceph-mon[113733]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-03-21T10:20:05.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:20:04 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi174.yyimst"}]: dispatch 2024-03-21T10:20:05.842 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:20:05 smithi174 ceph-mon[88004]: pgmap v247: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:20:06.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:20:05 smithi153 ceph-mon[113733]: pgmap v247: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:20:07.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:20:07 smithi174 ceph-mon[88004]: pgmap v248: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:20:07.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:20:07 smithi153 ceph-mon[113733]: pgmap v248: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:20:09.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:20:08 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:20:09.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:20:08 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:20:09.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:20:08 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:20:09.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:20:08 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:20:10.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:20:09 smithi153 ceph-mon[113733]: pgmap v249: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:20:10.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:20:09 smithi174 ceph-mon[88004]: pgmap v249: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:20:11.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:20:10 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:20:11.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:20:10 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:20:11.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:20:10 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:20:11.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:20:10 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:20:12.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:20:11 smithi174 ceph-mon[88004]: pgmap v250: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:20:12.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:20:11 smithi153 ceph-mon[113733]: pgmap v250: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:20:13.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:20:13 smithi174 ceph-mon[88004]: pgmap v251: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:20:13.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:20:13 smithi153 ceph-mon[113733]: pgmap v251: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:20:15.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:20:15 smithi174 ceph-mon[88004]: pgmap v252: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:20:15.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:20:15 smithi153 ceph-mon[113733]: pgmap v252: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:20:16.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:20:16 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:20:16.755 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:20:16 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:20:17.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:20:17 smithi174 ceph-mon[88004]: pgmap v253: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:20:17.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:20:17 smithi153 ceph-mon[113733]: pgmap v253: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:20:19.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:20:19 smithi174 ceph-mon[88004]: pgmap v254: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:20:19.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:20:19 smithi153 ceph-mon[113733]: pgmap v254: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:20:21.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:20:21 smithi174 ceph-mon[88004]: pgmap v255: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:20:21.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:20:21 smithi153 ceph-mon[113733]: pgmap v255: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:20:22.934 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:22.932+0000 7f84d5000700 1 -- 172.21.15.153:0/843934707 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f84d0101770 msgr2=0x7f84d01057c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:20:22.935 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:22.932+0000 7f84d5000700 1 --2- 172.21.15.153:0/843934707 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f84d0101770 0x7f84d01057c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f84bc009910 tx=0x7f84bc009c20 comp rx=0 tx=0).stop 2024-03-21T10:20:22.935 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:22.932+0000 7f84d5000700 1 -- 172.21.15.153:0/843934707 shutdown_connections 2024-03-21T10:20:22.935 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:22.932+0000 7f84d5000700 1 --2- 172.21.15.153:0/843934707 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f84d0101770 0x7f84d01057c0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:22.935 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:22.932+0000 7f84d5000700 1 --2- 172.21.15.153:0/843934707 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f84d0100dc0 0x7f84d01011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:22.935 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:22.932+0000 7f84d5000700 1 -- 172.21.15.153:0/843934707 >> 172.21.15.153:0/843934707 conn(0x7f84d00fc650 msgr2=0x7f84d00fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:20:22.935 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:22.932+0000 7f84d5000700 1 -- 172.21.15.153:0/843934707 shutdown_connections 2024-03-21T10:20:22.935 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:22.932+0000 7f84d5000700 1 -- 172.21.15.153:0/843934707 wait complete. 2024-03-21T10:20:22.936 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:22.934+0000 7f84d5000700 1 Processor -- start 2024-03-21T10:20:22.936 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:22.934+0000 7f84d5000700 1 -- start start 2024-03-21T10:20:22.936 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:22.934+0000 7f84d5000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f84d0100dc0 0x7f84d0198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:20:22.936 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:22.934+0000 7f84d5000700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f84d0101770 0x7f84d0198eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:20:22.936 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:22.934+0000 7f84d5000700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84d0199540 con 0x7f84d0100dc0 2024-03-21T10:20:22.936 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:22.934+0000 7f84d5000700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84d019d350 con 0x7f84d0101770 2024-03-21T10:20:22.937 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:22.935+0000 7f84cde00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f84d0100dc0 0x7f84d0198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:20:22.937 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:22.935+0000 7f84cde00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f84d0100dc0 0x7f84d0198970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:51896/0 (socket says 172.21.15.153:51896) 2024-03-21T10:20:22.937 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:22.935+0000 7f84cde00700 1 -- 172.21.15.153:0/3041436761 learned_addr learned my addr 172.21.15.153:0/3041436761 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:20:22.938 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:22.935+0000 7f84cde00700 1 -- 172.21.15.153:0/3041436761 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f84d0101770 msgr2=0x7f84d0198eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:20:22.938 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:22.935+0000 7f84cde00700 1 --2- 172.21.15.153:0/3041436761 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f84d0101770 0x7f84d0198eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:22.938 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:22.935+0000 7f84cde00700 1 -- 172.21.15.153:0/3041436761 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84bc0095f0 con 0x7f84d0100dc0 2024-03-21T10:20:22.938 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:22.935+0000 7f84cde00700 1 --2- 172.21.15.153:0/3041436761 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f84d0100dc0 0x7f84d0198970 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f84c000c8a0 tx=0x7f84c000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:20:22.938 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:22.936+0000 7f84c6a00700 1 -- 172.21.15.153:0/3041436761 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f84c000cea0 con 0x7f84d0100dc0 2024-03-21T10:20:22.938 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:22.936+0000 7f84c6a00700 1 -- 172.21.15.153:0/3041436761 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f84c0004830 con 0x7f84d0100dc0 2024-03-21T10:20:22.938 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:22.936+0000 7f84d5000700 1 -- 172.21.15.153:0/3041436761 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84d019d630 con 0x7f84d0100dc0 2024-03-21T10:20:22.938 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:22.936+0000 7f84c6a00700 1 -- 172.21.15.153:0/3041436761 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f84c00055b0 con 0x7f84d0100dc0 2024-03-21T10:20:22.938 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:22.936+0000 7f84d5000700 1 -- 172.21.15.153:0/3041436761 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84d019db80 con 0x7f84d0100dc0 2024-03-21T10:20:22.939 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:22.937+0000 7f84d5000700 1 -- 172.21.15.153:0/3041436761 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84d0104f40 con 0x7f84d0100dc0 2024-03-21T10:20:22.940 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:22.938+0000 7f84c6a00700 1 -- 172.21.15.153:0/3041436761 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f84c00049a0 con 0x7f84d0100dc0 2024-03-21T10:20:22.941 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:22.939+0000 7f84c6a00700 1 --2- 172.21.15.153:0/3041436761 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f84b406e460 0x7f84b4070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:20:22.941 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:22.940+0000 7f84c6a00700 1 -- 172.21.15.153:0/3041436761 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f84c008e170 con 0x7f84d0100dc0 2024-03-21T10:20:22.941 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:22.940+0000 7f84cd400700 1 --2- 172.21.15.153:0/3041436761 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f84b406e460 0x7f84b4070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:20:22.942 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:22.940+0000 7f84cd400700 1 --2- 172.21.15.153:0/3041436761 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f84b406e460 0x7f84b4070920 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f84bc006010 tx=0x7f84bc00b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:20:22.947 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:22.945+0000 7f84c6a00700 1 -- 172.21.15.153:0/3041436761 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f84c005ad40 con 0x7f84d0100dc0 2024-03-21T10:20:23.189 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:23.186+0000 7f84d5000700 1 -- 172.21.15.153:0/3041436761 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f84d019dec0 con 0x7f84b406e460 2024-03-21T10:20:23.191 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:23.189+0000 7f84c6a00700 1 -- 172.21.15.153:0/3041436761 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f84d019dec0 con 0x7f84b406e460 2024-03-21T10:20:23.194 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:23.192+0000 7f84d5000700 1 -- 172.21.15.153:0/3041436761 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f84b406e460 msgr2=0x7f84b4070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:20:23.194 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:23.192+0000 7f84d5000700 1 --2- 172.21.15.153:0/3041436761 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f84b406e460 0x7f84b4070920 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f84bc006010 tx=0x7f84bc00b540 comp rx=0 tx=0).stop 2024-03-21T10:20:23.194 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:23.192+0000 7f84d5000700 1 -- 172.21.15.153:0/3041436761 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f84d0100dc0 msgr2=0x7f84d0198970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:20:23.194 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:23.192+0000 7f84d5000700 1 --2- 172.21.15.153:0/3041436761 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f84d0100dc0 0x7f84d0198970 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f84c000c8a0 tx=0x7f84c000cc60 comp rx=0 tx=0).stop 2024-03-21T10:20:23.194 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:23.192+0000 7f84d5000700 1 -- 172.21.15.153:0/3041436761 shutdown_connections 2024-03-21T10:20:23.194 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:23.192+0000 7f84d5000700 1 --2- 172.21.15.153:0/3041436761 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f84b406e460 0x7f84b4070920 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:23.194 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:23.192+0000 7f84d5000700 1 --2- 172.21.15.153:0/3041436761 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f84d0100dc0 0x7f84d0198970 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:23.194 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:23.192+0000 7f84d5000700 1 --2- 172.21.15.153:0/3041436761 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f84d0101770 0x7f84d0198eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:23.194 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:23.192+0000 7f84d5000700 1 -- 172.21.15.153:0/3041436761 >> 172.21.15.153:0/3041436761 conn(0x7f84d00fc650 msgr2=0x7f84d00fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:20:23.195 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:23.192+0000 7f84d5000700 1 -- 172.21.15.153:0/3041436761 shutdown_connections 2024-03-21T10:20:23.195 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:23.193+0000 7f84d5000700 1 -- 172.21.15.153:0/3041436761 wait complete. 2024-03-21T10:20:23.210 INFO:teuthology.orchestra.run.smithi153.stdout:true 2024-03-21T10:20:23.359 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:23.357+0000 7f8745c00700 1 -- 172.21.15.153:0/83806427 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8740101be0 msgr2=0x7f8740101fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:20:23.360 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:23.357+0000 7f8745c00700 1 --2- 172.21.15.153:0/83806427 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8740101be0 0x7f8740101fc0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f8730009910 tx=0x7f8730009c20 comp rx=0 tx=0).stop 2024-03-21T10:20:23.360 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:23.357+0000 7f8745c00700 1 -- 172.21.15.153:0/83806427 shutdown_connections 2024-03-21T10:20:23.360 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:23.357+0000 7f8745c00700 1 --2- 172.21.15.153:0/83806427 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8740102500 0x7f874010a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:23.360 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:23.357+0000 7f8745c00700 1 --2- 172.21.15.153:0/83806427 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8740101be0 0x7f8740101fc0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:23.360 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:23.357+0000 7f8745c00700 1 -- 172.21.15.153:0/83806427 >> 172.21.15.153:0/83806427 conn(0x7f87400fb450 msgr2=0x7f87400fd870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:20:23.360 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:23.357+0000 7f8745c00700 1 -- 172.21.15.153:0/83806427 shutdown_connections 2024-03-21T10:20:23.360 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:23.357+0000 7f8745c00700 1 -- 172.21.15.153:0/83806427 wait complete. 2024-03-21T10:20:23.360 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:23.359+0000 7f8745c00700 1 Processor -- start 2024-03-21T10:20:23.361 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:23.359+0000 7f8745c00700 1 -- start start 2024-03-21T10:20:23.361 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:23.359+0000 7f8745c00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8740101be0 0x7f8740196630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:20:23.362 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:23.360+0000 7f8745c00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8740102500 0x7f8740196b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:20:23.362 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:23.360+0000 7f8745c00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8740197250 con 0x7f8740102500 2024-03-21T10:20:23.362 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:23.360+0000 7f8745c00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f874019afe0 con 0x7f8740101be0 2024-03-21T10:20:23.362 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:23.360+0000 7f873f400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8740102500 0x7f8740196b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:20:23.362 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:23.360+0000 7f873f400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8740102500 0x7f8740196b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:51926/0 (socket says 172.21.15.153:51926) 2024-03-21T10:20:23.362 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:23.360+0000 7f873f400700 1 -- 172.21.15.153:0/971694147 learned_addr learned my addr 172.21.15.153:0/971694147 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:20:23.362 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:23.360+0000 7f873fe00700 1 --2- 172.21.15.153:0/971694147 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8740101be0 0x7f8740196630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:20:23.363 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:23.360+0000 7f873f400700 1 -- 172.21.15.153:0/971694147 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8740101be0 msgr2=0x7f8740196630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:20:23.363 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:23.360+0000 7f873f400700 1 --2- 172.21.15.153:0/971694147 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8740101be0 0x7f8740196630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:23.364 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:23.360+0000 7f873f400700 1 -- 172.21.15.153:0/971694147 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87300095f0 con 0x7f8740102500 2024-03-21T10:20:23.364 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:23.361+0000 7f873f400700 1 --2- 172.21.15.153:0/971694147 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8740102500 0x7f8740196b70 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f872c00d900 tx=0x7f872c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:20:23.364 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:23.361+0000 7f873cc00700 1 -- 172.21.15.153:0/971694147 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f872c009900 con 0x7f8740102500 2024-03-21T10:20:23.364 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:23.361+0000 7f873cc00700 1 -- 172.21.15.153:0/971694147 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f872c010460 con 0x7f8740102500 2024-03-21T10:20:23.364 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:23.361+0000 7f873cc00700 1 -- 172.21.15.153:0/971694147 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f872c00f5f0 con 0x7f8740102500 2024-03-21T10:20:23.364 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:23.361+0000 7f8745c00700 1 -- 172.21.15.153:0/971694147 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f874019b2c0 con 0x7f8740102500 2024-03-21T10:20:23.364 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:23.361+0000 7f8745c00700 1 -- 172.21.15.153:0/971694147 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f874019b810 con 0x7f8740102500 2024-03-21T10:20:23.364 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:23.362+0000 7f8745c00700 1 -- 172.21.15.153:0/971694147 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87401080f0 con 0x7f8740102500 2024-03-21T10:20:23.366 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:23.365+0000 7f873cc00700 1 -- 172.21.15.153:0/971694147 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f872c0105d0 con 0x7f8740102500 2024-03-21T10:20:23.367 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:23.365+0000 7f873cc00700 1 --2- 172.21.15.153:0/971694147 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f872406e460 0x7f8724070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:20:23.367 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:23.366+0000 7f873cc00700 1 -- 172.21.15.153:0/971694147 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f872c08e030 con 0x7f8740102500 2024-03-21T10:20:23.367 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:23.366+0000 7f873fe00700 1 --2- 172.21.15.153:0/971694147 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f872406e460 0x7f8724070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:20:23.368 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:23.367+0000 7f873fe00700 1 --2- 172.21.15.153:0/971694147 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f872406e460 0x7f8724070920 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f873000b5c0 tx=0x7f873001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:20:23.374 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:23.372+0000 7f873cc00700 1 -- 172.21.15.153:0/971694147 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f872c057170 con 0x7f8740102500 2024-03-21T10:20:23.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:20:23 smithi153 ceph-mon[113733]: from='client.34506 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:20:23.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:20:23 smithi153 ceph-mon[113733]: pgmap v256: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:20:23.624 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:23.622+0000 7f8745c00700 1 -- 172.21.15.153:0/971694147 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f874019bbb0 con 0x7f872406e460 2024-03-21T10:20:23.626 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:23.624+0000 7f873cc00700 1 -- 172.21.15.153:0/971694147 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f874019bbb0 con 0x7f872406e460 2024-03-21T10:20:23.629 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:23.627+0000 7f8745c00700 1 -- 172.21.15.153:0/971694147 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f872406e460 msgr2=0x7f8724070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:20:23.629 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:23.627+0000 7f8745c00700 1 --2- 172.21.15.153:0/971694147 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f872406e460 0x7f8724070920 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f873000b5c0 tx=0x7f873001a040 comp rx=0 tx=0).stop 2024-03-21T10:20:23.630 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:23.627+0000 7f8745c00700 1 -- 172.21.15.153:0/971694147 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8740102500 msgr2=0x7f8740196b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:20:23.630 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:23.627+0000 7f8745c00700 1 --2- 172.21.15.153:0/971694147 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8740102500 0x7f8740196b70 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f872c00d900 tx=0x7f872c00dcc0 comp rx=0 tx=0).stop 2024-03-21T10:20:23.630 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:23.627+0000 7f8745c00700 1 -- 172.21.15.153:0/971694147 shutdown_connections 2024-03-21T10:20:23.630 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:23.628+0000 7f8745c00700 1 --2- 172.21.15.153:0/971694147 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8740101be0 0x7f8740196630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:23.630 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:23.628+0000 7f8745c00700 1 --2- 172.21.15.153:0/971694147 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f872406e460 0x7f8724070920 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:23.630 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:23.628+0000 7f8745c00700 1 --2- 172.21.15.153:0/971694147 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8740102500 0x7f8740196b70 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:23.630 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:23.628+0000 7f8745c00700 1 -- 172.21.15.153:0/971694147 >> 172.21.15.153:0/971694147 conn(0x7f87400fb450 msgr2=0x7f8740105230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:20:23.630 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:23.628+0000 7f8745c00700 1 -- 172.21.15.153:0/971694147 shutdown_connections 2024-03-21T10:20:23.630 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:23.628+0000 7f8745c00700 1 -- 172.21.15.153:0/971694147 wait complete. 2024-03-21T10:20:23.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:20:23 smithi174 ceph-mon[88004]: from='client.34506 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:20:23.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:20:23 smithi174 ceph-mon[88004]: pgmap v256: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:20:23.788 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:23.786+0000 7f457d800700 1 -- 172.21.15.153:0/1391979181 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4578101d00 msgr2=0x7f45781020e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:20:23.788 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:23.786+0000 7f457d800700 1 --2- 172.21.15.153:0/1391979181 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4578101d00 0x7f45781020e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4568009910 tx=0x7f4568009c20 comp rx=0 tx=0).stop 2024-03-21T10:20:23.788 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:23.786+0000 7f457d800700 1 -- 172.21.15.153:0/1391979181 shutdown_connections 2024-03-21T10:20:23.788 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:23.786+0000 7f457d800700 1 --2- 172.21.15.153:0/1391979181 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4578102620 0x7f457810ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:23.788 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:23.786+0000 7f457d800700 1 --2- 172.21.15.153:0/1391979181 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4578101d00 0x7f45781020e0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:23.788 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:23.786+0000 7f457d800700 1 -- 172.21.15.153:0/1391979181 >> 172.21.15.153:0/1391979181 conn(0x7f45780fb570 msgr2=0x7f45780fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:20:23.788 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:23.786+0000 7f457d800700 1 -- 172.21.15.153:0/1391979181 shutdown_connections 2024-03-21T10:20:23.789 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:23.786+0000 7f457d800700 1 -- 172.21.15.153:0/1391979181 wait complete. 2024-03-21T10:20:23.789 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:23.787+0000 7f457d800700 1 Processor -- start 2024-03-21T10:20:23.790 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:23.787+0000 7f457d800700 1 -- start start 2024-03-21T10:20:23.790 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:23.788+0000 7f457d800700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4578101d00 0x7f457819c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:20:23.790 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:23.788+0000 7f457d800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4578102620 0x7f457819ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:20:23.790 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:23.788+0000 7f457d800700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f457819d100 con 0x7f4578102620 2024-03-21T10:20:23.790 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:23.788+0000 7f457d800700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45781965b0 con 0x7f4578101d00 2024-03-21T10:20:23.790 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:23.788+0000 7f4576000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4578102620 0x7f457819ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:20:23.790 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:23.788+0000 7f4576000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4578102620 0x7f457819ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:51948/0 (socket says 172.21.15.153:51948) 2024-03-21T10:20:23.790 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:23.788+0000 7f4576000700 1 -- 172.21.15.153:0/2178947602 learned_addr learned my addr 172.21.15.153:0/2178947602 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:20:23.791 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:23.788+0000 7f4576a00700 1 --2- 172.21.15.153:0/2178947602 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4578101d00 0x7f457819c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:20:23.791 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:23.789+0000 7f4576000700 1 -- 172.21.15.153:0/2178947602 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4578101d00 msgr2=0x7f457819c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:20:23.791 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:23.789+0000 7f4576000700 1 --2- 172.21.15.153:0/2178947602 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4578101d00 0x7f457819c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:23.791 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:23.789+0000 7f4576000700 1 -- 172.21.15.153:0/2178947602 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45680095f0 con 0x7f4578102620 2024-03-21T10:20:23.791 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:23.789+0000 7f4576a00700 1 --2- 172.21.15.153:0/2178947602 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4578101d00 0x7f457819c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:20:23.791 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:23.789+0000 7f4576000700 1 --2- 172.21.15.153:0/2178947602 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4578102620 0x7f457819ca70 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f456400c8f0 tx=0x7f456400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:20:23.792 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:23.789+0000 7f456f400700 1 -- 172.21.15.153:0/2178947602 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4564007ad0 con 0x7f4578102620 2024-03-21T10:20:23.792 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:23.789+0000 7f456f400700 1 -- 172.21.15.153:0/2178947602 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f456400ce90 con 0x7f4578102620 2024-03-21T10:20:23.792 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:23.789+0000 7f456f400700 1 -- 172.21.15.153:0/2178947602 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4564018630 con 0x7f4578102620 2024-03-21T10:20:23.792 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:23.789+0000 7f457d800700 1 -- 172.21.15.153:0/2178947602 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4578196890 con 0x7f4578102620 2024-03-21T10:20:23.792 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:23.789+0000 7f457d800700 1 -- 172.21.15.153:0/2178947602 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4578196de0 con 0x7f4578102620 2024-03-21T10:20:23.792 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:23.791+0000 7f457d800700 1 -- 172.21.15.153:0/2178947602 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4578108210 con 0x7f4578102620 2024-03-21T10:20:23.794 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:23.793+0000 7f456f400700 1 -- 172.21.15.153:0/2178947602 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f4564007c30 con 0x7f4578102620 2024-03-21T10:20:23.795 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:23.793+0000 7f456f400700 1 --2- 172.21.15.153:0/2178947602 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f455c06e460 0x7f455c070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:20:23.795 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:23.794+0000 7f456f400700 1 -- 172.21.15.153:0/2178947602 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f456408e5a0 con 0x7f4578102620 2024-03-21T10:20:23.795 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:23.794+0000 7f4576a00700 1 --2- 172.21.15.153:0/2178947602 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f455c06e460 0x7f455c070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:20:23.796 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:23.795+0000 7f4576a00700 1 --2- 172.21.15.153:0/2178947602 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f455c06e460 0x7f455c070920 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4568005100 tx=0x7f456801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:20:23.800 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:23.798+0000 7f456f400700 1 -- 172.21.15.153:0/2178947602 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f456405b0f0 con 0x7f4578102620 2024-03-21T10:20:24.026 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:24.024+0000 7f457d800700 1 -- 172.21.15.153:0/2178947602 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f4578197150 con 0x7f455c06e460 2024-03-21T10:20:24.038 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:24.036+0000 7f456f400700 1 -- 172.21.15.153:0/2178947602 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f4578197150 con 0x7f455c06e460 2024-03-21T10:20:24.039 INFO:teuthology.orchestra.run.smithi153.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-21T10:20:24.039 INFO:teuthology.orchestra.run.smithi153.stdout:alertmanager.smithi153 smithi153 *:9093,9094 running (9m) 37s ago 22m 14.7M - 0.25.0 c8568f914cd2 9ed3f2200b5f 2024-03-21T10:20:24.039 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi153 smithi153 running (23m) 37s ago 23m 9424k - 18.2.1 5be31c24972a 5b61994396a6 2024-03-21T10:20:24.039 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi174 smithi174 running (21m) 16s ago 21m 9244k - 18.2.1 5be31c24972a 4f7a5fc3ac28 2024-03-21T10:20:24.039 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi153 smithi153 running (5m) 37s ago 23m 6656k - 19.0.0-2231-g2edfee68 237e43c7a65f 32c21104d5b0 2024-03-21T10:20:24.039 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi174 smithi174 running (5m) 16s ago 21m 6660k - 19.0.0-2231-g2edfee68 237e43c7a65f a04e48967ab6 2024-03-21T10:20:24.039 INFO:teuthology.orchestra.run.smithi153.stdout:grafana.smithi153 smithi153 *:3000 running (8m) 37s ago 22m 73.6M - 9.4.12 2bacad6d85d8 ee24026ab786 2024-03-21T10:20:24.039 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.iuyluj smithi153 running (45s) 37s ago 15m 10.9M - 19.0.0-2231-g2edfee68 237e43c7a65f 87aff8a5de70 2024-03-21T10:20:24.039 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.zipckr smithi153 running (69s) 37s ago 15m 11.8M - 19.0.0-2231-g2edfee68 237e43c7a65f ccc262bd5dd3 2024-03-21T10:20:24.039 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.nhozag smithi174 running (15m) 16s ago 15m 12.0M - 18.2.1 5be31c24972a 2fee4000308f 2024-03-21T10:20:24.039 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.yyimst smithi174 running (21s) 16s ago 15m 10.7M - 19.0.0-2231-g2edfee68 237e43c7a65f fc1962f9351e 2024-03-21T10:20:24.039 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi153.tluxni smithi153 *:8443,9283,8765 running (12m) 37s ago 24m 509M - 19.0.0-2231-g2edfee68 237e43c7a65f 7ffee0e55382 2024-03-21T10:20:24.040 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi174.qpcwat smithi174 *:8443,9283,8765 running (10m) 16s ago 21m 452M - 19.0.0-2231-g2edfee68 237e43c7a65f 748968f216ab 2024-03-21T10:20:24.040 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi153 smithi153 running (7m) 37s ago 25m 55.0M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f dd8038303dc3 2024-03-21T10:20:24.040 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi174 smithi174 running (6m) 16s ago 21m 44.2M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f 29d78a788f2c 2024-03-21T10:20:24.040 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi153 smithi153 *:9100 running (10m) 37s ago 23m 7889k - 1.5.0 0da6a335fe13 bb835edc0e0b 2024-03-21T10:20:24.040 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi174 smithi174 *:9100 running (10m) 16s ago 21m 8102k - 1.5.0 0da6a335fe13 7c4ecad790fa 2024-03-21T10:20:24.040 INFO:teuthology.orchestra.run.smithi153.stdout:osd.0 smithi153 running (4m) 37s ago 20m 116M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 507900fd4791 2024-03-21T10:20:24.040 INFO:teuthology.orchestra.run.smithi153.stdout:osd.1 smithi153 running (4m) 37s ago 19m 89.2M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 0d646452e9c1 2024-03-21T10:20:24.040 INFO:teuthology.orchestra.run.smithi153.stdout:osd.2 smithi153 running (3m) 37s ago 19m 87.4M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 1326f2afdfcd 2024-03-21T10:20:24.040 INFO:teuthology.orchestra.run.smithi153.stdout:osd.3 smithi174 running (2m) 16s ago 18m 83.6M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 70aafac71ad5 2024-03-21T10:20:24.040 INFO:teuthology.orchestra.run.smithi153.stdout:osd.4 smithi174 running (2m) 16s ago 18m 92.2M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 0d746ce6e79d 2024-03-21T10:20:24.040 INFO:teuthology.orchestra.run.smithi153.stdout:osd.5 smithi174 running (95s) 16s ago 17m 86.2M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 53e5f35b8cac 2024-03-21T10:20:24.040 INFO:teuthology.orchestra.run.smithi153.stdout:prometheus.smithi153 smithi153 *:9095 running (9m) 37s ago 22m 73.5M - 2.43.0 a07b618ecd1d 7d95561e50c2 2024-03-21T10:20:24.041 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:24.038+0000 7f457d800700 1 -- 172.21.15.153:0/2178947602 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f455c06e460 msgr2=0x7f455c070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:20:24.041 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:24.038+0000 7f457d800700 1 --2- 172.21.15.153:0/2178947602 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f455c06e460 0x7f455c070920 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4568005100 tx=0x7f456801a040 comp rx=0 tx=0).stop 2024-03-21T10:20:24.041 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:24.039+0000 7f457d800700 1 -- 172.21.15.153:0/2178947602 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4578102620 msgr2=0x7f457819ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:20:24.041 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:24.039+0000 7f457d800700 1 --2- 172.21.15.153:0/2178947602 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4578102620 0x7f457819ca70 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f456400c8f0 tx=0x7f456400ccb0 comp rx=0 tx=0).stop 2024-03-21T10:20:24.041 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:24.039+0000 7f457d800700 1 -- 172.21.15.153:0/2178947602 shutdown_connections 2024-03-21T10:20:24.042 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:24.039+0000 7f457d800700 1 --2- 172.21.15.153:0/2178947602 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4578101d00 0x7f457819c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:24.042 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:24.039+0000 7f457d800700 1 --2- 172.21.15.153:0/2178947602 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f455c06e460 0x7f455c070920 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:24.042 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:24.039+0000 7f457d800700 1 --2- 172.21.15.153:0/2178947602 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4578102620 0x7f457819ca70 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:24.042 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:24.039+0000 7f457d800700 1 -- 172.21.15.153:0/2178947602 >> 172.21.15.153:0/2178947602 conn(0x7f45780fb570 msgr2=0x7f4578105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:20:24.042 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:24.039+0000 7f457d800700 1 -- 172.21.15.153:0/2178947602 shutdown_connections 2024-03-21T10:20:24.042 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:24.039+0000 7f457d800700 1 -- 172.21.15.153:0/2178947602 wait complete. 2024-03-21T10:20:24.193 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:24.191+0000 7fc782400700 1 -- 172.21.15.153:0/849735524 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc77c0ffbd0 msgr2=0x7fc77c100050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:20:24.193 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:24.191+0000 7fc782400700 1 --2- 172.21.15.153:0/849735524 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc77c0ffbd0 0x7fc77c100050 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc768009910 tx=0x7fc768009c20 comp rx=0 tx=0).stop 2024-03-21T10:20:24.193 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:24.191+0000 7fc782400700 1 -- 172.21.15.153:0/849735524 shutdown_connections 2024-03-21T10:20:24.193 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:24.191+0000 7fc782400700 1 --2- 172.21.15.153:0/849735524 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc77c0ffbd0 0x7fc77c100050 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:24.193 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:24.191+0000 7fc782400700 1 --2- 172.21.15.153:0/849735524 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc77c105a40 0x7fc77c105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:24.193 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:24.191+0000 7fc782400700 1 -- 172.21.15.153:0/849735524 >> 172.21.15.153:0/849735524 conn(0x7fc77c0755a0 msgr2=0x7fc77c0759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:20:24.193 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:24.191+0000 7fc782400700 1 -- 172.21.15.153:0/849735524 shutdown_connections 2024-03-21T10:20:24.193 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:24.191+0000 7fc782400700 1 -- 172.21.15.153:0/849735524 wait complete. 2024-03-21T10:20:24.194 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:24.192+0000 7fc782400700 1 Processor -- start 2024-03-21T10:20:24.194 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:24.192+0000 7fc782400700 1 -- start start 2024-03-21T10:20:24.195 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:24.193+0000 7fc782400700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc77c0ffbd0 0x7fc77c19c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:20:24.195 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:24.193+0000 7fc782400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc77c105a40 0x7fc77c19ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:20:24.195 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:24.193+0000 7fc782400700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc77c19d0d0 con 0x7fc77c105a40 2024-03-21T10:20:24.195 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:24.193+0000 7fc782400700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc77c196580 con 0x7fc77c0ffbd0 2024-03-21T10:20:24.195 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:24.193+0000 7fc77aa00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc77c105a40 0x7fc77c19ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:20:24.195 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:24.193+0000 7fc77aa00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc77c105a40 0x7fc77c19ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:51966/0 (socket says 172.21.15.153:51966) 2024-03-21T10:20:24.195 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:24.193+0000 7fc77aa00700 1 -- 172.21.15.153:0/1892737862 learned_addr learned my addr 172.21.15.153:0/1892737862 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:20:24.196 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:24.193+0000 7fc77b400700 1 --2- 172.21.15.153:0/1892737862 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc77c0ffbd0 0x7fc77c19c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:20:24.196 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:24.193+0000 7fc77aa00700 1 -- 172.21.15.153:0/1892737862 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc77c0ffbd0 msgr2=0x7fc77c19c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:20:24.196 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:24.193+0000 7fc77aa00700 1 --2- 172.21.15.153:0/1892737862 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc77c0ffbd0 0x7fc77c19c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:24.196 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:24.193+0000 7fc77aa00700 1 -- 172.21.15.153:0/1892737862 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7680095f0 con 0x7fc77c105a40 2024-03-21T10:20:24.196 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:24.194+0000 7fc77aa00700 1 --2- 172.21.15.153:0/1892737862 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc77c105a40 0x7fc77c19ca40 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc76800b5c0 tx=0x7fc76800be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:20:24.196 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:24.194+0000 7fc773e00700 1 -- 172.21.15.153:0/1892737862 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc76801d070 con 0x7fc77c105a40 2024-03-21T10:20:24.196 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:24.194+0000 7fc773e00700 1 -- 172.21.15.153:0/1892737862 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc76800f460 con 0x7fc77c105a40 2024-03-21T10:20:24.196 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:24.194+0000 7fc782400700 1 -- 172.21.15.153:0/1892737862 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc77c196800 con 0x7fc77c105a40 2024-03-21T10:20:24.196 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:24.194+0000 7fc773e00700 1 -- 172.21.15.153:0/1892737862 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc768017610 con 0x7fc77c105a40 2024-03-21T10:20:24.197 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:24.194+0000 7fc782400700 1 -- 172.21.15.153:0/1892737862 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc77c196cf0 con 0x7fc77c105a40 2024-03-21T10:20:24.197 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:24.195+0000 7fc782400700 1 -- 172.21.15.153:0/1892737862 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc77c108fe0 con 0x7fc77c105a40 2024-03-21T10:20:24.198 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:24.196+0000 7fc773e00700 1 -- 172.21.15.153:0/1892737862 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fc76800f5d0 con 0x7fc77c105a40 2024-03-21T10:20:24.200 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:24.198+0000 7fc773e00700 1 --2- 172.21.15.153:0/1892737862 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fc76006e460 0x7fc760070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:20:24.200 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:24.198+0000 7fc773e00700 1 -- 172.21.15.153:0/1892737862 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fc76808fad0 con 0x7fc77c105a40 2024-03-21T10:20:24.200 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:24.198+0000 7fc77b400700 1 --2- 172.21.15.153:0/1892737862 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fc76006e460 0x7fc760070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:20:24.200 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:24.199+0000 7fc77b400700 1 --2- 172.21.15.153:0/1892737862 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fc76006e460 0x7fc760070920 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc76c005fd0 tx=0x7fc76c00a560 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:20:24.203 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:24.202+0000 7fc773e00700 1 -- 172.21.15.153:0/1892737862 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc76805c6a0 con 0x7fc77c105a40 2024-03-21T10:20:24.516 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:24.514+0000 7fc782400700 1 -- 172.21.15.153:0/1892737862 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fc77c04ea90 con 0x7fc77c105a40 2024-03-21T10:20:24.516 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:24.515+0000 7fc773e00700 1 -- 172.21.15.153:0/1892737862 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+805 (secure 0 0 0) 0x7fc76805c230 con 0x7fc77c105a40 2024-03-21T10:20:24.516 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:20:24.517 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": { 2024-03-21T10:20:24.517 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:20:24.517 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:20:24.517 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": { 2024-03-21T10:20:24.517 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:20:24.517 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:20:24.517 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": { 2024-03-21T10:20:24.517 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 6 2024-03-21T10:20:24.517 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:20:24.517 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": { 2024-03-21T10:20:24.517 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-03-21T10:20:24.517 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 3 2024-03-21T10:20:24.517 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:20:24.518 INFO:teuthology.orchestra.run.smithi153.stdout: "overall": { 2024-03-21T10:20:24.518 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-03-21T10:20:24.518 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 13 2024-03-21T10:20:24.518 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-03-21T10:20:24.518 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:20:24.519 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:24.517+0000 7fc782400700 1 -- 172.21.15.153:0/1892737862 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fc76006e460 msgr2=0x7fc760070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:20:24.519 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:24.517+0000 7fc782400700 1 --2- 172.21.15.153:0/1892737862 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fc76006e460 0x7fc760070920 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc76c005fd0 tx=0x7fc76c00a560 comp rx=0 tx=0).stop 2024-03-21T10:20:24.519 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:24.517+0000 7fc782400700 1 -- 172.21.15.153:0/1892737862 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc77c105a40 msgr2=0x7fc77c19ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:20:24.519 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:24.517+0000 7fc782400700 1 --2- 172.21.15.153:0/1892737862 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc77c105a40 0x7fc77c19ca40 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc76800b5c0 tx=0x7fc76800be00 comp rx=0 tx=0).stop 2024-03-21T10:20:24.520 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:24.517+0000 7fc782400700 1 -- 172.21.15.153:0/1892737862 shutdown_connections 2024-03-21T10:20:24.520 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:24.517+0000 7fc782400700 1 --2- 172.21.15.153:0/1892737862 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc77c0ffbd0 0x7fc77c19c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:24.520 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:24.517+0000 7fc782400700 1 --2- 172.21.15.153:0/1892737862 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fc76006e460 0x7fc760070920 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:24.520 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:24.518+0000 7fc782400700 1 --2- 172.21.15.153:0/1892737862 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc77c105a40 0x7fc77c19ca40 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:24.520 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:24.518+0000 7fc782400700 1 -- 172.21.15.153:0/1892737862 >> 172.21.15.153:0/1892737862 conn(0x7fc77c0755a0 msgr2=0x7fc77c068650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:20:24.520 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:24.518+0000 7fc782400700 1 -- 172.21.15.153:0/1892737862 shutdown_connections 2024-03-21T10:20:24.520 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:24.518+0000 7fc782400700 1 -- 172.21.15.153:0/1892737862 wait complete. 2024-03-21T10:20:24.673 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:24.671+0000 7f1935a00700 1 -- 172.21.15.153:0/611665534 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1930068670 msgr2=0x7f1930068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:20:24.673 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:24.671+0000 7f1935a00700 1 --2- 172.21.15.153:0/611665534 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1930068670 0x7f1930068a50 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f1920009910 tx=0x7f1920009c20 comp rx=0 tx=0).stop 2024-03-21T10:20:24.673 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:24.671+0000 7f1935a00700 1 -- 172.21.15.153:0/611665534 shutdown_connections 2024-03-21T10:20:24.674 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:24.671+0000 7f1935a00700 1 --2- 172.21.15.153:0/611665534 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1930069020 0x7f1930105790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:24.674 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:20:24 smithi153 ceph-mon[113733]: from='client.34510 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:20:24.674 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:20:24 smithi153 ceph-mon[113733]: from='client.34514 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:20:24.675 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:24.671+0000 7f1935a00700 1 --2- 172.21.15.153:0/611665534 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1930068670 0x7f1930068a50 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:24.675 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:24.671+0000 7f1935a00700 1 -- 172.21.15.153:0/611665534 >> 172.21.15.153:0/611665534 conn(0x7f19300755a0 msgr2=0x7f19300759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:20:24.675 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:24.672+0000 7f1935a00700 1 -- 172.21.15.153:0/611665534 shutdown_connections 2024-03-21T10:20:24.675 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:24.672+0000 7f1935a00700 1 -- 172.21.15.153:0/611665534 wait complete. 2024-03-21T10:20:24.675 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:24.673+0000 7f1935a00700 1 Processor -- start 2024-03-21T10:20:24.675 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:24.673+0000 7f1935a00700 1 -- start start 2024-03-21T10:20:24.676 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:24.674+0000 7f1935a00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1930068670 0x7f19300ffca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:20:24.676 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:24.674+0000 7f1935a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1930069020 0x7f19301001e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:20:24.676 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:24.674+0000 7f1935a00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1930100720 con 0x7f1930069020 2024-03-21T10:20:24.676 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:24.674+0000 7f1935a00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1930100860 con 0x7f1930068670 2024-03-21T10:20:24.676 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:24.674+0000 7f192e000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1930069020 0x7f19301001e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:20:24.676 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:24.674+0000 7f192e000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1930069020 0x7f19301001e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:51984/0 (socket says 172.21.15.153:51984) 2024-03-21T10:20:24.676 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:24.674+0000 7f192e000700 1 -- 172.21.15.153:0/1298003112 learned_addr learned my addr 172.21.15.153:0/1298003112 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:20:24.677 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:24.674+0000 7f192ea00700 1 --2- 172.21.15.153:0/1298003112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1930068670 0x7f19300ffca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:20:24.677 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:24.675+0000 7f192e000700 1 -- 172.21.15.153:0/1298003112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1930068670 msgr2=0x7f19300ffca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:20:24.677 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:24.675+0000 7f192e000700 1 --2- 172.21.15.153:0/1298003112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1930068670 0x7f19300ffca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:24.678 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:24.675+0000 7f192e000700 1 -- 172.21.15.153:0/1298003112 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19200095f0 con 0x7f1930069020 2024-03-21T10:20:24.678 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:24.675+0000 7f192e000700 1 --2- 172.21.15.153:0/1298003112 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1930069020 0x7f19301001e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f191c00c8f0 tx=0x7f191c00cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:20:24.678 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:24.676+0000 7f1927400700 1 -- 172.21.15.153:0/1298003112 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f191c0078a0 con 0x7f1930069020 2024-03-21T10:20:24.678 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:24.676+0000 7f1935a00700 1 -- 172.21.15.153:0/1298003112 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19301a2910 con 0x7f1930069020 2024-03-21T10:20:24.678 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:24.676+0000 7f1927400700 1 -- 172.21.15.153:0/1298003112 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f191c00f450 con 0x7f1930069020 2024-03-21T10:20:24.678 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:24.676+0000 7f1935a00700 1 -- 172.21.15.153:0/1298003112 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19301a2d20 con 0x7f1930069020 2024-03-21T10:20:24.678 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:24.676+0000 7f1927400700 1 -- 172.21.15.153:0/1298003112 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f191c018610 con 0x7f1930069020 2024-03-21T10:20:24.679 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:24.677+0000 7f1935a00700 1 -- 172.21.15.153:0/1298003112 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1930109130 con 0x7f1930069020 2024-03-21T10:20:24.680 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:24.678+0000 7f1927400700 1 -- 172.21.15.153:0/1298003112 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f191c00f5c0 con 0x7f1930069020 2024-03-21T10:20:24.681 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:24.679+0000 7f1927400700 1 --2- 172.21.15.153:0/1298003112 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f191406e460 0x7f1914070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:20:24.681 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:24.679+0000 7f1927400700 1 -- 172.21.15.153:0/1298003112 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f191c08e4f0 con 0x7f1930069020 2024-03-21T10:20:24.681 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:24.679+0000 7f192ea00700 1 --2- 172.21.15.153:0/1298003112 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f191406e460 0x7f1914070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:20:24.682 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:24.680+0000 7f192ea00700 1 --2- 172.21.15.153:0/1298003112 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f191406e460 0x7f1914070920 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1920006010 tx=0x7f192001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:20:24.685 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:24.684+0000 7f1927400700 1 -- 172.21.15.153:0/1298003112 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f191c057510 con 0x7f1930069020 2024-03-21T10:20:24.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:20:24 smithi174 ceph-mon[88004]: from='client.34510 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:20:24.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:20:24 smithi174 ceph-mon[88004]: from='client.34514 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:20:24.965 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:24.962+0000 7f1935a00700 1 -- 172.21.15.153:0/1298003112 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f1930100a20 con 0x7f1930069020 2024-03-21T10:20:24.965 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:24.963+0000 7f1927400700 1 -- 172.21.15.153:0/1298003112 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1876 (secure 0 0 0) 0x7f191c05abd0 con 0x7f1930069020 2024-03-21T10:20:24.965 INFO:teuthology.orchestra.run.smithi153.stdout:e30 2024-03-21T10:20:24.965 INFO:teuthology.orchestra.run.smithi153.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-21T10:20:24.965 INFO:teuthology.orchestra.run.smithi153.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:20:24.965 INFO:teuthology.orchestra.run.smithi153.stdout:legacy client fscid: 1 2024-03-21T10:20:24.965 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:20:24.966 INFO:teuthology.orchestra.run.smithi153.stdout:Filesystem 'cephfs' (1) 2024-03-21T10:20:24.966 INFO:teuthology.orchestra.run.smithi153.stdout:fs_name cephfs 2024-03-21T10:20:24.966 INFO:teuthology.orchestra.run.smithi153.stdout:epoch 23 2024-03-21T10:20:24.966 INFO:teuthology.orchestra.run.smithi153.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-21T10:20:24.966 INFO:teuthology.orchestra.run.smithi153.stdout:created 2024-03-21T10:04:27.086576+0000 2024-03-21T10:20:24.966 INFO:teuthology.orchestra.run.smithi153.stdout:modified 2024-03-21T10:19:05.022092+0000 2024-03-21T10:20:24.966 INFO:teuthology.orchestra.run.smithi153.stdout:tableserver 0 2024-03-21T10:20:24.966 INFO:teuthology.orchestra.run.smithi153.stdout:root 0 2024-03-21T10:20:24.966 INFO:teuthology.orchestra.run.smithi153.stdout:session_timeout 60 2024-03-21T10:20:24.966 INFO:teuthology.orchestra.run.smithi153.stdout:session_autoclose 300 2024-03-21T10:20:24.966 INFO:teuthology.orchestra.run.smithi153.stdout:max_file_size 1099511627776 2024-03-21T10:20:24.966 INFO:teuthology.orchestra.run.smithi153.stdout:max_xattr_size 65536 2024-03-21T10:20:24.966 INFO:teuthology.orchestra.run.smithi153.stdout:required_client_features {} 2024-03-21T10:20:24.967 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure 0 2024-03-21T10:20:24.967 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure_osd_epoch 94 2024-03-21T10:20:24.967 INFO:teuthology.orchestra.run.smithi153.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:20:24.967 INFO:teuthology.orchestra.run.smithi153.stdout:max_mds 2 2024-03-21T10:20:24.967 INFO:teuthology.orchestra.run.smithi153.stdout:in 0,1 2024-03-21T10:20:24.967 INFO:teuthology.orchestra.run.smithi153.stdout:up {} 2024-03-21T10:20:24.967 INFO:teuthology.orchestra.run.smithi153.stdout:failed 0,1 2024-03-21T10:20:24.967 INFO:teuthology.orchestra.run.smithi153.stdout:damaged 2024-03-21T10:20:24.967 INFO:teuthology.orchestra.run.smithi153.stdout:stopped 2024-03-21T10:20:24.967 INFO:teuthology.orchestra.run.smithi153.stdout:data_pools [3] 2024-03-21T10:20:24.967 INFO:teuthology.orchestra.run.smithi153.stdout:metadata_pool 2 2024-03-21T10:20:24.967 INFO:teuthology.orchestra.run.smithi153.stdout:inline_data enabled 2024-03-21T10:20:24.967 INFO:teuthology.orchestra.run.smithi153.stdout:balancer 2024-03-21T10:20:24.967 INFO:teuthology.orchestra.run.smithi153.stdout:bal_rank_mask -1 2024-03-21T10:20:24.967 INFO:teuthology.orchestra.run.smithi153.stdout:standby_count_wanted 1 2024-03-21T10:20:24.968 INFO:teuthology.orchestra.run.smithi153.stdout:qdb_cluster leader: 0 members: 2024-03-21T10:20:24.968 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:20:24.968 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:20:24.968 INFO:teuthology.orchestra.run.smithi153.stdout:Standby daemons: 2024-03-21T10:20:24.968 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:20:24.968 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.zipckr{-1:34442} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6826/1103846295,v1:172.21.15.153:6827/1103846295] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:20:24.968 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.iuyluj{-1:34474} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6828/1738630832,v1:172.21.15.153:6829/1738630832] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:20:24.968 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.nhozag{-1:44269} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.174:6826/3967223227,v1:172.21.15.174:6827/3967223227] compat {c=[1],r=[1],i=[7ff]}] 2024-03-21T10:20:24.968 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.yyimst{-1:44313} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.174:6824/4127786797,v1:172.21.15.174:6825/4127786797] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:20:24.969 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:24.966+0000 7f1935a00700 1 -- 172.21.15.153:0/1298003112 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f191406e460 msgr2=0x7f1914070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:20:24.969 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:24.966+0000 7f1935a00700 1 --2- 172.21.15.153:0/1298003112 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f191406e460 0x7f1914070920 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1920006010 tx=0x7f192001a040 comp rx=0 tx=0).stop 2024-03-21T10:20:24.969 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:24.966+0000 7f1935a00700 1 -- 172.21.15.153:0/1298003112 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1930069020 msgr2=0x7f19301001e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:20:24.969 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:24.966+0000 7f1935a00700 1 --2- 172.21.15.153:0/1298003112 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1930069020 0x7f19301001e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f191c00c8f0 tx=0x7f191c00cc00 comp rx=0 tx=0).stop 2024-03-21T10:20:24.969 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:24.966+0000 7f1935a00700 1 -- 172.21.15.153:0/1298003112 shutdown_connections 2024-03-21T10:20:24.969 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:24.966+0000 7f1935a00700 1 --2- 172.21.15.153:0/1298003112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1930068670 0x7f19300ffca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:24.969 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:24.966+0000 7f1935a00700 1 --2- 172.21.15.153:0/1298003112 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f191406e460 0x7f1914070920 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:24.969 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:24.966+0000 7f1935a00700 1 --2- 172.21.15.153:0/1298003112 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1930069020 0x7f19301001e0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:24.969 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:24.966+0000 7f1935a00700 1 -- 172.21.15.153:0/1298003112 >> 172.21.15.153:0/1298003112 conn(0x7f19300755a0 msgr2=0x7f19300fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:20:24.969 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:24.966+0000 7f1935a00700 1 -- 172.21.15.153:0/1298003112 shutdown_connections 2024-03-21T10:20:24.970 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:24.966+0000 7f1935a00700 1 -- 172.21.15.153:0/1298003112 wait complete. 2024-03-21T10:20:24.970 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 30 2024-03-21T10:20:25.118 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:25.116+0000 7fef6ec00700 1 -- 172.21.15.153:0/657597965 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fef68102fe0 msgr2=0x7fef681033c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:20:25.118 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:25.116+0000 7fef6ec00700 1 --2- 172.21.15.153:0/657597965 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fef68102fe0 0x7fef681033c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fef58009910 tx=0x7fef58009c20 comp rx=0 tx=0).stop 2024-03-21T10:20:25.118 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:25.116+0000 7fef6ec00700 1 -- 172.21.15.153:0/657597965 shutdown_connections 2024-03-21T10:20:25.118 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:25.116+0000 7fef6ec00700 1 --2- 172.21.15.153:0/657597965 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fef68103990 0x7fef681079e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:25.118 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:25.116+0000 7fef6ec00700 1 --2- 172.21.15.153:0/657597965 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fef68102fe0 0x7fef681033c0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:25.118 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:25.116+0000 7fef6ec00700 1 -- 172.21.15.153:0/657597965 >> 172.21.15.153:0/657597965 conn(0x7fef680fe850 msgr2=0x7fef68100c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:20:25.118 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:25.116+0000 7fef6ec00700 1 -- 172.21.15.153:0/657597965 shutdown_connections 2024-03-21T10:20:25.119 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:25.116+0000 7fef6ec00700 1 -- 172.21.15.153:0/657597965 wait complete. 2024-03-21T10:20:25.120 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:25.117+0000 7fef6ec00700 1 Processor -- start 2024-03-21T10:20:25.121 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:25.117+0000 7fef6ec00700 1 -- start start 2024-03-21T10:20:25.121 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:25.118+0000 7fef6ec00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fef68102fe0 0x7fef68198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:20:25.121 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:25.118+0000 7fef6ec00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fef68103990 0x7fef68198ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:20:25.121 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:25.118+0000 7fef6ec00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef68199560 con 0x7fef68102fe0 2024-03-21T10:20:25.121 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:25.118+0000 7fef6ec00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef6819d370 con 0x7fef68103990 2024-03-21T10:20:25.121 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:25.118+0000 7fef67e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fef68102fe0 0x7fef68198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:20:25.121 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:25.118+0000 7fef67e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fef68102fe0 0x7fef68198990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:52010/0 (socket says 172.21.15.153:52010) 2024-03-21T10:20:25.121 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:25.118+0000 7fef67e00700 1 -- 172.21.15.153:0/2027892276 learned_addr learned my addr 172.21.15.153:0/2027892276 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:20:25.121 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:25.118+0000 7fef5fe00700 1 --2- 172.21.15.153:0/2027892276 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fef68103990 0x7fef68198ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:20:25.121 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:25.118+0000 7fef67e00700 1 -- 172.21.15.153:0/2027892276 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fef68103990 msgr2=0x7fef68198ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:20:25.121 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:25.118+0000 7fef67e00700 1 --2- 172.21.15.153:0/2027892276 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fef68103990 0x7fef68198ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:25.121 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:25.118+0000 7fef67e00700 1 -- 172.21.15.153:0/2027892276 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef580095f0 con 0x7fef68102fe0 2024-03-21T10:20:25.122 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:25.119+0000 7fef67e00700 1 --2- 172.21.15.153:0/2027892276 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fef68102fe0 0x7fef68198990 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fef58006010 tx=0x7fef5800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:20:25.122 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:25.119+0000 7fef65600700 1 -- 172.21.15.153:0/2027892276 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fef5801d070 con 0x7fef68102fe0 2024-03-21T10:20:25.122 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:25.119+0000 7fef65600700 1 -- 172.21.15.153:0/2027892276 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fef5800f460 con 0x7fef68102fe0 2024-03-21T10:20:25.122 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:25.119+0000 7fef6ec00700 1 -- 172.21.15.153:0/2027892276 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef6819d5f0 con 0x7fef68102fe0 2024-03-21T10:20:25.122 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:25.119+0000 7fef65600700 1 -- 172.21.15.153:0/2027892276 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fef58017630 con 0x7fef68102fe0 2024-03-21T10:20:25.122 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:25.119+0000 7fef6ec00700 1 -- 172.21.15.153:0/2027892276 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef6819dae0 con 0x7fef68102fe0 2024-03-21T10:20:25.123 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:25.120+0000 7fef6ec00700 1 -- 172.21.15.153:0/2027892276 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef6810b380 con 0x7fef68102fe0 2024-03-21T10:20:25.123 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:25.121+0000 7fef65600700 1 -- 172.21.15.153:0/2027892276 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fef5800f5d0 con 0x7fef68102fe0 2024-03-21T10:20:25.123 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:25.122+0000 7fef65600700 1 --2- 172.21.15.153:0/2027892276 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fef4c06e460 0x7fef4c070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:20:25.124 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:25.122+0000 7fef65600700 1 -- 172.21.15.153:0/2027892276 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fef5808f9a0 con 0x7fef68102fe0 2024-03-21T10:20:25.124 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:25.122+0000 7fef5fe00700 1 --2- 172.21.15.153:0/2027892276 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fef4c06e460 0x7fef4c070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:20:25.125 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:25.123+0000 7fef5fe00700 1 --2- 172.21.15.153:0/2027892276 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fef4c06e460 0x7fef4c070920 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fef50005950 tx=0x7fef500058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:20:25.128 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:25.127+0000 7fef65600700 1 -- 172.21.15.153:0/2027892276 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fef5805c570 con 0x7fef68102fe0 2024-03-21T10:20:25.354 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:25.352+0000 7fef6ec00700 1 -- 172.21.15.153:0/2027892276 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fef6819dd00 con 0x7fef4c06e460 2024-03-21T10:20:25.358 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:25.357+0000 7fef65600700 1 -- 172.21.15.153:0/2027892276 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fef6819dd00 con 0x7fef4c06e460 2024-03-21T10:20:25.359 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:20:25.359 INFO:teuthology.orchestra.run.smithi153.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232", 2024-03-21T10:20:25.359 INFO:teuthology.orchestra.run.smithi153.stdout: "in_progress": true, 2024-03-21T10:20:25.359 INFO:teuthology.orchestra.run.smithi153.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-21T10:20:25.360 INFO:teuthology.orchestra.run.smithi153.stdout: "services_complete": [ 2024-03-21T10:20:25.360 INFO:teuthology.orchestra.run.smithi153.stdout: "crash", 2024-03-21T10:20:25.360 INFO:teuthology.orchestra.run.smithi153.stdout: "mon", 2024-03-21T10:20:25.360 INFO:teuthology.orchestra.run.smithi153.stdout: "osd", 2024-03-21T10:20:25.360 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr" 2024-03-21T10:20:25.360 INFO:teuthology.orchestra.run.smithi153.stdout: ], 2024-03-21T10:20:25.360 INFO:teuthology.orchestra.run.smithi153.stdout: "progress": "15/23 daemons upgraded", 2024-03-21T10:20:25.360 INFO:teuthology.orchestra.run.smithi153.stdout: "message": "Currently upgrading mds daemons", 2024-03-21T10:20:25.360 INFO:teuthology.orchestra.run.smithi153.stdout: "is_paused": false 2024-03-21T10:20:25.360 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:20:25.362 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:25.360+0000 7fef6ec00700 1 -- 172.21.15.153:0/2027892276 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fef4c06e460 msgr2=0x7fef4c070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:20:25.362 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:25.360+0000 7fef6ec00700 1 --2- 172.21.15.153:0/2027892276 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fef4c06e460 0x7fef4c070920 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fef50005950 tx=0x7fef500058e0 comp rx=0 tx=0).stop 2024-03-21T10:20:25.362 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:25.360+0000 7fef6ec00700 1 -- 172.21.15.153:0/2027892276 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fef68102fe0 msgr2=0x7fef68198990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:20:25.362 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:25.360+0000 7fef6ec00700 1 --2- 172.21.15.153:0/2027892276 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fef68102fe0 0x7fef68198990 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fef58006010 tx=0x7fef5800be20 comp rx=0 tx=0).stop 2024-03-21T10:20:25.362 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:25.360+0000 7fef6ec00700 1 -- 172.21.15.153:0/2027892276 shutdown_connections 2024-03-21T10:20:25.362 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:25.360+0000 7fef6ec00700 1 --2- 172.21.15.153:0/2027892276 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fef4c06e460 0x7fef4c070920 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:25.363 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:25.360+0000 7fef6ec00700 1 --2- 172.21.15.153:0/2027892276 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fef68102fe0 0x7fef68198990 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:25.363 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:25.360+0000 7fef6ec00700 1 --2- 172.21.15.153:0/2027892276 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fef68103990 0x7fef68198ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:25.363 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:25.360+0000 7fef6ec00700 1 -- 172.21.15.153:0/2027892276 >> 172.21.15.153:0/2027892276 conn(0x7fef680fe850 msgr2=0x7fef680fff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:20:25.363 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:25.360+0000 7fef6ec00700 1 -- 172.21.15.153:0/2027892276 shutdown_connections 2024-03-21T10:20:25.363 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:25.361+0000 7fef6ec00700 1 -- 172.21.15.153:0/2027892276 wait complete. 2024-03-21T10:20:25.517 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:25.515+0000 7f2cb3200700 1 -- 172.21.15.153:0/3326886727 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2cac103000 msgr2=0x7f2cac1033e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:20:25.518 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:25.515+0000 7f2cb3200700 1 --2- 172.21.15.153:0/3326886727 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2cac103000 0x7f2cac1033e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c009910 tx=0x7f2c9c009c20 comp rx=0 tx=0).stop 2024-03-21T10:20:25.518 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:25.515+0000 7f2cb3200700 1 -- 172.21.15.153:0/3326886727 shutdown_connections 2024-03-21T10:20:25.518 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:25.515+0000 7f2cb3200700 1 --2- 172.21.15.153:0/3326886727 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2cac1039b0 0x7f2cac107a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:25.518 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:25.515+0000 7f2cb3200700 1 --2- 172.21.15.153:0/3326886727 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2cac103000 0x7f2cac1033e0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:25.518 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:25.515+0000 7f2cb3200700 1 -- 172.21.15.153:0/3326886727 >> 172.21.15.153:0/3326886727 conn(0x7f2cac0fe8d0 msgr2=0x7f2cac100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:20:25.518 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:25.515+0000 7f2cb3200700 1 -- 172.21.15.153:0/3326886727 shutdown_connections 2024-03-21T10:20:25.518 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:25.515+0000 7f2cb3200700 1 -- 172.21.15.153:0/3326886727 wait complete. 2024-03-21T10:20:25.520 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:25.516+0000 7f2cb3200700 1 Processor -- start 2024-03-21T10:20:25.520 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:25.517+0000 7f2cb3200700 1 -- start start 2024-03-21T10:20:25.520 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:25.517+0000 7f2cb3200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2cac103000 0x7f2cac19e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:20:25.520 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:25.517+0000 7f2cb3200700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2cac1039b0 0x7f2cac19ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:20:25.520 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:25.517+0000 7f2cb3200700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cac19f4d0 con 0x7f2cac103000 2024-03-21T10:20:25.520 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:25.517+0000 7f2cb3200700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cac198980 con 0x7f2cac1039b0 2024-03-21T10:20:25.520 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:25.517+0000 7f2cabe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2cac103000 0x7f2cac19e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:20:25.520 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:25.517+0000 7f2cabe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2cac103000 0x7f2cac19e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:52038/0 (socket says 172.21.15.153:52038) 2024-03-21T10:20:25.520 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:25.517+0000 7f2cabe00700 1 -- 172.21.15.153:0/4138490304 learned_addr learned my addr 172.21.15.153:0/4138490304 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:20:25.520 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:25.518+0000 7f2ca3400700 1 --2- 172.21.15.153:0/4138490304 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2cac1039b0 0x7f2cac19ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:20:25.520 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:25.518+0000 7f2cabe00700 1 -- 172.21.15.153:0/4138490304 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2cac1039b0 msgr2=0x7f2cac19ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:20:25.521 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:25.518+0000 7f2cabe00700 1 --2- 172.21.15.153:0/4138490304 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2cac1039b0 0x7f2cac19ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:25.521 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:25.518+0000 7f2cabe00700 1 -- 172.21.15.153:0/4138490304 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c9c0095f0 con 0x7f2cac103000 2024-03-21T10:20:25.521 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:25.518+0000 7f2ca3400700 1 --2- 172.21.15.153:0/4138490304 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2cac1039b0 0x7f2cac19ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:20:25.521 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:25.518+0000 7f2cabe00700 1 --2- 172.21.15.153:0/4138490304 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2cac103000 0x7f2cac19e900 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c006010 tx=0x7f2c9c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:20:25.521 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:25.518+0000 7f2ca9600700 1 -- 172.21.15.153:0/4138490304 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2c9c01d070 con 0x7f2cac103000 2024-03-21T10:20:25.522 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:25.518+0000 7f2ca9600700 1 -- 172.21.15.153:0/4138490304 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2c9c004fb0 con 0x7f2cac103000 2024-03-21T10:20:25.522 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:25.519+0000 7f2ca9600700 1 -- 172.21.15.153:0/4138490304 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2c9c00f690 con 0x7f2cac103000 2024-03-21T10:20:25.522 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:25.519+0000 7f2cb3200700 1 -- 172.21.15.153:0/4138490304 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cac198c00 con 0x7f2cac103000 2024-03-21T10:20:25.522 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:25.519+0000 7f2cb3200700 1 -- 172.21.15.153:0/4138490304 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cac1990f0 con 0x7f2cac103000 2024-03-21T10:20:25.522 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:25.520+0000 7f2cb3200700 1 -- 172.21.15.153:0/4138490304 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cac10b3a0 con 0x7f2cac103000 2024-03-21T10:20:25.522 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:25.521+0000 7f2ca9600700 1 -- 172.21.15.153:0/4138490304 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f2c9c0228a0 con 0x7f2cac103000 2024-03-21T10:20:25.523 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:25.522+0000 7f2ca9600700 1 --2- 172.21.15.153:0/4138490304 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f2c9006e460 0x7f2c90070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:20:25.524 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:25.522+0000 7f2ca9600700 1 -- 172.21.15.153:0/4138490304 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f2c9c08fe10 con 0x7f2cac103000 2024-03-21T10:20:25.524 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:25.522+0000 7f2ca3400700 1 --2- 172.21.15.153:0/4138490304 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f2c9006e460 0x7f2c90070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:20:25.524 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:25.523+0000 7f2ca3400700 1 --2- 172.21.15.153:0/4138490304 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f2c9006e460 0x7f2c90070920 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2cac0ff140 tx=0x7f2c940073c0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:20:25.528 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:25.526+0000 7f2ca9600700 1 -- 172.21.15.153:0/4138490304 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2c9c05c960 con 0x7f2cac103000 2024-03-21T10:20:25.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:20:25 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/1892737862' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:20:25.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:20:25 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/1298003112' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:20:25.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:20:25 smithi174 ceph-mon[88004]: pgmap v257: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:20:25.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:20:25 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/1892737862' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:20:25.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:20:25 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/1298003112' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:20:25.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:20:25 smithi153 ceph-mon[113733]: pgmap v257: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:20:25.823 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:25.821+0000 7f2cb3200700 1 -- 172.21.15.153:0/4138490304 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f2cac02cd30 con 0x7f2cac103000 2024-03-21T10:20:25.824 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:25.822+0000 7f2ca9600700 1 -- 172.21.15.153:0/4138490304 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f2c9c0271c0 con 0x7f2cac103000 2024-03-21T10:20:25.824 INFO:teuthology.orchestra.run.smithi153.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-03-21T10:20:25.824 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-21T10:20:25.824 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is degraded 2024-03-21T10:20:25.824 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-21T10:20:25.824 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-21T10:20:25.825 INFO:teuthology.orchestra.run.smithi153.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-21T10:20:25.825 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-21T10:20:25.827 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:25.825+0000 7f2cb3200700 1 -- 172.21.15.153:0/4138490304 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f2c9006e460 msgr2=0x7f2c90070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:20:25.827 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:25.825+0000 7f2cb3200700 1 --2- 172.21.15.153:0/4138490304 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f2c9006e460 0x7f2c90070920 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2cac0ff140 tx=0x7f2c940073c0 comp rx=0 tx=0).stop 2024-03-21T10:20:25.827 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:25.825+0000 7f2cb3200700 1 -- 172.21.15.153:0/4138490304 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2cac103000 msgr2=0x7f2cac19e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:20:25.827 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:25.825+0000 7f2cb3200700 1 --2- 172.21.15.153:0/4138490304 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2cac103000 0x7f2cac19e900 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c006010 tx=0x7f2c9c00be20 comp rx=0 tx=0).stop 2024-03-21T10:20:25.827 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:25.825+0000 7f2cb3200700 1 -- 172.21.15.153:0/4138490304 shutdown_connections 2024-03-21T10:20:25.828 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:25.825+0000 7f2cb3200700 1 --2- 172.21.15.153:0/4138490304 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f2c9006e460 0x7f2c90070920 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:25.828 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:25.825+0000 7f2cb3200700 1 --2- 172.21.15.153:0/4138490304 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2cac103000 0x7f2cac19e900 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:25.828 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:25.825+0000 7f2cb3200700 1 --2- 172.21.15.153:0/4138490304 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2cac1039b0 0x7f2cac19ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:25.828 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:25.825+0000 7f2cb3200700 1 -- 172.21.15.153:0/4138490304 >> 172.21.15.153:0/4138490304 conn(0x7f2cac0fe8d0 msgr2=0x7f2cac0fff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:20:25.828 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:25.826+0000 7f2cb3200700 1 -- 172.21.15.153:0/4138490304 shutdown_connections 2024-03-21T10:20:25.828 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:25.826+0000 7f2cb3200700 1 -- 172.21.15.153:0/4138490304 wait complete. 2024-03-21T10:20:26.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:20:26 smithi174 ceph-mon[88004]: from='client.34526 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:20:26.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:20:26 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/4138490304' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:20:26.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:20:26 smithi153 ceph-mon[113733]: from='client.34526 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:20:26.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:20:26 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/4138490304' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:20:27.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:20:27 smithi174 ceph-mon[88004]: pgmap v258: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:20:27.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:20:27 smithi153 ceph-mon[113733]: pgmap v258: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:20:29.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:20:29 smithi174 ceph-mon[88004]: pgmap v259: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:20:29.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:20:29 smithi153 ceph-mon[113733]: pgmap v259: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:20:31.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:20:31 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:20:31.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:20:31 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:20:32.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:20:32 smithi153 ceph-mon[113733]: pgmap v260: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:20:32.603 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:20:32 smithi174 ceph-mon[88004]: pgmap v260: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:20:34.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:20:34 smithi153 ceph-mon[113733]: pgmap v261: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:20:34.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:20:34 smithi174 ceph-mon[88004]: pgmap v261: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:20:36.558 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:20:36 smithi153 ceph-mon[113733]: pgmap v262: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:20:36.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:20:36 smithi174 ceph-mon[88004]: pgmap v262: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:20:37.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:20:37 smithi174 ceph-mon[88004]: pgmap v263: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:20:37.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:20:37 smithi153 ceph-mon[113733]: pgmap v263: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:20:39.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:20:39 smithi174 ceph-mon[88004]: pgmap v264: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:20:39.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:20:39 smithi153 ceph-mon[113733]: pgmap v264: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:20:41.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:20:41 smithi174 ceph-mon[88004]: pgmap v265: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:20:41.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:20:41 smithi153 ceph-mon[113733]: pgmap v265: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:20:43.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:20:43 smithi174 ceph-mon[88004]: pgmap v266: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:20:43.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:20:43 smithi153 ceph-mon[113733]: pgmap v266: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:20:45.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:20:45 smithi174 ceph-mon[88004]: pgmap v267: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:20:45.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:20:45 smithi153 ceph-mon[113733]: pgmap v267: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:20:46.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:20:46 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:20:46.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:20:46 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:20:47.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:20:47 smithi174 ceph-mon[88004]: pgmap v268: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:20:47.730 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:20:47 smithi153 ceph-mon[113733]: pgmap v268: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:20:49.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:20:49 smithi174 ceph-mon[88004]: pgmap v269: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:20:49.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:20:49 smithi153 ceph-mon[113733]: pgmap v269: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:20:51.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:20:51 smithi174 ceph-mon[88004]: pgmap v270: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:20:51.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:20:51 smithi153 ceph-mon[113733]: pgmap v270: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:20:53.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:20:53 smithi174 ceph-mon[88004]: pgmap v271: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:20:53.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:20:53 smithi153 ceph-mon[113733]: pgmap v271: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:20:55.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:20:55 smithi174 ceph-mon[88004]: pgmap v272: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:20:55.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:20:55 smithi153 ceph-mon[113733]: pgmap v272: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:20:55.997 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:55.995+0000 7f2a6cc00700 1 -- 172.21.15.153:0/2079665023 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a68073ae0 msgr2=0x7f2a6810ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:20:55.997 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:55.995+0000 7f2a6cc00700 1 --2- 172.21.15.153:0/2079665023 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a68073ae0 0x7f2a6810ce40 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f2a54009910 tx=0x7f2a54009c20 comp rx=0 tx=0).stop 2024-03-21T10:20:55.997 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:55.995+0000 7f2a6cc00700 1 -- 172.21.15.153:0/2079665023 shutdown_connections 2024-03-21T10:20:55.997 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:55.995+0000 7f2a6cc00700 1 --2- 172.21.15.153:0/2079665023 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a68073ae0 0x7f2a6810ce40 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:55.997 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:55.995+0000 7f2a6cc00700 1 --2- 172.21.15.153:0/2079665023 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a680731c0 0x7f2a680735a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:55.997 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:55.995+0000 7f2a6cc00700 1 -- 172.21.15.153:0/2079665023 >> 172.21.15.153:0/2079665023 conn(0x7f2a680fc630 msgr2=0x7f2a680fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:20:55.997 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:55.995+0000 7f2a6cc00700 1 -- 172.21.15.153:0/2079665023 shutdown_connections 2024-03-21T10:20:55.997 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:55.996+0000 7f2a6cc00700 1 -- 172.21.15.153:0/2079665023 wait complete. 2024-03-21T10:20:55.999 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:55.996+0000 7f2a6cc00700 1 Processor -- start 2024-03-21T10:20:55.999 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:55.997+0000 7f2a6cc00700 1 -- start start 2024-03-21T10:20:55.999 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:55.997+0000 7f2a6cc00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a680731c0 0x7f2a6819e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:20:55.999 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:55.997+0000 7f2a6cc00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a68073ae0 0x7f2a6819ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:20:55.999 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:55.997+0000 7f2a6cc00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a6819f3e0 con 0x7f2a68073ae0 2024-03-21T10:20:55.999 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:55.997+0000 7f2a6cc00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a68198890 con 0x7f2a680731c0 2024-03-21T10:20:55.999 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:55.997+0000 7f2a65400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a68073ae0 0x7f2a6819ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:20:55.999 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:55.997+0000 7f2a65400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a68073ae0 0x7f2a6819ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:37022/0 (socket says 172.21.15.153:37022) 2024-03-21T10:20:55.999 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:55.997+0000 7f2a65400700 1 -- 172.21.15.153:0/537827720 learned_addr learned my addr 172.21.15.153:0/537827720 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:20:55.999 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:55.998+0000 7f2a65e00700 1 --2- 172.21.15.153:0/537827720 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a680731c0 0x7f2a6819e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:20:55.999 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:55.998+0000 7f2a65400700 1 -- 172.21.15.153:0/537827720 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a680731c0 msgr2=0x7f2a6819e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:20:56.000 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:55.998+0000 7f2a65400700 1 --2- 172.21.15.153:0/537827720 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a680731c0 0x7f2a6819e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:56.000 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:55.998+0000 7f2a65400700 1 -- 172.21.15.153:0/537827720 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a540095f0 con 0x7f2a68073ae0 2024-03-21T10:20:56.000 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:55.998+0000 7f2a65e00700 1 --2- 172.21.15.153:0/537827720 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a680731c0 0x7f2a6819e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:20:56.000 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:55.998+0000 7f2a65400700 1 --2- 172.21.15.153:0/537827720 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a68073ae0 0x7f2a6819ed50 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2a5400bd70 tx=0x7f2a5400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:20:56.000 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:55.999+0000 7f2a5ea00700 1 -- 172.21.15.153:0/537827720 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a5401d070 con 0x7f2a68073ae0 2024-03-21T10:20:56.001 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:55.999+0000 7f2a5ea00700 1 -- 172.21.15.153:0/537827720 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2a5400f460 con 0x7f2a68073ae0 2024-03-21T10:20:56.001 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:55.999+0000 7f2a5ea00700 1 -- 172.21.15.153:0/537827720 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a54017630 con 0x7f2a68073ae0 2024-03-21T10:20:56.001 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:55.999+0000 7f2a6cc00700 1 -- 172.21.15.153:0/537827720 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a68198b10 con 0x7f2a68073ae0 2024-03-21T10:20:56.001 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:55.999+0000 7f2a6cc00700 1 -- 172.21.15.153:0/537827720 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a68199000 con 0x7f2a68073ae0 2024-03-21T10:20:56.001 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:56.000+0000 7f2a6cc00700 1 -- 172.21.15.153:0/537827720 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a6810a540 con 0x7f2a68073ae0 2024-03-21T10:20:56.001 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:56.001+0000 7f2a5ea00700 1 -- 172.21.15.153:0/537827720 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f2a5400f5d0 con 0x7f2a68073ae0 2024-03-21T10:20:56.003 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:56.002+0000 7f2a5ea00700 1 --2- 172.21.15.153:0/537827720 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f2a4c06e460 0x7f2a4c070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:20:56.003 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:56.002+0000 7f2a5ea00700 1 -- 172.21.15.153:0/537827720 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f2a5408fae0 con 0x7f2a68073ae0 2024-03-21T10:20:56.003 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:56.002+0000 7f2a65e00700 1 --2- 172.21.15.153:0/537827720 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f2a4c06e460 0x7f2a4c070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:20:56.004 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:56.003+0000 7f2a65e00700 1 --2- 172.21.15.153:0/537827720 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f2a4c06e460 0x7f2a4c070920 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2a58007900 tx=0x7f2a58008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:20:56.008 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:56.007+0000 7f2a5ea00700 1 -- 172.21.15.153:0/537827720 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2a5405c6d0 con 0x7f2a68073ae0 2024-03-21T10:20:56.253 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:56.252+0000 7f2a6cc00700 1 -- 172.21.15.153:0/537827720 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2a681992e0 con 0x7f2a4c06e460 2024-03-21T10:20:56.255 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:56.254+0000 7f2a5ea00700 1 -- 172.21.15.153:0/537827720 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f2a681992e0 con 0x7f2a4c06e460 2024-03-21T10:20:56.258 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:56.256+0000 7f2a6cc00700 1 -- 172.21.15.153:0/537827720 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f2a4c06e460 msgr2=0x7f2a4c070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:20:56.258 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:56.256+0000 7f2a6cc00700 1 --2- 172.21.15.153:0/537827720 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f2a4c06e460 0x7f2a4c070920 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2a58007900 tx=0x7f2a58008040 comp rx=0 tx=0).stop 2024-03-21T10:20:56.258 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:56.257+0000 7f2a6cc00700 1 -- 172.21.15.153:0/537827720 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a68073ae0 msgr2=0x7f2a6819ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:20:56.258 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:56.257+0000 7f2a6cc00700 1 --2- 172.21.15.153:0/537827720 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a68073ae0 0x7f2a6819ed50 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2a5400bd70 tx=0x7f2a5400be50 comp rx=0 tx=0).stop 2024-03-21T10:20:56.258 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:56.257+0000 7f2a6cc00700 1 -- 172.21.15.153:0/537827720 shutdown_connections 2024-03-21T10:20:56.258 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:56.257+0000 7f2a6cc00700 1 --2- 172.21.15.153:0/537827720 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a680731c0 0x7f2a6819e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:56.259 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:56.257+0000 7f2a6cc00700 1 --2- 172.21.15.153:0/537827720 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f2a4c06e460 0x7f2a4c070920 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:56.259 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:56.257+0000 7f2a6cc00700 1 --2- 172.21.15.153:0/537827720 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a68073ae0 0x7f2a6819ed50 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:56.259 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:56.257+0000 7f2a6cc00700 1 -- 172.21.15.153:0/537827720 >> 172.21.15.153:0/537827720 conn(0x7f2a680fc630 msgr2=0x7f2a68107680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:20:56.259 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:56.257+0000 7f2a6cc00700 1 -- 172.21.15.153:0/537827720 shutdown_connections 2024-03-21T10:20:56.259 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:56.257+0000 7f2a6cc00700 1 -- 172.21.15.153:0/537827720 wait complete. 2024-03-21T10:20:56.273 INFO:teuthology.orchestra.run.smithi153.stdout:true 2024-03-21T10:20:56.414 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:56.413+0000 7fcae2e00700 1 -- 172.21.15.153:0/612484218 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcadc1037c0 msgr2=0x7fcadc107810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:20:56.414 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:56.413+0000 7fcae2e00700 1 --2- 172.21.15.153:0/612484218 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcadc1037c0 0x7fcadc107810 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fcac8009910 tx=0x7fcac8009c20 comp rx=0 tx=0).stop 2024-03-21T10:20:56.414 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:56.413+0000 7fcae2e00700 1 -- 172.21.15.153:0/612484218 shutdown_connections 2024-03-21T10:20:56.414 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:56.413+0000 7fcae2e00700 1 --2- 172.21.15.153:0/612484218 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcadc1037c0 0x7fcadc107810 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:56.414 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:56.413+0000 7fcae2e00700 1 --2- 172.21.15.153:0/612484218 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcadc102e10 0x7fcadc1031f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:56.415 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:56.413+0000 7fcae2e00700 1 -- 172.21.15.153:0/612484218 >> 172.21.15.153:0/612484218 conn(0x7fcadc0fe6c0 msgr2=0x7fcadc100ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:20:56.415 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:56.413+0000 7fcae2e00700 1 -- 172.21.15.153:0/612484218 shutdown_connections 2024-03-21T10:20:56.415 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:56.413+0000 7fcae2e00700 1 -- 172.21.15.153:0/612484218 wait complete. 2024-03-21T10:20:56.416 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:56.414+0000 7fcae2e00700 1 Processor -- start 2024-03-21T10:20:56.416 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:56.415+0000 7fcae2e00700 1 -- start start 2024-03-21T10:20:56.416 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:56.415+0000 7fcae2e00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcadc102e10 0x7fcadc19e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:20:56.416 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:56.415+0000 7fcae2e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcadc1037c0 0x7fcadc19ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:20:56.416 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:56.415+0000 7fcae2e00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcadc19f120 con 0x7fcadc1037c0 2024-03-21T10:20:56.416 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:56.415+0000 7fcae2e00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcadc1985d0 con 0x7fcadc102e10 2024-03-21T10:20:56.416 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:56.415+0000 7fcae1000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcadc1037c0 0x7fcadc19ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:20:56.416 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:56.415+0000 7fcae1000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcadc1037c0 0x7fcadc19ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:37042/0 (socket says 172.21.15.153:37042) 2024-03-21T10:20:56.416 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:56.415+0000 7fcae1000700 1 -- 172.21.15.153:0/221698402 learned_addr learned my addr 172.21.15.153:0/221698402 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:20:56.417 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:56.416+0000 7fcae1a00700 1 --2- 172.21.15.153:0/221698402 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcadc102e10 0x7fcadc19e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:20:56.417 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:56.416+0000 7fcae1000700 1 -- 172.21.15.153:0/221698402 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcadc102e10 msgr2=0x7fcadc19e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:20:56.417 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:56.416+0000 7fcae1000700 1 --2- 172.21.15.153:0/221698402 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcadc102e10 0x7fcadc19e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:56.418 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:56.416+0000 7fcae1000700 1 -- 172.21.15.153:0/221698402 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcac80095f0 con 0x7fcadc1037c0 2024-03-21T10:20:56.418 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:56.416+0000 7fcae1000700 1 --2- 172.21.15.153:0/221698402 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcadc1037c0 0x7fcadc19ea90 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fcac800bd70 tx=0x7fcac800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:20:56.418 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:56.416+0000 7fcada000700 1 -- 172.21.15.153:0/221698402 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcac801d070 con 0x7fcadc1037c0 2024-03-21T10:20:56.418 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:56.416+0000 7fcada000700 1 -- 172.21.15.153:0/221698402 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcac8005dc0 con 0x7fcadc1037c0 2024-03-21T10:20:56.418 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:56.416+0000 7fcae2e00700 1 -- 172.21.15.153:0/221698402 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcadc198850 con 0x7fcadc1037c0 2024-03-21T10:20:56.418 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:56.417+0000 7fcada000700 1 -- 172.21.15.153:0/221698402 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcac8017400 con 0x7fcadc1037c0 2024-03-21T10:20:56.418 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:56.417+0000 7fcae2e00700 1 -- 172.21.15.153:0/221698402 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcadc198d40 con 0x7fcadc1037c0 2024-03-21T10:20:56.418 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:56.418+0000 7fcae2e00700 1 -- 172.21.15.153:0/221698402 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcadc10b210 con 0x7fcadc1037c0 2024-03-21T10:20:56.422 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:56.419+0000 7fcada000700 1 -- 172.21.15.153:0/221698402 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fcac800f7f0 con 0x7fcadc1037c0 2024-03-21T10:20:56.422 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:56.420+0000 7fcada000700 1 --2- 172.21.15.153:0/221698402 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fcac406e460 0x7fcac4070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:20:56.423 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:56.420+0000 7fcada000700 1 -- 172.21.15.153:0/221698402 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fcac808fa60 con 0x7fcadc1037c0 2024-03-21T10:20:56.423 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:56.420+0000 7fcae1a00700 1 --2- 172.21.15.153:0/221698402 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fcac406e460 0x7fcac4070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:20:56.423 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:56.421+0000 7fcae1a00700 1 --2- 172.21.15.153:0/221698402 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fcac406e460 0x7fcac4070920 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fcad0005950 tx=0x7fcad00058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:20:56.425 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:56.425+0000 7fcada000700 1 -- 172.21.15.153:0/221698402 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fcac805c5b0 con 0x7fcadc1037c0 2024-03-21T10:20:56.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:20:56 smithi153 ceph-mon[113733]: from='client.34534 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:20:56.650 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:56.649+0000 7fcae2e00700 1 -- 172.21.15.153:0/221698402 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcadc02cd10 con 0x7fcac406e460 2024-03-21T10:20:56.651 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:56.650+0000 7fcada000700 1 -- 172.21.15.153:0/221698402 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fcadc02cd10 con 0x7fcac406e460 2024-03-21T10:20:56.654 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:56.654+0000 7fcae2e00700 1 -- 172.21.15.153:0/221698402 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fcac406e460 msgr2=0x7fcac4070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:20:56.654 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:56.654+0000 7fcae2e00700 1 --2- 172.21.15.153:0/221698402 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fcac406e460 0x7fcac4070920 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fcad0005950 tx=0x7fcad00058e0 comp rx=0 tx=0).stop 2024-03-21T10:20:56.654 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:56.654+0000 7fcae2e00700 1 -- 172.21.15.153:0/221698402 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcadc1037c0 msgr2=0x7fcadc19ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:20:56.655 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:56.654+0000 7fcae2e00700 1 --2- 172.21.15.153:0/221698402 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcadc1037c0 0x7fcadc19ea90 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fcac800bd70 tx=0x7fcac800be50 comp rx=0 tx=0).stop 2024-03-21T10:20:56.655 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:56.654+0000 7fcae2e00700 1 -- 172.21.15.153:0/221698402 shutdown_connections 2024-03-21T10:20:56.655 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:56.654+0000 7fcae2e00700 1 --2- 172.21.15.153:0/221698402 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcadc102e10 0x7fcadc19e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:56.655 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:56.654+0000 7fcae2e00700 1 --2- 172.21.15.153:0/221698402 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fcac406e460 0x7fcac4070920 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:56.655 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:56.654+0000 7fcae2e00700 1 --2- 172.21.15.153:0/221698402 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcadc1037c0 0x7fcadc19ea90 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:56.655 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:56.654+0000 7fcae2e00700 1 -- 172.21.15.153:0/221698402 >> 172.21.15.153:0/221698402 conn(0x7fcadc0fe6c0 msgr2=0x7fcadc100a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:20:56.655 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:56.654+0000 7fcae2e00700 1 -- 172.21.15.153:0/221698402 shutdown_connections 2024-03-21T10:20:56.655 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:56.654+0000 7fcae2e00700 1 -- 172.21.15.153:0/221698402 wait complete. 2024-03-21T10:20:56.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:20:56 smithi174 ceph-mon[88004]: from='client.34534 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:20:56.805 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:56.804+0000 7f92ad600700 1 -- 172.21.15.153:0/4230891070 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f92a8101750 msgr2=0x7f92a81057a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:20:56.805 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:56.804+0000 7f92ad600700 1 --2- 172.21.15.153:0/4230891070 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f92a8101750 0x7f92a81057a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f9294009910 tx=0x7f9294009c20 comp rx=0 tx=0).stop 2024-03-21T10:20:56.805 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:56.804+0000 7f92ad600700 1 -- 172.21.15.153:0/4230891070 shutdown_connections 2024-03-21T10:20:56.805 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:56.804+0000 7f92ad600700 1 --2- 172.21.15.153:0/4230891070 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f92a8101750 0x7f92a81057a0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:56.805 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:56.804+0000 7f92ad600700 1 --2- 172.21.15.153:0/4230891070 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f92a8100da0 0x7f92a8101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:56.806 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:56.804+0000 7f92ad600700 1 -- 172.21.15.153:0/4230891070 >> 172.21.15.153:0/4230891070 conn(0x7f92a80fc650 msgr2=0x7f92a80fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:20:56.806 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:56.804+0000 7f92ad600700 1 -- 172.21.15.153:0/4230891070 shutdown_connections 2024-03-21T10:20:56.806 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:56.804+0000 7f92ad600700 1 -- 172.21.15.153:0/4230891070 wait complete. 2024-03-21T10:20:56.807 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:56.805+0000 7f92ad600700 1 Processor -- start 2024-03-21T10:20:56.807 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:56.805+0000 7f92ad600700 1 -- start start 2024-03-21T10:20:56.807 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:56.806+0000 7f92ad600700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f92a8100da0 0x7f92a8073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:20:56.807 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:56.806+0000 7f92ad600700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f92a8101750 0x7f92a80735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:20:56.807 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:56.806+0000 7f92ad600700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92a8073b10 con 0x7f92a8101750 2024-03-21T10:20:56.807 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:56.806+0000 7f92ad600700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92a8073c50 con 0x7f92a8100da0 2024-03-21T10:20:56.807 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:56.806+0000 7f92a6000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f92a8101750 0x7f92a80735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:20:56.807 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:56.806+0000 7f92a6000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f92a8101750 0x7f92a80735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:37070/0 (socket says 172.21.15.153:37070) 2024-03-21T10:20:56.807 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:56.806+0000 7f92a6000700 1 -- 172.21.15.153:0/2307677987 learned_addr learned my addr 172.21.15.153:0/2307677987 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:20:56.807 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:56.806+0000 7f92a6a00700 1 --2- 172.21.15.153:0/2307677987 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f92a8100da0 0x7f92a8073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:20:56.808 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:56.807+0000 7f92a6000700 1 -- 172.21.15.153:0/2307677987 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f92a8100da0 msgr2=0x7f92a8073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:20:56.808 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:56.807+0000 7f92a6000700 1 --2- 172.21.15.153:0/2307677987 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f92a8100da0 0x7f92a8073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:56.808 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:56.807+0000 7f92a6000700 1 -- 172.21.15.153:0/2307677987 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92940095f0 con 0x7f92a8101750 2024-03-21T10:20:56.809 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:56.807+0000 7f92a6000700 1 --2- 172.21.15.153:0/2307677987 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f92a8101750 0x7f92a80735d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f929400bda0 tx=0x7f929400be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:20:56.809 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:56.807+0000 7f929f400700 1 -- 172.21.15.153:0/2307677987 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f929401d070 con 0x7f92a8101750 2024-03-21T10:20:56.809 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:56.807+0000 7f929f400700 1 -- 172.21.15.153:0/2307677987 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9294004fb0 con 0x7f92a8101750 2024-03-21T10:20:56.809 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:56.807+0000 7f92ad600700 1 -- 172.21.15.153:0/2307677987 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92a8073ed0 con 0x7f92a8101750 2024-03-21T10:20:56.809 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:56.807+0000 7f929f400700 1 -- 172.21.15.153:0/2307677987 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f929400f460 con 0x7f92a8101750 2024-03-21T10:20:56.809 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:56.807+0000 7f92ad600700 1 -- 172.21.15.153:0/2307677987 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92a81a2b90 con 0x7f92a8101750 2024-03-21T10:20:56.809 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:56.809+0000 7f92ad600700 1 -- 172.21.15.153:0/2307677987 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92a81091a0 con 0x7f92a8101750 2024-03-21T10:20:56.810 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:56.810+0000 7f929f400700 1 -- 172.21.15.153:0/2307677987 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f9294022800 con 0x7f92a8101750 2024-03-21T10:20:56.811 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:56.811+0000 7f929f400700 1 --2- 172.21.15.153:0/2307677987 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f928c06e460 0x7f928c070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:20:56.811 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:56.811+0000 7f929f400700 1 -- 172.21.15.153:0/2307677987 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f929408fa00 con 0x7f92a8101750 2024-03-21T10:20:56.812 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:56.811+0000 7f92a6a00700 1 --2- 172.21.15.153:0/2307677987 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f928c06e460 0x7f928c070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:20:56.812 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:56.812+0000 7f92a6a00700 1 --2- 172.21.15.153:0/2307677987 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f928c06e460 0x7f928c070920 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f9298005ea0 tx=0x7f9298005e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:20:56.816 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:56.815+0000 7f929f400700 1 -- 172.21.15.153:0/2307677987 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f9294058b50 con 0x7f92a8101750 2024-03-21T10:20:57.050 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:57.049+0000 7f92ad600700 1 -- 172.21.15.153:0/2307677987 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f92a802cd80 con 0x7f928c06e460 2024-03-21T10:20:57.086 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:57.085+0000 7f929f400700 1 -- 172.21.15.153:0/2307677987 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f92a802cd80 con 0x7f928c06e460 2024-03-21T10:20:57.087 INFO:teuthology.orchestra.run.smithi153.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-21T10:20:57.087 INFO:teuthology.orchestra.run.smithi153.stdout:alertmanager.smithi153 smithi153 *:9093,9094 running (9m) 70s ago 23m 14.7M - 0.25.0 c8568f914cd2 9ed3f2200b5f 2024-03-21T10:20:57.087 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi153 smithi153 running (23m) 70s ago 23m 9424k - 18.2.1 5be31c24972a 5b61994396a6 2024-03-21T10:20:57.087 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi174 smithi174 running (22m) 49s ago 22m 9244k - 18.2.1 5be31c24972a 4f7a5fc3ac28 2024-03-21T10:20:57.087 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi153 smithi153 running (5m) 70s ago 23m 6656k - 19.0.0-2231-g2edfee68 237e43c7a65f 32c21104d5b0 2024-03-21T10:20:57.087 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi174 smithi174 running (5m) 49s ago 22m 6660k - 19.0.0-2231-g2edfee68 237e43c7a65f a04e48967ab6 2024-03-21T10:20:57.087 INFO:teuthology.orchestra.run.smithi153.stdout:grafana.smithi153 smithi153 *:3000 running (9m) 70s ago 22m 73.6M - 9.4.12 2bacad6d85d8 ee24026ab786 2024-03-21T10:20:57.087 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.iuyluj smithi153 running (78s) 70s ago 16m 10.9M - 19.0.0-2231-g2edfee68 237e43c7a65f 87aff8a5de70 2024-03-21T10:20:57.087 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.zipckr smithi153 running (102s) 70s ago 16m 11.8M - 19.0.0-2231-g2edfee68 237e43c7a65f ccc262bd5dd3 2024-03-21T10:20:57.087 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.nhozag smithi174 running (16m) 49s ago 16m 12.0M - 18.2.1 5be31c24972a 2fee4000308f 2024-03-21T10:20:57.087 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.yyimst smithi174 running (54s) 49s ago 16m 10.7M - 19.0.0-2231-g2edfee68 237e43c7a65f fc1962f9351e 2024-03-21T10:20:57.087 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi153.tluxni smithi153 *:8443,9283,8765 running (12m) 70s ago 25m 509M - 19.0.0-2231-g2edfee68 237e43c7a65f 7ffee0e55382 2024-03-21T10:20:57.087 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi174.qpcwat smithi174 *:8443,9283,8765 running (11m) 49s ago 21m 452M - 19.0.0-2231-g2edfee68 237e43c7a65f 748968f216ab 2024-03-21T10:20:57.088 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi153 smithi153 running (7m) 70s ago 25m 55.0M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f dd8038303dc3 2024-03-21T10:20:57.088 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi174 smithi174 running (7m) 49s ago 21m 44.2M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f 29d78a788f2c 2024-03-21T10:20:57.088 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi153 smithi153 *:9100 running (10m) 70s ago 23m 7889k - 1.5.0 0da6a335fe13 bb835edc0e0b 2024-03-21T10:20:57.088 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi174 smithi174 *:9100 running (10m) 49s ago 22m 8102k - 1.5.0 0da6a335fe13 7c4ecad790fa 2024-03-21T10:20:57.088 INFO:teuthology.orchestra.run.smithi153.stdout:osd.0 smithi153 running (5m) 70s ago 20m 116M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 507900fd4791 2024-03-21T10:20:57.088 INFO:teuthology.orchestra.run.smithi153.stdout:osd.1 smithi153 running (4m) 70s ago 20m 89.2M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 0d646452e9c1 2024-03-21T10:20:57.088 INFO:teuthology.orchestra.run.smithi153.stdout:osd.2 smithi153 running (4m) 70s ago 19m 87.4M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 1326f2afdfcd 2024-03-21T10:20:57.088 INFO:teuthology.orchestra.run.smithi153.stdout:osd.3 smithi174 running (3m) 49s ago 19m 83.6M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 70aafac71ad5 2024-03-21T10:20:57.088 INFO:teuthology.orchestra.run.smithi153.stdout:osd.4 smithi174 running (2m) 49s ago 18m 92.2M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 0d746ce6e79d 2024-03-21T10:20:57.088 INFO:teuthology.orchestra.run.smithi153.stdout:osd.5 smithi174 running (2m) 49s ago 18m 86.2M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 53e5f35b8cac 2024-03-21T10:20:57.088 INFO:teuthology.orchestra.run.smithi153.stdout:prometheus.smithi153 smithi153 *:9095 running (10m) 70s ago 22m 73.5M - 2.43.0 a07b618ecd1d 7d95561e50c2 2024-03-21T10:20:57.089 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:57.088+0000 7f92ad600700 1 -- 172.21.15.153:0/2307677987 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f928c06e460 msgr2=0x7f928c070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:20:57.089 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:57.088+0000 7f92ad600700 1 --2- 172.21.15.153:0/2307677987 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f928c06e460 0x7f928c070920 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f9298005ea0 tx=0x7f9298005e30 comp rx=0 tx=0).stop 2024-03-21T10:20:57.090 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:57.088+0000 7f92ad600700 1 -- 172.21.15.153:0/2307677987 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f92a8101750 msgr2=0x7f92a80735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:20:57.090 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:57.088+0000 7f92ad600700 1 --2- 172.21.15.153:0/2307677987 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f92a8101750 0x7f92a80735d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f929400bda0 tx=0x7f929400be80 comp rx=0 tx=0).stop 2024-03-21T10:20:57.090 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:57.088+0000 7f92ad600700 1 -- 172.21.15.153:0/2307677987 shutdown_connections 2024-03-21T10:20:57.090 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:57.088+0000 7f92ad600700 1 --2- 172.21.15.153:0/2307677987 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f92a8100da0 0x7f92a8073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:57.090 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:57.088+0000 7f92ad600700 1 --2- 172.21.15.153:0/2307677987 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f928c06e460 0x7f928c070920 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:57.090 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:57.088+0000 7f92ad600700 1 --2- 172.21.15.153:0/2307677987 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f92a8101750 0x7f92a80735d0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:57.090 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:57.088+0000 7f92ad600700 1 -- 172.21.15.153:0/2307677987 >> 172.21.15.153:0/2307677987 conn(0x7f92a80fc650 msgr2=0x7f92a80fea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:20:57.090 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:57.089+0000 7f92ad600700 1 -- 172.21.15.153:0/2307677987 shutdown_connections 2024-03-21T10:20:57.090 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:57.089+0000 7f92ad600700 1 -- 172.21.15.153:0/2307677987 wait complete. 2024-03-21T10:20:57.246 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:57.244+0000 7fb780400700 1 -- 172.21.15.153:0/1057747345 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb778100da0 msgr2=0x7fb778101180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:20:57.246 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:57.244+0000 7fb780400700 1 --2- 172.21.15.153:0/1057747345 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb778100da0 0x7fb778101180 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb76c009910 tx=0x7fb76c009c20 comp rx=0 tx=0).stop 2024-03-21T10:20:57.246 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:57.244+0000 7fb780400700 1 -- 172.21.15.153:0/1057747345 shutdown_connections 2024-03-21T10:20:57.246 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:57.244+0000 7fb780400700 1 --2- 172.21.15.153:0/1057747345 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb778101750 0x7fb7781057a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:57.246 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:57.244+0000 7fb780400700 1 --2- 172.21.15.153:0/1057747345 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb778100da0 0x7fb778101180 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:57.247 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:57.244+0000 7fb780400700 1 -- 172.21.15.153:0/1057747345 >> 172.21.15.153:0/1057747345 conn(0x7fb7780fc630 msgr2=0x7fb7780fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:20:57.247 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:57.244+0000 7fb780400700 1 -- 172.21.15.153:0/1057747345 shutdown_connections 2024-03-21T10:20:57.247 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:57.244+0000 7fb780400700 1 -- 172.21.15.153:0/1057747345 wait complete. 2024-03-21T10:20:57.247 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:57.245+0000 7fb780400700 1 Processor -- start 2024-03-21T10:20:57.338 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:57.246+0000 7fb780400700 1 -- start start 2024-03-21T10:20:57.341 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:57.246+0000 7fb780400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb778100da0 0x7fb778198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:20:57.341 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:57.246+0000 7fb780400700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb778101750 0x7fb778198ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:20:57.341 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:57.246+0000 7fb780400700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb778199560 con 0x7fb778100da0 2024-03-21T10:20:57.341 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:57.246+0000 7fb780400700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb77819d370 con 0x7fb778101750 2024-03-21T10:20:57.341 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:57.246+0000 7fb77da00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb778100da0 0x7fb778198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:20:57.342 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:57.246+0000 7fb77da00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb778100da0 0x7fb778198990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:37092/0 (socket says 172.21.15.153:37092) 2024-03-21T10:20:57.342 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:57.246+0000 7fb77da00700 1 -- 172.21.15.153:0/1281611340 learned_addr learned my addr 172.21.15.153:0/1281611340 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:20:57.342 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:57.247+0000 7fb77d000700 1 --2- 172.21.15.153:0/1281611340 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb778101750 0x7fb778198ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:20:57.343 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:57.247+0000 7fb77da00700 1 -- 172.21.15.153:0/1281611340 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb778101750 msgr2=0x7fb778198ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:20:57.343 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:57.247+0000 7fb77da00700 1 --2- 172.21.15.153:0/1281611340 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb778101750 0x7fb778198ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:57.343 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:57.247+0000 7fb77da00700 1 -- 172.21.15.153:0/1281611340 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb76c0095f0 con 0x7fb778100da0 2024-03-21T10:20:57.343 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:57.247+0000 7fb77da00700 1 --2- 172.21.15.153:0/1281611340 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb778100da0 0x7fb778198990 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb76c00f690 tx=0x7fb76c00f770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:20:57.343 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:57.247+0000 7fb776000700 1 -- 172.21.15.153:0/1281611340 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb76c01d070 con 0x7fb778100da0 2024-03-21T10:20:57.343 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:57.247+0000 7fb780400700 1 -- 172.21.15.153:0/1281611340 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb77819d5f0 con 0x7fb778100da0 2024-03-21T10:20:57.343 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:57.247+0000 7fb776000700 1 -- 172.21.15.153:0/1281611340 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb76c0050d0 con 0x7fb778100da0 2024-03-21T10:20:57.343 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:57.247+0000 7fb780400700 1 -- 172.21.15.153:0/1281611340 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb77819dae0 con 0x7fb778100da0 2024-03-21T10:20:57.343 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:57.248+0000 7fb776000700 1 -- 172.21.15.153:0/1281611340 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb76c022470 con 0x7fb778100da0 2024-03-21T10:20:57.343 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:57.248+0000 7fb780400700 1 -- 172.21.15.153:0/1281611340 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb778104f20 con 0x7fb778100da0 2024-03-21T10:20:57.343 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:57.250+0000 7fb776000700 1 -- 172.21.15.153:0/1281611340 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fb76c003680 con 0x7fb778100da0 2024-03-21T10:20:57.343 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:57.250+0000 7fb776000700 1 --2- 172.21.15.153:0/1281611340 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb76006e460 0x7fb760070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:20:57.343 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:57.251+0000 7fb776000700 1 -- 172.21.15.153:0/1281611340 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fb76c08fa50 con 0x7fb778100da0 2024-03-21T10:20:57.344 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:57.251+0000 7fb77d000700 1 --2- 172.21.15.153:0/1281611340 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb76006e460 0x7fb760070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:20:57.344 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:57.251+0000 7fb77d000700 1 --2- 172.21.15.153:0/1281611340 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb76006e460 0x7fb760070920 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb778199fb0 tx=0x7fb764009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:20:57.344 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:57.255+0000 7fb776000700 1 -- 172.21.15.153:0/1281611340 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb76c05c5a0 con 0x7fb778100da0 2024-03-21T10:20:57.554 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:57.551+0000 7fb780400700 1 -- 172.21.15.153:0/1281611340 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fb778199df0 con 0x7fb778100da0 2024-03-21T10:20:57.554 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:57.552+0000 7fb776000700 1 -- 172.21.15.153:0/1281611340 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+805 (secure 0 0 0) 0x7fb76c05c130 con 0x7fb778100da0 2024-03-21T10:20:57.555 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:20:57.555 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": { 2024-03-21T10:20:57.555 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:20:57.555 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:20:57.555 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": { 2024-03-21T10:20:57.555 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:20:57.555 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:20:57.555 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": { 2024-03-21T10:20:57.555 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 6 2024-03-21T10:20:57.555 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:20:57.555 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": { 2024-03-21T10:20:57.556 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-03-21T10:20:57.556 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 3 2024-03-21T10:20:57.556 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:20:57.556 INFO:teuthology.orchestra.run.smithi153.stdout: "overall": { 2024-03-21T10:20:57.556 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-03-21T10:20:57.556 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 13 2024-03-21T10:20:57.556 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-03-21T10:20:57.556 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:20:57.557 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:57.555+0000 7fb780400700 1 -- 172.21.15.153:0/1281611340 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb76006e460 msgr2=0x7fb760070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:20:57.557 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:57.555+0000 7fb780400700 1 --2- 172.21.15.153:0/1281611340 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb76006e460 0x7fb760070920 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb778199fb0 tx=0x7fb764009260 comp rx=0 tx=0).stop 2024-03-21T10:20:57.557 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:57.555+0000 7fb780400700 1 -- 172.21.15.153:0/1281611340 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb778100da0 msgr2=0x7fb778198990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:20:57.557 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:57.555+0000 7fb780400700 1 --2- 172.21.15.153:0/1281611340 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb778100da0 0x7fb778198990 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb76c00f690 tx=0x7fb76c00f770 comp rx=0 tx=0).stop 2024-03-21T10:20:57.557 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:57.555+0000 7fb780400700 1 -- 172.21.15.153:0/1281611340 shutdown_connections 2024-03-21T10:20:57.557 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:57.555+0000 7fb780400700 1 --2- 172.21.15.153:0/1281611340 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb76006e460 0x7fb760070920 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:57.557 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:57.555+0000 7fb780400700 1 --2- 172.21.15.153:0/1281611340 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb778100da0 0x7fb778198990 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:57.557 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:57.555+0000 7fb780400700 1 --2- 172.21.15.153:0/1281611340 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb778101750 0x7fb778198ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:57.557 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:57.555+0000 7fb780400700 1 -- 172.21.15.153:0/1281611340 >> 172.21.15.153:0/1281611340 conn(0x7fb7780fc630 msgr2=0x7fb7780fea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:20:57.558 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:57.555+0000 7fb780400700 1 -- 172.21.15.153:0/1281611340 shutdown_connections 2024-03-21T10:20:57.558 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:57.555+0000 7fb780400700 1 -- 172.21.15.153:0/1281611340 wait complete. 2024-03-21T10:20:57.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:20:57 smithi174 ceph-mon[88004]: from='client.34538 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:20:57.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:20:57 smithi174 ceph-mon[88004]: from='client.34542 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:20:57.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:20:57 smithi174 ceph-mon[88004]: pgmap v273: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:20:57.703 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:57.702+0000 7fd72ee00700 1 -- 172.21.15.153:0/2353665428 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd72810d280 msgr2=0x7fd72810d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:20:57.703 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:57.702+0000 7fd72ee00700 1 --2- 172.21.15.153:0/2353665428 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd72810d280 0x7fd72810d660 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd718009910 tx=0x7fd718009c20 comp rx=0 tx=0).stop 2024-03-21T10:20:57.703 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:20:57 smithi153 ceph-mon[113733]: from='client.34538 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:20:57.704 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:20:57 smithi153 ceph-mon[113733]: from='client.34542 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:20:57.704 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:20:57 smithi153 ceph-mon[113733]: pgmap v273: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:20:57.704 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:57.703+0000 7fd72ee00700 1 -- 172.21.15.153:0/2353665428 shutdown_connections 2024-03-21T10:20:57.704 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:57.703+0000 7fd72ee00700 1 --2- 172.21.15.153:0/2353665428 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd7281071d0 0x7fd728107650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:57.704 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:57.703+0000 7fd72ee00700 1 --2- 172.21.15.153:0/2353665428 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd72810d280 0x7fd72810d660 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:57.704 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:57.703+0000 7fd72ee00700 1 -- 172.21.15.153:0/2353665428 >> 172.21.15.153:0/2353665428 conn(0x7fd7280769e0 msgr2=0x7fd728078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:20:57.704 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:57.703+0000 7fd72ee00700 1 -- 172.21.15.153:0/2353665428 shutdown_connections 2024-03-21T10:20:57.704 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:57.703+0000 7fd72ee00700 1 -- 172.21.15.153:0/2353665428 wait complete. 2024-03-21T10:20:57.705 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:57.705+0000 7fd72ee00700 1 Processor -- start 2024-03-21T10:20:57.706 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:57.705+0000 7fd72ee00700 1 -- start start 2024-03-21T10:20:57.706 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:57.706+0000 7fd72ee00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd7281071d0 0x7fd72819ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:20:57.706 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:57.706+0000 7fd72ee00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd72810d280 0x7fd72819d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:20:57.706 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:57.706+0000 7fd72ee00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd72819d8a0 con 0x7fd7281071d0 2024-03-21T10:20:57.706 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:57.706+0000 7fd72ee00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7281a05d0 con 0x7fd72810d280 2024-03-21T10:20:57.708 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:57.706+0000 7fd727e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd7281071d0 0x7fd72819ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:20:57.708 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:57.706+0000 7fd727e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd7281071d0 0x7fd72819ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:37116/0 (socket says 172.21.15.153:37116) 2024-03-21T10:20:57.708 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:57.706+0000 7fd727e00700 1 -- 172.21.15.153:0/3713366344 learned_addr learned my addr 172.21.15.153:0/3713366344 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:20:57.709 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:57.706+0000 7fd727400700 1 --2- 172.21.15.153:0/3713366344 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd72810d280 0x7fd72819d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:20:57.709 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:57.707+0000 7fd727e00700 1 -- 172.21.15.153:0/3713366344 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd72810d280 msgr2=0x7fd72819d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:20:57.709 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:57.707+0000 7fd727e00700 1 --2- 172.21.15.153:0/3713366344 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd72810d280 0x7fd72819d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:57.709 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:57.707+0000 7fd727e00700 1 -- 172.21.15.153:0/3713366344 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7180095f0 con 0x7fd7281071d0 2024-03-21T10:20:57.709 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:57.707+0000 7fd727e00700 1 --2- 172.21.15.153:0/3713366344 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd7281071d0 0x7fd72819ccd0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd71800f690 tx=0x7fd71800f6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:20:57.709 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:57.707+0000 7fd724c00700 1 -- 172.21.15.153:0/3713366344 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd71801d070 con 0x7fd7281071d0 2024-03-21T10:20:57.709 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:57.707+0000 7fd72ee00700 1 -- 172.21.15.153:0/3713366344 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7281a0850 con 0x7fd7281071d0 2024-03-21T10:20:57.709 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:57.708+0000 7fd724c00700 1 -- 172.21.15.153:0/3713366344 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd7180050d0 con 0x7fd7281071d0 2024-03-21T10:20:57.709 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:57.708+0000 7fd724c00700 1 -- 172.21.15.153:0/3713366344 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd718022470 con 0x7fd7281071d0 2024-03-21T10:20:57.709 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:57.708+0000 7fd72ee00700 1 -- 172.21.15.153:0/3713366344 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7281a0dc0 con 0x7fd7281071d0 2024-03-21T10:20:57.710 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:57.709+0000 7fd72ee00700 1 -- 172.21.15.153:0/3713366344 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd72804ea90 con 0x7fd7281071d0 2024-03-21T10:20:57.711 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:57.710+0000 7fd724c00700 1 -- 172.21.15.153:0/3713366344 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fd718003780 con 0x7fd7281071d0 2024-03-21T10:20:57.712 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:57.711+0000 7fd724c00700 1 --2- 172.21.15.153:0/3713366344 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd70c06e460 0x7fd70c070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:20:57.712 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:57.712+0000 7fd724c00700 1 -- 172.21.15.153:0/3713366344 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fd71808f870 con 0x7fd7281071d0 2024-03-21T10:20:57.712 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:57.712+0000 7fd727400700 1 --2- 172.21.15.153:0/3713366344 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd70c06e460 0x7fd70c070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:20:57.713 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:57.712+0000 7fd727400700 1 --2- 172.21.15.153:0/3713366344 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd70c06e460 0x7fd70c070920 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd72819e2b0 tx=0x7fd714009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:20:57.715 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:57.714+0000 7fd724c00700 1 -- 172.21.15.153:0/3713366344 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd718058b10 con 0x7fd7281071d0 2024-03-21T10:20:57.982 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:57.981+0000 7fd72ee00700 1 -- 172.21.15.153:0/3713366344 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fd72819dff0 con 0x7fd7281071d0 2024-03-21T10:20:57.982 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:57.982+0000 7fd724c00700 1 -- 172.21.15.153:0/3713366344 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1876 (secure 0 0 0) 0x7fd71805c1d0 con 0x7fd7281071d0 2024-03-21T10:20:57.983 INFO:teuthology.orchestra.run.smithi153.stdout:e30 2024-03-21T10:20:57.983 INFO:teuthology.orchestra.run.smithi153.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-21T10:20:57.983 INFO:teuthology.orchestra.run.smithi153.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:20:57.983 INFO:teuthology.orchestra.run.smithi153.stdout:legacy client fscid: 1 2024-03-21T10:20:57.983 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:20:57.983 INFO:teuthology.orchestra.run.smithi153.stdout:Filesystem 'cephfs' (1) 2024-03-21T10:20:57.983 INFO:teuthology.orchestra.run.smithi153.stdout:fs_name cephfs 2024-03-21T10:20:57.983 INFO:teuthology.orchestra.run.smithi153.stdout:epoch 23 2024-03-21T10:20:57.984 INFO:teuthology.orchestra.run.smithi153.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-21T10:20:57.984 INFO:teuthology.orchestra.run.smithi153.stdout:created 2024-03-21T10:04:27.086576+0000 2024-03-21T10:20:57.984 INFO:teuthology.orchestra.run.smithi153.stdout:modified 2024-03-21T10:19:05.022092+0000 2024-03-21T10:20:57.984 INFO:teuthology.orchestra.run.smithi153.stdout:tableserver 0 2024-03-21T10:20:57.984 INFO:teuthology.orchestra.run.smithi153.stdout:root 0 2024-03-21T10:20:57.984 INFO:teuthology.orchestra.run.smithi153.stdout:session_timeout 60 2024-03-21T10:20:57.984 INFO:teuthology.orchestra.run.smithi153.stdout:session_autoclose 300 2024-03-21T10:20:57.984 INFO:teuthology.orchestra.run.smithi153.stdout:max_file_size 1099511627776 2024-03-21T10:20:57.984 INFO:teuthology.orchestra.run.smithi153.stdout:max_xattr_size 65536 2024-03-21T10:20:57.984 INFO:teuthology.orchestra.run.smithi153.stdout:required_client_features {} 2024-03-21T10:20:57.984 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure 0 2024-03-21T10:20:57.984 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure_osd_epoch 94 2024-03-21T10:20:57.985 INFO:teuthology.orchestra.run.smithi153.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:20:57.985 INFO:teuthology.orchestra.run.smithi153.stdout:max_mds 2 2024-03-21T10:20:57.985 INFO:teuthology.orchestra.run.smithi153.stdout:in 0,1 2024-03-21T10:20:57.985 INFO:teuthology.orchestra.run.smithi153.stdout:up {} 2024-03-21T10:20:57.985 INFO:teuthology.orchestra.run.smithi153.stdout:failed 0,1 2024-03-21T10:20:57.985 INFO:teuthology.orchestra.run.smithi153.stdout:damaged 2024-03-21T10:20:57.985 INFO:teuthology.orchestra.run.smithi153.stdout:stopped 2024-03-21T10:20:57.985 INFO:teuthology.orchestra.run.smithi153.stdout:data_pools [3] 2024-03-21T10:20:57.985 INFO:teuthology.orchestra.run.smithi153.stdout:metadata_pool 2 2024-03-21T10:20:57.985 INFO:teuthology.orchestra.run.smithi153.stdout:inline_data enabled 2024-03-21T10:20:57.985 INFO:teuthology.orchestra.run.smithi153.stdout:balancer 2024-03-21T10:20:57.985 INFO:teuthology.orchestra.run.smithi153.stdout:bal_rank_mask -1 2024-03-21T10:20:57.986 INFO:teuthology.orchestra.run.smithi153.stdout:standby_count_wanted 1 2024-03-21T10:20:57.986 INFO:teuthology.orchestra.run.smithi153.stdout:qdb_cluster leader: 0 members: 2024-03-21T10:20:57.986 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:20:57.986 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:20:57.986 INFO:teuthology.orchestra.run.smithi153.stdout:Standby daemons: 2024-03-21T10:20:57.986 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:20:57.986 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.zipckr{-1:34442} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6826/1103846295,v1:172.21.15.153:6827/1103846295] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:20:57.986 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.iuyluj{-1:34474} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6828/1738630832,v1:172.21.15.153:6829/1738630832] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:20:57.986 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.nhozag{-1:44269} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.174:6826/3967223227,v1:172.21.15.174:6827/3967223227] compat {c=[1],r=[1],i=[7ff]}] 2024-03-21T10:20:57.986 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.yyimst{-1:44313} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.174:6824/4127786797,v1:172.21.15.174:6825/4127786797] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:20:57.988 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:57.985+0000 7fd72ee00700 1 -- 172.21.15.153:0/3713366344 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd70c06e460 msgr2=0x7fd70c070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:20:57.988 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:57.985+0000 7fd72ee00700 1 --2- 172.21.15.153:0/3713366344 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd70c06e460 0x7fd70c070920 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd72819e2b0 tx=0x7fd714009260 comp rx=0 tx=0).stop 2024-03-21T10:20:57.988 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:57.985+0000 7fd72ee00700 1 -- 172.21.15.153:0/3713366344 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd7281071d0 msgr2=0x7fd72819ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:20:57.988 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:57.985+0000 7fd72ee00700 1 --2- 172.21.15.153:0/3713366344 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd7281071d0 0x7fd72819ccd0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd71800f690 tx=0x7fd71800f6c0 comp rx=0 tx=0).stop 2024-03-21T10:20:57.988 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:57.985+0000 7fd72ee00700 1 -- 172.21.15.153:0/3713366344 shutdown_connections 2024-03-21T10:20:57.988 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:57.985+0000 7fd72ee00700 1 --2- 172.21.15.153:0/3713366344 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd70c06e460 0x7fd70c070920 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:57.988 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:57.985+0000 7fd72ee00700 1 --2- 172.21.15.153:0/3713366344 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd7281071d0 0x7fd72819ccd0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:57.988 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:57.985+0000 7fd72ee00700 1 --2- 172.21.15.153:0/3713366344 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd72810d280 0x7fd72819d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:57.988 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:57.985+0000 7fd72ee00700 1 -- 172.21.15.153:0/3713366344 >> 172.21.15.153:0/3713366344 conn(0x7fd7280769e0 msgr2=0x7fd728078320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:20:57.988 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:57.985+0000 7fd72ee00700 1 -- 172.21.15.153:0/3713366344 shutdown_connections 2024-03-21T10:20:57.988 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:57.985+0000 7fd72ee00700 1 -- 172.21.15.153:0/3713366344 wait complete. 2024-03-21T10:20:57.989 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 30 2024-03-21T10:20:58.151 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:58.149+0000 7f8c27e00700 1 -- 172.21.15.153:0/1822327312 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8c20069020 msgr2=0x7f8c20105790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:20:58.151 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:58.149+0000 7f8c27e00700 1 --2- 172.21.15.153:0/1822327312 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8c20069020 0x7f8c20105790 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c009910 tx=0x7f8c0c009c20 comp rx=0 tx=0).stop 2024-03-21T10:20:58.151 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:58.150+0000 7f8c27e00700 1 -- 172.21.15.153:0/1822327312 shutdown_connections 2024-03-21T10:20:58.151 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:58.150+0000 7f8c27e00700 1 --2- 172.21.15.153:0/1822327312 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8c20069020 0x7f8c20105790 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:58.151 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:58.150+0000 7f8c27e00700 1 --2- 172.21.15.153:0/1822327312 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c20068670 0x7f8c20068a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:58.151 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:58.150+0000 7f8c27e00700 1 -- 172.21.15.153:0/1822327312 >> 172.21.15.153:0/1822327312 conn(0x7f8c200755a0 msgr2=0x7f8c200759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:20:58.151 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:58.150+0000 7f8c27e00700 1 -- 172.21.15.153:0/1822327312 shutdown_connections 2024-03-21T10:20:58.151 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:58.150+0000 7f8c27e00700 1 -- 172.21.15.153:0/1822327312 wait complete. 2024-03-21T10:20:58.152 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:58.151+0000 7f8c27e00700 1 Processor -- start 2024-03-21T10:20:58.152 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:58.151+0000 7f8c27e00700 1 -- start start 2024-03-21T10:20:58.152 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:58.151+0000 7f8c27e00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c20068670 0x7f8c2019e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:20:58.152 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:58.151+0000 7f8c27e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8c20069020 0x7f8c2019ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:20:58.152 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:58.151+0000 7f8c27e00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c2019f3b0 con 0x7f8c20069020 2024-03-21T10:20:58.152 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:58.151+0000 7f8c27e00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c20198860 con 0x7f8c20068670 2024-03-21T10:20:58.153 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:58.152+0000 7f8c24a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8c20069020 0x7f8c2019ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:20:58.154 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:58.152+0000 7f8c24a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8c20069020 0x7f8c2019ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:37132/0 (socket says 172.21.15.153:37132) 2024-03-21T10:20:58.154 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:58.152+0000 7f8c24a00700 1 -- 172.21.15.153:0/2994642350 learned_addr learned my addr 172.21.15.153:0/2994642350 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:20:58.154 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:58.152+0000 7f8c25400700 1 --2- 172.21.15.153:0/2994642350 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c20068670 0x7f8c2019e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:20:58.154 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:58.152+0000 7f8c24a00700 1 -- 172.21.15.153:0/2994642350 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c20068670 msgr2=0x7f8c2019e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:20:58.154 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:58.152+0000 7f8c24a00700 1 --2- 172.21.15.153:0/2994642350 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c20068670 0x7f8c2019e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:58.154 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:58.152+0000 7f8c24a00700 1 -- 172.21.15.153:0/2994642350 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c0c0095f0 con 0x7f8c20069020 2024-03-21T10:20:58.154 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:58.152+0000 7f8c24a00700 1 --2- 172.21.15.153:0/2994642350 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8c20069020 0x7f8c2019ed20 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c00bda0 tx=0x7f8c0c00be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:20:58.154 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:58.153+0000 7f8c1e000700 1 -- 172.21.15.153:0/2994642350 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8c0c01d070 con 0x7f8c20069020 2024-03-21T10:20:58.154 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:58.153+0000 7f8c1e000700 1 -- 172.21.15.153:0/2994642350 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8c0c004fb0 con 0x7f8c20069020 2024-03-21T10:20:58.155 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:58.153+0000 7f8c27e00700 1 -- 172.21.15.153:0/2994642350 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c20198ae0 con 0x7f8c20069020 2024-03-21T10:20:58.155 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:58.153+0000 7f8c1e000700 1 -- 172.21.15.153:0/2994642350 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8c0c00f460 con 0x7f8c20069020 2024-03-21T10:20:58.155 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:58.153+0000 7f8c27e00700 1 -- 172.21.15.153:0/2994642350 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c20198fd0 con 0x7f8c20069020 2024-03-21T10:20:58.156 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:58.155+0000 7f8c27e00700 1 -- 172.21.15.153:0/2994642350 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c2004ea90 con 0x7f8c20069020 2024-03-21T10:20:58.156 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:58.155+0000 7f8c1e000700 1 -- 172.21.15.153:0/2994642350 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f8c0c022800 con 0x7f8c20069020 2024-03-21T10:20:58.156 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:58.156+0000 7f8c1e000700 1 --2- 172.21.15.153:0/2994642350 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f8c0806e460 0x7f8c08070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:20:58.157 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:58.156+0000 7f8c1e000700 1 -- 172.21.15.153:0/2994642350 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f8c0c08faf0 con 0x7f8c20069020 2024-03-21T10:20:58.157 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:58.156+0000 7f8c25400700 1 --2- 172.21.15.153:0/2994642350 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f8c0806e460 0x7f8c08070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:20:58.158 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:58.157+0000 7f8c25400700 1 --2- 172.21.15.153:0/2994642350 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f8c0806e460 0x7f8c08070920 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8c14005fd0 tx=0x7f8c1400a560 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:20:58.161 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:58.160+0000 7f8c1e000700 1 -- 172.21.15.153:0/2994642350 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f8c0c05c810 con 0x7f8c20069020 2024-03-21T10:20:58.391 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:58.390+0000 7f8c27e00700 1 -- 172.21.15.153:0/2994642350 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8c20199300 con 0x7f8c0806e460 2024-03-21T10:20:58.392 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:58.391+0000 7f8c1e000700 1 -- 172.21.15.153:0/2994642350 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f8c20199300 con 0x7f8c0806e460 2024-03-21T10:20:58.393 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:20:58.393 INFO:teuthology.orchestra.run.smithi153.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232", 2024-03-21T10:20:58.393 INFO:teuthology.orchestra.run.smithi153.stdout: "in_progress": true, 2024-03-21T10:20:58.393 INFO:teuthology.orchestra.run.smithi153.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-21T10:20:58.393 INFO:teuthology.orchestra.run.smithi153.stdout: "services_complete": [ 2024-03-21T10:20:58.393 INFO:teuthology.orchestra.run.smithi153.stdout: "crash", 2024-03-21T10:20:58.393 INFO:teuthology.orchestra.run.smithi153.stdout: "mon", 2024-03-21T10:20:58.393 INFO:teuthology.orchestra.run.smithi153.stdout: "osd", 2024-03-21T10:20:58.394 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr" 2024-03-21T10:20:58.394 INFO:teuthology.orchestra.run.smithi153.stdout: ], 2024-03-21T10:20:58.394 INFO:teuthology.orchestra.run.smithi153.stdout: "progress": "15/23 daemons upgraded", 2024-03-21T10:20:58.394 INFO:teuthology.orchestra.run.smithi153.stdout: "message": "Currently upgrading mds daemons", 2024-03-21T10:20:58.394 INFO:teuthology.orchestra.run.smithi153.stdout: "is_paused": false 2024-03-21T10:20:58.394 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:20:58.396 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:58.395+0000 7f8c27e00700 1 -- 172.21.15.153:0/2994642350 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f8c0806e460 msgr2=0x7f8c08070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:20:58.397 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:58.395+0000 7f8c27e00700 1 --2- 172.21.15.153:0/2994642350 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f8c0806e460 0x7f8c08070920 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8c14005fd0 tx=0x7f8c1400a560 comp rx=0 tx=0).stop 2024-03-21T10:20:58.397 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:58.395+0000 7f8c27e00700 1 -- 172.21.15.153:0/2994642350 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8c20069020 msgr2=0x7f8c2019ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:20:58.397 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:58.395+0000 7f8c27e00700 1 --2- 172.21.15.153:0/2994642350 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8c20069020 0x7f8c2019ed20 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c00bda0 tx=0x7f8c0c00be80 comp rx=0 tx=0).stop 2024-03-21T10:20:58.397 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:58.395+0000 7f8c27e00700 1 -- 172.21.15.153:0/2994642350 shutdown_connections 2024-03-21T10:20:58.397 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:58.395+0000 7f8c27e00700 1 --2- 172.21.15.153:0/2994642350 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c20068670 0x7f8c2019e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:58.397 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:58.395+0000 7f8c27e00700 1 --2- 172.21.15.153:0/2994642350 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f8c0806e460 0x7f8c08070920 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:58.397 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:58.395+0000 7f8c27e00700 1 --2- 172.21.15.153:0/2994642350 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8c20069020 0x7f8c2019ed20 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:58.397 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:58.395+0000 7f8c27e00700 1 -- 172.21.15.153:0/2994642350 >> 172.21.15.153:0/2994642350 conn(0x7f8c200755a0 msgr2=0x7f8c20102760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:20:58.397 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:58.396+0000 7f8c27e00700 1 -- 172.21.15.153:0/2994642350 shutdown_connections 2024-03-21T10:20:58.397 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:58.396+0000 7f8c27e00700 1 -- 172.21.15.153:0/2994642350 wait complete. 2024-03-21T10:20:58.547 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:58.546+0000 7f329f000700 1 -- 172.21.15.153:0/2501920866 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32980ffbd0 msgr2=0x7f3298100050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:20:58.548 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:58.546+0000 7f329f000700 1 --2- 172.21.15.153:0/2501920866 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32980ffbd0 0x7f3298100050 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f3284009910 tx=0x7f3284009c20 comp rx=0 tx=0).stop 2024-03-21T10:20:58.548 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:58.546+0000 7f329f000700 1 -- 172.21.15.153:0/2501920866 shutdown_connections 2024-03-21T10:20:58.548 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:58.546+0000 7f329f000700 1 --2- 172.21.15.153:0/2501920866 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32980ffbd0 0x7f3298100050 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:58.548 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:58.546+0000 7f329f000700 1 --2- 172.21.15.153:0/2501920866 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3298105a40 0x7f3298105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:58.548 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:58.546+0000 7f329f000700 1 -- 172.21.15.153:0/2501920866 >> 172.21.15.153:0/2501920866 conn(0x7f32980755a0 msgr2=0x7f32980759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:20:58.548 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:58.546+0000 7f329f000700 1 -- 172.21.15.153:0/2501920866 shutdown_connections 2024-03-21T10:20:58.548 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:58.546+0000 7f329f000700 1 -- 172.21.15.153:0/2501920866 wait complete. 2024-03-21T10:20:58.548 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:58.547+0000 7f329f000700 1 Processor -- start 2024-03-21T10:20:58.548 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:58.547+0000 7f329f000700 1 -- start start 2024-03-21T10:20:58.550 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:58.547+0000 7f329f000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32980ffbd0 0x7f3298101610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:20:58.550 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:58.547+0000 7f329f000700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3298105a40 0x7f3298101b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:20:58.550 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:58.547+0000 7f329f000700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3298102090 con 0x7f32980ffbd0 2024-03-21T10:20:58.550 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:58.547+0000 7f329f000700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32981021d0 con 0x7f3298105a40 2024-03-21T10:20:58.550 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:58.547+0000 7f3297e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32980ffbd0 0x7f3298101610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:20:58.550 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:58.548+0000 7f3297e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32980ffbd0 0x7f3298101610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:37152/0 (socket says 172.21.15.153:37152) 2024-03-21T10:20:58.550 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:58.548+0000 7f3297e00700 1 -- 172.21.15.153:0/3755407758 learned_addr learned my addr 172.21.15.153:0/3755407758 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:20:58.550 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:58.548+0000 7f3297e00700 1 -- 172.21.15.153:0/3755407758 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3298105a40 msgr2=0x7f3298101b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:20:58.550 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:58.548+0000 7f3297e00700 1 --2- 172.21.15.153:0/3755407758 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3298105a40 0x7f3298101b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:58.551 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:58.548+0000 7f3297e00700 1 -- 172.21.15.153:0/3755407758 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32840095f0 con 0x7f32980ffbd0 2024-03-21T10:20:58.551 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:58.548+0000 7f3297e00700 1 --2- 172.21.15.153:0/3755407758 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32980ffbd0 0x7f3298101610 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f328800c930 tx=0x7f328800ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:20:58.551 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:58.548+0000 7f3294c00700 1 -- 172.21.15.153:0/3755407758 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3288007ad0 con 0x7f32980ffbd0 2024-03-21T10:20:58.551 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:58.548+0000 7f3294c00700 1 -- 172.21.15.153:0/3755407758 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3288007c30 con 0x7f32980ffbd0 2024-03-21T10:20:58.551 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:58.548+0000 7f3294c00700 1 -- 172.21.15.153:0/3755407758 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3288018610 con 0x7f32980ffbd0 2024-03-21T10:20:58.551 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:58.548+0000 7f329f000700 1 -- 172.21.15.153:0/3755407758 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32981a28f0 con 0x7f32980ffbd0 2024-03-21T10:20:58.551 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:58.549+0000 7f329f000700 1 -- 172.21.15.153:0/3755407758 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32981a2c50 con 0x7f32980ffbd0 2024-03-21T10:20:58.552 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:58.550+0000 7f329f000700 1 -- 172.21.15.153:0/3755407758 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3298108fe0 con 0x7f32980ffbd0 2024-03-21T10:20:58.552 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:58.551+0000 7f3294c00700 1 -- 172.21.15.153:0/3755407758 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f328800f450 con 0x7f32980ffbd0 2024-03-21T10:20:58.552 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:58.552+0000 7f3294c00700 1 --2- 172.21.15.153:0/3755407758 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f327c06e460 0x7f327c070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:20:58.553 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:58.552+0000 7f3294c00700 1 -- 172.21.15.153:0/3755407758 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f328808e6e0 con 0x7f32980ffbd0 2024-03-21T10:20:58.553 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:58.552+0000 7f3297400700 1 --2- 172.21.15.153:0/3755407758 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f327c06e460 0x7f327c070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:20:58.553 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:58.553+0000 7f3297400700 1 --2- 172.21.15.153:0/3755407758 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f327c06e460 0x7f327c070920 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f328400b5c0 tx=0x7f3284005960 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:20:58.557 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:58.557+0000 7f3294c00700 1 -- 172.21.15.153:0/3755407758 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f3288057780 con 0x7f32980ffbd0 2024-03-21T10:20:58.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:20:58 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/1281611340' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:20:58.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:20:58 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/3713366344' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:20:58.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:20:58 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/1281611340' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:20:58.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:20:58 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/3713366344' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:20:58.858 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:58.857+0000 7f329f000700 1 -- 172.21.15.153:0/3755407758 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f329804ea90 con 0x7f32980ffbd0 2024-03-21T10:20:58.859 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:58.858+0000 7f3294c00700 1 -- 172.21.15.153:0/3755407758 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f328805ae40 con 0x7f32980ffbd0 2024-03-21T10:20:58.859 INFO:teuthology.orchestra.run.smithi153.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-03-21T10:20:58.859 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-21T10:20:58.859 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is degraded 2024-03-21T10:20:58.859 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-21T10:20:58.859 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-21T10:20:58.859 INFO:teuthology.orchestra.run.smithi153.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-21T10:20:58.859 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-21T10:20:58.862 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:58.861+0000 7f329f000700 1 -- 172.21.15.153:0/3755407758 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f327c06e460 msgr2=0x7f327c070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:20:58.862 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:58.861+0000 7f329f000700 1 --2- 172.21.15.153:0/3755407758 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f327c06e460 0x7f327c070920 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f328400b5c0 tx=0x7f3284005960 comp rx=0 tx=0).stop 2024-03-21T10:20:58.862 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:58.861+0000 7f329f000700 1 -- 172.21.15.153:0/3755407758 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32980ffbd0 msgr2=0x7f3298101610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:20:58.863 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:58.861+0000 7f329f000700 1 --2- 172.21.15.153:0/3755407758 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32980ffbd0 0x7f3298101610 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f328800c930 tx=0x7f328800ccf0 comp rx=0 tx=0).stop 2024-03-21T10:20:58.863 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:58.861+0000 7f329f000700 1 -- 172.21.15.153:0/3755407758 shutdown_connections 2024-03-21T10:20:58.863 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:58.861+0000 7f329f000700 1 --2- 172.21.15.153:0/3755407758 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f327c06e460 0x7f327c070920 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:58.863 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:58.861+0000 7f329f000700 1 --2- 172.21.15.153:0/3755407758 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32980ffbd0 0x7f3298101610 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:58.863 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:58.861+0000 7f329f000700 1 --2- 172.21.15.153:0/3755407758 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3298105a40 0x7f3298101b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:20:58.863 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:58.861+0000 7f329f000700 1 -- 172.21.15.153:0/3755407758 >> 172.21.15.153:0/3755407758 conn(0x7f32980755a0 msgr2=0x7f3298068650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:20:58.863 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:58.861+0000 7f329f000700 1 -- 172.21.15.153:0/3755407758 shutdown_connections 2024-03-21T10:20:58.863 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:20:58.862+0000 7f329f000700 1 -- 172.21.15.153:0/3755407758 wait complete. 2024-03-21T10:20:59.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:20:59 smithi174 ceph-mon[88004]: from='client.34554 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:20:59.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:20:59 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/3755407758' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:20:59.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:20:59 smithi174 ceph-mon[88004]: pgmap v274: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:20:59.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:20:59 smithi153 ceph-mon[113733]: from='client.34554 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:20:59.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:20:59 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/3755407758' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:20:59.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:20:59 smithi153 ceph-mon[113733]: pgmap v274: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:21:01.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:21:01 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:21:01.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:21:01 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:21:02.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:21:02 smithi153 ceph-mon[113733]: pgmap v275: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:21:02.604 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:21:02 smithi174 ceph-mon[88004]: pgmap v275: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:21:03.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:21:03 smithi153 ceph-mon[113733]: pgmap v276: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:21:03.638 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:21:03 smithi174 ceph-mon[88004]: pgmap v276: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:21:05.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:21:05 smithi174 ceph-mon[88004]: pgmap v277: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:21:05.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:21:05 smithi153 ceph-mon[113733]: pgmap v277: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:21:07.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:21:07 smithi174 ceph-mon[88004]: pgmap v278: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:21:07.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:21:07 smithi153 ceph-mon[113733]: pgmap v278: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:21:09.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:21:09 smithi174 ceph-mon[88004]: pgmap v279: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:21:09.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:21:09 smithi153 ceph-mon[113733]: pgmap v279: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:21:12.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:21:11 smithi153 ceph-mon[113733]: pgmap v280: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:21:12.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:21:11 smithi174 ceph-mon[88004]: pgmap v280: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:21:13.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:21:13 smithi174 ceph-mon[88004]: pgmap v281: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:21:13.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:21:13 smithi153 ceph-mon[113733]: pgmap v281: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:21:15.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:21:15 smithi174 ceph-mon[88004]: pgmap v282: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:21:15.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:21:15 smithi153 ceph-mon[113733]: pgmap v282: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:21:16.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:21:16 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:21:16.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:21:16 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:21:17.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:21:17 smithi174 ceph-mon[88004]: pgmap v283: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:21:17.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:21:17 smithi153 ceph-mon[113733]: pgmap v283: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:21:19.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:21:19 smithi174 ceph-mon[88004]: pgmap v284: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:21:19.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:21:19 smithi153 ceph-mon[113733]: pgmap v284: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:21:21.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:21:21 smithi174 ceph-mon[88004]: pgmap v285: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:21:21.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:21:21 smithi153 ceph-mon[113733]: pgmap v285: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:21:23.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:21:23 smithi174 ceph-mon[88004]: pgmap v286: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:21:23.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:21:23 smithi153 ceph-mon[113733]: pgmap v286: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:21:25.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:21:25 smithi174 ceph-mon[88004]: pgmap v287: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:21:25.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:21:25 smithi153 ceph-mon[113733]: pgmap v287: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:21:27.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:21:27 smithi174 ceph-mon[88004]: pgmap v288: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:21:27.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:21:27 smithi153 ceph-mon[113733]: pgmap v288: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:21:29.042 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.040+0000 7fb691600700 1 -- 172.21.15.153:0/3510840499 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb68c102620 msgr2=0x7fb68c10ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:29.043 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.040+0000 7fb691600700 1 --2- 172.21.15.153:0/3510840499 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb68c102620 0x7fb68c10ab10 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb678009910 tx=0x7fb678009c20 comp rx=0 tx=0).stop 2024-03-21T10:21:29.043 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.041+0000 7fb691600700 1 -- 172.21.15.153:0/3510840499 shutdown_connections 2024-03-21T10:21:29.043 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.041+0000 7fb691600700 1 --2- 172.21.15.153:0/3510840499 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb68c102620 0x7fb68c10ab10 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:29.043 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.041+0000 7fb691600700 1 --2- 172.21.15.153:0/3510840499 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb68c101d00 0x7fb68c1020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:29.043 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.041+0000 7fb691600700 1 -- 172.21.15.153:0/3510840499 >> 172.21.15.153:0/3510840499 conn(0x7fb68c0fb570 msgr2=0x7fb68c0fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:21:29.043 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.041+0000 7fb691600700 1 -- 172.21.15.153:0/3510840499 shutdown_connections 2024-03-21T10:21:29.043 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.041+0000 7fb691600700 1 -- 172.21.15.153:0/3510840499 wait complete. 2024-03-21T10:21:29.044 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.042+0000 7fb691600700 1 Processor -- start 2024-03-21T10:21:29.044 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.042+0000 7fb691600700 1 -- start start 2024-03-21T10:21:29.044 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.043+0000 7fb691600700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb68c101d00 0x7fb68c0ff960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:29.045 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.043+0000 7fb691600700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb68c102620 0x7fb68c0ffea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:29.045 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.043+0000 7fb691600700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb68c1003e0 con 0x7fb68c102620 2024-03-21T10:21:29.045 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.043+0000 7fb691600700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb68c100520 con 0x7fb68c101d00 2024-03-21T10:21:29.045 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.043+0000 7fb68a000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb68c102620 0x7fb68c0ffea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:29.045 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.043+0000 7fb68a000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb68c102620 0x7fb68c0ffea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:33562/0 (socket says 172.21.15.153:33562) 2024-03-21T10:21:29.045 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.043+0000 7fb68a000700 1 -- 172.21.15.153:0/2385011860 learned_addr learned my addr 172.21.15.153:0/2385011860 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:21:29.046 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.043+0000 7fb68aa00700 1 --2- 172.21.15.153:0/2385011860 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb68c101d00 0x7fb68c0ff960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:29.046 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.044+0000 7fb68a000700 1 -- 172.21.15.153:0/2385011860 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb68c101d00 msgr2=0x7fb68c0ff960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:29.046 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.044+0000 7fb68a000700 1 --2- 172.21.15.153:0/2385011860 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb68c101d00 0x7fb68c0ff960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:29.046 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.044+0000 7fb68a000700 1 -- 172.21.15.153:0/2385011860 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6780095f0 con 0x7fb68c102620 2024-03-21T10:21:29.047 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.044+0000 7fb68a000700 1 --2- 172.21.15.153:0/2385011860 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb68c102620 0x7fb68c0ffea0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb67800bd70 tx=0x7fb67800bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:21:29.047 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.045+0000 7fb683400700 1 -- 172.21.15.153:0/2385011860 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb67801d070 con 0x7fb68c102620 2024-03-21T10:21:29.047 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.045+0000 7fb683400700 1 -- 172.21.15.153:0/2385011860 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb678005dc0 con 0x7fb68c102620 2024-03-21T10:21:29.047 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.045+0000 7fb691600700 1 -- 172.21.15.153:0/2385011860 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb68c1007a0 con 0x7fb68c102620 2024-03-21T10:21:29.047 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.045+0000 7fb691600700 1 -- 172.21.15.153:0/2385011860 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb68c1a2cb0 con 0x7fb68c102620 2024-03-21T10:21:29.047 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.046+0000 7fb683400700 1 -- 172.21.15.153:0/2385011860 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb67800fe90 con 0x7fb68c102620 2024-03-21T10:21:29.048 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.047+0000 7fb691600700 1 -- 172.21.15.153:0/2385011860 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb68c108210 con 0x7fb68c102620 2024-03-21T10:21:29.049 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.048+0000 7fb683400700 1 -- 172.21.15.153:0/2385011860 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fb67800f6d0 con 0x7fb68c102620 2024-03-21T10:21:29.050 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.048+0000 7fb683400700 1 --2- 172.21.15.153:0/2385011860 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb67006e460 0x7fb670070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:29.050 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.049+0000 7fb683400700 1 -- 172.21.15.153:0/2385011860 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fb67808f960 con 0x7fb68c102620 2024-03-21T10:21:29.051 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.049+0000 7fb68aa00700 1 --2- 172.21.15.153:0/2385011860 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb67006e460 0x7fb670070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:29.051 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.050+0000 7fb68aa00700 1 --2- 172.21.15.153:0/2385011860 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb67006e460 0x7fb670070920 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb67c005950 tx=0x7fb67c00b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:21:29.059 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.058+0000 7fb683400700 1 -- 172.21.15.153:0/2385011860 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb678058a30 con 0x7fb68c102620 2024-03-21T10:21:29.301 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.299+0000 7fb691600700 1 -- 172.21.15.153:0/2385011860 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb68c0611d0 con 0x7fb67006e460 2024-03-21T10:21:29.304 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.302+0000 7fb683400700 1 -- 172.21.15.153:0/2385011860 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fb68c0611d0 con 0x7fb67006e460 2024-03-21T10:21:29.307 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.305+0000 7fb691600700 1 -- 172.21.15.153:0/2385011860 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb67006e460 msgr2=0x7fb670070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:29.308 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.305+0000 7fb691600700 1 --2- 172.21.15.153:0/2385011860 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb67006e460 0x7fb670070920 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb67c005950 tx=0x7fb67c00b410 comp rx=0 tx=0).stop 2024-03-21T10:21:29.308 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.305+0000 7fb691600700 1 -- 172.21.15.153:0/2385011860 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb68c102620 msgr2=0x7fb68c0ffea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:29.308 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.305+0000 7fb691600700 1 --2- 172.21.15.153:0/2385011860 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb68c102620 0x7fb68c0ffea0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb67800bd70 tx=0x7fb67800bda0 comp rx=0 tx=0).stop 2024-03-21T10:21:29.308 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.305+0000 7fb691600700 1 -- 172.21.15.153:0/2385011860 shutdown_connections 2024-03-21T10:21:29.308 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.305+0000 7fb691600700 1 --2- 172.21.15.153:0/2385011860 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb68c101d00 0x7fb68c0ff960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:29.308 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.306+0000 7fb691600700 1 --2- 172.21.15.153:0/2385011860 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb67006e460 0x7fb670070920 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:29.308 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.306+0000 7fb691600700 1 --2- 172.21.15.153:0/2385011860 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb68c102620 0x7fb68c0ffea0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:29.308 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.306+0000 7fb691600700 1 -- 172.21.15.153:0/2385011860 >> 172.21.15.153:0/2385011860 conn(0x7fb68c0fb570 msgr2=0x7fb68c105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:21:29.308 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.306+0000 7fb691600700 1 -- 172.21.15.153:0/2385011860 shutdown_connections 2024-03-21T10:21:29.308 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.306+0000 7fb691600700 1 -- 172.21.15.153:0/2385011860 wait complete. 2024-03-21T10:21:29.323 INFO:teuthology.orchestra.run.smithi153.stdout:true 2024-03-21T10:21:29.465 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.463+0000 7f0857a00700 1 -- 172.21.15.153:0/672951161 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0850073160 msgr2=0x7f0850073540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:29.465 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.463+0000 7f0857a00700 1 --2- 172.21.15.153:0/672951161 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0850073160 0x7f0850073540 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f0840009910 tx=0x7f0840009c20 comp rx=0 tx=0).stop 2024-03-21T10:21:29.465 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.463+0000 7f0857a00700 1 -- 172.21.15.153:0/672951161 shutdown_connections 2024-03-21T10:21:29.465 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.463+0000 7f0857a00700 1 --2- 172.21.15.153:0/672951161 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0850073a80 0x7f085010cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:29.465 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.463+0000 7f0857a00700 1 --2- 172.21.15.153:0/672951161 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0850073160 0x7f0850073540 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:29.465 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.463+0000 7f0857a00700 1 -- 172.21.15.153:0/672951161 >> 172.21.15.153:0/672951161 conn(0x7f08500fc6a0 msgr2=0x7f08500feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:21:29.465 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.463+0000 7f0857a00700 1 -- 172.21.15.153:0/672951161 shutdown_connections 2024-03-21T10:21:29.465 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.463+0000 7f0857a00700 1 -- 172.21.15.153:0/672951161 wait complete. 2024-03-21T10:21:29.466 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.464+0000 7f0857a00700 1 Processor -- start 2024-03-21T10:21:29.466 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.465+0000 7f0857a00700 1 -- start start 2024-03-21T10:21:29.467 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.465+0000 7f0857a00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0850073160 0x7f085019e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:29.467 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.465+0000 7f0857a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0850073a80 0x7f085019ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:29.467 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.465+0000 7f0857a00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f085019f3c0 con 0x7f0850073a80 2024-03-21T10:21:29.467 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.465+0000 7f0857a00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0850198820 con 0x7f0850073160 2024-03-21T10:21:29.468 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.465+0000 7f0847e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0850073a80 0x7f085019ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:29.468 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.466+0000 7f0847e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0850073a80 0x7f085019ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:33586/0 (socket says 172.21.15.153:33586) 2024-03-21T10:21:29.468 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.466+0000 7f0847e00700 1 -- 172.21.15.153:0/4225761105 learned_addr learned my addr 172.21.15.153:0/4225761105 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:21:29.468 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.466+0000 7f0855000700 1 --2- 172.21.15.153:0/4225761105 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0850073160 0x7f085019e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:29.468 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.466+0000 7f0847e00700 1 -- 172.21.15.153:0/4225761105 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0850073160 msgr2=0x7f085019e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:29.468 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.466+0000 7f0847e00700 1 --2- 172.21.15.153:0/4225761105 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0850073160 0x7f085019e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:29.468 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.466+0000 7f0847e00700 1 -- 172.21.15.153:0/4225761105 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08400095f0 con 0x7f0850073a80 2024-03-21T10:21:29.468 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.466+0000 7f0855000700 1 --2- 172.21.15.153:0/4225761105 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0850073160 0x7f085019e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:21:29.468 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.466+0000 7f0847e00700 1 --2- 172.21.15.153:0/4225761105 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0850073a80 0x7f085019ed70 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f083800c930 tx=0x7f083800ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:21:29.469 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.466+0000 7f084e000700 1 -- 172.21.15.153:0/4225761105 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0838007ad0 con 0x7f0850073a80 2024-03-21T10:21:29.469 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.467+0000 7f084e000700 1 -- 172.21.15.153:0/4225761105 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0838007c30 con 0x7f0850073a80 2024-03-21T10:21:29.469 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.467+0000 7f0857a00700 1 -- 172.21.15.153:0/4225761105 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0850198b00 con 0x7f0850073a80 2024-03-21T10:21:29.470 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.467+0000 7f084e000700 1 -- 172.21.15.153:0/4225761105 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0838018610 con 0x7f0850073a80 2024-03-21T10:21:29.470 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.467+0000 7f0857a00700 1 -- 172.21.15.153:0/4225761105 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0850199050 con 0x7f0850073a80 2024-03-21T10:21:29.470 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.468+0000 7f0857a00700 1 -- 172.21.15.153:0/4225761105 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f085010a5f0 con 0x7f0850073a80 2024-03-21T10:21:29.470 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.469+0000 7f084e000700 1 -- 172.21.15.153:0/4225761105 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f083800f450 con 0x7f0850073a80 2024-03-21T10:21:29.472 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.470+0000 7f084e000700 1 --2- 172.21.15.153:0/4225761105 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f083406e460 0x7f0834070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:29.472 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.470+0000 7f084e000700 1 -- 172.21.15.153:0/4225761105 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f083808e7b0 con 0x7f0850073a80 2024-03-21T10:21:29.472 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.470+0000 7f0855000700 1 --2- 172.21.15.153:0/4225761105 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f083406e460 0x7f0834070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:29.472 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.471+0000 7f0855000700 1 --2- 172.21.15.153:0/4225761105 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f083406e460 0x7f0834070920 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0840006010 tx=0x7f084000b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:21:29.476 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.475+0000 7f084e000700 1 -- 172.21.15.153:0/4225761105 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f08380577d0 con 0x7f0850073a80 2024-03-21T10:21:29.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:21:29 smithi153 ceph-mon[113733]: pgmap v289: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:21:29.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:21:29 smithi153 ceph-mon[113733]: from='client.34562 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:21:29.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:21:29 smithi174 ceph-mon[88004]: pgmap v289: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:21:29.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:21:29 smithi174 ceph-mon[88004]: from='client.34562 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:21:29.719 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.717+0000 7f0857a00700 1 -- 172.21.15.153:0/4225761105 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f08501992d0 con 0x7f083406e460 2024-03-21T10:21:29.721 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.719+0000 7f084e000700 1 -- 172.21.15.153:0/4225761105 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f08501992d0 con 0x7f083406e460 2024-03-21T10:21:29.724 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.722+0000 7f0857a00700 1 -- 172.21.15.153:0/4225761105 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f083406e460 msgr2=0x7f0834070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:29.724 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.722+0000 7f0857a00700 1 --2- 172.21.15.153:0/4225761105 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f083406e460 0x7f0834070920 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0840006010 tx=0x7f084000b540 comp rx=0 tx=0).stop 2024-03-21T10:21:29.724 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.722+0000 7f0857a00700 1 -- 172.21.15.153:0/4225761105 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0850073a80 msgr2=0x7f085019ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:29.724 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.722+0000 7f0857a00700 1 --2- 172.21.15.153:0/4225761105 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0850073a80 0x7f085019ed70 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f083800c930 tx=0x7f083800ccf0 comp rx=0 tx=0).stop 2024-03-21T10:21:29.724 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.722+0000 7f0857a00700 1 -- 172.21.15.153:0/4225761105 shutdown_connections 2024-03-21T10:21:29.725 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.722+0000 7f0857a00700 1 --2- 172.21.15.153:0/4225761105 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0850073160 0x7f085019e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:29.725 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.722+0000 7f0857a00700 1 --2- 172.21.15.153:0/4225761105 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f083406e460 0x7f0834070920 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:29.725 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.722+0000 7f0857a00700 1 --2- 172.21.15.153:0/4225761105 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0850073a80 0x7f085019ed70 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:29.725 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.722+0000 7f0857a00700 1 -- 172.21.15.153:0/4225761105 >> 172.21.15.153:0/4225761105 conn(0x7f08500fc6a0 msgr2=0x7f0850107730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:21:29.725 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.723+0000 7f0857a00700 1 -- 172.21.15.153:0/4225761105 shutdown_connections 2024-03-21T10:21:29.725 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.723+0000 7f0857a00700 1 -- 172.21.15.153:0/4225761105 wait complete. 2024-03-21T10:21:29.879 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.877+0000 7f1722000700 1 -- 172.21.15.153:0/835151726 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f171c10d280 msgr2=0x7f171c10d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:29.879 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.877+0000 7f1722000700 1 --2- 172.21.15.153:0/835151726 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f171c10d280 0x7f171c10d660 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f170c009910 tx=0x7f170c009c20 comp rx=0 tx=0).stop 2024-03-21T10:21:29.879 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.877+0000 7f1722000700 1 -- 172.21.15.153:0/835151726 shutdown_connections 2024-03-21T10:21:29.879 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.877+0000 7f1722000700 1 --2- 172.21.15.153:0/835151726 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f171c1071d0 0x7f171c107650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:29.880 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.877+0000 7f1722000700 1 --2- 172.21.15.153:0/835151726 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f171c10d280 0x7f171c10d660 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:29.880 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.877+0000 7f1722000700 1 -- 172.21.15.153:0/835151726 >> 172.21.15.153:0/835151726 conn(0x7f171c0769e0 msgr2=0x7f171c078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:21:29.880 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.877+0000 7f1722000700 1 -- 172.21.15.153:0/835151726 shutdown_connections 2024-03-21T10:21:29.880 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.877+0000 7f1722000700 1 -- 172.21.15.153:0/835151726 wait complete. 2024-03-21T10:21:29.881 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.878+0000 7f1722000700 1 Processor -- start 2024-03-21T10:21:29.881 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.878+0000 7f1722000700 1 -- start start 2024-03-21T10:21:29.881 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.879+0000 7f1722000700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f171c1071d0 0x7f171c1a2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:29.881 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.879+0000 7f1722000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f171c10d280 0x7f171c1a3010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:29.881 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.879+0000 7f1722000700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f171c1a36a0 con 0x7f171c10d280 2024-03-21T10:21:29.881 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.879+0000 7f1722000700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f171c19cb50 con 0x7f171c1071d0 2024-03-21T10:21:29.882 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.879+0000 7f171aa00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f171c10d280 0x7f171c1a3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:29.882 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.879+0000 7f171aa00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f171c10d280 0x7f171c1a3010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:33602/0 (socket says 172.21.15.153:33602) 2024-03-21T10:21:29.882 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.879+0000 7f171aa00700 1 -- 172.21.15.153:0/2685133639 learned_addr learned my addr 172.21.15.153:0/2685133639 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:21:29.882 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.879+0000 7f171b400700 1 --2- 172.21.15.153:0/2685133639 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f171c1071d0 0x7f171c1a2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:29.882 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.879+0000 7f171aa00700 1 -- 172.21.15.153:0/2685133639 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f171c1071d0 msgr2=0x7f171c1a2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:29.882 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.880+0000 7f171aa00700 1 --2- 172.21.15.153:0/2685133639 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f171c1071d0 0x7f171c1a2ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:29.882 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.880+0000 7f171aa00700 1 -- 172.21.15.153:0/2685133639 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f170c0095f0 con 0x7f171c10d280 2024-03-21T10:21:29.883 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.880+0000 7f171aa00700 1 --2- 172.21.15.153:0/2685133639 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f171c10d280 0x7f171c1a3010 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f170800c8f0 tx=0x7f170800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:21:29.883 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.880+0000 7f1713e00700 1 -- 172.21.15.153:0/2685133639 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1708007a70 con 0x7f171c10d280 2024-03-21T10:21:29.884 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.880+0000 7f1713e00700 1 -- 172.21.15.153:0/2685133639 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1708004cf0 con 0x7f171c10d280 2024-03-21T10:21:29.884 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.880+0000 7f1722000700 1 -- 172.21.15.153:0/2685133639 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f171c19ce30 con 0x7f171c10d280 2024-03-21T10:21:29.884 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.880+0000 7f1713e00700 1 -- 172.21.15.153:0/2685133639 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1708005610 con 0x7f171c10d280 2024-03-21T10:21:29.884 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.880+0000 7f1722000700 1 -- 172.21.15.153:0/2685133639 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f171c19d400 con 0x7f171c10d280 2024-03-21T10:21:29.884 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.882+0000 7f1722000700 1 -- 172.21.15.153:0/2685133639 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f171c04ea90 con 0x7f171c10d280 2024-03-21T10:21:29.885 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.883+0000 7f1713e00700 1 -- 172.21.15.153:0/2685133639 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f1708004750 con 0x7f171c10d280 2024-03-21T10:21:29.885 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.883+0000 7f1713e00700 1 --2- 172.21.15.153:0/2685133639 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f170006e460 0x7f1700070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:29.886 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.884+0000 7f1713e00700 1 -- 172.21.15.153:0/2685133639 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f1708013070 con 0x7f171c10d280 2024-03-21T10:21:29.886 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.884+0000 7f171b400700 1 --2- 172.21.15.153:0/2685133639 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f170006e460 0x7f1700070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:29.886 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.885+0000 7f171b400700 1 --2- 172.21.15.153:0/2685133639 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f170006e460 0x7f1700070920 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f170c000c00 tx=0x7f170c005fd0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:21:29.890 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:29.888+0000 7f1713e00700 1 -- 172.21.15.153:0/2685133639 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f170805af30 con 0x7f171c10d280 2024-03-21T10:21:30.108 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:30.106+0000 7f1722000700 1 -- 172.21.15.153:0/2685133639 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f171c19d7f0 con 0x7f170006e460 2024-03-21T10:21:30.125 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:30.124+0000 7f1713e00700 1 -- 172.21.15.153:0/2685133639 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f171c19d7f0 con 0x7f170006e460 2024-03-21T10:21:30.126 INFO:teuthology.orchestra.run.smithi153.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-21T10:21:30.126 INFO:teuthology.orchestra.run.smithi153.stdout:alertmanager.smithi153 smithi153 *:9093,9094 running (10m) 103s ago 24m 14.7M - 0.25.0 c8568f914cd2 9ed3f2200b5f 2024-03-21T10:21:30.126 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi153 smithi153 running (24m) 103s ago 24m 9424k - 18.2.1 5be31c24972a 5b61994396a6 2024-03-21T10:21:30.126 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi174 smithi174 running (22m) 82s ago 22m 9244k - 18.2.1 5be31c24972a 4f7a5fc3ac28 2024-03-21T10:21:30.126 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi153 smithi153 running (6m) 103s ago 24m 6656k - 19.0.0-2231-g2edfee68 237e43c7a65f 32c21104d5b0 2024-03-21T10:21:30.126 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi174 smithi174 running (6m) 82s ago 22m 6660k - 19.0.0-2231-g2edfee68 237e43c7a65f a04e48967ab6 2024-03-21T10:21:30.127 INFO:teuthology.orchestra.run.smithi153.stdout:grafana.smithi153 smithi153 *:3000 running (9m) 103s ago 23m 73.6M - 9.4.12 2bacad6d85d8 ee24026ab786 2024-03-21T10:21:30.127 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.iuyluj smithi153 running (111s) 103s ago 16m 10.9M - 19.0.0-2231-g2edfee68 237e43c7a65f 87aff8a5de70 2024-03-21T10:21:30.127 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.zipckr smithi153 running (2m) 103s ago 16m 11.8M - 19.0.0-2231-g2edfee68 237e43c7a65f ccc262bd5dd3 2024-03-21T10:21:30.127 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.nhozag smithi174 running (16m) 82s ago 16m 12.0M - 18.2.1 5be31c24972a 2fee4000308f 2024-03-21T10:21:30.127 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.yyimst smithi174 running (87s) 82s ago 16m 10.7M - 19.0.0-2231-g2edfee68 237e43c7a65f fc1962f9351e 2024-03-21T10:21:30.127 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi153.tluxni smithi153 *:8443,9283,8765 running (13m) 103s ago 25m 509M - 19.0.0-2231-g2edfee68 237e43c7a65f 7ffee0e55382 2024-03-21T10:21:30.127 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi174.qpcwat smithi174 *:8443,9283,8765 running (11m) 82s ago 22m 452M - 19.0.0-2231-g2edfee68 237e43c7a65f 748968f216ab 2024-03-21T10:21:30.127 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi153 smithi153 running (8m) 103s ago 26m 55.0M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f dd8038303dc3 2024-03-21T10:21:30.127 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi174 smithi174 running (7m) 82s ago 22m 44.2M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f 29d78a788f2c 2024-03-21T10:21:30.127 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi153 smithi153 *:9100 running (11m) 103s ago 24m 7889k - 1.5.0 0da6a335fe13 bb835edc0e0b 2024-03-21T10:21:30.127 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi174 smithi174 *:9100 running (11m) 82s ago 22m 8102k - 1.5.0 0da6a335fe13 7c4ecad790fa 2024-03-21T10:21:30.127 INFO:teuthology.orchestra.run.smithi153.stdout:osd.0 smithi153 running (5m) 103s ago 21m 116M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 507900fd4791 2024-03-21T10:21:30.127 INFO:teuthology.orchestra.run.smithi153.stdout:osd.1 smithi153 running (5m) 103s ago 20m 89.2M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 0d646452e9c1 2024-03-21T10:21:30.127 INFO:teuthology.orchestra.run.smithi153.stdout:osd.2 smithi153 running (4m) 103s ago 20m 87.4M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 1326f2afdfcd 2024-03-21T10:21:30.128 INFO:teuthology.orchestra.run.smithi153.stdout:osd.3 smithi174 running (4m) 82s ago 19m 83.6M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 70aafac71ad5 2024-03-21T10:21:30.128 INFO:teuthology.orchestra.run.smithi153.stdout:osd.4 smithi174 running (3m) 82s ago 19m 92.2M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 0d746ce6e79d 2024-03-21T10:21:30.128 INFO:teuthology.orchestra.run.smithi153.stdout:osd.5 smithi174 running (2m) 82s ago 18m 86.2M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 53e5f35b8cac 2024-03-21T10:21:30.128 INFO:teuthology.orchestra.run.smithi153.stdout:prometheus.smithi153 smithi153 *:9095 running (10m) 103s ago 23m 73.5M - 2.43.0 a07b618ecd1d 7d95561e50c2 2024-03-21T10:21:30.128 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:30.126+0000 7f1722000700 1 -- 172.21.15.153:0/2685133639 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f170006e460 msgr2=0x7f1700070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:30.129 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:30.126+0000 7f1722000700 1 --2- 172.21.15.153:0/2685133639 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f170006e460 0x7f1700070920 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f170c000c00 tx=0x7f170c005fd0 comp rx=0 tx=0).stop 2024-03-21T10:21:30.129 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:30.126+0000 7f1722000700 1 -- 172.21.15.153:0/2685133639 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f171c10d280 msgr2=0x7f171c1a3010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:30.129 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:30.126+0000 7f1722000700 1 --2- 172.21.15.153:0/2685133639 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f171c10d280 0x7f171c1a3010 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f170800c8f0 tx=0x7f170800ccb0 comp rx=0 tx=0).stop 2024-03-21T10:21:30.129 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:30.127+0000 7f1722000700 1 -- 172.21.15.153:0/2685133639 shutdown_connections 2024-03-21T10:21:30.129 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:30.127+0000 7f1722000700 1 --2- 172.21.15.153:0/2685133639 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f171c1071d0 0x7f171c1a2ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:30.129 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:30.127+0000 7f1722000700 1 --2- 172.21.15.153:0/2685133639 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f170006e460 0x7f1700070920 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:30.129 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:30.127+0000 7f1722000700 1 --2- 172.21.15.153:0/2685133639 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f171c10d280 0x7f171c1a3010 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:30.129 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:30.127+0000 7f1722000700 1 -- 172.21.15.153:0/2685133639 >> 172.21.15.153:0/2685133639 conn(0x7f171c0769e0 msgr2=0x7f171c0782d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:21:30.129 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:30.127+0000 7f1722000700 1 -- 172.21.15.153:0/2685133639 shutdown_connections 2024-03-21T10:21:30.129 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:30.127+0000 7f1722000700 1 -- 172.21.15.153:0/2685133639 wait complete. 2024-03-21T10:21:30.283 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:30.281+0000 7f7b18a00700 1 -- 172.21.15.153:0/1806249510 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7b14105a40 msgr2=0x7f7b14105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:30.283 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:30.281+0000 7f7b18a00700 1 --2- 172.21.15.153:0/1806249510 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7b14105a40 0x7f7b14105e20 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f7b04009910 tx=0x7f7b04009c20 comp rx=0 tx=0).stop 2024-03-21T10:21:30.283 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:30.281+0000 7f7b18a00700 1 -- 172.21.15.153:0/1806249510 shutdown_connections 2024-03-21T10:21:30.283 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:30.281+0000 7f7b18a00700 1 --2- 172.21.15.153:0/1806249510 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7b140ffbd0 0x7f7b14100050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:30.283 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:30.281+0000 7f7b18a00700 1 --2- 172.21.15.153:0/1806249510 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7b14105a40 0x7f7b14105e20 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:30.283 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:30.281+0000 7f7b18a00700 1 -- 172.21.15.153:0/1806249510 >> 172.21.15.153:0/1806249510 conn(0x7f7b140755a0 msgr2=0x7f7b140759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:21:30.283 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:30.281+0000 7f7b18a00700 1 -- 172.21.15.153:0/1806249510 shutdown_connections 2024-03-21T10:21:30.283 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:30.281+0000 7f7b18a00700 1 -- 172.21.15.153:0/1806249510 wait complete. 2024-03-21T10:21:30.284 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:30.282+0000 7f7b18a00700 1 Processor -- start 2024-03-21T10:21:30.284 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:30.282+0000 7f7b18a00700 1 -- start start 2024-03-21T10:21:30.284 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:30.283+0000 7f7b18a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7b140ffbd0 0x7f7b1419c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:30.284 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:30.283+0000 7f7b18a00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7b14105a40 0x7f7b1419c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:30.284 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:30.283+0000 7f7b18a00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b1419d060 con 0x7f7b140ffbd0 2024-03-21T10:21:30.285 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:30.283+0000 7f7b18a00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b14196520 con 0x7f7b14105a40 2024-03-21T10:21:30.286 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:30.283+0000 7f7b11e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7b140ffbd0 0x7f7b1419c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:30.286 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:30.283+0000 7f7b11e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7b140ffbd0 0x7f7b1419c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:33624/0 (socket says 172.21.15.153:33624) 2024-03-21T10:21:30.286 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:30.283+0000 7f7b11e00700 1 -- 172.21.15.153:0/836866614 learned_addr learned my addr 172.21.15.153:0/836866614 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:21:30.286 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:30.283+0000 7f7b11400700 1 --2- 172.21.15.153:0/836866614 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7b14105a40 0x7f7b1419c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:30.286 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:30.283+0000 7f7b11e00700 1 -- 172.21.15.153:0/836866614 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7b14105a40 msgr2=0x7f7b1419c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:30.286 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:30.283+0000 7f7b11e00700 1 --2- 172.21.15.153:0/836866614 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7b14105a40 0x7f7b1419c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:30.286 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:30.283+0000 7f7b11e00700 1 -- 172.21.15.153:0/836866614 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b040095f0 con 0x7f7b140ffbd0 2024-03-21T10:21:30.286 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:30.284+0000 7f7b11e00700 1 --2- 172.21.15.153:0/836866614 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7b140ffbd0 0x7f7b1419c440 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f7b0400b5c0 tx=0x7f7b0400be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:21:30.286 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:30.284+0000 7f7b0aa00700 1 -- 172.21.15.153:0/836866614 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7b0401d070 con 0x7f7b140ffbd0 2024-03-21T10:21:30.287 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:30.284+0000 7f7b0aa00700 1 -- 172.21.15.153:0/836866614 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7b0400f460 con 0x7f7b140ffbd0 2024-03-21T10:21:30.287 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:30.284+0000 7f7b0aa00700 1 -- 172.21.15.153:0/836866614 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7b04017610 con 0x7f7b140ffbd0 2024-03-21T10:21:30.287 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:30.284+0000 7f7b18a00700 1 -- 172.21.15.153:0/836866614 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b141967a0 con 0x7f7b140ffbd0 2024-03-21T10:21:30.287 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:30.284+0000 7f7b18a00700 1 -- 172.21.15.153:0/836866614 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b14196c90 con 0x7f7b140ffbd0 2024-03-21T10:21:30.287 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:30.285+0000 7f7b18a00700 1 -- 172.21.15.153:0/836866614 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b14109050 con 0x7f7b140ffbd0 2024-03-21T10:21:30.288 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:30.286+0000 7f7b0aa00700 1 -- 172.21.15.153:0/836866614 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f7b0400f5d0 con 0x7f7b140ffbd0 2024-03-21T10:21:30.289 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:30.287+0000 7f7b0aa00700 1 --2- 172.21.15.153:0/836866614 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f7af806e460 0x7f7af8070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:30.289 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:30.287+0000 7f7b0aa00700 1 -- 172.21.15.153:0/836866614 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f7b0408fad0 con 0x7f7b140ffbd0 2024-03-21T10:21:30.289 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:30.288+0000 7f7b11400700 1 --2- 172.21.15.153:0/836866614 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f7af806e460 0x7f7af8070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:30.290 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:30.288+0000 7f7b11400700 1 --2- 172.21.15.153:0/836866614 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f7af806e460 0x7f7af8070920 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f7b14197d10 tx=0x7f7b00005cf0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:21:30.294 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:30.292+0000 7f7b0aa00700 1 -- 172.21.15.153:0/836866614 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7b0405c6a0 con 0x7f7b140ffbd0 2024-03-21T10:21:30.604 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:30.602+0000 7f7b18a00700 1 -- 172.21.15.153:0/836866614 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f7b14108a30 con 0x7f7b140ffbd0 2024-03-21T10:21:30.605 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:30.603+0000 7f7b0aa00700 1 -- 172.21.15.153:0/836866614 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+805 (secure 0 0 0) 0x7f7b0405c230 con 0x7f7b140ffbd0 2024-03-21T10:21:30.606 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:21:30.606 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": { 2024-03-21T10:21:30.606 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:21:30.606 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:21:30.606 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": { 2024-03-21T10:21:30.606 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:21:30.606 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:21:30.606 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": { 2024-03-21T10:21:30.606 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 6 2024-03-21T10:21:30.606 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:21:30.606 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": { 2024-03-21T10:21:30.606 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-03-21T10:21:30.607 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 3 2024-03-21T10:21:30.607 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:21:30.607 INFO:teuthology.orchestra.run.smithi153.stdout: "overall": { 2024-03-21T10:21:30.607 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-03-21T10:21:30.607 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 13 2024-03-21T10:21:30.607 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-03-21T10:21:30.607 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:21:30.608 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:30.606+0000 7f7b18a00700 1 -- 172.21.15.153:0/836866614 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f7af806e460 msgr2=0x7f7af8070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:30.608 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:30.606+0000 7f7b18a00700 1 --2- 172.21.15.153:0/836866614 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f7af806e460 0x7f7af8070920 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f7b14197d10 tx=0x7f7b00005cf0 comp rx=0 tx=0).stop 2024-03-21T10:21:30.608 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:30.606+0000 7f7b18a00700 1 -- 172.21.15.153:0/836866614 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7b140ffbd0 msgr2=0x7f7b1419c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:30.608 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:30.606+0000 7f7b18a00700 1 --2- 172.21.15.153:0/836866614 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7b140ffbd0 0x7f7b1419c440 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f7b0400b5c0 tx=0x7f7b0400be00 comp rx=0 tx=0).stop 2024-03-21T10:21:30.609 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:30.607+0000 7f7b18a00700 1 -- 172.21.15.153:0/836866614 shutdown_connections 2024-03-21T10:21:30.609 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:30.607+0000 7f7b18a00700 1 --2- 172.21.15.153:0/836866614 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f7af806e460 0x7f7af8070920 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:30.609 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:30.607+0000 7f7b18a00700 1 --2- 172.21.15.153:0/836866614 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7b140ffbd0 0x7f7b1419c440 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:30.609 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:30.607+0000 7f7b18a00700 1 --2- 172.21.15.153:0/836866614 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7b14105a40 0x7f7b1419c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:30.609 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:30.607+0000 7f7b18a00700 1 -- 172.21.15.153:0/836866614 >> 172.21.15.153:0/836866614 conn(0x7f7b140755a0 msgr2=0x7f7b140685f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:21:30.609 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:30.607+0000 7f7b18a00700 1 -- 172.21.15.153:0/836866614 shutdown_connections 2024-03-21T10:21:30.609 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:30.607+0000 7f7b18a00700 1 -- 172.21.15.153:0/836866614 wait complete. 2024-03-21T10:21:30.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:21:30 smithi174 ceph-mon[88004]: from='client.34566 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:21:30.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:21:30 smithi174 ceph-mon[88004]: from='client.34570 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:21:30.764 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:30.761+0000 7f39a8800700 1 -- 172.21.15.153:0/1873777690 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f39a0102ff0 msgr2=0x7f39a01033d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:30.764 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:30.761+0000 7f39a8800700 1 --2- 172.21.15.153:0/1873777690 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f39a0102ff0 0x7f39a01033d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f3994009910 tx=0x7f3994009c20 comp rx=0 tx=0).stop 2024-03-21T10:21:30.764 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:30.762+0000 7f39a8800700 1 -- 172.21.15.153:0/1873777690 shutdown_connections 2024-03-21T10:21:30.764 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:30.762+0000 7f39a8800700 1 --2- 172.21.15.153:0/1873777690 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f39a01039a0 0x7f39a01079f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:30.764 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:30.762+0000 7f39a8800700 1 --2- 172.21.15.153:0/1873777690 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f39a0102ff0 0x7f39a01033d0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:30.764 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:30.762+0000 7f39a8800700 1 -- 172.21.15.153:0/1873777690 >> 172.21.15.153:0/1873777690 conn(0x7f39a00fe880 msgr2=0x7f39a0100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:21:30.764 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:30.762+0000 7f39a8800700 1 -- 172.21.15.153:0/1873777690 shutdown_connections 2024-03-21T10:21:30.764 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:21:30 smithi153 ceph-mon[113733]: from='client.34566 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:21:30.765 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:21:30 smithi153 ceph-mon[113733]: from='client.34570 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:21:30.765 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:30.762+0000 7f39a8800700 1 -- 172.21.15.153:0/1873777690 wait complete. 2024-03-21T10:21:30.765 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:30.763+0000 7f39a8800700 1 Processor -- start 2024-03-21T10:21:30.765 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:30.764+0000 7f39a8800700 1 -- start start 2024-03-21T10:21:30.766 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:30.764+0000 7f39a8800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f39a0102ff0 0x7f39a019e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:30.766 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:30.764+0000 7f39a8800700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f39a01039a0 0x7f39a019ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:30.766 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:30.764+0000 7f39a8800700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39a019f3f0 con 0x7f39a0102ff0 2024-03-21T10:21:30.766 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:30.764+0000 7f39a8800700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39a01988a0 con 0x7f39a01039a0 2024-03-21T10:21:30.767 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:30.765+0000 7f39a5e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f39a0102ff0 0x7f39a019e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:30.767 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:30.765+0000 7f39a5e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f39a0102ff0 0x7f39a019e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:33640/0 (socket says 172.21.15.153:33640) 2024-03-21T10:21:30.767 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:30.765+0000 7f39a5e00700 1 -- 172.21.15.153:0/3523340264 learned_addr learned my addr 172.21.15.153:0/3523340264 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:21:30.768 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:30.765+0000 7f39a5400700 1 --2- 172.21.15.153:0/3523340264 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f39a01039a0 0x7f39a019ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:30.768 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:30.765+0000 7f39a5e00700 1 -- 172.21.15.153:0/3523340264 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f39a01039a0 msgr2=0x7f39a019ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:30.768 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:30.765+0000 7f39a5e00700 1 --2- 172.21.15.153:0/3523340264 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f39a01039a0 0x7f39a019ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:30.768 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:30.765+0000 7f39a5e00700 1 -- 172.21.15.153:0/3523340264 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39940095f0 con 0x7f39a0102ff0 2024-03-21T10:21:30.768 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:30.765+0000 7f39a5400700 1 --2- 172.21.15.153:0/3523340264 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f39a01039a0 0x7f39a019ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:21:30.768 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:30.766+0000 7f39a5e00700 1 --2- 172.21.15.153:0/3523340264 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f39a0102ff0 0x7f39a019e820 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f399400b5c0 tx=0x7f399400be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:21:30.768 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:30.766+0000 7f399aa00700 1 -- 172.21.15.153:0/3523340264 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f399401d070 con 0x7f39a0102ff0 2024-03-21T10:21:30.768 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:30.766+0000 7f399aa00700 1 -- 172.21.15.153:0/3523340264 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f399400f460 con 0x7f39a0102ff0 2024-03-21T10:21:30.768 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:30.766+0000 7f39a8800700 1 -- 172.21.15.153:0/3523340264 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39a0198b20 con 0x7f39a0102ff0 2024-03-21T10:21:30.768 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:30.766+0000 7f399aa00700 1 -- 172.21.15.153:0/3523340264 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3994017610 con 0x7f39a0102ff0 2024-03-21T10:21:30.769 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:30.766+0000 7f39a8800700 1 -- 172.21.15.153:0/3523340264 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39a0199010 con 0x7f39a0102ff0 2024-03-21T10:21:30.769 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:30.767+0000 7f39a8800700 1 -- 172.21.15.153:0/3523340264 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39a010b390 con 0x7f39a0102ff0 2024-03-21T10:21:30.770 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:30.769+0000 7f399aa00700 1 -- 172.21.15.153:0/3523340264 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f399400f5d0 con 0x7f39a0102ff0 2024-03-21T10:21:30.771 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:30.770+0000 7f399aa00700 1 --2- 172.21.15.153:0/3523340264 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f398806e460 0x7f3988070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:30.771 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:30.770+0000 7f399aa00700 1 -- 172.21.15.153:0/3523340264 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f399408faf0 con 0x7f39a0102ff0 2024-03-21T10:21:30.771 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:30.770+0000 7f39a5400700 1 --2- 172.21.15.153:0/3523340264 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f398806e460 0x7f3988070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:30.772 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:30.771+0000 7f39a5400700 1 --2- 172.21.15.153:0/3523340264 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f398806e460 0x7f3988070920 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f39a00ff0d0 tx=0x7f3990008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:21:30.776 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:30.774+0000 7f399aa00700 1 -- 172.21.15.153:0/3523340264 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f399405c6e0 con 0x7f39a0102ff0 2024-03-21T10:21:31.041 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:31.039+0000 7f39a8800700 1 -- 172.21.15.153:0/3523340264 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f39a01992f0 con 0x7f39a0102ff0 2024-03-21T10:21:31.042 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:31.040+0000 7f399aa00700 1 -- 172.21.15.153:0/3523340264 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1876 (secure 0 0 0) 0x7f399405c270 con 0x7f39a0102ff0 2024-03-21T10:21:31.042 INFO:teuthology.orchestra.run.smithi153.stdout:e30 2024-03-21T10:21:31.042 INFO:teuthology.orchestra.run.smithi153.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-21T10:21:31.042 INFO:teuthology.orchestra.run.smithi153.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:21:31.042 INFO:teuthology.orchestra.run.smithi153.stdout:legacy client fscid: 1 2024-03-21T10:21:31.043 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:21:31.043 INFO:teuthology.orchestra.run.smithi153.stdout:Filesystem 'cephfs' (1) 2024-03-21T10:21:31.043 INFO:teuthology.orchestra.run.smithi153.stdout:fs_name cephfs 2024-03-21T10:21:31.043 INFO:teuthology.orchestra.run.smithi153.stdout:epoch 23 2024-03-21T10:21:31.043 INFO:teuthology.orchestra.run.smithi153.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-21T10:21:31.043 INFO:teuthology.orchestra.run.smithi153.stdout:created 2024-03-21T10:04:27.086576+0000 2024-03-21T10:21:31.043 INFO:teuthology.orchestra.run.smithi153.stdout:modified 2024-03-21T10:19:05.022092+0000 2024-03-21T10:21:31.043 INFO:teuthology.orchestra.run.smithi153.stdout:tableserver 0 2024-03-21T10:21:31.043 INFO:teuthology.orchestra.run.smithi153.stdout:root 0 2024-03-21T10:21:31.043 INFO:teuthology.orchestra.run.smithi153.stdout:session_timeout 60 2024-03-21T10:21:31.043 INFO:teuthology.orchestra.run.smithi153.stdout:session_autoclose 300 2024-03-21T10:21:31.043 INFO:teuthology.orchestra.run.smithi153.stdout:max_file_size 1099511627776 2024-03-21T10:21:31.043 INFO:teuthology.orchestra.run.smithi153.stdout:max_xattr_size 65536 2024-03-21T10:21:31.043 INFO:teuthology.orchestra.run.smithi153.stdout:required_client_features {} 2024-03-21T10:21:31.044 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure 0 2024-03-21T10:21:31.044 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure_osd_epoch 94 2024-03-21T10:21:31.044 INFO:teuthology.orchestra.run.smithi153.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:21:31.044 INFO:teuthology.orchestra.run.smithi153.stdout:max_mds 2 2024-03-21T10:21:31.044 INFO:teuthology.orchestra.run.smithi153.stdout:in 0,1 2024-03-21T10:21:31.044 INFO:teuthology.orchestra.run.smithi153.stdout:up {} 2024-03-21T10:21:31.044 INFO:teuthology.orchestra.run.smithi153.stdout:failed 0,1 2024-03-21T10:21:31.044 INFO:teuthology.orchestra.run.smithi153.stdout:damaged 2024-03-21T10:21:31.044 INFO:teuthology.orchestra.run.smithi153.stdout:stopped 2024-03-21T10:21:31.044 INFO:teuthology.orchestra.run.smithi153.stdout:data_pools [3] 2024-03-21T10:21:31.044 INFO:teuthology.orchestra.run.smithi153.stdout:metadata_pool 2 2024-03-21T10:21:31.044 INFO:teuthology.orchestra.run.smithi153.stdout:inline_data enabled 2024-03-21T10:21:31.044 INFO:teuthology.orchestra.run.smithi153.stdout:balancer 2024-03-21T10:21:31.044 INFO:teuthology.orchestra.run.smithi153.stdout:bal_rank_mask -1 2024-03-21T10:21:31.045 INFO:teuthology.orchestra.run.smithi153.stdout:standby_count_wanted 1 2024-03-21T10:21:31.045 INFO:teuthology.orchestra.run.smithi153.stdout:qdb_cluster leader: 0 members: 2024-03-21T10:21:31.045 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:21:31.045 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:21:31.045 INFO:teuthology.orchestra.run.smithi153.stdout:Standby daemons: 2024-03-21T10:21:31.045 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:21:31.045 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.zipckr{-1:34442} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6826/1103846295,v1:172.21.15.153:6827/1103846295] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:21:31.045 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.iuyluj{-1:34474} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6828/1738630832,v1:172.21.15.153:6829/1738630832] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:21:31.045 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.nhozag{-1:44269} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.174:6826/3967223227,v1:172.21.15.174:6827/3967223227] compat {c=[1],r=[1],i=[7ff]}] 2024-03-21T10:21:31.045 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.yyimst{-1:44313} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.174:6824/4127786797,v1:172.21.15.174:6825/4127786797] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:21:31.046 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:31.043+0000 7f39a8800700 1 -- 172.21.15.153:0/3523340264 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f398806e460 msgr2=0x7f3988070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:31.046 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:31.043+0000 7f39a8800700 1 --2- 172.21.15.153:0/3523340264 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f398806e460 0x7f3988070920 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f39a00ff0d0 tx=0x7f3990008040 comp rx=0 tx=0).stop 2024-03-21T10:21:31.046 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:31.043+0000 7f39a8800700 1 -- 172.21.15.153:0/3523340264 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f39a0102ff0 msgr2=0x7f39a019e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:31.046 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:31.043+0000 7f39a8800700 1 --2- 172.21.15.153:0/3523340264 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f39a0102ff0 0x7f39a019e820 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f399400b5c0 tx=0x7f399400be00 comp rx=0 tx=0).stop 2024-03-21T10:21:31.046 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:31.043+0000 7f39a8800700 1 -- 172.21.15.153:0/3523340264 shutdown_connections 2024-03-21T10:21:31.046 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:31.043+0000 7f39a8800700 1 --2- 172.21.15.153:0/3523340264 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f398806e460 0x7f3988070920 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:31.046 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:31.043+0000 7f39a8800700 1 --2- 172.21.15.153:0/3523340264 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f39a0102ff0 0x7f39a019e820 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:31.046 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:31.043+0000 7f39a8800700 1 --2- 172.21.15.153:0/3523340264 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f39a01039a0 0x7f39a019ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:31.046 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:31.043+0000 7f39a8800700 1 -- 172.21.15.153:0/3523340264 >> 172.21.15.153:0/3523340264 conn(0x7f39a00fe880 msgr2=0x7f39a0100000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:21:31.047 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:31.043+0000 7f39a8800700 1 -- 172.21.15.153:0/3523340264 shutdown_connections 2024-03-21T10:21:31.047 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:31.043+0000 7f39a8800700 1 -- 172.21.15.153:0/3523340264 wait complete. 2024-03-21T10:21:31.047 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 30 2024-03-21T10:21:31.191 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:31.189+0000 7ff758c00700 1 -- 172.21.15.153:0/3392216964 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff754102fa0 msgr2=0x7ff754103380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:31.191 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:31.189+0000 7ff758c00700 1 --2- 172.21.15.153:0/3392216964 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff754102fa0 0x7ff754103380 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff744009910 tx=0x7ff744009c20 comp rx=0 tx=0).stop 2024-03-21T10:21:31.191 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:31.189+0000 7ff758c00700 1 -- 172.21.15.153:0/3392216964 shutdown_connections 2024-03-21T10:21:31.191 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:31.189+0000 7ff758c00700 1 --2- 172.21.15.153:0/3392216964 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff754103950 0x7ff7541079a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:31.191 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:31.189+0000 7ff758c00700 1 --2- 172.21.15.153:0/3392216964 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff754102fa0 0x7ff754103380 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:31.192 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:31.189+0000 7ff758c00700 1 -- 172.21.15.153:0/3392216964 >> 172.21.15.153:0/3392216964 conn(0x7ff7540fe810 msgr2=0x7ff754100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:21:31.192 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:31.189+0000 7ff758c00700 1 -- 172.21.15.153:0/3392216964 shutdown_connections 2024-03-21T10:21:31.192 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:31.189+0000 7ff758c00700 1 -- 172.21.15.153:0/3392216964 wait complete. 2024-03-21T10:21:31.193 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:31.190+0000 7ff758c00700 1 Processor -- start 2024-03-21T10:21:31.193 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:31.190+0000 7ff758c00700 1 -- start start 2024-03-21T10:21:31.193 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:31.191+0000 7ff758c00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff754102fa0 0x7ff75419e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:31.193 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:31.191+0000 7ff758c00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff754103950 0x7ff75419ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:31.193 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:31.191+0000 7ff758c00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff75419f2e0 con 0x7ff754102fa0 2024-03-21T10:21:31.193 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:31.191+0000 7ff758c00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff754198790 con 0x7ff754103950 2024-03-21T10:21:31.193 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:31.191+0000 7ff751e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff754102fa0 0x7ff75419e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:31.193 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:31.191+0000 7ff751e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff754102fa0 0x7ff75419e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:33656/0 (socket says 172.21.15.153:33656) 2024-03-21T10:21:31.193 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:31.191+0000 7ff751e00700 1 -- 172.21.15.153:0/2178996219 learned_addr learned my addr 172.21.15.153:0/2178996219 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:21:31.194 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:31.191+0000 7ff74be00700 1 --2- 172.21.15.153:0/2178996219 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff754103950 0x7ff75419ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:31.195 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:31.192+0000 7ff751e00700 1 -- 172.21.15.153:0/2178996219 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff754103950 msgr2=0x7ff75419ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:31.195 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:31.192+0000 7ff751e00700 1 --2- 172.21.15.153:0/2178996219 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff754103950 0x7ff75419ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:31.195 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:31.192+0000 7ff751e00700 1 -- 172.21.15.153:0/2178996219 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7440095f0 con 0x7ff754102fa0 2024-03-21T10:21:31.195 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:31.192+0000 7ff751e00700 1 --2- 172.21.15.153:0/2178996219 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff754102fa0 0x7ff75419e710 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ff744006010 tx=0x7ff74400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:21:31.195 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:31.192+0000 7ff74aa00700 1 -- 172.21.15.153:0/2178996219 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff74401d070 con 0x7ff754102fa0 2024-03-21T10:21:31.195 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:31.192+0000 7ff74aa00700 1 -- 172.21.15.153:0/2178996219 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff744004fb0 con 0x7ff754102fa0 2024-03-21T10:21:31.195 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:31.192+0000 7ff74aa00700 1 -- 172.21.15.153:0/2178996219 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff74400f690 con 0x7ff754102fa0 2024-03-21T10:21:31.195 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:31.192+0000 7ff758c00700 1 -- 172.21.15.153:0/2178996219 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff754198a10 con 0x7ff754102fa0 2024-03-21T10:21:31.195 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:31.192+0000 7ff758c00700 1 -- 172.21.15.153:0/2178996219 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff754198f00 con 0x7ff754102fa0 2024-03-21T10:21:31.195 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:31.193+0000 7ff758c00700 1 -- 172.21.15.153:0/2178996219 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff75410b340 con 0x7ff754102fa0 2024-03-21T10:21:31.196 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:31.194+0000 7ff74aa00700 1 -- 172.21.15.153:0/2178996219 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7ff7440228a0 con 0x7ff754102fa0 2024-03-21T10:21:31.197 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:31.195+0000 7ff74aa00700 1 --2- 172.21.15.153:0/2178996219 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff73806e460 0x7ff738070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:31.197 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:31.195+0000 7ff74aa00700 1 -- 172.21.15.153:0/2178996219 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7ff74408fd30 con 0x7ff754102fa0 2024-03-21T10:21:31.197 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:31.195+0000 7ff74be00700 1 --2- 172.21.15.153:0/2178996219 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff73806e460 0x7ff738070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:31.198 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:31.196+0000 7ff74be00700 1 --2- 172.21.15.153:0/2178996219 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff73806e460 0x7ff738070920 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff754199c00 tx=0x7ff73c005cf0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:21:31.202 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:31.200+0000 7ff74aa00700 1 -- 172.21.15.153:0/2178996219 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff74405c900 con 0x7ff754102fa0 2024-03-21T10:21:31.446 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:31.444+0000 7ff758c00700 1 -- 172.21.15.153:0/2178996219 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff754199170 con 0x7ff73806e460 2024-03-21T10:21:31.455 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:31.453+0000 7ff74aa00700 1 -- 172.21.15.153:0/2178996219 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7ff754199170 con 0x7ff73806e460 2024-03-21T10:21:31.455 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:21:31.455 INFO:teuthology.orchestra.run.smithi153.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232", 2024-03-21T10:21:31.455 INFO:teuthology.orchestra.run.smithi153.stdout: "in_progress": true, 2024-03-21T10:21:31.455 INFO:teuthology.orchestra.run.smithi153.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-21T10:21:31.455 INFO:teuthology.orchestra.run.smithi153.stdout: "services_complete": [ 2024-03-21T10:21:31.456 INFO:teuthology.orchestra.run.smithi153.stdout: "crash", 2024-03-21T10:21:31.456 INFO:teuthology.orchestra.run.smithi153.stdout: "mon", 2024-03-21T10:21:31.456 INFO:teuthology.orchestra.run.smithi153.stdout: "osd", 2024-03-21T10:21:31.456 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr" 2024-03-21T10:21:31.456 INFO:teuthology.orchestra.run.smithi153.stdout: ], 2024-03-21T10:21:31.456 INFO:teuthology.orchestra.run.smithi153.stdout: "progress": "15/23 daemons upgraded", 2024-03-21T10:21:31.456 INFO:teuthology.orchestra.run.smithi153.stdout: "message": "Currently upgrading mds daemons", 2024-03-21T10:21:31.456 INFO:teuthology.orchestra.run.smithi153.stdout: "is_paused": false 2024-03-21T10:21:31.456 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:21:31.457 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:31.456+0000 7ff758c00700 1 -- 172.21.15.153:0/2178996219 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff73806e460 msgr2=0x7ff738070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:31.458 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:31.456+0000 7ff758c00700 1 --2- 172.21.15.153:0/2178996219 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff73806e460 0x7ff738070920 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff754199c00 tx=0x7ff73c005cf0 comp rx=0 tx=0).stop 2024-03-21T10:21:31.458 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:31.456+0000 7ff758c00700 1 -- 172.21.15.153:0/2178996219 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff754102fa0 msgr2=0x7ff75419e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:31.458 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:31.456+0000 7ff758c00700 1 --2- 172.21.15.153:0/2178996219 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff754102fa0 0x7ff75419e710 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ff744006010 tx=0x7ff74400be20 comp rx=0 tx=0).stop 2024-03-21T10:21:31.458 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:31.456+0000 7ff758c00700 1 -- 172.21.15.153:0/2178996219 shutdown_connections 2024-03-21T10:21:31.458 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:31.456+0000 7ff758c00700 1 --2- 172.21.15.153:0/2178996219 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff73806e460 0x7ff738070920 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:31.458 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:31.456+0000 7ff758c00700 1 --2- 172.21.15.153:0/2178996219 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff754102fa0 0x7ff75419e710 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:31.458 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:31.456+0000 7ff758c00700 1 --2- 172.21.15.153:0/2178996219 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff754103950 0x7ff75419ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:31.458 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:31.456+0000 7ff758c00700 1 -- 172.21.15.153:0/2178996219 >> 172.21.15.153:0/2178996219 conn(0x7ff7540fe810 msgr2=0x7ff7540fff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:21:31.459 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:31.456+0000 7ff758c00700 1 -- 172.21.15.153:0/2178996219 shutdown_connections 2024-03-21T10:21:31.459 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:31.456+0000 7ff758c00700 1 -- 172.21.15.153:0/2178996219 wait complete. 2024-03-21T10:21:31.605 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:31.602+0000 7f17e1800700 1 -- 172.21.15.153:0/1942328010 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f17dc068670 msgr2=0x7f17dc068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:31.605 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:31.602+0000 7f17e1800700 1 --2- 172.21.15.153:0/1942328010 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f17dc068670 0x7f17dc068a50 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f17cc009910 tx=0x7f17cc009c20 comp rx=0 tx=0).stop 2024-03-21T10:21:31.605 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:31.603+0000 7f17e1800700 1 -- 172.21.15.153:0/1942328010 shutdown_connections 2024-03-21T10:21:31.605 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:31.603+0000 7f17e1800700 1 --2- 172.21.15.153:0/1942328010 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f17dc069020 0x7f17dc105790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:31.605 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:31.603+0000 7f17e1800700 1 --2- 172.21.15.153:0/1942328010 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f17dc068670 0x7f17dc068a50 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:31.605 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:31.603+0000 7f17e1800700 1 -- 172.21.15.153:0/1942328010 >> 172.21.15.153:0/1942328010 conn(0x7f17dc0755a0 msgr2=0x7f17dc0759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:21:31.605 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:31.603+0000 7f17e1800700 1 -- 172.21.15.153:0/1942328010 shutdown_connections 2024-03-21T10:21:31.605 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:31.603+0000 7f17e1800700 1 -- 172.21.15.153:0/1942328010 wait complete. 2024-03-21T10:21:31.606 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:31.604+0000 7f17e1800700 1 Processor -- start 2024-03-21T10:21:31.606 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:31.604+0000 7f17e1800700 1 -- start start 2024-03-21T10:21:31.607 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:31.604+0000 7f17e1800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f17dc068670 0x7f17dc0ffca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:31.607 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:31.604+0000 7f17e1800700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f17dc069020 0x7f17dc1001e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:31.607 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:31.604+0000 7f17e1800700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17dc100720 con 0x7f17dc068670 2024-03-21T10:21:31.607 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:31.604+0000 7f17e1800700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17dc100860 con 0x7f17dc069020 2024-03-21T10:21:31.607 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:31.605+0000 7f17daa00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f17dc068670 0x7f17dc0ffca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:31.607 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:31.605+0000 7f17daa00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f17dc068670 0x7f17dc0ffca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:33670/0 (socket says 172.21.15.153:33670) 2024-03-21T10:21:31.607 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:31.605+0000 7f17daa00700 1 -- 172.21.15.153:0/629807737 learned_addr learned my addr 172.21.15.153:0/629807737 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:21:31.607 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:31.605+0000 7f17da000700 1 --2- 172.21.15.153:0/629807737 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f17dc069020 0x7f17dc1001e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:31.607 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:31.605+0000 7f17daa00700 1 -- 172.21.15.153:0/629807737 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f17dc069020 msgr2=0x7f17dc1001e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:31.607 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:31.605+0000 7f17daa00700 1 --2- 172.21.15.153:0/629807737 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f17dc069020 0x7f17dc1001e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:31.607 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:31.605+0000 7f17daa00700 1 -- 172.21.15.153:0/629807737 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17cc0095f0 con 0x7f17dc068670 2024-03-21T10:21:31.608 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:31.606+0000 7f17daa00700 1 --2- 172.21.15.153:0/629807737 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f17dc068670 0x7f17dc0ffca0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f17cc006010 tx=0x7f17cc00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:21:31.608 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:31.606+0000 7f17d3400700 1 -- 172.21.15.153:0/629807737 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f17cc01d070 con 0x7f17dc068670 2024-03-21T10:21:31.608 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:31.606+0000 7f17d3400700 1 -- 172.21.15.153:0/629807737 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f17cc004fb0 con 0x7f17dc068670 2024-03-21T10:21:31.608 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:31.606+0000 7f17d3400700 1 -- 172.21.15.153:0/629807737 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f17cc022ea0 con 0x7f17dc068670 2024-03-21T10:21:31.608 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:31.606+0000 7f17e1800700 1 -- 172.21.15.153:0/629807737 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17dc100ae0 con 0x7f17dc068670 2024-03-21T10:21:31.609 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:31.606+0000 7f17e1800700 1 -- 172.21.15.153:0/629807737 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17dc1a2df0 con 0x7f17dc068670 2024-03-21T10:21:31.609 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:31.607+0000 7f17e1800700 1 -- 172.21.15.153:0/629807737 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17dc109130 con 0x7f17dc068670 2024-03-21T10:21:31.609 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:31.608+0000 7f17d3400700 1 -- 172.21.15.153:0/629807737 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f17cc022470 con 0x7f17dc068670 2024-03-21T10:21:31.611 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:31.609+0000 7f17d3400700 1 --2- 172.21.15.153:0/629807737 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f17c006e460 0x7f17c0070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:21:31.611 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:31.609+0000 7f17d3400700 1 -- 172.21.15.153:0/629807737 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f17cc08fa80 con 0x7f17dc068670 2024-03-21T10:21:31.611 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:31.610+0000 7f17da000700 1 --2- 172.21.15.153:0/629807737 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f17c006e460 0x7f17c0070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:21:31.612 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:31.610+0000 7f17da000700 1 --2- 172.21.15.153:0/629807737 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f17c006e460 0x7f17c0070920 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f17c8006fd0 tx=0x7f17c8008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:21:31.616 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:31.614+0000 7f17d3400700 1 -- 172.21.15.153:0/629807737 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f17cc058bd0 con 0x7f17dc068670 2024-03-21T10:21:31.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:21:31 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/836866614' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:21:31.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:21:31 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/3523340264' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:21:31.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:21:31 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:21:31.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:21:31 smithi174 ceph-mon[88004]: pgmap v290: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:21:31.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:21:31 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/836866614' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:21:31.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:21:31 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/3523340264' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:21:31.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:21:31 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:21:31.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:21:31 smithi153 ceph-mon[113733]: pgmap v290: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:21:31.906 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:31.904+0000 7f17e1800700 1 -- 172.21.15.153:0/629807737 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f17dc04ea90 con 0x7f17dc068670 2024-03-21T10:21:31.907 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:31.905+0000 7f17d3400700 1 -- 172.21.15.153:0/629807737 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f17cc0271c0 con 0x7f17dc068670 2024-03-21T10:21:31.907 INFO:teuthology.orchestra.run.smithi153.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-03-21T10:21:31.907 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-21T10:21:31.907 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is degraded 2024-03-21T10:21:31.907 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-21T10:21:31.907 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-21T10:21:31.907 INFO:teuthology.orchestra.run.smithi153.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-21T10:21:31.908 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-21T10:21:31.909 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:31.908+0000 7f17e1800700 1 -- 172.21.15.153:0/629807737 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f17c006e460 msgr2=0x7f17c0070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:31.910 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:31.908+0000 7f17e1800700 1 --2- 172.21.15.153:0/629807737 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f17c006e460 0x7f17c0070920 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f17c8006fd0 tx=0x7f17c8008040 comp rx=0 tx=0).stop 2024-03-21T10:21:31.910 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:31.908+0000 7f17e1800700 1 -- 172.21.15.153:0/629807737 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f17dc068670 msgr2=0x7f17dc0ffca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:21:31.910 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:31.908+0000 7f17e1800700 1 --2- 172.21.15.153:0/629807737 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f17dc068670 0x7f17dc0ffca0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f17cc006010 tx=0x7f17cc00be20 comp rx=0 tx=0).stop 2024-03-21T10:21:31.910 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:31.908+0000 7f17e1800700 1 -- 172.21.15.153:0/629807737 shutdown_connections 2024-03-21T10:21:31.910 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:31.908+0000 7f17e1800700 1 --2- 172.21.15.153:0/629807737 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f17c006e460 0x7f17c0070920 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:31.910 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:31.908+0000 7f17e1800700 1 --2- 172.21.15.153:0/629807737 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f17dc068670 0x7f17dc0ffca0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:31.910 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:31.908+0000 7f17e1800700 1 --2- 172.21.15.153:0/629807737 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f17dc069020 0x7f17dc1001e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:21:31.910 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:31.908+0000 7f17e1800700 1 -- 172.21.15.153:0/629807737 >> 172.21.15.153:0/629807737 conn(0x7f17dc0755a0 msgr2=0x7f17dc0fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:21:31.911 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:31.908+0000 7f17e1800700 1 -- 172.21.15.153:0/629807737 shutdown_connections 2024-03-21T10:21:31.911 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:21:31.908+0000 7f17e1800700 1 -- 172.21.15.153:0/629807737 wait complete. 2024-03-21T10:21:32.650 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:21:32 smithi153 ceph-mon[113733]: from='client.34582 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:21:32.650 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:21:32 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/629807737' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:21:32.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:21:32 smithi174 ceph-mon[88004]: from='client.34582 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:21:32.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:21:32 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/629807737' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:21:33.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:21:33 smithi174 ceph-mon[88004]: pgmap v291: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:21:33.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:21:33 smithi153 ceph-mon[113733]: pgmap v291: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:21:35.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:21:35 smithi174 ceph-mon[88004]: pgmap v292: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:21:35.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:21:35 smithi153 ceph-mon[113733]: pgmap v292: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:21:37.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:21:37 smithi174 ceph-mon[88004]: pgmap v293: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:21:37.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:21:37 smithi153 ceph-mon[113733]: pgmap v293: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:21:39.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:21:39 smithi153 ceph-mon[113733]: pgmap v294: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:21:39.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:21:39 smithi174 ceph-mon[88004]: pgmap v294: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:21:41.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:21:41 smithi174 ceph-mon[88004]: pgmap v295: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:21:41.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:21:41 smithi153 ceph-mon[113733]: pgmap v295: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:21:43.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:21:43 smithi174 ceph-mon[88004]: pgmap v296: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:21:43.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:21:43 smithi153 ceph-mon[113733]: pgmap v296: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:21:45.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:21:45 smithi174 ceph-mon[88004]: pgmap v297: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:21:45.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:21:45 smithi153 ceph-mon[113733]: pgmap v297: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:21:46.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:21:46 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:21:46.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:21:46 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:21:47.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:21:47 smithi174 ceph-mon[88004]: pgmap v298: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:21:47.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:21:47 smithi153 ceph-mon[113733]: pgmap v298: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:21:49.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:21:49 smithi174 ceph-mon[88004]: pgmap v299: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:21:49.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:21:49 smithi153 ceph-mon[113733]: pgmap v299: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:21:51.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:21:51 smithi174 ceph-mon[88004]: pgmap v300: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:21:51.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:21:51 smithi153 ceph-mon[113733]: pgmap v300: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:21:53.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:21:53 smithi174 ceph-mon[88004]: pgmap v301: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:21:53.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:21:53 smithi153 ceph-mon[113733]: pgmap v301: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:21:55.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:21:55 smithi174 ceph-mon[88004]: pgmap v302: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:21:55.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:21:55 smithi153 ceph-mon[113733]: pgmap v302: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:21:57.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:21:57 smithi174 ceph-mon[88004]: pgmap v303: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:21:57.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:21:57 smithi153 ceph-mon[113733]: pgmap v303: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:21:59.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:21:59 smithi174 ceph-mon[88004]: pgmap v304: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:21:59.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:21:59 smithi153 ceph-mon[113733]: pgmap v304: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:22:01.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:22:01 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:22:01.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:22:01 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:22:02.088 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.086+0000 7fd38ca00700 1 -- 172.21.15.153:0/1902201215 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd388101af0 msgr2=0x7fd388101ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:02.088 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.086+0000 7fd38ca00700 1 --2- 172.21.15.153:0/1902201215 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd388101af0 0x7fd388101ed0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fd378009910 tx=0x7fd378009c20 comp rx=0 tx=0).stop 2024-03-21T10:22:02.089 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.086+0000 7fd38ca00700 1 -- 172.21.15.153:0/1902201215 shutdown_connections 2024-03-21T10:22:02.089 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.086+0000 7fd38ca00700 1 --2- 172.21.15.153:0/1902201215 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd388102410 0x7fd38810a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:02.089 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.086+0000 7fd38ca00700 1 --2- 172.21.15.153:0/1902201215 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd388101af0 0x7fd388101ed0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:02.089 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.086+0000 7fd38ca00700 1 -- 172.21.15.153:0/1902201215 >> 172.21.15.153:0/1902201215 conn(0x7fd3880fb360 msgr2=0x7fd3880fd780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:02.089 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.086+0000 7fd38ca00700 1 -- 172.21.15.153:0/1902201215 shutdown_connections 2024-03-21T10:22:02.089 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.086+0000 7fd38ca00700 1 -- 172.21.15.153:0/1902201215 wait complete. 2024-03-21T10:22:02.090 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.087+0000 7fd38ca00700 1 Processor -- start 2024-03-21T10:22:02.090 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.088+0000 7fd38ca00700 1 -- start start 2024-03-21T10:22:02.090 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.088+0000 7fd38ca00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd388101af0 0x7fd3880ff750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:02.090 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.088+0000 7fd38ca00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd388102410 0x7fd3880ffc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:02.090 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.088+0000 7fd38ca00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3881001d0 con 0x7fd388101af0 2024-03-21T10:22:02.090 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.088+0000 7fd38ca00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd388100310 con 0x7fd388102410 2024-03-21T10:22:02.090 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.088+0000 7fd386800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd388101af0 0x7fd3880ff750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:02.090 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.088+0000 7fd386800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd388101af0 0x7fd3880ff750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:33718/0 (socket says 172.21.15.153:33718) 2024-03-21T10:22:02.090 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.088+0000 7fd386800700 1 -- 172.21.15.153:0/2873527868 learned_addr learned my addr 172.21.15.153:0/2873527868 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:22:02.091 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.089+0000 7fd37de00700 1 --2- 172.21.15.153:0/2873527868 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd388102410 0x7fd3880ffc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:02.091 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.089+0000 7fd386800700 1 -- 172.21.15.153:0/2873527868 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd388102410 msgr2=0x7fd3880ffc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:02.092 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.089+0000 7fd386800700 1 --2- 172.21.15.153:0/2873527868 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd388102410 0x7fd3880ffc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:02.092 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.089+0000 7fd386800700 1 -- 172.21.15.153:0/2873527868 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3780095f0 con 0x7fd388101af0 2024-03-21T10:22:02.092 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.089+0000 7fd386800700 1 --2- 172.21.15.153:0/2873527868 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd388101af0 0x7fd3880ff750 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fd378006010 tx=0x7fd37800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:02.092 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.089+0000 7fd37fe00700 1 -- 172.21.15.153:0/2873527868 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd37801d070 con 0x7fd388101af0 2024-03-21T10:22:02.092 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.090+0000 7fd37fe00700 1 -- 172.21.15.153:0/2873527868 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd378004fb0 con 0x7fd388101af0 2024-03-21T10:22:02.092 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.090+0000 7fd38ca00700 1 -- 172.21.15.153:0/2873527868 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd388100590 con 0x7fd388101af0 2024-03-21T10:22:02.092 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.090+0000 7fd37fe00700 1 -- 172.21.15.153:0/2873527868 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd378022ea0 con 0x7fd388101af0 2024-03-21T10:22:02.092 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.090+0000 7fd38ca00700 1 -- 172.21.15.153:0/2873527868 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3881a2b40 con 0x7fd388101af0 2024-03-21T10:22:02.092 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.091+0000 7fd38ca00700 1 -- 172.21.15.153:0/2873527868 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd388108000 con 0x7fd388101af0 2024-03-21T10:22:02.093 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.092+0000 7fd37fe00700 1 -- 172.21.15.153:0/2873527868 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fd378022470 con 0x7fd388101af0 2024-03-21T10:22:02.094 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.093+0000 7fd37fe00700 1 --2- 172.21.15.153:0/2873527868 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd36c06e460 0x7fd36c070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:02.095 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.093+0000 7fd37fe00700 1 -- 172.21.15.153:0/2873527868 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fd37808fa60 con 0x7fd388101af0 2024-03-21T10:22:02.095 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.093+0000 7fd37de00700 1 --2- 172.21.15.153:0/2873527868 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd36c06e460 0x7fd36c070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:02.095 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.094+0000 7fd37de00700 1 --2- 172.21.15.153:0/2873527868 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd36c06e460 0x7fd36c070920 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd370006fd0 tx=0x7fd370008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:02.100 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.098+0000 7fd37fe00700 1 -- 172.21.15.153:0/2873527868 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd378058bd0 con 0x7fd388101af0 2024-03-21T10:22:02.346 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.343+0000 7fd38ca00700 1 -- 172.21.15.153:0/2873527868 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd3880611d0 con 0x7fd36c06e460 2024-03-21T10:22:02.346 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:22:02 smithi153 ceph-mon[113733]: pgmap v305: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:22:02.347 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.346+0000 7fd37fe00700 1 -- 172.21.15.153:0/2873527868 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fd3880611d0 con 0x7fd36c06e460 2024-03-21T10:22:02.354 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.348+0000 7fd38ca00700 1 -- 172.21.15.153:0/2873527868 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd36c06e460 msgr2=0x7fd36c070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:02.354 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.348+0000 7fd38ca00700 1 --2- 172.21.15.153:0/2873527868 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd36c06e460 0x7fd36c070920 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd370006fd0 tx=0x7fd370008040 comp rx=0 tx=0).stop 2024-03-21T10:22:02.354 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.348+0000 7fd38ca00700 1 -- 172.21.15.153:0/2873527868 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd388101af0 msgr2=0x7fd3880ff750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:02.354 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.348+0000 7fd38ca00700 1 --2- 172.21.15.153:0/2873527868 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd388101af0 0x7fd3880ff750 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fd378006010 tx=0x7fd37800be20 comp rx=0 tx=0).stop 2024-03-21T10:22:02.354 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.349+0000 7fd38ca00700 1 -- 172.21.15.153:0/2873527868 shutdown_connections 2024-03-21T10:22:02.355 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.349+0000 7fd38ca00700 1 --2- 172.21.15.153:0/2873527868 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd36c06e460 0x7fd36c070920 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:02.355 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.349+0000 7fd38ca00700 1 --2- 172.21.15.153:0/2873527868 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd388101af0 0x7fd3880ff750 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:02.355 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.349+0000 7fd38ca00700 1 --2- 172.21.15.153:0/2873527868 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd388102410 0x7fd3880ffc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:02.355 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.349+0000 7fd38ca00700 1 -- 172.21.15.153:0/2873527868 >> 172.21.15.153:0/2873527868 conn(0x7fd3880fb360 msgr2=0x7fd388105140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:02.355 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.349+0000 7fd38ca00700 1 -- 172.21.15.153:0/2873527868 shutdown_connections 2024-03-21T10:22:02.355 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.349+0000 7fd38ca00700 1 -- 172.21.15.153:0/2873527868 wait complete. 2024-03-21T10:22:02.367 INFO:teuthology.orchestra.run.smithi153.stdout:true 2024-03-21T10:22:02.510 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.507+0000 7f533d800700 1 -- 172.21.15.153:0/3954028876 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5338102fe0 msgr2=0x7f53381033c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:02.510 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.507+0000 7f533d800700 1 --2- 172.21.15.153:0/3954028876 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5338102fe0 0x7f53381033c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5328009910 tx=0x7f5328009c20 comp rx=0 tx=0).stop 2024-03-21T10:22:02.510 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.508+0000 7f533d800700 1 -- 172.21.15.153:0/3954028876 shutdown_connections 2024-03-21T10:22:02.510 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.508+0000 7f533d800700 1 --2- 172.21.15.153:0/3954028876 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5338103990 0x7f53381079e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:02.510 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.508+0000 7f533d800700 1 --2- 172.21.15.153:0/3954028876 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5338102fe0 0x7f53381033c0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:02.510 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.508+0000 7f533d800700 1 -- 172.21.15.153:0/3954028876 >> 172.21.15.153:0/3954028876 conn(0x7f53380fe870 msgr2=0x7f5338100c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:02.510 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.508+0000 7f533d800700 1 -- 172.21.15.153:0/3954028876 shutdown_connections 2024-03-21T10:22:02.510 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.508+0000 7f533d800700 1 -- 172.21.15.153:0/3954028876 wait complete. 2024-03-21T10:22:02.511 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.509+0000 7f533d800700 1 Processor -- start 2024-03-21T10:22:02.511 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.509+0000 7f533d800700 1 -- start start 2024-03-21T10:22:02.511 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.509+0000 7f533d800700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5338102fe0 0x7f533819e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:02.511 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.509+0000 7f533d800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5338103990 0x7f533819ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:02.511 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.509+0000 7f533d800700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f533819f360 con 0x7f5338103990 2024-03-21T10:22:02.512 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.509+0000 7f533d800700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5338198810 con 0x7f5338102fe0 2024-03-21T10:22:02.512 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.510+0000 7f532e000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5338103990 0x7f533819ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:02.512 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.510+0000 7f532e000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5338103990 0x7f533819ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:33734/0 (socket says 172.21.15.153:33734) 2024-03-21T10:22:02.512 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.510+0000 7f532e000700 1 -- 172.21.15.153:0/1040166466 learned_addr learned my addr 172.21.15.153:0/1040166466 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:22:02.513 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.510+0000 7f5336a00700 1 --2- 172.21.15.153:0/1040166466 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5338102fe0 0x7f533819e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:02.513 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.510+0000 7f532e000700 1 -- 172.21.15.153:0/1040166466 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5338102fe0 msgr2=0x7f533819e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:02.513 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.510+0000 7f532e000700 1 --2- 172.21.15.153:0/1040166466 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5338102fe0 0x7f533819e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:02.513 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.510+0000 7f532e000700 1 -- 172.21.15.153:0/1040166466 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53280095f0 con 0x7f5338103990 2024-03-21T10:22:02.513 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.510+0000 7f5336a00700 1 --2- 172.21.15.153:0/1040166466 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5338102fe0 0x7f533819e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:22:02.513 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.511+0000 7f532e000700 1 --2- 172.21.15.153:0/1040166466 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5338103990 0x7f533819ecd0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f532000c8f0 tx=0x7f532000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:02.513 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.511+0000 7f532fe00700 1 -- 172.21.15.153:0/1040166466 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5320007a50 con 0x7f5338103990 2024-03-21T10:22:02.513 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.511+0000 7f532fe00700 1 -- 172.21.15.153:0/1040166466 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5320007bb0 con 0x7f5338103990 2024-03-21T10:22:02.513 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.511+0000 7f533d800700 1 -- 172.21.15.153:0/1040166466 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5338198af0 con 0x7f5338103990 2024-03-21T10:22:02.513 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.511+0000 7f532fe00700 1 -- 172.21.15.153:0/1040166466 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5320018630 con 0x7f5338103990 2024-03-21T10:22:02.514 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.511+0000 7f533d800700 1 -- 172.21.15.153:0/1040166466 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5338199040 con 0x7f5338103990 2024-03-21T10:22:02.514 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.512+0000 7f533d800700 1 -- 172.21.15.153:0/1040166466 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f533810b380 con 0x7f5338103990 2024-03-21T10:22:02.515 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.513+0000 7f532fe00700 1 -- 172.21.15.153:0/1040166466 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f5320007d20 con 0x7f5338103990 2024-03-21T10:22:02.516 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.514+0000 7f532fe00700 1 --2- 172.21.15.153:0/1040166466 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f531c06e460 0x7f531c070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:02.516 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.514+0000 7f532fe00700 1 -- 172.21.15.153:0/1040166466 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f532008e550 con 0x7f5338103990 2024-03-21T10:22:02.516 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.515+0000 7f5336a00700 1 --2- 172.21.15.153:0/1040166466 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f531c06e460 0x7f531c070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:02.517 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.515+0000 7f5336a00700 1 --2- 172.21.15.153:0/1040166466 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f531c06e460 0x7f531c070920 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f532800b5c0 tx=0x7f5328005960 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:02.520 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.519+0000 7f532fe00700 1 -- 172.21.15.153:0/1040166466 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f532005b0a0 con 0x7f5338103990 2024-03-21T10:22:02.603 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:22:02 smithi174 ceph-mon[88004]: pgmap v305: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:22:02.765 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.762+0000 7f533d800700 1 -- 172.21.15.153:0/1040166466 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5338199330 con 0x7f531c06e460 2024-03-21T10:22:02.767 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.765+0000 7f532fe00700 1 -- 172.21.15.153:0/1040166466 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f5338199330 con 0x7f531c06e460 2024-03-21T10:22:02.771 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.768+0000 7f533d800700 1 -- 172.21.15.153:0/1040166466 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f531c06e460 msgr2=0x7f531c070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:02.771 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.768+0000 7f533d800700 1 --2- 172.21.15.153:0/1040166466 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f531c06e460 0x7f531c070920 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f532800b5c0 tx=0x7f5328005960 comp rx=0 tx=0).stop 2024-03-21T10:22:02.771 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.768+0000 7f533d800700 1 -- 172.21.15.153:0/1040166466 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5338103990 msgr2=0x7f533819ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:02.771 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.768+0000 7f533d800700 1 --2- 172.21.15.153:0/1040166466 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5338103990 0x7f533819ecd0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f532000c8f0 tx=0x7f532000ccb0 comp rx=0 tx=0).stop 2024-03-21T10:22:02.771 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.768+0000 7f533d800700 1 -- 172.21.15.153:0/1040166466 shutdown_connections 2024-03-21T10:22:02.771 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.768+0000 7f533d800700 1 --2- 172.21.15.153:0/1040166466 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5338102fe0 0x7f533819e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:02.771 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.768+0000 7f533d800700 1 --2- 172.21.15.153:0/1040166466 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f531c06e460 0x7f531c070920 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:02.771 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.768+0000 7f533d800700 1 --2- 172.21.15.153:0/1040166466 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5338103990 0x7f533819ecd0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:02.771 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.768+0000 7f533d800700 1 -- 172.21.15.153:0/1040166466 >> 172.21.15.153:0/1040166466 conn(0x7f53380fe870 msgr2=0x7f5338100c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:02.771 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.769+0000 7f533d800700 1 -- 172.21.15.153:0/1040166466 shutdown_connections 2024-03-21T10:22:02.771 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.769+0000 7f533d800700 1 -- 172.21.15.153:0/1040166466 wait complete. 2024-03-21T10:22:02.928 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.926+0000 7f4494400700 1 -- 172.21.15.153:0/2337072270 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f448c068670 msgr2=0x7f448c068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:02.928 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.926+0000 7f4494400700 1 --2- 172.21.15.153:0/2337072270 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f448c068670 0x7f448c068a50 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f4480009910 tx=0x7f4480009c20 comp rx=0 tx=0).stop 2024-03-21T10:22:02.928 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.926+0000 7f4494400700 1 -- 172.21.15.153:0/2337072270 shutdown_connections 2024-03-21T10:22:02.928 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.926+0000 7f4494400700 1 --2- 172.21.15.153:0/2337072270 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f448c069020 0x7f448c105790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:02.928 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.926+0000 7f4494400700 1 --2- 172.21.15.153:0/2337072270 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f448c068670 0x7f448c068a50 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:02.928 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.926+0000 7f4494400700 1 -- 172.21.15.153:0/2337072270 >> 172.21.15.153:0/2337072270 conn(0x7f448c0755a0 msgr2=0x7f448c0759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:02.928 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.926+0000 7f4494400700 1 -- 172.21.15.153:0/2337072270 shutdown_connections 2024-03-21T10:22:02.928 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.926+0000 7f4494400700 1 -- 172.21.15.153:0/2337072270 wait complete. 2024-03-21T10:22:02.929 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.927+0000 7f4494400700 1 Processor -- start 2024-03-21T10:22:02.929 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.927+0000 7f4494400700 1 -- start start 2024-03-21T10:22:02.929 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.928+0000 7f4494400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f448c068670 0x7f448c19e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:02.929 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.928+0000 7f4494400700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f448c069020 0x7f448c19edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:02.929 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.928+0000 7f4494400700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f448c19f480 con 0x7f448c068670 2024-03-21T10:22:02.930 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.928+0000 7f4494400700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f448c198930 con 0x7f448c069020 2024-03-21T10:22:02.930 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.928+0000 7f4491a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f448c068670 0x7f448c19e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:02.930 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.928+0000 7f4491a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f448c068670 0x7f448c19e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39880/0 (socket says 172.21.15.153:39880) 2024-03-21T10:22:02.931 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.928+0000 7f4491a00700 1 -- 172.21.15.153:0/3139075468 learned_addr learned my addr 172.21.15.153:0/3139075468 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:22:02.931 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.928+0000 7f4491000700 1 --2- 172.21.15.153:0/3139075468 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f448c069020 0x7f448c19edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:02.931 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.929+0000 7f4491a00700 1 -- 172.21.15.153:0/3139075468 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f448c069020 msgr2=0x7f448c19edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:02.931 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.929+0000 7f4491a00700 1 --2- 172.21.15.153:0/3139075468 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f448c069020 0x7f448c19edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:02.931 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.929+0000 7f4491a00700 1 -- 172.21.15.153:0/3139075468 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44800095f0 con 0x7f448c068670 2024-03-21T10:22:02.931 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.929+0000 7f4491a00700 1 --2- 172.21.15.153:0/3139075468 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f448c068670 0x7f448c19e8b0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f448000b5c0 tx=0x7f448000be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:02.932 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.929+0000 7f448a000700 1 -- 172.21.15.153:0/3139075468 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f448001d070 con 0x7f448c068670 2024-03-21T10:22:02.932 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.929+0000 7f448a000700 1 -- 172.21.15.153:0/3139075468 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f448000f460 con 0x7f448c068670 2024-03-21T10:22:02.932 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.929+0000 7f4494400700 1 -- 172.21.15.153:0/3139075468 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f448c198bb0 con 0x7f448c068670 2024-03-21T10:22:02.932 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.930+0000 7f4494400700 1 -- 172.21.15.153:0/3139075468 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f448c1990a0 con 0x7f448c068670 2024-03-21T10:22:02.932 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.930+0000 7f448a000700 1 -- 172.21.15.153:0/3139075468 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4480017610 con 0x7f448c068670 2024-03-21T10:22:02.932 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.931+0000 7f4494400700 1 -- 172.21.15.153:0/3139075468 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f448c04ea90 con 0x7f448c068670 2024-03-21T10:22:02.933 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.932+0000 7f448a000700 1 -- 172.21.15.153:0/3139075468 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f448000f5d0 con 0x7f448c068670 2024-03-21T10:22:02.934 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.933+0000 7f448a000700 1 --2- 172.21.15.153:0/3139075468 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f447406e460 0x7f4474070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:02.935 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.933+0000 7f448a000700 1 -- 172.21.15.153:0/3139075468 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f448008fae0 con 0x7f448c068670 2024-03-21T10:22:02.935 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.933+0000 7f4491000700 1 --2- 172.21.15.153:0/3139075468 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f447406e460 0x7f4474070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:02.935 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.934+0000 7f4491000700 1 --2- 172.21.15.153:0/3139075468 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f447406e460 0x7f4474070920 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f4478006fd0 tx=0x7f4478008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:02.940 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:02.938+0000 7f448a000700 1 -- 172.21.15.153:0/3139075468 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f448005c6d0 con 0x7f448c068670 2024-03-21T10:22:03.170 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:03.168+0000 7f4494400700 1 -- 172.21.15.153:0/3139075468 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f448c199490 con 0x7f447406e460 2024-03-21T10:22:03.189 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:03.187+0000 7f448a000700 1 -- 172.21.15.153:0/3139075468 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f448c199490 con 0x7f447406e460 2024-03-21T10:22:03.190 INFO:teuthology.orchestra.run.smithi153.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-21T10:22:03.190 INFO:teuthology.orchestra.run.smithi153.stdout:alertmanager.smithi153 smithi153 *:9093,9094 running (10m) 2m ago 24m 14.7M - 0.25.0 c8568f914cd2 9ed3f2200b5f 2024-03-21T10:22:03.190 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi153 smithi153 running (24m) 2m ago 24m 9424k - 18.2.1 5be31c24972a 5b61994396a6 2024-03-21T10:22:03.190 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi174 smithi174 running (23m) 115s ago 23m 9244k - 18.2.1 5be31c24972a 4f7a5fc3ac28 2024-03-21T10:22:03.190 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi153 smithi153 running (7m) 2m ago 24m 6656k - 19.0.0-2231-g2edfee68 237e43c7a65f 32c21104d5b0 2024-03-21T10:22:03.190 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi174 smithi174 running (6m) 115s ago 23m 6660k - 19.0.0-2231-g2edfee68 237e43c7a65f a04e48967ab6 2024-03-21T10:22:03.191 INFO:teuthology.orchestra.run.smithi153.stdout:grafana.smithi153 smithi153 *:3000 running (10m) 2m ago 24m 73.6M - 9.4.12 2bacad6d85d8 ee24026ab786 2024-03-21T10:22:03.191 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.iuyluj smithi153 running (2m) 2m ago 17m 10.9M - 19.0.0-2231-g2edfee68 237e43c7a65f 87aff8a5de70 2024-03-21T10:22:03.191 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.zipckr smithi153 running (2m) 2m ago 17m 11.8M - 19.0.0-2231-g2edfee68 237e43c7a65f ccc262bd5dd3 2024-03-21T10:22:03.191 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.nhozag smithi174 running (17m) 115s ago 17m 12.0M - 18.2.1 5be31c24972a 2fee4000308f 2024-03-21T10:22:03.191 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.yyimst smithi174 running (2m) 115s ago 17m 10.7M - 19.0.0-2231-g2edfee68 237e43c7a65f fc1962f9351e 2024-03-21T10:22:03.191 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi153.tluxni smithi153 *:8443,9283,8765 running (13m) 2m ago 26m 509M - 19.0.0-2231-g2edfee68 237e43c7a65f 7ffee0e55382 2024-03-21T10:22:03.191 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi174.qpcwat smithi174 *:8443,9283,8765 running (12m) 115s ago 23m 452M - 19.0.0-2231-g2edfee68 237e43c7a65f 748968f216ab 2024-03-21T10:22:03.191 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi153 smithi153 running (8m) 2m ago 26m 55.0M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f dd8038303dc3 2024-03-21T10:22:03.191 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi174 smithi174 running (8m) 115s ago 22m 44.2M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f 29d78a788f2c 2024-03-21T10:22:03.191 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi153 smithi153 *:9100 running (12m) 2m ago 24m 7889k - 1.5.0 0da6a335fe13 bb835edc0e0b 2024-03-21T10:22:03.191 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi174 smithi174 *:9100 running (11m) 115s ago 23m 8102k - 1.5.0 0da6a335fe13 7c4ecad790fa 2024-03-21T10:22:03.191 INFO:teuthology.orchestra.run.smithi153.stdout:osd.0 smithi153 running (6m) 2m ago 22m 116M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 507900fd4791 2024-03-21T10:22:03.191 INFO:teuthology.orchestra.run.smithi153.stdout:osd.1 smithi153 running (5m) 2m ago 21m 89.2M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 0d646452e9c1 2024-03-21T10:22:03.191 INFO:teuthology.orchestra.run.smithi153.stdout:osd.2 smithi153 running (5m) 2m ago 20m 87.4M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 1326f2afdfcd 2024-03-21T10:22:03.191 INFO:teuthology.orchestra.run.smithi153.stdout:osd.3 smithi174 running (4m) 115s ago 20m 83.6M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 70aafac71ad5 2024-03-21T10:22:03.192 INFO:teuthology.orchestra.run.smithi153.stdout:osd.4 smithi174 running (3m) 115s ago 19m 92.2M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 0d746ce6e79d 2024-03-21T10:22:03.192 INFO:teuthology.orchestra.run.smithi153.stdout:osd.5 smithi174 running (3m) 115s ago 19m 86.2M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 53e5f35b8cac 2024-03-21T10:22:03.192 INFO:teuthology.orchestra.run.smithi153.stdout:prometheus.smithi153 smithi153 *:9095 running (11m) 2m ago 23m 73.5M - 2.43.0 a07b618ecd1d 7d95561e50c2 2024-03-21T10:22:03.192 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:03.190+0000 7f4494400700 1 -- 172.21.15.153:0/3139075468 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f447406e460 msgr2=0x7f4474070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:03.193 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:03.190+0000 7f4494400700 1 --2- 172.21.15.153:0/3139075468 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f447406e460 0x7f4474070920 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f4478006fd0 tx=0x7f4478008040 comp rx=0 tx=0).stop 2024-03-21T10:22:03.193 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:03.190+0000 7f4494400700 1 -- 172.21.15.153:0/3139075468 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f448c068670 msgr2=0x7f448c19e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:03.193 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:03.190+0000 7f4494400700 1 --2- 172.21.15.153:0/3139075468 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f448c068670 0x7f448c19e8b0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f448000b5c0 tx=0x7f448000be00 comp rx=0 tx=0).stop 2024-03-21T10:22:03.193 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:03.190+0000 7f4494400700 1 -- 172.21.15.153:0/3139075468 shutdown_connections 2024-03-21T10:22:03.193 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:03.190+0000 7f4494400700 1 --2- 172.21.15.153:0/3139075468 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f447406e460 0x7f4474070920 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:03.193 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:03.190+0000 7f4494400700 1 --2- 172.21.15.153:0/3139075468 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f448c068670 0x7f448c19e8b0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:03.193 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:03.190+0000 7f4494400700 1 --2- 172.21.15.153:0/3139075468 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f448c069020 0x7f448c19edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:03.193 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:03.190+0000 7f4494400700 1 -- 172.21.15.153:0/3139075468 >> 172.21.15.153:0/3139075468 conn(0x7f448c0755a0 msgr2=0x7f448c102760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:03.193 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:03.191+0000 7f4494400700 1 -- 172.21.15.153:0/3139075468 shutdown_connections 2024-03-21T10:22:03.193 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:03.191+0000 7f4494400700 1 -- 172.21.15.153:0/3139075468 wait complete. 2024-03-21T10:22:03.340 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:03.338+0000 7f969ce00700 1 -- 172.21.15.153:0/2559430667 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f969810d280 msgr2=0x7f969810d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:03.341 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:03.338+0000 7f969ce00700 1 --2- 172.21.15.153:0/2559430667 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f969810d280 0x7f969810d660 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f9688009910 tx=0x7f9688009c20 comp rx=0 tx=0).stop 2024-03-21T10:22:03.341 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:03.338+0000 7f969ce00700 1 -- 172.21.15.153:0/2559430667 shutdown_connections 2024-03-21T10:22:03.341 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:03.338+0000 7f969ce00700 1 --2- 172.21.15.153:0/2559430667 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f96981071d0 0x7f9698107650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:03.341 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:03.338+0000 7f969ce00700 1 --2- 172.21.15.153:0/2559430667 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f969810d280 0x7f969810d660 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:03.341 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:03.338+0000 7f969ce00700 1 -- 172.21.15.153:0/2559430667 >> 172.21.15.153:0/2559430667 conn(0x7f9698076990 msgr2=0x7f9698078db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:03.341 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:03.338+0000 7f969ce00700 1 -- 172.21.15.153:0/2559430667 shutdown_connections 2024-03-21T10:22:03.341 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:03.338+0000 7f969ce00700 1 -- 172.21.15.153:0/2559430667 wait complete. 2024-03-21T10:22:03.342 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:03.339+0000 7f969ce00700 1 Processor -- start 2024-03-21T10:22:03.342 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:03.340+0000 7f969ce00700 1 -- start start 2024-03-21T10:22:03.343 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:03.340+0000 7f969ce00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f96981071d0 0x7f969819ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:03.343 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:03.340+0000 7f969ce00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f969810d280 0x7f969819d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:03.343 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:03.340+0000 7f969ce00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f969819d890 con 0x7f96981071d0 2024-03-21T10:22:03.343 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:03.340+0000 7f969ce00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96981a05c0 con 0x7f969810d280 2024-03-21T10:22:03.343 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:03.340+0000 7f9695e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f96981071d0 0x7f969819ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:03.343 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:03.340+0000 7f9695e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f96981071d0 0x7f969819ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39898/0 (socket says 172.21.15.153:39898) 2024-03-21T10:22:03.343 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:03.340+0000 7f9695e00700 1 -- 172.21.15.153:0/2664494347 learned_addr learned my addr 172.21.15.153:0/2664494347 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:22:03.343 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:03.340+0000 7f968fe00700 1 --2- 172.21.15.153:0/2664494347 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f969810d280 0x7f969819d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:03.343 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:03.341+0000 7f9695e00700 1 -- 172.21.15.153:0/2664494347 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f969810d280 msgr2=0x7f969819d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:03.343 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:03.341+0000 7f9695e00700 1 --2- 172.21.15.153:0/2664494347 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f969810d280 0x7f969819d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:03.343 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:03.341+0000 7f9695e00700 1 -- 172.21.15.153:0/2664494347 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96880095f0 con 0x7f96981071d0 2024-03-21T10:22:03.343 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:03.341+0000 7f968fe00700 1 --2- 172.21.15.153:0/2664494347 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f969810d280 0x7f969819d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:22:03.343 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:03.341+0000 7f9695e00700 1 --2- 172.21.15.153:0/2664494347 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f96981071d0 0x7f969819ccc0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f96880092e0 tx=0x7f968800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:03.344 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:03.341+0000 7f968ea00700 1 -- 172.21.15.153:0/2664494347 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f968801d070 con 0x7f96981071d0 2024-03-21T10:22:03.344 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:03.341+0000 7f968ea00700 1 -- 172.21.15.153:0/2664494347 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9688005e00 con 0x7f96981071d0 2024-03-21T10:22:03.344 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:03.341+0000 7f969ce00700 1 -- 172.21.15.153:0/2664494347 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96981a0840 con 0x7f96981071d0 2024-03-21T10:22:03.344 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:03.341+0000 7f968ea00700 1 -- 172.21.15.153:0/2664494347 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9688017400 con 0x7f96981071d0 2024-03-21T10:22:03.344 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:03.341+0000 7f969ce00700 1 -- 172.21.15.153:0/2664494347 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96981a0db0 con 0x7f96981071d0 2024-03-21T10:22:03.344 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:03.342+0000 7f969ce00700 1 -- 172.21.15.153:0/2664494347 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f969804ea90 con 0x7f96981071d0 2024-03-21T10:22:03.345 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:03.344+0000 7f968ea00700 1 -- 172.21.15.153:0/2664494347 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f968800f460 con 0x7f96981071d0 2024-03-21T10:22:03.346 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:03.344+0000 7f968ea00700 1 --2- 172.21.15.153:0/2664494347 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f967c06e460 0x7f967c070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:03.346 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:03.345+0000 7f968ea00700 1 -- 172.21.15.153:0/2664494347 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f968808f970 con 0x7f96981071d0 2024-03-21T10:22:03.346 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:03.345+0000 7f968fe00700 1 --2- 172.21.15.153:0/2664494347 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f967c06e460 0x7f967c070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:03.347 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:03.345+0000 7f968fe00700 1 --2- 172.21.15.153:0/2664494347 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f967c06e460 0x7f967c070920 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9680005fd0 tx=0x7f9680005ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:03.351 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:03.349+0000 7f968ea00700 1 -- 172.21.15.153:0/2664494347 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f9688058a50 con 0x7f96981071d0 2024-03-21T10:22:03.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:22:03 smithi153 ceph-mon[113733]: from='client.34590 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:22:03.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:22:03 smithi153 ceph-mon[113733]: from='client.34594 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:22:03.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:22:03 smithi153 ceph-mon[113733]: from='client.34598 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:22:03.650 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:03.648+0000 7f969ce00700 1 -- 172.21.15.153:0/2664494347 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f969819dfe0 con 0x7f96981071d0 2024-03-21T10:22:03.651 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:03.650+0000 7f968ea00700 1 -- 172.21.15.153:0/2664494347 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+805 (secure 0 0 0) 0x7f968805c110 con 0x7f96981071d0 2024-03-21T10:22:03.652 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:22:03.652 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": { 2024-03-21T10:22:03.652 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:22:03.652 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:22:03.652 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": { 2024-03-21T10:22:03.652 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:22:03.652 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:22:03.652 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": { 2024-03-21T10:22:03.652 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 6 2024-03-21T10:22:03.652 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:22:03.652 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": { 2024-03-21T10:22:03.652 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-03-21T10:22:03.652 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 3 2024-03-21T10:22:03.653 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:22:03.653 INFO:teuthology.orchestra.run.smithi153.stdout: "overall": { 2024-03-21T10:22:03.653 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-03-21T10:22:03.653 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 13 2024-03-21T10:22:03.653 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-03-21T10:22:03.653 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:22:03.654 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:03.652+0000 7f969ce00700 1 -- 172.21.15.153:0/2664494347 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f967c06e460 msgr2=0x7f967c070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:03.654 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:03.652+0000 7f969ce00700 1 --2- 172.21.15.153:0/2664494347 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f967c06e460 0x7f967c070920 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9680005fd0 tx=0x7f9680005ee0 comp rx=0 tx=0).stop 2024-03-21T10:22:03.655 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:03.652+0000 7f969ce00700 1 -- 172.21.15.153:0/2664494347 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f96981071d0 msgr2=0x7f969819ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:03.655 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:03.652+0000 7f969ce00700 1 --2- 172.21.15.153:0/2664494347 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f96981071d0 0x7f969819ccc0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f96880092e0 tx=0x7f968800be20 comp rx=0 tx=0).stop 2024-03-21T10:22:03.655 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:03.652+0000 7f969ce00700 1 -- 172.21.15.153:0/2664494347 shutdown_connections 2024-03-21T10:22:03.655 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:03.652+0000 7f969ce00700 1 --2- 172.21.15.153:0/2664494347 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f967c06e460 0x7f967c070920 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:03.655 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:03.652+0000 7f969ce00700 1 --2- 172.21.15.153:0/2664494347 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f96981071d0 0x7f969819ccc0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:03.655 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:03.652+0000 7f969ce00700 1 --2- 172.21.15.153:0/2664494347 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f969810d280 0x7f969819d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:03.655 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:03.653+0000 7f969ce00700 1 -- 172.21.15.153:0/2664494347 >> 172.21.15.153:0/2664494347 conn(0x7f9698076990 msgr2=0x7f9698078310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:03.655 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:03.653+0000 7f969ce00700 1 -- 172.21.15.153:0/2664494347 shutdown_connections 2024-03-21T10:22:03.655 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:03.653+0000 7f969ce00700 1 -- 172.21.15.153:0/2664494347 wait complete. 2024-03-21T10:22:03.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:22:03 smithi174 ceph-mon[88004]: from='client.34590 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:22:03.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:22:03 smithi174 ceph-mon[88004]: from='client.34594 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:22:03.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:22:03 smithi174 ceph-mon[88004]: from='client.34598 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:22:03.811 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:03.809+0000 7fb161800700 1 -- 172.21.15.153:0/1165498762 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb15c1039b0 msgr2=0x7fb15c107a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:03.863 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:03.809+0000 7fb161800700 1 --2- 172.21.15.153:0/1165498762 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb15c1039b0 0x7fb15c107a00 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fb148009910 tx=0x7fb148009c20 comp rx=0 tx=0).stop 2024-03-21T10:22:03.863 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:03.809+0000 7fb161800700 1 -- 172.21.15.153:0/1165498762 shutdown_connections 2024-03-21T10:22:03.863 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:03.809+0000 7fb161800700 1 --2- 172.21.15.153:0/1165498762 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb15c1039b0 0x7fb15c107a00 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:03.863 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:03.809+0000 7fb161800700 1 --2- 172.21.15.153:0/1165498762 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb15c103000 0x7fb15c1033e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:03.863 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:03.809+0000 7fb161800700 1 -- 172.21.15.153:0/1165498762 >> 172.21.15.153:0/1165498762 conn(0x7fb15c0fe8d0 msgr2=0x7fb15c100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:03.863 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:03.809+0000 7fb161800700 1 -- 172.21.15.153:0/1165498762 shutdown_connections 2024-03-21T10:22:03.863 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:03.809+0000 7fb161800700 1 -- 172.21.15.153:0/1165498762 wait complete. 2024-03-21T10:22:03.866 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:03.810+0000 7fb161800700 1 Processor -- start 2024-03-21T10:22:03.866 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:03.810+0000 7fb161800700 1 -- start start 2024-03-21T10:22:03.866 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:03.811+0000 7fb161800700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb15c103000 0x7fb15c19e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:03.866 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:03.811+0000 7fb161800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb15c1039b0 0x7fb15c19ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:03.866 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:03.811+0000 7fb161800700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb15c19f390 con 0x7fb15c1039b0 2024-03-21T10:22:03.866 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:03.811+0000 7fb161800700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb15c198840 con 0x7fb15c103000 2024-03-21T10:22:03.866 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:03.811+0000 7fb15a000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb15c1039b0 0x7fb15c19ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:03.866 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:03.811+0000 7fb15a000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb15c1039b0 0x7fb15c19ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39922/0 (socket says 172.21.15.153:39922) 2024-03-21T10:22:03.866 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:03.811+0000 7fb15a000700 1 -- 172.21.15.153:0/3819244799 learned_addr learned my addr 172.21.15.153:0/3819244799 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:22:03.866 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:03.811+0000 7fb15aa00700 1 --2- 172.21.15.153:0/3819244799 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb15c103000 0x7fb15c19e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:03.866 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:03.812+0000 7fb15a000700 1 -- 172.21.15.153:0/3819244799 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb15c103000 msgr2=0x7fb15c19e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:03.866 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:03.812+0000 7fb15a000700 1 --2- 172.21.15.153:0/3819244799 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb15c103000 0x7fb15c19e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:03.867 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:03.812+0000 7fb15a000700 1 -- 172.21.15.153:0/3819244799 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1480095f0 con 0x7fb15c1039b0 2024-03-21T10:22:03.867 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:03.812+0000 7fb15a000700 1 --2- 172.21.15.153:0/3819244799 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb15c1039b0 0x7fb15c19ed00 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fb14800bd70 tx=0x7fb14800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:03.867 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:03.812+0000 7fb153400700 1 -- 172.21.15.153:0/3819244799 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb14801d070 con 0x7fb15c1039b0 2024-03-21T10:22:03.867 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:03.812+0000 7fb153400700 1 -- 172.21.15.153:0/3819244799 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb148005dc0 con 0x7fb15c1039b0 2024-03-21T10:22:03.867 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:03.812+0000 7fb161800700 1 -- 172.21.15.153:0/3819244799 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb15c198ac0 con 0x7fb15c1039b0 2024-03-21T10:22:03.867 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:03.812+0000 7fb153400700 1 -- 172.21.15.153:0/3819244799 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb148017400 con 0x7fb15c1039b0 2024-03-21T10:22:03.867 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:03.812+0000 7fb161800700 1 -- 172.21.15.153:0/3819244799 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb15c198fb0 con 0x7fb15c1039b0 2024-03-21T10:22:03.867 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:03.814+0000 7fb161800700 1 -- 172.21.15.153:0/3819244799 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb15c10b400 con 0x7fb15c1039b0 2024-03-21T10:22:03.867 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:03.814+0000 7fb153400700 1 -- 172.21.15.153:0/3819244799 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fb14800f7f0 con 0x7fb15c1039b0 2024-03-21T10:22:03.867 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:03.815+0000 7fb153400700 1 --2- 172.21.15.153:0/3819244799 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb14006e460 0x7fb140070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:03.867 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:03.815+0000 7fb153400700 1 -- 172.21.15.153:0/3819244799 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fb14808f9c0 con 0x7fb15c1039b0 2024-03-21T10:22:03.867 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:03.816+0000 7fb15aa00700 1 --2- 172.21.15.153:0/3819244799 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb14006e460 0x7fb140070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:03.867 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:03.816+0000 7fb15aa00700 1 --2- 172.21.15.153:0/3819244799 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb14006e460 0x7fb140070920 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb14c005950 tx=0x7fb14c0058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:03.867 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:03.820+0000 7fb153400700 1 -- 172.21.15.153:0/3819244799 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb14805c590 con 0x7fb15c1039b0 2024-03-21T10:22:04.082 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:04.080+0000 7fb161800700 1 -- 172.21.15.153:0/3819244799 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fb15c1992e0 con 0x7fb15c1039b0 2024-03-21T10:22:04.083 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:04.082+0000 7fb153400700 1 -- 172.21.15.153:0/3819244799 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1876 (secure 0 0 0) 0x7fb14805c120 con 0x7fb15c1039b0 2024-03-21T10:22:04.085 INFO:teuthology.orchestra.run.smithi153.stdout:e30 2024-03-21T10:22:04.085 INFO:teuthology.orchestra.run.smithi153.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-21T10:22:04.085 INFO:teuthology.orchestra.run.smithi153.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:22:04.085 INFO:teuthology.orchestra.run.smithi153.stdout:legacy client fscid: 1 2024-03-21T10:22:04.085 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:22:04.085 INFO:teuthology.orchestra.run.smithi153.stdout:Filesystem 'cephfs' (1) 2024-03-21T10:22:04.085 INFO:teuthology.orchestra.run.smithi153.stdout:fs_name cephfs 2024-03-21T10:22:04.085 INFO:teuthology.orchestra.run.smithi153.stdout:epoch 23 2024-03-21T10:22:04.085 INFO:teuthology.orchestra.run.smithi153.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-21T10:22:04.086 INFO:teuthology.orchestra.run.smithi153.stdout:created 2024-03-21T10:04:27.086576+0000 2024-03-21T10:22:04.086 INFO:teuthology.orchestra.run.smithi153.stdout:modified 2024-03-21T10:19:05.022092+0000 2024-03-21T10:22:04.086 INFO:teuthology.orchestra.run.smithi153.stdout:tableserver 0 2024-03-21T10:22:04.086 INFO:teuthology.orchestra.run.smithi153.stdout:root 0 2024-03-21T10:22:04.086 INFO:teuthology.orchestra.run.smithi153.stdout:session_timeout 60 2024-03-21T10:22:04.086 INFO:teuthology.orchestra.run.smithi153.stdout:session_autoclose 300 2024-03-21T10:22:04.086 INFO:teuthology.orchestra.run.smithi153.stdout:max_file_size 1099511627776 2024-03-21T10:22:04.086 INFO:teuthology.orchestra.run.smithi153.stdout:max_xattr_size 65536 2024-03-21T10:22:04.086 INFO:teuthology.orchestra.run.smithi153.stdout:required_client_features {} 2024-03-21T10:22:04.086 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure 0 2024-03-21T10:22:04.086 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure_osd_epoch 94 2024-03-21T10:22:04.086 INFO:teuthology.orchestra.run.smithi153.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:22:04.086 INFO:teuthology.orchestra.run.smithi153.stdout:max_mds 2 2024-03-21T10:22:04.086 INFO:teuthology.orchestra.run.smithi153.stdout:in 0,1 2024-03-21T10:22:04.087 INFO:teuthology.orchestra.run.smithi153.stdout:up {} 2024-03-21T10:22:04.087 INFO:teuthology.orchestra.run.smithi153.stdout:failed 0,1 2024-03-21T10:22:04.087 INFO:teuthology.orchestra.run.smithi153.stdout:damaged 2024-03-21T10:22:04.087 INFO:teuthology.orchestra.run.smithi153.stdout:stopped 2024-03-21T10:22:04.087 INFO:teuthology.orchestra.run.smithi153.stdout:data_pools [3] 2024-03-21T10:22:04.087 INFO:teuthology.orchestra.run.smithi153.stdout:metadata_pool 2 2024-03-21T10:22:04.087 INFO:teuthology.orchestra.run.smithi153.stdout:inline_data enabled 2024-03-21T10:22:04.087 INFO:teuthology.orchestra.run.smithi153.stdout:balancer 2024-03-21T10:22:04.087 INFO:teuthology.orchestra.run.smithi153.stdout:bal_rank_mask -1 2024-03-21T10:22:04.087 INFO:teuthology.orchestra.run.smithi153.stdout:standby_count_wanted 1 2024-03-21T10:22:04.087 INFO:teuthology.orchestra.run.smithi153.stdout:qdb_cluster leader: 0 members: 2024-03-21T10:22:04.087 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:22:04.087 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:22:04.087 INFO:teuthology.orchestra.run.smithi153.stdout:Standby daemons: 2024-03-21T10:22:04.087 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:22:04.088 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.zipckr{-1:34442} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6826/1103846295,v1:172.21.15.153:6827/1103846295] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:22:04.088 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.iuyluj{-1:34474} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6828/1738630832,v1:172.21.15.153:6829/1738630832] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:22:04.088 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.nhozag{-1:44269} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.174:6826/3967223227,v1:172.21.15.174:6827/3967223227] compat {c=[1],r=[1],i=[7ff]}] 2024-03-21T10:22:04.088 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.yyimst{-1:44313} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.174:6824/4127786797,v1:172.21.15.174:6825/4127786797] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:22:04.088 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:04.084+0000 7fb161800700 1 -- 172.21.15.153:0/3819244799 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb14006e460 msgr2=0x7fb140070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:04.089 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:04.084+0000 7fb161800700 1 --2- 172.21.15.153:0/3819244799 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb14006e460 0x7fb140070920 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb14c005950 tx=0x7fb14c0058e0 comp rx=0 tx=0).stop 2024-03-21T10:22:04.089 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:04.084+0000 7fb161800700 1 -- 172.21.15.153:0/3819244799 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb15c1039b0 msgr2=0x7fb15c19ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:04.089 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:04.084+0000 7fb161800700 1 --2- 172.21.15.153:0/3819244799 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb15c1039b0 0x7fb15c19ed00 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fb14800bd70 tx=0x7fb14800be50 comp rx=0 tx=0).stop 2024-03-21T10:22:04.089 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:04.084+0000 7fb161800700 1 -- 172.21.15.153:0/3819244799 shutdown_connections 2024-03-21T10:22:04.089 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:04.084+0000 7fb161800700 1 --2- 172.21.15.153:0/3819244799 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb15c103000 0x7fb15c19e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:04.089 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:04.084+0000 7fb161800700 1 --2- 172.21.15.153:0/3819244799 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb14006e460 0x7fb140070920 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:04.089 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:04.084+0000 7fb161800700 1 --2- 172.21.15.153:0/3819244799 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb15c1039b0 0x7fb15c19ed00 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:04.089 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:04.084+0000 7fb161800700 1 -- 172.21.15.153:0/3819244799 >> 172.21.15.153:0/3819244799 conn(0x7fb15c0fe8d0 msgr2=0x7fb15c100cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:04.089 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:04.085+0000 7fb161800700 1 -- 172.21.15.153:0/3819244799 shutdown_connections 2024-03-21T10:22:04.089 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:04.085+0000 7fb161800700 1 -- 172.21.15.153:0/3819244799 wait complete. 2024-03-21T10:22:04.089 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 30 2024-03-21T10:22:04.245 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:04.242+0000 7f2faca00700 1 -- 172.21.15.153:0/1047831321 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2fa81071d0 msgr2=0x7f2fa8107650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:04.245 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:04.242+0000 7f2faca00700 1 --2- 172.21.15.153:0/1047831321 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2fa81071d0 0x7f2fa8107650 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f2f94009910 tx=0x7f2f94009c20 comp rx=0 tx=0).stop 2024-03-21T10:22:04.245 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:04.243+0000 7f2faca00700 1 -- 172.21.15.153:0/1047831321 shutdown_connections 2024-03-21T10:22:04.245 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:04.243+0000 7f2faca00700 1 --2- 172.21.15.153:0/1047831321 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2fa81071d0 0x7f2fa8107650 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:04.245 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:04.243+0000 7f2faca00700 1 --2- 172.21.15.153:0/1047831321 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2fa810d280 0x7f2fa810d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:04.246 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:04.243+0000 7f2faca00700 1 -- 172.21.15.153:0/1047831321 >> 172.21.15.153:0/1047831321 conn(0x7f2fa80769e0 msgr2=0x7f2fa8078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:04.246 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:04.243+0000 7f2faca00700 1 -- 172.21.15.153:0/1047831321 shutdown_connections 2024-03-21T10:22:04.246 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:04.243+0000 7f2faca00700 1 -- 172.21.15.153:0/1047831321 wait complete. 2024-03-21T10:22:04.248 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:04.244+0000 7f2faca00700 1 Processor -- start 2024-03-21T10:22:04.248 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:04.244+0000 7f2faca00700 1 -- start start 2024-03-21T10:22:04.248 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:04.244+0000 7f2faca00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2fa81071d0 0x7f2fa819ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:04.249 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:04.244+0000 7f2faca00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2fa810d280 0x7f2fa819d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:04.249 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:04.244+0000 7f2faca00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa819d880 con 0x7f2fa81071d0 2024-03-21T10:22:04.249 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:04.244+0000 7f2faca00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa81a05b0 con 0x7f2fa810d280 2024-03-21T10:22:04.249 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:04.245+0000 7f2fa5e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2fa81071d0 0x7f2fa819ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:04.249 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:04.245+0000 7f2fa5e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2fa81071d0 0x7f2fa819ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39936/0 (socket says 172.21.15.153:39936) 2024-03-21T10:22:04.249 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:04.245+0000 7f2fa5e00700 1 -- 172.21.15.153:0/1333619637 learned_addr learned my addr 172.21.15.153:0/1333619637 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:22:04.249 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:04.245+0000 7f2fa5400700 1 --2- 172.21.15.153:0/1333619637 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2fa810d280 0x7f2fa819d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:04.249 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:04.245+0000 7f2fa5e00700 1 -- 172.21.15.153:0/1333619637 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2fa810d280 msgr2=0x7f2fa819d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:04.249 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:04.245+0000 7f2fa5e00700 1 --2- 172.21.15.153:0/1333619637 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2fa810d280 0x7f2fa819d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:04.250 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:04.245+0000 7f2fa5e00700 1 -- 172.21.15.153:0/1333619637 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f940095f0 con 0x7f2fa81071d0 2024-03-21T10:22:04.250 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:04.245+0000 7f2fa5400700 1 --2- 172.21.15.153:0/1333619637 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2fa810d280 0x7f2fa819d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:22:04.250 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:04.245+0000 7f2fa5e00700 1 --2- 172.21.15.153:0/1333619637 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2fa81071d0 0x7f2fa819ccb0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f2f9800d8d0 tx=0x7f2f9800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:04.250 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:04.246+0000 7f2f9ea00700 1 -- 172.21.15.153:0/1333619637 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2f98009940 con 0x7f2fa81071d0 2024-03-21T10:22:04.250 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:04.246+0000 7f2f9ea00700 1 -- 172.21.15.153:0/1333619637 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2f98010460 con 0x7f2fa81071d0 2024-03-21T10:22:04.250 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:04.246+0000 7f2f9ea00700 1 -- 172.21.15.153:0/1333619637 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2f9800f5f0 con 0x7f2fa81071d0 2024-03-21T10:22:04.250 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:04.246+0000 7f2faca00700 1 -- 172.21.15.153:0/1333619637 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fa81a0890 con 0x7f2fa81071d0 2024-03-21T10:22:04.250 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:04.246+0000 7f2faca00700 1 -- 172.21.15.153:0/1333619637 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fa81a0e60 con 0x7f2fa81071d0 2024-03-21T10:22:04.252 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:04.247+0000 7f2faca00700 1 -- 172.21.15.153:0/1333619637 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fa810f6c0 con 0x7f2fa81071d0 2024-03-21T10:22:04.252 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:04.248+0000 7f2f9ea00700 1 -- 172.21.15.153:0/1333619637 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f2f980105d0 con 0x7f2fa81071d0 2024-03-21T10:22:04.252 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:04.249+0000 7f2f9ea00700 1 --2- 172.21.15.153:0/1333619637 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f2f8c06e460 0x7f2f8c070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:04.252 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:04.249+0000 7f2f9ea00700 1 -- 172.21.15.153:0/1333619637 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f2f9808e480 con 0x7f2fa81071d0 2024-03-21T10:22:04.252 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:04.249+0000 7f2fa5400700 1 --2- 172.21.15.153:0/1333619637 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f2f8c06e460 0x7f2f8c070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:04.252 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:04.250+0000 7f2fa5400700 1 --2- 172.21.15.153:0/1333619637 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f2f8c06e460 0x7f2f8c070920 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2f94006010 tx=0x7f2f9400b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:04.255 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:04.254+0000 7f2f9ea00700 1 -- 172.21.15.153:0/1333619637 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2f98057520 con 0x7f2fa81071d0 2024-03-21T10:22:04.496 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:04.494+0000 7f2faca00700 1 -- 172.21.15.153:0/1333619637 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2fa81a10d0 con 0x7f2f8c06e460 2024-03-21T10:22:04.498 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:04.496+0000 7f2f9ea00700 1 -- 172.21.15.153:0/1333619637 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f2fa81a10d0 con 0x7f2f8c06e460 2024-03-21T10:22:04.498 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:22:04.498 INFO:teuthology.orchestra.run.smithi153.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232", 2024-03-21T10:22:04.498 INFO:teuthology.orchestra.run.smithi153.stdout: "in_progress": true, 2024-03-21T10:22:04.498 INFO:teuthology.orchestra.run.smithi153.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-21T10:22:04.499 INFO:teuthology.orchestra.run.smithi153.stdout: "services_complete": [ 2024-03-21T10:22:04.499 INFO:teuthology.orchestra.run.smithi153.stdout: "crash", 2024-03-21T10:22:04.499 INFO:teuthology.orchestra.run.smithi153.stdout: "mon", 2024-03-21T10:22:04.499 INFO:teuthology.orchestra.run.smithi153.stdout: "osd", 2024-03-21T10:22:04.499 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr" 2024-03-21T10:22:04.499 INFO:teuthology.orchestra.run.smithi153.stdout: ], 2024-03-21T10:22:04.499 INFO:teuthology.orchestra.run.smithi153.stdout: "progress": "15/23 daemons upgraded", 2024-03-21T10:22:04.499 INFO:teuthology.orchestra.run.smithi153.stdout: "message": "Currently upgrading mds daemons", 2024-03-21T10:22:04.499 INFO:teuthology.orchestra.run.smithi153.stdout: "is_paused": false 2024-03-21T10:22:04.499 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:22:04.501 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:04.499+0000 7f2faca00700 1 -- 172.21.15.153:0/1333619637 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f2f8c06e460 msgr2=0x7f2f8c070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:04.501 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:04.499+0000 7f2faca00700 1 --2- 172.21.15.153:0/1333619637 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f2f8c06e460 0x7f2f8c070920 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2f94006010 tx=0x7f2f9400b540 comp rx=0 tx=0).stop 2024-03-21T10:22:04.501 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:04.499+0000 7f2faca00700 1 -- 172.21.15.153:0/1333619637 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2fa81071d0 msgr2=0x7f2fa819ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:04.502 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:04.499+0000 7f2faca00700 1 --2- 172.21.15.153:0/1333619637 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2fa81071d0 0x7f2fa819ccb0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f2f9800d8d0 tx=0x7f2f9800dc90 comp rx=0 tx=0).stop 2024-03-21T10:22:04.502 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:04.499+0000 7f2faca00700 1 -- 172.21.15.153:0/1333619637 shutdown_connections 2024-03-21T10:22:04.502 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:04.499+0000 7f2faca00700 1 --2- 172.21.15.153:0/1333619637 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f2f8c06e460 0x7f2f8c070920 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:04.502 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:04.499+0000 7f2faca00700 1 --2- 172.21.15.153:0/1333619637 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2fa81071d0 0x7f2fa819ccb0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:04.502 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:04.499+0000 7f2faca00700 1 --2- 172.21.15.153:0/1333619637 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2fa810d280 0x7f2fa819d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:04.502 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:04.499+0000 7f2faca00700 1 -- 172.21.15.153:0/1333619637 >> 172.21.15.153:0/1333619637 conn(0x7f2fa80769e0 msgr2=0x7f2fa8078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:04.502 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:04.499+0000 7f2faca00700 1 -- 172.21.15.153:0/1333619637 shutdown_connections 2024-03-21T10:22:04.502 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:04.499+0000 7f2faca00700 1 -- 172.21.15.153:0/1333619637 wait complete. 2024-03-21T10:22:04.643 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:04.641+0000 7f79bde00700 1 -- 172.21.15.153:0/1084423362 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f79b8073ae0 msgr2=0x7f79b810ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:04.644 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:04.641+0000 7f79bde00700 1 --2- 172.21.15.153:0/1084423362 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f79b8073ae0 0x7f79b810ce40 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f79a4009910 tx=0x7f79a4009c20 comp rx=0 tx=0).stop 2024-03-21T10:22:04.644 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:04.641+0000 7f79bde00700 1 -- 172.21.15.153:0/1084423362 shutdown_connections 2024-03-21T10:22:04.644 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:04.641+0000 7f79bde00700 1 --2- 172.21.15.153:0/1084423362 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f79b8073ae0 0x7f79b810ce40 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:04.644 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:04.641+0000 7f79bde00700 1 --2- 172.21.15.153:0/1084423362 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f79b80731c0 0x7f79b80735a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:04.644 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:04.641+0000 7f79bde00700 1 -- 172.21.15.153:0/1084423362 >> 172.21.15.153:0/1084423362 conn(0x7f79b80fc630 msgr2=0x7f79b80fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:04.645 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:22:04 smithi153 ceph-mon[113733]: pgmap v306: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:22:04.645 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:22:04 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/2664494347' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:22:04.645 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:22:04 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/3819244799' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:22:04.646 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:04.641+0000 7f79bde00700 1 -- 172.21.15.153:0/1084423362 shutdown_connections 2024-03-21T10:22:04.646 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:04.642+0000 7f79bde00700 1 -- 172.21.15.153:0/1084423362 wait complete. 2024-03-21T10:22:04.647 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:04.642+0000 7f79bde00700 1 Processor -- start 2024-03-21T10:22:04.647 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:04.643+0000 7f79bde00700 1 -- start start 2024-03-21T10:22:04.647 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:04.643+0000 7f79bde00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f79b80731c0 0x7f79b8198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:04.647 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:04.643+0000 7f79bde00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f79b8073ae0 0x7f79b8198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:04.647 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:04.643+0000 7f79bde00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79b8199530 con 0x7f79b8073ae0 2024-03-21T10:22:04.647 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:04.643+0000 7f79bde00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79b819d340 con 0x7f79b80731c0 2024-03-21T10:22:04.647 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:04.643+0000 7f79b6000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f79b8073ae0 0x7f79b8198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:04.647 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:04.643+0000 7f79b6000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f79b8073ae0 0x7f79b8198ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39952/0 (socket says 172.21.15.153:39952) 2024-03-21T10:22:04.647 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:04.643+0000 7f79b6000700 1 -- 172.21.15.153:0/3378024397 learned_addr learned my addr 172.21.15.153:0/3378024397 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:22:04.647 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:04.644+0000 7f79b6a00700 1 --2- 172.21.15.153:0/3378024397 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f79b80731c0 0x7f79b8198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:04.647 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:04.644+0000 7f79b6000700 1 -- 172.21.15.153:0/3378024397 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f79b80731c0 msgr2=0x7f79b8198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:04.647 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:04.644+0000 7f79b6000700 1 --2- 172.21.15.153:0/3378024397 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f79b80731c0 0x7f79b8198960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:04.647 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:04.644+0000 7f79b6000700 1 -- 172.21.15.153:0/3378024397 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79a40095f0 con 0x7f79b8073ae0 2024-03-21T10:22:04.648 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:04.644+0000 7f79b6000700 1 --2- 172.21.15.153:0/3378024397 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f79b8073ae0 0x7f79b8198ea0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f79a40092e0 tx=0x7f79a400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:04.648 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:04.644+0000 7f79bca00700 1 -- 172.21.15.153:0/3378024397 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f79a401d070 con 0x7f79b8073ae0 2024-03-21T10:22:04.648 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:04.645+0000 7f79bca00700 1 -- 172.21.15.153:0/3378024397 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f79a400f460 con 0x7f79b8073ae0 2024-03-21T10:22:04.648 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:04.645+0000 7f79bde00700 1 -- 172.21.15.153:0/3378024397 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79b819d5c0 con 0x7f79b8073ae0 2024-03-21T10:22:04.648 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:04.645+0000 7f79bca00700 1 -- 172.21.15.153:0/3378024397 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f79a4005e20 con 0x7f79b8073ae0 2024-03-21T10:22:04.648 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:04.645+0000 7f79bde00700 1 -- 172.21.15.153:0/3378024397 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79b819dab0 con 0x7f79b8073ae0 2024-03-21T10:22:04.648 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:04.646+0000 7f79bde00700 1 -- 172.21.15.153:0/3378024397 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79b810a540 con 0x7f79b8073ae0 2024-03-21T10:22:04.649 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:04.647+0000 7f79bca00700 1 -- 172.21.15.153:0/3378024397 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f79a400f5d0 con 0x7f79b8073ae0 2024-03-21T10:22:04.650 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:04.648+0000 7f79bca00700 1 --2- 172.21.15.153:0/3378024397 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f799c06e460 0x7f799c070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:04.651 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:04.648+0000 7f79bca00700 1 -- 172.21.15.153:0/3378024397 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f79a408f920 con 0x7f79b8073ae0 2024-03-21T10:22:04.651 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:04.648+0000 7f79b6a00700 1 --2- 172.21.15.153:0/3378024397 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f799c06e460 0x7f799c070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:04.651 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:04.649+0000 7f79b6a00700 1 --2- 172.21.15.153:0/3378024397 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f799c06e460 0x7f799c070920 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f79a8005950 tx=0x7f79a80058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:04.654 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:04.652+0000 7f79bca00700 1 -- 172.21.15.153:0/3378024397 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f79a405c650 con 0x7f79b8073ae0 2024-03-21T10:22:04.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:22:04 smithi174 ceph-mon[88004]: pgmap v306: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:22:04.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:22:04 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/2664494347' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:22:04.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:22:04 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/3819244799' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:22:04.944 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:04.942+0000 7f79bde00700 1 -- 172.21.15.153:0/3378024397 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f79b8109f20 con 0x7f79b8073ae0 2024-03-21T10:22:04.945 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:04.943+0000 7f79bca00700 1 -- 172.21.15.153:0/3378024397 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f79a405c1e0 con 0x7f79b8073ae0 2024-03-21T10:22:04.945 INFO:teuthology.orchestra.run.smithi153.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-03-21T10:22:04.945 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-21T10:22:04.945 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is degraded 2024-03-21T10:22:04.945 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-21T10:22:04.945 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-21T10:22:04.945 INFO:teuthology.orchestra.run.smithi153.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-21T10:22:04.945 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-21T10:22:04.948 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:04.946+0000 7f79bde00700 1 -- 172.21.15.153:0/3378024397 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f799c06e460 msgr2=0x7f799c070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:04.948 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:04.946+0000 7f79bde00700 1 --2- 172.21.15.153:0/3378024397 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f799c06e460 0x7f799c070920 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f79a8005950 tx=0x7f79a80058e0 comp rx=0 tx=0).stop 2024-03-21T10:22:04.948 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:04.946+0000 7f79bde00700 1 -- 172.21.15.153:0/3378024397 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f79b8073ae0 msgr2=0x7f79b8198ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:04.948 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:04.946+0000 7f79bde00700 1 --2- 172.21.15.153:0/3378024397 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f79b8073ae0 0x7f79b8198ea0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f79a40092e0 tx=0x7f79a400be20 comp rx=0 tx=0).stop 2024-03-21T10:22:04.948 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:04.946+0000 7f79bde00700 1 -- 172.21.15.153:0/3378024397 shutdown_connections 2024-03-21T10:22:04.949 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:04.946+0000 7f79bde00700 1 --2- 172.21.15.153:0/3378024397 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f79b80731c0 0x7f79b8198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:04.949 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:04.946+0000 7f79bde00700 1 --2- 172.21.15.153:0/3378024397 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f799c06e460 0x7f799c070920 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:04.949 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:04.946+0000 7f79bde00700 1 --2- 172.21.15.153:0/3378024397 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f79b8073ae0 0x7f79b8198ea0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:04.949 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:04.946+0000 7f79bde00700 1 -- 172.21.15.153:0/3378024397 >> 172.21.15.153:0/3378024397 conn(0x7f79b80fc630 msgr2=0x7f79b8107680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:04.949 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:04.946+0000 7f79bde00700 1 -- 172.21.15.153:0/3378024397 shutdown_connections 2024-03-21T10:22:04.949 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:04.946+0000 7f79bde00700 1 -- 172.21.15.153:0/3378024397 wait complete. 2024-03-21T10:22:05.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:22:05 smithi174 ceph-mon[88004]: from='client.34610 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:22:05.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:22:05 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/3378024397' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:22:05.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:22:05 smithi153 ceph-mon[113733]: from='client.34610 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:22:05.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:22:05 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/3378024397' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:22:06.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:22:06 smithi174 ceph-mon[88004]: pgmap v307: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:22:06.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:22:06 smithi153 ceph-mon[113733]: pgmap v307: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:22:08.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:22:08 smithi174 ceph-mon[88004]: pgmap v308: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:22:08.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:22:08 smithi153 ceph-mon[113733]: pgmap v308: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:22:10.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:22:10 smithi153 ceph-mon[113733]: pgmap v309: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:22:10.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:22:10 smithi174 ceph-mon[88004]: pgmap v309: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:22:12.650 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:22:12 smithi153 ceph-mon[113733]: pgmap v310: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:22:12.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:22:12 smithi174 ceph-mon[88004]: pgmap v310: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:22:14.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:22:14 smithi174 ceph-mon[88004]: pgmap v311: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:22:14.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:22:14 smithi153 ceph-mon[113733]: pgmap v311: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:22:16.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:22:16 smithi174 ceph-mon[88004]: pgmap v312: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:22:16.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:22:16 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:22:16.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:22:16 smithi153 ceph-mon[113733]: pgmap v312: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:22:16.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:22:16 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:22:18.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:22:18 smithi174 ceph-mon[88004]: pgmap v313: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:22:18.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:22:18 smithi153 ceph-mon[113733]: pgmap v313: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:22:20.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:22:20 smithi174 ceph-mon[88004]: pgmap v314: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:22:20.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:22:20 smithi153 ceph-mon[113733]: pgmap v314: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:22:22.609 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:22:22 smithi153 ceph-mon[113733]: pgmap v315: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:22:22.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:22:22 smithi174 ceph-mon[88004]: pgmap v315: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:22:24.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:22:24 smithi174 ceph-mon[88004]: pgmap v316: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:22:24.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:22:24 smithi153 ceph-mon[113733]: pgmap v316: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:22:26.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:22:26 smithi153 ceph-mon[113733]: pgmap v317: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:22:26.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:22:26 smithi174 ceph-mon[88004]: pgmap v317: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:22:28.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:22:28 smithi153 ceph-mon[113733]: pgmap v318: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:22:28.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:22:28 smithi174 ceph-mon[88004]: pgmap v318: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:22:30.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:22:30 smithi153 ceph-mon[113733]: pgmap v319: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:22:30.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:22:30 smithi174 ceph-mon[88004]: pgmap v319: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:22:31.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:22:31 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:22:31.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:22:31 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:22:32.650 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:22:32 smithi153 ceph-mon[113733]: pgmap v320: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:22:32.853 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:22:32 smithi174 ceph-mon[88004]: pgmap v320: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:22:34.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:22:34 smithi153 ceph-mon[113733]: pgmap v321: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:22:34.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:22:34 smithi174 ceph-mon[88004]: pgmap v321: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:22:35.126 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.124+0000 7efd79600700 1 -- 172.21.15.153:0/1028460723 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efd74075be0 msgr2=0x7efd741114a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:35.126 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.124+0000 7efd79600700 1 --2- 172.21.15.153:0/1028460723 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efd74075be0 0x7efd741114a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7efd60009910 tx=0x7efd60009c20 comp rx=0 tx=0).stop 2024-03-21T10:22:35.126 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.124+0000 7efd79600700 1 -- 172.21.15.153:0/1028460723 shutdown_connections 2024-03-21T10:22:35.126 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.124+0000 7efd79600700 1 --2- 172.21.15.153:0/1028460723 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efd74075be0 0x7efd741114a0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:35.126 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.124+0000 7efd79600700 1 --2- 172.21.15.153:0/1028460723 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efd740752c0 0x7efd740756a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:35.126 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.124+0000 7efd79600700 1 -- 172.21.15.153:0/1028460723 >> 172.21.15.153:0/1028460723 conn(0x7efd740fe810 msgr2=0x7efd74100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:35.127 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.124+0000 7efd79600700 1 -- 172.21.15.153:0/1028460723 shutdown_connections 2024-03-21T10:22:35.127 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.124+0000 7efd79600700 1 -- 172.21.15.153:0/1028460723 wait complete. 2024-03-21T10:22:35.127 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.125+0000 7efd79600700 1 Processor -- start 2024-03-21T10:22:35.128 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.125+0000 7efd79600700 1 -- start start 2024-03-21T10:22:35.128 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.126+0000 7efd79600700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efd740752c0 0x7efd741a2a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:35.128 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.126+0000 7efd79600700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efd74075be0 0x7efd741a2f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:35.128 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.126+0000 7efd79600700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd741a3620 con 0x7efd740752c0 2024-03-21T10:22:35.128 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.126+0000 7efd79600700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd7419cae0 con 0x7efd74075be0 2024-03-21T10:22:35.128 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.126+0000 7efd72a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efd740752c0 0x7efd741a2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:35.128 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.126+0000 7efd72a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efd740752c0 0x7efd741a2a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:48190/0 (socket says 172.21.15.153:48190) 2024-03-21T10:22:35.128 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.126+0000 7efd72a00700 1 -- 172.21.15.153:0/3127039378 learned_addr learned my addr 172.21.15.153:0/3127039378 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:22:35.129 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.126+0000 7efd72000700 1 --2- 172.21.15.153:0/3127039378 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efd74075be0 0x7efd741a2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:35.129 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.127+0000 7efd72a00700 1 -- 172.21.15.153:0/3127039378 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efd74075be0 msgr2=0x7efd741a2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:35.129 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.127+0000 7efd72a00700 1 --2- 172.21.15.153:0/3127039378 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efd74075be0 0x7efd741a2f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:35.129 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.127+0000 7efd72a00700 1 -- 172.21.15.153:0/3127039378 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd600095f0 con 0x7efd740752c0 2024-03-21T10:22:35.129 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.127+0000 7efd72a00700 1 --2- 172.21.15.153:0/3127039378 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efd740752c0 0x7efd741a2a00 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7efd6400d8d0 tx=0x7efd6400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:35.129 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.127+0000 7efd6b400700 1 -- 172.21.15.153:0/3127039378 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efd64009940 con 0x7efd740752c0 2024-03-21T10:22:35.130 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.127+0000 7efd6b400700 1 -- 172.21.15.153:0/3127039378 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efd64010460 con 0x7efd740752c0 2024-03-21T10:22:35.130 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.127+0000 7efd79600700 1 -- 172.21.15.153:0/3127039378 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd7419cdc0 con 0x7efd740752c0 2024-03-21T10:22:35.130 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.127+0000 7efd6b400700 1 -- 172.21.15.153:0/3127039378 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efd6400f5f0 con 0x7efd740752c0 2024-03-21T10:22:35.130 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.127+0000 7efd79600700 1 -- 172.21.15.153:0/3127039378 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd7419d310 con 0x7efd740752c0 2024-03-21T10:22:35.130 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.129+0000 7efd79600700 1 -- 172.21.15.153:0/3127039378 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd7410ec10 con 0x7efd740752c0 2024-03-21T10:22:35.131 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.130+0000 7efd6b400700 1 -- 172.21.15.153:0/3127039378 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7efd640105d0 con 0x7efd740752c0 2024-03-21T10:22:35.132 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.131+0000 7efd6b400700 1 --2- 172.21.15.153:0/3127039378 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7efd5806e460 0x7efd58070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:35.132 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.131+0000 7efd6b400700 1 -- 172.21.15.153:0/3127039378 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7efd6408e3e0 con 0x7efd740752c0 2024-03-21T10:22:35.133 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.131+0000 7efd72000700 1 --2- 172.21.15.153:0/3127039378 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7efd5806e460 0x7efd58070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:35.133 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.132+0000 7efd72000700 1 --2- 172.21.15.153:0/3127039378 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7efd5806e460 0x7efd58070920 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7efd60005b40 tx=0x7efd60005a90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:35.138 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.136+0000 7efd6b400700 1 -- 172.21.15.153:0/3127039378 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7efd6405b050 con 0x7efd740752c0 2024-03-21T10:22:35.380 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.378+0000 7efd79600700 1 -- 172.21.15.153:0/3127039378 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7efd7419d580 con 0x7efd5806e460 2024-03-21T10:22:35.381 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.379+0000 7efd6b400700 1 -- 172.21.15.153:0/3127039378 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7efd7419d580 con 0x7efd5806e460 2024-03-21T10:22:35.384 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.382+0000 7efd79600700 1 -- 172.21.15.153:0/3127039378 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7efd5806e460 msgr2=0x7efd58070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:35.384 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.382+0000 7efd79600700 1 --2- 172.21.15.153:0/3127039378 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7efd5806e460 0x7efd58070920 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7efd60005b40 tx=0x7efd60005a90 comp rx=0 tx=0).stop 2024-03-21T10:22:35.384 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.382+0000 7efd79600700 1 -- 172.21.15.153:0/3127039378 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efd740752c0 msgr2=0x7efd741a2a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:35.384 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.382+0000 7efd79600700 1 --2- 172.21.15.153:0/3127039378 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efd740752c0 0x7efd741a2a00 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7efd6400d8d0 tx=0x7efd6400dc90 comp rx=0 tx=0).stop 2024-03-21T10:22:35.384 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.382+0000 7efd79600700 1 -- 172.21.15.153:0/3127039378 shutdown_connections 2024-03-21T10:22:35.385 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.382+0000 7efd79600700 1 --2- 172.21.15.153:0/3127039378 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7efd5806e460 0x7efd58070920 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:35.385 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.382+0000 7efd79600700 1 --2- 172.21.15.153:0/3127039378 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efd740752c0 0x7efd741a2a00 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:35.385 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.382+0000 7efd79600700 1 --2- 172.21.15.153:0/3127039378 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efd74075be0 0x7efd741a2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:35.385 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.382+0000 7efd79600700 1 -- 172.21.15.153:0/3127039378 >> 172.21.15.153:0/3127039378 conn(0x7efd740fe810 msgr2=0x7efd740ff480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:35.385 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.382+0000 7efd79600700 1 -- 172.21.15.153:0/3127039378 shutdown_connections 2024-03-21T10:22:35.385 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.382+0000 7efd79600700 1 -- 172.21.15.153:0/3127039378 wait complete. 2024-03-21T10:22:35.402 INFO:teuthology.orchestra.run.smithi153.stdout:true 2024-03-21T10:22:35.547 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.544+0000 7fb5c6000700 1 -- 172.21.15.153:0/1448153884 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb5c0073160 msgr2=0x7fb5c0073540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:35.547 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.544+0000 7fb5c6000700 1 --2- 172.21.15.153:0/1448153884 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb5c0073160 0x7fb5c0073540 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb5b0009910 tx=0x7fb5b0009c20 comp rx=0 tx=0).stop 2024-03-21T10:22:35.547 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.544+0000 7fb5c6000700 1 -- 172.21.15.153:0/1448153884 shutdown_connections 2024-03-21T10:22:35.547 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.544+0000 7fb5c6000700 1 --2- 172.21.15.153:0/1448153884 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb5c0073a80 0x7fb5c010cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:35.547 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.544+0000 7fb5c6000700 1 --2- 172.21.15.153:0/1448153884 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb5c0073160 0x7fb5c0073540 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:35.547 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.544+0000 7fb5c6000700 1 -- 172.21.15.153:0/1448153884 >> 172.21.15.153:0/1448153884 conn(0x7fb5c00fc4d0 msgr2=0x7fb5c00fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:35.547 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.545+0000 7fb5c6000700 1 -- 172.21.15.153:0/1448153884 shutdown_connections 2024-03-21T10:22:35.547 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.545+0000 7fb5c6000700 1 -- 172.21.15.153:0/1448153884 wait complete. 2024-03-21T10:22:35.548 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.546+0000 7fb5c6000700 1 Processor -- start 2024-03-21T10:22:35.548 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.546+0000 7fb5c6000700 1 -- start start 2024-03-21T10:22:35.548 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.546+0000 7fb5c6000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb5c0073160 0x7fb5c019e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:35.548 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.546+0000 7fb5c6000700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb5c0073a80 0x7fb5c019ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:35.548 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.546+0000 7fb5c6000700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c019f280 con 0x7fb5c0073160 2024-03-21T10:22:35.549 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.546+0000 7fb5c6000700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c0198730 con 0x7fb5c0073a80 2024-03-21T10:22:35.549 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.547+0000 7fb5c4c00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb5c0073160 0x7fb5c019e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:35.549 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.547+0000 7fb5c4c00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb5c0073160 0x7fb5c019e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:48204/0 (socket says 172.21.15.153:48204) 2024-03-21T10:22:35.549 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.547+0000 7fb5c4c00700 1 -- 172.21.15.153:0/1281184731 learned_addr learned my addr 172.21.15.153:0/1281184731 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:22:35.550 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.547+0000 7fb5b7e00700 1 --2- 172.21.15.153:0/1281184731 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb5c0073a80 0x7fb5c019ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:35.550 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.547+0000 7fb5c4c00700 1 -- 172.21.15.153:0/1281184731 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb5c0073a80 msgr2=0x7fb5c019ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:35.550 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.547+0000 7fb5c4c00700 1 --2- 172.21.15.153:0/1281184731 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb5c0073a80 0x7fb5c019ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:35.550 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.547+0000 7fb5c4c00700 1 -- 172.21.15.153:0/1281184731 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5b00095f0 con 0x7fb5c0073160 2024-03-21T10:22:35.550 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.547+0000 7fb5b7e00700 1 --2- 172.21.15.153:0/1281184731 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb5c0073a80 0x7fb5c019ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:22:35.550 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.547+0000 7fb5c4c00700 1 --2- 172.21.15.153:0/1281184731 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb5c0073160 0x7fb5c019e6b0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fb5b00092e0 tx=0x7fb5b000be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:35.550 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.548+0000 7fb5be000700 1 -- 172.21.15.153:0/1281184731 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb5b001d070 con 0x7fb5c0073160 2024-03-21T10:22:35.550 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.548+0000 7fb5be000700 1 -- 172.21.15.153:0/1281184731 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb5b000f460 con 0x7fb5c0073160 2024-03-21T10:22:35.551 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.548+0000 7fb5be000700 1 -- 172.21.15.153:0/1281184731 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb5b0017630 con 0x7fb5c0073160 2024-03-21T10:22:35.551 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.548+0000 7fb5c6000700 1 -- 172.21.15.153:0/1281184731 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5c01989b0 con 0x7fb5c0073160 2024-03-21T10:22:35.551 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.548+0000 7fb5c6000700 1 -- 172.21.15.153:0/1281184731 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5c0198ea0 con 0x7fb5c0073160 2024-03-21T10:22:35.551 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.549+0000 7fb5c6000700 1 -- 172.21.15.153:0/1281184731 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5c010a420 con 0x7fb5c0073160 2024-03-21T10:22:35.552 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.550+0000 7fb5be000700 1 -- 172.21.15.153:0/1281184731 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fb5b000f5d0 con 0x7fb5c0073160 2024-03-21T10:22:35.553 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.551+0000 7fb5be000700 1 --2- 172.21.15.153:0/1281184731 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb5a406e460 0x7fb5a4070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:35.553 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.551+0000 7fb5be000700 1 -- 172.21.15.153:0/1281184731 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fb5b008fa10 con 0x7fb5c0073160 2024-03-21T10:22:35.553 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.551+0000 7fb5b7e00700 1 --2- 172.21.15.153:0/1281184731 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb5a406e460 0x7fb5a4070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:35.554 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.552+0000 7fb5b7e00700 1 --2- 172.21.15.153:0/1281184731 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb5a406e460 0x7fb5a4070920 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb5c0199ba0 tx=0x7fb5a8008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:35.558 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.556+0000 7fb5be000700 1 -- 172.21.15.153:0/1281184731 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb5b005c680 con 0x7fb5c0073160 2024-03-21T10:22:35.799 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.797+0000 7fb5c6000700 1 -- 172.21.15.153:0/1281184731 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb5c0199110 con 0x7fb5a406e460 2024-03-21T10:22:35.801 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.799+0000 7fb5be000700 1 -- 172.21.15.153:0/1281184731 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fb5c0199110 con 0x7fb5a406e460 2024-03-21T10:22:35.804 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.803+0000 7fb5c6000700 1 -- 172.21.15.153:0/1281184731 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb5a406e460 msgr2=0x7fb5a4070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:35.805 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.803+0000 7fb5c6000700 1 --2- 172.21.15.153:0/1281184731 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb5a406e460 0x7fb5a4070920 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb5c0199ba0 tx=0x7fb5a8008040 comp rx=0 tx=0).stop 2024-03-21T10:22:35.805 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.803+0000 7fb5c6000700 1 -- 172.21.15.153:0/1281184731 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb5c0073160 msgr2=0x7fb5c019e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:35.805 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.803+0000 7fb5c6000700 1 --2- 172.21.15.153:0/1281184731 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb5c0073160 0x7fb5c019e6b0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fb5b00092e0 tx=0x7fb5b000be00 comp rx=0 tx=0).stop 2024-03-21T10:22:35.805 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.803+0000 7fb5c6000700 1 -- 172.21.15.153:0/1281184731 shutdown_connections 2024-03-21T10:22:35.805 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.803+0000 7fb5c6000700 1 --2- 172.21.15.153:0/1281184731 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb5a406e460 0x7fb5a4070920 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:35.805 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.803+0000 7fb5c6000700 1 --2- 172.21.15.153:0/1281184731 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb5c0073160 0x7fb5c019e6b0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:35.805 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.803+0000 7fb5c6000700 1 --2- 172.21.15.153:0/1281184731 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb5c0073a80 0x7fb5c019ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:35.806 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.803+0000 7fb5c6000700 1 -- 172.21.15.153:0/1281184731 >> 172.21.15.153:0/1281184731 conn(0x7fb5c00fc4d0 msgr2=0x7fb5c0107560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:35.806 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.803+0000 7fb5c6000700 1 -- 172.21.15.153:0/1281184731 shutdown_connections 2024-03-21T10:22:35.806 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.803+0000 7fb5c6000700 1 -- 172.21.15.153:0/1281184731 wait complete. 2024-03-21T10:22:35.961 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.959+0000 7fb8ca200700 1 -- 172.21.15.153:0/3980779676 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb8c410d280 msgr2=0x7fb8c410d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:35.961 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.959+0000 7fb8ca200700 1 --2- 172.21.15.153:0/3980779676 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb8c410d280 0x7fb8c410d660 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fb8b4009910 tx=0x7fb8b4009c20 comp rx=0 tx=0).stop 2024-03-21T10:22:35.961 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.959+0000 7fb8ca200700 1 -- 172.21.15.153:0/3980779676 shutdown_connections 2024-03-21T10:22:35.961 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.959+0000 7fb8ca200700 1 --2- 172.21.15.153:0/3980779676 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb8c41071d0 0x7fb8c4107650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:35.961 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.959+0000 7fb8ca200700 1 --2- 172.21.15.153:0/3980779676 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb8c410d280 0x7fb8c410d660 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:35.961 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.959+0000 7fb8ca200700 1 -- 172.21.15.153:0/3980779676 >> 172.21.15.153:0/3980779676 conn(0x7fb8c40769e0 msgr2=0x7fb8c4078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:35.961 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.959+0000 7fb8ca200700 1 -- 172.21.15.153:0/3980779676 shutdown_connections 2024-03-21T10:22:35.962 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.959+0000 7fb8ca200700 1 -- 172.21.15.153:0/3980779676 wait complete. 2024-03-21T10:22:35.962 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.960+0000 7fb8ca200700 1 Processor -- start 2024-03-21T10:22:35.962 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.960+0000 7fb8ca200700 1 -- start start 2024-03-21T10:22:35.963 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.961+0000 7fb8ca200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb8c41071d0 0x7fb8c419ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:35.963 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.961+0000 7fb8ca200700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb8c410d280 0x7fb8c419d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:35.963 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.961+0000 7fb8ca200700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c419d810 con 0x7fb8c41071d0 2024-03-21T10:22:35.963 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.961+0000 7fb8ca200700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c419d980 con 0x7fb8c410d280 2024-03-21T10:22:35.963 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.961+0000 7fb8c3400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb8c41071d0 0x7fb8c419ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:35.963 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.961+0000 7fb8c3400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb8c41071d0 0x7fb8c419ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:48220/0 (socket says 172.21.15.153:48220) 2024-03-21T10:22:35.963 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.961+0000 7fb8c3400700 1 -- 172.21.15.153:0/295897577 learned_addr learned my addr 172.21.15.153:0/295897577 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:22:35.964 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.962+0000 7fb8baa00700 1 --2- 172.21.15.153:0/295897577 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb8c410d280 0x7fb8c419d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:35.964 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.962+0000 7fb8c3400700 1 -- 172.21.15.153:0/295897577 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb8c410d280 msgr2=0x7fb8c419d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:35.964 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.962+0000 7fb8c3400700 1 --2- 172.21.15.153:0/295897577 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb8c410d280 0x7fb8c419d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:35.964 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.962+0000 7fb8c3400700 1 -- 172.21.15.153:0/295897577 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8b40095f0 con 0x7fb8c41071d0 2024-03-21T10:22:35.965 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.962+0000 7fb8baa00700 1 --2- 172.21.15.153:0/295897577 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb8c410d280 0x7fb8c419d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:22:35.965 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.962+0000 7fb8c3400700 1 --2- 172.21.15.153:0/295897577 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb8c41071d0 0x7fb8c419ccd0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fb8b400b5c0 tx=0x7fb8b400be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:35.965 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.962+0000 7fb8c0c00700 1 -- 172.21.15.153:0/295897577 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb8b401d070 con 0x7fb8c41071d0 2024-03-21T10:22:35.965 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.962+0000 7fb8c0c00700 1 -- 172.21.15.153:0/295897577 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb8b400f460 con 0x7fb8c41071d0 2024-03-21T10:22:35.965 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.962+0000 7fb8c0c00700 1 -- 172.21.15.153:0/295897577 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb8b4017610 con 0x7fb8c41071d0 2024-03-21T10:22:35.965 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.962+0000 7fb8ca200700 1 -- 172.21.15.153:0/295897577 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8c41a06c0 con 0x7fb8c41071d0 2024-03-21T10:22:35.965 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.962+0000 7fb8ca200700 1 -- 172.21.15.153:0/295897577 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8c41a0c30 con 0x7fb8c41071d0 2024-03-21T10:22:35.965 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.964+0000 7fb8ca200700 1 -- 172.21.15.153:0/295897577 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8c404ea90 con 0x7fb8c41071d0 2024-03-21T10:22:35.966 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.965+0000 7fb8c0c00700 1 -- 172.21.15.153:0/295897577 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fb8b400f5d0 con 0x7fb8c41071d0 2024-03-21T10:22:35.967 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.966+0000 7fb8c0c00700 1 --2- 172.21.15.153:0/295897577 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb8a806e460 0x7fb8a8070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:35.967 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.966+0000 7fb8c0c00700 1 -- 172.21.15.153:0/295897577 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fb8b408fae0 con 0x7fb8c41071d0 2024-03-21T10:22:35.968 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.966+0000 7fb8baa00700 1 --2- 172.21.15.153:0/295897577 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb8a806e460 0x7fb8a8070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:35.968 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.966+0000 7fb8baa00700 1 --2- 172.21.15.153:0/295897577 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb8a806e460 0x7fb8a8070920 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac005fd0 tx=0x7fb8ac005e20 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:35.972 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:35.971+0000 7fb8c0c00700 1 -- 172.21.15.153:0/295897577 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb8b4058ba0 con 0x7fb8c41071d0 2024-03-21T10:22:36.215 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:36.213+0000 7fb8ca200700 1 -- 172.21.15.153:0/295897577 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fb8c41a11f0 con 0x7fb8a806e460 2024-03-21T10:22:36.245 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:36.243+0000 7fb8c0c00700 1 -- 172.21.15.153:0/295897577 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fb8c41a11f0 con 0x7fb8a806e460 2024-03-21T10:22:36.246 INFO:teuthology.orchestra.run.smithi153.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-21T10:22:36.246 INFO:teuthology.orchestra.run.smithi153.stdout:alertmanager.smithi153 smithi153 *:9093,9094 running (11m) 2m ago 25m 14.7M - 0.25.0 c8568f914cd2 9ed3f2200b5f 2024-03-21T10:22:36.246 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi153 smithi153 running (25m) 2m ago 25m 9424k - 18.2.1 5be31c24972a 5b61994396a6 2024-03-21T10:22:36.246 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi174 smithi174 running (23m) 2m ago 23m 9244k - 18.2.1 5be31c24972a 4f7a5fc3ac28 2024-03-21T10:22:36.246 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi153 smithi153 running (7m) 2m ago 25m 6656k - 19.0.0-2231-g2edfee68 237e43c7a65f 32c21104d5b0 2024-03-21T10:22:36.246 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi174 smithi174 running (7m) 2m ago 23m 6660k - 19.0.0-2231-g2edfee68 237e43c7a65f a04e48967ab6 2024-03-21T10:22:36.246 INFO:teuthology.orchestra.run.smithi153.stdout:grafana.smithi153 smithi153 *:3000 running (10m) 2m ago 24m 73.6M - 9.4.12 2bacad6d85d8 ee24026ab786 2024-03-21T10:22:36.246 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.iuyluj smithi153 running (2m) 2m ago 17m 10.9M - 19.0.0-2231-g2edfee68 237e43c7a65f 87aff8a5de70 2024-03-21T10:22:36.246 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.zipckr smithi153 running (3m) 2m ago 18m 11.8M - 19.0.0-2231-g2edfee68 237e43c7a65f ccc262bd5dd3 2024-03-21T10:22:36.246 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.nhozag smithi174 running (17m) 2m ago 17m 12.0M - 18.2.1 5be31c24972a 2fee4000308f 2024-03-21T10:22:36.247 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.yyimst smithi174 running (2m) 2m ago 18m 10.7M - 19.0.0-2231-g2edfee68 237e43c7a65f fc1962f9351e 2024-03-21T10:22:36.247 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi153.tluxni smithi153 *:8443,9283,8765 running (14m) 2m ago 26m 509M - 19.0.0-2231-g2edfee68 237e43c7a65f 7ffee0e55382 2024-03-21T10:22:36.247 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi174.qpcwat smithi174 *:8443,9283,8765 running (13m) 2m ago 23m 452M - 19.0.0-2231-g2edfee68 237e43c7a65f 748968f216ab 2024-03-21T10:22:36.247 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi153 smithi153 running (9m) 2m ago 27m 55.0M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f dd8038303dc3 2024-03-21T10:22:36.247 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi174 smithi174 running (8m) 2m ago 23m 44.2M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f 29d78a788f2c 2024-03-21T10:22:36.247 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi153 smithi153 *:9100 running (12m) 2m ago 25m 7889k - 1.5.0 0da6a335fe13 bb835edc0e0b 2024-03-21T10:22:36.247 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi174 smithi174 *:9100 running (12m) 2m ago 23m 8102k - 1.5.0 0da6a335fe13 7c4ecad790fa 2024-03-21T10:22:36.247 INFO:teuthology.orchestra.run.smithi153.stdout:osd.0 smithi153 running (6m) 2m ago 22m 116M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 507900fd4791 2024-03-21T10:22:36.247 INFO:teuthology.orchestra.run.smithi153.stdout:osd.1 smithi153 running (6m) 2m ago 22m 89.2M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 0d646452e9c1 2024-03-21T10:22:36.247 INFO:teuthology.orchestra.run.smithi153.stdout:osd.2 smithi153 running (5m) 2m ago 21m 87.4M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 1326f2afdfcd 2024-03-21T10:22:36.247 INFO:teuthology.orchestra.run.smithi153.stdout:osd.3 smithi174 running (5m) 2m ago 20m 83.6M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 70aafac71ad5 2024-03-21T10:22:36.247 INFO:teuthology.orchestra.run.smithi153.stdout:osd.4 smithi174 running (4m) 2m ago 20m 92.2M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 0d746ce6e79d 2024-03-21T10:22:36.247 INFO:teuthology.orchestra.run.smithi153.stdout:osd.5 smithi174 running (3m) 2m ago 19m 86.2M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 53e5f35b8cac 2024-03-21T10:22:36.247 INFO:teuthology.orchestra.run.smithi153.stdout:prometheus.smithi153 smithi153 *:9095 running (11m) 2m ago 24m 73.5M - 2.43.0 a07b618ecd1d 7d95561e50c2 2024-03-21T10:22:36.248 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:36.246+0000 7fb8ca200700 1 -- 172.21.15.153:0/295897577 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb8a806e460 msgr2=0x7fb8a8070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:36.248 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:36.246+0000 7fb8ca200700 1 --2- 172.21.15.153:0/295897577 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb8a806e460 0x7fb8a8070920 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac005fd0 tx=0x7fb8ac005e20 comp rx=0 tx=0).stop 2024-03-21T10:22:36.248 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:36.246+0000 7fb8ca200700 1 -- 172.21.15.153:0/295897577 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb8c41071d0 msgr2=0x7fb8c419ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:36.249 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:36.246+0000 7fb8ca200700 1 --2- 172.21.15.153:0/295897577 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb8c41071d0 0x7fb8c419ccd0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fb8b400b5c0 tx=0x7fb8b400be00 comp rx=0 tx=0).stop 2024-03-21T10:22:36.249 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:36.247+0000 7fb8ca200700 1 -- 172.21.15.153:0/295897577 shutdown_connections 2024-03-21T10:22:36.249 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:36.247+0000 7fb8ca200700 1 --2- 172.21.15.153:0/295897577 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb8a806e460 0x7fb8a8070920 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:36.249 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:36.247+0000 7fb8ca200700 1 --2- 172.21.15.153:0/295897577 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb8c41071d0 0x7fb8c419ccd0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:36.249 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:36.247+0000 7fb8ca200700 1 --2- 172.21.15.153:0/295897577 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb8c410d280 0x7fb8c419d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:36.249 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:36.247+0000 7fb8ca200700 1 -- 172.21.15.153:0/295897577 >> 172.21.15.153:0/295897577 conn(0x7fb8c40769e0 msgr2=0x7fb8c4078320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:36.249 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:36.247+0000 7fb8ca200700 1 -- 172.21.15.153:0/295897577 shutdown_connections 2024-03-21T10:22:36.249 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:36.247+0000 7fb8ca200700 1 -- 172.21.15.153:0/295897577 wait complete. 2024-03-21T10:22:36.406 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:36.403+0000 7f5542600700 1 -- 172.21.15.153:0/2675761722 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f553c1039b0 msgr2=0x7f553c107a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:36.406 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:36.403+0000 7f5542600700 1 --2- 172.21.15.153:0/2675761722 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f553c1039b0 0x7f553c107a00 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f5528009910 tx=0x7f5528009c20 comp rx=0 tx=0).stop 2024-03-21T10:22:36.406 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:36.404+0000 7f5542600700 1 -- 172.21.15.153:0/2675761722 shutdown_connections 2024-03-21T10:22:36.406 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:36.404+0000 7f5542600700 1 --2- 172.21.15.153:0/2675761722 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f553c1039b0 0x7f553c107a00 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:36.406 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:36.404+0000 7f5542600700 1 --2- 172.21.15.153:0/2675761722 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f553c103000 0x7f553c1033e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:36.406 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:36.404+0000 7f5542600700 1 -- 172.21.15.153:0/2675761722 >> 172.21.15.153:0/2675761722 conn(0x7f553c0fe8d0 msgr2=0x7f553c100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:36.406 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:36.404+0000 7f5542600700 1 -- 172.21.15.153:0/2675761722 shutdown_connections 2024-03-21T10:22:36.406 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:36.404+0000 7f5542600700 1 -- 172.21.15.153:0/2675761722 wait complete. 2024-03-21T10:22:36.408 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:36.405+0000 7f5542600700 1 Processor -- start 2024-03-21T10:22:36.408 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:36.405+0000 7f5542600700 1 -- start start 2024-03-21T10:22:36.408 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:36.405+0000 7f5542600700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f553c103000 0x7f553c19e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:36.408 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:36.405+0000 7f5542600700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f553c1039b0 0x7f553c19ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:36.408 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:36.405+0000 7f5542600700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f553c19f390 con 0x7f553c1039b0 2024-03-21T10:22:36.408 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:36.405+0000 7f5542600700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f553c198840 con 0x7f553c103000 2024-03-21T10:22:36.408 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:36.406+0000 7f553aa00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f553c1039b0 0x7f553c19ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:36.408 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:36.406+0000 7f553aa00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f553c1039b0 0x7f553c19ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:48238/0 (socket says 172.21.15.153:48238) 2024-03-21T10:22:36.408 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:36.406+0000 7f553aa00700 1 -- 172.21.15.153:0/3870046772 learned_addr learned my addr 172.21.15.153:0/3870046772 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:22:36.408 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:36.406+0000 7f553b400700 1 --2- 172.21.15.153:0/3870046772 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f553c103000 0x7f553c19e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:36.408 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:36.406+0000 7f553aa00700 1 -- 172.21.15.153:0/3870046772 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f553c103000 msgr2=0x7f553c19e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:36.409 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:36.406+0000 7f553aa00700 1 --2- 172.21.15.153:0/3870046772 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f553c103000 0x7f553c19e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:36.409 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:36.406+0000 7f553aa00700 1 -- 172.21.15.153:0/3870046772 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55280095f0 con 0x7f553c1039b0 2024-03-21T10:22:36.409 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:36.406+0000 7f553b400700 1 --2- 172.21.15.153:0/3870046772 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f553c103000 0x7f553c19e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:22:36.409 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:36.406+0000 7f553aa00700 1 --2- 172.21.15.153:0/3870046772 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f553c1039b0 0x7f553c19ed00 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f552800bda0 tx=0x7f552800be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:36.410 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:36.407+0000 7f5533e00700 1 -- 172.21.15.153:0/3870046772 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f552801d070 con 0x7f553c1039b0 2024-03-21T10:22:36.410 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:36.407+0000 7f5533e00700 1 -- 172.21.15.153:0/3870046772 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f552800f460 con 0x7f553c1039b0 2024-03-21T10:22:36.410 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:36.407+0000 7f5542600700 1 -- 172.21.15.153:0/3870046772 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f553c198ac0 con 0x7f553c1039b0 2024-03-21T10:22:36.410 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:36.407+0000 7f5533e00700 1 -- 172.21.15.153:0/3870046772 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5528017610 con 0x7f553c1039b0 2024-03-21T10:22:36.410 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:36.407+0000 7f5542600700 1 -- 172.21.15.153:0/3870046772 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f553c198fb0 con 0x7f553c1039b0 2024-03-21T10:22:36.410 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:36.408+0000 7f5542600700 1 -- 172.21.15.153:0/3870046772 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f553c10b400 con 0x7f553c1039b0 2024-03-21T10:22:36.411 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:36.409+0000 7f5533e00700 1 -- 172.21.15.153:0/3870046772 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f552800fa40 con 0x7f553c1039b0 2024-03-21T10:22:36.412 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:36.410+0000 7f5533e00700 1 --2- 172.21.15.153:0/3870046772 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f552006e460 0x7f5520070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:36.412 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:36.410+0000 7f5533e00700 1 -- 172.21.15.153:0/3870046772 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f552808f920 con 0x7f553c1039b0 2024-03-21T10:22:36.412 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:36.410+0000 7f553b400700 1 --2- 172.21.15.153:0/3870046772 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f552006e460 0x7f5520070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:36.412 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:36.411+0000 7f553b400700 1 --2- 172.21.15.153:0/3870046772 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f552006e460 0x7f5520070920 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f552c005fd0 tx=0x7f552c005ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:36.417 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:36.415+0000 7f5533e00700 1 -- 172.21.15.153:0/3870046772 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f552805c4f0 con 0x7f553c1039b0 2024-03-21T10:22:36.720 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:36.718+0000 7f5542600700 1 -- 172.21.15.153:0/3870046772 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f553c04ea90 con 0x7f553c1039b0 2024-03-21T10:22:36.721 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:36.720+0000 7f5533e00700 1 -- 172.21.15.153:0/3870046772 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+805 (secure 0 0 0) 0x7f552805c080 con 0x7f553c1039b0 2024-03-21T10:22:36.722 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:22:36.722 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": { 2024-03-21T10:22:36.722 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:22:36.722 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:22:36.722 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": { 2024-03-21T10:22:36.722 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:22:36.722 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:22:36.722 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": { 2024-03-21T10:22:36.722 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 6 2024-03-21T10:22:36.722 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:22:36.723 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": { 2024-03-21T10:22:36.723 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-03-21T10:22:36.723 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 3 2024-03-21T10:22:36.723 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:22:36.723 INFO:teuthology.orchestra.run.smithi153.stdout: "overall": { 2024-03-21T10:22:36.723 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-03-21T10:22:36.723 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 13 2024-03-21T10:22:36.723 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-03-21T10:22:36.723 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:22:36.725 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:36.722+0000 7f5542600700 1 -- 172.21.15.153:0/3870046772 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f552006e460 msgr2=0x7f5520070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:36.725 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:36.722+0000 7f5542600700 1 --2- 172.21.15.153:0/3870046772 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f552006e460 0x7f5520070920 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f552c005fd0 tx=0x7f552c005ee0 comp rx=0 tx=0).stop 2024-03-21T10:22:36.725 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:36.723+0000 7f5542600700 1 -- 172.21.15.153:0/3870046772 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f553c1039b0 msgr2=0x7f553c19ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:36.725 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:36.723+0000 7f5542600700 1 --2- 172.21.15.153:0/3870046772 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f553c1039b0 0x7f553c19ed00 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f552800bda0 tx=0x7f552800be80 comp rx=0 tx=0).stop 2024-03-21T10:22:36.725 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:36.723+0000 7f5542600700 1 -- 172.21.15.153:0/3870046772 shutdown_connections 2024-03-21T10:22:36.725 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:36.723+0000 7f5542600700 1 --2- 172.21.15.153:0/3870046772 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f553c103000 0x7f553c19e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:36.725 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:36.723+0000 7f5542600700 1 --2- 172.21.15.153:0/3870046772 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f552006e460 0x7f5520070920 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:36.725 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:36.723+0000 7f5542600700 1 --2- 172.21.15.153:0/3870046772 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f553c1039b0 0x7f553c19ed00 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:36.725 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:36.723+0000 7f5542600700 1 -- 172.21.15.153:0/3870046772 >> 172.21.15.153:0/3870046772 conn(0x7f553c0fe8d0 msgr2=0x7f553c100cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:36.725 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:36.723+0000 7f5542600700 1 -- 172.21.15.153:0/3870046772 shutdown_connections 2024-03-21T10:22:36.725 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:36.723+0000 7f5542600700 1 -- 172.21.15.153:0/3870046772 wait complete. 2024-03-21T10:22:36.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:22:36 smithi153 ceph-mon[113733]: pgmap v322: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:22:36.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:22:36 smithi153 ceph-mon[113733]: from='client.34618 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:22:36.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:22:36 smithi153 ceph-mon[113733]: from='client.34622 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:22:36.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:22:36 smithi153 ceph-mon[113733]: from='client.34626 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:22:36.867 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:36.865+0000 7f1f0dc00700 1 -- 172.21.15.153:0/600495245 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f0810d280 msgr2=0x7f1f0810d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:36.868 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:36.865+0000 7f1f0dc00700 1 --2- 172.21.15.153:0/600495245 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f0810d280 0x7f1f0810d660 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f1ef8009910 tx=0x7f1ef8009c20 comp rx=0 tx=0).stop 2024-03-21T10:22:36.868 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:36.865+0000 7f1f0dc00700 1 -- 172.21.15.153:0/600495245 shutdown_connections 2024-03-21T10:22:36.868 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:36.865+0000 7f1f0dc00700 1 --2- 172.21.15.153:0/600495245 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1f081071d0 0x7f1f08107650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:36.868 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:36.865+0000 7f1f0dc00700 1 --2- 172.21.15.153:0/600495245 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f0810d280 0x7f1f0810d660 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:36.868 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:36.865+0000 7f1f0dc00700 1 -- 172.21.15.153:0/600495245 >> 172.21.15.153:0/600495245 conn(0x7f1f080769e0 msgr2=0x7f1f08078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:36.868 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:36.866+0000 7f1f0dc00700 1 -- 172.21.15.153:0/600495245 shutdown_connections 2024-03-21T10:22:36.868 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:36.866+0000 7f1f0dc00700 1 -- 172.21.15.153:0/600495245 wait complete. 2024-03-21T10:22:36.869 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:36.867+0000 7f1f0dc00700 1 Processor -- start 2024-03-21T10:22:36.869 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:36.867+0000 7f1f0dc00700 1 -- start start 2024-03-21T10:22:36.869 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:36.867+0000 7f1f0dc00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f081071d0 0x7f1f0819ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:36.869 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:36.867+0000 7f1f0dc00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1f0810d280 0x7f1f0819d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:36.869 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:36.867+0000 7f1f0dc00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0819da30 con 0x7f1f081071d0 2024-03-21T10:22:36.869 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:36.867+0000 7f1f0dc00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f081a1770 con 0x7f1f0810d280 2024-03-21T10:22:36.869 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:36.867+0000 7f1f06a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f081071d0 0x7f1f0819ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:36.870 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:36.868+0000 7f1f06a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f081071d0 0x7f1f0819ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:48256/0 (socket says 172.21.15.153:48256) 2024-03-21T10:22:36.870 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:36.868+0000 7f1f06a00700 1 -- 172.21.15.153:0/1210321299 learned_addr learned my addr 172.21.15.153:0/1210321299 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:22:36.870 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:36.868+0000 7f1f06a00700 1 -- 172.21.15.153:0/1210321299 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1f0810d280 msgr2=0x7f1f0819d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:36.871 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:36.868+0000 7f1f06a00700 1 --2- 172.21.15.153:0/1210321299 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1f0810d280 0x7f1f0819d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:36.871 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:36.868+0000 7f1f06a00700 1 -- 172.21.15.153:0/1210321299 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ef80095f0 con 0x7f1f081071d0 2024-03-21T10:22:36.871 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:36.868+0000 7f1f06a00700 1 --2- 172.21.15.153:0/1210321299 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f081071d0 0x7f1f0819ce10 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f1ef8006010 tx=0x7f1ef800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:36.871 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:36.868+0000 7f1eff400700 1 -- 172.21.15.153:0/1210321299 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1ef801d070 con 0x7f1f081071d0 2024-03-21T10:22:36.871 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:36.868+0000 7f1eff400700 1 -- 172.21.15.153:0/1210321299 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1ef8004fb0 con 0x7f1f081071d0 2024-03-21T10:22:36.871 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:36.868+0000 7f1f0dc00700 1 -- 172.21.15.153:0/1210321299 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f081a19f0 con 0x7f1f081071d0 2024-03-21T10:22:36.871 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:36.869+0000 7f1eff400700 1 -- 172.21.15.153:0/1210321299 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1ef8022ea0 con 0x7f1f081071d0 2024-03-21T10:22:36.871 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:36.869+0000 7f1f0dc00700 1 -- 172.21.15.153:0/1210321299 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f081a1f60 con 0x7f1f081071d0 2024-03-21T10:22:36.872 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:36.870+0000 7f1f0dc00700 1 -- 172.21.15.153:0/1210321299 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f0810f6f0 con 0x7f1f081071d0 2024-03-21T10:22:36.872 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:36.871+0000 7f1eff400700 1 -- 172.21.15.153:0/1210321299 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f1ef8022470 con 0x7f1f081071d0 2024-03-21T10:22:36.873 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:36.871+0000 7f1eff400700 1 --2- 172.21.15.153:0/1210321299 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f1eec06e460 0x7f1eec070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:36.873 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:36.872+0000 7f1eff400700 1 -- 172.21.15.153:0/1210321299 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f1ef808fa80 con 0x7f1f081071d0 2024-03-21T10:22:36.873 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:36.872+0000 7f1f06000700 1 --2- 172.21.15.153:0/1210321299 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f1eec06e460 0x7f1eec070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:36.874 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:36.872+0000 7f1f06000700 1 --2- 172.21.15.153:0/1210321299 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f1eec06e460 0x7f1eec070920 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1f0819e3e0 tx=0x7f1ef4006d20 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:36.878 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:36.877+0000 7f1eff400700 1 -- 172.21.15.153:0/1210321299 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f1ef8058bd0 con 0x7f1f081071d0 2024-03-21T10:22:36.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:22:36 smithi174 ceph-mon[88004]: pgmap v322: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:22:36.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:22:36 smithi174 ceph-mon[88004]: from='client.34618 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:22:36.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:22:36 smithi174 ceph-mon[88004]: from='client.34622 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:22:36.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:22:36 smithi174 ceph-mon[88004]: from='client.34626 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:22:37.128 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:37.126+0000 7f1f0dc00700 1 -- 172.21.15.153:0/1210321299 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f1f081a22c0 con 0x7f1f081071d0 2024-03-21T10:22:37.128 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:37.127+0000 7f1eff400700 1 -- 172.21.15.153:0/1210321299 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1876 (secure 0 0 0) 0x7f1ef8027770 con 0x7f1f081071d0 2024-03-21T10:22:37.130 INFO:teuthology.orchestra.run.smithi153.stdout:e30 2024-03-21T10:22:37.130 INFO:teuthology.orchestra.run.smithi153.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-21T10:22:37.130 INFO:teuthology.orchestra.run.smithi153.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:22:37.131 INFO:teuthology.orchestra.run.smithi153.stdout:legacy client fscid: 1 2024-03-21T10:22:37.131 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:22:37.131 INFO:teuthology.orchestra.run.smithi153.stdout:Filesystem 'cephfs' (1) 2024-03-21T10:22:37.131 INFO:teuthology.orchestra.run.smithi153.stdout:fs_name cephfs 2024-03-21T10:22:37.131 INFO:teuthology.orchestra.run.smithi153.stdout:epoch 23 2024-03-21T10:22:37.131 INFO:teuthology.orchestra.run.smithi153.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-21T10:22:37.131 INFO:teuthology.orchestra.run.smithi153.stdout:created 2024-03-21T10:04:27.086576+0000 2024-03-21T10:22:37.131 INFO:teuthology.orchestra.run.smithi153.stdout:modified 2024-03-21T10:19:05.022092+0000 2024-03-21T10:22:37.131 INFO:teuthology.orchestra.run.smithi153.stdout:tableserver 0 2024-03-21T10:22:37.131 INFO:teuthology.orchestra.run.smithi153.stdout:root 0 2024-03-21T10:22:37.131 INFO:teuthology.orchestra.run.smithi153.stdout:session_timeout 60 2024-03-21T10:22:37.131 INFO:teuthology.orchestra.run.smithi153.stdout:session_autoclose 300 2024-03-21T10:22:37.131 INFO:teuthology.orchestra.run.smithi153.stdout:max_file_size 1099511627776 2024-03-21T10:22:37.131 INFO:teuthology.orchestra.run.smithi153.stdout:max_xattr_size 65536 2024-03-21T10:22:37.131 INFO:teuthology.orchestra.run.smithi153.stdout:required_client_features {} 2024-03-21T10:22:37.132 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure 0 2024-03-21T10:22:37.132 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure_osd_epoch 94 2024-03-21T10:22:37.132 INFO:teuthology.orchestra.run.smithi153.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:22:37.132 INFO:teuthology.orchestra.run.smithi153.stdout:max_mds 2 2024-03-21T10:22:37.132 INFO:teuthology.orchestra.run.smithi153.stdout:in 0,1 2024-03-21T10:22:37.132 INFO:teuthology.orchestra.run.smithi153.stdout:up {} 2024-03-21T10:22:37.132 INFO:teuthology.orchestra.run.smithi153.stdout:failed 0,1 2024-03-21T10:22:37.132 INFO:teuthology.orchestra.run.smithi153.stdout:damaged 2024-03-21T10:22:37.132 INFO:teuthology.orchestra.run.smithi153.stdout:stopped 2024-03-21T10:22:37.132 INFO:teuthology.orchestra.run.smithi153.stdout:data_pools [3] 2024-03-21T10:22:37.132 INFO:teuthology.orchestra.run.smithi153.stdout:metadata_pool 2 2024-03-21T10:22:37.132 INFO:teuthology.orchestra.run.smithi153.stdout:inline_data enabled 2024-03-21T10:22:37.132 INFO:teuthology.orchestra.run.smithi153.stdout:balancer 2024-03-21T10:22:37.132 INFO:teuthology.orchestra.run.smithi153.stdout:bal_rank_mask -1 2024-03-21T10:22:37.132 INFO:teuthology.orchestra.run.smithi153.stdout:standby_count_wanted 1 2024-03-21T10:22:37.133 INFO:teuthology.orchestra.run.smithi153.stdout:qdb_cluster leader: 0 members: 2024-03-21T10:22:37.133 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:22:37.133 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:22:37.133 INFO:teuthology.orchestra.run.smithi153.stdout:Standby daemons: 2024-03-21T10:22:37.133 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:22:37.133 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.zipckr{-1:34442} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6826/1103846295,v1:172.21.15.153:6827/1103846295] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:22:37.133 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.iuyluj{-1:34474} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6828/1738630832,v1:172.21.15.153:6829/1738630832] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:22:37.133 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.nhozag{-1:44269} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.174:6826/3967223227,v1:172.21.15.174:6827/3967223227] compat {c=[1],r=[1],i=[7ff]}] 2024-03-21T10:22:37.133 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.yyimst{-1:44313} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.174:6824/4127786797,v1:172.21.15.174:6825/4127786797] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:22:37.134 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:37.129+0000 7f1f0dc00700 1 -- 172.21.15.153:0/1210321299 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f1eec06e460 msgr2=0x7f1eec070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:37.134 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:37.129+0000 7f1f0dc00700 1 --2- 172.21.15.153:0/1210321299 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f1eec06e460 0x7f1eec070920 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1f0819e3e0 tx=0x7f1ef4006d20 comp rx=0 tx=0).stop 2024-03-21T10:22:37.134 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:37.129+0000 7f1f0dc00700 1 -- 172.21.15.153:0/1210321299 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f081071d0 msgr2=0x7f1f0819ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:37.134 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:37.129+0000 7f1f0dc00700 1 --2- 172.21.15.153:0/1210321299 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f081071d0 0x7f1f0819ce10 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f1ef8006010 tx=0x7f1ef800be20 comp rx=0 tx=0).stop 2024-03-21T10:22:37.134 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:37.130+0000 7f1f0dc00700 1 -- 172.21.15.153:0/1210321299 shutdown_connections 2024-03-21T10:22:37.134 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:37.130+0000 7f1f0dc00700 1 --2- 172.21.15.153:0/1210321299 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f1eec06e460 0x7f1eec070920 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:37.134 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:37.130+0000 7f1f0dc00700 1 --2- 172.21.15.153:0/1210321299 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f081071d0 0x7f1f0819ce10 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:37.135 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:37.130+0000 7f1f0dc00700 1 --2- 172.21.15.153:0/1210321299 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1f0810d280 0x7f1f0819d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:37.135 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:37.130+0000 7f1f0dc00700 1 -- 172.21.15.153:0/1210321299 >> 172.21.15.153:0/1210321299 conn(0x7f1f080769e0 msgr2=0x7f1f080785c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:37.135 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:37.130+0000 7f1f0dc00700 1 -- 172.21.15.153:0/1210321299 shutdown_connections 2024-03-21T10:22:37.135 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:37.130+0000 7f1f0dc00700 1 -- 172.21.15.153:0/1210321299 wait complete. 2024-03-21T10:22:37.135 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 30 2024-03-21T10:22:37.290 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:37.288+0000 7fc94c800700 1 -- 172.21.15.153:0/3966273268 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc944068670 msgr2=0x7fc944068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:37.291 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:37.288+0000 7fc94c800700 1 --2- 172.21.15.153:0/3966273268 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc944068670 0x7fc944068a50 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fc938009910 tx=0x7fc938009c20 comp rx=0 tx=0).stop 2024-03-21T10:22:37.291 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:37.288+0000 7fc94c800700 1 -- 172.21.15.153:0/3966273268 shutdown_connections 2024-03-21T10:22:37.291 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:37.288+0000 7fc94c800700 1 --2- 172.21.15.153:0/3966273268 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc944069020 0x7fc944105790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:37.291 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:37.288+0000 7fc94c800700 1 --2- 172.21.15.153:0/3966273268 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc944068670 0x7fc944068a50 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:37.291 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:37.288+0000 7fc94c800700 1 -- 172.21.15.153:0/3966273268 >> 172.21.15.153:0/3966273268 conn(0x7fc9440755a0 msgr2=0x7fc9440759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:37.291 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:37.288+0000 7fc94c800700 1 -- 172.21.15.153:0/3966273268 shutdown_connections 2024-03-21T10:22:37.291 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:37.288+0000 7fc94c800700 1 -- 172.21.15.153:0/3966273268 wait complete. 2024-03-21T10:22:37.292 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:37.289+0000 7fc94c800700 1 Processor -- start 2024-03-21T10:22:37.292 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:37.290+0000 7fc94c800700 1 -- start start 2024-03-21T10:22:37.293 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:37.290+0000 7fc94c800700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc944068670 0x7fc94419e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:37.293 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:37.290+0000 7fc94c800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc944069020 0x7fc94419ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:37.293 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:37.290+0000 7fc94c800700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc94419f2e0 con 0x7fc944069020 2024-03-21T10:22:37.293 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:37.290+0000 7fc94c800700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc944198790 con 0x7fc944068670 2024-03-21T10:22:37.293 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:37.290+0000 7fc949400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc944069020 0x7fc94419ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:37.293 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:37.290+0000 7fc949400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc944069020 0x7fc94419ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:48280/0 (socket says 172.21.15.153:48280) 2024-03-21T10:22:37.293 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:37.290+0000 7fc949400700 1 -- 172.21.15.153:0/674033723 learned_addr learned my addr 172.21.15.153:0/674033723 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:22:37.293 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:37.291+0000 7fc949e00700 1 --2- 172.21.15.153:0/674033723 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc944068670 0x7fc94419e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:37.293 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:37.291+0000 7fc949400700 1 -- 172.21.15.153:0/674033723 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc944068670 msgr2=0x7fc94419e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:37.293 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:37.291+0000 7fc949400700 1 --2- 172.21.15.153:0/674033723 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc944068670 0x7fc94419e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:37.293 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:37.291+0000 7fc949400700 1 -- 172.21.15.153:0/674033723 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9380095f0 con 0x7fc944069020 2024-03-21T10:22:37.294 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:37.291+0000 7fc949400700 1 --2- 172.21.15.153:0/674033723 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc944069020 0x7fc94419ec50 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc93400c8f0 tx=0x7fc93400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:37.294 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:37.291+0000 7fc93ea00700 1 -- 172.21.15.153:0/674033723 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc934007a50 con 0x7fc944069020 2024-03-21T10:22:37.295 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:37.292+0000 7fc94c800700 1 -- 172.21.15.153:0/674033723 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc944198a70 con 0x7fc944069020 2024-03-21T10:22:37.295 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:37.292+0000 7fc93ea00700 1 -- 172.21.15.153:0/674033723 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc934007bb0 con 0x7fc944069020 2024-03-21T10:22:37.295 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:37.292+0000 7fc93ea00700 1 -- 172.21.15.153:0/674033723 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc934018630 con 0x7fc944069020 2024-03-21T10:22:37.295 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:37.292+0000 7fc94c800700 1 -- 172.21.15.153:0/674033723 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc944198fc0 con 0x7fc944069020 2024-03-21T10:22:37.295 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:37.293+0000 7fc94c800700 1 -- 172.21.15.153:0/674033723 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc94404ea90 con 0x7fc944069020 2024-03-21T10:22:37.295 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:37.294+0000 7fc93ea00700 1 -- 172.21.15.153:0/674033723 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fc934007d20 con 0x7fc944069020 2024-03-21T10:22:37.297 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:37.295+0000 7fc93ea00700 1 --2- 172.21.15.153:0/674033723 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fc92c06e460 0x7fc92c070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:37.297 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:37.295+0000 7fc93ea00700 1 -- 172.21.15.153:0/674033723 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fc93408e8c0 con 0x7fc944069020 2024-03-21T10:22:37.297 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:37.295+0000 7fc949e00700 1 --2- 172.21.15.153:0/674033723 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fc92c06e460 0x7fc92c070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:37.297 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:37.296+0000 7fc949e00700 1 --2- 172.21.15.153:0/674033723 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fc92c06e460 0x7fc92c070920 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc938006010 tx=0x7fc93801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:37.300 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:37.298+0000 7fc93ea00700 1 -- 172.21.15.153:0/674033723 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc93405b510 con 0x7fc944069020 2024-03-21T10:22:37.547 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:37.545+0000 7fc94c800700 1 -- 172.21.15.153:0/674033723 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc9441994c0 con 0x7fc92c06e460 2024-03-21T10:22:37.549 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:37.547+0000 7fc93ea00700 1 -- 172.21.15.153:0/674033723 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fc9441994c0 con 0x7fc92c06e460 2024-03-21T10:22:37.549 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:22:37.549 INFO:teuthology.orchestra.run.smithi153.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232", 2024-03-21T10:22:37.549 INFO:teuthology.orchestra.run.smithi153.stdout: "in_progress": true, 2024-03-21T10:22:37.549 INFO:teuthology.orchestra.run.smithi153.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-21T10:22:37.549 INFO:teuthology.orchestra.run.smithi153.stdout: "services_complete": [ 2024-03-21T10:22:37.549 INFO:teuthology.orchestra.run.smithi153.stdout: "crash", 2024-03-21T10:22:37.549 INFO:teuthology.orchestra.run.smithi153.stdout: "mon", 2024-03-21T10:22:37.549 INFO:teuthology.orchestra.run.smithi153.stdout: "osd", 2024-03-21T10:22:37.549 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr" 2024-03-21T10:22:37.550 INFO:teuthology.orchestra.run.smithi153.stdout: ], 2024-03-21T10:22:37.550 INFO:teuthology.orchestra.run.smithi153.stdout: "progress": "15/23 daemons upgraded", 2024-03-21T10:22:37.550 INFO:teuthology.orchestra.run.smithi153.stdout: "message": "Currently upgrading mds daemons", 2024-03-21T10:22:37.550 INFO:teuthology.orchestra.run.smithi153.stdout: "is_paused": false 2024-03-21T10:22:37.550 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:22:37.551 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:37.550+0000 7fc94c800700 1 -- 172.21.15.153:0/674033723 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fc92c06e460 msgr2=0x7fc92c070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:37.551 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:37.550+0000 7fc94c800700 1 --2- 172.21.15.153:0/674033723 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fc92c06e460 0x7fc92c070920 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc938006010 tx=0x7fc93801a040 comp rx=0 tx=0).stop 2024-03-21T10:22:37.552 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:37.550+0000 7fc94c800700 1 -- 172.21.15.153:0/674033723 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc944069020 msgr2=0x7fc94419ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:37.552 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:37.550+0000 7fc94c800700 1 --2- 172.21.15.153:0/674033723 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc944069020 0x7fc94419ec50 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc93400c8f0 tx=0x7fc93400ccb0 comp rx=0 tx=0).stop 2024-03-21T10:22:37.552 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:37.550+0000 7fc94c800700 1 -- 172.21.15.153:0/674033723 shutdown_connections 2024-03-21T10:22:37.552 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:37.550+0000 7fc94c800700 1 --2- 172.21.15.153:0/674033723 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc944068670 0x7fc94419e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:37.552 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:37.550+0000 7fc94c800700 1 --2- 172.21.15.153:0/674033723 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fc92c06e460 0x7fc92c070920 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:37.553 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:37.550+0000 7fc94c800700 1 --2- 172.21.15.153:0/674033723 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc944069020 0x7fc94419ec50 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:37.553 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:37.550+0000 7fc94c800700 1 -- 172.21.15.153:0/674033723 >> 172.21.15.153:0/674033723 conn(0x7fc9440755a0 msgr2=0x7fc944102760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:37.553 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:37.550+0000 7fc94c800700 1 -- 172.21.15.153:0/674033723 shutdown_connections 2024-03-21T10:22:37.553 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:37.550+0000 7fc94c800700 1 -- 172.21.15.153:0/674033723 wait complete. 2024-03-21T10:22:37.706 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:37.704+0000 7fcf0ba00700 1 -- 172.21.15.153:0/3012549306 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcf04068670 msgr2=0x7fcf04068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:37.706 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:37.704+0000 7fcf0ba00700 1 --2- 172.21.15.153:0/3012549306 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcf04068670 0x7fcf04068a50 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fcef8009910 tx=0x7fcef8009c20 comp rx=0 tx=0).stop 2024-03-21T10:22:37.707 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:37.704+0000 7fcf0ba00700 1 -- 172.21.15.153:0/3012549306 shutdown_connections 2024-03-21T10:22:37.707 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:37.704+0000 7fcf0ba00700 1 --2- 172.21.15.153:0/3012549306 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcf04069020 0x7fcf04105790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:37.707 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:37.704+0000 7fcf0ba00700 1 --2- 172.21.15.153:0/3012549306 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcf04068670 0x7fcf04068a50 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:37.707 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:37.704+0000 7fcf0ba00700 1 -- 172.21.15.153:0/3012549306 >> 172.21.15.153:0/3012549306 conn(0x7fcf040755a0 msgr2=0x7fcf040759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:37.707 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:37.704+0000 7fcf0ba00700 1 -- 172.21.15.153:0/3012549306 shutdown_connections 2024-03-21T10:22:37.707 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:37.704+0000 7fcf0ba00700 1 -- 172.21.15.153:0/3012549306 wait complete. 2024-03-21T10:22:37.708 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:37.705+0000 7fcf0ba00700 1 Processor -- start 2024-03-21T10:22:37.708 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:37.705+0000 7fcf0ba00700 1 -- start start 2024-03-21T10:22:37.708 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:37.706+0000 7fcf0ba00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcf04068670 0x7fcf0419c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:37.708 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:37.706+0000 7fcf0ba00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcf04069020 0x7fcf0419ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:37.708 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:37.706+0000 7fcf0ba00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf0419d0e0 con 0x7fcf04069020 2024-03-21T10:22:37.708 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:37.706+0000 7fcf0ba00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf04196590 con 0x7fcf04068670 2024-03-21T10:22:37.708 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:37.706+0000 7fcf03e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcf04069020 0x7fcf0419ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:37.709 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:37.706+0000 7fcf03e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcf04069020 0x7fcf0419ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:48308/0 (socket says 172.21.15.153:48308) 2024-03-21T10:22:37.709 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:37.706+0000 7fcf03e00700 1 -- 172.21.15.153:0/1501770741 learned_addr learned my addr 172.21.15.153:0/1501770741 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:22:37.709 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:37.706+0000 7fcf09000700 1 --2- 172.21.15.153:0/1501770741 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcf04068670 0x7fcf0419c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:37.709 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:37.707+0000 7fcf03e00700 1 -- 172.21.15.153:0/1501770741 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcf04068670 msgr2=0x7fcf0419c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:37.709 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:37.707+0000 7fcf03e00700 1 --2- 172.21.15.153:0/1501770741 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcf04068670 0x7fcf0419c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:37.709 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:37.707+0000 7fcf03e00700 1 -- 172.21.15.153:0/1501770741 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcef80095f0 con 0x7fcf04069020 2024-03-21T10:22:37.710 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:37.707+0000 7fcf03e00700 1 --2- 172.21.15.153:0/1501770741 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcf04069020 0x7fcf0419ca50 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fcef000d900 tx=0x7fcef000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:37.710 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:37.707+0000 7fcf01600700 1 -- 172.21.15.153:0/1501770741 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcef0009900 con 0x7fcf04069020 2024-03-21T10:22:37.710 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:37.707+0000 7fcf01600700 1 -- 172.21.15.153:0/1501770741 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcef0010460 con 0x7fcf04069020 2024-03-21T10:22:37.710 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:37.707+0000 7fcf0ba00700 1 -- 172.21.15.153:0/1501770741 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf04196870 con 0x7fcf04069020 2024-03-21T10:22:37.710 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:37.707+0000 7fcf01600700 1 -- 172.21.15.153:0/1501770741 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcef000f5f0 con 0x7fcf04069020 2024-03-21T10:22:37.710 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:37.707+0000 7fcf0ba00700 1 -- 172.21.15.153:0/1501770741 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf04196dc0 con 0x7fcf04069020 2024-03-21T10:22:37.710 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:37.708+0000 7fcf0ba00700 1 -- 172.21.15.153:0/1501770741 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf04109130 con 0x7fcf04069020 2024-03-21T10:22:37.711 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:37.710+0000 7fcf01600700 1 -- 172.21.15.153:0/1501770741 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fcef00105d0 con 0x7fcf04069020 2024-03-21T10:22:37.712 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:37.710+0000 7fcf01600700 1 --2- 172.21.15.153:0/1501770741 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fceec06e460 0x7fceec070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:22:37.712 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:37.711+0000 7fcf01600700 1 -- 172.21.15.153:0/1501770741 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fcef008e3d0 con 0x7fcf04069020 2024-03-21T10:22:37.712 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:37.711+0000 7fcf09000700 1 --2- 172.21.15.153:0/1501770741 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fceec06e460 0x7fceec070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:22:37.713 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:37.711+0000 7fcf09000700 1 --2- 172.21.15.153:0/1501770741 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fceec06e460 0x7fceec070920 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fcef8006010 tx=0x7fcef801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:22:37.717 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:37.715+0000 7fcf01600700 1 -- 172.21.15.153:0/1501770741 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fcef005af20 con 0x7fcf04069020 2024-03-21T10:22:37.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:22:37 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/3870046772' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:22:37.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:22:37 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/1210321299' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:22:37.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:22:37 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/3870046772' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:22:37.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:22:37 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/1210321299' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:22:38.010 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:38.008+0000 7fcf0ba00700 1 -- 172.21.15.153:0/1501770741 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fcf0404ea90 con 0x7fcf04069020 2024-03-21T10:22:38.011 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:38.009+0000 7fcf01600700 1 -- 172.21.15.153:0/1501770741 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7fcef005aab0 con 0x7fcf04069020 2024-03-21T10:22:38.011 INFO:teuthology.orchestra.run.smithi153.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-03-21T10:22:38.011 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-21T10:22:38.011 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is degraded 2024-03-21T10:22:38.011 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-21T10:22:38.011 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-21T10:22:38.011 INFO:teuthology.orchestra.run.smithi153.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-21T10:22:38.011 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-21T10:22:38.013 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:38.011+0000 7fcf0ba00700 1 -- 172.21.15.153:0/1501770741 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fceec06e460 msgr2=0x7fceec070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:38.013 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:38.011+0000 7fcf0ba00700 1 --2- 172.21.15.153:0/1501770741 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fceec06e460 0x7fceec070920 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fcef8006010 tx=0x7fcef801a040 comp rx=0 tx=0).stop 2024-03-21T10:22:38.013 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:38.011+0000 7fcf0ba00700 1 -- 172.21.15.153:0/1501770741 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcf04069020 msgr2=0x7fcf0419ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:22:38.013 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:38.011+0000 7fcf0ba00700 1 --2- 172.21.15.153:0/1501770741 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcf04069020 0x7fcf0419ca50 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fcef000d900 tx=0x7fcef000dcc0 comp rx=0 tx=0).stop 2024-03-21T10:22:38.014 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:38.012+0000 7fcf0ba00700 1 -- 172.21.15.153:0/1501770741 shutdown_connections 2024-03-21T10:22:38.014 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:38.012+0000 7fcf0ba00700 1 --2- 172.21.15.153:0/1501770741 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcf04068670 0x7fcf0419c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:38.014 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:38.012+0000 7fcf0ba00700 1 --2- 172.21.15.153:0/1501770741 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fceec06e460 0x7fceec070920 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:38.014 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:38.012+0000 7fcf0ba00700 1 --2- 172.21.15.153:0/1501770741 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcf04069020 0x7fcf0419ca50 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:22:38.014 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:38.012+0000 7fcf0ba00700 1 -- 172.21.15.153:0/1501770741 >> 172.21.15.153:0/1501770741 conn(0x7fcf040755a0 msgr2=0x7fcf040fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:22:38.015 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:38.012+0000 7fcf0ba00700 1 -- 172.21.15.153:0/1501770741 shutdown_connections 2024-03-21T10:22:38.015 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:22:38.012+0000 7fcf0ba00700 1 -- 172.21.15.153:0/1501770741 wait complete. 2024-03-21T10:22:38.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:22:38 smithi153 ceph-mon[113733]: pgmap v323: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:22:38.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:22:38 smithi153 ceph-mon[113733]: from='client.34638 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:22:38.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:22:38 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/1501770741' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:22:38.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:22:38 smithi174 ceph-mon[88004]: pgmap v323: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:22:38.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:22:38 smithi174 ceph-mon[88004]: from='client.34638 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:22:38.952 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:22:38 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/1501770741' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:22:40.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:22:40 smithi153 ceph-mon[113733]: pgmap v324: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:22:40.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:22:40 smithi174 ceph-mon[88004]: pgmap v324: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:22:42.650 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:22:42 smithi153 ceph-mon[113733]: pgmap v325: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:22:42.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:22:42 smithi174 ceph-mon[88004]: pgmap v325: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:22:44.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:22:44 smithi153 ceph-mon[113733]: pgmap v326: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:22:44.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:22:44 smithi174 ceph-mon[88004]: pgmap v326: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:22:47.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:22:46 smithi174 ceph-mon[88004]: pgmap v327: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:22:47.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:22:46 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:22:47.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:22:46 smithi153 ceph-mon[113733]: pgmap v327: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:22:47.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:22:46 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:22:49.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:22:48 smithi174 ceph-mon[88004]: pgmap v328: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:22:49.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:22:48 smithi153 ceph-mon[113733]: pgmap v328: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:22:51.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:22:50 smithi174 ceph-mon[88004]: pgmap v329: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:22:51.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:22:50 smithi153 ceph-mon[113733]: pgmap v329: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:22:53.062 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:22:52 smithi153 ceph-mon[113733]: pgmap v330: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:22:53.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:22:52 smithi174 ceph-mon[88004]: pgmap v330: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:22:55.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:22:54 smithi174 ceph-mon[88004]: pgmap v331: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:22:55.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:22:54 smithi153 ceph-mon[113733]: pgmap v331: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:22:57.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:22:56 smithi153 ceph-mon[113733]: pgmap v332: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:22:57.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:22:56 smithi174 ceph-mon[88004]: pgmap v332: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:22:59.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:22:58 smithi174 ceph-mon[88004]: pgmap v333: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:22:59.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:22:58 smithi153 ceph-mon[113733]: pgmap v333: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:23:01.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:23:00 smithi174 ceph-mon[88004]: pgmap v334: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:23:01.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:23:00 smithi153 ceph-mon[113733]: pgmap v334: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:23:02.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:23:01 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:23:02.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:23:01 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:23:03.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:23:02 smithi153 ceph-mon[113733]: pgmap v335: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:23:03.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:23:02 smithi174 ceph-mon[88004]: pgmap v335: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:23:05.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:23:04 smithi174 ceph-mon[88004]: pgmap v336: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:23:05.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:23:04 smithi153 ceph-mon[113733]: pgmap v336: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:23:07.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:23:06 smithi174 ceph-mon[88004]: pgmap v337: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:23:07.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:23:06 smithi153 ceph-mon[113733]: pgmap v337: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:23:08.187 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:08.184+0000 7fa52b000700 1 -- 172.21.15.153:0/457114947 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa524068670 msgr2=0x7fa524068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:08.187 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:08.184+0000 7fa52b000700 1 --2- 172.21.15.153:0/457114947 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa524068670 0x7fa524068a50 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fa514009910 tx=0x7fa514009c20 comp rx=0 tx=0).stop 2024-03-21T10:23:08.187 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:08.184+0000 7fa52b000700 1 -- 172.21.15.153:0/457114947 shutdown_connections 2024-03-21T10:23:08.187 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:08.184+0000 7fa52b000700 1 --2- 172.21.15.153:0/457114947 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa524069020 0x7fa524105790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:08.187 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:08.184+0000 7fa52b000700 1 --2- 172.21.15.153:0/457114947 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa524068670 0x7fa524068a50 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:08.187 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:08.184+0000 7fa52b000700 1 -- 172.21.15.153:0/457114947 >> 172.21.15.153:0/457114947 conn(0x7fa5240755a0 msgr2=0x7fa5240759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:23:08.187 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:08.185+0000 7fa52b000700 1 -- 172.21.15.153:0/457114947 shutdown_connections 2024-03-21T10:23:08.187 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:08.185+0000 7fa52b000700 1 -- 172.21.15.153:0/457114947 wait complete. 2024-03-21T10:23:08.188 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:08.186+0000 7fa52b000700 1 Processor -- start 2024-03-21T10:23:08.188 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:08.186+0000 7fa52b000700 1 -- start start 2024-03-21T10:23:08.188 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:08.186+0000 7fa52b000700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa524068670 0x7fa52419e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:08.189 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:08.186+0000 7fa52b000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa524069020 0x7fa52419ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:08.189 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:08.186+0000 7fa52b000700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa52419f2e0 con 0x7fa524069020 2024-03-21T10:23:08.189 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:08.186+0000 7fa52b000700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa524198790 con 0x7fa524068670 2024-03-21T10:23:08.189 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:08.187+0000 7fa523400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa524069020 0x7fa52419ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:08.189 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:08.187+0000 7fa523400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa524069020 0x7fa52419ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:37742/0 (socket says 172.21.15.153:37742) 2024-03-21T10:23:08.189 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:08.187+0000 7fa523400700 1 -- 172.21.15.153:0/73570276 learned_addr learned my addr 172.21.15.153:0/73570276 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:23:08.189 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:08.187+0000 7fa523e00700 1 --2- 172.21.15.153:0/73570276 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa524068670 0x7fa52419e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:08.190 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:08.187+0000 7fa523400700 1 -- 172.21.15.153:0/73570276 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa524068670 msgr2=0x7fa52419e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:08.190 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:08.187+0000 7fa523400700 1 --2- 172.21.15.153:0/73570276 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa524068670 0x7fa52419e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:08.190 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:08.187+0000 7fa523400700 1 -- 172.21.15.153:0/73570276 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5140095f0 con 0x7fa524069020 2024-03-21T10:23:08.190 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:08.188+0000 7fa523400700 1 --2- 172.21.15.153:0/73570276 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa524069020 0x7fa52419ec50 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa51000c8f0 tx=0x7fa51000cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:23:08.190 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:08.188+0000 7fa520c00700 1 -- 172.21.15.153:0/73570276 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa5100078a0 con 0x7fa524069020 2024-03-21T10:23:08.190 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:08.188+0000 7fa520c00700 1 -- 172.21.15.153:0/73570276 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa51000f450 con 0x7fa524069020 2024-03-21T10:23:08.190 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:08.188+0000 7fa520c00700 1 -- 172.21.15.153:0/73570276 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa510018610 con 0x7fa524069020 2024-03-21T10:23:08.190 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:08.188+0000 7fa52b000700 1 -- 172.21.15.153:0/73570276 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa524198a70 con 0x7fa524069020 2024-03-21T10:23:08.191 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:08.188+0000 7fa52b000700 1 -- 172.21.15.153:0/73570276 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa524198fc0 con 0x7fa524069020 2024-03-21T10:23:08.191 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:08.189+0000 7fa52b000700 1 -- 172.21.15.153:0/73570276 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa52404ea90 con 0x7fa524069020 2024-03-21T10:23:08.192 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:08.190+0000 7fa520c00700 1 -- 172.21.15.153:0/73570276 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fa51000f5c0 con 0x7fa524069020 2024-03-21T10:23:08.194 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:08.192+0000 7fa520c00700 1 --2- 172.21.15.153:0/73570276 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa50806e460 0x7fa508070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:08.194 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:08.192+0000 7fa520c00700 1 -- 172.21.15.153:0/73570276 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fa51008e440 con 0x7fa524069020 2024-03-21T10:23:08.194 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:08.192+0000 7fa523e00700 1 --2- 172.21.15.153:0/73570276 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa50806e460 0x7fa508070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:08.194 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:08.193+0000 7fa523e00700 1 --2- 172.21.15.153:0/73570276 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa50806e460 0x7fa508070920 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa514006010 tx=0x7fa51401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:23:08.198 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:08.197+0000 7fa520c00700 1 -- 172.21.15.153:0/73570276 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa51005b010 con 0x7fa524069020 2024-03-21T10:23:08.444 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:08.442+0000 7fa52b000700 1 -- 172.21.15.153:0/73570276 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa524199400 con 0x7fa50806e460 2024-03-21T10:23:08.447 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:08.445+0000 7fa520c00700 1 -- 172.21.15.153:0/73570276 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fa524199400 con 0x7fa50806e460 2024-03-21T10:23:08.450 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:08.448+0000 7fa52b000700 1 -- 172.21.15.153:0/73570276 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa50806e460 msgr2=0x7fa508070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:08.450 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:08.448+0000 7fa52b000700 1 --2- 172.21.15.153:0/73570276 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa50806e460 0x7fa508070920 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa514006010 tx=0x7fa51401a040 comp rx=0 tx=0).stop 2024-03-21T10:23:08.451 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:08.448+0000 7fa52b000700 1 -- 172.21.15.153:0/73570276 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa524069020 msgr2=0x7fa52419ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:08.451 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:08.448+0000 7fa52b000700 1 --2- 172.21.15.153:0/73570276 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa524069020 0x7fa52419ec50 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa51000c8f0 tx=0x7fa51000cc00 comp rx=0 tx=0).stop 2024-03-21T10:23:08.451 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:08.448+0000 7fa52b000700 1 -- 172.21.15.153:0/73570276 shutdown_connections 2024-03-21T10:23:08.451 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:08.448+0000 7fa52b000700 1 --2- 172.21.15.153:0/73570276 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa524068670 0x7fa52419e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:08.451 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:08.448+0000 7fa52b000700 1 --2- 172.21.15.153:0/73570276 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa50806e460 0x7fa508070920 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:08.451 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:08.448+0000 7fa52b000700 1 --2- 172.21.15.153:0/73570276 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa524069020 0x7fa52419ec50 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:08.451 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:08.448+0000 7fa52b000700 1 -- 172.21.15.153:0/73570276 >> 172.21.15.153:0/73570276 conn(0x7fa5240755a0 msgr2=0x7fa5240fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:23:08.451 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:08.449+0000 7fa52b000700 1 -- 172.21.15.153:0/73570276 shutdown_connections 2024-03-21T10:23:08.451 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:08.449+0000 7fa52b000700 1 -- 172.21.15.153:0/73570276 wait complete. 2024-03-21T10:23:08.467 INFO:teuthology.orchestra.run.smithi153.stdout:true 2024-03-21T10:23:08.607 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:08.605+0000 7f3ada200700 1 -- 172.21.15.153:0/997472375 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ad4103990 msgr2=0x7f3ad41079e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:08.607 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:08.605+0000 7f3ada200700 1 --2- 172.21.15.153:0/997472375 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ad4103990 0x7f3ad41079e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f3abc00b0a0 tx=0x7f3abc00b3b0 comp rx=0 tx=0).stop 2024-03-21T10:23:08.607 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:08.605+0000 7f3ada200700 1 -- 172.21.15.153:0/997472375 shutdown_connections 2024-03-21T10:23:08.607 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:08.605+0000 7f3ada200700 1 --2- 172.21.15.153:0/997472375 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ad4103990 0x7f3ad41079e0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:08.607 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:08.605+0000 7f3ada200700 1 --2- 172.21.15.153:0/997472375 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3ad4102fe0 0x7f3ad41033c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:08.607 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:08.605+0000 7f3ada200700 1 -- 172.21.15.153:0/997472375 >> 172.21.15.153:0/997472375 conn(0x7f3ad40fe850 msgr2=0x7f3ad4100c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:23:08.607 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:08.605+0000 7f3ada200700 1 -- 172.21.15.153:0/997472375 shutdown_connections 2024-03-21T10:23:08.607 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:08.605+0000 7f3ada200700 1 -- 172.21.15.153:0/997472375 wait complete. 2024-03-21T10:23:08.608 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:08.606+0000 7f3ada200700 1 Processor -- start 2024-03-21T10:23:08.608 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:08.606+0000 7f3ada200700 1 -- start start 2024-03-21T10:23:08.608 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:08.607+0000 7f3ada200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ad4102fe0 0x7f3ad419e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:08.608 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:08.607+0000 7f3ada200700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3ad4103990 0x7f3ad419ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:08.608 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:08.607+0000 7f3ada200700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad419f390 con 0x7f3ad4102fe0 2024-03-21T10:23:08.609 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:08.607+0000 7f3ada200700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad4198840 con 0x7f3ad4103990 2024-03-21T10:23:08.610 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:08.607+0000 7f3ad3400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ad4102fe0 0x7f3ad419e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:08.610 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:08.607+0000 7f3ad3400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ad4102fe0 0x7f3ad419e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:37756/0 (socket says 172.21.15.153:37756) 2024-03-21T10:23:08.610 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:08.607+0000 7f3ad3400700 1 -- 172.21.15.153:0/2783209491 learned_addr learned my addr 172.21.15.153:0/2783209491 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:23:08.610 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:08.607+0000 7f3acaa00700 1 --2- 172.21.15.153:0/2783209491 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3ad4103990 0x7f3ad419ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:08.610 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:08.607+0000 7f3ad3400700 1 -- 172.21.15.153:0/2783209491 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3ad4103990 msgr2=0x7f3ad419ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:08.610 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:08.607+0000 7f3ad3400700 1 --2- 172.21.15.153:0/2783209491 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3ad4103990 0x7f3ad419ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:08.610 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:08.607+0000 7f3ad3400700 1 -- 172.21.15.153:0/2783209491 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3abc009d00 con 0x7f3ad4102fe0 2024-03-21T10:23:08.610 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:08.607+0000 7f3acaa00700 1 --2- 172.21.15.153:0/2783209491 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3ad4103990 0x7f3ad419ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:23:08.610 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:08.608+0000 7f3ad3400700 1 --2- 172.21.15.153:0/2783209491 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ad4102fe0 0x7f3ad419e7c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f3ac400d8d0 tx=0x7f3ac400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:23:08.610 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:08.608+0000 7f3ad0c00700 1 -- 172.21.15.153:0/2783209491 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3ac4009940 con 0x7f3ad4102fe0 2024-03-21T10:23:08.610 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:08.608+0000 7f3ad0c00700 1 -- 172.21.15.153:0/2783209491 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3ac4010460 con 0x7f3ad4102fe0 2024-03-21T10:23:08.610 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:08.608+0000 7f3ad0c00700 1 -- 172.21.15.153:0/2783209491 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3ac400f5f0 con 0x7f3ad4102fe0 2024-03-21T10:23:08.611 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:08.608+0000 7f3ada200700 1 -- 172.21.15.153:0/2783209491 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ad4198b20 con 0x7f3ad4102fe0 2024-03-21T10:23:08.611 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:08.608+0000 7f3ada200700 1 -- 172.21.15.153:0/2783209491 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ad4199070 con 0x7f3ad4102fe0 2024-03-21T10:23:08.611 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:08.609+0000 7f3ada200700 1 -- 172.21.15.153:0/2783209491 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ad410b3e0 con 0x7f3ad4102fe0 2024-03-21T10:23:08.612 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:08.611+0000 7f3ad0c00700 1 -- 172.21.15.153:0/2783209491 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f3ac40105d0 con 0x7f3ad4102fe0 2024-03-21T10:23:08.613 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:08.612+0000 7f3ad0c00700 1 --2- 172.21.15.153:0/2783209491 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f3ab806e460 0x7f3ab8070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:08.613 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:08.612+0000 7f3ad0c00700 1 -- 172.21.15.153:0/2783209491 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f3ac408e5b0 con 0x7f3ad4102fe0 2024-03-21T10:23:08.614 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:08.612+0000 7f3acaa00700 1 --2- 172.21.15.153:0/2783209491 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f3ab806e460 0x7f3ab8070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:08.614 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:08.613+0000 7f3acaa00700 1 --2- 172.21.15.153:0/2783209491 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f3ab806e460 0x7f3ab8070920 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3abc009480 tx=0x7f3abc000f40 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:23:08.618 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:08.616+0000 7f3ad0c00700 1 -- 172.21.15.153:0/2783209491 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f3ac405b100 con 0x7f3ad4102fe0 2024-03-21T10:23:08.855 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:08.852+0000 7f3ada200700 1 -- 172.21.15.153:0/2783209491 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3ad41993a0 con 0x7f3ab806e460 2024-03-21T10:23:08.857 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:08.855+0000 7f3ad0c00700 1 -- 172.21.15.153:0/2783209491 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f3ad41993a0 con 0x7f3ab806e460 2024-03-21T10:23:08.860 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:08.858+0000 7f3ada200700 1 -- 172.21.15.153:0/2783209491 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f3ab806e460 msgr2=0x7f3ab8070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:08.860 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:08.858+0000 7f3ada200700 1 --2- 172.21.15.153:0/2783209491 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f3ab806e460 0x7f3ab8070920 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3abc009480 tx=0x7f3abc000f40 comp rx=0 tx=0).stop 2024-03-21T10:23:08.860 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:08.858+0000 7f3ada200700 1 -- 172.21.15.153:0/2783209491 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ad4102fe0 msgr2=0x7f3ad419e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:08.860 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:08.858+0000 7f3ada200700 1 --2- 172.21.15.153:0/2783209491 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ad4102fe0 0x7f3ad419e7c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f3ac400d8d0 tx=0x7f3ac400dc90 comp rx=0 tx=0).stop 2024-03-21T10:23:08.860 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:08.858+0000 7f3ada200700 1 -- 172.21.15.153:0/2783209491 shutdown_connections 2024-03-21T10:23:08.860 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:08.858+0000 7f3ada200700 1 --2- 172.21.15.153:0/2783209491 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f3ab806e460 0x7f3ab8070920 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:08.860 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:08.858+0000 7f3ada200700 1 --2- 172.21.15.153:0/2783209491 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ad4102fe0 0x7f3ad419e7c0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:08.860 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:08.858+0000 7f3ada200700 1 --2- 172.21.15.153:0/2783209491 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3ad4103990 0x7f3ad419ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:08.860 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:08.858+0000 7f3ada200700 1 -- 172.21.15.153:0/2783209491 >> 172.21.15.153:0/2783209491 conn(0x7f3ad40fe850 msgr2=0x7f3ad40fff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:23:08.861 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:08.858+0000 7f3ada200700 1 -- 172.21.15.153:0/2783209491 shutdown_connections 2024-03-21T10:23:08.861 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:08.858+0000 7f3ada200700 1 -- 172.21.15.153:0/2783209491 wait complete. 2024-03-21T10:23:09.005 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.003+0000 7f222e800700 1 -- 172.21.15.153:0/1675144237 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f22281039b0 msgr2=0x7f2228107a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:09.005 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.003+0000 7f222e800700 1 --2- 172.21.15.153:0/1675144237 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f22281039b0 0x7f2228107a00 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f2214009910 tx=0x7f2214009c20 comp rx=0 tx=0).stop 2024-03-21T10:23:09.005 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.003+0000 7f222e800700 1 -- 172.21.15.153:0/1675144237 shutdown_connections 2024-03-21T10:23:09.005 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.003+0000 7f222e800700 1 --2- 172.21.15.153:0/1675144237 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f22281039b0 0x7f2228107a00 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:09.006 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.003+0000 7f222e800700 1 --2- 172.21.15.153:0/1675144237 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2228103000 0x7f22281033e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:09.006 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.003+0000 7f222e800700 1 -- 172.21.15.153:0/1675144237 >> 172.21.15.153:0/1675144237 conn(0x7f22280fe8d0 msgr2=0x7f2228100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:23:09.006 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.003+0000 7f222e800700 1 -- 172.21.15.153:0/1675144237 shutdown_connections 2024-03-21T10:23:09.006 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.003+0000 7f222e800700 1 -- 172.21.15.153:0/1675144237 wait complete. 2024-03-21T10:23:09.008 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.004+0000 7f222e800700 1 Processor -- start 2024-03-21T10:23:09.008 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.004+0000 7f222e800700 1 -- start start 2024-03-21T10:23:09.008 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.004+0000 7f222e800700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2228103000 0x7f222819e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:09.008 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.004+0000 7f222e800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f22281039b0 0x7f222819ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:09.008 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.005+0000 7f222e800700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f222819f300 con 0x7f22281039b0 2024-03-21T10:23:09.008 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.005+0000 7f222e800700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2228198840 con 0x7f2228103000 2024-03-21T10:23:09.008 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.005+0000 7f2226a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f22281039b0 0x7f222819ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:09.008 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.005+0000 7f2226a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f22281039b0 0x7f222819ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:37786/0 (socket says 172.21.15.153:37786) 2024-03-21T10:23:09.009 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.005+0000 7f2226a00700 1 -- 172.21.15.153:0/2980505810 learned_addr learned my addr 172.21.15.153:0/2980505810 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:23:09.009 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.005+0000 7f2227400700 1 --2- 172.21.15.153:0/2980505810 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2228103000 0x7f222819e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:09.009 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.005+0000 7f2226a00700 1 -- 172.21.15.153:0/2980505810 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2228103000 msgr2=0x7f222819e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:09.009 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.005+0000 7f2226a00700 1 --2- 172.21.15.153:0/2980505810 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2228103000 0x7f222819e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:09.009 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.005+0000 7f2226a00700 1 -- 172.21.15.153:0/2980505810 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22140095f0 con 0x7f22281039b0 2024-03-21T10:23:09.009 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.005+0000 7f2226a00700 1 --2- 172.21.15.153:0/2980505810 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f22281039b0 0x7f222819ed00 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f221400bda0 tx=0x7f221400be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:23:09.010 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.006+0000 7f221fe00700 1 -- 172.21.15.153:0/2980505810 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f221401d070 con 0x7f22281039b0 2024-03-21T10:23:09.010 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.006+0000 7f221fe00700 1 -- 172.21.15.153:0/2980505810 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2214004fb0 con 0x7f22281039b0 2024-03-21T10:23:09.010 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.006+0000 7f222e800700 1 -- 172.21.15.153:0/2980505810 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2228198ac0 con 0x7f22281039b0 2024-03-21T10:23:09.010 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.006+0000 7f221fe00700 1 -- 172.21.15.153:0/2980505810 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f221400f480 con 0x7f22281039b0 2024-03-21T10:23:09.010 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.006+0000 7f222e800700 1 -- 172.21.15.153:0/2980505810 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2228198fb0 con 0x7f22281039b0 2024-03-21T10:23:09.011 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.007+0000 7f222e800700 1 -- 172.21.15.153:0/2980505810 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f222810b400 con 0x7f22281039b0 2024-03-21T10:23:09.011 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.008+0000 7f221fe00700 1 -- 172.21.15.153:0/2980505810 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f2214022870 con 0x7f22281039b0 2024-03-21T10:23:09.011 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.009+0000 7f221fe00700 1 --2- 172.21.15.153:0/2980505810 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f220c06e460 0x7f220c070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:09.011 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.009+0000 7f221fe00700 1 -- 172.21.15.153:0/2980505810 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f221408fae0 con 0x7f22281039b0 2024-03-21T10:23:09.011 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.009+0000 7f2227400700 1 --2- 172.21.15.153:0/2980505810 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f220c06e460 0x7f220c070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:09.012 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.010+0000 7f2227400700 1 --2- 172.21.15.153:0/2980505810 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f220c06e460 0x7f220c070920 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f221800ba60 tx=0x7f2218005d50 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:23:09.015 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.014+0000 7f221fe00700 1 -- 172.21.15.153:0/2980505810 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f221405c6e0 con 0x7f22281039b0 2024-03-21T10:23:09.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:23:08 smithi174 ceph-mon[88004]: pgmap v338: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:23:09.242 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.240+0000 7f222e800700 1 -- 172.21.15.153:0/2980505810 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f22281991d0 con 0x7f220c06e460 2024-03-21T10:23:09.243 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:23:08 smithi153 ceph-mon[113733]: pgmap v338: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:23:09.254 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.252+0000 7f221fe00700 1 -- 172.21.15.153:0/2980505810 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f22281991d0 con 0x7f220c06e460 2024-03-21T10:23:09.254 INFO:teuthology.orchestra.run.smithi153.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-21T10:23:09.255 INFO:teuthology.orchestra.run.smithi153.stdout:alertmanager.smithi153 smithi153 *:9093,9094 running (11m) 3m ago 25m 14.7M - 0.25.0 c8568f914cd2 9ed3f2200b5f 2024-03-21T10:23:09.255 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi153 smithi153 running (26m) 3m ago 26m 9424k - 18.2.1 5be31c24972a 5b61994396a6 2024-03-21T10:23:09.255 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi174 smithi174 running (24m) 3m ago 24m 9244k - 18.2.1 5be31c24972a 4f7a5fc3ac28 2024-03-21T10:23:09.255 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi153 smithi153 running (8m) 3m ago 25m 6656k - 19.0.0-2231-g2edfee68 237e43c7a65f 32c21104d5b0 2024-03-21T10:23:09.255 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi174 smithi174 running (8m) 3m ago 24m 6660k - 19.0.0-2231-g2edfee68 237e43c7a65f a04e48967ab6 2024-03-21T10:23:09.256 INFO:teuthology.orchestra.run.smithi153.stdout:grafana.smithi153 smithi153 *:3000 running (11m) 3m ago 25m 73.6M - 9.4.12 2bacad6d85d8 ee24026ab786 2024-03-21T10:23:09.256 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.iuyluj smithi153 running (3m) 3m ago 18m 10.9M - 19.0.0-2231-g2edfee68 237e43c7a65f 87aff8a5de70 2024-03-21T10:23:09.256 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.zipckr smithi153 running (3m) 3m ago 18m 11.8M - 19.0.0-2231-g2edfee68 237e43c7a65f ccc262bd5dd3 2024-03-21T10:23:09.256 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.nhozag smithi174 running (18m) 3m ago 18m 12.0M - 18.2.1 5be31c24972a 2fee4000308f 2024-03-21T10:23:09.256 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.yyimst smithi174 running (3m) 3m ago 18m 10.7M - 19.0.0-2231-g2edfee68 237e43c7a65f fc1962f9351e 2024-03-21T10:23:09.256 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi153.tluxni smithi153 *:8443,9283,8765 running (14m) 3m ago 27m 509M - 19.0.0-2231-g2edfee68 237e43c7a65f 7ffee0e55382 2024-03-21T10:23:09.256 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi174.qpcwat smithi174 *:8443,9283,8765 running (13m) 3m ago 24m 452M - 19.0.0-2231-g2edfee68 237e43c7a65f 748968f216ab 2024-03-21T10:23:09.256 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi153 smithi153 running (9m) 3m ago 27m 55.0M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f dd8038303dc3 2024-03-21T10:23:09.257 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi174 smithi174 running (9m) 3m ago 24m 44.2M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f 29d78a788f2c 2024-03-21T10:23:09.257 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi153 smithi153 *:9100 running (13m) 3m ago 25m 7889k - 1.5.0 0da6a335fe13 bb835edc0e0b 2024-03-21T10:23:09.257 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi174 smithi174 *:9100 running (13m) 3m ago 24m 8102k - 1.5.0 0da6a335fe13 7c4ecad790fa 2024-03-21T10:23:09.257 INFO:teuthology.orchestra.run.smithi153.stdout:osd.0 smithi153 running (7m) 3m ago 23m 116M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 507900fd4791 2024-03-21T10:23:09.257 INFO:teuthology.orchestra.run.smithi153.stdout:osd.1 smithi153 running (6m) 3m ago 22m 89.2M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 0d646452e9c1 2024-03-21T10:23:09.257 INFO:teuthology.orchestra.run.smithi153.stdout:osd.2 smithi153 running (6m) 3m ago 21m 87.4M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 1326f2afdfcd 2024-03-21T10:23:09.257 INFO:teuthology.orchestra.run.smithi153.stdout:osd.3 smithi174 running (5m) 3m ago 21m 83.6M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 70aafac71ad5 2024-03-21T10:23:09.258 INFO:teuthology.orchestra.run.smithi153.stdout:osd.4 smithi174 running (5m) 3m ago 20m 92.2M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 0d746ce6e79d 2024-03-21T10:23:09.258 INFO:teuthology.orchestra.run.smithi153.stdout:osd.5 smithi174 running (4m) 3m ago 20m 86.2M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 53e5f35b8cac 2024-03-21T10:23:09.258 INFO:teuthology.orchestra.run.smithi153.stdout:prometheus.smithi153 smithi153 *:9095 running (12m) 3m ago 24m 73.5M - 2.43.0 a07b618ecd1d 7d95561e50c2 2024-03-21T10:23:09.259 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.254+0000 7f222e800700 1 -- 172.21.15.153:0/2980505810 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f220c06e460 msgr2=0x7f220c070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:09.259 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.254+0000 7f222e800700 1 --2- 172.21.15.153:0/2980505810 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f220c06e460 0x7f220c070920 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f221800ba60 tx=0x7f2218005d50 comp rx=0 tx=0).stop 2024-03-21T10:23:09.259 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.254+0000 7f222e800700 1 -- 172.21.15.153:0/2980505810 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f22281039b0 msgr2=0x7f222819ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:09.260 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.254+0000 7f222e800700 1 --2- 172.21.15.153:0/2980505810 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f22281039b0 0x7f222819ed00 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f221400bda0 tx=0x7f221400be80 comp rx=0 tx=0).stop 2024-03-21T10:23:09.260 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.255+0000 7f222e800700 1 -- 172.21.15.153:0/2980505810 shutdown_connections 2024-03-21T10:23:09.260 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.255+0000 7f222e800700 1 --2- 172.21.15.153:0/2980505810 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2228103000 0x7f222819e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:09.260 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.255+0000 7f222e800700 1 --2- 172.21.15.153:0/2980505810 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f220c06e460 0x7f220c070920 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:09.260 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.255+0000 7f222e800700 1 --2- 172.21.15.153:0/2980505810 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f22281039b0 0x7f222819ed00 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:09.260 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.255+0000 7f222e800700 1 -- 172.21.15.153:0/2980505810 >> 172.21.15.153:0/2980505810 conn(0x7f22280fe8d0 msgr2=0x7f2228100cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:23:09.261 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.255+0000 7f222e800700 1 -- 172.21.15.153:0/2980505810 shutdown_connections 2024-03-21T10:23:09.261 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.255+0000 7f222e800700 1 -- 172.21.15.153:0/2980505810 wait complete. 2024-03-21T10:23:09.405 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.403+0000 7efc48c00700 1 -- 172.21.15.153:0/1609982066 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efc44073a80 msgr2=0x7efc4410cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:09.406 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.403+0000 7efc48c00700 1 --2- 172.21.15.153:0/1609982066 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efc44073a80 0x7efc4410cee0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7efc30009910 tx=0x7efc30009c20 comp rx=0 tx=0).stop 2024-03-21T10:23:09.406 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.403+0000 7efc48c00700 1 -- 172.21.15.153:0/1609982066 shutdown_connections 2024-03-21T10:23:09.406 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.403+0000 7efc48c00700 1 --2- 172.21.15.153:0/1609982066 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efc44073a80 0x7efc4410cee0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:09.406 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.403+0000 7efc48c00700 1 --2- 172.21.15.153:0/1609982066 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efc44073160 0x7efc44073540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:09.406 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.403+0000 7efc48c00700 1 -- 172.21.15.153:0/1609982066 >> 172.21.15.153:0/1609982066 conn(0x7efc440fc690 msgr2=0x7efc440feab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:23:09.406 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.403+0000 7efc48c00700 1 -- 172.21.15.153:0/1609982066 shutdown_connections 2024-03-21T10:23:09.406 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.404+0000 7efc48c00700 1 -- 172.21.15.153:0/1609982066 wait complete. 2024-03-21T10:23:09.407 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.404+0000 7efc48c00700 1 Processor -- start 2024-03-21T10:23:09.407 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.405+0000 7efc48c00700 1 -- start start 2024-03-21T10:23:09.407 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.405+0000 7efc48c00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efc44073160 0x7efc44198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:09.408 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.405+0000 7efc48c00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efc44073a80 0x7efc44198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:09.408 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.405+0000 7efc48c00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc44199530 con 0x7efc44073160 2024-03-21T10:23:09.408 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.405+0000 7efc48c00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc4419d340 con 0x7efc44073a80 2024-03-21T10:23:09.408 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.405+0000 7efc41e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efc44073160 0x7efc44198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:09.408 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.405+0000 7efc41e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efc44073160 0x7efc44198960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:37810/0 (socket says 172.21.15.153:37810) 2024-03-21T10:23:09.408 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.405+0000 7efc41e00700 1 -- 172.21.15.153:0/3047103192 learned_addr learned my addr 172.21.15.153:0/3047103192 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:23:09.408 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.406+0000 7efc41400700 1 --2- 172.21.15.153:0/3047103192 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efc44073a80 0x7efc44198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:09.408 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.406+0000 7efc41e00700 1 -- 172.21.15.153:0/3047103192 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efc44073a80 msgr2=0x7efc44198ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:09.408 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.406+0000 7efc41e00700 1 --2- 172.21.15.153:0/3047103192 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efc44073a80 0x7efc44198ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:09.408 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.406+0000 7efc41e00700 1 -- 172.21.15.153:0/3047103192 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc300095f0 con 0x7efc44073160 2024-03-21T10:23:09.409 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.406+0000 7efc41e00700 1 --2- 172.21.15.153:0/3047103192 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efc44073160 0x7efc44198960 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7efc3400d8d0 tx=0x7efc3400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:23:09.409 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.406+0000 7efc3aa00700 1 -- 172.21.15.153:0/3047103192 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efc34009940 con 0x7efc44073160 2024-03-21T10:23:09.409 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.406+0000 7efc3aa00700 1 -- 172.21.15.153:0/3047103192 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efc34010460 con 0x7efc44073160 2024-03-21T10:23:09.409 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.406+0000 7efc48c00700 1 -- 172.21.15.153:0/3047103192 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc4419d620 con 0x7efc44073160 2024-03-21T10:23:09.409 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.406+0000 7efc3aa00700 1 -- 172.21.15.153:0/3047103192 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efc3400f5f0 con 0x7efc44073160 2024-03-21T10:23:09.409 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.407+0000 7efc48c00700 1 -- 172.21.15.153:0/3047103192 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc4419db70 con 0x7efc44073160 2024-03-21T10:23:09.409 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.408+0000 7efc48c00700 1 -- 172.21.15.153:0/3047103192 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc4410a5e0 con 0x7efc44073160 2024-03-21T10:23:09.410 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.409+0000 7efc3aa00700 1 -- 172.21.15.153:0/3047103192 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7efc340105d0 con 0x7efc44073160 2024-03-21T10:23:09.411 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.409+0000 7efc3aa00700 1 --2- 172.21.15.153:0/3047103192 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7efc2806e460 0x7efc28070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:09.411 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.410+0000 7efc3aa00700 1 -- 172.21.15.153:0/3047103192 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7efc3408e5b0 con 0x7efc44073160 2024-03-21T10:23:09.411 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.410+0000 7efc41400700 1 --2- 172.21.15.153:0/3047103192 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7efc2806e460 0x7efc28070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:09.412 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.410+0000 7efc41400700 1 --2- 172.21.15.153:0/3047103192 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7efc2806e460 0x7efc28070920 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7efc30006010 tx=0x7efc3000b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:23:09.416 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.414+0000 7efc3aa00700 1 -- 172.21.15.153:0/3047103192 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7efc3405b100 con 0x7efc44073160 2024-03-21T10:23:09.710 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.709+0000 7efc48c00700 1 -- 172.21.15.153:0/3047103192 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7efc4419dd90 con 0x7efc44073160 2024-03-21T10:23:09.711 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.710+0000 7efc3aa00700 1 -- 172.21.15.153:0/3047103192 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+805 (secure 0 0 0) 0x7efc3405ac90 con 0x7efc44073160 2024-03-21T10:23:09.712 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:23:09.712 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": { 2024-03-21T10:23:09.712 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:23:09.712 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:23:09.712 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": { 2024-03-21T10:23:09.712 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:23:09.712 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:23:09.712 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": { 2024-03-21T10:23:09.712 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 6 2024-03-21T10:23:09.712 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:23:09.713 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": { 2024-03-21T10:23:09.713 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-03-21T10:23:09.713 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 3 2024-03-21T10:23:09.713 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:23:09.713 INFO:teuthology.orchestra.run.smithi153.stdout: "overall": { 2024-03-21T10:23:09.713 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-03-21T10:23:09.713 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 13 2024-03-21T10:23:09.713 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-03-21T10:23:09.713 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:23:09.715 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.713+0000 7efc48c00700 1 -- 172.21.15.153:0/3047103192 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7efc2806e460 msgr2=0x7efc28070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:09.715 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.713+0000 7efc48c00700 1 --2- 172.21.15.153:0/3047103192 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7efc2806e460 0x7efc28070920 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7efc30006010 tx=0x7efc3000b540 comp rx=0 tx=0).stop 2024-03-21T10:23:09.715 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.713+0000 7efc48c00700 1 -- 172.21.15.153:0/3047103192 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efc44073160 msgr2=0x7efc44198960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:09.715 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.713+0000 7efc48c00700 1 --2- 172.21.15.153:0/3047103192 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efc44073160 0x7efc44198960 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7efc3400d8d0 tx=0x7efc3400dc90 comp rx=0 tx=0).stop 2024-03-21T10:23:09.715 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.713+0000 7efc48c00700 1 -- 172.21.15.153:0/3047103192 shutdown_connections 2024-03-21T10:23:09.715 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.713+0000 7efc48c00700 1 --2- 172.21.15.153:0/3047103192 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7efc2806e460 0x7efc28070920 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:09.715 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.713+0000 7efc48c00700 1 --2- 172.21.15.153:0/3047103192 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efc44073160 0x7efc44198960 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:09.715 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.713+0000 7efc48c00700 1 --2- 172.21.15.153:0/3047103192 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efc44073a80 0x7efc44198ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:09.715 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.713+0000 7efc48c00700 1 -- 172.21.15.153:0/3047103192 >> 172.21.15.153:0/3047103192 conn(0x7efc440fc690 msgr2=0x7efc44107720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:23:09.716 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.713+0000 7efc48c00700 1 -- 172.21.15.153:0/3047103192 shutdown_connections 2024-03-21T10:23:09.716 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.713+0000 7efc48c00700 1 -- 172.21.15.153:0/3047103192 wait complete. 2024-03-21T10:23:09.870 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.868+0000 7f9b00e00700 1 -- 172.21.15.153:0/1437749096 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9afc10d280 msgr2=0x7f9afc10d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:09.870 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.868+0000 7f9b00e00700 1 --2- 172.21.15.153:0/1437749096 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9afc10d280 0x7f9afc10d660 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f9aec009910 tx=0x7f9aec009c20 comp rx=0 tx=0).stop 2024-03-21T10:23:09.870 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.868+0000 7f9b00e00700 1 -- 172.21.15.153:0/1437749096 shutdown_connections 2024-03-21T10:23:09.870 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.868+0000 7f9b00e00700 1 --2- 172.21.15.153:0/1437749096 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9afc1071d0 0x7f9afc107650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:09.870 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.868+0000 7f9b00e00700 1 --2- 172.21.15.153:0/1437749096 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9afc10d280 0x7f9afc10d660 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:09.870 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.868+0000 7f9b00e00700 1 -- 172.21.15.153:0/1437749096 >> 172.21.15.153:0/1437749096 conn(0x7f9afc0769e0 msgr2=0x7f9afc078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:23:09.871 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.868+0000 7f9b00e00700 1 -- 172.21.15.153:0/1437749096 shutdown_connections 2024-03-21T10:23:09.871 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.868+0000 7f9b00e00700 1 -- 172.21.15.153:0/1437749096 wait complete. 2024-03-21T10:23:09.872 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.869+0000 7f9b00e00700 1 Processor -- start 2024-03-21T10:23:09.872 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.869+0000 7f9b00e00700 1 -- start start 2024-03-21T10:23:09.872 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.870+0000 7f9b00e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9afc1071d0 0x7f9afc19ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:09.872 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.870+0000 7f9b00e00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9afc10d280 0x7f9afc19d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:09.872 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.870+0000 7f9b00e00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9afc19d8a0 con 0x7f9afc1071d0 2024-03-21T10:23:09.872 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.870+0000 7f9b00e00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9afc1a05d0 con 0x7f9afc10d280 2024-03-21T10:23:09.872 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.870+0000 7f9af9e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9afc1071d0 0x7f9afc19ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:09.872 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.870+0000 7f9af9e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9afc1071d0 0x7f9afc19ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:37820/0 (socket says 172.21.15.153:37820) 2024-03-21T10:23:09.872 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.870+0000 7f9af9e00700 1 -- 172.21.15.153:0/1374585613 learned_addr learned my addr 172.21.15.153:0/1374585613 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:23:09.873 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.870+0000 7f9af9400700 1 --2- 172.21.15.153:0/1374585613 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9afc10d280 0x7f9afc19d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:09.873 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.870+0000 7f9af9e00700 1 -- 172.21.15.153:0/1374585613 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9afc10d280 msgr2=0x7f9afc19d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:09.873 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.870+0000 7f9af9e00700 1 --2- 172.21.15.153:0/1374585613 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9afc10d280 0x7f9afc19d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:09.873 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.870+0000 7f9af9e00700 1 -- 172.21.15.153:0/1374585613 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9aec0095f0 con 0x7f9afc1071d0 2024-03-21T10:23:09.873 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.870+0000 7f9af9400700 1 --2- 172.21.15.153:0/1374585613 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9afc10d280 0x7f9afc19d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:23:09.874 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.871+0000 7f9af9e00700 1 --2- 172.21.15.153:0/1374585613 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9afc1071d0 0x7f9afc19ccd0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f9aec00b5c0 tx=0x7f9aec00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:23:09.874 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.871+0000 7f9af2a00700 1 -- 172.21.15.153:0/1374585613 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9aec01d070 con 0x7f9afc1071d0 2024-03-21T10:23:09.874 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.871+0000 7f9af2a00700 1 -- 172.21.15.153:0/1374585613 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9aec005e00 con 0x7f9afc1071d0 2024-03-21T10:23:09.874 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.871+0000 7f9af2a00700 1 -- 172.21.15.153:0/1374585613 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9aec017400 con 0x7f9afc1071d0 2024-03-21T10:23:09.874 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.871+0000 7f9b00e00700 1 -- 172.21.15.153:0/1374585613 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9afc1a0850 con 0x7f9afc1071d0 2024-03-21T10:23:09.874 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.871+0000 7f9b00e00700 1 -- 172.21.15.153:0/1374585613 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9afc1a0dc0 con 0x7f9afc1071d0 2024-03-21T10:23:09.874 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.872+0000 7f9b00e00700 1 -- 172.21.15.153:0/1374585613 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9afc04ea90 con 0x7f9afc1071d0 2024-03-21T10:23:09.875 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.873+0000 7f9af2a00700 1 -- 172.21.15.153:0/1374585613 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f9aec00f460 con 0x7f9afc1071d0 2024-03-21T10:23:09.876 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.875+0000 7f9af2a00700 1 --2- 172.21.15.153:0/1374585613 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f9ae006e460 0x7f9ae0070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:09.876 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.875+0000 7f9af2a00700 1 -- 172.21.15.153:0/1374585613 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f9aec08fa70 con 0x7f9afc1071d0 2024-03-21T10:23:09.876 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.875+0000 7f9af9400700 1 --2- 172.21.15.153:0/1374585613 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f9ae006e460 0x7f9ae0070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:09.877 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.875+0000 7f9af9400700 1 --2- 172.21.15.153:0/1374585613 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f9ae006e460 0x7f9ae0070920 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9ae8005fd0 tx=0x7f9ae8005ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:23:09.881 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:09.879+0000 7f9af2a00700 1 -- 172.21.15.153:0/1374585613 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f9aec058a90 con 0x7f9afc1071d0 2024-03-21T10:23:10.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:23:09 smithi153 ceph-mon[113733]: from='client.34646 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:23:10.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:23:09 smithi153 ceph-mon[113733]: from='client.34650 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:23:10.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:23:09 smithi153 ceph-mon[113733]: from='client.34654 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:23:10.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:23:09 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/3047103192' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:23:10.134 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:10.132+0000 7f9b00e00700 1 -- 172.21.15.153:0/1374585613 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f9afc19dff0 con 0x7f9afc1071d0 2024-03-21T10:23:10.135 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:10.133+0000 7f9af2a00700 1 -- 172.21.15.153:0/1374585613 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1876 (secure 0 0 0) 0x7f9aec05c150 con 0x7f9afc1071d0 2024-03-21T10:23:10.135 INFO:teuthology.orchestra.run.smithi153.stdout:e30 2024-03-21T10:23:10.135 INFO:teuthology.orchestra.run.smithi153.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-21T10:23:10.136 INFO:teuthology.orchestra.run.smithi153.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:23:10.136 INFO:teuthology.orchestra.run.smithi153.stdout:legacy client fscid: 1 2024-03-21T10:23:10.136 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:23:10.136 INFO:teuthology.orchestra.run.smithi153.stdout:Filesystem 'cephfs' (1) 2024-03-21T10:23:10.136 INFO:teuthology.orchestra.run.smithi153.stdout:fs_name cephfs 2024-03-21T10:23:10.136 INFO:teuthology.orchestra.run.smithi153.stdout:epoch 23 2024-03-21T10:23:10.136 INFO:teuthology.orchestra.run.smithi153.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-21T10:23:10.136 INFO:teuthology.orchestra.run.smithi153.stdout:created 2024-03-21T10:04:27.086576+0000 2024-03-21T10:23:10.136 INFO:teuthology.orchestra.run.smithi153.stdout:modified 2024-03-21T10:19:05.022092+0000 2024-03-21T10:23:10.136 INFO:teuthology.orchestra.run.smithi153.stdout:tableserver 0 2024-03-21T10:23:10.136 INFO:teuthology.orchestra.run.smithi153.stdout:root 0 2024-03-21T10:23:10.136 INFO:teuthology.orchestra.run.smithi153.stdout:session_timeout 60 2024-03-21T10:23:10.137 INFO:teuthology.orchestra.run.smithi153.stdout:session_autoclose 300 2024-03-21T10:23:10.137 INFO:teuthology.orchestra.run.smithi153.stdout:max_file_size 1099511627776 2024-03-21T10:23:10.137 INFO:teuthology.orchestra.run.smithi153.stdout:max_xattr_size 65536 2024-03-21T10:23:10.137 INFO:teuthology.orchestra.run.smithi153.stdout:required_client_features {} 2024-03-21T10:23:10.137 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure 0 2024-03-21T10:23:10.137 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure_osd_epoch 94 2024-03-21T10:23:10.137 INFO:teuthology.orchestra.run.smithi153.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:23:10.137 INFO:teuthology.orchestra.run.smithi153.stdout:max_mds 2 2024-03-21T10:23:10.137 INFO:teuthology.orchestra.run.smithi153.stdout:in 0,1 2024-03-21T10:23:10.137 INFO:teuthology.orchestra.run.smithi153.stdout:up {} 2024-03-21T10:23:10.137 INFO:teuthology.orchestra.run.smithi153.stdout:failed 0,1 2024-03-21T10:23:10.137 INFO:teuthology.orchestra.run.smithi153.stdout:damaged 2024-03-21T10:23:10.137 INFO:teuthology.orchestra.run.smithi153.stdout:stopped 2024-03-21T10:23:10.137 INFO:teuthology.orchestra.run.smithi153.stdout:data_pools [3] 2024-03-21T10:23:10.138 INFO:teuthology.orchestra.run.smithi153.stdout:metadata_pool 2 2024-03-21T10:23:10.138 INFO:teuthology.orchestra.run.smithi153.stdout:inline_data enabled 2024-03-21T10:23:10.138 INFO:teuthology.orchestra.run.smithi153.stdout:balancer 2024-03-21T10:23:10.138 INFO:teuthology.orchestra.run.smithi153.stdout:bal_rank_mask -1 2024-03-21T10:23:10.138 INFO:teuthology.orchestra.run.smithi153.stdout:standby_count_wanted 1 2024-03-21T10:23:10.138 INFO:teuthology.orchestra.run.smithi153.stdout:qdb_cluster leader: 0 members: 2024-03-21T10:23:10.138 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:23:10.138 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:23:10.138 INFO:teuthology.orchestra.run.smithi153.stdout:Standby daemons: 2024-03-21T10:23:10.138 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:23:10.138 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.zipckr{-1:34442} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6826/1103846295,v1:172.21.15.153:6827/1103846295] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:23:10.138 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.iuyluj{-1:34474} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6828/1738630832,v1:172.21.15.153:6829/1738630832] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:23:10.138 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.nhozag{-1:44269} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.174:6826/3967223227,v1:172.21.15.174:6827/3967223227] compat {c=[1],r=[1],i=[7ff]}] 2024-03-21T10:23:10.138 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.yyimst{-1:44313} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.174:6824/4127786797,v1:172.21.15.174:6825/4127786797] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:23:10.139 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:10.137+0000 7f9b00e00700 1 -- 172.21.15.153:0/1374585613 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f9ae006e460 msgr2=0x7f9ae0070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:10.139 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:10.137+0000 7f9b00e00700 1 --2- 172.21.15.153:0/1374585613 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f9ae006e460 0x7f9ae0070920 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9ae8005fd0 tx=0x7f9ae8005ee0 comp rx=0 tx=0).stop 2024-03-21T10:23:10.139 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:10.137+0000 7f9b00e00700 1 -- 172.21.15.153:0/1374585613 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9afc1071d0 msgr2=0x7f9afc19ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:10.140 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:10.137+0000 7f9b00e00700 1 --2- 172.21.15.153:0/1374585613 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9afc1071d0 0x7f9afc19ccd0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f9aec00b5c0 tx=0x7f9aec00be20 comp rx=0 tx=0).stop 2024-03-21T10:23:10.140 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:10.137+0000 7f9b00e00700 1 -- 172.21.15.153:0/1374585613 shutdown_connections 2024-03-21T10:23:10.140 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:10.137+0000 7f9b00e00700 1 --2- 172.21.15.153:0/1374585613 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f9ae006e460 0x7f9ae0070920 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:10.140 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:10.137+0000 7f9b00e00700 1 --2- 172.21.15.153:0/1374585613 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9afc1071d0 0x7f9afc19ccd0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:10.140 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:10.137+0000 7f9b00e00700 1 --2- 172.21.15.153:0/1374585613 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9afc10d280 0x7f9afc19d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:10.140 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:10.137+0000 7f9b00e00700 1 -- 172.21.15.153:0/1374585613 >> 172.21.15.153:0/1374585613 conn(0x7f9afc0769e0 msgr2=0x7f9afc078320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:23:10.140 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:10.137+0000 7f9b00e00700 1 -- 172.21.15.153:0/1374585613 shutdown_connections 2024-03-21T10:23:10.140 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:10.137+0000 7f9b00e00700 1 -- 172.21.15.153:0/1374585613 wait complete. 2024-03-21T10:23:10.140 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 30 2024-03-21T10:23:10.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:23:09 smithi174 ceph-mon[88004]: from='client.34646 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:23:10.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:23:09 smithi174 ceph-mon[88004]: from='client.34650 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:23:10.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:23:09 smithi174 ceph-mon[88004]: from='client.34654 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:23:10.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:23:09 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/3047103192' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:23:10.291 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:10.289+0000 7fa290c00700 1 -- 172.21.15.153:0/3962870185 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa28c101750 msgr2=0x7fa28c1057a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:10.291 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:10.289+0000 7fa290c00700 1 --2- 172.21.15.153:0/3962870185 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa28c101750 0x7fa28c1057a0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fa278009910 tx=0x7fa278009c20 comp rx=0 tx=0).stop 2024-03-21T10:23:10.292 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:10.289+0000 7fa290c00700 1 -- 172.21.15.153:0/3962870185 shutdown_connections 2024-03-21T10:23:10.292 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:10.289+0000 7fa290c00700 1 --2- 172.21.15.153:0/3962870185 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa28c101750 0x7fa28c1057a0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:10.292 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:10.289+0000 7fa290c00700 1 --2- 172.21.15.153:0/3962870185 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa28c100da0 0x7fa28c101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:10.292 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:10.289+0000 7fa290c00700 1 -- 172.21.15.153:0/3962870185 >> 172.21.15.153:0/3962870185 conn(0x7fa28c0fc630 msgr2=0x7fa28c0fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:23:10.292 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:10.289+0000 7fa290c00700 1 -- 172.21.15.153:0/3962870185 shutdown_connections 2024-03-21T10:23:10.292 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:10.289+0000 7fa290c00700 1 -- 172.21.15.153:0/3962870185 wait complete. 2024-03-21T10:23:10.293 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:10.291+0000 7fa290c00700 1 Processor -- start 2024-03-21T10:23:10.293 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:10.291+0000 7fa290c00700 1 -- start start 2024-03-21T10:23:10.293 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:10.291+0000 7fa290c00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa28c100da0 0x7fa28c198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:10.293 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:10.291+0000 7fa290c00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa28c101750 0x7fa28c198e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:10.293 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:10.291+0000 7fa290c00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa28c199520 con 0x7fa28c100da0 2024-03-21T10:23:10.293 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:10.291+0000 7fa290c00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa28c19d330 con 0x7fa28c101750 2024-03-21T10:23:10.293 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:10.291+0000 7fa289e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa28c100da0 0x7fa28c198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:10.293 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:10.292+0000 7fa289e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa28c100da0 0x7fa28c198950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:37830/0 (socket says 172.21.15.153:37830) 2024-03-21T10:23:10.294 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:10.292+0000 7fa289e00700 1 -- 172.21.15.153:0/2696134360 learned_addr learned my addr 172.21.15.153:0/2696134360 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:23:10.295 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:10.292+0000 7fa289e00700 1 -- 172.21.15.153:0/2696134360 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa28c101750 msgr2=0x7fa28c198e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:10.295 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:10.292+0000 7fa289400700 1 --2- 172.21.15.153:0/2696134360 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa28c101750 0x7fa28c198e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:10.295 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:10.292+0000 7fa289e00700 1 --2- 172.21.15.153:0/2696134360 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa28c101750 0x7fa28c198e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:10.295 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:10.292+0000 7fa289e00700 1 -- 172.21.15.153:0/2696134360 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2780095f0 con 0x7fa28c100da0 2024-03-21T10:23:10.295 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:10.292+0000 7fa289e00700 1 --2- 172.21.15.153:0/2696134360 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa28c100da0 0x7fa28c198950 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fa27c0060b0 tx=0x7fa27c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:23:10.296 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:10.293+0000 7fa282a00700 1 -- 172.21.15.153:0/2696134360 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa27c00dda0 con 0x7fa28c100da0 2024-03-21T10:23:10.296 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:10.293+0000 7fa282a00700 1 -- 172.21.15.153:0/2696134360 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa27c00f040 con 0x7fa28c100da0 2024-03-21T10:23:10.296 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:10.293+0000 7fa290c00700 1 -- 172.21.15.153:0/2696134360 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa28c19d610 con 0x7fa28c100da0 2024-03-21T10:23:10.296 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:10.293+0000 7fa282a00700 1 -- 172.21.15.153:0/2696134360 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa27c014790 con 0x7fa28c100da0 2024-03-21T10:23:10.296 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:10.293+0000 7fa290c00700 1 -- 172.21.15.153:0/2696134360 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa28c19db60 con 0x7fa28c100da0 2024-03-21T10:23:10.296 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:10.294+0000 7fa290c00700 1 -- 172.21.15.153:0/2696134360 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa28c104f20 con 0x7fa28c100da0 2024-03-21T10:23:10.296 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:10.295+0000 7fa282a00700 1 -- 172.21.15.153:0/2696134360 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fa27c0148f0 con 0x7fa28c100da0 2024-03-21T10:23:10.298 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:10.296+0000 7fa282a00700 1 --2- 172.21.15.153:0/2696134360 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa27006e460 0x7fa270070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:10.298 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:10.296+0000 7fa282a00700 1 -- 172.21.15.153:0/2696134360 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fa27c08dfa0 con 0x7fa28c100da0 2024-03-21T10:23:10.298 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:10.296+0000 7fa289400700 1 --2- 172.21.15.153:0/2696134360 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa27006e460 0x7fa270070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:10.298 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:10.297+0000 7fa289400700 1 --2- 172.21.15.153:0/2696134360 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa27006e460 0x7fa270070920 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa278006010 tx=0x7fa27800b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:23:10.302 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:10.301+0000 7fa282a00700 1 -- 172.21.15.153:0/2696134360 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa27c05ac20 con 0x7fa28c100da0 2024-03-21T10:23:10.545 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:10.543+0000 7fa290c00700 1 -- 172.21.15.153:0/2696134360 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa28c19dea0 con 0x7fa27006e460 2024-03-21T10:23:10.547 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:10.546+0000 7fa282a00700 1 -- 172.21.15.153:0/2696134360 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fa28c19dea0 con 0x7fa27006e460 2024-03-21T10:23:10.548 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:23:10.548 INFO:teuthology.orchestra.run.smithi153.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232", 2024-03-21T10:23:10.548 INFO:teuthology.orchestra.run.smithi153.stdout: "in_progress": true, 2024-03-21T10:23:10.548 INFO:teuthology.orchestra.run.smithi153.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-21T10:23:10.548 INFO:teuthology.orchestra.run.smithi153.stdout: "services_complete": [ 2024-03-21T10:23:10.548 INFO:teuthology.orchestra.run.smithi153.stdout: "crash", 2024-03-21T10:23:10.548 INFO:teuthology.orchestra.run.smithi153.stdout: "mon", 2024-03-21T10:23:10.548 INFO:teuthology.orchestra.run.smithi153.stdout: "osd", 2024-03-21T10:23:10.549 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr" 2024-03-21T10:23:10.549 INFO:teuthology.orchestra.run.smithi153.stdout: ], 2024-03-21T10:23:10.549 INFO:teuthology.orchestra.run.smithi153.stdout: "progress": "15/23 daemons upgraded", 2024-03-21T10:23:10.549 INFO:teuthology.orchestra.run.smithi153.stdout: "message": "Currently upgrading mds daemons", 2024-03-21T10:23:10.549 INFO:teuthology.orchestra.run.smithi153.stdout: "is_paused": false 2024-03-21T10:23:10.549 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:23:10.550 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:10.548+0000 7fa290c00700 1 -- 172.21.15.153:0/2696134360 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa27006e460 msgr2=0x7fa270070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:10.550 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:10.548+0000 7fa290c00700 1 --2- 172.21.15.153:0/2696134360 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa27006e460 0x7fa270070920 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa278006010 tx=0x7fa27800b540 comp rx=0 tx=0).stop 2024-03-21T10:23:10.551 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:10.548+0000 7fa290c00700 1 -- 172.21.15.153:0/2696134360 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa28c100da0 msgr2=0x7fa28c198950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:10.551 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:10.548+0000 7fa290c00700 1 --2- 172.21.15.153:0/2696134360 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa28c100da0 0x7fa28c198950 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fa27c0060b0 tx=0x7fa27c00d3b0 comp rx=0 tx=0).stop 2024-03-21T10:23:10.551 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:10.549+0000 7fa290c00700 1 -- 172.21.15.153:0/2696134360 shutdown_connections 2024-03-21T10:23:10.551 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:10.549+0000 7fa290c00700 1 --2- 172.21.15.153:0/2696134360 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa27006e460 0x7fa270070920 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:10.551 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:10.549+0000 7fa290c00700 1 --2- 172.21.15.153:0/2696134360 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa28c100da0 0x7fa28c198950 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:10.551 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:10.549+0000 7fa290c00700 1 --2- 172.21.15.153:0/2696134360 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa28c101750 0x7fa28c198e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:10.551 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:10.549+0000 7fa290c00700 1 -- 172.21.15.153:0/2696134360 >> 172.21.15.153:0/2696134360 conn(0x7fa28c0fc630 msgr2=0x7fa28c0fe9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:23:10.551 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:10.549+0000 7fa290c00700 1 -- 172.21.15.153:0/2696134360 shutdown_connections 2024-03-21T10:23:10.551 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:10.549+0000 7fa290c00700 1 -- 172.21.15.153:0/2696134360 wait complete. 2024-03-21T10:23:10.705 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:10.703+0000 7f0645c00700 1 -- 172.21.15.153:0/3259853302 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0640069020 msgr2=0x7f0640105790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:10.705 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:10.703+0000 7f0645c00700 1 --2- 172.21.15.153:0/3259853302 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0640069020 0x7f0640105790 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f062c009910 tx=0x7f062c009c20 comp rx=0 tx=0).stop 2024-03-21T10:23:10.706 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:10.703+0000 7f0645c00700 1 -- 172.21.15.153:0/3259853302 shutdown_connections 2024-03-21T10:23:10.706 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:10.703+0000 7f0645c00700 1 --2- 172.21.15.153:0/3259853302 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0640069020 0x7f0640105790 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:10.706 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:10.703+0000 7f0645c00700 1 --2- 172.21.15.153:0/3259853302 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0640068670 0x7f0640068a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:10.706 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:10.703+0000 7f0645c00700 1 -- 172.21.15.153:0/3259853302 >> 172.21.15.153:0/3259853302 conn(0x7f06400755a0 msgr2=0x7f06400759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:23:10.706 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:10.703+0000 7f0645c00700 1 -- 172.21.15.153:0/3259853302 shutdown_connections 2024-03-21T10:23:10.706 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:10.703+0000 7f0645c00700 1 -- 172.21.15.153:0/3259853302 wait complete. 2024-03-21T10:23:10.708 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:10.704+0000 7f0645c00700 1 Processor -- start 2024-03-21T10:23:10.708 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:10.704+0000 7f0645c00700 1 -- start start 2024-03-21T10:23:10.708 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:10.705+0000 7f0645c00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0640068670 0x7f064019e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:10.708 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:10.705+0000 7f0645c00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0640069020 0x7f064019ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:10.708 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:10.705+0000 7f0645c00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f064019f3b0 con 0x7f0640068670 2024-03-21T10:23:10.708 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:10.705+0000 7f0645c00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0640198860 con 0x7f0640069020 2024-03-21T10:23:10.708 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:10.705+0000 7f063ea00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0640068670 0x7f064019e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:10.708 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:10.705+0000 7f063ea00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0640068670 0x7f064019e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:37848/0 (socket says 172.21.15.153:37848) 2024-03-21T10:23:10.708 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:10.705+0000 7f063ea00700 1 -- 172.21.15.153:0/498587989 learned_addr learned my addr 172.21.15.153:0/498587989 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:23:10.709 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:10.705+0000 7f063e000700 1 --2- 172.21.15.153:0/498587989 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0640069020 0x7f064019ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:10.709 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:10.705+0000 7f063ea00700 1 -- 172.21.15.153:0/498587989 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0640069020 msgr2=0x7f064019ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:10.709 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:10.705+0000 7f063ea00700 1 --2- 172.21.15.153:0/498587989 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0640069020 0x7f064019ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:10.709 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:10.705+0000 7f063ea00700 1 -- 172.21.15.153:0/498587989 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f062c0095f0 con 0x7f0640068670 2024-03-21T10:23:10.709 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:10.705+0000 7f063e000700 1 --2- 172.21.15.153:0/498587989 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0640069020 0x7f064019ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:23:10.709 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:10.706+0000 7f063ea00700 1 --2- 172.21.15.153:0/498587989 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0640068670 0x7f064019e7e0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f06300060b0 tx=0x7f063000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:23:10.709 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:10.706+0000 7f0637400700 1 -- 172.21.15.153:0/498587989 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f063000dda0 con 0x7f0640068670 2024-03-21T10:23:10.709 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:10.706+0000 7f0637400700 1 -- 172.21.15.153:0/498587989 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f063000f040 con 0x7f0640068670 2024-03-21T10:23:10.709 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:10.706+0000 7f0645c00700 1 -- 172.21.15.153:0/498587989 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0640198b40 con 0x7f0640068670 2024-03-21T10:23:10.709 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:10.706+0000 7f0637400700 1 -- 172.21.15.153:0/498587989 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0630014790 con 0x7f0640068670 2024-03-21T10:23:10.709 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:10.706+0000 7f0645c00700 1 -- 172.21.15.153:0/498587989 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0640199090 con 0x7f0640068670 2024-03-21T10:23:10.710 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:10.707+0000 7f0645c00700 1 -- 172.21.15.153:0/498587989 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f064004ea90 con 0x7f0640068670 2024-03-21T10:23:10.710 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:10.708+0000 7f0637400700 1 -- 172.21.15.153:0/498587989 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f0630014980 con 0x7f0640068670 2024-03-21T10:23:10.711 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:10.709+0000 7f0637400700 1 --2- 172.21.15.153:0/498587989 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f062406e460 0x7f0624070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:10.711 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:10.709+0000 7f0637400700 1 -- 172.21.15.153:0/498587989 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f063008dff0 con 0x7f0640068670 2024-03-21T10:23:10.711 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:10.709+0000 7f063e000700 1 --2- 172.21.15.153:0/498587989 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f062406e460 0x7f0624070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:10.711 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:10.710+0000 7f063e000700 1 --2- 172.21.15.153:0/498587989 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f062406e460 0x7f0624070920 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f062c006010 tx=0x7f062c00b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:23:10.715 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:10.714+0000 7f0637400700 1 -- 172.21.15.153:0/498587989 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f06300c2740 con 0x7f0640068670 2024-03-21T10:23:11.004 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:11.002+0000 7f0645c00700 1 -- 172.21.15.153:0/498587989 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f0640199310 con 0x7f0640068670 2024-03-21T10:23:11.005 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:11.003+0000 7f0637400700 1 -- 172.21.15.153:0/498587989 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f063001a1c0 con 0x7f0640068670 2024-03-21T10:23:11.005 INFO:teuthology.orchestra.run.smithi153.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-03-21T10:23:11.005 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-21T10:23:11.005 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is degraded 2024-03-21T10:23:11.005 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-21T10:23:11.005 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-21T10:23:11.005 INFO:teuthology.orchestra.run.smithi153.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-21T10:23:11.005 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-21T10:23:11.008 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:11.005+0000 7f0645c00700 1 -- 172.21.15.153:0/498587989 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f062406e460 msgr2=0x7f0624070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:11.008 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:11.005+0000 7f0645c00700 1 --2- 172.21.15.153:0/498587989 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f062406e460 0x7f0624070920 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f062c006010 tx=0x7f062c00b540 comp rx=0 tx=0).stop 2024-03-21T10:23:11.008 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:11.005+0000 7f0645c00700 1 -- 172.21.15.153:0/498587989 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0640068670 msgr2=0x7f064019e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:11.008 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:11.005+0000 7f0645c00700 1 --2- 172.21.15.153:0/498587989 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0640068670 0x7f064019e7e0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f06300060b0 tx=0x7f063000d3b0 comp rx=0 tx=0).stop 2024-03-21T10:23:11.008 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:11.006+0000 7f0645c00700 1 -- 172.21.15.153:0/498587989 shutdown_connections 2024-03-21T10:23:11.008 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:11.006+0000 7f0645c00700 1 --2- 172.21.15.153:0/498587989 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f062406e460 0x7f0624070920 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:11.008 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:11.006+0000 7f0645c00700 1 --2- 172.21.15.153:0/498587989 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0640068670 0x7f064019e7e0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:11.008 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:11.006+0000 7f0645c00700 1 --2- 172.21.15.153:0/498587989 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0640069020 0x7f064019ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:11.008 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:11.006+0000 7f0645c00700 1 -- 172.21.15.153:0/498587989 >> 172.21.15.153:0/498587989 conn(0x7f06400755a0 msgr2=0x7f0640102760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:23:11.008 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:11.006+0000 7f0645c00700 1 -- 172.21.15.153:0/498587989 shutdown_connections 2024-03-21T10:23:11.008 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:11.006+0000 7f0645c00700 1 -- 172.21.15.153:0/498587989 wait complete. 2024-03-21T10:23:11.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:23:10 smithi174 ceph-mon[88004]: pgmap v339: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:23:11.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:23:10 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/1374585613' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:23:11.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:23:10 smithi153 ceph-mon[113733]: pgmap v339: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:23:11.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:23:10 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/1374585613' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:23:12.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:23:11 smithi174 ceph-mon[88004]: from='client.34666 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:23:12.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:23:11 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/498587989' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:23:12.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:23:11 smithi153 ceph-mon[113733]: from='client.34666 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:23:12.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:23:11 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/498587989' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:23:13.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:23:12 smithi174 ceph-mon[88004]: pgmap v340: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:23:13.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:23:12 smithi153 ceph-mon[113733]: pgmap v340: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:23:15.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:23:14 smithi174 ceph-mon[88004]: pgmap v341: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:23:15.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:23:14 smithi153 ceph-mon[113733]: pgmap v341: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:23:17.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:23:16 smithi174 ceph-mon[88004]: pgmap v342: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:23:17.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:23:16 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:23:17.304 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:23:16 smithi153 ceph-mon[113733]: pgmap v342: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:23:17.305 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:23:16 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:23:19.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:23:18 smithi174 ceph-mon[88004]: pgmap v343: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:23:19.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:23:18 smithi153 ceph-mon[113733]: pgmap v343: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:23:21.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:23:20 smithi174 ceph-mon[88004]: pgmap v344: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:23:21.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:23:20 smithi153 ceph-mon[113733]: pgmap v344: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:23:23.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:23:22 smithi174 ceph-mon[88004]: pgmap v345: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:23:23.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:23:22 smithi153 ceph-mon[113733]: pgmap v345: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:23:25.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:23:24 smithi174 ceph-mon[88004]: pgmap v346: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:23:25.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:23:24 smithi153 ceph-mon[113733]: pgmap v346: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:23:27.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:23:26 smithi153 ceph-mon[113733]: pgmap v347: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:23:27.454 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:23:26 smithi174 ceph-mon[88004]: pgmap v347: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:23:29.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:23:28 smithi153 ceph-mon[113733]: pgmap v348: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:23:29.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:23:28 smithi174 ceph-mon[88004]: pgmap v348: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:23:31.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:23:30 smithi153 ceph-mon[113733]: pgmap v349: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:23:31.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:23:30 smithi174 ceph-mon[88004]: pgmap v349: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:23:32.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:23:31 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:23:32.353 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:23:31 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:23:33.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:23:32 smithi153 ceph-mon[113733]: pgmap v350: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:23:33.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:23:32 smithi174 ceph-mon[88004]: pgmap v350: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:23:35.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:23:34 smithi153 ceph-mon[113733]: pgmap v351: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:23:35.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:23:34 smithi174 ceph-mon[88004]: pgmap v351: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:23:37.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:23:36 smithi153 ceph-mon[113733]: pgmap v352: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:23:37.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:23:36 smithi174 ceph-mon[88004]: pgmap v352: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:23:39.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:23:38 smithi153 ceph-mon[113733]: pgmap v353: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:23:39.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:23:38 smithi174 ceph-mon[88004]: pgmap v353: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:23:41.176 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.175+0000 7f72d4400700 1 -- 172.21.15.153:0/2576029304 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72cc10d280 msgr2=0x7f72cc10d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:41.176 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.175+0000 7f72d4400700 1 --2- 172.21.15.153:0/2576029304 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72cc10d280 0x7f72cc10d660 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f72c0009910 tx=0x7f72c0009c20 comp rx=0 tx=0).stop 2024-03-21T10:23:41.177 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.175+0000 7f72d4400700 1 -- 172.21.15.153:0/2576029304 shutdown_connections 2024-03-21T10:23:41.177 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.175+0000 7f72d4400700 1 --2- 172.21.15.153:0/2576029304 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f72cc1071d0 0x7f72cc107650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:41.177 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.175+0000 7f72d4400700 1 --2- 172.21.15.153:0/2576029304 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72cc10d280 0x7f72cc10d660 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:41.177 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.175+0000 7f72d4400700 1 -- 172.21.15.153:0/2576029304 >> 172.21.15.153:0/2576029304 conn(0x7f72cc0769e0 msgr2=0x7f72cc078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:23:41.177 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.175+0000 7f72d4400700 1 -- 172.21.15.153:0/2576029304 shutdown_connections 2024-03-21T10:23:41.177 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.175+0000 7f72d4400700 1 -- 172.21.15.153:0/2576029304 wait complete. 2024-03-21T10:23:41.177 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.176+0000 7f72d4400700 1 Processor -- start 2024-03-21T10:23:41.178 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.176+0000 7f72d4400700 1 -- start start 2024-03-21T10:23:41.178 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.176+0000 7f72d4400700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f72cc1071d0 0x7f72cc19ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:41.178 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.177+0000 7f72d4400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72cc10d280 0x7f72cc19d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:41.178 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.177+0000 7f72d4400700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72cc19d8a0 con 0x7f72cc10d280 2024-03-21T10:23:41.179 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.177+0000 7f72d4400700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72cc1a05d0 con 0x7f72cc1071d0 2024-03-21T10:23:41.179 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.177+0000 7f72d1000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72cc10d280 0x7f72cc19d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:41.179 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.177+0000 7f72d1000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72cc10d280 0x7f72cc19d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:50766/0 (socket says 172.21.15.153:50766) 2024-03-21T10:23:41.179 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.177+0000 7f72d1000700 1 -- 172.21.15.153:0/1549530907 learned_addr learned my addr 172.21.15.153:0/1549530907 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:23:41.179 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.177+0000 7f72d1a00700 1 --2- 172.21.15.153:0/1549530907 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f72cc1071d0 0x7f72cc19ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:41.179 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.177+0000 7f72d1000700 1 -- 172.21.15.153:0/1549530907 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f72cc1071d0 msgr2=0x7f72cc19ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:41.179 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.177+0000 7f72d1000700 1 --2- 172.21.15.153:0/1549530907 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f72cc1071d0 0x7f72cc19ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:41.179 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.178+0000 7f72d1000700 1 -- 172.21.15.153:0/1549530907 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72c00095f0 con 0x7f72cc10d280 2024-03-21T10:23:41.179 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.178+0000 7f72d1000700 1 --2- 172.21.15.153:0/1549530907 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72cc10d280 0x7f72cc19d210 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f72b800c8a0 tx=0x7f72b800cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:23:41.180 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.178+0000 7f72ca000700 1 -- 172.21.15.153:0/1549530907 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f72b80078e0 con 0x7f72cc10d280 2024-03-21T10:23:41.180 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.178+0000 7f72ca000700 1 -- 172.21.15.153:0/1549530907 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f72b800f450 con 0x7f72cc10d280 2024-03-21T10:23:41.180 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.178+0000 7f72ca000700 1 -- 172.21.15.153:0/1549530907 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f72b800e5e0 con 0x7f72cc10d280 2024-03-21T10:23:41.180 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.178+0000 7f72d4400700 1 -- 172.21.15.153:0/1549530907 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72cc1a08b0 con 0x7f72cc10d280 2024-03-21T10:23:41.180 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.178+0000 7f72d4400700 1 -- 172.21.15.153:0/1549530907 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72cc1a0e80 con 0x7f72cc10d280 2024-03-21T10:23:41.180 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.179+0000 7f72d4400700 1 -- 172.21.15.153:0/1549530907 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72cc04ea90 con 0x7f72cc10d280 2024-03-21T10:23:41.182 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.181+0000 7f72ca000700 1 -- 172.21.15.153:0/1549530907 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f72b800f5c0 con 0x7f72cc10d280 2024-03-21T10:23:41.182 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.182+0000 7f72ca000700 1 --2- 172.21.15.153:0/1549530907 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f72b406e460 0x7f72b4070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:41.183 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.182+0000 7f72ca000700 1 -- 172.21.15.153:0/1549530907 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f72b808e520 con 0x7f72cc10d280 2024-03-21T10:23:41.183 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.182+0000 7f72d1a00700 1 --2- 172.21.15.153:0/1549530907 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f72b406e460 0x7f72b4070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:41.183 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.183+0000 7f72d1a00700 1 --2- 172.21.15.153:0/1549530907 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f72b406e460 0x7f72b4070920 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f72c0006010 tx=0x7f72c001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:23:41.187 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.187+0000 7f72ca000700 1 -- 172.21.15.153:0/1549530907 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f72b8057640 con 0x7f72cc10d280 2024-03-21T10:23:41.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:23:41 smithi174 ceph-mon[88004]: pgmap v354: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:23:41.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:23:41 smithi153 ceph-mon[113733]: pgmap v354: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:23:41.421 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.420+0000 7f72d4400700 1 -- 172.21.15.153:0/1549530907 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f72cc1a1300 con 0x7f72b406e460 2024-03-21T10:23:41.423 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.423+0000 7f72ca000700 1 -- 172.21.15.153:0/1549530907 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f72cc1a1300 con 0x7f72b406e460 2024-03-21T10:23:41.427 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.426+0000 7f72d4400700 1 -- 172.21.15.153:0/1549530907 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f72b406e460 msgr2=0x7f72b4070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:41.427 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.426+0000 7f72d4400700 1 --2- 172.21.15.153:0/1549530907 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f72b406e460 0x7f72b4070920 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f72c0006010 tx=0x7f72c001a040 comp rx=0 tx=0).stop 2024-03-21T10:23:41.427 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.426+0000 7f72d4400700 1 -- 172.21.15.153:0/1549530907 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72cc10d280 msgr2=0x7f72cc19d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:41.427 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.426+0000 7f72d4400700 1 --2- 172.21.15.153:0/1549530907 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72cc10d280 0x7f72cc19d210 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f72b800c8a0 tx=0x7f72b800cbb0 comp rx=0 tx=0).stop 2024-03-21T10:23:41.427 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.426+0000 7f72d4400700 1 -- 172.21.15.153:0/1549530907 shutdown_connections 2024-03-21T10:23:41.427 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.426+0000 7f72d4400700 1 --2- 172.21.15.153:0/1549530907 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f72cc1071d0 0x7f72cc19ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:41.427 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.426+0000 7f72d4400700 1 --2- 172.21.15.153:0/1549530907 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f72b406e460 0x7f72b4070920 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:41.427 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.426+0000 7f72d4400700 1 --2- 172.21.15.153:0/1549530907 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72cc10d280 0x7f72cc19d210 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:41.427 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.426+0000 7f72d4400700 1 -- 172.21.15.153:0/1549530907 >> 172.21.15.153:0/1549530907 conn(0x7f72cc0769e0 msgr2=0x7f72cc078320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:23:41.428 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.426+0000 7f72d4400700 1 -- 172.21.15.153:0/1549530907 shutdown_connections 2024-03-21T10:23:41.428 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.427+0000 7f72d4400700 1 -- 172.21.15.153:0/1549530907 wait complete. 2024-03-21T10:23:41.443 INFO:teuthology.orchestra.run.smithi153.stdout:true 2024-03-21T10:23:41.585 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.584+0000 7fc964a00700 1 -- 172.21.15.153:0/3614518307 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc960073a50 msgr2=0x7fc9601113f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:41.586 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.584+0000 7fc964a00700 1 --2- 172.21.15.153:0/3614518307 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc960073a50 0x7fc9601113f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fc94c009910 tx=0x7fc94c009c20 comp rx=0 tx=0).stop 2024-03-21T10:23:41.586 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.584+0000 7fc964a00700 1 -- 172.21.15.153:0/3614518307 shutdown_connections 2024-03-21T10:23:41.586 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.584+0000 7fc964a00700 1 --2- 172.21.15.153:0/3614518307 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc960073a50 0x7fc9601113f0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:41.586 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.584+0000 7fc964a00700 1 --2- 172.21.15.153:0/3614518307 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc960073130 0x7fc960073510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:41.586 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.584+0000 7fc964a00700 1 -- 172.21.15.153:0/3614518307 >> 172.21.15.153:0/3614518307 conn(0x7fc9600fc3f0 msgr2=0x7fc9600fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:23:41.586 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.584+0000 7fc964a00700 1 -- 172.21.15.153:0/3614518307 shutdown_connections 2024-03-21T10:23:41.586 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.584+0000 7fc964a00700 1 -- 172.21.15.153:0/3614518307 wait complete. 2024-03-21T10:23:41.587 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.585+0000 7fc964a00700 1 Processor -- start 2024-03-21T10:23:41.587 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.586+0000 7fc964a00700 1 -- start start 2024-03-21T10:23:41.588 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.586+0000 7fc964a00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc960073130 0x7fc9601a2830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:41.588 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.586+0000 7fc964a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc960073a50 0x7fc9601a2d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:41.588 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.586+0000 7fc964a00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9601a3400 con 0x7fc960073a50 2024-03-21T10:23:41.588 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.586+0000 7fc964a00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc96019c8b0 con 0x7fc960073130 2024-03-21T10:23:41.588 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.586+0000 7fc95ea00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc960073a50 0x7fc9601a2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:41.588 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.586+0000 7fc95ea00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc960073a50 0x7fc9601a2d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:50776/0 (socket says 172.21.15.153:50776) 2024-03-21T10:23:41.588 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.586+0000 7fc95ea00700 1 -- 172.21.15.153:0/922065878 learned_addr learned my addr 172.21.15.153:0/922065878 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:23:41.588 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.587+0000 7fc95f400700 1 --2- 172.21.15.153:0/922065878 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc960073130 0x7fc9601a2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:41.588 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.587+0000 7fc95ea00700 1 -- 172.21.15.153:0/922065878 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc960073130 msgr2=0x7fc9601a2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:41.588 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.587+0000 7fc95ea00700 1 --2- 172.21.15.153:0/922065878 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc960073130 0x7fc9601a2830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:41.588 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.587+0000 7fc95ea00700 1 -- 172.21.15.153:0/922065878 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc94c0095f0 con 0x7fc960073a50 2024-03-21T10:23:41.589 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.587+0000 7fc95ea00700 1 --2- 172.21.15.153:0/922065878 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc960073a50 0x7fc9601a2d70 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fc94c00b5c0 tx=0x7fc94c00be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:23:41.589 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.587+0000 7fc957e00700 1 -- 172.21.15.153:0/922065878 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc94c01d070 con 0x7fc960073a50 2024-03-21T10:23:41.589 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.588+0000 7fc957e00700 1 -- 172.21.15.153:0/922065878 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc94c00f460 con 0x7fc960073a50 2024-03-21T10:23:41.589 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.588+0000 7fc964a00700 1 -- 172.21.15.153:0/922065878 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc96019cb90 con 0x7fc960073a50 2024-03-21T10:23:41.589 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.588+0000 7fc957e00700 1 -- 172.21.15.153:0/922065878 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc94c017630 con 0x7fc960073a50 2024-03-21T10:23:41.589 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.588+0000 7fc964a00700 1 -- 172.21.15.153:0/922065878 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc96019d0e0 con 0x7fc960073a50 2024-03-21T10:23:41.590 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.589+0000 7fc964a00700 1 -- 172.21.15.153:0/922065878 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc96010eb70 con 0x7fc960073a50 2024-03-21T10:23:41.590 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.590+0000 7fc957e00700 1 -- 172.21.15.153:0/922065878 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fc94c00f5d0 con 0x7fc960073a50 2024-03-21T10:23:41.592 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.591+0000 7fc957e00700 1 --2- 172.21.15.153:0/922065878 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fc94406e460 0x7fc944070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:41.592 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.591+0000 7fc957e00700 1 -- 172.21.15.153:0/922065878 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fc94c08faa0 con 0x7fc960073a50 2024-03-21T10:23:41.592 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.591+0000 7fc95f400700 1 --2- 172.21.15.153:0/922065878 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fc94406e460 0x7fc944070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:41.592 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.592+0000 7fc95f400700 1 --2- 172.21.15.153:0/922065878 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fc94406e460 0x7fc944070920 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc950005fd0 tx=0x7fc95000a560 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:23:41.601 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.600+0000 7fc957e00700 1 -- 172.21.15.153:0/922065878 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc94c05c670 con 0x7fc960073a50 2024-03-21T10:23:41.834 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.834+0000 7fc964a00700 1 -- 172.21.15.153:0/922065878 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc96019d3c0 con 0x7fc94406e460 2024-03-21T10:23:41.836 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.835+0000 7fc957e00700 1 -- 172.21.15.153:0/922065878 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fc96019d3c0 con 0x7fc94406e460 2024-03-21T10:23:41.839 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.838+0000 7fc964a00700 1 -- 172.21.15.153:0/922065878 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fc94406e460 msgr2=0x7fc944070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:41.839 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.838+0000 7fc964a00700 1 --2- 172.21.15.153:0/922065878 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fc94406e460 0x7fc944070920 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc950005fd0 tx=0x7fc95000a560 comp rx=0 tx=0).stop 2024-03-21T10:23:41.839 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.838+0000 7fc964a00700 1 -- 172.21.15.153:0/922065878 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc960073a50 msgr2=0x7fc9601a2d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:41.839 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.838+0000 7fc964a00700 1 --2- 172.21.15.153:0/922065878 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc960073a50 0x7fc9601a2d70 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fc94c00b5c0 tx=0x7fc94c00be00 comp rx=0 tx=0).stop 2024-03-21T10:23:41.839 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.838+0000 7fc964a00700 1 -- 172.21.15.153:0/922065878 shutdown_connections 2024-03-21T10:23:41.839 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.838+0000 7fc964a00700 1 --2- 172.21.15.153:0/922065878 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc960073130 0x7fc9601a2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:41.839 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.838+0000 7fc964a00700 1 --2- 172.21.15.153:0/922065878 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fc94406e460 0x7fc944070920 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:41.839 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.838+0000 7fc964a00700 1 --2- 172.21.15.153:0/922065878 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc960073a50 0x7fc9601a2d70 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:41.840 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.838+0000 7fc964a00700 1 -- 172.21.15.153:0/922065878 >> 172.21.15.153:0/922065878 conn(0x7fc9600fc3f0 msgr2=0x7fc960102f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:23:41.840 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.838+0000 7fc964a00700 1 -- 172.21.15.153:0/922065878 shutdown_connections 2024-03-21T10:23:41.840 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.838+0000 7fc964a00700 1 -- 172.21.15.153:0/922065878 wait complete. 2024-03-21T10:23:41.983 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.982+0000 7f8d97200700 1 -- 172.21.15.153:0/3653578062 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d90101da0 msgr2=0x7f8d90102180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:41.984 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.982+0000 7f8d97200700 1 --2- 172.21.15.153:0/3653578062 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d90101da0 0x7f8d90102180 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f8d80009910 tx=0x7f8d80009c20 comp rx=0 tx=0).stop 2024-03-21T10:23:41.984 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.982+0000 7f8d97200700 1 -- 172.21.15.153:0/3653578062 shutdown_connections 2024-03-21T10:23:41.984 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.982+0000 7f8d97200700 1 --2- 172.21.15.153:0/3653578062 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8d901026c0 0x7f8d9010abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:41.984 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.982+0000 7f8d97200700 1 --2- 172.21.15.153:0/3653578062 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d90101da0 0x7f8d90102180 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:41.984 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.982+0000 7f8d97200700 1 -- 172.21.15.153:0/3653578062 >> 172.21.15.153:0/3653578062 conn(0x7f8d900fb570 msgr2=0x7f8d900fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:23:41.984 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.982+0000 7f8d97200700 1 -- 172.21.15.153:0/3653578062 shutdown_connections 2024-03-21T10:23:41.984 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.982+0000 7f8d97200700 1 -- 172.21.15.153:0/3653578062 wait complete. 2024-03-21T10:23:41.986 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.983+0000 7f8d97200700 1 Processor -- start 2024-03-21T10:23:41.986 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.983+0000 7f8d97200700 1 -- start start 2024-03-21T10:23:41.986 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.984+0000 7f8d97200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d90101da0 0x7f8d901989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:41.986 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.984+0000 7f8d97200700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8d901026c0 0x7f8d90198ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:41.986 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.984+0000 7f8d97200700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d90199570 con 0x7f8d90101da0 2024-03-21T10:23:41.986 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.984+0000 7f8d97200700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9019d380 con 0x7f8d901026c0 2024-03-21T10:23:41.986 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.984+0000 7f8d8fe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d90101da0 0x7f8d901989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:41.986 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.984+0000 7f8d8fe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d90101da0 0x7f8d901989a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:50790/0 (socket says 172.21.15.153:50790) 2024-03-21T10:23:41.986 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.984+0000 7f8d8fe00700 1 -- 172.21.15.153:0/313880222 learned_addr learned my addr 172.21.15.153:0/313880222 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:23:41.987 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.984+0000 7f8d8f400700 1 --2- 172.21.15.153:0/313880222 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8d901026c0 0x7f8d90198ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:41.987 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.984+0000 7f8d8fe00700 1 -- 172.21.15.153:0/313880222 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8d901026c0 msgr2=0x7f8d90198ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:41.987 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.984+0000 7f8d8fe00700 1 --2- 172.21.15.153:0/313880222 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8d901026c0 0x7f8d90198ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:41.987 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.984+0000 7f8d8fe00700 1 -- 172.21.15.153:0/313880222 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d800095f0 con 0x7f8d90101da0 2024-03-21T10:23:41.987 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.984+0000 7f8d8f400700 1 --2- 172.21.15.153:0/313880222 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8d901026c0 0x7f8d90198ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:23:41.987 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.985+0000 7f8d8fe00700 1 --2- 172.21.15.153:0/313880222 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d90101da0 0x7f8d901989a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f8d8000bd70 tx=0x7f8d8000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:23:41.987 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.985+0000 7f8d8cc00700 1 -- 172.21.15.153:0/313880222 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d8001d070 con 0x7f8d90101da0 2024-03-21T10:23:41.987 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.985+0000 7f8d8cc00700 1 -- 172.21.15.153:0/313880222 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8d8000f460 con 0x7f8d90101da0 2024-03-21T10:23:41.987 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.985+0000 7f8d8cc00700 1 -- 172.21.15.153:0/313880222 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d80017630 con 0x7f8d90101da0 2024-03-21T10:23:41.987 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.985+0000 7f8d97200700 1 -- 172.21.15.153:0/313880222 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d9019d600 con 0x7f8d90101da0 2024-03-21T10:23:41.987 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.985+0000 7f8d97200700 1 -- 172.21.15.153:0/313880222 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d9019daf0 con 0x7f8d90101da0 2024-03-21T10:23:41.988 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.986+0000 7f8d97200700 1 -- 172.21.15.153:0/313880222 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d9010a710 con 0x7f8d90101da0 2024-03-21T10:23:41.988 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.987+0000 7f8d8cc00700 1 -- 172.21.15.153:0/313880222 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f8d80003a60 con 0x7f8d90101da0 2024-03-21T10:23:41.988 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.988+0000 7f8d8cc00700 1 --2- 172.21.15.153:0/313880222 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f8d7406e460 0x7f8d74070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:41.989 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.988+0000 7f8d8cc00700 1 -- 172.21.15.153:0/313880222 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f8d8008f9b0 con 0x7f8d90101da0 2024-03-21T10:23:41.989 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.988+0000 7f8d8f400700 1 --2- 172.21.15.153:0/313880222 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f8d7406e460 0x7f8d74070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:41.989 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.989+0000 7f8d8f400700 1 --2- 172.21.15.153:0/313880222 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f8d7406e460 0x7f8d74070920 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c005fd0 tx=0x7f8d7c005ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:23:41.993 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:41.993+0000 7f8d8cc00700 1 -- 172.21.15.153:0/313880222 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f8d8005c640 con 0x7f8d90101da0 2024-03-21T10:23:42.209 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:42.208+0000 7f8d97200700 1 -- 172.21.15.153:0/313880222 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f8d9019de20 con 0x7f8d7406e460 2024-03-21T10:23:42.224 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:42.223+0000 7f8d8cc00700 1 -- 172.21.15.153:0/313880222 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f8d9019de20 con 0x7f8d7406e460 2024-03-21T10:23:42.225 INFO:teuthology.orchestra.run.smithi153.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-21T10:23:42.225 INFO:teuthology.orchestra.run.smithi153.stdout:alertmanager.smithi153 smithi153 *:9093,9094 running (12m) 3m ago 26m 14.7M - 0.25.0 c8568f914cd2 9ed3f2200b5f 2024-03-21T10:23:42.225 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi153 smithi153 running (26m) 3m ago 26m 9424k - 18.2.1 5be31c24972a 5b61994396a6 2024-03-21T10:23:42.225 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi174 smithi174 running (25m) 3m ago 25m 9244k - 18.2.1 5be31c24972a 4f7a5fc3ac28 2024-03-21T10:23:42.225 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi153 smithi153 running (8m) 3m ago 26m 6656k - 19.0.0-2231-g2edfee68 237e43c7a65f 32c21104d5b0 2024-03-21T10:23:42.225 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi174 smithi174 running (8m) 3m ago 24m 6660k - 19.0.0-2231-g2edfee68 237e43c7a65f a04e48967ab6 2024-03-21T10:23:42.225 INFO:teuthology.orchestra.run.smithi153.stdout:grafana.smithi153 smithi153 *:3000 running (11m) 3m ago 25m 73.6M - 9.4.12 2bacad6d85d8 ee24026ab786 2024-03-21T10:23:42.225 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.iuyluj smithi153 running (4m) 3m ago 19m 10.9M - 19.0.0-2231-g2edfee68 237e43c7a65f 87aff8a5de70 2024-03-21T10:23:42.225 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.zipckr smithi153 running (4m) 3m ago 19m 11.8M - 19.0.0-2231-g2edfee68 237e43c7a65f ccc262bd5dd3 2024-03-21T10:23:42.225 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.nhozag smithi174 running (18m) 3m ago 18m 12.0M - 18.2.1 5be31c24972a 2fee4000308f 2024-03-21T10:23:42.225 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.yyimst smithi174 running (3m) 3m ago 19m 10.7M - 19.0.0-2231-g2edfee68 237e43c7a65f fc1962f9351e 2024-03-21T10:23:42.225 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi153.tluxni smithi153 *:8443,9283,8765 running (15m) 3m ago 28m 509M - 19.0.0-2231-g2edfee68 237e43c7a65f 7ffee0e55382 2024-03-21T10:23:42.225 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi174.qpcwat smithi174 *:8443,9283,8765 running (14m) 3m ago 24m 452M - 19.0.0-2231-g2edfee68 237e43c7a65f 748968f216ab 2024-03-21T10:23:42.226 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi153 smithi153 running (10m) 3m ago 28m 55.0M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f dd8038303dc3 2024-03-21T10:23:42.226 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi174 smithi174 running (9m) 3m ago 24m 44.2M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f 29d78a788f2c 2024-03-21T10:23:42.226 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi153 smithi153 *:9100 running (13m) 3m ago 26m 7889k - 1.5.0 0da6a335fe13 bb835edc0e0b 2024-03-21T10:23:42.226 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi174 smithi174 *:9100 running (13m) 3m ago 24m 8102k - 1.5.0 0da6a335fe13 7c4ecad790fa 2024-03-21T10:23:42.226 INFO:teuthology.orchestra.run.smithi153.stdout:osd.0 smithi153 running (8m) 3m ago 23m 116M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 507900fd4791 2024-03-21T10:23:42.226 INFO:teuthology.orchestra.run.smithi153.stdout:osd.1 smithi153 running (7m) 3m ago 23m 89.2M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 0d646452e9c1 2024-03-21T10:23:42.226 INFO:teuthology.orchestra.run.smithi153.stdout:osd.2 smithi153 running (6m) 3m ago 22m 87.4M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 1326f2afdfcd 2024-03-21T10:23:42.226 INFO:teuthology.orchestra.run.smithi153.stdout:osd.3 smithi174 running (6m) 3m ago 21m 83.6M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 70aafac71ad5 2024-03-21T10:23:42.226 INFO:teuthology.orchestra.run.smithi153.stdout:osd.4 smithi174 running (5m) 3m ago 21m 92.2M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 0d746ce6e79d 2024-03-21T10:23:42.226 INFO:teuthology.orchestra.run.smithi153.stdout:osd.5 smithi174 running (4m) 3m ago 20m 86.2M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 53e5f35b8cac 2024-03-21T10:23:42.227 INFO:teuthology.orchestra.run.smithi153.stdout:prometheus.smithi153 smithi153 *:9095 running (12m) 3m ago 25m 73.5M - 2.43.0 a07b618ecd1d 7d95561e50c2 2024-03-21T10:23:42.227 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:42.226+0000 7f8d97200700 1 -- 172.21.15.153:0/313880222 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f8d7406e460 msgr2=0x7f8d74070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:42.227 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:42.226+0000 7f8d97200700 1 --2- 172.21.15.153:0/313880222 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f8d7406e460 0x7f8d74070920 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c005fd0 tx=0x7f8d7c005ee0 comp rx=0 tx=0).stop 2024-03-21T10:23:42.228 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:42.226+0000 7f8d97200700 1 -- 172.21.15.153:0/313880222 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d90101da0 msgr2=0x7f8d901989a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:42.228 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:42.226+0000 7f8d97200700 1 --2- 172.21.15.153:0/313880222 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d90101da0 0x7f8d901989a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f8d8000bd70 tx=0x7f8d8000be50 comp rx=0 tx=0).stop 2024-03-21T10:23:42.228 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:42.226+0000 7f8d97200700 1 -- 172.21.15.153:0/313880222 shutdown_connections 2024-03-21T10:23:42.228 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:42.226+0000 7f8d97200700 1 --2- 172.21.15.153:0/313880222 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f8d7406e460 0x7f8d74070920 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:42.228 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:42.226+0000 7f8d97200700 1 --2- 172.21.15.153:0/313880222 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d90101da0 0x7f8d901989a0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:42.228 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:42.226+0000 7f8d97200700 1 --2- 172.21.15.153:0/313880222 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8d901026c0 0x7f8d90198ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:42.228 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:42.226+0000 7f8d97200700 1 -- 172.21.15.153:0/313880222 >> 172.21.15.153:0/313880222 conn(0x7f8d900fb570 msgr2=0x7f8d901001c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:23:42.228 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:42.226+0000 7f8d97200700 1 -- 172.21.15.153:0/313880222 shutdown_connections 2024-03-21T10:23:42.228 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:42.226+0000 7f8d97200700 1 -- 172.21.15.153:0/313880222 wait complete. 2024-03-21T10:23:42.373 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:42.372+0000 7f5b3a200700 1 -- 172.21.15.153:0/1896721418 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5b34068410 msgr2=0x7f5b340687f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:42.373 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:42.372+0000 7f5b3a200700 1 --2- 172.21.15.153:0/1896721418 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5b34068410 0x7f5b340687f0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f5b24009910 tx=0x7f5b24009c20 comp rx=0 tx=0).stop 2024-03-21T10:23:42.374 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:42.372+0000 7f5b3a200700 1 -- 172.21.15.153:0/1896721418 shutdown_connections 2024-03-21T10:23:42.374 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:42.372+0000 7f5b3a200700 1 --2- 172.21.15.153:0/1896721418 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b34068d30 0x7f5b3410d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:42.374 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:42.372+0000 7f5b3a200700 1 --2- 172.21.15.153:0/1896721418 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5b34068410 0x7f5b340687f0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:42.374 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:42.372+0000 7f5b3a200700 1 -- 172.21.15.153:0/1896721418 >> 172.21.15.153:0/1896721418 conn(0x7f5b340755a0 msgr2=0x7f5b340759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:23:42.374 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:42.372+0000 7f5b3a200700 1 -- 172.21.15.153:0/1896721418 shutdown_connections 2024-03-21T10:23:42.374 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:42.372+0000 7f5b3a200700 1 -- 172.21.15.153:0/1896721418 wait complete. 2024-03-21T10:23:42.375 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:42.373+0000 7f5b3a200700 1 Processor -- start 2024-03-21T10:23:42.375 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:42.373+0000 7f5b3a200700 1 -- start start 2024-03-21T10:23:42.375 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:42.374+0000 7f5b3a200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5b34068410 0x7f5b3419e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:42.375 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:42.374+0000 7f5b3a200700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b34068d30 0x7f5b3419ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:42.375 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:42.374+0000 7f5b3a200700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b3419f280 con 0x7f5b34068410 2024-03-21T10:23:42.376 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:42.374+0000 7f5b3a200700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b34198730 con 0x7f5b34068d30 2024-03-21T10:23:42.376 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:42.374+0000 7f5b33400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5b34068410 0x7f5b3419e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:42.376 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:42.374+0000 7f5b33400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5b34068410 0x7f5b3419e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:50816/0 (socket says 172.21.15.153:50816) 2024-03-21T10:23:42.376 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:42.374+0000 7f5b33400700 1 -- 172.21.15.153:0/1922629941 learned_addr learned my addr 172.21.15.153:0/1922629941 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:23:42.376 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:42.374+0000 7f5b32a00700 1 --2- 172.21.15.153:0/1922629941 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b34068d30 0x7f5b3419ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:42.376 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:42.375+0000 7f5b33400700 1 -- 172.21.15.153:0/1922629941 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b34068d30 msgr2=0x7f5b3419ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:42.376 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:42.375+0000 7f5b33400700 1 --2- 172.21.15.153:0/1922629941 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b34068d30 0x7f5b3419ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:42.376 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:42.375+0000 7f5b33400700 1 -- 172.21.15.153:0/1922629941 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b240095f0 con 0x7f5b34068410 2024-03-21T10:23:42.377 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:42.375+0000 7f5b33400700 1 --2- 172.21.15.153:0/1922629941 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5b34068410 0x7f5b3419e6b0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f5b24005f50 tx=0x7f5b2400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:23:42.377 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:42.375+0000 7f5b2be00700 1 -- 172.21.15.153:0/1922629941 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5b2401d070 con 0x7f5b34068410 2024-03-21T10:23:42.377 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:42.375+0000 7f5b2be00700 1 -- 172.21.15.153:0/1922629941 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5b2400f460 con 0x7f5b34068410 2024-03-21T10:23:42.377 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:42.375+0000 7f5b3a200700 1 -- 172.21.15.153:0/1922629941 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b34198a10 con 0x7f5b34068410 2024-03-21T10:23:42.377 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:42.375+0000 7f5b2be00700 1 -- 172.21.15.153:0/1922629941 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5b24017610 con 0x7f5b34068410 2024-03-21T10:23:42.377 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:42.375+0000 7f5b3a200700 1 -- 172.21.15.153:0/1922629941 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b34198f60 con 0x7f5b34068410 2024-03-21T10:23:42.378 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:42.376+0000 7f5b3a200700 1 -- 172.21.15.153:0/1922629941 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b3410a9b0 con 0x7f5b34068410 2024-03-21T10:23:42.378 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:42.377+0000 7f5b2be00700 1 -- 172.21.15.153:0/1922629941 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f5b2400f5d0 con 0x7f5b34068410 2024-03-21T10:23:42.379 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:42.378+0000 7f5b2be00700 1 --2- 172.21.15.153:0/1922629941 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f5b1806e460 0x7f5b18070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:42.379 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:42.378+0000 7f5b2be00700 1 -- 172.21.15.153:0/1922629941 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f5b2408fb40 con 0x7f5b34068410 2024-03-21T10:23:42.379 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:42.379+0000 7f5b32a00700 1 --2- 172.21.15.153:0/1922629941 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f5b1806e460 0x7f5b18070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:42.380 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:42.379+0000 7f5b32a00700 1 --2- 172.21.15.153:0/1922629941 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f5b1806e460 0x7f5b18070920 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5b20005fd0 tx=0x7f5b2000a560 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:23:42.383 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:42.383+0000 7f5b2be00700 1 -- 172.21.15.153:0/1922629941 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f5b2405c690 con 0x7f5b34068410 2024-03-21T10:23:42.699 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:42.698+0000 7f5b3a200700 1 -- 172.21.15.153:0/1922629941 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f5b34199180 con 0x7f5b34068410 2024-03-21T10:23:42.700 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:42.699+0000 7f5b2be00700 1 -- 172.21.15.153:0/1922629941 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+805 (secure 0 0 0) 0x7f5b2405c220 con 0x7f5b34068410 2024-03-21T10:23:42.700 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:23:42.700 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": { 2024-03-21T10:23:42.700 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:23:42.700 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:23:42.700 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": { 2024-03-21T10:23:42.701 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:23:42.701 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:23:42.701 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": { 2024-03-21T10:23:42.701 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 6 2024-03-21T10:23:42.701 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:23:42.701 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": { 2024-03-21T10:23:42.701 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-03-21T10:23:42.701 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 3 2024-03-21T10:23:42.701 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:23:42.701 INFO:teuthology.orchestra.run.smithi153.stdout: "overall": { 2024-03-21T10:23:42.701 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-03-21T10:23:42.701 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 13 2024-03-21T10:23:42.701 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-03-21T10:23:42.701 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:23:42.703 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:42.702+0000 7f5b3a200700 1 -- 172.21.15.153:0/1922629941 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f5b1806e460 msgr2=0x7f5b18070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:42.703 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:42.702+0000 7f5b3a200700 1 --2- 172.21.15.153:0/1922629941 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f5b1806e460 0x7f5b18070920 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5b20005fd0 tx=0x7f5b2000a560 comp rx=0 tx=0).stop 2024-03-21T10:23:42.703 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:42.702+0000 7f5b3a200700 1 -- 172.21.15.153:0/1922629941 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5b34068410 msgr2=0x7f5b3419e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:42.703 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:42.702+0000 7f5b3a200700 1 --2- 172.21.15.153:0/1922629941 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5b34068410 0x7f5b3419e6b0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f5b24005f50 tx=0x7f5b2400be20 comp rx=0 tx=0).stop 2024-03-21T10:23:42.704 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:42.702+0000 7f5b3a200700 1 -- 172.21.15.153:0/1922629941 shutdown_connections 2024-03-21T10:23:42.704 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:42.702+0000 7f5b3a200700 1 --2- 172.21.15.153:0/1922629941 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f5b1806e460 0x7f5b18070920 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:42.704 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:42.702+0000 7f5b3a200700 1 --2- 172.21.15.153:0/1922629941 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5b34068410 0x7f5b3419e6b0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:42.704 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:42.702+0000 7f5b3a200700 1 --2- 172.21.15.153:0/1922629941 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b34068d30 0x7f5b3419ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:42.704 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:42.702+0000 7f5b3a200700 1 -- 172.21.15.153:0/1922629941 >> 172.21.15.153:0/1922629941 conn(0x7f5b340755a0 msgr2=0x7f5b340fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:23:42.704 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:42.702+0000 7f5b3a200700 1 -- 172.21.15.153:0/1922629941 shutdown_connections 2024-03-21T10:23:42.704 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:42.703+0000 7f5b3a200700 1 -- 172.21.15.153:0/1922629941 wait complete. 2024-03-21T10:23:42.855 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:42.854+0000 7fda71a00700 1 -- 172.21.15.153:0/3565851144 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fda6c10d280 msgr2=0x7fda6c10d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:42.855 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:42.854+0000 7fda71a00700 1 --2- 172.21.15.153:0/3565851144 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fda6c10d280 0x7fda6c10d660 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fda5c00b0a0 tx=0x7fda5c00b3b0 comp rx=0 tx=0).stop 2024-03-21T10:23:42.855 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:42.854+0000 7fda71a00700 1 -- 172.21.15.153:0/3565851144 shutdown_connections 2024-03-21T10:23:42.855 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:42.854+0000 7fda71a00700 1 --2- 172.21.15.153:0/3565851144 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fda6c1071d0 0x7fda6c107650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:42.855 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:42.854+0000 7fda71a00700 1 --2- 172.21.15.153:0/3565851144 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fda6c10d280 0x7fda6c10d660 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:42.855 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:42.854+0000 7fda71a00700 1 -- 172.21.15.153:0/3565851144 >> 172.21.15.153:0/3565851144 conn(0x7fda6c0769e0 msgr2=0x7fda6c078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:23:42.855 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:42.854+0000 7fda71a00700 1 -- 172.21.15.153:0/3565851144 shutdown_connections 2024-03-21T10:23:42.856 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:42.854+0000 7fda71a00700 1 -- 172.21.15.153:0/3565851144 wait complete. 2024-03-21T10:23:42.857 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:42.855+0000 7fda71a00700 1 Processor -- start 2024-03-21T10:23:42.857 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:42.855+0000 7fda71a00700 1 -- start start 2024-03-21T10:23:42.858 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:42.855+0000 7fda71a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fda6c1071d0 0x7fda6c19cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:42.858 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:42.855+0000 7fda71a00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fda6c10d280 0x7fda6c19d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:42.858 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:42.855+0000 7fda71a00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda6c19d940 con 0x7fda6c1071d0 2024-03-21T10:23:42.858 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:42.855+0000 7fda71a00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda6c1a16d0 con 0x7fda6c10d280 2024-03-21T10:23:42.858 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:42.855+0000 7fda6aa00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fda6c1071d0 0x7fda6c19cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:42.858 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:42.856+0000 7fda6aa00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fda6c1071d0 0x7fda6c19cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:50838/0 (socket says 172.21.15.153:50838) 2024-03-21T10:23:42.858 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:42.856+0000 7fda6aa00700 1 -- 172.21.15.153:0/3035686121 learned_addr learned my addr 172.21.15.153:0/3035686121 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:23:42.858 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:42.856+0000 7fda6aa00700 1 -- 172.21.15.153:0/3035686121 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fda6c10d280 msgr2=0x7fda6c19d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:42.858 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:42.856+0000 7fda6aa00700 1 --2- 172.21.15.153:0/3035686121 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fda6c10d280 0x7fda6c19d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:42.858 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:42.856+0000 7fda6aa00700 1 -- 172.21.15.153:0/3035686121 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda5c009d00 con 0x7fda6c1071d0 2024-03-21T10:23:42.858 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:42.856+0000 7fda6aa00700 1 --2- 172.21.15.153:0/3035686121 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fda6c1071d0 0x7fda6c19cd20 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fda5c00b070 tx=0x7fda5c003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:23:42.858 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:42.856+0000 7fda63e00700 1 -- 172.21.15.153:0/3035686121 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fda5c00e040 con 0x7fda6c1071d0 2024-03-21T10:23:42.858 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:42.856+0000 7fda63e00700 1 -- 172.21.15.153:0/3035686121 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fda5c007c00 con 0x7fda6c1071d0 2024-03-21T10:23:42.859 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:42.856+0000 7fda71a00700 1 -- 172.21.15.153:0/3035686121 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda6c1a1950 con 0x7fda6c1071d0 2024-03-21T10:23:42.859 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:42.856+0000 7fda63e00700 1 -- 172.21.15.153:0/3035686121 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fda5c01d740 con 0x7fda6c1071d0 2024-03-21T10:23:42.859 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:42.857+0000 7fda71a00700 1 -- 172.21.15.153:0/3035686121 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda6c1a1ec0 con 0x7fda6c1071d0 2024-03-21T10:23:42.859 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:42.858+0000 7fda71a00700 1 -- 172.21.15.153:0/3035686121 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda6c04ea90 con 0x7fda6c1071d0 2024-03-21T10:23:42.859 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:42.859+0000 7fda63e00700 1 -- 172.21.15.153:0/3035686121 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fda5c004040 con 0x7fda6c1071d0 2024-03-21T10:23:42.860 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:42.860+0000 7fda63e00700 1 --2- 172.21.15.153:0/3035686121 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fda5006e460 0x7fda50070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:42.860 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:42.860+0000 7fda63e00700 1 -- 172.21.15.153:0/3035686121 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fda5c08fdd0 con 0x7fda6c1071d0 2024-03-21T10:23:42.860 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:42.860+0000 7fda62000700 1 --2- 172.21.15.153:0/3035686121 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fda5006e460 0x7fda50070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:42.861 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:42.861+0000 7fda62000700 1 --2- 172.21.15.153:0/3035686121 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fda5006e460 0x7fda50070920 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fda6c19e340 tx=0x7fda54005c80 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:23:42.865 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:42.864+0000 7fda63e00700 1 -- 172.21.15.153:0/3035686121 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fda5c058e90 con 0x7fda6c1071d0 2024-03-21T10:23:43.134 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:43.133+0000 7fda71a00700 1 -- 172.21.15.153:0/3035686121 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fda6c1a22e0 con 0x7fda6c1071d0 2024-03-21T10:23:43.135 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:43.134+0000 7fda63e00700 1 -- 172.21.15.153:0/3035686121 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1876 (secure 0 0 0) 0x7fda5c05c550 con 0x7fda6c1071d0 2024-03-21T10:23:43.135 INFO:teuthology.orchestra.run.smithi153.stdout:e30 2024-03-21T10:23:43.135 INFO:teuthology.orchestra.run.smithi153.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-21T10:23:43.135 INFO:teuthology.orchestra.run.smithi153.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:23:43.136 INFO:teuthology.orchestra.run.smithi153.stdout:legacy client fscid: 1 2024-03-21T10:23:43.136 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:23:43.136 INFO:teuthology.orchestra.run.smithi153.stdout:Filesystem 'cephfs' (1) 2024-03-21T10:23:43.136 INFO:teuthology.orchestra.run.smithi153.stdout:fs_name cephfs 2024-03-21T10:23:43.136 INFO:teuthology.orchestra.run.smithi153.stdout:epoch 23 2024-03-21T10:23:43.136 INFO:teuthology.orchestra.run.smithi153.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-21T10:23:43.136 INFO:teuthology.orchestra.run.smithi153.stdout:created 2024-03-21T10:04:27.086576+0000 2024-03-21T10:23:43.136 INFO:teuthology.orchestra.run.smithi153.stdout:modified 2024-03-21T10:19:05.022092+0000 2024-03-21T10:23:43.136 INFO:teuthology.orchestra.run.smithi153.stdout:tableserver 0 2024-03-21T10:23:43.136 INFO:teuthology.orchestra.run.smithi153.stdout:root 0 2024-03-21T10:23:43.136 INFO:teuthology.orchestra.run.smithi153.stdout:session_timeout 60 2024-03-21T10:23:43.136 INFO:teuthology.orchestra.run.smithi153.stdout:session_autoclose 300 2024-03-21T10:23:43.136 INFO:teuthology.orchestra.run.smithi153.stdout:max_file_size 1099511627776 2024-03-21T10:23:43.136 INFO:teuthology.orchestra.run.smithi153.stdout:max_xattr_size 65536 2024-03-21T10:23:43.137 INFO:teuthology.orchestra.run.smithi153.stdout:required_client_features {} 2024-03-21T10:23:43.137 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure 0 2024-03-21T10:23:43.137 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure_osd_epoch 94 2024-03-21T10:23:43.137 INFO:teuthology.orchestra.run.smithi153.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:23:43.137 INFO:teuthology.orchestra.run.smithi153.stdout:max_mds 2 2024-03-21T10:23:43.137 INFO:teuthology.orchestra.run.smithi153.stdout:in 0,1 2024-03-21T10:23:43.137 INFO:teuthology.orchestra.run.smithi153.stdout:up {} 2024-03-21T10:23:43.137 INFO:teuthology.orchestra.run.smithi153.stdout:failed 0,1 2024-03-21T10:23:43.137 INFO:teuthology.orchestra.run.smithi153.stdout:damaged 2024-03-21T10:23:43.137 INFO:teuthology.orchestra.run.smithi153.stdout:stopped 2024-03-21T10:23:43.137 INFO:teuthology.orchestra.run.smithi153.stdout:data_pools [3] 2024-03-21T10:23:43.137 INFO:teuthology.orchestra.run.smithi153.stdout:metadata_pool 2 2024-03-21T10:23:43.137 INFO:teuthology.orchestra.run.smithi153.stdout:inline_data enabled 2024-03-21T10:23:43.137 INFO:teuthology.orchestra.run.smithi153.stdout:balancer 2024-03-21T10:23:43.137 INFO:teuthology.orchestra.run.smithi153.stdout:bal_rank_mask -1 2024-03-21T10:23:43.137 INFO:teuthology.orchestra.run.smithi153.stdout:standby_count_wanted 1 2024-03-21T10:23:43.138 INFO:teuthology.orchestra.run.smithi153.stdout:qdb_cluster leader: 0 members: 2024-03-21T10:23:43.138 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:23:43.138 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:23:43.138 INFO:teuthology.orchestra.run.smithi153.stdout:Standby daemons: 2024-03-21T10:23:43.138 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:23:43.138 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.zipckr{-1:34442} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6826/1103846295,v1:172.21.15.153:6827/1103846295] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:23:43.138 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.iuyluj{-1:34474} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6828/1738630832,v1:172.21.15.153:6829/1738630832] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:23:43.138 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.nhozag{-1:44269} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.174:6826/3967223227,v1:172.21.15.174:6827/3967223227] compat {c=[1],r=[1],i=[7ff]}] 2024-03-21T10:23:43.138 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.yyimst{-1:44313} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.174:6824/4127786797,v1:172.21.15.174:6825/4127786797] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:23:43.139 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:43.138+0000 7fda71a00700 1 -- 172.21.15.153:0/3035686121 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fda5006e460 msgr2=0x7fda50070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:43.139 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:43.138+0000 7fda71a00700 1 --2- 172.21.15.153:0/3035686121 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fda5006e460 0x7fda50070920 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fda6c19e340 tx=0x7fda54005c80 comp rx=0 tx=0).stop 2024-03-21T10:23:43.139 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:43.138+0000 7fda71a00700 1 -- 172.21.15.153:0/3035686121 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fda6c1071d0 msgr2=0x7fda6c19cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:43.139 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:43.138+0000 7fda71a00700 1 --2- 172.21.15.153:0/3035686121 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fda6c1071d0 0x7fda6c19cd20 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fda5c00b070 tx=0x7fda5c003ce0 comp rx=0 tx=0).stop 2024-03-21T10:23:43.140 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:43.138+0000 7fda71a00700 1 -- 172.21.15.153:0/3035686121 shutdown_connections 2024-03-21T10:23:43.140 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:43.138+0000 7fda71a00700 1 --2- 172.21.15.153:0/3035686121 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fda5006e460 0x7fda50070920 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:43.140 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:43.138+0000 7fda71a00700 1 --2- 172.21.15.153:0/3035686121 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fda6c1071d0 0x7fda6c19cd20 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:43.140 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:43.138+0000 7fda71a00700 1 --2- 172.21.15.153:0/3035686121 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fda6c10d280 0x7fda6c19d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:43.140 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:43.138+0000 7fda71a00700 1 -- 172.21.15.153:0/3035686121 >> 172.21.15.153:0/3035686121 conn(0x7fda6c0769e0 msgr2=0x7fda6c0783c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:23:43.140 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:43.139+0000 7fda71a00700 1 -- 172.21.15.153:0/3035686121 shutdown_connections 2024-03-21T10:23:43.140 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:43.139+0000 7fda71a00700 1 -- 172.21.15.153:0/3035686121 wait complete. 2024-03-21T10:23:43.141 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 30 2024-03-21T10:23:43.291 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:43.290+0000 7f62f4000700 1 -- 172.21.15.153:0/28058927 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62ec1071d0 msgr2=0x7f62ec107650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:43.291 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:43.290+0000 7f62f4000700 1 --2- 172.21.15.153:0/28058927 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62ec1071d0 0x7f62ec107650 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f62d8009910 tx=0x7f62d8009c20 comp rx=0 tx=0).stop 2024-03-21T10:23:43.291 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:43.290+0000 7f62f4000700 1 -- 172.21.15.153:0/28058927 shutdown_connections 2024-03-21T10:23:43.292 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:43.290+0000 7f62f4000700 1 --2- 172.21.15.153:0/28058927 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62ec1071d0 0x7f62ec107650 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:43.292 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:43.290+0000 7f62f4000700 1 --2- 172.21.15.153:0/28058927 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f62ec10d280 0x7f62ec10d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:43.292 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:43.290+0000 7f62f4000700 1 -- 172.21.15.153:0/28058927 >> 172.21.15.153:0/28058927 conn(0x7f62ec0769e0 msgr2=0x7f62ec078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:23:43.292 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:43.290+0000 7f62f4000700 1 -- 172.21.15.153:0/28058927 shutdown_connections 2024-03-21T10:23:43.292 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:23:43 smithi153 ceph-mon[113733]: pgmap v355: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:23:43.292 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:23:43 smithi153 ceph-mon[113733]: from='client.34674 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:23:43.292 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:23:43 smithi153 ceph-mon[113733]: from='client.34678 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:23:43.293 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:23:43 smithi153 ceph-mon[113733]: from='client.34682 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:23:43.293 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:23:43 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/1922629941' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:23:43.294 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:43.290+0000 7f62f4000700 1 -- 172.21.15.153:0/28058927 wait complete. 2024-03-21T10:23:43.294 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:43.292+0000 7f62f4000700 1 Processor -- start 2024-03-21T10:23:43.294 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:43.292+0000 7f62f4000700 1 -- start start 2024-03-21T10:23:43.294 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:43.292+0000 7f62f4000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62ec1071d0 0x7f62ec19ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:43.294 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:43.292+0000 7f62f4000700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f62ec10d280 0x7f62ec19d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:43.294 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:43.292+0000 7f62f4000700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62ec19d880 con 0x7f62ec1071d0 2024-03-21T10:23:43.294 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:43.292+0000 7f62f4000700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62ec1a05b0 con 0x7f62ec10d280 2024-03-21T10:23:43.295 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:43.292+0000 7f62f1600700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62ec1071d0 0x7f62ec19ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:43.295 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:43.292+0000 7f62f1600700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62ec1071d0 0x7f62ec19ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:38410/0 (socket says 172.21.15.153:38410) 2024-03-21T10:23:43.295 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:43.292+0000 7f62f1600700 1 -- 172.21.15.153:0/770309997 learned_addr learned my addr 172.21.15.153:0/770309997 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:23:43.295 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:43.293+0000 7f62f0c00700 1 --2- 172.21.15.153:0/770309997 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f62ec10d280 0x7f62ec19d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:43.295 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:43.293+0000 7f62f1600700 1 -- 172.21.15.153:0/770309997 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f62ec10d280 msgr2=0x7f62ec19d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:43.295 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:43.293+0000 7f62f1600700 1 --2- 172.21.15.153:0/770309997 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f62ec10d280 0x7f62ec19d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:43.295 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:43.293+0000 7f62f1600700 1 -- 172.21.15.153:0/770309997 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62d80095f0 con 0x7f62ec1071d0 2024-03-21T10:23:43.295 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:43.293+0000 7f62f1600700 1 --2- 172.21.15.153:0/770309997 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62ec1071d0 0x7f62ec19ccb0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f62e000c8a0 tx=0x7f62e000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:23:43.296 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:43.294+0000 7f62ea000700 1 -- 172.21.15.153:0/770309997 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f62e000cea0 con 0x7f62ec1071d0 2024-03-21T10:23:43.296 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:43.294+0000 7f62ea000700 1 -- 172.21.15.153:0/770309997 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f62e0004830 con 0x7f62ec1071d0 2024-03-21T10:23:43.296 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:43.294+0000 7f62ea000700 1 -- 172.21.15.153:0/770309997 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f62e00055b0 con 0x7f62ec1071d0 2024-03-21T10:23:43.296 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:43.294+0000 7f62f4000700 1 -- 172.21.15.153:0/770309997 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62ec1a0890 con 0x7f62ec1071d0 2024-03-21T10:23:43.296 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:43.294+0000 7f62f4000700 1 -- 172.21.15.153:0/770309997 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62ec1a0e60 con 0x7f62ec1071d0 2024-03-21T10:23:43.296 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:43.295+0000 7f62f4000700 1 -- 172.21.15.153:0/770309997 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62ec10f6c0 con 0x7f62ec1071d0 2024-03-21T10:23:43.297 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:43.296+0000 7f62ea000700 1 -- 172.21.15.153:0/770309997 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f62e00049a0 con 0x7f62ec1071d0 2024-03-21T10:23:43.297 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:43.297+0000 7f62ea000700 1 --2- 172.21.15.153:0/770309997 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f62d406e460 0x7f62d4070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:43.298 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:43.297+0000 7f62ea000700 1 -- 172.21.15.153:0/770309997 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f62e008e270 con 0x7f62ec1071d0 2024-03-21T10:23:43.298 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:43.297+0000 7f62f0c00700 1 --2- 172.21.15.153:0/770309997 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f62d406e460 0x7f62d4070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:43.298 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:43.298+0000 7f62f0c00700 1 --2- 172.21.15.153:0/770309997 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f62d406e460 0x7f62d4070920 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f62ec19e290 tx=0x7f62d801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:23:43.302 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:43.302+0000 7f62ea000700 1 -- 172.21.15.153:0/770309997 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f62e0057290 con 0x7f62ec1071d0 2024-03-21T10:23:43.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:23:43 smithi174 ceph-mon[88004]: pgmap v355: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:23:43.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:23:43 smithi174 ceph-mon[88004]: from='client.34674 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:23:43.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:23:43 smithi174 ceph-mon[88004]: from='client.34678 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:23:43.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:23:43 smithi174 ceph-mon[88004]: from='client.34682 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:23:43.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:23:43 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/1922629941' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:23:43.529 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:43.528+0000 7f62f4000700 1 -- 172.21.15.153:0/770309997 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f62ec1a10e0 con 0x7f62d406e460 2024-03-21T10:23:43.530 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:43.530+0000 7f62ea000700 1 -- 172.21.15.153:0/770309997 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f62ec1a10e0 con 0x7f62d406e460 2024-03-21T10:23:43.531 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:23:43.531 INFO:teuthology.orchestra.run.smithi153.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232", 2024-03-21T10:23:43.531 INFO:teuthology.orchestra.run.smithi153.stdout: "in_progress": true, 2024-03-21T10:23:43.531 INFO:teuthology.orchestra.run.smithi153.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-21T10:23:43.531 INFO:teuthology.orchestra.run.smithi153.stdout: "services_complete": [ 2024-03-21T10:23:43.531 INFO:teuthology.orchestra.run.smithi153.stdout: "crash", 2024-03-21T10:23:43.531 INFO:teuthology.orchestra.run.smithi153.stdout: "mon", 2024-03-21T10:23:43.531 INFO:teuthology.orchestra.run.smithi153.stdout: "osd", 2024-03-21T10:23:43.531 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr" 2024-03-21T10:23:43.531 INFO:teuthology.orchestra.run.smithi153.stdout: ], 2024-03-21T10:23:43.532 INFO:teuthology.orchestra.run.smithi153.stdout: "progress": "15/23 daemons upgraded", 2024-03-21T10:23:43.532 INFO:teuthology.orchestra.run.smithi153.stdout: "message": "Currently upgrading mds daemons", 2024-03-21T10:23:43.532 INFO:teuthology.orchestra.run.smithi153.stdout: "is_paused": false 2024-03-21T10:23:43.532 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:23:43.533 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:43.532+0000 7f62f4000700 1 -- 172.21.15.153:0/770309997 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f62d406e460 msgr2=0x7f62d4070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:43.533 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:43.532+0000 7f62f4000700 1 --2- 172.21.15.153:0/770309997 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f62d406e460 0x7f62d4070920 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f62ec19e290 tx=0x7f62d801a040 comp rx=0 tx=0).stop 2024-03-21T10:23:43.533 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:43.532+0000 7f62f4000700 1 -- 172.21.15.153:0/770309997 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62ec1071d0 msgr2=0x7f62ec19ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:43.533 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:43.532+0000 7f62f4000700 1 --2- 172.21.15.153:0/770309997 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62ec1071d0 0x7f62ec19ccb0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f62e000c8a0 tx=0x7f62e000cc60 comp rx=0 tx=0).stop 2024-03-21T10:23:43.534 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:43.532+0000 7f62f4000700 1 -- 172.21.15.153:0/770309997 shutdown_connections 2024-03-21T10:23:43.534 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:43.532+0000 7f62f4000700 1 --2- 172.21.15.153:0/770309997 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f62d406e460 0x7f62d4070920 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:43.534 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:43.532+0000 7f62f4000700 1 --2- 172.21.15.153:0/770309997 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62ec1071d0 0x7f62ec19ccb0 secure :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f62e000c8a0 tx=0x7f62e000cc60 comp rx=0 tx=0).stop 2024-03-21T10:23:43.534 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:43.532+0000 7f62f4000700 1 --2- 172.21.15.153:0/770309997 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f62ec10d280 0x7f62ec19d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:43.534 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:43.532+0000 7f62f4000700 1 -- 172.21.15.153:0/770309997 >> 172.21.15.153:0/770309997 conn(0x7f62ec0769e0 msgr2=0x7f62ec078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:23:43.534 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:43.533+0000 7f62f4000700 1 -- 172.21.15.153:0/770309997 shutdown_connections 2024-03-21T10:23:43.534 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:43.533+0000 7f62f4000700 1 -- 172.21.15.153:0/770309997 wait complete. 2024-03-21T10:23:43.675 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:43.674+0000 7f6ab2400700 1 -- 172.21.15.153:0/820952854 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6aac10d280 msgr2=0x7f6aac10d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:43.676 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:43.674+0000 7f6ab2400700 1 --2- 172.21.15.153:0/820952854 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6aac10d280 0x7f6aac10d660 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c009910 tx=0x7f6a9c009c20 comp rx=0 tx=0).stop 2024-03-21T10:23:43.676 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:43.674+0000 7f6ab2400700 1 -- 172.21.15.153:0/820952854 shutdown_connections 2024-03-21T10:23:43.676 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:43.674+0000 7f6ab2400700 1 --2- 172.21.15.153:0/820952854 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6aac1071d0 0x7f6aac107650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:43.676 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:43.674+0000 7f6ab2400700 1 --2- 172.21.15.153:0/820952854 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6aac10d280 0x7f6aac10d660 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:43.676 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:43.674+0000 7f6ab2400700 1 -- 172.21.15.153:0/820952854 >> 172.21.15.153:0/820952854 conn(0x7f6aac0769e0 msgr2=0x7f6aac078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:23:43.676 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:43.674+0000 7f6ab2400700 1 -- 172.21.15.153:0/820952854 shutdown_connections 2024-03-21T10:23:43.676 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:43.674+0000 7f6ab2400700 1 -- 172.21.15.153:0/820952854 wait complete. 2024-03-21T10:23:43.676 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:43.675+0000 7f6ab2400700 1 Processor -- start 2024-03-21T10:23:43.676 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:43.675+0000 7f6ab2400700 1 -- start start 2024-03-21T10:23:43.678 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:43.675+0000 7f6ab2400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6aac1071d0 0x7f6aac19ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:43.678 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:43.675+0000 7f6ab2400700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6aac10d280 0x7f6aac19d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:43.678 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:43.675+0000 7f6ab2400700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aac19d8a0 con 0x7f6aac1071d0 2024-03-21T10:23:43.678 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:43.675+0000 7f6ab2400700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aac1a05d0 con 0x7f6aac10d280 2024-03-21T10:23:43.678 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:43.676+0000 7f6aab400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6aac1071d0 0x7f6aac19ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:43.678 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:43.676+0000 7f6aab400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6aac1071d0 0x7f6aac19ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:38434/0 (socket says 172.21.15.153:38434) 2024-03-21T10:23:43.678 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:43.676+0000 7f6aab400700 1 -- 172.21.15.153:0/874841268 learned_addr learned my addr 172.21.15.153:0/874841268 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:23:43.678 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:43.676+0000 7f6aab400700 1 -- 172.21.15.153:0/874841268 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6aac10d280 msgr2=0x7f6aac19d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:43.678 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:43.676+0000 7f6aab400700 1 --2- 172.21.15.153:0/874841268 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6aac10d280 0x7f6aac19d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:43.679 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:43.676+0000 7f6aab400700 1 -- 172.21.15.153:0/874841268 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a9c0095f0 con 0x7f6aac1071d0 2024-03-21T10:23:43.679 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:43.676+0000 7f6aab400700 1 --2- 172.21.15.153:0/874841268 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6aac1071d0 0x7f6aac19ccd0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c00b5c0 tx=0x7f6a9c00be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:23:43.679 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:43.677+0000 7f6aa3e00700 1 -- 172.21.15.153:0/874841268 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6a9c01d070 con 0x7f6aac1071d0 2024-03-21T10:23:43.679 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:43.677+0000 7f6aa3e00700 1 -- 172.21.15.153:0/874841268 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6a9c00f460 con 0x7f6aac1071d0 2024-03-21T10:23:43.679 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:43.677+0000 7f6ab2400700 1 -- 172.21.15.153:0/874841268 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6aac1a0850 con 0x7f6aac1071d0 2024-03-21T10:23:43.679 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:43.677+0000 7f6aa3e00700 1 -- 172.21.15.153:0/874841268 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6a9c017610 con 0x7f6aac1071d0 2024-03-21T10:23:43.679 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:43.677+0000 7f6ab2400700 1 -- 172.21.15.153:0/874841268 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6aac1a0dc0 con 0x7f6aac1071d0 2024-03-21T10:23:43.679 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:43.678+0000 7f6ab2400700 1 -- 172.21.15.153:0/874841268 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6aac04ea90 con 0x7f6aac1071d0 2024-03-21T10:23:43.680 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:43.679+0000 7f6aa3e00700 1 -- 172.21.15.153:0/874841268 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f6a9c00f5d0 con 0x7f6aac1071d0 2024-03-21T10:23:43.680 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:43.680+0000 7f6aa3e00700 1 --2- 172.21.15.153:0/874841268 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f6a9006e460 0x7f6a90070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:23:43.680 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:43.680+0000 7f6aa3e00700 1 -- 172.21.15.153:0/874841268 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f6a9c08fb10 con 0x7f6aac1071d0 2024-03-21T10:23:43.681 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:43.680+0000 7f6aaaa00700 1 --2- 172.21.15.153:0/874841268 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f6a9006e460 0x7f6a90070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:23:43.681 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:43.681+0000 7f6aaaa00700 1 --2- 172.21.15.153:0/874841268 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f6a9006e460 0x7f6a90070920 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6aac19e2b0 tx=0x7f6a98005c80 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:23:43.685 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:43.684+0000 7f6aa3e00700 1 -- 172.21.15.153:0/874841268 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f6a9c058bb0 con 0x7f6aac1071d0 2024-03-21T10:23:43.992 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:43.991+0000 7f6ab2400700 1 -- 172.21.15.153:0/874841268 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f6aac19dff0 con 0x7f6aac1071d0 2024-03-21T10:23:43.992 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:43.992+0000 7f6aa3e00700 1 -- 172.21.15.153:0/874841268 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f6a9c05c270 con 0x7f6aac1071d0 2024-03-21T10:23:43.993 INFO:teuthology.orchestra.run.smithi153.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-03-21T10:23:43.993 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-21T10:23:43.993 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is degraded 2024-03-21T10:23:43.993 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-21T10:23:43.993 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-21T10:23:43.993 INFO:teuthology.orchestra.run.smithi153.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-21T10:23:43.993 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-21T10:23:43.995 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:43.995+0000 7f6ab2400700 1 -- 172.21.15.153:0/874841268 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f6a9006e460 msgr2=0x7f6a90070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:43.995 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:43.995+0000 7f6ab2400700 1 --2- 172.21.15.153:0/874841268 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f6a9006e460 0x7f6a90070920 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6aac19e2b0 tx=0x7f6a98005c80 comp rx=0 tx=0).stop 2024-03-21T10:23:43.996 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:43.995+0000 7f6ab2400700 1 -- 172.21.15.153:0/874841268 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6aac1071d0 msgr2=0x7f6aac19ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:23:43.996 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:43.995+0000 7f6ab2400700 1 --2- 172.21.15.153:0/874841268 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6aac1071d0 0x7f6aac19ccd0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c00b5c0 tx=0x7f6a9c00be00 comp rx=0 tx=0).stop 2024-03-21T10:23:43.996 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:43.995+0000 7f6ab2400700 1 -- 172.21.15.153:0/874841268 shutdown_connections 2024-03-21T10:23:43.996 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:43.995+0000 7f6ab2400700 1 --2- 172.21.15.153:0/874841268 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f6a9006e460 0x7f6a90070920 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:43.997 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:43.995+0000 7f6ab2400700 1 --2- 172.21.15.153:0/874841268 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6aac1071d0 0x7f6aac19ccd0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:43.997 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:43.995+0000 7f6ab2400700 1 --2- 172.21.15.153:0/874841268 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6aac10d280 0x7f6aac19d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:23:43.997 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:43.995+0000 7f6ab2400700 1 -- 172.21.15.153:0/874841268 >> 172.21.15.153:0/874841268 conn(0x7f6aac0769e0 msgr2=0x7f6aac078320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:23:43.997 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:43.995+0000 7f6ab2400700 1 -- 172.21.15.153:0/874841268 shutdown_connections 2024-03-21T10:23:43.997 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:23:43.996+0000 7f6ab2400700 1 -- 172.21.15.153:0/874841268 wait complete. 2024-03-21T10:23:44.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:23:44 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/3035686121' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:23:44.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:23:44 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/874841268' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:23:44.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:23:44 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/3035686121' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:23:44.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:23:44 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/874841268' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:23:45.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:23:45 smithi153 ceph-mon[113733]: pgmap v356: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:23:45.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:23:45 smithi153 ceph-mon[113733]: from='client.34694 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:23:45.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:23:45 smithi174 ceph-mon[88004]: pgmap v356: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:23:45.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:23:45 smithi174 ceph-mon[88004]: from='client.34694 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:23:47.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:23:47 smithi153 ceph-mon[113733]: pgmap v357: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:23:47.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:23:47 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:23:47.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:23:47 smithi174 ceph-mon[88004]: pgmap v357: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:23:47.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:23:47 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:23:49.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:23:49 smithi153 ceph-mon[113733]: pgmap v358: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:23:49.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:23:49 smithi174 ceph-mon[88004]: pgmap v358: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:23:51.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:23:51 smithi153 ceph-mon[113733]: pgmap v359: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:23:51.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:23:51 smithi174 ceph-mon[88004]: pgmap v359: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:23:53.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:23:53 smithi153 ceph-mon[113733]: pgmap v360: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:23:53.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:23:53 smithi174 ceph-mon[88004]: pgmap v360: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:23:55.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:23:55 smithi153 ceph-mon[113733]: pgmap v361: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:23:55.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:23:55 smithi174 ceph-mon[88004]: pgmap v361: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:23:57.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:23:57 smithi153 ceph-mon[113733]: pgmap v362: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:23:57.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:23:57 smithi174 ceph-mon[88004]: pgmap v362: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:23:59.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:23:59 smithi174 ceph-mon[88004]: pgmap v363: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:23:59.558 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:23:59 smithi153 ceph-mon[113733]: pgmap v363: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:24:01.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:24:01 smithi174 ceph-mon[88004]: pgmap v364: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:24:01.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:24:01 smithi153 ceph-mon[113733]: pgmap v364: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:24:02.158 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:24:02 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:24:02.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:24:02 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:24:03.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:24:03 smithi174 ceph-mon[88004]: pgmap v365: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:24:03.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:24:03 smithi153 ceph-mon[113733]: pgmap v365: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:24:05.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:24:05 smithi174 ceph-mon[88004]: pgmap v366: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:24:05.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:24:05 smithi153 ceph-mon[113733]: pgmap v366: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:24:07.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:24:07 smithi174 ceph-mon[88004]: pgmap v367: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:24:07.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:24:07 smithi153 ceph-mon[113733]: pgmap v367: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:24:09.307 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:24:09 smithi153 ceph-mon[113733]: pgmap v368: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:24:09.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:24:09 smithi174 ceph-mon[88004]: pgmap v368: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:24:11.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:24:11 smithi174 ceph-mon[88004]: pgmap v369: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:24:11.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:24:11 smithi153 ceph-mon[113733]: pgmap v369: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:24:13.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:24:13 smithi174 ceph-mon[88004]: pgmap v370: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:24:13.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:24:13 smithi153 ceph-mon[113733]: pgmap v370: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:24:14.169 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:14.167+0000 7f5644400700 1 -- 172.21.15.153:0/1326309880 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f563c101750 msgr2=0x7f563c1057a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:14.170 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:14.167+0000 7f5644400700 1 --2- 172.21.15.153:0/1326309880 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f563c101750 0x7f563c1057a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f5628009910 tx=0x7f5628009c20 comp rx=0 tx=0).stop 2024-03-21T10:24:14.170 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:14.167+0000 7f5644400700 1 -- 172.21.15.153:0/1326309880 shutdown_connections 2024-03-21T10:24:14.170 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:14.167+0000 7f5644400700 1 --2- 172.21.15.153:0/1326309880 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f563c101750 0x7f563c1057a0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:14.170 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:14.167+0000 7f5644400700 1 --2- 172.21.15.153:0/1326309880 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f563c100da0 0x7f563c101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:14.170 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:14.167+0000 7f5644400700 1 -- 172.21.15.153:0/1326309880 >> 172.21.15.153:0/1326309880 conn(0x7f563c0fc630 msgr2=0x7f563c0fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:14.170 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:14.167+0000 7f5644400700 1 -- 172.21.15.153:0/1326309880 shutdown_connections 2024-03-21T10:24:14.170 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:14.167+0000 7f5644400700 1 -- 172.21.15.153:0/1326309880 wait complete. 2024-03-21T10:24:14.171 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:14.168+0000 7f5644400700 1 Processor -- start 2024-03-21T10:24:14.171 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:14.169+0000 7f5644400700 1 -- start start 2024-03-21T10:24:14.171 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:14.169+0000 7f5644400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f563c100da0 0x7f563c19e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:14.171 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:14.169+0000 7f5644400700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f563c101750 0x7f563c19ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:14.171 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:14.169+0000 7f5644400700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f563c19f420 con 0x7f563c100da0 2024-03-21T10:24:14.171 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:14.169+0000 7f5644400700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f563c198940 con 0x7f563c101750 2024-03-21T10:24:14.172 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:14.170+0000 7f5641a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f563c100da0 0x7f563c19e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:14.172 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:14.170+0000 7f5641a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f563c100da0 0x7f563c19e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:46498/0 (socket says 172.21.15.153:46498) 2024-03-21T10:24:14.172 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:14.170+0000 7f5641a00700 1 -- 172.21.15.153:0/4072052731 learned_addr learned my addr 172.21.15.153:0/4072052731 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:24:14.172 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:14.170+0000 7f5641000700 1 --2- 172.21.15.153:0/4072052731 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f563c101750 0x7f563c19ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:14.172 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:14.170+0000 7f5641a00700 1 -- 172.21.15.153:0/4072052731 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f563c101750 msgr2=0x7f563c19ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:14.172 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:14.170+0000 7f5641a00700 1 --2- 172.21.15.153:0/4072052731 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f563c101750 0x7f563c19ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:14.173 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:14.170+0000 7f5641a00700 1 -- 172.21.15.153:0/4072052731 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56280095f0 con 0x7f563c100da0 2024-03-21T10:24:14.173 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:14.171+0000 7f5641a00700 1 --2- 172.21.15.153:0/4072052731 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f563c100da0 0x7f563c19e8c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f563000b4b0 tx=0x7f563000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:14.173 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:14.171+0000 7f563a000700 1 -- 172.21.15.153:0/4072052731 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5630011800 con 0x7f563c100da0 2024-03-21T10:24:14.173 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:14.171+0000 7f563a000700 1 -- 172.21.15.153:0/4072052731 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5630011e20 con 0x7f563c100da0 2024-03-21T10:24:14.173 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:14.171+0000 7f563a000700 1 -- 172.21.15.153:0/4072052731 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f563000f2e0 con 0x7f563c100da0 2024-03-21T10:24:14.173 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:14.171+0000 7f5644400700 1 -- 172.21.15.153:0/4072052731 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f563c198c20 con 0x7f563c100da0 2024-03-21T10:24:14.174 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:14.171+0000 7f5644400700 1 -- 172.21.15.153:0/4072052731 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f563c199170 con 0x7f563c100da0 2024-03-21T10:24:14.174 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:14.172+0000 7f5644400700 1 -- 172.21.15.153:0/4072052731 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f563c104f20 con 0x7f563c100da0 2024-03-21T10:24:14.175 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:14.174+0000 7f563a000700 1 -- 172.21.15.153:0/4072052731 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f5630010380 con 0x7f563c100da0 2024-03-21T10:24:14.176 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:14.175+0000 7f563a000700 1 --2- 172.21.15.153:0/4072052731 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f562406e460 0x7f5624070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:14.176 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:14.175+0000 7f563a000700 1 -- 172.21.15.153:0/4072052731 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f563008e020 con 0x7f563c100da0 2024-03-21T10:24:14.177 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:14.175+0000 7f5641000700 1 --2- 172.21.15.153:0/4072052731 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f562406e460 0x7f5624070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:14.177 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:14.176+0000 7f5641000700 1 --2- 172.21.15.153:0/4072052731 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f562406e460 0x7f5624070920 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f563c19a0a0 tx=0x7f562801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:14.181 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:14.180+0000 7f563a000700 1 -- 172.21.15.153:0/4072052731 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f563005ad40 con 0x7f563c100da0 2024-03-21T10:24:14.428 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:14.426+0000 7f5644400700 1 -- 172.21.15.153:0/4072052731 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f563c199500 con 0x7f562406e460 2024-03-21T10:24:14.429 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:14.428+0000 7f563a000700 1 -- 172.21.15.153:0/4072052731 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f563c199500 con 0x7f562406e460 2024-03-21T10:24:14.432 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:14.430+0000 7f5644400700 1 -- 172.21.15.153:0/4072052731 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f562406e460 msgr2=0x7f5624070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:14.433 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:14.430+0000 7f5644400700 1 --2- 172.21.15.153:0/4072052731 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f562406e460 0x7f5624070920 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f563c19a0a0 tx=0x7f562801a040 comp rx=0 tx=0).stop 2024-03-21T10:24:14.433 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:14.430+0000 7f5644400700 1 -- 172.21.15.153:0/4072052731 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f563c100da0 msgr2=0x7f563c19e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:14.433 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:14.430+0000 7f5644400700 1 --2- 172.21.15.153:0/4072052731 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f563c100da0 0x7f563c19e8c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f563000b4b0 tx=0x7f563000b870 comp rx=0 tx=0).stop 2024-03-21T10:24:14.433 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:14.431+0000 7f5644400700 1 -- 172.21.15.153:0/4072052731 shutdown_connections 2024-03-21T10:24:14.433 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:14.431+0000 7f5644400700 1 --2- 172.21.15.153:0/4072052731 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f562406e460 0x7f5624070920 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:14.433 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:14.431+0000 7f5644400700 1 --2- 172.21.15.153:0/4072052731 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f563c100da0 0x7f563c19e8c0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:14.433 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:14.431+0000 7f5644400700 1 --2- 172.21.15.153:0/4072052731 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f563c101750 0x7f563c19ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:14.433 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:14.431+0000 7f5644400700 1 -- 172.21.15.153:0/4072052731 >> 172.21.15.153:0/4072052731 conn(0x7f563c0fc630 msgr2=0x7f563c0fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:14.433 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:14.431+0000 7f5644400700 1 -- 172.21.15.153:0/4072052731 shutdown_connections 2024-03-21T10:24:14.433 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:14.431+0000 7f5644400700 1 -- 172.21.15.153:0/4072052731 wait complete. 2024-03-21T10:24:14.449 INFO:teuthology.orchestra.run.smithi153.stdout:true 2024-03-21T10:24:14.591 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:14.588+0000 7ff4db200700 1 -- 172.21.15.153:0/2097273444 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff4dc1037c0 msgr2=0x7ff4dc107810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:14.591 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:14.588+0000 7ff4db200700 1 --2- 172.21.15.153:0/2097273444 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff4dc1037c0 0x7ff4dc107810 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7ff4c8009910 tx=0x7ff4c8009c20 comp rx=0 tx=0).stop 2024-03-21T10:24:14.591 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:14.589+0000 7ff4db200700 1 -- 172.21.15.153:0/2097273444 shutdown_connections 2024-03-21T10:24:14.591 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:14.589+0000 7ff4db200700 1 --2- 172.21.15.153:0/2097273444 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff4dc1037c0 0x7ff4dc107810 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:14.591 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:14.589+0000 7ff4db200700 1 --2- 172.21.15.153:0/2097273444 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff4dc102e10 0x7ff4dc1031f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:14.591 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:14.589+0000 7ff4db200700 1 -- 172.21.15.153:0/2097273444 >> 172.21.15.153:0/2097273444 conn(0x7ff4dc0fe6c0 msgr2=0x7ff4dc100ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:14.591 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:14.589+0000 7ff4db200700 1 -- 172.21.15.153:0/2097273444 shutdown_connections 2024-03-21T10:24:14.591 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:14.589+0000 7ff4db200700 1 -- 172.21.15.153:0/2097273444 wait complete. 2024-03-21T10:24:14.593 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:14.590+0000 7ff4db200700 1 Processor -- start 2024-03-21T10:24:14.593 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:14.590+0000 7ff4db200700 1 -- start start 2024-03-21T10:24:14.593 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:14.590+0000 7ff4db200700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff4dc102e10 0x7ff4dc19e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:14.593 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:14.590+0000 7ff4db200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff4dc1037c0 0x7ff4dc19ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:14.593 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:14.590+0000 7ff4db200700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4dc19f260 con 0x7ff4dc1037c0 2024-03-21T10:24:14.593 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:14.590+0000 7ff4db200700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4dc198710 con 0x7ff4dc102e10 2024-03-21T10:24:14.593 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:14.591+0000 7ff4d9400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff4dc1037c0 0x7ff4dc19ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:14.593 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:14.591+0000 7ff4d9400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff4dc1037c0 0x7ff4dc19ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:46520/0 (socket says 172.21.15.153:46520) 2024-03-21T10:24:14.593 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:14.591+0000 7ff4d9400700 1 -- 172.21.15.153:0/1827669747 learned_addr learned my addr 172.21.15.153:0/1827669747 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:24:14.593 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:14.591+0000 7ff4d9e00700 1 --2- 172.21.15.153:0/1827669747 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff4dc102e10 0x7ff4dc19e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:14.593 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:14.591+0000 7ff4d9400700 1 -- 172.21.15.153:0/1827669747 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff4dc102e10 msgr2=0x7ff4dc19e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:14.594 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:14.591+0000 7ff4d9400700 1 --2- 172.21.15.153:0/1827669747 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff4dc102e10 0x7ff4dc19e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:14.594 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:14.591+0000 7ff4d9400700 1 -- 172.21.15.153:0/1827669747 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4c80095f0 con 0x7ff4dc1037c0 2024-03-21T10:24:14.594 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:14.591+0000 7ff4d9e00700 1 --2- 172.21.15.153:0/1827669747 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff4dc102e10 0x7ff4dc19e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:24:14.595 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:14.591+0000 7ff4d9400700 1 --2- 172.21.15.153:0/1827669747 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff4dc1037c0 0x7ff4dc19ebd0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7ff4c800bda0 tx=0x7ff4c800be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:14.595 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:14.592+0000 7ff4d2a00700 1 -- 172.21.15.153:0/1827669747 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff4c801d070 con 0x7ff4dc1037c0 2024-03-21T10:24:14.595 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:14.592+0000 7ff4d2a00700 1 -- 172.21.15.153:0/1827669747 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff4c8004fb0 con 0x7ff4dc1037c0 2024-03-21T10:24:14.595 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:14.592+0000 7ff4db200700 1 -- 172.21.15.153:0/1827669747 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4dc198990 con 0x7ff4dc1037c0 2024-03-21T10:24:14.595 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:14.592+0000 7ff4d2a00700 1 -- 172.21.15.153:0/1827669747 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff4c800f460 con 0x7ff4dc1037c0 2024-03-21T10:24:14.595 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:14.592+0000 7ff4db200700 1 -- 172.21.15.153:0/1827669747 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4dc198e80 con 0x7ff4dc1037c0 2024-03-21T10:24:14.595 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:14.593+0000 7ff4db200700 1 -- 172.21.15.153:0/1827669747 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4dc10b160 con 0x7ff4dc1037c0 2024-03-21T10:24:14.595 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:14.594+0000 7ff4d2a00700 1 -- 172.21.15.153:0/1827669747 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7ff4c8022800 con 0x7ff4dc1037c0 2024-03-21T10:24:14.597 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:14.595+0000 7ff4d2a00700 1 --2- 172.21.15.153:0/1827669747 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff4c006e460 0x7ff4c0070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:14.597 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:14.595+0000 7ff4d2a00700 1 -- 172.21.15.153:0/1827669747 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7ff4c808faf0 con 0x7ff4dc1037c0 2024-03-21T10:24:14.597 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:14.595+0000 7ff4d9e00700 1 --2- 172.21.15.153:0/1827669747 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff4c006e460 0x7ff4c0070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:14.597 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:14.596+0000 7ff4d9e00700 1 --2- 172.21.15.153:0/1827669747 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff4c006e460 0x7ff4c0070920 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc007900 tx=0x7ff4cc008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:14.601 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:14.600+0000 7ff4d2a00700 1 -- 172.21.15.153:0/1827669747 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff4c805c810 con 0x7ff4dc1037c0 2024-03-21T10:24:14.843 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:14.841+0000 7ff4db200700 1 -- 172.21.15.153:0/1827669747 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff4dc02cd50 con 0x7ff4c006e460 2024-03-21T10:24:14.846 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:14.844+0000 7ff4d2a00700 1 -- 172.21.15.153:0/1827669747 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7ff4dc02cd50 con 0x7ff4c006e460 2024-03-21T10:24:14.849 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:14.847+0000 7ff4db200700 1 -- 172.21.15.153:0/1827669747 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff4c006e460 msgr2=0x7ff4c0070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:14.849 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:14.847+0000 7ff4db200700 1 --2- 172.21.15.153:0/1827669747 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff4c006e460 0x7ff4c0070920 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc007900 tx=0x7ff4cc008040 comp rx=0 tx=0).stop 2024-03-21T10:24:14.849 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:14.847+0000 7ff4db200700 1 -- 172.21.15.153:0/1827669747 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff4dc1037c0 msgr2=0x7ff4dc19ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:14.849 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:14.847+0000 7ff4db200700 1 --2- 172.21.15.153:0/1827669747 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff4dc1037c0 0x7ff4dc19ebd0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7ff4c800bda0 tx=0x7ff4c800be80 comp rx=0 tx=0).stop 2024-03-21T10:24:14.849 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:14.847+0000 7ff4db200700 1 -- 172.21.15.153:0/1827669747 shutdown_connections 2024-03-21T10:24:14.849 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:14.847+0000 7ff4db200700 1 --2- 172.21.15.153:0/1827669747 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff4dc102e10 0x7ff4dc19e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:14.849 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:14.847+0000 7ff4db200700 1 --2- 172.21.15.153:0/1827669747 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff4c006e460 0x7ff4c0070920 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:14.849 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:14.847+0000 7ff4db200700 1 --2- 172.21.15.153:0/1827669747 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff4dc1037c0 0x7ff4dc19ebd0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:14.849 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:14.847+0000 7ff4db200700 1 -- 172.21.15.153:0/1827669747 >> 172.21.15.153:0/1827669747 conn(0x7ff4dc0fe6c0 msgr2=0x7ff4dc0ffea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:14.849 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:14.847+0000 7ff4db200700 1 -- 172.21.15.153:0/1827669747 shutdown_connections 2024-03-21T10:24:14.850 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:14.847+0000 7ff4db200700 1 -- 172.21.15.153:0/1827669747 wait complete. 2024-03-21T10:24:15.003 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.001+0000 7f7ea7200700 1 -- 172.21.15.153:0/2317039222 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ea81009c0 msgr2=0x7f7ea8100e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:15.003 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.001+0000 7f7ea7200700 1 --2- 172.21.15.153:0/2317039222 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ea81009c0 0x7f7ea8100e40 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f7e94009910 tx=0x7f7e94009c20 comp rx=0 tx=0).stop 2024-03-21T10:24:15.003 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.001+0000 7f7ea7200700 1 -- 172.21.15.153:0/2317039222 shutdown_connections 2024-03-21T10:24:15.003 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.001+0000 7f7ea7200700 1 --2- 172.21.15.153:0/2317039222 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ea81009c0 0x7f7ea8100e40 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:15.003 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.001+0000 7f7ea7200700 1 --2- 172.21.15.153:0/2317039222 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7ea81069c0 0x7f7ea8106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:15.004 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.001+0000 7f7ea7200700 1 -- 172.21.15.153:0/2317039222 >> 172.21.15.153:0/2317039222 conn(0x7f7ea80fc4f0 msgr2=0x7f7ea80fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:15.004 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.001+0000 7f7ea7200700 1 -- 172.21.15.153:0/2317039222 shutdown_connections 2024-03-21T10:24:15.004 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.001+0000 7f7ea7200700 1 -- 172.21.15.153:0/2317039222 wait complete. 2024-03-21T10:24:15.005 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.002+0000 7f7ea7200700 1 Processor -- start 2024-03-21T10:24:15.005 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.002+0000 7f7ea7200700 1 -- start start 2024-03-21T10:24:15.005 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.003+0000 7f7ea7200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ea81009c0 0x7f7ea8073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:15.005 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.003+0000 7f7ea7200700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7ea81069c0 0x7f7ea80735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:15.005 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.003+0000 7f7ea7200700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea8073b10 con 0x7f7ea81009c0 2024-03-21T10:24:15.005 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.003+0000 7f7ea7200700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea8073c50 con 0x7f7ea81069c0 2024-03-21T10:24:15.005 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.003+0000 7f7ea5e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ea81009c0 0x7f7ea8073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:15.005 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.003+0000 7f7ea5e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ea81009c0 0x7f7ea8073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:46544/0 (socket says 172.21.15.153:46544) 2024-03-21T10:24:15.005 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.003+0000 7f7ea5e00700 1 -- 172.21.15.153:0/739179310 learned_addr learned my addr 172.21.15.153:0/739179310 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:24:15.006 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.004+0000 7f7ea5400700 1 --2- 172.21.15.153:0/739179310 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7ea81069c0 0x7f7ea80735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:15.007 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.004+0000 7f7ea5e00700 1 -- 172.21.15.153:0/739179310 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7ea81069c0 msgr2=0x7f7ea80735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:15.007 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.004+0000 7f7ea5e00700 1 --2- 172.21.15.153:0/739179310 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7ea81069c0 0x7f7ea80735d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:15.007 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.004+0000 7f7ea5e00700 1 -- 172.21.15.153:0/739179310 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e940095f0 con 0x7f7ea81009c0 2024-03-21T10:24:15.007 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.004+0000 7f7ea5400700 1 --2- 172.21.15.153:0/739179310 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7ea81069c0 0x7f7ea80735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:24:15.007 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.004+0000 7f7ea5e00700 1 --2- 172.21.15.153:0/739179310 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ea81009c0 0x7f7ea8073090 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f7e9800d8d0 tx=0x7f7e9800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:15.007 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.004+0000 7f7e9ea00700 1 -- 172.21.15.153:0/739179310 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7e98009940 con 0x7f7ea81009c0 2024-03-21T10:24:15.007 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.004+0000 7f7e9ea00700 1 -- 172.21.15.153:0/739179310 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7e98010460 con 0x7f7ea81009c0 2024-03-21T10:24:15.007 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.004+0000 7f7e9ea00700 1 -- 172.21.15.153:0/739179310 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7e9800f5f0 con 0x7f7ea81009c0 2024-03-21T10:24:15.007 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.004+0000 7f7ea7200700 1 -- 172.21.15.153:0/739179310 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ea81a2770 con 0x7f7ea81009c0 2024-03-21T10:24:15.007 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.004+0000 7f7ea7200700 1 -- 172.21.15.153:0/739179310 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ea81a2b80 con 0x7f7ea81009c0 2024-03-21T10:24:15.008 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.006+0000 7f7ea7200700 1 -- 172.21.15.153:0/739179310 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ea8108f50 con 0x7f7ea81009c0 2024-03-21T10:24:15.008 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.006+0000 7f7e9ea00700 1 -- 172.21.15.153:0/739179310 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f7e980105d0 con 0x7f7ea81009c0 2024-03-21T10:24:15.009 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.007+0000 7f7e9ea00700 1 --2- 172.21.15.153:0/739179310 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f7e8c06e460 0x7f7e8c070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:15.009 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.007+0000 7f7e9ea00700 1 -- 172.21.15.153:0/739179310 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f7e9808e480 con 0x7f7ea81009c0 2024-03-21T10:24:15.009 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.008+0000 7f7ea5400700 1 --2- 172.21.15.153:0/739179310 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f7e8c06e460 0x7f7e8c070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:15.009 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.008+0000 7f7ea5400700 1 --2- 172.21.15.153:0/739179310 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f7e8c06e460 0x7f7e8c070920 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7e9400b5c0 tx=0x7f7e940053a0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:15.013 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.012+0000 7f7e9ea00700 1 -- 172.21.15.153:0/739179310 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7e98057520 con 0x7f7ea81009c0 2024-03-21T10:24:15.246 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.244+0000 7f7ea7200700 1 -- 172.21.15.153:0/739179310 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f7ea802cdd0 con 0x7f7e8c06e460 2024-03-21T10:24:15.263 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.261+0000 7f7e9ea00700 1 -- 172.21.15.153:0/739179310 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f7ea802cdd0 con 0x7f7e8c06e460 2024-03-21T10:24:15.263 INFO:teuthology.orchestra.run.smithi153.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-21T10:24:15.263 INFO:teuthology.orchestra.run.smithi153.stdout:alertmanager.smithi153 smithi153 *:9093,9094 running (13m) 4m ago 26m 14.7M - 0.25.0 c8568f914cd2 9ed3f2200b5f 2024-03-21T10:24:15.263 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi153 smithi153 running (27m) 4m ago 27m 9424k - 18.2.1 5be31c24972a 5b61994396a6 2024-03-21T10:24:15.264 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi174 smithi174 running (25m) 4m ago 25m 9244k - 18.2.1 5be31c24972a 4f7a5fc3ac28 2024-03-21T10:24:15.264 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi153 smithi153 running (9m) 4m ago 27m 6656k - 19.0.0-2231-g2edfee68 237e43c7a65f 32c21104d5b0 2024-03-21T10:24:15.264 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi174 smithi174 running (9m) 4m ago 25m 6660k - 19.0.0-2231-g2edfee68 237e43c7a65f a04e48967ab6 2024-03-21T10:24:15.264 INFO:teuthology.orchestra.run.smithi153.stdout:grafana.smithi153 smithi153 *:3000 running (12m) 4m ago 26m 73.6M - 9.4.12 2bacad6d85d8 ee24026ab786 2024-03-21T10:24:15.264 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.iuyluj smithi153 running (4m) 4m ago 19m 10.9M - 19.0.0-2231-g2edfee68 237e43c7a65f 87aff8a5de70 2024-03-21T10:24:15.264 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.zipckr smithi153 running (5m) 4m ago 19m 11.8M - 19.0.0-2231-g2edfee68 237e43c7a65f ccc262bd5dd3 2024-03-21T10:24:15.264 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.nhozag smithi174 running (19m) 4m ago 19m 12.0M - 18.2.1 5be31c24972a 2fee4000308f 2024-03-21T10:24:15.264 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.yyimst smithi174 running (4m) 4m ago 19m 10.7M - 19.0.0-2231-g2edfee68 237e43c7a65f fc1962f9351e 2024-03-21T10:24:15.264 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi153.tluxni smithi153 *:8443,9283,8765 running (16m) 4m ago 28m 509M - 19.0.0-2231-g2edfee68 237e43c7a65f 7ffee0e55382 2024-03-21T10:24:15.264 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi174.qpcwat smithi174 *:8443,9283,8765 running (14m) 4m ago 25m 452M - 19.0.0-2231-g2edfee68 237e43c7a65f 748968f216ab 2024-03-21T10:24:15.264 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi153 smithi153 running (10m) 4m ago 28m 55.0M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f dd8038303dc3 2024-03-21T10:24:15.264 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi174 smithi174 running (10m) 4m ago 25m 44.2M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f 29d78a788f2c 2024-03-21T10:24:15.264 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi153 smithi153 *:9100 running (14m) 4m ago 26m 7889k - 1.5.0 0da6a335fe13 bb835edc0e0b 2024-03-21T10:24:15.264 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi174 smithi174 *:9100 running (14m) 4m ago 25m 8102k - 1.5.0 0da6a335fe13 7c4ecad790fa 2024-03-21T10:24:15.265 INFO:teuthology.orchestra.run.smithi153.stdout:osd.0 smithi153 running (8m) 4m ago 24m 116M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 507900fd4791 2024-03-21T10:24:15.265 INFO:teuthology.orchestra.run.smithi153.stdout:osd.1 smithi153 running (7m) 4m ago 23m 89.2M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 0d646452e9c1 2024-03-21T10:24:15.265 INFO:teuthology.orchestra.run.smithi153.stdout:osd.2 smithi153 running (7m) 4m ago 23m 87.4M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 1326f2afdfcd 2024-03-21T10:24:15.265 INFO:teuthology.orchestra.run.smithi153.stdout:osd.3 smithi174 running (6m) 4m ago 22m 83.6M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 70aafac71ad5 2024-03-21T10:24:15.265 INFO:teuthology.orchestra.run.smithi153.stdout:osd.4 smithi174 running (6m) 4m ago 21m 92.2M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 0d746ce6e79d 2024-03-21T10:24:15.265 INFO:teuthology.orchestra.run.smithi153.stdout:osd.5 smithi174 running (5m) 4m ago 21m 86.2M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 53e5f35b8cac 2024-03-21T10:24:15.265 INFO:teuthology.orchestra.run.smithi153.stdout:prometheus.smithi153 smithi153 *:9095 running (13m) 4m ago 26m 73.5M - 2.43.0 a07b618ecd1d 7d95561e50c2 2024-03-21T10:24:15.266 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.264+0000 7f7ea7200700 1 -- 172.21.15.153:0/739179310 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f7e8c06e460 msgr2=0x7f7e8c070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:15.266 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.264+0000 7f7ea7200700 1 --2- 172.21.15.153:0/739179310 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f7e8c06e460 0x7f7e8c070920 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7e9400b5c0 tx=0x7f7e940053a0 comp rx=0 tx=0).stop 2024-03-21T10:24:15.266 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.264+0000 7f7ea7200700 1 -- 172.21.15.153:0/739179310 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ea81009c0 msgr2=0x7f7ea8073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:15.267 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.264+0000 7f7ea7200700 1 --2- 172.21.15.153:0/739179310 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ea81009c0 0x7f7ea8073090 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f7e9800d8d0 tx=0x7f7e9800dc90 comp rx=0 tx=0).stop 2024-03-21T10:24:15.267 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.264+0000 7f7ea7200700 1 -- 172.21.15.153:0/739179310 shutdown_connections 2024-03-21T10:24:15.267 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.264+0000 7f7ea7200700 1 --2- 172.21.15.153:0/739179310 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f7e8c06e460 0x7f7e8c070920 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:15.267 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.264+0000 7f7ea7200700 1 --2- 172.21.15.153:0/739179310 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ea81009c0 0x7f7ea8073090 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:15.267 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.264+0000 7f7ea7200700 1 --2- 172.21.15.153:0/739179310 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7ea81069c0 0x7f7ea80735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:15.267 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.264+0000 7f7ea7200700 1 -- 172.21.15.153:0/739179310 >> 172.21.15.153:0/739179310 conn(0x7f7ea80fc4f0 msgr2=0x7f7ea80fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:15.267 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.264+0000 7f7ea7200700 1 -- 172.21.15.153:0/739179310 shutdown_connections 2024-03-21T10:24:15.267 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.264+0000 7f7ea7200700 1 -- 172.21.15.153:0/739179310 wait complete. 2024-03-21T10:24:15.420 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.418+0000 7f8625400700 1 -- 172.21.15.153:0/718216443 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f86201039b0 msgr2=0x7f8620107a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:15.420 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.418+0000 7f8625400700 1 --2- 172.21.15.153:0/718216443 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f86201039b0 0x7f8620107a00 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f860c009910 tx=0x7f860c009c20 comp rx=0 tx=0).stop 2024-03-21T10:24:15.421 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.418+0000 7f8625400700 1 -- 172.21.15.153:0/718216443 shutdown_connections 2024-03-21T10:24:15.421 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.418+0000 7f8625400700 1 --2- 172.21.15.153:0/718216443 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f86201039b0 0x7f8620107a00 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:15.421 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.418+0000 7f8625400700 1 --2- 172.21.15.153:0/718216443 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8620103000 0x7f86201033e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:15.421 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.418+0000 7f8625400700 1 -- 172.21.15.153:0/718216443 >> 172.21.15.153:0/718216443 conn(0x7f86200fe8d0 msgr2=0x7f8620100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:15.421 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:24:15 smithi153 ceph-mon[113733]: pgmap v371: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:24:15.422 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.419+0000 7f8625400700 1 -- 172.21.15.153:0/718216443 shutdown_connections 2024-03-21T10:24:15.422 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.419+0000 7f8625400700 1 -- 172.21.15.153:0/718216443 wait complete. 2024-03-21T10:24:15.422 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.420+0000 7f8625400700 1 Processor -- start 2024-03-21T10:24:15.422 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.420+0000 7f8625400700 1 -- start start 2024-03-21T10:24:15.423 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.420+0000 7f8625400700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8620103000 0x7f862019e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:15.423 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.420+0000 7f8625400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f86201039b0 0x7f862019ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:15.423 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.420+0000 7f8625400700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f862019f390 con 0x7f86201039b0 2024-03-21T10:24:15.423 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.420+0000 7f8625400700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8620198840 con 0x7f8620103000 2024-03-21T10:24:15.423 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.421+0000 7f861de00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f86201039b0 0x7f862019ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:15.423 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.421+0000 7f861de00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f86201039b0 0x7f862019ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:46566/0 (socket says 172.21.15.153:46566) 2024-03-21T10:24:15.423 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.421+0000 7f861de00700 1 -- 172.21.15.153:0/660512452 learned_addr learned my addr 172.21.15.153:0/660512452 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:24:15.424 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.421+0000 7f861e800700 1 --2- 172.21.15.153:0/660512452 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8620103000 0x7f862019e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:15.424 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.421+0000 7f861de00700 1 -- 172.21.15.153:0/660512452 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8620103000 msgr2=0x7f862019e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:15.424 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.421+0000 7f861de00700 1 --2- 172.21.15.153:0/660512452 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8620103000 0x7f862019e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:15.424 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.421+0000 7f861de00700 1 -- 172.21.15.153:0/660512452 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f860c0095f0 con 0x7f86201039b0 2024-03-21T10:24:15.424 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.421+0000 7f861e800700 1 --2- 172.21.15.153:0/660512452 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8620103000 0x7f862019e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:24:15.424 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.422+0000 7f861de00700 1 --2- 172.21.15.153:0/660512452 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f86201039b0 0x7f862019ed00 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f860c00bda0 tx=0x7f860c00be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:15.425 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.422+0000 7f8617400700 1 -- 172.21.15.153:0/660512452 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f860c01d070 con 0x7f86201039b0 2024-03-21T10:24:15.425 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.422+0000 7f8617400700 1 -- 172.21.15.153:0/660512452 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f860c00f460 con 0x7f86201039b0 2024-03-21T10:24:15.425 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.422+0000 7f8625400700 1 -- 172.21.15.153:0/660512452 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8620198ac0 con 0x7f86201039b0 2024-03-21T10:24:15.425 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.422+0000 7f8617400700 1 -- 172.21.15.153:0/660512452 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f860c017610 con 0x7f86201039b0 2024-03-21T10:24:15.425 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.422+0000 7f8625400700 1 -- 172.21.15.153:0/660512452 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8620198fb0 con 0x7f86201039b0 2024-03-21T10:24:15.425 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.424+0000 7f8625400700 1 -- 172.21.15.153:0/660512452 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f862010b400 con 0x7f86201039b0 2024-03-21T10:24:15.426 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.424+0000 7f8617400700 1 -- 172.21.15.153:0/660512452 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f860c00fa40 con 0x7f86201039b0 2024-03-21T10:24:15.427 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.426+0000 7f8617400700 1 --2- 172.21.15.153:0/660512452 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f860406e460 0x7f8604070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:15.427 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.426+0000 7f8617400700 1 -- 172.21.15.153:0/660512452 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f860c08fa10 con 0x7f86201039b0 2024-03-21T10:24:15.428 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.426+0000 7f861e800700 1 --2- 172.21.15.153:0/660512452 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f860406e460 0x7f8604070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:15.428 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.427+0000 7f861e800700 1 --2- 172.21.15.153:0/660512452 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f860406e460 0x7f8604070920 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8610005950 tx=0x7f86100058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:15.432 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.430+0000 7f8617400700 1 -- 172.21.15.153:0/660512452 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f860c05c560 con 0x7f86201039b0 2024-03-21T10:24:15.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:24:15 smithi174 ceph-mon[88004]: pgmap v371: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:24:15.753 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.751+0000 7f8625400700 1 -- 172.21.15.153:0/660512452 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f862002cc70 con 0x7f86201039b0 2024-03-21T10:24:15.754 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.752+0000 7f8617400700 1 -- 172.21.15.153:0/660512452 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+805 (secure 0 0 0) 0x7f860c08de60 con 0x7f86201039b0 2024-03-21T10:24:15.754 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:24:15.754 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": { 2024-03-21T10:24:15.754 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:24:15.754 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:24:15.755 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": { 2024-03-21T10:24:15.755 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:24:15.755 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:24:15.755 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": { 2024-03-21T10:24:15.755 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 6 2024-03-21T10:24:15.755 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:24:15.755 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": { 2024-03-21T10:24:15.755 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-03-21T10:24:15.755 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 3 2024-03-21T10:24:15.755 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:24:15.755 INFO:teuthology.orchestra.run.smithi153.stdout: "overall": { 2024-03-21T10:24:15.755 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-03-21T10:24:15.755 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 13 2024-03-21T10:24:15.755 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-03-21T10:24:15.755 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:24:15.758 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.756+0000 7f8625400700 1 -- 172.21.15.153:0/660512452 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f860406e460 msgr2=0x7f8604070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:15.758 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.756+0000 7f8625400700 1 --2- 172.21.15.153:0/660512452 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f860406e460 0x7f8604070920 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8610005950 tx=0x7f86100058e0 comp rx=0 tx=0).stop 2024-03-21T10:24:15.758 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.756+0000 7f8625400700 1 -- 172.21.15.153:0/660512452 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f86201039b0 msgr2=0x7f862019ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:15.758 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.756+0000 7f8625400700 1 --2- 172.21.15.153:0/660512452 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f86201039b0 0x7f862019ed00 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f860c00bda0 tx=0x7f860c00be80 comp rx=0 tx=0).stop 2024-03-21T10:24:15.758 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.756+0000 7f8625400700 1 -- 172.21.15.153:0/660512452 shutdown_connections 2024-03-21T10:24:15.758 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.756+0000 7f8625400700 1 --2- 172.21.15.153:0/660512452 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8620103000 0x7f862019e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:15.758 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.756+0000 7f8625400700 1 --2- 172.21.15.153:0/660512452 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f860406e460 0x7f8604070920 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:15.759 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.756+0000 7f8625400700 1 --2- 172.21.15.153:0/660512452 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f86201039b0 0x7f862019ed00 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:15.759 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.756+0000 7f8625400700 1 -- 172.21.15.153:0/660512452 >> 172.21.15.153:0/660512452 conn(0x7f86200fe8d0 msgr2=0x7f8620100cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:15.759 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.756+0000 7f8625400700 1 -- 172.21.15.153:0/660512452 shutdown_connections 2024-03-21T10:24:15.759 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.757+0000 7f8625400700 1 -- 172.21.15.153:0/660512452 wait complete. 2024-03-21T10:24:15.915 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.913+0000 7feee4e00700 1 -- 172.21.15.153:0/4012588388 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feee0068950 msgr2=0x7feee0068dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:15.915 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.913+0000 7feee4e00700 1 --2- 172.21.15.153:0/4012588388 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feee0068950 0x7feee0068dd0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7feec8009910 tx=0x7feec8009c20 comp rx=0 tx=0).stop 2024-03-21T10:24:15.915 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.913+0000 7feee4e00700 1 -- 172.21.15.153:0/4012588388 shutdown_connections 2024-03-21T10:24:15.916 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.913+0000 7feee4e00700 1 --2- 172.21.15.153:0/4012588388 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feee0068950 0x7feee0068dd0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:15.916 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.913+0000 7feee4e00700 1 --2- 172.21.15.153:0/4012588388 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feee0105b50 0x7feee0068410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:15.916 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.913+0000 7feee4e00700 1 -- 172.21.15.153:0/4012588388 >> 172.21.15.153:0/4012588388 conn(0x7feee00755a0 msgr2=0x7feee00759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:15.916 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.913+0000 7feee4e00700 1 -- 172.21.15.153:0/4012588388 shutdown_connections 2024-03-21T10:24:15.916 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.913+0000 7feee4e00700 1 -- 172.21.15.153:0/4012588388 wait complete. 2024-03-21T10:24:15.917 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.914+0000 7feee4e00700 1 Processor -- start 2024-03-21T10:24:15.917 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.914+0000 7feee4e00700 1 -- start start 2024-03-21T10:24:15.917 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.915+0000 7feee4e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feee0068950 0x7feee01a2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:15.917 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.915+0000 7feee4e00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feee0105b50 0x7feee01a3010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:15.918 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.915+0000 7feee4e00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feee01a36a0 con 0x7feee0068950 2024-03-21T10:24:15.918 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.915+0000 7feee4e00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feee019cb50 con 0x7feee0105b50 2024-03-21T10:24:15.918 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.915+0000 7feedde00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feee0068950 0x7feee01a2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:15.918 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.915+0000 7feedde00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feee0068950 0x7feee01a2ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:46590/0 (socket says 172.21.15.153:46590) 2024-03-21T10:24:15.918 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.915+0000 7feedde00700 1 -- 172.21.15.153:0/2910701117 learned_addr learned my addr 172.21.15.153:0/2910701117 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:24:15.918 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.915+0000 7feed7e00700 1 --2- 172.21.15.153:0/2910701117 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feee0105b50 0x7feee01a3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:15.918 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.916+0000 7feedde00700 1 -- 172.21.15.153:0/2910701117 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feee0105b50 msgr2=0x7feee01a3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:15.918 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.916+0000 7feedde00700 1 --2- 172.21.15.153:0/2910701117 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feee0105b50 0x7feee01a3010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:15.918 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.916+0000 7feedde00700 1 -- 172.21.15.153:0/2910701117 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feec80095f0 con 0x7feee0068950 2024-03-21T10:24:15.919 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.916+0000 7feedde00700 1 --2- 172.21.15.153:0/2910701117 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feee0068950 0x7feee01a2ad0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7feed000d900 tx=0x7feed000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:15.919 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.916+0000 7feed6a00700 1 -- 172.21.15.153:0/2910701117 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feed00041f0 con 0x7feee0068950 2024-03-21T10:24:15.919 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.916+0000 7feed6a00700 1 -- 172.21.15.153:0/2910701117 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7feed0004350 con 0x7feee0068950 2024-03-21T10:24:15.919 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.916+0000 7feed6a00700 1 -- 172.21.15.153:0/2910701117 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feed0003dc0 con 0x7feee0068950 2024-03-21T10:24:15.919 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.916+0000 7feee4e00700 1 -- 172.21.15.153:0/2910701117 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feee019ce30 con 0x7feee0068950 2024-03-21T10:24:15.919 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.916+0000 7feee4e00700 1 -- 172.21.15.153:0/2910701117 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feee019d330 con 0x7feee0068950 2024-03-21T10:24:15.919 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.917+0000 7feee4e00700 1 -- 172.21.15.153:0/2910701117 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feee004ea90 con 0x7feee0068950 2024-03-21T10:24:15.920 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.918+0000 7feed6a00700 1 -- 172.21.15.153:0/2910701117 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7feed0009730 con 0x7feee0068950 2024-03-21T10:24:15.921 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.919+0000 7feed6a00700 1 --2- 172.21.15.153:0/2910701117 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7feec406e460 0x7feec4070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:15.921 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.920+0000 7feed6a00700 1 -- 172.21.15.153:0/2910701117 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7feed008e6e0 con 0x7feee0068950 2024-03-21T10:24:15.921 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.920+0000 7feed7e00700 1 --2- 172.21.15.153:0/2910701117 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7feec406e460 0x7feec4070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:15.922 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.920+0000 7feed7e00700 1 --2- 172.21.15.153:0/2910701117 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7feec406e460 0x7feec4070920 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7feec8005b40 tx=0x7feec801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:15.926 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:15.924+0000 7feed6a00700 1 -- 172.21.15.153:0/2910701117 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7feed005b2b0 con 0x7feee0068950 2024-03-21T10:24:16.193 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:16.191+0000 7feee4e00700 1 -- 172.21.15.153:0/2910701117 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7feee019d7f0 con 0x7feee0068950 2024-03-21T10:24:16.193 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:16.192+0000 7feed6a00700 1 -- 172.21.15.153:0/2910701117 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1876 (secure 0 0 0) 0x7feed005ae40 con 0x7feee0068950 2024-03-21T10:24:16.194 INFO:teuthology.orchestra.run.smithi153.stdout:e30 2024-03-21T10:24:16.195 INFO:teuthology.orchestra.run.smithi153.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-21T10:24:16.195 INFO:teuthology.orchestra.run.smithi153.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:24:16.195 INFO:teuthology.orchestra.run.smithi153.stdout:legacy client fscid: 1 2024-03-21T10:24:16.195 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:24:16.195 INFO:teuthology.orchestra.run.smithi153.stdout:Filesystem 'cephfs' (1) 2024-03-21T10:24:16.195 INFO:teuthology.orchestra.run.smithi153.stdout:fs_name cephfs 2024-03-21T10:24:16.195 INFO:teuthology.orchestra.run.smithi153.stdout:epoch 23 2024-03-21T10:24:16.195 INFO:teuthology.orchestra.run.smithi153.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-21T10:24:16.195 INFO:teuthology.orchestra.run.smithi153.stdout:created 2024-03-21T10:04:27.086576+0000 2024-03-21T10:24:16.195 INFO:teuthology.orchestra.run.smithi153.stdout:modified 2024-03-21T10:19:05.022092+0000 2024-03-21T10:24:16.195 INFO:teuthology.orchestra.run.smithi153.stdout:tableserver 0 2024-03-21T10:24:16.195 INFO:teuthology.orchestra.run.smithi153.stdout:root 0 2024-03-21T10:24:16.196 INFO:teuthology.orchestra.run.smithi153.stdout:session_timeout 60 2024-03-21T10:24:16.196 INFO:teuthology.orchestra.run.smithi153.stdout:session_autoclose 300 2024-03-21T10:24:16.196 INFO:teuthology.orchestra.run.smithi153.stdout:max_file_size 1099511627776 2024-03-21T10:24:16.196 INFO:teuthology.orchestra.run.smithi153.stdout:max_xattr_size 65536 2024-03-21T10:24:16.196 INFO:teuthology.orchestra.run.smithi153.stdout:required_client_features {} 2024-03-21T10:24:16.196 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure 0 2024-03-21T10:24:16.196 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure_osd_epoch 94 2024-03-21T10:24:16.196 INFO:teuthology.orchestra.run.smithi153.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:24:16.196 INFO:teuthology.orchestra.run.smithi153.stdout:max_mds 2 2024-03-21T10:24:16.196 INFO:teuthology.orchestra.run.smithi153.stdout:in 0,1 2024-03-21T10:24:16.196 INFO:teuthology.orchestra.run.smithi153.stdout:up {} 2024-03-21T10:24:16.196 INFO:teuthology.orchestra.run.smithi153.stdout:failed 0,1 2024-03-21T10:24:16.196 INFO:teuthology.orchestra.run.smithi153.stdout:damaged 2024-03-21T10:24:16.196 INFO:teuthology.orchestra.run.smithi153.stdout:stopped 2024-03-21T10:24:16.196 INFO:teuthology.orchestra.run.smithi153.stdout:data_pools [3] 2024-03-21T10:24:16.197 INFO:teuthology.orchestra.run.smithi153.stdout:metadata_pool 2 2024-03-21T10:24:16.197 INFO:teuthology.orchestra.run.smithi153.stdout:inline_data enabled 2024-03-21T10:24:16.197 INFO:teuthology.orchestra.run.smithi153.stdout:balancer 2024-03-21T10:24:16.197 INFO:teuthology.orchestra.run.smithi153.stdout:bal_rank_mask -1 2024-03-21T10:24:16.197 INFO:teuthology.orchestra.run.smithi153.stdout:standby_count_wanted 1 2024-03-21T10:24:16.197 INFO:teuthology.orchestra.run.smithi153.stdout:qdb_cluster leader: 0 members: 2024-03-21T10:24:16.197 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:24:16.197 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:24:16.197 INFO:teuthology.orchestra.run.smithi153.stdout:Standby daemons: 2024-03-21T10:24:16.197 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:24:16.197 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.zipckr{-1:34442} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6826/1103846295,v1:172.21.15.153:6827/1103846295] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:24:16.197 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.iuyluj{-1:34474} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6828/1738630832,v1:172.21.15.153:6829/1738630832] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:24:16.197 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.nhozag{-1:44269} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.174:6826/3967223227,v1:172.21.15.174:6827/3967223227] compat {c=[1],r=[1],i=[7ff]}] 2024-03-21T10:24:16.197 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.yyimst{-1:44313} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.174:6824/4127786797,v1:172.21.15.174:6825/4127786797] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:24:16.198 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:16.195+0000 7feee4e00700 1 -- 172.21.15.153:0/2910701117 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7feec406e460 msgr2=0x7feec4070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:16.198 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:16.195+0000 7feee4e00700 1 --2- 172.21.15.153:0/2910701117 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7feec406e460 0x7feec4070920 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7feec8005b40 tx=0x7feec801a040 comp rx=0 tx=0).stop 2024-03-21T10:24:16.199 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:16.195+0000 7feee4e00700 1 -- 172.21.15.153:0/2910701117 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feee0068950 msgr2=0x7feee01a2ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:16.199 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:16.195+0000 7feee4e00700 1 --2- 172.21.15.153:0/2910701117 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feee0068950 0x7feee01a2ad0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7feed000d900 tx=0x7feed000dcc0 comp rx=0 tx=0).stop 2024-03-21T10:24:16.199 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:16.195+0000 7feee4e00700 1 -- 172.21.15.153:0/2910701117 shutdown_connections 2024-03-21T10:24:16.199 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:16.195+0000 7feee4e00700 1 --2- 172.21.15.153:0/2910701117 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7feec406e460 0x7feec4070920 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:16.199 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:16.195+0000 7feee4e00700 1 --2- 172.21.15.153:0/2910701117 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feee0068950 0x7feee01a2ad0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:16.199 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:16.195+0000 7feee4e00700 1 --2- 172.21.15.153:0/2910701117 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feee0105b50 0x7feee01a3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:16.199 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:16.195+0000 7feee4e00700 1 -- 172.21.15.153:0/2910701117 >> 172.21.15.153:0/2910701117 conn(0x7feee00755a0 msgr2=0x7feee0101510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:16.199 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:16.195+0000 7feee4e00700 1 -- 172.21.15.153:0/2910701117 shutdown_connections 2024-03-21T10:24:16.199 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:16.195+0000 7feee4e00700 1 -- 172.21.15.153:0/2910701117 wait complete. 2024-03-21T10:24:16.199 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 30 2024-03-21T10:24:16.350 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:16.348+0000 7f3cb2a00700 1 -- 172.21.15.153:0/684352509 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3cac1071d0 msgr2=0x7f3cac107650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:16.350 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:16.348+0000 7f3cb2a00700 1 --2- 172.21.15.153:0/684352509 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3cac1071d0 0x7f3cac107650 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f3c98009910 tx=0x7f3c98009c20 comp rx=0 tx=0).stop 2024-03-21T10:24:16.350 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:16.348+0000 7f3cb2a00700 1 -- 172.21.15.153:0/684352509 shutdown_connections 2024-03-21T10:24:16.351 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:16.348+0000 7f3cb2a00700 1 --2- 172.21.15.153:0/684352509 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3cac1071d0 0x7f3cac107650 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:16.351 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:16.348+0000 7f3cb2a00700 1 --2- 172.21.15.153:0/684352509 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3cac10d280 0x7f3cac10d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:16.351 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:16.348+0000 7f3cb2a00700 1 -- 172.21.15.153:0/684352509 >> 172.21.15.153:0/684352509 conn(0x7f3cac0769e0 msgr2=0x7f3cac078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:16.351 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:16.348+0000 7f3cb2a00700 1 -- 172.21.15.153:0/684352509 shutdown_connections 2024-03-21T10:24:16.351 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:16.348+0000 7f3cb2a00700 1 -- 172.21.15.153:0/684352509 wait complete. 2024-03-21T10:24:16.352 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:16.349+0000 7f3cb2a00700 1 Processor -- start 2024-03-21T10:24:16.352 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:16.349+0000 7f3cb2a00700 1 -- start start 2024-03-21T10:24:16.352 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:16.350+0000 7f3cb2a00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3cac1071d0 0x7f3cac19ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:16.352 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:16.350+0000 7f3cb2a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3cac10d280 0x7f3cac19d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:16.352 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:16.350+0000 7f3cb2a00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cac19d880 con 0x7f3cac10d280 2024-03-21T10:24:16.352 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:16.350+0000 7f3cb2a00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cac1a05b0 con 0x7f3cac1071d0 2024-03-21T10:24:16.353 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:16.350+0000 7f3cab400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3cac10d280 0x7f3cac19d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:16.353 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:16.350+0000 7f3cab400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3cac10d280 0x7f3cac19d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:46604/0 (socket says 172.21.15.153:46604) 2024-03-21T10:24:16.353 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:16.350+0000 7f3cab400700 1 -- 172.21.15.153:0/1125909454 learned_addr learned my addr 172.21.15.153:0/1125909454 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:24:16.353 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:16.350+0000 7f3cabe00700 1 --2- 172.21.15.153:0/1125909454 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3cac1071d0 0x7f3cac19ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:16.353 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:16.350+0000 7f3cab400700 1 -- 172.21.15.153:0/1125909454 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3cac1071d0 msgr2=0x7f3cac19ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:16.353 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:16.350+0000 7f3cab400700 1 --2- 172.21.15.153:0/1125909454 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3cac1071d0 0x7f3cac19ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:16.353 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:16.350+0000 7f3cab400700 1 -- 172.21.15.153:0/1125909454 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c980095f0 con 0x7f3cac10d280 2024-03-21T10:24:16.353 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:16.351+0000 7f3cab400700 1 --2- 172.21.15.153:0/1125909454 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3cac10d280 0x7f3cac19d1f0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f3c9800b5c0 tx=0x7f3c9800be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:16.354 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:16.351+0000 7f3ca8c00700 1 -- 172.21.15.153:0/1125909454 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3c9801d070 con 0x7f3cac10d280 2024-03-21T10:24:16.354 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:16.351+0000 7f3ca8c00700 1 -- 172.21.15.153:0/1125909454 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3c9800f460 con 0x7f3cac10d280 2024-03-21T10:24:16.354 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:16.351+0000 7f3ca8c00700 1 -- 172.21.15.153:0/1125909454 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3c98017610 con 0x7f3cac10d280 2024-03-21T10:24:16.354 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:16.351+0000 7f3cb2a00700 1 -- 172.21.15.153:0/1125909454 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cac1a0830 con 0x7f3cac10d280 2024-03-21T10:24:16.354 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:16.351+0000 7f3cb2a00700 1 -- 172.21.15.153:0/1125909454 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cac1a0da0 con 0x7f3cac10d280 2024-03-21T10:24:16.354 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:16.352+0000 7f3cb2a00700 1 -- 172.21.15.153:0/1125909454 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cac10f6c0 con 0x7f3cac10d280 2024-03-21T10:24:16.355 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:16.353+0000 7f3ca8c00700 1 -- 172.21.15.153:0/1125909454 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f3c9800f5d0 con 0x7f3cac10d280 2024-03-21T10:24:16.356 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:16.354+0000 7f3ca8c00700 1 --2- 172.21.15.153:0/1125909454 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f3c9006e460 0x7f3c90070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:16.356 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:16.354+0000 7f3ca8c00700 1 -- 172.21.15.153:0/1125909454 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f3c9808fad0 con 0x7f3cac10d280 2024-03-21T10:24:16.356 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:16.354+0000 7f3cabe00700 1 --2- 172.21.15.153:0/1125909454 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f3c9006e460 0x7f3c90070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:16.356 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:16.355+0000 7f3cabe00700 1 --2- 172.21.15.153:0/1125909454 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f3c9006e460 0x7f3c90070920 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c005fd0 tx=0x7f3c9c00a560 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:16.360 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:16.359+0000 7f3ca8c00700 1 -- 172.21.15.153:0/1125909454 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f3c98058b70 con 0x7f3cac10d280 2024-03-21T10:24:16.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:24:16 smithi174 ceph-mon[88004]: from='client.34702 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:24:16.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:24:16 smithi174 ceph-mon[88004]: from='client.34706 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:24:16.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:24:16 smithi174 ceph-mon[88004]: from='client.34710 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:24:16.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:24:16 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/660512452' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:24:16.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:24:16 smithi153 ceph-mon[113733]: from='client.34702 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:24:16.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:24:16 smithi153 ceph-mon[113733]: from='client.34706 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:24:16.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:24:16 smithi153 ceph-mon[113733]: from='client.34710 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:24:16.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:24:16 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/660512452' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:24:16.600 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:16.598+0000 7f3cb2a00700 1 -- 172.21.15.153:0/1125909454 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3cac1a1080 con 0x7f3c9006e460 2024-03-21T10:24:16.602 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:16.600+0000 7f3ca8c00700 1 -- 172.21.15.153:0/1125909454 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f3cac1a1080 con 0x7f3c9006e460 2024-03-21T10:24:16.602 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:24:16.602 INFO:teuthology.orchestra.run.smithi153.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232", 2024-03-21T10:24:16.603 INFO:teuthology.orchestra.run.smithi153.stdout: "in_progress": true, 2024-03-21T10:24:16.603 INFO:teuthology.orchestra.run.smithi153.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-21T10:24:16.603 INFO:teuthology.orchestra.run.smithi153.stdout: "services_complete": [ 2024-03-21T10:24:16.603 INFO:teuthology.orchestra.run.smithi153.stdout: "crash", 2024-03-21T10:24:16.603 INFO:teuthology.orchestra.run.smithi153.stdout: "mon", 2024-03-21T10:24:16.603 INFO:teuthology.orchestra.run.smithi153.stdout: "osd", 2024-03-21T10:24:16.603 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr" 2024-03-21T10:24:16.603 INFO:teuthology.orchestra.run.smithi153.stdout: ], 2024-03-21T10:24:16.603 INFO:teuthology.orchestra.run.smithi153.stdout: "progress": "15/23 daemons upgraded", 2024-03-21T10:24:16.603 INFO:teuthology.orchestra.run.smithi153.stdout: "message": "Currently upgrading mds daemons", 2024-03-21T10:24:16.603 INFO:teuthology.orchestra.run.smithi153.stdout: "is_paused": false 2024-03-21T10:24:16.603 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:24:16.605 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:16.603+0000 7f3cb2a00700 1 -- 172.21.15.153:0/1125909454 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f3c9006e460 msgr2=0x7f3c90070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:16.605 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:16.603+0000 7f3cb2a00700 1 --2- 172.21.15.153:0/1125909454 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f3c9006e460 0x7f3c90070920 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c005fd0 tx=0x7f3c9c00a560 comp rx=0 tx=0).stop 2024-03-21T10:24:16.605 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:16.603+0000 7f3cb2a00700 1 -- 172.21.15.153:0/1125909454 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3cac10d280 msgr2=0x7f3cac19d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:16.605 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:16.603+0000 7f3cb2a00700 1 --2- 172.21.15.153:0/1125909454 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3cac10d280 0x7f3cac19d1f0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f3c9800b5c0 tx=0x7f3c9800be00 comp rx=0 tx=0).stop 2024-03-21T10:24:16.605 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:16.603+0000 7f3cb2a00700 1 -- 172.21.15.153:0/1125909454 shutdown_connections 2024-03-21T10:24:16.606 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:16.603+0000 7f3cb2a00700 1 --2- 172.21.15.153:0/1125909454 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3cac1071d0 0x7f3cac19ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:16.606 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:16.603+0000 7f3cb2a00700 1 --2- 172.21.15.153:0/1125909454 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f3c9006e460 0x7f3c90070920 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:16.606 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:16.603+0000 7f3cb2a00700 1 --2- 172.21.15.153:0/1125909454 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3cac10d280 0x7f3cac19d1f0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:16.606 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:16.603+0000 7f3cb2a00700 1 -- 172.21.15.153:0/1125909454 >> 172.21.15.153:0/1125909454 conn(0x7f3cac0769e0 msgr2=0x7f3cac078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:16.606 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:16.603+0000 7f3cb2a00700 1 -- 172.21.15.153:0/1125909454 shutdown_connections 2024-03-21T10:24:16.606 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:16.603+0000 7f3cb2a00700 1 -- 172.21.15.153:0/1125909454 wait complete. 2024-03-21T10:24:16.759 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:16.757+0000 7f3ec3400700 1 -- 172.21.15.153:0/240482346 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ebc102620 msgr2=0x7f3ebc10ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:16.759 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:16.757+0000 7f3ec3400700 1 --2- 172.21.15.153:0/240482346 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ebc102620 0x7f3ebc10ab10 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f3ea8009910 tx=0x7f3ea8009c20 comp rx=0 tx=0).stop 2024-03-21T10:24:16.760 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:16.757+0000 7f3ec3400700 1 -- 172.21.15.153:0/240482346 shutdown_connections 2024-03-21T10:24:16.760 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:16.757+0000 7f3ec3400700 1 --2- 172.21.15.153:0/240482346 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ebc102620 0x7f3ebc10ab10 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:16.760 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:16.757+0000 7f3ec3400700 1 --2- 172.21.15.153:0/240482346 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3ebc101d00 0x7f3ebc1020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:16.760 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:16.757+0000 7f3ec3400700 1 -- 172.21.15.153:0/240482346 >> 172.21.15.153:0/240482346 conn(0x7f3ebc0fb570 msgr2=0x7f3ebc0fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:16.760 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:16.757+0000 7f3ec3400700 1 -- 172.21.15.153:0/240482346 shutdown_connections 2024-03-21T10:24:16.760 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:16.757+0000 7f3ec3400700 1 -- 172.21.15.153:0/240482346 wait complete. 2024-03-21T10:24:16.762 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:16.758+0000 7f3ec3400700 1 Processor -- start 2024-03-21T10:24:16.762 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:16.758+0000 7f3ec3400700 1 -- start start 2024-03-21T10:24:16.762 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:16.759+0000 7f3ec3400700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3ebc101d00 0x7f3ebc19c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:16.762 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:16.759+0000 7f3ec3400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ebc102620 0x7f3ebc19cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:16.762 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:16.759+0000 7f3ec3400700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ebc19d150 con 0x7f3ebc102620 2024-03-21T10:24:16.762 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:16.759+0000 7f3ec3400700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ebc196600 con 0x7f3ebc101d00 2024-03-21T10:24:16.762 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:16.759+0000 7f3ebbe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ebc102620 0x7f3ebc19cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:16.762 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:16.759+0000 7f3ebbe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ebc102620 0x7f3ebc19cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:46624/0 (socket says 172.21.15.153:46624) 2024-03-21T10:24:16.763 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:16.759+0000 7f3ebbe00700 1 -- 172.21.15.153:0/4194192235 learned_addr learned my addr 172.21.15.153:0/4194192235 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:24:16.763 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:16.759+0000 7f3ec0a00700 1 --2- 172.21.15.153:0/4194192235 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3ebc101d00 0x7f3ebc19c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:16.763 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:16.760+0000 7f3ebbe00700 1 -- 172.21.15.153:0/4194192235 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3ebc101d00 msgr2=0x7f3ebc19c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:16.763 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:16.760+0000 7f3ebbe00700 1 --2- 172.21.15.153:0/4194192235 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3ebc101d00 0x7f3ebc19c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:16.763 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:16.760+0000 7f3ebbe00700 1 -- 172.21.15.153:0/4194192235 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ea80095f0 con 0x7f3ebc102620 2024-03-21T10:24:16.763 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:16.760+0000 7f3ebbe00700 1 --2- 172.21.15.153:0/4194192235 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ebc102620 0x7f3ebc19cac0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f3ea800bda0 tx=0x7f3ea800be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:16.763 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:16.760+0000 7f3eb9600700 1 -- 172.21.15.153:0/4194192235 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3ea801d070 con 0x7f3ebc102620 2024-03-21T10:24:16.763 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:16.760+0000 7f3eb9600700 1 -- 172.21.15.153:0/4194192235 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3ea800f460 con 0x7f3ebc102620 2024-03-21T10:24:16.763 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:16.760+0000 7f3eb9600700 1 -- 172.21.15.153:0/4194192235 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3ea8017630 con 0x7f3ebc102620 2024-03-21T10:24:16.763 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:16.760+0000 7f3ec3400700 1 -- 172.21.15.153:0/4194192235 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ebc196880 con 0x7f3ebc102620 2024-03-21T10:24:16.763 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:16.760+0000 7f3ec3400700 1 -- 172.21.15.153:0/4194192235 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ebc196d70 con 0x7f3ebc102620 2024-03-21T10:24:16.764 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:16.761+0000 7f3ec3400700 1 -- 172.21.15.153:0/4194192235 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ebc108210 con 0x7f3ebc102620 2024-03-21T10:24:16.764 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:16.762+0000 7f3eb9600700 1 -- 172.21.15.153:0/4194192235 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f3ea800fa40 con 0x7f3ebc102620 2024-03-21T10:24:16.765 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:16.763+0000 7f3eb9600700 1 --2- 172.21.15.153:0/4194192235 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f3ea406e460 0x7f3ea4070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:16.765 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:16.763+0000 7f3eb9600700 1 -- 172.21.15.153:0/4194192235 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f3ea808f9a0 con 0x7f3ebc102620 2024-03-21T10:24:16.765 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:16.764+0000 7f3ec0a00700 1 --2- 172.21.15.153:0/4194192235 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f3ea406e460 0x7f3ea4070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:16.766 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:16.764+0000 7f3ec0a00700 1 --2- 172.21.15.153:0/4194192235 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f3ea406e460 0x7f3ea4070920 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3eb0005950 tx=0x7f3eb00058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:16.770 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:16.768+0000 7f3eb9600700 1 -- 172.21.15.153:0/4194192235 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f3ea805c4f0 con 0x7f3ebc102620 2024-03-21T10:24:17.077 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:17.075+0000 7f3ec3400700 1 -- 172.21.15.153:0/4194192235 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f3ebc04ea90 con 0x7f3ebc102620 2024-03-21T10:24:17.078 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:17.076+0000 7f3eb9600700 1 -- 172.21.15.153:0/4194192235 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f3ea805c080 con 0x7f3ebc102620 2024-03-21T10:24:17.078 INFO:teuthology.orchestra.run.smithi153.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-03-21T10:24:17.078 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-21T10:24:17.078 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is degraded 2024-03-21T10:24:17.078 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-21T10:24:17.078 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-21T10:24:17.078 INFO:teuthology.orchestra.run.smithi153.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-21T10:24:17.079 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-21T10:24:17.081 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:17.079+0000 7f3ec3400700 1 -- 172.21.15.153:0/4194192235 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f3ea406e460 msgr2=0x7f3ea4070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:17.081 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:17.079+0000 7f3ec3400700 1 --2- 172.21.15.153:0/4194192235 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f3ea406e460 0x7f3ea4070920 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3eb0005950 tx=0x7f3eb00058e0 comp rx=0 tx=0).stop 2024-03-21T10:24:17.081 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:17.079+0000 7f3ec3400700 1 -- 172.21.15.153:0/4194192235 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ebc102620 msgr2=0x7f3ebc19cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:17.081 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:17.079+0000 7f3ec3400700 1 --2- 172.21.15.153:0/4194192235 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ebc102620 0x7f3ebc19cac0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f3ea800bda0 tx=0x7f3ea800be80 comp rx=0 tx=0).stop 2024-03-21T10:24:17.081 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:17.079+0000 7f3ec3400700 1 -- 172.21.15.153:0/4194192235 shutdown_connections 2024-03-21T10:24:17.082 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:17.079+0000 7f3ec3400700 1 --2- 172.21.15.153:0/4194192235 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3ebc101d00 0x7f3ebc19c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:17.082 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:17.079+0000 7f3ec3400700 1 --2- 172.21.15.153:0/4194192235 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f3ea406e460 0x7f3ea4070920 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:17.082 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:17.079+0000 7f3ec3400700 1 --2- 172.21.15.153:0/4194192235 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ebc102620 0x7f3ebc19cac0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:17.082 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:17.079+0000 7f3ec3400700 1 -- 172.21.15.153:0/4194192235 >> 172.21.15.153:0/4194192235 conn(0x7f3ebc0fb570 msgr2=0x7f3ebc105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:17.082 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:17.079+0000 7f3ec3400700 1 -- 172.21.15.153:0/4194192235 shutdown_connections 2024-03-21T10:24:17.082 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:17.079+0000 7f3ec3400700 1 -- 172.21.15.153:0/4194192235 wait complete. 2024-03-21T10:24:17.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:24:17 smithi174 ceph-mon[88004]: pgmap v372: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:24:17.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:24:17 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/2910701117' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:24:17.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:24:17 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:24:17.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:24:17 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/4194192235' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:24:17.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:24:17 smithi153 ceph-mon[113733]: pgmap v372: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:24:17.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:24:17 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/2910701117' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:24:17.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:24:17 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:24:17.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:24:17 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/4194192235' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:24:18.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:24:18 smithi174 ceph-mon[88004]: from='client.34722 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:24:18.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:24:18 smithi153 ceph-mon[113733]: from='client.34722 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:24:19.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:24:19 smithi174 ceph-mon[88004]: pgmap v373: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:24:19.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:24:19 smithi153 ceph-mon[113733]: pgmap v373: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:24:21.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:24:21 smithi174 ceph-mon[88004]: pgmap v374: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:24:21.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:24:21 smithi153 ceph-mon[113733]: pgmap v374: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:24:23.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:24:23 smithi174 ceph-mon[88004]: pgmap v375: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:24:23.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:24:23 smithi153 ceph-mon[113733]: pgmap v375: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:24:25.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:24:25 smithi174 ceph-mon[88004]: pgmap v376: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:24:25.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:24:25 smithi153 ceph-mon[113733]: pgmap v376: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:24:27.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:24:27 smithi174 ceph-mon[88004]: pgmap v377: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:24:27.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:24:27 smithi153 ceph-mon[113733]: pgmap v377: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:24:29.452 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:24:29 smithi174 ceph-mon[88004]: pgmap v378: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:24:29.504 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:24:29 smithi153 ceph-mon[113733]: pgmap v378: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:24:31.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:24:31 smithi174 ceph-mon[88004]: pgmap v379: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:24:31.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:24:31 smithi153 ceph-mon[113733]: pgmap v379: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:24:32.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:24:32 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:24:32.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:24:32 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:24:33.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:24:33 smithi174 ceph-mon[88004]: pgmap v380: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:24:33.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:24:33 smithi153 ceph-mon[113733]: pgmap v380: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:24:35.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:24:35 smithi153 ceph-mon[113733]: pgmap v381: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:24:35.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:24:35 smithi174 ceph-mon[88004]: pgmap v381: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:24:37.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:24:37 smithi153 ceph-mon[113733]: pgmap v382: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:24:37.626 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:24:37 smithi174 ceph-mon[88004]: pgmap v382: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:24:39.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:24:39 smithi153 ceph-mon[113733]: pgmap v383: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:24:39.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:24:39 smithi174 ceph-mon[88004]: pgmap v383: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:24:41.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:24:41 smithi153 ceph-mon[113733]: pgmap v384: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:24:41.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:24:41 smithi174 ceph-mon[88004]: pgmap v384: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:24:43.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:24:43 smithi153 ceph-mon[113733]: pgmap v385: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:24:43.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:24:43 smithi174 ceph-mon[88004]: pgmap v385: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:24:45.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:24:45 smithi153 ceph-mon[113733]: pgmap v386: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:24:45.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:24:45 smithi174 ceph-mon[88004]: pgmap v386: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:24:46.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:24:46 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:24:46.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:24:46 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:24:47.251 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:47.250+0000 7ff61ba00700 1 -- 172.21.15.153:0/316849842 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff614102ff0 msgr2=0x7ff6141033d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:47.252 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:47.250+0000 7ff61ba00700 1 --2- 172.21.15.153:0/316849842 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff614102ff0 0x7ff6141033d0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7ff608009910 tx=0x7ff608009c20 comp rx=0 tx=0).stop 2024-03-21T10:24:47.252 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:47.250+0000 7ff61ba00700 1 -- 172.21.15.153:0/316849842 shutdown_connections 2024-03-21T10:24:47.252 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:47.250+0000 7ff61ba00700 1 --2- 172.21.15.153:0/316849842 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff6141039a0 0x7ff6141079f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:47.252 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:47.250+0000 7ff61ba00700 1 --2- 172.21.15.153:0/316849842 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff614102ff0 0x7ff6141033d0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:47.252 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:47.250+0000 7ff61ba00700 1 -- 172.21.15.153:0/316849842 >> 172.21.15.153:0/316849842 conn(0x7ff6140fe880 msgr2=0x7ff614100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:47.252 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:47.250+0000 7ff61ba00700 1 -- 172.21.15.153:0/316849842 shutdown_connections 2024-03-21T10:24:47.252 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:47.251+0000 7ff61ba00700 1 -- 172.21.15.153:0/316849842 wait complete. 2024-03-21T10:24:47.253 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:47.251+0000 7ff61ba00700 1 Processor -- start 2024-03-21T10:24:47.253 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:47.252+0000 7ff61ba00700 1 -- start start 2024-03-21T10:24:47.253 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:47.252+0000 7ff61ba00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff614102ff0 0x7ff61419e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:47.254 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:47.252+0000 7ff61ba00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff6141039a0 0x7ff61419ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:47.254 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:47.252+0000 7ff61ba00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff61419f3f0 con 0x7ff6141039a0 2024-03-21T10:24:47.254 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:47.252+0000 7ff61ba00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6141988a0 con 0x7ff614102ff0 2024-03-21T10:24:47.254 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:47.253+0000 7ff613e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff6141039a0 0x7ff61419ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:47.254 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:47.253+0000 7ff619000700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff614102ff0 0x7ff61419e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:47.254 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:47.253+0000 7ff619000700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff614102ff0 0x7ff61419e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.153:57960/0 (socket says 172.21.15.153:57960) 2024-03-21T10:24:47.254 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:47.253+0000 7ff619000700 1 -- 172.21.15.153:0/849978649 learned_addr learned my addr 172.21.15.153:0/849978649 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:24:47.255 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:47.253+0000 7ff619000700 1 -- 172.21.15.153:0/849978649 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff6141039a0 msgr2=0x7ff61419ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:47.255 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:47.253+0000 7ff619000700 1 --2- 172.21.15.153:0/849978649 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff6141039a0 0x7ff61419ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:47.255 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:47.253+0000 7ff619000700 1 -- 172.21.15.153:0/849978649 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6080095f0 con 0x7ff614102ff0 2024-03-21T10:24:47.255 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:47.254+0000 7ff619000700 1 --2- 172.21.15.153:0/849978649 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff614102ff0 0x7ff61419e820 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ff60800b5c0 tx=0x7ff60800bd50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:47.255 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:47.254+0000 7ff611600700 1 -- 172.21.15.153:0/849978649 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff60801d070 con 0x7ff614102ff0 2024-03-21T10:24:47.256 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:47.255+0000 7ff61ba00700 1 -- 172.21.15.153:0/849978649 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff614198b20 con 0x7ff614102ff0 2024-03-21T10:24:47.256 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:47.255+0000 7ff611600700 1 -- 172.21.15.153:0/849978649 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff60800f460 con 0x7ff614102ff0 2024-03-21T10:24:47.256 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:47.255+0000 7ff611600700 1 -- 172.21.15.153:0/849978649 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff608017630 con 0x7ff614102ff0 2024-03-21T10:24:47.256 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:47.255+0000 7ff61ba00700 1 -- 172.21.15.153:0/849978649 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff614199010 con 0x7ff614102ff0 2024-03-21T10:24:47.256 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:47.256+0000 7ff61ba00700 1 -- 172.21.15.153:0/849978649 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff61410b390 con 0x7ff614102ff0 2024-03-21T10:24:47.258 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:47.257+0000 7ff611600700 1 -- 172.21.15.153:0/849978649 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7ff60800f5d0 con 0x7ff614102ff0 2024-03-21T10:24:47.259 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:47.258+0000 7ff611600700 1 --2- 172.21.15.153:0/849978649 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff5fc06e460 0x7ff5fc070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:47.259 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:47.258+0000 7ff611600700 1 -- 172.21.15.153:0/849978649 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7ff60808fc20 con 0x7ff614102ff0 2024-03-21T10:24:47.259 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:47.259+0000 7ff613e00700 1 --2- 172.21.15.153:0/849978649 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff5fc06e460 0x7ff5fc070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:47.260 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:47.259+0000 7ff613e00700 1 --2- 172.21.15.153:0/849978649 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff5fc06e460 0x7ff5fc070920 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff6140ff0d0 tx=0x7ff600009190 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:47.276 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:47.274+0000 7ff611600700 1 -- 172.21.15.153:0/849978649 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff60805c890 con 0x7ff614102ff0 2024-03-21T10:24:47.520 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:47.518+0000 7ff61ba00700 1 -- 172.21.15.153:0/849978649 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff61402cce0 con 0x7ff5fc06e460 2024-03-21T10:24:47.520 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:24:47 smithi153 ceph-mon[113733]: pgmap v387: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:24:47.522 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:47.521+0000 7ff611600700 1 -- 172.21.15.153:0/849978649 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7ff61402cce0 con 0x7ff5fc06e460 2024-03-21T10:24:47.525 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:47.523+0000 7ff61ba00700 1 -- 172.21.15.153:0/849978649 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff5fc06e460 msgr2=0x7ff5fc070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:47.525 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:47.523+0000 7ff61ba00700 1 --2- 172.21.15.153:0/849978649 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff5fc06e460 0x7ff5fc070920 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff6140ff0d0 tx=0x7ff600009190 comp rx=0 tx=0).stop 2024-03-21T10:24:47.526 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:47.523+0000 7ff61ba00700 1 -- 172.21.15.153:0/849978649 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff614102ff0 msgr2=0x7ff61419e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:47.526 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:47.523+0000 7ff61ba00700 1 --2- 172.21.15.153:0/849978649 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff614102ff0 0x7ff61419e820 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ff60800b5c0 tx=0x7ff60800bd50 comp rx=0 tx=0).stop 2024-03-21T10:24:47.526 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:47.524+0000 7ff61ba00700 1 -- 172.21.15.153:0/849978649 shutdown_connections 2024-03-21T10:24:47.526 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:47.524+0000 7ff61ba00700 1 --2- 172.21.15.153:0/849978649 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff614102ff0 0x7ff61419e820 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:47.526 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:47.524+0000 7ff61ba00700 1 --2- 172.21.15.153:0/849978649 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff5fc06e460 0x7ff5fc070920 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:47.526 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:47.524+0000 7ff61ba00700 1 --2- 172.21.15.153:0/849978649 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff6141039a0 0x7ff61419ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:47.526 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:47.524+0000 7ff61ba00700 1 -- 172.21.15.153:0/849978649 >> 172.21.15.153:0/849978649 conn(0x7ff6140fe880 msgr2=0x7ff614100000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:47.526 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:47.524+0000 7ff61ba00700 1 -- 172.21.15.153:0/849978649 shutdown_connections 2024-03-21T10:24:47.526 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:47.524+0000 7ff61ba00700 1 -- 172.21.15.153:0/849978649 wait complete. 2024-03-21T10:24:47.542 INFO:teuthology.orchestra.run.smithi153.stdout:true 2024-03-21T10:24:47.686 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:47.684+0000 7f48bd000700 1 -- 172.21.15.153:0/1111701744 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f48b810d280 msgr2=0x7f48b810d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:47.686 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:47.684+0000 7f48bd000700 1 --2- 172.21.15.153:0/1111701744 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f48b810d280 0x7f48b810d660 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f48a8009910 tx=0x7f48a8009c20 comp rx=0 tx=0).stop 2024-03-21T10:24:47.686 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:47.684+0000 7f48bd000700 1 -- 172.21.15.153:0/1111701744 shutdown_connections 2024-03-21T10:24:47.686 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:47.684+0000 7f48bd000700 1 --2- 172.21.15.153:0/1111701744 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f48b81071d0 0x7f48b8107650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:47.686 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:47.684+0000 7f48bd000700 1 --2- 172.21.15.153:0/1111701744 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f48b810d280 0x7f48b810d660 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:47.686 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:47.684+0000 7f48bd000700 1 -- 172.21.15.153:0/1111701744 >> 172.21.15.153:0/1111701744 conn(0x7f48b80769e0 msgr2=0x7f48b8078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:47.687 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:47.684+0000 7f48bd000700 1 -- 172.21.15.153:0/1111701744 shutdown_connections 2024-03-21T10:24:47.687 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:47.684+0000 7f48bd000700 1 -- 172.21.15.153:0/1111701744 wait complete. 2024-03-21T10:24:47.687 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:47.685+0000 7f48bd000700 1 Processor -- start 2024-03-21T10:24:47.688 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:47.685+0000 7f48bd000700 1 -- start start 2024-03-21T10:24:47.688 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:47.686+0000 7f48bd000700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f48b81071d0 0x7f48b819ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:47.688 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:47.686+0000 7f48bd000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f48b810d280 0x7f48b819d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:47.688 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:47.686+0000 7f48bd000700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48b819d8a0 con 0x7f48b810d280 2024-03-21T10:24:47.688 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:47.686+0000 7f48bd000700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48b81a05d0 con 0x7f48b81071d0 2024-03-21T10:24:47.688 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:47.686+0000 7f48ad400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f48b810d280 0x7f48b819d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:47.688 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:47.686+0000 7f48ad400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f48b810d280 0x7f48b819d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:35178/0 (socket says 172.21.15.153:35178) 2024-03-21T10:24:47.688 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:47.686+0000 7f48ad400700 1 -- 172.21.15.153:0/1989146277 learned_addr learned my addr 172.21.15.153:0/1989146277 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:24:47.689 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:47.687+0000 7f48b5e00700 1 --2- 172.21.15.153:0/1989146277 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f48b81071d0 0x7f48b819ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:47.689 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:47.687+0000 7f48ad400700 1 -- 172.21.15.153:0/1989146277 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f48b81071d0 msgr2=0x7f48b819ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:47.689 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:47.687+0000 7f48ad400700 1 --2- 172.21.15.153:0/1989146277 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f48b81071d0 0x7f48b819ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:47.690 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:47.687+0000 7f48ad400700 1 -- 172.21.15.153:0/1989146277 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48a80095f0 con 0x7f48b810d280 2024-03-21T10:24:47.690 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:47.687+0000 7f48ad400700 1 --2- 172.21.15.153:0/1989146277 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f48b810d280 0x7f48b819d210 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f48a000d8d0 tx=0x7f48a000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:47.690 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:47.687+0000 7f48af400700 1 -- 172.21.15.153:0/1989146277 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f48a0009940 con 0x7f48b810d280 2024-03-21T10:24:47.690 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:47.687+0000 7f48af400700 1 -- 172.21.15.153:0/1989146277 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f48a0010460 con 0x7f48b810d280 2024-03-21T10:24:47.690 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:47.687+0000 7f48bd000700 1 -- 172.21.15.153:0/1989146277 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48b81a08b0 con 0x7f48b810d280 2024-03-21T10:24:47.690 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:47.688+0000 7f48af400700 1 -- 172.21.15.153:0/1989146277 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f48a000f5f0 con 0x7f48b810d280 2024-03-21T10:24:47.690 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:47.688+0000 7f48bd000700 1 -- 172.21.15.153:0/1989146277 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48b81a0e80 con 0x7f48b810d280 2024-03-21T10:24:47.690 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:47.689+0000 7f48bd000700 1 -- 172.21.15.153:0/1989146277 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48b804ea90 con 0x7f48b810d280 2024-03-21T10:24:47.691 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:47.690+0000 7f48af400700 1 -- 172.21.15.153:0/1989146277 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f48a00105d0 con 0x7f48b810d280 2024-03-21T10:24:47.692 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:47.691+0000 7f48af400700 1 --2- 172.21.15.153:0/1989146277 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f489806e460 0x7f4898070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:47.692 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:47.691+0000 7f48af400700 1 -- 172.21.15.153:0/1989146277 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f48a008e5b0 con 0x7f48b810d280 2024-03-21T10:24:47.692 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:47.691+0000 7f48b5e00700 1 --2- 172.21.15.153:0/1989146277 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f489806e460 0x7f4898070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:47.693 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:47.692+0000 7f48b5e00700 1 --2- 172.21.15.153:0/1989146277 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f489806e460 0x7f4898070920 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f48a8006010 tx=0x7f48a801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:47.697 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:47.696+0000 7f48af400700 1 -- 172.21.15.153:0/1989146277 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f48a00575d0 con 0x7f48b810d280 2024-03-21T10:24:47.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:24:47 smithi174 ceph-mon[88004]: pgmap v387: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:24:47.933 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:47.931+0000 7f48bd000700 1 -- 172.21.15.153:0/1989146277 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f48b81a1210 con 0x7f489806e460 2024-03-21T10:24:47.935 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:47.934+0000 7f48af400700 1 -- 172.21.15.153:0/1989146277 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f48b81a1210 con 0x7f489806e460 2024-03-21T10:24:47.938 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:47.936+0000 7f48bd000700 1 -- 172.21.15.153:0/1989146277 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f489806e460 msgr2=0x7f4898070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:47.938 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:47.936+0000 7f48bd000700 1 --2- 172.21.15.153:0/1989146277 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f489806e460 0x7f4898070920 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f48a8006010 tx=0x7f48a801a040 comp rx=0 tx=0).stop 2024-03-21T10:24:47.938 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:47.936+0000 7f48bd000700 1 -- 172.21.15.153:0/1989146277 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f48b810d280 msgr2=0x7f48b819d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:47.938 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:47.936+0000 7f48bd000700 1 --2- 172.21.15.153:0/1989146277 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f48b810d280 0x7f48b819d210 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f48a000d8d0 tx=0x7f48a000dc90 comp rx=0 tx=0).stop 2024-03-21T10:24:47.938 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:47.936+0000 7f48bd000700 1 -- 172.21.15.153:0/1989146277 shutdown_connections 2024-03-21T10:24:47.939 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:47.936+0000 7f48bd000700 1 --2- 172.21.15.153:0/1989146277 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f48b81071d0 0x7f48b819ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:47.939 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:47.936+0000 7f48bd000700 1 --2- 172.21.15.153:0/1989146277 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f489806e460 0x7f4898070920 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:47.939 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:47.936+0000 7f48bd000700 1 --2- 172.21.15.153:0/1989146277 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f48b810d280 0x7f48b819d210 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:47.939 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:47.936+0000 7f48bd000700 1 -- 172.21.15.153:0/1989146277 >> 172.21.15.153:0/1989146277 conn(0x7f48b80769e0 msgr2=0x7f48b8078320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:47.939 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:47.937+0000 7f48bd000700 1 -- 172.21.15.153:0/1989146277 shutdown_connections 2024-03-21T10:24:47.939 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:47.937+0000 7f48bd000700 1 -- 172.21.15.153:0/1989146277 wait complete. 2024-03-21T10:24:48.082 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.080+0000 7f3630000700 1 -- 172.21.15.153:0/3974624647 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f36281071d0 msgr2=0x7f3628107650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:48.083 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.080+0000 7f3630000700 1 --2- 172.21.15.153:0/3974624647 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f36281071d0 0x7f3628107650 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f3614009910 tx=0x7f3614009c20 comp rx=0 tx=0).stop 2024-03-21T10:24:48.083 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.080+0000 7f3630000700 1 -- 172.21.15.153:0/3974624647 shutdown_connections 2024-03-21T10:24:48.083 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.080+0000 7f3630000700 1 --2- 172.21.15.153:0/3974624647 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f36281071d0 0x7f3628107650 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:48.083 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.081+0000 7f3630000700 1 --2- 172.21.15.153:0/3974624647 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f362810d280 0x7f362810d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:48.083 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.081+0000 7f3630000700 1 -- 172.21.15.153:0/3974624647 >> 172.21.15.153:0/3974624647 conn(0x7f3628076990 msgr2=0x7f3628078db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:48.083 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.081+0000 7f3630000700 1 -- 172.21.15.153:0/3974624647 shutdown_connections 2024-03-21T10:24:48.083 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.081+0000 7f3630000700 1 -- 172.21.15.153:0/3974624647 wait complete. 2024-03-21T10:24:48.084 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.082+0000 7f3630000700 1 Processor -- start 2024-03-21T10:24:48.084 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.082+0000 7f3630000700 1 -- start start 2024-03-21T10:24:48.084 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.082+0000 7f3630000700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f36281071d0 0x7f362819cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:48.085 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.082+0000 7f3630000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f362810d280 0x7f362819d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:48.085 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.082+0000 7f3630000700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f362819d940 con 0x7f362810d280 2024-03-21T10:24:48.085 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.082+0000 7f3630000700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36281a16d0 con 0x7f36281071d0 2024-03-21T10:24:48.085 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.083+0000 7f362cc00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f362810d280 0x7f362819d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:48.085 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.083+0000 7f362cc00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f362810d280 0x7f362819d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:35198/0 (socket says 172.21.15.153:35198) 2024-03-21T10:24:48.085 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.083+0000 7f362cc00700 1 -- 172.21.15.153:0/1956508048 learned_addr learned my addr 172.21.15.153:0/1956508048 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:24:48.085 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.083+0000 7f362d600700 1 --2- 172.21.15.153:0/1956508048 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f36281071d0 0x7f362819cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:48.086 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.083+0000 7f362cc00700 1 -- 172.21.15.153:0/1956508048 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f36281071d0 msgr2=0x7f362819cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:48.086 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.083+0000 7f362cc00700 1 --2- 172.21.15.153:0/1956508048 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f36281071d0 0x7f362819cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:48.086 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.083+0000 7f362cc00700 1 -- 172.21.15.153:0/1956508048 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36140095f0 con 0x7f362810d280 2024-03-21T10:24:48.086 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.084+0000 7f362cc00700 1 --2- 172.21.15.153:0/1956508048 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f362810d280 0x7f362819d260 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f3614006010 tx=0x7f361400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:48.086 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.084+0000 7f3626000700 1 -- 172.21.15.153:0/1956508048 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f361401d070 con 0x7f362810d280 2024-03-21T10:24:48.086 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.084+0000 7f3626000700 1 -- 172.21.15.153:0/1956508048 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3614004fb0 con 0x7f362810d280 2024-03-21T10:24:48.087 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.084+0000 7f3630000700 1 -- 172.21.15.153:0/1956508048 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36281a1950 con 0x7f362810d280 2024-03-21T10:24:48.087 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.084+0000 7f3626000700 1 -- 172.21.15.153:0/1956508048 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3614022ea0 con 0x7f362810d280 2024-03-21T10:24:48.087 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.084+0000 7f3630000700 1 -- 172.21.15.153:0/1956508048 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36281a1e40 con 0x7f362810d280 2024-03-21T10:24:48.087 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.085+0000 7f3630000700 1 -- 172.21.15.153:0/1956508048 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f362810f6c0 con 0x7f362810d280 2024-03-21T10:24:48.087 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.086+0000 7f3626000700 1 -- 172.21.15.153:0/1956508048 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f3614022470 con 0x7f362810d280 2024-03-21T10:24:48.088 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.087+0000 7f3626000700 1 --2- 172.21.15.153:0/1956508048 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f361006e460 0x7f3610070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:48.088 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.087+0000 7f3626000700 1 -- 172.21.15.153:0/1956508048 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f361408fa50 con 0x7f362810d280 2024-03-21T10:24:48.089 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.087+0000 7f362d600700 1 --2- 172.21.15.153:0/1956508048 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f361006e460 0x7f3610070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:48.089 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.088+0000 7f362d600700 1 --2- 172.21.15.153:0/1956508048 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f361006e460 0x7f3610070920 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f361c005fd0 tx=0x7f361c00a560 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:48.093 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.092+0000 7f3626000700 1 -- 172.21.15.153:0/1956508048 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f3614058ba0 con 0x7f362810d280 2024-03-21T10:24:48.323 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.321+0000 7f3630000700 1 -- 172.21.15.153:0/1956508048 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f362802cd60 con 0x7f361006e460 2024-03-21T10:24:48.335 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.333+0000 7f3626000700 1 -- 172.21.15.153:0/1956508048 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f362802cd60 con 0x7f361006e460 2024-03-21T10:24:48.336 INFO:teuthology.orchestra.run.smithi153.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-21T10:24:48.336 INFO:teuthology.orchestra.run.smithi153.stdout:alertmanager.smithi153 smithi153 *:9093,9094 running (13m) 5m ago 27m 14.7M - 0.25.0 c8568f914cd2 9ed3f2200b5f 2024-03-21T10:24:48.336 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi153 smithi153 running (27m) 5m ago 27m 9424k - 18.2.1 5be31c24972a 5b61994396a6 2024-03-21T10:24:48.336 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi174 smithi174 running (26m) 4m ago 26m 9244k - 18.2.1 5be31c24972a 4f7a5fc3ac28 2024-03-21T10:24:48.336 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi153 smithi153 running (9m) 5m ago 27m 6656k - 19.0.0-2231-g2edfee68 237e43c7a65f 32c21104d5b0 2024-03-21T10:24:48.336 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi174 smithi174 running (9m) 4m ago 26m 6660k - 19.0.0-2231-g2edfee68 237e43c7a65f a04e48967ab6 2024-03-21T10:24:48.336 INFO:teuthology.orchestra.run.smithi153.stdout:grafana.smithi153 smithi153 *:3000 running (13m) 5m ago 26m 73.6M - 9.4.12 2bacad6d85d8 ee24026ab786 2024-03-21T10:24:48.336 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.iuyluj smithi153 running (5m) 5m ago 20m 10.9M - 19.0.0-2231-g2edfee68 237e43c7a65f 87aff8a5de70 2024-03-21T10:24:48.336 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.zipckr smithi153 running (5m) 5m ago 20m 11.8M - 19.0.0-2231-g2edfee68 237e43c7a65f ccc262bd5dd3 2024-03-21T10:24:48.336 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.nhozag smithi174 running (20m) 4m ago 20m 12.0M - 18.2.1 5be31c24972a 2fee4000308f 2024-03-21T10:24:48.336 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.yyimst smithi174 running (4m) 4m ago 20m 10.7M - 19.0.0-2231-g2edfee68 237e43c7a65f fc1962f9351e 2024-03-21T10:24:48.336 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi153.tluxni smithi153 *:8443,9283,8765 running (16m) 5m ago 29m 509M - 19.0.0-2231-g2edfee68 237e43c7a65f 7ffee0e55382 2024-03-21T10:24:48.337 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi174.qpcwat smithi174 *:8443,9283,8765 running (15m) 4m ago 25m 452M - 19.0.0-2231-g2edfee68 237e43c7a65f 748968f216ab 2024-03-21T10:24:48.337 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi153 smithi153 running (11m) 5m ago 29m 55.0M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f dd8038303dc3 2024-03-21T10:24:48.337 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi174 smithi174 running (10m) 4m ago 25m 44.2M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f 29d78a788f2c 2024-03-21T10:24:48.337 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi153 smithi153 *:9100 running (14m) 5m ago 27m 7889k - 1.5.0 0da6a335fe13 bb835edc0e0b 2024-03-21T10:24:48.337 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi174 smithi174 *:9100 running (14m) 4m ago 25m 8102k - 1.5.0 0da6a335fe13 7c4ecad790fa 2024-03-21T10:24:48.337 INFO:teuthology.orchestra.run.smithi153.stdout:osd.0 smithi153 running (9m) 5m ago 24m 116M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 507900fd4791 2024-03-21T10:24:48.337 INFO:teuthology.orchestra.run.smithi153.stdout:osd.1 smithi153 running (8m) 5m ago 24m 89.2M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 0d646452e9c1 2024-03-21T10:24:48.337 INFO:teuthology.orchestra.run.smithi153.stdout:osd.2 smithi153 running (7m) 5m ago 23m 87.4M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 1326f2afdfcd 2024-03-21T10:24:48.337 INFO:teuthology.orchestra.run.smithi153.stdout:osd.3 smithi174 running (7m) 4m ago 23m 83.6M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 70aafac71ad5 2024-03-21T10:24:48.337 INFO:teuthology.orchestra.run.smithi153.stdout:osd.4 smithi174 running (6m) 4m ago 22m 92.2M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 0d746ce6e79d 2024-03-21T10:24:48.337 INFO:teuthology.orchestra.run.smithi153.stdout:osd.5 smithi174 running (5m) 4m ago 21m 86.2M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 53e5f35b8cac 2024-03-21T10:24:48.337 INFO:teuthology.orchestra.run.smithi153.stdout:prometheus.smithi153 smithi153 *:9095 running (14m) 5m ago 26m 73.5M - 2.43.0 a07b618ecd1d 7d95561e50c2 2024-03-21T10:24:48.338 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.336+0000 7f3630000700 1 -- 172.21.15.153:0/1956508048 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f361006e460 msgr2=0x7f3610070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:48.338 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.336+0000 7f3630000700 1 --2- 172.21.15.153:0/1956508048 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f361006e460 0x7f3610070920 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f361c005fd0 tx=0x7f361c00a560 comp rx=0 tx=0).stop 2024-03-21T10:24:48.338 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.336+0000 7f3630000700 1 -- 172.21.15.153:0/1956508048 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f362810d280 msgr2=0x7f362819d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:48.338 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.336+0000 7f3630000700 1 --2- 172.21.15.153:0/1956508048 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f362810d280 0x7f362819d260 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f3614006010 tx=0x7f361400be20 comp rx=0 tx=0).stop 2024-03-21T10:24:48.339 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.336+0000 7f3630000700 1 -- 172.21.15.153:0/1956508048 shutdown_connections 2024-03-21T10:24:48.339 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.336+0000 7f3630000700 1 --2- 172.21.15.153:0/1956508048 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f36281071d0 0x7f362819cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:48.339 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.336+0000 7f3630000700 1 --2- 172.21.15.153:0/1956508048 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f361006e460 0x7f3610070920 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:48.339 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.336+0000 7f3630000700 1 --2- 172.21.15.153:0/1956508048 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f362810d280 0x7f362819d260 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:48.339 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.336+0000 7f3630000700 1 -- 172.21.15.153:0/1956508048 >> 172.21.15.153:0/1956508048 conn(0x7f3628076990 msgr2=0x7f3628078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:48.339 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.337+0000 7f3630000700 1 -- 172.21.15.153:0/1956508048 shutdown_connections 2024-03-21T10:24:48.339 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.337+0000 7f3630000700 1 -- 172.21.15.153:0/1956508048 wait complete. 2024-03-21T10:24:48.481 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.479+0000 7fb890e00700 1 -- 172.21.15.153:0/148461623 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb88c107220 msgr2=0x7fb88c107600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:48.481 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.479+0000 7fb890e00700 1 --2- 172.21.15.153:0/148461623 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb88c107220 0x7fb88c107600 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb87c009910 tx=0x7fb87c009c20 comp rx=0 tx=0).stop 2024-03-21T10:24:48.481 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.479+0000 7fb890e00700 1 -- 172.21.15.153:0/148461623 shutdown_connections 2024-03-21T10:24:48.481 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.479+0000 7fb890e00700 1 --2- 172.21.15.153:0/148461623 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb88c107bd0 0x7fb88c10bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:48.481 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.479+0000 7fb890e00700 1 --2- 172.21.15.153:0/148461623 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb88c107220 0x7fb88c107600 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:48.481 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.479+0000 7fb890e00700 1 -- 172.21.15.153:0/148461623 >> 172.21.15.153:0/148461623 conn(0x7fb88c0767e0 msgr2=0x7fb88c078c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:48.481 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.479+0000 7fb890e00700 1 -- 172.21.15.153:0/148461623 shutdown_connections 2024-03-21T10:24:48.481 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.479+0000 7fb890e00700 1 -- 172.21.15.153:0/148461623 wait complete. 2024-03-21T10:24:48.482 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.480+0000 7fb890e00700 1 Processor -- start 2024-03-21T10:24:48.482 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.480+0000 7fb890e00700 1 -- start start 2024-03-21T10:24:48.483 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.481+0000 7fb890e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb88c107220 0x7fb88c1a2a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:48.483 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.481+0000 7fb890e00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb88c107bd0 0x7fb88c1a2f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:48.483 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.481+0000 7fb890e00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb88c1a35f0 con 0x7fb88c107220 2024-03-21T10:24:48.483 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.481+0000 7fb890e00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb88c19caa0 con 0x7fb88c107bd0 2024-03-21T10:24:48.483 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.481+0000 7fb88b400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb88c107220 0x7fb88c1a2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:48.483 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.481+0000 7fb88b400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb88c107220 0x7fb88c1a2a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:35212/0 (socket says 172.21.15.153:35212) 2024-03-21T10:24:48.483 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.481+0000 7fb88b400700 1 -- 172.21.15.153:0/3704210092 learned_addr learned my addr 172.21.15.153:0/3704210092 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:24:48.484 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.482+0000 7fb883e00700 1 --2- 172.21.15.153:0/3704210092 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb88c107bd0 0x7fb88c1a2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:48.484 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.482+0000 7fb88b400700 1 -- 172.21.15.153:0/3704210092 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb88c107bd0 msgr2=0x7fb88c1a2f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:48.484 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.482+0000 7fb88b400700 1 --2- 172.21.15.153:0/3704210092 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb88c107bd0 0x7fb88c1a2f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:48.484 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.482+0000 7fb88b400700 1 -- 172.21.15.153:0/3704210092 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb87c0095f0 con 0x7fb88c107220 2024-03-21T10:24:48.484 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.482+0000 7fb88b400700 1 --2- 172.21.15.153:0/3704210092 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb88c107220 0x7fb88c1a2a20 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fb87c006010 tx=0x7fb87c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:48.485 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.482+0000 7fb888c00700 1 -- 172.21.15.153:0/3704210092 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb87c01d070 con 0x7fb88c107220 2024-03-21T10:24:48.485 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.482+0000 7fb888c00700 1 -- 172.21.15.153:0/3704210092 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb87c004fb0 con 0x7fb88c107220 2024-03-21T10:24:48.485 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.482+0000 7fb888c00700 1 -- 172.21.15.153:0/3704210092 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb87c00f690 con 0x7fb88c107220 2024-03-21T10:24:48.485 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.482+0000 7fb890e00700 1 -- 172.21.15.153:0/3704210092 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb88c19cd20 con 0x7fb88c107220 2024-03-21T10:24:48.485 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.482+0000 7fb890e00700 1 -- 172.21.15.153:0/3704210092 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb88c19d210 con 0x7fb88c107220 2024-03-21T10:24:48.485 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.484+0000 7fb890e00700 1 -- 172.21.15.153:0/3704210092 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb88c10f5b0 con 0x7fb88c107220 2024-03-21T10:24:48.486 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.485+0000 7fb888c00700 1 -- 172.21.15.153:0/3704210092 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fb87c0228a0 con 0x7fb88c107220 2024-03-21T10:24:48.487 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.485+0000 7fb888c00700 1 --2- 172.21.15.153:0/3704210092 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb87006e460 0x7fb870070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:48.487 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.486+0000 7fb888c00700 1 -- 172.21.15.153:0/3704210092 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fb87c08fc90 con 0x7fb88c107220 2024-03-21T10:24:48.487 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.486+0000 7fb883e00700 1 --2- 172.21.15.153:0/3704210092 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb87006e460 0x7fb870070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:48.488 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.486+0000 7fb883e00700 1 --2- 172.21.15.153:0/3704210092 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb87006e460 0x7fb870070920 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb88c077000 tx=0x7fb874005cf0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:48.491 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.490+0000 7fb888c00700 1 -- 172.21.15.153:0/3704210092 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb87c05c900 con 0x7fb88c107220 2024-03-21T10:24:48.783 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.781+0000 7fb890e00700 1 -- 172.21.15.153:0/3704210092 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fb88c10ef90 con 0x7fb88c107220 2024-03-21T10:24:48.783 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.782+0000 7fb888c00700 1 -- 172.21.15.153:0/3704210092 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+805 (secure 0 0 0) 0x7fb87c027340 con 0x7fb88c107220 2024-03-21T10:24:48.784 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:24:48.784 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": { 2024-03-21T10:24:48.784 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:24:48.784 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:24:48.784 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": { 2024-03-21T10:24:48.784 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:24:48.784 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:24:48.784 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": { 2024-03-21T10:24:48.784 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 6 2024-03-21T10:24:48.785 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:24:48.785 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": { 2024-03-21T10:24:48.785 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-03-21T10:24:48.785 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 3 2024-03-21T10:24:48.785 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:24:48.785 INFO:teuthology.orchestra.run.smithi153.stdout: "overall": { 2024-03-21T10:24:48.785 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-03-21T10:24:48.785 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 13 2024-03-21T10:24:48.785 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-03-21T10:24:48.785 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:24:48.787 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.784+0000 7fb890e00700 1 -- 172.21.15.153:0/3704210092 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb87006e460 msgr2=0x7fb870070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:48.787 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.785+0000 7fb890e00700 1 --2- 172.21.15.153:0/3704210092 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb87006e460 0x7fb870070920 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb88c077000 tx=0x7fb874005cf0 comp rx=0 tx=0).stop 2024-03-21T10:24:48.787 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.785+0000 7fb890e00700 1 -- 172.21.15.153:0/3704210092 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb88c107220 msgr2=0x7fb88c1a2a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:48.787 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.785+0000 7fb890e00700 1 --2- 172.21.15.153:0/3704210092 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb88c107220 0x7fb88c1a2a20 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fb87c006010 tx=0x7fb87c00be20 comp rx=0 tx=0).stop 2024-03-21T10:24:48.787 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.785+0000 7fb890e00700 1 -- 172.21.15.153:0/3704210092 shutdown_connections 2024-03-21T10:24:48.787 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.785+0000 7fb890e00700 1 --2- 172.21.15.153:0/3704210092 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb87006e460 0x7fb870070920 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:48.787 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.785+0000 7fb890e00700 1 --2- 172.21.15.153:0/3704210092 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb88c107220 0x7fb88c1a2a20 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:48.787 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.785+0000 7fb890e00700 1 --2- 172.21.15.153:0/3704210092 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb88c107bd0 0x7fb88c1a2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:48.787 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.785+0000 7fb890e00700 1 -- 172.21.15.153:0/3704210092 >> 172.21.15.153:0/3704210092 conn(0x7fb88c0767e0 msgr2=0x7fb88c077e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:48.787 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.785+0000 7fb890e00700 1 -- 172.21.15.153:0/3704210092 shutdown_connections 2024-03-21T10:24:48.787 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.785+0000 7fb890e00700 1 -- 172.21.15.153:0/3704210092 wait complete. 2024-03-21T10:24:48.930 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.928+0000 7f4995800700 1 -- 172.21.15.153:0/1626834509 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4990106990 msgr2=0x7f4990106d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:48.931 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.928+0000 7f4995800700 1 --2- 172.21.15.153:0/1626834509 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4990106990 0x7f4990106d70 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f4980009910 tx=0x7f4980009c20 comp rx=0 tx=0).stop 2024-03-21T10:24:48.931 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.928+0000 7f4995800700 1 -- 172.21.15.153:0/1626834509 shutdown_connections 2024-03-21T10:24:48.931 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.929+0000 7f4995800700 1 --2- 172.21.15.153:0/1626834509 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4990100990 0x7f4990100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:48.931 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.929+0000 7f4995800700 1 --2- 172.21.15.153:0/1626834509 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4990106990 0x7f4990106d70 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:48.931 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.929+0000 7f4995800700 1 -- 172.21.15.153:0/1626834509 >> 172.21.15.153:0/1626834509 conn(0x7f49900fc480 msgr2=0x7f49900fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:48.931 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.929+0000 7f4995800700 1 -- 172.21.15.153:0/1626834509 shutdown_connections 2024-03-21T10:24:48.931 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.929+0000 7f4995800700 1 -- 172.21.15.153:0/1626834509 wait complete. 2024-03-21T10:24:48.932 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.930+0000 7f4995800700 1 Processor -- start 2024-03-21T10:24:48.932 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.930+0000 7f4995800700 1 -- start start 2024-03-21T10:24:48.932 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.930+0000 7f4995800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4990100990 0x7f499019c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:48.932 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.930+0000 7f4995800700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4990106990 0x7f499019cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:48.932 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.930+0000 7f4995800700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4990196610 con 0x7f4990100990 2024-03-21T10:24:48.932 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.930+0000 7f4995800700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4990196780 con 0x7f4990106990 2024-03-21T10:24:48.932 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.930+0000 7f498fe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4990100990 0x7f499019c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:48.932 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.931+0000 7f498fe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4990100990 0x7f499019c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:35232/0 (socket says 172.21.15.153:35232) 2024-03-21T10:24:48.933 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.931+0000 7f498fe00700 1 -- 172.21.15.153:0/89563876 learned_addr learned my addr 172.21.15.153:0/89563876 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:24:48.934 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.931+0000 7f498fe00700 1 -- 172.21.15.153:0/89563876 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4990106990 msgr2=0x7f499019cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:48.934 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.931+0000 7f498f400700 1 --2- 172.21.15.153:0/89563876 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4990106990 0x7f499019cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:48.934 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.931+0000 7f498fe00700 1 --2- 172.21.15.153:0/89563876 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4990106990 0x7f499019cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:48.934 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.931+0000 7f498fe00700 1 -- 172.21.15.153:0/89563876 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49800095f0 con 0x7f4990100990 2024-03-21T10:24:48.934 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.931+0000 7f498f400700 1 --2- 172.21.15.153:0/89563876 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4990106990 0x7f499019cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:24:48.934 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.931+0000 7f498fe00700 1 --2- 172.21.15.153:0/89563876 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4990100990 0x7f499019c620 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f498000b5c0 tx=0x7f498000be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:48.934 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.931+0000 7f498cc00700 1 -- 172.21.15.153:0/89563876 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f498001d070 con 0x7f4990100990 2024-03-21T10:24:48.934 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.932+0000 7f4995800700 1 -- 172.21.15.153:0/89563876 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4990196a00 con 0x7f4990100990 2024-03-21T10:24:48.934 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.932+0000 7f498cc00700 1 -- 172.21.15.153:0/89563876 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f498000f460 con 0x7f4990100990 2024-03-21T10:24:48.934 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.932+0000 7f498cc00700 1 -- 172.21.15.153:0/89563876 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4980017610 con 0x7f4990100990 2024-03-21T10:24:48.934 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.932+0000 7f4995800700 1 -- 172.21.15.153:0/89563876 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4990196ef0 con 0x7f4990100990 2024-03-21T10:24:48.935 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.933+0000 7f4995800700 1 -- 172.21.15.153:0/89563876 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4990108f90 con 0x7f4990100990 2024-03-21T10:24:48.935 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.934+0000 7f498cc00700 1 -- 172.21.15.153:0/89563876 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f498000f5d0 con 0x7f4990100990 2024-03-21T10:24:48.936 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.935+0000 7f498cc00700 1 --2- 172.21.15.153:0/89563876 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f497406e460 0x7f4974070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:48.937 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.935+0000 7f498cc00700 1 -- 172.21.15.153:0/89563876 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f498008fad0 con 0x7f4990100990 2024-03-21T10:24:48.937 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.935+0000 7f498f400700 1 --2- 172.21.15.153:0/89563876 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f497406e460 0x7f4974070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:48.937 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.936+0000 7f498f400700 1 --2- 172.21.15.153:0/89563876 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f497406e460 0x7f4974070920 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4990197eb0 tx=0x7f497c008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:48.941 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:48.940+0000 7f498cc00700 1 -- 172.21.15.153:0/89563876 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4980058b70 con 0x7f4990100990 2024-03-21T10:24:49.205 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:49.202+0000 7f4995800700 1 -- 172.21.15.153:0/89563876 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f4990197c50 con 0x7f4990100990 2024-03-21T10:24:49.205 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:49.203+0000 7f498cc00700 1 -- 172.21.15.153:0/89563876 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1876 (secure 0 0 0) 0x7f498005c230 con 0x7f4990100990 2024-03-21T10:24:49.206 INFO:teuthology.orchestra.run.smithi153.stdout:e30 2024-03-21T10:24:49.206 INFO:teuthology.orchestra.run.smithi153.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-21T10:24:49.206 INFO:teuthology.orchestra.run.smithi153.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:24:49.206 INFO:teuthology.orchestra.run.smithi153.stdout:legacy client fscid: 1 2024-03-21T10:24:49.207 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:24:49.207 INFO:teuthology.orchestra.run.smithi153.stdout:Filesystem 'cephfs' (1) 2024-03-21T10:24:49.207 INFO:teuthology.orchestra.run.smithi153.stdout:fs_name cephfs 2024-03-21T10:24:49.207 INFO:teuthology.orchestra.run.smithi153.stdout:epoch 23 2024-03-21T10:24:49.207 INFO:teuthology.orchestra.run.smithi153.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-21T10:24:49.207 INFO:teuthology.orchestra.run.smithi153.stdout:created 2024-03-21T10:04:27.086576+0000 2024-03-21T10:24:49.208 INFO:teuthology.orchestra.run.smithi153.stdout:modified 2024-03-21T10:19:05.022092+0000 2024-03-21T10:24:49.208 INFO:teuthology.orchestra.run.smithi153.stdout:tableserver 0 2024-03-21T10:24:49.208 INFO:teuthology.orchestra.run.smithi153.stdout:root 0 2024-03-21T10:24:49.208 INFO:teuthology.orchestra.run.smithi153.stdout:session_timeout 60 2024-03-21T10:24:49.208 INFO:teuthology.orchestra.run.smithi153.stdout:session_autoclose 300 2024-03-21T10:24:49.208 INFO:teuthology.orchestra.run.smithi153.stdout:max_file_size 1099511627776 2024-03-21T10:24:49.209 INFO:teuthology.orchestra.run.smithi153.stdout:max_xattr_size 65536 2024-03-21T10:24:49.209 INFO:teuthology.orchestra.run.smithi153.stdout:required_client_features {} 2024-03-21T10:24:49.209 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure 0 2024-03-21T10:24:49.209 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure_osd_epoch 94 2024-03-21T10:24:49.210 INFO:teuthology.orchestra.run.smithi153.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:24:49.210 INFO:teuthology.orchestra.run.smithi153.stdout:max_mds 2 2024-03-21T10:24:49.210 INFO:teuthology.orchestra.run.smithi153.stdout:in 0,1 2024-03-21T10:24:49.211 INFO:teuthology.orchestra.run.smithi153.stdout:up {} 2024-03-21T10:24:49.211 INFO:teuthology.orchestra.run.smithi153.stdout:failed 0,1 2024-03-21T10:24:49.211 INFO:teuthology.orchestra.run.smithi153.stdout:damaged 2024-03-21T10:24:49.211 INFO:teuthology.orchestra.run.smithi153.stdout:stopped 2024-03-21T10:24:49.211 INFO:teuthology.orchestra.run.smithi153.stdout:data_pools [3] 2024-03-21T10:24:49.211 INFO:teuthology.orchestra.run.smithi153.stdout:metadata_pool 2 2024-03-21T10:24:49.211 INFO:teuthology.orchestra.run.smithi153.stdout:inline_data enabled 2024-03-21T10:24:49.211 INFO:teuthology.orchestra.run.smithi153.stdout:balancer 2024-03-21T10:24:49.211 INFO:teuthology.orchestra.run.smithi153.stdout:bal_rank_mask -1 2024-03-21T10:24:49.212 INFO:teuthology.orchestra.run.smithi153.stdout:standby_count_wanted 1 2024-03-21T10:24:49.212 INFO:teuthology.orchestra.run.smithi153.stdout:qdb_cluster leader: 0 members: 2024-03-21T10:24:49.212 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:24:49.212 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:24:49.212 INFO:teuthology.orchestra.run.smithi153.stdout:Standby daemons: 2024-03-21T10:24:49.212 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:24:49.212 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.zipckr{-1:34442} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6826/1103846295,v1:172.21.15.153:6827/1103846295] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:24:49.212 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.iuyluj{-1:34474} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6828/1738630832,v1:172.21.15.153:6829/1738630832] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:24:49.213 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.nhozag{-1:44269} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.174:6826/3967223227,v1:172.21.15.174:6827/3967223227] compat {c=[1],r=[1],i=[7ff]}] 2024-03-21T10:24:49.213 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.yyimst{-1:44313} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.174:6824/4127786797,v1:172.21.15.174:6825/4127786797] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:24:49.215 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:49.206+0000 7f4995800700 1 -- 172.21.15.153:0/89563876 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f497406e460 msgr2=0x7f4974070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:49.215 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:49.206+0000 7f4995800700 1 --2- 172.21.15.153:0/89563876 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f497406e460 0x7f4974070920 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4990197eb0 tx=0x7f497c008040 comp rx=0 tx=0).stop 2024-03-21T10:24:49.215 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:49.206+0000 7f4995800700 1 -- 172.21.15.153:0/89563876 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4990100990 msgr2=0x7f499019c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:49.215 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:49.206+0000 7f4995800700 1 --2- 172.21.15.153:0/89563876 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4990100990 0x7f499019c620 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f498000b5c0 tx=0x7f498000be00 comp rx=0 tx=0).stop 2024-03-21T10:24:49.215 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:49.207+0000 7f4995800700 1 -- 172.21.15.153:0/89563876 shutdown_connections 2024-03-21T10:24:49.215 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:49.207+0000 7f4995800700 1 --2- 172.21.15.153:0/89563876 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f497406e460 0x7f4974070920 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.215 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:49.207+0000 7f4995800700 1 --2- 172.21.15.153:0/89563876 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4990100990 0x7f499019c620 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.216 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:49.207+0000 7f4995800700 1 --2- 172.21.15.153:0/89563876 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4990106990 0x7f499019cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.216 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:49.207+0000 7f4995800700 1 -- 172.21.15.153:0/89563876 >> 172.21.15.153:0/89563876 conn(0x7f49900fc480 msgr2=0x7f49900fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:49.216 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:49.207+0000 7f4995800700 1 -- 172.21.15.153:0/89563876 shutdown_connections 2024-03-21T10:24:49.216 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:49.207+0000 7f4995800700 1 -- 172.21.15.153:0/89563876 wait complete. 2024-03-21T10:24:49.216 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 30 2024-03-21T10:24:49.363 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:49.361+0000 7f7a4e600700 1 -- 172.21.15.153:0/2826411109 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7a48100dc0 msgr2=0x7f7a481011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:49.363 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:49.361+0000 7f7a4e600700 1 --2- 172.21.15.153:0/2826411109 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7a48100dc0 0x7f7a481011a0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f7a3800b0a0 tx=0x7f7a3800b3b0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.364 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:49.362+0000 7f7a4e600700 1 -- 172.21.15.153:0/2826411109 shutdown_connections 2024-03-21T10:24:49.364 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:49.362+0000 7f7a4e600700 1 --2- 172.21.15.153:0/2826411109 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7a48101770 0x7f7a481057c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.364 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:49.362+0000 7f7a4e600700 1 --2- 172.21.15.153:0/2826411109 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7a48100dc0 0x7f7a481011a0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.364 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:49.362+0000 7f7a4e600700 1 -- 172.21.15.153:0/2826411109 >> 172.21.15.153:0/2826411109 conn(0x7f7a480fc650 msgr2=0x7f7a480fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:49.364 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:49.362+0000 7f7a4e600700 1 -- 172.21.15.153:0/2826411109 shutdown_connections 2024-03-21T10:24:49.364 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:49.362+0000 7f7a4e600700 1 -- 172.21.15.153:0/2826411109 wait complete. 2024-03-21T10:24:49.365 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:49.363+0000 7f7a4e600700 1 Processor -- start 2024-03-21T10:24:49.365 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:49.363+0000 7f7a4e600700 1 -- start start 2024-03-21T10:24:49.365 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:49.363+0000 7f7a4e600700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7a48100dc0 0x7f7a4819e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:49.365 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:49.363+0000 7f7a4e600700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7a48101770 0x7f7a4819ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:49.365 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:49.363+0000 7f7a4e600700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a4819f2c0 con 0x7f7a48101770 2024-03-21T10:24:49.365 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:49.363+0000 7f7a4e600700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a48198770 con 0x7f7a48100dc0 2024-03-21T10:24:49.366 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:49.363+0000 7f7a3ea00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7a48101770 0x7f7a4819ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:49.366 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:49.364+0000 7f7a3ea00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7a48101770 0x7f7a4819ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:35242/0 (socket says 172.21.15.153:35242) 2024-03-21T10:24:49.366 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:49.364+0000 7f7a3ea00700 1 -- 172.21.15.153:0/772895767 learned_addr learned my addr 172.21.15.153:0/772895767 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:24:49.366 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:49.364+0000 7f7a3ea00700 1 -- 172.21.15.153:0/772895767 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7a48100dc0 msgr2=0x7f7a4819e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:49.366 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:49.364+0000 7f7a47400700 1 --2- 172.21.15.153:0/772895767 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7a48100dc0 0x7f7a4819e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:49.366 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:49.364+0000 7f7a3ea00700 1 --2- 172.21.15.153:0/772895767 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7a48100dc0 0x7f7a4819e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.366 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:49.364+0000 7f7a3ea00700 1 -- 172.21.15.153:0/772895767 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a38009d00 con 0x7f7a48101770 2024-03-21T10:24:49.366 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:49.364+0000 7f7a3ea00700 1 --2- 172.21.15.153:0/772895767 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7a48101770 0x7f7a4819ec30 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f7a3000d900 tx=0x7f7a3000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:49.366 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:49.364+0000 7f7a44c00700 1 -- 172.21.15.153:0/772895767 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7a30009900 con 0x7f7a48101770 2024-03-21T10:24:49.367 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:49.364+0000 7f7a44c00700 1 -- 172.21.15.153:0/772895767 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7a30010460 con 0x7f7a48101770 2024-03-21T10:24:49.367 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:49.364+0000 7f7a4e600700 1 -- 172.21.15.153:0/772895767 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a48198a50 con 0x7f7a48101770 2024-03-21T10:24:49.367 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:49.365+0000 7f7a44c00700 1 -- 172.21.15.153:0/772895767 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7a3000f5f0 con 0x7f7a48101770 2024-03-21T10:24:49.367 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:49.365+0000 7f7a4e600700 1 -- 172.21.15.153:0/772895767 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a48198fa0 con 0x7f7a48101770 2024-03-21T10:24:49.367 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:49.366+0000 7f7a4e600700 1 -- 172.21.15.153:0/772895767 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a48104f40 con 0x7f7a48101770 2024-03-21T10:24:49.368 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:49.367+0000 7f7a44c00700 1 -- 172.21.15.153:0/772895767 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f7a300105d0 con 0x7f7a48101770 2024-03-21T10:24:49.369 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:49.367+0000 7f7a44c00700 1 --2- 172.21.15.153:0/772895767 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f7a2c06e460 0x7f7a2c070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:49.369 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:49.368+0000 7f7a44c00700 1 -- 172.21.15.153:0/772895767 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f7a3008e410 con 0x7f7a48101770 2024-03-21T10:24:49.370 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:49.368+0000 7f7a47400700 1 --2- 172.21.15.153:0/772895767 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f7a2c06e460 0x7f7a2c070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:49.370 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:49.368+0000 7f7a47400700 1 --2- 172.21.15.153:0/772895767 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f7a2c06e460 0x7f7a2c070920 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f7a3800b830 tx=0x7f7a38006000 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:49.374 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:49.372+0000 7f7a44c00700 1 -- 172.21.15.153:0/772895767 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7a3005af60 con 0x7f7a48101770 2024-03-21T10:24:49.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:24:49 smithi153 ceph-mon[113733]: pgmap v388: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:24:49.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:24:49 smithi153 ceph-mon[113733]: from='client.44409 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:24:49.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:24:49 smithi153 ceph-mon[113733]: from='client.34734 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:24:49.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:24:49 smithi153 ceph-mon[113733]: from='client.34738 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:24:49.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:24:49 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/3704210092' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:24:49.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:24:49 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/89563876' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:24:49.616 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:49.614+0000 7f7a4e600700 1 -- 172.21.15.153:0/772895767 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7a48199230 con 0x7f7a2c06e460 2024-03-21T10:24:49.618 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:49.616+0000 7f7a44c00700 1 -- 172.21.15.153:0/772895767 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f7a48199230 con 0x7f7a2c06e460 2024-03-21T10:24:49.619 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:24:49.619 INFO:teuthology.orchestra.run.smithi153.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232", 2024-03-21T10:24:49.619 INFO:teuthology.orchestra.run.smithi153.stdout: "in_progress": true, 2024-03-21T10:24:49.619 INFO:teuthology.orchestra.run.smithi153.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-21T10:24:49.619 INFO:teuthology.orchestra.run.smithi153.stdout: "services_complete": [ 2024-03-21T10:24:49.619 INFO:teuthology.orchestra.run.smithi153.stdout: "crash", 2024-03-21T10:24:49.619 INFO:teuthology.orchestra.run.smithi153.stdout: "mon", 2024-03-21T10:24:49.619 INFO:teuthology.orchestra.run.smithi153.stdout: "osd", 2024-03-21T10:24:49.619 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr" 2024-03-21T10:24:49.619 INFO:teuthology.orchestra.run.smithi153.stdout: ], 2024-03-21T10:24:49.619 INFO:teuthology.orchestra.run.smithi153.stdout: "progress": "15/23 daemons upgraded", 2024-03-21T10:24:49.619 INFO:teuthology.orchestra.run.smithi153.stdout: "message": "Currently upgrading mds daemons", 2024-03-21T10:24:49.619 INFO:teuthology.orchestra.run.smithi153.stdout: "is_paused": false 2024-03-21T10:24:49.619 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:24:49.621 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:49.619+0000 7f7a4e600700 1 -- 172.21.15.153:0/772895767 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f7a2c06e460 msgr2=0x7f7a2c070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:49.621 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:49.619+0000 7f7a4e600700 1 --2- 172.21.15.153:0/772895767 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f7a2c06e460 0x7f7a2c070920 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f7a3800b830 tx=0x7f7a38006000 comp rx=0 tx=0).stop 2024-03-21T10:24:49.621 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:49.619+0000 7f7a4e600700 1 -- 172.21.15.153:0/772895767 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7a48101770 msgr2=0x7f7a4819ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:49.621 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:49.619+0000 7f7a4e600700 1 --2- 172.21.15.153:0/772895767 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7a48101770 0x7f7a4819ec30 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f7a3000d900 tx=0x7f7a3000dcc0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.621 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:49.619+0000 7f7a4e600700 1 -- 172.21.15.153:0/772895767 shutdown_connections 2024-03-21T10:24:49.621 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:49.619+0000 7f7a4e600700 1 --2- 172.21.15.153:0/772895767 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7a48100dc0 0x7f7a4819e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.622 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:49.619+0000 7f7a4e600700 1 --2- 172.21.15.153:0/772895767 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f7a2c06e460 0x7f7a2c070920 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.622 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:49.619+0000 7f7a4e600700 1 --2- 172.21.15.153:0/772895767 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7a48101770 0x7f7a4819ec30 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.622 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:49.619+0000 7f7a4e600700 1 -- 172.21.15.153:0/772895767 >> 172.21.15.153:0/772895767 conn(0x7f7a480fc650 msgr2=0x7f7a480fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:49.622 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:49.620+0000 7f7a4e600700 1 -- 172.21.15.153:0/772895767 shutdown_connections 2024-03-21T10:24:49.622 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:49.620+0000 7f7a4e600700 1 -- 172.21.15.153:0/772895767 wait complete. 2024-03-21T10:24:49.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:24:49 smithi174 ceph-mon[88004]: pgmap v388: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:24:49.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:24:49 smithi174 ceph-mon[88004]: from='client.44409 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:24:49.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:24:49 smithi174 ceph-mon[88004]: from='client.34734 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:24:49.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:24:49 smithi174 ceph-mon[88004]: from='client.34738 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:24:49.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:24:49 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/3704210092' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:24:49.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:24:49 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/89563876' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:24:49.783 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:49.778+0000 7fc075200700 1 -- 172.21.15.153:0/4061690109 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc070107220 msgr2=0x7fc070107600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:49.784 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:49.778+0000 7fc075200700 1 --2- 172.21.15.153:0/4061690109 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc070107220 0x7fc070107600 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fc060009910 tx=0x7fc060009c20 comp rx=0 tx=0).stop 2024-03-21T10:24:49.784 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:49.778+0000 7fc075200700 1 -- 172.21.15.153:0/4061690109 shutdown_connections 2024-03-21T10:24:49.784 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:49.778+0000 7fc075200700 1 --2- 172.21.15.153:0/4061690109 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc070107bd0 0x7fc07010bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.784 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:49.778+0000 7fc075200700 1 --2- 172.21.15.153:0/4061690109 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc070107220 0x7fc070107600 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.784 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:49.778+0000 7fc075200700 1 -- 172.21.15.153:0/4061690109 >> 172.21.15.153:0/4061690109 conn(0x7fc0700767e0 msgr2=0x7fc070078c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:49.784 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:49.778+0000 7fc075200700 1 -- 172.21.15.153:0/4061690109 shutdown_connections 2024-03-21T10:24:49.785 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:49.778+0000 7fc075200700 1 -- 172.21.15.153:0/4061690109 wait complete. 2024-03-21T10:24:49.785 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:49.779+0000 7fc075200700 1 Processor -- start 2024-03-21T10:24:49.785 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:49.779+0000 7fc075200700 1 -- start start 2024-03-21T10:24:49.785 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:49.780+0000 7fc075200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc070107220 0x7fc0701a2a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:49.785 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:49.780+0000 7fc075200700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc070107bd0 0x7fc0701a2f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:49.785 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:49.780+0000 7fc075200700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0701a35f0 con 0x7fc070107220 2024-03-21T10:24:49.785 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:49.780+0000 7fc075200700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc07019caa0 con 0x7fc070107bd0 2024-03-21T10:24:49.785 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:49.780+0000 7fc06f400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc070107220 0x7fc0701a2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:49.785 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:49.780+0000 7fc06f400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc070107220 0x7fc0701a2a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:35266/0 (socket says 172.21.15.153:35266) 2024-03-21T10:24:49.786 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:49.780+0000 7fc06f400700 1 -- 172.21.15.153:0/382439726 learned_addr learned my addr 172.21.15.153:0/382439726 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:24:49.786 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:49.780+0000 7fc06ea00700 1 --2- 172.21.15.153:0/382439726 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc070107bd0 0x7fc0701a2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:49.786 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:49.780+0000 7fc06f400700 1 -- 172.21.15.153:0/382439726 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc070107bd0 msgr2=0x7fc0701a2f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:49.786 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:49.780+0000 7fc06f400700 1 --2- 172.21.15.153:0/382439726 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc070107bd0 0x7fc0701a2f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:49.786 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:49.780+0000 7fc06f400700 1 -- 172.21.15.153:0/382439726 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0600095f0 con 0x7fc070107220 2024-03-21T10:24:49.786 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:49.781+0000 7fc06f400700 1 --2- 172.21.15.153:0/382439726 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc070107220 0x7fc0701a2a20 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fc0600092e0 tx=0x7fc06000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:49.786 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:49.781+0000 7fc067e00700 1 -- 172.21.15.153:0/382439726 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc06001d070 con 0x7fc070107220 2024-03-21T10:24:49.786 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:49.781+0000 7fc067e00700 1 -- 172.21.15.153:0/382439726 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc060005e00 con 0x7fc070107220 2024-03-21T10:24:49.787 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:49.781+0000 7fc067e00700 1 -- 172.21.15.153:0/382439726 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc060017400 con 0x7fc070107220 2024-03-21T10:24:49.787 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:49.781+0000 7fc075200700 1 -- 172.21.15.153:0/382439726 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc07019cd20 con 0x7fc070107220 2024-03-21T10:24:49.787 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:49.781+0000 7fc075200700 1 -- 172.21.15.153:0/382439726 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc07019d210 con 0x7fc070107220 2024-03-21T10:24:49.788 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:49.782+0000 7fc075200700 1 -- 172.21.15.153:0/382439726 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc07010f5b0 con 0x7fc070107220 2024-03-21T10:24:49.788 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:49.783+0000 7fc067e00700 1 -- 172.21.15.153:0/382439726 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fc06000f460 con 0x7fc070107220 2024-03-21T10:24:49.789 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:49.784+0000 7fc067e00700 1 --2- 172.21.15.153:0/382439726 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fc05406e460 0x7fc054070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:24:49.789 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:49.784+0000 7fc067e00700 1 -- 172.21.15.153:0/382439726 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fc06008f9b0 con 0x7fc070107220 2024-03-21T10:24:49.789 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:49.784+0000 7fc06ea00700 1 --2- 172.21.15.153:0/382439726 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fc05406e460 0x7fc054070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:24:49.789 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:49.785+0000 7fc06ea00700 1 --2- 172.21.15.153:0/382439726 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fc05406e460 0x7fc054070920 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc05c005950 tx=0x7fc05c0058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:24:49.790 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:49.789+0000 7fc067e00700 1 -- 172.21.15.153:0/382439726 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc06005c580 con 0x7fc070107220 2024-03-21T10:24:50.099 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:50.097+0000 7fc075200700 1 -- 172.21.15.153:0/382439726 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fc07010ef90 con 0x7fc070107220 2024-03-21T10:24:50.099 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:50.098+0000 7fc067e00700 1 -- 172.21.15.153:0/382439726 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7fc06005c110 con 0x7fc070107220 2024-03-21T10:24:50.100 INFO:teuthology.orchestra.run.smithi153.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-03-21T10:24:50.100 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-21T10:24:50.100 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is degraded 2024-03-21T10:24:50.100 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-21T10:24:50.100 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-21T10:24:50.100 INFO:teuthology.orchestra.run.smithi153.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-21T10:24:50.100 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-21T10:24:50.107 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:50.101+0000 7fc075200700 1 -- 172.21.15.153:0/382439726 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fc05406e460 msgr2=0x7fc054070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:50.107 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:50.101+0000 7fc075200700 1 --2- 172.21.15.153:0/382439726 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fc05406e460 0x7fc054070920 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc05c005950 tx=0x7fc05c0058e0 comp rx=0 tx=0).stop 2024-03-21T10:24:50.107 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:50.101+0000 7fc075200700 1 -- 172.21.15.153:0/382439726 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc070107220 msgr2=0x7fc0701a2a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:24:50.107 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:50.101+0000 7fc075200700 1 --2- 172.21.15.153:0/382439726 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc070107220 0x7fc0701a2a20 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fc0600092e0 tx=0x7fc06000be20 comp rx=0 tx=0).stop 2024-03-21T10:24:50.107 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:50.101+0000 7fc075200700 1 -- 172.21.15.153:0/382439726 shutdown_connections 2024-03-21T10:24:50.107 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:50.101+0000 7fc075200700 1 --2- 172.21.15.153:0/382439726 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fc05406e460 0x7fc054070920 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:50.107 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:50.101+0000 7fc075200700 1 --2- 172.21.15.153:0/382439726 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc070107220 0x7fc0701a2a20 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:50.108 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:50.101+0000 7fc075200700 1 --2- 172.21.15.153:0/382439726 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc070107bd0 0x7fc0701a2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:24:50.108 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:50.101+0000 7fc075200700 1 -- 172.21.15.153:0/382439726 >> 172.21.15.153:0/382439726 conn(0x7fc0700767e0 msgr2=0x7fc070077e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:24:50.108 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:50.101+0000 7fc075200700 1 -- 172.21.15.153:0/382439726 shutdown_connections 2024-03-21T10:24:50.108 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:24:50.101+0000 7fc075200700 1 -- 172.21.15.153:0/382439726 wait complete. 2024-03-21T10:24:50.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:24:50 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/382439726' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:24:50.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:24:50 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/382439726' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:24:51.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:24:51 smithi153 ceph-mon[113733]: pgmap v389: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:24:51.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:24:51 smithi153 ceph-mon[113733]: from='client.34750 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:24:51.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:24:51 smithi174 ceph-mon[88004]: pgmap v389: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:24:51.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:24:51 smithi174 ceph-mon[88004]: from='client.34750 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:24:53.451 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:24:53 smithi174 ceph-mon[88004]: pgmap v390: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:24:53.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:24:53 smithi153 ceph-mon[113733]: pgmap v390: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:24:55.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:24:55 smithi153 ceph-mon[113733]: pgmap v391: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:24:55.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:24:55 smithi174 ceph-mon[88004]: pgmap v391: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:24:57.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:24:57 smithi153 ceph-mon[113733]: pgmap v392: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:24:57.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:24:57 smithi174 ceph-mon[88004]: pgmap v392: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:24:59.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:24:59 smithi153 ceph-mon[113733]: pgmap v393: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:24:59.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:24:59 smithi174 ceph-mon[88004]: pgmap v393: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:25:01.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:25:01 smithi153 ceph-mon[113733]: pgmap v394: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:25:01.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:25:01 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:25:01.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:25:01 smithi174 ceph-mon[88004]: pgmap v394: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:25:01.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:25:01 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:25:03.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:25:03 smithi153 ceph-mon[113733]: pgmap v395: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:25:03.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:25:03 smithi174 ceph-mon[88004]: pgmap v395: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:25:05.621 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:25:05 smithi153 ceph-mon[113733]: pgmap v396: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:25:05.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:25:05 smithi174 ceph-mon[88004]: pgmap v396: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:25:07.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:25:07 smithi174 ceph-mon[88004]: pgmap v397: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:25:07.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:25:07 smithi153 ceph-mon[113733]: pgmap v397: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:25:09.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:25:09 smithi174 ceph-mon[88004]: pgmap v398: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:25:09.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:25:09 smithi153 ceph-mon[113733]: pgmap v398: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:25:11.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:25:11 smithi174 ceph-mon[88004]: pgmap v399: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:25:11.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:25:11 smithi153 ceph-mon[113733]: pgmap v399: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:25:13.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:25:13 smithi174 ceph-mon[88004]: pgmap v400: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:25:13.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:25:13 smithi153 ceph-mon[113733]: pgmap v400: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:25:15.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:25:15 smithi174 ceph-mon[88004]: pgmap v401: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:25:15.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:25:15 smithi153 ceph-mon[113733]: pgmap v401: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:25:16.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:25:16 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:25:16.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:25:16 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:25:17.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:25:17 smithi174 ceph-mon[88004]: pgmap v402: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:25:17.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:25:17 smithi153 ceph-mon[113733]: pgmap v402: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:25:18.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:25:18 smithi174 ceph-mon[88004]: pgmap v403: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:25:18.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:25:18 smithi153 ceph-mon[113733]: pgmap v403: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:25:20.272 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:20.270+0000 7ff9fe200700 1 -- 172.21.15.153:0/4233643104 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff9f8100da0 msgr2=0x7ff9f8101180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:20.272 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:20.270+0000 7ff9fe200700 1 --2- 172.21.15.153:0/4233643104 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff9f8100da0 0x7ff9f8101180 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7ff9e8009910 tx=0x7ff9e8009c20 comp rx=0 tx=0).stop 2024-03-21T10:25:20.272 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:20.270+0000 7ff9fe200700 1 -- 172.21.15.153:0/4233643104 shutdown_connections 2024-03-21T10:25:20.273 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:20.270+0000 7ff9fe200700 1 --2- 172.21.15.153:0/4233643104 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9f8101750 0x7ff9f81057a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:20.273 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:20.270+0000 7ff9fe200700 1 --2- 172.21.15.153:0/4233643104 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff9f8100da0 0x7ff9f8101180 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:20.273 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:20.270+0000 7ff9fe200700 1 -- 172.21.15.153:0/4233643104 >> 172.21.15.153:0/4233643104 conn(0x7ff9f80fc630 msgr2=0x7ff9f80fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:20.273 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:20.271+0000 7ff9fe200700 1 -- 172.21.15.153:0/4233643104 shutdown_connections 2024-03-21T10:25:20.273 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:20.271+0000 7ff9fe200700 1 -- 172.21.15.153:0/4233643104 wait complete. 2024-03-21T10:25:20.274 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:20.271+0000 7ff9fe200700 1 Processor -- start 2024-03-21T10:25:20.274 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:20.272+0000 7ff9fe200700 1 -- start start 2024-03-21T10:25:20.275 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:20.272+0000 7ff9fe200700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9f8100da0 0x7ff9f819c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:20.275 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:20.272+0000 7ff9fe200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff9f8101750 0x7ff9f819cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:20.275 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:20.272+0000 7ff9fe200700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f819d0e0 con 0x7ff9f8101750 2024-03-21T10:25:20.275 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:20.272+0000 7ff9fe200700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f8196620 con 0x7ff9f8100da0 2024-03-21T10:25:20.275 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:20.272+0000 7ff9eea00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff9f8101750 0x7ff9f819cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:20.275 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:20.272+0000 7ff9eea00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff9f8101750 0x7ff9f819cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:60280/0 (socket says 172.21.15.153:60280) 2024-03-21T10:25:20.275 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:20.272+0000 7ff9eea00700 1 -- 172.21.15.153:0/1313955038 learned_addr learned my addr 172.21.15.153:0/1313955038 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:25:20.275 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:20.273+0000 7ff9f7400700 1 --2- 172.21.15.153:0/1313955038 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9f8100da0 0x7ff9f819c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:20.275 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:20.273+0000 7ff9eea00700 1 -- 172.21.15.153:0/1313955038 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9f8100da0 msgr2=0x7ff9f819c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:20.275 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:20.273+0000 7ff9eea00700 1 --2- 172.21.15.153:0/1313955038 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9f8100da0 0x7ff9f819c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:20.275 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:20.273+0000 7ff9eea00700 1 -- 172.21.15.153:0/1313955038 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9e80095f0 con 0x7ff9f8101750 2024-03-21T10:25:20.275 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:20.273+0000 7ff9eea00700 1 --2- 172.21.15.153:0/1313955038 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff9f8101750 0x7ff9f819cae0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7ff9e000c960 tx=0x7ff9e000cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:20.276 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:20.274+0000 7ff9f4c00700 1 -- 172.21.15.153:0/1313955038 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff9e0007a70 con 0x7ff9f8101750 2024-03-21T10:25:20.276 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:20.274+0000 7ff9f4c00700 1 -- 172.21.15.153:0/1313955038 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff9e0007bd0 con 0x7ff9f8101750 2024-03-21T10:25:20.276 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:20.274+0000 7ff9f4c00700 1 -- 172.21.15.153:0/1313955038 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff9e00186c0 con 0x7ff9f8101750 2024-03-21T10:25:20.276 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:20.274+0000 7ff9fe200700 1 -- 172.21.15.153:0/1313955038 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9f8196900 con 0x7ff9f8101750 2024-03-21T10:25:20.276 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:20.274+0000 7ff9fe200700 1 -- 172.21.15.153:0/1313955038 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9f8196e50 con 0x7ff9f8101750 2024-03-21T10:25:20.277 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:20.275+0000 7ff9fe200700 1 -- 172.21.15.153:0/1313955038 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9f8109140 con 0x7ff9f8101750 2024-03-21T10:25:20.277 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:20.276+0000 7ff9f4c00700 1 -- 172.21.15.153:0/1313955038 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7ff9e000faa0 con 0x7ff9f8101750 2024-03-21T10:25:20.279 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:20.277+0000 7ff9f4c00700 1 --2- 172.21.15.153:0/1313955038 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff9dc06e460 0x7ff9dc070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:20.279 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:20.277+0000 7ff9f4c00700 1 -- 172.21.15.153:0/1313955038 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7ff9e008e520 con 0x7ff9f8101750 2024-03-21T10:25:20.279 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:20.277+0000 7ff9f7400700 1 --2- 172.21.15.153:0/1313955038 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff9dc06e460 0x7ff9dc070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:20.279 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:20.278+0000 7ff9f7400700 1 --2- 172.21.15.153:0/1313955038 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff9dc06e460 0x7ff9dc070920 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff9e8006010 tx=0x7ff9e801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:20.283 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:20.282+0000 7ff9f4c00700 1 -- 172.21.15.153:0/1313955038 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff9e005ae60 con 0x7ff9f8101750 2024-03-21T10:25:20.528 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:20.526+0000 7ff9fe200700 1 -- 172.21.15.153:0/1313955038 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff9f8197170 con 0x7ff9dc06e460 2024-03-21T10:25:20.528 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:25:20 smithi153 ceph-mon[113733]: pgmap v404: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:25:20.529 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:20.528+0000 7ff9f4c00700 1 -- 172.21.15.153:0/1313955038 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7ff9f8197170 con 0x7ff9dc06e460 2024-03-21T10:25:20.532 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:20.531+0000 7ff9fe200700 1 -- 172.21.15.153:0/1313955038 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff9dc06e460 msgr2=0x7ff9dc070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:20.532 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:20.531+0000 7ff9fe200700 1 --2- 172.21.15.153:0/1313955038 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff9dc06e460 0x7ff9dc070920 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff9e8006010 tx=0x7ff9e801a040 comp rx=0 tx=0).stop 2024-03-21T10:25:20.533 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:20.531+0000 7ff9fe200700 1 -- 172.21.15.153:0/1313955038 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff9f8101750 msgr2=0x7ff9f819cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:20.533 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:20.531+0000 7ff9fe200700 1 --2- 172.21.15.153:0/1313955038 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff9f8101750 0x7ff9f819cae0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7ff9e000c960 tx=0x7ff9e000cd20 comp rx=0 tx=0).stop 2024-03-21T10:25:20.533 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:20.531+0000 7ff9fe200700 1 -- 172.21.15.153:0/1313955038 shutdown_connections 2024-03-21T10:25:20.533 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:20.531+0000 7ff9fe200700 1 --2- 172.21.15.153:0/1313955038 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9f8100da0 0x7ff9f819c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:20.533 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:20.531+0000 7ff9fe200700 1 --2- 172.21.15.153:0/1313955038 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff9dc06e460 0x7ff9dc070920 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:20.533 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:20.531+0000 7ff9fe200700 1 --2- 172.21.15.153:0/1313955038 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff9f8101750 0x7ff9f819cae0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:20.534 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:20.531+0000 7ff9fe200700 1 -- 172.21.15.153:0/1313955038 >> 172.21.15.153:0/1313955038 conn(0x7ff9f80fc630 msgr2=0x7ff9f80fdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:20.534 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:20.532+0000 7ff9fe200700 1 -- 172.21.15.153:0/1313955038 shutdown_connections 2024-03-21T10:25:20.534 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:20.532+0000 7ff9fe200700 1 -- 172.21.15.153:0/1313955038 wait complete. 2024-03-21T10:25:20.549 INFO:teuthology.orchestra.run.smithi153.stdout:true 2024-03-21T10:25:20.688 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:20.686+0000 7f9722600700 1 -- 172.21.15.153:0/3052277461 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f971c100da0 msgr2=0x7f971c101180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:20.688 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:20.686+0000 7f9722600700 1 --2- 172.21.15.153:0/3052277461 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f971c100da0 0x7f971c101180 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f970c00b0a0 tx=0x7f970c00b3b0 comp rx=0 tx=0).stop 2024-03-21T10:25:20.689 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:20.686+0000 7f9722600700 1 -- 172.21.15.153:0/3052277461 shutdown_connections 2024-03-21T10:25:20.689 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:20.686+0000 7f9722600700 1 --2- 172.21.15.153:0/3052277461 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f971c101750 0x7f971c1057a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:20.689 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:20.686+0000 7f9722600700 1 --2- 172.21.15.153:0/3052277461 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f971c100da0 0x7f971c101180 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:20.689 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:20.686+0000 7f9722600700 1 -- 172.21.15.153:0/3052277461 >> 172.21.15.153:0/3052277461 conn(0x7f971c0fc630 msgr2=0x7f971c0fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:20.689 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:20.687+0000 7f9722600700 1 -- 172.21.15.153:0/3052277461 shutdown_connections 2024-03-21T10:25:20.689 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:20.687+0000 7f9722600700 1 -- 172.21.15.153:0/3052277461 wait complete. 2024-03-21T10:25:20.691 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:20.688+0000 7f9722600700 1 Processor -- start 2024-03-21T10:25:20.691 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:20.688+0000 7f9722600700 1 -- start start 2024-03-21T10:25:20.691 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:20.688+0000 7f9722600700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f971c100da0 0x7f971c19c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:20.691 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:20.688+0000 7f9722600700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f971c101750 0x7f971c19cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:20.692 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:20.688+0000 7f9722600700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f971c19d150 con 0x7f971c101750 2024-03-21T10:25:20.692 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:20.688+0000 7f9722600700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f971c196600 con 0x7f971c100da0 2024-03-21T10:25:20.692 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:20.688+0000 7f971aa00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f971c101750 0x7f971c19cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:20.692 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:20.689+0000 7f971aa00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f971c101750 0x7f971c19cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:60294/0 (socket says 172.21.15.153:60294) 2024-03-21T10:25:20.692 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:20.689+0000 7f971aa00700 1 -- 172.21.15.153:0/28911867 learned_addr learned my addr 172.21.15.153:0/28911867 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:25:20.692 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:20.689+0000 7f971b400700 1 --2- 172.21.15.153:0/28911867 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f971c100da0 0x7f971c19c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:20.692 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:20.689+0000 7f971aa00700 1 -- 172.21.15.153:0/28911867 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f971c100da0 msgr2=0x7f971c19c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:20.692 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:20.689+0000 7f971aa00700 1 --2- 172.21.15.153:0/28911867 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f971c100da0 0x7f971c19c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:20.692 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:20.689+0000 7f971aa00700 1 -- 172.21.15.153:0/28911867 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f970c009d00 con 0x7f971c101750 2024-03-21T10:25:20.692 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:20.689+0000 7f971b400700 1 --2- 172.21.15.153:0/28911867 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f971c100da0 0x7f971c19c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:25:20.693 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:20.689+0000 7f971aa00700 1 --2- 172.21.15.153:0/28911867 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f971c101750 0x7f971c19cac0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f970800d900 tx=0x7f970800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:20.693 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:20.689+0000 7f9713e00700 1 -- 172.21.15.153:0/28911867 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9708009900 con 0x7f971c101750 2024-03-21T10:25:20.693 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:20.690+0000 7f9713e00700 1 -- 172.21.15.153:0/28911867 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9708010460 con 0x7f971c101750 2024-03-21T10:25:20.693 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:20.690+0000 7f9713e00700 1 -- 172.21.15.153:0/28911867 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f970800f5f0 con 0x7f971c101750 2024-03-21T10:25:20.693 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:20.690+0000 7f9722600700 1 -- 172.21.15.153:0/28911867 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f971c1968e0 con 0x7f971c101750 2024-03-21T10:25:20.693 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:20.690+0000 7f9722600700 1 -- 172.21.15.153:0/28911867 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f971c196e30 con 0x7f971c101750 2024-03-21T10:25:20.694 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:20.691+0000 7f9722600700 1 -- 172.21.15.153:0/28911867 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f971c109140 con 0x7f971c101750 2024-03-21T10:25:20.694 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:20.692+0000 7f9713e00700 1 -- 172.21.15.153:0/28911867 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f97080105d0 con 0x7f971c101750 2024-03-21T10:25:20.694 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:20.693+0000 7f9713e00700 1 --2- 172.21.15.153:0/28911867 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f970006e460 0x7f9700070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:20.695 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:20.693+0000 7f9713e00700 1 -- 172.21.15.153:0/28911867 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f970808e360 con 0x7f971c101750 2024-03-21T10:25:20.695 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:20.693+0000 7f971b400700 1 --2- 172.21.15.153:0/28911867 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f970006e460 0x7f9700070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:20.695 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:20.694+0000 7f971b400700 1 --2- 172.21.15.153:0/28911867 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f970006e460 0x7f9700070920 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f970c009480 tx=0x7f970c000f40 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:20.699 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:20.698+0000 7f9713e00700 1 -- 172.21.15.153:0/28911867 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f970805af30 con 0x7f971c101750 2024-03-21T10:25:20.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:25:20 smithi174 ceph-mon[88004]: pgmap v404: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:25:20.943 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:20.941+0000 7f9722600700 1 -- 172.21.15.153:0/28911867 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f971c197130 con 0x7f970006e460 2024-03-21T10:25:20.946 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:20.944+0000 7f9713e00700 1 -- 172.21.15.153:0/28911867 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f971c197130 con 0x7f970006e460 2024-03-21T10:25:20.949 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:20.947+0000 7f9722600700 1 -- 172.21.15.153:0/28911867 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f970006e460 msgr2=0x7f9700070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:20.949 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:20.947+0000 7f9722600700 1 --2- 172.21.15.153:0/28911867 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f970006e460 0x7f9700070920 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f970c009480 tx=0x7f970c000f40 comp rx=0 tx=0).stop 2024-03-21T10:25:20.949 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:20.947+0000 7f9722600700 1 -- 172.21.15.153:0/28911867 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f971c101750 msgr2=0x7f971c19cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:20.949 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:20.947+0000 7f9722600700 1 --2- 172.21.15.153:0/28911867 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f971c101750 0x7f971c19cac0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f970800d900 tx=0x7f970800dcc0 comp rx=0 tx=0).stop 2024-03-21T10:25:20.949 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:20.947+0000 7f9722600700 1 -- 172.21.15.153:0/28911867 shutdown_connections 2024-03-21T10:25:20.949 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:20.947+0000 7f9722600700 1 --2- 172.21.15.153:0/28911867 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f971c100da0 0x7f971c19c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:20.950 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:20.947+0000 7f9722600700 1 --2- 172.21.15.153:0/28911867 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f970006e460 0x7f9700070920 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:20.950 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:20.947+0000 7f9722600700 1 --2- 172.21.15.153:0/28911867 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f971c101750 0x7f971c19cac0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:20.950 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:20.947+0000 7f9722600700 1 -- 172.21.15.153:0/28911867 >> 172.21.15.153:0/28911867 conn(0x7f971c0fc630 msgr2=0x7f971c0fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:20.950 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:20.948+0000 7f9722600700 1 -- 172.21.15.153:0/28911867 shutdown_connections 2024-03-21T10:25:20.950 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:20.948+0000 7f9722600700 1 -- 172.21.15.153:0/28911867 wait complete. 2024-03-21T10:25:21.099 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.097+0000 7ffad6a00700 1 -- 172.21.15.153:0/2304905963 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffad0073130 msgr2=0x7ffad0073510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:21.099 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.097+0000 7ffad6a00700 1 --2- 172.21.15.153:0/2304905963 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffad0073130 0x7ffad0073510 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7ffac0009910 tx=0x7ffac0009c20 comp rx=0 tx=0).stop 2024-03-21T10:25:21.099 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.097+0000 7ffad6a00700 1 -- 172.21.15.153:0/2304905963 shutdown_connections 2024-03-21T10:25:21.099 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.097+0000 7ffad6a00700 1 --2- 172.21.15.153:0/2304905963 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffad0073a50 0x7ffad0111610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:21.099 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.097+0000 7ffad6a00700 1 --2- 172.21.15.153:0/2304905963 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffad0073130 0x7ffad0073510 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:21.099 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.097+0000 7ffad6a00700 1 -- 172.21.15.153:0/2304905963 >> 172.21.15.153:0/2304905963 conn(0x7ffad00fc630 msgr2=0x7ffad00fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:21.100 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.097+0000 7ffad6a00700 1 -- 172.21.15.153:0/2304905963 shutdown_connections 2024-03-21T10:25:21.100 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.097+0000 7ffad6a00700 1 -- 172.21.15.153:0/2304905963 wait complete. 2024-03-21T10:25:21.101 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.098+0000 7ffad6a00700 1 Processor -- start 2024-03-21T10:25:21.101 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.099+0000 7ffad6a00700 1 -- start start 2024-03-21T10:25:21.101 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.099+0000 7ffad6a00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffad0073130 0x7ffad019cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:21.101 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.099+0000 7ffad6a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffad0073a50 0x7ffad019d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:21.101 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.099+0000 7ffad6a00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffad019d870 con 0x7ffad0073a50 2024-03-21T10:25:21.101 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.099+0000 7ffad6a00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffad01a05a0 con 0x7ffad0073130 2024-03-21T10:25:21.102 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.099+0000 7ffacf400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffad0073a50 0x7ffad019d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:21.102 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.099+0000 7ffacf400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffad0073a50 0x7ffad019d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:60306/0 (socket says 172.21.15.153:60306) 2024-03-21T10:25:21.102 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.099+0000 7ffacf400700 1 -- 172.21.15.153:0/3200201159 learned_addr learned my addr 172.21.15.153:0/3200201159 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:25:21.102 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.100+0000 7ffacfe00700 1 --2- 172.21.15.153:0/3200201159 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffad0073130 0x7ffad019cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:21.102 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.100+0000 7ffacf400700 1 -- 172.21.15.153:0/3200201159 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffad0073130 msgr2=0x7ffad019cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:21.102 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.100+0000 7ffacf400700 1 --2- 172.21.15.153:0/3200201159 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffad0073130 0x7ffad019cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:21.102 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.100+0000 7ffacf400700 1 -- 172.21.15.153:0/3200201159 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffac00095f0 con 0x7ffad0073a50 2024-03-21T10:25:21.102 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.100+0000 7ffacfe00700 1 --2- 172.21.15.153:0/3200201159 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffad0073130 0x7ffad019cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:25:21.102 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.100+0000 7ffacf400700 1 --2- 172.21.15.153:0/3200201159 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffad0073a50 0x7ffad019d1e0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7ffabc00c8a0 tx=0x7ffabc00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:21.103 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.100+0000 7ffaccc00700 1 -- 172.21.15.153:0/3200201159 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffabc00cea0 con 0x7ffad0073a50 2024-03-21T10:25:21.103 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.100+0000 7ffad6a00700 1 -- 172.21.15.153:0/3200201159 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffad01a0880 con 0x7ffad0073a50 2024-03-21T10:25:21.103 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.100+0000 7ffaccc00700 1 -- 172.21.15.153:0/3200201159 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ffabc004830 con 0x7ffad0073a50 2024-03-21T10:25:21.103 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.100+0000 7ffaccc00700 1 -- 172.21.15.153:0/3200201159 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffabc0055b0 con 0x7ffad0073a50 2024-03-21T10:25:21.103 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.100+0000 7ffad6a00700 1 -- 172.21.15.153:0/3200201159 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffad01a0dd0 con 0x7ffad0073a50 2024-03-21T10:25:21.103 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.102+0000 7ffad6a00700 1 -- 172.21.15.153:0/3200201159 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffad010ed90 con 0x7ffad0073a50 2024-03-21T10:25:21.104 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.103+0000 7ffaccc00700 1 -- 172.21.15.153:0/3200201159 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7ffabc0049a0 con 0x7ffad0073a50 2024-03-21T10:25:21.105 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.104+0000 7ffaccc00700 1 --2- 172.21.15.153:0/3200201159 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ffab406e460 0x7ffab4070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:21.105 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.104+0000 7ffaccc00700 1 -- 172.21.15.153:0/3200201159 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7ffabc08e270 con 0x7ffad0073a50 2024-03-21T10:25:21.105 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.104+0000 7ffacfe00700 1 --2- 172.21.15.153:0/3200201159 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ffab406e460 0x7ffab4070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:21.106 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.105+0000 7ffacfe00700 1 --2- 172.21.15.153:0/3200201159 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ffab406e460 0x7ffab4070920 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ffac000b5c0 tx=0x7ffac0005480 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:21.110 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.109+0000 7ffaccc00700 1 -- 172.21.15.153:0/3200201159 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ffabc057290 con 0x7ffad0073a50 2024-03-21T10:25:21.353 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.351+0000 7ffad6a00700 1 -- 172.21.15.153:0/3200201159 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7ffad01a10c0 con 0x7ffab406e460 2024-03-21T10:25:21.384 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.382+0000 7ffaccc00700 1 -- 172.21.15.153:0/3200201159 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7ffad01a10c0 con 0x7ffab406e460 2024-03-21T10:25:21.384 INFO:teuthology.orchestra.run.smithi153.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-21T10:25:21.384 INFO:teuthology.orchestra.run.smithi153.stdout:alertmanager.smithi153 smithi153 *:9093,9094 running (14m) 5m ago 27m 14.7M - 0.25.0 c8568f914cd2 9ed3f2200b5f 2024-03-21T10:25:21.384 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi153 smithi153 running (28m) 5m ago 28m 9424k - 18.2.1 5be31c24972a 5b61994396a6 2024-03-21T10:25:21.385 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi174 smithi174 running (26m) 5m ago 26m 9244k - 18.2.1 5be31c24972a 4f7a5fc3ac28 2024-03-21T10:25:21.385 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi153 smithi153 running (10m) 5m ago 28m 6656k - 19.0.0-2231-g2edfee68 237e43c7a65f 32c21104d5b0 2024-03-21T10:25:21.385 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi174 smithi174 running (10m) 5m ago 26m 6660k - 19.0.0-2231-g2edfee68 237e43c7a65f a04e48967ab6 2024-03-21T10:25:21.385 INFO:teuthology.orchestra.run.smithi153.stdout:grafana.smithi153 smithi153 *:3000 running (13m) 5m ago 27m 73.6M - 9.4.12 2bacad6d85d8 ee24026ab786 2024-03-21T10:25:21.385 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.iuyluj smithi153 running (5m) 5m ago 20m 10.9M - 19.0.0-2231-g2edfee68 237e43c7a65f 87aff8a5de70 2024-03-21T10:25:21.385 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.zipckr smithi153 running (6m) 5m ago 20m 11.8M - 19.0.0-2231-g2edfee68 237e43c7a65f ccc262bd5dd3 2024-03-21T10:25:21.385 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.nhozag smithi174 running (20m) 5m ago 20m 12.0M - 18.2.1 5be31c24972a 2fee4000308f 2024-03-21T10:25:21.385 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.yyimst smithi174 running (5m) 5m ago 20m 10.7M - 19.0.0-2231-g2edfee68 237e43c7a65f fc1962f9351e 2024-03-21T10:25:21.385 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi153.tluxni smithi153 *:8443,9283,8765 running (17m) 5m ago 29m 509M - 19.0.0-2231-g2edfee68 237e43c7a65f 7ffee0e55382 2024-03-21T10:25:21.385 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi174.qpcwat smithi174 *:8443,9283,8765 running (15m) 5m ago 26m 452M - 19.0.0-2231-g2edfee68 237e43c7a65f 748968f216ab 2024-03-21T10:25:21.385 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi153 smithi153 running (12m) 5m ago 30m 55.0M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f dd8038303dc3 2024-03-21T10:25:21.385 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi174 smithi174 running (11m) 5m ago 26m 44.2M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f 29d78a788f2c 2024-03-21T10:25:21.385 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi153 smithi153 *:9100 running (15m) 5m ago 28m 7889k - 1.5.0 0da6a335fe13 bb835edc0e0b 2024-03-21T10:25:21.385 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi174 smithi174 *:9100 running (15m) 5m ago 26m 8102k - 1.5.0 0da6a335fe13 7c4ecad790fa 2024-03-21T10:25:21.386 INFO:teuthology.orchestra.run.smithi153.stdout:osd.0 smithi153 running (9m) 5m ago 25m 116M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 507900fd4791 2024-03-21T10:25:21.386 INFO:teuthology.orchestra.run.smithi153.stdout:osd.1 smithi153 running (8m) 5m ago 24m 89.2M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 0d646452e9c1 2024-03-21T10:25:21.386 INFO:teuthology.orchestra.run.smithi153.stdout:osd.2 smithi153 running (8m) 5m ago 24m 87.4M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 1326f2afdfcd 2024-03-21T10:25:21.386 INFO:teuthology.orchestra.run.smithi153.stdout:osd.3 smithi174 running (7m) 5m ago 23m 83.6M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 70aafac71ad5 2024-03-21T10:25:21.386 INFO:teuthology.orchestra.run.smithi153.stdout:osd.4 smithi174 running (7m) 5m ago 23m 92.2M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 0d746ce6e79d 2024-03-21T10:25:21.386 INFO:teuthology.orchestra.run.smithi153.stdout:osd.5 smithi174 running (6m) 5m ago 22m 86.2M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 53e5f35b8cac 2024-03-21T10:25:21.386 INFO:teuthology.orchestra.run.smithi153.stdout:prometheus.smithi153 smithi153 *:9095 running (14m) 5m ago 27m 73.5M - 2.43.0 a07b618ecd1d 7d95561e50c2 2024-03-21T10:25:21.387 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.385+0000 7ffad6a00700 1 -- 172.21.15.153:0/3200201159 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ffab406e460 msgr2=0x7ffab4070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:21.387 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.385+0000 7ffad6a00700 1 --2- 172.21.15.153:0/3200201159 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ffab406e460 0x7ffab4070920 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ffac000b5c0 tx=0x7ffac0005480 comp rx=0 tx=0).stop 2024-03-21T10:25:21.387 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.385+0000 7ffad6a00700 1 -- 172.21.15.153:0/3200201159 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffad0073a50 msgr2=0x7ffad019d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:21.387 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.385+0000 7ffad6a00700 1 --2- 172.21.15.153:0/3200201159 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffad0073a50 0x7ffad019d1e0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7ffabc00c8a0 tx=0x7ffabc00cc60 comp rx=0 tx=0).stop 2024-03-21T10:25:21.387 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.385+0000 7ffad6a00700 1 -- 172.21.15.153:0/3200201159 shutdown_connections 2024-03-21T10:25:21.387 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.385+0000 7ffad6a00700 1 --2- 172.21.15.153:0/3200201159 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffad0073130 0x7ffad019cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:21.387 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.385+0000 7ffad6a00700 1 --2- 172.21.15.153:0/3200201159 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ffab406e460 0x7ffab4070920 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:21.387 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.385+0000 7ffad6a00700 1 --2- 172.21.15.153:0/3200201159 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffad0073a50 0x7ffad019d1e0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:21.387 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.385+0000 7ffad6a00700 1 -- 172.21.15.153:0/3200201159 >> 172.21.15.153:0/3200201159 conn(0x7ffad00fc630 msgr2=0x7ffad0103120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:21.387 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.385+0000 7ffad6a00700 1 -- 172.21.15.153:0/3200201159 shutdown_connections 2024-03-21T10:25:21.388 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.386+0000 7ffad6a00700 1 -- 172.21.15.153:0/3200201159 wait complete. 2024-03-21T10:25:21.531 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.529+0000 7fb30ba00700 1 -- 172.21.15.153:0/3744774088 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb304069020 msgr2=0x7fb304105790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:21.531 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.529+0000 7fb30ba00700 1 --2- 172.21.15.153:0/3744774088 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb304069020 0x7fb304105790 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fb2f0009910 tx=0x7fb2f0009c20 comp rx=0 tx=0).stop 2024-03-21T10:25:21.531 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.529+0000 7fb30ba00700 1 -- 172.21.15.153:0/3744774088 shutdown_connections 2024-03-21T10:25:21.531 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.529+0000 7fb30ba00700 1 --2- 172.21.15.153:0/3744774088 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb304069020 0x7fb304105790 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:21.531 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.529+0000 7fb30ba00700 1 --2- 172.21.15.153:0/3744774088 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb304068670 0x7fb304068a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:21.531 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.529+0000 7fb30ba00700 1 -- 172.21.15.153:0/3744774088 >> 172.21.15.153:0/3744774088 conn(0x7fb3040755a0 msgr2=0x7fb3040759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:21.531 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.529+0000 7fb30ba00700 1 -- 172.21.15.153:0/3744774088 shutdown_connections 2024-03-21T10:25:21.531 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.529+0000 7fb30ba00700 1 -- 172.21.15.153:0/3744774088 wait complete. 2024-03-21T10:25:21.532 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.530+0000 7fb30ba00700 1 Processor -- start 2024-03-21T10:25:21.532 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.530+0000 7fb30ba00700 1 -- start start 2024-03-21T10:25:21.532 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.531+0000 7fb30ba00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb304068670 0x7fb304196700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:21.532 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.531+0000 7fb30ba00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb304069020 0x7fb304196c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:21.532 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.531+0000 7fb30ba00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3041972d0 con 0x7fb304068670 2024-03-21T10:25:21.532 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.531+0000 7fb30ba00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb30419a000 con 0x7fb304069020 2024-03-21T10:25:21.533 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.531+0000 7fb309000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb304068670 0x7fb304196700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:21.533 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.531+0000 7fb309000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb304068670 0x7fb304196700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:60316/0 (socket says 172.21.15.153:60316) 2024-03-21T10:25:21.533 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.531+0000 7fb309000700 1 -- 172.21.15.153:0/2143475931 learned_addr learned my addr 172.21.15.153:0/2143475931 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:25:21.534 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.531+0000 7fb303e00700 1 --2- 172.21.15.153:0/2143475931 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb304069020 0x7fb304196c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:21.534 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.532+0000 7fb309000700 1 -- 172.21.15.153:0/2143475931 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb304069020 msgr2=0x7fb304196c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:21.534 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.532+0000 7fb309000700 1 --2- 172.21.15.153:0/2143475931 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb304069020 0x7fb304196c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:21.534 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.532+0000 7fb309000700 1 -- 172.21.15.153:0/2143475931 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2f00095f0 con 0x7fb304068670 2024-03-21T10:25:21.534 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.532+0000 7fb309000700 1 --2- 172.21.15.153:0/2143475931 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb304068670 0x7fb304196700 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fb2f800d8d0 tx=0x7fb2f800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:21.534 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.532+0000 7fb301600700 1 -- 172.21.15.153:0/2143475931 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb2f8009940 con 0x7fb304068670 2024-03-21T10:25:21.534 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.532+0000 7fb301600700 1 -- 172.21.15.153:0/2143475931 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb2f8010460 con 0x7fb304068670 2024-03-21T10:25:21.534 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.532+0000 7fb30ba00700 1 -- 172.21.15.153:0/2143475931 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb30419a2e0 con 0x7fb304068670 2024-03-21T10:25:21.535 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.532+0000 7fb301600700 1 -- 172.21.15.153:0/2143475931 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb2f800f5f0 con 0x7fb304068670 2024-03-21T10:25:21.535 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.532+0000 7fb30ba00700 1 -- 172.21.15.153:0/2143475931 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb30419a800 con 0x7fb304068670 2024-03-21T10:25:21.535 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.533+0000 7fb30ba00700 1 -- 172.21.15.153:0/2143475931 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3041090a0 con 0x7fb304068670 2024-03-21T10:25:21.536 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.534+0000 7fb301600700 1 -- 172.21.15.153:0/2143475931 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fb2f80105d0 con 0x7fb304068670 2024-03-21T10:25:21.537 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.535+0000 7fb301600700 1 --2- 172.21.15.153:0/2143475931 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb2ec06e460 0x7fb2ec070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:21.537 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.535+0000 7fb301600700 1 -- 172.21.15.153:0/2143475931 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fb2f808e480 con 0x7fb304068670 2024-03-21T10:25:21.537 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.536+0000 7fb303e00700 1 --2- 172.21.15.153:0/2143475931 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb2ec06e460 0x7fb2ec070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:21.537 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.536+0000 7fb303e00700 1 --2- 172.21.15.153:0/2143475931 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb2ec06e460 0x7fb2ec070920 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb304197e20 tx=0x7fb2f001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:21.542 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.541+0000 7fb301600700 1 -- 172.21.15.153:0/2143475931 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb2f8057520 con 0x7fb304068670 2024-03-21T10:25:21.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:25:21 smithi174 ceph-mon[88004]: from='client.34758 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:25:21.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:25:21 smithi174 ceph-mon[88004]: from='client.34762 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:25:21.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:25:21 smithi153 ceph-mon[113733]: from='client.34758 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:25:21.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:25:21 smithi153 ceph-mon[113733]: from='client.34762 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:25:21.833 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.832+0000 7fb30ba00700 1 -- 172.21.15.153:0/2143475931 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fb30404ea90 con 0x7fb304068670 2024-03-21T10:25:21.834 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.833+0000 7fb301600700 1 -- 172.21.15.153:0/2143475931 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+805 (secure 0 0 0) 0x7fb2f805abe0 con 0x7fb304068670 2024-03-21T10:25:21.834 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:25:21.835 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": { 2024-03-21T10:25:21.835 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:25:21.835 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:25:21.835 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": { 2024-03-21T10:25:21.835 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:25:21.835 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:25:21.835 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": { 2024-03-21T10:25:21.835 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 6 2024-03-21T10:25:21.835 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:25:21.835 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": { 2024-03-21T10:25:21.835 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-03-21T10:25:21.835 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 3 2024-03-21T10:25:21.835 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:25:21.835 INFO:teuthology.orchestra.run.smithi153.stdout: "overall": { 2024-03-21T10:25:21.836 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-03-21T10:25:21.836 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 13 2024-03-21T10:25:21.836 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-03-21T10:25:21.836 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:25:21.837 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.836+0000 7fb30ba00700 1 -- 172.21.15.153:0/2143475931 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb2ec06e460 msgr2=0x7fb2ec070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:21.837 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.836+0000 7fb30ba00700 1 --2- 172.21.15.153:0/2143475931 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb2ec06e460 0x7fb2ec070920 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb304197e20 tx=0x7fb2f001a040 comp rx=0 tx=0).stop 2024-03-21T10:25:21.838 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.836+0000 7fb30ba00700 1 -- 172.21.15.153:0/2143475931 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb304068670 msgr2=0x7fb304196700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:21.838 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.836+0000 7fb30ba00700 1 --2- 172.21.15.153:0/2143475931 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb304068670 0x7fb304196700 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fb2f800d8d0 tx=0x7fb2f800dc90 comp rx=0 tx=0).stop 2024-03-21T10:25:21.838 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.836+0000 7fb30ba00700 1 -- 172.21.15.153:0/2143475931 shutdown_connections 2024-03-21T10:25:21.838 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.836+0000 7fb30ba00700 1 --2- 172.21.15.153:0/2143475931 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb2ec06e460 0x7fb2ec070920 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:21.838 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.836+0000 7fb30ba00700 1 --2- 172.21.15.153:0/2143475931 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb304068670 0x7fb304196700 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:21.838 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.836+0000 7fb30ba00700 1 --2- 172.21.15.153:0/2143475931 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb304069020 0x7fb304196c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:21.838 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.836+0000 7fb30ba00700 1 -- 172.21.15.153:0/2143475931 >> 172.21.15.153:0/2143475931 conn(0x7fb3040755a0 msgr2=0x7fb304105000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:21.838 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.836+0000 7fb30ba00700 1 -- 172.21.15.153:0/2143475931 shutdown_connections 2024-03-21T10:25:21.838 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.836+0000 7fb30ba00700 1 -- 172.21.15.153:0/2143475931 wait complete. 2024-03-21T10:25:21.992 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.990+0000 7ff2b1a00700 1 -- 172.21.15.153:0/3970524059 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff2ac103000 msgr2=0x7ff2ac1033e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:21.992 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.990+0000 7ff2b1a00700 1 --2- 172.21.15.153:0/3970524059 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff2ac103000 0x7ff2ac1033e0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7ff29c00b0a0 tx=0x7ff29c00b3b0 comp rx=0 tx=0).stop 2024-03-21T10:25:21.993 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.990+0000 7ff2b1a00700 1 -- 172.21.15.153:0/3970524059 shutdown_connections 2024-03-21T10:25:21.993 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.990+0000 7ff2b1a00700 1 --2- 172.21.15.153:0/3970524059 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff2ac1039b0 0x7ff2ac107a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:21.993 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.990+0000 7ff2b1a00700 1 --2- 172.21.15.153:0/3970524059 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff2ac103000 0x7ff2ac1033e0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:21.993 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.990+0000 7ff2b1a00700 1 -- 172.21.15.153:0/3970524059 >> 172.21.15.153:0/3970524059 conn(0x7ff2ac0fe8d0 msgr2=0x7ff2ac100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:21.993 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.991+0000 7ff2b1a00700 1 -- 172.21.15.153:0/3970524059 shutdown_connections 2024-03-21T10:25:21.993 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.991+0000 7ff2b1a00700 1 -- 172.21.15.153:0/3970524059 wait complete. 2024-03-21T10:25:21.994 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.992+0000 7ff2b1a00700 1 Processor -- start 2024-03-21T10:25:21.994 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.992+0000 7ff2b1a00700 1 -- start start 2024-03-21T10:25:21.994 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.992+0000 7ff2b1a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff2ac103000 0x7ff2ac19e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:21.994 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.992+0000 7ff2b1a00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff2ac1039b0 0x7ff2ac19ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:21.994 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.992+0000 7ff2b1a00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2ac19f490 con 0x7ff2ac103000 2024-03-21T10:25:21.994 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.992+0000 7ff2b1a00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2ac198940 con 0x7ff2ac1039b0 2024-03-21T10:25:21.994 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.993+0000 7ff2aaa00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff2ac103000 0x7ff2ac19e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:21.994 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.993+0000 7ff2aaa00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff2ac103000 0x7ff2ac19e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:60330/0 (socket says 172.21.15.153:60330) 2024-03-21T10:25:21.995 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.993+0000 7ff2aaa00700 1 -- 172.21.15.153:0/1018400 learned_addr learned my addr 172.21.15.153:0/1018400 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:25:21.996 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.993+0000 7ff2a2000700 1 --2- 172.21.15.153:0/1018400 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff2ac1039b0 0x7ff2ac19ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:21.996 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.993+0000 7ff2aaa00700 1 -- 172.21.15.153:0/1018400 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff2ac1039b0 msgr2=0x7ff2ac19ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:21.996 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.993+0000 7ff2aaa00700 1 --2- 172.21.15.153:0/1018400 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff2ac1039b0 0x7ff2ac19ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:21.996 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.993+0000 7ff2aaa00700 1 -- 172.21.15.153:0/1018400 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff29c009d00 con 0x7ff2ac103000 2024-03-21T10:25:21.996 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.994+0000 7ff2aaa00700 1 --2- 172.21.15.153:0/1018400 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff2ac103000 0x7ff2ac19e8c0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7ff29c00b070 tx=0x7ff29c00cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:21.996 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.994+0000 7ff2a3e00700 1 -- 172.21.15.153:0/1018400 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff29c00e040 con 0x7ff2ac103000 2024-03-21T10:25:21.996 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.994+0000 7ff2a3e00700 1 -- 172.21.15.153:0/1018400 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff29c003e60 con 0x7ff2ac103000 2024-03-21T10:25:21.996 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.994+0000 7ff2a3e00700 1 -- 172.21.15.153:0/1018400 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff29c01d910 con 0x7ff2ac103000 2024-03-21T10:25:21.996 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.994+0000 7ff2b1a00700 1 -- 172.21.15.153:0/1018400 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2ac198bc0 con 0x7ff2ac103000 2024-03-21T10:25:21.996 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.994+0000 7ff2b1a00700 1 -- 172.21.15.153:0/1018400 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2ac1990b0 con 0x7ff2ac103000 2024-03-21T10:25:21.997 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.995+0000 7ff2b1a00700 1 -- 172.21.15.153:0/1018400 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2ac10b3a0 con 0x7ff2ac103000 2024-03-21T10:25:21.997 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.996+0000 7ff2a3e00700 1 -- 172.21.15.153:0/1018400 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7ff29c004820 con 0x7ff2ac103000 2024-03-21T10:25:21.998 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.997+0000 7ff2a3e00700 1 --2- 172.21.15.153:0/1018400 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff29006e460 0x7ff290070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:21.998 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.997+0000 7ff2a3e00700 1 -- 172.21.15.153:0/1018400 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7ff29c08ff10 con 0x7ff2ac103000 2024-03-21T10:25:21.999 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.998+0000 7ff2a2000700 1 --2- 172.21.15.153:0/1018400 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff29006e460 0x7ff290070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:21.999 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:21.998+0000 7ff2a2000700 1 --2- 172.21.15.153:0/1018400 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff29006e460 0x7ff290070920 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff294006fd0 tx=0x7ff294008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:22.003 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:22.002+0000 7ff2a3e00700 1 -- 172.21.15.153:0/1018400 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff29c05ca60 con 0x7ff2ac103000 2024-03-21T10:25:22.266 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:22.264+0000 7ff2b1a00700 1 -- 172.21.15.153:0/1018400 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7ff2ac02cca0 con 0x7ff2ac103000 2024-03-21T10:25:22.266 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:22.265+0000 7ff2a3e00700 1 -- 172.21.15.153:0/1018400 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1876 (secure 0 0 0) 0x7ff29c05c5f0 con 0x7ff2ac103000 2024-03-21T10:25:22.267 INFO:teuthology.orchestra.run.smithi153.stdout:e30 2024-03-21T10:25:22.267 INFO:teuthology.orchestra.run.smithi153.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-21T10:25:22.267 INFO:teuthology.orchestra.run.smithi153.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:25:22.267 INFO:teuthology.orchestra.run.smithi153.stdout:legacy client fscid: 1 2024-03-21T10:25:22.267 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:25:22.267 INFO:teuthology.orchestra.run.smithi153.stdout:Filesystem 'cephfs' (1) 2024-03-21T10:25:22.267 INFO:teuthology.orchestra.run.smithi153.stdout:fs_name cephfs 2024-03-21T10:25:22.267 INFO:teuthology.orchestra.run.smithi153.stdout:epoch 23 2024-03-21T10:25:22.267 INFO:teuthology.orchestra.run.smithi153.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-21T10:25:22.267 INFO:teuthology.orchestra.run.smithi153.stdout:created 2024-03-21T10:04:27.086576+0000 2024-03-21T10:25:22.267 INFO:teuthology.orchestra.run.smithi153.stdout:modified 2024-03-21T10:19:05.022092+0000 2024-03-21T10:25:22.267 INFO:teuthology.orchestra.run.smithi153.stdout:tableserver 0 2024-03-21T10:25:22.267 INFO:teuthology.orchestra.run.smithi153.stdout:root 0 2024-03-21T10:25:22.268 INFO:teuthology.orchestra.run.smithi153.stdout:session_timeout 60 2024-03-21T10:25:22.268 INFO:teuthology.orchestra.run.smithi153.stdout:session_autoclose 300 2024-03-21T10:25:22.268 INFO:teuthology.orchestra.run.smithi153.stdout:max_file_size 1099511627776 2024-03-21T10:25:22.268 INFO:teuthology.orchestra.run.smithi153.stdout:max_xattr_size 65536 2024-03-21T10:25:22.268 INFO:teuthology.orchestra.run.smithi153.stdout:required_client_features {} 2024-03-21T10:25:22.268 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure 0 2024-03-21T10:25:22.268 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure_osd_epoch 94 2024-03-21T10:25:22.268 INFO:teuthology.orchestra.run.smithi153.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:25:22.268 INFO:teuthology.orchestra.run.smithi153.stdout:max_mds 2 2024-03-21T10:25:22.268 INFO:teuthology.orchestra.run.smithi153.stdout:in 0,1 2024-03-21T10:25:22.268 INFO:teuthology.orchestra.run.smithi153.stdout:up {} 2024-03-21T10:25:22.268 INFO:teuthology.orchestra.run.smithi153.stdout:failed 0,1 2024-03-21T10:25:22.268 INFO:teuthology.orchestra.run.smithi153.stdout:damaged 2024-03-21T10:25:22.268 INFO:teuthology.orchestra.run.smithi153.stdout:stopped 2024-03-21T10:25:22.269 INFO:teuthology.orchestra.run.smithi153.stdout:data_pools [3] 2024-03-21T10:25:22.269 INFO:teuthology.orchestra.run.smithi153.stdout:metadata_pool 2 2024-03-21T10:25:22.269 INFO:teuthology.orchestra.run.smithi153.stdout:inline_data enabled 2024-03-21T10:25:22.269 INFO:teuthology.orchestra.run.smithi153.stdout:balancer 2024-03-21T10:25:22.269 INFO:teuthology.orchestra.run.smithi153.stdout:bal_rank_mask -1 2024-03-21T10:25:22.269 INFO:teuthology.orchestra.run.smithi153.stdout:standby_count_wanted 1 2024-03-21T10:25:22.269 INFO:teuthology.orchestra.run.smithi153.stdout:qdb_cluster leader: 0 members: 2024-03-21T10:25:22.269 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:25:22.269 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:25:22.269 INFO:teuthology.orchestra.run.smithi153.stdout:Standby daemons: 2024-03-21T10:25:22.269 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:25:22.270 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.zipckr{-1:34442} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6826/1103846295,v1:172.21.15.153:6827/1103846295] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:25:22.270 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.iuyluj{-1:34474} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6828/1738630832,v1:172.21.15.153:6829/1738630832] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:25:22.270 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.nhozag{-1:44269} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.174:6826/3967223227,v1:172.21.15.174:6827/3967223227] compat {c=[1],r=[1],i=[7ff]}] 2024-03-21T10:25:22.270 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.yyimst{-1:44313} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.174:6824/4127786797,v1:172.21.15.174:6825/4127786797] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:25:22.271 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:22.268+0000 7ff2b1a00700 1 -- 172.21.15.153:0/1018400 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff29006e460 msgr2=0x7ff290070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:22.271 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:22.268+0000 7ff2b1a00700 1 --2- 172.21.15.153:0/1018400 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff29006e460 0x7ff290070920 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff294006fd0 tx=0x7ff294008040 comp rx=0 tx=0).stop 2024-03-21T10:25:22.271 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:22.268+0000 7ff2b1a00700 1 -- 172.21.15.153:0/1018400 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff2ac103000 msgr2=0x7ff2ac19e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:22.271 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:22.268+0000 7ff2b1a00700 1 --2- 172.21.15.153:0/1018400 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff2ac103000 0x7ff2ac19e8c0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7ff29c00b070 tx=0x7ff29c00cdd0 comp rx=0 tx=0).stop 2024-03-21T10:25:22.271 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:22.268+0000 7ff2b1a00700 1 -- 172.21.15.153:0/1018400 shutdown_connections 2024-03-21T10:25:22.271 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:22.268+0000 7ff2b1a00700 1 --2- 172.21.15.153:0/1018400 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff29006e460 0x7ff290070920 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:22.271 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:22.268+0000 7ff2b1a00700 1 --2- 172.21.15.153:0/1018400 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff2ac103000 0x7ff2ac19e8c0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:22.271 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:22.268+0000 7ff2b1a00700 1 --2- 172.21.15.153:0/1018400 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff2ac1039b0 0x7ff2ac19ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:22.271 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:22.268+0000 7ff2b1a00700 1 -- 172.21.15.153:0/1018400 >> 172.21.15.153:0/1018400 conn(0x7ff2ac0fe8d0 msgr2=0x7ff2ac0fffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:22.271 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:22.268+0000 7ff2b1a00700 1 -- 172.21.15.153:0/1018400 shutdown_connections 2024-03-21T10:25:22.271 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:22.268+0000 7ff2b1a00700 1 -- 172.21.15.153:0/1018400 wait complete. 2024-03-21T10:25:22.271 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 30 2024-03-21T10:25:22.414 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:22.412+0000 7f2d0b200700 1 -- 172.21.15.153:0/4291701219 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2d040752c0 msgr2=0x7f2d040756a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:22.415 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:22.412+0000 7f2d0b200700 1 --2- 172.21.15.153:0/4291701219 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2d040752c0 0x7f2d040756a0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f2cf4009910 tx=0x7f2cf4009c20 comp rx=0 tx=0).stop 2024-03-21T10:25:22.415 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:22.413+0000 7f2d0b200700 1 -- 172.21.15.153:0/4291701219 shutdown_connections 2024-03-21T10:25:22.415 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:22.413+0000 7f2d0b200700 1 --2- 172.21.15.153:0/4291701219 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d04075be0 0x7f2d041114e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:22.415 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:22.413+0000 7f2d0b200700 1 --2- 172.21.15.153:0/4291701219 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2d040752c0 0x7f2d040756a0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:22.415 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:22.413+0000 7f2d0b200700 1 -- 172.21.15.153:0/4291701219 >> 172.21.15.153:0/4291701219 conn(0x7f2d040fe850 msgr2=0x7f2d04100c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:22.415 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:22.413+0000 7f2d0b200700 1 -- 172.21.15.153:0/4291701219 shutdown_connections 2024-03-21T10:25:22.415 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:22.413+0000 7f2d0b200700 1 -- 172.21.15.153:0/4291701219 wait complete. 2024-03-21T10:25:22.417 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:22.414+0000 7f2d0b200700 1 Processor -- start 2024-03-21T10:25:22.417 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:22.414+0000 7f2d0b200700 1 -- start start 2024-03-21T10:25:22.417 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:22.414+0000 7f2d0b200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2d040752c0 0x7f2d0419cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:22.417 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:22.414+0000 7f2d0b200700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d04075be0 0x7f2d0419d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:22.417 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:22.414+0000 7f2d0b200700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0419d950 con 0x7f2d040752c0 2024-03-21T10:25:22.417 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:22.414+0000 7f2d0b200700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d041a16e0 con 0x7f2d04075be0 2024-03-21T10:25:22.417 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:22.415+0000 7f2d03e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2d040752c0 0x7f2d0419cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:22.417 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:22.415+0000 7f2d03e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2d040752c0 0x7f2d0419cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:60340/0 (socket says 172.21.15.153:60340) 2024-03-21T10:25:22.417 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:22.415+0000 7f2d03e00700 1 -- 172.21.15.153:0/2748882137 learned_addr learned my addr 172.21.15.153:0/2748882137 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:25:22.417 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:22.415+0000 7f2d03400700 1 --2- 172.21.15.153:0/2748882137 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d04075be0 0x7f2d0419d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:22.417 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:22.415+0000 7f2d03e00700 1 -- 172.21.15.153:0/2748882137 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d04075be0 msgr2=0x7f2d0419d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:22.418 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:22.415+0000 7f2d03e00700 1 --2- 172.21.15.153:0/2748882137 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d04075be0 0x7f2d0419d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:22.418 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:22.415+0000 7f2d03e00700 1 -- 172.21.15.153:0/2748882137 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cf40095f0 con 0x7f2d040752c0 2024-03-21T10:25:22.418 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:22.416+0000 7f2d03e00700 1 --2- 172.21.15.153:0/2748882137 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2d040752c0 0x7f2d0419cd30 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f2cf4000c00 tx=0x7f2cf40056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:22.418 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:22.416+0000 7f2d00c00700 1 -- 172.21.15.153:0/2748882137 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2cf401d070 con 0x7f2d040752c0 2024-03-21T10:25:22.419 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:22.416+0000 7f2d00c00700 1 -- 172.21.15.153:0/2748882137 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2cf400f460 con 0x7f2d040752c0 2024-03-21T10:25:22.419 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:22.416+0000 7f2d00c00700 1 -- 172.21.15.153:0/2748882137 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2cf4021640 con 0x7f2d040752c0 2024-03-21T10:25:22.419 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:22.416+0000 7f2d0b200700 1 -- 172.21.15.153:0/2748882137 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d041a19c0 con 0x7f2d040752c0 2024-03-21T10:25:22.419 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:22.416+0000 7f2d0b200700 1 -- 172.21.15.153:0/2748882137 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d041a1f10 con 0x7f2d040752c0 2024-03-21T10:25:22.419 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:22.417+0000 7f2d0b200700 1 -- 172.21.15.153:0/2748882137 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d0410ec50 con 0x7f2d040752c0 2024-03-21T10:25:22.419 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:22.418+0000 7f2d00c00700 1 -- 172.21.15.153:0/2748882137 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f2cf400fa60 con 0x7f2d040752c0 2024-03-21T10:25:22.420 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:22.419+0000 7f2d00c00700 1 --2- 172.21.15.153:0/2748882137 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f2ce806e460 0x7f2ce8070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:22.421 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:22.419+0000 7f2d00c00700 1 -- 172.21.15.153:0/2748882137 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f2cf408f790 con 0x7f2d040752c0 2024-03-21T10:25:22.421 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:22.419+0000 7f2d03400700 1 --2- 172.21.15.153:0/2748882137 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f2ce806e460 0x7f2ce8070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:22.421 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:22.420+0000 7f2d03400700 1 --2- 172.21.15.153:0/2748882137 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f2ce806e460 0x7f2ce8070920 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f2cec005950 tx=0x7f2cec0058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:22.425 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:22.424+0000 7f2d00c00700 1 -- 172.21.15.153:0/2748882137 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2cf40587b0 con 0x7f2d040752c0 2024-03-21T10:25:22.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:25:22 smithi153 ceph-mon[113733]: from='client.34766 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:25:22.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:25:22 smithi153 ceph-mon[113733]: pgmap v405: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:25:22.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:25:22 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/2143475931' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:25:22.557 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:25:22 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/1018400' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:25:22.654 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:22.652+0000 7f2d0b200700 1 -- 172.21.15.153:0/2748882137 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2d041a2190 con 0x7f2ce806e460 2024-03-21T10:25:22.656 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:22.655+0000 7f2d00c00700 1 -- 172.21.15.153:0/2748882137 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f2d041a2190 con 0x7f2ce806e460 2024-03-21T10:25:22.656 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:25:22.657 INFO:teuthology.orchestra.run.smithi153.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232", 2024-03-21T10:25:22.657 INFO:teuthology.orchestra.run.smithi153.stdout: "in_progress": true, 2024-03-21T10:25:22.657 INFO:teuthology.orchestra.run.smithi153.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-21T10:25:22.657 INFO:teuthology.orchestra.run.smithi153.stdout: "services_complete": [ 2024-03-21T10:25:22.657 INFO:teuthology.orchestra.run.smithi153.stdout: "crash", 2024-03-21T10:25:22.657 INFO:teuthology.orchestra.run.smithi153.stdout: "mon", 2024-03-21T10:25:22.657 INFO:teuthology.orchestra.run.smithi153.stdout: "osd", 2024-03-21T10:25:22.657 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr" 2024-03-21T10:25:22.657 INFO:teuthology.orchestra.run.smithi153.stdout: ], 2024-03-21T10:25:22.657 INFO:teuthology.orchestra.run.smithi153.stdout: "progress": "15/23 daemons upgraded", 2024-03-21T10:25:22.658 INFO:teuthology.orchestra.run.smithi153.stdout: "message": "Currently upgrading mds daemons", 2024-03-21T10:25:22.658 INFO:teuthology.orchestra.run.smithi153.stdout: "is_paused": false 2024-03-21T10:25:22.658 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:25:22.660 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:22.657+0000 7f2d0b200700 1 -- 172.21.15.153:0/2748882137 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f2ce806e460 msgr2=0x7f2ce8070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:22.660 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:22.657+0000 7f2d0b200700 1 --2- 172.21.15.153:0/2748882137 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f2ce806e460 0x7f2ce8070920 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f2cec005950 tx=0x7f2cec0058e0 comp rx=0 tx=0).stop 2024-03-21T10:25:22.660 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:22.657+0000 7f2d0b200700 1 -- 172.21.15.153:0/2748882137 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2d040752c0 msgr2=0x7f2d0419cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:22.660 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:22.658+0000 7f2d0b200700 1 --2- 172.21.15.153:0/2748882137 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2d040752c0 0x7f2d0419cd30 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f2cf4000c00 tx=0x7f2cf40056a0 comp rx=0 tx=0).stop 2024-03-21T10:25:22.660 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:22.658+0000 7f2d0b200700 1 -- 172.21.15.153:0/2748882137 shutdown_connections 2024-03-21T10:25:22.660 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:22.658+0000 7f2d0b200700 1 --2- 172.21.15.153:0/2748882137 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f2ce806e460 0x7f2ce8070920 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:22.660 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:22.658+0000 7f2d0b200700 1 --2- 172.21.15.153:0/2748882137 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2d040752c0 0x7f2d0419cd30 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:22.660 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:22.658+0000 7f2d0b200700 1 --2- 172.21.15.153:0/2748882137 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d04075be0 0x7f2d0419d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:22.661 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:22.658+0000 7f2d0b200700 1 -- 172.21.15.153:0/2748882137 >> 172.21.15.153:0/2748882137 conn(0x7f2d040fe850 msgr2=0x7f2d04100540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:22.661 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:22.658+0000 7f2d0b200700 1 -- 172.21.15.153:0/2748882137 shutdown_connections 2024-03-21T10:25:22.661 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:22.658+0000 7f2d0b200700 1 -- 172.21.15.153:0/2748882137 wait complete. 2024-03-21T10:25:22.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:25:22 smithi174 ceph-mon[88004]: from='client.34766 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:25:22.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:25:22 smithi174 ceph-mon[88004]: pgmap v405: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:25:22.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:25:22 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/2143475931' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:25:22.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:25:22 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/1018400' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:25:22.809 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:22.807+0000 7f65eee00700 1 -- 172.21.15.153:0/2609566252 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f65e8107220 msgr2=0x7f65e8107600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:22.809 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:22.807+0000 7f65eee00700 1 --2- 172.21.15.153:0/2609566252 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f65e8107220 0x7f65e8107600 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f65dc009910 tx=0x7f65dc009c20 comp rx=0 tx=0).stop 2024-03-21T10:25:22.809 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:22.807+0000 7f65eee00700 1 -- 172.21.15.153:0/2609566252 shutdown_connections 2024-03-21T10:25:22.809 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:22.807+0000 7f65eee00700 1 --2- 172.21.15.153:0/2609566252 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f65e8107bd0 0x7f65e810bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:22.809 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:22.807+0000 7f65eee00700 1 --2- 172.21.15.153:0/2609566252 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f65e8107220 0x7f65e8107600 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:22.809 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:22.807+0000 7f65eee00700 1 -- 172.21.15.153:0/2609566252 >> 172.21.15.153:0/2609566252 conn(0x7f65e80767e0 msgr2=0x7f65e8078c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:22.809 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:22.807+0000 7f65eee00700 1 -- 172.21.15.153:0/2609566252 shutdown_connections 2024-03-21T10:25:22.809 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:22.807+0000 7f65eee00700 1 -- 172.21.15.153:0/2609566252 wait complete. 2024-03-21T10:25:22.811 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:22.808+0000 7f65eee00700 1 Processor -- start 2024-03-21T10:25:22.811 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:22.808+0000 7f65eee00700 1 -- start start 2024-03-21T10:25:22.811 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:22.808+0000 7f65eee00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f65e8107220 0x7f65e81a2a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:22.811 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:22.809+0000 7f65eee00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f65e8107bd0 0x7f65e81a2f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:22.811 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:22.809+0000 7f65eee00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65e81a35f0 con 0x7f65e8107bd0 2024-03-21T10:25:22.812 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:22.809+0000 7f65eee00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65e819caa0 con 0x7f65e8107220 2024-03-21T10:25:22.812 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:22.809+0000 7f65ed000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f65e8107bd0 0x7f65e81a2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:22.812 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:22.809+0000 7f65ed000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f65e8107bd0 0x7f65e81a2f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:60366/0 (socket says 172.21.15.153:60366) 2024-03-21T10:25:22.812 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:22.809+0000 7f65ed000700 1 -- 172.21.15.153:0/2834864675 learned_addr learned my addr 172.21.15.153:0/2834864675 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:25:22.812 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:22.809+0000 7f65eda00700 1 --2- 172.21.15.153:0/2834864675 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f65e8107220 0x7f65e81a2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:22.812 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:22.809+0000 7f65ed000700 1 -- 172.21.15.153:0/2834864675 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f65e8107220 msgr2=0x7f65e81a2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:22.812 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:22.809+0000 7f65ed000700 1 --2- 172.21.15.153:0/2834864675 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f65e8107220 0x7f65e81a2a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:22.812 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:22.809+0000 7f65ed000700 1 -- 172.21.15.153:0/2834864675 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65dc0095f0 con 0x7f65e8107bd0 2024-03-21T10:25:22.812 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:22.810+0000 7f65ed000700 1 --2- 172.21.15.153:0/2834864675 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f65e8107bd0 0x7f65e81a2f60 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f65d400d900 tx=0x7f65d400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:22.812 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:22.810+0000 7f65e6000700 1 -- 172.21.15.153:0/2834864675 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f65d4009900 con 0x7f65e8107bd0 2024-03-21T10:25:22.812 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:22.810+0000 7f65e6000700 1 -- 172.21.15.153:0/2834864675 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f65d4010460 con 0x7f65e8107bd0 2024-03-21T10:25:22.812 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:22.810+0000 7f65eee00700 1 -- 172.21.15.153:0/2834864675 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65e819cd80 con 0x7f65e8107bd0 2024-03-21T10:25:22.812 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:22.810+0000 7f65e6000700 1 -- 172.21.15.153:0/2834864675 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f65d400f5f0 con 0x7f65e8107bd0 2024-03-21T10:25:22.812 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:22.810+0000 7f65eee00700 1 -- 172.21.15.153:0/2834864675 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65e819d2d0 con 0x7f65e8107bd0 2024-03-21T10:25:22.813 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:22.811+0000 7f65eee00700 1 -- 172.21.15.153:0/2834864675 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65e810f5b0 con 0x7f65e8107bd0 2024-03-21T10:25:22.813 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:22.812+0000 7f65e6000700 1 -- 172.21.15.153:0/2834864675 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f65d40105d0 con 0x7f65e8107bd0 2024-03-21T10:25:22.815 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:22.813+0000 7f65e6000700 1 --2- 172.21.15.153:0/2834864675 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f65d006e460 0x7f65d0070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:22.815 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:22.814+0000 7f65e6000700 1 -- 172.21.15.153:0/2834864675 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f65d408e410 con 0x7f65e8107bd0 2024-03-21T10:25:22.815 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:22.814+0000 7f65eda00700 1 --2- 172.21.15.153:0/2834864675 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f65d006e460 0x7f65d0070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:22.815 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:22.814+0000 7f65eda00700 1 --2- 172.21.15.153:0/2834864675 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f65d006e460 0x7f65d0070920 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f65dc00b5c0 tx=0x7f65dc01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:22.819 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:22.818+0000 7f65e6000700 1 -- 172.21.15.153:0/2834864675 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f65d405af60 con 0x7f65e8107bd0 2024-03-21T10:25:23.117 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:23.115+0000 7f65eee00700 1 -- 172.21.15.153:0/2834864675 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f65e819d4f0 con 0x7f65e8107bd0 2024-03-21T10:25:23.118 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:23.116+0000 7f65e6000700 1 -- 172.21.15.153:0/2834864675 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f65d405aaf0 con 0x7f65e8107bd0 2024-03-21T10:25:23.118 INFO:teuthology.orchestra.run.smithi153.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-03-21T10:25:23.118 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-21T10:25:23.118 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is degraded 2024-03-21T10:25:23.118 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-21T10:25:23.119 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-21T10:25:23.119 INFO:teuthology.orchestra.run.smithi153.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-21T10:25:23.119 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-21T10:25:23.121 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:23.119+0000 7f65eee00700 1 -- 172.21.15.153:0/2834864675 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f65d006e460 msgr2=0x7f65d0070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:23.121 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:23.119+0000 7f65eee00700 1 --2- 172.21.15.153:0/2834864675 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f65d006e460 0x7f65d0070920 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f65dc00b5c0 tx=0x7f65dc01a040 comp rx=0 tx=0).stop 2024-03-21T10:25:23.121 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:23.119+0000 7f65eee00700 1 -- 172.21.15.153:0/2834864675 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f65e8107bd0 msgr2=0x7f65e81a2f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:23.121 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:23.119+0000 7f65eee00700 1 --2- 172.21.15.153:0/2834864675 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f65e8107bd0 0x7f65e81a2f60 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f65d400d900 tx=0x7f65d400dcc0 comp rx=0 tx=0).stop 2024-03-21T10:25:23.121 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:23.119+0000 7f65eee00700 1 -- 172.21.15.153:0/2834864675 shutdown_connections 2024-03-21T10:25:23.121 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:23.119+0000 7f65eee00700 1 --2- 172.21.15.153:0/2834864675 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f65e8107220 0x7f65e81a2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:23.121 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:23.119+0000 7f65eee00700 1 --2- 172.21.15.153:0/2834864675 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f65d006e460 0x7f65d0070920 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:23.121 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:23.119+0000 7f65eee00700 1 --2- 172.21.15.153:0/2834864675 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f65e8107bd0 0x7f65e81a2f60 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:23.122 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:23.119+0000 7f65eee00700 1 -- 172.21.15.153:0/2834864675 >> 172.21.15.153:0/2834864675 conn(0x7f65e80767e0 msgr2=0x7f65e8077e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:23.122 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:23.120+0000 7f65eee00700 1 -- 172.21.15.153:0/2834864675 shutdown_connections 2024-03-21T10:25:23.122 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:23.120+0000 7f65eee00700 1 -- 172.21.15.153:0/2834864675 wait complete. 2024-03-21T10:25:23.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:25:23 smithi174 ceph-mon[88004]: from='client.34778 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:25:23.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:25:23 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/2834864675' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:25:23.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:25:23 smithi153 ceph-mon[113733]: from='client.34778 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:25:23.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:25:23 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/2834864675' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:25:24.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:25:24 smithi174 ceph-mon[88004]: pgmap v406: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:25:24.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:25:24 smithi153 ceph-mon[113733]: pgmap v406: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:25:26.556 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:25:26 smithi153 ceph-mon[113733]: pgmap v407: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:25:26.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:25:26 smithi174 ceph-mon[88004]: pgmap v407: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:25:28.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:25:28 smithi153 ceph-mon[113733]: pgmap v408: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:25:28.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:25:28 smithi174 ceph-mon[88004]: pgmap v408: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:25:30.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:25:30 smithi153 ceph-mon[113733]: pgmap v409: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:25:30.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:25:30 smithi174 ceph-mon[88004]: pgmap v409: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:25:31.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:25:31 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:25:31.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:25:31 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:25:32.649 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:25:32 smithi153 ceph-mon[113733]: pgmap v410: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:25:32.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:25:32 smithi174 ceph-mon[88004]: pgmap v410: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:25:34.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:25:34 smithi174 ceph-mon[88004]: pgmap v411: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:25:34.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:25:34 smithi153 ceph-mon[113733]: pgmap v411: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:25:36.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:25:36 smithi153 ceph-mon[113733]: pgmap v412: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:25:36.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:25:36 smithi174 ceph-mon[88004]: pgmap v412: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:25:38.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:25:38 smithi153 ceph-mon[113733]: pgmap v413: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:25:38.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:25:38 smithi174 ceph-mon[88004]: pgmap v413: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:25:40.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:25:40 smithi153 ceph-mon[113733]: pgmap v414: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:25:40.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:25:40 smithi174 ceph-mon[88004]: pgmap v414: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:25:42.650 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:25:42 smithi153 ceph-mon[113733]: pgmap v415: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:25:42.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:25:42 smithi174 ceph-mon[88004]: pgmap v415: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:25:44.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:25:44 smithi153 ceph-mon[113733]: pgmap v416: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:25:44.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:25:44 smithi174 ceph-mon[88004]: pgmap v416: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:25:46.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:25:46 smithi153 ceph-mon[113733]: pgmap v417: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:25:46.807 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:25:46 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:25:46.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:25:46 smithi174 ceph-mon[88004]: pgmap v417: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:25:46.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:25:46 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:25:48.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:25:48 smithi153 ceph-mon[113733]: pgmap v418: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:25:48.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:25:48 smithi174 ceph-mon[88004]: pgmap v418: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:25:50.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:25:50 smithi153 ceph-mon[113733]: pgmap v419: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:25:50.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:25:50 smithi174 ceph-mon[88004]: pgmap v419: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:25:52.649 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:25:52 smithi153 ceph-mon[113733]: pgmap v420: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:25:52.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:25:52 smithi174 ceph-mon[88004]: pgmap v420: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:25:53.290 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:53.288+0000 7fbb51e00700 1 -- 172.21.15.153:0/1717792126 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbb4c1039b0 msgr2=0x7fbb4c107a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:53.290 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:53.288+0000 7fbb51e00700 1 --2- 172.21.15.153:0/1717792126 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbb4c1039b0 0x7fbb4c107a00 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fbb34009910 tx=0x7fbb34009c20 comp rx=0 tx=0).stop 2024-03-21T10:25:53.290 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:53.288+0000 7fbb51e00700 1 -- 172.21.15.153:0/1717792126 shutdown_connections 2024-03-21T10:25:53.291 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:53.288+0000 7fbb51e00700 1 --2- 172.21.15.153:0/1717792126 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbb4c1039b0 0x7fbb4c107a00 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:53.291 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:53.289+0000 7fbb51e00700 1 --2- 172.21.15.153:0/1717792126 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbb4c103000 0x7fbb4c1033e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:53.291 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:53.289+0000 7fbb51e00700 1 -- 172.21.15.153:0/1717792126 >> 172.21.15.153:0/1717792126 conn(0x7fbb4c0fe8d0 msgr2=0x7fbb4c100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:53.291 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:53.289+0000 7fbb51e00700 1 -- 172.21.15.153:0/1717792126 shutdown_connections 2024-03-21T10:25:53.291 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:53.289+0000 7fbb51e00700 1 -- 172.21.15.153:0/1717792126 wait complete. 2024-03-21T10:25:53.292 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:53.290+0000 7fbb51e00700 1 Processor -- start 2024-03-21T10:25:53.292 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:53.290+0000 7fbb51e00700 1 -- start start 2024-03-21T10:25:53.292 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:53.290+0000 7fbb51e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbb4c103000 0x7fbb4c19e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:53.292 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:53.290+0000 7fbb51e00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbb4c1039b0 0x7fbb4c19ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:53.293 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:53.290+0000 7fbb51e00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb4c19f490 con 0x7fbb4c103000 2024-03-21T10:25:53.293 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:53.290+0000 7fbb51e00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb4c198940 con 0x7fbb4c1039b0 2024-03-21T10:25:53.293 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:53.291+0000 7fbb4aa00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbb4c103000 0x7fbb4c19e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:53.293 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:53.291+0000 7fbb4aa00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbb4c103000 0x7fbb4c19e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:50652/0 (socket says 172.21.15.153:50652) 2024-03-21T10:25:53.293 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:53.291+0000 7fbb4aa00700 1 -- 172.21.15.153:0/3213153695 learned_addr learned my addr 172.21.15.153:0/3213153695 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:25:53.293 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:53.291+0000 7fbb42000700 1 --2- 172.21.15.153:0/3213153695 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbb4c1039b0 0x7fbb4c19ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:53.293 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:53.291+0000 7fbb4aa00700 1 -- 172.21.15.153:0/3213153695 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbb4c1039b0 msgr2=0x7fbb4c19ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:53.293 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:53.291+0000 7fbb4aa00700 1 --2- 172.21.15.153:0/3213153695 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbb4c1039b0 0x7fbb4c19ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:53.293 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:53.291+0000 7fbb4aa00700 1 -- 172.21.15.153:0/3213153695 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb340095f0 con 0x7fbb4c103000 2024-03-21T10:25:53.294 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:53.292+0000 7fbb4aa00700 1 --2- 172.21.15.153:0/3213153695 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbb4c103000 0x7fbb4c19e8c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c00b4b0 tx=0x7fbb3c00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:53.294 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:53.292+0000 7fbb50a00700 1 -- 172.21.15.153:0/3213153695 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbb3c011800 con 0x7fbb4c103000 2024-03-21T10:25:53.294 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:53.292+0000 7fbb50a00700 1 -- 172.21.15.153:0/3213153695 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbb3c011e20 con 0x7fbb4c103000 2024-03-21T10:25:53.294 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:53.292+0000 7fbb50a00700 1 -- 172.21.15.153:0/3213153695 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbb3c00f2e0 con 0x7fbb4c103000 2024-03-21T10:25:53.294 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:53.292+0000 7fbb51e00700 1 -- 172.21.15.153:0/3213153695 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb4c198c20 con 0x7fbb4c103000 2024-03-21T10:25:53.294 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:53.292+0000 7fbb51e00700 1 -- 172.21.15.153:0/3213153695 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb4c199170 con 0x7fbb4c103000 2024-03-21T10:25:53.295 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:53.293+0000 7fbb51e00700 1 -- 172.21.15.153:0/3213153695 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb4c10b330 con 0x7fbb4c103000 2024-03-21T10:25:53.295 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:53.294+0000 7fbb50a00700 1 -- 172.21.15.153:0/3213153695 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fbb3c010380 con 0x7fbb4c103000 2024-03-21T10:25:53.296 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:53.295+0000 7fbb50a00700 1 --2- 172.21.15.153:0/3213153695 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fbb2c06e460 0x7fbb2c070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:53.296 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:53.295+0000 7fbb50a00700 1 -- 172.21.15.153:0/3213153695 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fbb3c08e0c0 con 0x7fbb4c103000 2024-03-21T10:25:53.297 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:53.296+0000 7fbb42000700 1 --2- 172.21.15.153:0/3213153695 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fbb2c06e460 0x7fbb2c070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:53.297 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:53.296+0000 7fbb42000700 1 --2- 172.21.15.153:0/3213153695 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fbb2c06e460 0x7fbb2c070920 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fbb34006010 tx=0x7fbb34005960 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:53.301 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:53.300+0000 7fbb50a00700 1 -- 172.21.15.153:0/3213153695 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fbb3c05ad40 con 0x7fbb4c103000 2024-03-21T10:25:53.545 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:53.544+0000 7fbb51e00700 1 -- 172.21.15.153:0/3213153695 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbb4c199500 con 0x7fbb2c06e460 2024-03-21T10:25:53.547 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:53.546+0000 7fbb50a00700 1 -- 172.21.15.153:0/3213153695 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fbb4c199500 con 0x7fbb2c06e460 2024-03-21T10:25:53.551 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:53.550+0000 7fbb51e00700 1 -- 172.21.15.153:0/3213153695 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fbb2c06e460 msgr2=0x7fbb2c070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:53.551 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:53.550+0000 7fbb51e00700 1 --2- 172.21.15.153:0/3213153695 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fbb2c06e460 0x7fbb2c070920 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fbb34006010 tx=0x7fbb34005960 comp rx=0 tx=0).stop 2024-03-21T10:25:53.551 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:53.550+0000 7fbb51e00700 1 -- 172.21.15.153:0/3213153695 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbb4c103000 msgr2=0x7fbb4c19e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:53.552 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:53.550+0000 7fbb51e00700 1 --2- 172.21.15.153:0/3213153695 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbb4c103000 0x7fbb4c19e8c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c00b4b0 tx=0x7fbb3c00b870 comp rx=0 tx=0).stop 2024-03-21T10:25:53.552 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:53.550+0000 7fbb51e00700 1 -- 172.21.15.153:0/3213153695 shutdown_connections 2024-03-21T10:25:53.552 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:53.550+0000 7fbb51e00700 1 --2- 172.21.15.153:0/3213153695 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fbb2c06e460 0x7fbb2c070920 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:53.552 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:53.550+0000 7fbb51e00700 1 --2- 172.21.15.153:0/3213153695 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbb4c103000 0x7fbb4c19e8c0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:53.552 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:53.550+0000 7fbb51e00700 1 --2- 172.21.15.153:0/3213153695 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbb4c1039b0 0x7fbb4c19ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:53.552 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:53.550+0000 7fbb51e00700 1 -- 172.21.15.153:0/3213153695 >> 172.21.15.153:0/3213153695 conn(0x7fbb4c0fe8d0 msgr2=0x7fbb4c0fffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:53.552 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:53.550+0000 7fbb51e00700 1 -- 172.21.15.153:0/3213153695 shutdown_connections 2024-03-21T10:25:53.552 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:53.550+0000 7fbb51e00700 1 -- 172.21.15.153:0/3213153695 wait complete. 2024-03-21T10:25:53.568 INFO:teuthology.orchestra.run.smithi153.stdout:true 2024-03-21T10:25:53.712 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:53.710+0000 7fe492a00700 1 -- 172.21.15.153:0/3564268523 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe48c102b40 msgr2=0x7fe48c102fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:53.712 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:53.710+0000 7fe492a00700 1 --2- 172.21.15.153:0/3564268523 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe48c102b40 0x7fe48c102fc0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fe478009910 tx=0x7fe478009c20 comp rx=0 tx=0).stop 2024-03-21T10:25:53.712 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:53.710+0000 7fe492a00700 1 -- 172.21.15.153:0/3564268523 shutdown_connections 2024-03-21T10:25:53.713 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:53.710+0000 7fe492a00700 1 --2- 172.21.15.153:0/3564268523 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe48c102b40 0x7fe48c102fc0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:53.713 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:53.710+0000 7fe492a00700 1 --2- 172.21.15.153:0/3564268523 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe48c108bf0 0x7fe48c108fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:53.713 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:53.710+0000 7fe492a00700 1 -- 172.21.15.153:0/3564268523 >> 172.21.15.153:0/3564268523 conn(0x7fe48c0fe650 msgr2=0x7fe48c100a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:53.713 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:53.711+0000 7fe492a00700 1 -- 172.21.15.153:0/3564268523 shutdown_connections 2024-03-21T10:25:53.713 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:53.711+0000 7fe492a00700 1 -- 172.21.15.153:0/3564268523 wait complete. 2024-03-21T10:25:53.715 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:53.712+0000 7fe492a00700 1 Processor -- start 2024-03-21T10:25:53.715 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:53.712+0000 7fe492a00700 1 -- start start 2024-03-21T10:25:53.715 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:53.712+0000 7fe492a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe48c102b40 0x7fe48c198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:53.715 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:53.712+0000 7fe492a00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe48c108bf0 0x7fe48c198b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:53.715 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:53.712+0000 7fe492a00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe48c199270 con 0x7fe48c102b40 2024-03-21T10:25:53.715 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:53.712+0000 7fe492a00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe48c19d000 con 0x7fe48c108bf0 2024-03-21T10:25:53.715 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:53.712+0000 7fe491600700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe48c102b40 0x7fe48c198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:53.715 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:53.713+0000 7fe491600700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe48c102b40 0x7fe48c198650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:50674/0 (socket says 172.21.15.153:50674) 2024-03-21T10:25:53.715 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:53.713+0000 7fe491600700 1 -- 172.21.15.153:0/1426985864 learned_addr learned my addr 172.21.15.153:0/1426985864 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:25:53.715 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:53.713+0000 7fe490c00700 1 --2- 172.21.15.153:0/1426985864 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe48c108bf0 0x7fe48c198b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:53.716 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:53.713+0000 7fe491600700 1 -- 172.21.15.153:0/1426985864 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe48c108bf0 msgr2=0x7fe48c198b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:53.716 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:53.713+0000 7fe491600700 1 --2- 172.21.15.153:0/1426985864 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe48c108bf0 0x7fe48c198b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:53.716 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:53.713+0000 7fe491600700 1 -- 172.21.15.153:0/1426985864 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4780095f0 con 0x7fe48c102b40 2024-03-21T10:25:53.716 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:53.713+0000 7fe491600700 1 --2- 172.21.15.153:0/1426985864 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe48c102b40 0x7fe48c198650 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fe48000d8d0 tx=0x7fe48000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:53.716 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:53.714+0000 7fe48a000700 1 -- 172.21.15.153:0/1426985864 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe480009940 con 0x7fe48c102b40 2024-03-21T10:25:53.716 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:53.714+0000 7fe48a000700 1 -- 172.21.15.153:0/1426985864 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe480010460 con 0x7fe48c102b40 2024-03-21T10:25:53.716 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:53.714+0000 7fe48a000700 1 -- 172.21.15.153:0/1426985864 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe48000f5f0 con 0x7fe48c102b40 2024-03-21T10:25:53.716 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:53.714+0000 7fe492a00700 1 -- 172.21.15.153:0/1426985864 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe48c19d2e0 con 0x7fe48c102b40 2024-03-21T10:25:53.716 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:53.714+0000 7fe492a00700 1 -- 172.21.15.153:0/1426985864 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe48c19d8b0 con 0x7fe48c102b40 2024-03-21T10:25:53.716 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:53.715+0000 7fe492a00700 1 -- 172.21.15.153:0/1426985864 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe48c04f2e0 con 0x7fe48c102b40 2024-03-21T10:25:53.717 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:53.716+0000 7fe48a000700 1 -- 172.21.15.153:0/1426985864 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fe4800105d0 con 0x7fe48c102b40 2024-03-21T10:25:53.719 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:53.717+0000 7fe48a000700 1 --2- 172.21.15.153:0/1426985864 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe47406e460 0x7fe474070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:53.719 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:53.718+0000 7fe48a000700 1 -- 172.21.15.153:0/1426985864 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fe48008e480 con 0x7fe48c102b40 2024-03-21T10:25:53.719 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:53.718+0000 7fe490c00700 1 --2- 172.21.15.153:0/1426985864 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe47406e460 0x7fe474070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:53.719 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:53.719+0000 7fe490c00700 1 --2- 172.21.15.153:0/1426985864 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe47406e460 0x7fe474070920 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe478000c00 tx=0x7fe478005fd0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:53.723 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:53.723+0000 7fe48a000700 1 -- 172.21.15.153:0/1426985864 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe480057520 con 0x7fe48c102b40 2024-03-21T10:25:53.954 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:53.953+0000 7fe492a00700 1 -- 172.21.15.153:0/1426985864 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe48c19de30 con 0x7fe47406e460 2024-03-21T10:25:53.956 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:53.955+0000 7fe48a000700 1 -- 172.21.15.153:0/1426985864 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fe48c19de30 con 0x7fe47406e460 2024-03-21T10:25:53.959 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:53.957+0000 7fe492a00700 1 -- 172.21.15.153:0/1426985864 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe47406e460 msgr2=0x7fe474070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:53.959 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:53.957+0000 7fe492a00700 1 --2- 172.21.15.153:0/1426985864 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe47406e460 0x7fe474070920 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe478000c00 tx=0x7fe478005fd0 comp rx=0 tx=0).stop 2024-03-21T10:25:53.959 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:53.957+0000 7fe492a00700 1 -- 172.21.15.153:0/1426985864 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe48c102b40 msgr2=0x7fe48c198650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:53.959 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:53.957+0000 7fe492a00700 1 --2- 172.21.15.153:0/1426985864 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe48c102b40 0x7fe48c198650 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fe48000d8d0 tx=0x7fe48000dc90 comp rx=0 tx=0).stop 2024-03-21T10:25:53.959 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:53.958+0000 7fe492a00700 1 -- 172.21.15.153:0/1426985864 shutdown_connections 2024-03-21T10:25:53.959 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:53.958+0000 7fe492a00700 1 --2- 172.21.15.153:0/1426985864 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe47406e460 0x7fe474070920 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:53.960 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:53.958+0000 7fe492a00700 1 --2- 172.21.15.153:0/1426985864 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe48c102b40 0x7fe48c198650 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:53.960 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:53.958+0000 7fe492a00700 1 --2- 172.21.15.153:0/1426985864 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe48c108bf0 0x7fe48c198b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:53.960 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:53.958+0000 7fe492a00700 1 -- 172.21.15.153:0/1426985864 >> 172.21.15.153:0/1426985864 conn(0x7fe48c0fe650 msgr2=0x7fe48c0ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:53.960 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:53.958+0000 7fe492a00700 1 -- 172.21.15.153:0/1426985864 shutdown_connections 2024-03-21T10:25:53.960 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:53.958+0000 7fe492a00700 1 -- 172.21.15.153:0/1426985864 wait complete. 2024-03-21T10:25:54.118 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.117+0000 7fe578a00700 1 -- 172.21.15.153:0/3942954806 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe574102e10 msgr2=0x7fe5741031f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:54.119 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.117+0000 7fe578a00700 1 --2- 172.21.15.153:0/3942954806 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe574102e10 0x7fe5741031f0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fe564009910 tx=0x7fe564009c20 comp rx=0 tx=0).stop 2024-03-21T10:25:54.119 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.117+0000 7fe578a00700 1 -- 172.21.15.153:0/3942954806 shutdown_connections 2024-03-21T10:25:54.119 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.117+0000 7fe578a00700 1 --2- 172.21.15.153:0/3942954806 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe5741037c0 0x7fe574107810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:54.119 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.117+0000 7fe578a00700 1 --2- 172.21.15.153:0/3942954806 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe574102e10 0x7fe5741031f0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:54.119 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.117+0000 7fe578a00700 1 -- 172.21.15.153:0/3942954806 >> 172.21.15.153:0/3942954806 conn(0x7fe5740fe6c0 msgr2=0x7fe574100ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:54.119 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.117+0000 7fe578a00700 1 -- 172.21.15.153:0/3942954806 shutdown_connections 2024-03-21T10:25:54.119 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.117+0000 7fe578a00700 1 -- 172.21.15.153:0/3942954806 wait complete. 2024-03-21T10:25:54.122 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.118+0000 7fe578a00700 1 Processor -- start 2024-03-21T10:25:54.122 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.118+0000 7fe578a00700 1 -- start start 2024-03-21T10:25:54.123 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.118+0000 7fe578a00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe574102e10 0x7fe57419e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:54.123 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.118+0000 7fe578a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe5741037c0 0x7fe57419eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:54.123 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.118+0000 7fe578a00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe57419f0c0 con 0x7fe5741037c0 2024-03-21T10:25:54.123 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.118+0000 7fe578a00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe574198600 con 0x7fe574102e10 2024-03-21T10:25:54.123 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.119+0000 7fe571e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe5741037c0 0x7fe57419eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:54.123 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.119+0000 7fe571e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe5741037c0 0x7fe57419eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:50698/0 (socket says 172.21.15.153:50698) 2024-03-21T10:25:54.123 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.119+0000 7fe571e00700 1 -- 172.21.15.153:0/1166603645 learned_addr learned my addr 172.21.15.153:0/1166603645 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:25:54.124 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.119+0000 7fe572800700 1 --2- 172.21.15.153:0/1166603645 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe574102e10 0x7fe57419e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:54.124 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.119+0000 7fe571e00700 1 -- 172.21.15.153:0/1166603645 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe574102e10 msgr2=0x7fe57419e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:54.124 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.119+0000 7fe571e00700 1 --2- 172.21.15.153:0/1166603645 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe574102e10 0x7fe57419e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:54.124 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.119+0000 7fe571e00700 1 -- 172.21.15.153:0/1166603645 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5640095f0 con 0x7fe5741037c0 2024-03-21T10:25:54.124 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.120+0000 7fe571e00700 1 --2- 172.21.15.153:0/1166603645 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe5741037c0 0x7fe57419eac0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fe56000c960 tx=0x7fe56000cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:54.124 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.120+0000 7fe56b400700 1 -- 172.21.15.153:0/1166603645 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe560007a70 con 0x7fe5741037c0 2024-03-21T10:25:54.124 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.120+0000 7fe56b400700 1 -- 172.21.15.153:0/1166603645 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe560007bd0 con 0x7fe5741037c0 2024-03-21T10:25:54.124 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.120+0000 7fe56b400700 1 -- 172.21.15.153:0/1166603645 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe5600186c0 con 0x7fe5741037c0 2024-03-21T10:25:54.125 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.120+0000 7fe578a00700 1 -- 172.21.15.153:0/1166603645 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5741988e0 con 0x7fe5741037c0 2024-03-21T10:25:54.125 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.120+0000 7fe578a00700 1 -- 172.21.15.153:0/1166603645 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe574198de0 con 0x7fe5741037c0 2024-03-21T10:25:54.125 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.121+0000 7fe578a00700 1 -- 172.21.15.153:0/1166603645 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe57410b1b0 con 0x7fe5741037c0 2024-03-21T10:25:54.126 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.122+0000 7fe56b400700 1 -- 172.21.15.153:0/1166603645 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fe56000faa0 con 0x7fe5741037c0 2024-03-21T10:25:54.126 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.123+0000 7fe56b400700 1 --2- 172.21.15.153:0/1166603645 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe55806e460 0x7fe558070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:54.126 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.123+0000 7fe56b400700 1 -- 172.21.15.153:0/1166603645 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fe56008e5d0 con 0x7fe5741037c0 2024-03-21T10:25:54.126 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.124+0000 7fe572800700 1 --2- 172.21.15.153:0/1166603645 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe55806e460 0x7fe558070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:54.127 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.124+0000 7fe572800700 1 --2- 172.21.15.153:0/1166603645 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe55806e460 0x7fe558070920 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe564000c00 tx=0x7fe564005fd0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:54.129 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.128+0000 7fe56b400700 1 -- 172.21.15.153:0/1166603645 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe56005ae90 con 0x7fe5741037c0 2024-03-21T10:25:54.362 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.360+0000 7fe578a00700 1 -- 172.21.15.153:0/1166603645 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fe5741990e0 con 0x7fe55806e460 2024-03-21T10:25:54.380 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.379+0000 7fe56b400700 1 -- 172.21.15.153:0/1166603645 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fe5741990e0 con 0x7fe55806e460 2024-03-21T10:25:54.381 INFO:teuthology.orchestra.run.smithi153.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-21T10:25:54.381 INFO:teuthology.orchestra.run.smithi153.stdout:alertmanager.smithi153 smithi153 *:9093,9094 running (14m) 6m ago 28m 14.7M - 0.25.0 c8568f914cd2 9ed3f2200b5f 2024-03-21T10:25:54.381 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi153 smithi153 running (28m) 6m ago 28m 9424k - 18.2.1 5be31c24972a 5b61994396a6 2024-03-21T10:25:54.381 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi174 smithi174 running (27m) 5m ago 27m 9244k - 18.2.1 5be31c24972a 4f7a5fc3ac28 2024-03-21T10:25:54.381 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi153 smithi153 running (10m) 6m ago 28m 6656k - 19.0.0-2231-g2edfee68 237e43c7a65f 32c21104d5b0 2024-03-21T10:25:54.381 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi174 smithi174 running (10m) 5m ago 27m 6660k - 19.0.0-2231-g2edfee68 237e43c7a65f a04e48967ab6 2024-03-21T10:25:54.381 INFO:teuthology.orchestra.run.smithi153.stdout:grafana.smithi153 smithi153 *:3000 running (14m) 6m ago 27m 73.6M - 9.4.12 2bacad6d85d8 ee24026ab786 2024-03-21T10:25:54.382 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.iuyluj smithi153 running (6m) 6m ago 21m 10.9M - 19.0.0-2231-g2edfee68 237e43c7a65f 87aff8a5de70 2024-03-21T10:25:54.382 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.zipckr smithi153 running (6m) 6m ago 21m 11.8M - 19.0.0-2231-g2edfee68 237e43c7a65f ccc262bd5dd3 2024-03-21T10:25:54.382 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.nhozag smithi174 running (21m) 5m ago 21m 12.0M - 18.2.1 5be31c24972a 2fee4000308f 2024-03-21T10:25:54.382 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.yyimst smithi174 running (5m) 5m ago 21m 10.7M - 19.0.0-2231-g2edfee68 237e43c7a65f fc1962f9351e 2024-03-21T10:25:54.382 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi153.tluxni smithi153 *:8443,9283,8765 running (17m) 6m ago 30m 509M - 19.0.0-2231-g2edfee68 237e43c7a65f 7ffee0e55382 2024-03-21T10:25:54.382 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi174.qpcwat smithi174 *:8443,9283,8765 running (16m) 5m ago 26m 452M - 19.0.0-2231-g2edfee68 237e43c7a65f 748968f216ab 2024-03-21T10:25:54.382 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi153 smithi153 running (12m) 6m ago 30m 55.0M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f dd8038303dc3 2024-03-21T10:25:54.382 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi174 smithi174 running (11m) 5m ago 26m 44.2M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f 29d78a788f2c 2024-03-21T10:25:54.382 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi153 smithi153 *:9100 running (15m) 6m ago 28m 7889k - 1.5.0 0da6a335fe13 bb835edc0e0b 2024-03-21T10:25:54.382 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi174 smithi174 *:9100 running (15m) 5m ago 27m 8102k - 1.5.0 0da6a335fe13 7c4ecad790fa 2024-03-21T10:25:54.382 INFO:teuthology.orchestra.run.smithi153.stdout:osd.0 smithi153 running (10m) 6m ago 25m 116M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 507900fd4791 2024-03-21T10:25:54.382 INFO:teuthology.orchestra.run.smithi153.stdout:osd.1 smithi153 running (9m) 6m ago 25m 89.2M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 0d646452e9c1 2024-03-21T10:25:54.382 INFO:teuthology.orchestra.run.smithi153.stdout:osd.2 smithi153 running (9m) 6m ago 24m 87.4M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 1326f2afdfcd 2024-03-21T10:25:54.382 INFO:teuthology.orchestra.run.smithi153.stdout:osd.3 smithi174 running (8m) 5m ago 24m 83.6M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 70aafac71ad5 2024-03-21T10:25:54.383 INFO:teuthology.orchestra.run.smithi153.stdout:osd.4 smithi174 running (7m) 5m ago 23m 92.2M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 0d746ce6e79d 2024-03-21T10:25:54.383 INFO:teuthology.orchestra.run.smithi153.stdout:osd.5 smithi174 running (7m) 5m ago 23m 86.2M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 53e5f35b8cac 2024-03-21T10:25:54.383 INFO:teuthology.orchestra.run.smithi153.stdout:prometheus.smithi153 smithi153 *:9095 running (15m) 6m ago 27m 73.5M - 2.43.0 a07b618ecd1d 7d95561e50c2 2024-03-21T10:25:54.384 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.383+0000 7fe578a00700 1 -- 172.21.15.153:0/1166603645 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe55806e460 msgr2=0x7fe558070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:54.384 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.383+0000 7fe578a00700 1 --2- 172.21.15.153:0/1166603645 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe55806e460 0x7fe558070920 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe564000c00 tx=0x7fe564005fd0 comp rx=0 tx=0).stop 2024-03-21T10:25:54.385 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.383+0000 7fe578a00700 1 -- 172.21.15.153:0/1166603645 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe5741037c0 msgr2=0x7fe57419eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:54.385 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.383+0000 7fe578a00700 1 --2- 172.21.15.153:0/1166603645 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe5741037c0 0x7fe57419eac0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fe56000c960 tx=0x7fe56000cd20 comp rx=0 tx=0).stop 2024-03-21T10:25:54.385 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.383+0000 7fe578a00700 1 -- 172.21.15.153:0/1166603645 shutdown_connections 2024-03-21T10:25:54.385 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.383+0000 7fe578a00700 1 --2- 172.21.15.153:0/1166603645 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe574102e10 0x7fe57419e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:54.385 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.383+0000 7fe578a00700 1 --2- 172.21.15.153:0/1166603645 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe55806e460 0x7fe558070920 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:54.385 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.383+0000 7fe578a00700 1 --2- 172.21.15.153:0/1166603645 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe5741037c0 0x7fe57419eac0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:54.385 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.383+0000 7fe578a00700 1 -- 172.21.15.153:0/1166603645 >> 172.21.15.153:0/1166603645 conn(0x7fe5740fe6c0 msgr2=0x7fe5740ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:54.385 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.384+0000 7fe578a00700 1 -- 172.21.15.153:0/1166603645 shutdown_connections 2024-03-21T10:25:54.385 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.384+0000 7fe578a00700 1 -- 172.21.15.153:0/1166603645 wait complete. 2024-03-21T10:25:54.535 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.533+0000 7fa91bc00700 1 -- 172.21.15.153:0/615275007 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa914103000 msgr2=0x7fa9141033e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:54.535 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.533+0000 7fa91bc00700 1 --2- 172.21.15.153:0/615275007 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa914103000 0x7fa9141033e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fa908009910 tx=0x7fa908009c20 comp rx=0 tx=0).stop 2024-03-21T10:25:54.535 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.533+0000 7fa91bc00700 1 -- 172.21.15.153:0/615275007 shutdown_connections 2024-03-21T10:25:54.535 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.533+0000 7fa91bc00700 1 --2- 172.21.15.153:0/615275007 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa9141039b0 0x7fa914107a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:54.535 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.533+0000 7fa91bc00700 1 --2- 172.21.15.153:0/615275007 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa914103000 0x7fa9141033e0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:54.535 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.533+0000 7fa91bc00700 1 -- 172.21.15.153:0/615275007 >> 172.21.15.153:0/615275007 conn(0x7fa9140fe8d0 msgr2=0x7fa914100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:54.535 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.534+0000 7fa91bc00700 1 -- 172.21.15.153:0/615275007 shutdown_connections 2024-03-21T10:25:54.535 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.534+0000 7fa91bc00700 1 -- 172.21.15.153:0/615275007 wait complete. 2024-03-21T10:25:54.536 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.534+0000 7fa91bc00700 1 Processor -- start 2024-03-21T10:25:54.536 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.535+0000 7fa91bc00700 1 -- start start 2024-03-21T10:25:54.536 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.535+0000 7fa91bc00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa914103000 0x7fa91419e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:54.536 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.535+0000 7fa91bc00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa9141039b0 0x7fa91419ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:54.537 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.535+0000 7fa91bc00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa91419f3c0 con 0x7fa9141039b0 2024-03-21T10:25:54.537 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.535+0000 7fa91bc00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa914198870 con 0x7fa914103000 2024-03-21T10:25:54.537 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.535+0000 7fa913e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa9141039b0 0x7fa91419ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:54.537 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.535+0000 7fa913e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa9141039b0 0x7fa91419ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:50722/0 (socket says 172.21.15.153:50722) 2024-03-21T10:25:54.537 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.535+0000 7fa913e00700 1 -- 172.21.15.153:0/1722644804 learned_addr learned my addr 172.21.15.153:0/1722644804 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:25:54.538 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.536+0000 7fa913e00700 1 -- 172.21.15.153:0/1722644804 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa914103000 msgr2=0x7fa91419e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:54.538 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.536+0000 7fa913e00700 1 --2- 172.21.15.153:0/1722644804 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa914103000 0x7fa91419e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:54.538 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.536+0000 7fa913e00700 1 -- 172.21.15.153:0/1722644804 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9080095f0 con 0x7fa9141039b0 2024-03-21T10:25:54.538 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.536+0000 7fa913e00700 1 --2- 172.21.15.153:0/1722644804 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa9141039b0 0x7fa91419ed30 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fa90000d900 tx=0x7fa90000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:54.538 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.536+0000 7fa911600700 1 -- 172.21.15.153:0/1722644804 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa900009900 con 0x7fa9141039b0 2024-03-21T10:25:54.538 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.536+0000 7fa911600700 1 -- 172.21.15.153:0/1722644804 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa900010460 con 0x7fa9141039b0 2024-03-21T10:25:54.538 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.536+0000 7fa91bc00700 1 -- 172.21.15.153:0/1722644804 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa914198b50 con 0x7fa9141039b0 2024-03-21T10:25:54.538 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.536+0000 7fa911600700 1 -- 172.21.15.153:0/1722644804 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa90000f5f0 con 0x7fa9141039b0 2024-03-21T10:25:54.538 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.536+0000 7fa91bc00700 1 -- 172.21.15.153:0/1722644804 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9141990a0 con 0x7fa9141039b0 2024-03-21T10:25:54.539 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.537+0000 7fa91bc00700 1 -- 172.21.15.153:0/1722644804 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa91410b3a0 con 0x7fa9141039b0 2024-03-21T10:25:54.539 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.538+0000 7fa911600700 1 -- 172.21.15.153:0/1722644804 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fa9000105d0 con 0x7fa9141039b0 2024-03-21T10:25:54.540 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.539+0000 7fa911600700 1 --2- 172.21.15.153:0/1722644804 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa8fc06e460 0x7fa8fc070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:54.541 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.539+0000 7fa911600700 1 -- 172.21.15.153:0/1722644804 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fa90008e360 con 0x7fa9141039b0 2024-03-21T10:25:54.541 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.540+0000 7fa919200700 1 --2- 172.21.15.153:0/1722644804 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa8fc06e460 0x7fa8fc070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:54.541 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.540+0000 7fa919200700 1 --2- 172.21.15.153:0/1722644804 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa8fc06e460 0x7fa8fc070920 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa908006010 tx=0x7fa90800b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:54.545 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.544+0000 7fa911600700 1 -- 172.21.15.153:0/1722644804 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa90005af30 con 0x7fa9141039b0 2024-03-21T10:25:54.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:25:54 smithi153 ceph-mon[113733]: pgmap v421: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:25:54.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:25:54 smithi153 ceph-mon[113733]: from='client.34786 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:25:54.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:25:54 smithi153 ceph-mon[113733]: from='client.34790 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:25:54.833 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.832+0000 7fa91bc00700 1 -- 172.21.15.153:0/1722644804 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fa914199310 con 0x7fa9141039b0 2024-03-21T10:25:54.834 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.833+0000 7fa911600700 1 -- 172.21.15.153:0/1722644804 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+805 (secure 0 0 0) 0x7fa90005aac0 con 0x7fa9141039b0 2024-03-21T10:25:54.835 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:25:54.835 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": { 2024-03-21T10:25:54.835 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:25:54.835 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:25:54.835 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": { 2024-03-21T10:25:54.835 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:25:54.835 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:25:54.835 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": { 2024-03-21T10:25:54.835 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 6 2024-03-21T10:25:54.835 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:25:54.835 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": { 2024-03-21T10:25:54.836 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-03-21T10:25:54.836 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 3 2024-03-21T10:25:54.836 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:25:54.836 INFO:teuthology.orchestra.run.smithi153.stdout: "overall": { 2024-03-21T10:25:54.836 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-03-21T10:25:54.836 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 13 2024-03-21T10:25:54.836 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-03-21T10:25:54.836 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:25:54.837 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.836+0000 7fa91bc00700 1 -- 172.21.15.153:0/1722644804 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa8fc06e460 msgr2=0x7fa8fc070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:54.837 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.836+0000 7fa91bc00700 1 --2- 172.21.15.153:0/1722644804 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa8fc06e460 0x7fa8fc070920 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa908006010 tx=0x7fa90800b540 comp rx=0 tx=0).stop 2024-03-21T10:25:54.837 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.836+0000 7fa91bc00700 1 -- 172.21.15.153:0/1722644804 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa9141039b0 msgr2=0x7fa91419ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:54.838 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.836+0000 7fa91bc00700 1 --2- 172.21.15.153:0/1722644804 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa9141039b0 0x7fa91419ed30 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fa90000d900 tx=0x7fa90000dcc0 comp rx=0 tx=0).stop 2024-03-21T10:25:54.838 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.836+0000 7fa91bc00700 1 -- 172.21.15.153:0/1722644804 shutdown_connections 2024-03-21T10:25:54.838 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.836+0000 7fa91bc00700 1 --2- 172.21.15.153:0/1722644804 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa914103000 0x7fa91419e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:54.838 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.836+0000 7fa91bc00700 1 --2- 172.21.15.153:0/1722644804 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa8fc06e460 0x7fa8fc070920 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:54.838 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.836+0000 7fa91bc00700 1 --2- 172.21.15.153:0/1722644804 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa9141039b0 0x7fa91419ed30 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:54.838 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.836+0000 7fa91bc00700 1 -- 172.21.15.153:0/1722644804 >> 172.21.15.153:0/1722644804 conn(0x7fa9140fe8d0 msgr2=0x7fa9140ffee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:54.838 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.836+0000 7fa91bc00700 1 -- 172.21.15.153:0/1722644804 shutdown_connections 2024-03-21T10:25:54.838 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.836+0000 7fa91bc00700 1 -- 172.21.15.153:0/1722644804 wait complete. 2024-03-21T10:25:54.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:25:54 smithi174 ceph-mon[88004]: pgmap v421: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:25:54.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:25:54 smithi174 ceph-mon[88004]: from='client.34786 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:25:54.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:25:54 smithi174 ceph-mon[88004]: from='client.34790 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:25:54.993 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.991+0000 7fbe6d800700 1 -- 172.21.15.153:0/1644616236 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbe6810d280 msgr2=0x7fbe6810d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:54.993 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.991+0000 7fbe6d800700 1 --2- 172.21.15.153:0/1644616236 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbe6810d280 0x7fbe6810d660 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fbe58009910 tx=0x7fbe58009c20 comp rx=0 tx=0).stop 2024-03-21T10:25:54.993 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.992+0000 7fbe6d800700 1 -- 172.21.15.153:0/1644616236 shutdown_connections 2024-03-21T10:25:54.993 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.992+0000 7fbe6d800700 1 --2- 172.21.15.153:0/1644616236 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbe681071d0 0x7fbe68107650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:54.993 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.992+0000 7fbe6d800700 1 --2- 172.21.15.153:0/1644616236 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbe6810d280 0x7fbe6810d660 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:54.993 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.992+0000 7fbe6d800700 1 -- 172.21.15.153:0/1644616236 >> 172.21.15.153:0/1644616236 conn(0x7fbe680769e0 msgr2=0x7fbe68078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:54.993 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.992+0000 7fbe6d800700 1 -- 172.21.15.153:0/1644616236 shutdown_connections 2024-03-21T10:25:54.993 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.992+0000 7fbe6d800700 1 -- 172.21.15.153:0/1644616236 wait complete. 2024-03-21T10:25:54.995 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.993+0000 7fbe6d800700 1 Processor -- start 2024-03-21T10:25:54.995 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.993+0000 7fbe6d800700 1 -- start start 2024-03-21T10:25:54.995 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.993+0000 7fbe6d800700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbe681071d0 0x7fbe6819ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:54.995 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.993+0000 7fbe6d800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbe6810d280 0x7fbe6819d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:54.995 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.993+0000 7fbe6d800700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe6819d8a0 con 0x7fbe6810d280 2024-03-21T10:25:54.995 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.993+0000 7fbe6d800700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe681a05d0 con 0x7fbe681071d0 2024-03-21T10:25:54.995 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.994+0000 7fbe66000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbe6810d280 0x7fbe6819d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:54.996 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.994+0000 7fbe66000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbe6810d280 0x7fbe6819d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:50750/0 (socket says 172.21.15.153:50750) 2024-03-21T10:25:54.996 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.994+0000 7fbe66000700 1 -- 172.21.15.153:0/668916025 learned_addr learned my addr 172.21.15.153:0/668916025 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:25:54.996 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.994+0000 7fbe66a00700 1 --2- 172.21.15.153:0/668916025 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbe681071d0 0x7fbe6819ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:54.996 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.994+0000 7fbe66000700 1 -- 172.21.15.153:0/668916025 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbe681071d0 msgr2=0x7fbe6819ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:54.996 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.994+0000 7fbe66000700 1 --2- 172.21.15.153:0/668916025 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbe681071d0 0x7fbe6819ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:54.996 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.994+0000 7fbe66000700 1 -- 172.21.15.153:0/668916025 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe580095f0 con 0x7fbe6810d280 2024-03-21T10:25:54.996 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.994+0000 7fbe66a00700 1 --2- 172.21.15.153:0/668916025 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbe681071d0 0x7fbe6819ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:25:54.996 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.994+0000 7fbe66000700 1 --2- 172.21.15.153:0/668916025 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbe6810d280 0x7fbe6819d210 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fbe5400d8d0 tx=0x7fbe5400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:54.997 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.995+0000 7fbe5f400700 1 -- 172.21.15.153:0/668916025 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe54009940 con 0x7fbe6810d280 2024-03-21T10:25:54.997 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.995+0000 7fbe5f400700 1 -- 172.21.15.153:0/668916025 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbe54010460 con 0x7fbe6810d280 2024-03-21T10:25:54.997 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.995+0000 7fbe6d800700 1 -- 172.21.15.153:0/668916025 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe681a08b0 con 0x7fbe6810d280 2024-03-21T10:25:54.997 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.995+0000 7fbe5f400700 1 -- 172.21.15.153:0/668916025 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe5400f5f0 con 0x7fbe6810d280 2024-03-21T10:25:54.997 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.995+0000 7fbe6d800700 1 -- 172.21.15.153:0/668916025 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe681a0e80 con 0x7fbe6810d280 2024-03-21T10:25:54.997 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.996+0000 7fbe6d800700 1 -- 172.21.15.153:0/668916025 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe6804ea90 con 0x7fbe6810d280 2024-03-21T10:25:54.997 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.997+0000 7fbe5f400700 1 -- 172.21.15.153:0/668916025 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fbe540105d0 con 0x7fbe6810d280 2024-03-21T10:25:54.999 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.997+0000 7fbe5f400700 1 --2- 172.21.15.153:0/668916025 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fbe4c06e460 0x7fbe4c070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:54.999 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.998+0000 7fbe5f400700 1 -- 172.21.15.153:0/668916025 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fbe5408e510 con 0x7fbe6810d280 2024-03-21T10:25:54.999 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.998+0000 7fbe66a00700 1 --2- 172.21.15.153:0/668916025 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fbe4c06e460 0x7fbe4c070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:54.999 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:54.998+0000 7fbe66a00700 1 --2- 172.21.15.153:0/668916025 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fbe4c06e460 0x7fbe4c070920 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fbe58006010 tx=0x7fbe5800b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:55.003 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:55.002+0000 7fbe5f400700 1 -- 172.21.15.153:0/668916025 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fbe540575d0 con 0x7fbe6810d280 2024-03-21T10:25:55.259 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:55.257+0000 7fbe6d800700 1 -- 172.21.15.153:0/668916025 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fbe6819dff0 con 0x7fbe6810d280 2024-03-21T10:25:55.259 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:55.258+0000 7fbe5f400700 1 -- 172.21.15.153:0/668916025 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1876 (secure 0 0 0) 0x7fbe5405ac90 con 0x7fbe6810d280 2024-03-21T10:25:55.260 INFO:teuthology.orchestra.run.smithi153.stdout:e30 2024-03-21T10:25:55.260 INFO:teuthology.orchestra.run.smithi153.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-21T10:25:55.260 INFO:teuthology.orchestra.run.smithi153.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:25:55.260 INFO:teuthology.orchestra.run.smithi153.stdout:legacy client fscid: 1 2024-03-21T10:25:55.260 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:25:55.260 INFO:teuthology.orchestra.run.smithi153.stdout:Filesystem 'cephfs' (1) 2024-03-21T10:25:55.260 INFO:teuthology.orchestra.run.smithi153.stdout:fs_name cephfs 2024-03-21T10:25:55.260 INFO:teuthology.orchestra.run.smithi153.stdout:epoch 23 2024-03-21T10:25:55.260 INFO:teuthology.orchestra.run.smithi153.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-21T10:25:55.260 INFO:teuthology.orchestra.run.smithi153.stdout:created 2024-03-21T10:04:27.086576+0000 2024-03-21T10:25:55.260 INFO:teuthology.orchestra.run.smithi153.stdout:modified 2024-03-21T10:19:05.022092+0000 2024-03-21T10:25:55.261 INFO:teuthology.orchestra.run.smithi153.stdout:tableserver 0 2024-03-21T10:25:55.261 INFO:teuthology.orchestra.run.smithi153.stdout:root 0 2024-03-21T10:25:55.261 INFO:teuthology.orchestra.run.smithi153.stdout:session_timeout 60 2024-03-21T10:25:55.261 INFO:teuthology.orchestra.run.smithi153.stdout:session_autoclose 300 2024-03-21T10:25:55.261 INFO:teuthology.orchestra.run.smithi153.stdout:max_file_size 1099511627776 2024-03-21T10:25:55.261 INFO:teuthology.orchestra.run.smithi153.stdout:max_xattr_size 65536 2024-03-21T10:25:55.261 INFO:teuthology.orchestra.run.smithi153.stdout:required_client_features {} 2024-03-21T10:25:55.261 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure 0 2024-03-21T10:25:55.261 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure_osd_epoch 94 2024-03-21T10:25:55.261 INFO:teuthology.orchestra.run.smithi153.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:25:55.261 INFO:teuthology.orchestra.run.smithi153.stdout:max_mds 2 2024-03-21T10:25:55.261 INFO:teuthology.orchestra.run.smithi153.stdout:in 0,1 2024-03-21T10:25:55.261 INFO:teuthology.orchestra.run.smithi153.stdout:up {} 2024-03-21T10:25:55.262 INFO:teuthology.orchestra.run.smithi153.stdout:failed 0,1 2024-03-21T10:25:55.262 INFO:teuthology.orchestra.run.smithi153.stdout:damaged 2024-03-21T10:25:55.262 INFO:teuthology.orchestra.run.smithi153.stdout:stopped 2024-03-21T10:25:55.262 INFO:teuthology.orchestra.run.smithi153.stdout:data_pools [3] 2024-03-21T10:25:55.262 INFO:teuthology.orchestra.run.smithi153.stdout:metadata_pool 2 2024-03-21T10:25:55.262 INFO:teuthology.orchestra.run.smithi153.stdout:inline_data enabled 2024-03-21T10:25:55.262 INFO:teuthology.orchestra.run.smithi153.stdout:balancer 2024-03-21T10:25:55.262 INFO:teuthology.orchestra.run.smithi153.stdout:bal_rank_mask -1 2024-03-21T10:25:55.262 INFO:teuthology.orchestra.run.smithi153.stdout:standby_count_wanted 1 2024-03-21T10:25:55.262 INFO:teuthology.orchestra.run.smithi153.stdout:qdb_cluster leader: 0 members: 2024-03-21T10:25:55.262 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:25:55.262 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:25:55.262 INFO:teuthology.orchestra.run.smithi153.stdout:Standby daemons: 2024-03-21T10:25:55.262 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:25:55.263 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.zipckr{-1:34442} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6826/1103846295,v1:172.21.15.153:6827/1103846295] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:25:55.263 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.iuyluj{-1:34474} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6828/1738630832,v1:172.21.15.153:6829/1738630832] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:25:55.263 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.nhozag{-1:44269} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.174:6826/3967223227,v1:172.21.15.174:6827/3967223227] compat {c=[1],r=[1],i=[7ff]}] 2024-03-21T10:25:55.263 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.yyimst{-1:44313} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.174:6824/4127786797,v1:172.21.15.174:6825/4127786797] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:25:55.264 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:55.261+0000 7fbe6d800700 1 -- 172.21.15.153:0/668916025 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fbe4c06e460 msgr2=0x7fbe4c070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:55.264 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:55.261+0000 7fbe6d800700 1 --2- 172.21.15.153:0/668916025 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fbe4c06e460 0x7fbe4c070920 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fbe58006010 tx=0x7fbe5800b540 comp rx=0 tx=0).stop 2024-03-21T10:25:55.264 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:55.261+0000 7fbe6d800700 1 -- 172.21.15.153:0/668916025 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbe6810d280 msgr2=0x7fbe6819d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:55.264 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:55.261+0000 7fbe6d800700 1 --2- 172.21.15.153:0/668916025 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbe6810d280 0x7fbe6819d210 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fbe5400d8d0 tx=0x7fbe5400dc90 comp rx=0 tx=0).stop 2024-03-21T10:25:55.264 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:55.261+0000 7fbe6d800700 1 -- 172.21.15.153:0/668916025 shutdown_connections 2024-03-21T10:25:55.264 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:55.261+0000 7fbe6d800700 1 --2- 172.21.15.153:0/668916025 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbe681071d0 0x7fbe6819ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:55.264 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:55.261+0000 7fbe6d800700 1 --2- 172.21.15.153:0/668916025 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fbe4c06e460 0x7fbe4c070920 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:55.264 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:55.261+0000 7fbe6d800700 1 --2- 172.21.15.153:0/668916025 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbe6810d280 0x7fbe6819d210 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:55.264 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:55.261+0000 7fbe6d800700 1 -- 172.21.15.153:0/668916025 >> 172.21.15.153:0/668916025 conn(0x7fbe680769e0 msgr2=0x7fbe68078320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:55.264 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:55.261+0000 7fbe6d800700 1 -- 172.21.15.153:0/668916025 shutdown_connections 2024-03-21T10:25:55.264 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:55.261+0000 7fbe6d800700 1 -- 172.21.15.153:0/668916025 wait complete. 2024-03-21T10:25:55.264 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 30 2024-03-21T10:25:55.408 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:55.407+0000 7f4ea0c00700 1 -- 172.21.15.153:0/2760451068 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e9c107220 msgr2=0x7f4e9c107600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:55.408 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:55.407+0000 7f4ea0c00700 1 --2- 172.21.15.153:0/2760451068 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e9c107220 0x7f4e9c107600 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c009910 tx=0x7f4e8c009c20 comp rx=0 tx=0).stop 2024-03-21T10:25:55.409 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:55.407+0000 7f4ea0c00700 1 -- 172.21.15.153:0/2760451068 shutdown_connections 2024-03-21T10:25:55.409 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:55.407+0000 7f4ea0c00700 1 --2- 172.21.15.153:0/2760451068 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4e9c107bd0 0x7f4e9c10bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:55.409 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:55.407+0000 7f4ea0c00700 1 --2- 172.21.15.153:0/2760451068 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e9c107220 0x7f4e9c107600 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:55.409 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:55.407+0000 7f4ea0c00700 1 -- 172.21.15.153:0/2760451068 >> 172.21.15.153:0/2760451068 conn(0x7f4e9c0767e0 msgr2=0x7f4e9c078c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:55.409 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:55.407+0000 7f4ea0c00700 1 -- 172.21.15.153:0/2760451068 shutdown_connections 2024-03-21T10:25:55.409 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:55.407+0000 7f4ea0c00700 1 -- 172.21.15.153:0/2760451068 wait complete. 2024-03-21T10:25:55.411 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:55.408+0000 7f4ea0c00700 1 Processor -- start 2024-03-21T10:25:55.411 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:55.408+0000 7f4ea0c00700 1 -- start start 2024-03-21T10:25:55.411 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:55.408+0000 7f4ea0c00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4e9c107220 0x7f4e9c1a2a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:55.411 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:55.408+0000 7f4ea0c00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e9c107bd0 0x7f4e9c1a2f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:55.411 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:55.408+0000 7f4ea0c00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e9c1a35f0 con 0x7f4e9c107bd0 2024-03-21T10:25:55.411 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:55.408+0000 7f4ea0c00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e9c19caa0 con 0x7f4e9c107220 2024-03-21T10:25:55.411 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:55.409+0000 7f4e9aa00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e9c107bd0 0x7f4e9c1a2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:55.411 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:55.409+0000 7f4e9aa00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e9c107bd0 0x7f4e9c1a2f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:50778/0 (socket says 172.21.15.153:50778) 2024-03-21T10:25:55.411 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:55.409+0000 7f4e9aa00700 1 -- 172.21.15.153:0/752749555 learned_addr learned my addr 172.21.15.153:0/752749555 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:25:55.411 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:55.409+0000 7f4e9b400700 1 --2- 172.21.15.153:0/752749555 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4e9c107220 0x7f4e9c1a2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:55.412 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:55.409+0000 7f4e9aa00700 1 -- 172.21.15.153:0/752749555 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4e9c107220 msgr2=0x7f4e9c1a2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:55.412 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:55.409+0000 7f4e9aa00700 1 --2- 172.21.15.153:0/752749555 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4e9c107220 0x7f4e9c1a2a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:55.412 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:55.409+0000 7f4e9aa00700 1 -- 172.21.15.153:0/752749555 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e8c0095f0 con 0x7f4e9c107bd0 2024-03-21T10:25:55.412 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:55.409+0000 7f4e9aa00700 1 --2- 172.21.15.153:0/752749555 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e9c107bd0 0x7f4e9c1a2f60 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f4e8800c8f0 tx=0x7f4e8800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:55.412 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:55.410+0000 7f4e93e00700 1 -- 172.21.15.153:0/752749555 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4e88007a50 con 0x7f4e9c107bd0 2024-03-21T10:25:55.412 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:55.410+0000 7f4e93e00700 1 -- 172.21.15.153:0/752749555 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4e88007bb0 con 0x7f4e9c107bd0 2024-03-21T10:25:55.412 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:55.410+0000 7f4ea0c00700 1 -- 172.21.15.153:0/752749555 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e9c19cd80 con 0x7f4e9c107bd0 2024-03-21T10:25:55.412 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:55.410+0000 7f4e93e00700 1 -- 172.21.15.153:0/752749555 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4e88018630 con 0x7f4e9c107bd0 2024-03-21T10:25:55.412 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:55.410+0000 7f4ea0c00700 1 -- 172.21.15.153:0/752749555 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e9c19d2d0 con 0x7f4e9c107bd0 2024-03-21T10:25:55.413 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:55.411+0000 7f4ea0c00700 1 -- 172.21.15.153:0/752749555 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e9c10f5b0 con 0x7f4e9c107bd0 2024-03-21T10:25:55.413 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:55.412+0000 7f4e93e00700 1 -- 172.21.15.153:0/752749555 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f4e88007d20 con 0x7f4e9c107bd0 2024-03-21T10:25:55.414 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:55.413+0000 7f4e93e00700 1 --2- 172.21.15.153:0/752749555 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f4e8006e460 0x7f4e80070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:55.414 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:55.413+0000 7f4e93e00700 1 -- 172.21.15.153:0/752749555 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f4e8808e450 con 0x7f4e9c107bd0 2024-03-21T10:25:55.414 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:55.413+0000 7f4e9b400700 1 --2- 172.21.15.153:0/752749555 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f4e8006e460 0x7f4e80070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:55.414 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:55.414+0000 7f4e9b400700 1 --2- 172.21.15.153:0/752749555 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f4e8006e460 0x7f4e80070920 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c000c00 tx=0x7f4e8c005fd0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:55.418 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:55.418+0000 7f4e93e00700 1 -- 172.21.15.153:0/752749555 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4e8805b020 con 0x7f4e9c107bd0 2024-03-21T10:25:55.646 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:55.644+0000 7f4ea0c00700 1 -- 172.21.15.153:0/752749555 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4e9c19d5a0 con 0x7f4e8006e460 2024-03-21T10:25:55.648 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:55.647+0000 7f4e93e00700 1 -- 172.21.15.153:0/752749555 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f4e9c19d5a0 con 0x7f4e8006e460 2024-03-21T10:25:55.648 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:25:55.648 INFO:teuthology.orchestra.run.smithi153.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232", 2024-03-21T10:25:55.648 INFO:teuthology.orchestra.run.smithi153.stdout: "in_progress": true, 2024-03-21T10:25:55.648 INFO:teuthology.orchestra.run.smithi153.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-21T10:25:55.648 INFO:teuthology.orchestra.run.smithi153.stdout: "services_complete": [ 2024-03-21T10:25:55.648 INFO:teuthology.orchestra.run.smithi153.stdout: "crash", 2024-03-21T10:25:55.648 INFO:teuthology.orchestra.run.smithi153.stdout: "mon", 2024-03-21T10:25:55.648 INFO:teuthology.orchestra.run.smithi153.stdout: "osd", 2024-03-21T10:25:55.648 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr" 2024-03-21T10:25:55.649 INFO:teuthology.orchestra.run.smithi153.stdout: ], 2024-03-21T10:25:55.649 INFO:teuthology.orchestra.run.smithi153.stdout: "progress": "15/23 daemons upgraded", 2024-03-21T10:25:55.649 INFO:teuthology.orchestra.run.smithi153.stdout: "message": "Currently upgrading mds daemons", 2024-03-21T10:25:55.649 INFO:teuthology.orchestra.run.smithi153.stdout: "is_paused": false 2024-03-21T10:25:55.649 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:25:55.650 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:55.649+0000 7f4ea0c00700 1 -- 172.21.15.153:0/752749555 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f4e8006e460 msgr2=0x7f4e80070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:55.651 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:55.649+0000 7f4ea0c00700 1 --2- 172.21.15.153:0/752749555 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f4e8006e460 0x7f4e80070920 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c000c00 tx=0x7f4e8c005fd0 comp rx=0 tx=0).stop 2024-03-21T10:25:55.651 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:55.649+0000 7f4ea0c00700 1 -- 172.21.15.153:0/752749555 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e9c107bd0 msgr2=0x7f4e9c1a2f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:55.651 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:55.649+0000 7f4ea0c00700 1 --2- 172.21.15.153:0/752749555 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e9c107bd0 0x7f4e9c1a2f60 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f4e8800c8f0 tx=0x7f4e8800ccb0 comp rx=0 tx=0).stop 2024-03-21T10:25:55.651 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:55.649+0000 7f4ea0c00700 1 -- 172.21.15.153:0/752749555 shutdown_connections 2024-03-21T10:25:55.651 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:55.649+0000 7f4ea0c00700 1 --2- 172.21.15.153:0/752749555 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4e9c107220 0x7f4e9c1a2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:55.651 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:55.649+0000 7f4ea0c00700 1 --2- 172.21.15.153:0/752749555 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f4e8006e460 0x7f4e80070920 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:55.651 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:55.649+0000 7f4ea0c00700 1 --2- 172.21.15.153:0/752749555 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4e9c107bd0 0x7f4e9c1a2f60 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:55.651 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:55.649+0000 7f4ea0c00700 1 -- 172.21.15.153:0/752749555 >> 172.21.15.153:0/752749555 conn(0x7f4e9c0767e0 msgr2=0x7f4e9c077e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:55.651 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:55.649+0000 7f4ea0c00700 1 -- 172.21.15.153:0/752749555 shutdown_connections 2024-03-21T10:25:55.651 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:55.650+0000 7f4ea0c00700 1 -- 172.21.15.153:0/752749555 wait complete. 2024-03-21T10:25:55.792 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:55.791+0000 7f6744600700 1 -- 172.21.15.153:0/3482439442 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f673c1071d0 msgr2=0x7f673c107650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:55.792 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:55.791+0000 7f6744600700 1 --2- 172.21.15.153:0/3482439442 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f673c1071d0 0x7f673c107650 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f6728009910 tx=0x7f6728009c20 comp rx=0 tx=0).stop 2024-03-21T10:25:55.792 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:55.791+0000 7f6744600700 1 -- 172.21.15.153:0/3482439442 shutdown_connections 2024-03-21T10:25:55.792 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:55.791+0000 7f6744600700 1 --2- 172.21.15.153:0/3482439442 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f673c1071d0 0x7f673c107650 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:55.793 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:55.791+0000 7f6744600700 1 --2- 172.21.15.153:0/3482439442 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f673c10d280 0x7f673c10d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:55.793 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:55.791+0000 7f6744600700 1 -- 172.21.15.153:0/3482439442 >> 172.21.15.153:0/3482439442 conn(0x7f673c0769e0 msgr2=0x7f673c078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:55.793 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:55.791+0000 7f6744600700 1 -- 172.21.15.153:0/3482439442 shutdown_connections 2024-03-21T10:25:55.793 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:55.791+0000 7f6744600700 1 -- 172.21.15.153:0/3482439442 wait complete. 2024-03-21T10:25:55.794 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:55.792+0000 7f6744600700 1 Processor -- start 2024-03-21T10:25:55.794 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:55.792+0000 7f6744600700 1 -- start start 2024-03-21T10:25:55.794 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:55.792+0000 7f6744600700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f673c1071d0 0x7f673c19ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:55.794 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:55.792+0000 7f6744600700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f673c10d280 0x7f673c19d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:55.794 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:55.792+0000 7f6744600700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f673c19d880 con 0x7f673c10d280 2024-03-21T10:25:55.795 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:55.792+0000 7f6744600700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f673c1a05b0 con 0x7f673c1071d0 2024-03-21T10:25:55.795 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:55.793+0000 7f6741200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f673c10d280 0x7f673c19d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:55.795 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:55.793+0000 7f6741200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f673c10d280 0x7f673c19d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:50794/0 (socket says 172.21.15.153:50794) 2024-03-21T10:25:55.795 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:55.793+0000 7f6741200700 1 -- 172.21.15.153:0/2856766924 learned_addr learned my addr 172.21.15.153:0/2856766924 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:25:55.795 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:55.793+0000 7f6741c00700 1 --2- 172.21.15.153:0/2856766924 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f673c1071d0 0x7f673c19ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:55.795 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:55.793+0000 7f6741200700 1 -- 172.21.15.153:0/2856766924 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f673c1071d0 msgr2=0x7f673c19ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:55.795 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:55.793+0000 7f6741200700 1 --2- 172.21.15.153:0/2856766924 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f673c1071d0 0x7f673c19ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:55.795 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:55.793+0000 7f6741200700 1 -- 172.21.15.153:0/2856766924 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67280095f0 con 0x7f673c10d280 2024-03-21T10:25:55.795 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:55.793+0000 7f6741c00700 1 --2- 172.21.15.153:0/2856766924 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f673c1071d0 0x7f673c19ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:25:55.795 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:55.793+0000 7f6741200700 1 --2- 172.21.15.153:0/2856766924 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f673c10d280 0x7f673c19d1f0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f67280092e0 tx=0x7f672800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:55.796 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:55.794+0000 7f673a000700 1 -- 172.21.15.153:0/2856766924 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f672801d070 con 0x7f673c10d280 2024-03-21T10:25:55.796 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:55.794+0000 7f673a000700 1 -- 172.21.15.153:0/2856766924 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6728005e00 con 0x7f673c10d280 2024-03-21T10:25:55.796 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:55.794+0000 7f673a000700 1 -- 172.21.15.153:0/2856766924 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6728017400 con 0x7f673c10d280 2024-03-21T10:25:55.796 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:55.794+0000 7f6744600700 1 -- 172.21.15.153:0/2856766924 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f673c1a0830 con 0x7f673c10d280 2024-03-21T10:25:55.796 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:55.794+0000 7f6744600700 1 -- 172.21.15.153:0/2856766924 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f673c1a0da0 con 0x7f673c10d280 2024-03-21T10:25:55.796 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:55.795+0000 7f6744600700 1 -- 172.21.15.153:0/2856766924 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f673c10f6c0 con 0x7f673c10d280 2024-03-21T10:25:55.797 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:55.796+0000 7f673a000700 1 -- 172.21.15.153:0/2856766924 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f672800f460 con 0x7f673c10d280 2024-03-21T10:25:55.798 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:55.797+0000 7f673a000700 1 --2- 172.21.15.153:0/2856766924 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f672406e460 0x7f6724070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:25:55.798 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:55.797+0000 7f673a000700 1 -- 172.21.15.153:0/2856766924 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f672808fa80 con 0x7f673c10d280 2024-03-21T10:25:55.798 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:55.797+0000 7f6741c00700 1 --2- 172.21.15.153:0/2856766924 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f672406e460 0x7f6724070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:25:55.799 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:55.798+0000 7f6741c00700 1 --2- 172.21.15.153:0/2856766924 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f672406e460 0x7f6724070920 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f6730005fd0 tx=0x7f6730005ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:25:55.802 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:55.801+0000 7f673a000700 1 -- 172.21.15.153:0/2856766924 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f6728058aa0 con 0x7f673c10d280 2024-03-21T10:25:55.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:25:55 smithi174 ceph-mon[88004]: from='client.34794 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:25:55.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:25:55 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/1722644804' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:25:55.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:25:55 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/668916025' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:25:56.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:25:55 smithi153 ceph-mon[113733]: from='client.34794 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:25:56.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:25:55 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/1722644804' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:25:56.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:25:55 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/668916025' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:25:56.096 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:56.095+0000 7f6744600700 1 -- 172.21.15.153:0/2856766924 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f673c04ea90 con 0x7f673c10d280 2024-03-21T10:25:56.096 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:56.095+0000 7f673a000700 1 -- 172.21.15.153:0/2856766924 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f672805c160 con 0x7f673c10d280 2024-03-21T10:25:56.097 INFO:teuthology.orchestra.run.smithi153.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-03-21T10:25:56.097 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-21T10:25:56.097 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is degraded 2024-03-21T10:25:56.097 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-21T10:25:56.097 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-21T10:25:56.097 INFO:teuthology.orchestra.run.smithi153.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-21T10:25:56.097 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-21T10:25:56.099 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:56.098+0000 7f6744600700 1 -- 172.21.15.153:0/2856766924 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f672406e460 msgr2=0x7f6724070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:56.099 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:56.098+0000 7f6744600700 1 --2- 172.21.15.153:0/2856766924 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f672406e460 0x7f6724070920 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f6730005fd0 tx=0x7f6730005ee0 comp rx=0 tx=0).stop 2024-03-21T10:25:56.099 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:56.098+0000 7f6744600700 1 -- 172.21.15.153:0/2856766924 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f673c10d280 msgr2=0x7f673c19d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:25:56.100 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:56.098+0000 7f6744600700 1 --2- 172.21.15.153:0/2856766924 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f673c10d280 0x7f673c19d1f0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f67280092e0 tx=0x7f672800be20 comp rx=0 tx=0).stop 2024-03-21T10:25:56.100 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:56.098+0000 7f6744600700 1 -- 172.21.15.153:0/2856766924 shutdown_connections 2024-03-21T10:25:56.100 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:56.099+0000 7f6744600700 1 --2- 172.21.15.153:0/2856766924 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f673c1071d0 0x7f673c19ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:56.100 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:56.099+0000 7f6744600700 1 --2- 172.21.15.153:0/2856766924 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f672406e460 0x7f6724070920 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:56.100 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:56.099+0000 7f6744600700 1 --2- 172.21.15.153:0/2856766924 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f673c10d280 0x7f673c19d1f0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:25:56.100 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:56.099+0000 7f6744600700 1 -- 172.21.15.153:0/2856766924 >> 172.21.15.153:0/2856766924 conn(0x7f673c0769e0 msgr2=0x7f673c078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:25:56.100 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:56.099+0000 7f6744600700 1 -- 172.21.15.153:0/2856766924 shutdown_connections 2024-03-21T10:25:56.100 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:25:56.099+0000 7f6744600700 1 -- 172.21.15.153:0/2856766924 wait complete. 2024-03-21T10:25:56.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:25:56 smithi174 ceph-mon[88004]: pgmap v422: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:25:56.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:25:56 smithi174 ceph-mon[88004]: from='client.34806 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:25:56.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:25:56 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/2856766924' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:25:57.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:25:56 smithi153 ceph-mon[113733]: pgmap v422: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:25:57.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:25:56 smithi153 ceph-mon[113733]: from='client.34806 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:25:57.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:25:56 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/2856766924' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:25:58.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:25:58 smithi174 ceph-mon[88004]: pgmap v423: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:25:59.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:25:58 smithi153 ceph-mon[113733]: pgmap v423: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:26:00.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:26:00 smithi174 ceph-mon[88004]: pgmap v424: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:26:01.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:26:00 smithi153 ceph-mon[113733]: pgmap v424: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:26:01.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:26:01 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:26:02.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:26:01 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:26:02.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:26:02 smithi174 ceph-mon[88004]: pgmap v425: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:26:03.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:26:02 smithi153 ceph-mon[113733]: pgmap v425: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:26:04.862 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:26:04 smithi174 ceph-mon[88004]: pgmap v426: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:26:05.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:26:04 smithi153 ceph-mon[113733]: pgmap v426: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:26:06.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:26:06 smithi174 ceph-mon[88004]: pgmap v427: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:26:07.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:26:06 smithi153 ceph-mon[113733]: pgmap v427: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:26:08.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:26:08 smithi174 ceph-mon[88004]: pgmap v428: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:26:09.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:26:08 smithi153 ceph-mon[113733]: pgmap v428: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:26:10.701 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:26:10 smithi174 ceph-mon[88004]: pgmap v429: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:26:11.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:26:10 smithi153 ceph-mon[113733]: pgmap v429: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:26:12.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:26:12 smithi174 ceph-mon[88004]: pgmap v430: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:26:13.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:26:12 smithi153 ceph-mon[113733]: pgmap v430: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:26:14.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:26:14 smithi174 ceph-mon[88004]: pgmap v431: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:26:15.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:26:14 smithi153 ceph-mon[113733]: pgmap v431: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:26:16.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:26:16 smithi174 ceph-mon[88004]: pgmap v432: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:26:16.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:26:16 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:26:17.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:26:16 smithi153 ceph-mon[113733]: pgmap v432: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:26:17.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:26:16 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:26:18.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:26:18 smithi174 ceph-mon[88004]: pgmap v433: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:26:19.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:26:18 smithi153 ceph-mon[113733]: pgmap v433: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:26:20.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:26:20 smithi174 ceph-mon[88004]: pgmap v434: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:26:21.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:26:20 smithi153 ceph-mon[113733]: pgmap v434: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:26:22.950 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:26:22 smithi174 ceph-mon[88004]: pgmap v435: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:26:22.987 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:26:22 smithi153 ceph-mon[113733]: pgmap v435: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:26:24.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:26:24 smithi174 ceph-mon[88004]: pgmap v436: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:26:25.055 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:26:24 smithi153 ceph-mon[113733]: pgmap v436: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:26:26.268 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:26.268+0000 7ffa46200700 1 -- 172.21.15.153:0/4046652708 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffa40073a80 msgr2=0x7ffa4010cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:26.268 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:26.268+0000 7ffa46200700 1 --2- 172.21.15.153:0/4046652708 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffa40073a80 0x7ffa4010cec0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7ffa28009910 tx=0x7ffa28009c20 comp rx=0 tx=0).stop 2024-03-21T10:26:26.268 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:26.268+0000 7ffa46200700 1 -- 172.21.15.153:0/4046652708 shutdown_connections 2024-03-21T10:26:26.269 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:26.268+0000 7ffa46200700 1 --2- 172.21.15.153:0/4046652708 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffa40073a80 0x7ffa4010cec0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:26.269 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:26.268+0000 7ffa46200700 1 --2- 172.21.15.153:0/4046652708 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffa40073160 0x7ffa40073540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:26.269 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:26.268+0000 7ffa46200700 1 -- 172.21.15.153:0/4046652708 >> 172.21.15.153:0/4046652708 conn(0x7ffa400fc650 msgr2=0x7ffa400fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:26.269 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:26.268+0000 7ffa46200700 1 -- 172.21.15.153:0/4046652708 shutdown_connections 2024-03-21T10:26:26.269 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:26.268+0000 7ffa46200700 1 -- 172.21.15.153:0/4046652708 wait complete. 2024-03-21T10:26:26.270 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:26.269+0000 7ffa46200700 1 Processor -- start 2024-03-21T10:26:26.270 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:26.269+0000 7ffa46200700 1 -- start start 2024-03-21T10:26:26.270 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:26.270+0000 7ffa46200700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffa40073160 0x7ffa40198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:26.271 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:26.270+0000 7ffa46200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffa40073a80 0x7ffa40198eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:26.271 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:26.270+0000 7ffa46200700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa40199540 con 0x7ffa40073a80 2024-03-21T10:26:26.271 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:26.270+0000 7ffa46200700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4019d350 con 0x7ffa40073160 2024-03-21T10:26:26.271 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:26.270+0000 7ffa36a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffa40073a80 0x7ffa40198eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:26.271 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:26.270+0000 7ffa36a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffa40073a80 0x7ffa40198eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:54830/0 (socket says 172.21.15.153:54830) 2024-03-21T10:26:26.271 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:26.270+0000 7ffa36a00700 1 -- 172.21.15.153:0/886894709 learned_addr learned my addr 172.21.15.153:0/886894709 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:26:26.271 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:26.270+0000 7ffa3f400700 1 --2- 172.21.15.153:0/886894709 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffa40073160 0x7ffa40198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:26.271 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:26.270+0000 7ffa36a00700 1 -- 172.21.15.153:0/886894709 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffa40073160 msgr2=0x7ffa40198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:26.271 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:26.270+0000 7ffa36a00700 1 --2- 172.21.15.153:0/886894709 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffa40073160 0x7ffa40198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:26.271 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:26.270+0000 7ffa36a00700 1 -- 172.21.15.153:0/886894709 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa280095f0 con 0x7ffa40073a80 2024-03-21T10:26:26.271 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:26.270+0000 7ffa3f400700 1 --2- 172.21.15.153:0/886894709 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffa40073160 0x7ffa40198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:26:26.271 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:26.271+0000 7ffa36a00700 1 --2- 172.21.15.153:0/886894709 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffa40073a80 0x7ffa40198eb0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7ffa280092e0 tx=0x7ffa2800be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:26.272 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:26.271+0000 7ffa3cc00700 1 -- 172.21.15.153:0/886894709 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffa2801d070 con 0x7ffa40073a80 2024-03-21T10:26:26.272 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:26.271+0000 7ffa3cc00700 1 -- 172.21.15.153:0/886894709 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ffa2800f460 con 0x7ffa40073a80 2024-03-21T10:26:26.272 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:26.271+0000 7ffa3cc00700 1 -- 172.21.15.153:0/886894709 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffa28017630 con 0x7ffa40073a80 2024-03-21T10:26:26.272 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:26.271+0000 7ffa46200700 1 -- 172.21.15.153:0/886894709 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa4019d5d0 con 0x7ffa40073a80 2024-03-21T10:26:26.272 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:26.271+0000 7ffa46200700 1 -- 172.21.15.153:0/886894709 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa4019dac0 con 0x7ffa40073a80 2024-03-21T10:26:26.273 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:26.272+0000 7ffa46200700 1 -- 172.21.15.153:0/886894709 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa4010a5c0 con 0x7ffa40073a80 2024-03-21T10:26:26.273 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:26.273+0000 7ffa3cc00700 1 -- 172.21.15.153:0/886894709 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7ffa2800f5d0 con 0x7ffa40073a80 2024-03-21T10:26:26.274 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:26.274+0000 7ffa3cc00700 1 --2- 172.21.15.153:0/886894709 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ffa2406e460 0x7ffa24070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:26.274 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:26.274+0000 7ffa3cc00700 1 -- 172.21.15.153:0/886894709 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7ffa2808fab0 con 0x7ffa40073a80 2024-03-21T10:26:26.274 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:26.275+0000 7ffa3f400700 1 --2- 172.21.15.153:0/886894709 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ffa2406e460 0x7ffa24070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:26.275 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:26.275+0000 7ffa3f400700 1 --2- 172.21.15.153:0/886894709 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ffa2406e460 0x7ffa24070920 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ffa300060b0 tx=0x7ffa300076a0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:26.279 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:26.280+0000 7ffa3cc00700 1 -- 172.21.15.153:0/886894709 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ffa28091e60 con 0x7ffa40073a80 2024-03-21T10:26:26.521 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:26.521+0000 7ffa46200700 1 -- 172.21.15.153:0/886894709 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ffa4019ddf0 con 0x7ffa2406e460 2024-03-21T10:26:26.525 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:26.524+0000 7ffa3cc00700 1 -- 172.21.15.153:0/886894709 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7ffa4019ddf0 con 0x7ffa2406e460 2024-03-21T10:26:26.528 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:26.527+0000 7ffa46200700 1 -- 172.21.15.153:0/886894709 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ffa2406e460 msgr2=0x7ffa24070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:26.528 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:26.527+0000 7ffa46200700 1 --2- 172.21.15.153:0/886894709 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ffa2406e460 0x7ffa24070920 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ffa300060b0 tx=0x7ffa300076a0 comp rx=0 tx=0).stop 2024-03-21T10:26:26.528 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:26.527+0000 7ffa46200700 1 -- 172.21.15.153:0/886894709 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffa40073a80 msgr2=0x7ffa40198eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:26.528 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:26.527+0000 7ffa46200700 1 --2- 172.21.15.153:0/886894709 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffa40073a80 0x7ffa40198eb0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7ffa280092e0 tx=0x7ffa2800be00 comp rx=0 tx=0).stop 2024-03-21T10:26:26.528 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:26.527+0000 7ffa46200700 1 -- 172.21.15.153:0/886894709 shutdown_connections 2024-03-21T10:26:26.528 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:26.527+0000 7ffa46200700 1 --2- 172.21.15.153:0/886894709 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffa40073160 0x7ffa40198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:26.528 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:26.527+0000 7ffa46200700 1 --2- 172.21.15.153:0/886894709 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ffa2406e460 0x7ffa24070920 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:26.528 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:26.527+0000 7ffa46200700 1 --2- 172.21.15.153:0/886894709 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffa40073a80 0x7ffa40198eb0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:26.528 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:26.528+0000 7ffa46200700 1 -- 172.21.15.153:0/886894709 >> 172.21.15.153:0/886894709 conn(0x7ffa400fc650 msgr2=0x7ffa40107700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:26.528 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:26.528+0000 7ffa46200700 1 -- 172.21.15.153:0/886894709 shutdown_connections 2024-03-21T10:26:26.528 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:26.528+0000 7ffa46200700 1 -- 172.21.15.153:0/886894709 wait complete. 2024-03-21T10:26:26.544 INFO:teuthology.orchestra.run.smithi153.stdout:true 2024-03-21T10:26:26.686 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:26.686+0000 7fa5c3200700 1 -- 172.21.15.153:0/2319593319 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa5bc101770 msgr2=0x7fa5bc1057c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:26.687 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:26.686+0000 7fa5c3200700 1 --2- 172.21.15.153:0/2319593319 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa5bc101770 0x7fa5bc1057c0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fa5a8009910 tx=0x7fa5a8009c20 comp rx=0 tx=0).stop 2024-03-21T10:26:26.687 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:26.686+0000 7fa5c3200700 1 -- 172.21.15.153:0/2319593319 shutdown_connections 2024-03-21T10:26:26.687 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:26.686+0000 7fa5c3200700 1 --2- 172.21.15.153:0/2319593319 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa5bc101770 0x7fa5bc1057c0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:26.687 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:26.686+0000 7fa5c3200700 1 --2- 172.21.15.153:0/2319593319 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa5bc100dc0 0x7fa5bc1011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:26.687 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:26.686+0000 7fa5c3200700 1 -- 172.21.15.153:0/2319593319 >> 172.21.15.153:0/2319593319 conn(0x7fa5bc0fc650 msgr2=0x7fa5bc0fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:26.687 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:26.686+0000 7fa5c3200700 1 -- 172.21.15.153:0/2319593319 shutdown_connections 2024-03-21T10:26:26.687 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:26.686+0000 7fa5c3200700 1 -- 172.21.15.153:0/2319593319 wait complete. 2024-03-21T10:26:26.689 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:26.687+0000 7fa5c3200700 1 Processor -- start 2024-03-21T10:26:26.689 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:26.687+0000 7fa5c3200700 1 -- start start 2024-03-21T10:26:26.689 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:26.688+0000 7fa5c3200700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa5bc100dc0 0x7fa5bc198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:26.689 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:26.688+0000 7fa5c3200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa5bc101770 0x7fa5bc198ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:26.689 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:26.688+0000 7fa5c3200700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5bc1994c0 con 0x7fa5bc101770 2024-03-21T10:26:26.689 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:26.688+0000 7fa5c3200700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5bc199630 con 0x7fa5bc100dc0 2024-03-21T10:26:26.690 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:26.688+0000 7fa5bb400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa5bc101770 0x7fa5bc198ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:26.690 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:26.688+0000 7fa5bb400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa5bc101770 0x7fa5bc198ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:54844/0 (socket says 172.21.15.153:54844) 2024-03-21T10:26:26.690 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:26.688+0000 7fa5bb400700 1 -- 172.21.15.153:0/3485183352 learned_addr learned my addr 172.21.15.153:0/3485183352 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:26:26.690 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:26.688+0000 7fa5bbe00700 1 --2- 172.21.15.153:0/3485183352 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa5bc100dc0 0x7fa5bc198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:26.690 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:26.689+0000 7fa5bb400700 1 -- 172.21.15.153:0/3485183352 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa5bc100dc0 msgr2=0x7fa5bc198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:26.690 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:26.689+0000 7fa5bb400700 1 --2- 172.21.15.153:0/3485183352 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa5bc100dc0 0x7fa5bc198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:26.690 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:26.689+0000 7fa5bb400700 1 -- 172.21.15.153:0/3485183352 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5a80095f0 con 0x7fa5bc101770 2024-03-21T10:26:26.690 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:26.689+0000 7fa5bb400700 1 --2- 172.21.15.153:0/3485183352 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa5bc101770 0x7fa5bc198ec0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fa5a80092e0 tx=0x7fa5a800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:26.690 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:26.689+0000 7fa5b8c00700 1 -- 172.21.15.153:0/3485183352 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa5a801d070 con 0x7fa5bc101770 2024-03-21T10:26:26.690 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:26.689+0000 7fa5b8c00700 1 -- 172.21.15.153:0/3485183352 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa5a800f460 con 0x7fa5bc101770 2024-03-21T10:26:26.690 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:26.689+0000 7fa5c3200700 1 -- 172.21.15.153:0/3485183352 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5bc19d450 con 0x7fa5bc101770 2024-03-21T10:26:26.690 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:26.689+0000 7fa5b8c00700 1 -- 172.21.15.153:0/3485183352 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa5a80175f0 con 0x7fa5bc101770 2024-03-21T10:26:26.691 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:26.689+0000 7fa5c3200700 1 -- 172.21.15.153:0/3485183352 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5bc19d940 con 0x7fa5bc101770 2024-03-21T10:26:26.691 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:26.690+0000 7fa5c3200700 1 -- 172.21.15.153:0/3485183352 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5bc104f40 con 0x7fa5bc101770 2024-03-21T10:26:26.691 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:26.691+0000 7fa5b8c00700 1 -- 172.21.15.153:0/3485183352 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fa5a800f5d0 con 0x7fa5bc101770 2024-03-21T10:26:26.693 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:26.693+0000 7fa5b8c00700 1 --2- 172.21.15.153:0/3485183352 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa5a006e460 0x7fa5a0070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:26.693 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:26.693+0000 7fa5b8c00700 1 -- 172.21.15.153:0/3485183352 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fa5a808fa60 con 0x7fa5bc101770 2024-03-21T10:26:26.693 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:26.693+0000 7fa5bbe00700 1 --2- 172.21.15.153:0/3485183352 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa5a006e460 0x7fa5a0070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:26.693 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:26.694+0000 7fa5bbe00700 1 --2- 172.21.15.153:0/3485183352 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa5a006e460 0x7fa5a0070920 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac005ea0 tx=0x7fa5ac005e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:26.697 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:26.698+0000 7fa5b8c00700 1 -- 172.21.15.153:0/3485183352 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa5a805c650 con 0x7fa5bc101770 2024-03-21T10:26:26.806 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:26:26 smithi153 ceph-mon[113733]: pgmap v437: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:26:26.941 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:26.940+0000 7fa5c3200700 1 -- 172.21.15.153:0/3485183352 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa5bc19dc20 con 0x7fa5a006e460 2024-03-21T10:26:26.943 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:26.943+0000 7fa5b8c00700 1 -- 172.21.15.153:0/3485183352 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fa5bc19dc20 con 0x7fa5a006e460 2024-03-21T10:26:26.945 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:26.946+0000 7fa5c3200700 1 -- 172.21.15.153:0/3485183352 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa5a006e460 msgr2=0x7fa5a0070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:26.946 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:26.946+0000 7fa5c3200700 1 --2- 172.21.15.153:0/3485183352 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa5a006e460 0x7fa5a0070920 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac005ea0 tx=0x7fa5ac005e30 comp rx=0 tx=0).stop 2024-03-21T10:26:26.946 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:26.946+0000 7fa5c3200700 1 -- 172.21.15.153:0/3485183352 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa5bc101770 msgr2=0x7fa5bc198ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:26.946 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:26.946+0000 7fa5c3200700 1 --2- 172.21.15.153:0/3485183352 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa5bc101770 0x7fa5bc198ec0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fa5a80092e0 tx=0x7fa5a800be20 comp rx=0 tx=0).stop 2024-03-21T10:26:26.946 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:26.946+0000 7fa5c3200700 1 -- 172.21.15.153:0/3485183352 shutdown_connections 2024-03-21T10:26:26.946 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:26.946+0000 7fa5c3200700 1 --2- 172.21.15.153:0/3485183352 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa5bc100dc0 0x7fa5bc198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:26.947 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:26.946+0000 7fa5c3200700 1 --2- 172.21.15.153:0/3485183352 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa5a006e460 0x7fa5a0070920 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:26.947 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:26.946+0000 7fa5c3200700 1 --2- 172.21.15.153:0/3485183352 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa5bc101770 0x7fa5bc198ec0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:26.947 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:26.946+0000 7fa5c3200700 1 -- 172.21.15.153:0/3485183352 >> 172.21.15.153:0/3485183352 conn(0x7fa5bc0fc650 msgr2=0x7fa5bc0fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:26.947 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:26.946+0000 7fa5c3200700 1 -- 172.21.15.153:0/3485183352 shutdown_connections 2024-03-21T10:26:26.947 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:26.946+0000 7fa5c3200700 1 -- 172.21.15.153:0/3485183352 wait complete. 2024-03-21T10:26:26.950 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:26:26 smithi174 ceph-mon[88004]: pgmap v437: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:26:27.102 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.102+0000 7fa28c600700 1 -- 172.21.15.153:0/2466470863 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa284100dc0 msgr2=0x7fa2841011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:27.102 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.102+0000 7fa28c600700 1 --2- 172.21.15.153:0/2466470863 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa284100dc0 0x7fa2841011a0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fa278009910 tx=0x7fa278009c20 comp rx=0 tx=0).stop 2024-03-21T10:26:27.102 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.102+0000 7fa28c600700 1 -- 172.21.15.153:0/2466470863 shutdown_connections 2024-03-21T10:26:27.102 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.102+0000 7fa28c600700 1 --2- 172.21.15.153:0/2466470863 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa284101770 0x7fa2841057c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:27.102 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.102+0000 7fa28c600700 1 --2- 172.21.15.153:0/2466470863 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa284100dc0 0x7fa2841011a0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:27.103 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.102+0000 7fa28c600700 1 -- 172.21.15.153:0/2466470863 >> 172.21.15.153:0/2466470863 conn(0x7fa2840fc650 msgr2=0x7fa2840fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:27.103 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.102+0000 7fa28c600700 1 -- 172.21.15.153:0/2466470863 shutdown_connections 2024-03-21T10:26:27.103 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.102+0000 7fa28c600700 1 -- 172.21.15.153:0/2466470863 wait complete. 2024-03-21T10:26:27.104 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.103+0000 7fa28c600700 1 Processor -- start 2024-03-21T10:26:27.104 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.103+0000 7fa28c600700 1 -- start start 2024-03-21T10:26:27.104 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.103+0000 7fa28c600700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa284100dc0 0x7fa28419e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:27.104 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.104+0000 7fa28c600700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa284101770 0x7fa28419ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:27.104 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.104+0000 7fa28c600700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa28419f2c0 con 0x7fa284101770 2024-03-21T10:26:27.104 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.104+0000 7fa28c600700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa284198770 con 0x7fa284100dc0 2024-03-21T10:26:27.104 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.104+0000 7fa289200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa284101770 0x7fa28419ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:27.105 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.104+0000 7fa289200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa284101770 0x7fa28419ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:54858/0 (socket says 172.21.15.153:54858) 2024-03-21T10:26:27.105 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.104+0000 7fa289200700 1 -- 172.21.15.153:0/1166732488 learned_addr learned my addr 172.21.15.153:0/1166732488 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:26:27.105 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.104+0000 7fa289c00700 1 --2- 172.21.15.153:0/1166732488 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa284100dc0 0x7fa28419e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:27.105 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.104+0000 7fa289200700 1 -- 172.21.15.153:0/1166732488 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa284100dc0 msgr2=0x7fa28419e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:27.105 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.104+0000 7fa289200700 1 --2- 172.21.15.153:0/1166732488 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa284100dc0 0x7fa28419e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:27.105 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.104+0000 7fa289200700 1 -- 172.21.15.153:0/1166732488 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2780095f0 con 0x7fa284101770 2024-03-21T10:26:27.105 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.105+0000 7fa289200700 1 --2- 172.21.15.153:0/1166732488 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa284101770 0x7fa28419ec30 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fa27000d900 tx=0x7fa27000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:27.106 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.105+0000 7fa282000700 1 -- 172.21.15.153:0/1166732488 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa270009900 con 0x7fa284101770 2024-03-21T10:26:27.106 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.105+0000 7fa282000700 1 -- 172.21.15.153:0/1166732488 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa270010460 con 0x7fa284101770 2024-03-21T10:26:27.106 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.105+0000 7fa28c600700 1 -- 172.21.15.153:0/1166732488 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa284198a50 con 0x7fa284101770 2024-03-21T10:26:27.106 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.105+0000 7fa282000700 1 -- 172.21.15.153:0/1166732488 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa27000f5f0 con 0x7fa284101770 2024-03-21T10:26:27.106 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.105+0000 7fa28c600700 1 -- 172.21.15.153:0/1166732488 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa284198fa0 con 0x7fa284101770 2024-03-21T10:26:27.106 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.106+0000 7fa28c600700 1 -- 172.21.15.153:0/1166732488 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa284104f40 con 0x7fa284101770 2024-03-21T10:26:27.107 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.107+0000 7fa282000700 1 -- 172.21.15.153:0/1166732488 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fa2700105d0 con 0x7fa284101770 2024-03-21T10:26:27.108 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.108+0000 7fa282000700 1 --2- 172.21.15.153:0/1166732488 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa26c06e460 0x7fa26c070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:27.108 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.108+0000 7fa282000700 1 -- 172.21.15.153:0/1166732488 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fa27008e1d0 con 0x7fa284101770 2024-03-21T10:26:27.108 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.108+0000 7fa289c00700 1 --2- 172.21.15.153:0/1166732488 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa26c06e460 0x7fa26c070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:27.109 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.109+0000 7fa289c00700 1 --2- 172.21.15.153:0/1166732488 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa26c06e460 0x7fa26c070920 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa278006010 tx=0x7fa27801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:27.113 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.113+0000 7fa282000700 1 -- 172.21.15.153:0/1166732488 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa27005ad20 con 0x7fa284101770 2024-03-21T10:26:27.329 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.329+0000 7fa28c600700 1 -- 172.21.15.153:0/1166732488 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fa284199280 con 0x7fa26c06e460 2024-03-21T10:26:27.361 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.361+0000 7fa282000700 1 -- 172.21.15.153:0/1166732488 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fa284199280 con 0x7fa26c06e460 2024-03-21T10:26:27.361 INFO:teuthology.orchestra.run.smithi153.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-21T10:26:27.361 INFO:teuthology.orchestra.run.smithi153.stdout:alertmanager.smithi153 smithi153 *:9093,9094 running (15m) 6m ago 28m 14.7M - 0.25.0 c8568f914cd2 9ed3f2200b5f 2024-03-21T10:26:27.362 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi153 smithi153 running (29m) 6m ago 29m 9424k - 18.2.1 5be31c24972a 5b61994396a6 2024-03-21T10:26:27.362 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi174 smithi174 running (27m) 6m ago 27m 9244k - 18.2.1 5be31c24972a 4f7a5fc3ac28 2024-03-21T10:26:27.362 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi153 smithi153 running (11m) 6m ago 29m 6656k - 19.0.0-2231-g2edfee68 237e43c7a65f 32c21104d5b0 2024-03-21T10:26:27.362 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi174 smithi174 running (11m) 6m ago 27m 6660k - 19.0.0-2231-g2edfee68 237e43c7a65f a04e48967ab6 2024-03-21T10:26:27.362 INFO:teuthology.orchestra.run.smithi153.stdout:grafana.smithi153 smithi153 *:3000 running (14m) 6m ago 28m 73.6M - 9.4.12 2bacad6d85d8 ee24026ab786 2024-03-21T10:26:27.362 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.iuyluj smithi153 running (6m) 6m ago 21m 10.9M - 19.0.0-2231-g2edfee68 237e43c7a65f 87aff8a5de70 2024-03-21T10:26:27.362 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.zipckr smithi153 running (7m) 6m ago 21m 11.8M - 19.0.0-2231-g2edfee68 237e43c7a65f ccc262bd5dd3 2024-03-21T10:26:27.362 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.nhozag smithi174 running (21m) 6m ago 21m 12.0M - 18.2.1 5be31c24972a 2fee4000308f 2024-03-21T10:26:27.362 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.yyimst smithi174 running (6m) 6m ago 21m 10.7M - 19.0.0-2231-g2edfee68 237e43c7a65f fc1962f9351e 2024-03-21T10:26:27.362 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi153.tluxni smithi153 *:8443,9283,8765 running (18m) 6m ago 30m 509M - 19.0.0-2231-g2edfee68 237e43c7a65f 7ffee0e55382 2024-03-21T10:26:27.362 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi174.qpcwat smithi174 *:8443,9283,8765 running (16m) 6m ago 27m 452M - 19.0.0-2231-g2edfee68 237e43c7a65f 748968f216ab 2024-03-21T10:26:27.362 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi153 smithi153 running (13m) 6m ago 31m 55.0M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f dd8038303dc3 2024-03-21T10:26:27.362 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi174 smithi174 running (12m) 6m ago 27m 44.2M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f 29d78a788f2c 2024-03-21T10:26:27.363 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi153 smithi153 *:9100 running (16m) 6m ago 29m 7889k - 1.5.0 0da6a335fe13 bb835edc0e0b 2024-03-21T10:26:27.363 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi174 smithi174 *:9100 running (16m) 6m ago 27m 8102k - 1.5.0 0da6a335fe13 7c4ecad790fa 2024-03-21T10:26:27.363 INFO:teuthology.orchestra.run.smithi153.stdout:osd.0 smithi153 running (10m) 6m ago 26m 116M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 507900fd4791 2024-03-21T10:26:27.363 INFO:teuthology.orchestra.run.smithi153.stdout:osd.1 smithi153 running (10m) 6m ago 25m 89.2M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 0d646452e9c1 2024-03-21T10:26:27.363 INFO:teuthology.orchestra.run.smithi153.stdout:osd.2 smithi153 running (9m) 6m ago 25m 87.4M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 1326f2afdfcd 2024-03-21T10:26:27.363 INFO:teuthology.orchestra.run.smithi153.stdout:osd.3 smithi174 running (9m) 6m ago 24m 83.6M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 70aafac71ad5 2024-03-21T10:26:27.363 INFO:teuthology.orchestra.run.smithi153.stdout:osd.4 smithi174 running (8m) 6m ago 24m 92.2M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 0d746ce6e79d 2024-03-21T10:26:27.363 INFO:teuthology.orchestra.run.smithi153.stdout:osd.5 smithi174 running (7m) 6m ago 23m 86.2M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 53e5f35b8cac 2024-03-21T10:26:27.363 INFO:teuthology.orchestra.run.smithi153.stdout:prometheus.smithi153 smithi153 *:9095 running (15m) 6m ago 28m 73.5M - 2.43.0 a07b618ecd1d 7d95561e50c2 2024-03-21T10:26:27.364 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.363+0000 7fa28c600700 1 -- 172.21.15.153:0/1166732488 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa26c06e460 msgr2=0x7fa26c070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:27.364 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.363+0000 7fa28c600700 1 --2- 172.21.15.153:0/1166732488 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa26c06e460 0x7fa26c070920 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa278006010 tx=0x7fa27801a040 comp rx=0 tx=0).stop 2024-03-21T10:26:27.364 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.363+0000 7fa28c600700 1 -- 172.21.15.153:0/1166732488 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa284101770 msgr2=0x7fa28419ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:27.364 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.364+0000 7fa28c600700 1 --2- 172.21.15.153:0/1166732488 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa284101770 0x7fa28419ec30 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fa27000d900 tx=0x7fa27000dcc0 comp rx=0 tx=0).stop 2024-03-21T10:26:27.364 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.364+0000 7fa28c600700 1 -- 172.21.15.153:0/1166732488 shutdown_connections 2024-03-21T10:26:27.364 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.364+0000 7fa28c600700 1 --2- 172.21.15.153:0/1166732488 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa284100dc0 0x7fa28419e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:27.364 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.364+0000 7fa28c600700 1 --2- 172.21.15.153:0/1166732488 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa26c06e460 0x7fa26c070920 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:27.364 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.364+0000 7fa28c600700 1 --2- 172.21.15.153:0/1166732488 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa284101770 0x7fa28419ec30 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:27.364 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.364+0000 7fa28c600700 1 -- 172.21.15.153:0/1166732488 >> 172.21.15.153:0/1166732488 conn(0x7fa2840fc650 msgr2=0x7fa2840fddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:27.364 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.364+0000 7fa28c600700 1 -- 172.21.15.153:0/1166732488 shutdown_connections 2024-03-21T10:26:27.365 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.364+0000 7fa28c600700 1 -- 172.21.15.153:0/1166732488 wait complete. 2024-03-21T10:26:27.518 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.518+0000 7febffa00700 1 -- 172.21.15.153:0/3343810340 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7febf8069020 msgr2=0x7febf8105790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:27.519 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.518+0000 7febffa00700 1 --2- 172.21.15.153:0/3343810340 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7febf8069020 0x7febf8105790 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7febe0009910 tx=0x7febe0009c20 comp rx=0 tx=0).stop 2024-03-21T10:26:27.519 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.518+0000 7febffa00700 1 -- 172.21.15.153:0/3343810340 shutdown_connections 2024-03-21T10:26:27.519 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.518+0000 7febffa00700 1 --2- 172.21.15.153:0/3343810340 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7febf8069020 0x7febf8105790 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:27.519 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.518+0000 7febffa00700 1 --2- 172.21.15.153:0/3343810340 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7febf8068670 0x7febf8068a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:27.519 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.518+0000 7febffa00700 1 -- 172.21.15.153:0/3343810340 >> 172.21.15.153:0/3343810340 conn(0x7febf80755a0 msgr2=0x7febf80759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:27.519 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.518+0000 7febffa00700 1 -- 172.21.15.153:0/3343810340 shutdown_connections 2024-03-21T10:26:27.519 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.518+0000 7febffa00700 1 -- 172.21.15.153:0/3343810340 wait complete. 2024-03-21T10:26:27.520 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.520+0000 7febffa00700 1 Processor -- start 2024-03-21T10:26:27.520 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.520+0000 7febffa00700 1 -- start start 2024-03-21T10:26:27.521 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.520+0000 7febffa00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7febf8068670 0x7febf819e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:27.521 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.520+0000 7febffa00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7febf8069020 0x7febf819ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:27.521 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.520+0000 7febffa00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febf819f320 con 0x7febf8068670 2024-03-21T10:26:27.521 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.520+0000 7febffa00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febf8198860 con 0x7febf8069020 2024-03-21T10:26:27.521 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.520+0000 7febfd000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7febf8068670 0x7febf819e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:27.521 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.521+0000 7febfd000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7febf8068670 0x7febf819e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:54870/0 (socket says 172.21.15.153:54870) 2024-03-21T10:26:27.521 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.521+0000 7febfd000700 1 -- 172.21.15.153:0/54860374 learned_addr learned my addr 172.21.15.153:0/54860374 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:26:27.521 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.521+0000 7febefe00700 1 --2- 172.21.15.153:0/54860374 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7febf8069020 0x7febf819ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:27.521 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.521+0000 7febfd000700 1 -- 172.21.15.153:0/54860374 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7febf8069020 msgr2=0x7febf819ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:27.521 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.521+0000 7febfd000700 1 --2- 172.21.15.153:0/54860374 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7febf8069020 0x7febf819ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:27.522 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.521+0000 7febfd000700 1 -- 172.21.15.153:0/54860374 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febe00095f0 con 0x7febf8068670 2024-03-21T10:26:27.522 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.521+0000 7febfd000700 1 --2- 172.21.15.153:0/54860374 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7febf8068670 0x7febf819e7e0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7febe800c930 tx=0x7febe800ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:27.522 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.522+0000 7febf6000700 1 -- 172.21.15.153:0/54860374 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7febe8007ad0 con 0x7febf8068670 2024-03-21T10:26:27.522 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.522+0000 7febf6000700 1 -- 172.21.15.153:0/54860374 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7febe8007c30 con 0x7febf8068670 2024-03-21T10:26:27.523 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.522+0000 7febffa00700 1 -- 172.21.15.153:0/54860374 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febf8198b40 con 0x7febf8068670 2024-03-21T10:26:27.523 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.522+0000 7febf6000700 1 -- 172.21.15.153:0/54860374 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7febe8018610 con 0x7febf8068670 2024-03-21T10:26:27.523 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.522+0000 7febffa00700 1 -- 172.21.15.153:0/54860374 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febf8199090 con 0x7febf8068670 2024-03-21T10:26:27.523 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.523+0000 7febffa00700 1 -- 172.21.15.153:0/54860374 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febf804ea90 con 0x7febf8068670 2024-03-21T10:26:27.523 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.524+0000 7febf6000700 1 -- 172.21.15.153:0/54860374 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7febe800f450 con 0x7febf8068670 2024-03-21T10:26:27.524 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.525+0000 7febf6000700 1 --2- 172.21.15.153:0/54860374 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7febdc06e460 0x7febdc070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:27.525 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.525+0000 7febf6000700 1 -- 172.21.15.153:0/54860374 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7febe808e710 con 0x7febf8068670 2024-03-21T10:26:27.525 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.525+0000 7febefe00700 1 --2- 172.21.15.153:0/54860374 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7febdc06e460 0x7febdc070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:27.525 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.526+0000 7febefe00700 1 --2- 172.21.15.153:0/54860374 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7febdc06e460 0x7febdc070920 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7febf80fe9d0 tx=0x7febe001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:27.529 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.530+0000 7febf6000700 1 -- 172.21.15.153:0/54860374 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7febe805b300 con 0x7febf8068670 2024-03-21T10:26:27.840 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.840+0000 7febffa00700 1 -- 172.21.15.153:0/54860374 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7febf8199310 con 0x7febf8068670 2024-03-21T10:26:27.841 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.841+0000 7febf6000700 1 -- 172.21.15.153:0/54860374 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+805 (secure 0 0 0) 0x7febe805ae90 con 0x7febf8068670 2024-03-21T10:26:27.841 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:26:27.841 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": { 2024-03-21T10:26:27.841 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:26:27.841 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:26:27.842 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": { 2024-03-21T10:26:27.842 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:26:27.842 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:26:27.842 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": { 2024-03-21T10:26:27.842 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 6 2024-03-21T10:26:27.842 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:26:27.842 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": { 2024-03-21T10:26:27.842 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-03-21T10:26:27.842 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 3 2024-03-21T10:26:27.842 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:26:27.842 INFO:teuthology.orchestra.run.smithi153.stdout: "overall": { 2024-03-21T10:26:27.842 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-03-21T10:26:27.842 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 13 2024-03-21T10:26:27.843 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-03-21T10:26:27.843 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:26:27.844 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.844+0000 7febffa00700 1 -- 172.21.15.153:0/54860374 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7febdc06e460 msgr2=0x7febdc070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:27.844 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.844+0000 7febffa00700 1 --2- 172.21.15.153:0/54860374 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7febdc06e460 0x7febdc070920 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7febf80fe9d0 tx=0x7febe001a040 comp rx=0 tx=0).stop 2024-03-21T10:26:27.844 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.844+0000 7febffa00700 1 -- 172.21.15.153:0/54860374 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7febf8068670 msgr2=0x7febf819e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:27.844 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.844+0000 7febffa00700 1 --2- 172.21.15.153:0/54860374 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7febf8068670 0x7febf819e7e0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7febe800c930 tx=0x7febe800ccf0 comp rx=0 tx=0).stop 2024-03-21T10:26:27.844 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.844+0000 7febffa00700 1 -- 172.21.15.153:0/54860374 shutdown_connections 2024-03-21T10:26:27.844 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.844+0000 7febffa00700 1 --2- 172.21.15.153:0/54860374 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7febdc06e460 0x7febdc070920 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:27.844 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.844+0000 7febffa00700 1 --2- 172.21.15.153:0/54860374 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7febf8068670 0x7febf819e7e0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:27.844 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.844+0000 7febffa00700 1 --2- 172.21.15.153:0/54860374 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7febf8069020 0x7febf819ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:27.844 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.844+0000 7febffa00700 1 -- 172.21.15.153:0/54860374 >> 172.21.15.153:0/54860374 conn(0x7febf80755a0 msgr2=0x7febf8102760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:27.844 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.844+0000 7febffa00700 1 -- 172.21.15.153:0/54860374 shutdown_connections 2024-03-21T10:26:27.845 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.844+0000 7febffa00700 1 -- 172.21.15.153:0/54860374 wait complete. 2024-03-21T10:26:27.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:26:27 smithi174 ceph-mon[88004]: from='client.34814 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:26:27.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:26:27 smithi174 ceph-mon[88004]: from='client.34818 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:26:27.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:26:27 smithi174 ceph-mon[88004]: from='client.34822 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:26:27.994 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.994+0000 7f0483200700 1 -- 172.21.15.153:0/3260338106 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f047c10d280 msgr2=0x7f047c10d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:27.994 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.994+0000 7f0483200700 1 --2- 172.21.15.153:0/3260338106 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f047c10d280 0x7f047c10d660 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f046c009910 tx=0x7f046c009c20 comp rx=0 tx=0).stop 2024-03-21T10:26:27.994 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.994+0000 7f0483200700 1 -- 172.21.15.153:0/3260338106 shutdown_connections 2024-03-21T10:26:27.994 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.994+0000 7f0483200700 1 --2- 172.21.15.153:0/3260338106 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f047c1071d0 0x7f047c107650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:27.994 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.994+0000 7f0483200700 1 --2- 172.21.15.153:0/3260338106 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f047c10d280 0x7f047c10d660 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:27.994 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.994+0000 7f0483200700 1 -- 172.21.15.153:0/3260338106 >> 172.21.15.153:0/3260338106 conn(0x7f047c0769e0 msgr2=0x7f047c078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:27.995 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:26:27 smithi153 ceph-mon[113733]: from='client.34814 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:26:27.995 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:26:27 smithi153 ceph-mon[113733]: from='client.34818 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:26:27.995 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:26:27 smithi153 ceph-mon[113733]: from='client.34822 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:26:27.996 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.994+0000 7f0483200700 1 -- 172.21.15.153:0/3260338106 shutdown_connections 2024-03-21T10:26:27.996 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.994+0000 7f0483200700 1 -- 172.21.15.153:0/3260338106 wait complete. 2024-03-21T10:26:27.996 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.995+0000 7f0483200700 1 Processor -- start 2024-03-21T10:26:27.996 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.995+0000 7f0483200700 1 -- start start 2024-03-21T10:26:27.996 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.996+0000 7f0483200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f047c1071d0 0x7f047c19ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:27.996 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.996+0000 7f0483200700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f047c10d280 0x7f047c19d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:27.996 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.996+0000 7f0483200700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f047c19d8a0 con 0x7f047c1071d0 2024-03-21T10:26:27.996 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.996+0000 7f0483200700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f047c1a05d0 con 0x7f047c10d280 2024-03-21T10:26:27.997 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.996+0000 7f047be00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f047c1071d0 0x7f047c19ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:27.997 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.996+0000 7f047be00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f047c1071d0 0x7f047c19ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:54892/0 (socket says 172.21.15.153:54892) 2024-03-21T10:26:27.997 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.996+0000 7f047be00700 1 -- 172.21.15.153:0/3826941344 learned_addr learned my addr 172.21.15.153:0/3826941344 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:26:27.998 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.996+0000 7f047b400700 1 --2- 172.21.15.153:0/3826941344 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f047c10d280 0x7f047c19d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:27.998 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.997+0000 7f047be00700 1 -- 172.21.15.153:0/3826941344 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f047c10d280 msgr2=0x7f047c19d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:27.998 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.997+0000 7f047be00700 1 --2- 172.21.15.153:0/3826941344 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f047c10d280 0x7f047c19d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:27.998 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.997+0000 7f047be00700 1 -- 172.21.15.153:0/3826941344 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f046c0095f0 con 0x7f047c1071d0 2024-03-21T10:26:27.998 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.997+0000 7f047b400700 1 --2- 172.21.15.153:0/3826941344 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f047c10d280 0x7f047c19d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:26:27.998 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.997+0000 7f047be00700 1 --2- 172.21.15.153:0/3826941344 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f047c1071d0 0x7f047c19ccd0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f046c0092e0 tx=0x7f046c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:27.998 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.997+0000 7f0478c00700 1 -- 172.21.15.153:0/3826941344 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f046c01d070 con 0x7f047c1071d0 2024-03-21T10:26:27.998 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.997+0000 7f0478c00700 1 -- 172.21.15.153:0/3826941344 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f046c005e00 con 0x7f047c1071d0 2024-03-21T10:26:27.998 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.997+0000 7f0483200700 1 -- 172.21.15.153:0/3826941344 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f047c1a0850 con 0x7f047c1071d0 2024-03-21T10:26:27.998 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.997+0000 7f0478c00700 1 -- 172.21.15.153:0/3826941344 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f046c017400 con 0x7f047c1071d0 2024-03-21T10:26:27.999 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.997+0000 7f0483200700 1 -- 172.21.15.153:0/3826941344 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f047c1a0dc0 con 0x7f047c1071d0 2024-03-21T10:26:27.999 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:27.998+0000 7f0483200700 1 -- 172.21.15.153:0/3826941344 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f047c04ea90 con 0x7f047c1071d0 2024-03-21T10:26:27.999 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:28.000+0000 7f0478c00700 1 -- 172.21.15.153:0/3826941344 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f046c00f460 con 0x7f047c1071d0 2024-03-21T10:26:28.001 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:28.001+0000 7f0478c00700 1 --2- 172.21.15.153:0/3826941344 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f046006e460 0x7f0460070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:28.001 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:28.002+0000 7f0478c00700 1 -- 172.21.15.153:0/3826941344 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f046c08fa80 con 0x7f047c1071d0 2024-03-21T10:26:28.001 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:28.002+0000 7f047b400700 1 --2- 172.21.15.153:0/3826941344 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f046006e460 0x7f0460070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:28.002 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:28.002+0000 7f047b400700 1 --2- 172.21.15.153:0/3826941344 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f046006e460 0x7f0460070920 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0468005fd0 tx=0x7f0468005ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:28.006 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:28.006+0000 7f0478c00700 1 -- 172.21.15.153:0/3826941344 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f046c058aa0 con 0x7f047c1071d0 2024-03-21T10:26:28.263 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:28.263+0000 7f0483200700 1 -- 172.21.15.153:0/3826941344 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f047c19dff0 con 0x7f047c1071d0 2024-03-21T10:26:28.264 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:28.264+0000 7f0478c00700 1 -- 172.21.15.153:0/3826941344 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1876 (secure 0 0 0) 0x7f046c05c160 con 0x7f047c1071d0 2024-03-21T10:26:28.264 INFO:teuthology.orchestra.run.smithi153.stdout:e30 2024-03-21T10:26:28.264 INFO:teuthology.orchestra.run.smithi153.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-21T10:26:28.264 INFO:teuthology.orchestra.run.smithi153.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:26:28.265 INFO:teuthology.orchestra.run.smithi153.stdout:legacy client fscid: 1 2024-03-21T10:26:28.265 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:26:28.265 INFO:teuthology.orchestra.run.smithi153.stdout:Filesystem 'cephfs' (1) 2024-03-21T10:26:28.265 INFO:teuthology.orchestra.run.smithi153.stdout:fs_name cephfs 2024-03-21T10:26:28.265 INFO:teuthology.orchestra.run.smithi153.stdout:epoch 23 2024-03-21T10:26:28.265 INFO:teuthology.orchestra.run.smithi153.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-21T10:26:28.265 INFO:teuthology.orchestra.run.smithi153.stdout:created 2024-03-21T10:04:27.086576+0000 2024-03-21T10:26:28.265 INFO:teuthology.orchestra.run.smithi153.stdout:modified 2024-03-21T10:19:05.022092+0000 2024-03-21T10:26:28.265 INFO:teuthology.orchestra.run.smithi153.stdout:tableserver 0 2024-03-21T10:26:28.265 INFO:teuthology.orchestra.run.smithi153.stdout:root 0 2024-03-21T10:26:28.265 INFO:teuthology.orchestra.run.smithi153.stdout:session_timeout 60 2024-03-21T10:26:28.265 INFO:teuthology.orchestra.run.smithi153.stdout:session_autoclose 300 2024-03-21T10:26:28.265 INFO:teuthology.orchestra.run.smithi153.stdout:max_file_size 1099511627776 2024-03-21T10:26:28.265 INFO:teuthology.orchestra.run.smithi153.stdout:max_xattr_size 65536 2024-03-21T10:26:28.266 INFO:teuthology.orchestra.run.smithi153.stdout:required_client_features {} 2024-03-21T10:26:28.266 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure 0 2024-03-21T10:26:28.266 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure_osd_epoch 94 2024-03-21T10:26:28.266 INFO:teuthology.orchestra.run.smithi153.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:26:28.266 INFO:teuthology.orchestra.run.smithi153.stdout:max_mds 2 2024-03-21T10:26:28.266 INFO:teuthology.orchestra.run.smithi153.stdout:in 0,1 2024-03-21T10:26:28.266 INFO:teuthology.orchestra.run.smithi153.stdout:up {} 2024-03-21T10:26:28.266 INFO:teuthology.orchestra.run.smithi153.stdout:failed 0,1 2024-03-21T10:26:28.266 INFO:teuthology.orchestra.run.smithi153.stdout:damaged 2024-03-21T10:26:28.266 INFO:teuthology.orchestra.run.smithi153.stdout:stopped 2024-03-21T10:26:28.266 INFO:teuthology.orchestra.run.smithi153.stdout:data_pools [3] 2024-03-21T10:26:28.266 INFO:teuthology.orchestra.run.smithi153.stdout:metadata_pool 2 2024-03-21T10:26:28.266 INFO:teuthology.orchestra.run.smithi153.stdout:inline_data enabled 2024-03-21T10:26:28.267 INFO:teuthology.orchestra.run.smithi153.stdout:balancer 2024-03-21T10:26:28.267 INFO:teuthology.orchestra.run.smithi153.stdout:bal_rank_mask -1 2024-03-21T10:26:28.267 INFO:teuthology.orchestra.run.smithi153.stdout:standby_count_wanted 1 2024-03-21T10:26:28.267 INFO:teuthology.orchestra.run.smithi153.stdout:qdb_cluster leader: 0 members: 2024-03-21T10:26:28.267 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:26:28.267 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:26:28.267 INFO:teuthology.orchestra.run.smithi153.stdout:Standby daemons: 2024-03-21T10:26:28.267 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:26:28.267 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.zipckr{-1:34442} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6826/1103846295,v1:172.21.15.153:6827/1103846295] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:26:28.267 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.iuyluj{-1:34474} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6828/1738630832,v1:172.21.15.153:6829/1738630832] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:26:28.267 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.nhozag{-1:44269} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.174:6826/3967223227,v1:172.21.15.174:6827/3967223227] compat {c=[1],r=[1],i=[7ff]}] 2024-03-21T10:26:28.267 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.yyimst{-1:44313} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.174:6824/4127786797,v1:172.21.15.174:6825/4127786797] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:26:28.268 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:28.267+0000 7f0483200700 1 -- 172.21.15.153:0/3826941344 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f046006e460 msgr2=0x7f0460070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:28.268 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:28.267+0000 7f0483200700 1 --2- 172.21.15.153:0/3826941344 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f046006e460 0x7f0460070920 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0468005fd0 tx=0x7f0468005ee0 comp rx=0 tx=0).stop 2024-03-21T10:26:28.268 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:28.267+0000 7f0483200700 1 -- 172.21.15.153:0/3826941344 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f047c1071d0 msgr2=0x7f047c19ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:28.268 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:28.267+0000 7f0483200700 1 --2- 172.21.15.153:0/3826941344 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f047c1071d0 0x7f047c19ccd0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f046c0092e0 tx=0x7f046c00be20 comp rx=0 tx=0).stop 2024-03-21T10:26:28.268 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:28.267+0000 7f0483200700 1 -- 172.21.15.153:0/3826941344 shutdown_connections 2024-03-21T10:26:28.268 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:28.267+0000 7f0483200700 1 --2- 172.21.15.153:0/3826941344 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f046006e460 0x7f0460070920 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:28.268 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:28.267+0000 7f0483200700 1 --2- 172.21.15.153:0/3826941344 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f047c1071d0 0x7f047c19ccd0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:28.268 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:28.267+0000 7f0483200700 1 --2- 172.21.15.153:0/3826941344 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f047c10d280 0x7f047c19d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:28.269 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:28.267+0000 7f0483200700 1 -- 172.21.15.153:0/3826941344 >> 172.21.15.153:0/3826941344 conn(0x7f047c0769e0 msgr2=0x7f047c078320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:28.269 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:28.267+0000 7f0483200700 1 -- 172.21.15.153:0/3826941344 shutdown_connections 2024-03-21T10:26:28.269 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:28.267+0000 7f0483200700 1 -- 172.21.15.153:0/3826941344 wait complete. 2024-03-21T10:26:28.269 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 30 2024-03-21T10:26:28.415 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:28.415+0000 7fd536200700 1 -- 172.21.15.153:0/3015615525 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd53010d280 msgr2=0x7fd53010d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:28.415 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:28.415+0000 7fd536200700 1 --2- 172.21.15.153:0/3015615525 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd53010d280 0x7fd53010d660 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fd520009910 tx=0x7fd520009c20 comp rx=0 tx=0).stop 2024-03-21T10:26:28.415 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:28.415+0000 7fd536200700 1 -- 172.21.15.153:0/3015615525 shutdown_connections 2024-03-21T10:26:28.415 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:28.415+0000 7fd536200700 1 --2- 172.21.15.153:0/3015615525 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd5301071d0 0x7fd530107650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:28.415 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:28.415+0000 7fd536200700 1 --2- 172.21.15.153:0/3015615525 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd53010d280 0x7fd53010d660 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:28.416 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:28.415+0000 7fd536200700 1 -- 172.21.15.153:0/3015615525 >> 172.21.15.153:0/3015615525 conn(0x7fd5300769e0 msgr2=0x7fd530078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:28.416 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:28.415+0000 7fd536200700 1 -- 172.21.15.153:0/3015615525 shutdown_connections 2024-03-21T10:26:28.416 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:28.415+0000 7fd536200700 1 -- 172.21.15.153:0/3015615525 wait complete. 2024-03-21T10:26:28.417 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:28.416+0000 7fd536200700 1 Processor -- start 2024-03-21T10:26:28.417 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:28.416+0000 7fd536200700 1 -- start start 2024-03-21T10:26:28.417 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:28.417+0000 7fd536200700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd5301071d0 0x7fd53019ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:28.417 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:28.417+0000 7fd536200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd53010d280 0x7fd53019d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:28.417 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:28.417+0000 7fd536200700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd53019d8a0 con 0x7fd53010d280 2024-03-21T10:26:28.418 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:28.417+0000 7fd536200700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5301a05d0 con 0x7fd5301071d0 2024-03-21T10:26:28.418 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:28.417+0000 7fd52ea00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd53010d280 0x7fd53019d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:28.418 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:28.417+0000 7fd52ea00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd53010d280 0x7fd53019d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:54908/0 (socket says 172.21.15.153:54908) 2024-03-21T10:26:28.418 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:28.417+0000 7fd52ea00700 1 -- 172.21.15.153:0/2779453995 learned_addr learned my addr 172.21.15.153:0/2779453995 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:26:28.418 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:28.417+0000 7fd52f400700 1 --2- 172.21.15.153:0/2779453995 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd5301071d0 0x7fd53019ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:28.418 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:28.417+0000 7fd52ea00700 1 -- 172.21.15.153:0/2779453995 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd5301071d0 msgr2=0x7fd53019ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:28.418 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:28.417+0000 7fd52ea00700 1 --2- 172.21.15.153:0/2779453995 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd5301071d0 0x7fd53019ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:28.418 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:28.417+0000 7fd52ea00700 1 -- 172.21.15.153:0/2779453995 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5200095f0 con 0x7fd53010d280 2024-03-21T10:26:28.418 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:28.418+0000 7fd52ea00700 1 --2- 172.21.15.153:0/2779453995 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd53010d280 0x7fd53019d210 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fd51c00e920 tx=0x7fd51c00ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:28.419 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:28.418+0000 7fd527e00700 1 -- 172.21.15.153:0/2779453995 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd51c009790 con 0x7fd53010d280 2024-03-21T10:26:28.419 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:28.418+0000 7fd527e00700 1 -- 172.21.15.153:0/2779453995 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd51c00cba0 con 0x7fd53010d280 2024-03-21T10:26:28.419 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:28.418+0000 7fd536200700 1 -- 172.21.15.153:0/2779453995 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5301a08b0 con 0x7fd53010d280 2024-03-21T10:26:28.419 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:28.418+0000 7fd527e00700 1 -- 172.21.15.153:0/2779453995 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd51c018650 con 0x7fd53010d280 2024-03-21T10:26:28.419 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:28.418+0000 7fd536200700 1 -- 172.21.15.153:0/2779453995 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5301a0e80 con 0x7fd53010d280 2024-03-21T10:26:28.419 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:28.419+0000 7fd536200700 1 -- 172.21.15.153:0/2779453995 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd53004ea90 con 0x7fd53010d280 2024-03-21T10:26:28.420 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:28.420+0000 7fd527e00700 1 -- 172.21.15.153:0/2779453995 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fd51c00cd10 con 0x7fd53010d280 2024-03-21T10:26:28.421 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:28.421+0000 7fd527e00700 1 --2- 172.21.15.153:0/2779453995 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd51406e460 0x7fd514070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:28.421 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:28.421+0000 7fd527e00700 1 -- 172.21.15.153:0/2779453995 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fd51c014070 con 0x7fd53010d280 2024-03-21T10:26:28.421 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:28.421+0000 7fd52f400700 1 --2- 172.21.15.153:0/2779453995 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd51406e460 0x7fd514070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:28.421 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:28.422+0000 7fd52f400700 1 --2- 172.21.15.153:0/2779453995 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd51406e460 0x7fd514070920 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd520009910 tx=0x7fd520009f70 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:28.425 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:28.425+0000 7fd527e00700 1 -- 172.21.15.153:0/2779453995 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd51c053740 con 0x7fd53010d280 2024-03-21T10:26:28.652 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:28.652+0000 7fd536200700 1 -- 172.21.15.153:0/2779453995 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd5301a1260 con 0x7fd51406e460 2024-03-21T10:26:28.654 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:28.654+0000 7fd527e00700 1 -- 172.21.15.153:0/2779453995 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fd5301a1260 con 0x7fd51406e460 2024-03-21T10:26:28.655 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:26:28.655 INFO:teuthology.orchestra.run.smithi153.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232", 2024-03-21T10:26:28.655 INFO:teuthology.orchestra.run.smithi153.stdout: "in_progress": true, 2024-03-21T10:26:28.655 INFO:teuthology.orchestra.run.smithi153.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-21T10:26:28.655 INFO:teuthology.orchestra.run.smithi153.stdout: "services_complete": [ 2024-03-21T10:26:28.655 INFO:teuthology.orchestra.run.smithi153.stdout: "crash", 2024-03-21T10:26:28.655 INFO:teuthology.orchestra.run.smithi153.stdout: "mon", 2024-03-21T10:26:28.655 INFO:teuthology.orchestra.run.smithi153.stdout: "osd", 2024-03-21T10:26:28.655 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr" 2024-03-21T10:26:28.655 INFO:teuthology.orchestra.run.smithi153.stdout: ], 2024-03-21T10:26:28.655 INFO:teuthology.orchestra.run.smithi153.stdout: "progress": "15/23 daemons upgraded", 2024-03-21T10:26:28.655 INFO:teuthology.orchestra.run.smithi153.stdout: "message": "Currently upgrading mds daemons", 2024-03-21T10:26:28.655 INFO:teuthology.orchestra.run.smithi153.stdout: "is_paused": false 2024-03-21T10:26:28.656 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:26:28.657 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:28.657+0000 7fd536200700 1 -- 172.21.15.153:0/2779453995 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd51406e460 msgr2=0x7fd514070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:28.657 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:28.657+0000 7fd536200700 1 --2- 172.21.15.153:0/2779453995 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd51406e460 0x7fd514070920 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd520009910 tx=0x7fd520009f70 comp rx=0 tx=0).stop 2024-03-21T10:26:28.657 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:28.657+0000 7fd536200700 1 -- 172.21.15.153:0/2779453995 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd53010d280 msgr2=0x7fd53019d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:28.658 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:28.657+0000 7fd536200700 1 --2- 172.21.15.153:0/2779453995 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd53010d280 0x7fd53019d210 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fd51c00e920 tx=0x7fd51c00ece0 comp rx=0 tx=0).stop 2024-03-21T10:26:28.658 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:28.657+0000 7fd536200700 1 -- 172.21.15.153:0/2779453995 shutdown_connections 2024-03-21T10:26:28.658 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:28.657+0000 7fd536200700 1 --2- 172.21.15.153:0/2779453995 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd5301071d0 0x7fd53019ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:28.658 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:28.658+0000 7fd536200700 1 --2- 172.21.15.153:0/2779453995 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd51406e460 0x7fd514070920 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:28.658 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:28.658+0000 7fd536200700 1 --2- 172.21.15.153:0/2779453995 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd53010d280 0x7fd53019d210 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:28.658 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:28.658+0000 7fd536200700 1 -- 172.21.15.153:0/2779453995 >> 172.21.15.153:0/2779453995 conn(0x7fd5300769e0 msgr2=0x7fd530078320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:28.658 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:28.658+0000 7fd536200700 1 -- 172.21.15.153:0/2779453995 shutdown_connections 2024-03-21T10:26:28.658 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:28.658+0000 7fd536200700 1 -- 172.21.15.153:0/2779453995 wait complete. 2024-03-21T10:26:28.814 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:28.813+0000 7f0d1d800700 1 -- 172.21.15.153:0/2326316591 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0d18100dc0 msgr2=0x7f0d181011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:28.814 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:28.813+0000 7f0d1d800700 1 --2- 172.21.15.153:0/2326316591 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0d18100dc0 0x7f0d181011a0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f0d08009910 tx=0x7f0d08009c20 comp rx=0 tx=0).stop 2024-03-21T10:26:28.814 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:28.813+0000 7f0d1d800700 1 -- 172.21.15.153:0/2326316591 shutdown_connections 2024-03-21T10:26:28.814 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:28.813+0000 7f0d1d800700 1 --2- 172.21.15.153:0/2326316591 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d18101770 0x7f0d181057c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:28.814 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:28.813+0000 7f0d1d800700 1 --2- 172.21.15.153:0/2326316591 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0d18100dc0 0x7f0d181011a0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:28.814 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:28.814+0000 7f0d1d800700 1 -- 172.21.15.153:0/2326316591 >> 172.21.15.153:0/2326316591 conn(0x7f0d180fc650 msgr2=0x7f0d180fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:28.814 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:28.814+0000 7f0d1d800700 1 -- 172.21.15.153:0/2326316591 shutdown_connections 2024-03-21T10:26:28.814 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:28.814+0000 7f0d1d800700 1 -- 172.21.15.153:0/2326316591 wait complete. 2024-03-21T10:26:28.816 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:28.815+0000 7f0d1d800700 1 Processor -- start 2024-03-21T10:26:28.816 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:28.815+0000 7f0d1d800700 1 -- start start 2024-03-21T10:26:28.816 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:28.815+0000 7f0d1d800700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d18100dc0 0x7f0d18198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:28.816 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:28.815+0000 7f0d1d800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0d18101770 0x7f0d18198eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:28.816 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:28.815+0000 7f0d1d800700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d18199540 con 0x7f0d18101770 2024-03-21T10:26:28.816 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:28.815+0000 7f0d1d800700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d1819d350 con 0x7f0d18100dc0 2024-03-21T10:26:28.817 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:28.816+0000 7f0d16000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0d18101770 0x7f0d18198eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:28.817 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:28.816+0000 7f0d16000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0d18101770 0x7f0d18198eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:54924/0 (socket says 172.21.15.153:54924) 2024-03-21T10:26:28.817 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:28.816+0000 7f0d16000700 1 -- 172.21.15.153:0/2870244484 learned_addr learned my addr 172.21.15.153:0/2870244484 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:26:28.817 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:28.816+0000 7f0d16a00700 1 --2- 172.21.15.153:0/2870244484 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d18100dc0 0x7f0d18198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:28.817 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:28.816+0000 7f0d16000700 1 -- 172.21.15.153:0/2870244484 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d18100dc0 msgr2=0x7f0d18198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:28.817 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:28.816+0000 7f0d16000700 1 --2- 172.21.15.153:0/2870244484 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d18100dc0 0x7f0d18198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:28.817 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:28.816+0000 7f0d16000700 1 -- 172.21.15.153:0/2870244484 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d080095f0 con 0x7f0d18101770 2024-03-21T10:26:28.817 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:28.816+0000 7f0d16000700 1 --2- 172.21.15.153:0/2870244484 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0d18101770 0x7f0d18198eb0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f0d0400d8d0 tx=0x7f0d0400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:28.818 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:28.817+0000 7f0d0f400700 1 -- 172.21.15.153:0/2870244484 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0d04004a00 con 0x7f0d18101770 2024-03-21T10:26:28.818 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:28.817+0000 7f0d0f400700 1 -- 172.21.15.153:0/2870244484 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0d040054e0 con 0x7f0d18101770 2024-03-21T10:26:28.818 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:28.817+0000 7f0d0f400700 1 -- 172.21.15.153:0/2870244484 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0d04009d40 con 0x7f0d18101770 2024-03-21T10:26:28.818 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:28.817+0000 7f0d1d800700 1 -- 172.21.15.153:0/2870244484 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d1819d630 con 0x7f0d18101770 2024-03-21T10:26:28.818 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:28.817+0000 7f0d1d800700 1 -- 172.21.15.153:0/2870244484 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d1819db80 con 0x7f0d18101770 2024-03-21T10:26:28.818 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:28.818+0000 7f0d1d800700 1 -- 172.21.15.153:0/2870244484 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d18104f40 con 0x7f0d18101770 2024-03-21T10:26:28.819 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:28.819+0000 7f0d0f400700 1 -- 172.21.15.153:0/2870244484 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f0d04005020 con 0x7f0d18101770 2024-03-21T10:26:28.821 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:28.821+0000 7f0d0f400700 1 --2- 172.21.15.153:0/2870244484 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f0cfc06e460 0x7f0cfc070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:28.821 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:28.821+0000 7f0d0f400700 1 -- 172.21.15.153:0/2870244484 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f0d0408de50 con 0x7f0d18101770 2024-03-21T10:26:28.821 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:28.821+0000 7f0d16a00700 1 --2- 172.21.15.153:0/2870244484 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f0cfc06e460 0x7f0cfc070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:28.821 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:28.822+0000 7f0d16a00700 1 --2- 172.21.15.153:0/2870244484 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f0cfc06e460 0x7f0cfc070920 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0d08006010 tx=0x7f0d0801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:28.825 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:28.825+0000 7f0d0f400700 1 -- 172.21.15.153:0/2870244484 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0d0405aa20 con 0x7f0d18101770 2024-03-21T10:26:28.950 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:26:28 smithi174 ceph-mon[88004]: pgmap v438: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:26:28.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:26:28 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/54860374' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:26:28.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:26:28 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/3826941344' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:26:29.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:26:28 smithi153 ceph-mon[113733]: pgmap v438: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:26:29.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:26:28 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/54860374' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:26:29.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:26:28 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/3826941344' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:26:29.144 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:29.144+0000 7f0d1d800700 1 -- 172.21.15.153:0/2870244484 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f0d1804ea90 con 0x7f0d18101770 2024-03-21T10:26:29.145 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:29.145+0000 7f0d0f400700 1 -- 172.21.15.153:0/2870244484 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f0d040201c0 con 0x7f0d18101770 2024-03-21T10:26:29.145 INFO:teuthology.orchestra.run.smithi153.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-03-21T10:26:29.145 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-21T10:26:29.145 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is degraded 2024-03-21T10:26:29.145 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-21T10:26:29.145 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-21T10:26:29.145 INFO:teuthology.orchestra.run.smithi153.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-21T10:26:29.145 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-21T10:26:29.148 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:29.147+0000 7f0d1d800700 1 -- 172.21.15.153:0/2870244484 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f0cfc06e460 msgr2=0x7f0cfc070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:29.148 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:29.147+0000 7f0d1d800700 1 --2- 172.21.15.153:0/2870244484 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f0cfc06e460 0x7f0cfc070920 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0d08006010 tx=0x7f0d0801a040 comp rx=0 tx=0).stop 2024-03-21T10:26:29.148 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:29.147+0000 7f0d1d800700 1 -- 172.21.15.153:0/2870244484 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0d18101770 msgr2=0x7f0d18198eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:29.149 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:29.147+0000 7f0d1d800700 1 --2- 172.21.15.153:0/2870244484 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0d18101770 0x7f0d18198eb0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f0d0400d8d0 tx=0x7f0d0400dc90 comp rx=0 tx=0).stop 2024-03-21T10:26:29.149 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:29.148+0000 7f0d1d800700 1 -- 172.21.15.153:0/2870244484 shutdown_connections 2024-03-21T10:26:29.149 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:29.148+0000 7f0d1d800700 1 --2- 172.21.15.153:0/2870244484 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d18100dc0 0x7f0d18198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:29.149 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:29.148+0000 7f0d1d800700 1 --2- 172.21.15.153:0/2870244484 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f0cfc06e460 0x7f0cfc070920 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:29.149 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:29.148+0000 7f0d1d800700 1 --2- 172.21.15.153:0/2870244484 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0d18101770 0x7f0d18198eb0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:29.149 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:29.148+0000 7f0d1d800700 1 -- 172.21.15.153:0/2870244484 >> 172.21.15.153:0/2870244484 conn(0x7f0d180fc650 msgr2=0x7f0d180fddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:29.149 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:29.148+0000 7f0d1d800700 1 -- 172.21.15.153:0/2870244484 shutdown_connections 2024-03-21T10:26:29.149 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:29.148+0000 7f0d1d800700 1 -- 172.21.15.153:0/2870244484 wait complete. 2024-03-21T10:26:29.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:26:29 smithi174 ceph-mon[88004]: from='client.34834 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:26:29.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:26:29 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/2870244484' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:26:30.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:26:29 smithi153 ceph-mon[113733]: from='client.34834 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:26:30.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:26:29 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/2870244484' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:26:30.950 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:26:30 smithi174 ceph-mon[88004]: pgmap v439: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:26:31.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:26:30 smithi153 ceph-mon[113733]: pgmap v439: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:26:32.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:26:31 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:26:32.200 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:26:31 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:26:33.055 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:26:32 smithi153 ceph-mon[113733]: pgmap v440: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:26:33.200 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:26:32 smithi174 ceph-mon[88004]: pgmap v440: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:26:35.003 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:26:34 smithi174 ceph-mon[88004]: pgmap v441: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:26:35.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:26:34 smithi153 ceph-mon[113733]: pgmap v441: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:26:37.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:26:36 smithi153 ceph-mon[113733]: pgmap v442: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:26:37.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:26:36 smithi174 ceph-mon[88004]: pgmap v442: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:26:39.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:26:38 smithi153 ceph-mon[113733]: pgmap v443: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:26:39.200 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:26:38 smithi174 ceph-mon[88004]: pgmap v443: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:26:41.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:26:40 smithi153 ceph-mon[113733]: pgmap v444: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:26:41.200 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:26:40 smithi174 ceph-mon[88004]: pgmap v444: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:26:43.013 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:26:42 smithi153 ceph-mon[113733]: pgmap v445: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:26:43.200 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:26:42 smithi174 ceph-mon[88004]: pgmap v445: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:26:45.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:26:44 smithi153 ceph-mon[113733]: pgmap v446: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:26:45.200 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:26:44 smithi174 ceph-mon[88004]: pgmap v446: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:26:46.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:26:46 smithi174 ceph-mon[88004]: pgmap v447: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:26:46.951 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:26:46 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:26:47.055 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:26:46 smithi153 ceph-mon[113733]: pgmap v447: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:26:47.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:26:46 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:26:49.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:26:48 smithi153 ceph-mon[113733]: pgmap v448: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:26:49.200 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:26:48 smithi174 ceph-mon[88004]: pgmap v448: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:26:51.055 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:26:50 smithi153 ceph-mon[113733]: pgmap v449: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:26:51.200 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:26:50 smithi174 ceph-mon[88004]: pgmap v449: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:26:53.059 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:26:52 smithi153 ceph-mon[113733]: pgmap v450: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:26:53.200 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:26:52 smithi174 ceph-mon[88004]: pgmap v450: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:26:55.055 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:26:54 smithi153 ceph-mon[113733]: pgmap v451: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:26:55.200 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:26:54 smithi174 ceph-mon[88004]: pgmap v451: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:26:57.055 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:26:56 smithi153 ceph-mon[113733]: pgmap v452: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:26:57.200 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:26:56 smithi174 ceph-mon[88004]: pgmap v452: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:26:59.055 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:26:58 smithi153 ceph-mon[113733]: pgmap v453: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:26:59.200 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:26:58 smithi174 ceph-mon[88004]: pgmap v453: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:26:59.323 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:59.322+0000 7fd152e00700 1 -- 172.21.15.153:0/3618110816 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd14c100dc0 msgr2=0x7fd14c1011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:59.323 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:59.322+0000 7fd152e00700 1 --2- 172.21.15.153:0/3618110816 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd14c100dc0 0x7fd14c1011a0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fd13c009910 tx=0x7fd13c009c20 comp rx=0 tx=0).stop 2024-03-21T10:26:59.324 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:59.322+0000 7fd152e00700 1 -- 172.21.15.153:0/3618110816 shutdown_connections 2024-03-21T10:26:59.324 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:59.322+0000 7fd152e00700 1 --2- 172.21.15.153:0/3618110816 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd14c101770 0x7fd14c1057c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:59.324 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:59.322+0000 7fd152e00700 1 --2- 172.21.15.153:0/3618110816 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd14c100dc0 0x7fd14c1011a0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:59.324 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:59.322+0000 7fd152e00700 1 -- 172.21.15.153:0/3618110816 >> 172.21.15.153:0/3618110816 conn(0x7fd14c0fc650 msgr2=0x7fd14c0fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:59.324 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:59.322+0000 7fd152e00700 1 -- 172.21.15.153:0/3618110816 shutdown_connections 2024-03-21T10:26:59.324 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:59.322+0000 7fd152e00700 1 -- 172.21.15.153:0/3618110816 wait complete. 2024-03-21T10:26:59.325 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:59.323+0000 7fd152e00700 1 Processor -- start 2024-03-21T10:26:59.325 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:59.324+0000 7fd152e00700 1 -- start start 2024-03-21T10:26:59.325 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:59.324+0000 7fd152e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd14c100dc0 0x7fd14c19e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:59.325 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:59.324+0000 7fd152e00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd14c101770 0x7fd14c19ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:59.325 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:59.324+0000 7fd152e00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd14c19f3e0 con 0x7fd14c100dc0 2024-03-21T10:26:59.325 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:59.324+0000 7fd152e00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd14c198890 con 0x7fd14c101770 2024-03-21T10:26:59.325 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:59.324+0000 7fd14be00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd14c100dc0 0x7fd14c19e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:59.325 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:59.324+0000 7fd14be00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd14c100dc0 0x7fd14c19e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:52612/0 (socket says 172.21.15.153:52612) 2024-03-21T10:26:59.325 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:59.324+0000 7fd14be00700 1 -- 172.21.15.153:0/1704566681 learned_addr learned my addr 172.21.15.153:0/1704566681 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:26:59.326 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:59.325+0000 7fd14b400700 1 --2- 172.21.15.153:0/1704566681 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd14c101770 0x7fd14c19ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:59.326 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:59.325+0000 7fd14be00700 1 -- 172.21.15.153:0/1704566681 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd14c101770 msgr2=0x7fd14c19ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:59.327 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:59.325+0000 7fd14be00700 1 --2- 172.21.15.153:0/1704566681 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd14c101770 0x7fd14c19ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:59.327 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:59.325+0000 7fd14be00700 1 -- 172.21.15.153:0/1704566681 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd13c0095f0 con 0x7fd14c100dc0 2024-03-21T10:26:59.327 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:59.325+0000 7fd14be00700 1 --2- 172.21.15.153:0/1704566681 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd14c100dc0 0x7fd14c19e810 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fd13c00b5c0 tx=0x7fd13c00be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:59.327 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:59.325+0000 7fd148c00700 1 -- 172.21.15.153:0/1704566681 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd13c01d070 con 0x7fd14c100dc0 2024-03-21T10:26:59.327 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:59.325+0000 7fd148c00700 1 -- 172.21.15.153:0/1704566681 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd13c00f460 con 0x7fd14c100dc0 2024-03-21T10:26:59.327 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:59.325+0000 7fd148c00700 1 -- 172.21.15.153:0/1704566681 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd13c017610 con 0x7fd14c100dc0 2024-03-21T10:26:59.327 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:59.325+0000 7fd152e00700 1 -- 172.21.15.153:0/1704566681 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd14c198b10 con 0x7fd14c100dc0 2024-03-21T10:26:59.327 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:59.326+0000 7fd152e00700 1 -- 172.21.15.153:0/1704566681 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd14c199000 con 0x7fd14c100dc0 2024-03-21T10:26:59.327 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:59.327+0000 7fd152e00700 1 -- 172.21.15.153:0/1704566681 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd14c104f40 con 0x7fd14c100dc0 2024-03-21T10:26:59.328 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:59.328+0000 7fd148c00700 1 -- 172.21.15.153:0/1704566681 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fd13c00f5d0 con 0x7fd14c100dc0 2024-03-21T10:26:59.329 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:59.329+0000 7fd148c00700 1 --2- 172.21.15.153:0/1704566681 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd13006e460 0x7fd130070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:59.330 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:59.329+0000 7fd148c00700 1 -- 172.21.15.153:0/1704566681 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fd13c08fb10 con 0x7fd14c100dc0 2024-03-21T10:26:59.330 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:59.329+0000 7fd14b400700 1 --2- 172.21.15.153:0/1704566681 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd13006e460 0x7fd130070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:59.330 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:59.330+0000 7fd14b400700 1 --2- 172.21.15.153:0/1704566681 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd13006e460 0x7fd130070920 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd138006fd0 tx=0x7fd138008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:59.335 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:59.334+0000 7fd148c00700 1 -- 172.21.15.153:0/1704566681 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd13c05c6e0 con 0x7fd14c100dc0 2024-03-21T10:26:59.578 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:59.577+0000 7fd152e00700 1 -- 172.21.15.153:0/1704566681 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd14c1993a0 con 0x7fd13006e460 2024-03-21T10:26:59.580 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:59.580+0000 7fd148c00700 1 -- 172.21.15.153:0/1704566681 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fd14c1993a0 con 0x7fd13006e460 2024-03-21T10:26:59.584 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:59.582+0000 7fd152e00700 1 -- 172.21.15.153:0/1704566681 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd13006e460 msgr2=0x7fd130070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:59.584 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:59.582+0000 7fd152e00700 1 --2- 172.21.15.153:0/1704566681 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd13006e460 0x7fd130070920 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd138006fd0 tx=0x7fd138008040 comp rx=0 tx=0).stop 2024-03-21T10:26:59.584 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:59.582+0000 7fd152e00700 1 -- 172.21.15.153:0/1704566681 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd14c100dc0 msgr2=0x7fd14c19e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:59.584 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:59.582+0000 7fd152e00700 1 --2- 172.21.15.153:0/1704566681 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd14c100dc0 0x7fd14c19e810 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fd13c00b5c0 tx=0x7fd13c00be00 comp rx=0 tx=0).stop 2024-03-21T10:26:59.584 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:59.583+0000 7fd152e00700 1 -- 172.21.15.153:0/1704566681 shutdown_connections 2024-03-21T10:26:59.584 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:59.583+0000 7fd152e00700 1 --2- 172.21.15.153:0/1704566681 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd13006e460 0x7fd130070920 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:59.584 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:59.583+0000 7fd152e00700 1 --2- 172.21.15.153:0/1704566681 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd14c100dc0 0x7fd14c19e810 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:59.584 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:59.583+0000 7fd152e00700 1 --2- 172.21.15.153:0/1704566681 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd14c101770 0x7fd14c19ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:59.584 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:59.583+0000 7fd152e00700 1 -- 172.21.15.153:0/1704566681 >> 172.21.15.153:0/1704566681 conn(0x7fd14c0fc650 msgr2=0x7fd14c0fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:59.584 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:59.583+0000 7fd152e00700 1 -- 172.21.15.153:0/1704566681 shutdown_connections 2024-03-21T10:26:59.584 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:59.583+0000 7fd152e00700 1 -- 172.21.15.153:0/1704566681 wait complete. 2024-03-21T10:26:59.600 INFO:teuthology.orchestra.run.smithi153.stdout:true 2024-03-21T10:26:59.736 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:59.735+0000 7f689b800700 1 -- 172.21.15.153:0/3207443593 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6894103000 msgr2=0x7f68941033e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:59.736 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:59.735+0000 7f689b800700 1 --2- 172.21.15.153:0/3207443593 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6894103000 0x7f68941033e0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f6888009910 tx=0x7f6888009c20 comp rx=0 tx=0).stop 2024-03-21T10:26:59.736 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:59.735+0000 7f689b800700 1 -- 172.21.15.153:0/3207443593 shutdown_connections 2024-03-21T10:26:59.736 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:59.735+0000 7f689b800700 1 --2- 172.21.15.153:0/3207443593 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f68941039b0 0x7f6894107a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:59.736 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:59.735+0000 7f689b800700 1 --2- 172.21.15.153:0/3207443593 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6894103000 0x7f68941033e0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:59.736 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:59.735+0000 7f689b800700 1 -- 172.21.15.153:0/3207443593 >> 172.21.15.153:0/3207443593 conn(0x7f68940fe8d0 msgr2=0x7f6894100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:59.736 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:59.735+0000 7f689b800700 1 -- 172.21.15.153:0/3207443593 shutdown_connections 2024-03-21T10:26:59.737 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:59.735+0000 7f689b800700 1 -- 172.21.15.153:0/3207443593 wait complete. 2024-03-21T10:26:59.737 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:59.736+0000 7f689b800700 1 Processor -- start 2024-03-21T10:26:59.737 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:59.736+0000 7f689b800700 1 -- start start 2024-03-21T10:26:59.737 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:59.737+0000 7f689b800700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6894103000 0x7f689419e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:59.737 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:59.737+0000 7f689b800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f68941039b0 0x7f689419ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:59.738 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:59.737+0000 7f689b800700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f689419f3c0 con 0x7f68941039b0 2024-03-21T10:26:59.738 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:59.737+0000 7f689b800700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6894198870 con 0x7f6894103000 2024-03-21T10:26:59.739 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:59.737+0000 7f6893e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f68941039b0 0x7f689419ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:59.739 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:59.737+0000 7f6893e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f68941039b0 0x7f689419ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:52630/0 (socket says 172.21.15.153:52630) 2024-03-21T10:26:59.739 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:59.737+0000 7f6893e00700 1 -- 172.21.15.153:0/905110387 learned_addr learned my addr 172.21.15.153:0/905110387 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:26:59.739 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:59.737+0000 7f6898e00700 1 --2- 172.21.15.153:0/905110387 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6894103000 0x7f689419e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:59.739 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:59.737+0000 7f6893e00700 1 -- 172.21.15.153:0/905110387 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6894103000 msgr2=0x7f689419e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:59.739 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:59.737+0000 7f6893e00700 1 --2- 172.21.15.153:0/905110387 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6894103000 0x7f689419e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:59.739 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:59.737+0000 7f6893e00700 1 -- 172.21.15.153:0/905110387 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68880095f0 con 0x7f68941039b0 2024-03-21T10:26:59.739 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:59.738+0000 7f6893e00700 1 --2- 172.21.15.153:0/905110387 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f68941039b0 0x7f689419ed30 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f688000d900 tx=0x7f688000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:59.739 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:59.738+0000 7f6891600700 1 -- 172.21.15.153:0/905110387 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6880009900 con 0x7f68941039b0 2024-03-21T10:26:59.740 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:59.738+0000 7f6891600700 1 -- 172.21.15.153:0/905110387 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6880010460 con 0x7f68941039b0 2024-03-21T10:26:59.740 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:59.738+0000 7f689b800700 1 -- 172.21.15.153:0/905110387 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6894198b50 con 0x7f68941039b0 2024-03-21T10:26:59.740 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:59.738+0000 7f6891600700 1 -- 172.21.15.153:0/905110387 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f688000f5f0 con 0x7f68941039b0 2024-03-21T10:26:59.740 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:59.738+0000 7f689b800700 1 -- 172.21.15.153:0/905110387 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68941990a0 con 0x7f68941039b0 2024-03-21T10:26:59.740 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:59.739+0000 7f689b800700 1 -- 172.21.15.153:0/905110387 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f689410b3a0 con 0x7f68941039b0 2024-03-21T10:26:59.741 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:59.740+0000 7f6891600700 1 -- 172.21.15.153:0/905110387 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f68800105d0 con 0x7f68941039b0 2024-03-21T10:26:59.742 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:59.742+0000 7f6891600700 1 --2- 172.21.15.153:0/905110387 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f687c06e460 0x7f687c070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:26:59.742 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:59.742+0000 7f6891600700 1 -- 172.21.15.153:0/905110387 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f688008e360 con 0x7f68941039b0 2024-03-21T10:26:59.743 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:59.742+0000 7f6898e00700 1 --2- 172.21.15.153:0/905110387 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f687c06e460 0x7f687c070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:26:59.743 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:59.743+0000 7f6898e00700 1 --2- 172.21.15.153:0/905110387 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f687c06e460 0x7f687c070920 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6888006010 tx=0x7f688800b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:26:59.747 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:59.747+0000 7f6891600700 1 -- 172.21.15.153:0/905110387 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f688005af30 con 0x7f68941039b0 2024-03-21T10:26:59.972 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:59.971+0000 7f689b800700 1 -- 172.21.15.153:0/905110387 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6894199380 con 0x7f687c06e460 2024-03-21T10:26:59.973 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:59.973+0000 7f6891600700 1 -- 172.21.15.153:0/905110387 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f6894199380 con 0x7f687c06e460 2024-03-21T10:26:59.976 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:59.975+0000 7f689b800700 1 -- 172.21.15.153:0/905110387 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f687c06e460 msgr2=0x7f687c070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:59.976 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:59.975+0000 7f689b800700 1 --2- 172.21.15.153:0/905110387 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f687c06e460 0x7f687c070920 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6888006010 tx=0x7f688800b540 comp rx=0 tx=0).stop 2024-03-21T10:26:59.976 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:59.975+0000 7f689b800700 1 -- 172.21.15.153:0/905110387 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f68941039b0 msgr2=0x7f689419ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:26:59.976 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:59.975+0000 7f689b800700 1 --2- 172.21.15.153:0/905110387 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f68941039b0 0x7f689419ed30 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f688000d900 tx=0x7f688000dcc0 comp rx=0 tx=0).stop 2024-03-21T10:26:59.977 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:59.976+0000 7f689b800700 1 -- 172.21.15.153:0/905110387 shutdown_connections 2024-03-21T10:26:59.977 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:59.976+0000 7f689b800700 1 --2- 172.21.15.153:0/905110387 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6894103000 0x7f689419e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:59.977 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:59.976+0000 7f689b800700 1 --2- 172.21.15.153:0/905110387 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f687c06e460 0x7f687c070920 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:59.977 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:59.976+0000 7f689b800700 1 --2- 172.21.15.153:0/905110387 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f68941039b0 0x7f689419ed30 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:26:59.977 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:59.976+0000 7f689b800700 1 -- 172.21.15.153:0/905110387 >> 172.21.15.153:0/905110387 conn(0x7f68940fe8d0 msgr2=0x7f68940ffee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:26:59.977 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:59.976+0000 7f689b800700 1 -- 172.21.15.153:0/905110387 shutdown_connections 2024-03-21T10:26:59.977 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:26:59.976+0000 7f689b800700 1 -- 172.21.15.153:0/905110387 wait complete. 2024-03-21T10:27:00.134 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:00.133+0000 7fe2dde00700 1 -- 172.21.15.153:0/1695284058 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe2d8102620 msgr2=0x7fe2d810ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:00.134 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:00.133+0000 7fe2dde00700 1 --2- 172.21.15.153:0/1695284058 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe2d8102620 0x7fe2d810ab10 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fe2c4009910 tx=0x7fe2c4009c20 comp rx=0 tx=0).stop 2024-03-21T10:27:00.134 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:00.133+0000 7fe2dde00700 1 -- 172.21.15.153:0/1695284058 shutdown_connections 2024-03-21T10:27:00.134 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:00.133+0000 7fe2dde00700 1 --2- 172.21.15.153:0/1695284058 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe2d8102620 0x7fe2d810ab10 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:00.134 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:00.133+0000 7fe2dde00700 1 --2- 172.21.15.153:0/1695284058 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe2d8101d00 0x7fe2d81020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:00.134 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:00.133+0000 7fe2dde00700 1 -- 172.21.15.153:0/1695284058 >> 172.21.15.153:0/1695284058 conn(0x7fe2d80fb570 msgr2=0x7fe2d80fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:00.134 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:00.133+0000 7fe2dde00700 1 -- 172.21.15.153:0/1695284058 shutdown_connections 2024-03-21T10:27:00.134 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:00.133+0000 7fe2dde00700 1 -- 172.21.15.153:0/1695284058 wait complete. 2024-03-21T10:27:00.135 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:00.134+0000 7fe2dde00700 1 Processor -- start 2024-03-21T10:27:00.135 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:00.135+0000 7fe2dde00700 1 -- start start 2024-03-21T10:27:00.136 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:00.135+0000 7fe2dde00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe2d8101d00 0x7fe2d819c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:00.136 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:00.135+0000 7fe2dde00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe2d8102620 0x7fe2d819cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:00.136 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:00.135+0000 7fe2dde00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d819d150 con 0x7fe2d8101d00 2024-03-21T10:27:00.136 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:00.135+0000 7fe2dde00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d8196600 con 0x7fe2d8102620 2024-03-21T10:27:00.136 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:00.136+0000 7fe2d6a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe2d8101d00 0x7fe2d819c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:00.136 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:00.136+0000 7fe2d6a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe2d8101d00 0x7fe2d819c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:52640/0 (socket says 172.21.15.153:52640) 2024-03-21T10:27:00.136 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:00.136+0000 7fe2d6a00700 1 -- 172.21.15.153:0/2221194334 learned_addr learned my addr 172.21.15.153:0/2221194334 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:27:00.137 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:00.136+0000 7fe2d6a00700 1 -- 172.21.15.153:0/2221194334 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe2d8102620 msgr2=0x7fe2d819cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:00.137 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:00.136+0000 7fe2d6a00700 1 --2- 172.21.15.153:0/2221194334 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe2d8102620 0x7fe2d819cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:00.137 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:00.136+0000 7fe2d6a00700 1 -- 172.21.15.153:0/2221194334 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2c40095f0 con 0x7fe2d8101d00 2024-03-21T10:27:00.137 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:00.136+0000 7fe2d6a00700 1 --2- 172.21.15.153:0/2221194334 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe2d8101d00 0x7fe2d819c580 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fe2c800d8d0 tx=0x7fe2c800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:00.137 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:00.137+0000 7fe2dca00700 1 -- 172.21.15.153:0/2221194334 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe2c8009940 con 0x7fe2d8101d00 2024-03-21T10:27:00.138 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:00.137+0000 7fe2dca00700 1 -- 172.21.15.153:0/2221194334 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe2c8010460 con 0x7fe2d8101d00 2024-03-21T10:27:00.138 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:00.137+0000 7fe2dca00700 1 -- 172.21.15.153:0/2221194334 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe2c800f5f0 con 0x7fe2d8101d00 2024-03-21T10:27:00.138 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:00.137+0000 7fe2dde00700 1 -- 172.21.15.153:0/2221194334 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2d81968e0 con 0x7fe2d8101d00 2024-03-21T10:27:00.138 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:00.137+0000 7fe2dde00700 1 -- 172.21.15.153:0/2221194334 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2d8196e30 con 0x7fe2d8101d00 2024-03-21T10:27:00.138 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:00.138+0000 7fe2dde00700 1 -- 172.21.15.153:0/2221194334 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2d8108210 con 0x7fe2d8101d00 2024-03-21T10:27:00.139 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:00.139+0000 7fe2dca00700 1 -- 172.21.15.153:0/2221194334 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fe2c80105d0 con 0x7fe2d8101d00 2024-03-21T10:27:00.141 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:00.140+0000 7fe2dca00700 1 --2- 172.21.15.153:0/2221194334 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe2bc06e460 0x7fe2bc070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:00.141 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:00.140+0000 7fe2dca00700 1 -- 172.21.15.153:0/2221194334 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fe2c808e5b0 con 0x7fe2d8101d00 2024-03-21T10:27:00.141 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:00.140+0000 7fe2d6000700 1 --2- 172.21.15.153:0/2221194334 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe2bc06e460 0x7fe2bc070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:00.141 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:00.141+0000 7fe2d6000700 1 --2- 172.21.15.153:0/2221194334 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe2bc06e460 0x7fe2bc070920 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe2c400b5c0 tx=0x7fe2c4005fd0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:00.145 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:00.145+0000 7fe2dca00700 1 -- 172.21.15.153:0/2221194334 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe2c805b100 con 0x7fe2d8101d00 2024-03-21T10:27:00.377 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:00.376+0000 7fe2dde00700 1 -- 172.21.15.153:0/2221194334 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fe2d81970a0 con 0x7fe2bc06e460 2024-03-21T10:27:00.406 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:00.405+0000 7fe2dca00700 1 -- 172.21.15.153:0/2221194334 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fe2d81970a0 con 0x7fe2bc06e460 2024-03-21T10:27:00.407 INFO:teuthology.orchestra.run.smithi153.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-21T10:27:00.407 INFO:teuthology.orchestra.run.smithi153.stdout:alertmanager.smithi153 smithi153 *:9093,9094 running (15m) 7m ago 29m 14.7M - 0.25.0 c8568f914cd2 9ed3f2200b5f 2024-03-21T10:27:00.407 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi153 smithi153 running (29m) 7m ago 29m 9424k - 18.2.1 5be31c24972a 5b61994396a6 2024-03-21T10:27:00.407 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi174 smithi174 running (28m) 6m ago 28m 9244k - 18.2.1 5be31c24972a 4f7a5fc3ac28 2024-03-21T10:27:00.407 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi153 smithi153 running (12m) 7m ago 29m 6656k - 19.0.0-2231-g2edfee68 237e43c7a65f 32c21104d5b0 2024-03-21T10:27:00.407 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi174 smithi174 running (11m) 6m ago 28m 6660k - 19.0.0-2231-g2edfee68 237e43c7a65f a04e48967ab6 2024-03-21T10:27:00.407 INFO:teuthology.orchestra.run.smithi153.stdout:grafana.smithi153 smithi153 *:3000 running (15m) 7m ago 29m 73.6M - 9.4.12 2bacad6d85d8 ee24026ab786 2024-03-21T10:27:00.407 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.iuyluj smithi153 running (7m) 7m ago 22m 10.9M - 19.0.0-2231-g2edfee68 237e43c7a65f 87aff8a5de70 2024-03-21T10:27:00.407 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.zipckr smithi153 running (7m) 7m ago 22m 11.8M - 19.0.0-2231-g2edfee68 237e43c7a65f ccc262bd5dd3 2024-03-21T10:27:00.407 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.nhozag smithi174 running (22m) 6m ago 22m 12.0M - 18.2.1 5be31c24972a 2fee4000308f 2024-03-21T10:27:00.407 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.yyimst smithi174 running (6m) 6m ago 22m 10.7M - 19.0.0-2231-g2edfee68 237e43c7a65f fc1962f9351e 2024-03-21T10:27:00.408 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi153.tluxni smithi153 *:8443,9283,8765 running (18m) 7m ago 31m 509M - 19.0.0-2231-g2edfee68 237e43c7a65f 7ffee0e55382 2024-03-21T10:27:00.408 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi174.qpcwat smithi174 *:8443,9283,8765 running (17m) 6m ago 28m 452M - 19.0.0-2231-g2edfee68 237e43c7a65f 748968f216ab 2024-03-21T10:27:00.408 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi153 smithi153 running (13m) 7m ago 31m 55.0M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f dd8038303dc3 2024-03-21T10:27:00.408 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi174 smithi174 running (13m) 6m ago 27m 44.2M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f 29d78a788f2c 2024-03-21T10:27:00.408 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi153 smithi153 *:9100 running (16m) 7m ago 29m 7889k - 1.5.0 0da6a335fe13 bb835edc0e0b 2024-03-21T10:27:00.408 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi174 smithi174 *:9100 running (16m) 6m ago 28m 8102k - 1.5.0 0da6a335fe13 7c4ecad790fa 2024-03-21T10:27:00.408 INFO:teuthology.orchestra.run.smithi153.stdout:osd.0 smithi153 running (11m) 7m ago 27m 116M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 507900fd4791 2024-03-21T10:27:00.408 INFO:teuthology.orchestra.run.smithi153.stdout:osd.1 smithi153 running (10m) 7m ago 26m 89.2M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 0d646452e9c1 2024-03-21T10:27:00.408 INFO:teuthology.orchestra.run.smithi153.stdout:osd.2 smithi153 running (10m) 7m ago 25m 87.4M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 1326f2afdfcd 2024-03-21T10:27:00.408 INFO:teuthology.orchestra.run.smithi153.stdout:osd.3 smithi174 running (9m) 6m ago 25m 83.6M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 70aafac71ad5 2024-03-21T10:27:00.408 INFO:teuthology.orchestra.run.smithi153.stdout:osd.4 smithi174 running (8m) 6m ago 24m 92.2M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 0d746ce6e79d 2024-03-21T10:27:00.408 INFO:teuthology.orchestra.run.smithi153.stdout:osd.5 smithi174 running (8m) 6m ago 24m 86.2M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 53e5f35b8cac 2024-03-21T10:27:00.408 INFO:teuthology.orchestra.run.smithi153.stdout:prometheus.smithi153 smithi153 *:9095 running (16m) 7m ago 28m 73.5M - 2.43.0 a07b618ecd1d 7d95561e50c2 2024-03-21T10:27:00.410 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:00.409+0000 7fe2dde00700 1 -- 172.21.15.153:0/2221194334 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe2bc06e460 msgr2=0x7fe2bc070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:00.410 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:00.409+0000 7fe2dde00700 1 --2- 172.21.15.153:0/2221194334 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe2bc06e460 0x7fe2bc070920 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe2c400b5c0 tx=0x7fe2c4005fd0 comp rx=0 tx=0).stop 2024-03-21T10:27:00.410 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:00.409+0000 7fe2dde00700 1 -- 172.21.15.153:0/2221194334 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe2d8101d00 msgr2=0x7fe2d819c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:00.410 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:00.409+0000 7fe2dde00700 1 --2- 172.21.15.153:0/2221194334 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe2d8101d00 0x7fe2d819c580 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fe2c800d8d0 tx=0x7fe2c800dc90 comp rx=0 tx=0).stop 2024-03-21T10:27:00.410 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:00.409+0000 7fe2dde00700 1 -- 172.21.15.153:0/2221194334 shutdown_connections 2024-03-21T10:27:00.411 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:00.409+0000 7fe2dde00700 1 --2- 172.21.15.153:0/2221194334 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe2bc06e460 0x7fe2bc070920 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:00.411 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:00.409+0000 7fe2dde00700 1 --2- 172.21.15.153:0/2221194334 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe2d8101d00 0x7fe2d819c580 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:00.411 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:00.409+0000 7fe2dde00700 1 --2- 172.21.15.153:0/2221194334 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe2d8102620 0x7fe2d819cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:00.411 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:00.409+0000 7fe2dde00700 1 -- 172.21.15.153:0/2221194334 >> 172.21.15.153:0/2221194334 conn(0x7fe2d80fb570 msgr2=0x7fe2d8105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:00.411 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:00.410+0000 7fe2dde00700 1 -- 172.21.15.153:0/2221194334 shutdown_connections 2024-03-21T10:27:00.411 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:00.410+0000 7fe2dde00700 1 -- 172.21.15.153:0/2221194334 wait complete. 2024-03-21T10:27:00.563 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:00.562+0000 7f6369400700 1 -- 172.21.15.153:0/1957528351 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f63640ff8f0 msgr2=0x7f63640ffcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:00.563 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:00.562+0000 7f6369400700 1 --2- 172.21.15.153:0/1957528351 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f63640ff8f0 0x7f63640ffcd0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f6354009910 tx=0x7f6354009c20 comp rx=0 tx=0).stop 2024-03-21T10:27:00.563 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:00.562+0000 7f6369400700 1 -- 172.21.15.153:0/1957528351 shutdown_connections 2024-03-21T10:27:00.563 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:00.562+0000 7f6369400700 1 --2- 172.21.15.153:0/1957528351 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f63641002a0 0x7f636410cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:00.563 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:00.562+0000 7f6369400700 1 --2- 172.21.15.153:0/1957528351 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f63640ff8f0 0x7f63640ffcd0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:00.563 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:00.562+0000 7f6369400700 1 -- 172.21.15.153:0/1957528351 >> 172.21.15.153:0/1957528351 conn(0x7f63640fb570 msgr2=0x7f63640fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:00.563 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:00.562+0000 7f6369400700 1 -- 172.21.15.153:0/1957528351 shutdown_connections 2024-03-21T10:27:00.563 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:00.562+0000 7f6369400700 1 -- 172.21.15.153:0/1957528351 wait complete. 2024-03-21T10:27:00.564 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:00.563+0000 7f6369400700 1 Processor -- start 2024-03-21T10:27:00.564 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:00.563+0000 7f6369400700 1 -- start start 2024-03-21T10:27:00.565 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:00.563+0000 7f6369400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f63640ff8f0 0x7f636419e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:00.565 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:00.563+0000 7f6369400700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f63641002a0 0x7f636419ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:00.565 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:00.563+0000 7f6369400700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f636419f2a0 con 0x7f63640ff8f0 2024-03-21T10:27:00.565 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:00.563+0000 7f6369400700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6364198750 con 0x7f63641002a0 2024-03-21T10:27:00.565 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:00.564+0000 7f6362800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f63640ff8f0 0x7f636419e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:00.565 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:00.564+0000 7f6362800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f63640ff8f0 0x7f636419e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:52670/0 (socket says 172.21.15.153:52670) 2024-03-21T10:27:00.565 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:00.564+0000 7f6362800700 1 -- 172.21.15.153:0/1120268557 learned_addr learned my addr 172.21.15.153:0/1120268557 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:27:00.565 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:00.564+0000 7f6362800700 1 -- 172.21.15.153:0/1120268557 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f63641002a0 msgr2=0x7f636419ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:00.565 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:00.564+0000 7f6362800700 1 --2- 172.21.15.153:0/1120268557 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f63641002a0 0x7f636419ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:00.565 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:00.564+0000 7f6362800700 1 -- 172.21.15.153:0/1120268557 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63540095f0 con 0x7f63640ff8f0 2024-03-21T10:27:00.566 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:00.564+0000 7f6362800700 1 --2- 172.21.15.153:0/1120268557 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f63640ff8f0 0x7f636419e6d0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f6354005b40 tx=0x7f63540056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:00.566 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:00.565+0000 7f635b400700 1 -- 172.21.15.153:0/1120268557 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f635401d070 con 0x7f63640ff8f0 2024-03-21T10:27:00.566 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:00.565+0000 7f635b400700 1 -- 172.21.15.153:0/1120268557 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f635400f460 con 0x7f63640ff8f0 2024-03-21T10:27:00.566 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:00.565+0000 7f6369400700 1 -- 172.21.15.153:0/1120268557 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6364198a30 con 0x7f63640ff8f0 2024-03-21T10:27:00.566 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:00.565+0000 7f635b400700 1 -- 172.21.15.153:0/1120268557 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6354021640 con 0x7f63640ff8f0 2024-03-21T10:27:00.566 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:00.565+0000 7f6369400700 1 -- 172.21.15.153:0/1120268557 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6364198f80 con 0x7f63640ff8f0 2024-03-21T10:27:00.567 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:00.566+0000 7f6369400700 1 -- 172.21.15.153:0/1120268557 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f636410a6d0 con 0x7f63640ff8f0 2024-03-21T10:27:00.567 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:00.567+0000 7f635b400700 1 -- 172.21.15.153:0/1120268557 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f635400f5d0 con 0x7f63640ff8f0 2024-03-21T10:27:00.568 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:00.568+0000 7f635b400700 1 --2- 172.21.15.153:0/1120268557 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f634806e460 0x7f6348070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:00.568 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:00.568+0000 7f635b400700 1 -- 172.21.15.153:0/1120268557 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f635408f950 con 0x7f63640ff8f0 2024-03-21T10:27:00.568 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:00.568+0000 7f6361e00700 1 --2- 172.21.15.153:0/1120268557 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f634806e460 0x7f6348070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:00.569 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:00.569+0000 7f6361e00700 1 --2- 172.21.15.153:0/1120268557 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f634806e460 0x7f6348070920 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f6364199bc0 tx=0x7f6350005c80 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:00.573 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:00.572+0000 7f635b400700 1 -- 172.21.15.153:0/1120268557 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f635405c4a0 con 0x7f63640ff8f0 2024-03-21T10:27:00.884 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:00.883+0000 7f6369400700 1 -- 172.21.15.153:0/1120268557 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f636404ea90 con 0x7f63640ff8f0 2024-03-21T10:27:00.884 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:00.884+0000 7f635b400700 1 -- 172.21.15.153:0/1120268557 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+805 (secure 0 0 0) 0x7f635405c030 con 0x7f63640ff8f0 2024-03-21T10:27:00.885 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:27:00.885 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": { 2024-03-21T10:27:00.885 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:27:00.885 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:27:00.885 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": { 2024-03-21T10:27:00.885 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:27:00.885 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:27:00.885 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": { 2024-03-21T10:27:00.886 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 6 2024-03-21T10:27:00.886 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:27:00.886 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": { 2024-03-21T10:27:00.886 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-03-21T10:27:00.886 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 3 2024-03-21T10:27:00.886 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:27:00.886 INFO:teuthology.orchestra.run.smithi153.stdout: "overall": { 2024-03-21T10:27:00.886 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-03-21T10:27:00.886 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 13 2024-03-21T10:27:00.886 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-03-21T10:27:00.886 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:27:00.888 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:00.887+0000 7f6369400700 1 -- 172.21.15.153:0/1120268557 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f634806e460 msgr2=0x7f6348070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:00.888 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:00.887+0000 7f6369400700 1 --2- 172.21.15.153:0/1120268557 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f634806e460 0x7f6348070920 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f6364199bc0 tx=0x7f6350005c80 comp rx=0 tx=0).stop 2024-03-21T10:27:00.888 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:00.887+0000 7f6369400700 1 -- 172.21.15.153:0/1120268557 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f63640ff8f0 msgr2=0x7f636419e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:00.888 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:00.887+0000 7f6369400700 1 --2- 172.21.15.153:0/1120268557 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f63640ff8f0 0x7f636419e6d0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f6354005b40 tx=0x7f63540056a0 comp rx=0 tx=0).stop 2024-03-21T10:27:00.888 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:00.887+0000 7f6369400700 1 -- 172.21.15.153:0/1120268557 shutdown_connections 2024-03-21T10:27:00.888 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:00.887+0000 7f6369400700 1 --2- 172.21.15.153:0/1120268557 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f634806e460 0x7f6348070920 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:00.888 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:00.887+0000 7f6369400700 1 --2- 172.21.15.153:0/1120268557 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f63640ff8f0 0x7f636419e6d0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:00.889 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:00.887+0000 7f6369400700 1 --2- 172.21.15.153:0/1120268557 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f63641002a0 0x7f636419ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:00.889 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:00.887+0000 7f6369400700 1 -- 172.21.15.153:0/1120268557 >> 172.21.15.153:0/1120268557 conn(0x7f63640fb570 msgr2=0x7f63640fcbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:00.889 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:00.887+0000 7f6369400700 1 -- 172.21.15.153:0/1120268557 shutdown_connections 2024-03-21T10:27:00.889 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:00.887+0000 7f6369400700 1 -- 172.21.15.153:0/1120268557 wait complete. 2024-03-21T10:27:01.035 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.034+0000 7f669d600700 1 -- 172.21.15.153:0/378460062 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6698075be0 msgr2=0x7f66981114e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:01.035 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.034+0000 7f669d600700 1 --2- 172.21.15.153:0/378460062 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6698075be0 0x7f66981114e0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f6684009910 tx=0x7f6684009c20 comp rx=0 tx=0).stop 2024-03-21T10:27:01.035 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.034+0000 7f669d600700 1 -- 172.21.15.153:0/378460062 shutdown_connections 2024-03-21T10:27:01.035 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.034+0000 7f669d600700 1 --2- 172.21.15.153:0/378460062 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6698075be0 0x7f66981114e0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:01.035 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.034+0000 7f669d600700 1 --2- 172.21.15.153:0/378460062 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66980752c0 0x7f66980756a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:01.035 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.034+0000 7f669d600700 1 -- 172.21.15.153:0/378460062 >> 172.21.15.153:0/378460062 conn(0x7f66980fe870 msgr2=0x7f6698100c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:01.035 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.034+0000 7f669d600700 1 -- 172.21.15.153:0/378460062 shutdown_connections 2024-03-21T10:27:01.035 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.034+0000 7f669d600700 1 -- 172.21.15.153:0/378460062 wait complete. 2024-03-21T10:27:01.036 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.035+0000 7f669d600700 1 Processor -- start 2024-03-21T10:27:01.036 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.035+0000 7f669d600700 1 -- start start 2024-03-21T10:27:01.036 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.035+0000 7f669d600700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66980752c0 0x7f669819cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:01.036 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.035+0000 7f669d600700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6698075be0 0x7f669819d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:01.037 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.035+0000 7f669d600700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f669819dac0 con 0x7f6698075be0 2024-03-21T10:27:01.037 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.035+0000 7f669d600700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66981a1850 con 0x7f66980752c0 2024-03-21T10:27:01.037 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.036+0000 7f6696000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6698075be0 0x7f669819d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:01.037 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.036+0000 7f6696000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6698075be0 0x7f669819d3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:52684/0 (socket says 172.21.15.153:52684) 2024-03-21T10:27:01.037 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.036+0000 7f6696000700 1 -- 172.21.15.153:0/708362363 learned_addr learned my addr 172.21.15.153:0/708362363 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:27:01.038 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.036+0000 7f6696a00700 1 --2- 172.21.15.153:0/708362363 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66980752c0 0x7f669819cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:01.038 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.036+0000 7f6696000700 1 -- 172.21.15.153:0/708362363 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66980752c0 msgr2=0x7f669819cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:01.038 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.036+0000 7f6696000700 1 --2- 172.21.15.153:0/708362363 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66980752c0 0x7f669819cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:01.038 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.036+0000 7f6696000700 1 -- 172.21.15.153:0/708362363 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66840095f0 con 0x7f6698075be0 2024-03-21T10:27:01.038 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.036+0000 7f6696a00700 1 --2- 172.21.15.153:0/708362363 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66980752c0 0x7f669819cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:27:01.038 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.037+0000 7f6696000700 1 --2- 172.21.15.153:0/708362363 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6698075be0 0x7f669819d3e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f6684006010 tx=0x7f66840056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:01.038 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.037+0000 7f668f400700 1 -- 172.21.15.153:0/708362363 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f668401d070 con 0x7f6698075be0 2024-03-21T10:27:01.038 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.037+0000 7f668f400700 1 -- 172.21.15.153:0/708362363 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f668400f460 con 0x7f6698075be0 2024-03-21T10:27:01.038 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.037+0000 7f669d600700 1 -- 172.21.15.153:0/708362363 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66981a1b30 con 0x7f6698075be0 2024-03-21T10:27:01.038 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.037+0000 7f668f400700 1 -- 172.21.15.153:0/708362363 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6684021640 con 0x7f6698075be0 2024-03-21T10:27:01.039 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.037+0000 7f669d600700 1 -- 172.21.15.153:0/708362363 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66981a2080 con 0x7f6698075be0 2024-03-21T10:27:01.039 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.038+0000 7f669d600700 1 -- 172.21.15.153:0/708362363 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f669810ec50 con 0x7f6698075be0 2024-03-21T10:27:01.040 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.039+0000 7f668f400700 1 -- 172.21.15.153:0/708362363 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f668400f5d0 con 0x7f6698075be0 2024-03-21T10:27:01.041 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.040+0000 7f668f400700 1 --2- 172.21.15.153:0/708362363 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f667c06e460 0x7f667c070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:01.041 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.040+0000 7f668f400700 1 -- 172.21.15.153:0/708362363 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f668408f830 con 0x7f6698075be0 2024-03-21T10:27:01.041 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.040+0000 7f6696a00700 1 --2- 172.21.15.153:0/708362363 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f667c06e460 0x7f667c070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:01.042 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.041+0000 7f6696a00700 1 --2- 172.21.15.153:0/708362363 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f667c06e460 0x7f667c070920 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6688007900 tx=0x7f6688008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:01.046 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.045+0000 7f668f400700 1 -- 172.21.15.153:0/708362363 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f6684058850 con 0x7f6698075be0 2024-03-21T10:27:01.055 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:27:00 smithi153 ceph-mon[113733]: pgmap v454: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:27:01.055 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:27:00 smithi153 ceph-mon[113733]: from='client.34842 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:27:01.055 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:27:00 smithi153 ceph-mon[113733]: from='client.34846 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:27:01.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:27:00 smithi174 ceph-mon[88004]: pgmap v454: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:27:01.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:27:00 smithi174 ceph-mon[88004]: from='client.34842 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:27:01.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:27:00 smithi174 ceph-mon[88004]: from='client.34846 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:27:01.310 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.310+0000 7f669d600700 1 -- 172.21.15.153:0/708362363 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f66981a1cc0 con 0x7f6698075be0 2024-03-21T10:27:01.311 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.310+0000 7f668f400700 1 -- 172.21.15.153:0/708362363 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1876 (secure 0 0 0) 0x7f66981a1cc0 con 0x7f6698075be0 2024-03-21T10:27:01.311 INFO:teuthology.orchestra.run.smithi153.stdout:e30 2024-03-21T10:27:01.311 INFO:teuthology.orchestra.run.smithi153.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-21T10:27:01.311 INFO:teuthology.orchestra.run.smithi153.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:27:01.312 INFO:teuthology.orchestra.run.smithi153.stdout:legacy client fscid: 1 2024-03-21T10:27:01.312 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:27:01.312 INFO:teuthology.orchestra.run.smithi153.stdout:Filesystem 'cephfs' (1) 2024-03-21T10:27:01.312 INFO:teuthology.orchestra.run.smithi153.stdout:fs_name cephfs 2024-03-21T10:27:01.312 INFO:teuthology.orchestra.run.smithi153.stdout:epoch 23 2024-03-21T10:27:01.312 INFO:teuthology.orchestra.run.smithi153.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-21T10:27:01.312 INFO:teuthology.orchestra.run.smithi153.stdout:created 2024-03-21T10:04:27.086576+0000 2024-03-21T10:27:01.312 INFO:teuthology.orchestra.run.smithi153.stdout:modified 2024-03-21T10:19:05.022092+0000 2024-03-21T10:27:01.312 INFO:teuthology.orchestra.run.smithi153.stdout:tableserver 0 2024-03-21T10:27:01.312 INFO:teuthology.orchestra.run.smithi153.stdout:root 0 2024-03-21T10:27:01.312 INFO:teuthology.orchestra.run.smithi153.stdout:session_timeout 60 2024-03-21T10:27:01.312 INFO:teuthology.orchestra.run.smithi153.stdout:session_autoclose 300 2024-03-21T10:27:01.312 INFO:teuthology.orchestra.run.smithi153.stdout:max_file_size 1099511627776 2024-03-21T10:27:01.312 INFO:teuthology.orchestra.run.smithi153.stdout:max_xattr_size 65536 2024-03-21T10:27:01.313 INFO:teuthology.orchestra.run.smithi153.stdout:required_client_features {} 2024-03-21T10:27:01.313 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure 0 2024-03-21T10:27:01.313 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure_osd_epoch 94 2024-03-21T10:27:01.313 INFO:teuthology.orchestra.run.smithi153.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:27:01.313 INFO:teuthology.orchestra.run.smithi153.stdout:max_mds 2 2024-03-21T10:27:01.313 INFO:teuthology.orchestra.run.smithi153.stdout:in 0,1 2024-03-21T10:27:01.313 INFO:teuthology.orchestra.run.smithi153.stdout:up {} 2024-03-21T10:27:01.313 INFO:teuthology.orchestra.run.smithi153.stdout:failed 0,1 2024-03-21T10:27:01.313 INFO:teuthology.orchestra.run.smithi153.stdout:damaged 2024-03-21T10:27:01.313 INFO:teuthology.orchestra.run.smithi153.stdout:stopped 2024-03-21T10:27:01.313 INFO:teuthology.orchestra.run.smithi153.stdout:data_pools [3] 2024-03-21T10:27:01.313 INFO:teuthology.orchestra.run.smithi153.stdout:metadata_pool 2 2024-03-21T10:27:01.313 INFO:teuthology.orchestra.run.smithi153.stdout:inline_data enabled 2024-03-21T10:27:01.313 INFO:teuthology.orchestra.run.smithi153.stdout:balancer 2024-03-21T10:27:01.313 INFO:teuthology.orchestra.run.smithi153.stdout:bal_rank_mask -1 2024-03-21T10:27:01.314 INFO:teuthology.orchestra.run.smithi153.stdout:standby_count_wanted 1 2024-03-21T10:27:01.314 INFO:teuthology.orchestra.run.smithi153.stdout:qdb_cluster leader: 0 members: 2024-03-21T10:27:01.314 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:27:01.314 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:27:01.314 INFO:teuthology.orchestra.run.smithi153.stdout:Standby daemons: 2024-03-21T10:27:01.314 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:27:01.314 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.zipckr{-1:34442} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6826/1103846295,v1:172.21.15.153:6827/1103846295] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:27:01.314 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.iuyluj{-1:34474} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6828/1738630832,v1:172.21.15.153:6829/1738630832] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:27:01.314 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.nhozag{-1:44269} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.174:6826/3967223227,v1:172.21.15.174:6827/3967223227] compat {c=[1],r=[1],i=[7ff]}] 2024-03-21T10:27:01.314 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.yyimst{-1:44313} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.174:6824/4127786797,v1:172.21.15.174:6825/4127786797] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:27:01.315 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.313+0000 7f669d600700 1 -- 172.21.15.153:0/708362363 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f667c06e460 msgr2=0x7f667c070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:01.315 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.313+0000 7f669d600700 1 --2- 172.21.15.153:0/708362363 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f667c06e460 0x7f667c070920 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6688007900 tx=0x7f6688008040 comp rx=0 tx=0).stop 2024-03-21T10:27:01.315 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.313+0000 7f669d600700 1 -- 172.21.15.153:0/708362363 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6698075be0 msgr2=0x7f669819d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:01.315 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.313+0000 7f669d600700 1 --2- 172.21.15.153:0/708362363 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6698075be0 0x7f669819d3e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f6684006010 tx=0x7f66840056a0 comp rx=0 tx=0).stop 2024-03-21T10:27:01.315 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.313+0000 7f669d600700 1 -- 172.21.15.153:0/708362363 shutdown_connections 2024-03-21T10:27:01.315 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.313+0000 7f669d600700 1 --2- 172.21.15.153:0/708362363 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66980752c0 0x7f669819cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:01.316 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.313+0000 7f669d600700 1 --2- 172.21.15.153:0/708362363 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f667c06e460 0x7f667c070920 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:01.316 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.313+0000 7f669d600700 1 --2- 172.21.15.153:0/708362363 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6698075be0 0x7f669819d3e0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:01.316 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.313+0000 7f669d600700 1 -- 172.21.15.153:0/708362363 >> 172.21.15.153:0/708362363 conn(0x7f66980fe870 msgr2=0x7f6698100560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:01.316 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.313+0000 7f669d600700 1 -- 172.21.15.153:0/708362363 shutdown_connections 2024-03-21T10:27:01.316 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.313+0000 7f669d600700 1 -- 172.21.15.153:0/708362363 wait complete. 2024-03-21T10:27:01.316 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 30 2024-03-21T10:27:01.459 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.458+0000 7f7b71c00700 1 -- 172.21.15.153:0/1483579412 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7b6c103000 msgr2=0x7f7b6c1033e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:01.459 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.458+0000 7f7b71c00700 1 --2- 172.21.15.153:0/1483579412 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7b6c103000 0x7f7b6c1033e0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c009910 tx=0x7f7b5c009c20 comp rx=0 tx=0).stop 2024-03-21T10:27:01.459 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.458+0000 7f7b71c00700 1 -- 172.21.15.153:0/1483579412 shutdown_connections 2024-03-21T10:27:01.459 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.458+0000 7f7b71c00700 1 --2- 172.21.15.153:0/1483579412 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7b6c1039b0 0x7f7b6c107a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:01.459 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.458+0000 7f7b71c00700 1 --2- 172.21.15.153:0/1483579412 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7b6c103000 0x7f7b6c1033e0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:01.459 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.458+0000 7f7b71c00700 1 -- 172.21.15.153:0/1483579412 >> 172.21.15.153:0/1483579412 conn(0x7f7b6c0fe8d0 msgr2=0x7f7b6c100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:01.459 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.458+0000 7f7b71c00700 1 -- 172.21.15.153:0/1483579412 shutdown_connections 2024-03-21T10:27:01.459 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.458+0000 7f7b71c00700 1 -- 172.21.15.153:0/1483579412 wait complete. 2024-03-21T10:27:01.460 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.459+0000 7f7b71c00700 1 Processor -- start 2024-03-21T10:27:01.460 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.459+0000 7f7b71c00700 1 -- start start 2024-03-21T10:27:01.460 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.460+0000 7f7b71c00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7b6c103000 0x7f7b6c19e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:01.461 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.460+0000 7f7b71c00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7b6c1039b0 0x7f7b6c19ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:01.461 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.460+0000 7f7b71c00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b6c19f3c0 con 0x7f7b6c1039b0 2024-03-21T10:27:01.461 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.460+0000 7f7b71c00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b6c198870 con 0x7f7b6c103000 2024-03-21T10:27:01.462 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.460+0000 7f7b6a000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7b6c1039b0 0x7f7b6c19ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:01.462 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.460+0000 7f7b6a000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7b6c1039b0 0x7f7b6c19ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:52704/0 (socket says 172.21.15.153:52704) 2024-03-21T10:27:01.462 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.460+0000 7f7b6a000700 1 -- 172.21.15.153:0/1604762865 learned_addr learned my addr 172.21.15.153:0/1604762865 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:27:01.462 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.461+0000 7f7b6aa00700 1 --2- 172.21.15.153:0/1604762865 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7b6c103000 0x7f7b6c19e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:01.462 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.461+0000 7f7b6a000700 1 -- 172.21.15.153:0/1604762865 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7b6c103000 msgr2=0x7f7b6c19e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:01.462 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.461+0000 7f7b6a000700 1 --2- 172.21.15.153:0/1604762865 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7b6c103000 0x7f7b6c19e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:01.462 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.461+0000 7f7b6a000700 1 -- 172.21.15.153:0/1604762865 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b5c0095f0 con 0x7f7b6c1039b0 2024-03-21T10:27:01.462 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.461+0000 7f7b6a000700 1 --2- 172.21.15.153:0/1604762865 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7b6c1039b0 0x7f7b6c19ed30 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f7b5800c8f0 tx=0x7f7b5800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:01.462 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.461+0000 7f7b63400700 1 -- 172.21.15.153:0/1604762865 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7b58007a50 con 0x7f7b6c1039b0 2024-03-21T10:27:01.463 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.461+0000 7f7b63400700 1 -- 172.21.15.153:0/1604762865 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7b58007bb0 con 0x7f7b6c1039b0 2024-03-21T10:27:01.463 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.461+0000 7f7b71c00700 1 -- 172.21.15.153:0/1604762865 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b6c198b50 con 0x7f7b6c1039b0 2024-03-21T10:27:01.463 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.461+0000 7f7b63400700 1 -- 172.21.15.153:0/1604762865 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7b58018630 con 0x7f7b6c1039b0 2024-03-21T10:27:01.463 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.461+0000 7f7b71c00700 1 -- 172.21.15.153:0/1604762865 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b6c1990a0 con 0x7f7b6c1039b0 2024-03-21T10:27:01.463 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.462+0000 7f7b71c00700 1 -- 172.21.15.153:0/1604762865 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b6c10b3a0 con 0x7f7b6c1039b0 2024-03-21T10:27:01.464 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.463+0000 7f7b63400700 1 -- 172.21.15.153:0/1604762865 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f7b58007d20 con 0x7f7b6c1039b0 2024-03-21T10:27:01.465 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.465+0000 7f7b63400700 1 --2- 172.21.15.153:0/1604762865 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f7b5006e460 0x7f7b50070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:01.465 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.465+0000 7f7b63400700 1 -- 172.21.15.153:0/1604762865 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f7b5808e550 con 0x7f7b6c1039b0 2024-03-21T10:27:01.466 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.465+0000 7f7b6aa00700 1 --2- 172.21.15.153:0/1604762865 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f7b5006e460 0x7f7b50070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:01.466 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.466+0000 7f7b6aa00700 1 --2- 172.21.15.153:0/1604762865 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f7b5006e460 0x7f7b50070920 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c00b5c0 tx=0x7f7b5c005960 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:01.470 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.469+0000 7f7b63400700 1 -- 172.21.15.153:0/1604762865 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7b5805b0a0 con 0x7f7b6c1039b0 2024-03-21T10:27:01.707 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.706+0000 7f7b71c00700 1 -- 172.21.15.153:0/1604762865 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7b6c199320 con 0x7f7b5006e460 2024-03-21T10:27:01.709 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.708+0000 7f7b63400700 1 -- 172.21.15.153:0/1604762865 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f7b6c199320 con 0x7f7b5006e460 2024-03-21T10:27:01.709 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:27:01.709 INFO:teuthology.orchestra.run.smithi153.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232", 2024-03-21T10:27:01.709 INFO:teuthology.orchestra.run.smithi153.stdout: "in_progress": true, 2024-03-21T10:27:01.709 INFO:teuthology.orchestra.run.smithi153.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-21T10:27:01.709 INFO:teuthology.orchestra.run.smithi153.stdout: "services_complete": [ 2024-03-21T10:27:01.710 INFO:teuthology.orchestra.run.smithi153.stdout: "crash", 2024-03-21T10:27:01.710 INFO:teuthology.orchestra.run.smithi153.stdout: "mon", 2024-03-21T10:27:01.710 INFO:teuthology.orchestra.run.smithi153.stdout: "osd", 2024-03-21T10:27:01.710 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr" 2024-03-21T10:27:01.710 INFO:teuthology.orchestra.run.smithi153.stdout: ], 2024-03-21T10:27:01.710 INFO:teuthology.orchestra.run.smithi153.stdout: "progress": "15/23 daemons upgraded", 2024-03-21T10:27:01.710 INFO:teuthology.orchestra.run.smithi153.stdout: "message": "Currently upgrading mds daemons", 2024-03-21T10:27:01.710 INFO:teuthology.orchestra.run.smithi153.stdout: "is_paused": false 2024-03-21T10:27:01.710 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:27:01.712 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.712+0000 7f7b71c00700 1 -- 172.21.15.153:0/1604762865 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f7b5006e460 msgr2=0x7f7b50070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:01.712 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.712+0000 7f7b71c00700 1 --2- 172.21.15.153:0/1604762865 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f7b5006e460 0x7f7b50070920 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c00b5c0 tx=0x7f7b5c005960 comp rx=0 tx=0).stop 2024-03-21T10:27:01.712 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.712+0000 7f7b71c00700 1 -- 172.21.15.153:0/1604762865 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7b6c1039b0 msgr2=0x7f7b6c19ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:01.712 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.712+0000 7f7b71c00700 1 --2- 172.21.15.153:0/1604762865 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7b6c1039b0 0x7f7b6c19ed30 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f7b5800c8f0 tx=0x7f7b5800ccb0 comp rx=0 tx=0).stop 2024-03-21T10:27:01.713 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.712+0000 7f7b71c00700 1 -- 172.21.15.153:0/1604762865 shutdown_connections 2024-03-21T10:27:01.713 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.712+0000 7f7b71c00700 1 --2- 172.21.15.153:0/1604762865 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7b6c103000 0x7f7b6c19e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:01.713 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.712+0000 7f7b71c00700 1 --2- 172.21.15.153:0/1604762865 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f7b5006e460 0x7f7b50070920 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:01.713 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.712+0000 7f7b71c00700 1 --2- 172.21.15.153:0/1604762865 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7b6c1039b0 0x7f7b6c19ed30 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:01.713 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.712+0000 7f7b71c00700 1 -- 172.21.15.153:0/1604762865 >> 172.21.15.153:0/1604762865 conn(0x7f7b6c0fe8d0 msgr2=0x7f7b6c0ffee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:01.714 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.712+0000 7f7b71c00700 1 -- 172.21.15.153:0/1604762865 shutdown_connections 2024-03-21T10:27:01.714 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.712+0000 7f7b71c00700 1 -- 172.21.15.153:0/1604762865 wait complete. 2024-03-21T10:27:01.876 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.874+0000 7fbab9c00700 1 -- 172.21.15.153:0/1766108314 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbab4107220 msgr2=0x7fbab4107600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:01.876 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.874+0000 7fbab9c00700 1 --2- 172.21.15.153:0/1766108314 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbab4107220 0x7fbab4107600 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fbaa4009910 tx=0x7fbaa4009c20 comp rx=0 tx=0).stop 2024-03-21T10:27:01.876 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.875+0000 7fbab9c00700 1 -- 172.21.15.153:0/1766108314 shutdown_connections 2024-03-21T10:27:01.876 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.875+0000 7fbab9c00700 1 --2- 172.21.15.153:0/1766108314 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbab4107bd0 0x7fbab410bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:01.876 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.875+0000 7fbab9c00700 1 --2- 172.21.15.153:0/1766108314 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbab4107220 0x7fbab4107600 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:01.876 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.875+0000 7fbab9c00700 1 -- 172.21.15.153:0/1766108314 >> 172.21.15.153:0/1766108314 conn(0x7fbab40767e0 msgr2=0x7fbab4078c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:01.876 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.875+0000 7fbab9c00700 1 -- 172.21.15.153:0/1766108314 shutdown_connections 2024-03-21T10:27:01.876 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.875+0000 7fbab9c00700 1 -- 172.21.15.153:0/1766108314 wait complete. 2024-03-21T10:27:01.877 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.876+0000 7fbab9c00700 1 Processor -- start 2024-03-21T10:27:01.877 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.876+0000 7fbab9c00700 1 -- start start 2024-03-21T10:27:01.878 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.876+0000 7fbab9c00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbab4107220 0x7fbab41a2a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:01.878 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.876+0000 7fbab9c00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbab4107bd0 0x7fbab41a2f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:01.878 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.876+0000 7fbab9c00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbab41a3610 con 0x7fbab4107bd0 2024-03-21T10:27:01.878 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.876+0000 7fbab9c00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbab419cac0 con 0x7fbab4107220 2024-03-21T10:27:01.878 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.877+0000 7fbab3400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbab4107bd0 0x7fbab41a2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:01.878 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.877+0000 7fbab3400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbab4107bd0 0x7fbab41a2f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:52728/0 (socket says 172.21.15.153:52728) 2024-03-21T10:27:01.878 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.877+0000 7fbab3400700 1 -- 172.21.15.153:0/2260918893 learned_addr learned my addr 172.21.15.153:0/2260918893 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:27:01.878 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.877+0000 7fbab3e00700 1 --2- 172.21.15.153:0/2260918893 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbab4107220 0x7fbab41a2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:01.879 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.877+0000 7fbab3400700 1 -- 172.21.15.153:0/2260918893 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbab4107220 msgr2=0x7fbab41a2a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:01.879 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.877+0000 7fbab3400700 1 --2- 172.21.15.153:0/2260918893 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbab4107220 0x7fbab41a2a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:01.879 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.877+0000 7fbab3400700 1 -- 172.21.15.153:0/2260918893 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaa40095f0 con 0x7fbab4107bd0 2024-03-21T10:27:01.879 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.878+0000 7fbab3400700 1 --2- 172.21.15.153:0/2260918893 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbab4107bd0 0x7fbab41a2f80 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fbaa000e970 tx=0x7fbaa000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:01.879 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.878+0000 7fbab0c00700 1 -- 172.21.15.153:0/2260918893 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbaa0009750 con 0x7fbab4107bd0 2024-03-21T10:27:01.879 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.878+0000 7fbab0c00700 1 -- 172.21.15.153:0/2260918893 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbaa0004cf0 con 0x7fbab4107bd0 2024-03-21T10:27:01.879 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.878+0000 7fbab0c00700 1 -- 172.21.15.153:0/2260918893 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbaa0005660 con 0x7fbab4107bd0 2024-03-21T10:27:01.880 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.878+0000 7fbab9c00700 1 -- 172.21.15.153:0/2260918893 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbab419cda0 con 0x7fbab4107bd0 2024-03-21T10:27:01.880 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.878+0000 7fbab9c00700 1 -- 172.21.15.153:0/2260918893 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbab419d2f0 con 0x7fbab4107bd0 2024-03-21T10:27:01.880 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.879+0000 7fbab9c00700 1 -- 172.21.15.153:0/2260918893 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbab410f5b0 con 0x7fbab4107bd0 2024-03-21T10:27:01.881 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.880+0000 7fbab0c00700 1 -- 172.21.15.153:0/2260918893 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fbaa0004750 con 0x7fbab4107bd0 2024-03-21T10:27:01.882 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.882+0000 7fbab0c00700 1 --2- 172.21.15.153:0/2260918893 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fba9806e460 0x7fba98070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:01.882 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.882+0000 7fbab0c00700 1 -- 172.21.15.153:0/2260918893 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fbaa0014070 con 0x7fbab4107bd0 2024-03-21T10:27:01.882 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.882+0000 7fbab3e00700 1 --2- 172.21.15.153:0/2260918893 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fba9806e460 0x7fba98070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:01.883 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.883+0000 7fbab3e00700 1 --2- 172.21.15.153:0/2260918893 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fba9806e460 0x7fba98070920 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fbaa400b760 tx=0x7fbaa4009f90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:01.887 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:01.886+0000 7fbab0c00700 1 -- 172.21.15.153:0/2260918893 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fbaa005b470 con 0x7fbab4107bd0 2024-03-21T10:27:02.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:27:01 smithi153 ceph-mon[113733]: from='client.34850 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:27:02.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:27:01 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/1120268557' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:27:02.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:27:01 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:27:02.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:27:01 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/708362363' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:27:02.183 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:02.182+0000 7fbab9c00700 1 -- 172.21.15.153:0/2260918893 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fbab404f2e0 con 0x7fbab4107bd0 2024-03-21T10:27:02.184 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:02.183+0000 7fbab0c00700 1 -- 172.21.15.153:0/2260918893 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7fbaa005b000 con 0x7fbab4107bd0 2024-03-21T10:27:02.184 INFO:teuthology.orchestra.run.smithi153.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-03-21T10:27:02.184 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-21T10:27:02.184 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is degraded 2024-03-21T10:27:02.184 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-21T10:27:02.184 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-21T10:27:02.184 INFO:teuthology.orchestra.run.smithi153.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-21T10:27:02.184 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-21T10:27:02.187 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:02.186+0000 7fbab9c00700 1 -- 172.21.15.153:0/2260918893 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fba9806e460 msgr2=0x7fba98070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:02.187 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:02.186+0000 7fbab9c00700 1 --2- 172.21.15.153:0/2260918893 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fba9806e460 0x7fba98070920 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fbaa400b760 tx=0x7fbaa4009f90 comp rx=0 tx=0).stop 2024-03-21T10:27:02.187 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:02.186+0000 7fbab9c00700 1 -- 172.21.15.153:0/2260918893 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbab4107bd0 msgr2=0x7fbab41a2f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:02.187 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:02.186+0000 7fbab9c00700 1 --2- 172.21.15.153:0/2260918893 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbab4107bd0 0x7fbab41a2f80 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fbaa000e970 tx=0x7fbaa000ed30 comp rx=0 tx=0).stop 2024-03-21T10:27:02.187 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:02.186+0000 7fbab9c00700 1 -- 172.21.15.153:0/2260918893 shutdown_connections 2024-03-21T10:27:02.187 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:02.186+0000 7fbab9c00700 1 --2- 172.21.15.153:0/2260918893 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbab4107220 0x7fbab41a2a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:02.187 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:02.186+0000 7fbab9c00700 1 --2- 172.21.15.153:0/2260918893 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fba9806e460 0x7fba98070920 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:02.187 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:02.186+0000 7fbab9c00700 1 --2- 172.21.15.153:0/2260918893 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbab4107bd0 0x7fbab41a2f80 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:02.187 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:02.186+0000 7fbab9c00700 1 -- 172.21.15.153:0/2260918893 >> 172.21.15.153:0/2260918893 conn(0x7fbab40767e0 msgr2=0x7fbab4077e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:02.187 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:02.186+0000 7fbab9c00700 1 -- 172.21.15.153:0/2260918893 shutdown_connections 2024-03-21T10:27:02.188 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:02.186+0000 7fbab9c00700 1 -- 172.21.15.153:0/2260918893 wait complete. 2024-03-21T10:27:02.200 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:27:01 smithi174 ceph-mon[88004]: from='client.34850 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:27:02.200 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:27:01 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/1120268557' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:27:02.200 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:27:01 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:27:02.200 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:27:01 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/708362363' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:27:03.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:27:02 smithi153 ceph-mon[113733]: pgmap v455: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:27:03.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:27:02 smithi153 ceph-mon[113733]: from='client.34862 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:27:03.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:27:02 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/2260918893' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:27:03.200 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:27:02 smithi174 ceph-mon[88004]: pgmap v455: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:27:03.200 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:27:02 smithi174 ceph-mon[88004]: from='client.34862 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:27:03.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:27:02 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/2260918893' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:27:05.200 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:27:04 smithi174 ceph-mon[88004]: pgmap v456: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:27:05.305 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:27:04 smithi153 ceph-mon[113733]: pgmap v456: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:27:07.181 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:27:06 smithi174 ceph-mon[88004]: pgmap v457: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:27:07.305 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:27:06 smithi153 ceph-mon[113733]: pgmap v457: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:27:09.200 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:27:08 smithi174 ceph-mon[88004]: pgmap v458: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:27:09.305 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:27:08 smithi153 ceph-mon[113733]: pgmap v458: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:27:11.200 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:27:10 smithi174 ceph-mon[88004]: pgmap v459: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:27:11.305 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:27:10 smithi153 ceph-mon[113733]: pgmap v459: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:27:13.055 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:27:12 smithi153 ceph-mon[113733]: pgmap v460: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:27:13.200 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:27:12 smithi174 ceph-mon[88004]: pgmap v460: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:27:15.200 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:27:14 smithi174 ceph-mon[88004]: pgmap v461: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:27:15.305 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:27:14 smithi153 ceph-mon[113733]: pgmap v461: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:27:17.200 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:27:16 smithi174 ceph-mon[88004]: pgmap v462: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:27:17.200 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:27:16 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:27:17.305 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:27:16 smithi153 ceph-mon[113733]: pgmap v462: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:27:17.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:27:16 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:27:19.200 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:27:18 smithi174 ceph-mon[88004]: pgmap v463: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:27:19.305 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:27:18 smithi153 ceph-mon[113733]: pgmap v463: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:27:21.200 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:27:20 smithi174 ceph-mon[88004]: pgmap v464: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:27:21.305 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:27:20 smithi153 ceph-mon[113733]: pgmap v464: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:27:22.950 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:27:22 smithi174 ceph-mon[88004]: pgmap v465: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:27:23.055 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:27:22 smithi153 ceph-mon[113733]: pgmap v465: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:27:25.200 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:27:24 smithi174 ceph-mon[88004]: pgmap v466: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:27:25.305 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:27:24 smithi153 ceph-mon[113733]: pgmap v466: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:27:27.200 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:27:26 smithi174 ceph-mon[88004]: pgmap v467: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:27:27.305 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:27:26 smithi153 ceph-mon[113733]: pgmap v467: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:27:29.200 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:27:28 smithi174 ceph-mon[88004]: pgmap v468: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:27:29.305 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:27:28 smithi153 ceph-mon[113733]: pgmap v468: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:27:31.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:27:30 smithi174 ceph-mon[88004]: pgmap v469: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:27:31.305 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:27:30 smithi153 ceph-mon[113733]: pgmap v469: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:27:32.200 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:27:31 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:27:32.305 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:27:31 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:27:32.358 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:32.358+0000 7f8f25e00700 1 -- 172.21.15.153:0/1258892373 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f20073160 msgr2=0x7f8f20073540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:32.358 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:32.358+0000 7f8f25e00700 1 --2- 172.21.15.153:0/1258892373 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f20073160 0x7f8f20073540 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f8f10009910 tx=0x7f8f10009c20 comp rx=0 tx=0).stop 2024-03-21T10:27:32.358 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:32.358+0000 7f8f25e00700 1 -- 172.21.15.153:0/1258892373 shutdown_connections 2024-03-21T10:27:32.358 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:32.358+0000 7f8f25e00700 1 --2- 172.21.15.153:0/1258892373 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8f20073a80 0x7f8f2010cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:32.358 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:32.358+0000 7f8f25e00700 1 --2- 172.21.15.153:0/1258892373 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f20073160 0x7f8f20073540 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:32.358 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:32.358+0000 7f8f25e00700 1 -- 172.21.15.153:0/1258892373 >> 172.21.15.153:0/1258892373 conn(0x7f8f200fc650 msgr2=0x7f8f200fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:32.358 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:32.358+0000 7f8f25e00700 1 -- 172.21.15.153:0/1258892373 shutdown_connections 2024-03-21T10:27:32.359 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:32.358+0000 7f8f25e00700 1 -- 172.21.15.153:0/1258892373 wait complete. 2024-03-21T10:27:32.359 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:32.359+0000 7f8f25e00700 1 Processor -- start 2024-03-21T10:27:32.359 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:32.360+0000 7f8f25e00700 1 -- start start 2024-03-21T10:27:32.360 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:32.360+0000 7f8f25e00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8f20073160 0x7f8f2019e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:32.360 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:32.360+0000 7f8f25e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f20073a80 0x7f8f2019ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:32.360 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:32.360+0000 7f8f25e00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f2019f2b0 con 0x7f8f20073a80 2024-03-21T10:27:32.360 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:32.360+0000 7f8f25e00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f20198760 con 0x7f8f20073160 2024-03-21T10:27:32.360 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:32.360+0000 7f8f1e000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f20073a80 0x7f8f2019ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:32.361 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:32.360+0000 7f8f1e000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f20073a80 0x7f8f2019ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:52392/0 (socket says 172.21.15.153:52392) 2024-03-21T10:27:32.361 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:32.360+0000 7f8f1e000700 1 -- 172.21.15.153:0/3957208891 learned_addr learned my addr 172.21.15.153:0/3957208891 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:27:32.361 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:32.361+0000 7f8f1ea00700 1 --2- 172.21.15.153:0/3957208891 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8f20073160 0x7f8f2019e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:32.361 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:32.361+0000 7f8f1e000700 1 -- 172.21.15.153:0/3957208891 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8f20073160 msgr2=0x7f8f2019e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:32.361 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:32.361+0000 7f8f1e000700 1 --2- 172.21.15.153:0/3957208891 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8f20073160 0x7f8f2019e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:32.361 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:32.361+0000 7f8f1e000700 1 -- 172.21.15.153:0/3957208891 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f100095f0 con 0x7f8f20073a80 2024-03-21T10:27:32.362 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:32.361+0000 7f8f1e000700 1 --2- 172.21.15.153:0/3957208891 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f20073a80 0x7f8f2019ec20 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c00c8a0 tx=0x7f8f0c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:32.362 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:32.362+0000 7f8f24a00700 1 -- 172.21.15.153:0/3957208891 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8f0c00cea0 con 0x7f8f20073a80 2024-03-21T10:27:32.362 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:32.362+0000 7f8f24a00700 1 -- 172.21.15.153:0/3957208891 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8f0c004830 con 0x7f8f20073a80 2024-03-21T10:27:32.362 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:32.362+0000 7f8f24a00700 1 -- 172.21.15.153:0/3957208891 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8f0c0055b0 con 0x7f8f20073a80 2024-03-21T10:27:32.362 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:32.362+0000 7f8f25e00700 1 -- 172.21.15.153:0/3957208891 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f20198a40 con 0x7f8f20073a80 2024-03-21T10:27:32.362 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:32.362+0000 7f8f25e00700 1 -- 172.21.15.153:0/3957208891 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f20198f90 con 0x7f8f20073a80 2024-03-21T10:27:32.362 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:32.363+0000 7f8f25e00700 1 -- 172.21.15.153:0/3957208891 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f2010a5a0 con 0x7f8f20073a80 2024-03-21T10:27:32.363 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:32.364+0000 7f8f24a00700 1 -- 172.21.15.153:0/3957208891 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f8f0c0049a0 con 0x7f8f20073a80 2024-03-21T10:27:32.364 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:32.365+0000 7f8f24a00700 1 --2- 172.21.15.153:0/3957208891 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f8f0406e460 0x7f8f04070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:32.364 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:32.365+0000 7f8f24a00700 1 -- 172.21.15.153:0/3957208891 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f8f0c08e270 con 0x7f8f20073a80 2024-03-21T10:27:32.364 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:32.365+0000 7f8f1ea00700 1 --2- 172.21.15.153:0/3957208891 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f8f0406e460 0x7f8f04070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:32.365 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:32.366+0000 7f8f1ea00700 1 --2- 172.21.15.153:0/3957208891 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f8f0406e460 0x7f8f04070920 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f8f1000b5c0 tx=0x7f8f1001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:32.369 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:32.370+0000 7f8f24a00700 1 -- 172.21.15.153:0/3957208891 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f8f0c05adc0 con 0x7f8f20073a80 2024-03-21T10:27:32.612 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:32.612+0000 7f8f25e00700 1 -- 172.21.15.153:0/3957208891 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8f20199280 con 0x7f8f0406e460 2024-03-21T10:27:32.614 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:32.615+0000 7f8f24a00700 1 -- 172.21.15.153:0/3957208891 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f8f20199280 con 0x7f8f0406e460 2024-03-21T10:27:32.618 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:32.617+0000 7f8f25e00700 1 -- 172.21.15.153:0/3957208891 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f8f0406e460 msgr2=0x7f8f04070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:32.618 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:32.617+0000 7f8f25e00700 1 --2- 172.21.15.153:0/3957208891 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f8f0406e460 0x7f8f04070920 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f8f1000b5c0 tx=0x7f8f1001a040 comp rx=0 tx=0).stop 2024-03-21T10:27:32.618 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:32.617+0000 7f8f25e00700 1 -- 172.21.15.153:0/3957208891 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f20073a80 msgr2=0x7f8f2019ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:32.618 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:32.618+0000 7f8f25e00700 1 --2- 172.21.15.153:0/3957208891 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f20073a80 0x7f8f2019ec20 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c00c8a0 tx=0x7f8f0c00cc60 comp rx=0 tx=0).stop 2024-03-21T10:27:32.618 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:32.618+0000 7f8f25e00700 1 -- 172.21.15.153:0/3957208891 shutdown_connections 2024-03-21T10:27:32.618 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:32.618+0000 7f8f25e00700 1 --2- 172.21.15.153:0/3957208891 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8f20073160 0x7f8f2019e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:32.618 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:32.618+0000 7f8f25e00700 1 --2- 172.21.15.153:0/3957208891 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f8f0406e460 0x7f8f04070920 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:32.618 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:32.618+0000 7f8f25e00700 1 --2- 172.21.15.153:0/3957208891 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f20073a80 0x7f8f2019ec20 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:32.618 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:32.618+0000 7f8f25e00700 1 -- 172.21.15.153:0/3957208891 >> 172.21.15.153:0/3957208891 conn(0x7f8f200fc650 msgr2=0x7f8f201076e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:32.618 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:32.618+0000 7f8f25e00700 1 -- 172.21.15.153:0/3957208891 shutdown_connections 2024-03-21T10:27:32.618 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:32.618+0000 7f8f25e00700 1 -- 172.21.15.153:0/3957208891 wait complete. 2024-03-21T10:27:32.637 INFO:teuthology.orchestra.run.smithi153.stdout:true 2024-03-21T10:27:32.778 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:32.778+0000 7fe171e00700 1 -- 172.21.15.153:0/4283753216 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe16c103000 msgr2=0x7fe16c1033e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:32.778 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:32.778+0000 7fe171e00700 1 --2- 172.21.15.153:0/4283753216 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe16c103000 0x7fe16c1033e0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fe15c009910 tx=0x7fe15c009c20 comp rx=0 tx=0).stop 2024-03-21T10:27:32.778 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:32.778+0000 7fe171e00700 1 -- 172.21.15.153:0/4283753216 shutdown_connections 2024-03-21T10:27:32.778 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:32.778+0000 7fe171e00700 1 --2- 172.21.15.153:0/4283753216 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe16c1039b0 0x7fe16c107a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:32.779 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:32.778+0000 7fe171e00700 1 --2- 172.21.15.153:0/4283753216 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe16c103000 0x7fe16c1033e0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:32.779 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:32.778+0000 7fe171e00700 1 -- 172.21.15.153:0/4283753216 >> 172.21.15.153:0/4283753216 conn(0x7fe16c0fe8d0 msgr2=0x7fe16c100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:32.779 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:32.778+0000 7fe171e00700 1 -- 172.21.15.153:0/4283753216 shutdown_connections 2024-03-21T10:27:32.779 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:32.778+0000 7fe171e00700 1 -- 172.21.15.153:0/4283753216 wait complete. 2024-03-21T10:27:32.782 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:32.779+0000 7fe171e00700 1 Processor -- start 2024-03-21T10:27:32.782 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:32.779+0000 7fe171e00700 1 -- start start 2024-03-21T10:27:32.782 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:32.780+0000 7fe171e00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe16c103000 0x7fe16c19e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:32.782 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:32.780+0000 7fe171e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe16c1039b0 0x7fe16c19ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:32.782 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:32.780+0000 7fe171e00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe16c19f3c0 con 0x7fe16c1039b0 2024-03-21T10:27:32.783 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:32.780+0000 7fe171e00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe16c198870 con 0x7fe16c103000 2024-03-21T10:27:32.783 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:32.780+0000 7fe16a000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe16c1039b0 0x7fe16c19ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:32.783 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:32.780+0000 7fe16a000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe16c1039b0 0x7fe16c19ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:52402/0 (socket says 172.21.15.153:52402) 2024-03-21T10:27:32.783 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:32.780+0000 7fe16a000700 1 -- 172.21.15.153:0/4058753998 learned_addr learned my addr 172.21.15.153:0/4058753998 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:27:32.783 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:32.780+0000 7fe16aa00700 1 --2- 172.21.15.153:0/4058753998 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe16c103000 0x7fe16c19e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:32.783 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:32.780+0000 7fe16a000700 1 -- 172.21.15.153:0/4058753998 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe16c103000 msgr2=0x7fe16c19e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:32.784 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:32.780+0000 7fe16a000700 1 --2- 172.21.15.153:0/4058753998 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe16c103000 0x7fe16c19e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:32.784 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:32.780+0000 7fe16a000700 1 -- 172.21.15.153:0/4058753998 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe15c0095f0 con 0x7fe16c1039b0 2024-03-21T10:27:32.784 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:32.781+0000 7fe16a000700 1 --2- 172.21.15.153:0/4058753998 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe16c1039b0 0x7fe16c19ed30 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fe15800c8f0 tx=0x7fe15800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:32.784 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:32.781+0000 7fe170a00700 1 -- 172.21.15.153:0/4058753998 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe158007a50 con 0x7fe16c1039b0 2024-03-21T10:27:32.784 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:32.781+0000 7fe170a00700 1 -- 172.21.15.153:0/4058753998 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe158007bb0 con 0x7fe16c1039b0 2024-03-21T10:27:32.784 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:32.781+0000 7fe170a00700 1 -- 172.21.15.153:0/4058753998 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe158018630 con 0x7fe16c1039b0 2024-03-21T10:27:32.784 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:32.781+0000 7fe171e00700 1 -- 172.21.15.153:0/4058753998 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe16c198b50 con 0x7fe16c1039b0 2024-03-21T10:27:32.785 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:32.781+0000 7fe171e00700 1 -- 172.21.15.153:0/4058753998 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe16c1990a0 con 0x7fe16c1039b0 2024-03-21T10:27:32.785 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:32.782+0000 7fe171e00700 1 -- 172.21.15.153:0/4058753998 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe16c10b3a0 con 0x7fe16c1039b0 2024-03-21T10:27:32.786 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:32.784+0000 7fe170a00700 1 -- 172.21.15.153:0/4058753998 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fe158007d20 con 0x7fe16c1039b0 2024-03-21T10:27:32.786 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:32.784+0000 7fe170a00700 1 --2- 172.21.15.153:0/4058753998 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe15006e460 0x7fe150070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:32.786 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:32.785+0000 7fe170a00700 1 -- 172.21.15.153:0/4058753998 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fe15808e450 con 0x7fe16c1039b0 2024-03-21T10:27:32.786 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:32.785+0000 7fe16aa00700 1 --2- 172.21.15.153:0/4058753998 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe15006e460 0x7fe150070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:32.786 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:32.785+0000 7fe16aa00700 1 --2- 172.21.15.153:0/4058753998 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe15006e460 0x7fe150070920 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe15c006010 tx=0x7fe15c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:32.788 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:32.789+0000 7fe170a00700 1 -- 172.21.15.153:0/4058753998 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe15805b020 con 0x7fe16c1039b0 2024-03-21T10:27:33.034 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:33.034+0000 7fe171e00700 1 -- 172.21.15.153:0/4058753998 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe16c1993e0 con 0x7fe15006e460 2024-03-21T10:27:33.037 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:33.037+0000 7fe170a00700 1 -- 172.21.15.153:0/4058753998 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fe16c1993e0 con 0x7fe15006e460 2024-03-21T10:27:33.040 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:33.040+0000 7fe171e00700 1 -- 172.21.15.153:0/4058753998 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe15006e460 msgr2=0x7fe150070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:33.040 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:33.040+0000 7fe171e00700 1 --2- 172.21.15.153:0/4058753998 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe15006e460 0x7fe150070920 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe15c006010 tx=0x7fe15c01a040 comp rx=0 tx=0).stop 2024-03-21T10:27:33.041 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:33.040+0000 7fe171e00700 1 -- 172.21.15.153:0/4058753998 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe16c1039b0 msgr2=0x7fe16c19ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:33.041 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:33.040+0000 7fe171e00700 1 --2- 172.21.15.153:0/4058753998 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe16c1039b0 0x7fe16c19ed30 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fe15800c8f0 tx=0x7fe15800ccb0 comp rx=0 tx=0).stop 2024-03-21T10:27:33.041 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:33.040+0000 7fe171e00700 1 -- 172.21.15.153:0/4058753998 shutdown_connections 2024-03-21T10:27:33.041 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:33.040+0000 7fe171e00700 1 --2- 172.21.15.153:0/4058753998 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe16c103000 0x7fe16c19e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:33.041 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:33.040+0000 7fe171e00700 1 --2- 172.21.15.153:0/4058753998 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe15006e460 0x7fe150070920 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:33.041 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:33.040+0000 7fe171e00700 1 --2- 172.21.15.153:0/4058753998 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe16c1039b0 0x7fe16c19ed30 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:33.041 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:33.040+0000 7fe171e00700 1 -- 172.21.15.153:0/4058753998 >> 172.21.15.153:0/4058753998 conn(0x7fe16c0fe8d0 msgr2=0x7fe16c0ffee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:33.041 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:33.040+0000 7fe171e00700 1 -- 172.21.15.153:0/4058753998 shutdown_connections 2024-03-21T10:27:33.042 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:33.040+0000 7fe171e00700 1 -- 172.21.15.153:0/4058753998 wait complete. 2024-03-21T10:27:33.192 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:33.192+0000 7f8ea8200700 1 -- 172.21.15.153:0/4158946257 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8ea0068950 msgr2=0x7f8ea0068dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:33.192 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:33.192+0000 7f8ea8200700 1 --2- 172.21.15.153:0/4158946257 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8ea0068950 0x7f8ea0068dd0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f8e8c009910 tx=0x7f8e8c009c20 comp rx=0 tx=0).stop 2024-03-21T10:27:33.192 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:33.193+0000 7f8ea8200700 1 -- 172.21.15.153:0/4158946257 shutdown_connections 2024-03-21T10:27:33.192 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:33.193+0000 7f8ea8200700 1 --2- 172.21.15.153:0/4158946257 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8ea0068950 0x7f8ea0068dd0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:33.192 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:33.193+0000 7f8ea8200700 1 --2- 172.21.15.153:0/4158946257 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8ea0105b50 0x7f8ea0068410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:33.193 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:33.193+0000 7f8ea8200700 1 -- 172.21.15.153:0/4158946257 >> 172.21.15.153:0/4158946257 conn(0x7f8ea00755a0 msgr2=0x7f8ea00759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:33.193 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:27:32 smithi153 ceph-mon[113733]: pgmap v470: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:27:33.194 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:33.193+0000 7f8ea8200700 1 -- 172.21.15.153:0/4158946257 shutdown_connections 2024-03-21T10:27:33.194 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:33.193+0000 7f8ea8200700 1 -- 172.21.15.153:0/4158946257 wait complete. 2024-03-21T10:27:33.194 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:33.194+0000 7f8ea8200700 1 Processor -- start 2024-03-21T10:27:33.194 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:33.194+0000 7f8ea8200700 1 -- start start 2024-03-21T10:27:33.194 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:33.194+0000 7f8ea8200700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8ea0068950 0x7f8ea01a2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:33.194 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:33.194+0000 7f8ea8200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8ea0105b50 0x7f8ea01a3010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:33.194 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:33.194+0000 7f8ea8200700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea01a36a0 con 0x7f8ea0105b50 2024-03-21T10:27:33.194 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:33.194+0000 7f8ea8200700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea019cb50 con 0x7f8ea0068950 2024-03-21T10:27:33.195 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:33.195+0000 7f8ea4e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8ea0105b50 0x7f8ea01a3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:33.195 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:33.195+0000 7f8ea4e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8ea0105b50 0x7f8ea01a3010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:53248/0 (socket says 172.21.15.153:53248) 2024-03-21T10:27:33.195 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:33.195+0000 7f8ea4e00700 1 -- 172.21.15.153:0/479050368 learned_addr learned my addr 172.21.15.153:0/479050368 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:27:33.196 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:33.195+0000 7f8ea5800700 1 --2- 172.21.15.153:0/479050368 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8ea0068950 0x7f8ea01a2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:33.196 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:33.195+0000 7f8ea4e00700 1 -- 172.21.15.153:0/479050368 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8ea0068950 msgr2=0x7f8ea01a2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:33.196 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:33.195+0000 7f8ea4e00700 1 --2- 172.21.15.153:0/479050368 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8ea0068950 0x7f8ea01a2ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:33.196 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:33.195+0000 7f8ea4e00700 1 -- 172.21.15.153:0/479050368 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e8c0095f0 con 0x7f8ea0105b50 2024-03-21T10:27:33.196 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:33.195+0000 7f8ea4e00700 1 --2- 172.21.15.153:0/479050368 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8ea0105b50 0x7f8ea01a3010 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f8e8c006010 tx=0x7f8e8c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:33.196 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:33.196+0000 7f8e9e000700 1 -- 172.21.15.153:0/479050368 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8e8c01d070 con 0x7f8ea0105b50 2024-03-21T10:27:33.196 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:33.196+0000 7f8e9e000700 1 -- 172.21.15.153:0/479050368 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8e8c004fb0 con 0x7f8ea0105b50 2024-03-21T10:27:33.196 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:33.196+0000 7f8ea8200700 1 -- 172.21.15.153:0/479050368 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ea019cdd0 con 0x7f8ea0105b50 2024-03-21T10:27:33.196 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:33.196+0000 7f8e9e000700 1 -- 172.21.15.153:0/479050368 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8e8c022ea0 con 0x7f8ea0105b50 2024-03-21T10:27:33.196 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:33.196+0000 7f8ea8200700 1 -- 172.21.15.153:0/479050368 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ea019d2c0 con 0x7f8ea0105b50 2024-03-21T10:27:33.197 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:33.197+0000 7f8ea8200700 1 -- 172.21.15.153:0/479050368 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ea004ea90 con 0x7f8ea0105b50 2024-03-21T10:27:33.197 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:33.198+0000 7f8e9e000700 1 -- 172.21.15.153:0/479050368 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f8e8c022470 con 0x7f8ea0105b50 2024-03-21T10:27:33.198 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:33.199+0000 7f8e9e000700 1 --2- 172.21.15.153:0/479050368 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f8e8806e460 0x7f8e88070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:33.199 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:33.199+0000 7f8e9e000700 1 -- 172.21.15.153:0/479050368 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f8e8c08faf0 con 0x7f8ea0105b50 2024-03-21T10:27:33.199 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:33.199+0000 7f8ea5800700 1 --2- 172.21.15.153:0/479050368 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f8e8806e460 0x7f8e88070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:33.199 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:33.200+0000 7f8ea5800700 1 --2- 172.21.15.153:0/479050368 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f8e8806e460 0x7f8e88070920 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8e94005fd0 tx=0x7f8e94005dd0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:33.200 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:27:32 smithi174 ceph-mon[88004]: pgmap v470: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:27:33.203 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:33.204+0000 7f8e9e000700 1 -- 172.21.15.153:0/479050368 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f8e8c05c730 con 0x7f8ea0105b50 2024-03-21T10:27:33.419 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:33.419+0000 7f8ea8200700 1 -- 172.21.15.153:0/479050368 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f8ea019d750 con 0x7f8e8806e460 2024-03-21T10:27:33.430 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:33.431+0000 7f8e9e000700 1 -- 172.21.15.153:0/479050368 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f8ea019d750 con 0x7f8e8806e460 2024-03-21T10:27:33.431 INFO:teuthology.orchestra.run.smithi153.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-21T10:27:33.431 INFO:teuthology.orchestra.run.smithi153.stdout:alertmanager.smithi153 smithi153 *:9093,9094 running (16m) 7m ago 30m 14.7M - 0.25.0 c8568f914cd2 9ed3f2200b5f 2024-03-21T10:27:33.431 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi153 smithi153 running (30m) 7m ago 30m 9424k - 18.2.1 5be31c24972a 5b61994396a6 2024-03-21T10:27:33.431 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi174 smithi174 running (28m) 7m ago 28m 9244k - 18.2.1 5be31c24972a 4f7a5fc3ac28 2024-03-21T10:27:33.431 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi153 smithi153 running (12m) 7m ago 30m 6656k - 19.0.0-2231-g2edfee68 237e43c7a65f 32c21104d5b0 2024-03-21T10:27:33.431 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi174 smithi174 running (12m) 7m ago 28m 6660k - 19.0.0-2231-g2edfee68 237e43c7a65f a04e48967ab6 2024-03-21T10:27:33.431 INFO:teuthology.orchestra.run.smithi153.stdout:grafana.smithi153 smithi153 *:3000 running (15m) 7m ago 29m 73.6M - 9.4.12 2bacad6d85d8 ee24026ab786 2024-03-21T10:27:33.431 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.iuyluj smithi153 running (7m) 7m ago 22m 10.9M - 19.0.0-2231-g2edfee68 237e43c7a65f 87aff8a5de70 2024-03-21T10:27:33.431 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.zipckr smithi153 running (8m) 7m ago 23m 11.8M - 19.0.0-2231-g2edfee68 237e43c7a65f ccc262bd5dd3 2024-03-21T10:27:33.431 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.nhozag smithi174 running (22m) 7m ago 22m 12.0M - 18.2.1 5be31c24972a 2fee4000308f 2024-03-21T10:27:33.431 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.yyimst smithi174 running (7m) 7m ago 22m 10.7M - 19.0.0-2231-g2edfee68 237e43c7a65f fc1962f9351e 2024-03-21T10:27:33.431 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi153.tluxni smithi153 *:8443,9283,8765 running (19m) 7m ago 31m 509M - 19.0.0-2231-g2edfee68 237e43c7a65f 7ffee0e55382 2024-03-21T10:27:33.432 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi174.qpcwat smithi174 *:8443,9283,8765 running (17m) 7m ago 28m 452M - 19.0.0-2231-g2edfee68 237e43c7a65f 748968f216ab 2024-03-21T10:27:33.432 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi153 smithi153 running (14m) 7m ago 32m 55.0M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f dd8038303dc3 2024-03-21T10:27:33.432 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi174 smithi174 running (13m) 7m ago 28m 44.2M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f 29d78a788f2c 2024-03-21T10:27:33.432 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi153 smithi153 *:9100 running (17m) 7m ago 30m 7889k - 1.5.0 0da6a335fe13 bb835edc0e0b 2024-03-21T10:27:33.432 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi174 smithi174 *:9100 running (17m) 7m ago 28m 8102k - 1.5.0 0da6a335fe13 7c4ecad790fa 2024-03-21T10:27:33.432 INFO:teuthology.orchestra.run.smithi153.stdout:osd.0 smithi153 running (11m) 7m ago 27m 116M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 507900fd4791 2024-03-21T10:27:33.432 INFO:teuthology.orchestra.run.smithi153.stdout:osd.1 smithi153 running (11m) 7m ago 26m 89.2M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 0d646452e9c1 2024-03-21T10:27:33.432 INFO:teuthology.orchestra.run.smithi153.stdout:osd.2 smithi153 running (10m) 7m ago 26m 87.4M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 1326f2afdfcd 2024-03-21T10:27:33.432 INFO:teuthology.orchestra.run.smithi153.stdout:osd.3 smithi174 running (10m) 7m ago 25m 83.6M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 70aafac71ad5 2024-03-21T10:27:33.432 INFO:teuthology.orchestra.run.smithi153.stdout:osd.4 smithi174 running (9m) 7m ago 25m 92.2M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 0d746ce6e79d 2024-03-21T10:27:33.432 INFO:teuthology.orchestra.run.smithi153.stdout:osd.5 smithi174 running (8m) 7m ago 24m 86.2M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 53e5f35b8cac 2024-03-21T10:27:33.432 INFO:teuthology.orchestra.run.smithi153.stdout:prometheus.smithi153 smithi153 *:9095 running (16m) 7m ago 29m 73.5M - 2.43.0 a07b618ecd1d 7d95561e50c2 2024-03-21T10:27:33.433 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:33.433+0000 7f8ea8200700 1 -- 172.21.15.153:0/479050368 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f8e8806e460 msgr2=0x7f8e88070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:33.433 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:33.433+0000 7f8ea8200700 1 --2- 172.21.15.153:0/479050368 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f8e8806e460 0x7f8e88070920 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8e94005fd0 tx=0x7f8e94005dd0 comp rx=0 tx=0).stop 2024-03-21T10:27:33.433 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:33.433+0000 7f8ea8200700 1 -- 172.21.15.153:0/479050368 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8ea0105b50 msgr2=0x7f8ea01a3010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:33.433 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:33.433+0000 7f8ea8200700 1 --2- 172.21.15.153:0/479050368 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8ea0105b50 0x7f8ea01a3010 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f8e8c006010 tx=0x7f8e8c00be20 comp rx=0 tx=0).stop 2024-03-21T10:27:33.434 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:33.434+0000 7f8ea8200700 1 -- 172.21.15.153:0/479050368 shutdown_connections 2024-03-21T10:27:33.434 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:33.434+0000 7f8ea8200700 1 --2- 172.21.15.153:0/479050368 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8ea0068950 0x7f8ea01a2ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:33.434 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:33.434+0000 7f8ea8200700 1 --2- 172.21.15.153:0/479050368 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f8e8806e460 0x7f8e88070920 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:33.434 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:33.434+0000 7f8ea8200700 1 --2- 172.21.15.153:0/479050368 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8ea0105b50 0x7f8ea01a3010 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:33.434 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:33.434+0000 7f8ea8200700 1 -- 172.21.15.153:0/479050368 >> 172.21.15.153:0/479050368 conn(0x7f8ea00755a0 msgr2=0x7f8ea0101510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:33.434 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:33.434+0000 7f8ea8200700 1 -- 172.21.15.153:0/479050368 shutdown_connections 2024-03-21T10:27:33.434 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:33.434+0000 7f8ea8200700 1 -- 172.21.15.153:0/479050368 wait complete. 2024-03-21T10:27:33.587 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:33.587+0000 7fa619c00700 1 -- 172.21.15.153:0/472390287 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa614102fb0 msgr2=0x7fa614103390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:33.587 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:33.587+0000 7fa619c00700 1 --2- 172.21.15.153:0/472390287 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa614102fb0 0x7fa614103390 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fa604009910 tx=0x7fa604009c20 comp rx=0 tx=0).stop 2024-03-21T10:27:33.587 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:33.587+0000 7fa619c00700 1 -- 172.21.15.153:0/472390287 shutdown_connections 2024-03-21T10:27:33.587 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:33.587+0000 7fa619c00700 1 --2- 172.21.15.153:0/472390287 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa614103960 0x7fa6141079b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:33.588 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:33.587+0000 7fa619c00700 1 --2- 172.21.15.153:0/472390287 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa614102fb0 0x7fa614103390 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:33.588 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:33.587+0000 7fa619c00700 1 -- 172.21.15.153:0/472390287 >> 172.21.15.153:0/472390287 conn(0x7fa6140fe840 msgr2=0x7fa614100c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:33.588 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:33.587+0000 7fa619c00700 1 -- 172.21.15.153:0/472390287 shutdown_connections 2024-03-21T10:27:33.588 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:33.587+0000 7fa619c00700 1 -- 172.21.15.153:0/472390287 wait complete. 2024-03-21T10:27:33.590 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:33.588+0000 7fa619c00700 1 Processor -- start 2024-03-21T10:27:33.590 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:33.588+0000 7fa619c00700 1 -- start start 2024-03-21T10:27:33.590 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:33.589+0000 7fa619c00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa614102fb0 0x7fa61419e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:33.590 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:33.589+0000 7fa619c00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa614103960 0x7fa61419ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:33.590 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:33.589+0000 7fa619c00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa61419f2d0 con 0x7fa614103960 2024-03-21T10:27:33.590 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:33.589+0000 7fa619c00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa614198780 con 0x7fa614102fb0 2024-03-21T10:27:33.590 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:33.589+0000 7fa612000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa614103960 0x7fa61419ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:33.590 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:33.589+0000 7fa612000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa614103960 0x7fa61419ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:53268/0 (socket says 172.21.15.153:53268) 2024-03-21T10:27:33.591 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:33.589+0000 7fa612000700 1 -- 172.21.15.153:0/1452470725 learned_addr learned my addr 172.21.15.153:0/1452470725 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:27:33.591 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:33.589+0000 7fa612a00700 1 --2- 172.21.15.153:0/1452470725 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa614102fb0 0x7fa61419e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:33.591 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:33.590+0000 7fa612000700 1 -- 172.21.15.153:0/1452470725 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa614102fb0 msgr2=0x7fa61419e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:33.591 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:33.590+0000 7fa612000700 1 --2- 172.21.15.153:0/1452470725 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa614102fb0 0x7fa61419e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:33.591 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:33.590+0000 7fa612000700 1 -- 172.21.15.153:0/1452470725 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6040095f0 con 0x7fa614103960 2024-03-21T10:27:33.591 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:33.590+0000 7fa612000700 1 --2- 172.21.15.153:0/1452470725 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa614103960 0x7fa61419ec40 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fa60000d900 tx=0x7fa60000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:33.591 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:33.590+0000 7fa60b400700 1 -- 172.21.15.153:0/1452470725 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa600009900 con 0x7fa614103960 2024-03-21T10:27:33.591 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:33.590+0000 7fa60b400700 1 -- 172.21.15.153:0/1452470725 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa600010460 con 0x7fa614103960 2024-03-21T10:27:33.591 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:33.590+0000 7fa619c00700 1 -- 172.21.15.153:0/1452470725 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa614198a60 con 0x7fa614103960 2024-03-21T10:27:33.591 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:33.590+0000 7fa60b400700 1 -- 172.21.15.153:0/1452470725 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa60000f5f0 con 0x7fa614103960 2024-03-21T10:27:33.591 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:33.590+0000 7fa619c00700 1 -- 172.21.15.153:0/1452470725 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa614198fb0 con 0x7fa614103960 2024-03-21T10:27:33.592 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:33.591+0000 7fa619c00700 1 -- 172.21.15.153:0/1452470725 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa61410b350 con 0x7fa614103960 2024-03-21T10:27:33.592 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:33.592+0000 7fa60b400700 1 -- 172.21.15.153:0/1452470725 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fa6000105d0 con 0x7fa614103960 2024-03-21T10:27:33.593 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:33.593+0000 7fa60b400700 1 --2- 172.21.15.153:0/1452470725 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa5f806e460 0x7fa5f8070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:33.593 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:33.593+0000 7fa60b400700 1 -- 172.21.15.153:0/1452470725 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fa60008e360 con 0x7fa614103960 2024-03-21T10:27:33.593 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:33.594+0000 7fa612a00700 1 --2- 172.21.15.153:0/1452470725 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa5f806e460 0x7fa5f8070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:33.593 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:33.594+0000 7fa612a00700 1 --2- 172.21.15.153:0/1452470725 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa5f806e460 0x7fa5f8070920 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa604006010 tx=0x7fa60401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:33.597 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:33.598+0000 7fa60b400700 1 -- 172.21.15.153:0/1452470725 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa60005af30 con 0x7fa614103960 2024-03-21T10:27:33.900 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:33.899+0000 7fa619c00700 1 -- 172.21.15.153:0/1452470725 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fa614199220 con 0x7fa614103960 2024-03-21T10:27:33.900 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:33.900+0000 7fa60b400700 1 -- 172.21.15.153:0/1452470725 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+805 (secure 0 0 0) 0x7fa60005aac0 con 0x7fa614103960 2024-03-21T10:27:33.900 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:27:33.900 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": { 2024-03-21T10:27:33.901 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:27:33.901 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:27:33.901 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": { 2024-03-21T10:27:33.901 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:27:33.901 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:27:33.901 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": { 2024-03-21T10:27:33.901 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 6 2024-03-21T10:27:33.901 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:27:33.901 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": { 2024-03-21T10:27:33.901 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-03-21T10:27:33.901 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 3 2024-03-21T10:27:33.901 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:27:33.901 INFO:teuthology.orchestra.run.smithi153.stdout: "overall": { 2024-03-21T10:27:33.901 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-03-21T10:27:33.901 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 13 2024-03-21T10:27:33.902 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-03-21T10:27:33.902 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:27:33.904 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:33.904+0000 7fa619c00700 1 -- 172.21.15.153:0/1452470725 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa5f806e460 msgr2=0x7fa5f8070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:33.904 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:33.904+0000 7fa619c00700 1 --2- 172.21.15.153:0/1452470725 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa5f806e460 0x7fa5f8070920 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa604006010 tx=0x7fa60401a040 comp rx=0 tx=0).stop 2024-03-21T10:27:33.904 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:33.904+0000 7fa619c00700 1 -- 172.21.15.153:0/1452470725 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa614103960 msgr2=0x7fa61419ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:33.904 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:33.904+0000 7fa619c00700 1 --2- 172.21.15.153:0/1452470725 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa614103960 0x7fa61419ec40 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fa60000d900 tx=0x7fa60000dcc0 comp rx=0 tx=0).stop 2024-03-21T10:27:33.904 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:33.904+0000 7fa619c00700 1 -- 172.21.15.153:0/1452470725 shutdown_connections 2024-03-21T10:27:33.904 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:33.904+0000 7fa619c00700 1 --2- 172.21.15.153:0/1452470725 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa614102fb0 0x7fa61419e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:33.904 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:33.904+0000 7fa619c00700 1 --2- 172.21.15.153:0/1452470725 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa5f806e460 0x7fa5f8070920 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:33.905 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:33.904+0000 7fa619c00700 1 --2- 172.21.15.153:0/1452470725 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa614103960 0x7fa61419ec40 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:33.905 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:33.904+0000 7fa619c00700 1 -- 172.21.15.153:0/1452470725 >> 172.21.15.153:0/1452470725 conn(0x7fa6140fe840 msgr2=0x7fa614100c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:33.905 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:33.905+0000 7fa619c00700 1 -- 172.21.15.153:0/1452470725 shutdown_connections 2024-03-21T10:27:33.905 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:33.905+0000 7fa619c00700 1 -- 172.21.15.153:0/1452470725 wait complete. 2024-03-21T10:27:34.050 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.051+0000 7fb02d800700 1 -- 172.21.15.153:0/1135350848 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb02810d280 msgr2=0x7fb02810d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:34.051 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.051+0000 7fb02d800700 1 --2- 172.21.15.153:0/1135350848 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb02810d280 0x7fb02810d660 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fb018009910 tx=0x7fb018009c20 comp rx=0 tx=0).stop 2024-03-21T10:27:34.051 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.051+0000 7fb02d800700 1 -- 172.21.15.153:0/1135350848 shutdown_connections 2024-03-21T10:27:34.051 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.051+0000 7fb02d800700 1 --2- 172.21.15.153:0/1135350848 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0281071d0 0x7fb028107650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:34.051 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.051+0000 7fb02d800700 1 --2- 172.21.15.153:0/1135350848 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb02810d280 0x7fb02810d660 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:34.051 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.051+0000 7fb02d800700 1 -- 172.21.15.153:0/1135350848 >> 172.21.15.153:0/1135350848 conn(0x7fb0280769e0 msgr2=0x7fb028078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:34.051 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.051+0000 7fb02d800700 1 -- 172.21.15.153:0/1135350848 shutdown_connections 2024-03-21T10:27:34.051 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.051+0000 7fb02d800700 1 -- 172.21.15.153:0/1135350848 wait complete. 2024-03-21T10:27:34.053 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.052+0000 7fb02d800700 1 Processor -- start 2024-03-21T10:27:34.053 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.052+0000 7fb02d800700 1 -- start start 2024-03-21T10:27:34.053 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.052+0000 7fb02d800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb0281071d0 0x7fb0281a2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:34.053 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.052+0000 7fb02d800700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb02810d280 0x7fb0281a3010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:34.053 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.052+0000 7fb02d800700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0281a36a0 con 0x7fb0281071d0 2024-03-21T10:27:34.053 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.052+0000 7fb02d800700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb02819cb50 con 0x7fb02810d280 2024-03-21T10:27:34.053 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.053+0000 7fb026a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb0281071d0 0x7fb0281a2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:34.053 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.053+0000 7fb026a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb0281071d0 0x7fb0281a2ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:53290/0 (socket says 172.21.15.153:53290) 2024-03-21T10:27:34.053 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.053+0000 7fb026a00700 1 -- 172.21.15.153:0/39099108 learned_addr learned my addr 172.21.15.153:0/39099108 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:27:34.054 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.053+0000 7fb026a00700 1 -- 172.21.15.153:0/39099108 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb02810d280 msgr2=0x7fb0281a3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:34.054 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.053+0000 7fb026a00700 1 --2- 172.21.15.153:0/39099108 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb02810d280 0x7fb0281a3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:34.054 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.053+0000 7fb026a00700 1 -- 172.21.15.153:0/39099108 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0180095f0 con 0x7fb0281071d0 2024-03-21T10:27:34.054 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.053+0000 7fb026a00700 1 --2- 172.21.15.153:0/39099108 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb0281071d0 0x7fb0281a2ad0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fb01800b5c0 tx=0x7fb01800be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:34.054 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.054+0000 7fb01f400700 1 -- 172.21.15.153:0/39099108 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb01801d070 con 0x7fb0281071d0 2024-03-21T10:27:34.054 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.054+0000 7fb01f400700 1 -- 172.21.15.153:0/39099108 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb01800f460 con 0x7fb0281071d0 2024-03-21T10:27:34.054 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.054+0000 7fb02d800700 1 -- 172.21.15.153:0/39099108 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb02819cdd0 con 0x7fb0281071d0 2024-03-21T10:27:34.054 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.054+0000 7fb01f400700 1 -- 172.21.15.153:0/39099108 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb018017610 con 0x7fb0281071d0 2024-03-21T10:27:34.055 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.054+0000 7fb02d800700 1 -- 172.21.15.153:0/39099108 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb02819d340 con 0x7fb0281071d0 2024-03-21T10:27:34.055 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.055+0000 7fb02d800700 1 -- 172.21.15.153:0/39099108 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb02804ea90 con 0x7fb0281071d0 2024-03-21T10:27:34.055 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.056+0000 7fb01f400700 1 -- 172.21.15.153:0/39099108 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fb01800f5d0 con 0x7fb0281071d0 2024-03-21T10:27:34.056 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.057+0000 7fb01f400700 1 --2- 172.21.15.153:0/39099108 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb00c06e460 0x7fb00c070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:34.056 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.057+0000 7fb01f400700 1 -- 172.21.15.153:0/39099108 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fb01808fa30 con 0x7fb0281071d0 2024-03-21T10:27:34.057 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.057+0000 7fb026000700 1 --2- 172.21.15.153:0/39099108 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb00c06e460 0x7fb00c070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:34.057 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.058+0000 7fb026000700 1 --2- 172.21.15.153:0/39099108 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb00c06e460 0x7fb00c070920 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb0281064e0 tx=0x7fb014005c80 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:34.061 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.062+0000 7fb01f400700 1 -- 172.21.15.153:0/39099108 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb01805c6a0 con 0x7fb0281071d0 2024-03-21T10:27:34.200 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:27:33 smithi174 ceph-mon[88004]: from='client.34870 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:27:34.200 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:27:33 smithi174 ceph-mon[88004]: from='client.34874 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:27:34.200 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:27:33 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/1452470725' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:27:34.305 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:27:33 smithi153 ceph-mon[113733]: from='client.34870 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:27:34.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:27:33 smithi153 ceph-mon[113733]: from='client.34874 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:27:34.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:27:33 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/1452470725' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:27:34.315 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.315+0000 7fb02d800700 1 -- 172.21.15.153:0/39099108 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fb02819dff0 con 0x7fb0281071d0 2024-03-21T10:27:34.315 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.316+0000 7fb01f400700 1 -- 172.21.15.153:0/39099108 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1876 (secure 0 0 0) 0x7fb01805c230 con 0x7fb0281071d0 2024-03-21T10:27:34.316 INFO:teuthology.orchestra.run.smithi153.stdout:e30 2024-03-21T10:27:34.316 INFO:teuthology.orchestra.run.smithi153.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-21T10:27:34.316 INFO:teuthology.orchestra.run.smithi153.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:27:34.316 INFO:teuthology.orchestra.run.smithi153.stdout:legacy client fscid: 1 2024-03-21T10:27:34.317 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:27:34.317 INFO:teuthology.orchestra.run.smithi153.stdout:Filesystem 'cephfs' (1) 2024-03-21T10:27:34.317 INFO:teuthology.orchestra.run.smithi153.stdout:fs_name cephfs 2024-03-21T10:27:34.317 INFO:teuthology.orchestra.run.smithi153.stdout:epoch 23 2024-03-21T10:27:34.317 INFO:teuthology.orchestra.run.smithi153.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-21T10:27:34.317 INFO:teuthology.orchestra.run.smithi153.stdout:created 2024-03-21T10:04:27.086576+0000 2024-03-21T10:27:34.317 INFO:teuthology.orchestra.run.smithi153.stdout:modified 2024-03-21T10:19:05.022092+0000 2024-03-21T10:27:34.317 INFO:teuthology.orchestra.run.smithi153.stdout:tableserver 0 2024-03-21T10:27:34.317 INFO:teuthology.orchestra.run.smithi153.stdout:root 0 2024-03-21T10:27:34.317 INFO:teuthology.orchestra.run.smithi153.stdout:session_timeout 60 2024-03-21T10:27:34.317 INFO:teuthology.orchestra.run.smithi153.stdout:session_autoclose 300 2024-03-21T10:27:34.317 INFO:teuthology.orchestra.run.smithi153.stdout:max_file_size 1099511627776 2024-03-21T10:27:34.317 INFO:teuthology.orchestra.run.smithi153.stdout:max_xattr_size 65536 2024-03-21T10:27:34.318 INFO:teuthology.orchestra.run.smithi153.stdout:required_client_features {} 2024-03-21T10:27:34.318 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure 0 2024-03-21T10:27:34.318 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure_osd_epoch 94 2024-03-21T10:27:34.318 INFO:teuthology.orchestra.run.smithi153.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:27:34.318 INFO:teuthology.orchestra.run.smithi153.stdout:max_mds 2 2024-03-21T10:27:34.318 INFO:teuthology.orchestra.run.smithi153.stdout:in 0,1 2024-03-21T10:27:34.318 INFO:teuthology.orchestra.run.smithi153.stdout:up {} 2024-03-21T10:27:34.318 INFO:teuthology.orchestra.run.smithi153.stdout:failed 0,1 2024-03-21T10:27:34.318 INFO:teuthology.orchestra.run.smithi153.stdout:damaged 2024-03-21T10:27:34.318 INFO:teuthology.orchestra.run.smithi153.stdout:stopped 2024-03-21T10:27:34.318 INFO:teuthology.orchestra.run.smithi153.stdout:data_pools [3] 2024-03-21T10:27:34.318 INFO:teuthology.orchestra.run.smithi153.stdout:metadata_pool 2 2024-03-21T10:27:34.318 INFO:teuthology.orchestra.run.smithi153.stdout:inline_data enabled 2024-03-21T10:27:34.318 INFO:teuthology.orchestra.run.smithi153.stdout:balancer 2024-03-21T10:27:34.319 INFO:teuthology.orchestra.run.smithi153.stdout:bal_rank_mask -1 2024-03-21T10:27:34.319 INFO:teuthology.orchestra.run.smithi153.stdout:standby_count_wanted 1 2024-03-21T10:27:34.319 INFO:teuthology.orchestra.run.smithi153.stdout:qdb_cluster leader: 0 members: 2024-03-21T10:27:34.319 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:27:34.319 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:27:34.319 INFO:teuthology.orchestra.run.smithi153.stdout:Standby daemons: 2024-03-21T10:27:34.319 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:27:34.319 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.zipckr{-1:34442} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6826/1103846295,v1:172.21.15.153:6827/1103846295] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:27:34.319 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.iuyluj{-1:34474} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6828/1738630832,v1:172.21.15.153:6829/1738630832] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:27:34.319 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.nhozag{-1:44269} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.174:6826/3967223227,v1:172.21.15.174:6827/3967223227] compat {c=[1],r=[1],i=[7ff]}] 2024-03-21T10:27:34.319 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.yyimst{-1:44313} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.174:6824/4127786797,v1:172.21.15.174:6825/4127786797] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:27:34.320 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.318+0000 7fb02d800700 1 -- 172.21.15.153:0/39099108 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb00c06e460 msgr2=0x7fb00c070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:34.320 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.318+0000 7fb02d800700 1 --2- 172.21.15.153:0/39099108 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb00c06e460 0x7fb00c070920 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb0281064e0 tx=0x7fb014005c80 comp rx=0 tx=0).stop 2024-03-21T10:27:34.320 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.318+0000 7fb02d800700 1 -- 172.21.15.153:0/39099108 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb0281071d0 msgr2=0x7fb0281a2ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:34.320 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.318+0000 7fb02d800700 1 --2- 172.21.15.153:0/39099108 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb0281071d0 0x7fb0281a2ad0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fb01800b5c0 tx=0x7fb01800be00 comp rx=0 tx=0).stop 2024-03-21T10:27:34.320 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.319+0000 7fb02d800700 1 -- 172.21.15.153:0/39099108 shutdown_connections 2024-03-21T10:27:34.320 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.319+0000 7fb02d800700 1 --2- 172.21.15.153:0/39099108 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb00c06e460 0x7fb00c070920 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:34.320 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.319+0000 7fb02d800700 1 --2- 172.21.15.153:0/39099108 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb0281071d0 0x7fb0281a2ad0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:34.320 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.319+0000 7fb02d800700 1 --2- 172.21.15.153:0/39099108 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb02810d280 0x7fb0281a3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:34.321 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.319+0000 7fb02d800700 1 -- 172.21.15.153:0/39099108 >> 172.21.15.153:0/39099108 conn(0x7fb0280769e0 msgr2=0x7fb0280782d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:34.321 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.319+0000 7fb02d800700 1 -- 172.21.15.153:0/39099108 shutdown_connections 2024-03-21T10:27:34.321 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.319+0000 7fb02d800700 1 -- 172.21.15.153:0/39099108 wait complete. 2024-03-21T10:27:34.321 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 30 2024-03-21T10:27:34.474 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.474+0000 7f18cfc00700 1 -- 172.21.15.153:0/3641883004 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f18c8100da0 msgr2=0x7f18c8101180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:34.475 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.474+0000 7f18cfc00700 1 --2- 172.21.15.153:0/3641883004 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f18c8100da0 0x7f18c8101180 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f18b8009910 tx=0x7f18b8009c20 comp rx=0 tx=0).stop 2024-03-21T10:27:34.475 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.474+0000 7f18cfc00700 1 -- 172.21.15.153:0/3641883004 shutdown_connections 2024-03-21T10:27:34.475 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.474+0000 7f18cfc00700 1 --2- 172.21.15.153:0/3641883004 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f18c8101750 0x7f18c81057a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:34.475 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.474+0000 7f18cfc00700 1 --2- 172.21.15.153:0/3641883004 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f18c8100da0 0x7f18c8101180 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:34.475 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.474+0000 7f18cfc00700 1 -- 172.21.15.153:0/3641883004 >> 172.21.15.153:0/3641883004 conn(0x7f18c80fc630 msgr2=0x7f18c80fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:34.475 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.475+0000 7f18cfc00700 1 -- 172.21.15.153:0/3641883004 shutdown_connections 2024-03-21T10:27:34.475 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.475+0000 7f18cfc00700 1 -- 172.21.15.153:0/3641883004 wait complete. 2024-03-21T10:27:34.476 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.476+0000 7f18cfc00700 1 Processor -- start 2024-03-21T10:27:34.476 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.476+0000 7f18cfc00700 1 -- start start 2024-03-21T10:27:34.477 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.477+0000 7f18cfc00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f18c8100da0 0x7f18c8073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:34.477 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.477+0000 7f18cfc00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f18c8101750 0x7f18c80735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:34.477 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.477+0000 7f18cfc00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18c8073b10 con 0x7f18c8101750 2024-03-21T10:27:34.477 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.477+0000 7f18cfc00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18c8073c50 con 0x7f18c8100da0 2024-03-21T10:27:34.477 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.477+0000 7f18bfe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f18c8101750 0x7f18c80735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:34.477 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.477+0000 7f18bfe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f18c8101750 0x7f18c80735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:53304/0 (socket says 172.21.15.153:53304) 2024-03-21T10:27:34.477 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.477+0000 7f18bfe00700 1 -- 172.21.15.153:0/904327516 learned_addr learned my addr 172.21.15.153:0/904327516 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:27:34.477 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.477+0000 7f18cd200700 1 --2- 172.21.15.153:0/904327516 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f18c8100da0 0x7f18c8073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:34.477 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.477+0000 7f18bfe00700 1 -- 172.21.15.153:0/904327516 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f18c8100da0 msgr2=0x7f18c8073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:34.478 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.477+0000 7f18bfe00700 1 --2- 172.21.15.153:0/904327516 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f18c8100da0 0x7f18c8073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:34.478 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.477+0000 7f18bfe00700 1 -- 172.21.15.153:0/904327516 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18b80095f0 con 0x7f18c8101750 2024-03-21T10:27:34.478 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.478+0000 7f18bfe00700 1 --2- 172.21.15.153:0/904327516 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f18c8101750 0x7f18c80735d0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f18b000d900 tx=0x7f18b000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:34.478 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.478+0000 7f18c6000700 1 -- 172.21.15.153:0/904327516 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f18b0009900 con 0x7f18c8101750 2024-03-21T10:27:34.479 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.478+0000 7f18c6000700 1 -- 172.21.15.153:0/904327516 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f18b0010460 con 0x7f18c8101750 2024-03-21T10:27:34.479 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.478+0000 7f18cfc00700 1 -- 172.21.15.153:0/904327516 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18c81a29a0 con 0x7f18c8101750 2024-03-21T10:27:34.479 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.478+0000 7f18c6000700 1 -- 172.21.15.153:0/904327516 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f18b000f5f0 con 0x7f18c8101750 2024-03-21T10:27:34.479 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.478+0000 7f18cfc00700 1 -- 172.21.15.153:0/904327516 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18c81a2db0 con 0x7f18c8101750 2024-03-21T10:27:34.479 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.479+0000 7f18cfc00700 1 -- 172.21.15.153:0/904327516 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18c8109140 con 0x7f18c8101750 2024-03-21T10:27:34.479 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.480+0000 7f18c6000700 1 -- 172.21.15.153:0/904327516 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f18b00105d0 con 0x7f18c8101750 2024-03-21T10:27:34.481 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.482+0000 7f18c6000700 1 --2- 172.21.15.153:0/904327516 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f18ac06e4b0 0x7f18ac070970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:34.481 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.482+0000 7f18c6000700 1 -- 172.21.15.153:0/904327516 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f18b008e180 con 0x7f18c8101750 2024-03-21T10:27:34.481 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.482+0000 7f18cd200700 1 --2- 172.21.15.153:0/904327516 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f18ac06e4b0 0x7f18ac070970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:34.482 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.483+0000 7f18cd200700 1 --2- 172.21.15.153:0/904327516 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f18ac06e4b0 0x7f18ac070970 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f18b8006010 tx=0x7f18b801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:34.486 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.487+0000 7f18c6000700 1 -- 172.21.15.153:0/904327516 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f18b0057360 con 0x7f18c8101750 2024-03-21T10:27:34.719 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.719+0000 7f18cfc00700 1 -- 172.21.15.153:0/904327516 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f18c802cd50 con 0x7f18ac06e4b0 2024-03-21T10:27:34.721 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.721+0000 7f18c6000700 1 -- 172.21.15.153:0/904327516 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f18c802cd50 con 0x7f18ac06e4b0 2024-03-21T10:27:34.721 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:27:34.721 INFO:teuthology.orchestra.run.smithi153.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232", 2024-03-21T10:27:34.721 INFO:teuthology.orchestra.run.smithi153.stdout: "in_progress": true, 2024-03-21T10:27:34.721 INFO:teuthology.orchestra.run.smithi153.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-21T10:27:34.721 INFO:teuthology.orchestra.run.smithi153.stdout: "services_complete": [ 2024-03-21T10:27:34.721 INFO:teuthology.orchestra.run.smithi153.stdout: "crash", 2024-03-21T10:27:34.722 INFO:teuthology.orchestra.run.smithi153.stdout: "mon", 2024-03-21T10:27:34.722 INFO:teuthology.orchestra.run.smithi153.stdout: "osd", 2024-03-21T10:27:34.722 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr" 2024-03-21T10:27:34.722 INFO:teuthology.orchestra.run.smithi153.stdout: ], 2024-03-21T10:27:34.722 INFO:teuthology.orchestra.run.smithi153.stdout: "progress": "15/23 daemons upgraded", 2024-03-21T10:27:34.722 INFO:teuthology.orchestra.run.smithi153.stdout: "message": "Currently upgrading mds daemons", 2024-03-21T10:27:34.722 INFO:teuthology.orchestra.run.smithi153.stdout: "is_paused": false 2024-03-21T10:27:34.722 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:27:34.725 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.725+0000 7f18cfc00700 1 -- 172.21.15.153:0/904327516 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f18ac06e4b0 msgr2=0x7f18ac070970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:34.726 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.725+0000 7f18cfc00700 1 --2- 172.21.15.153:0/904327516 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f18ac06e4b0 0x7f18ac070970 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f18b8006010 tx=0x7f18b801a040 comp rx=0 tx=0).stop 2024-03-21T10:27:34.726 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.725+0000 7f18cfc00700 1 -- 172.21.15.153:0/904327516 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f18c8101750 msgr2=0x7f18c80735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:34.726 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.725+0000 7f18cfc00700 1 --2- 172.21.15.153:0/904327516 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f18c8101750 0x7f18c80735d0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f18b000d900 tx=0x7f18b000dcc0 comp rx=0 tx=0).stop 2024-03-21T10:27:34.726 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.725+0000 7f18cfc00700 1 -- 172.21.15.153:0/904327516 shutdown_connections 2024-03-21T10:27:34.726 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.725+0000 7f18cfc00700 1 --2- 172.21.15.153:0/904327516 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f18c8100da0 0x7f18c8073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:34.726 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.725+0000 7f18cfc00700 1 --2- 172.21.15.153:0/904327516 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f18ac06e4b0 0x7f18ac070970 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:34.726 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.725+0000 7f18cfc00700 1 --2- 172.21.15.153:0/904327516 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f18c8101750 0x7f18c80735d0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:34.726 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.725+0000 7f18cfc00700 1 -- 172.21.15.153:0/904327516 >> 172.21.15.153:0/904327516 conn(0x7f18c80fc630 msgr2=0x7f18c80fdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:34.726 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.726+0000 7f18cfc00700 1 -- 172.21.15.153:0/904327516 shutdown_connections 2024-03-21T10:27:34.726 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.726+0000 7f18cfc00700 1 -- 172.21.15.153:0/904327516 wait complete. 2024-03-21T10:27:34.875 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.875+0000 7fb61da00700 1 -- 172.21.15.153:0/4145828429 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb6181071d0 msgr2=0x7fb618107650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:34.875 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.875+0000 7fb61da00700 1 --2- 172.21.15.153:0/4145828429 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb6181071d0 0x7fb618107650 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fb600009910 tx=0x7fb600009c20 comp rx=0 tx=0).stop 2024-03-21T10:27:34.875 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.875+0000 7fb61da00700 1 -- 172.21.15.153:0/4145828429 shutdown_connections 2024-03-21T10:27:34.875 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.875+0000 7fb61da00700 1 --2- 172.21.15.153:0/4145828429 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb6181071d0 0x7fb618107650 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:34.876 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.875+0000 7fb61da00700 1 --2- 172.21.15.153:0/4145828429 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb61810d280 0x7fb61810d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:34.876 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.875+0000 7fb61da00700 1 -- 172.21.15.153:0/4145828429 >> 172.21.15.153:0/4145828429 conn(0x7fb6180769e0 msgr2=0x7fb618078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:34.876 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.875+0000 7fb61da00700 1 -- 172.21.15.153:0/4145828429 shutdown_connections 2024-03-21T10:27:34.876 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.875+0000 7fb61da00700 1 -- 172.21.15.153:0/4145828429 wait complete. 2024-03-21T10:27:34.882 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.876+0000 7fb61da00700 1 Processor -- start 2024-03-21T10:27:34.883 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.876+0000 7fb61da00700 1 -- start start 2024-03-21T10:27:34.883 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.877+0000 7fb61da00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb6181071d0 0x7fb61819ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:34.883 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.877+0000 7fb61da00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb61810d280 0x7fb61819d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:34.883 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.877+0000 7fb61da00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb61819d880 con 0x7fb61810d280 2024-03-21T10:27:34.883 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.877+0000 7fb61da00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6181a05b0 con 0x7fb6181071d0 2024-03-21T10:27:34.884 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.877+0000 7fb60e000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb61810d280 0x7fb61819d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:34.884 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.877+0000 7fb60e000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb61810d280 0x7fb61819d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:53310/0 (socket says 172.21.15.153:53310) 2024-03-21T10:27:34.884 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.877+0000 7fb60e000700 1 -- 172.21.15.153:0/3801450392 learned_addr learned my addr 172.21.15.153:0/3801450392 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:27:34.884 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.877+0000 7fb616a00700 1 --2- 172.21.15.153:0/3801450392 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb6181071d0 0x7fb61819ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:34.884 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.878+0000 7fb60e000700 1 -- 172.21.15.153:0/3801450392 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb6181071d0 msgr2=0x7fb61819ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:34.884 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.878+0000 7fb60e000700 1 --2- 172.21.15.153:0/3801450392 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb6181071d0 0x7fb61819ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:34.884 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.878+0000 7fb60e000700 1 -- 172.21.15.153:0/3801450392 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6000095f0 con 0x7fb61810d280 2024-03-21T10:27:34.884 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.878+0000 7fb60e000700 1 --2- 172.21.15.153:0/3801450392 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb61810d280 0x7fb61819d1f0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fb600006010 tx=0x7fb60000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:34.884 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.878+0000 7fb60fe00700 1 -- 172.21.15.153:0/3801450392 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb60001d070 con 0x7fb61810d280 2024-03-21T10:27:34.884 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.878+0000 7fb60fe00700 1 -- 172.21.15.153:0/3801450392 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb600004fb0 con 0x7fb61810d280 2024-03-21T10:27:34.884 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.878+0000 7fb61da00700 1 -- 172.21.15.153:0/3801450392 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6181a0830 con 0x7fb61810d280 2024-03-21T10:27:34.884 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.878+0000 7fb60fe00700 1 -- 172.21.15.153:0/3801450392 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb600022ea0 con 0x7fb61810d280 2024-03-21T10:27:34.884 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.878+0000 7fb61da00700 1 -- 172.21.15.153:0/3801450392 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6181a0da0 con 0x7fb61810d280 2024-03-21T10:27:34.884 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.879+0000 7fb61da00700 1 -- 172.21.15.153:0/3801450392 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb61810f6c0 con 0x7fb61810d280 2024-03-21T10:27:34.885 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.880+0000 7fb60fe00700 1 -- 172.21.15.153:0/3801450392 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fb600022470 con 0x7fb61810d280 2024-03-21T10:27:34.885 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.881+0000 7fb60fe00700 1 --2- 172.21.15.153:0/3801450392 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb5f806e460 0x7fb5f8070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:27:34.885 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.881+0000 7fb60fe00700 1 -- 172.21.15.153:0/3801450392 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fb60008fb30 con 0x7fb61810d280 2024-03-21T10:27:34.885 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.881+0000 7fb616a00700 1 --2- 172.21.15.153:0/3801450392 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb5f806e460 0x7fb5f8070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:27:34.885 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.882+0000 7fb616a00700 1 --2- 172.21.15.153:0/3801450392 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb5f806e460 0x7fb5f8070920 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb608005fd0 tx=0x7fb60800a560 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:27:34.885 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:34.886+0000 7fb60fe00700 1 -- 172.21.15.153:0/3801450392 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb600058c00 con 0x7fb61810d280 2024-03-21T10:27:35.055 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:27:34 smithi153 ceph-mon[113733]: from='client.34878 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:27:35.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:27:34 smithi153 ceph-mon[113733]: pgmap v471: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:27:35.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:27:34 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/39099108' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:27:35.185 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:35.185+0000 7fb61da00700 1 -- 172.21.15.153:0/3801450392 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fb61804ea90 con 0x7fb61810d280 2024-03-21T10:27:35.186 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:35.186+0000 7fb60fe00700 1 -- 172.21.15.153:0/3801450392 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7fb600022720 con 0x7fb61810d280 2024-03-21T10:27:35.186 INFO:teuthology.orchestra.run.smithi153.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-03-21T10:27:35.186 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-21T10:27:35.186 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is degraded 2024-03-21T10:27:35.186 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-21T10:27:35.187 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-21T10:27:35.187 INFO:teuthology.orchestra.run.smithi153.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-21T10:27:35.187 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-21T10:27:35.188 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:35.189+0000 7fb61da00700 1 -- 172.21.15.153:0/3801450392 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb5f806e460 msgr2=0x7fb5f8070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:35.189 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:35.189+0000 7fb61da00700 1 --2- 172.21.15.153:0/3801450392 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb5f806e460 0x7fb5f8070920 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb608005fd0 tx=0x7fb60800a560 comp rx=0 tx=0).stop 2024-03-21T10:27:35.189 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:35.189+0000 7fb61da00700 1 -- 172.21.15.153:0/3801450392 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb61810d280 msgr2=0x7fb61819d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:27:35.189 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:35.189+0000 7fb61da00700 1 --2- 172.21.15.153:0/3801450392 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb61810d280 0x7fb61819d1f0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fb600006010 tx=0x7fb60000be20 comp rx=0 tx=0).stop 2024-03-21T10:27:35.189 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:35.189+0000 7fb61da00700 1 -- 172.21.15.153:0/3801450392 shutdown_connections 2024-03-21T10:27:35.189 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:35.189+0000 7fb61da00700 1 --2- 172.21.15.153:0/3801450392 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb6181071d0 0x7fb61819ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:35.189 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:35.189+0000 7fb61da00700 1 --2- 172.21.15.153:0/3801450392 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb5f806e460 0x7fb5f8070920 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:35.189 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:35.189+0000 7fb61da00700 1 --2- 172.21.15.153:0/3801450392 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb61810d280 0x7fb61819d1f0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:27:35.189 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:35.189+0000 7fb61da00700 1 -- 172.21.15.153:0/3801450392 >> 172.21.15.153:0/3801450392 conn(0x7fb6180769e0 msgr2=0x7fb618078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:27:35.189 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:35.189+0000 7fb61da00700 1 -- 172.21.15.153:0/3801450392 shutdown_connections 2024-03-21T10:27:35.189 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:27:35.189+0000 7fb61da00700 1 -- 172.21.15.153:0/3801450392 wait complete. 2024-03-21T10:27:35.200 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:27:34 smithi174 ceph-mon[88004]: from='client.34878 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:27:35.200 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:27:34 smithi174 ceph-mon[88004]: pgmap v471: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:27:35.200 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:27:34 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/39099108' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:27:36.200 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:27:35 smithi174 ceph-mon[88004]: from='client.34890 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:27:36.200 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:27:35 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/3801450392' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:27:36.305 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:27:35 smithi153 ceph-mon[113733]: from='client.34890 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:27:36.305 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:27:35 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/3801450392' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:27:37.200 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:27:36 smithi174 ceph-mon[88004]: pgmap v472: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:27:37.305 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:27:36 smithi153 ceph-mon[113733]: pgmap v472: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:27:39.306 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:27:38 smithi153 ceph-mon[113733]: pgmap v473: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:27:39.450 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:27:38 smithi174 ceph-mon[88004]: pgmap v473: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:27:41.305 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:27:40 smithi153 ceph-mon[113733]: pgmap v474: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:27:41.450 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:27:40 smithi174 ceph-mon[88004]: pgmap v474: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:27:43.286 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:27:42 smithi174 ceph-mon[88004]: pgmap v475: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:27:43.305 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:27:42 smithi153 ceph-mon[113733]: pgmap v475: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:27:45.305 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:27:44 smithi153 ceph-mon[113733]: pgmap v476: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:27:45.450 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:27:44 smithi174 ceph-mon[88004]: pgmap v476: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:27:47.305 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:27:46 smithi153 ceph-mon[113733]: pgmap v477: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:27:47.305 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:27:46 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:27:47.450 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:27:46 smithi174 ceph-mon[88004]: pgmap v477: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:27:47.450 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:27:46 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:27:49.305 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:27:48 smithi153 ceph-mon[113733]: pgmap v478: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:27:49.450 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:27:48 smithi174 ceph-mon[88004]: pgmap v478: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:27:51.305 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:27:50 smithi153 ceph-mon[113733]: pgmap v479: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:27:51.450 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:27:50 smithi174 ceph-mon[88004]: pgmap v479: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:27:53.305 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:27:52 smithi153 ceph-mon[113733]: pgmap v480: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:27:53.450 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:27:52 smithi174 ceph-mon[88004]: pgmap v480: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:27:55.305 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:27:55 smithi153 ceph-mon[113733]: pgmap v481: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:27:55.450 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:27:55 smithi174 ceph-mon[88004]: pgmap v481: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:27:57.305 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:27:57 smithi153 ceph-mon[113733]: pgmap v482: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:27:57.450 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:27:57 smithi174 ceph-mon[88004]: pgmap v482: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:27:59.305 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:27:59 smithi153 ceph-mon[113733]: pgmap v483: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:27:59.450 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:27:59 smithi174 ceph-mon[88004]: pgmap v483: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:28:01.305 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:28:01 smithi153 ceph-mon[113733]: pgmap v484: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:28:01.450 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:28:01 smithi174 ceph-mon[88004]: pgmap v484: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:28:02.305 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:28:02 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:28:02.450 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:28:02 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:28:03.305 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:28:03 smithi153 ceph-mon[113733]: pgmap v485: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:28:03.450 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:28:03 smithi174 ceph-mon[88004]: pgmap v485: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:28:05.305 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:28:05 smithi153 ceph-mon[113733]: pgmap v486: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:28:05.357 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:05.356+0000 7f45a9800700 1 -- 172.21.15.153:0/885456854 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f45a4068670 msgr2=0x7f45a4068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:05.358 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:05.356+0000 7f45a9800700 1 --2- 172.21.15.153:0/885456854 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f45a4068670 0x7f45a4068a50 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f459400b0a0 tx=0x7f459400b3b0 comp rx=0 tx=0).stop 2024-03-21T10:28:05.358 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:05.357+0000 7f45a9800700 1 -- 172.21.15.153:0/885456854 shutdown_connections 2024-03-21T10:28:05.358 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:05.357+0000 7f45a9800700 1 --2- 172.21.15.153:0/885456854 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f45a4069020 0x7f45a4105790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:05.358 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:05.357+0000 7f45a9800700 1 --2- 172.21.15.153:0/885456854 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f45a4068670 0x7f45a4068a50 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:05.358 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:05.357+0000 7f45a9800700 1 -- 172.21.15.153:0/885456854 >> 172.21.15.153:0/885456854 conn(0x7f45a40755a0 msgr2=0x7f45a40759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:28:05.358 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:05.357+0000 7f45a9800700 1 -- 172.21.15.153:0/885456854 shutdown_connections 2024-03-21T10:28:05.358 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:05.357+0000 7f45a9800700 1 -- 172.21.15.153:0/885456854 wait complete. 2024-03-21T10:28:05.360 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:05.358+0000 7f45a9800700 1 Processor -- start 2024-03-21T10:28:05.360 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:05.358+0000 7f45a9800700 1 -- start start 2024-03-21T10:28:05.360 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:05.358+0000 7f45a9800700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f45a4068670 0x7f45a40ffca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:05.360 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:05.358+0000 7f45a9800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f45a4069020 0x7f45a41001e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:05.360 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:05.358+0000 7f45a9800700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45a4100720 con 0x7f45a4069020 2024-03-21T10:28:05.360 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:05.358+0000 7f45a9800700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45a4100860 con 0x7f45a4068670 2024-03-21T10:28:05.360 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:05.359+0000 7f45a2000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f45a4069020 0x7f45a41001e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:05.361 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:05.359+0000 7f45a2000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f45a4069020 0x7f45a41001e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:46656/0 (socket says 172.21.15.153:46656) 2024-03-21T10:28:05.361 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:05.359+0000 7f45a2000700 1 -- 172.21.15.153:0/4177413735 learned_addr learned my addr 172.21.15.153:0/4177413735 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:28:05.361 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:05.359+0000 7f45a2a00700 1 --2- 172.21.15.153:0/4177413735 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f45a4068670 0x7f45a40ffca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:05.361 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:05.359+0000 7f45a2000700 1 -- 172.21.15.153:0/4177413735 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f45a4068670 msgr2=0x7f45a40ffca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:05.361 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:05.359+0000 7f45a2000700 1 --2- 172.21.15.153:0/4177413735 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f45a4068670 0x7f45a40ffca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:05.361 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:05.359+0000 7f45a2000700 1 -- 172.21.15.153:0/4177413735 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4594009d00 con 0x7f45a4069020 2024-03-21T10:28:05.361 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:05.360+0000 7f45a2000700 1 --2- 172.21.15.153:0/4177413735 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f45a4069020 0x7f45a41001e0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f459000d900 tx=0x7f459000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:28:05.362 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:05.360+0000 7f459b400700 1 -- 172.21.15.153:0/4177413735 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4590009900 con 0x7f45a4069020 2024-03-21T10:28:05.362 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:05.360+0000 7f459b400700 1 -- 172.21.15.153:0/4177413735 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4590010460 con 0x7f45a4069020 2024-03-21T10:28:05.362 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:05.360+0000 7f45a9800700 1 -- 172.21.15.153:0/4177413735 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45a41a2910 con 0x7f45a4069020 2024-03-21T10:28:05.362 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:05.360+0000 7f459b400700 1 -- 172.21.15.153:0/4177413735 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f459000f5f0 con 0x7f45a4069020 2024-03-21T10:28:05.362 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:05.360+0000 7f45a9800700 1 -- 172.21.15.153:0/4177413735 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45a41a2d20 con 0x7f45a4069020 2024-03-21T10:28:05.363 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:05.361+0000 7f45a9800700 1 -- 172.21.15.153:0/4177413735 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45a4109130 con 0x7f45a4069020 2024-03-21T10:28:05.363 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:05.362+0000 7f459b400700 1 -- 172.21.15.153:0/4177413735 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f45900105d0 con 0x7f45a4069020 2024-03-21T10:28:05.363 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:05.363+0000 7f459b400700 1 --2- 172.21.15.153:0/4177413735 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f458806e460 0x7f4588070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:05.364 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:05.363+0000 7f459b400700 1 -- 172.21.15.153:0/4177413735 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f459008e2c0 con 0x7f45a4069020 2024-03-21T10:28:05.364 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:05.363+0000 7f45a2a00700 1 --2- 172.21.15.153:0/4177413735 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f458806e460 0x7f4588070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:05.364 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:05.364+0000 7f45a2a00700 1 --2- 172.21.15.153:0/4177413735 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f458806e460 0x7f4588070920 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f459400b830 tx=0x7f45940061f0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:28:05.368 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:05.368+0000 7f459b400700 1 -- 172.21.15.153:0/4177413735 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4590057400 con 0x7f45a4069020 2024-03-21T10:28:05.450 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:28:05 smithi174 ceph-mon[88004]: pgmap v486: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:28:05.616 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:05.615+0000 7f45a9800700 1 -- 172.21.15.153:0/4177413735 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f45a402cda0 con 0x7f458806e460 2024-03-21T10:28:05.618 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:05.617+0000 7f459b400700 1 -- 172.21.15.153:0/4177413735 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f45a402cda0 con 0x7f458806e460 2024-03-21T10:28:05.620 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:05.620+0000 7f45a9800700 1 -- 172.21.15.153:0/4177413735 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f458806e460 msgr2=0x7f4588070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:05.621 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:05.620+0000 7f45a9800700 1 --2- 172.21.15.153:0/4177413735 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f458806e460 0x7f4588070920 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f459400b830 tx=0x7f45940061f0 comp rx=0 tx=0).stop 2024-03-21T10:28:05.621 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:05.620+0000 7f45a9800700 1 -- 172.21.15.153:0/4177413735 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f45a4069020 msgr2=0x7f45a41001e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:05.621 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:05.620+0000 7f45a9800700 1 --2- 172.21.15.153:0/4177413735 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f45a4069020 0x7f45a41001e0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f459000d900 tx=0x7f459000dcc0 comp rx=0 tx=0).stop 2024-03-21T10:28:05.621 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:05.620+0000 7f45a9800700 1 -- 172.21.15.153:0/4177413735 shutdown_connections 2024-03-21T10:28:05.621 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:05.620+0000 7f45a9800700 1 --2- 172.21.15.153:0/4177413735 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f45a4068670 0x7f45a40ffca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:05.621 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:05.620+0000 7f45a9800700 1 --2- 172.21.15.153:0/4177413735 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f458806e460 0x7f4588070920 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:05.621 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:05.620+0000 7f45a9800700 1 --2- 172.21.15.153:0/4177413735 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f45a4069020 0x7f45a41001e0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:05.621 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:05.620+0000 7f45a9800700 1 -- 172.21.15.153:0/4177413735 >> 172.21.15.153:0/4177413735 conn(0x7f45a40755a0 msgr2=0x7f45a40fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:28:05.622 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:05.620+0000 7f45a9800700 1 -- 172.21.15.153:0/4177413735 shutdown_connections 2024-03-21T10:28:05.622 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:05.621+0000 7f45a9800700 1 -- 172.21.15.153:0/4177413735 wait complete. 2024-03-21T10:28:05.636 INFO:teuthology.orchestra.run.smithi153.stdout:true 2024-03-21T10:28:05.779 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:05.778+0000 7fd8a3c00700 1 -- 172.21.15.153:0/1056229564 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd89c073160 msgr2=0x7fd89c073540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:05.779 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:05.778+0000 7fd8a3c00700 1 --2- 172.21.15.153:0/1056229564 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd89c073160 0x7fd89c073540 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fd890009910 tx=0x7fd890009c20 comp rx=0 tx=0).stop 2024-03-21T10:28:05.779 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:05.778+0000 7fd8a3c00700 1 -- 172.21.15.153:0/1056229564 shutdown_connections 2024-03-21T10:28:05.779 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:05.778+0000 7fd8a3c00700 1 --2- 172.21.15.153:0/1056229564 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd89c073a80 0x7fd89c10cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:05.779 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:05.778+0000 7fd8a3c00700 1 --2- 172.21.15.153:0/1056229564 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd89c073160 0x7fd89c073540 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:05.779 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:05.778+0000 7fd8a3c00700 1 -- 172.21.15.153:0/1056229564 >> 172.21.15.153:0/1056229564 conn(0x7fd89c0fc630 msgr2=0x7fd89c0fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:28:05.779 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:05.778+0000 7fd8a3c00700 1 -- 172.21.15.153:0/1056229564 shutdown_connections 2024-03-21T10:28:05.780 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:05.779+0000 7fd8a3c00700 1 -- 172.21.15.153:0/1056229564 wait complete. 2024-03-21T10:28:05.780 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:05.780+0000 7fd8a3c00700 1 Processor -- start 2024-03-21T10:28:05.781 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:05.780+0000 7fd8a3c00700 1 -- start start 2024-03-21T10:28:05.781 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:05.780+0000 7fd8a3c00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd89c073160 0x7fd89c198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:05.781 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:05.780+0000 7fd8a3c00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd89c073a80 0x7fd89c198ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:05.781 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:05.780+0000 7fd8a3c00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd89c199550 con 0x7fd89c073a80 2024-03-21T10:28:05.781 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:05.780+0000 7fd8a3c00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd89c19d360 con 0x7fd89c073160 2024-03-21T10:28:05.781 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:05.780+0000 7fd89be00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd89c073a80 0x7fd89c198ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:05.781 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:05.781+0000 7fd89be00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd89c073a80 0x7fd89c198ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:46682/0 (socket says 172.21.15.153:46682) 2024-03-21T10:28:05.781 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:05.781+0000 7fd89be00700 1 -- 172.21.15.153:0/515638449 learned_addr learned my addr 172.21.15.153:0/515638449 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:28:05.782 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:05.781+0000 7fd8a1200700 1 --2- 172.21.15.153:0/515638449 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd89c073160 0x7fd89c198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:05.782 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:05.781+0000 7fd89be00700 1 -- 172.21.15.153:0/515638449 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd89c073160 msgr2=0x7fd89c198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:05.782 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:05.781+0000 7fd89be00700 1 --2- 172.21.15.153:0/515638449 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd89c073160 0x7fd89c198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:05.782 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:05.781+0000 7fd89be00700 1 -- 172.21.15.153:0/515638449 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8900095f0 con 0x7fd89c073a80 2024-03-21T10:28:05.782 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:05.781+0000 7fd89be00700 1 --2- 172.21.15.153:0/515638449 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd89c073a80 0x7fd89c198ec0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fd88800d8d0 tx=0x7fd88800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:28:05.783 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:05.782+0000 7fd899600700 1 -- 172.21.15.153:0/515638449 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd888009940 con 0x7fd89c073a80 2024-03-21T10:28:05.783 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:05.782+0000 7fd899600700 1 -- 172.21.15.153:0/515638449 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd888010460 con 0x7fd89c073a80 2024-03-21T10:28:05.783 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:05.782+0000 7fd8a3c00700 1 -- 172.21.15.153:0/515638449 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd89c19d640 con 0x7fd89c073a80 2024-03-21T10:28:05.783 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:05.782+0000 7fd899600700 1 -- 172.21.15.153:0/515638449 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd88800f5f0 con 0x7fd89c073a80 2024-03-21T10:28:05.783 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:05.782+0000 7fd8a3c00700 1 -- 172.21.15.153:0/515638449 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd89c19db90 con 0x7fd89c073a80 2024-03-21T10:28:05.783 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:05.783+0000 7fd8a3c00700 1 -- 172.21.15.153:0/515638449 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd89c10a5a0 con 0x7fd89c073a80 2024-03-21T10:28:05.784 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:05.784+0000 7fd899600700 1 -- 172.21.15.153:0/515638449 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fd8880105d0 con 0x7fd89c073a80 2024-03-21T10:28:05.785 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:05.785+0000 7fd899600700 1 --2- 172.21.15.153:0/515638449 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd88406e460 0x7fd884070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:05.785 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:05.785+0000 7fd899600700 1 -- 172.21.15.153:0/515638449 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fd88808e480 con 0x7fd89c073a80 2024-03-21T10:28:05.786 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:05.785+0000 7fd8a1200700 1 --2- 172.21.15.153:0/515638449 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd88406e460 0x7fd884070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:05.787 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:05.786+0000 7fd8a1200700 1 --2- 172.21.15.153:0/515638449 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd88406e460 0x7fd884070920 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd89000b5c0 tx=0x7fd89001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:28:05.790 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:05.790+0000 7fd899600700 1 -- 172.21.15.153:0/515638449 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd88805b050 con 0x7fd89c073a80 2024-03-21T10:28:06.034 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:06.031+0000 7fd8a3c00700 1 -- 172.21.15.153:0/515638449 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd89c19de00 con 0x7fd88406e460 2024-03-21T10:28:06.035 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:06.035+0000 7fd899600700 1 -- 172.21.15.153:0/515638449 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fd89c19de00 con 0x7fd88406e460 2024-03-21T10:28:06.039 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:06.038+0000 7fd8a3c00700 1 -- 172.21.15.153:0/515638449 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd88406e460 msgr2=0x7fd884070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:06.039 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:06.038+0000 7fd8a3c00700 1 --2- 172.21.15.153:0/515638449 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd88406e460 0x7fd884070920 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd89000b5c0 tx=0x7fd89001a040 comp rx=0 tx=0).stop 2024-03-21T10:28:06.039 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:06.038+0000 7fd8a3c00700 1 -- 172.21.15.153:0/515638449 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd89c073a80 msgr2=0x7fd89c198ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:06.039 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:06.038+0000 7fd8a3c00700 1 --2- 172.21.15.153:0/515638449 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd89c073a80 0x7fd89c198ec0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fd88800d8d0 tx=0x7fd88800dc90 comp rx=0 tx=0).stop 2024-03-21T10:28:06.039 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:06.038+0000 7fd8a3c00700 1 -- 172.21.15.153:0/515638449 shutdown_connections 2024-03-21T10:28:06.039 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:06.038+0000 7fd8a3c00700 1 --2- 172.21.15.153:0/515638449 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd89c073160 0x7fd89c198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:06.040 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:06.038+0000 7fd8a3c00700 1 --2- 172.21.15.153:0/515638449 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd88406e460 0x7fd884070920 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:06.040 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:06.038+0000 7fd8a3c00700 1 --2- 172.21.15.153:0/515638449 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd89c073a80 0x7fd89c198ec0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:06.040 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:06.038+0000 7fd8a3c00700 1 -- 172.21.15.153:0/515638449 >> 172.21.15.153:0/515638449 conn(0x7fd89c0fc630 msgr2=0x7fd89c1076e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:28:06.040 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:06.039+0000 7fd8a3c00700 1 -- 172.21.15.153:0/515638449 shutdown_connections 2024-03-21T10:28:06.040 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:06.039+0000 7fd8a3c00700 1 -- 172.21.15.153:0/515638449 wait complete. 2024-03-21T10:28:06.190 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:06.189+0000 7fc2c0400700 1 -- 172.21.15.153:0/2262386068 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc2b8102620 msgr2=0x7fc2b810ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:06.190 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:06.189+0000 7fc2c0400700 1 --2- 172.21.15.153:0/2262386068 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc2b8102620 0x7fc2b810ab10 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fc2a4009910 tx=0x7fc2a4009c20 comp rx=0 tx=0).stop 2024-03-21T10:28:06.191 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:06.190+0000 7fc2c0400700 1 -- 172.21.15.153:0/2262386068 shutdown_connections 2024-03-21T10:28:06.191 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:06.190+0000 7fc2c0400700 1 --2- 172.21.15.153:0/2262386068 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc2b8102620 0x7fc2b810ab10 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:06.191 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:06.190+0000 7fc2c0400700 1 --2- 172.21.15.153:0/2262386068 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc2b8101c70 0x7fc2b8102050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:06.191 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:06.190+0000 7fc2c0400700 1 -- 172.21.15.153:0/2262386068 >> 172.21.15.153:0/2262386068 conn(0x7fc2b80fb570 msgr2=0x7fc2b80fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:28:06.191 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:06.190+0000 7fc2c0400700 1 -- 172.21.15.153:0/2262386068 shutdown_connections 2024-03-21T10:28:06.191 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:06.190+0000 7fc2c0400700 1 -- 172.21.15.153:0/2262386068 wait complete. 2024-03-21T10:28:06.191 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:06.191+0000 7fc2c0400700 1 Processor -- start 2024-03-21T10:28:06.192 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:06.191+0000 7fc2c0400700 1 -- start start 2024-03-21T10:28:06.192 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:06.191+0000 7fc2c0400700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc2b8101c70 0x7fc2b80ff960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:06.192 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:06.191+0000 7fc2c0400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc2b8102620 0x7fc2b80ffea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:06.192 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:06.191+0000 7fc2c0400700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b81003e0 con 0x7fc2b8102620 2024-03-21T10:28:06.192 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:06.191+0000 7fc2c0400700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b8100520 con 0x7fc2b8101c70 2024-03-21T10:28:06.193 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:06.192+0000 7fc2bd000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc2b8102620 0x7fc2b80ffea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:06.193 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:06.192+0000 7fc2bd000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc2b8102620 0x7fc2b80ffea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:46692/0 (socket says 172.21.15.153:46692) 2024-03-21T10:28:06.193 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:06.192+0000 7fc2bd000700 1 -- 172.21.15.153:0/4227854133 learned_addr learned my addr 172.21.15.153:0/4227854133 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:28:06.193 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:06.192+0000 7fc2bda00700 1 --2- 172.21.15.153:0/4227854133 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc2b8101c70 0x7fc2b80ff960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:06.193 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:06.192+0000 7fc2bd000700 1 -- 172.21.15.153:0/4227854133 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc2b8101c70 msgr2=0x7fc2b80ff960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:06.194 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:06.192+0000 7fc2bd000700 1 --2- 172.21.15.153:0/4227854133 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc2b8101c70 0x7fc2b80ff960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:06.194 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:06.192+0000 7fc2bd000700 1 -- 172.21.15.153:0/4227854133 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2a40095f0 con 0x7fc2b8102620 2024-03-21T10:28:06.194 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:06.193+0000 7fc2bd000700 1 --2- 172.21.15.153:0/4227854133 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc2b8102620 0x7fc2b80ffea0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fc2a400bda0 tx=0x7fc2a400be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:28:06.194 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:06.193+0000 7fc2b6000700 1 -- 172.21.15.153:0/4227854133 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc2a401d070 con 0x7fc2b8102620 2024-03-21T10:28:06.194 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:06.193+0000 7fc2c0400700 1 -- 172.21.15.153:0/4227854133 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2b81007a0 con 0x7fc2b8102620 2024-03-21T10:28:06.194 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:06.193+0000 7fc2b6000700 1 -- 172.21.15.153:0/4227854133 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc2a400f460 con 0x7fc2b8102620 2024-03-21T10:28:06.194 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:06.193+0000 7fc2b6000700 1 -- 172.21.15.153:0/4227854133 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc2a4017630 con 0x7fc2b8102620 2024-03-21T10:28:06.194 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:06.193+0000 7fc2c0400700 1 -- 172.21.15.153:0/4227854133 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2b81a2c50 con 0x7fc2b8102620 2024-03-21T10:28:06.194 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:06.194+0000 7fc2c0400700 1 -- 172.21.15.153:0/4227854133 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2b8108210 con 0x7fc2b8102620 2024-03-21T10:28:06.196 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:06.196+0000 7fc2b6000700 1 -- 172.21.15.153:0/4227854133 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fc2a400fa40 con 0x7fc2b8102620 2024-03-21T10:28:06.197 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:06.197+0000 7fc2b6000700 1 --2- 172.21.15.153:0/4227854133 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fc2a006e460 0x7fc2a0070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:06.197 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:06.197+0000 7fc2b6000700 1 -- 172.21.15.153:0/4227854133 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fc2a408fa30 con 0x7fc2b8102620 2024-03-21T10:28:06.197 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:06.197+0000 7fc2bda00700 1 --2- 172.21.15.153:0/4227854133 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fc2a006e460 0x7fc2a0070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:06.198 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:06.198+0000 7fc2bda00700 1 --2- 172.21.15.153:0/4227854133 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fc2a006e460 0x7fc2a0070920 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac005950 tx=0x7fc2ac0058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:28:06.203 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:06.203+0000 7fc2b6000700 1 -- 172.21.15.153:0/4227854133 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc2a4058a50 con 0x7fc2b8102620 2024-03-21T10:28:06.440 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:06.439+0000 7fc2c0400700 1 -- 172.21.15.153:0/4227854133 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fc2b802cce0 con 0x7fc2a006e460 2024-03-21T10:28:06.457 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:06.456+0000 7fc2b6000700 1 -- 172.21.15.153:0/4227854133 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fc2b802cce0 con 0x7fc2a006e460 2024-03-21T10:28:06.458 INFO:teuthology.orchestra.run.smithi153.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-21T10:28:06.458 INFO:teuthology.orchestra.run.smithi153.stdout:alertmanager.smithi153 smithi153 *:9093,9094 running (16m) 8m ago 30m 14.7M - 0.25.0 c8568f914cd2 9ed3f2200b5f 2024-03-21T10:28:06.458 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi153 smithi153 running (31m) 8m ago 31m 9424k - 18.2.1 5be31c24972a 5b61994396a6 2024-03-21T10:28:06.458 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi174 smithi174 running (29m) 7m ago 29m 9244k - 18.2.1 5be31c24972a 4f7a5fc3ac28 2024-03-21T10:28:06.458 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi153 smithi153 running (13m) 8m ago 30m 6656k - 19.0.0-2231-g2edfee68 237e43c7a65f 32c21104d5b0 2024-03-21T10:28:06.458 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi174 smithi174 running (12m) 7m ago 29m 6660k - 19.0.0-2231-g2edfee68 237e43c7a65f a04e48967ab6 2024-03-21T10:28:06.458 INFO:teuthology.orchestra.run.smithi153.stdout:grafana.smithi153 smithi153 *:3000 running (16m) 8m ago 30m 73.6M - 9.4.12 2bacad6d85d8 ee24026ab786 2024-03-21T10:28:06.458 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.iuyluj smithi153 running (8m) 8m ago 23m 10.9M - 19.0.0-2231-g2edfee68 237e43c7a65f 87aff8a5de70 2024-03-21T10:28:06.458 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.zipckr smithi153 running (8m) 8m ago 23m 11.8M - 19.0.0-2231-g2edfee68 237e43c7a65f ccc262bd5dd3 2024-03-21T10:28:06.458 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.nhozag smithi174 running (23m) 7m ago 23m 12.0M - 18.2.1 5be31c24972a 2fee4000308f 2024-03-21T10:28:06.458 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.yyimst smithi174 running (8m) 7m ago 23m 10.7M - 19.0.0-2231-g2edfee68 237e43c7a65f fc1962f9351e 2024-03-21T10:28:06.458 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi153.tluxni smithi153 *:8443,9283,8765 running (19m) 8m ago 32m 509M - 19.0.0-2231-g2edfee68 237e43c7a65f 7ffee0e55382 2024-03-21T10:28:06.458 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi174.qpcwat smithi174 *:8443,9283,8765 running (18m) 7m ago 29m 452M - 19.0.0-2231-g2edfee68 237e43c7a65f 748968f216ab 2024-03-21T10:28:06.459 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi153 smithi153 running (14m) 8m ago 32m 55.0M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f dd8038303dc3 2024-03-21T10:28:06.459 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi174 smithi174 running (14m) 7m ago 29m 44.2M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f 29d78a788f2c 2024-03-21T10:28:06.459 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi153 smithi153 *:9100 running (18m) 8m ago 30m 7889k - 1.5.0 0da6a335fe13 bb835edc0e0b 2024-03-21T10:28:06.459 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi174 smithi174 *:9100 running (17m) 7m ago 29m 8102k - 1.5.0 0da6a335fe13 7c4ecad790fa 2024-03-21T10:28:06.459 INFO:teuthology.orchestra.run.smithi153.stdout:osd.0 smithi153 running (12m) 8m ago 28m 116M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 507900fd4791 2024-03-21T10:28:06.459 INFO:teuthology.orchestra.run.smithi153.stdout:osd.1 smithi153 running (11m) 8m ago 27m 89.2M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 0d646452e9c1 2024-03-21T10:28:06.459 INFO:teuthology.orchestra.run.smithi153.stdout:osd.2 smithi153 running (11m) 8m ago 26m 87.4M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 1326f2afdfcd 2024-03-21T10:28:06.459 INFO:teuthology.orchestra.run.smithi153.stdout:osd.3 smithi174 running (10m) 7m ago 26m 83.6M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 70aafac71ad5 2024-03-21T10:28:06.459 INFO:teuthology.orchestra.run.smithi153.stdout:osd.4 smithi174 running (9m) 7m ago 25m 92.2M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 0d746ce6e79d 2024-03-21T10:28:06.459 INFO:teuthology.orchestra.run.smithi153.stdout:osd.5 smithi174 running (9m) 7m ago 25m 86.2M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 53e5f35b8cac 2024-03-21T10:28:06.459 INFO:teuthology.orchestra.run.smithi153.stdout:prometheus.smithi153 smithi153 *:9095 running (17m) 8m ago 29m 73.5M - 2.43.0 a07b618ecd1d 7d95561e50c2 2024-03-21T10:28:06.460 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:06.459+0000 7fc2c0400700 1 -- 172.21.15.153:0/4227854133 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fc2a006e460 msgr2=0x7fc2a0070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:06.460 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:06.459+0000 7fc2c0400700 1 --2- 172.21.15.153:0/4227854133 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fc2a006e460 0x7fc2a0070920 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac005950 tx=0x7fc2ac0058e0 comp rx=0 tx=0).stop 2024-03-21T10:28:06.460 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:06.459+0000 7fc2c0400700 1 -- 172.21.15.153:0/4227854133 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc2b8102620 msgr2=0x7fc2b80ffea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:06.460 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:06.459+0000 7fc2c0400700 1 --2- 172.21.15.153:0/4227854133 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc2b8102620 0x7fc2b80ffea0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fc2a400bda0 tx=0x7fc2a400be80 comp rx=0 tx=0).stop 2024-03-21T10:28:06.460 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:06.459+0000 7fc2c0400700 1 -- 172.21.15.153:0/4227854133 shutdown_connections 2024-03-21T10:28:06.461 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:06.459+0000 7fc2c0400700 1 --2- 172.21.15.153:0/4227854133 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc2b8101c70 0x7fc2b80ff960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:06.461 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:06.459+0000 7fc2c0400700 1 --2- 172.21.15.153:0/4227854133 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fc2a006e460 0x7fc2a0070920 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:06.461 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:06.459+0000 7fc2c0400700 1 --2- 172.21.15.153:0/4227854133 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc2b8102620 0x7fc2b80ffea0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:06.461 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:06.459+0000 7fc2c0400700 1 -- 172.21.15.153:0/4227854133 >> 172.21.15.153:0/4227854133 conn(0x7fc2b80fb570 msgr2=0x7fc2b8105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:28:06.461 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:06.460+0000 7fc2c0400700 1 -- 172.21.15.153:0/4227854133 shutdown_connections 2024-03-21T10:28:06.461 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:06.460+0000 7fc2c0400700 1 -- 172.21.15.153:0/4227854133 wait complete. 2024-03-21T10:28:06.600 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:06.600+0000 7f97b2400700 1 -- 172.21.15.153:0/2199673567 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f97ac100970 msgr2=0x7f97ac100df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:06.601 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:06.600+0000 7f97b2400700 1 --2- 172.21.15.153:0/2199673567 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f97ac100970 0x7f97ac100df0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f9794009910 tx=0x7f9794009c20 comp rx=0 tx=0).stop 2024-03-21T10:28:06.601 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:06.600+0000 7f97b2400700 1 -- 172.21.15.153:0/2199673567 shutdown_connections 2024-03-21T10:28:06.601 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:06.600+0000 7f97b2400700 1 --2- 172.21.15.153:0/2199673567 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f97ac100970 0x7f97ac100df0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:06.601 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:06.600+0000 7f97b2400700 1 --2- 172.21.15.153:0/2199673567 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f97ac106970 0x7f97ac106d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:06.601 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:06.600+0000 7f97b2400700 1 -- 172.21.15.153:0/2199673567 >> 172.21.15.153:0/2199673567 conn(0x7f97ac0fc440 msgr2=0x7f97ac0fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:28:06.601 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:06.600+0000 7f97b2400700 1 -- 172.21.15.153:0/2199673567 shutdown_connections 2024-03-21T10:28:06.601 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:06.600+0000 7f97b2400700 1 -- 172.21.15.153:0/2199673567 wait complete. 2024-03-21T10:28:06.602 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:06.601+0000 7f97b2400700 1 Processor -- start 2024-03-21T10:28:06.603 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:06.601+0000 7f97b2400700 1 -- start start 2024-03-21T10:28:06.603 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:06.601+0000 7f97b2400700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f97ac100970 0x7f97ac073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:06.603 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:06.601+0000 7f97b2400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f97ac106970 0x7f97ac0735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:06.603 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:06.601+0000 7f97b2400700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97ac073b10 con 0x7f97ac106970 2024-03-21T10:28:06.603 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:06.602+0000 7f97b2400700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97ac073c50 con 0x7f97ac100970 2024-03-21T10:28:06.603 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:06.602+0000 7f97a3e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f97ac106970 0x7f97ac0735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:06.603 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:06.602+0000 7f97a3e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f97ac106970 0x7f97ac0735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:46706/0 (socket says 172.21.15.153:46706) 2024-03-21T10:28:06.603 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:06.602+0000 7f97a3e00700 1 -- 172.21.15.153:0/2234286650 learned_addr learned my addr 172.21.15.153:0/2234286650 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:28:06.603 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:06.602+0000 7f97b1000700 1 --2- 172.21.15.153:0/2234286650 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f97ac100970 0x7f97ac073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:06.603 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:06.602+0000 7f97a3e00700 1 -- 172.21.15.153:0/2234286650 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f97ac100970 msgr2=0x7f97ac073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:06.603 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:06.602+0000 7f97a3e00700 1 --2- 172.21.15.153:0/2234286650 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f97ac100970 0x7f97ac073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:06.603 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:06.602+0000 7f97a3e00700 1 -- 172.21.15.153:0/2234286650 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97940095f0 con 0x7f97ac106970 2024-03-21T10:28:06.603 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:06.602+0000 7f97b1000700 1 --2- 172.21.15.153:0/2234286650 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f97ac100970 0x7f97ac073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:28:06.603 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:06.602+0000 7f97a3e00700 1 --2- 172.21.15.153:0/2234286650 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f97ac106970 0x7f97ac0735d0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f9794006010 tx=0x7f979400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:28:06.604 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:06.603+0000 7f97aa000700 1 -- 172.21.15.153:0/2234286650 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f979401d070 con 0x7f97ac106970 2024-03-21T10:28:06.604 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:06.603+0000 7f97aa000700 1 -- 172.21.15.153:0/2234286650 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9794004fb0 con 0x7f97ac106970 2024-03-21T10:28:06.604 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:06.603+0000 7f97aa000700 1 -- 172.21.15.153:0/2234286650 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9794022ea0 con 0x7f97ac106970 2024-03-21T10:28:06.604 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:06.603+0000 7f97b2400700 1 -- 172.21.15.153:0/2234286650 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97ac073ed0 con 0x7f97ac106970 2024-03-21T10:28:06.605 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:06.603+0000 7f97b2400700 1 -- 172.21.15.153:0/2234286650 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97ac1a2a00 con 0x7f97ac106970 2024-03-21T10:28:06.605 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:06.604+0000 7f97b2400700 1 -- 172.21.15.153:0/2234286650 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97ac108f00 con 0x7f97ac106970 2024-03-21T10:28:06.605 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:06.605+0000 7f97aa000700 1 -- 172.21.15.153:0/2234286650 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f9794022470 con 0x7f97ac106970 2024-03-21T10:28:06.606 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:06.606+0000 7f97aa000700 1 --2- 172.21.15.153:0/2234286650 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f978c06e460 0x7f978c070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:06.606 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:06.606+0000 7f97aa000700 1 -- 172.21.15.153:0/2234286650 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f979408fa70 con 0x7f97ac106970 2024-03-21T10:28:06.606 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:06.606+0000 7f97b1000700 1 --2- 172.21.15.153:0/2234286650 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f978c06e460 0x7f978c070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:06.607 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:06.607+0000 7f97b1000700 1 --2- 172.21.15.153:0/2234286650 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f978c06e460 0x7f978c070920 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f979c005fd0 tx=0x7f979c005e20 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:28:06.610 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:06.610+0000 7f97aa000700 1 -- 172.21.15.153:0/2234286650 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f9794058ce0 con 0x7f97ac106970 2024-03-21T10:28:06.914 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:06.914+0000 7f97b2400700 1 -- 172.21.15.153:0/2234286650 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f97ac02cc70 con 0x7f97ac106970 2024-03-21T10:28:06.915 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:06.914+0000 7f97aa000700 1 -- 172.21.15.153:0/2234286650 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+805 (secure 0 0 0) 0x7f9794022720 con 0x7f97ac106970 2024-03-21T10:28:06.915 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:28:06.915 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": { 2024-03-21T10:28:06.915 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:28:06.915 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:28:06.915 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": { 2024-03-21T10:28:06.915 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:28:06.915 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:28:06.916 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": { 2024-03-21T10:28:06.916 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 6 2024-03-21T10:28:06.916 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:28:06.916 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": { 2024-03-21T10:28:06.916 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-03-21T10:28:06.916 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 3 2024-03-21T10:28:06.916 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:28:06.916 INFO:teuthology.orchestra.run.smithi153.stdout: "overall": { 2024-03-21T10:28:06.916 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-03-21T10:28:06.916 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 13 2024-03-21T10:28:06.916 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-03-21T10:28:06.916 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:28:06.918 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:06.918+0000 7f97b2400700 1 -- 172.21.15.153:0/2234286650 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f978c06e460 msgr2=0x7f978c070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:06.918 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:06.918+0000 7f97b2400700 1 --2- 172.21.15.153:0/2234286650 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f978c06e460 0x7f978c070920 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f979c005fd0 tx=0x7f979c005e20 comp rx=0 tx=0).stop 2024-03-21T10:28:06.918 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:06.918+0000 7f97b2400700 1 -- 172.21.15.153:0/2234286650 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f97ac106970 msgr2=0x7f97ac0735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:06.919 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:06.918+0000 7f97b2400700 1 --2- 172.21.15.153:0/2234286650 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f97ac106970 0x7f97ac0735d0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f9794006010 tx=0x7f979400be20 comp rx=0 tx=0).stop 2024-03-21T10:28:06.919 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:06.918+0000 7f97b2400700 1 -- 172.21.15.153:0/2234286650 shutdown_connections 2024-03-21T10:28:06.919 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:06.918+0000 7f97b2400700 1 --2- 172.21.15.153:0/2234286650 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f97ac100970 0x7f97ac073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:06.919 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:06.918+0000 7f97b2400700 1 --2- 172.21.15.153:0/2234286650 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f978c06e460 0x7f978c070920 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:06.919 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:06.918+0000 7f97b2400700 1 --2- 172.21.15.153:0/2234286650 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f97ac106970 0x7f97ac0735d0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:06.919 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:06.918+0000 7f97b2400700 1 -- 172.21.15.153:0/2234286650 >> 172.21.15.153:0/2234286650 conn(0x7f97ac0fc440 msgr2=0x7f97ac0fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:28:06.919 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:06.918+0000 7f97b2400700 1 -- 172.21.15.153:0/2234286650 shutdown_connections 2024-03-21T10:28:06.919 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:06.918+0000 7f97b2400700 1 -- 172.21.15.153:0/2234286650 wait complete. 2024-03-21T10:28:07.067 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.067+0000 7fdbd3200700 1 -- 172.21.15.153:0/1462466629 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdbcc073ae0 msgr2=0x7fdbcc10ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:07.067 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.067+0000 7fdbd3200700 1 --2- 172.21.15.153:0/1462466629 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdbcc073ae0 0x7fdbcc10ce40 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fdbb4009910 tx=0x7fdbb4009c20 comp rx=0 tx=0).stop 2024-03-21T10:28:07.068 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.067+0000 7fdbd3200700 1 -- 172.21.15.153:0/1462466629 shutdown_connections 2024-03-21T10:28:07.068 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.067+0000 7fdbd3200700 1 --2- 172.21.15.153:0/1462466629 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdbcc073ae0 0x7fdbcc10ce40 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:07.068 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.067+0000 7fdbd3200700 1 --2- 172.21.15.153:0/1462466629 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdbcc0731c0 0x7fdbcc0735a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:07.068 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.067+0000 7fdbd3200700 1 -- 172.21.15.153:0/1462466629 >> 172.21.15.153:0/1462466629 conn(0x7fdbcc0fc630 msgr2=0x7fdbcc0fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:28:07.068 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.067+0000 7fdbd3200700 1 -- 172.21.15.153:0/1462466629 shutdown_connections 2024-03-21T10:28:07.068 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.067+0000 7fdbd3200700 1 -- 172.21.15.153:0/1462466629 wait complete. 2024-03-21T10:28:07.069 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.068+0000 7fdbd3200700 1 Processor -- start 2024-03-21T10:28:07.069 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.068+0000 7fdbd3200700 1 -- start start 2024-03-21T10:28:07.069 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.068+0000 7fdbd3200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdbcc0731c0 0x7fdbcc198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:07.069 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.068+0000 7fdbd3200700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdbcc073ae0 0x7fdbcc198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:07.069 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.068+0000 7fdbd3200700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbcc199530 con 0x7fdbcc0731c0 2024-03-21T10:28:07.070 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.068+0000 7fdbd3200700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbcc19d340 con 0x7fdbcc073ae0 2024-03-21T10:28:07.070 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.069+0000 7fdbcbe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdbcc0731c0 0x7fdbcc198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:07.070 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.069+0000 7fdbcbe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdbcc0731c0 0x7fdbcc198960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:46718/0 (socket says 172.21.15.153:46718) 2024-03-21T10:28:07.070 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.069+0000 7fdbcbe00700 1 -- 172.21.15.153:0/1250178948 learned_addr learned my addr 172.21.15.153:0/1250178948 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:28:07.070 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.069+0000 7fdbc3400700 1 --2- 172.21.15.153:0/1250178948 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdbcc073ae0 0x7fdbcc198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:07.070 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.069+0000 7fdbcbe00700 1 -- 172.21.15.153:0/1250178948 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdbcc073ae0 msgr2=0x7fdbcc198ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:07.070 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.069+0000 7fdbcbe00700 1 --2- 172.21.15.153:0/1250178948 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdbcc073ae0 0x7fdbcc198ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:07.070 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.069+0000 7fdbcbe00700 1 -- 172.21.15.153:0/1250178948 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbb40095f0 con 0x7fdbcc0731c0 2024-03-21T10:28:07.071 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.069+0000 7fdbcbe00700 1 --2- 172.21.15.153:0/1250178948 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdbcc0731c0 0x7fdbcc198960 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fdbbc00ada0 tx=0x7fdbbc00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:28:07.071 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.070+0000 7fdbc9600700 1 -- 172.21.15.153:0/1250178948 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdbbc00f070 con 0x7fdbcc0731c0 2024-03-21T10:28:07.071 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.070+0000 7fdbc9600700 1 -- 172.21.15.153:0/1250178948 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdbbc007bf0 con 0x7fdbcc0731c0 2024-03-21T10:28:07.071 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.070+0000 7fdbc9600700 1 -- 172.21.15.153:0/1250178948 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdbbc005690 con 0x7fdbcc0731c0 2024-03-21T10:28:07.071 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.070+0000 7fdbd3200700 1 -- 172.21.15.153:0/1250178948 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbcc19d620 con 0x7fdbcc0731c0 2024-03-21T10:28:07.071 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.070+0000 7fdbd3200700 1 -- 172.21.15.153:0/1250178948 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbcc19db70 con 0x7fdbcc0731c0 2024-03-21T10:28:07.071 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.071+0000 7fdbd3200700 1 -- 172.21.15.153:0/1250178948 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbcc10a540 con 0x7fdbcc0731c0 2024-03-21T10:28:07.072 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.072+0000 7fdbc9600700 1 -- 172.21.15.153:0/1250178948 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fdbbc005850 con 0x7fdbcc0731c0 2024-03-21T10:28:07.073 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.073+0000 7fdbc9600700 1 --2- 172.21.15.153:0/1250178948 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fdbb006e460 0x7fdbb0070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:07.073 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.073+0000 7fdbc9600700 1 -- 172.21.15.153:0/1250178948 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fdbbc08df10 con 0x7fdbcc0731c0 2024-03-21T10:28:07.073 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.073+0000 7fdbc3400700 1 --2- 172.21.15.153:0/1250178948 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fdbb006e460 0x7fdbb0070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:07.074 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.074+0000 7fdbc3400700 1 --2- 172.21.15.153:0/1250178948 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fdbb006e460 0x7fdbb0070920 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fdbb4006010 tx=0x7fdbb400b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:28:07.077 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.077+0000 7fdbc9600700 1 -- 172.21.15.153:0/1250178948 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fdbbc05aae0 con 0x7fdbcc0731c0 2024-03-21T10:28:07.305 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:28:07 smithi153 ceph-mon[113733]: pgmap v487: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:28:07.305 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:28:07 smithi153 ceph-mon[113733]: from='client.34898 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:28:07.305 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:28:07 smithi153 ceph-mon[113733]: from='client.34902 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:28:07.305 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:28:07 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/2234286650' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:28:07.331 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.330+0000 7fdbd3200700 1 -- 172.21.15.153:0/1250178948 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fdbcc04ea90 con 0x7fdbcc0731c0 2024-03-21T10:28:07.331 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.331+0000 7fdbc9600700 1 -- 172.21.15.153:0/1250178948 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1876 (secure 0 0 0) 0x7fdbbc05a670 con 0x7fdbcc0731c0 2024-03-21T10:28:07.332 INFO:teuthology.orchestra.run.smithi153.stdout:e30 2024-03-21T10:28:07.332 INFO:teuthology.orchestra.run.smithi153.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-21T10:28:07.332 INFO:teuthology.orchestra.run.smithi153.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:28:07.332 INFO:teuthology.orchestra.run.smithi153.stdout:legacy client fscid: 1 2024-03-21T10:28:07.332 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:28:07.332 INFO:teuthology.orchestra.run.smithi153.stdout:Filesystem 'cephfs' (1) 2024-03-21T10:28:07.332 INFO:teuthology.orchestra.run.smithi153.stdout:fs_name cephfs 2024-03-21T10:28:07.332 INFO:teuthology.orchestra.run.smithi153.stdout:epoch 23 2024-03-21T10:28:07.333 INFO:teuthology.orchestra.run.smithi153.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-21T10:28:07.333 INFO:teuthology.orchestra.run.smithi153.stdout:created 2024-03-21T10:04:27.086576+0000 2024-03-21T10:28:07.333 INFO:teuthology.orchestra.run.smithi153.stdout:modified 2024-03-21T10:19:05.022092+0000 2024-03-21T10:28:07.333 INFO:teuthology.orchestra.run.smithi153.stdout:tableserver 0 2024-03-21T10:28:07.333 INFO:teuthology.orchestra.run.smithi153.stdout:root 0 2024-03-21T10:28:07.333 INFO:teuthology.orchestra.run.smithi153.stdout:session_timeout 60 2024-03-21T10:28:07.333 INFO:teuthology.orchestra.run.smithi153.stdout:session_autoclose 300 2024-03-21T10:28:07.333 INFO:teuthology.orchestra.run.smithi153.stdout:max_file_size 1099511627776 2024-03-21T10:28:07.333 INFO:teuthology.orchestra.run.smithi153.stdout:max_xattr_size 65536 2024-03-21T10:28:07.333 INFO:teuthology.orchestra.run.smithi153.stdout:required_client_features {} 2024-03-21T10:28:07.333 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure 0 2024-03-21T10:28:07.333 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure_osd_epoch 94 2024-03-21T10:28:07.333 INFO:teuthology.orchestra.run.smithi153.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:28:07.334 INFO:teuthology.orchestra.run.smithi153.stdout:max_mds 2 2024-03-21T10:28:07.334 INFO:teuthology.orchestra.run.smithi153.stdout:in 0,1 2024-03-21T10:28:07.334 INFO:teuthology.orchestra.run.smithi153.stdout:up {} 2024-03-21T10:28:07.334 INFO:teuthology.orchestra.run.smithi153.stdout:failed 0,1 2024-03-21T10:28:07.334 INFO:teuthology.orchestra.run.smithi153.stdout:damaged 2024-03-21T10:28:07.334 INFO:teuthology.orchestra.run.smithi153.stdout:stopped 2024-03-21T10:28:07.334 INFO:teuthology.orchestra.run.smithi153.stdout:data_pools [3] 2024-03-21T10:28:07.334 INFO:teuthology.orchestra.run.smithi153.stdout:metadata_pool 2 2024-03-21T10:28:07.334 INFO:teuthology.orchestra.run.smithi153.stdout:inline_data enabled 2024-03-21T10:28:07.334 INFO:teuthology.orchestra.run.smithi153.stdout:balancer 2024-03-21T10:28:07.334 INFO:teuthology.orchestra.run.smithi153.stdout:bal_rank_mask -1 2024-03-21T10:28:07.334 INFO:teuthology.orchestra.run.smithi153.stdout:standby_count_wanted 1 2024-03-21T10:28:07.335 INFO:teuthology.orchestra.run.smithi153.stdout:qdb_cluster leader: 0 members: 2024-03-21T10:28:07.335 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:28:07.335 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:28:07.335 INFO:teuthology.orchestra.run.smithi153.stdout:Standby daemons: 2024-03-21T10:28:07.335 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:28:07.335 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.zipckr{-1:34442} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6826/1103846295,v1:172.21.15.153:6827/1103846295] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:28:07.335 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.iuyluj{-1:34474} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6828/1738630832,v1:172.21.15.153:6829/1738630832] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:28:07.335 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.nhozag{-1:44269} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.174:6826/3967223227,v1:172.21.15.174:6827/3967223227] compat {c=[1],r=[1],i=[7ff]}] 2024-03-21T10:28:07.335 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.yyimst{-1:44313} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.174:6824/4127786797,v1:172.21.15.174:6825/4127786797] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:28:07.336 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.334+0000 7fdbd3200700 1 -- 172.21.15.153:0/1250178948 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fdbb006e460 msgr2=0x7fdbb0070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:07.336 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.334+0000 7fdbd3200700 1 --2- 172.21.15.153:0/1250178948 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fdbb006e460 0x7fdbb0070920 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fdbb4006010 tx=0x7fdbb400b540 comp rx=0 tx=0).stop 2024-03-21T10:28:07.336 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.334+0000 7fdbd3200700 1 -- 172.21.15.153:0/1250178948 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdbcc0731c0 msgr2=0x7fdbcc198960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:07.336 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.334+0000 7fdbd3200700 1 --2- 172.21.15.153:0/1250178948 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdbcc0731c0 0x7fdbcc198960 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fdbbc00ada0 tx=0x7fdbbc00c330 comp rx=0 tx=0).stop 2024-03-21T10:28:07.336 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.334+0000 7fdbd3200700 1 -- 172.21.15.153:0/1250178948 shutdown_connections 2024-03-21T10:28:07.336 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.334+0000 7fdbd3200700 1 --2- 172.21.15.153:0/1250178948 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fdbb006e460 0x7fdbb0070920 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:07.336 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.334+0000 7fdbd3200700 1 --2- 172.21.15.153:0/1250178948 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdbcc0731c0 0x7fdbcc198960 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:07.336 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.334+0000 7fdbd3200700 1 --2- 172.21.15.153:0/1250178948 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdbcc073ae0 0x7fdbcc198ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:07.336 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.334+0000 7fdbd3200700 1 -- 172.21.15.153:0/1250178948 >> 172.21.15.153:0/1250178948 conn(0x7fdbcc0fc630 msgr2=0x7fdbcc107680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:28:07.337 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.334+0000 7fdbd3200700 1 -- 172.21.15.153:0/1250178948 shutdown_connections 2024-03-21T10:28:07.337 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.334+0000 7fdbd3200700 1 -- 172.21.15.153:0/1250178948 wait complete. 2024-03-21T10:28:07.337 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 30 2024-03-21T10:28:07.450 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:28:07 smithi174 ceph-mon[88004]: pgmap v487: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:28:07.450 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:28:07 smithi174 ceph-mon[88004]: from='client.34898 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:28:07.450 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:28:07 smithi174 ceph-mon[88004]: from='client.34902 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:28:07.450 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:28:07 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/2234286650' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:28:07.485 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.485+0000 7efc35c00700 1 -- 172.21.15.153:0/192980003 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efc30102e10 msgr2=0x7efc301031f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:07.486 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.485+0000 7efc35c00700 1 --2- 172.21.15.153:0/192980003 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efc30102e10 0x7efc301031f0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7efc20009910 tx=0x7efc20009c20 comp rx=0 tx=0).stop 2024-03-21T10:28:07.486 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.485+0000 7efc35c00700 1 -- 172.21.15.153:0/192980003 shutdown_connections 2024-03-21T10:28:07.486 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.485+0000 7efc35c00700 1 --2- 172.21.15.153:0/192980003 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efc301037c0 0x7efc30107810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:07.486 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.485+0000 7efc35c00700 1 --2- 172.21.15.153:0/192980003 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efc30102e10 0x7efc301031f0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:07.486 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.485+0000 7efc35c00700 1 -- 172.21.15.153:0/192980003 >> 172.21.15.153:0/192980003 conn(0x7efc300fe6c0 msgr2=0x7efc30100ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:28:07.486 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.485+0000 7efc35c00700 1 -- 172.21.15.153:0/192980003 shutdown_connections 2024-03-21T10:28:07.486 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.485+0000 7efc35c00700 1 -- 172.21.15.153:0/192980003 wait complete. 2024-03-21T10:28:07.487 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.486+0000 7efc35c00700 1 Processor -- start 2024-03-21T10:28:07.487 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.486+0000 7efc35c00700 1 -- start start 2024-03-21T10:28:07.487 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.486+0000 7efc35c00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efc30102e10 0x7efc3019e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:07.488 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.486+0000 7efc35c00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efc301037c0 0x7efc3019eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:07.488 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.487+0000 7efc35c00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc3019f210 con 0x7efc301037c0 2024-03-21T10:28:07.488 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.487+0000 7efc35c00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc301986c0 con 0x7efc30102e10 2024-03-21T10:28:07.488 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.487+0000 7efc2f400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efc301037c0 0x7efc3019eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:07.488 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.487+0000 7efc2f400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efc301037c0 0x7efc3019eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:46738/0 (socket says 172.21.15.153:46738) 2024-03-21T10:28:07.488 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.487+0000 7efc2f400700 1 -- 172.21.15.153:0/3314781842 learned_addr learned my addr 172.21.15.153:0/3314781842 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:28:07.488 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.487+0000 7efc2fe00700 1 --2- 172.21.15.153:0/3314781842 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efc30102e10 0x7efc3019e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:07.489 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.487+0000 7efc2f400700 1 -- 172.21.15.153:0/3314781842 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efc30102e10 msgr2=0x7efc3019e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:07.489 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.487+0000 7efc2f400700 1 --2- 172.21.15.153:0/3314781842 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efc30102e10 0x7efc3019e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:07.489 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.487+0000 7efc2f400700 1 -- 172.21.15.153:0/3314781842 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc200095f0 con 0x7efc301037c0 2024-03-21T10:28:07.489 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.487+0000 7efc2fe00700 1 --2- 172.21.15.153:0/3314781842 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efc30102e10 0x7efc3019e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:28:07.489 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.488+0000 7efc2f400700 1 --2- 172.21.15.153:0/3314781842 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efc301037c0 0x7efc3019eb80 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7efc1c00b480 tx=0x7efc1c00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:28:07.489 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.488+0000 7efc2cc00700 1 -- 172.21.15.153:0/3314781842 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efc1c0118c0 con 0x7efc301037c0 2024-03-21T10:28:07.489 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.488+0000 7efc2cc00700 1 -- 172.21.15.153:0/3314781842 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efc1c010380 con 0x7efc301037c0 2024-03-21T10:28:07.489 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.488+0000 7efc35c00700 1 -- 172.21.15.153:0/3314781842 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc301989a0 con 0x7efc301037c0 2024-03-21T10:28:07.489 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.488+0000 7efc2cc00700 1 -- 172.21.15.153:0/3314781842 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efc1c00f4f0 con 0x7efc301037c0 2024-03-21T10:28:07.489 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.488+0000 7efc35c00700 1 -- 172.21.15.153:0/3314781842 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc30198ef0 con 0x7efc301037c0 2024-03-21T10:28:07.490 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.489+0000 7efc35c00700 1 -- 172.21.15.153:0/3314781842 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc3010b1b0 con 0x7efc301037c0 2024-03-21T10:28:07.491 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.491+0000 7efc2cc00700 1 -- 172.21.15.153:0/3314781842 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7efc1c0104f0 con 0x7efc301037c0 2024-03-21T10:28:07.492 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.492+0000 7efc2cc00700 1 --2- 172.21.15.153:0/3314781842 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7efc1406e460 0x7efc14070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:07.492 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.492+0000 7efc2cc00700 1 -- 172.21.15.153:0/3314781842 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7efc1c08e120 con 0x7efc301037c0 2024-03-21T10:28:07.493 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.492+0000 7efc2fe00700 1 --2- 172.21.15.153:0/3314781842 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7efc1406e460 0x7efc14070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:07.493 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.493+0000 7efc2fe00700 1 --2- 172.21.15.153:0/3314781842 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7efc1406e460 0x7efc14070920 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7efc20005100 tx=0x7efc2001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:28:07.497 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.497+0000 7efc2cc00700 1 -- 172.21.15.153:0/3314781842 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7efc1c05acf0 con 0x7efc301037c0 2024-03-21T10:28:07.727 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.726+0000 7efc35c00700 1 -- 172.21.15.153:0/3314781842 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7efc301992a0 con 0x7efc1406e460 2024-03-21T10:28:07.729 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.729+0000 7efc2cc00700 1 -- 172.21.15.153:0/3314781842 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7efc301992a0 con 0x7efc1406e460 2024-03-21T10:28:07.730 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:28:07.730 INFO:teuthology.orchestra.run.smithi153.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232", 2024-03-21T10:28:07.730 INFO:teuthology.orchestra.run.smithi153.stdout: "in_progress": true, 2024-03-21T10:28:07.730 INFO:teuthology.orchestra.run.smithi153.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-21T10:28:07.730 INFO:teuthology.orchestra.run.smithi153.stdout: "services_complete": [ 2024-03-21T10:28:07.730 INFO:teuthology.orchestra.run.smithi153.stdout: "crash", 2024-03-21T10:28:07.730 INFO:teuthology.orchestra.run.smithi153.stdout: "mon", 2024-03-21T10:28:07.730 INFO:teuthology.orchestra.run.smithi153.stdout: "osd", 2024-03-21T10:28:07.730 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr" 2024-03-21T10:28:07.730 INFO:teuthology.orchestra.run.smithi153.stdout: ], 2024-03-21T10:28:07.730 INFO:teuthology.orchestra.run.smithi153.stdout: "progress": "15/23 daemons upgraded", 2024-03-21T10:28:07.730 INFO:teuthology.orchestra.run.smithi153.stdout: "message": "Currently upgrading mds daemons", 2024-03-21T10:28:07.730 INFO:teuthology.orchestra.run.smithi153.stdout: "is_paused": false 2024-03-21T10:28:07.731 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:28:07.733 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.733+0000 7efc35c00700 1 -- 172.21.15.153:0/3314781842 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7efc1406e460 msgr2=0x7efc14070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:07.733 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.733+0000 7efc35c00700 1 --2- 172.21.15.153:0/3314781842 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7efc1406e460 0x7efc14070920 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7efc20005100 tx=0x7efc2001a040 comp rx=0 tx=0).stop 2024-03-21T10:28:07.733 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.733+0000 7efc35c00700 1 -- 172.21.15.153:0/3314781842 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efc301037c0 msgr2=0x7efc3019eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:07.733 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.733+0000 7efc35c00700 1 --2- 172.21.15.153:0/3314781842 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efc301037c0 0x7efc3019eb80 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7efc1c00b480 tx=0x7efc1c00b840 comp rx=0 tx=0).stop 2024-03-21T10:28:07.734 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.733+0000 7efc35c00700 1 -- 172.21.15.153:0/3314781842 shutdown_connections 2024-03-21T10:28:07.734 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.733+0000 7efc35c00700 1 --2- 172.21.15.153:0/3314781842 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efc30102e10 0x7efc3019e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:07.734 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.733+0000 7efc35c00700 1 --2- 172.21.15.153:0/3314781842 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7efc1406e460 0x7efc14070920 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:07.734 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.733+0000 7efc35c00700 1 --2- 172.21.15.153:0/3314781842 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efc301037c0 0x7efc3019eb80 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:07.734 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.733+0000 7efc35c00700 1 -- 172.21.15.153:0/3314781842 >> 172.21.15.153:0/3314781842 conn(0x7efc300fe6c0 msgr2=0x7efc300ffe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:28:07.734 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.733+0000 7efc35c00700 1 -- 172.21.15.153:0/3314781842 shutdown_connections 2024-03-21T10:28:07.734 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.734+0000 7efc35c00700 1 -- 172.21.15.153:0/3314781842 wait complete. 2024-03-21T10:28:07.883 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.882+0000 7f3f5d600700 1 -- 172.21.15.153:0/246370788 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3f581071d0 msgr2=0x7f3f58107650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:07.883 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.882+0000 7f3f5d600700 1 --2- 172.21.15.153:0/246370788 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3f581071d0 0x7f3f58107650 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f3f44009910 tx=0x7f3f44009c20 comp rx=0 tx=0).stop 2024-03-21T10:28:07.883 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.882+0000 7f3f5d600700 1 -- 172.21.15.153:0/246370788 shutdown_connections 2024-03-21T10:28:07.883 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.882+0000 7f3f5d600700 1 --2- 172.21.15.153:0/246370788 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3f581071d0 0x7f3f58107650 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:07.883 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.882+0000 7f3f5d600700 1 --2- 172.21.15.153:0/246370788 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3f5810d280 0x7f3f5810d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:07.883 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.882+0000 7f3f5d600700 1 -- 172.21.15.153:0/246370788 >> 172.21.15.153:0/246370788 conn(0x7f3f580769e0 msgr2=0x7f3f58078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:28:07.883 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.882+0000 7f3f5d600700 1 -- 172.21.15.153:0/246370788 shutdown_connections 2024-03-21T10:28:07.883 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.882+0000 7f3f5d600700 1 -- 172.21.15.153:0/246370788 wait complete. 2024-03-21T10:28:07.884 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.883+0000 7f3f5d600700 1 Processor -- start 2024-03-21T10:28:07.884 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.883+0000 7f3f5d600700 1 -- start start 2024-03-21T10:28:07.885 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.883+0000 7f3f5d600700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3f581071d0 0x7f3f5819ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:07.885 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.883+0000 7f3f5d600700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3f5810d280 0x7f3f5819d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:07.885 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.883+0000 7f3f5d600700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f5819d8a0 con 0x7f3f581071d0 2024-03-21T10:28:07.885 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.883+0000 7f3f5d600700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f581a05d0 con 0x7f3f5810d280 2024-03-21T10:28:07.885 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.884+0000 7f3f56a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3f581071d0 0x7f3f5819ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:07.885 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.884+0000 7f3f56a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3f581071d0 0x7f3f5819ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:46748/0 (socket says 172.21.15.153:46748) 2024-03-21T10:28:07.885 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.884+0000 7f3f56a00700 1 -- 172.21.15.153:0/659092963 learned_addr learned my addr 172.21.15.153:0/659092963 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:28:07.885 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.884+0000 7f3f56a00700 1 -- 172.21.15.153:0/659092963 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3f5810d280 msgr2=0x7f3f5819d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:07.885 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.884+0000 7f3f56a00700 1 --2- 172.21.15.153:0/659092963 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3f5810d280 0x7f3f5819d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:07.885 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.884+0000 7f3f56a00700 1 -- 172.21.15.153:0/659092963 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f440095f0 con 0x7f3f581071d0 2024-03-21T10:28:07.885 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.884+0000 7f3f56a00700 1 --2- 172.21.15.153:0/659092963 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3f581071d0 0x7f3f5819ccd0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f3f4800d900 tx=0x7f3f4800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:28:07.886 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.885+0000 7f3f4f400700 1 -- 172.21.15.153:0/659092963 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3f48004a00 con 0x7f3f581071d0 2024-03-21T10:28:07.886 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.885+0000 7f3f4f400700 1 -- 172.21.15.153:0/659092963 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3f480054e0 con 0x7f3f581071d0 2024-03-21T10:28:07.886 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.885+0000 7f3f4f400700 1 -- 172.21.15.153:0/659092963 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3f48009d60 con 0x7f3f581071d0 2024-03-21T10:28:07.886 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.885+0000 7f3f5d600700 1 -- 172.21.15.153:0/659092963 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f581a08b0 con 0x7f3f581071d0 2024-03-21T10:28:07.886 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.885+0000 7f3f5d600700 1 -- 172.21.15.153:0/659092963 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f581a0e80 con 0x7f3f581071d0 2024-03-21T10:28:07.886 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.886+0000 7f3f5d600700 1 -- 172.21.15.153:0/659092963 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f5804ea90 con 0x7f3f581071d0 2024-03-21T10:28:07.887 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.887+0000 7f3f4f400700 1 -- 172.21.15.153:0/659092963 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f3f48005020 con 0x7f3f581071d0 2024-03-21T10:28:07.889 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.888+0000 7f3f4f400700 1 --2- 172.21.15.153:0/659092963 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f3f3c06e460 0x7f3f3c070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:07.889 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.888+0000 7f3f4f400700 1 -- 172.21.15.153:0/659092963 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f3f4808e650 con 0x7f3f581071d0 2024-03-21T10:28:07.889 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.889+0000 7f3f56000700 1 --2- 172.21.15.153:0/659092963 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f3f3c06e460 0x7f3f3c070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:07.889 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.889+0000 7f3f56000700 1 --2- 172.21.15.153:0/659092963 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f3f3c06e460 0x7f3f3c070920 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f3f44006010 tx=0x7f3f4400b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:28:07.893 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:07.893+0000 7f3f4f400700 1 -- 172.21.15.153:0/659092963 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f3f48057790 con 0x7f3f581071d0 2024-03-21T10:28:08.200 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:08.199+0000 7f3f5d600700 1 -- 172.21.15.153:0/659092963 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f3f5819dff0 con 0x7f3f581071d0 2024-03-21T10:28:08.200 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:28:08 smithi174 ceph-mon[88004]: from='client.34906 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:28:08.200 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:28:08 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/1250178948' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:28:08.201 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:08.200+0000 7f3f4f400700 1 -- 172.21.15.153:0/659092963 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f3f480171c0 con 0x7f3f581071d0 2024-03-21T10:28:08.201 INFO:teuthology.orchestra.run.smithi153.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-03-21T10:28:08.201 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-21T10:28:08.201 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is degraded 2024-03-21T10:28:08.201 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-21T10:28:08.201 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-21T10:28:08.201 INFO:teuthology.orchestra.run.smithi153.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-21T10:28:08.201 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-21T10:28:08.203 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:08.203+0000 7f3f5d600700 1 -- 172.21.15.153:0/659092963 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f3f3c06e460 msgr2=0x7f3f3c070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:08.203 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:08.203+0000 7f3f5d600700 1 --2- 172.21.15.153:0/659092963 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f3f3c06e460 0x7f3f3c070920 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f3f44006010 tx=0x7f3f4400b540 comp rx=0 tx=0).stop 2024-03-21T10:28:08.203 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:08.203+0000 7f3f5d600700 1 -- 172.21.15.153:0/659092963 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3f581071d0 msgr2=0x7f3f5819ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:08.204 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:08.203+0000 7f3f5d600700 1 --2- 172.21.15.153:0/659092963 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3f581071d0 0x7f3f5819ccd0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f3f4800d900 tx=0x7f3f4800dcc0 comp rx=0 tx=0).stop 2024-03-21T10:28:08.204 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:08.203+0000 7f3f5d600700 1 -- 172.21.15.153:0/659092963 shutdown_connections 2024-03-21T10:28:08.204 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:08.203+0000 7f3f5d600700 1 --2- 172.21.15.153:0/659092963 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f3f3c06e460 0x7f3f3c070920 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:08.204 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:08.203+0000 7f3f5d600700 1 --2- 172.21.15.153:0/659092963 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3f581071d0 0x7f3f5819ccd0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:08.204 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:08.203+0000 7f3f5d600700 1 --2- 172.21.15.153:0/659092963 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3f5810d280 0x7f3f5819d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:08.204 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:08.203+0000 7f3f5d600700 1 -- 172.21.15.153:0/659092963 >> 172.21.15.153:0/659092963 conn(0x7f3f580769e0 msgr2=0x7f3f580783c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:28:08.204 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:08.203+0000 7f3f5d600700 1 -- 172.21.15.153:0/659092963 shutdown_connections 2024-03-21T10:28:08.204 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:08.203+0000 7f3f5d600700 1 -- 172.21.15.153:0/659092963 wait complete. 2024-03-21T10:28:08.305 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:28:08 smithi153 ceph-mon[113733]: from='client.34906 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:28:08.305 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:28:08 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/1250178948' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:28:09.450 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:28:09 smithi174 ceph-mon[88004]: pgmap v488: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:28:09.450 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:28:09 smithi174 ceph-mon[88004]: from='client.34918 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:28:09.450 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:28:09 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/659092963' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:28:09.555 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:28:09 smithi153 ceph-mon[113733]: pgmap v488: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:28:09.555 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:28:09 smithi153 ceph-mon[113733]: from='client.34918 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:28:09.555 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:28:09 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/659092963' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:28:11.450 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:28:11 smithi174 ceph-mon[88004]: pgmap v489: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:28:11.555 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:28:11 smithi153 ceph-mon[113733]: pgmap v489: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:28:13.450 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:28:13 smithi174 ceph-mon[88004]: pgmap v490: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:28:13.555 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:28:13 smithi153 ceph-mon[113733]: pgmap v490: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:28:15.450 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:28:15 smithi174 ceph-mon[88004]: pgmap v491: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:28:15.555 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:28:15 smithi153 ceph-mon[113733]: pgmap v491: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:28:17.450 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:28:17 smithi174 ceph-mon[88004]: pgmap v492: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:28:17.450 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:28:17 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:28:17.555 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:28:17 smithi153 ceph-mon[113733]: pgmap v492: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:28:17.555 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:28:17 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:28:19.390 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:28:19 smithi174 ceph-mon[88004]: pgmap v493: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:28:19.555 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:28:19 smithi153 ceph-mon[113733]: pgmap v493: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:28:21.450 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:28:21 smithi174 ceph-mon[88004]: pgmap v494: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:28:21.555 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:28:21 smithi153 ceph-mon[113733]: pgmap v494: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:28:23.450 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:28:23 smithi174 ceph-mon[88004]: pgmap v495: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:28:23.555 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:28:23 smithi153 ceph-mon[113733]: pgmap v495: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:28:25.450 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:28:25 smithi174 ceph-mon[88004]: pgmap v496: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:28:25.555 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:28:25 smithi153 ceph-mon[113733]: pgmap v496: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:28:27.450 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:28:27 smithi174 ceph-mon[88004]: pgmap v497: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:28:27.555 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:28:27 smithi153 ceph-mon[113733]: pgmap v497: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:28:29.450 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:28:29 smithi174 ceph-mon[88004]: pgmap v498: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:28:29.555 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:28:29 smithi153 ceph-mon[113733]: pgmap v498: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:28:31.450 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:28:31 smithi174 ceph-mon[88004]: pgmap v499: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:28:31.555 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:28:31 smithi153 ceph-mon[113733]: pgmap v499: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:28:32.305 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:28:32 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:28:32.450 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:28:32 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:28:33.450 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:28:33 smithi174 ceph-mon[88004]: pgmap v500: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:28:33.555 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:28:33 smithi153 ceph-mon[113733]: pgmap v500: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:28:35.450 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:28:35 smithi174 ceph-mon[88004]: pgmap v501: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:28:35.555 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:28:35 smithi153 ceph-mon[113733]: pgmap v501: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:28:37.450 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:28:37 smithi174 ceph-mon[88004]: pgmap v502: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:28:37.555 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:28:37 smithi153 ceph-mon[113733]: pgmap v502: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:28:38.373 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:38.372+0000 7fcc5b400700 1 -- 172.21.15.153:0/1094533570 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcc541039b0 msgr2=0x7fcc54107a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:38.373 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:38.372+0000 7fcc5b400700 1 --2- 172.21.15.153:0/1094533570 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcc541039b0 0x7fcc54107a00 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fcc40009910 tx=0x7fcc40009c20 comp rx=0 tx=0).stop 2024-03-21T10:28:38.373 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:38.373+0000 7fcc5b400700 1 -- 172.21.15.153:0/1094533570 shutdown_connections 2024-03-21T10:28:38.373 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:38.373+0000 7fcc5b400700 1 --2- 172.21.15.153:0/1094533570 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcc541039b0 0x7fcc54107a00 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:38.373 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:38.373+0000 7fcc5b400700 1 --2- 172.21.15.153:0/1094533570 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcc54103000 0x7fcc541033e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:38.373 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:38.373+0000 7fcc5b400700 1 -- 172.21.15.153:0/1094533570 >> 172.21.15.153:0/1094533570 conn(0x7fcc540fe8d0 msgr2=0x7fcc54100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:28:38.373 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:38.373+0000 7fcc5b400700 1 -- 172.21.15.153:0/1094533570 shutdown_connections 2024-03-21T10:28:38.374 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:38.373+0000 7fcc5b400700 1 -- 172.21.15.153:0/1094533570 wait complete. 2024-03-21T10:28:38.375 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:38.374+0000 7fcc5b400700 1 Processor -- start 2024-03-21T10:28:38.375 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:38.374+0000 7fcc5b400700 1 -- start start 2024-03-21T10:28:38.375 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:38.374+0000 7fcc5b400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcc54103000 0x7fcc5419e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:38.375 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:38.374+0000 7fcc5b400700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcc541039b0 0x7fcc5419ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:38.375 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:38.374+0000 7fcc5b400700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc5419f300 con 0x7fcc54103000 2024-03-21T10:28:38.375 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:38.374+0000 7fcc5b400700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc54198840 con 0x7fcc541039b0 2024-03-21T10:28:38.375 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:38.375+0000 7fcc58a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcc54103000 0x7fcc5419e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:38.375 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:38.375+0000 7fcc58a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcc54103000 0x7fcc5419e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:60084/0 (socket says 172.21.15.153:60084) 2024-03-21T10:28:38.375 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:38.375+0000 7fcc58a00700 1 -- 172.21.15.153:0/4048087896 learned_addr learned my addr 172.21.15.153:0/4048087896 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:28:38.376 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:38.375+0000 7fcc58a00700 1 -- 172.21.15.153:0/4048087896 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcc541039b0 msgr2=0x7fcc5419ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:38.376 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:38.375+0000 7fcc53e00700 1 --2- 172.21.15.153:0/4048087896 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcc541039b0 0x7fcc5419ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:38.376 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:38.375+0000 7fcc58a00700 1 --2- 172.21.15.153:0/4048087896 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcc541039b0 0x7fcc5419ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:38.377 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:38.375+0000 7fcc58a00700 1 -- 172.21.15.153:0/4048087896 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc400095f0 con 0x7fcc54103000 2024-03-21T10:28:38.377 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:38.375+0000 7fcc53e00700 1 --2- 172.21.15.153:0/4048087896 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcc541039b0 0x7fcc5419ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:28:38.377 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:38.376+0000 7fcc58a00700 1 --2- 172.21.15.153:0/4048087896 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcc54103000 0x7fcc5419e7c0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fcc4800c930 tx=0x7fcc4800ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:28:38.377 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:38.376+0000 7fcc51600700 1 -- 172.21.15.153:0/4048087896 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcc48007ad0 con 0x7fcc54103000 2024-03-21T10:28:38.377 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:38.376+0000 7fcc51600700 1 -- 172.21.15.153:0/4048087896 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcc48007c30 con 0x7fcc54103000 2024-03-21T10:28:38.377 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:38.376+0000 7fcc51600700 1 -- 172.21.15.153:0/4048087896 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcc48018610 con 0x7fcc54103000 2024-03-21T10:28:38.377 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:38.376+0000 7fcc5b400700 1 -- 172.21.15.153:0/4048087896 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc54198b20 con 0x7fcc54103000 2024-03-21T10:28:38.377 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:38.376+0000 7fcc5b400700 1 -- 172.21.15.153:0/4048087896 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc54199070 con 0x7fcc54103000 2024-03-21T10:28:38.377 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:38.377+0000 7fcc5b400700 1 -- 172.21.15.153:0/4048087896 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc5410b400 con 0x7fcc54103000 2024-03-21T10:28:38.378 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:38.378+0000 7fcc51600700 1 -- 172.21.15.153:0/4048087896 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fcc4800f450 con 0x7fcc54103000 2024-03-21T10:28:38.379 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:38.379+0000 7fcc51600700 1 --2- 172.21.15.153:0/4048087896 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fcc3c06e460 0x7fcc3c070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:38.379 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:38.379+0000 7fcc51600700 1 -- 172.21.15.153:0/4048087896 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fcc4808e6a0 con 0x7fcc54103000 2024-03-21T10:28:38.379 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:38.379+0000 7fcc53e00700 1 --2- 172.21.15.153:0/4048087896 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fcc3c06e460 0x7fcc3c070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:38.380 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:38.380+0000 7fcc53e00700 1 --2- 172.21.15.153:0/4048087896 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fcc3c06e460 0x7fcc3c070920 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fcc40006010 tx=0x7fcc4000b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:28:38.384 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:38.384+0000 7fcc51600700 1 -- 172.21.15.153:0/4048087896 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fcc4805b270 con 0x7fcc54103000 2024-03-21T10:28:38.626 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:38.625+0000 7fcc5b400700 1 -- 172.21.15.153:0/4048087896 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcc541992f0 con 0x7fcc3c06e460 2024-03-21T10:28:38.627 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:38.627+0000 7fcc51600700 1 -- 172.21.15.153:0/4048087896 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fcc541992f0 con 0x7fcc3c06e460 2024-03-21T10:28:38.630 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:38.630+0000 7fcc5b400700 1 -- 172.21.15.153:0/4048087896 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fcc3c06e460 msgr2=0x7fcc3c070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:38.630 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:38.630+0000 7fcc5b400700 1 --2- 172.21.15.153:0/4048087896 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fcc3c06e460 0x7fcc3c070920 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fcc40006010 tx=0x7fcc4000b540 comp rx=0 tx=0).stop 2024-03-21T10:28:38.630 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:38.630+0000 7fcc5b400700 1 -- 172.21.15.153:0/4048087896 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcc54103000 msgr2=0x7fcc5419e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:38.630 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:38.630+0000 7fcc5b400700 1 --2- 172.21.15.153:0/4048087896 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcc54103000 0x7fcc5419e7c0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fcc4800c930 tx=0x7fcc4800ccf0 comp rx=0 tx=0).stop 2024-03-21T10:28:38.630 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:38.630+0000 7fcc5b400700 1 -- 172.21.15.153:0/4048087896 shutdown_connections 2024-03-21T10:28:38.630 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:38.630+0000 7fcc5b400700 1 --2- 172.21.15.153:0/4048087896 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fcc3c06e460 0x7fcc3c070920 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:38.631 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:38.630+0000 7fcc5b400700 1 --2- 172.21.15.153:0/4048087896 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcc54103000 0x7fcc5419e7c0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:38.631 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:38.630+0000 7fcc5b400700 1 --2- 172.21.15.153:0/4048087896 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcc541039b0 0x7fcc5419ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:38.631 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:38.630+0000 7fcc5b400700 1 -- 172.21.15.153:0/4048087896 >> 172.21.15.153:0/4048087896 conn(0x7fcc540fe8d0 msgr2=0x7fcc54100cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:28:38.631 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:38.630+0000 7fcc5b400700 1 -- 172.21.15.153:0/4048087896 shutdown_connections 2024-03-21T10:28:38.631 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:38.630+0000 7fcc5b400700 1 -- 172.21.15.153:0/4048087896 wait complete. 2024-03-21T10:28:38.645 INFO:teuthology.orchestra.run.smithi153.stdout:true 2024-03-21T10:28:38.788 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:38.788+0000 7f41c0600700 1 -- 172.21.15.153:0/2237028334 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f41b8101d00 msgr2=0x7f41b81020e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:38.788 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:38.788+0000 7f41c0600700 1 --2- 172.21.15.153:0/2237028334 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f41b8101d00 0x7f41b81020e0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f41ac009910 tx=0x7f41ac009c20 comp rx=0 tx=0).stop 2024-03-21T10:28:38.788 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:38.788+0000 7f41c0600700 1 -- 172.21.15.153:0/2237028334 shutdown_connections 2024-03-21T10:28:38.789 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:38.788+0000 7f41c0600700 1 --2- 172.21.15.153:0/2237028334 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f41b8102620 0x7f41b810ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:38.789 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:38.788+0000 7f41c0600700 1 --2- 172.21.15.153:0/2237028334 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f41b8101d00 0x7f41b81020e0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:38.789 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:38.788+0000 7f41c0600700 1 -- 172.21.15.153:0/2237028334 >> 172.21.15.153:0/2237028334 conn(0x7f41b80fb570 msgr2=0x7f41b80fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:28:38.789 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:38.788+0000 7f41c0600700 1 -- 172.21.15.153:0/2237028334 shutdown_connections 2024-03-21T10:28:38.789 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:38.788+0000 7f41c0600700 1 -- 172.21.15.153:0/2237028334 wait complete. 2024-03-21T10:28:38.790 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:38.789+0000 7f41c0600700 1 Processor -- start 2024-03-21T10:28:38.790 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:38.789+0000 7f41c0600700 1 -- start start 2024-03-21T10:28:38.790 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:38.790+0000 7f41c0600700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f41b8101d00 0x7f41b819c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:38.791 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:38.790+0000 7f41c0600700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f41b8102620 0x7f41b819ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:38.791 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:38.790+0000 7f41c0600700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41b819d070 con 0x7f41b8101d00 2024-03-21T10:28:38.791 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:38.790+0000 7f41c0600700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41b81965b0 con 0x7f41b8102620 2024-03-21T10:28:38.791 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:38.790+0000 7f41bdc00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f41b8101d00 0x7f41b819c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:38.791 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:38.790+0000 7f41bdc00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f41b8101d00 0x7f41b819c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:60098/0 (socket says 172.21.15.153:60098) 2024-03-21T10:28:38.791 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:38.790+0000 7f41bdc00700 1 -- 172.21.15.153:0/1968105433 learned_addr learned my addr 172.21.15.153:0/1968105433 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:28:38.791 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:38.790+0000 7f41bd200700 1 --2- 172.21.15.153:0/1968105433 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f41b8102620 0x7f41b819ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:38.791 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:38.790+0000 7f41bdc00700 1 -- 172.21.15.153:0/1968105433 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f41b8102620 msgr2=0x7f41b819ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:38.791 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:38.790+0000 7f41bdc00700 1 --2- 172.21.15.153:0/1968105433 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f41b8102620 0x7f41b819ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:38.791 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:38.790+0000 7f41bdc00700 1 -- 172.21.15.153:0/1968105433 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41ac0095f0 con 0x7f41b8101d00 2024-03-21T10:28:38.792 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:38.791+0000 7f41bdc00700 1 --2- 172.21.15.153:0/1968105433 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f41b8101d00 0x7f41b819c530 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f41ac006010 tx=0x7f41ac00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:28:38.792 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:38.791+0000 7f41b6000700 1 -- 172.21.15.153:0/1968105433 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f41ac01d070 con 0x7f41b8101d00 2024-03-21T10:28:38.792 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:38.791+0000 7f41b6000700 1 -- 172.21.15.153:0/1968105433 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f41ac004fb0 con 0x7f41b8101d00 2024-03-21T10:28:38.792 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:38.791+0000 7f41c0600700 1 -- 172.21.15.153:0/1968105433 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41b8196830 con 0x7f41b8101d00 2024-03-21T10:28:38.792 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:38.791+0000 7f41b6000700 1 -- 172.21.15.153:0/1968105433 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f41ac022ea0 con 0x7f41b8101d00 2024-03-21T10:28:38.793 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:38.791+0000 7f41c0600700 1 -- 172.21.15.153:0/1968105433 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41b8196d20 con 0x7f41b8101d00 2024-03-21T10:28:38.793 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:38.792+0000 7f41c0600700 1 -- 172.21.15.153:0/1968105433 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41b8108210 con 0x7f41b8101d00 2024-03-21T10:28:38.793 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:38.793+0000 7f41b6000700 1 -- 172.21.15.153:0/1968105433 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f41ac022470 con 0x7f41b8101d00 2024-03-21T10:28:38.794 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:38.794+0000 7f41b6000700 1 --2- 172.21.15.153:0/1968105433 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f41a006e460 0x7f41a0070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:38.794 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:38.794+0000 7f41b6000700 1 -- 172.21.15.153:0/1968105433 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f41ac08fa70 con 0x7f41b8101d00 2024-03-21T10:28:38.794 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:38.794+0000 7f41bd200700 1 --2- 172.21.15.153:0/1968105433 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f41a006e460 0x7f41a0070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:38.795 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:38.795+0000 7f41bd200700 1 --2- 172.21.15.153:0/1968105433 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f41a006e460 0x7f41a0070920 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f41a4005ea0 tx=0x7f41a4005e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:28:38.798 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:38.799+0000 7f41b6000700 1 -- 172.21.15.153:0/1968105433 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f41ac05c810 con 0x7f41b8101d00 2024-03-21T10:28:39.034 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:39.034+0000 7f41c0600700 1 -- 172.21.15.153:0/1968105433 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f41b8197050 con 0x7f41a006e460 2024-03-21T10:28:39.037 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:39.037+0000 7f41b6000700 1 -- 172.21.15.153:0/1968105433 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f41b8197050 con 0x7f41a006e460 2024-03-21T10:28:39.043 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:39.040+0000 7f41c0600700 1 -- 172.21.15.153:0/1968105433 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f41a006e460 msgr2=0x7f41a0070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:39.043 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:39.040+0000 7f41c0600700 1 --2- 172.21.15.153:0/1968105433 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f41a006e460 0x7f41a0070920 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f41a4005ea0 tx=0x7f41a4005e30 comp rx=0 tx=0).stop 2024-03-21T10:28:39.043 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:39.040+0000 7f41c0600700 1 -- 172.21.15.153:0/1968105433 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f41b8101d00 msgr2=0x7f41b819c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:39.043 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:39.040+0000 7f41c0600700 1 --2- 172.21.15.153:0/1968105433 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f41b8101d00 0x7f41b819c530 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f41ac006010 tx=0x7f41ac00be20 comp rx=0 tx=0).stop 2024-03-21T10:28:39.043 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:39.040+0000 7f41c0600700 1 -- 172.21.15.153:0/1968105433 shutdown_connections 2024-03-21T10:28:39.043 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:39.040+0000 7f41c0600700 1 --2- 172.21.15.153:0/1968105433 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f41a006e460 0x7f41a0070920 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:39.043 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:39.040+0000 7f41c0600700 1 --2- 172.21.15.153:0/1968105433 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f41b8101d00 0x7f41b819c530 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:39.043 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:39.040+0000 7f41c0600700 1 --2- 172.21.15.153:0/1968105433 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f41b8102620 0x7f41b819ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:39.043 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:39.040+0000 7f41c0600700 1 -- 172.21.15.153:0/1968105433 >> 172.21.15.153:0/1968105433 conn(0x7f41b80fb570 msgr2=0x7f41b8105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:28:39.043 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:39.041+0000 7f41c0600700 1 -- 172.21.15.153:0/1968105433 shutdown_connections 2024-03-21T10:28:39.044 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:39.041+0000 7f41c0600700 1 -- 172.21.15.153:0/1968105433 wait complete. 2024-03-21T10:28:39.186 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:39.185+0000 7f11fbe00700 1 -- 172.21.15.153:0/1671911844 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f11f41071d0 msgr2=0x7f11f4107650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:39.186 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:39.185+0000 7f11fbe00700 1 --2- 172.21.15.153:0/1671911844 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f11f41071d0 0x7f11f4107650 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f11e0009910 tx=0x7f11e0009c20 comp rx=0 tx=0).stop 2024-03-21T10:28:39.186 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:39.185+0000 7f11fbe00700 1 -- 172.21.15.153:0/1671911844 shutdown_connections 2024-03-21T10:28:39.186 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:39.185+0000 7f11fbe00700 1 --2- 172.21.15.153:0/1671911844 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f11f41071d0 0x7f11f4107650 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:39.186 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:39.185+0000 7f11fbe00700 1 --2- 172.21.15.153:0/1671911844 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f11f410d280 0x7f11f410d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:39.186 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:39.185+0000 7f11fbe00700 1 -- 172.21.15.153:0/1671911844 >> 172.21.15.153:0/1671911844 conn(0x7f11f40769e0 msgr2=0x7f11f4078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:28:39.186 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:39.185+0000 7f11fbe00700 1 -- 172.21.15.153:0/1671911844 shutdown_connections 2024-03-21T10:28:39.187 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:39.186+0000 7f11fbe00700 1 -- 172.21.15.153:0/1671911844 wait complete. 2024-03-21T10:28:39.188 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:39.186+0000 7f11fbe00700 1 Processor -- start 2024-03-21T10:28:39.188 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:39.187+0000 7f11fbe00700 1 -- start start 2024-03-21T10:28:39.188 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:39.187+0000 7f11fbe00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f11f41071d0 0x7f11f419ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:39.188 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:39.187+0000 7f11fbe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f11f410d280 0x7f11f419d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:39.188 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:39.187+0000 7f11fbe00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11f419d7f0 con 0x7f11f410d280 2024-03-21T10:28:39.188 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:39.187+0000 7f11fbe00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11f419d960 con 0x7f11f41071d0 2024-03-21T10:28:39.188 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:39.187+0000 7f11f8a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f11f410d280 0x7f11f419d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:39.188 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:39.187+0000 7f11f8a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f11f410d280 0x7f11f419d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:60112/0 (socket says 172.21.15.153:60112) 2024-03-21T10:28:39.188 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:39.187+0000 7f11f8a00700 1 -- 172.21.15.153:0/2792794256 learned_addr learned my addr 172.21.15.153:0/2792794256 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:28:39.189 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:39.188+0000 7f11f9400700 1 --2- 172.21.15.153:0/2792794256 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f11f41071d0 0x7f11f419ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:39.189 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:39.188+0000 7f11f8a00700 1 -- 172.21.15.153:0/2792794256 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f11f41071d0 msgr2=0x7f11f419ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:39.189 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:39.188+0000 7f11f8a00700 1 --2- 172.21.15.153:0/2792794256 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f11f41071d0 0x7f11f419ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:39.189 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:39.188+0000 7f11f8a00700 1 -- 172.21.15.153:0/2792794256 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11e00095f0 con 0x7f11f410d280 2024-03-21T10:28:39.189 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:39.188+0000 7f11f9400700 1 --2- 172.21.15.153:0/2792794256 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f11f41071d0 0x7f11f419ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:28:39.189 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:39.188+0000 7f11f8a00700 1 --2- 172.21.15.153:0/2792794256 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f11f410d280 0x7f11f419d1f0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f11e0006010 tx=0x7f11e000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:28:39.190 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:39.188+0000 7f11f2000700 1 -- 172.21.15.153:0/2792794256 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f11e001d070 con 0x7f11f410d280 2024-03-21T10:28:39.190 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:39.188+0000 7f11fbe00700 1 -- 172.21.15.153:0/2792794256 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11f41a06a0 con 0x7f11f410d280 2024-03-21T10:28:39.190 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:39.188+0000 7f11f2000700 1 -- 172.21.15.153:0/2792794256 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f11e0004fb0 con 0x7f11f410d280 2024-03-21T10:28:39.190 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:39.188+0000 7f11f2000700 1 -- 172.21.15.153:0/2792794256 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f11e000f690 con 0x7f11f410d280 2024-03-21T10:28:39.190 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:39.189+0000 7f11fbe00700 1 -- 172.21.15.153:0/2792794256 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11f41a0c10 con 0x7f11f410d280 2024-03-21T10:28:39.190 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:39.190+0000 7f11fbe00700 1 -- 172.21.15.153:0/2792794256 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11f410f6c0 con 0x7f11f410d280 2024-03-21T10:28:39.190 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:39.191+0000 7f11f2000700 1 -- 172.21.15.153:0/2792794256 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f11e00228a0 con 0x7f11f410d280 2024-03-21T10:28:39.192 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:39.191+0000 7f11f2000700 1 --2- 172.21.15.153:0/2792794256 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f11dc06e460 0x7f11dc070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:39.192 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:39.192+0000 7f11f2000700 1 -- 172.21.15.153:0/2792794256 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f11e008fe10 con 0x7f11f410d280 2024-03-21T10:28:39.192 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:39.192+0000 7f11f9400700 1 --2- 172.21.15.153:0/2792794256 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f11dc06e460 0x7f11dc070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:39.192 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:39.192+0000 7f11f9400700 1 --2- 172.21.15.153:0/2792794256 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f11dc06e460 0x7f11dc070920 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f11e8005fd0 tx=0x7f11e8005e20 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:28:39.196 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:39.196+0000 7f11f2000700 1 -- 172.21.15.153:0/2792794256 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f11e0058e30 con 0x7f11f410d280 2024-03-21T10:28:39.305 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:28:39 smithi153 ceph-mon[113733]: pgmap v503: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:28:39.419 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:39.419+0000 7f11fbe00700 1 -- 172.21.15.153:0/2792794256 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f11f402cd60 con 0x7f11dc06e460 2024-03-21T10:28:39.431 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:39.431+0000 7f11f2000700 1 -- 172.21.15.153:0/2792794256 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f11f402cd60 con 0x7f11dc06e460 2024-03-21T10:28:39.432 INFO:teuthology.orchestra.run.smithi153.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-21T10:28:39.432 INFO:teuthology.orchestra.run.smithi153.stdout:alertmanager.smithi153 smithi153 *:9093,9094 running (17m) 8m ago 31m 14.7M - 0.25.0 c8568f914cd2 9ed3f2200b5f 2024-03-21T10:28:39.432 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi153 smithi153 running (31m) 8m ago 31m 9424k - 18.2.1 5be31c24972a 5b61994396a6 2024-03-21T10:28:39.432 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi174 smithi174 running (29m) 8m ago 29m 9244k - 18.2.1 5be31c24972a 4f7a5fc3ac28 2024-03-21T10:28:39.432 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi153 smithi153 running (13m) 8m ago 31m 6656k - 19.0.0-2231-g2edfee68 237e43c7a65f 32c21104d5b0 2024-03-21T10:28:39.432 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi174 smithi174 running (13m) 8m ago 29m 6660k - 19.0.0-2231-g2edfee68 237e43c7a65f a04e48967ab6 2024-03-21T10:28:39.432 INFO:teuthology.orchestra.run.smithi153.stdout:grafana.smithi153 smithi153 *:3000 running (16m) 8m ago 30m 73.6M - 9.4.12 2bacad6d85d8 ee24026ab786 2024-03-21T10:28:39.432 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.iuyluj smithi153 running (9m) 8m ago 23m 10.9M - 19.0.0-2231-g2edfee68 237e43c7a65f 87aff8a5de70 2024-03-21T10:28:39.432 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.zipckr smithi153 running (9m) 8m ago 24m 11.8M - 19.0.0-2231-g2edfee68 237e43c7a65f ccc262bd5dd3 2024-03-21T10:28:39.432 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.nhozag smithi174 running (23m) 8m ago 23m 12.0M - 18.2.1 5be31c24972a 2fee4000308f 2024-03-21T10:28:39.432 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.yyimst smithi174 running (8m) 8m ago 24m 10.7M - 19.0.0-2231-g2edfee68 237e43c7a65f fc1962f9351e 2024-03-21T10:28:39.433 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi153.tluxni smithi153 *:8443,9283,8765 running (20m) 8m ago 33m 509M - 19.0.0-2231-g2edfee68 237e43c7a65f 7ffee0e55382 2024-03-21T10:28:39.433 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi174.qpcwat smithi174 *:8443,9283,8765 running (19m) 8m ago 29m 452M - 19.0.0-2231-g2edfee68 237e43c7a65f 748968f216ab 2024-03-21T10:28:39.433 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi153 smithi153 running (15m) 8m ago 33m 55.0M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f dd8038303dc3 2024-03-21T10:28:39.433 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi174 smithi174 running (14m) 8m ago 29m 44.2M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f 29d78a788f2c 2024-03-21T10:28:39.433 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi153 smithi153 *:9100 running (18m) 8m ago 31m 7889k - 1.5.0 0da6a335fe13 bb835edc0e0b 2024-03-21T10:28:39.433 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi174 smithi174 *:9100 running (18m) 8m ago 29m 8102k - 1.5.0 0da6a335fe13 7c4ecad790fa 2024-03-21T10:28:39.433 INFO:teuthology.orchestra.run.smithi153.stdout:osd.0 smithi153 running (13m) 8m ago 28m 116M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 507900fd4791 2024-03-21T10:28:39.433 INFO:teuthology.orchestra.run.smithi153.stdout:osd.1 smithi153 running (12m) 8m ago 28m 89.2M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 0d646452e9c1 2024-03-21T10:28:39.433 INFO:teuthology.orchestra.run.smithi153.stdout:osd.2 smithi153 running (11m) 8m ago 27m 87.4M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 1326f2afdfcd 2024-03-21T10:28:39.433 INFO:teuthology.orchestra.run.smithi153.stdout:osd.3 smithi174 running (11m) 8m ago 26m 83.6M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 70aafac71ad5 2024-03-21T10:28:39.433 INFO:teuthology.orchestra.run.smithi153.stdout:osd.4 smithi174 running (10m) 8m ago 26m 92.2M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 0d746ce6e79d 2024-03-21T10:28:39.433 INFO:teuthology.orchestra.run.smithi153.stdout:osd.5 smithi174 running (9m) 8m ago 25m 86.2M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 53e5f35b8cac 2024-03-21T10:28:39.433 INFO:teuthology.orchestra.run.smithi153.stdout:prometheus.smithi153 smithi153 *:9095 running (17m) 8m ago 30m 73.5M - 2.43.0 a07b618ecd1d 7d95561e50c2 2024-03-21T10:28:39.434 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:39.434+0000 7f11fbe00700 1 -- 172.21.15.153:0/2792794256 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f11dc06e460 msgr2=0x7f11dc070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:39.434 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:39.434+0000 7f11fbe00700 1 --2- 172.21.15.153:0/2792794256 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f11dc06e460 0x7f11dc070920 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f11e8005fd0 tx=0x7f11e8005e20 comp rx=0 tx=0).stop 2024-03-21T10:28:39.434 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:39.434+0000 7f11fbe00700 1 -- 172.21.15.153:0/2792794256 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f11f410d280 msgr2=0x7f11f419d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:39.434 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:39.434+0000 7f11fbe00700 1 --2- 172.21.15.153:0/2792794256 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f11f410d280 0x7f11f419d1f0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f11e0006010 tx=0x7f11e000be20 comp rx=0 tx=0).stop 2024-03-21T10:28:39.435 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:39.434+0000 7f11fbe00700 1 -- 172.21.15.153:0/2792794256 shutdown_connections 2024-03-21T10:28:39.435 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:39.434+0000 7f11fbe00700 1 --2- 172.21.15.153:0/2792794256 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f11f41071d0 0x7f11f419ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:39.435 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:39.434+0000 7f11fbe00700 1 --2- 172.21.15.153:0/2792794256 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f11dc06e460 0x7f11dc070920 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:39.435 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:39.434+0000 7f11fbe00700 1 --2- 172.21.15.153:0/2792794256 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f11f410d280 0x7f11f419d1f0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:39.435 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:39.434+0000 7f11fbe00700 1 -- 172.21.15.153:0/2792794256 >> 172.21.15.153:0/2792794256 conn(0x7f11f40769e0 msgr2=0x7f11f4078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:28:39.435 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:39.434+0000 7f11fbe00700 1 -- 172.21.15.153:0/2792794256 shutdown_connections 2024-03-21T10:28:39.435 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:39.434+0000 7f11fbe00700 1 -- 172.21.15.153:0/2792794256 wait complete. 2024-03-21T10:28:39.450 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:28:39 smithi174 ceph-mon[88004]: pgmap v503: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:28:39.586 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:39.585+0000 7f1652600700 1 -- 172.21.15.153:0/3108842124 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f164c10d280 msgr2=0x7f164c10d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:39.586 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:39.585+0000 7f1652600700 1 --2- 172.21.15.153:0/3108842124 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f164c10d280 0x7f164c10d660 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f163c009910 tx=0x7f163c009c20 comp rx=0 tx=0).stop 2024-03-21T10:28:39.586 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:39.585+0000 7f1652600700 1 -- 172.21.15.153:0/3108842124 shutdown_connections 2024-03-21T10:28:39.586 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:39.585+0000 7f1652600700 1 --2- 172.21.15.153:0/3108842124 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f164c1071d0 0x7f164c107650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:39.586 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:39.585+0000 7f1652600700 1 --2- 172.21.15.153:0/3108842124 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f164c10d280 0x7f164c10d660 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:39.586 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:39.585+0000 7f1652600700 1 -- 172.21.15.153:0/3108842124 >> 172.21.15.153:0/3108842124 conn(0x7f164c076990 msgr2=0x7f164c078db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:28:39.586 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:39.585+0000 7f1652600700 1 -- 172.21.15.153:0/3108842124 shutdown_connections 2024-03-21T10:28:39.586 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:39.585+0000 7f1652600700 1 -- 172.21.15.153:0/3108842124 wait complete. 2024-03-21T10:28:39.588 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:39.587+0000 7f1652600700 1 Processor -- start 2024-03-21T10:28:39.588 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:39.587+0000 7f1652600700 1 -- start start 2024-03-21T10:28:39.588 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:39.587+0000 7f1652600700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f164c1071d0 0x7f164c19ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:39.588 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:39.587+0000 7f1652600700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f164c10d280 0x7f164c19d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:39.588 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:39.587+0000 7f1652600700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f164c19d890 con 0x7f164c10d280 2024-03-21T10:28:39.588 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:39.587+0000 7f1652600700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f164c1a05c0 con 0x7f164c1071d0 2024-03-21T10:28:39.588 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:39.587+0000 7f164aa00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f164c10d280 0x7f164c19d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:39.588 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:39.587+0000 7f164aa00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f164c10d280 0x7f164c19d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:60126/0 (socket says 172.21.15.153:60126) 2024-03-21T10:28:39.588 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:39.587+0000 7f164aa00700 1 -- 172.21.15.153:0/482925073 learned_addr learned my addr 172.21.15.153:0/482925073 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:28:39.588 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:39.588+0000 7f164b400700 1 --2- 172.21.15.153:0/482925073 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f164c1071d0 0x7f164c19ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:39.588 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:39.588+0000 7f164aa00700 1 -- 172.21.15.153:0/482925073 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f164c1071d0 msgr2=0x7f164c19ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:39.588 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:39.588+0000 7f164aa00700 1 --2- 172.21.15.153:0/482925073 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f164c1071d0 0x7f164c19ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:39.589 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:39.588+0000 7f164aa00700 1 -- 172.21.15.153:0/482925073 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f163c0095f0 con 0x7f164c10d280 2024-03-21T10:28:39.589 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:39.588+0000 7f164aa00700 1 --2- 172.21.15.153:0/482925073 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f164c10d280 0x7f164c19d200 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f163800c8a0 tx=0x7f163800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:28:39.589 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:39.588+0000 7f1643e00700 1 -- 172.21.15.153:0/482925073 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f163800cea0 con 0x7f164c10d280 2024-03-21T10:28:39.589 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:39.588+0000 7f1643e00700 1 -- 172.21.15.153:0/482925073 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1638004830 con 0x7f164c10d280 2024-03-21T10:28:39.590 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:39.588+0000 7f1652600700 1 -- 172.21.15.153:0/482925073 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f164c1a08a0 con 0x7f164c10d280 2024-03-21T10:28:39.590 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:39.588+0000 7f1643e00700 1 -- 172.21.15.153:0/482925073 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f16380055b0 con 0x7f164c10d280 2024-03-21T10:28:39.590 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:39.589+0000 7f1652600700 1 -- 172.21.15.153:0/482925073 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f164c1a0e70 con 0x7f164c10d280 2024-03-21T10:28:39.590 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:39.590+0000 7f1652600700 1 -- 172.21.15.153:0/482925073 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f164c04ea90 con 0x7f164c10d280 2024-03-21T10:28:39.590 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:39.591+0000 7f1643e00700 1 -- 172.21.15.153:0/482925073 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f16380049a0 con 0x7f164c10d280 2024-03-21T10:28:39.592 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:39.592+0000 7f1643e00700 1 --2- 172.21.15.153:0/482925073 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f163006e460 0x7f1630070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:39.592 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:39.592+0000 7f1643e00700 1 -- 172.21.15.153:0/482925073 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f163808e270 con 0x7f164c10d280 2024-03-21T10:28:39.592 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:39.592+0000 7f164b400700 1 --2- 172.21.15.153:0/482925073 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f163006e460 0x7f1630070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:39.593 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:39.593+0000 7f164b400700 1 --2- 172.21.15.153:0/482925073 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f163006e460 0x7f1630070920 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f163c006010 tx=0x7f163c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:28:39.597 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:39.597+0000 7f1643e00700 1 -- 172.21.15.153:0/482925073 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f1638057290 con 0x7f164c10d280 2024-03-21T10:28:39.894 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:39.894+0000 7f1652600700 1 -- 172.21.15.153:0/482925073 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f164c1a0a30 con 0x7f164c10d280 2024-03-21T10:28:39.895 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:39.895+0000 7f1643e00700 1 -- 172.21.15.153:0/482925073 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+805 (secure 0 0 0) 0x7f164c1a0a30 con 0x7f164c10d280 2024-03-21T10:28:39.896 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:28:39.896 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": { 2024-03-21T10:28:39.896 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:28:39.896 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:28:39.896 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": { 2024-03-21T10:28:39.896 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:28:39.896 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:28:39.897 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": { 2024-03-21T10:28:39.897 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 6 2024-03-21T10:28:39.897 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:28:39.897 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": { 2024-03-21T10:28:39.897 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-03-21T10:28:39.897 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 3 2024-03-21T10:28:39.897 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:28:39.897 INFO:teuthology.orchestra.run.smithi153.stdout: "overall": { 2024-03-21T10:28:39.897 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-03-21T10:28:39.897 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 13 2024-03-21T10:28:39.897 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-03-21T10:28:39.897 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:28:39.899 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:39.898+0000 7f1652600700 1 -- 172.21.15.153:0/482925073 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f163006e460 msgr2=0x7f1630070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:39.899 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:39.898+0000 7f1652600700 1 --2- 172.21.15.153:0/482925073 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f163006e460 0x7f1630070920 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f163c006010 tx=0x7f163c01a040 comp rx=0 tx=0).stop 2024-03-21T10:28:39.899 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:39.898+0000 7f1652600700 1 -- 172.21.15.153:0/482925073 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f164c10d280 msgr2=0x7f164c19d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:39.899 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:39.898+0000 7f1652600700 1 --2- 172.21.15.153:0/482925073 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f164c10d280 0x7f164c19d200 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f163800c8a0 tx=0x7f163800cc60 comp rx=0 tx=0).stop 2024-03-21T10:28:39.899 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:39.898+0000 7f1652600700 1 -- 172.21.15.153:0/482925073 shutdown_connections 2024-03-21T10:28:39.899 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:39.898+0000 7f1652600700 1 --2- 172.21.15.153:0/482925073 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f164c1071d0 0x7f164c19ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:39.899 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:39.898+0000 7f1652600700 1 --2- 172.21.15.153:0/482925073 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f163006e460 0x7f1630070920 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:39.899 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:39.898+0000 7f1652600700 1 --2- 172.21.15.153:0/482925073 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f164c10d280 0x7f164c19d200 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:39.899 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:39.899+0000 7f1652600700 1 -- 172.21.15.153:0/482925073 >> 172.21.15.153:0/482925073 conn(0x7f164c076990 msgr2=0x7f164c078310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:28:39.900 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:39.899+0000 7f1652600700 1 -- 172.21.15.153:0/482925073 shutdown_connections 2024-03-21T10:28:39.900 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:39.899+0000 7f1652600700 1 -- 172.21.15.153:0/482925073 wait complete. 2024-03-21T10:28:40.048 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.048+0000 7fceac200700 1 -- 172.21.15.153:0/2819615549 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcea41071d0 msgr2=0x7fcea4107650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:40.048 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.048+0000 7fceac200700 1 --2- 172.21.15.153:0/2819615549 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcea41071d0 0x7fcea4107650 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fce90009910 tx=0x7fce90009c20 comp rx=0 tx=0).stop 2024-03-21T10:28:40.048 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.048+0000 7fceac200700 1 -- 172.21.15.153:0/2819615549 shutdown_connections 2024-03-21T10:28:40.048 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.048+0000 7fceac200700 1 --2- 172.21.15.153:0/2819615549 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcea41071d0 0x7fcea4107650 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:40.049 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.048+0000 7fceac200700 1 --2- 172.21.15.153:0/2819615549 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcea410d280 0x7fcea410d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:40.049 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.048+0000 7fceac200700 1 -- 172.21.15.153:0/2819615549 >> 172.21.15.153:0/2819615549 conn(0x7fcea40769e0 msgr2=0x7fcea4078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:28:40.049 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.048+0000 7fceac200700 1 -- 172.21.15.153:0/2819615549 shutdown_connections 2024-03-21T10:28:40.049 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.048+0000 7fceac200700 1 -- 172.21.15.153:0/2819615549 wait complete. 2024-03-21T10:28:40.050 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.049+0000 7fceac200700 1 Processor -- start 2024-03-21T10:28:40.050 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.049+0000 7fceac200700 1 -- start start 2024-03-21T10:28:40.050 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.049+0000 7fceac200700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcea41071d0 0x7fcea419ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:40.050 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.049+0000 7fceac200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcea410d280 0x7fcea419d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:40.050 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.049+0000 7fceac200700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcea419d880 con 0x7fcea410d280 2024-03-21T10:28:40.050 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.049+0000 7fceac200700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcea41a05b0 con 0x7fcea41071d0 2024-03-21T10:28:40.050 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.050+0000 7fcea8e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcea410d280 0x7fcea419d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:40.050 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.050+0000 7fcea8e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcea410d280 0x7fcea419d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:60148/0 (socket says 172.21.15.153:60148) 2024-03-21T10:28:40.050 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.050+0000 7fcea8e00700 1 -- 172.21.15.153:0/217871949 learned_addr learned my addr 172.21.15.153:0/217871949 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:28:40.051 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.050+0000 7fcea9800700 1 --2- 172.21.15.153:0/217871949 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcea41071d0 0x7fcea419ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:40.052 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.050+0000 7fcea8e00700 1 -- 172.21.15.153:0/217871949 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcea41071d0 msgr2=0x7fcea419ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:40.052 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.050+0000 7fcea8e00700 1 --2- 172.21.15.153:0/217871949 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcea41071d0 0x7fcea419ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:40.052 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.050+0000 7fcea8e00700 1 -- 172.21.15.153:0/217871949 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce900095f0 con 0x7fcea410d280 2024-03-21T10:28:40.052 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.051+0000 7fcea8e00700 1 --2- 172.21.15.153:0/217871949 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcea410d280 0x7fcea419d1f0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fce90006010 tx=0x7fce9000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:28:40.052 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.051+0000 7fcea2000700 1 -- 172.21.15.153:0/217871949 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fce9001d070 con 0x7fcea410d280 2024-03-21T10:28:40.052 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.051+0000 7fcea2000700 1 -- 172.21.15.153:0/217871949 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fce90004fb0 con 0x7fcea410d280 2024-03-21T10:28:40.052 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.051+0000 7fceac200700 1 -- 172.21.15.153:0/217871949 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcea41a0830 con 0x7fcea410d280 2024-03-21T10:28:40.052 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.051+0000 7fcea2000700 1 -- 172.21.15.153:0/217871949 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fce9000f690 con 0x7fcea410d280 2024-03-21T10:28:40.052 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.051+0000 7fceac200700 1 -- 172.21.15.153:0/217871949 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcea41a0da0 con 0x7fcea410d280 2024-03-21T10:28:40.053 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.052+0000 7fceac200700 1 -- 172.21.15.153:0/217871949 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcea410f6c0 con 0x7fcea410d280 2024-03-21T10:28:40.053 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.053+0000 7fcea2000700 1 -- 172.21.15.153:0/217871949 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fce900228a0 con 0x7fcea410d280 2024-03-21T10:28:40.054 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.054+0000 7fcea2000700 1 --2- 172.21.15.153:0/217871949 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fce8c06e460 0x7fce8c070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:40.054 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.054+0000 7fcea2000700 1 -- 172.21.15.153:0/217871949 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fce9008fd30 con 0x7fcea410d280 2024-03-21T10:28:40.054 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.054+0000 7fcea9800700 1 --2- 172.21.15.153:0/217871949 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fce8c06e460 0x7fce8c070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:40.055 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.055+0000 7fcea9800700 1 --2- 172.21.15.153:0/217871949 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fce8c06e460 0x7fce8c070920 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fce98005fd0 tx=0x7fce9800a560 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:28:40.059 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.059+0000 7fcea2000700 1 -- 172.21.15.153:0/217871949 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fce90058dd0 con 0x7fcea410d280 2024-03-21T10:28:40.322 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.321+0000 7fceac200700 1 -- 172.21.15.153:0/217871949 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fcea404ea90 con 0x7fcea410d280 2024-03-21T10:28:40.322 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.322+0000 7fcea2000700 1 -- 172.21.15.153:0/217871949 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1876 (secure 0 0 0) 0x7fce90027070 con 0x7fcea410d280 2024-03-21T10:28:40.323 INFO:teuthology.orchestra.run.smithi153.stdout:e30 2024-03-21T10:28:40.323 INFO:teuthology.orchestra.run.smithi153.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-21T10:28:40.323 INFO:teuthology.orchestra.run.smithi153.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:28:40.323 INFO:teuthology.orchestra.run.smithi153.stdout:legacy client fscid: 1 2024-03-21T10:28:40.324 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:28:40.324 INFO:teuthology.orchestra.run.smithi153.stdout:Filesystem 'cephfs' (1) 2024-03-21T10:28:40.324 INFO:teuthology.orchestra.run.smithi153.stdout:fs_name cephfs 2024-03-21T10:28:40.324 INFO:teuthology.orchestra.run.smithi153.stdout:epoch 23 2024-03-21T10:28:40.324 INFO:teuthology.orchestra.run.smithi153.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-21T10:28:40.324 INFO:teuthology.orchestra.run.smithi153.stdout:created 2024-03-21T10:04:27.086576+0000 2024-03-21T10:28:40.324 INFO:teuthology.orchestra.run.smithi153.stdout:modified 2024-03-21T10:19:05.022092+0000 2024-03-21T10:28:40.324 INFO:teuthology.orchestra.run.smithi153.stdout:tableserver 0 2024-03-21T10:28:40.324 INFO:teuthology.orchestra.run.smithi153.stdout:root 0 2024-03-21T10:28:40.324 INFO:teuthology.orchestra.run.smithi153.stdout:session_timeout 60 2024-03-21T10:28:40.324 INFO:teuthology.orchestra.run.smithi153.stdout:session_autoclose 300 2024-03-21T10:28:40.324 INFO:teuthology.orchestra.run.smithi153.stdout:max_file_size 1099511627776 2024-03-21T10:28:40.324 INFO:teuthology.orchestra.run.smithi153.stdout:max_xattr_size 65536 2024-03-21T10:28:40.324 INFO:teuthology.orchestra.run.smithi153.stdout:required_client_features {} 2024-03-21T10:28:40.324 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure 0 2024-03-21T10:28:40.325 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure_osd_epoch 94 2024-03-21T10:28:40.325 INFO:teuthology.orchestra.run.smithi153.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:28:40.325 INFO:teuthology.orchestra.run.smithi153.stdout:max_mds 2 2024-03-21T10:28:40.325 INFO:teuthology.orchestra.run.smithi153.stdout:in 0,1 2024-03-21T10:28:40.325 INFO:teuthology.orchestra.run.smithi153.stdout:up {} 2024-03-21T10:28:40.325 INFO:teuthology.orchestra.run.smithi153.stdout:failed 0,1 2024-03-21T10:28:40.325 INFO:teuthology.orchestra.run.smithi153.stdout:damaged 2024-03-21T10:28:40.325 INFO:teuthology.orchestra.run.smithi153.stdout:stopped 2024-03-21T10:28:40.325 INFO:teuthology.orchestra.run.smithi153.stdout:data_pools [3] 2024-03-21T10:28:40.325 INFO:teuthology.orchestra.run.smithi153.stdout:metadata_pool 2 2024-03-21T10:28:40.325 INFO:teuthology.orchestra.run.smithi153.stdout:inline_data enabled 2024-03-21T10:28:40.325 INFO:teuthology.orchestra.run.smithi153.stdout:balancer 2024-03-21T10:28:40.325 INFO:teuthology.orchestra.run.smithi153.stdout:bal_rank_mask -1 2024-03-21T10:28:40.325 INFO:teuthology.orchestra.run.smithi153.stdout:standby_count_wanted 1 2024-03-21T10:28:40.325 INFO:teuthology.orchestra.run.smithi153.stdout:qdb_cluster leader: 0 members: 2024-03-21T10:28:40.326 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:28:40.326 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:28:40.326 INFO:teuthology.orchestra.run.smithi153.stdout:Standby daemons: 2024-03-21T10:28:40.326 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:28:40.326 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.zipckr{-1:34442} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6826/1103846295,v1:172.21.15.153:6827/1103846295] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:28:40.326 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.iuyluj{-1:34474} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6828/1738630832,v1:172.21.15.153:6829/1738630832] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:28:40.326 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.nhozag{-1:44269} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.174:6826/3967223227,v1:172.21.15.174:6827/3967223227] compat {c=[1],r=[1],i=[7ff]}] 2024-03-21T10:28:40.326 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.yyimst{-1:44313} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.174:6824/4127786797,v1:172.21.15.174:6825/4127786797] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:28:40.327 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.325+0000 7fceac200700 1 -- 172.21.15.153:0/217871949 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fce8c06e460 msgr2=0x7fce8c070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:40.327 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.325+0000 7fceac200700 1 --2- 172.21.15.153:0/217871949 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fce8c06e460 0x7fce8c070920 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fce98005fd0 tx=0x7fce9800a560 comp rx=0 tx=0).stop 2024-03-21T10:28:40.327 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.325+0000 7fceac200700 1 -- 172.21.15.153:0/217871949 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcea410d280 msgr2=0x7fcea419d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:40.327 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.325+0000 7fceac200700 1 --2- 172.21.15.153:0/217871949 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcea410d280 0x7fcea419d1f0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fce90006010 tx=0x7fce9000be20 comp rx=0 tx=0).stop 2024-03-21T10:28:40.327 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.325+0000 7fceac200700 1 -- 172.21.15.153:0/217871949 shutdown_connections 2024-03-21T10:28:40.327 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.325+0000 7fceac200700 1 --2- 172.21.15.153:0/217871949 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcea41071d0 0x7fcea419ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:40.327 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.325+0000 7fceac200700 1 --2- 172.21.15.153:0/217871949 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fce8c06e460 0x7fce8c070920 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:40.327 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.325+0000 7fceac200700 1 --2- 172.21.15.153:0/217871949 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcea410d280 0x7fcea419d1f0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:40.327 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.325+0000 7fceac200700 1 -- 172.21.15.153:0/217871949 >> 172.21.15.153:0/217871949 conn(0x7fcea40769e0 msgr2=0x7fcea4078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:28:40.328 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.325+0000 7fceac200700 1 -- 172.21.15.153:0/217871949 shutdown_connections 2024-03-21T10:28:40.328 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.325+0000 7fceac200700 1 -- 172.21.15.153:0/217871949 wait complete. 2024-03-21T10:28:40.328 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 30 2024-03-21T10:28:40.450 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:28:40 smithi174 ceph-mon[88004]: from='client.34926 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:28:40.450 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:28:40 smithi174 ceph-mon[88004]: from='client.34930 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:28:40.450 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:28:40 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/482925073' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:28:40.479 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.478+0000 7fb592200700 1 -- 172.21.15.153:0/1124499229 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb58c1037a0 msgr2=0x7fb58c1077f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:40.479 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.478+0000 7fb592200700 1 --2- 172.21.15.153:0/1124499229 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb58c1037a0 0x7fb58c1077f0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fb57c009910 tx=0x7fb57c009c20 comp rx=0 tx=0).stop 2024-03-21T10:28:40.479 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.478+0000 7fb592200700 1 -- 172.21.15.153:0/1124499229 shutdown_connections 2024-03-21T10:28:40.479 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.478+0000 7fb592200700 1 --2- 172.21.15.153:0/1124499229 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb58c1037a0 0x7fb58c1077f0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:40.479 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.478+0000 7fb592200700 1 --2- 172.21.15.153:0/1124499229 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb58c102df0 0x7fb58c1031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:40.479 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.478+0000 7fb592200700 1 -- 172.21.15.153:0/1124499229 >> 172.21.15.153:0/1124499229 conn(0x7fb58c0fe6a0 msgr2=0x7fb58c100ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:28:40.479 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.478+0000 7fb592200700 1 -- 172.21.15.153:0/1124499229 shutdown_connections 2024-03-21T10:28:40.480 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:28:40 smithi153 ceph-mon[113733]: from='client.34926 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:28:40.480 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:28:40 smithi153 ceph-mon[113733]: from='client.34930 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:28:40.480 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:28:40 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/482925073' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:28:40.481 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.478+0000 7fb592200700 1 -- 172.21.15.153:0/1124499229 wait complete. 2024-03-21T10:28:40.481 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.479+0000 7fb592200700 1 Processor -- start 2024-03-21T10:28:40.481 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.480+0000 7fb592200700 1 -- start start 2024-03-21T10:28:40.481 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.480+0000 7fb592200700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb58c102df0 0x7fb58c19e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:40.481 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.480+0000 7fb592200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb58c1037a0 0x7fb58c19ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:40.481 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.480+0000 7fb592200700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb58c19f0e0 con 0x7fb58c1037a0 2024-03-21T10:28:40.481 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.480+0000 7fb592200700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb58c198590 con 0x7fb58c102df0 2024-03-21T10:28:40.481 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.480+0000 7fb58be00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb58c1037a0 0x7fb58c19ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:40.481 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.480+0000 7fb58be00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb58c1037a0 0x7fb58c19ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:60158/0 (socket says 172.21.15.153:60158) 2024-03-21T10:28:40.482 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.480+0000 7fb58be00700 1 -- 172.21.15.153:0/813406410 learned_addr learned my addr 172.21.15.153:0/813406410 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:28:40.482 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.481+0000 7fb590e00700 1 --2- 172.21.15.153:0/813406410 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb58c102df0 0x7fb58c19e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:40.482 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.481+0000 7fb58be00700 1 -- 172.21.15.153:0/813406410 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb58c102df0 msgr2=0x7fb58c19e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:40.482 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.481+0000 7fb58be00700 1 --2- 172.21.15.153:0/813406410 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb58c102df0 0x7fb58c19e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:40.482 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.481+0000 7fb58be00700 1 -- 172.21.15.153:0/813406410 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb57c0095f0 con 0x7fb58c1037a0 2024-03-21T10:28:40.483 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.481+0000 7fb58be00700 1 --2- 172.21.15.153:0/813406410 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb58c1037a0 0x7fb58c19ea50 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fb57c00bda0 tx=0x7fb57c00be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:28:40.483 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.482+0000 7fb589600700 1 -- 172.21.15.153:0/813406410 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb57c01d070 con 0x7fb58c1037a0 2024-03-21T10:28:40.483 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.482+0000 7fb589600700 1 -- 172.21.15.153:0/813406410 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb57c004fb0 con 0x7fb58c1037a0 2024-03-21T10:28:40.483 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.482+0000 7fb592200700 1 -- 172.21.15.153:0/813406410 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb58c198810 con 0x7fb58c1037a0 2024-03-21T10:28:40.483 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.482+0000 7fb589600700 1 -- 172.21.15.153:0/813406410 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb57c00f460 con 0x7fb58c1037a0 2024-03-21T10:28:40.483 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.482+0000 7fb592200700 1 -- 172.21.15.153:0/813406410 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb58c198d00 con 0x7fb58c1037a0 2024-03-21T10:28:40.483 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.483+0000 7fb592200700 1 -- 172.21.15.153:0/813406410 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb58c10b1f0 con 0x7fb58c1037a0 2024-03-21T10:28:40.484 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.484+0000 7fb589600700 1 -- 172.21.15.153:0/813406410 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fb57c022800 con 0x7fb58c1037a0 2024-03-21T10:28:40.485 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.485+0000 7fb589600700 1 --2- 172.21.15.153:0/813406410 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb57406e460 0x7fb574070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:40.485 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.485+0000 7fb589600700 1 -- 172.21.15.153:0/813406410 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fb57c08fa50 con 0x7fb58c1037a0 2024-03-21T10:28:40.485 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.485+0000 7fb590e00700 1 --2- 172.21.15.153:0/813406410 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb57406e460 0x7fb574070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:40.486 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.486+0000 7fb590e00700 1 --2- 172.21.15.153:0/813406410 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb57406e460 0x7fb574070920 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fb580005fd0 tx=0x7fb58000a560 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:28:40.489 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.490+0000 7fb589600700 1 -- 172.21.15.153:0/813406410 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb57c05c810 con 0x7fb58c1037a0 2024-03-21T10:28:40.716 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.715+0000 7fb592200700 1 -- 172.21.15.153:0/813406410 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb58c02cd10 con 0x7fb57406e460 2024-03-21T10:28:40.718 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.718+0000 7fb589600700 1 -- 172.21.15.153:0/813406410 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fb58c02cd10 con 0x7fb57406e460 2024-03-21T10:28:40.718 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:28:40.718 INFO:teuthology.orchestra.run.smithi153.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232", 2024-03-21T10:28:40.718 INFO:teuthology.orchestra.run.smithi153.stdout: "in_progress": true, 2024-03-21T10:28:40.718 INFO:teuthology.orchestra.run.smithi153.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-21T10:28:40.719 INFO:teuthology.orchestra.run.smithi153.stdout: "services_complete": [ 2024-03-21T10:28:40.719 INFO:teuthology.orchestra.run.smithi153.stdout: "crash", 2024-03-21T10:28:40.719 INFO:teuthology.orchestra.run.smithi153.stdout: "mon", 2024-03-21T10:28:40.719 INFO:teuthology.orchestra.run.smithi153.stdout: "osd", 2024-03-21T10:28:40.719 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr" 2024-03-21T10:28:40.719 INFO:teuthology.orchestra.run.smithi153.stdout: ], 2024-03-21T10:28:40.719 INFO:teuthology.orchestra.run.smithi153.stdout: "progress": "15/23 daemons upgraded", 2024-03-21T10:28:40.719 INFO:teuthology.orchestra.run.smithi153.stdout: "message": "Currently upgrading mds daemons", 2024-03-21T10:28:40.719 INFO:teuthology.orchestra.run.smithi153.stdout: "is_paused": false 2024-03-21T10:28:40.719 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:28:40.721 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.721+0000 7fb592200700 1 -- 172.21.15.153:0/813406410 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb57406e460 msgr2=0x7fb574070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:40.721 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.721+0000 7fb592200700 1 --2- 172.21.15.153:0/813406410 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb57406e460 0x7fb574070920 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fb580005fd0 tx=0x7fb58000a560 comp rx=0 tx=0).stop 2024-03-21T10:28:40.721 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.721+0000 7fb592200700 1 -- 172.21.15.153:0/813406410 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb58c1037a0 msgr2=0x7fb58c19ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:40.721 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.721+0000 7fb592200700 1 --2- 172.21.15.153:0/813406410 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb58c1037a0 0x7fb58c19ea50 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fb57c00bda0 tx=0x7fb57c00be80 comp rx=0 tx=0).stop 2024-03-21T10:28:40.722 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.721+0000 7fb592200700 1 -- 172.21.15.153:0/813406410 shutdown_connections 2024-03-21T10:28:40.722 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.721+0000 7fb592200700 1 --2- 172.21.15.153:0/813406410 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb58c102df0 0x7fb58c19e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:40.722 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.721+0000 7fb592200700 1 --2- 172.21.15.153:0/813406410 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb57406e460 0x7fb574070920 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:40.722 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.721+0000 7fb592200700 1 --2- 172.21.15.153:0/813406410 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb58c1037a0 0x7fb58c19ea50 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:40.722 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.721+0000 7fb592200700 1 -- 172.21.15.153:0/813406410 >> 172.21.15.153:0/813406410 conn(0x7fb58c0fe6a0 msgr2=0x7fb58c100a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:28:40.722 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.721+0000 7fb592200700 1 -- 172.21.15.153:0/813406410 shutdown_connections 2024-03-21T10:28:40.722 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.721+0000 7fb592200700 1 -- 172.21.15.153:0/813406410 wait complete. 2024-03-21T10:28:40.873 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.873+0000 7faa6f200700 1 -- 172.21.15.153:0/1367993529 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faa681009c0 msgr2=0x7faa68100e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:40.873 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.873+0000 7faa6f200700 1 --2- 172.21.15.153:0/1367993529 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faa681009c0 0x7faa68100e40 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7faa58009910 tx=0x7faa58009c20 comp rx=0 tx=0).stop 2024-03-21T10:28:40.873 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.873+0000 7faa6f200700 1 -- 172.21.15.153:0/1367993529 shutdown_connections 2024-03-21T10:28:40.873 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.873+0000 7faa6f200700 1 --2- 172.21.15.153:0/1367993529 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faa681009c0 0x7faa68100e40 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:40.874 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.873+0000 7faa6f200700 1 --2- 172.21.15.153:0/1367993529 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faa681069c0 0x7faa68106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:40.874 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.873+0000 7faa6f200700 1 -- 172.21.15.153:0/1367993529 >> 172.21.15.153:0/1367993529 conn(0x7faa680fc4f0 msgr2=0x7faa680fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:28:40.874 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.873+0000 7faa6f200700 1 -- 172.21.15.153:0/1367993529 shutdown_connections 2024-03-21T10:28:40.874 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.873+0000 7faa6f200700 1 -- 172.21.15.153:0/1367993529 wait complete. 2024-03-21T10:28:40.874 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.874+0000 7faa6f200700 1 Processor -- start 2024-03-21T10:28:40.874 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.874+0000 7faa6f200700 1 -- start start 2024-03-21T10:28:40.875 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.875+0000 7faa6f200700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faa681009c0 0x7faa68073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:40.875 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.875+0000 7faa6f200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faa681069c0 0x7faa680735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:40.875 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.875+0000 7faa6f200700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa68073b10 con 0x7faa681069c0 2024-03-21T10:28:40.875 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.875+0000 7faa6f200700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa68073c50 con 0x7faa681009c0 2024-03-21T10:28:40.875 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.875+0000 7faa6d400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faa681069c0 0x7faa680735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:40.876 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.875+0000 7faa6d400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faa681069c0 0x7faa680735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:60172/0 (socket says 172.21.15.153:60172) 2024-03-21T10:28:40.876 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.875+0000 7faa6d400700 1 -- 172.21.15.153:0/1947092719 learned_addr learned my addr 172.21.15.153:0/1947092719 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:28:40.876 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.876+0000 7faa6de00700 1 --2- 172.21.15.153:0/1947092719 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faa681009c0 0x7faa68073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:40.876 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.876+0000 7faa6d400700 1 -- 172.21.15.153:0/1947092719 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faa681009c0 msgr2=0x7faa68073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:40.876 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.876+0000 7faa6d400700 1 --2- 172.21.15.153:0/1947092719 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faa681009c0 0x7faa68073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:40.876 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.876+0000 7faa6d400700 1 -- 172.21.15.153:0/1947092719 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa580095f0 con 0x7faa681069c0 2024-03-21T10:28:40.876 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.876+0000 7faa6d400700 1 --2- 172.21.15.153:0/1947092719 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faa681069c0 0x7faa680735d0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7faa58006010 tx=0x7faa5800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:28:40.877 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.876+0000 7faa62a00700 1 -- 172.21.15.153:0/1947092719 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faa5801d070 con 0x7faa681069c0 2024-03-21T10:28:40.877 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.876+0000 7faa62a00700 1 -- 172.21.15.153:0/1947092719 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faa58004fb0 con 0x7faa681069c0 2024-03-21T10:28:40.877 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.876+0000 7faa62a00700 1 -- 172.21.15.153:0/1947092719 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faa5800f690 con 0x7faa681069c0 2024-03-21T10:28:40.877 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.876+0000 7faa6f200700 1 -- 172.21.15.153:0/1947092719 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa68073ed0 con 0x7faa681069c0 2024-03-21T10:28:40.877 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.877+0000 7faa6f200700 1 -- 172.21.15.153:0/1947092719 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa681a2aa0 con 0x7faa681069c0 2024-03-21T10:28:40.878 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.878+0000 7faa6f200700 1 -- 172.21.15.153:0/1947092719 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa68108f50 con 0x7faa681069c0 2024-03-21T10:28:40.879 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.879+0000 7faa62a00700 1 -- 172.21.15.153:0/1947092719 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7faa580228a0 con 0x7faa681069c0 2024-03-21T10:28:40.880 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.880+0000 7faa62a00700 1 --2- 172.21.15.153:0/1947092719 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7faa5006e460 0x7faa50070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:28:40.880 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.880+0000 7faa62a00700 1 -- 172.21.15.153:0/1947092719 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7faa5808fdd0 con 0x7faa681069c0 2024-03-21T10:28:40.880 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.880+0000 7faa6de00700 1 --2- 172.21.15.153:0/1947092719 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7faa5006e460 0x7faa50070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:28:40.880 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.881+0000 7faa6de00700 1 --2- 172.21.15.153:0/1947092719 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7faa5006e460 0x7faa50070920 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7faa5c005fd0 tx=0x7faa5c00a560 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:28:40.884 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:40.885+0000 7faa62a00700 1 -- 172.21.15.153:0/1947092719 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7faa58058df0 con 0x7faa681069c0 2024-03-21T10:28:41.194 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:41.193+0000 7faa6f200700 1 -- 172.21.15.153:0/1947092719 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7faa6804f2e0 con 0x7faa681069c0 2024-03-21T10:28:41.194 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:41.194+0000 7faa62a00700 1 -- 172.21.15.153:0/1947092719 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7faa580271c0 con 0x7faa681069c0 2024-03-21T10:28:41.195 INFO:teuthology.orchestra.run.smithi153.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-03-21T10:28:41.195 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-21T10:28:41.195 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is degraded 2024-03-21T10:28:41.195 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-21T10:28:41.195 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-21T10:28:41.195 INFO:teuthology.orchestra.run.smithi153.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-21T10:28:41.195 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-21T10:28:41.197 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:41.197+0000 7faa6f200700 1 -- 172.21.15.153:0/1947092719 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7faa5006e460 msgr2=0x7faa50070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:41.197 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:41.197+0000 7faa6f200700 1 --2- 172.21.15.153:0/1947092719 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7faa5006e460 0x7faa50070920 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7faa5c005fd0 tx=0x7faa5c00a560 comp rx=0 tx=0).stop 2024-03-21T10:28:41.197 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:41.197+0000 7faa6f200700 1 -- 172.21.15.153:0/1947092719 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faa681069c0 msgr2=0x7faa680735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:28:41.197 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:41.197+0000 7faa6f200700 1 --2- 172.21.15.153:0/1947092719 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faa681069c0 0x7faa680735d0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7faa58006010 tx=0x7faa5800be20 comp rx=0 tx=0).stop 2024-03-21T10:28:41.198 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:41.197+0000 7faa6f200700 1 -- 172.21.15.153:0/1947092719 shutdown_connections 2024-03-21T10:28:41.198 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:41.197+0000 7faa6f200700 1 --2- 172.21.15.153:0/1947092719 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faa681009c0 0x7faa68073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:41.198 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:41.197+0000 7faa6f200700 1 --2- 172.21.15.153:0/1947092719 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7faa5006e460 0x7faa50070920 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:41.198 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:41.197+0000 7faa6f200700 1 --2- 172.21.15.153:0/1947092719 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faa681069c0 0x7faa680735d0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:28:41.198 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:41.197+0000 7faa6f200700 1 -- 172.21.15.153:0/1947092719 >> 172.21.15.153:0/1947092719 conn(0x7faa680fc4f0 msgr2=0x7faa680fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:28:41.198 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:41.197+0000 7faa6f200700 1 -- 172.21.15.153:0/1947092719 shutdown_connections 2024-03-21T10:28:41.198 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:28:41.197+0000 7faa6f200700 1 -- 172.21.15.153:0/1947092719 wait complete. 2024-03-21T10:28:41.450 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:28:41 smithi174 ceph-mon[88004]: from='client.34934 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:28:41.450 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:28:41 smithi174 ceph-mon[88004]: pgmap v504: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:28:41.450 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:28:41 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/217871949' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:28:41.555 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:28:41 smithi153 ceph-mon[113733]: from='client.34934 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:28:41.555 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:28:41 smithi153 ceph-mon[113733]: pgmap v504: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:28:41.555 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:28:41 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/217871949' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:28:42.450 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:28:42 smithi174 ceph-mon[88004]: from='client.34946 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:28:42.450 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:28:42 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/1947092719' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:28:42.555 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:28:42 smithi153 ceph-mon[113733]: from='client.34946 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:28:42.555 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:28:42 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/1947092719' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:28:43.450 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:28:43 smithi174 ceph-mon[88004]: pgmap v505: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:28:43.555 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:28:43 smithi153 ceph-mon[113733]: pgmap v505: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:28:45.450 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:28:45 smithi174 ceph-mon[88004]: pgmap v506: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:28:45.555 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:28:45 smithi153 ceph-mon[113733]: pgmap v506: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:28:47.450 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:28:47 smithi174 ceph-mon[88004]: pgmap v507: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:28:47.450 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:28:47 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:28:47.555 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:28:47 smithi153 ceph-mon[113733]: pgmap v507: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:28:47.555 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:28:47 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:28:49.555 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:28:49 smithi153 ceph-mon[113733]: pgmap v508: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:28:49.702 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:28:49 smithi174 ceph-mon[88004]: pgmap v508: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:28:51.555 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:28:51 smithi153 ceph-mon[113733]: pgmap v509: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:28:51.700 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:28:51 smithi174 ceph-mon[88004]: pgmap v509: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:28:53.555 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:28:53 smithi153 ceph-mon[113733]: pgmap v510: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:28:53.700 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:28:53 smithi174 ceph-mon[88004]: pgmap v510: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:28:55.512 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:28:55 smithi174 ceph-mon[88004]: pgmap v511: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:28:55.555 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:28:55 smithi153 ceph-mon[113733]: pgmap v511: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:28:57.555 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:28:57 smithi153 ceph-mon[113733]: pgmap v512: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:28:57.700 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:28:57 smithi174 ceph-mon[88004]: pgmap v512: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:28:59.555 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:28:59 smithi153 ceph-mon[113733]: pgmap v513: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:28:59.700 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:28:59 smithi174 ceph-mon[88004]: pgmap v513: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:29:01.450 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:29:01 smithi174 ceph-mon[88004]: pgmap v514: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:29:01.450 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:29:01 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:29:01.555 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:29:01 smithi153 ceph-mon[113733]: pgmap v514: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:29:01.555 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:29:01 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:29:03.555 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:29:03 smithi153 ceph-mon[113733]: pgmap v515: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:29:03.700 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:29:03 smithi174 ceph-mon[88004]: pgmap v515: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:29:05.502 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:29:05 smithi174 ceph-mon[88004]: pgmap v516: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:29:05.555 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:29:05 smithi153 ceph-mon[113733]: pgmap v516: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:29:07.555 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:29:07 smithi153 ceph-mon[113733]: pgmap v517: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:29:07.700 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:29:07 smithi174 ceph-mon[88004]: pgmap v517: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:29:09.555 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:29:09 smithi153 ceph-mon[113733]: pgmap v518: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:29:09.700 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:29:09 smithi174 ceph-mon[88004]: pgmap v518: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:29:11.373 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:11.368+0000 7f16c2600700 1 -- 172.21.15.153:0/868052308 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f16bc101770 msgr2=0x7f16bc1057c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:11.374 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:11.368+0000 7f16c2600700 1 --2- 172.21.15.153:0/868052308 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f16bc101770 0x7f16bc1057c0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f16a8009910 tx=0x7f16a8009c20 comp rx=0 tx=0).stop 2024-03-21T10:29:11.374 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:11.368+0000 7f16c2600700 1 -- 172.21.15.153:0/868052308 shutdown_connections 2024-03-21T10:29:11.374 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:11.368+0000 7f16c2600700 1 --2- 172.21.15.153:0/868052308 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f16bc101770 0x7f16bc1057c0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:11.374 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:11.368+0000 7f16c2600700 1 --2- 172.21.15.153:0/868052308 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f16bc100dc0 0x7f16bc1011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:11.374 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:11.368+0000 7f16c2600700 1 -- 172.21.15.153:0/868052308 >> 172.21.15.153:0/868052308 conn(0x7f16bc0fc670 msgr2=0x7f16bc0fea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:11.374 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:11.368+0000 7f16c2600700 1 -- 172.21.15.153:0/868052308 shutdown_connections 2024-03-21T10:29:11.374 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:11.369+0000 7f16c2600700 1 -- 172.21.15.153:0/868052308 wait complete. 2024-03-21T10:29:11.374 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:11.369+0000 7f16c2600700 1 Processor -- start 2024-03-21T10:29:11.375 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:11.370+0000 7f16c2600700 1 -- start start 2024-03-21T10:29:11.375 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:11.370+0000 7f16c2600700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f16bc100dc0 0x7f16bc19e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:11.375 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:11.370+0000 7f16c2600700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f16bc101770 0x7f16bc19ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:11.375 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:11.370+0000 7f16c2600700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16bc19f3e0 con 0x7f16bc101770 2024-03-21T10:29:11.375 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:11.370+0000 7f16c2600700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16bc198890 con 0x7f16bc100dc0 2024-03-21T10:29:11.375 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:11.370+0000 7f16baa00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f16bc101770 0x7f16bc19ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:11.375 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:11.370+0000 7f16baa00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f16bc101770 0x7f16bc19ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42680/0 (socket says 172.21.15.153:42680) 2024-03-21T10:29:11.375 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:11.370+0000 7f16baa00700 1 -- 172.21.15.153:0/194251953 learned_addr learned my addr 172.21.15.153:0/194251953 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:29:11.375 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:11.371+0000 7f16bb400700 1 --2- 172.21.15.153:0/194251953 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f16bc100dc0 0x7f16bc19e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:11.378 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:11.371+0000 7f16baa00700 1 -- 172.21.15.153:0/194251953 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f16bc100dc0 msgr2=0x7f16bc19e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:11.378 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:11.371+0000 7f16baa00700 1 --2- 172.21.15.153:0/194251953 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f16bc100dc0 0x7f16bc19e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:11.378 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:11.371+0000 7f16baa00700 1 -- 172.21.15.153:0/194251953 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16a80095f0 con 0x7f16bc101770 2024-03-21T10:29:11.378 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:11.371+0000 7f16baa00700 1 --2- 172.21.15.153:0/194251953 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f16bc101770 0x7f16bc19ed50 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f16a8005f50 tx=0x7f16a800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:11.379 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:11.371+0000 7f16b3e00700 1 -- 172.21.15.153:0/194251953 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f16a801d070 con 0x7f16bc101770 2024-03-21T10:29:11.379 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:11.372+0000 7f16b3e00700 1 -- 172.21.15.153:0/194251953 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f16a800f460 con 0x7f16bc101770 2024-03-21T10:29:11.379 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:11.372+0000 7f16c2600700 1 -- 172.21.15.153:0/194251953 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16bc198b10 con 0x7f16bc101770 2024-03-21T10:29:11.379 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:11.372+0000 7f16b3e00700 1 -- 172.21.15.153:0/194251953 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f16a8017630 con 0x7f16bc101770 2024-03-21T10:29:11.379 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:11.372+0000 7f16c2600700 1 -- 172.21.15.153:0/194251953 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16bc199000 con 0x7f16bc101770 2024-03-21T10:29:11.379 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:11.373+0000 7f16c2600700 1 -- 172.21.15.153:0/194251953 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16bc104f40 con 0x7f16bc101770 2024-03-21T10:29:11.379 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:11.374+0000 7f16b3e00700 1 -- 172.21.15.153:0/194251953 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f16a800f5d0 con 0x7f16bc101770 2024-03-21T10:29:11.379 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:11.375+0000 7f16b3e00700 1 --2- 172.21.15.153:0/194251953 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f16a006e460 0x7f16a0070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:11.379 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:11.375+0000 7f16b3e00700 1 -- 172.21.15.153:0/194251953 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f16a808faa0 con 0x7f16bc101770 2024-03-21T10:29:11.380 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:11.375+0000 7f16bb400700 1 --2- 172.21.15.153:0/194251953 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f16a006e460 0x7f16a0070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:11.380 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:11.376+0000 7f16bb400700 1 --2- 172.21.15.153:0/194251953 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f16a006e460 0x7f16a0070920 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f16ac006fd0 tx=0x7f16ac008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:11.380 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:11.380+0000 7f16b3e00700 1 -- 172.21.15.153:0/194251953 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f16a805c670 con 0x7f16bc101770 2024-03-21T10:29:11.554 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:29:11 smithi153 ceph-mon[113733]: pgmap v519: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:29:11.654 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:11.654+0000 7f16c2600700 1 -- 172.21.15.153:0/194251953 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f16bc02cd10 con 0x7f16a006e460 2024-03-21T10:29:11.656 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:11.656+0000 7f16b3e00700 1 -- 172.21.15.153:0/194251953 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f16bc02cd10 con 0x7f16a006e460 2024-03-21T10:29:11.659 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:11.659+0000 7f16c2600700 1 -- 172.21.15.153:0/194251953 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f16a006e460 msgr2=0x7f16a0070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:11.660 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:11.659+0000 7f16c2600700 1 --2- 172.21.15.153:0/194251953 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f16a006e460 0x7f16a0070920 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f16ac006fd0 tx=0x7f16ac008040 comp rx=0 tx=0).stop 2024-03-21T10:29:11.660 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:11.659+0000 7f16c2600700 1 -- 172.21.15.153:0/194251953 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f16bc101770 msgr2=0x7f16bc19ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:11.660 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:11.659+0000 7f16c2600700 1 --2- 172.21.15.153:0/194251953 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f16bc101770 0x7f16bc19ed50 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f16a8005f50 tx=0x7f16a800be20 comp rx=0 tx=0).stop 2024-03-21T10:29:11.660 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:11.659+0000 7f16c2600700 1 -- 172.21.15.153:0/194251953 shutdown_connections 2024-03-21T10:29:11.660 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:11.659+0000 7f16c2600700 1 --2- 172.21.15.153:0/194251953 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f16bc100dc0 0x7f16bc19e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:11.660 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:11.659+0000 7f16c2600700 1 --2- 172.21.15.153:0/194251953 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f16a006e460 0x7f16a0070920 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:11.660 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:11.659+0000 7f16c2600700 1 --2- 172.21.15.153:0/194251953 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f16bc101770 0x7f16bc19ed50 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:11.660 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:11.659+0000 7f16c2600700 1 -- 172.21.15.153:0/194251953 >> 172.21.15.153:0/194251953 conn(0x7f16bc0fc670 msgr2=0x7f16bc0fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:11.660 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:11.660+0000 7f16c2600700 1 -- 172.21.15.153:0/194251953 shutdown_connections 2024-03-21T10:29:11.660 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:11.660+0000 7f16c2600700 1 -- 172.21.15.153:0/194251953 wait complete. 2024-03-21T10:29:11.678 INFO:teuthology.orchestra.run.smithi153.stdout:true 2024-03-21T10:29:11.700 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:29:11 smithi174 ceph-mon[88004]: pgmap v519: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:29:11.819 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:11.819+0000 7f9f8f800700 1 -- 172.21.15.153:0/1347678328 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9f88102fd0 msgr2=0x7f9f881033b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:11.820 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:11.819+0000 7f9f8f800700 1 --2- 172.21.15.153:0/1347678328 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9f88102fd0 0x7f9f881033b0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c009910 tx=0x7f9f7c009c20 comp rx=0 tx=0).stop 2024-03-21T10:29:11.820 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:11.819+0000 7f9f8f800700 1 -- 172.21.15.153:0/1347678328 shutdown_connections 2024-03-21T10:29:11.820 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:11.819+0000 7f9f8f800700 1 --2- 172.21.15.153:0/1347678328 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f88103980 0x7f9f881079d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:11.820 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:11.819+0000 7f9f8f800700 1 --2- 172.21.15.153:0/1347678328 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9f88102fd0 0x7f9f881033b0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:11.820 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:11.819+0000 7f9f8f800700 1 -- 172.21.15.153:0/1347678328 >> 172.21.15.153:0/1347678328 conn(0x7f9f880fe840 msgr2=0x7f9f88100c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:11.820 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:11.819+0000 7f9f8f800700 1 -- 172.21.15.153:0/1347678328 shutdown_connections 2024-03-21T10:29:11.820 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:11.819+0000 7f9f8f800700 1 -- 172.21.15.153:0/1347678328 wait complete. 2024-03-21T10:29:11.821 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:11.820+0000 7f9f8f800700 1 Processor -- start 2024-03-21T10:29:11.821 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:11.820+0000 7f9f8f800700 1 -- start start 2024-03-21T10:29:11.821 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:11.821+0000 7f9f8f800700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f88102fd0 0x7f9f8819e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:11.822 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:11.821+0000 7f9f8f800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9f88103980 0x7f9f8819ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:11.822 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:11.821+0000 7f9f8f800700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f8819f3b0 con 0x7f9f88103980 2024-03-21T10:29:11.822 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:11.821+0000 7f9f8f800700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f88198860 con 0x7f9f88102fd0 2024-03-21T10:29:11.822 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:11.821+0000 7f9f87e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9f88103980 0x7f9f8819ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:11.822 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:11.821+0000 7f9f87e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9f88103980 0x7f9f8819ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42688/0 (socket says 172.21.15.153:42688) 2024-03-21T10:29:11.822 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:11.821+0000 7f9f87e00700 1 -- 172.21.15.153:0/1524867445 learned_addr learned my addr 172.21.15.153:0/1524867445 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:29:11.822 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:11.821+0000 7f9f8ce00700 1 --2- 172.21.15.153:0/1524867445 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f88102fd0 0x7f9f8819e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:11.822 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:11.822+0000 7f9f87e00700 1 -- 172.21.15.153:0/1524867445 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f88102fd0 msgr2=0x7f9f8819e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:11.822 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:11.822+0000 7f9f87e00700 1 --2- 172.21.15.153:0/1524867445 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f88102fd0 0x7f9f8819e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:11.822 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:11.822+0000 7f9f87e00700 1 -- 172.21.15.153:0/1524867445 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f7c0095f0 con 0x7f9f88103980 2024-03-21T10:29:11.823 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:11.822+0000 7f9f87e00700 1 --2- 172.21.15.153:0/1524867445 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9f88103980 0x7f9f8819ed20 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f9f7800c8f0 tx=0x7f9f7800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:11.823 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:11.822+0000 7f9f85600700 1 -- 172.21.15.153:0/1524867445 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f78007a50 con 0x7f9f88103980 2024-03-21T10:29:11.823 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:11.822+0000 7f9f85600700 1 -- 172.21.15.153:0/1524867445 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9f78007bb0 con 0x7f9f88103980 2024-03-21T10:29:11.823 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:11.822+0000 7f9f8f800700 1 -- 172.21.15.153:0/1524867445 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f88198b40 con 0x7f9f88103980 2024-03-21T10:29:11.823 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:11.822+0000 7f9f85600700 1 -- 172.21.15.153:0/1524867445 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f78018630 con 0x7f9f88103980 2024-03-21T10:29:11.823 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:11.822+0000 7f9f8f800700 1 -- 172.21.15.153:0/1524867445 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f88199090 con 0x7f9f88103980 2024-03-21T10:29:11.824 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:11.823+0000 7f9f8f800700 1 -- 172.21.15.153:0/1524867445 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f8810b370 con 0x7f9f88103980 2024-03-21T10:29:11.824 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:11.825+0000 7f9f85600700 1 -- 172.21.15.153:0/1524867445 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f9f78007d20 con 0x7f9f88103980 2024-03-21T10:29:11.825 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:11.825+0000 7f9f85600700 1 --2- 172.21.15.153:0/1524867445 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f9f7006e460 0x7f9f70070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:11.825 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:11.826+0000 7f9f85600700 1 -- 172.21.15.153:0/1524867445 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f9f7808e450 con 0x7f9f88103980 2024-03-21T10:29:11.825 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:11.826+0000 7f9f8ce00700 1 --2- 172.21.15.153:0/1524867445 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f9f7006e460 0x7f9f70070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:11.826 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:11.827+0000 7f9f8ce00700 1 --2- 172.21.15.153:0/1524867445 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f9f7006e460 0x7f9f70070920 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c006010 tx=0x7f9f7c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:11.830 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:11.831+0000 7f9f85600700 1 -- 172.21.15.153:0/1524867445 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f9f7805b020 con 0x7f9f88103980 2024-03-21T10:29:12.075 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:12.074+0000 7f9f8f800700 1 -- 172.21.15.153:0/1524867445 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9f881993d0 con 0x7f9f7006e460 2024-03-21T10:29:12.077 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:12.077+0000 7f9f85600700 1 -- 172.21.15.153:0/1524867445 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f9f881993d0 con 0x7f9f7006e460 2024-03-21T10:29:12.080 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:12.080+0000 7f9f8f800700 1 -- 172.21.15.153:0/1524867445 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f9f7006e460 msgr2=0x7f9f70070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:12.081 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:12.080+0000 7f9f8f800700 1 --2- 172.21.15.153:0/1524867445 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f9f7006e460 0x7f9f70070920 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c006010 tx=0x7f9f7c01a040 comp rx=0 tx=0).stop 2024-03-21T10:29:12.081 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:12.080+0000 7f9f8f800700 1 -- 172.21.15.153:0/1524867445 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9f88103980 msgr2=0x7f9f8819ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:12.081 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:12.080+0000 7f9f8f800700 1 --2- 172.21.15.153:0/1524867445 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9f88103980 0x7f9f8819ed20 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f9f7800c8f0 tx=0x7f9f7800ccb0 comp rx=0 tx=0).stop 2024-03-21T10:29:12.081 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:12.080+0000 7f9f8f800700 1 -- 172.21.15.153:0/1524867445 shutdown_connections 2024-03-21T10:29:12.081 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:12.080+0000 7f9f8f800700 1 --2- 172.21.15.153:0/1524867445 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f88102fd0 0x7f9f8819e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:12.081 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:12.080+0000 7f9f8f800700 1 --2- 172.21.15.153:0/1524867445 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f9f7006e460 0x7f9f70070920 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:12.081 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:12.080+0000 7f9f8f800700 1 --2- 172.21.15.153:0/1524867445 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9f88103980 0x7f9f8819ed20 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:12.081 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:12.080+0000 7f9f8f800700 1 -- 172.21.15.153:0/1524867445 >> 172.21.15.153:0/1524867445 conn(0x7f9f880fe840 msgr2=0x7f9f880ffe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:12.081 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:12.081+0000 7f9f8f800700 1 -- 172.21.15.153:0/1524867445 shutdown_connections 2024-03-21T10:29:12.081 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:12.081+0000 7f9f8f800700 1 -- 172.21.15.153:0/1524867445 wait complete. 2024-03-21T10:29:12.234 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:12.233+0000 7f60c4200700 1 -- 172.21.15.153:0/3492717675 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f60bc102ff0 msgr2=0x7f60bc1033d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:12.234 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:12.233+0000 7f60c4200700 1 --2- 172.21.15.153:0/3492717675 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f60bc102ff0 0x7f60bc1033d0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f60b0009910 tx=0x7f60b0009c20 comp rx=0 tx=0).stop 2024-03-21T10:29:12.234 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:12.234+0000 7f60c4200700 1 -- 172.21.15.153:0/3492717675 shutdown_connections 2024-03-21T10:29:12.234 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:12.234+0000 7f60c4200700 1 --2- 172.21.15.153:0/3492717675 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f60bc1039a0 0x7f60bc1079f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:12.234 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:12.234+0000 7f60c4200700 1 --2- 172.21.15.153:0/3492717675 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f60bc102ff0 0x7f60bc1033d0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:12.234 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:12.234+0000 7f60c4200700 1 -- 172.21.15.153:0/3492717675 >> 172.21.15.153:0/3492717675 conn(0x7f60bc0fe8a0 msgr2=0x7f60bc100cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:12.234 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:12.234+0000 7f60c4200700 1 -- 172.21.15.153:0/3492717675 shutdown_connections 2024-03-21T10:29:12.234 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:12.234+0000 7f60c4200700 1 -- 172.21.15.153:0/3492717675 wait complete. 2024-03-21T10:29:12.235 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:12.235+0000 7f60c4200700 1 Processor -- start 2024-03-21T10:29:12.235 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:12.235+0000 7f60c4200700 1 -- start start 2024-03-21T10:29:12.236 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:12.236+0000 7f60c4200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f60bc102ff0 0x7f60bc19e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:12.236 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:12.236+0000 7f60c4200700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f60bc1039a0 0x7f60bc19ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:12.236 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:12.236+0000 7f60c4200700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60bc19f3b0 con 0x7f60bc102ff0 2024-03-21T10:29:12.236 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:12.236+0000 7f60c4200700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60bc198860 con 0x7f60bc1039a0 2024-03-21T10:29:12.236 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:12.236+0000 7f60c1800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f60bc102ff0 0x7f60bc19e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:12.236 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:12.236+0000 7f60c1800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f60bc102ff0 0x7f60bc19e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42714/0 (socket says 172.21.15.153:42714) 2024-03-21T10:29:12.237 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:12.236+0000 7f60c1800700 1 -- 172.21.15.153:0/3098244082 learned_addr learned my addr 172.21.15.153:0/3098244082 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:29:12.237 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:12.237+0000 7f60c0e00700 1 --2- 172.21.15.153:0/3098244082 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f60bc1039a0 0x7f60bc19ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:12.237 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:12.237+0000 7f60c1800700 1 -- 172.21.15.153:0/3098244082 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f60bc1039a0 msgr2=0x7f60bc19ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:12.237 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:12.237+0000 7f60c1800700 1 --2- 172.21.15.153:0/3098244082 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f60bc1039a0 0x7f60bc19ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:12.238 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:12.237+0000 7f60c1800700 1 -- 172.21.15.153:0/3098244082 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60b00095f0 con 0x7f60bc102ff0 2024-03-21T10:29:12.238 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:12.237+0000 7f60c1800700 1 --2- 172.21.15.153:0/3098244082 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f60bc102ff0 0x7f60bc19e7e0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f60b000b5c0 tx=0x7f60b000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:12.238 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:12.237+0000 7f60ba000700 1 -- 172.21.15.153:0/3098244082 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f60b001d070 con 0x7f60bc102ff0 2024-03-21T10:29:12.238 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:12.238+0000 7f60ba000700 1 -- 172.21.15.153:0/3098244082 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f60b0005e00 con 0x7f60bc102ff0 2024-03-21T10:29:12.238 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:12.238+0000 7f60c4200700 1 -- 172.21.15.153:0/3098244082 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60bc198ae0 con 0x7f60bc102ff0 2024-03-21T10:29:12.238 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:12.238+0000 7f60ba000700 1 -- 172.21.15.153:0/3098244082 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f60b0017400 con 0x7f60bc102ff0 2024-03-21T10:29:12.238 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:12.238+0000 7f60c4200700 1 -- 172.21.15.153:0/3098244082 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60bc198fd0 con 0x7f60bc102ff0 2024-03-21T10:29:12.238 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:12.239+0000 7f60c4200700 1 -- 172.21.15.153:0/3098244082 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60bc10b390 con 0x7f60bc102ff0 2024-03-21T10:29:12.240 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:12.240+0000 7f60ba000700 1 -- 172.21.15.153:0/3098244082 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f60b000f460 con 0x7f60bc102ff0 2024-03-21T10:29:12.241 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:12.241+0000 7f60ba000700 1 --2- 172.21.15.153:0/3098244082 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f60a406e460 0x7f60a4070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:12.241 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:12.241+0000 7f60ba000700 1 -- 172.21.15.153:0/3098244082 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f60b008fa70 con 0x7f60bc102ff0 2024-03-21T10:29:12.241 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:12.241+0000 7f60c0e00700 1 --2- 172.21.15.153:0/3098244082 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f60a406e460 0x7f60a4070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:12.242 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:12.242+0000 7f60c0e00700 1 --2- 172.21.15.153:0/3098244082 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f60a406e460 0x7f60a4070920 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f60a8005950 tx=0x7f60a80058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:12.246 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:12.246+0000 7f60ba000700 1 -- 172.21.15.153:0/3098244082 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f60b005c5c0 con 0x7f60bc102ff0 2024-03-21T10:29:12.479 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:12.479+0000 7f60c4200700 1 -- 172.21.15.153:0/3098244082 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f60bc1992b0 con 0x7f60a406e460 2024-03-21T10:29:12.510 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:12.511+0000 7f60ba000700 1 -- 172.21.15.153:0/3098244082 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f60bc1992b0 con 0x7f60a406e460 2024-03-21T10:29:12.511 INFO:teuthology.orchestra.run.smithi153.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-21T10:29:12.511 INFO:teuthology.orchestra.run.smithi153.stdout:alertmanager.smithi153 smithi153 *:9093,9094 running (18m) 9m ago 31m 14.7M - 0.25.0 c8568f914cd2 9ed3f2200b5f 2024-03-21T10:29:12.511 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi153 smithi153 running (32m) 9m ago 32m 9424k - 18.2.1 5be31c24972a 5b61994396a6 2024-03-21T10:29:12.511 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi174 smithi174 running (30m) 9m ago 30m 9244k - 18.2.1 5be31c24972a 4f7a5fc3ac28 2024-03-21T10:29:12.511 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi153 smithi153 running (14m) 9m ago 32m 6656k - 19.0.0-2231-g2edfee68 237e43c7a65f 32c21104d5b0 2024-03-21T10:29:12.511 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi174 smithi174 running (14m) 9m ago 30m 6660k - 19.0.0-2231-g2edfee68 237e43c7a65f a04e48967ab6 2024-03-21T10:29:12.511 INFO:teuthology.orchestra.run.smithi153.stdout:grafana.smithi153 smithi153 *:3000 running (17m) 9m ago 31m 73.6M - 9.4.12 2bacad6d85d8 ee24026ab786 2024-03-21T10:29:12.512 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.iuyluj smithi153 running (9m) 9m ago 24m 10.9M - 19.0.0-2231-g2edfee68 237e43c7a65f 87aff8a5de70 2024-03-21T10:29:12.512 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.zipckr smithi153 running (9m) 9m ago 24m 11.8M - 19.0.0-2231-g2edfee68 237e43c7a65f ccc262bd5dd3 2024-03-21T10:29:12.512 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.nhozag smithi174 running (24m) 9m ago 24m 12.0M - 18.2.1 5be31c24972a 2fee4000308f 2024-03-21T10:29:12.512 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.yyimst smithi174 running (9m) 9m ago 24m 10.7M - 19.0.0-2231-g2edfee68 237e43c7a65f fc1962f9351e 2024-03-21T10:29:12.512 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi153.tluxni smithi153 *:8443,9283,8765 running (21m) 9m ago 33m 509M - 19.0.0-2231-g2edfee68 237e43c7a65f 7ffee0e55382 2024-03-21T10:29:12.512 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi174.qpcwat smithi174 *:8443,9283,8765 running (19m) 9m ago 30m 452M - 19.0.0-2231-g2edfee68 237e43c7a65f 748968f216ab 2024-03-21T10:29:12.512 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi153 smithi153 running (15m) 9m ago 33m 55.0M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f dd8038303dc3 2024-03-21T10:29:12.512 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi174 smithi174 running (15m) 9m ago 30m 44.2M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f 29d78a788f2c 2024-03-21T10:29:12.512 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi153 smithi153 *:9100 running (19m) 9m ago 31m 7889k - 1.5.0 0da6a335fe13 bb835edc0e0b 2024-03-21T10:29:12.512 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi174 smithi174 *:9100 running (19m) 9m ago 30m 8102k - 1.5.0 0da6a335fe13 7c4ecad790fa 2024-03-21T10:29:12.512 INFO:teuthology.orchestra.run.smithi153.stdout:osd.0 smithi153 running (13m) 9m ago 29m 116M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 507900fd4791 2024-03-21T10:29:12.512 INFO:teuthology.orchestra.run.smithi153.stdout:osd.1 smithi153 running (12m) 9m ago 28m 89.2M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 0d646452e9c1 2024-03-21T10:29:12.512 INFO:teuthology.orchestra.run.smithi153.stdout:osd.2 smithi153 running (12m) 9m ago 28m 87.4M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 1326f2afdfcd 2024-03-21T10:29:12.512 INFO:teuthology.orchestra.run.smithi153.stdout:osd.3 smithi174 running (11m) 9m ago 27m 83.6M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 70aafac71ad5 2024-03-21T10:29:12.512 INFO:teuthology.orchestra.run.smithi153.stdout:osd.4 smithi174 running (11m) 9m ago 26m 92.2M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 0d746ce6e79d 2024-03-21T10:29:12.513 INFO:teuthology.orchestra.run.smithi153.stdout:osd.5 smithi174 running (10m) 9m ago 26m 86.2M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 53e5f35b8cac 2024-03-21T10:29:12.513 INFO:teuthology.orchestra.run.smithi153.stdout:prometheus.smithi153 smithi153 *:9095 running (18m) 9m ago 30m 73.5M - 2.43.0 a07b618ecd1d 7d95561e50c2 2024-03-21T10:29:12.513 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:12.513+0000 7f60c4200700 1 -- 172.21.15.153:0/3098244082 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f60a406e460 msgr2=0x7f60a4070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:12.513 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:12.513+0000 7f60c4200700 1 --2- 172.21.15.153:0/3098244082 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f60a406e460 0x7f60a4070920 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f60a8005950 tx=0x7f60a80058e0 comp rx=0 tx=0).stop 2024-03-21T10:29:12.513 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:12.513+0000 7f60c4200700 1 -- 172.21.15.153:0/3098244082 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f60bc102ff0 msgr2=0x7f60bc19e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:12.514 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:12.513+0000 7f60c4200700 1 --2- 172.21.15.153:0/3098244082 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f60bc102ff0 0x7f60bc19e7e0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f60b000b5c0 tx=0x7f60b000be20 comp rx=0 tx=0).stop 2024-03-21T10:29:12.514 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:12.514+0000 7f60c4200700 1 -- 172.21.15.153:0/3098244082 shutdown_connections 2024-03-21T10:29:12.514 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:12.514+0000 7f60c4200700 1 --2- 172.21.15.153:0/3098244082 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f60a406e460 0x7f60a4070920 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:12.514 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:12.514+0000 7f60c4200700 1 --2- 172.21.15.153:0/3098244082 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f60bc102ff0 0x7f60bc19e7e0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:12.514 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:12.514+0000 7f60c4200700 1 --2- 172.21.15.153:0/3098244082 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f60bc1039a0 0x7f60bc19ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:12.514 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:12.514+0000 7f60c4200700 1 -- 172.21.15.153:0/3098244082 >> 172.21.15.153:0/3098244082 conn(0x7f60bc0fe8a0 msgr2=0x7f60bc0ffef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:12.514 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:12.514+0000 7f60c4200700 1 -- 172.21.15.153:0/3098244082 shutdown_connections 2024-03-21T10:29:12.514 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:12.514+0000 7f60c4200700 1 -- 172.21.15.153:0/3098244082 wait complete. 2024-03-21T10:29:12.666 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:12.666+0000 7fa808800700 1 -- 172.21.15.153:0/2940723114 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa800068670 msgr2=0x7fa800068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:12.667 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:12.666+0000 7fa808800700 1 --2- 172.21.15.153:0/2940723114 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa800068670 0x7fa800068a50 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fa7f4009910 tx=0x7fa7f4009c20 comp rx=0 tx=0).stop 2024-03-21T10:29:12.667 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:12.666+0000 7fa808800700 1 -- 172.21.15.153:0/2940723114 shutdown_connections 2024-03-21T10:29:12.667 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:12.666+0000 7fa808800700 1 --2- 172.21.15.153:0/2940723114 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa800069020 0x7fa800105790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:12.667 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:12.666+0000 7fa808800700 1 --2- 172.21.15.153:0/2940723114 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa800068670 0x7fa800068a50 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:12.667 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:12.666+0000 7fa808800700 1 -- 172.21.15.153:0/2940723114 >> 172.21.15.153:0/2940723114 conn(0x7fa8000755a0 msgr2=0x7fa8000759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:12.667 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:12.666+0000 7fa808800700 1 -- 172.21.15.153:0/2940723114 shutdown_connections 2024-03-21T10:29:12.667 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:12.666+0000 7fa808800700 1 -- 172.21.15.153:0/2940723114 wait complete. 2024-03-21T10:29:12.669 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:12.667+0000 7fa808800700 1 Processor -- start 2024-03-21T10:29:12.669 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:12.667+0000 7fa808800700 1 -- start start 2024-03-21T10:29:12.669 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:12.668+0000 7fa808800700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa800068670 0x7fa80019c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:12.669 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:12.668+0000 7fa808800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa800069020 0x7fa80019ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:12.669 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:12.668+0000 7fa808800700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa80019d050 con 0x7fa800069020 2024-03-21T10:29:12.669 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:12.668+0000 7fa808800700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa800196590 con 0x7fa800068670 2024-03-21T10:29:12.669 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:12.668+0000 7fa805400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa800069020 0x7fa80019ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:12.669 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:12.668+0000 7fa805400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa800069020 0x7fa80019ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42734/0 (socket says 172.21.15.153:42734) 2024-03-21T10:29:12.669 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:12.668+0000 7fa805400700 1 -- 172.21.15.153:0/3235655477 learned_addr learned my addr 172.21.15.153:0/3235655477 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:29:12.669 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:12.668+0000 7fa805e00700 1 --2- 172.21.15.153:0/3235655477 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa800068670 0x7fa80019c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:12.670 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:12.669+0000 7fa805400700 1 -- 172.21.15.153:0/3235655477 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa800068670 msgr2=0x7fa80019c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:12.670 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:12.669+0000 7fa805400700 1 --2- 172.21.15.153:0/3235655477 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa800068670 0x7fa80019c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:12.670 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:12.669+0000 7fa805400700 1 -- 172.21.15.153:0/3235655477 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7f40095f0 con 0x7fa800069020 2024-03-21T10:29:12.670 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:12.669+0000 7fa805400700 1 --2- 172.21.15.153:0/3235655477 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa800069020 0x7fa80019ca50 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fa7f000c960 tx=0x7fa7f000cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:12.670 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:12.669+0000 7fa7faa00700 1 -- 172.21.15.153:0/3235655477 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa7f0007a70 con 0x7fa800069020 2024-03-21T10:29:12.670 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:12.669+0000 7fa7faa00700 1 -- 172.21.15.153:0/3235655477 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa7f0007bd0 con 0x7fa800069020 2024-03-21T10:29:12.670 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:12.669+0000 7fa808800700 1 -- 172.21.15.153:0/3235655477 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa800196870 con 0x7fa800069020 2024-03-21T10:29:12.670 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:12.669+0000 7fa7faa00700 1 -- 172.21.15.153:0/3235655477 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa7f00186c0 con 0x7fa800069020 2024-03-21T10:29:12.671 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:12.669+0000 7fa808800700 1 -- 172.21.15.153:0/3235655477 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa800196dc0 con 0x7fa800069020 2024-03-21T10:29:12.671 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:12.670+0000 7fa808800700 1 -- 172.21.15.153:0/3235655477 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa800109130 con 0x7fa800069020 2024-03-21T10:29:12.671 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:12.671+0000 7fa7faa00700 1 -- 172.21.15.153:0/3235655477 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fa7f000faa0 con 0x7fa800069020 2024-03-21T10:29:12.672 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:12.672+0000 7fa7faa00700 1 --2- 172.21.15.153:0/3235655477 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa7e806e460 0x7fa7e8070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:12.672 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:12.672+0000 7fa7faa00700 1 -- 172.21.15.153:0/3235655477 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fa7f008e520 con 0x7fa800069020 2024-03-21T10:29:12.672 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:12.673+0000 7fa805e00700 1 --2- 172.21.15.153:0/3235655477 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa7e806e460 0x7fa7e8070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:12.673 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:12.673+0000 7fa805e00700 1 --2- 172.21.15.153:0/3235655477 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa7e806e460 0x7fa7e8070920 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa7f4006010 tx=0x7fa7f401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:12.677 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:12.678+0000 7fa7faa00700 1 -- 172.21.15.153:0/3235655477 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa7f005ae60 con 0x7fa800069020 2024-03-21T10:29:12.988 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:12.989+0000 7fa808800700 1 -- 172.21.15.153:0/3235655477 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fa80004ea90 con 0x7fa800069020 2024-03-21T10:29:12.989 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:12.990+0000 7fa7faa00700 1 -- 172.21.15.153:0/3235655477 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+805 (secure 0 0 0) 0x7fa7f005ac80 con 0x7fa800069020 2024-03-21T10:29:12.989 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:29:12.990 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": { 2024-03-21T10:29:12.990 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:29:12.990 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:29:12.990 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": { 2024-03-21T10:29:12.990 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:29:12.990 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:29:12.990 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": { 2024-03-21T10:29:12.990 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 6 2024-03-21T10:29:12.990 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:29:12.990 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": { 2024-03-21T10:29:12.990 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-03-21T10:29:12.990 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 3 2024-03-21T10:29:12.991 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:29:12.991 INFO:teuthology.orchestra.run.smithi153.stdout: "overall": { 2024-03-21T10:29:12.991 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-03-21T10:29:12.991 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 13 2024-03-21T10:29:12.991 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-03-21T10:29:12.991 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:29:12.991 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:12.993+0000 7fa808800700 1 -- 172.21.15.153:0/3235655477 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa7e806e460 msgr2=0x7fa7e8070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:12.992 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:12.993+0000 7fa808800700 1 --2- 172.21.15.153:0/3235655477 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa7e806e460 0x7fa7e8070920 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa7f4006010 tx=0x7fa7f401a040 comp rx=0 tx=0).stop 2024-03-21T10:29:12.992 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:12.993+0000 7fa808800700 1 -- 172.21.15.153:0/3235655477 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa800069020 msgr2=0x7fa80019ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:12.992 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:12.993+0000 7fa808800700 1 --2- 172.21.15.153:0/3235655477 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa800069020 0x7fa80019ca50 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fa7f000c960 tx=0x7fa7f000cd20 comp rx=0 tx=0).stop 2024-03-21T10:29:12.992 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:12.993+0000 7fa808800700 1 -- 172.21.15.153:0/3235655477 shutdown_connections 2024-03-21T10:29:12.992 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:12.993+0000 7fa808800700 1 --2- 172.21.15.153:0/3235655477 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa800068670 0x7fa80019c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:12.992 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:12.993+0000 7fa808800700 1 --2- 172.21.15.153:0/3235655477 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa7e806e460 0x7fa7e8070920 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:12.993 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:12.993+0000 7fa808800700 1 --2- 172.21.15.153:0/3235655477 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa800069020 0x7fa80019ca50 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:12.993 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:12.993+0000 7fa808800700 1 -- 172.21.15.153:0/3235655477 >> 172.21.15.153:0/3235655477 conn(0x7fa8000755a0 msgr2=0x7fa8000fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:12.993 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:12.993+0000 7fa808800700 1 -- 172.21.15.153:0/3235655477 shutdown_connections 2024-03-21T10:29:12.993 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:12.993+0000 7fa808800700 1 -- 172.21.15.153:0/3235655477 wait complete. 2024-03-21T10:29:13.135 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.135+0000 7f688ba00700 1 -- 172.21.15.153:0/2163987251 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f68840ff510 msgr2=0x7f68840ff8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:13.135 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.135+0000 7f688ba00700 1 --2- 172.21.15.153:0/2163987251 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f68840ff510 0x7f68840ff8f0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f6878009910 tx=0x7f6878009c20 comp rx=0 tx=0).stop 2024-03-21T10:29:13.135 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.135+0000 7f688ba00700 1 -- 172.21.15.153:0/2163987251 shutdown_connections 2024-03-21T10:29:13.135 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.135+0000 7f688ba00700 1 --2- 172.21.15.153:0/2163987251 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f68840ffec0 0x7f68841057b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:13.135 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.135+0000 7f688ba00700 1 --2- 172.21.15.153:0/2163987251 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f68840ff510 0x7f68840ff8f0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:13.135 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.135+0000 7f688ba00700 1 -- 172.21.15.153:0/2163987251 >> 172.21.15.153:0/2163987251 conn(0x7f6884074e00 msgr2=0x7f6884075210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:13.135 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.136+0000 7f688ba00700 1 -- 172.21.15.153:0/2163987251 shutdown_connections 2024-03-21T10:29:13.135 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.136+0000 7f688ba00700 1 -- 172.21.15.153:0/2163987251 wait complete. 2024-03-21T10:29:13.136 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.137+0000 7f688ba00700 1 Processor -- start 2024-03-21T10:29:13.136 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.137+0000 7f688ba00700 1 -- start start 2024-03-21T10:29:13.137 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.138+0000 7f688ba00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f68840ff510 0x7f6884100490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:13.137 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.138+0000 7f688ba00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f68840ffec0 0x7f68841009d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:13.137 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.138+0000 7f688ba00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6884100f10 con 0x7f68840ffec0 2024-03-21T10:29:13.137 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.138+0000 7f688ba00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6884101050 con 0x7f68840ff510 2024-03-21T10:29:13.137 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.138+0000 7f6883e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f68840ffec0 0x7f68841009d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:13.137 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.138+0000 7f6883e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f68840ffec0 0x7f68841009d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:48780/0 (socket says 172.21.15.153:48780) 2024-03-21T10:29:13.137 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.138+0000 7f6883e00700 1 -- 172.21.15.153:0/1039741383 learned_addr learned my addr 172.21.15.153:0/1039741383 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:29:13.138 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.138+0000 7f6889000700 1 --2- 172.21.15.153:0/1039741383 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f68840ff510 0x7f6884100490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:13.138 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.138+0000 7f6883e00700 1 -- 172.21.15.153:0/1039741383 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f68840ff510 msgr2=0x7f6884100490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:13.138 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.138+0000 7f6883e00700 1 --2- 172.21.15.153:0/1039741383 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f68840ff510 0x7f6884100490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:13.138 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.138+0000 7f6883e00700 1 -- 172.21.15.153:0/1039741383 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68780095f0 con 0x7f68840ffec0 2024-03-21T10:29:13.138 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.139+0000 7f6883e00700 1 --2- 172.21.15.153:0/1039741383 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f68840ffec0 0x7f68841009d0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f687000d900 tx=0x7f687000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:13.138 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.139+0000 7f6881600700 1 -- 172.21.15.153:0/1039741383 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6870009900 con 0x7f68840ffec0 2024-03-21T10:29:13.139 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.139+0000 7f6881600700 1 -- 172.21.15.153:0/1039741383 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6870010460 con 0x7f68840ffec0 2024-03-21T10:29:13.139 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.139+0000 7f688ba00700 1 -- 172.21.15.153:0/1039741383 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68841a2970 con 0x7f68840ffec0 2024-03-21T10:29:13.139 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.139+0000 7f6881600700 1 -- 172.21.15.153:0/1039741383 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f687000f5f0 con 0x7f68840ffec0 2024-03-21T10:29:13.139 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.139+0000 7f688ba00700 1 -- 172.21.15.153:0/1039741383 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68841a2d80 con 0x7f68840ffec0 2024-03-21T10:29:13.139 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.140+0000 7f688ba00700 1 -- 172.21.15.153:0/1039741383 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68841091e0 con 0x7f68840ffec0 2024-03-21T10:29:13.140 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.141+0000 7f6881600700 1 -- 172.21.15.153:0/1039741383 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f68700105d0 con 0x7f68840ffec0 2024-03-21T10:29:13.142 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.143+0000 7f6881600700 1 --2- 172.21.15.153:0/1039741383 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f686c06e460 0x7f686c070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:13.142 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.144+0000 7f6881600700 1 -- 172.21.15.153:0/1039741383 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f687008e360 con 0x7f68840ffec0 2024-03-21T10:29:13.142 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.144+0000 7f6889000700 1 --2- 172.21.15.153:0/1039741383 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f686c06e460 0x7f686c070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:13.143 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.144+0000 7f6889000700 1 --2- 172.21.15.153:0/1039741383 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f686c06e460 0x7f686c070920 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f6878006010 tx=0x7f687801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:13.149 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.150+0000 7f6881600700 1 -- 172.21.15.153:0/1039741383 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f6870057400 con 0x7f68840ffec0 2024-03-21T10:29:13.416 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.417+0000 7f688ba00700 1 -- 172.21.15.153:0/1039741383 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f688404ea90 con 0x7f68840ffec0 2024-03-21T10:29:13.416 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.418+0000 7f6881600700 1 -- 172.21.15.153:0/1039741383 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1876 (secure 0 0 0) 0x7f6870017770 con 0x7f68840ffec0 2024-03-21T10:29:13.417 INFO:teuthology.orchestra.run.smithi153.stdout:e30 2024-03-21T10:29:13.417 INFO:teuthology.orchestra.run.smithi153.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-21T10:29:13.417 INFO:teuthology.orchestra.run.smithi153.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:29:13.417 INFO:teuthology.orchestra.run.smithi153.stdout:legacy client fscid: 1 2024-03-21T10:29:13.417 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:29:13.417 INFO:teuthology.orchestra.run.smithi153.stdout:Filesystem 'cephfs' (1) 2024-03-21T10:29:13.417 INFO:teuthology.orchestra.run.smithi153.stdout:fs_name cephfs 2024-03-21T10:29:13.417 INFO:teuthology.orchestra.run.smithi153.stdout:epoch 23 2024-03-21T10:29:13.417 INFO:teuthology.orchestra.run.smithi153.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-21T10:29:13.418 INFO:teuthology.orchestra.run.smithi153.stdout:created 2024-03-21T10:04:27.086576+0000 2024-03-21T10:29:13.418 INFO:teuthology.orchestra.run.smithi153.stdout:modified 2024-03-21T10:19:05.022092+0000 2024-03-21T10:29:13.418 INFO:teuthology.orchestra.run.smithi153.stdout:tableserver 0 2024-03-21T10:29:13.418 INFO:teuthology.orchestra.run.smithi153.stdout:root 0 2024-03-21T10:29:13.418 INFO:teuthology.orchestra.run.smithi153.stdout:session_timeout 60 2024-03-21T10:29:13.418 INFO:teuthology.orchestra.run.smithi153.stdout:session_autoclose 300 2024-03-21T10:29:13.418 INFO:teuthology.orchestra.run.smithi153.stdout:max_file_size 1099511627776 2024-03-21T10:29:13.418 INFO:teuthology.orchestra.run.smithi153.stdout:max_xattr_size 65536 2024-03-21T10:29:13.418 INFO:teuthology.orchestra.run.smithi153.stdout:required_client_features {} 2024-03-21T10:29:13.418 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure 0 2024-03-21T10:29:13.418 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure_osd_epoch 94 2024-03-21T10:29:13.418 INFO:teuthology.orchestra.run.smithi153.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:29:13.418 INFO:teuthology.orchestra.run.smithi153.stdout:max_mds 2 2024-03-21T10:29:13.418 INFO:teuthology.orchestra.run.smithi153.stdout:in 0,1 2024-03-21T10:29:13.419 INFO:teuthology.orchestra.run.smithi153.stdout:up {} 2024-03-21T10:29:13.419 INFO:teuthology.orchestra.run.smithi153.stdout:failed 0,1 2024-03-21T10:29:13.419 INFO:teuthology.orchestra.run.smithi153.stdout:damaged 2024-03-21T10:29:13.419 INFO:teuthology.orchestra.run.smithi153.stdout:stopped 2024-03-21T10:29:13.419 INFO:teuthology.orchestra.run.smithi153.stdout:data_pools [3] 2024-03-21T10:29:13.419 INFO:teuthology.orchestra.run.smithi153.stdout:metadata_pool 2 2024-03-21T10:29:13.419 INFO:teuthology.orchestra.run.smithi153.stdout:inline_data enabled 2024-03-21T10:29:13.419 INFO:teuthology.orchestra.run.smithi153.stdout:balancer 2024-03-21T10:29:13.419 INFO:teuthology.orchestra.run.smithi153.stdout:bal_rank_mask -1 2024-03-21T10:29:13.419 INFO:teuthology.orchestra.run.smithi153.stdout:standby_count_wanted 1 2024-03-21T10:29:13.419 INFO:teuthology.orchestra.run.smithi153.stdout:qdb_cluster leader: 0 members: 2024-03-21T10:29:13.419 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:29:13.419 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:29:13.419 INFO:teuthology.orchestra.run.smithi153.stdout:Standby daemons: 2024-03-21T10:29:13.420 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:29:13.420 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.zipckr{-1:34442} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6826/1103846295,v1:172.21.15.153:6827/1103846295] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:29:13.420 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.iuyluj{-1:34474} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6828/1738630832,v1:172.21.15.153:6829/1738630832] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:29:13.420 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.nhozag{-1:44269} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.174:6826/3967223227,v1:172.21.15.174:6827/3967223227] compat {c=[1],r=[1],i=[7ff]}] 2024-03-21T10:29:13.420 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.yyimst{-1:44313} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.174:6824/4127786797,v1:172.21.15.174:6825/4127786797] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:29:13.420 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.421+0000 7f688ba00700 1 -- 172.21.15.153:0/1039741383 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f686c06e460 msgr2=0x7f686c070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:13.421 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.421+0000 7f688ba00700 1 --2- 172.21.15.153:0/1039741383 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f686c06e460 0x7f686c070920 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f6878006010 tx=0x7f687801a040 comp rx=0 tx=0).stop 2024-03-21T10:29:13.421 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.421+0000 7f688ba00700 1 -- 172.21.15.153:0/1039741383 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f68840ffec0 msgr2=0x7f68841009d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:13.421 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.421+0000 7f688ba00700 1 --2- 172.21.15.153:0/1039741383 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f68840ffec0 0x7f68841009d0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f687000d900 tx=0x7f687000dcc0 comp rx=0 tx=0).stop 2024-03-21T10:29:13.421 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.421+0000 7f688ba00700 1 -- 172.21.15.153:0/1039741383 shutdown_connections 2024-03-21T10:29:13.421 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.421+0000 7f688ba00700 1 --2- 172.21.15.153:0/1039741383 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f68840ff510 0x7f6884100490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:13.421 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.421+0000 7f688ba00700 1 --2- 172.21.15.153:0/1039741383 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f686c06e460 0x7f686c070920 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:13.421 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.421+0000 7f688ba00700 1 --2- 172.21.15.153:0/1039741383 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f68840ffec0 0x7f68841009d0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:13.421 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.421+0000 7f688ba00700 1 -- 172.21.15.153:0/1039741383 >> 172.21.15.153:0/1039741383 conn(0x7f6884074e00 msgr2=0x7f68840feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:13.421 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.421+0000 7f688ba00700 1 -- 172.21.15.153:0/1039741383 shutdown_connections 2024-03-21T10:29:13.421 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.421+0000 7f688ba00700 1 -- 172.21.15.153:0/1039741383 wait complete. 2024-03-21T10:29:13.422 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 30 2024-03-21T10:29:13.554 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:29:13 smithi153 ceph-mon[113733]: pgmap v520: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:29:13.555 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:29:13 smithi153 ceph-mon[113733]: from='client.34954 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:29:13.555 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:29:13 smithi153 ceph-mon[113733]: from='client.34958 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:29:13.555 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:29:13 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/3235655477' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:29:13.573 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.574+0000 7f5231000700 1 -- 172.21.15.153:0/3122514592 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f522c0731c0 msgr2=0x7f522c0735a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:13.573 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.574+0000 7f5231000700 1 --2- 172.21.15.153:0/3122514592 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f522c0731c0 0x7f522c0735a0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f521c009910 tx=0x7f521c009c20 comp rx=0 tx=0).stop 2024-03-21T10:29:13.573 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.574+0000 7f5231000700 1 -- 172.21.15.153:0/3122514592 shutdown_connections 2024-03-21T10:29:13.574 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.574+0000 7f5231000700 1 --2- 172.21.15.153:0/3122514592 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f522c073ae0 0x7f522c10ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:13.574 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.574+0000 7f5231000700 1 --2- 172.21.15.153:0/3122514592 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f522c0731c0 0x7f522c0735a0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:13.574 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.574+0000 7f5231000700 1 -- 172.21.15.153:0/3122514592 >> 172.21.15.153:0/3122514592 conn(0x7f522c0fc630 msgr2=0x7f522c0fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:13.574 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.574+0000 7f5231000700 1 -- 172.21.15.153:0/3122514592 shutdown_connections 2024-03-21T10:29:13.574 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.574+0000 7f5231000700 1 -- 172.21.15.153:0/3122514592 wait complete. 2024-03-21T10:29:13.574 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.575+0000 7f5231000700 1 Processor -- start 2024-03-21T10:29:13.574 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.576+0000 7f5231000700 1 -- start start 2024-03-21T10:29:13.575 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.576+0000 7f5231000700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f522c0731c0 0x7f522c198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:13.576 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.576+0000 7f5231000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f522c073ae0 0x7f522c198e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:13.576 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.576+0000 7f5231000700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f522c199520 con 0x7f522c073ae0 2024-03-21T10:29:13.576 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.576+0000 7f5231000700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f522c19d330 con 0x7f522c0731c0 2024-03-21T10:29:13.576 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.576+0000 7f5221400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f522c073ae0 0x7f522c198e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:13.576 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.576+0000 7f5221400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f522c073ae0 0x7f522c198e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:48788/0 (socket says 172.21.15.153:48788) 2024-03-21T10:29:13.576 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.576+0000 7f5221400700 1 -- 172.21.15.153:0/3221838342 learned_addr learned my addr 172.21.15.153:0/3221838342 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:29:13.576 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.577+0000 7f5229e00700 1 --2- 172.21.15.153:0/3221838342 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f522c0731c0 0x7f522c198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:13.576 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.577+0000 7f5221400700 1 -- 172.21.15.153:0/3221838342 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f522c0731c0 msgr2=0x7f522c198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:13.576 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.577+0000 7f5221400700 1 --2- 172.21.15.153:0/3221838342 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f522c0731c0 0x7f522c198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:13.576 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.577+0000 7f5221400700 1 -- 172.21.15.153:0/3221838342 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f521c0095f0 con 0x7f522c073ae0 2024-03-21T10:29:13.577 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.577+0000 7f5221400700 1 --2- 172.21.15.153:0/3221838342 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f522c073ae0 0x7f522c198e90 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f521400d8d0 tx=0x7f521400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:13.577 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.577+0000 7f5223400700 1 -- 172.21.15.153:0/3221838342 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5214009940 con 0x7f522c073ae0 2024-03-21T10:29:13.577 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.577+0000 7f5223400700 1 -- 172.21.15.153:0/3221838342 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5214010460 con 0x7f522c073ae0 2024-03-21T10:29:13.577 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.577+0000 7f5231000700 1 -- 172.21.15.153:0/3221838342 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f522c19d610 con 0x7f522c073ae0 2024-03-21T10:29:13.577 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.577+0000 7f5223400700 1 -- 172.21.15.153:0/3221838342 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f521400f5f0 con 0x7f522c073ae0 2024-03-21T10:29:13.577 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.578+0000 7f5231000700 1 -- 172.21.15.153:0/3221838342 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f522c19db60 con 0x7f522c073ae0 2024-03-21T10:29:13.578 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.579+0000 7f5231000700 1 -- 172.21.15.153:0/3221838342 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f522c10a540 con 0x7f522c073ae0 2024-03-21T10:29:13.578 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.580+0000 7f5223400700 1 -- 172.21.15.153:0/3221838342 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f52140105d0 con 0x7f522c073ae0 2024-03-21T10:29:13.579 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.580+0000 7f5223400700 1 --2- 172.21.15.153:0/3221838342 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f521006e460 0x7f5210070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:13.579 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.581+0000 7f5223400700 1 -- 172.21.15.153:0/3221838342 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f521408e510 con 0x7f522c073ae0 2024-03-21T10:29:13.579 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.581+0000 7f5229e00700 1 --2- 172.21.15.153:0/3221838342 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f521006e460 0x7f5210070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:13.580 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.581+0000 7f5229e00700 1 --2- 172.21.15.153:0/3221838342 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f521006e460 0x7f5210070920 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f521c00b5c0 tx=0x7f521c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:13.584 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.585+0000 7f5223400700 1 -- 172.21.15.153:0/3221838342 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f521405b100 con 0x7f522c073ae0 2024-03-21T10:29:13.700 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:29:13 smithi174 ceph-mon[88004]: pgmap v520: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:29:13.700 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:29:13 smithi174 ceph-mon[88004]: from='client.34954 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:29:13.700 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:29:13 smithi174 ceph-mon[88004]: from='client.34958 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:29:13.700 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:29:13 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/3235655477' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:29:13.809 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.810+0000 7f5231000700 1 -- 172.21.15.153:0/3221838342 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f522c19dd80 con 0x7f521006e460 2024-03-21T10:29:13.811 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.812+0000 7f5223400700 1 -- 172.21.15.153:0/3221838342 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f522c19dd80 con 0x7f521006e460 2024-03-21T10:29:13.811 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:29:13.811 INFO:teuthology.orchestra.run.smithi153.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232", 2024-03-21T10:29:13.811 INFO:teuthology.orchestra.run.smithi153.stdout: "in_progress": true, 2024-03-21T10:29:13.811 INFO:teuthology.orchestra.run.smithi153.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-21T10:29:13.811 INFO:teuthology.orchestra.run.smithi153.stdout: "services_complete": [ 2024-03-21T10:29:13.811 INFO:teuthology.orchestra.run.smithi153.stdout: "crash", 2024-03-21T10:29:13.811 INFO:teuthology.orchestra.run.smithi153.stdout: "mon", 2024-03-21T10:29:13.812 INFO:teuthology.orchestra.run.smithi153.stdout: "osd", 2024-03-21T10:29:13.812 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr" 2024-03-21T10:29:13.812 INFO:teuthology.orchestra.run.smithi153.stdout: ], 2024-03-21T10:29:13.812 INFO:teuthology.orchestra.run.smithi153.stdout: "progress": "15/23 daemons upgraded", 2024-03-21T10:29:13.812 INFO:teuthology.orchestra.run.smithi153.stdout: "message": "Currently upgrading mds daemons", 2024-03-21T10:29:13.812 INFO:teuthology.orchestra.run.smithi153.stdout: "is_paused": false 2024-03-21T10:29:13.812 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:29:13.814 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.814+0000 7f5231000700 1 -- 172.21.15.153:0/3221838342 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f521006e460 msgr2=0x7f5210070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:13.814 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.814+0000 7f5231000700 1 --2- 172.21.15.153:0/3221838342 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f521006e460 0x7f5210070920 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f521c00b5c0 tx=0x7f521c01a040 comp rx=0 tx=0).stop 2024-03-21T10:29:13.814 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.814+0000 7f5231000700 1 -- 172.21.15.153:0/3221838342 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f522c073ae0 msgr2=0x7f522c198e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:13.814 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.814+0000 7f5231000700 1 --2- 172.21.15.153:0/3221838342 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f522c073ae0 0x7f522c198e90 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f521400d8d0 tx=0x7f521400dc90 comp rx=0 tx=0).stop 2024-03-21T10:29:13.814 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.814+0000 7f5231000700 1 -- 172.21.15.153:0/3221838342 shutdown_connections 2024-03-21T10:29:13.814 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.815+0000 7f5231000700 1 --2- 172.21.15.153:0/3221838342 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f522c0731c0 0x7f522c198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:13.814 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.815+0000 7f5231000700 1 --2- 172.21.15.153:0/3221838342 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f521006e460 0x7f5210070920 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:13.814 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.815+0000 7f5231000700 1 --2- 172.21.15.153:0/3221838342 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f522c073ae0 0x7f522c198e90 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:13.814 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.815+0000 7f5231000700 1 -- 172.21.15.153:0/3221838342 >> 172.21.15.153:0/3221838342 conn(0x7f522c0fc630 msgr2=0x7f522c107680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:13.814 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.815+0000 7f5231000700 1 -- 172.21.15.153:0/3221838342 shutdown_connections 2024-03-21T10:29:13.814 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.815+0000 7f5231000700 1 -- 172.21.15.153:0/3221838342 wait complete. 2024-03-21T10:29:13.964 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.964+0000 7f13db000700 1 -- 172.21.15.153:0/3519698568 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f13d40731c0 msgr2=0x7f13d40735a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:13.964 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.964+0000 7f13db000700 1 --2- 172.21.15.153:0/3519698568 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f13d40731c0 0x7f13d40735a0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f13c4009910 tx=0x7f13c4009c20 comp rx=0 tx=0).stop 2024-03-21T10:29:13.964 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.965+0000 7f13db000700 1 -- 172.21.15.153:0/3519698568 shutdown_connections 2024-03-21T10:29:13.964 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.965+0000 7f13db000700 1 --2- 172.21.15.153:0/3519698568 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f13d4073ae0 0x7f13d410ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:13.964 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.965+0000 7f13db000700 1 --2- 172.21.15.153:0/3519698568 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f13d40731c0 0x7f13d40735a0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:13.964 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.965+0000 7f13db000700 1 -- 172.21.15.153:0/3519698568 >> 172.21.15.153:0/3519698568 conn(0x7f13d40fc650 msgr2=0x7f13d40fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:13.964 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.965+0000 7f13db000700 1 -- 172.21.15.153:0/3519698568 shutdown_connections 2024-03-21T10:29:13.964 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.965+0000 7f13db000700 1 -- 172.21.15.153:0/3519698568 wait complete. 2024-03-21T10:29:13.965 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.966+0000 7f13db000700 1 Processor -- start 2024-03-21T10:29:13.965 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.966+0000 7f13db000700 1 -- start start 2024-03-21T10:29:13.966 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.967+0000 7f13db000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f13d40731c0 0x7f13d419e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:13.966 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.967+0000 7f13db000700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f13d4073ae0 0x7f13d419ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:13.966 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.967+0000 7f13db000700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13d419f2c0 con 0x7f13d40731c0 2024-03-21T10:29:13.966 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.967+0000 7f13db000700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13d4198770 con 0x7f13d4073ae0 2024-03-21T10:29:13.966 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.967+0000 7f13d3e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f13d40731c0 0x7f13d419e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:13.966 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.967+0000 7f13d3e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f13d40731c0 0x7f13d419e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:48812/0 (socket says 172.21.15.153:48812) 2024-03-21T10:29:13.966 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.967+0000 7f13d3e00700 1 -- 172.21.15.153:0/3131448822 learned_addr learned my addr 172.21.15.153:0/3131448822 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:29:13.967 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.968+0000 7f13d3400700 1 --2- 172.21.15.153:0/3131448822 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f13d4073ae0 0x7f13d419ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:13.967 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.968+0000 7f13d3e00700 1 -- 172.21.15.153:0/3131448822 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f13d4073ae0 msgr2=0x7f13d419ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:13.967 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.968+0000 7f13d3e00700 1 --2- 172.21.15.153:0/3131448822 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f13d4073ae0 0x7f13d419ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:13.968 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.968+0000 7f13d3e00700 1 -- 172.21.15.153:0/3131448822 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13c40095f0 con 0x7f13d40731c0 2024-03-21T10:29:13.968 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.968+0000 7f13d3400700 1 --2- 172.21.15.153:0/3131448822 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f13d4073ae0 0x7f13d419ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:29:13.968 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.968+0000 7f13d3e00700 1 --2- 172.21.15.153:0/3131448822 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f13d40731c0 0x7f13d419e6f0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f13c400bd70 tx=0x7f13c400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:13.968 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.969+0000 7f13d0c00700 1 -- 172.21.15.153:0/3131448822 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f13c401d070 con 0x7f13d40731c0 2024-03-21T10:29:13.968 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.969+0000 7f13d0c00700 1 -- 172.21.15.153:0/3131448822 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f13c400f460 con 0x7f13d40731c0 2024-03-21T10:29:13.968 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.969+0000 7f13db000700 1 -- 172.21.15.153:0/3131448822 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13d41989f0 con 0x7f13d40731c0 2024-03-21T10:29:13.968 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.969+0000 7f13d0c00700 1 -- 172.21.15.153:0/3131448822 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f13c4017630 con 0x7f13d40731c0 2024-03-21T10:29:13.968 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.969+0000 7f13db000700 1 -- 172.21.15.153:0/3131448822 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13d4198ee0 con 0x7f13d40731c0 2024-03-21T10:29:13.969 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.970+0000 7f13db000700 1 -- 172.21.15.153:0/3131448822 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13d410a540 con 0x7f13d40731c0 2024-03-21T10:29:13.970 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.971+0000 7f13d0c00700 1 -- 172.21.15.153:0/3131448822 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f13c4003a60 con 0x7f13d40731c0 2024-03-21T10:29:13.971 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.972+0000 7f13d0c00700 1 --2- 172.21.15.153:0/3131448822 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f13b806e460 0x7f13b8070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:13.971 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.972+0000 7f13d0c00700 1 -- 172.21.15.153:0/3131448822 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f13c408f910 con 0x7f13d40731c0 2024-03-21T10:29:13.971 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.973+0000 7f13d3400700 1 --2- 172.21.15.153:0/3131448822 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f13b806e460 0x7f13b8070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:13.972 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.973+0000 7f13d3400700 1 --2- 172.21.15.153:0/3131448822 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f13b806e460 0x7f13b8070920 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f13c0005fd0 tx=0x7f13c0005ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:13.976 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:13.977+0000 7f13d0c00700 1 -- 172.21.15.153:0/3131448822 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f13c405c640 con 0x7f13d40731c0 2024-03-21T10:29:14.283 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:14.284+0000 7f13db000700 1 -- 172.21.15.153:0/3131448822 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f13d404ea90 con 0x7f13d40731c0 2024-03-21T10:29:14.287 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:14.288+0000 7f13d0c00700 1 -- 172.21.15.153:0/3131448822 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f13c405c1d0 con 0x7f13d40731c0 2024-03-21T10:29:14.287 INFO:teuthology.orchestra.run.smithi153.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-03-21T10:29:14.287 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-21T10:29:14.287 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is degraded 2024-03-21T10:29:14.287 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-21T10:29:14.288 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-21T10:29:14.288 INFO:teuthology.orchestra.run.smithi153.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-21T10:29:14.288 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-21T10:29:14.290 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:14.291+0000 7f13db000700 1 -- 172.21.15.153:0/3131448822 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f13b806e460 msgr2=0x7f13b8070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:14.290 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:14.291+0000 7f13db000700 1 --2- 172.21.15.153:0/3131448822 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f13b806e460 0x7f13b8070920 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f13c0005fd0 tx=0x7f13c0005ee0 comp rx=0 tx=0).stop 2024-03-21T10:29:14.290 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:14.291+0000 7f13db000700 1 -- 172.21.15.153:0/3131448822 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f13d40731c0 msgr2=0x7f13d419e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:14.290 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:14.291+0000 7f13db000700 1 --2- 172.21.15.153:0/3131448822 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f13d40731c0 0x7f13d419e6f0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f13c400bd70 tx=0x7f13c400be50 comp rx=0 tx=0).stop 2024-03-21T10:29:14.290 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:14.291+0000 7f13db000700 1 -- 172.21.15.153:0/3131448822 shutdown_connections 2024-03-21T10:29:14.290 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:14.291+0000 7f13db000700 1 --2- 172.21.15.153:0/3131448822 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f13b806e460 0x7f13b8070920 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:14.290 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:14.291+0000 7f13db000700 1 --2- 172.21.15.153:0/3131448822 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f13d40731c0 0x7f13d419e6f0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:14.291 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:14.291+0000 7f13db000700 1 --2- 172.21.15.153:0/3131448822 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f13d4073ae0 0x7f13d419ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:14.291 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:14.291+0000 7f13db000700 1 -- 172.21.15.153:0/3131448822 >> 172.21.15.153:0/3131448822 conn(0x7f13d40fc650 msgr2=0x7f13d4107680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:14.291 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:14.291+0000 7f13db000700 1 -- 172.21.15.153:0/3131448822 shutdown_connections 2024-03-21T10:29:14.291 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:14.291+0000 7f13db000700 1 -- 172.21.15.153:0/3131448822 wait complete. 2024-03-21T10:29:14.555 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:29:14 smithi153 ceph-mon[113733]: from='client.34962 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:29:14.555 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:29:14 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/1039741383' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:29:14.700 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:29:14 smithi174 ceph-mon[88004]: from='client.34962 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:29:14.700 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:29:14 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/1039741383' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:29:15.555 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:29:15 smithi153 ceph-mon[113733]: pgmap v521: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:29:15.555 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:29:15 smithi153 ceph-mon[113733]: from='client.34974 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:29:15.555 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:29:15 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/3131448822' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:29:15.699 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:29:15 smithi174 ceph-mon[88004]: pgmap v521: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:29:15.700 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:29:15 smithi174 ceph-mon[88004]: from='client.34974 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:29:15.700 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:29:15 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/3131448822' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:29:16.449 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:29:16 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:29:16.554 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:29:16 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:29:17.555 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:29:17 smithi153 ceph-mon[113733]: pgmap v522: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:29:17.699 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:29:17 smithi174 ceph-mon[88004]: pgmap v522: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:29:19.700 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:29:19 smithi174 ceph-mon[88004]: pgmap v523: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:29:19.804 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:29:19 smithi153 ceph-mon[113733]: pgmap v523: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:29:21.700 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:29:21 smithi174 ceph-mon[88004]: pgmap v524: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:29:21.804 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:29:21 smithi153 ceph-mon[113733]: pgmap v524: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:29:23.700 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:29:23 smithi174 ceph-mon[88004]: pgmap v525: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:29:23.805 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:29:23 smithi153 ceph-mon[113733]: pgmap v525: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:29:25.700 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:29:25 smithi174 ceph-mon[88004]: pgmap v526: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:29:25.804 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:29:25 smithi153 ceph-mon[113733]: pgmap v526: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:29:27.699 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:29:27 smithi174 ceph-mon[88004]: pgmap v527: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:29:27.804 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:29:27 smithi153 ceph-mon[113733]: pgmap v527: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:29:29.700 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:29:29 smithi174 ceph-mon[88004]: pgmap v528: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:29:29.805 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:29:29 smithi153 ceph-mon[113733]: pgmap v528: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:29:31.625 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:29:31 smithi174 ceph-mon[88004]: pgmap v529: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:29:31.625 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:29:31 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:29:31.805 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:29:31 smithi153 ceph-mon[113733]: pgmap v529: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:29:31.805 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:29:31 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:29:33.700 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:29:33 smithi174 ceph-mon[88004]: pgmap v530: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:29:33.805 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:29:33 smithi153 ceph-mon[113733]: pgmap v530: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:29:35.699 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:29:35 smithi174 ceph-mon[88004]: pgmap v531: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:29:35.804 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:29:35 smithi153 ceph-mon[113733]: pgmap v531: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:29:37.699 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:29:37 smithi174 ceph-mon[88004]: pgmap v532: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:29:37.805 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:29:37 smithi153 ceph-mon[113733]: pgmap v532: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:29:39.699 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:29:39 smithi174 ceph-mon[88004]: pgmap v533: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:29:39.804 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:29:39 smithi153 ceph-mon[113733]: pgmap v533: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:29:41.699 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:29:41 smithi174 ceph-mon[88004]: pgmap v534: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:29:41.804 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:29:41 smithi153 ceph-mon[113733]: pgmap v534: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:29:43.699 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:29:43 smithi174 ceph-mon[88004]: pgmap v535: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:29:43.804 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:29:43 smithi153 ceph-mon[113733]: pgmap v535: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:29:44.462 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:44.462+0000 7f7de3a00700 1 -- 172.21.15.153:0/3314244185 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ddc101d10 msgr2=0x7f7ddc1020f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:44.463 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:44.462+0000 7f7de3a00700 1 --2- 172.21.15.153:0/3314244185 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ddc101d10 0x7f7ddc1020f0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f7dd0009910 tx=0x7f7dd0009c20 comp rx=0 tx=0).stop 2024-03-21T10:29:44.463 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:44.462+0000 7f7de3a00700 1 -- 172.21.15.153:0/3314244185 shutdown_connections 2024-03-21T10:29:44.463 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:44.462+0000 7f7de3a00700 1 --2- 172.21.15.153:0/3314244185 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7ddc102630 0x7f7ddc10ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:44.463 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:44.462+0000 7f7de3a00700 1 --2- 172.21.15.153:0/3314244185 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ddc101d10 0x7f7ddc1020f0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:44.463 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:44.462+0000 7f7de3a00700 1 -- 172.21.15.153:0/3314244185 >> 172.21.15.153:0/3314244185 conn(0x7f7ddc0fb540 msgr2=0x7f7ddc0fd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:44.463 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:44.462+0000 7f7de3a00700 1 -- 172.21.15.153:0/3314244185 shutdown_connections 2024-03-21T10:29:44.463 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:44.462+0000 7f7de3a00700 1 -- 172.21.15.153:0/3314244185 wait complete. 2024-03-21T10:29:44.464 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:44.464+0000 7f7de3a00700 1 Processor -- start 2024-03-21T10:29:44.464 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:44.464+0000 7f7de3a00700 1 -- start start 2024-03-21T10:29:44.464 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:44.464+0000 7f7de3a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ddc101d10 0x7f7ddc0ff970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:44.464 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:44.464+0000 7f7de3a00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7ddc102630 0x7f7ddc0ffeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:44.464 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:44.464+0000 7f7de3a00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ddc1003f0 con 0x7f7ddc101d10 2024-03-21T10:29:44.464 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:44.464+0000 7f7de3a00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ddc100530 con 0x7f7ddc102630 2024-03-21T10:29:44.464 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:44.464+0000 7f7de1000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ddc101d10 0x7f7ddc0ff970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:44.464 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:44.464+0000 7f7de1000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ddc101d10 0x7f7ddc0ff970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:50754/0 (socket says 172.21.15.153:50754) 2024-03-21T10:29:44.465 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:44.464+0000 7f7de1000700 1 -- 172.21.15.153:0/4237252262 learned_addr learned my addr 172.21.15.153:0/4237252262 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:29:44.465 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:44.465+0000 7f7ddbe00700 1 --2- 172.21.15.153:0/4237252262 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7ddc102630 0x7f7ddc0ffeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:44.465 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:44.465+0000 7f7de1000700 1 -- 172.21.15.153:0/4237252262 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7ddc102630 msgr2=0x7f7ddc0ffeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:44.466 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:44.465+0000 7f7de1000700 1 --2- 172.21.15.153:0/4237252262 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7ddc102630 0x7f7ddc0ffeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:44.466 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:44.465+0000 7f7de1000700 1 -- 172.21.15.153:0/4237252262 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dd00095f0 con 0x7f7ddc101d10 2024-03-21T10:29:44.466 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:44.465+0000 7f7de1000700 1 --2- 172.21.15.153:0/4237252262 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ddc101d10 0x7f7ddc0ff970 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f7dd000b5c0 tx=0x7f7dd000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:44.466 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:44.466+0000 7f7dd9600700 1 -- 172.21.15.153:0/4237252262 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7dd001d070 con 0x7f7ddc101d10 2024-03-21T10:29:44.466 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:44.466+0000 7f7dd9600700 1 -- 172.21.15.153:0/4237252262 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7dd0005e00 con 0x7f7ddc101d10 2024-03-21T10:29:44.466 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:44.466+0000 7f7dd9600700 1 -- 172.21.15.153:0/4237252262 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7dd0017400 con 0x7f7ddc101d10 2024-03-21T10:29:44.466 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:44.466+0000 7f7de3a00700 1 -- 172.21.15.153:0/4237252262 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ddc1007b0 con 0x7f7ddc101d10 2024-03-21T10:29:44.466 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:44.466+0000 7f7de3a00700 1 -- 172.21.15.153:0/4237252262 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ddc1a2c70 con 0x7f7ddc101d10 2024-03-21T10:29:44.466 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:44.467+0000 7f7de3a00700 1 -- 172.21.15.153:0/4237252262 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ddc108220 con 0x7f7ddc101d10 2024-03-21T10:29:44.467 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:44.468+0000 7f7dd9600700 1 -- 172.21.15.153:0/4237252262 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f7dd000f460 con 0x7f7ddc101d10 2024-03-21T10:29:44.469 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:44.469+0000 7f7dd9600700 1 --2- 172.21.15.153:0/4237252262 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f7dc406e460 0x7f7dc4070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:44.469 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:44.469+0000 7f7dd9600700 1 -- 172.21.15.153:0/4237252262 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f7dd008f9b0 con 0x7f7ddc101d10 2024-03-21T10:29:44.469 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:44.469+0000 7f7ddbe00700 1 --2- 172.21.15.153:0/4237252262 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f7dc406e460 0x7f7dc4070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:44.470 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:44.470+0000 7f7ddbe00700 1 --2- 172.21.15.153:0/4237252262 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f7dc406e460 0x7f7dc4070920 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f7dc8005fd0 tx=0x7f7dc8005f40 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:44.474 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:44.474+0000 7f7dd9600700 1 -- 172.21.15.153:0/4237252262 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7dd0058a50 con 0x7f7ddc101d10 2024-03-21T10:29:44.718 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:44.718+0000 7f7de3a00700 1 -- 172.21.15.153:0/4237252262 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7ddc02cce0 con 0x7f7dc406e460 2024-03-21T10:29:44.719 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:44.720+0000 7f7dd9600700 1 -- 172.21.15.153:0/4237252262 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f7ddc02cce0 con 0x7f7dc406e460 2024-03-21T10:29:44.723 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:44.723+0000 7f7de3a00700 1 -- 172.21.15.153:0/4237252262 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f7dc406e460 msgr2=0x7f7dc4070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:44.723 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:44.723+0000 7f7de3a00700 1 --2- 172.21.15.153:0/4237252262 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f7dc406e460 0x7f7dc4070920 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f7dc8005fd0 tx=0x7f7dc8005f40 comp rx=0 tx=0).stop 2024-03-21T10:29:44.723 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:44.723+0000 7f7de3a00700 1 -- 172.21.15.153:0/4237252262 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ddc101d10 msgr2=0x7f7ddc0ff970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:44.723 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:44.723+0000 7f7de3a00700 1 --2- 172.21.15.153:0/4237252262 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ddc101d10 0x7f7ddc0ff970 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f7dd000b5c0 tx=0x7f7dd000be20 comp rx=0 tx=0).stop 2024-03-21T10:29:44.724 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:44.723+0000 7f7de3a00700 1 -- 172.21.15.153:0/4237252262 shutdown_connections 2024-03-21T10:29:44.724 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:44.723+0000 7f7de3a00700 1 --2- 172.21.15.153:0/4237252262 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f7dc406e460 0x7f7dc4070920 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:44.724 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:44.723+0000 7f7de3a00700 1 --2- 172.21.15.153:0/4237252262 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7ddc101d10 0x7f7ddc0ff970 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:44.724 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:44.723+0000 7f7de3a00700 1 --2- 172.21.15.153:0/4237252262 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7ddc102630 0x7f7ddc0ffeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:44.724 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:44.724+0000 7f7de3a00700 1 -- 172.21.15.153:0/4237252262 >> 172.21.15.153:0/4237252262 conn(0x7f7ddc0fb540 msgr2=0x7f7ddc105360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:44.724 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:44.724+0000 7f7de3a00700 1 -- 172.21.15.153:0/4237252262 shutdown_connections 2024-03-21T10:29:44.724 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:44.724+0000 7f7de3a00700 1 -- 172.21.15.153:0/4237252262 wait complete. 2024-03-21T10:29:44.740 INFO:teuthology.orchestra.run.smithi153.stdout:true 2024-03-21T10:29:44.884 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:44.884+0000 7f3ea9c00700 1 -- 172.21.15.153:0/3054255444 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ea410d280 msgr2=0x7f3ea410d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:44.884 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:44.884+0000 7f3ea9c00700 1 --2- 172.21.15.153:0/3054255444 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ea410d280 0x7f3ea410d660 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f3e94009910 tx=0x7f3e94009c20 comp rx=0 tx=0).stop 2024-03-21T10:29:44.884 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:44.884+0000 7f3ea9c00700 1 -- 172.21.15.153:0/3054255444 shutdown_connections 2024-03-21T10:29:44.884 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:44.884+0000 7f3ea9c00700 1 --2- 172.21.15.153:0/3054255444 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3ea41071d0 0x7f3ea4107650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:44.884 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:44.884+0000 7f3ea9c00700 1 --2- 172.21.15.153:0/3054255444 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ea410d280 0x7f3ea410d660 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:44.885 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:44.884+0000 7f3ea9c00700 1 -- 172.21.15.153:0/3054255444 >> 172.21.15.153:0/3054255444 conn(0x7f3ea40769e0 msgr2=0x7f3ea4078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:44.885 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:44.884+0000 7f3ea9c00700 1 -- 172.21.15.153:0/3054255444 shutdown_connections 2024-03-21T10:29:44.885 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:44.884+0000 7f3ea9c00700 1 -- 172.21.15.153:0/3054255444 wait complete. 2024-03-21T10:29:44.886 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:44.885+0000 7f3ea9c00700 1 Processor -- start 2024-03-21T10:29:44.886 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:44.885+0000 7f3ea9c00700 1 -- start start 2024-03-21T10:29:44.886 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:44.886+0000 7f3ea9c00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ea41071d0 0x7f3ea419ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:44.886 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:44.886+0000 7f3ea9c00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3ea410d280 0x7f3ea419d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:44.886 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:44.886+0000 7f3ea9c00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea419d8a0 con 0x7f3ea41071d0 2024-03-21T10:29:44.886 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:44.886+0000 7f3ea9c00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea41a05d0 con 0x7f3ea410d280 2024-03-21T10:29:44.886 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:44.886+0000 7f3ea2a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ea41071d0 0x7f3ea419ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:44.886 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:44.886+0000 7f3ea2a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ea41071d0 0x7f3ea419ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:50766/0 (socket says 172.21.15.153:50766) 2024-03-21T10:29:44.886 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:44.886+0000 7f3ea2a00700 1 -- 172.21.15.153:0/3267584607 learned_addr learned my addr 172.21.15.153:0/3267584607 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:29:44.887 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:44.886+0000 7f3ea2000700 1 --2- 172.21.15.153:0/3267584607 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3ea410d280 0x7f3ea419d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:44.887 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:44.886+0000 7f3ea2a00700 1 -- 172.21.15.153:0/3267584607 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3ea410d280 msgr2=0x7f3ea419d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:44.887 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:44.886+0000 7f3ea2a00700 1 --2- 172.21.15.153:0/3267584607 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3ea410d280 0x7f3ea419d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:44.887 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:44.886+0000 7f3ea2a00700 1 -- 172.21.15.153:0/3267584607 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e940095f0 con 0x7f3ea41071d0 2024-03-21T10:29:44.887 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:44.887+0000 7f3ea2a00700 1 --2- 172.21.15.153:0/3267584607 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ea41071d0 0x7f3ea419ccd0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f3e940092e0 tx=0x7f3e9400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:44.888 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:44.887+0000 7f3e9b400700 1 -- 172.21.15.153:0/3267584607 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3e9401d070 con 0x7f3ea41071d0 2024-03-21T10:29:44.888 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:44.887+0000 7f3e9b400700 1 -- 172.21.15.153:0/3267584607 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3e94005e00 con 0x7f3ea41071d0 2024-03-21T10:29:44.888 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:44.887+0000 7f3ea9c00700 1 -- 172.21.15.153:0/3267584607 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ea41a0850 con 0x7f3ea41071d0 2024-03-21T10:29:44.888 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:44.887+0000 7f3e9b400700 1 -- 172.21.15.153:0/3267584607 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3e94017400 con 0x7f3ea41071d0 2024-03-21T10:29:44.888 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:44.887+0000 7f3ea9c00700 1 -- 172.21.15.153:0/3267584607 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ea41a0dc0 con 0x7f3ea41071d0 2024-03-21T10:29:44.888 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:44.888+0000 7f3ea9c00700 1 -- 172.21.15.153:0/3267584607 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ea404ea90 con 0x7f3ea41071d0 2024-03-21T10:29:44.889 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:44.889+0000 7f3e9b400700 1 -- 172.21.15.153:0/3267584607 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f3e9400f460 con 0x7f3ea41071d0 2024-03-21T10:29:44.890 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:44.890+0000 7f3e9b400700 1 --2- 172.21.15.153:0/3267584607 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f3e8806e460 0x7f3e88070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:44.890 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:44.890+0000 7f3e9b400700 1 -- 172.21.15.153:0/3267584607 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f3e9408fae0 con 0x7f3ea41071d0 2024-03-21T10:29:44.890 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:44.891+0000 7f3ea2000700 1 --2- 172.21.15.153:0/3267584607 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f3e8806e460 0x7f3e88070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:44.891 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:44.891+0000 7f3ea2000700 1 --2- 172.21.15.153:0/3267584607 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f3e8806e460 0x7f3e88070920 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f3e90005950 tx=0x7f3e900058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:44.894 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:44.895+0000 7f3e9b400700 1 -- 172.21.15.153:0/3267584607 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f3e94058b00 con 0x7f3ea41071d0 2024-03-21T10:29:45.141 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:45.141+0000 7f3ea9c00700 1 -- 172.21.15.153:0/3267584607 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3ea41a11e0 con 0x7f3e8806e460 2024-03-21T10:29:45.143 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:45.144+0000 7f3e9b400700 1 -- 172.21.15.153:0/3267584607 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f3ea41a11e0 con 0x7f3e8806e460 2024-03-21T10:29:45.146 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:45.146+0000 7f3ea9c00700 1 -- 172.21.15.153:0/3267584607 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f3e8806e460 msgr2=0x7f3e88070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:45.147 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:45.146+0000 7f3ea9c00700 1 --2- 172.21.15.153:0/3267584607 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f3e8806e460 0x7f3e88070920 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f3e90005950 tx=0x7f3e900058e0 comp rx=0 tx=0).stop 2024-03-21T10:29:45.147 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:45.146+0000 7f3ea9c00700 1 -- 172.21.15.153:0/3267584607 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ea41071d0 msgr2=0x7f3ea419ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:45.147 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:45.146+0000 7f3ea9c00700 1 --2- 172.21.15.153:0/3267584607 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ea41071d0 0x7f3ea419ccd0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f3e940092e0 tx=0x7f3e9400be20 comp rx=0 tx=0).stop 2024-03-21T10:29:45.147 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:45.147+0000 7f3ea9c00700 1 -- 172.21.15.153:0/3267584607 shutdown_connections 2024-03-21T10:29:45.147 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:45.147+0000 7f3ea9c00700 1 --2- 172.21.15.153:0/3267584607 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f3e8806e460 0x7f3e88070920 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:45.147 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:45.147+0000 7f3ea9c00700 1 --2- 172.21.15.153:0/3267584607 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ea41071d0 0x7f3ea419ccd0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:45.147 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:45.147+0000 7f3ea9c00700 1 --2- 172.21.15.153:0/3267584607 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3ea410d280 0x7f3ea419d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:45.147 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:45.147+0000 7f3ea9c00700 1 -- 172.21.15.153:0/3267584607 >> 172.21.15.153:0/3267584607 conn(0x7f3ea40769e0 msgr2=0x7f3ea4078320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:45.147 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:45.147+0000 7f3ea9c00700 1 -- 172.21.15.153:0/3267584607 shutdown_connections 2024-03-21T10:29:45.147 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:45.147+0000 7f3ea9c00700 1 -- 172.21.15.153:0/3267584607 wait complete. 2024-03-21T10:29:45.292 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:45.292+0000 7f78b6600700 1 -- 172.21.15.153:0/881369106 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f78b00ff790 msgr2=0x7f78b00ffb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:45.293 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:45.292+0000 7f78b6600700 1 --2- 172.21.15.153:0/881369106 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f78b00ff790 0x7f78b00ffb70 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f78a4009910 tx=0x7f78a4009c20 comp rx=0 tx=0).stop 2024-03-21T10:29:45.293 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:45.292+0000 7f78b6600700 1 -- 172.21.15.153:0/881369106 shutdown_connections 2024-03-21T10:29:45.293 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:45.292+0000 7f78b6600700 1 --2- 172.21.15.153:0/881369106 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f78b0100140 0x7f78b010ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:45.293 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:45.292+0000 7f78b6600700 1 --2- 172.21.15.153:0/881369106 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f78b00ff790 0x7f78b00ffb70 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:45.293 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:45.292+0000 7f78b6600700 1 -- 172.21.15.153:0/881369106 >> 172.21.15.153:0/881369106 conn(0x7f78b00fb3d0 msgr2=0x7f78b00fd7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:45.293 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:45.292+0000 7f78b6600700 1 -- 172.21.15.153:0/881369106 shutdown_connections 2024-03-21T10:29:45.293 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:45.292+0000 7f78b6600700 1 -- 172.21.15.153:0/881369106 wait complete. 2024-03-21T10:29:45.294 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:45.293+0000 7f78b6600700 1 Processor -- start 2024-03-21T10:29:45.294 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:45.294+0000 7f78b6600700 1 -- start start 2024-03-21T10:29:45.295 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:45.294+0000 7f78b6600700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f78b00ff790 0x7f78b019e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:45.295 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:45.294+0000 7f78b6600700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f78b0100140 0x7f78b019eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:45.295 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:45.294+0000 7f78b6600700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78b019f170 con 0x7f78b00ff790 2024-03-21T10:29:45.295 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:45.294+0000 7f78b6600700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78b0198620 con 0x7f78b0100140 2024-03-21T10:29:45.295 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:45.294+0000 7f78b5200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f78b00ff790 0x7f78b019e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:45.295 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:45.294+0000 7f78b5200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f78b00ff790 0x7f78b019e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:50782/0 (socket says 172.21.15.153:50782) 2024-03-21T10:29:45.295 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:45.294+0000 7f78b5200700 1 -- 172.21.15.153:0/472131 learned_addr learned my addr 172.21.15.153:0/472131 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:29:45.295 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:45.295+0000 7f78afe00700 1 --2- 172.21.15.153:0/472131 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f78b0100140 0x7f78b019eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:45.295 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:45.295+0000 7f78b5200700 1 -- 172.21.15.153:0/472131 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f78b0100140 msgr2=0x7f78b019eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:45.295 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:45.295+0000 7f78b5200700 1 --2- 172.21.15.153:0/472131 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f78b0100140 0x7f78b019eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:45.295 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:45.295+0000 7f78b5200700 1 -- 172.21.15.153:0/472131 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78a40095f0 con 0x7f78b00ff790 2024-03-21T10:29:45.295 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:45.295+0000 7f78b5200700 1 --2- 172.21.15.153:0/472131 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f78b00ff790 0x7f78b019e5a0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f78a4000c00 tx=0x7f78a40056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:45.296 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:45.295+0000 7f78ad600700 1 -- 172.21.15.153:0/472131 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f78a401d070 con 0x7f78b00ff790 2024-03-21T10:29:45.296 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:45.295+0000 7f78ad600700 1 -- 172.21.15.153:0/472131 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f78a400f460 con 0x7f78b00ff790 2024-03-21T10:29:45.296 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:45.295+0000 7f78b6600700 1 -- 172.21.15.153:0/472131 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78b01988a0 con 0x7f78b00ff790 2024-03-21T10:29:45.296 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:45.295+0000 7f78ad600700 1 -- 172.21.15.153:0/472131 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f78a4021640 con 0x7f78b00ff790 2024-03-21T10:29:45.296 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:45.295+0000 7f78b6600700 1 -- 172.21.15.153:0/472131 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78b0198df0 con 0x7f78b00ff790 2024-03-21T10:29:45.297 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:45.296+0000 7f78b6600700 1 -- 172.21.15.153:0/472131 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78b010a570 con 0x7f78b00ff790 2024-03-21T10:29:45.297 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:45.297+0000 7f78ad600700 1 -- 172.21.15.153:0/472131 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f78a400fa60 con 0x7f78b00ff790 2024-03-21T10:29:45.298 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:45.298+0000 7f78ad600700 1 --2- 172.21.15.153:0/472131 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f789806e460 0x7f7898070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:45.298 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:45.298+0000 7f78ad600700 1 -- 172.21.15.153:0/472131 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f78a408f6e0 con 0x7f78b00ff790 2024-03-21T10:29:45.298 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:45.298+0000 7f78afe00700 1 --2- 172.21.15.153:0/472131 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f789806e460 0x7f7898070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:45.299 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:45.299+0000 7f78afe00700 1 --2- 172.21.15.153:0/472131 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f789806e460 0x7f7898070920 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f789c005950 tx=0x7f789c0058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:45.305 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:45.303+0000 7f78ad600700 1 -- 172.21.15.153:0/472131 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f78a405c2b0 con 0x7f78b00ff790 2024-03-21T10:29:45.538 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:45.538+0000 7f78b6600700 1 -- 172.21.15.153:0/472131 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f78b01990b0 con 0x7f789806e460 2024-03-21T10:29:45.556 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:45.556+0000 7f78ad600700 1 -- 172.21.15.153:0/472131 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f78b01990b0 con 0x7f789806e460 2024-03-21T10:29:45.557 INFO:teuthology.orchestra.run.smithi153.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-21T10:29:45.557 INFO:teuthology.orchestra.run.smithi153.stdout:alertmanager.smithi153 smithi153 *:9093,9094 running (18m) 9m ago 32m 14.7M - 0.25.0 c8568f914cd2 9ed3f2200b5f 2024-03-21T10:29:45.557 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi153 smithi153 running (32m) 9m ago 32m 9424k - 18.2.1 5be31c24972a 5b61994396a6 2024-03-21T10:29:45.557 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi174 smithi174 running (31m) 9m ago 31m 9244k - 18.2.1 5be31c24972a 4f7a5fc3ac28 2024-03-21T10:29:45.557 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi153 smithi153 running (14m) 9m ago 32m 6656k - 19.0.0-2231-g2edfee68 237e43c7a65f 32c21104d5b0 2024-03-21T10:29:45.557 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi174 smithi174 running (14m) 9m ago 30m 6660k - 19.0.0-2231-g2edfee68 237e43c7a65f a04e48967ab6 2024-03-21T10:29:45.557 INFO:teuthology.orchestra.run.smithi153.stdout:grafana.smithi153 smithi153 *:3000 running (18m) 9m ago 31m 73.6M - 9.4.12 2bacad6d85d8 ee24026ab786 2024-03-21T10:29:45.557 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.iuyluj smithi153 running (10m) 9m ago 25m 10.9M - 19.0.0-2231-g2edfee68 237e43c7a65f 87aff8a5de70 2024-03-21T10:29:45.557 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.zipckr smithi153 running (10m) 9m ago 25m 11.8M - 19.0.0-2231-g2edfee68 237e43c7a65f ccc262bd5dd3 2024-03-21T10:29:45.557 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.nhozag smithi174 running (25m) 9m ago 25m 12.0M - 18.2.1 5be31c24972a 2fee4000308f 2024-03-21T10:29:45.557 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.yyimst smithi174 running (9m) 9m ago 25m 10.7M - 19.0.0-2231-g2edfee68 237e43c7a65f fc1962f9351e 2024-03-21T10:29:45.557 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi153.tluxni smithi153 *:8443,9283,8765 running (21m) 9m ago 34m 509M - 19.0.0-2231-g2edfee68 237e43c7a65f 7ffee0e55382 2024-03-21T10:29:45.557 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi174.qpcwat smithi174 *:8443,9283,8765 running (20m) 9m ago 30m 452M - 19.0.0-2231-g2edfee68 237e43c7a65f 748968f216ab 2024-03-21T10:29:45.557 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi153 smithi153 running (16m) 9m ago 34m 55.0M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f dd8038303dc3 2024-03-21T10:29:45.558 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi174 smithi174 running (15m) 9m ago 30m 44.2M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f 29d78a788f2c 2024-03-21T10:29:45.558 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi153 smithi153 *:9100 running (19m) 9m ago 32m 7889k - 1.5.0 0da6a335fe13 bb835edc0e0b 2024-03-21T10:29:45.558 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi174 smithi174 *:9100 running (19m) 9m ago 30m 8102k - 1.5.0 0da6a335fe13 7c4ecad790fa 2024-03-21T10:29:45.558 INFO:teuthology.orchestra.run.smithi153.stdout:osd.0 smithi153 running (14m) 9m ago 29m 116M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 507900fd4791 2024-03-21T10:29:45.558 INFO:teuthology.orchestra.run.smithi153.stdout:osd.1 smithi153 running (13m) 9m ago 29m 89.2M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 0d646452e9c1 2024-03-21T10:29:45.558 INFO:teuthology.orchestra.run.smithi153.stdout:osd.2 smithi153 running (12m) 9m ago 28m 87.4M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 1326f2afdfcd 2024-03-21T10:29:45.558 INFO:teuthology.orchestra.run.smithi153.stdout:osd.3 smithi174 running (12m) 9m ago 28m 83.6M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 70aafac71ad5 2024-03-21T10:29:45.558 INFO:teuthology.orchestra.run.smithi153.stdout:osd.4 smithi174 running (11m) 9m ago 27m 92.2M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 0d746ce6e79d 2024-03-21T10:29:45.558 INFO:teuthology.orchestra.run.smithi153.stdout:osd.5 smithi174 running (10m) 9m ago 26m 86.2M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 53e5f35b8cac 2024-03-21T10:29:45.558 INFO:teuthology.orchestra.run.smithi153.stdout:prometheus.smithi153 smithi153 *:9095 running (19m) 9m ago 31m 73.5M - 2.43.0 a07b618ecd1d 7d95561e50c2 2024-03-21T10:29:45.559 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:45.559+0000 7f78b6600700 1 -- 172.21.15.153:0/472131 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f789806e460 msgr2=0x7f7898070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:45.559 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:45.559+0000 7f78b6600700 1 --2- 172.21.15.153:0/472131 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f789806e460 0x7f7898070920 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f789c005950 tx=0x7f789c0058e0 comp rx=0 tx=0).stop 2024-03-21T10:29:45.559 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:45.559+0000 7f78b6600700 1 -- 172.21.15.153:0/472131 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f78b00ff790 msgr2=0x7f78b019e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:45.559 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:45.559+0000 7f78b6600700 1 --2- 172.21.15.153:0/472131 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f78b00ff790 0x7f78b019e5a0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f78a4000c00 tx=0x7f78a40056a0 comp rx=0 tx=0).stop 2024-03-21T10:29:45.560 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:45.559+0000 7f78b6600700 1 -- 172.21.15.153:0/472131 shutdown_connections 2024-03-21T10:29:45.560 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:45.559+0000 7f78b6600700 1 --2- 172.21.15.153:0/472131 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f789806e460 0x7f7898070920 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:45.560 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:45.559+0000 7f78b6600700 1 --2- 172.21.15.153:0/472131 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f78b00ff790 0x7f78b019e5a0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:45.560 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:45.559+0000 7f78b6600700 1 --2- 172.21.15.153:0/472131 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f78b0100140 0x7f78b019eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:45.560 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:45.559+0000 7f78b6600700 1 -- 172.21.15.153:0/472131 >> 172.21.15.153:0/472131 conn(0x7f78b00fb3d0 msgr2=0x7f78b00fca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:45.560 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:45.560+0000 7f78b6600700 1 -- 172.21.15.153:0/472131 shutdown_connections 2024-03-21T10:29:45.560 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:45.560+0000 7f78b6600700 1 -- 172.21.15.153:0/472131 wait complete. 2024-03-21T10:29:45.699 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:29:45 smithi174 ceph-mon[88004]: pgmap v536: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:29:45.707 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:45.707+0000 7f2864e00700 1 -- 172.21.15.153:0/1446818336 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f286010d280 msgr2=0x7f286010d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:45.708 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:45.707+0000 7f2864e00700 1 --2- 172.21.15.153:0/1446818336 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f286010d280 0x7f286010d660 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f2850009910 tx=0x7f2850009c20 comp rx=0 tx=0).stop 2024-03-21T10:29:45.708 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:45.707+0000 7f2864e00700 1 -- 172.21.15.153:0/1446818336 shutdown_connections 2024-03-21T10:29:45.708 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:45.707+0000 7f2864e00700 1 --2- 172.21.15.153:0/1446818336 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f28601071d0 0x7f2860107650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:45.708 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:45.707+0000 7f2864e00700 1 --2- 172.21.15.153:0/1446818336 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f286010d280 0x7f286010d660 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:45.708 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:45.707+0000 7f2864e00700 1 -- 172.21.15.153:0/1446818336 >> 172.21.15.153:0/1446818336 conn(0x7f28600769e0 msgr2=0x7f2860078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:45.708 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:29:45 smithi153 ceph-mon[113733]: pgmap v536: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:29:45.709 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:45.708+0000 7f2864e00700 1 -- 172.21.15.153:0/1446818336 shutdown_connections 2024-03-21T10:29:45.709 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:45.708+0000 7f2864e00700 1 -- 172.21.15.153:0/1446818336 wait complete. 2024-03-21T10:29:45.709 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:45.709+0000 7f2864e00700 1 Processor -- start 2024-03-21T10:29:45.709 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:45.709+0000 7f2864e00700 1 -- start start 2024-03-21T10:29:45.710 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:45.710+0000 7f2864e00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f28601071d0 0x7f286019ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:45.710 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:45.710+0000 7f2864e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f286010d280 0x7f286019d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:45.710 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:45.710+0000 7f2864e00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f286019d8a0 con 0x7f286010d280 2024-03-21T10:29:45.710 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:45.710+0000 7f2864e00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28601a05d0 con 0x7f28601071d0 2024-03-21T10:29:45.710 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:45.710+0000 7f285d400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f286010d280 0x7f286019d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:45.710 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:45.710+0000 7f285d400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f286010d280 0x7f286019d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:50788/0 (socket says 172.21.15.153:50788) 2024-03-21T10:29:45.710 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:45.710+0000 7f285d400700 1 -- 172.21.15.153:0/901408841 learned_addr learned my addr 172.21.15.153:0/901408841 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:29:45.711 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:45.710+0000 7f285de00700 1 --2- 172.21.15.153:0/901408841 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f28601071d0 0x7f286019ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:45.711 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:45.711+0000 7f285d400700 1 -- 172.21.15.153:0/901408841 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f28601071d0 msgr2=0x7f286019ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:45.711 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:45.711+0000 7f285d400700 1 --2- 172.21.15.153:0/901408841 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f28601071d0 0x7f286019ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:45.711 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:45.711+0000 7f285d400700 1 -- 172.21.15.153:0/901408841 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28500095f0 con 0x7f286010d280 2024-03-21T10:29:45.711 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:45.711+0000 7f285d400700 1 --2- 172.21.15.153:0/901408841 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f286010d280 0x7f286019d210 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f284c00d8d0 tx=0x7f284c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:45.711 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:45.711+0000 7f2856a00700 1 -- 172.21.15.153:0/901408841 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f284c009940 con 0x7f286010d280 2024-03-21T10:29:45.712 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:45.711+0000 7f2856a00700 1 -- 172.21.15.153:0/901408841 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f284c010460 con 0x7f286010d280 2024-03-21T10:29:45.712 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:45.711+0000 7f2856a00700 1 -- 172.21.15.153:0/901408841 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f284c00f5f0 con 0x7f286010d280 2024-03-21T10:29:45.712 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:45.711+0000 7f2864e00700 1 -- 172.21.15.153:0/901408841 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28601a08b0 con 0x7f286010d280 2024-03-21T10:29:45.712 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:45.711+0000 7f2864e00700 1 -- 172.21.15.153:0/901408841 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28601a0e30 con 0x7f286010d280 2024-03-21T10:29:45.712 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:45.712+0000 7f2864e00700 1 -- 172.21.15.153:0/901408841 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f286004ea90 con 0x7f286010d280 2024-03-21T10:29:45.713 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:45.713+0000 7f2856a00700 1 -- 172.21.15.153:0/901408841 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f284c0105d0 con 0x7f286010d280 2024-03-21T10:29:45.714 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:45.714+0000 7f2856a00700 1 --2- 172.21.15.153:0/901408841 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f284406e460 0x7f2844070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:45.714 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:45.714+0000 7f2856a00700 1 -- 172.21.15.153:0/901408841 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f284c08e5b0 con 0x7f286010d280 2024-03-21T10:29:45.714 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:45.715+0000 7f285de00700 1 --2- 172.21.15.153:0/901408841 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f284406e460 0x7f2844070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:45.715 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:45.715+0000 7f285de00700 1 --2- 172.21.15.153:0/901408841 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f284406e460 0x7f2844070920 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f285000b5c0 tx=0x7f285001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:45.719 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:45.719+0000 7f2856a00700 1 -- 172.21.15.153:0/901408841 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f284c0575d0 con 0x7f286010d280 2024-03-21T10:29:46.007 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.007+0000 7f2864e00700 1 -- 172.21.15.153:0/901408841 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f28601a11a0 con 0x7f286010d280 2024-03-21T10:29:46.008 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.008+0000 7f2856a00700 1 -- 172.21.15.153:0/901408841 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+805 (secure 0 0 0) 0x7f284c05ac90 con 0x7f286010d280 2024-03-21T10:29:46.008 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:29:46.008 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": { 2024-03-21T10:29:46.008 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:29:46.008 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:29:46.008 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": { 2024-03-21T10:29:46.008 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:29:46.008 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:29:46.008 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": { 2024-03-21T10:29:46.009 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 6 2024-03-21T10:29:46.009 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:29:46.009 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": { 2024-03-21T10:29:46.009 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-03-21T10:29:46.009 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 3 2024-03-21T10:29:46.009 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:29:46.009 INFO:teuthology.orchestra.run.smithi153.stdout: "overall": { 2024-03-21T10:29:46.009 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-03-21T10:29:46.009 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 13 2024-03-21T10:29:46.009 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-03-21T10:29:46.009 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:29:46.010 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.011+0000 7f2864e00700 1 -- 172.21.15.153:0/901408841 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f284406e460 msgr2=0x7f2844070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:46.010 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.011+0000 7f2864e00700 1 --2- 172.21.15.153:0/901408841 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f284406e460 0x7f2844070920 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f285000b5c0 tx=0x7f285001a040 comp rx=0 tx=0).stop 2024-03-21T10:29:46.010 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.011+0000 7f2864e00700 1 -- 172.21.15.153:0/901408841 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f286010d280 msgr2=0x7f286019d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:46.011 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.011+0000 7f2864e00700 1 --2- 172.21.15.153:0/901408841 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f286010d280 0x7f286019d210 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f284c00d8d0 tx=0x7f284c00dc90 comp rx=0 tx=0).stop 2024-03-21T10:29:46.011 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.011+0000 7f2864e00700 1 -- 172.21.15.153:0/901408841 shutdown_connections 2024-03-21T10:29:46.011 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.011+0000 7f2864e00700 1 --2- 172.21.15.153:0/901408841 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f28601071d0 0x7f286019ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:46.011 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.011+0000 7f2864e00700 1 --2- 172.21.15.153:0/901408841 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f284406e460 0x7f2844070920 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:46.011 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.011+0000 7f2864e00700 1 --2- 172.21.15.153:0/901408841 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f286010d280 0x7f286019d210 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:46.012 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.011+0000 7f2864e00700 1 -- 172.21.15.153:0/901408841 >> 172.21.15.153:0/901408841 conn(0x7f28600769e0 msgr2=0x7f2860078320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:46.012 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.011+0000 7f2864e00700 1 -- 172.21.15.153:0/901408841 shutdown_connections 2024-03-21T10:29:46.012 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.011+0000 7f2864e00700 1 -- 172.21.15.153:0/901408841 wait complete. 2024-03-21T10:29:46.163 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.163+0000 7fc548600700 1 -- 172.21.15.153:0/2068420751 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc5401039b0 msgr2=0x7fc540107a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:46.163 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.163+0000 7fc548600700 1 --2- 172.21.15.153:0/2068420751 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc5401039b0 0x7fc540107a00 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fc52c009910 tx=0x7fc52c009c20 comp rx=0 tx=0).stop 2024-03-21T10:29:46.164 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.163+0000 7fc548600700 1 -- 172.21.15.153:0/2068420751 shutdown_connections 2024-03-21T10:29:46.164 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.163+0000 7fc548600700 1 --2- 172.21.15.153:0/2068420751 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc5401039b0 0x7fc540107a00 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:46.164 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.163+0000 7fc548600700 1 --2- 172.21.15.153:0/2068420751 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc540103000 0x7fc5401033e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:46.164 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.163+0000 7fc548600700 1 -- 172.21.15.153:0/2068420751 >> 172.21.15.153:0/2068420751 conn(0x7fc5400fe8d0 msgr2=0x7fc540100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:46.164 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.163+0000 7fc548600700 1 -- 172.21.15.153:0/2068420751 shutdown_connections 2024-03-21T10:29:46.164 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.163+0000 7fc548600700 1 -- 172.21.15.153:0/2068420751 wait complete. 2024-03-21T10:29:46.165 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.164+0000 7fc548600700 1 Processor -- start 2024-03-21T10:29:46.165 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.165+0000 7fc548600700 1 -- start start 2024-03-21T10:29:46.165 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.165+0000 7fc548600700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc540103000 0x7fc54019e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:46.165 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.165+0000 7fc548600700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc5401039b0 0x7fc54019ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:46.165 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.165+0000 7fc548600700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc54019f390 con 0x7fc5401039b0 2024-03-21T10:29:46.165 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.165+0000 7fc548600700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc540198840 con 0x7fc540103000 2024-03-21T10:29:46.165 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.165+0000 7fc545200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc5401039b0 0x7fc54019ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:46.165 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.165+0000 7fc545200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc5401039b0 0x7fc54019ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:50820/0 (socket says 172.21.15.153:50820) 2024-03-21T10:29:46.165 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.165+0000 7fc545200700 1 -- 172.21.15.153:0/3136648262 learned_addr learned my addr 172.21.15.153:0/3136648262 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:29:46.166 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.166+0000 7fc545c00700 1 --2- 172.21.15.153:0/3136648262 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc540103000 0x7fc54019e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:46.166 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.166+0000 7fc545200700 1 -- 172.21.15.153:0/3136648262 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc540103000 msgr2=0x7fc54019e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:46.166 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.166+0000 7fc545200700 1 --2- 172.21.15.153:0/3136648262 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc540103000 0x7fc54019e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:46.167 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.166+0000 7fc545200700 1 -- 172.21.15.153:0/3136648262 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc52c0095f0 con 0x7fc5401039b0 2024-03-21T10:29:46.167 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.166+0000 7fc545c00700 1 --2- 172.21.15.153:0/3136648262 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc540103000 0x7fc54019e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:29:46.167 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.166+0000 7fc545200700 1 --2- 172.21.15.153:0/3136648262 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc5401039b0 0x7fc54019ed00 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fc52c00bda0 tx=0x7fc52c00be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:46.167 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.166+0000 7fc53e000700 1 -- 172.21.15.153:0/3136648262 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc52c01d070 con 0x7fc5401039b0 2024-03-21T10:29:46.167 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.166+0000 7fc53e000700 1 -- 172.21.15.153:0/3136648262 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc52c00f460 con 0x7fc5401039b0 2024-03-21T10:29:46.167 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.166+0000 7fc53e000700 1 -- 172.21.15.153:0/3136648262 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc52c017610 con 0x7fc5401039b0 2024-03-21T10:29:46.167 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.166+0000 7fc548600700 1 -- 172.21.15.153:0/3136648262 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc540198ac0 con 0x7fc5401039b0 2024-03-21T10:29:46.167 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.167+0000 7fc548600700 1 -- 172.21.15.153:0/3136648262 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc540198fb0 con 0x7fc5401039b0 2024-03-21T10:29:46.167 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.168+0000 7fc548600700 1 -- 172.21.15.153:0/3136648262 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc54010b400 con 0x7fc5401039b0 2024-03-21T10:29:46.168 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.169+0000 7fc53e000700 1 -- 172.21.15.153:0/3136648262 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fc52c00fa40 con 0x7fc5401039b0 2024-03-21T10:29:46.169 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.169+0000 7fc53e000700 1 --2- 172.21.15.153:0/3136648262 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fc52806e460 0x7fc528070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:46.169 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.170+0000 7fc53e000700 1 -- 172.21.15.153:0/3136648262 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fc52c08f880 con 0x7fc5401039b0 2024-03-21T10:29:46.169 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.170+0000 7fc545c00700 1 --2- 172.21.15.153:0/3136648262 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fc52806e460 0x7fc528070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:46.170 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.170+0000 7fc545c00700 1 --2- 172.21.15.153:0/3136648262 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fc52806e460 0x7fc528070920 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc534005fd0 tx=0x7fc534005ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:46.174 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.174+0000 7fc53e000700 1 -- 172.21.15.153:0/3136648262 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc52c05c4f0 con 0x7fc5401039b0 2024-03-21T10:29:46.439 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.439+0000 7fc548600700 1 -- 172.21.15.153:0/3136648262 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fc54004ea90 con 0x7fc5401039b0 2024-03-21T10:29:46.439 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.440+0000 7fc53e000700 1 -- 172.21.15.153:0/3136648262 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1876 (secure 0 0 0) 0x7fc52c05c080 con 0x7fc5401039b0 2024-03-21T10:29:46.440 INFO:teuthology.orchestra.run.smithi153.stdout:e30 2024-03-21T10:29:46.440 INFO:teuthology.orchestra.run.smithi153.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-21T10:29:46.440 INFO:teuthology.orchestra.run.smithi153.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:29:46.440 INFO:teuthology.orchestra.run.smithi153.stdout:legacy client fscid: 1 2024-03-21T10:29:46.440 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:29:46.441 INFO:teuthology.orchestra.run.smithi153.stdout:Filesystem 'cephfs' (1) 2024-03-21T10:29:46.441 INFO:teuthology.orchestra.run.smithi153.stdout:fs_name cephfs 2024-03-21T10:29:46.441 INFO:teuthology.orchestra.run.smithi153.stdout:epoch 23 2024-03-21T10:29:46.441 INFO:teuthology.orchestra.run.smithi153.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-21T10:29:46.441 INFO:teuthology.orchestra.run.smithi153.stdout:created 2024-03-21T10:04:27.086576+0000 2024-03-21T10:29:46.441 INFO:teuthology.orchestra.run.smithi153.stdout:modified 2024-03-21T10:19:05.022092+0000 2024-03-21T10:29:46.441 INFO:teuthology.orchestra.run.smithi153.stdout:tableserver 0 2024-03-21T10:29:46.441 INFO:teuthology.orchestra.run.smithi153.stdout:root 0 2024-03-21T10:29:46.441 INFO:teuthology.orchestra.run.smithi153.stdout:session_timeout 60 2024-03-21T10:29:46.441 INFO:teuthology.orchestra.run.smithi153.stdout:session_autoclose 300 2024-03-21T10:29:46.441 INFO:teuthology.orchestra.run.smithi153.stdout:max_file_size 1099511627776 2024-03-21T10:29:46.441 INFO:teuthology.orchestra.run.smithi153.stdout:max_xattr_size 65536 2024-03-21T10:29:46.441 INFO:teuthology.orchestra.run.smithi153.stdout:required_client_features {} 2024-03-21T10:29:46.442 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure 0 2024-03-21T10:29:46.442 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure_osd_epoch 94 2024-03-21T10:29:46.442 INFO:teuthology.orchestra.run.smithi153.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:29:46.442 INFO:teuthology.orchestra.run.smithi153.stdout:max_mds 2 2024-03-21T10:29:46.442 INFO:teuthology.orchestra.run.smithi153.stdout:in 0,1 2024-03-21T10:29:46.442 INFO:teuthology.orchestra.run.smithi153.stdout:up {} 2024-03-21T10:29:46.442 INFO:teuthology.orchestra.run.smithi153.stdout:failed 0,1 2024-03-21T10:29:46.442 INFO:teuthology.orchestra.run.smithi153.stdout:damaged 2024-03-21T10:29:46.442 INFO:teuthology.orchestra.run.smithi153.stdout:stopped 2024-03-21T10:29:46.442 INFO:teuthology.orchestra.run.smithi153.stdout:data_pools [3] 2024-03-21T10:29:46.442 INFO:teuthology.orchestra.run.smithi153.stdout:metadata_pool 2 2024-03-21T10:29:46.442 INFO:teuthology.orchestra.run.smithi153.stdout:inline_data enabled 2024-03-21T10:29:46.442 INFO:teuthology.orchestra.run.smithi153.stdout:balancer 2024-03-21T10:29:46.442 INFO:teuthology.orchestra.run.smithi153.stdout:bal_rank_mask -1 2024-03-21T10:29:46.443 INFO:teuthology.orchestra.run.smithi153.stdout:standby_count_wanted 1 2024-03-21T10:29:46.443 INFO:teuthology.orchestra.run.smithi153.stdout:qdb_cluster leader: 0 members: 2024-03-21T10:29:46.443 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:29:46.443 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:29:46.443 INFO:teuthology.orchestra.run.smithi153.stdout:Standby daemons: 2024-03-21T10:29:46.443 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:29:46.443 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.zipckr{-1:34442} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6826/1103846295,v1:172.21.15.153:6827/1103846295] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:29:46.443 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.iuyluj{-1:34474} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6828/1738630832,v1:172.21.15.153:6829/1738630832] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:29:46.443 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.nhozag{-1:44269} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.174:6826/3967223227,v1:172.21.15.174:6827/3967223227] compat {c=[1],r=[1],i=[7ff]}] 2024-03-21T10:29:46.443 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.yyimst{-1:44313} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.174:6824/4127786797,v1:172.21.15.174:6825/4127786797] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:29:46.444 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.444+0000 7fc548600700 1 -- 172.21.15.153:0/3136648262 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fc52806e460 msgr2=0x7fc528070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:46.444 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.444+0000 7fc548600700 1 --2- 172.21.15.153:0/3136648262 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fc52806e460 0x7fc528070920 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc534005fd0 tx=0x7fc534005ee0 comp rx=0 tx=0).stop 2024-03-21T10:29:46.445 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.444+0000 7fc548600700 1 -- 172.21.15.153:0/3136648262 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc5401039b0 msgr2=0x7fc54019ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:46.445 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.444+0000 7fc548600700 1 --2- 172.21.15.153:0/3136648262 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc5401039b0 0x7fc54019ed00 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fc52c00bda0 tx=0x7fc52c00be80 comp rx=0 tx=0).stop 2024-03-21T10:29:46.445 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.444+0000 7fc548600700 1 -- 172.21.15.153:0/3136648262 shutdown_connections 2024-03-21T10:29:46.445 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.444+0000 7fc548600700 1 --2- 172.21.15.153:0/3136648262 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc540103000 0x7fc54019e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:46.445 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.444+0000 7fc548600700 1 --2- 172.21.15.153:0/3136648262 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fc52806e460 0x7fc528070920 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:46.445 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.444+0000 7fc548600700 1 --2- 172.21.15.153:0/3136648262 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc5401039b0 0x7fc54019ed00 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:46.445 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.444+0000 7fc548600700 1 -- 172.21.15.153:0/3136648262 >> 172.21.15.153:0/3136648262 conn(0x7fc5400fe8d0 msgr2=0x7fc540100cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:46.445 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.445+0000 7fc548600700 1 -- 172.21.15.153:0/3136648262 shutdown_connections 2024-03-21T10:29:46.445 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.445+0000 7fc548600700 1 -- 172.21.15.153:0/3136648262 wait complete. 2024-03-21T10:29:46.446 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 30 2024-03-21T10:29:46.586 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.586+0000 7f20af400700 1 -- 172.21.15.153:0/3636163725 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f20a810d280 msgr2=0x7f20a810d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:46.586 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.586+0000 7f20af400700 1 --2- 172.21.15.153:0/3636163725 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f20a810d280 0x7f20a810d660 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f209c009910 tx=0x7f209c009c20 comp rx=0 tx=0).stop 2024-03-21T10:29:46.586 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.586+0000 7f20af400700 1 -- 172.21.15.153:0/3636163725 shutdown_connections 2024-03-21T10:29:46.587 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.586+0000 7f20af400700 1 --2- 172.21.15.153:0/3636163725 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f20a81071d0 0x7f20a8107650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:46.587 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.586+0000 7f20af400700 1 --2- 172.21.15.153:0/3636163725 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f20a810d280 0x7f20a810d660 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:46.587 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.586+0000 7f20af400700 1 -- 172.21.15.153:0/3636163725 >> 172.21.15.153:0/3636163725 conn(0x7f20a80769e0 msgr2=0x7f20a8078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:46.587 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.586+0000 7f20af400700 1 -- 172.21.15.153:0/3636163725 shutdown_connections 2024-03-21T10:29:46.587 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.586+0000 7f20af400700 1 -- 172.21.15.153:0/3636163725 wait complete. 2024-03-21T10:29:46.588 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.587+0000 7f20af400700 1 Processor -- start 2024-03-21T10:29:46.588 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.587+0000 7f20af400700 1 -- start start 2024-03-21T10:29:46.588 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.588+0000 7f20af400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f20a81071d0 0x7f20a819ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:46.588 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.588+0000 7f20af400700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f20a810d280 0x7f20a819d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:46.589 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.588+0000 7f20af400700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20a819d8a0 con 0x7f20a81071d0 2024-03-21T10:29:46.589 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.588+0000 7f20af400700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20a81a05d0 con 0x7f20a810d280 2024-03-21T10:29:46.589 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.588+0000 7f20aca00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f20a81071d0 0x7f20a819ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:46.589 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.588+0000 7f20aca00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f20a81071d0 0x7f20a819ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:50832/0 (socket says 172.21.15.153:50832) 2024-03-21T10:29:46.589 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.588+0000 7f20aca00700 1 -- 172.21.15.153:0/4112975215 learned_addr learned my addr 172.21.15.153:0/4112975215 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:29:46.589 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.588+0000 7f20a7e00700 1 --2- 172.21.15.153:0/4112975215 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f20a810d280 0x7f20a819d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:46.589 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.588+0000 7f20aca00700 1 -- 172.21.15.153:0/4112975215 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f20a810d280 msgr2=0x7f20a819d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:46.589 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.588+0000 7f20aca00700 1 --2- 172.21.15.153:0/4112975215 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f20a810d280 0x7f20a819d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:46.589 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.588+0000 7f20aca00700 1 -- 172.21.15.153:0/4112975215 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f209c0095f0 con 0x7f20a81071d0 2024-03-21T10:29:46.589 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.588+0000 7f20a7e00700 1 --2- 172.21.15.153:0/4112975215 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f20a810d280 0x7f20a819d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:29:46.589 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.589+0000 7f20aca00700 1 --2- 172.21.15.153:0/4112975215 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f20a81071d0 0x7f20a819ccd0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f209c00b5c0 tx=0x7f209c00be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:46.590 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.589+0000 7f20a5600700 1 -- 172.21.15.153:0/4112975215 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f209c01d070 con 0x7f20a81071d0 2024-03-21T10:29:46.590 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.589+0000 7f20af400700 1 -- 172.21.15.153:0/4112975215 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20a81a0850 con 0x7f20a81071d0 2024-03-21T10:29:46.590 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.589+0000 7f20a5600700 1 -- 172.21.15.153:0/4112975215 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f209c00f460 con 0x7f20a81071d0 2024-03-21T10:29:46.590 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.589+0000 7f20a5600700 1 -- 172.21.15.153:0/4112975215 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f209c017610 con 0x7f20a81071d0 2024-03-21T10:29:46.590 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.589+0000 7f20af400700 1 -- 172.21.15.153:0/4112975215 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20a81a0dc0 con 0x7f20a81071d0 2024-03-21T10:29:46.591 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.590+0000 7f20af400700 1 -- 172.21.15.153:0/4112975215 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20a804ea90 con 0x7f20a81071d0 2024-03-21T10:29:46.591 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.591+0000 7f20a5600700 1 -- 172.21.15.153:0/4112975215 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f209c00f5d0 con 0x7f20a81071d0 2024-03-21T10:29:46.592 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.593+0000 7f20a5600700 1 --2- 172.21.15.153:0/4112975215 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f209006e460 0x7f2090070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:46.592 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.593+0000 7f20a5600700 1 -- 172.21.15.153:0/4112975215 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f209c08fb10 con 0x7f20a81071d0 2024-03-21T10:29:46.592 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.593+0000 7f20a7e00700 1 --2- 172.21.15.153:0/4112975215 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f209006e460 0x7f2090070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:46.593 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.594+0000 7f20a7e00700 1 --2- 172.21.15.153:0/4112975215 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f209006e460 0x7f2090070920 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f2094005fd0 tx=0x7f2094005dc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:46.597 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.597+0000 7f20a5600700 1 -- 172.21.15.153:0/4112975215 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f209c058bb0 con 0x7f20a81071d0 2024-03-21T10:29:46.699 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:29:46 smithi174 ceph-mon[88004]: from='client.34982 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:29:46.700 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:29:46 smithi174 ceph-mon[88004]: from='client.34986 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:29:46.700 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:29:46 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/901408841' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:29:46.700 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:29:46 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:29:46.804 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:29:46 smithi153 ceph-mon[113733]: from='client.34982 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:29:46.805 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:29:46 smithi153 ceph-mon[113733]: from='client.34986 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:29:46.805 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:29:46 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/901408841' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:29:46.805 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:29:46 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:29:46.825 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.825+0000 7f20af400700 1 -- 172.21.15.153:0/4112975215 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f20a81a1190 con 0x7f209006e460 2024-03-21T10:29:46.827 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.827+0000 7f20a5600700 1 -- 172.21.15.153:0/4112975215 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f20a81a1190 con 0x7f209006e460 2024-03-21T10:29:46.827 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:29:46.827 INFO:teuthology.orchestra.run.smithi153.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232", 2024-03-21T10:29:46.827 INFO:teuthology.orchestra.run.smithi153.stdout: "in_progress": true, 2024-03-21T10:29:46.827 INFO:teuthology.orchestra.run.smithi153.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-21T10:29:46.827 INFO:teuthology.orchestra.run.smithi153.stdout: "services_complete": [ 2024-03-21T10:29:46.828 INFO:teuthology.orchestra.run.smithi153.stdout: "crash", 2024-03-21T10:29:46.828 INFO:teuthology.orchestra.run.smithi153.stdout: "mon", 2024-03-21T10:29:46.828 INFO:teuthology.orchestra.run.smithi153.stdout: "osd", 2024-03-21T10:29:46.828 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr" 2024-03-21T10:29:46.828 INFO:teuthology.orchestra.run.smithi153.stdout: ], 2024-03-21T10:29:46.828 INFO:teuthology.orchestra.run.smithi153.stdout: "progress": "15/23 daemons upgraded", 2024-03-21T10:29:46.828 INFO:teuthology.orchestra.run.smithi153.stdout: "message": "Currently upgrading mds daemons", 2024-03-21T10:29:46.828 INFO:teuthology.orchestra.run.smithi153.stdout: "is_paused": false 2024-03-21T10:29:46.828 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:29:46.830 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.830+0000 7f20af400700 1 -- 172.21.15.153:0/4112975215 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f209006e460 msgr2=0x7f2090070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:46.830 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.830+0000 7f20af400700 1 --2- 172.21.15.153:0/4112975215 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f209006e460 0x7f2090070920 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f2094005fd0 tx=0x7f2094005dc0 comp rx=0 tx=0).stop 2024-03-21T10:29:46.830 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.830+0000 7f20af400700 1 -- 172.21.15.153:0/4112975215 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f20a81071d0 msgr2=0x7f20a819ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:46.830 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.830+0000 7f20af400700 1 --2- 172.21.15.153:0/4112975215 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f20a81071d0 0x7f20a819ccd0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f209c00b5c0 tx=0x7f209c00be00 comp rx=0 tx=0).stop 2024-03-21T10:29:46.830 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.830+0000 7f20af400700 1 -- 172.21.15.153:0/4112975215 shutdown_connections 2024-03-21T10:29:46.830 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.830+0000 7f20af400700 1 --2- 172.21.15.153:0/4112975215 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f209006e460 0x7f2090070920 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:46.831 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.830+0000 7f20af400700 1 --2- 172.21.15.153:0/4112975215 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f20a81071d0 0x7f20a819ccd0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:46.831 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.830+0000 7f20af400700 1 --2- 172.21.15.153:0/4112975215 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f20a810d280 0x7f20a819d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:46.831 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.830+0000 7f20af400700 1 -- 172.21.15.153:0/4112975215 >> 172.21.15.153:0/4112975215 conn(0x7f20a80769e0 msgr2=0x7f20a8078320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:46.831 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.830+0000 7f20af400700 1 -- 172.21.15.153:0/4112975215 shutdown_connections 2024-03-21T10:29:46.831 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.831+0000 7f20af400700 1 -- 172.21.15.153:0/4112975215 wait complete. 2024-03-21T10:29:46.983 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.983+0000 7f38b7e00700 1 -- 172.21.15.153:0/2647438724 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f38b8106a40 msgr2=0x7f38b8106e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:46.983 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.983+0000 7f38b7e00700 1 --2- 172.21.15.153:0/2647438724 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f38b8106a40 0x7f38b8106e20 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f38a800b0a0 tx=0x7f38a800b3b0 comp rx=0 tx=0).stop 2024-03-21T10:29:46.983 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.983+0000 7f38b7e00700 1 -- 172.21.15.153:0/2647438724 shutdown_connections 2024-03-21T10:29:46.983 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.983+0000 7f38b7e00700 1 --2- 172.21.15.153:0/2647438724 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f38b8068410 0x7f38b8068890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:46.984 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.983+0000 7f38b7e00700 1 --2- 172.21.15.153:0/2647438724 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f38b8106a40 0x7f38b8106e20 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:46.984 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.983+0000 7f38b7e00700 1 -- 172.21.15.153:0/2647438724 >> 172.21.15.153:0/2647438724 conn(0x7f38b8075330 msgr2=0x7f38b8075740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:46.984 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.983+0000 7f38b7e00700 1 -- 172.21.15.153:0/2647438724 shutdown_connections 2024-03-21T10:29:46.984 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.983+0000 7f38b7e00700 1 -- 172.21.15.153:0/2647438724 wait complete. 2024-03-21T10:29:46.985 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.984+0000 7f38b7e00700 1 Processor -- start 2024-03-21T10:29:46.985 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.985+0000 7f38b7e00700 1 -- start start 2024-03-21T10:29:46.985 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.985+0000 7f38b7e00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f38b8068410 0x7f38b819e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:46.985 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.985+0000 7f38b7e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f38b8106a40 0x7f38b819ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:46.985 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.985+0000 7f38b7e00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38b819f120 con 0x7f38b8106a40 2024-03-21T10:29:46.985 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.985+0000 7f38b7e00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38b81985d0 con 0x7f38b8068410 2024-03-21T10:29:46.986 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.985+0000 7f38ae000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f38b8106a40 0x7f38b819ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:46.986 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.985+0000 7f38ae000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f38b8106a40 0x7f38b819ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:50852/0 (socket says 172.21.15.153:50852) 2024-03-21T10:29:46.986 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.985+0000 7f38ae000700 1 -- 172.21.15.153:0/1134766883 learned_addr learned my addr 172.21.15.153:0/1134766883 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:29:46.986 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.986+0000 7f38b6a00700 1 --2- 172.21.15.153:0/1134766883 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f38b8068410 0x7f38b819e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:46.987 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.986+0000 7f38ae000700 1 -- 172.21.15.153:0/1134766883 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f38b8068410 msgr2=0x7f38b819e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:46.987 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.986+0000 7f38ae000700 1 --2- 172.21.15.153:0/1134766883 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f38b8068410 0x7f38b819e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:46.987 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.986+0000 7f38ae000700 1 -- 172.21.15.153:0/1134766883 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38a8009d00 con 0x7f38b8106a40 2024-03-21T10:29:46.987 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.986+0000 7f38ae000700 1 --2- 172.21.15.153:0/1134766883 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f38b8106a40 0x7f38b819ea90 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f38a000d8d0 tx=0x7f38a000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:46.987 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.986+0000 7f38afe00700 1 -- 172.21.15.153:0/1134766883 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f38a0009940 con 0x7f38b8106a40 2024-03-21T10:29:46.987 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.986+0000 7f38afe00700 1 -- 172.21.15.153:0/1134766883 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f38a0010460 con 0x7f38b8106a40 2024-03-21T10:29:46.987 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.986+0000 7f38afe00700 1 -- 172.21.15.153:0/1134766883 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f38a000f5f0 con 0x7f38b8106a40 2024-03-21T10:29:46.987 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.986+0000 7f38b7e00700 1 -- 172.21.15.153:0/1134766883 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38b81988b0 con 0x7f38b8106a40 2024-03-21T10:29:46.987 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.987+0000 7f38b7e00700 1 -- 172.21.15.153:0/1134766883 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38b8198e00 con 0x7f38b8106a40 2024-03-21T10:29:46.987 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.988+0000 7f38b7e00700 1 -- 172.21.15.153:0/1134766883 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38b804f2e0 con 0x7f38b8106a40 2024-03-21T10:29:46.988 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.989+0000 7f38afe00700 1 -- 172.21.15.153:0/1134766883 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f38a00105d0 con 0x7f38b8106a40 2024-03-21T10:29:46.989 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.990+0000 7f38afe00700 1 --2- 172.21.15.153:0/1134766883 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f389c06e460 0x7f389c070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:29:46.989 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.990+0000 7f38afe00700 1 -- 172.21.15.153:0/1134766883 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f38a008e480 con 0x7f38b8106a40 2024-03-21T10:29:46.989 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.990+0000 7f38b6a00700 1 --2- 172.21.15.153:0/1134766883 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f389c06e460 0x7f389c070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:29:46.990 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.990+0000 7f38b6a00700 1 --2- 172.21.15.153:0/1134766883 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f389c06e460 0x7f389c070920 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f38a8009480 tx=0x7f38a800b7f0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:29:46.994 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:46.994+0000 7f38afe00700 1 -- 172.21.15.153:0/1134766883 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f38a005b050 con 0x7f38b8106a40 2024-03-21T10:29:47.300 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:47.300+0000 7f38b7e00700 1 -- 172.21.15.153:0/1134766883 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f38b8199ae0 con 0x7f38b8106a40 2024-03-21T10:29:47.300 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:47.301+0000 7f38afe00700 1 -- 172.21.15.153:0/1134766883 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f38a005abe0 con 0x7f38b8106a40 2024-03-21T10:29:47.300 INFO:teuthology.orchestra.run.smithi153.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-03-21T10:29:47.301 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-21T10:29:47.301 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is degraded 2024-03-21T10:29:47.301 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-21T10:29:47.301 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-21T10:29:47.301 INFO:teuthology.orchestra.run.smithi153.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-21T10:29:47.301 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-21T10:29:47.303 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:47.303+0000 7f38b7e00700 1 -- 172.21.15.153:0/1134766883 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f389c06e460 msgr2=0x7f389c070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:47.303 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:47.303+0000 7f38b7e00700 1 --2- 172.21.15.153:0/1134766883 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f389c06e460 0x7f389c070920 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f38a8009480 tx=0x7f38a800b7f0 comp rx=0 tx=0).stop 2024-03-21T10:29:47.303 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:47.303+0000 7f38b7e00700 1 -- 172.21.15.153:0/1134766883 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f38b8106a40 msgr2=0x7f38b819ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:29:47.303 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:47.303+0000 7f38b7e00700 1 --2- 172.21.15.153:0/1134766883 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f38b8106a40 0x7f38b819ea90 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f38a000d8d0 tx=0x7f38a000dc90 comp rx=0 tx=0).stop 2024-03-21T10:29:47.304 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:47.304+0000 7f38b7e00700 1 -- 172.21.15.153:0/1134766883 shutdown_connections 2024-03-21T10:29:47.304 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:47.304+0000 7f38b7e00700 1 --2- 172.21.15.153:0/1134766883 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f38b8068410 0x7f38b819e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:47.304 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:47.304+0000 7f38b7e00700 1 --2- 172.21.15.153:0/1134766883 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f389c06e460 0x7f389c070920 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:47.304 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:47.304+0000 7f38b7e00700 1 --2- 172.21.15.153:0/1134766883 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f38b8106a40 0x7f38b819ea90 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:29:47.304 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:47.304+0000 7f38b7e00700 1 -- 172.21.15.153:0/1134766883 >> 172.21.15.153:0/1134766883 conn(0x7f38b8075330 msgr2=0x7f38b80ffe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:29:47.305 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:47.304+0000 7f38b7e00700 1 -- 172.21.15.153:0/1134766883 shutdown_connections 2024-03-21T10:29:47.305 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:29:47.304+0000 7f38b7e00700 1 -- 172.21.15.153:0/1134766883 wait complete. 2024-03-21T10:29:47.555 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:29:47 smithi153 ceph-mon[113733]: pgmap v537: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:29:47.555 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:29:47 smithi153 ceph-mon[113733]: from='client.34990 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:29:47.555 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:29:47 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/3136648262' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:29:47.555 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:29:47 smithi153 ceph-mon[113733]: from='client.35002 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:29:47.555 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:29:47 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/1134766883' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:29:47.700 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:29:47 smithi174 ceph-mon[88004]: pgmap v537: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:29:47.700 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:29:47 smithi174 ceph-mon[88004]: from='client.34990 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:29:47.700 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:29:47 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/3136648262' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:29:47.700 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:29:47 smithi174 ceph-mon[88004]: from='client.35002 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:29:47.700 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:29:47 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/1134766883' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:29:48.699 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:29:48 smithi174 ceph-mon[88004]: pgmap v538: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:29:48.804 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:29:48 smithi153 ceph-mon[113733]: pgmap v538: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:29:50.804 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:29:50 smithi153 ceph-mon[113733]: pgmap v539: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:29:50.949 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:29:50 smithi174 ceph-mon[88004]: pgmap v539: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:29:52.606 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:29:52 smithi153 ceph-mon[113733]: pgmap v540: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:29:52.949 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:29:52 smithi174 ceph-mon[88004]: pgmap v540: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:29:54.804 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:29:54 smithi153 ceph-mon[113733]: pgmap v541: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:29:54.949 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:29:54 smithi174 ceph-mon[88004]: pgmap v541: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:29:56.949 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:29:56 smithi174 ceph-mon[88004]: pgmap v542: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:29:57.054 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:29:56 smithi153 ceph-mon[113733]: pgmap v542: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:29:58.949 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:29:58 smithi174 ceph-mon[88004]: pgmap v543: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:29:59.054 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:29:58 smithi153 ceph-mon[113733]: pgmap v543: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:30:00.304 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:30:00 smithi153 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-mon-smithi153[113729]: 2024-03-21T10:29:59.999+0000 7f2169e00700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-03-21T10:30:00.949 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:30:00 smithi174 ceph-mon[88004]: pgmap v544: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:30:00.949 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:30:00 smithi174 ceph-mon[88004]: overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-03-21T10:30:01.054 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:30:00 smithi153 ceph-mon[113733]: pgmap v544: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:30:01.055 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:30:00 smithi153 ceph-mon[113733]: overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-03-21T10:30:01.949 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:30:01 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:30:02.054 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:30:01 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:30:02.949 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:30:02 smithi174 ceph-mon[88004]: pgmap v545: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:30:03.054 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:30:02 smithi153 ceph-mon[113733]: pgmap v545: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:30:04.949 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:30:04 smithi174 ceph-mon[88004]: pgmap v546: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:30:05.054 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:30:04 smithi153 ceph-mon[113733]: pgmap v546: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:30:06.949 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:30:06 smithi174 ceph-mon[88004]: pgmap v547: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:30:07.054 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:30:06 smithi153 ceph-mon[113733]: pgmap v547: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:30:08.949 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:30:08 smithi174 ceph-mon[88004]: pgmap v548: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:30:09.054 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:30:08 smithi153 ceph-mon[113733]: pgmap v548: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:30:10.796 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:30:10 smithi174 ceph-mon[88004]: pgmap v549: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:30:11.054 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:30:10 smithi153 ceph-mon[113733]: pgmap v549: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:30:12.898 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:30:12 smithi174 ceph-mon[88004]: pgmap v550: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:30:13.054 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:30:12 smithi153 ceph-mon[113733]: pgmap v550: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:30:14.949 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:30:14 smithi174 ceph-mon[88004]: pgmap v551: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:30:15.054 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:30:14 smithi153 ceph-mon[113733]: pgmap v551: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:30:16.950 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:30:16 smithi174 ceph-mon[88004]: pgmap v552: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:30:16.950 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:30:16 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:30:17.054 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:30:16 smithi153 ceph-mon[113733]: pgmap v552: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:30:17.055 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:30:16 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:30:17.473 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:17.473+0000 7f06fac00700 1 -- 172.21.15.153:0/2232759284 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f06f41071d0 msgr2=0x7f06f4107650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:17.474 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:17.473+0000 7f06fac00700 1 --2- 172.21.15.153:0/2232759284 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f06f41071d0 0x7f06f4107650 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f06dc009910 tx=0x7f06dc009c20 comp rx=0 tx=0).stop 2024-03-21T10:30:17.474 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:17.474+0000 7f06fac00700 1 -- 172.21.15.153:0/2232759284 shutdown_connections 2024-03-21T10:30:17.474 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:17.474+0000 7f06fac00700 1 --2- 172.21.15.153:0/2232759284 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f06f41071d0 0x7f06f4107650 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:17.474 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:17.474+0000 7f06fac00700 1 --2- 172.21.15.153:0/2232759284 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f06f410d280 0x7f06f410d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:17.474 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:17.474+0000 7f06fac00700 1 -- 172.21.15.153:0/2232759284 >> 172.21.15.153:0/2232759284 conn(0x7f06f40769e0 msgr2=0x7f06f4078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:17.474 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:17.474+0000 7f06fac00700 1 -- 172.21.15.153:0/2232759284 shutdown_connections 2024-03-21T10:30:17.474 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:17.474+0000 7f06fac00700 1 -- 172.21.15.153:0/2232759284 wait complete. 2024-03-21T10:30:17.474 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:17.475+0000 7f06fac00700 1 Processor -- start 2024-03-21T10:30:17.474 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:17.475+0000 7f06fac00700 1 -- start start 2024-03-21T10:30:17.476 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:17.475+0000 7f06fac00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f06f41071d0 0x7f06f419ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:17.476 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:17.475+0000 7f06fac00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f06f410d280 0x7f06f419d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:17.476 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:17.475+0000 7f06fac00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06f419d880 con 0x7f06f41071d0 2024-03-21T10:30:17.476 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:17.475+0000 7f06fac00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06f41a05b0 con 0x7f06f410d280 2024-03-21T10:30:17.476 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:17.476+0000 7f06f3e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f06f41071d0 0x7f06f419ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:17.477 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:17.476+0000 7f06f3e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f06f41071d0 0x7f06f419ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41972/0 (socket says 172.21.15.153:41972) 2024-03-21T10:30:17.477 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:17.476+0000 7f06f3e00700 1 -- 172.21.15.153:0/1796782096 learned_addr learned my addr 172.21.15.153:0/1796782096 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:30:17.477 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:17.476+0000 7f06ebe00700 1 --2- 172.21.15.153:0/1796782096 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f06f410d280 0x7f06f419d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:17.477 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:17.476+0000 7f06f3e00700 1 -- 172.21.15.153:0/1796782096 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f06f410d280 msgr2=0x7f06f419d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:17.477 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:17.476+0000 7f06f3e00700 1 --2- 172.21.15.153:0/1796782096 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f06f410d280 0x7f06f419d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:17.477 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:17.476+0000 7f06f3e00700 1 -- 172.21.15.153:0/1796782096 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06dc0095f0 con 0x7f06f41071d0 2024-03-21T10:30:17.477 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:17.476+0000 7f06f3e00700 1 --2- 172.21.15.153:0/1796782096 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f06f41071d0 0x7f06f419ccb0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f06e400c8a0 tx=0x7f06e400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:17.477 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:17.477+0000 7f06f1600700 1 -- 172.21.15.153:0/1796782096 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f06e400cea0 con 0x7f06f41071d0 2024-03-21T10:30:17.477 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:17.477+0000 7f06f1600700 1 -- 172.21.15.153:0/1796782096 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f06e4004830 con 0x7f06f41071d0 2024-03-21T10:30:17.477 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:17.477+0000 7f06fac00700 1 -- 172.21.15.153:0/1796782096 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06f41a0890 con 0x7f06f41071d0 2024-03-21T10:30:17.477 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:17.477+0000 7f06f1600700 1 -- 172.21.15.153:0/1796782096 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f06e40055b0 con 0x7f06f41071d0 2024-03-21T10:30:17.477 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:17.477+0000 7f06fac00700 1 -- 172.21.15.153:0/1796782096 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06f41a0e60 con 0x7f06f41071d0 2024-03-21T10:30:17.478 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:17.478+0000 7f06fac00700 1 -- 172.21.15.153:0/1796782096 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06f410f6c0 con 0x7f06f41071d0 2024-03-21T10:30:17.478 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:17.479+0000 7f06f1600700 1 -- 172.21.15.153:0/1796782096 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f06e40049a0 con 0x7f06f41071d0 2024-03-21T10:30:17.479 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:17.480+0000 7f06f1600700 1 --2- 172.21.15.153:0/1796782096 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f06d806e460 0x7f06d8070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:17.479 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:17.480+0000 7f06f1600700 1 -- 172.21.15.153:0/1796782096 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f06e408e170 con 0x7f06f41071d0 2024-03-21T10:30:17.479 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:17.481+0000 7f06ebe00700 1 --2- 172.21.15.153:0/1796782096 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f06d806e460 0x7f06d8070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:17.480 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:17.481+0000 7f06ebe00700 1 --2- 172.21.15.153:0/1796782096 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f06d806e460 0x7f06d8070920 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f06f419e290 tx=0x7f06dc01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:17.484 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:17.486+0000 7f06f1600700 1 -- 172.21.15.153:0/1796782096 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f06e4057210 con 0x7f06f41071d0 2024-03-21T10:30:17.728 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:17.729+0000 7f06fac00700 1 -- 172.21.15.153:0/1796782096 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f06f41a1130 con 0x7f06d806e460 2024-03-21T10:30:17.731 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:17.733+0000 7f06f1600700 1 -- 172.21.15.153:0/1796782096 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f06f41a1130 con 0x7f06d806e460 2024-03-21T10:30:17.734 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:17.735+0000 7f06fac00700 1 -- 172.21.15.153:0/1796782096 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f06d806e460 msgr2=0x7f06d8070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:17.734 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:17.735+0000 7f06fac00700 1 --2- 172.21.15.153:0/1796782096 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f06d806e460 0x7f06d8070920 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f06f419e290 tx=0x7f06dc01a040 comp rx=0 tx=0).stop 2024-03-21T10:30:17.734 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:17.735+0000 7f06fac00700 1 -- 172.21.15.153:0/1796782096 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f06f41071d0 msgr2=0x7f06f419ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:17.734 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:17.735+0000 7f06fac00700 1 --2- 172.21.15.153:0/1796782096 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f06f41071d0 0x7f06f419ccb0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f06e400c8a0 tx=0x7f06e400cc60 comp rx=0 tx=0).stop 2024-03-21T10:30:17.734 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:17.736+0000 7f06fac00700 1 -- 172.21.15.153:0/1796782096 shutdown_connections 2024-03-21T10:30:17.735 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:17.736+0000 7f06fac00700 1 --2- 172.21.15.153:0/1796782096 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f06d806e460 0x7f06d8070920 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:17.735 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:17.736+0000 7f06fac00700 1 --2- 172.21.15.153:0/1796782096 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f06f41071d0 0x7f06f419ccb0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:17.735 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:17.736+0000 7f06fac00700 1 --2- 172.21.15.153:0/1796782096 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f06f410d280 0x7f06f419d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:17.735 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:17.736+0000 7f06fac00700 1 -- 172.21.15.153:0/1796782096 >> 172.21.15.153:0/1796782096 conn(0x7f06f40769e0 msgr2=0x7f06f4078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:17.735 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:17.736+0000 7f06fac00700 1 -- 172.21.15.153:0/1796782096 shutdown_connections 2024-03-21T10:30:17.735 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:17.736+0000 7f06fac00700 1 -- 172.21.15.153:0/1796782096 wait complete. 2024-03-21T10:30:17.750 INFO:teuthology.orchestra.run.smithi153.stdout:true 2024-03-21T10:30:17.892 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:17.893+0000 7f1c49600700 1 -- 172.21.15.153:0/463955464 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c44073a80 msgr2=0x7f1c4410cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:17.892 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:17.893+0000 7f1c49600700 1 --2- 172.21.15.153:0/463955464 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c44073a80 0x7f1c4410cea0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f1c2c009910 tx=0x7f1c2c009c20 comp rx=0 tx=0).stop 2024-03-21T10:30:17.892 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:17.893+0000 7f1c49600700 1 -- 172.21.15.153:0/463955464 shutdown_connections 2024-03-21T10:30:17.892 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:17.893+0000 7f1c49600700 1 --2- 172.21.15.153:0/463955464 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c44073a80 0x7f1c4410cea0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:17.892 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:17.893+0000 7f1c49600700 1 --2- 172.21.15.153:0/463955464 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1c44073160 0x7f1c44073540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:17.893 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:17.893+0000 7f1c49600700 1 -- 172.21.15.153:0/463955464 >> 172.21.15.153:0/463955464 conn(0x7f1c440fc630 msgr2=0x7f1c440fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:17.893 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:17.893+0000 7f1c49600700 1 -- 172.21.15.153:0/463955464 shutdown_connections 2024-03-21T10:30:17.893 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:17.893+0000 7f1c49600700 1 -- 172.21.15.153:0/463955464 wait complete. 2024-03-21T10:30:17.894 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:17.894+0000 7f1c49600700 1 Processor -- start 2024-03-21T10:30:17.894 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:17.894+0000 7f1c49600700 1 -- start start 2024-03-21T10:30:17.894 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:17.894+0000 7f1c49600700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c44073160 0x7f1c4419e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:17.894 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:17.895+0000 7f1c49600700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1c44073a80 0x7f1c4419ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:17.894 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:17.895+0000 7f1c49600700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c4419f240 con 0x7f1c44073160 2024-03-21T10:30:17.894 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:17.895+0000 7f1c49600700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c44198780 con 0x7f1c44073a80 2024-03-21T10:30:17.895 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:17.895+0000 7f1c42a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c44073160 0x7f1c4419e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:17.895 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:17.895+0000 7f1c42a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c44073160 0x7f1c4419e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41980/0 (socket says 172.21.15.153:41980) 2024-03-21T10:30:17.895 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:17.895+0000 7f1c42a00700 1 -- 172.21.15.153:0/652433281 learned_addr learned my addr 172.21.15.153:0/652433281 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:30:17.895 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:17.895+0000 7f1c42000700 1 --2- 172.21.15.153:0/652433281 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1c44073a80 0x7f1c4419ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:17.895 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:17.895+0000 7f1c42a00700 1 -- 172.21.15.153:0/652433281 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1c44073a80 msgr2=0x7f1c4419ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:17.895 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:17.895+0000 7f1c42a00700 1 --2- 172.21.15.153:0/652433281 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1c44073a80 0x7f1c4419ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:17.895 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:17.895+0000 7f1c42a00700 1 -- 172.21.15.153:0/652433281 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c2c0095f0 con 0x7f1c44073160 2024-03-21T10:30:17.895 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:17.896+0000 7f1c42a00700 1 --2- 172.21.15.153:0/652433281 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c44073160 0x7f1c4419e700 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f1c3400c930 tx=0x7f1c3400ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:17.896 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:17.896+0000 7f1c3b400700 1 -- 172.21.15.153:0/652433281 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1c34007ad0 con 0x7f1c44073160 2024-03-21T10:30:17.896 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:17.896+0000 7f1c3b400700 1 -- 172.21.15.153:0/652433281 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1c34007c30 con 0x7f1c44073160 2024-03-21T10:30:17.896 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:17.896+0000 7f1c49600700 1 -- 172.21.15.153:0/652433281 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c44198a60 con 0x7f1c44073160 2024-03-21T10:30:17.896 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:17.896+0000 7f1c3b400700 1 -- 172.21.15.153:0/652433281 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1c34018610 con 0x7f1c44073160 2024-03-21T10:30:17.896 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:17.896+0000 7f1c49600700 1 -- 172.21.15.153:0/652433281 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c44198fb0 con 0x7f1c44073160 2024-03-21T10:30:17.896 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:17.897+0000 7f1c49600700 1 -- 172.21.15.153:0/652433281 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c4410a5a0 con 0x7f1c44073160 2024-03-21T10:30:17.897 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:17.898+0000 7f1c3b400700 1 -- 172.21.15.153:0/652433281 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f1c3400f450 con 0x7f1c44073160 2024-03-21T10:30:17.898 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:17.899+0000 7f1c3b400700 1 --2- 172.21.15.153:0/652433281 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f1c2806e460 0x7f1c28070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:17.898 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:17.899+0000 7f1c3b400700 1 -- 172.21.15.153:0/652433281 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f1c3408e7b0 con 0x7f1c44073160 2024-03-21T10:30:17.898 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:17.900+0000 7f1c42000700 1 --2- 172.21.15.153:0/652433281 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f1c2806e460 0x7f1c28070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:17.899 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:17.900+0000 7f1c42000700 1 --2- 172.21.15.153:0/652433281 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f1c2806e460 0x7f1c28070920 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f1c2c005f50 tx=0x7f1c2c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:17.903 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:17.904+0000 7f1c3b400700 1 -- 172.21.15.153:0/652433281 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f1c3405b300 con 0x7f1c44073160 2024-03-21T10:30:18.146 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:18.148+0000 7f1c49600700 1 -- 172.21.15.153:0/652433281 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1c44199280 con 0x7f1c2806e460 2024-03-21T10:30:18.149 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:18.151+0000 7f1c3b400700 1 -- 172.21.15.153:0/652433281 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f1c44199280 con 0x7f1c2806e460 2024-03-21T10:30:18.152 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:18.153+0000 7f1c49600700 1 -- 172.21.15.153:0/652433281 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f1c2806e460 msgr2=0x7f1c28070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:18.152 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:18.153+0000 7f1c49600700 1 --2- 172.21.15.153:0/652433281 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f1c2806e460 0x7f1c28070920 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f1c2c005f50 tx=0x7f1c2c01a040 comp rx=0 tx=0).stop 2024-03-21T10:30:18.152 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:18.154+0000 7f1c49600700 1 -- 172.21.15.153:0/652433281 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c44073160 msgr2=0x7f1c4419e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:18.153 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:18.154+0000 7f1c49600700 1 --2- 172.21.15.153:0/652433281 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c44073160 0x7f1c4419e700 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f1c3400c930 tx=0x7f1c3400ccf0 comp rx=0 tx=0).stop 2024-03-21T10:30:18.153 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:18.154+0000 7f1c49600700 1 -- 172.21.15.153:0/652433281 shutdown_connections 2024-03-21T10:30:18.153 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:18.154+0000 7f1c49600700 1 --2- 172.21.15.153:0/652433281 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f1c2806e460 0x7f1c28070920 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:18.153 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:18.154+0000 7f1c49600700 1 --2- 172.21.15.153:0/652433281 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c44073160 0x7f1c4419e700 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:18.153 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:18.154+0000 7f1c49600700 1 --2- 172.21.15.153:0/652433281 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1c44073a80 0x7f1c4419ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:18.153 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:18.154+0000 7f1c49600700 1 -- 172.21.15.153:0/652433281 >> 172.21.15.153:0/652433281 conn(0x7f1c440fc630 msgr2=0x7f1c441076e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:18.153 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:18.154+0000 7f1c49600700 1 -- 172.21.15.153:0/652433281 shutdown_connections 2024-03-21T10:30:18.153 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:18.154+0000 7f1c49600700 1 -- 172.21.15.153:0/652433281 wait complete. 2024-03-21T10:30:18.310 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:18.311+0000 7f9c93600700 1 -- 172.21.15.153:0/938304846 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c8c0ff910 msgr2=0x7f9c8c0ffcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:18.311 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:18.311+0000 7f9c93600700 1 --2- 172.21.15.153:0/938304846 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c8c0ff910 0x7f9c8c0ffcf0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f9c74009910 tx=0x7f9c74009c20 comp rx=0 tx=0).stop 2024-03-21T10:30:18.311 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:18.311+0000 7f9c93600700 1 -- 172.21.15.153:0/938304846 shutdown_connections 2024-03-21T10:30:18.311 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:18.311+0000 7f9c93600700 1 --2- 172.21.15.153:0/938304846 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9c8c1002c0 0x7f9c8c10cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:18.311 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:18.311+0000 7f9c93600700 1 --2- 172.21.15.153:0/938304846 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c8c0ff910 0x7f9c8c0ffcf0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:18.311 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:18.311+0000 7f9c93600700 1 -- 172.21.15.153:0/938304846 >> 172.21.15.153:0/938304846 conn(0x7f9c8c0fb570 msgr2=0x7f9c8c0fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:18.311 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:18.312+0000 7f9c93600700 1 -- 172.21.15.153:0/938304846 shutdown_connections 2024-03-21T10:30:18.311 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:18.312+0000 7f9c93600700 1 -- 172.21.15.153:0/938304846 wait complete. 2024-03-21T10:30:18.312 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:18.313+0000 7f9c93600700 1 Processor -- start 2024-03-21T10:30:18.312 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:18.313+0000 7f9c93600700 1 -- start start 2024-03-21T10:30:18.312 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:18.313+0000 7f9c93600700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9c8c0ff910 0x7f9c8c19e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:18.312 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:18.313+0000 7f9c93600700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c8c1002c0 0x7f9c8c19ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:18.312 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:18.313+0000 7f9c93600700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c8c19f3e0 con 0x7f9c8c1002c0 2024-03-21T10:30:18.312 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:18.313+0000 7f9c93600700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c8c198890 con 0x7f9c8c0ff910 2024-03-21T10:30:18.312 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:18.314+0000 7f9c83e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c8c1002c0 0x7f9c8c19ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:18.312 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:18.314+0000 7f9c83e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c8c1002c0 0x7f9c8c19ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42000/0 (socket says 172.21.15.153:42000) 2024-03-21T10:30:18.313 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:18.314+0000 7f9c83e00700 1 -- 172.21.15.153:0/1933844510 learned_addr learned my addr 172.21.15.153:0/1933844510 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:30:18.314 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:18.314+0000 7f9c90c00700 1 --2- 172.21.15.153:0/1933844510 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9c8c0ff910 0x7f9c8c19e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:18.314 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:18.314+0000 7f9c83e00700 1 -- 172.21.15.153:0/1933844510 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9c8c0ff910 msgr2=0x7f9c8c19e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:18.314 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:18.314+0000 7f9c83e00700 1 --2- 172.21.15.153:0/1933844510 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9c8c0ff910 0x7f9c8c19e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:18.314 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:18.314+0000 7f9c83e00700 1 -- 172.21.15.153:0/1933844510 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c740095f0 con 0x7f9c8c1002c0 2024-03-21T10:30:18.314 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:18.314+0000 7f9c83e00700 1 --2- 172.21.15.153:0/1933844510 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c8c1002c0 0x7f9c8c19ed50 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c00b4b0 tx=0x7f9c7c00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:18.314 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:18.315+0000 7f9c8a000700 1 -- 172.21.15.153:0/1933844510 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9c7c011800 con 0x7f9c8c1002c0 2024-03-21T10:30:18.314 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:18.315+0000 7f9c8a000700 1 -- 172.21.15.153:0/1933844510 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9c7c011e20 con 0x7f9c8c1002c0 2024-03-21T10:30:18.314 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:18.315+0000 7f9c93600700 1 -- 172.21.15.153:0/1933844510 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c8c198b70 con 0x7f9c8c1002c0 2024-03-21T10:30:18.314 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:18.315+0000 7f9c8a000700 1 -- 172.21.15.153:0/1933844510 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9c7c00f2e0 con 0x7f9c8c1002c0 2024-03-21T10:30:18.314 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:18.315+0000 7f9c93600700 1 -- 172.21.15.153:0/1933844510 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c8c1990c0 con 0x7f9c8c1002c0 2024-03-21T10:30:18.315 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:18.316+0000 7f9c93600700 1 -- 172.21.15.153:0/1933844510 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c8c10a6f0 con 0x7f9c8c1002c0 2024-03-21T10:30:18.315 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:18.317+0000 7f9c8a000700 1 -- 172.21.15.153:0/1933844510 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f9c7c010380 con 0x7f9c8c1002c0 2024-03-21T10:30:18.316 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:18.318+0000 7f9c8a000700 1 --2- 172.21.15.153:0/1933844510 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f9c6c06e460 0x7f9c6c070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:18.317 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:18.318+0000 7f9c8a000700 1 -- 172.21.15.153:0/1933844510 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f9c7c08e190 con 0x7f9c8c1002c0 2024-03-21T10:30:18.317 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:18.318+0000 7f9c90c00700 1 --2- 172.21.15.153:0/1933844510 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f9c6c06e460 0x7f9c6c070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:18.317 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:18.319+0000 7f9c90c00700 1 --2- 172.21.15.153:0/1933844510 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f9c6c06e460 0x7f9c6c070920 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9c74005b40 tx=0x7f9c74005a90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:18.321 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:18.322+0000 7f9c8a000700 1 -- 172.21.15.153:0/1933844510 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f9c7c05ace0 con 0x7f9c8c1002c0 2024-03-21T10:30:18.555 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:18.556+0000 7f9c93600700 1 -- 172.21.15.153:0/1933844510 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f9c8c199340 con 0x7f9c6c06e460 2024-03-21T10:30:18.567 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:18.569+0000 7f9c8a000700 1 -- 172.21.15.153:0/1933844510 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f9c8c199340 con 0x7f9c6c06e460 2024-03-21T10:30:18.568 INFO:teuthology.orchestra.run.smithi153.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-21T10:30:18.568 INFO:teuthology.orchestra.run.smithi153.stdout:alertmanager.smithi153 smithi153 *:9093,9094 running (19m) 10m ago 32m 14.7M - 0.25.0 c8568f914cd2 9ed3f2200b5f 2024-03-21T10:30:18.568 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi153 smithi153 running (33m) 10m ago 33m 9424k - 18.2.1 5be31c24972a 5b61994396a6 2024-03-21T10:30:18.568 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi174 smithi174 running (31m) 10m ago 31m 9244k - 18.2.1 5be31c24972a 4f7a5fc3ac28 2024-03-21T10:30:18.568 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi153 smithi153 running (15m) 10m ago 33m 6656k - 19.0.0-2231-g2edfee68 237e43c7a65f 32c21104d5b0 2024-03-21T10:30:18.568 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi174 smithi174 running (15m) 10m ago 31m 6660k - 19.0.0-2231-g2edfee68 237e43c7a65f a04e48967ab6 2024-03-21T10:30:18.568 INFO:teuthology.orchestra.run.smithi153.stdout:grafana.smithi153 smithi153 *:3000 running (18m) 10m ago 32m 73.6M - 9.4.12 2bacad6d85d8 ee24026ab786 2024-03-21T10:30:18.568 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.iuyluj smithi153 running (10m) 10m ago 25m 10.9M - 19.0.0-2231-g2edfee68 237e43c7a65f 87aff8a5de70 2024-03-21T10:30:18.569 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.zipckr smithi153 running (11m) 10m ago 25m 11.8M - 19.0.0-2231-g2edfee68 237e43c7a65f ccc262bd5dd3 2024-03-21T10:30:18.569 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.nhozag smithi174 running (25m) 10m ago 25m 12.0M - 18.2.1 5be31c24972a 2fee4000308f 2024-03-21T10:30:18.569 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.yyimst smithi174 running (10m) 10m ago 25m 10.7M - 19.0.0-2231-g2edfee68 237e43c7a65f fc1962f9351e 2024-03-21T10:30:18.569 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi153.tluxni smithi153 *:8443,9283,8765 running (22m) 10m ago 34m 509M - 19.0.0-2231-g2edfee68 237e43c7a65f 7ffee0e55382 2024-03-21T10:30:18.569 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi174.qpcwat smithi174 *:8443,9283,8765 running (20m) 10m ago 31m 452M - 19.0.0-2231-g2edfee68 237e43c7a65f 748968f216ab 2024-03-21T10:30:18.569 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi153 smithi153 running (16m) 10m ago 34m 55.0M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f dd8038303dc3 2024-03-21T10:30:18.569 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi174 smithi174 running (16m) 10m ago 31m 44.2M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f 29d78a788f2c 2024-03-21T10:30:18.569 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi153 smithi153 *:9100 running (20m) 10m ago 33m 7889k - 1.5.0 0da6a335fe13 bb835edc0e0b 2024-03-21T10:30:18.569 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi174 smithi174 *:9100 running (20m) 10m ago 31m 8102k - 1.5.0 0da6a335fe13 7c4ecad790fa 2024-03-21T10:30:18.569 INFO:teuthology.orchestra.run.smithi153.stdout:osd.0 smithi153 running (14m) 10m ago 30m 116M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 507900fd4791 2024-03-21T10:30:18.569 INFO:teuthology.orchestra.run.smithi153.stdout:osd.1 smithi153 running (13m) 10m ago 29m 89.2M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 0d646452e9c1 2024-03-21T10:30:18.569 INFO:teuthology.orchestra.run.smithi153.stdout:osd.2 smithi153 running (13m) 10m ago 29m 87.4M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 1326f2afdfcd 2024-03-21T10:30:18.569 INFO:teuthology.orchestra.run.smithi153.stdout:osd.3 smithi174 running (12m) 10m ago 28m 83.6M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 70aafac71ad5 2024-03-21T10:30:18.569 INFO:teuthology.orchestra.run.smithi153.stdout:osd.4 smithi174 running (12m) 10m ago 28m 92.2M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 0d746ce6e79d 2024-03-21T10:30:18.569 INFO:teuthology.orchestra.run.smithi153.stdout:osd.5 smithi174 running (11m) 10m ago 27m 86.2M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 53e5f35b8cac 2024-03-21T10:30:18.570 INFO:teuthology.orchestra.run.smithi153.stdout:prometheus.smithi153 smithi153 *:9095 running (19m) 10m ago 32m 73.5M - 2.43.0 a07b618ecd1d 7d95561e50c2 2024-03-21T10:30:18.571 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:18.572+0000 7f9c93600700 1 -- 172.21.15.153:0/1933844510 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f9c6c06e460 msgr2=0x7f9c6c070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:18.571 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:18.572+0000 7f9c93600700 1 --2- 172.21.15.153:0/1933844510 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f9c6c06e460 0x7f9c6c070920 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9c74005b40 tx=0x7f9c74005a90 comp rx=0 tx=0).stop 2024-03-21T10:30:18.571 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:18.572+0000 7f9c93600700 1 -- 172.21.15.153:0/1933844510 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c8c1002c0 msgr2=0x7f9c8c19ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:18.571 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:18.572+0000 7f9c93600700 1 --2- 172.21.15.153:0/1933844510 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c8c1002c0 0x7f9c8c19ed50 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c00b4b0 tx=0x7f9c7c00b870 comp rx=0 tx=0).stop 2024-03-21T10:30:18.571 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:18.572+0000 7f9c93600700 1 -- 172.21.15.153:0/1933844510 shutdown_connections 2024-03-21T10:30:18.571 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:18.572+0000 7f9c93600700 1 --2- 172.21.15.153:0/1933844510 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9c8c0ff910 0x7f9c8c19e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:18.571 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:18.572+0000 7f9c93600700 1 --2- 172.21.15.153:0/1933844510 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f9c6c06e460 0x7f9c6c070920 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:18.571 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:18.572+0000 7f9c93600700 1 --2- 172.21.15.153:0/1933844510 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9c8c1002c0 0x7f9c8c19ed50 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:18.571 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:18.572+0000 7f9c93600700 1 -- 172.21.15.153:0/1933844510 >> 172.21.15.153:0/1933844510 conn(0x7f9c8c0fb570 msgr2=0x7f9c8c0fcc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:18.571 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:18.572+0000 7f9c93600700 1 -- 172.21.15.153:0/1933844510 shutdown_connections 2024-03-21T10:30:18.571 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:18.572+0000 7f9c93600700 1 -- 172.21.15.153:0/1933844510 wait complete. 2024-03-21T10:30:18.722 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:18.723+0000 7f0465600700 1 -- 172.21.15.153:0/2145137171 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0460106970 msgr2=0x7f0460106d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:18.722 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:18.723+0000 7f0465600700 1 --2- 172.21.15.153:0/2145137171 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0460106970 0x7f0460106d50 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f0450009910 tx=0x7f0450009c20 comp rx=0 tx=0).stop 2024-03-21T10:30:18.722 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:18.723+0000 7f0465600700 1 -- 172.21.15.153:0/2145137171 shutdown_connections 2024-03-21T10:30:18.722 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:18.723+0000 7f0465600700 1 --2- 172.21.15.153:0/2145137171 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0460100970 0x7f0460100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:18.722 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:18.723+0000 7f0465600700 1 --2- 172.21.15.153:0/2145137171 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0460106970 0x7f0460106d50 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:18.723 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:18.723+0000 7f0465600700 1 -- 172.21.15.153:0/2145137171 >> 172.21.15.153:0/2145137171 conn(0x7f04600fc440 msgr2=0x7f04600fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:18.723 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:18.723+0000 7f0465600700 1 -- 172.21.15.153:0/2145137171 shutdown_connections 2024-03-21T10:30:18.723 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:18.723+0000 7f0465600700 1 -- 172.21.15.153:0/2145137171 wait complete. 2024-03-21T10:30:18.723 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:18.724+0000 7f0465600700 1 Processor -- start 2024-03-21T10:30:18.724 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:18.725+0000 7f0465600700 1 -- start start 2024-03-21T10:30:18.724 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:18.725+0000 7f0465600700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0460100970 0x7f0460073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:18.724 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:18.725+0000 7f0465600700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0460106970 0x7f04600735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:18.724 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:18.725+0000 7f0465600700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0460073b10 con 0x7f0460106970 2024-03-21T10:30:18.724 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:18.725+0000 7f0465600700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0460073c50 con 0x7f0460100970 2024-03-21T10:30:18.724 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:18.725+0000 7f045f400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0460106970 0x7f04600735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:18.724 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:18.725+0000 7f045f400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0460106970 0x7f04600735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42026/0 (socket says 172.21.15.153:42026) 2024-03-21T10:30:18.724 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:18.725+0000 7f045f400700 1 -- 172.21.15.153:0/4202109755 learned_addr learned my addr 172.21.15.153:0/4202109755 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:30:18.725 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:18.726+0000 7f045fe00700 1 --2- 172.21.15.153:0/4202109755 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0460100970 0x7f0460073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:18.725 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:18.726+0000 7f045f400700 1 -- 172.21.15.153:0/4202109755 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0460100970 msgr2=0x7f0460073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:18.725 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:18.726+0000 7f045f400700 1 --2- 172.21.15.153:0/4202109755 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0460100970 0x7f0460073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:18.725 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:18.726+0000 7f045f400700 1 -- 172.21.15.153:0/4202109755 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04500095f0 con 0x7f0460106970 2024-03-21T10:30:18.725 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:18.726+0000 7f045f400700 1 --2- 172.21.15.153:0/4202109755 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0460106970 0x7f04600735d0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f044c00d8d0 tx=0x7f044c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:18.725 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:18.726+0000 7f045cc00700 1 -- 172.21.15.153:0/4202109755 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f044c009940 con 0x7f0460106970 2024-03-21T10:30:18.725 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:18.726+0000 7f045cc00700 1 -- 172.21.15.153:0/4202109755 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f044c010460 con 0x7f0460106970 2024-03-21T10:30:18.726 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:18.726+0000 7f0465600700 1 -- 172.21.15.153:0/4202109755 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04601a2710 con 0x7f0460106970 2024-03-21T10:30:18.726 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:18.726+0000 7f0465600700 1 -- 172.21.15.153:0/4202109755 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04601a2b20 con 0x7f0460106970 2024-03-21T10:30:18.726 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:18.727+0000 7f045cc00700 1 -- 172.21.15.153:0/4202109755 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f044c00f5f0 con 0x7f0460106970 2024-03-21T10:30:18.726 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:18.727+0000 7f0465600700 1 -- 172.21.15.153:0/4202109755 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0460108f70 con 0x7f0460106970 2024-03-21T10:30:18.727 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:18.729+0000 7f045cc00700 1 -- 172.21.15.153:0/4202109755 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f044c0105d0 con 0x7f0460106970 2024-03-21T10:30:18.728 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:18.730+0000 7f045cc00700 1 --2- 172.21.15.153:0/4202109755 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f044406e460 0x7f0444070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:18.728 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:18.730+0000 7f045cc00700 1 -- 172.21.15.153:0/4202109755 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f044c08e5b0 con 0x7f0460106970 2024-03-21T10:30:18.728 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:18.730+0000 7f045fe00700 1 --2- 172.21.15.153:0/4202109755 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f044406e460 0x7f0444070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:18.729 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:18.731+0000 7f045fe00700 1 --2- 172.21.15.153:0/4202109755 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f044406e460 0x7f0444070920 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f0450006010 tx=0x7f045001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:18.733 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:18.734+0000 7f045cc00700 1 -- 172.21.15.153:0/4202109755 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f044c0575d0 con 0x7f0460106970 2024-03-21T10:30:18.949 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:30:18 smithi174 ceph-mon[88004]: pgmap v553: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:30:18.949 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:30:18 smithi174 ceph-mon[88004]: from='client.35010 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:30:18.950 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:30:18 smithi174 ceph-mon[88004]: from='client.35014 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:30:19.026 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:19.027+0000 7f0465600700 1 -- 172.21.15.153:0/4202109755 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f046004f2e0 con 0x7f0460106970 2024-03-21T10:30:19.027 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:30:18 smithi153 ceph-mon[113733]: pgmap v553: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:30:19.027 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:30:18 smithi153 ceph-mon[113733]: from='client.35010 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:30:19.027 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:30:18 smithi153 ceph-mon[113733]: from='client.35014 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:30:19.027 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:19.028+0000 7f045cc00700 1 -- 172.21.15.153:0/4202109755 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+805 (secure 0 0 0) 0x7f044c05ac90 con 0x7f0460106970 2024-03-21T10:30:19.028 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:30:19.028 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": { 2024-03-21T10:30:19.028 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:30:19.028 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:30:19.028 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": { 2024-03-21T10:30:19.028 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:30:19.028 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:30:19.028 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": { 2024-03-21T10:30:19.028 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 6 2024-03-21T10:30:19.028 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:30:19.028 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": { 2024-03-21T10:30:19.028 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-03-21T10:30:19.028 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 3 2024-03-21T10:30:19.029 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:30:19.029 INFO:teuthology.orchestra.run.smithi153.stdout: "overall": { 2024-03-21T10:30:19.029 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-03-21T10:30:19.029 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 13 2024-03-21T10:30:19.029 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-03-21T10:30:19.029 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:30:19.031 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:19.032+0000 7f0465600700 1 -- 172.21.15.153:0/4202109755 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f044406e460 msgr2=0x7f0444070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:19.031 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:19.032+0000 7f0465600700 1 --2- 172.21.15.153:0/4202109755 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f044406e460 0x7f0444070920 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f0450006010 tx=0x7f045001a040 comp rx=0 tx=0).stop 2024-03-21T10:30:19.031 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:19.032+0000 7f0465600700 1 -- 172.21.15.153:0/4202109755 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0460106970 msgr2=0x7f04600735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:19.031 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:19.032+0000 7f0465600700 1 --2- 172.21.15.153:0/4202109755 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0460106970 0x7f04600735d0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f044c00d8d0 tx=0x7f044c00dc90 comp rx=0 tx=0).stop 2024-03-21T10:30:19.031 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:19.032+0000 7f0465600700 1 -- 172.21.15.153:0/4202109755 shutdown_connections 2024-03-21T10:30:19.031 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:19.032+0000 7f0465600700 1 --2- 172.21.15.153:0/4202109755 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0460100970 0x7f0460073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:19.031 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:19.032+0000 7f0465600700 1 --2- 172.21.15.153:0/4202109755 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f044406e460 0x7f0444070920 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:19.032 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:19.032+0000 7f0465600700 1 --2- 172.21.15.153:0/4202109755 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0460106970 0x7f04600735d0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:19.032 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:19.032+0000 7f0465600700 1 -- 172.21.15.153:0/4202109755 >> 172.21.15.153:0/4202109755 conn(0x7f04600fc440 msgr2=0x7f04600fcd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:19.032 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:19.032+0000 7f0465600700 1 -- 172.21.15.153:0/4202109755 shutdown_connections 2024-03-21T10:30:19.032 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:19.032+0000 7f0465600700 1 -- 172.21.15.153:0/4202109755 wait complete. 2024-03-21T10:30:19.186 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:19.187+0000 7f0b1e400700 1 -- 172.21.15.153:0/2465289438 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0b181071d0 msgr2=0x7f0b18107650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:19.187 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:19.187+0000 7f0b1e400700 1 --2- 172.21.15.153:0/2465289438 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0b181071d0 0x7f0b18107650 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f0b04009910 tx=0x7f0b04009c20 comp rx=0 tx=0).stop 2024-03-21T10:30:19.187 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:19.187+0000 7f0b1e400700 1 -- 172.21.15.153:0/2465289438 shutdown_connections 2024-03-21T10:30:19.187 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:19.187+0000 7f0b1e400700 1 --2- 172.21.15.153:0/2465289438 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0b181071d0 0x7f0b18107650 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:19.187 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:19.187+0000 7f0b1e400700 1 --2- 172.21.15.153:0/2465289438 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b1810d280 0x7f0b1810d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:19.187 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:19.187+0000 7f0b1e400700 1 -- 172.21.15.153:0/2465289438 >> 172.21.15.153:0/2465289438 conn(0x7f0b180769e0 msgr2=0x7f0b18078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:19.187 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:19.188+0000 7f0b1e400700 1 -- 172.21.15.153:0/2465289438 shutdown_connections 2024-03-21T10:30:19.187 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:19.188+0000 7f0b1e400700 1 -- 172.21.15.153:0/2465289438 wait complete. 2024-03-21T10:30:19.189 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:19.189+0000 7f0b1e400700 1 Processor -- start 2024-03-21T10:30:19.189 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:19.189+0000 7f0b1e400700 1 -- start start 2024-03-21T10:30:19.189 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:19.189+0000 7f0b1e400700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b181071d0 0x7f0b1819ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:19.189 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:19.189+0000 7f0b1e400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0b1810d280 0x7f0b1819d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:19.189 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:19.189+0000 7f0b1e400700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b1819d880 con 0x7f0b1810d280 2024-03-21T10:30:19.189 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:19.189+0000 7f0b1e400700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b181a05b0 con 0x7f0b181071d0 2024-03-21T10:30:19.189 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:19.189+0000 7f0b16a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0b1810d280 0x7f0b1819d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:19.189 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:19.189+0000 7f0b16a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0b1810d280 0x7f0b1819d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42054/0 (socket says 172.21.15.153:42054) 2024-03-21T10:30:19.190 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:19.189+0000 7f0b16a00700 1 -- 172.21.15.153:0/1749694811 learned_addr learned my addr 172.21.15.153:0/1749694811 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:30:19.190 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:19.190+0000 7f0b17400700 1 --2- 172.21.15.153:0/1749694811 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b181071d0 0x7f0b1819ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:19.190 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:19.190+0000 7f0b16a00700 1 -- 172.21.15.153:0/1749694811 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b181071d0 msgr2=0x7f0b1819ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:19.190 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:19.190+0000 7f0b16a00700 1 --2- 172.21.15.153:0/1749694811 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b181071d0 0x7f0b1819ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:19.190 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:19.190+0000 7f0b16a00700 1 -- 172.21.15.153:0/1749694811 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b040095f0 con 0x7f0b1810d280 2024-03-21T10:30:19.190 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:19.190+0000 7f0b16a00700 1 --2- 172.21.15.153:0/1749694811 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0b1810d280 0x7f0b1819d1f0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f0b0400b5c0 tx=0x7f0b0400be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:19.190 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:19.190+0000 7f0b0fe00700 1 -- 172.21.15.153:0/1749694811 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0b0401d070 con 0x7f0b1810d280 2024-03-21T10:30:19.190 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:19.190+0000 7f0b0fe00700 1 -- 172.21.15.153:0/1749694811 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0b0400f460 con 0x7f0b1810d280 2024-03-21T10:30:19.190 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:19.190+0000 7f0b1e400700 1 -- 172.21.15.153:0/1749694811 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b181a0830 con 0x7f0b1810d280 2024-03-21T10:30:19.190 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:19.191+0000 7f0b0fe00700 1 -- 172.21.15.153:0/1749694811 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0b04017610 con 0x7f0b1810d280 2024-03-21T10:30:19.190 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:19.191+0000 7f0b1e400700 1 -- 172.21.15.153:0/1749694811 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b181a0da0 con 0x7f0b1810d280 2024-03-21T10:30:19.191 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:19.192+0000 7f0b1e400700 1 -- 172.21.15.153:0/1749694811 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b1810f6c0 con 0x7f0b1810d280 2024-03-21T10:30:19.191 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:19.193+0000 7f0b0fe00700 1 -- 172.21.15.153:0/1749694811 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f0b0400f5d0 con 0x7f0b1810d280 2024-03-21T10:30:19.192 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:19.194+0000 7f0b0fe00700 1 --2- 172.21.15.153:0/1749694811 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f0afc06e460 0x7f0afc070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:19.192 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:19.194+0000 7f0b0fe00700 1 -- 172.21.15.153:0/1749694811 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f0b0408fad0 con 0x7f0b1810d280 2024-03-21T10:30:19.192 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:19.194+0000 7f0b17400700 1 --2- 172.21.15.153:0/1749694811 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f0afc06e460 0x7f0afc070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:19.193 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:19.195+0000 7f0b17400700 1 --2- 172.21.15.153:0/1749694811 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f0afc06e460 0x7f0afc070920 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0b08005fd0 tx=0x7f0b0800a560 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:19.197 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:19.198+0000 7f0b0fe00700 1 -- 172.21.15.153:0/1749694811 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0b04058b70 con 0x7f0b1810d280 2024-03-21T10:30:19.461 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:19.462+0000 7f0b1e400700 1 -- 172.21.15.153:0/1749694811 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f0b1804ea90 con 0x7f0b1810d280 2024-03-21T10:30:19.461 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:19.462+0000 7f0b0fe00700 1 -- 172.21.15.153:0/1749694811 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1876 (secure 0 0 0) 0x7f0b0405c230 con 0x7f0b1810d280 2024-03-21T10:30:19.461 INFO:teuthology.orchestra.run.smithi153.stdout:e30 2024-03-21T10:30:19.462 INFO:teuthology.orchestra.run.smithi153.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-21T10:30:19.462 INFO:teuthology.orchestra.run.smithi153.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:30:19.462 INFO:teuthology.orchestra.run.smithi153.stdout:legacy client fscid: 1 2024-03-21T10:30:19.462 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:30:19.462 INFO:teuthology.orchestra.run.smithi153.stdout:Filesystem 'cephfs' (1) 2024-03-21T10:30:19.462 INFO:teuthology.orchestra.run.smithi153.stdout:fs_name cephfs 2024-03-21T10:30:19.462 INFO:teuthology.orchestra.run.smithi153.stdout:epoch 23 2024-03-21T10:30:19.462 INFO:teuthology.orchestra.run.smithi153.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-21T10:30:19.462 INFO:teuthology.orchestra.run.smithi153.stdout:created 2024-03-21T10:04:27.086576+0000 2024-03-21T10:30:19.462 INFO:teuthology.orchestra.run.smithi153.stdout:modified 2024-03-21T10:19:05.022092+0000 2024-03-21T10:30:19.462 INFO:teuthology.orchestra.run.smithi153.stdout:tableserver 0 2024-03-21T10:30:19.462 INFO:teuthology.orchestra.run.smithi153.stdout:root 0 2024-03-21T10:30:19.463 INFO:teuthology.orchestra.run.smithi153.stdout:session_timeout 60 2024-03-21T10:30:19.463 INFO:teuthology.orchestra.run.smithi153.stdout:session_autoclose 300 2024-03-21T10:30:19.463 INFO:teuthology.orchestra.run.smithi153.stdout:max_file_size 1099511627776 2024-03-21T10:30:19.463 INFO:teuthology.orchestra.run.smithi153.stdout:max_xattr_size 65536 2024-03-21T10:30:19.463 INFO:teuthology.orchestra.run.smithi153.stdout:required_client_features {} 2024-03-21T10:30:19.463 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure 0 2024-03-21T10:30:19.463 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure_osd_epoch 94 2024-03-21T10:30:19.463 INFO:teuthology.orchestra.run.smithi153.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:30:19.463 INFO:teuthology.orchestra.run.smithi153.stdout:max_mds 2 2024-03-21T10:30:19.463 INFO:teuthology.orchestra.run.smithi153.stdout:in 0,1 2024-03-21T10:30:19.463 INFO:teuthology.orchestra.run.smithi153.stdout:up {} 2024-03-21T10:30:19.463 INFO:teuthology.orchestra.run.smithi153.stdout:failed 0,1 2024-03-21T10:30:19.463 INFO:teuthology.orchestra.run.smithi153.stdout:damaged 2024-03-21T10:30:19.463 INFO:teuthology.orchestra.run.smithi153.stdout:stopped 2024-03-21T10:30:19.464 INFO:teuthology.orchestra.run.smithi153.stdout:data_pools [3] 2024-03-21T10:30:19.464 INFO:teuthology.orchestra.run.smithi153.stdout:metadata_pool 2 2024-03-21T10:30:19.464 INFO:teuthology.orchestra.run.smithi153.stdout:inline_data enabled 2024-03-21T10:30:19.464 INFO:teuthology.orchestra.run.smithi153.stdout:balancer 2024-03-21T10:30:19.464 INFO:teuthology.orchestra.run.smithi153.stdout:bal_rank_mask -1 2024-03-21T10:30:19.464 INFO:teuthology.orchestra.run.smithi153.stdout:standby_count_wanted 1 2024-03-21T10:30:19.464 INFO:teuthology.orchestra.run.smithi153.stdout:qdb_cluster leader: 0 members: 2024-03-21T10:30:19.464 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:30:19.464 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:30:19.464 INFO:teuthology.orchestra.run.smithi153.stdout:Standby daemons: 2024-03-21T10:30:19.464 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:30:19.464 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.zipckr{-1:34442} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6826/1103846295,v1:172.21.15.153:6827/1103846295] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:30:19.464 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.iuyluj{-1:34474} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6828/1738630832,v1:172.21.15.153:6829/1738630832] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:30:19.465 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.nhozag{-1:44269} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.174:6826/3967223227,v1:172.21.15.174:6827/3967223227] compat {c=[1],r=[1],i=[7ff]}] 2024-03-21T10:30:19.465 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.yyimst{-1:44313} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.174:6824/4127786797,v1:172.21.15.174:6825/4127786797] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:30:19.465 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:19.465+0000 7f0b1e400700 1 -- 172.21.15.153:0/1749694811 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f0afc06e460 msgr2=0x7f0afc070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:19.465 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:19.465+0000 7f0b1e400700 1 --2- 172.21.15.153:0/1749694811 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f0afc06e460 0x7f0afc070920 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0b08005fd0 tx=0x7f0b0800a560 comp rx=0 tx=0).stop 2024-03-21T10:30:19.466 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:19.465+0000 7f0b1e400700 1 -- 172.21.15.153:0/1749694811 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0b1810d280 msgr2=0x7f0b1819d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:19.466 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:19.465+0000 7f0b1e400700 1 --2- 172.21.15.153:0/1749694811 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0b1810d280 0x7f0b1819d1f0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f0b0400b5c0 tx=0x7f0b0400be00 comp rx=0 tx=0).stop 2024-03-21T10:30:19.466 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:19.465+0000 7f0b1e400700 1 -- 172.21.15.153:0/1749694811 shutdown_connections 2024-03-21T10:30:19.466 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:19.465+0000 7f0b1e400700 1 --2- 172.21.15.153:0/1749694811 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b181071d0 0x7f0b1819ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:19.466 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:19.465+0000 7f0b1e400700 1 --2- 172.21.15.153:0/1749694811 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f0afc06e460 0x7f0afc070920 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:19.466 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:19.465+0000 7f0b1e400700 1 --2- 172.21.15.153:0/1749694811 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0b1810d280 0x7f0b1819d1f0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:19.466 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:19.465+0000 7f0b1e400700 1 -- 172.21.15.153:0/1749694811 >> 172.21.15.153:0/1749694811 conn(0x7f0b180769e0 msgr2=0x7f0b18078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:19.466 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:19.466+0000 7f0b1e400700 1 -- 172.21.15.153:0/1749694811 shutdown_connections 2024-03-21T10:30:19.466 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:19.466+0000 7f0b1e400700 1 -- 172.21.15.153:0/1749694811 wait complete. 2024-03-21T10:30:19.466 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 30 2024-03-21T10:30:19.616 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:19.617+0000 7f8ac4800700 1 -- 172.21.15.153:0/597458708 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8abc1071d0 msgr2=0x7f8abc107650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:19.616 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:19.617+0000 7f8ac4800700 1 --2- 172.21.15.153:0/597458708 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8abc1071d0 0x7f8abc107650 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f8aac009910 tx=0x7f8aac009c20 comp rx=0 tx=0).stop 2024-03-21T10:30:19.617 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:19.617+0000 7f8ac4800700 1 -- 172.21.15.153:0/597458708 shutdown_connections 2024-03-21T10:30:19.617 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:19.617+0000 7f8ac4800700 1 --2- 172.21.15.153:0/597458708 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8abc1071d0 0x7f8abc107650 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:19.617 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:19.617+0000 7f8ac4800700 1 --2- 172.21.15.153:0/597458708 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8abc10d280 0x7f8abc10d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:19.617 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:19.617+0000 7f8ac4800700 1 -- 172.21.15.153:0/597458708 >> 172.21.15.153:0/597458708 conn(0x7f8abc0769e0 msgr2=0x7f8abc078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:19.617 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:19.618+0000 7f8ac4800700 1 -- 172.21.15.153:0/597458708 shutdown_connections 2024-03-21T10:30:19.617 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:19.618+0000 7f8ac4800700 1 -- 172.21.15.153:0/597458708 wait complete. 2024-03-21T10:30:19.617 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:19.619+0000 7f8ac4800700 1 Processor -- start 2024-03-21T10:30:19.618 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:19.619+0000 7f8ac4800700 1 -- start start 2024-03-21T10:30:19.619 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:19.619+0000 7f8ac4800700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8abc1071d0 0x7f8abc19ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:19.619 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:19.619+0000 7f8ac4800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8abc10d280 0x7f8abc19d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:19.619 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:19.619+0000 7f8ac4800700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8abc19d880 con 0x7f8abc10d280 2024-03-21T10:30:19.620 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:19.619+0000 7f8ac4800700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8abc1a05b0 con 0x7f8abc1071d0 2024-03-21T10:30:19.620 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:19.620+0000 7f8ac1400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8abc10d280 0x7f8abc19d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:19.620 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:19.620+0000 7f8ac1400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8abc10d280 0x7f8abc19d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42074/0 (socket says 172.21.15.153:42074) 2024-03-21T10:30:19.620 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:19.620+0000 7f8ac1400700 1 -- 172.21.15.153:0/3280868614 learned_addr learned my addr 172.21.15.153:0/3280868614 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:30:19.620 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:19.620+0000 7f8ac1e00700 1 --2- 172.21.15.153:0/3280868614 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8abc1071d0 0x7f8abc19ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:19.620 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:19.620+0000 7f8ac1400700 1 -- 172.21.15.153:0/3280868614 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8abc1071d0 msgr2=0x7f8abc19ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:19.620 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:19.620+0000 7f8ac1400700 1 --2- 172.21.15.153:0/3280868614 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8abc1071d0 0x7f8abc19ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:19.620 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:19.620+0000 7f8ac1400700 1 -- 172.21.15.153:0/3280868614 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8aac0095f0 con 0x7f8abc10d280 2024-03-21T10:30:19.620 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:19.620+0000 7f8ac1e00700 1 --2- 172.21.15.153:0/3280868614 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8abc1071d0 0x7f8abc19ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:30:19.620 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:19.621+0000 7f8ac1400700 1 --2- 172.21.15.153:0/3280868614 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8abc10d280 0x7f8abc19d1f0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f8aac00b5c0 tx=0x7f8aac00be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:19.620 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:19.621+0000 7f8ab6a00700 1 -- 172.21.15.153:0/3280868614 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8aac01d070 con 0x7f8abc10d280 2024-03-21T10:30:19.620 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:19.621+0000 7f8ab6a00700 1 -- 172.21.15.153:0/3280868614 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8aac00f460 con 0x7f8abc10d280 2024-03-21T10:30:19.620 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:19.621+0000 7f8ac4800700 1 -- 172.21.15.153:0/3280868614 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8abc1a0830 con 0x7f8abc10d280 2024-03-21T10:30:19.620 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:19.621+0000 7f8ab6a00700 1 -- 172.21.15.153:0/3280868614 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8aac017610 con 0x7f8abc10d280 2024-03-21T10:30:19.621 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:19.621+0000 7f8ac4800700 1 -- 172.21.15.153:0/3280868614 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8abc1a0da0 con 0x7f8abc10d280 2024-03-21T10:30:19.621 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:19.622+0000 7f8ac4800700 1 -- 172.21.15.153:0/3280868614 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8abc10f6c0 con 0x7f8abc10d280 2024-03-21T10:30:19.621 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:19.623+0000 7f8ab6a00700 1 -- 172.21.15.153:0/3280868614 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f8aac00f5d0 con 0x7f8abc10d280 2024-03-21T10:30:19.622 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:19.624+0000 7f8ab6a00700 1 --2- 172.21.15.153:0/3280868614 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f8aa406e460 0x7f8aa4070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:19.623 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:19.624+0000 7f8ab6a00700 1 -- 172.21.15.153:0/3280868614 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f8aac08fb90 con 0x7f8abc10d280 2024-03-21T10:30:19.623 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:19.624+0000 7f8ac1e00700 1 --2- 172.21.15.153:0/3280868614 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f8aa406e460 0x7f8aa4070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:19.624 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:19.625+0000 7f8ac1e00700 1 --2- 172.21.15.153:0/3280868614 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f8aa406e460 0x7f8aa4070920 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f8ab0005fd0 tx=0x7f8ab0005dc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:19.627 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:19.628+0000 7f8ab6a00700 1 -- 172.21.15.153:0/3280868614 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f8aac058bb0 con 0x7f8abc10d280 2024-03-21T10:30:19.804 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:30:19 smithi153 ceph-mon[113733]: from='client.35018 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:30:19.805 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:30:19 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/4202109755' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:30:19.805 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:30:19 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/1749694811' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:30:19.870 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:19.871+0000 7f8ac4800700 1 -- 172.21.15.153:0/3280868614 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8abc1a1080 con 0x7f8aa406e460 2024-03-21T10:30:19.872 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:19.874+0000 7f8ab6a00700 1 -- 172.21.15.153:0/3280868614 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f8abc1a1080 con 0x7f8aa406e460 2024-03-21T10:30:19.873 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:30:19.873 INFO:teuthology.orchestra.run.smithi153.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232", 2024-03-21T10:30:19.873 INFO:teuthology.orchestra.run.smithi153.stdout: "in_progress": true, 2024-03-21T10:30:19.873 INFO:teuthology.orchestra.run.smithi153.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-21T10:30:19.873 INFO:teuthology.orchestra.run.smithi153.stdout: "services_complete": [ 2024-03-21T10:30:19.873 INFO:teuthology.orchestra.run.smithi153.stdout: "crash", 2024-03-21T10:30:19.873 INFO:teuthology.orchestra.run.smithi153.stdout: "mon", 2024-03-21T10:30:19.873 INFO:teuthology.orchestra.run.smithi153.stdout: "osd", 2024-03-21T10:30:19.873 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr" 2024-03-21T10:30:19.873 INFO:teuthology.orchestra.run.smithi153.stdout: ], 2024-03-21T10:30:19.873 INFO:teuthology.orchestra.run.smithi153.stdout: "progress": "15/23 daemons upgraded", 2024-03-21T10:30:19.873 INFO:teuthology.orchestra.run.smithi153.stdout: "message": "Currently upgrading mds daemons", 2024-03-21T10:30:19.873 INFO:teuthology.orchestra.run.smithi153.stdout: "is_paused": false 2024-03-21T10:30:19.874 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:30:19.876 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:19.877+0000 7f8ac4800700 1 -- 172.21.15.153:0/3280868614 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f8aa406e460 msgr2=0x7f8aa4070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:19.876 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:19.877+0000 7f8ac4800700 1 --2- 172.21.15.153:0/3280868614 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f8aa406e460 0x7f8aa4070920 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f8ab0005fd0 tx=0x7f8ab0005dc0 comp rx=0 tx=0).stop 2024-03-21T10:30:19.876 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:19.877+0000 7f8ac4800700 1 -- 172.21.15.153:0/3280868614 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8abc10d280 msgr2=0x7f8abc19d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:19.876 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:19.877+0000 7f8ac4800700 1 --2- 172.21.15.153:0/3280868614 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8abc10d280 0x7f8abc19d1f0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f8aac00b5c0 tx=0x7f8aac00be00 comp rx=0 tx=0).stop 2024-03-21T10:30:19.876 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:19.877+0000 7f8ac4800700 1 -- 172.21.15.153:0/3280868614 shutdown_connections 2024-03-21T10:30:19.876 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:19.877+0000 7f8ac4800700 1 --2- 172.21.15.153:0/3280868614 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8abc1071d0 0x7f8abc19ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:19.876 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:19.877+0000 7f8ac4800700 1 --2- 172.21.15.153:0/3280868614 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f8aa406e460 0x7f8aa4070920 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:19.876 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:19.877+0000 7f8ac4800700 1 --2- 172.21.15.153:0/3280868614 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8abc10d280 0x7f8abc19d1f0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:19.876 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:19.877+0000 7f8ac4800700 1 -- 172.21.15.153:0/3280868614 >> 172.21.15.153:0/3280868614 conn(0x7f8abc0769e0 msgr2=0x7f8abc078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:19.876 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:19.877+0000 7f8ac4800700 1 -- 172.21.15.153:0/3280868614 shutdown_connections 2024-03-21T10:30:19.876 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:19.877+0000 7f8ac4800700 1 -- 172.21.15.153:0/3280868614 wait complete. 2024-03-21T10:30:19.949 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:30:19 smithi174 ceph-mon[88004]: from='client.35018 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:30:19.950 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:30:19 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/4202109755' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:30:19.950 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:30:19 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/1749694811' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:30:20.020 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:20.021+0000 7f83d6400700 1 -- 172.21.15.153:0/1115388755 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f83d00752c0 msgr2=0x7f83d00756a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:20.020 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:20.021+0000 7f83d6400700 1 --2- 172.21.15.153:0/1115388755 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f83d00752c0 0x7f83d00756a0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f83c0009910 tx=0x7f83c0009c20 comp rx=0 tx=0).stop 2024-03-21T10:30:20.020 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:20.021+0000 7f83d6400700 1 -- 172.21.15.153:0/1115388755 shutdown_connections 2024-03-21T10:30:20.020 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:20.021+0000 7f83d6400700 1 --2- 172.21.15.153:0/1115388755 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f83d0075be0 0x7f83d01114d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:20.020 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:20.021+0000 7f83d6400700 1 --2- 172.21.15.153:0/1115388755 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f83d00752c0 0x7f83d00756a0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:20.021 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:20.021+0000 7f83d6400700 1 -- 172.21.15.153:0/1115388755 >> 172.21.15.153:0/1115388755 conn(0x7f83d00fe800 msgr2=0x7f83d0100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:20.021 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:20.021+0000 7f83d6400700 1 -- 172.21.15.153:0/1115388755 shutdown_connections 2024-03-21T10:30:20.021 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:20.021+0000 7f83d6400700 1 -- 172.21.15.153:0/1115388755 wait complete. 2024-03-21T10:30:20.021 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:20.023+0000 7f83d6400700 1 Processor -- start 2024-03-21T10:30:20.021 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:20.023+0000 7f83d6400700 1 -- start start 2024-03-21T10:30:20.022 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:20.023+0000 7f83d6400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f83d00752c0 0x7f83d01a2a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:20.022 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:20.024+0000 7f83d6400700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f83d0075be0 0x7f83d01a2f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:20.022 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:20.024+0000 7f83d6400700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83d01a3620 con 0x7f83d00752c0 2024-03-21T10:30:20.023 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:20.024+0000 7f83d6400700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83d019cad0 con 0x7f83d0075be0 2024-03-21T10:30:20.023 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:20.024+0000 7f83cf400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f83d00752c0 0x7f83d01a2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:20.023 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:20.024+0000 7f83cf400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f83d00752c0 0x7f83d01a2a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42088/0 (socket says 172.21.15.153:42088) 2024-03-21T10:30:20.023 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:20.024+0000 7f83cf400700 1 -- 172.21.15.153:0/1872442335 learned_addr learned my addr 172.21.15.153:0/1872442335 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:30:20.024 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:20.024+0000 7f83cf400700 1 -- 172.21.15.153:0/1872442335 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f83d0075be0 msgr2=0x7f83d01a2f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:20.024 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:20.024+0000 7f83cf400700 1 --2- 172.21.15.153:0/1872442335 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f83d0075be0 0x7f83d01a2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:20.024 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:20.024+0000 7f83cf400700 1 -- 172.21.15.153:0/1872442335 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83c00095f0 con 0x7f83d00752c0 2024-03-21T10:30:20.024 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:20.024+0000 7f83cf400700 1 --2- 172.21.15.153:0/1872442335 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f83d00752c0 0x7f83d01a2a50 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f83c0000c00 tx=0x7f83c00056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:20.024 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:20.025+0000 7f83c7e00700 1 -- 172.21.15.153:0/1872442335 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f83c001d070 con 0x7f83d00752c0 2024-03-21T10:30:20.024 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:20.025+0000 7f83c7e00700 1 -- 172.21.15.153:0/1872442335 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f83c000f460 con 0x7f83d00752c0 2024-03-21T10:30:20.024 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:20.025+0000 7f83d6400700 1 -- 172.21.15.153:0/1872442335 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83d019cdb0 con 0x7f83d00752c0 2024-03-21T10:30:20.024 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:20.025+0000 7f83c7e00700 1 -- 172.21.15.153:0/1872442335 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f83c0021640 con 0x7f83d00752c0 2024-03-21T10:30:20.024 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:20.025+0000 7f83d6400700 1 -- 172.21.15.153:0/1872442335 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83d019d300 con 0x7f83d00752c0 2024-03-21T10:30:20.025 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:20.026+0000 7f83d6400700 1 -- 172.21.15.153:0/1872442335 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83d010ec40 con 0x7f83d00752c0 2024-03-21T10:30:20.025 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:20.027+0000 7f83c7e00700 1 -- 172.21.15.153:0/1872442335 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f83c000fa60 con 0x7f83d00752c0 2024-03-21T10:30:20.027 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:20.028+0000 7f83c7e00700 1 --2- 172.21.15.153:0/1872442335 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f83b406e460 0x7f83b4070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:20.027 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:20.028+0000 7f83c7e00700 1 -- 172.21.15.153:0/1872442335 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f83c008f6b0 con 0x7f83d00752c0 2024-03-21T10:30:20.027 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:20.028+0000 7f83cea00700 1 --2- 172.21.15.153:0/1872442335 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f83b406e460 0x7f83b4070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:20.027 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:20.029+0000 7f83cea00700 1 --2- 172.21.15.153:0/1872442335 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f83b406e460 0x7f83b4070920 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f83b8005950 tx=0x7f83b80058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:20.031 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:20.033+0000 7f83c7e00700 1 -- 172.21.15.153:0/1872442335 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f83c005c280 con 0x7f83d00752c0 2024-03-21T10:30:20.335 INFO:teuthology.orchestra.run.smithi153.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-03-21T10:30:20.335 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-21T10:30:20.335 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is degraded 2024-03-21T10:30:20.335 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-21T10:30:20.336 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-21T10:30:20.336 INFO:teuthology.orchestra.run.smithi153.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-21T10:30:20.336 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-21T10:30:20.336 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:20.335+0000 7f83d6400700 1 -- 172.21.15.153:0/1872442335 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f83d004ea90 con 0x7f83d00752c0 2024-03-21T10:30:20.336 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:20.336+0000 7f83c7e00700 1 -- 172.21.15.153:0/1872442335 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f83c005be10 con 0x7f83d00752c0 2024-03-21T10:30:20.338 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:20.339+0000 7f83d6400700 1 -- 172.21.15.153:0/1872442335 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f83b406e460 msgr2=0x7f83b4070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:20.338 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:20.339+0000 7f83d6400700 1 --2- 172.21.15.153:0/1872442335 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f83b406e460 0x7f83b4070920 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f83b8005950 tx=0x7f83b80058e0 comp rx=0 tx=0).stop 2024-03-21T10:30:20.338 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:20.339+0000 7f83d6400700 1 -- 172.21.15.153:0/1872442335 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f83d00752c0 msgr2=0x7f83d01a2a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:20.338 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:20.339+0000 7f83d6400700 1 --2- 172.21.15.153:0/1872442335 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f83d00752c0 0x7f83d01a2a50 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f83c0000c00 tx=0x7f83c00056a0 comp rx=0 tx=0).stop 2024-03-21T10:30:20.338 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:20.339+0000 7f83d6400700 1 -- 172.21.15.153:0/1872442335 shutdown_connections 2024-03-21T10:30:20.338 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:20.339+0000 7f83d6400700 1 --2- 172.21.15.153:0/1872442335 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f83b406e460 0x7f83b4070920 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:20.339 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:20.339+0000 7f83d6400700 1 --2- 172.21.15.153:0/1872442335 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f83d00752c0 0x7f83d01a2a50 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:20.339 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:20.339+0000 7f83d6400700 1 --2- 172.21.15.153:0/1872442335 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f83d0075be0 0x7f83d01a2f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:20.339 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:20.339+0000 7f83d6400700 1 -- 172.21.15.153:0/1872442335 >> 172.21.15.153:0/1872442335 conn(0x7f83d00fe800 msgr2=0x7f83d00ff470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:20.339 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:20.339+0000 7f83d6400700 1 -- 172.21.15.153:0/1872442335 shutdown_connections 2024-03-21T10:30:20.339 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:20.340+0000 7f83d6400700 1 -- 172.21.15.153:0/1872442335 wait complete. 2024-03-21T10:30:20.949 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:30:20 smithi174 ceph-mon[88004]: pgmap v554: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:30:20.950 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:30:20 smithi174 ceph-mon[88004]: from='client.35030 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:30:20.950 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:30:20 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/1872442335' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:30:21.054 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:30:20 smithi153 ceph-mon[113733]: pgmap v554: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:30:21.055 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:30:20 smithi153 ceph-mon[113733]: from='client.35030 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:30:21.055 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:30:20 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/1872442335' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:30:22.949 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:30:22 smithi174 ceph-mon[88004]: pgmap v555: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:30:23.054 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:30:22 smithi153 ceph-mon[113733]: pgmap v555: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:30:24.949 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:30:24 smithi174 ceph-mon[88004]: pgmap v556: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:30:25.054 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:30:24 smithi153 ceph-mon[113733]: pgmap v556: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:30:26.949 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:30:26 smithi174 ceph-mon[88004]: pgmap v557: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:30:27.054 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:30:26 smithi153 ceph-mon[113733]: pgmap v557: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:30:28.949 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:30:28 smithi174 ceph-mon[88004]: pgmap v558: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:30:29.058 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:30:28 smithi153 ceph-mon[113733]: pgmap v558: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:30:30.949 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:30:30 smithi174 ceph-mon[88004]: pgmap v559: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:30:31.054 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:30:30 smithi153 ceph-mon[113733]: pgmap v559: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:30:31.949 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:30:31 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:30:32.054 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:30:31 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:30:32.949 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:30:32 smithi174 ceph-mon[88004]: pgmap v560: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:30:33.054 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:30:32 smithi153 ceph-mon[113733]: pgmap v560: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:30:35.054 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:30:34 smithi153 ceph-mon[113733]: pgmap v561: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:30:35.199 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:30:34 smithi174 ceph-mon[88004]: pgmap v561: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:30:37.002 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:30:36 smithi174 ceph-mon[88004]: pgmap v562: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:30:37.054 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:30:36 smithi153 ceph-mon[113733]: pgmap v562: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:30:38.949 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:30:38 smithi174 ceph-mon[88004]: pgmap v563: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:30:39.054 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:30:38 smithi153 ceph-mon[113733]: pgmap v563: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:30:41.054 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:30:40 smithi153 ceph-mon[113733]: pgmap v564: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:30:41.199 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:30:40 smithi174 ceph-mon[88004]: pgmap v564: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:30:43.054 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:30:42 smithi153 ceph-mon[113733]: pgmap v565: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:30:43.199 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:30:42 smithi174 ceph-mon[88004]: pgmap v565: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:30:45.054 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:30:44 smithi153 ceph-mon[113733]: pgmap v566: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:30:45.199 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:30:44 smithi174 ceph-mon[88004]: pgmap v566: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:30:47.055 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:30:46 smithi153 ceph-mon[113733]: pgmap v567: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:30:47.055 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:30:46 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:30:47.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:30:46 smithi174 ceph-mon[88004]: pgmap v567: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:30:47.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:30:46 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:30:49.012 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:30:48 smithi174 ceph-mon[88004]: pgmap v568: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:30:49.054 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:30:48 smithi153 ceph-mon[113733]: pgmap v568: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:30:50.509 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:50.509+0000 7f7936e00700 1 -- 172.21.15.153:0/4091544149 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f793010d280 msgr2=0x7f793010d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:50.509 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:50.509+0000 7f7936e00700 1 --2- 172.21.15.153:0/4091544149 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f793010d280 0x7f793010d660 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f7920009910 tx=0x7f7920009c20 comp rx=0 tx=0).stop 2024-03-21T10:30:50.509 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:50.509+0000 7f7936e00700 1 -- 172.21.15.153:0/4091544149 shutdown_connections 2024-03-21T10:30:50.509 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:50.509+0000 7f7936e00700 1 --2- 172.21.15.153:0/4091544149 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f79301071d0 0x7f7930107650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:50.509 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:50.509+0000 7f7936e00700 1 --2- 172.21.15.153:0/4091544149 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f793010d280 0x7f793010d660 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:50.509 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:50.509+0000 7f7936e00700 1 -- 172.21.15.153:0/4091544149 >> 172.21.15.153:0/4091544149 conn(0x7f79300769e0 msgr2=0x7f7930078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:50.510 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:50.509+0000 7f7936e00700 1 -- 172.21.15.153:0/4091544149 shutdown_connections 2024-03-21T10:30:50.510 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:50.509+0000 7f7936e00700 1 -- 172.21.15.153:0/4091544149 wait complete. 2024-03-21T10:30:50.511 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:50.510+0000 7f7936e00700 1 Processor -- start 2024-03-21T10:30:50.511 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:50.511+0000 7f7936e00700 1 -- start start 2024-03-21T10:30:50.511 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:50.511+0000 7f7936e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f79301071d0 0x7f793019ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:50.511 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:50.511+0000 7f7936e00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f793010d280 0x7f793019d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:50.511 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:50.511+0000 7f7936e00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f793019d8a0 con 0x7f79301071d0 2024-03-21T10:30:50.511 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:50.511+0000 7f7936e00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79301a05d0 con 0x7f793010d280 2024-03-21T10:30:50.512 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:50.511+0000 7f792fe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f79301071d0 0x7f793019ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:50.512 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:50.511+0000 7f792fe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f79301071d0 0x7f793019ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:36758/0 (socket says 172.21.15.153:36758) 2024-03-21T10:30:50.512 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:50.511+0000 7f792fe00700 1 -- 172.21.15.153:0/4156466330 learned_addr learned my addr 172.21.15.153:0/4156466330 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:30:50.512 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:50.512+0000 7f7927e00700 1 --2- 172.21.15.153:0/4156466330 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f793010d280 0x7f793019d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:50.512 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:50.512+0000 7f792fe00700 1 -- 172.21.15.153:0/4156466330 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f793010d280 msgr2=0x7f793019d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:50.512 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:50.512+0000 7f792fe00700 1 --2- 172.21.15.153:0/4156466330 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f793010d280 0x7f793019d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:50.512 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:50.512+0000 7f792fe00700 1 -- 172.21.15.153:0/4156466330 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79200095f0 con 0x7f79301071d0 2024-03-21T10:30:50.513 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:50.512+0000 7f792fe00700 1 --2- 172.21.15.153:0/4156466330 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f79301071d0 0x7f793019ccd0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f7920006010 tx=0x7f792000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:50.513 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:50.512+0000 7f792d600700 1 -- 172.21.15.153:0/4156466330 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f792001d070 con 0x7f79301071d0 2024-03-21T10:30:50.513 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:50.513+0000 7f792d600700 1 -- 172.21.15.153:0/4156466330 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f792000f460 con 0x7f79301071d0 2024-03-21T10:30:50.513 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:50.513+0000 7f7936e00700 1 -- 172.21.15.153:0/4156466330 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79301a0850 con 0x7f79301071d0 2024-03-21T10:30:50.513 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:50.513+0000 7f792d600700 1 -- 172.21.15.153:0/4156466330 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7920017630 con 0x7f79301071d0 2024-03-21T10:30:50.513 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:50.513+0000 7f7936e00700 1 -- 172.21.15.153:0/4156466330 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79301a0d40 con 0x7f79301071d0 2024-03-21T10:30:50.513 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:50.514+0000 7f7936e00700 1 -- 172.21.15.153:0/4156466330 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f793004ea90 con 0x7f79301071d0 2024-03-21T10:30:50.514 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:50.515+0000 7f792d600700 1 -- 172.21.15.153:0/4156466330 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f792000f5d0 con 0x7f79301071d0 2024-03-21T10:30:50.516 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:50.516+0000 7f792d600700 1 --2- 172.21.15.153:0/4156466330 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f791406e460 0x7f7914070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:50.516 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:50.517+0000 7f792d600700 1 -- 172.21.15.153:0/4156466330 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f792008f9a0 con 0x7f79301071d0 2024-03-21T10:30:50.517 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:50.517+0000 7f7927e00700 1 --2- 172.21.15.153:0/4156466330 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f791406e460 0x7f7914070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:50.517 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:50.517+0000 7f7927e00700 1 --2- 172.21.15.153:0/4156466330 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f791406e460 0x7f7914070920 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f793019e2b0 tx=0x7f791800b5b0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:50.521 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:50.522+0000 7f792d600700 1 -- 172.21.15.153:0/4156466330 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7920058a40 con 0x7f79301071d0 2024-03-21T10:30:50.766 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:50.766+0000 7f7936e00700 1 -- 172.21.15.153:0/4156466330 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f79301a10d0 con 0x7f791406e460 2024-03-21T10:30:50.766 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:30:50 smithi153 ceph-mon[113733]: pgmap v569: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:30:50.767 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:50.768+0000 7f792d600700 1 -- 172.21.15.153:0/4156466330 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f79301a10d0 con 0x7f791406e460 2024-03-21T10:30:50.770 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:50.771+0000 7f7936e00700 1 -- 172.21.15.153:0/4156466330 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f791406e460 msgr2=0x7f7914070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:50.770 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:50.771+0000 7f7936e00700 1 --2- 172.21.15.153:0/4156466330 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f791406e460 0x7f7914070920 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f793019e2b0 tx=0x7f791800b5b0 comp rx=0 tx=0).stop 2024-03-21T10:30:50.771 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:50.771+0000 7f7936e00700 1 -- 172.21.15.153:0/4156466330 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f79301071d0 msgr2=0x7f793019ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:50.771 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:50.771+0000 7f7936e00700 1 --2- 172.21.15.153:0/4156466330 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f79301071d0 0x7f793019ccd0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f7920006010 tx=0x7f792000be20 comp rx=0 tx=0).stop 2024-03-21T10:30:50.771 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:50.771+0000 7f7936e00700 1 -- 172.21.15.153:0/4156466330 shutdown_connections 2024-03-21T10:30:50.771 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:50.771+0000 7f7936e00700 1 --2- 172.21.15.153:0/4156466330 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f791406e460 0x7f7914070920 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:50.771 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:50.771+0000 7f7936e00700 1 --2- 172.21.15.153:0/4156466330 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f79301071d0 0x7f793019ccd0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:50.771 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:50.771+0000 7f7936e00700 1 --2- 172.21.15.153:0/4156466330 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f793010d280 0x7f793019d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:50.771 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:50.771+0000 7f7936e00700 1 -- 172.21.15.153:0/4156466330 >> 172.21.15.153:0/4156466330 conn(0x7f79300769e0 msgr2=0x7f7930078320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:50.771 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:50.771+0000 7f7936e00700 1 -- 172.21.15.153:0/4156466330 shutdown_connections 2024-03-21T10:30:50.771 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:50.771+0000 7f7936e00700 1 -- 172.21.15.153:0/4156466330 wait complete. 2024-03-21T10:30:50.787 INFO:teuthology.orchestra.run.smithi153.stdout:true 2024-03-21T10:30:50.932 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:50.932+0000 7fd420800700 1 -- 172.21.15.153:0/2583628110 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd418068670 msgr2=0x7fd418068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:50.933 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:50.932+0000 7fd420800700 1 --2- 172.21.15.153:0/2583628110 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd418068670 0x7fd418068a50 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fd40c009910 tx=0x7fd40c009c20 comp rx=0 tx=0).stop 2024-03-21T10:30:50.933 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:50.933+0000 7fd420800700 1 -- 172.21.15.153:0/2583628110 shutdown_connections 2024-03-21T10:30:50.933 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:50.933+0000 7fd420800700 1 --2- 172.21.15.153:0/2583628110 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd418069020 0x7fd418105790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:50.933 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:50.933+0000 7fd420800700 1 --2- 172.21.15.153:0/2583628110 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd418068670 0x7fd418068a50 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:50.933 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:50.933+0000 7fd420800700 1 -- 172.21.15.153:0/2583628110 >> 172.21.15.153:0/2583628110 conn(0x7fd4180755a0 msgr2=0x7fd4180759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:50.933 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:50.933+0000 7fd420800700 1 -- 172.21.15.153:0/2583628110 shutdown_connections 2024-03-21T10:30:50.933 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:50.933+0000 7fd420800700 1 -- 172.21.15.153:0/2583628110 wait complete. 2024-03-21T10:30:50.933 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:50.934+0000 7fd420800700 1 Processor -- start 2024-03-21T10:30:50.934 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:50.934+0000 7fd420800700 1 -- start start 2024-03-21T10:30:50.934 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:50.935+0000 7fd420800700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd418068670 0x7fd418196740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:50.934 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:50.935+0000 7fd420800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd418069020 0x7fd418196c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:50.934 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:50.935+0000 7fd420800700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd418197310 con 0x7fd418069020 2024-03-21T10:30:50.934 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:50.935+0000 7fd420800700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd418199fd0 con 0x7fd418068670 2024-03-21T10:30:50.935 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:50.935+0000 7fd41d400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd418069020 0x7fd418196c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:50.935 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:50.935+0000 7fd41d400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd418069020 0x7fd418196c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:36776/0 (socket says 172.21.15.153:36776) 2024-03-21T10:30:50.935 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:50.935+0000 7fd41d400700 1 -- 172.21.15.153:0/1954683085 learned_addr learned my addr 172.21.15.153:0/1954683085 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:30:50.935 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:50.936+0000 7fd41de00700 1 --2- 172.21.15.153:0/1954683085 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd418068670 0x7fd418196740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:50.935 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:50.936+0000 7fd41d400700 1 -- 172.21.15.153:0/1954683085 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd418068670 msgr2=0x7fd418196740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:50.936 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:50.936+0000 7fd41d400700 1 --2- 172.21.15.153:0/1954683085 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd418068670 0x7fd418196740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:50.936 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:50.936+0000 7fd41d400700 1 -- 172.21.15.153:0/1954683085 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd40c0095f0 con 0x7fd418069020 2024-03-21T10:30:50.936 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:50.936+0000 7fd41d400700 1 --2- 172.21.15.153:0/1954683085 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd418069020 0x7fd418196c80 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fd40800d900 tx=0x7fd40800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:50.936 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:50.936+0000 7fd412a00700 1 -- 172.21.15.153:0/1954683085 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd408009900 con 0x7fd418069020 2024-03-21T10:30:50.936 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:50.937+0000 7fd412a00700 1 -- 172.21.15.153:0/1954683085 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd408010460 con 0x7fd418069020 2024-03-21T10:30:50.936 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:50.937+0000 7fd412a00700 1 -- 172.21.15.153:0/1954683085 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd40800f5f0 con 0x7fd418069020 2024-03-21T10:30:50.937 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:50.937+0000 7fd420800700 1 -- 172.21.15.153:0/1954683085 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd41819a2b0 con 0x7fd418069020 2024-03-21T10:30:50.937 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:50.937+0000 7fd420800700 1 -- 172.21.15.153:0/1954683085 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd41819a800 con 0x7fd418069020 2024-03-21T10:30:50.937 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:50.938+0000 7fd420800700 1 -- 172.21.15.153:0/1954683085 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd418109130 con 0x7fd418069020 2024-03-21T10:30:50.939 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:50.939+0000 7fd412a00700 1 -- 172.21.15.153:0/1954683085 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fd4080105d0 con 0x7fd418069020 2024-03-21T10:30:50.941 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:50.941+0000 7fd412a00700 1 --2- 172.21.15.153:0/1954683085 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd40006e460 0x7fd400070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:50.941 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:50.941+0000 7fd412a00700 1 -- 172.21.15.153:0/1954683085 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fd40808e360 con 0x7fd418069020 2024-03-21T10:30:50.941 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:50.941+0000 7fd41de00700 1 --2- 172.21.15.153:0/1954683085 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd40006e460 0x7fd400070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:50.941 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:50.942+0000 7fd41de00700 1 --2- 172.21.15.153:0/1954683085 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd40006e460 0x7fd400070920 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd40c00b5c0 tx=0x7fd40c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:50.944 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:50.945+0000 7fd412a00700 1 -- 172.21.15.153:0/1954683085 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd408057400 con 0x7fd418069020 2024-03-21T10:30:51.185 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:51.186+0000 7fd420800700 1 -- 172.21.15.153:0/1954683085 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd41819aae0 con 0x7fd40006e460 2024-03-21T10:30:51.188 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:51.189+0000 7fd412a00700 1 -- 172.21.15.153:0/1954683085 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fd41819aae0 con 0x7fd40006e460 2024-03-21T10:30:51.191 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:51.191+0000 7fd420800700 1 -- 172.21.15.153:0/1954683085 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd40006e460 msgr2=0x7fd400070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:51.191 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:51.191+0000 7fd420800700 1 --2- 172.21.15.153:0/1954683085 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd40006e460 0x7fd400070920 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd40c00b5c0 tx=0x7fd40c01a040 comp rx=0 tx=0).stop 2024-03-21T10:30:51.191 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:51.191+0000 7fd420800700 1 -- 172.21.15.153:0/1954683085 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd418069020 msgr2=0x7fd418196c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:51.191 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:51.191+0000 7fd420800700 1 --2- 172.21.15.153:0/1954683085 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd418069020 0x7fd418196c80 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fd40800d900 tx=0x7fd40800dcc0 comp rx=0 tx=0).stop 2024-03-21T10:30:51.192 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:51.191+0000 7fd420800700 1 -- 172.21.15.153:0/1954683085 shutdown_connections 2024-03-21T10:30:51.192 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:51.191+0000 7fd420800700 1 --2- 172.21.15.153:0/1954683085 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd418068670 0x7fd418196740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:51.192 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:51.191+0000 7fd420800700 1 --2- 172.21.15.153:0/1954683085 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd40006e460 0x7fd400070920 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:51.192 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:51.191+0000 7fd420800700 1 --2- 172.21.15.153:0/1954683085 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd418069020 0x7fd418196c80 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:51.192 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:51.191+0000 7fd420800700 1 -- 172.21.15.153:0/1954683085 >> 172.21.15.153:0/1954683085 conn(0x7fd4180755a0 msgr2=0x7fd4180fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:51.192 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:51.192+0000 7fd420800700 1 -- 172.21.15.153:0/1954683085 shutdown_connections 2024-03-21T10:30:51.192 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:51.192+0000 7fd420800700 1 -- 172.21.15.153:0/1954683085 wait complete. 2024-03-21T10:30:51.199 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:30:50 smithi174 ceph-mon[88004]: pgmap v569: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:30:51.355 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:51.356+0000 7f975dc00700 1 -- 172.21.15.153:0/2559982710 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9758073a80 msgr2=0x7f975810cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:51.356 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:51.356+0000 7f975dc00700 1 --2- 172.21.15.153:0/2559982710 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9758073a80 0x7f975810cec0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f9740009910 tx=0x7f9740009c20 comp rx=0 tx=0).stop 2024-03-21T10:30:51.356 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:51.356+0000 7f975dc00700 1 -- 172.21.15.153:0/2559982710 shutdown_connections 2024-03-21T10:30:51.356 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:51.356+0000 7f975dc00700 1 --2- 172.21.15.153:0/2559982710 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9758073a80 0x7f975810cec0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:51.356 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:51.356+0000 7f975dc00700 1 --2- 172.21.15.153:0/2559982710 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9758073160 0x7f9758073540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:51.356 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:51.356+0000 7f975dc00700 1 -- 172.21.15.153:0/2559982710 >> 172.21.15.153:0/2559982710 conn(0x7f97580fc650 msgr2=0x7f97580fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:51.356 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:51.356+0000 7f975dc00700 1 -- 172.21.15.153:0/2559982710 shutdown_connections 2024-03-21T10:30:51.356 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:51.356+0000 7f975dc00700 1 -- 172.21.15.153:0/2559982710 wait complete. 2024-03-21T10:30:51.357 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:51.357+0000 7f975dc00700 1 Processor -- start 2024-03-21T10:30:51.357 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:51.357+0000 7f975dc00700 1 -- start start 2024-03-21T10:30:51.357 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:51.357+0000 7f975dc00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9758073160 0x7f9758198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:51.357 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:51.357+0000 7f975dc00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9758073a80 0x7f9758198eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:51.357 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:51.357+0000 7f975dc00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97581994b0 con 0x7f9758073a80 2024-03-21T10:30:51.357 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:51.357+0000 7f975dc00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9758199620 con 0x7f9758073160 2024-03-21T10:30:51.357 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:51.358+0000 7f974e000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9758073a80 0x7f9758198eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:51.358 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:51.358+0000 7f974e000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9758073a80 0x7f9758198eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:36788/0 (socket says 172.21.15.153:36788) 2024-03-21T10:30:51.358 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:51.358+0000 7f974e000700 1 -- 172.21.15.153:0/2820889759 learned_addr learned my addr 172.21.15.153:0/2820889759 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:30:51.358 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:51.358+0000 7f9756a00700 1 --2- 172.21.15.153:0/2820889759 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9758073160 0x7f9758198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:51.358 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:51.358+0000 7f974e000700 1 -- 172.21.15.153:0/2820889759 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9758073160 msgr2=0x7f9758198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:51.359 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:51.358+0000 7f974e000700 1 --2- 172.21.15.153:0/2820889759 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9758073160 0x7f9758198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:51.359 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:51.358+0000 7f974e000700 1 -- 172.21.15.153:0/2820889759 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97400095f0 con 0x7f9758073a80 2024-03-21T10:30:51.359 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:51.358+0000 7f9756a00700 1 --2- 172.21.15.153:0/2820889759 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9758073160 0x7f9758198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:30:51.359 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:51.359+0000 7f974e000700 1 --2- 172.21.15.153:0/2820889759 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9758073a80 0x7f9758198eb0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f9740006010 tx=0x7f974000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:51.359 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:51.359+0000 7f974fe00700 1 -- 172.21.15.153:0/2820889759 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f974001d070 con 0x7f9758073a80 2024-03-21T10:30:51.359 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:51.359+0000 7f974fe00700 1 -- 172.21.15.153:0/2820889759 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9740004fb0 con 0x7f9758073a80 2024-03-21T10:30:51.359 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:51.359+0000 7f974fe00700 1 -- 172.21.15.153:0/2820889759 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f974000f690 con 0x7f9758073a80 2024-03-21T10:30:51.359 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:51.359+0000 7f975dc00700 1 -- 172.21.15.153:0/2820889759 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f975819d440 con 0x7f9758073a80 2024-03-21T10:30:51.359 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:51.359+0000 7f975dc00700 1 -- 172.21.15.153:0/2820889759 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f975819d930 con 0x7f9758073a80 2024-03-21T10:30:51.359 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:51.360+0000 7f975dc00700 1 -- 172.21.15.153:0/2820889759 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f975810a5c0 con 0x7f9758073a80 2024-03-21T10:30:51.360 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:51.361+0000 7f974fe00700 1 -- 172.21.15.153:0/2820889759 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f9740022470 con 0x7f9758073a80 2024-03-21T10:30:51.362 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:51.362+0000 7f974fe00700 1 --2- 172.21.15.153:0/2820889759 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f973806e460 0x7f9738070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:51.362 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:51.362+0000 7f974fe00700 1 -- 172.21.15.153:0/2820889759 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f974008fb70 con 0x7f9758073a80 2024-03-21T10:30:51.362 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:51.362+0000 7f9756a00700 1 --2- 172.21.15.153:0/2820889759 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f973806e460 0x7f9738070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:51.362 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:51.363+0000 7f9756a00700 1 --2- 172.21.15.153:0/2820889759 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f973806e460 0x7f9738070920 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f974800d340 tx=0x7f9748009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:51.366 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:51.367+0000 7f974fe00700 1 -- 172.21.15.153:0/2820889759 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f974005c7e0 con 0x7f9758073a80 2024-03-21T10:30:51.586 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:51.586+0000 7f975dc00700 1 -- 172.21.15.153:0/2820889759 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f975819dc60 con 0x7f973806e460 2024-03-21T10:30:51.598 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:51.599+0000 7f974fe00700 1 -- 172.21.15.153:0/2820889759 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f975819dc60 con 0x7f973806e460 2024-03-21T10:30:51.599 INFO:teuthology.orchestra.run.smithi153.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-21T10:30:51.599 INFO:teuthology.orchestra.run.smithi153.stdout:alertmanager.smithi153 smithi153 *:9093,9094 running (19m) 11m ago 33m 14.7M - 0.25.0 c8568f914cd2 9ed3f2200b5f 2024-03-21T10:30:51.599 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi153 smithi153 running (33m) 11m ago 33m 9424k - 18.2.1 5be31c24972a 5b61994396a6 2024-03-21T10:30:51.599 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi174 smithi174 running (32m) 10m ago 32m 9244k - 18.2.1 5be31c24972a 4f7a5fc3ac28 2024-03-21T10:30:51.599 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi153 smithi153 running (15m) 11m ago 33m 6656k - 19.0.0-2231-g2edfee68 237e43c7a65f 32c21104d5b0 2024-03-21T10:30:51.599 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi174 smithi174 running (15m) 10m ago 32m 6660k - 19.0.0-2231-g2edfee68 237e43c7a65f a04e48967ab6 2024-03-21T10:30:51.599 INFO:teuthology.orchestra.run.smithi153.stdout:grafana.smithi153 smithi153 *:3000 running (19m) 11m ago 32m 73.6M - 9.4.12 2bacad6d85d8 ee24026ab786 2024-03-21T10:30:51.599 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.iuyluj smithi153 running (11m) 11m ago 26m 10.9M - 19.0.0-2231-g2edfee68 237e43c7a65f 87aff8a5de70 2024-03-21T10:30:51.599 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.zipckr smithi153 running (11m) 11m ago 26m 11.8M - 19.0.0-2231-g2edfee68 237e43c7a65f ccc262bd5dd3 2024-03-21T10:30:51.599 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.nhozag smithi174 running (26m) 10m ago 26m 12.0M - 18.2.1 5be31c24972a 2fee4000308f 2024-03-21T10:30:51.600 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.yyimst smithi174 running (10m) 10m ago 26m 10.7M - 19.0.0-2231-g2edfee68 237e43c7a65f fc1962f9351e 2024-03-21T10:30:51.600 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi153.tluxni smithi153 *:8443,9283,8765 running (22m) 11m ago 35m 509M - 19.0.0-2231-g2edfee68 237e43c7a65f 7ffee0e55382 2024-03-21T10:30:51.600 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi174.qpcwat smithi174 *:8443,9283,8765 running (21m) 10m ago 31m 452M - 19.0.0-2231-g2edfee68 237e43c7a65f 748968f216ab 2024-03-21T10:30:51.600 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi153 smithi153 running (17m) 11m ago 35m 55.0M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f dd8038303dc3 2024-03-21T10:30:51.600 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi174 smithi174 running (16m) 10m ago 31m 44.2M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f 29d78a788f2c 2024-03-21T10:30:51.600 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi153 smithi153 *:9100 running (20m) 11m ago 33m 7889k - 1.5.0 0da6a335fe13 bb835edc0e0b 2024-03-21T10:30:51.600 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi174 smithi174 *:9100 running (20m) 10m ago 31m 8102k - 1.5.0 0da6a335fe13 7c4ecad790fa 2024-03-21T10:30:51.600 INFO:teuthology.orchestra.run.smithi153.stdout:osd.0 smithi153 running (15m) 11m ago 30m 116M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 507900fd4791 2024-03-21T10:30:51.600 INFO:teuthology.orchestra.run.smithi153.stdout:osd.1 smithi153 running (14m) 11m ago 30m 89.2M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 0d646452e9c1 2024-03-21T10:30:51.600 INFO:teuthology.orchestra.run.smithi153.stdout:osd.2 smithi153 running (13m) 11m ago 29m 87.4M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 1326f2afdfcd 2024-03-21T10:30:51.600 INFO:teuthology.orchestra.run.smithi153.stdout:osd.3 smithi174 running (13m) 10m ago 29m 83.6M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 70aafac71ad5 2024-03-21T10:30:51.600 INFO:teuthology.orchestra.run.smithi153.stdout:osd.4 smithi174 running (12m) 10m ago 28m 92.2M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 0d746ce6e79d 2024-03-21T10:30:51.600 INFO:teuthology.orchestra.run.smithi153.stdout:osd.5 smithi174 running (12m) 10m ago 28m 86.2M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 53e5f35b8cac 2024-03-21T10:30:51.600 INFO:teuthology.orchestra.run.smithi153.stdout:prometheus.smithi153 smithi153 *:9095 running (20m) 11m ago 32m 73.5M - 2.43.0 a07b618ecd1d 7d95561e50c2 2024-03-21T10:30:51.601 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:51.602+0000 7f975dc00700 1 -- 172.21.15.153:0/2820889759 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f973806e460 msgr2=0x7f9738070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:51.601 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:51.602+0000 7f975dc00700 1 --2- 172.21.15.153:0/2820889759 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f973806e460 0x7f9738070920 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f974800d340 tx=0x7f9748009040 comp rx=0 tx=0).stop 2024-03-21T10:30:51.601 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:51.602+0000 7f975dc00700 1 -- 172.21.15.153:0/2820889759 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9758073a80 msgr2=0x7f9758198eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:51.601 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:51.602+0000 7f975dc00700 1 --2- 172.21.15.153:0/2820889759 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9758073a80 0x7f9758198eb0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f9740006010 tx=0x7f974000be20 comp rx=0 tx=0).stop 2024-03-21T10:30:51.602 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:51.602+0000 7f975dc00700 1 -- 172.21.15.153:0/2820889759 shutdown_connections 2024-03-21T10:30:51.602 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:51.602+0000 7f975dc00700 1 --2- 172.21.15.153:0/2820889759 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9758073160 0x7f9758198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:51.602 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:51.602+0000 7f975dc00700 1 --2- 172.21.15.153:0/2820889759 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f973806e460 0x7f9738070920 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:51.602 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:51.602+0000 7f975dc00700 1 --2- 172.21.15.153:0/2820889759 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9758073a80 0x7f9758198eb0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:51.603 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:51.602+0000 7f975dc00700 1 -- 172.21.15.153:0/2820889759 >> 172.21.15.153:0/2820889759 conn(0x7f97580fc650 msgr2=0x7f9758107700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:51.603 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:51.602+0000 7f975dc00700 1 -- 172.21.15.153:0/2820889759 shutdown_connections 2024-03-21T10:30:51.603 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:51.602+0000 7f975dc00700 1 -- 172.21.15.153:0/2820889759 wait complete. 2024-03-21T10:30:51.746 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:51.747+0000 7f1946c00700 1 -- 172.21.15.153:0/943298082 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f194010d280 msgr2=0x7f194010d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:51.747 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:51.747+0000 7f1946c00700 1 --2- 172.21.15.153:0/943298082 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f194010d280 0x7f194010d660 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f1930009910 tx=0x7f1930009c20 comp rx=0 tx=0).stop 2024-03-21T10:30:51.747 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:51.747+0000 7f1946c00700 1 -- 172.21.15.153:0/943298082 shutdown_connections 2024-03-21T10:30:51.747 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:51.747+0000 7f1946c00700 1 --2- 172.21.15.153:0/943298082 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f19401071d0 0x7f1940107650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:51.747 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:51.747+0000 7f1946c00700 1 --2- 172.21.15.153:0/943298082 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f194010d280 0x7f194010d660 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:51.747 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:51.747+0000 7f1946c00700 1 -- 172.21.15.153:0/943298082 >> 172.21.15.153:0/943298082 conn(0x7f19400769e0 msgr2=0x7f1940078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:51.747 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:51.747+0000 7f1946c00700 1 -- 172.21.15.153:0/943298082 shutdown_connections 2024-03-21T10:30:51.747 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:51.747+0000 7f1946c00700 1 -- 172.21.15.153:0/943298082 wait complete. 2024-03-21T10:30:51.749 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:51.748+0000 7f1946c00700 1 Processor -- start 2024-03-21T10:30:51.749 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:51.748+0000 7f1946c00700 1 -- start start 2024-03-21T10:30:51.749 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:51.748+0000 7f1946c00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f19401071d0 0x7f194019ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:51.749 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:51.748+0000 7f1946c00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f194010d280 0x7f194019d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:51.749 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:51.748+0000 7f1946c00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f194019d8a0 con 0x7f194010d280 2024-03-21T10:30:51.750 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:51.748+0000 7f1946c00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19401a05d0 con 0x7f19401071d0 2024-03-21T10:30:51.750 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:51.749+0000 7f193f400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f194010d280 0x7f194019d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:51.750 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:51.749+0000 7f193f400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f194010d280 0x7f194019d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:36802/0 (socket says 172.21.15.153:36802) 2024-03-21T10:30:51.750 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:51.749+0000 7f193f400700 1 -- 172.21.15.153:0/1855229016 learned_addr learned my addr 172.21.15.153:0/1855229016 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:30:51.750 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:51.749+0000 7f193fe00700 1 --2- 172.21.15.153:0/1855229016 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f19401071d0 0x7f194019ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:51.750 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:51.749+0000 7f193f400700 1 -- 172.21.15.153:0/1855229016 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f19401071d0 msgr2=0x7f194019ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:51.750 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:51.749+0000 7f193f400700 1 --2- 172.21.15.153:0/1855229016 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f19401071d0 0x7f194019ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:51.750 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:51.749+0000 7f193f400700 1 -- 172.21.15.153:0/1855229016 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19300095f0 con 0x7f194010d280 2024-03-21T10:30:51.750 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:51.749+0000 7f193f400700 1 --2- 172.21.15.153:0/1855229016 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f194010d280 0x7f194019d210 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f192c00d8d0 tx=0x7f192c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:51.750 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:51.750+0000 7f193cc00700 1 -- 172.21.15.153:0/1855229016 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f192c009940 con 0x7f194010d280 2024-03-21T10:30:51.750 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:51.750+0000 7f193cc00700 1 -- 172.21.15.153:0/1855229016 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f192c010460 con 0x7f194010d280 2024-03-21T10:30:51.750 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:51.750+0000 7f1946c00700 1 -- 172.21.15.153:0/1855229016 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19401a08b0 con 0x7f194010d280 2024-03-21T10:30:51.750 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:51.750+0000 7f193cc00700 1 -- 172.21.15.153:0/1855229016 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f192c00f5f0 con 0x7f194010d280 2024-03-21T10:30:51.750 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:51.750+0000 7f1946c00700 1 -- 172.21.15.153:0/1855229016 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19401a0e80 con 0x7f194010d280 2024-03-21T10:30:51.751 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:51.751+0000 7f1946c00700 1 -- 172.21.15.153:0/1855229016 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f194004ea90 con 0x7f194010d280 2024-03-21T10:30:51.751 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:51.752+0000 7f193cc00700 1 -- 172.21.15.153:0/1855229016 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f192c0105d0 con 0x7f194010d280 2024-03-21T10:30:51.752 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:51.753+0000 7f193cc00700 1 --2- 172.21.15.153:0/1855229016 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f192406e460 0x7f1924070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:51.752 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:51.753+0000 7f193cc00700 1 -- 172.21.15.153:0/1855229016 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f192c08e480 con 0x7f194010d280 2024-03-21T10:30:51.752 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:51.753+0000 7f193fe00700 1 --2- 172.21.15.153:0/1855229016 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f192406e460 0x7f1924070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:51.753 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:51.754+0000 7f193fe00700 1 --2- 172.21.15.153:0/1855229016 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f192406e460 0x7f1924070920 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f1930006010 tx=0x7f193001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:51.757 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:51.758+0000 7f193cc00700 1 -- 172.21.15.153:0/1855229016 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f192c057520 con 0x7f194010d280 2024-03-21T10:30:52.046 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:52.046+0000 7f1946c00700 1 -- 172.21.15.153:0/1855229016 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f19401a11f0 con 0x7f194010d280 2024-03-21T10:30:52.046 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:30:51 smithi153 ceph-mon[113733]: from='client.35038 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:30:52.046 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:30:51 smithi153 ceph-mon[113733]: from='client.35042 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:30:52.046 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:52.047+0000 7f193cc00700 1 -- 172.21.15.153:0/1855229016 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+805 (secure 0 0 0) 0x7f192c05abe0 con 0x7f194010d280 2024-03-21T10:30:52.047 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:30:52.047 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": { 2024-03-21T10:30:52.047 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:30:52.047 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:30:52.047 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": { 2024-03-21T10:30:52.047 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:30:52.047 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:30:52.047 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": { 2024-03-21T10:30:52.047 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 6 2024-03-21T10:30:52.047 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:30:52.048 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": { 2024-03-21T10:30:52.048 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-03-21T10:30:52.048 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 3 2024-03-21T10:30:52.048 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:30:52.048 INFO:teuthology.orchestra.run.smithi153.stdout: "overall": { 2024-03-21T10:30:52.048 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-03-21T10:30:52.048 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 13 2024-03-21T10:30:52.048 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-03-21T10:30:52.048 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:30:52.050 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:52.051+0000 7f1946c00700 1 -- 172.21.15.153:0/1855229016 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f192406e460 msgr2=0x7f1924070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:52.050 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:52.051+0000 7f1946c00700 1 --2- 172.21.15.153:0/1855229016 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f192406e460 0x7f1924070920 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f1930006010 tx=0x7f193001a040 comp rx=0 tx=0).stop 2024-03-21T10:30:52.050 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:52.051+0000 7f1946c00700 1 -- 172.21.15.153:0/1855229016 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f194010d280 msgr2=0x7f194019d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:52.050 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:52.051+0000 7f1946c00700 1 --2- 172.21.15.153:0/1855229016 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f194010d280 0x7f194019d210 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f192c00d8d0 tx=0x7f192c00dc90 comp rx=0 tx=0).stop 2024-03-21T10:30:52.051 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:52.051+0000 7f1946c00700 1 -- 172.21.15.153:0/1855229016 shutdown_connections 2024-03-21T10:30:52.051 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:52.051+0000 7f1946c00700 1 --2- 172.21.15.153:0/1855229016 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f19401071d0 0x7f194019ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:52.051 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:52.051+0000 7f1946c00700 1 --2- 172.21.15.153:0/1855229016 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f192406e460 0x7f1924070920 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:52.051 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:52.051+0000 7f1946c00700 1 --2- 172.21.15.153:0/1855229016 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f194010d280 0x7f194019d210 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:52.051 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:52.051+0000 7f1946c00700 1 -- 172.21.15.153:0/1855229016 >> 172.21.15.153:0/1855229016 conn(0x7f19400769e0 msgr2=0x7f1940078320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:52.051 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:52.051+0000 7f1946c00700 1 -- 172.21.15.153:0/1855229016 shutdown_connections 2024-03-21T10:30:52.051 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:52.051+0000 7f1946c00700 1 -- 172.21.15.153:0/1855229016 wait complete. 2024-03-21T10:30:52.192 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:52.192+0000 7fa4a1400700 1 -- 172.21.15.153:0/3568150277 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa4900054b0 con 0x7fa4a41071d0 2024-03-21T10:30:52.193 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:52.193+0000 7fa4a9400700 1 -- 172.21.15.153:0/3568150277 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa4a41071d0 msgr2=0x7fa4a4107650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:52.193 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:52.193+0000 7fa4a9400700 1 --2- 172.21.15.153:0/3568150277 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa4a41071d0 0x7fa4a4107650 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fa490009910 tx=0x7fa490009c20 comp rx=0 tx=0).stop 2024-03-21T10:30:52.193 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:52.193+0000 7fa4a9400700 1 -- 172.21.15.153:0/3568150277 shutdown_connections 2024-03-21T10:30:52.193 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:52.193+0000 7fa4a9400700 1 --2- 172.21.15.153:0/3568150277 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa4a41071d0 0x7fa4a4107650 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:52.193 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:52.193+0000 7fa4a9400700 1 --2- 172.21.15.153:0/3568150277 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa4a410d280 0x7fa4a410d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:52.193 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:52.193+0000 7fa4a9400700 1 -- 172.21.15.153:0/3568150277 >> 172.21.15.153:0/3568150277 conn(0x7fa4a40769e0 msgr2=0x7fa4a4078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:52.193 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:52.193+0000 7fa4a9400700 1 -- 172.21.15.153:0/3568150277 shutdown_connections 2024-03-21T10:30:52.193 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:52.193+0000 7fa4a9400700 1 -- 172.21.15.153:0/3568150277 wait complete. 2024-03-21T10:30:52.194 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:52.194+0000 7fa4a9400700 1 Processor -- start 2024-03-21T10:30:52.194 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:52.194+0000 7fa4a9400700 1 -- start start 2024-03-21T10:30:52.194 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:52.195+0000 7fa4a9400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa4a41071d0 0x7fa4a419ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:52.194 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:52.195+0000 7fa4a9400700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa4a410d280 0x7fa4a419d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:52.194 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:52.195+0000 7fa4a9400700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a419d880 con 0x7fa4a41071d0 2024-03-21T10:30:52.195 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:52.195+0000 7fa4a9400700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a41a05b0 con 0x7fa4a410d280 2024-03-21T10:30:52.195 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:52.195+0000 7fa4a2800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa4a41071d0 0x7fa4a419ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:52.195 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:52.195+0000 7fa4a2800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa4a41071d0 0x7fa4a419ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:36822/0 (socket says 172.21.15.153:36822) 2024-03-21T10:30:52.195 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:52.195+0000 7fa4a2800700 1 -- 172.21.15.153:0/3442839133 learned_addr learned my addr 172.21.15.153:0/3442839133 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:30:52.196 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:52.195+0000 7fa4a2800700 1 -- 172.21.15.153:0/3442839133 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa4a410d280 msgr2=0x7fa4a419d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:52.196 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:52.195+0000 7fa4a1e00700 1 --2- 172.21.15.153:0/3442839133 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa4a410d280 0x7fa4a419d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:52.196 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:52.195+0000 7fa4a2800700 1 --2- 172.21.15.153:0/3442839133 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa4a410d280 0x7fa4a419d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:52.196 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:52.195+0000 7fa4a2800700 1 -- 172.21.15.153:0/3442839133 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4900095f0 con 0x7fa4a41071d0 2024-03-21T10:30:52.196 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:52.196+0000 7fa4a2800700 1 --2- 172.21.15.153:0/3442839133 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa4a41071d0 0x7fa4a419ccb0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fa4940060b0 tx=0x7fa49400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:52.196 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:52.196+0000 7fa49b400700 1 -- 172.21.15.153:0/3442839133 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa49400dda0 con 0x7fa4a41071d0 2024-03-21T10:30:52.196 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:52.196+0000 7fa4a9400700 1 -- 172.21.15.153:0/3442839133 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4a41a0890 con 0x7fa4a41071d0 2024-03-21T10:30:52.196 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:52.196+0000 7fa49b400700 1 -- 172.21.15.153:0/3442839133 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa49400f040 con 0x7fa4a41071d0 2024-03-21T10:30:52.196 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:52.196+0000 7fa49b400700 1 -- 172.21.15.153:0/3442839133 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa494014790 con 0x7fa4a41071d0 2024-03-21T10:30:52.196 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:52.196+0000 7fa4a9400700 1 -- 172.21.15.153:0/3442839133 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4a41a0e60 con 0x7fa4a41071d0 2024-03-21T10:30:52.197 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:52.197+0000 7fa4a9400700 1 -- 172.21.15.153:0/3442839133 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4a410f6c0 con 0x7fa4a41071d0 2024-03-21T10:30:52.197 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:52.198+0000 7fa49b400700 1 -- 172.21.15.153:0/3442839133 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fa494014980 con 0x7fa4a41071d0 2024-03-21T10:30:52.199 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:52.199+0000 7fa49b400700 1 --2- 172.21.15.153:0/3442839133 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa48806e460 0x7fa488070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:52.199 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:52.200+0000 7fa49b400700 1 -- 172.21.15.153:0/3442839133 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fa49408dfe0 con 0x7fa4a41071d0 2024-03-21T10:30:52.199 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:52.200+0000 7fa4a1e00700 1 --2- 172.21.15.153:0/3442839133 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa48806e460 0x7fa488070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:52.200 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:30:51 smithi174 ceph-mon[88004]: from='client.35038 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:30:52.200 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:30:51 smithi174 ceph-mon[88004]: from='client.35042 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:30:52.200 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:52.200+0000 7fa4a1e00700 1 --2- 172.21.15.153:0/3442839133 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa48806e460 0x7fa488070920 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa490006010 tx=0x7fa490005960 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:52.203 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:52.204+0000 7fa49b400700 1 -- 172.21.15.153:0/3442839133 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa4940c2740 con 0x7fa4a41071d0 2024-03-21T10:30:52.454 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:52.454+0000 7fa4a9400700 1 -- 172.21.15.153:0/3442839133 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fa4a404ea90 con 0x7fa4a41071d0 2024-03-21T10:30:52.455 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:52.455+0000 7fa49b400700 1 -- 172.21.15.153:0/3442839133 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1876 (secure 0 0 0) 0x7fa4940570d0 con 0x7fa4a41071d0 2024-03-21T10:30:52.456 INFO:teuthology.orchestra.run.smithi153.stdout:e30 2024-03-21T10:30:52.456 INFO:teuthology.orchestra.run.smithi153.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-21T10:30:52.456 INFO:teuthology.orchestra.run.smithi153.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:30:52.456 INFO:teuthology.orchestra.run.smithi153.stdout:legacy client fscid: 1 2024-03-21T10:30:52.456 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:30:52.456 INFO:teuthology.orchestra.run.smithi153.stdout:Filesystem 'cephfs' (1) 2024-03-21T10:30:52.456 INFO:teuthology.orchestra.run.smithi153.stdout:fs_name cephfs 2024-03-21T10:30:52.456 INFO:teuthology.orchestra.run.smithi153.stdout:epoch 23 2024-03-21T10:30:52.456 INFO:teuthology.orchestra.run.smithi153.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-21T10:30:52.456 INFO:teuthology.orchestra.run.smithi153.stdout:created 2024-03-21T10:04:27.086576+0000 2024-03-21T10:30:52.457 INFO:teuthology.orchestra.run.smithi153.stdout:modified 2024-03-21T10:19:05.022092+0000 2024-03-21T10:30:52.457 INFO:teuthology.orchestra.run.smithi153.stdout:tableserver 0 2024-03-21T10:30:52.457 INFO:teuthology.orchestra.run.smithi153.stdout:root 0 2024-03-21T10:30:52.457 INFO:teuthology.orchestra.run.smithi153.stdout:session_timeout 60 2024-03-21T10:30:52.457 INFO:teuthology.orchestra.run.smithi153.stdout:session_autoclose 300 2024-03-21T10:30:52.457 INFO:teuthology.orchestra.run.smithi153.stdout:max_file_size 1099511627776 2024-03-21T10:30:52.457 INFO:teuthology.orchestra.run.smithi153.stdout:max_xattr_size 65536 2024-03-21T10:30:52.457 INFO:teuthology.orchestra.run.smithi153.stdout:required_client_features {} 2024-03-21T10:30:52.457 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure 0 2024-03-21T10:30:52.457 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure_osd_epoch 94 2024-03-21T10:30:52.457 INFO:teuthology.orchestra.run.smithi153.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:30:52.457 INFO:teuthology.orchestra.run.smithi153.stdout:max_mds 2 2024-03-21T10:30:52.457 INFO:teuthology.orchestra.run.smithi153.stdout:in 0,1 2024-03-21T10:30:52.458 INFO:teuthology.orchestra.run.smithi153.stdout:up {} 2024-03-21T10:30:52.458 INFO:teuthology.orchestra.run.smithi153.stdout:failed 0,1 2024-03-21T10:30:52.458 INFO:teuthology.orchestra.run.smithi153.stdout:damaged 2024-03-21T10:30:52.458 INFO:teuthology.orchestra.run.smithi153.stdout:stopped 2024-03-21T10:30:52.458 INFO:teuthology.orchestra.run.smithi153.stdout:data_pools [3] 2024-03-21T10:30:52.458 INFO:teuthology.orchestra.run.smithi153.stdout:metadata_pool 2 2024-03-21T10:30:52.458 INFO:teuthology.orchestra.run.smithi153.stdout:inline_data enabled 2024-03-21T10:30:52.458 INFO:teuthology.orchestra.run.smithi153.stdout:balancer 2024-03-21T10:30:52.458 INFO:teuthology.orchestra.run.smithi153.stdout:bal_rank_mask -1 2024-03-21T10:30:52.458 INFO:teuthology.orchestra.run.smithi153.stdout:standby_count_wanted 1 2024-03-21T10:30:52.458 INFO:teuthology.orchestra.run.smithi153.stdout:qdb_cluster leader: 0 members: 2024-03-21T10:30:52.458 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:30:52.459 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:30:52.459 INFO:teuthology.orchestra.run.smithi153.stdout:Standby daemons: 2024-03-21T10:30:52.459 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:30:52.459 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.zipckr{-1:34442} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6826/1103846295,v1:172.21.15.153:6827/1103846295] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:30:52.459 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.iuyluj{-1:34474} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6828/1738630832,v1:172.21.15.153:6829/1738630832] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:30:52.459 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.nhozag{-1:44269} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.174:6826/3967223227,v1:172.21.15.174:6827/3967223227] compat {c=[1],r=[1],i=[7ff]}] 2024-03-21T10:30:52.459 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.yyimst{-1:44313} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.174:6824/4127786797,v1:172.21.15.174:6825/4127786797] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:30:52.460 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:52.458+0000 7fa4a9400700 1 -- 172.21.15.153:0/3442839133 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa48806e460 msgr2=0x7fa488070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:52.460 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:52.458+0000 7fa4a9400700 1 --2- 172.21.15.153:0/3442839133 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa48806e460 0x7fa488070920 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa490006010 tx=0x7fa490005960 comp rx=0 tx=0).stop 2024-03-21T10:30:52.460 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:52.458+0000 7fa4a9400700 1 -- 172.21.15.153:0/3442839133 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa4a41071d0 msgr2=0x7fa4a419ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:52.460 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:52.458+0000 7fa4a9400700 1 --2- 172.21.15.153:0/3442839133 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa4a41071d0 0x7fa4a419ccb0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fa4940060b0 tx=0x7fa49400d3b0 comp rx=0 tx=0).stop 2024-03-21T10:30:52.460 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:52.459+0000 7fa4a9400700 1 -- 172.21.15.153:0/3442839133 shutdown_connections 2024-03-21T10:30:52.460 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:52.459+0000 7fa4a9400700 1 --2- 172.21.15.153:0/3442839133 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa48806e460 0x7fa488070920 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:52.460 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:52.459+0000 7fa4a9400700 1 --2- 172.21.15.153:0/3442839133 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa4a41071d0 0x7fa4a419ccb0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:52.460 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:52.459+0000 7fa4a9400700 1 --2- 172.21.15.153:0/3442839133 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa4a410d280 0x7fa4a419d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:52.460 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:52.459+0000 7fa4a9400700 1 -- 172.21.15.153:0/3442839133 >> 172.21.15.153:0/3442839133 conn(0x7fa4a40769e0 msgr2=0x7fa4a4078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:52.460 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:52.459+0000 7fa4a9400700 1 -- 172.21.15.153:0/3442839133 shutdown_connections 2024-03-21T10:30:52.461 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:52.459+0000 7fa4a9400700 1 -- 172.21.15.153:0/3442839133 wait complete. 2024-03-21T10:30:52.461 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 30 2024-03-21T10:30:52.614 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:52.614+0000 7f9859c00700 1 -- 172.21.15.153:0/677895185 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f985410d280 msgr2=0x7f985410d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:52.614 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:52.614+0000 7f9859c00700 1 --2- 172.21.15.153:0/677895185 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f985410d280 0x7f985410d660 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f9844009910 tx=0x7f9844009c20 comp rx=0 tx=0).stop 2024-03-21T10:30:52.614 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:52.614+0000 7f9859c00700 1 -- 172.21.15.153:0/677895185 shutdown_connections 2024-03-21T10:30:52.614 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:52.614+0000 7f9859c00700 1 --2- 172.21.15.153:0/677895185 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f98541071d0 0x7f9854107650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:52.615 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:52.614+0000 7f9859c00700 1 --2- 172.21.15.153:0/677895185 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f985410d280 0x7f985410d660 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:52.615 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:52.614+0000 7f9859c00700 1 -- 172.21.15.153:0/677895185 >> 172.21.15.153:0/677895185 conn(0x7f98540769e0 msgr2=0x7f9854078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:52.615 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:52.615+0000 7f9859c00700 1 -- 172.21.15.153:0/677895185 shutdown_connections 2024-03-21T10:30:52.615 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:52.615+0000 7f9859c00700 1 -- 172.21.15.153:0/677895185 wait complete. 2024-03-21T10:30:52.616 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:52.616+0000 7f9859c00700 1 Processor -- start 2024-03-21T10:30:52.616 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:52.617+0000 7f9859c00700 1 -- start start 2024-03-21T10:30:52.617 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:52.617+0000 7f9859c00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f98541071d0 0x7f985419ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:52.617 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:52.617+0000 7f9859c00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f985410d280 0x7f985419d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:52.617 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:52.617+0000 7f9859c00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f985419d8a0 con 0x7f985410d280 2024-03-21T10:30:52.618 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:52.617+0000 7f9859c00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98541a05d0 con 0x7f98541071d0 2024-03-21T10:30:52.618 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:52.617+0000 7f984be00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f985410d280 0x7f985419d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:52.618 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:52.617+0000 7f984be00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f985410d280 0x7f985419d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:36838/0 (socket says 172.21.15.153:36838) 2024-03-21T10:30:52.618 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:52.617+0000 7f984be00700 1 -- 172.21.15.153:0/306563446 learned_addr learned my addr 172.21.15.153:0/306563446 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:30:52.618 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:52.618+0000 7f9852a00700 1 --2- 172.21.15.153:0/306563446 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f98541071d0 0x7f985419ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:52.618 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:52.618+0000 7f984be00700 1 -- 172.21.15.153:0/306563446 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f98541071d0 msgr2=0x7f985419ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:52.618 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:52.618+0000 7f984be00700 1 --2- 172.21.15.153:0/306563446 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f98541071d0 0x7f985419ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:52.618 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:52.618+0000 7f984be00700 1 -- 172.21.15.153:0/306563446 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98440095f0 con 0x7f985410d280 2024-03-21T10:30:52.618 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:52.618+0000 7f984be00700 1 --2- 172.21.15.153:0/306563446 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f985410d280 0x7f985419d210 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f983c00c8a0 tx=0x7f983c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:52.619 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:52.618+0000 7f984b400700 1 -- 172.21.15.153:0/306563446 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f983c00cea0 con 0x7f985410d280 2024-03-21T10:30:52.619 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:52.618+0000 7f984b400700 1 -- 172.21.15.153:0/306563446 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f983c004830 con 0x7f985410d280 2024-03-21T10:30:52.619 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:52.619+0000 7f9859c00700 1 -- 172.21.15.153:0/306563446 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98541a08b0 con 0x7f985410d280 2024-03-21T10:30:52.619 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:52.619+0000 7f984b400700 1 -- 172.21.15.153:0/306563446 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f983c0055b0 con 0x7f985410d280 2024-03-21T10:30:52.620 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:52.619+0000 7f9859c00700 1 -- 172.21.15.153:0/306563446 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98541a0e80 con 0x7f985410d280 2024-03-21T10:30:52.620 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:52.620+0000 7f9859c00700 1 -- 172.21.15.153:0/306563446 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f985404ea90 con 0x7f985410d280 2024-03-21T10:30:52.620 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:52.621+0000 7f984b400700 1 -- 172.21.15.153:0/306563446 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f983c0049a0 con 0x7f985410d280 2024-03-21T10:30:52.621 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:52.622+0000 7f984b400700 1 --2- 172.21.15.153:0/306563446 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f983806e460 0x7f9838070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:52.621 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:52.622+0000 7f984b400700 1 -- 172.21.15.153:0/306563446 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f983c08e270 con 0x7f985410d280 2024-03-21T10:30:52.622 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:52.622+0000 7f9852a00700 1 --2- 172.21.15.153:0/306563446 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f983806e460 0x7f9838070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:52.622 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:52.623+0000 7f9852a00700 1 --2- 172.21.15.153:0/306563446 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f983806e460 0x7f9838070920 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f9844006010 tx=0x7f984401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:52.626 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:52.627+0000 7f984b400700 1 -- 172.21.15.153:0/306563446 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f983c057290 con 0x7f985410d280 2024-03-21T10:30:52.865 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:52.865+0000 7f9859c00700 1 -- 172.21.15.153:0/306563446 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f98541a12f0 con 0x7f983806e460 2024-03-21T10:30:52.865 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:30:52 smithi153 ceph-mon[113733]: pgmap v570: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:30:52.865 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:30:52 smithi153 ceph-mon[113733]: from='client.35046 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:30:52.865 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:30:52 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/1855229016' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:30:52.865 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:30:52 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/3442839133' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:30:52.867 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:52.867+0000 7f984b400700 1 -- 172.21.15.153:0/306563446 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f98541a12f0 con 0x7f983806e460 2024-03-21T10:30:52.867 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:30:52.867 INFO:teuthology.orchestra.run.smithi153.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232", 2024-03-21T10:30:52.867 INFO:teuthology.orchestra.run.smithi153.stdout: "in_progress": true, 2024-03-21T10:30:52.867 INFO:teuthology.orchestra.run.smithi153.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-21T10:30:52.867 INFO:teuthology.orchestra.run.smithi153.stdout: "services_complete": [ 2024-03-21T10:30:52.867 INFO:teuthology.orchestra.run.smithi153.stdout: "crash", 2024-03-21T10:30:52.867 INFO:teuthology.orchestra.run.smithi153.stdout: "mon", 2024-03-21T10:30:52.867 INFO:teuthology.orchestra.run.smithi153.stdout: "osd", 2024-03-21T10:30:52.867 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr" 2024-03-21T10:30:52.868 INFO:teuthology.orchestra.run.smithi153.stdout: ], 2024-03-21T10:30:52.868 INFO:teuthology.orchestra.run.smithi153.stdout: "progress": "15/23 daemons upgraded", 2024-03-21T10:30:52.868 INFO:teuthology.orchestra.run.smithi153.stdout: "message": "Currently upgrading mds daemons", 2024-03-21T10:30:52.868 INFO:teuthology.orchestra.run.smithi153.stdout: "is_paused": false 2024-03-21T10:30:52.868 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:30:52.869 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:52.870+0000 7f9859c00700 1 -- 172.21.15.153:0/306563446 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f983806e460 msgr2=0x7f9838070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:52.870 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:52.870+0000 7f9859c00700 1 --2- 172.21.15.153:0/306563446 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f983806e460 0x7f9838070920 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f9844006010 tx=0x7f984401a040 comp rx=0 tx=0).stop 2024-03-21T10:30:52.870 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:52.870+0000 7f9859c00700 1 -- 172.21.15.153:0/306563446 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f985410d280 msgr2=0x7f985419d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:52.870 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:52.870+0000 7f9859c00700 1 --2- 172.21.15.153:0/306563446 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f985410d280 0x7f985419d210 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f983c00c8a0 tx=0x7f983c00cc60 comp rx=0 tx=0).stop 2024-03-21T10:30:52.870 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:52.870+0000 7f9859c00700 1 -- 172.21.15.153:0/306563446 shutdown_connections 2024-03-21T10:30:52.870 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:52.870+0000 7f9859c00700 1 --2- 172.21.15.153:0/306563446 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f98541071d0 0x7f985419ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:52.870 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:52.870+0000 7f9859c00700 1 --2- 172.21.15.153:0/306563446 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f983806e460 0x7f9838070920 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:52.870 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:52.870+0000 7f9859c00700 1 --2- 172.21.15.153:0/306563446 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f985410d280 0x7f985419d210 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:52.870 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:52.870+0000 7f9859c00700 1 -- 172.21.15.153:0/306563446 >> 172.21.15.153:0/306563446 conn(0x7f98540769e0 msgr2=0x7f9854078320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:52.870 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:52.870+0000 7f9859c00700 1 -- 172.21.15.153:0/306563446 shutdown_connections 2024-03-21T10:30:52.870 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:52.870+0000 7f9859c00700 1 -- 172.21.15.153:0/306563446 wait complete. 2024-03-21T10:30:53.021 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:53.021+0000 7f0fda600700 1 -- 172.21.15.153:0/1674997293 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0fd4100dc0 msgr2=0x7f0fd41011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:53.021 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:53.021+0000 7f0fda600700 1 --2- 172.21.15.153:0/1674997293 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0fd4100dc0 0x7f0fd41011a0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f0fc4009910 tx=0x7f0fc4009c20 comp rx=0 tx=0).stop 2024-03-21T10:30:53.021 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:53.021+0000 7f0fda600700 1 -- 172.21.15.153:0/1674997293 shutdown_connections 2024-03-21T10:30:53.021 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:53.021+0000 7f0fda600700 1 --2- 172.21.15.153:0/1674997293 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0fd4101770 0x7f0fd41057c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:53.021 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:53.021+0000 7f0fda600700 1 --2- 172.21.15.153:0/1674997293 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0fd4100dc0 0x7f0fd41011a0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:53.021 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:53.021+0000 7f0fda600700 1 -- 172.21.15.153:0/1674997293 >> 172.21.15.153:0/1674997293 conn(0x7f0fd40fc650 msgr2=0x7f0fd40fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:53.021 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:53.021+0000 7f0fda600700 1 -- 172.21.15.153:0/1674997293 shutdown_connections 2024-03-21T10:30:53.021 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:53.021+0000 7f0fda600700 1 -- 172.21.15.153:0/1674997293 wait complete. 2024-03-21T10:30:53.022 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:53.022+0000 7f0fda600700 1 Processor -- start 2024-03-21T10:30:53.023 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:53.022+0000 7f0fda600700 1 -- start start 2024-03-21T10:30:53.023 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:53.023+0000 7f0fda600700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0fd4100dc0 0x7f0fd419c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:53.023 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:53.023+0000 7f0fda600700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0fd4101770 0x7f0fd419ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:53.023 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:53.023+0000 7f0fda600700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd419d110 con 0x7f0fd4101770 2024-03-21T10:30:53.023 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:53.023+0000 7f0fda600700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd41965c0 con 0x7f0fd4100dc0 2024-03-21T10:30:53.023 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:53.023+0000 7f0fd2a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0fd4101770 0x7f0fd419ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:53.023 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:53.023+0000 7f0fd2a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0fd4101770 0x7f0fd419ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:57076/0 (socket says 172.21.15.153:57076) 2024-03-21T10:30:53.023 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:53.023+0000 7f0fd2a00700 1 -- 172.21.15.153:0/1435279978 learned_addr learned my addr 172.21.15.153:0/1435279978 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:30:53.023 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:53.023+0000 7f0fd3400700 1 --2- 172.21.15.153:0/1435279978 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0fd4100dc0 0x7f0fd419c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:53.024 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:53.024+0000 7f0fd2a00700 1 -- 172.21.15.153:0/1435279978 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0fd4100dc0 msgr2=0x7f0fd419c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:53.024 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:53.024+0000 7f0fd2a00700 1 --2- 172.21.15.153:0/1435279978 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0fd4100dc0 0x7f0fd419c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:53.024 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:53.024+0000 7f0fd2a00700 1 -- 172.21.15.153:0/1435279978 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fc40095f0 con 0x7f0fd4101770 2024-03-21T10:30:53.024 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:53.024+0000 7f0fd2a00700 1 --2- 172.21.15.153:0/1435279978 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0fd4101770 0x7f0fd419ca80 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f0fc000c8f0 tx=0x7f0fc000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:53.024 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:53.024+0000 7f0fcbe00700 1 -- 172.21.15.153:0/1435279978 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0fc0007a50 con 0x7f0fd4101770 2024-03-21T10:30:53.024 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:53.024+0000 7f0fcbe00700 1 -- 172.21.15.153:0/1435279978 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0fc0007bb0 con 0x7f0fd4101770 2024-03-21T10:30:53.024 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:53.024+0000 7f0fcbe00700 1 -- 172.21.15.153:0/1435279978 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0fc0018630 con 0x7f0fd4101770 2024-03-21T10:30:53.025 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:53.024+0000 7f0fda600700 1 -- 172.21.15.153:0/1435279978 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fd41968a0 con 0x7f0fd4101770 2024-03-21T10:30:53.025 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:53.024+0000 7f0fda600700 1 -- 172.21.15.153:0/1435279978 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fd4196df0 con 0x7f0fd4101770 2024-03-21T10:30:53.025 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:53.025+0000 7f0fda600700 1 -- 172.21.15.153:0/1435279978 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fd4109160 con 0x7f0fd4101770 2024-03-21T10:30:53.026 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:53.026+0000 7f0fcbe00700 1 -- 172.21.15.153:0/1435279978 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f0fc0007d20 con 0x7f0fd4101770 2024-03-21T10:30:53.026 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:53.027+0000 7f0fcbe00700 1 --2- 172.21.15.153:0/1435279978 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f0fb806e460 0x7f0fb8070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:30:53.027 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:53.027+0000 7f0fcbe00700 1 -- 172.21.15.153:0/1435279978 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f0fc008e4c0 con 0x7f0fd4101770 2024-03-21T10:30:53.027 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:53.028+0000 7f0fd3400700 1 --2- 172.21.15.153:0/1435279978 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f0fb806e460 0x7f0fb8070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:30:53.028 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:53.028+0000 7f0fd3400700 1 --2- 172.21.15.153:0/1435279978 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f0fb806e460 0x7f0fb8070920 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f0fc4006010 tx=0x7f0fc401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:30:53.034 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:53.035+0000 7f0fcbe00700 1 -- 172.21.15.153:0/1435279978 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0fc005b090 con 0x7f0fd4101770 2024-03-21T10:30:53.199 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:30:52 smithi174 ceph-mon[88004]: pgmap v570: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:30:53.199 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:30:52 smithi174 ceph-mon[88004]: from='client.35046 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:30:53.200 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:30:52 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/1855229016' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:30:53.200 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:30:52 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/3442839133' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:30:53.351 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:53.352+0000 7f0fda600700 1 -- 172.21.15.153:0/1435279978 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f0fd404ea90 con 0x7f0fd4101770 2024-03-21T10:30:53.352 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:53.353+0000 7f0fcbe00700 1 -- 172.21.15.153:0/1435279978 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f0fc005ac20 con 0x7f0fd4101770 2024-03-21T10:30:53.352 INFO:teuthology.orchestra.run.smithi153.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-03-21T10:30:53.352 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-21T10:30:53.352 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is degraded 2024-03-21T10:30:53.352 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-21T10:30:53.352 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-21T10:30:53.353 INFO:teuthology.orchestra.run.smithi153.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-21T10:30:53.353 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-21T10:30:53.355 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:53.355+0000 7f0fda600700 1 -- 172.21.15.153:0/1435279978 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f0fb806e460 msgr2=0x7f0fb8070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:53.356 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:53.355+0000 7f0fda600700 1 --2- 172.21.15.153:0/1435279978 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f0fb806e460 0x7f0fb8070920 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f0fc4006010 tx=0x7f0fc401a040 comp rx=0 tx=0).stop 2024-03-21T10:30:53.356 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:53.355+0000 7f0fda600700 1 -- 172.21.15.153:0/1435279978 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0fd4101770 msgr2=0x7f0fd419ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:30:53.356 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:53.355+0000 7f0fda600700 1 --2- 172.21.15.153:0/1435279978 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0fd4101770 0x7f0fd419ca80 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f0fc000c8f0 tx=0x7f0fc000ccb0 comp rx=0 tx=0).stop 2024-03-21T10:30:53.356 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:53.356+0000 7f0fda600700 1 -- 172.21.15.153:0/1435279978 shutdown_connections 2024-03-21T10:30:53.356 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:53.356+0000 7f0fda600700 1 --2- 172.21.15.153:0/1435279978 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0fd4100dc0 0x7f0fd419c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:53.356 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:53.356+0000 7f0fda600700 1 --2- 172.21.15.153:0/1435279978 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f0fb806e460 0x7f0fb8070920 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:53.356 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:53.356+0000 7f0fda600700 1 --2- 172.21.15.153:0/1435279978 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0fd4101770 0x7f0fd419ca80 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:30:53.356 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:53.356+0000 7f0fda600700 1 -- 172.21.15.153:0/1435279978 >> 172.21.15.153:0/1435279978 conn(0x7f0fd40fc650 msgr2=0x7f0fd40fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:30:53.356 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:53.356+0000 7f0fda600700 1 -- 172.21.15.153:0/1435279978 shutdown_connections 2024-03-21T10:30:53.356 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:30:53.356+0000 7f0fda600700 1 -- 172.21.15.153:0/1435279978 wait complete. 2024-03-21T10:30:54.054 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:30:53 smithi153 ceph-mon[113733]: from='client.35058 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:30:54.054 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:30:53 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/1435279978' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:30:54.171 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:30:53 smithi174 ceph-mon[88004]: from='client.35058 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:30:54.171 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:30:53 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/1435279978' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:30:55.054 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:30:54 smithi153 ceph-mon[113733]: pgmap v571: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:30:55.199 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:30:54 smithi174 ceph-mon[88004]: pgmap v571: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:30:57.054 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:30:56 smithi153 ceph-mon[113733]: pgmap v572: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:30:57.199 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:30:56 smithi174 ceph-mon[88004]: pgmap v572: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:30:59.054 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:30:58 smithi153 ceph-mon[113733]: pgmap v573: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:30:59.199 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:30:58 smithi174 ceph-mon[88004]: pgmap v573: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:31:01.054 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:31:00 smithi153 ceph-mon[113733]: pgmap v574: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:31:01.199 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:31:00 smithi174 ceph-mon[88004]: pgmap v574: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:31:02.199 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:31:01 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:31:02.304 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:31:01 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:31:03.054 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:31:02 smithi153 ceph-mon[113733]: pgmap v575: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:31:03.199 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:31:02 smithi174 ceph-mon[88004]: pgmap v575: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:31:05.199 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:31:04 smithi174 ceph-mon[88004]: pgmap v576: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:31:05.304 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:31:04 smithi153 ceph-mon[113733]: pgmap v576: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:31:07.304 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:31:06 smithi153 ceph-mon[113733]: pgmap v577: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:31:07.449 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:31:06 smithi174 ceph-mon[88004]: pgmap v577: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:31:09.304 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:31:08 smithi153 ceph-mon[113733]: pgmap v578: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:31:09.449 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:31:08 smithi174 ceph-mon[88004]: pgmap v578: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:31:11.304 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:31:10 smithi153 ceph-mon[113733]: pgmap v579: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:31:11.449 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:31:10 smithi174 ceph-mon[88004]: pgmap v579: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:31:13.304 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:31:12 smithi153 ceph-mon[113733]: pgmap v580: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:31:13.449 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:31:12 smithi174 ceph-mon[88004]: pgmap v580: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:31:15.199 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:31:14 smithi174 ceph-mon[88004]: pgmap v581: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:31:15.304 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:31:14 smithi153 ceph-mon[113733]: pgmap v581: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:31:17.304 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:31:16 smithi153 ceph-mon[113733]: pgmap v582: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:31:17.305 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:31:16 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:31:17.449 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:31:16 smithi174 ceph-mon[88004]: pgmap v582: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:31:17.449 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:31:16 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:31:19.304 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:31:18 smithi153 ceph-mon[113733]: pgmap v583: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:31:19.449 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:31:18 smithi174 ceph-mon[88004]: pgmap v583: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:31:21.304 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:31:21 smithi153 ceph-mon[113733]: pgmap v584: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:31:21.449 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:31:21 smithi174 ceph-mon[88004]: pgmap v584: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:31:23.304 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:31:23 smithi153 ceph-mon[113733]: pgmap v585: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:31:23.449 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:31:23 smithi174 ceph-mon[88004]: pgmap v585: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:31:23.527 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:23.527+0000 7fe82f200700 1 -- 172.21.15.153:0/4233517310 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe830100970 msgr2=0x7fe830100df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:23.527 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:23.527+0000 7fe82f200700 1 --2- 172.21.15.153:0/4233517310 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe830100970 0x7fe830100df0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fe81c009910 tx=0x7fe81c009c20 comp rx=0 tx=0).stop 2024-03-21T10:31:23.527 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:23.527+0000 7fe82f200700 1 -- 172.21.15.153:0/4233517310 shutdown_connections 2024-03-21T10:31:23.527 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:23.527+0000 7fe82f200700 1 --2- 172.21.15.153:0/4233517310 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe830100970 0x7fe830100df0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.528 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:23.527+0000 7fe82f200700 1 --2- 172.21.15.153:0/4233517310 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe830106970 0x7fe830106d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.528 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:23.527+0000 7fe82f200700 1 -- 172.21.15.153:0/4233517310 >> 172.21.15.153:0/4233517310 conn(0x7fe8300fc440 msgr2=0x7fe8300fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:23.528 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:23.527+0000 7fe82f200700 1 -- 172.21.15.153:0/4233517310 shutdown_connections 2024-03-21T10:31:23.528 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:23.527+0000 7fe82f200700 1 -- 172.21.15.153:0/4233517310 wait complete. 2024-03-21T10:31:23.529 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:23.528+0000 7fe82f200700 1 Processor -- start 2024-03-21T10:31:23.529 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:23.528+0000 7fe82f200700 1 -- start start 2024-03-21T10:31:23.529 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:23.529+0000 7fe82f200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe830100970 0x7fe830073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:23.529 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:23.529+0000 7fe82f200700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe830106970 0x7fe8300735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:23.529 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:23.529+0000 7fe82f200700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe830073b10 con 0x7fe830100970 2024-03-21T10:31:23.529 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:23.529+0000 7fe82f200700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe830073c50 con 0x7fe830106970 2024-03-21T10:31:23.530 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:23.529+0000 7fe82de00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe830100970 0x7fe830073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:23.530 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:23.529+0000 7fe82de00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe830100970 0x7fe830073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41506/0 (socket says 172.21.15.153:41506) 2024-03-21T10:31:23.530 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:23.529+0000 7fe82de00700 1 -- 172.21.15.153:0/2090653710 learned_addr learned my addr 172.21.15.153:0/2090653710 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:31:23.530 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:23.530+0000 7fe82d400700 1 --2- 172.21.15.153:0/2090653710 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe830106970 0x7fe8300735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:23.530 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:23.530+0000 7fe82de00700 1 -- 172.21.15.153:0/2090653710 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe830106970 msgr2=0x7fe8300735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:23.530 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:23.530+0000 7fe82de00700 1 --2- 172.21.15.153:0/2090653710 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe830106970 0x7fe8300735d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.530 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:23.530+0000 7fe82de00700 1 -- 172.21.15.153:0/2090653710 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe81c0095f0 con 0x7fe830100970 2024-03-21T10:31:23.530 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:23.530+0000 7fe82de00700 1 --2- 172.21.15.153:0/2090653710 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe830100970 0x7fe830073090 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fe82000d8d0 tx=0x7fe82000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:23.531 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:23.530+0000 7fe826a00700 1 -- 172.21.15.153:0/2090653710 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe820009940 con 0x7fe830100970 2024-03-21T10:31:23.531 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:23.530+0000 7fe826a00700 1 -- 172.21.15.153:0/2090653710 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe820010460 con 0x7fe830100970 2024-03-21T10:31:23.531 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:23.531+0000 7fe82f200700 1 -- 172.21.15.153:0/2090653710 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8301a2720 con 0x7fe830100970 2024-03-21T10:31:23.531 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:23.531+0000 7fe826a00700 1 -- 172.21.15.153:0/2090653710 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe82000f5f0 con 0x7fe830100970 2024-03-21T10:31:23.531 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:23.531+0000 7fe82f200700 1 -- 172.21.15.153:0/2090653710 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8301a2b30 con 0x7fe830100970 2024-03-21T10:31:23.531 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:23.532+0000 7fe82f200700 1 -- 172.21.15.153:0/2090653710 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe830108f00 con 0x7fe830100970 2024-03-21T10:31:23.532 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:23.533+0000 7fe826a00700 1 -- 172.21.15.153:0/2090653710 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fe8200105d0 con 0x7fe830100970 2024-03-21T10:31:23.533 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:23.534+0000 7fe826a00700 1 --2- 172.21.15.153:0/2090653710 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe81406e460 0x7fe814070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:23.533 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:23.534+0000 7fe826a00700 1 -- 172.21.15.153:0/2090653710 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fe82008e510 con 0x7fe830100970 2024-03-21T10:31:23.533 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:23.534+0000 7fe82d400700 1 --2- 172.21.15.153:0/2090653710 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe81406e460 0x7fe814070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:23.534 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:23.535+0000 7fe82d400700 1 --2- 172.21.15.153:0/2090653710 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe81406e460 0x7fe814070920 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe830074880 tx=0x7fe81c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:23.538 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:23.539+0000 7fe826a00700 1 -- 172.21.15.153:0/2090653710 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe8200575d0 con 0x7fe830100970 2024-03-21T10:31:23.782 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:23.782+0000 7fe82f200700 1 -- 172.21.15.153:0/2090653710 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe83002cdd0 con 0x7fe81406e460 2024-03-21T10:31:23.784 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:23.784+0000 7fe826a00700 1 -- 172.21.15.153:0/2090653710 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fe83002cdd0 con 0x7fe81406e460 2024-03-21T10:31:23.787 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:23.788+0000 7fe82f200700 1 -- 172.21.15.153:0/2090653710 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe81406e460 msgr2=0x7fe814070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:23.787 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:23.788+0000 7fe82f200700 1 --2- 172.21.15.153:0/2090653710 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe81406e460 0x7fe814070920 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe830074880 tx=0x7fe81c01a040 comp rx=0 tx=0).stop 2024-03-21T10:31:23.787 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:23.788+0000 7fe82f200700 1 -- 172.21.15.153:0/2090653710 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe830100970 msgr2=0x7fe830073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:23.787 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:23.788+0000 7fe82f200700 1 --2- 172.21.15.153:0/2090653710 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe830100970 0x7fe830073090 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fe82000d8d0 tx=0x7fe82000dc90 comp rx=0 tx=0).stop 2024-03-21T10:31:23.788 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:23.788+0000 7fe82f200700 1 -- 172.21.15.153:0/2090653710 shutdown_connections 2024-03-21T10:31:23.788 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:23.788+0000 7fe82f200700 1 --2- 172.21.15.153:0/2090653710 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe81406e460 0x7fe814070920 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.788 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:23.788+0000 7fe82f200700 1 --2- 172.21.15.153:0/2090653710 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe830100970 0x7fe830073090 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.788 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:23.788+0000 7fe82f200700 1 --2- 172.21.15.153:0/2090653710 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe830106970 0x7fe8300735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.788 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:23.788+0000 7fe82f200700 1 -- 172.21.15.153:0/2090653710 >> 172.21.15.153:0/2090653710 conn(0x7fe8300fc440 msgr2=0x7fe8300fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:23.788 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:23.788+0000 7fe82f200700 1 -- 172.21.15.153:0/2090653710 shutdown_connections 2024-03-21T10:31:23.788 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:23.788+0000 7fe82f200700 1 -- 172.21.15.153:0/2090653710 wait complete. 2024-03-21T10:31:23.805 INFO:teuthology.orchestra.run.smithi153.stdout:true 2024-03-21T10:31:23.949 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:23.948+0000 7fd454a00700 1 -- 172.21.15.153:0/3556216598 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd450101750 msgr2=0x7fd4501057a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:23.949 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:23.948+0000 7fd454a00700 1 --2- 172.21.15.153:0/3556216598 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd450101750 0x7fd4501057a0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fd438009910 tx=0x7fd438009c20 comp rx=0 tx=0).stop 2024-03-21T10:31:23.949 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:23.948+0000 7fd454a00700 1 -- 172.21.15.153:0/3556216598 shutdown_connections 2024-03-21T10:31:23.949 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:23.948+0000 7fd454a00700 1 --2- 172.21.15.153:0/3556216598 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd450101750 0x7fd4501057a0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.949 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:23.948+0000 7fd454a00700 1 --2- 172.21.15.153:0/3556216598 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd450100da0 0x7fd450101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.949 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:23.948+0000 7fd454a00700 1 -- 172.21.15.153:0/3556216598 >> 172.21.15.153:0/3556216598 conn(0x7fd4500fc630 msgr2=0x7fd4500fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:23.949 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:23.949+0000 7fd454a00700 1 -- 172.21.15.153:0/3556216598 shutdown_connections 2024-03-21T10:31:23.950 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:23.949+0000 7fd454a00700 1 -- 172.21.15.153:0/3556216598 wait complete. 2024-03-21T10:31:23.950 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:23.950+0000 7fd454a00700 1 Processor -- start 2024-03-21T10:31:23.950 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:23.950+0000 7fd454a00700 1 -- start start 2024-03-21T10:31:23.951 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:23.950+0000 7fd454a00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd450100da0 0x7fd450073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:23.951 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:23.950+0000 7fd454a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd450101750 0x7fd4500735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:23.952 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:23.950+0000 7fd454a00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd450074e40 con 0x7fd450101750 2024-03-21T10:31:23.952 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:23.950+0000 7fd454a00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd450074fb0 con 0x7fd450100da0 2024-03-21T10:31:23.952 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:23.951+0000 7fd445400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd450101750 0x7fd4500735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:23.952 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:23.951+0000 7fd445400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd450101750 0x7fd4500735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41522/0 (socket says 172.21.15.153:41522) 2024-03-21T10:31:23.952 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:23.951+0000 7fd445400700 1 -- 172.21.15.153:0/771288791 learned_addr learned my addr 172.21.15.153:0/771288791 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:31:23.952 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:23.951+0000 7fd44de00700 1 --2- 172.21.15.153:0/771288791 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd450100da0 0x7fd450073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:23.952 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:23.951+0000 7fd445400700 1 -- 172.21.15.153:0/771288791 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd450100da0 msgr2=0x7fd450073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:23.952 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:23.951+0000 7fd445400700 1 --2- 172.21.15.153:0/771288791 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd450100da0 0x7fd450073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:23.952 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:23.951+0000 7fd445400700 1 -- 172.21.15.153:0/771288791 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4380095f0 con 0x7fd450101750 2024-03-21T10:31:23.952 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:23.951+0000 7fd445400700 1 --2- 172.21.15.153:0/771288791 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd450101750 0x7fd4500735d0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fd43800bda0 tx=0x7fd43800be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:23.952 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:23.952+0000 7fd447400700 1 -- 172.21.15.153:0/771288791 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd43801d070 con 0x7fd450101750 2024-03-21T10:31:23.953 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:23.952+0000 7fd447400700 1 -- 172.21.15.153:0/771288791 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd438004fb0 con 0x7fd450101750 2024-03-21T10:31:23.953 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:23.952+0000 7fd454a00700 1 -- 172.21.15.153:0/771288791 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd450073b70 con 0x7fd450101750 2024-03-21T10:31:23.953 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:23.952+0000 7fd447400700 1 -- 172.21.15.153:0/771288791 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd43800f460 con 0x7fd450101750 2024-03-21T10:31:23.953 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:23.952+0000 7fd454a00700 1 -- 172.21.15.153:0/771288791 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4501a70d0 con 0x7fd450101750 2024-03-21T10:31:23.953 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:23.953+0000 7fd454a00700 1 -- 172.21.15.153:0/771288791 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4501091a0 con 0x7fd450101750 2024-03-21T10:31:23.953 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:23.954+0000 7fd447400700 1 -- 172.21.15.153:0/771288791 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fd438022800 con 0x7fd450101750 2024-03-21T10:31:23.954 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:23.955+0000 7fd447400700 1 --2- 172.21.15.153:0/771288791 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd43406e460 0x7fd434070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:23.955 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:23.955+0000 7fd447400700 1 -- 172.21.15.153:0/771288791 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fd43808fb00 con 0x7fd450101750 2024-03-21T10:31:23.955 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:23.955+0000 7fd44de00700 1 --2- 172.21.15.153:0/771288791 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd43406e460 0x7fd434070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:23.956 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:23.956+0000 7fd44de00700 1 --2- 172.21.15.153:0/771288791 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd43406e460 0x7fd434070920 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fd4400060b0 tx=0x7fd440007b80 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:23.959 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:23.960+0000 7fd447400700 1 -- 172.21.15.153:0/771288791 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd438058cf0 con 0x7fd450101750 2024-03-21T10:31:24.199 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:24.199+0000 7fd454a00700 1 -- 172.21.15.153:0/771288791 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd45002cce0 con 0x7fd43406e460 2024-03-21T10:31:24.202 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:24.202+0000 7fd447400700 1 -- 172.21.15.153:0/771288791 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fd45002cce0 con 0x7fd43406e460 2024-03-21T10:31:24.205 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:24.205+0000 7fd454a00700 1 -- 172.21.15.153:0/771288791 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd43406e460 msgr2=0x7fd434070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:24.205 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:24.205+0000 7fd454a00700 1 --2- 172.21.15.153:0/771288791 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd43406e460 0x7fd434070920 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fd4400060b0 tx=0x7fd440007b80 comp rx=0 tx=0).stop 2024-03-21T10:31:24.205 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:24.205+0000 7fd454a00700 1 -- 172.21.15.153:0/771288791 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd450101750 msgr2=0x7fd4500735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:24.205 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:24.205+0000 7fd454a00700 1 --2- 172.21.15.153:0/771288791 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd450101750 0x7fd4500735d0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fd43800bda0 tx=0x7fd43800be80 comp rx=0 tx=0).stop 2024-03-21T10:31:24.205 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:24.205+0000 7fd454a00700 1 -- 172.21.15.153:0/771288791 shutdown_connections 2024-03-21T10:31:24.205 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:24.205+0000 7fd454a00700 1 --2- 172.21.15.153:0/771288791 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd450100da0 0x7fd450073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:24.205 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:24.205+0000 7fd454a00700 1 --2- 172.21.15.153:0/771288791 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd43406e460 0x7fd434070920 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:24.205 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:24.205+0000 7fd454a00700 1 --2- 172.21.15.153:0/771288791 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd450101750 0x7fd4500735d0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:24.205 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:24.205+0000 7fd454a00700 1 -- 172.21.15.153:0/771288791 >> 172.21.15.153:0/771288791 conn(0x7fd4500fc630 msgr2=0x7fd4500fea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:24.205 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:24.206+0000 7fd454a00700 1 -- 172.21.15.153:0/771288791 shutdown_connections 2024-03-21T10:31:24.205 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:24.206+0000 7fd454a00700 1 -- 172.21.15.153:0/771288791 wait complete. 2024-03-21T10:31:24.360 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:24.361+0000 7f01c2600700 1 -- 172.21.15.153:0/1150907707 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f01bc1071d0 msgr2=0x7f01bc107650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:24.361 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:24.361+0000 7f01c2600700 1 --2- 172.21.15.153:0/1150907707 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f01bc1071d0 0x7f01bc107650 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f01a4009910 tx=0x7f01a4009c20 comp rx=0 tx=0).stop 2024-03-21T10:31:24.361 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:24.361+0000 7f01c2600700 1 -- 172.21.15.153:0/1150907707 shutdown_connections 2024-03-21T10:31:24.361 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:24.361+0000 7f01c2600700 1 --2- 172.21.15.153:0/1150907707 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f01bc1071d0 0x7f01bc107650 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:24.361 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:24.361+0000 7f01c2600700 1 --2- 172.21.15.153:0/1150907707 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f01bc10d280 0x7f01bc10d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:24.361 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:24.361+0000 7f01c2600700 1 -- 172.21.15.153:0/1150907707 >> 172.21.15.153:0/1150907707 conn(0x7f01bc0769e0 msgr2=0x7f01bc078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:24.361 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:24.361+0000 7f01c2600700 1 -- 172.21.15.153:0/1150907707 shutdown_connections 2024-03-21T10:31:24.361 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:24.361+0000 7f01c2600700 1 -- 172.21.15.153:0/1150907707 wait complete. 2024-03-21T10:31:24.362 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:24.362+0000 7f01c2600700 1 Processor -- start 2024-03-21T10:31:24.362 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:24.362+0000 7f01c2600700 1 -- start start 2024-03-21T10:31:24.362 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:24.362+0000 7f01c2600700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f01bc1071d0 0x7f01bc19ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:24.363 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:24.362+0000 7f01c2600700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f01bc10d280 0x7f01bc19d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:24.363 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:24.362+0000 7f01c2600700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01bc19d880 con 0x7f01bc10d280 2024-03-21T10:31:24.363 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:24.362+0000 7f01c2600700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01bc1a05b0 con 0x7f01bc1071d0 2024-03-21T10:31:24.363 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:24.363+0000 7f01b2a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f01bc10d280 0x7f01bc19d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:24.363 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:24.363+0000 7f01b2a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f01bc10d280 0x7f01bc19d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41542/0 (socket says 172.21.15.153:41542) 2024-03-21T10:31:24.363 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:24.363+0000 7f01b2a00700 1 -- 172.21.15.153:0/2691379530 learned_addr learned my addr 172.21.15.153:0/2691379530 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:31:24.363 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:24.363+0000 7f01bb400700 1 --2- 172.21.15.153:0/2691379530 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f01bc1071d0 0x7f01bc19ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:24.363 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:24.363+0000 7f01b2a00700 1 -- 172.21.15.153:0/2691379530 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f01bc1071d0 msgr2=0x7f01bc19ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:24.363 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:24.363+0000 7f01b2a00700 1 --2- 172.21.15.153:0/2691379530 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f01bc1071d0 0x7f01bc19ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:24.363 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:24.363+0000 7f01b2a00700 1 -- 172.21.15.153:0/2691379530 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01a40095f0 con 0x7f01bc10d280 2024-03-21T10:31:24.364 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:24.364+0000 7f01b2a00700 1 --2- 172.21.15.153:0/2691379530 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f01bc10d280 0x7f01bc19d1f0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f01a4006010 tx=0x7f01a400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:24.364 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:24.364+0000 7f01b8c00700 1 -- 172.21.15.153:0/2691379530 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f01a401d070 con 0x7f01bc10d280 2024-03-21T10:31:24.364 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:24.364+0000 7f01b8c00700 1 -- 172.21.15.153:0/2691379530 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f01a4004fb0 con 0x7f01bc10d280 2024-03-21T10:31:24.364 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:24.364+0000 7f01c2600700 1 -- 172.21.15.153:0/2691379530 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01bc1a0830 con 0x7f01bc10d280 2024-03-21T10:31:24.364 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:24.364+0000 7f01b8c00700 1 -- 172.21.15.153:0/2691379530 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f01a4022ea0 con 0x7f01bc10d280 2024-03-21T10:31:24.364 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:24.364+0000 7f01c2600700 1 -- 172.21.15.153:0/2691379530 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01bc1a0da0 con 0x7f01bc10d280 2024-03-21T10:31:24.365 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:24.365+0000 7f01c2600700 1 -- 172.21.15.153:0/2691379530 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01bc10f6c0 con 0x7f01bc10d280 2024-03-21T10:31:24.365 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:24.366+0000 7f01b8c00700 1 -- 172.21.15.153:0/2691379530 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f01a4022470 con 0x7f01bc10d280 2024-03-21T10:31:24.366 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:24.367+0000 7f01b8c00700 1 --2- 172.21.15.153:0/2691379530 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f01a006e460 0x7f01a0070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:24.366 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:24.367+0000 7f01b8c00700 1 -- 172.21.15.153:0/2691379530 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f01a408fa50 con 0x7f01bc10d280 2024-03-21T10:31:24.366 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:24.367+0000 7f01bb400700 1 --2- 172.21.15.153:0/2691379530 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f01a006e460 0x7f01a0070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:24.367 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:24.368+0000 7f01bb400700 1 --2- 172.21.15.153:0/2691379530 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f01a006e460 0x7f01a0070920 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f01ac005fd0 tx=0x7f01ac00a560 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:24.371 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:24.372+0000 7f01b8c00700 1 -- 172.21.15.153:0/2691379530 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f01a4058ba0 con 0x7f01bc10d280 2024-03-21T10:31:24.604 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:24.605+0000 7f01c2600700 1 -- 172.21.15.153:0/2691379530 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f01bc1a1080 con 0x7f01a006e460 2024-03-21T10:31:24.635 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:24.636+0000 7f01b8c00700 1 -- 172.21.15.153:0/2691379530 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f01bc1a1080 con 0x7f01a006e460 2024-03-21T10:31:24.636 INFO:teuthology.orchestra.run.smithi153.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-21T10:31:24.636 INFO:teuthology.orchestra.run.smithi153.stdout:alertmanager.smithi153 smithi153 *:9093,9094 running (20m) 11m ago 33m 14.7M - 0.25.0 c8568f914cd2 9ed3f2200b5f 2024-03-21T10:31:24.636 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi153 smithi153 running (34m) 11m ago 34m 9424k - 18.2.1 5be31c24972a 5b61994396a6 2024-03-21T10:31:24.636 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi174 smithi174 running (32m) 11m ago 32m 9244k - 18.2.1 5be31c24972a 4f7a5fc3ac28 2024-03-21T10:31:24.636 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi153 smithi153 running (16m) 11m ago 34m 6656k - 19.0.0-2231-g2edfee68 237e43c7a65f 32c21104d5b0 2024-03-21T10:31:24.636 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi174 smithi174 running (16m) 11m ago 32m 6660k - 19.0.0-2231-g2edfee68 237e43c7a65f a04e48967ab6 2024-03-21T10:31:24.636 INFO:teuthology.orchestra.run.smithi153.stdout:grafana.smithi153 smithi153 *:3000 running (19m) 11m ago 33m 73.6M - 9.4.12 2bacad6d85d8 ee24026ab786 2024-03-21T10:31:24.637 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.iuyluj smithi153 running (11m) 11m ago 26m 10.9M - 19.0.0-2231-g2edfee68 237e43c7a65f 87aff8a5de70 2024-03-21T10:31:24.637 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.zipckr smithi153 running (12m) 11m ago 26m 11.8M - 19.0.0-2231-g2edfee68 237e43c7a65f ccc262bd5dd3 2024-03-21T10:31:24.637 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.nhozag smithi174 running (26m) 11m ago 26m 12.0M - 18.2.1 5be31c24972a 2fee4000308f 2024-03-21T10:31:24.637 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.yyimst smithi174 running (11m) 11m ago 26m 10.7M - 19.0.0-2231-g2edfee68 237e43c7a65f fc1962f9351e 2024-03-21T10:31:24.637 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi153.tluxni smithi153 *:8443,9283,8765 running (23m) 11m ago 35m 509M - 19.0.0-2231-g2edfee68 237e43c7a65f 7ffee0e55382 2024-03-21T10:31:24.637 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi174.qpcwat smithi174 *:8443,9283,8765 running (21m) 11m ago 32m 452M - 19.0.0-2231-g2edfee68 237e43c7a65f 748968f216ab 2024-03-21T10:31:24.637 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi153 smithi153 running (18m) 11m ago 36m 55.0M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f dd8038303dc3 2024-03-21T10:31:24.637 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi174 smithi174 running (17m) 11m ago 32m 44.2M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f 29d78a788f2c 2024-03-21T10:31:24.637 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi153 smithi153 *:9100 running (21m) 11m ago 34m 7889k - 1.5.0 0da6a335fe13 bb835edc0e0b 2024-03-21T10:31:24.637 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi174 smithi174 *:9100 running (21m) 11m ago 32m 8102k - 1.5.0 0da6a335fe13 7c4ecad790fa 2024-03-21T10:31:24.637 INFO:teuthology.orchestra.run.smithi153.stdout:osd.0 smithi153 running (15m) 11m ago 31m 116M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 507900fd4791 2024-03-21T10:31:24.637 INFO:teuthology.orchestra.run.smithi153.stdout:osd.1 smithi153 running (15m) 11m ago 30m 89.2M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 0d646452e9c1 2024-03-21T10:31:24.637 INFO:teuthology.orchestra.run.smithi153.stdout:osd.2 smithi153 running (14m) 11m ago 30m 87.4M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 1326f2afdfcd 2024-03-21T10:31:24.637 INFO:teuthology.orchestra.run.smithi153.stdout:osd.3 smithi174 running (13m) 11m ago 29m 83.6M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 70aafac71ad5 2024-03-21T10:31:24.637 INFO:teuthology.orchestra.run.smithi153.stdout:osd.4 smithi174 running (13m) 11m ago 29m 92.2M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 0d746ce6e79d 2024-03-21T10:31:24.638 INFO:teuthology.orchestra.run.smithi153.stdout:osd.5 smithi174 running (12m) 11m ago 28m 86.2M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 53e5f35b8cac 2024-03-21T10:31:24.638 INFO:teuthology.orchestra.run.smithi153.stdout:prometheus.smithi153 smithi153 *:9095 running (20m) 11m ago 33m 73.5M - 2.43.0 a07b618ecd1d 7d95561e50c2 2024-03-21T10:31:24.638 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:24.639+0000 7f01c2600700 1 -- 172.21.15.153:0/2691379530 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f01a006e460 msgr2=0x7f01a0070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:24.638 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:24.639+0000 7f01c2600700 1 --2- 172.21.15.153:0/2691379530 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f01a006e460 0x7f01a0070920 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f01ac005fd0 tx=0x7f01ac00a560 comp rx=0 tx=0).stop 2024-03-21T10:31:24.638 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:24.639+0000 7f01c2600700 1 -- 172.21.15.153:0/2691379530 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f01bc10d280 msgr2=0x7f01bc19d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:24.639 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:24.639+0000 7f01c2600700 1 --2- 172.21.15.153:0/2691379530 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f01bc10d280 0x7f01bc19d1f0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f01a4006010 tx=0x7f01a400be20 comp rx=0 tx=0).stop 2024-03-21T10:31:24.639 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:24.639+0000 7f01c2600700 1 -- 172.21.15.153:0/2691379530 shutdown_connections 2024-03-21T10:31:24.639 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:24.639+0000 7f01c2600700 1 --2- 172.21.15.153:0/2691379530 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f01bc1071d0 0x7f01bc19ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:24.639 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:24.639+0000 7f01c2600700 1 --2- 172.21.15.153:0/2691379530 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f01a006e460 0x7f01a0070920 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:24.639 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:24.639+0000 7f01c2600700 1 --2- 172.21.15.153:0/2691379530 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f01bc10d280 0x7f01bc19d1f0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:24.639 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:24.639+0000 7f01c2600700 1 -- 172.21.15.153:0/2691379530 >> 172.21.15.153:0/2691379530 conn(0x7f01bc0769e0 msgr2=0x7f01bc078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:24.639 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:24.640+0000 7f01c2600700 1 -- 172.21.15.153:0/2691379530 shutdown_connections 2024-03-21T10:31:24.639 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:24.640+0000 7f01c2600700 1 -- 172.21.15.153:0/2691379530 wait complete. 2024-03-21T10:31:24.787 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:24.788+0000 7f36b7e00700 1 -- 172.21.15.153:0/1740482238 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f36b8075be0 msgr2=0x7f36b8111330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:24.787 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:24.788+0000 7f36b7e00700 1 --2- 172.21.15.153:0/1740482238 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f36b8075be0 0x7f36b8111330 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f36a4009910 tx=0x7f36a4009c20 comp rx=0 tx=0).stop 2024-03-21T10:31:24.787 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:24.788+0000 7f36b7e00700 1 -- 172.21.15.153:0/1740482238 shutdown_connections 2024-03-21T10:31:24.787 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:24.788+0000 7f36b7e00700 1 --2- 172.21.15.153:0/1740482238 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f36b8075be0 0x7f36b8111330 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:24.788 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:24.788+0000 7f36b7e00700 1 --2- 172.21.15.153:0/1740482238 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f36b80752c0 0x7f36b80756a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:24.788 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:24.788+0000 7f36b7e00700 1 -- 172.21.15.153:0/1740482238 >> 172.21.15.153:0/1740482238 conn(0x7f36b80fe700 msgr2=0x7f36b8100b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:24.788 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:24.788+0000 7f36b7e00700 1 -- 172.21.15.153:0/1740482238 shutdown_connections 2024-03-21T10:31:24.788 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:24.788+0000 7f36b7e00700 1 -- 172.21.15.153:0/1740482238 wait complete. 2024-03-21T10:31:24.788 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:24.789+0000 7f36b7e00700 1 Processor -- start 2024-03-21T10:31:24.788 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:24.789+0000 7f36b7e00700 1 -- start start 2024-03-21T10:31:24.789 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:24.789+0000 7f36b7e00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f36b80752c0 0x7f36b81a2900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:24.789 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:24.789+0000 7f36b7e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f36b8075be0 0x7f36b81a2e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:24.789 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:24.789+0000 7f36b7e00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36b81a34d0 con 0x7f36b8075be0 2024-03-21T10:31:24.789 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:24.789+0000 7f36b7e00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36b819c980 con 0x7f36b80752c0 2024-03-21T10:31:24.790 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:24.790+0000 7f36b6000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f36b8075be0 0x7f36b81a2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:24.790 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:24.790+0000 7f36b6000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f36b8075be0 0x7f36b81a2e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41566/0 (socket says 172.21.15.153:41566) 2024-03-21T10:31:24.790 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:24.790+0000 7f36b6000700 1 -- 172.21.15.153:0/1530513865 learned_addr learned my addr 172.21.15.153:0/1530513865 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:31:24.790 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:24.790+0000 7f36b6000700 1 -- 172.21.15.153:0/1530513865 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f36b80752c0 msgr2=0x7f36b81a2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:24.790 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:24.790+0000 7f36b6000700 1 --2- 172.21.15.153:0/1530513865 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f36b80752c0 0x7f36b81a2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:24.790 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:24.790+0000 7f36b6000700 1 -- 172.21.15.153:0/1530513865 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36a40095f0 con 0x7f36b8075be0 2024-03-21T10:31:24.791 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:24.790+0000 7f36b6000700 1 --2- 172.21.15.153:0/1530513865 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f36b8075be0 0x7f36b81a2e40 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f36a4006010 tx=0x7f36a40056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:24.791 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:24.791+0000 7f36af400700 1 -- 172.21.15.153:0/1530513865 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f36a401d070 con 0x7f36b8075be0 2024-03-21T10:31:24.791 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:24.791+0000 7f36af400700 1 -- 172.21.15.153:0/1530513865 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f36a400f460 con 0x7f36b8075be0 2024-03-21T10:31:24.791 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:24.791+0000 7f36b7e00700 1 -- 172.21.15.153:0/1530513865 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36b819cc60 con 0x7f36b8075be0 2024-03-21T10:31:24.791 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:24.791+0000 7f36af400700 1 -- 172.21.15.153:0/1530513865 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f36a4021640 con 0x7f36b8075be0 2024-03-21T10:31:24.791 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:24.791+0000 7f36b7e00700 1 -- 172.21.15.153:0/1530513865 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36b819d1b0 con 0x7f36b8075be0 2024-03-21T10:31:24.791 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:24.792+0000 7f36b7e00700 1 -- 172.21.15.153:0/1530513865 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36b810eaa0 con 0x7f36b8075be0 2024-03-21T10:31:24.792 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:24.793+0000 7f36af400700 1 -- 172.21.15.153:0/1530513865 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f36a400f5d0 con 0x7f36b8075be0 2024-03-21T10:31:24.793 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:24.794+0000 7f36af400700 1 --2- 172.21.15.153:0/1530513865 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f369c06e460 0x7f369c070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:24.793 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:24.794+0000 7f36af400700 1 -- 172.21.15.153:0/1530513865 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f36a408f850 con 0x7f36b8075be0 2024-03-21T10:31:24.793 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:24.794+0000 7f36b6a00700 1 --2- 172.21.15.153:0/1530513865 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f369c06e460 0x7f369c070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:24.794 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:24.795+0000 7f36b6a00700 1 --2- 172.21.15.153:0/1530513865 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f369c06e460 0x7f369c070920 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f36a8005950 tx=0x7f36a80058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:24.798 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:24.799+0000 7f36af400700 1 -- 172.21.15.153:0/1530513865 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f36a405c3a0 con 0x7f36b8075be0 2024-03-21T10:31:25.092 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:25.093+0000 7f36b7e00700 1 -- 172.21.15.153:0/1530513865 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f36b819d430 con 0x7f36b8075be0 2024-03-21T10:31:25.093 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:25.094+0000 7f36af400700 1 -- 172.21.15.153:0/1530513865 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+805 (secure 0 0 0) 0x7f36a405bf30 con 0x7f36b8075be0 2024-03-21T10:31:25.093 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:31:25.093 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": { 2024-03-21T10:31:25.093 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:31:25.093 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:31:25.094 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": { 2024-03-21T10:31:25.094 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:31:25.094 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:31:25.094 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": { 2024-03-21T10:31:25.094 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 6 2024-03-21T10:31:25.094 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:31:25.094 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": { 2024-03-21T10:31:25.094 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-03-21T10:31:25.094 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 3 2024-03-21T10:31:25.094 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:31:25.094 INFO:teuthology.orchestra.run.smithi153.stdout: "overall": { 2024-03-21T10:31:25.094 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-03-21T10:31:25.094 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 13 2024-03-21T10:31:25.095 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-03-21T10:31:25.095 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:31:25.096 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:25.096+0000 7f36b7e00700 1 -- 172.21.15.153:0/1530513865 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f369c06e460 msgr2=0x7f369c070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:25.096 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:25.096+0000 7f36b7e00700 1 --2- 172.21.15.153:0/1530513865 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f369c06e460 0x7f369c070920 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f36a8005950 tx=0x7f36a80058e0 comp rx=0 tx=0).stop 2024-03-21T10:31:25.096 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:25.096+0000 7f36b7e00700 1 -- 172.21.15.153:0/1530513865 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f36b8075be0 msgr2=0x7f36b81a2e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:25.096 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:25.096+0000 7f36b7e00700 1 --2- 172.21.15.153:0/1530513865 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f36b8075be0 0x7f36b81a2e40 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f36a4006010 tx=0x7f36a40056a0 comp rx=0 tx=0).stop 2024-03-21T10:31:25.096 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:25.097+0000 7f36b7e00700 1 -- 172.21.15.153:0/1530513865 shutdown_connections 2024-03-21T10:31:25.097 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:25.097+0000 7f36b7e00700 1 --2- 172.21.15.153:0/1530513865 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f36b80752c0 0x7f36b81a2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:25.097 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:25.097+0000 7f36b7e00700 1 --2- 172.21.15.153:0/1530513865 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f369c06e460 0x7f369c070920 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:25.097 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:25.097+0000 7f36b7e00700 1 --2- 172.21.15.153:0/1530513865 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f36b8075be0 0x7f36b81a2e40 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:25.097 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:25.097+0000 7f36b7e00700 1 -- 172.21.15.153:0/1530513865 >> 172.21.15.153:0/1530513865 conn(0x7f36b80fe700 msgr2=0x7f36b80ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:25.097 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:25.097+0000 7f36b7e00700 1 -- 172.21.15.153:0/1530513865 shutdown_connections 2024-03-21T10:31:25.097 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:25.097+0000 7f36b7e00700 1 -- 172.21.15.153:0/1530513865 wait complete. 2024-03-21T10:31:25.237 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:25.238+0000 7f1867e00700 1 -- 172.21.15.153:0/2841380708 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1860073a80 msgr2=0x7f186010cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:25.237 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:25.238+0000 7f1867e00700 1 --2- 172.21.15.153:0/2841380708 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1860073a80 0x7f186010cea0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f184c009910 tx=0x7f184c009c20 comp rx=0 tx=0).stop 2024-03-21T10:31:25.238 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:25.238+0000 7f1867e00700 1 -- 172.21.15.153:0/2841380708 shutdown_connections 2024-03-21T10:31:25.238 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:25.238+0000 7f1867e00700 1 --2- 172.21.15.153:0/2841380708 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1860073a80 0x7f186010cea0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:25.238 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:25.238+0000 7f1867e00700 1 --2- 172.21.15.153:0/2841380708 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1860073160 0x7f1860073540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:25.238 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:25.238+0000 7f1867e00700 1 -- 172.21.15.153:0/2841380708 >> 172.21.15.153:0/2841380708 conn(0x7f18600fc630 msgr2=0x7f18600fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:25.238 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:25.238+0000 7f1867e00700 1 -- 172.21.15.153:0/2841380708 shutdown_connections 2024-03-21T10:31:25.238 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:25.238+0000 7f1867e00700 1 -- 172.21.15.153:0/2841380708 wait complete. 2024-03-21T10:31:25.239 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:25.239+0000 7f1867e00700 1 Processor -- start 2024-03-21T10:31:25.239 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:25.239+0000 7f1867e00700 1 -- start start 2024-03-21T10:31:25.239 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:25.239+0000 7f1867e00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1860073160 0x7f186019e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:25.239 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:25.240+0000 7f1867e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1860073a80 0x7f186019ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:25.239 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:25.240+0000 7f1867e00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f186019f2d0 con 0x7f1860073a80 2024-03-21T10:31:25.239 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:25.240+0000 7f1867e00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1860198780 con 0x7f1860073160 2024-03-21T10:31:25.239 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:25.240+0000 7f1864a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1860073a80 0x7f186019ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:25.239 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:25.240+0000 7f1864a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1860073a80 0x7f186019ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41572/0 (socket says 172.21.15.153:41572) 2024-03-21T10:31:25.239 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:25.240+0000 7f1864a00700 1 -- 172.21.15.153:0/27942613 learned_addr learned my addr 172.21.15.153:0/27942613 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:31:25.240 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:25.240+0000 7f1864a00700 1 -- 172.21.15.153:0/27942613 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1860073160 msgr2=0x7f186019e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:25.240 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:25.240+0000 7f1864a00700 1 --2- 172.21.15.153:0/27942613 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1860073160 0x7f186019e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:25.240 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:25.240+0000 7f1864a00700 1 -- 172.21.15.153:0/27942613 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f184c0095f0 con 0x7f1860073a80 2024-03-21T10:31:25.241 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:25.240+0000 7f1864a00700 1 --2- 172.21.15.153:0/27942613 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1860073a80 0x7f186019ec40 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f184c006010 tx=0x7f184c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:25.241 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:25.241+0000 7f185e000700 1 -- 172.21.15.153:0/27942613 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f184c01d070 con 0x7f1860073a80 2024-03-21T10:31:25.241 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:25.241+0000 7f185e000700 1 -- 172.21.15.153:0/27942613 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f184c004fb0 con 0x7f1860073a80 2024-03-21T10:31:25.241 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:25.241+0000 7f185e000700 1 -- 172.21.15.153:0/27942613 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f184c00f480 con 0x7f1860073a80 2024-03-21T10:31:25.241 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:25.241+0000 7f1867e00700 1 -- 172.21.15.153:0/27942613 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1860198a00 con 0x7f1860073a80 2024-03-21T10:31:25.241 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:25.241+0000 7f1867e00700 1 -- 172.21.15.153:0/27942613 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1860198ea0 con 0x7f1860073a80 2024-03-21T10:31:25.241 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:25.242+0000 7f1867e00700 1 -- 172.21.15.153:0/27942613 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f186010a5a0 con 0x7f1860073a80 2024-03-21T10:31:25.242 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:25.243+0000 7f185e000700 1 -- 172.21.15.153:0/27942613 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f184c022890 con 0x7f1860073a80 2024-03-21T10:31:25.243 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:25.244+0000 7f185e000700 1 --2- 172.21.15.153:0/27942613 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f184806e460 0x7f1848070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:25.243 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:25.244+0000 7f185e000700 1 -- 172.21.15.153:0/27942613 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f184c08fc90 con 0x7f1860073a80 2024-03-21T10:31:25.243 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:25.244+0000 7f1865400700 1 --2- 172.21.15.153:0/27942613 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f184806e460 0x7f1848070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:25.244 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:25.245+0000 7f1865400700 1 --2- 172.21.15.153:0/27942613 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f184806e460 0x7f1848070920 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f185400a990 tx=0x7f1854005c80 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:25.248 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:25.249+0000 7f185e000700 1 -- 172.21.15.153:0/27942613 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f184c05c890 con 0x7f1860073a80 2024-03-21T10:31:25.304 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:31:25 smithi153 ceph-mon[113733]: pgmap v586: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:31:25.304 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:31:25 smithi153 ceph-mon[113733]: from='client.35066 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:31:25.304 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:31:25 smithi153 ceph-mon[113733]: from='client.35070 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:31:25.449 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:31:25 smithi174 ceph-mon[88004]: pgmap v586: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:31:25.449 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:31:25 smithi174 ceph-mon[88004]: from='client.35066 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:31:25.449 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:31:25 smithi174 ceph-mon[88004]: from='client.35070 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:31:25.501 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:25.501+0000 7f1867e00700 1 -- 172.21.15.153:0/27942613 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f186004ea90 con 0x7f1860073a80 2024-03-21T10:31:25.501 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:25.502+0000 7f185e000700 1 -- 172.21.15.153:0/27942613 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1876 (secure 0 0 0) 0x7f184c05c420 con 0x7f1860073a80 2024-03-21T10:31:25.503 INFO:teuthology.orchestra.run.smithi153.stdout:e30 2024-03-21T10:31:25.503 INFO:teuthology.orchestra.run.smithi153.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-21T10:31:25.503 INFO:teuthology.orchestra.run.smithi153.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:31:25.503 INFO:teuthology.orchestra.run.smithi153.stdout:legacy client fscid: 1 2024-03-21T10:31:25.503 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:31:25.503 INFO:teuthology.orchestra.run.smithi153.stdout:Filesystem 'cephfs' (1) 2024-03-21T10:31:25.503 INFO:teuthology.orchestra.run.smithi153.stdout:fs_name cephfs 2024-03-21T10:31:25.503 INFO:teuthology.orchestra.run.smithi153.stdout:epoch 23 2024-03-21T10:31:25.503 INFO:teuthology.orchestra.run.smithi153.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-21T10:31:25.503 INFO:teuthology.orchestra.run.smithi153.stdout:created 2024-03-21T10:04:27.086576+0000 2024-03-21T10:31:25.503 INFO:teuthology.orchestra.run.smithi153.stdout:modified 2024-03-21T10:19:05.022092+0000 2024-03-21T10:31:25.503 INFO:teuthology.orchestra.run.smithi153.stdout:tableserver 0 2024-03-21T10:31:25.503 INFO:teuthology.orchestra.run.smithi153.stdout:root 0 2024-03-21T10:31:25.504 INFO:teuthology.orchestra.run.smithi153.stdout:session_timeout 60 2024-03-21T10:31:25.504 INFO:teuthology.orchestra.run.smithi153.stdout:session_autoclose 300 2024-03-21T10:31:25.504 INFO:teuthology.orchestra.run.smithi153.stdout:max_file_size 1099511627776 2024-03-21T10:31:25.504 INFO:teuthology.orchestra.run.smithi153.stdout:max_xattr_size 65536 2024-03-21T10:31:25.504 INFO:teuthology.orchestra.run.smithi153.stdout:required_client_features {} 2024-03-21T10:31:25.504 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure 0 2024-03-21T10:31:25.504 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure_osd_epoch 94 2024-03-21T10:31:25.504 INFO:teuthology.orchestra.run.smithi153.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:31:25.504 INFO:teuthology.orchestra.run.smithi153.stdout:max_mds 2 2024-03-21T10:31:25.504 INFO:teuthology.orchestra.run.smithi153.stdout:in 0,1 2024-03-21T10:31:25.504 INFO:teuthology.orchestra.run.smithi153.stdout:up {} 2024-03-21T10:31:25.504 INFO:teuthology.orchestra.run.smithi153.stdout:failed 0,1 2024-03-21T10:31:25.504 INFO:teuthology.orchestra.run.smithi153.stdout:damaged 2024-03-21T10:31:25.504 INFO:teuthology.orchestra.run.smithi153.stdout:stopped 2024-03-21T10:31:25.505 INFO:teuthology.orchestra.run.smithi153.stdout:data_pools [3] 2024-03-21T10:31:25.505 INFO:teuthology.orchestra.run.smithi153.stdout:metadata_pool 2 2024-03-21T10:31:25.505 INFO:teuthology.orchestra.run.smithi153.stdout:inline_data enabled 2024-03-21T10:31:25.505 INFO:teuthology.orchestra.run.smithi153.stdout:balancer 2024-03-21T10:31:25.505 INFO:teuthology.orchestra.run.smithi153.stdout:bal_rank_mask -1 2024-03-21T10:31:25.505 INFO:teuthology.orchestra.run.smithi153.stdout:standby_count_wanted 1 2024-03-21T10:31:25.505 INFO:teuthology.orchestra.run.smithi153.stdout:qdb_cluster leader: 0 members: 2024-03-21T10:31:25.505 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:31:25.505 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:31:25.505 INFO:teuthology.orchestra.run.smithi153.stdout:Standby daemons: 2024-03-21T10:31:25.505 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:31:25.505 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.zipckr{-1:34442} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6826/1103846295,v1:172.21.15.153:6827/1103846295] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:31:25.505 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.iuyluj{-1:34474} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6828/1738630832,v1:172.21.15.153:6829/1738630832] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:31:25.506 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.nhozag{-1:44269} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.174:6826/3967223227,v1:172.21.15.174:6827/3967223227] compat {c=[1],r=[1],i=[7ff]}] 2024-03-21T10:31:25.506 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.yyimst{-1:44313} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.174:6824/4127786797,v1:172.21.15.174:6825/4127786797] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:31:25.506 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:25.505+0000 7f1867e00700 1 -- 172.21.15.153:0/27942613 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f184806e460 msgr2=0x7f1848070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:25.506 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:25.505+0000 7f1867e00700 1 --2- 172.21.15.153:0/27942613 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f184806e460 0x7f1848070920 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f185400a990 tx=0x7f1854005c80 comp rx=0 tx=0).stop 2024-03-21T10:31:25.507 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:25.505+0000 7f1867e00700 1 -- 172.21.15.153:0/27942613 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1860073a80 msgr2=0x7f186019ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:25.507 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:25.505+0000 7f1867e00700 1 --2- 172.21.15.153:0/27942613 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1860073a80 0x7f186019ec40 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f184c006010 tx=0x7f184c00be20 comp rx=0 tx=0).stop 2024-03-21T10:31:25.507 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:25.505+0000 7f1867e00700 1 -- 172.21.15.153:0/27942613 shutdown_connections 2024-03-21T10:31:25.507 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:25.505+0000 7f1867e00700 1 --2- 172.21.15.153:0/27942613 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1860073160 0x7f186019e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:25.507 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:25.505+0000 7f1867e00700 1 --2- 172.21.15.153:0/27942613 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f184806e460 0x7f1848070920 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:25.507 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:25.505+0000 7f1867e00700 1 --2- 172.21.15.153:0/27942613 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1860073a80 0x7f186019ec40 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:25.507 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:25.505+0000 7f1867e00700 1 -- 172.21.15.153:0/27942613 >> 172.21.15.153:0/27942613 conn(0x7f18600fc630 msgr2=0x7f18601076e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:25.507 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:25.505+0000 7f1867e00700 1 -- 172.21.15.153:0/27942613 shutdown_connections 2024-03-21T10:31:25.507 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:25.505+0000 7f1867e00700 1 -- 172.21.15.153:0/27942613 wait complete. 2024-03-21T10:31:25.507 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 30 2024-03-21T10:31:25.660 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:25.661+0000 7fd771200700 1 -- 172.21.15.153:0/1778285680 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd76c10d280 msgr2=0x7fd76c10d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:25.660 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:25.661+0000 7fd771200700 1 --2- 172.21.15.153:0/1778285680 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd76c10d280 0x7fd76c10d660 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fd75c009910 tx=0x7fd75c009c20 comp rx=0 tx=0).stop 2024-03-21T10:31:25.660 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:25.661+0000 7fd771200700 1 -- 172.21.15.153:0/1778285680 shutdown_connections 2024-03-21T10:31:25.661 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:25.661+0000 7fd771200700 1 --2- 172.21.15.153:0/1778285680 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd76c1071d0 0x7fd76c107650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:25.661 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:25.661+0000 7fd771200700 1 --2- 172.21.15.153:0/1778285680 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd76c10d280 0x7fd76c10d660 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:25.661 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:25.661+0000 7fd771200700 1 -- 172.21.15.153:0/1778285680 >> 172.21.15.153:0/1778285680 conn(0x7fd76c0769e0 msgr2=0x7fd76c078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:25.661 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:25.661+0000 7fd771200700 1 -- 172.21.15.153:0/1778285680 shutdown_connections 2024-03-21T10:31:25.661 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:25.661+0000 7fd771200700 1 -- 172.21.15.153:0/1778285680 wait complete. 2024-03-21T10:31:25.662 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:25.662+0000 7fd771200700 1 Processor -- start 2024-03-21T10:31:25.662 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:25.662+0000 7fd771200700 1 -- start start 2024-03-21T10:31:25.662 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:25.662+0000 7fd771200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd76c1071d0 0x7fd76c19ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:25.662 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:25.662+0000 7fd771200700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd76c10d280 0x7fd76c19d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:25.662 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:25.662+0000 7fd771200700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd76c19d810 con 0x7fd76c1071d0 2024-03-21T10:31:25.662 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:25.662+0000 7fd771200700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd76c19d980 con 0x7fd76c10d280 2024-03-21T10:31:25.662 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:25.663+0000 7fd769e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd76c1071d0 0x7fd76c19ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:25.663 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:25.663+0000 7fd769e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd76c1071d0 0x7fd76c19ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41588/0 (socket says 172.21.15.153:41588) 2024-03-21T10:31:25.663 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:25.663+0000 7fd769e00700 1 -- 172.21.15.153:0/1363996861 learned_addr learned my addr 172.21.15.153:0/1363996861 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:31:25.664 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:25.663+0000 7fd769e00700 1 -- 172.21.15.153:0/1363996861 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd76c10d280 msgr2=0x7fd76c19d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:25.664 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:25.663+0000 7fd769400700 1 --2- 172.21.15.153:0/1363996861 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd76c10d280 0x7fd76c19d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:25.664 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:25.663+0000 7fd769e00700 1 --2- 172.21.15.153:0/1363996861 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd76c10d280 0x7fd76c19d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:25.664 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:25.663+0000 7fd769e00700 1 -- 172.21.15.153:0/1363996861 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd75c0095f0 con 0x7fd76c1071d0 2024-03-21T10:31:25.664 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:25.664+0000 7fd769e00700 1 --2- 172.21.15.153:0/1363996861 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd76c1071d0 0x7fd76c19ccd0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fd75c00b5c0 tx=0x7fd75c00be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:25.664 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:25.664+0000 7fd762a00700 1 -- 172.21.15.153:0/1363996861 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd75c01d070 con 0x7fd76c1071d0 2024-03-21T10:31:25.664 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:25.664+0000 7fd762a00700 1 -- 172.21.15.153:0/1363996861 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd75c00f460 con 0x7fd76c1071d0 2024-03-21T10:31:25.664 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:25.664+0000 7fd771200700 1 -- 172.21.15.153:0/1363996861 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd76c1a06c0 con 0x7fd76c1071d0 2024-03-21T10:31:25.664 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:25.664+0000 7fd762a00700 1 -- 172.21.15.153:0/1363996861 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd75c017610 con 0x7fd76c1071d0 2024-03-21T10:31:25.664 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:25.664+0000 7fd771200700 1 -- 172.21.15.153:0/1363996861 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd76c1a0c30 con 0x7fd76c1071d0 2024-03-21T10:31:25.665 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:25.665+0000 7fd771200700 1 -- 172.21.15.153:0/1363996861 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd76c04ea90 con 0x7fd76c1071d0 2024-03-21T10:31:25.665 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:25.666+0000 7fd762a00700 1 -- 172.21.15.153:0/1363996861 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fd75c00f5d0 con 0x7fd76c1071d0 2024-03-21T10:31:25.667 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:25.668+0000 7fd762a00700 1 --2- 172.21.15.153:0/1363996861 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd75006e460 0x7fd750070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:25.667 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:25.668+0000 7fd762a00700 1 -- 172.21.15.153:0/1363996861 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fd75c08faf0 con 0x7fd76c1071d0 2024-03-21T10:31:25.667 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:25.668+0000 7fd769400700 1 --2- 172.21.15.153:0/1363996861 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd75006e460 0x7fd750070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:25.668 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:25.669+0000 7fd769400700 1 --2- 172.21.15.153:0/1363996861 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd75006e460 0x7fd750070920 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fd76c19e2b0 tx=0x7fd758005d50 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:25.672 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:25.673+0000 7fd762a00700 1 -- 172.21.15.153:0/1363996861 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd75c058bb0 con 0x7fd76c1071d0 2024-03-21T10:31:25.902 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:25.903+0000 7fd771200700 1 -- 172.21.15.153:0/1363996861 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd76c1a10f0 con 0x7fd75006e460 2024-03-21T10:31:25.904 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:25.905+0000 7fd762a00700 1 -- 172.21.15.153:0/1363996861 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fd76c1a10f0 con 0x7fd75006e460 2024-03-21T10:31:25.904 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:31:25.904 INFO:teuthology.orchestra.run.smithi153.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232", 2024-03-21T10:31:25.905 INFO:teuthology.orchestra.run.smithi153.stdout: "in_progress": true, 2024-03-21T10:31:25.905 INFO:teuthology.orchestra.run.smithi153.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-21T10:31:25.905 INFO:teuthology.orchestra.run.smithi153.stdout: "services_complete": [ 2024-03-21T10:31:25.905 INFO:teuthology.orchestra.run.smithi153.stdout: "crash", 2024-03-21T10:31:25.905 INFO:teuthology.orchestra.run.smithi153.stdout: "mon", 2024-03-21T10:31:25.905 INFO:teuthology.orchestra.run.smithi153.stdout: "osd", 2024-03-21T10:31:25.905 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr" 2024-03-21T10:31:25.905 INFO:teuthology.orchestra.run.smithi153.stdout: ], 2024-03-21T10:31:25.905 INFO:teuthology.orchestra.run.smithi153.stdout: "progress": "15/23 daemons upgraded", 2024-03-21T10:31:25.905 INFO:teuthology.orchestra.run.smithi153.stdout: "message": "Currently upgrading mds daemons", 2024-03-21T10:31:25.905 INFO:teuthology.orchestra.run.smithi153.stdout: "is_paused": false 2024-03-21T10:31:25.905 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:31:25.907 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:25.907+0000 7fd771200700 1 -- 172.21.15.153:0/1363996861 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd75006e460 msgr2=0x7fd750070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:25.907 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:25.907+0000 7fd771200700 1 --2- 172.21.15.153:0/1363996861 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd75006e460 0x7fd750070920 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fd76c19e2b0 tx=0x7fd758005d50 comp rx=0 tx=0).stop 2024-03-21T10:31:25.907 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:25.907+0000 7fd771200700 1 -- 172.21.15.153:0/1363996861 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd76c1071d0 msgr2=0x7fd76c19ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:25.907 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:25.907+0000 7fd771200700 1 --2- 172.21.15.153:0/1363996861 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd76c1071d0 0x7fd76c19ccd0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fd75c00b5c0 tx=0x7fd75c00be00 comp rx=0 tx=0).stop 2024-03-21T10:31:25.907 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:25.908+0000 7fd771200700 1 -- 172.21.15.153:0/1363996861 shutdown_connections 2024-03-21T10:31:25.907 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:25.908+0000 7fd771200700 1 --2- 172.21.15.153:0/1363996861 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd75006e460 0x7fd750070920 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:25.907 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:25.908+0000 7fd771200700 1 --2- 172.21.15.153:0/1363996861 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd76c1071d0 0x7fd76c19ccd0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:25.907 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:25.908+0000 7fd771200700 1 --2- 172.21.15.153:0/1363996861 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd76c10d280 0x7fd76c19d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:25.907 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:25.908+0000 7fd771200700 1 -- 172.21.15.153:0/1363996861 >> 172.21.15.153:0/1363996861 conn(0x7fd76c0769e0 msgr2=0x7fd76c078320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:25.908 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:25.908+0000 7fd771200700 1 -- 172.21.15.153:0/1363996861 shutdown_connections 2024-03-21T10:31:25.908 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:25.908+0000 7fd771200700 1 -- 172.21.15.153:0/1363996861 wait complete. 2024-03-21T10:31:26.064 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:26.064+0000 7f3241e00700 1 -- 172.21.15.153:0/3871980324 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f323c100da0 msgr2=0x7f323c101180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:26.064 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:26.064+0000 7f3241e00700 1 --2- 172.21.15.153:0/3871980324 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f323c100da0 0x7f323c101180 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f322c009910 tx=0x7f322c009c20 comp rx=0 tx=0).stop 2024-03-21T10:31:26.064 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:26.064+0000 7f3241e00700 1 -- 172.21.15.153:0/3871980324 shutdown_connections 2024-03-21T10:31:26.064 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:26.064+0000 7f3241e00700 1 --2- 172.21.15.153:0/3871980324 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f323c101750 0x7f323c1057a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:26.064 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:26.064+0000 7f3241e00700 1 --2- 172.21.15.153:0/3871980324 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f323c100da0 0x7f323c101180 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:26.064 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:26.064+0000 7f3241e00700 1 -- 172.21.15.153:0/3871980324 >> 172.21.15.153:0/3871980324 conn(0x7f323c0fc630 msgr2=0x7f323c0fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:26.064 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:26.064+0000 7f3241e00700 1 -- 172.21.15.153:0/3871980324 shutdown_connections 2024-03-21T10:31:26.064 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:26.064+0000 7f3241e00700 1 -- 172.21.15.153:0/3871980324 wait complete. 2024-03-21T10:31:26.065 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:26.065+0000 7f3241e00700 1 Processor -- start 2024-03-21T10:31:26.065 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:26.065+0000 7f3241e00700 1 -- start start 2024-03-21T10:31:26.065 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:26.066+0000 7f3241e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f323c100da0 0x7f323c10bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:26.065 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:26.066+0000 7f3241e00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f323c101750 0x7f323c10c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:26.066 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:26.066+0000 7f3241e00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f323c10fca0 con 0x7f323c100da0 2024-03-21T10:31:26.066 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:26.066+0000 7f3241e00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f323c10c590 con 0x7f323c101750 2024-03-21T10:31:26.066 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:26.066+0000 7f323aa00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f323c100da0 0x7f323c10bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:26.066 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:26.066+0000 7f323aa00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f323c100da0 0x7f323c10bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41612/0 (socket says 172.21.15.153:41612) 2024-03-21T10:31:26.066 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:26.066+0000 7f323aa00700 1 -- 172.21.15.153:0/3587102216 learned_addr learned my addr 172.21.15.153:0/3587102216 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:31:26.067 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:26.066+0000 7f323a000700 1 --2- 172.21.15.153:0/3587102216 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f323c101750 0x7f323c10c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:26.067 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:26.067+0000 7f323aa00700 1 -- 172.21.15.153:0/3587102216 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f323c101750 msgr2=0x7f323c10c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:26.067 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:26.067+0000 7f323aa00700 1 --2- 172.21.15.153:0/3587102216 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f323c101750 0x7f323c10c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:26.067 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:26.067+0000 7f323aa00700 1 -- 172.21.15.153:0/3587102216 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f322c0095f0 con 0x7f323c100da0 2024-03-21T10:31:26.067 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:26.067+0000 7f323aa00700 1 --2- 172.21.15.153:0/3587102216 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f323c100da0 0x7f323c10bb10 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f322c0092e0 tx=0x7f322c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:26.067 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:26.067+0000 7f3240a00700 1 -- 172.21.15.153:0/3587102216 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f322c01d070 con 0x7f323c100da0 2024-03-21T10:31:26.067 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:26.067+0000 7f3240a00700 1 -- 172.21.15.153:0/3587102216 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f322c005e00 con 0x7f323c100da0 2024-03-21T10:31:26.067 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:26.067+0000 7f3240a00700 1 -- 172.21.15.153:0/3587102216 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f322c017400 con 0x7f323c100da0 2024-03-21T10:31:26.068 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:26.067+0000 7f3241e00700 1 -- 172.21.15.153:0/3587102216 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f323c10c810 con 0x7f323c100da0 2024-03-21T10:31:26.068 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:26.067+0000 7f3241e00700 1 -- 172.21.15.153:0/3587102216 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f323c1a70d0 con 0x7f323c100da0 2024-03-21T10:31:26.068 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:26.069+0000 7f3241e00700 1 -- 172.21.15.153:0/3587102216 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f323c109140 con 0x7f323c100da0 2024-03-21T10:31:26.069 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:26.070+0000 7f3240a00700 1 -- 172.21.15.153:0/3587102216 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f322c00f460 con 0x7f323c100da0 2024-03-21T10:31:26.071 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:26.071+0000 7f3240a00700 1 --2- 172.21.15.153:0/3587102216 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f322006e460 0x7f3220070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:26.071 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:26.072+0000 7f3240a00700 1 -- 172.21.15.153:0/3587102216 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f322c08f950 con 0x7f323c100da0 2024-03-21T10:31:26.071 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:26.072+0000 7f323a000700 1 --2- 172.21.15.153:0/3587102216 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f322006e460 0x7f3220070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:26.071 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:26.072+0000 7f323a000700 1 --2- 172.21.15.153:0/3587102216 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f322006e460 0x7f3220070920 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f3228005950 tx=0x7f32280058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:26.076 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:26.077+0000 7f3240a00700 1 -- 172.21.15.153:0/3587102216 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f322c058a90 con 0x7f323c100da0 2024-03-21T10:31:26.304 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:31:26 smithi153 ceph-mon[113733]: from='client.35074 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:31:26.304 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:31:26 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/1530513865' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:31:26.304 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:31:26 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/27942613' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:31:26.383 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:26.383+0000 7f3241e00700 1 -- 172.21.15.153:0/3587102216 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f323c04ea90 con 0x7f323c100da0 2024-03-21T10:31:26.384 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:26.384+0000 7f3240a00700 1 -- 172.21.15.153:0/3587102216 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f322c05c150 con 0x7f323c100da0 2024-03-21T10:31:26.384 INFO:teuthology.orchestra.run.smithi153.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-03-21T10:31:26.384 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-21T10:31:26.384 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is degraded 2024-03-21T10:31:26.384 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-21T10:31:26.384 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-21T10:31:26.384 INFO:teuthology.orchestra.run.smithi153.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-21T10:31:26.384 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-21T10:31:26.386 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:26.387+0000 7f3241e00700 1 -- 172.21.15.153:0/3587102216 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f322006e460 msgr2=0x7f3220070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:26.386 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:26.387+0000 7f3241e00700 1 --2- 172.21.15.153:0/3587102216 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f322006e460 0x7f3220070920 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f3228005950 tx=0x7f32280058e0 comp rx=0 tx=0).stop 2024-03-21T10:31:26.386 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:26.387+0000 7f3241e00700 1 -- 172.21.15.153:0/3587102216 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f323c100da0 msgr2=0x7f323c10bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:26.386 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:26.387+0000 7f3241e00700 1 --2- 172.21.15.153:0/3587102216 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f323c100da0 0x7f323c10bb10 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f322c0092e0 tx=0x7f322c00be20 comp rx=0 tx=0).stop 2024-03-21T10:31:26.387 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:26.387+0000 7f3241e00700 1 -- 172.21.15.153:0/3587102216 shutdown_connections 2024-03-21T10:31:26.387 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:26.387+0000 7f3241e00700 1 --2- 172.21.15.153:0/3587102216 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f322006e460 0x7f3220070920 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:26.387 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:26.387+0000 7f3241e00700 1 --2- 172.21.15.153:0/3587102216 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f323c100da0 0x7f323c10bb10 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:26.387 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:26.387+0000 7f3241e00700 1 --2- 172.21.15.153:0/3587102216 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f323c101750 0x7f323c10c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:26.387 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:26.387+0000 7f3241e00700 1 -- 172.21.15.153:0/3587102216 >> 172.21.15.153:0/3587102216 conn(0x7f323c0fc630 msgr2=0x7f323c0fdcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:26.387 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:26.387+0000 7f3241e00700 1 -- 172.21.15.153:0/3587102216 shutdown_connections 2024-03-21T10:31:26.387 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:26.387+0000 7f3241e00700 1 -- 172.21.15.153:0/3587102216 wait complete. 2024-03-21T10:31:26.449 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:31:26 smithi174 ceph-mon[88004]: from='client.35074 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:31:26.449 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:31:26 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/1530513865' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:31:26.449 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:31:26 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/27942613' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:31:27.304 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:31:27 smithi153 ceph-mon[113733]: pgmap v587: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:31:27.305 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:31:27 smithi153 ceph-mon[113733]: from='client.35086 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:31:27.305 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:31:27 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/3587102216' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:31:27.449 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:31:27 smithi174 ceph-mon[88004]: pgmap v587: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:31:27.449 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:31:27 smithi174 ceph-mon[88004]: from='client.35086 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:31:27.449 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:31:27 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/3587102216' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:31:29.304 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:31:29 smithi153 ceph-mon[113733]: pgmap v588: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:31:29.449 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:31:29 smithi174 ceph-mon[88004]: pgmap v588: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:31:31.304 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:31:31 smithi153 ceph-mon[113733]: pgmap v589: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:31:31.449 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:31:31 smithi174 ceph-mon[88004]: pgmap v589: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:31:32.304 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:31:32 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:31:32.449 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:31:32 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:31:33.449 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:31:33 smithi174 ceph-mon[88004]: pgmap v590: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:31:33.554 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:31:33 smithi153 ceph-mon[113733]: pgmap v590: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:31:35.449 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:31:35 smithi174 ceph-mon[88004]: pgmap v591: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:31:35.554 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:31:35 smithi153 ceph-mon[113733]: pgmap v591: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:31:37.449 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:31:37 smithi174 ceph-mon[88004]: pgmap v592: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:31:37.554 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:31:37 smithi153 ceph-mon[113733]: pgmap v592: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:31:39.449 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:31:39 smithi174 ceph-mon[88004]: pgmap v593: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:31:39.554 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:31:39 smithi153 ceph-mon[113733]: pgmap v593: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:31:41.449 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:31:41 smithi174 ceph-mon[88004]: pgmap v594: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:31:41.554 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:31:41 smithi153 ceph-mon[113733]: pgmap v594: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:31:43.449 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:31:43 smithi174 ceph-mon[88004]: pgmap v595: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:31:43.554 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:31:43 smithi153 ceph-mon[113733]: pgmap v595: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:31:45.449 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:31:45 smithi174 ceph-mon[88004]: pgmap v596: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:31:45.554 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:31:45 smithi153 ceph-mon[113733]: pgmap v596: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:31:47.449 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:31:47 smithi174 ceph-mon[88004]: pgmap v597: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:31:47.449 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:31:47 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:31:47.554 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:31:47 smithi153 ceph-mon[113733]: pgmap v597: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:31:47.554 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:31:47 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:31:49.449 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:31:49 smithi174 ceph-mon[88004]: pgmap v598: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:31:49.554 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:31:49 smithi153 ceph-mon[113733]: pgmap v598: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:31:51.199 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:31:51 smithi174 ceph-mon[88004]: pgmap v599: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:31:51.554 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:31:51 smithi153 ceph-mon[113733]: pgmap v599: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:31:53.449 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:31:53 smithi174 ceph-mon[88004]: pgmap v600: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:31:53.554 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:31:53 smithi153 ceph-mon[113733]: pgmap v600: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:31:55.449 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:31:55 smithi174 ceph-mon[88004]: pgmap v601: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:31:55.554 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:31:55 smithi153 ceph-mon[113733]: pgmap v601: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:31:56.555 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:56.556+0000 7f56fee00700 1 -- 172.21.15.153:0/3507551471 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f56f8100dc0 msgr2=0x7f56f81011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:56.555 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:56.556+0000 7f56fee00700 1 --2- 172.21.15.153:0/3507551471 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f56f8100dc0 0x7f56f81011a0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f56e8009910 tx=0x7f56e8009c20 comp rx=0 tx=0).stop 2024-03-21T10:31:56.556 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:56.556+0000 7f56fee00700 1 -- 172.21.15.153:0/3507551471 shutdown_connections 2024-03-21T10:31:56.556 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:56.556+0000 7f56fee00700 1 --2- 172.21.15.153:0/3507551471 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f56f8101770 0x7f56f81057c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:56.556 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:56.556+0000 7f56fee00700 1 --2- 172.21.15.153:0/3507551471 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f56f8100dc0 0x7f56f81011a0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:56.556 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:56.556+0000 7f56fee00700 1 -- 172.21.15.153:0/3507551471 >> 172.21.15.153:0/3507551471 conn(0x7f56f80fc650 msgr2=0x7f56f80fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:56.556 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:56.556+0000 7f56fee00700 1 -- 172.21.15.153:0/3507551471 shutdown_connections 2024-03-21T10:31:56.556 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:56.556+0000 7f56fee00700 1 -- 172.21.15.153:0/3507551471 wait complete. 2024-03-21T10:31:56.558 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:56.557+0000 7f56fee00700 1 Processor -- start 2024-03-21T10:31:56.558 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:56.557+0000 7f56fee00700 1 -- start start 2024-03-21T10:31:56.558 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:56.557+0000 7f56fee00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f56f8100dc0 0x7f56f819e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:56.558 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:56.557+0000 7f56fee00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f56f8101770 0x7f56f819ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:56.558 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:56.557+0000 7f56fee00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56f819f250 con 0x7f56f8100dc0 2024-03-21T10:31:56.558 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:56.557+0000 7f56fee00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56f8198830 con 0x7f56f8101770 2024-03-21T10:31:56.558 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:56.558+0000 7f56f7e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f56f8100dc0 0x7f56f819e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:56.558 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:56.558+0000 7f56f7e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f56f8100dc0 0x7f56f819e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:55866/0 (socket says 172.21.15.153:55866) 2024-03-21T10:31:56.558 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:56.558+0000 7f56f7e00700 1 -- 172.21.15.153:0/153288448 learned_addr learned my addr 172.21.15.153:0/153288448 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:31:56.558 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:56.558+0000 7f56f7400700 1 --2- 172.21.15.153:0/153288448 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f56f8101770 0x7f56f819ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:56.558 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:56.558+0000 7f56f7e00700 1 -- 172.21.15.153:0/153288448 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f56f8101770 msgr2=0x7f56f819ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:56.558 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:56.558+0000 7f56f7e00700 1 --2- 172.21.15.153:0/153288448 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f56f8101770 0x7f56f819ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:56.559 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:56.558+0000 7f56f7e00700 1 -- 172.21.15.153:0/153288448 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56e80095f0 con 0x7f56f8100dc0 2024-03-21T10:31:56.559 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:56.558+0000 7f56f7400700 1 --2- 172.21.15.153:0/153288448 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f56f8101770 0x7f56f819ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:31:56.559 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:56.559+0000 7f56f7e00700 1 --2- 172.21.15.153:0/153288448 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f56f8100dc0 0x7f56f819e710 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f56e800b5c0 tx=0x7f56e800be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:56.559 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:56.559+0000 7f56f4c00700 1 -- 172.21.15.153:0/153288448 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f56e801d070 con 0x7f56f8100dc0 2024-03-21T10:31:56.559 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:56.559+0000 7f56f4c00700 1 -- 172.21.15.153:0/153288448 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f56e800f460 con 0x7f56f8100dc0 2024-03-21T10:31:56.559 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:56.559+0000 7f56f4c00700 1 -- 172.21.15.153:0/153288448 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f56e8017610 con 0x7f56f8100dc0 2024-03-21T10:31:56.560 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:56.559+0000 7f56fee00700 1 -- 172.21.15.153:0/153288448 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56f8198ab0 con 0x7f56f8100dc0 2024-03-21T10:31:56.560 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:56.559+0000 7f56fee00700 1 -- 172.21.15.153:0/153288448 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56f8198fa0 con 0x7f56f8100dc0 2024-03-21T10:31:56.560 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:56.560+0000 7f56fee00700 1 -- 172.21.15.153:0/153288448 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56f8104f40 con 0x7f56f8100dc0 2024-03-21T10:31:56.560 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:56.561+0000 7f56f4c00700 1 -- 172.21.15.153:0/153288448 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f56e800f5d0 con 0x7f56f8100dc0 2024-03-21T10:31:56.561 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:56.562+0000 7f56f4c00700 1 --2- 172.21.15.153:0/153288448 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f56dc06e460 0x7f56dc070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:56.561 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:56.562+0000 7f56f4c00700 1 -- 172.21.15.153:0/153288448 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f56e808fb10 con 0x7f56f8100dc0 2024-03-21T10:31:56.561 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:56.562+0000 7f56f7400700 1 --2- 172.21.15.153:0/153288448 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f56dc06e460 0x7f56dc070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:56.562 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:56.563+0000 7f56f7400700 1 --2- 172.21.15.153:0/153288448 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f56dc06e460 0x7f56dc070920 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f56e4005fd0 tx=0x7f56e4005e20 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:56.566 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:56.567+0000 7f56f4c00700 1 -- 172.21.15.153:0/153288448 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f56e805c6e0 con 0x7f56f8100dc0 2024-03-21T10:31:56.808 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:56.809+0000 7f56fee00700 1 -- 172.21.15.153:0/153288448 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f56f81992d0 con 0x7f56dc06e460 2024-03-21T10:31:56.810 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:56.811+0000 7f56f4c00700 1 -- 172.21.15.153:0/153288448 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f56f81992d0 con 0x7f56dc06e460 2024-03-21T10:31:56.813 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:56.814+0000 7f56fee00700 1 -- 172.21.15.153:0/153288448 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f56dc06e460 msgr2=0x7f56dc070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:56.813 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:56.814+0000 7f56fee00700 1 --2- 172.21.15.153:0/153288448 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f56dc06e460 0x7f56dc070920 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f56e4005fd0 tx=0x7f56e4005e20 comp rx=0 tx=0).stop 2024-03-21T10:31:56.814 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:56.814+0000 7f56fee00700 1 -- 172.21.15.153:0/153288448 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f56f8100dc0 msgr2=0x7f56f819e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:56.814 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:56.814+0000 7f56fee00700 1 --2- 172.21.15.153:0/153288448 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f56f8100dc0 0x7f56f819e710 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f56e800b5c0 tx=0x7f56e800be00 comp rx=0 tx=0).stop 2024-03-21T10:31:56.814 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:56.814+0000 7f56fee00700 1 -- 172.21.15.153:0/153288448 shutdown_connections 2024-03-21T10:31:56.814 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:56.814+0000 7f56fee00700 1 --2- 172.21.15.153:0/153288448 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f56dc06e460 0x7f56dc070920 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:56.814 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:56.814+0000 7f56fee00700 1 --2- 172.21.15.153:0/153288448 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f56f8100dc0 0x7f56f819e710 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:56.814 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:56.814+0000 7f56fee00700 1 --2- 172.21.15.153:0/153288448 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f56f8101770 0x7f56f819ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:56.814 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:56.814+0000 7f56fee00700 1 -- 172.21.15.153:0/153288448 >> 172.21.15.153:0/153288448 conn(0x7f56f80fc650 msgr2=0x7f56f80fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:56.814 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:56.814+0000 7f56fee00700 1 -- 172.21.15.153:0/153288448 shutdown_connections 2024-03-21T10:31:56.814 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:56.814+0000 7f56fee00700 1 -- 172.21.15.153:0/153288448 wait complete. 2024-03-21T10:31:56.830 INFO:teuthology.orchestra.run.smithi153.stdout:true 2024-03-21T10:31:56.973 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:56.974+0000 7f0d31200700 1 -- 172.21.15.153:0/4088496168 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0d2c069020 msgr2=0x7f0d2c105790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:56.974 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:56.974+0000 7f0d31200700 1 --2- 172.21.15.153:0/4088496168 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0d2c069020 0x7f0d2c105790 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f0d18009910 tx=0x7f0d18009c20 comp rx=0 tx=0).stop 2024-03-21T10:31:56.974 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:56.974+0000 7f0d31200700 1 -- 172.21.15.153:0/4088496168 shutdown_connections 2024-03-21T10:31:56.974 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:56.974+0000 7f0d31200700 1 --2- 172.21.15.153:0/4088496168 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0d2c069020 0x7f0d2c105790 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:56.974 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:56.974+0000 7f0d31200700 1 --2- 172.21.15.153:0/4088496168 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d2c068670 0x7f0d2c068a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:56.974 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:56.974+0000 7f0d31200700 1 -- 172.21.15.153:0/4088496168 >> 172.21.15.153:0/4088496168 conn(0x7f0d2c0755a0 msgr2=0x7f0d2c0759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:56.974 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:56.974+0000 7f0d31200700 1 -- 172.21.15.153:0/4088496168 shutdown_connections 2024-03-21T10:31:56.974 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:56.974+0000 7f0d31200700 1 -- 172.21.15.153:0/4088496168 wait complete. 2024-03-21T10:31:56.975 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:56.975+0000 7f0d31200700 1 Processor -- start 2024-03-21T10:31:56.975 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:56.975+0000 7f0d31200700 1 -- start start 2024-03-21T10:31:56.975 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:56.976+0000 7f0d31200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0d2c068670 0x7f0d2c19e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:56.975 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:56.976+0000 7f0d31200700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d2c069020 0x7f0d2c19ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:56.975 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:56.976+0000 7f0d31200700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d2c19f3b0 con 0x7f0d2c068670 2024-03-21T10:31:56.975 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:56.976+0000 7f0d31200700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d2c198860 con 0x7f0d2c069020 2024-03-21T10:31:56.976 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:56.976+0000 7f0d29e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0d2c068670 0x7f0d2c19e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:56.976 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:56.976+0000 7f0d29e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0d2c068670 0x7f0d2c19e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:55880/0 (socket says 172.21.15.153:55880) 2024-03-21T10:31:56.976 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:56.976+0000 7f0d29e00700 1 -- 172.21.15.153:0/6743435 learned_addr learned my addr 172.21.15.153:0/6743435 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:31:56.977 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:56.976+0000 7f0d29400700 1 --2- 172.21.15.153:0/6743435 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d2c069020 0x7f0d2c19ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:56.977 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:56.976+0000 7f0d29e00700 1 -- 172.21.15.153:0/6743435 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d2c069020 msgr2=0x7f0d2c19ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:56.977 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:56.976+0000 7f0d29e00700 1 --2- 172.21.15.153:0/6743435 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d2c069020 0x7f0d2c19ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:56.977 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:56.976+0000 7f0d29e00700 1 -- 172.21.15.153:0/6743435 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d180095f0 con 0x7f0d2c068670 2024-03-21T10:31:56.977 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:56.977+0000 7f0d29e00700 1 --2- 172.21.15.153:0/6743435 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0d2c068670 0x7f0d2c19e7e0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c00d8d0 tx=0x7f0d1c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:56.977 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:56.977+0000 7f0d22a00700 1 -- 172.21.15.153:0/6743435 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0d1c009940 con 0x7f0d2c068670 2024-03-21T10:31:56.977 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:56.977+0000 7f0d22a00700 1 -- 172.21.15.153:0/6743435 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0d1c010460 con 0x7f0d2c068670 2024-03-21T10:31:56.977 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:56.977+0000 7f0d22a00700 1 -- 172.21.15.153:0/6743435 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0d1c00f5f0 con 0x7f0d2c068670 2024-03-21T10:31:56.977 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:56.977+0000 7f0d31200700 1 -- 172.21.15.153:0/6743435 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d2c198b40 con 0x7f0d2c068670 2024-03-21T10:31:56.977 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:56.977+0000 7f0d31200700 1 -- 172.21.15.153:0/6743435 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d2c199040 con 0x7f0d2c068670 2024-03-21T10:31:56.978 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:56.978+0000 7f0d31200700 1 -- 172.21.15.153:0/6743435 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d2c04ea90 con 0x7f0d2c068670 2024-03-21T10:31:56.978 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:56.979+0000 7f0d22a00700 1 -- 172.21.15.153:0/6743435 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f0d1c0105d0 con 0x7f0d2c068670 2024-03-21T10:31:56.980 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:56.980+0000 7f0d22a00700 1 --2- 172.21.15.153:0/6743435 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f0d1006e460 0x7f0d10070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:56.980 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:56.980+0000 7f0d22a00700 1 -- 172.21.15.153:0/6743435 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f0d1c08e5b0 con 0x7f0d2c068670 2024-03-21T10:31:56.980 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:56.981+0000 7f0d29400700 1 --2- 172.21.15.153:0/6743435 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f0d1006e460 0x7f0d10070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:56.981 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:56.981+0000 7f0d29400700 1 --2- 172.21.15.153:0/6743435 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f0d1006e460 0x7f0d10070920 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f0d18000c00 tx=0x7f0d18005fd0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:56.984 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:56.985+0000 7f0d22a00700 1 -- 172.21.15.153:0/6743435 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0d1c05b100 con 0x7f0d2c068670 2024-03-21T10:31:57.234 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:57.235+0000 7f0d31200700 1 -- 172.21.15.153:0/6743435 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0d2c199390 con 0x7f0d1006e460 2024-03-21T10:31:57.236 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:57.237+0000 7f0d22a00700 1 -- 172.21.15.153:0/6743435 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f0d2c199390 con 0x7f0d1006e460 2024-03-21T10:31:57.239 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:57.240+0000 7f0d31200700 1 -- 172.21.15.153:0/6743435 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f0d1006e460 msgr2=0x7f0d10070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:57.239 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:57.240+0000 7f0d31200700 1 --2- 172.21.15.153:0/6743435 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f0d1006e460 0x7f0d10070920 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f0d18000c00 tx=0x7f0d18005fd0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.240 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:57.240+0000 7f0d31200700 1 -- 172.21.15.153:0/6743435 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0d2c068670 msgr2=0x7f0d2c19e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:57.240 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:57.240+0000 7f0d31200700 1 --2- 172.21.15.153:0/6743435 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0d2c068670 0x7f0d2c19e7e0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c00d8d0 tx=0x7f0d1c00dc90 comp rx=0 tx=0).stop 2024-03-21T10:31:57.240 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:57.240+0000 7f0d31200700 1 -- 172.21.15.153:0/6743435 shutdown_connections 2024-03-21T10:31:57.240 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:57.240+0000 7f0d31200700 1 --2- 172.21.15.153:0/6743435 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f0d1006e460 0x7f0d10070920 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.240 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:57.240+0000 7f0d31200700 1 --2- 172.21.15.153:0/6743435 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0d2c068670 0x7f0d2c19e7e0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.240 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:57.240+0000 7f0d31200700 1 --2- 172.21.15.153:0/6743435 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d2c069020 0x7f0d2c19ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.240 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:57.240+0000 7f0d31200700 1 -- 172.21.15.153:0/6743435 >> 172.21.15.153:0/6743435 conn(0x7f0d2c0755a0 msgr2=0x7f0d2c102760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:57.240 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:57.240+0000 7f0d31200700 1 -- 172.21.15.153:0/6743435 shutdown_connections 2024-03-21T10:31:57.240 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:57.240+0000 7f0d31200700 1 -- 172.21.15.153:0/6743435 wait complete. 2024-03-21T10:31:57.398 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:57.398+0000 7fbce5200700 1 -- 172.21.15.153:0/3281276178 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbce01039b0 msgr2=0x7fbce0107a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:57.398 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:57.398+0000 7fbce5200700 1 --2- 172.21.15.153:0/3281276178 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbce01039b0 0x7fbce0107a00 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fbccc009910 tx=0x7fbccc009c20 comp rx=0 tx=0).stop 2024-03-21T10:31:57.398 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:57.398+0000 7fbce5200700 1 -- 172.21.15.153:0/3281276178 shutdown_connections 2024-03-21T10:31:57.398 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:57.398+0000 7fbce5200700 1 --2- 172.21.15.153:0/3281276178 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbce01039b0 0x7fbce0107a00 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.398 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:57.398+0000 7fbce5200700 1 --2- 172.21.15.153:0/3281276178 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbce0103000 0x7fbce01033e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.398 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:57.398+0000 7fbce5200700 1 -- 172.21.15.153:0/3281276178 >> 172.21.15.153:0/3281276178 conn(0x7fbce00fe8d0 msgr2=0x7fbce0100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:57.399 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:31:57 smithi153 ceph-mon[113733]: pgmap v602: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:31:57.399 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:57.399+0000 7fbce5200700 1 -- 172.21.15.153:0/3281276178 shutdown_connections 2024-03-21T10:31:57.399 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:57.399+0000 7fbce5200700 1 -- 172.21.15.153:0/3281276178 wait complete. 2024-03-21T10:31:57.400 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:57.400+0000 7fbce5200700 1 Processor -- start 2024-03-21T10:31:57.400 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:57.400+0000 7fbce5200700 1 -- start start 2024-03-21T10:31:57.400 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:57.401+0000 7fbce5200700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbce0103000 0x7fbce01989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:57.401 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:57.401+0000 7fbce5200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbce01039b0 0x7fbce0198f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:57.401 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:57.401+0000 7fbce5200700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbce01995c0 con 0x7fbce01039b0 2024-03-21T10:31:57.401 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:57.401+0000 7fbce5200700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbce019d360 con 0x7fbce0103000 2024-03-21T10:31:57.401 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:57.401+0000 7fbcdd400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbce01039b0 0x7fbce0198f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:57.401 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:57.401+0000 7fbcdd400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbce01039b0 0x7fbce0198f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:55890/0 (socket says 172.21.15.153:55890) 2024-03-21T10:31:57.401 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:57.401+0000 7fbcdd400700 1 -- 172.21.15.153:0/3324117155 learned_addr learned my addr 172.21.15.153:0/3324117155 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:31:57.401 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:57.401+0000 7fbcdde00700 1 --2- 172.21.15.153:0/3324117155 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbce0103000 0x7fbce01989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:57.401 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:57.402+0000 7fbcdd400700 1 -- 172.21.15.153:0/3324117155 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbce0103000 msgr2=0x7fbce01989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:57.401 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:57.402+0000 7fbcdd400700 1 --2- 172.21.15.153:0/3324117155 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbce0103000 0x7fbce01989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.401 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:57.402+0000 7fbcdd400700 1 -- 172.21.15.153:0/3324117155 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbccc0095f0 con 0x7fbce01039b0 2024-03-21T10:31:57.402 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:57.402+0000 7fbcdd400700 1 --2- 172.21.15.153:0/3324117155 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbce01039b0 0x7fbce0198f30 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fbccc00bda0 tx=0x7fbccc00be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:57.402 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:57.402+0000 7fbcd6a00700 1 -- 172.21.15.153:0/3324117155 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbccc01d070 con 0x7fbce01039b0 2024-03-21T10:31:57.402 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:57.402+0000 7fbcd6a00700 1 -- 172.21.15.153:0/3324117155 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbccc004fb0 con 0x7fbce01039b0 2024-03-21T10:31:57.402 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:57.402+0000 7fbcd6a00700 1 -- 172.21.15.153:0/3324117155 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbccc00f460 con 0x7fbce01039b0 2024-03-21T10:31:57.402 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:57.402+0000 7fbce5200700 1 -- 172.21.15.153:0/3324117155 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbce019d5e0 con 0x7fbce01039b0 2024-03-21T10:31:57.402 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:57.402+0000 7fbce5200700 1 -- 172.21.15.153:0/3324117155 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbce019dad0 con 0x7fbce01039b0 2024-03-21T10:31:57.403 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:57.404+0000 7fbce5200700 1 -- 172.21.15.153:0/3324117155 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbce010b400 con 0x7fbce01039b0 2024-03-21T10:31:57.403 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:57.405+0000 7fbcd6a00700 1 -- 172.21.15.153:0/3324117155 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fbccc022800 con 0x7fbce01039b0 2024-03-21T10:31:57.405 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:57.406+0000 7fbcd6a00700 1 --2- 172.21.15.153:0/3324117155 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fbcc406e460 0x7fbcc4070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:57.405 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:57.406+0000 7fbcd6a00700 1 -- 172.21.15.153:0/3324117155 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fbccc08fa20 con 0x7fbce01039b0 2024-03-21T10:31:57.405 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:57.406+0000 7fbcdde00700 1 --2- 172.21.15.153:0/3324117155 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fbcc406e460 0x7fbcc4070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:57.406 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:57.407+0000 7fbcdde00700 1 --2- 172.21.15.153:0/3324117155 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fbcc406e460 0x7fbcc4070920 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fbcd0005fd0 tx=0x7fbcd000a560 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:57.411 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:57.411+0000 7fbcd6a00700 1 -- 172.21.15.153:0/3324117155 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fbccc05c7c0 con 0x7fbce01039b0 2024-03-21T10:31:57.449 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:31:57 smithi174 ceph-mon[88004]: pgmap v602: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:31:57.634 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:57.635+0000 7fbce5200700 1 -- 172.21.15.153:0/3324117155 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fbce019de00 con 0x7fbcc406e460 2024-03-21T10:31:57.666 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:57.666+0000 7fbcd6a00700 1 -- 172.21.15.153:0/3324117155 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fbce019de00 con 0x7fbcc406e460 2024-03-21T10:31:57.666 INFO:teuthology.orchestra.run.smithi153.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-21T10:31:57.666 INFO:teuthology.orchestra.run.smithi153.stdout:alertmanager.smithi153 smithi153 *:9093,9094 running (20m) 12m ago 34m 14.7M - 0.25.0 c8568f914cd2 9ed3f2200b5f 2024-03-21T10:31:57.666 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi153 smithi153 running (34m) 12m ago 34m 9424k - 18.2.1 5be31c24972a 5b61994396a6 2024-03-21T10:31:57.666 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi174 smithi174 running (33m) 11m ago 33m 9244k - 18.2.1 5be31c24972a 4f7a5fc3ac28 2024-03-21T10:31:57.667 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi153 smithi153 running (16m) 12m ago 34m 6656k - 19.0.0-2231-g2edfee68 237e43c7a65f 32c21104d5b0 2024-03-21T10:31:57.667 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi174 smithi174 running (16m) 11m ago 33m 6660k - 19.0.0-2231-g2edfee68 237e43c7a65f a04e48967ab6 2024-03-21T10:31:57.667 INFO:teuthology.orchestra.run.smithi153.stdout:grafana.smithi153 smithi153 *:3000 running (20m) 12m ago 33m 73.6M - 9.4.12 2bacad6d85d8 ee24026ab786 2024-03-21T10:31:57.667 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.iuyluj smithi153 running (12m) 12m ago 27m 10.9M - 19.0.0-2231-g2edfee68 237e43c7a65f 87aff8a5de70 2024-03-21T10:31:57.667 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.zipckr smithi153 running (12m) 12m ago 27m 11.8M - 19.0.0-2231-g2edfee68 237e43c7a65f ccc262bd5dd3 2024-03-21T10:31:57.667 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.nhozag smithi174 running (27m) 11m ago 27m 12.0M - 18.2.1 5be31c24972a 2fee4000308f 2024-03-21T10:31:57.667 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.yyimst smithi174 running (11m) 11m ago 27m 10.7M - 19.0.0-2231-g2edfee68 237e43c7a65f fc1962f9351e 2024-03-21T10:31:57.667 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi153.tluxni smithi153 *:8443,9283,8765 running (23m) 12m ago 36m 509M - 19.0.0-2231-g2edfee68 237e43c7a65f 7ffee0e55382 2024-03-21T10:31:57.667 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi174.qpcwat smithi174 *:8443,9283,8765 running (22m) 11m ago 32m 452M - 19.0.0-2231-g2edfee68 237e43c7a65f 748968f216ab 2024-03-21T10:31:57.667 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi153 smithi153 running (18m) 12m ago 36m 55.0M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f dd8038303dc3 2024-03-21T10:31:57.667 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi174 smithi174 running (18m) 11m ago 32m 44.2M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f 29d78a788f2c 2024-03-21T10:31:57.667 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi153 smithi153 *:9100 running (21m) 12m ago 34m 7889k - 1.5.0 0da6a335fe13 bb835edc0e0b 2024-03-21T10:31:57.667 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi174 smithi174 *:9100 running (21m) 11m ago 33m 8102k - 1.5.0 0da6a335fe13 7c4ecad790fa 2024-03-21T10:31:57.667 INFO:teuthology.orchestra.run.smithi153.stdout:osd.0 smithi153 running (16m) 12m ago 31m 116M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 507900fd4791 2024-03-21T10:31:57.667 INFO:teuthology.orchestra.run.smithi153.stdout:osd.1 smithi153 running (15m) 12m ago 31m 89.2M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 0d646452e9c1 2024-03-21T10:31:57.668 INFO:teuthology.orchestra.run.smithi153.stdout:osd.2 smithi153 running (15m) 12m ago 30m 87.4M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 1326f2afdfcd 2024-03-21T10:31:57.668 INFO:teuthology.orchestra.run.smithi153.stdout:osd.3 smithi174 running (14m) 11m ago 30m 83.6M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 70aafac71ad5 2024-03-21T10:31:57.668 INFO:teuthology.orchestra.run.smithi153.stdout:osd.4 smithi174 running (13m) 11m ago 29m 92.2M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 0d746ce6e79d 2024-03-21T10:31:57.668 INFO:teuthology.orchestra.run.smithi153.stdout:osd.5 smithi174 running (13m) 11m ago 29m 86.2M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 53e5f35b8cac 2024-03-21T10:31:57.668 INFO:teuthology.orchestra.run.smithi153.stdout:prometheus.smithi153 smithi153 *:9095 running (21m) 12m ago 33m 73.5M - 2.43.0 a07b618ecd1d 7d95561e50c2 2024-03-21T10:31:57.669 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:57.670+0000 7fbce5200700 1 -- 172.21.15.153:0/3324117155 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fbcc406e460 msgr2=0x7fbcc4070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:57.670 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:57.670+0000 7fbce5200700 1 --2- 172.21.15.153:0/3324117155 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fbcc406e460 0x7fbcc4070920 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fbcd0005fd0 tx=0x7fbcd000a560 comp rx=0 tx=0).stop 2024-03-21T10:31:57.670 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:57.670+0000 7fbce5200700 1 -- 172.21.15.153:0/3324117155 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbce01039b0 msgr2=0x7fbce0198f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:57.670 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:57.670+0000 7fbce5200700 1 --2- 172.21.15.153:0/3324117155 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbce01039b0 0x7fbce0198f30 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fbccc00bda0 tx=0x7fbccc00be80 comp rx=0 tx=0).stop 2024-03-21T10:31:57.670 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:57.670+0000 7fbce5200700 1 -- 172.21.15.153:0/3324117155 shutdown_connections 2024-03-21T10:31:57.670 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:57.670+0000 7fbce5200700 1 --2- 172.21.15.153:0/3324117155 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbce0103000 0x7fbce01989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.670 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:57.670+0000 7fbce5200700 1 --2- 172.21.15.153:0/3324117155 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fbcc406e460 0x7fbcc4070920 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.670 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:57.670+0000 7fbce5200700 1 --2- 172.21.15.153:0/3324117155 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbce01039b0 0x7fbce0198f30 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.671 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:57.670+0000 7fbce5200700 1 -- 172.21.15.153:0/3324117155 >> 172.21.15.153:0/3324117155 conn(0x7fbce00fe8d0 msgr2=0x7fbce0100cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:57.671 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:57.671+0000 7fbce5200700 1 -- 172.21.15.153:0/3324117155 shutdown_connections 2024-03-21T10:31:57.671 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:57.671+0000 7fbce5200700 1 -- 172.21.15.153:0/3324117155 wait complete. 2024-03-21T10:31:57.827 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:57.828+0000 7f80bf200700 1 -- 172.21.15.153:0/984137337 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f80b8100de0 msgr2=0x7f80b81011c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:57.828 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:57.828+0000 7f80bf200700 1 --2- 172.21.15.153:0/984137337 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f80b8100de0 0x7f80b81011c0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f80a8009910 tx=0x7f80a8009c20 comp rx=0 tx=0).stop 2024-03-21T10:31:57.828 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:57.828+0000 7f80bf200700 1 -- 172.21.15.153:0/984137337 shutdown_connections 2024-03-21T10:31:57.828 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:57.828+0000 7f80bf200700 1 --2- 172.21.15.153:0/984137337 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f80b8101790 0x7f80b81057e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.828 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:57.828+0000 7f80bf200700 1 --2- 172.21.15.153:0/984137337 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f80b8100de0 0x7f80b81011c0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.828 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:57.828+0000 7f80bf200700 1 -- 172.21.15.153:0/984137337 >> 172.21.15.153:0/984137337 conn(0x7f80b80fc6b0 msgr2=0x7f80b80fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:57.828 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:57.828+0000 7f80bf200700 1 -- 172.21.15.153:0/984137337 shutdown_connections 2024-03-21T10:31:57.828 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:57.828+0000 7f80bf200700 1 -- 172.21.15.153:0/984137337 wait complete. 2024-03-21T10:31:57.829 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:57.829+0000 7f80bf200700 1 Processor -- start 2024-03-21T10:31:57.830 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:57.829+0000 7f80bf200700 1 -- start start 2024-03-21T10:31:57.830 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:57.830+0000 7f80bf200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f80b8100de0 0x7f80b8198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:57.830 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:57.830+0000 7f80bf200700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f80b8101790 0x7f80b8198ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:57.830 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:57.830+0000 7f80bf200700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b8199550 con 0x7f80b8100de0 2024-03-21T10:31:57.830 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:57.830+0000 7f80bf200700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b819d360 con 0x7f80b8101790 2024-03-21T10:31:57.830 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:57.830+0000 7f80b7e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f80b8100de0 0x7f80b8198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:57.830 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:57.830+0000 7f80b7e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f80b8100de0 0x7f80b8198980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:55906/0 (socket says 172.21.15.153:55906) 2024-03-21T10:31:57.830 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:57.830+0000 7f80b7e00700 1 -- 172.21.15.153:0/2783973005 learned_addr learned my addr 172.21.15.153:0/2783973005 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:31:57.830 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:57.830+0000 7f80b7400700 1 --2- 172.21.15.153:0/2783973005 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f80b8101790 0x7f80b8198ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:57.830 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:57.830+0000 7f80b7e00700 1 -- 172.21.15.153:0/2783973005 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f80b8101790 msgr2=0x7f80b8198ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:57.830 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:57.830+0000 7f80b7e00700 1 --2- 172.21.15.153:0/2783973005 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f80b8101790 0x7f80b8198ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:57.831 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:57.830+0000 7f80b7e00700 1 -- 172.21.15.153:0/2783973005 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80a80095f0 con 0x7f80b8100de0 2024-03-21T10:31:57.831 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:57.830+0000 7f80b7400700 1 --2- 172.21.15.153:0/2783973005 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f80b8101790 0x7f80b8198ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:31:57.831 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:57.831+0000 7f80b7e00700 1 --2- 172.21.15.153:0/2783973005 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f80b8100de0 0x7f80b8198980 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f80a80092e0 tx=0x7f80a800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:57.832 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:57.831+0000 7f80b4c00700 1 -- 172.21.15.153:0/2783973005 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f80a801d070 con 0x7f80b8100de0 2024-03-21T10:31:57.832 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:57.831+0000 7f80b4c00700 1 -- 172.21.15.153:0/2783973005 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f80a8005e00 con 0x7f80b8100de0 2024-03-21T10:31:57.832 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:57.831+0000 7f80bf200700 1 -- 172.21.15.153:0/2783973005 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80b819d5e0 con 0x7f80b8100de0 2024-03-21T10:31:57.832 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:57.831+0000 7f80b4c00700 1 -- 172.21.15.153:0/2783973005 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f80a8017400 con 0x7f80b8100de0 2024-03-21T10:31:57.832 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:57.831+0000 7f80bf200700 1 -- 172.21.15.153:0/2783973005 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80b819dad0 con 0x7f80b8100de0 2024-03-21T10:31:57.832 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:57.832+0000 7f80bf200700 1 -- 172.21.15.153:0/2783973005 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80b8104f60 con 0x7f80b8100de0 2024-03-21T10:31:57.832 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:57.833+0000 7f80b4c00700 1 -- 172.21.15.153:0/2783973005 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f80a800f460 con 0x7f80b8100de0 2024-03-21T10:31:57.833 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:57.834+0000 7f80b4c00700 1 --2- 172.21.15.153:0/2783973005 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f809c06e460 0x7f809c070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:57.833 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:57.834+0000 7f80b4c00700 1 -- 172.21.15.153:0/2783973005 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f80a808faa0 con 0x7f80b8100de0 2024-03-21T10:31:57.834 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:57.835+0000 7f80b7400700 1 --2- 172.21.15.153:0/2783973005 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f809c06e460 0x7f809c070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:57.834 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:57.835+0000 7f80b7400700 1 --2- 172.21.15.153:0/2783973005 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f809c06e460 0x7f809c070920 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f80a4005fd0 tx=0x7f80a4005ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:57.838 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:57.839+0000 7f80b4c00700 1 -- 172.21.15.153:0/2783973005 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f80a805c5f0 con 0x7f80b8100de0 2024-03-21T10:31:58.128 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:58.128+0000 7f80bf200700 1 -- 172.21.15.153:0/2783973005 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f80b819ddb0 con 0x7f80b8100de0 2024-03-21T10:31:58.129 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:58.130+0000 7f80b4c00700 1 -- 172.21.15.153:0/2783973005 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+805 (secure 0 0 0) 0x7f80a805c180 con 0x7f80b8100de0 2024-03-21T10:31:58.129 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:31:58.129 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": { 2024-03-21T10:31:58.130 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:31:58.130 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:31:58.130 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": { 2024-03-21T10:31:58.130 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:31:58.130 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:31:58.130 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": { 2024-03-21T10:31:58.130 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 6 2024-03-21T10:31:58.130 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:31:58.130 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": { 2024-03-21T10:31:58.130 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-03-21T10:31:58.130 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 3 2024-03-21T10:31:58.130 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:31:58.130 INFO:teuthology.orchestra.run.smithi153.stdout: "overall": { 2024-03-21T10:31:58.130 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-03-21T10:31:58.131 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 13 2024-03-21T10:31:58.131 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-03-21T10:31:58.131 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:31:58.132 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:58.133+0000 7f80bf200700 1 -- 172.21.15.153:0/2783973005 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f809c06e460 msgr2=0x7f809c070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:58.132 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:58.133+0000 7f80bf200700 1 --2- 172.21.15.153:0/2783973005 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f809c06e460 0x7f809c070920 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f80a4005fd0 tx=0x7f80a4005ee0 comp rx=0 tx=0).stop 2024-03-21T10:31:58.132 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:58.133+0000 7f80bf200700 1 -- 172.21.15.153:0/2783973005 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f80b8100de0 msgr2=0x7f80b8198980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:58.132 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:58.133+0000 7f80bf200700 1 --2- 172.21.15.153:0/2783973005 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f80b8100de0 0x7f80b8198980 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f80a80092e0 tx=0x7f80a800be20 comp rx=0 tx=0).stop 2024-03-21T10:31:58.132 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:58.133+0000 7f80bf200700 1 -- 172.21.15.153:0/2783973005 shutdown_connections 2024-03-21T10:31:58.133 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:58.133+0000 7f80bf200700 1 --2- 172.21.15.153:0/2783973005 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f809c06e460 0x7f809c070920 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:58.133 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:58.133+0000 7f80bf200700 1 --2- 172.21.15.153:0/2783973005 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f80b8100de0 0x7f80b8198980 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:58.133 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:58.133+0000 7f80bf200700 1 --2- 172.21.15.153:0/2783973005 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f80b8101790 0x7f80b8198ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:58.133 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:58.133+0000 7f80bf200700 1 -- 172.21.15.153:0/2783973005 >> 172.21.15.153:0/2783973005 conn(0x7f80b80fc6b0 msgr2=0x7f80b80fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:58.133 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:58.133+0000 7f80bf200700 1 -- 172.21.15.153:0/2783973005 shutdown_connections 2024-03-21T10:31:58.133 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:58.133+0000 7f80bf200700 1 -- 172.21.15.153:0/2783973005 wait complete. 2024-03-21T10:31:58.289 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:58.289+0000 7f5193000700 1 -- 172.21.15.153:0/2369788226 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f518c069020 msgr2=0x7f518c105790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:58.289 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:58.289+0000 7f5193000700 1 --2- 172.21.15.153:0/2369788226 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f518c069020 0x7f518c105790 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f5174009910 tx=0x7f5174009c20 comp rx=0 tx=0).stop 2024-03-21T10:31:58.289 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:58.289+0000 7f5193000700 1 -- 172.21.15.153:0/2369788226 shutdown_connections 2024-03-21T10:31:58.289 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:58.289+0000 7f5193000700 1 --2- 172.21.15.153:0/2369788226 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f518c069020 0x7f518c105790 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:58.289 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:58.289+0000 7f5193000700 1 --2- 172.21.15.153:0/2369788226 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f518c068670 0x7f518c068a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:58.289 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:58.289+0000 7f5193000700 1 -- 172.21.15.153:0/2369788226 >> 172.21.15.153:0/2369788226 conn(0x7f518c0755a0 msgr2=0x7f518c0759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:58.289 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:58.289+0000 7f5193000700 1 -- 172.21.15.153:0/2369788226 shutdown_connections 2024-03-21T10:31:58.289 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:58.289+0000 7f5193000700 1 -- 172.21.15.153:0/2369788226 wait complete. 2024-03-21T10:31:58.290 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:58.290+0000 7f5193000700 1 Processor -- start 2024-03-21T10:31:58.291 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:58.291+0000 7f5193000700 1 -- start start 2024-03-21T10:31:58.291 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:58.291+0000 7f5193000700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f518c068670 0x7f518c0ffca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:58.291 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:58.291+0000 7f5193000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f518c069020 0x7f518c1001e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:58.291 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:58.291+0000 7f5193000700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f518c100720 con 0x7f518c069020 2024-03-21T10:31:58.291 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:58.291+0000 7f5193000700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f518c100860 con 0x7f518c068670 2024-03-21T10:31:58.291 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:58.291+0000 7f5183e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f518c069020 0x7f518c1001e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:58.291 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:58.291+0000 7f5183e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f518c069020 0x7f518c1001e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:55916/0 (socket says 172.21.15.153:55916) 2024-03-21T10:31:58.291 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:58.291+0000 7f5183e00700 1 -- 172.21.15.153:0/686142966 learned_addr learned my addr 172.21.15.153:0/686142966 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:31:58.291 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:58.292+0000 7f518be00700 1 --2- 172.21.15.153:0/686142966 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f518c068670 0x7f518c0ffca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:58.291 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:58.292+0000 7f5183e00700 1 -- 172.21.15.153:0/686142966 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f518c068670 msgr2=0x7f518c0ffca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:58.291 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:58.292+0000 7f5183e00700 1 --2- 172.21.15.153:0/686142966 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f518c068670 0x7f518c0ffca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:58.291 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:58.292+0000 7f5183e00700 1 -- 172.21.15.153:0/686142966 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51740095f0 con 0x7f518c069020 2024-03-21T10:31:58.292 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:58.292+0000 7f5183e00700 1 --2- 172.21.15.153:0/686142966 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f518c069020 0x7f518c1001e0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f517400bda0 tx=0x7f517400be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:58.292 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:58.292+0000 7f5189600700 1 -- 172.21.15.153:0/686142966 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f517401d070 con 0x7f518c069020 2024-03-21T10:31:58.292 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:58.292+0000 7f5189600700 1 -- 172.21.15.153:0/686142966 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5174004fb0 con 0x7f518c069020 2024-03-21T10:31:58.292 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:58.292+0000 7f5189600700 1 -- 172.21.15.153:0/686142966 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f517400f460 con 0x7f518c069020 2024-03-21T10:31:58.293 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:58.292+0000 7f5193000700 1 -- 172.21.15.153:0/686142966 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f518c100ae0 con 0x7f518c069020 2024-03-21T10:31:58.293 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:58.293+0000 7f5193000700 1 -- 172.21.15.153:0/686142966 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f518c1a2bb0 con 0x7f518c069020 2024-03-21T10:31:58.293 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:58.294+0000 7f5193000700 1 -- 172.21.15.153:0/686142966 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f518c1090a0 con 0x7f518c069020 2024-03-21T10:31:58.293 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:58.295+0000 7f5189600700 1 -- 172.21.15.153:0/686142966 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f5174022800 con 0x7f518c069020 2024-03-21T10:31:58.294 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:58.296+0000 7f5189600700 1 --2- 172.21.15.153:0/686142966 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f517006e460 0x7f5170070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:58.295 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:58.296+0000 7f5189600700 1 -- 172.21.15.153:0/686142966 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f517408fa20 con 0x7f518c069020 2024-03-21T10:31:58.295 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:58.296+0000 7f518be00700 1 --2- 172.21.15.153:0/686142966 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f517006e460 0x7f5170070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:58.295 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:58.297+0000 7f518be00700 1 --2- 172.21.15.153:0/686142966 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f517006e460 0x7f5170070920 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f517c005ea0 tx=0x7f517c005e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:58.299 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:58.300+0000 7f5189600700 1 -- 172.21.15.153:0/686142966 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f5174058c90 con 0x7f518c069020 2024-03-21T10:31:58.449 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:31:58 smithi174 ceph-mon[88004]: from='client.35094 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:31:58.449 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:31:58 smithi174 ceph-mon[88004]: from='client.35098 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:31:58.449 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:31:58 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/2783973005' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:31:58.554 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:31:58 smithi153 ceph-mon[113733]: from='client.35094 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:31:58.554 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:31:58 smithi153 ceph-mon[113733]: from='client.35098 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:31:58.554 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:31:58 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/2783973005' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:31:58.580 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:58.581+0000 7f5193000700 1 -- 172.21.15.153:0/686142966 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f518c04ea90 con 0x7f518c069020 2024-03-21T10:31:58.581 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:58.582+0000 7f5189600700 1 -- 172.21.15.153:0/686142966 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1876 (secure 0 0 0) 0x7f517405c350 con 0x7f518c069020 2024-03-21T10:31:58.582 INFO:teuthology.orchestra.run.smithi153.stdout:e30 2024-03-21T10:31:58.582 INFO:teuthology.orchestra.run.smithi153.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-21T10:31:58.582 INFO:teuthology.orchestra.run.smithi153.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:31:58.582 INFO:teuthology.orchestra.run.smithi153.stdout:legacy client fscid: 1 2024-03-21T10:31:58.582 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:31:58.582 INFO:teuthology.orchestra.run.smithi153.stdout:Filesystem 'cephfs' (1) 2024-03-21T10:31:58.582 INFO:teuthology.orchestra.run.smithi153.stdout:fs_name cephfs 2024-03-21T10:31:58.582 INFO:teuthology.orchestra.run.smithi153.stdout:epoch 23 2024-03-21T10:31:58.583 INFO:teuthology.orchestra.run.smithi153.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-21T10:31:58.583 INFO:teuthology.orchestra.run.smithi153.stdout:created 2024-03-21T10:04:27.086576+0000 2024-03-21T10:31:58.583 INFO:teuthology.orchestra.run.smithi153.stdout:modified 2024-03-21T10:19:05.022092+0000 2024-03-21T10:31:58.583 INFO:teuthology.orchestra.run.smithi153.stdout:tableserver 0 2024-03-21T10:31:58.583 INFO:teuthology.orchestra.run.smithi153.stdout:root 0 2024-03-21T10:31:58.583 INFO:teuthology.orchestra.run.smithi153.stdout:session_timeout 60 2024-03-21T10:31:58.583 INFO:teuthology.orchestra.run.smithi153.stdout:session_autoclose 300 2024-03-21T10:31:58.583 INFO:teuthology.orchestra.run.smithi153.stdout:max_file_size 1099511627776 2024-03-21T10:31:58.583 INFO:teuthology.orchestra.run.smithi153.stdout:max_xattr_size 65536 2024-03-21T10:31:58.583 INFO:teuthology.orchestra.run.smithi153.stdout:required_client_features {} 2024-03-21T10:31:58.583 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure 0 2024-03-21T10:31:58.583 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure_osd_epoch 94 2024-03-21T10:31:58.583 INFO:teuthology.orchestra.run.smithi153.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:31:58.583 INFO:teuthology.orchestra.run.smithi153.stdout:max_mds 2 2024-03-21T10:31:58.584 INFO:teuthology.orchestra.run.smithi153.stdout:in 0,1 2024-03-21T10:31:58.584 INFO:teuthology.orchestra.run.smithi153.stdout:up {} 2024-03-21T10:31:58.584 INFO:teuthology.orchestra.run.smithi153.stdout:failed 0,1 2024-03-21T10:31:58.584 INFO:teuthology.orchestra.run.smithi153.stdout:damaged 2024-03-21T10:31:58.584 INFO:teuthology.orchestra.run.smithi153.stdout:stopped 2024-03-21T10:31:58.584 INFO:teuthology.orchestra.run.smithi153.stdout:data_pools [3] 2024-03-21T10:31:58.584 INFO:teuthology.orchestra.run.smithi153.stdout:metadata_pool 2 2024-03-21T10:31:58.584 INFO:teuthology.orchestra.run.smithi153.stdout:inline_data enabled 2024-03-21T10:31:58.584 INFO:teuthology.orchestra.run.smithi153.stdout:balancer 2024-03-21T10:31:58.584 INFO:teuthology.orchestra.run.smithi153.stdout:bal_rank_mask -1 2024-03-21T10:31:58.584 INFO:teuthology.orchestra.run.smithi153.stdout:standby_count_wanted 1 2024-03-21T10:31:58.584 INFO:teuthology.orchestra.run.smithi153.stdout:qdb_cluster leader: 0 members: 2024-03-21T10:31:58.584 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:31:58.584 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:31:58.584 INFO:teuthology.orchestra.run.smithi153.stdout:Standby daemons: 2024-03-21T10:31:58.584 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:31:58.585 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.zipckr{-1:34442} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6826/1103846295,v1:172.21.15.153:6827/1103846295] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:31:58.585 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.iuyluj{-1:34474} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6828/1738630832,v1:172.21.15.153:6829/1738630832] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:31:58.585 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.nhozag{-1:44269} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.174:6826/3967223227,v1:172.21.15.174:6827/3967223227] compat {c=[1],r=[1],i=[7ff]}] 2024-03-21T10:31:58.585 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.yyimst{-1:44313} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.174:6824/4127786797,v1:172.21.15.174:6825/4127786797] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:31:58.586 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:58.586+0000 7f5193000700 1 -- 172.21.15.153:0/686142966 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f517006e460 msgr2=0x7f5170070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:58.586 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:58.586+0000 7f5193000700 1 --2- 172.21.15.153:0/686142966 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f517006e460 0x7f5170070920 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f517c005ea0 tx=0x7f517c005e30 comp rx=0 tx=0).stop 2024-03-21T10:31:58.586 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:58.586+0000 7f5193000700 1 -- 172.21.15.153:0/686142966 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f518c069020 msgr2=0x7f518c1001e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:58.586 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:58.586+0000 7f5193000700 1 --2- 172.21.15.153:0/686142966 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f518c069020 0x7f518c1001e0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f517400bda0 tx=0x7f517400be80 comp rx=0 tx=0).stop 2024-03-21T10:31:58.586 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:58.586+0000 7f5193000700 1 -- 172.21.15.153:0/686142966 shutdown_connections 2024-03-21T10:31:58.586 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:58.586+0000 7f5193000700 1 --2- 172.21.15.153:0/686142966 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f518c068670 0x7f518c0ffca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:58.586 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:58.586+0000 7f5193000700 1 --2- 172.21.15.153:0/686142966 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f517006e460 0x7f5170070920 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:58.586 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:58.586+0000 7f5193000700 1 --2- 172.21.15.153:0/686142966 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f518c069020 0x7f518c1001e0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:58.586 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:58.586+0000 7f5193000700 1 -- 172.21.15.153:0/686142966 >> 172.21.15.153:0/686142966 conn(0x7f518c0755a0 msgr2=0x7f518c105000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:58.586 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:58.586+0000 7f5193000700 1 -- 172.21.15.153:0/686142966 shutdown_connections 2024-03-21T10:31:58.586 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:58.586+0000 7f5193000700 1 -- 172.21.15.153:0/686142966 wait complete. 2024-03-21T10:31:58.587 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 30 2024-03-21T10:31:58.745 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:58.745+0000 7fede7c00700 1 -- 172.21.15.153:0/1081421431 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fede0068670 msgr2=0x7fede0068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:58.745 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:58.745+0000 7fede7c00700 1 --2- 172.21.15.153:0/1081421431 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fede0068670 0x7fede0068a50 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fedd000b0a0 tx=0x7fedd000b3b0 comp rx=0 tx=0).stop 2024-03-21T10:31:58.745 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:58.745+0000 7fede7c00700 1 -- 172.21.15.153:0/1081421431 shutdown_connections 2024-03-21T10:31:58.745 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:58.745+0000 7fede7c00700 1 --2- 172.21.15.153:0/1081421431 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fede0069020 0x7fede0105790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:58.745 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:58.745+0000 7fede7c00700 1 --2- 172.21.15.153:0/1081421431 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fede0068670 0x7fede0068a50 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:58.745 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:58.745+0000 7fede7c00700 1 -- 172.21.15.153:0/1081421431 >> 172.21.15.153:0/1081421431 conn(0x7fede00755a0 msgr2=0x7fede00759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:58.745 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:58.745+0000 7fede7c00700 1 -- 172.21.15.153:0/1081421431 shutdown_connections 2024-03-21T10:31:58.745 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:58.745+0000 7fede7c00700 1 -- 172.21.15.153:0/1081421431 wait complete. 2024-03-21T10:31:58.747 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:58.746+0000 7fede7c00700 1 Processor -- start 2024-03-21T10:31:58.747 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:58.746+0000 7fede7c00700 1 -- start start 2024-03-21T10:31:58.747 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:58.747+0000 7fede7c00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fede0068670 0x7fede019c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:58.747 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:58.747+0000 7fede7c00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fede0069020 0x7fede019ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:58.747 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:58.747+0000 7fede7c00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fede019d0e0 con 0x7fede0069020 2024-03-21T10:31:58.747 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:58.747+0000 7fede7c00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fede0196590 con 0x7fede0068670 2024-03-21T10:31:58.747 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:58.747+0000 7fedd7e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fede0069020 0x7fede019ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:58.747 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:58.747+0000 7fedd7e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fede0069020 0x7fede019ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:55934/0 (socket says 172.21.15.153:55934) 2024-03-21T10:31:58.747 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:58.747+0000 7fedd7e00700 1 -- 172.21.15.153:0/4244500811 learned_addr learned my addr 172.21.15.153:0/4244500811 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:31:58.747 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:58.747+0000 7fede5200700 1 --2- 172.21.15.153:0/4244500811 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fede0068670 0x7fede019c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:58.748 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:58.747+0000 7fedd7e00700 1 -- 172.21.15.153:0/4244500811 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fede0068670 msgr2=0x7fede019c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:58.748 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:58.747+0000 7fedd7e00700 1 --2- 172.21.15.153:0/4244500811 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fede0068670 0x7fede019c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:58.748 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:58.747+0000 7fedd7e00700 1 -- 172.21.15.153:0/4244500811 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedd0009d00 con 0x7fede0069020 2024-03-21T10:31:58.748 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:58.748+0000 7fedd7e00700 1 --2- 172.21.15.153:0/4244500811 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fede0069020 0x7fede019ca50 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fedcc00d900 tx=0x7fedcc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:58.748 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:58.748+0000 7fedde000700 1 -- 172.21.15.153:0/4244500811 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fedcc009900 con 0x7fede0069020 2024-03-21T10:31:58.748 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:58.748+0000 7fedde000700 1 -- 172.21.15.153:0/4244500811 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fedcc010460 con 0x7fede0069020 2024-03-21T10:31:58.749 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:58.748+0000 7fede7c00700 1 -- 172.21.15.153:0/4244500811 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fede0196870 con 0x7fede0069020 2024-03-21T10:31:58.749 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:58.748+0000 7fedde000700 1 -- 172.21.15.153:0/4244500811 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fedcc00f5f0 con 0x7fede0069020 2024-03-21T10:31:58.749 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:58.748+0000 7fede7c00700 1 -- 172.21.15.153:0/4244500811 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fede0196dc0 con 0x7fede0069020 2024-03-21T10:31:58.749 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:58.749+0000 7fede7c00700 1 -- 172.21.15.153:0/4244500811 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fede0109130 con 0x7fede0069020 2024-03-21T10:31:58.749 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:58.750+0000 7fedde000700 1 -- 172.21.15.153:0/4244500811 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fedcc0105d0 con 0x7fede0069020 2024-03-21T10:31:58.750 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:58.751+0000 7fedde000700 1 --2- 172.21.15.153:0/4244500811 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fedc406e460 0x7fedc4070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:58.750 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:58.751+0000 7fedde000700 1 -- 172.21.15.153:0/4244500811 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fedcc08e410 con 0x7fede0069020 2024-03-21T10:31:58.751 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:58.752+0000 7fede5200700 1 --2- 172.21.15.153:0/4244500811 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fedc406e460 0x7fedc4070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:58.751 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:58.752+0000 7fede5200700 1 --2- 172.21.15.153:0/4244500811 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fedc406e460 0x7fedc4070920 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fedd000b830 tx=0x7fedd00061f0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:58.755 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:58.757+0000 7fedde000700 1 -- 172.21.15.153:0/4244500811 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fedcc05af60 con 0x7fede0069020 2024-03-21T10:31:58.993 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:58.994+0000 7fede7c00700 1 -- 172.21.15.153:0/4244500811 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fede0197080 con 0x7fedc406e460 2024-03-21T10:31:58.996 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:58.997+0000 7fedde000700 1 -- 172.21.15.153:0/4244500811 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fede0197080 con 0x7fedc406e460 2024-03-21T10:31:58.997 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:31:58.997 INFO:teuthology.orchestra.run.smithi153.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232", 2024-03-21T10:31:58.997 INFO:teuthology.orchestra.run.smithi153.stdout: "in_progress": true, 2024-03-21T10:31:58.997 INFO:teuthology.orchestra.run.smithi153.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-21T10:31:58.997 INFO:teuthology.orchestra.run.smithi153.stdout: "services_complete": [ 2024-03-21T10:31:58.997 INFO:teuthology.orchestra.run.smithi153.stdout: "crash", 2024-03-21T10:31:58.997 INFO:teuthology.orchestra.run.smithi153.stdout: "mon", 2024-03-21T10:31:58.997 INFO:teuthology.orchestra.run.smithi153.stdout: "osd", 2024-03-21T10:31:58.997 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr" 2024-03-21T10:31:58.997 INFO:teuthology.orchestra.run.smithi153.stdout: ], 2024-03-21T10:31:58.998 INFO:teuthology.orchestra.run.smithi153.stdout: "progress": "15/23 daemons upgraded", 2024-03-21T10:31:58.998 INFO:teuthology.orchestra.run.smithi153.stdout: "message": "Currently upgrading mds daemons", 2024-03-21T10:31:58.998 INFO:teuthology.orchestra.run.smithi153.stdout: "is_paused": false 2024-03-21T10:31:58.998 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:31:59.000 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:59.002+0000 7fede7c00700 1 -- 172.21.15.153:0/4244500811 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fedc406e460 msgr2=0x7fedc4070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:59.000 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:59.002+0000 7fede7c00700 1 --2- 172.21.15.153:0/4244500811 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fedc406e460 0x7fedc4070920 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fedd000b830 tx=0x7fedd00061f0 comp rx=0 tx=0).stop 2024-03-21T10:31:59.000 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:59.002+0000 7fede7c00700 1 -- 172.21.15.153:0/4244500811 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fede0069020 msgr2=0x7fede019ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:59.001 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:59.002+0000 7fede7c00700 1 --2- 172.21.15.153:0/4244500811 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fede0069020 0x7fede019ca50 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fedcc00d900 tx=0x7fedcc00dcc0 comp rx=0 tx=0).stop 2024-03-21T10:31:59.001 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:59.002+0000 7fede7c00700 1 -- 172.21.15.153:0/4244500811 shutdown_connections 2024-03-21T10:31:59.001 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:59.002+0000 7fede7c00700 1 --2- 172.21.15.153:0/4244500811 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fede0068670 0x7fede019c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:59.001 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:59.002+0000 7fede7c00700 1 --2- 172.21.15.153:0/4244500811 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fedc406e460 0x7fedc4070920 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:59.001 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:59.002+0000 7fede7c00700 1 --2- 172.21.15.153:0/4244500811 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fede0069020 0x7fede019ca50 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:59.001 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:59.002+0000 7fede7c00700 1 -- 172.21.15.153:0/4244500811 >> 172.21.15.153:0/4244500811 conn(0x7fede00755a0 msgr2=0x7fede00fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:59.001 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:59.002+0000 7fede7c00700 1 -- 172.21.15.153:0/4244500811 shutdown_connections 2024-03-21T10:31:59.001 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:59.002+0000 7fede7c00700 1 -- 172.21.15.153:0/4244500811 wait complete. 2024-03-21T10:31:59.153 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:59.154+0000 7f3aec400700 1 -- 172.21.15.153:0/3433616702 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ae4075be0 msgr2=0x7f3ae41114f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:59.153 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:59.154+0000 7f3aec400700 1 --2- 172.21.15.153:0/3433616702 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ae4075be0 0x7f3ae41114f0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f3ad0009910 tx=0x7f3ad0009c20 comp rx=0 tx=0).stop 2024-03-21T10:31:59.153 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:59.154+0000 7f3aec400700 1 -- 172.21.15.153:0/3433616702 shutdown_connections 2024-03-21T10:31:59.153 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:59.154+0000 7f3aec400700 1 --2- 172.21.15.153:0/3433616702 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ae4075be0 0x7f3ae41114f0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:59.153 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:59.154+0000 7f3aec400700 1 --2- 172.21.15.153:0/3433616702 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3ae40752c0 0x7f3ae40756a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:59.153 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:59.154+0000 7f3aec400700 1 -- 172.21.15.153:0/3433616702 >> 172.21.15.153:0/3433616702 conn(0x7f3ae40fe880 msgr2=0x7f3ae4100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:59.153 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:59.154+0000 7f3aec400700 1 -- 172.21.15.153:0/3433616702 shutdown_connections 2024-03-21T10:31:59.153 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:59.155+0000 7f3aec400700 1 -- 172.21.15.153:0/3433616702 wait complete. 2024-03-21T10:31:59.155 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:59.155+0000 7f3aec400700 1 Processor -- start 2024-03-21T10:31:59.155 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:59.156+0000 7f3aec400700 1 -- start start 2024-03-21T10:31:59.155 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:59.156+0000 7f3aec400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ae40752c0 0x7f3ae41a2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:59.155 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:59.156+0000 7f3aec400700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3ae4075be0 0x7f3ae41a3000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:59.155 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:59.156+0000 7f3aec400700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae41a3690 con 0x7f3ae40752c0 2024-03-21T10:31:59.155 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:59.156+0000 7f3aec400700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae419cb40 con 0x7f3ae4075be0 2024-03-21T10:31:59.155 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:59.156+0000 7f3ae9a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ae40752c0 0x7f3ae41a2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:59.155 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:59.156+0000 7f3ae9a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ae40752c0 0x7f3ae41a2ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:55952/0 (socket says 172.21.15.153:55952) 2024-03-21T10:31:59.155 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:59.156+0000 7f3ae9a00700 1 -- 172.21.15.153:0/733781256 learned_addr learned my addr 172.21.15.153:0/733781256 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:31:59.155 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:59.157+0000 7f3ae9000700 1 --2- 172.21.15.153:0/733781256 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3ae4075be0 0x7f3ae41a3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:59.155 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:59.157+0000 7f3ae9a00700 1 -- 172.21.15.153:0/733781256 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3ae4075be0 msgr2=0x7f3ae41a3000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:59.156 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:59.157+0000 7f3ae9a00700 1 --2- 172.21.15.153:0/733781256 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3ae4075be0 0x7f3ae41a3000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:59.156 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:59.157+0000 7f3ae9a00700 1 -- 172.21.15.153:0/733781256 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ad00095f0 con 0x7f3ae40752c0 2024-03-21T10:31:59.156 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:59.157+0000 7f3ae9a00700 1 --2- 172.21.15.153:0/733781256 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ae40752c0 0x7f3ae41a2ac0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f3ad800d8d0 tx=0x7f3ad800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:59.157 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:59.157+0000 7f3ae2000700 1 -- 172.21.15.153:0/733781256 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3ad8009940 con 0x7f3ae40752c0 2024-03-21T10:31:59.157 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:59.157+0000 7f3ae2000700 1 -- 172.21.15.153:0/733781256 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3ad8010460 con 0x7f3ae40752c0 2024-03-21T10:31:59.157 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:59.157+0000 7f3aec400700 1 -- 172.21.15.153:0/733781256 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ae419ce20 con 0x7f3ae40752c0 2024-03-21T10:31:59.157 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:59.157+0000 7f3ae2000700 1 -- 172.21.15.153:0/733781256 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3ad800f5f0 con 0x7f3ae40752c0 2024-03-21T10:31:59.157 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:59.157+0000 7f3aec400700 1 -- 172.21.15.153:0/733781256 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ae419d370 con 0x7f3ae40752c0 2024-03-21T10:31:59.157 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:59.158+0000 7f3aec400700 1 -- 172.21.15.153:0/733781256 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ae410ec60 con 0x7f3ae40752c0 2024-03-21T10:31:59.157 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:59.159+0000 7f3ae2000700 1 -- 172.21.15.153:0/733781256 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f3ad80105d0 con 0x7f3ae40752c0 2024-03-21T10:31:59.159 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:59.160+0000 7f3ae2000700 1 --2- 172.21.15.153:0/733781256 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f3acc06e460 0x7f3acc070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:31:59.159 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:59.160+0000 7f3ae2000700 1 -- 172.21.15.153:0/733781256 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f3ad808e3e0 con 0x7f3ae40752c0 2024-03-21T10:31:59.159 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:59.161+0000 7f3ae9000700 1 --2- 172.21.15.153:0/733781256 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f3acc06e460 0x7f3acc070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:31:59.159 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:59.161+0000 7f3ae9000700 1 --2- 172.21.15.153:0/733781256 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f3acc06e460 0x7f3acc070920 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f3ad0006010 tx=0x7f3ad000b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:31:59.163 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:59.165+0000 7f3ae2000700 1 -- 172.21.15.153:0/733781256 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f3ad805b050 con 0x7f3ae40752c0 2024-03-21T10:31:59.304 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:31:59 smithi153 ceph-mon[113733]: pgmap v603: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:31:59.305 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:31:59 smithi153 ceph-mon[113733]: from='client.35102 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:31:59.305 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:31:59 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/686142966' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:31:59.449 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:31:59 smithi174 ceph-mon[88004]: pgmap v603: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:31:59.449 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:31:59 smithi174 ceph-mon[88004]: from='client.35102 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:31:59.449 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:31:59 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/686142966' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:31:59.468 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:59.470+0000 7f3aec400700 1 -- 172.21.15.153:0/733781256 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f3ae419d6a0 con 0x7f3ae40752c0 2024-03-21T10:31:59.469 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:59.470+0000 7f3ae2000700 1 -- 172.21.15.153:0/733781256 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f3ad805abe0 con 0x7f3ae40752c0 2024-03-21T10:31:59.470 INFO:teuthology.orchestra.run.smithi153.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-03-21T10:31:59.470 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-21T10:31:59.470 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is degraded 2024-03-21T10:31:59.470 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-21T10:31:59.470 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-21T10:31:59.470 INFO:teuthology.orchestra.run.smithi153.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-21T10:31:59.471 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-21T10:31:59.472 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:59.473+0000 7f3aec400700 1 -- 172.21.15.153:0/733781256 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f3acc06e460 msgr2=0x7f3acc070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:59.472 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:59.473+0000 7f3aec400700 1 --2- 172.21.15.153:0/733781256 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f3acc06e460 0x7f3acc070920 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f3ad0006010 tx=0x7f3ad000b540 comp rx=0 tx=0).stop 2024-03-21T10:31:59.472 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:59.473+0000 7f3aec400700 1 -- 172.21.15.153:0/733781256 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ae40752c0 msgr2=0x7f3ae41a2ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:31:59.473 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:59.473+0000 7f3aec400700 1 --2- 172.21.15.153:0/733781256 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ae40752c0 0x7f3ae41a2ac0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f3ad800d8d0 tx=0x7f3ad800dc90 comp rx=0 tx=0).stop 2024-03-21T10:31:59.473 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:59.473+0000 7f3aec400700 1 -- 172.21.15.153:0/733781256 shutdown_connections 2024-03-21T10:31:59.473 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:59.473+0000 7f3aec400700 1 --2- 172.21.15.153:0/733781256 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f3acc06e460 0x7f3acc070920 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:59.473 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:59.473+0000 7f3aec400700 1 --2- 172.21.15.153:0/733781256 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3ae40752c0 0x7f3ae41a2ac0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:59.473 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:59.473+0000 7f3aec400700 1 --2- 172.21.15.153:0/733781256 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3ae4075be0 0x7f3ae41a3000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:31:59.473 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:59.473+0000 7f3aec400700 1 -- 172.21.15.153:0/733781256 >> 172.21.15.153:0/733781256 conn(0x7f3ae40fe880 msgr2=0x7f3ae4100570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:31:59.474 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:59.473+0000 7f3aec400700 1 -- 172.21.15.153:0/733781256 shutdown_connections 2024-03-21T10:31:59.474 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:31:59.473+0000 7f3aec400700 1 -- 172.21.15.153:0/733781256 wait complete. 2024-03-21T10:32:00.449 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:32:00 smithi174 ceph-mon[88004]: from='client.35114 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:32:00.449 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:32:00 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/733781256' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:32:00.554 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:32:00 smithi153 ceph-mon[113733]: from='client.35114 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:32:00.554 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:32:00 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/733781256' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:32:01.449 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:32:01 smithi174 ceph-mon[88004]: pgmap v604: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:32:01.554 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:32:01 smithi153 ceph-mon[113733]: pgmap v604: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:32:02.449 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:32:02 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:32:02.554 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:32:02 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:32:03.449 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:32:03 smithi174 ceph-mon[88004]: pgmap v605: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:32:03.449 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:32:03 smithi153 ceph-mon[113733]: pgmap v605: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:32:05.450 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:32:05 smithi174 ceph-mon[88004]: pgmap v606: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:32:05.554 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:32:05 smithi153 ceph-mon[113733]: pgmap v606: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:32:07.449 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:32:07 smithi174 ceph-mon[88004]: pgmap v607: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:32:07.554 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:32:07 smithi153 ceph-mon[113733]: pgmap v607: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:32:09.449 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:32:09 smithi174 ceph-mon[88004]: pgmap v608: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:32:09.554 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:32:09 smithi153 ceph-mon[113733]: pgmap v608: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:32:11.554 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:32:11 smithi153 ceph-mon[113733]: pgmap v609: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:32:11.562 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:32:11 smithi174 ceph-mon[88004]: pgmap v609: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:32:13.457 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:32:13 smithi174 ceph-mon[88004]: pgmap v610: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:32:13.554 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:32:13 smithi153 ceph-mon[113733]: pgmap v610: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:32:15.554 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:32:15 smithi153 ceph-mon[113733]: pgmap v611: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:32:15.699 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:32:15 smithi174 ceph-mon[88004]: pgmap v611: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:32:17.554 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:32:17 smithi153 ceph-mon[113733]: pgmap v612: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:32:17.554 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:32:17 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:32:17.699 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:32:17 smithi174 ceph-mon[88004]: pgmap v612: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:32:17.699 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:32:17 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:32:19.554 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:32:19 smithi153 ceph-mon[113733]: pgmap v613: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:32:19.699 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:32:19 smithi174 ceph-mon[88004]: pgmap v613: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:32:21.554 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:32:21 smithi153 ceph-mon[113733]: pgmap v614: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:32:21.699 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:32:21 smithi174 ceph-mon[88004]: pgmap v614: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:32:23.554 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:32:23 smithi153 ceph-mon[113733]: pgmap v615: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:32:23.699 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:32:23 smithi174 ceph-mon[88004]: pgmap v615: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:32:25.554 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:32:25 smithi153 ceph-mon[113733]: pgmap v616: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:32:25.699 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:32:25 smithi174 ceph-mon[88004]: pgmap v616: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:32:27.449 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:32:27 smithi174 ceph-mon[88004]: pgmap v617: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:32:27.554 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:32:27 smithi153 ceph-mon[113733]: pgmap v617: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:32:29.554 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:32:29 smithi153 ceph-mon[113733]: pgmap v618: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:32:29.642 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:29.642+0000 7f4a05200700 1 -- 172.21.15.153:0/4079917889 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4a00101750 msgr2=0x7f4a001057a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:29.642 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:29.642+0000 7f4a05200700 1 --2- 172.21.15.153:0/4079917889 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4a00101750 0x7f4a001057a0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f49e8009910 tx=0x7f49e8009c20 comp rx=0 tx=0).stop 2024-03-21T10:32:29.642 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:29.642+0000 7f4a05200700 1 -- 172.21.15.153:0/4079917889 shutdown_connections 2024-03-21T10:32:29.642 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:29.642+0000 7f4a05200700 1 --2- 172.21.15.153:0/4079917889 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4a00101750 0x7f4a001057a0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:29.642 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:29.642+0000 7f4a05200700 1 --2- 172.21.15.153:0/4079917889 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4a00100da0 0x7f4a00101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:29.642 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:29.642+0000 7f4a05200700 1 -- 172.21.15.153:0/4079917889 >> 172.21.15.153:0/4079917889 conn(0x7f4a000fc630 msgr2=0x7f4a000fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:32:29.642 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:29.642+0000 7f4a05200700 1 -- 172.21.15.153:0/4079917889 shutdown_connections 2024-03-21T10:32:29.642 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:29.642+0000 7f4a05200700 1 -- 172.21.15.153:0/4079917889 wait complete. 2024-03-21T10:32:29.643 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:29.644+0000 7f4a05200700 1 Processor -- start 2024-03-21T10:32:29.643 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:29.644+0000 7f4a05200700 1 -- start start 2024-03-21T10:32:29.643 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:29.644+0000 7f4a05200700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4a00100da0 0x7f4a00198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:29.643 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:29.644+0000 7f4a05200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4a00101750 0x7f4a00198e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:29.644 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:29.644+0000 7f4a05200700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a00199490 con 0x7f4a00101750 2024-03-21T10:32:29.644 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:29.644+0000 7f4a05200700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a00199600 con 0x7f4a00100da0 2024-03-21T10:32:29.646 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:29.645+0000 7f49f7e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4a00101750 0x7f4a00198e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:29.646 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:29.645+0000 7f49f7e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4a00101750 0x7f4a00198e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:47926/0 (socket says 172.21.15.153:47926) 2024-03-21T10:32:29.646 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:29.645+0000 7f49f7e00700 1 -- 172.21.15.153:0/1531252440 learned_addr learned my addr 172.21.15.153:0/1531252440 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:32:29.646 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:29.645+0000 7f49fde00700 1 --2- 172.21.15.153:0/1531252440 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4a00100da0 0x7f4a00198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:29.646 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:29.645+0000 7f49f7e00700 1 -- 172.21.15.153:0/1531252440 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4a00100da0 msgr2=0x7f4a00198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:29.646 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:29.645+0000 7f49f7e00700 1 --2- 172.21.15.153:0/1531252440 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4a00100da0 0x7f4a00198950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:29.646 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:29.645+0000 7f49f7e00700 1 -- 172.21.15.153:0/1531252440 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49e80095f0 con 0x7f4a00101750 2024-03-21T10:32:29.646 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:29.645+0000 7f49f7e00700 1 --2- 172.21.15.153:0/1531252440 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4a00101750 0x7f4a00198e90 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f49e80092e0 tx=0x7f49e800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:32:29.647 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:29.646+0000 7f49f6a00700 1 -- 172.21.15.153:0/1531252440 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f49e801d070 con 0x7f4a00101750 2024-03-21T10:32:29.647 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:29.646+0000 7f4a05200700 1 -- 172.21.15.153:0/1531252440 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a0019d420 con 0x7f4a00101750 2024-03-21T10:32:29.647 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:29.646+0000 7f49f6a00700 1 -- 172.21.15.153:0/1531252440 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f49e800f460 con 0x7f4a00101750 2024-03-21T10:32:29.647 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:29.646+0000 7f49f6a00700 1 -- 172.21.15.153:0/1531252440 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f49e80175f0 con 0x7f4a00101750 2024-03-21T10:32:29.647 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:29.646+0000 7f4a05200700 1 -- 172.21.15.153:0/1531252440 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a0019d910 con 0x7f4a00101750 2024-03-21T10:32:29.647 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:29.647+0000 7f4a05200700 1 -- 172.21.15.153:0/1531252440 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a0004ea90 con 0x7f4a00101750 2024-03-21T10:32:29.648 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:29.648+0000 7f49f6a00700 1 -- 172.21.15.153:0/1531252440 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f49e800f5d0 con 0x7f4a00101750 2024-03-21T10:32:29.648 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:29.649+0000 7f49f6a00700 1 --2- 172.21.15.153:0/1531252440 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f49e406e460 0x7f49e4070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:29.648 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:29.649+0000 7f49f6a00700 1 -- 172.21.15.153:0/1531252440 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f49e808faa0 con 0x7f4a00101750 2024-03-21T10:32:29.648 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:29.649+0000 7f49fde00700 1 --2- 172.21.15.153:0/1531252440 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f49e406e460 0x7f49e4070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:29.649 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:29.650+0000 7f49fde00700 1 --2- 172.21.15.153:0/1531252440 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f49e406e460 0x7f49e4070920 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f49f000ba60 tx=0x7f49f0005d50 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:32:29.653 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:29.654+0000 7f49f6a00700 1 -- 172.21.15.153:0/1531252440 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f49e805c5f0 con 0x7f4a00101750 2024-03-21T10:32:29.699 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:32:29 smithi174 ceph-mon[88004]: pgmap v618: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:32:29.894 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:29.894+0000 7f4a05200700 1 -- 172.21.15.153:0/1531252440 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4a00104b70 con 0x7f49e406e460 2024-03-21T10:32:29.896 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:29.897+0000 7f49f6a00700 1 -- 172.21.15.153:0/1531252440 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f4a00104b70 con 0x7f49e406e460 2024-03-21T10:32:29.900 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:29.901+0000 7f4a05200700 1 -- 172.21.15.153:0/1531252440 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f49e406e460 msgr2=0x7f49e4070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:29.900 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:29.901+0000 7f4a05200700 1 --2- 172.21.15.153:0/1531252440 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f49e406e460 0x7f49e4070920 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f49f000ba60 tx=0x7f49f0005d50 comp rx=0 tx=0).stop 2024-03-21T10:32:29.900 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:29.901+0000 7f4a05200700 1 -- 172.21.15.153:0/1531252440 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4a00101750 msgr2=0x7f4a00198e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:29.900 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:29.901+0000 7f4a05200700 1 --2- 172.21.15.153:0/1531252440 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4a00101750 0x7f4a00198e90 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f49e80092e0 tx=0x7f49e800be20 comp rx=0 tx=0).stop 2024-03-21T10:32:29.900 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:29.901+0000 7f4a05200700 1 -- 172.21.15.153:0/1531252440 shutdown_connections 2024-03-21T10:32:29.900 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:29.901+0000 7f4a05200700 1 --2- 172.21.15.153:0/1531252440 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4a00100da0 0x7f4a00198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:29.901 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:29.901+0000 7f4a05200700 1 --2- 172.21.15.153:0/1531252440 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f49e406e460 0x7f49e4070920 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:29.901 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:29.901+0000 7f4a05200700 1 --2- 172.21.15.153:0/1531252440 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4a00101750 0x7f4a00198e90 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:29.901 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:29.901+0000 7f4a05200700 1 -- 172.21.15.153:0/1531252440 >> 172.21.15.153:0/1531252440 conn(0x7f4a000fc630 msgr2=0x7f4a000fe9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:32:29.901 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:29.901+0000 7f4a05200700 1 -- 172.21.15.153:0/1531252440 shutdown_connections 2024-03-21T10:32:29.901 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:29.901+0000 7f4a05200700 1 -- 172.21.15.153:0/1531252440 wait complete. 2024-03-21T10:32:29.916 INFO:teuthology.orchestra.run.smithi153.stdout:true 2024-03-21T10:32:30.054 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.055+0000 7f0bfd600700 1 -- 172.21.15.153:0/3805326666 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0bf810d280 msgr2=0x7f0bf810d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:30.055 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.055+0000 7f0bfd600700 1 --2- 172.21.15.153:0/3805326666 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0bf810d280 0x7f0bf810d660 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f0be8009910 tx=0x7f0be8009c20 comp rx=0 tx=0).stop 2024-03-21T10:32:30.055 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.055+0000 7f0bfd600700 1 -- 172.21.15.153:0/3805326666 shutdown_connections 2024-03-21T10:32:30.055 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.055+0000 7f0bfd600700 1 --2- 172.21.15.153:0/3805326666 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0bf81071d0 0x7f0bf8107650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:30.055 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.055+0000 7f0bfd600700 1 --2- 172.21.15.153:0/3805326666 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0bf810d280 0x7f0bf810d660 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:30.055 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.055+0000 7f0bfd600700 1 -- 172.21.15.153:0/3805326666 >> 172.21.15.153:0/3805326666 conn(0x7f0bf80769e0 msgr2=0x7f0bf8078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:32:30.055 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.055+0000 7f0bfd600700 1 -- 172.21.15.153:0/3805326666 shutdown_connections 2024-03-21T10:32:30.055 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.055+0000 7f0bfd600700 1 -- 172.21.15.153:0/3805326666 wait complete. 2024-03-21T10:32:30.055 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.056+0000 7f0bfd600700 1 Processor -- start 2024-03-21T10:32:30.056 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.057+0000 7f0bfd600700 1 -- start start 2024-03-21T10:32:30.057 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.057+0000 7f0bfd600700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0bf81071d0 0x7f0bf819ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:30.057 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.057+0000 7f0bfd600700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0bf810d280 0x7f0bf819d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:30.057 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.057+0000 7f0bfd600700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf819d8a0 con 0x7f0bf810d280 2024-03-21T10:32:30.057 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.057+0000 7f0bfd600700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf81a05d0 con 0x7f0bf81071d0 2024-03-21T10:32:30.057 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.057+0000 7f0bf6000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0bf810d280 0x7f0bf819d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:30.057 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.057+0000 7f0bf6000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0bf810d280 0x7f0bf819d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:47946/0 (socket says 172.21.15.153:47946) 2024-03-21T10:32:30.057 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.057+0000 7f0bf6000700 1 -- 172.21.15.153:0/448014586 learned_addr learned my addr 172.21.15.153:0/448014586 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:32:30.057 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.058+0000 7f0bf6a00700 1 --2- 172.21.15.153:0/448014586 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0bf81071d0 0x7f0bf819ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:30.057 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.058+0000 7f0bf6000700 1 -- 172.21.15.153:0/448014586 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0bf81071d0 msgr2=0x7f0bf819ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:30.057 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.058+0000 7f0bf6000700 1 --2- 172.21.15.153:0/448014586 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0bf81071d0 0x7f0bf819ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:30.057 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.058+0000 7f0bf6000700 1 -- 172.21.15.153:0/448014586 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0be80095f0 con 0x7f0bf810d280 2024-03-21T10:32:30.058 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.058+0000 7f0bf6000700 1 --2- 172.21.15.153:0/448014586 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0bf810d280 0x7f0bf819d210 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f0be400d8d0 tx=0x7f0be400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:32:30.058 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.058+0000 7f0bef400700 1 -- 172.21.15.153:0/448014586 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0be4009940 con 0x7f0bf810d280 2024-03-21T10:32:30.058 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.058+0000 7f0bef400700 1 -- 172.21.15.153:0/448014586 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0be4010460 con 0x7f0bf810d280 2024-03-21T10:32:30.058 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.059+0000 7f0bfd600700 1 -- 172.21.15.153:0/448014586 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bf81a08b0 con 0x7f0bf810d280 2024-03-21T10:32:30.058 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.059+0000 7f0bef400700 1 -- 172.21.15.153:0/448014586 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0be400f5f0 con 0x7f0bf810d280 2024-03-21T10:32:30.059 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.059+0000 7f0bfd600700 1 -- 172.21.15.153:0/448014586 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bf81a0e80 con 0x7f0bf810d280 2024-03-21T10:32:30.059 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.060+0000 7f0bfd600700 1 -- 172.21.15.153:0/448014586 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bf804ea90 con 0x7f0bf810d280 2024-03-21T10:32:30.060 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.061+0000 7f0bef400700 1 -- 172.21.15.153:0/448014586 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f0be40105d0 con 0x7f0bf810d280 2024-03-21T10:32:30.061 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.062+0000 7f0bef400700 1 --2- 172.21.15.153:0/448014586 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f0bdc06e460 0x7f0bdc070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:30.061 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.062+0000 7f0bef400700 1 -- 172.21.15.153:0/448014586 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f0be408e5b0 con 0x7f0bf810d280 2024-03-21T10:32:30.061 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.062+0000 7f0bf6a00700 1 --2- 172.21.15.153:0/448014586 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f0bdc06e460 0x7f0bdc070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:30.062 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.063+0000 7f0bf6a00700 1 --2- 172.21.15.153:0/448014586 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f0bdc06e460 0x7f0bdc070920 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f0be8006010 tx=0x7f0be801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:32:30.065 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.066+0000 7f0bef400700 1 -- 172.21.15.153:0/448014586 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0be40575d0 con 0x7f0bf810d280 2024-03-21T10:32:30.295 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.296+0000 7f0bfd600700 1 -- 172.21.15.153:0/448014586 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0bf81a1290 con 0x7f0bdc06e460 2024-03-21T10:32:30.296 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.297+0000 7f0bef400700 1 -- 172.21.15.153:0/448014586 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f0bf81a1290 con 0x7f0bdc06e460 2024-03-21T10:32:30.299 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.300+0000 7f0bfd600700 1 -- 172.21.15.153:0/448014586 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f0bdc06e460 msgr2=0x7f0bdc070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:30.299 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.300+0000 7f0bfd600700 1 --2- 172.21.15.153:0/448014586 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f0bdc06e460 0x7f0bdc070920 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f0be8006010 tx=0x7f0be801a040 comp rx=0 tx=0).stop 2024-03-21T10:32:30.299 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.300+0000 7f0bfd600700 1 -- 172.21.15.153:0/448014586 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0bf810d280 msgr2=0x7f0bf819d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:30.299 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.300+0000 7f0bfd600700 1 --2- 172.21.15.153:0/448014586 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0bf810d280 0x7f0bf819d210 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f0be400d8d0 tx=0x7f0be400dc90 comp rx=0 tx=0).stop 2024-03-21T10:32:30.299 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.300+0000 7f0bfd600700 1 -- 172.21.15.153:0/448014586 shutdown_connections 2024-03-21T10:32:30.300 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.300+0000 7f0bfd600700 1 --2- 172.21.15.153:0/448014586 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0bf81071d0 0x7f0bf819ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:30.300 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.300+0000 7f0bfd600700 1 --2- 172.21.15.153:0/448014586 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f0bdc06e460 0x7f0bdc070920 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:30.300 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.300+0000 7f0bfd600700 1 --2- 172.21.15.153:0/448014586 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0bf810d280 0x7f0bf819d210 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:30.300 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.300+0000 7f0bfd600700 1 -- 172.21.15.153:0/448014586 >> 172.21.15.153:0/448014586 conn(0x7f0bf80769e0 msgr2=0x7f0bf8078320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:32:30.300 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.300+0000 7f0bfd600700 1 -- 172.21.15.153:0/448014586 shutdown_connections 2024-03-21T10:32:30.300 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.300+0000 7f0bfd600700 1 -- 172.21.15.153:0/448014586 wait complete. 2024-03-21T10:32:30.454 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.454+0000 7f974f000700 1 -- 172.21.15.153:0/3363573056 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9748101d00 msgr2=0x7f97481020e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:30.454 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.454+0000 7f974f000700 1 --2- 172.21.15.153:0/3363573056 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9748101d00 0x7f97481020e0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f9738009910 tx=0x7f9738009c20 comp rx=0 tx=0).stop 2024-03-21T10:32:30.454 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.455+0000 7f974f000700 1 -- 172.21.15.153:0/3363573056 shutdown_connections 2024-03-21T10:32:30.454 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.455+0000 7f974f000700 1 --2- 172.21.15.153:0/3363573056 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9748102620 0x7f974810ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:30.454 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.455+0000 7f974f000700 1 --2- 172.21.15.153:0/3363573056 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9748101d00 0x7f97481020e0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:30.455 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.455+0000 7f974f000700 1 -- 172.21.15.153:0/3363573056 >> 172.21.15.153:0/3363573056 conn(0x7f97480fb570 msgr2=0x7f97480fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:32:30.455 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.455+0000 7f974f000700 1 -- 172.21.15.153:0/3363573056 shutdown_connections 2024-03-21T10:32:30.455 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.455+0000 7f974f000700 1 -- 172.21.15.153:0/3363573056 wait complete. 2024-03-21T10:32:30.456 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.456+0000 7f974f000700 1 Processor -- start 2024-03-21T10:32:30.456 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.456+0000 7f974f000700 1 -- start start 2024-03-21T10:32:30.456 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.456+0000 7f974f000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9748101d00 0x7f974819c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:30.456 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.456+0000 7f974f000700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9748102620 0x7f974819ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:30.456 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.456+0000 7f974f000700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f974819d100 con 0x7f9748101d00 2024-03-21T10:32:30.456 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.456+0000 7f974f000700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97481965b0 con 0x7f9748102620 2024-03-21T10:32:30.456 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.456+0000 7f9747e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9748101d00 0x7f974819c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:30.456 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.457+0000 7f9747e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9748101d00 0x7f974819c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:47968/0 (socket says 172.21.15.153:47968) 2024-03-21T10:32:30.456 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.457+0000 7f9747e00700 1 -- 172.21.15.153:0/845580270 learned_addr learned my addr 172.21.15.153:0/845580270 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:32:30.456 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.457+0000 7f9747400700 1 --2- 172.21.15.153:0/845580270 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9748102620 0x7f974819ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:30.456 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.457+0000 7f9747e00700 1 -- 172.21.15.153:0/845580270 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9748102620 msgr2=0x7f974819ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:30.456 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.457+0000 7f9747e00700 1 --2- 172.21.15.153:0/845580270 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9748102620 0x7f974819ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:30.457 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.457+0000 7f9747e00700 1 -- 172.21.15.153:0/845580270 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97380095f0 con 0x7f9748101d00 2024-03-21T10:32:30.457 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.457+0000 7f9747e00700 1 --2- 172.21.15.153:0/845580270 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9748101d00 0x7f974819c530 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f9738006010 tx=0x7f973800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:32:30.457 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.458+0000 7f9744c00700 1 -- 172.21.15.153:0/845580270 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f973801d070 con 0x7f9748101d00 2024-03-21T10:32:30.457 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.458+0000 7f9744c00700 1 -- 172.21.15.153:0/845580270 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f973800f460 con 0x7f9748101d00 2024-03-21T10:32:30.457 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.458+0000 7f974f000700 1 -- 172.21.15.153:0/845580270 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9748196830 con 0x7f9748101d00 2024-03-21T10:32:30.458 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.458+0000 7f974f000700 1 -- 172.21.15.153:0/845580270 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9748196d20 con 0x7f9748101d00 2024-03-21T10:32:30.458 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.458+0000 7f9744c00700 1 -- 172.21.15.153:0/845580270 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9738017610 con 0x7f9748101d00 2024-03-21T10:32:30.458 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.459+0000 7f974f000700 1 -- 172.21.15.153:0/845580270 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9748108210 con 0x7f9748101d00 2024-03-21T10:32:30.459 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.460+0000 7f9744c00700 1 -- 172.21.15.153:0/845580270 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f973800f5d0 con 0x7f9748101d00 2024-03-21T10:32:30.460 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.461+0000 7f9744c00700 1 --2- 172.21.15.153:0/845580270 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f972c06e460 0x7f972c070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:30.460 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.461+0000 7f9744c00700 1 -- 172.21.15.153:0/845580270 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f973808f9d0 con 0x7f9748101d00 2024-03-21T10:32:30.460 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.461+0000 7f9747400700 1 --2- 172.21.15.153:0/845580270 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f972c06e460 0x7f972c070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:30.461 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.462+0000 7f9747400700 1 --2- 172.21.15.153:0/845580270 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f972c06e460 0x7f972c070920 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f9734005950 tx=0x7f97340058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:32:30.465 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.466+0000 7f9744c00700 1 -- 172.21.15.153:0/845580270 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f973805c5e0 con 0x7f9748101d00 2024-03-21T10:32:30.699 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.700+0000 7f974f000700 1 -- 172.21.15.153:0/845580270 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f974802cca0 con 0x7f972c06e460 2024-03-21T10:32:30.730 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.731+0000 7f9744c00700 1 -- 172.21.15.153:0/845580270 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f974802cca0 con 0x7f972c06e460 2024-03-21T10:32:30.731 INFO:teuthology.orchestra.run.smithi153.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-21T10:32:30.731 INFO:teuthology.orchestra.run.smithi153.stdout:alertmanager.smithi153 smithi153 *:9093,9094 running (21m) 12m ago 35m 14.7M - 0.25.0 c8568f914cd2 9ed3f2200b5f 2024-03-21T10:32:30.731 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi153 smithi153 running (35m) 12m ago 35m 9424k - 18.2.1 5be31c24972a 5b61994396a6 2024-03-21T10:32:30.731 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi174 smithi174 running (33m) 12m ago 33m 9244k - 18.2.1 5be31c24972a 4f7a5fc3ac28 2024-03-21T10:32:30.731 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi153 smithi153 running (17m) 12m ago 35m 6656k - 19.0.0-2231-g2edfee68 237e43c7a65f 32c21104d5b0 2024-03-21T10:32:30.731 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi174 smithi174 running (17m) 12m ago 33m 6660k - 19.0.0-2231-g2edfee68 237e43c7a65f a04e48967ab6 2024-03-21T10:32:30.731 INFO:teuthology.orchestra.run.smithi153.stdout:grafana.smithi153 smithi153 *:3000 running (20m) 12m ago 34m 73.6M - 9.4.12 2bacad6d85d8 ee24026ab786 2024-03-21T10:32:30.732 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.iuyluj smithi153 running (12m) 12m ago 27m 10.9M - 19.0.0-2231-g2edfee68 237e43c7a65f 87aff8a5de70 2024-03-21T10:32:30.732 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.zipckr smithi153 running (13m) 12m ago 27m 11.8M - 19.0.0-2231-g2edfee68 237e43c7a65f ccc262bd5dd3 2024-03-21T10:32:30.732 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.nhozag smithi174 running (27m) 12m ago 27m 12.0M - 18.2.1 5be31c24972a 2fee4000308f 2024-03-21T10:32:30.732 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.yyimst smithi174 running (12m) 12m ago 27m 10.7M - 19.0.0-2231-g2edfee68 237e43c7a65f fc1962f9351e 2024-03-21T10:32:30.732 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi153.tluxni smithi153 *:8443,9283,8765 running (24m) 12m ago 36m 509M - 19.0.0-2231-g2edfee68 237e43c7a65f 7ffee0e55382 2024-03-21T10:32:30.732 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi174.qpcwat smithi174 *:8443,9283,8765 running (22m) 12m ago 33m 452M - 19.0.0-2231-g2edfee68 237e43c7a65f 748968f216ab 2024-03-21T10:32:30.732 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi153 smithi153 running (19m) 12m ago 37m 55.0M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f dd8038303dc3 2024-03-21T10:32:30.732 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi174 smithi174 running (18m) 12m ago 33m 44.2M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f 29d78a788f2c 2024-03-21T10:32:30.732 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi153 smithi153 *:9100 running (22m) 12m ago 35m 7889k - 1.5.0 0da6a335fe13 bb835edc0e0b 2024-03-21T10:32:30.732 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi174 smithi174 *:9100 running (22m) 12m ago 33m 8102k - 1.5.0 0da6a335fe13 7c4ecad790fa 2024-03-21T10:32:30.732 INFO:teuthology.orchestra.run.smithi153.stdout:osd.0 smithi153 running (16m) 12m ago 32m 116M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 507900fd4791 2024-03-21T10:32:30.732 INFO:teuthology.orchestra.run.smithi153.stdout:osd.1 smithi153 running (16m) 12m ago 31m 89.2M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 0d646452e9c1 2024-03-21T10:32:30.733 INFO:teuthology.orchestra.run.smithi153.stdout:osd.2 smithi153 running (15m) 12m ago 31m 87.4M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 1326f2afdfcd 2024-03-21T10:32:30.733 INFO:teuthology.orchestra.run.smithi153.stdout:osd.3 smithi174 running (15m) 12m ago 30m 83.6M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 70aafac71ad5 2024-03-21T10:32:30.733 INFO:teuthology.orchestra.run.smithi153.stdout:osd.4 smithi174 running (14m) 12m ago 30m 92.2M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 0d746ce6e79d 2024-03-21T10:32:30.733 INFO:teuthology.orchestra.run.smithi153.stdout:osd.5 smithi174 running (13m) 12m ago 29m 86.2M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 53e5f35b8cac 2024-03-21T10:32:30.733 INFO:teuthology.orchestra.run.smithi153.stdout:prometheus.smithi153 smithi153 *:9095 running (21m) 12m ago 34m 73.5M - 2.43.0 a07b618ecd1d 7d95561e50c2 2024-03-21T10:32:30.734 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.734+0000 7f974f000700 1 -- 172.21.15.153:0/845580270 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f972c06e460 msgr2=0x7f972c070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:30.734 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.734+0000 7f974f000700 1 --2- 172.21.15.153:0/845580270 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f972c06e460 0x7f972c070920 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f9734005950 tx=0x7f97340058e0 comp rx=0 tx=0).stop 2024-03-21T10:32:30.734 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.734+0000 7f974f000700 1 -- 172.21.15.153:0/845580270 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9748101d00 msgr2=0x7f974819c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:30.734 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.734+0000 7f974f000700 1 --2- 172.21.15.153:0/845580270 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9748101d00 0x7f974819c530 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f9738006010 tx=0x7f973800be20 comp rx=0 tx=0).stop 2024-03-21T10:32:30.734 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.734+0000 7f974f000700 1 -- 172.21.15.153:0/845580270 shutdown_connections 2024-03-21T10:32:30.735 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.734+0000 7f974f000700 1 --2- 172.21.15.153:0/845580270 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f972c06e460 0x7f972c070920 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:30.735 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.734+0000 7f974f000700 1 --2- 172.21.15.153:0/845580270 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9748101d00 0x7f974819c530 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:30.735 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.734+0000 7f974f000700 1 --2- 172.21.15.153:0/845580270 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9748102620 0x7f974819ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:30.735 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.734+0000 7f974f000700 1 -- 172.21.15.153:0/845580270 >> 172.21.15.153:0/845580270 conn(0x7f97480fb570 msgr2=0x7f9748105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:32:30.735 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.734+0000 7f974f000700 1 -- 172.21.15.153:0/845580270 shutdown_connections 2024-03-21T10:32:30.735 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.735+0000 7f974f000700 1 -- 172.21.15.153:0/845580270 wait complete. 2024-03-21T10:32:30.892 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.891+0000 7fcdbae00700 1 -- 172.21.15.153:0/436224792 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcdb40731c0 msgr2=0x7fcdb40735a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:30.892 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.891+0000 7fcdbae00700 1 --2- 172.21.15.153:0/436224792 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcdb40731c0 0x7fcdb40735a0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fcda4009910 tx=0x7fcda4009c20 comp rx=0 tx=0).stop 2024-03-21T10:32:30.892 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.891+0000 7fcdbae00700 1 -- 172.21.15.153:0/436224792 shutdown_connections 2024-03-21T10:32:30.892 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.891+0000 7fcdbae00700 1 --2- 172.21.15.153:0/436224792 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcdb4073ae0 0x7fcdb410ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:30.893 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.891+0000 7fcdbae00700 1 --2- 172.21.15.153:0/436224792 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcdb40731c0 0x7fcdb40735a0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:30.893 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.891+0000 7fcdbae00700 1 -- 172.21.15.153:0/436224792 >> 172.21.15.153:0/436224792 conn(0x7fcdb40fc650 msgr2=0x7fcdb40fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:32:30.894 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.891+0000 7fcdbae00700 1 -- 172.21.15.153:0/436224792 shutdown_connections 2024-03-21T10:32:30.894 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.891+0000 7fcdbae00700 1 -- 172.21.15.153:0/436224792 wait complete. 2024-03-21T10:32:30.894 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.892+0000 7fcdbae00700 1 Processor -- start 2024-03-21T10:32:30.894 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.892+0000 7fcdbae00700 1 -- start start 2024-03-21T10:32:30.898 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.893+0000 7fcdbae00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcdb40731c0 0x7fcdb4198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:30.898 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.893+0000 7fcdbae00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcdb4073ae0 0x7fcdb4198ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:30.898 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.893+0000 7fcdbae00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdb4199560 con 0x7fcdb4073ae0 2024-03-21T10:32:30.898 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.893+0000 7fcdbae00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdb419d370 con 0x7fcdb40731c0 2024-03-21T10:32:30.899 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.893+0000 7fcdb3400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcdb4073ae0 0x7fcdb4198ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:30.899 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.893+0000 7fcdb3400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcdb4073ae0 0x7fcdb4198ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:47988/0 (socket says 172.21.15.153:47988) 2024-03-21T10:32:30.899 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.893+0000 7fcdb3400700 1 -- 172.21.15.153:0/3764573262 learned_addr learned my addr 172.21.15.153:0/3764573262 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:32:30.899 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.893+0000 7fcdb3e00700 1 --2- 172.21.15.153:0/3764573262 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcdb40731c0 0x7fcdb4198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:30.899 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.894+0000 7fcdb3400700 1 -- 172.21.15.153:0/3764573262 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcdb40731c0 msgr2=0x7fcdb4198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:30.900 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.894+0000 7fcdb3400700 1 --2- 172.21.15.153:0/3764573262 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcdb40731c0 0x7fcdb4198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:30.900 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.894+0000 7fcdb3400700 1 -- 172.21.15.153:0/3764573262 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcda40095f0 con 0x7fcdb4073ae0 2024-03-21T10:32:30.900 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.894+0000 7fcdb3400700 1 --2- 172.21.15.153:0/3764573262 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcdb4073ae0 0x7fcdb4198ed0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fcda000d8d0 tx=0x7fcda000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:32:30.900 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.894+0000 7fcdb0c00700 1 -- 172.21.15.153:0/3764573262 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcda0009940 con 0x7fcdb4073ae0 2024-03-21T10:32:30.900 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.894+0000 7fcdb0c00700 1 -- 172.21.15.153:0/3764573262 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcda0010460 con 0x7fcdb4073ae0 2024-03-21T10:32:30.900 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.894+0000 7fcdbae00700 1 -- 172.21.15.153:0/3764573262 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdb419d650 con 0x7fcdb4073ae0 2024-03-21T10:32:30.901 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.894+0000 7fcdb0c00700 1 -- 172.21.15.153:0/3764573262 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcda000f5f0 con 0x7fcdb4073ae0 2024-03-21T10:32:30.901 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.894+0000 7fcdbae00700 1 -- 172.21.15.153:0/3764573262 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdb419dba0 con 0x7fcdb4073ae0 2024-03-21T10:32:30.901 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.895+0000 7fcdbae00700 1 -- 172.21.15.153:0/3764573262 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdb410a560 con 0x7fcdb4073ae0 2024-03-21T10:32:30.901 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.897+0000 7fcdb0c00700 1 -- 172.21.15.153:0/3764573262 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fcda00105d0 con 0x7fcdb4073ae0 2024-03-21T10:32:30.901 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.897+0000 7fcdb0c00700 1 --2- 172.21.15.153:0/3764573262 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fcd9806e460 0x7fcd98070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:30.901 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.898+0000 7fcdb0c00700 1 -- 172.21.15.153:0/3764573262 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fcda008e480 con 0x7fcdb4073ae0 2024-03-21T10:32:30.901 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.898+0000 7fcdb3e00700 1 --2- 172.21.15.153:0/3764573262 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fcd9806e460 0x7fcd98070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:30.902 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.898+0000 7fcdb3e00700 1 --2- 172.21.15.153:0/3764573262 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fcd9806e460 0x7fcd98070920 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fcda4005b40 tx=0x7fcda401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:32:30.902 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:30.902+0000 7fcdb0c00700 1 -- 172.21.15.153:0/3764573262 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fcda005b050 con 0x7fcdb4073ae0 2024-03-21T10:32:31.197 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:31.197+0000 7fcdbae00700 1 -- 172.21.15.153:0/3764573262 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fcdb419de10 con 0x7fcdb4073ae0 2024-03-21T10:32:31.198 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:31.198+0000 7fcdb0c00700 1 -- 172.21.15.153:0/3764573262 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+805 (secure 0 0 0) 0x7fcda005abe0 con 0x7fcdb4073ae0 2024-03-21T10:32:31.198 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:32:31.198 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": { 2024-03-21T10:32:31.198 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:32:31.199 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:32:31.199 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": { 2024-03-21T10:32:31.199 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:32:31.199 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:32:31.199 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": { 2024-03-21T10:32:31.199 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 6 2024-03-21T10:32:31.199 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:32:31.200 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": { 2024-03-21T10:32:31.200 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-03-21T10:32:31.200 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 3 2024-03-21T10:32:31.200 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:32:31.200 INFO:teuthology.orchestra.run.smithi153.stdout: "overall": { 2024-03-21T10:32:31.200 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-03-21T10:32:31.201 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 13 2024-03-21T10:32:31.201 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-03-21T10:32:31.201 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:32:31.202 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:31.201+0000 7fcdbae00700 1 -- 172.21.15.153:0/3764573262 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fcd9806e460 msgr2=0x7fcd98070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:31.202 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:31.201+0000 7fcdbae00700 1 --2- 172.21.15.153:0/3764573262 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fcd9806e460 0x7fcd98070920 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fcda4005b40 tx=0x7fcda401a040 comp rx=0 tx=0).stop 2024-03-21T10:32:31.202 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:31.201+0000 7fcdbae00700 1 -- 172.21.15.153:0/3764573262 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcdb4073ae0 msgr2=0x7fcdb4198ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:31.203 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:31.201+0000 7fcdbae00700 1 --2- 172.21.15.153:0/3764573262 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcdb4073ae0 0x7fcdb4198ed0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fcda000d8d0 tx=0x7fcda000dc90 comp rx=0 tx=0).stop 2024-03-21T10:32:31.203 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:31.201+0000 7fcdbae00700 1 -- 172.21.15.153:0/3764573262 shutdown_connections 2024-03-21T10:32:31.203 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:31.201+0000 7fcdbae00700 1 --2- 172.21.15.153:0/3764573262 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcdb40731c0 0x7fcdb4198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.203 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:31.201+0000 7fcdbae00700 1 --2- 172.21.15.153:0/3764573262 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fcd9806e460 0x7fcd98070920 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.203 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:31.202+0000 7fcdbae00700 1 --2- 172.21.15.153:0/3764573262 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcdb4073ae0 0x7fcdb4198ed0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.203 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:31.202+0000 7fcdbae00700 1 -- 172.21.15.153:0/3764573262 >> 172.21.15.153:0/3764573262 conn(0x7fcdb40fc650 msgr2=0x7fcdb41076a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:32:31.204 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:31.202+0000 7fcdbae00700 1 -- 172.21.15.153:0/3764573262 shutdown_connections 2024-03-21T10:32:31.204 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:31.202+0000 7fcdbae00700 1 -- 172.21.15.153:0/3764573262 wait complete. 2024-03-21T10:32:31.353 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:31.354+0000 7fb595800700 1 -- 172.21.15.153:0/2976663764 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb590073160 msgr2=0x7fb590073540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:31.353 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:31.354+0000 7fb595800700 1 --2- 172.21.15.153:0/2976663764 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb590073160 0x7fb590073540 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fb580009910 tx=0x7fb580009c20 comp rx=0 tx=0).stop 2024-03-21T10:32:31.353 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:31.354+0000 7fb595800700 1 -- 172.21.15.153:0/2976663764 shutdown_connections 2024-03-21T10:32:31.353 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:31.354+0000 7fb595800700 1 --2- 172.21.15.153:0/2976663764 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb590073a80 0x7fb59010cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.353 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:31.354+0000 7fb595800700 1 --2- 172.21.15.153:0/2976663764 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb590073160 0x7fb590073540 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.354 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:31.354+0000 7fb595800700 1 -- 172.21.15.153:0/2976663764 >> 172.21.15.153:0/2976663764 conn(0x7fb5900fc650 msgr2=0x7fb5900fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:32:31.354 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:31.354+0000 7fb595800700 1 -- 172.21.15.153:0/2976663764 shutdown_connections 2024-03-21T10:32:31.354 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:31.354+0000 7fb595800700 1 -- 172.21.15.153:0/2976663764 wait complete. 2024-03-21T10:32:31.354 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:31.355+0000 7fb595800700 1 Processor -- start 2024-03-21T10:32:31.355 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:31.355+0000 7fb595800700 1 -- start start 2024-03-21T10:32:31.355 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:31.355+0000 7fb595800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb590073160 0x7fb59019e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:31.355 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:31.355+0000 7fb595800700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb590073a80 0x7fb59019ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:31.355 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:31.355+0000 7fb595800700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb59019f400 con 0x7fb590073160 2024-03-21T10:32:31.355 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:31.355+0000 7fb595800700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5901988b0 con 0x7fb590073a80 2024-03-21T10:32:31.355 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:31.356+0000 7fb58ea00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb590073160 0x7fb59019e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:31.355 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:31.356+0000 7fb58ea00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb590073160 0x7fb59019e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:48006/0 (socket says 172.21.15.153:48006) 2024-03-21T10:32:31.355 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:31.356+0000 7fb58ea00700 1 -- 172.21.15.153:0/277928445 learned_addr learned my addr 172.21.15.153:0/277928445 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:32:31.356 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:31.356+0000 7fb58ea00700 1 -- 172.21.15.153:0/277928445 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb590073a80 msgr2=0x7fb59019ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:31.356 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:31.356+0000 7fb58ea00700 1 --2- 172.21.15.153:0/277928445 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb590073a80 0x7fb59019ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.356 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:31.356+0000 7fb58ea00700 1 -- 172.21.15.153:0/277928445 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5800095f0 con 0x7fb590073160 2024-03-21T10:32:31.356 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:31.356+0000 7fb58ea00700 1 --2- 172.21.15.153:0/277928445 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb590073160 0x7fb59019e830 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fb5800092e0 tx=0x7fb58000be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:32:31.356 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:31.357+0000 7fb587400700 1 -- 172.21.15.153:0/277928445 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb58001d070 con 0x7fb590073160 2024-03-21T10:32:31.356 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:31.357+0000 7fb587400700 1 -- 172.21.15.153:0/277928445 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb58000f460 con 0x7fb590073160 2024-03-21T10:32:31.356 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:31.357+0000 7fb595800700 1 -- 172.21.15.153:0/277928445 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb590198b30 con 0x7fb590073160 2024-03-21T10:32:31.356 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:31.357+0000 7fb587400700 1 -- 172.21.15.153:0/277928445 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb580017630 con 0x7fb590073160 2024-03-21T10:32:31.356 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:31.357+0000 7fb595800700 1 -- 172.21.15.153:0/277928445 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb590199020 con 0x7fb590073160 2024-03-21T10:32:31.357 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:31.358+0000 7fb595800700 1 -- 172.21.15.153:0/277928445 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb59010a5c0 con 0x7fb590073160 2024-03-21T10:32:31.358 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:31.359+0000 7fb587400700 1 -- 172.21.15.153:0/277928445 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fb58000f5d0 con 0x7fb590073160 2024-03-21T10:32:31.359 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:31.360+0000 7fb587400700 1 --2- 172.21.15.153:0/277928445 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb57406e460 0x7fb574070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:31.359 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:31.360+0000 7fb587400700 1 -- 172.21.15.153:0/277928445 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fb58008fad0 con 0x7fb590073160 2024-03-21T10:32:31.359 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:31.360+0000 7fb587e00700 1 --2- 172.21.15.153:0/277928445 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb57406e460 0x7fb574070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:31.360 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:31.361+0000 7fb587e00700 1 --2- 172.21.15.153:0/277928445 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb57406e460 0x7fb574070920 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fb5900fee20 tx=0x7fb578006d20 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:32:31.364 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:31.365+0000 7fb587400700 1 -- 172.21.15.153:0/277928445 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb58005c6c0 con 0x7fb590073160 2024-03-21T10:32:31.554 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:32:31 smithi153 ceph-mon[113733]: pgmap v619: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:32:31.554 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:32:31 smithi153 ceph-mon[113733]: from='client.35122 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:32:31.554 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:32:31 smithi153 ceph-mon[113733]: from='client.35126 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:32:31.554 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:32:31 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/3764573262' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:32:31.554 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:32:31 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:32:31.640 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:31.641+0000 7fb595800700 1 -- 172.21.15.153:0/277928445 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fb59004ea90 con 0x7fb590073160 2024-03-21T10:32:31.641 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:31.642+0000 7fb587400700 1 -- 172.21.15.153:0/277928445 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1876 (secure 0 0 0) 0x7fb58005c250 con 0x7fb590073160 2024-03-21T10:32:31.642 INFO:teuthology.orchestra.run.smithi153.stdout:e30 2024-03-21T10:32:31.642 INFO:teuthology.orchestra.run.smithi153.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-21T10:32:31.642 INFO:teuthology.orchestra.run.smithi153.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:32:31.642 INFO:teuthology.orchestra.run.smithi153.stdout:legacy client fscid: 1 2024-03-21T10:32:31.642 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:32:31.642 INFO:teuthology.orchestra.run.smithi153.stdout:Filesystem 'cephfs' (1) 2024-03-21T10:32:31.642 INFO:teuthology.orchestra.run.smithi153.stdout:fs_name cephfs 2024-03-21T10:32:31.642 INFO:teuthology.orchestra.run.smithi153.stdout:epoch 23 2024-03-21T10:32:31.642 INFO:teuthology.orchestra.run.smithi153.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-21T10:32:31.642 INFO:teuthology.orchestra.run.smithi153.stdout:created 2024-03-21T10:04:27.086576+0000 2024-03-21T10:32:31.642 INFO:teuthology.orchestra.run.smithi153.stdout:modified 2024-03-21T10:19:05.022092+0000 2024-03-21T10:32:31.642 INFO:teuthology.orchestra.run.smithi153.stdout:tableserver 0 2024-03-21T10:32:31.642 INFO:teuthology.orchestra.run.smithi153.stdout:root 0 2024-03-21T10:32:31.643 INFO:teuthology.orchestra.run.smithi153.stdout:session_timeout 60 2024-03-21T10:32:31.643 INFO:teuthology.orchestra.run.smithi153.stdout:session_autoclose 300 2024-03-21T10:32:31.643 INFO:teuthology.orchestra.run.smithi153.stdout:max_file_size 1099511627776 2024-03-21T10:32:31.643 INFO:teuthology.orchestra.run.smithi153.stdout:max_xattr_size 65536 2024-03-21T10:32:31.643 INFO:teuthology.orchestra.run.smithi153.stdout:required_client_features {} 2024-03-21T10:32:31.643 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure 0 2024-03-21T10:32:31.643 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure_osd_epoch 94 2024-03-21T10:32:31.643 INFO:teuthology.orchestra.run.smithi153.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:32:31.643 INFO:teuthology.orchestra.run.smithi153.stdout:max_mds 2 2024-03-21T10:32:31.643 INFO:teuthology.orchestra.run.smithi153.stdout:in 0,1 2024-03-21T10:32:31.643 INFO:teuthology.orchestra.run.smithi153.stdout:up {} 2024-03-21T10:32:31.643 INFO:teuthology.orchestra.run.smithi153.stdout:failed 0,1 2024-03-21T10:32:31.643 INFO:teuthology.orchestra.run.smithi153.stdout:damaged 2024-03-21T10:32:31.643 INFO:teuthology.orchestra.run.smithi153.stdout:stopped 2024-03-21T10:32:31.643 INFO:teuthology.orchestra.run.smithi153.stdout:data_pools [3] 2024-03-21T10:32:31.644 INFO:teuthology.orchestra.run.smithi153.stdout:metadata_pool 2 2024-03-21T10:32:31.644 INFO:teuthology.orchestra.run.smithi153.stdout:inline_data enabled 2024-03-21T10:32:31.644 INFO:teuthology.orchestra.run.smithi153.stdout:balancer 2024-03-21T10:32:31.644 INFO:teuthology.orchestra.run.smithi153.stdout:bal_rank_mask -1 2024-03-21T10:32:31.644 INFO:teuthology.orchestra.run.smithi153.stdout:standby_count_wanted 1 2024-03-21T10:32:31.644 INFO:teuthology.orchestra.run.smithi153.stdout:qdb_cluster leader: 0 members: 2024-03-21T10:32:31.644 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:32:31.644 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:32:31.644 INFO:teuthology.orchestra.run.smithi153.stdout:Standby daemons: 2024-03-21T10:32:31.644 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:32:31.644 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.zipckr{-1:34442} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6826/1103846295,v1:172.21.15.153:6827/1103846295] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:32:31.644 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.iuyluj{-1:34474} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6828/1738630832,v1:172.21.15.153:6829/1738630832] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:32:31.644 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.nhozag{-1:44269} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.174:6826/3967223227,v1:172.21.15.174:6827/3967223227] compat {c=[1],r=[1],i=[7ff]}] 2024-03-21T10:32:31.644 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.yyimst{-1:44313} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.174:6824/4127786797,v1:172.21.15.174:6825/4127786797] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:32:31.646 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:31.646+0000 7fb595800700 1 -- 172.21.15.153:0/277928445 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb57406e460 msgr2=0x7fb574070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:31.646 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:31.646+0000 7fb595800700 1 --2- 172.21.15.153:0/277928445 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb57406e460 0x7fb574070920 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fb5900fee20 tx=0x7fb578006d20 comp rx=0 tx=0).stop 2024-03-21T10:32:31.646 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:31.646+0000 7fb595800700 1 -- 172.21.15.153:0/277928445 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb590073160 msgr2=0x7fb59019e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:31.646 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:31.646+0000 7fb595800700 1 --2- 172.21.15.153:0/277928445 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb590073160 0x7fb59019e830 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fb5800092e0 tx=0x7fb58000be00 comp rx=0 tx=0).stop 2024-03-21T10:32:31.646 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:31.646+0000 7fb595800700 1 -- 172.21.15.153:0/277928445 shutdown_connections 2024-03-21T10:32:31.646 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:31.646+0000 7fb595800700 1 --2- 172.21.15.153:0/277928445 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb57406e460 0x7fb574070920 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.646 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:31.646+0000 7fb595800700 1 --2- 172.21.15.153:0/277928445 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb590073160 0x7fb59019e830 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.646 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:31.646+0000 7fb595800700 1 --2- 172.21.15.153:0/277928445 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb590073a80 0x7fb59019ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.646 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:31.646+0000 7fb595800700 1 -- 172.21.15.153:0/277928445 >> 172.21.15.153:0/277928445 conn(0x7fb5900fc650 msgr2=0x7fb590107700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:32:31.646 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:31.647+0000 7fb595800700 1 -- 172.21.15.153:0/277928445 shutdown_connections 2024-03-21T10:32:31.646 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:31.647+0000 7fb595800700 1 -- 172.21.15.153:0/277928445 wait complete. 2024-03-21T10:32:31.647 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 30 2024-03-21T10:32:31.699 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:32:31 smithi174 ceph-mon[88004]: pgmap v619: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:32:31.699 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:32:31 smithi174 ceph-mon[88004]: from='client.35122 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:32:31.699 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:32:31 smithi174 ceph-mon[88004]: from='client.35126 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:32:31.699 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:32:31 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/3764573262' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:32:31.699 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:32:31 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:32:31.799 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:31.800+0000 7f06a3c00700 1 -- 172.21.15.153:0/3836733648 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f069c0731c0 msgr2=0x7f069c0735a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:31.799 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:31.800+0000 7f06a3c00700 1 --2- 172.21.15.153:0/3836733648 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f069c0731c0 0x7f069c0735a0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f0690009910 tx=0x7f0690009c20 comp rx=0 tx=0).stop 2024-03-21T10:32:31.799 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:31.800+0000 7f06a3c00700 1 -- 172.21.15.153:0/3836733648 shutdown_connections 2024-03-21T10:32:31.799 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:31.800+0000 7f06a3c00700 1 --2- 172.21.15.153:0/3836733648 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f069c073ae0 0x7f069c10ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.800 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:31.800+0000 7f06a3c00700 1 --2- 172.21.15.153:0/3836733648 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f069c0731c0 0x7f069c0735a0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.800 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:31.800+0000 7f06a3c00700 1 -- 172.21.15.153:0/3836733648 >> 172.21.15.153:0/3836733648 conn(0x7f069c0fc650 msgr2=0x7f069c0fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:32:31.800 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:31.800+0000 7f06a3c00700 1 -- 172.21.15.153:0/3836733648 shutdown_connections 2024-03-21T10:32:31.800 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:31.800+0000 7f06a3c00700 1 -- 172.21.15.153:0/3836733648 wait complete. 2024-03-21T10:32:31.801 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:31.801+0000 7f06a3c00700 1 Processor -- start 2024-03-21T10:32:31.801 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:31.801+0000 7f06a3c00700 1 -- start start 2024-03-21T10:32:31.801 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:31.802+0000 7f06a3c00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f069c0731c0 0x7f069c19e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:31.801 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:31.802+0000 7f06a3c00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f069c073ae0 0x7f069c19ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:31.801 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:31.802+0000 7f06a3c00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f069c19f2c0 con 0x7f069c073ae0 2024-03-21T10:32:31.801 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:31.802+0000 7f06a3c00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f069c1987e0 con 0x7f069c0731c0 2024-03-21T10:32:31.801 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:31.802+0000 7f069be00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f069c073ae0 0x7f069c19ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:31.801 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:31.802+0000 7f069be00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f069c073ae0 0x7f069c19ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:48024/0 (socket says 172.21.15.153:48024) 2024-03-21T10:32:31.801 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:31.802+0000 7f069be00700 1 -- 172.21.15.153:0/4213162479 learned_addr learned my addr 172.21.15.153:0/4213162479 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:32:31.801 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:31.802+0000 7f06a1200700 1 --2- 172.21.15.153:0/4213162479 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f069c0731c0 0x7f069c19e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:31.802 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:31.802+0000 7f069be00700 1 -- 172.21.15.153:0/4213162479 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f069c0731c0 msgr2=0x7f069c19e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:31.802 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:31.802+0000 7f069be00700 1 --2- 172.21.15.153:0/4213162479 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f069c0731c0 0x7f069c19e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:31.803 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:31.802+0000 7f069be00700 1 -- 172.21.15.153:0/4213162479 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06900095f0 con 0x7f069c073ae0 2024-03-21T10:32:31.803 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:31.802+0000 7f06a1200700 1 --2- 172.21.15.153:0/4213162479 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f069c0731c0 0x7f069c19e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:32:31.803 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:31.803+0000 7f069be00700 1 --2- 172.21.15.153:0/4213162479 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f069c073ae0 0x7f069c19ec30 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f068800c8a0 tx=0x7f068800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:32:31.803 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:31.803+0000 7f0699600700 1 -- 172.21.15.153:0/4213162479 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f068800cea0 con 0x7f069c073ae0 2024-03-21T10:32:31.803 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:31.803+0000 7f06a3c00700 1 -- 172.21.15.153:0/4213162479 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f069c198ac0 con 0x7f069c073ae0 2024-03-21T10:32:31.803 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:31.803+0000 7f0699600700 1 -- 172.21.15.153:0/4213162479 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0688004830 con 0x7f069c073ae0 2024-03-21T10:32:31.803 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:31.803+0000 7f0699600700 1 -- 172.21.15.153:0/4213162479 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f06880055b0 con 0x7f069c073ae0 2024-03-21T10:32:31.803 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:31.803+0000 7f06a3c00700 1 -- 172.21.15.153:0/4213162479 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f069c199010 con 0x7f069c073ae0 2024-03-21T10:32:31.803 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:31.804+0000 7f06a3c00700 1 -- 172.21.15.153:0/4213162479 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f069c10a560 con 0x7f069c073ae0 2024-03-21T10:32:31.804 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:31.805+0000 7f0699600700 1 -- 172.21.15.153:0/4213162479 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f06880049a0 con 0x7f069c073ae0 2024-03-21T10:32:31.805 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:31.806+0000 7f0699600700 1 --2- 172.21.15.153:0/4213162479 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f068406e460 0x7f0684070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:31.805 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:31.806+0000 7f0699600700 1 -- 172.21.15.153:0/4213162479 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f068808e190 con 0x7f069c073ae0 2024-03-21T10:32:31.805 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:31.806+0000 7f06a1200700 1 --2- 172.21.15.153:0/4213162479 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f068406e460 0x7f0684070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:31.806 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:31.807+0000 7f06a1200700 1 --2- 172.21.15.153:0/4213162479 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f068406e460 0x7f0684070920 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f0690006010 tx=0x7f069000b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:32:31.810 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:31.811+0000 7f0699600700 1 -- 172.21.15.153:0/4213162479 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f068805ad60 con 0x7f069c073ae0 2024-03-21T10:32:32.037 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:32.038+0000 7f06a3c00700 1 -- 172.21.15.153:0/4213162479 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f069c199340 con 0x7f068406e460 2024-03-21T10:32:32.041 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:32.041+0000 7f0699600700 1 -- 172.21.15.153:0/4213162479 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f069c199340 con 0x7f068406e460 2024-03-21T10:32:32.042 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:32:32.042 INFO:teuthology.orchestra.run.smithi153.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232", 2024-03-21T10:32:32.042 INFO:teuthology.orchestra.run.smithi153.stdout: "in_progress": true, 2024-03-21T10:32:32.042 INFO:teuthology.orchestra.run.smithi153.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-21T10:32:32.042 INFO:teuthology.orchestra.run.smithi153.stdout: "services_complete": [ 2024-03-21T10:32:32.042 INFO:teuthology.orchestra.run.smithi153.stdout: "crash", 2024-03-21T10:32:32.043 INFO:teuthology.orchestra.run.smithi153.stdout: "mon", 2024-03-21T10:32:32.043 INFO:teuthology.orchestra.run.smithi153.stdout: "osd", 2024-03-21T10:32:32.043 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr" 2024-03-21T10:32:32.043 INFO:teuthology.orchestra.run.smithi153.stdout: ], 2024-03-21T10:32:32.043 INFO:teuthology.orchestra.run.smithi153.stdout: "progress": "15/23 daemons upgraded", 2024-03-21T10:32:32.043 INFO:teuthology.orchestra.run.smithi153.stdout: "message": "Currently upgrading mds daemons", 2024-03-21T10:32:32.043 INFO:teuthology.orchestra.run.smithi153.stdout: "is_paused": false 2024-03-21T10:32:32.043 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:32:32.043 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:32.044+0000 7f06a3c00700 1 -- 172.21.15.153:0/4213162479 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f068406e460 msgr2=0x7f0684070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:32.044 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:32.044+0000 7f06a3c00700 1 --2- 172.21.15.153:0/4213162479 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f068406e460 0x7f0684070920 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f0690006010 tx=0x7f069000b540 comp rx=0 tx=0).stop 2024-03-21T10:32:32.044 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:32.044+0000 7f06a3c00700 1 -- 172.21.15.153:0/4213162479 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f069c073ae0 msgr2=0x7f069c19ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:32.044 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:32.044+0000 7f06a3c00700 1 --2- 172.21.15.153:0/4213162479 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f069c073ae0 0x7f069c19ec30 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f068800c8a0 tx=0x7f068800cc60 comp rx=0 tx=0).stop 2024-03-21T10:32:32.044 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:32.045+0000 7f06a3c00700 1 -- 172.21.15.153:0/4213162479 shutdown_connections 2024-03-21T10:32:32.044 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:32.045+0000 7f06a3c00700 1 --2- 172.21.15.153:0/4213162479 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f069c0731c0 0x7f069c19e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:32.044 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:32.045+0000 7f06a3c00700 1 --2- 172.21.15.153:0/4213162479 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f068406e460 0x7f0684070920 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:32.044 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:32.045+0000 7f06a3c00700 1 --2- 172.21.15.153:0/4213162479 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f069c073ae0 0x7f069c19ec30 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:32.045 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:32.045+0000 7f06a3c00700 1 -- 172.21.15.153:0/4213162479 >> 172.21.15.153:0/4213162479 conn(0x7f069c0fc650 msgr2=0x7f069c1076a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:32:32.045 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:32.045+0000 7f06a3c00700 1 -- 172.21.15.153:0/4213162479 shutdown_connections 2024-03-21T10:32:32.045 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:32.045+0000 7f06a3c00700 1 -- 172.21.15.153:0/4213162479 wait complete. 2024-03-21T10:32:32.194 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:32.195+0000 7f1788200700 1 -- 172.21.15.153:0/2496314556 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1780101770 msgr2=0x7f17801057c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:32.194 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:32.195+0000 7f1788200700 1 --2- 172.21.15.153:0/2496314556 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1780101770 0x7f17801057c0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f176c009910 tx=0x7f176c009c20 comp rx=0 tx=0).stop 2024-03-21T10:32:32.194 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:32.195+0000 7f1788200700 1 -- 172.21.15.153:0/2496314556 shutdown_connections 2024-03-21T10:32:32.194 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:32.195+0000 7f1788200700 1 --2- 172.21.15.153:0/2496314556 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1780101770 0x7f17801057c0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:32.195 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:32.195+0000 7f1788200700 1 --2- 172.21.15.153:0/2496314556 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1780100dc0 0x7f17801011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:32.195 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:32.195+0000 7f1788200700 1 -- 172.21.15.153:0/2496314556 >> 172.21.15.153:0/2496314556 conn(0x7f17800fc650 msgr2=0x7f17800fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:32:32.195 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:32.195+0000 7f1788200700 1 -- 172.21.15.153:0/2496314556 shutdown_connections 2024-03-21T10:32:32.195 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:32.195+0000 7f1788200700 1 -- 172.21.15.153:0/2496314556 wait complete. 2024-03-21T10:32:32.196 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:32.196+0000 7f1788200700 1 Processor -- start 2024-03-21T10:32:32.196 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:32.196+0000 7f1788200700 1 -- start start 2024-03-21T10:32:32.196 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:32.197+0000 7f1788200700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1780100dc0 0x7f178019c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:32.196 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:32.197+0000 7f1788200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1780101770 0x7f178019ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:32.196 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:32.197+0000 7f1788200700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f178019d110 con 0x7f1780101770 2024-03-21T10:32:32.196 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:32.197+0000 7f1788200700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17801965c0 con 0x7f1780100dc0 2024-03-21T10:32:32.197 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:32.197+0000 7f1784e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1780101770 0x7f178019ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:32.197 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:32.197+0000 7f1784e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1780101770 0x7f178019ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:48042/0 (socket says 172.21.15.153:48042) 2024-03-21T10:32:32.197 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:32.197+0000 7f1784e00700 1 -- 172.21.15.153:0/3930921785 learned_addr learned my addr 172.21.15.153:0/3930921785 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:32:32.197 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:32.197+0000 7f1785800700 1 --2- 172.21.15.153:0/3930921785 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1780100dc0 0x7f178019c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:32.197 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:32.197+0000 7f1784e00700 1 -- 172.21.15.153:0/3930921785 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1780100dc0 msgr2=0x7f178019c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:32.197 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:32.197+0000 7f1784e00700 1 --2- 172.21.15.153:0/3930921785 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1780100dc0 0x7f178019c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:32.197 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:32.197+0000 7f1784e00700 1 -- 172.21.15.153:0/3930921785 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f176c0095f0 con 0x7f1780101770 2024-03-21T10:32:32.198 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:32.198+0000 7f1784e00700 1 --2- 172.21.15.153:0/3930921785 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1780101770 0x7f178019ca80 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f176c0092e0 tx=0x7f176c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:32:32.198 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:32.198+0000 7f177e000700 1 -- 172.21.15.153:0/3930921785 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f176c01d070 con 0x7f1780101770 2024-03-21T10:32:32.198 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:32.198+0000 7f177e000700 1 -- 172.21.15.153:0/3930921785 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f176c005dc0 con 0x7f1780101770 2024-03-21T10:32:32.198 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:32.198+0000 7f1788200700 1 -- 172.21.15.153:0/3930921785 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1780196840 con 0x7f1780101770 2024-03-21T10:32:32.198 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:32.198+0000 7f177e000700 1 -- 172.21.15.153:0/3930921785 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f176c017400 con 0x7f1780101770 2024-03-21T10:32:32.198 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:32.198+0000 7f1788200700 1 -- 172.21.15.153:0/3930921785 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1780196d30 con 0x7f1780101770 2024-03-21T10:32:32.198 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:32.199+0000 7f1788200700 1 -- 172.21.15.153:0/3930921785 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17801091c0 con 0x7f1780101770 2024-03-21T10:32:32.199 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:32.200+0000 7f177e000700 1 -- 172.21.15.153:0/3930921785 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f176c00f460 con 0x7f1780101770 2024-03-21T10:32:32.200 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:32.201+0000 7f177e000700 1 --2- 172.21.15.153:0/3930921785 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f176806e460 0x7f1768070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:32:32.200 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:32.201+0000 7f177e000700 1 -- 172.21.15.153:0/3930921785 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f176c08f990 con 0x7f1780101770 2024-03-21T10:32:32.200 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:32.201+0000 7f1785800700 1 --2- 172.21.15.153:0/3930921785 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f176806e460 0x7f1768070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:32:32.201 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:32.202+0000 7f1785800700 1 --2- 172.21.15.153:0/3930921785 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f176806e460 0x7f1768070920 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f1774005950 tx=0x7f17740058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:32:32.205 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:32.206+0000 7f177e000700 1 -- 172.21.15.153:0/3930921785 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f176c05c560 con 0x7f1780101770 2024-03-21T10:32:32.304 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:32:32 smithi153 ceph-mon[113733]: from='client.35130 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:32:32.304 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:32:32 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/277928445' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:32:32.449 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:32:32 smithi174 ceph-mon[88004]: from='client.35130 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:32:32.450 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:32:32 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/277928445' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:32:32.500 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:32.501+0000 7f1788200700 1 -- 172.21.15.153:0/3930921785 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f178004ea90 con 0x7f1780101770 2024-03-21T10:32:32.501 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:32.502+0000 7f177e000700 1 -- 172.21.15.153:0/3930921785 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f176c05c0f0 con 0x7f1780101770 2024-03-21T10:32:32.501 INFO:teuthology.orchestra.run.smithi153.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-03-21T10:32:32.501 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-21T10:32:32.501 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is degraded 2024-03-21T10:32:32.501 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-21T10:32:32.501 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-21T10:32:32.501 INFO:teuthology.orchestra.run.smithi153.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-21T10:32:32.501 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-21T10:32:32.504 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:32.505+0000 7f1788200700 1 -- 172.21.15.153:0/3930921785 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f176806e460 msgr2=0x7f1768070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:32.504 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:32.505+0000 7f1788200700 1 --2- 172.21.15.153:0/3930921785 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f176806e460 0x7f1768070920 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f1774005950 tx=0x7f17740058e0 comp rx=0 tx=0).stop 2024-03-21T10:32:32.504 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:32.505+0000 7f1788200700 1 -- 172.21.15.153:0/3930921785 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1780101770 msgr2=0x7f178019ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:32:32.504 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:32.505+0000 7f1788200700 1 --2- 172.21.15.153:0/3930921785 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1780101770 0x7f178019ca80 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f176c0092e0 tx=0x7f176c00be20 comp rx=0 tx=0).stop 2024-03-21T10:32:32.504 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:32.505+0000 7f1788200700 1 -- 172.21.15.153:0/3930921785 shutdown_connections 2024-03-21T10:32:32.504 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:32.505+0000 7f1788200700 1 --2- 172.21.15.153:0/3930921785 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1780100dc0 0x7f178019c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:32.504 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:32.505+0000 7f1788200700 1 --2- 172.21.15.153:0/3930921785 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f176806e460 0x7f1768070920 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:32.504 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:32.505+0000 7f1788200700 1 --2- 172.21.15.153:0/3930921785 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1780101770 0x7f178019ca80 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:32:32.504 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:32.505+0000 7f1788200700 1 -- 172.21.15.153:0/3930921785 >> 172.21.15.153:0/3930921785 conn(0x7f17800fc650 msgr2=0x7f17800fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:32:32.505 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:32.505+0000 7f1788200700 1 -- 172.21.15.153:0/3930921785 shutdown_connections 2024-03-21T10:32:32.505 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:32:32.505+0000 7f1788200700 1 -- 172.21.15.153:0/3930921785 wait complete. 2024-03-21T10:32:33.554 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:32:33 smithi153 ceph-mon[113733]: pgmap v620: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:32:33.554 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:32:33 smithi153 ceph-mon[113733]: from='client.35142 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:32:33.554 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:32:33 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/3930921785' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:32:33.699 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:32:33 smithi174 ceph-mon[88004]: pgmap v620: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:32:33.699 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:32:33 smithi174 ceph-mon[88004]: from='client.35142 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:32:33.699 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:32:33 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/3930921785' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:32:35.554 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:32:35 smithi153 ceph-mon[113733]: pgmap v621: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:32:35.699 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:32:35 smithi174 ceph-mon[88004]: pgmap v621: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:32:37.554 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:32:37 smithi153 ceph-mon[113733]: pgmap v622: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:32:37.699 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:32:37 smithi174 ceph-mon[88004]: pgmap v622: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:32:39.554 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:32:39 smithi153 ceph-mon[113733]: pgmap v623: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:32:39.699 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:32:39 smithi174 ceph-mon[88004]: pgmap v623: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:32:41.554 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:32:41 smithi153 ceph-mon[113733]: pgmap v624: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:32:41.699 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:32:41 smithi174 ceph-mon[88004]: pgmap v624: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:32:43.699 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:32:43 smithi174 ceph-mon[88004]: pgmap v625: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:32:43.804 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:32:43 smithi153 ceph-mon[113733]: pgmap v625: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:32:45.699 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:32:45 smithi174 ceph-mon[88004]: pgmap v626: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:32:45.804 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:32:45 smithi153 ceph-mon[113733]: pgmap v626: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:32:46.699 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:32:46 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:32:46.803 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:32:46 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:32:47.677 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:32:47 smithi174 ceph-mon[88004]: pgmap v627: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:32:47.804 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:32:47 smithi153 ceph-mon[113733]: pgmap v627: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:32:49.699 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:32:49 smithi174 ceph-mon[88004]: pgmap v628: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:32:49.804 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:32:49 smithi153 ceph-mon[113733]: pgmap v628: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:32:51.699 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:32:51 smithi174 ceph-mon[88004]: pgmap v629: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:32:51.803 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:32:51 smithi153 ceph-mon[113733]: pgmap v629: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:32:53.699 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:32:53 smithi174 ceph-mon[88004]: pgmap v630: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:32:53.804 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:32:53 smithi153 ceph-mon[113733]: pgmap v630: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:32:55.699 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:32:55 smithi174 ceph-mon[88004]: pgmap v631: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:32:55.804 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:32:55 smithi153 ceph-mon[113733]: pgmap v631: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:32:57.699 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:32:57 smithi174 ceph-mon[88004]: pgmap v632: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:32:57.804 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:32:57 smithi153 ceph-mon[113733]: pgmap v632: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:32:59.699 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:32:59 smithi174 ceph-mon[88004]: pgmap v633: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:32:59.848 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:32:59 smithi153 ceph-mon[113733]: pgmap v633: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:33:01.699 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:33:01 smithi174 ceph-mon[88004]: pgmap v634: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:33:01.699 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:33:01 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:33:01.804 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:33:01 smithi153 ceph-mon[113733]: pgmap v634: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:33:01.804 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:33:01 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:33:02.673 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:02.674+0000 7fa7e9600700 1 -- 172.21.15.153:0/3229294667 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa7e4102e10 msgr2=0x7fa7e41031f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:02.673 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:02.674+0000 7fa7e9600700 1 --2- 172.21.15.153:0/3229294667 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa7e4102e10 0x7fa7e41031f0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fa7d4009910 tx=0x7fa7d4009c20 comp rx=0 tx=0).stop 2024-03-21T10:33:02.673 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:02.674+0000 7fa7e9600700 1 -- 172.21.15.153:0/3229294667 shutdown_connections 2024-03-21T10:33:02.673 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:02.674+0000 7fa7e9600700 1 --2- 172.21.15.153:0/3229294667 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa7e41037c0 0x7fa7e4107810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:02.673 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:02.674+0000 7fa7e9600700 1 --2- 172.21.15.153:0/3229294667 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa7e4102e10 0x7fa7e41031f0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:02.673 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:02.674+0000 7fa7e9600700 1 -- 172.21.15.153:0/3229294667 >> 172.21.15.153:0/3229294667 conn(0x7fa7e40fe6c0 msgr2=0x7fa7e4100ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:02.673 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:02.674+0000 7fa7e9600700 1 -- 172.21.15.153:0/3229294667 shutdown_connections 2024-03-21T10:33:02.673 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:02.674+0000 7fa7e9600700 1 -- 172.21.15.153:0/3229294667 wait complete. 2024-03-21T10:33:02.675 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:02.675+0000 7fa7e9600700 1 Processor -- start 2024-03-21T10:33:02.675 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:02.676+0000 7fa7e9600700 1 -- start start 2024-03-21T10:33:02.675 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:02.676+0000 7fa7e9600700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa7e4102e10 0x7fa7e4198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:02.675 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:02.676+0000 7fa7e9600700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa7e41037c0 0x7fa7e4198ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:02.675 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:02.676+0000 7fa7e9600700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e4199550 con 0x7fa7e41037c0 2024-03-21T10:33:02.675 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:02.676+0000 7fa7e9600700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e419d360 con 0x7fa7e4102e10 2024-03-21T10:33:02.675 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:02.676+0000 7fa7e3400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa7e41037c0 0x7fa7e4198ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:02.675 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:02.676+0000 7fa7e3400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa7e41037c0 0x7fa7e4198ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:33908/0 (socket says 172.21.15.153:33908) 2024-03-21T10:33:02.675 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:02.676+0000 7fa7e3400700 1 -- 172.21.15.153:0/1643220801 learned_addr learned my addr 172.21.15.153:0/1643220801 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:33:02.675 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:02.677+0000 7fa7e3e00700 1 --2- 172.21.15.153:0/1643220801 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa7e4102e10 0x7fa7e4198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:02.676 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:02.677+0000 7fa7e3400700 1 -- 172.21.15.153:0/1643220801 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa7e4102e10 msgr2=0x7fa7e4198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:02.676 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:02.677+0000 7fa7e3400700 1 --2- 172.21.15.153:0/1643220801 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa7e4102e10 0x7fa7e4198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:02.676 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:02.677+0000 7fa7e3400700 1 -- 172.21.15.153:0/1643220801 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7d40095f0 con 0x7fa7e41037c0 2024-03-21T10:33:02.676 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:02.677+0000 7fa7e3400700 1 --2- 172.21.15.153:0/1643220801 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa7e41037c0 0x7fa7e4198ec0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fa7d000b480 tx=0x7fa7d000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:02.676 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:02.677+0000 7fa7e0c00700 1 -- 172.21.15.153:0/1643220801 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa7d00118c0 con 0x7fa7e41037c0 2024-03-21T10:33:02.677 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:02.677+0000 7fa7e0c00700 1 -- 172.21.15.153:0/1643220801 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa7d0010380 con 0x7fa7e41037c0 2024-03-21T10:33:02.677 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:02.677+0000 7fa7e0c00700 1 -- 172.21.15.153:0/1643220801 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa7d000f4f0 con 0x7fa7e41037c0 2024-03-21T10:33:02.677 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:02.678+0000 7fa7e9600700 1 -- 172.21.15.153:0/1643220801 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7e419d640 con 0x7fa7e41037c0 2024-03-21T10:33:02.677 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:02.678+0000 7fa7e9600700 1 -- 172.21.15.153:0/1643220801 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7e419db90 con 0x7fa7e41037c0 2024-03-21T10:33:02.677 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:02.679+0000 7fa7e9600700 1 -- 172.21.15.153:0/1643220801 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7e410b1b0 con 0x7fa7e41037c0 2024-03-21T10:33:02.677 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:02.680+0000 7fa7e0c00700 1 -- 172.21.15.153:0/1643220801 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fa7d00104f0 con 0x7fa7e41037c0 2024-03-21T10:33:02.679 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:02.681+0000 7fa7e0c00700 1 --2- 172.21.15.153:0/1643220801 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa7c806e460 0x7fa7c8070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:02.679 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:02.681+0000 7fa7e0c00700 1 -- 172.21.15.153:0/1643220801 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fa7d008e120 con 0x7fa7e41037c0 2024-03-21T10:33:02.679 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:02.681+0000 7fa7e3e00700 1 --2- 172.21.15.153:0/1643220801 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa7c806e460 0x7fa7c8070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:02.679 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:02.682+0000 7fa7e3e00700 1 --2- 172.21.15.153:0/1643220801 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa7c806e460 0x7fa7c8070920 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa7d4006010 tx=0x7fa7d401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:02.684 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:02.686+0000 7fa7e0c00700 1 -- 172.21.15.153:0/1643220801 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa7d005acf0 con 0x7fa7e41037c0 2024-03-21T10:33:02.929 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:02.930+0000 7fa7e9600700 1 -- 172.21.15.153:0/1643220801 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa7e4199df0 con 0x7fa7c806e460 2024-03-21T10:33:02.931 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:02.933+0000 7fa7e0c00700 1 -- 172.21.15.153:0/1643220801 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fa7e4199df0 con 0x7fa7c806e460 2024-03-21T10:33:02.934 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:02.936+0000 7fa7e9600700 1 -- 172.21.15.153:0/1643220801 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa7c806e460 msgr2=0x7fa7c8070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:02.934 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:02.936+0000 7fa7e9600700 1 --2- 172.21.15.153:0/1643220801 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa7c806e460 0x7fa7c8070920 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa7d4006010 tx=0x7fa7d401a040 comp rx=0 tx=0).stop 2024-03-21T10:33:02.934 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:02.936+0000 7fa7e9600700 1 -- 172.21.15.153:0/1643220801 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa7e41037c0 msgr2=0x7fa7e4198ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:02.934 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:02.936+0000 7fa7e9600700 1 --2- 172.21.15.153:0/1643220801 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa7e41037c0 0x7fa7e4198ec0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fa7d000b480 tx=0x7fa7d000b840 comp rx=0 tx=0).stop 2024-03-21T10:33:02.934 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:02.936+0000 7fa7e9600700 1 -- 172.21.15.153:0/1643220801 shutdown_connections 2024-03-21T10:33:02.934 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:02.936+0000 7fa7e9600700 1 --2- 172.21.15.153:0/1643220801 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa7e4102e10 0x7fa7e4198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:02.934 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:02.936+0000 7fa7e9600700 1 --2- 172.21.15.153:0/1643220801 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa7c806e460 0x7fa7c8070920 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:02.935 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:02.936+0000 7fa7e9600700 1 --2- 172.21.15.153:0/1643220801 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa7e41037c0 0x7fa7e4198ec0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:02.935 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:02.936+0000 7fa7e9600700 1 -- 172.21.15.153:0/1643220801 >> 172.21.15.153:0/1643220801 conn(0x7fa7e40fe6c0 msgr2=0x7fa7e40ffda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:02.935 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:02.936+0000 7fa7e9600700 1 -- 172.21.15.153:0/1643220801 shutdown_connections 2024-03-21T10:33:02.935 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:02.936+0000 7fa7e9600700 1 -- 172.21.15.153:0/1643220801 wait complete. 2024-03-21T10:33:02.949 INFO:teuthology.orchestra.run.smithi153.stdout:true 2024-03-21T10:33:03.078 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.080+0000 7f79a0000700 1 -- 172.21.15.153:0/151529257 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f79981071d0 msgr2=0x7f7998107650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:03.078 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.080+0000 7f79a0000700 1 --2- 172.21.15.153:0/151529257 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f79981071d0 0x7f7998107650 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f7984009910 tx=0x7f7984009c20 comp rx=0 tx=0).stop 2024-03-21T10:33:03.078 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.080+0000 7f79a0000700 1 -- 172.21.15.153:0/151529257 shutdown_connections 2024-03-21T10:33:03.078 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.080+0000 7f79a0000700 1 --2- 172.21.15.153:0/151529257 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f79981071d0 0x7f7998107650 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:03.078 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.080+0000 7f79a0000700 1 --2- 172.21.15.153:0/151529257 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f799810d280 0x7f799810d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:03.078 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.080+0000 7f79a0000700 1 -- 172.21.15.153:0/151529257 >> 172.21.15.153:0/151529257 conn(0x7f79980769e0 msgr2=0x7f7998078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:03.079 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.080+0000 7f79a0000700 1 -- 172.21.15.153:0/151529257 shutdown_connections 2024-03-21T10:33:03.079 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.080+0000 7f79a0000700 1 -- 172.21.15.153:0/151529257 wait complete. 2024-03-21T10:33:03.080 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.081+0000 7f79a0000700 1 Processor -- start 2024-03-21T10:33:03.080 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.081+0000 7f79a0000700 1 -- start start 2024-03-21T10:33:03.080 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.081+0000 7f79a0000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f79981071d0 0x7f799819ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:03.080 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.081+0000 7f79a0000700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f799810d280 0x7f799819d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:03.080 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.081+0000 7f79a0000700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f799819d880 con 0x7f79981071d0 2024-03-21T10:33:03.080 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.081+0000 7f79a0000700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79981a05b0 con 0x7f799810d280 2024-03-21T10:33:03.080 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.082+0000 7f799d600700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f79981071d0 0x7f799819ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:03.080 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.082+0000 7f799d600700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f79981071d0 0x7f799819ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:45710/0 (socket says 172.21.15.153:45710) 2024-03-21T10:33:03.080 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.082+0000 7f799d600700 1 -- 172.21.15.153:0/648225834 learned_addr learned my addr 172.21.15.153:0/648225834 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:33:03.080 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.082+0000 7f799d600700 1 -- 172.21.15.153:0/648225834 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f799810d280 msgr2=0x7f799819d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:03.081 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.082+0000 7f799cc00700 1 --2- 172.21.15.153:0/648225834 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f799810d280 0x7f799819d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:03.081 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.082+0000 7f799d600700 1 --2- 172.21.15.153:0/648225834 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f799810d280 0x7f799819d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:03.081 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.082+0000 7f799d600700 1 -- 172.21.15.153:0/648225834 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79840095f0 con 0x7f79981071d0 2024-03-21T10:33:03.081 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.082+0000 7f799d600700 1 --2- 172.21.15.153:0/648225834 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f79981071d0 0x7f799819ccb0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f798c00c8a0 tx=0x7f798c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:03.081 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.083+0000 7f7996000700 1 -- 172.21.15.153:0/648225834 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f798c00cea0 con 0x7f79981071d0 2024-03-21T10:33:03.081 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.083+0000 7f7996000700 1 -- 172.21.15.153:0/648225834 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f798c004830 con 0x7f79981071d0 2024-03-21T10:33:03.081 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.083+0000 7f79a0000700 1 -- 172.21.15.153:0/648225834 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79981a0890 con 0x7f79981071d0 2024-03-21T10:33:03.082 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.083+0000 7f7996000700 1 -- 172.21.15.153:0/648225834 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f798c0055b0 con 0x7f79981071d0 2024-03-21T10:33:03.082 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.083+0000 7f79a0000700 1 -- 172.21.15.153:0/648225834 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79981a0e60 con 0x7f79981071d0 2024-03-21T10:33:03.082 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.084+0000 7f79a0000700 1 -- 172.21.15.153:0/648225834 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f799810f6c0 con 0x7f79981071d0 2024-03-21T10:33:03.083 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.085+0000 7f7996000700 1 -- 172.21.15.153:0/648225834 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f798c0049a0 con 0x7f79981071d0 2024-03-21T10:33:03.084 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.086+0000 7f7996000700 1 --2- 172.21.15.153:0/648225834 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f798006e460 0x7f7980070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:03.084 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.086+0000 7f7996000700 1 -- 172.21.15.153:0/648225834 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f798c08e0b0 con 0x7f79981071d0 2024-03-21T10:33:03.084 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.086+0000 7f799cc00700 1 --2- 172.21.15.153:0/648225834 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f798006e460 0x7f7980070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:03.085 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.087+0000 7f799cc00700 1 --2- 172.21.15.153:0/648225834 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f798006e460 0x7f7980070920 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f7984000c00 tx=0x7f7984005fd0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:03.088 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.091+0000 7f7996000700 1 -- 172.21.15.153:0/648225834 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f798c0571f0 con 0x7f79981071d0 2024-03-21T10:33:03.314 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.315+0000 7f79a0000700 1 -- 172.21.15.153:0/648225834 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f79981a1130 con 0x7f798006e460 2024-03-21T10:33:03.316 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.318+0000 7f7996000700 1 -- 172.21.15.153:0/648225834 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f79981a1130 con 0x7f798006e460 2024-03-21T10:33:03.318 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.320+0000 7f79a0000700 1 -- 172.21.15.153:0/648225834 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f798006e460 msgr2=0x7f7980070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:03.318 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.320+0000 7f79a0000700 1 --2- 172.21.15.153:0/648225834 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f798006e460 0x7f7980070920 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f7984000c00 tx=0x7f7984005fd0 comp rx=0 tx=0).stop 2024-03-21T10:33:03.318 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.320+0000 7f79a0000700 1 -- 172.21.15.153:0/648225834 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f79981071d0 msgr2=0x7f799819ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:03.319 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.320+0000 7f79a0000700 1 --2- 172.21.15.153:0/648225834 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f79981071d0 0x7f799819ccb0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f798c00c8a0 tx=0x7f798c00cc60 comp rx=0 tx=0).stop 2024-03-21T10:33:03.319 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.321+0000 7f79a0000700 1 -- 172.21.15.153:0/648225834 shutdown_connections 2024-03-21T10:33:03.319 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.321+0000 7f79a0000700 1 --2- 172.21.15.153:0/648225834 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f798006e460 0x7f7980070920 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:03.319 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.321+0000 7f79a0000700 1 --2- 172.21.15.153:0/648225834 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f79981071d0 0x7f799819ccb0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:03.319 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.321+0000 7f79a0000700 1 --2- 172.21.15.153:0/648225834 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f799810d280 0x7f799819d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:03.319 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.321+0000 7f79a0000700 1 -- 172.21.15.153:0/648225834 >> 172.21.15.153:0/648225834 conn(0x7f79980769e0 msgr2=0x7f7998078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:03.319 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.321+0000 7f79a0000700 1 -- 172.21.15.153:0/648225834 shutdown_connections 2024-03-21T10:33:03.320 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.321+0000 7f79a0000700 1 -- 172.21.15.153:0/648225834 wait complete. 2024-03-21T10:33:03.448 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:33:03 smithi174 ceph-mon[88004]: pgmap v635: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:33:03.473 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.475+0000 7f49b2e00700 1 -- 172.21.15.153:0/4175832378 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49ac073160 msgr2=0x7f49ac073540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:03.473 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.475+0000 7f49b2e00700 1 --2- 172.21.15.153:0/4175832378 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49ac073160 0x7f49ac073540 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f499c009910 tx=0x7f499c009c20 comp rx=0 tx=0).stop 2024-03-21T10:33:03.473 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.475+0000 7f49b2e00700 1 -- 172.21.15.153:0/4175832378 shutdown_connections 2024-03-21T10:33:03.473 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.475+0000 7f49b2e00700 1 --2- 172.21.15.153:0/4175832378 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f49ac073a80 0x7f49ac10cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:03.474 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.475+0000 7f49b2e00700 1 --2- 172.21.15.153:0/4175832378 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49ac073160 0x7f49ac073540 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:03.474 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.475+0000 7f49b2e00700 1 -- 172.21.15.153:0/4175832378 >> 172.21.15.153:0/4175832378 conn(0x7f49ac0fc630 msgr2=0x7f49ac0fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:03.474 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.475+0000 7f49b2e00700 1 -- 172.21.15.153:0/4175832378 shutdown_connections 2024-03-21T10:33:03.474 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.475+0000 7f49b2e00700 1 -- 172.21.15.153:0/4175832378 wait complete. 2024-03-21T10:33:03.474 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.476+0000 7f49b2e00700 1 Processor -- start 2024-03-21T10:33:03.474 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.477+0000 7f49b2e00700 1 -- start start 2024-03-21T10:33:03.475 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.477+0000 7f49b2e00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f49ac073160 0x7f49ac198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:03.475 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.477+0000 7f49b2e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49ac073a80 0x7f49ac198ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:03.476 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.477+0000 7f49b2e00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49ac199550 con 0x7f49ac073a80 2024-03-21T10:33:03.476 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.477+0000 7f49b2e00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49ac19d360 con 0x7f49ac073160 2024-03-21T10:33:03.476 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.477+0000 7f49ab400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49ac073a80 0x7f49ac198ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:03.476 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.477+0000 7f49ab400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49ac073a80 0x7f49ac198ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:45726/0 (socket says 172.21.15.153:45726) 2024-03-21T10:33:03.476 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.477+0000 7f49ab400700 1 -- 172.21.15.153:0/261787980 learned_addr learned my addr 172.21.15.153:0/261787980 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:33:03.476 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.478+0000 7f49abe00700 1 --2- 172.21.15.153:0/261787980 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f49ac073160 0x7f49ac198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:03.476 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.478+0000 7f49ab400700 1 -- 172.21.15.153:0/261787980 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f49ac073160 msgr2=0x7f49ac198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:03.476 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.478+0000 7f49ab400700 1 --2- 172.21.15.153:0/261787980 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f49ac073160 0x7f49ac198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:03.476 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.478+0000 7f49ab400700 1 -- 172.21.15.153:0/261787980 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f499c0095f0 con 0x7f49ac073a80 2024-03-21T10:33:03.477 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.478+0000 7f49ab400700 1 --2- 172.21.15.153:0/261787980 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49ac073a80 0x7f49ac198ec0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f499800d8d0 tx=0x7f499800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:03.477 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.478+0000 7f49a8c00700 1 -- 172.21.15.153:0/261787980 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4998009940 con 0x7f49ac073a80 2024-03-21T10:33:03.477 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.478+0000 7f49a8c00700 1 -- 172.21.15.153:0/261787980 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4998010460 con 0x7f49ac073a80 2024-03-21T10:33:03.477 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.478+0000 7f49b2e00700 1 -- 172.21.15.153:0/261787980 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49ac19d640 con 0x7f49ac073a80 2024-03-21T10:33:03.477 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.479+0000 7f49a8c00700 1 -- 172.21.15.153:0/261787980 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f499800f5f0 con 0x7f49ac073a80 2024-03-21T10:33:03.477 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.479+0000 7f49b2e00700 1 -- 172.21.15.153:0/261787980 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49ac19db90 con 0x7f49ac073a80 2024-03-21T10:33:03.478 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.480+0000 7f49b2e00700 1 -- 172.21.15.153:0/261787980 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49ac10a5a0 con 0x7f49ac073a80 2024-03-21T10:33:03.478 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.481+0000 7f49a8c00700 1 -- 172.21.15.153:0/261787980 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f49980105d0 con 0x7f49ac073a80 2024-03-21T10:33:03.480 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.482+0000 7f49a8c00700 1 --2- 172.21.15.153:0/261787980 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f499006e460 0x7f4990070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:03.480 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.482+0000 7f49a8c00700 1 -- 172.21.15.153:0/261787980 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f499808e480 con 0x7f49ac073a80 2024-03-21T10:33:03.480 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.482+0000 7f49abe00700 1 --2- 172.21.15.153:0/261787980 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f499006e460 0x7f4990070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:03.480 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.483+0000 7f49abe00700 1 --2- 172.21.15.153:0/261787980 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f499006e460 0x7f4990070920 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f499c00b5c0 tx=0x7f499c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:03.484 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.486+0000 7f49a8c00700 1 -- 172.21.15.153:0/261787980 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f499805b050 con 0x7f49ac073a80 2024-03-21T10:33:03.704 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.706+0000 7f49b2e00700 1 -- 172.21.15.153:0/261787980 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f49ac19de00 con 0x7f499006e460 2024-03-21T10:33:03.704 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:33:03 smithi153 ceph-mon[113733]: pgmap v635: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:33:03.736 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.738+0000 7f49a8c00700 1 -- 172.21.15.153:0/261787980 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f49ac19de00 con 0x7f499006e460 2024-03-21T10:33:03.737 INFO:teuthology.orchestra.run.smithi153.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-21T10:33:03.737 INFO:teuthology.orchestra.run.smithi153.stdout:alertmanager.smithi153 smithi153 *:9093,9094 running (21m) 13m ago 35m 14.7M - 0.25.0 c8568f914cd2 9ed3f2200b5f 2024-03-21T10:33:03.737 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi153 smithi153 running (35m) 13m ago 35m 9424k - 18.2.1 5be31c24972a 5b61994396a6 2024-03-21T10:33:03.737 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi174 smithi174 running (34m) 12m ago 34m 9244k - 18.2.1 5be31c24972a 4f7a5fc3ac28 2024-03-21T10:33:03.738 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi153 smithi153 running (18m) 13m ago 35m 6656k - 19.0.0-2231-g2edfee68 237e43c7a65f 32c21104d5b0 2024-03-21T10:33:03.738 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi174 smithi174 running (17m) 12m ago 34m 6660k - 19.0.0-2231-g2edfee68 237e43c7a65f a04e48967ab6 2024-03-21T10:33:03.738 INFO:teuthology.orchestra.run.smithi153.stdout:grafana.smithi153 smithi153 *:3000 running (21m) 13m ago 35m 73.6M - 9.4.12 2bacad6d85d8 ee24026ab786 2024-03-21T10:33:03.738 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.iuyluj smithi153 running (13m) 13m ago 28m 10.9M - 19.0.0-2231-g2edfee68 237e43c7a65f 87aff8a5de70 2024-03-21T10:33:03.738 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.zipckr smithi153 running (13m) 13m ago 28m 11.8M - 19.0.0-2231-g2edfee68 237e43c7a65f ccc262bd5dd3 2024-03-21T10:33:03.738 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.nhozag smithi174 running (28m) 12m ago 28m 12.0M - 18.2.1 5be31c24972a 2fee4000308f 2024-03-21T10:33:03.738 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.yyimst smithi174 running (13m) 12m ago 28m 10.7M - 19.0.0-2231-g2edfee68 237e43c7a65f fc1962f9351e 2024-03-21T10:33:03.738 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi153.tluxni smithi153 *:8443,9283,8765 running (24m) 13m ago 37m 509M - 19.0.0-2231-g2edfee68 237e43c7a65f 7ffee0e55382 2024-03-21T10:33:03.738 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi174.qpcwat smithi174 *:8443,9283,8765 running (23m) 12m ago 34m 452M - 19.0.0-2231-g2edfee68 237e43c7a65f 748968f216ab 2024-03-21T10:33:03.738 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi153 smithi153 running (19m) 13m ago 37m 55.0M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f dd8038303dc3 2024-03-21T10:33:03.739 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi174 smithi174 running (19m) 12m ago 33m 44.2M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f 29d78a788f2c 2024-03-21T10:33:03.739 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi153 smithi153 *:9100 running (23m) 13m ago 35m 7889k - 1.5.0 0da6a335fe13 bb835edc0e0b 2024-03-21T10:33:03.739 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi174 smithi174 *:9100 running (22m) 12m ago 34m 8102k - 1.5.0 0da6a335fe13 7c4ecad790fa 2024-03-21T10:33:03.739 INFO:teuthology.orchestra.run.smithi153.stdout:osd.0 smithi153 running (17m) 13m ago 33m 116M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 507900fd4791 2024-03-21T10:33:03.739 INFO:teuthology.orchestra.run.smithi153.stdout:osd.1 smithi153 running (16m) 13m ago 32m 89.2M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 0d646452e9c1 2024-03-21T10:33:03.739 INFO:teuthology.orchestra.run.smithi153.stdout:osd.2 smithi153 running (16m) 13m ago 31m 87.4M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 1326f2afdfcd 2024-03-21T10:33:03.739 INFO:teuthology.orchestra.run.smithi153.stdout:osd.3 smithi174 running (15m) 12m ago 31m 83.6M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 70aafac71ad5 2024-03-21T10:33:03.739 INFO:teuthology.orchestra.run.smithi153.stdout:osd.4 smithi174 running (14m) 12m ago 30m 92.2M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 0d746ce6e79d 2024-03-21T10:33:03.739 INFO:teuthology.orchestra.run.smithi153.stdout:osd.5 smithi174 running (14m) 12m ago 30m 86.2M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 53e5f35b8cac 2024-03-21T10:33:03.739 INFO:teuthology.orchestra.run.smithi153.stdout:prometheus.smithi153 smithi153 *:9095 running (22m) 13m ago 34m 73.5M - 2.43.0 a07b618ecd1d 7d95561e50c2 2024-03-21T10:33:03.740 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.741+0000 7f49b2e00700 1 -- 172.21.15.153:0/261787980 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f499006e460 msgr2=0x7f4990070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:03.740 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.742+0000 7f49b2e00700 1 --2- 172.21.15.153:0/261787980 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f499006e460 0x7f4990070920 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f499c00b5c0 tx=0x7f499c01a040 comp rx=0 tx=0).stop 2024-03-21T10:33:03.741 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.742+0000 7f49b2e00700 1 -- 172.21.15.153:0/261787980 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49ac073a80 msgr2=0x7f49ac198ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:03.741 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.742+0000 7f49b2e00700 1 --2- 172.21.15.153:0/261787980 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49ac073a80 0x7f49ac198ec0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f499800d8d0 tx=0x7f499800dc90 comp rx=0 tx=0).stop 2024-03-21T10:33:03.741 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.742+0000 7f49b2e00700 1 -- 172.21.15.153:0/261787980 shutdown_connections 2024-03-21T10:33:03.741 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.742+0000 7f49b2e00700 1 --2- 172.21.15.153:0/261787980 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f49ac073160 0x7f49ac198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:03.741 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.742+0000 7f49b2e00700 1 --2- 172.21.15.153:0/261787980 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f499006e460 0x7f4990070920 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:03.741 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.742+0000 7f49b2e00700 1 --2- 172.21.15.153:0/261787980 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49ac073a80 0x7f49ac198ec0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:03.741 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.742+0000 7f49b2e00700 1 -- 172.21.15.153:0/261787980 >> 172.21.15.153:0/261787980 conn(0x7f49ac0fc630 msgr2=0x7f49ac1076e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:03.741 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.742+0000 7f49b2e00700 1 -- 172.21.15.153:0/261787980 shutdown_connections 2024-03-21T10:33:03.741 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.742+0000 7f49b2e00700 1 -- 172.21.15.153:0/261787980 wait complete. 2024-03-21T10:33:03.881 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.882+0000 7f37f2800700 1 -- 172.21.15.153:0/50753842 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f37ec103980 msgr2=0x7f37ec1079d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:03.881 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.882+0000 7f37f2800700 1 --2- 172.21.15.153:0/50753842 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f37ec103980 0x7f37ec1079d0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f37d8009910 tx=0x7f37d8009c20 comp rx=0 tx=0).stop 2024-03-21T10:33:03.881 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.883+0000 7f37f2800700 1 -- 172.21.15.153:0/50753842 shutdown_connections 2024-03-21T10:33:03.881 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.883+0000 7f37f2800700 1 --2- 172.21.15.153:0/50753842 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f37ec103980 0x7f37ec1079d0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:03.881 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.883+0000 7f37f2800700 1 --2- 172.21.15.153:0/50753842 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f37ec102fd0 0x7f37ec1033b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:03.881 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.883+0000 7f37f2800700 1 -- 172.21.15.153:0/50753842 >> 172.21.15.153:0/50753842 conn(0x7f37ec0fe840 msgr2=0x7f37ec100c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:03.881 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.883+0000 7f37f2800700 1 -- 172.21.15.153:0/50753842 shutdown_connections 2024-03-21T10:33:03.881 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.883+0000 7f37f2800700 1 -- 172.21.15.153:0/50753842 wait complete. 2024-03-21T10:33:03.882 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.884+0000 7f37f2800700 1 Processor -- start 2024-03-21T10:33:03.882 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.884+0000 7f37f2800700 1 -- start start 2024-03-21T10:33:03.883 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.885+0000 7f37f2800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f37ec102fd0 0x7f37ec19e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:03.883 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.885+0000 7f37f2800700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f37ec103980 0x7f37ec19ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:03.883 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.885+0000 7f37f2800700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37ec19f310 con 0x7f37ec102fd0 2024-03-21T10:33:03.883 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.885+0000 7f37f2800700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37ec198850 con 0x7f37ec103980 2024-03-21T10:33:03.883 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.885+0000 7f37eb400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f37ec102fd0 0x7f37ec19e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:03.883 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.885+0000 7f37eb400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f37ec102fd0 0x7f37ec19e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:45740/0 (socket says 172.21.15.153:45740) 2024-03-21T10:33:03.883 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.885+0000 7f37eb400700 1 -- 172.21.15.153:0/1772505661 learned_addr learned my addr 172.21.15.153:0/1772505661 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:33:03.884 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.886+0000 7f37eaa00700 1 --2- 172.21.15.153:0/1772505661 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f37ec103980 0x7f37ec19ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:03.885 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.886+0000 7f37eb400700 1 -- 172.21.15.153:0/1772505661 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f37ec103980 msgr2=0x7f37ec19ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:03.885 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.886+0000 7f37eb400700 1 --2- 172.21.15.153:0/1772505661 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f37ec103980 0x7f37ec19ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:03.885 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.886+0000 7f37eb400700 1 -- 172.21.15.153:0/1772505661 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37d80095f0 con 0x7f37ec102fd0 2024-03-21T10:33:03.885 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.886+0000 7f37eb400700 1 --2- 172.21.15.153:0/1772505661 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f37ec102fd0 0x7f37ec19e7d0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f37dc00c930 tx=0x7f37dc00ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:03.885 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.886+0000 7f37e3e00700 1 -- 172.21.15.153:0/1772505661 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f37dc007ad0 con 0x7f37ec102fd0 2024-03-21T10:33:03.885 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.886+0000 7f37e3e00700 1 -- 172.21.15.153:0/1772505661 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f37dc007c30 con 0x7f37ec102fd0 2024-03-21T10:33:03.885 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.887+0000 7f37f2800700 1 -- 172.21.15.153:0/1772505661 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37ec198b30 con 0x7f37ec102fd0 2024-03-21T10:33:03.885 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.887+0000 7f37e3e00700 1 -- 172.21.15.153:0/1772505661 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f37dc018610 con 0x7f37ec102fd0 2024-03-21T10:33:03.885 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.887+0000 7f37f2800700 1 -- 172.21.15.153:0/1772505661 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37ec199080 con 0x7f37ec102fd0 2024-03-21T10:33:03.885 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.888+0000 7f37f2800700 1 -- 172.21.15.153:0/1772505661 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37ec10b3d0 con 0x7f37ec102fd0 2024-03-21T10:33:03.886 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.889+0000 7f37e3e00700 1 -- 172.21.15.153:0/1772505661 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f37dc00f450 con 0x7f37ec102fd0 2024-03-21T10:33:03.887 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.890+0000 7f37e3e00700 1 --2- 172.21.15.153:0/1772505661 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f37d006e460 0x7f37d0070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:03.887 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.890+0000 7f37e3e00700 1 -- 172.21.15.153:0/1772505661 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f37dc08e7b0 con 0x7f37ec102fd0 2024-03-21T10:33:03.888 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.890+0000 7f37eaa00700 1 --2- 172.21.15.153:0/1772505661 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f37d006e460 0x7f37d0070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:03.888 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.891+0000 7f37eaa00700 1 --2- 172.21.15.153:0/1772505661 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f37d006e460 0x7f37d0070920 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f37d80059d0 tx=0x7f37d8005960 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:03.892 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:03.894+0000 7f37e3e00700 1 -- 172.21.15.153:0/1772505661 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f37dc05b300 con 0x7f37ec102fd0 2024-03-21T10:33:04.199 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:04.201+0000 7f37f2800700 1 -- 172.21.15.153:0/1772505661 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f37ec04ea90 con 0x7f37ec102fd0 2024-03-21T10:33:04.199 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:04.202+0000 7f37e3e00700 1 -- 172.21.15.153:0/1772505661 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+805 (secure 0 0 0) 0x7f37dc05ae90 con 0x7f37ec102fd0 2024-03-21T10:33:04.200 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:33:04.200 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": { 2024-03-21T10:33:04.200 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:33:04.200 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:33:04.200 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": { 2024-03-21T10:33:04.200 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:33:04.201 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:33:04.201 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": { 2024-03-21T10:33:04.201 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 6 2024-03-21T10:33:04.201 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:33:04.201 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": { 2024-03-21T10:33:04.201 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-03-21T10:33:04.201 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 3 2024-03-21T10:33:04.201 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:33:04.201 INFO:teuthology.orchestra.run.smithi153.stdout: "overall": { 2024-03-21T10:33:04.201 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-03-21T10:33:04.201 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 13 2024-03-21T10:33:04.201 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-03-21T10:33:04.201 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:33:04.202 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:04.204+0000 7f37f2800700 1 -- 172.21.15.153:0/1772505661 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f37d006e460 msgr2=0x7f37d0070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:04.202 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:04.204+0000 7f37f2800700 1 --2- 172.21.15.153:0/1772505661 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f37d006e460 0x7f37d0070920 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f37d80059d0 tx=0x7f37d8005960 comp rx=0 tx=0).stop 2024-03-21T10:33:04.202 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:04.204+0000 7f37f2800700 1 -- 172.21.15.153:0/1772505661 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f37ec102fd0 msgr2=0x7f37ec19e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:04.202 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:04.204+0000 7f37f2800700 1 --2- 172.21.15.153:0/1772505661 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f37ec102fd0 0x7f37ec19e7d0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f37dc00c930 tx=0x7f37dc00ccf0 comp rx=0 tx=0).stop 2024-03-21T10:33:04.203 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:04.204+0000 7f37f2800700 1 -- 172.21.15.153:0/1772505661 shutdown_connections 2024-03-21T10:33:04.203 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:04.204+0000 7f37f2800700 1 --2- 172.21.15.153:0/1772505661 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f37d006e460 0x7f37d0070920 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:04.203 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:04.204+0000 7f37f2800700 1 --2- 172.21.15.153:0/1772505661 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f37ec102fd0 0x7f37ec19e7d0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:04.203 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:04.204+0000 7f37f2800700 1 --2- 172.21.15.153:0/1772505661 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f37ec103980 0x7f37ec19ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:04.203 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:04.204+0000 7f37f2800700 1 -- 172.21.15.153:0/1772505661 >> 172.21.15.153:0/1772505661 conn(0x7f37ec0fe840 msgr2=0x7f37ec100c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:04.203 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:04.205+0000 7f37f2800700 1 -- 172.21.15.153:0/1772505661 shutdown_connections 2024-03-21T10:33:04.203 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:04.205+0000 7f37f2800700 1 -- 172.21.15.153:0/1772505661 wait complete. 2024-03-21T10:33:04.355 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:04.357+0000 7f712be00700 1 -- 172.21.15.153:0/392939626 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f712c107220 msgr2=0x7f712c107600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:04.355 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:04.357+0000 7f712be00700 1 --2- 172.21.15.153:0/392939626 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f712c107220 0x7f712c107600 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f711c009910 tx=0x7f711c009c20 comp rx=0 tx=0).stop 2024-03-21T10:33:04.355 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:04.357+0000 7f712be00700 1 -- 172.21.15.153:0/392939626 shutdown_connections 2024-03-21T10:33:04.356 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:04.357+0000 7f712be00700 1 --2- 172.21.15.153:0/392939626 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f712c107bd0 0x7f712c10bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:04.356 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:04.357+0000 7f712be00700 1 --2- 172.21.15.153:0/392939626 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f712c107220 0x7f712c107600 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:04.356 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:04.357+0000 7f712be00700 1 -- 172.21.15.153:0/392939626 >> 172.21.15.153:0/392939626 conn(0x7f712c0767e0 msgr2=0x7f712c078c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:04.356 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:04.357+0000 7f712be00700 1 -- 172.21.15.153:0/392939626 shutdown_connections 2024-03-21T10:33:04.356 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:04.357+0000 7f712be00700 1 -- 172.21.15.153:0/392939626 wait complete. 2024-03-21T10:33:04.357 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:04.358+0000 7f712be00700 1 Processor -- start 2024-03-21T10:33:04.357 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:04.358+0000 7f712be00700 1 -- start start 2024-03-21T10:33:04.357 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:04.359+0000 7f712be00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f712c107220 0x7f712c1a2a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:04.357 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:04.359+0000 7f712be00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f712c107bd0 0x7f712c1a2f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:04.357 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:04.359+0000 7f712be00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f712c1a35f0 con 0x7f712c107bd0 2024-03-21T10:33:04.357 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:04.359+0000 7f712be00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f712c19caa0 con 0x7f712c107220 2024-03-21T10:33:04.358 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:04.359+0000 7f712a000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f712c107bd0 0x7f712c1a2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:04.358 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:04.359+0000 7f712a000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f712c107bd0 0x7f712c1a2f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:45772/0 (socket says 172.21.15.153:45772) 2024-03-21T10:33:04.358 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:04.359+0000 7f712a000700 1 -- 172.21.15.153:0/420295229 learned_addr learned my addr 172.21.15.153:0/420295229 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:33:04.358 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:04.359+0000 7f712aa00700 1 --2- 172.21.15.153:0/420295229 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f712c107220 0x7f712c1a2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:04.358 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:04.359+0000 7f712a000700 1 -- 172.21.15.153:0/420295229 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f712c107220 msgr2=0x7f712c1a2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:04.358 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:04.359+0000 7f712a000700 1 --2- 172.21.15.153:0/420295229 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f712c107220 0x7f712c1a2a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:04.358 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:04.359+0000 7f712a000700 1 -- 172.21.15.153:0/420295229 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f711c0095f0 con 0x7f712c107bd0 2024-03-21T10:33:04.359 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:04.360+0000 7f712a000700 1 --2- 172.21.15.153:0/420295229 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f712c107bd0 0x7f712c1a2f60 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f711800d900 tx=0x7f711800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:04.359 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:04.360+0000 7f7123400700 1 -- 172.21.15.153:0/420295229 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7118009900 con 0x7f712c107bd0 2024-03-21T10:33:04.359 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:04.360+0000 7f7123400700 1 -- 172.21.15.153:0/420295229 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7118010460 con 0x7f712c107bd0 2024-03-21T10:33:04.359 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:04.360+0000 7f7123400700 1 -- 172.21.15.153:0/420295229 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f711800f5f0 con 0x7f712c107bd0 2024-03-21T10:33:04.359 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:04.360+0000 7f712be00700 1 -- 172.21.15.153:0/420295229 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f712c19cd80 con 0x7f712c107bd0 2024-03-21T10:33:04.359 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:04.360+0000 7f712be00700 1 -- 172.21.15.153:0/420295229 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f712c19d2d0 con 0x7f712c107bd0 2024-03-21T10:33:04.360 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:04.361+0000 7f712be00700 1 -- 172.21.15.153:0/420295229 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f712c10f5b0 con 0x7f712c107bd0 2024-03-21T10:33:04.360 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:04.362+0000 7f7123400700 1 -- 172.21.15.153:0/420295229 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f71180105d0 con 0x7f712c107bd0 2024-03-21T10:33:04.361 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:04.363+0000 7f7123400700 1 --2- 172.21.15.153:0/420295229 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f711006e460 0x7f7110070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:04.361 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:04.363+0000 7f7123400700 1 -- 172.21.15.153:0/420295229 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f711808e360 con 0x7f712c107bd0 2024-03-21T10:33:04.361 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:04.363+0000 7f712aa00700 1 --2- 172.21.15.153:0/420295229 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f711006e460 0x7f7110070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:04.362 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:04.364+0000 7f712aa00700 1 --2- 172.21.15.153:0/420295229 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f711006e460 0x7f7110070920 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f711c006010 tx=0x7f711c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:04.366 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:04.368+0000 7f7123400700 1 -- 172.21.15.153:0/420295229 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f711805af30 con 0x7f712c107bd0 2024-03-21T10:33:04.554 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:33:04 smithi153 ceph-mon[113733]: from='client.35150 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:33:04.554 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:33:04 smithi153 ceph-mon[113733]: from='client.35154 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:33:04.554 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:33:04 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/1772505661' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:33:04.635 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:04.637+0000 7f712be00700 1 -- 172.21.15.153:0/420295229 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f712c19d540 con 0x7f712c107bd0 2024-03-21T10:33:04.636 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:04.638+0000 7f7123400700 1 -- 172.21.15.153:0/420295229 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1876 (secure 0 0 0) 0x7f7118017770 con 0x7f712c107bd0 2024-03-21T10:33:04.637 INFO:teuthology.orchestra.run.smithi153.stdout:e30 2024-03-21T10:33:04.637 INFO:teuthology.orchestra.run.smithi153.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-21T10:33:04.637 INFO:teuthology.orchestra.run.smithi153.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:33:04.637 INFO:teuthology.orchestra.run.smithi153.stdout:legacy client fscid: 1 2024-03-21T10:33:04.638 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:33:04.638 INFO:teuthology.orchestra.run.smithi153.stdout:Filesystem 'cephfs' (1) 2024-03-21T10:33:04.638 INFO:teuthology.orchestra.run.smithi153.stdout:fs_name cephfs 2024-03-21T10:33:04.638 INFO:teuthology.orchestra.run.smithi153.stdout:epoch 23 2024-03-21T10:33:04.638 INFO:teuthology.orchestra.run.smithi153.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-21T10:33:04.638 INFO:teuthology.orchestra.run.smithi153.stdout:created 2024-03-21T10:04:27.086576+0000 2024-03-21T10:33:04.638 INFO:teuthology.orchestra.run.smithi153.stdout:modified 2024-03-21T10:19:05.022092+0000 2024-03-21T10:33:04.638 INFO:teuthology.orchestra.run.smithi153.stdout:tableserver 0 2024-03-21T10:33:04.638 INFO:teuthology.orchestra.run.smithi153.stdout:root 0 2024-03-21T10:33:04.638 INFO:teuthology.orchestra.run.smithi153.stdout:session_timeout 60 2024-03-21T10:33:04.638 INFO:teuthology.orchestra.run.smithi153.stdout:session_autoclose 300 2024-03-21T10:33:04.638 INFO:teuthology.orchestra.run.smithi153.stdout:max_file_size 1099511627776 2024-03-21T10:33:04.638 INFO:teuthology.orchestra.run.smithi153.stdout:max_xattr_size 65536 2024-03-21T10:33:04.638 INFO:teuthology.orchestra.run.smithi153.stdout:required_client_features {} 2024-03-21T10:33:04.639 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure 0 2024-03-21T10:33:04.639 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure_osd_epoch 94 2024-03-21T10:33:04.639 INFO:teuthology.orchestra.run.smithi153.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:33:04.639 INFO:teuthology.orchestra.run.smithi153.stdout:max_mds 2 2024-03-21T10:33:04.639 INFO:teuthology.orchestra.run.smithi153.stdout:in 0,1 2024-03-21T10:33:04.639 INFO:teuthology.orchestra.run.smithi153.stdout:up {} 2024-03-21T10:33:04.639 INFO:teuthology.orchestra.run.smithi153.stdout:failed 0,1 2024-03-21T10:33:04.639 INFO:teuthology.orchestra.run.smithi153.stdout:damaged 2024-03-21T10:33:04.639 INFO:teuthology.orchestra.run.smithi153.stdout:stopped 2024-03-21T10:33:04.639 INFO:teuthology.orchestra.run.smithi153.stdout:data_pools [3] 2024-03-21T10:33:04.639 INFO:teuthology.orchestra.run.smithi153.stdout:metadata_pool 2 2024-03-21T10:33:04.639 INFO:teuthology.orchestra.run.smithi153.stdout:inline_data enabled 2024-03-21T10:33:04.639 INFO:teuthology.orchestra.run.smithi153.stdout:balancer 2024-03-21T10:33:04.639 INFO:teuthology.orchestra.run.smithi153.stdout:bal_rank_mask -1 2024-03-21T10:33:04.640 INFO:teuthology.orchestra.run.smithi153.stdout:standby_count_wanted 1 2024-03-21T10:33:04.640 INFO:teuthology.orchestra.run.smithi153.stdout:qdb_cluster leader: 0 members: 2024-03-21T10:33:04.640 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:33:04.640 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:33:04.640 INFO:teuthology.orchestra.run.smithi153.stdout:Standby daemons: 2024-03-21T10:33:04.640 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:33:04.640 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.zipckr{-1:34442} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6826/1103846295,v1:172.21.15.153:6827/1103846295] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:33:04.640 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.iuyluj{-1:34474} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6828/1738630832,v1:172.21.15.153:6829/1738630832] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:33:04.640 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.nhozag{-1:44269} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.174:6826/3967223227,v1:172.21.15.174:6827/3967223227] compat {c=[1],r=[1],i=[7ff]}] 2024-03-21T10:33:04.640 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.yyimst{-1:44313} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.174:6824/4127786797,v1:172.21.15.174:6825/4127786797] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:33:04.641 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:04.641+0000 7f712be00700 1 -- 172.21.15.153:0/420295229 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f711006e460 msgr2=0x7f7110070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:04.641 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:04.641+0000 7f712be00700 1 --2- 172.21.15.153:0/420295229 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f711006e460 0x7f7110070920 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f711c006010 tx=0x7f711c01a040 comp rx=0 tx=0).stop 2024-03-21T10:33:04.641 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:04.641+0000 7f712be00700 1 -- 172.21.15.153:0/420295229 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f712c107bd0 msgr2=0x7f712c1a2f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:04.641 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:04.641+0000 7f712be00700 1 --2- 172.21.15.153:0/420295229 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f712c107bd0 0x7f712c1a2f60 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f711800d900 tx=0x7f711800dcc0 comp rx=0 tx=0).stop 2024-03-21T10:33:04.641 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:04.641+0000 7f712be00700 1 -- 172.21.15.153:0/420295229 shutdown_connections 2024-03-21T10:33:04.641 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:04.641+0000 7f712be00700 1 --2- 172.21.15.153:0/420295229 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f712c107220 0x7f712c1a2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:04.641 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:04.641+0000 7f712be00700 1 --2- 172.21.15.153:0/420295229 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f711006e460 0x7f7110070920 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:04.641 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:04.641+0000 7f712be00700 1 --2- 172.21.15.153:0/420295229 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f712c107bd0 0x7f712c1a2f60 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:04.641 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:04.641+0000 7f712be00700 1 -- 172.21.15.153:0/420295229 >> 172.21.15.153:0/420295229 conn(0x7f712c0767e0 msgr2=0x7f712c077e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:04.642 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:04.642+0000 7f712be00700 1 -- 172.21.15.153:0/420295229 shutdown_connections 2024-03-21T10:33:04.642 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:04.642+0000 7f712be00700 1 -- 172.21.15.153:0/420295229 wait complete. 2024-03-21T10:33:04.642 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 30 2024-03-21T10:33:04.699 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:33:04 smithi174 ceph-mon[88004]: from='client.35150 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:33:04.699 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:33:04 smithi174 ceph-mon[88004]: from='client.35154 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:33:04.699 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:33:04 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/1772505661' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:33:04.795 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:04.796+0000 7fb5bd400700 1 -- 172.21.15.153:0/3996897163 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb5b8107220 msgr2=0x7fb5b8107600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:04.795 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:04.796+0000 7fb5bd400700 1 --2- 172.21.15.153:0/3996897163 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb5b8107220 0x7fb5b8107600 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fb5a8009910 tx=0x7fb5a8009c20 comp rx=0 tx=0).stop 2024-03-21T10:33:04.795 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:04.796+0000 7fb5bd400700 1 -- 172.21.15.153:0/3996897163 shutdown_connections 2024-03-21T10:33:04.795 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:04.796+0000 7fb5bd400700 1 --2- 172.21.15.153:0/3996897163 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb5b8107bd0 0x7fb5b810bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:04.795 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:04.796+0000 7fb5bd400700 1 --2- 172.21.15.153:0/3996897163 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb5b8107220 0x7fb5b8107600 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:04.795 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:04.796+0000 7fb5bd400700 1 -- 172.21.15.153:0/3996897163 >> 172.21.15.153:0/3996897163 conn(0x7fb5b80767e0 msgr2=0x7fb5b8078c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:04.795 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:04.796+0000 7fb5bd400700 1 -- 172.21.15.153:0/3996897163 shutdown_connections 2024-03-21T10:33:04.795 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:04.797+0000 7fb5bd400700 1 -- 172.21.15.153:0/3996897163 wait complete. 2024-03-21T10:33:04.797 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:04.798+0000 7fb5bd400700 1 Processor -- start 2024-03-21T10:33:04.797 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:04.798+0000 7fb5bd400700 1 -- start start 2024-03-21T10:33:04.797 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:04.798+0000 7fb5bd400700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb5b8107220 0x7fb5b81a2a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:04.797 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:04.798+0000 7fb5bd400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb5b8107bd0 0x7fb5b81a2f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:04.797 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:04.798+0000 7fb5bd400700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5b81a35f0 con 0x7fb5b8107bd0 2024-03-21T10:33:04.797 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:04.798+0000 7fb5bd400700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5b819caa0 con 0x7fb5b8107220 2024-03-21T10:33:04.797 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:04.798+0000 7fb5b7400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb5b8107bd0 0x7fb5b81a2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:04.798 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:04.799+0000 7fb5b7400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb5b8107bd0 0x7fb5b81a2f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:45788/0 (socket says 172.21.15.153:45788) 2024-03-21T10:33:04.798 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:04.799+0000 7fb5b7400700 1 -- 172.21.15.153:0/204864155 learned_addr learned my addr 172.21.15.153:0/204864155 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:33:04.798 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:04.799+0000 7fb5b7e00700 1 --2- 172.21.15.153:0/204864155 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb5b8107220 0x7fb5b81a2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:04.798 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:04.799+0000 7fb5b7400700 1 -- 172.21.15.153:0/204864155 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb5b8107220 msgr2=0x7fb5b81a2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:04.798 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:04.799+0000 7fb5b7400700 1 --2- 172.21.15.153:0/204864155 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb5b8107220 0x7fb5b81a2a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:04.798 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:04.799+0000 7fb5b7400700 1 -- 172.21.15.153:0/204864155 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5a80095f0 con 0x7fb5b8107bd0 2024-03-21T10:33:04.798 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:04.799+0000 7fb5b7400700 1 --2- 172.21.15.153:0/204864155 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb5b8107bd0 0x7fb5b81a2f60 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fb5a400c8f0 tx=0x7fb5a400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:04.799 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:04.800+0000 7fb5b4c00700 1 -- 172.21.15.153:0/204864155 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb5a4007a50 con 0x7fb5b8107bd0 2024-03-21T10:33:04.799 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:04.800+0000 7fb5b4c00700 1 -- 172.21.15.153:0/204864155 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb5a4007bb0 con 0x7fb5b8107bd0 2024-03-21T10:33:04.799 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:04.800+0000 7fb5b4c00700 1 -- 172.21.15.153:0/204864155 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb5a4018630 con 0x7fb5b8107bd0 2024-03-21T10:33:04.799 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:04.800+0000 7fb5bd400700 1 -- 172.21.15.153:0/204864155 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5b819cd80 con 0x7fb5b8107bd0 2024-03-21T10:33:04.799 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:04.800+0000 7fb5bd400700 1 -- 172.21.15.153:0/204864155 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5b819d2d0 con 0x7fb5b8107bd0 2024-03-21T10:33:04.799 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:04.801+0000 7fb5bd400700 1 -- 172.21.15.153:0/204864155 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5b810f5b0 con 0x7fb5b8107bd0 2024-03-21T10:33:04.800 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:04.802+0000 7fb5b4c00700 1 -- 172.21.15.153:0/204864155 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fb5a4007d20 con 0x7fb5b8107bd0 2024-03-21T10:33:04.800 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:04.803+0000 7fb5b4c00700 1 --2- 172.21.15.153:0/204864155 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb59c06e460 0x7fb59c070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:04.801 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:04.803+0000 7fb5b4c00700 1 -- 172.21.15.153:0/204864155 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fb5a408e450 con 0x7fb5b8107bd0 2024-03-21T10:33:04.801 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:04.803+0000 7fb5b7e00700 1 --2- 172.21.15.153:0/204864155 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb59c06e460 0x7fb59c070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:04.801 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:04.804+0000 7fb5b7e00700 1 --2- 172.21.15.153:0/204864155 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb59c06e460 0x7fb59c070920 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb5a800b5c0 tx=0x7fb5a801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:04.805 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:04.807+0000 7fb5b4c00700 1 -- 172.21.15.153:0/204864155 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb5a405b020 con 0x7fb5b8107bd0 2024-03-21T10:33:05.046 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:05.048+0000 7fb5bd400700 1 -- 172.21.15.153:0/204864155 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb5b819d610 con 0x7fb59c06e460 2024-03-21T10:33:05.050 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:05.052+0000 7fb5b4c00700 1 -- 172.21.15.153:0/204864155 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fb5b819d610 con 0x7fb59c06e460 2024-03-21T10:33:05.050 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:33:05.050 INFO:teuthology.orchestra.run.smithi153.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232", 2024-03-21T10:33:05.050 INFO:teuthology.orchestra.run.smithi153.stdout: "in_progress": true, 2024-03-21T10:33:05.051 INFO:teuthology.orchestra.run.smithi153.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-21T10:33:05.051 INFO:teuthology.orchestra.run.smithi153.stdout: "services_complete": [ 2024-03-21T10:33:05.051 INFO:teuthology.orchestra.run.smithi153.stdout: "crash", 2024-03-21T10:33:05.051 INFO:teuthology.orchestra.run.smithi153.stdout: "mon", 2024-03-21T10:33:05.051 INFO:teuthology.orchestra.run.smithi153.stdout: "osd", 2024-03-21T10:33:05.051 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr" 2024-03-21T10:33:05.051 INFO:teuthology.orchestra.run.smithi153.stdout: ], 2024-03-21T10:33:05.051 INFO:teuthology.orchestra.run.smithi153.stdout: "progress": "15/23 daemons upgraded", 2024-03-21T10:33:05.051 INFO:teuthology.orchestra.run.smithi153.stdout: "message": "Currently upgrading mds daemons", 2024-03-21T10:33:05.051 INFO:teuthology.orchestra.run.smithi153.stdout: "is_paused": false 2024-03-21T10:33:05.051 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:33:05.052 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:05.055+0000 7fb5bd400700 1 -- 172.21.15.153:0/204864155 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb59c06e460 msgr2=0x7fb59c070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:05.053 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:05.055+0000 7fb5bd400700 1 --2- 172.21.15.153:0/204864155 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb59c06e460 0x7fb59c070920 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb5a800b5c0 tx=0x7fb5a801a040 comp rx=0 tx=0).stop 2024-03-21T10:33:05.053 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:05.055+0000 7fb5bd400700 1 -- 172.21.15.153:0/204864155 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb5b8107bd0 msgr2=0x7fb5b81a2f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:05.053 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:05.055+0000 7fb5bd400700 1 --2- 172.21.15.153:0/204864155 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb5b8107bd0 0x7fb5b81a2f60 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fb5a400c8f0 tx=0x7fb5a400ccb0 comp rx=0 tx=0).stop 2024-03-21T10:33:05.053 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:05.055+0000 7fb5bd400700 1 -- 172.21.15.153:0/204864155 shutdown_connections 2024-03-21T10:33:05.054 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:05.055+0000 7fb5bd400700 1 --2- 172.21.15.153:0/204864155 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb5b8107220 0x7fb5b81a2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:05.054 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:05.055+0000 7fb5bd400700 1 --2- 172.21.15.153:0/204864155 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb59c06e460 0x7fb59c070920 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:05.054 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:05.055+0000 7fb5bd400700 1 --2- 172.21.15.153:0/204864155 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb5b8107bd0 0x7fb5b81a2f60 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:05.054 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:05.055+0000 7fb5bd400700 1 -- 172.21.15.153:0/204864155 >> 172.21.15.153:0/204864155 conn(0x7fb5b80767e0 msgr2=0x7fb5b8077e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:05.054 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:05.055+0000 7fb5bd400700 1 -- 172.21.15.153:0/204864155 shutdown_connections 2024-03-21T10:33:05.054 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:05.055+0000 7fb5bd400700 1 -- 172.21.15.153:0/204864155 wait complete. 2024-03-21T10:33:05.202 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:05.204+0000 7fd54ea00700 1 -- 172.21.15.153:0/3073360556 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd54810d280 msgr2=0x7fd54810d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:05.203 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:05.204+0000 7fd54ea00700 1 --2- 172.21.15.153:0/3073360556 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd54810d280 0x7fd54810d660 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fd538009910 tx=0x7fd538009c20 comp rx=0 tx=0).stop 2024-03-21T10:33:05.203 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:05.204+0000 7fd54ea00700 1 -- 172.21.15.153:0/3073360556 shutdown_connections 2024-03-21T10:33:05.203 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:05.204+0000 7fd54ea00700 1 --2- 172.21.15.153:0/3073360556 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd5481071d0 0x7fd548107650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:05.203 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:05.204+0000 7fd54ea00700 1 --2- 172.21.15.153:0/3073360556 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd54810d280 0x7fd54810d660 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:05.203 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:05.204+0000 7fd54ea00700 1 -- 172.21.15.153:0/3073360556 >> 172.21.15.153:0/3073360556 conn(0x7fd5480769e0 msgr2=0x7fd548078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:05.203 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:05.205+0000 7fd54ea00700 1 -- 172.21.15.153:0/3073360556 shutdown_connections 2024-03-21T10:33:05.203 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:05.205+0000 7fd54ea00700 1 -- 172.21.15.153:0/3073360556 wait complete. 2024-03-21T10:33:05.205 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:05.205+0000 7fd54ea00700 1 Processor -- start 2024-03-21T10:33:05.205 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:05.206+0000 7fd54ea00700 1 -- start start 2024-03-21T10:33:05.205 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:05.206+0000 7fd54ea00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd5481071d0 0x7fd54819ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:05.205 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:05.206+0000 7fd54ea00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd54810d280 0x7fd54819d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:05.205 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:05.206+0000 7fd54ea00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd54819d8a0 con 0x7fd5481071d0 2024-03-21T10:33:05.205 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:05.206+0000 7fd54ea00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5481a05d0 con 0x7fd54810d280 2024-03-21T10:33:05.205 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:05.206+0000 7fd547e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd5481071d0 0x7fd54819ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:05.205 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:05.206+0000 7fd547e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd5481071d0 0x7fd54819ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:45812/0 (socket says 172.21.15.153:45812) 2024-03-21T10:33:05.205 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:05.206+0000 7fd547e00700 1 -- 172.21.15.153:0/3969943947 learned_addr learned my addr 172.21.15.153:0/3969943947 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:33:05.205 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:05.207+0000 7fd547e00700 1 -- 172.21.15.153:0/3969943947 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd54810d280 msgr2=0x7fd54819d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:05.205 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:05.207+0000 7fd547400700 1 --2- 172.21.15.153:0/3969943947 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd54810d280 0x7fd54819d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:05.205 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:05.207+0000 7fd547e00700 1 --2- 172.21.15.153:0/3969943947 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd54810d280 0x7fd54819d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:05.206 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:05.207+0000 7fd547e00700 1 -- 172.21.15.153:0/3969943947 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5380095f0 con 0x7fd5481071d0 2024-03-21T10:33:05.206 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:05.207+0000 7fd547e00700 1 --2- 172.21.15.153:0/3969943947 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd5481071d0 0x7fd54819ccd0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fd53800b5c0 tx=0x7fd53800be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:05.210 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:05.207+0000 7fd544c00700 1 -- 172.21.15.153:0/3969943947 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd53801d070 con 0x7fd5481071d0 2024-03-21T10:33:05.211 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:05.207+0000 7fd54ea00700 1 -- 172.21.15.153:0/3969943947 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5481a0850 con 0x7fd5481071d0 2024-03-21T10:33:05.211 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:05.207+0000 7fd544c00700 1 -- 172.21.15.153:0/3969943947 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd53800f460 con 0x7fd5481071d0 2024-03-21T10:33:05.211 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:05.207+0000 7fd544c00700 1 -- 172.21.15.153:0/3969943947 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd538017610 con 0x7fd5481071d0 2024-03-21T10:33:05.211 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:05.207+0000 7fd54ea00700 1 -- 172.21.15.153:0/3969943947 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5481a0dc0 con 0x7fd5481071d0 2024-03-21T10:33:05.211 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:05.208+0000 7fd54ea00700 1 -- 172.21.15.153:0/3969943947 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd54804ea90 con 0x7fd5481071d0 2024-03-21T10:33:05.211 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:05.210+0000 7fd544c00700 1 -- 172.21.15.153:0/3969943947 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fd53800f5d0 con 0x7fd5481071d0 2024-03-21T10:33:05.211 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:05.210+0000 7fd544c00700 1 --2- 172.21.15.153:0/3969943947 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd52c06e460 0x7fd52c070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:05.211 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:05.211+0000 7fd544c00700 1 -- 172.21.15.153:0/3969943947 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fd53808fb90 con 0x7fd5481071d0 2024-03-21T10:33:05.211 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:05.211+0000 7fd547400700 1 --2- 172.21.15.153:0/3969943947 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd52c06e460 0x7fd52c070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:05.211 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:05.211+0000 7fd547400700 1 --2- 172.21.15.153:0/3969943947 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd52c06e460 0x7fd52c070920 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fd54819e2b0 tx=0x7fd534005cf0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:05.213 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:05.215+0000 7fd544c00700 1 -- 172.21.15.153:0/3969943947 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd538058bb0 con 0x7fd5481071d0 2024-03-21T10:33:05.514 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:05.516+0000 7fd54ea00700 1 -- 172.21.15.153:0/3969943947 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fd54819dff0 con 0x7fd5481071d0 2024-03-21T10:33:05.515 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:05.517+0000 7fd544c00700 1 -- 172.21.15.153:0/3969943947 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7fd53805c270 con 0x7fd5481071d0 2024-03-21T10:33:05.515 INFO:teuthology.orchestra.run.smithi153.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-03-21T10:33:05.515 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-21T10:33:05.515 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is degraded 2024-03-21T10:33:05.515 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-21T10:33:05.515 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-21T10:33:05.515 INFO:teuthology.orchestra.run.smithi153.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-21T10:33:05.515 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-21T10:33:05.518 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:05.520+0000 7fd54ea00700 1 -- 172.21.15.153:0/3969943947 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd52c06e460 msgr2=0x7fd52c070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:05.518 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:05.520+0000 7fd54ea00700 1 --2- 172.21.15.153:0/3969943947 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd52c06e460 0x7fd52c070920 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fd54819e2b0 tx=0x7fd534005cf0 comp rx=0 tx=0).stop 2024-03-21T10:33:05.518 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:05.520+0000 7fd54ea00700 1 -- 172.21.15.153:0/3969943947 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd5481071d0 msgr2=0x7fd54819ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:05.518 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:05.520+0000 7fd54ea00700 1 --2- 172.21.15.153:0/3969943947 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd5481071d0 0x7fd54819ccd0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fd53800b5c0 tx=0x7fd53800be00 comp rx=0 tx=0).stop 2024-03-21T10:33:05.519 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:05.520+0000 7fd54ea00700 1 -- 172.21.15.153:0/3969943947 shutdown_connections 2024-03-21T10:33:05.519 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:05.520+0000 7fd54ea00700 1 --2- 172.21.15.153:0/3969943947 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd52c06e460 0x7fd52c070920 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:05.519 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:05.520+0000 7fd54ea00700 1 --2- 172.21.15.153:0/3969943947 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd5481071d0 0x7fd54819ccd0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:05.519 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:05.520+0000 7fd54ea00700 1 --2- 172.21.15.153:0/3969943947 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd54810d280 0x7fd54819d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:05.519 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:05.520+0000 7fd54ea00700 1 -- 172.21.15.153:0/3969943947 >> 172.21.15.153:0/3969943947 conn(0x7fd5480769e0 msgr2=0x7fd548078320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:05.519 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:05.521+0000 7fd54ea00700 1 -- 172.21.15.153:0/3969943947 shutdown_connections 2024-03-21T10:33:05.519 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:05.521+0000 7fd54ea00700 1 -- 172.21.15.153:0/3969943947 wait complete. 2024-03-21T10:33:05.699 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:33:05 smithi174 ceph-mon[88004]: pgmap v636: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:33:05.699 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:33:05 smithi174 ceph-mon[88004]: from='client.35158 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:33:05.699 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:33:05 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/420295229' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:33:05.804 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:33:05 smithi153 ceph-mon[113733]: pgmap v636: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:33:05.804 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:33:05 smithi153 ceph-mon[113733]: from='client.35158 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:33:05.804 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:33:05 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/420295229' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:33:06.699 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:33:06 smithi174 ceph-mon[88004]: from='client.35170 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:33:06.699 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:33:06 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/3969943947' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:33:06.803 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:33:06 smithi153 ceph-mon[113733]: from='client.35170 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:33:06.804 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:33:06 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/3969943947' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:33:07.698 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:33:07 smithi174 ceph-mon[88004]: pgmap v637: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:33:07.803 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:33:07 smithi153 ceph-mon[113733]: pgmap v637: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:33:09.699 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:33:09 smithi174 ceph-mon[88004]: pgmap v638: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:33:09.803 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:33:09 smithi153 ceph-mon[113733]: pgmap v638: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:33:11.699 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:33:11 smithi174 ceph-mon[88004]: pgmap v639: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:33:11.803 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:33:11 smithi153 ceph-mon[113733]: pgmap v639: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:33:13.699 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:33:13 smithi174 ceph-mon[88004]: pgmap v640: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:33:13.803 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:33:13 smithi153 ceph-mon[113733]: pgmap v640: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:33:15.687 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:33:15 smithi153 ceph-mon[113733]: pgmap v641: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:33:15.698 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:33:15 smithi174 ceph-mon[88004]: pgmap v641: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:33:16.699 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:33:16 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:33:16.803 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:33:16 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:33:17.803 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:33:17 smithi153 ceph-mon[113733]: pgmap v642: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:33:17.949 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:33:17 smithi174 ceph-mon[88004]: pgmap v642: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:33:18.803 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:33:18 smithi153 ceph-mon[113733]: pgmap v643: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:33:18.949 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:33:18 smithi174 ceph-mon[88004]: pgmap v643: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:33:20.948 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:33:20 smithi174 ceph-mon[88004]: pgmap v644: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:33:21.053 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:33:20 smithi153 ceph-mon[113733]: pgmap v644: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:33:22.949 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:33:22 smithi174 ceph-mon[88004]: pgmap v645: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:33:23.053 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:33:22 smithi153 ceph-mon[113733]: pgmap v645: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:33:24.949 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:33:24 smithi174 ceph-mon[88004]: pgmap v646: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:33:25.054 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:33:24 smithi153 ceph-mon[113733]: pgmap v646: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:33:26.949 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:33:26 smithi174 ceph-mon[88004]: pgmap v647: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:33:27.053 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:33:26 smithi153 ceph-mon[113733]: pgmap v647: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:33:28.945 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:33:28 smithi174 ceph-mon[88004]: pgmap v648: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:33:29.054 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:33:28 smithi153 ceph-mon[113733]: pgmap v648: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:33:30.949 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:33:30 smithi174 ceph-mon[88004]: pgmap v649: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:33:31.053 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:33:30 smithi153 ceph-mon[113733]: pgmap v649: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:33:31.948 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:33:31 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:33:32.053 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:33:31 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:33:32.948 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:33:32 smithi174 ceph-mon[88004]: pgmap v650: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:33:33.053 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:33:32 smithi153 ceph-mon[113733]: pgmap v650: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:33:34.948 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:33:34 smithi174 ceph-mon[88004]: pgmap v651: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:33:35.054 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:33:34 smithi153 ceph-mon[113733]: pgmap v651: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:33:35.689 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:35.690+0000 7f001fc00700 1 -- 172.21.15.153:0/2680061617 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0018101750 msgr2=0x7f00181057a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:35.690 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:35.690+0000 7f001fc00700 1 --2- 172.21.15.153:0/2680061617 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0018101750 0x7f00181057a0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f0008009910 tx=0x7f0008009c20 comp rx=0 tx=0).stop 2024-03-21T10:33:35.690 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:35.690+0000 7f001fc00700 1 -- 172.21.15.153:0/2680061617 shutdown_connections 2024-03-21T10:33:35.690 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:35.690+0000 7f001fc00700 1 --2- 172.21.15.153:0/2680061617 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0018101750 0x7f00181057a0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:35.690 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:35.690+0000 7f001fc00700 1 --2- 172.21.15.153:0/2680061617 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0018100da0 0x7f0018101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:35.690 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:35.690+0000 7f001fc00700 1 -- 172.21.15.153:0/2680061617 >> 172.21.15.153:0/2680061617 conn(0x7f00180fc630 msgr2=0x7f00180fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:35.690 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:35.690+0000 7f001fc00700 1 -- 172.21.15.153:0/2680061617 shutdown_connections 2024-03-21T10:33:35.690 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:35.690+0000 7f001fc00700 1 -- 172.21.15.153:0/2680061617 wait complete. 2024-03-21T10:33:35.690 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:35.692+0000 7f001fc00700 1 Processor -- start 2024-03-21T10:33:35.691 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:35.692+0000 7f001fc00700 1 -- start start 2024-03-21T10:33:35.691 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:35.692+0000 7f001fc00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0018100da0 0x7f0018198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:35.691 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:35.692+0000 7f001fc00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0018101750 0x7f0018198e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:35.691 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:35.692+0000 7f001fc00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0018199490 con 0x7f0018101750 2024-03-21T10:33:35.691 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:35.692+0000 7f001fc00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0018199600 con 0x7f0018100da0 2024-03-21T10:33:35.692 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:35.693+0000 7f000fe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0018101750 0x7f0018198e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:35.692 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:35.693+0000 7f000fe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0018101750 0x7f0018198e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:37952/0 (socket says 172.21.15.153:37952) 2024-03-21T10:33:35.692 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:35.693+0000 7f000fe00700 1 -- 172.21.15.153:0/1982216588 learned_addr learned my addr 172.21.15.153:0/1982216588 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:33:35.693 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:35.693+0000 7f001d200700 1 --2- 172.21.15.153:0/1982216588 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0018100da0 0x7f0018198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:35.693 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:35.693+0000 7f000fe00700 1 -- 172.21.15.153:0/1982216588 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0018100da0 msgr2=0x7f0018198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:35.693 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:35.693+0000 7f000fe00700 1 --2- 172.21.15.153:0/1982216588 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0018100da0 0x7f0018198950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:35.693 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:35.693+0000 7f000fe00700 1 -- 172.21.15.153:0/1982216588 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00080095f0 con 0x7f0018101750 2024-03-21T10:33:35.693 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:35.693+0000 7f000fe00700 1 --2- 172.21.15.153:0/1982216588 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0018101750 0x7f0018198e90 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f000800bda0 tx=0x7f000800be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:35.693 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:35.694+0000 7f0016000700 1 -- 172.21.15.153:0/1982216588 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f000801d070 con 0x7f0018101750 2024-03-21T10:33:35.693 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:35.694+0000 7f0016000700 1 -- 172.21.15.153:0/1982216588 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0008004fb0 con 0x7f0018101750 2024-03-21T10:33:35.693 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:35.694+0000 7f001fc00700 1 -- 172.21.15.153:0/1982216588 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f001819d420 con 0x7f0018101750 2024-03-21T10:33:35.693 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:35.694+0000 7f0016000700 1 -- 172.21.15.153:0/1982216588 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f000800f460 con 0x7f0018101750 2024-03-21T10:33:35.693 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:35.694+0000 7f001fc00700 1 -- 172.21.15.153:0/1982216588 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f001819d910 con 0x7f0018101750 2024-03-21T10:33:35.694 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:35.695+0000 7f001fc00700 1 -- 172.21.15.153:0/1982216588 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0018104f20 con 0x7f0018101750 2024-03-21T10:33:35.695 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:35.696+0000 7f0016000700 1 -- 172.21.15.153:0/1982216588 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f0008022800 con 0x7f0018101750 2024-03-21T10:33:35.696 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:35.697+0000 7f0016000700 1 --2- 172.21.15.153:0/1982216588 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7efff806e460 0x7efff8070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:35.696 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:35.697+0000 7f0016000700 1 -- 172.21.15.153:0/1982216588 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f000808fa20 con 0x7f0018101750 2024-03-21T10:33:35.696 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:35.698+0000 7f001d200700 1 --2- 172.21.15.153:0/1982216588 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7efff806e460 0x7efff8070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:35.697 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:35.698+0000 7f001d200700 1 --2- 172.21.15.153:0/1982216588 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7efff806e460 0x7efff8070920 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f000000ba60 tx=0x7f0000005d50 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:35.701 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:35.703+0000 7f0016000700 1 -- 172.21.15.153:0/1982216588 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f000805c7c0 con 0x7f0018101750 2024-03-21T10:33:35.946 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:35.947+0000 7f001fc00700 1 -- 172.21.15.153:0/1982216588 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f001802cce0 con 0x7efff806e460 2024-03-21T10:33:35.950 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:35.950+0000 7f0016000700 1 -- 172.21.15.153:0/1982216588 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f001802cce0 con 0x7efff806e460 2024-03-21T10:33:35.953 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:35.954+0000 7f001fc00700 1 -- 172.21.15.153:0/1982216588 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7efff806e460 msgr2=0x7efff8070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:35.953 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:35.954+0000 7f001fc00700 1 --2- 172.21.15.153:0/1982216588 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7efff806e460 0x7efff8070920 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f000000ba60 tx=0x7f0000005d50 comp rx=0 tx=0).stop 2024-03-21T10:33:35.953 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:35.954+0000 7f001fc00700 1 -- 172.21.15.153:0/1982216588 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0018101750 msgr2=0x7f0018198e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:35.953 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:35.954+0000 7f001fc00700 1 --2- 172.21.15.153:0/1982216588 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0018101750 0x7f0018198e90 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f000800bda0 tx=0x7f000800be80 comp rx=0 tx=0).stop 2024-03-21T10:33:35.953 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:35.954+0000 7f001fc00700 1 -- 172.21.15.153:0/1982216588 shutdown_connections 2024-03-21T10:33:35.953 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:35.954+0000 7f001fc00700 1 --2- 172.21.15.153:0/1982216588 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0018100da0 0x7f0018198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:35.953 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:35.954+0000 7f001fc00700 1 --2- 172.21.15.153:0/1982216588 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7efff806e460 0x7efff8070920 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:35.953 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:35.954+0000 7f001fc00700 1 --2- 172.21.15.153:0/1982216588 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0018101750 0x7f0018198e90 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:35.953 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:35.954+0000 7f001fc00700 1 -- 172.21.15.153:0/1982216588 >> 172.21.15.153:0/1982216588 conn(0x7f00180fc630 msgr2=0x7f00180fe9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:35.953 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:35.954+0000 7f001fc00700 1 -- 172.21.15.153:0/1982216588 shutdown_connections 2024-03-21T10:33:35.954 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:35.954+0000 7f001fc00700 1 -- 172.21.15.153:0/1982216588 wait complete. 2024-03-21T10:33:35.969 INFO:teuthology.orchestra.run.smithi153.stdout:true 2024-03-21T10:33:36.107 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.107+0000 7fb463800700 1 -- 172.21.15.153:0/1611715846 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb45c103980 msgr2=0x7fb45c1079d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:36.107 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.107+0000 7fb463800700 1 --2- 172.21.15.153:0/1611715846 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb45c103980 0x7fb45c1079d0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fb448009910 tx=0x7fb448009c20 comp rx=0 tx=0).stop 2024-03-21T10:33:36.107 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.108+0000 7fb463800700 1 -- 172.21.15.153:0/1611715846 shutdown_connections 2024-03-21T10:33:36.107 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.108+0000 7fb463800700 1 --2- 172.21.15.153:0/1611715846 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb45c103980 0x7fb45c1079d0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:36.107 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.108+0000 7fb463800700 1 --2- 172.21.15.153:0/1611715846 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb45c102fd0 0x7fb45c1033b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:36.107 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.108+0000 7fb463800700 1 -- 172.21.15.153:0/1611715846 >> 172.21.15.153:0/1611715846 conn(0x7fb45c0fe840 msgr2=0x7fb45c100c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:36.107 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.108+0000 7fb463800700 1 -- 172.21.15.153:0/1611715846 shutdown_connections 2024-03-21T10:33:36.107 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.108+0000 7fb463800700 1 -- 172.21.15.153:0/1611715846 wait complete. 2024-03-21T10:33:36.108 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.109+0000 7fb463800700 1 Processor -- start 2024-03-21T10:33:36.108 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.109+0000 7fb463800700 1 -- start start 2024-03-21T10:33:36.108 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.109+0000 7fb463800700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb45c102fd0 0x7fb45c19e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:36.108 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.109+0000 7fb463800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb45c103980 0x7fb45c19ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:36.108 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.109+0000 7fb463800700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb45c19f3a0 con 0x7fb45c103980 2024-03-21T10:33:36.108 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.109+0000 7fb463800700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb45c198850 con 0x7fb45c102fd0 2024-03-21T10:33:36.108 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.110+0000 7fb45be00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb45c103980 0x7fb45c19ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:36.109 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.110+0000 7fb45be00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb45c103980 0x7fb45c19ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:37974/0 (socket says 172.21.15.153:37974) 2024-03-21T10:33:36.109 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.110+0000 7fb45be00700 1 -- 172.21.15.153:0/1700832848 learned_addr learned my addr 172.21.15.153:0/1700832848 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:33:36.110 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.110+0000 7fb460e00700 1 --2- 172.21.15.153:0/1700832848 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb45c102fd0 0x7fb45c19e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:36.110 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.110+0000 7fb45be00700 1 -- 172.21.15.153:0/1700832848 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb45c102fd0 msgr2=0x7fb45c19e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:36.110 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.110+0000 7fb45be00700 1 --2- 172.21.15.153:0/1700832848 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb45c102fd0 0x7fb45c19e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:36.110 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.110+0000 7fb45be00700 1 -- 172.21.15.153:0/1700832848 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4480095f0 con 0x7fb45c103980 2024-03-21T10:33:36.110 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.110+0000 7fb460e00700 1 --2- 172.21.15.153:0/1700832848 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb45c102fd0 0x7fb45c19e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:33:36.110 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.110+0000 7fb45be00700 1 --2- 172.21.15.153:0/1700832848 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb45c103980 0x7fb45c19ed10 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fb44800bd70 tx=0x7fb44800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:36.110 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.111+0000 7fb459600700 1 -- 172.21.15.153:0/1700832848 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb44801d070 con 0x7fb45c103980 2024-03-21T10:33:36.110 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.111+0000 7fb459600700 1 -- 172.21.15.153:0/1700832848 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb448005dc0 con 0x7fb45c103980 2024-03-21T10:33:36.110 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.111+0000 7fb459600700 1 -- 172.21.15.153:0/1700832848 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb448017400 con 0x7fb45c103980 2024-03-21T10:33:36.110 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.111+0000 7fb463800700 1 -- 172.21.15.153:0/1700832848 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb45c198ad0 con 0x7fb45c103980 2024-03-21T10:33:36.110 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.111+0000 7fb463800700 1 -- 172.21.15.153:0/1700832848 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb45c198fc0 con 0x7fb45c103980 2024-03-21T10:33:36.111 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.112+0000 7fb463800700 1 -- 172.21.15.153:0/1700832848 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb45c10b3d0 con 0x7fb45c103980 2024-03-21T10:33:36.111 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.113+0000 7fb459600700 1 -- 172.21.15.153:0/1700832848 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fb44800f7f0 con 0x7fb45c103980 2024-03-21T10:33:36.113 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.114+0000 7fb459600700 1 --2- 172.21.15.153:0/1700832848 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb44406e460 0x7fb444070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:36.113 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.114+0000 7fb459600700 1 -- 172.21.15.153:0/1700832848 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fb44808f990 con 0x7fb45c103980 2024-03-21T10:33:36.113 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.114+0000 7fb460e00700 1 --2- 172.21.15.153:0/1700832848 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb44406e460 0x7fb444070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:36.113 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.115+0000 7fb460e00700 1 --2- 172.21.15.153:0/1700832848 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb44406e460 0x7fb444070920 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fb450005fd0 tx=0x7fb450005ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:36.118 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.119+0000 7fb459600700 1 -- 172.21.15.153:0/1700832848 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb44805c560 con 0x7fb45c103980 2024-03-21T10:33:36.356 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.357+0000 7fb463800700 1 -- 172.21.15.153:0/1700832848 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb45c1992f0 con 0x7fb44406e460 2024-03-21T10:33:36.358 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.360+0000 7fb459600700 1 -- 172.21.15.153:0/1700832848 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fb45c1992f0 con 0x7fb44406e460 2024-03-21T10:33:36.362 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.363+0000 7fb463800700 1 -- 172.21.15.153:0/1700832848 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb44406e460 msgr2=0x7fb444070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:36.362 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.363+0000 7fb463800700 1 --2- 172.21.15.153:0/1700832848 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb44406e460 0x7fb444070920 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fb450005fd0 tx=0x7fb450005ee0 comp rx=0 tx=0).stop 2024-03-21T10:33:36.362 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.363+0000 7fb463800700 1 -- 172.21.15.153:0/1700832848 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb45c103980 msgr2=0x7fb45c19ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:36.363 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.363+0000 7fb463800700 1 --2- 172.21.15.153:0/1700832848 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb45c103980 0x7fb45c19ed10 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fb44800bd70 tx=0x7fb44800be50 comp rx=0 tx=0).stop 2024-03-21T10:33:36.363 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.364+0000 7fb463800700 1 -- 172.21.15.153:0/1700832848 shutdown_connections 2024-03-21T10:33:36.363 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.364+0000 7fb463800700 1 --2- 172.21.15.153:0/1700832848 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb45c102fd0 0x7fb45c19e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:36.363 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.364+0000 7fb463800700 1 --2- 172.21.15.153:0/1700832848 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb44406e460 0x7fb444070920 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:36.363 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.364+0000 7fb463800700 1 --2- 172.21.15.153:0/1700832848 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb45c103980 0x7fb45c19ed10 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:36.363 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.364+0000 7fb463800700 1 -- 172.21.15.153:0/1700832848 >> 172.21.15.153:0/1700832848 conn(0x7fb45c0fe840 msgr2=0x7fb45c100c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:36.363 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.364+0000 7fb463800700 1 -- 172.21.15.153:0/1700832848 shutdown_connections 2024-03-21T10:33:36.363 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.364+0000 7fb463800700 1 -- 172.21.15.153:0/1700832848 wait complete. 2024-03-21T10:33:36.523 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.524+0000 7f339fa00700 1 -- 172.21.15.153:0/834923623 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3398105a60 msgr2=0x7f3398105e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:36.523 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.524+0000 7f339fa00700 1 --2- 172.21.15.153:0/834923623 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3398105a60 0x7f3398105e40 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f3388009910 tx=0x7f3388009c20 comp rx=0 tx=0).stop 2024-03-21T10:33:36.523 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.524+0000 7f339fa00700 1 -- 172.21.15.153:0/834923623 shutdown_connections 2024-03-21T10:33:36.523 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.524+0000 7f339fa00700 1 --2- 172.21.15.153:0/834923623 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f33980ffa60 0x7f33980ffee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:36.523 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.524+0000 7f339fa00700 1 --2- 172.21.15.153:0/834923623 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3398105a60 0x7f3398105e40 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:36.523 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.524+0000 7f339fa00700 1 -- 172.21.15.153:0/834923623 >> 172.21.15.153:0/834923623 conn(0x7f33980fb550 msgr2=0x7f33980fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:36.523 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.524+0000 7f339fa00700 1 -- 172.21.15.153:0/834923623 shutdown_connections 2024-03-21T10:33:36.523 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.524+0000 7f339fa00700 1 -- 172.21.15.153:0/834923623 wait complete. 2024-03-21T10:33:36.524 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.525+0000 7f339fa00700 1 Processor -- start 2024-03-21T10:33:36.524 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.525+0000 7f339fa00700 1 -- start start 2024-03-21T10:33:36.524 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.526+0000 7f339fa00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f33980ffa60 0x7f33981989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:36.525 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.526+0000 7f339fa00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3398105a60 0x7f3398198ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:36.525 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.526+0000 7f339fa00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3398199570 con 0x7f3398105a60 2024-03-21T10:33:36.525 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.526+0000 7f339fa00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f339819d380 con 0x7f33980ffa60 2024-03-21T10:33:36.526 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.526+0000 7f338fe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3398105a60 0x7f3398198ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:36.526 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.526+0000 7f338fe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3398105a60 0x7f3398198ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:37996/0 (socket says 172.21.15.153:37996) 2024-03-21T10:33:36.526 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.526+0000 7f338fe00700 1 -- 172.21.15.153:0/1325272612 learned_addr learned my addr 172.21.15.153:0/1325272612 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:33:36.526 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.527+0000 7f339d000700 1 --2- 172.21.15.153:0/1325272612 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f33980ffa60 0x7f33981989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:36.526 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.527+0000 7f338fe00700 1 -- 172.21.15.153:0/1325272612 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f33980ffa60 msgr2=0x7f33981989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:36.526 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.527+0000 7f338fe00700 1 --2- 172.21.15.153:0/1325272612 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f33980ffa60 0x7f33981989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:36.526 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.527+0000 7f338fe00700 1 -- 172.21.15.153:0/1325272612 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33880095f0 con 0x7f3398105a60 2024-03-21T10:33:36.526 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.527+0000 7f339d000700 1 --2- 172.21.15.153:0/1325272612 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f33980ffa60 0x7f33981989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:33:36.526 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.527+0000 7f338fe00700 1 --2- 172.21.15.153:0/1325272612 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3398105a60 0x7f3398198ee0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f338400d8d0 tx=0x7f338400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:36.527 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.527+0000 7f3396000700 1 -- 172.21.15.153:0/1325272612 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3384009940 con 0x7f3398105a60 2024-03-21T10:33:36.527 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.527+0000 7f3396000700 1 -- 172.21.15.153:0/1325272612 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3384010460 con 0x7f3398105a60 2024-03-21T10:33:36.527 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.527+0000 7f3396000700 1 -- 172.21.15.153:0/1325272612 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f338400f5f0 con 0x7f3398105a60 2024-03-21T10:33:36.527 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.527+0000 7f339fa00700 1 -- 172.21.15.153:0/1325272612 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f339819d660 con 0x7f3398105a60 2024-03-21T10:33:36.527 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.528+0000 7f339fa00700 1 -- 172.21.15.153:0/1325272612 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f339819dbb0 con 0x7f3398105a60 2024-03-21T10:33:36.528 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.529+0000 7f339fa00700 1 -- 172.21.15.153:0/1325272612 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f339804ea90 con 0x7f3398105a60 2024-03-21T10:33:36.528 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.530+0000 7f3396000700 1 -- 172.21.15.153:0/1325272612 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f33840105d0 con 0x7f3398105a60 2024-03-21T10:33:36.529 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.531+0000 7f3396000700 1 --2- 172.21.15.153:0/1325272612 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f337c06e460 0x7f337c070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:36.529 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.531+0000 7f3396000700 1 -- 172.21.15.153:0/1325272612 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f338408e3e0 con 0x7f3398105a60 2024-03-21T10:33:36.530 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.531+0000 7f339d000700 1 --2- 172.21.15.153:0/1325272612 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f337c06e460 0x7f337c070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:36.530 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.532+0000 7f339d000700 1 --2- 172.21.15.153:0/1325272612 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f337c06e460 0x7f337c070920 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f3388005b40 tx=0x7f338801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:36.534 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.535+0000 7f3396000700 1 -- 172.21.15.153:0/1325272612 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f338405b050 con 0x7f3398105a60 2024-03-21T10:33:36.768 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.769+0000 7f339fa00700 1 -- 172.21.15.153:0/1325272612 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f339819df00 con 0x7f337c06e460 2024-03-21T10:33:36.780 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.781+0000 7f3396000700 1 -- 172.21.15.153:0/1325272612 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f339819df00 con 0x7f337c06e460 2024-03-21T10:33:36.780 INFO:teuthology.orchestra.run.smithi153.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-21T10:33:36.781 INFO:teuthology.orchestra.run.smithi153.stdout:alertmanager.smithi153 smithi153 *:9093,9094 running (22m) 13m ago 36m 14.7M - 0.25.0 c8568f914cd2 9ed3f2200b5f 2024-03-21T10:33:36.781 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi153 smithi153 running (36m) 13m ago 36m 9424k - 18.2.1 5be31c24972a 5b61994396a6 2024-03-21T10:33:36.781 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi174 smithi174 running (34m) 13m ago 34m 9244k - 18.2.1 5be31c24972a 4f7a5fc3ac28 2024-03-21T10:33:36.781 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi153 smithi153 running (18m) 13m ago 36m 6656k - 19.0.0-2231-g2edfee68 237e43c7a65f 32c21104d5b0 2024-03-21T10:33:36.781 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi174 smithi174 running (18m) 13m ago 34m 6660k - 19.0.0-2231-g2edfee68 237e43c7a65f a04e48967ab6 2024-03-21T10:33:36.781 INFO:teuthology.orchestra.run.smithi153.stdout:grafana.smithi153 smithi153 *:3000 running (21m) 13m ago 35m 73.6M - 9.4.12 2bacad6d85d8 ee24026ab786 2024-03-21T10:33:36.781 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.iuyluj smithi153 running (13m) 13m ago 28m 10.9M - 19.0.0-2231-g2edfee68 237e43c7a65f 87aff8a5de70 2024-03-21T10:33:36.781 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.zipckr smithi153 running (14m) 13m ago 29m 11.8M - 19.0.0-2231-g2edfee68 237e43c7a65f ccc262bd5dd3 2024-03-21T10:33:36.781 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.nhozag smithi174 running (28m) 13m ago 28m 12.0M - 18.2.1 5be31c24972a 2fee4000308f 2024-03-21T10:33:36.781 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.yyimst smithi174 running (13m) 13m ago 29m 10.7M - 19.0.0-2231-g2edfee68 237e43c7a65f fc1962f9351e 2024-03-21T10:33:36.781 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi153.tluxni smithi153 *:8443,9283,8765 running (25m) 13m ago 37m 509M - 19.0.0-2231-g2edfee68 237e43c7a65f 7ffee0e55382 2024-03-21T10:33:36.781 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi174.qpcwat smithi174 *:8443,9283,8765 running (24m) 13m ago 34m 452M - 19.0.0-2231-g2edfee68 237e43c7a65f 748968f216ab 2024-03-21T10:33:36.781 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi153 smithi153 running (20m) 13m ago 38m 55.0M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f dd8038303dc3 2024-03-21T10:33:36.782 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi174 smithi174 running (19m) 13m ago 34m 44.2M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f 29d78a788f2c 2024-03-21T10:33:36.782 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi153 smithi153 *:9100 running (23m) 13m ago 36m 7889k - 1.5.0 0da6a335fe13 bb835edc0e0b 2024-03-21T10:33:36.782 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi174 smithi174 *:9100 running (23m) 13m ago 34m 8102k - 1.5.0 0da6a335fe13 7c4ecad790fa 2024-03-21T10:33:36.782 INFO:teuthology.orchestra.run.smithi153.stdout:osd.0 smithi153 running (17m) 13m ago 33m 116M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 507900fd4791 2024-03-21T10:33:36.782 INFO:teuthology.orchestra.run.smithi153.stdout:osd.1 smithi153 running (17m) 13m ago 33m 89.2M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 0d646452e9c1 2024-03-21T10:33:36.782 INFO:teuthology.orchestra.run.smithi153.stdout:osd.2 smithi153 running (16m) 13m ago 32m 87.4M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 1326f2afdfcd 2024-03-21T10:33:36.782 INFO:teuthology.orchestra.run.smithi153.stdout:osd.3 smithi174 running (16m) 13m ago 31m 83.6M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 70aafac71ad5 2024-03-21T10:33:36.782 INFO:teuthology.orchestra.run.smithi153.stdout:osd.4 smithi174 running (15m) 13m ago 31m 92.2M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 0d746ce6e79d 2024-03-21T10:33:36.782 INFO:teuthology.orchestra.run.smithi153.stdout:osd.5 smithi174 running (14m) 13m ago 30m 86.2M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 53e5f35b8cac 2024-03-21T10:33:36.782 INFO:teuthology.orchestra.run.smithi153.stdout:prometheus.smithi153 smithi153 *:9095 running (22m) 13m ago 35m 73.5M - 2.43.0 a07b618ecd1d 7d95561e50c2 2024-03-21T10:33:36.783 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.784+0000 7f339fa00700 1 -- 172.21.15.153:0/1325272612 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f337c06e460 msgr2=0x7f337c070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:36.784 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.784+0000 7f339fa00700 1 --2- 172.21.15.153:0/1325272612 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f337c06e460 0x7f337c070920 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f3388005b40 tx=0x7f338801a040 comp rx=0 tx=0).stop 2024-03-21T10:33:36.784 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.784+0000 7f339fa00700 1 -- 172.21.15.153:0/1325272612 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3398105a60 msgr2=0x7f3398198ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:36.784 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.784+0000 7f339fa00700 1 --2- 172.21.15.153:0/1325272612 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3398105a60 0x7f3398198ee0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f338400d8d0 tx=0x7f338400dc90 comp rx=0 tx=0).stop 2024-03-21T10:33:36.784 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.785+0000 7f339fa00700 1 -- 172.21.15.153:0/1325272612 shutdown_connections 2024-03-21T10:33:36.784 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.785+0000 7f339fa00700 1 --2- 172.21.15.153:0/1325272612 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f33980ffa60 0x7f33981989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:36.784 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.785+0000 7f339fa00700 1 --2- 172.21.15.153:0/1325272612 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f337c06e460 0x7f337c070920 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:36.784 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.785+0000 7f339fa00700 1 --2- 172.21.15.153:0/1325272612 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3398105a60 0x7f3398198ee0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:36.784 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.785+0000 7f339fa00700 1 -- 172.21.15.153:0/1325272612 >> 172.21.15.153:0/1325272612 conn(0x7f33980fb550 msgr2=0x7f33980fc350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:36.784 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.785+0000 7f339fa00700 1 -- 172.21.15.153:0/1325272612 shutdown_connections 2024-03-21T10:33:36.784 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.785+0000 7f339fa00700 1 -- 172.21.15.153:0/1325272612 wait complete. 2024-03-21T10:33:36.943 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.944+0000 7f58b7000700 1 -- 172.21.15.153:0/3810747293 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f58b0102fc0 msgr2=0x7f58b01033a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:36.943 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.944+0000 7f58b7000700 1 --2- 172.21.15.153:0/3810747293 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f58b0102fc0 0x7f58b01033a0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f58a0009910 tx=0x7f58a0009c20 comp rx=0 tx=0).stop 2024-03-21T10:33:36.943 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.944+0000 7f58b7000700 1 -- 172.21.15.153:0/3810747293 shutdown_connections 2024-03-21T10:33:36.943 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.944+0000 7f58b7000700 1 --2- 172.21.15.153:0/3810747293 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f58b0103970 0x7f58b01079c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:36.944 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.944+0000 7f58b7000700 1 --2- 172.21.15.153:0/3810747293 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f58b0102fc0 0x7f58b01033a0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:36.944 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.944+0000 7f58b7000700 1 -- 172.21.15.153:0/3810747293 >> 172.21.15.153:0/3810747293 conn(0x7f58b00fe870 msgr2=0x7f58b0100c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:36.944 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.944+0000 7f58b7000700 1 -- 172.21.15.153:0/3810747293 shutdown_connections 2024-03-21T10:33:36.944 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.944+0000 7f58b7000700 1 -- 172.21.15.153:0/3810747293 wait complete. 2024-03-21T10:33:36.944 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.945+0000 7f58b7000700 1 Processor -- start 2024-03-21T10:33:36.945 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.945+0000 7f58b7000700 1 -- start start 2024-03-21T10:33:36.945 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.946+0000 7f58b7000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f58b0102fc0 0x7f58b019e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:36.945 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.946+0000 7f58b7000700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f58b0103970 0x7f58b019ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:36.945 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.946+0000 7f58b7000700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58b019f2c0 con 0x7f58b0102fc0 2024-03-21T10:33:36.945 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.946+0000 7f58b7000700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58b0198770 con 0x7f58b0103970 2024-03-21T10:33:36.945 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.946+0000 7f58afe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f58b0102fc0 0x7f58b019e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:36.945 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.946+0000 7f58afe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f58b0102fc0 0x7f58b019e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:38014/0 (socket says 172.21.15.153:38014) 2024-03-21T10:33:36.945 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.946+0000 7f58afe00700 1 -- 172.21.15.153:0/3334271542 learned_addr learned my addr 172.21.15.153:0/3334271542 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:33:36.946 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.946+0000 7f58af400700 1 --2- 172.21.15.153:0/3334271542 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f58b0103970 0x7f58b019ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:36.946 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.947+0000 7f58afe00700 1 -- 172.21.15.153:0/3334271542 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f58b0103970 msgr2=0x7f58b019ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:36.946 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.947+0000 7f58afe00700 1 --2- 172.21.15.153:0/3334271542 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f58b0103970 0x7f58b019ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:36.946 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.947+0000 7f58afe00700 1 -- 172.21.15.153:0/3334271542 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58a00095f0 con 0x7f58b0102fc0 2024-03-21T10:33:36.946 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.947+0000 7f58af400700 1 --2- 172.21.15.153:0/3334271542 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f58b0103970 0x7f58b019ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:33:36.946 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.947+0000 7f58afe00700 1 --2- 172.21.15.153:0/3334271542 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f58b0102fc0 0x7f58b019e6f0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f58a000b5c0 tx=0x7f58a000be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:36.946 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.947+0000 7f58acc00700 1 -- 172.21.15.153:0/3334271542 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f58a001d070 con 0x7f58b0102fc0 2024-03-21T10:33:36.947 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.947+0000 7f58b7000700 1 -- 172.21.15.153:0/3334271542 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58b01989f0 con 0x7f58b0102fc0 2024-03-21T10:33:36.947 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.947+0000 7f58acc00700 1 -- 172.21.15.153:0/3334271542 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f58a000f460 con 0x7f58b0102fc0 2024-03-21T10:33:36.947 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.947+0000 7f58acc00700 1 -- 172.21.15.153:0/3334271542 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f58a0017610 con 0x7f58b0102fc0 2024-03-21T10:33:36.947 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.948+0000 7f58b7000700 1 -- 172.21.15.153:0/3334271542 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58b0198ee0 con 0x7f58b0102fc0 2024-03-21T10:33:36.948 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.949+0000 7f58b7000700 1 -- 172.21.15.153:0/3334271542 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58b010b360 con 0x7f58b0102fc0 2024-03-21T10:33:36.949 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.950+0000 7f58acc00700 1 -- 172.21.15.153:0/3334271542 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f58a000f5d0 con 0x7f58b0102fc0 2024-03-21T10:33:36.950 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.952+0000 7f58acc00700 1 --2- 172.21.15.153:0/3334271542 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f589406e460 0x7f5894070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:36.951 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.952+0000 7f58acc00700 1 -- 172.21.15.153:0/3334271542 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f58a008fad0 con 0x7f58b0102fc0 2024-03-21T10:33:36.951 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.952+0000 7f58af400700 1 --2- 172.21.15.153:0/3334271542 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f589406e460 0x7f5894070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:36.951 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.953+0000 7f58af400700 1 --2- 172.21.15.153:0/3334271542 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f589406e460 0x7f5894070920 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f589c005fd0 tx=0x7f589c005dc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:36.955 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:36.957+0000 7f58acc00700 1 -- 172.21.15.153:0/3334271542 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f58a005c6a0 con 0x7f58b0102fc0 2024-03-21T10:33:36.956 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:33:36 smithi153 ceph-mon[113733]: pgmap v652: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:33:36.956 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:33:36 smithi153 ceph-mon[113733]: from='client.35178 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:33:36.956 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:33:36 smithi153 ceph-mon[113733]: from='client.35182 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:33:37.198 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:33:36 smithi174 ceph-mon[88004]: pgmap v652: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:33:37.199 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:33:36 smithi174 ceph-mon[88004]: from='client.35178 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:33:37.199 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:33:36 smithi174 ceph-mon[88004]: from='client.35182 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:33:37.246 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:37.247+0000 7f58b7000700 1 -- 172.21.15.153:0/3334271542 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f58b002cd30 con 0x7f58b0102fc0 2024-03-21T10:33:37.247 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:37.248+0000 7f58acc00700 1 -- 172.21.15.153:0/3334271542 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+805 (secure 0 0 0) 0x7f58a005c230 con 0x7f58b0102fc0 2024-03-21T10:33:37.248 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:33:37.248 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": { 2024-03-21T10:33:37.248 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:33:37.248 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:33:37.248 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": { 2024-03-21T10:33:37.248 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:33:37.248 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:33:37.248 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": { 2024-03-21T10:33:37.248 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 6 2024-03-21T10:33:37.248 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:33:37.248 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": { 2024-03-21T10:33:37.249 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-03-21T10:33:37.249 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 3 2024-03-21T10:33:37.249 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:33:37.249 INFO:teuthology.orchestra.run.smithi153.stdout: "overall": { 2024-03-21T10:33:37.249 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-03-21T10:33:37.249 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 13 2024-03-21T10:33:37.249 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-03-21T10:33:37.249 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:33:37.250 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:37.251+0000 7f58b7000700 1 -- 172.21.15.153:0/3334271542 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f589406e460 msgr2=0x7f5894070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:37.251 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:37.251+0000 7f58b7000700 1 --2- 172.21.15.153:0/3334271542 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f589406e460 0x7f5894070920 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f589c005fd0 tx=0x7f589c005dc0 comp rx=0 tx=0).stop 2024-03-21T10:33:37.251 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:37.251+0000 7f58b7000700 1 -- 172.21.15.153:0/3334271542 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f58b0102fc0 msgr2=0x7f58b019e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:37.251 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:37.251+0000 7f58b7000700 1 --2- 172.21.15.153:0/3334271542 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f58b0102fc0 0x7f58b019e6f0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f58a000b5c0 tx=0x7f58a000be00 comp rx=0 tx=0).stop 2024-03-21T10:33:37.251 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:37.252+0000 7f58b7000700 1 -- 172.21.15.153:0/3334271542 shutdown_connections 2024-03-21T10:33:37.251 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:37.252+0000 7f58b7000700 1 --2- 172.21.15.153:0/3334271542 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f589406e460 0x7f5894070920 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:37.251 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:37.252+0000 7f58b7000700 1 --2- 172.21.15.153:0/3334271542 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f58b0102fc0 0x7f58b019e6f0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:37.251 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:37.252+0000 7f58b7000700 1 --2- 172.21.15.153:0/3334271542 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f58b0103970 0x7f58b019ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:37.251 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:37.252+0000 7f58b7000700 1 -- 172.21.15.153:0/3334271542 >> 172.21.15.153:0/3334271542 conn(0x7f58b00fe870 msgr2=0x7f58b0100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:37.251 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:37.252+0000 7f58b7000700 1 -- 172.21.15.153:0/3334271542 shutdown_connections 2024-03-21T10:33:37.251 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:37.252+0000 7f58b7000700 1 -- 172.21.15.153:0/3334271542 wait complete. 2024-03-21T10:33:37.404 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:37.405+0000 7ff88ca00700 1 -- 172.21.15.153:0/3201476154 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff8880739d0 msgr2=0x7ff88810cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:37.404 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:37.405+0000 7ff88ca00700 1 --2- 172.21.15.153:0/3201476154 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff8880739d0 0x7ff88810cd40 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7ff874009910 tx=0x7ff874009c20 comp rx=0 tx=0).stop 2024-03-21T10:33:37.404 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:37.405+0000 7ff88ca00700 1 -- 172.21.15.153:0/3201476154 shutdown_connections 2024-03-21T10:33:37.404 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:37.405+0000 7ff88ca00700 1 --2- 172.21.15.153:0/3201476154 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff8880739d0 0x7ff88810cd40 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:37.404 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:37.405+0000 7ff88ca00700 1 --2- 172.21.15.153:0/3201476154 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff8880730b0 0x7ff888073490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:37.405 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:37.405+0000 7ff88ca00700 1 -- 172.21.15.153:0/3201476154 >> 172.21.15.153:0/3201476154 conn(0x7ff8880fc4f0 msgr2=0x7ff8880fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:37.405 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:37.405+0000 7ff88ca00700 1 -- 172.21.15.153:0/3201476154 shutdown_connections 2024-03-21T10:33:37.405 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:37.405+0000 7ff88ca00700 1 -- 172.21.15.153:0/3201476154 wait complete. 2024-03-21T10:33:37.406 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:37.406+0000 7ff88ca00700 1 Processor -- start 2024-03-21T10:33:37.406 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:37.407+0000 7ff88ca00700 1 -- start start 2024-03-21T10:33:37.406 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:37.407+0000 7ff88ca00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff8880730b0 0x7ff88819e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:37.406 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:37.407+0000 7ff88ca00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff8880739d0 0x7ff88819ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:37.406 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:37.407+0000 7ff88ca00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff88819f0f0 con 0x7ff8880739d0 2024-03-21T10:33:37.406 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:37.407+0000 7ff88ca00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8881985a0 con 0x7ff8880730b0 2024-03-21T10:33:37.406 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:37.407+0000 7ff886a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff8880739d0 0x7ff88819ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:37.406 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:37.407+0000 7ff886a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff8880739d0 0x7ff88819ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:38042/0 (socket says 172.21.15.153:38042) 2024-03-21T10:33:37.406 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:37.407+0000 7ff886a00700 1 -- 172.21.15.153:0/2811601306 learned_addr learned my addr 172.21.15.153:0/2811601306 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:33:37.407 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:37.408+0000 7ff887400700 1 --2- 172.21.15.153:0/2811601306 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff8880730b0 0x7ff88819e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:37.407 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:37.408+0000 7ff886a00700 1 -- 172.21.15.153:0/2811601306 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff8880730b0 msgr2=0x7ff88819e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:37.407 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:37.408+0000 7ff886a00700 1 --2- 172.21.15.153:0/2811601306 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff8880730b0 0x7ff88819e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:37.407 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:37.408+0000 7ff886a00700 1 -- 172.21.15.153:0/2811601306 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8740095f0 con 0x7ff8880739d0 2024-03-21T10:33:37.408 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:37.408+0000 7ff886a00700 1 --2- 172.21.15.153:0/2811601306 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff8880739d0 0x7ff88819ea60 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7ff874005f50 tx=0x7ff87400f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:37.408 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:37.408+0000 7ff87fe00700 1 -- 172.21.15.153:0/2811601306 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff87401d070 con 0x7ff8880739d0 2024-03-21T10:33:37.408 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:37.408+0000 7ff88ca00700 1 -- 172.21.15.153:0/2811601306 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff888198820 con 0x7ff8880739d0 2024-03-21T10:33:37.408 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:37.408+0000 7ff87fe00700 1 -- 172.21.15.153:0/2811601306 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff87400b610 con 0x7ff8880739d0 2024-03-21T10:33:37.408 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:37.408+0000 7ff87fe00700 1 -- 172.21.15.153:0/2811601306 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff874022470 con 0x7ff8880739d0 2024-03-21T10:33:37.408 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:37.408+0000 7ff88ca00700 1 -- 172.21.15.153:0/2811601306 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff888198d10 con 0x7ff8880739d0 2024-03-21T10:33:37.408 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:37.409+0000 7ff88ca00700 1 -- 172.21.15.153:0/2811601306 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff88810a440 con 0x7ff8880739d0 2024-03-21T10:33:37.409 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:37.410+0000 7ff87fe00700 1 -- 172.21.15.153:0/2811601306 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7ff87400fca0 con 0x7ff8880739d0 2024-03-21T10:33:37.410 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:37.411+0000 7ff87fe00700 1 --2- 172.21.15.153:0/2811601306 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff86c06e460 0x7ff86c070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:37.410 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:37.411+0000 7ff87fe00700 1 -- 172.21.15.153:0/2811601306 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7ff87408f8f0 con 0x7ff8880739d0 2024-03-21T10:33:37.410 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:37.412+0000 7ff887400700 1 --2- 172.21.15.153:0/2811601306 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff86c06e460 0x7ff86c070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:37.411 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:37.412+0000 7ff887400700 1 --2- 172.21.15.153:0/2811601306 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff86c06e460 0x7ff86c070920 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff878009b30 tx=0x7ff878009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:37.415 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:37.416+0000 7ff87fe00700 1 -- 172.21.15.153:0/2811601306 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff87405c4c0 con 0x7ff8880739d0 2024-03-21T10:33:37.665 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:37.666+0000 7ff88ca00700 1 -- 172.21.15.153:0/2811601306 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7ff88804f2e0 con 0x7ff8880739d0 2024-03-21T10:33:37.666 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:37.667+0000 7ff87fe00700 1 -- 172.21.15.153:0/2811601306 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1876 (secure 0 0 0) 0x7ff87405c050 con 0x7ff8880739d0 2024-03-21T10:33:37.667 INFO:teuthology.orchestra.run.smithi153.stdout:e30 2024-03-21T10:33:37.667 INFO:teuthology.orchestra.run.smithi153.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-21T10:33:37.667 INFO:teuthology.orchestra.run.smithi153.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:33:37.667 INFO:teuthology.orchestra.run.smithi153.stdout:legacy client fscid: 1 2024-03-21T10:33:37.667 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:33:37.667 INFO:teuthology.orchestra.run.smithi153.stdout:Filesystem 'cephfs' (1) 2024-03-21T10:33:37.667 INFO:teuthology.orchestra.run.smithi153.stdout:fs_name cephfs 2024-03-21T10:33:37.667 INFO:teuthology.orchestra.run.smithi153.stdout:epoch 23 2024-03-21T10:33:37.667 INFO:teuthology.orchestra.run.smithi153.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-21T10:33:37.667 INFO:teuthology.orchestra.run.smithi153.stdout:created 2024-03-21T10:04:27.086576+0000 2024-03-21T10:33:37.667 INFO:teuthology.orchestra.run.smithi153.stdout:modified 2024-03-21T10:19:05.022092+0000 2024-03-21T10:33:37.667 INFO:teuthology.orchestra.run.smithi153.stdout:tableserver 0 2024-03-21T10:33:37.668 INFO:teuthology.orchestra.run.smithi153.stdout:root 0 2024-03-21T10:33:37.668 INFO:teuthology.orchestra.run.smithi153.stdout:session_timeout 60 2024-03-21T10:33:37.668 INFO:teuthology.orchestra.run.smithi153.stdout:session_autoclose 300 2024-03-21T10:33:37.668 INFO:teuthology.orchestra.run.smithi153.stdout:max_file_size 1099511627776 2024-03-21T10:33:37.668 INFO:teuthology.orchestra.run.smithi153.stdout:max_xattr_size 65536 2024-03-21T10:33:37.668 INFO:teuthology.orchestra.run.smithi153.stdout:required_client_features {} 2024-03-21T10:33:37.668 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure 0 2024-03-21T10:33:37.668 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure_osd_epoch 94 2024-03-21T10:33:37.668 INFO:teuthology.orchestra.run.smithi153.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:33:37.668 INFO:teuthology.orchestra.run.smithi153.stdout:max_mds 2 2024-03-21T10:33:37.668 INFO:teuthology.orchestra.run.smithi153.stdout:in 0,1 2024-03-21T10:33:37.668 INFO:teuthology.orchestra.run.smithi153.stdout:up {} 2024-03-21T10:33:37.668 INFO:teuthology.orchestra.run.smithi153.stdout:failed 0,1 2024-03-21T10:33:37.668 INFO:teuthology.orchestra.run.smithi153.stdout:damaged 2024-03-21T10:33:37.668 INFO:teuthology.orchestra.run.smithi153.stdout:stopped 2024-03-21T10:33:37.669 INFO:teuthology.orchestra.run.smithi153.stdout:data_pools [3] 2024-03-21T10:33:37.669 INFO:teuthology.orchestra.run.smithi153.stdout:metadata_pool 2 2024-03-21T10:33:37.669 INFO:teuthology.orchestra.run.smithi153.stdout:inline_data enabled 2024-03-21T10:33:37.669 INFO:teuthology.orchestra.run.smithi153.stdout:balancer 2024-03-21T10:33:37.669 INFO:teuthology.orchestra.run.smithi153.stdout:bal_rank_mask -1 2024-03-21T10:33:37.669 INFO:teuthology.orchestra.run.smithi153.stdout:standby_count_wanted 1 2024-03-21T10:33:37.669 INFO:teuthology.orchestra.run.smithi153.stdout:qdb_cluster leader: 0 members: 2024-03-21T10:33:37.669 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:33:37.669 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:33:37.669 INFO:teuthology.orchestra.run.smithi153.stdout:Standby daemons: 2024-03-21T10:33:37.669 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:33:37.669 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.zipckr{-1:34442} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6826/1103846295,v1:172.21.15.153:6827/1103846295] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:33:37.669 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.iuyluj{-1:34474} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6828/1738630832,v1:172.21.15.153:6829/1738630832] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:33:37.669 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.nhozag{-1:44269} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.174:6826/3967223227,v1:172.21.15.174:6827/3967223227] compat {c=[1],r=[1],i=[7ff]}] 2024-03-21T10:33:37.669 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.yyimst{-1:44313} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.174:6824/4127786797,v1:172.21.15.174:6825/4127786797] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:33:37.672 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:37.671+0000 7ff88ca00700 1 -- 172.21.15.153:0/2811601306 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff86c06e460 msgr2=0x7ff86c070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:37.672 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:37.671+0000 7ff88ca00700 1 --2- 172.21.15.153:0/2811601306 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff86c06e460 0x7ff86c070920 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff878009b30 tx=0x7ff878009260 comp rx=0 tx=0).stop 2024-03-21T10:33:37.672 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:37.671+0000 7ff88ca00700 1 -- 172.21.15.153:0/2811601306 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff8880739d0 msgr2=0x7ff88819ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:37.672 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:37.671+0000 7ff88ca00700 1 --2- 172.21.15.153:0/2811601306 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff8880739d0 0x7ff88819ea60 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7ff874005f50 tx=0x7ff87400f740 comp rx=0 tx=0).stop 2024-03-21T10:33:37.672 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:37.671+0000 7ff88ca00700 1 -- 172.21.15.153:0/2811601306 shutdown_connections 2024-03-21T10:33:37.672 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:37.671+0000 7ff88ca00700 1 --2- 172.21.15.153:0/2811601306 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff8880730b0 0x7ff88819e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:37.672 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:37.671+0000 7ff88ca00700 1 --2- 172.21.15.153:0/2811601306 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff86c06e460 0x7ff86c070920 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:37.672 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:37.671+0000 7ff88ca00700 1 --2- 172.21.15.153:0/2811601306 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff8880739d0 0x7ff88819ea60 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:37.672 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:37.672+0000 7ff88ca00700 1 -- 172.21.15.153:0/2811601306 >> 172.21.15.153:0/2811601306 conn(0x7ff8880fc4f0 msgr2=0x7ff888107580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:37.672 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:37.672+0000 7ff88ca00700 1 -- 172.21.15.153:0/2811601306 shutdown_connections 2024-03-21T10:33:37.672 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:37.672+0000 7ff88ca00700 1 -- 172.21.15.153:0/2811601306 wait complete. 2024-03-21T10:33:37.673 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 30 2024-03-21T10:33:37.815 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:37.816+0000 7f3840200700 1 -- 172.21.15.153:0/3256778263 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f383810d280 msgr2=0x7f383810d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:37.816 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:37.816+0000 7f3840200700 1 --2- 172.21.15.153:0/3256778263 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f383810d280 0x7f383810d660 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f382c009910 tx=0x7f382c009c20 comp rx=0 tx=0).stop 2024-03-21T10:33:37.816 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:37.816+0000 7f3840200700 1 -- 172.21.15.153:0/3256778263 shutdown_connections 2024-03-21T10:33:37.816 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:37.816+0000 7f3840200700 1 --2- 172.21.15.153:0/3256778263 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f38381071d0 0x7f3838107650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:37.816 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:37.816+0000 7f3840200700 1 --2- 172.21.15.153:0/3256778263 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f383810d280 0x7f383810d660 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:37.816 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:37.816+0000 7f3840200700 1 -- 172.21.15.153:0/3256778263 >> 172.21.15.153:0/3256778263 conn(0x7f38380769e0 msgr2=0x7f3838078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:37.816 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:37.816+0000 7f3840200700 1 -- 172.21.15.153:0/3256778263 shutdown_connections 2024-03-21T10:33:37.816 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:37.816+0000 7f3840200700 1 -- 172.21.15.153:0/3256778263 wait complete. 2024-03-21T10:33:37.817 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:37.817+0000 7f3840200700 1 Processor -- start 2024-03-21T10:33:37.817 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:37.817+0000 7f3840200700 1 -- start start 2024-03-21T10:33:37.817 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:37.818+0000 7f3840200700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f38381071d0 0x7f383819ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:37.818 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:37.818+0000 7f3840200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f383810d280 0x7f383819d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:37.818 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:37.818+0000 7f3840200700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f383819da30 con 0x7f383810d280 2024-03-21T10:33:37.818 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:37.818+0000 7f3840200700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38381a1770 con 0x7f38381071d0 2024-03-21T10:33:37.818 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:37.818+0000 7f383ce00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f383810d280 0x7f383819d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:37.818 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:37.818+0000 7f383ce00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f383810d280 0x7f383819d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:38070/0 (socket says 172.21.15.153:38070) 2024-03-21T10:33:37.818 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:37.818+0000 7f383ce00700 1 -- 172.21.15.153:0/902452349 learned_addr learned my addr 172.21.15.153:0/902452349 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:33:37.818 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:37.818+0000 7f383d800700 1 --2- 172.21.15.153:0/902452349 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f38381071d0 0x7f383819ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:37.818 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:37.819+0000 7f383ce00700 1 -- 172.21.15.153:0/902452349 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f38381071d0 msgr2=0x7f383819ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:37.818 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:37.819+0000 7f383ce00700 1 --2- 172.21.15.153:0/902452349 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f38381071d0 0x7f383819ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:37.818 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:37.819+0000 7f383ce00700 1 -- 172.21.15.153:0/902452349 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f382c0095f0 con 0x7f383810d280 2024-03-21T10:33:37.818 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:37.819+0000 7f383ce00700 1 --2- 172.21.15.153:0/902452349 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f383810d280 0x7f383819d350 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f382400b4b0 tx=0x7f382400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:37.819 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:37.819+0000 7f3836000700 1 -- 172.21.15.153:0/902452349 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3824011800 con 0x7f383810d280 2024-03-21T10:33:37.819 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:37.819+0000 7f3836000700 1 -- 172.21.15.153:0/902452349 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3824011e20 con 0x7f383810d280 2024-03-21T10:33:37.819 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:37.819+0000 7f3840200700 1 -- 172.21.15.153:0/902452349 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38381a1a50 con 0x7f383810d280 2024-03-21T10:33:37.819 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:37.819+0000 7f3836000700 1 -- 172.21.15.153:0/902452349 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f382400f2e0 con 0x7f383810d280 2024-03-21T10:33:37.819 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:37.819+0000 7f3840200700 1 -- 172.21.15.153:0/902452349 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38381a2020 con 0x7f383810d280 2024-03-21T10:33:37.819 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:37.820+0000 7f3840200700 1 -- 172.21.15.153:0/902452349 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f383810f6f0 con 0x7f383810d280 2024-03-21T10:33:37.820 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:37.821+0000 7f3836000700 1 -- 172.21.15.153:0/902452349 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f3824010380 con 0x7f383810d280 2024-03-21T10:33:37.821 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:37.822+0000 7f3836000700 1 --2- 172.21.15.153:0/902452349 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f382006e460 0x7f3820070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:37.821 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:37.822+0000 7f3836000700 1 -- 172.21.15.153:0/902452349 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f382408e190 con 0x7f383810d280 2024-03-21T10:33:37.821 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:37.823+0000 7f383d800700 1 --2- 172.21.15.153:0/902452349 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f382006e460 0x7f3820070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:37.822 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:37.823+0000 7f383d800700 1 --2- 172.21.15.153:0/902452349 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f382006e460 0x7f3820070920 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f382c00b5c0 tx=0x7f382c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:37.825 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:37.827+0000 7f3836000700 1 -- 172.21.15.153:0/902452349 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f38240571b0 con 0x7f383810d280 2024-03-21T10:33:38.053 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:33:37 smithi153 ceph-mon[113733]: from='client.35186 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:33:38.054 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:33:37 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/3334271542' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:33:38.054 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:33:37 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/2811601306' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:33:38.066 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:38.068+0000 7f3840200700 1 -- 172.21.15.153:0/902452349 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f38381a2360 con 0x7f382006e460 2024-03-21T10:33:38.068 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:38.069+0000 7f3836000700 1 -- 172.21.15.153:0/902452349 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f38381a2360 con 0x7f382006e460 2024-03-21T10:33:38.068 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:33:38.069 INFO:teuthology.orchestra.run.smithi153.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232", 2024-03-21T10:33:38.069 INFO:teuthology.orchestra.run.smithi153.stdout: "in_progress": true, 2024-03-21T10:33:38.069 INFO:teuthology.orchestra.run.smithi153.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-21T10:33:38.069 INFO:teuthology.orchestra.run.smithi153.stdout: "services_complete": [ 2024-03-21T10:33:38.069 INFO:teuthology.orchestra.run.smithi153.stdout: "crash", 2024-03-21T10:33:38.069 INFO:teuthology.orchestra.run.smithi153.stdout: "mon", 2024-03-21T10:33:38.069 INFO:teuthology.orchestra.run.smithi153.stdout: "osd", 2024-03-21T10:33:38.069 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr" 2024-03-21T10:33:38.069 INFO:teuthology.orchestra.run.smithi153.stdout: ], 2024-03-21T10:33:38.069 INFO:teuthology.orchestra.run.smithi153.stdout: "progress": "15/23 daemons upgraded", 2024-03-21T10:33:38.069 INFO:teuthology.orchestra.run.smithi153.stdout: "message": "Currently upgrading mds daemons", 2024-03-21T10:33:38.070 INFO:teuthology.orchestra.run.smithi153.stdout: "is_paused": false 2024-03-21T10:33:38.070 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:33:38.072 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:38.073+0000 7f3840200700 1 -- 172.21.15.153:0/902452349 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f382006e460 msgr2=0x7f3820070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:38.072 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:38.073+0000 7f3840200700 1 --2- 172.21.15.153:0/902452349 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f382006e460 0x7f3820070920 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f382c00b5c0 tx=0x7f382c01a040 comp rx=0 tx=0).stop 2024-03-21T10:33:38.072 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:38.073+0000 7f3840200700 1 -- 172.21.15.153:0/902452349 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f383810d280 msgr2=0x7f383819d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:38.072 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:38.073+0000 7f3840200700 1 --2- 172.21.15.153:0/902452349 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f383810d280 0x7f383819d350 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f382400b4b0 tx=0x7f382400b870 comp rx=0 tx=0).stop 2024-03-21T10:33:38.072 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:38.073+0000 7f3840200700 1 -- 172.21.15.153:0/902452349 shutdown_connections 2024-03-21T10:33:38.072 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:38.073+0000 7f3840200700 1 --2- 172.21.15.153:0/902452349 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f38381071d0 0x7f383819ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:38.072 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:38.073+0000 7f3840200700 1 --2- 172.21.15.153:0/902452349 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f382006e460 0x7f3820070920 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:38.072 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:38.073+0000 7f3840200700 1 --2- 172.21.15.153:0/902452349 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f383810d280 0x7f383819d350 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:38.072 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:38.073+0000 7f3840200700 1 -- 172.21.15.153:0/902452349 >> 172.21.15.153:0/902452349 conn(0x7f38380769e0 msgr2=0x7f38380785c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:38.072 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:38.073+0000 7f3840200700 1 -- 172.21.15.153:0/902452349 shutdown_connections 2024-03-21T10:33:38.072 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:38.073+0000 7f3840200700 1 -- 172.21.15.153:0/902452349 wait complete. 2024-03-21T10:33:38.100 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:33:37 smithi174 ceph-mon[88004]: from='client.35186 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:33:38.100 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:33:37 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/3334271542' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:33:38.100 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:33:37 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/2811601306' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:33:38.222 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:38.223+0000 7f5654400700 1 -- 172.21.15.153:0/2037865314 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f564c10d280 msgr2=0x7f564c10d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:38.222 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:38.223+0000 7f5654400700 1 --2- 172.21.15.153:0/2037865314 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f564c10d280 0x7f564c10d660 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f5640009910 tx=0x7f5640009c20 comp rx=0 tx=0).stop 2024-03-21T10:33:38.223 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:38.223+0000 7f5654400700 1 -- 172.21.15.153:0/2037865314 shutdown_connections 2024-03-21T10:33:38.223 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:38.223+0000 7f5654400700 1 --2- 172.21.15.153:0/2037865314 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f564c1071d0 0x7f564c107650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:38.223 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:38.223+0000 7f5654400700 1 --2- 172.21.15.153:0/2037865314 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f564c10d280 0x7f564c10d660 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:38.223 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:38.223+0000 7f5654400700 1 -- 172.21.15.153:0/2037865314 >> 172.21.15.153:0/2037865314 conn(0x7f564c0769e0 msgr2=0x7f564c078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:38.223 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:38.223+0000 7f5654400700 1 -- 172.21.15.153:0/2037865314 shutdown_connections 2024-03-21T10:33:38.223 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:38.223+0000 7f5654400700 1 -- 172.21.15.153:0/2037865314 wait complete. 2024-03-21T10:33:38.223 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:38.225+0000 7f5654400700 1 Processor -- start 2024-03-21T10:33:38.223 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:38.225+0000 7f5654400700 1 -- start start 2024-03-21T10:33:38.224 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:38.225+0000 7f5654400700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f564c1071d0 0x7f564c19ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:38.224 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:38.225+0000 7f5654400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f564c10d280 0x7f564c19d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:38.224 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:38.225+0000 7f5654400700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f564c19d8a0 con 0x7f564c10d280 2024-03-21T10:33:38.225 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:38.225+0000 7f5654400700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f564c1a05d0 con 0x7f564c1071d0 2024-03-21T10:33:38.225 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:38.226+0000 7f5651000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f564c10d280 0x7f564c19d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:38.225 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:38.226+0000 7f5651000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f564c10d280 0x7f564c19d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:38084/0 (socket says 172.21.15.153:38084) 2024-03-21T10:33:38.225 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:38.226+0000 7f5651000700 1 -- 172.21.15.153:0/1057266982 learned_addr learned my addr 172.21.15.153:0/1057266982 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:33:38.225 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:38.226+0000 7f5651a00700 1 --2- 172.21.15.153:0/1057266982 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f564c1071d0 0x7f564c19ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:38.226 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:38.226+0000 7f5651000700 1 -- 172.21.15.153:0/1057266982 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f564c1071d0 msgr2=0x7f564c19ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:38.226 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:38.226+0000 7f5651000700 1 --2- 172.21.15.153:0/1057266982 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f564c1071d0 0x7f564c19ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:38.226 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:38.226+0000 7f5651000700 1 -- 172.21.15.153:0/1057266982 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56400095f0 con 0x7f564c10d280 2024-03-21T10:33:38.226 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:38.226+0000 7f5651000700 1 --2- 172.21.15.153:0/1057266982 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f564c10d280 0x7f564c19d210 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f563c00d8d0 tx=0x7f563c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:38.226 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:38.227+0000 7f564a000700 1 -- 172.21.15.153:0/1057266982 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f563c009940 con 0x7f564c10d280 2024-03-21T10:33:38.226 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:38.227+0000 7f564a000700 1 -- 172.21.15.153:0/1057266982 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f563c010460 con 0x7f564c10d280 2024-03-21T10:33:38.226 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:38.227+0000 7f5654400700 1 -- 172.21.15.153:0/1057266982 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f564c1a08b0 con 0x7f564c10d280 2024-03-21T10:33:38.227 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:38.227+0000 7f564a000700 1 -- 172.21.15.153:0/1057266982 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f563c00f5f0 con 0x7f564c10d280 2024-03-21T10:33:38.227 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:38.227+0000 7f5654400700 1 -- 172.21.15.153:0/1057266982 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f564c1a0e80 con 0x7f564c10d280 2024-03-21T10:33:38.227 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:38.228+0000 7f5654400700 1 -- 172.21.15.153:0/1057266982 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f564c04ea90 con 0x7f564c10d280 2024-03-21T10:33:38.227 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:38.229+0000 7f564a000700 1 -- 172.21.15.153:0/1057266982 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f563c0105d0 con 0x7f564c10d280 2024-03-21T10:33:38.228 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:38.230+0000 7f564a000700 1 --2- 172.21.15.153:0/1057266982 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f563406e460 0x7f5634070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:33:38.228 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:38.230+0000 7f564a000700 1 -- 172.21.15.153:0/1057266982 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f563c08e5b0 con 0x7f564c10d280 2024-03-21T10:33:38.229 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:38.230+0000 7f5651a00700 1 --2- 172.21.15.153:0/1057266982 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f563406e460 0x7f5634070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:33:38.229 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:38.231+0000 7f5651a00700 1 --2- 172.21.15.153:0/1057266982 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f563406e460 0x7f5634070920 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f5640006010 tx=0x7f564001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:33:38.233 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:38.234+0000 7f564a000700 1 -- 172.21.15.153:0/1057266982 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f563c0575d0 con 0x7f564c10d280 2024-03-21T10:33:38.543 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:38.543+0000 7f5654400700 1 -- 172.21.15.153:0/1057266982 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f564c1a11a0 con 0x7f564c10d280 2024-03-21T10:33:38.543 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:38.544+0000 7f564a000700 1 -- 172.21.15.153:0/1057266982 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f563c05ac90 con 0x7f564c10d280 2024-03-21T10:33:38.543 INFO:teuthology.orchestra.run.smithi153.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-03-21T10:33:38.544 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-21T10:33:38.544 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is degraded 2024-03-21T10:33:38.544 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-21T10:33:38.544 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-21T10:33:38.544 INFO:teuthology.orchestra.run.smithi153.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-21T10:33:38.544 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-21T10:33:38.546 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:38.547+0000 7f5654400700 1 -- 172.21.15.153:0/1057266982 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f563406e460 msgr2=0x7f5634070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:38.547 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:38.547+0000 7f5654400700 1 --2- 172.21.15.153:0/1057266982 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f563406e460 0x7f5634070920 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f5640006010 tx=0x7f564001a040 comp rx=0 tx=0).stop 2024-03-21T10:33:38.547 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:38.547+0000 7f5654400700 1 -- 172.21.15.153:0/1057266982 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f564c10d280 msgr2=0x7f564c19d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:33:38.547 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:38.547+0000 7f5654400700 1 --2- 172.21.15.153:0/1057266982 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f564c10d280 0x7f564c19d210 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f563c00d8d0 tx=0x7f563c00dc90 comp rx=0 tx=0).stop 2024-03-21T10:33:38.547 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:38.547+0000 7f5654400700 1 -- 172.21.15.153:0/1057266982 shutdown_connections 2024-03-21T10:33:38.547 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:38.547+0000 7f5654400700 1 --2- 172.21.15.153:0/1057266982 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f564c1071d0 0x7f564c19ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:38.547 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:38.547+0000 7f5654400700 1 --2- 172.21.15.153:0/1057266982 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f563406e460 0x7f5634070920 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:38.547 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:38.547+0000 7f5654400700 1 --2- 172.21.15.153:0/1057266982 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f564c10d280 0x7f564c19d210 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:33:38.547 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:38.547+0000 7f5654400700 1 -- 172.21.15.153:0/1057266982 >> 172.21.15.153:0/1057266982 conn(0x7f564c0769e0 msgr2=0x7f564c078320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:33:38.547 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:38.547+0000 7f5654400700 1 -- 172.21.15.153:0/1057266982 shutdown_connections 2024-03-21T10:33:38.547 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:33:38.547+0000 7f5654400700 1 -- 172.21.15.153:0/1057266982 wait complete. 2024-03-21T10:33:38.804 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:33:38 smithi153 ceph-mon[113733]: pgmap v653: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:33:38.804 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:33:38 smithi153 ceph-mon[113733]: from='client.35198 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:33:38.804 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:33:38 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/1057266982' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:33:39.199 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:33:38 smithi174 ceph-mon[88004]: pgmap v653: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:33:39.199 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:33:38 smithi174 ceph-mon[88004]: from='client.35198 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:33:39.199 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:33:38 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/1057266982' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:33:41.054 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:33:40 smithi153 ceph-mon[113733]: pgmap v654: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:33:41.198 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:33:40 smithi174 ceph-mon[88004]: pgmap v654: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:33:43.054 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:33:42 smithi153 ceph-mon[113733]: pgmap v655: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:33:43.198 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:33:42 smithi174 ceph-mon[88004]: pgmap v655: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:33:45.054 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:33:44 smithi153 ceph-mon[113733]: pgmap v656: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:33:45.198 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:33:44 smithi174 ceph-mon[88004]: pgmap v656: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:33:47.053 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:33:46 smithi153 ceph-mon[113733]: pgmap v657: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:33:47.054 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:33:46 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:33:47.198 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:33:46 smithi174 ceph-mon[88004]: pgmap v657: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:33:47.199 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:33:46 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:33:49.053 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:33:48 smithi153 ceph-mon[113733]: pgmap v658: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:33:49.198 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:33:48 smithi174 ceph-mon[88004]: pgmap v658: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:33:51.053 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:33:50 smithi153 ceph-mon[113733]: pgmap v659: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:33:51.198 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:33:50 smithi174 ceph-mon[88004]: pgmap v659: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:33:53.053 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:33:52 smithi153 ceph-mon[113733]: pgmap v660: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:33:53.198 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:33:52 smithi174 ceph-mon[88004]: pgmap v660: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:33:55.053 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:33:54 smithi153 ceph-mon[113733]: pgmap v661: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:33:55.198 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:33:54 smithi174 ceph-mon[88004]: pgmap v661: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:33:57.054 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:33:56 smithi153 ceph-mon[113733]: pgmap v662: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:33:57.198 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:33:56 smithi174 ceph-mon[88004]: pgmap v662: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:33:59.053 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:33:58 smithi153 ceph-mon[113733]: pgmap v663: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:33:59.198 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:33:58 smithi174 ceph-mon[88004]: pgmap v663: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:34:01.053 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:34:00 smithi153 ceph-mon[113733]: pgmap v664: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:34:01.198 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:34:00 smithi174 ceph-mon[88004]: pgmap v664: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:34:02.053 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:34:01 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:34:02.198 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:34:01 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:34:03.053 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:34:02 smithi153 ceph-mon[113733]: pgmap v665: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:34:03.198 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:34:02 smithi174 ceph-mon[88004]: pgmap v665: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:34:05.198 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:34:04 smithi174 ceph-mon[88004]: pgmap v666: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:34:05.303 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:34:04 smithi153 ceph-mon[113733]: pgmap v666: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:34:07.199 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:34:06 smithi174 ceph-mon[88004]: pgmap v667: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:34:07.278 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:34:06 smithi153 ceph-mon[113733]: pgmap v667: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:34:08.716 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:08.717+0000 7feab5a00700 1 -- 172.21.15.153:0/315942532 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feab0100da0 msgr2=0x7feab0101180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:08.717 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:08.717+0000 7feab5a00700 1 --2- 172.21.15.153:0/315942532 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feab0100da0 0x7feab0101180 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7feaa0009910 tx=0x7feaa0009c20 comp rx=0 tx=0).stop 2024-03-21T10:34:08.717 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:08.717+0000 7feab5a00700 1 -- 172.21.15.153:0/315942532 shutdown_connections 2024-03-21T10:34:08.717 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:08.717+0000 7feab5a00700 1 --2- 172.21.15.153:0/315942532 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feab0101750 0x7feab01057a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:08.717 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:08.717+0000 7feab5a00700 1 --2- 172.21.15.153:0/315942532 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feab0100da0 0x7feab0101180 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:08.717 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:08.717+0000 7feab5a00700 1 -- 172.21.15.153:0/315942532 >> 172.21.15.153:0/315942532 conn(0x7feab00fc630 msgr2=0x7feab00fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:08.717 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:08.717+0000 7feab5a00700 1 -- 172.21.15.153:0/315942532 shutdown_connections 2024-03-21T10:34:08.717 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:08.717+0000 7feab5a00700 1 -- 172.21.15.153:0/315942532 wait complete. 2024-03-21T10:34:08.718 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:08.718+0000 7feab5a00700 1 Processor -- start 2024-03-21T10:34:08.718 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:08.719+0000 7feab5a00700 1 -- start start 2024-03-21T10:34:08.718 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:08.719+0000 7feab5a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feab0100da0 0x7feab0198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:08.718 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:08.719+0000 7feab5a00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feab0101750 0x7feab0198ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:08.718 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:08.719+0000 7feab5a00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feab0199560 con 0x7feab0100da0 2024-03-21T10:34:08.719 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:08.719+0000 7feab5a00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feab019d370 con 0x7feab0101750 2024-03-21T10:34:08.719 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:08.719+0000 7feaaea00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feab0100da0 0x7feab0198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:08.719 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:08.719+0000 7feaaea00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feab0100da0 0x7feab0198990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:59550/0 (socket says 172.21.15.153:59550) 2024-03-21T10:34:08.719 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:08.719+0000 7feaaea00700 1 -- 172.21.15.153:0/1903801588 learned_addr learned my addr 172.21.15.153:0/1903801588 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:34:08.719 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:08.720+0000 7feaae000700 1 --2- 172.21.15.153:0/1903801588 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feab0101750 0x7feab0198ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:08.719 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:08.720+0000 7feaaea00700 1 -- 172.21.15.153:0/1903801588 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feab0101750 msgr2=0x7feab0198ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:08.719 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:08.720+0000 7feaaea00700 1 --2- 172.21.15.153:0/1903801588 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feab0101750 0x7feab0198ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:08.719 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:08.720+0000 7feaaea00700 1 -- 172.21.15.153:0/1903801588 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaa00095f0 con 0x7feab0100da0 2024-03-21T10:34:08.720 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:08.720+0000 7feaaea00700 1 --2- 172.21.15.153:0/1903801588 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feab0100da0 0x7feab0198990 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7feaa0006010 tx=0x7feaa000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:08.720 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:08.720+0000 7feaa7400700 1 -- 172.21.15.153:0/1903801588 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feaa001d070 con 0x7feab0100da0 2024-03-21T10:34:08.720 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:08.720+0000 7feaa7400700 1 -- 172.21.15.153:0/1903801588 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7feaa000f460 con 0x7feab0100da0 2024-03-21T10:34:08.720 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:08.720+0000 7feaa7400700 1 -- 172.21.15.153:0/1903801588 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feaa0017630 con 0x7feab0100da0 2024-03-21T10:34:08.720 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:08.720+0000 7feab5a00700 1 -- 172.21.15.153:0/1903801588 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feab019d5f0 con 0x7feab0100da0 2024-03-21T10:34:08.720 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:08.721+0000 7feab5a00700 1 -- 172.21.15.153:0/1903801588 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feab019dae0 con 0x7feab0100da0 2024-03-21T10:34:08.720 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:08.722+0000 7feab5a00700 1 -- 172.21.15.153:0/1903801588 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feab0104f20 con 0x7feab0100da0 2024-03-21T10:34:08.721 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:08.723+0000 7feaa7400700 1 -- 172.21.15.153:0/1903801588 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7feaa000f5d0 con 0x7feab0100da0 2024-03-21T10:34:08.722 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:08.724+0000 7feaa7400700 1 --2- 172.21.15.153:0/1903801588 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fea9406e460 0x7fea94070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:08.722 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:08.724+0000 7feaa7400700 1 -- 172.21.15.153:0/1903801588 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7feaa008f9a0 con 0x7feab0100da0 2024-03-21T10:34:08.723 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:08.724+0000 7feaae000700 1 --2- 172.21.15.153:0/1903801588 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fea9406e460 0x7fea94070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:08.723 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:08.725+0000 7feaae000700 1 --2- 172.21.15.153:0/1903801588 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fea9406e460 0x7fea94070920 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fea9c005fd0 tx=0x7fea9c005f60 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:08.727 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:08.729+0000 7feaa7400700 1 -- 172.21.15.153:0/1903801588 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7feaa005c570 con 0x7feab0100da0 2024-03-21T10:34:08.972 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:08.973+0000 7feab5a00700 1 -- 172.21.15.153:0/1903801588 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7feab019de10 con 0x7fea9406e460 2024-03-21T10:34:08.973 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:34:08 smithi153 ceph-mon[113733]: pgmap v668: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:34:08.976 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:08.977+0000 7feaa7400700 1 -- 172.21.15.153:0/1903801588 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7feab019de10 con 0x7fea9406e460 2024-03-21T10:34:08.979 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:08.980+0000 7feab5a00700 1 -- 172.21.15.153:0/1903801588 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fea9406e460 msgr2=0x7fea94070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:08.979 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:08.980+0000 7feab5a00700 1 --2- 172.21.15.153:0/1903801588 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fea9406e460 0x7fea94070920 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fea9c005fd0 tx=0x7fea9c005f60 comp rx=0 tx=0).stop 2024-03-21T10:34:08.979 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:08.980+0000 7feab5a00700 1 -- 172.21.15.153:0/1903801588 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feab0100da0 msgr2=0x7feab0198990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:08.979 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:08.980+0000 7feab5a00700 1 --2- 172.21.15.153:0/1903801588 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feab0100da0 0x7feab0198990 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7feaa0006010 tx=0x7feaa000be20 comp rx=0 tx=0).stop 2024-03-21T10:34:08.979 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:08.980+0000 7feab5a00700 1 -- 172.21.15.153:0/1903801588 shutdown_connections 2024-03-21T10:34:08.980 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:08.980+0000 7feab5a00700 1 --2- 172.21.15.153:0/1903801588 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fea9406e460 0x7fea94070920 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:08.980 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:08.980+0000 7feab5a00700 1 --2- 172.21.15.153:0/1903801588 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feab0100da0 0x7feab0198990 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:08.980 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:08.980+0000 7feab5a00700 1 --2- 172.21.15.153:0/1903801588 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feab0101750 0x7feab0198ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:08.980 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:08.980+0000 7feab5a00700 1 -- 172.21.15.153:0/1903801588 >> 172.21.15.153:0/1903801588 conn(0x7feab00fc630 msgr2=0x7feab00fea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:08.980 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:08.981+0000 7feab5a00700 1 -- 172.21.15.153:0/1903801588 shutdown_connections 2024-03-21T10:34:08.980 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:08.981+0000 7feab5a00700 1 -- 172.21.15.153:0/1903801588 wait complete. 2024-03-21T10:34:08.995 INFO:teuthology.orchestra.run.smithi153.stdout:true 2024-03-21T10:34:09.137 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.138+0000 7fcb17200700 1 -- 172.21.15.153:0/2158490653 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb181009c0 msgr2=0x7fcb18100e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:09.138 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.138+0000 7fcb17200700 1 --2- 172.21.15.153:0/2158490653 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb181009c0 0x7fcb18100e40 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fcb00009910 tx=0x7fcb00009c20 comp rx=0 tx=0).stop 2024-03-21T10:34:09.138 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.138+0000 7fcb17200700 1 -- 172.21.15.153:0/2158490653 shutdown_connections 2024-03-21T10:34:09.138 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.138+0000 7fcb17200700 1 --2- 172.21.15.153:0/2158490653 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb181009c0 0x7fcb18100e40 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:09.138 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.138+0000 7fcb17200700 1 --2- 172.21.15.153:0/2158490653 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcb181069c0 0x7fcb18106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:09.138 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.138+0000 7fcb17200700 1 -- 172.21.15.153:0/2158490653 >> 172.21.15.153:0/2158490653 conn(0x7fcb180fc4f0 msgr2=0x7fcb180fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:09.138 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.138+0000 7fcb17200700 1 -- 172.21.15.153:0/2158490653 shutdown_connections 2024-03-21T10:34:09.138 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.139+0000 7fcb17200700 1 -- 172.21.15.153:0/2158490653 wait complete. 2024-03-21T10:34:09.139 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.139+0000 7fcb17200700 1 Processor -- start 2024-03-21T10:34:09.140 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.140+0000 7fcb17200700 1 -- start start 2024-03-21T10:34:09.140 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.140+0000 7fcb17200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb181009c0 0x7fcb18073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:09.140 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.140+0000 7fcb17200700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcb181069c0 0x7fcb180735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:09.140 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.140+0000 7fcb17200700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb18073b10 con 0x7fcb181009c0 2024-03-21T10:34:09.140 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.140+0000 7fcb17200700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb18073c50 con 0x7fcb181069c0 2024-03-21T10:34:09.140 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.140+0000 7fcb15e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb181009c0 0x7fcb18073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:09.140 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.140+0000 7fcb15e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb181009c0 0x7fcb18073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:59570/0 (socket says 172.21.15.153:59570) 2024-03-21T10:34:09.140 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.140+0000 7fcb15e00700 1 -- 172.21.15.153:0/2549687171 learned_addr learned my addr 172.21.15.153:0/2549687171 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:34:09.140 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.141+0000 7fcb0fe00700 1 --2- 172.21.15.153:0/2549687171 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcb181069c0 0x7fcb180735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:09.140 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.141+0000 7fcb15e00700 1 -- 172.21.15.153:0/2549687171 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcb181069c0 msgr2=0x7fcb180735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:09.140 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.141+0000 7fcb15e00700 1 --2- 172.21.15.153:0/2549687171 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcb181069c0 0x7fcb180735d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:09.140 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.141+0000 7fcb15e00700 1 -- 172.21.15.153:0/2549687171 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb000095f0 con 0x7fcb181009c0 2024-03-21T10:34:09.140 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.141+0000 7fcb15e00700 1 --2- 172.21.15.153:0/2549687171 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb181009c0 0x7fcb18073090 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fcb0800d8d0 tx=0x7fcb0800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:09.141 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.141+0000 7fcb0ea00700 1 -- 172.21.15.153:0/2549687171 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcb08009940 con 0x7fcb181009c0 2024-03-21T10:34:09.141 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.141+0000 7fcb0ea00700 1 -- 172.21.15.153:0/2549687171 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcb08010460 con 0x7fcb181009c0 2024-03-21T10:34:09.141 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.141+0000 7fcb0ea00700 1 -- 172.21.15.153:0/2549687171 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcb0800f5f0 con 0x7fcb181009c0 2024-03-21T10:34:09.141 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.141+0000 7fcb17200700 1 -- 172.21.15.153:0/2549687171 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb181a2770 con 0x7fcb181009c0 2024-03-21T10:34:09.142 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.142+0000 7fcb17200700 1 -- 172.21.15.153:0/2549687171 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb181a2b20 con 0x7fcb181009c0 2024-03-21T10:34:09.142 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.143+0000 7fcb17200700 1 -- 172.21.15.153:0/2549687171 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb18108f50 con 0x7fcb181009c0 2024-03-21T10:34:09.142 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.144+0000 7fcb0ea00700 1 -- 172.21.15.153:0/2549687171 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fcb080105d0 con 0x7fcb181009c0 2024-03-21T10:34:09.144 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.145+0000 7fcb0ea00700 1 --2- 172.21.15.153:0/2549687171 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fcafc06e460 0x7fcafc070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:09.144 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.145+0000 7fcb0ea00700 1 -- 172.21.15.153:0/2549687171 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fcb0808e5b0 con 0x7fcb181009c0 2024-03-21T10:34:09.144 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.146+0000 7fcb0fe00700 1 --2- 172.21.15.153:0/2549687171 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fcafc06e460 0x7fcafc070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:09.145 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.146+0000 7fcb0fe00700 1 --2- 172.21.15.153:0/2549687171 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fcafc06e460 0x7fcafc070920 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fcb18074880 tx=0x7fcb0001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:09.149 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.150+0000 7fcb0ea00700 1 -- 172.21.15.153:0/2549687171 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fcb080575d0 con 0x7fcb181009c0 2024-03-21T10:34:09.198 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:34:08 smithi174 ceph-mon[88004]: pgmap v668: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:34:09.383 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.384+0000 7fcb17200700 1 -- 172.21.15.153:0/2549687171 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcb1802cd80 con 0x7fcafc06e460 2024-03-21T10:34:09.385 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.387+0000 7fcb0ea00700 1 -- 172.21.15.153:0/2549687171 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fcb1802cd80 con 0x7fcafc06e460 2024-03-21T10:34:09.388 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.389+0000 7fcb17200700 1 -- 172.21.15.153:0/2549687171 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fcafc06e460 msgr2=0x7fcafc070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:09.388 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.389+0000 7fcb17200700 1 --2- 172.21.15.153:0/2549687171 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fcafc06e460 0x7fcafc070920 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fcb18074880 tx=0x7fcb0001a040 comp rx=0 tx=0).stop 2024-03-21T10:34:09.388 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.389+0000 7fcb17200700 1 -- 172.21.15.153:0/2549687171 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb181009c0 msgr2=0x7fcb18073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:09.388 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.389+0000 7fcb17200700 1 --2- 172.21.15.153:0/2549687171 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb181009c0 0x7fcb18073090 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fcb0800d8d0 tx=0x7fcb0800dc90 comp rx=0 tx=0).stop 2024-03-21T10:34:09.389 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.390+0000 7fcb17200700 1 -- 172.21.15.153:0/2549687171 shutdown_connections 2024-03-21T10:34:09.389 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.390+0000 7fcb17200700 1 --2- 172.21.15.153:0/2549687171 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fcafc06e460 0x7fcafc070920 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:09.389 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.390+0000 7fcb17200700 1 --2- 172.21.15.153:0/2549687171 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb181009c0 0x7fcb18073090 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:09.389 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.390+0000 7fcb17200700 1 --2- 172.21.15.153:0/2549687171 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcb181069c0 0x7fcb180735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:09.389 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.390+0000 7fcb17200700 1 -- 172.21.15.153:0/2549687171 >> 172.21.15.153:0/2549687171 conn(0x7fcb180fc4f0 msgr2=0x7fcb180fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:09.389 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.390+0000 7fcb17200700 1 -- 172.21.15.153:0/2549687171 shutdown_connections 2024-03-21T10:34:09.389 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.390+0000 7fcb17200700 1 -- 172.21.15.153:0/2549687171 wait complete. 2024-03-21T10:34:09.532 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.533+0000 7f71fd000700 1 -- 172.21.15.153:0/2277517506 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f71f81071d0 msgr2=0x7f71f8107650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:09.532 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.533+0000 7f71fd000700 1 --2- 172.21.15.153:0/2277517506 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f71f81071d0 0x7f71f8107650 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f71e4009910 tx=0x7f71e4009c20 comp rx=0 tx=0).stop 2024-03-21T10:34:09.533 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.533+0000 7f71fd000700 1 -- 172.21.15.153:0/2277517506 shutdown_connections 2024-03-21T10:34:09.533 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.533+0000 7f71fd000700 1 --2- 172.21.15.153:0/2277517506 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f71f81071d0 0x7f71f8107650 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:09.533 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.533+0000 7f71fd000700 1 --2- 172.21.15.153:0/2277517506 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f71f810d280 0x7f71f810d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:09.533 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.533+0000 7f71fd000700 1 -- 172.21.15.153:0/2277517506 >> 172.21.15.153:0/2277517506 conn(0x7f71f80769e0 msgr2=0x7f71f8078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:09.533 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.533+0000 7f71fd000700 1 -- 172.21.15.153:0/2277517506 shutdown_connections 2024-03-21T10:34:09.533 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.533+0000 7f71fd000700 1 -- 172.21.15.153:0/2277517506 wait complete. 2024-03-21T10:34:09.535 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.534+0000 7f71fd000700 1 Processor -- start 2024-03-21T10:34:09.535 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.534+0000 7f71fd000700 1 -- start start 2024-03-21T10:34:09.535 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.535+0000 7f71fd000700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f71f81071d0 0x7f71f819ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:09.535 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.535+0000 7f71fd000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f71f810d280 0x7f71f819d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:09.535 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.535+0000 7f71fd000700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71f819da30 con 0x7f71f810d280 2024-03-21T10:34:09.535 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.535+0000 7f71fd000700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71f81a17c0 con 0x7f71f81071d0 2024-03-21T10:34:09.535 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.535+0000 7f71f5400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f71f810d280 0x7f71f819d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:09.535 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.535+0000 7f71f5400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f71f810d280 0x7f71f819d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:59582/0 (socket says 172.21.15.153:59582) 2024-03-21T10:34:09.535 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.535+0000 7f71f5400700 1 -- 172.21.15.153:0/2989998590 learned_addr learned my addr 172.21.15.153:0/2989998590 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:34:09.536 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.535+0000 7f71f5e00700 1 --2- 172.21.15.153:0/2989998590 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f71f81071d0 0x7f71f819ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:09.536 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.536+0000 7f71f5400700 1 -- 172.21.15.153:0/2989998590 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f71f81071d0 msgr2=0x7f71f819ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:09.536 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.536+0000 7f71f5400700 1 --2- 172.21.15.153:0/2989998590 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f71f81071d0 0x7f71f819ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:09.536 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.536+0000 7f71f5400700 1 -- 172.21.15.153:0/2989998590 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71e40095f0 con 0x7f71f810d280 2024-03-21T10:34:09.536 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.536+0000 7f71f5400700 1 --2- 172.21.15.153:0/2989998590 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f71f810d280 0x7f71f819d350 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f71e400b5c0 tx=0x7f71e400be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:09.536 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.536+0000 7f71eea00700 1 -- 172.21.15.153:0/2989998590 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f71e401d070 con 0x7f71f810d280 2024-03-21T10:34:09.536 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.536+0000 7f71eea00700 1 -- 172.21.15.153:0/2989998590 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f71e400f460 con 0x7f71f810d280 2024-03-21T10:34:09.536 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.536+0000 7f71fd000700 1 -- 172.21.15.153:0/2989998590 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71f81a1a40 con 0x7f71f810d280 2024-03-21T10:34:09.536 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.536+0000 7f71eea00700 1 -- 172.21.15.153:0/2989998590 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f71e4017610 con 0x7f71f810d280 2024-03-21T10:34:09.536 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.536+0000 7f71fd000700 1 -- 172.21.15.153:0/2989998590 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71f81a1fb0 con 0x7f71f810d280 2024-03-21T10:34:09.537 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.537+0000 7f71fd000700 1 -- 172.21.15.153:0/2989998590 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71f810f710 con 0x7f71f810d280 2024-03-21T10:34:09.537 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.539+0000 7f71eea00700 1 -- 172.21.15.153:0/2989998590 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f71e400f5d0 con 0x7f71f810d280 2024-03-21T10:34:09.538 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.539+0000 7f71eea00700 1 --2- 172.21.15.153:0/2989998590 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f71dc06e460 0x7f71dc070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:09.538 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.539+0000 7f71eea00700 1 -- 172.21.15.153:0/2989998590 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f71e408fb80 con 0x7f71f810d280 2024-03-21T10:34:09.538 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.540+0000 7f71f5e00700 1 --2- 172.21.15.153:0/2989998590 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f71dc06e460 0x7f71dc070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:09.539 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.540+0000 7f71f5e00700 1 --2- 172.21.15.153:0/2989998590 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f71dc06e460 0x7f71dc070920 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f71e8006fd0 tx=0x7f71e8008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:09.543 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.544+0000 7f71eea00700 1 -- 172.21.15.153:0/2989998590 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f71e4058ba0 con 0x7f71f810d280 2024-03-21T10:34:09.759 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.760+0000 7f71fd000700 1 -- 172.21.15.153:0/2989998590 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f71f81a2300 con 0x7f71dc06e460 2024-03-21T10:34:09.792 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.793+0000 7f71eea00700 1 -- 172.21.15.153:0/2989998590 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f71f81a2300 con 0x7f71dc06e460 2024-03-21T10:34:09.793 INFO:teuthology.orchestra.run.smithi153.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-21T10:34:09.793 INFO:teuthology.orchestra.run.smithi153.stdout:alertmanager.smithi153 smithi153 *:9093,9094 running (22m) 14m ago 36m 14.7M - 0.25.0 c8568f914cd2 9ed3f2200b5f 2024-03-21T10:34:09.793 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi153 smithi153 running (37m) 14m ago 37m 9424k - 18.2.1 5be31c24972a 5b61994396a6 2024-03-21T10:34:09.793 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi174 smithi174 running (35m) 14m ago 35m 9244k - 18.2.1 5be31c24972a 4f7a5fc3ac28 2024-03-21T10:34:09.793 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi153 smithi153 running (19m) 14m ago 36m 6656k - 19.0.0-2231-g2edfee68 237e43c7a65f 32c21104d5b0 2024-03-21T10:34:09.794 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi174 smithi174 running (19m) 14m ago 35m 6660k - 19.0.0-2231-g2edfee68 237e43c7a65f a04e48967ab6 2024-03-21T10:34:09.794 INFO:teuthology.orchestra.run.smithi153.stdout:grafana.smithi153 smithi153 *:3000 running (22m) 14m ago 36m 73.6M - 9.4.12 2bacad6d85d8 ee24026ab786 2024-03-21T10:34:09.794 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.iuyluj smithi153 running (14m) 14m ago 29m 10.9M - 19.0.0-2231-g2edfee68 237e43c7a65f 87aff8a5de70 2024-03-21T10:34:09.794 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.zipckr smithi153 running (14m) 14m ago 29m 11.8M - 19.0.0-2231-g2edfee68 237e43c7a65f ccc262bd5dd3 2024-03-21T10:34:09.794 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.nhozag smithi174 running (29m) 14m ago 29m 12.0M - 18.2.1 5be31c24972a 2fee4000308f 2024-03-21T10:34:09.794 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.yyimst smithi174 running (14m) 14m ago 29m 10.7M - 19.0.0-2231-g2edfee68 237e43c7a65f fc1962f9351e 2024-03-21T10:34:09.794 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi153.tluxni smithi153 *:8443,9283,8765 running (25m) 14m ago 38m 509M - 19.0.0-2231-g2edfee68 237e43c7a65f 7ffee0e55382 2024-03-21T10:34:09.794 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi174.qpcwat smithi174 *:8443,9283,8765 running (24m) 14m ago 35m 452M - 19.0.0-2231-g2edfee68 237e43c7a65f 748968f216ab 2024-03-21T10:34:09.794 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi153 smithi153 running (20m) 14m ago 38m 55.0M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f dd8038303dc3 2024-03-21T10:34:09.794 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi174 smithi174 running (20m) 14m ago 35m 44.2M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f 29d78a788f2c 2024-03-21T10:34:09.794 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi153 smithi153 *:9100 running (24m) 14m ago 36m 7889k - 1.5.0 0da6a335fe13 bb835edc0e0b 2024-03-21T10:34:09.794 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi174 smithi174 *:9100 running (24m) 14m ago 35m 8102k - 1.5.0 0da6a335fe13 7c4ecad790fa 2024-03-21T10:34:09.794 INFO:teuthology.orchestra.run.smithi153.stdout:osd.0 smithi153 running (18m) 14m ago 34m 116M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 507900fd4791 2024-03-21T10:34:09.795 INFO:teuthology.orchestra.run.smithi153.stdout:osd.1 smithi153 running (17m) 14m ago 33m 89.2M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 0d646452e9c1 2024-03-21T10:34:09.795 INFO:teuthology.orchestra.run.smithi153.stdout:osd.2 smithi153 running (17m) 14m ago 32m 87.4M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 1326f2afdfcd 2024-03-21T10:34:09.795 INFO:teuthology.orchestra.run.smithi153.stdout:osd.3 smithi174 running (16m) 14m ago 32m 83.6M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 70aafac71ad5 2024-03-21T10:34:09.795 INFO:teuthology.orchestra.run.smithi153.stdout:osd.4 smithi174 running (16m) 14m ago 31m 92.2M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 0d746ce6e79d 2024-03-21T10:34:09.795 INFO:teuthology.orchestra.run.smithi153.stdout:osd.5 smithi174 running (15m) 14m ago 31m 86.2M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 53e5f35b8cac 2024-03-21T10:34:09.795 INFO:teuthology.orchestra.run.smithi153.stdout:prometheus.smithi153 smithi153 *:9095 running (23m) 14m ago 35m 73.5M - 2.43.0 a07b618ecd1d 7d95561e50c2 2024-03-21T10:34:09.796 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.797+0000 7f71fd000700 1 -- 172.21.15.153:0/2989998590 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f71dc06e460 msgr2=0x7f71dc070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:09.796 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.797+0000 7f71fd000700 1 --2- 172.21.15.153:0/2989998590 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f71dc06e460 0x7f71dc070920 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f71e8006fd0 tx=0x7f71e8008040 comp rx=0 tx=0).stop 2024-03-21T10:34:09.796 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.797+0000 7f71fd000700 1 -- 172.21.15.153:0/2989998590 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f71f810d280 msgr2=0x7f71f819d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:09.796 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.797+0000 7f71fd000700 1 --2- 172.21.15.153:0/2989998590 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f71f810d280 0x7f71f819d350 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f71e400b5c0 tx=0x7f71e400be00 comp rx=0 tx=0).stop 2024-03-21T10:34:09.796 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.797+0000 7f71fd000700 1 -- 172.21.15.153:0/2989998590 shutdown_connections 2024-03-21T10:34:09.796 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.797+0000 7f71fd000700 1 --2- 172.21.15.153:0/2989998590 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f71f81071d0 0x7f71f819ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:09.796 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.797+0000 7f71fd000700 1 --2- 172.21.15.153:0/2989998590 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f71dc06e460 0x7f71dc070920 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:09.796 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.797+0000 7f71fd000700 1 --2- 172.21.15.153:0/2989998590 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f71f810d280 0x7f71f819d350 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:09.796 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.797+0000 7f71fd000700 1 -- 172.21.15.153:0/2989998590 >> 172.21.15.153:0/2989998590 conn(0x7f71f80769e0 msgr2=0x7f71f80784d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:09.796 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.797+0000 7f71fd000700 1 -- 172.21.15.153:0/2989998590 shutdown_connections 2024-03-21T10:34:09.796 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.797+0000 7f71fd000700 1 -- 172.21.15.153:0/2989998590 wait complete. 2024-03-21T10:34:09.937 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.938+0000 7efbfd000700 1 -- 172.21.15.153:0/353013859 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efbf8102fd0 msgr2=0x7efbf81033b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:09.937 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.938+0000 7efbfd000700 1 --2- 172.21.15.153:0/353013859 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efbf8102fd0 0x7efbf81033b0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7efbe8009910 tx=0x7efbe8009c20 comp rx=0 tx=0).stop 2024-03-21T10:34:09.938 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.938+0000 7efbfd000700 1 -- 172.21.15.153:0/353013859 shutdown_connections 2024-03-21T10:34:09.938 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.938+0000 7efbfd000700 1 --2- 172.21.15.153:0/353013859 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efbf8103980 0x7efbf81079d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:09.938 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.938+0000 7efbfd000700 1 --2- 172.21.15.153:0/353013859 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efbf8102fd0 0x7efbf81033b0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:09.938 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.938+0000 7efbfd000700 1 -- 172.21.15.153:0/353013859 >> 172.21.15.153:0/353013859 conn(0x7efbf80fe840 msgr2=0x7efbf8100c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:09.938 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.938+0000 7efbfd000700 1 -- 172.21.15.153:0/353013859 shutdown_connections 2024-03-21T10:34:09.938 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.938+0000 7efbfd000700 1 -- 172.21.15.153:0/353013859 wait complete. 2024-03-21T10:34:09.940 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.939+0000 7efbfd000700 1 Processor -- start 2024-03-21T10:34:09.940 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.939+0000 7efbfd000700 1 -- start start 2024-03-21T10:34:09.940 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.940+0000 7efbfd000700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efbf8102fd0 0x7efbf819e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:09.941 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.940+0000 7efbfd000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efbf8103980 0x7efbf819ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:09.941 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.940+0000 7efbfd000700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efbf819f320 con 0x7efbf8103980 2024-03-21T10:34:09.941 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.940+0000 7efbfd000700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efbf8198860 con 0x7efbf8102fd0 2024-03-21T10:34:09.941 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.940+0000 7efbed400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efbf8103980 0x7efbf819ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:09.941 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.940+0000 7efbed400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efbf8103980 0x7efbf819ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:59592/0 (socket says 172.21.15.153:59592) 2024-03-21T10:34:09.941 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.940+0000 7efbed400700 1 -- 172.21.15.153:0/2438199469 learned_addr learned my addr 172.21.15.153:0/2438199469 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:34:09.941 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.940+0000 7efbf5e00700 1 --2- 172.21.15.153:0/2438199469 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efbf8102fd0 0x7efbf819e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:09.941 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.941+0000 7efbed400700 1 -- 172.21.15.153:0/2438199469 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efbf8102fd0 msgr2=0x7efbf819e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:09.941 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.941+0000 7efbed400700 1 --2- 172.21.15.153:0/2438199469 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efbf8102fd0 0x7efbf819e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:09.941 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.941+0000 7efbed400700 1 -- 172.21.15.153:0/2438199469 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efbe80095f0 con 0x7efbf8103980 2024-03-21T10:34:09.941 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.941+0000 7efbf5e00700 1 --2- 172.21.15.153:0/2438199469 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efbf8102fd0 0x7efbf819e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:34:09.941 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.941+0000 7efbed400700 1 --2- 172.21.15.153:0/2438199469 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efbf8103980 0x7efbf819ed20 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7efbe000c960 tx=0x7efbe000cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:09.941 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.941+0000 7efbef400700 1 -- 172.21.15.153:0/2438199469 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efbe0007a70 con 0x7efbf8103980 2024-03-21T10:34:09.942 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.941+0000 7efbef400700 1 -- 172.21.15.153:0/2438199469 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efbe0007bd0 con 0x7efbf8103980 2024-03-21T10:34:09.942 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.941+0000 7efbef400700 1 -- 172.21.15.153:0/2438199469 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efbe00186c0 con 0x7efbf8103980 2024-03-21T10:34:09.942 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.941+0000 7efbfd000700 1 -- 172.21.15.153:0/2438199469 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efbf8198b40 con 0x7efbf8103980 2024-03-21T10:34:09.942 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.941+0000 7efbfd000700 1 -- 172.21.15.153:0/2438199469 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efbf8199090 con 0x7efbf8103980 2024-03-21T10:34:09.942 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.942+0000 7efbfd000700 1 -- 172.21.15.153:0/2438199469 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efbf810b370 con 0x7efbf8103980 2024-03-21T10:34:09.942 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.943+0000 7efbef400700 1 -- 172.21.15.153:0/2438199469 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7efbe000faa0 con 0x7efbf8103980 2024-03-21T10:34:09.943 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.944+0000 7efbef400700 1 --2- 172.21.15.153:0/2438199469 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7efbdc06e460 0x7efbdc070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:09.943 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.944+0000 7efbef400700 1 -- 172.21.15.153:0/2438199469 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7efbe008e5d0 con 0x7efbf8103980 2024-03-21T10:34:09.943 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.945+0000 7efbf5e00700 1 --2- 172.21.15.153:0/2438199469 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7efbdc06e460 0x7efbdc070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:09.944 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.945+0000 7efbf5e00700 1 --2- 172.21.15.153:0/2438199469 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7efbdc06e460 0x7efbdc070920 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7efbe8006010 tx=0x7efbe800b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:09.948 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:09.949+0000 7efbef400700 1 -- 172.21.15.153:0/2438199469 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7efbe005ae90 con 0x7efbf8103980 2024-03-21T10:34:10.054 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:34:09 smithi153 ceph-mon[113733]: from='client.35206 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:34:10.054 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:34:09 smithi153 ceph-mon[113733]: from='client.35210 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:34:10.198 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:34:09 smithi174 ceph-mon[88004]: from='client.35206 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:34:10.199 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:34:09 smithi174 ceph-mon[88004]: from='client.35210 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:34:10.256 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:10.257+0000 7efbfd000700 1 -- 172.21.15.153:0/2438199469 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7efbf804ea90 con 0x7efbf8103980 2024-03-21T10:34:10.257 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:10.258+0000 7efbef400700 1 -- 172.21.15.153:0/2438199469 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+805 (secure 0 0 0) 0x7efbe005acb0 con 0x7efbf8103980 2024-03-21T10:34:10.257 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:34:10.257 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": { 2024-03-21T10:34:10.257 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:34:10.257 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:34:10.258 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": { 2024-03-21T10:34:10.258 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:34:10.258 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:34:10.258 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": { 2024-03-21T10:34:10.258 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 6 2024-03-21T10:34:10.258 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:34:10.258 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": { 2024-03-21T10:34:10.258 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-03-21T10:34:10.258 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 3 2024-03-21T10:34:10.258 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:34:10.258 INFO:teuthology.orchestra.run.smithi153.stdout: "overall": { 2024-03-21T10:34:10.258 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-03-21T10:34:10.258 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 13 2024-03-21T10:34:10.259 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-03-21T10:34:10.259 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:34:10.261 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:10.262+0000 7efbfd000700 1 -- 172.21.15.153:0/2438199469 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7efbdc06e460 msgr2=0x7efbdc070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:10.261 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:10.262+0000 7efbfd000700 1 --2- 172.21.15.153:0/2438199469 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7efbdc06e460 0x7efbdc070920 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7efbe8006010 tx=0x7efbe800b540 comp rx=0 tx=0).stop 2024-03-21T10:34:10.261 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:10.262+0000 7efbfd000700 1 -- 172.21.15.153:0/2438199469 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efbf8103980 msgr2=0x7efbf819ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:10.261 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:10.262+0000 7efbfd000700 1 --2- 172.21.15.153:0/2438199469 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efbf8103980 0x7efbf819ed20 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7efbe000c960 tx=0x7efbe000cd20 comp rx=0 tx=0).stop 2024-03-21T10:34:10.261 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:10.262+0000 7efbfd000700 1 -- 172.21.15.153:0/2438199469 shutdown_connections 2024-03-21T10:34:10.261 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:10.262+0000 7efbfd000700 1 --2- 172.21.15.153:0/2438199469 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efbf8102fd0 0x7efbf819e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:10.261 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:10.262+0000 7efbfd000700 1 --2- 172.21.15.153:0/2438199469 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7efbdc06e460 0x7efbdc070920 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:10.261 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:10.262+0000 7efbfd000700 1 --2- 172.21.15.153:0/2438199469 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efbf8103980 0x7efbf819ed20 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:10.261 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:10.262+0000 7efbfd000700 1 -- 172.21.15.153:0/2438199469 >> 172.21.15.153:0/2438199469 conn(0x7efbf80fe840 msgr2=0x7efbf80ffe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:10.261 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:10.262+0000 7efbfd000700 1 -- 172.21.15.153:0/2438199469 shutdown_connections 2024-03-21T10:34:10.261 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:10.262+0000 7efbfd000700 1 -- 172.21.15.153:0/2438199469 wait complete. 2024-03-21T10:34:10.404 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:10.405+0000 7f605ce00700 1 -- 172.21.15.153:0/188305037 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6058102620 msgr2=0x7f605810ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:10.404 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:10.405+0000 7f605ce00700 1 --2- 172.21.15.153:0/188305037 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6058102620 0x7f605810ab10 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f6044009910 tx=0x7f6044009c20 comp rx=0 tx=0).stop 2024-03-21T10:34:10.404 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:10.405+0000 7f605ce00700 1 -- 172.21.15.153:0/188305037 shutdown_connections 2024-03-21T10:34:10.405 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:10.405+0000 7f605ce00700 1 --2- 172.21.15.153:0/188305037 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6058102620 0x7f605810ab10 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:10.405 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:10.405+0000 7f605ce00700 1 --2- 172.21.15.153:0/188305037 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6058101d00 0x7f60581020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:10.405 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:10.405+0000 7f605ce00700 1 -- 172.21.15.153:0/188305037 >> 172.21.15.153:0/188305037 conn(0x7f60580fb570 msgr2=0x7f60580fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:10.405 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:10.406+0000 7f605ce00700 1 -- 172.21.15.153:0/188305037 shutdown_connections 2024-03-21T10:34:10.405 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:10.406+0000 7f605ce00700 1 -- 172.21.15.153:0/188305037 wait complete. 2024-03-21T10:34:10.406 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:10.407+0000 7f605ce00700 1 Processor -- start 2024-03-21T10:34:10.406 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:10.407+0000 7f605ce00700 1 -- start start 2024-03-21T10:34:10.406 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:10.407+0000 7f605ce00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6058101d00 0x7f605819c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:10.406 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:10.407+0000 7f605ce00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6058102620 0x7f605819cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:10.406 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:10.407+0000 7f605ce00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f605819d150 con 0x7f6058102620 2024-03-21T10:34:10.406 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:10.407+0000 7f605ce00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6058196600 con 0x7f6058101d00 2024-03-21T10:34:10.406 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:10.407+0000 7f6055400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6058102620 0x7f605819cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:10.406 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:10.407+0000 7f6055400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6058102620 0x7f605819cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:59618/0 (socket says 172.21.15.153:59618) 2024-03-21T10:34:10.406 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:10.407+0000 7f6055400700 1 -- 172.21.15.153:0/1234498474 learned_addr learned my addr 172.21.15.153:0/1234498474 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:34:10.407 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:10.408+0000 7f6055400700 1 -- 172.21.15.153:0/1234498474 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6058101d00 msgr2=0x7f605819c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:10.407 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:10.408+0000 7f6055400700 1 --2- 172.21.15.153:0/1234498474 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6058101d00 0x7f605819c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:10.408 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:10.408+0000 7f6055400700 1 -- 172.21.15.153:0/1234498474 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60440095f0 con 0x7f6058102620 2024-03-21T10:34:10.408 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:10.408+0000 7f6055400700 1 --2- 172.21.15.153:0/1234498474 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6058102620 0x7f605819cac0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f604400bda0 tx=0x7f604400be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:10.408 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:10.408+0000 7f604ea00700 1 -- 172.21.15.153:0/1234498474 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f604401d070 con 0x7f6058102620 2024-03-21T10:34:10.408 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:10.408+0000 7f604ea00700 1 -- 172.21.15.153:0/1234498474 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f604400f460 con 0x7f6058102620 2024-03-21T10:34:10.408 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:10.408+0000 7f605ce00700 1 -- 172.21.15.153:0/1234498474 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6058196880 con 0x7f6058102620 2024-03-21T10:34:10.408 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:10.408+0000 7f604ea00700 1 -- 172.21.15.153:0/1234498474 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6044017630 con 0x7f6058102620 2024-03-21T10:34:10.408 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:10.408+0000 7f605ce00700 1 -- 172.21.15.153:0/1234498474 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6058196d70 con 0x7f6058102620 2024-03-21T10:34:10.408 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:10.409+0000 7f605ce00700 1 -- 172.21.15.153:0/1234498474 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6058108210 con 0x7f6058102620 2024-03-21T10:34:10.409 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:10.410+0000 7f604ea00700 1 -- 172.21.15.153:0/1234498474 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f604400fa60 con 0x7f6058102620 2024-03-21T10:34:10.410 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:10.411+0000 7f604ea00700 1 --2- 172.21.15.153:0/1234498474 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f603c06e460 0x7f603c070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:10.410 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:10.411+0000 7f604ea00700 1 -- 172.21.15.153:0/1234498474 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f604408fa30 con 0x7f6058102620 2024-03-21T10:34:10.410 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:10.412+0000 7f6055e00700 1 --2- 172.21.15.153:0/1234498474 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f603c06e460 0x7f603c070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:10.411 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:10.412+0000 7f6055e00700 1 --2- 172.21.15.153:0/1234498474 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f603c06e460 0x7f603c070920 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f604800a990 tx=0x7f6048005c80 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:10.415 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:10.416+0000 7f604ea00700 1 -- 172.21.15.153:0/1234498474 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f604405c580 con 0x7f6058102620 2024-03-21T10:34:10.668 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:10.670+0000 7f605ce00700 1 -- 172.21.15.153:0/1234498474 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f605804ea90 con 0x7f6058102620 2024-03-21T10:34:10.669 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:10.671+0000 7f604ea00700 1 -- 172.21.15.153:0/1234498474 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1876 (secure 0 0 0) 0x7f604405c110 con 0x7f6058102620 2024-03-21T10:34:10.670 INFO:teuthology.orchestra.run.smithi153.stdout:e30 2024-03-21T10:34:10.670 INFO:teuthology.orchestra.run.smithi153.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-21T10:34:10.670 INFO:teuthology.orchestra.run.smithi153.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:34:10.671 INFO:teuthology.orchestra.run.smithi153.stdout:legacy client fscid: 1 2024-03-21T10:34:10.671 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:34:10.671 INFO:teuthology.orchestra.run.smithi153.stdout:Filesystem 'cephfs' (1) 2024-03-21T10:34:10.671 INFO:teuthology.orchestra.run.smithi153.stdout:fs_name cephfs 2024-03-21T10:34:10.671 INFO:teuthology.orchestra.run.smithi153.stdout:epoch 23 2024-03-21T10:34:10.671 INFO:teuthology.orchestra.run.smithi153.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-21T10:34:10.671 INFO:teuthology.orchestra.run.smithi153.stdout:created 2024-03-21T10:04:27.086576+0000 2024-03-21T10:34:10.671 INFO:teuthology.orchestra.run.smithi153.stdout:modified 2024-03-21T10:19:05.022092+0000 2024-03-21T10:34:10.671 INFO:teuthology.orchestra.run.smithi153.stdout:tableserver 0 2024-03-21T10:34:10.671 INFO:teuthology.orchestra.run.smithi153.stdout:root 0 2024-03-21T10:34:10.671 INFO:teuthology.orchestra.run.smithi153.stdout:session_timeout 60 2024-03-21T10:34:10.671 INFO:teuthology.orchestra.run.smithi153.stdout:session_autoclose 300 2024-03-21T10:34:10.671 INFO:teuthology.orchestra.run.smithi153.stdout:max_file_size 1099511627776 2024-03-21T10:34:10.671 INFO:teuthology.orchestra.run.smithi153.stdout:max_xattr_size 65536 2024-03-21T10:34:10.672 INFO:teuthology.orchestra.run.smithi153.stdout:required_client_features {} 2024-03-21T10:34:10.672 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure 0 2024-03-21T10:34:10.672 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure_osd_epoch 94 2024-03-21T10:34:10.672 INFO:teuthology.orchestra.run.smithi153.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:34:10.672 INFO:teuthology.orchestra.run.smithi153.stdout:max_mds 2 2024-03-21T10:34:10.672 INFO:teuthology.orchestra.run.smithi153.stdout:in 0,1 2024-03-21T10:34:10.672 INFO:teuthology.orchestra.run.smithi153.stdout:up {} 2024-03-21T10:34:10.672 INFO:teuthology.orchestra.run.smithi153.stdout:failed 0,1 2024-03-21T10:34:10.672 INFO:teuthology.orchestra.run.smithi153.stdout:damaged 2024-03-21T10:34:10.672 INFO:teuthology.orchestra.run.smithi153.stdout:stopped 2024-03-21T10:34:10.672 INFO:teuthology.orchestra.run.smithi153.stdout:data_pools [3] 2024-03-21T10:34:10.672 INFO:teuthology.orchestra.run.smithi153.stdout:metadata_pool 2 2024-03-21T10:34:10.672 INFO:teuthology.orchestra.run.smithi153.stdout:inline_data enabled 2024-03-21T10:34:10.672 INFO:teuthology.orchestra.run.smithi153.stdout:balancer 2024-03-21T10:34:10.673 INFO:teuthology.orchestra.run.smithi153.stdout:bal_rank_mask -1 2024-03-21T10:34:10.673 INFO:teuthology.orchestra.run.smithi153.stdout:standby_count_wanted 1 2024-03-21T10:34:10.673 INFO:teuthology.orchestra.run.smithi153.stdout:qdb_cluster leader: 0 members: 2024-03-21T10:34:10.673 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:34:10.673 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:34:10.673 INFO:teuthology.orchestra.run.smithi153.stdout:Standby daemons: 2024-03-21T10:34:10.673 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:34:10.673 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.zipckr{-1:34442} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6826/1103846295,v1:172.21.15.153:6827/1103846295] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:34:10.673 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.iuyluj{-1:34474} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6828/1738630832,v1:172.21.15.153:6829/1738630832] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:34:10.673 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.nhozag{-1:44269} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.174:6826/3967223227,v1:172.21.15.174:6827/3967223227] compat {c=[1],r=[1],i=[7ff]}] 2024-03-21T10:34:10.673 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.yyimst{-1:44313} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.174:6824/4127786797,v1:172.21.15.174:6825/4127786797] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:34:10.674 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:10.673+0000 7f605ce00700 1 -- 172.21.15.153:0/1234498474 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f603c06e460 msgr2=0x7f603c070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:10.674 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:10.673+0000 7f605ce00700 1 --2- 172.21.15.153:0/1234498474 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f603c06e460 0x7f603c070920 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f604800a990 tx=0x7f6048005c80 comp rx=0 tx=0).stop 2024-03-21T10:34:10.674 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:10.673+0000 7f605ce00700 1 -- 172.21.15.153:0/1234498474 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6058102620 msgr2=0x7f605819cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:10.675 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:10.673+0000 7f605ce00700 1 --2- 172.21.15.153:0/1234498474 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6058102620 0x7f605819cac0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f604400bda0 tx=0x7f604400be80 comp rx=0 tx=0).stop 2024-03-21T10:34:10.675 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:10.674+0000 7f605ce00700 1 -- 172.21.15.153:0/1234498474 shutdown_connections 2024-03-21T10:34:10.675 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:10.674+0000 7f605ce00700 1 --2- 172.21.15.153:0/1234498474 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6058101d00 0x7f605819c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:10.675 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:10.674+0000 7f605ce00700 1 --2- 172.21.15.153:0/1234498474 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f603c06e460 0x7f603c070920 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:10.675 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:10.674+0000 7f605ce00700 1 --2- 172.21.15.153:0/1234498474 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6058102620 0x7f605819cac0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:10.675 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:10.674+0000 7f605ce00700 1 -- 172.21.15.153:0/1234498474 >> 172.21.15.153:0/1234498474 conn(0x7f60580fb570 msgr2=0x7f6058105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:10.675 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:10.674+0000 7f605ce00700 1 -- 172.21.15.153:0/1234498474 shutdown_connections 2024-03-21T10:34:10.675 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:10.674+0000 7f605ce00700 1 -- 172.21.15.153:0/1234498474 wait complete. 2024-03-21T10:34:10.675 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 30 2024-03-21T10:34:10.818 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:10.818+0000 7fa4d7e00700 1 -- 172.21.15.153:0/948017815 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa4d81037c0 msgr2=0x7fa4d8107810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:10.818 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:10.818+0000 7fa4d7e00700 1 --2- 172.21.15.153:0/948017815 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa4d81037c0 0x7fa4d8107810 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fa4c4009910 tx=0x7fa4c4009c20 comp rx=0 tx=0).stop 2024-03-21T10:34:10.819 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:10.818+0000 7fa4d7e00700 1 -- 172.21.15.153:0/948017815 shutdown_connections 2024-03-21T10:34:10.819 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:10.818+0000 7fa4d7e00700 1 --2- 172.21.15.153:0/948017815 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa4d81037c0 0x7fa4d8107810 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:10.819 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:10.818+0000 7fa4d7e00700 1 --2- 172.21.15.153:0/948017815 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa4d8102e10 0x7fa4d81031f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:10.819 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:10.818+0000 7fa4d7e00700 1 -- 172.21.15.153:0/948017815 >> 172.21.15.153:0/948017815 conn(0x7fa4d80fe6c0 msgr2=0x7fa4d8100ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:10.819 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:10.818+0000 7fa4d7e00700 1 -- 172.21.15.153:0/948017815 shutdown_connections 2024-03-21T10:34:10.819 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:10.818+0000 7fa4d7e00700 1 -- 172.21.15.153:0/948017815 wait complete. 2024-03-21T10:34:10.819 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:10.819+0000 7fa4d7e00700 1 Processor -- start 2024-03-21T10:34:10.820 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:10.819+0000 7fa4d7e00700 1 -- start start 2024-03-21T10:34:10.824 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:10.820+0000 7fa4d7e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa4d8102e10 0x7fa4d819e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:10.824 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:10.820+0000 7fa4d7e00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa4d81037c0 0x7fa4d819eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:10.824 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:10.820+0000 7fa4d7e00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4d819f150 con 0x7fa4d8102e10 2024-03-21T10:34:10.824 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:10.820+0000 7fa4d7e00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4d8198600 con 0x7fa4d81037c0 2024-03-21T10:34:10.824 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:10.820+0000 7fa4d6a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa4d8102e10 0x7fa4d819e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:10.824 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:10.820+0000 7fa4d6a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa4d8102e10 0x7fa4d819e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:59636/0 (socket says 172.21.15.153:59636) 2024-03-21T10:34:10.824 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:10.820+0000 7fa4d6a00700 1 -- 172.21.15.153:0/4233188270 learned_addr learned my addr 172.21.15.153:0/4233188270 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:34:10.824 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:10.820+0000 7fa4d6000700 1 --2- 172.21.15.153:0/4233188270 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa4d81037c0 0x7fa4d819eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:10.825 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:10.821+0000 7fa4d6a00700 1 -- 172.21.15.153:0/4233188270 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa4d81037c0 msgr2=0x7fa4d819eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:10.825 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:10.821+0000 7fa4d6a00700 1 --2- 172.21.15.153:0/4233188270 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa4d81037c0 0x7fa4d819eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:10.825 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:10.821+0000 7fa4d6a00700 1 -- 172.21.15.153:0/4233188270 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4c40095f0 con 0x7fa4d8102e10 2024-03-21T10:34:10.825 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:10.821+0000 7fa4d6000700 1 --2- 172.21.15.153:0/4233188270 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa4d81037c0 0x7fa4d819eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:34:10.825 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:10.821+0000 7fa4d6a00700 1 --2- 172.21.15.153:0/4233188270 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa4d8102e10 0x7fa4d819e580 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fa4c800d8d0 tx=0x7fa4c800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:10.825 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:10.821+0000 7fa4dca00700 1 -- 172.21.15.153:0/4233188270 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa4c8009940 con 0x7fa4d8102e10 2024-03-21T10:34:10.825 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:10.821+0000 7fa4dca00700 1 -- 172.21.15.153:0/4233188270 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa4c8010460 con 0x7fa4d8102e10 2024-03-21T10:34:10.825 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:10.821+0000 7fa4d7e00700 1 -- 172.21.15.153:0/4233188270 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4d81988e0 con 0x7fa4d8102e10 2024-03-21T10:34:10.825 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:10.821+0000 7fa4dca00700 1 -- 172.21.15.153:0/4233188270 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa4c800f5f0 con 0x7fa4d8102e10 2024-03-21T10:34:10.825 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:10.821+0000 7fa4d7e00700 1 -- 172.21.15.153:0/4233188270 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4d8198e30 con 0x7fa4d8102e10 2024-03-21T10:34:10.825 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:10.822+0000 7fa4d7e00700 1 -- 172.21.15.153:0/4233188270 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4d810b210 con 0x7fa4d8102e10 2024-03-21T10:34:10.825 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:10.824+0000 7fa4dca00700 1 -- 172.21.15.153:0/4233188270 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fa4c80105d0 con 0x7fa4d8102e10 2024-03-21T10:34:10.825 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:10.824+0000 7fa4dca00700 1 --2- 172.21.15.153:0/4233188270 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa4bc06e460 0x7fa4bc070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:10.825 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:10.825+0000 7fa4dca00700 1 -- 172.21.15.153:0/4233188270 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fa4c808e480 con 0x7fa4d8102e10 2024-03-21T10:34:10.826 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:10.825+0000 7fa4d6000700 1 --2- 172.21.15.153:0/4233188270 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa4bc06e460 0x7fa4bc070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:10.826 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:10.825+0000 7fa4d6000700 1 --2- 172.21.15.153:0/4233188270 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa4bc06e460 0x7fa4bc070920 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fa4c4006010 tx=0x7fa4c4005960 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:10.827 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:10.829+0000 7fa4dca00700 1 -- 172.21.15.153:0/4233188270 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa4c805b050 con 0x7fa4d8102e10 2024-03-21T10:34:11.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:34:10 smithi153 ceph-mon[113733]: pgmap v669: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:34:11.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:34:10 smithi153 ceph-mon[113733]: from='client.35214 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:34:11.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:34:10 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/2438199469' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:34:11.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:34:10 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/1234498474' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:34:11.067 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:11.068+0000 7fa4d7e00700 1 -- 172.21.15.153:0/4233188270 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa4d8199170 con 0x7fa4bc06e460 2024-03-21T10:34:11.071 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:11.073+0000 7fa4dca00700 1 -- 172.21.15.153:0/4233188270 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fa4d8199170 con 0x7fa4bc06e460 2024-03-21T10:34:11.072 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:34:11.072 INFO:teuthology.orchestra.run.smithi153.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232", 2024-03-21T10:34:11.072 INFO:teuthology.orchestra.run.smithi153.stdout: "in_progress": true, 2024-03-21T10:34:11.072 INFO:teuthology.orchestra.run.smithi153.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-21T10:34:11.072 INFO:teuthology.orchestra.run.smithi153.stdout: "services_complete": [ 2024-03-21T10:34:11.072 INFO:teuthology.orchestra.run.smithi153.stdout: "crash", 2024-03-21T10:34:11.072 INFO:teuthology.orchestra.run.smithi153.stdout: "mon", 2024-03-21T10:34:11.072 INFO:teuthology.orchestra.run.smithi153.stdout: "osd", 2024-03-21T10:34:11.072 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr" 2024-03-21T10:34:11.072 INFO:teuthology.orchestra.run.smithi153.stdout: ], 2024-03-21T10:34:11.072 INFO:teuthology.orchestra.run.smithi153.stdout: "progress": "15/23 daemons upgraded", 2024-03-21T10:34:11.072 INFO:teuthology.orchestra.run.smithi153.stdout: "message": "Currently upgrading mds daemons", 2024-03-21T10:34:11.072 INFO:teuthology.orchestra.run.smithi153.stdout: "is_paused": false 2024-03-21T10:34:11.072 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:34:11.075 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:11.075+0000 7fa4d7e00700 1 -- 172.21.15.153:0/4233188270 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa4bc06e460 msgr2=0x7fa4bc070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:11.075 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:11.075+0000 7fa4d7e00700 1 --2- 172.21.15.153:0/4233188270 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa4bc06e460 0x7fa4bc070920 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fa4c4006010 tx=0x7fa4c4005960 comp rx=0 tx=0).stop 2024-03-21T10:34:11.075 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:11.076+0000 7fa4d7e00700 1 -- 172.21.15.153:0/4233188270 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa4d8102e10 msgr2=0x7fa4d819e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:11.075 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:11.076+0000 7fa4d7e00700 1 --2- 172.21.15.153:0/4233188270 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa4d8102e10 0x7fa4d819e580 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fa4c800d8d0 tx=0x7fa4c800dc90 comp rx=0 tx=0).stop 2024-03-21T10:34:11.075 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:11.076+0000 7fa4d7e00700 1 -- 172.21.15.153:0/4233188270 shutdown_connections 2024-03-21T10:34:11.075 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:11.076+0000 7fa4d7e00700 1 --2- 172.21.15.153:0/4233188270 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa4bc06e460 0x7fa4bc070920 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:11.075 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:11.076+0000 7fa4d7e00700 1 --2- 172.21.15.153:0/4233188270 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa4d8102e10 0x7fa4d819e580 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:11.075 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:11.076+0000 7fa4d7e00700 1 --2- 172.21.15.153:0/4233188270 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa4d81037c0 0x7fa4d819eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:11.075 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:11.076+0000 7fa4d7e00700 1 -- 172.21.15.153:0/4233188270 >> 172.21.15.153:0/4233188270 conn(0x7fa4d80fe6c0 msgr2=0x7fa4d8100aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:11.075 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:11.076+0000 7fa4d7e00700 1 -- 172.21.15.153:0/4233188270 shutdown_connections 2024-03-21T10:34:11.075 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:11.076+0000 7fa4d7e00700 1 -- 172.21.15.153:0/4233188270 wait complete. 2024-03-21T10:34:11.198 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:34:10 smithi174 ceph-mon[88004]: pgmap v669: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:34:11.199 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:34:10 smithi174 ceph-mon[88004]: from='client.35214 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:34:11.199 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:34:10 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/2438199469' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:34:11.199 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:34:10 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/1234498474' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:34:11.222 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:11.223+0000 7fd2b9e00700 1 -- 172.21.15.153:0/1054524235 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd2b40731c0 msgr2=0x7fd2b40735a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:11.222 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:11.223+0000 7fd2b9e00700 1 --2- 172.21.15.153:0/1054524235 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd2b40731c0 0x7fd2b40735a0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fd2a8009910 tx=0x7fd2a8009c20 comp rx=0 tx=0).stop 2024-03-21T10:34:11.222 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:11.223+0000 7fd2b9e00700 1 -- 172.21.15.153:0/1054524235 shutdown_connections 2024-03-21T10:34:11.222 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:11.223+0000 7fd2b9e00700 1 --2- 172.21.15.153:0/1054524235 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd2b4073ae0 0x7fd2b410cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:11.222 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:11.223+0000 7fd2b9e00700 1 --2- 172.21.15.153:0/1054524235 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd2b40731c0 0x7fd2b40735a0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:11.222 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:11.223+0000 7fd2b9e00700 1 -- 172.21.15.153:0/1054524235 >> 172.21.15.153:0/1054524235 conn(0x7fd2b40fc440 msgr2=0x7fd2b40fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:11.222 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:11.223+0000 7fd2b9e00700 1 -- 172.21.15.153:0/1054524235 shutdown_connections 2024-03-21T10:34:11.222 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:11.223+0000 7fd2b9e00700 1 -- 172.21.15.153:0/1054524235 wait complete. 2024-03-21T10:34:11.223 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:11.224+0000 7fd2b9e00700 1 Processor -- start 2024-03-21T10:34:11.223 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:11.224+0000 7fd2b9e00700 1 -- start start 2024-03-21T10:34:11.223 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:11.225+0000 7fd2b9e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd2b40731c0 0x7fd2b419e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:11.224 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:11.225+0000 7fd2b9e00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd2b4073ae0 0x7fd2b419eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:11.224 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:11.225+0000 7fd2b9e00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b419f0a0 con 0x7fd2b40731c0 2024-03-21T10:34:11.224 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:11.225+0000 7fd2b9e00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b41985e0 con 0x7fd2b4073ae0 2024-03-21T10:34:11.224 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:11.225+0000 7fd2b8a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd2b40731c0 0x7fd2b419e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:11.224 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:11.225+0000 7fd2b8a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd2b40731c0 0x7fd2b419e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:59652/0 (socket says 172.21.15.153:59652) 2024-03-21T10:34:11.224 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:11.225+0000 7fd2b8a00700 1 -- 172.21.15.153:0/1752376734 learned_addr learned my addr 172.21.15.153:0/1752376734 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:34:11.225 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:11.225+0000 7fd2b3e00700 1 --2- 172.21.15.153:0/1752376734 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd2b4073ae0 0x7fd2b419eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:11.225 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:11.225+0000 7fd2b8a00700 1 -- 172.21.15.153:0/1752376734 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd2b4073ae0 msgr2=0x7fd2b419eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:11.225 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:11.225+0000 7fd2b8a00700 1 --2- 172.21.15.153:0/1752376734 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd2b4073ae0 0x7fd2b419eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:11.225 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:11.225+0000 7fd2b8a00700 1 -- 172.21.15.153:0/1752376734 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2a80095f0 con 0x7fd2b40731c0 2024-03-21T10:34:11.225 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:11.225+0000 7fd2b3e00700 1 --2- 172.21.15.153:0/1752376734 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd2b4073ae0 0x7fd2b419eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:34:11.225 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:11.226+0000 7fd2b8a00700 1 --2- 172.21.15.153:0/1752376734 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd2b40731c0 0x7fd2b419e560 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fd2a80092e0 tx=0x7fd2a800be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:11.225 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:11.226+0000 7fd2b1600700 1 -- 172.21.15.153:0/1752376734 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd2a801d070 con 0x7fd2b40731c0 2024-03-21T10:34:11.225 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:11.226+0000 7fd2b1600700 1 -- 172.21.15.153:0/1752376734 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd2a800f460 con 0x7fd2b40731c0 2024-03-21T10:34:11.226 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:11.226+0000 7fd2b9e00700 1 -- 172.21.15.153:0/1752376734 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2b4198860 con 0x7fd2b40731c0 2024-03-21T10:34:11.226 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:11.226+0000 7fd2b1600700 1 -- 172.21.15.153:0/1752376734 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd2a8017630 con 0x7fd2b40731c0 2024-03-21T10:34:11.226 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:11.226+0000 7fd2b9e00700 1 -- 172.21.15.153:0/1752376734 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2b4198d50 con 0x7fd2b40731c0 2024-03-21T10:34:11.226 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:11.227+0000 7fd2b9e00700 1 -- 172.21.15.153:0/1752376734 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2b410a370 con 0x7fd2b40731c0 2024-03-21T10:34:11.226 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:11.228+0000 7fd2b1600700 1 -- 172.21.15.153:0/1752376734 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fd2a800f5d0 con 0x7fd2b40731c0 2024-03-21T10:34:11.228 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:11.229+0000 7fd2b1600700 1 --2- 172.21.15.153:0/1752376734 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd29c06e460 0x7fd29c070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:11.228 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:11.229+0000 7fd2b1600700 1 -- 172.21.15.153:0/1752376734 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fd2a808fb70 con 0x7fd2b40731c0 2024-03-21T10:34:11.228 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:11.229+0000 7fd2b3e00700 1 --2- 172.21.15.153:0/1752376734 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd29c06e460 0x7fd29c070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:11.228 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:11.230+0000 7fd2b3e00700 1 --2- 172.21.15.153:0/1752376734 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd29c06e460 0x7fd29c070920 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fd2a0005fd0 tx=0x7fd2a0005e20 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:11.233 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:11.234+0000 7fd2b1600700 1 -- 172.21.15.153:0/1752376734 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd2a805c6c0 con 0x7fd2b40731c0 2024-03-21T10:34:11.522 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:11.523+0000 7fd2b9e00700 1 -- 172.21.15.153:0/1752376734 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fd2b4199030 con 0x7fd2b40731c0 2024-03-21T10:34:11.522 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:11.524+0000 7fd2b1600700 1 -- 172.21.15.153:0/1752376734 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7fd2a805c250 con 0x7fd2b40731c0 2024-03-21T10:34:11.522 INFO:teuthology.orchestra.run.smithi153.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-03-21T10:34:11.522 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-21T10:34:11.523 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is degraded 2024-03-21T10:34:11.523 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-21T10:34:11.523 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-21T10:34:11.523 INFO:teuthology.orchestra.run.smithi153.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-21T10:34:11.523 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-21T10:34:11.524 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:11.526+0000 7fd2b9e00700 1 -- 172.21.15.153:0/1752376734 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd29c06e460 msgr2=0x7fd29c070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:11.525 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:11.526+0000 7fd2b9e00700 1 --2- 172.21.15.153:0/1752376734 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd29c06e460 0x7fd29c070920 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fd2a0005fd0 tx=0x7fd2a0005e20 comp rx=0 tx=0).stop 2024-03-21T10:34:11.525 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:11.526+0000 7fd2b9e00700 1 -- 172.21.15.153:0/1752376734 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd2b40731c0 msgr2=0x7fd2b419e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:11.525 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:11.526+0000 7fd2b9e00700 1 --2- 172.21.15.153:0/1752376734 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd2b40731c0 0x7fd2b419e560 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fd2a80092e0 tx=0x7fd2a800be00 comp rx=0 tx=0).stop 2024-03-21T10:34:11.525 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:11.526+0000 7fd2b9e00700 1 -- 172.21.15.153:0/1752376734 shutdown_connections 2024-03-21T10:34:11.526 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:11.526+0000 7fd2b9e00700 1 --2- 172.21.15.153:0/1752376734 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd29c06e460 0x7fd29c070920 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:11.526 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:11.526+0000 7fd2b9e00700 1 --2- 172.21.15.153:0/1752376734 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd2b40731c0 0x7fd2b419e560 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:11.526 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:11.526+0000 7fd2b9e00700 1 --2- 172.21.15.153:0/1752376734 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd2b4073ae0 0x7fd2b419eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:11.526 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:11.526+0000 7fd2b9e00700 1 -- 172.21.15.153:0/1752376734 >> 172.21.15.153:0/1752376734 conn(0x7fd2b40fc440 msgr2=0x7fd2b41074b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:11.526 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:11.527+0000 7fd2b9e00700 1 -- 172.21.15.153:0/1752376734 shutdown_connections 2024-03-21T10:34:11.526 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:11.527+0000 7fd2b9e00700 1 -- 172.21.15.153:0/1752376734 wait complete. 2024-03-21T10:34:12.198 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:34:11 smithi174 ceph-mon[88004]: from='client.35226 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:34:12.199 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:34:11 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/1752376734' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:34:12.303 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:34:11 smithi153 ceph-mon[113733]: from='client.35226 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:34:12.303 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:34:11 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/1752376734' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:34:13.053 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:34:12 smithi153 ceph-mon[113733]: pgmap v670: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:34:13.198 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:34:12 smithi174 ceph-mon[88004]: pgmap v670: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:34:15.198 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:34:14 smithi174 ceph-mon[88004]: pgmap v671: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:34:15.303 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:34:14 smithi153 ceph-mon[113733]: pgmap v671: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:34:17.198 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:34:16 smithi174 ceph-mon[88004]: pgmap v672: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:34:17.199 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:34:16 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:34:17.303 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:34:16 smithi153 ceph-mon[113733]: pgmap v672: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:34:17.303 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:34:16 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:34:19.198 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:34:18 smithi174 ceph-mon[88004]: pgmap v673: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:34:19.303 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:34:18 smithi153 ceph-mon[113733]: pgmap v673: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:34:21.198 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:34:20 smithi174 ceph-mon[88004]: pgmap v674: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:34:21.303 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:34:20 smithi153 ceph-mon[113733]: pgmap v674: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:34:23.053 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:34:22 smithi153 ceph-mon[113733]: pgmap v675: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:34:23.198 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:34:22 smithi174 ceph-mon[88004]: pgmap v675: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:34:25.198 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:34:24 smithi174 ceph-mon[88004]: pgmap v676: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:34:25.303 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:34:24 smithi153 ceph-mon[113733]: pgmap v676: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:34:27.198 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:34:26 smithi174 ceph-mon[88004]: pgmap v677: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:34:27.303 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:34:26 smithi153 ceph-mon[113733]: pgmap v677: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:34:29.198 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:34:28 smithi174 ceph-mon[88004]: pgmap v678: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:34:29.303 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:34:28 smithi153 ceph-mon[113733]: pgmap v678: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:34:31.198 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:34:30 smithi174 ceph-mon[88004]: pgmap v679: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:34:31.303 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:34:30 smithi153 ceph-mon[113733]: pgmap v679: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:34:32.199 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:34:31 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:34:32.303 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:34:31 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:34:33.198 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:34:32 smithi174 ceph-mon[88004]: pgmap v680: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:34:33.303 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:34:32 smithi153 ceph-mon[113733]: pgmap v680: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:34:35.198 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:34:34 smithi174 ceph-mon[88004]: pgmap v681: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:34:35.303 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:34:34 smithi153 ceph-mon[113733]: pgmap v681: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:34:37.198 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:34:36 smithi174 ceph-mon[88004]: pgmap v682: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:34:37.303 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:34:36 smithi153 ceph-mon[113733]: pgmap v682: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:34:39.198 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:34:38 smithi174 ceph-mon[88004]: pgmap v683: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:34:39.304 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:34:38 smithi153 ceph-mon[113733]: pgmap v683: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:34:41.198 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:34:40 smithi174 ceph-mon[88004]: pgmap v684: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:34:41.303 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:34:40 smithi153 ceph-mon[113733]: pgmap v684: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:34:41.698 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:41.699+0000 7f4723e00700 1 -- 172.21.15.153:0/3160410022 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4724106a40 msgr2=0x7f4724106e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:41.698 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:41.699+0000 7f4723e00700 1 --2- 172.21.15.153:0/3160410022 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4724106a40 0x7f4724106e20 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f4714009910 tx=0x7f4714009c20 comp rx=0 tx=0).stop 2024-03-21T10:34:41.698 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:41.699+0000 7f4723e00700 1 -- 172.21.15.153:0/3160410022 shutdown_connections 2024-03-21T10:34:41.698 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:41.699+0000 7f4723e00700 1 --2- 172.21.15.153:0/3160410022 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4724068410 0x7f4724068890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:41.698 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:41.699+0000 7f4723e00700 1 --2- 172.21.15.153:0/3160410022 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4724106a40 0x7f4724106e20 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:41.698 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:41.699+0000 7f4723e00700 1 -- 172.21.15.153:0/3160410022 >> 172.21.15.153:0/3160410022 conn(0x7f4724075330 msgr2=0x7f4724075740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:41.698 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:41.699+0000 7f4723e00700 1 -- 172.21.15.153:0/3160410022 shutdown_connections 2024-03-21T10:34:41.698 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:41.699+0000 7f4723e00700 1 -- 172.21.15.153:0/3160410022 wait complete. 2024-03-21T10:34:41.704 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:41.700+0000 7f4723e00700 1 Processor -- start 2024-03-21T10:34:41.704 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:41.700+0000 7f4723e00700 1 -- start start 2024-03-21T10:34:41.704 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:41.701+0000 7f4723e00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4724068410 0x7f472419e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:41.704 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:41.701+0000 7f4723e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4724106a40 0x7f472419eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:41.704 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:41.701+0000 7f4723e00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f472419f1e0 con 0x7f4724106a40 2024-03-21T10:34:41.705 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:41.701+0000 7f4723e00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4724198690 con 0x7f4724068410 2024-03-21T10:34:41.705 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:41.701+0000 7f471a000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4724106a40 0x7f472419eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:41.705 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:41.701+0000 7f471a000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4724106a40 0x7f472419eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:48002/0 (socket says 172.21.15.153:48002) 2024-03-21T10:34:41.705 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:41.701+0000 7f471a000700 1 -- 172.21.15.153:0/828217506 learned_addr learned my addr 172.21.15.153:0/828217506 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:34:41.705 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:41.701+0000 7f4722a00700 1 --2- 172.21.15.153:0/828217506 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4724068410 0x7f472419e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:41.705 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:41.702+0000 7f471a000700 1 -- 172.21.15.153:0/828217506 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4724068410 msgr2=0x7f472419e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:41.705 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:41.702+0000 7f471a000700 1 --2- 172.21.15.153:0/828217506 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4724068410 0x7f472419e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:41.705 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:41.702+0000 7f471a000700 1 -- 172.21.15.153:0/828217506 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47140095f0 con 0x7f4724106a40 2024-03-21T10:34:41.705 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:41.702+0000 7f471a000700 1 --2- 172.21.15.153:0/828217506 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4724106a40 0x7f472419eb50 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f470c00b4b0 tx=0x7f470c00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:41.705 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:41.702+0000 7f471be00700 1 -- 172.21.15.153:0/828217506 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f470c011800 con 0x7f4724106a40 2024-03-21T10:34:41.705 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:41.702+0000 7f471be00700 1 -- 172.21.15.153:0/828217506 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f470c011e20 con 0x7f4724106a40 2024-03-21T10:34:41.705 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:41.702+0000 7f4723e00700 1 -- 172.21.15.153:0/828217506 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4724198970 con 0x7f4724106a40 2024-03-21T10:34:41.705 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:41.702+0000 7f471be00700 1 -- 172.21.15.153:0/828217506 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f470c00f2e0 con 0x7f4724106a40 2024-03-21T10:34:41.705 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:41.703+0000 7f4723e00700 1 -- 172.21.15.153:0/828217506 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4724198ec0 con 0x7f4724106a40 2024-03-21T10:34:41.706 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:41.704+0000 7f4723e00700 1 -- 172.21.15.153:0/828217506 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f472404f2e0 con 0x7f4724106a40 2024-03-21T10:34:41.706 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:41.705+0000 7f471be00700 1 -- 172.21.15.153:0/828217506 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f470c010380 con 0x7f4724106a40 2024-03-21T10:34:41.706 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:41.706+0000 7f471be00700 1 --2- 172.21.15.153:0/828217506 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f470806e460 0x7f4708070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:41.707 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:41.706+0000 7f471be00700 1 -- 172.21.15.153:0/828217506 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f470c08e0c0 con 0x7f4724106a40 2024-03-21T10:34:41.707 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:41.706+0000 7f4722a00700 1 --2- 172.21.15.153:0/828217506 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f470806e460 0x7f4708070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:41.707 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:41.707+0000 7f4722a00700 1 --2- 172.21.15.153:0/828217506 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f470806e460 0x7f4708070920 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f471400b5c0 tx=0x7f471401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:41.709 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:41.711+0000 7f471be00700 1 -- 172.21.15.153:0/828217506 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f470c05ad40 con 0x7f4724106a40 2024-03-21T10:34:41.950 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:41.951+0000 7f4723e00700 1 -- 172.21.15.153:0/828217506 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f47241993c0 con 0x7f470806e460 2024-03-21T10:34:41.953 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:41.954+0000 7f471be00700 1 -- 172.21.15.153:0/828217506 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f47241993c0 con 0x7f470806e460 2024-03-21T10:34:41.956 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:41.957+0000 7f4723e00700 1 -- 172.21.15.153:0/828217506 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f470806e460 msgr2=0x7f4708070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:41.956 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:41.957+0000 7f4723e00700 1 --2- 172.21.15.153:0/828217506 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f470806e460 0x7f4708070920 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f471400b5c0 tx=0x7f471401a040 comp rx=0 tx=0).stop 2024-03-21T10:34:41.956 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:41.957+0000 7f4723e00700 1 -- 172.21.15.153:0/828217506 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4724106a40 msgr2=0x7f472419eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:41.956 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:41.957+0000 7f4723e00700 1 --2- 172.21.15.153:0/828217506 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4724106a40 0x7f472419eb50 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f470c00b4b0 tx=0x7f470c00b870 comp rx=0 tx=0).stop 2024-03-21T10:34:41.956 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:41.957+0000 7f4723e00700 1 -- 172.21.15.153:0/828217506 shutdown_connections 2024-03-21T10:34:41.956 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:41.957+0000 7f4723e00700 1 --2- 172.21.15.153:0/828217506 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4724068410 0x7f472419e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:41.956 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:41.957+0000 7f4723e00700 1 --2- 172.21.15.153:0/828217506 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f470806e460 0x7f4708070920 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:41.956 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:41.957+0000 7f4723e00700 1 --2- 172.21.15.153:0/828217506 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4724106a40 0x7f472419eb50 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:41.957 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:41.957+0000 7f4723e00700 1 -- 172.21.15.153:0/828217506 >> 172.21.15.153:0/828217506 conn(0x7f4724075330 msgr2=0x7f47240fe950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:41.957 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:41.958+0000 7f4723e00700 1 -- 172.21.15.153:0/828217506 shutdown_connections 2024-03-21T10:34:41.957 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:41.958+0000 7f4723e00700 1 -- 172.21.15.153:0/828217506 wait complete. 2024-03-21T10:34:41.972 INFO:teuthology.orchestra.run.smithi153.stdout:true 2024-03-21T10:34:42.115 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.116+0000 7f7813a00700 1 -- 172.21.15.153:0/2580088852 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f780c1071d0 msgr2=0x7f780c107650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:42.116 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.116+0000 7f7813a00700 1 --2- 172.21.15.153:0/2580088852 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f780c1071d0 0x7f780c107650 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f77f8009910 tx=0x7f77f8009c20 comp rx=0 tx=0).stop 2024-03-21T10:34:42.116 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.116+0000 7f7813a00700 1 -- 172.21.15.153:0/2580088852 shutdown_connections 2024-03-21T10:34:42.116 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.116+0000 7f7813a00700 1 --2- 172.21.15.153:0/2580088852 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f780c1071d0 0x7f780c107650 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:42.116 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.116+0000 7f7813a00700 1 --2- 172.21.15.153:0/2580088852 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f780c10d280 0x7f780c10d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:42.116 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.116+0000 7f7813a00700 1 -- 172.21.15.153:0/2580088852 >> 172.21.15.153:0/2580088852 conn(0x7f780c0769e0 msgr2=0x7f780c078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:42.116 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.116+0000 7f7813a00700 1 -- 172.21.15.153:0/2580088852 shutdown_connections 2024-03-21T10:34:42.116 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.117+0000 7f7813a00700 1 -- 172.21.15.153:0/2580088852 wait complete. 2024-03-21T10:34:42.118 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.118+0000 7f7813a00700 1 Processor -- start 2024-03-21T10:34:42.118 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.118+0000 7f7813a00700 1 -- start start 2024-03-21T10:34:42.118 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.118+0000 7f7813a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f780c1071d0 0x7f780c19cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:42.118 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.118+0000 7f7813a00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f780c10d280 0x7f780c19d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:42.118 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.118+0000 7f7813a00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f780c19d9b0 con 0x7f780c1071d0 2024-03-21T10:34:42.118 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.118+0000 7f7813a00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f780c1a1740 con 0x7f780c10d280 2024-03-21T10:34:42.118 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.118+0000 7f7811000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f780c1071d0 0x7f780c19cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:42.118 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.118+0000 7f7811000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f780c1071d0 0x7f780c19cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:48006/0 (socket says 172.21.15.153:48006) 2024-03-21T10:34:42.118 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.118+0000 7f7811000700 1 -- 172.21.15.153:0/515416548 learned_addr learned my addr 172.21.15.153:0/515416548 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:34:42.118 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.119+0000 7f780be00700 1 --2- 172.21.15.153:0/515416548 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f780c10d280 0x7f780c19d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:42.118 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.119+0000 7f7811000700 1 -- 172.21.15.153:0/515416548 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f780c10d280 msgr2=0x7f780c19d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:42.118 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.119+0000 7f7811000700 1 --2- 172.21.15.153:0/515416548 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f780c10d280 0x7f780c19d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:42.119 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.119+0000 7f7811000700 1 -- 172.21.15.153:0/515416548 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77f80095f0 con 0x7f780c1071d0 2024-03-21T10:34:42.119 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.119+0000 7f7811000700 1 --2- 172.21.15.153:0/515416548 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f780c1071d0 0x7f780c19cd90 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f780000d8d0 tx=0x7f780000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:42.119 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.119+0000 7f7809600700 1 -- 172.21.15.153:0/515416548 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7800009940 con 0x7f780c1071d0 2024-03-21T10:34:42.119 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.120+0000 7f7809600700 1 -- 172.21.15.153:0/515416548 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7800010460 con 0x7f780c1071d0 2024-03-21T10:34:42.119 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.120+0000 7f7813a00700 1 -- 172.21.15.153:0/515416548 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f780c1a1a20 con 0x7f780c1071d0 2024-03-21T10:34:42.120 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.120+0000 7f7809600700 1 -- 172.21.15.153:0/515416548 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f780000f5f0 con 0x7f780c1071d0 2024-03-21T10:34:42.120 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.120+0000 7f7813a00700 1 -- 172.21.15.153:0/515416548 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f780c1a1f70 con 0x7f780c1071d0 2024-03-21T10:34:42.120 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.121+0000 7f7813a00700 1 -- 172.21.15.153:0/515416548 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f780c04ea90 con 0x7f780c1071d0 2024-03-21T10:34:42.120 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.122+0000 7f7809600700 1 -- 172.21.15.153:0/515416548 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f78000105d0 con 0x7f780c1071d0 2024-03-21T10:34:42.122 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.123+0000 7f7809600700 1 --2- 172.21.15.153:0/515416548 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f77f406e460 0x7f77f4070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:42.122 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.123+0000 7f7809600700 1 -- 172.21.15.153:0/515416548 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f780008e480 con 0x7f780c1071d0 2024-03-21T10:34:42.122 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.124+0000 7f780be00700 1 --2- 172.21.15.153:0/515416548 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f77f406e460 0x7f77f4070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:42.123 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.124+0000 7f780be00700 1 --2- 172.21.15.153:0/515416548 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f77f406e460 0x7f77f4070920 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f780c19e3b0 tx=0x7f77f801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:42.126 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.128+0000 7f7809600700 1 -- 172.21.15.153:0/515416548 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7800057520 con 0x7f780c1071d0 2024-03-21T10:34:42.373 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.374+0000 7f7813a00700 1 -- 172.21.15.153:0/515416548 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f780c0008d0 con 0x7f77f406e460 2024-03-21T10:34:42.375 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.377+0000 7f7809600700 1 -- 172.21.15.153:0/515416548 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f780c0008d0 con 0x7f77f406e460 2024-03-21T10:34:42.378 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.379+0000 7f7813a00700 1 -- 172.21.15.153:0/515416548 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f77f406e460 msgr2=0x7f77f4070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:42.378 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.379+0000 7f7813a00700 1 --2- 172.21.15.153:0/515416548 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f77f406e460 0x7f77f4070920 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f780c19e3b0 tx=0x7f77f801a040 comp rx=0 tx=0).stop 2024-03-21T10:34:42.378 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.379+0000 7f7813a00700 1 -- 172.21.15.153:0/515416548 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f780c1071d0 msgr2=0x7f780c19cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:42.378 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.379+0000 7f7813a00700 1 --2- 172.21.15.153:0/515416548 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f780c1071d0 0x7f780c19cd90 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f780000d8d0 tx=0x7f780000dc90 comp rx=0 tx=0).stop 2024-03-21T10:34:42.378 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.379+0000 7f7813a00700 1 -- 172.21.15.153:0/515416548 shutdown_connections 2024-03-21T10:34:42.378 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.379+0000 7f7813a00700 1 --2- 172.21.15.153:0/515416548 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f77f406e460 0x7f77f4070920 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:42.378 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.379+0000 7f7813a00700 1 --2- 172.21.15.153:0/515416548 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f780c1071d0 0x7f780c19cd90 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:42.379 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.379+0000 7f7813a00700 1 --2- 172.21.15.153:0/515416548 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f780c10d280 0x7f780c19d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:42.379 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.379+0000 7f7813a00700 1 -- 172.21.15.153:0/515416548 >> 172.21.15.153:0/515416548 conn(0x7f780c0769e0 msgr2=0x7f780c078480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:42.379 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.380+0000 7f7813a00700 1 -- 172.21.15.153:0/515416548 shutdown_connections 2024-03-21T10:34:42.379 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.380+0000 7f7813a00700 1 -- 172.21.15.153:0/515416548 wait complete. 2024-03-21T10:34:42.529 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.530+0000 7f35f2c00700 1 -- 172.21.15.153:0/4212030752 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f35ec10d280 msgr2=0x7f35ec10d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:42.529 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.530+0000 7f35f2c00700 1 --2- 172.21.15.153:0/4212030752 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f35ec10d280 0x7f35ec10d660 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f35dc009910 tx=0x7f35dc009c20 comp rx=0 tx=0).stop 2024-03-21T10:34:42.529 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.530+0000 7f35f2c00700 1 -- 172.21.15.153:0/4212030752 shutdown_connections 2024-03-21T10:34:42.529 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.530+0000 7f35f2c00700 1 --2- 172.21.15.153:0/4212030752 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f35ec1071d0 0x7f35ec107650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:42.529 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.530+0000 7f35f2c00700 1 --2- 172.21.15.153:0/4212030752 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f35ec10d280 0x7f35ec10d660 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:42.529 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.530+0000 7f35f2c00700 1 -- 172.21.15.153:0/4212030752 >> 172.21.15.153:0/4212030752 conn(0x7f35ec0769e0 msgr2=0x7f35ec078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:42.529 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.530+0000 7f35f2c00700 1 -- 172.21.15.153:0/4212030752 shutdown_connections 2024-03-21T10:34:42.529 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.530+0000 7f35f2c00700 1 -- 172.21.15.153:0/4212030752 wait complete. 2024-03-21T10:34:42.530 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.531+0000 7f35f2c00700 1 Processor -- start 2024-03-21T10:34:42.530 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.532+0000 7f35f2c00700 1 -- start start 2024-03-21T10:34:42.530 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.532+0000 7f35f2c00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f35ec1071d0 0x7f35ec19ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:42.531 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.532+0000 7f35f2c00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f35ec10d280 0x7f35ec19d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:42.531 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.532+0000 7f35f2c00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35ec19d8a0 con 0x7f35ec10d280 2024-03-21T10:34:42.531 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.532+0000 7f35f2c00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35ec1a05d0 con 0x7f35ec1071d0 2024-03-21T10:34:42.531 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.532+0000 7f35eb400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f35ec10d280 0x7f35ec19d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:42.531 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.532+0000 7f35eb400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f35ec10d280 0x7f35ec19d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:48028/0 (socket says 172.21.15.153:48028) 2024-03-21T10:34:42.532 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.532+0000 7f35eb400700 1 -- 172.21.15.153:0/244525072 learned_addr learned my addr 172.21.15.153:0/244525072 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:34:42.532 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.533+0000 7f35ebe00700 1 --2- 172.21.15.153:0/244525072 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f35ec1071d0 0x7f35ec19ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:42.532 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.533+0000 7f35eb400700 1 -- 172.21.15.153:0/244525072 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f35ec1071d0 msgr2=0x7f35ec19ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:42.532 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.533+0000 7f35eb400700 1 --2- 172.21.15.153:0/244525072 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f35ec1071d0 0x7f35ec19ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:42.532 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.533+0000 7f35eb400700 1 -- 172.21.15.153:0/244525072 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35dc0095f0 con 0x7f35ec10d280 2024-03-21T10:34:42.532 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.533+0000 7f35eb400700 1 --2- 172.21.15.153:0/244525072 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f35ec10d280 0x7f35ec19d210 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f35d800d8d0 tx=0x7f35d800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:42.532 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.533+0000 7f35e8c00700 1 -- 172.21.15.153:0/244525072 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f35d8009940 con 0x7f35ec10d280 2024-03-21T10:34:42.533 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.533+0000 7f35e8c00700 1 -- 172.21.15.153:0/244525072 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f35d8010460 con 0x7f35ec10d280 2024-03-21T10:34:42.533 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.534+0000 7f35f2c00700 1 -- 172.21.15.153:0/244525072 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35ec1a08b0 con 0x7f35ec10d280 2024-03-21T10:34:42.533 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.534+0000 7f35e8c00700 1 -- 172.21.15.153:0/244525072 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f35d800f5f0 con 0x7f35ec10d280 2024-03-21T10:34:42.533 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.534+0000 7f35f2c00700 1 -- 172.21.15.153:0/244525072 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35ec1a0e80 con 0x7f35ec10d280 2024-03-21T10:34:42.533 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.535+0000 7f35f2c00700 1 -- 172.21.15.153:0/244525072 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35ec04ea90 con 0x7f35ec10d280 2024-03-21T10:34:42.534 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.536+0000 7f35e8c00700 1 -- 172.21.15.153:0/244525072 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f35d80105d0 con 0x7f35ec10d280 2024-03-21T10:34:42.535 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.537+0000 7f35e8c00700 1 --2- 172.21.15.153:0/244525072 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f35d006e460 0x7f35d0070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:42.535 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.537+0000 7f35e8c00700 1 -- 172.21.15.153:0/244525072 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f35d808e4b0 con 0x7f35ec10d280 2024-03-21T10:34:42.536 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.537+0000 7f35ebe00700 1 --2- 172.21.15.153:0/244525072 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f35d006e460 0x7f35d0070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:42.536 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.538+0000 7f35ebe00700 1 --2- 172.21.15.153:0/244525072 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f35d006e460 0x7f35d0070920 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f35dc00b5c0 tx=0x7f35dc005960 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:42.540 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.542+0000 7f35e8c00700 1 -- 172.21.15.153:0/244525072 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f35d8057550 con 0x7f35ec10d280 2024-03-21T10:34:42.769 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.770+0000 7f35f2c00700 1 -- 172.21.15.153:0/244525072 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f35ec114510 con 0x7f35d006e460 2024-03-21T10:34:42.800 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.801+0000 7f35e8c00700 1 -- 172.21.15.153:0/244525072 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f35ec114510 con 0x7f35d006e460 2024-03-21T10:34:42.801 INFO:teuthology.orchestra.run.smithi153.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-21T10:34:42.801 INFO:teuthology.orchestra.run.smithi153.stdout:alertmanager.smithi153 smithi153 *:9093,9094 running (23m) 14m ago 37m 14.7M - 0.25.0 c8568f914cd2 9ed3f2200b5f 2024-03-21T10:34:42.801 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi153 smithi153 running (37m) 14m ago 37m 9424k - 18.2.1 5be31c24972a 5b61994396a6 2024-03-21T10:34:42.801 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi174 smithi174 running (36m) 14m ago 36m 9244k - 18.2.1 5be31c24972a 4f7a5fc3ac28 2024-03-21T10:34:42.801 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi153 smithi153 running (19m) 14m ago 37m 6656k - 19.0.0-2231-g2edfee68 237e43c7a65f 32c21104d5b0 2024-03-21T10:34:42.801 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi174 smithi174 running (19m) 14m ago 35m 6660k - 19.0.0-2231-g2edfee68 237e43c7a65f a04e48967ab6 2024-03-21T10:34:42.801 INFO:teuthology.orchestra.run.smithi153.stdout:grafana.smithi153 smithi153 *:3000 running (23m) 14m ago 36m 73.6M - 9.4.12 2bacad6d85d8 ee24026ab786 2024-03-21T10:34:42.801 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.iuyluj smithi153 running (15m) 14m ago 30m 10.9M - 19.0.0-2231-g2edfee68 237e43c7a65f 87aff8a5de70 2024-03-21T10:34:42.801 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.zipckr smithi153 running (15m) 14m ago 30m 11.8M - 19.0.0-2231-g2edfee68 237e43c7a65f ccc262bd5dd3 2024-03-21T10:34:42.802 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.nhozag smithi174 running (29m) 14m ago 29m 12.0M - 18.2.1 5be31c24972a 2fee4000308f 2024-03-21T10:34:42.802 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.yyimst smithi174 running (14m) 14m ago 30m 10.7M - 19.0.0-2231-g2edfee68 237e43c7a65f fc1962f9351e 2024-03-21T10:34:42.802 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi153.tluxni smithi153 *:8443,9283,8765 running (26m) 14m ago 39m 509M - 19.0.0-2231-g2edfee68 237e43c7a65f 7ffee0e55382 2024-03-21T10:34:42.802 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi174.qpcwat smithi174 *:8443,9283,8765 running (25m) 14m ago 35m 452M - 19.0.0-2231-g2edfee68 237e43c7a65f 748968f216ab 2024-03-21T10:34:42.802 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi153 smithi153 running (21m) 14m ago 39m 55.0M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f dd8038303dc3 2024-03-21T10:34:42.802 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi174 smithi174 running (20m) 14m ago 35m 44.2M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f 29d78a788f2c 2024-03-21T10:34:42.802 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi153 smithi153 *:9100 running (24m) 14m ago 37m 7889k - 1.5.0 0da6a335fe13 bb835edc0e0b 2024-03-21T10:34:42.802 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi174 smithi174 *:9100 running (24m) 14m ago 35m 8102k - 1.5.0 0da6a335fe13 7c4ecad790fa 2024-03-21T10:34:42.802 INFO:teuthology.orchestra.run.smithi153.stdout:osd.0 smithi153 running (19m) 14m ago 34m 116M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 507900fd4791 2024-03-21T10:34:42.802 INFO:teuthology.orchestra.run.smithi153.stdout:osd.1 smithi153 running (18m) 14m ago 34m 89.2M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 0d646452e9c1 2024-03-21T10:34:42.802 INFO:teuthology.orchestra.run.smithi153.stdout:osd.2 smithi153 running (17m) 14m ago 33m 87.4M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 1326f2afdfcd 2024-03-21T10:34:42.802 INFO:teuthology.orchestra.run.smithi153.stdout:osd.3 smithi174 running (17m) 14m ago 32m 83.6M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 70aafac71ad5 2024-03-21T10:34:42.802 INFO:teuthology.orchestra.run.smithi153.stdout:osd.4 smithi174 running (16m) 14m ago 32m 92.2M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 0d746ce6e79d 2024-03-21T10:34:42.803 INFO:teuthology.orchestra.run.smithi153.stdout:osd.5 smithi174 running (15m) 14m ago 31m 86.2M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 53e5f35b8cac 2024-03-21T10:34:42.803 INFO:teuthology.orchestra.run.smithi153.stdout:prometheus.smithi153 smithi153 *:9095 running (23m) 14m ago 36m 73.5M - 2.43.0 a07b618ecd1d 7d95561e50c2 2024-03-21T10:34:42.803 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.804+0000 7f35f2c00700 1 -- 172.21.15.153:0/244525072 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f35d006e460 msgr2=0x7f35d0070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:42.803 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.804+0000 7f35f2c00700 1 --2- 172.21.15.153:0/244525072 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f35d006e460 0x7f35d0070920 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f35dc00b5c0 tx=0x7f35dc005960 comp rx=0 tx=0).stop 2024-03-21T10:34:42.804 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.804+0000 7f35f2c00700 1 -- 172.21.15.153:0/244525072 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f35ec10d280 msgr2=0x7f35ec19d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:42.804 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.804+0000 7f35f2c00700 1 --2- 172.21.15.153:0/244525072 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f35ec10d280 0x7f35ec19d210 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f35d800d8d0 tx=0x7f35d800dc90 comp rx=0 tx=0).stop 2024-03-21T10:34:42.804 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.805+0000 7f35f2c00700 1 -- 172.21.15.153:0/244525072 shutdown_connections 2024-03-21T10:34:42.804 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.805+0000 7f35f2c00700 1 --2- 172.21.15.153:0/244525072 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f35ec1071d0 0x7f35ec19ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:42.804 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.805+0000 7f35f2c00700 1 --2- 172.21.15.153:0/244525072 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f35d006e460 0x7f35d0070920 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:42.804 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.805+0000 7f35f2c00700 1 --2- 172.21.15.153:0/244525072 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f35ec10d280 0x7f35ec19d210 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:42.804 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.805+0000 7f35f2c00700 1 -- 172.21.15.153:0/244525072 >> 172.21.15.153:0/244525072 conn(0x7f35ec0769e0 msgr2=0x7f35ec078320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:42.804 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.805+0000 7f35f2c00700 1 -- 172.21.15.153:0/244525072 shutdown_connections 2024-03-21T10:34:42.804 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.805+0000 7f35f2c00700 1 -- 172.21.15.153:0/244525072 wait complete. 2024-03-21T10:34:42.946 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:34:42 smithi153 ceph-mon[113733]: pgmap v685: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:34:42.946 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:34:42 smithi153 ceph-mon[113733]: from='client.35234 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:34:42.946 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:34:42 smithi153 ceph-mon[113733]: from='client.35238 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:34:42.959 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.960+0000 7f7d85a00700 1 -- 172.21.15.153:0/3200043036 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7d80101750 msgr2=0x7f7d801057a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:42.959 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.960+0000 7f7d85a00700 1 --2- 172.21.15.153:0/3200043036 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7d80101750 0x7f7d801057a0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c009910 tx=0x7f7d6c009c20 comp rx=0 tx=0).stop 2024-03-21T10:34:42.959 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.960+0000 7f7d85a00700 1 -- 172.21.15.153:0/3200043036 shutdown_connections 2024-03-21T10:34:42.959 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.960+0000 7f7d85a00700 1 --2- 172.21.15.153:0/3200043036 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7d80101750 0x7f7d801057a0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:42.959 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.960+0000 7f7d85a00700 1 --2- 172.21.15.153:0/3200043036 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7d80100da0 0x7f7d80101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:42.959 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.960+0000 7f7d85a00700 1 -- 172.21.15.153:0/3200043036 >> 172.21.15.153:0/3200043036 conn(0x7f7d800fc630 msgr2=0x7f7d800fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:42.959 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.960+0000 7f7d85a00700 1 -- 172.21.15.153:0/3200043036 shutdown_connections 2024-03-21T10:34:42.959 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.960+0000 7f7d85a00700 1 -- 172.21.15.153:0/3200043036 wait complete. 2024-03-21T10:34:42.960 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.961+0000 7f7d85a00700 1 Processor -- start 2024-03-21T10:34:42.960 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.961+0000 7f7d85a00700 1 -- start start 2024-03-21T10:34:42.960 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.962+0000 7f7d85a00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7d80100da0 0x7f7d8019c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:42.960 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.962+0000 7f7d85a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7d80101750 0x7f7d8019ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:42.960 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.962+0000 7f7d85a00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d8019d0d0 con 0x7f7d80101750 2024-03-21T10:34:42.960 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.962+0000 7f7d85a00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d80196580 con 0x7f7d80100da0 2024-03-21T10:34:42.961 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.962+0000 7f7d7e000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7d80101750 0x7f7d8019ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:42.961 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.962+0000 7f7d7e000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7d80101750 0x7f7d8019ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:57968/0 (socket says 172.21.15.153:57968) 2024-03-21T10:34:42.961 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.962+0000 7f7d7e000700 1 -- 172.21.15.153:0/4172762099 learned_addr learned my addr 172.21.15.153:0/4172762099 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:34:42.961 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.962+0000 7f7d7ea00700 1 --2- 172.21.15.153:0/4172762099 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7d80100da0 0x7f7d8019c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:42.962 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.962+0000 7f7d7e000700 1 -- 172.21.15.153:0/4172762099 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7d80100da0 msgr2=0x7f7d8019c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:42.962 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.962+0000 7f7d7e000700 1 --2- 172.21.15.153:0/4172762099 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7d80100da0 0x7f7d8019c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:42.962 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.963+0000 7f7d7e000700 1 -- 172.21.15.153:0/4172762099 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d6c0095f0 con 0x7f7d80101750 2024-03-21T10:34:42.962 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.963+0000 7f7d7e000700 1 --2- 172.21.15.153:0/4172762099 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7d80101750 0x7f7d8019ca40 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c000c00 tx=0x7f7d6c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:42.962 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.963+0000 7f7d77400700 1 -- 172.21.15.153:0/4172762099 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d6c01d070 con 0x7f7d80101750 2024-03-21T10:34:42.962 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.963+0000 7f7d77400700 1 -- 172.21.15.153:0/4172762099 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7d6c005dc0 con 0x7f7d80101750 2024-03-21T10:34:42.963 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.963+0000 7f7d85a00700 1 -- 172.21.15.153:0/4172762099 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d80196800 con 0x7f7d80101750 2024-03-21T10:34:42.963 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.963+0000 7f7d77400700 1 -- 172.21.15.153:0/4172762099 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d6c017400 con 0x7f7d80101750 2024-03-21T10:34:42.963 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.963+0000 7f7d85a00700 1 -- 172.21.15.153:0/4172762099 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d80196cf0 con 0x7f7d80101750 2024-03-21T10:34:42.963 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.964+0000 7f7d85a00700 1 -- 172.21.15.153:0/4172762099 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d801091a0 con 0x7f7d80101750 2024-03-21T10:34:42.964 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.965+0000 7f7d77400700 1 -- 172.21.15.153:0/4172762099 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f7d6c00f7f0 con 0x7f7d80101750 2024-03-21T10:34:42.965 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.966+0000 7f7d77400700 1 --2- 172.21.15.153:0/4172762099 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f7d6406e460 0x7f7d64070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:42.965 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.967+0000 7f7d77400700 1 -- 172.21.15.153:0/4172762099 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f7d6c08fa50 con 0x7f7d80101750 2024-03-21T10:34:42.965 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.967+0000 7f7d7ea00700 1 --2- 172.21.15.153:0/4172762099 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f7d6406e460 0x7f7d64070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:42.966 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.967+0000 7f7d7ea00700 1 --2- 172.21.15.153:0/4172762099 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f7d6406e460 0x7f7d64070920 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f7d70005950 tx=0x7f7d700058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:42.969 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:42.971+0000 7f7d77400700 1 -- 172.21.15.153:0/4172762099 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7d6c05c5a0 con 0x7f7d80101750 2024-03-21T10:34:43.198 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:34:42 smithi174 ceph-mon[88004]: pgmap v685: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:34:43.199 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:34:42 smithi174 ceph-mon[88004]: from='client.35234 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:34:43.199 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:34:42 smithi174 ceph-mon[88004]: from='client.35238 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:34:43.271 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:43.272+0000 7f7d85a00700 1 -- 172.21.15.153:0/4172762099 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f7d8004ea90 con 0x7f7d80101750 2024-03-21T10:34:43.272 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:43.273+0000 7f7d77400700 1 -- 172.21.15.153:0/4172762099 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+805 (secure 0 0 0) 0x7f7d6c05c130 con 0x7f7d80101750 2024-03-21T10:34:43.272 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:34:43.272 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": { 2024-03-21T10:34:43.272 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:34:43.272 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:34:43.272 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": { 2024-03-21T10:34:43.272 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:34:43.272 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:34:43.273 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": { 2024-03-21T10:34:43.273 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 6 2024-03-21T10:34:43.273 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:34:43.273 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": { 2024-03-21T10:34:43.273 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-03-21T10:34:43.273 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 3 2024-03-21T10:34:43.273 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:34:43.273 INFO:teuthology.orchestra.run.smithi153.stdout: "overall": { 2024-03-21T10:34:43.273 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-03-21T10:34:43.273 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 13 2024-03-21T10:34:43.273 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-03-21T10:34:43.273 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:34:43.276 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:43.277+0000 7f7d85a00700 1 -- 172.21.15.153:0/4172762099 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f7d6406e460 msgr2=0x7f7d64070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:43.276 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:43.277+0000 7f7d85a00700 1 --2- 172.21.15.153:0/4172762099 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f7d6406e460 0x7f7d64070920 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f7d70005950 tx=0x7f7d700058e0 comp rx=0 tx=0).stop 2024-03-21T10:34:43.276 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:43.278+0000 7f7d85a00700 1 -- 172.21.15.153:0/4172762099 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7d80101750 msgr2=0x7f7d8019ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:43.277 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:43.278+0000 7f7d85a00700 1 --2- 172.21.15.153:0/4172762099 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7d80101750 0x7f7d8019ca40 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c000c00 tx=0x7f7d6c00be20 comp rx=0 tx=0).stop 2024-03-21T10:34:43.277 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:43.278+0000 7f7d85a00700 1 -- 172.21.15.153:0/4172762099 shutdown_connections 2024-03-21T10:34:43.277 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:43.278+0000 7f7d85a00700 1 --2- 172.21.15.153:0/4172762099 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7d80100da0 0x7f7d8019c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:43.277 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:43.278+0000 7f7d85a00700 1 --2- 172.21.15.153:0/4172762099 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f7d6406e460 0x7f7d64070920 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:43.277 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:43.278+0000 7f7d85a00700 1 --2- 172.21.15.153:0/4172762099 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7d80101750 0x7f7d8019ca40 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:43.277 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:43.278+0000 7f7d85a00700 1 -- 172.21.15.153:0/4172762099 >> 172.21.15.153:0/4172762099 conn(0x7f7d800fc630 msgr2=0x7f7d800fea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:43.277 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:43.278+0000 7f7d85a00700 1 -- 172.21.15.153:0/4172762099 shutdown_connections 2024-03-21T10:34:43.277 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:43.278+0000 7f7d85a00700 1 -- 172.21.15.153:0/4172762099 wait complete. 2024-03-21T10:34:43.426 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:43.426+0000 7ff482200700 1 -- 172.21.15.153:0/2277515368 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff47c10d280 msgr2=0x7ff47c10d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:43.426 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:43.426+0000 7ff482200700 1 --2- 172.21.15.153:0/2277515368 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff47c10d280 0x7ff47c10d660 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7ff46c009910 tx=0x7ff46c009c20 comp rx=0 tx=0).stop 2024-03-21T10:34:43.426 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:43.427+0000 7ff482200700 1 -- 172.21.15.153:0/2277515368 shutdown_connections 2024-03-21T10:34:43.426 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:43.427+0000 7ff482200700 1 --2- 172.21.15.153:0/2277515368 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff47c1071d0 0x7ff47c107650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:43.426 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:43.427+0000 7ff482200700 1 --2- 172.21.15.153:0/2277515368 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff47c10d280 0x7ff47c10d660 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:43.426 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:43.427+0000 7ff482200700 1 -- 172.21.15.153:0/2277515368 >> 172.21.15.153:0/2277515368 conn(0x7ff47c0769e0 msgr2=0x7ff47c078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:43.426 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:43.427+0000 7ff482200700 1 -- 172.21.15.153:0/2277515368 shutdown_connections 2024-03-21T10:34:43.426 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:43.427+0000 7ff482200700 1 -- 172.21.15.153:0/2277515368 wait complete. 2024-03-21T10:34:43.427 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:43.428+0000 7ff482200700 1 Processor -- start 2024-03-21T10:34:43.427 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:43.428+0000 7ff482200700 1 -- start start 2024-03-21T10:34:43.428 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:43.429+0000 7ff482200700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff47c1071d0 0x7ff47c19ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:43.428 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:43.429+0000 7ff482200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff47c10d280 0x7ff47c19d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:43.428 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:43.429+0000 7ff482200700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff47c19da30 con 0x7ff47c10d280 2024-03-21T10:34:43.428 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:43.429+0000 7ff482200700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff47c1a1770 con 0x7ff47c1071d0 2024-03-21T10:34:43.428 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:43.429+0000 7ff47aa00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff47c10d280 0x7ff47c19d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:43.428 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:43.429+0000 7ff47aa00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff47c10d280 0x7ff47c19d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:57980/0 (socket says 172.21.15.153:57980) 2024-03-21T10:34:43.428 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:43.429+0000 7ff47aa00700 1 -- 172.21.15.153:0/1678497690 learned_addr learned my addr 172.21.15.153:0/1678497690 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:34:43.428 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:43.429+0000 7ff47b400700 1 --2- 172.21.15.153:0/1678497690 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff47c1071d0 0x7ff47c19ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:43.429 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:43.430+0000 7ff47aa00700 1 -- 172.21.15.153:0/1678497690 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff47c1071d0 msgr2=0x7ff47c19ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:43.429 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:43.430+0000 7ff47aa00700 1 --2- 172.21.15.153:0/1678497690 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff47c1071d0 0x7ff47c19ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:43.429 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:43.430+0000 7ff47aa00700 1 -- 172.21.15.153:0/1678497690 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff46c0095f0 con 0x7ff47c10d280 2024-03-21T10:34:43.429 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:43.430+0000 7ff47aa00700 1 --2- 172.21.15.153:0/1678497690 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff47c10d280 0x7ff47c19d350 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7ff46800b4b0 tx=0x7ff46800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:43.429 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:43.430+0000 7ff473e00700 1 -- 172.21.15.153:0/1678497690 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff468011800 con 0x7ff47c10d280 2024-03-21T10:34:43.429 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:43.430+0000 7ff473e00700 1 -- 172.21.15.153:0/1678497690 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff468011e20 con 0x7ff47c10d280 2024-03-21T10:34:43.429 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:43.430+0000 7ff473e00700 1 -- 172.21.15.153:0/1678497690 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff46800f2e0 con 0x7ff47c10d280 2024-03-21T10:34:43.429 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:43.430+0000 7ff482200700 1 -- 172.21.15.153:0/1678497690 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff47c1a1a50 con 0x7ff47c10d280 2024-03-21T10:34:43.430 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:43.431+0000 7ff482200700 1 -- 172.21.15.153:0/1678497690 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff47c1a2020 con 0x7ff47c10d280 2024-03-21T10:34:43.430 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:43.432+0000 7ff482200700 1 -- 172.21.15.153:0/1678497690 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff47c10f6f0 con 0x7ff47c10d280 2024-03-21T10:34:43.431 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:43.433+0000 7ff473e00700 1 -- 172.21.15.153:0/1678497690 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7ff468010380 con 0x7ff47c10d280 2024-03-21T10:34:43.432 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:43.434+0000 7ff473e00700 1 --2- 172.21.15.153:0/1678497690 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff46006e460 0x7ff460070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:43.432 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:43.434+0000 7ff473e00700 1 -- 172.21.15.153:0/1678497690 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7ff46808e0c0 con 0x7ff47c10d280 2024-03-21T10:34:43.432 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:43.434+0000 7ff47b400700 1 --2- 172.21.15.153:0/1678497690 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff46006e460 0x7ff460070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:43.433 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:43.435+0000 7ff47b400700 1 --2- 172.21.15.153:0/1678497690 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff46006e460 0x7ff460070920 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ff46c00b5c0 tx=0x7ff46c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:43.440 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:43.442+0000 7ff473e00700 1 -- 172.21.15.153:0/1678497690 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff468057210 con 0x7ff47c10d280 2024-03-21T10:34:43.704 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:43.705+0000 7ff482200700 1 -- 172.21.15.153:0/1678497690 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7ff47c04ea90 con 0x7ff47c10d280 2024-03-21T10:34:43.704 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:43.706+0000 7ff473e00700 1 -- 172.21.15.153:0/1678497690 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1876 (secure 0 0 0) 0x7ff46805a8d0 con 0x7ff47c10d280 2024-03-21T10:34:43.705 INFO:teuthology.orchestra.run.smithi153.stdout:e30 2024-03-21T10:34:43.705 INFO:teuthology.orchestra.run.smithi153.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-21T10:34:43.705 INFO:teuthology.orchestra.run.smithi153.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:34:43.705 INFO:teuthology.orchestra.run.smithi153.stdout:legacy client fscid: 1 2024-03-21T10:34:43.705 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:34:43.705 INFO:teuthology.orchestra.run.smithi153.stdout:Filesystem 'cephfs' (1) 2024-03-21T10:34:43.705 INFO:teuthology.orchestra.run.smithi153.stdout:fs_name cephfs 2024-03-21T10:34:43.705 INFO:teuthology.orchestra.run.smithi153.stdout:epoch 23 2024-03-21T10:34:43.705 INFO:teuthology.orchestra.run.smithi153.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-21T10:34:43.706 INFO:teuthology.orchestra.run.smithi153.stdout:created 2024-03-21T10:04:27.086576+0000 2024-03-21T10:34:43.706 INFO:teuthology.orchestra.run.smithi153.stdout:modified 2024-03-21T10:19:05.022092+0000 2024-03-21T10:34:43.706 INFO:teuthology.orchestra.run.smithi153.stdout:tableserver 0 2024-03-21T10:34:43.706 INFO:teuthology.orchestra.run.smithi153.stdout:root 0 2024-03-21T10:34:43.706 INFO:teuthology.orchestra.run.smithi153.stdout:session_timeout 60 2024-03-21T10:34:43.706 INFO:teuthology.orchestra.run.smithi153.stdout:session_autoclose 300 2024-03-21T10:34:43.706 INFO:teuthology.orchestra.run.smithi153.stdout:max_file_size 1099511627776 2024-03-21T10:34:43.706 INFO:teuthology.orchestra.run.smithi153.stdout:max_xattr_size 65536 2024-03-21T10:34:43.706 INFO:teuthology.orchestra.run.smithi153.stdout:required_client_features {} 2024-03-21T10:34:43.706 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure 0 2024-03-21T10:34:43.706 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure_osd_epoch 94 2024-03-21T10:34:43.706 INFO:teuthology.orchestra.run.smithi153.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:34:43.706 INFO:teuthology.orchestra.run.smithi153.stdout:max_mds 2 2024-03-21T10:34:43.707 INFO:teuthology.orchestra.run.smithi153.stdout:in 0,1 2024-03-21T10:34:43.707 INFO:teuthology.orchestra.run.smithi153.stdout:up {} 2024-03-21T10:34:43.707 INFO:teuthology.orchestra.run.smithi153.stdout:failed 0,1 2024-03-21T10:34:43.707 INFO:teuthology.orchestra.run.smithi153.stdout:damaged 2024-03-21T10:34:43.707 INFO:teuthology.orchestra.run.smithi153.stdout:stopped 2024-03-21T10:34:43.707 INFO:teuthology.orchestra.run.smithi153.stdout:data_pools [3] 2024-03-21T10:34:43.707 INFO:teuthology.orchestra.run.smithi153.stdout:metadata_pool 2 2024-03-21T10:34:43.707 INFO:teuthology.orchestra.run.smithi153.stdout:inline_data enabled 2024-03-21T10:34:43.707 INFO:teuthology.orchestra.run.smithi153.stdout:balancer 2024-03-21T10:34:43.707 INFO:teuthology.orchestra.run.smithi153.stdout:bal_rank_mask -1 2024-03-21T10:34:43.707 INFO:teuthology.orchestra.run.smithi153.stdout:standby_count_wanted 1 2024-03-21T10:34:43.707 INFO:teuthology.orchestra.run.smithi153.stdout:qdb_cluster leader: 0 members: 2024-03-21T10:34:43.707 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:34:43.707 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:34:43.707 INFO:teuthology.orchestra.run.smithi153.stdout:Standby daemons: 2024-03-21T10:34:43.708 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:34:43.708 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.zipckr{-1:34442} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6826/1103846295,v1:172.21.15.153:6827/1103846295] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:34:43.708 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.iuyluj{-1:34474} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6828/1738630832,v1:172.21.15.153:6829/1738630832] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:34:43.708 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.nhozag{-1:44269} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.174:6826/3967223227,v1:172.21.15.174:6827/3967223227] compat {c=[1],r=[1],i=[7ff]}] 2024-03-21T10:34:43.708 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.yyimst{-1:44313} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.174:6824/4127786797,v1:172.21.15.174:6825/4127786797] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:34:43.708 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:43.709+0000 7ff482200700 1 -- 172.21.15.153:0/1678497690 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff46006e460 msgr2=0x7ff460070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:43.709 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:43.709+0000 7ff482200700 1 --2- 172.21.15.153:0/1678497690 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff46006e460 0x7ff460070920 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ff46c00b5c0 tx=0x7ff46c01a040 comp rx=0 tx=0).stop 2024-03-21T10:34:43.709 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:43.709+0000 7ff482200700 1 -- 172.21.15.153:0/1678497690 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff47c10d280 msgr2=0x7ff47c19d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:43.709 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:43.709+0000 7ff482200700 1 --2- 172.21.15.153:0/1678497690 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff47c10d280 0x7ff47c19d350 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7ff46800b4b0 tx=0x7ff46800b870 comp rx=0 tx=0).stop 2024-03-21T10:34:43.709 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:43.709+0000 7ff482200700 1 -- 172.21.15.153:0/1678497690 shutdown_connections 2024-03-21T10:34:43.709 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:43.709+0000 7ff482200700 1 --2- 172.21.15.153:0/1678497690 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff47c1071d0 0x7ff47c19ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:43.709 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:43.709+0000 7ff482200700 1 --2- 172.21.15.153:0/1678497690 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff46006e460 0x7ff460070920 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:43.709 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:43.709+0000 7ff482200700 1 --2- 172.21.15.153:0/1678497690 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff47c10d280 0x7ff47c19d350 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:43.709 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:43.709+0000 7ff482200700 1 -- 172.21.15.153:0/1678497690 >> 172.21.15.153:0/1678497690 conn(0x7ff47c0769e0 msgr2=0x7ff47c0785c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:43.709 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:43.709+0000 7ff482200700 1 -- 172.21.15.153:0/1678497690 shutdown_connections 2024-03-21T10:34:43.709 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:43.709+0000 7ff482200700 1 -- 172.21.15.153:0/1678497690 wait complete. 2024-03-21T10:34:43.710 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 30 2024-03-21T10:34:43.860 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:43.862+0000 7f0d32a00700 1 -- 172.21.15.153:0/1601791023 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0d2c102620 msgr2=0x7f0d2c10ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:43.860 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:43.862+0000 7f0d32a00700 1 --2- 172.21.15.153:0/1601791023 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0d2c102620 0x7f0d2c10ab10 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f0d14009910 tx=0x7f0d14009c20 comp rx=0 tx=0).stop 2024-03-21T10:34:43.860 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:43.862+0000 7f0d32a00700 1 -- 172.21.15.153:0/1601791023 shutdown_connections 2024-03-21T10:34:43.860 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:43.862+0000 7f0d32a00700 1 --2- 172.21.15.153:0/1601791023 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0d2c102620 0x7f0d2c10ab10 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:43.861 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:43.862+0000 7f0d32a00700 1 --2- 172.21.15.153:0/1601791023 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d2c101d00 0x7f0d2c1020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:43.861 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:43.862+0000 7f0d32a00700 1 -- 172.21.15.153:0/1601791023 >> 172.21.15.153:0/1601791023 conn(0x7f0d2c0fb570 msgr2=0x7f0d2c0fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:43.861 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:43.862+0000 7f0d32a00700 1 -- 172.21.15.153:0/1601791023 shutdown_connections 2024-03-21T10:34:43.861 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:43.862+0000 7f0d32a00700 1 -- 172.21.15.153:0/1601791023 wait complete. 2024-03-21T10:34:43.862 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:43.863+0000 7f0d32a00700 1 Processor -- start 2024-03-21T10:34:43.862 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:43.863+0000 7f0d32a00700 1 -- start start 2024-03-21T10:34:43.862 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:43.864+0000 7f0d32a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0d2c101d00 0x7f0d2c198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:43.862 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:43.864+0000 7f0d32a00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d2c102620 0x7f0d2c198e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:43.863 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:43.864+0000 7f0d32a00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d2c199520 con 0x7f0d2c101d00 2024-03-21T10:34:43.863 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:43.864+0000 7f0d32a00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d2c19d330 con 0x7f0d2c102620 2024-03-21T10:34:43.863 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:43.864+0000 7f0d2be00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0d2c101d00 0x7f0d2c198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:43.863 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:43.864+0000 7f0d2be00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0d2c101d00 0x7f0d2c198950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:58004/0 (socket says 172.21.15.153:58004) 2024-03-21T10:34:43.863 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:43.864+0000 7f0d2be00700 1 -- 172.21.15.153:0/3246600647 learned_addr learned my addr 172.21.15.153:0/3246600647 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:34:43.863 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:43.865+0000 7f0d23400700 1 --2- 172.21.15.153:0/3246600647 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d2c102620 0x7f0d2c198e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:43.863 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:43.865+0000 7f0d2be00700 1 -- 172.21.15.153:0/3246600647 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d2c102620 msgr2=0x7f0d2c198e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:43.863 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:43.865+0000 7f0d2be00700 1 --2- 172.21.15.153:0/3246600647 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d2c102620 0x7f0d2c198e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:43.863 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:43.865+0000 7f0d2be00700 1 -- 172.21.15.153:0/3246600647 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d140095f0 con 0x7f0d2c101d00 2024-03-21T10:34:43.863 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:43.865+0000 7f0d2be00700 1 --2- 172.21.15.153:0/3246600647 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0d2c101d00 0x7f0d2c198950 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c00c8a0 tx=0x7f0d1c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:43.864 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:43.865+0000 7f0d29600700 1 -- 172.21.15.153:0/3246600647 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0d1c00cea0 con 0x7f0d2c101d00 2024-03-21T10:34:43.864 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:43.865+0000 7f0d32a00700 1 -- 172.21.15.153:0/3246600647 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d2c19d610 con 0x7f0d2c101d00 2024-03-21T10:34:43.864 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:43.865+0000 7f0d29600700 1 -- 172.21.15.153:0/3246600647 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0d1c004830 con 0x7f0d2c101d00 2024-03-21T10:34:43.864 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:43.865+0000 7f0d29600700 1 -- 172.21.15.153:0/3246600647 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0d1c0055b0 con 0x7f0d2c101d00 2024-03-21T10:34:43.864 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:43.865+0000 7f0d32a00700 1 -- 172.21.15.153:0/3246600647 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d2c19db60 con 0x7f0d2c101d00 2024-03-21T10:34:43.865 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:43.866+0000 7f0d32a00700 1 -- 172.21.15.153:0/3246600647 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d2c10a670 con 0x7f0d2c101d00 2024-03-21T10:34:43.865 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:43.867+0000 7f0d29600700 1 -- 172.21.15.153:0/3246600647 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f0d1c0049a0 con 0x7f0d2c101d00 2024-03-21T10:34:43.866 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:43.868+0000 7f0d29600700 1 --2- 172.21.15.153:0/3246600647 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f0d1006e460 0x7f0d10070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:43.866 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:43.868+0000 7f0d29600700 1 -- 172.21.15.153:0/3246600647 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f0d1c08e270 con 0x7f0d2c101d00 2024-03-21T10:34:43.866 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:43.869+0000 7f0d23400700 1 --2- 172.21.15.153:0/3246600647 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f0d1006e460 0x7f0d10070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:43.867 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:43.869+0000 7f0d23400700 1 --2- 172.21.15.153:0/3246600647 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f0d1006e460 0x7f0d10070920 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f0d1400b5c0 tx=0x7f0d14005fd0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:43.870 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:43.873+0000 7f0d29600700 1 -- 172.21.15.153:0/3246600647 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0d1c05adc0 con 0x7f0d2c101d00 2024-03-21T10:34:44.118 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:44.121+0000 7f0d32a00700 1 -- 172.21.15.153:0/3246600647 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0d2c19dde0 con 0x7f0d1006e460 2024-03-21T10:34:44.120 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:44.123+0000 7f0d29600700 1 -- 172.21.15.153:0/3246600647 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f0d2c19dde0 con 0x7f0d1006e460 2024-03-21T10:34:44.121 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:34:44.121 INFO:teuthology.orchestra.run.smithi153.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232", 2024-03-21T10:34:44.121 INFO:teuthology.orchestra.run.smithi153.stdout: "in_progress": true, 2024-03-21T10:34:44.121 INFO:teuthology.orchestra.run.smithi153.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-21T10:34:44.121 INFO:teuthology.orchestra.run.smithi153.stdout: "services_complete": [ 2024-03-21T10:34:44.121 INFO:teuthology.orchestra.run.smithi153.stdout: "crash", 2024-03-21T10:34:44.121 INFO:teuthology.orchestra.run.smithi153.stdout: "mon", 2024-03-21T10:34:44.121 INFO:teuthology.orchestra.run.smithi153.stdout: "osd", 2024-03-21T10:34:44.121 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr" 2024-03-21T10:34:44.121 INFO:teuthology.orchestra.run.smithi153.stdout: ], 2024-03-21T10:34:44.122 INFO:teuthology.orchestra.run.smithi153.stdout: "progress": "15/23 daemons upgraded", 2024-03-21T10:34:44.122 INFO:teuthology.orchestra.run.smithi153.stdout: "message": "Currently upgrading mds daemons", 2024-03-21T10:34:44.122 INFO:teuthology.orchestra.run.smithi153.stdout: "is_paused": false 2024-03-21T10:34:44.122 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:34:44.123 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:44.125+0000 7f0d32a00700 1 -- 172.21.15.153:0/3246600647 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f0d1006e460 msgr2=0x7f0d10070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:44.124 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:44.126+0000 7f0d32a00700 1 --2- 172.21.15.153:0/3246600647 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f0d1006e460 0x7f0d10070920 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f0d1400b5c0 tx=0x7f0d14005fd0 comp rx=0 tx=0).stop 2024-03-21T10:34:44.124 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:44.126+0000 7f0d32a00700 1 -- 172.21.15.153:0/3246600647 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0d2c101d00 msgr2=0x7f0d2c198950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:44.124 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:44.126+0000 7f0d32a00700 1 --2- 172.21.15.153:0/3246600647 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0d2c101d00 0x7f0d2c198950 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c00c8a0 tx=0x7f0d1c00cc60 comp rx=0 tx=0).stop 2024-03-21T10:34:44.124 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:44.126+0000 7f0d32a00700 1 -- 172.21.15.153:0/3246600647 shutdown_connections 2024-03-21T10:34:44.124 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:44.126+0000 7f0d32a00700 1 --2- 172.21.15.153:0/3246600647 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f0d1006e460 0x7f0d10070920 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:44.124 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:44.126+0000 7f0d32a00700 1 --2- 172.21.15.153:0/3246600647 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0d2c101d00 0x7f0d2c198950 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:44.124 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:44.126+0000 7f0d32a00700 1 --2- 172.21.15.153:0/3246600647 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d2c102620 0x7f0d2c198e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:44.124 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:44.126+0000 7f0d32a00700 1 -- 172.21.15.153:0/3246600647 >> 172.21.15.153:0/3246600647 conn(0x7f0d2c0fb570 msgr2=0x7f0d2c100120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:44.124 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:44.126+0000 7f0d32a00700 1 -- 172.21.15.153:0/3246600647 shutdown_connections 2024-03-21T10:34:44.124 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:44.126+0000 7f0d32a00700 1 -- 172.21.15.153:0/3246600647 wait complete. 2024-03-21T10:34:44.271 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:44.274+0000 7fa141000700 1 -- 172.21.15.153:0/2359677252 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa13c102fb0 msgr2=0x7fa13c103390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:44.272 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:44.274+0000 7fa141000700 1 --2- 172.21.15.153:0/2359677252 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa13c102fb0 0x7fa13c103390 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fa12c009910 tx=0x7fa12c009c20 comp rx=0 tx=0).stop 2024-03-21T10:34:44.272 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:44.274+0000 7fa141000700 1 -- 172.21.15.153:0/2359677252 shutdown_connections 2024-03-21T10:34:44.272 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:44.274+0000 7fa141000700 1 --2- 172.21.15.153:0/2359677252 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa13c103960 0x7fa13c1079b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:44.272 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:44.274+0000 7fa141000700 1 --2- 172.21.15.153:0/2359677252 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa13c102fb0 0x7fa13c103390 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:44.272 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:44.274+0000 7fa141000700 1 -- 172.21.15.153:0/2359677252 >> 172.21.15.153:0/2359677252 conn(0x7fa13c0fe840 msgr2=0x7fa13c100c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:44.272 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:34:43 smithi153 ceph-mon[113733]: from='client.35242 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:34:44.272 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:34:43 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/4172762099' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:34:44.272 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:34:43 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/1678497690' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:34:44.274 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:44.274+0000 7fa141000700 1 -- 172.21.15.153:0/2359677252 shutdown_connections 2024-03-21T10:34:44.274 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:44.274+0000 7fa141000700 1 -- 172.21.15.153:0/2359677252 wait complete. 2024-03-21T10:34:44.274 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:44.275+0000 7fa141000700 1 Processor -- start 2024-03-21T10:34:44.274 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:44.275+0000 7fa141000700 1 -- start start 2024-03-21T10:34:44.274 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:44.275+0000 7fa141000700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa13c102fb0 0x7fa13c198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:44.274 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:44.275+0000 7fa141000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa13c103960 0x7fa13c198ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:44.274 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:44.275+0000 7fa141000700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa13c1994d0 con 0x7fa13c103960 2024-03-21T10:34:44.274 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:44.275+0000 7fa141000700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa13c199640 con 0x7fa13c102fb0 2024-03-21T10:34:44.274 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:44.276+0000 7fa139400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa13c103960 0x7fa13c198ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:44.274 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:44.276+0000 7fa139400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa13c103960 0x7fa13c198ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:58020/0 (socket says 172.21.15.153:58020) 2024-03-21T10:34:44.274 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:44.276+0000 7fa139400700 1 -- 172.21.15.153:0/1637542116 learned_addr learned my addr 172.21.15.153:0/1637542116 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:34:44.274 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:44.276+0000 7fa139e00700 1 --2- 172.21.15.153:0/1637542116 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa13c102fb0 0x7fa13c198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:44.275 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:44.276+0000 7fa139400700 1 -- 172.21.15.153:0/1637542116 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa13c102fb0 msgr2=0x7fa13c198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:44.275 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:44.276+0000 7fa139400700 1 --2- 172.21.15.153:0/1637542116 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa13c102fb0 0x7fa13c198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:44.275 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:44.276+0000 7fa139400700 1 -- 172.21.15.153:0/1637542116 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa12c0095f0 con 0x7fa13c103960 2024-03-21T10:34:44.275 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:44.276+0000 7fa139e00700 1 --2- 172.21.15.153:0/1637542116 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa13c102fb0 0x7fa13c198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:34:44.275 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:44.277+0000 7fa139400700 1 --2- 172.21.15.153:0/1637542116 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa13c103960 0x7fa13c198ed0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fa12800c960 tx=0x7fa12800cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:44.276 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:44.277+0000 7fa132a00700 1 -- 172.21.15.153:0/1637542116 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa128007a70 con 0x7fa13c103960 2024-03-21T10:34:44.276 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:44.277+0000 7fa132a00700 1 -- 172.21.15.153:0/1637542116 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa128007bd0 con 0x7fa13c103960 2024-03-21T10:34:44.276 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:44.277+0000 7fa132a00700 1 -- 172.21.15.153:0/1637542116 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa1280186c0 con 0x7fa13c103960 2024-03-21T10:34:44.276 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:44.277+0000 7fa141000700 1 -- 172.21.15.153:0/1637542116 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa13c19d4c0 con 0x7fa13c103960 2024-03-21T10:34:44.276 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:44.277+0000 7fa141000700 1 -- 172.21.15.153:0/1637542116 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa13c19da10 con 0x7fa13c103960 2024-03-21T10:34:44.276 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:44.278+0000 7fa141000700 1 -- 172.21.15.153:0/1637542116 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa13c10b350 con 0x7fa13c103960 2024-03-21T10:34:44.278 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:44.280+0000 7fa132a00700 1 -- 172.21.15.153:0/1637542116 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fa12800faa0 con 0x7fa13c103960 2024-03-21T10:34:44.279 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:44.281+0000 7fa132a00700 1 --2- 172.21.15.153:0/1637542116 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa12006e460 0x7fa120070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:34:44.279 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:44.281+0000 7fa132a00700 1 -- 172.21.15.153:0/1637542116 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fa12808e5d0 con 0x7fa13c103960 2024-03-21T10:34:44.279 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:44.281+0000 7fa139e00700 1 --2- 172.21.15.153:0/1637542116 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa12006e460 0x7fa120070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:34:44.279 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:44.282+0000 7fa139e00700 1 --2- 172.21.15.153:0/1637542116 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa12006e460 0x7fa120070920 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fa12c006010 tx=0x7fa12c00b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:34:44.283 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:44.286+0000 7fa132a00700 1 -- 172.21.15.153:0/1637542116 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa12805ae90 con 0x7fa13c103960 2024-03-21T10:34:44.448 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:34:43 smithi174 ceph-mon[88004]: from='client.35242 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:34:44.449 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:34:43 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/4172762099' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:34:44.449 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:34:43 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/1678497690' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:34:44.595 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:44.597+0000 7fa141000700 1 -- 172.21.15.153:0/1637542116 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fa13c04ea90 con 0x7fa13c103960 2024-03-21T10:34:44.595 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:44.598+0000 7fa132a00700 1 -- 172.21.15.153:0/1637542116 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7fa12805acb0 con 0x7fa13c103960 2024-03-21T10:34:44.596 INFO:teuthology.orchestra.run.smithi153.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-03-21T10:34:44.596 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-21T10:34:44.596 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is degraded 2024-03-21T10:34:44.596 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-21T10:34:44.596 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-21T10:34:44.596 INFO:teuthology.orchestra.run.smithi153.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-21T10:34:44.596 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-21T10:34:44.599 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:44.601+0000 7fa141000700 1 -- 172.21.15.153:0/1637542116 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa12006e460 msgr2=0x7fa120070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:44.599 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:44.601+0000 7fa141000700 1 --2- 172.21.15.153:0/1637542116 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa12006e460 0x7fa120070920 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fa12c006010 tx=0x7fa12c00b540 comp rx=0 tx=0).stop 2024-03-21T10:34:44.599 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:44.601+0000 7fa141000700 1 -- 172.21.15.153:0/1637542116 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa13c103960 msgr2=0x7fa13c198ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:34:44.599 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:44.601+0000 7fa141000700 1 --2- 172.21.15.153:0/1637542116 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa13c103960 0x7fa13c198ed0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fa12800c960 tx=0x7fa12800cd20 comp rx=0 tx=0).stop 2024-03-21T10:34:44.599 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:44.601+0000 7fa141000700 1 -- 172.21.15.153:0/1637542116 shutdown_connections 2024-03-21T10:34:44.599 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:44.601+0000 7fa141000700 1 --2- 172.21.15.153:0/1637542116 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa13c102fb0 0x7fa13c198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:44.599 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:44.601+0000 7fa141000700 1 --2- 172.21.15.153:0/1637542116 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa12006e460 0x7fa120070920 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:44.599 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:44.601+0000 7fa141000700 1 --2- 172.21.15.153:0/1637542116 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa13c103960 0x7fa13c198ed0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:34:44.599 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:44.601+0000 7fa141000700 1 -- 172.21.15.153:0/1637542116 >> 172.21.15.153:0/1637542116 conn(0x7fa13c0fe840 msgr2=0x7fa13c0fff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:34:44.599 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:44.601+0000 7fa141000700 1 -- 172.21.15.153:0/1637542116 shutdown_connections 2024-03-21T10:34:44.599 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:34:44.601+0000 7fa141000700 1 -- 172.21.15.153:0/1637542116 wait complete. 2024-03-21T10:34:45.303 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:34:44 smithi153 ceph-mon[113733]: pgmap v686: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:34:45.303 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:34:44 smithi153 ceph-mon[113733]: from='client.35254 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:34:45.304 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:34:44 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/1637542116' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:34:45.448 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:34:44 smithi174 ceph-mon[88004]: pgmap v686: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:34:45.448 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:34:44 smithi174 ceph-mon[88004]: from='client.35254 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:34:45.449 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:34:44 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/1637542116' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:34:47.303 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:34:46 smithi153 ceph-mon[113733]: pgmap v687: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:34:47.304 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:34:46 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:34:47.448 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:34:46 smithi174 ceph-mon[88004]: pgmap v687: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:34:47.449 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:34:46 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:34:49.303 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:34:48 smithi153 ceph-mon[113733]: pgmap v688: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:34:49.448 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:34:48 smithi174 ceph-mon[88004]: pgmap v688: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:34:51.303 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:34:50 smithi153 ceph-mon[113733]: pgmap v689: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:34:51.448 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:34:50 smithi174 ceph-mon[88004]: pgmap v689: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:34:53.303 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:34:52 smithi153 ceph-mon[113733]: pgmap v690: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:34:53.448 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:34:52 smithi174 ceph-mon[88004]: pgmap v690: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:34:55.303 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:34:54 smithi153 ceph-mon[113733]: pgmap v691: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:34:55.448 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:34:54 smithi174 ceph-mon[88004]: pgmap v691: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:34:57.303 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:34:56 smithi153 ceph-mon[113733]: pgmap v692: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:34:57.448 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:34:56 smithi174 ceph-mon[88004]: pgmap v692: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:34:59.303 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:34:59 smithi153 ceph-mon[113733]: pgmap v693: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:34:59.448 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:34:59 smithi174 ceph-mon[88004]: pgmap v693: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:35:01.303 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:35:01 smithi153 ceph-mon[113733]: pgmap v694: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:35:01.448 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:35:01 smithi174 ceph-mon[88004]: pgmap v694: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:35:02.198 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:35:02 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:35:02.303 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:35:02 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:35:03.303 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:35:03 smithi153 ceph-mon[113733]: pgmap v695: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:35:03.448 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:35:03 smithi174 ceph-mon[88004]: pgmap v695: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:35:05.303 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:35:05 smithi153 ceph-mon[113733]: pgmap v696: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:35:05.448 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:35:05 smithi174 ceph-mon[88004]: pgmap v696: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:35:07.303 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:35:07 smithi153 ceph-mon[113733]: pgmap v697: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:35:07.448 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:35:07 smithi174 ceph-mon[88004]: pgmap v697: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:35:08.304 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:35:08 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:35:08.304 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:35:08 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:35:08.304 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:35:08 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:35:08.304 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:35:08 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:35:08.304 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:35:08 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:35:08.304 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:35:08 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:35:08.304 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:35:08 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:35:08.304 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:35:08 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:35:08.448 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:35:08 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:35:08.449 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:35:08 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:35:08.449 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:35:08 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:35:08.449 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:35:08 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:35:08.449 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:35:08 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:35:08.449 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:35:08 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:35:08.449 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:35:08 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:35:08.449 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:35:08 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:35:09.196 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:35:09 smithi174 ceph-mon[88004]: pgmap v698: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:35:09.196 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:35:09 smithi174 ceph-mon[88004]: pgmap v699: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:35:09.196 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:35:09 smithi174 ceph-mon[88004]: Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED) 2024-03-21T10:35:09.303 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:35:09 smithi153 ceph-mon[113733]: pgmap v698: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:35:09.303 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:35:09 smithi153 ceph-mon[113733]: pgmap v699: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:35:09.304 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:35:09 smithi153 ceph-mon[113733]: Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED) 2024-03-21T10:35:11.449 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:35:11 smithi174 ceph-mon[88004]: pgmap v700: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:35:11.449 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:35:11 smithi174 ceph-mon[88004]: Upgrade: Updating mds.cephfs.smithi174.nhozag 2024-03-21T10:35:11.449 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:35:11 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:35:11.449 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:35:11 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi174.nhozag", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T10:35:11.449 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:35:11 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:35:11.449 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:35:11 smithi174 ceph-mon[88004]: Deploying daemon mds.cephfs.smithi174.nhozag on smithi174 2024-03-21T10:35:11.553 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:35:11 smithi153 ceph-mon[113733]: pgmap v700: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:35:11.553 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:35:11 smithi153 ceph-mon[113733]: Upgrade: Updating mds.cephfs.smithi174.nhozag 2024-03-21T10:35:11.554 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:35:11 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:35:11.554 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:35:11 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi174.nhozag", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T10:35:11.554 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:35:11 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:35:11.554 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:35:11 smithi153 ceph-mon[113733]: Deploying daemon mds.cephfs.smithi174.nhozag on smithi174 2024-03-21T10:35:13.448 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:35:13 smithi174 ceph-mon[88004]: pgmap v701: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:35:13.553 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:35:13 smithi153 ceph-mon[113733]: pgmap v701: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:35:14.553 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:35:14 smithi153 ceph-mon[113733]: osdmap e98: 6 total, 6 up, 6 in 2024-03-21T10:35:14.554 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:35:14 smithi153 ceph-mon[113733]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-03-21T10:35:14.698 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:35:14 smithi174 ceph-mon[88004]: osdmap e98: 6 total, 6 up, 6 in 2024-03-21T10:35:14.698 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:35:14 smithi174 ceph-mon[88004]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-03-21T10:35:14.769 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:14.770+0000 7fa59ae00700 1 -- 172.21.15.153:0/11261215 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa594103000 msgr2=0x7fa5941033e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:14.770 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:14.770+0000 7fa59ae00700 1 --2- 172.21.15.153:0/11261215 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa594103000 0x7fa5941033e0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fa58400b0a0 tx=0x7fa58400b3b0 comp rx=0 tx=0).stop 2024-03-21T10:35:14.770 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:14.771+0000 7fa59ae00700 1 -- 172.21.15.153:0/11261215 shutdown_connections 2024-03-21T10:35:14.770 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:14.771+0000 7fa59ae00700 1 --2- 172.21.15.153:0/11261215 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa5941039b0 0x7fa594107a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:14.770 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:14.771+0000 7fa59ae00700 1 --2- 172.21.15.153:0/11261215 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa594103000 0x7fa5941033e0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:14.770 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:14.771+0000 7fa59ae00700 1 -- 172.21.15.153:0/11261215 >> 172.21.15.153:0/11261215 conn(0x7fa5940fe8d0 msgr2=0x7fa594100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:35:14.770 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:14.771+0000 7fa59ae00700 1 -- 172.21.15.153:0/11261215 shutdown_connections 2024-03-21T10:35:14.770 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:14.771+0000 7fa59ae00700 1 -- 172.21.15.153:0/11261215 wait complete. 2024-03-21T10:35:14.771 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:14.772+0000 7fa59ae00700 1 Processor -- start 2024-03-21T10:35:14.771 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:14.772+0000 7fa59ae00700 1 -- start start 2024-03-21T10:35:14.771 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:14.773+0000 7fa59ae00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa594103000 0x7fa59419e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:14.771 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:14.773+0000 7fa59ae00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa5941039b0 0x7fa59419ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:14.771 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:14.773+0000 7fa59ae00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa59419f400 con 0x7fa5941039b0 2024-03-21T10:35:14.771 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:14.773+0000 7fa59ae00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5941988b0 con 0x7fa594103000 2024-03-21T10:35:14.772 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:14.773+0000 7fa58b400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa5941039b0 0x7fa59419ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:14.772 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:14.773+0000 7fa58b400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa5941039b0 0x7fa59419ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:53680/0 (socket says 172.21.15.153:53680) 2024-03-21T10:35:14.772 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:14.773+0000 7fa58b400700 1 -- 172.21.15.153:0/2599219581 learned_addr learned my addr 172.21.15.153:0/2599219581 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:35:14.772 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:14.773+0000 7fa593e00700 1 --2- 172.21.15.153:0/2599219581 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa594103000 0x7fa59419e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:14.773 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:14.774+0000 7fa58b400700 1 -- 172.21.15.153:0/2599219581 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa594103000 msgr2=0x7fa59419e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:14.773 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:14.774+0000 7fa58b400700 1 --2- 172.21.15.153:0/2599219581 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa594103000 0x7fa59419e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:14.773 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:14.774+0000 7fa58b400700 1 -- 172.21.15.153:0/2599219581 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa584009d00 con 0x7fa5941039b0 2024-03-21T10:35:14.773 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:14.774+0000 7fa58b400700 1 --2- 172.21.15.153:0/2599219581 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa5941039b0 0x7fa59419ed70 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fa57c00e8f0 tx=0x7fa57c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:35:14.773 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:14.774+0000 7fa591600700 1 -- 172.21.15.153:0/2599219581 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa57c009790 con 0x7fa5941039b0 2024-03-21T10:35:14.774 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:14.774+0000 7fa591600700 1 -- 172.21.15.153:0/2599219581 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa57c004cf0 con 0x7fa5941039b0 2024-03-21T10:35:14.774 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:14.774+0000 7fa591600700 1 -- 172.21.15.153:0/2599219581 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa57c005690 con 0x7fa5941039b0 2024-03-21T10:35:14.774 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:14.774+0000 7fa59ae00700 1 -- 172.21.15.153:0/2599219581 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa594198b90 con 0x7fa5941039b0 2024-03-21T10:35:14.774 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:14.775+0000 7fa59ae00700 1 -- 172.21.15.153:0/2599219581 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5941990e0 con 0x7fa5941039b0 2024-03-21T10:35:14.774 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:14.776+0000 7fa59ae00700 1 -- 172.21.15.153:0/2599219581 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa59410b3a0 con 0x7fa5941039b0 2024-03-21T10:35:14.775 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:14.777+0000 7fa591600700 1 -- 172.21.15.153:0/2599219581 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fa57c004750 con 0x7fa5941039b0 2024-03-21T10:35:14.776 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:14.778+0000 7fa591600700 1 --2- 172.21.15.153:0/2599219581 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa57806e460 0x7fa578070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:14.776 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:14.778+0000 7fa591600700 1 -- 172.21.15.153:0/2599219581 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fa57c014070 con 0x7fa5941039b0 2024-03-21T10:35:14.777 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:14.778+0000 7fa593e00700 1 --2- 172.21.15.153:0/2599219581 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa57806e460 0x7fa578070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:14.777 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:14.779+0000 7fa593e00700 1 --2- 172.21.15.153:0/2599219581 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa57806e460 0x7fa578070920 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fa584009480 tx=0x7fa58400b740 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:35:14.781 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:14.783+0000 7fa591600700 1 -- 172.21.15.153:0/2599219581 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa57c05b4b0 con 0x7fa5941039b0 2024-03-21T10:35:15.030 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:15.031+0000 7fa59ae00700 1 -- 172.21.15.153:0/2599219581 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa5941993d0 con 0x7fa57806e460 2024-03-21T10:35:15.032 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:15.033+0000 7fa591600700 1 -- 172.21.15.153:0/2599219581 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fa5941993d0 con 0x7fa57806e460 2024-03-21T10:35:15.037 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:15.038+0000 7fa59ae00700 1 -- 172.21.15.153:0/2599219581 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa57806e460 msgr2=0x7fa578070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:15.037 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:15.038+0000 7fa59ae00700 1 --2- 172.21.15.153:0/2599219581 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa57806e460 0x7fa578070920 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fa584009480 tx=0x7fa58400b740 comp rx=0 tx=0).stop 2024-03-21T10:35:15.037 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:15.038+0000 7fa59ae00700 1 -- 172.21.15.153:0/2599219581 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa5941039b0 msgr2=0x7fa59419ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:15.037 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:15.038+0000 7fa59ae00700 1 --2- 172.21.15.153:0/2599219581 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa5941039b0 0x7fa59419ed70 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fa57c00e8f0 tx=0x7fa57c00ecb0 comp rx=0 tx=0).stop 2024-03-21T10:35:15.037 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:15.038+0000 7fa59ae00700 1 -- 172.21.15.153:0/2599219581 shutdown_connections 2024-03-21T10:35:15.037 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:15.038+0000 7fa59ae00700 1 --2- 172.21.15.153:0/2599219581 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa594103000 0x7fa59419e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:15.037 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:15.038+0000 7fa59ae00700 1 --2- 172.21.15.153:0/2599219581 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa57806e460 0x7fa578070920 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:15.037 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:15.038+0000 7fa59ae00700 1 --2- 172.21.15.153:0/2599219581 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa5941039b0 0x7fa59419ed70 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:15.037 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:15.038+0000 7fa59ae00700 1 -- 172.21.15.153:0/2599219581 >> 172.21.15.153:0/2599219581 conn(0x7fa5940fe8d0 msgr2=0x7fa5940ffef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:35:15.037 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:15.038+0000 7fa59ae00700 1 -- 172.21.15.153:0/2599219581 shutdown_connections 2024-03-21T10:35:15.037 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:15.038+0000 7fa59ae00700 1 -- 172.21.15.153:0/2599219581 wait complete. 2024-03-21T10:35:15.054 INFO:teuthology.orchestra.run.smithi153.stdout:true 2024-03-21T10:35:15.197 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:15.198+0000 7f0cf9400700 1 -- 172.21.15.153:0/1086761095 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0cf4103000 msgr2=0x7f0cf41033e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:15.197 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:15.198+0000 7f0cf9400700 1 --2- 172.21.15.153:0/1086761095 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0cf4103000 0x7f0cf41033e0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f0ce4009910 tx=0x7f0ce4009c20 comp rx=0 tx=0).stop 2024-03-21T10:35:15.197 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:15.198+0000 7f0cf9400700 1 -- 172.21.15.153:0/1086761095 shutdown_connections 2024-03-21T10:35:15.197 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:15.198+0000 7f0cf9400700 1 --2- 172.21.15.153:0/1086761095 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0cf41039b0 0x7f0cf4107a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:15.198 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:15.198+0000 7f0cf9400700 1 --2- 172.21.15.153:0/1086761095 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0cf4103000 0x7f0cf41033e0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:15.198 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:15.198+0000 7f0cf9400700 1 -- 172.21.15.153:0/1086761095 >> 172.21.15.153:0/1086761095 conn(0x7f0cf40fe8d0 msgr2=0x7f0cf4100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:35:15.198 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:15.199+0000 7f0cf9400700 1 -- 172.21.15.153:0/1086761095 shutdown_connections 2024-03-21T10:35:15.198 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:15.199+0000 7f0cf9400700 1 -- 172.21.15.153:0/1086761095 wait complete. 2024-03-21T10:35:15.199 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:15.200+0000 7f0cf9400700 1 Processor -- start 2024-03-21T10:35:15.199 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:15.200+0000 7f0cf9400700 1 -- start start 2024-03-21T10:35:15.199 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:15.200+0000 7f0cf9400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0cf4103000 0x7f0cf419e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:15.200 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:15.200+0000 7f0cf9400700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0cf41039b0 0x7f0cf419ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:15.200 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:15.200+0000 7f0cf9400700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf419f4d0 con 0x7f0cf4103000 2024-03-21T10:35:15.200 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:15.200+0000 7f0cf9400700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf4198980 con 0x7f0cf41039b0 2024-03-21T10:35:15.200 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:15.200+0000 7f0cf2800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0cf4103000 0x7f0cf419e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:15.200 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:15.200+0000 7f0cf2800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0cf4103000 0x7f0cf419e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:53698/0 (socket says 172.21.15.153:53698) 2024-03-21T10:35:15.200 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:15.200+0000 7f0cf2800700 1 -- 172.21.15.153:0/3394950018 learned_addr learned my addr 172.21.15.153:0/3394950018 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:35:15.200 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:15.201+0000 7f0cf2800700 1 -- 172.21.15.153:0/3394950018 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0cf41039b0 msgr2=0x7f0cf419ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:15.200 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:15.201+0000 7f0cebe00700 1 --2- 172.21.15.153:0/3394950018 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0cf41039b0 0x7f0cf419ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:15.200 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:15.201+0000 7f0cf2800700 1 --2- 172.21.15.153:0/3394950018 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0cf41039b0 0x7f0cf419ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:15.200 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:15.201+0000 7f0cf2800700 1 -- 172.21.15.153:0/3394950018 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ce40095f0 con 0x7f0cf4103000 2024-03-21T10:35:15.200 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:15.201+0000 7f0cf2800700 1 --2- 172.21.15.153:0/3394950018 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0cf4103000 0x7f0cf419e900 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f0ce400b5c0 tx=0x7f0ce400be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:35:15.201 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:15.201+0000 7f0ceb400700 1 -- 172.21.15.153:0/3394950018 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ce401d070 con 0x7f0cf4103000 2024-03-21T10:35:15.201 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:15.202+0000 7f0ceb400700 1 -- 172.21.15.153:0/3394950018 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0ce400f460 con 0x7f0cf4103000 2024-03-21T10:35:15.201 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:15.202+0000 7f0cf9400700 1 -- 172.21.15.153:0/3394950018 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cf4198c00 con 0x7f0cf4103000 2024-03-21T10:35:15.201 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:15.202+0000 7f0ceb400700 1 -- 172.21.15.153:0/3394950018 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ce4017610 con 0x7f0cf4103000 2024-03-21T10:35:15.201 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:15.202+0000 7f0cf9400700 1 -- 172.21.15.153:0/3394950018 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cf41990f0 con 0x7f0cf4103000 2024-03-21T10:35:15.201 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:15.203+0000 7f0cf9400700 1 -- 172.21.15.153:0/3394950018 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cf410b3a0 con 0x7f0cf4103000 2024-03-21T10:35:15.202 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:15.204+0000 7f0ceb400700 1 -- 172.21.15.153:0/3394950018 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f0ce400f5d0 con 0x7f0cf4103000 2024-03-21T10:35:15.203 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:15.205+0000 7f0ceb400700 1 --2- 172.21.15.153:0/3394950018 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f0cd806e460 0x7f0cd8070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:15.203 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:15.205+0000 7f0ceb400700 1 -- 172.21.15.153:0/3394950018 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f0ce408fad0 con 0x7f0cf4103000 2024-03-21T10:35:15.203 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:15.205+0000 7f0cebe00700 1 --2- 172.21.15.153:0/3394950018 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f0cd806e460 0x7f0cd8070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:15.204 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:15.206+0000 7f0cebe00700 1 --2- 172.21.15.153:0/3394950018 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f0cd806e460 0x7f0cd8070920 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc006fd0 tx=0x7f0cdc008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:35:15.208 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:15.210+0000 7f0ceb400700 1 -- 172.21.15.153:0/3394950018 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0ce405c6a0 con 0x7f0cf4103000 2024-03-21T10:35:15.458 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:15.460+0000 7f0cf9400700 1 -- 172.21.15.153:0/3394950018 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0cf4199360 con 0x7f0cd806e460 2024-03-21T10:35:15.460 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:15.462+0000 7f0ceb400700 1 -- 172.21.15.153:0/3394950018 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f0cf4199360 con 0x7f0cd806e460 2024-03-21T10:35:15.463 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:15.464+0000 7f0cf9400700 1 -- 172.21.15.153:0/3394950018 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f0cd806e460 msgr2=0x7f0cd8070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:15.463 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:15.464+0000 7f0cf9400700 1 --2- 172.21.15.153:0/3394950018 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f0cd806e460 0x7f0cd8070920 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc006fd0 tx=0x7f0cdc008040 comp rx=0 tx=0).stop 2024-03-21T10:35:15.463 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:15.464+0000 7f0cf9400700 1 -- 172.21.15.153:0/3394950018 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0cf4103000 msgr2=0x7f0cf419e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:15.463 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:15.464+0000 7f0cf9400700 1 --2- 172.21.15.153:0/3394950018 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0cf4103000 0x7f0cf419e900 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f0ce400b5c0 tx=0x7f0ce400be00 comp rx=0 tx=0).stop 2024-03-21T10:35:15.464 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:15.465+0000 7f0cf9400700 1 -- 172.21.15.153:0/3394950018 shutdown_connections 2024-03-21T10:35:15.464 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:15.465+0000 7f0cf9400700 1 --2- 172.21.15.153:0/3394950018 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f0cd806e460 0x7f0cd8070920 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:15.464 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:15.465+0000 7f0cf9400700 1 --2- 172.21.15.153:0/3394950018 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0cf4103000 0x7f0cf419e900 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:15.464 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:15.465+0000 7f0cf9400700 1 --2- 172.21.15.153:0/3394950018 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0cf41039b0 0x7f0cf419ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:15.464 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:15.465+0000 7f0cf9400700 1 -- 172.21.15.153:0/3394950018 >> 172.21.15.153:0/3394950018 conn(0x7f0cf40fe8d0 msgr2=0x7f0cf40fff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:35:15.464 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:15.465+0000 7f0cf9400700 1 -- 172.21.15.153:0/3394950018 shutdown_connections 2024-03-21T10:35:15.464 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:15.465+0000 7f0cf9400700 1 -- 172.21.15.153:0/3394950018 wait complete. 2024-03-21T10:35:15.553 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:35:15 smithi153 ceph-mon[113733]: pgmap v703: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:35:15.553 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:35:15 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi174.nhozag"}]: dispatch 2024-03-21T10:35:15.621 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:15.623+0000 7f6a41400700 1 -- 172.21.15.153:0/2734727717 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6a3c068670 msgr2=0x7f6a3c068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:15.622 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:15.623+0000 7f6a41400700 1 --2- 172.21.15.153:0/2734727717 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6a3c068670 0x7f6a3c068a50 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c00b0a0 tx=0x7f6a2c00b3b0 comp rx=0 tx=0).stop 2024-03-21T10:35:15.622 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:15.623+0000 7f6a41400700 1 -- 172.21.15.153:0/2734727717 shutdown_connections 2024-03-21T10:35:15.622 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:15.623+0000 7f6a41400700 1 --2- 172.21.15.153:0/2734727717 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6a3c069020 0x7f6a3c105790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:15.622 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:15.623+0000 7f6a41400700 1 --2- 172.21.15.153:0/2734727717 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6a3c068670 0x7f6a3c068a50 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:15.622 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:15.623+0000 7f6a41400700 1 -- 172.21.15.153:0/2734727717 >> 172.21.15.153:0/2734727717 conn(0x7f6a3c0755a0 msgr2=0x7f6a3c0759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:35:15.622 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:15.623+0000 7f6a41400700 1 -- 172.21.15.153:0/2734727717 shutdown_connections 2024-03-21T10:35:15.622 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:15.623+0000 7f6a41400700 1 -- 172.21.15.153:0/2734727717 wait complete. 2024-03-21T10:35:15.623 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:15.624+0000 7f6a41400700 1 Processor -- start 2024-03-21T10:35:15.623 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:15.624+0000 7f6a41400700 1 -- start start 2024-03-21T10:35:15.623 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:15.624+0000 7f6a41400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6a3c068670 0x7f6a3c19e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:15.623 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:15.625+0000 7f6a41400700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6a3c069020 0x7f6a3c19edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:15.623 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:15.625+0000 7f6a41400700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a3c19f450 con 0x7f6a3c068670 2024-03-21T10:35:15.623 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:15.625+0000 7f6a41400700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a3c198900 con 0x7f6a3c069020 2024-03-21T10:35:15.624 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:15.625+0000 7f6a3a800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6a3c068670 0x7f6a3c19e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:15.625 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:15.625+0000 7f6a3a800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6a3c068670 0x7f6a3c19e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:53720/0 (socket says 172.21.15.153:53720) 2024-03-21T10:35:15.625 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:15.625+0000 7f6a3a800700 1 -- 172.21.15.153:0/350940546 learned_addr learned my addr 172.21.15.153:0/350940546 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:35:15.625 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:15.625+0000 7f6a31e00700 1 --2- 172.21.15.153:0/350940546 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6a3c069020 0x7f6a3c19edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:15.625 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:15.625+0000 7f6a3a800700 1 -- 172.21.15.153:0/350940546 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6a3c069020 msgr2=0x7f6a3c19edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:15.625 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:15.625+0000 7f6a3a800700 1 --2- 172.21.15.153:0/350940546 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6a3c069020 0x7f6a3c19edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:15.625 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:15.625+0000 7f6a3a800700 1 -- 172.21.15.153:0/350940546 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a2c009d00 con 0x7f6a3c068670 2024-03-21T10:35:15.625 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:15.625+0000 7f6a31e00700 1 --2- 172.21.15.153:0/350940546 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6a3c069020 0x7f6a3c19edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:35:15.625 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:15.626+0000 7f6a3a800700 1 --2- 172.21.15.153:0/350940546 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6a3c068670 0x7f6a3c19e880 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c00b070 tx=0x7f6a2c00cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:35:15.625 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:15.626+0000 7f6a33e00700 1 -- 172.21.15.153:0/350940546 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6a2c00e040 con 0x7f6a3c068670 2024-03-21T10:35:15.625 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:15.626+0000 7f6a33e00700 1 -- 172.21.15.153:0/350940546 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6a2c003e60 con 0x7f6a3c068670 2024-03-21T10:35:15.625 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:15.626+0000 7f6a33e00700 1 -- 172.21.15.153:0/350940546 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6a2c01d910 con 0x7f6a3c068670 2024-03-21T10:35:15.625 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:15.626+0000 7f6a41400700 1 -- 172.21.15.153:0/350940546 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a3c198b80 con 0x7f6a3c068670 2024-03-21T10:35:15.625 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:15.626+0000 7f6a41400700 1 -- 172.21.15.153:0/350940546 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a3c199070 con 0x7f6a3c068670 2024-03-21T10:35:15.626 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:15.627+0000 7f6a41400700 1 -- 172.21.15.153:0/350940546 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a3c04ea90 con 0x7f6a3c068670 2024-03-21T10:35:15.626 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:15.628+0000 7f6a33e00700 1 -- 172.21.15.153:0/350940546 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f6a2c004820 con 0x7f6a3c068670 2024-03-21T10:35:15.628 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:15.629+0000 7f6a33e00700 1 --2- 172.21.15.153:0/350940546 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f6a2006e460 0x7f6a20070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:15.628 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:15.629+0000 7f6a33e00700 1 -- 172.21.15.153:0/350940546 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f6a2c08fdb0 con 0x7f6a3c068670 2024-03-21T10:35:15.628 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:15.629+0000 7f6a31e00700 1 --2- 172.21.15.153:0/350940546 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f6a2006e460 0x7f6a20070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:15.628 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:15.630+0000 7f6a31e00700 1 --2- 172.21.15.153:0/350940546 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f6a2006e460 0x7f6a20070920 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c107730 tx=0x7f6a24008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:35:15.632 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:15.634+0000 7f6a33e00700 1 -- 172.21.15.153:0/350940546 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f6a2c05c980 con 0x7f6a3c068670 2024-03-21T10:35:15.698 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:35:15 smithi174 ceph-mon[88004]: pgmap v703: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:35:15.698 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:35:15 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi174.nhozag"}]: dispatch 2024-03-21T10:35:15.865 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:15.866+0000 7f6a41400700 1 -- 172.21.15.153:0/350940546 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f6a3c199530 con 0x7f6a2006e460 2024-03-21T10:35:15.884 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:15.885+0000 7f6a33e00700 1 -- 172.21.15.153:0/350940546 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f6a3c199530 con 0x7f6a2006e460 2024-03-21T10:35:15.884 INFO:teuthology.orchestra.run.smithi153.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-21T10:35:15.884 INFO:teuthology.orchestra.run.smithi153.stdout:alertmanager.smithi153 smithi153 *:9093,9094 running (24m) 15m ago 37m 14.7M - 0.25.0 c8568f914cd2 9ed3f2200b5f 2024-03-21T10:35:15.884 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi153 smithi153 running (38m) 15m ago 38m 9424k - 18.2.1 5be31c24972a 5b61994396a6 2024-03-21T10:35:15.884 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi174 smithi174 running (36m) 15m ago 36m 9244k - 18.2.1 5be31c24972a 4f7a5fc3ac28 2024-03-21T10:35:15.884 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi153 smithi153 running (20m) 15m ago 38m 6656k - 19.0.0-2231-g2edfee68 237e43c7a65f 32c21104d5b0 2024-03-21T10:35:15.884 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi174 smithi174 running (20m) 15m ago 36m 6660k - 19.0.0-2231-g2edfee68 237e43c7a65f a04e48967ab6 2024-03-21T10:35:15.884 INFO:teuthology.orchestra.run.smithi153.stdout:grafana.smithi153 smithi153 *:3000 running (23m) 15m ago 37m 73.6M - 9.4.12 2bacad6d85d8 ee24026ab786 2024-03-21T10:35:15.885 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.iuyluj smithi153 running (15m) 15m ago 30m 10.9M - 19.0.0-2231-g2edfee68 237e43c7a65f 87aff8a5de70 2024-03-21T10:35:15.885 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.zipckr smithi153 running (16m) 15m ago 30m 11.8M - 19.0.0-2231-g2edfee68 237e43c7a65f ccc262bd5dd3 2024-03-21T10:35:15.885 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.nhozag smithi174 running (30m) 15m ago 30m 12.0M - 18.2.1 5be31c24972a 2fee4000308f 2024-03-21T10:35:15.885 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.yyimst smithi174 running (15m) 15m ago 30m 10.7M - 19.0.0-2231-g2edfee68 237e43c7a65f fc1962f9351e 2024-03-21T10:35:15.885 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi153.tluxni smithi153 *:8443,9283,8765 running (27m) 15m ago 39m 509M - 19.0.0-2231-g2edfee68 237e43c7a65f 7ffee0e55382 2024-03-21T10:35:15.885 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi174.qpcwat smithi174 *:8443,9283,8765 running (25m) 15m ago 36m 452M - 19.0.0-2231-g2edfee68 237e43c7a65f 748968f216ab 2024-03-21T10:35:15.885 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi153 smithi153 running (21m) 15m ago 39m 55.0M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f dd8038303dc3 2024-03-21T10:35:15.885 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi174 smithi174 running (21m) 15m ago 36m 44.2M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f 29d78a788f2c 2024-03-21T10:35:15.885 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi153 smithi153 *:9100 running (25m) 15m ago 37m 7889k - 1.5.0 0da6a335fe13 bb835edc0e0b 2024-03-21T10:35:15.885 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi174 smithi174 *:9100 running (25m) 15m ago 36m 8102k - 1.5.0 0da6a335fe13 7c4ecad790fa 2024-03-21T10:35:15.885 INFO:teuthology.orchestra.run.smithi153.stdout:osd.0 smithi153 running (19m) 15m ago 35m 116M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 507900fd4791 2024-03-21T10:35:15.885 INFO:teuthology.orchestra.run.smithi153.stdout:osd.1 smithi153 running (18m) 15m ago 34m 89.2M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 0d646452e9c1 2024-03-21T10:35:15.885 INFO:teuthology.orchestra.run.smithi153.stdout:osd.2 smithi153 running (18m) 15m ago 34m 87.4M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 1326f2afdfcd 2024-03-21T10:35:15.885 INFO:teuthology.orchestra.run.smithi153.stdout:osd.3 smithi174 running (17m) 15m ago 33m 83.6M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 70aafac71ad5 2024-03-21T10:35:15.885 INFO:teuthology.orchestra.run.smithi153.stdout:osd.4 smithi174 running (17m) 15m ago 32m 92.2M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 0d746ce6e79d 2024-03-21T10:35:15.886 INFO:teuthology.orchestra.run.smithi153.stdout:osd.5 smithi174 running (16m) 15m ago 32m 86.2M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 53e5f35b8cac 2024-03-21T10:35:15.886 INFO:teuthology.orchestra.run.smithi153.stdout:prometheus.smithi153 smithi153 *:9095 running (24m) 15m ago 37m 73.5M - 2.43.0 a07b618ecd1d 7d95561e50c2 2024-03-21T10:35:15.888 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:15.889+0000 7f6a41400700 1 -- 172.21.15.153:0/350940546 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f6a2006e460 msgr2=0x7f6a20070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:15.888 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:15.889+0000 7f6a41400700 1 --2- 172.21.15.153:0/350940546 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f6a2006e460 0x7f6a20070920 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c107730 tx=0x7f6a24008040 comp rx=0 tx=0).stop 2024-03-21T10:35:15.888 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:15.889+0000 7f6a41400700 1 -- 172.21.15.153:0/350940546 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6a3c068670 msgr2=0x7f6a3c19e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:15.888 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:15.889+0000 7f6a41400700 1 --2- 172.21.15.153:0/350940546 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6a3c068670 0x7f6a3c19e880 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c00b070 tx=0x7f6a2c00cdd0 comp rx=0 tx=0).stop 2024-03-21T10:35:15.889 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:15.890+0000 7f6a41400700 1 -- 172.21.15.153:0/350940546 shutdown_connections 2024-03-21T10:35:15.889 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:15.890+0000 7f6a41400700 1 --2- 172.21.15.153:0/350940546 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f6a2006e460 0x7f6a20070920 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:15.889 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:15.890+0000 7f6a41400700 1 --2- 172.21.15.153:0/350940546 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6a3c068670 0x7f6a3c19e880 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:15.889 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:15.890+0000 7f6a41400700 1 --2- 172.21.15.153:0/350940546 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6a3c069020 0x7f6a3c19edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:15.889 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:15.890+0000 7f6a41400700 1 -- 172.21.15.153:0/350940546 >> 172.21.15.153:0/350940546 conn(0x7f6a3c0755a0 msgr2=0x7f6a3c102760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:35:15.889 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:15.890+0000 7f6a41400700 1 -- 172.21.15.153:0/350940546 shutdown_connections 2024-03-21T10:35:15.889 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:15.890+0000 7f6a41400700 1 -- 172.21.15.153:0/350940546 wait complete. 2024-03-21T10:35:16.046 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.047+0000 7f6585c00700 1 -- 172.21.15.153:0/1382308860 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6580069020 msgr2=0x7f6580105790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:16.047 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.047+0000 7f6585c00700 1 --2- 172.21.15.153:0/1382308860 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6580069020 0x7f6580105790 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f6568009910 tx=0x7f6568009c20 comp rx=0 tx=0).stop 2024-03-21T10:35:16.047 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.047+0000 7f6585c00700 1 -- 172.21.15.153:0/1382308860 shutdown_connections 2024-03-21T10:35:16.047 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.047+0000 7f6585c00700 1 --2- 172.21.15.153:0/1382308860 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6580069020 0x7f6580105790 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:16.047 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.047+0000 7f6585c00700 1 --2- 172.21.15.153:0/1382308860 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6580068670 0x7f6580068a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:16.047 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.047+0000 7f6585c00700 1 -- 172.21.15.153:0/1382308860 >> 172.21.15.153:0/1382308860 conn(0x7f65800755a0 msgr2=0x7f65800759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:35:16.048 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.047+0000 7f6585c00700 1 -- 172.21.15.153:0/1382308860 shutdown_connections 2024-03-21T10:35:16.048 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.048+0000 7f6585c00700 1 -- 172.21.15.153:0/1382308860 wait complete. 2024-03-21T10:35:16.050 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.048+0000 7f6585c00700 1 Processor -- start 2024-03-21T10:35:16.050 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.049+0000 7f6585c00700 1 -- start start 2024-03-21T10:35:16.050 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.049+0000 7f6585c00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6580068670 0x7f6580196700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:16.050 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.049+0000 7f6585c00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6580069020 0x7f6580196c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:16.050 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.049+0000 7f6585c00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65801972d0 con 0x7f6580069020 2024-03-21T10:35:16.050 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.049+0000 7f6585c00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f658019a000 con 0x7f6580068670 2024-03-21T10:35:16.051 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.049+0000 7f6577e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6580069020 0x7f6580196c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:16.051 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.049+0000 7f6577e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6580069020 0x7f6580196c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:53734/0 (socket says 172.21.15.153:53734) 2024-03-21T10:35:16.051 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.049+0000 7f6577e00700 1 -- 172.21.15.153:0/4294359870 learned_addr learned my addr 172.21.15.153:0/4294359870 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:35:16.051 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.050+0000 7f657ea00700 1 --2- 172.21.15.153:0/4294359870 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6580068670 0x7f6580196700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:16.051 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.050+0000 7f6577e00700 1 -- 172.21.15.153:0/4294359870 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6580068670 msgr2=0x7f6580196700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:16.051 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.050+0000 7f6577e00700 1 --2- 172.21.15.153:0/4294359870 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6580068670 0x7f6580196700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:16.051 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.050+0000 7f6577e00700 1 -- 172.21.15.153:0/4294359870 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65680095f0 con 0x7f6580069020 2024-03-21T10:35:16.051 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.050+0000 7f6577e00700 1 --2- 172.21.15.153:0/4294359870 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6580069020 0x7f6580196c40 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f656800bda0 tx=0x7f656800be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:35:16.051 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.051+0000 7f6577400700 1 -- 172.21.15.153:0/4294359870 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f656801d070 con 0x7f6580069020 2024-03-21T10:35:16.052 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.051+0000 7f6577400700 1 -- 172.21.15.153:0/4294359870 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6568004fb0 con 0x7f6580069020 2024-03-21T10:35:16.052 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.051+0000 7f6585c00700 1 -- 172.21.15.153:0/4294359870 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f658019a280 con 0x7f6580069020 2024-03-21T10:35:16.052 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.051+0000 7f6577400700 1 -- 172.21.15.153:0/4294359870 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f656800f460 con 0x7f6580069020 2024-03-21T10:35:16.052 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.051+0000 7f6585c00700 1 -- 172.21.15.153:0/4294359870 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f658019a770 con 0x7f6580069020 2024-03-21T10:35:16.054 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.052+0000 7f6585c00700 1 -- 172.21.15.153:0/4294359870 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65801090a0 con 0x7f6580069020 2024-03-21T10:35:16.054 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.053+0000 7f6577400700 1 -- 172.21.15.153:0/4294359870 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f6568022800 con 0x7f6580069020 2024-03-21T10:35:16.054 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.055+0000 7f6577400700 1 --2- 172.21.15.153:0/4294359870 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f656406e460 0x7f6564070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:16.054 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.055+0000 7f6577400700 1 -- 172.21.15.153:0/4294359870 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f656808fa20 con 0x7f6580069020 2024-03-21T10:35:16.054 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.055+0000 7f657ea00700 1 --2- 172.21.15.153:0/4294359870 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f656406e460 0x7f6564070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:16.055 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.056+0000 7f657ea00700 1 --2- 172.21.15.153:0/4294359870 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f656406e460 0x7f6564070920 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f6570005fd0 tx=0x7f657000a560 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:35:16.058 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.060+0000 7f6577400700 1 -- 172.21.15.153:0/4294359870 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f6568058c90 con 0x7f6580069020 2024-03-21T10:35:16.366 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.367+0000 7f6585c00700 1 -- 172.21.15.153:0/4294359870 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f658004ea90 con 0x7f6580069020 2024-03-21T10:35:16.367 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.368+0000 7f6577400700 1 -- 172.21.15.153:0/4294359870 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f656805c350 con 0x7f6580069020 2024-03-21T10:35:16.367 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:35:16.367 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": { 2024-03-21T10:35:16.367 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:35:16.367 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:35:16.367 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": { 2024-03-21T10:35:16.367 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:35:16.367 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:35:16.367 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": { 2024-03-21T10:35:16.367 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 6 2024-03-21T10:35:16.368 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:35:16.368 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": { 2024-03-21T10:35:16.368 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 3 2024-03-21T10:35:16.368 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:35:16.368 INFO:teuthology.orchestra.run.smithi153.stdout: "overall": { 2024-03-21T10:35:16.368 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 13 2024-03-21T10:35:16.368 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-03-21T10:35:16.368 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:35:16.370 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.371+0000 7f6585c00700 1 -- 172.21.15.153:0/4294359870 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f656406e460 msgr2=0x7f6564070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:16.370 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.371+0000 7f6585c00700 1 --2- 172.21.15.153:0/4294359870 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f656406e460 0x7f6564070920 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f6570005fd0 tx=0x7f657000a560 comp rx=0 tx=0).stop 2024-03-21T10:35:16.370 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.371+0000 7f6585c00700 1 -- 172.21.15.153:0/4294359870 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6580069020 msgr2=0x7f6580196c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:16.370 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.371+0000 7f6585c00700 1 --2- 172.21.15.153:0/4294359870 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6580069020 0x7f6580196c40 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f656800bda0 tx=0x7f656800be80 comp rx=0 tx=0).stop 2024-03-21T10:35:16.370 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.371+0000 7f6585c00700 1 -- 172.21.15.153:0/4294359870 shutdown_connections 2024-03-21T10:35:16.370 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.371+0000 7f6585c00700 1 --2- 172.21.15.153:0/4294359870 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6580068670 0x7f6580196700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:16.370 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.371+0000 7f6585c00700 1 --2- 172.21.15.153:0/4294359870 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f656406e460 0x7f6564070920 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:16.370 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.371+0000 7f6585c00700 1 --2- 172.21.15.153:0/4294359870 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6580069020 0x7f6580196c40 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:16.370 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.371+0000 7f6585c00700 1 -- 172.21.15.153:0/4294359870 >> 172.21.15.153:0/4294359870 conn(0x7f65800755a0 msgr2=0x7f6580105000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:35:16.371 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.372+0000 7f6585c00700 1 -- 172.21.15.153:0/4294359870 shutdown_connections 2024-03-21T10:35:16.371 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.372+0000 7f6585c00700 1 -- 172.21.15.153:0/4294359870 wait complete. 2024-03-21T10:35:16.524 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.525+0000 7f8817000700 1 -- 172.21.15.153:0/565870231 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8810101d40 msgr2=0x7f8810102120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:16.524 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.525+0000 7f8817000700 1 --2- 172.21.15.153:0/565870231 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8810101d40 0x7f8810102120 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f8800009910 tx=0x7f8800009c20 comp rx=0 tx=0).stop 2024-03-21T10:35:16.524 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.525+0000 7f8817000700 1 -- 172.21.15.153:0/565870231 shutdown_connections 2024-03-21T10:35:16.524 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.525+0000 7f8817000700 1 --2- 172.21.15.153:0/565870231 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8810102660 0x7f881010ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:16.524 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.525+0000 7f8817000700 1 --2- 172.21.15.153:0/565870231 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8810101d40 0x7f8810102120 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:16.525 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.525+0000 7f8817000700 1 -- 172.21.15.153:0/565870231 >> 172.21.15.153:0/565870231 conn(0x7f88100fb570 msgr2=0x7f88100fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:35:16.525 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.525+0000 7f8817000700 1 -- 172.21.15.153:0/565870231 shutdown_connections 2024-03-21T10:35:16.525 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.525+0000 7f8817000700 1 -- 172.21.15.153:0/565870231 wait complete. 2024-03-21T10:35:16.525 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.526+0000 7f8817000700 1 Processor -- start 2024-03-21T10:35:16.526 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.527+0000 7f8817000700 1 -- start start 2024-03-21T10:35:16.526 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.527+0000 7f8817000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8810101d40 0x7f88100ff9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:16.526 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.527+0000 7f8817000700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8810102660 0x7f88100ffee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:16.526 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.527+0000 7f8817000700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8810100420 con 0x7f8810101d40 2024-03-21T10:35:16.526 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.527+0000 7f8817000700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8810100560 con 0x7f8810102660 2024-03-21T10:35:16.526 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.527+0000 7f880fe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8810101d40 0x7f88100ff9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:16.526 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.527+0000 7f880fe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8810101d40 0x7f88100ff9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:53754/0 (socket says 172.21.15.153:53754) 2024-03-21T10:35:16.526 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.527+0000 7f880fe00700 1 -- 172.21.15.153:0/2373730355 learned_addr learned my addr 172.21.15.153:0/2373730355 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:35:16.527 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.528+0000 7f8807400700 1 --2- 172.21.15.153:0/2373730355 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8810102660 0x7f88100ffee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:16.527 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.528+0000 7f880fe00700 1 -- 172.21.15.153:0/2373730355 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8810102660 msgr2=0x7f88100ffee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:16.527 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.528+0000 7f880fe00700 1 --2- 172.21.15.153:0/2373730355 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8810102660 0x7f88100ffee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:16.527 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.528+0000 7f880fe00700 1 -- 172.21.15.153:0/2373730355 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88000095f0 con 0x7f8810101d40 2024-03-21T10:35:16.528 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.528+0000 7f880fe00700 1 --2- 172.21.15.153:0/2373730355 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8810101d40 0x7f88100ff9a0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f8800006010 tx=0x7f880000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:35:16.528 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.528+0000 7f880d600700 1 -- 172.21.15.153:0/2373730355 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f880001d070 con 0x7f8810101d40 2024-03-21T10:35:16.528 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.528+0000 7f880d600700 1 -- 172.21.15.153:0/2373730355 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8800004fb0 con 0x7f8810101d40 2024-03-21T10:35:16.528 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.528+0000 7f880d600700 1 -- 172.21.15.153:0/2373730355 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f880000f690 con 0x7f8810101d40 2024-03-21T10:35:16.528 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.528+0000 7f8817000700 1 -- 172.21.15.153:0/2373730355 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88101007e0 con 0x7f8810101d40 2024-03-21T10:35:16.528 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.528+0000 7f8817000700 1 -- 172.21.15.153:0/2373730355 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88101a2d40 con 0x7f8810101d40 2024-03-21T10:35:16.528 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.530+0000 7f8817000700 1 -- 172.21.15.153:0/2373730355 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8810108250 con 0x7f8810101d40 2024-03-21T10:35:16.529 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.531+0000 7f880d600700 1 -- 172.21.15.153:0/2373730355 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f88000228a0 con 0x7f8810101d40 2024-03-21T10:35:16.530 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.532+0000 7f880d600700 1 --2- 172.21.15.153:0/2373730355 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f87f006e460 0x7f87f0070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:16.530 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.532+0000 7f880d600700 1 -- 172.21.15.153:0/2373730355 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f880008fd30 con 0x7f8810101d40 2024-03-21T10:35:16.531 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.532+0000 7f8807400700 1 --2- 172.21.15.153:0/2373730355 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f87f006e460 0x7f87f0070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:16.531 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.533+0000 7f8807400700 1 --2- 172.21.15.153:0/2373730355 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f87f006e460 0x7f87f0070920 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f87f8006fd0 tx=0x7f87f8008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:35:16.535 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.537+0000 7f880d600700 1 -- 172.21.15.153:0/2373730355 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f8800058dd0 con 0x7f8810101d40 2024-03-21T10:35:16.698 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:35:16 smithi174 ceph-mon[88004]: from='client.35262 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:35:16.698 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:35:16 smithi174 ceph-mon[88004]: from='client.35266 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:35:16.699 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:35:16 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:35:16.699 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:35:16 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:35:16.803 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:35:16 smithi153 ceph-mon[113733]: from='client.35262 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:35:16.803 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:35:16 smithi153 ceph-mon[113733]: from='client.35266 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:35:16.804 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:35:16 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:35:16.804 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:35:16 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:35:16.804 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.806+0000 7f8817000700 1 -- 172.21.15.153:0/2373730355 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f881004ea90 con 0x7f8810101d40 2024-03-21T10:35:16.804 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.806+0000 7f880d600700 1 -- 172.21.15.153:0/2373730355 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1698 (secure 0 0 0) 0x7f880005c490 con 0x7f8810101d40 2024-03-21T10:35:16.806 INFO:teuthology.orchestra.run.smithi153.stdout:e31 2024-03-21T10:35:16.806 INFO:teuthology.orchestra.run.smithi153.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-21T10:35:16.806 INFO:teuthology.orchestra.run.smithi153.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:35:16.806 INFO:teuthology.orchestra.run.smithi153.stdout:legacy client fscid: 1 2024-03-21T10:35:16.806 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:35:16.806 INFO:teuthology.orchestra.run.smithi153.stdout:Filesystem 'cephfs' (1) 2024-03-21T10:35:16.806 INFO:teuthology.orchestra.run.smithi153.stdout:fs_name cephfs 2024-03-21T10:35:16.806 INFO:teuthology.orchestra.run.smithi153.stdout:epoch 23 2024-03-21T10:35:16.806 INFO:teuthology.orchestra.run.smithi153.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-21T10:35:16.806 INFO:teuthology.orchestra.run.smithi153.stdout:created 2024-03-21T10:04:27.086576+0000 2024-03-21T10:35:16.807 INFO:teuthology.orchestra.run.smithi153.stdout:modified 2024-03-21T10:19:05.022092+0000 2024-03-21T10:35:16.807 INFO:teuthology.orchestra.run.smithi153.stdout:tableserver 0 2024-03-21T10:35:16.807 INFO:teuthology.orchestra.run.smithi153.stdout:root 0 2024-03-21T10:35:16.807 INFO:teuthology.orchestra.run.smithi153.stdout:session_timeout 60 2024-03-21T10:35:16.807 INFO:teuthology.orchestra.run.smithi153.stdout:session_autoclose 300 2024-03-21T10:35:16.807 INFO:teuthology.orchestra.run.smithi153.stdout:max_file_size 1099511627776 2024-03-21T10:35:16.807 INFO:teuthology.orchestra.run.smithi153.stdout:max_xattr_size 65536 2024-03-21T10:35:16.807 INFO:teuthology.orchestra.run.smithi153.stdout:required_client_features {} 2024-03-21T10:35:16.807 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure 0 2024-03-21T10:35:16.807 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure_osd_epoch 94 2024-03-21T10:35:16.807 INFO:teuthology.orchestra.run.smithi153.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:35:16.807 INFO:teuthology.orchestra.run.smithi153.stdout:max_mds 2 2024-03-21T10:35:16.807 INFO:teuthology.orchestra.run.smithi153.stdout:in 0,1 2024-03-21T10:35:16.807 INFO:teuthology.orchestra.run.smithi153.stdout:up {} 2024-03-21T10:35:16.807 INFO:teuthology.orchestra.run.smithi153.stdout:failed 0,1 2024-03-21T10:35:16.808 INFO:teuthology.orchestra.run.smithi153.stdout:damaged 2024-03-21T10:35:16.808 INFO:teuthology.orchestra.run.smithi153.stdout:stopped 2024-03-21T10:35:16.808 INFO:teuthology.orchestra.run.smithi153.stdout:data_pools [3] 2024-03-21T10:35:16.808 INFO:teuthology.orchestra.run.smithi153.stdout:metadata_pool 2 2024-03-21T10:35:16.808 INFO:teuthology.orchestra.run.smithi153.stdout:inline_data enabled 2024-03-21T10:35:16.808 INFO:teuthology.orchestra.run.smithi153.stdout:balancer 2024-03-21T10:35:16.808 INFO:teuthology.orchestra.run.smithi153.stdout:bal_rank_mask -1 2024-03-21T10:35:16.808 INFO:teuthology.orchestra.run.smithi153.stdout:standby_count_wanted 1 2024-03-21T10:35:16.808 INFO:teuthology.orchestra.run.smithi153.stdout:qdb_cluster leader: 0 members: 2024-03-21T10:35:16.808 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:35:16.808 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:35:16.808 INFO:teuthology.orchestra.run.smithi153.stdout:Standby daemons: 2024-03-21T10:35:16.808 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:35:16.808 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.zipckr{-1:34442} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6826/1103846295,v1:172.21.15.153:6827/1103846295] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:35:16.809 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.iuyluj{-1:34474} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6828/1738630832,v1:172.21.15.153:6829/1738630832] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:35:16.809 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.yyimst{-1:44313} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.174:6824/4127786797,v1:172.21.15.174:6825/4127786797] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:35:16.809 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.810+0000 7f8817000700 1 -- 172.21.15.153:0/2373730355 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f87f006e460 msgr2=0x7f87f0070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:16.809 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.810+0000 7f8817000700 1 --2- 172.21.15.153:0/2373730355 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f87f006e460 0x7f87f0070920 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f87f8006fd0 tx=0x7f87f8008040 comp rx=0 tx=0).stop 2024-03-21T10:35:16.809 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.810+0000 7f8817000700 1 -- 172.21.15.153:0/2373730355 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8810101d40 msgr2=0x7f88100ff9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:16.809 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.810+0000 7f8817000700 1 --2- 172.21.15.153:0/2373730355 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8810101d40 0x7f88100ff9a0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f8800006010 tx=0x7f880000be20 comp rx=0 tx=0).stop 2024-03-21T10:35:16.809 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.810+0000 7f8817000700 1 -- 172.21.15.153:0/2373730355 shutdown_connections 2024-03-21T10:35:16.809 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.810+0000 7f8817000700 1 --2- 172.21.15.153:0/2373730355 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f87f006e460 0x7f87f0070920 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:16.810 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.810+0000 7f8817000700 1 --2- 172.21.15.153:0/2373730355 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8810101d40 0x7f88100ff9a0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:16.810 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.810+0000 7f8817000700 1 --2- 172.21.15.153:0/2373730355 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8810102660 0x7f88100ffee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:16.810 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.810+0000 7f8817000700 1 -- 172.21.15.153:0/2373730355 >> 172.21.15.153:0/2373730355 conn(0x7f88100fb570 msgr2=0x7f8810105390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:35:16.810 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.811+0000 7f8817000700 1 -- 172.21.15.153:0/2373730355 shutdown_connections 2024-03-21T10:35:16.810 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.811+0000 7f8817000700 1 -- 172.21.15.153:0/2373730355 wait complete. 2024-03-21T10:35:16.811 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 31 2024-03-21T10:35:16.966 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.967+0000 7f9f5b600700 1 -- 172.21.15.153:0/818057770 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9f54103000 msgr2=0x7f9f541033e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:16.967 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.967+0000 7f9f5b600700 1 --2- 172.21.15.153:0/818057770 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9f54103000 0x7f9f541033e0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f9f48009910 tx=0x7f9f48009c20 comp rx=0 tx=0).stop 2024-03-21T10:35:16.967 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.967+0000 7f9f5b600700 1 -- 172.21.15.153:0/818057770 shutdown_connections 2024-03-21T10:35:16.967 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.967+0000 7f9f5b600700 1 --2- 172.21.15.153:0/818057770 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f541039b0 0x7f9f54107a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:16.967 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.967+0000 7f9f5b600700 1 --2- 172.21.15.153:0/818057770 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9f54103000 0x7f9f541033e0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:16.967 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.967+0000 7f9f5b600700 1 -- 172.21.15.153:0/818057770 >> 172.21.15.153:0/818057770 conn(0x7f9f540fe8d0 msgr2=0x7f9f54100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:35:16.967 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.967+0000 7f9f5b600700 1 -- 172.21.15.153:0/818057770 shutdown_connections 2024-03-21T10:35:16.967 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.967+0000 7f9f5b600700 1 -- 172.21.15.153:0/818057770 wait complete. 2024-03-21T10:35:16.969 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.968+0000 7f9f5b600700 1 Processor -- start 2024-03-21T10:35:16.969 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.968+0000 7f9f5b600700 1 -- start start 2024-03-21T10:35:16.969 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.969+0000 7f9f5b600700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f54103000 0x7f9f5419e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:16.969 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.969+0000 7f9f5b600700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9f541039b0 0x7f9f5419ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:16.969 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.969+0000 7f9f5b600700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5419f330 con 0x7f9f541039b0 2024-03-21T10:35:16.969 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.969+0000 7f9f5b600700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f54198870 con 0x7f9f54103000 2024-03-21T10:35:16.970 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.969+0000 7f9f53e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9f541039b0 0x7f9f5419ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:16.970 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.969+0000 7f9f53e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9f541039b0 0x7f9f5419ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:53768/0 (socket says 172.21.15.153:53768) 2024-03-21T10:35:16.970 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.969+0000 7f9f53e00700 1 -- 172.21.15.153:0/3405400395 learned_addr learned my addr 172.21.15.153:0/3405400395 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:35:16.970 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.969+0000 7f9f58c00700 1 --2- 172.21.15.153:0/3405400395 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f54103000 0x7f9f5419e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:16.970 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.969+0000 7f9f53e00700 1 -- 172.21.15.153:0/3405400395 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f54103000 msgr2=0x7f9f5419e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:16.970 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.969+0000 7f9f53e00700 1 --2- 172.21.15.153:0/3405400395 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f54103000 0x7f9f5419e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:16.970 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.969+0000 7f9f53e00700 1 -- 172.21.15.153:0/3405400395 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f480095f0 con 0x7f9f541039b0 2024-03-21T10:35:16.970 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.970+0000 7f9f53e00700 1 --2- 172.21.15.153:0/3405400395 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9f541039b0 0x7f9f5419ed30 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f9f4000c960 tx=0x7f9f4000cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:35:16.970 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.970+0000 7f9f51600700 1 -- 172.21.15.153:0/3405400395 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f40007a70 con 0x7f9f541039b0 2024-03-21T10:35:16.970 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.970+0000 7f9f51600700 1 -- 172.21.15.153:0/3405400395 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9f40007bd0 con 0x7f9f541039b0 2024-03-21T10:35:16.970 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.970+0000 7f9f51600700 1 -- 172.21.15.153:0/3405400395 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f400186c0 con 0x7f9f541039b0 2024-03-21T10:35:16.970 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.970+0000 7f9f5b600700 1 -- 172.21.15.153:0/3405400395 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f54198b50 con 0x7f9f541039b0 2024-03-21T10:35:16.970 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.970+0000 7f9f5b600700 1 -- 172.21.15.153:0/3405400395 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f541990a0 con 0x7f9f541039b0 2024-03-21T10:35:16.971 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.971+0000 7f9f5b600700 1 -- 172.21.15.153:0/3405400395 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f5410b3a0 con 0x7f9f541039b0 2024-03-21T10:35:16.971 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.972+0000 7f9f51600700 1 -- 172.21.15.153:0/3405400395 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f9f4000faa0 con 0x7f9f541039b0 2024-03-21T10:35:16.971 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.973+0000 7f9f51600700 1 --2- 172.21.15.153:0/3405400395 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f9f3c06e460 0x7f9f3c070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:16.972 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.973+0000 7f9f51600700 1 -- 172.21.15.153:0/3405400395 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f9f4008e520 con 0x7f9f541039b0 2024-03-21T10:35:16.972 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.974+0000 7f9f58c00700 1 --2- 172.21.15.153:0/3405400395 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f9f3c06e460 0x7f9f3c070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:16.973 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.974+0000 7f9f58c00700 1 --2- 172.21.15.153:0/3405400395 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f9f3c06e460 0x7f9f3c070920 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f9f4800b5c0 tx=0x7f9f4801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:35:16.977 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:16.978+0000 7f9f51600700 1 -- 172.21.15.153:0/3405400395 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f9f4005ae60 con 0x7f9f541039b0 2024-03-21T10:35:17.201 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:17.202+0000 7f9f5b600700 1 -- 172.21.15.153:0/3405400395 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9f54199370 con 0x7f9f3c06e460 2024-03-21T10:35:17.203 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:17.205+0000 7f9f51600700 1 -- 172.21.15.153:0/3405400395 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f9f54199370 con 0x7f9f3c06e460 2024-03-21T10:35:17.203 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:35:17.203 INFO:teuthology.orchestra.run.smithi153.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232", 2024-03-21T10:35:17.203 INFO:teuthology.orchestra.run.smithi153.stdout: "in_progress": true, 2024-03-21T10:35:17.204 INFO:teuthology.orchestra.run.smithi153.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-21T10:35:17.204 INFO:teuthology.orchestra.run.smithi153.stdout: "services_complete": [ 2024-03-21T10:35:17.204 INFO:teuthology.orchestra.run.smithi153.stdout: "crash", 2024-03-21T10:35:17.204 INFO:teuthology.orchestra.run.smithi153.stdout: "mon", 2024-03-21T10:35:17.204 INFO:teuthology.orchestra.run.smithi153.stdout: "osd", 2024-03-21T10:35:17.204 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr" 2024-03-21T10:35:17.204 INFO:teuthology.orchestra.run.smithi153.stdout: ], 2024-03-21T10:35:17.204 INFO:teuthology.orchestra.run.smithi153.stdout: "progress": "15/23 daemons upgraded", 2024-03-21T10:35:17.204 INFO:teuthology.orchestra.run.smithi153.stdout: "message": "Currently upgrading mds daemons", 2024-03-21T10:35:17.204 INFO:teuthology.orchestra.run.smithi153.stdout: "is_paused": false 2024-03-21T10:35:17.204 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:35:17.207 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:17.208+0000 7f9f5b600700 1 -- 172.21.15.153:0/3405400395 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f9f3c06e460 msgr2=0x7f9f3c070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:17.207 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:17.208+0000 7f9f5b600700 1 --2- 172.21.15.153:0/3405400395 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f9f3c06e460 0x7f9f3c070920 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f9f4800b5c0 tx=0x7f9f4801a040 comp rx=0 tx=0).stop 2024-03-21T10:35:17.207 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:17.208+0000 7f9f5b600700 1 -- 172.21.15.153:0/3405400395 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9f541039b0 msgr2=0x7f9f5419ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:17.207 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:17.208+0000 7f9f5b600700 1 --2- 172.21.15.153:0/3405400395 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9f541039b0 0x7f9f5419ed30 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f9f4000c960 tx=0x7f9f4000cd20 comp rx=0 tx=0).stop 2024-03-21T10:35:17.207 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:17.208+0000 7f9f5b600700 1 -- 172.21.15.153:0/3405400395 shutdown_connections 2024-03-21T10:35:17.207 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:17.208+0000 7f9f5b600700 1 --2- 172.21.15.153:0/3405400395 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f54103000 0x7f9f5419e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:17.207 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:17.208+0000 7f9f5b600700 1 --2- 172.21.15.153:0/3405400395 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f9f3c06e460 0x7f9f3c070920 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:17.207 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:17.208+0000 7f9f5b600700 1 --2- 172.21.15.153:0/3405400395 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9f541039b0 0x7f9f5419ed30 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:17.207 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:17.208+0000 7f9f5b600700 1 -- 172.21.15.153:0/3405400395 >> 172.21.15.153:0/3405400395 conn(0x7f9f540fe8d0 msgr2=0x7f9f540ffee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:35:17.208 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:17.209+0000 7f9f5b600700 1 -- 172.21.15.153:0/3405400395 shutdown_connections 2024-03-21T10:35:17.208 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:17.209+0000 7f9f5b600700 1 -- 172.21.15.153:0/3405400395 wait complete. 2024-03-21T10:35:17.363 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:17.364+0000 7f16a1e00700 1 -- 172.21.15.153:0/2604252442 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f169c1039b0 msgr2=0x7f169c107a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:17.363 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:17.364+0000 7f16a1e00700 1 --2- 172.21.15.153:0/2604252442 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f169c1039b0 0x7f169c107a00 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f1688009910 tx=0x7f1688009c20 comp rx=0 tx=0).stop 2024-03-21T10:35:17.363 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:17.364+0000 7f16a1e00700 1 -- 172.21.15.153:0/2604252442 shutdown_connections 2024-03-21T10:35:17.363 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:17.364+0000 7f16a1e00700 1 --2- 172.21.15.153:0/2604252442 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f169c1039b0 0x7f169c107a00 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:17.364 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:17.364+0000 7f16a1e00700 1 --2- 172.21.15.153:0/2604252442 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f169c103000 0x7f169c1033e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:17.364 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:17.364+0000 7f16a1e00700 1 -- 172.21.15.153:0/2604252442 >> 172.21.15.153:0/2604252442 conn(0x7f169c0fe8d0 msgr2=0x7f169c100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:35:17.364 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:17.364+0000 7f16a1e00700 1 -- 172.21.15.153:0/2604252442 shutdown_connections 2024-03-21T10:35:17.364 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:17.364+0000 7f16a1e00700 1 -- 172.21.15.153:0/2604252442 wait complete. 2024-03-21T10:35:17.365 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:17.365+0000 7f16a1e00700 1 Processor -- start 2024-03-21T10:35:17.365 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:17.366+0000 7f16a1e00700 1 -- start start 2024-03-21T10:35:17.365 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:17.366+0000 7f16a1e00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f169c103000 0x7f169c19e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:17.365 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:17.366+0000 7f16a1e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f169c1039b0 0x7f169c19ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:17.365 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:17.366+0000 7f16a1e00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f169c19f390 con 0x7f169c1039b0 2024-03-21T10:35:17.365 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:17.366+0000 7f16a1e00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f169c198840 con 0x7f169c103000 2024-03-21T10:35:17.365 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:17.366+0000 7f169a000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f169c1039b0 0x7f169c19ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:17.365 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:17.366+0000 7f169a000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f169c1039b0 0x7f169c19ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:53788/0 (socket says 172.21.15.153:53788) 2024-03-21T10:35:17.365 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:17.366+0000 7f169a000700 1 -- 172.21.15.153:0/2583323067 learned_addr learned my addr 172.21.15.153:0/2583323067 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:35:17.367 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:17.367+0000 7f169aa00700 1 --2- 172.21.15.153:0/2583323067 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f169c103000 0x7f169c19e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:17.367 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:17.367+0000 7f169a000700 1 -- 172.21.15.153:0/2583323067 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f169c103000 msgr2=0x7f169c19e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:17.367 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:17.367+0000 7f169a000700 1 --2- 172.21.15.153:0/2583323067 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f169c103000 0x7f169c19e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:17.367 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:17.367+0000 7f169a000700 1 -- 172.21.15.153:0/2583323067 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16880095f0 con 0x7f169c1039b0 2024-03-21T10:35:17.367 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:17.367+0000 7f169aa00700 1 --2- 172.21.15.153:0/2583323067 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f169c103000 0x7f169c19e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:35:17.367 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:17.367+0000 7f169a000700 1 --2- 172.21.15.153:0/2583323067 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f169c1039b0 0x7f169c19ed00 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f168800bd70 tx=0x7f168800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:35:17.368 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:17.367+0000 7f16a0a00700 1 -- 172.21.15.153:0/2583323067 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f168801d070 con 0x7f169c1039b0 2024-03-21T10:35:17.368 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:17.367+0000 7f16a0a00700 1 -- 172.21.15.153:0/2583323067 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1688005dc0 con 0x7f169c1039b0 2024-03-21T10:35:17.368 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:17.367+0000 7f16a0a00700 1 -- 172.21.15.153:0/2583323067 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1688017400 con 0x7f169c1039b0 2024-03-21T10:35:17.368 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:17.367+0000 7f16a1e00700 1 -- 172.21.15.153:0/2583323067 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f169c198ac0 con 0x7f169c1039b0 2024-03-21T10:35:17.368 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:17.368+0000 7f16a1e00700 1 -- 172.21.15.153:0/2583323067 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f169c198fb0 con 0x7f169c1039b0 2024-03-21T10:35:17.368 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:17.369+0000 7f16a1e00700 1 -- 172.21.15.153:0/2583323067 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f169c10b400 con 0x7f169c1039b0 2024-03-21T10:35:17.368 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:17.370+0000 7f16a0a00700 1 -- 172.21.15.153:0/2583323067 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f168800f7f0 con 0x7f169c1039b0 2024-03-21T10:35:17.369 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:17.371+0000 7f16a0a00700 1 --2- 172.21.15.153:0/2583323067 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f168006e460 0x7f1680070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:17.369 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:17.371+0000 7f16a0a00700 1 -- 172.21.15.153:0/2583323067 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f168808f990 con 0x7f169c1039b0 2024-03-21T10:35:17.369 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:17.371+0000 7f169aa00700 1 --2- 172.21.15.153:0/2583323067 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f168006e460 0x7f1680070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:17.370 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:17.371+0000 7f169aa00700 1 --2- 172.21.15.153:0/2583323067 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f168006e460 0x7f1680070920 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f168c005fd0 tx=0x7f168c005ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:35:17.373 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:17.375+0000 7f16a0a00700 1 -- 172.21.15.153:0/2583323067 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f168805c560 con 0x7f169c1039b0 2024-03-21T10:35:17.553 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:35:17 smithi153 ceph-mon[113733]: pgmap v704: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:35:17.553 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:35:17 smithi153 ceph-mon[113733]: from='client.35270 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:35:17.554 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:35:17 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/4294359870' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:35:17.554 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:35:17 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/2373730355' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:35:17.661 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:35:17 smithi174 ceph-mon[88004]: pgmap v704: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:35:17.661 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:35:17 smithi174 ceph-mon[88004]: from='client.35270 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:35:17.661 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:35:17 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/4294359870' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:35:17.661 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:35:17 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/2373730355' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:35:17.677 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:17.678+0000 7f16a1e00700 1 -- 172.21.15.153:0/2583323067 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f169c04ea90 con 0x7f169c1039b0 2024-03-21T10:35:17.678 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:17.679+0000 7f16a0a00700 1 -- 172.21.15.153:0/2583323067 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4493 (secure 0 0 0) 0x7f168805c0f0 con 0x7f169c1039b0 2024-03-21T10:35:17.678 INFO:teuthology.orchestra.run.smithi153.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-03-21T10:35:17.679 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-03-21T10:35:17.679 INFO:teuthology.orchestra.run.smithi153.stdout: host smithi174 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi174/config 2024-03-21T10:35:17.679 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/podman: timeout after 895 seconds 2024-03-21T10:35:17.679 INFO:teuthology.orchestra.run.smithi153.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 -e NODE_NAME=smithi174 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/run/ceph:z -v /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpd910r3_i:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 inventory --format=json-pretty --filter-for-batch 2024-03-21T10:35:17.679 INFO:teuthology.orchestra.run.smithi153.stdout:Traceback (most recent call last): 2024-03-21T10:35:17.679 INFO:teuthology.orchestra.run.smithi153.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-03-21T10:35:17.679 INFO:teuthology.orchestra.run.smithi153.stdout: return _run_code(code, main_globals, None, 2024-03-21T10:35:17.679 INFO:teuthology.orchestra.run.smithi153.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-03-21T10:35:17.679 INFO:teuthology.orchestra.run.smithi153.stdout: exec(code, run_globals) 2024-03-21T10:35:17.679 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 5606, in 2024-03-21T10:35:17.679 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 5594, in main 2024-03-21T10:35:17.679 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 406, in _infer_config 2024-03-21T10:35:17.679 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 321, in _infer_fsid 2024-03-21T10:35:17.680 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 434, in _infer_image 2024-03-21T10:35:17.680 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 308, in _validate_fsid 2024-03-21T10:35:17.680 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 3357, in command_ceph_volume 2024-03-21T10:35:17.680 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-03-21T10:35:17.680 INFO:teuthology.orchestra.run.smithi153.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 -e NODE_NAME=smithi174 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/run/ceph:z -v /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpd910r3_i:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 inventory --format=json-pretty --filter-for-batch 2024-03-21T10:35:17.680 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-21T10:35:17.680 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is degraded 2024-03-21T10:35:17.680 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-21T10:35:17.680 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-21T10:35:17.680 INFO:teuthology.orchestra.run.smithi153.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-21T10:35:17.680 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-21T10:35:17.681 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:17.682+0000 7f16a1e00700 1 -- 172.21.15.153:0/2583323067 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f168006e460 msgr2=0x7f1680070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:17.681 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:17.682+0000 7f16a1e00700 1 --2- 172.21.15.153:0/2583323067 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f168006e460 0x7f1680070920 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f168c005fd0 tx=0x7f168c005ee0 comp rx=0 tx=0).stop 2024-03-21T10:35:17.681 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:17.682+0000 7f16a1e00700 1 -- 172.21.15.153:0/2583323067 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f169c1039b0 msgr2=0x7f169c19ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:17.681 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:17.682+0000 7f16a1e00700 1 --2- 172.21.15.153:0/2583323067 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f169c1039b0 0x7f169c19ed00 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f168800bd70 tx=0x7f168800be50 comp rx=0 tx=0).stop 2024-03-21T10:35:17.681 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:17.682+0000 7f16a1e00700 1 -- 172.21.15.153:0/2583323067 shutdown_connections 2024-03-21T10:35:17.681 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:17.682+0000 7f16a1e00700 1 --2- 172.21.15.153:0/2583323067 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f169c103000 0x7f169c19e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:17.681 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:17.682+0000 7f16a1e00700 1 --2- 172.21.15.153:0/2583323067 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f168006e460 0x7f1680070920 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:17.682 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:17.682+0000 7f16a1e00700 1 --2- 172.21.15.153:0/2583323067 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f169c1039b0 0x7f169c19ed00 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:17.682 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:17.682+0000 7f16a1e00700 1 -- 172.21.15.153:0/2583323067 >> 172.21.15.153:0/2583323067 conn(0x7f169c0fe8d0 msgr2=0x7f169c100cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:35:17.682 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:17.683+0000 7f16a1e00700 1 -- 172.21.15.153:0/2583323067 shutdown_connections 2024-03-21T10:35:17.682 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:17.683+0000 7f16a1e00700 1 -- 172.21.15.153:0/2583323067 wait complete. 2024-03-21T10:35:18.448 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:35:18 smithi174 ceph-mon[88004]: from='client.35282 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:35:18.449 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:35:18 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/2583323067' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:35:18.803 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:35:18 smithi153 ceph-mon[113733]: from='client.35282 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:35:18.803 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:35:18 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/2583323067' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:35:19.448 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:35:19 smithi174 ceph-mon[88004]: pgmap v705: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:35:19.448 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:35:19 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:35:19.448 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:35:19 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:35:19.803 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:35:19 smithi153 ceph-mon[113733]: pgmap v705: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:35:19.803 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:35:19 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:35:19.803 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:35:19 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:35:20.699 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:35:20 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:35:20.699 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:35:20 smithi174 ceph-mon[88004]: mds.? [v2:172.21.15.174:6826/2767517845,v1:172.21.15.174:6827/2767517845] up:boot 2024-03-21T10:35:20.699 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:35:20 smithi174 ceph-mon[88004]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-03-21T10:35:20.699 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:35:20 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi174.nhozag"}]: dispatch 2024-03-21T10:35:20.699 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:35:20 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:35:20.699 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:35:20 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:35:20.699 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:35:20 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:35:20.699 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:35:20 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:35:20.803 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:35:20 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:35:20.803 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:35:20 smithi153 ceph-mon[113733]: mds.? [v2:172.21.15.174:6826/2767517845,v1:172.21.15.174:6827/2767517845] up:boot 2024-03-21T10:35:20.803 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:35:20 smithi153 ceph-mon[113733]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-03-21T10:35:20.803 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:35:20 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi174.nhozag"}]: dispatch 2024-03-21T10:35:20.803 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:35:20 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:35:20.804 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:35:20 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:35:20.804 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:35:20 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:35:20.804 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:35:20 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:35:21.600 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:35:21 smithi174 ceph-mon[88004]: pgmap v706: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:35:21.804 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:35:21 smithi153 ceph-mon[113733]: pgmap v706: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:35:23.448 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:35:23 smithi174 ceph-mon[88004]: pgmap v707: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:35:23.803 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:35:23 smithi153 ceph-mon[113733]: pgmap v707: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:35:25.698 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:35:25 smithi174 ceph-mon[88004]: pgmap v708: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:35:25.698 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:35:25 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:35:25.699 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:35:25 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:35:25.803 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:35:25 smithi153 ceph-mon[113733]: pgmap v708: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:35:25.803 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:35:25 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:35:25.804 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:35:25 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:35:27.683 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:35:27 smithi174 ceph-mon[88004]: pgmap v709: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:35:27.684 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:35:27 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:35:27.684 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:35:27 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:35:27.684 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:35:27 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:35:27.684 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:35:27 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:35:27.803 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:35:27 smithi153 ceph-mon[113733]: pgmap v709: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:35:27.803 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:35:27 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:35:27.803 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:35:27 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:35:27.803 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:35:27 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:35:27.803 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:35:27 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:35:29.691 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:35:29 smithi153 ceph-mon[113733]: pgmap v710: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:35:29.698 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:35:29 smithi174 ceph-mon[88004]: pgmap v710: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:35:31.698 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:35:31 smithi174 ceph-mon[88004]: pgmap v711: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:35:31.699 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:35:31 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:35:31.803 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:35:31 smithi153 ceph-mon[113733]: pgmap v711: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:35:31.803 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:35:31 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:35:33.742 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:35:33 smithi153 ceph-mon[113733]: pgmap v712: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:35:33.948 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:35:33 smithi174 ceph-mon[88004]: pgmap v712: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:35:35.803 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:35:35 smithi153 ceph-mon[113733]: pgmap v713: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:35:35.948 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:35:35 smithi174 ceph-mon[88004]: pgmap v713: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:35:37.803 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:35:37 smithi153 ceph-mon[113733]: pgmap v714: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:35:37.809 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:35:37 smithi174 ceph-mon[88004]: pgmap v714: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:35:39.804 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:35:39 smithi153 ceph-mon[113733]: pgmap v715: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:35:39.948 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:35:39 smithi174 ceph-mon[88004]: pgmap v715: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:35:41.803 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:35:41 smithi153 ceph-mon[113733]: pgmap v716: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:35:41.948 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:35:41 smithi174 ceph-mon[88004]: pgmap v716: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:35:42.646 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:35:42 smithi153 ceph-mon[113733]: pgmap v717: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:35:42.948 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:35:42 smithi174 ceph-mon[88004]: pgmap v717: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:35:45.053 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:35:44 smithi153 ceph-mon[113733]: pgmap v718: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:35:45.198 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:35:44 smithi174 ceph-mon[88004]: pgmap v718: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:35:47.053 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:35:46 smithi153 ceph-mon[113733]: pgmap v719: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:35:47.054 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:35:46 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:35:47.198 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:35:46 smithi174 ceph-mon[88004]: pgmap v719: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:35:47.198 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:35:46 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:35:47.855 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:47.857+0000 7fd22e000700 1 -- 172.21.15.153:0/1829467674 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd228103000 msgr2=0x7fd2281033e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:47.855 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:47.857+0000 7fd22e000700 1 --2- 172.21.15.153:0/1829467674 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd228103000 0x7fd2281033e0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fd218009910 tx=0x7fd218009c20 comp rx=0 tx=0).stop 2024-03-21T10:35:47.855 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:47.857+0000 7fd22e000700 1 -- 172.21.15.153:0/1829467674 shutdown_connections 2024-03-21T10:35:47.855 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:47.857+0000 7fd22e000700 1 --2- 172.21.15.153:0/1829467674 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd2281039b0 0x7fd228107a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:47.855 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:47.857+0000 7fd22e000700 1 --2- 172.21.15.153:0/1829467674 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd228103000 0x7fd2281033e0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:47.855 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:47.857+0000 7fd22e000700 1 -- 172.21.15.153:0/1829467674 >> 172.21.15.153:0/1829467674 conn(0x7fd2280fe8d0 msgr2=0x7fd228100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:35:47.855 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:47.857+0000 7fd22e000700 1 -- 172.21.15.153:0/1829467674 shutdown_connections 2024-03-21T10:35:47.855 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:47.857+0000 7fd22e000700 1 -- 172.21.15.153:0/1829467674 wait complete. 2024-03-21T10:35:47.857 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:47.858+0000 7fd22e000700 1 Processor -- start 2024-03-21T10:35:47.857 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:47.858+0000 7fd22e000700 1 -- start start 2024-03-21T10:35:47.857 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:47.859+0000 7fd22e000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd228103000 0x7fd22819e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:47.857 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:47.859+0000 7fd22e000700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd2281039b0 0x7fd22819ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:47.857 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:47.859+0000 7fd22e000700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd22819f3c0 con 0x7fd228103000 2024-03-21T10:35:47.857 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:47.859+0000 7fd22e000700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd228198870 con 0x7fd2281039b0 2024-03-21T10:35:47.857 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:47.859+0000 7fd227400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd228103000 0x7fd22819e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:47.857 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:47.859+0000 7fd227400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd228103000 0x7fd22819e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:57752/0 (socket says 172.21.15.153:57752) 2024-03-21T10:35:47.857 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:47.859+0000 7fd227400700 1 -- 172.21.15.153:0/3655263693 learned_addr learned my addr 172.21.15.153:0/3655263693 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:35:47.857 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:47.859+0000 7fd226a00700 1 --2- 172.21.15.153:0/3655263693 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd2281039b0 0x7fd22819ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:47.858 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:47.860+0000 7fd227400700 1 -- 172.21.15.153:0/3655263693 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd2281039b0 msgr2=0x7fd22819ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:47.858 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:47.860+0000 7fd227400700 1 --2- 172.21.15.153:0/3655263693 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd2281039b0 0x7fd22819ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:47.858 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:47.860+0000 7fd227400700 1 -- 172.21.15.153:0/3655263693 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2180095f0 con 0x7fd228103000 2024-03-21T10:35:47.858 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:47.860+0000 7fd227400700 1 --2- 172.21.15.153:0/3655263693 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd228103000 0x7fd22819e7f0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fd21800f690 tx=0x7fd21800f6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:35:47.858 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:47.861+0000 7fd21fe00700 1 -- 172.21.15.153:0/3655263693 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd21801d070 con 0x7fd228103000 2024-03-21T10:35:47.858 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:47.861+0000 7fd21fe00700 1 -- 172.21.15.153:0/3655263693 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd2180050d0 con 0x7fd228103000 2024-03-21T10:35:47.859 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:47.861+0000 7fd22e000700 1 -- 172.21.15.153:0/3655263693 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd228198af0 con 0x7fd228103000 2024-03-21T10:35:47.859 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:47.861+0000 7fd21fe00700 1 -- 172.21.15.153:0/3655263693 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd218022470 con 0x7fd228103000 2024-03-21T10:35:47.859 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:47.861+0000 7fd22e000700 1 -- 172.21.15.153:0/3655263693 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd228198fe0 con 0x7fd228103000 2024-03-21T10:35:47.859 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:47.862+0000 7fd22e000700 1 -- 172.21.15.153:0/3655263693 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd22810b3a0 con 0x7fd228103000 2024-03-21T10:35:47.860 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:47.863+0000 7fd21fe00700 1 -- 172.21.15.153:0/3655263693 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fd218003780 con 0x7fd228103000 2024-03-21T10:35:47.862 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:47.864+0000 7fd21fe00700 1 --2- 172.21.15.153:0/3655263693 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd20c06e460 0x7fd20c070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:47.862 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:47.864+0000 7fd21fe00700 1 -- 172.21.15.153:0/3655263693 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fd21808f870 con 0x7fd228103000 2024-03-21T10:35:47.862 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:47.864+0000 7fd226a00700 1 --2- 172.21.15.153:0/3655263693 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd20c06e460 0x7fd20c070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:47.862 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:47.865+0000 7fd226a00700 1 --2- 172.21.15.153:0/3655263693 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd20c06e460 0x7fd20c070920 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fd2280ff0a0 tx=0x7fd214009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:35:47.866 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:47.869+0000 7fd21fe00700 1 -- 172.21.15.153:0/3655263693 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd21805c590 con 0x7fd228103000 2024-03-21T10:35:48.110 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:48.113+0000 7fd22e000700 1 -- 172.21.15.153:0/3655263693 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd228199250 con 0x7fd20c06e460 2024-03-21T10:35:48.112 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:48.115+0000 7fd21fe00700 1 -- 172.21.15.153:0/3655263693 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fd228199250 con 0x7fd20c06e460 2024-03-21T10:35:48.115 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:48.117+0000 7fd22e000700 1 -- 172.21.15.153:0/3655263693 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd20c06e460 msgr2=0x7fd20c070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:48.115 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:48.117+0000 7fd22e000700 1 --2- 172.21.15.153:0/3655263693 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd20c06e460 0x7fd20c070920 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fd2280ff0a0 tx=0x7fd214009260 comp rx=0 tx=0).stop 2024-03-21T10:35:48.115 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:48.117+0000 7fd22e000700 1 -- 172.21.15.153:0/3655263693 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd228103000 msgr2=0x7fd22819e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:48.115 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:48.117+0000 7fd22e000700 1 --2- 172.21.15.153:0/3655263693 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd228103000 0x7fd22819e7f0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fd21800f690 tx=0x7fd21800f6c0 comp rx=0 tx=0).stop 2024-03-21T10:35:48.116 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:48.118+0000 7fd22e000700 1 -- 172.21.15.153:0/3655263693 shutdown_connections 2024-03-21T10:35:48.116 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:48.118+0000 7fd22e000700 1 --2- 172.21.15.153:0/3655263693 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd20c06e460 0x7fd20c070920 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:48.116 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:48.118+0000 7fd22e000700 1 --2- 172.21.15.153:0/3655263693 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd228103000 0x7fd22819e7f0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:48.116 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:48.118+0000 7fd22e000700 1 --2- 172.21.15.153:0/3655263693 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd2281039b0 0x7fd22819ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:48.116 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:48.118+0000 7fd22e000700 1 -- 172.21.15.153:0/3655263693 >> 172.21.15.153:0/3655263693 conn(0x7fd2280fe8d0 msgr2=0x7fd2280ffee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:35:48.116 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:48.118+0000 7fd22e000700 1 -- 172.21.15.153:0/3655263693 shutdown_connections 2024-03-21T10:35:48.116 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:48.118+0000 7fd22e000700 1 -- 172.21.15.153:0/3655263693 wait complete. 2024-03-21T10:35:48.132 INFO:teuthology.orchestra.run.smithi153.stdout:true 2024-03-21T10:35:48.270 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:48.272+0000 7fa58ca00700 1 -- 172.21.15.153:0/2623954575 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa588069020 msgr2=0x7fa588105790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:48.270 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:48.272+0000 7fa58ca00700 1 --2- 172.21.15.153:0/2623954575 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa588069020 0x7fa588105790 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fa570009910 tx=0x7fa570009c20 comp rx=0 tx=0).stop 2024-03-21T10:35:48.270 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:48.272+0000 7fa58ca00700 1 -- 172.21.15.153:0/2623954575 shutdown_connections 2024-03-21T10:35:48.271 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:48.272+0000 7fa58ca00700 1 --2- 172.21.15.153:0/2623954575 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa588069020 0x7fa588105790 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:48.271 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:48.272+0000 7fa58ca00700 1 --2- 172.21.15.153:0/2623954575 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa588068670 0x7fa588068a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:48.271 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:48.272+0000 7fa58ca00700 1 -- 172.21.15.153:0/2623954575 >> 172.21.15.153:0/2623954575 conn(0x7fa5880755a0 msgr2=0x7fa5880759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:35:48.271 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:48.273+0000 7fa58ca00700 1 -- 172.21.15.153:0/2623954575 shutdown_connections 2024-03-21T10:35:48.271 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:48.273+0000 7fa58ca00700 1 -- 172.21.15.153:0/2623954575 wait complete. 2024-03-21T10:35:48.272 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:48.274+0000 7fa58ca00700 1 Processor -- start 2024-03-21T10:35:48.272 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:48.274+0000 7fa58ca00700 1 -- start start 2024-03-21T10:35:48.272 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:48.274+0000 7fa58ca00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa588068670 0x7fa5880ffca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:48.273 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:48.274+0000 7fa58ca00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa588069020 0x7fa5881001e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:48.273 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:48.274+0000 7fa58ca00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa588100720 con 0x7fa588068670 2024-03-21T10:35:48.273 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:48.274+0000 7fa58ca00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa588100860 con 0x7fa588069020 2024-03-21T10:35:48.273 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:48.274+0000 7fa585e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa588068670 0x7fa5880ffca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:48.273 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:48.275+0000 7fa585e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa588068670 0x7fa5880ffca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:57758/0 (socket says 172.21.15.153:57758) 2024-03-21T10:35:48.273 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:48.275+0000 7fa585e00700 1 -- 172.21.15.153:0/3868027174 learned_addr learned my addr 172.21.15.153:0/3868027174 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:35:48.273 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:48.275+0000 7fa57d400700 1 --2- 172.21.15.153:0/3868027174 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa588069020 0x7fa5881001e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:48.273 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:48.275+0000 7fa585e00700 1 -- 172.21.15.153:0/3868027174 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa588069020 msgr2=0x7fa5881001e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:48.273 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:48.275+0000 7fa585e00700 1 --2- 172.21.15.153:0/3868027174 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa588069020 0x7fa5881001e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:48.273 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:48.275+0000 7fa585e00700 1 -- 172.21.15.153:0/3868027174 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5700095f0 con 0x7fa588068670 2024-03-21T10:35:48.273 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:48.275+0000 7fa57d400700 1 --2- 172.21.15.153:0/3868027174 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa588069020 0x7fa5881001e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:35:48.273 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:48.275+0000 7fa585e00700 1 --2- 172.21.15.153:0/3868027174 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa588068670 0x7fa5880ffca0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fa57800b4b0 tx=0x7fa57800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:35:48.274 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:48.275+0000 7fa57f400700 1 -- 172.21.15.153:0/3868027174 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa578011800 con 0x7fa588068670 2024-03-21T10:35:48.274 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:48.276+0000 7fa57f400700 1 -- 172.21.15.153:0/3868027174 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa578011e20 con 0x7fa588068670 2024-03-21T10:35:48.274 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:48.276+0000 7fa57f400700 1 -- 172.21.15.153:0/3868027174 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa57800f2e0 con 0x7fa588068670 2024-03-21T10:35:48.274 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:48.276+0000 7fa58ca00700 1 -- 172.21.15.153:0/3868027174 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5881a2a50 con 0x7fa588068670 2024-03-21T10:35:48.275 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:48.276+0000 7fa58ca00700 1 -- 172.21.15.153:0/3868027174 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5881a2e60 con 0x7fa588068670 2024-03-21T10:35:48.275 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:48.277+0000 7fa58ca00700 1 -- 172.21.15.153:0/3868027174 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5881090c0 con 0x7fa588068670 2024-03-21T10:35:48.275 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:48.278+0000 7fa57f400700 1 -- 172.21.15.153:0/3868027174 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fa578010380 con 0x7fa588068670 2024-03-21T10:35:48.276 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:48.279+0000 7fa57f400700 1 --2- 172.21.15.153:0/3868027174 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa56c06e460 0x7fa56c070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:48.276 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:48.279+0000 7fa57f400700 1 -- 172.21.15.153:0/3868027174 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fa57808e020 con 0x7fa588068670 2024-03-21T10:35:48.277 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:48.279+0000 7fa57d400700 1 --2- 172.21.15.153:0/3868027174 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa56c06e460 0x7fa56c070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:48.277 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:48.280+0000 7fa57d400700 1 --2- 172.21.15.153:0/3868027174 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa56c06e460 0x7fa56c070920 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fa57000b5c0 tx=0x7fa570005480 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:35:48.281 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:48.284+0000 7fa57f400700 1 -- 172.21.15.153:0/3868027174 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa578057160 con 0x7fa588068670 2024-03-21T10:35:48.523 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:48.525+0000 7fa58ca00700 1 -- 172.21.15.153:0/3868027174 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa58802cd80 con 0x7fa56c06e460 2024-03-21T10:35:48.526 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:48.528+0000 7fa57f400700 1 -- 172.21.15.153:0/3868027174 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fa58802cd80 con 0x7fa56c06e460 2024-03-21T10:35:48.529 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:48.531+0000 7fa58ca00700 1 -- 172.21.15.153:0/3868027174 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa56c06e460 msgr2=0x7fa56c070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:48.529 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:48.531+0000 7fa58ca00700 1 --2- 172.21.15.153:0/3868027174 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa56c06e460 0x7fa56c070920 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fa57000b5c0 tx=0x7fa570005480 comp rx=0 tx=0).stop 2024-03-21T10:35:48.529 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:48.531+0000 7fa58ca00700 1 -- 172.21.15.153:0/3868027174 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa588068670 msgr2=0x7fa5880ffca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:48.529 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:48.531+0000 7fa58ca00700 1 --2- 172.21.15.153:0/3868027174 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa588068670 0x7fa5880ffca0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fa57800b4b0 tx=0x7fa57800b870 comp rx=0 tx=0).stop 2024-03-21T10:35:48.529 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:48.532+0000 7fa58ca00700 1 -- 172.21.15.153:0/3868027174 shutdown_connections 2024-03-21T10:35:48.529 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:48.532+0000 7fa58ca00700 1 --2- 172.21.15.153:0/3868027174 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa56c06e460 0x7fa56c070920 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:48.530 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:48.532+0000 7fa58ca00700 1 --2- 172.21.15.153:0/3868027174 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa588068670 0x7fa5880ffca0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:48.530 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:48.532+0000 7fa58ca00700 1 --2- 172.21.15.153:0/3868027174 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa588069020 0x7fa5881001e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:48.530 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:48.532+0000 7fa58ca00700 1 -- 172.21.15.153:0/3868027174 >> 172.21.15.153:0/3868027174 conn(0x7fa5880755a0 msgr2=0x7fa5880fe870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:35:48.530 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:48.532+0000 7fa58ca00700 1 -- 172.21.15.153:0/3868027174 shutdown_connections 2024-03-21T10:35:48.530 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:48.532+0000 7fa58ca00700 1 -- 172.21.15.153:0/3868027174 wait complete. 2024-03-21T10:35:48.690 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:48.692+0000 7feb12200700 1 -- 172.21.15.153:0/1308087259 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feb0c105b50 msgr2=0x7feb0c105f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:48.691 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:48.692+0000 7feb12200700 1 --2- 172.21.15.153:0/1308087259 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feb0c105b50 0x7feb0c105f30 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7feafc009910 tx=0x7feafc009c20 comp rx=0 tx=0).stop 2024-03-21T10:35:48.691 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:48.693+0000 7feb12200700 1 -- 172.21.15.153:0/1308087259 shutdown_connections 2024-03-21T10:35:48.691 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:48.693+0000 7feb12200700 1 --2- 172.21.15.153:0/1308087259 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb0c0fef70 0x7feb0c0ff3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:48.691 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:48.693+0000 7feb12200700 1 --2- 172.21.15.153:0/1308087259 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feb0c105b50 0x7feb0c105f30 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:48.691 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:48.693+0000 7feb12200700 1 -- 172.21.15.153:0/1308087259 >> 172.21.15.153:0/1308087259 conn(0x7feb0c075e30 msgr2=0x7feb0c0fd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:35:48.691 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:48.693+0000 7feb12200700 1 -- 172.21.15.153:0/1308087259 shutdown_connections 2024-03-21T10:35:48.691 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:48.693+0000 7feb12200700 1 -- 172.21.15.153:0/1308087259 wait complete. 2024-03-21T10:35:48.693 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:48.694+0000 7feb12200700 1 Processor -- start 2024-03-21T10:35:48.693 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:48.694+0000 7feb12200700 1 -- start start 2024-03-21T10:35:48.693 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:48.694+0000 7feb12200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feb0c0fef70 0x7feb0c19e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:48.693 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:48.694+0000 7feb12200700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb0c105b50 0x7feb0c19ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:48.693 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:48.694+0000 7feb12200700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb0c19f360 con 0x7feb0c0fef70 2024-03-21T10:35:48.693 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:48.694+0000 7feb12200700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb0c198810 con 0x7feb0c105b50 2024-03-21T10:35:48.693 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:48.695+0000 7feb0b400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feb0c0fef70 0x7feb0c19e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:48.693 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:48.695+0000 7feb0b400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feb0c0fef70 0x7feb0c19e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:57786/0 (socket says 172.21.15.153:57786) 2024-03-21T10:35:48.693 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:48.695+0000 7feb0b400700 1 -- 172.21.15.153:0/3733871592 learned_addr learned my addr 172.21.15.153:0/3733871592 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:35:48.693 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:48.695+0000 7feb0b400700 1 -- 172.21.15.153:0/3733871592 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb0c105b50 msgr2=0x7feb0c19ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:48.693 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:48.695+0000 7feb02a00700 1 --2- 172.21.15.153:0/3733871592 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb0c105b50 0x7feb0c19ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:48.693 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:48.695+0000 7feb0b400700 1 --2- 172.21.15.153:0/3733871592 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb0c105b50 0x7feb0c19ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:48.693 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:48.695+0000 7feb0b400700 1 -- 172.21.15.153:0/3733871592 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feafc0095f0 con 0x7feb0c0fef70 2024-03-21T10:35:48.694 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:48.695+0000 7feb0b400700 1 --2- 172.21.15.153:0/3733871592 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feb0c0fef70 0x7feb0c19e790 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7feafc00b5c0 tx=0x7feafc00be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:35:48.694 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:48.696+0000 7feb08c00700 1 -- 172.21.15.153:0/3733871592 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feafc01d070 con 0x7feb0c0fef70 2024-03-21T10:35:48.695 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:48.696+0000 7feb08c00700 1 -- 172.21.15.153:0/3733871592 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7feafc00f460 con 0x7feb0c0fef70 2024-03-21T10:35:48.695 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:48.696+0000 7feb12200700 1 -- 172.21.15.153:0/3733871592 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb0c198a90 con 0x7feb0c0fef70 2024-03-21T10:35:48.695 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:48.696+0000 7feb08c00700 1 -- 172.21.15.153:0/3733871592 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feafc017610 con 0x7feb0c0fef70 2024-03-21T10:35:48.695 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:48.696+0000 7feb12200700 1 -- 172.21.15.153:0/3733871592 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb0c198f80 con 0x7feb0c0fef70 2024-03-21T10:35:48.695 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:48.697+0000 7feb12200700 1 -- 172.21.15.153:0/3733871592 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb0c0fd980 con 0x7feb0c0fef70 2024-03-21T10:35:48.695 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:48.698+0000 7feb08c00700 1 -- 172.21.15.153:0/3733871592 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7feafc00f5d0 con 0x7feb0c0fef70 2024-03-21T10:35:48.696 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:48.699+0000 7feb08c00700 1 --2- 172.21.15.153:0/3733871592 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7feaf006e460 0x7feaf0070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:48.697 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:48.700+0000 7feb08c00700 1 -- 172.21.15.153:0/3733871592 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7feafc08fb90 con 0x7feb0c0fef70 2024-03-21T10:35:48.697 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:48.700+0000 7feb02a00700 1 --2- 172.21.15.153:0/3733871592 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7feaf006e460 0x7feaf0070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:48.698 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:48.700+0000 7feb02a00700 1 --2- 172.21.15.153:0/3733871592 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7feaf006e460 0x7feaf0070920 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7feb0c107240 tx=0x7feaf4005cf0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:35:48.702 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:48.704+0000 7feb08c00700 1 -- 172.21.15.153:0/3733871592 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7feafc05c6e0 con 0x7feb0c0fef70 2024-03-21T10:35:48.803 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:35:48 smithi153 ceph-mon[113733]: pgmap v720: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:35:48.803 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:35:48 smithi153 ceph-mon[113733]: from='client.35294 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:35:48.936 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:48.938+0000 7feb12200700 1 -- 172.21.15.153:0/3733871592 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7feb0c1991a0 con 0x7feaf006e460 2024-03-21T10:35:48.967 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:48.969+0000 7feb08c00700 1 -- 172.21.15.153:0/3733871592 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7feb0c1991a0 con 0x7feaf006e460 2024-03-21T10:35:48.967 INFO:teuthology.orchestra.run.smithi153.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-21T10:35:48.967 INFO:teuthology.orchestra.run.smithi153.stdout:alertmanager.smithi153 smithi153 *:9093,9094 running (24m) 21s ago 38m 15.6M - 0.25.0 c8568f914cd2 9ed3f2200b5f 2024-03-21T10:35:48.967 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi153 smithi153 running (38m) 21s ago 38m 10.4M - 18.2.1 5be31c24972a 5b61994396a6 2024-03-21T10:35:48.968 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi174 smithi174 running (37m) 24s ago 37m 10.1M - 18.2.1 5be31c24972a 4f7a5fc3ac28 2024-03-21T10:35:48.968 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi153 smithi153 running (20m) 21s ago 38m 6656k - 19.0.0-2231-g2edfee68 237e43c7a65f 32c21104d5b0 2024-03-21T10:35:48.968 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi174 smithi174 running (20m) 24s ago 37m 6660k - 19.0.0-2231-g2edfee68 237e43c7a65f a04e48967ab6 2024-03-21T10:35:48.968 INFO:teuthology.orchestra.run.smithi153.stdout:grafana.smithi153 smithi153 *:3000 running (24m) 21s ago 37m 75.2M - 9.4.12 2bacad6d85d8 ee24026ab786 2024-03-21T10:35:48.968 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.iuyluj smithi153 running (16m) 21s ago 31m 18.7M - 19.0.0-2231-g2edfee68 237e43c7a65f 87aff8a5de70 2024-03-21T10:35:48.968 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.zipckr smithi153 running (16m) 21s ago 31m 18.9M - 19.0.0-2231-g2edfee68 237e43c7a65f ccc262bd5dd3 2024-03-21T10:35:48.968 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.nhozag smithi174 running (30s) 24s ago 31m 10.9M - 19.0.0-2231-g2edfee68 237e43c7a65f 9e02aea72c96 2024-03-21T10:35:48.968 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.yyimst smithi174 running (15m) 24s ago 31m 18.5M - 19.0.0-2231-g2edfee68 237e43c7a65f fc1962f9351e 2024-03-21T10:35:48.968 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi153.tluxni smithi153 *:8443,9283,8765 running (27m) 21s ago 40m 512M - 19.0.0-2231-g2edfee68 237e43c7a65f 7ffee0e55382 2024-03-21T10:35:48.968 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi174.qpcwat smithi174 *:8443,9283,8765 running (26m) 24s ago 36m 459M - 19.0.0-2231-g2edfee68 237e43c7a65f 748968f216ab 2024-03-21T10:35:48.968 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi153 smithi153 running (22m) 21s ago 40m 70.8M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f dd8038303dc3 2024-03-21T10:35:48.968 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi174 smithi174 running (21m) 24s ago 36m 51.9M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f 29d78a788f2c 2024-03-21T10:35:48.968 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi153 smithi153 *:9100 running (25m) 21s ago 38m 8140k - 1.5.0 0da6a335fe13 bb835edc0e0b 2024-03-21T10:35:48.968 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi174 smithi174 *:9100 running (25m) 24s ago 36m 8610k - 1.5.0 0da6a335fe13 7c4ecad790fa 2024-03-21T10:35:48.968 INFO:teuthology.orchestra.run.smithi153.stdout:osd.0 smithi153 running (20m) 21s ago 35m 118M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 507900fd4791 2024-03-21T10:35:48.968 INFO:teuthology.orchestra.run.smithi153.stdout:osd.1 smithi153 running (19m) 21s ago 35m 93.0M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 0d646452e9c1 2024-03-21T10:35:48.969 INFO:teuthology.orchestra.run.smithi153.stdout:osd.2 smithi153 running (18m) 21s ago 34m 91.2M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 1326f2afdfcd 2024-03-21T10:35:48.969 INFO:teuthology.orchestra.run.smithi153.stdout:osd.3 smithi174 running (18m) 24s ago 34m 87.5M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 70aafac71ad5 2024-03-21T10:35:48.969 INFO:teuthology.orchestra.run.smithi153.stdout:osd.4 smithi174 running (17m) 24s ago 33m 95.8M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 0d746ce6e79d 2024-03-21T10:35:48.969 INFO:teuthology.orchestra.run.smithi153.stdout:osd.5 smithi174 running (17m) 24s ago 32m 90.3M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 53e5f35b8cac 2024-03-21T10:35:48.969 INFO:teuthology.orchestra.run.smithi153.stdout:prometheus.smithi153 smithi153 *:9095 running (25m) 21s ago 37m 88.1M - 2.43.0 a07b618ecd1d 7d95561e50c2 2024-03-21T10:35:48.970 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:48.972+0000 7feb12200700 1 -- 172.21.15.153:0/3733871592 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7feaf006e460 msgr2=0x7feaf0070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:48.970 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:48.972+0000 7feb12200700 1 --2- 172.21.15.153:0/3733871592 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7feaf006e460 0x7feaf0070920 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7feb0c107240 tx=0x7feaf4005cf0 comp rx=0 tx=0).stop 2024-03-21T10:35:48.970 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:48.972+0000 7feb12200700 1 -- 172.21.15.153:0/3733871592 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feb0c0fef70 msgr2=0x7feb0c19e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:48.970 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:48.972+0000 7feb12200700 1 --2- 172.21.15.153:0/3733871592 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feb0c0fef70 0x7feb0c19e790 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7feafc00b5c0 tx=0x7feafc00be00 comp rx=0 tx=0).stop 2024-03-21T10:35:48.970 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:48.972+0000 7feb12200700 1 -- 172.21.15.153:0/3733871592 shutdown_connections 2024-03-21T10:35:48.970 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:48.972+0000 7feb12200700 1 --2- 172.21.15.153:0/3733871592 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7feaf006e460 0x7feaf0070920 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:48.970 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:48.972+0000 7feb12200700 1 --2- 172.21.15.153:0/3733871592 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feb0c0fef70 0x7feb0c19e790 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:48.970 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:48.972+0000 7feb12200700 1 --2- 172.21.15.153:0/3733871592 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb0c105b50 0x7feb0c19ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:48.970 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:48.972+0000 7feb12200700 1 -- 172.21.15.153:0/3733871592 >> 172.21.15.153:0/3733871592 conn(0x7feb0c075e30 msgr2=0x7feb0c104500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:35:48.971 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:48.972+0000 7feb12200700 1 -- 172.21.15.153:0/3733871592 shutdown_connections 2024-03-21T10:35:48.971 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:48.973+0000 7feb12200700 1 -- 172.21.15.153:0/3733871592 wait complete. 2024-03-21T10:35:49.124 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:49.126+0000 7fec6e000700 1 -- 172.21.15.153:0/142287683 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec68102fc0 msgr2=0x7fec681033a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:49.125 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:49.126+0000 7fec6e000700 1 --2- 172.21.15.153:0/142287683 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec68102fc0 0x7fec681033a0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fec58009910 tx=0x7fec58009c20 comp rx=0 tx=0).stop 2024-03-21T10:35:49.125 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:49.127+0000 7fec6e000700 1 -- 172.21.15.153:0/142287683 shutdown_connections 2024-03-21T10:35:49.125 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:49.127+0000 7fec6e000700 1 --2- 172.21.15.153:0/142287683 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fec68103970 0x7fec681079c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:49.125 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:49.127+0000 7fec6e000700 1 --2- 172.21.15.153:0/142287683 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec68102fc0 0x7fec681033a0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:49.126 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:49.127+0000 7fec6e000700 1 -- 172.21.15.153:0/142287683 >> 172.21.15.153:0/142287683 conn(0x7fec680fe870 msgr2=0x7fec68100c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:35:49.126 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:49.127+0000 7fec6e000700 1 -- 172.21.15.153:0/142287683 shutdown_connections 2024-03-21T10:35:49.126 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:49.127+0000 7fec6e000700 1 -- 172.21.15.153:0/142287683 wait complete. 2024-03-21T10:35:49.129 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:49.128+0000 7fec6e000700 1 Processor -- start 2024-03-21T10:35:49.130 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:49.128+0000 7fec6e000700 1 -- start start 2024-03-21T10:35:49.130 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:49.128+0000 7fec6e000700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fec68102fc0 0x7fec68198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:49.130 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:49.128+0000 7fec6e000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec68103970 0x7fec68198e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:49.130 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:49.128+0000 7fec6e000700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec68199520 con 0x7fec68103970 2024-03-21T10:35:49.130 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:49.128+0000 7fec6e000700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec6819d330 con 0x7fec68102fc0 2024-03-21T10:35:49.131 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:49.129+0000 7fec5ea00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec68103970 0x7fec68198e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:49.131 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:49.129+0000 7fec5ea00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec68103970 0x7fec68198e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:57814/0 (socket says 172.21.15.153:57814) 2024-03-21T10:35:49.131 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:49.129+0000 7fec5ea00700 1 -- 172.21.15.153:0/1722269924 learned_addr learned my addr 172.21.15.153:0/1722269924 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:35:49.131 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:49.129+0000 7fec67400700 1 --2- 172.21.15.153:0/1722269924 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fec68102fc0 0x7fec68198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:49.131 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:49.129+0000 7fec5ea00700 1 -- 172.21.15.153:0/1722269924 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fec68102fc0 msgr2=0x7fec68198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:49.131 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:49.129+0000 7fec5ea00700 1 --2- 172.21.15.153:0/1722269924 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fec68102fc0 0x7fec68198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:49.131 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:49.129+0000 7fec5ea00700 1 -- 172.21.15.153:0/1722269924 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec580095f0 con 0x7fec68103970 2024-03-21T10:35:49.132 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:49.129+0000 7fec5ea00700 1 --2- 172.21.15.153:0/1722269924 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec68103970 0x7fec68198e90 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fec50007e90 tx=0x7fec5000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:35:49.132 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:49.130+0000 7fec64c00700 1 -- 172.21.15.153:0/1722269924 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fec5000dda0 con 0x7fec68103970 2024-03-21T10:35:49.132 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:49.130+0000 7fec64c00700 1 -- 172.21.15.153:0/1722269924 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fec5000f040 con 0x7fec68103970 2024-03-21T10:35:49.132 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:49.130+0000 7fec6e000700 1 -- 172.21.15.153:0/1722269924 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec6819d610 con 0x7fec68103970 2024-03-21T10:35:49.132 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:49.130+0000 7fec64c00700 1 -- 172.21.15.153:0/1722269924 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fec500125f0 con 0x7fec68103970 2024-03-21T10:35:49.132 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:49.130+0000 7fec6e000700 1 -- 172.21.15.153:0/1722269924 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec6819db60 con 0x7fec68103970 2024-03-21T10:35:49.133 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:49.131+0000 7fec6e000700 1 -- 172.21.15.153:0/1722269924 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec6810b360 con 0x7fec68103970 2024-03-21T10:35:49.133 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:49.132+0000 7fec64c00700 1 -- 172.21.15.153:0/1722269924 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fec500128b0 con 0x7fec68103970 2024-03-21T10:35:49.134 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:49.133+0000 7fec64c00700 1 --2- 172.21.15.153:0/1722269924 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fec4c06e460 0x7fec4c070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:49.134 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:49.133+0000 7fec64c00700 1 -- 172.21.15.153:0/1722269924 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fec5008def0 con 0x7fec68103970 2024-03-21T10:35:49.135 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:49.133+0000 7fec67400700 1 --2- 172.21.15.153:0/1722269924 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fec4c06e460 0x7fec4c070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:49.135 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:49.134+0000 7fec67400700 1 --2- 172.21.15.153:0/1722269924 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fec4c06e460 0x7fec4c070920 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fec58006010 tx=0x7fec5801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:35:49.135 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:49.137+0000 7fec64c00700 1 -- 172.21.15.153:0/1722269924 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fec5005aa40 con 0x7fec68103970 2024-03-21T10:35:49.198 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:35:48 smithi174 ceph-mon[88004]: pgmap v720: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:35:49.198 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:35:48 smithi174 ceph-mon[88004]: from='client.35294 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:35:49.441 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:49.444+0000 7fec6e000700 1 -- 172.21.15.153:0/1722269924 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fec6819dd80 con 0x7fec68103970 2024-03-21T10:35:49.443 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:49.445+0000 7fec64c00700 1 -- 172.21.15.153:0/1722269924 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fec5005a5d0 con 0x7fec68103970 2024-03-21T10:35:49.443 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:35:49.443 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": { 2024-03-21T10:35:49.443 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:35:49.443 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:35:49.443 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": { 2024-03-21T10:35:49.443 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:35:49.443 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:35:49.443 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": { 2024-03-21T10:35:49.443 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 6 2024-03-21T10:35:49.444 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:35:49.444 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": { 2024-03-21T10:35:49.444 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 4 2024-03-21T10:35:49.444 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:35:49.444 INFO:teuthology.orchestra.run.smithi153.stdout: "overall": { 2024-03-21T10:35:49.444 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 14 2024-03-21T10:35:49.444 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-03-21T10:35:49.444 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:35:49.446 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:49.448+0000 7fec6e000700 1 -- 172.21.15.153:0/1722269924 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fec4c06e460 msgr2=0x7fec4c070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:49.446 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:49.448+0000 7fec6e000700 1 --2- 172.21.15.153:0/1722269924 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fec4c06e460 0x7fec4c070920 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fec58006010 tx=0x7fec5801a040 comp rx=0 tx=0).stop 2024-03-21T10:35:49.446 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:49.448+0000 7fec6e000700 1 -- 172.21.15.153:0/1722269924 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec68103970 msgr2=0x7fec68198e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:49.446 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:49.448+0000 7fec6e000700 1 --2- 172.21.15.153:0/1722269924 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec68103970 0x7fec68198e90 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fec50007e90 tx=0x7fec5000d3b0 comp rx=0 tx=0).stop 2024-03-21T10:35:49.446 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:49.448+0000 7fec6e000700 1 -- 172.21.15.153:0/1722269924 shutdown_connections 2024-03-21T10:35:49.447 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:49.448+0000 7fec6e000700 1 --2- 172.21.15.153:0/1722269924 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fec68102fc0 0x7fec68198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:49.447 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:49.448+0000 7fec6e000700 1 --2- 172.21.15.153:0/1722269924 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fec4c06e460 0x7fec4c070920 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:49.447 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:49.448+0000 7fec6e000700 1 --2- 172.21.15.153:0/1722269924 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec68103970 0x7fec68198e90 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:49.447 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:49.448+0000 7fec6e000700 1 -- 172.21.15.153:0/1722269924 >> 172.21.15.153:0/1722269924 conn(0x7fec680fe870 msgr2=0x7fec680fff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:35:49.447 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:49.448+0000 7fec6e000700 1 -- 172.21.15.153:0/1722269924 shutdown_connections 2024-03-21T10:35:49.448 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:49.448+0000 7fec6e000700 1 -- 172.21.15.153:0/1722269924 wait complete. 2024-03-21T10:35:49.595 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:49.597+0000 7fd617e00700 1 -- 172.21.15.153:0/1447232082 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd6181069c0 msgr2=0x7fd618106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:49.595 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:49.597+0000 7fd617e00700 1 --2- 172.21.15.153:0/1447232082 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd6181069c0 0x7fd618106da0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fd608009910 tx=0x7fd608009c20 comp rx=0 tx=0).stop 2024-03-21T10:35:49.595 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:49.597+0000 7fd617e00700 1 -- 172.21.15.153:0/1447232082 shutdown_connections 2024-03-21T10:35:49.595 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:49.597+0000 7fd617e00700 1 --2- 172.21.15.153:0/1447232082 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd6181009c0 0x7fd618100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:49.595 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:49.597+0000 7fd617e00700 1 --2- 172.21.15.153:0/1447232082 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd6181069c0 0x7fd618106da0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:49.596 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:49.597+0000 7fd617e00700 1 -- 172.21.15.153:0/1447232082 >> 172.21.15.153:0/1447232082 conn(0x7fd6180fc4f0 msgr2=0x7fd6180fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:35:49.596 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:49.598+0000 7fd617e00700 1 -- 172.21.15.153:0/1447232082 shutdown_connections 2024-03-21T10:35:49.596 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:49.598+0000 7fd617e00700 1 -- 172.21.15.153:0/1447232082 wait complete. 2024-03-21T10:35:49.597 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:49.598+0000 7fd617e00700 1 Processor -- start 2024-03-21T10:35:49.597 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:49.599+0000 7fd617e00700 1 -- start start 2024-03-21T10:35:49.597 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:49.599+0000 7fd617e00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd6181009c0 0x7fd618073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:49.597 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:49.599+0000 7fd617e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd6181069c0 0x7fd6180735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:49.597 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:49.599+0000 7fd617e00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd618073b10 con 0x7fd6181069c0 2024-03-21T10:35:49.597 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:49.599+0000 7fd617e00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd618073c50 con 0x7fd6181009c0 2024-03-21T10:35:49.597 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:49.599+0000 7fd616000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd6181069c0 0x7fd6180735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:49.598 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:49.599+0000 7fd616000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd6181069c0 0x7fd6180735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:57836/0 (socket says 172.21.15.153:57836) 2024-03-21T10:35:49.598 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:49.599+0000 7fd616000700 1 -- 172.21.15.153:0/775160728 learned_addr learned my addr 172.21.15.153:0/775160728 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:35:49.598 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:49.600+0000 7fd616a00700 1 --2- 172.21.15.153:0/775160728 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd6181009c0 0x7fd618073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:49.598 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:49.600+0000 7fd616000700 1 -- 172.21.15.153:0/775160728 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd6181009c0 msgr2=0x7fd618073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:49.598 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:49.600+0000 7fd616000700 1 --2- 172.21.15.153:0/775160728 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd6181009c0 0x7fd618073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:49.598 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:49.600+0000 7fd616000700 1 -- 172.21.15.153:0/775160728 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6080095f0 con 0x7fd6181069c0 2024-03-21T10:35:49.598 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:49.600+0000 7fd616000700 1 --2- 172.21.15.153:0/775160728 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd6181069c0 0x7fd6180735d0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fd60400c8a0 tx=0x7fd60400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:35:49.599 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:49.600+0000 7fd60f400700 1 -- 172.21.15.153:0/775160728 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd60400cea0 con 0x7fd6181069c0 2024-03-21T10:35:49.599 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:49.600+0000 7fd60f400700 1 -- 172.21.15.153:0/775160728 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd604004830 con 0x7fd6181069c0 2024-03-21T10:35:49.599 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:49.600+0000 7fd617e00700 1 -- 172.21.15.153:0/775160728 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6181a2760 con 0x7fd6181069c0 2024-03-21T10:35:49.599 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:49.600+0000 7fd60f400700 1 -- 172.21.15.153:0/775160728 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd6040055b0 con 0x7fd6181069c0 2024-03-21T10:35:49.599 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:49.600+0000 7fd617e00700 1 -- 172.21.15.153:0/775160728 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6181a2b70 con 0x7fd6181069c0 2024-03-21T10:35:49.599 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:49.601+0000 7fd617e00700 1 -- 172.21.15.153:0/775160728 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd618108fc0 con 0x7fd6181069c0 2024-03-21T10:35:49.600 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:49.602+0000 7fd60f400700 1 -- 172.21.15.153:0/775160728 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fd6040049a0 con 0x7fd6181069c0 2024-03-21T10:35:49.600 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:49.603+0000 7fd60f400700 1 --2- 172.21.15.153:0/775160728 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd5fc06e460 0x7fd5fc070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:49.601 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:49.603+0000 7fd60f400700 1 -- 172.21.15.153:0/775160728 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fd60408e270 con 0x7fd6181069c0 2024-03-21T10:35:49.601 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:49.604+0000 7fd616a00700 1 --2- 172.21.15.153:0/775160728 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd5fc06e460 0x7fd5fc070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:49.601 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:49.604+0000 7fd616a00700 1 --2- 172.21.15.153:0/775160728 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd5fc06e460 0x7fd5fc070920 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fd608006010 tx=0x7fd60800b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:35:49.605 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:49.608+0000 7fd60f400700 1 -- 172.21.15.153:0/775160728 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd604057290 con 0x7fd6181069c0 2024-03-21T10:35:49.859 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:49.862+0000 7fd617e00700 1 -- 172.21.15.153:0/775160728 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fd61804f2e0 con 0x7fd6181069c0 2024-03-21T10:35:49.860 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:49.863+0000 7fd60f400700 1 -- 172.21.15.153:0/775160728 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1876 (secure 0 0 0) 0x7fd604004c50 con 0x7fd6181069c0 2024-03-21T10:35:49.860 INFO:teuthology.orchestra.run.smithi153.stdout:e32 2024-03-21T10:35:49.861 INFO:teuthology.orchestra.run.smithi153.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-21T10:35:49.861 INFO:teuthology.orchestra.run.smithi153.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:35:49.861 INFO:teuthology.orchestra.run.smithi153.stdout:legacy client fscid: 1 2024-03-21T10:35:49.861 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:35:49.861 INFO:teuthology.orchestra.run.smithi153.stdout:Filesystem 'cephfs' (1) 2024-03-21T10:35:49.861 INFO:teuthology.orchestra.run.smithi153.stdout:fs_name cephfs 2024-03-21T10:35:49.861 INFO:teuthology.orchestra.run.smithi153.stdout:epoch 23 2024-03-21T10:35:49.861 INFO:teuthology.orchestra.run.smithi153.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-21T10:35:49.861 INFO:teuthology.orchestra.run.smithi153.stdout:created 2024-03-21T10:04:27.086576+0000 2024-03-21T10:35:49.861 INFO:teuthology.orchestra.run.smithi153.stdout:modified 2024-03-21T10:19:05.022092+0000 2024-03-21T10:35:49.861 INFO:teuthology.orchestra.run.smithi153.stdout:tableserver 0 2024-03-21T10:35:49.861 INFO:teuthology.orchestra.run.smithi153.stdout:root 0 2024-03-21T10:35:49.861 INFO:teuthology.orchestra.run.smithi153.stdout:session_timeout 60 2024-03-21T10:35:49.861 INFO:teuthology.orchestra.run.smithi153.stdout:session_autoclose 300 2024-03-21T10:35:49.862 INFO:teuthology.orchestra.run.smithi153.stdout:max_file_size 1099511627776 2024-03-21T10:35:49.862 INFO:teuthology.orchestra.run.smithi153.stdout:max_xattr_size 65536 2024-03-21T10:35:49.862 INFO:teuthology.orchestra.run.smithi153.stdout:required_client_features {} 2024-03-21T10:35:49.862 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure 0 2024-03-21T10:35:49.862 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure_osd_epoch 94 2024-03-21T10:35:49.862 INFO:teuthology.orchestra.run.smithi153.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:35:49.862 INFO:teuthology.orchestra.run.smithi153.stdout:max_mds 2 2024-03-21T10:35:49.862 INFO:teuthology.orchestra.run.smithi153.stdout:in 0,1 2024-03-21T10:35:49.862 INFO:teuthology.orchestra.run.smithi153.stdout:up {} 2024-03-21T10:35:49.862 INFO:teuthology.orchestra.run.smithi153.stdout:failed 0,1 2024-03-21T10:35:49.862 INFO:teuthology.orchestra.run.smithi153.stdout:damaged 2024-03-21T10:35:49.862 INFO:teuthology.orchestra.run.smithi153.stdout:stopped 2024-03-21T10:35:49.862 INFO:teuthology.orchestra.run.smithi153.stdout:data_pools [3] 2024-03-21T10:35:49.862 INFO:teuthology.orchestra.run.smithi153.stdout:metadata_pool 2 2024-03-21T10:35:49.863 INFO:teuthology.orchestra.run.smithi153.stdout:inline_data enabled 2024-03-21T10:35:49.863 INFO:teuthology.orchestra.run.smithi153.stdout:balancer 2024-03-21T10:35:49.863 INFO:teuthology.orchestra.run.smithi153.stdout:bal_rank_mask -1 2024-03-21T10:35:49.863 INFO:teuthology.orchestra.run.smithi153.stdout:standby_count_wanted 1 2024-03-21T10:35:49.863 INFO:teuthology.orchestra.run.smithi153.stdout:qdb_cluster leader: 0 members: 2024-03-21T10:35:49.863 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:35:49.863 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:35:49.863 INFO:teuthology.orchestra.run.smithi153.stdout:Standby daemons: 2024-03-21T10:35:49.863 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:35:49.863 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.zipckr{-1:34442} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6826/1103846295,v1:172.21.15.153:6827/1103846295] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:35:49.863 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.iuyluj{-1:34474} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6828/1738630832,v1:172.21.15.153:6829/1738630832] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:35:49.863 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.yyimst{-1:44313} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.174:6824/4127786797,v1:172.21.15.174:6825/4127786797] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:35:49.863 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.nhozag{-1:44687} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.174:6826/2767517845,v1:172.21.15.174:6827/2767517845] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:35:49.864 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:49.866+0000 7fd617e00700 1 -- 172.21.15.153:0/775160728 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd5fc06e460 msgr2=0x7fd5fc070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:49.864 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:49.866+0000 7fd617e00700 1 --2- 172.21.15.153:0/775160728 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd5fc06e460 0x7fd5fc070920 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fd608006010 tx=0x7fd60800b540 comp rx=0 tx=0).stop 2024-03-21T10:35:49.864 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:49.866+0000 7fd617e00700 1 -- 172.21.15.153:0/775160728 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd6181069c0 msgr2=0x7fd6180735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:49.865 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:49.866+0000 7fd617e00700 1 --2- 172.21.15.153:0/775160728 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd6181069c0 0x7fd6180735d0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fd60400c8a0 tx=0x7fd60400cc60 comp rx=0 tx=0).stop 2024-03-21T10:35:49.865 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:49.866+0000 7fd617e00700 1 -- 172.21.15.153:0/775160728 shutdown_connections 2024-03-21T10:35:49.865 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:49.866+0000 7fd617e00700 1 --2- 172.21.15.153:0/775160728 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd6181009c0 0x7fd618073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:49.865 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:49.866+0000 7fd617e00700 1 --2- 172.21.15.153:0/775160728 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd5fc06e460 0x7fd5fc070920 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:49.865 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:49.866+0000 7fd617e00700 1 --2- 172.21.15.153:0/775160728 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd6181069c0 0x7fd6180735d0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:49.865 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:49.866+0000 7fd617e00700 1 -- 172.21.15.153:0/775160728 >> 172.21.15.153:0/775160728 conn(0x7fd6180fc4f0 msgr2=0x7fd6180fce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:35:49.865 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:49.866+0000 7fd617e00700 1 -- 172.21.15.153:0/775160728 shutdown_connections 2024-03-21T10:35:49.865 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:49.866+0000 7fd617e00700 1 -- 172.21.15.153:0/775160728 wait complete. 2024-03-21T10:35:49.865 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 32 2024-03-21T10:35:50.015 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:50.018+0000 7f1f4e600700 1 -- 172.21.15.153:0/1621778478 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f48102fe0 msgr2=0x7f1f481033c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:50.016 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:50.018+0000 7f1f4e600700 1 --2- 172.21.15.153:0/1621778478 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f48102fe0 0x7f1f481033c0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f1f38009910 tx=0x7f1f38009c20 comp rx=0 tx=0).stop 2024-03-21T10:35:50.016 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:50.018+0000 7f1f4e600700 1 -- 172.21.15.153:0/1621778478 shutdown_connections 2024-03-21T10:35:50.016 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:50.018+0000 7f1f4e600700 1 --2- 172.21.15.153:0/1621778478 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1f48103990 0x7f1f481079e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:50.016 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:50.018+0000 7f1f4e600700 1 --2- 172.21.15.153:0/1621778478 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f48102fe0 0x7f1f481033c0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:50.016 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:50.018+0000 7f1f4e600700 1 -- 172.21.15.153:0/1621778478 >> 172.21.15.153:0/1621778478 conn(0x7f1f480fe870 msgr2=0x7f1f48100c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:35:50.016 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:35:49 smithi153 ceph-mon[113733]: from='client.35298 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:35:50.016 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:35:49 smithi153 ceph-mon[113733]: from='client.35302 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:35:50.016 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:35:49 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/1722269924' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:35:50.018 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:50.018+0000 7f1f4e600700 1 -- 172.21.15.153:0/1621778478 shutdown_connections 2024-03-21T10:35:50.018 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:50.018+0000 7f1f4e600700 1 -- 172.21.15.153:0/1621778478 wait complete. 2024-03-21T10:35:50.018 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:50.019+0000 7f1f4e600700 1 Processor -- start 2024-03-21T10:35:50.018 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:50.019+0000 7f1f4e600700 1 -- start start 2024-03-21T10:35:50.018 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:50.020+0000 7f1f4e600700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f48102fe0 0x7f1f4819e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:50.018 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:50.020+0000 7f1f4e600700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1f48103990 0x7f1f4819ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:50.018 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:50.020+0000 7f1f4e600700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f4819f360 con 0x7f1f48102fe0 2024-03-21T10:35:50.018 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:50.020+0000 7f1f4e600700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f48198810 con 0x7f1f48103990 2024-03-21T10:35:50.018 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:50.020+0000 7f1f47400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f48102fe0 0x7f1f4819e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:50.018 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:50.020+0000 7f1f47400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f48102fe0 0x7f1f4819e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:57852/0 (socket says 172.21.15.153:57852) 2024-03-21T10:35:50.018 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:50.020+0000 7f1f47400700 1 -- 172.21.15.153:0/3955895495 learned_addr learned my addr 172.21.15.153:0/3955895495 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:35:50.019 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:50.020+0000 7f1f3fe00700 1 --2- 172.21.15.153:0/3955895495 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1f48103990 0x7f1f4819ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:50.019 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:50.021+0000 7f1f47400700 1 -- 172.21.15.153:0/3955895495 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1f48103990 msgr2=0x7f1f4819ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:50.019 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:50.021+0000 7f1f47400700 1 --2- 172.21.15.153:0/3955895495 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1f48103990 0x7f1f4819ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:50.019 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:50.021+0000 7f1f47400700 1 -- 172.21.15.153:0/3955895495 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f380095f0 con 0x7f1f48102fe0 2024-03-21T10:35:50.019 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:50.021+0000 7f1f3fe00700 1 --2- 172.21.15.153:0/3955895495 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1f48103990 0x7f1f4819ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:35:50.020 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:50.021+0000 7f1f47400700 1 --2- 172.21.15.153:0/3955895495 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f48102fe0 0x7f1f4819e790 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f1f3800b5c0 tx=0x7f1f3800be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:35:50.020 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:50.021+0000 7f1f44c00700 1 -- 172.21.15.153:0/3955895495 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f3801d070 con 0x7f1f48102fe0 2024-03-21T10:35:50.020 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:50.021+0000 7f1f44c00700 1 -- 172.21.15.153:0/3955895495 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1f3800f460 con 0x7f1f48102fe0 2024-03-21T10:35:50.020 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:50.021+0000 7f1f44c00700 1 -- 172.21.15.153:0/3955895495 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f38017610 con 0x7f1f48102fe0 2024-03-21T10:35:50.020 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:50.021+0000 7f1f4e600700 1 -- 172.21.15.153:0/3955895495 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f48198a90 con 0x7f1f48102fe0 2024-03-21T10:35:50.020 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:50.021+0000 7f1f4e600700 1 -- 172.21.15.153:0/3955895495 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f48198f80 con 0x7f1f48102fe0 2024-03-21T10:35:50.020 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:50.022+0000 7f1f4e600700 1 -- 172.21.15.153:0/3955895495 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f4810b380 con 0x7f1f48102fe0 2024-03-21T10:35:50.021 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:50.024+0000 7f1f44c00700 1 -- 172.21.15.153:0/3955895495 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f1f3800f5d0 con 0x7f1f48102fe0 2024-03-21T10:35:50.023 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:50.025+0000 7f1f44c00700 1 --2- 172.21.15.153:0/3955895495 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f1f2c06e460 0x7f1f2c070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:50.023 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:50.025+0000 7f1f44c00700 1 -- 172.21.15.153:0/3955895495 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f1f3808fa70 con 0x7f1f48102fe0 2024-03-21T10:35:50.023 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:50.025+0000 7f1f3fe00700 1 --2- 172.21.15.153:0/3955895495 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f1f2c06e460 0x7f1f2c070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:50.023 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:50.026+0000 7f1f3fe00700 1 --2- 172.21.15.153:0/3955895495 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f1f2c06e460 0x7f1f2c070920 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f1f30005fd0 tx=0x7f1f30005dc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:35:50.027 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:50.030+0000 7f1f44c00700 1 -- 172.21.15.153:0/3955895495 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f1f3805c6e0 con 0x7f1f48102fe0 2024-03-21T10:35:50.198 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:35:49 smithi174 ceph-mon[88004]: from='client.35298 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:35:50.199 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:35:49 smithi174 ceph-mon[88004]: from='client.35302 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:35:50.199 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:35:49 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/1722269924' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:35:50.267 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:50.269+0000 7f1f4e600700 1 -- 172.21.15.153:0/3955895495 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1f481992b0 con 0x7f1f2c06e460 2024-03-21T10:35:50.268 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:50.271+0000 7f1f44c00700 1 -- 172.21.15.153:0/3955895495 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f1f481992b0 con 0x7f1f2c06e460 2024-03-21T10:35:50.268 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:35:50.268 INFO:teuthology.orchestra.run.smithi153.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232", 2024-03-21T10:35:50.269 INFO:teuthology.orchestra.run.smithi153.stdout: "in_progress": true, 2024-03-21T10:35:50.269 INFO:teuthology.orchestra.run.smithi153.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-21T10:35:50.269 INFO:teuthology.orchestra.run.smithi153.stdout: "services_complete": [ 2024-03-21T10:35:50.269 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr", 2024-03-21T10:35:50.269 INFO:teuthology.orchestra.run.smithi153.stdout: "mon", 2024-03-21T10:35:50.269 INFO:teuthology.orchestra.run.smithi153.stdout: "mds", 2024-03-21T10:35:50.269 INFO:teuthology.orchestra.run.smithi153.stdout: "crash", 2024-03-21T10:35:50.269 INFO:teuthology.orchestra.run.smithi153.stdout: "osd" 2024-03-21T10:35:50.269 INFO:teuthology.orchestra.run.smithi153.stdout: ], 2024-03-21T10:35:50.269 INFO:teuthology.orchestra.run.smithi153.stdout: "progress": "16/23 daemons upgraded", 2024-03-21T10:35:50.269 INFO:teuthology.orchestra.run.smithi153.stdout: "message": "Currently upgrading mds daemons", 2024-03-21T10:35:50.269 INFO:teuthology.orchestra.run.smithi153.stdout: "is_paused": false 2024-03-21T10:35:50.269 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:35:50.272 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:50.274+0000 7f1f4e600700 1 -- 172.21.15.153:0/3955895495 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f1f2c06e460 msgr2=0x7f1f2c070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:50.272 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:50.274+0000 7f1f4e600700 1 --2- 172.21.15.153:0/3955895495 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f1f2c06e460 0x7f1f2c070920 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f1f30005fd0 tx=0x7f1f30005dc0 comp rx=0 tx=0).stop 2024-03-21T10:35:50.272 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:50.274+0000 7f1f4e600700 1 -- 172.21.15.153:0/3955895495 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f48102fe0 msgr2=0x7f1f4819e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:50.272 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:50.274+0000 7f1f4e600700 1 --2- 172.21.15.153:0/3955895495 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f48102fe0 0x7f1f4819e790 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f1f3800b5c0 tx=0x7f1f3800be00 comp rx=0 tx=0).stop 2024-03-21T10:35:50.272 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:50.274+0000 7f1f4e600700 1 -- 172.21.15.153:0/3955895495 shutdown_connections 2024-03-21T10:35:50.272 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:50.274+0000 7f1f4e600700 1 --2- 172.21.15.153:0/3955895495 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f1f2c06e460 0x7f1f2c070920 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:50.272 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:50.274+0000 7f1f4e600700 1 --2- 172.21.15.153:0/3955895495 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f48102fe0 0x7f1f4819e790 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:50.273 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:50.274+0000 7f1f4e600700 1 --2- 172.21.15.153:0/3955895495 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1f48103990 0x7f1f4819ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:50.273 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:50.274+0000 7f1f4e600700 1 -- 172.21.15.153:0/3955895495 >> 172.21.15.153:0/3955895495 conn(0x7f1f480fe870 msgr2=0x7f1f48100c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:35:50.273 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:50.274+0000 7f1f4e600700 1 -- 172.21.15.153:0/3955895495 shutdown_connections 2024-03-21T10:35:50.273 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:50.275+0000 7f1f4e600700 1 -- 172.21.15.153:0/3955895495 wait complete. 2024-03-21T10:35:50.431 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:50.433+0000 7f0ee2a00700 1 -- 172.21.15.153:0/3515067537 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0edc101750 msgr2=0x7f0edc1057a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:50.431 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:50.433+0000 7f0ee2a00700 1 --2- 172.21.15.153:0/3515067537 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0edc101750 0x7f0edc1057a0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f0ec8009910 tx=0x7f0ec8009c20 comp rx=0 tx=0).stop 2024-03-21T10:35:50.431 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:50.433+0000 7f0ee2a00700 1 -- 172.21.15.153:0/3515067537 shutdown_connections 2024-03-21T10:35:50.431 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:50.433+0000 7f0ee2a00700 1 --2- 172.21.15.153:0/3515067537 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0edc101750 0x7f0edc1057a0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:50.431 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:50.433+0000 7f0ee2a00700 1 --2- 172.21.15.153:0/3515067537 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0edc100da0 0x7f0edc101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:50.431 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:50.433+0000 7f0ee2a00700 1 -- 172.21.15.153:0/3515067537 >> 172.21.15.153:0/3515067537 conn(0x7f0edc0fc650 msgr2=0x7f0edc0fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:35:50.431 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:50.434+0000 7f0ee2a00700 1 -- 172.21.15.153:0/3515067537 shutdown_connections 2024-03-21T10:35:50.432 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:50.434+0000 7f0ee2a00700 1 -- 172.21.15.153:0/3515067537 wait complete. 2024-03-21T10:35:50.432 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:50.435+0000 7f0ee2a00700 1 Processor -- start 2024-03-21T10:35:50.433 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:50.436+0000 7f0ee2a00700 1 -- start start 2024-03-21T10:35:50.433 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:50.436+0000 7f0ee2a00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0edc100da0 0x7f0edc19c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:50.433 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:50.436+0000 7f0ee2a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0edc101750 0x7f0edc19cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:50.433 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:50.436+0000 7f0ee2a00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0edc19d1f0 con 0x7f0edc101750 2024-03-21T10:35:50.433 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:50.436+0000 7f0ee2a00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0edc1966a0 con 0x7f0edc100da0 2024-03-21T10:35:50.434 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:50.436+0000 7f0edb400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0edc101750 0x7f0edc19cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:50.434 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:50.437+0000 7f0edb400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0edc101750 0x7f0edc19cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:57866/0 (socket says 172.21.15.153:57866) 2024-03-21T10:35:50.434 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:50.437+0000 7f0edb400700 1 -- 172.21.15.153:0/1069676748 learned_addr learned my addr 172.21.15.153:0/1069676748 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:35:50.435 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:50.437+0000 7f0edbe00700 1 --2- 172.21.15.153:0/1069676748 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0edc100da0 0x7f0edc19c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:50.435 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:50.437+0000 7f0edb400700 1 -- 172.21.15.153:0/1069676748 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0edc100da0 msgr2=0x7f0edc19c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:50.435 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:50.437+0000 7f0edb400700 1 --2- 172.21.15.153:0/1069676748 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0edc100da0 0x7f0edc19c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:50.435 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:50.437+0000 7f0edb400700 1 -- 172.21.15.153:0/1069676748 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ec80095f0 con 0x7f0edc101750 2024-03-21T10:35:50.435 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:50.437+0000 7f0edb400700 1 --2- 172.21.15.153:0/1069676748 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0edc101750 0x7f0edc19cb60 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f0ec800bda0 tx=0x7f0ec800be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:35:50.435 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:50.438+0000 7f0ed8c00700 1 -- 172.21.15.153:0/1069676748 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ec801d070 con 0x7f0edc101750 2024-03-21T10:35:50.436 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:50.438+0000 7f0ee2a00700 1 -- 172.21.15.153:0/1069676748 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0edc196920 con 0x7f0edc101750 2024-03-21T10:35:50.436 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:50.438+0000 7f0ed8c00700 1 -- 172.21.15.153:0/1069676748 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0ec8004fb0 con 0x7f0edc101750 2024-03-21T10:35:50.436 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:50.438+0000 7f0ed8c00700 1 -- 172.21.15.153:0/1069676748 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ec800f480 con 0x7f0edc101750 2024-03-21T10:35:50.436 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:50.438+0000 7f0ee2a00700 1 -- 172.21.15.153:0/1069676748 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0edc196e10 con 0x7f0edc101750 2024-03-21T10:35:50.436 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:50.439+0000 7f0ee2a00700 1 -- 172.21.15.153:0/1069676748 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0edc1090f0 con 0x7f0edc101750 2024-03-21T10:35:50.437 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:50.440+0000 7f0ed8c00700 1 -- 172.21.15.153:0/1069676748 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f0ec8022870 con 0x7f0edc101750 2024-03-21T10:35:50.438 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:50.441+0000 7f0ed8c00700 1 --2- 172.21.15.153:0/1069676748 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f0ec006e460 0x7f0ec0070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:35:50.439 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:50.442+0000 7f0ed8c00700 1 -- 172.21.15.153:0/1069676748 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f0ec808fad0 con 0x7f0edc101750 2024-03-21T10:35:50.439 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:50.442+0000 7f0edbe00700 1 --2- 172.21.15.153:0/1069676748 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f0ec006e460 0x7f0ec0070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:35:50.440 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:50.443+0000 7f0edbe00700 1 --2- 172.21.15.153:0/1069676748 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f0ec006e460 0x7f0ec0070920 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc006fd0 tx=0x7f0ecc008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:35:50.443 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:50.446+0000 7f0ed8c00700 1 -- 172.21.15.153:0/1069676748 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0ec805c6d0 con 0x7f0edc101750 2024-03-21T10:35:50.752 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:50.755+0000 7f0ee2a00700 1 -- 172.21.15.153:0/1069676748 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f0edc04ea90 con 0x7f0edc101750 2024-03-21T10:35:50.754 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:50.757+0000 7f0ed8c00700 1 -- 172.21.15.153:0/1069676748 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4493 (secure 0 0 0) 0x7f0ec805c260 con 0x7f0edc101750 2024-03-21T10:35:50.755 INFO:teuthology.orchestra.run.smithi153.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-03-21T10:35:50.755 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-03-21T10:35:50.755 INFO:teuthology.orchestra.run.smithi153.stdout: host smithi174 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi174/config 2024-03-21T10:35:50.755 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/podman: timeout after 895 seconds 2024-03-21T10:35:50.755 INFO:teuthology.orchestra.run.smithi153.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 -e NODE_NAME=smithi174 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/run/ceph:z -v /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpd910r3_i:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 inventory --format=json-pretty --filter-for-batch 2024-03-21T10:35:50.755 INFO:teuthology.orchestra.run.smithi153.stdout:Traceback (most recent call last): 2024-03-21T10:35:50.755 INFO:teuthology.orchestra.run.smithi153.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-03-21T10:35:50.755 INFO:teuthology.orchestra.run.smithi153.stdout: return _run_code(code, main_globals, None, 2024-03-21T10:35:50.755 INFO:teuthology.orchestra.run.smithi153.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-03-21T10:35:50.756 INFO:teuthology.orchestra.run.smithi153.stdout: exec(code, run_globals) 2024-03-21T10:35:50.756 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 5606, in 2024-03-21T10:35:50.756 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 5594, in main 2024-03-21T10:35:50.756 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 406, in _infer_config 2024-03-21T10:35:50.756 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 321, in _infer_fsid 2024-03-21T10:35:50.756 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 434, in _infer_image 2024-03-21T10:35:50.756 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 308, in _validate_fsid 2024-03-21T10:35:50.756 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 3357, in command_ceph_volume 2024-03-21T10:35:50.756 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-03-21T10:35:50.756 INFO:teuthology.orchestra.run.smithi153.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 -e NODE_NAME=smithi174 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/run/ceph:z -v /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpd910r3_i:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 inventory --format=json-pretty --filter-for-batch 2024-03-21T10:35:50.756 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-21T10:35:50.756 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is degraded 2024-03-21T10:35:50.756 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-21T10:35:50.756 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-21T10:35:50.756 INFO:teuthology.orchestra.run.smithi153.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-21T10:35:50.756 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-21T10:35:50.758 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:50.761+0000 7f0ee2a00700 1 -- 172.21.15.153:0/1069676748 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f0ec006e460 msgr2=0x7f0ec0070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:50.758 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:50.761+0000 7f0ee2a00700 1 --2- 172.21.15.153:0/1069676748 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f0ec006e460 0x7f0ec0070920 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc006fd0 tx=0x7f0ecc008040 comp rx=0 tx=0).stop 2024-03-21T10:35:50.758 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:50.761+0000 7f0ee2a00700 1 -- 172.21.15.153:0/1069676748 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0edc101750 msgr2=0x7f0edc19cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:35:50.759 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:50.761+0000 7f0ee2a00700 1 --2- 172.21.15.153:0/1069676748 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0edc101750 0x7f0edc19cb60 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f0ec800bda0 tx=0x7f0ec800be80 comp rx=0 tx=0).stop 2024-03-21T10:35:50.759 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:50.761+0000 7f0ee2a00700 1 -- 172.21.15.153:0/1069676748 shutdown_connections 2024-03-21T10:35:50.759 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:50.761+0000 7f0ee2a00700 1 --2- 172.21.15.153:0/1069676748 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0edc100da0 0x7f0edc19c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:50.759 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:50.761+0000 7f0ee2a00700 1 --2- 172.21.15.153:0/1069676748 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f0ec006e460 0x7f0ec0070920 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:50.759 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:50.761+0000 7f0ee2a00700 1 --2- 172.21.15.153:0/1069676748 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0edc101750 0x7f0edc19cb60 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:35:50.759 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:50.761+0000 7f0ee2a00700 1 -- 172.21.15.153:0/1069676748 >> 172.21.15.153:0/1069676748 conn(0x7f0edc0fc650 msgr2=0x7f0edc0fddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:35:50.759 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:50.762+0000 7f0ee2a00700 1 -- 172.21.15.153:0/1069676748 shutdown_connections 2024-03-21T10:35:50.760 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:35:50.762+0000 7f0ee2a00700 1 -- 172.21.15.153:0/1069676748 wait complete. 2024-03-21T10:35:51.053 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:35:50 smithi153 ceph-mon[113733]: pgmap v721: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:35:51.053 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:35:50 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/775160728' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:35:51.054 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:35:50 smithi153 ceph-mon[113733]: from='client.35314 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:35:51.198 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:35:50 smithi174 ceph-mon[88004]: pgmap v721: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:35:51.198 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:35:50 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/775160728' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:35:51.198 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:35:50 smithi174 ceph-mon[88004]: from='client.35314 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:35:51.948 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:35:51 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/1069676748' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:35:52.053 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:35:51 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/1069676748' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:35:53.027 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:35:52 smithi153 ceph-mon[113733]: pgmap v722: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:35:53.198 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:35:52 smithi174 ceph-mon[88004]: pgmap v722: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:35:55.053 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:35:54 smithi153 ceph-mon[113733]: pgmap v723: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:35:55.198 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:35:54 smithi174 ceph-mon[88004]: pgmap v723: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:35:57.053 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:35:56 smithi153 ceph-mon[113733]: pgmap v724: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:35:57.198 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:35:56 smithi174 ceph-mon[88004]: pgmap v724: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:35:59.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:35:58 smithi153 ceph-mon[113733]: pgmap v725: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:35:59.198 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:35:58 smithi174 ceph-mon[88004]: pgmap v725: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:36:01.053 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:36:00 smithi153 ceph-mon[113733]: pgmap v726: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:36:01.198 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:36:00 smithi174 ceph-mon[88004]: pgmap v726: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:36:02.053 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:36:01 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:36:02.198 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:36:01 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:36:03.053 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:36:02 smithi153 ceph-mon[113733]: pgmap v727: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:36:03.198 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:36:02 smithi174 ceph-mon[88004]: pgmap v727: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:36:05.198 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:36:04 smithi174 ceph-mon[88004]: pgmap v728: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:36:05.303 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:36:04 smithi153 ceph-mon[113733]: pgmap v728: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:36:07.198 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:36:06 smithi174 ceph-mon[88004]: pgmap v729: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:36:07.303 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:36:06 smithi153 ceph-mon[113733]: pgmap v729: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:36:09.198 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:36:08 smithi174 ceph-mon[88004]: pgmap v730: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:36:09.303 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:36:08 smithi153 ceph-mon[113733]: pgmap v730: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:36:10.948 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:36:10 smithi174 ceph-mon[88004]: pgmap v731: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:36:11.303 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:36:10 smithi153 ceph-mon[113733]: pgmap v731: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:36:13.053 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:36:12 smithi153 ceph-mon[113733]: pgmap v732: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:36:13.198 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:36:12 smithi174 ceph-mon[88004]: pgmap v732: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:36:15.198 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:36:14 smithi174 ceph-mon[88004]: pgmap v733: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:36:15.303 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:36:14 smithi153 ceph-mon[113733]: pgmap v733: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:36:17.198 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:36:16 smithi174 ceph-mon[88004]: pgmap v734: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:36:17.198 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:36:16 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:36:17.303 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:36:16 smithi153 ceph-mon[113733]: pgmap v734: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:36:17.303 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:36:16 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:36:19.181 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:36:18 smithi174 ceph-mon[88004]: pgmap v735: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:36:19.303 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:36:18 smithi153 ceph-mon[113733]: pgmap v735: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:36:20.934 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:20.935+0000 7f3205600700 1 -- 172.21.15.153:0/1877742330 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3200101750 msgr2=0x7f32001057a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:20.934 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:20.935+0000 7f3205600700 1 --2- 172.21.15.153:0/1877742330 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3200101750 0x7f32001057a0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f31ec009910 tx=0x7f31ec009c20 comp rx=0 tx=0).stop 2024-03-21T10:36:20.934 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:20.935+0000 7f3205600700 1 -- 172.21.15.153:0/1877742330 shutdown_connections 2024-03-21T10:36:20.934 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:20.935+0000 7f3205600700 1 --2- 172.21.15.153:0/1877742330 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3200101750 0x7f32001057a0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:20.934 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:20.935+0000 7f3205600700 1 --2- 172.21.15.153:0/1877742330 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3200100da0 0x7f3200101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:20.934 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:20.935+0000 7f3205600700 1 -- 172.21.15.153:0/1877742330 >> 172.21.15.153:0/1877742330 conn(0x7f32000fc630 msgr2=0x7f32000fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:36:20.934 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:20.935+0000 7f3205600700 1 -- 172.21.15.153:0/1877742330 shutdown_connections 2024-03-21T10:36:20.935 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:20.935+0000 7f3205600700 1 -- 172.21.15.153:0/1877742330 wait complete. 2024-03-21T10:36:20.936 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:20.936+0000 7f3205600700 1 Processor -- start 2024-03-21T10:36:20.936 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:20.937+0000 7f3205600700 1 -- start start 2024-03-21T10:36:20.936 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:20.937+0000 7f3205600700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3200100da0 0x7f320019e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:20.936 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:20.937+0000 7f3205600700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3200101750 0x7f320019ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:20.936 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:20.937+0000 7f3205600700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f320019f2d0 con 0x7f3200100da0 2024-03-21T10:36:20.936 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:20.937+0000 7f3205600700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3200198780 con 0x7f3200101750 2024-03-21T10:36:20.936 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:20.937+0000 7f31fea00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3200100da0 0x7f320019e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:20.936 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:20.937+0000 7f31fea00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3200100da0 0x7f320019e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:48064/0 (socket says 172.21.15.153:48064) 2024-03-21T10:36:20.936 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:20.937+0000 7f31fea00700 1 -- 172.21.15.153:0/4133525015 learned_addr learned my addr 172.21.15.153:0/4133525015 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:36:20.936 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:20.938+0000 7f31fe000700 1 --2- 172.21.15.153:0/4133525015 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3200101750 0x7f320019ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:20.937 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:20.938+0000 7f31fea00700 1 -- 172.21.15.153:0/4133525015 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3200101750 msgr2=0x7f320019ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:20.937 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:20.938+0000 7f31fea00700 1 --2- 172.21.15.153:0/4133525015 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3200101750 0x7f320019ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:20.937 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:20.938+0000 7f31fea00700 1 -- 172.21.15.153:0/4133525015 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31ec0095f0 con 0x7f3200100da0 2024-03-21T10:36:20.937 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:20.938+0000 7f31fe000700 1 --2- 172.21.15.153:0/4133525015 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3200101750 0x7f320019ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:36:20.937 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:20.938+0000 7f31fea00700 1 --2- 172.21.15.153:0/4133525015 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3200100da0 0x7f320019e700 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f31f000d8d0 tx=0x7f31f000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:36:20.938 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:20.938+0000 7f31f7400700 1 -- 172.21.15.153:0/4133525015 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f31f0009940 con 0x7f3200100da0 2024-03-21T10:36:20.938 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:20.938+0000 7f31f7400700 1 -- 172.21.15.153:0/4133525015 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f31f0010460 con 0x7f3200100da0 2024-03-21T10:36:20.938 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:20.938+0000 7f31f7400700 1 -- 172.21.15.153:0/4133525015 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f31f000f5f0 con 0x7f3200100da0 2024-03-21T10:36:20.938 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:20.938+0000 7f3205600700 1 -- 172.21.15.153:0/4133525015 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3200198a60 con 0x7f3200100da0 2024-03-21T10:36:20.938 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:20.939+0000 7f3205600700 1 -- 172.21.15.153:0/4133525015 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3200198fb0 con 0x7f3200100da0 2024-03-21T10:36:20.938 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:20.940+0000 7f3205600700 1 -- 172.21.15.153:0/4133525015 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3200104f20 con 0x7f3200100da0 2024-03-21T10:36:20.939 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:20.941+0000 7f31f7400700 1 -- 172.21.15.153:0/4133525015 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f31f00105d0 con 0x7f3200100da0 2024-03-21T10:36:20.939 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:20.941+0000 7f31f7400700 1 --2- 172.21.15.153:0/4133525015 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f31e406e460 0x7f31e4070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:20.940 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:20.942+0000 7f31f7400700 1 -- 172.21.15.153:0/4133525015 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f31f008e480 con 0x7f3200100da0 2024-03-21T10:36:20.940 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:20.942+0000 7f31fe000700 1 --2- 172.21.15.153:0/4133525015 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f31e406e460 0x7f31e4070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:20.941 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:20.943+0000 7f31fe000700 1 --2- 172.21.15.153:0/4133525015 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f31e406e460 0x7f31e4070920 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f3200199bf0 tx=0x7f31ec01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:36:20.945 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:20.947+0000 7f31f7400700 1 -- 172.21.15.153:0/4133525015 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f31f005b050 con 0x7f3200100da0 2024-03-21T10:36:21.182 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:21.184+0000 7f3205600700 1 -- 172.21.15.153:0/4133525015 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3200199220 con 0x7f31e406e460 2024-03-21T10:36:21.183 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:36:20 smithi153 ceph-mon[113733]: pgmap v736: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:36:21.184 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:21.186+0000 7f31f7400700 1 -- 172.21.15.153:0/4133525015 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f3200199220 con 0x7f31e406e460 2024-03-21T10:36:21.187 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:21.189+0000 7f3205600700 1 -- 172.21.15.153:0/4133525015 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f31e406e460 msgr2=0x7f31e4070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:21.187 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:21.189+0000 7f3205600700 1 --2- 172.21.15.153:0/4133525015 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f31e406e460 0x7f31e4070920 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f3200199bf0 tx=0x7f31ec01a040 comp rx=0 tx=0).stop 2024-03-21T10:36:21.188 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:21.189+0000 7f3205600700 1 -- 172.21.15.153:0/4133525015 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3200100da0 msgr2=0x7f320019e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:21.188 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:21.189+0000 7f3205600700 1 --2- 172.21.15.153:0/4133525015 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3200100da0 0x7f320019e700 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f31f000d8d0 tx=0x7f31f000dc90 comp rx=0 tx=0).stop 2024-03-21T10:36:21.188 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:21.189+0000 7f3205600700 1 -- 172.21.15.153:0/4133525015 shutdown_connections 2024-03-21T10:36:21.188 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:21.189+0000 7f3205600700 1 --2- 172.21.15.153:0/4133525015 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f31e406e460 0x7f31e4070920 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:21.188 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:21.189+0000 7f3205600700 1 --2- 172.21.15.153:0/4133525015 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3200100da0 0x7f320019e700 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:21.188 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:21.189+0000 7f3205600700 1 --2- 172.21.15.153:0/4133525015 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3200101750 0x7f320019ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:21.188 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:21.189+0000 7f3205600700 1 -- 172.21.15.153:0/4133525015 >> 172.21.15.153:0/4133525015 conn(0x7f32000fc630 msgr2=0x7f32000fea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:36:21.188 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:21.189+0000 7f3205600700 1 -- 172.21.15.153:0/4133525015 shutdown_connections 2024-03-21T10:36:21.188 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:21.189+0000 7f3205600700 1 -- 172.21.15.153:0/4133525015 wait complete. 2024-03-21T10:36:21.198 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:36:20 smithi174 ceph-mon[88004]: pgmap v736: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:36:21.202 INFO:teuthology.orchestra.run.smithi153.stdout:true 2024-03-21T10:36:21.356 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:21.357+0000 7f37e7000700 1 -- 172.21.15.153:0/1192279924 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f37e0100da0 msgr2=0x7f37e0101180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:21.356 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:21.357+0000 7f37e7000700 1 --2- 172.21.15.153:0/1192279924 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f37e0100da0 0x7f37e0101180 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f37d0009910 tx=0x7f37d0009c20 comp rx=0 tx=0).stop 2024-03-21T10:36:21.356 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:21.358+0000 7f37e7000700 1 -- 172.21.15.153:0/1192279924 shutdown_connections 2024-03-21T10:36:21.356 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:21.358+0000 7f37e7000700 1 --2- 172.21.15.153:0/1192279924 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f37e0101750 0x7f37e01057a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:21.357 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:21.358+0000 7f37e7000700 1 --2- 172.21.15.153:0/1192279924 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f37e0100da0 0x7f37e0101180 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:21.357 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:21.358+0000 7f37e7000700 1 -- 172.21.15.153:0/1192279924 >> 172.21.15.153:0/1192279924 conn(0x7f37e00fc630 msgr2=0x7f37e00fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:36:21.357 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:21.358+0000 7f37e7000700 1 -- 172.21.15.153:0/1192279924 shutdown_connections 2024-03-21T10:36:21.357 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:21.358+0000 7f37e7000700 1 -- 172.21.15.153:0/1192279924 wait complete. 2024-03-21T10:36:21.358 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:21.359+0000 7f37e7000700 1 Processor -- start 2024-03-21T10:36:21.358 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:21.359+0000 7f37e7000700 1 -- start start 2024-03-21T10:36:21.358 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:21.359+0000 7f37e7000700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f37e0100da0 0x7f37e019c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:21.358 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:21.359+0000 7f37e7000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f37e0101750 0x7f37e019cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:21.358 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:21.359+0000 7f37e7000700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37e019d0e0 con 0x7f37e0101750 2024-03-21T10:36:21.358 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:21.359+0000 7f37e7000700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37e0196620 con 0x7f37e0100da0 2024-03-21T10:36:21.358 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:21.360+0000 7f37df400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f37e0101750 0x7f37e019cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:21.358 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:21.360+0000 7f37df400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f37e0101750 0x7f37e019cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:48088/0 (socket says 172.21.15.153:48088) 2024-03-21T10:36:21.358 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:21.360+0000 7f37df400700 1 -- 172.21.15.153:0/1260697024 learned_addr learned my addr 172.21.15.153:0/1260697024 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:36:21.359 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:21.360+0000 7f37dfe00700 1 --2- 172.21.15.153:0/1260697024 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f37e0100da0 0x7f37e019c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:21.359 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:21.360+0000 7f37df400700 1 -- 172.21.15.153:0/1260697024 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f37e0100da0 msgr2=0x7f37e019c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:21.360 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:21.360+0000 7f37df400700 1 --2- 172.21.15.153:0/1260697024 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f37e0100da0 0x7f37e019c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:21.360 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:21.360+0000 7f37df400700 1 -- 172.21.15.153:0/1260697024 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37d00095f0 con 0x7f37e0101750 2024-03-21T10:36:21.360 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:21.360+0000 7f37df400700 1 --2- 172.21.15.153:0/1260697024 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f37e0101750 0x7f37e019cae0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f37cc00c960 tx=0x7f37cc00cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:36:21.360 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:21.361+0000 7f37dcc00700 1 -- 172.21.15.153:0/1260697024 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f37cc007a70 con 0x7f37e0101750 2024-03-21T10:36:21.360 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:21.361+0000 7f37dcc00700 1 -- 172.21.15.153:0/1260697024 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f37cc007bd0 con 0x7f37e0101750 2024-03-21T10:36:21.360 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:21.361+0000 7f37dcc00700 1 -- 172.21.15.153:0/1260697024 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f37cc0186c0 con 0x7f37e0101750 2024-03-21T10:36:21.360 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:21.361+0000 7f37e7000700 1 -- 172.21.15.153:0/1260697024 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37e0196900 con 0x7f37e0101750 2024-03-21T10:36:21.360 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:21.361+0000 7f37e7000700 1 -- 172.21.15.153:0/1260697024 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37e0196e50 con 0x7f37e0101750 2024-03-21T10:36:21.360 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:21.362+0000 7f37e7000700 1 -- 172.21.15.153:0/1260697024 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37e0109140 con 0x7f37e0101750 2024-03-21T10:36:21.362 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:21.363+0000 7f37dcc00700 1 -- 172.21.15.153:0/1260697024 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f37cc00faa0 con 0x7f37e0101750 2024-03-21T10:36:21.362 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:21.364+0000 7f37dcc00700 1 --2- 172.21.15.153:0/1260697024 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f37c406e460 0x7f37c4070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:21.363 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:21.364+0000 7f37dcc00700 1 -- 172.21.15.153:0/1260697024 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f37cc08e480 con 0x7f37e0101750 2024-03-21T10:36:21.363 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:21.365+0000 7f37dfe00700 1 --2- 172.21.15.153:0/1260697024 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f37c406e460 0x7f37c4070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:21.363 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:21.365+0000 7f37dfe00700 1 --2- 172.21.15.153:0/1260697024 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f37c406e460 0x7f37c4070920 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f37d0000c00 tx=0x7f37d0005fd0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:36:21.372 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:21.374+0000 7f37dcc00700 1 -- 172.21.15.153:0/1260697024 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f37cc05ae60 con 0x7f37e0101750 2024-03-21T10:36:21.620 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:21.622+0000 7f37e7000700 1 -- 172.21.15.153:0/1260697024 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f37e0197120 con 0x7f37c406e460 2024-03-21T10:36:21.623 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:21.625+0000 7f37dcc00700 1 -- 172.21.15.153:0/1260697024 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f37e0197120 con 0x7f37c406e460 2024-03-21T10:36:21.626 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:21.628+0000 7f37e7000700 1 -- 172.21.15.153:0/1260697024 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f37c406e460 msgr2=0x7f37c4070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:21.626 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:21.628+0000 7f37e7000700 1 --2- 172.21.15.153:0/1260697024 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f37c406e460 0x7f37c4070920 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f37d0000c00 tx=0x7f37d0005fd0 comp rx=0 tx=0).stop 2024-03-21T10:36:21.626 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:21.628+0000 7f37e7000700 1 -- 172.21.15.153:0/1260697024 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f37e0101750 msgr2=0x7f37e019cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:21.626 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:21.628+0000 7f37e7000700 1 --2- 172.21.15.153:0/1260697024 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f37e0101750 0x7f37e019cae0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f37cc00c960 tx=0x7f37cc00cd20 comp rx=0 tx=0).stop 2024-03-21T10:36:21.627 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:21.628+0000 7f37e7000700 1 -- 172.21.15.153:0/1260697024 shutdown_connections 2024-03-21T10:36:21.627 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:21.628+0000 7f37e7000700 1 --2- 172.21.15.153:0/1260697024 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f37e0100da0 0x7f37e019c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:21.627 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:21.628+0000 7f37e7000700 1 --2- 172.21.15.153:0/1260697024 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f37c406e460 0x7f37c4070920 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:21.627 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:21.628+0000 7f37e7000700 1 --2- 172.21.15.153:0/1260697024 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f37e0101750 0x7f37e019cae0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:21.627 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:21.628+0000 7f37e7000700 1 -- 172.21.15.153:0/1260697024 >> 172.21.15.153:0/1260697024 conn(0x7f37e00fc630 msgr2=0x7f37e00fdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:36:21.627 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:21.628+0000 7f37e7000700 1 -- 172.21.15.153:0/1260697024 shutdown_connections 2024-03-21T10:36:21.627 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:21.628+0000 7f37e7000700 1 -- 172.21.15.153:0/1260697024 wait complete. 2024-03-21T10:36:21.771 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:21.773+0000 7f9311400700 1 -- 172.21.15.153:0/671339510 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f930c1039b0 msgr2=0x7f930c107a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:21.771 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:21.773+0000 7f9311400700 1 --2- 172.21.15.153:0/671339510 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f930c1039b0 0x7f930c107a00 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f92f8009910 tx=0x7f92f8009c20 comp rx=0 tx=0).stop 2024-03-21T10:36:21.772 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:21.773+0000 7f9311400700 1 -- 172.21.15.153:0/671339510 shutdown_connections 2024-03-21T10:36:21.772 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:21.773+0000 7f9311400700 1 --2- 172.21.15.153:0/671339510 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f930c1039b0 0x7f930c107a00 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:21.772 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:21.773+0000 7f9311400700 1 --2- 172.21.15.153:0/671339510 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f930c103000 0x7f930c1033e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:21.772 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:21.773+0000 7f9311400700 1 -- 172.21.15.153:0/671339510 >> 172.21.15.153:0/671339510 conn(0x7f930c0fe8d0 msgr2=0x7f930c100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:36:21.772 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:21.773+0000 7f9311400700 1 -- 172.21.15.153:0/671339510 shutdown_connections 2024-03-21T10:36:21.772 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:21.773+0000 7f9311400700 1 -- 172.21.15.153:0/671339510 wait complete. 2024-03-21T10:36:21.772 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:21.774+0000 7f9311400700 1 Processor -- start 2024-03-21T10:36:21.773 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:21.774+0000 7f9311400700 1 -- start start 2024-03-21T10:36:21.774 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:21.775+0000 7f9311400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f930c103000 0x7f930c19e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:21.774 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:21.775+0000 7f9311400700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f930c1039b0 0x7f930c19ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:21.774 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:21.775+0000 7f9311400700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f930c19f490 con 0x7f930c103000 2024-03-21T10:36:21.774 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:21.775+0000 7f9311400700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f930c198940 con 0x7f930c1039b0 2024-03-21T10:36:21.774 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:21.775+0000 7f930a800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f930c103000 0x7f930c19e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:21.774 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:21.775+0000 7f930a800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f930c103000 0x7f930c19e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:48114/0 (socket says 172.21.15.153:48114) 2024-03-21T10:36:21.774 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:21.775+0000 7f930a800700 1 -- 172.21.15.153:0/953798831 learned_addr learned my addr 172.21.15.153:0/953798831 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:36:21.774 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:21.775+0000 7f9309e00700 1 --2- 172.21.15.153:0/953798831 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f930c1039b0 0x7f930c19ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:21.774 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:21.775+0000 7f930a800700 1 -- 172.21.15.153:0/953798831 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f930c1039b0 msgr2=0x7f930c19ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:21.774 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:21.775+0000 7f930a800700 1 --2- 172.21.15.153:0/953798831 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f930c1039b0 0x7f930c19ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:21.774 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:21.775+0000 7f930a800700 1 -- 172.21.15.153:0/953798831 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92f80095f0 con 0x7f930c103000 2024-03-21T10:36:21.775 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:21.776+0000 7f930a800700 1 --2- 172.21.15.153:0/953798831 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f930c103000 0x7f930c19e8c0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f92fc00b4b0 tx=0x7f92fc00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:36:21.775 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:21.776+0000 7f9303400700 1 -- 172.21.15.153:0/953798831 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f92fc011800 con 0x7f930c103000 2024-03-21T10:36:21.775 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:21.776+0000 7f9303400700 1 -- 172.21.15.153:0/953798831 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f92fc011e20 con 0x7f930c103000 2024-03-21T10:36:21.775 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:21.776+0000 7f9311400700 1 -- 172.21.15.153:0/953798831 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f930c198c20 con 0x7f930c103000 2024-03-21T10:36:21.775 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:21.776+0000 7f9303400700 1 -- 172.21.15.153:0/953798831 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f92fc00f2e0 con 0x7f930c103000 2024-03-21T10:36:21.775 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:21.776+0000 7f9311400700 1 -- 172.21.15.153:0/953798831 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f930c199170 con 0x7f930c103000 2024-03-21T10:36:21.776 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:21.777+0000 7f9311400700 1 -- 172.21.15.153:0/953798831 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f930c10b330 con 0x7f930c103000 2024-03-21T10:36:21.776 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:21.778+0000 7f9303400700 1 -- 172.21.15.153:0/953798831 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f92fc010380 con 0x7f930c103000 2024-03-21T10:36:21.777 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:21.779+0000 7f9303400700 1 --2- 172.21.15.153:0/953798831 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f92f006e460 0x7f92f0070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:21.778 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:21.779+0000 7f9303400700 1 -- 172.21.15.153:0/953798831 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f92fc08e190 con 0x7f930c103000 2024-03-21T10:36:21.778 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:21.780+0000 7f9309e00700 1 --2- 172.21.15.153:0/953798831 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f92f006e460 0x7f92f0070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:21.778 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:21.780+0000 7f9309e00700 1 --2- 172.21.15.153:0/953798831 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f92f006e460 0x7f92f0070920 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f92f8000c00 tx=0x7f92f8005fd0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:36:21.782 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:21.784+0000 7f9303400700 1 -- 172.21.15.153:0/953798831 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f92fc05ace0 con 0x7f930c103000 2024-03-21T10:36:21.948 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:36:21 smithi174 ceph-mon[88004]: from='client.35322 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:36:22.000 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:22.002+0000 7f9311400700 1 -- 172.21.15.153:0/953798831 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f930c1993f0 con 0x7f92f006e460 2024-03-21T10:36:22.016 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:22.018+0000 7f9303400700 1 -- 172.21.15.153:0/953798831 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f930c1993f0 con 0x7f92f006e460 2024-03-21T10:36:22.016 INFO:teuthology.orchestra.run.smithi153.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-21T10:36:22.016 INFO:teuthology.orchestra.run.smithi153.stdout:alertmanager.smithi153 smithi153 *:9093,9094 running (25m) 54s ago 38m 15.6M - 0.25.0 c8568f914cd2 9ed3f2200b5f 2024-03-21T10:36:22.017 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi153 smithi153 running (39m) 54s ago 39m 10.4M - 18.2.1 5be31c24972a 5b61994396a6 2024-03-21T10:36:22.017 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi174 smithi174 running (37m) 57s ago 37m 10.1M - 18.2.1 5be31c24972a 4f7a5fc3ac28 2024-03-21T10:36:22.017 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi153 smithi153 running (21m) 54s ago 39m 6656k - 19.0.0-2231-g2edfee68 237e43c7a65f 32c21104d5b0 2024-03-21T10:36:22.017 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi174 smithi174 running (21m) 57s ago 37m 6660k - 19.0.0-2231-g2edfee68 237e43c7a65f a04e48967ab6 2024-03-21T10:36:22.017 INFO:teuthology.orchestra.run.smithi153.stdout:grafana.smithi153 smithi153 *:3000 running (24m) 54s ago 38m 75.2M - 9.4.12 2bacad6d85d8 ee24026ab786 2024-03-21T10:36:22.017 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.iuyluj smithi153 running (16m) 54s ago 31m 18.7M - 19.0.0-2231-g2edfee68 237e43c7a65f 87aff8a5de70 2024-03-21T10:36:22.017 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.zipckr smithi153 running (17m) 54s ago 31m 18.9M - 19.0.0-2231-g2edfee68 237e43c7a65f ccc262bd5dd3 2024-03-21T10:36:22.017 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.nhozag smithi174 running (63s) 57s ago 31m 10.9M - 19.0.0-2231-g2edfee68 237e43c7a65f 9e02aea72c96 2024-03-21T10:36:22.017 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.yyimst smithi174 running (16m) 57s ago 31m 18.5M - 19.0.0-2231-g2edfee68 237e43c7a65f fc1962f9351e 2024-03-21T10:36:22.017 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi153.tluxni smithi153 *:8443,9283,8765 running (28m) 54s ago 40m 512M - 19.0.0-2231-g2edfee68 237e43c7a65f 7ffee0e55382 2024-03-21T10:36:22.017 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi174.qpcwat smithi174 *:8443,9283,8765 running (26m) 57s ago 37m 459M - 19.0.0-2231-g2edfee68 237e43c7a65f 748968f216ab 2024-03-21T10:36:22.017 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi153 smithi153 running (23m) 54s ago 41m 70.8M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f dd8038303dc3 2024-03-21T10:36:22.017 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi174 smithi174 running (22m) 57s ago 37m 51.9M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f 29d78a788f2c 2024-03-21T10:36:22.018 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi153 smithi153 *:9100 running (26m) 54s ago 39m 8140k - 1.5.0 0da6a335fe13 bb835edc0e0b 2024-03-21T10:36:22.018 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi174 smithi174 *:9100 running (26m) 57s ago 37m 8610k - 1.5.0 0da6a335fe13 7c4ecad790fa 2024-03-21T10:36:22.018 INFO:teuthology.orchestra.run.smithi153.stdout:osd.0 smithi153 running (20m) 54s ago 36m 118M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 507900fd4791 2024-03-21T10:36:22.018 INFO:teuthology.orchestra.run.smithi153.stdout:osd.1 smithi153 running (19m) 54s ago 35m 93.0M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 0d646452e9c1 2024-03-21T10:36:22.018 INFO:teuthology.orchestra.run.smithi153.stdout:osd.2 smithi153 running (19m) 54s ago 35m 91.2M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 1326f2afdfcd 2024-03-21T10:36:22.018 INFO:teuthology.orchestra.run.smithi153.stdout:osd.3 smithi174 running (18m) 57s ago 34m 87.5M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 70aafac71ad5 2024-03-21T10:36:22.018 INFO:teuthology.orchestra.run.smithi153.stdout:osd.4 smithi174 running (18m) 57s ago 34m 95.8M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 0d746ce6e79d 2024-03-21T10:36:22.018 INFO:teuthology.orchestra.run.smithi153.stdout:osd.5 smithi174 running (17m) 57s ago 33m 90.3M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 53e5f35b8cac 2024-03-21T10:36:22.018 INFO:teuthology.orchestra.run.smithi153.stdout:prometheus.smithi153 smithi153 *:9095 running (25m) 54s ago 38m 88.1M - 2.43.0 a07b618ecd1d 7d95561e50c2 2024-03-21T10:36:22.019 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:22.020+0000 7f9311400700 1 -- 172.21.15.153:0/953798831 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f92f006e460 msgr2=0x7f92f0070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:22.019 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:22.020+0000 7f9311400700 1 --2- 172.21.15.153:0/953798831 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f92f006e460 0x7f92f0070920 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f92f8000c00 tx=0x7f92f8005fd0 comp rx=0 tx=0).stop 2024-03-21T10:36:22.019 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:22.020+0000 7f9311400700 1 -- 172.21.15.153:0/953798831 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f930c103000 msgr2=0x7f930c19e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:22.019 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:22.020+0000 7f9311400700 1 --2- 172.21.15.153:0/953798831 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f930c103000 0x7f930c19e8c0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f92fc00b4b0 tx=0x7f92fc00b870 comp rx=0 tx=0).stop 2024-03-21T10:36:22.019 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:22.021+0000 7f9311400700 1 -- 172.21.15.153:0/953798831 shutdown_connections 2024-03-21T10:36:22.019 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:22.021+0000 7f9311400700 1 --2- 172.21.15.153:0/953798831 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f92f006e460 0x7f92f0070920 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:22.019 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:22.021+0000 7f9311400700 1 --2- 172.21.15.153:0/953798831 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f930c103000 0x7f930c19e8c0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:22.019 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:22.021+0000 7f9311400700 1 --2- 172.21.15.153:0/953798831 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f930c1039b0 0x7f930c19ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:22.019 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:22.021+0000 7f9311400700 1 -- 172.21.15.153:0/953798831 >> 172.21.15.153:0/953798831 conn(0x7f930c0fe8d0 msgr2=0x7f930c0fffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:36:22.020 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:22.021+0000 7f9311400700 1 -- 172.21.15.153:0/953798831 shutdown_connections 2024-03-21T10:36:22.020 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:22.021+0000 7f9311400700 1 -- 172.21.15.153:0/953798831 wait complete. 2024-03-21T10:36:22.168 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:22.169+0000 7fabf0c00700 1 -- 172.21.15.153:0/3950816303 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fabec107bd0 msgr2=0x7fabec10bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:22.168 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:22.169+0000 7fabf0c00700 1 --2- 172.21.15.153:0/3950816303 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fabec107bd0 0x7fabec10bc10 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fabd8009910 tx=0x7fabd8009c20 comp rx=0 tx=0).stop 2024-03-21T10:36:22.168 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:22.169+0000 7fabf0c00700 1 -- 172.21.15.153:0/3950816303 shutdown_connections 2024-03-21T10:36:22.168 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:22.169+0000 7fabf0c00700 1 --2- 172.21.15.153:0/3950816303 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fabec107bd0 0x7fabec10bc10 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:22.168 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:22.169+0000 7fabf0c00700 1 --2- 172.21.15.153:0/3950816303 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fabec107220 0x7fabec107600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:22.168 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:22.169+0000 7fabf0c00700 1 -- 172.21.15.153:0/3950816303 >> 172.21.15.153:0/3950816303 conn(0x7fabec0767e0 msgr2=0x7fabec078c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:36:22.168 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:36:21 smithi153 ceph-mon[113733]: from='client.35322 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:36:22.169 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:22.169+0000 7fabf0c00700 1 -- 172.21.15.153:0/3950816303 shutdown_connections 2024-03-21T10:36:22.169 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:22.170+0000 7fabf0c00700 1 -- 172.21.15.153:0/3950816303 wait complete. 2024-03-21T10:36:22.169 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:22.171+0000 7fabf0c00700 1 Processor -- start 2024-03-21T10:36:22.169 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:22.171+0000 7fabf0c00700 1 -- start start 2024-03-21T10:36:22.170 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:22.172+0000 7fabf0c00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fabec107220 0x7fabec1a29f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:22.170 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:22.172+0000 7fabf0c00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fabec107bd0 0x7fabec1a2f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:22.170 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:22.172+0000 7fabf0c00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabec1a3530 con 0x7fabec107220 2024-03-21T10:36:22.170 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:22.172+0000 7fabf0c00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabec19ca70 con 0x7fabec107bd0 2024-03-21T10:36:22.170 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:22.172+0000 7fabeb400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fabec107220 0x7fabec1a29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:22.171 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:22.172+0000 7fabeb400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fabec107220 0x7fabec1a29f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:48122/0 (socket says 172.21.15.153:48122) 2024-03-21T10:36:22.171 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:22.172+0000 7fabeb400700 1 -- 172.21.15.153:0/1606304654 learned_addr learned my addr 172.21.15.153:0/1606304654 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:36:22.172 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:22.172+0000 7fabeaa00700 1 --2- 172.21.15.153:0/1606304654 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fabec107bd0 0x7fabec1a2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:22.172 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:22.173+0000 7fabeb400700 1 -- 172.21.15.153:0/1606304654 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fabec107bd0 msgr2=0x7fabec1a2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:22.172 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:22.173+0000 7fabeb400700 1 --2- 172.21.15.153:0/1606304654 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fabec107bd0 0x7fabec1a2f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:22.172 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:22.173+0000 7fabeb400700 1 -- 172.21.15.153:0/1606304654 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabd80095f0 con 0x7fabec107220 2024-03-21T10:36:22.172 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:22.173+0000 7fabeb400700 1 --2- 172.21.15.153:0/1606304654 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fabec107220 0x7fabec1a29f0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fabdc00c930 tx=0x7fabdc00ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:36:22.172 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:22.173+0000 7fabe3e00700 1 -- 172.21.15.153:0/1606304654 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fabdc007ad0 con 0x7fabec107220 2024-03-21T10:36:22.172 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:22.173+0000 7fabe3e00700 1 -- 172.21.15.153:0/1606304654 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fabdc007c30 con 0x7fabec107220 2024-03-21T10:36:22.172 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:22.173+0000 7fabf0c00700 1 -- 172.21.15.153:0/1606304654 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabec19cd50 con 0x7fabec107220 2024-03-21T10:36:22.172 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:22.174+0000 7fabe3e00700 1 -- 172.21.15.153:0/1606304654 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fabdc018610 con 0x7fabec107220 2024-03-21T10:36:22.173 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:22.174+0000 7fabf0c00700 1 -- 172.21.15.153:0/1606304654 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabec19d2a0 con 0x7fabec107220 2024-03-21T10:36:22.174 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:22.176+0000 7fabf0c00700 1 -- 172.21.15.153:0/1606304654 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabec10f610 con 0x7fabec107220 2024-03-21T10:36:22.174 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:22.176+0000 7fabe3e00700 1 -- 172.21.15.153:0/1606304654 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fabdc00f450 con 0x7fabec107220 2024-03-21T10:36:22.176 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:22.177+0000 7fabe3e00700 1 --2- 172.21.15.153:0/1606304654 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fabd006e460 0x7fabd0070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:22.176 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:22.178+0000 7fabe3e00700 1 -- 172.21.15.153:0/1606304654 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fabdc08e710 con 0x7fabec107220 2024-03-21T10:36:22.176 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:22.178+0000 7fabeaa00700 1 --2- 172.21.15.153:0/1606304654 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fabd006e460 0x7fabd0070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:22.177 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:22.179+0000 7fabeaa00700 1 --2- 172.21.15.153:0/1606304654 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fabd006e460 0x7fabd0070920 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fabec077000 tx=0x7fabd801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:36:22.180 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:22.182+0000 7fabe3e00700 1 -- 172.21.15.153:0/1606304654 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fabdc05b430 con 0x7fabec107220 2024-03-21T10:36:22.482 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:22.484+0000 7fabf0c00700 1 -- 172.21.15.153:0/1606304654 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fabec19e020 con 0x7fabec107220 2024-03-21T10:36:22.483 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:22.485+0000 7fabe3e00700 1 -- 172.21.15.153:0/1606304654 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fabdc05afc0 con 0x7fabec107220 2024-03-21T10:36:22.484 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:36:22.484 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": { 2024-03-21T10:36:22.484 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:36:22.484 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:36:22.484 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": { 2024-03-21T10:36:22.484 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:36:22.484 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:36:22.484 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": { 2024-03-21T10:36:22.484 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 6 2024-03-21T10:36:22.485 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:36:22.485 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": { 2024-03-21T10:36:22.485 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 4 2024-03-21T10:36:22.485 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:36:22.485 INFO:teuthology.orchestra.run.smithi153.stdout: "overall": { 2024-03-21T10:36:22.485 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 14 2024-03-21T10:36:22.485 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-03-21T10:36:22.485 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:36:22.487 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:22.488+0000 7fabf0c00700 1 -- 172.21.15.153:0/1606304654 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fabd006e460 msgr2=0x7fabd0070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:22.487 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:22.488+0000 7fabf0c00700 1 --2- 172.21.15.153:0/1606304654 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fabd006e460 0x7fabd0070920 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fabec077000 tx=0x7fabd801a040 comp rx=0 tx=0).stop 2024-03-21T10:36:22.487 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:22.488+0000 7fabf0c00700 1 -- 172.21.15.153:0/1606304654 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fabec107220 msgr2=0x7fabec1a29f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:22.487 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:22.488+0000 7fabf0c00700 1 --2- 172.21.15.153:0/1606304654 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fabec107220 0x7fabec1a29f0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fabdc00c930 tx=0x7fabdc00ccf0 comp rx=0 tx=0).stop 2024-03-21T10:36:22.487 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:22.488+0000 7fabf0c00700 1 -- 172.21.15.153:0/1606304654 shutdown_connections 2024-03-21T10:36:22.487 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:22.488+0000 7fabf0c00700 1 --2- 172.21.15.153:0/1606304654 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fabd006e460 0x7fabd0070920 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:22.487 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:22.488+0000 7fabf0c00700 1 --2- 172.21.15.153:0/1606304654 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fabec107220 0x7fabec1a29f0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:22.487 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:22.488+0000 7fabf0c00700 1 --2- 172.21.15.153:0/1606304654 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fabec107bd0 0x7fabec1a2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:22.487 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:22.488+0000 7fabf0c00700 1 -- 172.21.15.153:0/1606304654 >> 172.21.15.153:0/1606304654 conn(0x7fabec0767e0 msgr2=0x7fabec078bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:36:22.487 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:22.488+0000 7fabf0c00700 1 -- 172.21.15.153:0/1606304654 shutdown_connections 2024-03-21T10:36:22.487 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:22.488+0000 7fabf0c00700 1 -- 172.21.15.153:0/1606304654 wait complete. 2024-03-21T10:36:22.640 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:22.641+0000 7f63a2a00700 1 -- 172.21.15.153:0/2624534414 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f639c1037c0 msgr2=0x7f639c107810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:22.640 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:22.641+0000 7f63a2a00700 1 --2- 172.21.15.153:0/2624534414 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f639c1037c0 0x7f639c107810 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f6388009910 tx=0x7f6388009c20 comp rx=0 tx=0).stop 2024-03-21T10:36:22.640 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:22.641+0000 7f63a2a00700 1 -- 172.21.15.153:0/2624534414 shutdown_connections 2024-03-21T10:36:22.640 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:22.641+0000 7f63a2a00700 1 --2- 172.21.15.153:0/2624534414 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f639c1037c0 0x7f639c107810 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:22.640 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:22.641+0000 7f63a2a00700 1 --2- 172.21.15.153:0/2624534414 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f639c102e10 0x7f639c1031f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:22.640 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:22.641+0000 7f63a2a00700 1 -- 172.21.15.153:0/2624534414 >> 172.21.15.153:0/2624534414 conn(0x7f639c0fe6c0 msgr2=0x7f639c100ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:36:22.640 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:22.641+0000 7f63a2a00700 1 -- 172.21.15.153:0/2624534414 shutdown_connections 2024-03-21T10:36:22.641 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:22.641+0000 7f63a2a00700 1 -- 172.21.15.153:0/2624534414 wait complete. 2024-03-21T10:36:22.642 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:22.642+0000 7f63a2a00700 1 Processor -- start 2024-03-21T10:36:22.642 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:22.642+0000 7f63a2a00700 1 -- start start 2024-03-21T10:36:22.642 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:22.643+0000 7f63a2a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f639c102e10 0x7f639c19e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:22.642 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:22.643+0000 7f63a2a00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f639c1037c0 0x7f639c19ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:22.642 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:22.643+0000 7f63a2a00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f639c19f260 con 0x7f639c102e10 2024-03-21T10:36:22.642 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:22.643+0000 7f63a2a00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f639c198710 con 0x7f639c1037c0 2024-03-21T10:36:22.642 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:22.643+0000 7f63a1600700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f639c102e10 0x7f639c19e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:22.643 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:22.643+0000 7f63a1600700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f639c102e10 0x7f639c19e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:48138/0 (socket says 172.21.15.153:48138) 2024-03-21T10:36:22.643 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:22.643+0000 7f63a1600700 1 -- 172.21.15.153:0/3033830844 learned_addr learned my addr 172.21.15.153:0/3033830844 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:36:22.643 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:22.643+0000 7f63a0c00700 1 --2- 172.21.15.153:0/3033830844 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f639c1037c0 0x7f639c19ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:22.643 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:22.644+0000 7f63a1600700 1 -- 172.21.15.153:0/3033830844 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f639c1037c0 msgr2=0x7f639c19ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:22.643 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:22.644+0000 7f63a1600700 1 --2- 172.21.15.153:0/3033830844 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f639c1037c0 0x7f639c19ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:22.643 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:22.644+0000 7f63a1600700 1 -- 172.21.15.153:0/3033830844 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63880095f0 con 0x7f639c102e10 2024-03-21T10:36:22.643 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:22.644+0000 7f63a0c00700 1 --2- 172.21.15.153:0/3033830844 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f639c1037c0 0x7f639c19ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:36:22.643 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:22.644+0000 7f63a1600700 1 --2- 172.21.15.153:0/3033830844 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f639c102e10 0x7f639c19e690 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f639000b4b0 tx=0x7f639000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:36:22.644 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:22.644+0000 7f639a000700 1 -- 172.21.15.153:0/3033830844 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6390011800 con 0x7f639c102e10 2024-03-21T10:36:22.644 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:22.644+0000 7f639a000700 1 -- 172.21.15.153:0/3033830844 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6390011e20 con 0x7f639c102e10 2024-03-21T10:36:22.644 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:22.644+0000 7f63a2a00700 1 -- 172.21.15.153:0/3033830844 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f639c1989f0 con 0x7f639c102e10 2024-03-21T10:36:22.644 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:22.644+0000 7f639a000700 1 -- 172.21.15.153:0/3033830844 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f639000f2e0 con 0x7f639c102e10 2024-03-21T10:36:22.644 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:22.644+0000 7f63a2a00700 1 -- 172.21.15.153:0/3033830844 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f639c198f40 con 0x7f639c102e10 2024-03-21T10:36:22.644 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:22.645+0000 7f63a2a00700 1 -- 172.21.15.153:0/3033830844 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f639c10b140 con 0x7f639c102e10 2024-03-21T10:36:22.645 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:22.646+0000 7f639a000700 1 -- 172.21.15.153:0/3033830844 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f6390010380 con 0x7f639c102e10 2024-03-21T10:36:22.645 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:22.647+0000 7f639a000700 1 --2- 172.21.15.153:0/3033830844 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f638406e460 0x7f6384070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:22.645 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:22.647+0000 7f639a000700 1 -- 172.21.15.153:0/3033830844 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f639008e0c0 con 0x7f639c102e10 2024-03-21T10:36:22.646 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:22.647+0000 7f63a0c00700 1 --2- 172.21.15.153:0/3033830844 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f638406e460 0x7f6384070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:22.646 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:22.648+0000 7f63a0c00700 1 --2- 172.21.15.153:0/3033830844 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f638406e460 0x7f6384070920 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f6388006010 tx=0x7f638800b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:36:22.650 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:22.652+0000 7f639a000700 1 -- 172.21.15.153:0/3033830844 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f639005ad40 con 0x7f639c102e10 2024-03-21T10:36:22.904 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:22.906+0000 7f63a2a00700 1 -- 172.21.15.153:0/3033830844 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f639c199210 con 0x7f639c102e10 2024-03-21T10:36:22.905 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:22.906+0000 7f639a000700 1 -- 172.21.15.153:0/3033830844 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1876 (secure 0 0 0) 0x7f639005a8d0 con 0x7f639c102e10 2024-03-21T10:36:22.906 INFO:teuthology.orchestra.run.smithi153.stdout:e32 2024-03-21T10:36:22.906 INFO:teuthology.orchestra.run.smithi153.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-21T10:36:22.906 INFO:teuthology.orchestra.run.smithi153.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:36:22.906 INFO:teuthology.orchestra.run.smithi153.stdout:legacy client fscid: 1 2024-03-21T10:36:22.906 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:36:22.906 INFO:teuthology.orchestra.run.smithi153.stdout:Filesystem 'cephfs' (1) 2024-03-21T10:36:22.906 INFO:teuthology.orchestra.run.smithi153.stdout:fs_name cephfs 2024-03-21T10:36:22.906 INFO:teuthology.orchestra.run.smithi153.stdout:epoch 23 2024-03-21T10:36:22.906 INFO:teuthology.orchestra.run.smithi153.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-21T10:36:22.906 INFO:teuthology.orchestra.run.smithi153.stdout:created 2024-03-21T10:04:27.086576+0000 2024-03-21T10:36:22.907 INFO:teuthology.orchestra.run.smithi153.stdout:modified 2024-03-21T10:19:05.022092+0000 2024-03-21T10:36:22.907 INFO:teuthology.orchestra.run.smithi153.stdout:tableserver 0 2024-03-21T10:36:22.907 INFO:teuthology.orchestra.run.smithi153.stdout:root 0 2024-03-21T10:36:22.907 INFO:teuthology.orchestra.run.smithi153.stdout:session_timeout 60 2024-03-21T10:36:22.907 INFO:teuthology.orchestra.run.smithi153.stdout:session_autoclose 300 2024-03-21T10:36:22.907 INFO:teuthology.orchestra.run.smithi153.stdout:max_file_size 1099511627776 2024-03-21T10:36:22.907 INFO:teuthology.orchestra.run.smithi153.stdout:max_xattr_size 65536 2024-03-21T10:36:22.907 INFO:teuthology.orchestra.run.smithi153.stdout:required_client_features {} 2024-03-21T10:36:22.907 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure 0 2024-03-21T10:36:22.907 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure_osd_epoch 94 2024-03-21T10:36:22.907 INFO:teuthology.orchestra.run.smithi153.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:36:22.907 INFO:teuthology.orchestra.run.smithi153.stdout:max_mds 2 2024-03-21T10:36:22.907 INFO:teuthology.orchestra.run.smithi153.stdout:in 0,1 2024-03-21T10:36:22.907 INFO:teuthology.orchestra.run.smithi153.stdout:up {} 2024-03-21T10:36:22.908 INFO:teuthology.orchestra.run.smithi153.stdout:failed 0,1 2024-03-21T10:36:22.908 INFO:teuthology.orchestra.run.smithi153.stdout:damaged 2024-03-21T10:36:22.908 INFO:teuthology.orchestra.run.smithi153.stdout:stopped 2024-03-21T10:36:22.908 INFO:teuthology.orchestra.run.smithi153.stdout:data_pools [3] 2024-03-21T10:36:22.908 INFO:teuthology.orchestra.run.smithi153.stdout:metadata_pool 2 2024-03-21T10:36:22.908 INFO:teuthology.orchestra.run.smithi153.stdout:inline_data enabled 2024-03-21T10:36:22.908 INFO:teuthology.orchestra.run.smithi153.stdout:balancer 2024-03-21T10:36:22.908 INFO:teuthology.orchestra.run.smithi153.stdout:bal_rank_mask -1 2024-03-21T10:36:22.908 INFO:teuthology.orchestra.run.smithi153.stdout:standby_count_wanted 1 2024-03-21T10:36:22.908 INFO:teuthology.orchestra.run.smithi153.stdout:qdb_cluster leader: 0 members: 2024-03-21T10:36:22.908 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:36:22.908 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:36:22.908 INFO:teuthology.orchestra.run.smithi153.stdout:Standby daemons: 2024-03-21T10:36:22.908 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:36:22.909 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.zipckr{-1:34442} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6826/1103846295,v1:172.21.15.153:6827/1103846295] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:36:22.909 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.iuyluj{-1:34474} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6828/1738630832,v1:172.21.15.153:6829/1738630832] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:36:22.909 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.yyimst{-1:44313} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.174:6824/4127786797,v1:172.21.15.174:6825/4127786797] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:36:22.909 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.nhozag{-1:44687} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.174:6826/2767517845,v1:172.21.15.174:6827/2767517845] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:36:22.910 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:22.909+0000 7f63a2a00700 1 -- 172.21.15.153:0/3033830844 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f638406e460 msgr2=0x7f6384070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:22.910 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:22.909+0000 7f63a2a00700 1 --2- 172.21.15.153:0/3033830844 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f638406e460 0x7f6384070920 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f6388006010 tx=0x7f638800b540 comp rx=0 tx=0).stop 2024-03-21T10:36:22.910 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:22.909+0000 7f63a2a00700 1 -- 172.21.15.153:0/3033830844 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f639c102e10 msgr2=0x7f639c19e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:22.910 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:22.909+0000 7f63a2a00700 1 --2- 172.21.15.153:0/3033830844 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f639c102e10 0x7f639c19e690 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f639000b4b0 tx=0x7f639000b870 comp rx=0 tx=0).stop 2024-03-21T10:36:22.910 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:22.909+0000 7f63a2a00700 1 -- 172.21.15.153:0/3033830844 shutdown_connections 2024-03-21T10:36:22.910 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:22.909+0000 7f63a2a00700 1 --2- 172.21.15.153:0/3033830844 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f638406e460 0x7f6384070920 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:22.910 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:22.909+0000 7f63a2a00700 1 --2- 172.21.15.153:0/3033830844 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f639c102e10 0x7f639c19e690 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:22.910 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:22.909+0000 7f63a2a00700 1 --2- 172.21.15.153:0/3033830844 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f639c1037c0 0x7f639c19ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:22.910 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:22.909+0000 7f63a2a00700 1 -- 172.21.15.153:0/3033830844 >> 172.21.15.153:0/3033830844 conn(0x7f639c0fe6c0 msgr2=0x7f639c0ffea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:36:22.910 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:22.909+0000 7f63a2a00700 1 -- 172.21.15.153:0/3033830844 shutdown_connections 2024-03-21T10:36:22.910 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:22.910+0000 7f63a2a00700 1 -- 172.21.15.153:0/3033830844 wait complete. 2024-03-21T10:36:22.910 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 32 2024-03-21T10:36:23.062 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:23.063+0000 7fe6dd600700 1 -- 172.21.15.153:0/1541720034 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6d8069020 msgr2=0x7fe6d8105790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:23.062 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:23.063+0000 7fe6dd600700 1 --2- 172.21.15.153:0/1541720034 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6d8069020 0x7fe6d8105790 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fe6c4009910 tx=0x7fe6c4009c20 comp rx=0 tx=0).stop 2024-03-21T10:36:23.062 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:23.064+0000 7fe6dd600700 1 -- 172.21.15.153:0/1541720034 shutdown_connections 2024-03-21T10:36:23.062 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:23.064+0000 7fe6dd600700 1 --2- 172.21.15.153:0/1541720034 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6d8069020 0x7fe6d8105790 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:23.063 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:23.064+0000 7fe6dd600700 1 --2- 172.21.15.153:0/1541720034 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe6d8068670 0x7fe6d8068a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:23.063 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:23.064+0000 7fe6dd600700 1 -- 172.21.15.153:0/1541720034 >> 172.21.15.153:0/1541720034 conn(0x7fe6d80755a0 msgr2=0x7fe6d80759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:36:23.063 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:23.064+0000 7fe6dd600700 1 -- 172.21.15.153:0/1541720034 shutdown_connections 2024-03-21T10:36:23.063 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:23.064+0000 7fe6dd600700 1 -- 172.21.15.153:0/1541720034 wait complete. 2024-03-21T10:36:23.064 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:23.066+0000 7fe6dd600700 1 Processor -- start 2024-03-21T10:36:23.064 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:23.066+0000 7fe6dd600700 1 -- start start 2024-03-21T10:36:23.065 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:23.066+0000 7fe6dd600700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe6d8068670 0x7fe6d819c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:23.065 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:23.066+0000 7fe6dd600700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6d8069020 0x7fe6d819cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:23.066 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:23.066+0000 7fe6dd600700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6d819d140 con 0x7fe6d8069020 2024-03-21T10:36:23.066 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:23.067+0000 7fe6dd600700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6d81966d0 con 0x7fe6d8068670 2024-03-21T10:36:23.066 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:23.067+0000 7fe6d6000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6d8069020 0x7fe6d819cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:23.067 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:23.067+0000 7fe6d6000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6d8069020 0x7fe6d819cb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:33464/0 (socket says 172.21.15.153:33464) 2024-03-21T10:36:23.067 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:23.067+0000 7fe6d6000700 1 -- 172.21.15.153:0/3251068923 learned_addr learned my addr 172.21.15.153:0/3251068923 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:36:23.067 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:23.067+0000 7fe6d6a00700 1 --2- 172.21.15.153:0/3251068923 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe6d8068670 0x7fe6d819c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:23.067 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:23.067+0000 7fe6d6000700 1 -- 172.21.15.153:0/3251068923 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe6d8068670 msgr2=0x7fe6d819c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:23.067 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:23.067+0000 7fe6d6000700 1 --2- 172.21.15.153:0/3251068923 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe6d8068670 0x7fe6d819c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:23.067 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:23.067+0000 7fe6d6000700 1 -- 172.21.15.153:0/3251068923 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6c40095f0 con 0x7fe6d8069020 2024-03-21T10:36:23.068 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:23.068+0000 7fe6d6000700 1 --2- 172.21.15.153:0/3251068923 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6d8069020 0x7fe6d819cb20 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fe6c400bda0 tx=0x7fe6c400be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:36:23.068 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:23.068+0000 7fe6cf400700 1 -- 172.21.15.153:0/3251068923 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe6c401d070 con 0x7fe6d8069020 2024-03-21T10:36:23.068 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:23.068+0000 7fe6cf400700 1 -- 172.21.15.153:0/3251068923 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe6c4004fb0 con 0x7fe6d8069020 2024-03-21T10:36:23.068 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:23.068+0000 7fe6dd600700 1 -- 172.21.15.153:0/3251068923 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6d8196950 con 0x7fe6d8069020 2024-03-21T10:36:23.068 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:23.068+0000 7fe6cf400700 1 -- 172.21.15.153:0/3251068923 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe6c400f460 con 0x7fe6d8069020 2024-03-21T10:36:23.068 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:23.068+0000 7fe6dd600700 1 -- 172.21.15.153:0/3251068923 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6d8196e40 con 0x7fe6d8069020 2024-03-21T10:36:23.068 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:23.069+0000 7fe6dd600700 1 -- 172.21.15.153:0/3251068923 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6d81090e0 con 0x7fe6d8069020 2024-03-21T10:36:23.069 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:23.070+0000 7fe6cf400700 1 -- 172.21.15.153:0/3251068923 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fe6c4022800 con 0x7fe6d8069020 2024-03-21T10:36:23.069 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:23.071+0000 7fe6cf400700 1 --2- 172.21.15.153:0/3251068923 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe6bc06e460 0x7fe6bc070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:23.070 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:23.071+0000 7fe6cf400700 1 -- 172.21.15.153:0/3251068923 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fe6c408faf0 con 0x7fe6d8069020 2024-03-21T10:36:23.070 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:23.072+0000 7fe6d6a00700 1 --2- 172.21.15.153:0/3251068923 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe6bc06e460 0x7fe6bc070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:23.070 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:23.072+0000 7fe6d6a00700 1 --2- 172.21.15.153:0/3251068923 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe6bc06e460 0x7fe6bc070920 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fe6c8006fd0 tx=0x7fe6c8008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:36:23.075 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:23.077+0000 7fe6cf400700 1 -- 172.21.15.153:0/3251068923 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe6c405c810 con 0x7fe6d8069020 2024-03-21T10:36:23.198 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:36:22 smithi174 ceph-mon[88004]: from='client.35326 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:36:23.198 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:36:22 smithi174 ceph-mon[88004]: pgmap v737: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:36:23.199 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:36:22 smithi174 ceph-mon[88004]: from='client.35330 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:36:23.199 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:36:22 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/1606304654' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:36:23.303 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:36:22 smithi153 ceph-mon[113733]: from='client.35326 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:36:23.303 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:36:22 smithi153 ceph-mon[113733]: pgmap v737: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:36:23.303 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:36:22 smithi153 ceph-mon[113733]: from='client.35330 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:36:23.303 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:36:22 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/1606304654' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:36:23.304 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:23.306+0000 7fe6dd600700 1 -- 172.21.15.153:0/3251068923 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe6d8197120 con 0x7fe6bc06e460 2024-03-21T10:36:23.306 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:23.307+0000 7fe6cf400700 1 -- 172.21.15.153:0/3251068923 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fe6d8197120 con 0x7fe6bc06e460 2024-03-21T10:36:23.306 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:36:23.306 INFO:teuthology.orchestra.run.smithi153.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232", 2024-03-21T10:36:23.306 INFO:teuthology.orchestra.run.smithi153.stdout: "in_progress": true, 2024-03-21T10:36:23.306 INFO:teuthology.orchestra.run.smithi153.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-21T10:36:23.306 INFO:teuthology.orchestra.run.smithi153.stdout: "services_complete": [ 2024-03-21T10:36:23.306 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr", 2024-03-21T10:36:23.306 INFO:teuthology.orchestra.run.smithi153.stdout: "mon", 2024-03-21T10:36:23.306 INFO:teuthology.orchestra.run.smithi153.stdout: "mds", 2024-03-21T10:36:23.306 INFO:teuthology.orchestra.run.smithi153.stdout: "crash", 2024-03-21T10:36:23.306 INFO:teuthology.orchestra.run.smithi153.stdout: "osd" 2024-03-21T10:36:23.307 INFO:teuthology.orchestra.run.smithi153.stdout: ], 2024-03-21T10:36:23.307 INFO:teuthology.orchestra.run.smithi153.stdout: "progress": "16/23 daemons upgraded", 2024-03-21T10:36:23.307 INFO:teuthology.orchestra.run.smithi153.stdout: "message": "Currently upgrading mds daemons", 2024-03-21T10:36:23.307 INFO:teuthology.orchestra.run.smithi153.stdout: "is_paused": false 2024-03-21T10:36:23.307 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:36:23.308 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:23.310+0000 7fe6dd600700 1 -- 172.21.15.153:0/3251068923 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe6bc06e460 msgr2=0x7fe6bc070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:23.308 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:23.310+0000 7fe6dd600700 1 --2- 172.21.15.153:0/3251068923 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe6bc06e460 0x7fe6bc070920 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fe6c8006fd0 tx=0x7fe6c8008040 comp rx=0 tx=0).stop 2024-03-21T10:36:23.308 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:23.310+0000 7fe6dd600700 1 -- 172.21.15.153:0/3251068923 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6d8069020 msgr2=0x7fe6d819cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:23.308 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:23.310+0000 7fe6dd600700 1 --2- 172.21.15.153:0/3251068923 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6d8069020 0x7fe6d819cb20 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fe6c400bda0 tx=0x7fe6c400be80 comp rx=0 tx=0).stop 2024-03-21T10:36:23.309 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:23.310+0000 7fe6dd600700 1 -- 172.21.15.153:0/3251068923 shutdown_connections 2024-03-21T10:36:23.309 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:23.310+0000 7fe6dd600700 1 --2- 172.21.15.153:0/3251068923 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe6d8068670 0x7fe6d819c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:23.309 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:23.310+0000 7fe6dd600700 1 --2- 172.21.15.153:0/3251068923 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe6bc06e460 0x7fe6bc070920 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:23.309 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:23.310+0000 7fe6dd600700 1 --2- 172.21.15.153:0/3251068923 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6d8069020 0x7fe6d819cb20 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:23.309 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:23.310+0000 7fe6dd600700 1 -- 172.21.15.153:0/3251068923 >> 172.21.15.153:0/3251068923 conn(0x7fe6d80755a0 msgr2=0x7fe6d80fe870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:36:23.309 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:23.311+0000 7fe6dd600700 1 -- 172.21.15.153:0/3251068923 shutdown_connections 2024-03-21T10:36:23.310 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:23.311+0000 7fe6dd600700 1 -- 172.21.15.153:0/3251068923 wait complete. 2024-03-21T10:36:23.452 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:23.453+0000 7fe219600700 1 -- 172.21.15.153:0/1817938859 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe2141071d0 msgr2=0x7fe214107650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:23.452 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:23.453+0000 7fe219600700 1 --2- 172.21.15.153:0/1817938859 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe2141071d0 0x7fe214107650 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fe200009910 tx=0x7fe200009c20 comp rx=0 tx=0).stop 2024-03-21T10:36:23.452 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:23.453+0000 7fe219600700 1 -- 172.21.15.153:0/1817938859 shutdown_connections 2024-03-21T10:36:23.452 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:23.453+0000 7fe219600700 1 --2- 172.21.15.153:0/1817938859 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe2141071d0 0x7fe214107650 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:23.452 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:23.453+0000 7fe219600700 1 --2- 172.21.15.153:0/1817938859 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe21410d280 0x7fe21410d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:23.452 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:23.453+0000 7fe219600700 1 -- 172.21.15.153:0/1817938859 >> 172.21.15.153:0/1817938859 conn(0x7fe2140769e0 msgr2=0x7fe214078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:36:23.452 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:23.453+0000 7fe219600700 1 -- 172.21.15.153:0/1817938859 shutdown_connections 2024-03-21T10:36:23.452 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:23.453+0000 7fe219600700 1 -- 172.21.15.153:0/1817938859 wait complete. 2024-03-21T10:36:23.454 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:23.454+0000 7fe219600700 1 Processor -- start 2024-03-21T10:36:23.454 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:23.455+0000 7fe219600700 1 -- start start 2024-03-21T10:36:23.454 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:23.455+0000 7fe219600700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe2141071d0 0x7fe21419ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:23.454 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:23.455+0000 7fe219600700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe21410d280 0x7fe21419d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:23.454 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:23.455+0000 7fe219600700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe21419da30 con 0x7fe2141071d0 2024-03-21T10:36:23.454 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:23.455+0000 7fe219600700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2141a17c0 con 0x7fe21410d280 2024-03-21T10:36:23.454 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:23.455+0000 7fe212a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe2141071d0 0x7fe21419ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:23.454 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:23.455+0000 7fe212a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe2141071d0 0x7fe21419ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:33474/0 (socket says 172.21.15.153:33474) 2024-03-21T10:36:23.455 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:23.455+0000 7fe212a00700 1 -- 172.21.15.153:0/785954655 learned_addr learned my addr 172.21.15.153:0/785954655 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:36:23.455 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:23.456+0000 7fe212000700 1 --2- 172.21.15.153:0/785954655 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe21410d280 0x7fe21419d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:23.455 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:23.456+0000 7fe212a00700 1 -- 172.21.15.153:0/785954655 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe21410d280 msgr2=0x7fe21419d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:23.455 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:23.456+0000 7fe212a00700 1 --2- 172.21.15.153:0/785954655 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe21410d280 0x7fe21419d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:23.455 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:23.456+0000 7fe212a00700 1 -- 172.21.15.153:0/785954655 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2000095f0 con 0x7fe2141071d0 2024-03-21T10:36:23.455 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:23.456+0000 7fe212a00700 1 --2- 172.21.15.153:0/785954655 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe2141071d0 0x7fe21419ce10 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fe20400b4b0 tx=0x7fe20400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:36:23.456 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:23.456+0000 7fe20b400700 1 -- 172.21.15.153:0/785954655 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe204011800 con 0x7fe2141071d0 2024-03-21T10:36:23.456 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:23.456+0000 7fe20b400700 1 -- 172.21.15.153:0/785954655 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe204011e20 con 0x7fe2141071d0 2024-03-21T10:36:23.456 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:23.456+0000 7fe20b400700 1 -- 172.21.15.153:0/785954655 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe20400f2e0 con 0x7fe2141071d0 2024-03-21T10:36:23.456 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:23.456+0000 7fe219600700 1 -- 172.21.15.153:0/785954655 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2141a1aa0 con 0x7fe2141071d0 2024-03-21T10:36:23.456 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:23.456+0000 7fe219600700 1 -- 172.21.15.153:0/785954655 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2141a2070 con 0x7fe2141071d0 2024-03-21T10:36:23.456 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:23.457+0000 7fe219600700 1 -- 172.21.15.153:0/785954655 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe21410f6f0 con 0x7fe2141071d0 2024-03-21T10:36:23.456 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:23.458+0000 7fe20b400700 1 -- 172.21.15.153:0/785954655 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fe204010380 con 0x7fe2141071d0 2024-03-21T10:36:23.458 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:23.459+0000 7fe20b400700 1 --2- 172.21.15.153:0/785954655 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe1f806e460 0x7fe1f8070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:23.458 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:23.459+0000 7fe20b400700 1 -- 172.21.15.153:0/785954655 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fe20408e0c0 con 0x7fe2141071d0 2024-03-21T10:36:23.458 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:23.460+0000 7fe212000700 1 --2- 172.21.15.153:0/785954655 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe1f806e460 0x7fe1f8070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:23.458 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:23.460+0000 7fe212000700 1 --2- 172.21.15.153:0/785954655 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe1f806e460 0x7fe1f8070920 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fe200006010 tx=0x7fe20000b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:36:23.462 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:23.464+0000 7fe20b400700 1 -- 172.21.15.153:0/785954655 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe204057210 con 0x7fe2141071d0 2024-03-21T10:36:23.773 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:23.774+0000 7fe219600700 1 -- 172.21.15.153:0/785954655 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fe2141a2400 con 0x7fe2141071d0 2024-03-21T10:36:23.773 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:23.775+0000 7fe20b400700 1 -- 172.21.15.153:0/785954655 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4493 (secure 0 0 0) 0x7fe20405a8d0 con 0x7fe2141071d0 2024-03-21T10:36:23.774 INFO:teuthology.orchestra.run.smithi153.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-03-21T10:36:23.774 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-03-21T10:36:23.775 INFO:teuthology.orchestra.run.smithi153.stdout: host smithi174 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi174/config 2024-03-21T10:36:23.775 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/podman: timeout after 895 seconds 2024-03-21T10:36:23.775 INFO:teuthology.orchestra.run.smithi153.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 -e NODE_NAME=smithi174 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/run/ceph:z -v /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpd910r3_i:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 inventory --format=json-pretty --filter-for-batch 2024-03-21T10:36:23.775 INFO:teuthology.orchestra.run.smithi153.stdout:Traceback (most recent call last): 2024-03-21T10:36:23.775 INFO:teuthology.orchestra.run.smithi153.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-03-21T10:36:23.775 INFO:teuthology.orchestra.run.smithi153.stdout: return _run_code(code, main_globals, None, 2024-03-21T10:36:23.775 INFO:teuthology.orchestra.run.smithi153.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-03-21T10:36:23.775 INFO:teuthology.orchestra.run.smithi153.stdout: exec(code, run_globals) 2024-03-21T10:36:23.775 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 5606, in 2024-03-21T10:36:23.775 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 5594, in main 2024-03-21T10:36:23.775 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 406, in _infer_config 2024-03-21T10:36:23.775 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 321, in _infer_fsid 2024-03-21T10:36:23.775 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 434, in _infer_image 2024-03-21T10:36:23.775 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 308, in _validate_fsid 2024-03-21T10:36:23.775 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 3357, in command_ceph_volume 2024-03-21T10:36:23.776 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-03-21T10:36:23.776 INFO:teuthology.orchestra.run.smithi153.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 -e NODE_NAME=smithi174 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/run/ceph:z -v /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpd910r3_i:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 inventory --format=json-pretty --filter-for-batch 2024-03-21T10:36:23.776 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-21T10:36:23.776 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is degraded 2024-03-21T10:36:23.776 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-21T10:36:23.776 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-21T10:36:23.776 INFO:teuthology.orchestra.run.smithi153.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-21T10:36:23.776 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-21T10:36:23.777 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:23.778+0000 7fe219600700 1 -- 172.21.15.153:0/785954655 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe1f806e460 msgr2=0x7fe1f8070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:23.777 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:23.778+0000 7fe219600700 1 --2- 172.21.15.153:0/785954655 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe1f806e460 0x7fe1f8070920 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fe200006010 tx=0x7fe20000b540 comp rx=0 tx=0).stop 2024-03-21T10:36:23.777 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:23.778+0000 7fe219600700 1 -- 172.21.15.153:0/785954655 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe2141071d0 msgr2=0x7fe21419ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:23.777 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:23.778+0000 7fe219600700 1 --2- 172.21.15.153:0/785954655 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe2141071d0 0x7fe21419ce10 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fe20400b4b0 tx=0x7fe20400b870 comp rx=0 tx=0).stop 2024-03-21T10:36:23.777 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:23.779+0000 7fe219600700 1 -- 172.21.15.153:0/785954655 shutdown_connections 2024-03-21T10:36:23.777 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:23.779+0000 7fe219600700 1 --2- 172.21.15.153:0/785954655 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe1f806e460 0x7fe1f8070920 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:23.777 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:23.779+0000 7fe219600700 1 --2- 172.21.15.153:0/785954655 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe2141071d0 0x7fe21419ce10 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:23.777 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:23.779+0000 7fe219600700 1 --2- 172.21.15.153:0/785954655 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe21410d280 0x7fe21419d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:23.777 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:23.779+0000 7fe219600700 1 -- 172.21.15.153:0/785954655 >> 172.21.15.153:0/785954655 conn(0x7fe2140769e0 msgr2=0x7fe2140784d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:36:23.777 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:23.779+0000 7fe219600700 1 -- 172.21.15.153:0/785954655 shutdown_connections 2024-03-21T10:36:23.778 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:23.779+0000 7fe219600700 1 -- 172.21.15.153:0/785954655 wait complete. 2024-03-21T10:36:24.053 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:36:23 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/3033830844' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:36:24.053 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:36:23 smithi153 ceph-mon[113733]: from='client.35342 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:36:24.053 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:36:23 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/785954655' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:36:24.198 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:36:23 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/3033830844' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:36:24.198 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:36:23 smithi174 ceph-mon[88004]: from='client.35342 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:36:24.198 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:36:23 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/785954655' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:36:25.198 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:36:24 smithi174 ceph-mon[88004]: pgmap v738: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:36:25.303 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:36:24 smithi153 ceph-mon[113733]: pgmap v738: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:36:27.198 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:36:26 smithi174 ceph-mon[88004]: pgmap v739: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:36:27.303 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:36:26 smithi153 ceph-mon[113733]: pgmap v739: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:36:29.198 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:36:28 smithi174 ceph-mon[88004]: pgmap v740: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:36:29.303 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:36:28 smithi153 ceph-mon[113733]: pgmap v740: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:36:31.198 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:36:30 smithi174 ceph-mon[88004]: pgmap v741: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:36:31.303 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:36:30 smithi153 ceph-mon[113733]: pgmap v741: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:36:32.198 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:36:31 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:36:32.304 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:36:31 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:36:33.198 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:36:32 smithi174 ceph-mon[88004]: pgmap v742: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:36:33.303 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:36:32 smithi153 ceph-mon[113733]: pgmap v742: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:36:35.198 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:36:34 smithi174 ceph-mon[88004]: pgmap v743: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:36:35.303 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:36:34 smithi153 ceph-mon[113733]: pgmap v743: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:36:37.053 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:36:36 smithi153 ceph-mon[113733]: pgmap v744: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:36:37.198 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:36:36 smithi174 ceph-mon[88004]: pgmap v744: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:36:39.198 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:36:38 smithi174 ceph-mon[88004]: pgmap v745: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:36:39.303 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:36:38 smithi153 ceph-mon[113733]: pgmap v745: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:36:41.303 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:36:40 smithi153 ceph-mon[113733]: pgmap v746: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:36:41.448 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:36:40 smithi174 ceph-mon[88004]: pgmap v746: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:36:43.303 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:36:42 smithi153 ceph-mon[113733]: pgmap v747: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:36:43.448 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:36:42 smithi174 ceph-mon[88004]: pgmap v747: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:36:45.198 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:36:44 smithi174 ceph-mon[88004]: pgmap v748: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:36:45.303 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:36:44 smithi153 ceph-mon[113733]: pgmap v748: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:36:47.303 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:36:46 smithi153 ceph-mon[113733]: pgmap v749: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:36:47.303 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:36:46 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:36:47.448 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:36:46 smithi174 ceph-mon[88004]: pgmap v749: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:36:47.448 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:36:46 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:36:49.303 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:36:48 smithi153 ceph-mon[113733]: pgmap v750: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:36:49.448 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:36:48 smithi174 ceph-mon[88004]: pgmap v750: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:36:51.303 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:36:50 smithi153 ceph-mon[113733]: pgmap v751: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:36:51.448 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:36:50 smithi174 ceph-mon[88004]: pgmap v751: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:36:53.303 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:36:52 smithi153 ceph-mon[113733]: pgmap v752: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:36:53.448 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:36:52 smithi174 ceph-mon[88004]: pgmap v752: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:36:53.948 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:53.949+0000 7f0008e00700 1 -- 172.21.15.153:0/4196920530 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0004102620 msgr2=0x7f000410ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:53.948 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:53.949+0000 7f0008e00700 1 --2- 172.21.15.153:0/4196920530 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0004102620 0x7f000410ab10 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7efff0009910 tx=0x7efff0009c20 comp rx=0 tx=0).stop 2024-03-21T10:36:53.948 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:53.949+0000 7f0008e00700 1 -- 172.21.15.153:0/4196920530 shutdown_connections 2024-03-21T10:36:53.948 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:53.949+0000 7f0008e00700 1 --2- 172.21.15.153:0/4196920530 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0004102620 0x7f000410ab10 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:53.948 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:53.949+0000 7f0008e00700 1 --2- 172.21.15.153:0/4196920530 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0004101d00 0x7f00041020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:53.948 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:53.949+0000 7f0008e00700 1 -- 172.21.15.153:0/4196920530 >> 172.21.15.153:0/4196920530 conn(0x7f00040fb570 msgr2=0x7f00040fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:36:53.948 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:53.949+0000 7f0008e00700 1 -- 172.21.15.153:0/4196920530 shutdown_connections 2024-03-21T10:36:53.948 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:53.949+0000 7f0008e00700 1 -- 172.21.15.153:0/4196920530 wait complete. 2024-03-21T10:36:53.949 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:53.951+0000 7f0008e00700 1 Processor -- start 2024-03-21T10:36:53.949 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:53.951+0000 7f0008e00700 1 -- start start 2024-03-21T10:36:53.949 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:53.951+0000 7f0008e00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0004101d00 0x7f000419c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:53.949 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:53.951+0000 7f0008e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0004102620 0x7f000419cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:53.949 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:53.951+0000 7f0008e00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f000419d150 con 0x7f0004102620 2024-03-21T10:36:53.949 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:53.951+0000 7f0008e00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0004196600 con 0x7f0004101d00 2024-03-21T10:36:53.951 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:53.951+0000 7f0001400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0004102620 0x7f000419cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:53.951 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:53.951+0000 7f0001400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0004102620 0x7f000419cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:48324/0 (socket says 172.21.15.153:48324) 2024-03-21T10:36:53.951 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:53.951+0000 7f0001400700 1 -- 172.21.15.153:0/17958045 learned_addr learned my addr 172.21.15.153:0/17958045 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:36:53.951 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:53.952+0000 7f0001e00700 1 --2- 172.21.15.153:0/17958045 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0004101d00 0x7f000419c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:53.951 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:53.952+0000 7f0001400700 1 -- 172.21.15.153:0/17958045 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0004101d00 msgr2=0x7f000419c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:53.951 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:53.952+0000 7f0001400700 1 --2- 172.21.15.153:0/17958045 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0004101d00 0x7f000419c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:53.951 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:53.952+0000 7f0001400700 1 -- 172.21.15.153:0/17958045 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efff00095f0 con 0x7f0004102620 2024-03-21T10:36:53.951 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:53.952+0000 7f0001400700 1 --2- 172.21.15.153:0/17958045 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0004102620 0x7f000419cac0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7efff000bda0 tx=0x7efff000be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:36:53.951 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:53.952+0000 7efffaa00700 1 -- 172.21.15.153:0/17958045 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efff001d070 con 0x7f0004102620 2024-03-21T10:36:53.951 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:53.953+0000 7efffaa00700 1 -- 172.21.15.153:0/17958045 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efff000f460 con 0x7f0004102620 2024-03-21T10:36:53.952 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:53.953+0000 7f0008e00700 1 -- 172.21.15.153:0/17958045 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0004196880 con 0x7f0004102620 2024-03-21T10:36:53.952 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:53.953+0000 7efffaa00700 1 -- 172.21.15.153:0/17958045 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efff0017610 con 0x7f0004102620 2024-03-21T10:36:53.952 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:53.953+0000 7f0008e00700 1 -- 172.21.15.153:0/17958045 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0004196d70 con 0x7f0004102620 2024-03-21T10:36:53.952 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:53.954+0000 7f0008e00700 1 -- 172.21.15.153:0/17958045 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0004108210 con 0x7f0004102620 2024-03-21T10:36:53.953 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:53.955+0000 7efffaa00700 1 -- 172.21.15.153:0/17958045 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7efff000fa40 con 0x7f0004102620 2024-03-21T10:36:53.954 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:53.956+0000 7efffaa00700 1 --2- 172.21.15.153:0/17958045 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7effe806e460 0x7effe8070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:53.954 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:53.956+0000 7efffaa00700 1 -- 172.21.15.153:0/17958045 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7efff008fb00 con 0x7f0004102620 2024-03-21T10:36:53.954 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:53.956+0000 7f0001e00700 1 --2- 172.21.15.153:0/17958045 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7effe806e460 0x7effe8070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:53.955 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:53.957+0000 7f0001e00700 1 --2- 172.21.15.153:0/17958045 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7effe806e460 0x7effe8070920 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7efff4005950 tx=0x7efff40058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:36:53.959 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:53.961+0000 7efffaa00700 1 -- 172.21.15.153:0/17958045 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7efff005c650 con 0x7f0004102620 2024-03-21T10:36:54.202 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:54.204+0000 7f0008e00700 1 -- 172.21.15.153:0/17958045 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0004197050 con 0x7effe806e460 2024-03-21T10:36:54.205 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:54.206+0000 7efffaa00700 1 -- 172.21.15.153:0/17958045 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f0004197050 con 0x7effe806e460 2024-03-21T10:36:54.208 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:54.209+0000 7f0008e00700 1 -- 172.21.15.153:0/17958045 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7effe806e460 msgr2=0x7effe8070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:54.208 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:54.209+0000 7f0008e00700 1 --2- 172.21.15.153:0/17958045 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7effe806e460 0x7effe8070920 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7efff4005950 tx=0x7efff40058e0 comp rx=0 tx=0).stop 2024-03-21T10:36:54.208 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:54.209+0000 7f0008e00700 1 -- 172.21.15.153:0/17958045 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0004102620 msgr2=0x7f000419cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:54.208 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:54.209+0000 7f0008e00700 1 --2- 172.21.15.153:0/17958045 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0004102620 0x7f000419cac0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7efff000bda0 tx=0x7efff000be80 comp rx=0 tx=0).stop 2024-03-21T10:36:54.208 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:54.209+0000 7f0008e00700 1 -- 172.21.15.153:0/17958045 shutdown_connections 2024-03-21T10:36:54.208 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:54.209+0000 7f0008e00700 1 --2- 172.21.15.153:0/17958045 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0004101d00 0x7f000419c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:54.208 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:54.209+0000 7f0008e00700 1 --2- 172.21.15.153:0/17958045 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7effe806e460 0x7effe8070920 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:54.208 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:54.209+0000 7f0008e00700 1 --2- 172.21.15.153:0/17958045 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0004102620 0x7f000419cac0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:54.208 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:54.209+0000 7f0008e00700 1 -- 172.21.15.153:0/17958045 >> 172.21.15.153:0/17958045 conn(0x7f00040fb570 msgr2=0x7f0004105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:36:54.209 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:54.210+0000 7f0008e00700 1 -- 172.21.15.153:0/17958045 shutdown_connections 2024-03-21T10:36:54.209 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:54.210+0000 7f0008e00700 1 -- 172.21.15.153:0/17958045 wait complete. 2024-03-21T10:36:54.224 INFO:teuthology.orchestra.run.smithi153.stdout:true 2024-03-21T10:36:54.368 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:54.369+0000 7f5e65a00700 1 -- 172.21.15.153:0/3709223191 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5e60101be0 msgr2=0x7f5e60101fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:54.368 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:54.369+0000 7f5e65a00700 1 --2- 172.21.15.153:0/3709223191 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5e60101be0 0x7f5e60101fc0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f5e50009940 tx=0x7f5e50009c50 comp rx=0 tx=0).stop 2024-03-21T10:36:54.368 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:54.369+0000 7f5e65a00700 1 -- 172.21.15.153:0/3709223191 shutdown_connections 2024-03-21T10:36:54.368 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:54.369+0000 7f5e65a00700 1 --2- 172.21.15.153:0/3709223191 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5e60102500 0x7f5e6010a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:54.368 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:54.369+0000 7f5e65a00700 1 --2- 172.21.15.153:0/3709223191 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5e60101be0 0x7f5e60101fc0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:54.368 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:54.369+0000 7f5e65a00700 1 -- 172.21.15.153:0/3709223191 >> 172.21.15.153:0/3709223191 conn(0x7f5e600fb390 msgr2=0x7f5e600fd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:36:54.368 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:54.369+0000 7f5e65a00700 1 -- 172.21.15.153:0/3709223191 shutdown_connections 2024-03-21T10:36:54.368 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:54.369+0000 7f5e65a00700 1 -- 172.21.15.153:0/3709223191 wait complete. 2024-03-21T10:36:54.369 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:54.370+0000 7f5e65a00700 1 Processor -- start 2024-03-21T10:36:54.369 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:54.371+0000 7f5e65a00700 1 -- start start 2024-03-21T10:36:54.369 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:54.371+0000 7f5e65a00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5e60101be0 0x7f5e60196560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:54.369 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:54.371+0000 7f5e65a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5e60102500 0x7f5e60196aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:54.369 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:54.371+0000 7f5e65a00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e60197180 con 0x7f5e60102500 2024-03-21T10:36:54.369 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:54.371+0000 7f5e65a00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e6019af10 con 0x7f5e60101be0 2024-03-21T10:36:54.371 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:54.371+0000 7f5e5f400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5e60102500 0x7f5e60196aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:54.371 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:54.371+0000 7f5e5f400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5e60102500 0x7f5e60196aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:48334/0 (socket says 172.21.15.153:48334) 2024-03-21T10:36:54.371 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:54.371+0000 7f5e5f400700 1 -- 172.21.15.153:0/2661073941 learned_addr learned my addr 172.21.15.153:0/2661073941 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:36:54.371 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:54.372+0000 7f5e5fe00700 1 --2- 172.21.15.153:0/2661073941 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5e60101be0 0x7f5e60196560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:54.371 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:54.372+0000 7f5e5f400700 1 -- 172.21.15.153:0/2661073941 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5e60101be0 msgr2=0x7f5e60196560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:54.372 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:54.372+0000 7f5e5f400700 1 --2- 172.21.15.153:0/2661073941 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5e60101be0 0x7f5e60196560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:54.372 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:54.372+0000 7f5e5f400700 1 -- 172.21.15.153:0/2661073941 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e500095f0 con 0x7f5e60102500 2024-03-21T10:36:54.372 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:54.372+0000 7f5e5f400700 1 --2- 172.21.15.153:0/2661073941 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5e60102500 0x7f5e60196aa0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c00d900 tx=0x7f5e4c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:36:54.372 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:54.372+0000 7f5e5cc00700 1 -- 172.21.15.153:0/2661073941 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e4c009900 con 0x7f5e60102500 2024-03-21T10:36:54.372 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:54.373+0000 7f5e5cc00700 1 -- 172.21.15.153:0/2661073941 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5e4c010460 con 0x7f5e60102500 2024-03-21T10:36:54.372 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:54.373+0000 7f5e65a00700 1 -- 172.21.15.153:0/2661073941 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e6019b1f0 con 0x7f5e60102500 2024-03-21T10:36:54.372 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:54.373+0000 7f5e5cc00700 1 -- 172.21.15.153:0/2661073941 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e4c00f5f0 con 0x7f5e60102500 2024-03-21T10:36:54.372 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:54.373+0000 7f5e65a00700 1 -- 172.21.15.153:0/2661073941 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e6019b740 con 0x7f5e60102500 2024-03-21T10:36:54.372 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:54.374+0000 7f5e65a00700 1 -- 172.21.15.153:0/2661073941 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e601080f0 con 0x7f5e60102500 2024-03-21T10:36:54.373 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:54.375+0000 7f5e5cc00700 1 -- 172.21.15.153:0/2661073941 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f5e4c0105d0 con 0x7f5e60102500 2024-03-21T10:36:54.374 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:54.376+0000 7f5e5cc00700 1 --2- 172.21.15.153:0/2661073941 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f5e4406e460 0x7f5e44070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:54.374 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:54.376+0000 7f5e5cc00700 1 -- 172.21.15.153:0/2661073941 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f5e4c08e2c0 con 0x7f5e60102500 2024-03-21T10:36:54.374 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:54.376+0000 7f5e5fe00700 1 --2- 172.21.15.153:0/2661073941 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f5e4406e460 0x7f5e44070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:54.375 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:54.377+0000 7f5e5fe00700 1 --2- 172.21.15.153:0/2661073941 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f5e4406e460 0x7f5e44070920 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f5e50005b40 tx=0x7f5e50005ab0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:36:54.379 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:54.381+0000 7f5e5cc00700 1 -- 172.21.15.153:0/2661073941 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f5e4c057400 con 0x7f5e60102500 2024-03-21T10:36:54.622 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:54.623+0000 7f5e65a00700 1 -- 172.21.15.153:0/2661073941 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5e6019baa0 con 0x7f5e4406e460 2024-03-21T10:36:54.624 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:54.626+0000 7f5e5cc00700 1 -- 172.21.15.153:0/2661073941 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f5e6019baa0 con 0x7f5e4406e460 2024-03-21T10:36:54.627 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:54.628+0000 7f5e65a00700 1 -- 172.21.15.153:0/2661073941 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f5e4406e460 msgr2=0x7f5e44070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:54.627 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:54.628+0000 7f5e65a00700 1 --2- 172.21.15.153:0/2661073941 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f5e4406e460 0x7f5e44070920 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f5e50005b40 tx=0x7f5e50005ab0 comp rx=0 tx=0).stop 2024-03-21T10:36:54.627 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:54.628+0000 7f5e65a00700 1 -- 172.21.15.153:0/2661073941 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5e60102500 msgr2=0x7f5e60196aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:54.627 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:54.628+0000 7f5e65a00700 1 --2- 172.21.15.153:0/2661073941 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5e60102500 0x7f5e60196aa0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c00d900 tx=0x7f5e4c00dcc0 comp rx=0 tx=0).stop 2024-03-21T10:36:54.628 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:54.629+0000 7f5e65a00700 1 -- 172.21.15.153:0/2661073941 shutdown_connections 2024-03-21T10:36:54.628 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:54.629+0000 7f5e65a00700 1 --2- 172.21.15.153:0/2661073941 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5e60101be0 0x7f5e60196560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:54.628 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:54.629+0000 7f5e65a00700 1 --2- 172.21.15.153:0/2661073941 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f5e4406e460 0x7f5e44070920 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:54.628 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:54.629+0000 7f5e65a00700 1 --2- 172.21.15.153:0/2661073941 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5e60102500 0x7f5e60196aa0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:54.628 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:54.629+0000 7f5e65a00700 1 -- 172.21.15.153:0/2661073941 >> 172.21.15.153:0/2661073941 conn(0x7f5e600fb390 msgr2=0x7f5e60105230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:36:54.628 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:54.629+0000 7f5e65a00700 1 -- 172.21.15.153:0/2661073941 shutdown_connections 2024-03-21T10:36:54.628 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:54.629+0000 7f5e65a00700 1 -- 172.21.15.153:0/2661073941 wait complete. 2024-03-21T10:36:54.776 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:54.777+0000 7fbd2ae00700 1 -- 172.21.15.153:0/2918190635 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbd24106a40 msgr2=0x7fbd24106e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:54.776 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:54.777+0000 7fbd2ae00700 1 --2- 172.21.15.153:0/2918190635 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbd24106a40 0x7fbd24106e20 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fbd18009910 tx=0x7fbd18009c20 comp rx=0 tx=0).stop 2024-03-21T10:36:54.776 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:54.777+0000 7fbd2ae00700 1 -- 172.21.15.153:0/2918190635 shutdown_connections 2024-03-21T10:36:54.776 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:54.777+0000 7fbd2ae00700 1 --2- 172.21.15.153:0/2918190635 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbd24068410 0x7fbd24068890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:54.776 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:54.777+0000 7fbd2ae00700 1 --2- 172.21.15.153:0/2918190635 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbd24106a40 0x7fbd24106e20 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:54.776 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:54.777+0000 7fbd2ae00700 1 -- 172.21.15.153:0/2918190635 >> 172.21.15.153:0/2918190635 conn(0x7fbd24075330 msgr2=0x7fbd24075740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:36:54.776 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:54.777+0000 7fbd2ae00700 1 -- 172.21.15.153:0/2918190635 shutdown_connections 2024-03-21T10:36:54.777 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:54.778+0000 7fbd2ae00700 1 -- 172.21.15.153:0/2918190635 wait complete. 2024-03-21T10:36:54.778 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:54.778+0000 7fbd2ae00700 1 Processor -- start 2024-03-21T10:36:54.778 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:54.779+0000 7fbd2ae00700 1 -- start start 2024-03-21T10:36:54.778 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:54.779+0000 7fbd2ae00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbd24068410 0x7fbd24196590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:54.778 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:54.779+0000 7fbd2ae00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbd24106a40 0x7fbd24196ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:54.778 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:54.779+0000 7fbd2ae00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd241971b0 con 0x7fbd24068410 2024-03-21T10:36:54.778 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:54.779+0000 7fbd2ae00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd2419af40 con 0x7fbd24106a40 2024-03-21T10:36:54.779 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:54.779+0000 7fbd29a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbd24068410 0x7fbd24196590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:54.779 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:54.779+0000 7fbd29a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbd24068410 0x7fbd24196590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:48346/0 (socket says 172.21.15.153:48346) 2024-03-21T10:36:54.779 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:54.779+0000 7fbd29a00700 1 -- 172.21.15.153:0/1399351409 learned_addr learned my addr 172.21.15.153:0/1399351409 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:36:54.779 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:54.780+0000 7fbd29000700 1 --2- 172.21.15.153:0/1399351409 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbd24106a40 0x7fbd24196ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:54.779 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:54.780+0000 7fbd29a00700 1 -- 172.21.15.153:0/1399351409 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbd24106a40 msgr2=0x7fbd24196ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:54.779 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:54.780+0000 7fbd29a00700 1 --2- 172.21.15.153:0/1399351409 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbd24106a40 0x7fbd24196ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:54.779 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:54.780+0000 7fbd29a00700 1 -- 172.21.15.153:0/1399351409 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd180095f0 con 0x7fbd24068410 2024-03-21T10:36:54.779 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:54.780+0000 7fbd29000700 1 --2- 172.21.15.153:0/1399351409 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbd24106a40 0x7fbd24196ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:36:54.779 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:54.780+0000 7fbd29a00700 1 --2- 172.21.15.153:0/1399351409 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbd24068410 0x7fbd24196590 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fbd1800bd70 tx=0x7fbd1800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:36:54.780 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:54.780+0000 7fbd22000700 1 -- 172.21.15.153:0/1399351409 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbd1801d070 con 0x7fbd24068410 2024-03-21T10:36:54.780 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:54.780+0000 7fbd22000700 1 -- 172.21.15.153:0/1399351409 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbd1800f460 con 0x7fbd24068410 2024-03-21T10:36:54.780 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:54.781+0000 7fbd22000700 1 -- 172.21.15.153:0/1399351409 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbd18017630 con 0x7fbd24068410 2024-03-21T10:36:54.780 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:54.781+0000 7fbd2ae00700 1 -- 172.21.15.153:0/1399351409 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd2419b1c0 con 0x7fbd24068410 2024-03-21T10:36:54.780 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:54.781+0000 7fbd2ae00700 1 -- 172.21.15.153:0/1399351409 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd2419b6b0 con 0x7fbd24068410 2024-03-21T10:36:54.780 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:54.782+0000 7fbd2ae00700 1 -- 172.21.15.153:0/1399351409 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd24103ce0 con 0x7fbd24068410 2024-03-21T10:36:54.781 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:54.783+0000 7fbd22000700 1 -- 172.21.15.153:0/1399351409 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fbd18003a60 con 0x7fbd24068410 2024-03-21T10:36:54.782 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:54.783+0000 7fbd22000700 1 --2- 172.21.15.153:0/1399351409 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fbd0c06e460 0x7fbd0c070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:54.782 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:54.784+0000 7fbd22000700 1 -- 172.21.15.153:0/1399351409 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fbd1808f9b0 con 0x7fbd24068410 2024-03-21T10:36:54.782 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:54.784+0000 7fbd29000700 1 --2- 172.21.15.153:0/1399351409 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fbd0c06e460 0x7fbd0c070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:54.782 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:54.784+0000 7fbd29000700 1 --2- 172.21.15.153:0/1399351409 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fbd0c06e460 0x7fbd0c070920 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fbd10005950 tx=0x7fbd100058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:36:54.786 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:54.788+0000 7fbd22000700 1 -- 172.21.15.153:0/1399351409 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fbd18058b10 con 0x7fbd24068410 2024-03-21T10:36:55.015 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:55.016+0000 7fbd2ae00700 1 -- 172.21.15.153:0/1399351409 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fbd2402cd80 con 0x7fbd0c06e460 2024-03-21T10:36:55.046 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:55.047+0000 7fbd22000700 1 -- 172.21.15.153:0/1399351409 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fbd2402cd80 con 0x7fbd0c06e460 2024-03-21T10:36:55.046 INFO:teuthology.orchestra.run.smithi153.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-21T10:36:55.047 INFO:teuthology.orchestra.run.smithi153.stdout:alertmanager.smithi153 smithi153 *:9093,9094 running (25m) 87s ago 39m 15.6M - 0.25.0 c8568f914cd2 9ed3f2200b5f 2024-03-21T10:36:55.047 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi153 smithi153 running (39m) 87s ago 39m 10.4M - 18.2.1 5be31c24972a 5b61994396a6 2024-03-21T10:36:55.047 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi174 smithi174 running (38m) 90s ago 38m 10.1M - 18.2.1 5be31c24972a 4f7a5fc3ac28 2024-03-21T10:36:55.047 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi153 smithi153 running (21m) 87s ago 39m 6656k - 19.0.0-2231-g2edfee68 237e43c7a65f 32c21104d5b0 2024-03-21T10:36:55.047 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi174 smithi174 running (21m) 90s ago 38m 6660k - 19.0.0-2231-g2edfee68 237e43c7a65f a04e48967ab6 2024-03-21T10:36:55.047 INFO:teuthology.orchestra.run.smithi153.stdout:grafana.smithi153 smithi153 *:3000 running (25m) 87s ago 38m 75.2M - 9.4.12 2bacad6d85d8 ee24026ab786 2024-03-21T10:36:55.047 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.iuyluj smithi153 running (17m) 87s ago 32m 18.7M - 19.0.0-2231-g2edfee68 237e43c7a65f 87aff8a5de70 2024-03-21T10:36:55.047 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.zipckr smithi153 running (17m) 87s ago 32m 18.9M - 19.0.0-2231-g2edfee68 237e43c7a65f ccc262bd5dd3 2024-03-21T10:36:55.047 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.nhozag smithi174 running (96s) 90s ago 32m 10.9M - 19.0.0-2231-g2edfee68 237e43c7a65f 9e02aea72c96 2024-03-21T10:36:55.047 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.yyimst smithi174 running (16m) 90s ago 32m 18.5M - 19.0.0-2231-g2edfee68 237e43c7a65f fc1962f9351e 2024-03-21T10:36:55.047 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi153.tluxni smithi153 *:8443,9283,8765 running (28m) 87s ago 41m 512M - 19.0.0-2231-g2edfee68 237e43c7a65f 7ffee0e55382 2024-03-21T10:36:55.047 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi174.qpcwat smithi174 *:8443,9283,8765 running (27m) 90s ago 37m 459M - 19.0.0-2231-g2edfee68 237e43c7a65f 748968f216ab 2024-03-21T10:36:55.047 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi153 smithi153 running (23m) 87s ago 41m 70.8M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f dd8038303dc3 2024-03-21T10:36:55.048 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi174 smithi174 running (22m) 90s ago 37m 51.9M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f 29d78a788f2c 2024-03-21T10:36:55.048 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi153 smithi153 *:9100 running (26m) 87s ago 39m 8140k - 1.5.0 0da6a335fe13 bb835edc0e0b 2024-03-21T10:36:55.048 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi174 smithi174 *:9100 running (26m) 90s ago 38m 8610k - 1.5.0 0da6a335fe13 7c4ecad790fa 2024-03-21T10:36:55.048 INFO:teuthology.orchestra.run.smithi153.stdout:osd.0 smithi153 running (21m) 87s ago 36m 118M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 507900fd4791 2024-03-21T10:36:55.048 INFO:teuthology.orchestra.run.smithi153.stdout:osd.1 smithi153 running (20m) 87s ago 36m 93.0M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 0d646452e9c1 2024-03-21T10:36:55.048 INFO:teuthology.orchestra.run.smithi153.stdout:osd.2 smithi153 running (20m) 87s ago 35m 91.2M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 1326f2afdfcd 2024-03-21T10:36:55.048 INFO:teuthology.orchestra.run.smithi153.stdout:osd.3 smithi174 running (19m) 90s ago 35m 87.5M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 70aafac71ad5 2024-03-21T10:36:55.048 INFO:teuthology.orchestra.run.smithi153.stdout:osd.4 smithi174 running (18m) 90s ago 34m 95.8M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 0d746ce6e79d 2024-03-21T10:36:55.048 INFO:teuthology.orchestra.run.smithi153.stdout:osd.5 smithi174 running (18m) 90s ago 34m 90.3M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 53e5f35b8cac 2024-03-21T10:36:55.048 INFO:teuthology.orchestra.run.smithi153.stdout:prometheus.smithi153 smithi153 *:9095 running (26m) 87s ago 38m 88.1M - 2.43.0 a07b618ecd1d 7d95561e50c2 2024-03-21T10:36:55.049 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:55.051+0000 7fbd2ae00700 1 -- 172.21.15.153:0/1399351409 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fbd0c06e460 msgr2=0x7fbd0c070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:55.049 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:55.051+0000 7fbd2ae00700 1 --2- 172.21.15.153:0/1399351409 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fbd0c06e460 0x7fbd0c070920 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fbd10005950 tx=0x7fbd100058e0 comp rx=0 tx=0).stop 2024-03-21T10:36:55.050 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:55.051+0000 7fbd2ae00700 1 -- 172.21.15.153:0/1399351409 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbd24068410 msgr2=0x7fbd24196590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:55.050 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:55.051+0000 7fbd2ae00700 1 --2- 172.21.15.153:0/1399351409 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbd24068410 0x7fbd24196590 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fbd1800bd70 tx=0x7fbd1800be50 comp rx=0 tx=0).stop 2024-03-21T10:36:55.050 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:55.051+0000 7fbd2ae00700 1 -- 172.21.15.153:0/1399351409 shutdown_connections 2024-03-21T10:36:55.050 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:55.051+0000 7fbd2ae00700 1 --2- 172.21.15.153:0/1399351409 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fbd0c06e460 0x7fbd0c070920 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:55.050 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:55.051+0000 7fbd2ae00700 1 --2- 172.21.15.153:0/1399351409 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbd24068410 0x7fbd24196590 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:55.050 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:55.051+0000 7fbd2ae00700 1 --2- 172.21.15.153:0/1399351409 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbd24106a40 0x7fbd24196ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:55.050 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:55.051+0000 7fbd2ae00700 1 -- 172.21.15.153:0/1399351409 >> 172.21.15.153:0/1399351409 conn(0x7fbd24075330 msgr2=0x7fbd240fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:36:55.050 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:55.051+0000 7fbd2ae00700 1 -- 172.21.15.153:0/1399351409 shutdown_connections 2024-03-21T10:36:55.050 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:55.051+0000 7fbd2ae00700 1 -- 172.21.15.153:0/1399351409 wait complete. 2024-03-21T10:36:55.206 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:55.207+0000 7fcffec00700 1 -- 172.21.15.153:0/689470078 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcff8073130 msgr2=0x7fcff8073510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:55.206 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:55.207+0000 7fcffec00700 1 --2- 172.21.15.153:0/689470078 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcff8073130 0x7fcff8073510 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fcfe800b0a0 tx=0x7fcfe800b3b0 comp rx=0 tx=0).stop 2024-03-21T10:36:55.206 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:55.207+0000 7fcffec00700 1 -- 172.21.15.153:0/689470078 shutdown_connections 2024-03-21T10:36:55.206 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:55.207+0000 7fcffec00700 1 --2- 172.21.15.153:0/689470078 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcff8073a50 0x7fcff8111610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:55.206 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:55.207+0000 7fcffec00700 1 --2- 172.21.15.153:0/689470078 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcff8073130 0x7fcff8073510 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:55.206 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:55.207+0000 7fcffec00700 1 -- 172.21.15.153:0/689470078 >> 172.21.15.153:0/689470078 conn(0x7fcff80fc630 msgr2=0x7fcff80fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:36:55.206 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:55.207+0000 7fcffec00700 1 -- 172.21.15.153:0/689470078 shutdown_connections 2024-03-21T10:36:55.206 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:55.207+0000 7fcffec00700 1 -- 172.21.15.153:0/689470078 wait complete. 2024-03-21T10:36:55.208 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:55.208+0000 7fcffec00700 1 Processor -- start 2024-03-21T10:36:55.208 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:55.208+0000 7fcffec00700 1 -- start start 2024-03-21T10:36:55.208 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:55.209+0000 7fcffec00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcff8073130 0x7fcff81a2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:55.208 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:55.209+0000 7fcffec00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcff8073a50 0x7fcff81a3000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:55.208 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:55.209+0000 7fcffec00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcff81a3690 con 0x7fcff8073130 2024-03-21T10:36:55.209 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:55.209+0000 7fcffec00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcff819cb40 con 0x7fcff8073a50 2024-03-21T10:36:55.209 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:55.209+0000 7fcff7e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcff8073130 0x7fcff81a2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:55.209 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:55.209+0000 7fcff7e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcff8073130 0x7fcff81a2ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:48368/0 (socket says 172.21.15.153:48368) 2024-03-21T10:36:55.209 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:55.209+0000 7fcff7e00700 1 -- 172.21.15.153:0/256336849 learned_addr learned my addr 172.21.15.153:0/256336849 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:36:55.209 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:55.209+0000 7fcfef400700 1 --2- 172.21.15.153:0/256336849 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcff8073a50 0x7fcff81a3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:55.209 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:55.209+0000 7fcff7e00700 1 -- 172.21.15.153:0/256336849 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcff8073a50 msgr2=0x7fcff81a3000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:55.209 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:55.209+0000 7fcff7e00700 1 --2- 172.21.15.153:0/256336849 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcff8073a50 0x7fcff81a3000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:55.209 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:55.209+0000 7fcff7e00700 1 -- 172.21.15.153:0/256336849 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfe8009d00 con 0x7fcff8073130 2024-03-21T10:36:55.209 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:55.210+0000 7fcff7e00700 1 --2- 172.21.15.153:0/256336849 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcff8073130 0x7fcff81a2ac0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fcfe800b070 tx=0x7fcfe8009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:36:55.209 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:55.210+0000 7fcff5600700 1 -- 172.21.15.153:0/256336849 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcfe800e040 con 0x7fcff8073130 2024-03-21T10:36:55.209 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:55.210+0000 7fcff5600700 1 -- 172.21.15.153:0/256336849 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcfe8007c00 con 0x7fcff8073130 2024-03-21T10:36:55.209 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:55.210+0000 7fcff5600700 1 -- 172.21.15.153:0/256336849 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcfe801d670 con 0x7fcff8073130 2024-03-21T10:36:55.209 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:55.210+0000 7fcffec00700 1 -- 172.21.15.153:0/256336849 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcff819cdc0 con 0x7fcff8073130 2024-03-21T10:36:55.210 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:55.210+0000 7fcffec00700 1 -- 172.21.15.153:0/256336849 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcff819d310 con 0x7fcff8073130 2024-03-21T10:36:55.210 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:55.211+0000 7fcffec00700 1 -- 172.21.15.153:0/256336849 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcff810ed90 con 0x7fcff8073130 2024-03-21T10:36:55.210 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:55.212+0000 7fcff5600700 1 -- 172.21.15.153:0/256336849 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fcfe8004600 con 0x7fcff8073130 2024-03-21T10:36:55.211 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:55.213+0000 7fcff5600700 1 --2- 172.21.15.153:0/256336849 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fcfdc06e460 0x7fcfdc070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:55.211 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:55.213+0000 7fcff5600700 1 -- 172.21.15.153:0/256336849 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fcfe808fd10 con 0x7fcff8073130 2024-03-21T10:36:55.212 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:55.213+0000 7fcfef400700 1 --2- 172.21.15.153:0/256336849 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fcfdc06e460 0x7fcfdc070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:55.212 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:55.214+0000 7fcfef400700 1 --2- 172.21.15.153:0/256336849 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fcfdc06e460 0x7fcfdc070920 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fcfe0005950 tx=0x7fcfe00058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:36:55.216 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:55.218+0000 7fcff5600700 1 -- 172.21.15.153:0/256336849 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fcfe805c860 con 0x7fcff8073130 2024-03-21T10:36:55.303 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:36:54 smithi153 ceph-mon[113733]: pgmap v753: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:36:55.303 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:36:54 smithi153 ceph-mon[113733]: from='client.35350 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:36:55.448 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:36:54 smithi174 ceph-mon[88004]: pgmap v753: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:36:55.448 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:36:54 smithi174 ceph-mon[88004]: from='client.35350 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:36:55.511 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:55.513+0000 7fcffec00700 1 -- 172.21.15.153:0/256336849 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fcff804ea90 con 0x7fcff8073130 2024-03-21T10:36:55.511 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:55.513+0000 7fcff5600700 1 -- 172.21.15.153:0/256336849 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fcfe8007d70 con 0x7fcff8073130 2024-03-21T10:36:55.512 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:36:55.512 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": { 2024-03-21T10:36:55.512 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:36:55.512 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:36:55.512 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": { 2024-03-21T10:36:55.512 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:36:55.512 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:36:55.513 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": { 2024-03-21T10:36:55.513 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 6 2024-03-21T10:36:55.513 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:36:55.513 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": { 2024-03-21T10:36:55.513 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 4 2024-03-21T10:36:55.513 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:36:55.513 INFO:teuthology.orchestra.run.smithi153.stdout: "overall": { 2024-03-21T10:36:55.513 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 14 2024-03-21T10:36:55.513 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-03-21T10:36:55.513 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:36:55.515 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:55.516+0000 7fcffec00700 1 -- 172.21.15.153:0/256336849 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fcfdc06e460 msgr2=0x7fcfdc070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:55.515 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:55.516+0000 7fcffec00700 1 --2- 172.21.15.153:0/256336849 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fcfdc06e460 0x7fcfdc070920 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fcfe0005950 tx=0x7fcfe00058e0 comp rx=0 tx=0).stop 2024-03-21T10:36:55.515 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:55.516+0000 7fcffec00700 1 -- 172.21.15.153:0/256336849 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcff8073130 msgr2=0x7fcff81a2ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:55.515 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:55.516+0000 7fcffec00700 1 --2- 172.21.15.153:0/256336849 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcff8073130 0x7fcff81a2ac0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fcfe800b070 tx=0x7fcfe8009810 comp rx=0 tx=0).stop 2024-03-21T10:36:55.515 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:55.516+0000 7fcffec00700 1 -- 172.21.15.153:0/256336849 shutdown_connections 2024-03-21T10:36:55.515 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:55.516+0000 7fcffec00700 1 --2- 172.21.15.153:0/256336849 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fcfdc06e460 0x7fcfdc070920 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:55.515 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:55.516+0000 7fcffec00700 1 --2- 172.21.15.153:0/256336849 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcff8073130 0x7fcff81a2ac0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:55.515 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:55.516+0000 7fcffec00700 1 --2- 172.21.15.153:0/256336849 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcff8073a50 0x7fcff81a3000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:55.515 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:55.516+0000 7fcffec00700 1 -- 172.21.15.153:0/256336849 >> 172.21.15.153:0/256336849 conn(0x7fcff80fc630 msgr2=0x7fcff8103120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:36:55.515 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:55.517+0000 7fcffec00700 1 -- 172.21.15.153:0/256336849 shutdown_connections 2024-03-21T10:36:55.515 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:55.517+0000 7fcffec00700 1 -- 172.21.15.153:0/256336849 wait complete. 2024-03-21T10:36:55.666 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:55.667+0000 7fd34b400700 1 -- 172.21.15.153:0/253165063 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd34410d280 msgr2=0x7fd34410d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:55.666 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:55.667+0000 7fd34b400700 1 --2- 172.21.15.153:0/253165063 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd34410d280 0x7fd34410d660 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fd338009910 tx=0x7fd338009c20 comp rx=0 tx=0).stop 2024-03-21T10:36:55.667 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:55.668+0000 7fd34b400700 1 -- 172.21.15.153:0/253165063 shutdown_connections 2024-03-21T10:36:55.667 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:55.668+0000 7fd34b400700 1 --2- 172.21.15.153:0/253165063 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd3441071d0 0x7fd344107650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:55.667 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:55.668+0000 7fd34b400700 1 --2- 172.21.15.153:0/253165063 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd34410d280 0x7fd34410d660 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:55.667 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:55.668+0000 7fd34b400700 1 -- 172.21.15.153:0/253165063 >> 172.21.15.153:0/253165063 conn(0x7fd3440769e0 msgr2=0x7fd344078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:36:55.667 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:55.668+0000 7fd34b400700 1 -- 172.21.15.153:0/253165063 shutdown_connections 2024-03-21T10:36:55.667 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:55.668+0000 7fd34b400700 1 -- 172.21.15.153:0/253165063 wait complete. 2024-03-21T10:36:55.668 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:55.669+0000 7fd34b400700 1 Processor -- start 2024-03-21T10:36:55.668 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:55.669+0000 7fd34b400700 1 -- start start 2024-03-21T10:36:55.668 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:55.669+0000 7fd34b400700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd3441071d0 0x7fd34419ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:55.668 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:55.669+0000 7fd34b400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd34410d280 0x7fd34419d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:55.668 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:55.669+0000 7fd34b400700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd34419da30 con 0x7fd34410d280 2024-03-21T10:36:55.669 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:55.669+0000 7fd34b400700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3441a1770 con 0x7fd3441071d0 2024-03-21T10:36:55.669 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:55.670+0000 7fd343e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd34410d280 0x7fd34419d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:55.669 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:55.670+0000 7fd343e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd34410d280 0x7fd34419d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:48386/0 (socket says 172.21.15.153:48386) 2024-03-21T10:36:55.669 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:55.670+0000 7fd343e00700 1 -- 172.21.15.153:0/3141787811 learned_addr learned my addr 172.21.15.153:0/3141787811 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:36:55.669 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:55.670+0000 7fd348a00700 1 --2- 172.21.15.153:0/3141787811 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd3441071d0 0x7fd34419ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:55.669 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:55.670+0000 7fd343e00700 1 -- 172.21.15.153:0/3141787811 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd3441071d0 msgr2=0x7fd34419ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:55.669 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:55.670+0000 7fd343e00700 1 --2- 172.21.15.153:0/3141787811 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd3441071d0 0x7fd34419ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:55.669 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:55.670+0000 7fd343e00700 1 -- 172.21.15.153:0/3141787811 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3380095f0 con 0x7fd34410d280 2024-03-21T10:36:55.669 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:55.670+0000 7fd348a00700 1 --2- 172.21.15.153:0/3141787811 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd3441071d0 0x7fd34419ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:36:55.670 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:55.670+0000 7fd343e00700 1 --2- 172.21.15.153:0/3141787811 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd34410d280 0x7fd34419d350 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fd33000b4b0 tx=0x7fd33000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:36:55.670 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:55.671+0000 7fd341600700 1 -- 172.21.15.153:0/3141787811 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd330011800 con 0x7fd34410d280 2024-03-21T10:36:55.670 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:55.671+0000 7fd341600700 1 -- 172.21.15.153:0/3141787811 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd330011e20 con 0x7fd34410d280 2024-03-21T10:36:55.670 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:55.671+0000 7fd34b400700 1 -- 172.21.15.153:0/3141787811 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3441a1a50 con 0x7fd34410d280 2024-03-21T10:36:55.670 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:55.671+0000 7fd341600700 1 -- 172.21.15.153:0/3141787811 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd33000f2e0 con 0x7fd34410d280 2024-03-21T10:36:55.670 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:55.671+0000 7fd34b400700 1 -- 172.21.15.153:0/3141787811 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3441a2020 con 0x7fd34410d280 2024-03-21T10:36:55.670 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:55.672+0000 7fd34b400700 1 -- 172.21.15.153:0/3141787811 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd34410f6f0 con 0x7fd34410d280 2024-03-21T10:36:55.671 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:55.673+0000 7fd341600700 1 -- 172.21.15.153:0/3141787811 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fd330010380 con 0x7fd34410d280 2024-03-21T10:36:55.672 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:55.674+0000 7fd341600700 1 --2- 172.21.15.153:0/3141787811 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd32c06e460 0x7fd32c070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:55.672 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:55.674+0000 7fd341600700 1 -- 172.21.15.153:0/3141787811 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fd33008e190 con 0x7fd34410d280 2024-03-21T10:36:55.672 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:55.674+0000 7fd348a00700 1 --2- 172.21.15.153:0/3141787811 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd32c06e460 0x7fd32c070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:55.673 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:55.675+0000 7fd348a00700 1 --2- 172.21.15.153:0/3141787811 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd32c06e460 0x7fd32c070920 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fd33800b5c0 tx=0x7fd338005960 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:36:55.677 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:55.679+0000 7fd341600700 1 -- 172.21.15.153:0/3141787811 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd3300571b0 con 0x7fd34410d280 2024-03-21T10:36:55.947 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:55.949+0000 7fd34b400700 1 -- 172.21.15.153:0/3141787811 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fd34404ea90 con 0x7fd34410d280 2024-03-21T10:36:55.948 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:55.950+0000 7fd341600700 1 -- 172.21.15.153:0/3141787811 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1876 (secure 0 0 0) 0x7fd33005a870 con 0x7fd34410d280 2024-03-21T10:36:55.949 INFO:teuthology.orchestra.run.smithi153.stdout:e32 2024-03-21T10:36:55.949 INFO:teuthology.orchestra.run.smithi153.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-21T10:36:55.949 INFO:teuthology.orchestra.run.smithi153.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:36:55.949 INFO:teuthology.orchestra.run.smithi153.stdout:legacy client fscid: 1 2024-03-21T10:36:55.949 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:36:55.949 INFO:teuthology.orchestra.run.smithi153.stdout:Filesystem 'cephfs' (1) 2024-03-21T10:36:55.949 INFO:teuthology.orchestra.run.smithi153.stdout:fs_name cephfs 2024-03-21T10:36:55.949 INFO:teuthology.orchestra.run.smithi153.stdout:epoch 23 2024-03-21T10:36:55.949 INFO:teuthology.orchestra.run.smithi153.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-21T10:36:55.949 INFO:teuthology.orchestra.run.smithi153.stdout:created 2024-03-21T10:04:27.086576+0000 2024-03-21T10:36:55.950 INFO:teuthology.orchestra.run.smithi153.stdout:modified 2024-03-21T10:19:05.022092+0000 2024-03-21T10:36:55.950 INFO:teuthology.orchestra.run.smithi153.stdout:tableserver 0 2024-03-21T10:36:55.950 INFO:teuthology.orchestra.run.smithi153.stdout:root 0 2024-03-21T10:36:55.950 INFO:teuthology.orchestra.run.smithi153.stdout:session_timeout 60 2024-03-21T10:36:55.950 INFO:teuthology.orchestra.run.smithi153.stdout:session_autoclose 300 2024-03-21T10:36:55.950 INFO:teuthology.orchestra.run.smithi153.stdout:max_file_size 1099511627776 2024-03-21T10:36:55.950 INFO:teuthology.orchestra.run.smithi153.stdout:max_xattr_size 65536 2024-03-21T10:36:55.950 INFO:teuthology.orchestra.run.smithi153.stdout:required_client_features {} 2024-03-21T10:36:55.950 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure 0 2024-03-21T10:36:55.950 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure_osd_epoch 94 2024-03-21T10:36:55.950 INFO:teuthology.orchestra.run.smithi153.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:36:55.950 INFO:teuthology.orchestra.run.smithi153.stdout:max_mds 2 2024-03-21T10:36:55.950 INFO:teuthology.orchestra.run.smithi153.stdout:in 0,1 2024-03-21T10:36:55.950 INFO:teuthology.orchestra.run.smithi153.stdout:up {} 2024-03-21T10:36:55.951 INFO:teuthology.orchestra.run.smithi153.stdout:failed 0,1 2024-03-21T10:36:55.951 INFO:teuthology.orchestra.run.smithi153.stdout:damaged 2024-03-21T10:36:55.951 INFO:teuthology.orchestra.run.smithi153.stdout:stopped 2024-03-21T10:36:55.951 INFO:teuthology.orchestra.run.smithi153.stdout:data_pools [3] 2024-03-21T10:36:55.951 INFO:teuthology.orchestra.run.smithi153.stdout:metadata_pool 2 2024-03-21T10:36:55.951 INFO:teuthology.orchestra.run.smithi153.stdout:inline_data enabled 2024-03-21T10:36:55.951 INFO:teuthology.orchestra.run.smithi153.stdout:balancer 2024-03-21T10:36:55.951 INFO:teuthology.orchestra.run.smithi153.stdout:bal_rank_mask -1 2024-03-21T10:36:55.951 INFO:teuthology.orchestra.run.smithi153.stdout:standby_count_wanted 1 2024-03-21T10:36:55.951 INFO:teuthology.orchestra.run.smithi153.stdout:qdb_cluster leader: 0 members: 2024-03-21T10:36:55.951 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:36:55.951 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:36:55.951 INFO:teuthology.orchestra.run.smithi153.stdout:Standby daemons: 2024-03-21T10:36:55.951 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:36:55.951 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.zipckr{-1:34442} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6826/1103846295,v1:172.21.15.153:6827/1103846295] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:36:55.951 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.iuyluj{-1:34474} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6828/1738630832,v1:172.21.15.153:6829/1738630832] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:36:55.952 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.yyimst{-1:44313} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.174:6824/4127786797,v1:172.21.15.174:6825/4127786797] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:36:55.952 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.nhozag{-1:44687} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.174:6826/2767517845,v1:172.21.15.174:6827/2767517845] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:36:55.953 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:55.954+0000 7fd34b400700 1 -- 172.21.15.153:0/3141787811 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd32c06e460 msgr2=0x7fd32c070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:55.953 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:55.954+0000 7fd34b400700 1 --2- 172.21.15.153:0/3141787811 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd32c06e460 0x7fd32c070920 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fd33800b5c0 tx=0x7fd338005960 comp rx=0 tx=0).stop 2024-03-21T10:36:55.953 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:55.954+0000 7fd34b400700 1 -- 172.21.15.153:0/3141787811 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd34410d280 msgr2=0x7fd34419d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:55.953 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:55.954+0000 7fd34b400700 1 --2- 172.21.15.153:0/3141787811 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd34410d280 0x7fd34419d350 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fd33000b4b0 tx=0x7fd33000b870 comp rx=0 tx=0).stop 2024-03-21T10:36:55.953 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:55.954+0000 7fd34b400700 1 -- 172.21.15.153:0/3141787811 shutdown_connections 2024-03-21T10:36:55.953 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:55.954+0000 7fd34b400700 1 --2- 172.21.15.153:0/3141787811 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd3441071d0 0x7fd34419ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:55.953 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:55.954+0000 7fd34b400700 1 --2- 172.21.15.153:0/3141787811 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd32c06e460 0x7fd32c070920 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:55.953 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:55.954+0000 7fd34b400700 1 --2- 172.21.15.153:0/3141787811 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd34410d280 0x7fd34419d350 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:55.953 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:55.954+0000 7fd34b400700 1 -- 172.21.15.153:0/3141787811 >> 172.21.15.153:0/3141787811 conn(0x7fd3440769e0 msgr2=0x7fd3440785c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:36:55.953 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:55.954+0000 7fd34b400700 1 -- 172.21.15.153:0/3141787811 shutdown_connections 2024-03-21T10:36:55.953 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:55.955+0000 7fd34b400700 1 -- 172.21.15.153:0/3141787811 wait complete. 2024-03-21T10:36:55.954 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 32 2024-03-21T10:36:56.117 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:56.118+0000 7ff072800700 1 -- 172.21.15.153:0/3162456695 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff06c101790 msgr2=0x7ff06c1057e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:56.117 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:56.118+0000 7ff072800700 1 --2- 172.21.15.153:0/3162456695 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff06c101790 0x7ff06c1057e0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7ff058009910 tx=0x7ff058009c20 comp rx=0 tx=0).stop 2024-03-21T10:36:56.117 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:56.118+0000 7ff072800700 1 -- 172.21.15.153:0/3162456695 shutdown_connections 2024-03-21T10:36:56.117 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:56.118+0000 7ff072800700 1 --2- 172.21.15.153:0/3162456695 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff06c101790 0x7ff06c1057e0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:56.117 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:56.118+0000 7ff072800700 1 --2- 172.21.15.153:0/3162456695 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff06c100de0 0x7ff06c1011c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:56.117 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:56.118+0000 7ff072800700 1 -- 172.21.15.153:0/3162456695 >> 172.21.15.153:0/3162456695 conn(0x7ff06c0fc670 msgr2=0x7ff06c0fea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:36:56.117 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:56.118+0000 7ff072800700 1 -- 172.21.15.153:0/3162456695 shutdown_connections 2024-03-21T10:36:56.117 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:56.118+0000 7ff072800700 1 -- 172.21.15.153:0/3162456695 wait complete. 2024-03-21T10:36:56.118 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:56.120+0000 7ff072800700 1 Processor -- start 2024-03-21T10:36:56.118 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:56.120+0000 7ff072800700 1 -- start start 2024-03-21T10:36:56.118 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:56.120+0000 7ff072800700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff06c100de0 0x7ff06c19c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:56.118 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:56.120+0000 7ff072800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff06c101790 0x7ff06c19caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:56.118 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:56.120+0000 7ff072800700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff06c19d0a0 con 0x7ff06c101790 2024-03-21T10:36:56.118 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:56.120+0000 7ff072800700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff06c1965e0 con 0x7ff06c100de0 2024-03-21T10:36:56.119 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:56.121+0000 7ff06aa00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff06c101790 0x7ff06c19caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:56.120 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:56.121+0000 7ff06aa00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff06c101790 0x7ff06c19caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:48400/0 (socket says 172.21.15.153:48400) 2024-03-21T10:36:56.120 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:56.121+0000 7ff06aa00700 1 -- 172.21.15.153:0/3227875492 learned_addr learned my addr 172.21.15.153:0/3227875492 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:36:56.120 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:56.121+0000 7ff06b400700 1 --2- 172.21.15.153:0/3227875492 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff06c100de0 0x7ff06c19c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:56.120 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:56.121+0000 7ff06aa00700 1 -- 172.21.15.153:0/3227875492 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff06c100de0 msgr2=0x7ff06c19c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:56.120 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:56.121+0000 7ff06aa00700 1 --2- 172.21.15.153:0/3227875492 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff06c100de0 0x7ff06c19c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:56.120 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:56.121+0000 7ff06aa00700 1 -- 172.21.15.153:0/3227875492 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0580095f0 con 0x7ff06c101790 2024-03-21T10:36:56.120 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:56.121+0000 7ff06aa00700 1 --2- 172.21.15.153:0/3227875492 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff06c101790 0x7ff06c19caa0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7ff058005f50 tx=0x7ff05800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:36:56.120 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:56.122+0000 7ff063e00700 1 -- 172.21.15.153:0/3227875492 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff05801d070 con 0x7ff06c101790 2024-03-21T10:36:56.121 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:56.122+0000 7ff063e00700 1 -- 172.21.15.153:0/3227875492 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff05800f460 con 0x7ff06c101790 2024-03-21T10:36:56.121 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:56.122+0000 7ff072800700 1 -- 172.21.15.153:0/3227875492 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff06c196860 con 0x7ff06c101790 2024-03-21T10:36:56.121 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:56.122+0000 7ff063e00700 1 -- 172.21.15.153:0/3227875492 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff058017630 con 0x7ff06c101790 2024-03-21T10:36:56.121 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:56.122+0000 7ff072800700 1 -- 172.21.15.153:0/3227875492 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff06c196d50 con 0x7ff06c101790 2024-03-21T10:36:56.121 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:56.123+0000 7ff072800700 1 -- 172.21.15.153:0/3227875492 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff06c1091e0 con 0x7ff06c101790 2024-03-21T10:36:56.122 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:56.124+0000 7ff063e00700 1 -- 172.21.15.153:0/3227875492 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7ff05800f5d0 con 0x7ff06c101790 2024-03-21T10:36:56.123 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:56.125+0000 7ff063e00700 1 --2- 172.21.15.153:0/3227875492 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff05006e460 0x7ff050070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:56.123 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:56.125+0000 7ff063e00700 1 -- 172.21.15.153:0/3227875492 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7ff05808fb90 con 0x7ff06c101790 2024-03-21T10:36:56.123 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:56.125+0000 7ff06b400700 1 --2- 172.21.15.153:0/3227875492 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff05006e460 0x7ff050070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:56.124 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:56.126+0000 7ff06b400700 1 --2- 172.21.15.153:0/3227875492 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff05006e460 0x7ff050070920 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7ff05c005ea0 tx=0x7ff05c005e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:36:56.128 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:56.130+0000 7ff063e00700 1 -- 172.21.15.153:0/3227875492 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff05805c6e0 con 0x7ff06c101790 2024-03-21T10:36:56.303 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:36:56 smithi153 ceph-mon[113733]: from='client.35354 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:36:56.303 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:36:56 smithi153 ceph-mon[113733]: from='client.35358 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:36:56.303 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:36:56 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/256336849' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:36:56.303 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:36:56 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/3141787811' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:36:56.375 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:56.376+0000 7ff072800700 1 -- 172.21.15.153:0/3227875492 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff06c197030 con 0x7ff05006e460 2024-03-21T10:36:56.377 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:56.379+0000 7ff063e00700 1 -- 172.21.15.153:0/3227875492 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7ff06c197030 con 0x7ff05006e460 2024-03-21T10:36:56.377 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:36:56.377 INFO:teuthology.orchestra.run.smithi153.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232", 2024-03-21T10:36:56.377 INFO:teuthology.orchestra.run.smithi153.stdout: "in_progress": true, 2024-03-21T10:36:56.377 INFO:teuthology.orchestra.run.smithi153.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-21T10:36:56.377 INFO:teuthology.orchestra.run.smithi153.stdout: "services_complete": [ 2024-03-21T10:36:56.377 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr", 2024-03-21T10:36:56.378 INFO:teuthology.orchestra.run.smithi153.stdout: "mon", 2024-03-21T10:36:56.378 INFO:teuthology.orchestra.run.smithi153.stdout: "mds", 2024-03-21T10:36:56.378 INFO:teuthology.orchestra.run.smithi153.stdout: "crash", 2024-03-21T10:36:56.378 INFO:teuthology.orchestra.run.smithi153.stdout: "osd" 2024-03-21T10:36:56.378 INFO:teuthology.orchestra.run.smithi153.stdout: ], 2024-03-21T10:36:56.378 INFO:teuthology.orchestra.run.smithi153.stdout: "progress": "16/23 daemons upgraded", 2024-03-21T10:36:56.378 INFO:teuthology.orchestra.run.smithi153.stdout: "message": "Currently upgrading mds daemons", 2024-03-21T10:36:56.378 INFO:teuthology.orchestra.run.smithi153.stdout: "is_paused": false 2024-03-21T10:36:56.378 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:36:56.380 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:56.381+0000 7ff072800700 1 -- 172.21.15.153:0/3227875492 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff05006e460 msgr2=0x7ff050070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:56.380 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:56.381+0000 7ff072800700 1 --2- 172.21.15.153:0/3227875492 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff05006e460 0x7ff050070920 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7ff05c005ea0 tx=0x7ff05c005e30 comp rx=0 tx=0).stop 2024-03-21T10:36:56.380 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:56.381+0000 7ff072800700 1 -- 172.21.15.153:0/3227875492 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff06c101790 msgr2=0x7ff06c19caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:56.380 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:56.381+0000 7ff072800700 1 --2- 172.21.15.153:0/3227875492 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff06c101790 0x7ff06c19caa0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7ff058005f50 tx=0x7ff05800be20 comp rx=0 tx=0).stop 2024-03-21T10:36:56.380 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:56.382+0000 7ff072800700 1 -- 172.21.15.153:0/3227875492 shutdown_connections 2024-03-21T10:36:56.380 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:56.382+0000 7ff072800700 1 --2- 172.21.15.153:0/3227875492 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff06c100de0 0x7ff06c19c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:56.380 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:56.382+0000 7ff072800700 1 --2- 172.21.15.153:0/3227875492 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff05006e460 0x7ff050070920 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:56.380 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:56.382+0000 7ff072800700 1 --2- 172.21.15.153:0/3227875492 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff06c101790 0x7ff06c19caa0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:56.381 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:56.382+0000 7ff072800700 1 -- 172.21.15.153:0/3227875492 >> 172.21.15.153:0/3227875492 conn(0x7ff06c0fc670 msgr2=0x7ff06c0fea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:36:56.381 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:56.382+0000 7ff072800700 1 -- 172.21.15.153:0/3227875492 shutdown_connections 2024-03-21T10:36:56.381 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:56.382+0000 7ff072800700 1 -- 172.21.15.153:0/3227875492 wait complete. 2024-03-21T10:36:56.448 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:36:56 smithi174 ceph-mon[88004]: from='client.35354 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:36:56.448 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:36:56 smithi174 ceph-mon[88004]: from='client.35358 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:36:56.448 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:36:56 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/256336849' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:36:56.448 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:36:56 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/3141787811' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:36:56.534 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:56.536+0000 7f285c000700 1 -- 172.21.15.153:0/2631029839 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2854100df0 msgr2=0x7f28541011d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:56.535 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:56.536+0000 7f285c000700 1 --2- 172.21.15.153:0/2631029839 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2854100df0 0x7f28541011d0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f2848009910 tx=0x7f2848009c20 comp rx=0 tx=0).stop 2024-03-21T10:36:56.535 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:56.536+0000 7f285c000700 1 -- 172.21.15.153:0/2631029839 shutdown_connections 2024-03-21T10:36:56.535 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:56.536+0000 7f285c000700 1 --2- 172.21.15.153:0/2631029839 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f28541017a0 0x7f28541057f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:56.535 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:56.536+0000 7f285c000700 1 --2- 172.21.15.153:0/2631029839 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2854100df0 0x7f28541011d0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:56.535 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:56.536+0000 7f285c000700 1 -- 172.21.15.153:0/2631029839 >> 172.21.15.153:0/2631029839 conn(0x7f28540fc6a0 msgr2=0x7f28540feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:36:56.535 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:56.536+0000 7f285c000700 1 -- 172.21.15.153:0/2631029839 shutdown_connections 2024-03-21T10:36:56.535 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:56.536+0000 7f285c000700 1 -- 172.21.15.153:0/2631029839 wait complete. 2024-03-21T10:36:56.536 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:56.537+0000 7f285c000700 1 Processor -- start 2024-03-21T10:36:56.536 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:56.537+0000 7f285c000700 1 -- start start 2024-03-21T10:36:56.536 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:56.538+0000 7f285c000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2854100df0 0x7f285419c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:56.536 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:56.538+0000 7f285c000700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f28541017a0 0x7f285419ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:56.536 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:56.538+0000 7f285c000700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f285419d100 con 0x7f2854100df0 2024-03-21T10:36:56.536 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:56.538+0000 7f285c000700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28541965b0 con 0x7f28541017a0 2024-03-21T10:36:56.536 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:56.538+0000 7f2859600700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2854100df0 0x7f285419c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:56.536 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:56.538+0000 7f2859600700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2854100df0 0x7f285419c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:48422/0 (socket says 172.21.15.153:48422) 2024-03-21T10:36:56.537 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:56.538+0000 7f2859600700 1 -- 172.21.15.153:0/4108359866 learned_addr learned my addr 172.21.15.153:0/4108359866 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:36:56.538 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:56.538+0000 7f2858c00700 1 --2- 172.21.15.153:0/4108359866 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f28541017a0 0x7f285419ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:56.538 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:56.538+0000 7f2859600700 1 -- 172.21.15.153:0/4108359866 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f28541017a0 msgr2=0x7f285419ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:56.538 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:56.538+0000 7f2859600700 1 --2- 172.21.15.153:0/4108359866 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f28541017a0 0x7f285419ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:56.538 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:56.538+0000 7f2859600700 1 -- 172.21.15.153:0/4108359866 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28480095f0 con 0x7f2854100df0 2024-03-21T10:36:56.538 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:56.539+0000 7f2858c00700 1 --2- 172.21.15.153:0/4108359866 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f28541017a0 0x7f285419ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:36:56.538 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:56.539+0000 7f2859600700 1 --2- 172.21.15.153:0/4108359866 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2854100df0 0x7f285419c530 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f284800b5c0 tx=0x7f284800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:36:56.538 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:56.539+0000 7f2852000700 1 -- 172.21.15.153:0/4108359866 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f284801d070 con 0x7f2854100df0 2024-03-21T10:36:56.538 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:56.539+0000 7f2852000700 1 -- 172.21.15.153:0/4108359866 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2848005e00 con 0x7f2854100df0 2024-03-21T10:36:56.538 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:56.539+0000 7f285c000700 1 -- 172.21.15.153:0/4108359866 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2854196830 con 0x7f2854100df0 2024-03-21T10:36:56.539 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:56.539+0000 7f2852000700 1 -- 172.21.15.153:0/4108359866 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2848017400 con 0x7f2854100df0 2024-03-21T10:36:56.539 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:56.539+0000 7f285c000700 1 -- 172.21.15.153:0/4108359866 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2854196d20 con 0x7f2854100df0 2024-03-21T10:36:56.539 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:56.540+0000 7f285c000700 1 -- 172.21.15.153:0/4108359866 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2854109190 con 0x7f2854100df0 2024-03-21T10:36:56.539 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:56.541+0000 7f2852000700 1 -- 172.21.15.153:0/4108359866 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f284800f460 con 0x7f2854100df0 2024-03-21T10:36:56.541 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:56.542+0000 7f2852000700 1 --2- 172.21.15.153:0/4108359866 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f283c06e460 0x7f283c070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:36:56.541 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:56.542+0000 7f2852000700 1 -- 172.21.15.153:0/4108359866 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f284808fab0 con 0x7f2854100df0 2024-03-21T10:36:56.541 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:56.543+0000 7f2858c00700 1 --2- 172.21.15.153:0/4108359866 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f283c06e460 0x7f283c070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:36:56.541 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:56.543+0000 7f2858c00700 1 --2- 172.21.15.153:0/4108359866 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f283c06e460 0x7f283c070920 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f2840005d90 tx=0x7f2840005d00 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:36:56.545 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:56.547+0000 7f2852000700 1 -- 172.21.15.153:0/4108359866 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2848091e60 con 0x7f2854100df0 2024-03-21T10:36:56.846 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:56.847+0000 7f285c000700 1 -- 172.21.15.153:0/4108359866 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f2854108b70 con 0x7f2854100df0 2024-03-21T10:36:56.846 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:56.848+0000 7f2852000700 1 -- 172.21.15.153:0/4108359866 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4493 (secure 0 0 0) 0x7f284805c600 con 0x7f2854100df0 2024-03-21T10:36:56.847 INFO:teuthology.orchestra.run.smithi153.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-03-21T10:36:56.847 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-03-21T10:36:56.847 INFO:teuthology.orchestra.run.smithi153.stdout: host smithi174 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi174/config 2024-03-21T10:36:56.847 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/podman: timeout after 895 seconds 2024-03-21T10:36:56.847 INFO:teuthology.orchestra.run.smithi153.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 -e NODE_NAME=smithi174 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/run/ceph:z -v /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpd910r3_i:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 inventory --format=json-pretty --filter-for-batch 2024-03-21T10:36:56.847 INFO:teuthology.orchestra.run.smithi153.stdout:Traceback (most recent call last): 2024-03-21T10:36:56.847 INFO:teuthology.orchestra.run.smithi153.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-03-21T10:36:56.848 INFO:teuthology.orchestra.run.smithi153.stdout: return _run_code(code, main_globals, None, 2024-03-21T10:36:56.848 INFO:teuthology.orchestra.run.smithi153.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-03-21T10:36:56.848 INFO:teuthology.orchestra.run.smithi153.stdout: exec(code, run_globals) 2024-03-21T10:36:56.848 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 5606, in 2024-03-21T10:36:56.848 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 5594, in main 2024-03-21T10:36:56.848 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 406, in _infer_config 2024-03-21T10:36:56.848 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 321, in _infer_fsid 2024-03-21T10:36:56.848 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 434, in _infer_image 2024-03-21T10:36:56.848 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 308, in _validate_fsid 2024-03-21T10:36:56.848 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 3357, in command_ceph_volume 2024-03-21T10:36:56.848 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-03-21T10:36:56.848 INFO:teuthology.orchestra.run.smithi153.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 -e NODE_NAME=smithi174 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/run/ceph:z -v /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpd910r3_i:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 inventory --format=json-pretty --filter-for-batch 2024-03-21T10:36:56.848 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-21T10:36:56.848 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is degraded 2024-03-21T10:36:56.849 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-21T10:36:56.849 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-21T10:36:56.849 INFO:teuthology.orchestra.run.smithi153.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-21T10:36:56.849 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-21T10:36:56.849 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:56.851+0000 7f285c000700 1 -- 172.21.15.153:0/4108359866 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f283c06e460 msgr2=0x7f283c070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:56.849 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:56.851+0000 7f285c000700 1 --2- 172.21.15.153:0/4108359866 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f283c06e460 0x7f283c070920 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f2840005d90 tx=0x7f2840005d00 comp rx=0 tx=0).stop 2024-03-21T10:36:56.850 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:56.851+0000 7f285c000700 1 -- 172.21.15.153:0/4108359866 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2854100df0 msgr2=0x7f285419c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:36:56.850 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:56.851+0000 7f285c000700 1 --2- 172.21.15.153:0/4108359866 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2854100df0 0x7f285419c530 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f284800b5c0 tx=0x7f284800be20 comp rx=0 tx=0).stop 2024-03-21T10:36:56.850 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:56.851+0000 7f285c000700 1 -- 172.21.15.153:0/4108359866 shutdown_connections 2024-03-21T10:36:56.850 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:56.851+0000 7f285c000700 1 --2- 172.21.15.153:0/4108359866 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f283c06e460 0x7f283c070920 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:56.850 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:56.851+0000 7f285c000700 1 --2- 172.21.15.153:0/4108359866 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2854100df0 0x7f285419c530 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:56.850 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:56.851+0000 7f285c000700 1 --2- 172.21.15.153:0/4108359866 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f28541017a0 0x7f285419ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:36:56.850 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:56.851+0000 7f285c000700 1 -- 172.21.15.153:0/4108359866 >> 172.21.15.153:0/4108359866 conn(0x7f28540fc6a0 msgr2=0x7f28540fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:36:56.850 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:56.852+0000 7f285c000700 1 -- 172.21.15.153:0/4108359866 shutdown_connections 2024-03-21T10:36:56.850 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:36:56.852+0000 7f285c000700 1 -- 172.21.15.153:0/4108359866 wait complete. 2024-03-21T10:36:57.303 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:36:57 smithi153 ceph-mon[113733]: pgmap v754: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:36:57.303 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:36:57 smithi153 ceph-mon[113733]: from='client.35370 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:36:57.303 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:36:57 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/4108359866' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:36:57.448 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:36:57 smithi174 ceph-mon[88004]: pgmap v754: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:36:57.448 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:36:57 smithi174 ceph-mon[88004]: from='client.35370 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:36:57.448 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:36:57 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/4108359866' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:36:59.303 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:36:59 smithi153 ceph-mon[113733]: pgmap v755: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:36:59.448 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:36:59 smithi174 ceph-mon[88004]: pgmap v755: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:37:01.303 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:37:01 smithi153 ceph-mon[113733]: pgmap v756: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:37:01.448 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:37:01 smithi174 ceph-mon[88004]: pgmap v756: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:37:02.303 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:37:02 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:37:02.448 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:37:02 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:37:03.303 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:37:03 smithi153 ceph-mon[113733]: pgmap v757: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:37:03.448 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:37:03 smithi174 ceph-mon[88004]: pgmap v757: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:37:05.303 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:37:05 smithi153 ceph-mon[113733]: pgmap v758: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:37:05.448 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:37:05 smithi174 ceph-mon[88004]: pgmap v758: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:37:07.448 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:37:07 smithi174 ceph-mon[88004]: pgmap v759: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:37:07.553 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:37:07 smithi153 ceph-mon[113733]: pgmap v759: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:37:09.448 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:37:09 smithi174 ceph-mon[88004]: pgmap v760: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:37:09.553 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:37:09 smithi153 ceph-mon[113733]: pgmap v760: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:37:11.448 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:37:11 smithi174 ceph-mon[88004]: pgmap v761: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:37:11.553 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:37:11 smithi153 ceph-mon[113733]: pgmap v761: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:37:13.448 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:37:13 smithi174 ceph-mon[88004]: pgmap v762: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:37:13.553 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:37:13 smithi153 ceph-mon[113733]: pgmap v762: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:37:15.448 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:37:15 smithi174 ceph-mon[88004]: pgmap v763: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:37:15.553 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:37:15 smithi153 ceph-mon[113733]: pgmap v763: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:37:17.448 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:37:17 smithi174 ceph-mon[88004]: pgmap v764: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:37:17.448 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:37:17 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:37:17.553 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:37:17 smithi153 ceph-mon[113733]: pgmap v764: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:37:17.553 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:37:17 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:37:19.448 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:37:19 smithi174 ceph-mon[88004]: pgmap v765: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:37:19.553 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:37:19 smithi153 ceph-mon[113733]: pgmap v765: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:37:21.448 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:37:21 smithi174 ceph-mon[88004]: pgmap v766: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:37:21.553 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:37:21 smithi153 ceph-mon[113733]: pgmap v766: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:37:23.448 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:37:23 smithi174 ceph-mon[88004]: pgmap v767: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:37:23.553 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:37:23 smithi153 ceph-mon[113733]: pgmap v767: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:37:25.448 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:37:25 smithi174 ceph-mon[88004]: pgmap v768: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:37:25.553 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:37:25 smithi153 ceph-mon[113733]: pgmap v768: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:37:27.020 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.020+0000 7fb0f6400700 1 -- 172.21.15.153:0/2429871504 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb0f01071d0 msgr2=0x7fb0f0107650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:37:27.020 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.020+0000 7fb0f6400700 1 --2- 172.21.15.153:0/2429871504 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb0f01071d0 0x7fb0f0107650 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc009910 tx=0x7fb0dc009c20 comp rx=0 tx=0).stop 2024-03-21T10:37:27.020 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.020+0000 7fb0f6400700 1 -- 172.21.15.153:0/2429871504 shutdown_connections 2024-03-21T10:37:27.020 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.021+0000 7fb0f6400700 1 --2- 172.21.15.153:0/2429871504 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb0f01071d0 0x7fb0f0107650 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:37:27.020 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.021+0000 7fb0f6400700 1 --2- 172.21.15.153:0/2429871504 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0f010d280 0x7fb0f010d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:37:27.020 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.021+0000 7fb0f6400700 1 -- 172.21.15.153:0/2429871504 >> 172.21.15.153:0/2429871504 conn(0x7fb0f00769e0 msgr2=0x7fb0f0078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:37:27.020 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.021+0000 7fb0f6400700 1 -- 172.21.15.153:0/2429871504 shutdown_connections 2024-03-21T10:37:27.020 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.021+0000 7fb0f6400700 1 -- 172.21.15.153:0/2429871504 wait complete. 2024-03-21T10:37:27.020 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.022+0000 7fb0f6400700 1 Processor -- start 2024-03-21T10:37:27.022 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.022+0000 7fb0f6400700 1 -- start start 2024-03-21T10:37:27.022 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.022+0000 7fb0f6400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb0f01071d0 0x7fb0f019ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:37:27.022 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.022+0000 7fb0f6400700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0f010d280 0x7fb0f019d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:37:27.022 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.022+0000 7fb0f6400700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f019d880 con 0x7fb0f01071d0 2024-03-21T10:37:27.022 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.022+0000 7fb0f6400700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f01a05b0 con 0x7fb0f010d280 2024-03-21T10:37:27.022 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.023+0000 7fb0ef400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb0f01071d0 0x7fb0f019ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:37:27.022 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.023+0000 7fb0ef400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb0f01071d0 0x7fb0f019ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:53326/0 (socket says 172.21.15.153:53326) 2024-03-21T10:37:27.023 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.023+0000 7fb0ef400700 1 -- 172.21.15.153:0/2970426693 learned_addr learned my addr 172.21.15.153:0/2970426693 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:37:27.023 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.023+0000 7fb0ef400700 1 -- 172.21.15.153:0/2970426693 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0f010d280 msgr2=0x7fb0f019d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:37:27.023 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.023+0000 7fb0ef400700 1 --2- 172.21.15.153:0/2970426693 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0f010d280 0x7fb0f019d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:37:27.023 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.023+0000 7fb0ef400700 1 -- 172.21.15.153:0/2970426693 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0dc0095f0 con 0x7fb0f01071d0 2024-03-21T10:37:27.023 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.023+0000 7fb0ef400700 1 --2- 172.21.15.153:0/2970426693 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb0f01071d0 0x7fb0f019ccb0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fb0e000d8d0 tx=0x7fb0e000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:37:27.023 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.024+0000 7fb0e7e00700 1 -- 172.21.15.153:0/2970426693 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb0e0009940 con 0x7fb0f01071d0 2024-03-21T10:37:27.023 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.024+0000 7fb0f6400700 1 -- 172.21.15.153:0/2970426693 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0f01a0890 con 0x7fb0f01071d0 2024-03-21T10:37:27.023 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.024+0000 7fb0e7e00700 1 -- 172.21.15.153:0/2970426693 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb0e0010460 con 0x7fb0f01071d0 2024-03-21T10:37:27.023 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.024+0000 7fb0e7e00700 1 -- 172.21.15.153:0/2970426693 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb0e000f5f0 con 0x7fb0f01071d0 2024-03-21T10:37:27.023 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.024+0000 7fb0f6400700 1 -- 172.21.15.153:0/2970426693 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0f01a0e60 con 0x7fb0f01071d0 2024-03-21T10:37:27.024 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.025+0000 7fb0f6400700 1 -- 172.21.15.153:0/2970426693 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0f010f6c0 con 0x7fb0f01071d0 2024-03-21T10:37:27.024 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.026+0000 7fb0e7e00700 1 -- 172.21.15.153:0/2970426693 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fb0e00105d0 con 0x7fb0f01071d0 2024-03-21T10:37:27.025 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.027+0000 7fb0e7e00700 1 --2- 172.21.15.153:0/2970426693 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb0d406e4b0 0x7fb0d4070970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:37:27.025 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.027+0000 7fb0e7e00700 1 -- 172.21.15.153:0/2970426693 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fb0e008e480 con 0x7fb0f01071d0 2024-03-21T10:37:27.025 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.027+0000 7fb0eea00700 1 --2- 172.21.15.153:0/2970426693 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb0d406e4b0 0x7fb0d4070970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:37:27.026 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.028+0000 7fb0eea00700 1 --2- 172.21.15.153:0/2970426693 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb0d406e4b0 0x7fb0d4070970 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc00b5c0 tx=0x7fb0dc005fd0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:37:27.030 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.032+0000 7fb0e7e00700 1 -- 172.21.15.153:0/2970426693 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb0e0057520 con 0x7fb0f01071d0 2024-03-21T10:37:27.274 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.276+0000 7fb0f6400700 1 -- 172.21.15.153:0/2970426693 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb0f01a1140 con 0x7fb0d406e4b0 2024-03-21T10:37:27.274 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:37:27 smithi153 ceph-mon[113733]: pgmap v769: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:37:27.276 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.278+0000 7fb0e7e00700 1 -- 172.21.15.153:0/2970426693 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fb0f01a1140 con 0x7fb0d406e4b0 2024-03-21T10:37:27.279 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.281+0000 7fb0f6400700 1 -- 172.21.15.153:0/2970426693 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb0d406e4b0 msgr2=0x7fb0d4070970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:37:27.279 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.281+0000 7fb0f6400700 1 --2- 172.21.15.153:0/2970426693 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb0d406e4b0 0x7fb0d4070970 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc00b5c0 tx=0x7fb0dc005fd0 comp rx=0 tx=0).stop 2024-03-21T10:37:27.279 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.281+0000 7fb0f6400700 1 -- 172.21.15.153:0/2970426693 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb0f01071d0 msgr2=0x7fb0f019ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:37:27.279 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.281+0000 7fb0f6400700 1 --2- 172.21.15.153:0/2970426693 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb0f01071d0 0x7fb0f019ccb0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fb0e000d8d0 tx=0x7fb0e000dc90 comp rx=0 tx=0).stop 2024-03-21T10:37:27.280 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.281+0000 7fb0f6400700 1 -- 172.21.15.153:0/2970426693 shutdown_connections 2024-03-21T10:37:27.280 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.281+0000 7fb0f6400700 1 --2- 172.21.15.153:0/2970426693 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb0d406e4b0 0x7fb0d4070970 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:37:27.280 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.281+0000 7fb0f6400700 1 --2- 172.21.15.153:0/2970426693 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb0f01071d0 0x7fb0f019ccb0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:37:27.280 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.281+0000 7fb0f6400700 1 --2- 172.21.15.153:0/2970426693 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0f010d280 0x7fb0f019d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:37:27.280 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.281+0000 7fb0f6400700 1 -- 172.21.15.153:0/2970426693 >> 172.21.15.153:0/2970426693 conn(0x7fb0f00769e0 msgr2=0x7fb0f0078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:37:27.280 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.281+0000 7fb0f6400700 1 -- 172.21.15.153:0/2970426693 shutdown_connections 2024-03-21T10:37:27.280 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.281+0000 7fb0f6400700 1 -- 172.21.15.153:0/2970426693 wait complete. 2024-03-21T10:37:27.295 INFO:teuthology.orchestra.run.smithi153.stdout:true 2024-03-21T10:37:27.436 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.437+0000 7f4614e00700 1 -- 172.21.15.153:0/295223296 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f461010d280 msgr2=0x7f461010d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:37:27.436 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.437+0000 7f4614e00700 1 --2- 172.21.15.153:0/295223296 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f461010d280 0x7f461010d660 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f4600009910 tx=0x7f4600009c20 comp rx=0 tx=0).stop 2024-03-21T10:37:27.436 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.437+0000 7f4614e00700 1 -- 172.21.15.153:0/295223296 shutdown_connections 2024-03-21T10:37:27.436 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.437+0000 7f4614e00700 1 --2- 172.21.15.153:0/295223296 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f46101071d0 0x7f4610107650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:37:27.436 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.437+0000 7f4614e00700 1 --2- 172.21.15.153:0/295223296 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f461010d280 0x7f461010d660 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:37:27.436 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.437+0000 7f4614e00700 1 -- 172.21.15.153:0/295223296 >> 172.21.15.153:0/295223296 conn(0x7f4610076990 msgr2=0x7f4610078db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:37:27.436 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.437+0000 7f4614e00700 1 -- 172.21.15.153:0/295223296 shutdown_connections 2024-03-21T10:37:27.436 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.437+0000 7f4614e00700 1 -- 172.21.15.153:0/295223296 wait complete. 2024-03-21T10:37:27.438 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.438+0000 7f4614e00700 1 Processor -- start 2024-03-21T10:37:27.438 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.439+0000 7f4614e00700 1 -- start start 2024-03-21T10:37:27.438 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.439+0000 7f4614e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f46101071d0 0x7f461019ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:37:27.438 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.439+0000 7f4614e00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f461010d280 0x7f461019d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:37:27.438 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.439+0000 7f4614e00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f461019d890 con 0x7f46101071d0 2024-03-21T10:37:27.438 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.439+0000 7f4614e00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46101a05c0 con 0x7f461010d280 2024-03-21T10:37:27.438 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.439+0000 7f460de00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f46101071d0 0x7f461019ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:37:27.438 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.440+0000 7f460de00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f46101071d0 0x7f461019ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:53344/0 (socket says 172.21.15.153:53344) 2024-03-21T10:37:27.438 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.440+0000 7f460de00700 1 -- 172.21.15.153:0/4017277212 learned_addr learned my addr 172.21.15.153:0/4017277212 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:37:27.438 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.440+0000 7f460d400700 1 --2- 172.21.15.153:0/4017277212 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f461010d280 0x7f461019d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:37:27.439 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.440+0000 7f460de00700 1 -- 172.21.15.153:0/4017277212 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f461010d280 msgr2=0x7f461019d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:37:27.439 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.440+0000 7f460de00700 1 --2- 172.21.15.153:0/4017277212 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f461010d280 0x7f461019d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:37:27.439 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.440+0000 7f460de00700 1 -- 172.21.15.153:0/4017277212 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46000095f0 con 0x7f46101071d0 2024-03-21T10:37:27.439 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.440+0000 7f460de00700 1 --2- 172.21.15.153:0/4017277212 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f46101071d0 0x7f461019ccc0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f4600006010 tx=0x7f460000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:37:27.439 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.441+0000 7f4606a00700 1 -- 172.21.15.153:0/4017277212 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f460001d070 con 0x7f46101071d0 2024-03-21T10:37:27.440 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.441+0000 7f4606a00700 1 -- 172.21.15.153:0/4017277212 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f460000f460 con 0x7f46101071d0 2024-03-21T10:37:27.440 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.441+0000 7f4606a00700 1 -- 172.21.15.153:0/4017277212 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4600017630 con 0x7f46101071d0 2024-03-21T10:37:27.440 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.441+0000 7f4614e00700 1 -- 172.21.15.153:0/4017277212 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46101a0840 con 0x7f46101071d0 2024-03-21T10:37:27.440 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.441+0000 7f4614e00700 1 -- 172.21.15.153:0/4017277212 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46101a0db0 con 0x7f46101071d0 2024-03-21T10:37:27.440 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.442+0000 7f4614e00700 1 -- 172.21.15.153:0/4017277212 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f461004ea90 con 0x7f46101071d0 2024-03-21T10:37:27.441 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.443+0000 7f4606a00700 1 -- 172.21.15.153:0/4017277212 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f460000f5d0 con 0x7f46101071d0 2024-03-21T10:37:27.442 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.444+0000 7f4606a00700 1 --2- 172.21.15.153:0/4017277212 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f45f406e460 0x7f45f4070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:37:27.442 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.444+0000 7f4606a00700 1 -- 172.21.15.153:0/4017277212 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f460008f9a0 con 0x7f46101071d0 2024-03-21T10:37:27.442 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.444+0000 7f460d400700 1 --2- 172.21.15.153:0/4017277212 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f45f406e460 0x7f45f4070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:37:27.443 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.445+0000 7f460d400700 1 --2- 172.21.15.153:0/4017277212 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f45f406e460 0x7f45f4070920 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f45fc005950 tx=0x7f45fc0058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:37:27.447 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.449+0000 7f4606a00700 1 -- 172.21.15.153:0/4017277212 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4600058a40 con 0x7f46101071d0 2024-03-21T10:37:27.447 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:37:27 smithi174 ceph-mon[88004]: pgmap v769: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:37:27.688 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.690+0000 7f4614e00700 1 -- 172.21.15.153:0/4017277212 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f46101a1200 con 0x7f45f406e460 2024-03-21T10:37:27.691 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.694+0000 7f4606a00700 1 -- 172.21.15.153:0/4017277212 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f46101a1200 con 0x7f45f406e460 2024-03-21T10:37:27.694 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.696+0000 7f4614e00700 1 -- 172.21.15.153:0/4017277212 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f45f406e460 msgr2=0x7f45f4070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:37:27.694 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.696+0000 7f4614e00700 1 --2- 172.21.15.153:0/4017277212 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f45f406e460 0x7f45f4070920 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f45fc005950 tx=0x7f45fc0058e0 comp rx=0 tx=0).stop 2024-03-21T10:37:27.694 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.696+0000 7f4614e00700 1 -- 172.21.15.153:0/4017277212 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f46101071d0 msgr2=0x7f461019ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:37:27.694 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.696+0000 7f4614e00700 1 --2- 172.21.15.153:0/4017277212 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f46101071d0 0x7f461019ccc0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f4600006010 tx=0x7f460000be20 comp rx=0 tx=0).stop 2024-03-21T10:37:27.694 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.696+0000 7f4614e00700 1 -- 172.21.15.153:0/4017277212 shutdown_connections 2024-03-21T10:37:27.694 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.696+0000 7f4614e00700 1 --2- 172.21.15.153:0/4017277212 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f45f406e460 0x7f45f4070920 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:37:27.694 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.696+0000 7f4614e00700 1 --2- 172.21.15.153:0/4017277212 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f46101071d0 0x7f461019ccc0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:37:27.694 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.696+0000 7f4614e00700 1 --2- 172.21.15.153:0/4017277212 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f461010d280 0x7f461019d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:37:27.694 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.696+0000 7f4614e00700 1 -- 172.21.15.153:0/4017277212 >> 172.21.15.153:0/4017277212 conn(0x7f4610076990 msgr2=0x7f4610078310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:37:27.695 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.696+0000 7f4614e00700 1 -- 172.21.15.153:0/4017277212 shutdown_connections 2024-03-21T10:37:27.695 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.696+0000 7f4614e00700 1 -- 172.21.15.153:0/4017277212 wait complete. 2024-03-21T10:37:27.843 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.844+0000 7f449de00700 1 -- 172.21.15.153:0/1748594455 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4498100da0 msgr2=0x7f4498101180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:37:27.843 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.844+0000 7f449de00700 1 --2- 172.21.15.153:0/1748594455 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4498100da0 0x7f4498101180 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f4488009910 tx=0x7f4488009c20 comp rx=0 tx=0).stop 2024-03-21T10:37:27.843 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.844+0000 7f449de00700 1 -- 172.21.15.153:0/1748594455 shutdown_connections 2024-03-21T10:37:27.843 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.844+0000 7f449de00700 1 --2- 172.21.15.153:0/1748594455 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4498101750 0x7f44981057a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:37:27.843 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.844+0000 7f449de00700 1 --2- 172.21.15.153:0/1748594455 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4498100da0 0x7f4498101180 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:37:27.843 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.844+0000 7f449de00700 1 -- 172.21.15.153:0/1748594455 >> 172.21.15.153:0/1748594455 conn(0x7f44980fc630 msgr2=0x7f44980fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:37:27.843 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.844+0000 7f449de00700 1 -- 172.21.15.153:0/1748594455 shutdown_connections 2024-03-21T10:37:27.843 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.845+0000 7f449de00700 1 -- 172.21.15.153:0/1748594455 wait complete. 2024-03-21T10:37:27.845 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.845+0000 7f449de00700 1 Processor -- start 2024-03-21T10:37:27.845 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.846+0000 7f449de00700 1 -- start start 2024-03-21T10:37:27.845 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.846+0000 7f449de00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4498100da0 0x7f449819e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:37:27.845 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.846+0000 7f449de00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4498101750 0x7f449819ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:37:27.845 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.846+0000 7f449de00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f449819f350 con 0x7f4498100da0 2024-03-21T10:37:27.845 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.846+0000 7f449de00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4498198800 con 0x7f4498101750 2024-03-21T10:37:27.845 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.846+0000 7f4496a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4498100da0 0x7f449819e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:37:27.845 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.846+0000 7f4496a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4498100da0 0x7f449819e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:53360/0 (socket says 172.21.15.153:53360) 2024-03-21T10:37:27.845 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.846+0000 7f4496a00700 1 -- 172.21.15.153:0/53883804 learned_addr learned my addr 172.21.15.153:0/53883804 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:37:27.845 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.847+0000 7f4496000700 1 --2- 172.21.15.153:0/53883804 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4498101750 0x7f449819ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:37:27.845 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.847+0000 7f4496a00700 1 -- 172.21.15.153:0/53883804 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4498101750 msgr2=0x7f449819ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:37:27.846 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.847+0000 7f4496a00700 1 --2- 172.21.15.153:0/53883804 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4498101750 0x7f449819ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:37:27.846 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.847+0000 7f4496a00700 1 -- 172.21.15.153:0/53883804 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44880095f0 con 0x7f4498100da0 2024-03-21T10:37:27.846 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.847+0000 7f4496a00700 1 --2- 172.21.15.153:0/53883804 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4498100da0 0x7f449819e780 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f4488006010 tx=0x7f448800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:37:27.846 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.847+0000 7f449ca00700 1 -- 172.21.15.153:0/53883804 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f448801d070 con 0x7f4498100da0 2024-03-21T10:37:27.846 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.847+0000 7f449ca00700 1 -- 172.21.15.153:0/53883804 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f448800f460 con 0x7f4498100da0 2024-03-21T10:37:27.847 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.847+0000 7f449ca00700 1 -- 172.21.15.153:0/53883804 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4488017630 con 0x7f4498100da0 2024-03-21T10:37:27.847 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.847+0000 7f449de00700 1 -- 172.21.15.153:0/53883804 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4498198a80 con 0x7f4498100da0 2024-03-21T10:37:27.847 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.847+0000 7f449de00700 1 -- 172.21.15.153:0/53883804 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4498198f20 con 0x7f4498100da0 2024-03-21T10:37:27.847 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.848+0000 7f449de00700 1 -- 172.21.15.153:0/53883804 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4498104f20 con 0x7f4498100da0 2024-03-21T10:37:27.847 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.849+0000 7f449ca00700 1 -- 172.21.15.153:0/53883804 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f448800f5d0 con 0x7f4498100da0 2024-03-21T10:37:27.848 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.850+0000 7f449ca00700 1 --2- 172.21.15.153:0/53883804 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f447c06e460 0x7f447c070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:37:27.849 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.850+0000 7f449ca00700 1 -- 172.21.15.153:0/53883804 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f448808f9a0 con 0x7f4498100da0 2024-03-21T10:37:27.849 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.851+0000 7f4496000700 1 --2- 172.21.15.153:0/53883804 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f447c06e460 0x7f447c070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:37:27.849 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.851+0000 7f4496000700 1 --2- 172.21.15.153:0/53883804 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f447c06e460 0x7f447c070920 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f4484005950 tx=0x7f44840058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:37:27.853 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:27.855+0000 7f449ca00700 1 -- 172.21.15.153:0/53883804 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f448805c570 con 0x7f4498100da0 2024-03-21T10:37:28.072 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:28.074+0000 7f449de00700 1 -- 172.21.15.153:0/53883804 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f449802cce0 con 0x7f447c06e460 2024-03-21T10:37:28.091 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:28.093+0000 7f449ca00700 1 -- 172.21.15.153:0/53883804 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f449802cce0 con 0x7f447c06e460 2024-03-21T10:37:28.092 INFO:teuthology.orchestra.run.smithi153.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-21T10:37:28.092 INFO:teuthology.orchestra.run.smithi153.stdout:alertmanager.smithi153 smithi153 *:9093,9094 running (26m) 2m ago 39m 15.6M - 0.25.0 c8568f914cd2 9ed3f2200b5f 2024-03-21T10:37:28.092 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi153 smithi153 running (40m) 2m ago 40m 10.4M - 18.2.1 5be31c24972a 5b61994396a6 2024-03-21T10:37:28.092 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi174 smithi174 running (38m) 2m ago 38m 10.1M - 18.2.1 5be31c24972a 4f7a5fc3ac28 2024-03-21T10:37:28.092 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi153 smithi153 running (22m) 2m ago 40m 6656k - 19.0.0-2231-g2edfee68 237e43c7a65f 32c21104d5b0 2024-03-21T10:37:28.092 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi174 smithi174 running (22m) 2m ago 38m 6660k - 19.0.0-2231-g2edfee68 237e43c7a65f a04e48967ab6 2024-03-21T10:37:28.092 INFO:teuthology.orchestra.run.smithi153.stdout:grafana.smithi153 smithi153 *:3000 running (25m) 2m ago 39m 75.2M - 9.4.12 2bacad6d85d8 ee24026ab786 2024-03-21T10:37:28.092 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.iuyluj smithi153 running (17m) 2m ago 32m 18.7M - 19.0.0-2231-g2edfee68 237e43c7a65f 87aff8a5de70 2024-03-21T10:37:28.092 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.zipckr smithi153 running (18m) 2m ago 32m 18.9M - 19.0.0-2231-g2edfee68 237e43c7a65f ccc262bd5dd3 2024-03-21T10:37:28.092 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.nhozag smithi174 running (2m) 2m ago 32m 10.9M - 19.0.0-2231-g2edfee68 237e43c7a65f 9e02aea72c96 2024-03-21T10:37:28.092 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.yyimst smithi174 running (17m) 2m ago 32m 18.5M - 19.0.0-2231-g2edfee68 237e43c7a65f fc1962f9351e 2024-03-21T10:37:28.092 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi153.tluxni smithi153 *:8443,9283,8765 running (29m) 2m ago 41m 512M - 19.0.0-2231-g2edfee68 237e43c7a65f 7ffee0e55382 2024-03-21T10:37:28.093 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi174.qpcwat smithi174 *:8443,9283,8765 running (27m) 2m ago 38m 459M - 19.0.0-2231-g2edfee68 237e43c7a65f 748968f216ab 2024-03-21T10:37:28.093 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi153 smithi153 running (24m) 2m ago 42m 70.8M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f dd8038303dc3 2024-03-21T10:37:28.093 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi174 smithi174 running (23m) 2m ago 38m 51.9M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f 29d78a788f2c 2024-03-21T10:37:28.093 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi153 smithi153 *:9100 running (27m) 2m ago 40m 8140k - 1.5.0 0da6a335fe13 bb835edc0e0b 2024-03-21T10:37:28.093 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi174 smithi174 *:9100 running (27m) 2m ago 38m 8610k - 1.5.0 0da6a335fe13 7c4ecad790fa 2024-03-21T10:37:28.093 INFO:teuthology.orchestra.run.smithi153.stdout:osd.0 smithi153 running (21m) 2m ago 37m 118M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 507900fd4791 2024-03-21T10:37:28.093 INFO:teuthology.orchestra.run.smithi153.stdout:osd.1 smithi153 running (21m) 2m ago 36m 93.0M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 0d646452e9c1 2024-03-21T10:37:28.093 INFO:teuthology.orchestra.run.smithi153.stdout:osd.2 smithi153 running (20m) 2m ago 36m 91.2M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 1326f2afdfcd 2024-03-21T10:37:28.093 INFO:teuthology.orchestra.run.smithi153.stdout:osd.3 smithi174 running (20m) 2m ago 35m 87.5M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 70aafac71ad5 2024-03-21T10:37:28.093 INFO:teuthology.orchestra.run.smithi153.stdout:osd.4 smithi174 running (19m) 2m ago 35m 95.8M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 0d746ce6e79d 2024-03-21T10:37:28.093 INFO:teuthology.orchestra.run.smithi153.stdout:osd.5 smithi174 running (18m) 2m ago 34m 90.3M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 53e5f35b8cac 2024-03-21T10:37:28.093 INFO:teuthology.orchestra.run.smithi153.stdout:prometheus.smithi153 smithi153 *:9095 running (26m) 2m ago 39m 88.1M - 2.43.0 a07b618ecd1d 7d95561e50c2 2024-03-21T10:37:28.094 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:28.095+0000 7f449de00700 1 -- 172.21.15.153:0/53883804 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f447c06e460 msgr2=0x7f447c070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:37:28.094 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:28.095+0000 7f449de00700 1 --2- 172.21.15.153:0/53883804 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f447c06e460 0x7f447c070920 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f4484005950 tx=0x7f44840058e0 comp rx=0 tx=0).stop 2024-03-21T10:37:28.094 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:28.095+0000 7f449de00700 1 -- 172.21.15.153:0/53883804 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4498100da0 msgr2=0x7f449819e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:37:28.094 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:28.096+0000 7f449de00700 1 --2- 172.21.15.153:0/53883804 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4498100da0 0x7f449819e780 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f4488006010 tx=0x7f448800be20 comp rx=0 tx=0).stop 2024-03-21T10:37:28.094 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:28.096+0000 7f449de00700 1 -- 172.21.15.153:0/53883804 shutdown_connections 2024-03-21T10:37:28.094 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:28.096+0000 7f449de00700 1 --2- 172.21.15.153:0/53883804 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f447c06e460 0x7f447c070920 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:37:28.095 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:28.096+0000 7f449de00700 1 --2- 172.21.15.153:0/53883804 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4498100da0 0x7f449819e780 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:37:28.095 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:28.096+0000 7f449de00700 1 --2- 172.21.15.153:0/53883804 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4498101750 0x7f449819ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:37:28.095 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:28.096+0000 7f449de00700 1 -- 172.21.15.153:0/53883804 >> 172.21.15.153:0/53883804 conn(0x7f44980fc630 msgr2=0x7f44980fea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:37:28.095 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:28.096+0000 7f449de00700 1 -- 172.21.15.153:0/53883804 shutdown_connections 2024-03-21T10:37:28.095 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:28.096+0000 7f449de00700 1 -- 172.21.15.153:0/53883804 wait complete. 2024-03-21T10:37:28.239 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:28.241+0000 7f1c9cc00700 1 -- 172.21.15.153:0/3121817363 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c981039b0 msgr2=0x7f1c98107a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:37:28.240 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:28.241+0000 7f1c9cc00700 1 --2- 172.21.15.153:0/3121817363 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c981039b0 0x7f1c98107a00 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f1c80009910 tx=0x7f1c80009c20 comp rx=0 tx=0).stop 2024-03-21T10:37:28.240 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:28.241+0000 7f1c9cc00700 1 -- 172.21.15.153:0/3121817363 shutdown_connections 2024-03-21T10:37:28.240 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:28.241+0000 7f1c9cc00700 1 --2- 172.21.15.153:0/3121817363 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c981039b0 0x7f1c98107a00 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:37:28.240 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:28.241+0000 7f1c9cc00700 1 --2- 172.21.15.153:0/3121817363 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1c98103000 0x7f1c981033e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:37:28.240 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:28.241+0000 7f1c9cc00700 1 -- 172.21.15.153:0/3121817363 >> 172.21.15.153:0/3121817363 conn(0x7f1c980fe8d0 msgr2=0x7f1c98100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:37:28.240 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:28.241+0000 7f1c9cc00700 1 -- 172.21.15.153:0/3121817363 shutdown_connections 2024-03-21T10:37:28.240 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:28.241+0000 7f1c9cc00700 1 -- 172.21.15.153:0/3121817363 wait complete. 2024-03-21T10:37:28.241 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:28.242+0000 7f1c9cc00700 1 Processor -- start 2024-03-21T10:37:28.242 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:28.242+0000 7f1c9cc00700 1 -- start start 2024-03-21T10:37:28.242 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:28.243+0000 7f1c9cc00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1c98103000 0x7f1c9819e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:37:28.242 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:28.243+0000 7f1c9cc00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c981039b0 0x7f1c9819ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:37:28.242 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:28.243+0000 7f1c9cc00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c9819f2e0 con 0x7f1c981039b0 2024-03-21T10:37:28.242 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:28.243+0000 7f1c9cc00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c98198790 con 0x7f1c98103000 2024-03-21T10:37:28.242 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:28.243+0000 7f1c8d400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c981039b0 0x7f1c9819ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:37:28.242 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:28.243+0000 7f1c8d400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c981039b0 0x7f1c9819ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:53370/0 (socket says 172.21.15.153:53370) 2024-03-21T10:37:28.242 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:28.243+0000 7f1c8d400700 1 -- 172.21.15.153:0/245729578 learned_addr learned my addr 172.21.15.153:0/245729578 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:37:28.242 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:28.243+0000 7f1c95e00700 1 --2- 172.21.15.153:0/245729578 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1c98103000 0x7f1c9819e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:37:28.242 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:28.243+0000 7f1c8d400700 1 -- 172.21.15.153:0/245729578 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1c98103000 msgr2=0x7f1c9819e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:37:28.242 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:28.243+0000 7f1c8d400700 1 --2- 172.21.15.153:0/245729578 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1c98103000 0x7f1c9819e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:37:28.242 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:28.243+0000 7f1c8d400700 1 -- 172.21.15.153:0/245729578 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c800095f0 con 0x7f1c981039b0 2024-03-21T10:37:28.242 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:28.244+0000 7f1c8d400700 1 --2- 172.21.15.153:0/245729578 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c981039b0 0x7f1c9819ec50 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f1c8000bda0 tx=0x7f1c8000be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:37:28.243 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:28.244+0000 7f1c8f400700 1 -- 172.21.15.153:0/245729578 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1c8001d070 con 0x7f1c981039b0 2024-03-21T10:37:28.243 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:28.244+0000 7f1c8f400700 1 -- 172.21.15.153:0/245729578 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1c8000f460 con 0x7f1c981039b0 2024-03-21T10:37:28.243 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:28.244+0000 7f1c9cc00700 1 -- 172.21.15.153:0/245729578 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c98198a10 con 0x7f1c981039b0 2024-03-21T10:37:28.244 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:28.244+0000 7f1c8f400700 1 -- 172.21.15.153:0/245729578 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1c80017630 con 0x7f1c981039b0 2024-03-21T10:37:28.244 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:28.244+0000 7f1c9cc00700 1 -- 172.21.15.153:0/245729578 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c98198f00 con 0x7f1c981039b0 2024-03-21T10:37:28.244 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:28.245+0000 7f1c9cc00700 1 -- 172.21.15.153:0/245729578 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c9810b310 con 0x7f1c981039b0 2024-03-21T10:37:28.244 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:28.246+0000 7f1c8f400700 1 -- 172.21.15.153:0/245729578 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f1c8000fa40 con 0x7f1c981039b0 2024-03-21T10:37:28.245 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:28.247+0000 7f1c8f400700 1 --2- 172.21.15.153:0/245729578 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f1c7c06e460 0x7f1c7c070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:37:28.245 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:28.247+0000 7f1c8f400700 1 -- 172.21.15.153:0/245729578 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f1c8008f9f0 con 0x7f1c981039b0 2024-03-21T10:37:28.245 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:28.247+0000 7f1c95e00700 1 --2- 172.21.15.153:0/245729578 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f1c7c06e460 0x7f1c7c070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:37:28.246 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:28.248+0000 7f1c95e00700 1 --2- 172.21.15.153:0/245729578 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f1c7c06e460 0x7f1c7c070920 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f1c88005950 tx=0x7f1c880058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:37:28.249 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:28.252+0000 7f1c8f400700 1 -- 172.21.15.153:0/245729578 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f1c8005c540 con 0x7f1c981039b0 2024-03-21T10:37:28.339 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:37:28 smithi153 ceph-mon[113733]: from='client.35378 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:37:28.448 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:37:28 smithi174 ceph-mon[88004]: from='client.35378 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:37:28.554 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:28.555+0000 7f1c9cc00700 1 -- 172.21.15.153:0/245729578 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f1c9804ea90 con 0x7f1c981039b0 2024-03-21T10:37:28.554 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:28.556+0000 7f1c8f400700 1 -- 172.21.15.153:0/245729578 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f1c8005c0d0 con 0x7f1c981039b0 2024-03-21T10:37:28.555 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:37:28.555 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": { 2024-03-21T10:37:28.555 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:37:28.555 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:37:28.555 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": { 2024-03-21T10:37:28.555 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:37:28.555 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:37:28.555 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": { 2024-03-21T10:37:28.555 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 6 2024-03-21T10:37:28.555 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:37:28.555 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": { 2024-03-21T10:37:28.556 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 4 2024-03-21T10:37:28.556 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:37:28.556 INFO:teuthology.orchestra.run.smithi153.stdout: "overall": { 2024-03-21T10:37:28.556 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 14 2024-03-21T10:37:28.556 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-03-21T10:37:28.556 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:37:28.557 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:28.559+0000 7f1c9cc00700 1 -- 172.21.15.153:0/245729578 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f1c7c06e460 msgr2=0x7f1c7c070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:37:28.557 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:28.559+0000 7f1c9cc00700 1 --2- 172.21.15.153:0/245729578 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f1c7c06e460 0x7f1c7c070920 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f1c88005950 tx=0x7f1c880058e0 comp rx=0 tx=0).stop 2024-03-21T10:37:28.557 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:28.559+0000 7f1c9cc00700 1 -- 172.21.15.153:0/245729578 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c981039b0 msgr2=0x7f1c9819ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:37:28.557 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:28.559+0000 7f1c9cc00700 1 --2- 172.21.15.153:0/245729578 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c981039b0 0x7f1c9819ec50 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f1c8000bda0 tx=0x7f1c8000be80 comp rx=0 tx=0).stop 2024-03-21T10:37:28.558 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:28.559+0000 7f1c9cc00700 1 -- 172.21.15.153:0/245729578 shutdown_connections 2024-03-21T10:37:28.558 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:28.559+0000 7f1c9cc00700 1 --2- 172.21.15.153:0/245729578 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1c98103000 0x7f1c9819e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:37:28.558 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:28.559+0000 7f1c9cc00700 1 --2- 172.21.15.153:0/245729578 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f1c7c06e460 0x7f1c7c070920 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:37:28.558 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:28.559+0000 7f1c9cc00700 1 --2- 172.21.15.153:0/245729578 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1c981039b0 0x7f1c9819ec50 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:37:28.558 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:28.559+0000 7f1c9cc00700 1 -- 172.21.15.153:0/245729578 >> 172.21.15.153:0/245729578 conn(0x7f1c980fe8d0 msgr2=0x7f1c98107270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:37:28.558 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:28.559+0000 7f1c9cc00700 1 -- 172.21.15.153:0/245729578 shutdown_connections 2024-03-21T10:37:28.558 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:28.559+0000 7f1c9cc00700 1 -- 172.21.15.153:0/245729578 wait complete. 2024-03-21T10:37:28.715 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:28.716+0000 7f1ca6800700 1 -- 172.21.15.153:0/2240128932 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1ca01071d0 msgr2=0x7f1ca0107650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:37:28.715 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:28.716+0000 7f1ca6800700 1 --2- 172.21.15.153:0/2240128932 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1ca01071d0 0x7f1ca0107650 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c009910 tx=0x7f1c8c009c20 comp rx=0 tx=0).stop 2024-03-21T10:37:28.715 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:28.716+0000 7f1ca6800700 1 -- 172.21.15.153:0/2240128932 shutdown_connections 2024-03-21T10:37:28.715 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:28.716+0000 7f1ca6800700 1 --2- 172.21.15.153:0/2240128932 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1ca01071d0 0x7f1ca0107650 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:37:28.715 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:28.716+0000 7f1ca6800700 1 --2- 172.21.15.153:0/2240128932 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1ca010d280 0x7f1ca010d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:37:28.715 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:28.716+0000 7f1ca6800700 1 -- 172.21.15.153:0/2240128932 >> 172.21.15.153:0/2240128932 conn(0x7f1ca00769e0 msgr2=0x7f1ca0078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:37:28.715 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:28.716+0000 7f1ca6800700 1 -- 172.21.15.153:0/2240128932 shutdown_connections 2024-03-21T10:37:28.716 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:28.718+0000 7f1ca6800700 1 -- 172.21.15.153:0/2240128932 wait complete. 2024-03-21T10:37:28.717 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:28.718+0000 7f1ca6800700 1 Processor -- start 2024-03-21T10:37:28.717 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:28.718+0000 7f1ca6800700 1 -- start start 2024-03-21T10:37:28.717 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:28.718+0000 7f1ca6800700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1ca01071d0 0x7f1ca019ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:37:28.717 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:28.718+0000 7f1ca6800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1ca010d280 0x7f1ca019d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:37:28.717 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:28.718+0000 7f1ca6800700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca019d880 con 0x7f1ca010d280 2024-03-21T10:37:28.717 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:28.718+0000 7f1ca6800700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca01a05b0 con 0x7f1ca01071d0 2024-03-21T10:37:28.717 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:28.719+0000 7f1c9ea00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1ca010d280 0x7f1ca019d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:37:28.717 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:28.719+0000 7f1c9ea00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1ca010d280 0x7f1ca019d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:53384/0 (socket says 172.21.15.153:53384) 2024-03-21T10:37:28.717 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:28.719+0000 7f1c9ea00700 1 -- 172.21.15.153:0/756149737 learned_addr learned my addr 172.21.15.153:0/756149737 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:37:28.717 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:28.719+0000 7f1c9f400700 1 --2- 172.21.15.153:0/756149737 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1ca01071d0 0x7f1ca019ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:37:28.718 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:28.719+0000 7f1c9ea00700 1 -- 172.21.15.153:0/756149737 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1ca01071d0 msgr2=0x7f1ca019ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:37:28.718 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:28.719+0000 7f1c9ea00700 1 --2- 172.21.15.153:0/756149737 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1ca01071d0 0x7f1ca019ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:37:28.718 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:28.719+0000 7f1c9ea00700 1 -- 172.21.15.153:0/756149737 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c8c0095f0 con 0x7f1ca010d280 2024-03-21T10:37:28.718 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:28.719+0000 7f1c9ea00700 1 --2- 172.21.15.153:0/756149737 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1ca010d280 0x7f1ca019d1f0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c0092e0 tx=0x7f1c8c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:37:28.719 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:28.720+0000 7f1c97e00700 1 -- 172.21.15.153:0/756149737 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1c8c01d070 con 0x7f1ca010d280 2024-03-21T10:37:28.719 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:28.720+0000 7f1c97e00700 1 -- 172.21.15.153:0/756149737 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1c8c005e00 con 0x7f1ca010d280 2024-03-21T10:37:28.719 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:28.720+0000 7f1ca6800700 1 -- 172.21.15.153:0/756149737 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ca01a0830 con 0x7f1ca010d280 2024-03-21T10:37:28.719 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:28.720+0000 7f1c97e00700 1 -- 172.21.15.153:0/756149737 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1c8c017400 con 0x7f1ca010d280 2024-03-21T10:37:28.719 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:28.720+0000 7f1ca6800700 1 -- 172.21.15.153:0/756149737 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ca01a0da0 con 0x7f1ca010d280 2024-03-21T10:37:28.719 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:28.721+0000 7f1ca6800700 1 -- 172.21.15.153:0/756149737 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ca010f6c0 con 0x7f1ca010d280 2024-03-21T10:37:28.721 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:28.723+0000 7f1c97e00700 1 -- 172.21.15.153:0/756149737 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f1c8c00f460 con 0x7f1ca010d280 2024-03-21T10:37:28.722 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:28.724+0000 7f1c97e00700 1 --2- 172.21.15.153:0/756149737 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f1c8406e460 0x7f1c84070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:37:28.722 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:28.724+0000 7f1c97e00700 1 -- 172.21.15.153:0/756149737 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f1c8c08fa80 con 0x7f1ca010d280 2024-03-21T10:37:28.722 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:28.724+0000 7f1c9f400700 1 --2- 172.21.15.153:0/756149737 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f1c8406e460 0x7f1c84070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:37:28.722 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:28.725+0000 7f1c9f400700 1 --2- 172.21.15.153:0/756149737 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f1c8406e460 0x7f1c84070920 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f1c90005950 tx=0x7f1c900058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:37:28.726 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:28.728+0000 7f1c97e00700 1 -- 172.21.15.153:0/756149737 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f1c8c058aa0 con 0x7f1ca010d280 2024-03-21T10:37:28.984 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:28.987+0000 7f1ca6800700 1 -- 172.21.15.153:0/756149737 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f1ca004ea90 con 0x7f1ca010d280 2024-03-21T10:37:28.985 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:28.988+0000 7f1c97e00700 1 -- 172.21.15.153:0/756149737 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1876 (secure 0 0 0) 0x7f1c8c05c160 con 0x7f1ca010d280 2024-03-21T10:37:28.985 INFO:teuthology.orchestra.run.smithi153.stdout:e32 2024-03-21T10:37:28.985 INFO:teuthology.orchestra.run.smithi153.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-21T10:37:28.985 INFO:teuthology.orchestra.run.smithi153.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:37:28.986 INFO:teuthology.orchestra.run.smithi153.stdout:legacy client fscid: 1 2024-03-21T10:37:28.986 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:37:28.986 INFO:teuthology.orchestra.run.smithi153.stdout:Filesystem 'cephfs' (1) 2024-03-21T10:37:28.986 INFO:teuthology.orchestra.run.smithi153.stdout:fs_name cephfs 2024-03-21T10:37:28.986 INFO:teuthology.orchestra.run.smithi153.stdout:epoch 23 2024-03-21T10:37:28.986 INFO:teuthology.orchestra.run.smithi153.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-21T10:37:28.986 INFO:teuthology.orchestra.run.smithi153.stdout:created 2024-03-21T10:04:27.086576+0000 2024-03-21T10:37:28.986 INFO:teuthology.orchestra.run.smithi153.stdout:modified 2024-03-21T10:19:05.022092+0000 2024-03-21T10:37:28.986 INFO:teuthology.orchestra.run.smithi153.stdout:tableserver 0 2024-03-21T10:37:28.986 INFO:teuthology.orchestra.run.smithi153.stdout:root 0 2024-03-21T10:37:28.986 INFO:teuthology.orchestra.run.smithi153.stdout:session_timeout 60 2024-03-21T10:37:28.986 INFO:teuthology.orchestra.run.smithi153.stdout:session_autoclose 300 2024-03-21T10:37:28.987 INFO:teuthology.orchestra.run.smithi153.stdout:max_file_size 1099511627776 2024-03-21T10:37:28.987 INFO:teuthology.orchestra.run.smithi153.stdout:max_xattr_size 65536 2024-03-21T10:37:28.987 INFO:teuthology.orchestra.run.smithi153.stdout:required_client_features {} 2024-03-21T10:37:28.987 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure 0 2024-03-21T10:37:28.987 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure_osd_epoch 94 2024-03-21T10:37:28.987 INFO:teuthology.orchestra.run.smithi153.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:37:28.987 INFO:teuthology.orchestra.run.smithi153.stdout:max_mds 2 2024-03-21T10:37:28.987 INFO:teuthology.orchestra.run.smithi153.stdout:in 0,1 2024-03-21T10:37:28.987 INFO:teuthology.orchestra.run.smithi153.stdout:up {} 2024-03-21T10:37:28.987 INFO:teuthology.orchestra.run.smithi153.stdout:failed 0,1 2024-03-21T10:37:28.987 INFO:teuthology.orchestra.run.smithi153.stdout:damaged 2024-03-21T10:37:28.987 INFO:teuthology.orchestra.run.smithi153.stdout:stopped 2024-03-21T10:37:28.987 INFO:teuthology.orchestra.run.smithi153.stdout:data_pools [3] 2024-03-21T10:37:28.988 INFO:teuthology.orchestra.run.smithi153.stdout:metadata_pool 2 2024-03-21T10:37:28.988 INFO:teuthology.orchestra.run.smithi153.stdout:inline_data enabled 2024-03-21T10:37:28.988 INFO:teuthology.orchestra.run.smithi153.stdout:balancer 2024-03-21T10:37:28.988 INFO:teuthology.orchestra.run.smithi153.stdout:bal_rank_mask -1 2024-03-21T10:37:28.988 INFO:teuthology.orchestra.run.smithi153.stdout:standby_count_wanted 1 2024-03-21T10:37:28.988 INFO:teuthology.orchestra.run.smithi153.stdout:qdb_cluster leader: 0 members: 2024-03-21T10:37:28.988 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:37:28.988 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:37:28.988 INFO:teuthology.orchestra.run.smithi153.stdout:Standby daemons: 2024-03-21T10:37:28.988 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:37:28.988 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.zipckr{-1:34442} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6826/1103846295,v1:172.21.15.153:6827/1103846295] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:37:28.988 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.iuyluj{-1:34474} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6828/1738630832,v1:172.21.15.153:6829/1738630832] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:37:28.989 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.yyimst{-1:44313} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.174:6824/4127786797,v1:172.21.15.174:6825/4127786797] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:37:28.989 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.nhozag{-1:44687} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.174:6826/2767517845,v1:172.21.15.174:6827/2767517845] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:37:28.989 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:28.990+0000 7f1ca6800700 1 -- 172.21.15.153:0/756149737 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f1c8406e460 msgr2=0x7f1c84070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:37:28.989 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:28.991+0000 7f1ca6800700 1 --2- 172.21.15.153:0/756149737 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f1c8406e460 0x7f1c84070920 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f1c90005950 tx=0x7f1c900058e0 comp rx=0 tx=0).stop 2024-03-21T10:37:28.990 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:28.991+0000 7f1ca6800700 1 -- 172.21.15.153:0/756149737 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1ca010d280 msgr2=0x7f1ca019d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:37:28.990 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:28.991+0000 7f1ca6800700 1 --2- 172.21.15.153:0/756149737 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1ca010d280 0x7f1ca019d1f0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c0092e0 tx=0x7f1c8c00be20 comp rx=0 tx=0).stop 2024-03-21T10:37:28.990 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:28.991+0000 7f1ca6800700 1 -- 172.21.15.153:0/756149737 shutdown_connections 2024-03-21T10:37:28.990 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:28.991+0000 7f1ca6800700 1 --2- 172.21.15.153:0/756149737 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1ca01071d0 0x7f1ca019ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:37:28.990 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:28.991+0000 7f1ca6800700 1 --2- 172.21.15.153:0/756149737 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f1c8406e460 0x7f1c84070920 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:37:28.990 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:28.991+0000 7f1ca6800700 1 --2- 172.21.15.153:0/756149737 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1ca010d280 0x7f1ca019d1f0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:37:28.990 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:28.991+0000 7f1ca6800700 1 -- 172.21.15.153:0/756149737 >> 172.21.15.153:0/756149737 conn(0x7f1ca00769e0 msgr2=0x7f1ca0078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:37:28.990 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:28.991+0000 7f1ca6800700 1 -- 172.21.15.153:0/756149737 shutdown_connections 2024-03-21T10:37:28.990 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:28.991+0000 7f1ca6800700 1 -- 172.21.15.153:0/756149737 wait complete. 2024-03-21T10:37:28.990 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 32 2024-03-21T10:37:29.155 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:29.155+0000 7f0248000700 1 -- 172.21.15.153:0/1616499047 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0240073a80 msgr2=0x7f024010cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:37:29.155 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:29.155+0000 7f0248000700 1 --2- 172.21.15.153:0/1616499047 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0240073a80 0x7f024010cec0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f022c009910 tx=0x7f022c009c20 comp rx=0 tx=0).stop 2024-03-21T10:37:29.155 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:29.155+0000 7f0248000700 1 -- 172.21.15.153:0/1616499047 shutdown_connections 2024-03-21T10:37:29.155 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:29.155+0000 7f0248000700 1 --2- 172.21.15.153:0/1616499047 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0240073a80 0x7f024010cec0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:37:29.155 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:29.155+0000 7f0248000700 1 --2- 172.21.15.153:0/1616499047 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0240073160 0x7f0240073540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:37:29.155 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:29.155+0000 7f0248000700 1 -- 172.21.15.153:0/1616499047 >> 172.21.15.153:0/1616499047 conn(0x7f02400fc650 msgr2=0x7f02400fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:37:29.156 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:29.156+0000 7f0248000700 1 -- 172.21.15.153:0/1616499047 shutdown_connections 2024-03-21T10:37:29.156 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:29.156+0000 7f0248000700 1 -- 172.21.15.153:0/1616499047 wait complete. 2024-03-21T10:37:29.156 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:29.157+0000 7f0248000700 1 Processor -- start 2024-03-21T10:37:29.156 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:29.157+0000 7f0248000700 1 -- start start 2024-03-21T10:37:29.156 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:29.157+0000 7f0248000700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0240073160 0x7f0240198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:37:29.156 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:29.157+0000 7f0248000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0240073a80 0x7f0240198eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:37:29.156 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:29.157+0000 7f0248000700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0240199540 con 0x7f0240073a80 2024-03-21T10:37:29.156 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:29.157+0000 7f0248000700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f024019d350 con 0x7f0240073160 2024-03-21T10:37:29.156 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:29.158+0000 7f0244c00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0240073a80 0x7f0240198eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:37:29.156 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:29.158+0000 7f0244c00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0240073a80 0x7f0240198eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:53406/0 (socket says 172.21.15.153:53406) 2024-03-21T10:37:29.156 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:29.158+0000 7f0244c00700 1 -- 172.21.15.153:0/4229647703 learned_addr learned my addr 172.21.15.153:0/4229647703 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:37:29.156 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:29.158+0000 7f0245600700 1 --2- 172.21.15.153:0/4229647703 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0240073160 0x7f0240198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:37:29.156 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:29.158+0000 7f0244c00700 1 -- 172.21.15.153:0/4229647703 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0240073160 msgr2=0x7f0240198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:37:29.156 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:29.158+0000 7f0244c00700 1 --2- 172.21.15.153:0/4229647703 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0240073160 0x7f0240198970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:37:29.157 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:29.158+0000 7f0244c00700 1 -- 172.21.15.153:0/4229647703 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f022c0095f0 con 0x7f0240073a80 2024-03-21T10:37:29.157 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:29.158+0000 7f0244c00700 1 --2- 172.21.15.153:0/4229647703 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0240073a80 0x7f0240198eb0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f022c000c00 tx=0x7f022c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:37:29.157 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:29.159+0000 7f023e000700 1 -- 172.21.15.153:0/4229647703 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f022c01d070 con 0x7f0240073a80 2024-03-21T10:37:29.157 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:29.159+0000 7f023e000700 1 -- 172.21.15.153:0/4229647703 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f022c00f460 con 0x7f0240073a80 2024-03-21T10:37:29.158 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:29.159+0000 7f023e000700 1 -- 172.21.15.153:0/4229647703 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f022c005e20 con 0x7f0240073a80 2024-03-21T10:37:29.158 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:29.159+0000 7f0248000700 1 -- 172.21.15.153:0/4229647703 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f024019d5d0 con 0x7f0240073a80 2024-03-21T10:37:29.158 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:29.159+0000 7f0248000700 1 -- 172.21.15.153:0/4229647703 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f024019dac0 con 0x7f0240073a80 2024-03-21T10:37:29.158 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:29.160+0000 7f0248000700 1 -- 172.21.15.153:0/4229647703 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f024010a5c0 con 0x7f0240073a80 2024-03-21T10:37:29.158 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:29.161+0000 7f023e000700 1 -- 172.21.15.153:0/4229647703 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f022c00f5d0 con 0x7f0240073a80 2024-03-21T10:37:29.159 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:29.162+0000 7f023e000700 1 --2- 172.21.15.153:0/4229647703 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f022806e460 0x7f0228070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:37:29.159 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:29.162+0000 7f023e000700 1 -- 172.21.15.153:0/4229647703 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f022c08f900 con 0x7f0240073a80 2024-03-21T10:37:29.159 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:29.162+0000 7f0245600700 1 --2- 172.21.15.153:0/4229647703 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f022806e460 0x7f0228070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:37:29.160 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:29.163+0000 7f0245600700 1 --2- 172.21.15.153:0/4229647703 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f022806e460 0x7f0228070920 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f0234005950 tx=0x7f02340058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:37:29.163 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:29.167+0000 7f023e000700 1 -- 172.21.15.153:0/4229647703 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f022c05c630 con 0x7f0240073a80 2024-03-21T10:37:29.303 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:37:29 smithi153 ceph-mon[113733]: from='client.35382 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:37:29.303 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:37:29 smithi153 ceph-mon[113733]: pgmap v770: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:37:29.303 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:37:29 smithi153 ceph-mon[113733]: from='client.35386 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:37:29.303 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:37:29 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/245729578' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:37:29.303 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:37:29 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/756149737' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:37:29.406 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:29.409+0000 7f0248000700 1 -- 172.21.15.153:0/4229647703 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f024019ddf0 con 0x7f022806e460 2024-03-21T10:37:29.409 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:29.411+0000 7f023e000700 1 -- 172.21.15.153:0/4229647703 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f024019ddf0 con 0x7f022806e460 2024-03-21T10:37:29.409 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:37:29.409 INFO:teuthology.orchestra.run.smithi153.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232", 2024-03-21T10:37:29.409 INFO:teuthology.orchestra.run.smithi153.stdout: "in_progress": true, 2024-03-21T10:37:29.409 INFO:teuthology.orchestra.run.smithi153.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-21T10:37:29.409 INFO:teuthology.orchestra.run.smithi153.stdout: "services_complete": [ 2024-03-21T10:37:29.409 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr", 2024-03-21T10:37:29.409 INFO:teuthology.orchestra.run.smithi153.stdout: "mon", 2024-03-21T10:37:29.409 INFO:teuthology.orchestra.run.smithi153.stdout: "mds", 2024-03-21T10:37:29.409 INFO:teuthology.orchestra.run.smithi153.stdout: "crash", 2024-03-21T10:37:29.410 INFO:teuthology.orchestra.run.smithi153.stdout: "osd" 2024-03-21T10:37:29.410 INFO:teuthology.orchestra.run.smithi153.stdout: ], 2024-03-21T10:37:29.410 INFO:teuthology.orchestra.run.smithi153.stdout: "progress": "16/23 daemons upgraded", 2024-03-21T10:37:29.410 INFO:teuthology.orchestra.run.smithi153.stdout: "message": "Currently upgrading mds daemons", 2024-03-21T10:37:29.410 INFO:teuthology.orchestra.run.smithi153.stdout: "is_paused": false 2024-03-21T10:37:29.410 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:37:29.412 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:29.414+0000 7f0248000700 1 -- 172.21.15.153:0/4229647703 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f022806e460 msgr2=0x7f0228070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:37:29.412 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:29.414+0000 7f0248000700 1 --2- 172.21.15.153:0/4229647703 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f022806e460 0x7f0228070920 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f0234005950 tx=0x7f02340058e0 comp rx=0 tx=0).stop 2024-03-21T10:37:29.412 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:29.414+0000 7f0248000700 1 -- 172.21.15.153:0/4229647703 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0240073a80 msgr2=0x7f0240198eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:37:29.412 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:29.414+0000 7f0248000700 1 --2- 172.21.15.153:0/4229647703 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0240073a80 0x7f0240198eb0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f022c000c00 tx=0x7f022c00be20 comp rx=0 tx=0).stop 2024-03-21T10:37:29.412 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:29.414+0000 7f0248000700 1 -- 172.21.15.153:0/4229647703 shutdown_connections 2024-03-21T10:37:29.412 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:29.415+0000 7f0248000700 1 --2- 172.21.15.153:0/4229647703 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0240073160 0x7f0240198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:37:29.412 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:29.415+0000 7f0248000700 1 --2- 172.21.15.153:0/4229647703 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f022806e460 0x7f0228070920 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:37:29.412 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:29.415+0000 7f0248000700 1 --2- 172.21.15.153:0/4229647703 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0240073a80 0x7f0240198eb0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:37:29.412 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:29.415+0000 7f0248000700 1 -- 172.21.15.153:0/4229647703 >> 172.21.15.153:0/4229647703 conn(0x7f02400fc650 msgr2=0x7f0240107700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:37:29.413 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:29.415+0000 7f0248000700 1 -- 172.21.15.153:0/4229647703 shutdown_connections 2024-03-21T10:37:29.413 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:29.415+0000 7f0248000700 1 -- 172.21.15.153:0/4229647703 wait complete. 2024-03-21T10:37:29.448 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:37:29 smithi174 ceph-mon[88004]: from='client.35382 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:37:29.448 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:37:29 smithi174 ceph-mon[88004]: pgmap v770: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:37:29.448 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:37:29 smithi174 ceph-mon[88004]: from='client.35386 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:37:29.448 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:37:29 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/245729578' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:37:29.448 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:37:29 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/756149737' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:37:29.564 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:29.567+0000 7fbd76c00700 1 -- 172.21.15.153:0/2698648877 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbd700731c0 msgr2=0x7fbd700735a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:37:29.564 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:29.567+0000 7fbd76c00700 1 --2- 172.21.15.153:0/2698648877 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbd700731c0 0x7fbd700735a0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fbd60009910 tx=0x7fbd60009c20 comp rx=0 tx=0).stop 2024-03-21T10:37:29.565 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:29.567+0000 7fbd76c00700 1 -- 172.21.15.153:0/2698648877 shutdown_connections 2024-03-21T10:37:29.565 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:29.567+0000 7fbd76c00700 1 --2- 172.21.15.153:0/2698648877 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbd70073ae0 0x7fbd7010ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:37:29.565 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:29.567+0000 7fbd76c00700 1 --2- 172.21.15.153:0/2698648877 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbd700731c0 0x7fbd700735a0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:37:29.565 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:29.567+0000 7fbd76c00700 1 -- 172.21.15.153:0/2698648877 >> 172.21.15.153:0/2698648877 conn(0x7fbd700fc630 msgr2=0x7fbd700fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:37:29.565 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:29.567+0000 7fbd76c00700 1 -- 172.21.15.153:0/2698648877 shutdown_connections 2024-03-21T10:37:29.565 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:29.567+0000 7fbd76c00700 1 -- 172.21.15.153:0/2698648877 wait complete. 2024-03-21T10:37:29.566 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:29.568+0000 7fbd76c00700 1 Processor -- start 2024-03-21T10:37:29.566 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:29.568+0000 7fbd76c00700 1 -- start start 2024-03-21T10:37:29.566 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:29.568+0000 7fbd76c00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbd700731c0 0x7fbd70198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:37:29.567 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:29.569+0000 7fbd76c00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbd70073ae0 0x7fbd70198e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:37:29.567 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:29.569+0000 7fbd76c00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd70199520 con 0x7fbd70073ae0 2024-03-21T10:37:29.567 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:29.569+0000 7fbd76c00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd7019d330 con 0x7fbd700731c0 2024-03-21T10:37:29.567 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:29.569+0000 7fbd67e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbd70073ae0 0x7fbd70198e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:37:29.567 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:29.569+0000 7fbd67e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbd70073ae0 0x7fbd70198e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:53422/0 (socket says 172.21.15.153:53422) 2024-03-21T10:37:29.567 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:29.569+0000 7fbd67e00700 1 -- 172.21.15.153:0/1997232007 learned_addr learned my addr 172.21.15.153:0/1997232007 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:37:29.567 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:29.569+0000 7fbd6fe00700 1 --2- 172.21.15.153:0/1997232007 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbd700731c0 0x7fbd70198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:37:29.567 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:29.569+0000 7fbd67e00700 1 -- 172.21.15.153:0/1997232007 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbd700731c0 msgr2=0x7fbd70198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:37:29.567 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:29.569+0000 7fbd67e00700 1 --2- 172.21.15.153:0/1997232007 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbd700731c0 0x7fbd70198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:37:29.567 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:29.569+0000 7fbd67e00700 1 -- 172.21.15.153:0/1997232007 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd600095f0 con 0x7fbd70073ae0 2024-03-21T10:37:29.567 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:29.569+0000 7fbd6fe00700 1 --2- 172.21.15.153:0/1997232007 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbd700731c0 0x7fbd70198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:37:29.567 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:29.570+0000 7fbd67e00700 1 --2- 172.21.15.153:0/1997232007 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbd70073ae0 0x7fbd70198e90 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fbd5800c8a0 tx=0x7fbd5800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:37:29.568 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:29.570+0000 7fbd6d600700 1 -- 172.21.15.153:0/1997232007 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbd5800cea0 con 0x7fbd70073ae0 2024-03-21T10:37:29.568 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:29.570+0000 7fbd6d600700 1 -- 172.21.15.153:0/1997232007 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbd58004830 con 0x7fbd70073ae0 2024-03-21T10:37:29.568 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:29.570+0000 7fbd6d600700 1 -- 172.21.15.153:0/1997232007 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbd580055b0 con 0x7fbd70073ae0 2024-03-21T10:37:29.568 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:29.570+0000 7fbd76c00700 1 -- 172.21.15.153:0/1997232007 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd7019d610 con 0x7fbd70073ae0 2024-03-21T10:37:29.568 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:29.570+0000 7fbd76c00700 1 -- 172.21.15.153:0/1997232007 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd7019db60 con 0x7fbd70073ae0 2024-03-21T10:37:29.568 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:29.571+0000 7fbd76c00700 1 -- 172.21.15.153:0/1997232007 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd7010a540 con 0x7fbd70073ae0 2024-03-21T10:37:29.569 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:29.572+0000 7fbd6d600700 1 -- 172.21.15.153:0/1997232007 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fbd580049a0 con 0x7fbd70073ae0 2024-03-21T10:37:29.570 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:29.573+0000 7fbd6d600700 1 --2- 172.21.15.153:0/1997232007 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fbd5406e4b0 0x7fbd54070970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:37:29.570 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:29.573+0000 7fbd6d600700 1 -- 172.21.15.153:0/1997232007 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fbd5808e270 con 0x7fbd70073ae0 2024-03-21T10:37:29.571 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:29.573+0000 7fbd6fe00700 1 --2- 172.21.15.153:0/1997232007 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fbd5406e4b0 0x7fbd54070970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:37:29.571 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:29.574+0000 7fbd6fe00700 1 --2- 172.21.15.153:0/1997232007 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fbd5406e4b0 0x7fbd54070970 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fbd60009fd0 tx=0x7fbd6000b560 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:37:29.574 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:29.578+0000 7fbd6d600700 1 -- 172.21.15.153:0/1997232007 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fbd5805adc0 con 0x7fbd70073ae0 2024-03-21T10:37:29.862 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:29.864+0000 7fbd76c00700 1 -- 172.21.15.153:0/1997232007 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fbd7004ea90 con 0x7fbd70073ae0 2024-03-21T10:37:29.862 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:29.865+0000 7fbd6d600700 1 -- 172.21.15.153:0/1997232007 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4493 (secure 0 0 0) 0x7fbd58004c50 con 0x7fbd70073ae0 2024-03-21T10:37:29.863 INFO:teuthology.orchestra.run.smithi153.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-03-21T10:37:29.863 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-03-21T10:37:29.863 INFO:teuthology.orchestra.run.smithi153.stdout: host smithi174 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi174/config 2024-03-21T10:37:29.863 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/podman: timeout after 895 seconds 2024-03-21T10:37:29.863 INFO:teuthology.orchestra.run.smithi153.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 -e NODE_NAME=smithi174 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/run/ceph:z -v /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpd910r3_i:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 inventory --format=json-pretty --filter-for-batch 2024-03-21T10:37:29.863 INFO:teuthology.orchestra.run.smithi153.stdout:Traceback (most recent call last): 2024-03-21T10:37:29.864 INFO:teuthology.orchestra.run.smithi153.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-03-21T10:37:29.864 INFO:teuthology.orchestra.run.smithi153.stdout: return _run_code(code, main_globals, None, 2024-03-21T10:37:29.864 INFO:teuthology.orchestra.run.smithi153.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-03-21T10:37:29.864 INFO:teuthology.orchestra.run.smithi153.stdout: exec(code, run_globals) 2024-03-21T10:37:29.864 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 5606, in 2024-03-21T10:37:29.864 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 5594, in main 2024-03-21T10:37:29.864 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 406, in _infer_config 2024-03-21T10:37:29.864 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 321, in _infer_fsid 2024-03-21T10:37:29.864 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 434, in _infer_image 2024-03-21T10:37:29.864 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 308, in _validate_fsid 2024-03-21T10:37:29.864 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 3357, in command_ceph_volume 2024-03-21T10:37:29.864 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-03-21T10:37:29.864 INFO:teuthology.orchestra.run.smithi153.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 -e NODE_NAME=smithi174 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/run/ceph:z -v /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpd910r3_i:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 inventory --format=json-pretty --filter-for-batch 2024-03-21T10:37:29.864 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-21T10:37:29.864 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is degraded 2024-03-21T10:37:29.864 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-21T10:37:29.865 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-21T10:37:29.865 INFO:teuthology.orchestra.run.smithi153.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-21T10:37:29.865 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-21T10:37:29.866 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:29.869+0000 7fbd76c00700 1 -- 172.21.15.153:0/1997232007 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fbd5406e4b0 msgr2=0x7fbd54070970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:37:29.866 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:29.869+0000 7fbd76c00700 1 --2- 172.21.15.153:0/1997232007 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fbd5406e4b0 0x7fbd54070970 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fbd60009fd0 tx=0x7fbd6000b560 comp rx=0 tx=0).stop 2024-03-21T10:37:29.866 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:29.869+0000 7fbd76c00700 1 -- 172.21.15.153:0/1997232007 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbd70073ae0 msgr2=0x7fbd70198e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:37:29.866 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:29.869+0000 7fbd76c00700 1 --2- 172.21.15.153:0/1997232007 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbd70073ae0 0x7fbd70198e90 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fbd5800c8a0 tx=0x7fbd5800cc60 comp rx=0 tx=0).stop 2024-03-21T10:37:29.866 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:29.869+0000 7fbd76c00700 1 -- 172.21.15.153:0/1997232007 shutdown_connections 2024-03-21T10:37:29.867 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:29.869+0000 7fbd76c00700 1 --2- 172.21.15.153:0/1997232007 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbd700731c0 0x7fbd70198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:37:29.867 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:29.869+0000 7fbd76c00700 1 --2- 172.21.15.153:0/1997232007 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fbd5406e4b0 0x7fbd54070970 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:37:29.867 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:29.869+0000 7fbd76c00700 1 --2- 172.21.15.153:0/1997232007 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbd70073ae0 0x7fbd70198e90 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:37:29.867 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:29.869+0000 7fbd76c00700 1 -- 172.21.15.153:0/1997232007 >> 172.21.15.153:0/1997232007 conn(0x7fbd700fc630 msgr2=0x7fbd70107680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:37:29.867 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:29.869+0000 7fbd76c00700 1 -- 172.21.15.153:0/1997232007 shutdown_connections 2024-03-21T10:37:29.867 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:37:29.869+0000 7fbd76c00700 1 -- 172.21.15.153:0/1997232007 wait complete. 2024-03-21T10:37:30.448 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:37:30 smithi174 ceph-mon[88004]: from='client.35398 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:37:30.448 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:37:30 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/1997232007' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:37:30.553 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:37:30 smithi153 ceph-mon[113733]: from='client.35398 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:37:30.553 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:37:30 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/1997232007' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:37:31.448 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:37:31 smithi174 ceph-mon[88004]: pgmap v771: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:37:31.553 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:37:31 smithi153 ceph-mon[113733]: pgmap v771: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:37:32.448 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:37:32 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:37:32.553 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:37:32 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:37:33.448 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:37:33 smithi174 ceph-mon[88004]: pgmap v772: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:37:33.497 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:37:33 smithi153 ceph-mon[113733]: pgmap v772: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:37:35.448 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:37:35 smithi174 ceph-mon[88004]: pgmap v773: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:37:35.553 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:37:35 smithi153 ceph-mon[113733]: pgmap v773: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:37:37.448 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:37:37 smithi174 ceph-mon[88004]: pgmap v774: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:37:37.553 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:37:37 smithi153 ceph-mon[113733]: pgmap v774: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:37:39.448 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:37:39 smithi174 ceph-mon[88004]: pgmap v775: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:37:39.553 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:37:39 smithi153 ceph-mon[113733]: pgmap v775: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:37:41.448 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:37:41 smithi174 ceph-mon[88004]: pgmap v776: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:37:41.553 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:37:41 smithi153 ceph-mon[113733]: pgmap v776: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:37:43.448 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:37:43 smithi174 ceph-mon[88004]: pgmap v777: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:37:43.553 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:37:43 smithi153 ceph-mon[113733]: pgmap v777: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:37:45.448 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:37:45 smithi174 ceph-mon[88004]: pgmap v778: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:37:45.553 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:37:45 smithi153 ceph-mon[113733]: pgmap v778: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:37:47.553 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:37:47 smithi153 ceph-mon[113733]: pgmap v779: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:37:47.553 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:37:47 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:37:47.698 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:37:47 smithi174 ceph-mon[88004]: pgmap v779: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:37:47.698 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:37:47 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:37:49.303 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:37:49 smithi153 ceph-mon[113733]: pgmap v780: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:37:49.698 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:37:49 smithi174 ceph-mon[88004]: pgmap v780: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:37:51.553 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:37:51 smithi153 ceph-mon[113733]: pgmap v781: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:37:51.698 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:37:51 smithi174 ceph-mon[88004]: pgmap v781: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:37:53.553 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:37:53 smithi153 ceph-mon[113733]: pgmap v782: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:37:53.698 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:37:53 smithi174 ceph-mon[88004]: pgmap v782: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:37:55.553 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:37:55 smithi153 ceph-mon[113733]: pgmap v783: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:37:55.698 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:37:55 smithi174 ceph-mon[88004]: pgmap v783: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:37:57.448 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:37:57 smithi174 ceph-mon[88004]: pgmap v784: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:37:57.553 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:37:57 smithi153 ceph-mon[113733]: pgmap v784: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:37:59.553 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:37:59 smithi153 ceph-mon[113733]: pgmap v785: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:37:59.698 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:37:59 smithi174 ceph-mon[88004]: pgmap v785: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:38:00.036 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.037+0000 7f01dfa00700 1 -- 172.21.15.153:0/2539800259 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f01d8101750 msgr2=0x7f01d81057a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:38:00.036 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.037+0000 7f01dfa00700 1 --2- 172.21.15.153:0/2539800259 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f01d8101750 0x7f01d81057a0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f01c8009910 tx=0x7f01c8009c20 comp rx=0 tx=0).stop 2024-03-21T10:38:00.036 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.037+0000 7f01dfa00700 1 -- 172.21.15.153:0/2539800259 shutdown_connections 2024-03-21T10:38:00.036 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.037+0000 7f01dfa00700 1 --2- 172.21.15.153:0/2539800259 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f01d8101750 0x7f01d81057a0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:38:00.036 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.037+0000 7f01dfa00700 1 --2- 172.21.15.153:0/2539800259 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f01d8100da0 0x7f01d8101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:38:00.036 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.037+0000 7f01dfa00700 1 -- 172.21.15.153:0/2539800259 >> 172.21.15.153:0/2539800259 conn(0x7f01d80fc630 msgr2=0x7f01d80fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:38:00.036 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.037+0000 7f01dfa00700 1 -- 172.21.15.153:0/2539800259 shutdown_connections 2024-03-21T10:38:00.036 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.037+0000 7f01dfa00700 1 -- 172.21.15.153:0/2539800259 wait complete. 2024-03-21T10:38:00.037 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.038+0000 7f01dfa00700 1 Processor -- start 2024-03-21T10:38:00.037 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.039+0000 7f01dfa00700 1 -- start start 2024-03-21T10:38:00.037 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.039+0000 7f01dfa00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f01d8100da0 0x7f01d819c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:38:00.037 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.039+0000 7f01dfa00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f01d8101750 0x7f01d819ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:38:00.038 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.039+0000 7f01dfa00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01d819d0d0 con 0x7f01d8100da0 2024-03-21T10:38:00.038 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.039+0000 7f01dfa00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01d8196580 con 0x7f01d8101750 2024-03-21T10:38:00.038 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.039+0000 7f01dd000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f01d8100da0 0x7f01d819c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:38:00.038 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.039+0000 7f01dd000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f01d8100da0 0x7f01d819c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:34890/0 (socket says 172.21.15.153:34890) 2024-03-21T10:38:00.038 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.039+0000 7f01dd000700 1 -- 172.21.15.153:0/4225904583 learned_addr learned my addr 172.21.15.153:0/4225904583 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:38:00.039 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.040+0000 7f01cfe00700 1 --2- 172.21.15.153:0/4225904583 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f01d8101750 0x7f01d819ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:38:00.039 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.040+0000 7f01dd000700 1 -- 172.21.15.153:0/4225904583 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f01d8101750 msgr2=0x7f01d819ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:38:00.039 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.040+0000 7f01dd000700 1 --2- 172.21.15.153:0/4225904583 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f01d8101750 0x7f01d819ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:38:00.039 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.040+0000 7f01dd000700 1 -- 172.21.15.153:0/4225904583 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01c80095f0 con 0x7f01d8100da0 2024-03-21T10:38:00.039 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.040+0000 7f01dd000700 1 --2- 172.21.15.153:0/4225904583 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f01d8100da0 0x7f01d819c500 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f01c000d8d0 tx=0x7f01c000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:38:00.039 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.040+0000 7f01d6000700 1 -- 172.21.15.153:0/4225904583 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f01c0009940 con 0x7f01d8100da0 2024-03-21T10:38:00.039 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.040+0000 7f01d6000700 1 -- 172.21.15.153:0/4225904583 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f01c0010460 con 0x7f01d8100da0 2024-03-21T10:38:00.039 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.040+0000 7f01d6000700 1 -- 172.21.15.153:0/4225904583 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f01c000f5f0 con 0x7f01d8100da0 2024-03-21T10:38:00.039 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.040+0000 7f01dfa00700 1 -- 172.21.15.153:0/4225904583 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01d8196860 con 0x7f01d8100da0 2024-03-21T10:38:00.040 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.041+0000 7f01dfa00700 1 -- 172.21.15.153:0/4225904583 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01d8196d60 con 0x7f01d8100da0 2024-03-21T10:38:00.040 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.042+0000 7f01dfa00700 1 -- 172.21.15.153:0/4225904583 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01d81091a0 con 0x7f01d8100da0 2024-03-21T10:38:00.040 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.043+0000 7f01d6000700 1 -- 172.21.15.153:0/4225904583 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f01c00105d0 con 0x7f01d8100da0 2024-03-21T10:38:00.042 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.044+0000 7f01d6000700 1 --2- 172.21.15.153:0/4225904583 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f01b806e460 0x7f01b8070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:38:00.042 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.044+0000 7f01d6000700 1 -- 172.21.15.153:0/4225904583 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f01c008e5b0 con 0x7f01d8100da0 2024-03-21T10:38:00.042 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.044+0000 7f01cfe00700 1 --2- 172.21.15.153:0/4225904583 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f01b806e460 0x7f01b8070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:38:00.042 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.045+0000 7f01cfe00700 1 --2- 172.21.15.153:0/4225904583 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f01b806e460 0x7f01b8070920 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f01c8000c00 tx=0x7f01c8005fd0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:38:00.047 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.049+0000 7f01d6000700 1 -- 172.21.15.153:0/4225904583 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f01c005b100 con 0x7f01d8100da0 2024-03-21T10:38:00.288 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.289+0000 7f01dfa00700 1 -- 172.21.15.153:0/4225904583 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f01d81970a0 con 0x7f01b806e460 2024-03-21T10:38:00.291 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.293+0000 7f01d6000700 1 -- 172.21.15.153:0/4225904583 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f01d81970a0 con 0x7f01b806e460 2024-03-21T10:38:00.295 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.296+0000 7f01dfa00700 1 -- 172.21.15.153:0/4225904583 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f01b806e460 msgr2=0x7f01b8070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:38:00.295 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.296+0000 7f01dfa00700 1 --2- 172.21.15.153:0/4225904583 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f01b806e460 0x7f01b8070920 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f01c8000c00 tx=0x7f01c8005fd0 comp rx=0 tx=0).stop 2024-03-21T10:38:00.295 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.296+0000 7f01dfa00700 1 -- 172.21.15.153:0/4225904583 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f01d8100da0 msgr2=0x7f01d819c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:38:00.295 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.296+0000 7f01dfa00700 1 --2- 172.21.15.153:0/4225904583 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f01d8100da0 0x7f01d819c500 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f01c000d8d0 tx=0x7f01c000dc90 comp rx=0 tx=0).stop 2024-03-21T10:38:00.295 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.297+0000 7f01dfa00700 1 -- 172.21.15.153:0/4225904583 shutdown_connections 2024-03-21T10:38:00.295 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.297+0000 7f01dfa00700 1 --2- 172.21.15.153:0/4225904583 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f01b806e460 0x7f01b8070920 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:38:00.295 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.297+0000 7f01dfa00700 1 --2- 172.21.15.153:0/4225904583 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f01d8100da0 0x7f01d819c500 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:38:00.295 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.297+0000 7f01dfa00700 1 --2- 172.21.15.153:0/4225904583 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f01d8101750 0x7f01d819ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:38:00.295 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.297+0000 7f01dfa00700 1 -- 172.21.15.153:0/4225904583 >> 172.21.15.153:0/4225904583 conn(0x7f01d80fc630 msgr2=0x7f01d80fea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:38:00.296 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.297+0000 7f01dfa00700 1 -- 172.21.15.153:0/4225904583 shutdown_connections 2024-03-21T10:38:00.296 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.297+0000 7f01dfa00700 1 -- 172.21.15.153:0/4225904583 wait complete. 2024-03-21T10:38:00.311 INFO:teuthology.orchestra.run.smithi153.stdout:true 2024-03-21T10:38:00.454 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.455+0000 7f35a5e00700 1 -- 172.21.15.153:0/1370779011 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f35a01039b0 msgr2=0x7f35a0107a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:38:00.454 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.455+0000 7f35a5e00700 1 --2- 172.21.15.153:0/1370779011 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f35a01039b0 0x7f35a0107a00 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f3588009910 tx=0x7f3588009c20 comp rx=0 tx=0).stop 2024-03-21T10:38:00.454 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.456+0000 7f35a5e00700 1 -- 172.21.15.153:0/1370779011 shutdown_connections 2024-03-21T10:38:00.454 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.456+0000 7f35a5e00700 1 --2- 172.21.15.153:0/1370779011 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f35a01039b0 0x7f35a0107a00 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:38:00.454 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.456+0000 7f35a5e00700 1 --2- 172.21.15.153:0/1370779011 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f35a0103000 0x7f35a01033e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:38:00.454 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.456+0000 7f35a5e00700 1 -- 172.21.15.153:0/1370779011 >> 172.21.15.153:0/1370779011 conn(0x7f35a00fe8d0 msgr2=0x7f35a0100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:38:00.454 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.456+0000 7f35a5e00700 1 -- 172.21.15.153:0/1370779011 shutdown_connections 2024-03-21T10:38:00.454 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.456+0000 7f35a5e00700 1 -- 172.21.15.153:0/1370779011 wait complete. 2024-03-21T10:38:00.455 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.457+0000 7f35a5e00700 1 Processor -- start 2024-03-21T10:38:00.455 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.457+0000 7f35a5e00700 1 -- start start 2024-03-21T10:38:00.456 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.457+0000 7f35a5e00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f35a0103000 0x7f35a019e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:38:00.456 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.457+0000 7f35a5e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f35a01039b0 0x7f35a019ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:38:00.456 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.457+0000 7f35a5e00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35a019f390 con 0x7f35a01039b0 2024-03-21T10:38:00.456 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.457+0000 7f35a5e00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35a0198840 con 0x7f35a0103000 2024-03-21T10:38:00.456 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.458+0000 7f3597e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f35a01039b0 0x7f35a019ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:38:00.456 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.458+0000 7f3597e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f35a01039b0 0x7f35a019ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:34904/0 (socket says 172.21.15.153:34904) 2024-03-21T10:38:00.456 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.458+0000 7f3597e00700 1 -- 172.21.15.153:0/2143248887 learned_addr learned my addr 172.21.15.153:0/2143248887 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:38:00.457 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.458+0000 7f359ea00700 1 --2- 172.21.15.153:0/2143248887 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f35a0103000 0x7f35a019e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:38:00.457 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.458+0000 7f3597e00700 1 -- 172.21.15.153:0/2143248887 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f35a0103000 msgr2=0x7f35a019e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:38:00.457 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.458+0000 7f3597e00700 1 --2- 172.21.15.153:0/2143248887 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f35a0103000 0x7f35a019e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:38:00.457 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.458+0000 7f3597e00700 1 -- 172.21.15.153:0/2143248887 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35880095f0 con 0x7f35a01039b0 2024-03-21T10:38:00.457 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.459+0000 7f3597e00700 1 --2- 172.21.15.153:0/2143248887 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f35a01039b0 0x7f35a019ed00 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f358800bd70 tx=0x7f358800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:38:00.458 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.459+0000 7f35a4a00700 1 -- 172.21.15.153:0/2143248887 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f358801d070 con 0x7f35a01039b0 2024-03-21T10:38:00.458 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.459+0000 7f35a4a00700 1 -- 172.21.15.153:0/2143248887 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3588005dc0 con 0x7f35a01039b0 2024-03-21T10:38:00.458 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.459+0000 7f35a5e00700 1 -- 172.21.15.153:0/2143248887 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35a0198ac0 con 0x7f35a01039b0 2024-03-21T10:38:00.458 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.459+0000 7f35a4a00700 1 -- 172.21.15.153:0/2143248887 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3588017400 con 0x7f35a01039b0 2024-03-21T10:38:00.458 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.459+0000 7f35a5e00700 1 -- 172.21.15.153:0/2143248887 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35a0198fb0 con 0x7f35a01039b0 2024-03-21T10:38:00.458 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.460+0000 7f35a5e00700 1 -- 172.21.15.153:0/2143248887 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35a010b400 con 0x7f35a01039b0 2024-03-21T10:38:00.459 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.461+0000 7f35a4a00700 1 -- 172.21.15.153:0/2143248887 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f358800f7f0 con 0x7f35a01039b0 2024-03-21T10:38:00.461 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.463+0000 7f35a4a00700 1 --2- 172.21.15.153:0/2143248887 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f358406e460 0x7f3584070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:38:00.461 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.463+0000 7f35a4a00700 1 -- 172.21.15.153:0/2143248887 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f358808f9c0 con 0x7f35a01039b0 2024-03-21T10:38:00.461 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.463+0000 7f359ea00700 1 --2- 172.21.15.153:0/2143248887 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f358406e460 0x7f3584070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:38:00.462 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.464+0000 7f359ea00700 1 --2- 172.21.15.153:0/2143248887 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f358406e460 0x7f3584070920 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f3590005950 tx=0x7f35900058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:38:00.466 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.468+0000 7f35a4a00700 1 -- 172.21.15.153:0/2143248887 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f358805c5b0 con 0x7f35a01039b0 2024-03-21T10:38:00.712 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.714+0000 7f35a5e00700 1 -- 172.21.15.153:0/2143248887 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f35a002cd80 con 0x7f358406e460 2024-03-21T10:38:00.715 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.717+0000 7f35a4a00700 1 -- 172.21.15.153:0/2143248887 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f35a002cd80 con 0x7f358406e460 2024-03-21T10:38:00.718 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.719+0000 7f35a5e00700 1 -- 172.21.15.153:0/2143248887 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f358406e460 msgr2=0x7f3584070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:38:00.718 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.719+0000 7f35a5e00700 1 --2- 172.21.15.153:0/2143248887 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f358406e460 0x7f3584070920 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f3590005950 tx=0x7f35900058e0 comp rx=0 tx=0).stop 2024-03-21T10:38:00.718 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.719+0000 7f35a5e00700 1 -- 172.21.15.153:0/2143248887 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f35a01039b0 msgr2=0x7f35a019ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:38:00.718 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.719+0000 7f35a5e00700 1 --2- 172.21.15.153:0/2143248887 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f35a01039b0 0x7f35a019ed00 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f358800bd70 tx=0x7f358800be50 comp rx=0 tx=0).stop 2024-03-21T10:38:00.718 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.720+0000 7f35a5e00700 1 -- 172.21.15.153:0/2143248887 shutdown_connections 2024-03-21T10:38:00.718 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.720+0000 7f35a5e00700 1 --2- 172.21.15.153:0/2143248887 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f35a0103000 0x7f35a019e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:38:00.719 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.720+0000 7f35a5e00700 1 --2- 172.21.15.153:0/2143248887 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f358406e460 0x7f3584070920 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:38:00.719 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.720+0000 7f35a5e00700 1 --2- 172.21.15.153:0/2143248887 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f35a01039b0 0x7f35a019ed00 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:38:00.719 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.720+0000 7f35a5e00700 1 -- 172.21.15.153:0/2143248887 >> 172.21.15.153:0/2143248887 conn(0x7f35a00fe8d0 msgr2=0x7f35a0100cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:38:00.719 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.720+0000 7f35a5e00700 1 -- 172.21.15.153:0/2143248887 shutdown_connections 2024-03-21T10:38:00.719 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.720+0000 7f35a5e00700 1 -- 172.21.15.153:0/2143248887 wait complete. 2024-03-21T10:38:00.877 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.878+0000 7f2a90a00700 1 -- 172.21.15.153:0/2376160439 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a8c068670 msgr2=0x7f2a8c068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:38:00.877 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.878+0000 7f2a90a00700 1 --2- 172.21.15.153:0/2376160439 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a8c068670 0x7f2a8c068a50 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c009910 tx=0x7f2a7c009c20 comp rx=0 tx=0).stop 2024-03-21T10:38:00.877 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.878+0000 7f2a90a00700 1 -- 172.21.15.153:0/2376160439 shutdown_connections 2024-03-21T10:38:00.877 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.878+0000 7f2a90a00700 1 --2- 172.21.15.153:0/2376160439 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a8c069020 0x7f2a8c105790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:38:00.877 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.878+0000 7f2a90a00700 1 --2- 172.21.15.153:0/2376160439 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a8c068670 0x7f2a8c068a50 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:38:00.877 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.878+0000 7f2a90a00700 1 -- 172.21.15.153:0/2376160439 >> 172.21.15.153:0/2376160439 conn(0x7f2a8c0755a0 msgr2=0x7f2a8c0759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:38:00.877 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.878+0000 7f2a90a00700 1 -- 172.21.15.153:0/2376160439 shutdown_connections 2024-03-21T10:38:00.877 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.878+0000 7f2a90a00700 1 -- 172.21.15.153:0/2376160439 wait complete. 2024-03-21T10:38:00.879 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.879+0000 7f2a90a00700 1 Processor -- start 2024-03-21T10:38:00.879 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.880+0000 7f2a90a00700 1 -- start start 2024-03-21T10:38:00.879 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.880+0000 7f2a90a00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a8c068670 0x7f2a8c19c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:38:00.879 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.880+0000 7f2a90a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a8c069020 0x7f2a8c19ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:38:00.879 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.880+0000 7f2a90a00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a8c19d0e0 con 0x7f2a8c069020 2024-03-21T10:38:00.879 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.880+0000 7f2a90a00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a8c196590 con 0x7f2a8c068670 2024-03-21T10:38:00.879 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.880+0000 7f2a89400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a8c069020 0x7f2a8c19ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:38:00.879 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.880+0000 7f2a89400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a8c069020 0x7f2a8c19ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:34934/0 (socket says 172.21.15.153:34934) 2024-03-21T10:38:00.880 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.880+0000 7f2a89400700 1 -- 172.21.15.153:0/1163148817 learned_addr learned my addr 172.21.15.153:0/1163148817 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:38:00.880 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.881+0000 7f2a89e00700 1 --2- 172.21.15.153:0/1163148817 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a8c068670 0x7f2a8c19c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:38:00.880 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.881+0000 7f2a89400700 1 -- 172.21.15.153:0/1163148817 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a8c068670 msgr2=0x7f2a8c19c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:38:00.880 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.881+0000 7f2a89400700 1 --2- 172.21.15.153:0/1163148817 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a8c068670 0x7f2a8c19c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:38:00.880 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.881+0000 7f2a89400700 1 -- 172.21.15.153:0/1163148817 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a7c0095f0 con 0x7f2a8c069020 2024-03-21T10:38:00.881 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.881+0000 7f2a89400700 1 --2- 172.21.15.153:0/1163148817 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a8c069020 0x7f2a8c19ca50 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f2a7800d900 tx=0x7f2a7800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:38:00.881 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.881+0000 7f2a82a00700 1 -- 172.21.15.153:0/1163148817 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a78009900 con 0x7f2a8c069020 2024-03-21T10:38:00.881 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.882+0000 7f2a82a00700 1 -- 172.21.15.153:0/1163148817 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2a78010460 con 0x7f2a8c069020 2024-03-21T10:38:00.881 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.882+0000 7f2a82a00700 1 -- 172.21.15.153:0/1163148817 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a7800f5f0 con 0x7f2a8c069020 2024-03-21T10:38:00.881 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.882+0000 7f2a90a00700 1 -- 172.21.15.153:0/1163148817 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a8c196870 con 0x7f2a8c069020 2024-03-21T10:38:00.881 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.882+0000 7f2a90a00700 1 -- 172.21.15.153:0/1163148817 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a8c196dc0 con 0x7f2a8c069020 2024-03-21T10:38:00.881 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.883+0000 7f2a90a00700 1 -- 172.21.15.153:0/1163148817 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a8c109130 con 0x7f2a8c069020 2024-03-21T10:38:00.882 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.884+0000 7f2a82a00700 1 -- 172.21.15.153:0/1163148817 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f2a780105d0 con 0x7f2a8c069020 2024-03-21T10:38:00.883 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.885+0000 7f2a82a00700 1 --2- 172.21.15.153:0/1163148817 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f2a7006e460 0x7f2a70070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:38:00.883 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.885+0000 7f2a82a00700 1 -- 172.21.15.153:0/1163148817 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f2a7808e0d0 con 0x7f2a8c069020 2024-03-21T10:38:00.883 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.885+0000 7f2a89e00700 1 --2- 172.21.15.153:0/1163148817 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f2a7006e460 0x7f2a70070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:38:00.884 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.886+0000 7f2a89e00700 1 --2- 172.21.15.153:0/1163148817 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f2a7006e460 0x7f2a70070920 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c00b5c0 tx=0x7f2a7c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:38:00.890 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:00.890+0000 7f2a82a00700 1 -- 172.21.15.153:0/1163148817 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2a7805aca0 con 0x7f2a8c069020 2024-03-21T10:38:01.118 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:01.120+0000 7f2a90a00700 1 -- 172.21.15.153:0/1163148817 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f2a8c197120 con 0x7f2a7006e460 2024-03-21T10:38:01.134 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:01.136+0000 7f2a82a00700 1 -- 172.21.15.153:0/1163148817 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f2a8c197120 con 0x7f2a7006e460 2024-03-21T10:38:01.134 INFO:teuthology.orchestra.run.smithi153.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-21T10:38:01.134 INFO:teuthology.orchestra.run.smithi153.stdout:alertmanager.smithi153 smithi153 *:9093,9094 running (26m) 2m ago 40m 15.6M - 0.25.0 c8568f914cd2 9ed3f2200b5f 2024-03-21T10:38:01.134 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi153 smithi153 running (40m) 2m ago 40m 10.4M - 18.2.1 5be31c24972a 5b61994396a6 2024-03-21T10:38:01.134 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi174 smithi174 running (39m) 2m ago 39m 10.1M - 18.2.1 5be31c24972a 4f7a5fc3ac28 2024-03-21T10:38:01.135 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi153 smithi153 running (23m) 2m ago 40m 6656k - 19.0.0-2231-g2edfee68 237e43c7a65f 32c21104d5b0 2024-03-21T10:38:01.135 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi174 smithi174 running (22m) 2m ago 39m 6660k - 19.0.0-2231-g2edfee68 237e43c7a65f a04e48967ab6 2024-03-21T10:38:01.135 INFO:teuthology.orchestra.run.smithi153.stdout:grafana.smithi153 smithi153 *:3000 running (26m) 2m ago 40m 75.2M - 9.4.12 2bacad6d85d8 ee24026ab786 2024-03-21T10:38:01.135 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.iuyluj smithi153 running (18m) 2m ago 33m 18.7M - 19.0.0-2231-g2edfee68 237e43c7a65f 87aff8a5de70 2024-03-21T10:38:01.135 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.zipckr smithi153 running (18m) 2m ago 33m 18.9M - 19.0.0-2231-g2edfee68 237e43c7a65f ccc262bd5dd3 2024-03-21T10:38:01.135 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.nhozag smithi174 running (2m) 2m ago 33m 10.9M - 19.0.0-2231-g2edfee68 237e43c7a65f 9e02aea72c96 2024-03-21T10:38:01.135 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.yyimst smithi174 running (17m) 2m ago 33m 18.5M - 19.0.0-2231-g2edfee68 237e43c7a65f fc1962f9351e 2024-03-21T10:38:01.135 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi153.tluxni smithi153 *:8443,9283,8765 running (29m) 2m ago 42m 512M - 19.0.0-2231-g2edfee68 237e43c7a65f 7ffee0e55382 2024-03-21T10:38:01.135 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi174.qpcwat smithi174 *:8443,9283,8765 running (28m) 2m ago 39m 459M - 19.0.0-2231-g2edfee68 237e43c7a65f 748968f216ab 2024-03-21T10:38:01.135 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi153 smithi153 running (24m) 2m ago 42m 70.8M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f dd8038303dc3 2024-03-21T10:38:01.135 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi174 smithi174 running (24m) 2m ago 38m 51.9M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f 29d78a788f2c 2024-03-21T10:38:01.135 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi153 smithi153 *:9100 running (27m) 2m ago 40m 8140k - 1.5.0 0da6a335fe13 bb835edc0e0b 2024-03-21T10:38:01.135 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi174 smithi174 *:9100 running (27m) 2m ago 39m 8610k - 1.5.0 0da6a335fe13 7c4ecad790fa 2024-03-21T10:38:01.135 INFO:teuthology.orchestra.run.smithi153.stdout:osd.0 smithi153 running (22m) 2m ago 38m 118M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 507900fd4791 2024-03-21T10:38:01.136 INFO:teuthology.orchestra.run.smithi153.stdout:osd.1 smithi153 running (21m) 2m ago 37m 93.0M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 0d646452e9c1 2024-03-21T10:38:01.136 INFO:teuthology.orchestra.run.smithi153.stdout:osd.2 smithi153 running (21m) 2m ago 36m 91.2M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 1326f2afdfcd 2024-03-21T10:38:01.136 INFO:teuthology.orchestra.run.smithi153.stdout:osd.3 smithi174 running (20m) 2m ago 36m 87.5M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 70aafac71ad5 2024-03-21T10:38:01.136 INFO:teuthology.orchestra.run.smithi153.stdout:osd.4 smithi174 running (19m) 2m ago 35m 95.8M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 0d746ce6e79d 2024-03-21T10:38:01.136 INFO:teuthology.orchestra.run.smithi153.stdout:osd.5 smithi174 running (19m) 2m ago 35m 90.3M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 53e5f35b8cac 2024-03-21T10:38:01.136 INFO:teuthology.orchestra.run.smithi153.stdout:prometheus.smithi153 smithi153 *:9095 running (27m) 2m ago 39m 88.1M - 2.43.0 a07b618ecd1d 7d95561e50c2 2024-03-21T10:38:01.137 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:01.138+0000 7f2a90a00700 1 -- 172.21.15.153:0/1163148817 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f2a7006e460 msgr2=0x7f2a70070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:38:01.137 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:01.138+0000 7f2a90a00700 1 --2- 172.21.15.153:0/1163148817 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f2a7006e460 0x7f2a70070920 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c00b5c0 tx=0x7f2a7c01a040 comp rx=0 tx=0).stop 2024-03-21T10:38:01.137 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:01.138+0000 7f2a90a00700 1 -- 172.21.15.153:0/1163148817 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a8c069020 msgr2=0x7f2a8c19ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:38:01.137 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:01.138+0000 7f2a90a00700 1 --2- 172.21.15.153:0/1163148817 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a8c069020 0x7f2a8c19ca50 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f2a7800d900 tx=0x7f2a7800dcc0 comp rx=0 tx=0).stop 2024-03-21T10:38:01.137 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:01.138+0000 7f2a90a00700 1 -- 172.21.15.153:0/1163148817 shutdown_connections 2024-03-21T10:38:01.137 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:01.138+0000 7f2a90a00700 1 --2- 172.21.15.153:0/1163148817 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a8c068670 0x7f2a8c19c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:38:01.137 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:01.138+0000 7f2a90a00700 1 --2- 172.21.15.153:0/1163148817 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f2a7006e460 0x7f2a70070920 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:38:01.137 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:01.138+0000 7f2a90a00700 1 --2- 172.21.15.153:0/1163148817 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a8c069020 0x7f2a8c19ca50 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:38:01.137 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:01.138+0000 7f2a90a00700 1 -- 172.21.15.153:0/1163148817 >> 172.21.15.153:0/1163148817 conn(0x7f2a8c0755a0 msgr2=0x7f2a8c0fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:38:01.137 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:01.139+0000 7f2a90a00700 1 -- 172.21.15.153:0/1163148817 shutdown_connections 2024-03-21T10:38:01.138 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:01.139+0000 7f2a90a00700 1 -- 172.21.15.153:0/1163148817 wait complete. 2024-03-21T10:38:01.279 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:01.281+0000 7fb8fbe00700 1 -- 172.21.15.153:0/534483769 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb8fc100970 msgr2=0x7fb8fc100df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:38:01.279 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:01.281+0000 7fb8fbe00700 1 --2- 172.21.15.153:0/534483769 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb8fc100970 0x7fb8fc100df0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fb8e8009910 tx=0x7fb8e8009c20 comp rx=0 tx=0).stop 2024-03-21T10:38:01.279 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:01.281+0000 7fb8fbe00700 1 -- 172.21.15.153:0/534483769 shutdown_connections 2024-03-21T10:38:01.279 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:01.281+0000 7fb8fbe00700 1 --2- 172.21.15.153:0/534483769 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb8fc100970 0x7fb8fc100df0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:38:01.279 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:01.281+0000 7fb8fbe00700 1 --2- 172.21.15.153:0/534483769 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb8fc106970 0x7fb8fc106d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:38:01.279 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:01.281+0000 7fb8fbe00700 1 -- 172.21.15.153:0/534483769 >> 172.21.15.153:0/534483769 conn(0x7fb8fc0fc440 msgr2=0x7fb8fc0fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:38:01.279 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:01.281+0000 7fb8fbe00700 1 -- 172.21.15.153:0/534483769 shutdown_connections 2024-03-21T10:38:01.279 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:01.281+0000 7fb8fbe00700 1 -- 172.21.15.153:0/534483769 wait complete. 2024-03-21T10:38:01.280 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:01.282+0000 7fb8fbe00700 1 Processor -- start 2024-03-21T10:38:01.280 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:01.282+0000 7fb8fbe00700 1 -- start start 2024-03-21T10:38:01.281 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:01.282+0000 7fb8fbe00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb8fc100970 0x7fb8fc19e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:38:01.281 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:01.282+0000 7fb8fbe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb8fc106970 0x7fb8fc19eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:38:01.281 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:01.282+0000 7fb8fbe00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8fc19f210 con 0x7fb8fc106970 2024-03-21T10:38:01.281 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:01.282+0000 7fb8fbe00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8fc1986c0 con 0x7fb8fc100970 2024-03-21T10:38:01.281 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:01.283+0000 7fb8fa000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb8fc106970 0x7fb8fc19eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:38:01.281 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:01.283+0000 7fb8fa000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb8fc106970 0x7fb8fc19eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:34952/0 (socket says 172.21.15.153:34952) 2024-03-21T10:38:01.281 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:01.283+0000 7fb8fa000700 1 -- 172.21.15.153:0/4090497230 learned_addr learned my addr 172.21.15.153:0/4090497230 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:38:01.282 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:01.283+0000 7fb8faa00700 1 --2- 172.21.15.153:0/4090497230 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb8fc100970 0x7fb8fc19e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:38:01.282 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:01.283+0000 7fb8fa000700 1 -- 172.21.15.153:0/4090497230 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb8fc100970 msgr2=0x7fb8fc19e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:38:01.282 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:01.283+0000 7fb8fa000700 1 --2- 172.21.15.153:0/4090497230 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb8fc100970 0x7fb8fc19e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:38:01.282 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:01.283+0000 7fb8fa000700 1 -- 172.21.15.153:0/4090497230 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8e80095f0 con 0x7fb8fc106970 2024-03-21T10:38:01.282 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:01.283+0000 7fb8fa000700 1 --2- 172.21.15.153:0/4090497230 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb8fc106970 0x7fb8fc19eb80 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fb8e800b5c0 tx=0x7fb8e800be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:38:01.283 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:01.284+0000 7fb8f3400700 1 -- 172.21.15.153:0/4090497230 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb8e801d070 con 0x7fb8fc106970 2024-03-21T10:38:01.283 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:01.284+0000 7fb8f3400700 1 -- 172.21.15.153:0/4090497230 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb8e800f460 con 0x7fb8fc106970 2024-03-21T10:38:01.283 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:01.284+0000 7fb8f3400700 1 -- 172.21.15.153:0/4090497230 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb8e8017610 con 0x7fb8fc106970 2024-03-21T10:38:01.283 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:01.284+0000 7fb8fbe00700 1 -- 172.21.15.153:0/4090497230 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8fc198940 con 0x7fb8fc106970 2024-03-21T10:38:01.283 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:01.284+0000 7fb8fbe00700 1 -- 172.21.15.153:0/4090497230 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8fc198de0 con 0x7fb8fc106970 2024-03-21T10:38:01.283 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:01.285+0000 7fb8fbe00700 1 -- 172.21.15.153:0/4090497230 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8fc04f2e0 con 0x7fb8fc106970 2024-03-21T10:38:01.284 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:01.286+0000 7fb8f3400700 1 -- 172.21.15.153:0/4090497230 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fb8e800f5d0 con 0x7fb8fc106970 2024-03-21T10:38:01.285 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:01.287+0000 7fb8f3400700 1 --2- 172.21.15.153:0/4090497230 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb8e006e460 0x7fb8e0070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:38:01.285 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:01.287+0000 7fb8f3400700 1 -- 172.21.15.153:0/4090497230 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fb8e808fb80 con 0x7fb8fc106970 2024-03-21T10:38:01.285 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:01.287+0000 7fb8faa00700 1 --2- 172.21.15.153:0/4090497230 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb8e006e460 0x7fb8e0070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:38:01.286 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:01.288+0000 7fb8faa00700 1 --2- 172.21.15.153:0/4090497230 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb8e006e460 0x7fb8e0070920 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec006fd0 tx=0x7fb8ec008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:38:01.291 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:01.293+0000 7fb8f3400700 1 -- 172.21.15.153:0/4090497230 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb8e805c6d0 con 0x7fb8fc106970 2024-03-21T10:38:01.553 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:38:01 smithi153 ceph-mon[113733]: pgmap v786: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:38:01.553 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:38:01 smithi153 ceph-mon[113733]: from='client.35406 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:38:01.597 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:01.599+0000 7fb8fbe00700 1 -- 172.21.15.153:0/4090497230 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fb8fc199bd0 con 0x7fb8fc106970 2024-03-21T10:38:01.597 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:01.599+0000 7fb8f3400700 1 -- 172.21.15.153:0/4090497230 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fb8e805c260 con 0x7fb8fc106970 2024-03-21T10:38:01.598 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:38:01.598 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": { 2024-03-21T10:38:01.598 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:38:01.598 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:38:01.598 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": { 2024-03-21T10:38:01.598 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:38:01.598 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:38:01.598 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": { 2024-03-21T10:38:01.598 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 6 2024-03-21T10:38:01.599 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:38:01.599 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": { 2024-03-21T10:38:01.599 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 4 2024-03-21T10:38:01.599 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:38:01.599 INFO:teuthology.orchestra.run.smithi153.stdout: "overall": { 2024-03-21T10:38:01.599 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 14 2024-03-21T10:38:01.599 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-03-21T10:38:01.599 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:38:01.601 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:01.603+0000 7fb8fbe00700 1 -- 172.21.15.153:0/4090497230 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb8e006e460 msgr2=0x7fb8e0070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:38:01.601 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:01.603+0000 7fb8fbe00700 1 --2- 172.21.15.153:0/4090497230 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb8e006e460 0x7fb8e0070920 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec006fd0 tx=0x7fb8ec008040 comp rx=0 tx=0).stop 2024-03-21T10:38:01.601 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:01.603+0000 7fb8fbe00700 1 -- 172.21.15.153:0/4090497230 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb8fc106970 msgr2=0x7fb8fc19eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:38:01.601 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:01.603+0000 7fb8fbe00700 1 --2- 172.21.15.153:0/4090497230 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb8fc106970 0x7fb8fc19eb80 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fb8e800b5c0 tx=0x7fb8e800be00 comp rx=0 tx=0).stop 2024-03-21T10:38:01.601 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:01.603+0000 7fb8fbe00700 1 -- 172.21.15.153:0/4090497230 shutdown_connections 2024-03-21T10:38:01.601 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:01.603+0000 7fb8fbe00700 1 --2- 172.21.15.153:0/4090497230 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb8fc100970 0x7fb8fc19e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:38:01.601 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:01.603+0000 7fb8fbe00700 1 --2- 172.21.15.153:0/4090497230 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb8e006e460 0x7fb8e0070920 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:38:01.602 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:01.603+0000 7fb8fbe00700 1 --2- 172.21.15.153:0/4090497230 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb8fc106970 0x7fb8fc19eb80 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:38:01.602 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:01.603+0000 7fb8fbe00700 1 -- 172.21.15.153:0/4090497230 >> 172.21.15.153:0/4090497230 conn(0x7fb8fc0fc440 msgr2=0x7fb8fc0fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:38:01.602 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:01.603+0000 7fb8fbe00700 1 -- 172.21.15.153:0/4090497230 shutdown_connections 2024-03-21T10:38:01.602 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:01.603+0000 7fb8fbe00700 1 -- 172.21.15.153:0/4090497230 wait complete. 2024-03-21T10:38:01.698 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:38:01 smithi174 ceph-mon[88004]: pgmap v786: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:38:01.698 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:38:01 smithi174 ceph-mon[88004]: from='client.35406 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:38:01.747 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:01.748+0000 7fe393600700 1 -- 172.21.15.153:0/2396041612 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe38c103000 msgr2=0x7fe38c1033e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:38:01.747 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:01.748+0000 7fe393600700 1 --2- 172.21.15.153:0/2396041612 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe38c103000 0x7fe38c1033e0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fe380009910 tx=0x7fe380009c20 comp rx=0 tx=0).stop 2024-03-21T10:38:01.747 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:01.748+0000 7fe393600700 1 -- 172.21.15.153:0/2396041612 shutdown_connections 2024-03-21T10:38:01.747 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:01.748+0000 7fe393600700 1 --2- 172.21.15.153:0/2396041612 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe38c1039b0 0x7fe38c107a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:38:01.747 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:01.748+0000 7fe393600700 1 --2- 172.21.15.153:0/2396041612 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe38c103000 0x7fe38c1033e0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:38:01.747 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:01.748+0000 7fe393600700 1 -- 172.21.15.153:0/2396041612 >> 172.21.15.153:0/2396041612 conn(0x7fe38c0fe8d0 msgr2=0x7fe38c100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:38:01.747 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:01.748+0000 7fe393600700 1 -- 172.21.15.153:0/2396041612 shutdown_connections 2024-03-21T10:38:01.747 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:01.748+0000 7fe393600700 1 -- 172.21.15.153:0/2396041612 wait complete. 2024-03-21T10:38:01.748 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:01.749+0000 7fe393600700 1 Processor -- start 2024-03-21T10:38:01.748 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:01.750+0000 7fe393600700 1 -- start start 2024-03-21T10:38:01.748 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:01.750+0000 7fe393600700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe38c103000 0x7fe38c19e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:38:01.748 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:01.750+0000 7fe393600700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe38c1039b0 0x7fe38c19ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:38:01.748 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:01.750+0000 7fe393600700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe38c19f4d0 con 0x7fe38c1039b0 2024-03-21T10:38:01.748 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:01.750+0000 7fe393600700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe38c198980 con 0x7fe38c103000 2024-03-21T10:38:01.749 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:01.750+0000 7fe38be00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe38c1039b0 0x7fe38c19ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:38:01.749 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:01.750+0000 7fe38be00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe38c1039b0 0x7fe38c19ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:34984/0 (socket says 172.21.15.153:34984) 2024-03-21T10:38:01.749 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:01.750+0000 7fe38be00700 1 -- 172.21.15.153:0/4115240640 learned_addr learned my addr 172.21.15.153:0/4115240640 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:38:01.750 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:01.751+0000 7fe390c00700 1 --2- 172.21.15.153:0/4115240640 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe38c103000 0x7fe38c19e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:38:01.750 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:01.751+0000 7fe38be00700 1 -- 172.21.15.153:0/4115240640 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe38c103000 msgr2=0x7fe38c19e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:38:01.750 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:01.751+0000 7fe38be00700 1 --2- 172.21.15.153:0/4115240640 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe38c103000 0x7fe38c19e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:38:01.750 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:01.751+0000 7fe38be00700 1 -- 172.21.15.153:0/4115240640 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3800095f0 con 0x7fe38c1039b0 2024-03-21T10:38:01.750 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:01.751+0000 7fe38be00700 1 --2- 172.21.15.153:0/4115240640 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe38c1039b0 0x7fe38c19ee40 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fe37800b480 tx=0x7fe37800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:38:01.750 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:01.751+0000 7fe389600700 1 -- 172.21.15.153:0/4115240640 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe3780118c0 con 0x7fe38c1039b0 2024-03-21T10:38:01.750 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:01.751+0000 7fe389600700 1 -- 172.21.15.153:0/4115240640 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe378010380 con 0x7fe38c1039b0 2024-03-21T10:38:01.750 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:01.751+0000 7fe393600700 1 -- 172.21.15.153:0/4115240640 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe38c198c60 con 0x7fe38c1039b0 2024-03-21T10:38:01.750 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:01.751+0000 7fe389600700 1 -- 172.21.15.153:0/4115240640 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe37800f4f0 con 0x7fe38c1039b0 2024-03-21T10:38:01.751 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:01.751+0000 7fe393600700 1 -- 172.21.15.153:0/4115240640 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe38c1991b0 con 0x7fe38c1039b0 2024-03-21T10:38:01.751 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:01.752+0000 7fe393600700 1 -- 172.21.15.153:0/4115240640 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe38c10b3a0 con 0x7fe38c1039b0 2024-03-21T10:38:01.751 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:01.754+0000 7fe389600700 1 -- 172.21.15.153:0/4115240640 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fe3780104f0 con 0x7fe38c1039b0 2024-03-21T10:38:01.752 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:01.754+0000 7fe389600700 1 --2- 172.21.15.153:0/4115240640 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe37406e460 0x7fe374070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:38:01.753 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:01.755+0000 7fe389600700 1 -- 172.21.15.153:0/4115240640 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fe37808e120 con 0x7fe38c1039b0 2024-03-21T10:38:01.753 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:01.755+0000 7fe390c00700 1 --2- 172.21.15.153:0/4115240640 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe37406e460 0x7fe374070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:38:01.753 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:01.755+0000 7fe390c00700 1 --2- 172.21.15.153:0/4115240640 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe37406e460 0x7fe374070920 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fe380006010 tx=0x7fe38001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:38:01.757 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:01.759+0000 7fe389600700 1 -- 172.21.15.153:0/4115240640 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe37805acf0 con 0x7fe38c1039b0 2024-03-21T10:38:02.020 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:02.022+0000 7fe393600700 1 -- 172.21.15.153:0/4115240640 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fe38c04ea90 con 0x7fe38c1039b0 2024-03-21T10:38:02.021 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:02.023+0000 7fe389600700 1 -- 172.21.15.153:0/4115240640 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1876 (secure 0 0 0) 0x7fe37805a880 con 0x7fe38c1039b0 2024-03-21T10:38:02.021 INFO:teuthology.orchestra.run.smithi153.stdout:e32 2024-03-21T10:38:02.021 INFO:teuthology.orchestra.run.smithi153.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-21T10:38:02.022 INFO:teuthology.orchestra.run.smithi153.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:38:02.022 INFO:teuthology.orchestra.run.smithi153.stdout:legacy client fscid: 1 2024-03-21T10:38:02.022 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:38:02.022 INFO:teuthology.orchestra.run.smithi153.stdout:Filesystem 'cephfs' (1) 2024-03-21T10:38:02.022 INFO:teuthology.orchestra.run.smithi153.stdout:fs_name cephfs 2024-03-21T10:38:02.022 INFO:teuthology.orchestra.run.smithi153.stdout:epoch 23 2024-03-21T10:38:02.022 INFO:teuthology.orchestra.run.smithi153.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-21T10:38:02.022 INFO:teuthology.orchestra.run.smithi153.stdout:created 2024-03-21T10:04:27.086576+0000 2024-03-21T10:38:02.022 INFO:teuthology.orchestra.run.smithi153.stdout:modified 2024-03-21T10:19:05.022092+0000 2024-03-21T10:38:02.022 INFO:teuthology.orchestra.run.smithi153.stdout:tableserver 0 2024-03-21T10:38:02.022 INFO:teuthology.orchestra.run.smithi153.stdout:root 0 2024-03-21T10:38:02.022 INFO:teuthology.orchestra.run.smithi153.stdout:session_timeout 60 2024-03-21T10:38:02.022 INFO:teuthology.orchestra.run.smithi153.stdout:session_autoclose 300 2024-03-21T10:38:02.023 INFO:teuthology.orchestra.run.smithi153.stdout:max_file_size 1099511627776 2024-03-21T10:38:02.023 INFO:teuthology.orchestra.run.smithi153.stdout:max_xattr_size 65536 2024-03-21T10:38:02.023 INFO:teuthology.orchestra.run.smithi153.stdout:required_client_features {} 2024-03-21T10:38:02.023 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure 0 2024-03-21T10:38:02.023 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure_osd_epoch 94 2024-03-21T10:38:02.023 INFO:teuthology.orchestra.run.smithi153.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:38:02.023 INFO:teuthology.orchestra.run.smithi153.stdout:max_mds 2 2024-03-21T10:38:02.023 INFO:teuthology.orchestra.run.smithi153.stdout:in 0,1 2024-03-21T10:38:02.023 INFO:teuthology.orchestra.run.smithi153.stdout:up {} 2024-03-21T10:38:02.023 INFO:teuthology.orchestra.run.smithi153.stdout:failed 0,1 2024-03-21T10:38:02.023 INFO:teuthology.orchestra.run.smithi153.stdout:damaged 2024-03-21T10:38:02.023 INFO:teuthology.orchestra.run.smithi153.stdout:stopped 2024-03-21T10:38:02.023 INFO:teuthology.orchestra.run.smithi153.stdout:data_pools [3] 2024-03-21T10:38:02.023 INFO:teuthology.orchestra.run.smithi153.stdout:metadata_pool 2 2024-03-21T10:38:02.023 INFO:teuthology.orchestra.run.smithi153.stdout:inline_data enabled 2024-03-21T10:38:02.024 INFO:teuthology.orchestra.run.smithi153.stdout:balancer 2024-03-21T10:38:02.024 INFO:teuthology.orchestra.run.smithi153.stdout:bal_rank_mask -1 2024-03-21T10:38:02.024 INFO:teuthology.orchestra.run.smithi153.stdout:standby_count_wanted 1 2024-03-21T10:38:02.024 INFO:teuthology.orchestra.run.smithi153.stdout:qdb_cluster leader: 0 members: 2024-03-21T10:38:02.024 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:38:02.024 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:38:02.024 INFO:teuthology.orchestra.run.smithi153.stdout:Standby daemons: 2024-03-21T10:38:02.024 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:38:02.024 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.zipckr{-1:34442} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6826/1103846295,v1:172.21.15.153:6827/1103846295] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:38:02.024 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.iuyluj{-1:34474} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6828/1738630832,v1:172.21.15.153:6829/1738630832] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:38:02.024 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.yyimst{-1:44313} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.174:6824/4127786797,v1:172.21.15.174:6825/4127786797] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:38:02.024 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.nhozag{-1:44687} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.174:6826/2767517845,v1:172.21.15.174:6827/2767517845] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:38:02.025 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:02.025+0000 7fe393600700 1 -- 172.21.15.153:0/4115240640 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe37406e460 msgr2=0x7fe374070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:38:02.025 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:02.025+0000 7fe393600700 1 --2- 172.21.15.153:0/4115240640 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe37406e460 0x7fe374070920 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fe380006010 tx=0x7fe38001a040 comp rx=0 tx=0).stop 2024-03-21T10:38:02.025 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:02.025+0000 7fe393600700 1 -- 172.21.15.153:0/4115240640 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe38c1039b0 msgr2=0x7fe38c19ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:38:02.025 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:02.025+0000 7fe393600700 1 --2- 172.21.15.153:0/4115240640 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe38c1039b0 0x7fe38c19ee40 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fe37800b480 tx=0x7fe37800b840 comp rx=0 tx=0).stop 2024-03-21T10:38:02.025 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:02.025+0000 7fe393600700 1 -- 172.21.15.153:0/4115240640 shutdown_connections 2024-03-21T10:38:02.025 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:02.025+0000 7fe393600700 1 --2- 172.21.15.153:0/4115240640 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe38c103000 0x7fe38c19e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:38:02.025 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:02.025+0000 7fe393600700 1 --2- 172.21.15.153:0/4115240640 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe37406e460 0x7fe374070920 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:38:02.025 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:02.025+0000 7fe393600700 1 --2- 172.21.15.153:0/4115240640 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe38c1039b0 0x7fe38c19ee40 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:38:02.025 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:02.025+0000 7fe393600700 1 -- 172.21.15.153:0/4115240640 >> 172.21.15.153:0/4115240640 conn(0x7fe38c0fe8d0 msgr2=0x7fe38c0fff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:38:02.026 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:02.026+0000 7fe393600700 1 -- 172.21.15.153:0/4115240640 shutdown_connections 2024-03-21T10:38:02.026 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:02.026+0000 7fe393600700 1 -- 172.21.15.153:0/4115240640 wait complete. 2024-03-21T10:38:02.026 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 32 2024-03-21T10:38:02.184 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:02.185+0000 7ff983200700 1 -- 172.21.15.153:0/1017345949 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff984102e10 msgr2=0x7ff9841031f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:38:02.184 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:02.185+0000 7ff983200700 1 --2- 172.21.15.153:0/1017345949 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff984102e10 0x7ff9841031f0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7ff974009910 tx=0x7ff974009c20 comp rx=0 tx=0).stop 2024-03-21T10:38:02.184 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:02.185+0000 7ff983200700 1 -- 172.21.15.153:0/1017345949 shutdown_connections 2024-03-21T10:38:02.184 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:02.185+0000 7ff983200700 1 --2- 172.21.15.153:0/1017345949 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9841037c0 0x7ff984107810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:38:02.184 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:02.185+0000 7ff983200700 1 --2- 172.21.15.153:0/1017345949 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff984102e10 0x7ff9841031f0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:38:02.184 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:02.185+0000 7ff983200700 1 -- 172.21.15.153:0/1017345949 >> 172.21.15.153:0/1017345949 conn(0x7ff9840fe6c0 msgr2=0x7ff984100ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:38:02.184 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:02.185+0000 7ff983200700 1 -- 172.21.15.153:0/1017345949 shutdown_connections 2024-03-21T10:38:02.184 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:02.185+0000 7ff983200700 1 -- 172.21.15.153:0/1017345949 wait complete. 2024-03-21T10:38:02.185 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:02.186+0000 7ff983200700 1 Processor -- start 2024-03-21T10:38:02.185 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:02.187+0000 7ff983200700 1 -- start start 2024-03-21T10:38:02.185 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:02.187+0000 7ff983200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff984102e10 0x7ff98419e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:38:02.185 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:02.187+0000 7ff983200700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9841037c0 0x7ff98419eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:38:02.186 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:02.187+0000 7ff983200700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff98419f150 con 0x7ff984102e10 2024-03-21T10:38:02.186 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:02.187+0000 7ff983200700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff984198600 con 0x7ff9841037c0 2024-03-21T10:38:02.186 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:02.187+0000 7ff981e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff984102e10 0x7ff98419e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:38:02.186 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:02.187+0000 7ff981e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff984102e10 0x7ff98419e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:35012/0 (socket says 172.21.15.153:35012) 2024-03-21T10:38:02.186 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:02.187+0000 7ff981e00700 1 -- 172.21.15.153:0/2767504244 learned_addr learned my addr 172.21.15.153:0/2767504244 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:38:02.186 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:02.188+0000 7ff981400700 1 --2- 172.21.15.153:0/2767504244 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9841037c0 0x7ff98419eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:38:02.187 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:02.188+0000 7ff981e00700 1 -- 172.21.15.153:0/2767504244 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9841037c0 msgr2=0x7ff98419eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:38:02.187 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:02.188+0000 7ff981e00700 1 --2- 172.21.15.153:0/2767504244 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9841037c0 0x7ff98419eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:38:02.187 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:02.188+0000 7ff981e00700 1 -- 172.21.15.153:0/2767504244 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9740095f0 con 0x7ff984102e10 2024-03-21T10:38:02.187 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:02.188+0000 7ff981e00700 1 --2- 172.21.15.153:0/2767504244 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff984102e10 0x7ff98419e580 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7ff9740092e0 tx=0x7ff97400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:38:02.187 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:02.188+0000 7ff97aa00700 1 -- 172.21.15.153:0/2767504244 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff97401d070 con 0x7ff984102e10 2024-03-21T10:38:02.187 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:02.188+0000 7ff97aa00700 1 -- 172.21.15.153:0/2767504244 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff974005e00 con 0x7ff984102e10 2024-03-21T10:38:02.187 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:02.188+0000 7ff983200700 1 -- 172.21.15.153:0/2767504244 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff984198880 con 0x7ff984102e10 2024-03-21T10:38:02.187 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:02.188+0000 7ff97aa00700 1 -- 172.21.15.153:0/2767504244 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff974017400 con 0x7ff984102e10 2024-03-21T10:38:02.188 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:02.189+0000 7ff983200700 1 -- 172.21.15.153:0/2767504244 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff984198d70 con 0x7ff984102e10 2024-03-21T10:38:02.188 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:02.190+0000 7ff983200700 1 -- 172.21.15.153:0/2767504244 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff98410b1b0 con 0x7ff984102e10 2024-03-21T10:38:02.188 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:02.191+0000 7ff97aa00700 1 -- 172.21.15.153:0/2767504244 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7ff97400f460 con 0x7ff984102e10 2024-03-21T10:38:02.190 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:02.192+0000 7ff97aa00700 1 --2- 172.21.15.153:0/2767504244 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff96806e460 0x7ff968070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:38:02.190 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:02.192+0000 7ff97aa00700 1 -- 172.21.15.153:0/2767504244 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7ff97408f910 con 0x7ff984102e10 2024-03-21T10:38:02.190 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:02.192+0000 7ff981400700 1 --2- 172.21.15.153:0/2767504244 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff96806e460 0x7ff968070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:38:02.190 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:02.193+0000 7ff981400700 1 --2- 172.21.15.153:0/2767504244 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff96806e460 0x7ff968070920 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7ff970005950 tx=0x7ff9700058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:38:02.194 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:02.196+0000 7ff97aa00700 1 -- 172.21.15.153:0/2767504244 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff97405c580 con 0x7ff984102e10 2024-03-21T10:38:02.303 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:38:02 smithi153 ceph-mon[113733]: from='client.35410 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:38:02.303 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:38:02 smithi153 ceph-mon[113733]: from='client.35414 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:38:02.303 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:38:02 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:38:02.303 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:38:02 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/4090497230' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:38:02.303 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:38:02 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/4115240640' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:38:02.438 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:02.440+0000 7ff983200700 1 -- 172.21.15.153:0/2767504244 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff9841990a0 con 0x7ff96806e460 2024-03-21T10:38:02.441 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:02.443+0000 7ff97aa00700 1 -- 172.21.15.153:0/2767504244 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7ff9841990a0 con 0x7ff96806e460 2024-03-21T10:38:02.441 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:38:02.442 INFO:teuthology.orchestra.run.smithi153.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232", 2024-03-21T10:38:02.442 INFO:teuthology.orchestra.run.smithi153.stdout: "in_progress": true, 2024-03-21T10:38:02.442 INFO:teuthology.orchestra.run.smithi153.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-21T10:38:02.442 INFO:teuthology.orchestra.run.smithi153.stdout: "services_complete": [ 2024-03-21T10:38:02.442 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr", 2024-03-21T10:38:02.442 INFO:teuthology.orchestra.run.smithi153.stdout: "mon", 2024-03-21T10:38:02.442 INFO:teuthology.orchestra.run.smithi153.stdout: "mds", 2024-03-21T10:38:02.442 INFO:teuthology.orchestra.run.smithi153.stdout: "crash", 2024-03-21T10:38:02.442 INFO:teuthology.orchestra.run.smithi153.stdout: "osd" 2024-03-21T10:38:02.443 INFO:teuthology.orchestra.run.smithi153.stdout: ], 2024-03-21T10:38:02.443 INFO:teuthology.orchestra.run.smithi153.stdout: "progress": "16/23 daemons upgraded", 2024-03-21T10:38:02.443 INFO:teuthology.orchestra.run.smithi153.stdout: "message": "Currently upgrading mds daemons", 2024-03-21T10:38:02.443 INFO:teuthology.orchestra.run.smithi153.stdout: "is_paused": false 2024-03-21T10:38:02.443 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:38:02.445 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:02.446+0000 7ff983200700 1 -- 172.21.15.153:0/2767504244 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff96806e460 msgr2=0x7ff968070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:38:02.445 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:02.446+0000 7ff983200700 1 --2- 172.21.15.153:0/2767504244 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff96806e460 0x7ff968070920 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7ff970005950 tx=0x7ff9700058e0 comp rx=0 tx=0).stop 2024-03-21T10:38:02.445 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:02.446+0000 7ff983200700 1 -- 172.21.15.153:0/2767504244 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff984102e10 msgr2=0x7ff98419e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:38:02.445 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:02.446+0000 7ff983200700 1 --2- 172.21.15.153:0/2767504244 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff984102e10 0x7ff98419e580 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7ff9740092e0 tx=0x7ff97400be20 comp rx=0 tx=0).stop 2024-03-21T10:38:02.445 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:02.446+0000 7ff983200700 1 -- 172.21.15.153:0/2767504244 shutdown_connections 2024-03-21T10:38:02.445 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:02.446+0000 7ff983200700 1 --2- 172.21.15.153:0/2767504244 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff96806e460 0x7ff968070920 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:38:02.445 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:02.446+0000 7ff983200700 1 --2- 172.21.15.153:0/2767504244 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff984102e10 0x7ff98419e580 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:38:02.445 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:02.446+0000 7ff983200700 1 --2- 172.21.15.153:0/2767504244 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9841037c0 0x7ff98419eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:38:02.445 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:02.446+0000 7ff983200700 1 -- 172.21.15.153:0/2767504244 >> 172.21.15.153:0/2767504244 conn(0x7ff9840fe6c0 msgr2=0x7ff9840ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:38:02.446 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:02.447+0000 7ff983200700 1 -- 172.21.15.153:0/2767504244 shutdown_connections 2024-03-21T10:38:02.446 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:02.447+0000 7ff983200700 1 -- 172.21.15.153:0/2767504244 wait complete. 2024-03-21T10:38:02.601 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:02.602+0000 7f02df800700 1 -- 172.21.15.153:0/155078737 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f02d8073160 msgr2=0x7f02d8073540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:38:02.601 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:02.602+0000 7f02df800700 1 --2- 172.21.15.153:0/155078737 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f02d8073160 0x7f02d8073540 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f02c8009910 tx=0x7f02c8009c20 comp rx=0 tx=0).stop 2024-03-21T10:38:02.601 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:02.603+0000 7f02df800700 1 -- 172.21.15.153:0/155078737 shutdown_connections 2024-03-21T10:38:02.601 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:02.603+0000 7f02df800700 1 --2- 172.21.15.153:0/155078737 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f02d8073a80 0x7f02d810cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:38:02.601 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:02.603+0000 7f02df800700 1 --2- 172.21.15.153:0/155078737 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f02d8073160 0x7f02d8073540 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:38:02.602 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:02.603+0000 7f02df800700 1 -- 172.21.15.153:0/155078737 >> 172.21.15.153:0/155078737 conn(0x7f02d80fc6a0 msgr2=0x7f02d80feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:38:02.602 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:02.603+0000 7f02df800700 1 -- 172.21.15.153:0/155078737 shutdown_connections 2024-03-21T10:38:02.602 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:02.603+0000 7f02df800700 1 -- 172.21.15.153:0/155078737 wait complete. 2024-03-21T10:38:02.602 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:02.605+0000 7f02df800700 1 Processor -- start 2024-03-21T10:38:02.603 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:02.605+0000 7f02df800700 1 -- start start 2024-03-21T10:38:02.603 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:02.605+0000 7f02df800700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f02d8073160 0x7f02d819e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:38:02.603 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:02.606+0000 7f02df800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f02d8073a80 0x7f02d819ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:38:02.604 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:02.606+0000 7f02df800700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02d8198820 con 0x7f02d8073a80 2024-03-21T10:38:02.604 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:02.606+0000 7f02df800700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02d8198990 con 0x7f02d8073160 2024-03-21T10:38:02.605 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:02.606+0000 7f02cfe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f02d8073a80 0x7f02d819ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:38:02.605 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:02.606+0000 7f02cfe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f02d8073a80 0x7f02d819ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:35028/0 (socket says 172.21.15.153:35028) 2024-03-21T10:38:02.605 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:02.606+0000 7f02cfe00700 1 -- 172.21.15.153:0/458733938 learned_addr learned my addr 172.21.15.153:0/458733938 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:38:02.605 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:02.606+0000 7f02dce00700 1 --2- 172.21.15.153:0/458733938 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f02d8073160 0x7f02d819e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:38:02.605 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:02.606+0000 7f02cfe00700 1 -- 172.21.15.153:0/458733938 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f02d8073160 msgr2=0x7f02d819e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:38:02.605 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:02.606+0000 7f02cfe00700 1 --2- 172.21.15.153:0/458733938 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f02d8073160 0x7f02d819e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:38:02.605 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:02.606+0000 7f02cfe00700 1 -- 172.21.15.153:0/458733938 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02c80095f0 con 0x7f02d8073a80 2024-03-21T10:38:02.605 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:02.607+0000 7f02cfe00700 1 --2- 172.21.15.153:0/458733938 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f02d8073a80 0x7f02d819ed70 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f02c400c8a0 tx=0x7f02c400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:38:02.605 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:02.607+0000 7f02d6000700 1 -- 172.21.15.153:0/458733938 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f02c400cea0 con 0x7f02d8073a80 2024-03-21T10:38:02.606 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:02.607+0000 7f02d6000700 1 -- 172.21.15.153:0/458733938 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f02c4004830 con 0x7f02d8073a80 2024-03-21T10:38:02.606 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:02.607+0000 7f02d6000700 1 -- 172.21.15.153:0/458733938 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f02c40055b0 con 0x7f02d8073a80 2024-03-21T10:38:02.606 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:02.607+0000 7f02df800700 1 -- 172.21.15.153:0/458733938 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02d8198c70 con 0x7f02d8073a80 2024-03-21T10:38:02.606 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:02.607+0000 7f02df800700 1 -- 172.21.15.153:0/458733938 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02d81991c0 con 0x7f02d8073a80 2024-03-21T10:38:02.606 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:02.608+0000 7f02df800700 1 -- 172.21.15.153:0/458733938 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02d810a5f0 con 0x7f02d8073a80 2024-03-21T10:38:02.607 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:02.609+0000 7f02d6000700 1 -- 172.21.15.153:0/458733938 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f02c40049a0 con 0x7f02d8073a80 2024-03-21T10:38:02.608 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:02.610+0000 7f02d6000700 1 --2- 172.21.15.153:0/458733938 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f02bc06e460 0x7f02bc070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:38:02.609 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:02.610+0000 7f02d6000700 1 -- 172.21.15.153:0/458733938 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f02c408e150 con 0x7f02d8073a80 2024-03-21T10:38:02.609 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:02.611+0000 7f02dce00700 1 --2- 172.21.15.153:0/458733938 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f02bc06e460 0x7f02bc070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:38:02.609 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:02.611+0000 7f02dce00700 1 --2- 172.21.15.153:0/458733938 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f02bc06e460 0x7f02bc070920 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f02c8005f50 tx=0x7f02c801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:38:02.614 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:02.615+0000 7f02d6000700 1 -- 172.21.15.153:0/458733938 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f02c40571f0 con 0x7f02d8073a80 2024-03-21T10:38:02.698 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:38:02 smithi174 ceph-mon[88004]: from='client.35410 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:38:02.698 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:38:02 smithi174 ceph-mon[88004]: from='client.35414 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:38:02.698 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:38:02 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:38:02.698 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:38:02 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/4090497230' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:38:02.699 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:38:02 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/4115240640' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:38:02.922 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:02.923+0000 7f02df800700 1 -- 172.21.15.153:0/458733938 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f02d804ea90 con 0x7f02d8073a80 2024-03-21T10:38:02.922 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:02.924+0000 7f02d6000700 1 -- 172.21.15.153:0/458733938 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4493 (secure 0 0 0) 0x7f02c4004c50 con 0x7f02d8073a80 2024-03-21T10:38:02.923 INFO:teuthology.orchestra.run.smithi153.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-03-21T10:38:02.923 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-03-21T10:38:02.923 INFO:teuthology.orchestra.run.smithi153.stdout: host smithi174 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi174/config 2024-03-21T10:38:02.923 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/podman: timeout after 895 seconds 2024-03-21T10:38:02.923 INFO:teuthology.orchestra.run.smithi153.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 -e NODE_NAME=smithi174 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/run/ceph:z -v /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpd910r3_i:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 inventory --format=json-pretty --filter-for-batch 2024-03-21T10:38:02.923 INFO:teuthology.orchestra.run.smithi153.stdout:Traceback (most recent call last): 2024-03-21T10:38:02.923 INFO:teuthology.orchestra.run.smithi153.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-03-21T10:38:02.923 INFO:teuthology.orchestra.run.smithi153.stdout: return _run_code(code, main_globals, None, 2024-03-21T10:38:02.923 INFO:teuthology.orchestra.run.smithi153.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-03-21T10:38:02.923 INFO:teuthology.orchestra.run.smithi153.stdout: exec(code, run_globals) 2024-03-21T10:38:02.924 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 5606, in 2024-03-21T10:38:02.924 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 5594, in main 2024-03-21T10:38:02.924 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 406, in _infer_config 2024-03-21T10:38:02.924 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 321, in _infer_fsid 2024-03-21T10:38:02.924 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 434, in _infer_image 2024-03-21T10:38:02.924 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 308, in _validate_fsid 2024-03-21T10:38:02.924 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 3357, in command_ceph_volume 2024-03-21T10:38:02.924 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-03-21T10:38:02.924 INFO:teuthology.orchestra.run.smithi153.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 -e NODE_NAME=smithi174 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/run/ceph:z -v /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpd910r3_i:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 inventory --format=json-pretty --filter-for-batch 2024-03-21T10:38:02.924 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-21T10:38:02.924 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is degraded 2024-03-21T10:38:02.924 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-21T10:38:02.924 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-21T10:38:02.924 INFO:teuthology.orchestra.run.smithi153.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-21T10:38:02.925 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-21T10:38:02.925 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:02.927+0000 7f02df800700 1 -- 172.21.15.153:0/458733938 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f02bc06e460 msgr2=0x7f02bc070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:38:02.925 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:02.927+0000 7f02df800700 1 --2- 172.21.15.153:0/458733938 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f02bc06e460 0x7f02bc070920 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f02c8005f50 tx=0x7f02c801a040 comp rx=0 tx=0).stop 2024-03-21T10:38:02.925 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:02.927+0000 7f02df800700 1 -- 172.21.15.153:0/458733938 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f02d8073a80 msgr2=0x7f02d819ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:38:02.925 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:02.927+0000 7f02df800700 1 --2- 172.21.15.153:0/458733938 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f02d8073a80 0x7f02d819ed70 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f02c400c8a0 tx=0x7f02c400cc60 comp rx=0 tx=0).stop 2024-03-21T10:38:02.926 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:02.927+0000 7f02df800700 1 -- 172.21.15.153:0/458733938 shutdown_connections 2024-03-21T10:38:02.926 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:02.927+0000 7f02df800700 1 --2- 172.21.15.153:0/458733938 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f02d8073160 0x7f02d819e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:38:02.926 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:02.927+0000 7f02df800700 1 --2- 172.21.15.153:0/458733938 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f02bc06e460 0x7f02bc070920 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:38:02.926 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:02.927+0000 7f02df800700 1 --2- 172.21.15.153:0/458733938 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f02d8073a80 0x7f02d819ed70 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:38:02.926 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:02.927+0000 7f02df800700 1 -- 172.21.15.153:0/458733938 >> 172.21.15.153:0/458733938 conn(0x7f02d80fc6a0 msgr2=0x7f02d8107730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:38:02.926 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:02.927+0000 7f02df800700 1 -- 172.21.15.153:0/458733938 shutdown_connections 2024-03-21T10:38:02.926 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:02.927+0000 7f02df800700 1 -- 172.21.15.153:0/458733938 wait complete. 2024-03-21T10:38:03.553 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:38:03 smithi153 ceph-mon[113733]: pgmap v787: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:38:03.553 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:38:03 smithi153 ceph-mon[113733]: from='client.35426 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:38:03.553 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:38:03 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/458733938' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:38:03.698 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:38:03 smithi174 ceph-mon[88004]: pgmap v787: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:38:03.698 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:38:03 smithi174 ceph-mon[88004]: from='client.35426 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:38:03.698 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:38:03 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/458733938' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:38:05.553 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:38:05 smithi153 ceph-mon[113733]: pgmap v788: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:38:05.698 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:38:05 smithi174 ceph-mon[88004]: pgmap v788: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:38:07.553 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:38:07 smithi153 ceph-mon[113733]: pgmap v789: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:38:07.698 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:38:07 smithi174 ceph-mon[88004]: pgmap v789: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:38:09.553 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:38:09 smithi153 ceph-mon[113733]: pgmap v790: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:38:09.698 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:38:09 smithi174 ceph-mon[88004]: pgmap v790: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:38:11.553 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:38:11 smithi153 ceph-mon[113733]: pgmap v791: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:38:11.698 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:38:11 smithi174 ceph-mon[88004]: pgmap v791: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:38:13.553 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:38:13 smithi153 ceph-mon[113733]: pgmap v792: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:38:13.698 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:38:13 smithi174 ceph-mon[88004]: pgmap v792: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:38:15.553 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:38:15 smithi153 ceph-mon[113733]: pgmap v793: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:38:15.698 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:38:15 smithi174 ceph-mon[88004]: pgmap v793: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:38:16.698 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:38:16 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:38:16.803 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:38:16 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:38:17.698 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:38:17 smithi174 ceph-mon[88004]: pgmap v794: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:38:17.803 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:38:17 smithi153 ceph-mon[113733]: pgmap v794: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:38:19.698 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:38:19 smithi174 ceph-mon[88004]: pgmap v795: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:38:19.803 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:38:19 smithi153 ceph-mon[113733]: pgmap v795: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:38:21.698 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:38:21 smithi174 ceph-mon[88004]: pgmap v796: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:38:21.803 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:38:21 smithi153 ceph-mon[113733]: pgmap v796: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:38:23.698 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:38:23 smithi174 ceph-mon[88004]: pgmap v797: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:38:23.803 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:38:23 smithi153 ceph-mon[113733]: pgmap v797: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:38:25.553 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:38:25 smithi153 ceph-mon[113733]: pgmap v798: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:38:25.698 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:38:25 smithi174 ceph-mon[88004]: pgmap v798: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:38:27.698 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:38:27 smithi174 ceph-mon[88004]: pgmap v799: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:38:27.803 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:38:27 smithi153 ceph-mon[113733]: pgmap v799: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:38:29.698 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:38:29 smithi174 ceph-mon[88004]: pgmap v800: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:38:29.803 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:38:29 smithi153 ceph-mon[113733]: pgmap v800: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:38:31.698 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:38:31 smithi174 ceph-mon[88004]: pgmap v801: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:38:31.698 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:38:31 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:38:31.803 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:38:31 smithi153 ceph-mon[113733]: pgmap v801: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:38:31.803 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:38:31 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:38:33.097 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.100+0000 7f72f8e00700 1 -- 172.21.15.153:0/1947958098 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72f4100da0 msgr2=0x7f72f4101180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:38:33.097 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.100+0000 7f72f8e00700 1 --2- 172.21.15.153:0/1947958098 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72f4100da0 0x7f72f4101180 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f72e4009910 tx=0x7f72e4009c20 comp rx=0 tx=0).stop 2024-03-21T10:38:33.097 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.100+0000 7f72f8e00700 1 -- 172.21.15.153:0/1947958098 shutdown_connections 2024-03-21T10:38:33.099 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.100+0000 7f72f8e00700 1 --2- 172.21.15.153:0/1947958098 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f72f4101750 0x7f72f41057a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:38:33.099 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.100+0000 7f72f8e00700 1 --2- 172.21.15.153:0/1947958098 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72f4100da0 0x7f72f4101180 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:38:33.099 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.100+0000 7f72f8e00700 1 -- 172.21.15.153:0/1947958098 >> 172.21.15.153:0/1947958098 conn(0x7f72f40fc630 msgr2=0x7f72f40fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:38:33.099 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.100+0000 7f72f8e00700 1 -- 172.21.15.153:0/1947958098 shutdown_connections 2024-03-21T10:38:33.099 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.100+0000 7f72f8e00700 1 -- 172.21.15.153:0/1947958098 wait complete. 2024-03-21T10:38:33.101 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.101+0000 7f72f8e00700 1 Processor -- start 2024-03-21T10:38:33.101 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.101+0000 7f72f8e00700 1 -- start start 2024-03-21T10:38:33.101 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.102+0000 7f72f8e00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f72f4100da0 0x7f72f419c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:38:33.101 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.102+0000 7f72f8e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72f4101750 0x7f72f419cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:38:33.101 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.102+0000 7f72f8e00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72f419d210 con 0x7f72f4101750 2024-03-21T10:38:33.101 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.102+0000 7f72f8e00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72f41966c0 con 0x7f72f4100da0 2024-03-21T10:38:33.101 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.102+0000 7f72f1400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72f4101750 0x7f72f419cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:38:33.101 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.102+0000 7f72f1400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72f4101750 0x7f72f419cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:55210/0 (socket says 172.21.15.153:55210) 2024-03-21T10:38:33.101 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.102+0000 7f72f1400700 1 -- 172.21.15.153:0/4139563095 learned_addr learned my addr 172.21.15.153:0/4139563095 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:38:33.101 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.102+0000 7f72f1e00700 1 --2- 172.21.15.153:0/4139563095 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f72f4100da0 0x7f72f419c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:38:33.101 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.103+0000 7f72f1400700 1 -- 172.21.15.153:0/4139563095 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f72f4100da0 msgr2=0x7f72f419c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:38:33.101 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.103+0000 7f72f1400700 1 --2- 172.21.15.153:0/4139563095 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f72f4100da0 0x7f72f419c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:38:33.101 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.103+0000 7f72f1400700 1 -- 172.21.15.153:0/4139563095 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72e40095f0 con 0x7f72f4101750 2024-03-21T10:38:33.102 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.103+0000 7f72f1400700 1 --2- 172.21.15.153:0/4139563095 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72f4101750 0x7f72f419cb80 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f72e000b480 tx=0x7f72e000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:38:33.102 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.103+0000 7f72eaa00700 1 -- 172.21.15.153:0/4139563095 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f72e00118c0 con 0x7f72f4101750 2024-03-21T10:38:33.103 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.103+0000 7f72eaa00700 1 -- 172.21.15.153:0/4139563095 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f72e0010380 con 0x7f72f4101750 2024-03-21T10:38:33.103 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.103+0000 7f72f8e00700 1 -- 172.21.15.153:0/4139563095 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72f41969a0 con 0x7f72f4101750 2024-03-21T10:38:33.103 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.103+0000 7f72eaa00700 1 -- 172.21.15.153:0/4139563095 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f72e000f4f0 con 0x7f72f4101750 2024-03-21T10:38:33.103 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.104+0000 7f72f8e00700 1 -- 172.21.15.153:0/4139563095 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72f4196ef0 con 0x7f72f4101750 2024-03-21T10:38:33.103 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.105+0000 7f72f8e00700 1 -- 172.21.15.153:0/4139563095 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72f4109140 con 0x7f72f4101750 2024-03-21T10:38:33.103 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.106+0000 7f72eaa00700 1 -- 172.21.15.153:0/4139563095 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f72e00104f0 con 0x7f72f4101750 2024-03-21T10:38:33.104 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.107+0000 7f72eaa00700 1 --2- 172.21.15.153:0/4139563095 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f72d806e4b0 0x7f72d8070970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:38:33.104 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.107+0000 7f72eaa00700 1 -- 172.21.15.153:0/4139563095 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f72e008dfe0 con 0x7f72f4101750 2024-03-21T10:38:33.104 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.107+0000 7f72f1e00700 1 --2- 172.21.15.153:0/4139563095 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f72d806e4b0 0x7f72d8070970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:38:33.105 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.108+0000 7f72f1e00700 1 --2- 172.21.15.153:0/4139563095 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f72d806e4b0 0x7f72d8070970 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f72e4006010 tx=0x7f72e401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:38:33.109 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.112+0000 7f72eaa00700 1 -- 172.21.15.153:0/4139563095 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f72e005ac70 con 0x7f72f4101750 2024-03-21T10:38:33.352 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.355+0000 7f72f8e00700 1 -- 172.21.15.153:0/4139563095 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f72f4197230 con 0x7f72d806e4b0 2024-03-21T10:38:33.354 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.357+0000 7f72eaa00700 1 -- 172.21.15.153:0/4139563095 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f72f4197230 con 0x7f72d806e4b0 2024-03-21T10:38:33.357 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.360+0000 7f72f8e00700 1 -- 172.21.15.153:0/4139563095 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f72d806e4b0 msgr2=0x7f72d8070970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:38:33.357 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.360+0000 7f72f8e00700 1 --2- 172.21.15.153:0/4139563095 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f72d806e4b0 0x7f72d8070970 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f72e4006010 tx=0x7f72e401a040 comp rx=0 tx=0).stop 2024-03-21T10:38:33.358 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.360+0000 7f72f8e00700 1 -- 172.21.15.153:0/4139563095 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72f4101750 msgr2=0x7f72f419cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:38:33.358 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.360+0000 7f72f8e00700 1 --2- 172.21.15.153:0/4139563095 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72f4101750 0x7f72f419cb80 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f72e000b480 tx=0x7f72e000b840 comp rx=0 tx=0).stop 2024-03-21T10:38:33.358 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.360+0000 7f72f8e00700 1 -- 172.21.15.153:0/4139563095 shutdown_connections 2024-03-21T10:38:33.358 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.360+0000 7f72f8e00700 1 --2- 172.21.15.153:0/4139563095 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f72f4100da0 0x7f72f419c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:38:33.358 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.360+0000 7f72f8e00700 1 --2- 172.21.15.153:0/4139563095 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f72d806e4b0 0x7f72d8070970 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:38:33.358 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.360+0000 7f72f8e00700 1 --2- 172.21.15.153:0/4139563095 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f72f4101750 0x7f72f419cb80 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:38:33.358 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.360+0000 7f72f8e00700 1 -- 172.21.15.153:0/4139563095 >> 172.21.15.153:0/4139563095 conn(0x7f72f40fc630 msgr2=0x7f72f40fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:38:33.358 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.361+0000 7f72f8e00700 1 -- 172.21.15.153:0/4139563095 shutdown_connections 2024-03-21T10:38:33.358 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.361+0000 7f72f8e00700 1 -- 172.21.15.153:0/4139563095 wait complete. 2024-03-21T10:38:33.376 INFO:teuthology.orchestra.run.smithi153.stdout:true 2024-03-21T10:38:33.519 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.522+0000 7fc1c9200700 1 -- 172.21.15.153:0/4289208261 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc1c4107bd0 msgr2=0x7fc1c410bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:38:33.520 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.522+0000 7fc1c9200700 1 --2- 172.21.15.153:0/4289208261 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc1c4107bd0 0x7fc1c410bc10 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac009910 tx=0x7fc1ac009c20 comp rx=0 tx=0).stop 2024-03-21T10:38:33.520 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.522+0000 7fc1c9200700 1 -- 172.21.15.153:0/4289208261 shutdown_connections 2024-03-21T10:38:33.520 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.522+0000 7fc1c9200700 1 --2- 172.21.15.153:0/4289208261 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc1c4107bd0 0x7fc1c410bc10 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:38:33.520 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.522+0000 7fc1c9200700 1 --2- 172.21.15.153:0/4289208261 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc1c4107220 0x7fc1c4107600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:38:33.520 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.522+0000 7fc1c9200700 1 -- 172.21.15.153:0/4289208261 >> 172.21.15.153:0/4289208261 conn(0x7fc1c40767e0 msgr2=0x7fc1c4078c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:38:33.520 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.522+0000 7fc1c9200700 1 -- 172.21.15.153:0/4289208261 shutdown_connections 2024-03-21T10:38:33.520 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.523+0000 7fc1c9200700 1 -- 172.21.15.153:0/4289208261 wait complete. 2024-03-21T10:38:33.522 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.523+0000 7fc1c9200700 1 Processor -- start 2024-03-21T10:38:33.522 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.524+0000 7fc1c9200700 1 -- start start 2024-03-21T10:38:33.522 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.524+0000 7fc1c9200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc1c4107220 0x7fc1c41a29f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:38:33.522 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.524+0000 7fc1c9200700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc1c4107bd0 0x7fc1c41a2f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:38:33.522 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.524+0000 7fc1c9200700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c41a35c0 con 0x7fc1c4107220 2024-03-21T10:38:33.522 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.524+0000 7fc1c9200700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c419ca70 con 0x7fc1c4107bd0 2024-03-21T10:38:33.522 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.524+0000 7fc1c3400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc1c4107220 0x7fc1c41a29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:38:33.522 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.524+0000 7fc1c3400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc1c4107220 0x7fc1c41a29f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:55218/0 (socket says 172.21.15.153:55218) 2024-03-21T10:38:33.522 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.524+0000 7fc1c3400700 1 -- 172.21.15.153:0/1461198002 learned_addr learned my addr 172.21.15.153:0/1461198002 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:38:33.522 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.525+0000 7fc1baa00700 1 --2- 172.21.15.153:0/1461198002 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc1c4107bd0 0x7fc1c41a2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:38:33.522 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.525+0000 7fc1c3400700 1 -- 172.21.15.153:0/1461198002 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc1c4107bd0 msgr2=0x7fc1c41a2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:38:33.523 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.525+0000 7fc1c3400700 1 --2- 172.21.15.153:0/1461198002 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc1c4107bd0 0x7fc1c41a2f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:38:33.523 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.525+0000 7fc1c3400700 1 -- 172.21.15.153:0/1461198002 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1ac0095f0 con 0x7fc1c4107220 2024-03-21T10:38:33.524 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.525+0000 7fc1c3400700 1 --2- 172.21.15.153:0/1461198002 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc1c4107220 0x7fc1c41a29f0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fc1b400ada0 tx=0x7fc1b400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:38:33.524 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.526+0000 7fc1c0c00700 1 -- 172.21.15.153:0/1461198002 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc1b400f070 con 0x7fc1c4107220 2024-03-21T10:38:33.524 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.526+0000 7fc1c0c00700 1 -- 172.21.15.153:0/1461198002 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc1b4007bf0 con 0x7fc1c4107220 2024-03-21T10:38:33.524 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.526+0000 7fc1c9200700 1 -- 172.21.15.153:0/1461198002 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1c419cd50 con 0x7fc1c4107220 2024-03-21T10:38:33.524 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.526+0000 7fc1c0c00700 1 -- 172.21.15.153:0/1461198002 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc1b4005690 con 0x7fc1c4107220 2024-03-21T10:38:33.525 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.526+0000 7fc1c9200700 1 -- 172.21.15.153:0/1461198002 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1c419d2a0 con 0x7fc1c4107220 2024-03-21T10:38:33.525 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.527+0000 7fc1c9200700 1 -- 172.21.15.153:0/1461198002 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1c410f610 con 0x7fc1c4107220 2024-03-21T10:38:33.525 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.528+0000 7fc1c0c00700 1 -- 172.21.15.153:0/1461198002 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fc1b4005850 con 0x7fc1c4107220 2024-03-21T10:38:33.526 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.529+0000 7fc1c0c00700 1 --2- 172.21.15.153:0/1461198002 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fc1a806e460 0x7fc1a8070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:38:33.526 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.529+0000 7fc1c0c00700 1 -- 172.21.15.153:0/1461198002 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fc1b408df10 con 0x7fc1c4107220 2024-03-21T10:38:33.526 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.529+0000 7fc1baa00700 1 --2- 172.21.15.153:0/1461198002 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fc1a806e460 0x7fc1a8070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:38:33.526 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.530+0000 7fc1baa00700 1 --2- 172.21.15.153:0/1461198002 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fc1a806e460 0x7fc1a8070920 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fc1c4077000 tx=0x7fc1ac01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:38:33.531 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.534+0000 7fc1c0c00700 1 -- 172.21.15.153:0/1461198002 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc1b405aae0 con 0x7fc1c4107220 2024-03-21T10:38:33.698 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:38:33 smithi174 ceph-mon[88004]: pgmap v802: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:38:33.773 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.775+0000 7fc1c9200700 1 -- 172.21.15.153:0/1461198002 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc1c419d580 con 0x7fc1a806e460 2024-03-21T10:38:33.773 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:38:33 smithi153 ceph-mon[113733]: pgmap v802: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:38:33.774 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.777+0000 7fc1c0c00700 1 -- 172.21.15.153:0/1461198002 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fc1c419d580 con 0x7fc1a806e460 2024-03-21T10:38:33.777 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.780+0000 7fc1c9200700 1 -- 172.21.15.153:0/1461198002 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fc1a806e460 msgr2=0x7fc1a8070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:38:33.777 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.780+0000 7fc1c9200700 1 --2- 172.21.15.153:0/1461198002 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fc1a806e460 0x7fc1a8070920 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fc1c4077000 tx=0x7fc1ac01a040 comp rx=0 tx=0).stop 2024-03-21T10:38:33.777 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.780+0000 7fc1c9200700 1 -- 172.21.15.153:0/1461198002 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc1c4107220 msgr2=0x7fc1c41a29f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:38:33.777 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.780+0000 7fc1c9200700 1 --2- 172.21.15.153:0/1461198002 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc1c4107220 0x7fc1c41a29f0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fc1b400ada0 tx=0x7fc1b400c330 comp rx=0 tx=0).stop 2024-03-21T10:38:33.777 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.780+0000 7fc1c9200700 1 -- 172.21.15.153:0/1461198002 shutdown_connections 2024-03-21T10:38:33.777 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.780+0000 7fc1c9200700 1 --2- 172.21.15.153:0/1461198002 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fc1a806e460 0x7fc1a8070920 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:38:33.777 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.780+0000 7fc1c9200700 1 --2- 172.21.15.153:0/1461198002 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc1c4107220 0x7fc1c41a29f0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:38:33.777 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.780+0000 7fc1c9200700 1 --2- 172.21.15.153:0/1461198002 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc1c4107bd0 0x7fc1c41a2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:38:33.778 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.780+0000 7fc1c9200700 1 -- 172.21.15.153:0/1461198002 >> 172.21.15.153:0/1461198002 conn(0x7fc1c40767e0 msgr2=0x7fc1c4078bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:38:33.778 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.780+0000 7fc1c9200700 1 -- 172.21.15.153:0/1461198002 shutdown_connections 2024-03-21T10:38:33.778 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.780+0000 7fc1c9200700 1 -- 172.21.15.153:0/1461198002 wait complete. 2024-03-21T10:38:33.935 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.937+0000 7f1fc2800700 1 -- 172.21.15.153:0/1024427644 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1fbc068670 msgr2=0x7f1fbc068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:38:33.935 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.937+0000 7f1fc2800700 1 --2- 172.21.15.153:0/1024427644 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1fbc068670 0x7f1fbc068a50 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f1fac009910 tx=0x7f1fac009c20 comp rx=0 tx=0).stop 2024-03-21T10:38:33.935 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.937+0000 7f1fc2800700 1 -- 172.21.15.153:0/1024427644 shutdown_connections 2024-03-21T10:38:33.935 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.937+0000 7f1fc2800700 1 --2- 172.21.15.153:0/1024427644 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1fbc069020 0x7f1fbc105790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:38:33.935 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.937+0000 7f1fc2800700 1 --2- 172.21.15.153:0/1024427644 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1fbc068670 0x7f1fbc068a50 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:38:33.935 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.937+0000 7f1fc2800700 1 -- 172.21.15.153:0/1024427644 >> 172.21.15.153:0/1024427644 conn(0x7f1fbc0755a0 msgr2=0x7f1fbc0759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:38:33.935 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.937+0000 7f1fc2800700 1 -- 172.21.15.153:0/1024427644 shutdown_connections 2024-03-21T10:38:33.935 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.937+0000 7f1fc2800700 1 -- 172.21.15.153:0/1024427644 wait complete. 2024-03-21T10:38:33.936 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.938+0000 7f1fc2800700 1 Processor -- start 2024-03-21T10:38:33.936 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.939+0000 7f1fc2800700 1 -- start start 2024-03-21T10:38:33.936 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.939+0000 7f1fc2800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1fbc068670 0x7f1fbc0ffca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:38:33.937 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.939+0000 7f1fc2800700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1fbc069020 0x7f1fbc1001e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:38:33.937 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.939+0000 7f1fc2800700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fbc100720 con 0x7f1fbc068670 2024-03-21T10:38:33.937 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.939+0000 7f1fc2800700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fbc100860 con 0x7f1fbc069020 2024-03-21T10:38:33.937 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.939+0000 7f1fbb400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1fbc068670 0x7f1fbc0ffca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:38:33.937 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.939+0000 7f1fbb400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1fbc068670 0x7f1fbc0ffca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:55226/0 (socket says 172.21.15.153:55226) 2024-03-21T10:38:33.937 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.939+0000 7f1fbb400700 1 -- 172.21.15.153:0/1596230685 learned_addr learned my addr 172.21.15.153:0/1596230685 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:38:33.937 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.940+0000 7f1fbb400700 1 -- 172.21.15.153:0/1596230685 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1fbc069020 msgr2=0x7f1fbc1001e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:38:33.937 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.940+0000 7f1fb2a00700 1 --2- 172.21.15.153:0/1596230685 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1fbc069020 0x7f1fbc1001e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:38:33.937 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.940+0000 7f1fbb400700 1 --2- 172.21.15.153:0/1596230685 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1fbc069020 0x7f1fbc1001e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:38:33.937 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.940+0000 7f1fbb400700 1 -- 172.21.15.153:0/1596230685 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fac0095f0 con 0x7f1fbc068670 2024-03-21T10:38:33.938 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.940+0000 7f1fbb400700 1 --2- 172.21.15.153:0/1596230685 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1fbc068670 0x7f1fbc0ffca0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f1fac00b5c0 tx=0x7f1fac00be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:38:33.938 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.940+0000 7f1fb8c00700 1 -- 172.21.15.153:0/1596230685 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1fac01d070 con 0x7f1fbc068670 2024-03-21T10:38:33.938 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.940+0000 7f1fb8c00700 1 -- 172.21.15.153:0/1596230685 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1fac00f460 con 0x7f1fbc068670 2024-03-21T10:38:33.939 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.940+0000 7f1fb8c00700 1 -- 172.21.15.153:0/1596230685 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1fac017610 con 0x7f1fbc068670 2024-03-21T10:38:33.939 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.940+0000 7f1fc2800700 1 -- 172.21.15.153:0/1596230685 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fbc100ae0 con 0x7f1fbc068670 2024-03-21T10:38:33.939 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.940+0000 7f1fc2800700 1 -- 172.21.15.153:0/1596230685 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fbc1a2df0 con 0x7f1fbc068670 2024-03-21T10:38:33.939 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.941+0000 7f1fc2800700 1 -- 172.21.15.153:0/1596230685 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fbc109130 con 0x7f1fbc068670 2024-03-21T10:38:33.939 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.942+0000 7f1fb8c00700 1 -- 172.21.15.153:0/1596230685 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f1fac00f5d0 con 0x7f1fbc068670 2024-03-21T10:38:33.940 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.943+0000 7f1fb8c00700 1 --2- 172.21.15.153:0/1596230685 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f1fa006e460 0x7f1fa0070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:38:33.940 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.943+0000 7f1fb8c00700 1 -- 172.21.15.153:0/1596230685 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f1fac08fad0 con 0x7f1fbc068670 2024-03-21T10:38:33.940 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.944+0000 7f1fb2a00700 1 --2- 172.21.15.153:0/1596230685 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f1fa006e460 0x7f1fa0070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:38:33.941 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.944+0000 7f1fb2a00700 1 --2- 172.21.15.153:0/1596230685 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f1fa006e460 0x7f1fa0070920 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc101490 tx=0x7f1fa40072f0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:38:33.945 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:33.948+0000 7f1fb8c00700 1 -- 172.21.15.153:0/1596230685 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f1fac058b70 con 0x7f1fbc068670 2024-03-21T10:38:34.180 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:34.183+0000 7f1fc2800700 1 -- 172.21.15.153:0/1596230685 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f1fbc0611d0 con 0x7f1fa006e460 2024-03-21T10:38:34.213 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:34.216+0000 7f1fb8c00700 1 -- 172.21.15.153:0/1596230685 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f1fbc0611d0 con 0x7f1fa006e460 2024-03-21T10:38:34.213 INFO:teuthology.orchestra.run.smithi153.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-21T10:38:34.213 INFO:teuthology.orchestra.run.smithi153.stdout:alertmanager.smithi153 smithi153 *:9093,9094 running (27m) 3m ago 41m 15.6M - 0.25.0 c8568f914cd2 9ed3f2200b5f 2024-03-21T10:38:34.214 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi153 smithi153 running (41m) 3m ago 41m 10.4M - 18.2.1 5be31c24972a 5b61994396a6 2024-03-21T10:38:34.214 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi174 smithi174 running (39m) 3m ago 39m 10.1M - 18.2.1 5be31c24972a 4f7a5fc3ac28 2024-03-21T10:38:34.214 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi153 smithi153 running (23m) 3m ago 41m 6656k - 19.0.0-2231-g2edfee68 237e43c7a65f 32c21104d5b0 2024-03-21T10:38:34.214 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi174 smithi174 running (23m) 3m ago 39m 6660k - 19.0.0-2231-g2edfee68 237e43c7a65f a04e48967ab6 2024-03-21T10:38:34.214 INFO:teuthology.orchestra.run.smithi153.stdout:grafana.smithi153 smithi153 *:3000 running (26m) 3m ago 40m 75.2M - 9.4.12 2bacad6d85d8 ee24026ab786 2024-03-21T10:38:34.214 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.iuyluj smithi153 running (18m) 3m ago 33m 18.7M - 19.0.0-2231-g2edfee68 237e43c7a65f 87aff8a5de70 2024-03-21T10:38:34.214 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.zipckr smithi153 running (19m) 3m ago 34m 18.9M - 19.0.0-2231-g2edfee68 237e43c7a65f ccc262bd5dd3 2024-03-21T10:38:34.214 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.nhozag smithi174 running (3m) 3m ago 33m 10.9M - 19.0.0-2231-g2edfee68 237e43c7a65f 9e02aea72c96 2024-03-21T10:38:34.214 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.yyimst smithi174 running (18m) 3m ago 33m 18.5M - 19.0.0-2231-g2edfee68 237e43c7a65f fc1962f9351e 2024-03-21T10:38:34.214 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi153.tluxni smithi153 *:8443,9283,8765 running (30m) 3m ago 42m 512M - 19.0.0-2231-g2edfee68 237e43c7a65f 7ffee0e55382 2024-03-21T10:38:34.214 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi174.qpcwat smithi174 *:8443,9283,8765 running (29m) 3m ago 39m 459M - 19.0.0-2231-g2edfee68 237e43c7a65f 748968f216ab 2024-03-21T10:38:34.214 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi153 smithi153 running (25m) 3m ago 43m 70.8M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f dd8038303dc3 2024-03-21T10:38:34.214 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi174 smithi174 running (24m) 3m ago 39m 51.9M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f 29d78a788f2c 2024-03-21T10:38:34.214 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi153 smithi153 *:9100 running (28m) 3m ago 41m 8140k - 1.5.0 0da6a335fe13 bb835edc0e0b 2024-03-21T10:38:34.215 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi174 smithi174 *:9100 running (28m) 3m ago 39m 8610k - 1.5.0 0da6a335fe13 7c4ecad790fa 2024-03-21T10:38:34.215 INFO:teuthology.orchestra.run.smithi153.stdout:osd.0 smithi153 running (22m) 3m ago 38m 118M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 507900fd4791 2024-03-21T10:38:34.215 INFO:teuthology.orchestra.run.smithi153.stdout:osd.1 smithi153 running (22m) 3m ago 37m 93.0M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 0d646452e9c1 2024-03-21T10:38:34.215 INFO:teuthology.orchestra.run.smithi153.stdout:osd.2 smithi153 running (21m) 3m ago 37m 91.2M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 1326f2afdfcd 2024-03-21T10:38:34.215 INFO:teuthology.orchestra.run.smithi153.stdout:osd.3 smithi174 running (21m) 3m ago 36m 87.5M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 70aafac71ad5 2024-03-21T10:38:34.215 INFO:teuthology.orchestra.run.smithi153.stdout:osd.4 smithi174 running (20m) 3m ago 36m 95.8M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 0d746ce6e79d 2024-03-21T10:38:34.215 INFO:teuthology.orchestra.run.smithi153.stdout:osd.5 smithi174 running (19m) 3m ago 35m 90.3M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 53e5f35b8cac 2024-03-21T10:38:34.215 INFO:teuthology.orchestra.run.smithi153.stdout:prometheus.smithi153 smithi153 *:9095 running (27m) 3m ago 40m 88.1M - 2.43.0 a07b618ecd1d 7d95561e50c2 2024-03-21T10:38:34.216 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:34.219+0000 7f1fc2800700 1 -- 172.21.15.153:0/1596230685 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f1fa006e460 msgr2=0x7f1fa0070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:38:34.216 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:34.219+0000 7f1fc2800700 1 --2- 172.21.15.153:0/1596230685 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f1fa006e460 0x7f1fa0070920 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc101490 tx=0x7f1fa40072f0 comp rx=0 tx=0).stop 2024-03-21T10:38:34.216 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:34.219+0000 7f1fc2800700 1 -- 172.21.15.153:0/1596230685 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1fbc068670 msgr2=0x7f1fbc0ffca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:38:34.216 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:34.219+0000 7f1fc2800700 1 --2- 172.21.15.153:0/1596230685 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1fbc068670 0x7f1fbc0ffca0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f1fac00b5c0 tx=0x7f1fac00be00 comp rx=0 tx=0).stop 2024-03-21T10:38:34.217 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:34.219+0000 7f1fc2800700 1 -- 172.21.15.153:0/1596230685 shutdown_connections 2024-03-21T10:38:34.217 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:34.219+0000 7f1fc2800700 1 --2- 172.21.15.153:0/1596230685 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f1fa006e460 0x7f1fa0070920 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:38:34.217 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:34.219+0000 7f1fc2800700 1 --2- 172.21.15.153:0/1596230685 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1fbc068670 0x7f1fbc0ffca0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:38:34.217 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:34.219+0000 7f1fc2800700 1 --2- 172.21.15.153:0/1596230685 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1fbc069020 0x7f1fbc1001e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:38:34.217 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:34.219+0000 7f1fc2800700 1 -- 172.21.15.153:0/1596230685 >> 172.21.15.153:0/1596230685 conn(0x7f1fbc0755a0 msgr2=0x7f1fbc0fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:38:34.217 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:34.219+0000 7f1fc2800700 1 -- 172.21.15.153:0/1596230685 shutdown_connections 2024-03-21T10:38:34.217 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:34.219+0000 7f1fc2800700 1 -- 172.21.15.153:0/1596230685 wait complete. 2024-03-21T10:38:34.369 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:34.372+0000 7f3cd5000700 1 -- 172.21.15.153:0/102326075 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3cd010d280 msgr2=0x7f3cd010d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:38:34.370 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:34.372+0000 7f3cd5000700 1 --2- 172.21.15.153:0/102326075 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3cd010d280 0x7f3cd010d660 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f3cc0009910 tx=0x7f3cc0009c20 comp rx=0 tx=0).stop 2024-03-21T10:38:34.370 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:34.372+0000 7f3cd5000700 1 -- 172.21.15.153:0/102326075 shutdown_connections 2024-03-21T10:38:34.370 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:34.372+0000 7f3cd5000700 1 --2- 172.21.15.153:0/102326075 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3cd01071d0 0x7f3cd0107650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:38:34.370 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:34.372+0000 7f3cd5000700 1 --2- 172.21.15.153:0/102326075 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3cd010d280 0x7f3cd010d660 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:38:34.370 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:34.372+0000 7f3cd5000700 1 -- 172.21.15.153:0/102326075 >> 172.21.15.153:0/102326075 conn(0x7f3cd00769e0 msgr2=0x7f3cd0078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:38:34.370 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:34.372+0000 7f3cd5000700 1 -- 172.21.15.153:0/102326075 shutdown_connections 2024-03-21T10:38:34.370 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:34.372+0000 7f3cd5000700 1 -- 172.21.15.153:0/102326075 wait complete. 2024-03-21T10:38:34.371 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:34.373+0000 7f3cd5000700 1 Processor -- start 2024-03-21T10:38:34.371 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:34.374+0000 7f3cd5000700 1 -- start start 2024-03-21T10:38:34.371 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:34.374+0000 7f3cd5000700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3cd01071d0 0x7f3cd01a2a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:38:34.371 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:34.374+0000 7f3cd5000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3cd010d280 0x7f3cd01a2f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:38:34.371 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:34.374+0000 7f3cd5000700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd01a3640 con 0x7f3cd010d280 2024-03-21T10:38:34.372 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:34.374+0000 7f3cd5000700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd019cb00 con 0x7f3cd01071d0 2024-03-21T10:38:34.372 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:34.374+0000 7f3ccd400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3cd010d280 0x7f3cd01a2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:38:34.372 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:34.374+0000 7f3ccd400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3cd010d280 0x7f3cd01a2f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:55244/0 (socket says 172.21.15.153:55244) 2024-03-21T10:38:34.372 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:34.374+0000 7f3ccd400700 1 -- 172.21.15.153:0/2343552186 learned_addr learned my addr 172.21.15.153:0/2343552186 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:38:34.372 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:34.375+0000 7f3ccde00700 1 --2- 172.21.15.153:0/2343552186 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3cd01071d0 0x7f3cd01a2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:38:34.373 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:34.375+0000 7f3ccd400700 1 -- 172.21.15.153:0/2343552186 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3cd01071d0 msgr2=0x7f3cd01a2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:38:34.373 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:34.375+0000 7f3ccd400700 1 --2- 172.21.15.153:0/2343552186 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3cd01071d0 0x7f3cd01a2a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:38:34.373 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:34.375+0000 7f3ccd400700 1 -- 172.21.15.153:0/2343552186 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cc00095f0 con 0x7f3cd010d280 2024-03-21T10:38:34.373 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:34.375+0000 7f3ccd400700 1 --2- 172.21.15.153:0/2343552186 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3cd010d280 0x7f3cd01a2f60 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc00d8d0 tx=0x7f3cbc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:38:34.373 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:34.375+0000 7f3cc6a00700 1 -- 172.21.15.153:0/2343552186 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3cbc009940 con 0x7f3cd010d280 2024-03-21T10:38:34.373 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:34.375+0000 7f3cc6a00700 1 -- 172.21.15.153:0/2343552186 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3cbc010460 con 0x7f3cd010d280 2024-03-21T10:38:34.373 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:34.375+0000 7f3cd5000700 1 -- 172.21.15.153:0/2343552186 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cd019cde0 con 0x7f3cd010d280 2024-03-21T10:38:34.374 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:34.376+0000 7f3cc6a00700 1 -- 172.21.15.153:0/2343552186 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3cbc00f5f0 con 0x7f3cd010d280 2024-03-21T10:38:34.374 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:34.376+0000 7f3cd5000700 1 -- 172.21.15.153:0/2343552186 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cd019d3b0 con 0x7f3cd010d280 2024-03-21T10:38:34.374 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:34.377+0000 7f3cd5000700 1 -- 172.21.15.153:0/2343552186 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cd004ea90 con 0x7f3cd010d280 2024-03-21T10:38:34.377 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:34.380+0000 7f3cc6a00700 1 -- 172.21.15.153:0/2343552186 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f3cbc0105d0 con 0x7f3cd010d280 2024-03-21T10:38:34.378 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:34.381+0000 7f3cc6a00700 1 --2- 172.21.15.153:0/2343552186 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f3cb406e460 0x7f3cb4070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:38:34.378 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:34.381+0000 7f3cc6a00700 1 -- 172.21.15.153:0/2343552186 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f3cbc08e480 con 0x7f3cd010d280 2024-03-21T10:38:34.378 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:34.381+0000 7f3ccde00700 1 --2- 172.21.15.153:0/2343552186 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f3cb406e460 0x7f3cb4070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:38:34.379 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:34.382+0000 7f3ccde00700 1 --2- 172.21.15.153:0/2343552186 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f3cb406e460 0x7f3cb4070920 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f3cc0006010 tx=0x7f3cc001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:38:34.381 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:34.385+0000 7f3cc6a00700 1 -- 172.21.15.153:0/2343552186 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f3cbc05b180 con 0x7f3cd010d280 2024-03-21T10:38:34.553 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:38:34 smithi153 ceph-mon[113733]: from='client.35434 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:38:34.674 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:34.677+0000 7f3cd5000700 1 -- 172.21.15.153:0/2343552186 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f3cd019dff0 con 0x7f3cd010d280 2024-03-21T10:38:34.675 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:34.678+0000 7f3cc6a00700 1 -- 172.21.15.153:0/2343552186 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f3cbc05ad10 con 0x7f3cd010d280 2024-03-21T10:38:34.675 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:38:34.676 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": { 2024-03-21T10:38:34.676 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:38:34.676 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:38:34.676 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": { 2024-03-21T10:38:34.676 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:38:34.676 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:38:34.676 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": { 2024-03-21T10:38:34.676 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 6 2024-03-21T10:38:34.676 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:38:34.676 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": { 2024-03-21T10:38:34.676 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 4 2024-03-21T10:38:34.676 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:38:34.676 INFO:teuthology.orchestra.run.smithi153.stdout: "overall": { 2024-03-21T10:38:34.676 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 14 2024-03-21T10:38:34.677 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-03-21T10:38:34.677 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:38:34.678 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:34.681+0000 7f3cd5000700 1 -- 172.21.15.153:0/2343552186 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f3cb406e460 msgr2=0x7f3cb4070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:38:34.678 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:34.681+0000 7f3cd5000700 1 --2- 172.21.15.153:0/2343552186 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f3cb406e460 0x7f3cb4070920 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f3cc0006010 tx=0x7f3cc001a040 comp rx=0 tx=0).stop 2024-03-21T10:38:34.678 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:34.681+0000 7f3cd5000700 1 -- 172.21.15.153:0/2343552186 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3cd010d280 msgr2=0x7f3cd01a2f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:38:34.678 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:34.681+0000 7f3cd5000700 1 --2- 172.21.15.153:0/2343552186 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3cd010d280 0x7f3cd01a2f60 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc00d8d0 tx=0x7f3cbc00dc90 comp rx=0 tx=0).stop 2024-03-21T10:38:34.678 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:34.681+0000 7f3cd5000700 1 -- 172.21.15.153:0/2343552186 shutdown_connections 2024-03-21T10:38:34.679 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:34.681+0000 7f3cd5000700 1 --2- 172.21.15.153:0/2343552186 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3cd01071d0 0x7f3cd01a2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:38:34.679 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:34.681+0000 7f3cd5000700 1 --2- 172.21.15.153:0/2343552186 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f3cb406e460 0x7f3cb4070920 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:38:34.679 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:34.681+0000 7f3cd5000700 1 --2- 172.21.15.153:0/2343552186 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3cd010d280 0x7f3cd01a2f60 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:38:34.679 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:34.681+0000 7f3cd5000700 1 -- 172.21.15.153:0/2343552186 >> 172.21.15.153:0/2343552186 conn(0x7f3cd00769e0 msgr2=0x7f3cd0078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:38:34.679 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:34.681+0000 7f3cd5000700 1 -- 172.21.15.153:0/2343552186 shutdown_connections 2024-03-21T10:38:34.679 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:34.681+0000 7f3cd5000700 1 -- 172.21.15.153:0/2343552186 wait complete. 2024-03-21T10:38:34.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:38:34 smithi174 ceph-mon[88004]: from='client.35434 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:38:34.835 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:34.838+0000 7fb8abe00700 1 -- 172.21.15.153:0/536378271 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb8a4075be0 msgr2=0x7fb8a41114b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:38:34.835 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:34.838+0000 7fb8abe00700 1 --2- 172.21.15.153:0/536378271 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb8a4075be0 0x7fb8a41114b0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fb890009910 tx=0x7fb890009c20 comp rx=0 tx=0).stop 2024-03-21T10:38:34.836 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:34.838+0000 7fb8abe00700 1 -- 172.21.15.153:0/536378271 shutdown_connections 2024-03-21T10:38:34.836 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:34.838+0000 7fb8abe00700 1 --2- 172.21.15.153:0/536378271 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb8a4075be0 0x7fb8a41114b0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:38:34.836 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:34.838+0000 7fb8abe00700 1 --2- 172.21.15.153:0/536378271 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb8a40752c0 0x7fb8a40756a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:38:34.836 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:34.838+0000 7fb8abe00700 1 -- 172.21.15.153:0/536378271 >> 172.21.15.153:0/536378271 conn(0x7fb8a40fe840 msgr2=0x7fb8a4100c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:38:34.836 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:34.838+0000 7fb8abe00700 1 -- 172.21.15.153:0/536378271 shutdown_connections 2024-03-21T10:38:34.836 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:34.838+0000 7fb8abe00700 1 -- 172.21.15.153:0/536378271 wait complete. 2024-03-21T10:38:34.837 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:34.839+0000 7fb8abe00700 1 Processor -- start 2024-03-21T10:38:34.837 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:34.839+0000 7fb8abe00700 1 -- start start 2024-03-21T10:38:34.837 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:34.840+0000 7fb8abe00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb8a40752c0 0x7fb8a419ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:38:34.837 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:34.840+0000 7fb8abe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb8a4075be0 0x7fb8a419d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:38:34.838 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:34.840+0000 7fb8abe00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a419d800 con 0x7fb8a4075be0 2024-03-21T10:38:34.838 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:34.840+0000 7fb8abe00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a419d970 con 0x7fb8a40752c0 2024-03-21T10:38:34.838 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:34.840+0000 7fb8a8a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb8a4075be0 0x7fb8a419d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:38:34.838 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:34.840+0000 7fb8a8a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb8a4075be0 0x7fb8a419d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:55260/0 (socket says 172.21.15.153:55260) 2024-03-21T10:38:34.838 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:34.840+0000 7fb8a8a00700 1 -- 172.21.15.153:0/1645314361 learned_addr learned my addr 172.21.15.153:0/1645314361 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:38:34.838 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:34.840+0000 7fb8a9400700 1 --2- 172.21.15.153:0/1645314361 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb8a40752c0 0x7fb8a419ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:38:34.838 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:34.841+0000 7fb8a8a00700 1 -- 172.21.15.153:0/1645314361 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb8a40752c0 msgr2=0x7fb8a419ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:38:34.838 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:34.841+0000 7fb8a8a00700 1 --2- 172.21.15.153:0/1645314361 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb8a40752c0 0x7fb8a419ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:38:34.838 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:34.841+0000 7fb8a8a00700 1 -- 172.21.15.153:0/1645314361 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8900095f0 con 0x7fb8a4075be0 2024-03-21T10:38:34.839 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:34.841+0000 7fb8a8a00700 1 --2- 172.21.15.153:0/1645314361 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb8a4075be0 0x7fb8a419d200 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fb890005b40 tx=0x7fb8900056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:38:34.839 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:34.841+0000 7fb8a2000700 1 -- 172.21.15.153:0/1645314361 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb89001d070 con 0x7fb8a4075be0 2024-03-21T10:38:34.839 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:34.841+0000 7fb8a2000700 1 -- 172.21.15.153:0/1645314361 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb89000f460 con 0x7fb8a4075be0 2024-03-21T10:38:34.839 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:34.841+0000 7fb8abe00700 1 -- 172.21.15.153:0/1645314361 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8a41a0710 con 0x7fb8a4075be0 2024-03-21T10:38:34.839 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:34.841+0000 7fb8a2000700 1 -- 172.21.15.153:0/1645314361 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb890021640 con 0x7fb8a4075be0 2024-03-21T10:38:34.839 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:34.841+0000 7fb8abe00700 1 -- 172.21.15.153:0/1645314361 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8a41a0c60 con 0x7fb8a4075be0 2024-03-21T10:38:34.840 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:34.843+0000 7fb8abe00700 1 -- 172.21.15.153:0/1645314361 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8a410ec20 con 0x7fb8a4075be0 2024-03-21T10:38:34.841 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:34.844+0000 7fb8a2000700 1 -- 172.21.15.153:0/1645314361 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fb89000f5d0 con 0x7fb8a4075be0 2024-03-21T10:38:34.841 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:34.844+0000 7fb8a2000700 1 --2- 172.21.15.153:0/1645314361 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb88c06e460 0x7fb88c070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:38:34.842 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:34.845+0000 7fb8a2000700 1 -- 172.21.15.153:0/1645314361 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fb89008f920 con 0x7fb8a4075be0 2024-03-21T10:38:34.842 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:34.845+0000 7fb8a9400700 1 --2- 172.21.15.153:0/1645314361 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb88c06e460 0x7fb88c070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:38:34.842 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:34.846+0000 7fb8a9400700 1 --2- 172.21.15.153:0/1645314361 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb88c06e460 0x7fb88c070920 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fb898005ea0 tx=0x7fb898005e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:38:34.846 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:34.849+0000 7fb8a2000700 1 -- 172.21.15.153:0/1645314361 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb890058940 con 0x7fb8a4075be0 2024-03-21T10:38:35.116 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:35.119+0000 7fb8abe00700 1 -- 172.21.15.153:0/1645314361 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fb8a404ea90 con 0x7fb8a4075be0 2024-03-21T10:38:35.117 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:35.120+0000 7fb8a2000700 1 -- 172.21.15.153:0/1645314361 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1876 (secure 0 0 0) 0x7fb89005c000 con 0x7fb8a4075be0 2024-03-21T10:38:35.118 INFO:teuthology.orchestra.run.smithi153.stdout:e32 2024-03-21T10:38:35.118 INFO:teuthology.orchestra.run.smithi153.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-21T10:38:35.118 INFO:teuthology.orchestra.run.smithi153.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:38:35.118 INFO:teuthology.orchestra.run.smithi153.stdout:legacy client fscid: 1 2024-03-21T10:38:35.118 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:38:35.118 INFO:teuthology.orchestra.run.smithi153.stdout:Filesystem 'cephfs' (1) 2024-03-21T10:38:35.118 INFO:teuthology.orchestra.run.smithi153.stdout:fs_name cephfs 2024-03-21T10:38:35.118 INFO:teuthology.orchestra.run.smithi153.stdout:epoch 23 2024-03-21T10:38:35.118 INFO:teuthology.orchestra.run.smithi153.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-21T10:38:35.119 INFO:teuthology.orchestra.run.smithi153.stdout:created 2024-03-21T10:04:27.086576+0000 2024-03-21T10:38:35.119 INFO:teuthology.orchestra.run.smithi153.stdout:modified 2024-03-21T10:19:05.022092+0000 2024-03-21T10:38:35.119 INFO:teuthology.orchestra.run.smithi153.stdout:tableserver 0 2024-03-21T10:38:35.119 INFO:teuthology.orchestra.run.smithi153.stdout:root 0 2024-03-21T10:38:35.119 INFO:teuthology.orchestra.run.smithi153.stdout:session_timeout 60 2024-03-21T10:38:35.119 INFO:teuthology.orchestra.run.smithi153.stdout:session_autoclose 300 2024-03-21T10:38:35.119 INFO:teuthology.orchestra.run.smithi153.stdout:max_file_size 1099511627776 2024-03-21T10:38:35.119 INFO:teuthology.orchestra.run.smithi153.stdout:max_xattr_size 65536 2024-03-21T10:38:35.119 INFO:teuthology.orchestra.run.smithi153.stdout:required_client_features {} 2024-03-21T10:38:35.119 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure 0 2024-03-21T10:38:35.119 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure_osd_epoch 94 2024-03-21T10:38:35.119 INFO:teuthology.orchestra.run.smithi153.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:38:35.119 INFO:teuthology.orchestra.run.smithi153.stdout:max_mds 2 2024-03-21T10:38:35.119 INFO:teuthology.orchestra.run.smithi153.stdout:in 0,1 2024-03-21T10:38:35.120 INFO:teuthology.orchestra.run.smithi153.stdout:up {} 2024-03-21T10:38:35.120 INFO:teuthology.orchestra.run.smithi153.stdout:failed 0,1 2024-03-21T10:38:35.120 INFO:teuthology.orchestra.run.smithi153.stdout:damaged 2024-03-21T10:38:35.120 INFO:teuthology.orchestra.run.smithi153.stdout:stopped 2024-03-21T10:38:35.120 INFO:teuthology.orchestra.run.smithi153.stdout:data_pools [3] 2024-03-21T10:38:35.120 INFO:teuthology.orchestra.run.smithi153.stdout:metadata_pool 2 2024-03-21T10:38:35.120 INFO:teuthology.orchestra.run.smithi153.stdout:inline_data enabled 2024-03-21T10:38:35.120 INFO:teuthology.orchestra.run.smithi153.stdout:balancer 2024-03-21T10:38:35.120 INFO:teuthology.orchestra.run.smithi153.stdout:bal_rank_mask -1 2024-03-21T10:38:35.120 INFO:teuthology.orchestra.run.smithi153.stdout:standby_count_wanted 1 2024-03-21T10:38:35.120 INFO:teuthology.orchestra.run.smithi153.stdout:qdb_cluster leader: 0 members: 2024-03-21T10:38:35.121 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:38:35.121 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:38:35.121 INFO:teuthology.orchestra.run.smithi153.stdout:Standby daemons: 2024-03-21T10:38:35.121 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:38:35.121 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.zipckr{-1:34442} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6826/1103846295,v1:172.21.15.153:6827/1103846295] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:38:35.121 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.iuyluj{-1:34474} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6828/1738630832,v1:172.21.15.153:6829/1738630832] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:38:35.121 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.yyimst{-1:44313} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.174:6824/4127786797,v1:172.21.15.174:6825/4127786797] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:38:35.121 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.nhozag{-1:44687} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.174:6826/2767517845,v1:172.21.15.174:6827/2767517845] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:38:35.122 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:35.123+0000 7fb8abe00700 1 -- 172.21.15.153:0/1645314361 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb88c06e460 msgr2=0x7fb88c070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:38:35.122 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:35.123+0000 7fb8abe00700 1 --2- 172.21.15.153:0/1645314361 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb88c06e460 0x7fb88c070920 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fb898005ea0 tx=0x7fb898005e30 comp rx=0 tx=0).stop 2024-03-21T10:38:35.122 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:35.123+0000 7fb8abe00700 1 -- 172.21.15.153:0/1645314361 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb8a4075be0 msgr2=0x7fb8a419d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:38:35.122 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:35.123+0000 7fb8abe00700 1 --2- 172.21.15.153:0/1645314361 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb8a4075be0 0x7fb8a419d200 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fb890005b40 tx=0x7fb8900056a0 comp rx=0 tx=0).stop 2024-03-21T10:38:35.122 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:35.123+0000 7fb8abe00700 1 -- 172.21.15.153:0/1645314361 shutdown_connections 2024-03-21T10:38:35.123 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:35.123+0000 7fb8abe00700 1 --2- 172.21.15.153:0/1645314361 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb8a40752c0 0x7fb8a419ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:38:35.123 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:35.123+0000 7fb8abe00700 1 --2- 172.21.15.153:0/1645314361 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb88c06e460 0x7fb88c070920 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:38:35.123 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:35.123+0000 7fb8abe00700 1 --2- 172.21.15.153:0/1645314361 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb8a4075be0 0x7fb8a419d200 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:38:35.123 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:35.123+0000 7fb8abe00700 1 -- 172.21.15.153:0/1645314361 >> 172.21.15.153:0/1645314361 conn(0x7fb8a40fe840 msgr2=0x7fb8a40ff4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:38:35.123 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:35.124+0000 7fb8abe00700 1 -- 172.21.15.153:0/1645314361 shutdown_connections 2024-03-21T10:38:35.123 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:35.124+0000 7fb8abe00700 1 -- 172.21.15.153:0/1645314361 wait complete. 2024-03-21T10:38:35.123 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 32 2024-03-21T10:38:35.276 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:35.279+0000 7f26b5000700 1 -- 172.21.15.153:0/2933510844 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26b0103000 msgr2=0x7f26b01033e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:38:35.277 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:35.279+0000 7f26b5000700 1 --2- 172.21.15.153:0/2933510844 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26b0103000 0x7f26b01033e0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f26a0009910 tx=0x7f26a0009c20 comp rx=0 tx=0).stop 2024-03-21T10:38:35.277 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:35.279+0000 7f26b5000700 1 -- 172.21.15.153:0/2933510844 shutdown_connections 2024-03-21T10:38:35.277 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:35.279+0000 7f26b5000700 1 --2- 172.21.15.153:0/2933510844 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26b01039b0 0x7f26b0107a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:38:35.277 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:35.279+0000 7f26b5000700 1 --2- 172.21.15.153:0/2933510844 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26b0103000 0x7f26b01033e0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:38:35.277 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:35.279+0000 7f26b5000700 1 -- 172.21.15.153:0/2933510844 >> 172.21.15.153:0/2933510844 conn(0x7f26b00fe8d0 msgr2=0x7f26b0100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:38:35.277 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:35.279+0000 7f26b5000700 1 -- 172.21.15.153:0/2933510844 shutdown_connections 2024-03-21T10:38:35.277 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:35.279+0000 7f26b5000700 1 -- 172.21.15.153:0/2933510844 wait complete. 2024-03-21T10:38:35.279 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:35.280+0000 7f26b5000700 1 Processor -- start 2024-03-21T10:38:35.279 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:35.280+0000 7f26b5000700 1 -- start start 2024-03-21T10:38:35.279 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:35.281+0000 7f26b5000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26b0103000 0x7f26b019e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:38:35.280 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:35.281+0000 7f26b5000700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26b01039b0 0x7f26b019ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:38:35.280 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:35.281+0000 7f26b5000700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26b019f3c0 con 0x7f26b0103000 2024-03-21T10:38:35.280 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:35.281+0000 7f26b5000700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26b0198870 con 0x7f26b01039b0 2024-03-21T10:38:35.280 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:35.281+0000 7f26ade00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26b0103000 0x7f26b019e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:38:35.280 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:35.281+0000 7f26ade00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26b0103000 0x7f26b019e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:55266/0 (socket says 172.21.15.153:55266) 2024-03-21T10:38:35.280 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:35.281+0000 7f26ade00700 1 -- 172.21.15.153:0/1953376738 learned_addr learned my addr 172.21.15.153:0/1953376738 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:38:35.280 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:35.282+0000 7f26ad400700 1 --2- 172.21.15.153:0/1953376738 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26b01039b0 0x7f26b019ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:38:35.280 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:35.282+0000 7f26ade00700 1 -- 172.21.15.153:0/1953376738 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26b01039b0 msgr2=0x7f26b019ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:38:35.280 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:35.282+0000 7f26ade00700 1 --2- 172.21.15.153:0/1953376738 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26b01039b0 0x7f26b019ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:38:35.280 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:35.282+0000 7f26ade00700 1 -- 172.21.15.153:0/1953376738 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26a00095f0 con 0x7f26b0103000 2024-03-21T10:38:35.280 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:35.282+0000 7f26ad400700 1 --2- 172.21.15.153:0/1953376738 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26b01039b0 0x7f26b019ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:38:35.280 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:35.282+0000 7f26ade00700 1 --2- 172.21.15.153:0/1953376738 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26b0103000 0x7f26b019e7f0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f26a00092e0 tx=0x7f26a000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:38:35.280 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:35.282+0000 7f26a6a00700 1 -- 172.21.15.153:0/1953376738 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f26a001d070 con 0x7f26b0103000 2024-03-21T10:38:35.281 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:35.282+0000 7f26a6a00700 1 -- 172.21.15.153:0/1953376738 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f26a0005e00 con 0x7f26b0103000 2024-03-21T10:38:35.281 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:35.282+0000 7f26b5000700 1 -- 172.21.15.153:0/1953376738 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26b0198af0 con 0x7f26b0103000 2024-03-21T10:38:35.281 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:35.282+0000 7f26a6a00700 1 -- 172.21.15.153:0/1953376738 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f26a0017400 con 0x7f26b0103000 2024-03-21T10:38:35.281 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:35.282+0000 7f26b5000700 1 -- 172.21.15.153:0/1953376738 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26b0198fe0 con 0x7f26b0103000 2024-03-21T10:38:35.281 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:35.284+0000 7f26b5000700 1 -- 172.21.15.153:0/1953376738 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26b010b3a0 con 0x7f26b0103000 2024-03-21T10:38:35.281 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:35.285+0000 7f26a6a00700 1 -- 172.21.15.153:0/1953376738 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f26a000f460 con 0x7f26b0103000 2024-03-21T10:38:35.282 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:35.285+0000 7f26a6a00700 1 --2- 172.21.15.153:0/1953376738 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f269406e460 0x7f2694070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:38:35.283 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:35.286+0000 7f26a6a00700 1 -- 172.21.15.153:0/1953376738 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f26a008f910 con 0x7f26b0103000 2024-03-21T10:38:35.283 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:35.286+0000 7f26ad400700 1 --2- 172.21.15.153:0/1953376738 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f269406e460 0x7f2694070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:38:35.283 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:35.286+0000 7f26ad400700 1 --2- 172.21.15.153:0/1953376738 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f269406e460 0x7f2694070920 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f269c005fd0 tx=0x7f269c005ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:38:35.287 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:35.290+0000 7f26a6a00700 1 -- 172.21.15.153:0/1953376738 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f26a005c580 con 0x7f26b0103000 2024-03-21T10:38:35.538 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:35.540+0000 7f26b5000700 1 -- 172.21.15.153:0/1953376738 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f26b002cce0 con 0x7f269406e460 2024-03-21T10:38:35.540 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:35.543+0000 7f26a6a00700 1 -- 172.21.15.153:0/1953376738 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f26b002cce0 con 0x7f269406e460 2024-03-21T10:38:35.540 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:38:35.541 INFO:teuthology.orchestra.run.smithi153.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232", 2024-03-21T10:38:35.541 INFO:teuthology.orchestra.run.smithi153.stdout: "in_progress": true, 2024-03-21T10:38:35.541 INFO:teuthology.orchestra.run.smithi153.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-21T10:38:35.541 INFO:teuthology.orchestra.run.smithi153.stdout: "services_complete": [ 2024-03-21T10:38:35.541 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr", 2024-03-21T10:38:35.541 INFO:teuthology.orchestra.run.smithi153.stdout: "mon", 2024-03-21T10:38:35.541 INFO:teuthology.orchestra.run.smithi153.stdout: "mds", 2024-03-21T10:38:35.541 INFO:teuthology.orchestra.run.smithi153.stdout: "crash", 2024-03-21T10:38:35.541 INFO:teuthology.orchestra.run.smithi153.stdout: "osd" 2024-03-21T10:38:35.541 INFO:teuthology.orchestra.run.smithi153.stdout: ], 2024-03-21T10:38:35.541 INFO:teuthology.orchestra.run.smithi153.stdout: "progress": "16/23 daemons upgraded", 2024-03-21T10:38:35.541 INFO:teuthology.orchestra.run.smithi153.stdout: "message": "Currently upgrading mds daemons", 2024-03-21T10:38:35.541 INFO:teuthology.orchestra.run.smithi153.stdout: "is_paused": false 2024-03-21T10:38:35.541 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:38:35.542 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:35.545+0000 7f26b5000700 1 -- 172.21.15.153:0/1953376738 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f269406e460 msgr2=0x7f2694070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:38:35.542 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:35.545+0000 7f26b5000700 1 --2- 172.21.15.153:0/1953376738 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f269406e460 0x7f2694070920 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f269c005fd0 tx=0x7f269c005ee0 comp rx=0 tx=0).stop 2024-03-21T10:38:35.542 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:35.545+0000 7f26b5000700 1 -- 172.21.15.153:0/1953376738 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26b0103000 msgr2=0x7f26b019e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:38:35.543 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:35.545+0000 7f26b5000700 1 --2- 172.21.15.153:0/1953376738 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26b0103000 0x7f26b019e7f0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f26a00092e0 tx=0x7f26a000be20 comp rx=0 tx=0).stop 2024-03-21T10:38:35.543 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:35.546+0000 7f26b5000700 1 -- 172.21.15.153:0/1953376738 shutdown_connections 2024-03-21T10:38:35.543 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:35.546+0000 7f26b5000700 1 --2- 172.21.15.153:0/1953376738 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f269406e460 0x7f2694070920 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:38:35.543 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:35.546+0000 7f26b5000700 1 --2- 172.21.15.153:0/1953376738 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26b0103000 0x7f26b019e7f0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:38:35.543 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:35.546+0000 7f26b5000700 1 --2- 172.21.15.153:0/1953376738 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26b01039b0 0x7f26b019ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:38:35.543 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:35.546+0000 7f26b5000700 1 -- 172.21.15.153:0/1953376738 >> 172.21.15.153:0/1953376738 conn(0x7f26b00fe8d0 msgr2=0x7f26b00ffee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:38:35.543 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:35.546+0000 7f26b5000700 1 -- 172.21.15.153:0/1953376738 shutdown_connections 2024-03-21T10:38:35.544 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:35.546+0000 7f26b5000700 1 -- 172.21.15.153:0/1953376738 wait complete. 2024-03-21T10:38:35.696 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:35.698+0000 7f2639e00700 1 -- 172.21.15.153:0/1027928172 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f263410d280 msgr2=0x7f263410d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:38:35.696 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:35.698+0000 7f2639e00700 1 --2- 172.21.15.153:0/1027928172 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f263410d280 0x7f263410d660 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f2624009910 tx=0x7f2624009c20 comp rx=0 tx=0).stop 2024-03-21T10:38:35.696 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:35.698+0000 7f2639e00700 1 -- 172.21.15.153:0/1027928172 shutdown_connections 2024-03-21T10:38:35.696 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:35.698+0000 7f2639e00700 1 --2- 172.21.15.153:0/1027928172 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26341071d0 0x7f2634107650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:38:35.696 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:35.698+0000 7f2639e00700 1 --2- 172.21.15.153:0/1027928172 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f263410d280 0x7f263410d660 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:38:35.696 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:35.698+0000 7f2639e00700 1 -- 172.21.15.153:0/1027928172 >> 172.21.15.153:0/1027928172 conn(0x7f2634076990 msgr2=0x7f2634078db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:38:35.696 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:38:35 smithi153 ceph-mon[113733]: pgmap v803: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:38:35.697 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:38:35 smithi153 ceph-mon[113733]: from='client.35438 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:38:35.697 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:38:35 smithi153 ceph-mon[113733]: from='client.35442 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:38:35.697 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:38:35 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/2343552186' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:38:35.697 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:38:35 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/1645314361' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:38:35.698 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:35.698+0000 7f2639e00700 1 -- 172.21.15.153:0/1027928172 shutdown_connections 2024-03-21T10:38:35.698 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:35.698+0000 7f2639e00700 1 -- 172.21.15.153:0/1027928172 wait complete. 2024-03-21T10:38:35.698 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:35.700+0000 7f2639e00700 1 Processor -- start 2024-03-21T10:38:35.698 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:35.700+0000 7f2639e00700 1 -- start start 2024-03-21T10:38:35.699 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:35.701+0000 7f2639e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26341071d0 0x7f263419ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:38:35.699 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:35.701+0000 7f2639e00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f263410d280 0x7f263419d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:38:35.699 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:35.701+0000 7f2639e00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f263419d890 con 0x7f26341071d0 2024-03-21T10:38:35.699 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:35.701+0000 7f2639e00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26341a05c0 con 0x7f263410d280 2024-03-21T10:38:35.699 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:35.701+0000 7f2632a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26341071d0 0x7f263419ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:38:35.699 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:35.701+0000 7f2632a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26341071d0 0x7f263419ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:55276/0 (socket says 172.21.15.153:55276) 2024-03-21T10:38:35.699 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:35.701+0000 7f2632a00700 1 -- 172.21.15.153:0/2809282883 learned_addr learned my addr 172.21.15.153:0/2809282883 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:38:35.699 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:35.701+0000 7f262be00700 1 --2- 172.21.15.153:0/2809282883 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f263410d280 0x7f263419d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:38:35.699 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:35.701+0000 7f2632a00700 1 -- 172.21.15.153:0/2809282883 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f263410d280 msgr2=0x7f263419d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:38:35.699 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:35.701+0000 7f2632a00700 1 --2- 172.21.15.153:0/2809282883 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f263410d280 0x7f263419d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:38:35.699 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:35.701+0000 7f2632a00700 1 -- 172.21.15.153:0/2809282883 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26240095f0 con 0x7f26341071d0 2024-03-21T10:38:35.700 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:38:35 smithi174 ceph-mon[88004]: pgmap v803: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:38:35.700 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:38:35 smithi174 ceph-mon[88004]: from='client.35438 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:38:35.700 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:38:35 smithi174 ceph-mon[88004]: from='client.35442 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:38:35.700 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:38:35 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/2343552186' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:38:35.700 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:38:35 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/1645314361' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:38:35.702 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:35.702+0000 7f2632a00700 1 --2- 172.21.15.153:0/2809282883 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26341071d0 0x7f263419ccc0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f26240092e0 tx=0x7f262400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:38:35.702 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:35.702+0000 7f2638a00700 1 -- 172.21.15.153:0/2809282883 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f262401d070 con 0x7f26341071d0 2024-03-21T10:38:35.702 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:35.702+0000 7f2638a00700 1 -- 172.21.15.153:0/2809282883 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2624005e00 con 0x7f26341071d0 2024-03-21T10:38:35.702 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:35.702+0000 7f2639e00700 1 -- 172.21.15.153:0/2809282883 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26341a0840 con 0x7f26341071d0 2024-03-21T10:38:35.702 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:35.702+0000 7f2639e00700 1 -- 172.21.15.153:0/2809282883 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26341a0db0 con 0x7f26341071d0 2024-03-21T10:38:35.702 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:35.703+0000 7f2638a00700 1 -- 172.21.15.153:0/2809282883 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2624017400 con 0x7f26341071d0 2024-03-21T10:38:35.702 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:35.703+0000 7f2639e00700 1 -- 172.21.15.153:0/2809282883 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f263404ea90 con 0x7f26341071d0 2024-03-21T10:38:35.702 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:35.705+0000 7f2638a00700 1 -- 172.21.15.153:0/2809282883 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f262400f460 con 0x7f26341071d0 2024-03-21T10:38:35.703 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:35.706+0000 7f2638a00700 1 --2- 172.21.15.153:0/2809282883 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f261806e460 0x7f2618070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:38:35.703 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:35.706+0000 7f2638a00700 1 -- 172.21.15.153:0/2809282883 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f262408fa80 con 0x7f26341071d0 2024-03-21T10:38:35.703 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:35.706+0000 7f262be00700 1 --2- 172.21.15.153:0/2809282883 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f261806e460 0x7f2618070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:38:35.704 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:35.707+0000 7f262be00700 1 --2- 172.21.15.153:0/2809282883 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f261806e460 0x7f2618070920 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f261c005950 tx=0x7f261c0058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:38:35.707 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:35.710+0000 7f2638a00700 1 -- 172.21.15.153:0/2809282883 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2624058aa0 con 0x7f26341071d0 2024-03-21T10:38:36.016 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:36.019+0000 7f2639e00700 1 -- 172.21.15.153:0/2809282883 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f263419dfe0 con 0x7f26341071d0 2024-03-21T10:38:36.016 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:36.019+0000 7f2638a00700 1 -- 172.21.15.153:0/2809282883 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4493 (secure 0 0 0) 0x7f262405c160 con 0x7f26341071d0 2024-03-21T10:38:36.017 INFO:teuthology.orchestra.run.smithi153.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-03-21T10:38:36.017 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-03-21T10:38:36.017 INFO:teuthology.orchestra.run.smithi153.stdout: host smithi174 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi174/config 2024-03-21T10:38:36.017 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/podman: timeout after 895 seconds 2024-03-21T10:38:36.017 INFO:teuthology.orchestra.run.smithi153.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 -e NODE_NAME=smithi174 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/run/ceph:z -v /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpd910r3_i:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 inventory --format=json-pretty --filter-for-batch 2024-03-21T10:38:36.018 INFO:teuthology.orchestra.run.smithi153.stdout:Traceback (most recent call last): 2024-03-21T10:38:36.018 INFO:teuthology.orchestra.run.smithi153.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-03-21T10:38:36.018 INFO:teuthology.orchestra.run.smithi153.stdout: return _run_code(code, main_globals, None, 2024-03-21T10:38:36.018 INFO:teuthology.orchestra.run.smithi153.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-03-21T10:38:36.018 INFO:teuthology.orchestra.run.smithi153.stdout: exec(code, run_globals) 2024-03-21T10:38:36.018 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 5606, in 2024-03-21T10:38:36.018 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 5594, in main 2024-03-21T10:38:36.018 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 406, in _infer_config 2024-03-21T10:38:36.018 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 321, in _infer_fsid 2024-03-21T10:38:36.018 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 434, in _infer_image 2024-03-21T10:38:36.018 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 308, in _validate_fsid 2024-03-21T10:38:36.018 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 3357, in command_ceph_volume 2024-03-21T10:38:36.018 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-03-21T10:38:36.018 INFO:teuthology.orchestra.run.smithi153.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 -e NODE_NAME=smithi174 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/run/ceph:z -v /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpd910r3_i:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 inventory --format=json-pretty --filter-for-batch 2024-03-21T10:38:36.018 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-21T10:38:36.019 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is degraded 2024-03-21T10:38:36.019 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-21T10:38:36.019 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-21T10:38:36.019 INFO:teuthology.orchestra.run.smithi153.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-21T10:38:36.019 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-21T10:38:36.020 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:36.023+0000 7f2639e00700 1 -- 172.21.15.153:0/2809282883 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f261806e460 msgr2=0x7f2618070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:38:36.020 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:36.023+0000 7f2639e00700 1 --2- 172.21.15.153:0/2809282883 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f261806e460 0x7f2618070920 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f261c005950 tx=0x7f261c0058e0 comp rx=0 tx=0).stop 2024-03-21T10:38:36.020 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:36.023+0000 7f2639e00700 1 -- 172.21.15.153:0/2809282883 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26341071d0 msgr2=0x7f263419ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:38:36.020 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:36.023+0000 7f2639e00700 1 --2- 172.21.15.153:0/2809282883 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26341071d0 0x7f263419ccc0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f26240092e0 tx=0x7f262400be20 comp rx=0 tx=0).stop 2024-03-21T10:38:36.020 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:36.023+0000 7f2639e00700 1 -- 172.21.15.153:0/2809282883 shutdown_connections 2024-03-21T10:38:36.020 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:36.023+0000 7f2639e00700 1 --2- 172.21.15.153:0/2809282883 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f261806e460 0x7f2618070920 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:38:36.020 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:36.023+0000 7f2639e00700 1 --2- 172.21.15.153:0/2809282883 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26341071d0 0x7f263419ccc0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:38:36.021 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:36.023+0000 7f2639e00700 1 --2- 172.21.15.153:0/2809282883 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f263410d280 0x7f263419d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:38:36.021 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:36.023+0000 7f2639e00700 1 -- 172.21.15.153:0/2809282883 >> 172.21.15.153:0/2809282883 conn(0x7f2634076990 msgr2=0x7f2634078310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:38:36.021 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:36.023+0000 7f2639e00700 1 -- 172.21.15.153:0/2809282883 shutdown_connections 2024-03-21T10:38:36.021 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:38:36.023+0000 7f2639e00700 1 -- 172.21.15.153:0/2809282883 wait complete. 2024-03-21T10:38:36.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:38:36 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/2809282883' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:38:36.803 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:38:36 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/2809282883' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:38:37.698 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:38:37 smithi174 ceph-mon[88004]: from='client.35454 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:38:37.698 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:38:37 smithi174 ceph-mon[88004]: pgmap v804: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:38:37.803 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:38:37 smithi153 ceph-mon[113733]: from='client.35454 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:38:37.803 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:38:37 smithi153 ceph-mon[113733]: pgmap v804: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:38:39.698 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:38:39 smithi174 ceph-mon[88004]: pgmap v805: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:38:39.803 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:38:39 smithi153 ceph-mon[113733]: pgmap v805: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:38:41.698 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:38:41 smithi174 ceph-mon[88004]: pgmap v806: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:38:41.803 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:38:41 smithi153 ceph-mon[113733]: pgmap v806: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:38:43.698 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:38:43 smithi174 ceph-mon[88004]: pgmap v807: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:38:43.803 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:38:43 smithi153 ceph-mon[113733]: pgmap v807: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:38:45.698 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:38:45 smithi174 ceph-mon[88004]: pgmap v808: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:38:45.734 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:38:45 smithi153 ceph-mon[113733]: pgmap v808: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:38:46.698 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:38:46 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:38:46.803 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:38:46 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:38:47.698 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:38:47 smithi174 ceph-mon[88004]: pgmap v809: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:38:47.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:38:47 smithi153 ceph-mon[113733]: pgmap v809: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:38:49.698 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:38:49 smithi174 ceph-mon[88004]: pgmap v810: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:38:49.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:38:49 smithi153 ceph-mon[113733]: pgmap v810: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:38:51.698 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:38:51 smithi174 ceph-mon[88004]: pgmap v811: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:38:51.803 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:38:51 smithi153 ceph-mon[113733]: pgmap v811: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:38:53.803 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:38:53 smithi153 ceph-mon[113733]: pgmap v812: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:38:53.861 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:38:53 smithi174 ceph-mon[88004]: pgmap v812: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:38:55.803 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:38:55 smithi153 ceph-mon[113733]: pgmap v813: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:38:55.948 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:38:55 smithi174 ceph-mon[88004]: pgmap v813: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:38:57.803 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:38:57 smithi153 ceph-mon[113733]: pgmap v814: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:38:57.948 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:38:57 smithi174 ceph-mon[88004]: pgmap v814: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:38:59.803 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:38:59 smithi153 ceph-mon[113733]: pgmap v815: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:38:59.948 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:38:59 smithi174 ceph-mon[88004]: pgmap v815: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:39:01.803 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:39:01 smithi153 ceph-mon[113733]: pgmap v816: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:39:01.803 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:39:01 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:39:01.948 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:39:01 smithi174 ceph-mon[88004]: pgmap v816: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:39:01.948 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:39:01 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:39:03.803 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:39:03 smithi153 ceph-mon[113733]: pgmap v817: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:39:03.948 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:39:03 smithi174 ceph-mon[88004]: pgmap v817: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:39:05.803 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:39:05 smithi153 ceph-mon[113733]: pgmap v818: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:39:05.948 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:39:05 smithi174 ceph-mon[88004]: pgmap v818: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:39:06.195 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:06.196+0000 7f8081400700 1 -- 172.21.15.153:0/4203056371 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f807c103000 msgr2=0x7f807c1033e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:39:06.195 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:06.196+0000 7f8081400700 1 --2- 172.21.15.153:0/4203056371 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f807c103000 0x7f807c1033e0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f806c009910 tx=0x7f806c009c20 comp rx=0 tx=0).stop 2024-03-21T10:39:06.195 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:06.197+0000 7f8081400700 1 -- 172.21.15.153:0/4203056371 shutdown_connections 2024-03-21T10:39:06.195 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:06.197+0000 7f8081400700 1 --2- 172.21.15.153:0/4203056371 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f807c1039b0 0x7f807c107a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:39:06.195 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:06.197+0000 7f8081400700 1 --2- 172.21.15.153:0/4203056371 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f807c103000 0x7f807c1033e0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:39:06.195 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:06.197+0000 7f8081400700 1 -- 172.21.15.153:0/4203056371 >> 172.21.15.153:0/4203056371 conn(0x7f807c0fe8d0 msgr2=0x7f807c100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:39:06.195 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:06.197+0000 7f8081400700 1 -- 172.21.15.153:0/4203056371 shutdown_connections 2024-03-21T10:39:06.195 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:06.197+0000 7f8081400700 1 -- 172.21.15.153:0/4203056371 wait complete. 2024-03-21T10:39:06.196 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:06.198+0000 7f8081400700 1 Processor -- start 2024-03-21T10:39:06.196 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:06.198+0000 7f8081400700 1 -- start start 2024-03-21T10:39:06.197 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:06.199+0000 7f8081400700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f807c103000 0x7f807c19e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:39:06.197 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:06.199+0000 7f8081400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f807c1039b0 0x7f807c19ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:39:06.197 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:06.199+0000 7f8081400700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f807c19f3c0 con 0x7f807c1039b0 2024-03-21T10:39:06.197 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:06.199+0000 7f8081400700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f807c198870 con 0x7f807c103000 2024-03-21T10:39:06.198 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:06.199+0000 7f8079e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f807c1039b0 0x7f807c19ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:39:06.198 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:06.199+0000 7f8079e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f807c1039b0 0x7f807c19ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:37238/0 (socket says 172.21.15.153:37238) 2024-03-21T10:39:06.198 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:06.199+0000 7f8079e00700 1 -- 172.21.15.153:0/2986979722 learned_addr learned my addr 172.21.15.153:0/2986979722 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:39:06.198 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:06.199+0000 7f807a800700 1 --2- 172.21.15.153:0/2986979722 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f807c103000 0x7f807c19e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:39:06.198 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:06.200+0000 7f8079e00700 1 -- 172.21.15.153:0/2986979722 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f807c103000 msgr2=0x7f807c19e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:39:06.198 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:06.200+0000 7f8079e00700 1 --2- 172.21.15.153:0/2986979722 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f807c103000 0x7f807c19e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:39:06.198 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:06.200+0000 7f8079e00700 1 -- 172.21.15.153:0/2986979722 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f806c0095f0 con 0x7f807c1039b0 2024-03-21T10:39:06.199 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:06.200+0000 7f8079e00700 1 --2- 172.21.15.153:0/2986979722 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f807c1039b0 0x7f807c19ed30 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f806800d900 tx=0x7f806800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:39:06.199 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:06.200+0000 7f8073400700 1 -- 172.21.15.153:0/2986979722 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8068009900 con 0x7f807c1039b0 2024-03-21T10:39:06.199 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:06.201+0000 7f8073400700 1 -- 172.21.15.153:0/2986979722 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8068010460 con 0x7f807c1039b0 2024-03-21T10:39:06.199 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:06.201+0000 7f8073400700 1 -- 172.21.15.153:0/2986979722 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f806800f5f0 con 0x7f807c1039b0 2024-03-21T10:39:06.199 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:06.201+0000 7f8081400700 1 -- 172.21.15.153:0/2986979722 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f807c198b50 con 0x7f807c1039b0 2024-03-21T10:39:06.199 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:06.201+0000 7f8081400700 1 -- 172.21.15.153:0/2986979722 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f807c1990a0 con 0x7f807c1039b0 2024-03-21T10:39:06.200 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:06.202+0000 7f8081400700 1 -- 172.21.15.153:0/2986979722 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f807c10b3a0 con 0x7f807c1039b0 2024-03-21T10:39:06.201 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:06.203+0000 7f8073400700 1 -- 172.21.15.153:0/2986979722 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f80680105d0 con 0x7f807c1039b0 2024-03-21T10:39:06.201 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:06.204+0000 7f8073400700 1 --2- 172.21.15.153:0/2986979722 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f806006e460 0x7f8060070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:39:06.202 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:06.204+0000 7f8073400700 1 -- 172.21.15.153:0/2986979722 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f806808dfe0 con 0x7f807c1039b0 2024-03-21T10:39:06.202 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:06.204+0000 7f807a800700 1 --2- 172.21.15.153:0/2986979722 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f806006e460 0x7f8060070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:39:06.202 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:06.205+0000 7f807a800700 1 --2- 172.21.15.153:0/2986979722 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f806006e460 0x7f8060070920 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f806c00b5c0 tx=0x7f806c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:39:06.207 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:06.209+0000 7f8073400700 1 -- 172.21.15.153:0/2986979722 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f806805abf0 con 0x7f807c1039b0 2024-03-21T10:39:06.455 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:06.457+0000 7f8081400700 1 -- 172.21.15.153:0/2986979722 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f807c199310 con 0x7f806006e460 2024-03-21T10:39:06.456 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:06.459+0000 7f8073400700 1 -- 172.21.15.153:0/2986979722 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f807c199310 con 0x7f806006e460 2024-03-21T10:39:06.460 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:06.461+0000 7f8081400700 1 -- 172.21.15.153:0/2986979722 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f806006e460 msgr2=0x7f8060070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:39:06.460 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:06.461+0000 7f8081400700 1 --2- 172.21.15.153:0/2986979722 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f806006e460 0x7f8060070920 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f806c00b5c0 tx=0x7f806c01a040 comp rx=0 tx=0).stop 2024-03-21T10:39:06.460 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:06.461+0000 7f8081400700 1 -- 172.21.15.153:0/2986979722 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f807c1039b0 msgr2=0x7f807c19ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:39:06.460 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:06.461+0000 7f8081400700 1 --2- 172.21.15.153:0/2986979722 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f807c1039b0 0x7f807c19ed30 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f806800d900 tx=0x7f806800dc10 comp rx=0 tx=0).stop 2024-03-21T10:39:06.460 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:06.462+0000 7f8081400700 1 -- 172.21.15.153:0/2986979722 shutdown_connections 2024-03-21T10:39:06.460 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:06.462+0000 7f8081400700 1 --2- 172.21.15.153:0/2986979722 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f807c103000 0x7f807c19e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:39:06.460 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:06.462+0000 7f8081400700 1 --2- 172.21.15.153:0/2986979722 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f806006e460 0x7f8060070920 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:39:06.460 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:06.462+0000 7f8081400700 1 --2- 172.21.15.153:0/2986979722 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f807c1039b0 0x7f807c19ed30 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:39:06.460 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:06.462+0000 7f8081400700 1 -- 172.21.15.153:0/2986979722 >> 172.21.15.153:0/2986979722 conn(0x7f807c0fe8d0 msgr2=0x7f807c0ffee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:39:06.461 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:06.462+0000 7f8081400700 1 -- 172.21.15.153:0/2986979722 shutdown_connections 2024-03-21T10:39:06.461 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:06.462+0000 7f8081400700 1 -- 172.21.15.153:0/2986979722 wait complete. 2024-03-21T10:39:06.476 INFO:teuthology.orchestra.run.smithi153.stdout:true 2024-03-21T10:39:06.614 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:06.616+0000 7f3c51800700 1 -- 172.21.15.153:0/1501349569 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c4c10d280 msgr2=0x7f3c4c10d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:39:06.614 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:06.616+0000 7f3c51800700 1 --2- 172.21.15.153:0/1501349569 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c4c10d280 0x7f3c4c10d660 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c009910 tx=0x7f3c3c009c20 comp rx=0 tx=0).stop 2024-03-21T10:39:06.614 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:06.616+0000 7f3c51800700 1 -- 172.21.15.153:0/1501349569 shutdown_connections 2024-03-21T10:39:06.614 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:06.616+0000 7f3c51800700 1 --2- 172.21.15.153:0/1501349569 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c4c1071d0 0x7f3c4c107650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:39:06.614 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:06.616+0000 7f3c51800700 1 --2- 172.21.15.153:0/1501349569 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c4c10d280 0x7f3c4c10d660 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:39:06.614 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:06.616+0000 7f3c51800700 1 -- 172.21.15.153:0/1501349569 >> 172.21.15.153:0/1501349569 conn(0x7f3c4c076990 msgr2=0x7f3c4c078db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:39:06.615 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:06.616+0000 7f3c51800700 1 -- 172.21.15.153:0/1501349569 shutdown_connections 2024-03-21T10:39:06.615 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:06.616+0000 7f3c51800700 1 -- 172.21.15.153:0/1501349569 wait complete. 2024-03-21T10:39:06.616 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:06.617+0000 7f3c51800700 1 Processor -- start 2024-03-21T10:39:06.616 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:06.617+0000 7f3c51800700 1 -- start start 2024-03-21T10:39:06.616 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:06.617+0000 7f3c51800700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c4c1071d0 0x7f3c4c19ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:39:06.616 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:06.617+0000 7f3c51800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c4c10d280 0x7f3c4c19d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:39:06.617 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:06.617+0000 7f3c51800700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c4c19d800 con 0x7f3c4c10d280 2024-03-21T10:39:06.617 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:06.617+0000 7f3c51800700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c4c19d970 con 0x7f3c4c1071d0 2024-03-21T10:39:06.617 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:06.618+0000 7f3c4a000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c4c10d280 0x7f3c4c19d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:39:06.617 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:06.618+0000 7f3c4a000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c4c10d280 0x7f3c4c19d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:37258/0 (socket says 172.21.15.153:37258) 2024-03-21T10:39:06.617 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:06.618+0000 7f3c4a000700 1 -- 172.21.15.153:0/596973269 learned_addr learned my addr 172.21.15.153:0/596973269 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:39:06.617 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:06.618+0000 7f3c4aa00700 1 --2- 172.21.15.153:0/596973269 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c4c1071d0 0x7f3c4c19ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:39:06.617 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:06.618+0000 7f3c4a000700 1 -- 172.21.15.153:0/596973269 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c4c1071d0 msgr2=0x7f3c4c19ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:39:06.617 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:06.618+0000 7f3c4a000700 1 --2- 172.21.15.153:0/596973269 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c4c1071d0 0x7f3c4c19ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:39:06.617 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:06.618+0000 7f3c4a000700 1 -- 172.21.15.153:0/596973269 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c3c0095f0 con 0x7f3c4c10d280 2024-03-21T10:39:06.617 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:06.619+0000 7f3c4a000700 1 --2- 172.21.15.153:0/596973269 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c4c10d280 0x7f3c4c19d200 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f3c3800c930 tx=0x7f3c3800ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:39:06.618 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:06.619+0000 7f3c43400700 1 -- 172.21.15.153:0/596973269 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3c38007ad0 con 0x7f3c4c10d280 2024-03-21T10:39:06.618 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:06.619+0000 7f3c43400700 1 -- 172.21.15.153:0/596973269 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3c38007c30 con 0x7f3c4c10d280 2024-03-21T10:39:06.618 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:06.619+0000 7f3c51800700 1 -- 172.21.15.153:0/596973269 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c4c1a0710 con 0x7f3c4c10d280 2024-03-21T10:39:06.618 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:06.619+0000 7f3c43400700 1 -- 172.21.15.153:0/596973269 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3c38018610 con 0x7f3c4c10d280 2024-03-21T10:39:06.618 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:06.619+0000 7f3c51800700 1 -- 172.21.15.153:0/596973269 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c4c1a0ce0 con 0x7f3c4c10d280 2024-03-21T10:39:06.618 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:06.620+0000 7f3c51800700 1 -- 172.21.15.153:0/596973269 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c4c04ea90 con 0x7f3c4c10d280 2024-03-21T10:39:06.619 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:06.621+0000 7f3c43400700 1 -- 172.21.15.153:0/596973269 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f3c3800f450 con 0x7f3c4c10d280 2024-03-21T10:39:06.620 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:06.622+0000 7f3c43400700 1 --2- 172.21.15.153:0/596973269 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f3c3006e460 0x7f3c30070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:39:06.620 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:06.622+0000 7f3c43400700 1 -- 172.21.15.153:0/596973269 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f3c3808e6a0 con 0x7f3c4c10d280 2024-03-21T10:39:06.620 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:06.622+0000 7f3c4aa00700 1 --2- 172.21.15.153:0/596973269 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f3c3006e460 0x7f3c30070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:39:06.621 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:06.623+0000 7f3c4aa00700 1 --2- 172.21.15.153:0/596973269 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f3c3006e460 0x7f3c30070920 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c006010 tx=0x7f3c3c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:39:06.625 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:06.627+0000 7f3c43400700 1 -- 172.21.15.153:0/596973269 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f3c38057740 con 0x7f3c4c10d280 2024-03-21T10:39:06.851 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:06.853+0000 7f3c51800700 1 -- 172.21.15.153:0/596973269 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3c4c1a1190 con 0x7f3c3006e460 2024-03-21T10:39:06.853 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:06.855+0000 7f3c43400700 1 -- 172.21.15.153:0/596973269 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f3c4c1a1190 con 0x7f3c3006e460 2024-03-21T10:39:06.856 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:06.857+0000 7f3c51800700 1 -- 172.21.15.153:0/596973269 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f3c3006e460 msgr2=0x7f3c30070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:39:06.856 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:06.857+0000 7f3c51800700 1 --2- 172.21.15.153:0/596973269 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f3c3006e460 0x7f3c30070920 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c006010 tx=0x7f3c3c01a040 comp rx=0 tx=0).stop 2024-03-21T10:39:06.856 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:06.858+0000 7f3c51800700 1 -- 172.21.15.153:0/596973269 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c4c10d280 msgr2=0x7f3c4c19d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:39:06.856 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:06.858+0000 7f3c51800700 1 --2- 172.21.15.153:0/596973269 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c4c10d280 0x7f3c4c19d200 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f3c3800c930 tx=0x7f3c3800ccf0 comp rx=0 tx=0).stop 2024-03-21T10:39:06.856 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:06.858+0000 7f3c51800700 1 -- 172.21.15.153:0/596973269 shutdown_connections 2024-03-21T10:39:06.856 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:06.858+0000 7f3c51800700 1 --2- 172.21.15.153:0/596973269 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c4c1071d0 0x7f3c4c19ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:39:06.856 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:06.858+0000 7f3c51800700 1 --2- 172.21.15.153:0/596973269 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f3c3006e460 0x7f3c30070920 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:39:06.856 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:06.858+0000 7f3c51800700 1 --2- 172.21.15.153:0/596973269 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3c4c10d280 0x7f3c4c19d200 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:39:06.856 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:06.858+0000 7f3c51800700 1 -- 172.21.15.153:0/596973269 >> 172.21.15.153:0/596973269 conn(0x7f3c4c076990 msgr2=0x7f3c4c078310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:39:06.857 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:06.858+0000 7f3c51800700 1 -- 172.21.15.153:0/596973269 shutdown_connections 2024-03-21T10:39:06.857 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:06.858+0000 7f3c51800700 1 -- 172.21.15.153:0/596973269 wait complete. 2024-03-21T10:39:07.006 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.008+0000 7f065c400700 1 -- 172.21.15.153:0/2571413339 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0654068670 msgr2=0x7f0654068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:39:07.007 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.008+0000 7f065c400700 1 --2- 172.21.15.153:0/2571413339 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0654068670 0x7f0654068a50 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f0648009910 tx=0x7f0648009c20 comp rx=0 tx=0).stop 2024-03-21T10:39:07.007 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.008+0000 7f065c400700 1 -- 172.21.15.153:0/2571413339 shutdown_connections 2024-03-21T10:39:07.007 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.008+0000 7f065c400700 1 --2- 172.21.15.153:0/2571413339 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0654069020 0x7f0654105790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:39:07.007 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.008+0000 7f065c400700 1 --2- 172.21.15.153:0/2571413339 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0654068670 0x7f0654068a50 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:39:07.007 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.008+0000 7f065c400700 1 -- 172.21.15.153:0/2571413339 >> 172.21.15.153:0/2571413339 conn(0x7f06540755a0 msgr2=0x7f06540759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:39:07.007 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.009+0000 7f065c400700 1 -- 172.21.15.153:0/2571413339 shutdown_connections 2024-03-21T10:39:07.007 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.009+0000 7f065c400700 1 -- 172.21.15.153:0/2571413339 wait complete. 2024-03-21T10:39:07.008 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.010+0000 7f065c400700 1 Processor -- start 2024-03-21T10:39:07.008 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.010+0000 7f065c400700 1 -- start start 2024-03-21T10:39:07.008 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.010+0000 7f065c400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0654068670 0x7f065419e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:39:07.008 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.010+0000 7f065c400700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0654069020 0x7f065419ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:39:07.008 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.010+0000 7f065c400700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f065419f2e0 con 0x7f0654068670 2024-03-21T10:39:07.008 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.010+0000 7f065c400700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0654198790 con 0x7f0654069020 2024-03-21T10:39:07.008 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.010+0000 7f0659a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0654068670 0x7f065419e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:39:07.009 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.010+0000 7f0659a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0654068670 0x7f065419e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:37290/0 (socket says 172.21.15.153:37290) 2024-03-21T10:39:07.009 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.010+0000 7f0659a00700 1 -- 172.21.15.153:0/2673370692 learned_addr learned my addr 172.21.15.153:0/2673370692 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:39:07.009 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.011+0000 7f0659000700 1 --2- 172.21.15.153:0/2673370692 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0654069020 0x7f065419ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:39:07.010 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.011+0000 7f0659a00700 1 -- 172.21.15.153:0/2673370692 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0654069020 msgr2=0x7f065419ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:39:07.010 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.011+0000 7f0659a00700 1 --2- 172.21.15.153:0/2673370692 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0654069020 0x7f065419ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:39:07.010 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.011+0000 7f0659a00700 1 -- 172.21.15.153:0/2673370692 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06480095f0 con 0x7f0654068670 2024-03-21T10:39:07.010 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.011+0000 7f0659000700 1 --2- 172.21.15.153:0/2673370692 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0654069020 0x7f065419ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:39:07.010 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.011+0000 7f0659a00700 1 --2- 172.21.15.153:0/2673370692 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0654068670 0x7f065419e710 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f06480092e0 tx=0x7f064800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:39:07.010 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.011+0000 7f0652000700 1 -- 172.21.15.153:0/2673370692 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f064801d070 con 0x7f0654068670 2024-03-21T10:39:07.010 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.011+0000 7f065c400700 1 -- 172.21.15.153:0/2673370692 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0654198a10 con 0x7f0654068670 2024-03-21T10:39:07.010 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.011+0000 7f0652000700 1 -- 172.21.15.153:0/2673370692 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0648005e00 con 0x7f0654068670 2024-03-21T10:39:07.010 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.012+0000 7f0652000700 1 -- 172.21.15.153:0/2673370692 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0648017400 con 0x7f0654068670 2024-03-21T10:39:07.010 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.012+0000 7f065c400700 1 -- 172.21.15.153:0/2673370692 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0654198f00 con 0x7f0654068670 2024-03-21T10:39:07.011 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.013+0000 7f065c400700 1 -- 172.21.15.153:0/2673370692 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f065404ea90 con 0x7f0654068670 2024-03-21T10:39:07.011 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.014+0000 7f0652000700 1 -- 172.21.15.153:0/2673370692 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f064800f460 con 0x7f0654068670 2024-03-21T10:39:07.012 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.015+0000 7f0652000700 1 --2- 172.21.15.153:0/2673370692 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f063c06e460 0x7f063c070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:39:07.013 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.015+0000 7f0652000700 1 -- 172.21.15.153:0/2673370692 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f064808fa00 con 0x7f0654068670 2024-03-21T10:39:07.013 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.015+0000 7f0659000700 1 --2- 172.21.15.153:0/2673370692 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f063c06e460 0x7f063c070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:39:07.013 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.016+0000 7f0659000700 1 --2- 172.21.15.153:0/2673370692 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f063c06e460 0x7f063c070920 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f0640005fd0 tx=0x7f0640005ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:39:07.017 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.019+0000 7f0652000700 1 -- 172.21.15.153:0/2673370692 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f064805c5f0 con 0x7f0654068670 2024-03-21T10:39:07.232 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.234+0000 7f065c400700 1 -- 172.21.15.153:0/2673370692 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f06541993c0 con 0x7f063c06e460 2024-03-21T10:39:07.253 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.255+0000 7f0652000700 1 -- 172.21.15.153:0/2673370692 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f06541993c0 con 0x7f063c06e460 2024-03-21T10:39:07.254 INFO:teuthology.orchestra.run.smithi153.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-21T10:39:07.254 INFO:teuthology.orchestra.run.smithi153.stdout:alertmanager.smithi153 smithi153 *:9093,9094 running (27m) 3m ago 41m 15.6M - 0.25.0 c8568f914cd2 9ed3f2200b5f 2024-03-21T10:39:07.254 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi153 smithi153 running (42m) 3m ago 42m 10.4M - 18.2.1 5be31c24972a 5b61994396a6 2024-03-21T10:39:07.254 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi174 smithi174 running (40m) 3m ago 40m 10.1M - 18.2.1 5be31c24972a 4f7a5fc3ac28 2024-03-21T10:39:07.254 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi153 smithi153 running (24m) 3m ago 41m 6656k - 19.0.0-2231-g2edfee68 237e43c7a65f 32c21104d5b0 2024-03-21T10:39:07.254 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi174 smithi174 running (24m) 3m ago 40m 6660k - 19.0.0-2231-g2edfee68 237e43c7a65f a04e48967ab6 2024-03-21T10:39:07.254 INFO:teuthology.orchestra.run.smithi153.stdout:grafana.smithi153 smithi153 *:3000 running (27m) 3m ago 41m 75.2M - 9.4.12 2bacad6d85d8 ee24026ab786 2024-03-21T10:39:07.254 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.iuyluj smithi153 running (19m) 3m ago 34m 18.7M - 19.0.0-2231-g2edfee68 237e43c7a65f 87aff8a5de70 2024-03-21T10:39:07.254 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.zipckr smithi153 running (19m) 3m ago 34m 18.9M - 19.0.0-2231-g2edfee68 237e43c7a65f ccc262bd5dd3 2024-03-21T10:39:07.254 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.nhozag smithi174 running (3m) 3m ago 34m 10.9M - 19.0.0-2231-g2edfee68 237e43c7a65f 9e02aea72c96 2024-03-21T10:39:07.254 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.yyimst smithi174 running (19m) 3m ago 34m 18.5M - 19.0.0-2231-g2edfee68 237e43c7a65f fc1962f9351e 2024-03-21T10:39:07.254 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi153.tluxni smithi153 *:8443,9283,8765 running (30m) 3m ago 43m 512M - 19.0.0-2231-g2edfee68 237e43c7a65f 7ffee0e55382 2024-03-21T10:39:07.255 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi174.qpcwat smithi174 *:8443,9283,8765 running (29m) 3m ago 40m 459M - 19.0.0-2231-g2edfee68 237e43c7a65f 748968f216ab 2024-03-21T10:39:07.255 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi153 smithi153 running (25m) 3m ago 43m 70.8M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f dd8038303dc3 2024-03-21T10:39:07.255 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi174 smithi174 running (25m) 3m ago 40m 51.9M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f 29d78a788f2c 2024-03-21T10:39:07.255 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi153 smithi153 *:9100 running (29m) 3m ago 41m 8140k - 1.5.0 0da6a335fe13 bb835edc0e0b 2024-03-21T10:39:07.255 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi174 smithi174 *:9100 running (28m) 3m ago 40m 8610k - 1.5.0 0da6a335fe13 7c4ecad790fa 2024-03-21T10:39:07.255 INFO:teuthology.orchestra.run.smithi153.stdout:osd.0 smithi153 running (23m) 3m ago 39m 118M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 507900fd4791 2024-03-21T10:39:07.255 INFO:teuthology.orchestra.run.smithi153.stdout:osd.1 smithi153 running (22m) 3m ago 38m 93.0M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 0d646452e9c1 2024-03-21T10:39:07.255 INFO:teuthology.orchestra.run.smithi153.stdout:osd.2 smithi153 running (22m) 3m ago 37m 91.2M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 1326f2afdfcd 2024-03-21T10:39:07.255 INFO:teuthology.orchestra.run.smithi153.stdout:osd.3 smithi174 running (21m) 3m ago 37m 87.5M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 70aafac71ad5 2024-03-21T10:39:07.255 INFO:teuthology.orchestra.run.smithi153.stdout:osd.4 smithi174 running (21m) 3m ago 36m 95.8M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 0d746ce6e79d 2024-03-21T10:39:07.255 INFO:teuthology.orchestra.run.smithi153.stdout:osd.5 smithi174 running (20m) 3m ago 36m 90.3M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 53e5f35b8cac 2024-03-21T10:39:07.255 INFO:teuthology.orchestra.run.smithi153.stdout:prometheus.smithi153 smithi153 *:9095 running (28m) 3m ago 40m 88.1M - 2.43.0 a07b618ecd1d 7d95561e50c2 2024-03-21T10:39:07.256 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.258+0000 7f065c400700 1 -- 172.21.15.153:0/2673370692 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f063c06e460 msgr2=0x7f063c070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:39:07.257 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.258+0000 7f065c400700 1 --2- 172.21.15.153:0/2673370692 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f063c06e460 0x7f063c070920 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f0640005fd0 tx=0x7f0640005ee0 comp rx=0 tx=0).stop 2024-03-21T10:39:07.257 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.258+0000 7f065c400700 1 -- 172.21.15.153:0/2673370692 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0654068670 msgr2=0x7f065419e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:39:07.257 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.258+0000 7f065c400700 1 --2- 172.21.15.153:0/2673370692 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0654068670 0x7f065419e710 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f06480092e0 tx=0x7f064800be20 comp rx=0 tx=0).stop 2024-03-21T10:39:07.257 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.258+0000 7f065c400700 1 -- 172.21.15.153:0/2673370692 shutdown_connections 2024-03-21T10:39:07.257 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.259+0000 7f065c400700 1 --2- 172.21.15.153:0/2673370692 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f063c06e460 0x7f063c070920 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:39:07.257 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.259+0000 7f065c400700 1 --2- 172.21.15.153:0/2673370692 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0654068670 0x7f065419e710 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:39:07.257 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.259+0000 7f065c400700 1 --2- 172.21.15.153:0/2673370692 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0654069020 0x7f065419ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:39:07.257 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.259+0000 7f065c400700 1 -- 172.21.15.153:0/2673370692 >> 172.21.15.153:0/2673370692 conn(0x7f06540755a0 msgr2=0x7f0654102760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:39:07.257 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.259+0000 7f065c400700 1 -- 172.21.15.153:0/2673370692 shutdown_connections 2024-03-21T10:39:07.257 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.259+0000 7f065c400700 1 -- 172.21.15.153:0/2673370692 wait complete. 2024-03-21T10:39:07.400 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.402+0000 7f54c3800700 1 -- 172.21.15.153:0/2400068384 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f54bc068d30 msgr2=0x7f54bc10d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:39:07.400 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.402+0000 7f54c3800700 1 --2- 172.21.15.153:0/2400068384 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f54bc068d30 0x7f54bc10d240 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f54a8009910 tx=0x7f54a8009c20 comp rx=0 tx=0).stop 2024-03-21T10:39:07.400 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.402+0000 7f54c3800700 1 -- 172.21.15.153:0/2400068384 shutdown_connections 2024-03-21T10:39:07.401 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.402+0000 7f54c3800700 1 --2- 172.21.15.153:0/2400068384 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f54bc068d30 0x7f54bc10d240 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:39:07.401 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.402+0000 7f54c3800700 1 --2- 172.21.15.153:0/2400068384 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f54bc068410 0x7f54bc0687f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:39:07.401 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.402+0000 7f54c3800700 1 -- 172.21.15.153:0/2400068384 >> 172.21.15.153:0/2400068384 conn(0x7f54bc0755a0 msgr2=0x7f54bc0759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:39:07.401 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.402+0000 7f54c3800700 1 -- 172.21.15.153:0/2400068384 shutdown_connections 2024-03-21T10:39:07.401 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.402+0000 7f54c3800700 1 -- 172.21.15.153:0/2400068384 wait complete. 2024-03-21T10:39:07.402 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.403+0000 7f54c3800700 1 Processor -- start 2024-03-21T10:39:07.402 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.403+0000 7f54c3800700 1 -- start start 2024-03-21T10:39:07.402 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.404+0000 7f54c3800700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f54bc068410 0x7f54bc19e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:39:07.402 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.404+0000 7f54c3800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f54bc068d30 0x7f54bc19ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:39:07.402 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.404+0000 7f54c3800700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54bc19f2b0 con 0x7f54bc068d30 2024-03-21T10:39:07.402 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.404+0000 7f54c3800700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54bc198760 con 0x7f54bc068410 2024-03-21T10:39:07.403 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.404+0000 7f54bbe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f54bc068d30 0x7f54bc19ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:39:07.403 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.404+0000 7f54bbe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f54bc068d30 0x7f54bc19ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:37310/0 (socket says 172.21.15.153:37310) 2024-03-21T10:39:07.403 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.404+0000 7f54bbe00700 1 -- 172.21.15.153:0/661709872 learned_addr learned my addr 172.21.15.153:0/661709872 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:39:07.403 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.404+0000 7f54c0e00700 1 --2- 172.21.15.153:0/661709872 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f54bc068410 0x7f54bc19e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:39:07.404 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.404+0000 7f54bbe00700 1 -- 172.21.15.153:0/661709872 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f54bc068410 msgr2=0x7f54bc19e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:39:07.404 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.404+0000 7f54bbe00700 1 --2- 172.21.15.153:0/661709872 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f54bc068410 0x7f54bc19e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:39:07.404 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.405+0000 7f54bbe00700 1 -- 172.21.15.153:0/661709872 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54a80095f0 con 0x7f54bc068d30 2024-03-21T10:39:07.404 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.405+0000 7f54bbe00700 1 --2- 172.21.15.153:0/661709872 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f54bc068d30 0x7f54bc19ec20 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f54a800b5c0 tx=0x7f54a800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:39:07.404 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.405+0000 7f54b9600700 1 -- 172.21.15.153:0/661709872 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f54a801d070 con 0x7f54bc068d30 2024-03-21T10:39:07.404 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.405+0000 7f54b9600700 1 -- 172.21.15.153:0/661709872 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f54a8005e00 con 0x7f54bc068d30 2024-03-21T10:39:07.404 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.405+0000 7f54b9600700 1 -- 172.21.15.153:0/661709872 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f54a8017400 con 0x7f54bc068d30 2024-03-21T10:39:07.404 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.405+0000 7f54c3800700 1 -- 172.21.15.153:0/661709872 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54bc198a40 con 0x7f54bc068d30 2024-03-21T10:39:07.404 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.405+0000 7f54c3800700 1 -- 172.21.15.153:0/661709872 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54bc198f90 con 0x7f54bc068d30 2024-03-21T10:39:07.405 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.406+0000 7f54c3800700 1 -- 172.21.15.153:0/661709872 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54bc10a9b0 con 0x7f54bc068d30 2024-03-21T10:39:07.405 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.407+0000 7f54b9600700 1 -- 172.21.15.153:0/661709872 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f54a800f6f0 con 0x7f54bc068d30 2024-03-21T10:39:07.406 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.408+0000 7f54b9600700 1 --2- 172.21.15.153:0/661709872 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f54a406e460 0x7f54a4070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:39:07.406 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.408+0000 7f54b9600700 1 -- 172.21.15.153:0/661709872 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f54a808f8c0 con 0x7f54bc068d30 2024-03-21T10:39:07.406 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.409+0000 7f54c0e00700 1 --2- 172.21.15.153:0/661709872 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f54a406e460 0x7f54a4070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:39:07.407 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.409+0000 7f54c0e00700 1 --2- 172.21.15.153:0/661709872 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f54a406e460 0x7f54a4070920 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f54bc076b00 tx=0x7f54b000b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:39:07.411 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.413+0000 7f54b9600700 1 -- 172.21.15.153:0/661709872 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f54a805c490 con 0x7f54bc068d30 2024-03-21T10:39:07.704 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.706+0000 7f54c3800700 1 -- 172.21.15.153:0/661709872 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f54bc04ea90 con 0x7f54bc068d30 2024-03-21T10:39:07.705 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.707+0000 7f54b9600700 1 -- 172.21.15.153:0/661709872 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f54a805c020 con 0x7f54bc068d30 2024-03-21T10:39:07.705 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:39:07.706 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": { 2024-03-21T10:39:07.706 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:39:07.706 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:39:07.706 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": { 2024-03-21T10:39:07.706 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:39:07.706 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:39:07.706 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": { 2024-03-21T10:39:07.706 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 6 2024-03-21T10:39:07.706 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:39:07.706 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": { 2024-03-21T10:39:07.706 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 4 2024-03-21T10:39:07.706 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:39:07.706 INFO:teuthology.orchestra.run.smithi153.stdout: "overall": { 2024-03-21T10:39:07.707 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 14 2024-03-21T10:39:07.707 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-03-21T10:39:07.707 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:39:07.708 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.710+0000 7f54c3800700 1 -- 172.21.15.153:0/661709872 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f54a406e460 msgr2=0x7f54a4070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:39:07.708 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.710+0000 7f54c3800700 1 --2- 172.21.15.153:0/661709872 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f54a406e460 0x7f54a4070920 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f54bc076b00 tx=0x7f54b000b410 comp rx=0 tx=0).stop 2024-03-21T10:39:07.708 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.710+0000 7f54c3800700 1 -- 172.21.15.153:0/661709872 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f54bc068d30 msgr2=0x7f54bc19ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:39:07.708 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.710+0000 7f54c3800700 1 --2- 172.21.15.153:0/661709872 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f54bc068d30 0x7f54bc19ec20 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f54a800b5c0 tx=0x7f54a800bd70 comp rx=0 tx=0).stop 2024-03-21T10:39:07.708 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.710+0000 7f54c3800700 1 -- 172.21.15.153:0/661709872 shutdown_connections 2024-03-21T10:39:07.708 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.710+0000 7f54c3800700 1 --2- 172.21.15.153:0/661709872 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f54bc068410 0x7f54bc19e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:39:07.708 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.710+0000 7f54c3800700 1 --2- 172.21.15.153:0/661709872 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f54a406e460 0x7f54a4070920 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:39:07.708 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.710+0000 7f54c3800700 1 --2- 172.21.15.153:0/661709872 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f54bc068d30 0x7f54bc19ec20 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:39:07.709 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.710+0000 7f54c3800700 1 -- 172.21.15.153:0/661709872 >> 172.21.15.153:0/661709872 conn(0x7f54bc0755a0 msgr2=0x7f54bc0fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:39:07.709 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.710+0000 7f54c3800700 1 -- 172.21.15.153:0/661709872 shutdown_connections 2024-03-21T10:39:07.709 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.710+0000 7f54c3800700 1 -- 172.21.15.153:0/661709872 wait complete. 2024-03-21T10:39:07.803 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:39:07 smithi153 ceph-mon[113733]: pgmap v819: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:39:07.803 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:39:07 smithi153 ceph-mon[113733]: from='client.35462 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:39:07.863 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.864+0000 7f771f400700 1 -- 172.21.15.153:0/718670427 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f77181039b0 msgr2=0x7f7718107a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:39:07.863 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.864+0000 7f771f400700 1 --2- 172.21.15.153:0/718670427 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f77181039b0 0x7f7718107a00 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f7704009910 tx=0x7f7704009c20 comp rx=0 tx=0).stop 2024-03-21T10:39:07.863 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.864+0000 7f771f400700 1 -- 172.21.15.153:0/718670427 shutdown_connections 2024-03-21T10:39:07.863 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.864+0000 7f771f400700 1 --2- 172.21.15.153:0/718670427 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f77181039b0 0x7f7718107a00 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:39:07.863 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.865+0000 7f771f400700 1 --2- 172.21.15.153:0/718670427 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7718103000 0x7f77181033e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:39:07.863 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.865+0000 7f771f400700 1 -- 172.21.15.153:0/718670427 >> 172.21.15.153:0/718670427 conn(0x7f77180fe8d0 msgr2=0x7f7718100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:39:07.863 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.865+0000 7f771f400700 1 -- 172.21.15.153:0/718670427 shutdown_connections 2024-03-21T10:39:07.863 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.865+0000 7f771f400700 1 -- 172.21.15.153:0/718670427 wait complete. 2024-03-21T10:39:07.864 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.866+0000 7f771f400700 1 Processor -- start 2024-03-21T10:39:07.864 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.866+0000 7f771f400700 1 -- start start 2024-03-21T10:39:07.865 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.866+0000 7f771f400700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7718103000 0x7f771819e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:39:07.866 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.866+0000 7f771f400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f77181039b0 0x7f771819ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:39:07.866 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.866+0000 7f771f400700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f771819f490 con 0x7f77181039b0 2024-03-21T10:39:07.866 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.866+0000 7f771f400700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7718198940 con 0x7f7718103000 2024-03-21T10:39:07.866 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.867+0000 7f7717e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f77181039b0 0x7f771819ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:39:07.866 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.867+0000 7f7717e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f77181039b0 0x7f771819ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:37324/0 (socket says 172.21.15.153:37324) 2024-03-21T10:39:07.866 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.867+0000 7f7717e00700 1 -- 172.21.15.153:0/2076407516 learned_addr learned my addr 172.21.15.153:0/2076407516 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:39:07.866 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.867+0000 7f7717e00700 1 -- 172.21.15.153:0/2076407516 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7718103000 msgr2=0x7f771819e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:39:07.866 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.867+0000 7f771ca00700 1 --2- 172.21.15.153:0/2076407516 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7718103000 0x7f771819e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:39:07.866 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.867+0000 7f7717e00700 1 --2- 172.21.15.153:0/2076407516 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7718103000 0x7f771819e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:39:07.866 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.867+0000 7f7717e00700 1 -- 172.21.15.153:0/2076407516 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77040095f0 con 0x7f77181039b0 2024-03-21T10:39:07.866 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.867+0000 7f771ca00700 1 --2- 172.21.15.153:0/2076407516 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7718103000 0x7f771819e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:39:07.867 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.868+0000 7f7717e00700 1 --2- 172.21.15.153:0/2076407516 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f77181039b0 0x7f771819ee00 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f770400bda0 tx=0x7f770400be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:39:07.867 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.868+0000 7f7715600700 1 -- 172.21.15.153:0/2076407516 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f770401d070 con 0x7f77181039b0 2024-03-21T10:39:07.867 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.868+0000 7f7715600700 1 -- 172.21.15.153:0/2076407516 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7704004fb0 con 0x7f77181039b0 2024-03-21T10:39:07.867 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.868+0000 7f771f400700 1 -- 172.21.15.153:0/2076407516 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7718198bc0 con 0x7f77181039b0 2024-03-21T10:39:07.867 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.868+0000 7f7715600700 1 -- 172.21.15.153:0/2076407516 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f770400f480 con 0x7f77181039b0 2024-03-21T10:39:07.867 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.868+0000 7f771f400700 1 -- 172.21.15.153:0/2076407516 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77181990b0 con 0x7f77181039b0 2024-03-21T10:39:07.867 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.869+0000 7f771f400700 1 -- 172.21.15.153:0/2076407516 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f771810b350 con 0x7f77181039b0 2024-03-21T10:39:07.868 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.870+0000 7f7715600700 1 -- 172.21.15.153:0/2076407516 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f7704022870 con 0x7f77181039b0 2024-03-21T10:39:07.869 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.871+0000 7f7715600700 1 --2- 172.21.15.153:0/2076407516 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f770006e460 0x7f7700070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:39:07.869 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.871+0000 7f7715600700 1 -- 172.21.15.153:0/2076407516 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f770408fa10 con 0x7f77181039b0 2024-03-21T10:39:07.869 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.871+0000 7f771ca00700 1 --2- 172.21.15.153:0/2076407516 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f770006e460 0x7f7700070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:39:07.869 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.872+0000 7f771ca00700 1 --2- 172.21.15.153:0/2076407516 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f770006e460 0x7f7700070920 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f770c007900 tx=0x7f770c008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:39:07.873 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:07.876+0000 7f7715600700 1 -- 172.21.15.153:0/2076407516 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f770405c690 con 0x7f77181039b0 2024-03-21T10:39:07.948 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:39:07 smithi174 ceph-mon[88004]: pgmap v819: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:39:07.948 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:39:07 smithi174 ceph-mon[88004]: from='client.35462 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:39:08.135 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:08.137+0000 7f771f400700 1 -- 172.21.15.153:0/2076407516 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f771804ea90 con 0x7f77181039b0 2024-03-21T10:39:08.135 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:08.138+0000 7f7715600700 1 -- 172.21.15.153:0/2076407516 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1876 (secure 0 0 0) 0x7f7704027770 con 0x7f77181039b0 2024-03-21T10:39:08.136 INFO:teuthology.orchestra.run.smithi153.stdout:e32 2024-03-21T10:39:08.136 INFO:teuthology.orchestra.run.smithi153.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-21T10:39:08.136 INFO:teuthology.orchestra.run.smithi153.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:39:08.136 INFO:teuthology.orchestra.run.smithi153.stdout:legacy client fscid: 1 2024-03-21T10:39:08.136 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:39:08.136 INFO:teuthology.orchestra.run.smithi153.stdout:Filesystem 'cephfs' (1) 2024-03-21T10:39:08.137 INFO:teuthology.orchestra.run.smithi153.stdout:fs_name cephfs 2024-03-21T10:39:08.137 INFO:teuthology.orchestra.run.smithi153.stdout:epoch 23 2024-03-21T10:39:08.137 INFO:teuthology.orchestra.run.smithi153.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-21T10:39:08.137 INFO:teuthology.orchestra.run.smithi153.stdout:created 2024-03-21T10:04:27.086576+0000 2024-03-21T10:39:08.137 INFO:teuthology.orchestra.run.smithi153.stdout:modified 2024-03-21T10:19:05.022092+0000 2024-03-21T10:39:08.137 INFO:teuthology.orchestra.run.smithi153.stdout:tableserver 0 2024-03-21T10:39:08.137 INFO:teuthology.orchestra.run.smithi153.stdout:root 0 2024-03-21T10:39:08.137 INFO:teuthology.orchestra.run.smithi153.stdout:session_timeout 60 2024-03-21T10:39:08.137 INFO:teuthology.orchestra.run.smithi153.stdout:session_autoclose 300 2024-03-21T10:39:08.137 INFO:teuthology.orchestra.run.smithi153.stdout:max_file_size 1099511627776 2024-03-21T10:39:08.137 INFO:teuthology.orchestra.run.smithi153.stdout:max_xattr_size 65536 2024-03-21T10:39:08.137 INFO:teuthology.orchestra.run.smithi153.stdout:required_client_features {} 2024-03-21T10:39:08.137 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure 0 2024-03-21T10:39:08.137 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure_osd_epoch 94 2024-03-21T10:39:08.137 INFO:teuthology.orchestra.run.smithi153.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:39:08.137 INFO:teuthology.orchestra.run.smithi153.stdout:max_mds 2 2024-03-21T10:39:08.138 INFO:teuthology.orchestra.run.smithi153.stdout:in 0,1 2024-03-21T10:39:08.138 INFO:teuthology.orchestra.run.smithi153.stdout:up {} 2024-03-21T10:39:08.138 INFO:teuthology.orchestra.run.smithi153.stdout:failed 0,1 2024-03-21T10:39:08.138 INFO:teuthology.orchestra.run.smithi153.stdout:damaged 2024-03-21T10:39:08.138 INFO:teuthology.orchestra.run.smithi153.stdout:stopped 2024-03-21T10:39:08.138 INFO:teuthology.orchestra.run.smithi153.stdout:data_pools [3] 2024-03-21T10:39:08.138 INFO:teuthology.orchestra.run.smithi153.stdout:metadata_pool 2 2024-03-21T10:39:08.138 INFO:teuthology.orchestra.run.smithi153.stdout:inline_data enabled 2024-03-21T10:39:08.138 INFO:teuthology.orchestra.run.smithi153.stdout:balancer 2024-03-21T10:39:08.138 INFO:teuthology.orchestra.run.smithi153.stdout:bal_rank_mask -1 2024-03-21T10:39:08.138 INFO:teuthology.orchestra.run.smithi153.stdout:standby_count_wanted 1 2024-03-21T10:39:08.138 INFO:teuthology.orchestra.run.smithi153.stdout:qdb_cluster leader: 0 members: 2024-03-21T10:39:08.138 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:39:08.139 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:39:08.139 INFO:teuthology.orchestra.run.smithi153.stdout:Standby daemons: 2024-03-21T10:39:08.139 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:39:08.139 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.zipckr{-1:34442} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6826/1103846295,v1:172.21.15.153:6827/1103846295] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:39:08.139 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.iuyluj{-1:34474} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6828/1738630832,v1:172.21.15.153:6829/1738630832] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:39:08.139 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.yyimst{-1:44313} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.174:6824/4127786797,v1:172.21.15.174:6825/4127786797] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:39:08.139 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.nhozag{-1:44687} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.174:6826/2767517845,v1:172.21.15.174:6827/2767517845] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:39:08.140 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:08.142+0000 7f771f400700 1 -- 172.21.15.153:0/2076407516 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f770006e460 msgr2=0x7f7700070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:39:08.140 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:08.142+0000 7f771f400700 1 --2- 172.21.15.153:0/2076407516 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f770006e460 0x7f7700070920 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f770c007900 tx=0x7f770c008040 comp rx=0 tx=0).stop 2024-03-21T10:39:08.140 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:08.142+0000 7f771f400700 1 -- 172.21.15.153:0/2076407516 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f77181039b0 msgr2=0x7f771819ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:39:08.140 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:08.142+0000 7f771f400700 1 --2- 172.21.15.153:0/2076407516 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f77181039b0 0x7f771819ee00 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f770400bda0 tx=0x7f770400be80 comp rx=0 tx=0).stop 2024-03-21T10:39:08.140 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:08.142+0000 7f771f400700 1 -- 172.21.15.153:0/2076407516 shutdown_connections 2024-03-21T10:39:08.140 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:08.142+0000 7f771f400700 1 --2- 172.21.15.153:0/2076407516 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7718103000 0x7f771819e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:39:08.140 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:08.142+0000 7f771f400700 1 --2- 172.21.15.153:0/2076407516 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f770006e460 0x7f7700070920 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:39:08.140 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:08.142+0000 7f771f400700 1 --2- 172.21.15.153:0/2076407516 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f77181039b0 0x7f771819ee00 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:39:08.140 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:08.142+0000 7f771f400700 1 -- 172.21.15.153:0/2076407516 >> 172.21.15.153:0/2076407516 conn(0x7f77180fe8d0 msgr2=0x7f77180fffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:39:08.140 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:08.142+0000 7f771f400700 1 -- 172.21.15.153:0/2076407516 shutdown_connections 2024-03-21T10:39:08.140 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:08.142+0000 7f771f400700 1 -- 172.21.15.153:0/2076407516 wait complete. 2024-03-21T10:39:08.141 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 32 2024-03-21T10:39:08.295 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:08.297+0000 7f5a9ae00700 1 -- 172.21.15.153:0/2239134269 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5a940730b0 msgr2=0x7f5a94073490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:39:08.295 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:08.297+0000 7f5a9ae00700 1 --2- 172.21.15.153:0/2239134269 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5a940730b0 0x7f5a94073490 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f5a84009910 tx=0x7f5a84009c20 comp rx=0 tx=0).stop 2024-03-21T10:39:08.296 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:08.297+0000 7f5a9ae00700 1 -- 172.21.15.153:0/2239134269 shutdown_connections 2024-03-21T10:39:08.296 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:08.297+0000 7f5a9ae00700 1 --2- 172.21.15.153:0/2239134269 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5a940739d0 0x7f5a9410cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:39:08.296 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:08.297+0000 7f5a9ae00700 1 --2- 172.21.15.153:0/2239134269 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5a940730b0 0x7f5a94073490 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:39:08.296 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:08.297+0000 7f5a9ae00700 1 -- 172.21.15.153:0/2239134269 >> 172.21.15.153:0/2239134269 conn(0x7f5a940fc630 msgr2=0x7f5a940fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:39:08.296 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:08.297+0000 7f5a9ae00700 1 -- 172.21.15.153:0/2239134269 shutdown_connections 2024-03-21T10:39:08.296 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:08.297+0000 7f5a9ae00700 1 -- 172.21.15.153:0/2239134269 wait complete. 2024-03-21T10:39:08.297 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:08.298+0000 7f5a9ae00700 1 Processor -- start 2024-03-21T10:39:08.297 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:08.298+0000 7f5a9ae00700 1 -- start start 2024-03-21T10:39:08.297 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:08.299+0000 7f5a9ae00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5a940730b0 0x7f5a9419e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:39:08.297 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:08.299+0000 7f5a9ae00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5a940739d0 0x7f5a9419ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:39:08.297 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:08.299+0000 7f5a9ae00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a9419f210 con 0x7f5a940730b0 2024-03-21T10:39:08.297 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:08.299+0000 7f5a9ae00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a941987a0 con 0x7f5a940739d0 2024-03-21T10:39:08.298 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:08.299+0000 7f5a93e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5a940730b0 0x7f5a9419e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:39:08.298 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:08.299+0000 7f5a93e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5a940730b0 0x7f5a9419e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:37340/0 (socket says 172.21.15.153:37340) 2024-03-21T10:39:08.298 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:08.299+0000 7f5a93e00700 1 -- 172.21.15.153:0/3218797176 learned_addr learned my addr 172.21.15.153:0/3218797176 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:39:08.298 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:08.299+0000 7f5a93400700 1 --2- 172.21.15.153:0/3218797176 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5a940739d0 0x7f5a9419ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:39:08.298 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:08.299+0000 7f5a93e00700 1 -- 172.21.15.153:0/3218797176 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5a940739d0 msgr2=0x7f5a9419ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:39:08.298 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:08.299+0000 7f5a93e00700 1 --2- 172.21.15.153:0/3218797176 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5a940739d0 0x7f5a9419ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:39:08.298 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:08.299+0000 7f5a93e00700 1 -- 172.21.15.153:0/3218797176 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a840095f0 con 0x7f5a940730b0 2024-03-21T10:39:08.298 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:08.299+0000 7f5a93400700 1 --2- 172.21.15.153:0/3218797176 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5a940739d0 0x7f5a9419ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:39:08.299 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:08.300+0000 7f5a93e00700 1 --2- 172.21.15.153:0/3218797176 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5a940730b0 0x7f5a9419e6d0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f5a8400b5c0 tx=0x7f5a8400be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:39:08.299 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:08.300+0000 7f5a90c00700 1 -- 172.21.15.153:0/3218797176 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5a8401d070 con 0x7f5a940730b0 2024-03-21T10:39:08.299 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:08.300+0000 7f5a90c00700 1 -- 172.21.15.153:0/3218797176 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5a8400f460 con 0x7f5a940730b0 2024-03-21T10:39:08.299 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:08.300+0000 7f5a9ae00700 1 -- 172.21.15.153:0/3218797176 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a94198a20 con 0x7f5a940730b0 2024-03-21T10:39:08.299 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:08.300+0000 7f5a90c00700 1 -- 172.21.15.153:0/3218797176 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5a84017610 con 0x7f5a940730b0 2024-03-21T10:39:08.299 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:08.300+0000 7f5a9ae00700 1 -- 172.21.15.153:0/3218797176 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a94198f10 con 0x7f5a940730b0 2024-03-21T10:39:08.300 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:08.301+0000 7f5a9ae00700 1 -- 172.21.15.153:0/3218797176 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a9410a5c0 con 0x7f5a940730b0 2024-03-21T10:39:08.300 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:08.303+0000 7f5a90c00700 1 -- 172.21.15.153:0/3218797176 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f5a8400f5d0 con 0x7f5a940730b0 2024-03-21T10:39:08.301 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:08.304+0000 7f5a90c00700 1 --2- 172.21.15.153:0/3218797176 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f5a7806e460 0x7f5a78070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:39:08.302 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:08.304+0000 7f5a90c00700 1 -- 172.21.15.153:0/3218797176 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f5a8408faf0 con 0x7f5a940730b0 2024-03-21T10:39:08.302 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:08.304+0000 7f5a93400700 1 --2- 172.21.15.153:0/3218797176 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f5a7806e460 0x7f5a78070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:39:08.302 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:08.305+0000 7f5a93400700 1 --2- 172.21.15.153:0/3218797176 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f5a7806e460 0x7f5a78070920 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f5a80005fd0 tx=0x7f5a80005e20 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:39:08.306 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:08.308+0000 7f5a90c00700 1 -- 172.21.15.153:0/3218797176 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f5a8405c6e0 con 0x7f5a940730b0 2024-03-21T10:39:08.547 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:08.549+0000 7f5a9ae00700 1 -- 172.21.15.153:0/3218797176 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5a941991f0 con 0x7f5a7806e460 2024-03-21T10:39:08.549 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:08.551+0000 7f5a90c00700 1 -- 172.21.15.153:0/3218797176 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f5a941991f0 con 0x7f5a7806e460 2024-03-21T10:39:08.549 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:39:08.549 INFO:teuthology.orchestra.run.smithi153.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232", 2024-03-21T10:39:08.549 INFO:teuthology.orchestra.run.smithi153.stdout: "in_progress": true, 2024-03-21T10:39:08.549 INFO:teuthology.orchestra.run.smithi153.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-21T10:39:08.550 INFO:teuthology.orchestra.run.smithi153.stdout: "services_complete": [ 2024-03-21T10:39:08.550 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr", 2024-03-21T10:39:08.550 INFO:teuthology.orchestra.run.smithi153.stdout: "mon", 2024-03-21T10:39:08.550 INFO:teuthology.orchestra.run.smithi153.stdout: "mds", 2024-03-21T10:39:08.550 INFO:teuthology.orchestra.run.smithi153.stdout: "crash", 2024-03-21T10:39:08.550 INFO:teuthology.orchestra.run.smithi153.stdout: "osd" 2024-03-21T10:39:08.550 INFO:teuthology.orchestra.run.smithi153.stdout: ], 2024-03-21T10:39:08.550 INFO:teuthology.orchestra.run.smithi153.stdout: "progress": "16/23 daemons upgraded", 2024-03-21T10:39:08.550 INFO:teuthology.orchestra.run.smithi153.stdout: "message": "Currently upgrading mds daemons", 2024-03-21T10:39:08.550 INFO:teuthology.orchestra.run.smithi153.stdout: "is_paused": false 2024-03-21T10:39:08.550 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:39:08.552 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:08.554+0000 7f5a9ae00700 1 -- 172.21.15.153:0/3218797176 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f5a7806e460 msgr2=0x7f5a78070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:39:08.552 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:08.554+0000 7f5a9ae00700 1 --2- 172.21.15.153:0/3218797176 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f5a7806e460 0x7f5a78070920 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f5a80005fd0 tx=0x7f5a80005e20 comp rx=0 tx=0).stop 2024-03-21T10:39:08.552 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:08.554+0000 7f5a9ae00700 1 -- 172.21.15.153:0/3218797176 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5a940730b0 msgr2=0x7f5a9419e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:39:08.552 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:08.554+0000 7f5a9ae00700 1 --2- 172.21.15.153:0/3218797176 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5a940730b0 0x7f5a9419e6d0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f5a8400b5c0 tx=0x7f5a8400be00 comp rx=0 tx=0).stop 2024-03-21T10:39:08.552 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:08.554+0000 7f5a9ae00700 1 -- 172.21.15.153:0/3218797176 shutdown_connections 2024-03-21T10:39:08.552 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:08.554+0000 7f5a9ae00700 1 --2- 172.21.15.153:0/3218797176 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f5a7806e460 0x7f5a78070920 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:39:08.552 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:08.554+0000 7f5a9ae00700 1 --2- 172.21.15.153:0/3218797176 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5a940730b0 0x7f5a9419e6d0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:39:08.552 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:08.554+0000 7f5a9ae00700 1 --2- 172.21.15.153:0/3218797176 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5a940739d0 0x7f5a9419ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:39:08.553 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:08.554+0000 7f5a9ae00700 1 -- 172.21.15.153:0/3218797176 >> 172.21.15.153:0/3218797176 conn(0x7f5a940fc630 msgr2=0x7f5a94107700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:39:08.553 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:08.554+0000 7f5a9ae00700 1 -- 172.21.15.153:0/3218797176 shutdown_connections 2024-03-21T10:39:08.553 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:08.554+0000 7f5a9ae00700 1 -- 172.21.15.153:0/3218797176 wait complete. 2024-03-21T10:39:08.695 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:08.697+0000 7fb6b2200700 1 -- 172.21.15.153:0/197696438 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb6ac106970 msgr2=0x7fb6ac106d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:39:08.695 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:08.697+0000 7fb6b2200700 1 --2- 172.21.15.153:0/197696438 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb6ac106970 0x7fb6ac106d50 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fb6a000b0a0 tx=0x7fb6a000b3b0 comp rx=0 tx=0).stop 2024-03-21T10:39:08.695 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:08.697+0000 7fb6b2200700 1 -- 172.21.15.153:0/197696438 shutdown_connections 2024-03-21T10:39:08.696 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:08.697+0000 7fb6b2200700 1 --2- 172.21.15.153:0/197696438 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb6ac100970 0x7fb6ac100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:39:08.696 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:08.697+0000 7fb6b2200700 1 --2- 172.21.15.153:0/197696438 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb6ac106970 0x7fb6ac106d50 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:39:08.696 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:08.697+0000 7fb6b2200700 1 -- 172.21.15.153:0/197696438 >> 172.21.15.153:0/197696438 conn(0x7fb6ac0fc440 msgr2=0x7fb6ac0fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:39:08.696 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:08.697+0000 7fb6b2200700 1 -- 172.21.15.153:0/197696438 shutdown_connections 2024-03-21T10:39:08.696 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:08.697+0000 7fb6b2200700 1 -- 172.21.15.153:0/197696438 wait complete. 2024-03-21T10:39:08.697 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:08.698+0000 7fb6b2200700 1 Processor -- start 2024-03-21T10:39:08.697 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:08.698+0000 7fb6b2200700 1 -- start start 2024-03-21T10:39:08.697 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:08.699+0000 7fb6b2200700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb6ac100970 0x7fb6ac19e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:39:08.697 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:08.699+0000 7fb6b2200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb6ac106970 0x7fb6ac19eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:39:08.697 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:08.699+0000 7fb6b2200700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6ac19f150 con 0x7fb6ac106970 2024-03-21T10:39:08.697 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:08.699+0000 7fb6b2200700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6ac198600 con 0x7fb6ac100970 2024-03-21T10:39:08.697 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:08.699+0000 7fb6abe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb6ac106970 0x7fb6ac19eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:39:08.697 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:08.699+0000 7fb6abe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb6ac106970 0x7fb6ac19eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:37366/0 (socket says 172.21.15.153:37366) 2024-03-21T10:39:08.697 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:08.699+0000 7fb6abe00700 1 -- 172.21.15.153:0/3001582093 learned_addr learned my addr 172.21.15.153:0/3001582093 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:39:08.697 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:08.699+0000 7fb6b0e00700 1 --2- 172.21.15.153:0/3001582093 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb6ac100970 0x7fb6ac19e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:39:08.698 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:08.699+0000 7fb6abe00700 1 -- 172.21.15.153:0/3001582093 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb6ac100970 msgr2=0x7fb6ac19e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:39:08.698 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:08.699+0000 7fb6abe00700 1 --2- 172.21.15.153:0/3001582093 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb6ac100970 0x7fb6ac19e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:39:08.698 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:08.699+0000 7fb6abe00700 1 -- 172.21.15.153:0/3001582093 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6a0009d00 con 0x7fb6ac106970 2024-03-21T10:39:08.698 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:08.700+0000 7fb6abe00700 1 --2- 172.21.15.153:0/3001582093 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb6ac106970 0x7fb6ac19eac0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fb69800d8d0 tx=0x7fb69800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:39:08.699 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:08.700+0000 7fb6a9600700 1 -- 172.21.15.153:0/3001582093 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb698009940 con 0x7fb6ac106970 2024-03-21T10:39:08.699 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:08.700+0000 7fb6a9600700 1 -- 172.21.15.153:0/3001582093 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb698010460 con 0x7fb6ac106970 2024-03-21T10:39:08.699 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:08.700+0000 7fb6b2200700 1 -- 172.21.15.153:0/3001582093 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6ac1988e0 con 0x7fb6ac106970 2024-03-21T10:39:08.699 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:08.700+0000 7fb6a9600700 1 -- 172.21.15.153:0/3001582093 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb69800f5f0 con 0x7fb6ac106970 2024-03-21T10:39:08.699 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:08.700+0000 7fb6b2200700 1 -- 172.21.15.153:0/3001582093 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6ac198e30 con 0x7fb6ac106970 2024-03-21T10:39:08.699 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:08.701+0000 7fb6b2200700 1 -- 172.21.15.153:0/3001582093 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6ac04f2e0 con 0x7fb6ac106970 2024-03-21T10:39:08.700 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:08.702+0000 7fb6a9600700 1 -- 172.21.15.153:0/3001582093 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fb6980105d0 con 0x7fb6ac106970 2024-03-21T10:39:08.701 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:08.703+0000 7fb6a9600700 1 --2- 172.21.15.153:0/3001582093 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb69406e460 0x7fb694070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:39:08.701 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:08.703+0000 7fb6a9600700 1 -- 172.21.15.153:0/3001582093 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fb69808e480 con 0x7fb6ac106970 2024-03-21T10:39:08.701 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:08.703+0000 7fb6b0e00700 1 --2- 172.21.15.153:0/3001582093 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb69406e460 0x7fb694070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:39:08.702 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:08.704+0000 7fb6b0e00700 1 --2- 172.21.15.153:0/3001582093 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb69406e460 0x7fb694070920 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fb6a0009480 tx=0x7fb6a000b7f0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:39:08.705 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:08.708+0000 7fb6a9600700 1 -- 172.21.15.153:0/3001582093 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb69805b050 con 0x7fb6ac106970 2024-03-21T10:39:08.803 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:39:08 smithi153 ceph-mon[113733]: from='client.35466 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:39:08.803 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:39:08 smithi153 ceph-mon[113733]: from='client.35470 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:39:08.803 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:39:08 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/661709872' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:39:08.803 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:39:08 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/2076407516' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:39:08.948 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:39:08 smithi174 ceph-mon[88004]: from='client.35466 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:39:08.948 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:39:08 smithi174 ceph-mon[88004]: from='client.35470 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:39:08.948 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:39:08 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/661709872' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:39:08.948 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:39:08 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/2076407516' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:39:09.005 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:09.007+0000 7fb6b2200700 1 -- 172.21.15.153:0/3001582093 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fb6ac199b10 con 0x7fb6ac106970 2024-03-21T10:39:09.005 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:09.008+0000 7fb6a9600700 1 -- 172.21.15.153:0/3001582093 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4493 (secure 0 0 0) 0x7fb69805abe0 con 0x7fb6ac106970 2024-03-21T10:39:09.006 INFO:teuthology.orchestra.run.smithi153.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-03-21T10:39:09.006 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-03-21T10:39:09.006 INFO:teuthology.orchestra.run.smithi153.stdout: host smithi174 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi174/config 2024-03-21T10:39:09.006 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/podman: timeout after 895 seconds 2024-03-21T10:39:09.006 INFO:teuthology.orchestra.run.smithi153.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 -e NODE_NAME=smithi174 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/run/ceph:z -v /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpd910r3_i:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 inventory --format=json-pretty --filter-for-batch 2024-03-21T10:39:09.007 INFO:teuthology.orchestra.run.smithi153.stdout:Traceback (most recent call last): 2024-03-21T10:39:09.007 INFO:teuthology.orchestra.run.smithi153.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-03-21T10:39:09.007 INFO:teuthology.orchestra.run.smithi153.stdout: return _run_code(code, main_globals, None, 2024-03-21T10:39:09.007 INFO:teuthology.orchestra.run.smithi153.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-03-21T10:39:09.007 INFO:teuthology.orchestra.run.smithi153.stdout: exec(code, run_globals) 2024-03-21T10:39:09.007 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 5606, in 2024-03-21T10:39:09.007 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 5594, in main 2024-03-21T10:39:09.007 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 406, in _infer_config 2024-03-21T10:39:09.007 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 321, in _infer_fsid 2024-03-21T10:39:09.007 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 434, in _infer_image 2024-03-21T10:39:09.007 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 308, in _validate_fsid 2024-03-21T10:39:09.007 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 3357, in command_ceph_volume 2024-03-21T10:39:09.007 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-03-21T10:39:09.007 INFO:teuthology.orchestra.run.smithi153.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 -e NODE_NAME=smithi174 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/run/ceph:z -v /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpd910r3_i:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 inventory --format=json-pretty --filter-for-batch 2024-03-21T10:39:09.007 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-21T10:39:09.007 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is degraded 2024-03-21T10:39:09.008 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-21T10:39:09.008 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-21T10:39:09.008 INFO:teuthology.orchestra.run.smithi153.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-21T10:39:09.008 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-21T10:39:09.009 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:09.010+0000 7fb6b2200700 1 -- 172.21.15.153:0/3001582093 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb69406e460 msgr2=0x7fb694070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:39:09.009 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:09.010+0000 7fb6b2200700 1 --2- 172.21.15.153:0/3001582093 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb69406e460 0x7fb694070920 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fb6a0009480 tx=0x7fb6a000b7f0 comp rx=0 tx=0).stop 2024-03-21T10:39:09.009 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:09.010+0000 7fb6b2200700 1 -- 172.21.15.153:0/3001582093 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb6ac106970 msgr2=0x7fb6ac19eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:39:09.009 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:09.010+0000 7fb6b2200700 1 --2- 172.21.15.153:0/3001582093 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb6ac106970 0x7fb6ac19eac0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fb69800d8d0 tx=0x7fb69800dc90 comp rx=0 tx=0).stop 2024-03-21T10:39:09.009 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:09.011+0000 7fb6b2200700 1 -- 172.21.15.153:0/3001582093 shutdown_connections 2024-03-21T10:39:09.009 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:09.011+0000 7fb6b2200700 1 --2- 172.21.15.153:0/3001582093 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb6ac100970 0x7fb6ac19e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:39:09.009 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:09.011+0000 7fb6b2200700 1 --2- 172.21.15.153:0/3001582093 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb69406e460 0x7fb694070920 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:39:09.009 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:09.011+0000 7fb6b2200700 1 --2- 172.21.15.153:0/3001582093 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb6ac106970 0x7fb6ac19eac0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:39:09.009 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:09.011+0000 7fb6b2200700 1 -- 172.21.15.153:0/3001582093 >> 172.21.15.153:0/3001582093 conn(0x7fb6ac0fc440 msgr2=0x7fb6ac072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:39:09.009 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:09.011+0000 7fb6b2200700 1 -- 172.21.15.153:0/3001582093 shutdown_connections 2024-03-21T10:39:09.009 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:09.011+0000 7fb6b2200700 1 -- 172.21.15.153:0/3001582093 wait complete. 2024-03-21T10:39:09.698 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:39:09 smithi174 ceph-mon[88004]: pgmap v820: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:39:09.698 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:39:09 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/3001582093' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:39:09.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:39:09 smithi153 ceph-mon[113733]: pgmap v820: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:39:09.803 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:39:09 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/3001582093' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:39:10.803 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:39:10 smithi153 ceph-mon[113733]: from='client.35482 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:39:10.948 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:39:10 smithi174 ceph-mon[88004]: from='client.35482 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:39:11.803 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:39:11 smithi153 ceph-mon[113733]: pgmap v821: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:39:11.948 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:39:11 smithi174 ceph-mon[88004]: pgmap v821: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:39:12.645 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:39:12 smithi153 ceph-mon[113733]: pgmap v822: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:39:12.948 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:39:12 smithi174 ceph-mon[88004]: pgmap v822: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:39:15.198 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:39:14 smithi174 ceph-mon[88004]: pgmap v823: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:39:15.302 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:39:14 smithi153 ceph-mon[113733]: pgmap v823: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:39:17.053 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:39:16 smithi153 ceph-mon[113733]: pgmap v824: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:39:17.053 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:39:16 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:39:17.198 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:39:16 smithi174 ceph-mon[88004]: pgmap v824: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:39:17.198 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:39:16 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:39:19.197 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:39:18 smithi174 ceph-mon[88004]: pgmap v825: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:39:19.302 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:39:18 smithi153 ceph-mon[113733]: pgmap v825: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:39:21.198 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:39:20 smithi174 ceph-mon[88004]: pgmap v826: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:39:21.303 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:39:20 smithi153 ceph-mon[113733]: pgmap v826: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:39:23.448 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:39:23 smithi174 ceph-mon[88004]: pgmap v827: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:39:23.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:39:23 smithi153 ceph-mon[113733]: pgmap v827: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:39:25.448 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:39:25 smithi174 ceph-mon[88004]: pgmap v828: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:39:25.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:39:25 smithi153 ceph-mon[113733]: pgmap v828: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:39:27.302 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:39:27 smithi153 ceph-mon[113733]: pgmap v829: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:39:27.448 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:39:27 smithi174 ceph-mon[88004]: pgmap v829: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:39:29.447 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:39:29 smithi174 ceph-mon[88004]: pgmap v830: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:39:29.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:39:29 smithi153 ceph-mon[113733]: pgmap v830: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:39:31.448 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:39:31 smithi174 ceph-mon[88004]: pgmap v831: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:39:31.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:39:31 smithi153 ceph-mon[113733]: pgmap v831: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:39:32.448 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:39:32 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:39:32.553 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:39:32 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:39:33.448 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:39:33 smithi174 ceph-mon[88004]: pgmap v832: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:39:33.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:39:33 smithi153 ceph-mon[113733]: pgmap v832: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:39:35.448 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:39:35 smithi174 ceph-mon[88004]: pgmap v833: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:39:35.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:39:35 smithi153 ceph-mon[113733]: pgmap v833: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:39:37.447 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:39:37 smithi174 ceph-mon[88004]: pgmap v834: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:39:37.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:39:37 smithi153 ceph-mon[113733]: pgmap v834: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:39:39.178 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.180+0000 7fcb41200700 1 -- 172.21.15.153:0/3476627455 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb3c068d30 msgr2=0x7fcb3c10d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:39:39.178 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.180+0000 7fcb41200700 1 --2- 172.21.15.153:0/3476627455 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb3c068d30 0x7fcb3c10d240 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fcb24009910 tx=0x7fcb24009c20 comp rx=0 tx=0).stop 2024-03-21T10:39:39.178 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.180+0000 7fcb41200700 1 -- 172.21.15.153:0/3476627455 shutdown_connections 2024-03-21T10:39:39.178 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.180+0000 7fcb41200700 1 --2- 172.21.15.153:0/3476627455 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb3c068d30 0x7fcb3c10d240 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:39:39.178 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.180+0000 7fcb41200700 1 --2- 172.21.15.153:0/3476627455 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcb3c068410 0x7fcb3c0687f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:39:39.179 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.180+0000 7fcb41200700 1 -- 172.21.15.153:0/3476627455 >> 172.21.15.153:0/3476627455 conn(0x7fcb3c0755a0 msgr2=0x7fcb3c0759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:39:39.179 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.180+0000 7fcb41200700 1 -- 172.21.15.153:0/3476627455 shutdown_connections 2024-03-21T10:39:39.179 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.180+0000 7fcb41200700 1 -- 172.21.15.153:0/3476627455 wait complete. 2024-03-21T10:39:39.179 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.181+0000 7fcb41200700 1 Processor -- start 2024-03-21T10:39:39.179 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.181+0000 7fcb41200700 1 -- start start 2024-03-21T10:39:39.180 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.182+0000 7fcb41200700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcb3c068410 0x7fcb3c19e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:39:39.180 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.182+0000 7fcb41200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb3c068d30 0x7fcb3c19ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:39:39.180 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.182+0000 7fcb41200700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb3c19f2b0 con 0x7fcb3c068d30 2024-03-21T10:39:39.180 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.182+0000 7fcb41200700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb3c198760 con 0x7fcb3c068410 2024-03-21T10:39:39.180 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.182+0000 7fcb31400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb3c068d30 0x7fcb3c19ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:39:39.180 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.182+0000 7fcb31400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb3c068d30 0x7fcb3c19ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:48094/0 (socket says 172.21.15.153:48094) 2024-03-21T10:39:39.180 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.182+0000 7fcb31400700 1 -- 172.21.15.153:0/1618647442 learned_addr learned my addr 172.21.15.153:0/1618647442 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:39:39.180 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.182+0000 7fcb39e00700 1 --2- 172.21.15.153:0/1618647442 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcb3c068410 0x7fcb3c19e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:39:39.181 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.182+0000 7fcb31400700 1 -- 172.21.15.153:0/1618647442 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcb3c068410 msgr2=0x7fcb3c19e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:39:39.182 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.182+0000 7fcb31400700 1 --2- 172.21.15.153:0/1618647442 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcb3c068410 0x7fcb3c19e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:39:39.182 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.183+0000 7fcb31400700 1 -- 172.21.15.153:0/1618647442 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb240095f0 con 0x7fcb3c068d30 2024-03-21T10:39:39.182 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.183+0000 7fcb31400700 1 --2- 172.21.15.153:0/1618647442 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb3c068d30 0x7fcb3c19ec20 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fcb240092e0 tx=0x7fcb2400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:39:39.182 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.183+0000 7fcb33400700 1 -- 172.21.15.153:0/1618647442 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcb2401d070 con 0x7fcb3c068d30 2024-03-21T10:39:39.182 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.183+0000 7fcb33400700 1 -- 172.21.15.153:0/1618647442 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcb24005e00 con 0x7fcb3c068d30 2024-03-21T10:39:39.182 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.183+0000 7fcb33400700 1 -- 172.21.15.153:0/1618647442 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcb24017400 con 0x7fcb3c068d30 2024-03-21T10:39:39.182 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.183+0000 7fcb41200700 1 -- 172.21.15.153:0/1618647442 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb3c198a40 con 0x7fcb3c068d30 2024-03-21T10:39:39.182 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.183+0000 7fcb41200700 1 -- 172.21.15.153:0/1618647442 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb3c198f40 con 0x7fcb3c068d30 2024-03-21T10:39:39.183 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.184+0000 7fcb41200700 1 -- 172.21.15.153:0/1618647442 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb3c10a9b0 con 0x7fcb3c068d30 2024-03-21T10:39:39.184 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.186+0000 7fcb33400700 1 -- 172.21.15.153:0/1618647442 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fcb2400f460 con 0x7fcb3c068d30 2024-03-21T10:39:39.185 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.186+0000 7fcb33400700 1 --2- 172.21.15.153:0/1618647442 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fcb2006e460 0x7fcb20070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:39:39.185 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.187+0000 7fcb33400700 1 -- 172.21.15.153:0/1618647442 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fcb2408fa60 con 0x7fcb3c068d30 2024-03-21T10:39:39.185 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.187+0000 7fcb39e00700 1 --2- 172.21.15.153:0/1618647442 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fcb2006e460 0x7fcb20070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:39:39.186 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.187+0000 7fcb39e00700 1 --2- 172.21.15.153:0/1618647442 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fcb2006e460 0x7fcb20070920 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c005950 tx=0x7fcb2c0058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:39:39.190 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.192+0000 7fcb33400700 1 -- 172.21.15.153:0/1618647442 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fcb2405c5b0 con 0x7fcb3c068d30 2024-03-21T10:39:39.433 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.435+0000 7fcb41200700 1 -- 172.21.15.153:0/1618647442 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcb3c199290 con 0x7fcb2006e460 2024-03-21T10:39:39.433 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:39:39 smithi153 ceph-mon[113733]: pgmap v835: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:39:39.434 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.437+0000 7fcb33400700 1 -- 172.21.15.153:0/1618647442 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fcb3c199290 con 0x7fcb2006e460 2024-03-21T10:39:39.438 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.439+0000 7fcb41200700 1 -- 172.21.15.153:0/1618647442 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fcb2006e460 msgr2=0x7fcb20070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:39:39.438 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.439+0000 7fcb41200700 1 --2- 172.21.15.153:0/1618647442 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fcb2006e460 0x7fcb20070920 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c005950 tx=0x7fcb2c0058e0 comp rx=0 tx=0).stop 2024-03-21T10:39:39.438 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.439+0000 7fcb41200700 1 -- 172.21.15.153:0/1618647442 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb3c068d30 msgr2=0x7fcb3c19ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:39:39.438 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.439+0000 7fcb41200700 1 --2- 172.21.15.153:0/1618647442 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb3c068d30 0x7fcb3c19ec20 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fcb240092e0 tx=0x7fcb2400be20 comp rx=0 tx=0).stop 2024-03-21T10:39:39.438 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.439+0000 7fcb41200700 1 -- 172.21.15.153:0/1618647442 shutdown_connections 2024-03-21T10:39:39.438 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.439+0000 7fcb41200700 1 --2- 172.21.15.153:0/1618647442 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcb3c068410 0x7fcb3c19e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:39:39.438 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.439+0000 7fcb41200700 1 --2- 172.21.15.153:0/1618647442 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fcb2006e460 0x7fcb20070920 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:39:39.438 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.439+0000 7fcb41200700 1 --2- 172.21.15.153:0/1618647442 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcb3c068d30 0x7fcb3c19ec20 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:39:39.439 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.440+0000 7fcb41200700 1 -- 172.21.15.153:0/1618647442 >> 172.21.15.153:0/1618647442 conn(0x7fcb3c0755a0 msgr2=0x7fcb3c0fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:39:39.439 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.440+0000 7fcb41200700 1 -- 172.21.15.153:0/1618647442 shutdown_connections 2024-03-21T10:39:39.439 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.440+0000 7fcb41200700 1 -- 172.21.15.153:0/1618647442 wait complete. 2024-03-21T10:39:39.447 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:39:39 smithi174 ceph-mon[88004]: pgmap v835: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:39:39.456 INFO:teuthology.orchestra.run.smithi153.stdout:true 2024-03-21T10:39:39.589 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.591+0000 7f2359a00700 1 -- 172.21.15.153:0/3019014160 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f23541039b0 msgr2=0x7f2354107a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:39:39.590 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.591+0000 7f2359a00700 1 --2- 172.21.15.153:0/3019014160 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f23541039b0 0x7f2354107a00 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f233c009910 tx=0x7f233c009c20 comp rx=0 tx=0).stop 2024-03-21T10:39:39.590 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.591+0000 7f2359a00700 1 -- 172.21.15.153:0/3019014160 shutdown_connections 2024-03-21T10:39:39.590 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.591+0000 7f2359a00700 1 --2- 172.21.15.153:0/3019014160 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f23541039b0 0x7f2354107a00 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:39:39.590 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.591+0000 7f2359a00700 1 --2- 172.21.15.153:0/3019014160 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2354103000 0x7f23541033e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:39:39.590 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.591+0000 7f2359a00700 1 -- 172.21.15.153:0/3019014160 >> 172.21.15.153:0/3019014160 conn(0x7f23540fe8d0 msgr2=0x7f2354100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:39:39.590 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.591+0000 7f2359a00700 1 -- 172.21.15.153:0/3019014160 shutdown_connections 2024-03-21T10:39:39.590 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.591+0000 7f2359a00700 1 -- 172.21.15.153:0/3019014160 wait complete. 2024-03-21T10:39:39.590 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.593+0000 7f2359a00700 1 Processor -- start 2024-03-21T10:39:39.591 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.593+0000 7f2359a00700 1 -- start start 2024-03-21T10:39:39.591 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.593+0000 7f2359a00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2354103000 0x7f235419e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:39:39.591 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.593+0000 7f2359a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f23541039b0 0x7f235419ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:39:39.591 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.593+0000 7f2359a00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f235419f390 con 0x7f23541039b0 2024-03-21T10:39:39.591 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.593+0000 7f2359a00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2354198840 con 0x7f2354103000 2024-03-21T10:39:39.591 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.594+0000 7f2352000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f23541039b0 0x7f235419ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:39:39.592 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.594+0000 7f2352000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f23541039b0 0x7f235419ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:48102/0 (socket says 172.21.15.153:48102) 2024-03-21T10:39:39.592 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.594+0000 7f2352000700 1 -- 172.21.15.153:0/1694215016 learned_addr learned my addr 172.21.15.153:0/1694215016 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:39:39.592 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.594+0000 7f2352a00700 1 --2- 172.21.15.153:0/1694215016 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2354103000 0x7f235419e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:39:39.592 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.594+0000 7f2352000700 1 -- 172.21.15.153:0/1694215016 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2354103000 msgr2=0x7f235419e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:39:39.592 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.594+0000 7f2352000700 1 --2- 172.21.15.153:0/1694215016 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2354103000 0x7f235419e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:39:39.592 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.594+0000 7f2352000700 1 -- 172.21.15.153:0/1694215016 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f233c0095f0 con 0x7f23541039b0 2024-03-21T10:39:39.592 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.594+0000 7f2352000700 1 --2- 172.21.15.153:0/1694215016 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f23541039b0 0x7f235419ed00 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f233c0092e0 tx=0x7f233c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:39:39.593 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.595+0000 7f234b400700 1 -- 172.21.15.153:0/1694215016 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f233c01d070 con 0x7f23541039b0 2024-03-21T10:39:39.593 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.595+0000 7f234b400700 1 -- 172.21.15.153:0/1694215016 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f233c005dc0 con 0x7f23541039b0 2024-03-21T10:39:39.593 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.595+0000 7f2359a00700 1 -- 172.21.15.153:0/1694215016 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2354198ac0 con 0x7f23541039b0 2024-03-21T10:39:39.593 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.595+0000 7f234b400700 1 -- 172.21.15.153:0/1694215016 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f233c017400 con 0x7f23541039b0 2024-03-21T10:39:39.593 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.595+0000 7f2359a00700 1 -- 172.21.15.153:0/1694215016 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2354198fb0 con 0x7f23541039b0 2024-03-21T10:39:39.594 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.596+0000 7f2359a00700 1 -- 172.21.15.153:0/1694215016 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f235410b400 con 0x7f23541039b0 2024-03-21T10:39:39.595 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.597+0000 7f234b400700 1 -- 172.21.15.153:0/1694215016 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f233c00f460 con 0x7f23541039b0 2024-03-21T10:39:39.595 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.598+0000 7f234b400700 1 --2- 172.21.15.153:0/1694215016 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f233806e460 0x7f2338070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:39:39.596 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.598+0000 7f234b400700 1 -- 172.21.15.153:0/1694215016 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f233c08f9b0 con 0x7f23541039b0 2024-03-21T10:39:39.596 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.598+0000 7f2352a00700 1 --2- 172.21.15.153:0/1694215016 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f233806e460 0x7f2338070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:39:39.597 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.599+0000 7f2352a00700 1 --2- 172.21.15.153:0/1694215016 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f233806e460 0x7f2338070920 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f2344005950 tx=0x7f23440058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:39:39.600 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.603+0000 7f234b400700 1 -- 172.21.15.153:0/1694215016 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f233c05c580 con 0x7f23541039b0 2024-03-21T10:39:39.826 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.828+0000 7f2359a00700 1 -- 172.21.15.153:0/1694215016 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2354199220 con 0x7f233806e460 2024-03-21T10:39:39.828 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.830+0000 7f234b400700 1 -- 172.21.15.153:0/1694215016 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f2354199220 con 0x7f233806e460 2024-03-21T10:39:39.831 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.833+0000 7f2359a00700 1 -- 172.21.15.153:0/1694215016 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f233806e460 msgr2=0x7f2338070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:39:39.831 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.833+0000 7f2359a00700 1 --2- 172.21.15.153:0/1694215016 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f233806e460 0x7f2338070920 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f2344005950 tx=0x7f23440058e0 comp rx=0 tx=0).stop 2024-03-21T10:39:39.831 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.833+0000 7f2359a00700 1 -- 172.21.15.153:0/1694215016 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f23541039b0 msgr2=0x7f235419ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:39:39.831 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.833+0000 7f2359a00700 1 --2- 172.21.15.153:0/1694215016 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f23541039b0 0x7f235419ed00 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f233c0092e0 tx=0x7f233c00be20 comp rx=0 tx=0).stop 2024-03-21T10:39:39.831 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.833+0000 7f2359a00700 1 -- 172.21.15.153:0/1694215016 shutdown_connections 2024-03-21T10:39:39.831 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.833+0000 7f2359a00700 1 --2- 172.21.15.153:0/1694215016 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2354103000 0x7f235419e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:39:39.832 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.833+0000 7f2359a00700 1 --2- 172.21.15.153:0/1694215016 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f233806e460 0x7f2338070920 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:39:39.832 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.833+0000 7f2359a00700 1 --2- 172.21.15.153:0/1694215016 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f23541039b0 0x7f235419ed00 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:39:39.832 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.833+0000 7f2359a00700 1 -- 172.21.15.153:0/1694215016 >> 172.21.15.153:0/1694215016 conn(0x7f23540fe8d0 msgr2=0x7f2354100cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:39:39.832 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.833+0000 7f2359a00700 1 -- 172.21.15.153:0/1694215016 shutdown_connections 2024-03-21T10:39:39.832 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.834+0000 7f2359a00700 1 -- 172.21.15.153:0/1694215016 wait complete. 2024-03-21T10:39:39.985 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.987+0000 7fa9fc800700 1 -- 172.21.15.153:0/517003252 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa9f4100dc0 msgr2=0x7fa9f41011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:39:39.986 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.987+0000 7fa9fc800700 1 --2- 172.21.15.153:0/517003252 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa9f4100dc0 0x7fa9f41011a0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fa9e8009910 tx=0x7fa9e8009c20 comp rx=0 tx=0).stop 2024-03-21T10:39:39.986 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.987+0000 7fa9fc800700 1 -- 172.21.15.153:0/517003252 shutdown_connections 2024-03-21T10:39:39.986 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.987+0000 7fa9fc800700 1 --2- 172.21.15.153:0/517003252 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa9f4101770 0x7fa9f41057c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:39:39.986 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.987+0000 7fa9fc800700 1 --2- 172.21.15.153:0/517003252 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa9f4100dc0 0x7fa9f41011a0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:39:39.986 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.987+0000 7fa9fc800700 1 -- 172.21.15.153:0/517003252 >> 172.21.15.153:0/517003252 conn(0x7fa9f40fc650 msgr2=0x7fa9f40fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:39:39.986 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.988+0000 7fa9fc800700 1 -- 172.21.15.153:0/517003252 shutdown_connections 2024-03-21T10:39:39.986 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.988+0000 7fa9fc800700 1 -- 172.21.15.153:0/517003252 wait complete. 2024-03-21T10:39:39.987 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.989+0000 7fa9fc800700 1 Processor -- start 2024-03-21T10:39:39.987 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.989+0000 7fa9fc800700 1 -- start start 2024-03-21T10:39:39.987 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.989+0000 7fa9fc800700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa9f4100dc0 0x7fa9f4198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:39:39.987 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.989+0000 7fa9fc800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa9f4101770 0x7fa9f4198e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:39:39.987 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.989+0000 7fa9fc800700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f4199520 con 0x7fa9f4101770 2024-03-21T10:39:39.987 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.989+0000 7fa9fc800700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f419d330 con 0x7fa9f4100dc0 2024-03-21T10:39:39.988 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.989+0000 7fa9f9400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa9f4101770 0x7fa9f4198e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:39:39.988 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.990+0000 7fa9f9400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa9f4101770 0x7fa9f4198e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:48120/0 (socket says 172.21.15.153:48120) 2024-03-21T10:39:39.988 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.990+0000 7fa9f9400700 1 -- 172.21.15.153:0/102509348 learned_addr learned my addr 172.21.15.153:0/102509348 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:39:39.988 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.990+0000 7fa9f9e00700 1 --2- 172.21.15.153:0/102509348 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa9f4100dc0 0x7fa9f4198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:39:39.988 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.990+0000 7fa9f9400700 1 -- 172.21.15.153:0/102509348 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa9f4100dc0 msgr2=0x7fa9f4198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:39:39.988 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.990+0000 7fa9f9400700 1 --2- 172.21.15.153:0/102509348 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa9f4100dc0 0x7fa9f4198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:39:39.989 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.990+0000 7fa9f9400700 1 -- 172.21.15.153:0/102509348 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9e80095f0 con 0x7fa9f4101770 2024-03-21T10:39:39.989 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.990+0000 7fa9f9400700 1 --2- 172.21.15.153:0/102509348 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa9f4101770 0x7fa9f4198e90 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fa9e400d8d0 tx=0x7fa9e400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:39:39.989 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.991+0000 7fa9eea00700 1 -- 172.21.15.153:0/102509348 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa9e40041f0 con 0x7fa9f4101770 2024-03-21T10:39:39.989 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.991+0000 7fa9fc800700 1 -- 172.21.15.153:0/102509348 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9f419d610 con 0x7fa9f4101770 2024-03-21T10:39:39.989 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.991+0000 7fa9eea00700 1 -- 172.21.15.153:0/102509348 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa9e4007620 con 0x7fa9f4101770 2024-03-21T10:39:39.989 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.991+0000 7fa9eea00700 1 -- 172.21.15.153:0/102509348 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa9e4003c50 con 0x7fa9f4101770 2024-03-21T10:39:39.989 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.991+0000 7fa9fc800700 1 -- 172.21.15.153:0/102509348 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9f419db60 con 0x7fa9f4101770 2024-03-21T10:39:39.990 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.992+0000 7fa9fc800700 1 -- 172.21.15.153:0/102509348 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9f4104f40 con 0x7fa9f4101770 2024-03-21T10:39:39.990 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.993+0000 7fa9eea00700 1 -- 172.21.15.153:0/102509348 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fa9e4004350 con 0x7fa9f4101770 2024-03-21T10:39:39.992 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.994+0000 7fa9eea00700 1 --2- 172.21.15.153:0/102509348 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa9dc06e460 0x7fa9dc070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:39:39.992 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.994+0000 7fa9eea00700 1 -- 172.21.15.153:0/102509348 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fa9e408e130 con 0x7fa9f4101770 2024-03-21T10:39:39.992 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.994+0000 7fa9f9e00700 1 --2- 172.21.15.153:0/102509348 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa9dc06e460 0x7fa9dc070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:39:39.992 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.995+0000 7fa9f9e00700 1 --2- 172.21.15.153:0/102509348 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa9dc06e460 0x7fa9dc070920 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fa9e800b5c0 tx=0x7fa9e801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:39:39.996 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:39.999+0000 7fa9eea00700 1 -- 172.21.15.153:0/102509348 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa9e405ac80 con 0x7fa9f4101770 2024-03-21T10:39:40.216 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:40.218+0000 7fa9fc800700 1 -- 172.21.15.153:0/102509348 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fa9f419ddf0 con 0x7fa9dc06e460 2024-03-21T10:39:40.245 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:40.247+0000 7fa9eea00700 1 -- 172.21.15.153:0/102509348 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fa9f419ddf0 con 0x7fa9dc06e460 2024-03-21T10:39:40.245 INFO:teuthology.orchestra.run.smithi153.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-21T10:39:40.246 INFO:teuthology.orchestra.run.smithi153.stdout:alertmanager.smithi153 smithi153 *:9093,9094 running (28m) 4m ago 42m 15.6M - 0.25.0 c8568f914cd2 9ed3f2200b5f 2024-03-21T10:39:40.246 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi153 smithi153 running (42m) 4m ago 42m 10.4M - 18.2.1 5be31c24972a 5b61994396a6 2024-03-21T10:39:40.246 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi174 smithi174 running (40m) 4m ago 40m 10.1M - 18.2.1 5be31c24972a 4f7a5fc3ac28 2024-03-21T10:39:40.246 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi153 smithi153 running (24m) 4m ago 42m 6656k - 19.0.0-2231-g2edfee68 237e43c7a65f 32c21104d5b0 2024-03-21T10:39:40.246 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi174 smithi174 running (24m) 4m ago 40m 6660k - 19.0.0-2231-g2edfee68 237e43c7a65f a04e48967ab6 2024-03-21T10:39:40.246 INFO:teuthology.orchestra.run.smithi153.stdout:grafana.smithi153 smithi153 *:3000 running (27m) 4m ago 41m 75.2M - 9.4.12 2bacad6d85d8 ee24026ab786 2024-03-21T10:39:40.246 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.iuyluj smithi153 running (20m) 4m ago 34m 18.7M - 19.0.0-2231-g2edfee68 237e43c7a65f 87aff8a5de70 2024-03-21T10:39:40.246 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.zipckr smithi153 running (20m) 4m ago 35m 18.9M - 19.0.0-2231-g2edfee68 237e43c7a65f ccc262bd5dd3 2024-03-21T10:39:40.246 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.nhozag smithi174 running (4m) 4m ago 34m 10.9M - 19.0.0-2231-g2edfee68 237e43c7a65f 9e02aea72c96 2024-03-21T10:39:40.246 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.yyimst smithi174 running (19m) 4m ago 35m 18.5M - 19.0.0-2231-g2edfee68 237e43c7a65f fc1962f9351e 2024-03-21T10:39:40.246 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi153.tluxni smithi153 *:8443,9283,8765 running (31m) 4m ago 44m 512M - 19.0.0-2231-g2edfee68 237e43c7a65f 7ffee0e55382 2024-03-21T10:39:40.246 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi174.qpcwat smithi174 *:8443,9283,8765 running (30m) 4m ago 40m 459M - 19.0.0-2231-g2edfee68 237e43c7a65f 748968f216ab 2024-03-21T10:39:40.246 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi153 smithi153 running (26m) 4m ago 44m 70.8M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f dd8038303dc3 2024-03-21T10:39:40.247 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi174 smithi174 running (25m) 4m ago 40m 51.9M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f 29d78a788f2c 2024-03-21T10:39:40.247 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi153 smithi153 *:9100 running (29m) 4m ago 42m 8140k - 1.5.0 0da6a335fe13 bb835edc0e0b 2024-03-21T10:39:40.247 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi174 smithi174 *:9100 running (29m) 4m ago 40m 8610k - 1.5.0 0da6a335fe13 7c4ecad790fa 2024-03-21T10:39:40.247 INFO:teuthology.orchestra.run.smithi153.stdout:osd.0 smithi153 running (24m) 4m ago 39m 118M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 507900fd4791 2024-03-21T10:39:40.247 INFO:teuthology.orchestra.run.smithi153.stdout:osd.1 smithi153 running (23m) 4m ago 39m 93.0M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 0d646452e9c1 2024-03-21T10:39:40.247 INFO:teuthology.orchestra.run.smithi153.stdout:osd.2 smithi153 running (22m) 4m ago 38m 91.2M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 1326f2afdfcd 2024-03-21T10:39:40.247 INFO:teuthology.orchestra.run.smithi153.stdout:osd.3 smithi174 running (22m) 4m ago 37m 87.5M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 70aafac71ad5 2024-03-21T10:39:40.247 INFO:teuthology.orchestra.run.smithi153.stdout:osd.4 smithi174 running (21m) 4m ago 37m 95.8M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 0d746ce6e79d 2024-03-21T10:39:40.247 INFO:teuthology.orchestra.run.smithi153.stdout:osd.5 smithi174 running (20m) 4m ago 36m 90.3M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 53e5f35b8cac 2024-03-21T10:39:40.247 INFO:teuthology.orchestra.run.smithi153.stdout:prometheus.smithi153 smithi153 *:9095 running (28m) 4m ago 41m 88.1M - 2.43.0 a07b618ecd1d 7d95561e50c2 2024-03-21T10:39:40.249 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:40.251+0000 7fa9fc800700 1 -- 172.21.15.153:0/102509348 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa9dc06e460 msgr2=0x7fa9dc070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:39:40.249 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:40.251+0000 7fa9fc800700 1 --2- 172.21.15.153:0/102509348 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa9dc06e460 0x7fa9dc070920 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fa9e800b5c0 tx=0x7fa9e801a040 comp rx=0 tx=0).stop 2024-03-21T10:39:40.249 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:40.251+0000 7fa9fc800700 1 -- 172.21.15.153:0/102509348 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa9f4101770 msgr2=0x7fa9f4198e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:39:40.249 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:40.251+0000 7fa9fc800700 1 --2- 172.21.15.153:0/102509348 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa9f4101770 0x7fa9f4198e90 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fa9e400d8d0 tx=0x7fa9e400dc90 comp rx=0 tx=0).stop 2024-03-21T10:39:40.249 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:40.251+0000 7fa9fc800700 1 -- 172.21.15.153:0/102509348 shutdown_connections 2024-03-21T10:39:40.249 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:40.251+0000 7fa9fc800700 1 --2- 172.21.15.153:0/102509348 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa9f4100dc0 0x7fa9f4198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:39:40.249 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:40.251+0000 7fa9fc800700 1 --2- 172.21.15.153:0/102509348 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa9dc06e460 0x7fa9dc070920 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:39:40.249 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:40.251+0000 7fa9fc800700 1 --2- 172.21.15.153:0/102509348 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa9f4101770 0x7fa9f4198e90 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:39:40.249 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:40.251+0000 7fa9fc800700 1 -- 172.21.15.153:0/102509348 >> 172.21.15.153:0/102509348 conn(0x7fa9f40fc650 msgr2=0x7fa9f40fddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:39:40.249 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:40.251+0000 7fa9fc800700 1 -- 172.21.15.153:0/102509348 shutdown_connections 2024-03-21T10:39:40.250 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:40.251+0000 7fa9fc800700 1 -- 172.21.15.153:0/102509348 wait complete. 2024-03-21T10:39:40.404 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:40.406+0000 7f9454000700 1 -- 172.21.15.153:0/4229087197 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f944c1071d0 msgr2=0x7f944c107650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:39:40.404 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:40.406+0000 7f9454000700 1 --2- 172.21.15.153:0/4229087197 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f944c1071d0 0x7f944c107650 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f9438009910 tx=0x7f9438009c20 comp rx=0 tx=0).stop 2024-03-21T10:39:40.404 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:40.406+0000 7f9454000700 1 -- 172.21.15.153:0/4229087197 shutdown_connections 2024-03-21T10:39:40.405 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:40.406+0000 7f9454000700 1 --2- 172.21.15.153:0/4229087197 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f944c1071d0 0x7f944c107650 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:39:40.405 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:40.406+0000 7f9454000700 1 --2- 172.21.15.153:0/4229087197 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f944c10d280 0x7f944c10d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:39:40.405 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:40.406+0000 7f9454000700 1 -- 172.21.15.153:0/4229087197 >> 172.21.15.153:0/4229087197 conn(0x7f944c0769e0 msgr2=0x7f944c078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:39:40.405 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:40.406+0000 7f9454000700 1 -- 172.21.15.153:0/4229087197 shutdown_connections 2024-03-21T10:39:40.405 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:40.406+0000 7f9454000700 1 -- 172.21.15.153:0/4229087197 wait complete. 2024-03-21T10:39:40.406 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:40.407+0000 7f9454000700 1 Processor -- start 2024-03-21T10:39:40.407 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:40.407+0000 7f9454000700 1 -- start start 2024-03-21T10:39:40.407 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:40.408+0000 7f9454000700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f944c1071d0 0x7f944c19ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:39:40.407 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:40.408+0000 7f9454000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f944c10d280 0x7f944c19d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:39:40.407 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:40.408+0000 7f9454000700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f944c19d880 con 0x7f944c10d280 2024-03-21T10:39:40.407 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:40.408+0000 7f9454000700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f944c1a05b0 con 0x7f944c1071d0 2024-03-21T10:39:40.407 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:40.408+0000 7f9450c00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f944c10d280 0x7f944c19d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:39:40.407 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:40.408+0000 7f9450c00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f944c10d280 0x7f944c19d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:48144/0 (socket says 172.21.15.153:48144) 2024-03-21T10:39:40.407 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:40.408+0000 7f9450c00700 1 -- 172.21.15.153:0/3845794685 learned_addr learned my addr 172.21.15.153:0/3845794685 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:39:40.407 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:40.408+0000 7f9451600700 1 --2- 172.21.15.153:0/3845794685 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f944c1071d0 0x7f944c19ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:39:40.407 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:40.408+0000 7f9450c00700 1 -- 172.21.15.153:0/3845794685 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f944c1071d0 msgr2=0x7f944c19ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:39:40.407 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:40.408+0000 7f9450c00700 1 --2- 172.21.15.153:0/3845794685 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f944c1071d0 0x7f944c19ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:39:40.407 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:40.408+0000 7f9450c00700 1 -- 172.21.15.153:0/3845794685 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94380095f0 con 0x7f944c10d280 2024-03-21T10:39:40.407 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:40.409+0000 7f9450c00700 1 --2- 172.21.15.153:0/3845794685 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f944c10d280 0x7f944c19d1f0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f9438006010 tx=0x7f943800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:39:40.408 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:40.409+0000 7f944a000700 1 -- 172.21.15.153:0/3845794685 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f943801d070 con 0x7f944c10d280 2024-03-21T10:39:40.408 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:40.409+0000 7f944a000700 1 -- 172.21.15.153:0/3845794685 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9438004fb0 con 0x7f944c10d280 2024-03-21T10:39:40.408 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:40.409+0000 7f944a000700 1 -- 172.21.15.153:0/3845794685 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9438022ea0 con 0x7f944c10d280 2024-03-21T10:39:40.408 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:40.409+0000 7f9454000700 1 -- 172.21.15.153:0/3845794685 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f944c1a0830 con 0x7f944c10d280 2024-03-21T10:39:40.408 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:40.409+0000 7f9454000700 1 -- 172.21.15.153:0/3845794685 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f944c1a0da0 con 0x7f944c10d280 2024-03-21T10:39:40.408 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:40.410+0000 7f9454000700 1 -- 172.21.15.153:0/3845794685 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f944c10f6c0 con 0x7f944c10d280 2024-03-21T10:39:40.409 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:40.411+0000 7f944a000700 1 -- 172.21.15.153:0/3845794685 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f9438022470 con 0x7f944c10d280 2024-03-21T10:39:40.411 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:40.413+0000 7f944a000700 1 --2- 172.21.15.153:0/3845794685 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f943406e460 0x7f9434070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:39:40.411 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:40.413+0000 7f944a000700 1 -- 172.21.15.153:0/3845794685 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f943808fb20 con 0x7f944c10d280 2024-03-21T10:39:40.411 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:40.413+0000 7f9451600700 1 --2- 172.21.15.153:0/3845794685 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f943406e460 0x7f9434070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:39:40.412 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:40.414+0000 7f9451600700 1 --2- 172.21.15.153:0/3845794685 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f943406e460 0x7f9434070920 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f9440005fd0 tx=0x7f944000a560 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:39:40.416 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:40.418+0000 7f944a000700 1 -- 172.21.15.153:0/3845794685 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f9438058bf0 con 0x7f944c10d280 2024-03-21T10:39:40.447 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:39:40 smithi174 ceph-mon[88004]: from='client.35490 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:39:40.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:39:40 smithi153 ceph-mon[113733]: from='client.35490 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:39:40.725 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:40.726+0000 7f9454000700 1 -- 172.21.15.153:0/3845794685 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f944c04ea90 con 0x7f944c10d280 2024-03-21T10:39:40.726 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:40.728+0000 7f944a000700 1 -- 172.21.15.153:0/3845794685 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f9438022720 con 0x7f944c10d280 2024-03-21T10:39:40.726 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:39:40.726 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": { 2024-03-21T10:39:40.726 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:39:40.726 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:39:40.726 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": { 2024-03-21T10:39:40.727 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:39:40.727 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:39:40.727 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": { 2024-03-21T10:39:40.727 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 6 2024-03-21T10:39:40.727 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:39:40.727 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": { 2024-03-21T10:39:40.727 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 4 2024-03-21T10:39:40.727 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:39:40.727 INFO:teuthology.orchestra.run.smithi153.stdout: "overall": { 2024-03-21T10:39:40.727 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 14 2024-03-21T10:39:40.727 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-03-21T10:39:40.727 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:39:40.729 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:40.731+0000 7f9454000700 1 -- 172.21.15.153:0/3845794685 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f943406e460 msgr2=0x7f9434070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:39:40.729 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:40.731+0000 7f9454000700 1 --2- 172.21.15.153:0/3845794685 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f943406e460 0x7f9434070920 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f9440005fd0 tx=0x7f944000a560 comp rx=0 tx=0).stop 2024-03-21T10:39:40.729 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:40.731+0000 7f9454000700 1 -- 172.21.15.153:0/3845794685 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f944c10d280 msgr2=0x7f944c19d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:39:40.729 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:40.731+0000 7f9454000700 1 --2- 172.21.15.153:0/3845794685 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f944c10d280 0x7f944c19d1f0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f9438006010 tx=0x7f943800be20 comp rx=0 tx=0).stop 2024-03-21T10:39:40.729 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:40.731+0000 7f9454000700 1 -- 172.21.15.153:0/3845794685 shutdown_connections 2024-03-21T10:39:40.729 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:40.731+0000 7f9454000700 1 --2- 172.21.15.153:0/3845794685 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f944c1071d0 0x7f944c19ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:39:40.729 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:40.731+0000 7f9454000700 1 --2- 172.21.15.153:0/3845794685 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f943406e460 0x7f9434070920 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:39:40.729 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:40.731+0000 7f9454000700 1 --2- 172.21.15.153:0/3845794685 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f944c10d280 0x7f944c19d1f0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:39:40.729 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:40.731+0000 7f9454000700 1 -- 172.21.15.153:0/3845794685 >> 172.21.15.153:0/3845794685 conn(0x7f944c0769e0 msgr2=0x7f944c078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:39:40.730 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:40.731+0000 7f9454000700 1 -- 172.21.15.153:0/3845794685 shutdown_connections 2024-03-21T10:39:40.730 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:40.731+0000 7f9454000700 1 -- 172.21.15.153:0/3845794685 wait complete. 2024-03-21T10:39:40.884 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:40.886+0000 7fb6b4c00700 1 -- 172.21.15.153:0/3131399397 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb6b0100dc0 msgr2=0x7fb6b01011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:39:40.885 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:40.886+0000 7fb6b4c00700 1 --2- 172.21.15.153:0/3131399397 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb6b0100dc0 0x7fb6b01011a0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fb6a0009910 tx=0x7fb6a0009c20 comp rx=0 tx=0).stop 2024-03-21T10:39:40.885 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:40.886+0000 7fb6b4c00700 1 -- 172.21.15.153:0/3131399397 shutdown_connections 2024-03-21T10:39:40.885 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:40.886+0000 7fb6b4c00700 1 --2- 172.21.15.153:0/3131399397 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb6b0101770 0x7fb6b01057c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:39:40.885 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:40.886+0000 7fb6b4c00700 1 --2- 172.21.15.153:0/3131399397 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb6b0100dc0 0x7fb6b01011a0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:39:40.885 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:40.886+0000 7fb6b4c00700 1 -- 172.21.15.153:0/3131399397 >> 172.21.15.153:0/3131399397 conn(0x7fb6b00fc650 msgr2=0x7fb6b00fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:39:40.885 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:40.886+0000 7fb6b4c00700 1 -- 172.21.15.153:0/3131399397 shutdown_connections 2024-03-21T10:39:40.885 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:40.886+0000 7fb6b4c00700 1 -- 172.21.15.153:0/3131399397 wait complete. 2024-03-21T10:39:40.887 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:40.887+0000 7fb6b4c00700 1 Processor -- start 2024-03-21T10:39:40.887 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:40.888+0000 7fb6b4c00700 1 -- start start 2024-03-21T10:39:40.887 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:40.888+0000 7fb6b4c00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb6b0100dc0 0x7fb6b0198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:39:40.887 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:40.888+0000 7fb6b4c00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb6b0101770 0x7fb6b0198e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:39:40.887 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:40.888+0000 7fb6b4c00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b0199520 con 0x7fb6b0100dc0 2024-03-21T10:39:40.887 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:40.888+0000 7fb6b4c00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b019d330 con 0x7fb6b0101770 2024-03-21T10:39:40.887 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:40.888+0000 7fb6ade00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb6b0100dc0 0x7fb6b0198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:39:40.887 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:40.888+0000 7fb6ade00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb6b0100dc0 0x7fb6b0198950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:48168/0 (socket says 172.21.15.153:48168) 2024-03-21T10:39:40.887 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:40.888+0000 7fb6ade00700 1 -- 172.21.15.153:0/3726881736 learned_addr learned my addr 172.21.15.153:0/3726881736 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:39:40.887 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:40.889+0000 7fb6ad400700 1 --2- 172.21.15.153:0/3726881736 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb6b0101770 0x7fb6b0198e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:39:40.887 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:40.889+0000 7fb6ade00700 1 -- 172.21.15.153:0/3726881736 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb6b0101770 msgr2=0x7fb6b0198e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:39:40.887 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:40.889+0000 7fb6ade00700 1 --2- 172.21.15.153:0/3726881736 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb6b0101770 0x7fb6b0198e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:39:40.888 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:40.889+0000 7fb6ade00700 1 -- 172.21.15.153:0/3726881736 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6a00095f0 con 0x7fb6b0100dc0 2024-03-21T10:39:40.888 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:40.889+0000 7fb6ade00700 1 --2- 172.21.15.153:0/3726881736 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb6b0100dc0 0x7fb6b0198950 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fb6a0006010 tx=0x7fb6a000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:39:40.888 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:40.889+0000 7fb6a6a00700 1 -- 172.21.15.153:0/3726881736 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb6a001d070 con 0x7fb6b0100dc0 2024-03-21T10:39:40.888 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:40.889+0000 7fb6a6a00700 1 -- 172.21.15.153:0/3726881736 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb6a000f460 con 0x7fb6b0100dc0 2024-03-21T10:39:40.889 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:40.889+0000 7fb6a6a00700 1 -- 172.21.15.153:0/3726881736 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb6a0017630 con 0x7fb6b0100dc0 2024-03-21T10:39:40.889 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:40.889+0000 7fb6b4c00700 1 -- 172.21.15.153:0/3726881736 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6b019d5b0 con 0x7fb6b0100dc0 2024-03-21T10:39:40.889 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:40.890+0000 7fb6b4c00700 1 -- 172.21.15.153:0/3726881736 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6b019da50 con 0x7fb6b0100dc0 2024-03-21T10:39:40.889 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:40.891+0000 7fb6b4c00700 1 -- 172.21.15.153:0/3726881736 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6b0104f40 con 0x7fb6b0100dc0 2024-03-21T10:39:40.889 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:40.892+0000 7fb6a6a00700 1 -- 172.21.15.153:0/3726881736 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fb6a000f5d0 con 0x7fb6b0100dc0 2024-03-21T10:39:40.891 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:40.893+0000 7fb6a6a00700 1 --2- 172.21.15.153:0/3726881736 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb69406e460 0x7fb694070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:39:40.891 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:40.893+0000 7fb6a6a00700 1 -- 172.21.15.153:0/3726881736 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fb6a008f9a0 con 0x7fb6b0100dc0 2024-03-21T10:39:40.891 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:40.893+0000 7fb6ad400700 1 --2- 172.21.15.153:0/3726881736 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb69406e460 0x7fb694070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:39:40.891 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:40.894+0000 7fb6ad400700 1 --2- 172.21.15.153:0/3726881736 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb69406e460 0x7fb694070920 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fb69c005950 tx=0x7fb69c0058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:39:40.895 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:40.898+0000 7fb6a6a00700 1 -- 172.21.15.153:0/3726881736 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb6a005c570 con 0x7fb6b0100dc0 2024-03-21T10:39:41.165 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:41.167+0000 7fb6b4c00700 1 -- 172.21.15.153:0/3726881736 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fb6b004ea90 con 0x7fb6b0100dc0 2024-03-21T10:39:41.165 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:41.168+0000 7fb6a6a00700 1 -- 172.21.15.153:0/3726881736 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1876 (secure 0 0 0) 0x7fb6a005c100 con 0x7fb6b0100dc0 2024-03-21T10:39:41.166 INFO:teuthology.orchestra.run.smithi153.stdout:e32 2024-03-21T10:39:41.166 INFO:teuthology.orchestra.run.smithi153.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-21T10:39:41.166 INFO:teuthology.orchestra.run.smithi153.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:39:41.167 INFO:teuthology.orchestra.run.smithi153.stdout:legacy client fscid: 1 2024-03-21T10:39:41.167 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:39:41.167 INFO:teuthology.orchestra.run.smithi153.stdout:Filesystem 'cephfs' (1) 2024-03-21T10:39:41.167 INFO:teuthology.orchestra.run.smithi153.stdout:fs_name cephfs 2024-03-21T10:39:41.167 INFO:teuthology.orchestra.run.smithi153.stdout:epoch 23 2024-03-21T10:39:41.167 INFO:teuthology.orchestra.run.smithi153.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-21T10:39:41.167 INFO:teuthology.orchestra.run.smithi153.stdout:created 2024-03-21T10:04:27.086576+0000 2024-03-21T10:39:41.167 INFO:teuthology.orchestra.run.smithi153.stdout:modified 2024-03-21T10:19:05.022092+0000 2024-03-21T10:39:41.167 INFO:teuthology.orchestra.run.smithi153.stdout:tableserver 0 2024-03-21T10:39:41.167 INFO:teuthology.orchestra.run.smithi153.stdout:root 0 2024-03-21T10:39:41.167 INFO:teuthology.orchestra.run.smithi153.stdout:session_timeout 60 2024-03-21T10:39:41.167 INFO:teuthology.orchestra.run.smithi153.stdout:session_autoclose 300 2024-03-21T10:39:41.167 INFO:teuthology.orchestra.run.smithi153.stdout:max_file_size 1099511627776 2024-03-21T10:39:41.167 INFO:teuthology.orchestra.run.smithi153.stdout:max_xattr_size 65536 2024-03-21T10:39:41.167 INFO:teuthology.orchestra.run.smithi153.stdout:required_client_features {} 2024-03-21T10:39:41.167 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure 0 2024-03-21T10:39:41.168 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure_osd_epoch 94 2024-03-21T10:39:41.168 INFO:teuthology.orchestra.run.smithi153.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:39:41.168 INFO:teuthology.orchestra.run.smithi153.stdout:max_mds 2 2024-03-21T10:39:41.168 INFO:teuthology.orchestra.run.smithi153.stdout:in 0,1 2024-03-21T10:39:41.168 INFO:teuthology.orchestra.run.smithi153.stdout:up {} 2024-03-21T10:39:41.168 INFO:teuthology.orchestra.run.smithi153.stdout:failed 0,1 2024-03-21T10:39:41.168 INFO:teuthology.orchestra.run.smithi153.stdout:damaged 2024-03-21T10:39:41.168 INFO:teuthology.orchestra.run.smithi153.stdout:stopped 2024-03-21T10:39:41.168 INFO:teuthology.orchestra.run.smithi153.stdout:data_pools [3] 2024-03-21T10:39:41.168 INFO:teuthology.orchestra.run.smithi153.stdout:metadata_pool 2 2024-03-21T10:39:41.168 INFO:teuthology.orchestra.run.smithi153.stdout:inline_data enabled 2024-03-21T10:39:41.168 INFO:teuthology.orchestra.run.smithi153.stdout:balancer 2024-03-21T10:39:41.168 INFO:teuthology.orchestra.run.smithi153.stdout:bal_rank_mask -1 2024-03-21T10:39:41.168 INFO:teuthology.orchestra.run.smithi153.stdout:standby_count_wanted 1 2024-03-21T10:39:41.168 INFO:teuthology.orchestra.run.smithi153.stdout:qdb_cluster leader: 0 members: 2024-03-21T10:39:41.168 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:39:41.169 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:39:41.169 INFO:teuthology.orchestra.run.smithi153.stdout:Standby daemons: 2024-03-21T10:39:41.169 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:39:41.169 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.zipckr{-1:34442} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6826/1103846295,v1:172.21.15.153:6827/1103846295] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:39:41.169 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.iuyluj{-1:34474} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6828/1738630832,v1:172.21.15.153:6829/1738630832] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:39:41.169 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.yyimst{-1:44313} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.174:6824/4127786797,v1:172.21.15.174:6825/4127786797] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:39:41.169 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.nhozag{-1:44687} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.174:6826/2767517845,v1:172.21.15.174:6827/2767517845] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:39:41.170 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:41.171+0000 7fb6b4c00700 1 -- 172.21.15.153:0/3726881736 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb69406e460 msgr2=0x7fb694070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:39:41.170 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:41.171+0000 7fb6b4c00700 1 --2- 172.21.15.153:0/3726881736 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb69406e460 0x7fb694070920 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fb69c005950 tx=0x7fb69c0058e0 comp rx=0 tx=0).stop 2024-03-21T10:39:41.170 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:41.171+0000 7fb6b4c00700 1 -- 172.21.15.153:0/3726881736 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb6b0100dc0 msgr2=0x7fb6b0198950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:39:41.170 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:41.171+0000 7fb6b4c00700 1 --2- 172.21.15.153:0/3726881736 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb6b0100dc0 0x7fb6b0198950 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fb6a0006010 tx=0x7fb6a000be20 comp rx=0 tx=0).stop 2024-03-21T10:39:41.170 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:41.171+0000 7fb6b4c00700 1 -- 172.21.15.153:0/3726881736 shutdown_connections 2024-03-21T10:39:41.170 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:41.171+0000 7fb6b4c00700 1 --2- 172.21.15.153:0/3726881736 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb69406e460 0x7fb694070920 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:39:41.170 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:41.171+0000 7fb6b4c00700 1 --2- 172.21.15.153:0/3726881736 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb6b0100dc0 0x7fb6b0198950 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:39:41.170 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:41.171+0000 7fb6b4c00700 1 --2- 172.21.15.153:0/3726881736 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb6b0101770 0x7fb6b0198e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:39:41.170 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:41.171+0000 7fb6b4c00700 1 -- 172.21.15.153:0/3726881736 >> 172.21.15.153:0/3726881736 conn(0x7fb6b00fc650 msgr2=0x7fb6b00fddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:39:41.170 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:41.171+0000 7fb6b4c00700 1 -- 172.21.15.153:0/3726881736 shutdown_connections 2024-03-21T10:39:41.170 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:41.171+0000 7fb6b4c00700 1 -- 172.21.15.153:0/3726881736 wait complete. 2024-03-21T10:39:41.171 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 32 2024-03-21T10:39:41.321 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:41.323+0000 7f121ec00700 1 -- 172.21.15.153:0/334051442 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1218073ae0 msgr2=0x7f121810cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:39:41.321 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:41.323+0000 7f121ec00700 1 --2- 172.21.15.153:0/334051442 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1218073ae0 0x7f121810cc70 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f1204009910 tx=0x7f1204009c20 comp rx=0 tx=0).stop 2024-03-21T10:39:41.321 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:41.323+0000 7f121ec00700 1 -- 172.21.15.153:0/334051442 shutdown_connections 2024-03-21T10:39:41.321 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:41.323+0000 7f121ec00700 1 --2- 172.21.15.153:0/334051442 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1218073ae0 0x7f121810cc70 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:39:41.321 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:41.323+0000 7f121ec00700 1 --2- 172.21.15.153:0/334051442 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f12180731c0 0x7f12180735a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:39:41.322 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:41.323+0000 7f121ec00700 1 -- 172.21.15.153:0/334051442 >> 172.21.15.153:0/334051442 conn(0x7f12180fc440 msgr2=0x7f12180fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:39:41.322 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:41.323+0000 7f121ec00700 1 -- 172.21.15.153:0/334051442 shutdown_connections 2024-03-21T10:39:41.322 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:41.323+0000 7f121ec00700 1 -- 172.21.15.153:0/334051442 wait complete. 2024-03-21T10:39:41.323 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:41.324+0000 7f121ec00700 1 Processor -- start 2024-03-21T10:39:41.323 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:41.324+0000 7f121ec00700 1 -- start start 2024-03-21T10:39:41.323 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:41.325+0000 7f121ec00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f12180731c0 0x7f121819e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:39:41.323 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:41.325+0000 7f121ec00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1218073ae0 0x7f121819eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:39:41.323 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:41.325+0000 7f121ec00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f121819f0c0 con 0x7f1218073ae0 2024-03-21T10:39:41.323 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:41.325+0000 7f121ec00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1218198600 con 0x7f12180731c0 2024-03-21T10:39:41.323 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:41.325+0000 7f121ce00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1218073ae0 0x7f121819eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:39:41.323 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:41.325+0000 7f121ce00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1218073ae0 0x7f121819eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:48184/0 (socket says 172.21.15.153:48184) 2024-03-21T10:39:41.323 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:41.325+0000 7f121ce00700 1 -- 172.21.15.153:0/1897625954 learned_addr learned my addr 172.21.15.153:0/1897625954 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:39:41.324 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:41.325+0000 7f121d800700 1 --2- 172.21.15.153:0/1897625954 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f12180731c0 0x7f121819e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:39:41.324 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:41.326+0000 7f121ce00700 1 -- 172.21.15.153:0/1897625954 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f12180731c0 msgr2=0x7f121819e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:39:41.324 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:41.326+0000 7f121ce00700 1 --2- 172.21.15.153:0/1897625954 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f12180731c0 0x7f121819e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:39:41.324 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:41.326+0000 7f121ce00700 1 -- 172.21.15.153:0/1897625954 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12040095f0 con 0x7f1218073ae0 2024-03-21T10:39:41.324 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:41.326+0000 7f121ce00700 1 --2- 172.21.15.153:0/1897625954 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1218073ae0 0x7f121819eac0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f120400bd70 tx=0x7f120400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:39:41.324 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:41.326+0000 7f1216000700 1 -- 172.21.15.153:0/1897625954 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f120401d070 con 0x7f1218073ae0 2024-03-21T10:39:41.325 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:41.326+0000 7f1216000700 1 -- 172.21.15.153:0/1897625954 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f120400f460 con 0x7f1218073ae0 2024-03-21T10:39:41.325 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:41.326+0000 7f1216000700 1 -- 172.21.15.153:0/1897625954 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1204017630 con 0x7f1218073ae0 2024-03-21T10:39:41.325 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:41.326+0000 7f121ec00700 1 -- 172.21.15.153:0/1897625954 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1218198880 con 0x7f1218073ae0 2024-03-21T10:39:41.325 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:41.326+0000 7f121ec00700 1 -- 172.21.15.153:0/1897625954 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1218198d20 con 0x7f1218073ae0 2024-03-21T10:39:41.325 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:41.327+0000 7f121ec00700 1 -- 172.21.15.153:0/1897625954 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f121810a370 con 0x7f1218073ae0 2024-03-21T10:39:41.326 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:41.328+0000 7f1216000700 1 -- 172.21.15.153:0/1897625954 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f120400f5d0 con 0x7f1218073ae0 2024-03-21T10:39:41.328 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:41.330+0000 7f1216000700 1 --2- 172.21.15.153:0/1897625954 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f120006e460 0x7f1200070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:39:41.328 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:41.330+0000 7f1216000700 1 -- 172.21.15.153:0/1897625954 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f120408fb80 con 0x7f1218073ae0 2024-03-21T10:39:41.328 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:41.330+0000 7f121d800700 1 --2- 172.21.15.153:0/1897625954 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f120006e460 0x7f1200070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:39:41.328 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:41.331+0000 7f121d800700 1 --2- 172.21.15.153:0/1897625954 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f120006e460 0x7f1200070920 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f120c005ea0 tx=0x7f120c005e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:39:41.333 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:41.335+0000 7f1216000700 1 -- 172.21.15.153:0/1897625954 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f120405c6d0 con 0x7f1218073ae0 2024-03-21T10:39:41.448 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:39:41 smithi174 ceph-mon[88004]: pgmap v836: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:39:41.448 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:39:41 smithi174 ceph-mon[88004]: from='client.35494 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:39:41.448 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:39:41 smithi174 ceph-mon[88004]: from='client.35498 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:39:41.448 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:39:41 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/3845794685' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:39:41.448 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:39:41 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/3726881736' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:39:41.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:39:41 smithi153 ceph-mon[113733]: pgmap v836: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:39:41.553 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:39:41 smithi153 ceph-mon[113733]: from='client.35494 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:39:41.553 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:39:41 smithi153 ceph-mon[113733]: from='client.35498 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:39:41.553 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:39:41 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/3845794685' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:39:41.553 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:39:41 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/3726881736' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:39:41.561 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:41.563+0000 7f121ec00700 1 -- 172.21.15.153:0/1897625954 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f121802cca0 con 0x7f120006e460 2024-03-21T10:39:41.565 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:41.567+0000 7f1216000700 1 -- 172.21.15.153:0/1897625954 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f121802cca0 con 0x7f120006e460 2024-03-21T10:39:41.565 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:39:41.565 INFO:teuthology.orchestra.run.smithi153.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232", 2024-03-21T10:39:41.565 INFO:teuthology.orchestra.run.smithi153.stdout: "in_progress": true, 2024-03-21T10:39:41.565 INFO:teuthology.orchestra.run.smithi153.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-21T10:39:41.565 INFO:teuthology.orchestra.run.smithi153.stdout: "services_complete": [ 2024-03-21T10:39:41.565 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr", 2024-03-21T10:39:41.565 INFO:teuthology.orchestra.run.smithi153.stdout: "mon", 2024-03-21T10:39:41.565 INFO:teuthology.orchestra.run.smithi153.stdout: "mds", 2024-03-21T10:39:41.565 INFO:teuthology.orchestra.run.smithi153.stdout: "crash", 2024-03-21T10:39:41.566 INFO:teuthology.orchestra.run.smithi153.stdout: "osd" 2024-03-21T10:39:41.566 INFO:teuthology.orchestra.run.smithi153.stdout: ], 2024-03-21T10:39:41.566 INFO:teuthology.orchestra.run.smithi153.stdout: "progress": "16/23 daemons upgraded", 2024-03-21T10:39:41.566 INFO:teuthology.orchestra.run.smithi153.stdout: "message": "Currently upgrading mds daemons", 2024-03-21T10:39:41.566 INFO:teuthology.orchestra.run.smithi153.stdout: "is_paused": false 2024-03-21T10:39:41.566 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:39:41.567 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:41.569+0000 7f121ec00700 1 -- 172.21.15.153:0/1897625954 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f120006e460 msgr2=0x7f1200070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:39:41.567 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:41.569+0000 7f121ec00700 1 --2- 172.21.15.153:0/1897625954 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f120006e460 0x7f1200070920 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f120c005ea0 tx=0x7f120c005e30 comp rx=0 tx=0).stop 2024-03-21T10:39:41.567 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:41.569+0000 7f121ec00700 1 -- 172.21.15.153:0/1897625954 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1218073ae0 msgr2=0x7f121819eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:39:41.567 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:41.569+0000 7f121ec00700 1 --2- 172.21.15.153:0/1897625954 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1218073ae0 0x7f121819eac0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f120400bd70 tx=0x7f120400be50 comp rx=0 tx=0).stop 2024-03-21T10:39:41.567 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:41.569+0000 7f121ec00700 1 -- 172.21.15.153:0/1897625954 shutdown_connections 2024-03-21T10:39:41.567 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:41.569+0000 7f121ec00700 1 --2- 172.21.15.153:0/1897625954 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f12180731c0 0x7f121819e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:39:41.567 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:41.569+0000 7f121ec00700 1 --2- 172.21.15.153:0/1897625954 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f120006e460 0x7f1200070920 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:39:41.568 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:41.569+0000 7f121ec00700 1 --2- 172.21.15.153:0/1897625954 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1218073ae0 0x7f121819eac0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:39:41.568 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:41.569+0000 7f121ec00700 1 -- 172.21.15.153:0/1897625954 >> 172.21.15.153:0/1897625954 conn(0x7f12180fc440 msgr2=0x7f12181074b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:39:41.568 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:41.569+0000 7f121ec00700 1 -- 172.21.15.153:0/1897625954 shutdown_connections 2024-03-21T10:39:41.568 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:41.569+0000 7f121ec00700 1 -- 172.21.15.153:0/1897625954 wait complete. 2024-03-21T10:39:41.722 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:41.723+0000 7f1e45400700 1 -- 172.21.15.153:0/3381802553 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1e4010d280 msgr2=0x7f1e4010d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:39:41.722 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:41.723+0000 7f1e45400700 1 --2- 172.21.15.153:0/3381802553 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1e4010d280 0x7f1e4010d660 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f1e30009910 tx=0x7f1e30009c20 comp rx=0 tx=0).stop 2024-03-21T10:39:41.722 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:41.723+0000 7f1e45400700 1 -- 172.21.15.153:0/3381802553 shutdown_connections 2024-03-21T10:39:41.722 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:41.723+0000 7f1e45400700 1 --2- 172.21.15.153:0/3381802553 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1e401071d0 0x7f1e40107650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:39:41.722 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:41.723+0000 7f1e45400700 1 --2- 172.21.15.153:0/3381802553 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1e4010d280 0x7f1e4010d660 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:39:41.722 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:41.724+0000 7f1e45400700 1 -- 172.21.15.153:0/3381802553 >> 172.21.15.153:0/3381802553 conn(0x7f1e400769e0 msgr2=0x7f1e40078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:39:41.722 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:41.724+0000 7f1e45400700 1 -- 172.21.15.153:0/3381802553 shutdown_connections 2024-03-21T10:39:41.722 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:41.724+0000 7f1e45400700 1 -- 172.21.15.153:0/3381802553 wait complete. 2024-03-21T10:39:41.723 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:41.725+0000 7f1e45400700 1 Processor -- start 2024-03-21T10:39:41.724 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:41.725+0000 7f1e45400700 1 -- start start 2024-03-21T10:39:41.724 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:41.725+0000 7f1e45400700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1e401071d0 0x7f1e4019ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:39:41.724 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:41.725+0000 7f1e45400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1e4010d280 0x7f1e4019d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:39:41.724 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:41.725+0000 7f1e45400700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e4019da30 con 0x7f1e4010d280 2024-03-21T10:39:41.724 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:41.725+0000 7f1e45400700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e401a1770 con 0x7f1e401071d0 2024-03-21T10:39:41.724 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:41.725+0000 7f1e3de00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1e4010d280 0x7f1e4019d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:39:41.724 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:41.725+0000 7f1e3de00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1e4010d280 0x7f1e4019d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:48202/0 (socket says 172.21.15.153:48202) 2024-03-21T10:39:41.724 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:41.726+0000 7f1e3de00700 1 -- 172.21.15.153:0/79305019 learned_addr learned my addr 172.21.15.153:0/79305019 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:39:41.724 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:41.726+0000 7f1e3e800700 1 --2- 172.21.15.153:0/79305019 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1e401071d0 0x7f1e4019ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:39:41.724 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:41.726+0000 7f1e3de00700 1 -- 172.21.15.153:0/79305019 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1e401071d0 msgr2=0x7f1e4019ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:39:41.724 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:41.726+0000 7f1e3de00700 1 --2- 172.21.15.153:0/79305019 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1e401071d0 0x7f1e4019ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:39:41.724 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:41.726+0000 7f1e3de00700 1 -- 172.21.15.153:0/79305019 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e300095f0 con 0x7f1e4010d280 2024-03-21T10:39:41.725 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:41.726+0000 7f1e3de00700 1 --2- 172.21.15.153:0/79305019 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1e4010d280 0x7f1e4019d350 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f1e2c00b4b0 tx=0x7f1e2c00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:39:41.725 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:41.727+0000 7f1e37400700 1 -- 172.21.15.153:0/79305019 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1e2c011800 con 0x7f1e4010d280 2024-03-21T10:39:41.725 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:41.727+0000 7f1e37400700 1 -- 172.21.15.153:0/79305019 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1e2c011e20 con 0x7f1e4010d280 2024-03-21T10:39:41.725 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:41.727+0000 7f1e37400700 1 -- 172.21.15.153:0/79305019 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1e2c00f2e0 con 0x7f1e4010d280 2024-03-21T10:39:41.725 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:41.727+0000 7f1e45400700 1 -- 172.21.15.153:0/79305019 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e401a1a50 con 0x7f1e4010d280 2024-03-21T10:39:41.725 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:41.727+0000 7f1e45400700 1 -- 172.21.15.153:0/79305019 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e401a2020 con 0x7f1e4010d280 2024-03-21T10:39:41.726 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:41.728+0000 7f1e45400700 1 -- 172.21.15.153:0/79305019 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e4010f6f0 con 0x7f1e4010d280 2024-03-21T10:39:41.726 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:41.729+0000 7f1e37400700 1 -- 172.21.15.153:0/79305019 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f1e2c010380 con 0x7f1e4010d280 2024-03-21T10:39:41.728 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:41.730+0000 7f1e37400700 1 --2- 172.21.15.153:0/79305019 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f1e2406e460 0x7f1e24070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:39:41.728 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:41.730+0000 7f1e37400700 1 -- 172.21.15.153:0/79305019 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f1e2c08e0c0 con 0x7f1e4010d280 2024-03-21T10:39:41.728 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:41.731+0000 7f1e3e800700 1 --2- 172.21.15.153:0/79305019 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f1e2406e460 0x7f1e24070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:39:41.729 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:41.731+0000 7f1e3e800700 1 --2- 172.21.15.153:0/79305019 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f1e2406e460 0x7f1e24070920 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f1e30006010 tx=0x7f1e3001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:39:41.733 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:41.735+0000 7f1e37400700 1 -- 172.21.15.153:0/79305019 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f1e2c057210 con 0x7f1e4010d280 2024-03-21T10:39:42.038 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:42.040+0000 7f1e45400700 1 -- 172.21.15.153:0/79305019 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f1e4004ea90 con 0x7f1e4010d280 2024-03-21T10:39:42.039 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:42.041+0000 7f1e37400700 1 -- 172.21.15.153:0/79305019 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4493 (secure 0 0 0) 0x7f1e2c05a8d0 con 0x7f1e4010d280 2024-03-21T10:39:42.039 INFO:teuthology.orchestra.run.smithi153.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-03-21T10:39:42.039 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-03-21T10:39:42.039 INFO:teuthology.orchestra.run.smithi153.stdout: host smithi174 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi174/config 2024-03-21T10:39:42.039 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/podman: timeout after 895 seconds 2024-03-21T10:39:42.040 INFO:teuthology.orchestra.run.smithi153.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 -e NODE_NAME=smithi174 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/run/ceph:z -v /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpd910r3_i:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 inventory --format=json-pretty --filter-for-batch 2024-03-21T10:39:42.040 INFO:teuthology.orchestra.run.smithi153.stdout:Traceback (most recent call last): 2024-03-21T10:39:42.040 INFO:teuthology.orchestra.run.smithi153.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-03-21T10:39:42.040 INFO:teuthology.orchestra.run.smithi153.stdout: return _run_code(code, main_globals, None, 2024-03-21T10:39:42.040 INFO:teuthology.orchestra.run.smithi153.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-03-21T10:39:42.040 INFO:teuthology.orchestra.run.smithi153.stdout: exec(code, run_globals) 2024-03-21T10:39:42.040 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 5606, in 2024-03-21T10:39:42.040 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 5594, in main 2024-03-21T10:39:42.040 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 406, in _infer_config 2024-03-21T10:39:42.040 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 321, in _infer_fsid 2024-03-21T10:39:42.040 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 434, in _infer_image 2024-03-21T10:39:42.040 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 308, in _validate_fsid 2024-03-21T10:39:42.040 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 3357, in command_ceph_volume 2024-03-21T10:39:42.040 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-03-21T10:39:42.040 INFO:teuthology.orchestra.run.smithi153.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 -e NODE_NAME=smithi174 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/run/ceph:z -v /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpd910r3_i:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 inventory --format=json-pretty --filter-for-batch 2024-03-21T10:39:42.040 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-21T10:39:42.041 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is degraded 2024-03-21T10:39:42.041 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-21T10:39:42.041 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-21T10:39:42.041 INFO:teuthology.orchestra.run.smithi153.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-21T10:39:42.041 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-21T10:39:42.042 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:42.043+0000 7f1e45400700 1 -- 172.21.15.153:0/79305019 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f1e2406e460 msgr2=0x7f1e24070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:39:42.042 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:42.044+0000 7f1e45400700 1 --2- 172.21.15.153:0/79305019 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f1e2406e460 0x7f1e24070920 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f1e30006010 tx=0x7f1e3001a040 comp rx=0 tx=0).stop 2024-03-21T10:39:42.042 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:42.044+0000 7f1e45400700 1 -- 172.21.15.153:0/79305019 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1e4010d280 msgr2=0x7f1e4019d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:39:42.042 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:42.044+0000 7f1e45400700 1 --2- 172.21.15.153:0/79305019 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1e4010d280 0x7f1e4019d350 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f1e2c00b4b0 tx=0x7f1e2c00b870 comp rx=0 tx=0).stop 2024-03-21T10:39:42.042 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:42.044+0000 7f1e45400700 1 -- 172.21.15.153:0/79305019 shutdown_connections 2024-03-21T10:39:42.042 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:42.044+0000 7f1e45400700 1 --2- 172.21.15.153:0/79305019 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1e401071d0 0x7f1e4019ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:39:42.042 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:42.044+0000 7f1e45400700 1 --2- 172.21.15.153:0/79305019 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f1e2406e460 0x7f1e24070920 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:39:42.042 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:42.044+0000 7f1e45400700 1 --2- 172.21.15.153:0/79305019 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1e4010d280 0x7f1e4019d350 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:39:42.042 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:42.044+0000 7f1e45400700 1 -- 172.21.15.153:0/79305019 >> 172.21.15.153:0/79305019 conn(0x7f1e400769e0 msgr2=0x7f1e400785c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:39:42.042 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:42.044+0000 7f1e45400700 1 -- 172.21.15.153:0/79305019 shutdown_connections 2024-03-21T10:39:42.042 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:39:42.044+0000 7f1e45400700 1 -- 172.21.15.153:0/79305019 wait complete. 2024-03-21T10:39:42.302 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:39:42 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/79305019' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:39:42.447 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:39:42 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/79305019' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:39:43.447 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:39:43 smithi174 ceph-mon[88004]: from='client.35510 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:39:43.448 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:39:43 smithi174 ceph-mon[88004]: pgmap v837: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:39:43.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:39:43 smithi153 ceph-mon[113733]: from='client.35510 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:39:43.553 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:39:43 smithi153 ceph-mon[113733]: pgmap v837: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:39:45.553 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:39:45 smithi153 ceph-mon[113733]: pgmap v838: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:39:45.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:39:45 smithi174 ceph-mon[88004]: pgmap v838: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:39:47.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:39:47 smithi153 ceph-mon[113733]: pgmap v839: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:39:47.553 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:39:47 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:39:47.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:39:47 smithi174 ceph-mon[88004]: pgmap v839: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:39:47.698 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:39:47 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:39:49.553 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:39:49 smithi153 ceph-mon[113733]: pgmap v840: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:39:49.698 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:39:49 smithi174 ceph-mon[88004]: pgmap v840: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:39:51.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:39:51 smithi153 ceph-mon[113733]: pgmap v841: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:39:51.698 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:39:51 smithi174 ceph-mon[88004]: pgmap v841: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:39:53.553 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:39:53 smithi153 ceph-mon[113733]: pgmap v842: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:39:53.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:39:53 smithi174 ceph-mon[88004]: pgmap v842: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:39:55.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:39:55 smithi153 ceph-mon[113733]: pgmap v843: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:39:55.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:39:55 smithi174 ceph-mon[88004]: pgmap v843: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:39:57.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:39:57 smithi153 ceph-mon[113733]: pgmap v844: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:39:57.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:39:57 smithi174 ceph-mon[88004]: pgmap v844: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:39:59.553 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:39:59 smithi153 ceph-mon[113733]: pgmap v845: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:39:59.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:39:59 smithi174 ceph-mon[88004]: pgmap v845: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:40:00.259 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:40:00 smithi153 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-mon-smithi153[113729]: 2024-03-21T10:39:59.998+0000 7f2169e00700 -1 log_channel(cluster) log [ERR] : Health detail: HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-03-21T10:40:00.259 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:40:00 smithi153 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-mon-smithi153[113729]: 2024-03-21T10:39:59.999+0000 7f2169e00700 -1 log_channel(cluster) log [ERR] : [WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-03-21T10:40:00.259 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:40:00 smithi153 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-mon-smithi153[113729]: 2024-03-21T10:39:59.999+0000 7f2169e00700 -1 log_channel(cluster) log [ERR] : host smithi174 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi174/config 2024-03-21T10:40:00.259 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:40:00 smithi153 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-mon-smithi153[113729]: 2024-03-21T10:39:59.999+0000 7f2169e00700 -1 log_channel(cluster) log [ERR] : /usr/bin/podman: timeout after 895 seconds 2024-03-21T10:40:00.260 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:40:00 smithi153 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-mon-smithi153[113729]: 2024-03-21T10:39:59.999+0000 7f2169e00700 -1 log_channel(cluster) log [ERR] : Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 -e NODE_NAME=smithi174 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/run/ceph:z -v /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpd910r3_i:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 inventory --format=json-pretty --filter-for-batch 2024-03-21T10:40:00.260 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:40:00 smithi153 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-mon-smithi153[113729]: 2024-03-21T10:39:59.999+0000 7f2169e00700 -1 log_channel(cluster) log [ERR] : Traceback (most recent call last): 2024-03-21T10:40:00.260 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:40:00 smithi153 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-mon-smithi153[113729]: 2024-03-21T10:39:59.999+0000 7f2169e00700 -1 log_channel(cluster) log [ERR] : File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-03-21T10:40:00.260 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:40:00 smithi153 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-mon-smithi153[113729]: 2024-03-21T10:39:59.999+0000 7f2169e00700 -1 log_channel(cluster) log [ERR] : return _run_code(code, main_globals, None, 2024-03-21T10:40:00.260 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:40:00 smithi153 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-mon-smithi153[113729]: 2024-03-21T10:39:59.999+0000 7f2169e00700 -1 log_channel(cluster) log [ERR] : File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-03-21T10:40:00.260 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:40:00 smithi153 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-mon-smithi153[113729]: 2024-03-21T10:39:59.999+0000 7f2169e00700 -1 log_channel(cluster) log [ERR] : exec(code, run_globals) 2024-03-21T10:40:00.260 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:40:00 smithi153 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-mon-smithi153[113729]: 2024-03-21T10:39:59.999+0000 7f2169e00700 -1 log_channel(cluster) log [ERR] : File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 5606, in 2024-03-21T10:40:00.260 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:40:00 smithi153 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-mon-smithi153[113729]: 2024-03-21T10:39:59.999+0000 7f2169e00700 -1 log_channel(cluster) log [ERR] : File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 5594, in main 2024-03-21T10:40:00.260 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:40:00 smithi153 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-mon-smithi153[113729]: 2024-03-21T10:39:59.999+0000 7f2169e00700 -1 log_channel(cluster) log [ERR] : File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 406, in _infer_config 2024-03-21T10:40:00.261 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:40:00 smithi153 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-mon-smithi153[113729]: 2024-03-21T10:39:59.999+0000 7f2169e00700 -1 log_channel(cluster) log [ERR] : File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 321, in _infer_fsid 2024-03-21T10:40:00.261 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:40:00 smithi153 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-mon-smithi153[113729]: 2024-03-21T10:39:59.999+0000 7f2169e00700 -1 log_channel(cluster) log [ERR] : File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 434, in _infer_image 2024-03-21T10:40:00.261 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:40:00 smithi153 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-mon-smithi153[113729]: 2024-03-21T10:39:59.999+0000 7f2169e00700 -1 log_channel(cluster) log [ERR] : File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 308, in _validate_fsid 2024-03-21T10:40:00.261 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:40:00 smithi153 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-mon-smithi153[113729]: 2024-03-21T10:39:59.999+0000 7f2169e00700 -1 log_channel(cluster) log [ERR] : File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 3357, in command_ceph_volume 2024-03-21T10:40:00.261 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:40:00 smithi153 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-mon-smithi153[113729]: 2024-03-21T10:39:59.999+0000 7f2169e00700 -1 log_channel(cluster) log [ERR] : File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-03-21T10:40:00.261 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:40:00 smithi153 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-mon-smithi153[113729]: 2024-03-21T10:39:59.999+0000 7f2169e00700 -1 log_channel(cluster) log [ERR] : RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 -e NODE_NAME=smithi174 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/run/ceph:z -v /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpd910r3_i:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 inventory --format=json-pretty --filter-for-batch 2024-03-21T10:40:00.261 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:40:00 smithi153 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-mon-smithi153[113729]: 2024-03-21T10:39:59.999+0000 7f2169e00700 -1 log_channel(cluster) log [ERR] : [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-21T10:40:00.261 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:40:00 smithi153 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-mon-smithi153[113729]: 2024-03-21T10:39:59.999+0000 7f2169e00700 -1 log_channel(cluster) log [ERR] : fs cephfs is degraded 2024-03-21T10:40:00.261 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:40:00 smithi153 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-mon-smithi153[113729]: 2024-03-21T10:39:59.999+0000 7f2169e00700 -1 log_channel(cluster) log [ERR] : [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-21T10:40:00.262 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:40:00 smithi153 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-mon-smithi153[113729]: 2024-03-21T10:39:59.999+0000 7f2169e00700 -1 log_channel(cluster) log [ERR] : fs cephfs has deprecated feature inline_data enabled. 2024-03-21T10:40:00.262 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:40:00 smithi153 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-mon-smithi153[113729]: 2024-03-21T10:39:59.999+0000 7f2169e00700 -1 log_channel(cluster) log [ERR] : [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-21T10:40:00.262 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:40:00 smithi153 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-mon-smithi153[113729]: 2024-03-21T10:39:59.999+0000 7f2169e00700 -1 log_channel(cluster) log [ERR] : fs cephfs is offline because no MDS is active for it. 2024-03-21T10:40:00.262 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:40:00 smithi153 ceph-mon[113733]: Health detail: HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-03-21T10:40:00.262 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:40:00 smithi153 ceph-mon[113733]: [WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-03-21T10:40:00.262 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:40:00 smithi153 ceph-mon[113733]: host smithi174 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi174/config 2024-03-21T10:40:00.262 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:40:00 smithi153 ceph-mon[113733]: /usr/bin/podman: timeout after 895 seconds 2024-03-21T10:40:00.262 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:40:00 smithi153 ceph-mon[113733]: Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 -e NODE_NAME=smithi174 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/run/ceph:z -v /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpd910r3_i:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 inventory --format=json-pretty --filter-for-batch 2024-03-21T10:40:00.263 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:40:00 smithi153 ceph-mon[113733]: Traceback (most recent call last): 2024-03-21T10:40:00.263 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:40:00 smithi153 ceph-mon[113733]: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-03-21T10:40:00.263 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:40:00 smithi153 ceph-mon[113733]: return _run_code(code, main_globals, None, 2024-03-21T10:40:00.263 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:40:00 smithi153 ceph-mon[113733]: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-03-21T10:40:00.263 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:40:00 smithi153 ceph-mon[113733]: exec(code, run_globals) 2024-03-21T10:40:00.263 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:40:00 smithi153 ceph-mon[113733]: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 5606, in 2024-03-21T10:40:00.263 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:40:00 smithi153 ceph-mon[113733]: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 5594, in main 2024-03-21T10:40:00.263 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:40:00 smithi153 ceph-mon[113733]: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 406, in _infer_config 2024-03-21T10:40:00.263 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:40:00 smithi153 ceph-mon[113733]: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 321, in _infer_fsid 2024-03-21T10:40:00.263 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:40:00 smithi153 ceph-mon[113733]: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 434, in _infer_image 2024-03-21T10:40:00.263 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:40:00 smithi153 ceph-mon[113733]: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 308, in _validate_fsid 2024-03-21T10:40:00.263 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:40:00 smithi153 ceph-mon[113733]: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 3357, in command_ceph_volume 2024-03-21T10:40:00.263 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:40:00 smithi153 ceph-mon[113733]: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-03-21T10:40:00.263 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:40:00 smithi153 ceph-mon[113733]: RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 -e NODE_NAME=smithi174 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/run/ceph:z -v /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpd910r3_i:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 inventory --format=json-pretty --filter-for-batch 2024-03-21T10:40:00.263 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:40:00 smithi153 ceph-mon[113733]: [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-21T10:40:00.264 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:40:00 smithi153 ceph-mon[113733]: fs cephfs is degraded 2024-03-21T10:40:00.264 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:40:00 smithi153 ceph-mon[113733]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-21T10:40:00.264 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:40:00 smithi153 ceph-mon[113733]: fs cephfs has deprecated feature inline_data enabled. 2024-03-21T10:40:00.264 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:40:00 smithi153 ceph-mon[113733]: [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-21T10:40:00.264 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:40:00 smithi153 ceph-mon[113733]: fs cephfs is offline because no MDS is active for it. 2024-03-21T10:40:00.698 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:40:00 smithi174 ceph-mon[88004]: Health detail: HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-03-21T10:40:00.699 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:40:00 smithi174 ceph-mon[88004]: [WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-03-21T10:40:00.699 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:40:00 smithi174 ceph-mon[88004]: host smithi174 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi174/config 2024-03-21T10:40:00.699 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:40:00 smithi174 ceph-mon[88004]: /usr/bin/podman: timeout after 895 seconds 2024-03-21T10:40:00.699 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:40:00 smithi174 ceph-mon[88004]: Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 -e NODE_NAME=smithi174 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/run/ceph:z -v /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpd910r3_i:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 inventory --format=json-pretty --filter-for-batch 2024-03-21T10:40:00.699 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:40:00 smithi174 ceph-mon[88004]: Traceback (most recent call last): 2024-03-21T10:40:00.699 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:40:00 smithi174 ceph-mon[88004]: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-03-21T10:40:00.699 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:40:00 smithi174 ceph-mon[88004]: return _run_code(code, main_globals, None, 2024-03-21T10:40:00.699 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:40:00 smithi174 ceph-mon[88004]: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-03-21T10:40:00.699 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:40:00 smithi174 ceph-mon[88004]: exec(code, run_globals) 2024-03-21T10:40:00.699 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:40:00 smithi174 ceph-mon[88004]: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 5606, in 2024-03-21T10:40:00.699 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:40:00 smithi174 ceph-mon[88004]: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 5594, in main 2024-03-21T10:40:00.699 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:40:00 smithi174 ceph-mon[88004]: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 406, in _infer_config 2024-03-21T10:40:00.700 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:40:00 smithi174 ceph-mon[88004]: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 321, in _infer_fsid 2024-03-21T10:40:00.700 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:40:00 smithi174 ceph-mon[88004]: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 434, in _infer_image 2024-03-21T10:40:00.700 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:40:00 smithi174 ceph-mon[88004]: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 308, in _validate_fsid 2024-03-21T10:40:00.700 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:40:00 smithi174 ceph-mon[88004]: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 3357, in command_ceph_volume 2024-03-21T10:40:00.700 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:40:00 smithi174 ceph-mon[88004]: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-03-21T10:40:00.700 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:40:00 smithi174 ceph-mon[88004]: RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 -e NODE_NAME=smithi174 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/run/ceph:z -v /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpd910r3_i:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 inventory --format=json-pretty --filter-for-batch 2024-03-21T10:40:00.700 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:40:00 smithi174 ceph-mon[88004]: [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-21T10:40:00.700 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:40:00 smithi174 ceph-mon[88004]: fs cephfs is degraded 2024-03-21T10:40:00.700 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:40:00 smithi174 ceph-mon[88004]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-21T10:40:00.700 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:40:00 smithi174 ceph-mon[88004]: fs cephfs has deprecated feature inline_data enabled. 2024-03-21T10:40:00.700 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:40:00 smithi174 ceph-mon[88004]: [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-21T10:40:00.700 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:40:00 smithi174 ceph-mon[88004]: fs cephfs is offline because no MDS is active for it. 2024-03-21T10:40:01.553 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:40:01 smithi153 ceph-mon[113733]: pgmap v846: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:40:01.553 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:40:01 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:40:01.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:40:01 smithi174 ceph-mon[88004]: pgmap v846: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:40:01.698 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:40:01 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:40:03.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:40:03 smithi153 ceph-mon[113733]: pgmap v847: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:40:03.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:40:03 smithi174 ceph-mon[88004]: pgmap v847: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:40:05.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:40:05 smithi153 ceph-mon[113733]: pgmap v848: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:40:05.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:40:05 smithi174 ceph-mon[88004]: pgmap v848: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:40:07.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:40:07 smithi153 ceph-mon[113733]: pgmap v849: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:40:07.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:40:07 smithi174 ceph-mon[88004]: pgmap v849: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:40:09.553 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:40:09 smithi153 ceph-mon[113733]: pgmap v850: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:40:09.698 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:40:09 smithi174 ceph-mon[88004]: pgmap v850: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:40:11.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:40:11 smithi153 ceph-mon[113733]: pgmap v851: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:40:11.698 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:40:11 smithi174 ceph-mon[88004]: pgmap v851: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:40:12.211 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:12.213+0000 7fe3b8000700 1 -- 172.21.15.153:0/1966194048 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe3b0103000 msgr2=0x7fe3b01033e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:40:12.211 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:12.213+0000 7fe3b8000700 1 --2- 172.21.15.153:0/1966194048 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe3b0103000 0x7fe3b01033e0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fe3a0009910 tx=0x7fe3a0009c20 comp rx=0 tx=0).stop 2024-03-21T10:40:12.211 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:12.213+0000 7fe3b8000700 1 -- 172.21.15.153:0/1966194048 shutdown_connections 2024-03-21T10:40:12.212 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:12.213+0000 7fe3b8000700 1 --2- 172.21.15.153:0/1966194048 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe3b01039b0 0x7fe3b0107a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:40:12.212 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:12.213+0000 7fe3b8000700 1 --2- 172.21.15.153:0/1966194048 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe3b0103000 0x7fe3b01033e0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:40:12.212 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:12.213+0000 7fe3b8000700 1 -- 172.21.15.153:0/1966194048 >> 172.21.15.153:0/1966194048 conn(0x7fe3b00fe8d0 msgr2=0x7fe3b0100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:40:12.212 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:12.213+0000 7fe3b8000700 1 -- 172.21.15.153:0/1966194048 shutdown_connections 2024-03-21T10:40:12.212 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:12.213+0000 7fe3b8000700 1 -- 172.21.15.153:0/1966194048 wait complete. 2024-03-21T10:40:12.213 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:12.214+0000 7fe3b8000700 1 Processor -- start 2024-03-21T10:40:12.213 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:12.214+0000 7fe3b8000700 1 -- start start 2024-03-21T10:40:12.213 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:12.215+0000 7fe3b8000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe3b0103000 0x7fe3b019e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:40:12.213 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:12.215+0000 7fe3b8000700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe3b01039b0 0x7fe3b019ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:40:12.214 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:12.215+0000 7fe3b8000700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b019f3c0 con 0x7fe3b0103000 2024-03-21T10:40:12.214 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:12.215+0000 7fe3b8000700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b0198870 con 0x7fe3b01039b0 2024-03-21T10:40:12.214 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:12.215+0000 7fe3b5600700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe3b0103000 0x7fe3b019e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:40:12.214 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:12.215+0000 7fe3b5600700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe3b0103000 0x7fe3b019e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:60546/0 (socket says 172.21.15.153:60546) 2024-03-21T10:40:12.214 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:12.215+0000 7fe3b5600700 1 -- 172.21.15.153:0/1820549401 learned_addr learned my addr 172.21.15.153:0/1820549401 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:40:12.214 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:12.215+0000 7fe3b4c00700 1 --2- 172.21.15.153:0/1820549401 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe3b01039b0 0x7fe3b019ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:40:12.214 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:12.216+0000 7fe3b5600700 1 -- 172.21.15.153:0/1820549401 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe3b01039b0 msgr2=0x7fe3b019ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:40:12.214 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:12.216+0000 7fe3b5600700 1 --2- 172.21.15.153:0/1820549401 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe3b01039b0 0x7fe3b019ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:40:12.214 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:12.216+0000 7fe3b5600700 1 -- 172.21.15.153:0/1820549401 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3a00095f0 con 0x7fe3b0103000 2024-03-21T10:40:12.214 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:12.216+0000 7fe3b4c00700 1 --2- 172.21.15.153:0/1820549401 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe3b01039b0 0x7fe3b019ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:40:12.214 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:12.216+0000 7fe3b5600700 1 --2- 172.21.15.153:0/1820549401 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe3b0103000 0x7fe3b019e7f0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fe3a0006010 tx=0x7fe3a000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:40:12.215 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:12.216+0000 7fe3ae000700 1 -- 172.21.15.153:0/1820549401 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe3a001d070 con 0x7fe3b0103000 2024-03-21T10:40:12.215 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:12.216+0000 7fe3ae000700 1 -- 172.21.15.153:0/1820549401 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe3a000f460 con 0x7fe3b0103000 2024-03-21T10:40:12.215 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:12.216+0000 7fe3b8000700 1 -- 172.21.15.153:0/1820549401 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3b0198af0 con 0x7fe3b0103000 2024-03-21T10:40:12.215 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:12.216+0000 7fe3ae000700 1 -- 172.21.15.153:0/1820549401 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe3a0017630 con 0x7fe3b0103000 2024-03-21T10:40:12.215 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:12.216+0000 7fe3b8000700 1 -- 172.21.15.153:0/1820549401 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3b0198fe0 con 0x7fe3b0103000 2024-03-21T10:40:12.216 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:12.217+0000 7fe3b8000700 1 -- 172.21.15.153:0/1820549401 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3b010b3a0 con 0x7fe3b0103000 2024-03-21T10:40:12.216 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:12.218+0000 7fe3ae000700 1 -- 172.21.15.153:0/1820549401 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fe3a000f5d0 con 0x7fe3b0103000 2024-03-21T10:40:12.217 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:12.219+0000 7fe3ae000700 1 --2- 172.21.15.153:0/1820549401 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe39406e460 0x7fe394070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:40:12.217 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:12.219+0000 7fe3ae000700 1 -- 172.21.15.153:0/1820549401 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fe3a008fa50 con 0x7fe3b0103000 2024-03-21T10:40:12.217 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:12.220+0000 7fe3b4c00700 1 --2- 172.21.15.153:0/1820549401 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe39406e460 0x7fe394070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:40:12.218 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:12.220+0000 7fe3b4c00700 1 --2- 172.21.15.153:0/1820549401 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe39406e460 0x7fe394070920 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fe39c005fd0 tx=0x7fe39c005ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:40:12.222 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:12.224+0000 7fe3ae000700 1 -- 172.21.15.153:0/1820549401 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe3a005c5a0 con 0x7fe3b0103000 2024-03-21T10:40:12.462 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:12.464+0000 7fe3b8000700 1 -- 172.21.15.153:0/1820549401 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe3b0199200 con 0x7fe39406e460 2024-03-21T10:40:12.465 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:12.467+0000 7fe3ae000700 1 -- 172.21.15.153:0/1820549401 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fe3b0199200 con 0x7fe39406e460 2024-03-21T10:40:12.467 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:12.470+0000 7fe3b8000700 1 -- 172.21.15.153:0/1820549401 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe39406e460 msgr2=0x7fe394070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:40:12.467 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:12.470+0000 7fe3b8000700 1 --2- 172.21.15.153:0/1820549401 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe39406e460 0x7fe394070920 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fe39c005fd0 tx=0x7fe39c005ee0 comp rx=0 tx=0).stop 2024-03-21T10:40:12.467 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:12.470+0000 7fe3b8000700 1 -- 172.21.15.153:0/1820549401 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe3b0103000 msgr2=0x7fe3b019e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:40:12.468 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:12.470+0000 7fe3b8000700 1 --2- 172.21.15.153:0/1820549401 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe3b0103000 0x7fe3b019e7f0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fe3a0006010 tx=0x7fe3a000be20 comp rx=0 tx=0).stop 2024-03-21T10:40:12.468 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:12.470+0000 7fe3b8000700 1 -- 172.21.15.153:0/1820549401 shutdown_connections 2024-03-21T10:40:12.468 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:12.470+0000 7fe3b8000700 1 --2- 172.21.15.153:0/1820549401 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe39406e460 0x7fe394070920 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:40:12.468 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:12.470+0000 7fe3b8000700 1 --2- 172.21.15.153:0/1820549401 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe3b0103000 0x7fe3b019e7f0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:40:12.468 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:12.470+0000 7fe3b8000700 1 --2- 172.21.15.153:0/1820549401 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe3b01039b0 0x7fe3b019ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:40:12.468 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:12.470+0000 7fe3b8000700 1 -- 172.21.15.153:0/1820549401 >> 172.21.15.153:0/1820549401 conn(0x7fe3b00fe8d0 msgr2=0x7fe3b00ffee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:40:12.468 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:12.470+0000 7fe3b8000700 1 -- 172.21.15.153:0/1820549401 shutdown_connections 2024-03-21T10:40:12.468 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:12.470+0000 7fe3b8000700 1 -- 172.21.15.153:0/1820549401 wait complete. 2024-03-21T10:40:12.483 INFO:teuthology.orchestra.run.smithi153.stdout:true 2024-03-21T10:40:12.626 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:12.627+0000 7fae50400700 1 -- 172.21.15.153:0/3939137598 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fae48103000 msgr2=0x7fae481033e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:40:12.626 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:12.627+0000 7fae50400700 1 --2- 172.21.15.153:0/3939137598 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fae48103000 0x7fae481033e0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fae3c009910 tx=0x7fae3c009c20 comp rx=0 tx=0).stop 2024-03-21T10:40:12.626 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:12.628+0000 7fae50400700 1 -- 172.21.15.153:0/3939137598 shutdown_connections 2024-03-21T10:40:12.626 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:12.628+0000 7fae50400700 1 --2- 172.21.15.153:0/3939137598 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fae481039b0 0x7fae48107a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:40:12.626 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:12.628+0000 7fae50400700 1 --2- 172.21.15.153:0/3939137598 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fae48103000 0x7fae481033e0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:40:12.626 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:12.628+0000 7fae50400700 1 -- 172.21.15.153:0/3939137598 >> 172.21.15.153:0/3939137598 conn(0x7fae480fe8d0 msgr2=0x7fae48100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:40:12.626 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:12.628+0000 7fae50400700 1 -- 172.21.15.153:0/3939137598 shutdown_connections 2024-03-21T10:40:12.626 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:12.628+0000 7fae50400700 1 -- 172.21.15.153:0/3939137598 wait complete. 2024-03-21T10:40:12.627 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:12.629+0000 7fae50400700 1 Processor -- start 2024-03-21T10:40:12.627 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:12.629+0000 7fae50400700 1 -- start start 2024-03-21T10:40:12.628 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:12.629+0000 7fae50400700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fae48103000 0x7fae4819e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:40:12.628 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:12.629+0000 7fae50400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fae481039b0 0x7fae4819ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:40:12.628 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:12.629+0000 7fae50400700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae4819f3c0 con 0x7fae481039b0 2024-03-21T10:40:12.628 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:12.629+0000 7fae50400700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae48198870 con 0x7fae48103000 2024-03-21T10:40:12.628 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:12.630+0000 7fae4d000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fae481039b0 0x7fae4819ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:40:12.628 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:12.630+0000 7fae4d000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fae481039b0 0x7fae4819ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:60556/0 (socket says 172.21.15.153:60556) 2024-03-21T10:40:12.628 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:12.630+0000 7fae4d000700 1 -- 172.21.15.153:0/1375556288 learned_addr learned my addr 172.21.15.153:0/1375556288 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:40:12.628 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:12.630+0000 7fae4da00700 1 --2- 172.21.15.153:0/1375556288 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fae48103000 0x7fae4819e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:40:12.629 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:12.630+0000 7fae4d000700 1 -- 172.21.15.153:0/1375556288 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fae48103000 msgr2=0x7fae4819e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:40:12.629 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:12.630+0000 7fae4d000700 1 --2- 172.21.15.153:0/1375556288 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fae48103000 0x7fae4819e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:40:12.629 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:12.630+0000 7fae4d000700 1 -- 172.21.15.153:0/1375556288 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae3c0095f0 con 0x7fae481039b0 2024-03-21T10:40:12.629 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:12.631+0000 7fae4d000700 1 --2- 172.21.15.153:0/1375556288 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fae481039b0 0x7fae4819ed30 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fae3400d900 tx=0x7fae3400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:40:12.629 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:12.631+0000 7fae46000700 1 -- 172.21.15.153:0/1375556288 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fae34009900 con 0x7fae481039b0 2024-03-21T10:40:12.629 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:12.631+0000 7fae46000700 1 -- 172.21.15.153:0/1375556288 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fae34010460 con 0x7fae481039b0 2024-03-21T10:40:12.630 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:12.631+0000 7fae46000700 1 -- 172.21.15.153:0/1375556288 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fae3400f5f0 con 0x7fae481039b0 2024-03-21T10:40:12.630 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:12.631+0000 7fae50400700 1 -- 172.21.15.153:0/1375556288 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae48198b50 con 0x7fae481039b0 2024-03-21T10:40:12.630 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:12.631+0000 7fae50400700 1 -- 172.21.15.153:0/1375556288 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae481990a0 con 0x7fae481039b0 2024-03-21T10:40:12.630 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:12.632+0000 7fae50400700 1 -- 172.21.15.153:0/1375556288 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae4810b3a0 con 0x7fae481039b0 2024-03-21T10:40:12.631 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:12.633+0000 7fae46000700 1 -- 172.21.15.153:0/1375556288 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fae340105d0 con 0x7fae481039b0 2024-03-21T10:40:12.632 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:12.634+0000 7fae46000700 1 --2- 172.21.15.153:0/1375556288 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fae3006e460 0x7fae30070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:40:12.632 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:12.634+0000 7fae46000700 1 -- 172.21.15.153:0/1375556288 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fae3408e0d0 con 0x7fae481039b0 2024-03-21T10:40:12.632 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:12.635+0000 7fae4da00700 1 --2- 172.21.15.153:0/1375556288 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fae3006e460 0x7fae30070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:40:12.633 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:12.635+0000 7fae4da00700 1 --2- 172.21.15.153:0/1375556288 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fae3006e460 0x7fae30070920 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fae3c006010 tx=0x7fae3c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:40:12.636 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:12.639+0000 7fae46000700 1 -- 172.21.15.153:0/1375556288 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fae3405aca0 con 0x7fae481039b0 2024-03-21T10:40:12.879 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:12.881+0000 7fae50400700 1 -- 172.21.15.153:0/1375556288 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fae481993d0 con 0x7fae3006e460 2024-03-21T10:40:12.881 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:12.883+0000 7fae46000700 1 -- 172.21.15.153:0/1375556288 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fae481993d0 con 0x7fae3006e460 2024-03-21T10:40:12.884 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:12.886+0000 7fae50400700 1 -- 172.21.15.153:0/1375556288 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fae3006e460 msgr2=0x7fae30070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:40:12.884 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:12.886+0000 7fae50400700 1 --2- 172.21.15.153:0/1375556288 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fae3006e460 0x7fae30070920 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fae3c006010 tx=0x7fae3c01a040 comp rx=0 tx=0).stop 2024-03-21T10:40:12.884 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:12.886+0000 7fae50400700 1 -- 172.21.15.153:0/1375556288 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fae481039b0 msgr2=0x7fae4819ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:40:12.884 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:12.886+0000 7fae50400700 1 --2- 172.21.15.153:0/1375556288 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fae481039b0 0x7fae4819ed30 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fae3400d900 tx=0x7fae3400dcc0 comp rx=0 tx=0).stop 2024-03-21T10:40:12.884 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:12.886+0000 7fae50400700 1 -- 172.21.15.153:0/1375556288 shutdown_connections 2024-03-21T10:40:12.884 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:12.886+0000 7fae50400700 1 --2- 172.21.15.153:0/1375556288 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fae48103000 0x7fae4819e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:40:12.885 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:12.886+0000 7fae50400700 1 --2- 172.21.15.153:0/1375556288 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fae3006e460 0x7fae30070920 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:40:12.885 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:12.886+0000 7fae50400700 1 --2- 172.21.15.153:0/1375556288 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fae481039b0 0x7fae4819ed30 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:40:12.885 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:12.886+0000 7fae50400700 1 -- 172.21.15.153:0/1375556288 >> 172.21.15.153:0/1375556288 conn(0x7fae480fe8d0 msgr2=0x7fae480ffee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:40:12.885 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:12.887+0000 7fae50400700 1 -- 172.21.15.153:0/1375556288 shutdown_connections 2024-03-21T10:40:12.885 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:12.887+0000 7fae50400700 1 -- 172.21.15.153:0/1375556288 wait complete. 2024-03-21T10:40:13.042 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.044+0000 7f36b3200700 1 -- 172.21.15.153:0/589842680 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f36b4107bd0 msgr2=0x7f36b410bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:40:13.042 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.044+0000 7f36b3200700 1 --2- 172.21.15.153:0/589842680 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f36b4107bd0 0x7f36b410bc10 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f36a0009910 tx=0x7f36a0009c20 comp rx=0 tx=0).stop 2024-03-21T10:40:13.042 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.044+0000 7f36b3200700 1 -- 172.21.15.153:0/589842680 shutdown_connections 2024-03-21T10:40:13.042 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.044+0000 7f36b3200700 1 --2- 172.21.15.153:0/589842680 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f36b4107bd0 0x7f36b410bc10 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:40:13.042 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.044+0000 7f36b3200700 1 --2- 172.21.15.153:0/589842680 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f36b4107220 0x7f36b4107600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:40:13.042 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.044+0000 7f36b3200700 1 -- 172.21.15.153:0/589842680 >> 172.21.15.153:0/589842680 conn(0x7f36b40767e0 msgr2=0x7f36b4078c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:40:13.042 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.044+0000 7f36b3200700 1 -- 172.21.15.153:0/589842680 shutdown_connections 2024-03-21T10:40:13.042 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.044+0000 7f36b3200700 1 -- 172.21.15.153:0/589842680 wait complete. 2024-03-21T10:40:13.043 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.045+0000 7f36b3200700 1 Processor -- start 2024-03-21T10:40:13.043 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.045+0000 7f36b3200700 1 -- start start 2024-03-21T10:40:13.044 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.046+0000 7f36b3200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f36b4107220 0x7f36b41a29f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:40:13.044 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.046+0000 7f36b3200700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f36b4107bd0 0x7f36b41a2f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:40:13.044 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.046+0000 7f36b3200700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36b41a35c0 con 0x7f36b4107220 2024-03-21T10:40:13.044 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.046+0000 7f36b3200700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36b419ca70 con 0x7f36b4107bd0 2024-03-21T10:40:13.044 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.046+0000 7f36b1e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f36b4107220 0x7f36b41a29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:40:13.044 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.046+0000 7f36b1e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f36b4107220 0x7f36b41a29f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:50598/0 (socket says 172.21.15.153:50598) 2024-03-21T10:40:13.044 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.046+0000 7f36b1e00700 1 -- 172.21.15.153:0/3098631423 learned_addr learned my addr 172.21.15.153:0/3098631423 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:40:13.045 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.047+0000 7f36b1400700 1 --2- 172.21.15.153:0/3098631423 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f36b4107bd0 0x7f36b41a2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:40:13.045 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.047+0000 7f36b1e00700 1 -- 172.21.15.153:0/3098631423 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f36b4107bd0 msgr2=0x7f36b41a2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:40:13.045 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.047+0000 7f36b1e00700 1 --2- 172.21.15.153:0/3098631423 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f36b4107bd0 0x7f36b41a2f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:40:13.045 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.047+0000 7f36b1e00700 1 -- 172.21.15.153:0/3098631423 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36a00095f0 con 0x7f36b4107220 2024-03-21T10:40:13.045 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.047+0000 7f36b1e00700 1 --2- 172.21.15.153:0/3098631423 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f36b4107220 0x7f36b41a29f0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f36a400d8d0 tx=0x7f36a400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:40:13.046 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.047+0000 7f36aaa00700 1 -- 172.21.15.153:0/3098631423 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f36a4009940 con 0x7f36b4107220 2024-03-21T10:40:13.046 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.047+0000 7f36aaa00700 1 -- 172.21.15.153:0/3098631423 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f36a4010460 con 0x7f36b4107220 2024-03-21T10:40:13.046 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.047+0000 7f36b3200700 1 -- 172.21.15.153:0/3098631423 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36b419cd50 con 0x7f36b4107220 2024-03-21T10:40:13.046 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.048+0000 7f36aaa00700 1 -- 172.21.15.153:0/3098631423 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f36a400f5f0 con 0x7f36b4107220 2024-03-21T10:40:13.046 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.048+0000 7f36b3200700 1 -- 172.21.15.153:0/3098631423 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36b419d2a0 con 0x7f36b4107220 2024-03-21T10:40:13.046 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.049+0000 7f36b3200700 1 -- 172.21.15.153:0/3098631423 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36b410f610 con 0x7f36b4107220 2024-03-21T10:40:13.047 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.050+0000 7f36aaa00700 1 -- 172.21.15.153:0/3098631423 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f36a40105d0 con 0x7f36b4107220 2024-03-21T10:40:13.048 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.051+0000 7f36aaa00700 1 --2- 172.21.15.153:0/3098631423 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f369806e460 0x7f3698070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:40:13.049 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.051+0000 7f36aaa00700 1 -- 172.21.15.153:0/3098631423 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f36a408e250 con 0x7f36b4107220 2024-03-21T10:40:13.049 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.051+0000 7f36b1400700 1 --2- 172.21.15.153:0/3098631423 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f369806e460 0x7f3698070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:40:13.049 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.052+0000 7f36b1400700 1 --2- 172.21.15.153:0/3098631423 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f369806e460 0x7f3698070920 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f36b4077000 tx=0x7f36a001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:40:13.053 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.056+0000 7f36aaa00700 1 -- 172.21.15.153:0/3098631423 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f36a405ae20 con 0x7f36b4107220 2024-03-21T10:40:13.283 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.285+0000 7f36b3200700 1 -- 172.21.15.153:0/3098631423 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f36b419d580 con 0x7f369806e460 2024-03-21T10:40:13.306 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.308+0000 7f36aaa00700 1 -- 172.21.15.153:0/3098631423 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f36b419d580 con 0x7f369806e460 2024-03-21T10:40:13.307 INFO:teuthology.orchestra.run.smithi153.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-21T10:40:13.307 INFO:teuthology.orchestra.run.smithi153.stdout:alertmanager.smithi153 smithi153 *:9093,9094 running (29m) 4m ago 42m 15.6M - 0.25.0 c8568f914cd2 9ed3f2200b5f 2024-03-21T10:40:13.307 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi153 smithi153 running (43m) 4m ago 43m 10.4M - 18.2.1 5be31c24972a 5b61994396a6 2024-03-21T10:40:13.307 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi174 smithi174 running (41m) 4m ago 41m 10.1M - 18.2.1 5be31c24972a 4f7a5fc3ac28 2024-03-21T10:40:13.308 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi153 smithi153 running (25m) 4m ago 43m 6656k - 19.0.0-2231-g2edfee68 237e43c7a65f 32c21104d5b0 2024-03-21T10:40:13.308 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi174 smithi174 running (25m) 4m ago 41m 6660k - 19.0.0-2231-g2edfee68 237e43c7a65f a04e48967ab6 2024-03-21T10:40:13.308 INFO:teuthology.orchestra.run.smithi153.stdout:grafana.smithi153 smithi153 *:3000 running (28m) 4m ago 42m 75.2M - 9.4.12 2bacad6d85d8 ee24026ab786 2024-03-21T10:40:13.308 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.iuyluj smithi153 running (20m) 4m ago 35m 18.7M - 19.0.0-2231-g2edfee68 237e43c7a65f 87aff8a5de70 2024-03-21T10:40:13.308 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.zipckr smithi153 running (20m) 4m ago 35m 18.9M - 19.0.0-2231-g2edfee68 237e43c7a65f ccc262bd5dd3 2024-03-21T10:40:13.308 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.nhozag smithi174 running (4m) 4m ago 35m 10.9M - 19.0.0-2231-g2edfee68 237e43c7a65f 9e02aea72c96 2024-03-21T10:40:13.309 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.yyimst smithi174 running (20m) 4m ago 35m 18.5M - 19.0.0-2231-g2edfee68 237e43c7a65f fc1962f9351e 2024-03-21T10:40:13.309 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi153.tluxni smithi153 *:8443,9283,8765 running (32m) 4m ago 44m 512M - 19.0.0-2231-g2edfee68 237e43c7a65f 7ffee0e55382 2024-03-21T10:40:13.309 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi174.qpcwat smithi174 *:8443,9283,8765 running (30m) 4m ago 41m 459M - 19.0.0-2231-g2edfee68 237e43c7a65f 748968f216ab 2024-03-21T10:40:13.309 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi153 smithi153 running (26m) 4m ago 44m 70.8M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f dd8038303dc3 2024-03-21T10:40:13.309 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi174 smithi174 running (26m) 4m ago 41m 51.9M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f 29d78a788f2c 2024-03-21T10:40:13.309 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi153 smithi153 *:9100 running (30m) 4m ago 42m 8140k - 1.5.0 0da6a335fe13 bb835edc0e0b 2024-03-21T10:40:13.309 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi174 smithi174 *:9100 running (30m) 4m ago 41m 8610k - 1.5.0 0da6a335fe13 7c4ecad790fa 2024-03-21T10:40:13.310 INFO:teuthology.orchestra.run.smithi153.stdout:osd.0 smithi153 running (24m) 4m ago 40m 118M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 507900fd4791 2024-03-21T10:40:13.310 INFO:teuthology.orchestra.run.smithi153.stdout:osd.1 smithi153 running (23m) 4m ago 39m 93.0M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 0d646452e9c1 2024-03-21T10:40:13.310 INFO:teuthology.orchestra.run.smithi153.stdout:osd.2 smithi153 running (23m) 4m ago 39m 91.2M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 1326f2afdfcd 2024-03-21T10:40:13.310 INFO:teuthology.orchestra.run.smithi153.stdout:osd.3 smithi174 running (22m) 4m ago 38m 87.5M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 70aafac71ad5 2024-03-21T10:40:13.310 INFO:teuthology.orchestra.run.smithi153.stdout:osd.4 smithi174 running (22m) 4m ago 37m 95.8M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 0d746ce6e79d 2024-03-21T10:40:13.310 INFO:teuthology.orchestra.run.smithi153.stdout:osd.5 smithi174 running (21m) 4m ago 37m 90.3M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 53e5f35b8cac 2024-03-21T10:40:13.311 INFO:teuthology.orchestra.run.smithi153.stdout:prometheus.smithi153 smithi153 *:9095 running (29m) 4m ago 41m 88.1M - 2.43.0 a07b618ecd1d 7d95561e50c2 2024-03-21T10:40:13.312 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.311+0000 7f36b3200700 1 -- 172.21.15.153:0/3098631423 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f369806e460 msgr2=0x7f3698070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:40:13.312 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.311+0000 7f36b3200700 1 --2- 172.21.15.153:0/3098631423 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f369806e460 0x7f3698070920 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f36b4077000 tx=0x7f36a001a040 comp rx=0 tx=0).stop 2024-03-21T10:40:13.312 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.311+0000 7f36b3200700 1 -- 172.21.15.153:0/3098631423 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f36b4107220 msgr2=0x7f36b41a29f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:40:13.312 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.311+0000 7f36b3200700 1 --2- 172.21.15.153:0/3098631423 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f36b4107220 0x7f36b41a29f0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f36a400d8d0 tx=0x7f36a400dc90 comp rx=0 tx=0).stop 2024-03-21T10:40:13.312 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.311+0000 7f36b3200700 1 -- 172.21.15.153:0/3098631423 shutdown_connections 2024-03-21T10:40:13.313 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.311+0000 7f36b3200700 1 --2- 172.21.15.153:0/3098631423 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f369806e460 0x7f3698070920 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:40:13.313 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.311+0000 7f36b3200700 1 --2- 172.21.15.153:0/3098631423 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f36b4107220 0x7f36b41a29f0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:40:13.313 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.311+0000 7f36b3200700 1 --2- 172.21.15.153:0/3098631423 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f36b4107bd0 0x7f36b41a2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:40:13.313 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.312+0000 7f36b3200700 1 -- 172.21.15.153:0/3098631423 >> 172.21.15.153:0/3098631423 conn(0x7f36b40767e0 msgr2=0x7f36b4078bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:40:13.313 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.312+0000 7f36b3200700 1 -- 172.21.15.153:0/3098631423 shutdown_connections 2024-03-21T10:40:13.313 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.312+0000 7f36b3200700 1 -- 172.21.15.153:0/3098631423 wait complete. 2024-03-21T10:40:13.466 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.468+0000 7f6451a00700 1 -- 172.21.15.153:0/2638512162 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f644c101c40 msgr2=0x7f644c0ff050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:40:13.467 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.468+0000 7f6451a00700 1 --2- 172.21.15.153:0/2638512162 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f644c101c40 0x7f644c0ff050 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f643c009910 tx=0x7f643c009c20 comp rx=0 tx=0).stop 2024-03-21T10:40:13.467 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.468+0000 7f6451a00700 1 -- 172.21.15.153:0/2638512162 shutdown_connections 2024-03-21T10:40:13.467 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.468+0000 7f6451a00700 1 --2- 172.21.15.153:0/2638512162 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f644c0ff590 0x7f644c0ffa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:40:13.467 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.468+0000 7f6451a00700 1 --2- 172.21.15.153:0/2638512162 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f644c101c40 0x7f644c0ff050 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:40:13.467 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.468+0000 7f6451a00700 1 -- 172.21.15.153:0/2638512162 >> 172.21.15.153:0/2638512162 conn(0x7f644c075e30 msgr2=0x7f644c0fd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:40:13.467 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.469+0000 7f6451a00700 1 -- 172.21.15.153:0/2638512162 shutdown_connections 2024-03-21T10:40:13.467 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.469+0000 7f6451a00700 1 -- 172.21.15.153:0/2638512162 wait complete. 2024-03-21T10:40:13.468 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.470+0000 7f6451a00700 1 Processor -- start 2024-03-21T10:40:13.468 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.470+0000 7f6451a00700 1 -- start start 2024-03-21T10:40:13.468 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.470+0000 7f6451a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f644c0ff590 0x7f644c19ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:40:13.468 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.470+0000 7f6451a00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f644c101c40 0x7f644c19ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:40:13.468 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.470+0000 7f6451a00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f644c198a40 con 0x7f644c0ff590 2024-03-21T10:40:13.469 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.470+0000 7f6451a00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f644c198bb0 con 0x7f644c101c40 2024-03-21T10:40:13.469 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.471+0000 7f644aa00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f644c0ff590 0x7f644c19ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:40:13.469 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.471+0000 7f644aa00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f644c0ff590 0x7f644c19ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:50608/0 (socket says 172.21.15.153:50608) 2024-03-21T10:40:13.469 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.471+0000 7f644aa00700 1 -- 172.21.15.153:0/3927421578 learned_addr learned my addr 172.21.15.153:0/3927421578 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:40:13.470 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.471+0000 7f6443e00700 1 --2- 172.21.15.153:0/3927421578 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f644c101c40 0x7f644c19ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:40:13.470 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.471+0000 7f644aa00700 1 -- 172.21.15.153:0/3927421578 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f644c101c40 msgr2=0x7f644c19ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:40:13.470 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.471+0000 7f644aa00700 1 --2- 172.21.15.153:0/3927421578 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f644c101c40 0x7f644c19ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:40:13.470 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.471+0000 7f644aa00700 1 -- 172.21.15.153:0/3927421578 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f643c0095f0 con 0x7f644c0ff590 2024-03-21T10:40:13.470 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.471+0000 7f6443e00700 1 --2- 172.21.15.153:0/3927421578 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f644c101c40 0x7f644c19ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T10:40:13.470 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.472+0000 7f644aa00700 1 --2- 172.21.15.153:0/3927421578 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f644c0ff590 0x7f644c19ea50 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f643c0092e0 tx=0x7f643c00f460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:40:13.470 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.472+0000 7f6443400700 1 -- 172.21.15.153:0/3927421578 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f643c01d070 con 0x7f644c0ff590 2024-03-21T10:40:13.470 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.472+0000 7f6443400700 1 -- 172.21.15.153:0/3927421578 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f643c0052a0 con 0x7f644c0ff590 2024-03-21T10:40:13.471 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.472+0000 7f6451a00700 1 -- 172.21.15.153:0/3927421578 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f644c198e30 con 0x7f644c0ff590 2024-03-21T10:40:13.471 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.472+0000 7f6443400700 1 -- 172.21.15.153:0/3927421578 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f643c017630 con 0x7f644c0ff590 2024-03-21T10:40:13.471 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.472+0000 7f6451a00700 1 -- 172.21.15.153:0/3927421578 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f644c199320 con 0x7f644c0ff590 2024-03-21T10:40:13.471 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.474+0000 7f6451a00700 1 -- 172.21.15.153:0/3927421578 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f644c10af40 con 0x7f644c0ff590 2024-03-21T10:40:13.472 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.474+0000 7f6443400700 1 -- 172.21.15.153:0/3927421578 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f643c0178a0 con 0x7f644c0ff590 2024-03-21T10:40:13.473 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.475+0000 7f6443400700 1 --2- 172.21.15.153:0/3927421578 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f643006e460 0x7f6430070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:40:13.473 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.475+0000 7f6443400700 1 -- 172.21.15.153:0/3927421578 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f643c08f8b0 con 0x7f644c0ff590 2024-03-21T10:40:13.473 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.476+0000 7f6443e00700 1 --2- 172.21.15.153:0/3927421578 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f643006e460 0x7f6430070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:40:13.474 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.477+0000 7f6443e00700 1 --2- 172.21.15.153:0/3927421578 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f643006e460 0x7f6430070920 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f644c19a270 tx=0x7f6434009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:40:13.478 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.481+0000 7f6443400700 1 -- 172.21.15.153:0/3927421578 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f643c058950 con 0x7f644c0ff590 2024-03-21T10:40:13.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:40:13 smithi153 ceph-mon[113733]: pgmap v852: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:40:13.553 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:40:13 smithi153 ceph-mon[113733]: from='client.35518 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:40:13.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:40:13 smithi174 ceph-mon[88004]: pgmap v852: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:40:13.698 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:40:13 smithi174 ceph-mon[88004]: from='client.35518 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:40:13.790 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.792+0000 7f6451a00700 1 -- 172.21.15.153:0/3927421578 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f644c199f70 con 0x7f644c0ff590 2024-03-21T10:40:13.790 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.793+0000 7f6443400700 1 -- 172.21.15.153:0/3927421578 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f643c003e60 con 0x7f644c0ff590 2024-03-21T10:40:13.791 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:40:13.791 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": { 2024-03-21T10:40:13.791 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:40:13.791 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:40:13.791 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": { 2024-03-21T10:40:13.791 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:40:13.791 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:40:13.792 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": { 2024-03-21T10:40:13.792 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 6 2024-03-21T10:40:13.792 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:40:13.792 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": { 2024-03-21T10:40:13.792 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 4 2024-03-21T10:40:13.792 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:40:13.792 INFO:teuthology.orchestra.run.smithi153.stdout: "overall": { 2024-03-21T10:40:13.792 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 14 2024-03-21T10:40:13.792 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-03-21T10:40:13.792 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:40:13.793 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.795+0000 7f6451a00700 1 -- 172.21.15.153:0/3927421578 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f643006e460 msgr2=0x7f6430070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:40:13.793 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.795+0000 7f6451a00700 1 --2- 172.21.15.153:0/3927421578 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f643006e460 0x7f6430070920 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f644c19a270 tx=0x7f6434009260 comp rx=0 tx=0).stop 2024-03-21T10:40:13.793 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.795+0000 7f6451a00700 1 -- 172.21.15.153:0/3927421578 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f644c0ff590 msgr2=0x7f644c19ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:40:13.793 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.795+0000 7f6451a00700 1 --2- 172.21.15.153:0/3927421578 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f644c0ff590 0x7f644c19ea50 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f643c0092e0 tx=0x7f643c00f460 comp rx=0 tx=0).stop 2024-03-21T10:40:13.794 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.795+0000 7f6451a00700 1 -- 172.21.15.153:0/3927421578 shutdown_connections 2024-03-21T10:40:13.794 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.795+0000 7f6451a00700 1 --2- 172.21.15.153:0/3927421578 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f643006e460 0x7f6430070920 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:40:13.794 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.795+0000 7f6451a00700 1 --2- 172.21.15.153:0/3927421578 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f644c0ff590 0x7f644c19ea50 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:40:13.794 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.795+0000 7f6451a00700 1 --2- 172.21.15.153:0/3927421578 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f644c101c40 0x7f644c19ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:40:13.794 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.796+0000 7f6451a00700 1 -- 172.21.15.153:0/3927421578 >> 172.21.15.153:0/3927421578 conn(0x7f644c075e30 msgr2=0x7f644c0fc770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:40:13.794 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.796+0000 7f6451a00700 1 -- 172.21.15.153:0/3927421578 shutdown_connections 2024-03-21T10:40:13.794 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.796+0000 7f6451a00700 1 -- 172.21.15.153:0/3927421578 wait complete. 2024-03-21T10:40:13.944 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.946+0000 7f589ee00700 1 -- 172.21.15.153:0/1833078590 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f58980ff530 msgr2=0x7f58980ff910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:40:13.944 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.946+0000 7f589ee00700 1 --2- 172.21.15.153:0/1833078590 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f58980ff530 0x7f58980ff910 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f5888009910 tx=0x7f5888009c20 comp rx=0 tx=0).stop 2024-03-21T10:40:13.945 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.946+0000 7f589ee00700 1 -- 172.21.15.153:0/1833078590 shutdown_connections 2024-03-21T10:40:13.945 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.946+0000 7f589ee00700 1 --2- 172.21.15.153:0/1833078590 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f58980ffee0 0x7f5898105810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:40:13.945 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.946+0000 7f589ee00700 1 --2- 172.21.15.153:0/1833078590 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f58980ff530 0x7f58980ff910 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:40:13.945 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.946+0000 7f589ee00700 1 -- 172.21.15.153:0/1833078590 >> 172.21.15.153:0/1833078590 conn(0x7f5898074d70 msgr2=0x7f5898075180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:40:13.945 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.947+0000 7f589ee00700 1 -- 172.21.15.153:0/1833078590 shutdown_connections 2024-03-21T10:40:13.945 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.947+0000 7f589ee00700 1 -- 172.21.15.153:0/1833078590 wait complete. 2024-03-21T10:40:13.945 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.948+0000 7f589ee00700 1 Processor -- start 2024-03-21T10:40:13.946 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.948+0000 7f589ee00700 1 -- start start 2024-03-21T10:40:13.946 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.948+0000 7f589ee00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f58980ff530 0x7f589819c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:40:13.946 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.948+0000 7f589ee00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f58980ffee0 0x7f589819ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:40:13.946 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.948+0000 7f589ee00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f589819d120 con 0x7f58980ffee0 2024-03-21T10:40:13.947 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.948+0000 7f589ee00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58981965d0 con 0x7f58980ff530 2024-03-21T10:40:13.947 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.949+0000 7f588fe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f58980ffee0 0x7f589819ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:40:13.947 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.949+0000 7f588fe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f58980ffee0 0x7f589819ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:50632/0 (socket says 172.21.15.153:50632) 2024-03-21T10:40:13.947 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.949+0000 7f588fe00700 1 -- 172.21.15.153:0/2779077211 learned_addr learned my addr 172.21.15.153:0/2779077211 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:40:13.948 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.949+0000 7f5897e00700 1 --2- 172.21.15.153:0/2779077211 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f58980ff530 0x7f589819c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:40:13.948 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.949+0000 7f588fe00700 1 -- 172.21.15.153:0/2779077211 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f58980ff530 msgr2=0x7f589819c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:40:13.948 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.949+0000 7f588fe00700 1 --2- 172.21.15.153:0/2779077211 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f58980ff530 0x7f589819c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:40:13.948 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.949+0000 7f588fe00700 1 -- 172.21.15.153:0/2779077211 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58880095f0 con 0x7f58980ffee0 2024-03-21T10:40:13.948 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.949+0000 7f5897e00700 1 --2- 172.21.15.153:0/2779077211 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f58980ff530 0x7f589819c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:40:13.948 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.949+0000 7f588fe00700 1 --2- 172.21.15.153:0/2779077211 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f58980ffee0 0x7f589819ca90 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f588000c840 tx=0x7f588000cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:40:13.948 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.950+0000 7f5895600700 1 -- 172.21.15.153:0/2779077211 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f58800041f0 con 0x7f58980ffee0 2024-03-21T10:40:13.948 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.950+0000 7f5895600700 1 -- 172.21.15.153:0/2779077211 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5880004cf0 con 0x7f58980ffee0 2024-03-21T10:40:13.948 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.950+0000 7f589ee00700 1 -- 172.21.15.153:0/2779077211 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58981968b0 con 0x7f58980ffee0 2024-03-21T10:40:13.948 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.950+0000 7f5895600700 1 -- 172.21.15.153:0/2779077211 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5880007740 con 0x7f58980ffee0 2024-03-21T10:40:13.949 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.950+0000 7f589ee00700 1 -- 172.21.15.153:0/2779077211 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5898196e00 con 0x7f58980ffee0 2024-03-21T10:40:13.949 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.951+0000 7f589ee00700 1 -- 172.21.15.153:0/2779077211 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58981091a0 con 0x7f58980ffee0 2024-03-21T10:40:13.950 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.952+0000 7f5895600700 1 -- 172.21.15.153:0/2779077211 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f5880004350 con 0x7f58980ffee0 2024-03-21T10:40:13.951 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.953+0000 7f5895600700 1 --2- 172.21.15.153:0/2779077211 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f587c06e460 0x7f587c070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:40:13.951 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.953+0000 7f5895600700 1 -- 172.21.15.153:0/2779077211 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f588008e100 con 0x7f58980ffee0 2024-03-21T10:40:13.951 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.953+0000 7f5897e00700 1 --2- 172.21.15.153:0/2779077211 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f587c06e460 0x7f587c070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:40:13.951 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.954+0000 7f5897e00700 1 --2- 172.21.15.153:0/2779077211 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f587c06e460 0x7f587c070920 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f5888006010 tx=0x7f588800b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:40:13.955 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:13.959+0000 7f5895600700 1 -- 172.21.15.153:0/2779077211 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f588005acd0 con 0x7f58980ffee0 2024-03-21T10:40:14.222 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:14.224+0000 7f589ee00700 1 -- 172.21.15.153:0/2779077211 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f589804ea90 con 0x7f58980ffee0 2024-03-21T10:40:14.223 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:14.226+0000 7f5895600700 1 -- 172.21.15.153:0/2779077211 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1876 (secure 0 0 0) 0x7f588001d070 con 0x7f58980ffee0 2024-03-21T10:40:14.223 INFO:teuthology.orchestra.run.smithi153.stdout:e32 2024-03-21T10:40:14.224 INFO:teuthology.orchestra.run.smithi153.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-21T10:40:14.224 INFO:teuthology.orchestra.run.smithi153.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:40:14.224 INFO:teuthology.orchestra.run.smithi153.stdout:legacy client fscid: 1 2024-03-21T10:40:14.224 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:40:14.224 INFO:teuthology.orchestra.run.smithi153.stdout:Filesystem 'cephfs' (1) 2024-03-21T10:40:14.224 INFO:teuthology.orchestra.run.smithi153.stdout:fs_name cephfs 2024-03-21T10:40:14.224 INFO:teuthology.orchestra.run.smithi153.stdout:epoch 23 2024-03-21T10:40:14.224 INFO:teuthology.orchestra.run.smithi153.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-21T10:40:14.224 INFO:teuthology.orchestra.run.smithi153.stdout:created 2024-03-21T10:04:27.086576+0000 2024-03-21T10:40:14.224 INFO:teuthology.orchestra.run.smithi153.stdout:modified 2024-03-21T10:19:05.022092+0000 2024-03-21T10:40:14.224 INFO:teuthology.orchestra.run.smithi153.stdout:tableserver 0 2024-03-21T10:40:14.224 INFO:teuthology.orchestra.run.smithi153.stdout:root 0 2024-03-21T10:40:14.224 INFO:teuthology.orchestra.run.smithi153.stdout:session_timeout 60 2024-03-21T10:40:14.225 INFO:teuthology.orchestra.run.smithi153.stdout:session_autoclose 300 2024-03-21T10:40:14.225 INFO:teuthology.orchestra.run.smithi153.stdout:max_file_size 1099511627776 2024-03-21T10:40:14.225 INFO:teuthology.orchestra.run.smithi153.stdout:max_xattr_size 65536 2024-03-21T10:40:14.225 INFO:teuthology.orchestra.run.smithi153.stdout:required_client_features {} 2024-03-21T10:40:14.225 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure 0 2024-03-21T10:40:14.225 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure_osd_epoch 94 2024-03-21T10:40:14.225 INFO:teuthology.orchestra.run.smithi153.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:40:14.225 INFO:teuthology.orchestra.run.smithi153.stdout:max_mds 2 2024-03-21T10:40:14.225 INFO:teuthology.orchestra.run.smithi153.stdout:in 0,1 2024-03-21T10:40:14.225 INFO:teuthology.orchestra.run.smithi153.stdout:up {} 2024-03-21T10:40:14.225 INFO:teuthology.orchestra.run.smithi153.stdout:failed 0,1 2024-03-21T10:40:14.225 INFO:teuthology.orchestra.run.smithi153.stdout:damaged 2024-03-21T10:40:14.225 INFO:teuthology.orchestra.run.smithi153.stdout:stopped 2024-03-21T10:40:14.225 INFO:teuthology.orchestra.run.smithi153.stdout:data_pools [3] 2024-03-21T10:40:14.225 INFO:teuthology.orchestra.run.smithi153.stdout:metadata_pool 2 2024-03-21T10:40:14.226 INFO:teuthology.orchestra.run.smithi153.stdout:inline_data enabled 2024-03-21T10:40:14.226 INFO:teuthology.orchestra.run.smithi153.stdout:balancer 2024-03-21T10:40:14.226 INFO:teuthology.orchestra.run.smithi153.stdout:bal_rank_mask -1 2024-03-21T10:40:14.226 INFO:teuthology.orchestra.run.smithi153.stdout:standby_count_wanted 1 2024-03-21T10:40:14.226 INFO:teuthology.orchestra.run.smithi153.stdout:qdb_cluster leader: 0 members: 2024-03-21T10:40:14.226 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:40:14.226 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:40:14.226 INFO:teuthology.orchestra.run.smithi153.stdout:Standby daemons: 2024-03-21T10:40:14.226 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:40:14.226 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.zipckr{-1:34442} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6826/1103846295,v1:172.21.15.153:6827/1103846295] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:40:14.226 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.iuyluj{-1:34474} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6828/1738630832,v1:172.21.15.153:6829/1738630832] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:40:14.226 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.yyimst{-1:44313} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.174:6824/4127786797,v1:172.21.15.174:6825/4127786797] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:40:14.226 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.nhozag{-1:44687} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.174:6826/2767517845,v1:172.21.15.174:6827/2767517845] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:40:14.227 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:14.230+0000 7f589ee00700 1 -- 172.21.15.153:0/2779077211 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f587c06e460 msgr2=0x7f587c070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:40:14.227 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:14.230+0000 7f589ee00700 1 --2- 172.21.15.153:0/2779077211 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f587c06e460 0x7f587c070920 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f5888006010 tx=0x7f588800b540 comp rx=0 tx=0).stop 2024-03-21T10:40:14.227 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:14.230+0000 7f589ee00700 1 -- 172.21.15.153:0/2779077211 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f58980ffee0 msgr2=0x7f589819ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:40:14.227 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:14.230+0000 7f589ee00700 1 --2- 172.21.15.153:0/2779077211 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f58980ffee0 0x7f589819ca90 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f588000c840 tx=0x7f588000cc00 comp rx=0 tx=0).stop 2024-03-21T10:40:14.227 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:14.230+0000 7f589ee00700 1 -- 172.21.15.153:0/2779077211 shutdown_connections 2024-03-21T10:40:14.228 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:14.230+0000 7f589ee00700 1 --2- 172.21.15.153:0/2779077211 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f58980ff530 0x7f589819c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:40:14.228 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:14.230+0000 7f589ee00700 1 --2- 172.21.15.153:0/2779077211 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f587c06e460 0x7f587c070920 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:40:14.228 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:14.230+0000 7f589ee00700 1 --2- 172.21.15.153:0/2779077211 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f58980ffee0 0x7f589819ca90 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:40:14.228 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:14.230+0000 7f589ee00700 1 -- 172.21.15.153:0/2779077211 >> 172.21.15.153:0/2779077211 conn(0x7f5898074d70 msgr2=0x7f58980fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:40:14.228 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:14.230+0000 7f589ee00700 1 -- 172.21.15.153:0/2779077211 shutdown_connections 2024-03-21T10:40:14.228 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:14.230+0000 7f589ee00700 1 -- 172.21.15.153:0/2779077211 wait complete. 2024-03-21T10:40:14.228 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 32 2024-03-21T10:40:14.380 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:14.383+0000 7f49e8a00700 1 -- 172.21.15.153:0/3552348999 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49e4073a50 msgr2=0x7f49e4111440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:40:14.380 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:14.383+0000 7f49e8a00700 1 --2- 172.21.15.153:0/3552348999 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49e4073a50 0x7f49e4111440 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f49d0009910 tx=0x7f49d0009c20 comp rx=0 tx=0).stop 2024-03-21T10:40:14.380 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:14.383+0000 7f49e8a00700 1 -- 172.21.15.153:0/3552348999 shutdown_connections 2024-03-21T10:40:14.380 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:14.383+0000 7f49e8a00700 1 --2- 172.21.15.153:0/3552348999 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49e4073a50 0x7f49e4111440 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:40:14.380 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:14.383+0000 7f49e8a00700 1 --2- 172.21.15.153:0/3552348999 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f49e4073130 0x7f49e4073510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:40:14.380 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:14.383+0000 7f49e8a00700 1 -- 172.21.15.153:0/3552348999 >> 172.21.15.153:0/3552348999 conn(0x7f49e40fc440 msgr2=0x7f49e40fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:40:14.380 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:14.383+0000 7f49e8a00700 1 -- 172.21.15.153:0/3552348999 shutdown_connections 2024-03-21T10:40:14.380 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:14.383+0000 7f49e8a00700 1 -- 172.21.15.153:0/3552348999 wait complete. 2024-03-21T10:40:14.381 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:14.384+0000 7f49e8a00700 1 Processor -- start 2024-03-21T10:40:14.381 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:14.384+0000 7f49e8a00700 1 -- start start 2024-03-21T10:40:14.382 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:14.385+0000 7f49e8a00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f49e4073130 0x7f49e41a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:40:14.382 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:14.385+0000 7f49e8a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49e4073a50 0x7f49e41a2f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:40:14.382 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:14.385+0000 7f49e8a00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49e41a35b0 con 0x7f49e4073a50 2024-03-21T10:40:14.382 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:14.385+0000 7f49e8a00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49e419ca60 con 0x7f49e4073130 2024-03-21T10:40:14.382 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:14.385+0000 7f49e1e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49e4073a50 0x7f49e41a2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:40:14.383 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:14.385+0000 7f49e1e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49e4073a50 0x7f49e41a2f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:50656/0 (socket says 172.21.15.153:50656) 2024-03-21T10:40:14.383 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:14.385+0000 7f49e1e00700 1 -- 172.21.15.153:0/1298194532 learned_addr learned my addr 172.21.15.153:0/1298194532 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:40:14.383 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:14.385+0000 7f49e2800700 1 --2- 172.21.15.153:0/1298194532 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f49e4073130 0x7f49e41a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:40:14.383 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:14.385+0000 7f49e1e00700 1 -- 172.21.15.153:0/1298194532 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f49e4073130 msgr2=0x7f49e41a29e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:40:14.383 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:14.385+0000 7f49e1e00700 1 --2- 172.21.15.153:0/1298194532 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f49e4073130 0x7f49e41a29e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:40:14.383 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:14.385+0000 7f49e1e00700 1 -- 172.21.15.153:0/1298194532 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49d00095f0 con 0x7f49e4073a50 2024-03-21T10:40:14.383 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:14.386+0000 7f49e1e00700 1 --2- 172.21.15.153:0/1298194532 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49e4073a50 0x7f49e41a2f20 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f49d000b5c0 tx=0x7f49d000be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:40:14.384 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:14.386+0000 7f49db400700 1 -- 172.21.15.153:0/1298194532 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f49d001d070 con 0x7f49e4073a50 2024-03-21T10:40:14.384 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:14.386+0000 7f49db400700 1 -- 172.21.15.153:0/1298194532 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f49d000f460 con 0x7f49e4073a50 2024-03-21T10:40:14.384 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:14.386+0000 7f49db400700 1 -- 172.21.15.153:0/1298194532 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f49d0017630 con 0x7f49e4073a50 2024-03-21T10:40:14.384 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:14.386+0000 7f49e8a00700 1 -- 172.21.15.153:0/1298194532 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49e419cd40 con 0x7f49e4073a50 2024-03-21T10:40:14.384 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:14.386+0000 7f49e8a00700 1 -- 172.21.15.153:0/1298194532 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49e419d240 con 0x7f49e4073a50 2024-03-21T10:40:14.384 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:14.387+0000 7f49e8a00700 1 -- 172.21.15.153:0/1298194532 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49e410ebc0 con 0x7f49e4073a50 2024-03-21T10:40:14.385 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:14.388+0000 7f49db400700 1 -- 172.21.15.153:0/1298194532 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f49d000f5d0 con 0x7f49e4073a50 2024-03-21T10:40:14.386 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:14.389+0000 7f49db400700 1 --2- 172.21.15.153:0/1298194532 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f49c806e460 0x7f49c8070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:40:14.386 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:14.389+0000 7f49db400700 1 -- 172.21.15.153:0/1298194532 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f49d008fb30 con 0x7f49e4073a50 2024-03-21T10:40:14.386 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:14.389+0000 7f49e2800700 1 --2- 172.21.15.153:0/1298194532 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f49c806e460 0x7f49c8070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:40:14.387 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:14.390+0000 7f49e2800700 1 --2- 172.21.15.153:0/1298194532 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f49c806e460 0x7f49c8070920 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f49d4009f20 tx=0x7f49d40094f0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:40:14.391 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:14.394+0000 7f49db400700 1 -- 172.21.15.153:0/1298194532 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f49d005c680 con 0x7f49e4073a50 2024-03-21T10:40:14.553 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:40:14 smithi153 ceph-mon[113733]: from='client.35522 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:40:14.553 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:40:14 smithi153 ceph-mon[113733]: from='client.35526 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:40:14.553 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:40:14 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/3927421578' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:40:14.553 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:40:14 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/2779077211' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:40:14.631 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:14.634+0000 7f49e8a00700 1 -- 172.21.15.153:0/1298194532 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f49e419d570 con 0x7f49c806e460 2024-03-21T10:40:14.633 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:14.636+0000 7f49db400700 1 -- 172.21.15.153:0/1298194532 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f49e419d570 con 0x7f49c806e460 2024-03-21T10:40:14.633 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:40:14.634 INFO:teuthology.orchestra.run.smithi153.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232", 2024-03-21T10:40:14.634 INFO:teuthology.orchestra.run.smithi153.stdout: "in_progress": true, 2024-03-21T10:40:14.634 INFO:teuthology.orchestra.run.smithi153.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-21T10:40:14.634 INFO:teuthology.orchestra.run.smithi153.stdout: "services_complete": [ 2024-03-21T10:40:14.634 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr", 2024-03-21T10:40:14.634 INFO:teuthology.orchestra.run.smithi153.stdout: "mon", 2024-03-21T10:40:14.634 INFO:teuthology.orchestra.run.smithi153.stdout: "mds", 2024-03-21T10:40:14.634 INFO:teuthology.orchestra.run.smithi153.stdout: "crash", 2024-03-21T10:40:14.634 INFO:teuthology.orchestra.run.smithi153.stdout: "osd" 2024-03-21T10:40:14.634 INFO:teuthology.orchestra.run.smithi153.stdout: ], 2024-03-21T10:40:14.634 INFO:teuthology.orchestra.run.smithi153.stdout: "progress": "16/23 daemons upgraded", 2024-03-21T10:40:14.634 INFO:teuthology.orchestra.run.smithi153.stdout: "message": "Currently upgrading mds daemons", 2024-03-21T10:40:14.635 INFO:teuthology.orchestra.run.smithi153.stdout: "is_paused": false 2024-03-21T10:40:14.635 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:40:14.636 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:14.639+0000 7f49e8a00700 1 -- 172.21.15.153:0/1298194532 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f49c806e460 msgr2=0x7f49c8070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:40:14.636 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:14.639+0000 7f49e8a00700 1 --2- 172.21.15.153:0/1298194532 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f49c806e460 0x7f49c8070920 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f49d4009f20 tx=0x7f49d40094f0 comp rx=0 tx=0).stop 2024-03-21T10:40:14.636 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:14.639+0000 7f49e8a00700 1 -- 172.21.15.153:0/1298194532 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49e4073a50 msgr2=0x7f49e41a2f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:40:14.636 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:14.639+0000 7f49e8a00700 1 --2- 172.21.15.153:0/1298194532 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49e4073a50 0x7f49e41a2f20 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f49d000b5c0 tx=0x7f49d000be00 comp rx=0 tx=0).stop 2024-03-21T10:40:14.637 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:14.639+0000 7f49e8a00700 1 -- 172.21.15.153:0/1298194532 shutdown_connections 2024-03-21T10:40:14.637 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:14.639+0000 7f49e8a00700 1 --2- 172.21.15.153:0/1298194532 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f49e4073130 0x7f49e41a29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:40:14.637 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:14.639+0000 7f49e8a00700 1 --2- 172.21.15.153:0/1298194532 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f49c806e460 0x7f49c8070920 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:40:14.637 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:14.639+0000 7f49e8a00700 1 --2- 172.21.15.153:0/1298194532 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49e4073a50 0x7f49e41a2f20 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:40:14.637 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:14.639+0000 7f49e8a00700 1 -- 172.21.15.153:0/1298194532 >> 172.21.15.153:0/1298194532 conn(0x7f49e40fc440 msgr2=0x7f49e4102f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:40:14.638 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:14.640+0000 7f49e8a00700 1 -- 172.21.15.153:0/1298194532 shutdown_connections 2024-03-21T10:40:14.638 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:14.640+0000 7f49e8a00700 1 -- 172.21.15.153:0/1298194532 wait complete. 2024-03-21T10:40:14.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:40:14 smithi174 ceph-mon[88004]: from='client.35522 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:40:14.698 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:40:14 smithi174 ceph-mon[88004]: from='client.35526 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:40:14.698 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:40:14 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/3927421578' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:40:14.698 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:40:14 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/2779077211' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:40:14.788 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:14.791+0000 7f167aa00700 1 -- 172.21.15.153:0/371862285 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1674107bd0 msgr2=0x7f167410bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:40:14.788 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:14.791+0000 7f167aa00700 1 --2- 172.21.15.153:0/371862285 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1674107bd0 0x7f167410bc10 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f1660009910 tx=0x7f1660009c20 comp rx=0 tx=0).stop 2024-03-21T10:40:14.788 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:14.791+0000 7f167aa00700 1 -- 172.21.15.153:0/371862285 shutdown_connections 2024-03-21T10:40:14.788 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:14.791+0000 7f167aa00700 1 --2- 172.21.15.153:0/371862285 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1674107bd0 0x7f167410bc10 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:40:14.788 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:14.791+0000 7f167aa00700 1 --2- 172.21.15.153:0/371862285 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1674107220 0x7f1674107600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:40:14.789 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:14.791+0000 7f167aa00700 1 -- 172.21.15.153:0/371862285 >> 172.21.15.153:0/371862285 conn(0x7f16740767e0 msgr2=0x7f1674078c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:40:14.789 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:14.791+0000 7f167aa00700 1 -- 172.21.15.153:0/371862285 shutdown_connections 2024-03-21T10:40:14.789 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:14.791+0000 7f167aa00700 1 -- 172.21.15.153:0/371862285 wait complete. 2024-03-21T10:40:14.790 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:14.792+0000 7f167aa00700 1 Processor -- start 2024-03-21T10:40:14.790 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:14.792+0000 7f167aa00700 1 -- start start 2024-03-21T10:40:14.790 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:14.792+0000 7f167aa00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1674107220 0x7f16741a29f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:40:14.790 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:14.792+0000 7f167aa00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1674107bd0 0x7f16741a2f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:40:14.791 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:14.792+0000 7f167aa00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16741a3530 con 0x7f1674107220 2024-03-21T10:40:14.791 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:14.792+0000 7f167aa00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f167419ca70 con 0x7f1674107bd0 2024-03-21T10:40:14.791 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:14.793+0000 7f1679600700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1674107220 0x7f16741a29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:40:14.791 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:14.793+0000 7f1679600700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1674107220 0x7f16741a29f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:50668/0 (socket says 172.21.15.153:50668) 2024-03-21T10:40:14.791 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:14.793+0000 7f1679600700 1 -- 172.21.15.153:0/2868021202 learned_addr learned my addr 172.21.15.153:0/2868021202 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:40:14.791 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:14.793+0000 7f1678c00700 1 --2- 172.21.15.153:0/2868021202 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1674107bd0 0x7f16741a2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:40:14.791 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:14.793+0000 7f1679600700 1 -- 172.21.15.153:0/2868021202 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1674107bd0 msgr2=0x7f16741a2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:40:14.791 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:14.793+0000 7f1679600700 1 --2- 172.21.15.153:0/2868021202 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1674107bd0 0x7f16741a2f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:40:14.791 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:14.793+0000 7f1679600700 1 -- 172.21.15.153:0/2868021202 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16600095f0 con 0x7f1674107220 2024-03-21T10:40:14.791 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:14.793+0000 7f1679600700 1 --2- 172.21.15.153:0/2868021202 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1674107220 0x7f16741a29f0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f166400cfd0 tx=0x7f166400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:40:14.792 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:14.794+0000 7f1672000700 1 -- 172.21.15.153:0/2868021202 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f166400ad70 con 0x7f1674107220 2024-03-21T10:40:14.792 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:14.794+0000 7f1672000700 1 -- 172.21.15.153:0/2868021202 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f16640128e0 con 0x7f1674107220 2024-03-21T10:40:14.792 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:14.794+0000 7f1672000700 1 -- 172.21.15.153:0/2868021202 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1664011880 con 0x7f1674107220 2024-03-21T10:40:14.792 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:14.794+0000 7f167aa00700 1 -- 172.21.15.153:0/2868021202 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f167419cd50 con 0x7f1674107220 2024-03-21T10:40:14.792 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:14.794+0000 7f167aa00700 1 -- 172.21.15.153:0/2868021202 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f167419d2a0 con 0x7f1674107220 2024-03-21T10:40:14.792 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:14.795+0000 7f167aa00700 1 -- 172.21.15.153:0/2868021202 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f167410f610 con 0x7f1674107220 2024-03-21T10:40:14.793 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:14.796+0000 7f1672000700 1 -- 172.21.15.153:0/2868021202 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f1664012420 con 0x7f1674107220 2024-03-21T10:40:14.793 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:14.797+0000 7f1672000700 1 --2- 172.21.15.153:0/2868021202 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f165806e460 0x7f1658070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:40:14.794 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:14.797+0000 7f1672000700 1 -- 172.21.15.153:0/2868021202 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f166408e720 con 0x7f1674107220 2024-03-21T10:40:14.794 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:14.797+0000 7f1678c00700 1 --2- 172.21.15.153:0/2868021202 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f165806e460 0x7f1658070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:40:14.794 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:14.798+0000 7f1678c00700 1 --2- 172.21.15.153:0/2868021202 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f165806e460 0x7f1658070920 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f1660000c00 tx=0x7f1660005fd0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:40:14.798 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:14.802+0000 7f1672000700 1 -- 172.21.15.153:0/2868021202 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f166405b270 con 0x7f1674107220 2024-03-21T10:40:15.094 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:15.096+0000 7f167aa00700 1 -- 172.21.15.153:0/2868021202 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f167419d520 con 0x7f1674107220 2024-03-21T10:40:15.094 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:15.097+0000 7f1672000700 1 -- 172.21.15.153:0/2868021202 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4493 (secure 0 0 0) 0x7f166405ae00 con 0x7f1674107220 2024-03-21T10:40:15.095 INFO:teuthology.orchestra.run.smithi153.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-03-21T10:40:15.095 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-03-21T10:40:15.095 INFO:teuthology.orchestra.run.smithi153.stdout: host smithi174 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi174/config 2024-03-21T10:40:15.095 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/podman: timeout after 895 seconds 2024-03-21T10:40:15.095 INFO:teuthology.orchestra.run.smithi153.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 -e NODE_NAME=smithi174 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/run/ceph:z -v /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpd910r3_i:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 inventory --format=json-pretty --filter-for-batch 2024-03-21T10:40:15.095 INFO:teuthology.orchestra.run.smithi153.stdout:Traceback (most recent call last): 2024-03-21T10:40:15.095 INFO:teuthology.orchestra.run.smithi153.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-03-21T10:40:15.095 INFO:teuthology.orchestra.run.smithi153.stdout: return _run_code(code, main_globals, None, 2024-03-21T10:40:15.095 INFO:teuthology.orchestra.run.smithi153.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-03-21T10:40:15.095 INFO:teuthology.orchestra.run.smithi153.stdout: exec(code, run_globals) 2024-03-21T10:40:15.095 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 5606, in 2024-03-21T10:40:15.095 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 5594, in main 2024-03-21T10:40:15.096 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 406, in _infer_config 2024-03-21T10:40:15.096 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 321, in _infer_fsid 2024-03-21T10:40:15.096 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 434, in _infer_image 2024-03-21T10:40:15.096 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 308, in _validate_fsid 2024-03-21T10:40:15.096 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 3357, in command_ceph_volume 2024-03-21T10:40:15.096 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-03-21T10:40:15.096 INFO:teuthology.orchestra.run.smithi153.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 -e NODE_NAME=smithi174 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/run/ceph:z -v /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpd910r3_i:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 inventory --format=json-pretty --filter-for-batch 2024-03-21T10:40:15.096 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-21T10:40:15.096 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is degraded 2024-03-21T10:40:15.096 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-21T10:40:15.096 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-21T10:40:15.096 INFO:teuthology.orchestra.run.smithi153.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-21T10:40:15.096 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-21T10:40:15.097 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:15.100+0000 7f167aa00700 1 -- 172.21.15.153:0/2868021202 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f165806e460 msgr2=0x7f1658070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:40:15.097 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:15.100+0000 7f167aa00700 1 --2- 172.21.15.153:0/2868021202 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f165806e460 0x7f1658070920 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f1660000c00 tx=0x7f1660005fd0 comp rx=0 tx=0).stop 2024-03-21T10:40:15.097 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:15.100+0000 7f167aa00700 1 -- 172.21.15.153:0/2868021202 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1674107220 msgr2=0x7f16741a29f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:40:15.098 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:15.100+0000 7f167aa00700 1 --2- 172.21.15.153:0/2868021202 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1674107220 0x7f16741a29f0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f166400cfd0 tx=0x7f166400a340 comp rx=0 tx=0).stop 2024-03-21T10:40:15.098 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:15.100+0000 7f167aa00700 1 -- 172.21.15.153:0/2868021202 shutdown_connections 2024-03-21T10:40:15.098 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:15.100+0000 7f167aa00700 1 --2- 172.21.15.153:0/2868021202 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f165806e460 0x7f1658070920 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:40:15.098 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:15.100+0000 7f167aa00700 1 --2- 172.21.15.153:0/2868021202 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1674107220 0x7f16741a29f0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:40:15.098 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:15.100+0000 7f167aa00700 1 --2- 172.21.15.153:0/2868021202 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1674107bd0 0x7f16741a2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:40:15.098 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:15.100+0000 7f167aa00700 1 -- 172.21.15.153:0/2868021202 >> 172.21.15.153:0/2868021202 conn(0x7f16740767e0 msgr2=0x7f1674078bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:40:15.098 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:15.100+0000 7f167aa00700 1 -- 172.21.15.153:0/2868021202 shutdown_connections 2024-03-21T10:40:15.098 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:15.100+0000 7f167aa00700 1 -- 172.21.15.153:0/2868021202 wait complete. 2024-03-21T10:40:15.553 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:40:15 smithi153 ceph-mon[113733]: pgmap v853: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:40:15.553 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:40:15 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/2868021202' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:40:15.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:40:15 smithi174 ceph-mon[88004]: pgmap v853: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:40:15.698 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:40:15 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/2868021202' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:40:16.698 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:40:16 smithi174 ceph-mon[88004]: from='client.35538 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:40:16.698 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:40:16 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:40:16.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:40:16 smithi153 ceph-mon[113733]: from='client.35538 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:40:16.803 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:40:16 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:40:17.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:40:17 smithi174 ceph-mon[88004]: pgmap v854: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:40:17.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:40:17 smithi153 ceph-mon[113733]: pgmap v854: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:40:19.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:40:19 smithi174 ceph-mon[88004]: pgmap v855: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:40:19.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:40:19 smithi153 ceph-mon[113733]: pgmap v855: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:40:21.698 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:40:21 smithi174 ceph-mon[88004]: pgmap v856: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:40:21.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:40:21 smithi153 ceph-mon[113733]: pgmap v856: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:40:23.685 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:40:23 smithi153 ceph-mon[113733]: pgmap v857: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:40:23.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:40:23 smithi174 ceph-mon[88004]: pgmap v857: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:40:25.448 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:40:25 smithi174 ceph-mon[88004]: pgmap v858: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:40:25.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:40:25 smithi153 ceph-mon[113733]: pgmap v858: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:40:27.698 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:40:27 smithi174 ceph-mon[88004]: pgmap v859: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:40:27.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:40:27 smithi153 ceph-mon[113733]: pgmap v859: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:40:29.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:40:29 smithi174 ceph-mon[88004]: pgmap v860: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:40:29.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:40:29 smithi153 ceph-mon[113733]: pgmap v860: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:40:31.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:40:31 smithi174 ceph-mon[88004]: pgmap v861: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:40:31.698 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:40:31 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:40:31.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:40:31 smithi153 ceph-mon[113733]: pgmap v861: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:40:31.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:40:31 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:40:33.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:40:33 smithi174 ceph-mon[88004]: pgmap v862: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:40:33.803 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:40:33 smithi153 ceph-mon[113733]: pgmap v862: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:40:35.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:40:35 smithi174 ceph-mon[88004]: pgmap v863: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:40:35.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:40:35 smithi153 ceph-mon[113733]: pgmap v863: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:40:37.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:40:37 smithi174 ceph-mon[88004]: pgmap v864: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:40:37.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:40:37 smithi153 ceph-mon[113733]: pgmap v864: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:40:39.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:40:39 smithi153 ceph-mon[113733]: pgmap v865: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:40:39.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:40:39 smithi174 ceph-mon[88004]: pgmap v865: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:40:41.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:40:41 smithi174 ceph-mon[88004]: pgmap v866: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:40:41.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:40:41 smithi153 ceph-mon[113733]: pgmap v866: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:40:43.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:40:43 smithi174 ceph-mon[88004]: pgmap v867: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:40:43.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:40:43 smithi153 ceph-mon[113733]: pgmap v867: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:40:45.266 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:45.268+0000 7fa814c00700 1 -- 172.21.15.153:0/4225564357 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa810073a50 msgr2=0x7fa810111610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:40:45.266 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:45.268+0000 7fa814c00700 1 --2- 172.21.15.153:0/4225564357 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa810073a50 0x7fa810111610 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fa7fc009910 tx=0x7fa7fc009c20 comp rx=0 tx=0).stop 2024-03-21T10:40:45.266 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:45.268+0000 7fa814c00700 1 -- 172.21.15.153:0/4225564357 shutdown_connections 2024-03-21T10:40:45.266 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:45.268+0000 7fa814c00700 1 --2- 172.21.15.153:0/4225564357 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa810073a50 0x7fa810111610 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:40:45.267 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:45.268+0000 7fa814c00700 1 --2- 172.21.15.153:0/4225564357 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa810073130 0x7fa810073510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:40:45.267 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:45.268+0000 7fa814c00700 1 -- 172.21.15.153:0/4225564357 >> 172.21.15.153:0/4225564357 conn(0x7fa8100fc630 msgr2=0x7fa8100fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:40:45.267 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:45.268+0000 7fa814c00700 1 -- 172.21.15.153:0/4225564357 shutdown_connections 2024-03-21T10:40:45.267 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:45.268+0000 7fa814c00700 1 -- 172.21.15.153:0/4225564357 wait complete. 2024-03-21T10:40:45.268 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:45.269+0000 7fa814c00700 1 Processor -- start 2024-03-21T10:40:45.268 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:45.269+0000 7fa814c00700 1 -- start start 2024-03-21T10:40:45.268 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:45.270+0000 7fa814c00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa810073130 0x7fa81019cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:40:45.268 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:45.270+0000 7fa814c00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa810073a50 0x7fa81019d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:40:45.268 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:45.270+0000 7fa814c00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa81019d930 con 0x7fa810073130 2024-03-21T10:40:45.268 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:45.270+0000 7fa814c00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8101a16c0 con 0x7fa810073a50 2024-03-21T10:40:45.268 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:45.270+0000 7fa80de00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa810073130 0x7fa81019cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:40:45.268 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:45.270+0000 7fa80de00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa810073130 0x7fa81019cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:58266/0 (socket says 172.21.15.153:58266) 2024-03-21T10:40:45.269 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:45.270+0000 7fa80de00700 1 -- 172.21.15.153:0/868468421 learned_addr learned my addr 172.21.15.153:0/868468421 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:40:45.269 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:45.270+0000 7fa80d400700 1 --2- 172.21.15.153:0/868468421 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa810073a50 0x7fa81019d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:40:45.269 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:45.270+0000 7fa80de00700 1 -- 172.21.15.153:0/868468421 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa810073a50 msgr2=0x7fa81019d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:40:45.269 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:45.270+0000 7fa80de00700 1 --2- 172.21.15.153:0/868468421 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa810073a50 0x7fa81019d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:40:45.269 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:45.270+0000 7fa80de00700 1 -- 172.21.15.153:0/868468421 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7fc0095f0 con 0x7fa810073130 2024-03-21T10:40:45.270 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:45.271+0000 7fa80de00700 1 --2- 172.21.15.153:0/868468421 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa810073130 0x7fa81019cd10 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fa80000c8a0 tx=0x7fa80000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:40:45.270 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:45.271+0000 7fa806a00700 1 -- 172.21.15.153:0/868468421 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa80000cea0 con 0x7fa810073130 2024-03-21T10:40:45.270 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:45.271+0000 7fa806a00700 1 -- 172.21.15.153:0/868468421 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa800004830 con 0x7fa810073130 2024-03-21T10:40:45.270 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:45.271+0000 7fa814c00700 1 -- 172.21.15.153:0/868468421 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8101a19a0 con 0x7fa810073130 2024-03-21T10:40:45.270 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:45.271+0000 7fa806a00700 1 -- 172.21.15.153:0/868468421 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa8000055b0 con 0x7fa810073130 2024-03-21T10:40:45.270 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:45.271+0000 7fa814c00700 1 -- 172.21.15.153:0/868468421 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8101a1ef0 con 0x7fa810073130 2024-03-21T10:40:45.270 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:45.272+0000 7fa814c00700 1 -- 172.21.15.153:0/868468421 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa81010ed90 con 0x7fa810073130 2024-03-21T10:40:45.271 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:45.273+0000 7fa806a00700 1 -- 172.21.15.153:0/868468421 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fa8000049a0 con 0x7fa810073130 2024-03-21T10:40:45.272 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:45.274+0000 7fa806a00700 1 --2- 172.21.15.153:0/868468421 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa7f406e460 0x7fa7f4070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:40:45.272 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:45.275+0000 7fa806a00700 1 -- 172.21.15.153:0/868468421 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fa80008e170 con 0x7fa810073130 2024-03-21T10:40:45.272 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:45.275+0000 7fa80d400700 1 --2- 172.21.15.153:0/868468421 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa7f406e460 0x7fa7f4070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:40:45.273 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:45.275+0000 7fa80d400700 1 --2- 172.21.15.153:0/868468421 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa7f406e460 0x7fa7f4070920 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fa81019e330 tx=0x7fa7fc01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:40:45.277 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:45.280+0000 7fa806a00700 1 -- 172.21.15.153:0/868468421 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa800057210 con 0x7fa810073130 2024-03-21T10:40:45.518 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:45.520+0000 7fa814c00700 1 -- 172.21.15.153:0/868468421 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa8101a2300 con 0x7fa7f406e460 2024-03-21T10:40:45.519 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:40:45 smithi153 ceph-mon[113733]: pgmap v868: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:40:45.520 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:45.522+0000 7fa806a00700 1 -- 172.21.15.153:0/868468421 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fa8101a2300 con 0x7fa7f406e460 2024-03-21T10:40:45.523 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:45.525+0000 7fa814c00700 1 -- 172.21.15.153:0/868468421 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa7f406e460 msgr2=0x7fa7f4070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:40:45.523 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:45.525+0000 7fa814c00700 1 --2- 172.21.15.153:0/868468421 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa7f406e460 0x7fa7f4070920 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fa81019e330 tx=0x7fa7fc01a040 comp rx=0 tx=0).stop 2024-03-21T10:40:45.523 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:45.525+0000 7fa814c00700 1 -- 172.21.15.153:0/868468421 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa810073130 msgr2=0x7fa81019cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:40:45.523 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:45.525+0000 7fa814c00700 1 --2- 172.21.15.153:0/868468421 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa810073130 0x7fa81019cd10 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fa80000c8a0 tx=0x7fa80000cc60 comp rx=0 tx=0).stop 2024-03-21T10:40:45.524 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:45.525+0000 7fa814c00700 1 -- 172.21.15.153:0/868468421 shutdown_connections 2024-03-21T10:40:45.524 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:45.525+0000 7fa814c00700 1 --2- 172.21.15.153:0/868468421 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa7f406e460 0x7fa7f4070920 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:40:45.524 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:45.525+0000 7fa814c00700 1 --2- 172.21.15.153:0/868468421 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa810073130 0x7fa81019cd10 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:40:45.524 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:45.525+0000 7fa814c00700 1 --2- 172.21.15.153:0/868468421 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa810073a50 0x7fa81019d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:40:45.524 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:45.525+0000 7fa814c00700 1 -- 172.21.15.153:0/868468421 >> 172.21.15.153:0/868468421 conn(0x7fa8100fc630 msgr2=0x7fa810103120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:40:45.524 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:45.526+0000 7fa814c00700 1 -- 172.21.15.153:0/868468421 shutdown_connections 2024-03-21T10:40:45.524 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:45.526+0000 7fa814c00700 1 -- 172.21.15.153:0/868468421 wait complete. 2024-03-21T10:40:45.540 INFO:teuthology.orchestra.run.smithi153.stdout:true 2024-03-21T10:40:45.685 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:45.687+0000 7fd67ac00700 1 -- 172.21.15.153:0/1848109540 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd674103000 msgr2=0x7fd6741033e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:40:45.685 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:45.687+0000 7fd67ac00700 1 --2- 172.21.15.153:0/1848109540 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd674103000 0x7fd6741033e0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fd664009910 tx=0x7fd664009c20 comp rx=0 tx=0).stop 2024-03-21T10:40:45.685 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:45.687+0000 7fd67ac00700 1 -- 172.21.15.153:0/1848109540 shutdown_connections 2024-03-21T10:40:45.686 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:45.687+0000 7fd67ac00700 1 --2- 172.21.15.153:0/1848109540 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd6741039b0 0x7fd674107a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:40:45.686 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:45.687+0000 7fd67ac00700 1 --2- 172.21.15.153:0/1848109540 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd674103000 0x7fd6741033e0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:40:45.686 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:45.687+0000 7fd67ac00700 1 -- 172.21.15.153:0/1848109540 >> 172.21.15.153:0/1848109540 conn(0x7fd6740fe8d0 msgr2=0x7fd674100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:40:45.686 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:45.687+0000 7fd67ac00700 1 -- 172.21.15.153:0/1848109540 shutdown_connections 2024-03-21T10:40:45.686 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:45.687+0000 7fd67ac00700 1 -- 172.21.15.153:0/1848109540 wait complete. 2024-03-21T10:40:45.687 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:45.688+0000 7fd67ac00700 1 Processor -- start 2024-03-21T10:40:45.687 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:45.688+0000 7fd67ac00700 1 -- start start 2024-03-21T10:40:45.687 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:45.689+0000 7fd67ac00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd674103000 0x7fd67419e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:40:45.687 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:45.689+0000 7fd67ac00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd6741039b0 0x7fd67419ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:40:45.687 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:45.689+0000 7fd67ac00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd67419f3c0 con 0x7fd6741039b0 2024-03-21T10:40:45.687 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:45.689+0000 7fd67ac00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd674198870 con 0x7fd674103000 2024-03-21T10:40:45.688 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:45.689+0000 7fd673400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd6741039b0 0x7fd67419ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:40:45.688 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:45.689+0000 7fd673400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd6741039b0 0x7fd67419ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:58276/0 (socket says 172.21.15.153:58276) 2024-03-21T10:40:45.688 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:45.689+0000 7fd673400700 1 -- 172.21.15.153:0/2435627406 learned_addr learned my addr 172.21.15.153:0/2435627406 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:40:45.688 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:45.689+0000 7fd673e00700 1 --2- 172.21.15.153:0/2435627406 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd674103000 0x7fd67419e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:40:45.688 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:45.690+0000 7fd673400700 1 -- 172.21.15.153:0/2435627406 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd674103000 msgr2=0x7fd67419e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:40:45.688 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:45.690+0000 7fd673400700 1 --2- 172.21.15.153:0/2435627406 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd674103000 0x7fd67419e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:40:45.688 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:45.690+0000 7fd673400700 1 -- 172.21.15.153:0/2435627406 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd660009520 con 0x7fd6741039b0 2024-03-21T10:40:45.689 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:45.690+0000 7fd673400700 1 --2- 172.21.15.153:0/2435627406 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd6741039b0 0x7fd67419ed30 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fd66000e9b0 tx=0x7fd66000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:40:45.689 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:45.690+0000 7fd670c00700 1 -- 172.21.15.153:0/2435627406 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd66000cba0 con 0x7fd6741039b0 2024-03-21T10:40:45.689 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:45.690+0000 7fd670c00700 1 -- 172.21.15.153:0/2435627406 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd6600044e0 con 0x7fd6741039b0 2024-03-21T10:40:45.689 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:45.690+0000 7fd670c00700 1 -- 172.21.15.153:0/2435627406 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd660005080 con 0x7fd6741039b0 2024-03-21T10:40:45.689 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:45.690+0000 7fd67ac00700 1 -- 172.21.15.153:0/2435627406 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6640095f0 con 0x7fd6741039b0 2024-03-21T10:40:45.689 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:45.690+0000 7fd67ac00700 1 -- 172.21.15.153:0/2435627406 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd674198ec0 con 0x7fd6741039b0 2024-03-21T10:40:45.689 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:45.692+0000 7fd67ac00700 1 -- 172.21.15.153:0/2435627406 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd67410b3a0 con 0x7fd6741039b0 2024-03-21T10:40:45.690 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:45.693+0000 7fd670c00700 1 -- 172.21.15.153:0/2435627406 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fd660004750 con 0x7fd6741039b0 2024-03-21T10:40:45.692 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:45.694+0000 7fd670c00700 1 --2- 172.21.15.153:0/2435627406 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd65806e460 0x7fd658070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:40:45.692 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:45.694+0000 7fd670c00700 1 -- 172.21.15.153:0/2435627406 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fd660014070 con 0x7fd6741039b0 2024-03-21T10:40:45.692 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:45.694+0000 7fd673e00700 1 --2- 172.21.15.153:0/2435627406 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd65806e460 0x7fd658070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:40:45.692 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:45.695+0000 7fd673e00700 1 --2- 172.21.15.153:0/2435627406 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd65806e460 0x7fd658070920 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fd66400b760 tx=0x7fd664005a90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:40:45.697 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:45.699+0000 7fd670c00700 1 -- 172.21.15.153:0/2435627406 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd66005b1a0 con 0x7fd6741039b0 2024-03-21T10:40:45.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:40:45 smithi174 ceph-mon[88004]: pgmap v868: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:40:45.936 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:45.938+0000 7fd67ac00700 1 -- 172.21.15.153:0/2435627406 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd67402cce0 con 0x7fd65806e460 2024-03-21T10:40:45.938 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:45.940+0000 7fd670c00700 1 -- 172.21.15.153:0/2435627406 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fd67402cce0 con 0x7fd65806e460 2024-03-21T10:40:45.941 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:45.943+0000 7fd67ac00700 1 -- 172.21.15.153:0/2435627406 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd65806e460 msgr2=0x7fd658070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:40:45.942 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:45.943+0000 7fd67ac00700 1 --2- 172.21.15.153:0/2435627406 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd65806e460 0x7fd658070920 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fd66400b760 tx=0x7fd664005a90 comp rx=0 tx=0).stop 2024-03-21T10:40:45.942 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:45.943+0000 7fd67ac00700 1 -- 172.21.15.153:0/2435627406 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd6741039b0 msgr2=0x7fd67419ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:40:45.942 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:45.943+0000 7fd67ac00700 1 --2- 172.21.15.153:0/2435627406 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd6741039b0 0x7fd67419ed30 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fd66000e9b0 tx=0x7fd66000ed70 comp rx=0 tx=0).stop 2024-03-21T10:40:45.942 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:45.944+0000 7fd67ac00700 1 -- 172.21.15.153:0/2435627406 shutdown_connections 2024-03-21T10:40:45.942 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:45.944+0000 7fd67ac00700 1 --2- 172.21.15.153:0/2435627406 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd674103000 0x7fd67419e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:40:45.942 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:45.944+0000 7fd67ac00700 1 --2- 172.21.15.153:0/2435627406 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd65806e460 0x7fd658070920 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:40:45.942 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:45.944+0000 7fd67ac00700 1 --2- 172.21.15.153:0/2435627406 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd6741039b0 0x7fd67419ed30 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:40:45.942 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:45.944+0000 7fd67ac00700 1 -- 172.21.15.153:0/2435627406 >> 172.21.15.153:0/2435627406 conn(0x7fd6740fe8d0 msgr2=0x7fd6740ffee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:40:45.942 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:45.944+0000 7fd67ac00700 1 -- 172.21.15.153:0/2435627406 shutdown_connections 2024-03-21T10:40:45.942 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:45.944+0000 7fd67ac00700 1 -- 172.21.15.153:0/2435627406 wait complete. 2024-03-21T10:40:46.092 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.094+0000 7f44e1600700 1 -- 172.21.15.153:0/4287502575 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f44dc10d280 msgr2=0x7f44dc10d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:40:46.092 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.094+0000 7f44e1600700 1 --2- 172.21.15.153:0/4287502575 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f44dc10d280 0x7f44dc10d660 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f44cc009910 tx=0x7f44cc009c20 comp rx=0 tx=0).stop 2024-03-21T10:40:46.092 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.094+0000 7f44e1600700 1 -- 172.21.15.153:0/4287502575 shutdown_connections 2024-03-21T10:40:46.092 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.094+0000 7f44e1600700 1 --2- 172.21.15.153:0/4287502575 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f44dc1071d0 0x7f44dc107650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:40:46.092 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.094+0000 7f44e1600700 1 --2- 172.21.15.153:0/4287502575 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f44dc10d280 0x7f44dc10d660 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:40:46.092 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.094+0000 7f44e1600700 1 -- 172.21.15.153:0/4287502575 >> 172.21.15.153:0/4287502575 conn(0x7f44dc0769e0 msgr2=0x7f44dc078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:40:46.092 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.094+0000 7f44e1600700 1 -- 172.21.15.153:0/4287502575 shutdown_connections 2024-03-21T10:40:46.092 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.094+0000 7f44e1600700 1 -- 172.21.15.153:0/4287502575 wait complete. 2024-03-21T10:40:46.093 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.095+0000 7f44e1600700 1 Processor -- start 2024-03-21T10:40:46.093 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.095+0000 7f44e1600700 1 -- start start 2024-03-21T10:40:46.093 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.095+0000 7f44e1600700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f44dc1071d0 0x7f44dc19ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:40:46.093 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.095+0000 7f44e1600700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f44dc10d280 0x7f44dc19d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:40:46.093 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.095+0000 7f44e1600700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44dc19d8a0 con 0x7f44dc1071d0 2024-03-21T10:40:46.094 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.095+0000 7f44e1600700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44dc1a05d0 con 0x7f44dc10d280 2024-03-21T10:40:46.094 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.096+0000 7f44daa00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f44dc1071d0 0x7f44dc19ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:40:46.094 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.096+0000 7f44daa00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f44dc1071d0 0x7f44dc19ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:58298/0 (socket says 172.21.15.153:58298) 2024-03-21T10:40:46.094 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.096+0000 7f44daa00700 1 -- 172.21.15.153:0/608277050 learned_addr learned my addr 172.21.15.153:0/608277050 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:40:46.095 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.096+0000 7f44da000700 1 --2- 172.21.15.153:0/608277050 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f44dc10d280 0x7f44dc19d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:40:46.095 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.096+0000 7f44daa00700 1 -- 172.21.15.153:0/608277050 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f44dc10d280 msgr2=0x7f44dc19d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:40:46.095 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.096+0000 7f44daa00700 1 --2- 172.21.15.153:0/608277050 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f44dc10d280 0x7f44dc19d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:40:46.095 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.096+0000 7f44daa00700 1 -- 172.21.15.153:0/608277050 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44cc0095f0 con 0x7f44dc1071d0 2024-03-21T10:40:46.095 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.097+0000 7f44daa00700 1 --2- 172.21.15.153:0/608277050 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f44dc1071d0 0x7f44dc19ccd0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f44cc006010 tx=0x7f44cc00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:40:46.095 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.097+0000 7f44d3400700 1 -- 172.21.15.153:0/608277050 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f44cc01d070 con 0x7f44dc1071d0 2024-03-21T10:40:46.095 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.097+0000 7f44d3400700 1 -- 172.21.15.153:0/608277050 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f44cc00f460 con 0x7f44dc1071d0 2024-03-21T10:40:46.095 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.097+0000 7f44e1600700 1 -- 172.21.15.153:0/608277050 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44dc1a0850 con 0x7f44dc1071d0 2024-03-21T10:40:46.096 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.097+0000 7f44d3400700 1 -- 172.21.15.153:0/608277050 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f44cc017630 con 0x7f44dc1071d0 2024-03-21T10:40:46.096 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.097+0000 7f44e1600700 1 -- 172.21.15.153:0/608277050 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44dc1a0dc0 con 0x7f44dc1071d0 2024-03-21T10:40:46.096 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.098+0000 7f44e1600700 1 -- 172.21.15.153:0/608277050 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44dc04ea90 con 0x7f44dc1071d0 2024-03-21T10:40:46.097 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.099+0000 7f44d3400700 1 -- 172.21.15.153:0/608277050 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f44cc00f5d0 con 0x7f44dc1071d0 2024-03-21T10:40:46.098 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.100+0000 7f44d3400700 1 --2- 172.21.15.153:0/608277050 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f44c006e460 0x7f44c0070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:40:46.098 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.100+0000 7f44d3400700 1 -- 172.21.15.153:0/608277050 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f44cc08fa00 con 0x7f44dc1071d0 2024-03-21T10:40:46.098 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.100+0000 7f44da000700 1 --2- 172.21.15.153:0/608277050 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f44c006e460 0x7f44c0070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:40:46.098 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.101+0000 7f44da000700 1 --2- 172.21.15.153:0/608277050 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f44c006e460 0x7f44c0070920 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f44c8005950 tx=0x7f44c80058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:40:46.102 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.105+0000 7f44d3400700 1 -- 172.21.15.153:0/608277050 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f44cc058a20 con 0x7f44dc1071d0 2024-03-21T10:40:46.331 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.333+0000 7f44e1600700 1 -- 172.21.15.153:0/608277050 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f44dc1a1150 con 0x7f44c006e460 2024-03-21T10:40:46.348 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.351+0000 7f44d3400700 1 -- 172.21.15.153:0/608277050 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f44dc1a1150 con 0x7f44c006e460 2024-03-21T10:40:46.349 INFO:teuthology.orchestra.run.smithi153.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-21T10:40:46.349 INFO:teuthology.orchestra.run.smithi153.stdout:alertmanager.smithi153 smithi153 *:9093,9094 running (29m) 5m ago 43m 15.6M - 0.25.0 c8568f914cd2 9ed3f2200b5f 2024-03-21T10:40:46.349 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi153 smithi153 running (43m) 5m ago 43m 10.4M - 18.2.1 5be31c24972a 5b61994396a6 2024-03-21T10:40:46.350 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi174 smithi174 running (42m) 5m ago 42m 10.1M - 18.2.1 5be31c24972a 4f7a5fc3ac28 2024-03-21T10:40:46.350 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi153 smithi153 running (25m) 5m ago 43m 6656k - 19.0.0-2231-g2edfee68 237e43c7a65f 32c21104d5b0 2024-03-21T10:40:46.350 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi174 smithi174 running (25m) 5m ago 41m 6660k - 19.0.0-2231-g2edfee68 237e43c7a65f a04e48967ab6 2024-03-21T10:40:46.350 INFO:teuthology.orchestra.run.smithi153.stdout:grafana.smithi153 smithi153 *:3000 running (29m) 5m ago 42m 75.2M - 9.4.12 2bacad6d85d8 ee24026ab786 2024-03-21T10:40:46.350 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.iuyluj smithi153 running (21m) 5m ago 36m 18.7M - 19.0.0-2231-g2edfee68 237e43c7a65f 87aff8a5de70 2024-03-21T10:40:46.350 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.zipckr smithi153 running (21m) 5m ago 36m 18.9M - 19.0.0-2231-g2edfee68 237e43c7a65f ccc262bd5dd3 2024-03-21T10:40:46.350 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.nhozag smithi174 running (5m) 5m ago 36m 10.9M - 19.0.0-2231-g2edfee68 237e43c7a65f 9e02aea72c96 2024-03-21T10:40:46.350 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.yyimst smithi174 running (20m) 5m ago 36m 18.5M - 19.0.0-2231-g2edfee68 237e43c7a65f fc1962f9351e 2024-03-21T10:40:46.350 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi153.tluxni smithi153 *:8443,9283,8765 running (32m) 5m ago 45m 512M - 19.0.0-2231-g2edfee68 237e43c7a65f 7ffee0e55382 2024-03-21T10:40:46.350 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi174.qpcwat smithi174 *:8443,9283,8765 running (31m) 5m ago 41m 459M - 19.0.0-2231-g2edfee68 237e43c7a65f 748968f216ab 2024-03-21T10:40:46.350 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi153 smithi153 running (27m) 5m ago 45m 70.8M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f dd8038303dc3 2024-03-21T10:40:46.350 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi174 smithi174 running (26m) 5m ago 41m 51.9M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f 29d78a788f2c 2024-03-21T10:40:46.350 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi153 smithi153 *:9100 running (30m) 5m ago 43m 8140k - 1.5.0 0da6a335fe13 bb835edc0e0b 2024-03-21T10:40:46.350 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi174 smithi174 *:9100 running (30m) 5m ago 41m 8610k - 1.5.0 0da6a335fe13 7c4ecad790fa 2024-03-21T10:40:46.351 INFO:teuthology.orchestra.run.smithi153.stdout:osd.0 smithi153 running (25m) 5m ago 40m 118M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 507900fd4791 2024-03-21T10:40:46.351 INFO:teuthology.orchestra.run.smithi153.stdout:osd.1 smithi153 running (24m) 5m ago 40m 93.0M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 0d646452e9c1 2024-03-21T10:40:46.351 INFO:teuthology.orchestra.run.smithi153.stdout:osd.2 smithi153 running (23m) 5m ago 39m 91.2M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 1326f2afdfcd 2024-03-21T10:40:46.351 INFO:teuthology.orchestra.run.smithi153.stdout:osd.3 smithi174 running (23m) 5m ago 39m 87.5M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 70aafac71ad5 2024-03-21T10:40:46.351 INFO:teuthology.orchestra.run.smithi153.stdout:osd.4 smithi174 running (22m) 5m ago 38m 95.8M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 0d746ce6e79d 2024-03-21T10:40:46.351 INFO:teuthology.orchestra.run.smithi153.stdout:osd.5 smithi174 running (21m) 5m ago 37m 90.3M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 53e5f35b8cac 2024-03-21T10:40:46.351 INFO:teuthology.orchestra.run.smithi153.stdout:prometheus.smithi153 smithi153 *:9095 running (30m) 5m ago 42m 88.1M - 2.43.0 a07b618ecd1d 7d95561e50c2 2024-03-21T10:40:46.351 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.353+0000 7f44e1600700 1 -- 172.21.15.153:0/608277050 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f44c006e460 msgr2=0x7f44c0070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:40:46.352 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.353+0000 7f44e1600700 1 --2- 172.21.15.153:0/608277050 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f44c006e460 0x7f44c0070920 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f44c8005950 tx=0x7f44c80058e0 comp rx=0 tx=0).stop 2024-03-21T10:40:46.352 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.353+0000 7f44e1600700 1 -- 172.21.15.153:0/608277050 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f44dc1071d0 msgr2=0x7f44dc19ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:40:46.352 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.353+0000 7f44e1600700 1 --2- 172.21.15.153:0/608277050 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f44dc1071d0 0x7f44dc19ccd0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f44cc006010 tx=0x7f44cc00be20 comp rx=0 tx=0).stop 2024-03-21T10:40:46.352 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.354+0000 7f44e1600700 1 -- 172.21.15.153:0/608277050 shutdown_connections 2024-03-21T10:40:46.352 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.354+0000 7f44e1600700 1 --2- 172.21.15.153:0/608277050 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f44c006e460 0x7f44c0070920 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:40:46.352 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.354+0000 7f44e1600700 1 --2- 172.21.15.153:0/608277050 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f44dc1071d0 0x7f44dc19ccd0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:40:46.352 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.354+0000 7f44e1600700 1 --2- 172.21.15.153:0/608277050 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f44dc10d280 0x7f44dc19d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:40:46.352 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.354+0000 7f44e1600700 1 -- 172.21.15.153:0/608277050 >> 172.21.15.153:0/608277050 conn(0x7f44dc0769e0 msgr2=0x7f44dc078320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:40:46.352 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.354+0000 7f44e1600700 1 -- 172.21.15.153:0/608277050 shutdown_connections 2024-03-21T10:40:46.352 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.354+0000 7f44e1600700 1 -- 172.21.15.153:0/608277050 wait complete. 2024-03-21T10:40:46.503 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.505+0000 7fe6e6800700 1 -- 172.21.15.153:0/1154247344 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6e01039b0 msgr2=0x7fe6e0107a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:40:46.503 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.505+0000 7fe6e6800700 1 --2- 172.21.15.153:0/1154247344 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6e01039b0 0x7fe6e0107a00 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc009910 tx=0x7fe6cc009c20 comp rx=0 tx=0).stop 2024-03-21T10:40:46.503 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.505+0000 7fe6e6800700 1 -- 172.21.15.153:0/1154247344 shutdown_connections 2024-03-21T10:40:46.504 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.505+0000 7fe6e6800700 1 --2- 172.21.15.153:0/1154247344 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6e01039b0 0x7fe6e0107a00 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:40:46.504 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.505+0000 7fe6e6800700 1 --2- 172.21.15.153:0/1154247344 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe6e0103000 0x7fe6e01033e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:40:46.504 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.505+0000 7fe6e6800700 1 -- 172.21.15.153:0/1154247344 >> 172.21.15.153:0/1154247344 conn(0x7fe6e00fe8d0 msgr2=0x7fe6e0100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:40:46.504 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.506+0000 7fe6e6800700 1 -- 172.21.15.153:0/1154247344 shutdown_connections 2024-03-21T10:40:46.504 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.506+0000 7fe6e6800700 1 -- 172.21.15.153:0/1154247344 wait complete. 2024-03-21T10:40:46.505 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.507+0000 7fe6e6800700 1 Processor -- start 2024-03-21T10:40:46.505 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.507+0000 7fe6e6800700 1 -- start start 2024-03-21T10:40:46.505 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.507+0000 7fe6e6800700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe6e0103000 0x7fe6e019e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:40:46.505 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.507+0000 7fe6e6800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6e01039b0 0x7fe6e019ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:40:46.505 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.507+0000 7fe6e6800700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e019f390 con 0x7fe6e01039b0 2024-03-21T10:40:46.505 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.507+0000 7fe6e6800700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e0198840 con 0x7fe6e0103000 2024-03-21T10:40:46.506 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.507+0000 7fe6dea00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6e01039b0 0x7fe6e019ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:40:46.506 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.507+0000 7fe6dea00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6e01039b0 0x7fe6e019ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:58324/0 (socket says 172.21.15.153:58324) 2024-03-21T10:40:46.506 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.508+0000 7fe6dea00700 1 -- 172.21.15.153:0/1551018820 learned_addr learned my addr 172.21.15.153:0/1551018820 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:40:46.507 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.508+0000 7fe6df400700 1 --2- 172.21.15.153:0/1551018820 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe6e0103000 0x7fe6e019e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:40:46.507 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.508+0000 7fe6dea00700 1 -- 172.21.15.153:0/1551018820 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe6e0103000 msgr2=0x7fe6e019e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:40:46.507 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.508+0000 7fe6dea00700 1 --2- 172.21.15.153:0/1551018820 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe6e0103000 0x7fe6e019e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:40:46.507 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.508+0000 7fe6dea00700 1 -- 172.21.15.153:0/1551018820 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6cc0095f0 con 0x7fe6e01039b0 2024-03-21T10:40:46.507 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.508+0000 7fe6dea00700 1 --2- 172.21.15.153:0/1551018820 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6e01039b0 0x7fe6e019ed00 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc00bda0 tx=0x7fe6cc00be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:40:46.507 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.508+0000 7fe6d7e00700 1 -- 172.21.15.153:0/1551018820 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe6cc01d070 con 0x7fe6e01039b0 2024-03-21T10:40:46.507 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.508+0000 7fe6d7e00700 1 -- 172.21.15.153:0/1551018820 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe6cc00f460 con 0x7fe6e01039b0 2024-03-21T10:40:46.507 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.508+0000 7fe6e6800700 1 -- 172.21.15.153:0/1551018820 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6e0198ac0 con 0x7fe6e01039b0 2024-03-21T10:40:46.507 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.509+0000 7fe6d7e00700 1 -- 172.21.15.153:0/1551018820 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe6cc017610 con 0x7fe6e01039b0 2024-03-21T10:40:46.507 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.509+0000 7fe6e6800700 1 -- 172.21.15.153:0/1551018820 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6e0198fb0 con 0x7fe6e01039b0 2024-03-21T10:40:46.508 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.510+0000 7fe6e6800700 1 -- 172.21.15.153:0/1551018820 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6e010b400 con 0x7fe6e01039b0 2024-03-21T10:40:46.508 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.511+0000 7fe6d7e00700 1 -- 172.21.15.153:0/1551018820 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fe6cc00fa40 con 0x7fe6e01039b0 2024-03-21T10:40:46.509 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.512+0000 7fe6d7e00700 1 --2- 172.21.15.153:0/1551018820 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe6c406e460 0x7fe6c4070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:40:46.509 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.512+0000 7fe6d7e00700 1 -- 172.21.15.153:0/1551018820 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fe6cc08f920 con 0x7fe6e01039b0 2024-03-21T10:40:46.509 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.512+0000 7fe6df400700 1 --2- 172.21.15.153:0/1551018820 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe6c406e460 0x7fe6c4070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:40:46.510 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.513+0000 7fe6df400700 1 --2- 172.21.15.153:0/1551018820 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe6c406e460 0x7fe6c4070920 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fe6d0005950 tx=0x7fe6d00058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:40:46.514 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.517+0000 7fe6d7e00700 1 -- 172.21.15.153:0/1551018820 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe6cc05c4f0 con 0x7fe6e01039b0 2024-03-21T10:40:46.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:40:46 smithi174 ceph-mon[88004]: from='client.35546 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:40:46.698 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:40:46 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:40:46.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:40:46 smithi153 ceph-mon[113733]: from='client.35546 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:40:46.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:40:46 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:40:46.824 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.826+0000 7fe6e6800700 1 -- 172.21.15.153:0/1551018820 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fe6e004ea90 con 0x7fe6e01039b0 2024-03-21T10:40:46.824 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.827+0000 7fe6d7e00700 1 -- 172.21.15.153:0/1551018820 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fe6cc05c080 con 0x7fe6e01039b0 2024-03-21T10:40:46.825 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:40:46.825 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": { 2024-03-21T10:40:46.825 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:40:46.825 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:40:46.825 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": { 2024-03-21T10:40:46.825 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:40:46.825 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:40:46.825 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": { 2024-03-21T10:40:46.825 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 6 2024-03-21T10:40:46.825 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:40:46.825 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": { 2024-03-21T10:40:46.825 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 4 2024-03-21T10:40:46.826 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:40:46.826 INFO:teuthology.orchestra.run.smithi153.stdout: "overall": { 2024-03-21T10:40:46.826 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 14 2024-03-21T10:40:46.826 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-03-21T10:40:46.826 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:40:46.827 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.830+0000 7fe6e6800700 1 -- 172.21.15.153:0/1551018820 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe6c406e460 msgr2=0x7fe6c4070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:40:46.827 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.830+0000 7fe6e6800700 1 --2- 172.21.15.153:0/1551018820 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe6c406e460 0x7fe6c4070920 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fe6d0005950 tx=0x7fe6d00058e0 comp rx=0 tx=0).stop 2024-03-21T10:40:46.827 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.830+0000 7fe6e6800700 1 -- 172.21.15.153:0/1551018820 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6e01039b0 msgr2=0x7fe6e019ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:40:46.828 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.830+0000 7fe6e6800700 1 --2- 172.21.15.153:0/1551018820 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6e01039b0 0x7fe6e019ed00 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc00bda0 tx=0x7fe6cc00be80 comp rx=0 tx=0).stop 2024-03-21T10:40:46.828 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.830+0000 7fe6e6800700 1 -- 172.21.15.153:0/1551018820 shutdown_connections 2024-03-21T10:40:46.828 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.830+0000 7fe6e6800700 1 --2- 172.21.15.153:0/1551018820 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe6e0103000 0x7fe6e019e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:40:46.828 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.830+0000 7fe6e6800700 1 --2- 172.21.15.153:0/1551018820 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe6c406e460 0x7fe6c4070920 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:40:46.828 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.830+0000 7fe6e6800700 1 --2- 172.21.15.153:0/1551018820 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6e01039b0 0x7fe6e019ed00 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:40:46.828 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.830+0000 7fe6e6800700 1 -- 172.21.15.153:0/1551018820 >> 172.21.15.153:0/1551018820 conn(0x7fe6e00fe8d0 msgr2=0x7fe6e0100cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:40:46.828 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.830+0000 7fe6e6800700 1 -- 172.21.15.153:0/1551018820 shutdown_connections 2024-03-21T10:40:46.829 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.830+0000 7fe6e6800700 1 -- 172.21.15.153:0/1551018820 wait complete. 2024-03-21T10:40:46.972 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.974+0000 7f2d87c00700 1 -- 172.21.15.153:0/69736800 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2d800739d0 msgr2=0x7f2d8010cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:40:46.973 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.974+0000 7f2d87c00700 1 --2- 172.21.15.153:0/69736800 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2d800739d0 0x7f2d8010cee0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c009910 tx=0x7f2d6c009c20 comp rx=0 tx=0).stop 2024-03-21T10:40:46.973 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.974+0000 7f2d87c00700 1 -- 172.21.15.153:0/69736800 shutdown_connections 2024-03-21T10:40:46.973 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.974+0000 7f2d87c00700 1 --2- 172.21.15.153:0/69736800 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2d800739d0 0x7f2d8010cee0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:40:46.973 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.974+0000 7f2d87c00700 1 --2- 172.21.15.153:0/69736800 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d800730b0 0x7f2d80073490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:40:46.973 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.974+0000 7f2d87c00700 1 -- 172.21.15.153:0/69736800 >> 172.21.15.153:0/69736800 conn(0x7f2d800fc650 msgr2=0x7f2d800fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:40:46.973 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.975+0000 7f2d87c00700 1 -- 172.21.15.153:0/69736800 shutdown_connections 2024-03-21T10:40:46.973 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.975+0000 7f2d87c00700 1 -- 172.21.15.153:0/69736800 wait complete. 2024-03-21T10:40:46.974 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.976+0000 7f2d87c00700 1 Processor -- start 2024-03-21T10:40:46.974 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.976+0000 7f2d87c00700 1 -- start start 2024-03-21T10:40:46.975 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.976+0000 7f2d87c00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2d800730b0 0x7f2d8019e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:40:46.975 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.976+0000 7f2d87c00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d800739d0 0x7f2d8019ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:40:46.975 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.976+0000 7f2d87c00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8019f2e0 con 0x7f2d800730b0 2024-03-21T10:40:46.975 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.976+0000 7f2d87c00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d80198790 con 0x7f2d800739d0 2024-03-21T10:40:46.975 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.976+0000 7f2d85200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2d800730b0 0x7f2d8019e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:40:46.975 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.976+0000 7f2d85200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2d800730b0 0x7f2d8019e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:58340/0 (socket says 172.21.15.153:58340) 2024-03-21T10:40:46.975 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.976+0000 7f2d85200700 1 -- 172.21.15.153:0/3124210722 learned_addr learned my addr 172.21.15.153:0/3124210722 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:40:46.975 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.977+0000 7f2d7fe00700 1 --2- 172.21.15.153:0/3124210722 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d800739d0 0x7f2d8019ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:40:46.975 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.977+0000 7f2d85200700 1 -- 172.21.15.153:0/3124210722 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d800739d0 msgr2=0x7f2d8019ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:40:46.975 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.977+0000 7f2d85200700 1 --2- 172.21.15.153:0/3124210722 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d800739d0 0x7f2d8019ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:40:46.975 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.977+0000 7f2d85200700 1 -- 172.21.15.153:0/3124210722 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d6c0095f0 con 0x7f2d800730b0 2024-03-21T10:40:46.975 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.977+0000 7f2d85200700 1 --2- 172.21.15.153:0/3124210722 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2d800730b0 0x7f2d8019e710 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f2d7400c8a0 tx=0x7f2d7400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:40:46.976 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.977+0000 7f2d7d600700 1 -- 172.21.15.153:0/3124210722 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2d7400cea0 con 0x7f2d800730b0 2024-03-21T10:40:46.976 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.977+0000 7f2d7d600700 1 -- 172.21.15.153:0/3124210722 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2d74004830 con 0x7f2d800730b0 2024-03-21T10:40:46.976 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.977+0000 7f2d7d600700 1 -- 172.21.15.153:0/3124210722 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2d740055b0 con 0x7f2d800730b0 2024-03-21T10:40:46.976 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.977+0000 7f2d87c00700 1 -- 172.21.15.153:0/3124210722 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d80198a70 con 0x7f2d800730b0 2024-03-21T10:40:46.976 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.978+0000 7f2d87c00700 1 -- 172.21.15.153:0/3124210722 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d80198f70 con 0x7f2d800730b0 2024-03-21T10:40:46.976 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.979+0000 7f2d87c00700 1 -- 172.21.15.153:0/3124210722 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d8010a5e0 con 0x7f2d800730b0 2024-03-21T10:40:46.977 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.980+0000 7f2d7d600700 1 -- 172.21.15.153:0/3124210722 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f2d740049a0 con 0x7f2d800730b0 2024-03-21T10:40:46.978 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.980+0000 7f2d7d600700 1 --2- 172.21.15.153:0/3124210722 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f2d6806e460 0x7f2d68070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:40:46.978 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.981+0000 7f2d7d600700 1 -- 172.21.15.153:0/3124210722 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f2d7408e1d0 con 0x7f2d800730b0 2024-03-21T10:40:46.978 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.981+0000 7f2d7fe00700 1 --2- 172.21.15.153:0/3124210722 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f2d6806e460 0x7f2d68070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:40:46.979 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.981+0000 7f2d7fe00700 1 --2- 172.21.15.153:0/3124210722 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f2d6806e460 0x7f2d68070920 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c000c00 tx=0x7f2d6c005fd0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:40:46.983 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:46.985+0000 7f2d7d600700 1 -- 172.21.15.153:0/3124210722 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2d7405adc0 con 0x7f2d800730b0 2024-03-21T10:40:47.253 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:47.255+0000 7f2d87c00700 1 -- 172.21.15.153:0/3124210722 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f2d8004ea90 con 0x7f2d800730b0 2024-03-21T10:40:47.254 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:47.256+0000 7f2d7d600700 1 -- 172.21.15.153:0/3124210722 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1876 (secure 0 0 0) 0x7f2d74004c50 con 0x7f2d800730b0 2024-03-21T10:40:47.255 INFO:teuthology.orchestra.run.smithi153.stdout:e32 2024-03-21T10:40:47.255 INFO:teuthology.orchestra.run.smithi153.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-21T10:40:47.255 INFO:teuthology.orchestra.run.smithi153.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:40:47.256 INFO:teuthology.orchestra.run.smithi153.stdout:legacy client fscid: 1 2024-03-21T10:40:47.256 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:40:47.256 INFO:teuthology.orchestra.run.smithi153.stdout:Filesystem 'cephfs' (1) 2024-03-21T10:40:47.256 INFO:teuthology.orchestra.run.smithi153.stdout:fs_name cephfs 2024-03-21T10:40:47.256 INFO:teuthology.orchestra.run.smithi153.stdout:epoch 23 2024-03-21T10:40:47.256 INFO:teuthology.orchestra.run.smithi153.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-21T10:40:47.256 INFO:teuthology.orchestra.run.smithi153.stdout:created 2024-03-21T10:04:27.086576+0000 2024-03-21T10:40:47.256 INFO:teuthology.orchestra.run.smithi153.stdout:modified 2024-03-21T10:19:05.022092+0000 2024-03-21T10:40:47.256 INFO:teuthology.orchestra.run.smithi153.stdout:tableserver 0 2024-03-21T10:40:47.256 INFO:teuthology.orchestra.run.smithi153.stdout:root 0 2024-03-21T10:40:47.256 INFO:teuthology.orchestra.run.smithi153.stdout:session_timeout 60 2024-03-21T10:40:47.256 INFO:teuthology.orchestra.run.smithi153.stdout:session_autoclose 300 2024-03-21T10:40:47.256 INFO:teuthology.orchestra.run.smithi153.stdout:max_file_size 1099511627776 2024-03-21T10:40:47.256 INFO:teuthology.orchestra.run.smithi153.stdout:max_xattr_size 65536 2024-03-21T10:40:47.256 INFO:teuthology.orchestra.run.smithi153.stdout:required_client_features {} 2024-03-21T10:40:47.257 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure 0 2024-03-21T10:40:47.257 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure_osd_epoch 94 2024-03-21T10:40:47.257 INFO:teuthology.orchestra.run.smithi153.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:40:47.257 INFO:teuthology.orchestra.run.smithi153.stdout:max_mds 2 2024-03-21T10:40:47.257 INFO:teuthology.orchestra.run.smithi153.stdout:in 0,1 2024-03-21T10:40:47.257 INFO:teuthology.orchestra.run.smithi153.stdout:up {} 2024-03-21T10:40:47.257 INFO:teuthology.orchestra.run.smithi153.stdout:failed 0,1 2024-03-21T10:40:47.257 INFO:teuthology.orchestra.run.smithi153.stdout:damaged 2024-03-21T10:40:47.257 INFO:teuthology.orchestra.run.smithi153.stdout:stopped 2024-03-21T10:40:47.257 INFO:teuthology.orchestra.run.smithi153.stdout:data_pools [3] 2024-03-21T10:40:47.257 INFO:teuthology.orchestra.run.smithi153.stdout:metadata_pool 2 2024-03-21T10:40:47.257 INFO:teuthology.orchestra.run.smithi153.stdout:inline_data enabled 2024-03-21T10:40:47.257 INFO:teuthology.orchestra.run.smithi153.stdout:balancer 2024-03-21T10:40:47.258 INFO:teuthology.orchestra.run.smithi153.stdout:bal_rank_mask -1 2024-03-21T10:40:47.258 INFO:teuthology.orchestra.run.smithi153.stdout:standby_count_wanted 1 2024-03-21T10:40:47.258 INFO:teuthology.orchestra.run.smithi153.stdout:qdb_cluster leader: 0 members: 2024-03-21T10:40:47.258 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:40:47.258 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:40:47.258 INFO:teuthology.orchestra.run.smithi153.stdout:Standby daemons: 2024-03-21T10:40:47.258 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:40:47.258 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.zipckr{-1:34442} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6826/1103846295,v1:172.21.15.153:6827/1103846295] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:40:47.258 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.iuyluj{-1:34474} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6828/1738630832,v1:172.21.15.153:6829/1738630832] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:40:47.258 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.yyimst{-1:44313} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.174:6824/4127786797,v1:172.21.15.174:6825/4127786797] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:40:47.258 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.nhozag{-1:44687} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.174:6826/2767517845,v1:172.21.15.174:6827/2767517845] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:40:47.259 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:47.260+0000 7f2d87c00700 1 -- 172.21.15.153:0/3124210722 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f2d6806e460 msgr2=0x7f2d68070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:40:47.259 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:47.260+0000 7f2d87c00700 1 --2- 172.21.15.153:0/3124210722 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f2d6806e460 0x7f2d68070920 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c000c00 tx=0x7f2d6c005fd0 comp rx=0 tx=0).stop 2024-03-21T10:40:47.259 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:47.260+0000 7f2d87c00700 1 -- 172.21.15.153:0/3124210722 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2d800730b0 msgr2=0x7f2d8019e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:40:47.259 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:47.260+0000 7f2d87c00700 1 --2- 172.21.15.153:0/3124210722 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2d800730b0 0x7f2d8019e710 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f2d7400c8a0 tx=0x7f2d7400cc60 comp rx=0 tx=0).stop 2024-03-21T10:40:47.259 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:47.260+0000 7f2d87c00700 1 -- 172.21.15.153:0/3124210722 shutdown_connections 2024-03-21T10:40:47.259 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:47.260+0000 7f2d87c00700 1 --2- 172.21.15.153:0/3124210722 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f2d6806e460 0x7f2d68070920 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:40:47.259 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:47.260+0000 7f2d87c00700 1 --2- 172.21.15.153:0/3124210722 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2d800730b0 0x7f2d8019e710 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:40:47.260 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:47.260+0000 7f2d87c00700 1 --2- 172.21.15.153:0/3124210722 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d800739d0 0x7f2d8019ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:40:47.260 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:47.260+0000 7f2d87c00700 1 -- 172.21.15.153:0/3124210722 >> 172.21.15.153:0/3124210722 conn(0x7f2d800fc650 msgr2=0x7f2d80107720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:40:47.260 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:47.260+0000 7f2d87c00700 1 -- 172.21.15.153:0/3124210722 shutdown_connections 2024-03-21T10:40:47.260 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:47.260+0000 7f2d87c00700 1 -- 172.21.15.153:0/3124210722 wait complete. 2024-03-21T10:40:47.260 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 32 2024-03-21T10:40:47.410 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:47.412+0000 7f6813200700 1 -- 172.21.15.153:0/140179506 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6814075be0 msgr2=0x7f6814111310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:40:47.410 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:47.412+0000 7f6813200700 1 --2- 172.21.15.153:0/140179506 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6814075be0 0x7f6814111310 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f6800009910 tx=0x7f6800009c20 comp rx=0 tx=0).stop 2024-03-21T10:40:47.410 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:47.412+0000 7f6813200700 1 -- 172.21.15.153:0/140179506 shutdown_connections 2024-03-21T10:40:47.410 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:47.412+0000 7f6813200700 1 --2- 172.21.15.153:0/140179506 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6814075be0 0x7f6814111310 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:40:47.410 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:47.412+0000 7f6813200700 1 --2- 172.21.15.153:0/140179506 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f68140752c0 0x7f68140756a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:40:47.410 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:47.412+0000 7f6813200700 1 -- 172.21.15.153:0/140179506 >> 172.21.15.153:0/140179506 conn(0x7f68140fe660 msgr2=0x7f6814100a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:40:47.410 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:47.412+0000 7f6813200700 1 -- 172.21.15.153:0/140179506 shutdown_connections 2024-03-21T10:40:47.410 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:47.412+0000 7f6813200700 1 -- 172.21.15.153:0/140179506 wait complete. 2024-03-21T10:40:47.411 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:47.413+0000 7f6813200700 1 Processor -- start 2024-03-21T10:40:47.411 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:47.413+0000 7f6813200700 1 -- start start 2024-03-21T10:40:47.411 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:47.414+0000 7f6813200700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f68140752c0 0x7f68141a2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:40:47.411 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:47.414+0000 7f6813200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6814075be0 0x7f68141a2e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:40:47.412 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:47.414+0000 7f6813200700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68141a3520 con 0x7f6814075be0 2024-03-21T10:40:47.412 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:47.414+0000 7f6813200700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f681419c9d0 con 0x7f68140752c0 2024-03-21T10:40:47.413 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:47.414+0000 7f6811400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6814075be0 0x7f68141a2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:40:47.413 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:47.414+0000 7f6811400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6814075be0 0x7f68141a2e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:58356/0 (socket says 172.21.15.153:58356) 2024-03-21T10:40:47.413 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:47.414+0000 7f6811400700 1 -- 172.21.15.153:0/2663346538 learned_addr learned my addr 172.21.15.153:0/2663346538 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:40:47.413 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:47.414+0000 7f6811e00700 1 --2- 172.21.15.153:0/2663346538 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f68140752c0 0x7f68141a2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:40:47.413 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:47.414+0000 7f6811400700 1 -- 172.21.15.153:0/2663346538 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f68140752c0 msgr2=0x7f68141a2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:40:47.413 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:47.414+0000 7f6811400700 1 --2- 172.21.15.153:0/2663346538 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f68140752c0 0x7f68141a2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:40:47.413 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:47.414+0000 7f6811400700 1 -- 172.21.15.153:0/2663346538 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68000095f0 con 0x7f6814075be0 2024-03-21T10:40:47.413 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:47.415+0000 7f6811400700 1 --2- 172.21.15.153:0/2663346538 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6814075be0 0x7f68141a2e90 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f68000092e0 tx=0x7f68000056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:40:47.413 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:47.415+0000 7f680aa00700 1 -- 172.21.15.153:0/2663346538 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f680001d070 con 0x7f6814075be0 2024-03-21T10:40:47.414 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:47.415+0000 7f680aa00700 1 -- 172.21.15.153:0/2663346538 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f680000f460 con 0x7f6814075be0 2024-03-21T10:40:47.414 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:47.415+0000 7f6813200700 1 -- 172.21.15.153:0/2663346538 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f681419ccb0 con 0x7f6814075be0 2024-03-21T10:40:47.414 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:47.415+0000 7f680aa00700 1 -- 172.21.15.153:0/2663346538 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6800021640 con 0x7f6814075be0 2024-03-21T10:40:47.414 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:47.415+0000 7f6813200700 1 -- 172.21.15.153:0/2663346538 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f681419d200 con 0x7f6814075be0 2024-03-21T10:40:47.414 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:47.416+0000 7f6813200700 1 -- 172.21.15.153:0/2663346538 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f681410ea80 con 0x7f6814075be0 2024-03-21T10:40:47.415 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:47.417+0000 7f680aa00700 1 -- 172.21.15.153:0/2663346538 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f680000f5d0 con 0x7f6814075be0 2024-03-21T10:40:47.416 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:47.418+0000 7f680aa00700 1 --2- 172.21.15.153:0/2663346538 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f67f806e460 0x7f67f8070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:40:47.416 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:47.418+0000 7f680aa00700 1 -- 172.21.15.153:0/2663346538 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f680008f7f0 con 0x7f6814075be0 2024-03-21T10:40:47.416 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:47.418+0000 7f6811e00700 1 --2- 172.21.15.153:0/2663346538 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f67f806e460 0x7f67f8070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:40:47.416 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:47.419+0000 7f6811e00700 1 --2- 172.21.15.153:0/2663346538 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f67f806e460 0x7f67f8070920 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f6804005950 tx=0x7f68040058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:40:47.420 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:47.423+0000 7f680aa00700 1 -- 172.21.15.153:0/2663346538 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f680005c340 con 0x7f6814075be0 2024-03-21T10:40:47.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:40:47 smithi153 ceph-mon[113733]: pgmap v869: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:40:47.553 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:40:47 smithi153 ceph-mon[113733]: from='client.35550 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:40:47.553 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:40:47 smithi153 ceph-mon[113733]: from='client.35554 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:40:47.553 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:40:47 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/1551018820' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:40:47.553 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:40:47 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/3124210722' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:40:47.660 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:47.662+0000 7f6813200700 1 -- 172.21.15.153:0/2663346538 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f681419d480 con 0x7f67f806e460 2024-03-21T10:40:47.663 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:47.665+0000 7f680aa00700 1 -- 172.21.15.153:0/2663346538 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f681419d480 con 0x7f67f806e460 2024-03-21T10:40:47.663 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:40:47.663 INFO:teuthology.orchestra.run.smithi153.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232", 2024-03-21T10:40:47.664 INFO:teuthology.orchestra.run.smithi153.stdout: "in_progress": true, 2024-03-21T10:40:47.664 INFO:teuthology.orchestra.run.smithi153.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-21T10:40:47.664 INFO:teuthology.orchestra.run.smithi153.stdout: "services_complete": [ 2024-03-21T10:40:47.664 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr", 2024-03-21T10:40:47.664 INFO:teuthology.orchestra.run.smithi153.stdout: "mon", 2024-03-21T10:40:47.664 INFO:teuthology.orchestra.run.smithi153.stdout: "mds", 2024-03-21T10:40:47.664 INFO:teuthology.orchestra.run.smithi153.stdout: "crash", 2024-03-21T10:40:47.664 INFO:teuthology.orchestra.run.smithi153.stdout: "osd" 2024-03-21T10:40:47.664 INFO:teuthology.orchestra.run.smithi153.stdout: ], 2024-03-21T10:40:47.664 INFO:teuthology.orchestra.run.smithi153.stdout: "progress": "16/23 daemons upgraded", 2024-03-21T10:40:47.664 INFO:teuthology.orchestra.run.smithi153.stdout: "message": "Currently upgrading mds daemons", 2024-03-21T10:40:47.664 INFO:teuthology.orchestra.run.smithi153.stdout: "is_paused": false 2024-03-21T10:40:47.664 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:40:47.667 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:47.669+0000 7f6813200700 1 -- 172.21.15.153:0/2663346538 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f67f806e460 msgr2=0x7f67f8070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:40:47.667 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:47.669+0000 7f6813200700 1 --2- 172.21.15.153:0/2663346538 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f67f806e460 0x7f67f8070920 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f6804005950 tx=0x7f68040058e0 comp rx=0 tx=0).stop 2024-03-21T10:40:47.667 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:47.669+0000 7f6813200700 1 -- 172.21.15.153:0/2663346538 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6814075be0 msgr2=0x7f68141a2e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:40:47.667 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:47.669+0000 7f6813200700 1 --2- 172.21.15.153:0/2663346538 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6814075be0 0x7f68141a2e90 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f68000092e0 tx=0x7f68000056a0 comp rx=0 tx=0).stop 2024-03-21T10:40:47.667 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:47.669+0000 7f6813200700 1 -- 172.21.15.153:0/2663346538 shutdown_connections 2024-03-21T10:40:47.667 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:47.669+0000 7f6813200700 1 --2- 172.21.15.153:0/2663346538 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f68140752c0 0x7f68141a2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:40:47.667 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:47.669+0000 7f6813200700 1 --2- 172.21.15.153:0/2663346538 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f67f806e460 0x7f67f8070920 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:40:47.667 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:47.669+0000 7f6813200700 1 --2- 172.21.15.153:0/2663346538 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6814075be0 0x7f68141a2e90 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:40:47.668 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:47.669+0000 7f6813200700 1 -- 172.21.15.153:0/2663346538 >> 172.21.15.153:0/2663346538 conn(0x7f68140fe660 msgr2=0x7f68140ff2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:40:47.668 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:47.669+0000 7f6813200700 1 -- 172.21.15.153:0/2663346538 shutdown_connections 2024-03-21T10:40:47.668 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:47.669+0000 7f6813200700 1 -- 172.21.15.153:0/2663346538 wait complete. 2024-03-21T10:40:47.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:40:47 smithi174 ceph-mon[88004]: pgmap v869: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:40:47.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:40:47 smithi174 ceph-mon[88004]: from='client.35550 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:40:47.698 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:40:47 smithi174 ceph-mon[88004]: from='client.35554 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:40:47.698 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:40:47 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/1551018820' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:40:47.698 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:40:47 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/3124210722' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:40:47.818 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:47.820+0000 7fb924200700 1 -- 172.21.15.153:0/4263175128 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb91c073a80 msgr2=0x7fb91c10cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:40:47.818 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:47.820+0000 7fb924200700 1 --2- 172.21.15.153:0/4263175128 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb91c073a80 0x7fb91c10cec0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fb908009910 tx=0x7fb908009c20 comp rx=0 tx=0).stop 2024-03-21T10:40:47.818 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:47.820+0000 7fb924200700 1 -- 172.21.15.153:0/4263175128 shutdown_connections 2024-03-21T10:40:47.818 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:47.820+0000 7fb924200700 1 --2- 172.21.15.153:0/4263175128 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb91c073a80 0x7fb91c10cec0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:40:47.818 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:47.820+0000 7fb924200700 1 --2- 172.21.15.153:0/4263175128 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb91c073160 0x7fb91c073540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:40:47.818 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:47.820+0000 7fb924200700 1 -- 172.21.15.153:0/4263175128 >> 172.21.15.153:0/4263175128 conn(0x7fb91c0fc650 msgr2=0x7fb91c0fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:40:47.818 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:47.820+0000 7fb924200700 1 -- 172.21.15.153:0/4263175128 shutdown_connections 2024-03-21T10:40:47.818 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:47.820+0000 7fb924200700 1 -- 172.21.15.153:0/4263175128 wait complete. 2024-03-21T10:40:47.819 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:47.821+0000 7fb924200700 1 Processor -- start 2024-03-21T10:40:47.819 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:47.821+0000 7fb924200700 1 -- start start 2024-03-21T10:40:47.819 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:47.821+0000 7fb924200700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb91c073160 0x7fb91c198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:40:47.819 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:47.821+0000 7fb924200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb91c073a80 0x7fb91c198eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:40:47.819 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:47.821+0000 7fb924200700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb91c199540 con 0x7fb91c073a80 2024-03-21T10:40:47.820 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:47.821+0000 7fb924200700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb91c19d350 con 0x7fb91c073160 2024-03-21T10:40:47.820 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:47.822+0000 7fb920e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb91c073a80 0x7fb91c198eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:40:47.820 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:47.822+0000 7fb920e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb91c073a80 0x7fb91c198eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:58378/0 (socket says 172.21.15.153:58378) 2024-03-21T10:40:47.820 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:47.822+0000 7fb920e00700 1 -- 172.21.15.153:0/1787162067 learned_addr learned my addr 172.21.15.153:0/1787162067 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:40:47.821 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:47.822+0000 7fb921800700 1 --2- 172.21.15.153:0/1787162067 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb91c073160 0x7fb91c198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:40:47.821 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:47.822+0000 7fb920e00700 1 -- 172.21.15.153:0/1787162067 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb91c073160 msgr2=0x7fb91c198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:40:47.821 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:47.822+0000 7fb920e00700 1 --2- 172.21.15.153:0/1787162067 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb91c073160 0x7fb91c198970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:40:47.821 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:47.822+0000 7fb920e00700 1 -- 172.21.15.153:0/1787162067 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9080095f0 con 0x7fb91c073a80 2024-03-21T10:40:47.821 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:47.822+0000 7fb920e00700 1 --2- 172.21.15.153:0/1787162067 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb91c073a80 0x7fb91c198eb0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fb90800bd70 tx=0x7fb90800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:40:47.821 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:47.823+0000 7fb91a000700 1 -- 172.21.15.153:0/1787162067 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb90801d070 con 0x7fb91c073a80 2024-03-21T10:40:47.821 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:47.823+0000 7fb91a000700 1 -- 172.21.15.153:0/1787162067 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb90800f460 con 0x7fb91c073a80 2024-03-21T10:40:47.821 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:47.823+0000 7fb91a000700 1 -- 172.21.15.153:0/1787162067 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb908017630 con 0x7fb91c073a80 2024-03-21T10:40:47.821 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:47.823+0000 7fb924200700 1 -- 172.21.15.153:0/1787162067 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb91c19d5d0 con 0x7fb91c073a80 2024-03-21T10:40:47.821 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:47.823+0000 7fb924200700 1 -- 172.21.15.153:0/1787162067 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb91c19dac0 con 0x7fb91c073a80 2024-03-21T10:40:47.822 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:47.824+0000 7fb924200700 1 -- 172.21.15.153:0/1787162067 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb91c10a5c0 con 0x7fb91c073a80 2024-03-21T10:40:47.823 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:47.825+0000 7fb91a000700 1 -- 172.21.15.153:0/1787162067 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fb908003a60 con 0x7fb91c073a80 2024-03-21T10:40:47.823 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:47.826+0000 7fb91a000700 1 --2- 172.21.15.153:0/1787162067 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb90406e460 0x7fb904070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:40:47.823 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:47.826+0000 7fb91a000700 1 -- 172.21.15.153:0/1787162067 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fb90808faa0 con 0x7fb91c073a80 2024-03-21T10:40:47.824 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:47.826+0000 7fb921800700 1 --2- 172.21.15.153:0/1787162067 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb90406e460 0x7fb904070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:40:47.824 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:47.827+0000 7fb921800700 1 --2- 172.21.15.153:0/1787162067 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb90406e460 0x7fb904070920 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fb910005950 tx=0x7fb9100058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:40:47.828 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:47.830+0000 7fb91a000700 1 -- 172.21.15.153:0/1787162067 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb90805c6b0 con 0x7fb91c073a80 2024-03-21T10:40:48.117 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:48.119+0000 7fb924200700 1 -- 172.21.15.153:0/1787162067 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fb91c04ea90 con 0x7fb91c073a80 2024-03-21T10:40:48.117 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:48.119+0000 7fb91a000700 1 -- 172.21.15.153:0/1787162067 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4493 (secure 0 0 0) 0x7fb90805c240 con 0x7fb91c073a80 2024-03-21T10:40:48.118 INFO:teuthology.orchestra.run.smithi153.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-03-21T10:40:48.118 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-03-21T10:40:48.118 INFO:teuthology.orchestra.run.smithi153.stdout: host smithi174 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi174/config 2024-03-21T10:40:48.118 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/podman: timeout after 895 seconds 2024-03-21T10:40:48.118 INFO:teuthology.orchestra.run.smithi153.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 -e NODE_NAME=smithi174 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/run/ceph:z -v /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpd910r3_i:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 inventory --format=json-pretty --filter-for-batch 2024-03-21T10:40:48.118 INFO:teuthology.orchestra.run.smithi153.stdout:Traceback (most recent call last): 2024-03-21T10:40:48.118 INFO:teuthology.orchestra.run.smithi153.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-03-21T10:40:48.118 INFO:teuthology.orchestra.run.smithi153.stdout: return _run_code(code, main_globals, None, 2024-03-21T10:40:48.118 INFO:teuthology.orchestra.run.smithi153.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-03-21T10:40:48.118 INFO:teuthology.orchestra.run.smithi153.stdout: exec(code, run_globals) 2024-03-21T10:40:48.118 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 5606, in 2024-03-21T10:40:48.118 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 5594, in main 2024-03-21T10:40:48.119 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 406, in _infer_config 2024-03-21T10:40:48.119 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 321, in _infer_fsid 2024-03-21T10:40:48.119 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 434, in _infer_image 2024-03-21T10:40:48.119 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 308, in _validate_fsid 2024-03-21T10:40:48.119 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 3357, in command_ceph_volume 2024-03-21T10:40:48.119 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-03-21T10:40:48.119 INFO:teuthology.orchestra.run.smithi153.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 -e NODE_NAME=smithi174 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/run/ceph:z -v /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpd910r3_i:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 inventory --format=json-pretty --filter-for-batch 2024-03-21T10:40:48.119 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-21T10:40:48.119 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is degraded 2024-03-21T10:40:48.119 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-21T10:40:48.119 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-21T10:40:48.119 INFO:teuthology.orchestra.run.smithi153.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-21T10:40:48.119 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-21T10:40:48.120 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:48.122+0000 7fb924200700 1 -- 172.21.15.153:0/1787162067 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb90406e460 msgr2=0x7fb904070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:40:48.120 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:48.122+0000 7fb924200700 1 --2- 172.21.15.153:0/1787162067 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb90406e460 0x7fb904070920 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fb910005950 tx=0x7fb9100058e0 comp rx=0 tx=0).stop 2024-03-21T10:40:48.120 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:48.122+0000 7fb924200700 1 -- 172.21.15.153:0/1787162067 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb91c073a80 msgr2=0x7fb91c198eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:40:48.120 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:48.122+0000 7fb924200700 1 --2- 172.21.15.153:0/1787162067 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb91c073a80 0x7fb91c198eb0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fb90800bd70 tx=0x7fb90800be50 comp rx=0 tx=0).stop 2024-03-21T10:40:48.120 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:48.122+0000 7fb924200700 1 -- 172.21.15.153:0/1787162067 shutdown_connections 2024-03-21T10:40:48.120 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:48.122+0000 7fb924200700 1 --2- 172.21.15.153:0/1787162067 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb91c073160 0x7fb91c198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:40:48.121 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:48.122+0000 7fb924200700 1 --2- 172.21.15.153:0/1787162067 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb90406e460 0x7fb904070920 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:40:48.121 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:48.122+0000 7fb924200700 1 --2- 172.21.15.153:0/1787162067 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb91c073a80 0x7fb91c198eb0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:40:48.121 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:48.122+0000 7fb924200700 1 -- 172.21.15.153:0/1787162067 >> 172.21.15.153:0/1787162067 conn(0x7fb91c0fc650 msgr2=0x7fb91c107700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:40:48.121 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:48.122+0000 7fb924200700 1 -- 172.21.15.153:0/1787162067 shutdown_connections 2024-03-21T10:40:48.121 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:40:48.123+0000 7fb924200700 1 -- 172.21.15.153:0/1787162067 wait complete. 2024-03-21T10:40:48.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:40:48 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/1787162067' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:40:48.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:40:48 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/1787162067' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:40:49.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:40:49 smithi153 ceph-mon[113733]: from='client.35566 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:40:49.803 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:40:49 smithi153 ceph-mon[113733]: pgmap v870: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:40:49.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:40:49 smithi174 ceph-mon[88004]: from='client.35566 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:40:49.948 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:40:49 smithi174 ceph-mon[88004]: pgmap v870: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:40:51.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:40:51 smithi153 ceph-mon[113733]: pgmap v871: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:40:51.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:40:51 smithi174 ceph-mon[88004]: pgmap v871: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:40:53.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:40:53 smithi153 ceph-mon[113733]: pgmap v872: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:40:53.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:40:53 smithi174 ceph-mon[88004]: pgmap v872: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:40:55.803 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:40:55 smithi153 ceph-mon[113733]: pgmap v873: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:40:55.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:40:55 smithi174 ceph-mon[88004]: pgmap v873: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:40:57.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:40:57 smithi153 ceph-mon[113733]: pgmap v874: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:40:57.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:40:57 smithi174 ceph-mon[88004]: pgmap v874: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:40:59.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:40:59 smithi153 ceph-mon[113733]: pgmap v875: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:40:59.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:40:59 smithi174 ceph-mon[88004]: pgmap v875: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:41:01.803 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:41:01 smithi153 ceph-mon[113733]: pgmap v876: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:41:01.803 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:41:01 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:41:01.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:41:01 smithi174 ceph-mon[88004]: pgmap v876: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:41:01.948 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:41:01 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:41:03.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:41:03 smithi153 ceph-mon[113733]: pgmap v877: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:41:03.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:41:03 smithi174 ceph-mon[88004]: pgmap v877: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:41:05.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:41:05 smithi153 ceph-mon[113733]: pgmap v878: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:41:05.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:41:05 smithi174 ceph-mon[88004]: pgmap v878: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:41:07.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:41:07 smithi153 ceph-mon[113733]: pgmap v879: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:41:07.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:41:07 smithi174 ceph-mon[88004]: pgmap v879: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:41:09.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:41:09 smithi153 ceph-mon[113733]: pgmap v880: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:41:09.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:41:09 smithi174 ceph-mon[88004]: pgmap v880: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:41:11.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:41:11 smithi153 ceph-mon[113733]: pgmap v881: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:41:11.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:41:11 smithi174 ceph-mon[88004]: pgmap v881: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:41:13.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:41:13 smithi153 ceph-mon[113733]: pgmap v882: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:41:13.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:41:13 smithi174 ceph-mon[88004]: pgmap v882: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:41:15.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:41:15 smithi153 ceph-mon[113733]: pgmap v883: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:41:15.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:41:15 smithi174 ceph-mon[88004]: pgmap v883: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:41:16.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:41:16 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:41:16.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:41:16 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:41:17.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:41:17 smithi153 ceph-mon[113733]: pgmap v884: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:41:17.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:41:17 smithi174 ceph-mon[88004]: pgmap v884: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:41:18.290 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:18.293+0000 7fbe62e00700 1 -- 172.21.15.153:0/2315488713 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbe5c1071d0 msgr2=0x7fbe5c107650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:41:18.290 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:18.293+0000 7fbe62e00700 1 --2- 172.21.15.153:0/2315488713 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbe5c1071d0 0x7fbe5c107650 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fbe48009910 tx=0x7fbe48009c20 comp rx=0 tx=0).stop 2024-03-21T10:41:18.290 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:18.293+0000 7fbe62e00700 1 -- 172.21.15.153:0/2315488713 shutdown_connections 2024-03-21T10:41:18.291 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:18.293+0000 7fbe62e00700 1 --2- 172.21.15.153:0/2315488713 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbe5c1071d0 0x7fbe5c107650 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:41:18.291 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:18.293+0000 7fbe62e00700 1 --2- 172.21.15.153:0/2315488713 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbe5c10d280 0x7fbe5c10d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:41:18.291 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:18.293+0000 7fbe62e00700 1 -- 172.21.15.153:0/2315488713 >> 172.21.15.153:0/2315488713 conn(0x7fbe5c0769e0 msgr2=0x7fbe5c078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:41:18.291 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:18.293+0000 7fbe62e00700 1 -- 172.21.15.153:0/2315488713 shutdown_connections 2024-03-21T10:41:18.291 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:18.293+0000 7fbe62e00700 1 -- 172.21.15.153:0/2315488713 wait complete. 2024-03-21T10:41:18.292 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:18.294+0000 7fbe62e00700 1 Processor -- start 2024-03-21T10:41:18.292 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:18.295+0000 7fbe62e00700 1 -- start start 2024-03-21T10:41:18.292 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:18.295+0000 7fbe62e00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbe5c1071d0 0x7fbe5c19ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:41:18.292 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:18.295+0000 7fbe62e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbe5c10d280 0x7fbe5c19d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:41:18.292 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:18.295+0000 7fbe62e00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe5c19d880 con 0x7fbe5c10d280 2024-03-21T10:41:18.292 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:18.295+0000 7fbe62e00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe5c1a05b0 con 0x7fbe5c1071d0 2024-03-21T10:41:18.292 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:18.295+0000 7fbe5b400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbe5c10d280 0x7fbe5c19d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:41:18.292 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:18.295+0000 7fbe5b400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbe5c10d280 0x7fbe5c19d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:35614/0 (socket says 172.21.15.153:35614) 2024-03-21T10:41:18.292 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:18.295+0000 7fbe5b400700 1 -- 172.21.15.153:0/332212495 learned_addr learned my addr 172.21.15.153:0/332212495 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:41:18.293 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:18.296+0000 7fbe5be00700 1 --2- 172.21.15.153:0/332212495 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbe5c1071d0 0x7fbe5c19ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:41:18.293 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:18.296+0000 7fbe5b400700 1 -- 172.21.15.153:0/332212495 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbe5c1071d0 msgr2=0x7fbe5c19ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:41:18.293 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:18.296+0000 7fbe5b400700 1 --2- 172.21.15.153:0/332212495 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbe5c1071d0 0x7fbe5c19ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:41:18.294 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:18.296+0000 7fbe5b400700 1 -- 172.21.15.153:0/332212495 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe480095f0 con 0x7fbe5c10d280 2024-03-21T10:41:18.294 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:18.296+0000 7fbe5b400700 1 --2- 172.21.15.153:0/332212495 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbe5c10d280 0x7fbe5c19d1f0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fbe480092e0 tx=0x7fbe4800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:41:18.294 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:18.296+0000 7fbe58c00700 1 -- 172.21.15.153:0/332212495 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe4801d070 con 0x7fbe5c10d280 2024-03-21T10:41:18.294 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:18.296+0000 7fbe58c00700 1 -- 172.21.15.153:0/332212495 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbe48005e00 con 0x7fbe5c10d280 2024-03-21T10:41:18.294 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:18.296+0000 7fbe58c00700 1 -- 172.21.15.153:0/332212495 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe48017400 con 0x7fbe5c10d280 2024-03-21T10:41:18.294 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:18.296+0000 7fbe62e00700 1 -- 172.21.15.153:0/332212495 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe5c1a0830 con 0x7fbe5c10d280 2024-03-21T10:41:18.294 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:18.297+0000 7fbe62e00700 1 -- 172.21.15.153:0/332212495 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe5c1a0da0 con 0x7fbe5c10d280 2024-03-21T10:41:18.294 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:18.298+0000 7fbe62e00700 1 -- 172.21.15.153:0/332212495 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe5c10f6c0 con 0x7fbe5c10d280 2024-03-21T10:41:18.295 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:18.299+0000 7fbe58c00700 1 -- 172.21.15.153:0/332212495 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fbe4800f460 con 0x7fbe5c10d280 2024-03-21T10:41:18.296 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:18.300+0000 7fbe58c00700 1 --2- 172.21.15.153:0/332212495 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fbe4006e460 0x7fbe40070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:41:18.296 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:18.300+0000 7fbe58c00700 1 -- 172.21.15.153:0/332212495 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fbe4808f9b0 con 0x7fbe5c10d280 2024-03-21T10:41:18.296 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:18.300+0000 7fbe5be00700 1 --2- 172.21.15.153:0/332212495 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fbe4006e460 0x7fbe40070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:41:18.297 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:18.301+0000 7fbe5be00700 1 --2- 172.21.15.153:0/332212495 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fbe4006e460 0x7fbe40070920 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c005950 tx=0x7fbe4c0058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:41:18.301 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:18.305+0000 7fbe58c00700 1 -- 172.21.15.153:0/332212495 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fbe48058a50 con 0x7fbe5c10d280 2024-03-21T10:41:18.542 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:18.545+0000 7fbe62e00700 1 -- 172.21.15.153:0/332212495 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbe5c1a1080 con 0x7fbe4006e460 2024-03-21T10:41:18.546 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:18.549+0000 7fbe58c00700 1 -- 172.21.15.153:0/332212495 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fbe5c1a1080 con 0x7fbe4006e460 2024-03-21T10:41:18.549 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:18.552+0000 7fbe62e00700 1 -- 172.21.15.153:0/332212495 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fbe4006e460 msgr2=0x7fbe40070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:41:18.549 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:18.552+0000 7fbe62e00700 1 --2- 172.21.15.153:0/332212495 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fbe4006e460 0x7fbe40070920 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c005950 tx=0x7fbe4c0058e0 comp rx=0 tx=0).stop 2024-03-21T10:41:18.550 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:18.552+0000 7fbe62e00700 1 -- 172.21.15.153:0/332212495 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbe5c10d280 msgr2=0x7fbe5c19d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:41:18.550 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:18.552+0000 7fbe62e00700 1 --2- 172.21.15.153:0/332212495 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbe5c10d280 0x7fbe5c19d1f0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fbe480092e0 tx=0x7fbe4800be20 comp rx=0 tx=0).stop 2024-03-21T10:41:18.550 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:18.552+0000 7fbe62e00700 1 -- 172.21.15.153:0/332212495 shutdown_connections 2024-03-21T10:41:18.550 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:18.552+0000 7fbe62e00700 1 --2- 172.21.15.153:0/332212495 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbe5c1071d0 0x7fbe5c19ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:41:18.550 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:18.552+0000 7fbe62e00700 1 --2- 172.21.15.153:0/332212495 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fbe4006e460 0x7fbe40070920 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:41:18.550 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:18.552+0000 7fbe62e00700 1 --2- 172.21.15.153:0/332212495 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbe5c10d280 0x7fbe5c19d1f0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:41:18.550 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:18.553+0000 7fbe62e00700 1 -- 172.21.15.153:0/332212495 >> 172.21.15.153:0/332212495 conn(0x7fbe5c0769e0 msgr2=0x7fbe5c078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:41:18.550 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:18.553+0000 7fbe62e00700 1 -- 172.21.15.153:0/332212495 shutdown_connections 2024-03-21T10:41:18.550 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:18.553+0000 7fbe62e00700 1 -- 172.21.15.153:0/332212495 wait complete. 2024-03-21T10:41:18.568 INFO:teuthology.orchestra.run.smithi153.stdout:true 2024-03-21T10:41:18.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:41:18 smithi174 ceph-mon[88004]: pgmap v885: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:41:18.704 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:18.707+0000 7fe7b4c00700 1 -- 172.21.15.153:0/379660570 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe7b0103000 msgr2=0x7fe7b01033e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:41:18.704 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:18.707+0000 7fe7b4c00700 1 --2- 172.21.15.153:0/379660570 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe7b0103000 0x7fe7b01033e0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fe7a0009910 tx=0x7fe7a0009c20 comp rx=0 tx=0).stop 2024-03-21T10:41:18.704 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:18.708+0000 7fe7b4c00700 1 -- 172.21.15.153:0/379660570 shutdown_connections 2024-03-21T10:41:18.705 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:18.708+0000 7fe7b4c00700 1 --2- 172.21.15.153:0/379660570 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe7b01039b0 0x7fe7b0107a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:41:18.705 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:18.708+0000 7fe7b4c00700 1 --2- 172.21.15.153:0/379660570 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe7b0103000 0x7fe7b01033e0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:41:18.705 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:18.708+0000 7fe7b4c00700 1 -- 172.21.15.153:0/379660570 >> 172.21.15.153:0/379660570 conn(0x7fe7b00fe8d0 msgr2=0x7fe7b0100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:41:18.705 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:18.708+0000 7fe7b4c00700 1 -- 172.21.15.153:0/379660570 shutdown_connections 2024-03-21T10:41:18.705 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:18.708+0000 7fe7b4c00700 1 -- 172.21.15.153:0/379660570 wait complete. 2024-03-21T10:41:18.705 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:18.709+0000 7fe7b4c00700 1 Processor -- start 2024-03-21T10:41:18.705 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:18.709+0000 7fe7b4c00700 1 -- start start 2024-03-21T10:41:18.706 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:18.710+0000 7fe7b4c00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe7b0103000 0x7fe7b019e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:41:18.707 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:18.710+0000 7fe7b4c00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe7b01039b0 0x7fe7b019ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:41:18.707 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:18.710+0000 7fe7b4c00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7b019f3c0 con 0x7fe7b01039b0 2024-03-21T10:41:18.707 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:18.710+0000 7fe7b4c00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7b0198870 con 0x7fe7b0103000 2024-03-21T10:41:18.707 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:18.710+0000 7fe7ad400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe7b01039b0 0x7fe7b019ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:41:18.707 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:18.710+0000 7fe7ad400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe7b01039b0 0x7fe7b019ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:35632/0 (socket says 172.21.15.153:35632) 2024-03-21T10:41:18.707 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:18.710+0000 7fe7ad400700 1 -- 172.21.15.153:0/1736002855 learned_addr learned my addr 172.21.15.153:0/1736002855 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:41:18.707 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:18.710+0000 7fe7ade00700 1 --2- 172.21.15.153:0/1736002855 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe7b0103000 0x7fe7b019e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:41:18.708 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:18.710+0000 7fe7ad400700 1 -- 172.21.15.153:0/1736002855 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe7b0103000 msgr2=0x7fe7b019e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:41:18.708 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:18.710+0000 7fe7ad400700 1 --2- 172.21.15.153:0/1736002855 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe7b0103000 0x7fe7b019e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:41:18.708 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:18.710+0000 7fe7ad400700 1 -- 172.21.15.153:0/1736002855 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7a00095f0 con 0x7fe7b01039b0 2024-03-21T10:41:18.708 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:18.711+0000 7fe7ad400700 1 --2- 172.21.15.153:0/1736002855 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe7b01039b0 0x7fe7b019ed30 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fe79c00c8f0 tx=0x7fe79c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:41:18.708 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:18.711+0000 7fe7a6a00700 1 -- 172.21.15.153:0/1736002855 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe79c007a50 con 0x7fe7b01039b0 2024-03-21T10:41:18.708 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:18.711+0000 7fe7a6a00700 1 -- 172.21.15.153:0/1736002855 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe79c007bb0 con 0x7fe7b01039b0 2024-03-21T10:41:18.708 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:18.711+0000 7fe7b4c00700 1 -- 172.21.15.153:0/1736002855 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7b0198b50 con 0x7fe7b01039b0 2024-03-21T10:41:18.708 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:18.711+0000 7fe7a6a00700 1 -- 172.21.15.153:0/1736002855 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe79c018630 con 0x7fe7b01039b0 2024-03-21T10:41:18.708 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:18.711+0000 7fe7b4c00700 1 -- 172.21.15.153:0/1736002855 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7b01990a0 con 0x7fe7b01039b0 2024-03-21T10:41:18.709 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:18.712+0000 7fe7b4c00700 1 -- 172.21.15.153:0/1736002855 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7b010b3a0 con 0x7fe7b01039b0 2024-03-21T10:41:18.710 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:18.713+0000 7fe7a6a00700 1 -- 172.21.15.153:0/1736002855 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fe79c007d20 con 0x7fe7b01039b0 2024-03-21T10:41:18.711 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:18.714+0000 7fe7a6a00700 1 --2- 172.21.15.153:0/1736002855 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe79406e460 0x7fe794070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:41:18.711 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:18.714+0000 7fe7a6a00700 1 -- 172.21.15.153:0/1736002855 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fe79c08e550 con 0x7fe7b01039b0 2024-03-21T10:41:18.711 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:18.715+0000 7fe7ade00700 1 --2- 172.21.15.153:0/1736002855 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe79406e460 0x7fe794070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:41:18.711 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:18.715+0000 7fe7ade00700 1 --2- 172.21.15.153:0/1736002855 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe79406e460 0x7fe794070920 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fe7a000b5c0 tx=0x7fe7a001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:41:18.716 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:18.719+0000 7fe7a6a00700 1 -- 172.21.15.153:0/1736002855 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe79c05b0a0 con 0x7fe7b01039b0 2024-03-21T10:41:18.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:41:18 smithi153 ceph-mon[113733]: pgmap v885: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:41:18.953 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:18.957+0000 7fe7b4c00700 1 -- 172.21.15.153:0/1736002855 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe7b0199390 con 0x7fe79406e460 2024-03-21T10:41:18.955 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:18.958+0000 7fe7a6a00700 1 -- 172.21.15.153:0/1736002855 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fe7b0199390 con 0x7fe79406e460 2024-03-21T10:41:18.957 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:18.961+0000 7fe7b4c00700 1 -- 172.21.15.153:0/1736002855 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe79406e460 msgr2=0x7fe794070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:41:18.958 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:18.961+0000 7fe7b4c00700 1 --2- 172.21.15.153:0/1736002855 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe79406e460 0x7fe794070920 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fe7a000b5c0 tx=0x7fe7a001a040 comp rx=0 tx=0).stop 2024-03-21T10:41:18.959 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:18.961+0000 7fe7b4c00700 1 -- 172.21.15.153:0/1736002855 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe7b01039b0 msgr2=0x7fe7b019ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:41:18.959 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:18.961+0000 7fe7b4c00700 1 --2- 172.21.15.153:0/1736002855 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe7b01039b0 0x7fe7b019ed30 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fe79c00c8f0 tx=0x7fe79c00ccb0 comp rx=0 tx=0).stop 2024-03-21T10:41:18.959 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:18.961+0000 7fe7b4c00700 1 -- 172.21.15.153:0/1736002855 shutdown_connections 2024-03-21T10:41:18.959 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:18.961+0000 7fe7b4c00700 1 --2- 172.21.15.153:0/1736002855 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe7b0103000 0x7fe7b019e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:41:18.959 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:18.961+0000 7fe7b4c00700 1 --2- 172.21.15.153:0/1736002855 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe79406e460 0x7fe794070920 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:41:18.959 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:18.961+0000 7fe7b4c00700 1 --2- 172.21.15.153:0/1736002855 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe7b01039b0 0x7fe7b019ed30 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:41:18.959 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:18.961+0000 7fe7b4c00700 1 -- 172.21.15.153:0/1736002855 >> 172.21.15.153:0/1736002855 conn(0x7fe7b00fe8d0 msgr2=0x7fe7b00ffee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:41:18.959 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:18.962+0000 7fe7b4c00700 1 -- 172.21.15.153:0/1736002855 shutdown_connections 2024-03-21T10:41:18.960 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:18.962+0000 7fe7b4c00700 1 -- 172.21.15.153:0/1736002855 wait complete. 2024-03-21T10:41:19.101 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.104+0000 7f0db7c00700 1 -- 172.21.15.153:0/3784234133 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0db0073160 msgr2=0x7f0db0073540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:41:19.101 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.104+0000 7f0db7c00700 1 --2- 172.21.15.153:0/3784234133 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0db0073160 0x7f0db0073540 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f0da4009910 tx=0x7f0da4009c20 comp rx=0 tx=0).stop 2024-03-21T10:41:19.101 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.104+0000 7f0db7c00700 1 -- 172.21.15.153:0/3784234133 shutdown_connections 2024-03-21T10:41:19.101 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.104+0000 7f0db7c00700 1 --2- 172.21.15.153:0/3784234133 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0db0073a80 0x7f0db010cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:41:19.101 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.104+0000 7f0db7c00700 1 --2- 172.21.15.153:0/3784234133 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0db0073160 0x7f0db0073540 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:41:19.101 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.104+0000 7f0db7c00700 1 -- 172.21.15.153:0/3784234133 >> 172.21.15.153:0/3784234133 conn(0x7f0db00fc650 msgr2=0x7f0db00fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:41:19.101 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.104+0000 7f0db7c00700 1 -- 172.21.15.153:0/3784234133 shutdown_connections 2024-03-21T10:41:19.101 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.104+0000 7f0db7c00700 1 -- 172.21.15.153:0/3784234133 wait complete. 2024-03-21T10:41:19.107 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.105+0000 7f0db7c00700 1 Processor -- start 2024-03-21T10:41:19.107 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.105+0000 7f0db7c00700 1 -- start start 2024-03-21T10:41:19.107 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.106+0000 7f0db7c00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0db0073160 0x7f0db0198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:41:19.107 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.106+0000 7f0db7c00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0db0073a80 0x7f0db0198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:41:19.107 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.106+0000 7f0db7c00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0db01994a0 con 0x7f0db0073a80 2024-03-21T10:41:19.107 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.106+0000 7f0db7c00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0db0199610 con 0x7f0db0073160 2024-03-21T10:41:19.107 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.106+0000 7f0dafe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0db0073a80 0x7f0db0198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:41:19.107 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.106+0000 7f0dafe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0db0073a80 0x7f0db0198ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:35646/0 (socket says 172.21.15.153:35646) 2024-03-21T10:41:19.107 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.106+0000 7f0dafe00700 1 -- 172.21.15.153:0/2132080591 learned_addr learned my addr 172.21.15.153:0/2132080591 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:41:19.107 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.106+0000 7f0db5200700 1 --2- 172.21.15.153:0/2132080591 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0db0073160 0x7f0db0198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:41:19.108 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.106+0000 7f0dafe00700 1 -- 172.21.15.153:0/2132080591 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0db0073160 msgr2=0x7f0db0198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:41:19.108 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.106+0000 7f0dafe00700 1 --2- 172.21.15.153:0/2132080591 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0db0073160 0x7f0db0198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:41:19.108 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.106+0000 7f0dafe00700 1 -- 172.21.15.153:0/2132080591 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0da40095f0 con 0x7f0db0073a80 2024-03-21T10:41:19.108 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.107+0000 7f0dafe00700 1 --2- 172.21.15.153:0/2132080591 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0db0073a80 0x7f0db0198ea0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c00c930 tx=0x7f0d9c00ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:41:19.108 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.107+0000 7f0dad600700 1 -- 172.21.15.153:0/2132080591 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0d9c007ad0 con 0x7f0db0073a80 2024-03-21T10:41:19.108 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.107+0000 7f0dad600700 1 -- 172.21.15.153:0/2132080591 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0d9c007c30 con 0x7f0db0073a80 2024-03-21T10:41:19.108 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.107+0000 7f0dad600700 1 -- 172.21.15.153:0/2132080591 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0d9c018610 con 0x7f0db0073a80 2024-03-21T10:41:19.108 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.107+0000 7f0db7c00700 1 -- 172.21.15.153:0/2132080591 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0db019d490 con 0x7f0db0073a80 2024-03-21T10:41:19.108 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.107+0000 7f0db7c00700 1 -- 172.21.15.153:0/2132080591 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0db019d990 con 0x7f0db0073a80 2024-03-21T10:41:19.108 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.108+0000 7f0db7c00700 1 -- 172.21.15.153:0/2132080591 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0db010a5c0 con 0x7f0db0073a80 2024-03-21T10:41:19.108 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.109+0000 7f0dad600700 1 -- 172.21.15.153:0/2132080591 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f0d9c00f450 con 0x7f0db0073a80 2024-03-21T10:41:19.109 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.110+0000 7f0dad600700 1 --2- 172.21.15.153:0/2132080591 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f0d9806e460 0x7f0d98070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:41:19.109 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.110+0000 7f0dad600700 1 -- 172.21.15.153:0/2132080591 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f0d9c08e6a0 con 0x7f0db0073a80 2024-03-21T10:41:19.109 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.110+0000 7f0db5200700 1 --2- 172.21.15.153:0/2132080591 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f0d9806e460 0x7f0d98070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:41:19.109 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.111+0000 7f0db5200700 1 --2- 172.21.15.153:0/2132080591 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f0d9806e460 0x7f0d98070920 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f0da4005f50 tx=0x7f0da401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:41:19.111 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.115+0000 7f0dad600700 1 -- 172.21.15.153:0/2132080591 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0d9c05b270 con 0x7f0db0073a80 2024-03-21T10:41:19.329 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.332+0000 7f0db7c00700 1 -- 172.21.15.153:0/2132080591 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f0db002cd80 con 0x7f0d9806e460 2024-03-21T10:41:19.346 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.349+0000 7f0dad600700 1 -- 172.21.15.153:0/2132080591 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f0db002cd80 con 0x7f0d9806e460 2024-03-21T10:41:19.346 INFO:teuthology.orchestra.run.smithi153.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-21T10:41:19.346 INFO:teuthology.orchestra.run.smithi153.stdout:alertmanager.smithi153 smithi153 *:9093,9094 running (30m) 5m ago 43m 15.6M - 0.25.0 c8568f914cd2 9ed3f2200b5f 2024-03-21T10:41:19.347 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi153 smithi153 running (44m) 5m ago 44m 10.4M - 18.2.1 5be31c24972a 5b61994396a6 2024-03-21T10:41:19.347 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi174 smithi174 running (42m) 5m ago 42m 10.1M - 18.2.1 5be31c24972a 4f7a5fc3ac28 2024-03-21T10:41:19.347 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi153 smithi153 running (26m) 5m ago 44m 6656k - 19.0.0-2231-g2edfee68 237e43c7a65f 32c21104d5b0 2024-03-21T10:41:19.347 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi174 smithi174 running (26m) 5m ago 42m 6660k - 19.0.0-2231-g2edfee68 237e43c7a65f a04e48967ab6 2024-03-21T10:41:19.347 INFO:teuthology.orchestra.run.smithi153.stdout:grafana.smithi153 smithi153 *:3000 running (29m) 5m ago 43m 75.2M - 9.4.12 2bacad6d85d8 ee24026ab786 2024-03-21T10:41:19.347 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.iuyluj smithi153 running (21m) 5m ago 36m 18.7M - 19.0.0-2231-g2edfee68 237e43c7a65f 87aff8a5de70 2024-03-21T10:41:19.347 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.zipckr smithi153 running (22m) 5m ago 36m 18.9M - 19.0.0-2231-g2edfee68 237e43c7a65f ccc262bd5dd3 2024-03-21T10:41:19.347 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.nhozag smithi174 running (6m) 5m ago 36m 10.9M - 19.0.0-2231-g2edfee68 237e43c7a65f 9e02aea72c96 2024-03-21T10:41:19.347 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.yyimst smithi174 running (21m) 5m ago 36m 18.5M - 19.0.0-2231-g2edfee68 237e43c7a65f fc1962f9351e 2024-03-21T10:41:19.347 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi153.tluxni smithi153 *:8443,9283,8765 running (33m) 5m ago 45m 512M - 19.0.0-2231-g2edfee68 237e43c7a65f 7ffee0e55382 2024-03-21T10:41:19.347 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi174.qpcwat smithi174 *:8443,9283,8765 running (31m) 5m ago 42m 459M - 19.0.0-2231-g2edfee68 237e43c7a65f 748968f216ab 2024-03-21T10:41:19.347 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi153 smithi153 running (27m) 5m ago 45m 70.8M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f dd8038303dc3 2024-03-21T10:41:19.347 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi174 smithi174 running (27m) 5m ago 42m 51.9M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f 29d78a788f2c 2024-03-21T10:41:19.347 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi153 smithi153 *:9100 running (31m) 5m ago 44m 8140k - 1.5.0 0da6a335fe13 bb835edc0e0b 2024-03-21T10:41:19.347 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi174 smithi174 *:9100 running (31m) 5m ago 42m 8610k - 1.5.0 0da6a335fe13 7c4ecad790fa 2024-03-21T10:41:19.348 INFO:teuthology.orchestra.run.smithi153.stdout:osd.0 smithi153 running (25m) 5m ago 41m 118M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 507900fd4791 2024-03-21T10:41:19.348 INFO:teuthology.orchestra.run.smithi153.stdout:osd.1 smithi153 running (24m) 5m ago 40m 93.0M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 0d646452e9c1 2024-03-21T10:41:19.348 INFO:teuthology.orchestra.run.smithi153.stdout:osd.2 smithi153 running (24m) 5m ago 40m 91.2M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 1326f2afdfcd 2024-03-21T10:41:19.348 INFO:teuthology.orchestra.run.smithi153.stdout:osd.3 smithi174 running (23m) 5m ago 39m 87.5M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 70aafac71ad5 2024-03-21T10:41:19.348 INFO:teuthology.orchestra.run.smithi153.stdout:osd.4 smithi174 running (23m) 5m ago 39m 95.8M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 0d746ce6e79d 2024-03-21T10:41:19.348 INFO:teuthology.orchestra.run.smithi153.stdout:osd.5 smithi174 running (22m) 5m ago 38m 90.3M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 53e5f35b8cac 2024-03-21T10:41:19.348 INFO:teuthology.orchestra.run.smithi153.stdout:prometheus.smithi153 smithi153 *:9095 running (30m) 5m ago 43m 88.1M - 2.43.0 a07b618ecd1d 7d95561e50c2 2024-03-21T10:41:19.348 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.352+0000 7f0db7c00700 1 -- 172.21.15.153:0/2132080591 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f0d9806e460 msgr2=0x7f0d98070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:41:19.349 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.352+0000 7f0db7c00700 1 --2- 172.21.15.153:0/2132080591 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f0d9806e460 0x7f0d98070920 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f0da4005f50 tx=0x7f0da401a040 comp rx=0 tx=0).stop 2024-03-21T10:41:19.349 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.352+0000 7f0db7c00700 1 -- 172.21.15.153:0/2132080591 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0db0073a80 msgr2=0x7f0db0198ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:41:19.349 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.352+0000 7f0db7c00700 1 --2- 172.21.15.153:0/2132080591 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0db0073a80 0x7f0db0198ea0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c00c930 tx=0x7f0d9c00ccf0 comp rx=0 tx=0).stop 2024-03-21T10:41:19.349 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.352+0000 7f0db7c00700 1 -- 172.21.15.153:0/2132080591 shutdown_connections 2024-03-21T10:41:19.349 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.352+0000 7f0db7c00700 1 --2- 172.21.15.153:0/2132080591 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0db0073160 0x7f0db0198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:41:19.350 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.352+0000 7f0db7c00700 1 --2- 172.21.15.153:0/2132080591 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f0d9806e460 0x7f0d98070920 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:41:19.350 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.352+0000 7f0db7c00700 1 --2- 172.21.15.153:0/2132080591 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0db0073a80 0x7f0db0198ea0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:41:19.350 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.352+0000 7f0db7c00700 1 -- 172.21.15.153:0/2132080591 >> 172.21.15.153:0/2132080591 conn(0x7f0db00fc650 msgr2=0x7f0db0107700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:41:19.350 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.352+0000 7f0db7c00700 1 -- 172.21.15.153:0/2132080591 shutdown_connections 2024-03-21T10:41:19.350 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.353+0000 7f0db7c00700 1 -- 172.21.15.153:0/2132080591 wait complete. 2024-03-21T10:41:19.504 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.507+0000 7efd48800700 1 -- 172.21.15.153:0/1832169776 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efd40101790 msgr2=0x7efd401057e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:41:19.504 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.507+0000 7efd48800700 1 --2- 172.21.15.153:0/1832169776 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efd40101790 0x7efd401057e0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7efd30009910 tx=0x7efd30009c20 comp rx=0 tx=0).stop 2024-03-21T10:41:19.504 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.507+0000 7efd48800700 1 -- 172.21.15.153:0/1832169776 shutdown_connections 2024-03-21T10:41:19.504 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.507+0000 7efd48800700 1 --2- 172.21.15.153:0/1832169776 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efd40101790 0x7efd401057e0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:41:19.504 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.507+0000 7efd48800700 1 --2- 172.21.15.153:0/1832169776 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efd40100de0 0x7efd401011c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:41:19.504 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.507+0000 7efd48800700 1 -- 172.21.15.153:0/1832169776 >> 172.21.15.153:0/1832169776 conn(0x7efd400fc670 msgr2=0x7efd400fea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:41:19.504 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.507+0000 7efd48800700 1 -- 172.21.15.153:0/1832169776 shutdown_connections 2024-03-21T10:41:19.505 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.507+0000 7efd48800700 1 -- 172.21.15.153:0/1832169776 wait complete. 2024-03-21T10:41:19.505 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.508+0000 7efd48800700 1 Processor -- start 2024-03-21T10:41:19.505 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.508+0000 7efd48800700 1 -- start start 2024-03-21T10:41:19.506 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.509+0000 7efd48800700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efd40100de0 0x7efd4019e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:41:19.506 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.509+0000 7efd48800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efd40101790 0x7efd4019ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:41:19.506 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.509+0000 7efd48800700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd4019f400 con 0x7efd40101790 2024-03-21T10:41:19.506 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.509+0000 7efd48800700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd401988b0 con 0x7efd40100de0 2024-03-21T10:41:19.506 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.509+0000 7efd45400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efd40101790 0x7efd4019ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:41:19.506 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.509+0000 7efd45400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efd40101790 0x7efd4019ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:35654/0 (socket says 172.21.15.153:35654) 2024-03-21T10:41:19.506 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.509+0000 7efd45400700 1 -- 172.21.15.153:0/761129711 learned_addr learned my addr 172.21.15.153:0/761129711 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:41:19.507 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.509+0000 7efd45e00700 1 --2- 172.21.15.153:0/761129711 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efd40100de0 0x7efd4019e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:41:19.507 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.509+0000 7efd45400700 1 -- 172.21.15.153:0/761129711 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efd40100de0 msgr2=0x7efd4019e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:41:19.507 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.509+0000 7efd45400700 1 --2- 172.21.15.153:0/761129711 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efd40100de0 0x7efd4019e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:41:19.507 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.509+0000 7efd45400700 1 -- 172.21.15.153:0/761129711 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd300095f0 con 0x7efd40101790 2024-03-21T10:41:19.507 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.510+0000 7efd45400700 1 --2- 172.21.15.153:0/761129711 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efd40101790 0x7efd4019ed70 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7efd30005f50 tx=0x7efd3000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:41:19.507 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.510+0000 7efd3aa00700 1 -- 172.21.15.153:0/761129711 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efd3001d070 con 0x7efd40101790 2024-03-21T10:41:19.507 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.510+0000 7efd3aa00700 1 -- 172.21.15.153:0/761129711 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efd3000f460 con 0x7efd40101790 2024-03-21T10:41:19.507 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.510+0000 7efd3aa00700 1 -- 172.21.15.153:0/761129711 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efd30017630 con 0x7efd40101790 2024-03-21T10:41:19.507 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.510+0000 7efd48800700 1 -- 172.21.15.153:0/761129711 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd40198b30 con 0x7efd40101790 2024-03-21T10:41:19.507 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.510+0000 7efd48800700 1 -- 172.21.15.153:0/761129711 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd40199020 con 0x7efd40101790 2024-03-21T10:41:19.508 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.511+0000 7efd48800700 1 -- 172.21.15.153:0/761129711 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd40104f60 con 0x7efd40101790 2024-03-21T10:41:19.509 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.512+0000 7efd3aa00700 1 -- 172.21.15.153:0/761129711 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7efd3000f5d0 con 0x7efd40101790 2024-03-21T10:41:19.510 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.513+0000 7efd3aa00700 1 --2- 172.21.15.153:0/761129711 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7efd2806e460 0x7efd28070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:41:19.510 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.513+0000 7efd3aa00700 1 -- 172.21.15.153:0/761129711 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7efd3008fb90 con 0x7efd40101790 2024-03-21T10:41:19.510 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.514+0000 7efd45e00700 1 --2- 172.21.15.153:0/761129711 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7efd2806e460 0x7efd28070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:41:19.511 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.514+0000 7efd45e00700 1 --2- 172.21.15.153:0/761129711 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7efd2806e460 0x7efd28070920 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7efd34006fd0 tx=0x7efd34008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:41:19.515 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.518+0000 7efd3aa00700 1 -- 172.21.15.153:0/761129711 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7efd3005c6e0 con 0x7efd40101790 2024-03-21T10:41:19.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:41:19 smithi153 ceph-mon[113733]: from='client.35574 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:41:19.803 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:41:19 smithi153 ceph-mon[113733]: from='client.35578 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:41:19.803 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:41:19 smithi153 ceph-mon[113733]: from='client.35582 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:41:19.825 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.828+0000 7efd48800700 1 -- 172.21.15.153:0/761129711 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7efd4004ea90 con 0x7efd40101790 2024-03-21T10:41:19.826 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.830+0000 7efd3aa00700 1 -- 172.21.15.153:0/761129711 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7efd3005c270 con 0x7efd40101790 2024-03-21T10:41:19.826 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:41:19.827 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": { 2024-03-21T10:41:19.827 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:41:19.827 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:41:19.827 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": { 2024-03-21T10:41:19.827 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:41:19.827 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:41:19.827 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": { 2024-03-21T10:41:19.827 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 6 2024-03-21T10:41:19.827 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:41:19.827 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": { 2024-03-21T10:41:19.827 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 4 2024-03-21T10:41:19.827 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:41:19.827 INFO:teuthology.orchestra.run.smithi153.stdout: "overall": { 2024-03-21T10:41:19.828 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 14 2024-03-21T10:41:19.828 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-03-21T10:41:19.828 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:41:19.829 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.833+0000 7efd48800700 1 -- 172.21.15.153:0/761129711 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7efd2806e460 msgr2=0x7efd28070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:41:19.829 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.833+0000 7efd48800700 1 --2- 172.21.15.153:0/761129711 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7efd2806e460 0x7efd28070920 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7efd34006fd0 tx=0x7efd34008040 comp rx=0 tx=0).stop 2024-03-21T10:41:19.829 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.833+0000 7efd48800700 1 -- 172.21.15.153:0/761129711 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efd40101790 msgr2=0x7efd4019ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:41:19.830 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.833+0000 7efd48800700 1 --2- 172.21.15.153:0/761129711 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efd40101790 0x7efd4019ed70 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7efd30005f50 tx=0x7efd3000be20 comp rx=0 tx=0).stop 2024-03-21T10:41:19.830 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.833+0000 7efd48800700 1 -- 172.21.15.153:0/761129711 shutdown_connections 2024-03-21T10:41:19.830 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.833+0000 7efd48800700 1 --2- 172.21.15.153:0/761129711 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efd40100de0 0x7efd4019e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:41:19.830 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.833+0000 7efd48800700 1 --2- 172.21.15.153:0/761129711 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7efd2806e460 0x7efd28070920 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:41:19.830 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.833+0000 7efd48800700 1 --2- 172.21.15.153:0/761129711 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efd40101790 0x7efd4019ed70 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:41:19.831 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.833+0000 7efd48800700 1 -- 172.21.15.153:0/761129711 >> 172.21.15.153:0/761129711 conn(0x7efd400fc670 msgr2=0x7efd400fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:41:19.831 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.833+0000 7efd48800700 1 -- 172.21.15.153:0/761129711 shutdown_connections 2024-03-21T10:41:19.831 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.833+0000 7efd48800700 1 -- 172.21.15.153:0/761129711 wait complete. 2024-03-21T10:41:19.948 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:41:19 smithi174 ceph-mon[88004]: from='client.35574 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:41:19.948 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:41:19 smithi174 ceph-mon[88004]: from='client.35578 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:41:19.948 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:41:19 smithi174 ceph-mon[88004]: from='client.35582 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:41:19.973 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.976+0000 7fab6da00700 1 -- 172.21.15.153:0/2850426082 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fab68107bd0 msgr2=0x7fab6810bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:41:19.973 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.976+0000 7fab6da00700 1 --2- 172.21.15.153:0/2850426082 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fab68107bd0 0x7fab6810bc10 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fab54009910 tx=0x7fab54009c20 comp rx=0 tx=0).stop 2024-03-21T10:41:19.973 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.976+0000 7fab6da00700 1 -- 172.21.15.153:0/2850426082 shutdown_connections 2024-03-21T10:41:19.973 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.976+0000 7fab6da00700 1 --2- 172.21.15.153:0/2850426082 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fab68107bd0 0x7fab6810bc10 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:41:19.973 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.976+0000 7fab6da00700 1 --2- 172.21.15.153:0/2850426082 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fab68107220 0x7fab68107600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:41:19.973 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.976+0000 7fab6da00700 1 -- 172.21.15.153:0/2850426082 >> 172.21.15.153:0/2850426082 conn(0x7fab680767e0 msgr2=0x7fab68078c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:41:19.974 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.976+0000 7fab6da00700 1 -- 172.21.15.153:0/2850426082 shutdown_connections 2024-03-21T10:41:19.974 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.976+0000 7fab6da00700 1 -- 172.21.15.153:0/2850426082 wait complete. 2024-03-21T10:41:19.975 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.977+0000 7fab6da00700 1 Processor -- start 2024-03-21T10:41:19.975 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.977+0000 7fab6da00700 1 -- start start 2024-03-21T10:41:19.975 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.978+0000 7fab6da00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fab68107220 0x7fab681a29f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:41:19.975 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.978+0000 7fab6da00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fab68107bd0 0x7fab681a2f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:41:19.975 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.978+0000 7fab6da00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab681a35c0 con 0x7fab68107220 2024-03-21T10:41:19.975 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.978+0000 7fab6da00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab6819ca70 con 0x7fab68107bd0 2024-03-21T10:41:19.976 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.978+0000 7fab67e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fab68107220 0x7fab681a29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:41:19.976 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.978+0000 7fab67e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fab68107220 0x7fab681a29f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:35668/0 (socket says 172.21.15.153:35668) 2024-03-21T10:41:19.976 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.978+0000 7fab67e00700 1 -- 172.21.15.153:0/3374825450 learned_addr learned my addr 172.21.15.153:0/3374825450 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:41:19.976 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.978+0000 7fab67400700 1 --2- 172.21.15.153:0/3374825450 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fab68107bd0 0x7fab681a2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:41:19.976 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.978+0000 7fab67e00700 1 -- 172.21.15.153:0/3374825450 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fab68107bd0 msgr2=0x7fab681a2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:41:19.976 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.978+0000 7fab67e00700 1 --2- 172.21.15.153:0/3374825450 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fab68107bd0 0x7fab681a2f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:41:19.976 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.978+0000 7fab67e00700 1 -- 172.21.15.153:0/3374825450 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab540095f0 con 0x7fab68107220 2024-03-21T10:41:19.976 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.979+0000 7fab67e00700 1 --2- 172.21.15.153:0/3374825450 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fab68107220 0x7fab681a29f0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fab5800c8a0 tx=0x7fab5800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:41:19.977 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.979+0000 7fab64c00700 1 -- 172.21.15.153:0/3374825450 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab5800cea0 con 0x7fab68107220 2024-03-21T10:41:19.977 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.979+0000 7fab64c00700 1 -- 172.21.15.153:0/3374825450 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fab58004830 con 0x7fab68107220 2024-03-21T10:41:19.977 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.979+0000 7fab6da00700 1 -- 172.21.15.153:0/3374825450 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab6819cd50 con 0x7fab68107220 2024-03-21T10:41:19.977 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.979+0000 7fab64c00700 1 -- 172.21.15.153:0/3374825450 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab580055b0 con 0x7fab68107220 2024-03-21T10:41:19.978 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.979+0000 7fab6da00700 1 -- 172.21.15.153:0/3374825450 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab6819d2a0 con 0x7fab68107220 2024-03-21T10:41:19.978 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.980+0000 7fab6da00700 1 -- 172.21.15.153:0/3374825450 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab6810f610 con 0x7fab68107220 2024-03-21T10:41:19.978 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.981+0000 7fab64c00700 1 -- 172.21.15.153:0/3374825450 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fab580049a0 con 0x7fab68107220 2024-03-21T10:41:19.979 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.982+0000 7fab64c00700 1 --2- 172.21.15.153:0/3374825450 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fab4c06e460 0x7fab4c070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:41:19.979 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.982+0000 7fab64c00700 1 -- 172.21.15.153:0/3374825450 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fab5808e1d0 con 0x7fab68107220 2024-03-21T10:41:19.979 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.983+0000 7fab67400700 1 --2- 172.21.15.153:0/3374825450 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fab4c06e460 0x7fab4c070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:41:19.979 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.983+0000 7fab67400700 1 --2- 172.21.15.153:0/3374825450 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fab4c06e460 0x7fab4c070920 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fab54000c00 tx=0x7fab54005fd0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:41:19.983 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:19.987+0000 7fab64c00700 1 -- 172.21.15.153:0/3374825450 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fab5805adc0 con 0x7fab68107220 2024-03-21T10:41:20.236 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:20.239+0000 7fab6da00700 1 -- 172.21.15.153:0/3374825450 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fab6819d520 con 0x7fab68107220 2024-03-21T10:41:20.236 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:20.240+0000 7fab64c00700 1 -- 172.21.15.153:0/3374825450 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1876 (secure 0 0 0) 0x7fab58004c50 con 0x7fab68107220 2024-03-21T10:41:20.237 INFO:teuthology.orchestra.run.smithi153.stdout:e32 2024-03-21T10:41:20.237 INFO:teuthology.orchestra.run.smithi153.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-21T10:41:20.237 INFO:teuthology.orchestra.run.smithi153.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:41:20.237 INFO:teuthology.orchestra.run.smithi153.stdout:legacy client fscid: 1 2024-03-21T10:41:20.237 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:41:20.238 INFO:teuthology.orchestra.run.smithi153.stdout:Filesystem 'cephfs' (1) 2024-03-21T10:41:20.238 INFO:teuthology.orchestra.run.smithi153.stdout:fs_name cephfs 2024-03-21T10:41:20.238 INFO:teuthology.orchestra.run.smithi153.stdout:epoch 23 2024-03-21T10:41:20.238 INFO:teuthology.orchestra.run.smithi153.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-21T10:41:20.238 INFO:teuthology.orchestra.run.smithi153.stdout:created 2024-03-21T10:04:27.086576+0000 2024-03-21T10:41:20.238 INFO:teuthology.orchestra.run.smithi153.stdout:modified 2024-03-21T10:19:05.022092+0000 2024-03-21T10:41:20.238 INFO:teuthology.orchestra.run.smithi153.stdout:tableserver 0 2024-03-21T10:41:20.238 INFO:teuthology.orchestra.run.smithi153.stdout:root 0 2024-03-21T10:41:20.238 INFO:teuthology.orchestra.run.smithi153.stdout:session_timeout 60 2024-03-21T10:41:20.238 INFO:teuthology.orchestra.run.smithi153.stdout:session_autoclose 300 2024-03-21T10:41:20.238 INFO:teuthology.orchestra.run.smithi153.stdout:max_file_size 1099511627776 2024-03-21T10:41:20.238 INFO:teuthology.orchestra.run.smithi153.stdout:max_xattr_size 65536 2024-03-21T10:41:20.238 INFO:teuthology.orchestra.run.smithi153.stdout:required_client_features {} 2024-03-21T10:41:20.239 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure 0 2024-03-21T10:41:20.239 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure_osd_epoch 94 2024-03-21T10:41:20.239 INFO:teuthology.orchestra.run.smithi153.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:41:20.239 INFO:teuthology.orchestra.run.smithi153.stdout:max_mds 2 2024-03-21T10:41:20.239 INFO:teuthology.orchestra.run.smithi153.stdout:in 0,1 2024-03-21T10:41:20.239 INFO:teuthology.orchestra.run.smithi153.stdout:up {} 2024-03-21T10:41:20.239 INFO:teuthology.orchestra.run.smithi153.stdout:failed 0,1 2024-03-21T10:41:20.239 INFO:teuthology.orchestra.run.smithi153.stdout:damaged 2024-03-21T10:41:20.239 INFO:teuthology.orchestra.run.smithi153.stdout:stopped 2024-03-21T10:41:20.239 INFO:teuthology.orchestra.run.smithi153.stdout:data_pools [3] 2024-03-21T10:41:20.239 INFO:teuthology.orchestra.run.smithi153.stdout:metadata_pool 2 2024-03-21T10:41:20.239 INFO:teuthology.orchestra.run.smithi153.stdout:inline_data enabled 2024-03-21T10:41:20.239 INFO:teuthology.orchestra.run.smithi153.stdout:balancer 2024-03-21T10:41:20.239 INFO:teuthology.orchestra.run.smithi153.stdout:bal_rank_mask -1 2024-03-21T10:41:20.240 INFO:teuthology.orchestra.run.smithi153.stdout:standby_count_wanted 1 2024-03-21T10:41:20.240 INFO:teuthology.orchestra.run.smithi153.stdout:qdb_cluster leader: 0 members: 2024-03-21T10:41:20.240 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:41:20.240 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:41:20.240 INFO:teuthology.orchestra.run.smithi153.stdout:Standby daemons: 2024-03-21T10:41:20.240 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:41:20.240 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.zipckr{-1:34442} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6826/1103846295,v1:172.21.15.153:6827/1103846295] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:41:20.240 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.iuyluj{-1:34474} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6828/1738630832,v1:172.21.15.153:6829/1738630832] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:41:20.240 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.yyimst{-1:44313} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.174:6824/4127786797,v1:172.21.15.174:6825/4127786797] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:41:20.240 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.nhozag{-1:44687} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.174:6826/2767517845,v1:172.21.15.174:6827/2767517845] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:41:20.241 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:20.242+0000 7fab6da00700 1 -- 172.21.15.153:0/3374825450 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fab4c06e460 msgr2=0x7fab4c070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:41:20.241 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:20.242+0000 7fab6da00700 1 --2- 172.21.15.153:0/3374825450 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fab4c06e460 0x7fab4c070920 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fab54000c00 tx=0x7fab54005fd0 comp rx=0 tx=0).stop 2024-03-21T10:41:20.241 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:20.242+0000 7fab6da00700 1 -- 172.21.15.153:0/3374825450 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fab68107220 msgr2=0x7fab681a29f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:41:20.241 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:20.242+0000 7fab6da00700 1 --2- 172.21.15.153:0/3374825450 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fab68107220 0x7fab681a29f0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fab5800c8a0 tx=0x7fab5800cc60 comp rx=0 tx=0).stop 2024-03-21T10:41:20.241 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:20.243+0000 7fab6da00700 1 -- 172.21.15.153:0/3374825450 shutdown_connections 2024-03-21T10:41:20.241 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:20.243+0000 7fab6da00700 1 --2- 172.21.15.153:0/3374825450 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fab4c06e460 0x7fab4c070920 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:41:20.242 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:20.243+0000 7fab6da00700 1 --2- 172.21.15.153:0/3374825450 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fab68107220 0x7fab681a29f0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:41:20.242 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:20.243+0000 7fab6da00700 1 --2- 172.21.15.153:0/3374825450 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fab68107bd0 0x7fab681a2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:41:20.242 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:20.243+0000 7fab6da00700 1 -- 172.21.15.153:0/3374825450 >> 172.21.15.153:0/3374825450 conn(0x7fab680767e0 msgr2=0x7fab68078bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:41:20.242 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:20.243+0000 7fab6da00700 1 -- 172.21.15.153:0/3374825450 shutdown_connections 2024-03-21T10:41:20.242 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:20.243+0000 7fab6da00700 1 -- 172.21.15.153:0/3374825450 wait complete. 2024-03-21T10:41:20.242 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 32 2024-03-21T10:41:20.395 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:20.398+0000 7efe49600700 1 -- 172.21.15.153:0/2866517828 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efe44100da0 msgr2=0x7efe44101180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:41:20.395 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:20.398+0000 7efe49600700 1 --2- 172.21.15.153:0/2866517828 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efe44100da0 0x7efe44101180 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7efe34009910 tx=0x7efe34009c20 comp rx=0 tx=0).stop 2024-03-21T10:41:20.396 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:20.398+0000 7efe49600700 1 -- 172.21.15.153:0/2866517828 shutdown_connections 2024-03-21T10:41:20.396 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:20.398+0000 7efe49600700 1 --2- 172.21.15.153:0/2866517828 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efe44101750 0x7efe441057a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:41:20.396 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:20.398+0000 7efe49600700 1 --2- 172.21.15.153:0/2866517828 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efe44100da0 0x7efe44101180 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:41:20.396 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:20.398+0000 7efe49600700 1 -- 172.21.15.153:0/2866517828 >> 172.21.15.153:0/2866517828 conn(0x7efe440fc630 msgr2=0x7efe440fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:41:20.396 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:20.399+0000 7efe49600700 1 -- 172.21.15.153:0/2866517828 shutdown_connections 2024-03-21T10:41:20.396 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:20.399+0000 7efe49600700 1 -- 172.21.15.153:0/2866517828 wait complete. 2024-03-21T10:41:20.396 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:20.400+0000 7efe49600700 1 Processor -- start 2024-03-21T10:41:20.396 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:20.400+0000 7efe49600700 1 -- start start 2024-03-21T10:41:20.397 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:20.400+0000 7efe49600700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efe44100da0 0x7efe4419e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:41:20.397 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:20.400+0000 7efe49600700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efe44101750 0x7efe4419ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:41:20.397 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:20.400+0000 7efe49600700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe4419f350 con 0x7efe44100da0 2024-03-21T10:41:20.397 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:20.400+0000 7efe49600700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe44198800 con 0x7efe44101750 2024-03-21T10:41:20.397 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:20.401+0000 7efe42a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efe44100da0 0x7efe4419e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:41:20.398 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:20.401+0000 7efe42a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efe44100da0 0x7efe4419e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:35682/0 (socket says 172.21.15.153:35682) 2024-03-21T10:41:20.398 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:20.401+0000 7efe42a00700 1 -- 172.21.15.153:0/3681881300 learned_addr learned my addr 172.21.15.153:0/3681881300 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:41:20.399 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:20.401+0000 7efe42a00700 1 -- 172.21.15.153:0/3681881300 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efe44101750 msgr2=0x7efe4419ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:41:20.399 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:20.401+0000 7efe42a00700 1 --2- 172.21.15.153:0/3681881300 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efe44101750 0x7efe4419ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:41:20.399 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:20.401+0000 7efe42a00700 1 -- 172.21.15.153:0/3681881300 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe340095f0 con 0x7efe44100da0 2024-03-21T10:41:20.399 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:20.402+0000 7efe42a00700 1 --2- 172.21.15.153:0/3681881300 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efe44100da0 0x7efe4419e780 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7efe34006010 tx=0x7efe3400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:41:20.399 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:20.402+0000 7efe3b400700 1 -- 172.21.15.153:0/3681881300 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe3401d070 con 0x7efe44100da0 2024-03-21T10:41:20.399 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:20.402+0000 7efe3b400700 1 -- 172.21.15.153:0/3681881300 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efe3400f460 con 0x7efe44100da0 2024-03-21T10:41:20.399 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:20.402+0000 7efe3b400700 1 -- 172.21.15.153:0/3681881300 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe34017630 con 0x7efe44100da0 2024-03-21T10:41:20.399 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:20.402+0000 7efe49600700 1 -- 172.21.15.153:0/3681881300 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe44198a80 con 0x7efe44100da0 2024-03-21T10:41:20.399 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:20.402+0000 7efe49600700 1 -- 172.21.15.153:0/3681881300 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe44198f20 con 0x7efe44100da0 2024-03-21T10:41:20.400 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:20.403+0000 7efe49600700 1 -- 172.21.15.153:0/3681881300 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe44104f20 con 0x7efe44100da0 2024-03-21T10:41:20.401 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:20.404+0000 7efe3b400700 1 -- 172.21.15.153:0/3681881300 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7efe3400f5d0 con 0x7efe44100da0 2024-03-21T10:41:20.402 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:20.405+0000 7efe3b400700 1 --2- 172.21.15.153:0/3681881300 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7efe2806e460 0x7efe28070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:41:20.402 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:20.405+0000 7efe3b400700 1 -- 172.21.15.153:0/3681881300 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7efe3408f900 con 0x7efe44100da0 2024-03-21T10:41:20.402 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:20.406+0000 7efe3be00700 1 --2- 172.21.15.153:0/3681881300 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7efe2806e460 0x7efe28070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:41:20.402 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:20.406+0000 7efe3be00700 1 --2- 172.21.15.153:0/3681881300 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7efe2806e460 0x7efe28070920 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7efe2c005950 tx=0x7efe2c0058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:41:20.406 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:20.410+0000 7efe3b400700 1 -- 172.21.15.153:0/3681881300 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7efe3405c570 con 0x7efe44100da0 2024-03-21T10:41:20.657 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:20.660+0000 7efe49600700 1 -- 172.21.15.153:0/3681881300 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7efe441992c0 con 0x7efe2806e460 2024-03-21T10:41:20.658 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:20.662+0000 7efe3b400700 1 -- 172.21.15.153:0/3681881300 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7efe441992c0 con 0x7efe2806e460 2024-03-21T10:41:20.659 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:41:20.659 INFO:teuthology.orchestra.run.smithi153.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232", 2024-03-21T10:41:20.659 INFO:teuthology.orchestra.run.smithi153.stdout: "in_progress": true, 2024-03-21T10:41:20.660 INFO:teuthology.orchestra.run.smithi153.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-21T10:41:20.660 INFO:teuthology.orchestra.run.smithi153.stdout: "services_complete": [ 2024-03-21T10:41:20.660 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr", 2024-03-21T10:41:20.660 INFO:teuthology.orchestra.run.smithi153.stdout: "mon", 2024-03-21T10:41:20.660 INFO:teuthology.orchestra.run.smithi153.stdout: "mds", 2024-03-21T10:41:20.660 INFO:teuthology.orchestra.run.smithi153.stdout: "crash", 2024-03-21T10:41:20.660 INFO:teuthology.orchestra.run.smithi153.stdout: "osd" 2024-03-21T10:41:20.660 INFO:teuthology.orchestra.run.smithi153.stdout: ], 2024-03-21T10:41:20.660 INFO:teuthology.orchestra.run.smithi153.stdout: "progress": "16/23 daemons upgraded", 2024-03-21T10:41:20.660 INFO:teuthology.orchestra.run.smithi153.stdout: "message": "Currently upgrading mds daemons", 2024-03-21T10:41:20.661 INFO:teuthology.orchestra.run.smithi153.stdout: "is_paused": false 2024-03-21T10:41:20.661 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:41:20.661 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:20.664+0000 7efe49600700 1 -- 172.21.15.153:0/3681881300 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7efe2806e460 msgr2=0x7efe28070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:41:20.661 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:20.664+0000 7efe49600700 1 --2- 172.21.15.153:0/3681881300 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7efe2806e460 0x7efe28070920 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7efe2c005950 tx=0x7efe2c0058e0 comp rx=0 tx=0).stop 2024-03-21T10:41:20.661 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:20.664+0000 7efe49600700 1 -- 172.21.15.153:0/3681881300 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efe44100da0 msgr2=0x7efe4419e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:41:20.662 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:20.664+0000 7efe49600700 1 --2- 172.21.15.153:0/3681881300 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efe44100da0 0x7efe4419e780 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7efe34006010 tx=0x7efe3400be20 comp rx=0 tx=0).stop 2024-03-21T10:41:20.662 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:20.665+0000 7efe49600700 1 -- 172.21.15.153:0/3681881300 shutdown_connections 2024-03-21T10:41:20.662 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:20.665+0000 7efe49600700 1 --2- 172.21.15.153:0/3681881300 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7efe2806e460 0x7efe28070920 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:41:20.662 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:20.665+0000 7efe49600700 1 --2- 172.21.15.153:0/3681881300 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efe44100da0 0x7efe4419e780 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:41:20.662 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:20.665+0000 7efe49600700 1 --2- 172.21.15.153:0/3681881300 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efe44101750 0x7efe4419ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:41:20.662 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:20.665+0000 7efe49600700 1 -- 172.21.15.153:0/3681881300 >> 172.21.15.153:0/3681881300 conn(0x7efe440fc630 msgr2=0x7efe440fea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:41:20.662 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:20.665+0000 7efe49600700 1 -- 172.21.15.153:0/3681881300 shutdown_connections 2024-03-21T10:41:20.662 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:20.665+0000 7efe49600700 1 -- 172.21.15.153:0/3681881300 wait complete. 2024-03-21T10:41:20.809 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:20.812+0000 7fa84a800700 1 -- 172.21.15.153:0/2130468201 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa8441039b0 msgr2=0x7fa844107a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:41:20.809 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:20.812+0000 7fa84a800700 1 --2- 172.21.15.153:0/2130468201 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa8441039b0 0x7fa844107a00 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fa830009910 tx=0x7fa830009c20 comp rx=0 tx=0).stop 2024-03-21T10:41:20.810 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:20.812+0000 7fa84a800700 1 -- 172.21.15.153:0/2130468201 shutdown_connections 2024-03-21T10:41:20.810 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:20.812+0000 7fa84a800700 1 --2- 172.21.15.153:0/2130468201 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa8441039b0 0x7fa844107a00 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:41:20.810 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:20.812+0000 7fa84a800700 1 --2- 172.21.15.153:0/2130468201 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa844103000 0x7fa8441033e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:41:20.810 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:20.812+0000 7fa84a800700 1 -- 172.21.15.153:0/2130468201 >> 172.21.15.153:0/2130468201 conn(0x7fa8440fe8d0 msgr2=0x7fa844100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:41:20.810 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:20.813+0000 7fa84a800700 1 -- 172.21.15.153:0/2130468201 shutdown_connections 2024-03-21T10:41:20.810 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:20.813+0000 7fa84a800700 1 -- 172.21.15.153:0/2130468201 wait complete. 2024-03-21T10:41:20.811 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:20.813+0000 7fa84a800700 1 Processor -- start 2024-03-21T10:41:20.811 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:20.814+0000 7fa84a800700 1 -- start start 2024-03-21T10:41:20.811 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:20.814+0000 7fa84a800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa844103000 0x7fa84419e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:41:20.812 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:20.814+0000 7fa84a800700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa8441039b0 0x7fa84419ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:41:20.812 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:20.814+0000 7fa84a800700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa84419f390 con 0x7fa844103000 2024-03-21T10:41:20.812 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:20.814+0000 7fa84a800700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa844198840 con 0x7fa8441039b0 2024-03-21T10:41:20.812 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:20.814+0000 7fa843400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa844103000 0x7fa84419e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:41:20.812 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:20.814+0000 7fa843400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa844103000 0x7fa84419e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:35692/0 (socket says 172.21.15.153:35692) 2024-03-21T10:41:20.812 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:20.814+0000 7fa843400700 1 -- 172.21.15.153:0/2525573589 learned_addr learned my addr 172.21.15.153:0/2525573589 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:41:20.812 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:20.815+0000 7fa842a00700 1 --2- 172.21.15.153:0/2525573589 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa8441039b0 0x7fa84419ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:41:20.812 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:20.815+0000 7fa843400700 1 -- 172.21.15.153:0/2525573589 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa8441039b0 msgr2=0x7fa84419ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:41:20.812 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:20.815+0000 7fa843400700 1 --2- 172.21.15.153:0/2525573589 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa8441039b0 0x7fa84419ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:41:20.812 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:20.815+0000 7fa843400700 1 -- 172.21.15.153:0/2525573589 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8300095f0 con 0x7fa844103000 2024-03-21T10:41:20.812 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:20.815+0000 7fa843400700 1 --2- 172.21.15.153:0/2525573589 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa844103000 0x7fa84419e7c0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fa83400c8a0 tx=0x7fa83400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:41:20.813 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:20.815+0000 7fa83be00700 1 -- 172.21.15.153:0/2525573589 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa83400cea0 con 0x7fa844103000 2024-03-21T10:41:20.813 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:20.815+0000 7fa83be00700 1 -- 172.21.15.153:0/2525573589 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa834004830 con 0x7fa844103000 2024-03-21T10:41:20.813 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:20.815+0000 7fa84a800700 1 -- 172.21.15.153:0/2525573589 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa844198b20 con 0x7fa844103000 2024-03-21T10:41:20.813 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:20.815+0000 7fa83be00700 1 -- 172.21.15.153:0/2525573589 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa8340055b0 con 0x7fa844103000 2024-03-21T10:41:20.813 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:20.815+0000 7fa84a800700 1 -- 172.21.15.153:0/2525573589 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa844199070 con 0x7fa844103000 2024-03-21T10:41:20.813 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:20.816+0000 7fa84a800700 1 -- 172.21.15.153:0/2525573589 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa84410b400 con 0x7fa844103000 2024-03-21T10:41:20.814 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:20.817+0000 7fa83be00700 1 -- 172.21.15.153:0/2525573589 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fa8340049a0 con 0x7fa844103000 2024-03-21T10:41:20.815 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:20.818+0000 7fa83be00700 1 --2- 172.21.15.153:0/2525573589 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa82806e460 0x7fa828070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:41:20.815 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:20.818+0000 7fa83be00700 1 -- 172.21.15.153:0/2525573589 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fa83408e170 con 0x7fa844103000 2024-03-21T10:41:20.815 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:20.819+0000 7fa842a00700 1 --2- 172.21.15.153:0/2525573589 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa82806e460 0x7fa828070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:41:20.816 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:41:20 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/761129711' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:41:20.816 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:41:20 smithi153 ceph-mon[113733]: pgmap v886: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:41:20.816 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:41:20 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/3374825450' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:41:20.816 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:20.819+0000 7fa842a00700 1 --2- 172.21.15.153:0/2525573589 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa82806e460 0x7fa828070920 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fa8440ff0a0 tx=0x7fa83001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:41:20.820 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:20.824+0000 7fa83be00700 1 -- 172.21.15.153:0/2525573589 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa83405ad40 con 0x7fa844103000 2024-03-21T10:41:20.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:41:20 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/761129711' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:41:20.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:41:20 smithi174 ceph-mon[88004]: pgmap v886: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:41:20.948 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:41:20 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/3374825450' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:41:21.127 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:21.130+0000 7fa84a800700 1 -- 172.21.15.153:0/2525573589 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fa844199340 con 0x7fa844103000 2024-03-21T10:41:21.128 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:21.131+0000 7fa83be00700 1 -- 172.21.15.153:0/2525573589 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4493 (secure 0 0 0) 0x7fa834004c50 con 0x7fa844103000 2024-03-21T10:41:21.128 INFO:teuthology.orchestra.run.smithi153.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-03-21T10:41:21.128 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-03-21T10:41:21.128 INFO:teuthology.orchestra.run.smithi153.stdout: host smithi174 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi174/config 2024-03-21T10:41:21.129 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/podman: timeout after 895 seconds 2024-03-21T10:41:21.129 INFO:teuthology.orchestra.run.smithi153.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 -e NODE_NAME=smithi174 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/run/ceph:z -v /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpd910r3_i:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 inventory --format=json-pretty --filter-for-batch 2024-03-21T10:41:21.129 INFO:teuthology.orchestra.run.smithi153.stdout:Traceback (most recent call last): 2024-03-21T10:41:21.129 INFO:teuthology.orchestra.run.smithi153.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-03-21T10:41:21.129 INFO:teuthology.orchestra.run.smithi153.stdout: return _run_code(code, main_globals, None, 2024-03-21T10:41:21.129 INFO:teuthology.orchestra.run.smithi153.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-03-21T10:41:21.129 INFO:teuthology.orchestra.run.smithi153.stdout: exec(code, run_globals) 2024-03-21T10:41:21.129 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 5606, in 2024-03-21T10:41:21.129 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 5594, in main 2024-03-21T10:41:21.129 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 406, in _infer_config 2024-03-21T10:41:21.129 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 321, in _infer_fsid 2024-03-21T10:41:21.129 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 434, in _infer_image 2024-03-21T10:41:21.129 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 308, in _validate_fsid 2024-03-21T10:41:21.129 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 3357, in command_ceph_volume 2024-03-21T10:41:21.129 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-03-21T10:41:21.130 INFO:teuthology.orchestra.run.smithi153.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 -e NODE_NAME=smithi174 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/run/ceph:z -v /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpd910r3_i:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 inventory --format=json-pretty --filter-for-batch 2024-03-21T10:41:21.130 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-21T10:41:21.130 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is degraded 2024-03-21T10:41:21.130 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-21T10:41:21.130 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-21T10:41:21.130 INFO:teuthology.orchestra.run.smithi153.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-21T10:41:21.130 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-21T10:41:21.131 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:21.134+0000 7fa84a800700 1 -- 172.21.15.153:0/2525573589 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa82806e460 msgr2=0x7fa828070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:41:21.131 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:21.134+0000 7fa84a800700 1 --2- 172.21.15.153:0/2525573589 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa82806e460 0x7fa828070920 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fa8440ff0a0 tx=0x7fa83001a040 comp rx=0 tx=0).stop 2024-03-21T10:41:21.131 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:21.134+0000 7fa84a800700 1 -- 172.21.15.153:0/2525573589 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa844103000 msgr2=0x7fa84419e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:41:21.131 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:21.134+0000 7fa84a800700 1 --2- 172.21.15.153:0/2525573589 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa844103000 0x7fa84419e7c0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fa83400c8a0 tx=0x7fa83400cc60 comp rx=0 tx=0).stop 2024-03-21T10:41:21.131 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:21.134+0000 7fa84a800700 1 -- 172.21.15.153:0/2525573589 shutdown_connections 2024-03-21T10:41:21.132 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:21.134+0000 7fa84a800700 1 --2- 172.21.15.153:0/2525573589 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa82806e460 0x7fa828070920 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:41:21.132 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:21.134+0000 7fa84a800700 1 --2- 172.21.15.153:0/2525573589 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa844103000 0x7fa84419e7c0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:41:21.132 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:21.134+0000 7fa84a800700 1 --2- 172.21.15.153:0/2525573589 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa8441039b0 0x7fa84419ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:41:21.132 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:21.134+0000 7fa84a800700 1 -- 172.21.15.153:0/2525573589 >> 172.21.15.153:0/2525573589 conn(0x7fa8440fe8d0 msgr2=0x7fa844100cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:41:21.132 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:21.135+0000 7fa84a800700 1 -- 172.21.15.153:0/2525573589 shutdown_connections 2024-03-21T10:41:21.132 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:21.135+0000 7fa84a800700 1 -- 172.21.15.153:0/2525573589 wait complete. 2024-03-21T10:41:21.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:41:21 smithi174 ceph-mon[88004]: from='client.35594 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:41:21.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:41:21 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/2525573589' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:41:22.052 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:41:21 smithi153 ceph-mon[113733]: from='client.35594 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:41:22.052 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:41:21 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/2525573589' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:41:22.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:41:22 smithi174 ceph-mon[88004]: pgmap v887: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:41:23.052 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:41:22 smithi153 ceph-mon[113733]: pgmap v887: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:41:25.197 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:41:24 smithi174 ceph-mon[88004]: pgmap v888: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:41:25.302 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:41:24 smithi153 ceph-mon[113733]: pgmap v888: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:41:27.197 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:41:26 smithi174 ceph-mon[88004]: pgmap v889: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:41:27.302 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:41:26 smithi153 ceph-mon[113733]: pgmap v889: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:41:29.197 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:41:28 smithi174 ceph-mon[88004]: pgmap v890: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:41:29.302 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:41:28 smithi153 ceph-mon[113733]: pgmap v890: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:41:31.052 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:41:30 smithi153 ceph-mon[113733]: pgmap v891: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:41:31.197 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:41:30 smithi174 ceph-mon[88004]: pgmap v891: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:41:32.197 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:41:31 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:41:32.302 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:41:31 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:41:33.197 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:41:32 smithi174 ceph-mon[88004]: pgmap v892: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:41:33.302 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:41:32 smithi153 ceph-mon[113733]: pgmap v892: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:41:35.197 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:41:34 smithi174 ceph-mon[88004]: pgmap v893: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:41:35.302 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:41:34 smithi153 ceph-mon[113733]: pgmap v893: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:41:37.197 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:41:36 smithi174 ceph-mon[88004]: pgmap v894: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:41:37.302 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:41:36 smithi153 ceph-mon[113733]: pgmap v894: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:41:39.302 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:41:38 smithi153 ceph-mon[113733]: pgmap v895: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:41:39.447 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:41:38 smithi174 ceph-mon[88004]: pgmap v895: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:41:41.302 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:41:40 smithi153 ceph-mon[113733]: pgmap v896: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:41:41.448 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:41:40 smithi174 ceph-mon[88004]: pgmap v896: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:41:43.302 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:41:42 smithi153 ceph-mon[113733]: pgmap v897: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:41:43.447 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:41:42 smithi174 ceph-mon[88004]: pgmap v897: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:41:45.302 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:41:44 smithi153 ceph-mon[113733]: pgmap v898: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:41:45.447 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:41:44 smithi174 ceph-mon[88004]: pgmap v898: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:41:47.302 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:41:46 smithi153 ceph-mon[113733]: pgmap v899: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:41:47.302 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:41:46 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:41:47.447 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:41:46 smithi174 ceph-mon[88004]: pgmap v899: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:41:47.447 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:41:46 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:41:49.273 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:41:48 smithi174 ceph-mon[88004]: pgmap v900: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:41:49.302 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:41:48 smithi153 ceph-mon[113733]: pgmap v900: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:41:51.300 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:51.302+0000 7f2195400700 1 -- 172.21.15.153:0/4293828731 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2190102e10 msgr2=0x7f21901031f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:41:51.300 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:51.302+0000 7f2195400700 1 --2- 172.21.15.153:0/4293828731 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2190102e10 0x7f21901031f0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f2180009910 tx=0x7f2180009c20 comp rx=0 tx=0).stop 2024-03-21T10:41:51.301 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:41:50 smithi153 ceph-mon[113733]: pgmap v901: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:41:51.301 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:51.303+0000 7f2195400700 1 -- 172.21.15.153:0/4293828731 shutdown_connections 2024-03-21T10:41:51.302 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:51.303+0000 7f2195400700 1 --2- 172.21.15.153:0/4293828731 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f21901037c0 0x7f2190107810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:41:51.302 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:51.303+0000 7f2195400700 1 --2- 172.21.15.153:0/4293828731 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2190102e10 0x7f21901031f0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:41:51.302 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:51.303+0000 7f2195400700 1 -- 172.21.15.153:0/4293828731 >> 172.21.15.153:0/4293828731 conn(0x7f21900fe6c0 msgr2=0x7f2190100ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:41:51.302 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:51.303+0000 7f2195400700 1 -- 172.21.15.153:0/4293828731 shutdown_connections 2024-03-21T10:41:51.302 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:51.303+0000 7f2195400700 1 -- 172.21.15.153:0/4293828731 wait complete. 2024-03-21T10:41:51.302 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:51.304+0000 7f2195400700 1 Processor -- start 2024-03-21T10:41:51.302 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:51.304+0000 7f2195400700 1 -- start start 2024-03-21T10:41:51.303 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:51.304+0000 7f2195400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2190102e10 0x7f219019e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:41:51.303 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:51.305+0000 7f2195400700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f21901037c0 0x7f219019eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:41:51.303 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:51.305+0000 7f2195400700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f219019f150 con 0x7f2190102e10 2024-03-21T10:41:51.303 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:51.305+0000 7f2195400700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2190198600 con 0x7f21901037c0 2024-03-21T10:41:51.303 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:51.305+0000 7f218fe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2190102e10 0x7f219019e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:41:51.304 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:51.305+0000 7f218fe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2190102e10 0x7f219019e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:50314/0 (socket says 172.21.15.153:50314) 2024-03-21T10:41:51.304 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:51.305+0000 7f218fe00700 1 -- 172.21.15.153:0/724923002 learned_addr learned my addr 172.21.15.153:0/724923002 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:41:51.304 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:51.305+0000 7f218f400700 1 --2- 172.21.15.153:0/724923002 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f21901037c0 0x7f219019eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:41:51.304 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:51.305+0000 7f218fe00700 1 -- 172.21.15.153:0/724923002 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f21901037c0 msgr2=0x7f219019eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:41:51.304 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:51.305+0000 7f218fe00700 1 --2- 172.21.15.153:0/724923002 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f21901037c0 0x7f219019eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:41:51.304 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:51.305+0000 7f218fe00700 1 -- 172.21.15.153:0/724923002 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21800095f0 con 0x7f2190102e10 2024-03-21T10:41:51.304 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:51.305+0000 7f218f400700 1 --2- 172.21.15.153:0/724923002 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f21901037c0 0x7f219019eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:41:51.304 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:51.306+0000 7f218fe00700 1 --2- 172.21.15.153:0/724923002 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2190102e10 0x7f219019e580 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f21800092e0 tx=0x7f218000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:41:51.305 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:51.306+0000 7f218cc00700 1 -- 172.21.15.153:0/724923002 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f218001d070 con 0x7f2190102e10 2024-03-21T10:41:51.305 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:51.306+0000 7f218cc00700 1 -- 172.21.15.153:0/724923002 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2180005e00 con 0x7f2190102e10 2024-03-21T10:41:51.305 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:51.306+0000 7f218cc00700 1 -- 172.21.15.153:0/724923002 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2180017400 con 0x7f2190102e10 2024-03-21T10:41:51.305 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:51.306+0000 7f2195400700 1 -- 172.21.15.153:0/724923002 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2190198880 con 0x7f2190102e10 2024-03-21T10:41:51.305 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:51.306+0000 7f2195400700 1 -- 172.21.15.153:0/724923002 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2190198d70 con 0x7f2190102e10 2024-03-21T10:41:51.305 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:51.307+0000 7f2195400700 1 -- 172.21.15.153:0/724923002 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f219010b1b0 con 0x7f2190102e10 2024-03-21T10:41:51.306 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:51.308+0000 7f218cc00700 1 -- 172.21.15.153:0/724923002 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f218000f460 con 0x7f2190102e10 2024-03-21T10:41:51.307 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:51.310+0000 7f218cc00700 1 --2- 172.21.15.153:0/724923002 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f217406e460 0x7f2174070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:41:51.308 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:51.310+0000 7f218cc00700 1 -- 172.21.15.153:0/724923002 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f218008f910 con 0x7f2190102e10 2024-03-21T10:41:51.308 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:51.310+0000 7f218f400700 1 --2- 172.21.15.153:0/724923002 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f217406e460 0x7f2174070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:41:51.308 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:51.311+0000 7f218f400700 1 --2- 172.21.15.153:0/724923002 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f217406e460 0x7f2174070920 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f217c005fd0 tx=0x7f217c005ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:41:51.312 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:51.315+0000 7f218cc00700 1 -- 172.21.15.153:0/724923002 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f218005c580 con 0x7f2190102e10 2024-03-21T10:41:51.447 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:41:50 smithi174 ceph-mon[88004]: pgmap v901: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:41:51.554 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:51.556+0000 7f2195400700 1 -- 172.21.15.153:0/724923002 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f219002ccf0 con 0x7f217406e460 2024-03-21T10:41:51.557 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:51.560+0000 7f218cc00700 1 -- 172.21.15.153:0/724923002 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f219002ccf0 con 0x7f217406e460 2024-03-21T10:41:51.560 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:51.562+0000 7f2195400700 1 -- 172.21.15.153:0/724923002 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f217406e460 msgr2=0x7f2174070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:41:51.560 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:51.562+0000 7f2195400700 1 --2- 172.21.15.153:0/724923002 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f217406e460 0x7f2174070920 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f217c005fd0 tx=0x7f217c005ee0 comp rx=0 tx=0).stop 2024-03-21T10:41:51.560 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:51.562+0000 7f2195400700 1 -- 172.21.15.153:0/724923002 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2190102e10 msgr2=0x7f219019e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:41:51.560 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:51.562+0000 7f2195400700 1 --2- 172.21.15.153:0/724923002 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2190102e10 0x7f219019e580 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f21800092e0 tx=0x7f218000be20 comp rx=0 tx=0).stop 2024-03-21T10:41:51.561 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:51.563+0000 7f2195400700 1 -- 172.21.15.153:0/724923002 shutdown_connections 2024-03-21T10:41:51.561 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:51.563+0000 7f2195400700 1 --2- 172.21.15.153:0/724923002 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f217406e460 0x7f2174070920 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:41:51.561 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:51.563+0000 7f2195400700 1 --2- 172.21.15.153:0/724923002 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2190102e10 0x7f219019e580 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:41:51.561 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:51.563+0000 7f2195400700 1 --2- 172.21.15.153:0/724923002 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f21901037c0 0x7f219019eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:41:51.561 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:51.563+0000 7f2195400700 1 -- 172.21.15.153:0/724923002 >> 172.21.15.153:0/724923002 conn(0x7f21900fe6c0 msgr2=0x7f21900ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:41:51.561 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:51.563+0000 7f2195400700 1 -- 172.21.15.153:0/724923002 shutdown_connections 2024-03-21T10:41:51.562 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:51.563+0000 7f2195400700 1 -- 172.21.15.153:0/724923002 wait complete. 2024-03-21T10:41:51.576 INFO:teuthology.orchestra.run.smithi153.stdout:true 2024-03-21T10:41:51.710 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:51.712+0000 7f23f4400700 1 -- 172.21.15.153:0/3985569203 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f23ec069020 msgr2=0x7f23ec105790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:41:51.710 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:51.712+0000 7f23f4400700 1 --2- 172.21.15.153:0/3985569203 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f23ec069020 0x7f23ec105790 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f23d8009910 tx=0x7f23d8009c20 comp rx=0 tx=0).stop 2024-03-21T10:41:51.710 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:51.712+0000 7f23f4400700 1 -- 172.21.15.153:0/3985569203 shutdown_connections 2024-03-21T10:41:51.711 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:51.712+0000 7f23f4400700 1 --2- 172.21.15.153:0/3985569203 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f23ec069020 0x7f23ec105790 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:41:51.711 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:51.712+0000 7f23f4400700 1 --2- 172.21.15.153:0/3985569203 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f23ec068670 0x7f23ec068a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:41:51.711 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:51.712+0000 7f23f4400700 1 -- 172.21.15.153:0/3985569203 >> 172.21.15.153:0/3985569203 conn(0x7f23ec0755a0 msgr2=0x7f23ec0759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:41:51.711 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:51.712+0000 7f23f4400700 1 -- 172.21.15.153:0/3985569203 shutdown_connections 2024-03-21T10:41:51.711 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:51.712+0000 7f23f4400700 1 -- 172.21.15.153:0/3985569203 wait complete. 2024-03-21T10:41:51.714 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:51.713+0000 7f23f4400700 1 Processor -- start 2024-03-21T10:41:51.714 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:51.713+0000 7f23f4400700 1 -- start start 2024-03-21T10:41:51.714 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:51.714+0000 7f23f4400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f23ec068670 0x7f23ec19e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:41:51.714 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:51.714+0000 7f23f4400700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f23ec069020 0x7f23ec19ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:41:51.714 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:51.714+0000 7f23f4400700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23ec19f3b0 con 0x7f23ec068670 2024-03-21T10:41:51.714 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:51.714+0000 7f23f4400700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23ec198860 con 0x7f23ec069020 2024-03-21T10:41:51.714 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:51.714+0000 7f23f1a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f23ec068670 0x7f23ec19e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:41:51.715 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:51.714+0000 7f23f1a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f23ec068670 0x7f23ec19e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:50342/0 (socket says 172.21.15.153:50342) 2024-03-21T10:41:51.715 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:51.714+0000 7f23f1a00700 1 -- 172.21.15.153:0/2733865365 learned_addr learned my addr 172.21.15.153:0/2733865365 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:41:51.715 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:51.714+0000 7f23f1000700 1 --2- 172.21.15.153:0/2733865365 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f23ec069020 0x7f23ec19ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:41:51.715 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:51.715+0000 7f23f1a00700 1 -- 172.21.15.153:0/2733865365 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f23ec069020 msgr2=0x7f23ec19ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:41:51.715 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:51.715+0000 7f23f1a00700 1 --2- 172.21.15.153:0/2733865365 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f23ec069020 0x7f23ec19ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:41:51.715 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:51.715+0000 7f23f1a00700 1 -- 172.21.15.153:0/2733865365 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23d80095f0 con 0x7f23ec068670 2024-03-21T10:41:51.715 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:51.715+0000 7f23f1a00700 1 --2- 172.21.15.153:0/2733865365 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f23ec068670 0x7f23ec19e7e0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f23e000d8d0 tx=0x7f23e000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:41:51.715 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:51.715+0000 7f23ea000700 1 -- 172.21.15.153:0/2733865365 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f23e0009940 con 0x7f23ec068670 2024-03-21T10:41:51.715 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:51.715+0000 7f23ea000700 1 -- 172.21.15.153:0/2733865365 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f23e0010460 con 0x7f23ec068670 2024-03-21T10:41:51.715 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:51.715+0000 7f23ea000700 1 -- 172.21.15.153:0/2733865365 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f23e000f5f0 con 0x7f23ec068670 2024-03-21T10:41:51.715 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:51.715+0000 7f23f4400700 1 -- 172.21.15.153:0/2733865365 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23ec198b40 con 0x7f23ec068670 2024-03-21T10:41:51.715 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:51.715+0000 7f23f4400700 1 -- 172.21.15.153:0/2733865365 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23ec199090 con 0x7f23ec068670 2024-03-21T10:41:51.716 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:51.716+0000 7f23f4400700 1 -- 172.21.15.153:0/2733865365 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23ec04ea90 con 0x7f23ec068670 2024-03-21T10:41:51.716 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:51.717+0000 7f23ea000700 1 -- 172.21.15.153:0/2733865365 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f23e00105d0 con 0x7f23ec068670 2024-03-21T10:41:51.716 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:51.718+0000 7f23ea000700 1 --2- 172.21.15.153:0/2733865365 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f23d406e460 0x7f23d4070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:41:51.717 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:51.718+0000 7f23ea000700 1 -- 172.21.15.153:0/2733865365 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f23e008e480 con 0x7f23ec068670 2024-03-21T10:41:51.717 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:51.718+0000 7f23f1000700 1 --2- 172.21.15.153:0/2733865365 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f23d406e460 0x7f23d4070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:41:51.717 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:51.719+0000 7f23f1000700 1 --2- 172.21.15.153:0/2733865365 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f23d406e460 0x7f23d4070920 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f23ec0fe9d0 tx=0x7f23d801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:41:51.720 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:51.723+0000 7f23ea000700 1 -- 172.21.15.153:0/2733865365 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f23e005b050 con 0x7f23ec068670 2024-03-21T10:41:51.948 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:51.950+0000 7f23f4400700 1 -- 172.21.15.153:0/2733865365 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f23ec1993e0 con 0x7f23d406e460 2024-03-21T10:41:51.951 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:51.953+0000 7f23ea000700 1 -- 172.21.15.153:0/2733865365 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f23ec1993e0 con 0x7f23d406e460 2024-03-21T10:41:51.954 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:51.956+0000 7f23f4400700 1 -- 172.21.15.153:0/2733865365 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f23d406e460 msgr2=0x7f23d4070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:41:51.954 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:51.956+0000 7f23f4400700 1 --2- 172.21.15.153:0/2733865365 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f23d406e460 0x7f23d4070920 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f23ec0fe9d0 tx=0x7f23d801a040 comp rx=0 tx=0).stop 2024-03-21T10:41:51.954 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:51.956+0000 7f23f4400700 1 -- 172.21.15.153:0/2733865365 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f23ec068670 msgr2=0x7f23ec19e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:41:51.954 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:51.956+0000 7f23f4400700 1 --2- 172.21.15.153:0/2733865365 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f23ec068670 0x7f23ec19e7e0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f23e000d8d0 tx=0x7f23e000dc90 comp rx=0 tx=0).stop 2024-03-21T10:41:51.955 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:51.956+0000 7f23f4400700 1 -- 172.21.15.153:0/2733865365 shutdown_connections 2024-03-21T10:41:51.955 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:51.956+0000 7f23f4400700 1 --2- 172.21.15.153:0/2733865365 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f23d406e460 0x7f23d4070920 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:41:51.955 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:51.956+0000 7f23f4400700 1 --2- 172.21.15.153:0/2733865365 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f23ec068670 0x7f23ec19e7e0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:41:51.955 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:51.956+0000 7f23f4400700 1 --2- 172.21.15.153:0/2733865365 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f23ec069020 0x7f23ec19ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:41:51.955 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:51.956+0000 7f23f4400700 1 -- 172.21.15.153:0/2733865365 >> 172.21.15.153:0/2733865365 conn(0x7f23ec0755a0 msgr2=0x7f23ec102760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:41:51.955 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:51.957+0000 7f23f4400700 1 -- 172.21.15.153:0/2733865365 shutdown_connections 2024-03-21T10:41:51.955 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:51.957+0000 7f23f4400700 1 -- 172.21.15.153:0/2733865365 wait complete. 2024-03-21T10:41:52.097 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.099+0000 7fd623a00700 1 -- 172.21.15.153:0/1784849588 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd61c1039b0 msgr2=0x7fd61c107a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:41:52.097 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.099+0000 7fd623a00700 1 --2- 172.21.15.153:0/1784849588 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd61c1039b0 0x7fd61c107a00 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fd608009910 tx=0x7fd608009c20 comp rx=0 tx=0).stop 2024-03-21T10:41:52.097 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.099+0000 7fd623a00700 1 -- 172.21.15.153:0/1784849588 shutdown_connections 2024-03-21T10:41:52.097 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.099+0000 7fd623a00700 1 --2- 172.21.15.153:0/1784849588 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd61c1039b0 0x7fd61c107a00 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:41:52.097 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.099+0000 7fd623a00700 1 --2- 172.21.15.153:0/1784849588 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd61c103000 0x7fd61c1033e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:41:52.097 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.099+0000 7fd623a00700 1 -- 172.21.15.153:0/1784849588 >> 172.21.15.153:0/1784849588 conn(0x7fd61c0fe8d0 msgr2=0x7fd61c100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:41:52.097 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.099+0000 7fd623a00700 1 -- 172.21.15.153:0/1784849588 shutdown_connections 2024-03-21T10:41:52.097 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.099+0000 7fd623a00700 1 -- 172.21.15.153:0/1784849588 wait complete. 2024-03-21T10:41:52.099 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.100+0000 7fd623a00700 1 Processor -- start 2024-03-21T10:41:52.099 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.100+0000 7fd623a00700 1 -- start start 2024-03-21T10:41:52.099 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.101+0000 7fd623a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd61c103000 0x7fd61c198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:41:52.099 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.101+0000 7fd623a00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd61c1039b0 0x7fd61c198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:41:52.099 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.101+0000 7fd623a00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd61c199530 con 0x7fd61c103000 2024-03-21T10:41:52.099 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.101+0000 7fd623a00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd61c19d340 con 0x7fd61c1039b0 2024-03-21T10:41:52.099 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.101+0000 7fd621000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd61c103000 0x7fd61c198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:41:52.099 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.101+0000 7fd621000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd61c103000 0x7fd61c198960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:50364/0 (socket says 172.21.15.153:50364) 2024-03-21T10:41:52.099 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.101+0000 7fd621000700 1 -- 172.21.15.153:0/927579377 learned_addr learned my addr 172.21.15.153:0/927579377 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:41:52.099 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.101+0000 7fd61be00700 1 --2- 172.21.15.153:0/927579377 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd61c1039b0 0x7fd61c198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:41:52.099 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.101+0000 7fd621000700 1 -- 172.21.15.153:0/927579377 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd61c1039b0 msgr2=0x7fd61c198ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:41:52.099 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.101+0000 7fd621000700 1 --2- 172.21.15.153:0/927579377 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd61c1039b0 0x7fd61c198ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:41:52.099 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.101+0000 7fd621000700 1 -- 172.21.15.153:0/927579377 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6080095f0 con 0x7fd61c103000 2024-03-21T10:41:52.099 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.101+0000 7fd61be00700 1 --2- 172.21.15.153:0/927579377 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd61c1039b0 0x7fd61c198ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:41:52.100 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.102+0000 7fd621000700 1 --2- 172.21.15.153:0/927579377 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd61c103000 0x7fd61c198960 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fd61000d900 tx=0x7fd61000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:41:52.100 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.102+0000 7fd619600700 1 -- 172.21.15.153:0/927579377 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd610004a00 con 0x7fd61c103000 2024-03-21T10:41:52.101 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.102+0000 7fd619600700 1 -- 172.21.15.153:0/927579377 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd6100054e0 con 0x7fd61c103000 2024-03-21T10:41:52.101 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.102+0000 7fd619600700 1 -- 172.21.15.153:0/927579377 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd610009d60 con 0x7fd61c103000 2024-03-21T10:41:52.101 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.102+0000 7fd623a00700 1 -- 172.21.15.153:0/927579377 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd61c19d620 con 0x7fd61c103000 2024-03-21T10:41:52.101 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.102+0000 7fd623a00700 1 -- 172.21.15.153:0/927579377 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd61c19db20 con 0x7fd61c103000 2024-03-21T10:41:52.101 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.103+0000 7fd623a00700 1 -- 172.21.15.153:0/927579377 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd61c10b310 con 0x7fd61c103000 2024-03-21T10:41:52.102 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.104+0000 7fd619600700 1 -- 172.21.15.153:0/927579377 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fd610005020 con 0x7fd61c103000 2024-03-21T10:41:52.103 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.105+0000 7fd619600700 1 --2- 172.21.15.153:0/927579377 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd60406e460 0x7fd604070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:41:52.103 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.105+0000 7fd619600700 1 -- 172.21.15.153:0/927579377 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fd61008e5b0 con 0x7fd61c103000 2024-03-21T10:41:52.103 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.105+0000 7fd61be00700 1 --2- 172.21.15.153:0/927579377 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd60406e460 0x7fd604070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:41:52.103 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.106+0000 7fd61be00700 1 --2- 172.21.15.153:0/927579377 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd60406e460 0x7fd604070920 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fd6080059d0 tx=0x7fd608005960 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:41:52.107 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.110+0000 7fd619600700 1 -- 172.21.15.153:0/927579377 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd61005b100 con 0x7fd61c103000 2024-03-21T10:41:52.340 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.343+0000 7fd623a00700 1 -- 172.21.15.153:0/927579377 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fd61c19de50 con 0x7fd60406e460 2024-03-21T10:41:52.371 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.374+0000 7fd619600700 1 -- 172.21.15.153:0/927579377 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fd61c19de50 con 0x7fd60406e460 2024-03-21T10:41:52.372 INFO:teuthology.orchestra.run.smithi153.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-21T10:41:52.372 INFO:teuthology.orchestra.run.smithi153.stdout:alertmanager.smithi153 smithi153 *:9093,9094 running (30m) 6m ago 44m 15.6M - 0.25.0 c8568f914cd2 9ed3f2200b5f 2024-03-21T10:41:52.372 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi153 smithi153 running (44m) 6m ago 44m 10.4M - 18.2.1 5be31c24972a 5b61994396a6 2024-03-21T10:41:52.372 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi174 smithi174 running (43m) 6m ago 43m 10.1M - 18.2.1 5be31c24972a 4f7a5fc3ac28 2024-03-21T10:41:52.372 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi153 smithi153 running (26m) 6m ago 44m 6656k - 19.0.0-2231-g2edfee68 237e43c7a65f 32c21104d5b0 2024-03-21T10:41:52.372 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi174 smithi174 running (26m) 6m ago 43m 6660k - 19.0.0-2231-g2edfee68 237e43c7a65f a04e48967ab6 2024-03-21T10:41:52.372 INFO:teuthology.orchestra.run.smithi153.stdout:grafana.smithi153 smithi153 *:3000 running (30m) 6m ago 43m 75.2M - 9.4.12 2bacad6d85d8 ee24026ab786 2024-03-21T10:41:52.372 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.iuyluj smithi153 running (22m) 6m ago 37m 18.7M - 19.0.0-2231-g2edfee68 237e43c7a65f 87aff8a5de70 2024-03-21T10:41:52.373 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.zipckr smithi153 running (22m) 6m ago 37m 18.9M - 19.0.0-2231-g2edfee68 237e43c7a65f ccc262bd5dd3 2024-03-21T10:41:52.373 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.nhozag smithi174 running (6m) 6m ago 37m 10.9M - 19.0.0-2231-g2edfee68 237e43c7a65f 9e02aea72c96 2024-03-21T10:41:52.373 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.yyimst smithi174 running (21m) 6m ago 37m 18.5M - 19.0.0-2231-g2edfee68 237e43c7a65f fc1962f9351e 2024-03-21T10:41:52.373 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi153.tluxni smithi153 *:8443,9283,8765 running (33m) 6m ago 46m 512M - 19.0.0-2231-g2edfee68 237e43c7a65f 7ffee0e55382 2024-03-21T10:41:52.373 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi174.qpcwat smithi174 *:8443,9283,8765 running (32m) 6m ago 42m 459M - 19.0.0-2231-g2edfee68 237e43c7a65f 748968f216ab 2024-03-21T10:41:52.373 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi153 smithi153 running (28m) 6m ago 46m 70.8M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f dd8038303dc3 2024-03-21T10:41:52.373 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi174 smithi174 running (27m) 6m ago 42m 51.9M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f 29d78a788f2c 2024-03-21T10:41:52.373 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi153 smithi153 *:9100 running (31m) 6m ago 44m 8140k - 1.5.0 0da6a335fe13 bb835edc0e0b 2024-03-21T10:41:52.373 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi174 smithi174 *:9100 running (31m) 6m ago 42m 8610k - 1.5.0 0da6a335fe13 7c4ecad790fa 2024-03-21T10:41:52.373 INFO:teuthology.orchestra.run.smithi153.stdout:osd.0 smithi153 running (26m) 6m ago 41m 118M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 507900fd4791 2024-03-21T10:41:52.373 INFO:teuthology.orchestra.run.smithi153.stdout:osd.1 smithi153 running (25m) 6m ago 41m 93.0M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 0d646452e9c1 2024-03-21T10:41:52.373 INFO:teuthology.orchestra.run.smithi153.stdout:osd.2 smithi153 running (24m) 6m ago 40m 91.2M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 1326f2afdfcd 2024-03-21T10:41:52.373 INFO:teuthology.orchestra.run.smithi153.stdout:osd.3 smithi174 running (24m) 6m ago 40m 87.5M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 70aafac71ad5 2024-03-21T10:41:52.373 INFO:teuthology.orchestra.run.smithi153.stdout:osd.4 smithi174 running (23m) 6m ago 39m 95.8M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 0d746ce6e79d 2024-03-21T10:41:52.373 INFO:teuthology.orchestra.run.smithi153.stdout:osd.5 smithi174 running (23m) 6m ago 39m 90.3M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 53e5f35b8cac 2024-03-21T10:41:52.373 INFO:teuthology.orchestra.run.smithi153.stdout:prometheus.smithi153 smithi153 *:9095 running (31m) 6m ago 43m 88.1M - 2.43.0 a07b618ecd1d 7d95561e50c2 2024-03-21T10:41:52.375 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.377+0000 7fd623a00700 1 -- 172.21.15.153:0/927579377 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd60406e460 msgr2=0x7fd604070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:41:52.375 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.377+0000 7fd623a00700 1 --2- 172.21.15.153:0/927579377 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd60406e460 0x7fd604070920 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fd6080059d0 tx=0x7fd608005960 comp rx=0 tx=0).stop 2024-03-21T10:41:52.375 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.377+0000 7fd623a00700 1 -- 172.21.15.153:0/927579377 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd61c103000 msgr2=0x7fd61c198960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:41:52.375 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.377+0000 7fd623a00700 1 --2- 172.21.15.153:0/927579377 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd61c103000 0x7fd61c198960 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fd61000d900 tx=0x7fd61000dcc0 comp rx=0 tx=0).stop 2024-03-21T10:41:52.375 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.377+0000 7fd623a00700 1 -- 172.21.15.153:0/927579377 shutdown_connections 2024-03-21T10:41:52.375 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.377+0000 7fd623a00700 1 --2- 172.21.15.153:0/927579377 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd60406e460 0x7fd604070920 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:41:52.375 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.377+0000 7fd623a00700 1 --2- 172.21.15.153:0/927579377 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd61c103000 0x7fd61c198960 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:41:52.375 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.377+0000 7fd623a00700 1 --2- 172.21.15.153:0/927579377 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd61c1039b0 0x7fd61c198ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:41:52.376 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.377+0000 7fd623a00700 1 -- 172.21.15.153:0/927579377 >> 172.21.15.153:0/927579377 conn(0x7fd61c0fe8d0 msgr2=0x7fd61c107270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:41:52.376 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.378+0000 7fd623a00700 1 -- 172.21.15.153:0/927579377 shutdown_connections 2024-03-21T10:41:52.376 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.378+0000 7fd623a00700 1 -- 172.21.15.153:0/927579377 wait complete. 2024-03-21T10:41:52.529 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.531+0000 7ff21aa00700 1 -- 172.21.15.153:0/3646323528 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff214102480 msgr2=0x7ff21410a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:41:52.529 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.531+0000 7ff21aa00700 1 --2- 172.21.15.153:0/3646323528 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff214102480 0x7ff21410a970 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7ff200009910 tx=0x7ff200009c20 comp rx=0 tx=0).stop 2024-03-21T10:41:52.529 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.531+0000 7ff21aa00700 1 -- 172.21.15.153:0/3646323528 shutdown_connections 2024-03-21T10:41:52.530 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.531+0000 7ff21aa00700 1 --2- 172.21.15.153:0/3646323528 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff214102480 0x7ff21410a970 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:41:52.530 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.531+0000 7ff21aa00700 1 --2- 172.21.15.153:0/3646323528 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff214101b60 0x7ff214101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:41:52.530 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.531+0000 7ff21aa00700 1 -- 172.21.15.153:0/3646323528 >> 172.21.15.153:0/3646323528 conn(0x7ff2140fb390 msgr2=0x7ff2140fd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:41:52.530 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.531+0000 7ff21aa00700 1 -- 172.21.15.153:0/3646323528 shutdown_connections 2024-03-21T10:41:52.530 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.532+0000 7ff21aa00700 1 -- 172.21.15.153:0/3646323528 wait complete. 2024-03-21T10:41:52.530 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.533+0000 7ff21aa00700 1 Processor -- start 2024-03-21T10:41:52.530 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.533+0000 7ff21aa00700 1 -- start start 2024-03-21T10:41:52.532 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.534+0000 7ff21aa00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff214101b60 0x7ff214196580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:41:52.532 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.534+0000 7ff21aa00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff214102480 0x7ff214196ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:41:52.532 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.534+0000 7ff21aa00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2141971a0 con 0x7ff214102480 2024-03-21T10:41:52.532 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.534+0000 7ff21aa00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff21419af30 con 0x7ff214101b60 2024-03-21T10:41:52.532 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.534+0000 7ff218c00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff214102480 0x7ff214196ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:41:52.532 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.534+0000 7ff218c00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff214102480 0x7ff214196ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:50384/0 (socket says 172.21.15.153:50384) 2024-03-21T10:41:52.532 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.534+0000 7ff218c00700 1 -- 172.21.15.153:0/3138368187 learned_addr learned my addr 172.21.15.153:0/3138368187 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:41:52.532 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.534+0000 7ff219600700 1 --2- 172.21.15.153:0/3138368187 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff214101b60 0x7ff214196580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:41:52.532 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.534+0000 7ff218c00700 1 -- 172.21.15.153:0/3138368187 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff214101b60 msgr2=0x7ff214196580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:41:52.533 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.534+0000 7ff218c00700 1 --2- 172.21.15.153:0/3138368187 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff214101b60 0x7ff214196580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:41:52.533 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.534+0000 7ff218c00700 1 -- 172.21.15.153:0/3138368187 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2000095f0 con 0x7ff214102480 2024-03-21T10:41:52.533 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.534+0000 7ff219600700 1 --2- 172.21.15.153:0/3138368187 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff214101b60 0x7ff214196580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:41:52.533 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.535+0000 7ff218c00700 1 --2- 172.21.15.153:0/3138368187 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff214102480 0x7ff214196ac0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7ff20000bda0 tx=0x7ff20000be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:41:52.534 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.535+0000 7ff212000700 1 -- 172.21.15.153:0/3138368187 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff20001d070 con 0x7ff214102480 2024-03-21T10:41:52.534 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.535+0000 7ff212000700 1 -- 172.21.15.153:0/3138368187 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff20000f460 con 0x7ff214102480 2024-03-21T10:41:52.534 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.535+0000 7ff212000700 1 -- 172.21.15.153:0/3138368187 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff200017610 con 0x7ff214102480 2024-03-21T10:41:52.534 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.535+0000 7ff21aa00700 1 -- 172.21.15.153:0/3138368187 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff21419b1b0 con 0x7ff214102480 2024-03-21T10:41:52.534 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.535+0000 7ff21aa00700 1 -- 172.21.15.153:0/3138368187 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff21419b6a0 con 0x7ff214102480 2024-03-21T10:41:52.534 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.536+0000 7ff21aa00700 1 -- 172.21.15.153:0/3138368187 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff214108070 con 0x7ff214102480 2024-03-21T10:41:52.535 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.537+0000 7ff212000700 1 -- 172.21.15.153:0/3138368187 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7ff20000fa40 con 0x7ff214102480 2024-03-21T10:41:52.536 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.538+0000 7ff212000700 1 --2- 172.21.15.153:0/3138368187 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff1fc06e460 0x7ff1fc070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:41:52.536 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.538+0000 7ff212000700 1 -- 172.21.15.153:0/3138368187 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7ff20008f8c0 con 0x7ff214102480 2024-03-21T10:41:52.536 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.539+0000 7ff219600700 1 --2- 172.21.15.153:0/3138368187 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff1fc06e460 0x7ff1fc070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:41:52.536 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.539+0000 7ff219600700 1 --2- 172.21.15.153:0/3138368187 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff1fc06e460 0x7ff1fc070920 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7ff208005fd0 tx=0x7ff208005ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:41:52.540 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.543+0000 7ff212000700 1 -- 172.21.15.153:0/3138368187 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff200058960 con 0x7ff214102480 2024-03-21T10:41:52.831 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.833+0000 7ff21aa00700 1 -- 172.21.15.153:0/3138368187 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff21402cc70 con 0x7ff214102480 2024-03-21T10:41:52.831 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.834+0000 7ff212000700 1 -- 172.21.15.153:0/3138368187 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7ff20005c020 con 0x7ff214102480 2024-03-21T10:41:52.832 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:41:52.833 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": { 2024-03-21T10:41:52.833 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:41:52.833 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:41:52.833 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": { 2024-03-21T10:41:52.833 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:41:52.833 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:41:52.833 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": { 2024-03-21T10:41:52.833 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 6 2024-03-21T10:41:52.833 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:41:52.833 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": { 2024-03-21T10:41:52.833 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 4 2024-03-21T10:41:52.833 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:41:52.833 INFO:teuthology.orchestra.run.smithi153.stdout: "overall": { 2024-03-21T10:41:52.833 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 14 2024-03-21T10:41:52.833 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-03-21T10:41:52.834 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:41:52.835 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.837+0000 7ff21aa00700 1 -- 172.21.15.153:0/3138368187 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff1fc06e460 msgr2=0x7ff1fc070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:41:52.835 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.837+0000 7ff21aa00700 1 --2- 172.21.15.153:0/3138368187 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff1fc06e460 0x7ff1fc070920 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7ff208005fd0 tx=0x7ff208005ee0 comp rx=0 tx=0).stop 2024-03-21T10:41:52.835 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.837+0000 7ff21aa00700 1 -- 172.21.15.153:0/3138368187 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff214102480 msgr2=0x7ff214196ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:41:52.835 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.837+0000 7ff21aa00700 1 --2- 172.21.15.153:0/3138368187 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff214102480 0x7ff214196ac0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7ff20000bda0 tx=0x7ff20000be80 comp rx=0 tx=0).stop 2024-03-21T10:41:52.835 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.837+0000 7ff21aa00700 1 -- 172.21.15.153:0/3138368187 shutdown_connections 2024-03-21T10:41:52.836 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.837+0000 7ff21aa00700 1 --2- 172.21.15.153:0/3138368187 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff214101b60 0x7ff214196580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:41:52.836 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.837+0000 7ff21aa00700 1 --2- 172.21.15.153:0/3138368187 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff1fc06e460 0x7ff1fc070920 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:41:52.836 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.837+0000 7ff21aa00700 1 --2- 172.21.15.153:0/3138368187 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff214102480 0x7ff214196ac0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:41:52.836 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.837+0000 7ff21aa00700 1 -- 172.21.15.153:0/3138368187 >> 172.21.15.153:0/3138368187 conn(0x7ff2140fb390 msgr2=0x7ff2141051b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:41:52.836 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.837+0000 7ff21aa00700 1 -- 172.21.15.153:0/3138368187 shutdown_connections 2024-03-21T10:41:52.836 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.837+0000 7ff21aa00700 1 -- 172.21.15.153:0/3138368187 wait complete. 2024-03-21T10:41:52.992 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.994+0000 7f62cee00700 1 -- 172.21.15.153:0/2382475425 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62c81009a0 msgr2=0x7f62c8100e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:41:52.993 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.994+0000 7f62cee00700 1 --2- 172.21.15.153:0/2382475425 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62c81009a0 0x7f62c8100e20 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f62b4009910 tx=0x7f62b4009c20 comp rx=0 tx=0).stop 2024-03-21T10:41:52.993 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.994+0000 7f62cee00700 1 -- 172.21.15.153:0/2382475425 shutdown_connections 2024-03-21T10:41:52.993 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.994+0000 7f62cee00700 1 --2- 172.21.15.153:0/2382475425 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62c81009a0 0x7f62c8100e20 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:41:52.993 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.994+0000 7f62cee00700 1 --2- 172.21.15.153:0/2382475425 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f62c81069a0 0x7f62c8106d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:41:52.993 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.994+0000 7f62cee00700 1 -- 172.21.15.153:0/2382475425 >> 172.21.15.153:0/2382475425 conn(0x7f62c80fc4d0 msgr2=0x7f62c80fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:41:52.993 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.994+0000 7f62cee00700 1 -- 172.21.15.153:0/2382475425 shutdown_connections 2024-03-21T10:41:52.993 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.995+0000 7f62cee00700 1 -- 172.21.15.153:0/2382475425 wait complete. 2024-03-21T10:41:52.993 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.996+0000 7f62cee00700 1 Processor -- start 2024-03-21T10:41:52.994 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.996+0000 7f62cee00700 1 -- start start 2024-03-21T10:41:52.994 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.997+0000 7f62cee00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62c81009a0 0x7f62c8073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:41:52.994 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.997+0000 7f62cee00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f62c81069a0 0x7f62c80735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:41:52.994 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.997+0000 7f62cee00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62c8073b10 con 0x7f62c81009a0 2024-03-21T10:41:52.994 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.997+0000 7f62cee00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62c8073c50 con 0x7f62c81069a0 2024-03-21T10:41:52.995 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.997+0000 7f62cda00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62c81009a0 0x7f62c8073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:41:52.995 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.997+0000 7f62cda00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62c81009a0 0x7f62c8073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:52412/0 (socket says 172.21.15.153:52412) 2024-03-21T10:41:52.995 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.997+0000 7f62cda00700 1 -- 172.21.15.153:0/1326421371 learned_addr learned my addr 172.21.15.153:0/1326421371 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:41:52.995 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.998+0000 7f62cd000700 1 --2- 172.21.15.153:0/1326421371 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f62c81069a0 0x7f62c80735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:41:52.996 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.998+0000 7f62cda00700 1 -- 172.21.15.153:0/1326421371 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f62c81069a0 msgr2=0x7f62c80735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:41:52.996 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.998+0000 7f62cda00700 1 --2- 172.21.15.153:0/1326421371 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f62c81069a0 0x7f62c80735d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:41:52.996 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.998+0000 7f62cda00700 1 -- 172.21.15.153:0/1326421371 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62b40095f0 con 0x7f62c81009a0 2024-03-21T10:41:52.996 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.998+0000 7f62cda00700 1 --2- 172.21.15.153:0/1326421371 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62c81009a0 0x7f62c8073090 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f62bc00b4b0 tx=0x7f62bc00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:41:52.996 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.998+0000 7f62c6000700 1 -- 172.21.15.153:0/1326421371 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f62bc011800 con 0x7f62c81009a0 2024-03-21T10:41:52.996 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.999+0000 7f62c6000700 1 -- 172.21.15.153:0/1326421371 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f62bc011e20 con 0x7f62c81009a0 2024-03-21T10:41:52.997 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.999+0000 7f62cee00700 1 -- 172.21.15.153:0/1326421371 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62c81a2860 con 0x7f62c81009a0 2024-03-21T10:41:52.997 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.999+0000 7f62c6000700 1 -- 172.21.15.153:0/1326421371 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f62bc00f2e0 con 0x7f62c81009a0 2024-03-21T10:41:52.997 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:52.999+0000 7f62cee00700 1 -- 172.21.15.153:0/1326421371 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62c81a2c70 con 0x7f62c81009a0 2024-03-21T10:41:52.998 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:53.000+0000 7f62cee00700 1 -- 172.21.15.153:0/1326421371 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62a8005320 con 0x7f62c81009a0 2024-03-21T10:41:52.998 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:53.001+0000 7f62c6000700 1 -- 172.21.15.153:0/1326421371 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f62bc011960 con 0x7f62c81009a0 2024-03-21T10:41:52.999 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:53.002+0000 7f62c6000700 1 --2- 172.21.15.153:0/1326421371 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f62b006e460 0x7f62b0070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:41:52.999 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:53.002+0000 7f62c6000700 1 -- 172.21.15.153:0/1326421371 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f62bc08df70 con 0x7f62c81009a0 2024-03-21T10:41:53.000 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:53.002+0000 7f62cd000700 1 --2- 172.21.15.153:0/1326421371 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f62b006e460 0x7f62b0070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:41:53.000 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:53.003+0000 7f62cd000700 1 --2- 172.21.15.153:0/1326421371 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f62b006e460 0x7f62b0070920 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f62c8074880 tx=0x7f62b401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:41:53.004 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:53.006+0000 7f62c6000700 1 -- 172.21.15.153:0/1326421371 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f62bc0c2740 con 0x7f62c81009a0 2024-03-21T10:41:53.270 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:53.272+0000 7f62cee00700 1 -- 172.21.15.153:0/1326421371 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f62a8006200 con 0x7f62c81009a0 2024-03-21T10:41:53.270 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:41:52 smithi153 ceph-mon[113733]: from='client.35602 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:41:53.270 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:41:52 smithi153 ceph-mon[113733]: pgmap v902: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:41:53.270 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:41:52 smithi153 ceph-mon[113733]: from='client.35606 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:41:53.271 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:41:52 smithi153 ceph-mon[113733]: from='client.35610 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:41:53.271 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:41:52 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/3138368187' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:41:53.271 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:53.273+0000 7f62c6000700 1 -- 172.21.15.153:0/1326421371 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1876 (secure 0 0 0) 0x7f62bc015770 con 0x7f62c81009a0 2024-03-21T10:41:53.272 INFO:teuthology.orchestra.run.smithi153.stdout:e32 2024-03-21T10:41:53.272 INFO:teuthology.orchestra.run.smithi153.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-21T10:41:53.272 INFO:teuthology.orchestra.run.smithi153.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:41:53.272 INFO:teuthology.orchestra.run.smithi153.stdout:legacy client fscid: 1 2024-03-21T10:41:53.272 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:41:53.272 INFO:teuthology.orchestra.run.smithi153.stdout:Filesystem 'cephfs' (1) 2024-03-21T10:41:53.272 INFO:teuthology.orchestra.run.smithi153.stdout:fs_name cephfs 2024-03-21T10:41:53.272 INFO:teuthology.orchestra.run.smithi153.stdout:epoch 23 2024-03-21T10:41:53.272 INFO:teuthology.orchestra.run.smithi153.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-21T10:41:53.272 INFO:teuthology.orchestra.run.smithi153.stdout:created 2024-03-21T10:04:27.086576+0000 2024-03-21T10:41:53.273 INFO:teuthology.orchestra.run.smithi153.stdout:modified 2024-03-21T10:19:05.022092+0000 2024-03-21T10:41:53.273 INFO:teuthology.orchestra.run.smithi153.stdout:tableserver 0 2024-03-21T10:41:53.273 INFO:teuthology.orchestra.run.smithi153.stdout:root 0 2024-03-21T10:41:53.273 INFO:teuthology.orchestra.run.smithi153.stdout:session_timeout 60 2024-03-21T10:41:53.273 INFO:teuthology.orchestra.run.smithi153.stdout:session_autoclose 300 2024-03-21T10:41:53.273 INFO:teuthology.orchestra.run.smithi153.stdout:max_file_size 1099511627776 2024-03-21T10:41:53.273 INFO:teuthology.orchestra.run.smithi153.stdout:max_xattr_size 65536 2024-03-21T10:41:53.273 INFO:teuthology.orchestra.run.smithi153.stdout:required_client_features {} 2024-03-21T10:41:53.273 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure 0 2024-03-21T10:41:53.273 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure_osd_epoch 94 2024-03-21T10:41:53.273 INFO:teuthology.orchestra.run.smithi153.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:41:53.273 INFO:teuthology.orchestra.run.smithi153.stdout:max_mds 2 2024-03-21T10:41:53.273 INFO:teuthology.orchestra.run.smithi153.stdout:in 0,1 2024-03-21T10:41:53.273 INFO:teuthology.orchestra.run.smithi153.stdout:up {} 2024-03-21T10:41:53.273 INFO:teuthology.orchestra.run.smithi153.stdout:failed 0,1 2024-03-21T10:41:53.274 INFO:teuthology.orchestra.run.smithi153.stdout:damaged 2024-03-21T10:41:53.274 INFO:teuthology.orchestra.run.smithi153.stdout:stopped 2024-03-21T10:41:53.274 INFO:teuthology.orchestra.run.smithi153.stdout:data_pools [3] 2024-03-21T10:41:53.274 INFO:teuthology.orchestra.run.smithi153.stdout:metadata_pool 2 2024-03-21T10:41:53.274 INFO:teuthology.orchestra.run.smithi153.stdout:inline_data enabled 2024-03-21T10:41:53.274 INFO:teuthology.orchestra.run.smithi153.stdout:balancer 2024-03-21T10:41:53.274 INFO:teuthology.orchestra.run.smithi153.stdout:bal_rank_mask -1 2024-03-21T10:41:53.274 INFO:teuthology.orchestra.run.smithi153.stdout:standby_count_wanted 1 2024-03-21T10:41:53.274 INFO:teuthology.orchestra.run.smithi153.stdout:qdb_cluster leader: 0 members: 2024-03-21T10:41:53.277 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:41:53.277 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:41:53.277 INFO:teuthology.orchestra.run.smithi153.stdout:Standby daemons: 2024-03-21T10:41:53.277 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:41:53.277 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.zipckr{-1:34442} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6826/1103846295,v1:172.21.15.153:6827/1103846295] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:41:53.277 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.iuyluj{-1:34474} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6828/1738630832,v1:172.21.15.153:6829/1738630832] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:41:53.277 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.yyimst{-1:44313} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.174:6824/4127786797,v1:172.21.15.174:6825/4127786797] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:41:53.277 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.nhozag{-1:44687} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.174:6826/2767517845,v1:172.21.15.174:6827/2767517845] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:41:53.278 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:53.276+0000 7f62cee00700 1 -- 172.21.15.153:0/1326421371 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f62b006e460 msgr2=0x7f62b0070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:41:53.279 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:53.276+0000 7f62cee00700 1 --2- 172.21.15.153:0/1326421371 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f62b006e460 0x7f62b0070920 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f62c8074880 tx=0x7f62b401a040 comp rx=0 tx=0).stop 2024-03-21T10:41:53.279 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:53.276+0000 7f62cee00700 1 -- 172.21.15.153:0/1326421371 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62c81009a0 msgr2=0x7f62c8073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:41:53.279 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:53.276+0000 7f62cee00700 1 --2- 172.21.15.153:0/1326421371 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62c81009a0 0x7f62c8073090 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f62bc00b4b0 tx=0x7f62bc00b870 comp rx=0 tx=0).stop 2024-03-21T10:41:53.279 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:53.276+0000 7f62cee00700 1 -- 172.21.15.153:0/1326421371 shutdown_connections 2024-03-21T10:41:53.279 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:53.276+0000 7f62cee00700 1 --2- 172.21.15.153:0/1326421371 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f62b006e460 0x7f62b0070920 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:41:53.279 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:53.276+0000 7f62cee00700 1 --2- 172.21.15.153:0/1326421371 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f62c81009a0 0x7f62c8073090 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:41:53.279 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:53.276+0000 7f62cee00700 1 --2- 172.21.15.153:0/1326421371 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f62c81069a0 0x7f62c80735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:41:53.279 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:53.276+0000 7f62cee00700 1 -- 172.21.15.153:0/1326421371 >> 172.21.15.153:0/1326421371 conn(0x7f62c80fc4d0 msgr2=0x7f62c80fcf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:41:53.279 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:53.276+0000 7f62cee00700 1 -- 172.21.15.153:0/1326421371 shutdown_connections 2024-03-21T10:41:53.279 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:53.277+0000 7f62cee00700 1 -- 172.21.15.153:0/1326421371 wait complete. 2024-03-21T10:41:53.279 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 32 2024-03-21T10:41:53.428 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:53.430+0000 7fc6c4200700 1 -- 172.21.15.153:0/1428394231 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc6bc10d280 msgr2=0x7fc6bc10d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:41:53.428 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:53.430+0000 7fc6c4200700 1 --2- 172.21.15.153:0/1428394231 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc6bc10d280 0x7fc6bc10d660 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fc6b0009910 tx=0x7fc6b0009c20 comp rx=0 tx=0).stop 2024-03-21T10:41:53.428 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:53.430+0000 7fc6c4200700 1 -- 172.21.15.153:0/1428394231 shutdown_connections 2024-03-21T10:41:53.428 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:53.430+0000 7fc6c4200700 1 --2- 172.21.15.153:0/1428394231 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc6bc1071d0 0x7fc6bc107650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:41:53.428 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:53.430+0000 7fc6c4200700 1 --2- 172.21.15.153:0/1428394231 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc6bc10d280 0x7fc6bc10d660 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:41:53.428 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:53.430+0000 7fc6c4200700 1 -- 172.21.15.153:0/1428394231 >> 172.21.15.153:0/1428394231 conn(0x7fc6bc0769e0 msgr2=0x7fc6bc078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:41:53.428 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:53.430+0000 7fc6c4200700 1 -- 172.21.15.153:0/1428394231 shutdown_connections 2024-03-21T10:41:53.428 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:53.430+0000 7fc6c4200700 1 -- 172.21.15.153:0/1428394231 wait complete. 2024-03-21T10:41:53.429 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:53.431+0000 7fc6c4200700 1 Processor -- start 2024-03-21T10:41:53.429 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:53.431+0000 7fc6c4200700 1 -- start start 2024-03-21T10:41:53.429 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:53.431+0000 7fc6c4200700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc6bc1071d0 0x7fc6bc19ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:41:53.429 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:53.431+0000 7fc6c4200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc6bc10d280 0x7fc6bc19d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:41:53.429 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:53.431+0000 7fc6c4200700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6bc19d8a0 con 0x7fc6bc10d280 2024-03-21T10:41:53.429 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:53.431+0000 7fc6c4200700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6bc1a05d0 con 0x7fc6bc1071d0 2024-03-21T10:41:53.430 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:53.432+0000 7fc6c0e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc6bc10d280 0x7fc6bc19d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:41:53.430 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:53.432+0000 7fc6c0e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc6bc10d280 0x7fc6bc19d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:52442/0 (socket says 172.21.15.153:52442) 2024-03-21T10:41:53.430 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:53.432+0000 7fc6c0e00700 1 -- 172.21.15.153:0/1819014144 learned_addr learned my addr 172.21.15.153:0/1819014144 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:41:53.431 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:53.432+0000 7fc6c0e00700 1 -- 172.21.15.153:0/1819014144 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc6bc1071d0 msgr2=0x7fc6bc19ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:41:53.431 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:53.432+0000 7fc6c0e00700 1 --2- 172.21.15.153:0/1819014144 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc6bc1071d0 0x7fc6bc19ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:41:53.431 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:53.432+0000 7fc6c0e00700 1 -- 172.21.15.153:0/1819014144 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6b00095f0 con 0x7fc6bc10d280 2024-03-21T10:41:53.431 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:53.432+0000 7fc6c0e00700 1 --2- 172.21.15.153:0/1819014144 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc6bc10d280 0x7fc6bc19d210 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fc6a800c8a0 tx=0x7fc6a800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:41:53.431 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:53.433+0000 7fc6ba000700 1 -- 172.21.15.153:0/1819014144 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc6a800cea0 con 0x7fc6bc10d280 2024-03-21T10:41:53.431 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:53.433+0000 7fc6ba000700 1 -- 172.21.15.153:0/1819014144 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc6a8004830 con 0x7fc6bc10d280 2024-03-21T10:41:53.431 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:53.433+0000 7fc6c4200700 1 -- 172.21.15.153:0/1819014144 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6bc1a08b0 con 0x7fc6bc10d280 2024-03-21T10:41:53.431 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:53.433+0000 7fc6ba000700 1 -- 172.21.15.153:0/1819014144 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc6a80055b0 con 0x7fc6bc10d280 2024-03-21T10:41:53.431 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:53.433+0000 7fc6c4200700 1 -- 172.21.15.153:0/1819014144 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6bc1a0e80 con 0x7fc6bc10d280 2024-03-21T10:41:53.432 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:53.434+0000 7fc6c4200700 1 -- 172.21.15.153:0/1819014144 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6bc04ea90 con 0x7fc6bc10d280 2024-03-21T10:41:53.432 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:53.435+0000 7fc6ba000700 1 -- 172.21.15.153:0/1819014144 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fc6a80049a0 con 0x7fc6bc10d280 2024-03-21T10:41:53.433 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:53.436+0000 7fc6ba000700 1 --2- 172.21.15.153:0/1819014144 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fc6a406e460 0x7fc6a4070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:41:53.434 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:53.436+0000 7fc6ba000700 1 -- 172.21.15.153:0/1819014144 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fc6a808e170 con 0x7fc6bc10d280 2024-03-21T10:41:53.434 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:53.436+0000 7fc6c1800700 1 --2- 172.21.15.153:0/1819014144 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fc6a406e460 0x7fc6a4070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:41:53.434 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:53.437+0000 7fc6c1800700 1 --2- 172.21.15.153:0/1819014144 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fc6a406e460 0x7fc6a4070920 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fc6b000b5c0 tx=0x7fc6b0005960 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:41:53.438 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:53.441+0000 7fc6ba000700 1 -- 172.21.15.153:0/1819014144 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc6a8057210 con 0x7fc6bc10d280 2024-03-21T10:41:53.447 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:41:52 smithi174 ceph-mon[88004]: from='client.35602 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:41:53.448 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:41:52 smithi174 ceph-mon[88004]: pgmap v902: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:41:53.448 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:41:52 smithi174 ceph-mon[88004]: from='client.35606 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:41:53.448 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:41:52 smithi174 ceph-mon[88004]: from='client.35610 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:41:53.448 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:41:52 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/3138368187' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:41:53.668 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:53.670+0000 7fc6c4200700 1 -- 172.21.15.153:0/1819014144 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc6bc1a13f0 con 0x7fc6a406e460 2024-03-21T10:41:53.670 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:53.673+0000 7fc6ba000700 1 -- 172.21.15.153:0/1819014144 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fc6bc1a13f0 con 0x7fc6a406e460 2024-03-21T10:41:53.671 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:41:53.671 INFO:teuthology.orchestra.run.smithi153.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232", 2024-03-21T10:41:53.671 INFO:teuthology.orchestra.run.smithi153.stdout: "in_progress": true, 2024-03-21T10:41:53.671 INFO:teuthology.orchestra.run.smithi153.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-21T10:41:53.671 INFO:teuthology.orchestra.run.smithi153.stdout: "services_complete": [ 2024-03-21T10:41:53.671 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr", 2024-03-21T10:41:53.671 INFO:teuthology.orchestra.run.smithi153.stdout: "mon", 2024-03-21T10:41:53.671 INFO:teuthology.orchestra.run.smithi153.stdout: "mds", 2024-03-21T10:41:53.671 INFO:teuthology.orchestra.run.smithi153.stdout: "crash", 2024-03-21T10:41:53.671 INFO:teuthology.orchestra.run.smithi153.stdout: "osd" 2024-03-21T10:41:53.671 INFO:teuthology.orchestra.run.smithi153.stdout: ], 2024-03-21T10:41:53.671 INFO:teuthology.orchestra.run.smithi153.stdout: "progress": "16/23 daemons upgraded", 2024-03-21T10:41:53.672 INFO:teuthology.orchestra.run.smithi153.stdout: "message": "Currently upgrading mds daemons", 2024-03-21T10:41:53.672 INFO:teuthology.orchestra.run.smithi153.stdout: "is_paused": false 2024-03-21T10:41:53.672 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:41:53.673 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:53.675+0000 7fc6c4200700 1 -- 172.21.15.153:0/1819014144 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fc6a406e460 msgr2=0x7fc6a4070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:41:53.673 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:53.675+0000 7fc6c4200700 1 --2- 172.21.15.153:0/1819014144 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fc6a406e460 0x7fc6a4070920 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fc6b000b5c0 tx=0x7fc6b0005960 comp rx=0 tx=0).stop 2024-03-21T10:41:53.674 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:53.676+0000 7fc6c4200700 1 -- 172.21.15.153:0/1819014144 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc6bc10d280 msgr2=0x7fc6bc19d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:41:53.674 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:53.676+0000 7fc6c4200700 1 --2- 172.21.15.153:0/1819014144 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc6bc10d280 0x7fc6bc19d210 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fc6a800c8a0 tx=0x7fc6a800cc60 comp rx=0 tx=0).stop 2024-03-21T10:41:53.674 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:53.676+0000 7fc6c4200700 1 -- 172.21.15.153:0/1819014144 shutdown_connections 2024-03-21T10:41:53.674 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:53.676+0000 7fc6c4200700 1 --2- 172.21.15.153:0/1819014144 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc6bc1071d0 0x7fc6bc19ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:41:53.674 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:53.676+0000 7fc6c4200700 1 --2- 172.21.15.153:0/1819014144 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fc6a406e460 0x7fc6a4070920 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:41:53.674 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:53.676+0000 7fc6c4200700 1 --2- 172.21.15.153:0/1819014144 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc6bc10d280 0x7fc6bc19d210 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:41:53.674 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:53.676+0000 7fc6c4200700 1 -- 172.21.15.153:0/1819014144 >> 172.21.15.153:0/1819014144 conn(0x7fc6bc0769e0 msgr2=0x7fc6bc078320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:41:53.674 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:53.676+0000 7fc6c4200700 1 -- 172.21.15.153:0/1819014144 shutdown_connections 2024-03-21T10:41:53.674 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:53.676+0000 7fc6c4200700 1 -- 172.21.15.153:0/1819014144 wait complete. 2024-03-21T10:41:53.822 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:53.824+0000 7f58baa00700 1 -- 172.21.15.153:0/3175241173 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f58a80054b0 con 0x7f58bc1071d0 2024-03-21T10:41:53.823 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:53.825+0000 7f58c2a00700 1 -- 172.21.15.153:0/3175241173 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f58bc1071d0 msgr2=0x7f58bc107650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:41:53.823 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:53.825+0000 7f58c2a00700 1 --2- 172.21.15.153:0/3175241173 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f58bc1071d0 0x7f58bc107650 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f58a8009910 tx=0x7f58a8009c20 comp rx=0 tx=0).stop 2024-03-21T10:41:53.823 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:53.825+0000 7f58c2a00700 1 -- 172.21.15.153:0/3175241173 shutdown_connections 2024-03-21T10:41:53.823 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:53.825+0000 7f58c2a00700 1 --2- 172.21.15.153:0/3175241173 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f58bc1071d0 0x7f58bc107650 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:41:53.823 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:53.825+0000 7f58c2a00700 1 --2- 172.21.15.153:0/3175241173 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f58bc10d280 0x7f58bc10d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:41:53.823 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:53.825+0000 7f58c2a00700 1 -- 172.21.15.153:0/3175241173 >> 172.21.15.153:0/3175241173 conn(0x7f58bc076990 msgr2=0x7f58bc078db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:41:53.823 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:53.825+0000 7f58c2a00700 1 -- 172.21.15.153:0/3175241173 shutdown_connections 2024-03-21T10:41:53.823 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:53.825+0000 7f58c2a00700 1 -- 172.21.15.153:0/3175241173 wait complete. 2024-03-21T10:41:53.823 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:53.826+0000 7f58c2a00700 1 Processor -- start 2024-03-21T10:41:53.823 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:53.826+0000 7f58c2a00700 1 -- start start 2024-03-21T10:41:53.824 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:53.827+0000 7f58c2a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f58bc1071d0 0x7f58bc19cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:41:53.824 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:53.827+0000 7f58c2a00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f58bc10d280 0x7f58bc19d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:41:53.824 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:53.827+0000 7f58c2a00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58bc19d940 con 0x7f58bc1071d0 2024-03-21T10:41:53.824 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:53.827+0000 7f58c2a00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58bc1a16d0 con 0x7f58bc10d280 2024-03-21T10:41:53.826 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:53.827+0000 7f58bbe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f58bc1071d0 0x7f58bc19cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:41:53.826 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:53.827+0000 7f58bbe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f58bc1071d0 0x7f58bc19cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:52448/0 (socket says 172.21.15.153:52448) 2024-03-21T10:41:53.826 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:53.827+0000 7f58bbe00700 1 -- 172.21.15.153:0/2229019974 learned_addr learned my addr 172.21.15.153:0/2229019974 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:41:53.826 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:53.827+0000 7f58bb400700 1 --2- 172.21.15.153:0/2229019974 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f58bc10d280 0x7f58bc19d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:41:53.826 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:53.827+0000 7f58bbe00700 1 -- 172.21.15.153:0/2229019974 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f58bc10d280 msgr2=0x7f58bc19d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:41:53.826 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:53.827+0000 7f58bbe00700 1 --2- 172.21.15.153:0/2229019974 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f58bc10d280 0x7f58bc19d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:41:53.826 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:53.827+0000 7f58bbe00700 1 -- 172.21.15.153:0/2229019974 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58a80095f0 con 0x7f58bc1071d0 2024-03-21T10:41:53.826 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:53.828+0000 7f58bbe00700 1 --2- 172.21.15.153:0/2229019974 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f58bc1071d0 0x7f58bc19cd20 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f58ac00d8d0 tx=0x7f58ac00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:41:53.826 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:53.828+0000 7f58b8c00700 1 -- 172.21.15.153:0/2229019974 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f58ac009940 con 0x7f58bc1071d0 2024-03-21T10:41:53.826 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:53.828+0000 7f58b8c00700 1 -- 172.21.15.153:0/2229019974 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f58ac010460 con 0x7f58bc1071d0 2024-03-21T10:41:53.826 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:53.828+0000 7f58b8c00700 1 -- 172.21.15.153:0/2229019974 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f58ac00f5f0 con 0x7f58bc1071d0 2024-03-21T10:41:53.827 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:53.828+0000 7f58c2a00700 1 -- 172.21.15.153:0/2229019974 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58bc1a19b0 con 0x7f58bc1071d0 2024-03-21T10:41:53.827 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:53.828+0000 7f58c2a00700 1 -- 172.21.15.153:0/2229019974 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58bc1a1f00 con 0x7f58bc1071d0 2024-03-21T10:41:53.827 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:53.829+0000 7f58c2a00700 1 -- 172.21.15.153:0/2229019974 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58bc10f6c0 con 0x7f58bc1071d0 2024-03-21T10:41:53.827 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:53.830+0000 7f58b8c00700 1 -- 172.21.15.153:0/2229019974 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f58ac0105d0 con 0x7f58bc1071d0 2024-03-21T10:41:53.829 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:53.831+0000 7f58b8c00700 1 --2- 172.21.15.153:0/2229019974 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f58a006e460 0x7f58a0070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:41:53.829 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:53.831+0000 7f58b8c00700 1 -- 172.21.15.153:0/2229019974 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f58ac08e480 con 0x7f58bc1071d0 2024-03-21T10:41:53.829 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:53.831+0000 7f58bb400700 1 --2- 172.21.15.153:0/2229019974 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f58a006e460 0x7f58a0070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:41:53.829 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:53.832+0000 7f58bb400700 1 --2- 172.21.15.153:0/2229019974 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f58a006e460 0x7f58a0070920 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f58a8005b40 tx=0x7f58a801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:41:53.833 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:53.836+0000 7f58b8c00700 1 -- 172.21.15.153:0/2229019974 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f58ac057520 con 0x7f58bc1071d0 2024-03-21T10:41:54.125 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:54.128+0000 7f58c2a00700 1 -- 172.21.15.153:0/2229019974 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f58bc04ea90 con 0x7f58bc1071d0 2024-03-21T10:41:54.126 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:54.128+0000 7f58b8c00700 1 -- 172.21.15.153:0/2229019974 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4493 (secure 0 0 0) 0x7f58ac05abe0 con 0x7f58bc1071d0 2024-03-21T10:41:54.126 INFO:teuthology.orchestra.run.smithi153.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-03-21T10:41:54.127 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-03-21T10:41:54.127 INFO:teuthology.orchestra.run.smithi153.stdout: host smithi174 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi174/config 2024-03-21T10:41:54.127 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/podman: timeout after 895 seconds 2024-03-21T10:41:54.127 INFO:teuthology.orchestra.run.smithi153.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 -e NODE_NAME=smithi174 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/run/ceph:z -v /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpd910r3_i:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 inventory --format=json-pretty --filter-for-batch 2024-03-21T10:41:54.127 INFO:teuthology.orchestra.run.smithi153.stdout:Traceback (most recent call last): 2024-03-21T10:41:54.127 INFO:teuthology.orchestra.run.smithi153.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-03-21T10:41:54.127 INFO:teuthology.orchestra.run.smithi153.stdout: return _run_code(code, main_globals, None, 2024-03-21T10:41:54.127 INFO:teuthology.orchestra.run.smithi153.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-03-21T10:41:54.128 INFO:teuthology.orchestra.run.smithi153.stdout: exec(code, run_globals) 2024-03-21T10:41:54.128 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 5606, in 2024-03-21T10:41:54.128 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 5594, in main 2024-03-21T10:41:54.128 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 406, in _infer_config 2024-03-21T10:41:54.128 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 321, in _infer_fsid 2024-03-21T10:41:54.128 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 434, in _infer_image 2024-03-21T10:41:54.128 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 308, in _validate_fsid 2024-03-21T10:41:54.128 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 3357, in command_ceph_volume 2024-03-21T10:41:54.128 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-03-21T10:41:54.128 INFO:teuthology.orchestra.run.smithi153.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 -e NODE_NAME=smithi174 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/run/ceph:z -v /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpd910r3_i:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 inventory --format=json-pretty --filter-for-batch 2024-03-21T10:41:54.129 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-21T10:41:54.129 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is degraded 2024-03-21T10:41:54.129 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-21T10:41:54.129 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-21T10:41:54.129 INFO:teuthology.orchestra.run.smithi153.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-21T10:41:54.129 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-21T10:41:54.130 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:54.131+0000 7f58c2a00700 1 -- 172.21.15.153:0/2229019974 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f58a006e460 msgr2=0x7f58a0070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:41:54.130 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:54.131+0000 7f58c2a00700 1 --2- 172.21.15.153:0/2229019974 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f58a006e460 0x7f58a0070920 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f58a8005b40 tx=0x7f58a801a040 comp rx=0 tx=0).stop 2024-03-21T10:41:54.130 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:54.131+0000 7f58c2a00700 1 -- 172.21.15.153:0/2229019974 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f58bc1071d0 msgr2=0x7f58bc19cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:41:54.130 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:54.131+0000 7f58c2a00700 1 --2- 172.21.15.153:0/2229019974 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f58bc1071d0 0x7f58bc19cd20 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f58ac00d8d0 tx=0x7f58ac00dc90 comp rx=0 tx=0).stop 2024-03-21T10:41:54.130 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:54.131+0000 7f58c2a00700 1 -- 172.21.15.153:0/2229019974 shutdown_connections 2024-03-21T10:41:54.131 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:54.131+0000 7f58c2a00700 1 --2- 172.21.15.153:0/2229019974 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f58a006e460 0x7f58a0070920 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:41:54.131 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:54.131+0000 7f58c2a00700 1 --2- 172.21.15.153:0/2229019974 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f58bc1071d0 0x7f58bc19cd20 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:41:54.131 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:54.131+0000 7f58c2a00700 1 --2- 172.21.15.153:0/2229019974 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f58bc10d280 0x7f58bc19d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:41:54.131 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:54.131+0000 7f58c2a00700 1 -- 172.21.15.153:0/2229019974 >> 172.21.15.153:0/2229019974 conn(0x7f58bc076990 msgr2=0x7f58bc078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:41:54.131 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:54.132+0000 7f58c2a00700 1 -- 172.21.15.153:0/2229019974 shutdown_connections 2024-03-21T10:41:54.131 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:41:54.132+0000 7f58c2a00700 1 -- 172.21.15.153:0/2229019974 wait complete. 2024-03-21T10:41:54.302 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:41:54 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/1326421371' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:41:54.432 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:41:54 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/1326421371' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:41:55.302 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:41:55 smithi153 ceph-mon[113733]: from='client.35622 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:41:55.303 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:41:55 smithi153 ceph-mon[113733]: pgmap v903: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:41:55.303 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:41:55 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/2229019974' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:41:55.447 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:41:55 smithi174 ceph-mon[88004]: from='client.35622 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:41:55.447 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:41:55 smithi174 ceph-mon[88004]: pgmap v903: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:41:55.448 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:41:55 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/2229019974' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:41:57.302 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:41:57 smithi153 ceph-mon[113733]: pgmap v904: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:41:57.447 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:41:57 smithi174 ceph-mon[88004]: pgmap v904: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:41:59.302 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:41:59 smithi153 ceph-mon[113733]: pgmap v905: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:41:59.447 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:41:59 smithi174 ceph-mon[88004]: pgmap v905: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:42:01.302 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:42:01 smithi153 ceph-mon[113733]: pgmap v906: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:42:01.447 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:42:01 smithi174 ceph-mon[88004]: pgmap v906: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:42:02.302 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:42:02 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:42:02.447 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:42:02 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:42:03.302 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:42:03 smithi153 ceph-mon[113733]: pgmap v907: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:42:03.447 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:42:03 smithi174 ceph-mon[88004]: pgmap v907: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:42:05.303 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:42:05 smithi153 ceph-mon[113733]: pgmap v908: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:42:05.447 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:42:05 smithi174 ceph-mon[88004]: pgmap v908: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:42:07.302 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:42:07 smithi153 ceph-mon[113733]: pgmap v909: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:42:07.447 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:42:07 smithi174 ceph-mon[88004]: pgmap v909: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:42:09.447 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:42:09 smithi174 ceph-mon[88004]: pgmap v910: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:42:09.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:42:09 smithi153 ceph-mon[113733]: pgmap v910: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:42:11.447 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:42:11 smithi174 ceph-mon[88004]: pgmap v911: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:42:11.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:42:11 smithi153 ceph-mon[113733]: pgmap v911: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:42:13.447 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:42:13 smithi174 ceph-mon[88004]: pgmap v912: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:42:13.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:42:13 smithi153 ceph-mon[113733]: pgmap v912: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:42:15.447 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:42:15 smithi174 ceph-mon[88004]: pgmap v913: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:42:15.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:42:15 smithi153 ceph-mon[113733]: pgmap v913: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:42:17.447 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:42:17 smithi174 ceph-mon[88004]: pgmap v914: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:42:17.447 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:42:17 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:42:17.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:42:17 smithi153 ceph-mon[113733]: pgmap v914: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:42:17.553 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:42:17 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:42:19.447 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:42:19 smithi174 ceph-mon[88004]: pgmap v915: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:42:19.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:42:19 smithi153 ceph-mon[113733]: pgmap v915: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:42:21.447 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:42:21 smithi174 ceph-mon[88004]: pgmap v916: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:42:21.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:42:21 smithi153 ceph-mon[113733]: pgmap v916: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:42:23.447 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:42:23 smithi174 ceph-mon[88004]: pgmap v917: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:42:23.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:42:23 smithi153 ceph-mon[113733]: pgmap v917: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:42:24.297 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:24.299+0000 7f7964200700 1 -- 172.21.15.153:0/1967384926 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f795c10d280 msgr2=0x7f795c10d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:42:24.297 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:24.299+0000 7f7964200700 1 --2- 172.21.15.153:0/1967384926 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f795c10d280 0x7f795c10d660 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f7950009910 tx=0x7f7950009c20 comp rx=0 tx=0).stop 2024-03-21T10:42:24.297 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:24.299+0000 7f7964200700 1 -- 172.21.15.153:0/1967384926 shutdown_connections 2024-03-21T10:42:24.297 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:24.299+0000 7f7964200700 1 --2- 172.21.15.153:0/1967384926 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f795c1071d0 0x7f795c107650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:42:24.298 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:24.299+0000 7f7964200700 1 --2- 172.21.15.153:0/1967384926 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f795c10d280 0x7f795c10d660 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:42:24.298 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:24.299+0000 7f7964200700 1 -- 172.21.15.153:0/1967384926 >> 172.21.15.153:0/1967384926 conn(0x7f795c0769e0 msgr2=0x7f795c078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:42:24.298 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:24.300+0000 7f7964200700 1 -- 172.21.15.153:0/1967384926 shutdown_connections 2024-03-21T10:42:24.298 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:24.300+0000 7f7964200700 1 -- 172.21.15.153:0/1967384926 wait complete. 2024-03-21T10:42:24.299 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:24.301+0000 7f7964200700 1 Processor -- start 2024-03-21T10:42:24.299 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:24.301+0000 7f7964200700 1 -- start start 2024-03-21T10:42:24.299 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:24.301+0000 7f7964200700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f795c1071d0 0x7f795c19ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:42:24.299 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:24.301+0000 7f7964200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f795c10d280 0x7f795c19d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:42:24.299 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:24.301+0000 7f7964200700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f795c19da30 con 0x7f795c10d280 2024-03-21T10:42:24.299 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:24.301+0000 7f7964200700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f795c1a1770 con 0x7f795c1071d0 2024-03-21T10:42:24.299 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:24.301+0000 7f7960e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f795c10d280 0x7f795c19d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:42:24.299 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:24.302+0000 7f7960e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f795c10d280 0x7f795c19d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:40930/0 (socket says 172.21.15.153:40930) 2024-03-21T10:42:24.299 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:24.302+0000 7f7960e00700 1 -- 172.21.15.153:0/3641586718 learned_addr learned my addr 172.21.15.153:0/3641586718 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:42:24.300 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:24.302+0000 7f7961800700 1 --2- 172.21.15.153:0/3641586718 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f795c1071d0 0x7f795c19ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:42:24.300 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:24.302+0000 7f7960e00700 1 -- 172.21.15.153:0/3641586718 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f795c1071d0 msgr2=0x7f795c19ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:42:24.300 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:24.302+0000 7f7960e00700 1 --2- 172.21.15.153:0/3641586718 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f795c1071d0 0x7f795c19ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:42:24.301 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:24.302+0000 7f7960e00700 1 -- 172.21.15.153:0/3641586718 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79500095f0 con 0x7f795c10d280 2024-03-21T10:42:24.301 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:24.302+0000 7f7960e00700 1 --2- 172.21.15.153:0/3641586718 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f795c10d280 0x7f795c19d350 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f794800b4b0 tx=0x7f794800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:42:24.301 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:24.303+0000 7f795a000700 1 -- 172.21.15.153:0/3641586718 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7948011800 con 0x7f795c10d280 2024-03-21T10:42:24.301 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:24.303+0000 7f795a000700 1 -- 172.21.15.153:0/3641586718 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7948011e20 con 0x7f795c10d280 2024-03-21T10:42:24.301 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:24.303+0000 7f795a000700 1 -- 172.21.15.153:0/3641586718 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f794800f2e0 con 0x7f795c10d280 2024-03-21T10:42:24.301 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:24.303+0000 7f7964200700 1 -- 172.21.15.153:0/3641586718 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f795c1a1a50 con 0x7f795c10d280 2024-03-21T10:42:24.301 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:24.303+0000 7f7964200700 1 -- 172.21.15.153:0/3641586718 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f795c1a2020 con 0x7f795c10d280 2024-03-21T10:42:24.301 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:24.304+0000 7f7964200700 1 -- 172.21.15.153:0/3641586718 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f795c10f6f0 con 0x7f795c10d280 2024-03-21T10:42:24.302 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:24.305+0000 7f795a000700 1 -- 172.21.15.153:0/3641586718 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f7948010380 con 0x7f795c10d280 2024-03-21T10:42:24.304 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:24.306+0000 7f795a000700 1 --2- 172.21.15.153:0/3641586718 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f794406e460 0x7f7944070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:42:24.304 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:24.306+0000 7f795a000700 1 -- 172.21.15.153:0/3641586718 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f794808e190 con 0x7f795c10d280 2024-03-21T10:42:24.304 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:24.306+0000 7f7961800700 1 --2- 172.21.15.153:0/3641586718 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f794406e460 0x7f7944070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:42:24.304 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:24.307+0000 7f7961800700 1 --2- 172.21.15.153:0/3641586718 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f794406e460 0x7f7944070920 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f7950006010 tx=0x7f795001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:42:24.309 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:24.311+0000 7f795a000700 1 -- 172.21.15.153:0/3641586718 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f79480571b0 con 0x7f795c10d280 2024-03-21T10:42:24.544 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:24.546+0000 7f7964200700 1 -- 172.21.15.153:0/3641586718 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f795c1a2310 con 0x7f794406e460 2024-03-21T10:42:24.546 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:24.548+0000 7f795a000700 1 -- 172.21.15.153:0/3641586718 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f795c1a2310 con 0x7f794406e460 2024-03-21T10:42:24.549 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:24.551+0000 7f7964200700 1 -- 172.21.15.153:0/3641586718 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f794406e460 msgr2=0x7f7944070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:42:24.549 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:24.551+0000 7f7964200700 1 --2- 172.21.15.153:0/3641586718 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f794406e460 0x7f7944070920 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f7950006010 tx=0x7f795001a040 comp rx=0 tx=0).stop 2024-03-21T10:42:24.549 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:24.551+0000 7f7964200700 1 -- 172.21.15.153:0/3641586718 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f795c10d280 msgr2=0x7f795c19d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:42:24.549 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:24.551+0000 7f7964200700 1 --2- 172.21.15.153:0/3641586718 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f795c10d280 0x7f795c19d350 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f794800b4b0 tx=0x7f794800b870 comp rx=0 tx=0).stop 2024-03-21T10:42:24.549 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:24.551+0000 7f7964200700 1 -- 172.21.15.153:0/3641586718 shutdown_connections 2024-03-21T10:42:24.549 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:24.551+0000 7f7964200700 1 --2- 172.21.15.153:0/3641586718 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f795c1071d0 0x7f795c19ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:42:24.549 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:24.551+0000 7f7964200700 1 --2- 172.21.15.153:0/3641586718 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f794406e460 0x7f7944070920 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:42:24.550 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:24.551+0000 7f7964200700 1 --2- 172.21.15.153:0/3641586718 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f795c10d280 0x7f795c19d350 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:42:24.550 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:24.551+0000 7f7964200700 1 -- 172.21.15.153:0/3641586718 >> 172.21.15.153:0/3641586718 conn(0x7f795c0769e0 msgr2=0x7f795c0785c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:42:24.550 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:24.552+0000 7f7964200700 1 -- 172.21.15.153:0/3641586718 shutdown_connections 2024-03-21T10:42:24.550 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:24.552+0000 7f7964200700 1 -- 172.21.15.153:0/3641586718 wait complete. 2024-03-21T10:42:24.564 INFO:teuthology.orchestra.run.smithi153.stdout:true 2024-03-21T10:42:24.704 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:24.706+0000 7fe3e3c00700 1 -- 172.21.15.153:0/3980727784 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe3dc073ae0 msgr2=0x7fe3dc10ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:42:24.704 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:24.706+0000 7fe3e3c00700 1 --2- 172.21.15.153:0/3980727784 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe3dc073ae0 0x7fe3dc10ce40 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fe3c4009910 tx=0x7fe3c4009c20 comp rx=0 tx=0).stop 2024-03-21T10:42:24.705 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:24.706+0000 7fe3e3c00700 1 -- 172.21.15.153:0/3980727784 shutdown_connections 2024-03-21T10:42:24.705 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:24.706+0000 7fe3e3c00700 1 --2- 172.21.15.153:0/3980727784 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe3dc073ae0 0x7fe3dc10ce40 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:42:24.705 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:24.706+0000 7fe3e3c00700 1 --2- 172.21.15.153:0/3980727784 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe3dc0731c0 0x7fe3dc0735a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:42:24.705 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:24.706+0000 7fe3e3c00700 1 -- 172.21.15.153:0/3980727784 >> 172.21.15.153:0/3980727784 conn(0x7fe3dc0fc630 msgr2=0x7fe3dc0fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:42:24.705 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:24.707+0000 7fe3e3c00700 1 -- 172.21.15.153:0/3980727784 shutdown_connections 2024-03-21T10:42:24.705 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:24.707+0000 7fe3e3c00700 1 -- 172.21.15.153:0/3980727784 wait complete. 2024-03-21T10:42:24.707 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:24.708+0000 7fe3e3c00700 1 Processor -- start 2024-03-21T10:42:24.707 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:24.708+0000 7fe3e3c00700 1 -- start start 2024-03-21T10:42:24.708 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:24.708+0000 7fe3e3c00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe3dc0731c0 0x7fe3dc198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:42:24.708 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:24.708+0000 7fe3e3c00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe3dc073ae0 0x7fe3dc198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:42:24.708 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:24.708+0000 7fe3e3c00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3dc199530 con 0x7fe3dc0731c0 2024-03-21T10:42:24.708 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:24.708+0000 7fe3e3c00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3dc19d340 con 0x7fe3dc073ae0 2024-03-21T10:42:24.708 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:24.708+0000 7fe3e1200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe3dc0731c0 0x7fe3dc198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:42:24.708 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:24.709+0000 7fe3e1200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe3dc0731c0 0x7fe3dc198960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:40946/0 (socket says 172.21.15.153:40946) 2024-03-21T10:42:24.708 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:24.709+0000 7fe3e1200700 1 -- 172.21.15.153:0/1819499501 learned_addr learned my addr 172.21.15.153:0/1819499501 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:42:24.708 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:24.709+0000 7fe3d3e00700 1 --2- 172.21.15.153:0/1819499501 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe3dc073ae0 0x7fe3dc198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:42:24.708 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:24.709+0000 7fe3e1200700 1 -- 172.21.15.153:0/1819499501 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe3dc073ae0 msgr2=0x7fe3dc198ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:42:24.708 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:24.709+0000 7fe3e1200700 1 --2- 172.21.15.153:0/1819499501 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe3dc073ae0 0x7fe3dc198ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:42:24.708 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:24.709+0000 7fe3e1200700 1 -- 172.21.15.153:0/1819499501 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3c40095f0 con 0x7fe3dc0731c0 2024-03-21T10:42:24.708 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:24.709+0000 7fe3e1200700 1 --2- 172.21.15.153:0/1819499501 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe3dc0731c0 0x7fe3dc198960 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc00d8d0 tx=0x7fe3cc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:42:24.708 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:24.709+0000 7fe3da000700 1 -- 172.21.15.153:0/1819499501 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe3cc009940 con 0x7fe3dc0731c0 2024-03-21T10:42:24.708 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:24.710+0000 7fe3e3c00700 1 -- 172.21.15.153:0/1819499501 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3dc19d620 con 0x7fe3dc0731c0 2024-03-21T10:42:24.709 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:24.710+0000 7fe3da000700 1 -- 172.21.15.153:0/1819499501 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe3cc010460 con 0x7fe3dc0731c0 2024-03-21T10:42:24.709 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:24.710+0000 7fe3da000700 1 -- 172.21.15.153:0/1819499501 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe3cc00f5f0 con 0x7fe3dc0731c0 2024-03-21T10:42:24.709 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:24.710+0000 7fe3e3c00700 1 -- 172.21.15.153:0/1819499501 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3dc19db70 con 0x7fe3dc0731c0 2024-03-21T10:42:24.709 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:24.711+0000 7fe3e3c00700 1 -- 172.21.15.153:0/1819499501 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3dc10a540 con 0x7fe3dc0731c0 2024-03-21T10:42:24.709 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:24.712+0000 7fe3da000700 1 -- 172.21.15.153:0/1819499501 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fe3cc0105d0 con 0x7fe3dc0731c0 2024-03-21T10:42:24.710 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:24.713+0000 7fe3da000700 1 --2- 172.21.15.153:0/1819499501 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe3bc06e460 0x7fe3bc070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:42:24.710 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:24.713+0000 7fe3da000700 1 -- 172.21.15.153:0/1819499501 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fe3cc08e480 con 0x7fe3dc0731c0 2024-03-21T10:42:24.710 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:24.713+0000 7fe3d3e00700 1 --2- 172.21.15.153:0/1819499501 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe3bc06e460 0x7fe3bc070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:42:24.711 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:24.714+0000 7fe3d3e00700 1 --2- 172.21.15.153:0/1819499501 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe3bc06e460 0x7fe3bc070920 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fe3c4006010 tx=0x7fe3c400b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:42:24.715 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:24.718+0000 7fe3da000700 1 -- 172.21.15.153:0/1819499501 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe3cc05b050 con 0x7fe3dc0731c0 2024-03-21T10:42:24.948 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:24.950+0000 7fe3e3c00700 1 -- 172.21.15.153:0/1819499501 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe3dc19de50 con 0x7fe3bc06e460 2024-03-21T10:42:24.950 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:24.953+0000 7fe3da000700 1 -- 172.21.15.153:0/1819499501 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fe3dc19de50 con 0x7fe3bc06e460 2024-03-21T10:42:24.954 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:24.956+0000 7fe3e3c00700 1 -- 172.21.15.153:0/1819499501 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe3bc06e460 msgr2=0x7fe3bc070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:42:24.954 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:24.956+0000 7fe3e3c00700 1 --2- 172.21.15.153:0/1819499501 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe3bc06e460 0x7fe3bc070920 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fe3c4006010 tx=0x7fe3c400b540 comp rx=0 tx=0).stop 2024-03-21T10:42:24.954 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:24.956+0000 7fe3e3c00700 1 -- 172.21.15.153:0/1819499501 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe3dc0731c0 msgr2=0x7fe3dc198960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:42:24.954 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:24.956+0000 7fe3e3c00700 1 --2- 172.21.15.153:0/1819499501 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe3dc0731c0 0x7fe3dc198960 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc00d8d0 tx=0x7fe3cc00dc90 comp rx=0 tx=0).stop 2024-03-21T10:42:24.954 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:24.956+0000 7fe3e3c00700 1 -- 172.21.15.153:0/1819499501 shutdown_connections 2024-03-21T10:42:24.954 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:24.956+0000 7fe3e3c00700 1 --2- 172.21.15.153:0/1819499501 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe3bc06e460 0x7fe3bc070920 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:42:24.954 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:24.956+0000 7fe3e3c00700 1 --2- 172.21.15.153:0/1819499501 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe3dc0731c0 0x7fe3dc198960 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:42:24.954 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:24.956+0000 7fe3e3c00700 1 --2- 172.21.15.153:0/1819499501 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe3dc073ae0 0x7fe3dc198ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:42:24.954 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:24.956+0000 7fe3e3c00700 1 -- 172.21.15.153:0/1819499501 >> 172.21.15.153:0/1819499501 conn(0x7fe3dc0fc630 msgr2=0x7fe3dc107680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:42:24.954 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:24.956+0000 7fe3e3c00700 1 -- 172.21.15.153:0/1819499501 shutdown_connections 2024-03-21T10:42:24.954 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:24.956+0000 7fe3e3c00700 1 -- 172.21.15.153:0/1819499501 wait complete. 2024-03-21T10:42:25.108 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.111+0000 7fe67a000700 1 -- 172.21.15.153:0/2146356422 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe674102e10 msgr2=0x7fe6741031f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:42:25.109 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.111+0000 7fe67a000700 1 --2- 172.21.15.153:0/2146356422 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe674102e10 0x7fe6741031f0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fe668009910 tx=0x7fe668009c20 comp rx=0 tx=0).stop 2024-03-21T10:42:25.109 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.111+0000 7fe67a000700 1 -- 172.21.15.153:0/2146356422 shutdown_connections 2024-03-21T10:42:25.109 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.111+0000 7fe67a000700 1 --2- 172.21.15.153:0/2146356422 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe6741037c0 0x7fe674107810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:42:25.109 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.111+0000 7fe67a000700 1 --2- 172.21.15.153:0/2146356422 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe674102e10 0x7fe6741031f0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:42:25.109 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.111+0000 7fe67a000700 1 -- 172.21.15.153:0/2146356422 >> 172.21.15.153:0/2146356422 conn(0x7fe6740fe6c0 msgr2=0x7fe674100ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:42:25.109 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.111+0000 7fe67a000700 1 -- 172.21.15.153:0/2146356422 shutdown_connections 2024-03-21T10:42:25.109 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.111+0000 7fe67a000700 1 -- 172.21.15.153:0/2146356422 wait complete. 2024-03-21T10:42:25.110 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.112+0000 7fe67a000700 1 Processor -- start 2024-03-21T10:42:25.110 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.112+0000 7fe67a000700 1 -- start start 2024-03-21T10:42:25.110 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.112+0000 7fe67a000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe674102e10 0x7fe6740752a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:42:25.110 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.112+0000 7fe67a000700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe6741037c0 0x7fe6740757e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:42:25.110 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.112+0000 7fe67a000700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe674079430 con 0x7fe674102e10 2024-03-21T10:42:25.110 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.112+0000 7fe67a000700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe674075d20 con 0x7fe6741037c0 2024-03-21T10:42:25.111 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.113+0000 7fe678c00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe674102e10 0x7fe6740752a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:42:25.111 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.113+0000 7fe678c00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe674102e10 0x7fe6740752a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:40972/0 (socket says 172.21.15.153:40972) 2024-03-21T10:42:25.111 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.113+0000 7fe678c00700 1 -- 172.21.15.153:0/2747658211 learned_addr learned my addr 172.21.15.153:0/2747658211 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:42:25.111 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.113+0000 7fe678c00700 1 -- 172.21.15.153:0/2747658211 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe6741037c0 msgr2=0x7fe6740757e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:42:25.111 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.113+0000 7fe678c00700 1 --2- 172.21.15.153:0/2747658211 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe6741037c0 0x7fe6740757e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:42:25.111 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.113+0000 7fe678c00700 1 -- 172.21.15.153:0/2747658211 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6680095f0 con 0x7fe674102e10 2024-03-21T10:42:25.111 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.113+0000 7fe678c00700 1 --2- 172.21.15.153:0/2747658211 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe674102e10 0x7fe6740752a0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fe6680092e0 tx=0x7fe66800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:42:25.112 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.115+0000 7fe671600700 1 -- 172.21.15.153:0/2747658211 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe66801d070 con 0x7fe674102e10 2024-03-21T10:42:25.112 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.115+0000 7fe671600700 1 -- 172.21.15.153:0/2747658211 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe668005e00 con 0x7fe674102e10 2024-03-21T10:42:25.112 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.115+0000 7fe671600700 1 -- 172.21.15.153:0/2747658211 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe668017400 con 0x7fe674102e10 2024-03-21T10:42:25.113 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.115+0000 7fe67a000700 1 -- 172.21.15.153:0/2747658211 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe674075fa0 con 0x7fe674102e10 2024-03-21T10:42:25.113 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.115+0000 7fe67a000700 1 -- 172.21.15.153:0/2747658211 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6741a6f30 con 0x7fe674102e10 2024-03-21T10:42:25.113 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.116+0000 7fe67a000700 1 -- 172.21.15.153:0/2747658211 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe67410b1b0 con 0x7fe674102e10 2024-03-21T10:42:25.114 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.117+0000 7fe671600700 1 -- 172.21.15.153:0/2747658211 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fe66800f460 con 0x7fe674102e10 2024-03-21T10:42:25.115 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.118+0000 7fe671600700 1 --2- 172.21.15.153:0/2747658211 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe65c06e460 0x7fe65c070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:42:25.116 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.118+0000 7fe671600700 1 -- 172.21.15.153:0/2747658211 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fe66808fa60 con 0x7fe674102e10 2024-03-21T10:42:25.116 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.118+0000 7fe673e00700 1 --2- 172.21.15.153:0/2747658211 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe65c06e460 0x7fe65c070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:42:25.116 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.119+0000 7fe673e00700 1 --2- 172.21.15.153:0/2747658211 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe65c06e460 0x7fe65c070920 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fe660005950 tx=0x7fe6600058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:42:25.120 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.123+0000 7fe671600700 1 -- 172.21.15.153:0/2747658211 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe668058a80 con 0x7fe674102e10 2024-03-21T10:42:25.302 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:42:25 smithi153 ceph-mon[113733]: pgmap v918: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:42:25.302 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:42:25 smithi153 ceph-mon[113733]: from='client.35630 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:42:25.352 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.355+0000 7fe67a000700 1 -- 172.21.15.153:0/2747658211 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fe67402cd80 con 0x7fe65c06e460 2024-03-21T10:42:25.372 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.374+0000 7fe671600700 1 -- 172.21.15.153:0/2747658211 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fe67402cd80 con 0x7fe65c06e460 2024-03-21T10:42:25.372 INFO:teuthology.orchestra.run.smithi153.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-21T10:42:25.373 INFO:teuthology.orchestra.run.smithi153.stdout:alertmanager.smithi153 smithi153 *:9093,9094 running (31m) 6m ago 44m 15.6M - 0.25.0 c8568f914cd2 9ed3f2200b5f 2024-03-21T10:42:25.373 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi153 smithi153 running (45m) 6m ago 45m 10.4M - 18.2.1 5be31c24972a 5b61994396a6 2024-03-21T10:42:25.373 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi174 smithi174 running (43m) 7m ago 43m 10.1M - 18.2.1 5be31c24972a 4f7a5fc3ac28 2024-03-21T10:42:25.373 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi153 smithi153 running (27m) 6m ago 45m 6656k - 19.0.0-2231-g2edfee68 237e43c7a65f 32c21104d5b0 2024-03-21T10:42:25.373 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi174 smithi174 running (27m) 7m ago 43m 6660k - 19.0.0-2231-g2edfee68 237e43c7a65f a04e48967ab6 2024-03-21T10:42:25.373 INFO:teuthology.orchestra.run.smithi153.stdout:grafana.smithi153 smithi153 *:3000 running (30m) 6m ago 44m 75.2M - 9.4.12 2bacad6d85d8 ee24026ab786 2024-03-21T10:42:25.373 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.iuyluj smithi153 running (22m) 6m ago 37m 18.7M - 19.0.0-2231-g2edfee68 237e43c7a65f 87aff8a5de70 2024-03-21T10:42:25.373 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.zipckr smithi153 running (23m) 6m ago 37m 18.9M - 19.0.0-2231-g2edfee68 237e43c7a65f ccc262bd5dd3 2024-03-21T10:42:25.373 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.nhozag smithi174 running (7m) 7m ago 37m 10.9M - 19.0.0-2231-g2edfee68 237e43c7a65f 9e02aea72c96 2024-03-21T10:42:25.373 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.yyimst smithi174 running (22m) 7m ago 37m 18.5M - 19.0.0-2231-g2edfee68 237e43c7a65f fc1962f9351e 2024-03-21T10:42:25.373 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi153.tluxni smithi153 *:8443,9283,8765 running (34m) 6m ago 46m 512M - 19.0.0-2231-g2edfee68 237e43c7a65f 7ffee0e55382 2024-03-21T10:42:25.373 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi174.qpcwat smithi174 *:8443,9283,8765 running (32m) 7m ago 43m 459M - 19.0.0-2231-g2edfee68 237e43c7a65f 748968f216ab 2024-03-21T10:42:25.373 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi153 smithi153 running (29m) 6m ago 47m 70.8M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f dd8038303dc3 2024-03-21T10:42:25.373 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi174 smithi174 running (28m) 7m ago 43m 51.9M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f 29d78a788f2c 2024-03-21T10:42:25.374 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi153 smithi153 *:9100 running (32m) 6m ago 45m 8140k - 1.5.0 0da6a335fe13 bb835edc0e0b 2024-03-21T10:42:25.374 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi174 smithi174 *:9100 running (32m) 7m ago 43m 8610k - 1.5.0 0da6a335fe13 7c4ecad790fa 2024-03-21T10:42:25.374 INFO:teuthology.orchestra.run.smithi153.stdout:osd.0 smithi153 running (26m) 6m ago 42m 118M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 507900fd4791 2024-03-21T10:42:25.374 INFO:teuthology.orchestra.run.smithi153.stdout:osd.1 smithi153 running (26m) 6m ago 41m 93.0M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 0d646452e9c1 2024-03-21T10:42:25.374 INFO:teuthology.orchestra.run.smithi153.stdout:osd.2 smithi153 running (25m) 6m ago 41m 91.2M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 1326f2afdfcd 2024-03-21T10:42:25.374 INFO:teuthology.orchestra.run.smithi153.stdout:osd.3 smithi174 running (25m) 7m ago 40m 87.5M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 70aafac71ad5 2024-03-21T10:42:25.374 INFO:teuthology.orchestra.run.smithi153.stdout:osd.4 smithi174 running (24m) 7m ago 40m 95.8M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 0d746ce6e79d 2024-03-21T10:42:25.374 INFO:teuthology.orchestra.run.smithi153.stdout:osd.5 smithi174 running (23m) 7m ago 39m 90.3M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 53e5f35b8cac 2024-03-21T10:42:25.374 INFO:teuthology.orchestra.run.smithi153.stdout:prometheus.smithi153 smithi153 *:9095 running (31m) 6m ago 44m 88.1M - 2.43.0 a07b618ecd1d 7d95561e50c2 2024-03-21T10:42:25.376 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.378+0000 7fe67a000700 1 -- 172.21.15.153:0/2747658211 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe65c06e460 msgr2=0x7fe65c070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:42:25.376 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.378+0000 7fe67a000700 1 --2- 172.21.15.153:0/2747658211 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe65c06e460 0x7fe65c070920 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fe660005950 tx=0x7fe6600058e0 comp rx=0 tx=0).stop 2024-03-21T10:42:25.376 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.378+0000 7fe67a000700 1 -- 172.21.15.153:0/2747658211 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe674102e10 msgr2=0x7fe6740752a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:42:25.376 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.378+0000 7fe67a000700 1 --2- 172.21.15.153:0/2747658211 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe674102e10 0x7fe6740752a0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fe6680092e0 tx=0x7fe66800be20 comp rx=0 tx=0).stop 2024-03-21T10:42:25.376 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.378+0000 7fe67a000700 1 -- 172.21.15.153:0/2747658211 shutdown_connections 2024-03-21T10:42:25.376 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.378+0000 7fe67a000700 1 --2- 172.21.15.153:0/2747658211 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe65c06e460 0x7fe65c070920 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:42:25.376 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.378+0000 7fe67a000700 1 --2- 172.21.15.153:0/2747658211 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe674102e10 0x7fe6740752a0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:42:25.376 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.378+0000 7fe67a000700 1 --2- 172.21.15.153:0/2747658211 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe6741037c0 0x7fe6740757e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:42:25.376 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.378+0000 7fe67a000700 1 -- 172.21.15.153:0/2747658211 >> 172.21.15.153:0/2747658211 conn(0x7fe6740fe6c0 msgr2=0x7fe6740ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:42:25.377 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.378+0000 7fe67a000700 1 -- 172.21.15.153:0/2747658211 shutdown_connections 2024-03-21T10:42:25.377 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.379+0000 7fe67a000700 1 -- 172.21.15.153:0/2747658211 wait complete. 2024-03-21T10:42:25.389 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:42:25 smithi174 ceph-mon[88004]: pgmap v918: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:42:25.389 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:42:25 smithi174 ceph-mon[88004]: from='client.35630 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:42:25.521 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.523+0000 7fce38c00700 1 -- 172.21.15.153:0/460665060 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fce34069020 msgr2=0x7fce34105790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:42:25.521 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.523+0000 7fce38c00700 1 --2- 172.21.15.153:0/460665060 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fce34069020 0x7fce34105790 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fce20009910 tx=0x7fce20009c20 comp rx=0 tx=0).stop 2024-03-21T10:42:25.522 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.523+0000 7fce38c00700 1 -- 172.21.15.153:0/460665060 shutdown_connections 2024-03-21T10:42:25.522 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.523+0000 7fce38c00700 1 --2- 172.21.15.153:0/460665060 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fce34069020 0x7fce34105790 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:42:25.522 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.523+0000 7fce38c00700 1 --2- 172.21.15.153:0/460665060 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fce34068670 0x7fce34068a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:42:25.522 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.523+0000 7fce38c00700 1 -- 172.21.15.153:0/460665060 >> 172.21.15.153:0/460665060 conn(0x7fce340755a0 msgr2=0x7fce340759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:42:25.522 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.524+0000 7fce38c00700 1 -- 172.21.15.153:0/460665060 shutdown_connections 2024-03-21T10:42:25.522 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.524+0000 7fce38c00700 1 -- 172.21.15.153:0/460665060 wait complete. 2024-03-21T10:42:25.523 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.525+0000 7fce38c00700 1 Processor -- start 2024-03-21T10:42:25.523 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.525+0000 7fce38c00700 1 -- start start 2024-03-21T10:42:25.523 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.525+0000 7fce38c00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fce34068670 0x7fce340ffca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:42:25.524 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.525+0000 7fce38c00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fce34069020 0x7fce341001e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:42:25.524 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.525+0000 7fce38c00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce34100720 con 0x7fce34069020 2024-03-21T10:42:25.524 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.525+0000 7fce38c00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce34100860 con 0x7fce34068670 2024-03-21T10:42:25.524 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.525+0000 7fce31400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fce34069020 0x7fce341001e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:42:25.524 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.526+0000 7fce31400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fce34069020 0x7fce341001e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:40994/0 (socket says 172.21.15.153:40994) 2024-03-21T10:42:25.524 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.526+0000 7fce31400700 1 -- 172.21.15.153:0/2759186669 learned_addr learned my addr 172.21.15.153:0/2759186669 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:42:25.524 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.526+0000 7fce31e00700 1 --2- 172.21.15.153:0/2759186669 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fce34068670 0x7fce340ffca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:42:25.524 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.526+0000 7fce31400700 1 -- 172.21.15.153:0/2759186669 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fce34068670 msgr2=0x7fce340ffca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:42:25.524 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.526+0000 7fce31400700 1 --2- 172.21.15.153:0/2759186669 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fce34068670 0x7fce340ffca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:42:25.524 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.526+0000 7fce31400700 1 -- 172.21.15.153:0/2759186669 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce200095f0 con 0x7fce34069020 2024-03-21T10:42:25.525 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.526+0000 7fce31400700 1 --2- 172.21.15.153:0/2759186669 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fce34069020 0x7fce341001e0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fce2000bda0 tx=0x7fce2000be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:42:25.525 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.526+0000 7fce2aa00700 1 -- 172.21.15.153:0/2759186669 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fce2001d070 con 0x7fce34069020 2024-03-21T10:42:25.525 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.527+0000 7fce2aa00700 1 -- 172.21.15.153:0/2759186669 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fce2000f460 con 0x7fce34069020 2024-03-21T10:42:25.525 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.527+0000 7fce38c00700 1 -- 172.21.15.153:0/2759186669 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce34100ae0 con 0x7fce34069020 2024-03-21T10:42:25.525 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.527+0000 7fce2aa00700 1 -- 172.21.15.153:0/2759186669 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fce20017610 con 0x7fce34069020 2024-03-21T10:42:25.525 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.527+0000 7fce38c00700 1 -- 172.21.15.153:0/2759186669 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce341a2c00 con 0x7fce34069020 2024-03-21T10:42:25.525 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.528+0000 7fce38c00700 1 -- 172.21.15.153:0/2759186669 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce341090a0 con 0x7fce34069020 2024-03-21T10:42:25.526 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.529+0000 7fce2aa00700 1 -- 172.21.15.153:0/2759186669 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fce2000fa40 con 0x7fce34069020 2024-03-21T10:42:25.527 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.530+0000 7fce2aa00700 1 --2- 172.21.15.153:0/2759186669 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fce1806e460 0x7fce18070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:42:25.527 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.530+0000 7fce2aa00700 1 -- 172.21.15.153:0/2759186669 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fce2008fa10 con 0x7fce34069020 2024-03-21T10:42:25.527 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.530+0000 7fce31e00700 1 --2- 172.21.15.153:0/2759186669 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fce1806e460 0x7fce18070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:42:25.528 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.531+0000 7fce31e00700 1 --2- 172.21.15.153:0/2759186669 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fce1806e460 0x7fce18070920 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fce24005950 tx=0x7fce240058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:42:25.532 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.534+0000 7fce2aa00700 1 -- 172.21.15.153:0/2759186669 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fce20058a30 con 0x7fce34069020 2024-03-21T10:42:25.822 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.825+0000 7fce38c00700 1 -- 172.21.15.153:0/2759186669 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fce3404ea90 con 0x7fce34069020 2024-03-21T10:42:25.824 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.826+0000 7fce2aa00700 1 -- 172.21.15.153:0/2759186669 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fce2008de60 con 0x7fce34069020 2024-03-21T10:42:25.824 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:42:25.824 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": { 2024-03-21T10:42:25.824 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:42:25.824 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:42:25.824 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": { 2024-03-21T10:42:25.824 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:42:25.824 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:42:25.824 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": { 2024-03-21T10:42:25.824 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 6 2024-03-21T10:42:25.824 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:42:25.825 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": { 2024-03-21T10:42:25.825 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 4 2024-03-21T10:42:25.825 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:42:25.825 INFO:teuthology.orchestra.run.smithi153.stdout: "overall": { 2024-03-21T10:42:25.825 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 14 2024-03-21T10:42:25.825 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-03-21T10:42:25.825 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:42:25.826 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.829+0000 7fce38c00700 1 -- 172.21.15.153:0/2759186669 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fce1806e460 msgr2=0x7fce18070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:42:25.826 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.829+0000 7fce38c00700 1 --2- 172.21.15.153:0/2759186669 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fce1806e460 0x7fce18070920 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fce24005950 tx=0x7fce240058e0 comp rx=0 tx=0).stop 2024-03-21T10:42:25.827 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.829+0000 7fce38c00700 1 -- 172.21.15.153:0/2759186669 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fce34069020 msgr2=0x7fce341001e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:42:25.827 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.829+0000 7fce38c00700 1 --2- 172.21.15.153:0/2759186669 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fce34069020 0x7fce341001e0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fce2000bda0 tx=0x7fce2000be80 comp rx=0 tx=0).stop 2024-03-21T10:42:25.830 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.829+0000 7fce38c00700 1 -- 172.21.15.153:0/2759186669 shutdown_connections 2024-03-21T10:42:25.830 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.829+0000 7fce38c00700 1 --2- 172.21.15.153:0/2759186669 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fce34068670 0x7fce340ffca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:42:25.830 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.829+0000 7fce38c00700 1 --2- 172.21.15.153:0/2759186669 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fce1806e460 0x7fce18070920 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:42:25.830 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.829+0000 7fce38c00700 1 --2- 172.21.15.153:0/2759186669 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fce34069020 0x7fce341001e0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:42:25.830 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.829+0000 7fce38c00700 1 -- 172.21.15.153:0/2759186669 >> 172.21.15.153:0/2759186669 conn(0x7fce340755a0 msgr2=0x7fce34105000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:42:25.830 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.830+0000 7fce38c00700 1 -- 172.21.15.153:0/2759186669 shutdown_connections 2024-03-21T10:42:25.830 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.830+0000 7fce38c00700 1 -- 172.21.15.153:0/2759186669 wait complete. 2024-03-21T10:42:25.985 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.987+0000 7f1044c00700 1 -- 172.21.15.153:0/4010879761 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1040107220 msgr2=0x7f1040107600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:42:25.985 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.987+0000 7f1044c00700 1 --2- 172.21.15.153:0/4010879761 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1040107220 0x7f1040107600 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f1030009910 tx=0x7f1030009c20 comp rx=0 tx=0).stop 2024-03-21T10:42:25.985 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.987+0000 7f1044c00700 1 -- 172.21.15.153:0/4010879761 shutdown_connections 2024-03-21T10:42:25.985 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.987+0000 7f1044c00700 1 --2- 172.21.15.153:0/4010879761 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1040107bd0 0x7f104010bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:42:25.985 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.987+0000 7f1044c00700 1 --2- 172.21.15.153:0/4010879761 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1040107220 0x7f1040107600 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:42:25.985 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.987+0000 7f1044c00700 1 -- 172.21.15.153:0/4010879761 >> 172.21.15.153:0/4010879761 conn(0x7f10400767e0 msgr2=0x7f1040078c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:42:25.985 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.987+0000 7f1044c00700 1 -- 172.21.15.153:0/4010879761 shutdown_connections 2024-03-21T10:42:25.985 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.987+0000 7f1044c00700 1 -- 172.21.15.153:0/4010879761 wait complete. 2024-03-21T10:42:25.986 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.988+0000 7f1044c00700 1 Processor -- start 2024-03-21T10:42:25.986 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.988+0000 7f1044c00700 1 -- start start 2024-03-21T10:42:25.986 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.988+0000 7f1044c00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1040107220 0x7f104019cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:42:25.986 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.988+0000 7f1044c00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1040107bd0 0x7f104019d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:42:25.986 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.988+0000 7f1044c00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f104019d960 con 0x7f1040107220 2024-03-21T10:42:25.987 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.988+0000 7f1044c00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10401a16f0 con 0x7f1040107bd0 2024-03-21T10:42:25.987 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.989+0000 7f103f400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1040107220 0x7f104019cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:42:25.987 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.989+0000 7f103f400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1040107220 0x7f104019cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41004/0 (socket says 172.21.15.153:41004) 2024-03-21T10:42:25.987 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.989+0000 7f103f400700 1 -- 172.21.15.153:0/1732387116 learned_addr learned my addr 172.21.15.153:0/1732387116 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:42:25.988 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.989+0000 7f103f400700 1 -- 172.21.15.153:0/1732387116 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1040107bd0 msgr2=0x7f104019d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:42:25.988 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.989+0000 7f103f400700 1 --2- 172.21.15.153:0/1732387116 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1040107bd0 0x7f104019d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:42:25.988 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.989+0000 7f103f400700 1 -- 172.21.15.153:0/1732387116 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10300095f0 con 0x7f1040107220 2024-03-21T10:42:25.988 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.989+0000 7f103f400700 1 --2- 172.21.15.153:0/1732387116 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1040107220 0x7f104019cd40 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f103000b5c0 tx=0x7f103000be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:42:25.988 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.990+0000 7f1037e00700 1 -- 172.21.15.153:0/1732387116 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f103001d070 con 0x7f1040107220 2024-03-21T10:42:25.988 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.990+0000 7f1044c00700 1 -- 172.21.15.153:0/1732387116 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10401a1970 con 0x7f1040107220 2024-03-21T10:42:25.988 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.990+0000 7f1037e00700 1 -- 172.21.15.153:0/1732387116 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f103000f460 con 0x7f1040107220 2024-03-21T10:42:25.988 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.990+0000 7f1037e00700 1 -- 172.21.15.153:0/1732387116 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1030017610 con 0x7f1040107220 2024-03-21T10:42:25.988 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.990+0000 7f1044c00700 1 -- 172.21.15.153:0/1732387116 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10401a1e60 con 0x7f1040107220 2024-03-21T10:42:25.989 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.991+0000 7f1044c00700 1 -- 172.21.15.153:0/1732387116 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f104010f5b0 con 0x7f1040107220 2024-03-21T10:42:25.989 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.992+0000 7f1037e00700 1 -- 172.21.15.153:0/1732387116 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f103000f5d0 con 0x7f1040107220 2024-03-21T10:42:25.990 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.993+0000 7f1037e00700 1 --2- 172.21.15.153:0/1732387116 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f102406e460 0x7f1024070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:42:25.990 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.993+0000 7f1037e00700 1 -- 172.21.15.153:0/1732387116 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f103008fad0 con 0x7f1040107220 2024-03-21T10:42:25.991 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.993+0000 7f103ea00700 1 --2- 172.21.15.153:0/1732387116 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f102406e460 0x7f1024070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:42:25.991 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.994+0000 7f103ea00700 1 --2- 172.21.15.153:0/1732387116 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f102406e460 0x7f1024070920 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f104019e360 tx=0x7f102c006d20 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:42:25.995 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:25.998+0000 7f1037e00700 1 -- 172.21.15.153:0/1732387116 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f1030058b70 con 0x7f1040107220 2024-03-21T10:42:26.254 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:26.256+0000 7f1044c00700 1 -- 172.21.15.153:0/1732387116 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f104004f2e0 con 0x7f1040107220 2024-03-21T10:42:26.254 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:26.256+0000 7f1037e00700 1 -- 172.21.15.153:0/1732387116 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1876 (secure 0 0 0) 0x7f103005c230 con 0x7f1040107220 2024-03-21T10:42:26.255 INFO:teuthology.orchestra.run.smithi153.stdout:e32 2024-03-21T10:42:26.255 INFO:teuthology.orchestra.run.smithi153.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-21T10:42:26.255 INFO:teuthology.orchestra.run.smithi153.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:42:26.255 INFO:teuthology.orchestra.run.smithi153.stdout:legacy client fscid: 1 2024-03-21T10:42:26.255 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:42:26.255 INFO:teuthology.orchestra.run.smithi153.stdout:Filesystem 'cephfs' (1) 2024-03-21T10:42:26.256 INFO:teuthology.orchestra.run.smithi153.stdout:fs_name cephfs 2024-03-21T10:42:26.256 INFO:teuthology.orchestra.run.smithi153.stdout:epoch 23 2024-03-21T10:42:26.256 INFO:teuthology.orchestra.run.smithi153.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-21T10:42:26.256 INFO:teuthology.orchestra.run.smithi153.stdout:created 2024-03-21T10:04:27.086576+0000 2024-03-21T10:42:26.256 INFO:teuthology.orchestra.run.smithi153.stdout:modified 2024-03-21T10:19:05.022092+0000 2024-03-21T10:42:26.256 INFO:teuthology.orchestra.run.smithi153.stdout:tableserver 0 2024-03-21T10:42:26.256 INFO:teuthology.orchestra.run.smithi153.stdout:root 0 2024-03-21T10:42:26.256 INFO:teuthology.orchestra.run.smithi153.stdout:session_timeout 60 2024-03-21T10:42:26.256 INFO:teuthology.orchestra.run.smithi153.stdout:session_autoclose 300 2024-03-21T10:42:26.256 INFO:teuthology.orchestra.run.smithi153.stdout:max_file_size 1099511627776 2024-03-21T10:42:26.256 INFO:teuthology.orchestra.run.smithi153.stdout:max_xattr_size 65536 2024-03-21T10:42:26.256 INFO:teuthology.orchestra.run.smithi153.stdout:required_client_features {} 2024-03-21T10:42:26.256 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure 0 2024-03-21T10:42:26.257 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure_osd_epoch 94 2024-03-21T10:42:26.257 INFO:teuthology.orchestra.run.smithi153.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:42:26.257 INFO:teuthology.orchestra.run.smithi153.stdout:max_mds 2 2024-03-21T10:42:26.257 INFO:teuthology.orchestra.run.smithi153.stdout:in 0,1 2024-03-21T10:42:26.257 INFO:teuthology.orchestra.run.smithi153.stdout:up {} 2024-03-21T10:42:26.257 INFO:teuthology.orchestra.run.smithi153.stdout:failed 0,1 2024-03-21T10:42:26.257 INFO:teuthology.orchestra.run.smithi153.stdout:damaged 2024-03-21T10:42:26.257 INFO:teuthology.orchestra.run.smithi153.stdout:stopped 2024-03-21T10:42:26.257 INFO:teuthology.orchestra.run.smithi153.stdout:data_pools [3] 2024-03-21T10:42:26.257 INFO:teuthology.orchestra.run.smithi153.stdout:metadata_pool 2 2024-03-21T10:42:26.257 INFO:teuthology.orchestra.run.smithi153.stdout:inline_data enabled 2024-03-21T10:42:26.257 INFO:teuthology.orchestra.run.smithi153.stdout:balancer 2024-03-21T10:42:26.257 INFO:teuthology.orchestra.run.smithi153.stdout:bal_rank_mask -1 2024-03-21T10:42:26.257 INFO:teuthology.orchestra.run.smithi153.stdout:standby_count_wanted 1 2024-03-21T10:42:26.257 INFO:teuthology.orchestra.run.smithi153.stdout:qdb_cluster leader: 0 members: 2024-03-21T10:42:26.257 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:42:26.258 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:42:26.258 INFO:teuthology.orchestra.run.smithi153.stdout:Standby daemons: 2024-03-21T10:42:26.258 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:42:26.258 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.zipckr{-1:34442} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6826/1103846295,v1:172.21.15.153:6827/1103846295] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:42:26.258 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.iuyluj{-1:34474} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6828/1738630832,v1:172.21.15.153:6829/1738630832] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:42:26.258 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.yyimst{-1:44313} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.174:6824/4127786797,v1:172.21.15.174:6825/4127786797] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:42:26.258 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.nhozag{-1:44687} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.174:6826/2767517845,v1:172.21.15.174:6827/2767517845] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:42:26.259 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:26.260+0000 7f1044c00700 1 -- 172.21.15.153:0/1732387116 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f102406e460 msgr2=0x7f1024070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:42:26.259 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:26.260+0000 7f1044c00700 1 --2- 172.21.15.153:0/1732387116 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f102406e460 0x7f1024070920 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f104019e360 tx=0x7f102c006d20 comp rx=0 tx=0).stop 2024-03-21T10:42:26.259 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:26.260+0000 7f1044c00700 1 -- 172.21.15.153:0/1732387116 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1040107220 msgr2=0x7f104019cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:42:26.259 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:26.260+0000 7f1044c00700 1 --2- 172.21.15.153:0/1732387116 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1040107220 0x7f104019cd40 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f103000b5c0 tx=0x7f103000be00 comp rx=0 tx=0).stop 2024-03-21T10:42:26.259 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:26.260+0000 7f1044c00700 1 -- 172.21.15.153:0/1732387116 shutdown_connections 2024-03-21T10:42:26.259 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:26.260+0000 7f1044c00700 1 --2- 172.21.15.153:0/1732387116 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f102406e460 0x7f1024070920 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:42:26.259 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:26.260+0000 7f1044c00700 1 --2- 172.21.15.153:0/1732387116 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1040107220 0x7f104019cd40 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:42:26.259 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:26.260+0000 7f1044c00700 1 --2- 172.21.15.153:0/1732387116 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1040107bd0 0x7f104019d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:42:26.259 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:26.260+0000 7f1044c00700 1 -- 172.21.15.153:0/1732387116 >> 172.21.15.153:0/1732387116 conn(0x7f10400767e0 msgr2=0x7f1040077ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:42:26.260 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:26.261+0000 7f1044c00700 1 -- 172.21.15.153:0/1732387116 shutdown_connections 2024-03-21T10:42:26.260 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:26.261+0000 7f1044c00700 1 -- 172.21.15.153:0/1732387116 wait complete. 2024-03-21T10:42:26.260 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 32 2024-03-21T10:42:26.414 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:26.416+0000 7f3037800700 1 -- 172.21.15.153:0/1151119436 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f30301039b0 msgr2=0x7f3030107a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:42:26.415 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:26.416+0000 7f3037800700 1 --2- 172.21.15.153:0/1151119436 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f30301039b0 0x7f3030107a00 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f301c009910 tx=0x7f301c009c20 comp rx=0 tx=0).stop 2024-03-21T10:42:26.415 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:42:26 smithi153 ceph-mon[113733]: from='client.35634 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:42:26.415 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:42:26 smithi153 ceph-mon[113733]: from='client.35638 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:42:26.415 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:42:26 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/2759186669' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:42:26.416 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:26.417+0000 7f3037800700 1 -- 172.21.15.153:0/1151119436 shutdown_connections 2024-03-21T10:42:26.416 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:26.417+0000 7f3037800700 1 --2- 172.21.15.153:0/1151119436 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f30301039b0 0x7f3030107a00 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:42:26.416 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:26.417+0000 7f3037800700 1 --2- 172.21.15.153:0/1151119436 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3030103000 0x7f30301033e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:42:26.416 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:26.417+0000 7f3037800700 1 -- 172.21.15.153:0/1151119436 >> 172.21.15.153:0/1151119436 conn(0x7f30300fe8d0 msgr2=0x7f3030100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:42:26.416 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:26.417+0000 7f3037800700 1 -- 172.21.15.153:0/1151119436 shutdown_connections 2024-03-21T10:42:26.416 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:26.417+0000 7f3037800700 1 -- 172.21.15.153:0/1151119436 wait complete. 2024-03-21T10:42:26.416 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:26.419+0000 7f3037800700 1 Processor -- start 2024-03-21T10:42:26.416 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:26.419+0000 7f3037800700 1 -- start start 2024-03-21T10:42:26.417 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:26.420+0000 7f3037800700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3030103000 0x7f303019e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:42:26.417 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:26.420+0000 7f3037800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f30301039b0 0x7f303019ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:42:26.418 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:26.420+0000 7f3037800700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f303019f300 con 0x7f30301039b0 2024-03-21T10:42:26.418 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:26.420+0000 7f3037800700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3030198840 con 0x7f3030103000 2024-03-21T10:42:26.419 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:26.420+0000 7f302fe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f30301039b0 0x7f303019ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:42:26.419 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:26.420+0000 7f302fe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f30301039b0 0x7f303019ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41014/0 (socket says 172.21.15.153:41014) 2024-03-21T10:42:26.419 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:26.421+0000 7f302fe00700 1 -- 172.21.15.153:0/3319404828 learned_addr learned my addr 172.21.15.153:0/3319404828 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:42:26.419 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:26.421+0000 7f3034e00700 1 --2- 172.21.15.153:0/3319404828 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3030103000 0x7f303019e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:42:26.419 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:26.421+0000 7f302fe00700 1 -- 172.21.15.153:0/3319404828 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3030103000 msgr2=0x7f303019e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:42:26.419 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:26.421+0000 7f302fe00700 1 --2- 172.21.15.153:0/3319404828 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3030103000 0x7f303019e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:42:26.419 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:26.421+0000 7f302fe00700 1 -- 172.21.15.153:0/3319404828 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f301c0095f0 con 0x7f30301039b0 2024-03-21T10:42:26.419 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:26.421+0000 7f302fe00700 1 --2- 172.21.15.153:0/3319404828 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f30301039b0 0x7f303019ed00 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f301c0092e0 tx=0x7f301c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:42:26.420 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:26.422+0000 7f302d600700 1 -- 172.21.15.153:0/3319404828 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f301c01d070 con 0x7f30301039b0 2024-03-21T10:42:26.420 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:26.422+0000 7f302d600700 1 -- 172.21.15.153:0/3319404828 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f301c00f460 con 0x7f30301039b0 2024-03-21T10:42:26.420 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:26.422+0000 7f3037800700 1 -- 172.21.15.153:0/3319404828 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3030198ac0 con 0x7f30301039b0 2024-03-21T10:42:26.420 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:26.422+0000 7f302d600700 1 -- 172.21.15.153:0/3319404828 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f301c0175f0 con 0x7f30301039b0 2024-03-21T10:42:26.420 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:26.422+0000 7f3037800700 1 -- 172.21.15.153:0/3319404828 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3030198fb0 con 0x7f30301039b0 2024-03-21T10:42:26.420 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:26.423+0000 7f3037800700 1 -- 172.21.15.153:0/3319404828 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f303010b400 con 0x7f30301039b0 2024-03-21T10:42:26.421 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:26.424+0000 7f302d600700 1 -- 172.21.15.153:0/3319404828 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f301c00f5d0 con 0x7f30301039b0 2024-03-21T10:42:26.422 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:26.425+0000 7f302d600700 1 --2- 172.21.15.153:0/3319404828 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f301806e460 0x7f3018070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:42:26.422 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:26.425+0000 7f302d600700 1 -- 172.21.15.153:0/3319404828 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f301c08f9d0 con 0x7f30301039b0 2024-03-21T10:42:26.423 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:26.425+0000 7f3034e00700 1 --2- 172.21.15.153:0/3319404828 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f301806e460 0x7f3018070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:42:26.423 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:26.426+0000 7f3034e00700 1 --2- 172.21.15.153:0/3319404828 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f301806e460 0x7f3018070920 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f3024005ea0 tx=0x7f3024005e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:42:26.427 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:26.430+0000 7f302d600700 1 -- 172.21.15.153:0/3319404828 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f301c05c5a0 con 0x7f30301039b0 2024-03-21T10:42:26.447 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:42:26 smithi174 ceph-mon[88004]: from='client.35634 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:42:26.447 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:42:26 smithi174 ceph-mon[88004]: from='client.35638 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:42:26.447 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:42:26 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/2759186669' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:42:26.673 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:26.675+0000 7f3037800700 1 -- 172.21.15.153:0/3319404828 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3030199220 con 0x7f301806e460 2024-03-21T10:42:26.675 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:26.678+0000 7f302d600700 1 -- 172.21.15.153:0/3319404828 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f3030199220 con 0x7f301806e460 2024-03-21T10:42:26.675 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:42:26.675 INFO:teuthology.orchestra.run.smithi153.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232", 2024-03-21T10:42:26.675 INFO:teuthology.orchestra.run.smithi153.stdout: "in_progress": true, 2024-03-21T10:42:26.675 INFO:teuthology.orchestra.run.smithi153.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-21T10:42:26.676 INFO:teuthology.orchestra.run.smithi153.stdout: "services_complete": [ 2024-03-21T10:42:26.676 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr", 2024-03-21T10:42:26.676 INFO:teuthology.orchestra.run.smithi153.stdout: "mon", 2024-03-21T10:42:26.676 INFO:teuthology.orchestra.run.smithi153.stdout: "mds", 2024-03-21T10:42:26.676 INFO:teuthology.orchestra.run.smithi153.stdout: "crash", 2024-03-21T10:42:26.676 INFO:teuthology.orchestra.run.smithi153.stdout: "osd" 2024-03-21T10:42:26.676 INFO:teuthology.orchestra.run.smithi153.stdout: ], 2024-03-21T10:42:26.676 INFO:teuthology.orchestra.run.smithi153.stdout: "progress": "16/23 daemons upgraded", 2024-03-21T10:42:26.676 INFO:teuthology.orchestra.run.smithi153.stdout: "message": "Currently upgrading mds daemons", 2024-03-21T10:42:26.676 INFO:teuthology.orchestra.run.smithi153.stdout: "is_paused": false 2024-03-21T10:42:26.676 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:42:26.678 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:26.680+0000 7f3037800700 1 -- 172.21.15.153:0/3319404828 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f301806e460 msgr2=0x7f3018070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:42:26.678 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:26.680+0000 7f3037800700 1 --2- 172.21.15.153:0/3319404828 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f301806e460 0x7f3018070920 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f3024005ea0 tx=0x7f3024005e30 comp rx=0 tx=0).stop 2024-03-21T10:42:26.678 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:26.680+0000 7f3037800700 1 -- 172.21.15.153:0/3319404828 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f30301039b0 msgr2=0x7f303019ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:42:26.678 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:26.680+0000 7f3037800700 1 --2- 172.21.15.153:0/3319404828 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f30301039b0 0x7f303019ed00 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f301c0092e0 tx=0x7f301c00be20 comp rx=0 tx=0).stop 2024-03-21T10:42:26.678 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:26.680+0000 7f3037800700 1 -- 172.21.15.153:0/3319404828 shutdown_connections 2024-03-21T10:42:26.678 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:26.680+0000 7f3037800700 1 --2- 172.21.15.153:0/3319404828 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3030103000 0x7f303019e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:42:26.679 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:26.680+0000 7f3037800700 1 --2- 172.21.15.153:0/3319404828 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f301806e460 0x7f3018070920 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:42:26.679 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:26.680+0000 7f3037800700 1 --2- 172.21.15.153:0/3319404828 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f30301039b0 0x7f303019ed00 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:42:26.679 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:26.680+0000 7f3037800700 1 -- 172.21.15.153:0/3319404828 >> 172.21.15.153:0/3319404828 conn(0x7f30300fe8d0 msgr2=0x7f3030100cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:42:26.679 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:26.681+0000 7f3037800700 1 -- 172.21.15.153:0/3319404828 shutdown_connections 2024-03-21T10:42:26.679 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:26.681+0000 7f3037800700 1 -- 172.21.15.153:0/3319404828 wait complete. 2024-03-21T10:42:26.829 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:26.831+0000 7f56c9800700 1 -- 172.21.15.153:0/1753369385 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f56c4068670 msgr2=0x7f56c4068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:42:26.829 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:26.831+0000 7f56c9800700 1 --2- 172.21.15.153:0/1753369385 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f56c4068670 0x7f56c4068a50 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f56b400b0a0 tx=0x7f56b400b3b0 comp rx=0 tx=0).stop 2024-03-21T10:42:26.829 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:26.831+0000 7f56c9800700 1 -- 172.21.15.153:0/1753369385 shutdown_connections 2024-03-21T10:42:26.829 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:26.831+0000 7f56c9800700 1 --2- 172.21.15.153:0/1753369385 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f56c4069020 0x7f56c4105790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:42:26.830 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:26.831+0000 7f56c9800700 1 --2- 172.21.15.153:0/1753369385 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f56c4068670 0x7f56c4068a50 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:42:26.830 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:26.831+0000 7f56c9800700 1 -- 172.21.15.153:0/1753369385 >> 172.21.15.153:0/1753369385 conn(0x7f56c40755a0 msgr2=0x7f56c40759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:42:26.830 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:26.831+0000 7f56c9800700 1 -- 172.21.15.153:0/1753369385 shutdown_connections 2024-03-21T10:42:26.830 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:26.832+0000 7f56c9800700 1 -- 172.21.15.153:0/1753369385 wait complete. 2024-03-21T10:42:26.830 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:26.833+0000 7f56c9800700 1 Processor -- start 2024-03-21T10:42:26.830 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:26.833+0000 7f56c9800700 1 -- start start 2024-03-21T10:42:26.832 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:26.833+0000 7f56c9800700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f56c4068670 0x7f56c419e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:42:26.832 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:26.833+0000 7f56c9800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f56c4069020 0x7f56c419ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:42:26.832 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:26.833+0000 7f56c9800700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56c419f350 con 0x7f56c4069020 2024-03-21T10:42:26.832 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:26.833+0000 7f56c9800700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56c4198800 con 0x7f56c4068670 2024-03-21T10:42:26.832 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:26.834+0000 7f56bbe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f56c4069020 0x7f56c419ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:42:26.832 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:26.834+0000 7f56bbe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f56c4069020 0x7f56c419ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41032/0 (socket says 172.21.15.153:41032) 2024-03-21T10:42:26.832 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:26.834+0000 7f56bbe00700 1 -- 172.21.15.153:0/1172956270 learned_addr learned my addr 172.21.15.153:0/1172956270 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:42:26.832 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:26.834+0000 7f56c2a00700 1 --2- 172.21.15.153:0/1172956270 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f56c4068670 0x7f56c419e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:42:26.832 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:26.834+0000 7f56bbe00700 1 -- 172.21.15.153:0/1172956270 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f56c4068670 msgr2=0x7f56c419e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:42:26.833 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:26.834+0000 7f56bbe00700 1 --2- 172.21.15.153:0/1172956270 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f56c4068670 0x7f56c419e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:42:26.833 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:26.834+0000 7f56bbe00700 1 -- 172.21.15.153:0/1172956270 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56b4009d00 con 0x7f56c4069020 2024-03-21T10:42:26.833 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:26.834+0000 7f56c2a00700 1 --2- 172.21.15.153:0/1172956270 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f56c4068670 0x7f56c419e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:42:26.833 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:26.834+0000 7f56bbe00700 1 --2- 172.21.15.153:0/1172956270 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f56c4069020 0x7f56c419ecc0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f56ac00d900 tx=0x7f56ac00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:42:26.833 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:26.835+0000 7f56bb400700 1 -- 172.21.15.153:0/1172956270 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f56ac009900 con 0x7f56c4069020 2024-03-21T10:42:26.834 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:26.835+0000 7f56bb400700 1 -- 172.21.15.153:0/1172956270 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f56ac010460 con 0x7f56c4069020 2024-03-21T10:42:26.834 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:26.835+0000 7f56bb400700 1 -- 172.21.15.153:0/1172956270 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f56ac00f5f0 con 0x7f56c4069020 2024-03-21T10:42:26.834 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:26.835+0000 7f56c9800700 1 -- 172.21.15.153:0/1172956270 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56c4198ae0 con 0x7f56c4069020 2024-03-21T10:42:26.834 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:26.835+0000 7f56c9800700 1 -- 172.21.15.153:0/1172956270 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56c4199030 con 0x7f56c4069020 2024-03-21T10:42:26.834 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:26.836+0000 7f56c9800700 1 -- 172.21.15.153:0/1172956270 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56c404ea90 con 0x7f56c4069020 2024-03-21T10:42:26.834 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:26.837+0000 7f56bb400700 1 -- 172.21.15.153:0/1172956270 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f56ac0105d0 con 0x7f56c4069020 2024-03-21T10:42:26.836 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:26.838+0000 7f56bb400700 1 --2- 172.21.15.153:0/1172956270 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f56a806e460 0x7f56a8070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:42:26.836 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:26.838+0000 7f56bb400700 1 -- 172.21.15.153:0/1172956270 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f56ac08e2c0 con 0x7f56c4069020 2024-03-21T10:42:26.836 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:26.838+0000 7f56c2a00700 1 --2- 172.21.15.153:0/1172956270 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f56a806e460 0x7f56a8070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:42:26.837 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:26.839+0000 7f56c2a00700 1 --2- 172.21.15.153:0/1172956270 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f56a806e460 0x7f56a8070920 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f56b4009480 tx=0x7f56b4000f40 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:42:26.840 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:26.843+0000 7f56bb400700 1 -- 172.21.15.153:0/1172956270 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f56ac05af30 con 0x7f56c4069020 2024-03-21T10:42:27.136 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:27.138+0000 7f56c9800700 1 -- 172.21.15.153:0/1172956270 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f56c4199d10 con 0x7f56c4069020 2024-03-21T10:42:27.137 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:27.139+0000 7f56bb400700 1 -- 172.21.15.153:0/1172956270 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4493 (secure 0 0 0) 0x7f56ac05aac0 con 0x7f56c4069020 2024-03-21T10:42:27.137 INFO:teuthology.orchestra.run.smithi153.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-03-21T10:42:27.137 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-03-21T10:42:27.137 INFO:teuthology.orchestra.run.smithi153.stdout: host smithi174 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi174/config 2024-03-21T10:42:27.137 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/podman: timeout after 895 seconds 2024-03-21T10:42:27.137 INFO:teuthology.orchestra.run.smithi153.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 -e NODE_NAME=smithi174 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/run/ceph:z -v /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpd910r3_i:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 inventory --format=json-pretty --filter-for-batch 2024-03-21T10:42:27.138 INFO:teuthology.orchestra.run.smithi153.stdout:Traceback (most recent call last): 2024-03-21T10:42:27.138 INFO:teuthology.orchestra.run.smithi153.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-03-21T10:42:27.138 INFO:teuthology.orchestra.run.smithi153.stdout: return _run_code(code, main_globals, None, 2024-03-21T10:42:27.138 INFO:teuthology.orchestra.run.smithi153.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-03-21T10:42:27.138 INFO:teuthology.orchestra.run.smithi153.stdout: exec(code, run_globals) 2024-03-21T10:42:27.138 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 5606, in 2024-03-21T10:42:27.138 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 5594, in main 2024-03-21T10:42:27.138 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 406, in _infer_config 2024-03-21T10:42:27.138 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 321, in _infer_fsid 2024-03-21T10:42:27.138 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 434, in _infer_image 2024-03-21T10:42:27.138 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 308, in _validate_fsid 2024-03-21T10:42:27.139 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 3357, in command_ceph_volume 2024-03-21T10:42:27.139 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-03-21T10:42:27.139 INFO:teuthology.orchestra.run.smithi153.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 -e NODE_NAME=smithi174 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/run/ceph:z -v /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpd910r3_i:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 inventory --format=json-pretty --filter-for-batch 2024-03-21T10:42:27.139 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-21T10:42:27.139 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is degraded 2024-03-21T10:42:27.139 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-21T10:42:27.140 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-21T10:42:27.140 INFO:teuthology.orchestra.run.smithi153.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-21T10:42:27.140 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-21T10:42:27.141 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:27.142+0000 7f56c9800700 1 -- 172.21.15.153:0/1172956270 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f56a806e460 msgr2=0x7f56a8070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:42:27.142 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:27.142+0000 7f56c9800700 1 --2- 172.21.15.153:0/1172956270 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f56a806e460 0x7f56a8070920 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f56b4009480 tx=0x7f56b4000f40 comp rx=0 tx=0).stop 2024-03-21T10:42:27.142 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:27.142+0000 7f56c9800700 1 -- 172.21.15.153:0/1172956270 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f56c4069020 msgr2=0x7f56c419ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:42:27.142 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:27.142+0000 7f56c9800700 1 --2- 172.21.15.153:0/1172956270 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f56c4069020 0x7f56c419ecc0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f56ac00d900 tx=0x7f56ac00dcc0 comp rx=0 tx=0).stop 2024-03-21T10:42:27.142 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:27.142+0000 7f56c9800700 1 -- 172.21.15.153:0/1172956270 shutdown_connections 2024-03-21T10:42:27.142 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:27.142+0000 7f56c9800700 1 --2- 172.21.15.153:0/1172956270 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f56c4068670 0x7f56c419e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:42:27.142 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:27.142+0000 7f56c9800700 1 --2- 172.21.15.153:0/1172956270 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f56a806e460 0x7f56a8070920 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:42:27.142 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:27.142+0000 7f56c9800700 1 --2- 172.21.15.153:0/1172956270 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f56c4069020 0x7f56c419ecc0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:42:27.142 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:27.142+0000 7f56c9800700 1 -- 172.21.15.153:0/1172956270 >> 172.21.15.153:0/1172956270 conn(0x7f56c40755a0 msgr2=0x7f56c4102760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:42:27.142 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:27.142+0000 7f56c9800700 1 -- 172.21.15.153:0/1172956270 shutdown_connections 2024-03-21T10:42:27.142 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:27.142+0000 7f56c9800700 1 -- 172.21.15.153:0/1172956270 wait complete. 2024-03-21T10:42:27.447 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:42:27 smithi174 ceph-mon[88004]: pgmap v919: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:42:27.447 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:42:27 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/1732387116' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:42:27.534 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:42:27 smithi153 ceph-mon[113733]: pgmap v919: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:42:27.534 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:42:27 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/1732387116' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:42:28.447 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:42:28 smithi174 ceph-mon[88004]: from='client.35650 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:42:28.447 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:42:28 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/1172956270' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:42:28.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:42:28 smithi153 ceph-mon[113733]: from='client.35650 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:42:28.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:42:28 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/1172956270' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:42:29.447 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:42:29 smithi174 ceph-mon[88004]: pgmap v920: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:42:29.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:42:29 smithi153 ceph-mon[113733]: pgmap v920: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:42:31.447 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:42:31 smithi174 ceph-mon[88004]: pgmap v921: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:42:31.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:42:31 smithi153 ceph-mon[113733]: pgmap v921: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:42:32.447 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:42:32 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:42:32.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:42:32 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:42:33.447 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:42:33 smithi174 ceph-mon[88004]: pgmap v922: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:42:33.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:42:33 smithi153 ceph-mon[113733]: pgmap v922: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:42:35.447 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:42:35 smithi174 ceph-mon[88004]: pgmap v923: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:42:35.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:42:35 smithi153 ceph-mon[113733]: pgmap v923: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:42:37.447 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:42:37 smithi174 ceph-mon[88004]: pgmap v924: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:42:37.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:42:37 smithi153 ceph-mon[113733]: pgmap v924: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:42:39.447 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:42:39 smithi174 ceph-mon[88004]: pgmap v925: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:42:39.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:42:39 smithi153 ceph-mon[113733]: pgmap v925: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:42:41.447 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:42:41 smithi174 ceph-mon[88004]: pgmap v926: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:42:41.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:42:41 smithi153 ceph-mon[113733]: pgmap v926: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:42:43.302 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:42:43 smithi153 ceph-mon[113733]: pgmap v927: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:42:43.447 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:42:43 smithi174 ceph-mon[88004]: pgmap v927: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:42:45.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:42:45 smithi153 ceph-mon[113733]: pgmap v928: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:42:45.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:42:45 smithi174 ceph-mon[88004]: pgmap v928: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:42:47.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:42:47 smithi153 ceph-mon[113733]: pgmap v929: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:42:47.553 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:42:47 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:42:47.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:42:47 smithi174 ceph-mon[88004]: pgmap v929: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:42:47.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:42:47 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:42:49.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:42:49 smithi153 ceph-mon[113733]: pgmap v930: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:42:49.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:42:49 smithi174 ceph-mon[88004]: pgmap v930: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:42:51.447 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:42:51 smithi174 ceph-mon[88004]: pgmap v931: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:42:51.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:42:51 smithi153 ceph-mon[113733]: pgmap v931: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:42:53.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:42:53 smithi153 ceph-mon[113733]: pgmap v932: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:42:53.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:42:53 smithi174 ceph-mon[88004]: pgmap v932: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:42:55.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:42:55 smithi153 ceph-mon[113733]: pgmap v933: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:42:55.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:42:55 smithi174 ceph-mon[88004]: pgmap v933: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:42:57.313 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:57.311+0000 7f1f31600700 1 -- 172.21.15.153:0/2531403624 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f2c101770 msgr2=0x7f1f2c1057c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:42:57.313 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:57.311+0000 7f1f31600700 1 --2- 172.21.15.153:0/2531403624 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f2c101770 0x7f1f2c1057c0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f1f18009910 tx=0x7f1f18009c20 comp rx=0 tx=0).stop 2024-03-21T10:42:57.313 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:57.311+0000 7f1f31600700 1 -- 172.21.15.153:0/2531403624 shutdown_connections 2024-03-21T10:42:57.313 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:57.311+0000 7f1f31600700 1 --2- 172.21.15.153:0/2531403624 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f2c101770 0x7f1f2c1057c0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:42:57.313 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:57.311+0000 7f1f31600700 1 --2- 172.21.15.153:0/2531403624 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1f2c100dc0 0x7f1f2c1011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:42:57.313 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:57.311+0000 7f1f31600700 1 -- 172.21.15.153:0/2531403624 >> 172.21.15.153:0/2531403624 conn(0x7f1f2c0fc650 msgr2=0x7f1f2c0fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:42:57.313 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:57.311+0000 7f1f31600700 1 -- 172.21.15.153:0/2531403624 shutdown_connections 2024-03-21T10:42:57.313 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:57.311+0000 7f1f31600700 1 -- 172.21.15.153:0/2531403624 wait complete. 2024-03-21T10:42:57.313 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:57.312+0000 7f1f31600700 1 Processor -- start 2024-03-21T10:42:57.313 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:57.313+0000 7f1f31600700 1 -- start start 2024-03-21T10:42:57.314 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:57.313+0000 7f1f31600700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1f2c100dc0 0x7f1f2c198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:42:57.314 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:57.313+0000 7f1f31600700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f2c101770 0x7f1f2c198e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:42:57.314 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:57.313+0000 7f1f31600700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f2c199520 con 0x7f1f2c101770 2024-03-21T10:42:57.314 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:57.313+0000 7f1f31600700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f2c19d330 con 0x7f1f2c100dc0 2024-03-21T10:42:57.314 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:57.313+0000 7f1f2a000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f2c101770 0x7f1f2c198e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:42:57.314 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:57.313+0000 7f1f2a000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f2c101770 0x7f1f2c198e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:35790/0 (socket says 172.21.15.153:35790) 2024-03-21T10:42:57.314 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:57.313+0000 7f1f2a000700 1 -- 172.21.15.153:0/1398825286 learned_addr learned my addr 172.21.15.153:0/1398825286 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:42:57.314 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:57.314+0000 7f1f2aa00700 1 --2- 172.21.15.153:0/1398825286 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1f2c100dc0 0x7f1f2c198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:42:57.314 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:57.314+0000 7f1f2a000700 1 -- 172.21.15.153:0/1398825286 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1f2c100dc0 msgr2=0x7f1f2c198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:42:57.314 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:57.314+0000 7f1f2a000700 1 --2- 172.21.15.153:0/1398825286 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1f2c100dc0 0x7f1f2c198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:42:57.314 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:57.314+0000 7f1f2a000700 1 -- 172.21.15.153:0/1398825286 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f180095f0 con 0x7f1f2c101770 2024-03-21T10:42:57.314 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:57.314+0000 7f1f2aa00700 1 --2- 172.21.15.153:0/1398825286 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1f2c100dc0 0x7f1f2c198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:42:57.315 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:57.314+0000 7f1f2a000700 1 --2- 172.21.15.153:0/1398825286 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f2c101770 0x7f1f2c198e90 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f1f1800bda0 tx=0x7f1f1800be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:42:57.315 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:57.314+0000 7f1f23400700 1 -- 172.21.15.153:0/1398825286 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f1801d070 con 0x7f1f2c101770 2024-03-21T10:42:57.315 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:57.314+0000 7f1f23400700 1 -- 172.21.15.153:0/1398825286 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1f18004fb0 con 0x7f1f2c101770 2024-03-21T10:42:57.315 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:57.314+0000 7f1f31600700 1 -- 172.21.15.153:0/1398825286 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f2c19d5b0 con 0x7f1f2c101770 2024-03-21T10:42:57.315 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:57.315+0000 7f1f23400700 1 -- 172.21.15.153:0/1398825286 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f1800f460 con 0x7f1f2c101770 2024-03-21T10:42:57.315 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:57.315+0000 7f1f31600700 1 -- 172.21.15.153:0/1398825286 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f2c19daa0 con 0x7f1f2c101770 2024-03-21T10:42:57.316 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:57.316+0000 7f1f31600700 1 -- 172.21.15.153:0/1398825286 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f2c104f40 con 0x7f1f2c101770 2024-03-21T10:42:57.316 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:57.317+0000 7f1f23400700 1 -- 172.21.15.153:0/1398825286 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f1f18022800 con 0x7f1f2c101770 2024-03-21T10:42:57.316 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:57.318+0000 7f1f23400700 1 --2- 172.21.15.153:0/1398825286 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f1f1006e460 0x7f1f10070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:42:57.316 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:57.318+0000 7f1f23400700 1 -- 172.21.15.153:0/1398825286 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f1f1808faf0 con 0x7f1f2c101770 2024-03-21T10:42:57.316 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:57.318+0000 7f1f2aa00700 1 --2- 172.21.15.153:0/1398825286 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f1f1006e460 0x7f1f10070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:42:57.316 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:57.319+0000 7f1f2aa00700 1 --2- 172.21.15.153:0/1398825286 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f1f1006e460 0x7f1f10070920 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c005fd0 tx=0x7f1f1c005dc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:42:57.321 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:57.324+0000 7f1f23400700 1 -- 172.21.15.153:0/1398825286 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f1f1805c810 con 0x7f1f2c101770 2024-03-21T10:42:57.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:42:57 smithi153 ceph-mon[113733]: pgmap v934: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:42:57.562 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:57.564+0000 7f1f31600700 1 -- 172.21.15.153:0/1398825286 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1f2c19dcc0 con 0x7f1f1006e460 2024-03-21T10:42:57.564 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:57.567+0000 7f1f23400700 1 -- 172.21.15.153:0/1398825286 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f1f2c19dcc0 con 0x7f1f1006e460 2024-03-21T10:42:57.569 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:57.571+0000 7f1f31600700 1 -- 172.21.15.153:0/1398825286 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f1f1006e460 msgr2=0x7f1f10070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:42:57.569 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:57.571+0000 7f1f31600700 1 --2- 172.21.15.153:0/1398825286 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f1f1006e460 0x7f1f10070920 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c005fd0 tx=0x7f1f1c005dc0 comp rx=0 tx=0).stop 2024-03-21T10:42:57.569 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:57.571+0000 7f1f31600700 1 -- 172.21.15.153:0/1398825286 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f2c101770 msgr2=0x7f1f2c198e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:42:57.569 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:57.571+0000 7f1f31600700 1 --2- 172.21.15.153:0/1398825286 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f2c101770 0x7f1f2c198e90 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f1f1800bda0 tx=0x7f1f1800be80 comp rx=0 tx=0).stop 2024-03-21T10:42:57.569 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:57.571+0000 7f1f31600700 1 -- 172.21.15.153:0/1398825286 shutdown_connections 2024-03-21T10:42:57.569 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:57.571+0000 7f1f31600700 1 --2- 172.21.15.153:0/1398825286 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1f2c100dc0 0x7f1f2c198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:42:57.569 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:57.571+0000 7f1f31600700 1 --2- 172.21.15.153:0/1398825286 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f1f1006e460 0x7f1f10070920 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:42:57.569 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:57.571+0000 7f1f31600700 1 --2- 172.21.15.153:0/1398825286 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1f2c101770 0x7f1f2c198e90 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:42:57.569 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:57.571+0000 7f1f31600700 1 -- 172.21.15.153:0/1398825286 >> 172.21.15.153:0/1398825286 conn(0x7f1f2c0fc650 msgr2=0x7f1f2c0fdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:42:57.569 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:57.571+0000 7f1f31600700 1 -- 172.21.15.153:0/1398825286 shutdown_connections 2024-03-21T10:42:57.570 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:57.571+0000 7f1f31600700 1 -- 172.21.15.153:0/1398825286 wait complete. 2024-03-21T10:42:57.584 INFO:teuthology.orchestra.run.smithi153.stdout:true 2024-03-21T10:42:57.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:42:57 smithi174 ceph-mon[88004]: pgmap v934: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:42:57.727 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:57.729+0000 7f2a56000700 1 -- 172.21.15.153:0/526184430 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a50103000 msgr2=0x7f2a501033e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:42:57.727 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:57.729+0000 7f2a56000700 1 --2- 172.21.15.153:0/526184430 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a50103000 0x7f2a501033e0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f2a40009910 tx=0x7f2a40009c20 comp rx=0 tx=0).stop 2024-03-21T10:42:57.727 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:57.729+0000 7f2a56000700 1 -- 172.21.15.153:0/526184430 shutdown_connections 2024-03-21T10:42:57.727 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:57.729+0000 7f2a56000700 1 --2- 172.21.15.153:0/526184430 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a501039b0 0x7f2a50107a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:42:57.727 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:57.729+0000 7f2a56000700 1 --2- 172.21.15.153:0/526184430 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a50103000 0x7f2a501033e0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:42:57.727 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:57.729+0000 7f2a56000700 1 -- 172.21.15.153:0/526184430 >> 172.21.15.153:0/526184430 conn(0x7f2a500fe8d0 msgr2=0x7f2a50100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:42:57.727 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:57.729+0000 7f2a56000700 1 -- 172.21.15.153:0/526184430 shutdown_connections 2024-03-21T10:42:57.727 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:57.729+0000 7f2a56000700 1 -- 172.21.15.153:0/526184430 wait complete. 2024-03-21T10:42:57.728 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:57.730+0000 7f2a56000700 1 Processor -- start 2024-03-21T10:42:57.728 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:57.731+0000 7f2a56000700 1 -- start start 2024-03-21T10:42:57.729 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:57.731+0000 7f2a56000700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a50103000 0x7f2a5019e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:42:57.729 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:57.731+0000 7f2a56000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a501039b0 0x7f2a5019ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:42:57.729 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:57.731+0000 7f2a56000700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a5019f3c0 con 0x7f2a501039b0 2024-03-21T10:42:57.729 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:57.731+0000 7f2a56000700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a50198870 con 0x7f2a50103000 2024-03-21T10:42:57.729 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:57.731+0000 7f2a4ea00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a501039b0 0x7f2a5019ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:42:57.729 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:57.731+0000 7f2a4ea00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a501039b0 0x7f2a5019ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:35800/0 (socket says 172.21.15.153:35800) 2024-03-21T10:42:57.729 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:57.731+0000 7f2a4ea00700 1 -- 172.21.15.153:0/160765926 learned_addr learned my addr 172.21.15.153:0/160765926 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:42:57.730 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:57.732+0000 7f2a4f400700 1 --2- 172.21.15.153:0/160765926 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a50103000 0x7f2a5019e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:42:57.730 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:57.732+0000 7f2a4ea00700 1 -- 172.21.15.153:0/160765926 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a50103000 msgr2=0x7f2a5019e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:42:57.730 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:57.732+0000 7f2a4ea00700 1 --2- 172.21.15.153:0/160765926 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a50103000 0x7f2a5019e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:42:57.730 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:57.732+0000 7f2a4ea00700 1 -- 172.21.15.153:0/160765926 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a400095f0 con 0x7f2a501039b0 2024-03-21T10:42:57.731 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:57.732+0000 7f2a4f400700 1 --2- 172.21.15.153:0/160765926 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a50103000 0x7f2a5019e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:42:57.731 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:57.732+0000 7f2a4ea00700 1 --2- 172.21.15.153:0/160765926 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a501039b0 0x7f2a5019ed30 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c00d900 tx=0x7f2a3c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:42:57.731 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:57.732+0000 7f2a47e00700 1 -- 172.21.15.153:0/160765926 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a3c009900 con 0x7f2a501039b0 2024-03-21T10:42:57.731 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:57.732+0000 7f2a56000700 1 -- 172.21.15.153:0/160765926 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a50198b50 con 0x7f2a501039b0 2024-03-21T10:42:57.731 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:57.733+0000 7f2a47e00700 1 -- 172.21.15.153:0/160765926 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2a3c010460 con 0x7f2a501039b0 2024-03-21T10:42:57.731 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:57.733+0000 7f2a56000700 1 -- 172.21.15.153:0/160765926 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a501990a0 con 0x7f2a501039b0 2024-03-21T10:42:57.731 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:57.733+0000 7f2a47e00700 1 -- 172.21.15.153:0/160765926 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a3c00f5f0 con 0x7f2a501039b0 2024-03-21T10:42:57.731 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:57.734+0000 7f2a56000700 1 -- 172.21.15.153:0/160765926 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a5010b3a0 con 0x7f2a501039b0 2024-03-21T10:42:57.732 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:57.735+0000 7f2a47e00700 1 -- 172.21.15.153:0/160765926 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f2a3c0105d0 con 0x7f2a501039b0 2024-03-21T10:42:57.734 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:57.736+0000 7f2a47e00700 1 --2- 172.21.15.153:0/160765926 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f2a3406e460 0x7f2a34070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:42:57.734 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:57.736+0000 7f2a47e00700 1 -- 172.21.15.153:0/160765926 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f2a3c08e1f0 con 0x7f2a501039b0 2024-03-21T10:42:57.734 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:57.737+0000 7f2a4f400700 1 --2- 172.21.15.153:0/160765926 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f2a3406e460 0x7f2a34070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:42:57.734 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:57.737+0000 7f2a4f400700 1 --2- 172.21.15.153:0/160765926 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f2a3406e460 0x7f2a34070920 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f2a4000b5c0 tx=0x7f2a400053a0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:42:57.739 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:57.741+0000 7f2a47e00700 1 -- 172.21.15.153:0/160765926 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2a3c05ad40 con 0x7f2a501039b0 2024-03-21T10:42:57.966 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:57.968+0000 7f2a56000700 1 -- 172.21.15.153:0/160765926 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2a501993d0 con 0x7f2a3406e460 2024-03-21T10:42:57.969 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:57.971+0000 7f2a47e00700 1 -- 172.21.15.153:0/160765926 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f2a501993d0 con 0x7f2a3406e460 2024-03-21T10:42:57.972 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:57.974+0000 7f2a56000700 1 -- 172.21.15.153:0/160765926 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f2a3406e460 msgr2=0x7f2a34070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:42:57.972 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:57.974+0000 7f2a56000700 1 --2- 172.21.15.153:0/160765926 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f2a3406e460 0x7f2a34070920 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f2a4000b5c0 tx=0x7f2a400053a0 comp rx=0 tx=0).stop 2024-03-21T10:42:57.972 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:57.974+0000 7f2a56000700 1 -- 172.21.15.153:0/160765926 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a501039b0 msgr2=0x7f2a5019ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:42:57.972 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:57.974+0000 7f2a56000700 1 --2- 172.21.15.153:0/160765926 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a501039b0 0x7f2a5019ed30 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c00d900 tx=0x7f2a3c00dcc0 comp rx=0 tx=0).stop 2024-03-21T10:42:57.972 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:57.974+0000 7f2a56000700 1 -- 172.21.15.153:0/160765926 shutdown_connections 2024-03-21T10:42:57.972 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:57.974+0000 7f2a56000700 1 --2- 172.21.15.153:0/160765926 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a50103000 0x7f2a5019e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:42:57.972 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:57.974+0000 7f2a56000700 1 --2- 172.21.15.153:0/160765926 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f2a3406e460 0x7f2a34070920 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:42:57.972 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:57.974+0000 7f2a56000700 1 --2- 172.21.15.153:0/160765926 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2a501039b0 0x7f2a5019ed30 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:42:57.972 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:57.974+0000 7f2a56000700 1 -- 172.21.15.153:0/160765926 >> 172.21.15.153:0/160765926 conn(0x7f2a500fe8d0 msgr2=0x7f2a500ffee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:42:57.972 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:57.974+0000 7f2a56000700 1 -- 172.21.15.153:0/160765926 shutdown_connections 2024-03-21T10:42:57.973 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:57.974+0000 7f2a56000700 1 -- 172.21.15.153:0/160765926 wait complete. 2024-03-21T10:42:58.129 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:58.131+0000 7ff2f6200700 1 -- 172.21.15.153:0/2892343461 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff2f0069020 msgr2=0x7ff2f0105790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:42:58.129 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:58.131+0000 7ff2f6200700 1 --2- 172.21.15.153:0/2892343461 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff2f0069020 0x7ff2f0105790 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc009910 tx=0x7ff2dc009c20 comp rx=0 tx=0).stop 2024-03-21T10:42:58.129 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:58.131+0000 7ff2f6200700 1 -- 172.21.15.153:0/2892343461 shutdown_connections 2024-03-21T10:42:58.129 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:58.131+0000 7ff2f6200700 1 --2- 172.21.15.153:0/2892343461 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff2f0069020 0x7ff2f0105790 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:42:58.129 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:58.131+0000 7ff2f6200700 1 --2- 172.21.15.153:0/2892343461 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff2f0068670 0x7ff2f0068a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:42:58.129 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:58.131+0000 7ff2f6200700 1 -- 172.21.15.153:0/2892343461 >> 172.21.15.153:0/2892343461 conn(0x7ff2f00755a0 msgr2=0x7ff2f00759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:42:58.129 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:58.131+0000 7ff2f6200700 1 -- 172.21.15.153:0/2892343461 shutdown_connections 2024-03-21T10:42:58.129 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:58.131+0000 7ff2f6200700 1 -- 172.21.15.153:0/2892343461 wait complete. 2024-03-21T10:42:58.131 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:58.132+0000 7ff2f6200700 1 Processor -- start 2024-03-21T10:42:58.131 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:58.132+0000 7ff2f6200700 1 -- start start 2024-03-21T10:42:58.131 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:58.133+0000 7ff2f6200700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff2f0068670 0x7ff2f019e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:42:58.131 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:58.133+0000 7ff2f6200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff2f0069020 0x7ff2f019ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:42:58.131 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:58.133+0000 7ff2f6200700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f019f210 con 0x7ff2f0069020 2024-03-21T10:42:58.131 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:58.133+0000 7ff2f6200700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f0198750 con 0x7ff2f0068670 2024-03-21T10:42:58.131 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:58.133+0000 7ff2eea00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff2f0069020 0x7ff2f019ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:42:58.131 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:58.133+0000 7ff2eea00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff2f0069020 0x7ff2f019ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:35822/0 (socket says 172.21.15.153:35822) 2024-03-21T10:42:58.131 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:58.133+0000 7ff2eea00700 1 -- 172.21.15.153:0/411348989 learned_addr learned my addr 172.21.15.153:0/411348989 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:42:58.131 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:58.133+0000 7ff2ef400700 1 --2- 172.21.15.153:0/411348989 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff2f0068670 0x7ff2f019e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:42:58.131 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:58.133+0000 7ff2eea00700 1 -- 172.21.15.153:0/411348989 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff2f0068670 msgr2=0x7ff2f019e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:42:58.131 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:58.133+0000 7ff2eea00700 1 --2- 172.21.15.153:0/411348989 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff2f0068670 0x7ff2f019e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:42:58.132 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:58.133+0000 7ff2eea00700 1 -- 172.21.15.153:0/411348989 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2dc0095f0 con 0x7ff2f0069020 2024-03-21T10:42:58.132 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:58.134+0000 7ff2eea00700 1 --2- 172.21.15.153:0/411348989 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff2f0069020 0x7ff2f019ec10 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc00bda0 tx=0x7ff2dc00be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:42:58.132 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:58.134+0000 7ff2e7e00700 1 -- 172.21.15.153:0/411348989 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff2dc01d070 con 0x7ff2f0069020 2024-03-21T10:42:58.132 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:58.134+0000 7ff2e7e00700 1 -- 172.21.15.153:0/411348989 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff2dc004fb0 con 0x7ff2f0069020 2024-03-21T10:42:58.132 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:58.134+0000 7ff2f6200700 1 -- 172.21.15.153:0/411348989 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2f01989d0 con 0x7ff2f0069020 2024-03-21T10:42:58.132 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:58.134+0000 7ff2e7e00700 1 -- 172.21.15.153:0/411348989 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff2dc00f480 con 0x7ff2f0069020 2024-03-21T10:42:58.133 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:58.134+0000 7ff2f6200700 1 -- 172.21.15.153:0/411348989 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2f0198ec0 con 0x7ff2f0069020 2024-03-21T10:42:58.133 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:58.135+0000 7ff2f6200700 1 -- 172.21.15.153:0/411348989 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2f004ea90 con 0x7ff2f0069020 2024-03-21T10:42:58.134 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:58.136+0000 7ff2e7e00700 1 -- 172.21.15.153:0/411348989 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7ff2dc022870 con 0x7ff2f0069020 2024-03-21T10:42:58.135 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:58.137+0000 7ff2e7e00700 1 --2- 172.21.15.153:0/411348989 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff2d406e460 0x7ff2d4070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:42:58.135 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:58.137+0000 7ff2e7e00700 1 -- 172.21.15.153:0/411348989 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7ff2dc08faa0 con 0x7ff2f0069020 2024-03-21T10:42:58.135 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:58.138+0000 7ff2ef400700 1 --2- 172.21.15.153:0/411348989 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff2d406e460 0x7ff2d4070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:42:58.135 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:58.138+0000 7ff2ef400700 1 --2- 172.21.15.153:0/411348989 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff2d406e460 0x7ff2d4070920 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7ff2e0005ea0 tx=0x7ff2e0005e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:42:58.139 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:58.143+0000 7ff2e7e00700 1 -- 172.21.15.153:0/411348989 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff2dc05c7c0 con 0x7ff2f0069020 2024-03-21T10:42:58.366 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:58.368+0000 7ff2f6200700 1 -- 172.21.15.153:0/411348989 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7ff2f01992b0 con 0x7ff2d406e460 2024-03-21T10:42:58.396 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:58.399+0000 7ff2e7e00700 1 -- 172.21.15.153:0/411348989 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7ff2f01992b0 con 0x7ff2d406e460 2024-03-21T10:42:58.397 INFO:teuthology.orchestra.run.smithi153.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-21T10:42:58.397 INFO:teuthology.orchestra.run.smithi153.stdout:alertmanager.smithi153 smithi153 *:9093,9094 running (31m) 7m ago 45m 15.6M - 0.25.0 c8568f914cd2 9ed3f2200b5f 2024-03-21T10:42:58.397 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi153 smithi153 running (45m) 7m ago 45m 10.4M - 18.2.1 5be31c24972a 5b61994396a6 2024-03-21T10:42:58.397 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi174 smithi174 running (44m) 7m ago 44m 10.1M - 18.2.1 5be31c24972a 4f7a5fc3ac28 2024-03-21T10:42:58.397 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi153 smithi153 running (27m) 7m ago 45m 6656k - 19.0.0-2231-g2edfee68 237e43c7a65f 32c21104d5b0 2024-03-21T10:42:58.397 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi174 smithi174 running (27m) 7m ago 44m 6660k - 19.0.0-2231-g2edfee68 237e43c7a65f a04e48967ab6 2024-03-21T10:42:58.397 INFO:teuthology.orchestra.run.smithi153.stdout:grafana.smithi153 smithi153 *:3000 running (31m) 7m ago 44m 75.2M - 9.4.12 2bacad6d85d8 ee24026ab786 2024-03-21T10:42:58.398 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.iuyluj smithi153 running (23m) 7m ago 38m 18.7M - 19.0.0-2231-g2edfee68 237e43c7a65f 87aff8a5de70 2024-03-21T10:42:58.398 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.zipckr smithi153 running (23m) 7m ago 38m 18.9M - 19.0.0-2231-g2edfee68 237e43c7a65f ccc262bd5dd3 2024-03-21T10:42:58.398 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.nhozag smithi174 running (7m) 7m ago 38m 10.9M - 19.0.0-2231-g2edfee68 237e43c7a65f 9e02aea72c96 2024-03-21T10:42:58.398 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.yyimst smithi174 running (22m) 7m ago 38m 18.5M - 19.0.0-2231-g2edfee68 237e43c7a65f fc1962f9351e 2024-03-21T10:42:58.398 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi153.tluxni smithi153 *:8443,9283,8765 running (34m) 7m ago 47m 512M - 19.0.0-2231-g2edfee68 237e43c7a65f 7ffee0e55382 2024-03-21T10:42:58.398 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi174.qpcwat smithi174 *:8443,9283,8765 running (33m) 7m ago 44m 459M - 19.0.0-2231-g2edfee68 237e43c7a65f 748968f216ab 2024-03-21T10:42:58.398 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi153 smithi153 running (29m) 7m ago 47m 70.8M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f dd8038303dc3 2024-03-21T10:42:58.398 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi174 smithi174 running (29m) 7m ago 43m 51.9M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f 29d78a788f2c 2024-03-21T10:42:58.398 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi153 smithi153 *:9100 running (32m) 7m ago 45m 8140k - 1.5.0 0da6a335fe13 bb835edc0e0b 2024-03-21T10:42:58.398 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi174 smithi174 *:9100 running (32m) 7m ago 44m 8610k - 1.5.0 0da6a335fe13 7c4ecad790fa 2024-03-21T10:42:58.398 INFO:teuthology.orchestra.run.smithi153.stdout:osd.0 smithi153 running (27m) 7m ago 42m 118M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 507900fd4791 2024-03-21T10:42:58.398 INFO:teuthology.orchestra.run.smithi153.stdout:osd.1 smithi153 running (26m) 7m ago 42m 93.0M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 0d646452e9c1 2024-03-21T10:42:58.398 INFO:teuthology.orchestra.run.smithi153.stdout:osd.2 smithi153 running (26m) 7m ago 41m 91.2M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 1326f2afdfcd 2024-03-21T10:42:58.398 INFO:teuthology.orchestra.run.smithi153.stdout:osd.3 smithi174 running (25m) 7m ago 41m 87.5M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 70aafac71ad5 2024-03-21T10:42:58.399 INFO:teuthology.orchestra.run.smithi153.stdout:osd.4 smithi174 running (24m) 7m ago 40m 95.8M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 0d746ce6e79d 2024-03-21T10:42:58.399 INFO:teuthology.orchestra.run.smithi153.stdout:osd.5 smithi174 running (24m) 7m ago 40m 90.3M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 53e5f35b8cac 2024-03-21T10:42:58.399 INFO:teuthology.orchestra.run.smithi153.stdout:prometheus.smithi153 smithi153 *:9095 running (32m) 7m ago 44m 88.1M - 2.43.0 a07b618ecd1d 7d95561e50c2 2024-03-21T10:42:58.399 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:58.402+0000 7ff2f6200700 1 -- 172.21.15.153:0/411348989 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff2d406e460 msgr2=0x7ff2d4070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:42:58.400 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:58.402+0000 7ff2f6200700 1 --2- 172.21.15.153:0/411348989 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff2d406e460 0x7ff2d4070920 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7ff2e0005ea0 tx=0x7ff2e0005e30 comp rx=0 tx=0).stop 2024-03-21T10:42:58.400 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:58.402+0000 7ff2f6200700 1 -- 172.21.15.153:0/411348989 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff2f0069020 msgr2=0x7ff2f019ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:42:58.400 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:58.402+0000 7ff2f6200700 1 --2- 172.21.15.153:0/411348989 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff2f0069020 0x7ff2f019ec10 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc00bda0 tx=0x7ff2dc00be80 comp rx=0 tx=0).stop 2024-03-21T10:42:58.400 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:58.402+0000 7ff2f6200700 1 -- 172.21.15.153:0/411348989 shutdown_connections 2024-03-21T10:42:58.400 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:58.402+0000 7ff2f6200700 1 --2- 172.21.15.153:0/411348989 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff2f0068670 0x7ff2f019e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:42:58.400 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:58.402+0000 7ff2f6200700 1 --2- 172.21.15.153:0/411348989 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff2d406e460 0x7ff2d4070920 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:42:58.400 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:58.402+0000 7ff2f6200700 1 --2- 172.21.15.153:0/411348989 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff2f0069020 0x7ff2f019ec10 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:42:58.400 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:58.402+0000 7ff2f6200700 1 -- 172.21.15.153:0/411348989 >> 172.21.15.153:0/411348989 conn(0x7ff2f00755a0 msgr2=0x7ff2f0105000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:42:58.400 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:58.402+0000 7ff2f6200700 1 -- 172.21.15.153:0/411348989 shutdown_connections 2024-03-21T10:42:58.400 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:58.402+0000 7ff2f6200700 1 -- 172.21.15.153:0/411348989 wait complete. 2024-03-21T10:42:58.553 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:58.555+0000 7f255e000700 1 -- 172.21.15.153:0/1201623554 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2558073a50 msgr2=0x7f2558111610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:42:58.553 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:58.555+0000 7f255e000700 1 --2- 172.21.15.153:0/1201623554 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2558073a50 0x7f2558111610 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f2544009910 tx=0x7f2544009c20 comp rx=0 tx=0).stop 2024-03-21T10:42:58.553 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:58.555+0000 7f255e000700 1 -- 172.21.15.153:0/1201623554 shutdown_connections 2024-03-21T10:42:58.553 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:58.555+0000 7f255e000700 1 --2- 172.21.15.153:0/1201623554 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2558073a50 0x7f2558111610 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:42:58.553 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:58.555+0000 7f255e000700 1 --2- 172.21.15.153:0/1201623554 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2558073130 0x7f2558073510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:42:58.553 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:58.555+0000 7f255e000700 1 -- 172.21.15.153:0/1201623554 >> 172.21.15.153:0/1201623554 conn(0x7f25580fc630 msgr2=0x7f25580fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:42:58.553 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:58.555+0000 7f255e000700 1 -- 172.21.15.153:0/1201623554 shutdown_connections 2024-03-21T10:42:58.553 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:58.555+0000 7f255e000700 1 -- 172.21.15.153:0/1201623554 wait complete. 2024-03-21T10:42:58.554 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:58.556+0000 7f255e000700 1 Processor -- start 2024-03-21T10:42:58.554 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:58.556+0000 7f255e000700 1 -- start start 2024-03-21T10:42:58.554 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:58.557+0000 7f255e000700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2558073130 0x7f255819cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:42:58.554 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:58.557+0000 7f255e000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2558073a50 0x7f255819d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:42:58.554 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:58.557+0000 7f255e000700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f255819d9e0 con 0x7f2558073a50 2024-03-21T10:42:58.554 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:58.557+0000 7f255e000700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25581a1770 con 0x7f2558073130 2024-03-21T10:42:58.555 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:58.557+0000 7f2556a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2558073a50 0x7f255819d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:42:58.555 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:58.557+0000 7f2556a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2558073a50 0x7f255819d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:35848/0 (socket says 172.21.15.153:35848) 2024-03-21T10:42:58.555 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:58.557+0000 7f2556a00700 1 -- 172.21.15.153:0/2980865480 learned_addr learned my addr 172.21.15.153:0/2980865480 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:42:58.555 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:58.558+0000 7f2557400700 1 --2- 172.21.15.153:0/2980865480 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2558073130 0x7f255819cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:42:58.555 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:58.558+0000 7f2556a00700 1 -- 172.21.15.153:0/2980865480 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2558073130 msgr2=0x7f255819cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:42:58.555 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:58.558+0000 7f2556a00700 1 --2- 172.21.15.153:0/2980865480 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2558073130 0x7f255819cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:42:58.555 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:58.558+0000 7f2556a00700 1 -- 172.21.15.153:0/2980865480 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25440095f0 con 0x7f2558073a50 2024-03-21T10:42:58.556 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:58.558+0000 7f2556a00700 1 --2- 172.21.15.153:0/2980865480 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2558073a50 0x7f255819d300 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f254400b5c0 tx=0x7f254400bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:42:58.556 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:58.558+0000 7f254fe00700 1 -- 172.21.15.153:0/2980865480 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f254401d070 con 0x7f2558073a50 2024-03-21T10:42:58.556 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:58.559+0000 7f254fe00700 1 -- 172.21.15.153:0/2980865480 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f254400f460 con 0x7f2558073a50 2024-03-21T10:42:58.556 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:58.559+0000 7f255e000700 1 -- 172.21.15.153:0/2980865480 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25581a1a50 con 0x7f2558073a50 2024-03-21T10:42:58.556 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:58.559+0000 7f254fe00700 1 -- 172.21.15.153:0/2980865480 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2544017610 con 0x7f2558073a50 2024-03-21T10:42:58.557 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:58.559+0000 7f255e000700 1 -- 172.21.15.153:0/2980865480 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25581a1fa0 con 0x7f2558073a50 2024-03-21T10:42:58.557 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:58.560+0000 7f255e000700 1 -- 172.21.15.153:0/2980865480 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f255810ed90 con 0x7f2558073a50 2024-03-21T10:42:58.558 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:58.561+0000 7f254fe00700 1 -- 172.21.15.153:0/2980865480 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f254400f5d0 con 0x7f2558073a50 2024-03-21T10:42:58.560 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:58.562+0000 7f254fe00700 1 --2- 172.21.15.153:0/2980865480 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f253c06e460 0x7f253c070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:42:58.560 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:58.563+0000 7f254fe00700 1 -- 172.21.15.153:0/2980865480 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f254408f9e0 con 0x7f2558073a50 2024-03-21T10:42:58.560 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:58.563+0000 7f2557400700 1 --2- 172.21.15.153:0/2980865480 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f253c06e460 0x7f253c070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:42:58.561 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:58.564+0000 7f2557400700 1 --2- 172.21.15.153:0/2980865480 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f253c06e460 0x7f253c070920 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f2548009a10 tx=0x7f2548009190 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:42:58.565 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:58.567+0000 7f254fe00700 1 -- 172.21.15.153:0/2980865480 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2544058aa0 con 0x7f2558073a50 2024-03-21T10:42:58.870 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:58.873+0000 7f255e000700 1 -- 172.21.15.153:0/2980865480 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f255804ea90 con 0x7f2558073a50 2024-03-21T10:42:58.871 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:58.874+0000 7f254fe00700 1 -- 172.21.15.153:0/2980865480 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f254405c160 con 0x7f2558073a50 2024-03-21T10:42:58.871 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:42:58.872 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": { 2024-03-21T10:42:58.872 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:42:58.872 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:42:58.872 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": { 2024-03-21T10:42:58.872 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:42:58.872 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:42:58.872 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": { 2024-03-21T10:42:58.872 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 6 2024-03-21T10:42:58.872 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:42:58.872 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": { 2024-03-21T10:42:58.872 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 4 2024-03-21T10:42:58.872 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:42:58.872 INFO:teuthology.orchestra.run.smithi153.stdout: "overall": { 2024-03-21T10:42:58.872 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 14 2024-03-21T10:42:58.872 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-03-21T10:42:58.873 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:42:58.874 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:58.877+0000 7f255e000700 1 -- 172.21.15.153:0/2980865480 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f253c06e460 msgr2=0x7f253c070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:42:58.874 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:58.877+0000 7f255e000700 1 --2- 172.21.15.153:0/2980865480 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f253c06e460 0x7f253c070920 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f2548009a10 tx=0x7f2548009190 comp rx=0 tx=0).stop 2024-03-21T10:42:58.874 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:58.877+0000 7f255e000700 1 -- 172.21.15.153:0/2980865480 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2558073a50 msgr2=0x7f255819d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:42:58.874 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:58.877+0000 7f255e000700 1 --2- 172.21.15.153:0/2980865480 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2558073a50 0x7f255819d300 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f254400b5c0 tx=0x7f254400bd50 comp rx=0 tx=0).stop 2024-03-21T10:42:58.875 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:58.877+0000 7f255e000700 1 -- 172.21.15.153:0/2980865480 shutdown_connections 2024-03-21T10:42:58.875 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:58.877+0000 7f255e000700 1 --2- 172.21.15.153:0/2980865480 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2558073130 0x7f255819cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:42:58.875 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:58.877+0000 7f255e000700 1 --2- 172.21.15.153:0/2980865480 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f253c06e460 0x7f253c070920 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:42:58.875 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:58.877+0000 7f255e000700 1 --2- 172.21.15.153:0/2980865480 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2558073a50 0x7f255819d300 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:42:58.875 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:58.877+0000 7f255e000700 1 -- 172.21.15.153:0/2980865480 >> 172.21.15.153:0/2980865480 conn(0x7f25580fc630 msgr2=0x7f2558103120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:42:58.875 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:58.877+0000 7f255e000700 1 -- 172.21.15.153:0/2980865480 shutdown_connections 2024-03-21T10:42:58.875 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:58.877+0000 7f255e000700 1 -- 172.21.15.153:0/2980865480 wait complete. 2024-03-21T10:42:59.031 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.033+0000 7faa6ca00700 1 -- 172.21.15.153:0/3132169007 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faa68102fe0 msgr2=0x7faa681033c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:42:59.031 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.033+0000 7faa6ca00700 1 --2- 172.21.15.153:0/3132169007 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faa68102fe0 0x7faa681033c0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7faa58009910 tx=0x7faa58009c20 comp rx=0 tx=0).stop 2024-03-21T10:42:59.031 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.034+0000 7faa6ca00700 1 -- 172.21.15.153:0/3132169007 shutdown_connections 2024-03-21T10:42:59.031 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.034+0000 7faa6ca00700 1 --2- 172.21.15.153:0/3132169007 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faa68103990 0x7faa681079e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:42:59.031 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.034+0000 7faa6ca00700 1 --2- 172.21.15.153:0/3132169007 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faa68102fe0 0x7faa681033c0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:42:59.032 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.034+0000 7faa6ca00700 1 -- 172.21.15.153:0/3132169007 >> 172.21.15.153:0/3132169007 conn(0x7faa680fe8d0 msgr2=0x7faa68100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:42:59.032 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.034+0000 7faa6ca00700 1 -- 172.21.15.153:0/3132169007 shutdown_connections 2024-03-21T10:42:59.032 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.034+0000 7faa6ca00700 1 -- 172.21.15.153:0/3132169007 wait complete. 2024-03-21T10:42:59.032 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.035+0000 7faa6ca00700 1 Processor -- start 2024-03-21T10:42:59.032 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.035+0000 7faa6ca00700 1 -- start start 2024-03-21T10:42:59.033 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.035+0000 7faa6ca00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faa68102fe0 0x7faa6819e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:42:59.033 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.035+0000 7faa6ca00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faa68103990 0x7faa6819ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:42:59.033 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.035+0000 7faa6ca00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa6819f3d0 con 0x7faa68103990 2024-03-21T10:42:59.033 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.035+0000 7faa6ca00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa68198880 con 0x7faa68102fe0 2024-03-21T10:42:59.033 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.036+0000 7faa65400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faa68103990 0x7faa6819ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:42:59.033 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.036+0000 7faa65400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faa68103990 0x7faa6819ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:35868/0 (socket says 172.21.15.153:35868) 2024-03-21T10:42:59.033 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.036+0000 7faa65400700 1 -- 172.21.15.153:0/1428172498 learned_addr learned my addr 172.21.15.153:0/1428172498 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:42:59.034 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.036+0000 7faa65e00700 1 --2- 172.21.15.153:0/1428172498 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faa68102fe0 0x7faa6819e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:42:59.034 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.036+0000 7faa65400700 1 -- 172.21.15.153:0/1428172498 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faa68102fe0 msgr2=0x7faa6819e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:42:59.034 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.036+0000 7faa65400700 1 --2- 172.21.15.153:0/1428172498 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faa68102fe0 0x7faa6819e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:42:59.034 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.036+0000 7faa65400700 1 -- 172.21.15.153:0/1428172498 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa580095f0 con 0x7faa68103990 2024-03-21T10:42:59.034 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.036+0000 7faa65e00700 1 --2- 172.21.15.153:0/1428172498 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faa68102fe0 0x7faa6819e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:42:59.034 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.036+0000 7faa65400700 1 --2- 172.21.15.153:0/1428172498 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faa68103990 0x7faa6819ed40 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7faa5400b480 tx=0x7faa5400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:42:59.034 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.037+0000 7faa5ea00700 1 -- 172.21.15.153:0/1428172498 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faa540118c0 con 0x7faa68103990 2024-03-21T10:42:59.035 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.037+0000 7faa5ea00700 1 -- 172.21.15.153:0/1428172498 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faa54010380 con 0x7faa68103990 2024-03-21T10:42:59.035 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.037+0000 7faa6ca00700 1 -- 172.21.15.153:0/1428172498 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa68198b60 con 0x7faa68103990 2024-03-21T10:42:59.035 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.037+0000 7faa5ea00700 1 -- 172.21.15.153:0/1428172498 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faa5400f4f0 con 0x7faa68103990 2024-03-21T10:42:59.035 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.037+0000 7faa6ca00700 1 -- 172.21.15.153:0/1428172498 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa681990b0 con 0x7faa68103990 2024-03-21T10:42:59.035 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.038+0000 7faa6ca00700 1 -- 172.21.15.153:0/1428172498 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa6810b380 con 0x7faa68103990 2024-03-21T10:42:59.036 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.039+0000 7faa5ea00700 1 -- 172.21.15.153:0/1428172498 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7faa540104f0 con 0x7faa68103990 2024-03-21T10:42:59.037 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.040+0000 7faa5ea00700 1 --2- 172.21.15.153:0/1428172498 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7faa4c06e460 0x7faa4c070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:42:59.038 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.040+0000 7faa5ea00700 1 -- 172.21.15.153:0/1428172498 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7faa5408e120 con 0x7faa68103990 2024-03-21T10:42:59.038 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.040+0000 7faa65e00700 1 --2- 172.21.15.153:0/1428172498 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7faa4c06e460 0x7faa4c070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:42:59.038 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.041+0000 7faa65e00700 1 --2- 172.21.15.153:0/1428172498 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7faa4c06e460 0x7faa4c070920 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7faa58006010 tx=0x7faa5800b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:42:59.042 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.045+0000 7faa5ea00700 1 -- 172.21.15.153:0/1428172498 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7faa5405acf0 con 0x7faa68103990 2024-03-21T10:42:59.309 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.311+0000 7faa6ca00700 1 -- 172.21.15.153:0/1428172498 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7faa6804ea90 con 0x7faa68103990 2024-03-21T10:42:59.309 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.313+0000 7faa5ea00700 1 -- 172.21.15.153:0/1428172498 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1876 (secure 0 0 0) 0x7faa5405a880 con 0x7faa68103990 2024-03-21T10:42:59.310 INFO:teuthology.orchestra.run.smithi153.stdout:e32 2024-03-21T10:42:59.310 INFO:teuthology.orchestra.run.smithi153.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-21T10:42:59.310 INFO:teuthology.orchestra.run.smithi153.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:42:59.310 INFO:teuthology.orchestra.run.smithi153.stdout:legacy client fscid: 1 2024-03-21T10:42:59.310 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:42:59.311 INFO:teuthology.orchestra.run.smithi153.stdout:Filesystem 'cephfs' (1) 2024-03-21T10:42:59.311 INFO:teuthology.orchestra.run.smithi153.stdout:fs_name cephfs 2024-03-21T10:42:59.311 INFO:teuthology.orchestra.run.smithi153.stdout:epoch 23 2024-03-21T10:42:59.311 INFO:teuthology.orchestra.run.smithi153.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-21T10:42:59.311 INFO:teuthology.orchestra.run.smithi153.stdout:created 2024-03-21T10:04:27.086576+0000 2024-03-21T10:42:59.311 INFO:teuthology.orchestra.run.smithi153.stdout:modified 2024-03-21T10:19:05.022092+0000 2024-03-21T10:42:59.311 INFO:teuthology.orchestra.run.smithi153.stdout:tableserver 0 2024-03-21T10:42:59.311 INFO:teuthology.orchestra.run.smithi153.stdout:root 0 2024-03-21T10:42:59.311 INFO:teuthology.orchestra.run.smithi153.stdout:session_timeout 60 2024-03-21T10:42:59.311 INFO:teuthology.orchestra.run.smithi153.stdout:session_autoclose 300 2024-03-21T10:42:59.311 INFO:teuthology.orchestra.run.smithi153.stdout:max_file_size 1099511627776 2024-03-21T10:42:59.311 INFO:teuthology.orchestra.run.smithi153.stdout:max_xattr_size 65536 2024-03-21T10:42:59.311 INFO:teuthology.orchestra.run.smithi153.stdout:required_client_features {} 2024-03-21T10:42:59.311 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure 0 2024-03-21T10:42:59.311 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure_osd_epoch 94 2024-03-21T10:42:59.311 INFO:teuthology.orchestra.run.smithi153.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:42:59.312 INFO:teuthology.orchestra.run.smithi153.stdout:max_mds 2 2024-03-21T10:42:59.312 INFO:teuthology.orchestra.run.smithi153.stdout:in 0,1 2024-03-21T10:42:59.312 INFO:teuthology.orchestra.run.smithi153.stdout:up {} 2024-03-21T10:42:59.312 INFO:teuthology.orchestra.run.smithi153.stdout:failed 0,1 2024-03-21T10:42:59.312 INFO:teuthology.orchestra.run.smithi153.stdout:damaged 2024-03-21T10:42:59.312 INFO:teuthology.orchestra.run.smithi153.stdout:stopped 2024-03-21T10:42:59.312 INFO:teuthology.orchestra.run.smithi153.stdout:data_pools [3] 2024-03-21T10:42:59.312 INFO:teuthology.orchestra.run.smithi153.stdout:metadata_pool 2 2024-03-21T10:42:59.312 INFO:teuthology.orchestra.run.smithi153.stdout:inline_data enabled 2024-03-21T10:42:59.312 INFO:teuthology.orchestra.run.smithi153.stdout:balancer 2024-03-21T10:42:59.312 INFO:teuthology.orchestra.run.smithi153.stdout:bal_rank_mask -1 2024-03-21T10:42:59.312 INFO:teuthology.orchestra.run.smithi153.stdout:standby_count_wanted 1 2024-03-21T10:42:59.312 INFO:teuthology.orchestra.run.smithi153.stdout:qdb_cluster leader: 0 members: 2024-03-21T10:42:59.312 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:42:59.312 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:42:59.313 INFO:teuthology.orchestra.run.smithi153.stdout:Standby daemons: 2024-03-21T10:42:59.313 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:42:59.313 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.zipckr{-1:34442} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6826/1103846295,v1:172.21.15.153:6827/1103846295] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:42:59.313 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.iuyluj{-1:34474} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6828/1738630832,v1:172.21.15.153:6829/1738630832] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:42:59.313 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.yyimst{-1:44313} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.174:6824/4127786797,v1:172.21.15.174:6825/4127786797] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:42:59.313 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.nhozag{-1:44687} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.174:6826/2767517845,v1:172.21.15.174:6827/2767517845] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:42:59.314 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.318+0000 7faa6ca00700 1 -- 172.21.15.153:0/1428172498 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7faa4c06e460 msgr2=0x7faa4c070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:42:59.314 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.318+0000 7faa6ca00700 1 --2- 172.21.15.153:0/1428172498 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7faa4c06e460 0x7faa4c070920 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7faa58006010 tx=0x7faa5800b540 comp rx=0 tx=0).stop 2024-03-21T10:42:59.315 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.318+0000 7faa6ca00700 1 -- 172.21.15.153:0/1428172498 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faa68103990 msgr2=0x7faa6819ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:42:59.315 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.318+0000 7faa6ca00700 1 --2- 172.21.15.153:0/1428172498 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faa68103990 0x7faa6819ed40 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7faa5400b480 tx=0x7faa5400b840 comp rx=0 tx=0).stop 2024-03-21T10:42:59.315 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.318+0000 7faa6ca00700 1 -- 172.21.15.153:0/1428172498 shutdown_connections 2024-03-21T10:42:59.315 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.318+0000 7faa6ca00700 1 --2- 172.21.15.153:0/1428172498 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faa68102fe0 0x7faa6819e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:42:59.315 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.318+0000 7faa6ca00700 1 --2- 172.21.15.153:0/1428172498 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7faa4c06e460 0x7faa4c070920 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:42:59.315 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.318+0000 7faa6ca00700 1 --2- 172.21.15.153:0/1428172498 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7faa68103990 0x7faa6819ed40 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:42:59.315 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.318+0000 7faa6ca00700 1 -- 172.21.15.153:0/1428172498 >> 172.21.15.153:0/1428172498 conn(0x7faa680fe8d0 msgr2=0x7faa680fff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:42:59.315 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.318+0000 7faa6ca00700 1 -- 172.21.15.153:0/1428172498 shutdown_connections 2024-03-21T10:42:59.315 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.318+0000 7faa6ca00700 1 -- 172.21.15.153:0/1428172498 wait complete. 2024-03-21T10:42:59.316 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 32 2024-03-21T10:42:59.475 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.478+0000 7f8754a00700 1 -- 172.21.15.153:0/3100272818 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f87501017c0 msgr2=0x7f8750105810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:42:59.475 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.478+0000 7f8754a00700 1 --2- 172.21.15.153:0/3100272818 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f87501017c0 0x7f8750105810 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f873c009910 tx=0x7f873c009c20 comp rx=0 tx=0).stop 2024-03-21T10:42:59.475 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.478+0000 7f8754a00700 1 -- 172.21.15.153:0/3100272818 shutdown_connections 2024-03-21T10:42:59.475 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.478+0000 7f8754a00700 1 --2- 172.21.15.153:0/3100272818 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f87501017c0 0x7f8750105810 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:42:59.476 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.478+0000 7f8754a00700 1 --2- 172.21.15.153:0/3100272818 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8750100e10 0x7f87501011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:42:59.476 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.478+0000 7f8754a00700 1 -- 172.21.15.153:0/3100272818 >> 172.21.15.153:0/3100272818 conn(0x7f87500fc6c0 msgr2=0x7f87500feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:42:59.476 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.479+0000 7f8754a00700 1 -- 172.21.15.153:0/3100272818 shutdown_connections 2024-03-21T10:42:59.476 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.479+0000 7f8754a00700 1 -- 172.21.15.153:0/3100272818 wait complete. 2024-03-21T10:42:59.477 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.480+0000 7f8754a00700 1 Processor -- start 2024-03-21T10:42:59.477 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.480+0000 7f8754a00700 1 -- start start 2024-03-21T10:42:59.477 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.480+0000 7f8754a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8750100e10 0x7f8750073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:42:59.477 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.480+0000 7f8754a00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f87501017c0 0x7f87500735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:42:59.477 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.480+0000 7f8754a00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8750073b10 con 0x7f8750100e10 2024-03-21T10:42:59.477 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.480+0000 7f8754a00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8750073c50 con 0x7f87501017c0 2024-03-21T10:42:59.477 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.480+0000 7f874de00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8750100e10 0x7f8750073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:42:59.477 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.481+0000 7f874de00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8750100e10 0x7f8750073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:35886/0 (socket says 172.21.15.153:35886) 2024-03-21T10:42:59.477 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.481+0000 7f874de00700 1 -- 172.21.15.153:0/3015853928 learned_addr learned my addr 172.21.15.153:0/3015853928 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:42:59.478 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.481+0000 7f874d400700 1 --2- 172.21.15.153:0/3015853928 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f87501017c0 0x7f87500735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:42:59.479 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.481+0000 7f874de00700 1 -- 172.21.15.153:0/3015853928 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f87501017c0 msgr2=0x7f87500735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:42:59.479 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.481+0000 7f874de00700 1 --2- 172.21.15.153:0/3015853928 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f87501017c0 0x7f87500735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:42:59.479 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.481+0000 7f874de00700 1 -- 172.21.15.153:0/3015853928 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f873c0095f0 con 0x7f8750100e10 2024-03-21T10:42:59.479 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.481+0000 7f874de00700 1 --2- 172.21.15.153:0/3015853928 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8750100e10 0x7f8750073090 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f874000d8d0 tx=0x7f874000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:42:59.479 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.481+0000 7f8746a00700 1 -- 172.21.15.153:0/3015853928 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8740009940 con 0x7f8750100e10 2024-03-21T10:42:59.479 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.482+0000 7f8746a00700 1 -- 172.21.15.153:0/3015853928 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8740010460 con 0x7f8750100e10 2024-03-21T10:42:59.479 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.482+0000 7f8754a00700 1 -- 172.21.15.153:0/3015853928 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87501a29d0 con 0x7f8750100e10 2024-03-21T10:42:59.479 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.482+0000 7f8746a00700 1 -- 172.21.15.153:0/3015853928 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f874000f5f0 con 0x7f8750100e10 2024-03-21T10:42:59.479 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.482+0000 7f8754a00700 1 -- 172.21.15.153:0/3015853928 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87501a2de0 con 0x7f8750100e10 2024-03-21T10:42:59.480 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.483+0000 7f8754a00700 1 -- 172.21.15.153:0/3015853928 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8750109210 con 0x7f8750100e10 2024-03-21T10:42:59.480 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.484+0000 7f8746a00700 1 -- 172.21.15.153:0/3015853928 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f87400105d0 con 0x7f8750100e10 2024-03-21T10:42:59.481 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.485+0000 7f8746a00700 1 --2- 172.21.15.153:0/3015853928 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f873406e460 0x7f8734070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:42:59.482 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.485+0000 7f8746a00700 1 -- 172.21.15.153:0/3015853928 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f874008e5b0 con 0x7f8750100e10 2024-03-21T10:42:59.482 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.485+0000 7f874d400700 1 --2- 172.21.15.153:0/3015853928 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f873406e460 0x7f8734070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:42:59.482 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.486+0000 7f874d400700 1 --2- 172.21.15.153:0/3015853928 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f873406e460 0x7f8734070920 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f873c006010 tx=0x7f873c00b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:42:59.486 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.490+0000 7f8746a00700 1 -- 172.21.15.153:0/3015853928 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f87400575d0 con 0x7f8750100e10 2024-03-21T10:42:59.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:42:59 smithi153 ceph-mon[113733]: from='client.35658 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:42:59.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:42:59 smithi153 ceph-mon[113733]: pgmap v935: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:42:59.553 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:42:59 smithi153 ceph-mon[113733]: from='client.35662 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:42:59.553 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:42:59 smithi153 ceph-mon[113733]: from='client.35666 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:42:59.553 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:42:59 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/2980865480' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:42:59.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:42:59 smithi174 ceph-mon[88004]: from='client.35658 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:42:59.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:42:59 smithi174 ceph-mon[88004]: pgmap v935: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:42:59.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:42:59 smithi174 ceph-mon[88004]: from='client.35662 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:42:59.698 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:42:59 smithi174 ceph-mon[88004]: from='client.35666 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:42:59.698 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:42:59 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/2980865480' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:42:59.730 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.734+0000 7f8754a00700 1 -- 172.21.15.153:0/3015853928 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f875002cd50 con 0x7f873406e460 2024-03-21T10:42:59.733 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.737+0000 7f8746a00700 1 -- 172.21.15.153:0/3015853928 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f875002cd50 con 0x7f873406e460 2024-03-21T10:42:59.733 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:42:59.733 INFO:teuthology.orchestra.run.smithi153.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232", 2024-03-21T10:42:59.734 INFO:teuthology.orchestra.run.smithi153.stdout: "in_progress": true, 2024-03-21T10:42:59.734 INFO:teuthology.orchestra.run.smithi153.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-21T10:42:59.734 INFO:teuthology.orchestra.run.smithi153.stdout: "services_complete": [ 2024-03-21T10:42:59.734 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr", 2024-03-21T10:42:59.734 INFO:teuthology.orchestra.run.smithi153.stdout: "mon", 2024-03-21T10:42:59.734 INFO:teuthology.orchestra.run.smithi153.stdout: "mds", 2024-03-21T10:42:59.734 INFO:teuthology.orchestra.run.smithi153.stdout: "crash", 2024-03-21T10:42:59.734 INFO:teuthology.orchestra.run.smithi153.stdout: "osd" 2024-03-21T10:42:59.734 INFO:teuthology.orchestra.run.smithi153.stdout: ], 2024-03-21T10:42:59.734 INFO:teuthology.orchestra.run.smithi153.stdout: "progress": "16/23 daemons upgraded", 2024-03-21T10:42:59.734 INFO:teuthology.orchestra.run.smithi153.stdout: "message": "Currently upgrading mds daemons", 2024-03-21T10:42:59.734 INFO:teuthology.orchestra.run.smithi153.stdout: "is_paused": false 2024-03-21T10:42:59.734 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:42:59.736 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.739+0000 7f8754a00700 1 -- 172.21.15.153:0/3015853928 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f873406e460 msgr2=0x7f8734070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:42:59.736 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.739+0000 7f8754a00700 1 --2- 172.21.15.153:0/3015853928 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f873406e460 0x7f8734070920 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f873c006010 tx=0x7f873c00b540 comp rx=0 tx=0).stop 2024-03-21T10:42:59.737 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.740+0000 7f8754a00700 1 -- 172.21.15.153:0/3015853928 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8750100e10 msgr2=0x7f8750073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:42:59.737 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.740+0000 7f8754a00700 1 --2- 172.21.15.153:0/3015853928 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8750100e10 0x7f8750073090 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f874000d8d0 tx=0x7f874000dc90 comp rx=0 tx=0).stop 2024-03-21T10:42:59.737 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.740+0000 7f8754a00700 1 -- 172.21.15.153:0/3015853928 shutdown_connections 2024-03-21T10:42:59.737 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.740+0000 7f8754a00700 1 --2- 172.21.15.153:0/3015853928 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f873406e460 0x7f8734070920 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:42:59.737 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.740+0000 7f8754a00700 1 --2- 172.21.15.153:0/3015853928 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8750100e10 0x7f8750073090 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:42:59.737 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.740+0000 7f8754a00700 1 --2- 172.21.15.153:0/3015853928 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f87501017c0 0x7f87500735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:42:59.737 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.740+0000 7f8754a00700 1 -- 172.21.15.153:0/3015853928 >> 172.21.15.153:0/3015853928 conn(0x7f87500fc6c0 msgr2=0x7f87500fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:42:59.737 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.740+0000 7f8754a00700 1 -- 172.21.15.153:0/3015853928 shutdown_connections 2024-03-21T10:42:59.737 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.740+0000 7f8754a00700 1 -- 172.21.15.153:0/3015853928 wait complete. 2024-03-21T10:42:59.891 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.894+0000 7fb9caa00700 1 -- 172.21.15.153:0/1363369442 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb9c4068d30 msgr2=0x7fb9c410d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:42:59.891 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.894+0000 7fb9caa00700 1 --2- 172.21.15.153:0/1363369442 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb9c4068d30 0x7fb9c410d240 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fb9b0009910 tx=0x7fb9b0009c20 comp rx=0 tx=0).stop 2024-03-21T10:42:59.891 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.894+0000 7fb9caa00700 1 -- 172.21.15.153:0/1363369442 shutdown_connections 2024-03-21T10:42:59.891 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.894+0000 7fb9caa00700 1 --2- 172.21.15.153:0/1363369442 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb9c4068d30 0x7fb9c410d240 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:42:59.891 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.894+0000 7fb9caa00700 1 --2- 172.21.15.153:0/1363369442 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb9c4068410 0x7fb9c40687f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:42:59.892 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.894+0000 7fb9caa00700 1 -- 172.21.15.153:0/1363369442 >> 172.21.15.153:0/1363369442 conn(0x7fb9c40755a0 msgr2=0x7fb9c40759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:42:59.892 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.894+0000 7fb9caa00700 1 -- 172.21.15.153:0/1363369442 shutdown_connections 2024-03-21T10:42:59.892 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.894+0000 7fb9caa00700 1 -- 172.21.15.153:0/1363369442 wait complete. 2024-03-21T10:42:59.893 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.895+0000 7fb9caa00700 1 Processor -- start 2024-03-21T10:42:59.893 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.895+0000 7fb9caa00700 1 -- start start 2024-03-21T10:42:59.893 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.896+0000 7fb9caa00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb9c4068410 0x7fb9c419e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:42:59.893 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.896+0000 7fb9caa00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb9c4068d30 0x7fb9c419ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:42:59.893 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.896+0000 7fb9caa00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c419f2b0 con 0x7fb9c4068d30 2024-03-21T10:42:59.893 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.896+0000 7fb9caa00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c4198760 con 0x7fb9c4068410 2024-03-21T10:42:59.893 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.896+0000 7fb9c3400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb9c4068d30 0x7fb9c419ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:42:59.893 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.896+0000 7fb9c3400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb9c4068d30 0x7fb9c419ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:35898/0 (socket says 172.21.15.153:35898) 2024-03-21T10:42:59.894 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.896+0000 7fb9c3400700 1 -- 172.21.15.153:0/4111997733 learned_addr learned my addr 172.21.15.153:0/4111997733 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:42:59.894 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.897+0000 7fb9c3e00700 1 --2- 172.21.15.153:0/4111997733 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb9c4068410 0x7fb9c419e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:42:59.894 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.897+0000 7fb9c3400700 1 -- 172.21.15.153:0/4111997733 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb9c4068410 msgr2=0x7fb9c419e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:42:59.894 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.897+0000 7fb9c3400700 1 --2- 172.21.15.153:0/4111997733 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb9c4068410 0x7fb9c419e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:42:59.894 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.897+0000 7fb9c3400700 1 -- 172.21.15.153:0/4111997733 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9b00095f0 con 0x7fb9c4068d30 2024-03-21T10:42:59.895 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.897+0000 7fb9c3400700 1 --2- 172.21.15.153:0/4111997733 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb9c4068d30 0x7fb9c419ec20 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fb9b000b5c0 tx=0x7fb9b000be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:42:59.895 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.897+0000 7fb9c0c00700 1 -- 172.21.15.153:0/4111997733 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb9b001d070 con 0x7fb9c4068d30 2024-03-21T10:42:59.895 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.897+0000 7fb9c0c00700 1 -- 172.21.15.153:0/4111997733 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb9b000f460 con 0x7fb9c4068d30 2024-03-21T10:42:59.895 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.897+0000 7fb9c0c00700 1 -- 172.21.15.153:0/4111997733 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb9b0017630 con 0x7fb9c4068d30 2024-03-21T10:42:59.895 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.897+0000 7fb9caa00700 1 -- 172.21.15.153:0/4111997733 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9c4198a40 con 0x7fb9c4068d30 2024-03-21T10:42:59.895 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.898+0000 7fb9caa00700 1 -- 172.21.15.153:0/4111997733 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9c4198f90 con 0x7fb9c4068d30 2024-03-21T10:42:59.895 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.899+0000 7fb9caa00700 1 -- 172.21.15.153:0/4111997733 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9c410a9b0 con 0x7fb9c4068d30 2024-03-21T10:42:59.896 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.900+0000 7fb9c0c00700 1 -- 172.21.15.153:0/4111997733 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fb9b000f5d0 con 0x7fb9c4068d30 2024-03-21T10:42:59.897 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.901+0000 7fb9c0c00700 1 --2- 172.21.15.153:0/4111997733 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb9a806e460 0x7fb9a8070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:42:59.897 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.901+0000 7fb9c0c00700 1 -- 172.21.15.153:0/4111997733 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fb9b008faa0 con 0x7fb9c4068d30 2024-03-21T10:42:59.897 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.901+0000 7fb9c3e00700 1 --2- 172.21.15.153:0/4111997733 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb9a806e460 0x7fb9a8070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:42:59.898 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.902+0000 7fb9c3e00700 1 --2- 172.21.15.153:0/4111997733 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb9a806e460 0x7fb9a8070920 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fb9b4007720 tx=0x7fb9b40076b0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:42:59.902 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:42:59.906+0000 7fb9c0c00700 1 -- 172.21.15.153:0/4111997733 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb9b005c670 con 0x7fb9c4068d30 2024-03-21T10:43:00.207 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:00.210+0000 7fb9caa00700 1 -- 172.21.15.153:0/4111997733 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fb9c404ea90 con 0x7fb9c4068d30 2024-03-21T10:43:00.208 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:00.211+0000 7fb9c0c00700 1 -- 172.21.15.153:0/4111997733 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4493 (secure 0 0 0) 0x7fb9b005c200 con 0x7fb9c4068d30 2024-03-21T10:43:00.208 INFO:teuthology.orchestra.run.smithi153.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-03-21T10:43:00.209 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-03-21T10:43:00.209 INFO:teuthology.orchestra.run.smithi153.stdout: host smithi174 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi174/config 2024-03-21T10:43:00.209 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/podman: timeout after 895 seconds 2024-03-21T10:43:00.209 INFO:teuthology.orchestra.run.smithi153.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 -e NODE_NAME=smithi174 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/run/ceph:z -v /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpd910r3_i:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 inventory --format=json-pretty --filter-for-batch 2024-03-21T10:43:00.209 INFO:teuthology.orchestra.run.smithi153.stdout:Traceback (most recent call last): 2024-03-21T10:43:00.209 INFO:teuthology.orchestra.run.smithi153.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-03-21T10:43:00.209 INFO:teuthology.orchestra.run.smithi153.stdout: return _run_code(code, main_globals, None, 2024-03-21T10:43:00.209 INFO:teuthology.orchestra.run.smithi153.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-03-21T10:43:00.209 INFO:teuthology.orchestra.run.smithi153.stdout: exec(code, run_globals) 2024-03-21T10:43:00.210 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 5606, in 2024-03-21T10:43:00.210 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 5594, in main 2024-03-21T10:43:00.210 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 406, in _infer_config 2024-03-21T10:43:00.210 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 321, in _infer_fsid 2024-03-21T10:43:00.210 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 434, in _infer_image 2024-03-21T10:43:00.210 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 308, in _validate_fsid 2024-03-21T10:43:00.210 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 3357, in command_ceph_volume 2024-03-21T10:43:00.210 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-03-21T10:43:00.210 INFO:teuthology.orchestra.run.smithi153.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 -e NODE_NAME=smithi174 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/run/ceph:z -v /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpd910r3_i:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 inventory --format=json-pretty --filter-for-batch 2024-03-21T10:43:00.210 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-21T10:43:00.211 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is degraded 2024-03-21T10:43:00.211 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-21T10:43:00.211 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-21T10:43:00.211 INFO:teuthology.orchestra.run.smithi153.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-21T10:43:00.211 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-21T10:43:00.212 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:00.214+0000 7fb9caa00700 1 -- 172.21.15.153:0/4111997733 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb9a806e460 msgr2=0x7fb9a8070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:43:00.212 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:00.214+0000 7fb9caa00700 1 --2- 172.21.15.153:0/4111997733 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb9a806e460 0x7fb9a8070920 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fb9b4007720 tx=0x7fb9b40076b0 comp rx=0 tx=0).stop 2024-03-21T10:43:00.212 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:00.214+0000 7fb9caa00700 1 -- 172.21.15.153:0/4111997733 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb9c4068d30 msgr2=0x7fb9c419ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:43:00.212 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:00.214+0000 7fb9caa00700 1 --2- 172.21.15.153:0/4111997733 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb9c4068d30 0x7fb9c419ec20 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fb9b000b5c0 tx=0x7fb9b000be00 comp rx=0 tx=0).stop 2024-03-21T10:43:00.212 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:00.214+0000 7fb9caa00700 1 -- 172.21.15.153:0/4111997733 shutdown_connections 2024-03-21T10:43:00.212 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:00.214+0000 7fb9caa00700 1 --2- 172.21.15.153:0/4111997733 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb9c4068410 0x7fb9c419e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:43:00.212 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:00.214+0000 7fb9caa00700 1 --2- 172.21.15.153:0/4111997733 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb9a806e460 0x7fb9a8070920 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:43:00.212 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:00.214+0000 7fb9caa00700 1 --2- 172.21.15.153:0/4111997733 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb9c4068d30 0x7fb9c419ec20 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:43:00.212 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:00.214+0000 7fb9caa00700 1 -- 172.21.15.153:0/4111997733 >> 172.21.15.153:0/4111997733 conn(0x7fb9c40755a0 msgr2=0x7fb9c40fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:43:00.213 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:00.215+0000 7fb9caa00700 1 -- 172.21.15.153:0/4111997733 shutdown_connections 2024-03-21T10:43:00.213 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:00.215+0000 7fb9caa00700 1 -- 172.21.15.153:0/4111997733 wait complete. 2024-03-21T10:43:00.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:43:00 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/1428172498' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:43:00.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:43:00 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/4111997733' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:43:00.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:43:00 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/1428172498' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:43:00.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:43:00 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/4111997733' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:43:01.507 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:43:01 smithi174 ceph-mon[88004]: from='client.35678 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:43:01.508 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:43:01 smithi174 ceph-mon[88004]: pgmap v936: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:43:01.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:43:01 smithi153 ceph-mon[113733]: from='client.35678 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:43:01.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:43:01 smithi153 ceph-mon[113733]: pgmap v936: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:43:02.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:43:02 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:43:02.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:43:02 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:43:03.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:43:03 smithi153 ceph-mon[113733]: pgmap v937: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:43:03.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:43:03 smithi174 ceph-mon[88004]: pgmap v937: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:43:05.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:43:05 smithi153 ceph-mon[113733]: pgmap v938: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:43:05.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:43:05 smithi174 ceph-mon[88004]: pgmap v938: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:43:07.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:43:07 smithi153 ceph-mon[113733]: pgmap v939: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:43:07.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:43:07 smithi174 ceph-mon[88004]: pgmap v939: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:43:09.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:43:09 smithi153 ceph-mon[113733]: pgmap v940: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:43:09.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:43:09 smithi174 ceph-mon[88004]: pgmap v940: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:43:11.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:43:11 smithi153 ceph-mon[113733]: pgmap v941: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:43:11.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:43:11 smithi174 ceph-mon[88004]: pgmap v941: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:43:13.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:43:13 smithi153 ceph-mon[113733]: pgmap v942: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:43:13.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:43:13 smithi174 ceph-mon[88004]: pgmap v942: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:43:15.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:43:15 smithi174 ceph-mon[88004]: pgmap v943: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:43:15.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:43:15 smithi153 ceph-mon[113733]: pgmap v943: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:43:16.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:43:16 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:43:16.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:43:16 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:43:17.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:43:17 smithi174 ceph-mon[88004]: pgmap v944: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:43:17.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:43:17 smithi153 ceph-mon[113733]: pgmap v944: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:43:19.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:43:19 smithi153 ceph-mon[113733]: pgmap v945: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:43:19.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:43:19 smithi174 ceph-mon[88004]: pgmap v945: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:43:21.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:43:21 smithi174 ceph-mon[88004]: pgmap v946: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:43:21.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:43:21 smithi153 ceph-mon[113733]: pgmap v946: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:43:23.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:43:23 smithi174 ceph-mon[88004]: pgmap v947: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:43:23.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:43:23 smithi153 ceph-mon[113733]: pgmap v947: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:43:25.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:43:25 smithi174 ceph-mon[88004]: pgmap v948: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:43:25.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:43:25 smithi153 ceph-mon[113733]: pgmap v948: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:43:27.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:43:27 smithi174 ceph-mon[88004]: pgmap v949: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:43:27.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:43:27 smithi153 ceph-mon[113733]: pgmap v949: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:43:29.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:43:29 smithi174 ceph-mon[88004]: pgmap v950: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:43:29.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:43:29 smithi153 ceph-mon[113733]: pgmap v950: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:43:30.383 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:30.385+0000 7fd316800700 1 -- 172.21.15.153:0/3044180880 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd310068410 msgr2=0x7fd310068890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:43:30.383 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:30.385+0000 7fd316800700 1 --2- 172.21.15.153:0/3044180880 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd310068410 0x7fd310068890 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc009910 tx=0x7fd2fc009c20 comp rx=0 tx=0).stop 2024-03-21T10:43:30.384 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:30.386+0000 7fd316800700 1 -- 172.21.15.153:0/3044180880 shutdown_connections 2024-03-21T10:43:30.384 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:30.386+0000 7fd316800700 1 --2- 172.21.15.153:0/3044180880 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd310068410 0x7fd310068890 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:43:30.384 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:30.386+0000 7fd316800700 1 --2- 172.21.15.153:0/3044180880 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd310106a40 0x7fd310106e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:43:30.384 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:30.386+0000 7fd316800700 1 -- 172.21.15.153:0/3044180880 >> 172.21.15.153:0/3044180880 conn(0x7fd310075330 msgr2=0x7fd310075740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:43:30.384 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:30.386+0000 7fd316800700 1 -- 172.21.15.153:0/3044180880 shutdown_connections 2024-03-21T10:43:30.384 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:30.386+0000 7fd316800700 1 -- 172.21.15.153:0/3044180880 wait complete. 2024-03-21T10:43:30.384 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:30.387+0000 7fd316800700 1 Processor -- start 2024-03-21T10:43:30.384 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:30.387+0000 7fd316800700 1 -- start start 2024-03-21T10:43:30.385 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:30.388+0000 7fd316800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd310068410 0x7fd3100ffae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:43:30.385 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:30.388+0000 7fd316800700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd310106a40 0x7fd310100020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:43:30.385 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:30.388+0000 7fd316800700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd310100560 con 0x7fd310068410 2024-03-21T10:43:30.385 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:30.388+0000 7fd316800700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3101006a0 con 0x7fd310106a40 2024-03-21T10:43:30.385 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:30.388+0000 7fd315400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd310068410 0x7fd3100ffae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:43:30.386 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:30.388+0000 7fd315400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd310068410 0x7fd3100ffae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:34714/0 (socket says 172.21.15.153:34714) 2024-03-21T10:43:30.386 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:30.388+0000 7fd315400700 1 -- 172.21.15.153:0/2635691244 learned_addr learned my addr 172.21.15.153:0/2635691244 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:43:30.387 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:30.388+0000 7fd314a00700 1 --2- 172.21.15.153:0/2635691244 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd310106a40 0x7fd310100020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:43:30.387 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:30.388+0000 7fd315400700 1 -- 172.21.15.153:0/2635691244 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd310106a40 msgr2=0x7fd310100020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:43:30.387 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:30.388+0000 7fd315400700 1 --2- 172.21.15.153:0/2635691244 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd310106a40 0x7fd310100020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:43:30.387 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:30.388+0000 7fd315400700 1 -- 172.21.15.153:0/2635691244 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2fc0095f0 con 0x7fd310068410 2024-03-21T10:43:30.387 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:30.389+0000 7fd315400700 1 --2- 172.21.15.153:0/2635691244 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd310068410 0x7fd3100ffae0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fd30400b4b0 tx=0x7fd30400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:43:30.387 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:30.389+0000 7fd30e000700 1 -- 172.21.15.153:0/2635691244 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd304011800 con 0x7fd310068410 2024-03-21T10:43:30.387 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:30.389+0000 7fd30e000700 1 -- 172.21.15.153:0/2635691244 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd304011e20 con 0x7fd310068410 2024-03-21T10:43:30.387 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:30.389+0000 7fd30e000700 1 -- 172.21.15.153:0/2635691244 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd30400f2e0 con 0x7fd310068410 2024-03-21T10:43:30.387 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:30.389+0000 7fd316800700 1 -- 172.21.15.153:0/2635691244 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3101a2850 con 0x7fd310068410 2024-03-21T10:43:30.387 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:30.389+0000 7fd316800700 1 -- 172.21.15.153:0/2635691244 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3101a2c60 con 0x7fd310068410 2024-03-21T10:43:30.388 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:30.390+0000 7fd316800700 1 -- 172.21.15.153:0/2635691244 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd310103ce0 con 0x7fd310068410 2024-03-21T10:43:30.388 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:30.391+0000 7fd30e000700 1 -- 172.21.15.153:0/2635691244 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fd304010380 con 0x7fd310068410 2024-03-21T10:43:30.389 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:30.392+0000 7fd30e000700 1 --2- 172.21.15.153:0/2635691244 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd2f806e460 0x7fd2f8070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:43:30.390 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:30.392+0000 7fd30e000700 1 -- 172.21.15.153:0/2635691244 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fd30408e020 con 0x7fd310068410 2024-03-21T10:43:30.390 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:30.392+0000 7fd314a00700 1 --2- 172.21.15.153:0/2635691244 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd2f806e460 0x7fd2f8070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:43:30.390 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:30.393+0000 7fd314a00700 1 --2- 172.21.15.153:0/2635691244 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd2f806e460 0x7fd2f8070920 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc006010 tx=0x7fd2fc00b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:43:30.395 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:30.397+0000 7fd30e000700 1 -- 172.21.15.153:0/2635691244 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd304057210 con 0x7fd310068410 2024-03-21T10:43:30.639 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:30.641+0000 7fd316800700 1 -- 172.21.15.153:0/2635691244 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd31002cda0 con 0x7fd2f806e460 2024-03-21T10:43:30.641 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:30.644+0000 7fd30e000700 1 -- 172.21.15.153:0/2635691244 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fd31002cda0 con 0x7fd2f806e460 2024-03-21T10:43:30.645 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:30.647+0000 7fd316800700 1 -- 172.21.15.153:0/2635691244 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd2f806e460 msgr2=0x7fd2f8070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:43:30.645 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:30.647+0000 7fd316800700 1 --2- 172.21.15.153:0/2635691244 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd2f806e460 0x7fd2f8070920 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc006010 tx=0x7fd2fc00b540 comp rx=0 tx=0).stop 2024-03-21T10:43:30.645 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:30.647+0000 7fd316800700 1 -- 172.21.15.153:0/2635691244 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd310068410 msgr2=0x7fd3100ffae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:43:30.645 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:30.647+0000 7fd316800700 1 --2- 172.21.15.153:0/2635691244 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd310068410 0x7fd3100ffae0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fd30400b4b0 tx=0x7fd30400b870 comp rx=0 tx=0).stop 2024-03-21T10:43:30.645 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:30.647+0000 7fd316800700 1 -- 172.21.15.153:0/2635691244 shutdown_connections 2024-03-21T10:43:30.645 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:30.647+0000 7fd316800700 1 --2- 172.21.15.153:0/2635691244 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd2f806e460 0x7fd2f8070920 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:43:30.645 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:30.647+0000 7fd316800700 1 --2- 172.21.15.153:0/2635691244 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd310068410 0x7fd3100ffae0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:43:30.645 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:30.647+0000 7fd316800700 1 --2- 172.21.15.153:0/2635691244 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd310106a40 0x7fd310100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:43:30.645 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:30.647+0000 7fd316800700 1 -- 172.21.15.153:0/2635691244 >> 172.21.15.153:0/2635691244 conn(0x7fd310075330 msgr2=0x7fd3100feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:43:30.645 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:30.647+0000 7fd316800700 1 -- 172.21.15.153:0/2635691244 shutdown_connections 2024-03-21T10:43:30.645 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:30.647+0000 7fd316800700 1 -- 172.21.15.153:0/2635691244 wait complete. 2024-03-21T10:43:30.661 INFO:teuthology.orchestra.run.smithi153.stdout:true 2024-03-21T10:43:30.791 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:30.794+0000 7fec81800700 1 -- 172.21.15.153:0/1076265235 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec7c1071d0 msgr2=0x7fec7c107650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:43:30.792 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:30.794+0000 7fec81800700 1 --2- 172.21.15.153:0/1076265235 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec7c1071d0 0x7fec7c107650 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fec68009910 tx=0x7fec68009c20 comp rx=0 tx=0).stop 2024-03-21T10:43:30.792 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:30.794+0000 7fec81800700 1 -- 172.21.15.153:0/1076265235 shutdown_connections 2024-03-21T10:43:30.792 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:30.794+0000 7fec81800700 1 --2- 172.21.15.153:0/1076265235 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec7c1071d0 0x7fec7c107650 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:43:30.792 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:30.794+0000 7fec81800700 1 --2- 172.21.15.153:0/1076265235 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fec7c10d280 0x7fec7c10d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:43:30.792 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:30.794+0000 7fec81800700 1 -- 172.21.15.153:0/1076265235 >> 172.21.15.153:0/1076265235 conn(0x7fec7c0769e0 msgr2=0x7fec7c078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:43:30.792 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:30.794+0000 7fec81800700 1 -- 172.21.15.153:0/1076265235 shutdown_connections 2024-03-21T10:43:30.792 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:30.794+0000 7fec81800700 1 -- 172.21.15.153:0/1076265235 wait complete. 2024-03-21T10:43:30.793 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:30.795+0000 7fec81800700 1 Processor -- start 2024-03-21T10:43:30.793 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:30.795+0000 7fec81800700 1 -- start start 2024-03-21T10:43:30.793 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:30.795+0000 7fec81800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec7c1071d0 0x7fec7c19cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:43:30.793 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:30.795+0000 7fec81800700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fec7c10d280 0x7fec7c19d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:43:30.793 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:30.795+0000 7fec81800700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec7c19d9b0 con 0x7fec7c1071d0 2024-03-21T10:43:30.793 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:30.795+0000 7fec81800700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec7c1a1740 con 0x7fec7c10d280 2024-03-21T10:43:30.794 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:30.796+0000 7fec7aa00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec7c1071d0 0x7fec7c19cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:43:30.794 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:30.796+0000 7fec7aa00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec7c1071d0 0x7fec7c19cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:34720/0 (socket says 172.21.15.153:34720) 2024-03-21T10:43:30.794 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:30.796+0000 7fec7aa00700 1 -- 172.21.15.153:0/3534836963 learned_addr learned my addr 172.21.15.153:0/3534836963 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:43:30.794 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:30.796+0000 7fec7aa00700 1 -- 172.21.15.153:0/3534836963 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fec7c10d280 msgr2=0x7fec7c19d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:43:30.794 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:30.796+0000 7fec7aa00700 1 --2- 172.21.15.153:0/3534836963 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fec7c10d280 0x7fec7c19d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:43:30.794 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:30.796+0000 7fec7aa00700 1 -- 172.21.15.153:0/3534836963 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec680095f0 con 0x7fec7c1071d0 2024-03-21T10:43:30.795 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:30.796+0000 7fec7aa00700 1 --2- 172.21.15.153:0/3534836963 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec7c1071d0 0x7fec7c19cd90 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fec6c00c8a0 tx=0x7fec6c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:43:30.795 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:30.797+0000 7fec73400700 1 -- 172.21.15.153:0/3534836963 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fec6c00cea0 con 0x7fec7c1071d0 2024-03-21T10:43:30.795 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:30.797+0000 7fec73400700 1 -- 172.21.15.153:0/3534836963 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fec6c004830 con 0x7fec7c1071d0 2024-03-21T10:43:30.795 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:30.797+0000 7fec73400700 1 -- 172.21.15.153:0/3534836963 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fec6c0055b0 con 0x7fec7c1071d0 2024-03-21T10:43:30.795 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:30.797+0000 7fec81800700 1 -- 172.21.15.153:0/3534836963 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec7c1a1a20 con 0x7fec7c1071d0 2024-03-21T10:43:30.795 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:30.797+0000 7fec81800700 1 -- 172.21.15.153:0/3534836963 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec7c1a1f70 con 0x7fec7c1071d0 2024-03-21T10:43:30.795 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:30.798+0000 7fec81800700 1 -- 172.21.15.153:0/3534836963 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec7c04ea90 con 0x7fec7c1071d0 2024-03-21T10:43:30.796 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:30.799+0000 7fec73400700 1 -- 172.21.15.153:0/3534836963 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fec6c0049a0 con 0x7fec7c1071d0 2024-03-21T10:43:30.797 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:30.800+0000 7fec73400700 1 --2- 172.21.15.153:0/3534836963 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fec6006e460 0x7fec60070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:43:30.797 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:30.800+0000 7fec73400700 1 -- 172.21.15.153:0/3534836963 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fec6c08e150 con 0x7fec7c1071d0 2024-03-21T10:43:30.798 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:30.801+0000 7fec7a000700 1 --2- 172.21.15.153:0/3534836963 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fec6006e460 0x7fec60070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:43:30.798 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:30.801+0000 7fec7a000700 1 --2- 172.21.15.153:0/3534836963 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fec6006e460 0x7fec60070920 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fec6800b5c0 tx=0x7fec68005960 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:43:30.802 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:30.805+0000 7fec73400700 1 -- 172.21.15.153:0/3534836963 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fec6c0571f0 con 0x7fec7c1071d0 2024-03-21T10:43:31.029 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:31.031+0000 7fec81800700 1 -- 172.21.15.153:0/3534836963 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fec7c1a2520 con 0x7fec6006e460 2024-03-21T10:43:31.031 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:31.034+0000 7fec73400700 1 -- 172.21.15.153:0/3534836963 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fec7c1a2520 con 0x7fec6006e460 2024-03-21T10:43:31.034 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:31.036+0000 7fec81800700 1 -- 172.21.15.153:0/3534836963 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fec6006e460 msgr2=0x7fec60070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:43:31.035 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:31.036+0000 7fec81800700 1 --2- 172.21.15.153:0/3534836963 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fec6006e460 0x7fec60070920 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fec6800b5c0 tx=0x7fec68005960 comp rx=0 tx=0).stop 2024-03-21T10:43:31.035 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:31.037+0000 7fec81800700 1 -- 172.21.15.153:0/3534836963 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec7c1071d0 msgr2=0x7fec7c19cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:43:31.035 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:31.037+0000 7fec81800700 1 --2- 172.21.15.153:0/3534836963 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec7c1071d0 0x7fec7c19cd90 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fec6c00c8a0 tx=0x7fec6c00cc60 comp rx=0 tx=0).stop 2024-03-21T10:43:31.035 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:31.037+0000 7fec81800700 1 -- 172.21.15.153:0/3534836963 shutdown_connections 2024-03-21T10:43:31.035 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:31.037+0000 7fec81800700 1 --2- 172.21.15.153:0/3534836963 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fec6006e460 0x7fec60070920 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:43:31.035 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:31.037+0000 7fec81800700 1 --2- 172.21.15.153:0/3534836963 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fec7c1071d0 0x7fec7c19cd90 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:43:31.035 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:31.037+0000 7fec81800700 1 --2- 172.21.15.153:0/3534836963 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fec7c10d280 0x7fec7c19d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:43:31.035 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:31.037+0000 7fec81800700 1 -- 172.21.15.153:0/3534836963 >> 172.21.15.153:0/3534836963 conn(0x7fec7c0769e0 msgr2=0x7fec7c078480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:43:31.035 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:31.037+0000 7fec81800700 1 -- 172.21.15.153:0/3534836963 shutdown_connections 2024-03-21T10:43:31.035 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:31.037+0000 7fec81800700 1 -- 172.21.15.153:0/3534836963 wait complete. 2024-03-21T10:43:31.187 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:31.189+0000 7f304b600700 1 -- 172.21.15.153:0/1134771642 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3044105a50 msgr2=0x7f3044105e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:43:31.187 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:31.189+0000 7f304b600700 1 --2- 172.21.15.153:0/1134771642 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3044105a50 0x7f3044105e30 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f3038009910 tx=0x7f3038009c20 comp rx=0 tx=0).stop 2024-03-21T10:43:31.187 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:31.189+0000 7f304b600700 1 -- 172.21.15.153:0/1134771642 shutdown_connections 2024-03-21T10:43:31.188 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:31.189+0000 7f304b600700 1 --2- 172.21.15.153:0/1134771642 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f30440ffa50 0x7f30440ffed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:43:31.188 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:31.189+0000 7f304b600700 1 --2- 172.21.15.153:0/1134771642 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3044105a50 0x7f3044105e30 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:43:31.188 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:31.189+0000 7f304b600700 1 -- 172.21.15.153:0/1134771642 >> 172.21.15.153:0/1134771642 conn(0x7f30440fb540 msgr2=0x7f30440fd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:43:31.188 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:31.190+0000 7f304b600700 1 -- 172.21.15.153:0/1134771642 shutdown_connections 2024-03-21T10:43:31.188 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:31.190+0000 7f304b600700 1 -- 172.21.15.153:0/1134771642 wait complete. 2024-03-21T10:43:31.189 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:31.191+0000 7f304b600700 1 Processor -- start 2024-03-21T10:43:31.189 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:31.191+0000 7f304b600700 1 -- start start 2024-03-21T10:43:31.189 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:31.191+0000 7f304b600700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f30440ffa50 0x7f304419e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:43:31.189 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:31.191+0000 7f304b600700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3044105a50 0x7f304419ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:43:31.189 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:31.191+0000 7f304b600700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f304419f3f0 con 0x7f30440ffa50 2024-03-21T10:43:31.190 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:31.191+0000 7f304b600700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30441988a0 con 0x7f3044105a50 2024-03-21T10:43:31.190 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:31.191+0000 7f3048c00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f30440ffa50 0x7f304419e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:43:31.190 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:31.191+0000 7f3048c00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f30440ffa50 0x7f304419e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:34736/0 (socket says 172.21.15.153:34736) 2024-03-21T10:43:31.190 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:31.192+0000 7f3048c00700 1 -- 172.21.15.153:0/1018634656 learned_addr learned my addr 172.21.15.153:0/1018634656 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:43:31.190 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:31.192+0000 7f3043e00700 1 --2- 172.21.15.153:0/1018634656 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3044105a50 0x7f304419ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:43:31.190 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:31.192+0000 7f3048c00700 1 -- 172.21.15.153:0/1018634656 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3044105a50 msgr2=0x7f304419ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:43:31.190 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:31.192+0000 7f3048c00700 1 --2- 172.21.15.153:0/1018634656 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3044105a50 0x7f304419ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:43:31.190 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:31.192+0000 7f3048c00700 1 -- 172.21.15.153:0/1018634656 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30380095f0 con 0x7f30440ffa50 2024-03-21T10:43:31.190 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:31.192+0000 7f3043e00700 1 --2- 172.21.15.153:0/1018634656 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3044105a50 0x7f304419ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:43:31.191 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:31.192+0000 7f3048c00700 1 --2- 172.21.15.153:0/1018634656 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f30440ffa50 0x7f304419e820 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f3038006010 tx=0x7f303800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:43:31.191 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:31.193+0000 7f3041600700 1 -- 172.21.15.153:0/1018634656 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f303801d070 con 0x7f30440ffa50 2024-03-21T10:43:31.191 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:31.193+0000 7f3041600700 1 -- 172.21.15.153:0/1018634656 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3038004fb0 con 0x7f30440ffa50 2024-03-21T10:43:31.191 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:31.193+0000 7f3041600700 1 -- 172.21.15.153:0/1018634656 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f303800f690 con 0x7f30440ffa50 2024-03-21T10:43:31.191 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:31.193+0000 7f304b600700 1 -- 172.21.15.153:0/1018634656 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3044198b20 con 0x7f30440ffa50 2024-03-21T10:43:31.191 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:31.193+0000 7f304b600700 1 -- 172.21.15.153:0/1018634656 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3044199010 con 0x7f30440ffa50 2024-03-21T10:43:31.191 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:31.194+0000 7f304b600700 1 -- 172.21.15.153:0/1018634656 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f304404ea90 con 0x7f30440ffa50 2024-03-21T10:43:31.192 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:31.195+0000 7f3041600700 1 -- 172.21.15.153:0/1018634656 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f30380228a0 con 0x7f30440ffa50 2024-03-21T10:43:31.193 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:31.196+0000 7f3041600700 1 --2- 172.21.15.153:0/1018634656 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f302c06e460 0x7f302c070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:43:31.193 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:31.196+0000 7f3041600700 1 -- 172.21.15.153:0/1018634656 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f303808fc90 con 0x7f30440ffa50 2024-03-21T10:43:31.193 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:31.196+0000 7f3043e00700 1 --2- 172.21.15.153:0/1018634656 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f302c06e460 0x7f302c070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:43:31.194 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:31.197+0000 7f3043e00700 1 --2- 172.21.15.153:0/1018634656 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f302c06e460 0x7f302c070920 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f3044104e10 tx=0x7f3030008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:43:31.201 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:31.204+0000 7f3041600700 1 -- 172.21.15.153:0/1018634656 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f303805c900 con 0x7f30440ffa50 2024-03-21T10:43:31.432 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:31.435+0000 7f304b600700 1 -- 172.21.15.153:0/1018634656 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f3044199490 con 0x7f302c06e460 2024-03-21T10:43:31.464 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:31.467+0000 7f3041600700 1 -- 172.21.15.153:0/1018634656 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f3044199490 con 0x7f302c06e460 2024-03-21T10:43:31.464 INFO:teuthology.orchestra.run.smithi153.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-21T10:43:31.465 INFO:teuthology.orchestra.run.smithi153.stdout:alertmanager.smithi153 smithi153 *:9093,9094 running (32m) 8m ago 46m 15.6M - 0.25.0 c8568f914cd2 9ed3f2200b5f 2024-03-21T10:43:31.465 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi153 smithi153 running (46m) 8m ago 46m 10.4M - 18.2.1 5be31c24972a 5b61994396a6 2024-03-21T10:43:31.465 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi174 smithi174 running (44m) 8m ago 44m 10.1M - 18.2.1 5be31c24972a 4f7a5fc3ac28 2024-03-21T10:43:31.465 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi153 smithi153 running (28m) 8m ago 46m 6656k - 19.0.0-2231-g2edfee68 237e43c7a65f 32c21104d5b0 2024-03-21T10:43:31.465 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi174 smithi174 running (28m) 8m ago 44m 6660k - 19.0.0-2231-g2edfee68 237e43c7a65f a04e48967ab6 2024-03-21T10:43:31.465 INFO:teuthology.orchestra.run.smithi153.stdout:grafana.smithi153 smithi153 *:3000 running (31m) 8m ago 45m 75.2M - 9.4.12 2bacad6d85d8 ee24026ab786 2024-03-21T10:43:31.465 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.iuyluj smithi153 running (23m) 8m ago 38m 18.7M - 19.0.0-2231-g2edfee68 237e43c7a65f 87aff8a5de70 2024-03-21T10:43:31.465 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.zipckr smithi153 running (24m) 8m ago 38m 18.9M - 19.0.0-2231-g2edfee68 237e43c7a65f ccc262bd5dd3 2024-03-21T10:43:31.465 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.nhozag smithi174 running (8m) 8m ago 38m 10.9M - 19.0.0-2231-g2edfee68 237e43c7a65f 9e02aea72c96 2024-03-21T10:43:31.465 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.yyimst smithi174 running (23m) 8m ago 38m 18.5M - 19.0.0-2231-g2edfee68 237e43c7a65f fc1962f9351e 2024-03-21T10:43:31.465 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi153.tluxni smithi153 *:8443,9283,8765 running (35m) 8m ago 47m 512M - 19.0.0-2231-g2edfee68 237e43c7a65f 7ffee0e55382 2024-03-21T10:43:31.465 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi174.qpcwat smithi174 *:8443,9283,8765 running (33m) 8m ago 44m 459M - 19.0.0-2231-g2edfee68 237e43c7a65f 748968f216ab 2024-03-21T10:43:31.465 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi153 smithi153 running (30m) 8m ago 48m 70.8M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f dd8038303dc3 2024-03-21T10:43:31.466 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi174 smithi174 running (29m) 8m ago 44m 51.9M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f 29d78a788f2c 2024-03-21T10:43:31.466 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi153 smithi153 *:9100 running (33m) 8m ago 46m 8140k - 1.5.0 0da6a335fe13 bb835edc0e0b 2024-03-21T10:43:31.466 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi174 smithi174 *:9100 running (33m) 8m ago 44m 8610k - 1.5.0 0da6a335fe13 7c4ecad790fa 2024-03-21T10:43:31.466 INFO:teuthology.orchestra.run.smithi153.stdout:osd.0 smithi153 running (27m) 8m ago 43m 118M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 507900fd4791 2024-03-21T10:43:31.466 INFO:teuthology.orchestra.run.smithi153.stdout:osd.1 smithi153 running (27m) 8m ago 42m 93.0M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 0d646452e9c1 2024-03-21T10:43:31.466 INFO:teuthology.orchestra.run.smithi153.stdout:osd.2 smithi153 running (26m) 8m ago 42m 91.2M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 1326f2afdfcd 2024-03-21T10:43:31.466 INFO:teuthology.orchestra.run.smithi153.stdout:osd.3 smithi174 running (26m) 8m ago 41m 87.5M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 70aafac71ad5 2024-03-21T10:43:31.466 INFO:teuthology.orchestra.run.smithi153.stdout:osd.4 smithi174 running (25m) 8m ago 41m 95.8M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 0d746ce6e79d 2024-03-21T10:43:31.466 INFO:teuthology.orchestra.run.smithi153.stdout:osd.5 smithi174 running (24m) 8m ago 40m 90.3M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 53e5f35b8cac 2024-03-21T10:43:31.466 INFO:teuthology.orchestra.run.smithi153.stdout:prometheus.smithi153 smithi153 *:9095 running (32m) 8m ago 45m 88.1M - 2.43.0 a07b618ecd1d 7d95561e50c2 2024-03-21T10:43:31.467 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:31.470+0000 7f304b600700 1 -- 172.21.15.153:0/1018634656 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f302c06e460 msgr2=0x7f302c070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:43:31.468 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:31.470+0000 7f304b600700 1 --2- 172.21.15.153:0/1018634656 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f302c06e460 0x7f302c070920 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f3044104e10 tx=0x7f3030008040 comp rx=0 tx=0).stop 2024-03-21T10:43:31.468 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:31.470+0000 7f304b600700 1 -- 172.21.15.153:0/1018634656 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f30440ffa50 msgr2=0x7f304419e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:43:31.468 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:31.470+0000 7f304b600700 1 --2- 172.21.15.153:0/1018634656 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f30440ffa50 0x7f304419e820 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f3038006010 tx=0x7f303800be20 comp rx=0 tx=0).stop 2024-03-21T10:43:31.468 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:31.470+0000 7f304b600700 1 -- 172.21.15.153:0/1018634656 shutdown_connections 2024-03-21T10:43:31.468 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:31.470+0000 7f304b600700 1 --2- 172.21.15.153:0/1018634656 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f302c06e460 0x7f302c070920 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:43:31.468 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:31.470+0000 7f304b600700 1 --2- 172.21.15.153:0/1018634656 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f30440ffa50 0x7f304419e820 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:43:31.468 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:31.470+0000 7f304b600700 1 --2- 172.21.15.153:0/1018634656 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3044105a50 0x7f304419ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:43:31.468 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:31.470+0000 7f304b600700 1 -- 172.21.15.153:0/1018634656 >> 172.21.15.153:0/1018634656 conn(0x7f30440fb540 msgr2=0x7f3044068d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:43:31.469 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:31.470+0000 7f304b600700 1 -- 172.21.15.153:0/1018634656 shutdown_connections 2024-03-21T10:43:31.469 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:31.471+0000 7f304b600700 1 -- 172.21.15.153:0/1018634656 wait complete. 2024-03-21T10:43:31.638 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:31.640+0000 7fee36400700 1 -- 172.21.15.153:0/447686395 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee301071d0 msgr2=0x7fee30107650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:43:31.639 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:31.640+0000 7fee36400700 1 --2- 172.21.15.153:0/447686395 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee301071d0 0x7fee30107650 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fee1c009910 tx=0x7fee1c009c20 comp rx=0 tx=0).stop 2024-03-21T10:43:31.639 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:31.641+0000 7fee36400700 1 -- 172.21.15.153:0/447686395 shutdown_connections 2024-03-21T10:43:31.639 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:31.641+0000 7fee36400700 1 --2- 172.21.15.153:0/447686395 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee301071d0 0x7fee30107650 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:43:31.639 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:31.641+0000 7fee36400700 1 --2- 172.21.15.153:0/447686395 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fee3010d280 0x7fee3010d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:43:31.639 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:31.641+0000 7fee36400700 1 -- 172.21.15.153:0/447686395 >> 172.21.15.153:0/447686395 conn(0x7fee300769e0 msgr2=0x7fee30078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:43:31.639 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:31.641+0000 7fee36400700 1 -- 172.21.15.153:0/447686395 shutdown_connections 2024-03-21T10:43:31.639 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:31.641+0000 7fee36400700 1 -- 172.21.15.153:0/447686395 wait complete. 2024-03-21T10:43:31.639 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:43:31 smithi153 ceph-mon[113733]: pgmap v951: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:43:31.640 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:43:31 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:43:31.641 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:31.642+0000 7fee36400700 1 Processor -- start 2024-03-21T10:43:31.641 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:31.642+0000 7fee36400700 1 -- start start 2024-03-21T10:43:31.641 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:31.642+0000 7fee36400700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fee301071d0 0x7fee3019ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:43:31.641 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:31.642+0000 7fee36400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee3010d280 0x7fee3019d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:43:31.641 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:31.642+0000 7fee36400700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee3019d7f0 con 0x7fee3010d280 2024-03-21T10:43:31.642 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:31.642+0000 7fee36400700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee3019d960 con 0x7fee301071d0 2024-03-21T10:43:31.642 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:31.643+0000 7fee2ea00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee3010d280 0x7fee3019d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:43:31.642 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:31.643+0000 7fee2ea00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee3010d280 0x7fee3019d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:34760/0 (socket says 172.21.15.153:34760) 2024-03-21T10:43:31.642 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:31.643+0000 7fee2ea00700 1 -- 172.21.15.153:0/4109078380 learned_addr learned my addr 172.21.15.153:0/4109078380 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:43:31.642 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:31.643+0000 7fee2f400700 1 --2- 172.21.15.153:0/4109078380 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fee301071d0 0x7fee3019ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:43:31.642 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:31.643+0000 7fee2ea00700 1 -- 172.21.15.153:0/4109078380 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fee301071d0 msgr2=0x7fee3019ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:43:31.642 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:31.643+0000 7fee2ea00700 1 --2- 172.21.15.153:0/4109078380 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fee301071d0 0x7fee3019ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:43:31.642 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:31.643+0000 7fee2ea00700 1 -- 172.21.15.153:0/4109078380 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee1c0095f0 con 0x7fee3010d280 2024-03-21T10:43:31.642 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:31.643+0000 7fee2ea00700 1 --2- 172.21.15.153:0/4109078380 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee3010d280 0x7fee3019d1f0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fee1c005e00 tx=0x7fee1c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:43:31.642 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:31.644+0000 7fee27e00700 1 -- 172.21.15.153:0/4109078380 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fee1c01d070 con 0x7fee3010d280 2024-03-21T10:43:31.642 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:31.644+0000 7fee36400700 1 -- 172.21.15.153:0/4109078380 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee301a06a0 con 0x7fee3010d280 2024-03-21T10:43:31.642 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:31.644+0000 7fee27e00700 1 -- 172.21.15.153:0/4109078380 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fee1c004fb0 con 0x7fee3010d280 2024-03-21T10:43:31.642 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:31.644+0000 7fee27e00700 1 -- 172.21.15.153:0/4109078380 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fee1c00f690 con 0x7fee3010d280 2024-03-21T10:43:31.643 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:31.644+0000 7fee36400700 1 -- 172.21.15.153:0/4109078380 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee301a0c10 con 0x7fee3010d280 2024-03-21T10:43:31.643 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:31.645+0000 7fee36400700 1 -- 172.21.15.153:0/4109078380 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee3010f6c0 con 0x7fee3010d280 2024-03-21T10:43:31.644 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:31.647+0000 7fee27e00700 1 -- 172.21.15.153:0/4109078380 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fee1c022470 con 0x7fee3010d280 2024-03-21T10:43:31.645 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:31.648+0000 7fee27e00700 1 --2- 172.21.15.153:0/4109078380 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fee1406e460 0x7fee14070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:43:31.645 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:31.648+0000 7fee27e00700 1 -- 172.21.15.153:0/4109078380 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fee1c08fe60 con 0x7fee3010d280 2024-03-21T10:43:31.645 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:31.648+0000 7fee2f400700 1 --2- 172.21.15.153:0/4109078380 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fee1406e460 0x7fee14070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:43:31.646 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:31.649+0000 7fee2f400700 1 --2- 172.21.15.153:0/4109078380 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fee1406e460 0x7fee14070920 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fee2000ba60 tx=0x7fee20005d50 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:43:31.650 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:31.653+0000 7fee27e00700 1 -- 172.21.15.153:0/4109078380 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fee1c058e80 con 0x7fee3010d280 2024-03-21T10:43:31.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:43:31 smithi174 ceph-mon[88004]: pgmap v951: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:43:31.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:43:31 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:43:31.961 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:31.964+0000 7fee36400700 1 -- 172.21.15.153:0/4109078380 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fee3004ea90 con 0x7fee3010d280 2024-03-21T10:43:31.963 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:31.965+0000 7fee27e00700 1 -- 172.21.15.153:0/4109078380 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fee1c05c540 con 0x7fee3010d280 2024-03-21T10:43:31.963 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:43:31.963 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": { 2024-03-21T10:43:31.963 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:43:31.964 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:43:31.964 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": { 2024-03-21T10:43:31.964 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:43:31.964 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:43:31.964 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": { 2024-03-21T10:43:31.964 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 6 2024-03-21T10:43:31.964 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:43:31.964 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": { 2024-03-21T10:43:31.964 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 4 2024-03-21T10:43:31.964 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:43:31.964 INFO:teuthology.orchestra.run.smithi153.stdout: "overall": { 2024-03-21T10:43:31.964 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 14 2024-03-21T10:43:31.964 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-03-21T10:43:31.964 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:43:31.965 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:31.968+0000 7fee36400700 1 -- 172.21.15.153:0/4109078380 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fee1406e460 msgr2=0x7fee14070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:43:31.965 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:31.968+0000 7fee36400700 1 --2- 172.21.15.153:0/4109078380 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fee1406e460 0x7fee14070920 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fee2000ba60 tx=0x7fee20005d50 comp rx=0 tx=0).stop 2024-03-21T10:43:31.965 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:31.968+0000 7fee36400700 1 -- 172.21.15.153:0/4109078380 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee3010d280 msgr2=0x7fee3019d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:43:31.966 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:31.968+0000 7fee36400700 1 --2- 172.21.15.153:0/4109078380 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee3010d280 0x7fee3019d1f0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fee1c005e00 tx=0x7fee1c00be20 comp rx=0 tx=0).stop 2024-03-21T10:43:31.966 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:31.968+0000 7fee36400700 1 -- 172.21.15.153:0/4109078380 shutdown_connections 2024-03-21T10:43:31.966 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:31.968+0000 7fee36400700 1 --2- 172.21.15.153:0/4109078380 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fee301071d0 0x7fee3019ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:43:31.966 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:31.968+0000 7fee36400700 1 --2- 172.21.15.153:0/4109078380 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fee1406e460 0x7fee14070920 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:43:31.966 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:31.968+0000 7fee36400700 1 --2- 172.21.15.153:0/4109078380 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee3010d280 0x7fee3019d1f0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:43:31.966 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:31.968+0000 7fee36400700 1 -- 172.21.15.153:0/4109078380 >> 172.21.15.153:0/4109078380 conn(0x7fee300769e0 msgr2=0x7fee30078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:43:31.966 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:31.969+0000 7fee36400700 1 -- 172.21.15.153:0/4109078380 shutdown_connections 2024-03-21T10:43:31.967 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:31.969+0000 7fee36400700 1 -- 172.21.15.153:0/4109078380 wait complete. 2024-03-21T10:43:32.123 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.125+0000 7fcac1400700 1 -- 172.21.15.153:0/853878126 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcabc101770 msgr2=0x7fcabc1057c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:43:32.123 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.125+0000 7fcac1400700 1 --2- 172.21.15.153:0/853878126 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcabc101770 0x7fcabc1057c0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fcaa8009910 tx=0x7fcaa8009c20 comp rx=0 tx=0).stop 2024-03-21T10:43:32.123 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.125+0000 7fcac1400700 1 -- 172.21.15.153:0/853878126 shutdown_connections 2024-03-21T10:43:32.123 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.125+0000 7fcac1400700 1 --2- 172.21.15.153:0/853878126 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcabc101770 0x7fcabc1057c0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:43:32.123 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.125+0000 7fcac1400700 1 --2- 172.21.15.153:0/853878126 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcabc100dc0 0x7fcabc1011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:43:32.123 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.125+0000 7fcac1400700 1 -- 172.21.15.153:0/853878126 >> 172.21.15.153:0/853878126 conn(0x7fcabc0fc650 msgr2=0x7fcabc0fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:43:32.123 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.125+0000 7fcac1400700 1 -- 172.21.15.153:0/853878126 shutdown_connections 2024-03-21T10:43:32.123 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.125+0000 7fcac1400700 1 -- 172.21.15.153:0/853878126 wait complete. 2024-03-21T10:43:32.125 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.126+0000 7fcac1400700 1 Processor -- start 2024-03-21T10:43:32.125 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.126+0000 7fcac1400700 1 -- start start 2024-03-21T10:43:32.125 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.127+0000 7fcac1400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcabc100dc0 0x7fcabc198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:43:32.125 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.127+0000 7fcac1400700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcabc101770 0x7fcabc198e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:43:32.125 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.127+0000 7fcac1400700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcabc199520 con 0x7fcabc100dc0 2024-03-21T10:43:32.125 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.127+0000 7fcac1400700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcabc19d330 con 0x7fcabc101770 2024-03-21T10:43:32.125 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.127+0000 7fcaba800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcabc100dc0 0x7fcabc198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:43:32.125 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.127+0000 7fcaba800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcabc100dc0 0x7fcabc198950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:34782/0 (socket says 172.21.15.153:34782) 2024-03-21T10:43:32.125 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.127+0000 7fcaba800700 1 -- 172.21.15.153:0/3722651247 learned_addr learned my addr 172.21.15.153:0/3722651247 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:43:32.125 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.127+0000 7fcab9e00700 1 --2- 172.21.15.153:0/3722651247 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcabc101770 0x7fcabc198e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:43:32.126 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.127+0000 7fcaba800700 1 -- 172.21.15.153:0/3722651247 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcabc101770 msgr2=0x7fcabc198e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:43:32.126 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.128+0000 7fcaba800700 1 --2- 172.21.15.153:0/3722651247 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcabc101770 0x7fcabc198e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:43:32.126 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.128+0000 7fcaba800700 1 -- 172.21.15.153:0/3722651247 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaa80095f0 con 0x7fcabc100dc0 2024-03-21T10:43:32.126 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.128+0000 7fcaba800700 1 --2- 172.21.15.153:0/3722651247 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcabc100dc0 0x7fcabc198950 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fcaac00d8d0 tx=0x7fcaac00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:43:32.126 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.128+0000 7fcab3400700 1 -- 172.21.15.153:0/3722651247 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcaac009940 con 0x7fcabc100dc0 2024-03-21T10:43:32.126 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.128+0000 7fcab3400700 1 -- 172.21.15.153:0/3722651247 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcaac010460 con 0x7fcabc100dc0 2024-03-21T10:43:32.127 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.128+0000 7fcac1400700 1 -- 172.21.15.153:0/3722651247 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcabc19d610 con 0x7fcabc100dc0 2024-03-21T10:43:32.129 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.128+0000 7fcab3400700 1 -- 172.21.15.153:0/3722651247 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcaac00f5f0 con 0x7fcabc100dc0 2024-03-21T10:43:32.129 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.128+0000 7fcac1400700 1 -- 172.21.15.153:0/3722651247 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcabc19db60 con 0x7fcabc100dc0 2024-03-21T10:43:32.129 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.129+0000 7fcac1400700 1 -- 172.21.15.153:0/3722651247 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcabc104f40 con 0x7fcabc100dc0 2024-03-21T10:43:32.129 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.130+0000 7fcab3400700 1 -- 172.21.15.153:0/3722651247 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fcaac0105d0 con 0x7fcabc100dc0 2024-03-21T10:43:32.130 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.131+0000 7fcab3400700 1 --2- 172.21.15.153:0/3722651247 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fcaa006e460 0x7fcaa0070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:43:32.130 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.131+0000 7fcab3400700 1 -- 172.21.15.153:0/3722651247 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fcaac08e480 con 0x7fcabc100dc0 2024-03-21T10:43:32.130 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.132+0000 7fcab9e00700 1 --2- 172.21.15.153:0/3722651247 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fcaa006e460 0x7fcaa0070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:43:32.132 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.132+0000 7fcab9e00700 1 --2- 172.21.15.153:0/3722651247 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fcaa006e460 0x7fcaa0070920 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fcaa800b5c0 tx=0x7fcaa8005fd0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:43:32.145 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.136+0000 7fcab3400700 1 -- 172.21.15.153:0/3722651247 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fcaac05b050 con 0x7fcabc100dc0 2024-03-21T10:43:32.419 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.421+0000 7fcac1400700 1 -- 172.21.15.153:0/3722651247 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fcabc04ea90 con 0x7fcabc100dc0 2024-03-21T10:43:32.419 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.422+0000 7fcab3400700 1 -- 172.21.15.153:0/3722651247 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1876 (secure 0 0 0) 0x7fcaac016770 con 0x7fcabc100dc0 2024-03-21T10:43:32.420 INFO:teuthology.orchestra.run.smithi153.stdout:e32 2024-03-21T10:43:32.420 INFO:teuthology.orchestra.run.smithi153.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-21T10:43:32.420 INFO:teuthology.orchestra.run.smithi153.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:43:32.420 INFO:teuthology.orchestra.run.smithi153.stdout:legacy client fscid: 1 2024-03-21T10:43:32.420 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:43:32.420 INFO:teuthology.orchestra.run.smithi153.stdout:Filesystem 'cephfs' (1) 2024-03-21T10:43:32.420 INFO:teuthology.orchestra.run.smithi153.stdout:fs_name cephfs 2024-03-21T10:43:32.421 INFO:teuthology.orchestra.run.smithi153.stdout:epoch 23 2024-03-21T10:43:32.421 INFO:teuthology.orchestra.run.smithi153.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-21T10:43:32.421 INFO:teuthology.orchestra.run.smithi153.stdout:created 2024-03-21T10:04:27.086576+0000 2024-03-21T10:43:32.421 INFO:teuthology.orchestra.run.smithi153.stdout:modified 2024-03-21T10:19:05.022092+0000 2024-03-21T10:43:32.421 INFO:teuthology.orchestra.run.smithi153.stdout:tableserver 0 2024-03-21T10:43:32.421 INFO:teuthology.orchestra.run.smithi153.stdout:root 0 2024-03-21T10:43:32.421 INFO:teuthology.orchestra.run.smithi153.stdout:session_timeout 60 2024-03-21T10:43:32.421 INFO:teuthology.orchestra.run.smithi153.stdout:session_autoclose 300 2024-03-21T10:43:32.421 INFO:teuthology.orchestra.run.smithi153.stdout:max_file_size 1099511627776 2024-03-21T10:43:32.421 INFO:teuthology.orchestra.run.smithi153.stdout:max_xattr_size 65536 2024-03-21T10:43:32.421 INFO:teuthology.orchestra.run.smithi153.stdout:required_client_features {} 2024-03-21T10:43:32.421 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure 0 2024-03-21T10:43:32.422 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure_osd_epoch 94 2024-03-21T10:43:32.422 INFO:teuthology.orchestra.run.smithi153.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:43:32.422 INFO:teuthology.orchestra.run.smithi153.stdout:max_mds 2 2024-03-21T10:43:32.422 INFO:teuthology.orchestra.run.smithi153.stdout:in 0,1 2024-03-21T10:43:32.422 INFO:teuthology.orchestra.run.smithi153.stdout:up {} 2024-03-21T10:43:32.422 INFO:teuthology.orchestra.run.smithi153.stdout:failed 0,1 2024-03-21T10:43:32.422 INFO:teuthology.orchestra.run.smithi153.stdout:damaged 2024-03-21T10:43:32.422 INFO:teuthology.orchestra.run.smithi153.stdout:stopped 2024-03-21T10:43:32.422 INFO:teuthology.orchestra.run.smithi153.stdout:data_pools [3] 2024-03-21T10:43:32.422 INFO:teuthology.orchestra.run.smithi153.stdout:metadata_pool 2 2024-03-21T10:43:32.422 INFO:teuthology.orchestra.run.smithi153.stdout:inline_data enabled 2024-03-21T10:43:32.422 INFO:teuthology.orchestra.run.smithi153.stdout:balancer 2024-03-21T10:43:32.423 INFO:teuthology.orchestra.run.smithi153.stdout:bal_rank_mask -1 2024-03-21T10:43:32.423 INFO:teuthology.orchestra.run.smithi153.stdout:standby_count_wanted 1 2024-03-21T10:43:32.423 INFO:teuthology.orchestra.run.smithi153.stdout:qdb_cluster leader: 0 members: 2024-03-21T10:43:32.423 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:43:32.423 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:43:32.423 INFO:teuthology.orchestra.run.smithi153.stdout:Standby daemons: 2024-03-21T10:43:32.423 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:43:32.423 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.zipckr{-1:34442} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6826/1103846295,v1:172.21.15.153:6827/1103846295] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:43:32.423 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.iuyluj{-1:34474} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6828/1738630832,v1:172.21.15.153:6829/1738630832] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:43:32.423 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.yyimst{-1:44313} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.174:6824/4127786797,v1:172.21.15.174:6825/4127786797] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:43:32.423 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.nhozag{-1:44687} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.174:6826/2767517845,v1:172.21.15.174:6827/2767517845] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:43:32.424 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.425+0000 7fcac1400700 1 -- 172.21.15.153:0/3722651247 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fcaa006e460 msgr2=0x7fcaa0070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:43:32.424 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.425+0000 7fcac1400700 1 --2- 172.21.15.153:0/3722651247 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fcaa006e460 0x7fcaa0070920 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fcaa800b5c0 tx=0x7fcaa8005fd0 comp rx=0 tx=0).stop 2024-03-21T10:43:32.424 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.425+0000 7fcac1400700 1 -- 172.21.15.153:0/3722651247 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcabc100dc0 msgr2=0x7fcabc198950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:43:32.425 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.425+0000 7fcac1400700 1 --2- 172.21.15.153:0/3722651247 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcabc100dc0 0x7fcabc198950 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fcaac00d8d0 tx=0x7fcaac00dc90 comp rx=0 tx=0).stop 2024-03-21T10:43:32.425 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.425+0000 7fcac1400700 1 -- 172.21.15.153:0/3722651247 shutdown_connections 2024-03-21T10:43:32.425 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.425+0000 7fcac1400700 1 --2- 172.21.15.153:0/3722651247 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fcaa006e460 0x7fcaa0070920 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:43:32.425 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.425+0000 7fcac1400700 1 --2- 172.21.15.153:0/3722651247 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcabc100dc0 0x7fcabc198950 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:43:32.425 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.425+0000 7fcac1400700 1 --2- 172.21.15.153:0/3722651247 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcabc101770 0x7fcabc198e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:43:32.425 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.425+0000 7fcac1400700 1 -- 172.21.15.153:0/3722651247 >> 172.21.15.153:0/3722651247 conn(0x7fcabc0fc650 msgr2=0x7fcabc0fdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:43:32.425 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.425+0000 7fcac1400700 1 -- 172.21.15.153:0/3722651247 shutdown_connections 2024-03-21T10:43:32.425 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.425+0000 7fcac1400700 1 -- 172.21.15.153:0/3722651247 wait complete. 2024-03-21T10:43:32.425 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 32 2024-03-21T10:43:32.577 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.579+0000 7f45ebe00700 1 -- 172.21.15.153:0/3412228389 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f45ec1009c0 msgr2=0x7f45ec100e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:43:32.577 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.579+0000 7f45ebe00700 1 --2- 172.21.15.153:0/3412228389 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f45ec1009c0 0x7f45ec100e40 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f45d8009910 tx=0x7f45d8009c20 comp rx=0 tx=0).stop 2024-03-21T10:43:32.577 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.579+0000 7f45ebe00700 1 -- 172.21.15.153:0/3412228389 shutdown_connections 2024-03-21T10:43:32.577 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.579+0000 7f45ebe00700 1 --2- 172.21.15.153:0/3412228389 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f45ec1009c0 0x7f45ec100e40 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:43:32.577 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.579+0000 7f45ebe00700 1 --2- 172.21.15.153:0/3412228389 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f45ec1069c0 0x7f45ec106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:43:32.577 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.579+0000 7f45ebe00700 1 -- 172.21.15.153:0/3412228389 >> 172.21.15.153:0/3412228389 conn(0x7f45ec0fc4f0 msgr2=0x7f45ec0fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:43:32.577 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.579+0000 7f45ebe00700 1 -- 172.21.15.153:0/3412228389 shutdown_connections 2024-03-21T10:43:32.577 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.579+0000 7f45ebe00700 1 -- 172.21.15.153:0/3412228389 wait complete. 2024-03-21T10:43:32.579 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.580+0000 7f45ebe00700 1 Processor -- start 2024-03-21T10:43:32.579 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.580+0000 7f45ebe00700 1 -- start start 2024-03-21T10:43:32.579 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.581+0000 7f45ebe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f45ec1009c0 0x7f45ec1965a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:43:32.579 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.581+0000 7f45ebe00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f45ec1069c0 0x7f45ec196ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:43:32.579 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.581+0000 7f45ebe00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45ec1971c0 con 0x7f45ec1009c0 2024-03-21T10:43:32.579 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.581+0000 7f45ebe00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45ec19af50 con 0x7f45ec1069c0 2024-03-21T10:43:32.579 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.581+0000 7f45eaa00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f45ec1009c0 0x7f45ec1965a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:43:32.579 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.581+0000 7f45eaa00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f45ec1009c0 0x7f45ec1965a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:34812/0 (socket says 172.21.15.153:34812) 2024-03-21T10:43:32.579 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.581+0000 7f45eaa00700 1 -- 172.21.15.153:0/2017140723 learned_addr learned my addr 172.21.15.153:0/2017140723 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:43:32.579 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.582+0000 7f45eaa00700 1 -- 172.21.15.153:0/2017140723 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f45ec1069c0 msgr2=0x7f45ec196ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:43:32.579 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.582+0000 7f45eaa00700 1 --2- 172.21.15.153:0/2017140723 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f45ec1069c0 0x7f45ec196ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:43:32.579 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.582+0000 7f45eaa00700 1 -- 172.21.15.153:0/2017140723 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45d80095f0 con 0x7f45ec1009c0 2024-03-21T10:43:32.580 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.582+0000 7f45eaa00700 1 --2- 172.21.15.153:0/2017140723 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f45ec1009c0 0x7f45ec1965a0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f45dc00d8d0 tx=0x7f45dc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:43:32.580 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.582+0000 7f45e3400700 1 -- 172.21.15.153:0/2017140723 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f45dc009940 con 0x7f45ec1009c0 2024-03-21T10:43:32.580 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.582+0000 7f45e3400700 1 -- 172.21.15.153:0/2017140723 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f45dc010460 con 0x7f45ec1009c0 2024-03-21T10:43:32.580 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.582+0000 7f45ebe00700 1 -- 172.21.15.153:0/2017140723 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45ec19b230 con 0x7f45ec1009c0 2024-03-21T10:43:32.580 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.582+0000 7f45e3400700 1 -- 172.21.15.153:0/2017140723 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f45dc00f5f0 con 0x7f45ec1009c0 2024-03-21T10:43:32.580 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.582+0000 7f45ebe00700 1 -- 172.21.15.153:0/2017140723 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45ec19b780 con 0x7f45ec1009c0 2024-03-21T10:43:32.581 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.583+0000 7f45ebe00700 1 -- 172.21.15.153:0/2017140723 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45ec108f50 con 0x7f45ec1009c0 2024-03-21T10:43:32.581 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.584+0000 7f45e3400700 1 -- 172.21.15.153:0/2017140723 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f45dc0105d0 con 0x7f45ec1009c0 2024-03-21T10:43:32.582 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.585+0000 7f45e3400700 1 --2- 172.21.15.153:0/2017140723 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f45d006e460 0x7f45d0070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:43:32.583 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.585+0000 7f45e3400700 1 -- 172.21.15.153:0/2017140723 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f45dc08e480 con 0x7f45ec1009c0 2024-03-21T10:43:32.583 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.586+0000 7f45ea000700 1 --2- 172.21.15.153:0/2017140723 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f45d006e460 0x7f45d0070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:43:32.583 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.586+0000 7f45ea000700 1 --2- 172.21.15.153:0/2017140723 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f45d006e460 0x7f45d0070920 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f45d8006010 tx=0x7f45d800b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:43:32.587 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.590+0000 7f45e3400700 1 -- 172.21.15.153:0/2017140723 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f45dc057520 con 0x7f45ec1009c0 2024-03-21T10:43:32.646 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:43:32 smithi153 ceph-mon[113733]: from='client.35686 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:43:32.646 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:43:32 smithi153 ceph-mon[113733]: from='client.35690 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:43:32.646 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:43:32 smithi153 ceph-mon[113733]: from='client.35694 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:43:32.646 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:43:32 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/4109078380' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:43:32.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:43:32 smithi174 ceph-mon[88004]: from='client.35686 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:43:32.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:43:32 smithi174 ceph-mon[88004]: from='client.35690 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:43:32.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:43:32 smithi174 ceph-mon[88004]: from='client.35694 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:43:32.698 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:43:32 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/4109078380' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:43:32.825 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.827+0000 7f45ebe00700 1 -- 172.21.15.153:0/2017140723 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f45ec19bad0 con 0x7f45d006e460 2024-03-21T10:43:32.827 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.829+0000 7f45e3400700 1 -- 172.21.15.153:0/2017140723 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f45ec19bad0 con 0x7f45d006e460 2024-03-21T10:43:32.827 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:43:32.827 INFO:teuthology.orchestra.run.smithi153.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232", 2024-03-21T10:43:32.827 INFO:teuthology.orchestra.run.smithi153.stdout: "in_progress": true, 2024-03-21T10:43:32.828 INFO:teuthology.orchestra.run.smithi153.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-21T10:43:32.828 INFO:teuthology.orchestra.run.smithi153.stdout: "services_complete": [ 2024-03-21T10:43:32.828 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr", 2024-03-21T10:43:32.828 INFO:teuthology.orchestra.run.smithi153.stdout: "mon", 2024-03-21T10:43:32.828 INFO:teuthology.orchestra.run.smithi153.stdout: "mds", 2024-03-21T10:43:32.828 INFO:teuthology.orchestra.run.smithi153.stdout: "crash", 2024-03-21T10:43:32.828 INFO:teuthology.orchestra.run.smithi153.stdout: "osd" 2024-03-21T10:43:32.828 INFO:teuthology.orchestra.run.smithi153.stdout: ], 2024-03-21T10:43:32.828 INFO:teuthology.orchestra.run.smithi153.stdout: "progress": "16/23 daemons upgraded", 2024-03-21T10:43:32.828 INFO:teuthology.orchestra.run.smithi153.stdout: "message": "Currently upgrading mds daemons", 2024-03-21T10:43:32.828 INFO:teuthology.orchestra.run.smithi153.stdout: "is_paused": false 2024-03-21T10:43:32.828 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:43:32.829 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.832+0000 7f45ebe00700 1 -- 172.21.15.153:0/2017140723 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f45d006e460 msgr2=0x7f45d0070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:43:32.829 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.832+0000 7f45ebe00700 1 --2- 172.21.15.153:0/2017140723 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f45d006e460 0x7f45d0070920 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f45d8006010 tx=0x7f45d800b540 comp rx=0 tx=0).stop 2024-03-21T10:43:32.829 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.832+0000 7f45ebe00700 1 -- 172.21.15.153:0/2017140723 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f45ec1009c0 msgr2=0x7f45ec1965a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:43:32.830 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.832+0000 7f45ebe00700 1 --2- 172.21.15.153:0/2017140723 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f45ec1009c0 0x7f45ec1965a0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f45dc00d8d0 tx=0x7f45dc00dc90 comp rx=0 tx=0).stop 2024-03-21T10:43:32.830 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.832+0000 7f45ebe00700 1 -- 172.21.15.153:0/2017140723 shutdown_connections 2024-03-21T10:43:32.830 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.832+0000 7f45ebe00700 1 --2- 172.21.15.153:0/2017140723 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f45d006e460 0x7f45d0070920 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:43:32.830 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.832+0000 7f45ebe00700 1 --2- 172.21.15.153:0/2017140723 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f45ec1009c0 0x7f45ec1965a0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:43:32.830 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.832+0000 7f45ebe00700 1 --2- 172.21.15.153:0/2017140723 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f45ec1069c0 0x7f45ec196ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:43:32.830 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.832+0000 7f45ebe00700 1 -- 172.21.15.153:0/2017140723 >> 172.21.15.153:0/2017140723 conn(0x7f45ec0fc4f0 msgr2=0x7f45ec0fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:43:32.830 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.832+0000 7f45ebe00700 1 -- 172.21.15.153:0/2017140723 shutdown_connections 2024-03-21T10:43:32.830 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.832+0000 7f45ebe00700 1 -- 172.21.15.153:0/2017140723 wait complete. 2024-03-21T10:43:32.985 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.987+0000 7fe14b600700 1 -- 172.21.15.153:0/2889928507 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe144073a80 msgr2=0x7fe14410cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:43:32.986 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.987+0000 7fe14b600700 1 --2- 172.21.15.153:0/2889928507 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe144073a80 0x7fe14410cea0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fe130009910 tx=0x7fe130009c20 comp rx=0 tx=0).stop 2024-03-21T10:43:32.986 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.987+0000 7fe14b600700 1 -- 172.21.15.153:0/2889928507 shutdown_connections 2024-03-21T10:43:32.986 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.987+0000 7fe14b600700 1 --2- 172.21.15.153:0/2889928507 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe144073a80 0x7fe14410cea0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:43:32.986 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.987+0000 7fe14b600700 1 --2- 172.21.15.153:0/2889928507 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe144073160 0x7fe144073540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:43:32.986 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.987+0000 7fe14b600700 1 -- 172.21.15.153:0/2889928507 >> 172.21.15.153:0/2889928507 conn(0x7fe1440fc630 msgr2=0x7fe1440fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:43:32.986 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.988+0000 7fe14b600700 1 -- 172.21.15.153:0/2889928507 shutdown_connections 2024-03-21T10:43:32.986 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.988+0000 7fe14b600700 1 -- 172.21.15.153:0/2889928507 wait complete. 2024-03-21T10:43:32.989 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.989+0000 7fe14b600700 1 Processor -- start 2024-03-21T10:43:32.989 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.989+0000 7fe14b600700 1 -- start start 2024-03-21T10:43:32.989 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.989+0000 7fe14b600700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe144073160 0x7fe14419e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:43:32.989 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.989+0000 7fe14b600700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe144073a80 0x7fe14419ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:43:32.989 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.989+0000 7fe14b600700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe14419f3f0 con 0x7fe144073a80 2024-03-21T10:43:32.990 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.989+0000 7fe14b600700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1441988a0 con 0x7fe144073160 2024-03-21T10:43:32.990 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.990+0000 7fe143e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe144073a80 0x7fe14419ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:43:32.990 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.990+0000 7fe143e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe144073a80 0x7fe14419ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:37844/0 (socket says 172.21.15.153:37844) 2024-03-21T10:43:32.990 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.990+0000 7fe143e00700 1 -- 172.21.15.153:0/566278729 learned_addr learned my addr 172.21.15.153:0/566278729 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:43:32.990 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.990+0000 7fe148c00700 1 --2- 172.21.15.153:0/566278729 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe144073160 0x7fe14419e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:43:32.990 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.990+0000 7fe143e00700 1 -- 172.21.15.153:0/566278729 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe144073160 msgr2=0x7fe14419e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:43:32.991 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.990+0000 7fe143e00700 1 --2- 172.21.15.153:0/566278729 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe144073160 0x7fe14419e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:43:32.991 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.990+0000 7fe143e00700 1 -- 172.21.15.153:0/566278729 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1300095f0 con 0x7fe144073a80 2024-03-21T10:43:32.991 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.991+0000 7fe143e00700 1 --2- 172.21.15.153:0/566278729 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe144073a80 0x7fe14419ed60 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fe130006010 tx=0x7fe13000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:43:32.991 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.991+0000 7fe141600700 1 -- 172.21.15.153:0/566278729 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe13001d070 con 0x7fe144073a80 2024-03-21T10:43:32.991 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.991+0000 7fe141600700 1 -- 172.21.15.153:0/566278729 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe130004fb0 con 0x7fe144073a80 2024-03-21T10:43:32.991 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.991+0000 7fe14b600700 1 -- 172.21.15.153:0/566278729 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe144198b20 con 0x7fe144073a80 2024-03-21T10:43:32.992 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.991+0000 7fe141600700 1 -- 172.21.15.153:0/566278729 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe13000f480 con 0x7fe144073a80 2024-03-21T10:43:32.992 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.991+0000 7fe14b600700 1 -- 172.21.15.153:0/566278729 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe144199010 con 0x7fe144073a80 2024-03-21T10:43:32.992 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.992+0000 7fe14b600700 1 -- 172.21.15.153:0/566278729 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe14410a5a0 con 0x7fe144073a80 2024-03-21T10:43:32.993 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.993+0000 7fe141600700 1 -- 172.21.15.153:0/566278729 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fe130022890 con 0x7fe144073a80 2024-03-21T10:43:32.993 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.994+0000 7fe141600700 1 --2- 172.21.15.153:0/566278729 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe12c06e460 0x7fe12c070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:43:32.993 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.994+0000 7fe141600700 1 -- 172.21.15.153:0/566278729 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fe13008fba0 con 0x7fe144073a80 2024-03-21T10:43:32.993 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.995+0000 7fe148c00700 1 --2- 172.21.15.153:0/566278729 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe12c06e460 0x7fe12c070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:43:32.993 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.995+0000 7fe148c00700 1 --2- 172.21.15.153:0/566278729 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe12c06e460 0x7fe12c070920 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fe138006fd0 tx=0x7fe138008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:43:32.996 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:32.999+0000 7fe141600700 1 -- 172.21.15.153:0/566278729 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe13005c840 con 0x7fe144073a80 2024-03-21T10:43:33.310 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:33.312+0000 7fe14b600700 1 -- 172.21.15.153:0/566278729 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fe144109f80 con 0x7fe144073a80 2024-03-21T10:43:33.310 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:33.313+0000 7fe141600700 1 -- 172.21.15.153:0/566278729 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4493 (secure 0 0 0) 0x7fe13005c3d0 con 0x7fe144073a80 2024-03-21T10:43:33.311 INFO:teuthology.orchestra.run.smithi153.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-03-21T10:43:33.311 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-03-21T10:43:33.311 INFO:teuthology.orchestra.run.smithi153.stdout: host smithi174 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi174/config 2024-03-21T10:43:33.312 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/podman: timeout after 895 seconds 2024-03-21T10:43:33.312 INFO:teuthology.orchestra.run.smithi153.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 -e NODE_NAME=smithi174 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/run/ceph:z -v /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpd910r3_i:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 inventory --format=json-pretty --filter-for-batch 2024-03-21T10:43:33.312 INFO:teuthology.orchestra.run.smithi153.stdout:Traceback (most recent call last): 2024-03-21T10:43:33.312 INFO:teuthology.orchestra.run.smithi153.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-03-21T10:43:33.312 INFO:teuthology.orchestra.run.smithi153.stdout: return _run_code(code, main_globals, None, 2024-03-21T10:43:33.312 INFO:teuthology.orchestra.run.smithi153.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-03-21T10:43:33.312 INFO:teuthology.orchestra.run.smithi153.stdout: exec(code, run_globals) 2024-03-21T10:43:33.312 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 5606, in 2024-03-21T10:43:33.312 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 5594, in main 2024-03-21T10:43:33.312 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 406, in _infer_config 2024-03-21T10:43:33.312 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 321, in _infer_fsid 2024-03-21T10:43:33.312 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 434, in _infer_image 2024-03-21T10:43:33.312 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 308, in _validate_fsid 2024-03-21T10:43:33.313 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 3357, in command_ceph_volume 2024-03-21T10:43:33.313 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-03-21T10:43:33.313 INFO:teuthology.orchestra.run.smithi153.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 -e NODE_NAME=smithi174 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/run/ceph:z -v /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpd910r3_i:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 inventory --format=json-pretty --filter-for-batch 2024-03-21T10:43:33.313 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-21T10:43:33.313 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is degraded 2024-03-21T10:43:33.313 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-21T10:43:33.313 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-21T10:43:33.313 INFO:teuthology.orchestra.run.smithi153.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-21T10:43:33.313 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-21T10:43:33.314 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:33.316+0000 7fe14b600700 1 -- 172.21.15.153:0/566278729 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe12c06e460 msgr2=0x7fe12c070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:43:33.314 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:33.316+0000 7fe14b600700 1 --2- 172.21.15.153:0/566278729 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe12c06e460 0x7fe12c070920 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fe138006fd0 tx=0x7fe138008040 comp rx=0 tx=0).stop 2024-03-21T10:43:33.314 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:33.316+0000 7fe14b600700 1 -- 172.21.15.153:0/566278729 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe144073a80 msgr2=0x7fe14419ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:43:33.314 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:33.316+0000 7fe14b600700 1 --2- 172.21.15.153:0/566278729 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe144073a80 0x7fe14419ed60 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fe130006010 tx=0x7fe13000be20 comp rx=0 tx=0).stop 2024-03-21T10:43:33.314 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:33.316+0000 7fe14b600700 1 -- 172.21.15.153:0/566278729 shutdown_connections 2024-03-21T10:43:33.314 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:33.316+0000 7fe14b600700 1 --2- 172.21.15.153:0/566278729 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe144073160 0x7fe14419e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:43:33.314 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:33.316+0000 7fe14b600700 1 --2- 172.21.15.153:0/566278729 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe12c06e460 0x7fe12c070920 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:43:33.314 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:33.316+0000 7fe14b600700 1 --2- 172.21.15.153:0/566278729 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe144073a80 0x7fe14419ed60 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:43:33.315 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:33.316+0000 7fe14b600700 1 -- 172.21.15.153:0/566278729 >> 172.21.15.153:0/566278729 conn(0x7fe1440fc630 msgr2=0x7fe1441076e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:43:33.315 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:33.316+0000 7fe14b600700 1 -- 172.21.15.153:0/566278729 shutdown_connections 2024-03-21T10:43:33.315 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:43:33.316+0000 7fe14b600700 1 -- 172.21.15.153:0/566278729 wait complete. 2024-03-21T10:43:33.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:43:33 smithi174 ceph-mon[88004]: pgmap v952: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:43:33.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:43:33 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/3722651247' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:43:33.698 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:43:33 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/566278729' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:43:33.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:43:33 smithi153 ceph-mon[113733]: pgmap v952: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:43:33.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:43:33 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/3722651247' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:43:33.803 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:43:33 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/566278729' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:43:34.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:43:34 smithi174 ceph-mon[88004]: from='client.35706 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:43:34.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:43:34 smithi153 ceph-mon[113733]: from='client.35706 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:43:35.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:43:35 smithi174 ceph-mon[88004]: pgmap v953: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:43:35.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:43:35 smithi153 ceph-mon[113733]: pgmap v953: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:43:37.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:43:37 smithi174 ceph-mon[88004]: pgmap v954: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:43:37.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:43:37 smithi153 ceph-mon[113733]: pgmap v954: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:43:39.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:43:39 smithi174 ceph-mon[88004]: pgmap v955: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:43:39.732 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:43:39 smithi153 ceph-mon[113733]: pgmap v955: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:43:41.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:43:41 smithi174 ceph-mon[88004]: pgmap v956: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:43:41.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:43:41 smithi153 ceph-mon[113733]: pgmap v956: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:43:43.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:43:43 smithi174 ceph-mon[88004]: pgmap v957: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:43:43.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:43:43 smithi153 ceph-mon[113733]: pgmap v957: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:43:45.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:43:45 smithi174 ceph-mon[88004]: pgmap v958: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:43:45.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:43:45 smithi153 ceph-mon[113733]: pgmap v958: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:43:46.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:43:46 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:43:46.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:43:46 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:43:47.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:43:47 smithi174 ceph-mon[88004]: pgmap v959: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:43:47.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:43:47 smithi153 ceph-mon[113733]: pgmap v959: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:43:49.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:43:49 smithi174 ceph-mon[88004]: pgmap v960: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:43:49.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:43:49 smithi153 ceph-mon[113733]: pgmap v960: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:43:51.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:43:51 smithi153 ceph-mon[113733]: pgmap v961: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:43:51.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:43:51 smithi174 ceph-mon[88004]: pgmap v961: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:43:53.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:43:53 smithi153 ceph-mon[113733]: pgmap v962: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:43:53.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:43:53 smithi174 ceph-mon[88004]: pgmap v962: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:43:55.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:43:55 smithi153 ceph-mon[113733]: pgmap v963: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:43:55.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:43:55 smithi174 ceph-mon[88004]: pgmap v963: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:43:57.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:43:57 smithi153 ceph-mon[113733]: pgmap v964: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:43:57.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:43:57 smithi174 ceph-mon[88004]: pgmap v964: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:43:59.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:43:59 smithi153 ceph-mon[113733]: pgmap v965: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:43:59.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:43:59 smithi174 ceph-mon[88004]: pgmap v965: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:44:01.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:44:01 smithi153 ceph-mon[113733]: pgmap v966: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:44:01.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:44:01 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:44:01.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:44:01 smithi174 ceph-mon[88004]: pgmap v966: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:44:01.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:44:01 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:44:03.482 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:03.485+0000 7f3274400700 1 -- 172.21.15.153:0/2999054483 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f326c0731c0 msgr2=0x7f326c0735a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:44:03.482 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:03.485+0000 7f3274400700 1 --2- 172.21.15.153:0/2999054483 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f326c0731c0 0x7f326c0735a0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f3260009910 tx=0x7f3260009c20 comp rx=0 tx=0).stop 2024-03-21T10:44:03.483 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:03.486+0000 7f3274400700 1 -- 172.21.15.153:0/2999054483 shutdown_connections 2024-03-21T10:44:03.483 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:03.486+0000 7f3274400700 1 --2- 172.21.15.153:0/2999054483 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f326c073ae0 0x7f326c10ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:44:03.483 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:03.486+0000 7f3274400700 1 --2- 172.21.15.153:0/2999054483 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f326c0731c0 0x7f326c0735a0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:44:03.483 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:03.486+0000 7f3274400700 1 -- 172.21.15.153:0/2999054483 >> 172.21.15.153:0/2999054483 conn(0x7f326c0fc650 msgr2=0x7f326c0fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:44:03.483 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:03.486+0000 7f3274400700 1 -- 172.21.15.153:0/2999054483 shutdown_connections 2024-03-21T10:44:03.483 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:03.486+0000 7f3274400700 1 -- 172.21.15.153:0/2999054483 wait complete. 2024-03-21T10:44:03.484 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:03.487+0000 7f3274400700 1 Processor -- start 2024-03-21T10:44:03.484 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:03.487+0000 7f3274400700 1 -- start start 2024-03-21T10:44:03.484 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:03.487+0000 7f3274400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f326c0731c0 0x7f326c19e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:44:03.484 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:03.487+0000 7f3274400700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f326c073ae0 0x7f326c19eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:44:03.484 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:03.487+0000 7f3274400700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f326c19f430 con 0x7f326c0731c0 2024-03-21T10:44:03.484 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:03.487+0000 7f3274400700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f326c1988e0 con 0x7f326c073ae0 2024-03-21T10:44:03.484 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:03.488+0000 7f3271a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f326c0731c0 0x7f326c19e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:44:03.484 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:03.488+0000 7f3271a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f326c0731c0 0x7f326c19e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:33444/0 (socket says 172.21.15.153:33444) 2024-03-21T10:44:03.485 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:03.488+0000 7f3271a00700 1 -- 172.21.15.153:0/1852915948 learned_addr learned my addr 172.21.15.153:0/1852915948 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:44:03.485 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:03.488+0000 7f3271a00700 1 -- 172.21.15.153:0/1852915948 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f326c073ae0 msgr2=0x7f326c19eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:44:03.486 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:03.488+0000 7f3271a00700 1 --2- 172.21.15.153:0/1852915948 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f326c073ae0 0x7f326c19eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:44:03.486 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:03.488+0000 7f3271a00700 1 -- 172.21.15.153:0/1852915948 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32600095f0 con 0x7f326c0731c0 2024-03-21T10:44:03.486 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:03.488+0000 7f3271a00700 1 --2- 172.21.15.153:0/1852915948 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f326c0731c0 0x7f326c19e860 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f32600092e0 tx=0x7f326000be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:44:03.486 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:03.489+0000 7f326a000700 1 -- 172.21.15.153:0/1852915948 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f326001d070 con 0x7f326c0731c0 2024-03-21T10:44:03.486 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:03.489+0000 7f326a000700 1 -- 172.21.15.153:0/1852915948 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f326000f460 con 0x7f326c0731c0 2024-03-21T10:44:03.486 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:03.489+0000 7f3274400700 1 -- 172.21.15.153:0/1852915948 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f326c198b60 con 0x7f326c0731c0 2024-03-21T10:44:03.486 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:03.489+0000 7f326a000700 1 -- 172.21.15.153:0/1852915948 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3260017630 con 0x7f326c0731c0 2024-03-21T10:44:03.486 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:03.489+0000 7f3274400700 1 -- 172.21.15.153:0/1852915948 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f326c199050 con 0x7f326c0731c0 2024-03-21T10:44:03.486 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:03.490+0000 7f3274400700 1 -- 172.21.15.153:0/1852915948 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f326c10a560 con 0x7f326c0731c0 2024-03-21T10:44:03.487 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:03.491+0000 7f326a000700 1 -- 172.21.15.153:0/1852915948 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f326000f5d0 con 0x7f326c0731c0 2024-03-21T10:44:03.489 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:03.492+0000 7f326a000700 1 --2- 172.21.15.153:0/1852915948 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f325406e460 0x7f3254070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:44:03.489 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:03.492+0000 7f326a000700 1 -- 172.21.15.153:0/1852915948 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f326008fb70 con 0x7f326c0731c0 2024-03-21T10:44:03.489 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:03.492+0000 7f3271000700 1 --2- 172.21.15.153:0/1852915948 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f325406e460 0x7f3254070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:44:03.489 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:03.493+0000 7f3271000700 1 --2- 172.21.15.153:0/1852915948 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f325406e460 0x7f3254070920 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f326c0fee20 tx=0x7f3258006d20 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:44:03.496 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:03.500+0000 7f326a000700 1 -- 172.21.15.153:0/1852915948 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f326005c6c0 con 0x7f326c0731c0 2024-03-21T10:44:03.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:44:03 smithi174 ceph-mon[88004]: pgmap v967: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:44:03.735 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:03.738+0000 7f3274400700 1 -- 172.21.15.153:0/1852915948 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f326c199330 con 0x7f325406e460 2024-03-21T10:44:03.735 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:44:03 smithi153 ceph-mon[113733]: pgmap v967: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:44:03.737 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:03.740+0000 7f326a000700 1 -- 172.21.15.153:0/1852915948 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f326c199330 con 0x7f325406e460 2024-03-21T10:44:03.740 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:03.743+0000 7f3274400700 1 -- 172.21.15.153:0/1852915948 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f325406e460 msgr2=0x7f3254070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:44:03.740 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:03.743+0000 7f3274400700 1 --2- 172.21.15.153:0/1852915948 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f325406e460 0x7f3254070920 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f326c0fee20 tx=0x7f3258006d20 comp rx=0 tx=0).stop 2024-03-21T10:44:03.740 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:03.743+0000 7f3274400700 1 -- 172.21.15.153:0/1852915948 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f326c0731c0 msgr2=0x7f326c19e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:44:03.740 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:03.743+0000 7f3274400700 1 --2- 172.21.15.153:0/1852915948 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f326c0731c0 0x7f326c19e860 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f32600092e0 tx=0x7f326000be00 comp rx=0 tx=0).stop 2024-03-21T10:44:03.740 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:03.743+0000 7f3274400700 1 -- 172.21.15.153:0/1852915948 shutdown_connections 2024-03-21T10:44:03.740 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:03.743+0000 7f3274400700 1 --2- 172.21.15.153:0/1852915948 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f325406e460 0x7f3254070920 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:44:03.740 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:03.743+0000 7f3274400700 1 --2- 172.21.15.153:0/1852915948 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f326c0731c0 0x7f326c19e860 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:44:03.741 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:03.743+0000 7f3274400700 1 --2- 172.21.15.153:0/1852915948 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f326c073ae0 0x7f326c19eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:44:03.741 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:03.743+0000 7f3274400700 1 -- 172.21.15.153:0/1852915948 >> 172.21.15.153:0/1852915948 conn(0x7f326c0fc650 msgr2=0x7f326c1076a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:44:03.741 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:03.744+0000 7f3274400700 1 -- 172.21.15.153:0/1852915948 shutdown_connections 2024-03-21T10:44:03.741 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:03.744+0000 7f3274400700 1 -- 172.21.15.153:0/1852915948 wait complete. 2024-03-21T10:44:03.756 INFO:teuthology.orchestra.run.smithi153.stdout:true 2024-03-21T10:44:03.896 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:03.899+0000 7f7cc0600700 1 -- 172.21.15.153:0/628613935 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7cb81071d0 msgr2=0x7f7cb8107650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:44:03.896 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:03.899+0000 7f7cc0600700 1 --2- 172.21.15.153:0/628613935 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7cb81071d0 0x7f7cb8107650 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f7ca4009910 tx=0x7f7ca4009c20 comp rx=0 tx=0).stop 2024-03-21T10:44:03.896 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:03.899+0000 7f7cc0600700 1 -- 172.21.15.153:0/628613935 shutdown_connections 2024-03-21T10:44:03.896 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:03.899+0000 7f7cc0600700 1 --2- 172.21.15.153:0/628613935 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7cb81071d0 0x7f7cb8107650 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:44:03.896 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:03.899+0000 7f7cc0600700 1 --2- 172.21.15.153:0/628613935 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7cb810d280 0x7f7cb810d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:44:03.896 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:03.899+0000 7f7cc0600700 1 -- 172.21.15.153:0/628613935 >> 172.21.15.153:0/628613935 conn(0x7f7cb80769e0 msgr2=0x7f7cb8078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:44:03.896 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:03.899+0000 7f7cc0600700 1 -- 172.21.15.153:0/628613935 shutdown_connections 2024-03-21T10:44:03.896 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:03.899+0000 7f7cc0600700 1 -- 172.21.15.153:0/628613935 wait complete. 2024-03-21T10:44:03.898 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:03.900+0000 7f7cc0600700 1 Processor -- start 2024-03-21T10:44:03.898 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:03.901+0000 7f7cc0600700 1 -- start start 2024-03-21T10:44:03.898 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:03.901+0000 7f7cc0600700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7cb81071d0 0x7f7cb819ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:44:03.898 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:03.901+0000 7f7cc0600700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7cb810d280 0x7f7cb819d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:44:03.898 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:03.901+0000 7f7cc0600700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb819d880 con 0x7f7cb810d280 2024-03-21T10:44:03.898 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:03.901+0000 7f7cc0600700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb81a05b0 con 0x7f7cb81071d0 2024-03-21T10:44:03.898 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:03.901+0000 7f7cbd200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7cb810d280 0x7f7cb819d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:44:03.898 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:03.901+0000 7f7cbd200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7cb810d280 0x7f7cb819d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:33466/0 (socket says 172.21.15.153:33466) 2024-03-21T10:44:03.898 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:03.901+0000 7f7cbd200700 1 -- 172.21.15.153:0/673210718 learned_addr learned my addr 172.21.15.153:0/673210718 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:44:03.898 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:03.902+0000 7f7cbdc00700 1 --2- 172.21.15.153:0/673210718 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7cb81071d0 0x7f7cb819ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:44:03.899 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:03.902+0000 7f7cbd200700 1 -- 172.21.15.153:0/673210718 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7cb81071d0 msgr2=0x7f7cb819ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:44:03.899 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:03.902+0000 7f7cbd200700 1 --2- 172.21.15.153:0/673210718 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7cb81071d0 0x7f7cb819ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:44:03.899 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:03.902+0000 7f7cbd200700 1 -- 172.21.15.153:0/673210718 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ca40095f0 con 0x7f7cb810d280 2024-03-21T10:44:03.899 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:03.902+0000 7f7cbd200700 1 --2- 172.21.15.153:0/673210718 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7cb810d280 0x7f7cb819d1f0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f7ca400b5c0 tx=0x7f7ca400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:44:03.899 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:03.902+0000 7f7cb6000700 1 -- 172.21.15.153:0/673210718 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7ca401d070 con 0x7f7cb810d280 2024-03-21T10:44:03.899 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:03.902+0000 7f7cb6000700 1 -- 172.21.15.153:0/673210718 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7ca4005e00 con 0x7f7cb810d280 2024-03-21T10:44:03.899 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:03.902+0000 7f7cc0600700 1 -- 172.21.15.153:0/673210718 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cb81a0830 con 0x7f7cb810d280 2024-03-21T10:44:03.900 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:03.903+0000 7f7cb6000700 1 -- 172.21.15.153:0/673210718 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7ca4017400 con 0x7f7cb810d280 2024-03-21T10:44:03.900 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:03.903+0000 7f7cc0600700 1 -- 172.21.15.153:0/673210718 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cb81a0da0 con 0x7f7cb810d280 2024-03-21T10:44:03.900 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:03.904+0000 7f7cc0600700 1 -- 172.21.15.153:0/673210718 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cb810f6c0 con 0x7f7cb810d280 2024-03-21T10:44:03.901 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:03.905+0000 7f7cb6000700 1 -- 172.21.15.153:0/673210718 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f7ca400f460 con 0x7f7cb810d280 2024-03-21T10:44:03.903 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:03.906+0000 7f7cb6000700 1 --2- 172.21.15.153:0/673210718 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f7ca006e460 0x7f7ca0070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:44:03.903 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:03.906+0000 7f7cb6000700 1 -- 172.21.15.153:0/673210718 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f7ca408fa70 con 0x7f7cb810d280 2024-03-21T10:44:03.903 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:03.907+0000 7f7cbdc00700 1 --2- 172.21.15.153:0/673210718 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f7ca006e460 0x7f7ca0070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:44:03.903 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:03.907+0000 7f7cbdc00700 1 --2- 172.21.15.153:0/673210718 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f7ca006e460 0x7f7ca0070920 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f7cac005950 tx=0x7f7cac0058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:44:03.908 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:03.911+0000 7f7cb6000700 1 -- 172.21.15.153:0/673210718 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7ca4058a90 con 0x7f7cb810d280 2024-03-21T10:44:04.143 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:04.146+0000 7f7cc0600700 1 -- 172.21.15.153:0/673210718 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7cb802cce0 con 0x7f7ca006e460 2024-03-21T10:44:04.145 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:04.149+0000 7f7cb6000700 1 -- 172.21.15.153:0/673210718 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f7cb802cce0 con 0x7f7ca006e460 2024-03-21T10:44:04.148 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:04.151+0000 7f7cc0600700 1 -- 172.21.15.153:0/673210718 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f7ca006e460 msgr2=0x7f7ca0070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:44:04.148 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:04.151+0000 7f7cc0600700 1 --2- 172.21.15.153:0/673210718 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f7ca006e460 0x7f7ca0070920 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f7cac005950 tx=0x7f7cac0058e0 comp rx=0 tx=0).stop 2024-03-21T10:44:04.148 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:04.151+0000 7f7cc0600700 1 -- 172.21.15.153:0/673210718 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7cb810d280 msgr2=0x7f7cb819d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:44:04.148 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:04.151+0000 7f7cc0600700 1 --2- 172.21.15.153:0/673210718 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7cb810d280 0x7f7cb819d1f0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f7ca400b5c0 tx=0x7f7ca400be20 comp rx=0 tx=0).stop 2024-03-21T10:44:04.148 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:04.152+0000 7f7cc0600700 1 -- 172.21.15.153:0/673210718 shutdown_connections 2024-03-21T10:44:04.149 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:04.152+0000 7f7cc0600700 1 --2- 172.21.15.153:0/673210718 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7cb81071d0 0x7f7cb819ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:44:04.149 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:04.152+0000 7f7cc0600700 1 --2- 172.21.15.153:0/673210718 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f7ca006e460 0x7f7ca0070920 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:44:04.149 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:04.152+0000 7f7cc0600700 1 --2- 172.21.15.153:0/673210718 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7cb810d280 0x7f7cb819d1f0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:44:04.149 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:04.152+0000 7f7cc0600700 1 -- 172.21.15.153:0/673210718 >> 172.21.15.153:0/673210718 conn(0x7f7cb80769e0 msgr2=0x7f7cb8078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:44:04.149 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:04.152+0000 7f7cc0600700 1 -- 172.21.15.153:0/673210718 shutdown_connections 2024-03-21T10:44:04.149 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:04.152+0000 7f7cc0600700 1 -- 172.21.15.153:0/673210718 wait complete. 2024-03-21T10:44:04.306 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:04.309+0000 7fa565200700 1 -- 172.21.15.153:0/2377789500 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa5601071d0 msgr2=0x7fa560107650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:44:04.306 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:04.309+0000 7fa565200700 1 --2- 172.21.15.153:0/2377789500 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa5601071d0 0x7fa560107650 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fa54c009910 tx=0x7fa54c009c20 comp rx=0 tx=0).stop 2024-03-21T10:44:04.306 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:04.309+0000 7fa565200700 1 -- 172.21.15.153:0/2377789500 shutdown_connections 2024-03-21T10:44:04.306 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:04.309+0000 7fa565200700 1 --2- 172.21.15.153:0/2377789500 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa5601071d0 0x7fa560107650 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:44:04.306 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:04.309+0000 7fa565200700 1 --2- 172.21.15.153:0/2377789500 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa56010d280 0x7fa56010d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:44:04.306 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:04.309+0000 7fa565200700 1 -- 172.21.15.153:0/2377789500 >> 172.21.15.153:0/2377789500 conn(0x7fa5600769e0 msgr2=0x7fa560078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:44:04.306 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:04.309+0000 7fa565200700 1 -- 172.21.15.153:0/2377789500 shutdown_connections 2024-03-21T10:44:04.306 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:04.309+0000 7fa565200700 1 -- 172.21.15.153:0/2377789500 wait complete. 2024-03-21T10:44:04.307 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:04.310+0000 7fa565200700 1 Processor -- start 2024-03-21T10:44:04.307 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:04.311+0000 7fa565200700 1 -- start start 2024-03-21T10:44:04.307 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:04.311+0000 7fa565200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa5601071d0 0x7fa56019cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:44:04.307 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:04.311+0000 7fa565200700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa56010d280 0x7fa56019d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:44:04.307 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:04.311+0000 7fa565200700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa56019d9b0 con 0x7fa5601071d0 2024-03-21T10:44:04.307 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:04.311+0000 7fa565200700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5601a1740 con 0x7fa56010d280 2024-03-21T10:44:04.308 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:04.311+0000 7fa55de00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa5601071d0 0x7fa56019cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:44:04.308 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:04.311+0000 7fa55de00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa5601071d0 0x7fa56019cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:33474/0 (socket says 172.21.15.153:33474) 2024-03-21T10:44:04.308 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:04.311+0000 7fa55de00700 1 -- 172.21.15.153:0/1462420154 learned_addr learned my addr 172.21.15.153:0/1462420154 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:44:04.308 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:04.312+0000 7fa55d400700 1 --2- 172.21.15.153:0/1462420154 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa56010d280 0x7fa56019d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:44:04.309 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:04.312+0000 7fa55de00700 1 -- 172.21.15.153:0/1462420154 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa56010d280 msgr2=0x7fa56019d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:44:04.309 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:04.312+0000 7fa55de00700 1 --2- 172.21.15.153:0/1462420154 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa56010d280 0x7fa56019d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:44:04.309 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:04.312+0000 7fa55de00700 1 -- 172.21.15.153:0/1462420154 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa54c0095f0 con 0x7fa5601071d0 2024-03-21T10:44:04.309 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:04.312+0000 7fa55de00700 1 --2- 172.21.15.153:0/1462420154 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa5601071d0 0x7fa56019cd90 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fa55000d8d0 tx=0x7fa55000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:44:04.309 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:04.312+0000 7fa556a00700 1 -- 172.21.15.153:0/1462420154 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa550009940 con 0x7fa5601071d0 2024-03-21T10:44:04.309 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:04.312+0000 7fa556a00700 1 -- 172.21.15.153:0/1462420154 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa550010460 con 0x7fa5601071d0 2024-03-21T10:44:04.309 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:04.312+0000 7fa565200700 1 -- 172.21.15.153:0/1462420154 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5601a1a20 con 0x7fa5601071d0 2024-03-21T10:44:04.310 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:04.312+0000 7fa556a00700 1 -- 172.21.15.153:0/1462420154 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa55000f5f0 con 0x7fa5601071d0 2024-03-21T10:44:04.310 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:04.313+0000 7fa565200700 1 -- 172.21.15.153:0/1462420154 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5601a1f70 con 0x7fa5601071d0 2024-03-21T10:44:04.310 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:04.314+0000 7fa565200700 1 -- 172.21.15.153:0/1462420154 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa56004ea90 con 0x7fa5601071d0 2024-03-21T10:44:04.311 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:04.315+0000 7fa556a00700 1 -- 172.21.15.153:0/1462420154 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fa5500105d0 con 0x7fa5601071d0 2024-03-21T10:44:04.312 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:04.316+0000 7fa556a00700 1 --2- 172.21.15.153:0/1462420154 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa54406e460 0x7fa544070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:44:04.312 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:04.316+0000 7fa556a00700 1 -- 172.21.15.153:0/1462420154 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fa55008e5b0 con 0x7fa5601071d0 2024-03-21T10:44:04.312 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:04.316+0000 7fa55d400700 1 --2- 172.21.15.153:0/1462420154 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa54406e460 0x7fa544070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:44:04.313 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:04.317+0000 7fa55d400700 1 --2- 172.21.15.153:0/1462420154 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa54406e460 0x7fa544070920 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fa54c00b5c0 tx=0x7fa54c005960 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:44:04.317 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:04.321+0000 7fa556a00700 1 -- 172.21.15.153:0/1462420154 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa5500575d0 con 0x7fa5601071d0 2024-03-21T10:44:04.551 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:04.554+0000 7fa565200700 1 -- 172.21.15.153:0/1462420154 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fa5601a2440 con 0x7fa54406e460 2024-03-21T10:44:04.589 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:04.593+0000 7fa556a00700 1 -- 172.21.15.153:0/1462420154 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fa5601a2440 con 0x7fa54406e460 2024-03-21T10:44:04.589 INFO:teuthology.orchestra.run.smithi153.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-21T10:44:04.590 INFO:teuthology.orchestra.run.smithi153.stdout:alertmanager.smithi153 smithi153 *:9093,9094 running (32m) 8m ago 46m 15.6M - 0.25.0 c8568f914cd2 9ed3f2200b5f 2024-03-21T10:44:04.590 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi153 smithi153 running (46m) 8m ago 46m 10.4M - 18.2.1 5be31c24972a 5b61994396a6 2024-03-21T10:44:04.590 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi174 smithi174 running (45m) 8m ago 45m 10.1M - 18.2.1 5be31c24972a 4f7a5fc3ac28 2024-03-21T10:44:04.590 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi153 smithi153 running (29m) 8m ago 46m 6656k - 19.0.0-2231-g2edfee68 237e43c7a65f 32c21104d5b0 2024-03-21T10:44:04.590 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi174 smithi174 running (28m) 8m ago 45m 6660k - 19.0.0-2231-g2edfee68 237e43c7a65f a04e48967ab6 2024-03-21T10:44:04.590 INFO:teuthology.orchestra.run.smithi153.stdout:grafana.smithi153 smithi153 *:3000 running (32m) 8m ago 46m 75.2M - 9.4.12 2bacad6d85d8 ee24026ab786 2024-03-21T10:44:04.590 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.iuyluj smithi153 running (24m) 8m ago 39m 18.7M - 19.0.0-2231-g2edfee68 237e43c7a65f 87aff8a5de70 2024-03-21T10:44:04.590 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.zipckr smithi153 running (24m) 8m ago 39m 18.9M - 19.0.0-2231-g2edfee68 237e43c7a65f ccc262bd5dd3 2024-03-21T10:44:04.590 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.nhozag smithi174 running (8m) 8m ago 39m 10.9M - 19.0.0-2231-g2edfee68 237e43c7a65f 9e02aea72c96 2024-03-21T10:44:04.590 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.yyimst smithi174 running (24m) 8m ago 39m 18.5M - 19.0.0-2231-g2edfee68 237e43c7a65f fc1962f9351e 2024-03-21T10:44:04.590 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi153.tluxni smithi153 *:8443,9283,8765 running (35m) 8m ago 48m 512M - 19.0.0-2231-g2edfee68 237e43c7a65f 7ffee0e55382 2024-03-21T10:44:04.590 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi174.qpcwat smithi174 *:8443,9283,8765 running (34m) 8m ago 45m 459M - 19.0.0-2231-g2edfee68 237e43c7a65f 748968f216ab 2024-03-21T10:44:04.591 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi153 smithi153 running (30m) 8m ago 48m 70.8M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f dd8038303dc3 2024-03-21T10:44:04.591 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi174 smithi174 running (30m) 8m ago 45m 51.9M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f 29d78a788f2c 2024-03-21T10:44:04.591 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi153 smithi153 *:9100 running (34m) 8m ago 46m 8140k - 1.5.0 0da6a335fe13 bb835edc0e0b 2024-03-21T10:44:04.591 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi174 smithi174 *:9100 running (33m) 8m ago 45m 8610k - 1.5.0 0da6a335fe13 7c4ecad790fa 2024-03-21T10:44:04.591 INFO:teuthology.orchestra.run.smithi153.stdout:osd.0 smithi153 running (28m) 8m ago 44m 118M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 507900fd4791 2024-03-21T10:44:04.591 INFO:teuthology.orchestra.run.smithi153.stdout:osd.1 smithi153 running (27m) 8m ago 43m 93.0M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 0d646452e9c1 2024-03-21T10:44:04.591 INFO:teuthology.orchestra.run.smithi153.stdout:osd.2 smithi153 running (27m) 8m ago 42m 91.2M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 1326f2afdfcd 2024-03-21T10:44:04.591 INFO:teuthology.orchestra.run.smithi153.stdout:osd.3 smithi174 running (26m) 8m ago 42m 87.5M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 70aafac71ad5 2024-03-21T10:44:04.591 INFO:teuthology.orchestra.run.smithi153.stdout:osd.4 smithi174 running (25m) 8m ago 41m 95.8M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 0d746ce6e79d 2024-03-21T10:44:04.591 INFO:teuthology.orchestra.run.smithi153.stdout:osd.5 smithi174 running (25m) 8m ago 41m 90.3M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 53e5f35b8cac 2024-03-21T10:44:04.591 INFO:teuthology.orchestra.run.smithi153.stdout:prometheus.smithi153 smithi153 *:9095 running (33m) 8m ago 45m 88.1M - 2.43.0 a07b618ecd1d 7d95561e50c2 2024-03-21T10:44:04.593 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:04.597+0000 7fa565200700 1 -- 172.21.15.153:0/1462420154 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa54406e460 msgr2=0x7fa544070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:44:04.594 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:04.597+0000 7fa565200700 1 --2- 172.21.15.153:0/1462420154 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa54406e460 0x7fa544070920 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fa54c00b5c0 tx=0x7fa54c005960 comp rx=0 tx=0).stop 2024-03-21T10:44:04.594 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:04.597+0000 7fa565200700 1 -- 172.21.15.153:0/1462420154 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa5601071d0 msgr2=0x7fa56019cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:44:04.594 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:04.597+0000 7fa565200700 1 --2- 172.21.15.153:0/1462420154 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa5601071d0 0x7fa56019cd90 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fa55000d8d0 tx=0x7fa55000dc90 comp rx=0 tx=0).stop 2024-03-21T10:44:04.594 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:04.597+0000 7fa565200700 1 -- 172.21.15.153:0/1462420154 shutdown_connections 2024-03-21T10:44:04.594 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:04.597+0000 7fa565200700 1 --2- 172.21.15.153:0/1462420154 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa54406e460 0x7fa544070920 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:44:04.594 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:04.597+0000 7fa565200700 1 --2- 172.21.15.153:0/1462420154 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa5601071d0 0x7fa56019cd90 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:44:04.594 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:04.597+0000 7fa565200700 1 --2- 172.21.15.153:0/1462420154 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa56010d280 0x7fa56019d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:44:04.594 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:04.597+0000 7fa565200700 1 -- 172.21.15.153:0/1462420154 >> 172.21.15.153:0/1462420154 conn(0x7fa5600769e0 msgr2=0x7fa560078480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:44:04.594 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:04.597+0000 7fa565200700 1 -- 172.21.15.153:0/1462420154 shutdown_connections 2024-03-21T10:44:04.594 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:04.597+0000 7fa565200700 1 -- 172.21.15.153:0/1462420154 wait complete. 2024-03-21T10:44:04.748 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:04.751+0000 7fb7c8600700 1 -- 172.21.15.153:0/3711749745 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb7c0101d60 msgr2=0x7fb7c0102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:44:04.748 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:04.751+0000 7fb7c8600700 1 --2- 172.21.15.153:0/3711749745 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb7c0101d60 0x7fb7c0102140 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fb7b4009940 tx=0x7fb7b4009c50 comp rx=0 tx=0).stop 2024-03-21T10:44:04.748 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:04.751+0000 7fb7c8600700 1 -- 172.21.15.153:0/3711749745 shutdown_connections 2024-03-21T10:44:04.748 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:04.751+0000 7fb7c8600700 1 --2- 172.21.15.153:0/3711749745 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb7c0102680 0x7fb7c010ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:44:04.748 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:04.751+0000 7fb7c8600700 1 --2- 172.21.15.153:0/3711749745 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb7c0101d60 0x7fb7c0102140 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:44:04.748 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:04.751+0000 7fb7c8600700 1 -- 172.21.15.153:0/3711749745 >> 172.21.15.153:0/3711749745 conn(0x7fb7c00fb570 msgr2=0x7fb7c00fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:44:04.749 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:04.752+0000 7fb7c8600700 1 -- 172.21.15.153:0/3711749745 shutdown_connections 2024-03-21T10:44:04.749 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:04.752+0000 7fb7c8600700 1 -- 172.21.15.153:0/3711749745 wait complete. 2024-03-21T10:44:04.749 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:04.753+0000 7fb7c8600700 1 Processor -- start 2024-03-21T10:44:04.749 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:04.753+0000 7fb7c8600700 1 -- start start 2024-03-21T10:44:04.750 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:04.753+0000 7fb7c8600700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb7c0101d60 0x7fb7c00ff9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:44:04.750 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:04.753+0000 7fb7c8600700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb7c0102680 0x7fb7c00fff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:44:04.750 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:04.753+0000 7fb7c8600700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c0100440 con 0x7fb7c0102680 2024-03-21T10:44:04.750 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:04.753+0000 7fb7c8600700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c0100580 con 0x7fb7c0101d60 2024-03-21T10:44:04.750 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:04.754+0000 7fb7c5200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb7c0102680 0x7fb7c00fff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:44:04.750 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:04.754+0000 7fb7c5200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb7c0102680 0x7fb7c00fff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:33504/0 (socket says 172.21.15.153:33504) 2024-03-21T10:44:04.750 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:04.754+0000 7fb7c5200700 1 -- 172.21.15.153:0/3626713700 learned_addr learned my addr 172.21.15.153:0/3626713700 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:44:04.751 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:04.754+0000 7fb7c5200700 1 -- 172.21.15.153:0/3626713700 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb7c0101d60 msgr2=0x7fb7c00ff9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:44:04.751 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:04.754+0000 7fb7c5200700 1 --2- 172.21.15.153:0/3626713700 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb7c0101d60 0x7fb7c00ff9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:44:04.751 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:04.754+0000 7fb7c5200700 1 -- 172.21.15.153:0/3626713700 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7b40095f0 con 0x7fb7c0102680 2024-03-21T10:44:04.751 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:04.754+0000 7fb7c5200700 1 --2- 172.21.15.153:0/3626713700 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb7c0102680 0x7fb7c00fff00 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac00d900 tx=0x7fb7ac00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:44:04.751 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:04.754+0000 7fb7be000700 1 -- 172.21.15.153:0/3626713700 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb7ac009900 con 0x7fb7c0102680 2024-03-21T10:44:04.751 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:04.755+0000 7fb7be000700 1 -- 172.21.15.153:0/3626713700 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb7ac010460 con 0x7fb7c0102680 2024-03-21T10:44:04.752 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:04.755+0000 7fb7c8600700 1 -- 172.21.15.153:0/3626713700 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7c01a2930 con 0x7fb7c0102680 2024-03-21T10:44:04.752 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:04.755+0000 7fb7be000700 1 -- 172.21.15.153:0/3626713700 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb7ac00f5f0 con 0x7fb7c0102680 2024-03-21T10:44:04.752 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:04.755+0000 7fb7c8600700 1 -- 172.21.15.153:0/3626713700 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7c01a2d40 con 0x7fb7c0102680 2024-03-21T10:44:04.752 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:04.756+0000 7fb7c8600700 1 -- 172.21.15.153:0/3626713700 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7c0108270 con 0x7fb7c0102680 2024-03-21T10:44:04.753 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:04.757+0000 7fb7be000700 1 -- 172.21.15.153:0/3626713700 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fb7ac0105d0 con 0x7fb7c0102680 2024-03-21T10:44:04.754 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:04.758+0000 7fb7be000700 1 --2- 172.21.15.153:0/3626713700 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb7a806e460 0x7fb7a8070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:44:04.754 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:04.758+0000 7fb7be000700 1 -- 172.21.15.153:0/3626713700 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fb7ac08e3d0 con 0x7fb7c0102680 2024-03-21T10:44:04.754 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:04.758+0000 7fb7c5c00700 1 --2- 172.21.15.153:0/3626713700 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb7a806e460 0x7fb7a8070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:44:04.755 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:04.759+0000 7fb7c5c00700 1 --2- 172.21.15.153:0/3626713700 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb7a806e460 0x7fb7a8070920 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fb7b4006010 tx=0x7fb7b4000bc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:44:04.759 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:04.762+0000 7fb7be000700 1 -- 172.21.15.153:0/3626713700 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb7ac0573f0 con 0x7fb7c0102680 2024-03-21T10:44:05.072 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:05.075+0000 7fb7c8600700 1 -- 172.21.15.153:0/3626713700 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fb7c002cc70 con 0x7fb7c0102680 2024-03-21T10:44:05.072 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:05.076+0000 7fb7be000700 1 -- 172.21.15.153:0/3626713700 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fb7ac05aab0 con 0x7fb7c0102680 2024-03-21T10:44:05.073 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:44:05.073 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": { 2024-03-21T10:44:05.073 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:44:05.074 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:44:05.074 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": { 2024-03-21T10:44:05.074 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:44:05.074 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:44:05.074 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": { 2024-03-21T10:44:05.074 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 6 2024-03-21T10:44:05.074 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:44:05.074 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": { 2024-03-21T10:44:05.074 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 4 2024-03-21T10:44:05.074 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:44:05.074 INFO:teuthology.orchestra.run.smithi153.stdout: "overall": { 2024-03-21T10:44:05.074 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 14 2024-03-21T10:44:05.074 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-03-21T10:44:05.075 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:44:05.075 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:05.079+0000 7fb7c8600700 1 -- 172.21.15.153:0/3626713700 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb7a806e460 msgr2=0x7fb7a8070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:44:05.075 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:05.079+0000 7fb7c8600700 1 --2- 172.21.15.153:0/3626713700 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb7a806e460 0x7fb7a8070920 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fb7b4006010 tx=0x7fb7b4000bc0 comp rx=0 tx=0).stop 2024-03-21T10:44:05.075 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:05.079+0000 7fb7c8600700 1 -- 172.21.15.153:0/3626713700 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb7c0102680 msgr2=0x7fb7c00fff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:44:05.075 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:05.079+0000 7fb7c8600700 1 --2- 172.21.15.153:0/3626713700 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb7c0102680 0x7fb7c00fff00 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac00d900 tx=0x7fb7ac00dcc0 comp rx=0 tx=0).stop 2024-03-21T10:44:05.076 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:05.079+0000 7fb7c8600700 1 -- 172.21.15.153:0/3626713700 shutdown_connections 2024-03-21T10:44:05.076 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:05.079+0000 7fb7c8600700 1 --2- 172.21.15.153:0/3626713700 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb7c0101d60 0x7fb7c00ff9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:44:05.076 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:05.079+0000 7fb7c8600700 1 --2- 172.21.15.153:0/3626713700 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb7a806e460 0x7fb7a8070920 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:44:05.076 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:05.079+0000 7fb7c8600700 1 --2- 172.21.15.153:0/3626713700 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb7c0102680 0x7fb7c00fff00 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:44:05.076 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:05.079+0000 7fb7c8600700 1 -- 172.21.15.153:0/3626713700 >> 172.21.15.153:0/3626713700 conn(0x7fb7c00fb570 msgr2=0x7fb7c01053b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:44:05.076 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:05.079+0000 7fb7c8600700 1 -- 172.21.15.153:0/3626713700 shutdown_connections 2024-03-21T10:44:05.076 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:05.079+0000 7fb7c8600700 1 -- 172.21.15.153:0/3626713700 wait complete. 2024-03-21T10:44:05.229 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:05.233+0000 7fdb6d200700 1 -- 172.21.15.153:0/4151698934 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdb68105b50 msgr2=0x7fdb68105f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:44:05.230 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:05.233+0000 7fdb6d200700 1 --2- 172.21.15.153:0/4151698934 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdb68105b50 0x7fdb68105f30 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fdb58009910 tx=0x7fdb58009c20 comp rx=0 tx=0).stop 2024-03-21T10:44:05.230 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:05.233+0000 7fdb6d200700 1 -- 172.21.15.153:0/4151698934 shutdown_connections 2024-03-21T10:44:05.230 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:05.233+0000 7fdb6d200700 1 --2- 172.21.15.153:0/4151698934 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb680fef70 0x7fdb680ff3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:44:05.230 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:05.233+0000 7fdb6d200700 1 --2- 172.21.15.153:0/4151698934 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdb68105b50 0x7fdb68105f30 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:44:05.230 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:05.233+0000 7fdb6d200700 1 -- 172.21.15.153:0/4151698934 >> 172.21.15.153:0/4151698934 conn(0x7fdb68075e30 msgr2=0x7fdb680fd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:44:05.230 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:05.233+0000 7fdb6d200700 1 -- 172.21.15.153:0/4151698934 shutdown_connections 2024-03-21T10:44:05.230 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:05.233+0000 7fdb6d200700 1 -- 172.21.15.153:0/4151698934 wait complete. 2024-03-21T10:44:05.231 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:05.234+0000 7fdb6d200700 1 Processor -- start 2024-03-21T10:44:05.231 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:05.234+0000 7fdb6d200700 1 -- start start 2024-03-21T10:44:05.231 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:05.234+0000 7fdb6d200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdb680fef70 0x7fdb68100150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:44:05.231 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:05.234+0000 7fdb6d200700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb68105b50 0x7fdb68100690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:44:05.231 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:05.234+0000 7fdb6d200700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb68100bd0 con 0x7fdb680fef70 2024-03-21T10:44:05.232 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:05.235+0000 7fdb6d200700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb68100d10 con 0x7fdb68105b50 2024-03-21T10:44:05.232 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:05.235+0000 7fdb65e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdb680fef70 0x7fdb68100150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:44:05.232 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:05.235+0000 7fdb65e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdb680fef70 0x7fdb68100150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:33524/0 (socket says 172.21.15.153:33524) 2024-03-21T10:44:05.232 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:05.235+0000 7fdb65e00700 1 -- 172.21.15.153:0/746822144 learned_addr learned my addr 172.21.15.153:0/746822144 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:44:05.232 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:05.235+0000 7fdb65e00700 1 -- 172.21.15.153:0/746822144 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb68105b50 msgr2=0x7fdb68100690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:44:05.232 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:05.235+0000 7fdb5d400700 1 --2- 172.21.15.153:0/746822144 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb68105b50 0x7fdb68100690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:44:05.232 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:05.235+0000 7fdb65e00700 1 --2- 172.21.15.153:0/746822144 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb68105b50 0x7fdb68100690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:44:05.232 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:05.235+0000 7fdb65e00700 1 -- 172.21.15.153:0/746822144 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb580095f0 con 0x7fdb680fef70 2024-03-21T10:44:05.233 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:05.235+0000 7fdb65e00700 1 --2- 172.21.15.153:0/746822144 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdb680fef70 0x7fdb68100150 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fdb580092e0 tx=0x7fdb5800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:44:05.233 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:05.236+0000 7fdb5f400700 1 -- 172.21.15.153:0/746822144 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdb5801d070 con 0x7fdb680fef70 2024-03-21T10:44:05.233 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:05.236+0000 7fdb5f400700 1 -- 172.21.15.153:0/746822144 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdb58005e00 con 0x7fdb680fef70 2024-03-21T10:44:05.233 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:05.236+0000 7fdb5f400700 1 -- 172.21.15.153:0/746822144 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdb58017400 con 0x7fdb680fef70 2024-03-21T10:44:05.233 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:05.236+0000 7fdb6d200700 1 -- 172.21.15.153:0/746822144 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb68100f90 con 0x7fdb680fef70 2024-03-21T10:44:05.233 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:05.236+0000 7fdb6d200700 1 -- 172.21.15.153:0/746822144 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb681a2ca0 con 0x7fdb680fef70 2024-03-21T10:44:05.233 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:05.237+0000 7fdb6d200700 1 -- 172.21.15.153:0/746822144 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb68109120 con 0x7fdb680fef70 2024-03-21T10:44:05.234 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:05.238+0000 7fdb5f400700 1 -- 172.21.15.153:0/746822144 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fdb5800f460 con 0x7fdb680fef70 2024-03-21T10:44:05.235 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:05.239+0000 7fdb5f400700 1 --2- 172.21.15.153:0/746822144 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fdb4c06e460 0x7fdb4c070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:44:05.235 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:05.239+0000 7fdb5f400700 1 -- 172.21.15.153:0/746822144 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fdb5808f9f0 con 0x7fdb680fef70 2024-03-21T10:44:05.235 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:05.239+0000 7fdb5d400700 1 --2- 172.21.15.153:0/746822144 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fdb4c06e460 0x7fdb4c070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:44:05.236 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:05.240+0000 7fdb5d400700 1 --2- 172.21.15.153:0/746822144 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fdb4c06e460 0x7fdb4c070920 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fdb50005950 tx=0x7fdb500058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:44:05.240 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:05.244+0000 7fdb5f400700 1 -- 172.21.15.153:0/746822144 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fdb58058a90 con 0x7fdb680fef70 2024-03-21T10:44:05.502 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:05.505+0000 7fdb6d200700 1 -- 172.21.15.153:0/746822144 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fdb68108b00 con 0x7fdb680fef70 2024-03-21T10:44:05.503 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:05.507+0000 7fdb5f400700 1 -- 172.21.15.153:0/746822144 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1876 (secure 0 0 0) 0x7fdb5805c150 con 0x7fdb680fef70 2024-03-21T10:44:05.504 INFO:teuthology.orchestra.run.smithi153.stdout:e32 2024-03-21T10:44:05.504 INFO:teuthology.orchestra.run.smithi153.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-21T10:44:05.504 INFO:teuthology.orchestra.run.smithi153.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:44:05.504 INFO:teuthology.orchestra.run.smithi153.stdout:legacy client fscid: 1 2024-03-21T10:44:05.504 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:44:05.504 INFO:teuthology.orchestra.run.smithi153.stdout:Filesystem 'cephfs' (1) 2024-03-21T10:44:05.504 INFO:teuthology.orchestra.run.smithi153.stdout:fs_name cephfs 2024-03-21T10:44:05.504 INFO:teuthology.orchestra.run.smithi153.stdout:epoch 23 2024-03-21T10:44:05.504 INFO:teuthology.orchestra.run.smithi153.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-21T10:44:05.504 INFO:teuthology.orchestra.run.smithi153.stdout:created 2024-03-21T10:04:27.086576+0000 2024-03-21T10:44:05.505 INFO:teuthology.orchestra.run.smithi153.stdout:modified 2024-03-21T10:19:05.022092+0000 2024-03-21T10:44:05.505 INFO:teuthology.orchestra.run.smithi153.stdout:tableserver 0 2024-03-21T10:44:05.505 INFO:teuthology.orchestra.run.smithi153.stdout:root 0 2024-03-21T10:44:05.505 INFO:teuthology.orchestra.run.smithi153.stdout:session_timeout 60 2024-03-21T10:44:05.505 INFO:teuthology.orchestra.run.smithi153.stdout:session_autoclose 300 2024-03-21T10:44:05.505 INFO:teuthology.orchestra.run.smithi153.stdout:max_file_size 1099511627776 2024-03-21T10:44:05.505 INFO:teuthology.orchestra.run.smithi153.stdout:max_xattr_size 65536 2024-03-21T10:44:05.505 INFO:teuthology.orchestra.run.smithi153.stdout:required_client_features {} 2024-03-21T10:44:05.505 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure 0 2024-03-21T10:44:05.505 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure_osd_epoch 94 2024-03-21T10:44:05.505 INFO:teuthology.orchestra.run.smithi153.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:44:05.505 INFO:teuthology.orchestra.run.smithi153.stdout:max_mds 2 2024-03-21T10:44:05.505 INFO:teuthology.orchestra.run.smithi153.stdout:in 0,1 2024-03-21T10:44:05.505 INFO:teuthology.orchestra.run.smithi153.stdout:up {} 2024-03-21T10:44:05.506 INFO:teuthology.orchestra.run.smithi153.stdout:failed 0,1 2024-03-21T10:44:05.506 INFO:teuthology.orchestra.run.smithi153.stdout:damaged 2024-03-21T10:44:05.506 INFO:teuthology.orchestra.run.smithi153.stdout:stopped 2024-03-21T10:44:05.506 INFO:teuthology.orchestra.run.smithi153.stdout:data_pools [3] 2024-03-21T10:44:05.506 INFO:teuthology.orchestra.run.smithi153.stdout:metadata_pool 2 2024-03-21T10:44:05.506 INFO:teuthology.orchestra.run.smithi153.stdout:inline_data enabled 2024-03-21T10:44:05.506 INFO:teuthology.orchestra.run.smithi153.stdout:balancer 2024-03-21T10:44:05.506 INFO:teuthology.orchestra.run.smithi153.stdout:bal_rank_mask -1 2024-03-21T10:44:05.506 INFO:teuthology.orchestra.run.smithi153.stdout:standby_count_wanted 1 2024-03-21T10:44:05.506 INFO:teuthology.orchestra.run.smithi153.stdout:qdb_cluster leader: 0 members: 2024-03-21T10:44:05.506 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:44:05.506 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:44:05.506 INFO:teuthology.orchestra.run.smithi153.stdout:Standby daemons: 2024-03-21T10:44:05.506 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:44:05.507 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.zipckr{-1:34442} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6826/1103846295,v1:172.21.15.153:6827/1103846295] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:44:05.507 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.iuyluj{-1:34474} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6828/1738630832,v1:172.21.15.153:6829/1738630832] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:44:05.507 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.yyimst{-1:44313} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.174:6824/4127786797,v1:172.21.15.174:6825/4127786797] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:44:05.507 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.nhozag{-1:44687} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.174:6826/2767517845,v1:172.21.15.174:6827/2767517845] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:44:05.507 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:05.511+0000 7fdb6d200700 1 -- 172.21.15.153:0/746822144 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fdb4c06e460 msgr2=0x7fdb4c070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:44:05.507 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:05.511+0000 7fdb6d200700 1 --2- 172.21.15.153:0/746822144 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fdb4c06e460 0x7fdb4c070920 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fdb50005950 tx=0x7fdb500058e0 comp rx=0 tx=0).stop 2024-03-21T10:44:05.507 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:05.511+0000 7fdb6d200700 1 -- 172.21.15.153:0/746822144 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdb680fef70 msgr2=0x7fdb68100150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:44:05.507 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:05.511+0000 7fdb6d200700 1 --2- 172.21.15.153:0/746822144 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdb680fef70 0x7fdb68100150 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fdb580092e0 tx=0x7fdb5800be20 comp rx=0 tx=0).stop 2024-03-21T10:44:05.508 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:05.511+0000 7fdb6d200700 1 -- 172.21.15.153:0/746822144 shutdown_connections 2024-03-21T10:44:05.508 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:05.511+0000 7fdb6d200700 1 --2- 172.21.15.153:0/746822144 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fdb4c06e460 0x7fdb4c070920 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:44:05.508 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:05.511+0000 7fdb6d200700 1 --2- 172.21.15.153:0/746822144 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdb680fef70 0x7fdb68100150 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:44:05.508 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:05.511+0000 7fdb6d200700 1 --2- 172.21.15.153:0/746822144 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb68105b50 0x7fdb68100690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:44:05.508 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:05.511+0000 7fdb6d200700 1 -- 172.21.15.153:0/746822144 >> 172.21.15.153:0/746822144 conn(0x7fdb68075e30 msgr2=0x7fdb68104520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:44:05.508 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:05.511+0000 7fdb6d200700 1 -- 172.21.15.153:0/746822144 shutdown_connections 2024-03-21T10:44:05.508 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:05.511+0000 7fdb6d200700 1 -- 172.21.15.153:0/746822144 wait complete. 2024-03-21T10:44:05.509 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 32 2024-03-21T10:44:05.675 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:05.678+0000 7f3287200700 1 -- 172.21.15.153:0/2097866801 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3280073a50 msgr2=0x7f3280111610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:44:05.675 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:05.678+0000 7f3287200700 1 --2- 172.21.15.153:0/2097866801 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3280073a50 0x7f3280111610 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f326c009910 tx=0x7f326c009c20 comp rx=0 tx=0).stop 2024-03-21T10:44:05.675 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:05.678+0000 7f3287200700 1 -- 172.21.15.153:0/2097866801 shutdown_connections 2024-03-21T10:44:05.675 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:05.678+0000 7f3287200700 1 --2- 172.21.15.153:0/2097866801 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3280073a50 0x7f3280111610 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:44:05.675 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:05.678+0000 7f3287200700 1 --2- 172.21.15.153:0/2097866801 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3280073130 0x7f3280073510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:44:05.675 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:05.678+0000 7f3287200700 1 -- 172.21.15.153:0/2097866801 >> 172.21.15.153:0/2097866801 conn(0x7f32800fc630 msgr2=0x7f32800fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:44:05.675 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:05.678+0000 7f3287200700 1 -- 172.21.15.153:0/2097866801 shutdown_connections 2024-03-21T10:44:05.675 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:05.678+0000 7f3287200700 1 -- 172.21.15.153:0/2097866801 wait complete. 2024-03-21T10:44:05.676 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:05.679+0000 7f3287200700 1 Processor -- start 2024-03-21T10:44:05.676 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:05.679+0000 7f3287200700 1 -- start start 2024-03-21T10:44:05.677 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:05.680+0000 7f3287200700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3280073130 0x7f328019cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:44:05.677 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:05.680+0000 7f3287200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3280073a50 0x7f328019d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:44:05.677 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:05.680+0000 7f3287200700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f328019d930 con 0x7f3280073a50 2024-03-21T10:44:05.677 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:05.680+0000 7f3287200700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32801a16c0 con 0x7f3280073130 2024-03-21T10:44:05.677 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:05.680+0000 7f327f400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3280073a50 0x7f328019d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:44:05.677 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:05.680+0000 7f327f400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3280073a50 0x7f328019d250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:33538/0 (socket says 172.21.15.153:33538) 2024-03-21T10:44:05.677 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:05.680+0000 7f327f400700 1 -- 172.21.15.153:0/3795619113 learned_addr learned my addr 172.21.15.153:0/3795619113 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:44:05.677 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:05.680+0000 7f327fe00700 1 --2- 172.21.15.153:0/3795619113 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3280073130 0x7f328019cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:44:05.677 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:05.681+0000 7f327f400700 1 -- 172.21.15.153:0/3795619113 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3280073130 msgr2=0x7f328019cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:44:05.677 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:05.681+0000 7f327f400700 1 --2- 172.21.15.153:0/3795619113 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3280073130 0x7f328019cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:44:05.678 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:05.681+0000 7f327f400700 1 -- 172.21.15.153:0/3795619113 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f326c0095f0 con 0x7f3280073a50 2024-03-21T10:44:05.678 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:05.681+0000 7f327f400700 1 --2- 172.21.15.153:0/3795619113 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3280073a50 0x7f328019d250 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f326c00b5c0 tx=0x7f326c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:44:05.678 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:05.681+0000 7f327cc00700 1 -- 172.21.15.153:0/3795619113 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f326c01d070 con 0x7f3280073a50 2024-03-21T10:44:05.678 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:05.681+0000 7f327cc00700 1 -- 172.21.15.153:0/3795619113 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f326c005e00 con 0x7f3280073a50 2024-03-21T10:44:05.678 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:05.681+0000 7f327cc00700 1 -- 172.21.15.153:0/3795619113 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f326c017400 con 0x7f3280073a50 2024-03-21T10:44:05.679 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:05.681+0000 7f3287200700 1 -- 172.21.15.153:0/3795619113 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32801a19a0 con 0x7f3280073a50 2024-03-21T10:44:05.679 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:05.681+0000 7f3287200700 1 -- 172.21.15.153:0/3795619113 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32801a1ef0 con 0x7f3280073a50 2024-03-21T10:44:05.679 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:05.683+0000 7f3287200700 1 -- 172.21.15.153:0/3795619113 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f328010ed90 con 0x7f3280073a50 2024-03-21T10:44:05.680 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:05.684+0000 7f327cc00700 1 -- 172.21.15.153:0/3795619113 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f326c00f460 con 0x7f3280073a50 2024-03-21T10:44:05.681 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:05.685+0000 7f327cc00700 1 --2- 172.21.15.153:0/3795619113 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f326406e460 0x7f3264070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:44:05.682 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:05.685+0000 7f327cc00700 1 -- 172.21.15.153:0/3795619113 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f326c08f9b0 con 0x7f3280073a50 2024-03-21T10:44:05.682 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:05.685+0000 7f327fe00700 1 --2- 172.21.15.153:0/3795619113 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f326406e460 0x7f3264070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:44:05.682 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:05.686+0000 7f327fe00700 1 --2- 172.21.15.153:0/3795619113 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f326406e460 0x7f3264070920 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f3270005950 tx=0x7f32700058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:44:05.686 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:05.690+0000 7f327cc00700 1 -- 172.21.15.153:0/3795619113 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f326c058a70 con 0x7f3280073a50 2024-03-21T10:44:05.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:44:05 smithi153 ceph-mon[113733]: from='client.35714 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:44:05.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:44:05 smithi153 ceph-mon[113733]: pgmap v968: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:44:05.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:44:05 smithi153 ceph-mon[113733]: from='client.35718 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:44:05.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:44:05 smithi153 ceph-mon[113733]: from='client.35722 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:44:05.803 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:44:05 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/3626713700' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:44:05.921 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:05.924+0000 7f3287200700 1 -- 172.21.15.153:0/3795619113 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f32801a2240 con 0x7f326406e460 2024-03-21T10:44:05.924 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:05.928+0000 7f327cc00700 1 -- 172.21.15.153:0/3795619113 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f32801a2240 con 0x7f326406e460 2024-03-21T10:44:05.924 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:44:05.924 INFO:teuthology.orchestra.run.smithi153.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232", 2024-03-21T10:44:05.925 INFO:teuthology.orchestra.run.smithi153.stdout: "in_progress": true, 2024-03-21T10:44:05.925 INFO:teuthology.orchestra.run.smithi153.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-21T10:44:05.925 INFO:teuthology.orchestra.run.smithi153.stdout: "services_complete": [ 2024-03-21T10:44:05.925 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr", 2024-03-21T10:44:05.925 INFO:teuthology.orchestra.run.smithi153.stdout: "mon", 2024-03-21T10:44:05.925 INFO:teuthology.orchestra.run.smithi153.stdout: "mds", 2024-03-21T10:44:05.925 INFO:teuthology.orchestra.run.smithi153.stdout: "crash", 2024-03-21T10:44:05.925 INFO:teuthology.orchestra.run.smithi153.stdout: "osd" 2024-03-21T10:44:05.925 INFO:teuthology.orchestra.run.smithi153.stdout: ], 2024-03-21T10:44:05.925 INFO:teuthology.orchestra.run.smithi153.stdout: "progress": "16/23 daemons upgraded", 2024-03-21T10:44:05.925 INFO:teuthology.orchestra.run.smithi153.stdout: "message": "Currently upgrading mds daemons", 2024-03-21T10:44:05.925 INFO:teuthology.orchestra.run.smithi153.stdout: "is_paused": false 2024-03-21T10:44:05.925 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:44:05.927 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:05.930+0000 7f3287200700 1 -- 172.21.15.153:0/3795619113 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f326406e460 msgr2=0x7f3264070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:44:05.927 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:05.930+0000 7f3287200700 1 --2- 172.21.15.153:0/3795619113 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f326406e460 0x7f3264070920 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f3270005950 tx=0x7f32700058e0 comp rx=0 tx=0).stop 2024-03-21T10:44:05.927 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:05.930+0000 7f3287200700 1 -- 172.21.15.153:0/3795619113 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3280073a50 msgr2=0x7f328019d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:44:05.927 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:05.930+0000 7f3287200700 1 --2- 172.21.15.153:0/3795619113 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3280073a50 0x7f328019d250 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f326c00b5c0 tx=0x7f326c00be20 comp rx=0 tx=0).stop 2024-03-21T10:44:05.927 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:05.930+0000 7f3287200700 1 -- 172.21.15.153:0/3795619113 shutdown_connections 2024-03-21T10:44:05.927 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:05.930+0000 7f3287200700 1 --2- 172.21.15.153:0/3795619113 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3280073130 0x7f328019cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:44:05.927 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:05.930+0000 7f3287200700 1 --2- 172.21.15.153:0/3795619113 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f326406e460 0x7f3264070920 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:44:05.927 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:05.930+0000 7f3287200700 1 --2- 172.21.15.153:0/3795619113 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3280073a50 0x7f328019d250 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:44:05.927 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:05.930+0000 7f3287200700 1 -- 172.21.15.153:0/3795619113 >> 172.21.15.153:0/3795619113 conn(0x7f32800fc630 msgr2=0x7f3280103120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:44:05.928 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:05.931+0000 7f3287200700 1 -- 172.21.15.153:0/3795619113 shutdown_connections 2024-03-21T10:44:05.928 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:05.931+0000 7f3287200700 1 -- 172.21.15.153:0/3795619113 wait complete. 2024-03-21T10:44:05.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:44:05 smithi174 ceph-mon[88004]: from='client.35714 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:44:05.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:44:05 smithi174 ceph-mon[88004]: pgmap v968: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:44:05.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:44:05 smithi174 ceph-mon[88004]: from='client.35718 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:44:05.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:44:05 smithi174 ceph-mon[88004]: from='client.35722 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:44:05.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:44:05 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/3626713700' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:44:06.077 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:06.080+0000 7fae7be00700 1 -- 172.21.15.153:0/1235577673 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fae740ffa40 msgr2=0x7fae740ffec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:44:06.077 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:06.080+0000 7fae7be00700 1 --2- 172.21.15.153:0/1235577673 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fae740ffa40 0x7fae740ffec0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fae60009910 tx=0x7fae60009c20 comp rx=0 tx=0).stop 2024-03-21T10:44:06.077 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:06.080+0000 7fae7be00700 1 -- 172.21.15.153:0/1235577673 shutdown_connections 2024-03-21T10:44:06.077 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:06.080+0000 7fae7be00700 1 --2- 172.21.15.153:0/1235577673 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fae740ffa40 0x7fae740ffec0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:44:06.077 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:06.080+0000 7fae7be00700 1 --2- 172.21.15.153:0/1235577673 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fae74105a40 0x7fae74105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:44:06.077 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:06.080+0000 7fae7be00700 1 -- 172.21.15.153:0/1235577673 >> 172.21.15.153:0/1235577673 conn(0x7fae740fb570 msgr2=0x7fae740fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:44:06.077 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:06.080+0000 7fae7be00700 1 -- 172.21.15.153:0/1235577673 shutdown_connections 2024-03-21T10:44:06.077 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:06.080+0000 7fae7be00700 1 -- 172.21.15.153:0/1235577673 wait complete. 2024-03-21T10:44:06.079 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:06.081+0000 7fae7be00700 1 Processor -- start 2024-03-21T10:44:06.079 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:06.081+0000 7fae7be00700 1 -- start start 2024-03-21T10:44:06.079 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:06.082+0000 7fae7be00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fae740ffa40 0x7fae740fbe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:44:06.079 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:06.082+0000 7fae7be00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fae74105a40 0x7fae740fc3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:44:06.079 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:06.082+0000 7fae7be00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae740fc910 con 0x7fae740ffa40 2024-03-21T10:44:06.079 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:06.082+0000 7fae7be00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae740fca80 con 0x7fae74105a40 2024-03-21T10:44:06.079 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:06.082+0000 7fae79400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fae740ffa40 0x7fae740fbe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:44:06.079 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:06.082+0000 7fae79400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fae740ffa40 0x7fae740fbe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:33556/0 (socket says 172.21.15.153:33556) 2024-03-21T10:44:06.079 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:06.082+0000 7fae79400700 1 -- 172.21.15.153:0/2415262446 learned_addr learned my addr 172.21.15.153:0/2415262446 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:44:06.079 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:06.082+0000 7fae78a00700 1 --2- 172.21.15.153:0/2415262446 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fae74105a40 0x7fae740fc3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:44:06.079 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:06.082+0000 7fae79400700 1 -- 172.21.15.153:0/2415262446 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fae74105a40 msgr2=0x7fae740fc3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:44:06.080 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:06.082+0000 7fae79400700 1 --2- 172.21.15.153:0/2415262446 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fae74105a40 0x7fae740fc3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:44:06.080 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:06.082+0000 7fae79400700 1 -- 172.21.15.153:0/2415262446 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae600095f0 con 0x7fae740ffa40 2024-03-21T10:44:06.080 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:06.083+0000 7fae79400700 1 --2- 172.21.15.153:0/2415262446 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fae740ffa40 0x7fae740fbe90 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fae6800c930 tx=0x7fae6800ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:44:06.080 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:06.083+0000 7fae72000700 1 -- 172.21.15.153:0/2415262446 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fae68007ad0 con 0x7fae740ffa40 2024-03-21T10:44:06.080 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:06.083+0000 7fae72000700 1 -- 172.21.15.153:0/2415262446 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fae68007c30 con 0x7fae740ffa40 2024-03-21T10:44:06.080 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:06.083+0000 7fae7be00700 1 -- 172.21.15.153:0/2415262446 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae740fcd00 con 0x7fae740ffa40 2024-03-21T10:44:06.081 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:06.083+0000 7fae72000700 1 -- 172.21.15.153:0/2415262446 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fae68018610 con 0x7fae740ffa40 2024-03-21T10:44:06.081 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:06.083+0000 7fae7be00700 1 -- 172.21.15.153:0/2415262446 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae741a2ce0 con 0x7fae740ffa40 2024-03-21T10:44:06.081 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:06.084+0000 7fae7be00700 1 -- 172.21.15.153:0/2415262446 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae74108fe0 con 0x7fae740ffa40 2024-03-21T10:44:06.081 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:06.085+0000 7fae72000700 1 -- 172.21.15.153:0/2415262446 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fae6800f450 con 0x7fae740ffa40 2024-03-21T10:44:06.082 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:06.086+0000 7fae72000700 1 --2- 172.21.15.153:0/2415262446 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fae5c06e460 0x7fae5c070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:44:06.082 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:06.086+0000 7fae72000700 1 -- 172.21.15.153:0/2415262446 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fae6808e510 con 0x7fae740ffa40 2024-03-21T10:44:06.082 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:06.086+0000 7fae78a00700 1 --2- 172.21.15.153:0/2415262446 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fae5c06e460 0x7fae5c070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:44:06.083 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:06.087+0000 7fae78a00700 1 --2- 172.21.15.153:0/2415262446 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fae5c06e460 0x7fae5c070920 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fae60000c00 tx=0x7fae60005fd0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:44:06.087 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:06.091+0000 7fae72000700 1 -- 172.21.15.153:0/2415262446 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fae680575b0 con 0x7fae740ffa40 2024-03-21T10:44:06.381 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:06.384+0000 7fae7be00700 1 -- 172.21.15.153:0/2415262446 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fae7404ea90 con 0x7fae740ffa40 2024-03-21T10:44:06.381 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:06.385+0000 7fae72000700 1 -- 172.21.15.153:0/2415262446 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4493 (secure 0 0 0) 0x7fae6805ac70 con 0x7fae740ffa40 2024-03-21T10:44:06.382 INFO:teuthology.orchestra.run.smithi153.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-03-21T10:44:06.382 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-03-21T10:44:06.382 INFO:teuthology.orchestra.run.smithi153.stdout: host smithi174 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi174/config 2024-03-21T10:44:06.382 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/podman: timeout after 895 seconds 2024-03-21T10:44:06.382 INFO:teuthology.orchestra.run.smithi153.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 -e NODE_NAME=smithi174 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/run/ceph:z -v /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpd910r3_i:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 inventory --format=json-pretty --filter-for-batch 2024-03-21T10:44:06.383 INFO:teuthology.orchestra.run.smithi153.stdout:Traceback (most recent call last): 2024-03-21T10:44:06.383 INFO:teuthology.orchestra.run.smithi153.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-03-21T10:44:06.383 INFO:teuthology.orchestra.run.smithi153.stdout: return _run_code(code, main_globals, None, 2024-03-21T10:44:06.383 INFO:teuthology.orchestra.run.smithi153.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-03-21T10:44:06.383 INFO:teuthology.orchestra.run.smithi153.stdout: exec(code, run_globals) 2024-03-21T10:44:06.383 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 5606, in 2024-03-21T10:44:06.383 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 5594, in main 2024-03-21T10:44:06.383 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 406, in _infer_config 2024-03-21T10:44:06.383 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 321, in _infer_fsid 2024-03-21T10:44:06.383 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 434, in _infer_image 2024-03-21T10:44:06.383 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 308, in _validate_fsid 2024-03-21T10:44:06.383 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 3357, in command_ceph_volume 2024-03-21T10:44:06.383 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-03-21T10:44:06.383 INFO:teuthology.orchestra.run.smithi153.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 -e NODE_NAME=smithi174 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/run/ceph:z -v /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpd910r3_i:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 inventory --format=json-pretty --filter-for-batch 2024-03-21T10:44:06.384 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-21T10:44:06.384 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is degraded 2024-03-21T10:44:06.384 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-21T10:44:06.384 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-21T10:44:06.384 INFO:teuthology.orchestra.run.smithi153.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-21T10:44:06.384 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-21T10:44:06.385 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:06.388+0000 7fae7be00700 1 -- 172.21.15.153:0/2415262446 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fae5c06e460 msgr2=0x7fae5c070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:44:06.385 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:06.388+0000 7fae7be00700 1 --2- 172.21.15.153:0/2415262446 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fae5c06e460 0x7fae5c070920 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fae60000c00 tx=0x7fae60005fd0 comp rx=0 tx=0).stop 2024-03-21T10:44:06.385 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:06.388+0000 7fae7be00700 1 -- 172.21.15.153:0/2415262446 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fae740ffa40 msgr2=0x7fae740fbe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:44:06.385 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:06.388+0000 7fae7be00700 1 --2- 172.21.15.153:0/2415262446 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fae740ffa40 0x7fae740fbe90 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fae6800c930 tx=0x7fae6800ccf0 comp rx=0 tx=0).stop 2024-03-21T10:44:06.385 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:06.388+0000 7fae7be00700 1 -- 172.21.15.153:0/2415262446 shutdown_connections 2024-03-21T10:44:06.385 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:06.388+0000 7fae7be00700 1 --2- 172.21.15.153:0/2415262446 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fae5c06e460 0x7fae5c070920 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:44:06.385 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:06.388+0000 7fae7be00700 1 --2- 172.21.15.153:0/2415262446 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fae740ffa40 0x7fae740fbe90 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:44:06.385 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:06.388+0000 7fae7be00700 1 --2- 172.21.15.153:0/2415262446 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fae74105a40 0x7fae740fc3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:44:06.385 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:06.388+0000 7fae7be00700 1 -- 172.21.15.153:0/2415262446 >> 172.21.15.153:0/2415262446 conn(0x7fae740fb570 msgr2=0x7fae74068f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:44:06.385 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:06.388+0000 7fae7be00700 1 -- 172.21.15.153:0/2415262446 shutdown_connections 2024-03-21T10:44:06.385 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:06.388+0000 7fae7be00700 1 -- 172.21.15.153:0/2415262446 wait complete. 2024-03-21T10:44:06.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:44:06 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/746822144' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:44:06.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:44:06 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/2415262446' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:44:06.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:44:06 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/746822144' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:44:06.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:44:06 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/2415262446' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:44:07.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:44:07 smithi153 ceph-mon[113733]: pgmap v969: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:44:07.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:44:07 smithi153 ceph-mon[113733]: from='client.35734 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:44:07.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:44:07 smithi174 ceph-mon[88004]: pgmap v969: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:44:07.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:44:07 smithi174 ceph-mon[88004]: from='client.35734 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:44:09.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:44:09 smithi153 ceph-mon[113733]: pgmap v970: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:44:09.849 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:44:09 smithi174 ceph-mon[88004]: pgmap v970: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:44:11.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:44:11 smithi153 ceph-mon[113733]: pgmap v971: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:44:11.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:44:11 smithi174 ceph-mon[88004]: pgmap v971: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:44:13.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:44:13 smithi153 ceph-mon[113733]: pgmap v972: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:44:13.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:44:13 smithi174 ceph-mon[88004]: pgmap v972: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:44:15.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:44:15 smithi153 ceph-mon[113733]: pgmap v973: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:44:15.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:44:15 smithi174 ceph-mon[88004]: pgmap v973: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:44:16.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:44:16 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:44:16.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:44:16 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:44:17.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:44:17 smithi153 ceph-mon[113733]: pgmap v974: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:44:17.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:44:17 smithi174 ceph-mon[88004]: pgmap v974: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:44:19.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:44:19 smithi174 ceph-mon[88004]: pgmap v975: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:44:20.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:44:19 smithi153 ceph-mon[113733]: pgmap v975: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:44:21.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:44:21 smithi174 ceph-mon[88004]: pgmap v976: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:44:22.052 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:44:21 smithi153 ceph-mon[113733]: pgmap v976: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:44:23.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:44:23 smithi174 ceph-mon[88004]: pgmap v977: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:44:24.052 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:44:23 smithi153 ceph-mon[113733]: pgmap v977: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:44:25.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:44:25 smithi174 ceph-mon[88004]: pgmap v978: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:44:26.052 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:44:25 smithi153 ceph-mon[113733]: pgmap v978: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:44:27.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:44:27 smithi174 ceph-mon[88004]: pgmap v979: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:44:28.052 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:44:27 smithi153 ceph-mon[113733]: pgmap v979: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:44:28.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:44:28 smithi174 ceph-mon[88004]: pgmap v980: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:44:29.052 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:44:28 smithi153 ceph-mon[113733]: pgmap v980: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:44:31.302 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:44:30 smithi153 ceph-mon[113733]: pgmap v981: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:44:31.447 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:44:30 smithi174 ceph-mon[88004]: pgmap v981: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:44:32.302 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:44:32 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:44:32.447 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:44:32 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:44:33.302 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:44:33 smithi153 ceph-mon[113733]: pgmap v982: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:44:33.447 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:44:33 smithi174 ceph-mon[88004]: pgmap v982: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:44:35.302 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:44:35 smithi153 ceph-mon[113733]: pgmap v983: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:44:35.447 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:44:35 smithi174 ceph-mon[88004]: pgmap v983: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:44:36.558 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:36.560+0000 7f484e000700 1 -- 172.21.15.153:0/441150075 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f484810d280 msgr2=0x7f484810d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:44:36.558 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:36.561+0000 7f484e000700 1 --2- 172.21.15.153:0/441150075 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f484810d280 0x7f484810d660 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f4838009910 tx=0x7f4838009c20 comp rx=0 tx=0).stop 2024-03-21T10:44:36.559 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:36.561+0000 7f484e000700 1 -- 172.21.15.153:0/441150075 shutdown_connections 2024-03-21T10:44:36.559 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:36.561+0000 7f484e000700 1 --2- 172.21.15.153:0/441150075 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f48481071d0 0x7f4848107650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:44:36.559 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:36.561+0000 7f484e000700 1 --2- 172.21.15.153:0/441150075 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f484810d280 0x7f484810d660 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:44:36.559 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:36.561+0000 7f484e000700 1 -- 172.21.15.153:0/441150075 >> 172.21.15.153:0/441150075 conn(0x7f48480769e0 msgr2=0x7f4848078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:44:36.559 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:36.561+0000 7f484e000700 1 -- 172.21.15.153:0/441150075 shutdown_connections 2024-03-21T10:44:36.559 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:36.561+0000 7f484e000700 1 -- 172.21.15.153:0/441150075 wait complete. 2024-03-21T10:44:36.559 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:36.562+0000 7f484e000700 1 Processor -- start 2024-03-21T10:44:36.560 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:36.562+0000 7f484e000700 1 -- start start 2024-03-21T10:44:36.560 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:36.563+0000 7f484e000700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f48481071d0 0x7f484819cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:44:36.561 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:36.563+0000 7f484e000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f484810d280 0x7f484819d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:44:36.561 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:36.563+0000 7f484e000700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f484819da10 con 0x7f484810d280 2024-03-21T10:44:36.561 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:36.563+0000 7f484e000700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48481a17a0 con 0x7f48481071d0 2024-03-21T10:44:36.561 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:36.563+0000 7f4846a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f484810d280 0x7f484819d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:44:36.561 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:36.563+0000 7f4846a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f484810d280 0x7f484819d330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41918/0 (socket says 172.21.15.153:41918) 2024-03-21T10:44:36.561 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:36.563+0000 7f4846a00700 1 -- 172.21.15.153:0/1470707 learned_addr learned my addr 172.21.15.153:0/1470707 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:44:36.561 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:36.564+0000 7f4847400700 1 --2- 172.21.15.153:0/1470707 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f48481071d0 0x7f484819cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:44:36.562 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:36.564+0000 7f4846a00700 1 -- 172.21.15.153:0/1470707 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f48481071d0 msgr2=0x7f484819cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:44:36.562 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:36.564+0000 7f4846a00700 1 --2- 172.21.15.153:0/1470707 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f48481071d0 0x7f484819cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:44:36.562 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:36.564+0000 7f4846a00700 1 -- 172.21.15.153:0/1470707 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48380095f0 con 0x7f484810d280 2024-03-21T10:44:36.562 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:36.564+0000 7f4846a00700 1 --2- 172.21.15.153:0/1470707 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f484810d280 0x7f484819d330 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f483400b4b0 tx=0x7f483400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:44:36.562 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:36.564+0000 7f483fe00700 1 -- 172.21.15.153:0/1470707 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4834011800 con 0x7f484810d280 2024-03-21T10:44:36.562 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:36.564+0000 7f483fe00700 1 -- 172.21.15.153:0/1470707 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4834011e20 con 0x7f484810d280 2024-03-21T10:44:36.563 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:36.564+0000 7f484e000700 1 -- 172.21.15.153:0/1470707 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48481a1a80 con 0x7f484810d280 2024-03-21T10:44:36.563 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:36.564+0000 7f483fe00700 1 -- 172.21.15.153:0/1470707 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f483400f2e0 con 0x7f484810d280 2024-03-21T10:44:36.563 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:36.565+0000 7f484e000700 1 -- 172.21.15.153:0/1470707 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48481a2050 con 0x7f484810d280 2024-03-21T10:44:36.563 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:36.566+0000 7f484e000700 1 -- 172.21.15.153:0/1470707 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f484810f6f0 con 0x7f484810d280 2024-03-21T10:44:36.564 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:36.567+0000 7f483fe00700 1 -- 172.21.15.153:0/1470707 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f4834010380 con 0x7f484810d280 2024-03-21T10:44:36.566 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:36.568+0000 7f483fe00700 1 --2- 172.21.15.153:0/1470707 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f482c06e460 0x7f482c070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:44:36.566 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:36.569+0000 7f483fe00700 1 -- 172.21.15.153:0/1470707 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f483408e190 con 0x7f484810d280 2024-03-21T10:44:36.566 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:36.569+0000 7f4847400700 1 --2- 172.21.15.153:0/1470707 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f482c06e460 0x7f482c070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:44:36.567 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:36.570+0000 7f4847400700 1 --2- 172.21.15.153:0/1470707 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f482c06e460 0x7f482c070920 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f483800b5c0 tx=0x7f483801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:44:36.570 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:36.573+0000 7f483fe00700 1 -- 172.21.15.153:0/1470707 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f48340571b0 con 0x7f484810d280 2024-03-21T10:44:36.810 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:36.812+0000 7f484e000700 1 -- 172.21.15.153:0/1470707 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f48481a23e0 con 0x7f482c06e460 2024-03-21T10:44:36.812 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:36.815+0000 7f483fe00700 1 -- 172.21.15.153:0/1470707 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f48481a23e0 con 0x7f482c06e460 2024-03-21T10:44:36.815 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:36.817+0000 7f484e000700 1 -- 172.21.15.153:0/1470707 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f482c06e460 msgr2=0x7f482c070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:44:36.815 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:36.818+0000 7f484e000700 1 --2- 172.21.15.153:0/1470707 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f482c06e460 0x7f482c070920 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f483800b5c0 tx=0x7f483801a040 comp rx=0 tx=0).stop 2024-03-21T10:44:36.815 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:36.818+0000 7f484e000700 1 -- 172.21.15.153:0/1470707 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f484810d280 msgr2=0x7f484819d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:44:36.815 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:36.818+0000 7f484e000700 1 --2- 172.21.15.153:0/1470707 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f484810d280 0x7f484819d330 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f483400b4b0 tx=0x7f483400b870 comp rx=0 tx=0).stop 2024-03-21T10:44:36.815 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:36.818+0000 7f484e000700 1 -- 172.21.15.153:0/1470707 shutdown_connections 2024-03-21T10:44:36.815 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:36.818+0000 7f484e000700 1 --2- 172.21.15.153:0/1470707 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f48481071d0 0x7f484819cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:44:36.815 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:36.818+0000 7f484e000700 1 --2- 172.21.15.153:0/1470707 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f482c06e460 0x7f482c070920 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:44:36.816 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:36.818+0000 7f484e000700 1 --2- 172.21.15.153:0/1470707 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f484810d280 0x7f484819d330 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:44:36.816 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:36.818+0000 7f484e000700 1 -- 172.21.15.153:0/1470707 >> 172.21.15.153:0/1470707 conn(0x7f48480769e0 msgr2=0x7f4848078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:44:36.816 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:36.818+0000 7f484e000700 1 -- 172.21.15.153:0/1470707 shutdown_connections 2024-03-21T10:44:36.816 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:36.818+0000 7f484e000700 1 -- 172.21.15.153:0/1470707 wait complete. 2024-03-21T10:44:36.830 INFO:teuthology.orchestra.run.smithi153.stdout:true 2024-03-21T10:44:36.966 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:36.968+0000 7f1bdaa00700 1 -- 172.21.15.153:0/2141841623 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1bd41071d0 msgr2=0x7f1bd4107650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:44:36.966 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:36.968+0000 7f1bdaa00700 1 --2- 172.21.15.153:0/2141841623 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1bd41071d0 0x7f1bd4107650 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f1bc0009910 tx=0x7f1bc0009c20 comp rx=0 tx=0).stop 2024-03-21T10:44:36.966 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:36.968+0000 7f1bdaa00700 1 -- 172.21.15.153:0/2141841623 shutdown_connections 2024-03-21T10:44:36.966 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:36.968+0000 7f1bdaa00700 1 --2- 172.21.15.153:0/2141841623 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1bd41071d0 0x7f1bd4107650 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:44:36.966 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:36.968+0000 7f1bdaa00700 1 --2- 172.21.15.153:0/2141841623 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1bd410d280 0x7f1bd410d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:44:36.966 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:36.968+0000 7f1bdaa00700 1 -- 172.21.15.153:0/2141841623 >> 172.21.15.153:0/2141841623 conn(0x7f1bd40769e0 msgr2=0x7f1bd4078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:44:36.966 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:36.968+0000 7f1bdaa00700 1 -- 172.21.15.153:0/2141841623 shutdown_connections 2024-03-21T10:44:36.966 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:36.968+0000 7f1bdaa00700 1 -- 172.21.15.153:0/2141841623 wait complete. 2024-03-21T10:44:36.967 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:36.969+0000 7f1bdaa00700 1 Processor -- start 2024-03-21T10:44:36.967 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:36.969+0000 7f1bdaa00700 1 -- start start 2024-03-21T10:44:36.967 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:36.970+0000 7f1bdaa00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1bd41071d0 0x7f1bd419ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:44:36.967 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:36.970+0000 7f1bdaa00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1bd410d280 0x7f1bd419d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:44:36.967 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:36.970+0000 7f1bdaa00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bd419d880 con 0x7f1bd41071d0 2024-03-21T10:44:36.967 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:36.970+0000 7f1bdaa00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bd41a05b0 con 0x7f1bd410d280 2024-03-21T10:44:36.968 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:36.970+0000 7f1bd3e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1bd41071d0 0x7f1bd419ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:44:36.968 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:36.970+0000 7f1bd3e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1bd41071d0 0x7f1bd419ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41944/0 (socket says 172.21.15.153:41944) 2024-03-21T10:44:36.968 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:36.970+0000 7f1bd3e00700 1 -- 172.21.15.153:0/175223216 learned_addr learned my addr 172.21.15.153:0/175223216 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:44:36.969 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:36.970+0000 7f1bd3400700 1 --2- 172.21.15.153:0/175223216 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1bd410d280 0x7f1bd419d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:44:36.969 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:36.971+0000 7f1bd3e00700 1 -- 172.21.15.153:0/175223216 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1bd410d280 msgr2=0x7f1bd419d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:44:36.969 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:36.971+0000 7f1bd3e00700 1 --2- 172.21.15.153:0/175223216 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1bd410d280 0x7f1bd419d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:44:36.969 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:36.971+0000 7f1bd3e00700 1 -- 172.21.15.153:0/175223216 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bc00095f0 con 0x7f1bd41071d0 2024-03-21T10:44:36.969 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:36.971+0000 7f1bd3e00700 1 --2- 172.21.15.153:0/175223216 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1bd41071d0 0x7f1bd419ccb0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f1bc400d8d0 tx=0x7f1bc400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:44:36.969 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:36.971+0000 7f1bd0c00700 1 -- 172.21.15.153:0/175223216 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1bc4009940 con 0x7f1bd41071d0 2024-03-21T10:44:36.969 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:36.971+0000 7f1bd0c00700 1 -- 172.21.15.153:0/175223216 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1bc4010460 con 0x7f1bd41071d0 2024-03-21T10:44:36.969 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:36.971+0000 7f1bdaa00700 1 -- 172.21.15.153:0/175223216 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bd41a0890 con 0x7f1bd41071d0 2024-03-21T10:44:36.969 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:36.971+0000 7f1bd0c00700 1 -- 172.21.15.153:0/175223216 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1bc400f5f0 con 0x7f1bd41071d0 2024-03-21T10:44:36.969 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:36.971+0000 7f1bdaa00700 1 -- 172.21.15.153:0/175223216 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bd41a0e60 con 0x7f1bd41071d0 2024-03-21T10:44:36.970 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:36.972+0000 7f1bdaa00700 1 -- 172.21.15.153:0/175223216 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bd410f6c0 con 0x7f1bd41071d0 2024-03-21T10:44:36.970 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:36.973+0000 7f1bd0c00700 1 -- 172.21.15.153:0/175223216 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f1bc40105d0 con 0x7f1bd41071d0 2024-03-21T10:44:36.972 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:36.974+0000 7f1bd0c00700 1 --2- 172.21.15.153:0/175223216 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f1bb806e460 0x7f1bb8070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:44:36.972 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:36.975+0000 7f1bd0c00700 1 -- 172.21.15.153:0/175223216 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f1bc408e480 con 0x7f1bd41071d0 2024-03-21T10:44:36.972 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:36.975+0000 7f1bd3400700 1 --2- 172.21.15.153:0/175223216 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f1bb806e460 0x7f1bb8070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:44:36.972 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:36.975+0000 7f1bd3400700 1 --2- 172.21.15.153:0/175223216 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f1bb806e460 0x7f1bb8070920 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f1bd419e290 tx=0x7f1bc001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:44:36.976 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:36.979+0000 7f1bd0c00700 1 -- 172.21.15.153:0/175223216 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f1bc4057520 con 0x7f1bd41071d0 2024-03-21T10:44:37.201 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:37.204+0000 7f1bdaa00700 1 -- 172.21.15.153:0/175223216 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1bd41a1140 con 0x7f1bb806e460 2024-03-21T10:44:37.202 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:44:37 smithi153 ceph-mon[113733]: pgmap v984: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:44:37.204 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:37.207+0000 7f1bd0c00700 1 -- 172.21.15.153:0/175223216 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f1bd41a1140 con 0x7f1bb806e460 2024-03-21T10:44:37.206 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:37.209+0000 7f1bdaa00700 1 -- 172.21.15.153:0/175223216 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f1bb806e460 msgr2=0x7f1bb8070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:44:37.207 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:37.209+0000 7f1bdaa00700 1 --2- 172.21.15.153:0/175223216 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f1bb806e460 0x7f1bb8070920 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f1bd419e290 tx=0x7f1bc001a040 comp rx=0 tx=0).stop 2024-03-21T10:44:37.207 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:37.209+0000 7f1bdaa00700 1 -- 172.21.15.153:0/175223216 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1bd41071d0 msgr2=0x7f1bd419ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:44:37.207 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:37.209+0000 7f1bdaa00700 1 --2- 172.21.15.153:0/175223216 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1bd41071d0 0x7f1bd419ccb0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f1bc400d8d0 tx=0x7f1bc400dc90 comp rx=0 tx=0).stop 2024-03-21T10:44:37.207 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:37.210+0000 7f1bdaa00700 1 -- 172.21.15.153:0/175223216 shutdown_connections 2024-03-21T10:44:37.207 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:37.210+0000 7f1bdaa00700 1 --2- 172.21.15.153:0/175223216 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f1bb806e460 0x7f1bb8070920 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:44:37.207 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:37.210+0000 7f1bdaa00700 1 --2- 172.21.15.153:0/175223216 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1bd41071d0 0x7f1bd419ccb0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:44:37.208 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:37.210+0000 7f1bdaa00700 1 --2- 172.21.15.153:0/175223216 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1bd410d280 0x7f1bd419d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:44:37.208 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:37.210+0000 7f1bdaa00700 1 -- 172.21.15.153:0/175223216 >> 172.21.15.153:0/175223216 conn(0x7f1bd40769e0 msgr2=0x7f1bd4078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:44:37.208 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:37.210+0000 7f1bdaa00700 1 -- 172.21.15.153:0/175223216 shutdown_connections 2024-03-21T10:44:37.208 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:37.210+0000 7f1bdaa00700 1 -- 172.21.15.153:0/175223216 wait complete. 2024-03-21T10:44:37.362 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:37.365+0000 7f33d2200700 1 -- 172.21.15.153:0/1696610848 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f33cc1071d0 msgr2=0x7f33cc107650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:44:37.363 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:37.365+0000 7f33d2200700 1 --2- 172.21.15.153:0/1696610848 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f33cc1071d0 0x7f33cc107650 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f33b4009910 tx=0x7f33b4009c20 comp rx=0 tx=0).stop 2024-03-21T10:44:37.363 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:37.365+0000 7f33d2200700 1 -- 172.21.15.153:0/1696610848 shutdown_connections 2024-03-21T10:44:37.363 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:37.365+0000 7f33d2200700 1 --2- 172.21.15.153:0/1696610848 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f33cc1071d0 0x7f33cc107650 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:44:37.363 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:37.365+0000 7f33d2200700 1 --2- 172.21.15.153:0/1696610848 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f33cc10d280 0x7f33cc10d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:44:37.363 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:37.365+0000 7f33d2200700 1 -- 172.21.15.153:0/1696610848 >> 172.21.15.153:0/1696610848 conn(0x7f33cc0769e0 msgr2=0x7f33cc078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:44:37.363 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:37.365+0000 7f33d2200700 1 -- 172.21.15.153:0/1696610848 shutdown_connections 2024-03-21T10:44:37.363 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:37.365+0000 7f33d2200700 1 -- 172.21.15.153:0/1696610848 wait complete. 2024-03-21T10:44:37.364 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:37.366+0000 7f33d2200700 1 Processor -- start 2024-03-21T10:44:37.364 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:37.366+0000 7f33d2200700 1 -- start start 2024-03-21T10:44:37.364 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:37.366+0000 7f33d2200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f33cc1071d0 0x7f33cc19ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:44:37.365 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:37.367+0000 7f33d2200700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f33cc10d280 0x7f33cc19d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:44:37.365 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:37.367+0000 7f33d2200700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33cc19d880 con 0x7f33cc1071d0 2024-03-21T10:44:37.365 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:37.367+0000 7f33d2200700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33cc1a05b0 con 0x7f33cc10d280 2024-03-21T10:44:37.365 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:37.367+0000 7f33cb400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f33cc1071d0 0x7f33cc19ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:44:37.365 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:37.367+0000 7f33cb400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f33cc1071d0 0x7f33cc19ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41960/0 (socket says 172.21.15.153:41960) 2024-03-21T10:44:37.365 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:37.367+0000 7f33cb400700 1 -- 172.21.15.153:0/4147688365 learned_addr learned my addr 172.21.15.153:0/4147688365 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:44:37.365 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:37.367+0000 7f33cb400700 1 -- 172.21.15.153:0/4147688365 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f33cc10d280 msgr2=0x7f33cc19d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:44:37.365 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:37.367+0000 7f33c2a00700 1 --2- 172.21.15.153:0/4147688365 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f33cc10d280 0x7f33cc19d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:44:37.365 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:37.367+0000 7f33cb400700 1 --2- 172.21.15.153:0/4147688365 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f33cc10d280 0x7f33cc19d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:44:37.365 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:37.367+0000 7f33cb400700 1 -- 172.21.15.153:0/4147688365 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33b40095f0 con 0x7f33cc1071d0 2024-03-21T10:44:37.366 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:37.368+0000 7f33cb400700 1 --2- 172.21.15.153:0/4147688365 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f33cc1071d0 0x7f33cc19ccb0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f33bc00d8d0 tx=0x7f33bc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:44:37.366 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:37.368+0000 7f33c8c00700 1 -- 172.21.15.153:0/4147688365 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f33bc009940 con 0x7f33cc1071d0 2024-03-21T10:44:37.366 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:37.368+0000 7f33c8c00700 1 -- 172.21.15.153:0/4147688365 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f33bc010460 con 0x7f33cc1071d0 2024-03-21T10:44:37.366 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:37.368+0000 7f33c8c00700 1 -- 172.21.15.153:0/4147688365 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f33bc00f5f0 con 0x7f33cc1071d0 2024-03-21T10:44:37.367 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:37.368+0000 7f33d2200700 1 -- 172.21.15.153:0/4147688365 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33cc1a0890 con 0x7f33cc1071d0 2024-03-21T10:44:37.367 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:37.368+0000 7f33d2200700 1 -- 172.21.15.153:0/4147688365 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33cc1a0e60 con 0x7f33cc1071d0 2024-03-21T10:44:37.367 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:37.369+0000 7f33d2200700 1 -- 172.21.15.153:0/4147688365 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33cc10f6c0 con 0x7f33cc1071d0 2024-03-21T10:44:37.368 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:37.371+0000 7f33c8c00700 1 -- 172.21.15.153:0/4147688365 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f33bc0105d0 con 0x7f33cc1071d0 2024-03-21T10:44:37.369 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:37.371+0000 7f33c8c00700 1 --2- 172.21.15.153:0/4147688365 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f33b006e460 0x7f33b0070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:44:37.369 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:37.372+0000 7f33c8c00700 1 -- 172.21.15.153:0/4147688365 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f33bc08e3e0 con 0x7f33cc1071d0 2024-03-21T10:44:37.369 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:37.372+0000 7f33c2a00700 1 --2- 172.21.15.153:0/4147688365 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f33b006e460 0x7f33b0070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:44:37.369 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:37.372+0000 7f33c2a00700 1 --2- 172.21.15.153:0/4147688365 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f33b006e460 0x7f33b0070920 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f33b4000c00 tx=0x7f33b4005fd0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:44:37.374 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:37.377+0000 7f33c8c00700 1 -- 172.21.15.153:0/4147688365 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f33bc057520 con 0x7f33cc1071d0 2024-03-21T10:44:37.447 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:44:37 smithi174 ceph-mon[88004]: pgmap v984: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:44:37.607 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:37.609+0000 7f33d2200700 1 -- 172.21.15.153:0/4147688365 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f33cc1a1140 con 0x7f33b006e460 2024-03-21T10:44:37.639 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:37.642+0000 7f33c8c00700 1 -- 172.21.15.153:0/4147688365 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f33cc1a1140 con 0x7f33b006e460 2024-03-21T10:44:37.639 INFO:teuthology.orchestra.run.smithi153.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-21T10:44:37.639 INFO:teuthology.orchestra.run.smithi153.stdout:alertmanager.smithi153 smithi153 *:9093,9094 running (33m) 9m ago 47m 15.6M - 0.25.0 c8568f914cd2 9ed3f2200b5f 2024-03-21T10:44:37.640 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi153 smithi153 running (47m) 9m ago 47m 10.4M - 18.2.1 5be31c24972a 5b61994396a6 2024-03-21T10:44:37.640 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi174 smithi174 running (45m) 9m ago 45m 10.1M - 18.2.1 5be31c24972a 4f7a5fc3ac28 2024-03-21T10:44:37.640 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi153 smithi153 running (29m) 9m ago 47m 6656k - 19.0.0-2231-g2edfee68 237e43c7a65f 32c21104d5b0 2024-03-21T10:44:37.640 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi174 smithi174 running (29m) 9m ago 45m 6660k - 19.0.0-2231-g2edfee68 237e43c7a65f a04e48967ab6 2024-03-21T10:44:37.640 INFO:teuthology.orchestra.run.smithi153.stdout:grafana.smithi153 smithi153 *:3000 running (32m) 9m ago 46m 75.2M - 9.4.12 2bacad6d85d8 ee24026ab786 2024-03-21T10:44:37.640 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.iuyluj smithi153 running (24m) 9m ago 39m 18.7M - 19.0.0-2231-g2edfee68 237e43c7a65f 87aff8a5de70 2024-03-21T10:44:37.640 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.zipckr smithi153 running (25m) 9m ago 40m 18.9M - 19.0.0-2231-g2edfee68 237e43c7a65f ccc262bd5dd3 2024-03-21T10:44:37.640 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.nhozag smithi174 running (9m) 9m ago 39m 10.9M - 19.0.0-2231-g2edfee68 237e43c7a65f 9e02aea72c96 2024-03-21T10:44:37.640 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.yyimst smithi174 running (24m) 9m ago 40m 18.5M - 19.0.0-2231-g2edfee68 237e43c7a65f fc1962f9351e 2024-03-21T10:44:37.640 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi153.tluxni smithi153 *:8443,9283,8765 running (36m) 9m ago 48m 512M - 19.0.0-2231-g2edfee68 237e43c7a65f 7ffee0e55382 2024-03-21T10:44:37.640 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi174.qpcwat smithi174 *:8443,9283,8765 running (35m) 9m ago 45m 459M - 19.0.0-2231-g2edfee68 237e43c7a65f 748968f216ab 2024-03-21T10:44:37.640 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi153 smithi153 running (31m) 9m ago 49m 70.8M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f dd8038303dc3 2024-03-21T10:44:37.640 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi174 smithi174 running (30m) 9m ago 45m 51.9M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f 29d78a788f2c 2024-03-21T10:44:37.640 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi153 smithi153 *:9100 running (34m) 9m ago 47m 8140k - 1.5.0 0da6a335fe13 bb835edc0e0b 2024-03-21T10:44:37.640 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi174 smithi174 *:9100 running (34m) 9m ago 45m 8610k - 1.5.0 0da6a335fe13 7c4ecad790fa 2024-03-21T10:44:37.641 INFO:teuthology.orchestra.run.smithi153.stdout:osd.0 smithi153 running (29m) 9m ago 44m 118M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 507900fd4791 2024-03-21T10:44:37.641 INFO:teuthology.orchestra.run.smithi153.stdout:osd.1 smithi153 running (28m) 9m ago 44m 93.0M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 0d646452e9c1 2024-03-21T10:44:37.641 INFO:teuthology.orchestra.run.smithi153.stdout:osd.2 smithi153 running (27m) 9m ago 43m 91.2M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 1326f2afdfcd 2024-03-21T10:44:37.641 INFO:teuthology.orchestra.run.smithi153.stdout:osd.3 smithi174 running (27m) 9m ago 42m 87.5M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 70aafac71ad5 2024-03-21T10:44:37.641 INFO:teuthology.orchestra.run.smithi153.stdout:osd.4 smithi174 running (26m) 9m ago 42m 95.8M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 0d746ce6e79d 2024-03-21T10:44:37.641 INFO:teuthology.orchestra.run.smithi153.stdout:osd.5 smithi174 running (25m) 9m ago 41m 90.3M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 53e5f35b8cac 2024-03-21T10:44:37.641 INFO:teuthology.orchestra.run.smithi153.stdout:prometheus.smithi153 smithi153 *:9095 running (33m) 9m ago 46m 88.1M - 2.43.0 a07b618ecd1d 7d95561e50c2 2024-03-21T10:44:37.643 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:37.645+0000 7f33d2200700 1 -- 172.21.15.153:0/4147688365 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f33b006e460 msgr2=0x7f33b0070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:44:37.643 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:37.645+0000 7f33d2200700 1 --2- 172.21.15.153:0/4147688365 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f33b006e460 0x7f33b0070920 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f33b4000c00 tx=0x7f33b4005fd0 comp rx=0 tx=0).stop 2024-03-21T10:44:37.643 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:37.645+0000 7f33d2200700 1 -- 172.21.15.153:0/4147688365 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f33cc1071d0 msgr2=0x7f33cc19ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:44:37.643 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:37.645+0000 7f33d2200700 1 --2- 172.21.15.153:0/4147688365 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f33cc1071d0 0x7f33cc19ccb0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f33bc00d8d0 tx=0x7f33bc00dc90 comp rx=0 tx=0).stop 2024-03-21T10:44:37.643 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:37.645+0000 7f33d2200700 1 -- 172.21.15.153:0/4147688365 shutdown_connections 2024-03-21T10:44:37.643 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:37.645+0000 7f33d2200700 1 --2- 172.21.15.153:0/4147688365 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f33b006e460 0x7f33b0070920 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:44:37.643 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:37.645+0000 7f33d2200700 1 --2- 172.21.15.153:0/4147688365 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f33cc1071d0 0x7f33cc19ccb0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:44:37.643 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:37.645+0000 7f33d2200700 1 --2- 172.21.15.153:0/4147688365 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f33cc10d280 0x7f33cc19d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:44:37.643 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:37.645+0000 7f33d2200700 1 -- 172.21.15.153:0/4147688365 >> 172.21.15.153:0/4147688365 conn(0x7f33cc0769e0 msgr2=0x7f33cc078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:44:37.643 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:37.645+0000 7f33d2200700 1 -- 172.21.15.153:0/4147688365 shutdown_connections 2024-03-21T10:44:37.643 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:37.645+0000 7f33d2200700 1 -- 172.21.15.153:0/4147688365 wait complete. 2024-03-21T10:44:37.787 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:37.790+0000 7f406f000700 1 -- 172.21.15.153:0/154880803 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4068103000 msgr2=0x7f40681033e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:44:37.787 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:37.790+0000 7f406f000700 1 --2- 172.21.15.153:0/154880803 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4068103000 0x7f40681033e0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f4058009910 tx=0x7f4058009c20 comp rx=0 tx=0).stop 2024-03-21T10:44:37.788 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:37.790+0000 7f406f000700 1 -- 172.21.15.153:0/154880803 shutdown_connections 2024-03-21T10:44:37.788 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:37.790+0000 7f406f000700 1 --2- 172.21.15.153:0/154880803 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f40681039b0 0x7f4068107a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:44:37.788 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:37.790+0000 7f406f000700 1 --2- 172.21.15.153:0/154880803 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4068103000 0x7f40681033e0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:44:37.788 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:37.790+0000 7f406f000700 1 -- 172.21.15.153:0/154880803 >> 172.21.15.153:0/154880803 conn(0x7f40680fe8d0 msgr2=0x7f4068100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:44:37.788 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:37.790+0000 7f406f000700 1 -- 172.21.15.153:0/154880803 shutdown_connections 2024-03-21T10:44:37.788 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:37.790+0000 7f406f000700 1 -- 172.21.15.153:0/154880803 wait complete. 2024-03-21T10:44:37.789 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:37.791+0000 7f406f000700 1 Processor -- start 2024-03-21T10:44:37.789 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:37.791+0000 7f406f000700 1 -- start start 2024-03-21T10:44:37.789 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:37.791+0000 7f406f000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4068103000 0x7f406819e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:44:37.789 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:37.791+0000 7f406f000700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f40681039b0 0x7f406819ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:44:37.789 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:37.791+0000 7f406f000700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f406819f330 con 0x7f4068103000 2024-03-21T10:44:37.789 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:37.791+0000 7f406f000700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4068198870 con 0x7f40681039b0 2024-03-21T10:44:37.789 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:37.792+0000 7f4067e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4068103000 0x7f406819e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:44:37.789 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:37.792+0000 7f4067e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4068103000 0x7f406819e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41974/0 (socket says 172.21.15.153:41974) 2024-03-21T10:44:37.790 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:37.792+0000 7f4067e00700 1 -- 172.21.15.153:0/2640913725 learned_addr learned my addr 172.21.15.153:0/2640913725 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:44:37.791 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:37.792+0000 7f4067400700 1 --2- 172.21.15.153:0/2640913725 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f40681039b0 0x7f406819ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:44:37.791 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:37.792+0000 7f4067e00700 1 -- 172.21.15.153:0/2640913725 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f40681039b0 msgr2=0x7f406819ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:44:37.791 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:37.792+0000 7f4067e00700 1 --2- 172.21.15.153:0/2640913725 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f40681039b0 0x7f406819ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:44:37.791 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:37.792+0000 7f4067e00700 1 -- 172.21.15.153:0/2640913725 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40580095f0 con 0x7f4068103000 2024-03-21T10:44:37.791 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:37.793+0000 7f4067e00700 1 --2- 172.21.15.153:0/2640913725 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4068103000 0x7f406819e7f0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f405800b5c0 tx=0x7f405800be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:44:37.791 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:37.793+0000 7f4064c00700 1 -- 172.21.15.153:0/2640913725 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f405801d070 con 0x7f4068103000 2024-03-21T10:44:37.791 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:37.793+0000 7f4064c00700 1 -- 172.21.15.153:0/2640913725 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f405800f460 con 0x7f4068103000 2024-03-21T10:44:37.791 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:37.793+0000 7f4064c00700 1 -- 172.21.15.153:0/2640913725 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4058017610 con 0x7f4068103000 2024-03-21T10:44:37.791 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:37.793+0000 7f406f000700 1 -- 172.21.15.153:0/2640913725 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4068198af0 con 0x7f4068103000 2024-03-21T10:44:37.791 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:37.793+0000 7f406f000700 1 -- 172.21.15.153:0/2640913725 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4068198fe0 con 0x7f4068103000 2024-03-21T10:44:37.792 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:37.794+0000 7f406f000700 1 -- 172.21.15.153:0/2640913725 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f406810b3a0 con 0x7f4068103000 2024-03-21T10:44:37.792 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:37.795+0000 7f4064c00700 1 -- 172.21.15.153:0/2640913725 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f405800f5d0 con 0x7f4068103000 2024-03-21T10:44:37.793 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:37.796+0000 7f4064c00700 1 --2- 172.21.15.153:0/2640913725 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f404c06e460 0x7f404c070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:44:37.793 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:37.796+0000 7f4064c00700 1 -- 172.21.15.153:0/2640913725 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f405808fb10 con 0x7f4068103000 2024-03-21T10:44:37.794 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:37.796+0000 7f4067400700 1 --2- 172.21.15.153:0/2640913725 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f404c06e460 0x7f404c070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:44:37.794 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:37.797+0000 7f4067400700 1 --2- 172.21.15.153:0/2640913725 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f404c06e460 0x7f404c070920 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f40680ff0a0 tx=0x7f4054005d50 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:44:37.798 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:37.801+0000 7f4064c00700 1 -- 172.21.15.153:0/2640913725 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f405805c6e0 con 0x7f4068103000 2024-03-21T10:44:38.109 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:38.112+0000 7f406f000700 1 -- 172.21.15.153:0/2640913725 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f4068199e10 con 0x7f4068103000 2024-03-21T10:44:38.110 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:38.113+0000 7f4064c00700 1 -- 172.21.15.153:0/2640913725 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f4068199e10 con 0x7f4068103000 2024-03-21T10:44:38.111 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:44:38.111 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": { 2024-03-21T10:44:38.111 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:44:38.111 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:44:38.111 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": { 2024-03-21T10:44:38.111 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:44:38.111 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:44:38.111 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": { 2024-03-21T10:44:38.111 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 6 2024-03-21T10:44:38.111 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:44:38.111 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": { 2024-03-21T10:44:38.111 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 4 2024-03-21T10:44:38.111 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:44:38.112 INFO:teuthology.orchestra.run.smithi153.stdout: "overall": { 2024-03-21T10:44:38.112 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 14 2024-03-21T10:44:38.112 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-03-21T10:44:38.112 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:44:38.113 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:38.116+0000 7f406f000700 1 -- 172.21.15.153:0/2640913725 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f404c06e460 msgr2=0x7f404c070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:44:38.113 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:38.116+0000 7f406f000700 1 --2- 172.21.15.153:0/2640913725 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f404c06e460 0x7f404c070920 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f40680ff0a0 tx=0x7f4054005d50 comp rx=0 tx=0).stop 2024-03-21T10:44:38.113 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:38.116+0000 7f406f000700 1 -- 172.21.15.153:0/2640913725 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4068103000 msgr2=0x7f406819e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:44:38.113 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:38.116+0000 7f406f000700 1 --2- 172.21.15.153:0/2640913725 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4068103000 0x7f406819e7f0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f405800b5c0 tx=0x7f405800be00 comp rx=0 tx=0).stop 2024-03-21T10:44:38.114 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:38.116+0000 7f406f000700 1 -- 172.21.15.153:0/2640913725 shutdown_connections 2024-03-21T10:44:38.114 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:38.116+0000 7f406f000700 1 --2- 172.21.15.153:0/2640913725 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f404c06e460 0x7f404c070920 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:44:38.114 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:38.116+0000 7f406f000700 1 --2- 172.21.15.153:0/2640913725 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4068103000 0x7f406819e7f0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:44:38.114 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:38.116+0000 7f406f000700 1 --2- 172.21.15.153:0/2640913725 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f40681039b0 0x7f406819ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:44:38.114 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:38.116+0000 7f406f000700 1 -- 172.21.15.153:0/2640913725 >> 172.21.15.153:0/2640913725 conn(0x7f40680fe8d0 msgr2=0x7f40680ffee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:44:38.114 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:38.116+0000 7f406f000700 1 -- 172.21.15.153:0/2640913725 shutdown_connections 2024-03-21T10:44:38.114 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:38.116+0000 7f406f000700 1 -- 172.21.15.153:0/2640913725 wait complete. 2024-03-21T10:44:38.266 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:38.268+0000 7f327a800700 1 -- 172.21.15.153:0/224026723 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3274068950 msgr2=0x7f3274068dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:44:38.266 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:38.268+0000 7f327a800700 1 --2- 172.21.15.153:0/224026723 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3274068950 0x7f3274068dd0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f325c00b0a0 tx=0x7f325c00b3b0 comp rx=0 tx=0).stop 2024-03-21T10:44:38.266 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:38.268+0000 7f327a800700 1 -- 172.21.15.153:0/224026723 shutdown_connections 2024-03-21T10:44:38.266 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:38.268+0000 7f327a800700 1 --2- 172.21.15.153:0/224026723 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3274068950 0x7f3274068dd0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:44:38.266 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:38.268+0000 7f327a800700 1 --2- 172.21.15.153:0/224026723 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3274105b50 0x7f3274068410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:44:38.266 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:38.268+0000 7f327a800700 1 -- 172.21.15.153:0/224026723 >> 172.21.15.153:0/224026723 conn(0x7f32740755a0 msgr2=0x7f32740759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:44:38.267 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:38.268+0000 7f327a800700 1 -- 172.21.15.153:0/224026723 shutdown_connections 2024-03-21T10:44:38.267 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:38.268+0000 7f327a800700 1 -- 172.21.15.153:0/224026723 wait complete. 2024-03-21T10:44:38.267 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:38.270+0000 7f327a800700 1 Processor -- start 2024-03-21T10:44:38.267 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:38.270+0000 7f327a800700 1 -- start start 2024-03-21T10:44:38.268 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:38.270+0000 7f327a800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3274068950 0x7f32741a2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:44:38.268 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:38.270+0000 7f327a800700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3274105b50 0x7f32741a3010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:44:38.269 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:38.270+0000 7f327a800700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32741a36a0 con 0x7f3274068950 2024-03-21T10:44:38.269 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:38.270+0000 7f327a800700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f327419cb50 con 0x7f3274105b50 2024-03-21T10:44:38.269 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:38.271+0000 7f3273400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3274068950 0x7f32741a2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:44:38.269 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:38.271+0000 7f3273400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3274068950 0x7f32741a2ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:41994/0 (socket says 172.21.15.153:41994) 2024-03-21T10:44:38.269 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:38.271+0000 7f3273400700 1 -- 172.21.15.153:0/2991954085 learned_addr learned my addr 172.21.15.153:0/2991954085 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:44:38.269 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:38.271+0000 7f3273400700 1 -- 172.21.15.153:0/2991954085 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3274105b50 msgr2=0x7f32741a3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:44:38.269 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:38.271+0000 7f3273400700 1 --2- 172.21.15.153:0/2991954085 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3274105b50 0x7f32741a3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:44:38.269 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:38.271+0000 7f3273400700 1 -- 172.21.15.153:0/2991954085 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f325c009d00 con 0x7f3274068950 2024-03-21T10:44:38.269 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:38.271+0000 7f3273400700 1 --2- 172.21.15.153:0/2991954085 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3274068950 0x7f32741a2ad0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f326400c8f0 tx=0x7f326400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:44:38.270 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:38.272+0000 7f3270c00700 1 -- 172.21.15.153:0/2991954085 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3264007a70 con 0x7f3274068950 2024-03-21T10:44:38.270 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:38.272+0000 7f3270c00700 1 -- 172.21.15.153:0/2991954085 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3264007bd0 con 0x7f3274068950 2024-03-21T10:44:38.270 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:38.272+0000 7f327a800700 1 -- 172.21.15.153:0/2991954085 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f327419ce30 con 0x7f3274068950 2024-03-21T10:44:38.270 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:38.272+0000 7f3270c00700 1 -- 172.21.15.153:0/2991954085 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3264018630 con 0x7f3274068950 2024-03-21T10:44:38.270 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:38.272+0000 7f327a800700 1 -- 172.21.15.153:0/2991954085 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f327419d380 con 0x7f3274068950 2024-03-21T10:44:38.270 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:38.273+0000 7f327a800700 1 -- 172.21.15.153:0/2991954085 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f327404ea90 con 0x7f3274068950 2024-03-21T10:44:38.271 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:38.274+0000 7f3270c00700 1 -- 172.21.15.153:0/2991954085 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f326400f450 con 0x7f3274068950 2024-03-21T10:44:38.272 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:38.275+0000 7f3270c00700 1 --2- 172.21.15.153:0/2991954085 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f325806e460 0x7f3258070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:44:38.272 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:38.275+0000 7f3270c00700 1 -- 172.21.15.153:0/2991954085 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f326408e510 con 0x7f3274068950 2024-03-21T10:44:38.272 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:38.275+0000 7f326aa00700 1 --2- 172.21.15.153:0/2991954085 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f325806e460 0x7f3258070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:44:38.273 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:38.276+0000 7f326aa00700 1 --2- 172.21.15.153:0/2991954085 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f325806e460 0x7f3258070920 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f325c009480 tx=0x7f325c006000 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:44:38.277 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:38.280+0000 7f3270c00700 1 -- 172.21.15.153:0/2991954085 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f326405b100 con 0x7f3274068950 2024-03-21T10:44:38.302 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:44:38 smithi153 ceph-mon[113733]: from='client.35742 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:44:38.302 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:44:38 smithi153 ceph-mon[113733]: from='client.35746 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:44:38.447 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:44:38 smithi174 ceph-mon[88004]: from='client.35742 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:44:38.447 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:44:38 smithi174 ceph-mon[88004]: from='client.35746 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:44:38.532 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:38.535+0000 7f327a800700 1 -- 172.21.15.153:0/2991954085 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f327419d7c0 con 0x7f3274068950 2024-03-21T10:44:38.533 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:38.536+0000 7f3270c00700 1 -- 172.21.15.153:0/2991954085 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1876 (secure 0 0 0) 0x7f326405ac90 con 0x7f3274068950 2024-03-21T10:44:38.534 INFO:teuthology.orchestra.run.smithi153.stdout:e32 2024-03-21T10:44:38.534 INFO:teuthology.orchestra.run.smithi153.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-21T10:44:38.534 INFO:teuthology.orchestra.run.smithi153.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:44:38.534 INFO:teuthology.orchestra.run.smithi153.stdout:legacy client fscid: 1 2024-03-21T10:44:38.534 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:44:38.535 INFO:teuthology.orchestra.run.smithi153.stdout:Filesystem 'cephfs' (1) 2024-03-21T10:44:38.535 INFO:teuthology.orchestra.run.smithi153.stdout:fs_name cephfs 2024-03-21T10:44:38.535 INFO:teuthology.orchestra.run.smithi153.stdout:epoch 23 2024-03-21T10:44:38.535 INFO:teuthology.orchestra.run.smithi153.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-21T10:44:38.535 INFO:teuthology.orchestra.run.smithi153.stdout:created 2024-03-21T10:04:27.086576+0000 2024-03-21T10:44:38.535 INFO:teuthology.orchestra.run.smithi153.stdout:modified 2024-03-21T10:19:05.022092+0000 2024-03-21T10:44:38.535 INFO:teuthology.orchestra.run.smithi153.stdout:tableserver 0 2024-03-21T10:44:38.535 INFO:teuthology.orchestra.run.smithi153.stdout:root 0 2024-03-21T10:44:38.535 INFO:teuthology.orchestra.run.smithi153.stdout:session_timeout 60 2024-03-21T10:44:38.535 INFO:teuthology.orchestra.run.smithi153.stdout:session_autoclose 300 2024-03-21T10:44:38.535 INFO:teuthology.orchestra.run.smithi153.stdout:max_file_size 1099511627776 2024-03-21T10:44:38.535 INFO:teuthology.orchestra.run.smithi153.stdout:max_xattr_size 65536 2024-03-21T10:44:38.535 INFO:teuthology.orchestra.run.smithi153.stdout:required_client_features {} 2024-03-21T10:44:38.535 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure 0 2024-03-21T10:44:38.535 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure_osd_epoch 94 2024-03-21T10:44:38.535 INFO:teuthology.orchestra.run.smithi153.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:44:38.535 INFO:teuthology.orchestra.run.smithi153.stdout:max_mds 2 2024-03-21T10:44:38.536 INFO:teuthology.orchestra.run.smithi153.stdout:in 0,1 2024-03-21T10:44:38.536 INFO:teuthology.orchestra.run.smithi153.stdout:up {} 2024-03-21T10:44:38.536 INFO:teuthology.orchestra.run.smithi153.stdout:failed 0,1 2024-03-21T10:44:38.536 INFO:teuthology.orchestra.run.smithi153.stdout:damaged 2024-03-21T10:44:38.536 INFO:teuthology.orchestra.run.smithi153.stdout:stopped 2024-03-21T10:44:38.536 INFO:teuthology.orchestra.run.smithi153.stdout:data_pools [3] 2024-03-21T10:44:38.536 INFO:teuthology.orchestra.run.smithi153.stdout:metadata_pool 2 2024-03-21T10:44:38.536 INFO:teuthology.orchestra.run.smithi153.stdout:inline_data enabled 2024-03-21T10:44:38.536 INFO:teuthology.orchestra.run.smithi153.stdout:balancer 2024-03-21T10:44:38.536 INFO:teuthology.orchestra.run.smithi153.stdout:bal_rank_mask -1 2024-03-21T10:44:38.536 INFO:teuthology.orchestra.run.smithi153.stdout:standby_count_wanted 1 2024-03-21T10:44:38.536 INFO:teuthology.orchestra.run.smithi153.stdout:qdb_cluster leader: 0 members: 2024-03-21T10:44:38.536 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:44:38.536 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:44:38.536 INFO:teuthology.orchestra.run.smithi153.stdout:Standby daemons: 2024-03-21T10:44:38.536 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:44:38.537 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.zipckr{-1:34442} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6826/1103846295,v1:172.21.15.153:6827/1103846295] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:44:38.537 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.iuyluj{-1:34474} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6828/1738630832,v1:172.21.15.153:6829/1738630832] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:44:38.537 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.yyimst{-1:44313} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.174:6824/4127786797,v1:172.21.15.174:6825/4127786797] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:44:38.537 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.nhozag{-1:44687} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.174:6826/2767517845,v1:172.21.15.174:6827/2767517845] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:44:38.537 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:38.540+0000 7f327a800700 1 -- 172.21.15.153:0/2991954085 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f325806e460 msgr2=0x7f3258070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:44:38.538 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:38.540+0000 7f327a800700 1 --2- 172.21.15.153:0/2991954085 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f325806e460 0x7f3258070920 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f325c009480 tx=0x7f325c006000 comp rx=0 tx=0).stop 2024-03-21T10:44:38.538 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:38.540+0000 7f327a800700 1 -- 172.21.15.153:0/2991954085 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3274068950 msgr2=0x7f32741a2ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:44:38.538 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:38.540+0000 7f327a800700 1 --2- 172.21.15.153:0/2991954085 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3274068950 0x7f32741a2ad0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f326400c8f0 tx=0x7f326400ccb0 comp rx=0 tx=0).stop 2024-03-21T10:44:38.538 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:38.540+0000 7f327a800700 1 -- 172.21.15.153:0/2991954085 shutdown_connections 2024-03-21T10:44:38.538 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:38.540+0000 7f327a800700 1 --2- 172.21.15.153:0/2991954085 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f325806e460 0x7f3258070920 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:44:38.538 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:38.540+0000 7f327a800700 1 --2- 172.21.15.153:0/2991954085 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3274068950 0x7f32741a2ad0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:44:38.538 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:38.540+0000 7f327a800700 1 --2- 172.21.15.153:0/2991954085 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3274105b50 0x7f32741a3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:44:38.538 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:38.540+0000 7f327a800700 1 -- 172.21.15.153:0/2991954085 >> 172.21.15.153:0/2991954085 conn(0x7f32740755a0 msgr2=0x7f3274101510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:44:38.538 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:38.540+0000 7f327a800700 1 -- 172.21.15.153:0/2991954085 shutdown_connections 2024-03-21T10:44:38.538 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:38.540+0000 7f327a800700 1 -- 172.21.15.153:0/2991954085 wait complete. 2024-03-21T10:44:38.539 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 32 2024-03-21T10:44:38.695 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:38.697+0000 7fded1000700 1 -- 172.21.15.153:0/1937275915 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdecc106fc0 msgr2=0x7fdecc107440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:44:38.695 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:38.697+0000 7fded1000700 1 --2- 172.21.15.153:0/1937275915 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdecc106fc0 0x7fdecc107440 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fdeb4009910 tx=0x7fdeb4009c20 comp rx=0 tx=0).stop 2024-03-21T10:44:38.695 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:38.697+0000 7fded1000700 1 -- 172.21.15.153:0/1937275915 shutdown_connections 2024-03-21T10:44:38.695 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:38.697+0000 7fded1000700 1 --2- 172.21.15.153:0/1937275915 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdecc106fc0 0x7fdecc107440 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:44:38.695 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:38.697+0000 7fded1000700 1 --2- 172.21.15.153:0/1937275915 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdecc10d070 0x7fdecc10d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:44:38.695 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:38.697+0000 7fded1000700 1 -- 172.21.15.153:0/1937275915 >> 172.21.15.153:0/1937275915 conn(0x7fdecc0767e0 msgr2=0x7fdecc078c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:44:38.695 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:38.697+0000 7fded1000700 1 -- 172.21.15.153:0/1937275915 shutdown_connections 2024-03-21T10:44:38.696 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:38.698+0000 7fded1000700 1 -- 172.21.15.153:0/1937275915 wait complete. 2024-03-21T10:44:38.697 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:38.698+0000 7fded1000700 1 Processor -- start 2024-03-21T10:44:38.697 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:38.699+0000 7fded1000700 1 -- start start 2024-03-21T10:44:38.697 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:38.699+0000 7fded1000700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdecc106fc0 0x7fdecc1a28c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:44:38.697 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:38.699+0000 7fded1000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdecc10d070 0x7fdecc1a2e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:44:38.697 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:38.699+0000 7fded1000700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdecc1a3490 con 0x7fdecc10d070 2024-03-21T10:44:38.697 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:38.699+0000 7fded1000700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdecc19c940 con 0x7fdecc106fc0 2024-03-21T10:44:38.697 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:38.699+0000 7fdec3e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdecc10d070 0x7fdecc1a2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:44:38.697 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:38.699+0000 7fdec3e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdecc10d070 0x7fdecc1a2e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42012/0 (socket says 172.21.15.153:42012) 2024-03-21T10:44:38.697 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:38.699+0000 7fdec3e00700 1 -- 172.21.15.153:0/2959661075 learned_addr learned my addr 172.21.15.153:0/2959661075 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:44:38.698 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:38.700+0000 7fdecb400700 1 --2- 172.21.15.153:0/2959661075 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdecc106fc0 0x7fdecc1a28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:44:38.698 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:38.700+0000 7fdec3e00700 1 -- 172.21.15.153:0/2959661075 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdecc106fc0 msgr2=0x7fdecc1a28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:44:38.698 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:38.700+0000 7fdec3e00700 1 --2- 172.21.15.153:0/2959661075 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdecc106fc0 0x7fdecc1a28c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:44:38.698 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:38.700+0000 7fdec3e00700 1 -- 172.21.15.153:0/2959661075 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdeb40095f0 con 0x7fdecc10d070 2024-03-21T10:44:38.698 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:38.700+0000 7fdec3e00700 1 --2- 172.21.15.153:0/2959661075 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdecc10d070 0x7fdecc1a2e00 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fdeb400b5c0 tx=0x7fdeb400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:44:38.699 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:38.700+0000 7fdec8c00700 1 -- 172.21.15.153:0/2959661075 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdeb401d070 con 0x7fdecc10d070 2024-03-21T10:44:38.699 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:38.700+0000 7fdec8c00700 1 -- 172.21.15.153:0/2959661075 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdeb4005e00 con 0x7fdecc10d070 2024-03-21T10:44:38.699 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:38.700+0000 7fded1000700 1 -- 172.21.15.153:0/2959661075 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdecc19cbc0 con 0x7fdecc10d070 2024-03-21T10:44:38.699 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:38.701+0000 7fdec8c00700 1 -- 172.21.15.153:0/2959661075 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdeb4017400 con 0x7fdecc10d070 2024-03-21T10:44:38.699 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:38.701+0000 7fded1000700 1 -- 172.21.15.153:0/2959661075 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdecc19d130 con 0x7fdecc10d070 2024-03-21T10:44:38.699 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:38.702+0000 7fded1000700 1 -- 172.21.15.153:0/2959661075 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdecc04f2e0 con 0x7fdecc10d070 2024-03-21T10:44:38.700 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:38.703+0000 7fdec8c00700 1 -- 172.21.15.153:0/2959661075 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fdeb400f460 con 0x7fdecc10d070 2024-03-21T10:44:38.701 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:38.704+0000 7fdec8c00700 1 --2- 172.21.15.153:0/2959661075 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fdeb006e460 0x7fdeb0070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:44:38.701 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:38.704+0000 7fdec8c00700 1 -- 172.21.15.153:0/2959661075 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fdeb408f930 con 0x7fdecc10d070 2024-03-21T10:44:38.701 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:38.704+0000 7fdecb400700 1 --2- 172.21.15.153:0/2959661075 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fdeb006e460 0x7fdeb0070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:44:38.702 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:38.705+0000 7fdecb400700 1 --2- 172.21.15.153:0/2959661075 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fdeb006e460 0x7fdeb0070920 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fdebc005950 tx=0x7fdebc0058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:44:38.705 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:38.708+0000 7fdec8c00700 1 -- 172.21.15.153:0/2959661075 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fdeb405c5a0 con 0x7fdecc10d070 2024-03-21T10:44:38.954 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:38.956+0000 7fded1000700 1 -- 172.21.15.153:0/2959661075 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdecc19d520 con 0x7fdeb006e460 2024-03-21T10:44:38.955 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:38.958+0000 7fdec8c00700 1 -- 172.21.15.153:0/2959661075 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fdecc19d520 con 0x7fdeb006e460 2024-03-21T10:44:38.956 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:44:38.956 INFO:teuthology.orchestra.run.smithi153.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232", 2024-03-21T10:44:38.956 INFO:teuthology.orchestra.run.smithi153.stdout: "in_progress": true, 2024-03-21T10:44:38.956 INFO:teuthology.orchestra.run.smithi153.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-21T10:44:38.956 INFO:teuthology.orchestra.run.smithi153.stdout: "services_complete": [ 2024-03-21T10:44:38.956 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr", 2024-03-21T10:44:38.956 INFO:teuthology.orchestra.run.smithi153.stdout: "mon", 2024-03-21T10:44:38.956 INFO:teuthology.orchestra.run.smithi153.stdout: "mds", 2024-03-21T10:44:38.956 INFO:teuthology.orchestra.run.smithi153.stdout: "crash", 2024-03-21T10:44:38.956 INFO:teuthology.orchestra.run.smithi153.stdout: "osd" 2024-03-21T10:44:38.957 INFO:teuthology.orchestra.run.smithi153.stdout: ], 2024-03-21T10:44:38.957 INFO:teuthology.orchestra.run.smithi153.stdout: "progress": "16/23 daemons upgraded", 2024-03-21T10:44:38.957 INFO:teuthology.orchestra.run.smithi153.stdout: "message": "Currently upgrading mds daemons", 2024-03-21T10:44:38.957 INFO:teuthology.orchestra.run.smithi153.stdout: "is_paused": false 2024-03-21T10:44:38.957 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:44:38.959 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:38.961+0000 7fded1000700 1 -- 172.21.15.153:0/2959661075 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fdeb006e460 msgr2=0x7fdeb0070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:44:38.959 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:38.961+0000 7fded1000700 1 --2- 172.21.15.153:0/2959661075 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fdeb006e460 0x7fdeb0070920 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fdebc005950 tx=0x7fdebc0058e0 comp rx=0 tx=0).stop 2024-03-21T10:44:38.959 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:38.961+0000 7fded1000700 1 -- 172.21.15.153:0/2959661075 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdecc10d070 msgr2=0x7fdecc1a2e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:44:38.959 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:38.961+0000 7fded1000700 1 --2- 172.21.15.153:0/2959661075 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdecc10d070 0x7fdecc1a2e00 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fdeb400b5c0 tx=0x7fdeb400be20 comp rx=0 tx=0).stop 2024-03-21T10:44:38.959 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:38.961+0000 7fded1000700 1 -- 172.21.15.153:0/2959661075 shutdown_connections 2024-03-21T10:44:38.959 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:38.961+0000 7fded1000700 1 --2- 172.21.15.153:0/2959661075 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdecc106fc0 0x7fdecc1a28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:44:38.959 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:38.961+0000 7fded1000700 1 --2- 172.21.15.153:0/2959661075 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fdeb006e460 0x7fdeb0070920 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:44:38.959 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:38.961+0000 7fded1000700 1 --2- 172.21.15.153:0/2959661075 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdecc10d070 0x7fdecc1a2e00 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:44:38.959 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:38.961+0000 7fded1000700 1 -- 172.21.15.153:0/2959661075 >> 172.21.15.153:0/2959661075 conn(0x7fdecc0767e0 msgr2=0x7fdecc077e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:44:38.959 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:38.962+0000 7fded1000700 1 -- 172.21.15.153:0/2959661075 shutdown_connections 2024-03-21T10:44:38.959 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:38.962+0000 7fded1000700 1 -- 172.21.15.153:0/2959661075 wait complete. 2024-03-21T10:44:39.118 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:39.121+0000 7f7d44c00700 1 -- 172.21.15.153:0/2770413440 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7d401071d0 msgr2=0x7f7d40107650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:44:39.119 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:39.121+0000 7f7d44c00700 1 --2- 172.21.15.153:0/2770413440 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7d401071d0 0x7f7d40107650 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f7d28009910 tx=0x7f7d28009c20 comp rx=0 tx=0).stop 2024-03-21T10:44:39.119 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:39.121+0000 7f7d44c00700 1 -- 172.21.15.153:0/2770413440 shutdown_connections 2024-03-21T10:44:39.119 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:39.121+0000 7f7d44c00700 1 --2- 172.21.15.153:0/2770413440 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7d401071d0 0x7f7d40107650 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:44:39.119 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:39.121+0000 7f7d44c00700 1 --2- 172.21.15.153:0/2770413440 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7d4010d280 0x7f7d4010d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:44:39.119 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:39.121+0000 7f7d44c00700 1 -- 172.21.15.153:0/2770413440 >> 172.21.15.153:0/2770413440 conn(0x7f7d400769e0 msgr2=0x7f7d40078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:44:39.119 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:39.121+0000 7f7d44c00700 1 -- 172.21.15.153:0/2770413440 shutdown_connections 2024-03-21T10:44:39.119 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:39.121+0000 7f7d44c00700 1 -- 172.21.15.153:0/2770413440 wait complete. 2024-03-21T10:44:39.121 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:39.122+0000 7f7d44c00700 1 Processor -- start 2024-03-21T10:44:39.121 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:39.122+0000 7f7d44c00700 1 -- start start 2024-03-21T10:44:39.121 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:39.123+0000 7f7d44c00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7d401071d0 0x7f7d4019ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:44:39.121 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:39.123+0000 7f7d44c00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7d4010d280 0x7f7d4019d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:44:39.121 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:39.123+0000 7f7d44c00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d4019d880 con 0x7f7d401071d0 2024-03-21T10:44:39.121 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:39.123+0000 7f7d44c00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d401a05b0 con 0x7f7d4010d280 2024-03-21T10:44:39.121 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:39.123+0000 7f7d3de00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7d401071d0 0x7f7d4019ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:44:39.121 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:39.123+0000 7f7d3de00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7d401071d0 0x7f7d4019ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42038/0 (socket says 172.21.15.153:42038) 2024-03-21T10:44:39.121 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:39.123+0000 7f7d3de00700 1 -- 172.21.15.153:0/2981989637 learned_addr learned my addr 172.21.15.153:0/2981989637 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:44:39.121 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:39.123+0000 7f7d35400700 1 --2- 172.21.15.153:0/2981989637 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7d4010d280 0x7f7d4019d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:44:39.121 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:39.124+0000 7f7d3de00700 1 -- 172.21.15.153:0/2981989637 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7d4010d280 msgr2=0x7f7d4019d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:44:39.121 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:39.124+0000 7f7d3de00700 1 --2- 172.21.15.153:0/2981989637 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7d4010d280 0x7f7d4019d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:44:39.122 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:39.124+0000 7f7d3de00700 1 -- 172.21.15.153:0/2981989637 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d280095f0 con 0x7f7d401071d0 2024-03-21T10:44:39.122 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:39.124+0000 7f7d35400700 1 --2- 172.21.15.153:0/2981989637 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7d4010d280 0x7f7d4019d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:44:39.123 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:39.124+0000 7f7d3de00700 1 --2- 172.21.15.153:0/2981989637 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7d401071d0 0x7f7d4019ccb0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f7d300060b0 tx=0x7f7d3000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:44:39.123 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:39.124+0000 7f7d37400700 1 -- 172.21.15.153:0/2981989637 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d3000dda0 con 0x7f7d401071d0 2024-03-21T10:44:39.123 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:39.124+0000 7f7d37400700 1 -- 172.21.15.153:0/2981989637 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7d3000f040 con 0x7f7d401071d0 2024-03-21T10:44:39.123 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:39.124+0000 7f7d44c00700 1 -- 172.21.15.153:0/2981989637 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d401a0890 con 0x7f7d401071d0 2024-03-21T10:44:39.123 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:39.124+0000 7f7d37400700 1 -- 172.21.15.153:0/2981989637 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d30014790 con 0x7f7d401071d0 2024-03-21T10:44:39.123 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:39.124+0000 7f7d44c00700 1 -- 172.21.15.153:0/2981989637 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d401a0e60 con 0x7f7d401071d0 2024-03-21T10:44:39.123 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:39.125+0000 7f7d44c00700 1 -- 172.21.15.153:0/2981989637 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d4010f6c0 con 0x7f7d401071d0 2024-03-21T10:44:39.123 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:39.126+0000 7f7d37400700 1 -- 172.21.15.153:0/2981989637 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f7d30014980 con 0x7f7d401071d0 2024-03-21T10:44:39.124 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:39.127+0000 7f7d37400700 1 --2- 172.21.15.153:0/2981989637 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f7d2406e460 0x7f7d24070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:44:39.125 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:39.127+0000 7f7d37400700 1 -- 172.21.15.153:0/2981989637 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f7d3008dfe0 con 0x7f7d401071d0 2024-03-21T10:44:39.125 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:39.128+0000 7f7d35400700 1 --2- 172.21.15.153:0/2981989637 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f7d2406e460 0x7f7d24070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:44:39.125 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:39.128+0000 7f7d35400700 1 --2- 172.21.15.153:0/2981989637 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f7d2406e460 0x7f7d24070920 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f7d28006010 tx=0x7f7d2800b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:44:39.129 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:39.132+0000 7f7d37400700 1 -- 172.21.15.153:0/2981989637 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7d300570d0 con 0x7f7d401071d0 2024-03-21T10:44:39.302 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:44:39 smithi153 ceph-mon[113733]: from='client.35750 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:44:39.302 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:44:39 smithi153 ceph-mon[113733]: pgmap v985: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:44:39.302 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:44:39 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/2640913725' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:44:39.302 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:44:39 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/2991954085' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:44:39.349 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:44:39 smithi174 ceph-mon[88004]: from='client.35750 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:44:39.349 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:44:39 smithi174 ceph-mon[88004]: pgmap v985: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:44:39.349 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:44:39 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/2640913725' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:44:39.349 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:44:39 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/2991954085' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:44:39.429 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:39.432+0000 7f7d44c00700 1 -- 172.21.15.153:0/2981989637 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f7d401a10e0 con 0x7f7d401071d0 2024-03-21T10:44:39.430 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:39.433+0000 7f7d37400700 1 -- 172.21.15.153:0/2981989637 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4493 (secure 0 0 0) 0x7f7d3005a790 con 0x7f7d401071d0 2024-03-21T10:44:39.430 INFO:teuthology.orchestra.run.smithi153.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-03-21T10:44:39.430 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-03-21T10:44:39.430 INFO:teuthology.orchestra.run.smithi153.stdout: host smithi174 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi174/config 2024-03-21T10:44:39.431 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/podman: timeout after 895 seconds 2024-03-21T10:44:39.431 INFO:teuthology.orchestra.run.smithi153.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 -e NODE_NAME=smithi174 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/run/ceph:z -v /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpd910r3_i:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 inventory --format=json-pretty --filter-for-batch 2024-03-21T10:44:39.431 INFO:teuthology.orchestra.run.smithi153.stdout:Traceback (most recent call last): 2024-03-21T10:44:39.431 INFO:teuthology.orchestra.run.smithi153.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-03-21T10:44:39.431 INFO:teuthology.orchestra.run.smithi153.stdout: return _run_code(code, main_globals, None, 2024-03-21T10:44:39.431 INFO:teuthology.orchestra.run.smithi153.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-03-21T10:44:39.431 INFO:teuthology.orchestra.run.smithi153.stdout: exec(code, run_globals) 2024-03-21T10:44:39.431 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 5606, in 2024-03-21T10:44:39.431 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 5594, in main 2024-03-21T10:44:39.431 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 406, in _infer_config 2024-03-21T10:44:39.431 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 321, in _infer_fsid 2024-03-21T10:44:39.431 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 434, in _infer_image 2024-03-21T10:44:39.431 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 308, in _validate_fsid 2024-03-21T10:44:39.431 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 3357, in command_ceph_volume 2024-03-21T10:44:39.432 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-03-21T10:44:39.432 INFO:teuthology.orchestra.run.smithi153.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 -e NODE_NAME=smithi174 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/run/ceph:z -v /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpd910r3_i:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 inventory --format=json-pretty --filter-for-batch 2024-03-21T10:44:39.432 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-21T10:44:39.432 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is degraded 2024-03-21T10:44:39.432 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-21T10:44:39.432 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-21T10:44:39.432 INFO:teuthology.orchestra.run.smithi153.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-21T10:44:39.432 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-21T10:44:39.433 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:39.435+0000 7f7d44c00700 1 -- 172.21.15.153:0/2981989637 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f7d2406e460 msgr2=0x7f7d24070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:44:39.433 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:39.435+0000 7f7d44c00700 1 --2- 172.21.15.153:0/2981989637 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f7d2406e460 0x7f7d24070920 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f7d28006010 tx=0x7f7d2800b540 comp rx=0 tx=0).stop 2024-03-21T10:44:39.433 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:39.435+0000 7f7d44c00700 1 -- 172.21.15.153:0/2981989637 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7d401071d0 msgr2=0x7f7d4019ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:44:39.433 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:39.435+0000 7f7d44c00700 1 --2- 172.21.15.153:0/2981989637 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7d401071d0 0x7f7d4019ccb0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f7d300060b0 tx=0x7f7d3000d3b0 comp rx=0 tx=0).stop 2024-03-21T10:44:39.433 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:39.436+0000 7f7d44c00700 1 -- 172.21.15.153:0/2981989637 shutdown_connections 2024-03-21T10:44:39.434 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:39.436+0000 7f7d44c00700 1 --2- 172.21.15.153:0/2981989637 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f7d2406e460 0x7f7d24070920 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:44:39.434 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:39.436+0000 7f7d44c00700 1 --2- 172.21.15.153:0/2981989637 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7d401071d0 0x7f7d4019ccb0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:44:39.434 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:39.436+0000 7f7d44c00700 1 --2- 172.21.15.153:0/2981989637 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7d4010d280 0x7f7d4019d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:44:39.434 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:39.436+0000 7f7d44c00700 1 -- 172.21.15.153:0/2981989637 >> 172.21.15.153:0/2981989637 conn(0x7f7d400769e0 msgr2=0x7f7d40078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:44:39.434 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:39.436+0000 7f7d44c00700 1 -- 172.21.15.153:0/2981989637 shutdown_connections 2024-03-21T10:44:39.434 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:44:39.436+0000 7f7d44c00700 1 -- 172.21.15.153:0/2981989637 wait complete. 2024-03-21T10:44:40.302 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:44:40 smithi153 ceph-mon[113733]: from='client.35762 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:44:40.302 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:44:40 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/2981989637' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:44:40.447 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:44:40 smithi174 ceph-mon[88004]: from='client.35762 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:44:40.447 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:44:40 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/2981989637' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:44:41.302 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:44:41 smithi153 ceph-mon[113733]: pgmap v986: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:44:41.447 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:44:41 smithi174 ceph-mon[88004]: pgmap v986: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:44:43.447 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:44:43 smithi174 ceph-mon[88004]: pgmap v987: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:44:43.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:44:43 smithi153 ceph-mon[113733]: pgmap v987: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:44:45.447 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:44:45 smithi174 ceph-mon[88004]: pgmap v988: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:44:45.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:44:45 smithi153 ceph-mon[113733]: pgmap v988: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:44:47.447 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:44:47 smithi174 ceph-mon[88004]: pgmap v989: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:44:47.447 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:44:47 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:44:47.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:44:47 smithi153 ceph-mon[113733]: pgmap v989: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:44:47.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:44:47 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:44:49.447 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:44:49 smithi174 ceph-mon[88004]: pgmap v990: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:44:49.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:44:49 smithi153 ceph-mon[113733]: pgmap v990: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:44:51.447 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:44:51 smithi174 ceph-mon[88004]: pgmap v991: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:44:51.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:44:51 smithi153 ceph-mon[113733]: pgmap v991: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:44:53.447 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:44:53 smithi174 ceph-mon[88004]: pgmap v992: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:44:53.554 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:44:53 smithi153 ceph-mon[113733]: pgmap v992: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:44:55.447 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:44:55 smithi174 ceph-mon[88004]: pgmap v993: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:44:55.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:44:55 smithi153 ceph-mon[113733]: pgmap v993: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:44:57.447 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:44:57 smithi174 ceph-mon[88004]: pgmap v994: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:44:57.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:44:57 smithi153 ceph-mon[113733]: pgmap v994: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:44:59.447 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:44:59 smithi174 ceph-mon[88004]: pgmap v995: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:44:59.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:44:59 smithi153 ceph-mon[113733]: pgmap v995: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:45:01.447 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:45:01 smithi174 ceph-mon[88004]: pgmap v996: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:45:01.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:45:01 smithi153 ceph-mon[113733]: pgmap v996: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:45:02.447 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:45:02 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:45:02.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:45:02 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:45:03.447 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:45:03 smithi174 ceph-mon[88004]: pgmap v997: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:45:03.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:45:03 smithi153 ceph-mon[113733]: pgmap v997: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:45:05.447 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:45:05 smithi174 ceph-mon[88004]: pgmap v998: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:45:05.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:45:05 smithi153 ceph-mon[113733]: pgmap v998: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:45:07.447 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:45:07 smithi174 ceph-mon[88004]: pgmap v999: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:45:07.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:45:07 smithi153 ceph-mon[113733]: pgmap v999: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:45:09.447 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:45:09 smithi174 ceph-mon[88004]: pgmap v1000: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:45:09.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:45:09 smithi153 ceph-mon[113733]: pgmap v1000: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:45:09.601 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:09.603+0000 7efea5800700 1 -- 172.21.15.153:0/1177543802 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efea010d280 msgr2=0x7efea010d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:45:09.601 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:09.603+0000 7efea5800700 1 --2- 172.21.15.153:0/1177543802 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efea010d280 0x7efea010d660 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7efe9000b0a0 tx=0x7efe9000b3b0 comp rx=0 tx=0).stop 2024-03-21T10:45:09.601 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:09.603+0000 7efea5800700 1 -- 172.21.15.153:0/1177543802 shutdown_connections 2024-03-21T10:45:09.601 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:09.603+0000 7efea5800700 1 --2- 172.21.15.153:0/1177543802 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efea01071d0 0x7efea0107650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:45:09.601 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:09.603+0000 7efea5800700 1 --2- 172.21.15.153:0/1177543802 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efea010d280 0x7efea010d660 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:45:09.601 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:09.603+0000 7efea5800700 1 -- 172.21.15.153:0/1177543802 >> 172.21.15.153:0/1177543802 conn(0x7efea00769e0 msgr2=0x7efea0078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:45:09.601 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:09.603+0000 7efea5800700 1 -- 172.21.15.153:0/1177543802 shutdown_connections 2024-03-21T10:45:09.601 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:09.603+0000 7efea5800700 1 -- 172.21.15.153:0/1177543802 wait complete. 2024-03-21T10:45:09.602 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:09.604+0000 7efea5800700 1 Processor -- start 2024-03-21T10:45:09.602 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:09.604+0000 7efea5800700 1 -- start start 2024-03-21T10:45:09.602 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:09.605+0000 7efea5800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efea01071d0 0x7efea019cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:45:09.602 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:09.605+0000 7efea5800700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efea010d280 0x7efea019d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:45:09.603 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:09.605+0000 7efea5800700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efea019d940 con 0x7efea01071d0 2024-03-21T10:45:09.603 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:09.605+0000 7efea5800700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efea01a16d0 con 0x7efea010d280 2024-03-21T10:45:09.603 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:09.605+0000 7efe9ea00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efea01071d0 0x7efea019cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:45:09.603 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:09.605+0000 7efe9ea00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efea01071d0 0x7efea019cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:46616/0 (socket says 172.21.15.153:46616) 2024-03-21T10:45:09.603 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:09.605+0000 7efe9ea00700 1 -- 172.21.15.153:0/281208271 learned_addr learned my addr 172.21.15.153:0/281208271 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:45:09.604 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:09.605+0000 7efe96000700 1 --2- 172.21.15.153:0/281208271 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efea010d280 0x7efea019d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:45:09.604 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:09.606+0000 7efe9ea00700 1 -- 172.21.15.153:0/281208271 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efea010d280 msgr2=0x7efea019d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:45:09.604 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:09.606+0000 7efe9ea00700 1 --2- 172.21.15.153:0/281208271 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efea010d280 0x7efea019d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:45:09.604 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:09.606+0000 7efe9ea00700 1 -- 172.21.15.153:0/281208271 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe90009d00 con 0x7efea01071d0 2024-03-21T10:45:09.604 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:09.606+0000 7efe96000700 1 --2- 172.21.15.153:0/281208271 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efea010d280 0x7efea019d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:45:09.605 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:09.606+0000 7efe9ea00700 1 --2- 172.21.15.153:0/281208271 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efea01071d0 0x7efea019cd20 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7efe9000b070 tx=0x7efe90003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:45:09.605 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:09.606+0000 7efe97e00700 1 -- 172.21.15.153:0/281208271 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe9000e040 con 0x7efea01071d0 2024-03-21T10:45:09.605 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:09.606+0000 7efe97e00700 1 -- 172.21.15.153:0/281208271 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efe90007c00 con 0x7efea01071d0 2024-03-21T10:45:09.605 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:09.606+0000 7efe97e00700 1 -- 172.21.15.153:0/281208271 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe9001d740 con 0x7efea01071d0 2024-03-21T10:45:09.605 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:09.606+0000 7efea5800700 1 -- 172.21.15.153:0/281208271 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efea01a1950 con 0x7efea01071d0 2024-03-21T10:45:09.605 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:09.607+0000 7efea5800700 1 -- 172.21.15.153:0/281208271 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efea01a1ec0 con 0x7efea01071d0 2024-03-21T10:45:09.605 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:09.608+0000 7efea5800700 1 -- 172.21.15.153:0/281208271 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efea004ea90 con 0x7efea01071d0 2024-03-21T10:45:09.606 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:09.609+0000 7efe97e00700 1 -- 172.21.15.153:0/281208271 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7efe90004040 con 0x7efea01071d0 2024-03-21T10:45:09.607 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:09.610+0000 7efe97e00700 1 --2- 172.21.15.153:0/281208271 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7efe8406e460 0x7efe84070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:45:09.607 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:09.610+0000 7efe97e00700 1 -- 172.21.15.153:0/281208271 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7efe9008fd70 con 0x7efea01071d0 2024-03-21T10:45:09.607 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:09.610+0000 7efe96000700 1 --2- 172.21.15.153:0/281208271 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7efe8406e460 0x7efe84070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:45:09.608 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:09.611+0000 7efe96000700 1 --2- 172.21.15.153:0/281208271 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7efe8406e460 0x7efe84070920 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7efe88005fd0 tx=0x7efe88005dc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:45:09.612 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:09.615+0000 7efe97e00700 1 -- 172.21.15.153:0/281208271 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7efe90058e10 con 0x7efea01071d0 2024-03-21T10:45:09.853 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:09.855+0000 7efea5800700 1 -- 172.21.15.153:0/281208271 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7efea01a2410 con 0x7efe8406e460 2024-03-21T10:45:09.855 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:09.858+0000 7efe97e00700 1 -- 172.21.15.153:0/281208271 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7efea01a2410 con 0x7efe8406e460 2024-03-21T10:45:09.858 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:09.861+0000 7efea5800700 1 -- 172.21.15.153:0/281208271 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7efe8406e460 msgr2=0x7efe84070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:45:09.858 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:09.861+0000 7efea5800700 1 --2- 172.21.15.153:0/281208271 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7efe8406e460 0x7efe84070920 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7efe88005fd0 tx=0x7efe88005dc0 comp rx=0 tx=0).stop 2024-03-21T10:45:09.858 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:09.861+0000 7efea5800700 1 -- 172.21.15.153:0/281208271 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efea01071d0 msgr2=0x7efea019cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:45:09.858 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:09.861+0000 7efea5800700 1 --2- 172.21.15.153:0/281208271 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efea01071d0 0x7efea019cd20 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7efe9000b070 tx=0x7efe90003ce0 comp rx=0 tx=0).stop 2024-03-21T10:45:09.859 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:09.861+0000 7efea5800700 1 -- 172.21.15.153:0/281208271 shutdown_connections 2024-03-21T10:45:09.859 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:09.861+0000 7efea5800700 1 --2- 172.21.15.153:0/281208271 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7efe8406e460 0x7efe84070920 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:45:09.859 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:09.861+0000 7efea5800700 1 --2- 172.21.15.153:0/281208271 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efea01071d0 0x7efea019cd20 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:45:09.859 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:09.861+0000 7efea5800700 1 --2- 172.21.15.153:0/281208271 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efea010d280 0x7efea019d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:45:09.859 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:09.861+0000 7efea5800700 1 -- 172.21.15.153:0/281208271 >> 172.21.15.153:0/281208271 conn(0x7efea00769e0 msgr2=0x7efea00783c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:45:09.859 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:09.861+0000 7efea5800700 1 -- 172.21.15.153:0/281208271 shutdown_connections 2024-03-21T10:45:09.859 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:09.861+0000 7efea5800700 1 -- 172.21.15.153:0/281208271 wait complete. 2024-03-21T10:45:09.875 INFO:teuthology.orchestra.run.smithi153.stdout:true 2024-03-21T10:45:10.015 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.017+0000 7f7056400700 1 -- 172.21.15.153:0/4013439254 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7050068670 msgr2=0x7f7050068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:45:10.015 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.017+0000 7f7056400700 1 --2- 172.21.15.153:0/4013439254 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7050068670 0x7f7050068a50 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f704000b0a0 tx=0x7f704000b3b0 comp rx=0 tx=0).stop 2024-03-21T10:45:10.015 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.018+0000 7f7056400700 1 -- 172.21.15.153:0/4013439254 shutdown_connections 2024-03-21T10:45:10.015 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.018+0000 7f7056400700 1 --2- 172.21.15.153:0/4013439254 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7050069020 0x7f7050105790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:45:10.015 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.018+0000 7f7056400700 1 --2- 172.21.15.153:0/4013439254 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7050068670 0x7f7050068a50 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:45:10.015 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.018+0000 7f7056400700 1 -- 172.21.15.153:0/4013439254 >> 172.21.15.153:0/4013439254 conn(0x7f70500755a0 msgr2=0x7f70500759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:45:10.015 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.018+0000 7f7056400700 1 -- 172.21.15.153:0/4013439254 shutdown_connections 2024-03-21T10:45:10.016 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.018+0000 7f7056400700 1 -- 172.21.15.153:0/4013439254 wait complete. 2024-03-21T10:45:10.016 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.019+0000 7f7056400700 1 Processor -- start 2024-03-21T10:45:10.016 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.019+0000 7f7056400700 1 -- start start 2024-03-21T10:45:10.016 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.019+0000 7f7056400700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7050068670 0x7f70500ffca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:45:10.017 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.019+0000 7f7056400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7050069020 0x7f70501001e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:45:10.017 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.019+0000 7f7056400700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7050100720 con 0x7f7050069020 2024-03-21T10:45:10.017 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.019+0000 7f7056400700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7050100860 con 0x7f7050068670 2024-03-21T10:45:10.018 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.020+0000 7f7046a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7050069020 0x7f70501001e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:45:10.018 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.020+0000 7f7046a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7050069020 0x7f70501001e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:46646/0 (socket says 172.21.15.153:46646) 2024-03-21T10:45:10.018 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.020+0000 7f7046a00700 1 -- 172.21.15.153:0/1302325374 learned_addr learned my addr 172.21.15.153:0/1302325374 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:45:10.018 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.020+0000 7f704f400700 1 --2- 172.21.15.153:0/1302325374 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7050068670 0x7f70500ffca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:45:10.018 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.020+0000 7f7046a00700 1 -- 172.21.15.153:0/1302325374 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7050068670 msgr2=0x7f70500ffca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:45:10.018 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.020+0000 7f7046a00700 1 --2- 172.21.15.153:0/1302325374 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7050068670 0x7f70500ffca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:45:10.018 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.020+0000 7f7046a00700 1 -- 172.21.15.153:0/1302325374 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7040009d00 con 0x7f7050069020 2024-03-21T10:45:10.019 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.020+0000 7f7046a00700 1 --2- 172.21.15.153:0/1302325374 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7050069020 0x7f70501001e0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f703800d900 tx=0x7f703800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:45:10.019 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.021+0000 7f704cc00700 1 -- 172.21.15.153:0/1302325374 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7038009900 con 0x7f7050069020 2024-03-21T10:45:10.019 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.021+0000 7f704cc00700 1 -- 172.21.15.153:0/1302325374 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7038010460 con 0x7f7050069020 2024-03-21T10:45:10.019 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.021+0000 7f704cc00700 1 -- 172.21.15.153:0/1302325374 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f703800f5f0 con 0x7f7050069020 2024-03-21T10:45:10.019 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.021+0000 7f7056400700 1 -- 172.21.15.153:0/1302325374 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70501a2910 con 0x7f7050069020 2024-03-21T10:45:10.019 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.021+0000 7f7056400700 1 -- 172.21.15.153:0/1302325374 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70501a2d20 con 0x7f7050069020 2024-03-21T10:45:10.019 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.022+0000 7f7056400700 1 -- 172.21.15.153:0/1302325374 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7050109130 con 0x7f7050069020 2024-03-21T10:45:10.020 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.023+0000 7f704cc00700 1 -- 172.21.15.153:0/1302325374 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f70380105d0 con 0x7f7050069020 2024-03-21T10:45:10.021 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.024+0000 7f704cc00700 1 --2- 172.21.15.153:0/1302325374 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f703406e460 0x7f7034070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:45:10.021 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.024+0000 7f704cc00700 1 -- 172.21.15.153:0/1302325374 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f703808e0d0 con 0x7f7050069020 2024-03-21T10:45:10.021 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.024+0000 7f704f400700 1 --2- 172.21.15.153:0/1302325374 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f703406e460 0x7f7034070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:45:10.022 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.025+0000 7f704f400700 1 --2- 172.21.15.153:0/1302325374 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f703406e460 0x7f7034070920 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f704000b830 tx=0x7f7040006000 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:45:10.026 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.029+0000 7f704cc00700 1 -- 172.21.15.153:0/1302325374 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7038057170 con 0x7f7050069020 2024-03-21T10:45:10.271 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.273+0000 7f7056400700 1 -- 172.21.15.153:0/1302325374 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f705002cda0 con 0x7f703406e460 2024-03-21T10:45:10.273 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.276+0000 7f704cc00700 1 -- 172.21.15.153:0/1302325374 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f705002cda0 con 0x7f703406e460 2024-03-21T10:45:10.276 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.279+0000 7f7056400700 1 -- 172.21.15.153:0/1302325374 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f703406e460 msgr2=0x7f7034070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:45:10.276 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.279+0000 7f7056400700 1 --2- 172.21.15.153:0/1302325374 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f703406e460 0x7f7034070920 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f704000b830 tx=0x7f7040006000 comp rx=0 tx=0).stop 2024-03-21T10:45:10.277 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.279+0000 7f7056400700 1 -- 172.21.15.153:0/1302325374 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7050069020 msgr2=0x7f70501001e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:45:10.277 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.279+0000 7f7056400700 1 --2- 172.21.15.153:0/1302325374 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7050069020 0x7f70501001e0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f703800d900 tx=0x7f703800dcc0 comp rx=0 tx=0).stop 2024-03-21T10:45:10.277 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.279+0000 7f7056400700 1 -- 172.21.15.153:0/1302325374 shutdown_connections 2024-03-21T10:45:10.277 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.279+0000 7f7056400700 1 --2- 172.21.15.153:0/1302325374 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7050068670 0x7f70500ffca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:45:10.277 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.279+0000 7f7056400700 1 --2- 172.21.15.153:0/1302325374 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f703406e460 0x7f7034070920 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:45:10.277 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.279+0000 7f7056400700 1 --2- 172.21.15.153:0/1302325374 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7050069020 0x7f70501001e0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:45:10.277 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.279+0000 7f7056400700 1 -- 172.21.15.153:0/1302325374 >> 172.21.15.153:0/1302325374 conn(0x7f70500755a0 msgr2=0x7f70500fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:45:10.277 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.279+0000 7f7056400700 1 -- 172.21.15.153:0/1302325374 shutdown_connections 2024-03-21T10:45:10.277 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.279+0000 7f7056400700 1 -- 172.21.15.153:0/1302325374 wait complete. 2024-03-21T10:45:10.428 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.430+0000 7f42ef000700 1 -- 172.21.15.153:0/1359815350 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f42e810d280 msgr2=0x7f42e810d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:45:10.428 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.430+0000 7f42ef000700 1 --2- 172.21.15.153:0/1359815350 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f42e810d280 0x7f42e810d660 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f42d8009910 tx=0x7f42d8009c20 comp rx=0 tx=0).stop 2024-03-21T10:45:10.428 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.430+0000 7f42ef000700 1 -- 172.21.15.153:0/1359815350 shutdown_connections 2024-03-21T10:45:10.428 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.430+0000 7f42ef000700 1 --2- 172.21.15.153:0/1359815350 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f42e81071d0 0x7f42e8107650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:45:10.428 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.430+0000 7f42ef000700 1 --2- 172.21.15.153:0/1359815350 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f42e810d280 0x7f42e810d660 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:45:10.428 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.430+0000 7f42ef000700 1 -- 172.21.15.153:0/1359815350 >> 172.21.15.153:0/1359815350 conn(0x7f42e80769e0 msgr2=0x7f42e8078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:45:10.428 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.430+0000 7f42ef000700 1 -- 172.21.15.153:0/1359815350 shutdown_connections 2024-03-21T10:45:10.428 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.431+0000 7f42ef000700 1 -- 172.21.15.153:0/1359815350 wait complete. 2024-03-21T10:45:10.429 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.432+0000 7f42ef000700 1 Processor -- start 2024-03-21T10:45:10.429 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.432+0000 7f42ef000700 1 -- start start 2024-03-21T10:45:10.430 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.433+0000 7f42ef000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f42e81071d0 0x7f42e819ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:45:10.430 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.433+0000 7f42ef000700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f42e810d280 0x7f42e819d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:45:10.430 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.433+0000 7f42ef000700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42e819d8a0 con 0x7f42e81071d0 2024-03-21T10:45:10.430 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.433+0000 7f42ef000700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42e81a05d0 con 0x7f42e810d280 2024-03-21T10:45:10.431 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.433+0000 7f42e7e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f42e81071d0 0x7f42e819ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:45:10.431 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.433+0000 7f42e7e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f42e81071d0 0x7f42e819ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:46674/0 (socket says 172.21.15.153:46674) 2024-03-21T10:45:10.431 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.433+0000 7f42e7e00700 1 -- 172.21.15.153:0/180447406 learned_addr learned my addr 172.21.15.153:0/180447406 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:45:10.431 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.433+0000 7f42e7400700 1 --2- 172.21.15.153:0/180447406 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f42e810d280 0x7f42e819d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:45:10.431 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.433+0000 7f42e7e00700 1 -- 172.21.15.153:0/180447406 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f42e810d280 msgr2=0x7f42e819d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:45:10.431 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.433+0000 7f42e7e00700 1 --2- 172.21.15.153:0/180447406 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f42e810d280 0x7f42e819d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:45:10.431 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.433+0000 7f42e7e00700 1 -- 172.21.15.153:0/180447406 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42d80095f0 con 0x7f42e81071d0 2024-03-21T10:45:10.432 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.434+0000 7f42e7e00700 1 --2- 172.21.15.153:0/180447406 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f42e81071d0 0x7f42e819ccd0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f42d8006010 tx=0x7f42d800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:45:10.432 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.434+0000 7f42e4c00700 1 -- 172.21.15.153:0/180447406 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f42d801d070 con 0x7f42e81071d0 2024-03-21T10:45:10.432 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.434+0000 7f42e4c00700 1 -- 172.21.15.153:0/180447406 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f42d800f460 con 0x7f42e81071d0 2024-03-21T10:45:10.432 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.434+0000 7f42ef000700 1 -- 172.21.15.153:0/180447406 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42e81a0850 con 0x7f42e81071d0 2024-03-21T10:45:10.432 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.434+0000 7f42e4c00700 1 -- 172.21.15.153:0/180447406 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f42d8017630 con 0x7f42e81071d0 2024-03-21T10:45:10.432 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.434+0000 7f42ef000700 1 -- 172.21.15.153:0/180447406 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42e81a0dc0 con 0x7f42e81071d0 2024-03-21T10:45:10.433 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.435+0000 7f42ef000700 1 -- 172.21.15.153:0/180447406 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42e804ea90 con 0x7f42e81071d0 2024-03-21T10:45:10.433 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.436+0000 7f42e4c00700 1 -- 172.21.15.153:0/180447406 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f42d800f5d0 con 0x7f42e81071d0 2024-03-21T10:45:10.434 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.437+0000 7f42e4c00700 1 --2- 172.21.15.153:0/180447406 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f42cc06e460 0x7f42cc070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:45:10.435 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.437+0000 7f42e4c00700 1 -- 172.21.15.153:0/180447406 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f42d808f9a0 con 0x7f42e81071d0 2024-03-21T10:45:10.435 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.438+0000 7f42e7400700 1 --2- 172.21.15.153:0/180447406 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f42cc06e460 0x7f42cc070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:45:10.435 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.438+0000 7f42e7400700 1 --2- 172.21.15.153:0/180447406 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f42cc06e460 0x7f42cc070920 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f42d4005950 tx=0x7f42d40058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:45:10.441 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.444+0000 7f42e4c00700 1 -- 172.21.15.153:0/180447406 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f42d8058a40 con 0x7f42e81071d0 2024-03-21T10:45:10.672 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.675+0000 7f42ef000700 1 -- 172.21.15.153:0/180447406 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f42e81a11d0 con 0x7f42cc06e460 2024-03-21T10:45:10.689 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.691+0000 7f42e4c00700 1 -- 172.21.15.153:0/180447406 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f42e81a11d0 con 0x7f42cc06e460 2024-03-21T10:45:10.689 INFO:teuthology.orchestra.run.smithi153.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-21T10:45:10.689 INFO:teuthology.orchestra.run.smithi153.stdout:alertmanager.smithi153 smithi153 *:9093,9094 running (34m) 9m ago 47m 15.6M - 0.25.0 c8568f914cd2 9ed3f2200b5f 2024-03-21T10:45:10.689 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi153 smithi153 running (48m) 9m ago 48m 10.4M - 18.2.1 5be31c24972a 5b61994396a6 2024-03-21T10:45:10.689 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi174 smithi174 running (46m) 9m ago 46m 10.1M - 18.2.1 5be31c24972a 4f7a5fc3ac28 2024-03-21T10:45:10.689 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi153 smithi153 running (30m) 9m ago 48m 6656k - 19.0.0-2231-g2edfee68 237e43c7a65f 32c21104d5b0 2024-03-21T10:45:10.689 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi174 smithi174 running (30m) 9m ago 46m 6660k - 19.0.0-2231-g2edfee68 237e43c7a65f a04e48967ab6 2024-03-21T10:45:10.689 INFO:teuthology.orchestra.run.smithi153.stdout:grafana.smithi153 smithi153 *:3000 running (33m) 9m ago 47m 75.2M - 9.4.12 2bacad6d85d8 ee24026ab786 2024-03-21T10:45:10.690 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.iuyluj smithi153 running (25m) 9m ago 40m 18.7M - 19.0.0-2231-g2edfee68 237e43c7a65f 87aff8a5de70 2024-03-21T10:45:10.690 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.zipckr smithi153 running (25m) 9m ago 40m 18.9M - 19.0.0-2231-g2edfee68 237e43c7a65f ccc262bd5dd3 2024-03-21T10:45:10.690 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.nhozag smithi174 running (9m) 9m ago 40m 10.9M - 19.0.0-2231-g2edfee68 237e43c7a65f 9e02aea72c96 2024-03-21T10:45:10.690 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.yyimst smithi174 running (25m) 9m ago 40m 18.5M - 19.0.0-2231-g2edfee68 237e43c7a65f fc1962f9351e 2024-03-21T10:45:10.690 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi153.tluxni smithi153 *:8443,9283,8765 running (36m) 9m ago 49m 512M - 19.0.0-2231-g2edfee68 237e43c7a65f 7ffee0e55382 2024-03-21T10:45:10.690 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi174.qpcwat smithi174 *:8443,9283,8765 running (35m) 9m ago 46m 459M - 19.0.0-2231-g2edfee68 237e43c7a65f 748968f216ab 2024-03-21T10:45:10.690 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi153 smithi153 running (31m) 9m ago 49m 70.8M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f dd8038303dc3 2024-03-21T10:45:10.690 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi174 smithi174 running (31m) 9m ago 46m 51.9M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f 29d78a788f2c 2024-03-21T10:45:10.690 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi153 smithi153 *:9100 running (35m) 9m ago 47m 8140k - 1.5.0 0da6a335fe13 bb835edc0e0b 2024-03-21T10:45:10.690 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi174 smithi174 *:9100 running (35m) 9m ago 46m 8610k - 1.5.0 0da6a335fe13 7c4ecad790fa 2024-03-21T10:45:10.690 INFO:teuthology.orchestra.run.smithi153.stdout:osd.0 smithi153 running (29m) 9m ago 45m 118M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 507900fd4791 2024-03-21T10:45:10.690 INFO:teuthology.orchestra.run.smithi153.stdout:osd.1 smithi153 running (28m) 9m ago 44m 93.0M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 0d646452e9c1 2024-03-21T10:45:10.690 INFO:teuthology.orchestra.run.smithi153.stdout:osd.2 smithi153 running (28m) 9m ago 44m 91.2M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 1326f2afdfcd 2024-03-21T10:45:10.690 INFO:teuthology.orchestra.run.smithi153.stdout:osd.3 smithi174 running (27m) 9m ago 43m 87.5M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 70aafac71ad5 2024-03-21T10:45:10.691 INFO:teuthology.orchestra.run.smithi153.stdout:osd.4 smithi174 running (27m) 9m ago 42m 95.8M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 0d746ce6e79d 2024-03-21T10:45:10.691 INFO:teuthology.orchestra.run.smithi153.stdout:osd.5 smithi174 running (26m) 9m ago 42m 90.3M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 53e5f35b8cac 2024-03-21T10:45:10.691 INFO:teuthology.orchestra.run.smithi153.stdout:prometheus.smithi153 smithi153 *:9095 running (34m) 9m ago 46m 88.1M - 2.43.0 a07b618ecd1d 7d95561e50c2 2024-03-21T10:45:10.692 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.694+0000 7f42ef000700 1 -- 172.21.15.153:0/180447406 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f42cc06e460 msgr2=0x7f42cc070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:45:10.692 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.694+0000 7f42ef000700 1 --2- 172.21.15.153:0/180447406 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f42cc06e460 0x7f42cc070920 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f42d4005950 tx=0x7f42d40058e0 comp rx=0 tx=0).stop 2024-03-21T10:45:10.692 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.694+0000 7f42ef000700 1 -- 172.21.15.153:0/180447406 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f42e81071d0 msgr2=0x7f42e819ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:45:10.692 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.694+0000 7f42ef000700 1 --2- 172.21.15.153:0/180447406 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f42e81071d0 0x7f42e819ccd0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f42d8006010 tx=0x7f42d800be20 comp rx=0 tx=0).stop 2024-03-21T10:45:10.692 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.694+0000 7f42ef000700 1 -- 172.21.15.153:0/180447406 shutdown_connections 2024-03-21T10:45:10.692 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.694+0000 7f42ef000700 1 --2- 172.21.15.153:0/180447406 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f42cc06e460 0x7f42cc070920 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:45:10.692 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.694+0000 7f42ef000700 1 --2- 172.21.15.153:0/180447406 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f42e81071d0 0x7f42e819ccd0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:45:10.692 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.694+0000 7f42ef000700 1 --2- 172.21.15.153:0/180447406 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f42e810d280 0x7f42e819d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:45:10.692 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.694+0000 7f42ef000700 1 -- 172.21.15.153:0/180447406 >> 172.21.15.153:0/180447406 conn(0x7f42e80769e0 msgr2=0x7f42e8078320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:45:10.692 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.695+0000 7f42ef000700 1 -- 172.21.15.153:0/180447406 shutdown_connections 2024-03-21T10:45:10.692 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.695+0000 7f42ef000700 1 -- 172.21.15.153:0/180447406 wait complete. 2024-03-21T10:45:10.833 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.835+0000 7f2882200700 1 -- 172.21.15.153:0/2526992532 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f287c073160 msgr2=0x7f287c073540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:45:10.833 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.835+0000 7f2882200700 1 --2- 172.21.15.153:0/2526992532 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f287c073160 0x7f287c073540 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f286c009910 tx=0x7f286c009c20 comp rx=0 tx=0).stop 2024-03-21T10:45:10.833 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.835+0000 7f2882200700 1 -- 172.21.15.153:0/2526992532 shutdown_connections 2024-03-21T10:45:10.833 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.835+0000 7f2882200700 1 --2- 172.21.15.153:0/2526992532 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f287c073a80 0x7f287c10cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:45:10.833 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.835+0000 7f2882200700 1 --2- 172.21.15.153:0/2526992532 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f287c073160 0x7f287c073540 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:45:10.833 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.835+0000 7f2882200700 1 -- 172.21.15.153:0/2526992532 >> 172.21.15.153:0/2526992532 conn(0x7f287c0fc650 msgr2=0x7f287c0fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:45:10.833 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.835+0000 7f2882200700 1 -- 172.21.15.153:0/2526992532 shutdown_connections 2024-03-21T10:45:10.833 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.835+0000 7f2882200700 1 -- 172.21.15.153:0/2526992532 wait complete. 2024-03-21T10:45:10.835 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.836+0000 7f2882200700 1 Processor -- start 2024-03-21T10:45:10.835 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.837+0000 7f2882200700 1 -- start start 2024-03-21T10:45:10.835 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.837+0000 7f2882200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f287c073160 0x7f287c100b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:45:10.835 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.837+0000 7f2882200700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f287c073a80 0x7f287c101060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:45:10.835 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.837+0000 7f2882200700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f287c104cb0 con 0x7f287c073160 2024-03-21T10:45:10.835 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.837+0000 7f2882200700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f287c1015a0 con 0x7f287c073a80 2024-03-21T10:45:10.835 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.837+0000 7f287b400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f287c073160 0x7f287c100b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:45:10.835 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.837+0000 7f287b400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f287c073160 0x7f287c100b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:46692/0 (socket says 172.21.15.153:46692) 2024-03-21T10:45:10.835 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.837+0000 7f287b400700 1 -- 172.21.15.153:0/2703765559 learned_addr learned my addr 172.21.15.153:0/2703765559 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:45:10.835 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.837+0000 7f2873e00700 1 --2- 172.21.15.153:0/2703765559 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f287c073a80 0x7f287c101060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:45:10.835 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.838+0000 7f287b400700 1 -- 172.21.15.153:0/2703765559 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f287c073a80 msgr2=0x7f287c101060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:45:10.836 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.838+0000 7f287b400700 1 --2- 172.21.15.153:0/2703765559 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f287c073a80 0x7f287c101060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:45:10.836 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.838+0000 7f287b400700 1 -- 172.21.15.153:0/2703765559 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f286c0095f0 con 0x7f287c073160 2024-03-21T10:45:10.836 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.838+0000 7f287b400700 1 --2- 172.21.15.153:0/2703765559 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f287c073160 0x7f287c100b20 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f286c006010 tx=0x7f286c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:45:10.837 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.838+0000 7f2878c00700 1 -- 172.21.15.153:0/2703765559 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f286c01d070 con 0x7f287c073160 2024-03-21T10:45:10.837 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.838+0000 7f2878c00700 1 -- 172.21.15.153:0/2703765559 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f286c004fb0 con 0x7f287c073160 2024-03-21T10:45:10.837 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.838+0000 7f2882200700 1 -- 172.21.15.153:0/2703765559 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f287c101820 con 0x7f287c073160 2024-03-21T10:45:10.837 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.838+0000 7f2878c00700 1 -- 172.21.15.153:0/2703765559 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f286c00f480 con 0x7f287c073160 2024-03-21T10:45:10.837 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.838+0000 7f2882200700 1 -- 172.21.15.153:0/2703765559 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f287c1a70e0 con 0x7f287c073160 2024-03-21T10:45:10.837 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.839+0000 7f2882200700 1 -- 172.21.15.153:0/2703765559 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f287c10a5c0 con 0x7f287c073160 2024-03-21T10:45:10.837 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.840+0000 7f2878c00700 1 -- 172.21.15.153:0/2703765559 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f286c022890 con 0x7f287c073160 2024-03-21T10:45:10.838 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.841+0000 7f2878c00700 1 --2- 172.21.15.153:0/2703765559 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f286006e460 0x7f2860070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:45:10.838 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.841+0000 7f2878c00700 1 -- 172.21.15.153:0/2703765559 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f286c08fad0 con 0x7f287c073160 2024-03-21T10:45:10.838 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.841+0000 7f2873e00700 1 --2- 172.21.15.153:0/2703765559 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f286006e460 0x7f2860070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:45:10.839 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.842+0000 7f2873e00700 1 --2- 172.21.15.153:0/2703765559 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f286006e460 0x7f2860070920 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f287c102310 tx=0x7f2864005cf0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:45:10.843 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:10.846+0000 7f2878c00700 1 -- 172.21.15.153:0/2703765559 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f286c058cc0 con 0x7f287c073160 2024-03-21T10:45:11.133 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:11.135+0000 7f2882200700 1 -- 172.21.15.153:0/2703765559 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f287c04ea90 con 0x7f287c073160 2024-03-21T10:45:11.134 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:11.136+0000 7f2878c00700 1 -- 172.21.15.153:0/2703765559 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f286c05c380 con 0x7f287c073160 2024-03-21T10:45:11.134 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:45:11.134 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": { 2024-03-21T10:45:11.134 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:45:11.134 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:45:11.134 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": { 2024-03-21T10:45:11.134 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:45:11.135 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:45:11.135 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": { 2024-03-21T10:45:11.135 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 6 2024-03-21T10:45:11.135 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:45:11.135 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": { 2024-03-21T10:45:11.135 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 4 2024-03-21T10:45:11.135 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:45:11.135 INFO:teuthology.orchestra.run.smithi153.stdout: "overall": { 2024-03-21T10:45:11.135 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 14 2024-03-21T10:45:11.135 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-03-21T10:45:11.135 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:45:11.137 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:11.139+0000 7f2882200700 1 -- 172.21.15.153:0/2703765559 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f286006e460 msgr2=0x7f2860070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:45:11.137 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:11.139+0000 7f2882200700 1 --2- 172.21.15.153:0/2703765559 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f286006e460 0x7f2860070920 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f287c102310 tx=0x7f2864005cf0 comp rx=0 tx=0).stop 2024-03-21T10:45:11.137 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:11.139+0000 7f2882200700 1 -- 172.21.15.153:0/2703765559 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f287c073160 msgr2=0x7f287c100b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:45:11.137 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:11.139+0000 7f2882200700 1 --2- 172.21.15.153:0/2703765559 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f287c073160 0x7f287c100b20 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f286c006010 tx=0x7f286c00be20 comp rx=0 tx=0).stop 2024-03-21T10:45:11.137 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:11.140+0000 7f2882200700 1 -- 172.21.15.153:0/2703765559 shutdown_connections 2024-03-21T10:45:11.138 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:11.140+0000 7f2882200700 1 --2- 172.21.15.153:0/2703765559 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f286006e460 0x7f2860070920 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:45:11.138 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:11.140+0000 7f2882200700 1 --2- 172.21.15.153:0/2703765559 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f287c073160 0x7f287c100b20 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:45:11.138 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:11.140+0000 7f2882200700 1 --2- 172.21.15.153:0/2703765559 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f287c073a80 0x7f287c101060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:45:11.138 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:11.140+0000 7f2882200700 1 -- 172.21.15.153:0/2703765559 >> 172.21.15.153:0/2703765559 conn(0x7f287c0fc650 msgr2=0x7f287c107700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:45:11.138 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:11.140+0000 7f2882200700 1 -- 172.21.15.153:0/2703765559 shutdown_connections 2024-03-21T10:45:11.138 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:11.140+0000 7f2882200700 1 -- 172.21.15.153:0/2703765559 wait complete. 2024-03-21T10:45:11.288 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:11.291+0000 7f90f7a00700 1 -- 172.21.15.153:0/2091583403 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f90f00752c0 msgr2=0x7f90f00756a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:45:11.288 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:11.291+0000 7f90f7a00700 1 --2- 172.21.15.153:0/2091583403 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f90f00752c0 0x7f90f00756a0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f90e4009910 tx=0x7f90e4009c20 comp rx=0 tx=0).stop 2024-03-21T10:45:11.289 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:11.291+0000 7f90f7a00700 1 -- 172.21.15.153:0/2091583403 shutdown_connections 2024-03-21T10:45:11.289 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:11.291+0000 7f90f7a00700 1 --2- 172.21.15.153:0/2091583403 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f90f0075be0 0x7f90f01114c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:45:11.289 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:11.291+0000 7f90f7a00700 1 --2- 172.21.15.153:0/2091583403 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f90f00752c0 0x7f90f00756a0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:45:11.289 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:11.291+0000 7f90f7a00700 1 -- 172.21.15.153:0/2091583403 >> 172.21.15.153:0/2091583403 conn(0x7f90f00fe870 msgr2=0x7f90f0100c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:45:11.289 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:11.291+0000 7f90f7a00700 1 -- 172.21.15.153:0/2091583403 shutdown_connections 2024-03-21T10:45:11.289 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:11.291+0000 7f90f7a00700 1 -- 172.21.15.153:0/2091583403 wait complete. 2024-03-21T10:45:11.290 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:11.292+0000 7f90f7a00700 1 Processor -- start 2024-03-21T10:45:11.290 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:11.292+0000 7f90f7a00700 1 -- start start 2024-03-21T10:45:11.290 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:11.293+0000 7f90f7a00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f90f00752c0 0x7f90f01a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:45:11.290 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:11.293+0000 7f90f7a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f90f0075be0 0x7f90f01a2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:45:11.290 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:11.293+0000 7f90f7a00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90f01a35a0 con 0x7f90f0075be0 2024-03-21T10:45:11.290 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:11.293+0000 7f90f7a00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90f019cae0 con 0x7f90f00752c0 2024-03-21T10:45:11.290 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:11.293+0000 7f90efe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f90f0075be0 0x7f90f01a2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:45:11.291 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:11.293+0000 7f90efe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f90f0075be0 0x7f90f01a2fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:46720/0 (socket says 172.21.15.153:46720) 2024-03-21T10:45:11.291 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:11.293+0000 7f90efe00700 1 -- 172.21.15.153:0/1835632646 learned_addr learned my addr 172.21.15.153:0/1835632646 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:45:11.291 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:11.293+0000 7f90f5000700 1 --2- 172.21.15.153:0/1835632646 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f90f00752c0 0x7f90f01a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:45:11.292 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:11.293+0000 7f90efe00700 1 -- 172.21.15.153:0/1835632646 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f90f00752c0 msgr2=0x7f90f01a2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:45:11.292 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:11.293+0000 7f90efe00700 1 --2- 172.21.15.153:0/1835632646 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f90f00752c0 0x7f90f01a2a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:45:11.292 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:11.293+0000 7f90efe00700 1 -- 172.21.15.153:0/1835632646 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90e40095f0 con 0x7f90f0075be0 2024-03-21T10:45:11.292 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:11.294+0000 7f90efe00700 1 --2- 172.21.15.153:0/1835632646 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f90f0075be0 0x7f90f01a2fa0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f90dc00c930 tx=0x7f90dc00ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:45:11.292 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:11.294+0000 7f90ed600700 1 -- 172.21.15.153:0/1835632646 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f90dc007ad0 con 0x7f90f0075be0 2024-03-21T10:45:11.292 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:11.294+0000 7f90ed600700 1 -- 172.21.15.153:0/1835632646 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f90dc007c30 con 0x7f90f0075be0 2024-03-21T10:45:11.292 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:11.294+0000 7f90f7a00700 1 -- 172.21.15.153:0/1835632646 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90f019cdc0 con 0x7f90f0075be0 2024-03-21T10:45:11.292 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:11.294+0000 7f90ed600700 1 -- 172.21.15.153:0/1835632646 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f90dc018610 con 0x7f90f0075be0 2024-03-21T10:45:11.292 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:11.294+0000 7f90f7a00700 1 -- 172.21.15.153:0/1835632646 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90f019d310 con 0x7f90f0075be0 2024-03-21T10:45:11.293 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:11.295+0000 7f90f7a00700 1 -- 172.21.15.153:0/1835632646 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90f010ec30 con 0x7f90f0075be0 2024-03-21T10:45:11.293 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:11.296+0000 7f90ed600700 1 -- 172.21.15.153:0/1835632646 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f90dc00f450 con 0x7f90f0075be0 2024-03-21T10:45:11.294 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:11.297+0000 7f90ed600700 1 --2- 172.21.15.153:0/1835632646 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f90d806e460 0x7f90d8070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:45:11.294 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:11.297+0000 7f90ed600700 1 -- 172.21.15.153:0/1835632646 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f90dc08e790 con 0x7f90f0075be0 2024-03-21T10:45:11.294 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:11.297+0000 7f90f5000700 1 --2- 172.21.15.153:0/1835632646 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f90d806e460 0x7f90d8070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:45:11.295 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:11.298+0000 7f90f5000700 1 --2- 172.21.15.153:0/1835632646 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f90d806e460 0x7f90d8070920 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f90e400b5c0 tx=0x7f90e4005fd0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:45:11.299 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:11.302+0000 7f90ed600700 1 -- 172.21.15.153:0/1835632646 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f90dc05b2e0 con 0x7f90f0075be0 2024-03-21T10:45:11.447 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:45:11 smithi174 ceph-mon[88004]: from='client.35770 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:45:11.447 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:45:11 smithi174 ceph-mon[88004]: pgmap v1001: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:45:11.447 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:45:11 smithi174 ceph-mon[88004]: from='client.35774 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:45:11.447 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:45:11 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/2703765559' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:45:11.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:45:11 smithi153 ceph-mon[113733]: from='client.35770 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:45:11.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:45:11 smithi153 ceph-mon[113733]: pgmap v1001: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:45:11.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:45:11 smithi153 ceph-mon[113733]: from='client.35774 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:45:11.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:45:11 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/2703765559' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:45:11.566 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:11.569+0000 7f90f7a00700 1 -- 172.21.15.153:0/1835632646 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f90f019d590 con 0x7f90f0075be0 2024-03-21T10:45:11.566 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:11.569+0000 7f90ed600700 1 -- 172.21.15.153:0/1835632646 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1876 (secure 0 0 0) 0x7f90dc05ae70 con 0x7f90f0075be0 2024-03-21T10:45:11.568 INFO:teuthology.orchestra.run.smithi153.stdout:e32 2024-03-21T10:45:11.568 INFO:teuthology.orchestra.run.smithi153.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-21T10:45:11.568 INFO:teuthology.orchestra.run.smithi153.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:45:11.568 INFO:teuthology.orchestra.run.smithi153.stdout:legacy client fscid: 1 2024-03-21T10:45:11.568 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:45:11.568 INFO:teuthology.orchestra.run.smithi153.stdout:Filesystem 'cephfs' (1) 2024-03-21T10:45:11.568 INFO:teuthology.orchestra.run.smithi153.stdout:fs_name cephfs 2024-03-21T10:45:11.568 INFO:teuthology.orchestra.run.smithi153.stdout:epoch 23 2024-03-21T10:45:11.568 INFO:teuthology.orchestra.run.smithi153.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-21T10:45:11.568 INFO:teuthology.orchestra.run.smithi153.stdout:created 2024-03-21T10:04:27.086576+0000 2024-03-21T10:45:11.568 INFO:teuthology.orchestra.run.smithi153.stdout:modified 2024-03-21T10:19:05.022092+0000 2024-03-21T10:45:11.568 INFO:teuthology.orchestra.run.smithi153.stdout:tableserver 0 2024-03-21T10:45:11.568 INFO:teuthology.orchestra.run.smithi153.stdout:root 0 2024-03-21T10:45:11.568 INFO:teuthology.orchestra.run.smithi153.stdout:session_timeout 60 2024-03-21T10:45:11.568 INFO:teuthology.orchestra.run.smithi153.stdout:session_autoclose 300 2024-03-21T10:45:11.569 INFO:teuthology.orchestra.run.smithi153.stdout:max_file_size 1099511627776 2024-03-21T10:45:11.569 INFO:teuthology.orchestra.run.smithi153.stdout:max_xattr_size 65536 2024-03-21T10:45:11.569 INFO:teuthology.orchestra.run.smithi153.stdout:required_client_features {} 2024-03-21T10:45:11.569 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure 0 2024-03-21T10:45:11.569 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure_osd_epoch 94 2024-03-21T10:45:11.569 INFO:teuthology.orchestra.run.smithi153.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:45:11.569 INFO:teuthology.orchestra.run.smithi153.stdout:max_mds 2 2024-03-21T10:45:11.569 INFO:teuthology.orchestra.run.smithi153.stdout:in 0,1 2024-03-21T10:45:11.569 INFO:teuthology.orchestra.run.smithi153.stdout:up {} 2024-03-21T10:45:11.569 INFO:teuthology.orchestra.run.smithi153.stdout:failed 0,1 2024-03-21T10:45:11.569 INFO:teuthology.orchestra.run.smithi153.stdout:damaged 2024-03-21T10:45:11.569 INFO:teuthology.orchestra.run.smithi153.stdout:stopped 2024-03-21T10:45:11.569 INFO:teuthology.orchestra.run.smithi153.stdout:data_pools [3] 2024-03-21T10:45:11.569 INFO:teuthology.orchestra.run.smithi153.stdout:metadata_pool 2 2024-03-21T10:45:11.569 INFO:teuthology.orchestra.run.smithi153.stdout:inline_data enabled 2024-03-21T10:45:11.569 INFO:teuthology.orchestra.run.smithi153.stdout:balancer 2024-03-21T10:45:11.570 INFO:teuthology.orchestra.run.smithi153.stdout:bal_rank_mask -1 2024-03-21T10:45:11.570 INFO:teuthology.orchestra.run.smithi153.stdout:standby_count_wanted 1 2024-03-21T10:45:11.570 INFO:teuthology.orchestra.run.smithi153.stdout:qdb_cluster leader: 0 members: 2024-03-21T10:45:11.570 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:45:11.570 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:45:11.570 INFO:teuthology.orchestra.run.smithi153.stdout:Standby daemons: 2024-03-21T10:45:11.570 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:45:11.570 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.zipckr{-1:34442} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6826/1103846295,v1:172.21.15.153:6827/1103846295] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:45:11.570 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.iuyluj{-1:34474} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6828/1738630832,v1:172.21.15.153:6829/1738630832] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:45:11.570 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.yyimst{-1:44313} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.174:6824/4127786797,v1:172.21.15.174:6825/4127786797] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:45:11.570 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.nhozag{-1:44687} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.174:6826/2767517845,v1:172.21.15.174:6827/2767517845] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:45:11.571 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:11.572+0000 7f90f7a00700 1 -- 172.21.15.153:0/1835632646 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f90d806e460 msgr2=0x7f90d8070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:45:11.571 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:11.572+0000 7f90f7a00700 1 --2- 172.21.15.153:0/1835632646 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f90d806e460 0x7f90d8070920 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f90e400b5c0 tx=0x7f90e4005fd0 comp rx=0 tx=0).stop 2024-03-21T10:45:11.571 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:11.572+0000 7f90f7a00700 1 -- 172.21.15.153:0/1835632646 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f90f0075be0 msgr2=0x7f90f01a2fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:45:11.571 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:11.572+0000 7f90f7a00700 1 --2- 172.21.15.153:0/1835632646 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f90f0075be0 0x7f90f01a2fa0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f90dc00c930 tx=0x7f90dc00ccf0 comp rx=0 tx=0).stop 2024-03-21T10:45:11.571 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:11.572+0000 7f90f7a00700 1 -- 172.21.15.153:0/1835632646 shutdown_connections 2024-03-21T10:45:11.571 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:11.572+0000 7f90f7a00700 1 --2- 172.21.15.153:0/1835632646 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f90f00752c0 0x7f90f01a2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:45:11.571 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:11.572+0000 7f90f7a00700 1 --2- 172.21.15.153:0/1835632646 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f90d806e460 0x7f90d8070920 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:45:11.571 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:11.572+0000 7f90f7a00700 1 --2- 172.21.15.153:0/1835632646 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f90f0075be0 0x7f90f01a2fa0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:45:11.571 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:11.572+0000 7f90f7a00700 1 -- 172.21.15.153:0/1835632646 >> 172.21.15.153:0/1835632646 conn(0x7f90f00fe870 msgr2=0x7f90f00ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:45:11.572 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:11.573+0000 7f90f7a00700 1 -- 172.21.15.153:0/1835632646 shutdown_connections 2024-03-21T10:45:11.572 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:11.573+0000 7f90f7a00700 1 -- 172.21.15.153:0/1835632646 wait complete. 2024-03-21T10:45:11.572 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 32 2024-03-21T10:45:11.724 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:11.727+0000 7f6669000700 1 -- 172.21.15.153:0/2531885533 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6664073160 msgr2=0x7f6664073540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:45:11.724 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:11.727+0000 7f6669000700 1 --2- 172.21.15.153:0/2531885533 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6664073160 0x7f6664073540 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f6654009910 tx=0x7f6654009c20 comp rx=0 tx=0).stop 2024-03-21T10:45:11.724 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:11.727+0000 7f6669000700 1 -- 172.21.15.153:0/2531885533 shutdown_connections 2024-03-21T10:45:11.724 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:11.727+0000 7f6669000700 1 --2- 172.21.15.153:0/2531885533 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6664073a80 0x7f666410cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:45:11.724 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:11.727+0000 7f6669000700 1 --2- 172.21.15.153:0/2531885533 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6664073160 0x7f6664073540 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:45:11.724 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:11.727+0000 7f6669000700 1 -- 172.21.15.153:0/2531885533 >> 172.21.15.153:0/2531885533 conn(0x7f66640fc650 msgr2=0x7f66640fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:45:11.725 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:11.727+0000 7f6669000700 1 -- 172.21.15.153:0/2531885533 shutdown_connections 2024-03-21T10:45:11.725 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:11.727+0000 7f6669000700 1 -- 172.21.15.153:0/2531885533 wait complete. 2024-03-21T10:45:11.725 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:11.728+0000 7f6669000700 1 Processor -- start 2024-03-21T10:45:11.725 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:11.728+0000 7f6669000700 1 -- start start 2024-03-21T10:45:11.725 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:11.728+0000 7f6669000700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6664073160 0x7f666419e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:45:11.725 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:11.728+0000 7f6669000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6664073a80 0x7f666419ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:45:11.726 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:11.728+0000 7f6669000700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f666419f3e0 con 0x7f6664073a80 2024-03-21T10:45:11.726 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:11.728+0000 7f6669000700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6664198890 con 0x7f6664073160 2024-03-21T10:45:11.727 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:11.729+0000 7f6661400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6664073a80 0x7f666419ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:45:11.727 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:11.729+0000 7f6661400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6664073a80 0x7f666419ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:46732/0 (socket says 172.21.15.153:46732) 2024-03-21T10:45:11.727 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:11.729+0000 7f6661400700 1 -- 172.21.15.153:0/3557505241 learned_addr learned my addr 172.21.15.153:0/3557505241 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:45:11.727 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:11.729+0000 7f6661e00700 1 --2- 172.21.15.153:0/3557505241 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6664073160 0x7f666419e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:45:11.727 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:11.729+0000 7f6661400700 1 -- 172.21.15.153:0/3557505241 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6664073160 msgr2=0x7f666419e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:45:11.727 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:11.729+0000 7f6661400700 1 --2- 172.21.15.153:0/3557505241 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6664073160 0x7f666419e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:45:11.727 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:11.729+0000 7f6661400700 1 -- 172.21.15.153:0/3557505241 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66540095f0 con 0x7f6664073a80 2024-03-21T10:45:11.727 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:11.729+0000 7f6661400700 1 --2- 172.21.15.153:0/3557505241 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6664073a80 0x7f666419ed50 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f665000b4b0 tx=0x7f665000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:45:11.727 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:11.730+0000 7f665aa00700 1 -- 172.21.15.153:0/3557505241 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6650011800 con 0x7f6664073a80 2024-03-21T10:45:11.727 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:11.730+0000 7f665aa00700 1 -- 172.21.15.153:0/3557505241 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6650011e20 con 0x7f6664073a80 2024-03-21T10:45:11.727 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:11.730+0000 7f665aa00700 1 -- 172.21.15.153:0/3557505241 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f665000f2e0 con 0x7f6664073a80 2024-03-21T10:45:11.727 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:11.730+0000 7f6669000700 1 -- 172.21.15.153:0/3557505241 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6664198b70 con 0x7f6664073a80 2024-03-21T10:45:11.727 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:11.730+0000 7f6669000700 1 -- 172.21.15.153:0/3557505241 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66641990c0 con 0x7f6664073a80 2024-03-21T10:45:11.728 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:11.731+0000 7f6669000700 1 -- 172.21.15.153:0/3557505241 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f666410a5a0 con 0x7f6664073a80 2024-03-21T10:45:11.729 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:11.732+0000 7f665aa00700 1 -- 172.21.15.153:0/3557505241 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f6650010380 con 0x7f6664073a80 2024-03-21T10:45:11.730 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:11.733+0000 7f665aa00700 1 --2- 172.21.15.153:0/3557505241 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f664806e460 0x7f6648070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:45:11.730 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:11.733+0000 7f665aa00700 1 -- 172.21.15.153:0/3557505241 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f665008e0c0 con 0x7f6664073a80 2024-03-21T10:45:11.730 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:11.733+0000 7f6661e00700 1 --2- 172.21.15.153:0/3557505241 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f664806e460 0x7f6648070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:45:11.731 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:11.734+0000 7f6661e00700 1 --2- 172.21.15.153:0/3557505241 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f664806e460 0x7f6648070920 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f6654006010 tx=0x7f665400b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:45:11.735 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:11.738+0000 7f665aa00700 1 -- 172.21.15.153:0/3557505241 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f665005ad40 con 0x7f6664073a80 2024-03-21T10:45:11.962 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:11.965+0000 7f6669000700 1 -- 172.21.15.153:0/3557505241 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6664199390 con 0x7f664806e460 2024-03-21T10:45:11.965 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:11.968+0000 7f665aa00700 1 -- 172.21.15.153:0/3557505241 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f6664199390 con 0x7f664806e460 2024-03-21T10:45:11.965 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:45:11.965 INFO:teuthology.orchestra.run.smithi153.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232", 2024-03-21T10:45:11.965 INFO:teuthology.orchestra.run.smithi153.stdout: "in_progress": true, 2024-03-21T10:45:11.965 INFO:teuthology.orchestra.run.smithi153.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-21T10:45:11.965 INFO:teuthology.orchestra.run.smithi153.stdout: "services_complete": [ 2024-03-21T10:45:11.965 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr", 2024-03-21T10:45:11.965 INFO:teuthology.orchestra.run.smithi153.stdout: "mon", 2024-03-21T10:45:11.965 INFO:teuthology.orchestra.run.smithi153.stdout: "mds", 2024-03-21T10:45:11.966 INFO:teuthology.orchestra.run.smithi153.stdout: "crash", 2024-03-21T10:45:11.966 INFO:teuthology.orchestra.run.smithi153.stdout: "osd" 2024-03-21T10:45:11.966 INFO:teuthology.orchestra.run.smithi153.stdout: ], 2024-03-21T10:45:11.966 INFO:teuthology.orchestra.run.smithi153.stdout: "progress": "16/23 daemons upgraded", 2024-03-21T10:45:11.966 INFO:teuthology.orchestra.run.smithi153.stdout: "message": "Currently upgrading mds daemons", 2024-03-21T10:45:11.966 INFO:teuthology.orchestra.run.smithi153.stdout: "is_paused": false 2024-03-21T10:45:11.966 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:45:11.968 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:11.970+0000 7f6669000700 1 -- 172.21.15.153:0/3557505241 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f664806e460 msgr2=0x7f6648070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:45:11.968 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:11.970+0000 7f6669000700 1 --2- 172.21.15.153:0/3557505241 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f664806e460 0x7f6648070920 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f6654006010 tx=0x7f665400b540 comp rx=0 tx=0).stop 2024-03-21T10:45:11.968 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:11.970+0000 7f6669000700 1 -- 172.21.15.153:0/3557505241 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6664073a80 msgr2=0x7f666419ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:45:11.968 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:11.970+0000 7f6669000700 1 --2- 172.21.15.153:0/3557505241 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6664073a80 0x7f666419ed50 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f665000b4b0 tx=0x7f665000b870 comp rx=0 tx=0).stop 2024-03-21T10:45:11.968 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:11.970+0000 7f6669000700 1 -- 172.21.15.153:0/3557505241 shutdown_connections 2024-03-21T10:45:11.968 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:11.970+0000 7f6669000700 1 --2- 172.21.15.153:0/3557505241 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6664073160 0x7f666419e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:45:11.968 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:11.970+0000 7f6669000700 1 --2- 172.21.15.153:0/3557505241 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f664806e460 0x7f6648070920 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:45:11.968 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:11.970+0000 7f6669000700 1 --2- 172.21.15.153:0/3557505241 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6664073a80 0x7f666419ed50 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:45:11.969 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:11.970+0000 7f6669000700 1 -- 172.21.15.153:0/3557505241 >> 172.21.15.153:0/3557505241 conn(0x7f66640fc650 msgr2=0x7f66641076e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:45:11.969 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:11.971+0000 7f6669000700 1 -- 172.21.15.153:0/3557505241 shutdown_connections 2024-03-21T10:45:11.969 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:11.971+0000 7f6669000700 1 -- 172.21.15.153:0/3557505241 wait complete. 2024-03-21T10:45:12.120 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:12.123+0000 7fab59c00700 1 -- 172.21.15.153:0/75299021 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fab541071d0 msgr2=0x7fab54107650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:45:12.120 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:12.123+0000 7fab59c00700 1 --2- 172.21.15.153:0/75299021 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fab541071d0 0x7fab54107650 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fab40009910 tx=0x7fab40009c20 comp rx=0 tx=0).stop 2024-03-21T10:45:12.120 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:12.123+0000 7fab59c00700 1 -- 172.21.15.153:0/75299021 shutdown_connections 2024-03-21T10:45:12.120 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:12.123+0000 7fab59c00700 1 --2- 172.21.15.153:0/75299021 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fab541071d0 0x7fab54107650 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:45:12.120 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:12.123+0000 7fab59c00700 1 --2- 172.21.15.153:0/75299021 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fab5410d280 0x7fab5410d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:45:12.121 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:12.123+0000 7fab59c00700 1 -- 172.21.15.153:0/75299021 >> 172.21.15.153:0/75299021 conn(0x7fab540769e0 msgr2=0x7fab54078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:45:12.121 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:12.123+0000 7fab59c00700 1 -- 172.21.15.153:0/75299021 shutdown_connections 2024-03-21T10:45:12.121 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:12.123+0000 7fab59c00700 1 -- 172.21.15.153:0/75299021 wait complete. 2024-03-21T10:45:12.121 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:12.124+0000 7fab59c00700 1 Processor -- start 2024-03-21T10:45:12.121 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:12.124+0000 7fab59c00700 1 -- start start 2024-03-21T10:45:12.122 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:12.125+0000 7fab59c00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fab541071d0 0x7fab5419ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:45:12.122 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:12.125+0000 7fab59c00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fab5410d280 0x7fab5419d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:45:12.122 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:12.125+0000 7fab59c00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab5419d880 con 0x7fab541071d0 2024-03-21T10:45:12.122 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:12.125+0000 7fab59c00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab541a05b0 con 0x7fab5410d280 2024-03-21T10:45:12.123 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:12.125+0000 7fab52a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fab541071d0 0x7fab5419ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:45:12.123 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:12.125+0000 7fab52a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fab541071d0 0x7fab5419ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:46756/0 (socket says 172.21.15.153:46756) 2024-03-21T10:45:12.123 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:12.125+0000 7fab52a00700 1 -- 172.21.15.153:0/3199844364 learned_addr learned my addr 172.21.15.153:0/3199844364 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:45:12.123 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:12.125+0000 7fab52000700 1 --2- 172.21.15.153:0/3199844364 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fab5410d280 0x7fab5419d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:45:12.123 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:12.125+0000 7fab52a00700 1 -- 172.21.15.153:0/3199844364 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fab5410d280 msgr2=0x7fab5419d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:45:12.123 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:12.125+0000 7fab52a00700 1 --2- 172.21.15.153:0/3199844364 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fab5410d280 0x7fab5419d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:45:12.123 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:12.125+0000 7fab52a00700 1 -- 172.21.15.153:0/3199844364 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab400095f0 con 0x7fab541071d0 2024-03-21T10:45:12.123 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:12.126+0000 7fab52a00700 1 --2- 172.21.15.153:0/3199844364 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fab541071d0 0x7fab5419ccb0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fab4400d8d0 tx=0x7fab4400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:45:12.124 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:12.126+0000 7fab4b400700 1 -- 172.21.15.153:0/3199844364 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab44009940 con 0x7fab541071d0 2024-03-21T10:45:12.124 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:12.126+0000 7fab4b400700 1 -- 172.21.15.153:0/3199844364 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fab44010460 con 0x7fab541071d0 2024-03-21T10:45:12.124 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:12.126+0000 7fab59c00700 1 -- 172.21.15.153:0/3199844364 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab541a0890 con 0x7fab541071d0 2024-03-21T10:45:12.124 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:12.126+0000 7fab4b400700 1 -- 172.21.15.153:0/3199844364 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab4400f5f0 con 0x7fab541071d0 2024-03-21T10:45:12.124 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:12.126+0000 7fab59c00700 1 -- 172.21.15.153:0/3199844364 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab541a0e60 con 0x7fab541071d0 2024-03-21T10:45:12.124 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:12.127+0000 7fab59c00700 1 -- 172.21.15.153:0/3199844364 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab5410f6c0 con 0x7fab541071d0 2024-03-21T10:45:12.125 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:12.128+0000 7fab4b400700 1 -- 172.21.15.153:0/3199844364 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fab440105d0 con 0x7fab541071d0 2024-03-21T10:45:12.126 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:12.129+0000 7fab4b400700 1 --2- 172.21.15.153:0/3199844364 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fab3806e460 0x7fab38070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:45:12.126 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:12.129+0000 7fab4b400700 1 -- 172.21.15.153:0/3199844364 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fab4408e5b0 con 0x7fab541071d0 2024-03-21T10:45:12.127 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:12.130+0000 7fab52000700 1 --2- 172.21.15.153:0/3199844364 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fab3806e460 0x7fab38070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:45:12.127 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:12.130+0000 7fab52000700 1 --2- 172.21.15.153:0/3199844364 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fab3806e460 0x7fab38070920 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fab5419e290 tx=0x7fab4001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:45:12.131 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:12.134+0000 7fab4b400700 1 -- 172.21.15.153:0/3199844364 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fab440575d0 con 0x7fab541071d0 2024-03-21T10:45:12.302 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:45:12 smithi153 ceph-mon[113733]: from='client.35778 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:45:12.302 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:45:12 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/1835632646' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:45:12.432 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:12.434+0000 7fab59c00700 1 -- 172.21.15.153:0/3199844364 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fab5404ea90 con 0x7fab541071d0 2024-03-21T10:45:12.432 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:12.435+0000 7fab4b400700 1 -- 172.21.15.153:0/3199844364 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4493 (secure 0 0 0) 0x7fab4405ac90 con 0x7fab541071d0 2024-03-21T10:45:12.433 INFO:teuthology.orchestra.run.smithi153.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-03-21T10:45:12.433 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-03-21T10:45:12.433 INFO:teuthology.orchestra.run.smithi153.stdout: host smithi174 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi174/config 2024-03-21T10:45:12.433 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/podman: timeout after 895 seconds 2024-03-21T10:45:12.433 INFO:teuthology.orchestra.run.smithi153.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 -e NODE_NAME=smithi174 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/run/ceph:z -v /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpd910r3_i:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 inventory --format=json-pretty --filter-for-batch 2024-03-21T10:45:12.433 INFO:teuthology.orchestra.run.smithi153.stdout:Traceback (most recent call last): 2024-03-21T10:45:12.433 INFO:teuthology.orchestra.run.smithi153.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-03-21T10:45:12.433 INFO:teuthology.orchestra.run.smithi153.stdout: return _run_code(code, main_globals, None, 2024-03-21T10:45:12.433 INFO:teuthology.orchestra.run.smithi153.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-03-21T10:45:12.434 INFO:teuthology.orchestra.run.smithi153.stdout: exec(code, run_globals) 2024-03-21T10:45:12.434 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 5606, in 2024-03-21T10:45:12.434 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 5594, in main 2024-03-21T10:45:12.434 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 406, in _infer_config 2024-03-21T10:45:12.434 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 321, in _infer_fsid 2024-03-21T10:45:12.434 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 434, in _infer_image 2024-03-21T10:45:12.434 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 308, in _validate_fsid 2024-03-21T10:45:12.434 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 3357, in command_ceph_volume 2024-03-21T10:45:12.434 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-03-21T10:45:12.434 INFO:teuthology.orchestra.run.smithi153.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 -e NODE_NAME=smithi174 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/run/ceph:z -v /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpd910r3_i:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 inventory --format=json-pretty --filter-for-batch 2024-03-21T10:45:12.434 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-21T10:45:12.434 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is degraded 2024-03-21T10:45:12.434 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-21T10:45:12.434 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-21T10:45:12.434 INFO:teuthology.orchestra.run.smithi153.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-21T10:45:12.435 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-21T10:45:12.436 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:12.438+0000 7fab59c00700 1 -- 172.21.15.153:0/3199844364 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fab3806e460 msgr2=0x7fab38070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:45:12.436 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:12.438+0000 7fab59c00700 1 --2- 172.21.15.153:0/3199844364 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fab3806e460 0x7fab38070920 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fab5419e290 tx=0x7fab4001a040 comp rx=0 tx=0).stop 2024-03-21T10:45:12.436 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:12.438+0000 7fab59c00700 1 -- 172.21.15.153:0/3199844364 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fab541071d0 msgr2=0x7fab5419ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:45:12.436 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:12.438+0000 7fab59c00700 1 --2- 172.21.15.153:0/3199844364 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fab541071d0 0x7fab5419ccb0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fab4400d8d0 tx=0x7fab4400dc90 comp rx=0 tx=0).stop 2024-03-21T10:45:12.436 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:12.438+0000 7fab59c00700 1 -- 172.21.15.153:0/3199844364 shutdown_connections 2024-03-21T10:45:12.436 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:12.438+0000 7fab59c00700 1 --2- 172.21.15.153:0/3199844364 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fab3806e460 0x7fab38070920 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:45:12.436 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:12.438+0000 7fab59c00700 1 --2- 172.21.15.153:0/3199844364 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fab541071d0 0x7fab5419ccb0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:45:12.436 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:12.438+0000 7fab59c00700 1 --2- 172.21.15.153:0/3199844364 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fab5410d280 0x7fab5419d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:45:12.436 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:12.438+0000 7fab59c00700 1 -- 172.21.15.153:0/3199844364 >> 172.21.15.153:0/3199844364 conn(0x7fab540769e0 msgr2=0x7fab54078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:45:12.436 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:12.438+0000 7fab59c00700 1 -- 172.21.15.153:0/3199844364 shutdown_connections 2024-03-21T10:45:12.436 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:12.438+0000 7fab59c00700 1 -- 172.21.15.153:0/3199844364 wait complete. 2024-03-21T10:45:12.447 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:45:12 smithi174 ceph-mon[88004]: from='client.35778 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:45:12.447 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:45:12 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/1835632646' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:45:13.447 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:45:13 smithi174 ceph-mon[88004]: pgmap v1002: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:45:13.447 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:45:13 smithi174 ceph-mon[88004]: from='client.35790 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:45:13.447 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:45:13 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/3199844364' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:45:13.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:45:13 smithi153 ceph-mon[113733]: pgmap v1002: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:45:13.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:45:13 smithi153 ceph-mon[113733]: from='client.35790 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:45:13.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:45:13 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/3199844364' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:45:15.447 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:45:15 smithi174 ceph-mon[88004]: pgmap v1003: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:45:15.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:45:15 smithi153 ceph-mon[113733]: pgmap v1003: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:45:17.447 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:45:17 smithi174 ceph-mon[88004]: pgmap v1004: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:45:17.447 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:45:17 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:45:17.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:45:17 smithi153 ceph-mon[113733]: pgmap v1004: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:45:17.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:45:17 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:45:19.447 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:45:19 smithi174 ceph-mon[88004]: pgmap v1005: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:45:19.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:45:19 smithi153 ceph-mon[113733]: pgmap v1005: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:45:21.447 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:45:21 smithi174 ceph-mon[88004]: pgmap v1006: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:45:21.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:45:21 smithi153 ceph-mon[113733]: pgmap v1006: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:45:23.447 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:45:23 smithi174 ceph-mon[88004]: pgmap v1007: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:45:23.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:45:23 smithi153 ceph-mon[113733]: pgmap v1007: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:45:25.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:45:25 smithi153 ceph-mon[113733]: pgmap v1008: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:45:25.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:45:25 smithi174 ceph-mon[88004]: pgmap v1008: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:45:27.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:45:27 smithi153 ceph-mon[113733]: pgmap v1009: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:45:27.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:45:27 smithi174 ceph-mon[88004]: pgmap v1009: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:45:29.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:45:29 smithi153 ceph-mon[113733]: pgmap v1010: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:45:29.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:45:29 smithi174 ceph-mon[88004]: pgmap v1010: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:45:31.447 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:45:31 smithi174 ceph-mon[88004]: pgmap v1011: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:45:31.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:45:31 smithi153 ceph-mon[113733]: pgmap v1011: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:45:32.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:45:32 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:45:32.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:45:32 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:45:33.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:45:33 smithi153 ceph-mon[113733]: pgmap v1012: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:45:33.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:45:33 smithi174 ceph-mon[88004]: pgmap v1012: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:45:35.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:45:35 smithi153 ceph-mon[113733]: pgmap v1013: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:45:35.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:45:35 smithi174 ceph-mon[88004]: pgmap v1013: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:45:37.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:45:37 smithi153 ceph-mon[113733]: pgmap v1014: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:45:37.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:45:37 smithi174 ceph-mon[88004]: pgmap v1014: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:45:39.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:45:39 smithi153 ceph-mon[113733]: pgmap v1015: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:45:39.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:45:39 smithi174 ceph-mon[88004]: pgmap v1015: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:45:41.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:45:41 smithi153 ceph-mon[113733]: pgmap v1016: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:45:41.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:45:41 smithi174 ceph-mon[88004]: pgmap v1016: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:45:42.604 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:42.606+0000 7f9d27600700 1 -- 172.21.15.153:0/2588178027 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9d201039b0 msgr2=0x7f9d20107a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:45:42.604 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:42.606+0000 7f9d27600700 1 --2- 172.21.15.153:0/2588178027 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9d201039b0 0x7f9d20107a00 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c009910 tx=0x7f9d0c009c20 comp rx=0 tx=0).stop 2024-03-21T10:45:42.604 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:42.607+0000 7f9d27600700 1 -- 172.21.15.153:0/2588178027 shutdown_connections 2024-03-21T10:45:42.604 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:42.607+0000 7f9d27600700 1 --2- 172.21.15.153:0/2588178027 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9d201039b0 0x7f9d20107a00 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:45:42.604 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:42.607+0000 7f9d27600700 1 --2- 172.21.15.153:0/2588178027 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d20103000 0x7f9d201033e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:45:42.604 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:42.607+0000 7f9d27600700 1 -- 172.21.15.153:0/2588178027 >> 172.21.15.153:0/2588178027 conn(0x7f9d200fe8d0 msgr2=0x7f9d20100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:45:42.605 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:42.607+0000 7f9d27600700 1 -- 172.21.15.153:0/2588178027 shutdown_connections 2024-03-21T10:45:42.605 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:42.607+0000 7f9d27600700 1 -- 172.21.15.153:0/2588178027 wait complete. 2024-03-21T10:45:42.605 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:42.608+0000 7f9d27600700 1 Processor -- start 2024-03-21T10:45:42.605 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:42.608+0000 7f9d27600700 1 -- start start 2024-03-21T10:45:42.606 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:42.608+0000 7f9d27600700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d20103000 0x7f9d2019e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:45:42.606 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:42.608+0000 7f9d27600700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9d201039b0 0x7f9d2019ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:45:42.606 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:42.609+0000 7f9d27600700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d2019f300 con 0x7f9d201039b0 2024-03-21T10:45:42.606 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:42.609+0000 7f9d27600700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d20198840 con 0x7f9d20103000 2024-03-21T10:45:42.606 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:42.609+0000 7f9d1fe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9d201039b0 0x7f9d2019ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:45:42.606 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:42.609+0000 7f9d1fe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9d201039b0 0x7f9d2019ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:36774/0 (socket says 172.21.15.153:36774) 2024-03-21T10:45:42.606 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:42.609+0000 7f9d1fe00700 1 -- 172.21.15.153:0/3651158473 learned_addr learned my addr 172.21.15.153:0/3651158473 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:45:42.607 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:42.609+0000 7f9d24c00700 1 --2- 172.21.15.153:0/3651158473 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d20103000 0x7f9d2019e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:45:42.607 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:42.609+0000 7f9d1fe00700 1 -- 172.21.15.153:0/3651158473 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d20103000 msgr2=0x7f9d2019e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:45:42.608 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:42.609+0000 7f9d1fe00700 1 --2- 172.21.15.153:0/3651158473 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d20103000 0x7f9d2019e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:45:42.608 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:42.609+0000 7f9d1fe00700 1 -- 172.21.15.153:0/3651158473 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d0c0095f0 con 0x7f9d201039b0 2024-03-21T10:45:42.608 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:42.610+0000 7f9d1fe00700 1 --2- 172.21.15.153:0/3651158473 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9d201039b0 0x7f9d2019ed00 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c00bda0 tx=0x7f9d0c00be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:45:42.608 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:42.610+0000 7f9d1d600700 1 -- 172.21.15.153:0/3651158473 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9d0c01d070 con 0x7f9d201039b0 2024-03-21T10:45:42.608 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:42.610+0000 7f9d1d600700 1 -- 172.21.15.153:0/3651158473 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9d0c004fb0 con 0x7f9d201039b0 2024-03-21T10:45:42.608 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:42.610+0000 7f9d27600700 1 -- 172.21.15.153:0/3651158473 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d20198ac0 con 0x7f9d201039b0 2024-03-21T10:45:42.608 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:42.610+0000 7f9d1d600700 1 -- 172.21.15.153:0/3651158473 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9d0c00f460 con 0x7f9d201039b0 2024-03-21T10:45:42.608 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:42.610+0000 7f9d27600700 1 -- 172.21.15.153:0/3651158473 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d20198fb0 con 0x7f9d201039b0 2024-03-21T10:45:42.608 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:42.612+0000 7f9d27600700 1 -- 172.21.15.153:0/3651158473 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d2010b400 con 0x7f9d201039b0 2024-03-21T10:45:42.609 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:42.612+0000 7f9d1d600700 1 -- 172.21.15.153:0/3651158473 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f9d0c022800 con 0x7f9d201039b0 2024-03-21T10:45:42.611 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:42.614+0000 7f9d1d600700 1 --2- 172.21.15.153:0/3651158473 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f9d0806e460 0x7f9d08070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:45:42.612 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:42.614+0000 7f9d1d600700 1 -- 172.21.15.153:0/3651158473 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f9d0c08f980 con 0x7f9d201039b0 2024-03-21T10:45:42.612 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:42.614+0000 7f9d24c00700 1 --2- 172.21.15.153:0/3651158473 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f9d0806e460 0x7f9d08070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:45:42.612 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:42.615+0000 7f9d24c00700 1 --2- 172.21.15.153:0/3651158473 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f9d0806e460 0x7f9d08070920 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f9d14005ea0 tx=0x7f9d14005e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:45:42.616 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:42.619+0000 7f9d1d600700 1 -- 172.21.15.153:0/3651158473 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f9d0c05c7c0 con 0x7f9d201039b0 2024-03-21T10:45:42.856 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:42.858+0000 7f9d27600700 1 -- 172.21.15.153:0/3651158473 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9d20199220 con 0x7f9d0806e460 2024-03-21T10:45:42.858 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:42.861+0000 7f9d1d600700 1 -- 172.21.15.153:0/3651158473 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f9d20199220 con 0x7f9d0806e460 2024-03-21T10:45:42.861 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:42.864+0000 7f9d27600700 1 -- 172.21.15.153:0/3651158473 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f9d0806e460 msgr2=0x7f9d08070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:45:42.861 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:42.864+0000 7f9d27600700 1 --2- 172.21.15.153:0/3651158473 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f9d0806e460 0x7f9d08070920 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f9d14005ea0 tx=0x7f9d14005e30 comp rx=0 tx=0).stop 2024-03-21T10:45:42.862 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:42.864+0000 7f9d27600700 1 -- 172.21.15.153:0/3651158473 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9d201039b0 msgr2=0x7f9d2019ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:45:42.862 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:42.864+0000 7f9d27600700 1 --2- 172.21.15.153:0/3651158473 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9d201039b0 0x7f9d2019ed00 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c00bda0 tx=0x7f9d0c00be80 comp rx=0 tx=0).stop 2024-03-21T10:45:42.862 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:42.864+0000 7f9d27600700 1 -- 172.21.15.153:0/3651158473 shutdown_connections 2024-03-21T10:45:42.862 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:42.864+0000 7f9d27600700 1 --2- 172.21.15.153:0/3651158473 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d20103000 0x7f9d2019e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:45:42.862 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:42.864+0000 7f9d27600700 1 --2- 172.21.15.153:0/3651158473 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f9d0806e460 0x7f9d08070920 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:45:42.863 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:42.864+0000 7f9d27600700 1 --2- 172.21.15.153:0/3651158473 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9d201039b0 0x7f9d2019ed00 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:45:42.863 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:42.864+0000 7f9d27600700 1 -- 172.21.15.153:0/3651158473 >> 172.21.15.153:0/3651158473 conn(0x7f9d200fe8d0 msgr2=0x7f9d20100cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:45:42.863 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:42.865+0000 7f9d27600700 1 -- 172.21.15.153:0/3651158473 shutdown_connections 2024-03-21T10:45:42.863 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:42.865+0000 7f9d27600700 1 -- 172.21.15.153:0/3651158473 wait complete. 2024-03-21T10:45:42.877 INFO:teuthology.orchestra.run.smithi153.stdout:true 2024-03-21T10:45:43.018 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.020+0000 7efbfce00700 1 -- 172.21.15.153:0/905118533 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efbf8068d30 msgr2=0x7efbf810d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:45:43.018 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.020+0000 7efbfce00700 1 --2- 172.21.15.153:0/905118533 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efbf8068d30 0x7efbf810d080 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7efbe0009910 tx=0x7efbe0009c20 comp rx=0 tx=0).stop 2024-03-21T10:45:43.018 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.020+0000 7efbfce00700 1 -- 172.21.15.153:0/905118533 shutdown_connections 2024-03-21T10:45:43.018 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.020+0000 7efbfce00700 1 --2- 172.21.15.153:0/905118533 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efbf8068d30 0x7efbf810d080 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:45:43.018 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.020+0000 7efbfce00700 1 --2- 172.21.15.153:0/905118533 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efbf8068410 0x7efbf80687f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:45:43.018 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.020+0000 7efbfce00700 1 -- 172.21.15.153:0/905118533 >> 172.21.15.153:0/905118533 conn(0x7efbf8075330 msgr2=0x7efbf8075740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:45:43.018 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.021+0000 7efbfce00700 1 -- 172.21.15.153:0/905118533 shutdown_connections 2024-03-21T10:45:43.018 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.021+0000 7efbfce00700 1 -- 172.21.15.153:0/905118533 wait complete. 2024-03-21T10:45:43.019 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.022+0000 7efbfce00700 1 Processor -- start 2024-03-21T10:45:43.019 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.022+0000 7efbfce00700 1 -- start start 2024-03-21T10:45:43.020 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.022+0000 7efbfce00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efbf8068410 0x7efbf819e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:45:43.020 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.022+0000 7efbfce00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efbf8068d30 0x7efbf819ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:45:43.020 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.022+0000 7efbfce00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efbf819f0a0 con 0x7efbf8068410 2024-03-21T10:45:43.020 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.022+0000 7efbfce00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efbf8198630 con 0x7efbf8068d30 2024-03-21T10:45:43.020 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.022+0000 7efbf7400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efbf8068410 0x7efbf819e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:45:43.020 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.022+0000 7efbf7400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efbf8068410 0x7efbf819e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:54718/0 (socket says 172.21.15.153:54718) 2024-03-21T10:45:43.020 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.022+0000 7efbf7400700 1 -- 172.21.15.153:0/344374564 learned_addr learned my addr 172.21.15.153:0/344374564 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:45:43.020 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.023+0000 7efbf7400700 1 -- 172.21.15.153:0/344374564 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efbf8068d30 msgr2=0x7efbf819ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:45:43.020 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.023+0000 7efbeea00700 1 --2- 172.21.15.153:0/344374564 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efbf8068d30 0x7efbf819ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:45:43.020 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.023+0000 7efbf7400700 1 --2- 172.21.15.153:0/344374564 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efbf8068d30 0x7efbf819ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:45:43.021 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.023+0000 7efbf7400700 1 -- 172.21.15.153:0/344374564 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efbe00095f0 con 0x7efbf8068410 2024-03-21T10:45:43.021 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.023+0000 7efbf7400700 1 --2- 172.21.15.153:0/344374564 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efbf8068410 0x7efbf819e540 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7efbe8007e90 tx=0x7efbe800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:45:43.021 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.023+0000 7efbf4c00700 1 -- 172.21.15.153:0/344374564 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efbe8017070 con 0x7efbf8068410 2024-03-21T10:45:43.021 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.024+0000 7efbf4c00700 1 -- 172.21.15.153:0/344374564 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efbe800f040 con 0x7efbf8068410 2024-03-21T10:45:43.021 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.024+0000 7efbfce00700 1 -- 172.21.15.153:0/344374564 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efbf8198910 con 0x7efbf8068410 2024-03-21T10:45:43.022 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.024+0000 7efbf4c00700 1 -- 172.21.15.153:0/344374564 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efbe80048f0 con 0x7efbf8068410 2024-03-21T10:45:43.022 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.024+0000 7efbfce00700 1 -- 172.21.15.153:0/344374564 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efbf8198e60 con 0x7efbf8068410 2024-03-21T10:45:43.022 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.025+0000 7efbfce00700 1 -- 172.21.15.153:0/344374564 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efbf810a7f0 con 0x7efbf8068410 2024-03-21T10:45:43.023 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.026+0000 7efbf4c00700 1 -- 172.21.15.153:0/344374564 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7efbe8004120 con 0x7efbf8068410 2024-03-21T10:45:43.024 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.027+0000 7efbf4c00700 1 --2- 172.21.15.153:0/344374564 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7efbdc06e400 0x7efbdc0708c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:45:43.024 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.027+0000 7efbf4c00700 1 -- 172.21.15.153:0/344374564 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7efbe808df00 con 0x7efbf8068410 2024-03-21T10:45:43.024 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.027+0000 7efbeea00700 1 --2- 172.21.15.153:0/344374564 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7efbdc06e400 0x7efbdc0708c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:45:43.025 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.028+0000 7efbeea00700 1 --2- 172.21.15.153:0/344374564 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7efbdc06e400 0x7efbdc0708c0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7efbe0006010 tx=0x7efbe000b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:45:43.028 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.031+0000 7efbf4c00700 1 -- 172.21.15.153:0/344374564 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7efbe805aaf0 con 0x7efbf8068410 2024-03-21T10:45:43.269 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.272+0000 7efbfce00700 1 -- 172.21.15.153:0/344374564 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7efbf8199100 con 0x7efbdc06e400 2024-03-21T10:45:43.272 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.275+0000 7efbf4c00700 1 -- 172.21.15.153:0/344374564 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7efbf8199100 con 0x7efbdc06e400 2024-03-21T10:45:43.275 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.278+0000 7efbfce00700 1 -- 172.21.15.153:0/344374564 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7efbdc06e400 msgr2=0x7efbdc0708c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:45:43.276 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.278+0000 7efbfce00700 1 --2- 172.21.15.153:0/344374564 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7efbdc06e400 0x7efbdc0708c0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7efbe0006010 tx=0x7efbe000b540 comp rx=0 tx=0).stop 2024-03-21T10:45:43.276 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.278+0000 7efbfce00700 1 -- 172.21.15.153:0/344374564 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efbf8068410 msgr2=0x7efbf819e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:45:43.276 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.278+0000 7efbfce00700 1 --2- 172.21.15.153:0/344374564 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efbf8068410 0x7efbf819e540 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7efbe8007e90 tx=0x7efbe800d3b0 comp rx=0 tx=0).stop 2024-03-21T10:45:43.276 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.278+0000 7efbfce00700 1 -- 172.21.15.153:0/344374564 shutdown_connections 2024-03-21T10:45:43.276 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.278+0000 7efbfce00700 1 --2- 172.21.15.153:0/344374564 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7efbdc06e400 0x7efbdc0708c0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:45:43.276 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.278+0000 7efbfce00700 1 --2- 172.21.15.153:0/344374564 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7efbf8068410 0x7efbf819e540 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:45:43.276 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.278+0000 7efbfce00700 1 --2- 172.21.15.153:0/344374564 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efbf8068d30 0x7efbf819ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:45:43.276 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.278+0000 7efbfce00700 1 -- 172.21.15.153:0/344374564 >> 172.21.15.153:0/344374564 conn(0x7efbf8075330 msgr2=0x7efbf80fe530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:45:43.276 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.279+0000 7efbfce00700 1 -- 172.21.15.153:0/344374564 shutdown_connections 2024-03-21T10:45:43.276 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.279+0000 7efbfce00700 1 -- 172.21.15.153:0/344374564 wait complete. 2024-03-21T10:45:43.436 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.438+0000 7fe169000700 1 -- 172.21.15.153:0/2489841325 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe164073ae0 msgr2=0x7fe16410ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:45:43.436 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.438+0000 7fe169000700 1 --2- 172.21.15.153:0/2489841325 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe164073ae0 0x7fe16410ce40 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fe150009910 tx=0x7fe150009c20 comp rx=0 tx=0).stop 2024-03-21T10:45:43.436 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.439+0000 7fe169000700 1 -- 172.21.15.153:0/2489841325 shutdown_connections 2024-03-21T10:45:43.436 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.439+0000 7fe169000700 1 --2- 172.21.15.153:0/2489841325 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe164073ae0 0x7fe16410ce40 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:45:43.436 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.439+0000 7fe169000700 1 --2- 172.21.15.153:0/2489841325 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe1640731c0 0x7fe1640735a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:45:43.436 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.439+0000 7fe169000700 1 -- 172.21.15.153:0/2489841325 >> 172.21.15.153:0/2489841325 conn(0x7fe1640fc630 msgr2=0x7fe1640fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:45:43.436 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.439+0000 7fe169000700 1 -- 172.21.15.153:0/2489841325 shutdown_connections 2024-03-21T10:45:43.437 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.439+0000 7fe169000700 1 -- 172.21.15.153:0/2489841325 wait complete. 2024-03-21T10:45:43.437 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.440+0000 7fe169000700 1 Processor -- start 2024-03-21T10:45:43.437 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.440+0000 7fe169000700 1 -- start start 2024-03-21T10:45:43.438 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.440+0000 7fe169000700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe1640731c0 0x7fe164198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:45:43.438 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.440+0000 7fe169000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe164073ae0 0x7fe164198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:45:43.438 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.440+0000 7fe169000700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe164199530 con 0x7fe164073ae0 2024-03-21T10:45:43.438 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.440+0000 7fe169000700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe16419d340 con 0x7fe1640731c0 2024-03-21T10:45:43.438 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.441+0000 7fe161400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe164073ae0 0x7fe164198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:45:43.438 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.441+0000 7fe161400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe164073ae0 0x7fe164198ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:54728/0 (socket says 172.21.15.153:54728) 2024-03-21T10:45:43.438 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.441+0000 7fe161400700 1 -- 172.21.15.153:0/27184854 learned_addr learned my addr 172.21.15.153:0/27184854 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:45:43.439 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.441+0000 7fe161e00700 1 --2- 172.21.15.153:0/27184854 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe1640731c0 0x7fe164198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:45:43.439 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.441+0000 7fe161400700 1 -- 172.21.15.153:0/27184854 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe1640731c0 msgr2=0x7fe164198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:45:43.439 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.441+0000 7fe161400700 1 --2- 172.21.15.153:0/27184854 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe1640731c0 0x7fe164198960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:45:43.439 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.441+0000 7fe161400700 1 -- 172.21.15.153:0/27184854 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1500095f0 con 0x7fe164073ae0 2024-03-21T10:45:43.439 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.442+0000 7fe161400700 1 --2- 172.21.15.153:0/27184854 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe164073ae0 0x7fe164198ea0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fe150000c00 tx=0x7fe15000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:45:43.440 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.442+0000 7fe15aa00700 1 -- 172.21.15.153:0/27184854 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe15001d070 con 0x7fe164073ae0 2024-03-21T10:45:43.440 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.442+0000 7fe15aa00700 1 -- 172.21.15.153:0/27184854 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe15000f460 con 0x7fe164073ae0 2024-03-21T10:45:43.440 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.442+0000 7fe15aa00700 1 -- 172.21.15.153:0/27184854 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe150005e20 con 0x7fe164073ae0 2024-03-21T10:45:43.440 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.442+0000 7fe169000700 1 -- 172.21.15.153:0/27184854 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe16419d5c0 con 0x7fe164073ae0 2024-03-21T10:45:43.440 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.442+0000 7fe169000700 1 -- 172.21.15.153:0/27184854 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe16419dab0 con 0x7fe164073ae0 2024-03-21T10:45:43.440 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.443+0000 7fe169000700 1 -- 172.21.15.153:0/27184854 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe16410a540 con 0x7fe164073ae0 2024-03-21T10:45:43.441 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.444+0000 7fe15aa00700 1 -- 172.21.15.153:0/27184854 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fe15000f5d0 con 0x7fe164073ae0 2024-03-21T10:45:43.442 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.445+0000 7fe15aa00700 1 --2- 172.21.15.153:0/27184854 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe14806e460 0x7fe148070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:45:43.442 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.445+0000 7fe15aa00700 1 -- 172.21.15.153:0/27184854 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fe15008f9a0 con 0x7fe164073ae0 2024-03-21T10:45:43.442 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.445+0000 7fe161e00700 1 --2- 172.21.15.153:0/27184854 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe14806e460 0x7fe148070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:45:43.443 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.446+0000 7fe161e00700 1 --2- 172.21.15.153:0/27184854 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe14806e460 0x7fe148070920 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fe154009af0 tx=0x7fe1540094f0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:45:43.447 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.450+0000 7fe15aa00700 1 -- 172.21.15.153:0/27184854 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe15005c630 con 0x7fe164073ae0 2024-03-21T10:45:43.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:45:43 smithi153 ceph-mon[113733]: pgmap v1017: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:45:43.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:45:43 smithi174 ceph-mon[88004]: pgmap v1017: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:45:43.697 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.700+0000 7fe169000700 1 -- 172.21.15.153:0/27184854 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fe16402cce0 con 0x7fe14806e460 2024-03-21T10:45:43.713 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.713+0000 7fe15aa00700 1 -- 172.21.15.153:0/27184854 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fe16402cce0 con 0x7fe14806e460 2024-03-21T10:45:43.714 INFO:teuthology.orchestra.run.smithi153.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-21T10:45:43.714 INFO:teuthology.orchestra.run.smithi153.stdout:alertmanager.smithi153 smithi153 *:9093,9094 running (34m) 10m ago 48m 15.6M - 0.25.0 c8568f914cd2 9ed3f2200b5f 2024-03-21T10:45:43.714 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi153 smithi153 running (48m) 10m ago 48m 10.4M - 18.2.1 5be31c24972a 5b61994396a6 2024-03-21T10:45:43.714 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi174 smithi174 running (47m) 10m ago 47m 10.1M - 18.2.1 5be31c24972a 4f7a5fc3ac28 2024-03-21T10:45:43.714 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi153 smithi153 running (30m) 10m ago 48m 6656k - 19.0.0-2231-g2edfee68 237e43c7a65f 32c21104d5b0 2024-03-21T10:45:43.714 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi174 smithi174 running (30m) 10m ago 46m 6660k - 19.0.0-2231-g2edfee68 237e43c7a65f a04e48967ab6 2024-03-21T10:45:43.714 INFO:teuthology.orchestra.run.smithi153.stdout:grafana.smithi153 smithi153 *:3000 running (34m) 10m ago 47m 75.2M - 9.4.12 2bacad6d85d8 ee24026ab786 2024-03-21T10:45:43.714 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.iuyluj smithi153 running (26m) 10m ago 41m 18.7M - 19.0.0-2231-g2edfee68 237e43c7a65f 87aff8a5de70 2024-03-21T10:45:43.714 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.zipckr smithi153 running (26m) 10m ago 41m 18.9M - 19.0.0-2231-g2edfee68 237e43c7a65f ccc262bd5dd3 2024-03-21T10:45:43.714 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.nhozag smithi174 running (10m) 10m ago 40m 10.9M - 19.0.0-2231-g2edfee68 237e43c7a65f 9e02aea72c96 2024-03-21T10:45:43.714 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.yyimst smithi174 running (25m) 10m ago 41m 18.5M - 19.0.0-2231-g2edfee68 237e43c7a65f fc1962f9351e 2024-03-21T10:45:43.715 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi153.tluxni smithi153 *:8443,9283,8765 running (37m) 10m ago 50m 512M - 19.0.0-2231-g2edfee68 237e43c7a65f 7ffee0e55382 2024-03-21T10:45:43.715 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi174.qpcwat smithi174 *:8443,9283,8765 running (36m) 10m ago 46m 459M - 19.0.0-2231-g2edfee68 237e43c7a65f 748968f216ab 2024-03-21T10:45:43.715 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi153 smithi153 running (32m) 10m ago 50m 70.8M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f dd8038303dc3 2024-03-21T10:45:43.715 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi174 smithi174 running (31m) 10m ago 46m 51.9M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f 29d78a788f2c 2024-03-21T10:45:43.715 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi153 smithi153 *:9100 running (35m) 10m ago 48m 8140k - 1.5.0 0da6a335fe13 bb835edc0e0b 2024-03-21T10:45:43.715 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi174 smithi174 *:9100 running (35m) 10m ago 46m 8610k - 1.5.0 0da6a335fe13 7c4ecad790fa 2024-03-21T10:45:43.715 INFO:teuthology.orchestra.run.smithi153.stdout:osd.0 smithi153 running (30m) 10m ago 45m 118M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 507900fd4791 2024-03-21T10:45:43.715 INFO:teuthology.orchestra.run.smithi153.stdout:osd.1 smithi153 running (29m) 10m ago 45m 93.0M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 0d646452e9c1 2024-03-21T10:45:43.715 INFO:teuthology.orchestra.run.smithi153.stdout:osd.2 smithi153 running (28m) 10m ago 44m 91.2M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 1326f2afdfcd 2024-03-21T10:45:43.715 INFO:teuthology.orchestra.run.smithi153.stdout:osd.3 smithi174 running (28m) 10m ago 44m 87.5M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 70aafac71ad5 2024-03-21T10:45:43.715 INFO:teuthology.orchestra.run.smithi153.stdout:osd.4 smithi174 running (27m) 10m ago 43m 95.8M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 0d746ce6e79d 2024-03-21T10:45:43.715 INFO:teuthology.orchestra.run.smithi153.stdout:osd.5 smithi174 running (26m) 10m ago 42m 90.3M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 53e5f35b8cac 2024-03-21T10:45:43.715 INFO:teuthology.orchestra.run.smithi153.stdout:prometheus.smithi153 smithi153 *:9095 running (34m) 10m ago 47m 88.1M - 2.43.0 a07b618ecd1d 7d95561e50c2 2024-03-21T10:45:43.716 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.716+0000 7fe169000700 1 -- 172.21.15.153:0/27184854 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe14806e460 msgr2=0x7fe148070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:45:43.716 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.716+0000 7fe169000700 1 --2- 172.21.15.153:0/27184854 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe14806e460 0x7fe148070920 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fe154009af0 tx=0x7fe1540094f0 comp rx=0 tx=0).stop 2024-03-21T10:45:43.716 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.716+0000 7fe169000700 1 -- 172.21.15.153:0/27184854 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe164073ae0 msgr2=0x7fe164198ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:45:43.716 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.716+0000 7fe169000700 1 --2- 172.21.15.153:0/27184854 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe164073ae0 0x7fe164198ea0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fe150000c00 tx=0x7fe15000be20 comp rx=0 tx=0).stop 2024-03-21T10:45:43.716 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.716+0000 7fe169000700 1 -- 172.21.15.153:0/27184854 shutdown_connections 2024-03-21T10:45:43.716 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.716+0000 7fe169000700 1 --2- 172.21.15.153:0/27184854 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe1640731c0 0x7fe164198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:45:43.716 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.716+0000 7fe169000700 1 --2- 172.21.15.153:0/27184854 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe14806e460 0x7fe148070920 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:45:43.716 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.716+0000 7fe169000700 1 --2- 172.21.15.153:0/27184854 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe164073ae0 0x7fe164198ea0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:45:43.716 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.716+0000 7fe169000700 1 -- 172.21.15.153:0/27184854 >> 172.21.15.153:0/27184854 conn(0x7fe1640fc630 msgr2=0x7fe164107680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:45:43.716 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.716+0000 7fe169000700 1 -- 172.21.15.153:0/27184854 shutdown_connections 2024-03-21T10:45:43.717 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.717+0000 7fe169000700 1 -- 172.21.15.153:0/27184854 wait complete. 2024-03-21T10:45:43.872 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.875+0000 7f256ea00700 1 -- 172.21.15.153:0/1752053173 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2568068670 msgr2=0x7f2568068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:45:43.872 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.875+0000 7f256ea00700 1 --2- 172.21.15.153:0/1752053173 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2568068670 0x7f2568068a50 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f255800b0a0 tx=0x7f255800b3b0 comp rx=0 tx=0).stop 2024-03-21T10:45:43.872 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.875+0000 7f256ea00700 1 -- 172.21.15.153:0/1752053173 shutdown_connections 2024-03-21T10:45:43.872 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.875+0000 7f256ea00700 1 --2- 172.21.15.153:0/1752053173 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2568069020 0x7f2568105790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:45:43.873 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.875+0000 7f256ea00700 1 --2- 172.21.15.153:0/1752053173 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2568068670 0x7f2568068a50 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:45:43.873 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.875+0000 7f256ea00700 1 -- 172.21.15.153:0/1752053173 >> 172.21.15.153:0/1752053173 conn(0x7f25680755a0 msgr2=0x7f25680759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:45:43.873 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.875+0000 7f256ea00700 1 -- 172.21.15.153:0/1752053173 shutdown_connections 2024-03-21T10:45:43.873 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.875+0000 7f256ea00700 1 -- 172.21.15.153:0/1752053173 wait complete. 2024-03-21T10:45:43.874 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.876+0000 7f256ea00700 1 Processor -- start 2024-03-21T10:45:43.874 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.876+0000 7f256ea00700 1 -- start start 2024-03-21T10:45:43.875 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.876+0000 7f256ea00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2568068670 0x7f256819c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:45:43.875 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.877+0000 7f256ea00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2568069020 0x7f256819cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:45:43.875 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.877+0000 7f256ea00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f256819d150 con 0x7f2568069020 2024-03-21T10:45:43.875 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.877+0000 7f256ea00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2568196600 con 0x7f2568068670 2024-03-21T10:45:43.875 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.877+0000 7f255f400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2568069020 0x7f256819cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:45:43.875 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.877+0000 7f255f400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2568069020 0x7f256819cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:54752/0 (socket says 172.21.15.153:54752) 2024-03-21T10:45:43.875 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.877+0000 7f255f400700 1 -- 172.21.15.153:0/2112115153 learned_addr learned my addr 172.21.15.153:0/2112115153 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:45:43.875 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.877+0000 7f2567e00700 1 --2- 172.21.15.153:0/2112115153 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2568068670 0x7f256819c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:45:43.875 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.877+0000 7f255f400700 1 -- 172.21.15.153:0/2112115153 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2568068670 msgr2=0x7f256819c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:45:43.875 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.877+0000 7f255f400700 1 --2- 172.21.15.153:0/2112115153 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2568068670 0x7f256819c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:45:43.875 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.877+0000 7f255f400700 1 -- 172.21.15.153:0/2112115153 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2558009d00 con 0x7f2568069020 2024-03-21T10:45:43.876 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.878+0000 7f255f400700 1 --2- 172.21.15.153:0/2112115153 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2568069020 0x7f256819cac0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f255000d900 tx=0x7f255000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:45:43.876 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.878+0000 7f2565600700 1 -- 172.21.15.153:0/2112115153 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2550009900 con 0x7f2568069020 2024-03-21T10:45:43.876 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.878+0000 7f2565600700 1 -- 172.21.15.153:0/2112115153 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2550010460 con 0x7f2568069020 2024-03-21T10:45:43.876 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.878+0000 7f256ea00700 1 -- 172.21.15.153:0/2112115153 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25681968e0 con 0x7f2568069020 2024-03-21T10:45:43.876 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.878+0000 7f2565600700 1 -- 172.21.15.153:0/2112115153 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f255000f5f0 con 0x7f2568069020 2024-03-21T10:45:43.876 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.878+0000 7f256ea00700 1 -- 172.21.15.153:0/2112115153 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2568196e30 con 0x7f2568069020 2024-03-21T10:45:43.877 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.880+0000 7f256ea00700 1 -- 172.21.15.153:0/2112115153 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2568109130 con 0x7f2568069020 2024-03-21T10:45:43.877 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.881+0000 7f2565600700 1 -- 172.21.15.153:0/2112115153 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f25500105d0 con 0x7f2568069020 2024-03-21T10:45:43.879 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.881+0000 7f2565600700 1 --2- 172.21.15.153:0/2112115153 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f254c06e460 0x7f254c070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:45:43.879 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.882+0000 7f2565600700 1 -- 172.21.15.153:0/2112115153 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f255008e360 con 0x7f2568069020 2024-03-21T10:45:43.879 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.882+0000 7f2567e00700 1 --2- 172.21.15.153:0/2112115153 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f254c06e460 0x7f254c070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:45:43.879 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.882+0000 7f2567e00700 1 --2- 172.21.15.153:0/2112115153 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f254c06e460 0x7f254c070920 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f2558009480 tx=0x7f255800b7f0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:45:43.883 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:43.886+0000 7f2565600700 1 -- 172.21.15.153:0/2112115153 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f255005af30 con 0x7f2568069020 2024-03-21T10:45:44.195 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:44.198+0000 7f256ea00700 1 -- 172.21.15.153:0/2112115153 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f256804ea90 con 0x7f2568069020 2024-03-21T10:45:44.196 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:44.199+0000 7f2565600700 1 -- 172.21.15.153:0/2112115153 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f255005aac0 con 0x7f2568069020 2024-03-21T10:45:44.196 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:45:44.196 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": { 2024-03-21T10:45:44.196 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:45:44.196 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:45:44.196 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": { 2024-03-21T10:45:44.196 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:45:44.196 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:45:44.197 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": { 2024-03-21T10:45:44.197 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 6 2024-03-21T10:45:44.197 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:45:44.197 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": { 2024-03-21T10:45:44.197 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 4 2024-03-21T10:45:44.197 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:45:44.197 INFO:teuthology.orchestra.run.smithi153.stdout: "overall": { 2024-03-21T10:45:44.197 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 14 2024-03-21T10:45:44.197 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-03-21T10:45:44.197 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:45:44.199 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:44.201+0000 7f256ea00700 1 -- 172.21.15.153:0/2112115153 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f254c06e460 msgr2=0x7f254c070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:45:44.199 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:44.201+0000 7f256ea00700 1 --2- 172.21.15.153:0/2112115153 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f254c06e460 0x7f254c070920 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f2558009480 tx=0x7f255800b7f0 comp rx=0 tx=0).stop 2024-03-21T10:45:44.199 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:44.201+0000 7f256ea00700 1 -- 172.21.15.153:0/2112115153 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2568069020 msgr2=0x7f256819cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:45:44.199 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:44.201+0000 7f256ea00700 1 --2- 172.21.15.153:0/2112115153 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2568069020 0x7f256819cac0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f255000d900 tx=0x7f255000dcc0 comp rx=0 tx=0).stop 2024-03-21T10:45:44.199 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:44.201+0000 7f256ea00700 1 -- 172.21.15.153:0/2112115153 shutdown_connections 2024-03-21T10:45:44.199 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:44.201+0000 7f256ea00700 1 --2- 172.21.15.153:0/2112115153 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2568068670 0x7f256819c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:45:44.199 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:44.201+0000 7f256ea00700 1 --2- 172.21.15.153:0/2112115153 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f254c06e460 0x7f254c070920 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:45:44.199 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:44.201+0000 7f256ea00700 1 --2- 172.21.15.153:0/2112115153 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f2568069020 0x7f256819cac0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:45:44.200 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:44.201+0000 7f256ea00700 1 -- 172.21.15.153:0/2112115153 >> 172.21.15.153:0/2112115153 conn(0x7f25680755a0 msgr2=0x7f25680fe740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:45:44.200 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:44.202+0000 7f256ea00700 1 -- 172.21.15.153:0/2112115153 shutdown_connections 2024-03-21T10:45:44.200 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:44.202+0000 7f256ea00700 1 -- 172.21.15.153:0/2112115153 wait complete. 2024-03-21T10:45:44.355 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:44.358+0000 7f7018a00700 1 -- 172.21.15.153:0/281406880 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f70141071d0 msgr2=0x7f7014107650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:45:44.356 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:44.358+0000 7f7018a00700 1 --2- 172.21.15.153:0/281406880 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f70141071d0 0x7f7014107650 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f7000009910 tx=0x7f7000009c20 comp rx=0 tx=0).stop 2024-03-21T10:45:44.356 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:44.358+0000 7f7018a00700 1 -- 172.21.15.153:0/281406880 shutdown_connections 2024-03-21T10:45:44.356 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:44.358+0000 7f7018a00700 1 --2- 172.21.15.153:0/281406880 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f70141071d0 0x7f7014107650 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:45:44.356 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:44.358+0000 7f7018a00700 1 --2- 172.21.15.153:0/281406880 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f701410d280 0x7f701410d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:45:44.356 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:44.358+0000 7f7018a00700 1 -- 172.21.15.153:0/281406880 >> 172.21.15.153:0/281406880 conn(0x7f70140769e0 msgr2=0x7f7014078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:45:44.356 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:44.358+0000 7f7018a00700 1 -- 172.21.15.153:0/281406880 shutdown_connections 2024-03-21T10:45:44.356 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:44.358+0000 7f7018a00700 1 -- 172.21.15.153:0/281406880 wait complete. 2024-03-21T10:45:44.357 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:44.359+0000 7f7018a00700 1 Processor -- start 2024-03-21T10:45:44.358 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:44.359+0000 7f7018a00700 1 -- start start 2024-03-21T10:45:44.358 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:44.359+0000 7f7018a00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f70141071d0 0x7f701419ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:45:44.358 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:44.359+0000 7f7018a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f701410d280 0x7f701419d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:45:44.358 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:44.359+0000 7f7018a00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f701419da30 con 0x7f701410d280 2024-03-21T10:45:44.358 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:44.359+0000 7f7018a00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70141a17c0 con 0x7f70141071d0 2024-03-21T10:45:44.358 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:44.360+0000 7f7011400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f701410d280 0x7f701419d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:45:44.358 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:44.360+0000 7f7011400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f701410d280 0x7f701419d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:54778/0 (socket says 172.21.15.153:54778) 2024-03-21T10:45:44.358 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:44.360+0000 7f7011400700 1 -- 172.21.15.153:0/874948975 learned_addr learned my addr 172.21.15.153:0/874948975 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:45:44.359 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:44.360+0000 7f7011e00700 1 --2- 172.21.15.153:0/874948975 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f70141071d0 0x7f701419ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:45:44.359 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:44.360+0000 7f7011400700 1 -- 172.21.15.153:0/874948975 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f70141071d0 msgr2=0x7f701419ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:45:44.359 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:44.360+0000 7f7011400700 1 --2- 172.21.15.153:0/874948975 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f70141071d0 0x7f701419ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:45:44.359 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:44.360+0000 7f7011400700 1 -- 172.21.15.153:0/874948975 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70000095f0 con 0x7f701410d280 2024-03-21T10:45:44.359 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:44.361+0000 7f7011400700 1 --2- 172.21.15.153:0/874948975 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f701410d280 0x7f701419d350 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f700000b5c0 tx=0x7f700000be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:45:44.359 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:44.361+0000 7f700aa00700 1 -- 172.21.15.153:0/874948975 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f700001d070 con 0x7f701410d280 2024-03-21T10:45:44.360 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:44.361+0000 7f700aa00700 1 -- 172.21.15.153:0/874948975 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f700000f460 con 0x7f701410d280 2024-03-21T10:45:44.360 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:44.361+0000 7f7018a00700 1 -- 172.21.15.153:0/874948975 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70141a1a40 con 0x7f701410d280 2024-03-21T10:45:44.360 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:44.361+0000 7f700aa00700 1 -- 172.21.15.153:0/874948975 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7000017610 con 0x7f701410d280 2024-03-21T10:45:44.360 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:44.361+0000 7f7018a00700 1 -- 172.21.15.153:0/874948975 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70141a1fb0 con 0x7f701410d280 2024-03-21T10:45:44.360 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:44.362+0000 7f7018a00700 1 -- 172.21.15.153:0/874948975 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f701410f710 con 0x7f701410d280 2024-03-21T10:45:44.360 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:44.363+0000 7f700aa00700 1 -- 172.21.15.153:0/874948975 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f700000f5d0 con 0x7f701410d280 2024-03-21T10:45:44.361 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:44.364+0000 7f700aa00700 1 --2- 172.21.15.153:0/874948975 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f6ff806e460 0x7f6ff8070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:45:44.361 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:44.364+0000 7f700aa00700 1 -- 172.21.15.153:0/874948975 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f700008fad0 con 0x7f701410d280 2024-03-21T10:45:44.361 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:44.364+0000 7f7011e00700 1 --2- 172.21.15.153:0/874948975 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f6ff806e460 0x7f6ff8070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:45:44.362 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:44.365+0000 7f7011e00700 1 --2- 172.21.15.153:0/874948975 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f6ff806e460 0x7f6ff8070920 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f7004006fd0 tx=0x7f7004008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:45:44.366 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:44.370+0000 7f700aa00700 1 -- 172.21.15.153:0/874948975 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7000058b70 con 0x7f701410d280 2024-03-21T10:45:44.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:45:44 smithi153 ceph-mon[113733]: from='client.35798 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:45:44.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:45:44 smithi153 ceph-mon[113733]: from='client.35802 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:45:44.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:45:44 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/2112115153' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:45:44.634 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:44.637+0000 7f7018a00700 1 -- 172.21.15.153:0/874948975 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f701404ea90 con 0x7f701410d280 2024-03-21T10:45:44.635 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:44.638+0000 7f700aa00700 1 -- 172.21.15.153:0/874948975 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1876 (secure 0 0 0) 0x7f700005c230 con 0x7f701410d280 2024-03-21T10:45:44.636 INFO:teuthology.orchestra.run.smithi153.stdout:e32 2024-03-21T10:45:44.636 INFO:teuthology.orchestra.run.smithi153.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-21T10:45:44.636 INFO:teuthology.orchestra.run.smithi153.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:45:44.636 INFO:teuthology.orchestra.run.smithi153.stdout:legacy client fscid: 1 2024-03-21T10:45:44.636 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:45:44.636 INFO:teuthology.orchestra.run.smithi153.stdout:Filesystem 'cephfs' (1) 2024-03-21T10:45:44.636 INFO:teuthology.orchestra.run.smithi153.stdout:fs_name cephfs 2024-03-21T10:45:44.637 INFO:teuthology.orchestra.run.smithi153.stdout:epoch 23 2024-03-21T10:45:44.637 INFO:teuthology.orchestra.run.smithi153.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-21T10:45:44.637 INFO:teuthology.orchestra.run.smithi153.stdout:created 2024-03-21T10:04:27.086576+0000 2024-03-21T10:45:44.637 INFO:teuthology.orchestra.run.smithi153.stdout:modified 2024-03-21T10:19:05.022092+0000 2024-03-21T10:45:44.637 INFO:teuthology.orchestra.run.smithi153.stdout:tableserver 0 2024-03-21T10:45:44.637 INFO:teuthology.orchestra.run.smithi153.stdout:root 0 2024-03-21T10:45:44.637 INFO:teuthology.orchestra.run.smithi153.stdout:session_timeout 60 2024-03-21T10:45:44.637 INFO:teuthology.orchestra.run.smithi153.stdout:session_autoclose 300 2024-03-21T10:45:44.637 INFO:teuthology.orchestra.run.smithi153.stdout:max_file_size 1099511627776 2024-03-21T10:45:44.637 INFO:teuthology.orchestra.run.smithi153.stdout:max_xattr_size 65536 2024-03-21T10:45:44.637 INFO:teuthology.orchestra.run.smithi153.stdout:required_client_features {} 2024-03-21T10:45:44.637 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure 0 2024-03-21T10:45:44.637 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure_osd_epoch 94 2024-03-21T10:45:44.637 INFO:teuthology.orchestra.run.smithi153.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:45:44.637 INFO:teuthology.orchestra.run.smithi153.stdout:max_mds 2 2024-03-21T10:45:44.638 INFO:teuthology.orchestra.run.smithi153.stdout:in 0,1 2024-03-21T10:45:44.638 INFO:teuthology.orchestra.run.smithi153.stdout:up {} 2024-03-21T10:45:44.638 INFO:teuthology.orchestra.run.smithi153.stdout:failed 0,1 2024-03-21T10:45:44.638 INFO:teuthology.orchestra.run.smithi153.stdout:damaged 2024-03-21T10:45:44.638 INFO:teuthology.orchestra.run.smithi153.stdout:stopped 2024-03-21T10:45:44.638 INFO:teuthology.orchestra.run.smithi153.stdout:data_pools [3] 2024-03-21T10:45:44.638 INFO:teuthology.orchestra.run.smithi153.stdout:metadata_pool 2 2024-03-21T10:45:44.638 INFO:teuthology.orchestra.run.smithi153.stdout:inline_data enabled 2024-03-21T10:45:44.638 INFO:teuthology.orchestra.run.smithi153.stdout:balancer 2024-03-21T10:45:44.638 INFO:teuthology.orchestra.run.smithi153.stdout:bal_rank_mask -1 2024-03-21T10:45:44.638 INFO:teuthology.orchestra.run.smithi153.stdout:standby_count_wanted 1 2024-03-21T10:45:44.638 INFO:teuthology.orchestra.run.smithi153.stdout:qdb_cluster leader: 0 members: 2024-03-21T10:45:44.638 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:45:44.638 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:45:44.638 INFO:teuthology.orchestra.run.smithi153.stdout:Standby daemons: 2024-03-21T10:45:44.639 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:45:44.639 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.zipckr{-1:34442} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6826/1103846295,v1:172.21.15.153:6827/1103846295] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:45:44.639 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.iuyluj{-1:34474} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6828/1738630832,v1:172.21.15.153:6829/1738630832] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:45:44.639 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.yyimst{-1:44313} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.174:6824/4127786797,v1:172.21.15.174:6825/4127786797] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:45:44.639 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.nhozag{-1:44687} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.174:6826/2767517845,v1:172.21.15.174:6827/2767517845] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:45:44.639 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:44.640+0000 7f7018a00700 1 -- 172.21.15.153:0/874948975 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f6ff806e460 msgr2=0x7f6ff8070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:45:44.640 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:44.640+0000 7f7018a00700 1 --2- 172.21.15.153:0/874948975 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f6ff806e460 0x7f6ff8070920 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f7004006fd0 tx=0x7f7004008040 comp rx=0 tx=0).stop 2024-03-21T10:45:44.640 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:44.640+0000 7f7018a00700 1 -- 172.21.15.153:0/874948975 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f701410d280 msgr2=0x7f701419d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:45:44.640 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:44.640+0000 7f7018a00700 1 --2- 172.21.15.153:0/874948975 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f701410d280 0x7f701419d350 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f700000b5c0 tx=0x7f700000be00 comp rx=0 tx=0).stop 2024-03-21T10:45:44.640 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:44.641+0000 7f7018a00700 1 -- 172.21.15.153:0/874948975 shutdown_connections 2024-03-21T10:45:44.640 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:44.641+0000 7f7018a00700 1 --2- 172.21.15.153:0/874948975 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f70141071d0 0x7f701419ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:45:44.640 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:44.641+0000 7f7018a00700 1 --2- 172.21.15.153:0/874948975 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f6ff806e460 0x7f6ff8070920 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:45:44.640 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:44.641+0000 7f7018a00700 1 --2- 172.21.15.153:0/874948975 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f701410d280 0x7f701419d350 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:45:44.640 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:44.641+0000 7f7018a00700 1 -- 172.21.15.153:0/874948975 >> 172.21.15.153:0/874948975 conn(0x7f70140769e0 msgr2=0x7f70140784d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:45:44.640 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:44.641+0000 7f7018a00700 1 -- 172.21.15.153:0/874948975 shutdown_connections 2024-03-21T10:45:44.640 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:44.641+0000 7f7018a00700 1 -- 172.21.15.153:0/874948975 wait complete. 2024-03-21T10:45:44.640 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 32 2024-03-21T10:45:44.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:45:44 smithi174 ceph-mon[88004]: from='client.35798 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:45:44.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:45:44 smithi174 ceph-mon[88004]: from='client.35802 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:45:44.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:45:44 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/2112115153' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:45:44.789 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:44.791+0000 7f9561200700 1 -- 172.21.15.153:0/2344240893 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f955c100da0 msgr2=0x7f955c101180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:45:44.789 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:44.791+0000 7f9561200700 1 --2- 172.21.15.153:0/2344240893 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f955c100da0 0x7f955c101180 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f954c009910 tx=0x7f954c009c20 comp rx=0 tx=0).stop 2024-03-21T10:45:44.789 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:44.791+0000 7f9561200700 1 -- 172.21.15.153:0/2344240893 shutdown_connections 2024-03-21T10:45:44.789 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:44.791+0000 7f9561200700 1 --2- 172.21.15.153:0/2344240893 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f955c101750 0x7f955c1057a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:45:44.789 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:44.791+0000 7f9561200700 1 --2- 172.21.15.153:0/2344240893 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f955c100da0 0x7f955c101180 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:45:44.789 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:44.791+0000 7f9561200700 1 -- 172.21.15.153:0/2344240893 >> 172.21.15.153:0/2344240893 conn(0x7f955c0fc630 msgr2=0x7f955c0fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:45:44.789 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:44.791+0000 7f9561200700 1 -- 172.21.15.153:0/2344240893 shutdown_connections 2024-03-21T10:45:44.789 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:44.791+0000 7f9561200700 1 -- 172.21.15.153:0/2344240893 wait complete. 2024-03-21T10:45:44.790 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:44.792+0000 7f9561200700 1 Processor -- start 2024-03-21T10:45:44.790 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:44.793+0000 7f9561200700 1 -- start start 2024-03-21T10:45:44.790 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:44.793+0000 7f9561200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f955c100da0 0x7f955c198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:45:44.790 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:44.793+0000 7f9561200700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f955c101750 0x7f955c198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:45:44.790 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:44.793+0000 7f9561200700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f955c199530 con 0x7f955c100da0 2024-03-21T10:45:44.791 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:44.793+0000 7f9561200700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f955c19d340 con 0x7f955c101750 2024-03-21T10:45:44.791 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:44.793+0000 7f9559e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f955c100da0 0x7f955c198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:45:44.791 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:44.793+0000 7f9559e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f955c100da0 0x7f955c198960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:54796/0 (socket says 172.21.15.153:54796) 2024-03-21T10:45:44.791 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:44.793+0000 7f9559e00700 1 -- 172.21.15.153:0/3703132583 learned_addr learned my addr 172.21.15.153:0/3703132583 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:45:44.792 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:44.794+0000 7f9559400700 1 --2- 172.21.15.153:0/3703132583 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f955c101750 0x7f955c198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:45:44.792 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:44.794+0000 7f9559e00700 1 -- 172.21.15.153:0/3703132583 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f955c101750 msgr2=0x7f955c198ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:45:44.792 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:44.794+0000 7f9559e00700 1 --2- 172.21.15.153:0/3703132583 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f955c101750 0x7f955c198ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:45:44.792 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:44.794+0000 7f9559e00700 1 -- 172.21.15.153:0/3703132583 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f954c0095f0 con 0x7f955c100da0 2024-03-21T10:45:44.792 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:44.794+0000 7f9559e00700 1 --2- 172.21.15.153:0/3703132583 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f955c100da0 0x7f955c198960 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f954c0092e0 tx=0x7f954c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:45:44.792 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:44.794+0000 7f9552a00700 1 -- 172.21.15.153:0/3703132583 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f954c01d070 con 0x7f955c100da0 2024-03-21T10:45:44.792 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:44.794+0000 7f9552a00700 1 -- 172.21.15.153:0/3703132583 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f954c005e00 con 0x7f955c100da0 2024-03-21T10:45:44.792 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:44.794+0000 7f9561200700 1 -- 172.21.15.153:0/3703132583 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f955c19d5c0 con 0x7f955c100da0 2024-03-21T10:45:44.792 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:44.794+0000 7f9552a00700 1 -- 172.21.15.153:0/3703132583 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f954c017400 con 0x7f955c100da0 2024-03-21T10:45:44.792 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:44.794+0000 7f9561200700 1 -- 172.21.15.153:0/3703132583 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f955c19dab0 con 0x7f955c100da0 2024-03-21T10:45:44.793 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:44.795+0000 7f9561200700 1 -- 172.21.15.153:0/3703132583 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f955c104f20 con 0x7f955c100da0 2024-03-21T10:45:44.793 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:44.796+0000 7f9552a00700 1 -- 172.21.15.153:0/3703132583 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f954c00f460 con 0x7f955c100da0 2024-03-21T10:45:44.794 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:44.797+0000 7f9552a00700 1 --2- 172.21.15.153:0/3703132583 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f954006e460 0x7f9540070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:45:44.794 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:44.797+0000 7f9552a00700 1 -- 172.21.15.153:0/3703132583 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f954c08f9b0 con 0x7f955c100da0 2024-03-21T10:45:44.794 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:44.799+0000 7f9559400700 1 --2- 172.21.15.153:0/3703132583 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f954006e460 0x7f9540070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:45:44.795 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:44.799+0000 7f9559400700 1 --2- 172.21.15.153:0/3703132583 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f954006e460 0x7f9540070920 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f9548005950 tx=0x7f95480058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:45:44.799 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:44.802+0000 7f9552a00700 1 -- 172.21.15.153:0/3703132583 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f954c05c580 con 0x7f955c100da0 2024-03-21T10:45:45.039 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:45.042+0000 7f9561200700 1 -- 172.21.15.153:0/3703132583 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f955c19dde0 con 0x7f954006e460 2024-03-21T10:45:45.042 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:45.045+0000 7f9552a00700 1 -- 172.21.15.153:0/3703132583 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f955c19dde0 con 0x7f954006e460 2024-03-21T10:45:45.042 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:45:45.042 INFO:teuthology.orchestra.run.smithi153.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232", 2024-03-21T10:45:45.042 INFO:teuthology.orchestra.run.smithi153.stdout: "in_progress": true, 2024-03-21T10:45:45.042 INFO:teuthology.orchestra.run.smithi153.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-21T10:45:45.042 INFO:teuthology.orchestra.run.smithi153.stdout: "services_complete": [ 2024-03-21T10:45:45.042 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr", 2024-03-21T10:45:45.042 INFO:teuthology.orchestra.run.smithi153.stdout: "mon", 2024-03-21T10:45:45.042 INFO:teuthology.orchestra.run.smithi153.stdout: "mds", 2024-03-21T10:45:45.042 INFO:teuthology.orchestra.run.smithi153.stdout: "crash", 2024-03-21T10:45:45.043 INFO:teuthology.orchestra.run.smithi153.stdout: "osd" 2024-03-21T10:45:45.043 INFO:teuthology.orchestra.run.smithi153.stdout: ], 2024-03-21T10:45:45.043 INFO:teuthology.orchestra.run.smithi153.stdout: "progress": "16/23 daemons upgraded", 2024-03-21T10:45:45.043 INFO:teuthology.orchestra.run.smithi153.stdout: "message": "Currently upgrading mds daemons", 2024-03-21T10:45:45.043 INFO:teuthology.orchestra.run.smithi153.stdout: "is_paused": false 2024-03-21T10:45:45.043 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:45:45.046 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:45.049+0000 7f9561200700 1 -- 172.21.15.153:0/3703132583 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f954006e460 msgr2=0x7f9540070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:45:45.046 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:45.049+0000 7f9561200700 1 --2- 172.21.15.153:0/3703132583 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f954006e460 0x7f9540070920 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f9548005950 tx=0x7f95480058e0 comp rx=0 tx=0).stop 2024-03-21T10:45:45.046 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:45.049+0000 7f9561200700 1 -- 172.21.15.153:0/3703132583 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f955c100da0 msgr2=0x7f955c198960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:45:45.046 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:45.049+0000 7f9561200700 1 --2- 172.21.15.153:0/3703132583 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f955c100da0 0x7f955c198960 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f954c0092e0 tx=0x7f954c00be20 comp rx=0 tx=0).stop 2024-03-21T10:45:45.046 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:45.049+0000 7f9561200700 1 -- 172.21.15.153:0/3703132583 shutdown_connections 2024-03-21T10:45:45.046 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:45.049+0000 7f9561200700 1 --2- 172.21.15.153:0/3703132583 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f954006e460 0x7f9540070920 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:45:45.046 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:45.049+0000 7f9561200700 1 --2- 172.21.15.153:0/3703132583 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f955c100da0 0x7f955c198960 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:45:45.046 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:45.049+0000 7f9561200700 1 --2- 172.21.15.153:0/3703132583 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f955c101750 0x7f955c198ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:45:45.046 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:45.049+0000 7f9561200700 1 -- 172.21.15.153:0/3703132583 >> 172.21.15.153:0/3703132583 conn(0x7f955c0fc630 msgr2=0x7f955c0fdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:45:45.046 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:45.050+0000 7f9561200700 1 -- 172.21.15.153:0/3703132583 shutdown_connections 2024-03-21T10:45:45.046 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:45.050+0000 7f9561200700 1 -- 172.21.15.153:0/3703132583 wait complete. 2024-03-21T10:45:45.197 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:45.201+0000 7f4f9dc00700 1 -- 172.21.15.153:0/3114564941 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f98100da0 msgr2=0x7f4f98101180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:45:45.198 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:45.201+0000 7f4f9dc00700 1 --2- 172.21.15.153:0/3114564941 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f98100da0 0x7f4f98101180 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f4f88009910 tx=0x7f4f88009c20 comp rx=0 tx=0).stop 2024-03-21T10:45:45.198 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:45.201+0000 7f4f9dc00700 1 -- 172.21.15.153:0/3114564941 shutdown_connections 2024-03-21T10:45:45.198 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:45.201+0000 7f4f9dc00700 1 --2- 172.21.15.153:0/3114564941 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4f98101750 0x7f4f981057a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:45:45.198 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:45.201+0000 7f4f9dc00700 1 --2- 172.21.15.153:0/3114564941 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f98100da0 0x7f4f98101180 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:45:45.198 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:45.201+0000 7f4f9dc00700 1 -- 172.21.15.153:0/3114564941 >> 172.21.15.153:0/3114564941 conn(0x7f4f980fc630 msgr2=0x7f4f980fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:45:45.198 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:45.201+0000 7f4f9dc00700 1 -- 172.21.15.153:0/3114564941 shutdown_connections 2024-03-21T10:45:45.198 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:45.201+0000 7f4f9dc00700 1 -- 172.21.15.153:0/3114564941 wait complete. 2024-03-21T10:45:45.199 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:45.202+0000 7f4f9dc00700 1 Processor -- start 2024-03-21T10:45:45.199 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:45.202+0000 7f4f9dc00700 1 -- start start 2024-03-21T10:45:45.199 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:45.203+0000 7f4f9dc00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f98100da0 0x7f4f98198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:45:45.200 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:45.203+0000 7f4f9dc00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4f98101750 0x7f4f98198ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:45:45.200 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:45.203+0000 7f4f9dc00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f98199560 con 0x7f4f98100da0 2024-03-21T10:45:45.200 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:45.203+0000 7f4f9dc00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f9819d370 con 0x7f4f98101750 2024-03-21T10:45:45.200 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:45.203+0000 7f4f96a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f98100da0 0x7f4f98198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:45:45.200 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:45.203+0000 7f4f96a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f98100da0 0x7f4f98198990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:54814/0 (socket says 172.21.15.153:54814) 2024-03-21T10:45:45.200 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:45.203+0000 7f4f96a00700 1 -- 172.21.15.153:0/2497921088 learned_addr learned my addr 172.21.15.153:0/2497921088 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:45:45.200 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:45.203+0000 7f4f96a00700 1 -- 172.21.15.153:0/2497921088 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4f98101750 msgr2=0x7f4f98198ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:45:45.200 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:45.203+0000 7f4f96a00700 1 --2- 172.21.15.153:0/2497921088 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4f98101750 0x7f4f98198ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:45:45.200 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:45.203+0000 7f4f96a00700 1 -- 172.21.15.153:0/2497921088 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f880095f0 con 0x7f4f98100da0 2024-03-21T10:45:45.200 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:45.203+0000 7f4f96a00700 1 --2- 172.21.15.153:0/2497921088 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f98100da0 0x7f4f98198990 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f4f880092e0 tx=0x7f4f8800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:45:45.201 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:45.204+0000 7f4f8f400700 1 -- 172.21.15.153:0/2497921088 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4f8801d070 con 0x7f4f98100da0 2024-03-21T10:45:45.201 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:45.204+0000 7f4f8f400700 1 -- 172.21.15.153:0/2497921088 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4f88005e00 con 0x7f4f98100da0 2024-03-21T10:45:45.201 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:45.204+0000 7f4f8f400700 1 -- 172.21.15.153:0/2497921088 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4f88017400 con 0x7f4f98100da0 2024-03-21T10:45:45.201 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:45.204+0000 7f4f9dc00700 1 -- 172.21.15.153:0/2497921088 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f9819d5f0 con 0x7f4f98100da0 2024-03-21T10:45:45.201 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:45.204+0000 7f4f9dc00700 1 -- 172.21.15.153:0/2497921088 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f9819dae0 con 0x7f4f98100da0 2024-03-21T10:45:45.201 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:45.205+0000 7f4f9dc00700 1 -- 172.21.15.153:0/2497921088 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f98104f20 con 0x7f4f98100da0 2024-03-21T10:45:45.202 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:45.206+0000 7f4f8f400700 1 -- 172.21.15.153:0/2497921088 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f4f8800f460 con 0x7f4f98100da0 2024-03-21T10:45:45.203 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:45.207+0000 7f4f8f400700 1 --2- 172.21.15.153:0/2497921088 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f4f7c06e460 0x7f4f7c070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:45:45.203 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:45.207+0000 7f4f8f400700 1 -- 172.21.15.153:0/2497921088 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f4f8808f9b0 con 0x7f4f98100da0 2024-03-21T10:45:45.203 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:45.207+0000 7f4f96000700 1 --2- 172.21.15.153:0/2497921088 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f4f7c06e460 0x7f4f7c070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:45:45.204 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:45.208+0000 7f4f96000700 1 --2- 172.21.15.153:0/2497921088 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f4f7c06e460 0x7f4f7c070920 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f4f84005950 tx=0x7f4f840058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:45:45.207 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:45.211+0000 7f4f8f400700 1 -- 172.21.15.153:0/2497921088 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4f8805c580 con 0x7f4f98100da0 2024-03-21T10:45:45.302 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:45:45 smithi153 ceph-mon[113733]: from='client.35806 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:45:45.302 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:45:45 smithi153 ceph-mon[113733]: pgmap v1018: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:45:45.302 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:45:45 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/874948975' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:45:45.500 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:45.504+0000 7f4f9dc00700 1 -- 172.21.15.153:0/2497921088 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f4f9804ea90 con 0x7f4f98100da0 2024-03-21T10:45:45.500 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:45.505+0000 7f4f8f400700 1 -- 172.21.15.153:0/2497921088 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4493 (secure 0 0 0) 0x7f4f8805c110 con 0x7f4f98100da0 2024-03-21T10:45:45.501 INFO:teuthology.orchestra.run.smithi153.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-03-21T10:45:45.501 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-03-21T10:45:45.501 INFO:teuthology.orchestra.run.smithi153.stdout: host smithi174 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi174/config 2024-03-21T10:45:45.501 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/podman: timeout after 895 seconds 2024-03-21T10:45:45.501 INFO:teuthology.orchestra.run.smithi153.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 -e NODE_NAME=smithi174 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/run/ceph:z -v /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpd910r3_i:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 inventory --format=json-pretty --filter-for-batch 2024-03-21T10:45:45.502 INFO:teuthology.orchestra.run.smithi153.stdout:Traceback (most recent call last): 2024-03-21T10:45:45.502 INFO:teuthology.orchestra.run.smithi153.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-03-21T10:45:45.502 INFO:teuthology.orchestra.run.smithi153.stdout: return _run_code(code, main_globals, None, 2024-03-21T10:45:45.502 INFO:teuthology.orchestra.run.smithi153.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-03-21T10:45:45.502 INFO:teuthology.orchestra.run.smithi153.stdout: exec(code, run_globals) 2024-03-21T10:45:45.502 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 5606, in 2024-03-21T10:45:45.502 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 5594, in main 2024-03-21T10:45:45.502 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 406, in _infer_config 2024-03-21T10:45:45.502 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 321, in _infer_fsid 2024-03-21T10:45:45.502 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 434, in _infer_image 2024-03-21T10:45:45.502 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 308, in _validate_fsid 2024-03-21T10:45:45.502 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 3357, in command_ceph_volume 2024-03-21T10:45:45.502 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-03-21T10:45:45.502 INFO:teuthology.orchestra.run.smithi153.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 -e NODE_NAME=smithi174 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/run/ceph:z -v /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpd910r3_i:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 inventory --format=json-pretty --filter-for-batch 2024-03-21T10:45:45.503 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-21T10:45:45.503 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is degraded 2024-03-21T10:45:45.503 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-21T10:45:45.503 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-21T10:45:45.503 INFO:teuthology.orchestra.run.smithi153.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-21T10:45:45.503 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-21T10:45:45.504 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:45.507+0000 7f4f9dc00700 1 -- 172.21.15.153:0/2497921088 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f4f7c06e460 msgr2=0x7f4f7c070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:45:45.504 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:45.507+0000 7f4f9dc00700 1 --2- 172.21.15.153:0/2497921088 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f4f7c06e460 0x7f4f7c070920 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f4f84005950 tx=0x7f4f840058e0 comp rx=0 tx=0).stop 2024-03-21T10:45:45.504 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:45.507+0000 7f4f9dc00700 1 -- 172.21.15.153:0/2497921088 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f98100da0 msgr2=0x7f4f98198990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:45:45.504 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:45.507+0000 7f4f9dc00700 1 --2- 172.21.15.153:0/2497921088 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f98100da0 0x7f4f98198990 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f4f880092e0 tx=0x7f4f8800be20 comp rx=0 tx=0).stop 2024-03-21T10:45:45.504 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:45.507+0000 7f4f9dc00700 1 -- 172.21.15.153:0/2497921088 shutdown_connections 2024-03-21T10:45:45.504 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:45.507+0000 7f4f9dc00700 1 --2- 172.21.15.153:0/2497921088 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f4f7c06e460 0x7f4f7c070920 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:45:45.504 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:45.507+0000 7f4f9dc00700 1 --2- 172.21.15.153:0/2497921088 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4f98100da0 0x7f4f98198990 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:45:45.504 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:45.507+0000 7f4f9dc00700 1 --2- 172.21.15.153:0/2497921088 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4f98101750 0x7f4f98198ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:45:45.504 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:45.507+0000 7f4f9dc00700 1 -- 172.21.15.153:0/2497921088 >> 172.21.15.153:0/2497921088 conn(0x7f4f980fc630 msgr2=0x7f4f980fea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:45:45.504 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:45.508+0000 7f4f9dc00700 1 -- 172.21.15.153:0/2497921088 shutdown_connections 2024-03-21T10:45:45.505 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:45:45.508+0000 7f4f9dc00700 1 -- 172.21.15.153:0/2497921088 wait complete. 2024-03-21T10:45:45.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:45:45 smithi174 ceph-mon[88004]: from='client.35806 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:45:45.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:45:45 smithi174 ceph-mon[88004]: pgmap v1018: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:45:45.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:45:45 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/874948975' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:45:46.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:45:46 smithi153 ceph-mon[113733]: from='client.35818 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:45:46.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:45:46 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/2497921088' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:45:46.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:45:46 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:45:46.617 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:45:46 smithi174 ceph-mon[88004]: from='client.35818 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:45:46.617 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:45:46 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/2497921088' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:45:46.617 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:45:46 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:45:47.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:45:47 smithi153 ceph-mon[113733]: pgmap v1019: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:45:47.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:45:47 smithi174 ceph-mon[88004]: pgmap v1019: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:45:49.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:45:49 smithi153 ceph-mon[113733]: pgmap v1020: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:45:49.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:45:49 smithi174 ceph-mon[88004]: pgmap v1020: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:45:51.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:45:51 smithi174 ceph-mon[88004]: pgmap v1021: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:45:51.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:45:51 smithi153 ceph-mon[113733]: pgmap v1021: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:45:53.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:45:53 smithi174 ceph-mon[88004]: pgmap v1022: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:45:53.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:45:53 smithi153 ceph-mon[113733]: pgmap v1022: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:45:55.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:45:55 smithi174 ceph-mon[88004]: pgmap v1023: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:45:55.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:45:55 smithi153 ceph-mon[113733]: pgmap v1023: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:45:57.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:45:57 smithi174 ceph-mon[88004]: pgmap v1024: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:45:57.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:45:57 smithi153 ceph-mon[113733]: pgmap v1024: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:45:59.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:45:59 smithi174 ceph-mon[88004]: pgmap v1025: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:45:59.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:45:59 smithi153 ceph-mon[113733]: pgmap v1025: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:46:01.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:46:01 smithi174 ceph-mon[88004]: pgmap v1026: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:46:01.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:46:01 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:46:01.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:46:01 smithi153 ceph-mon[113733]: pgmap v1026: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:46:01.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:46:01 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:46:03.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:46:03 smithi174 ceph-mon[88004]: pgmap v1027: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:46:03.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:46:03 smithi153 ceph-mon[113733]: pgmap v1027: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:46:05.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:46:05 smithi174 ceph-mon[88004]: pgmap v1028: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:46:05.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:46:05 smithi153 ceph-mon[113733]: pgmap v1028: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:46:07.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:46:07 smithi174 ceph-mon[88004]: pgmap v1029: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:46:07.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:46:07 smithi153 ceph-mon[113733]: pgmap v1029: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:46:09.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:46:09 smithi153 ceph-mon[113733]: pgmap v1030: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:46:09.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:46:09 smithi174 ceph-mon[88004]: pgmap v1030: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:46:11.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:46:11 smithi174 ceph-mon[88004]: pgmap v1031: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:46:11.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:46:11 smithi153 ceph-mon[113733]: pgmap v1031: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:46:13.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:46:13 smithi174 ceph-mon[88004]: pgmap v1032: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:46:13.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:46:13 smithi153 ceph-mon[113733]: pgmap v1032: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:46:15.677 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:15.679+0000 7fb893a00700 1 -- 172.21.15.153:0/2848250927 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb88c10d280 msgr2=0x7fb88c10d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:46:15.677 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:15.679+0000 7fb893a00700 1 --2- 172.21.15.153:0/2848250927 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb88c10d280 0x7fb88c10d660 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fb880009910 tx=0x7fb880009c20 comp rx=0 tx=0).stop 2024-03-21T10:46:15.677 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:15.679+0000 7fb893a00700 1 -- 172.21.15.153:0/2848250927 shutdown_connections 2024-03-21T10:46:15.677 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:15.679+0000 7fb893a00700 1 --2- 172.21.15.153:0/2848250927 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb88c1071d0 0x7fb88c107650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:46:15.677 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:15.679+0000 7fb893a00700 1 --2- 172.21.15.153:0/2848250927 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb88c10d280 0x7fb88c10d660 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:46:15.677 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:15.679+0000 7fb893a00700 1 -- 172.21.15.153:0/2848250927 >> 172.21.15.153:0/2848250927 conn(0x7fb88c0769e0 msgr2=0x7fb88c078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:46:15.678 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:46:15 smithi153 ceph-mon[113733]: pgmap v1033: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:46:15.679 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:15.680+0000 7fb893a00700 1 -- 172.21.15.153:0/2848250927 shutdown_connections 2024-03-21T10:46:15.679 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:15.680+0000 7fb893a00700 1 -- 172.21.15.153:0/2848250927 wait complete. 2024-03-21T10:46:15.679 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:15.681+0000 7fb893a00700 1 Processor -- start 2024-03-21T10:46:15.679 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:15.681+0000 7fb893a00700 1 -- start start 2024-03-21T10:46:15.679 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:15.682+0000 7fb893a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb88c1071d0 0x7fb88c19ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:46:15.679 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:15.682+0000 7fb893a00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb88c10d280 0x7fb88c19d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:46:15.679 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:15.682+0000 7fb893a00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb88c19d8a0 con 0x7fb88c1071d0 2024-03-21T10:46:15.679 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:15.682+0000 7fb893a00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb88c1a05d0 con 0x7fb88c10d280 2024-03-21T10:46:15.679 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:15.682+0000 7fb891000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb88c1071d0 0x7fb88c19ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:46:15.680 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:15.682+0000 7fb891000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb88c1071d0 0x7fb88c19ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:57002/0 (socket says 172.21.15.153:57002) 2024-03-21T10:46:15.680 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:15.682+0000 7fb891000700 1 -- 172.21.15.153:0/484855778 learned_addr learned my addr 172.21.15.153:0/484855778 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:46:15.681 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:15.682+0000 7fb88be00700 1 --2- 172.21.15.153:0/484855778 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb88c10d280 0x7fb88c19d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:46:15.681 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:15.682+0000 7fb891000700 1 -- 172.21.15.153:0/484855778 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb88c10d280 msgr2=0x7fb88c19d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:46:15.681 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:15.682+0000 7fb891000700 1 --2- 172.21.15.153:0/484855778 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb88c10d280 0x7fb88c19d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:46:15.681 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:15.682+0000 7fb891000700 1 -- 172.21.15.153:0/484855778 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8800095f0 con 0x7fb88c1071d0 2024-03-21T10:46:15.681 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:15.682+0000 7fb88be00700 1 --2- 172.21.15.153:0/484855778 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb88c10d280 0x7fb88c19d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:46:15.681 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:15.683+0000 7fb891000700 1 --2- 172.21.15.153:0/484855778 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb88c1071d0 0x7fb88c19ccd0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fb880006010 tx=0x7fb88000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:46:15.681 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:15.683+0000 7fb889600700 1 -- 172.21.15.153:0/484855778 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb88001d070 con 0x7fb88c1071d0 2024-03-21T10:46:15.681 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:15.683+0000 7fb889600700 1 -- 172.21.15.153:0/484855778 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb880004fb0 con 0x7fb88c1071d0 2024-03-21T10:46:15.681 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:15.683+0000 7fb893a00700 1 -- 172.21.15.153:0/484855778 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb88c1a0850 con 0x7fb88c1071d0 2024-03-21T10:46:15.681 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:15.683+0000 7fb889600700 1 -- 172.21.15.153:0/484855778 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb880022ea0 con 0x7fb88c1071d0 2024-03-21T10:46:15.682 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:15.683+0000 7fb893a00700 1 -- 172.21.15.153:0/484855778 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb88c1a0dc0 con 0x7fb88c1071d0 2024-03-21T10:46:15.682 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:15.684+0000 7fb893a00700 1 -- 172.21.15.153:0/484855778 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb88c04ea90 con 0x7fb88c1071d0 2024-03-21T10:46:15.682 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:15.685+0000 7fb889600700 1 -- 172.21.15.153:0/484855778 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fb880022470 con 0x7fb88c1071d0 2024-03-21T10:46:15.683 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:15.686+0000 7fb889600700 1 --2- 172.21.15.153:0/484855778 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb87406e460 0x7fb874070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:46:15.683 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:15.686+0000 7fb889600700 1 -- 172.21.15.153:0/484855778 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fb88008fa50 con 0x7fb88c1071d0 2024-03-21T10:46:15.683 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:15.686+0000 7fb88be00700 1 --2- 172.21.15.153:0/484855778 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb87406e460 0x7fb874070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:46:15.684 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:15.687+0000 7fb88be00700 1 --2- 172.21.15.153:0/484855778 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb87406e460 0x7fb874070920 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fb878005fd0 tx=0x7fb878005dc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:46:15.688 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:15.691+0000 7fb889600700 1 -- 172.21.15.153:0/484855778 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb880058ba0 con 0x7fb88c1071d0 2024-03-21T10:46:15.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:46:15 smithi174 ceph-mon[88004]: pgmap v1033: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:46:15.933 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:15.935+0000 7fb893a00700 1 -- 172.21.15.153:0/484855778 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb88c1a11b0 con 0x7fb87406e460 2024-03-21T10:46:15.934 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:15.937+0000 7fb889600700 1 -- 172.21.15.153:0/484855778 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fb88c1a11b0 con 0x7fb87406e460 2024-03-21T10:46:15.937 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:15.940+0000 7fb893a00700 1 -- 172.21.15.153:0/484855778 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb87406e460 msgr2=0x7fb874070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:46:15.937 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:15.940+0000 7fb893a00700 1 --2- 172.21.15.153:0/484855778 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb87406e460 0x7fb874070920 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fb878005fd0 tx=0x7fb878005dc0 comp rx=0 tx=0).stop 2024-03-21T10:46:15.938 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:15.940+0000 7fb893a00700 1 -- 172.21.15.153:0/484855778 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb88c1071d0 msgr2=0x7fb88c19ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:46:15.938 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:15.940+0000 7fb893a00700 1 --2- 172.21.15.153:0/484855778 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb88c1071d0 0x7fb88c19ccd0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fb880006010 tx=0x7fb88000be20 comp rx=0 tx=0).stop 2024-03-21T10:46:15.938 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:15.940+0000 7fb893a00700 1 -- 172.21.15.153:0/484855778 shutdown_connections 2024-03-21T10:46:15.938 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:15.940+0000 7fb893a00700 1 --2- 172.21.15.153:0/484855778 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb87406e460 0x7fb874070920 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:46:15.938 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:15.940+0000 7fb893a00700 1 --2- 172.21.15.153:0/484855778 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb88c1071d0 0x7fb88c19ccd0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:46:15.938 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:15.940+0000 7fb893a00700 1 --2- 172.21.15.153:0/484855778 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb88c10d280 0x7fb88c19d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:46:15.938 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:15.940+0000 7fb893a00700 1 -- 172.21.15.153:0/484855778 >> 172.21.15.153:0/484855778 conn(0x7fb88c0769e0 msgr2=0x7fb88c078320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:46:15.938 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:15.941+0000 7fb893a00700 1 -- 172.21.15.153:0/484855778 shutdown_connections 2024-03-21T10:46:15.939 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:15.941+0000 7fb893a00700 1 -- 172.21.15.153:0/484855778 wait complete. 2024-03-21T10:46:15.955 INFO:teuthology.orchestra.run.smithi153.stdout:true 2024-03-21T10:46:16.101 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.103+0000 7f0ae8c00700 1 -- 172.21.15.153:0/797777410 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0ae41069c0 msgr2=0x7f0ae4106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:46:16.101 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.103+0000 7f0ae8c00700 1 --2- 172.21.15.153:0/797777410 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0ae41069c0 0x7f0ae4106da0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f0ad4009910 tx=0x7f0ad4009c20 comp rx=0 tx=0).stop 2024-03-21T10:46:16.101 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.103+0000 7f0ae8c00700 1 -- 172.21.15.153:0/797777410 shutdown_connections 2024-03-21T10:46:16.101 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.103+0000 7f0ae8c00700 1 --2- 172.21.15.153:0/797777410 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0ae41009c0 0x7f0ae4100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:46:16.101 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.103+0000 7f0ae8c00700 1 --2- 172.21.15.153:0/797777410 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0ae41069c0 0x7f0ae4106da0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:46:16.101 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.103+0000 7f0ae8c00700 1 -- 172.21.15.153:0/797777410 >> 172.21.15.153:0/797777410 conn(0x7f0ae40fc4b0 msgr2=0x7f0ae40fe8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:46:16.101 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.103+0000 7f0ae8c00700 1 -- 172.21.15.153:0/797777410 shutdown_connections 2024-03-21T10:46:16.101 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.103+0000 7f0ae8c00700 1 -- 172.21.15.153:0/797777410 wait complete. 2024-03-21T10:46:16.102 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.104+0000 7f0ae8c00700 1 Processor -- start 2024-03-21T10:46:16.102 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.105+0000 7f0ae8c00700 1 -- start start 2024-03-21T10:46:16.102 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.105+0000 7f0ae8c00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0ae41009c0 0x7f0ae419c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:46:16.102 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.105+0000 7f0ae8c00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0ae41069c0 0x7f0ae419c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:46:16.102 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.105+0000 7f0ae8c00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ae419d050 con 0x7f0ae41069c0 2024-03-21T10:46:16.102 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.105+0000 7f0ae8c00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ae4196510 con 0x7f0ae41009c0 2024-03-21T10:46:16.103 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.105+0000 7f0adaa00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0ae41069c0 0x7f0ae419c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:46:16.104 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.105+0000 7f0adaa00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0ae41069c0 0x7f0ae419c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:57016/0 (socket says 172.21.15.153:57016) 2024-03-21T10:46:16.104 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.105+0000 7f0adaa00700 1 -- 172.21.15.153:0/3460263121 learned_addr learned my addr 172.21.15.153:0/3460263121 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:46:16.104 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.106+0000 7f0ae3400700 1 --2- 172.21.15.153:0/3460263121 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0ae41009c0 0x7f0ae419c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:46:16.104 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.106+0000 7f0adaa00700 1 -- 172.21.15.153:0/3460263121 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0ae41009c0 msgr2=0x7f0ae419c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:46:16.104 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.106+0000 7f0adaa00700 1 --2- 172.21.15.153:0/3460263121 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0ae41009c0 0x7f0ae419c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:46:16.104 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.106+0000 7f0adaa00700 1 -- 172.21.15.153:0/3460263121 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ad40095f0 con 0x7f0ae41069c0 2024-03-21T10:46:16.104 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.106+0000 7f0adaa00700 1 --2- 172.21.15.153:0/3460263121 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0ae41069c0 0x7f0ae419c970 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f0acc00b4b0 tx=0x7f0acc00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:46:16.104 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.106+0000 7f0ae0c00700 1 -- 172.21.15.153:0/3460263121 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0acc011800 con 0x7f0ae41069c0 2024-03-21T10:46:16.104 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.106+0000 7f0ae8c00700 1 -- 172.21.15.153:0/3460263121 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ae41967f0 con 0x7f0ae41069c0 2024-03-21T10:46:16.104 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.106+0000 7f0ae0c00700 1 -- 172.21.15.153:0/3460263121 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0acc011e20 con 0x7f0ae41069c0 2024-03-21T10:46:16.104 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.106+0000 7f0ae0c00700 1 -- 172.21.15.153:0/3460263121 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0acc00f2e0 con 0x7f0ae41069c0 2024-03-21T10:46:16.105 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.107+0000 7f0ae8c00700 1 -- 172.21.15.153:0/3460263121 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ae4196d40 con 0x7f0ae41069c0 2024-03-21T10:46:16.105 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.108+0000 7f0ae8c00700 1 -- 172.21.15.153:0/3460263121 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ae4108fc0 con 0x7f0ae41069c0 2024-03-21T10:46:16.106 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.109+0000 7f0ae0c00700 1 -- 172.21.15.153:0/3460263121 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f0acc010380 con 0x7f0ae41069c0 2024-03-21T10:46:16.107 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.110+0000 7f0ae0c00700 1 --2- 172.21.15.153:0/3460263121 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f0ac806e460 0x7f0ac8070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:46:16.107 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.110+0000 7f0ae0c00700 1 -- 172.21.15.153:0/3460263121 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f0acc08e190 con 0x7f0ae41069c0 2024-03-21T10:46:16.107 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.110+0000 7f0ae3400700 1 --2- 172.21.15.153:0/3460263121 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f0ac806e460 0x7f0ac8070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:46:16.108 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.111+0000 7f0ae3400700 1 --2- 172.21.15.153:0/3460263121 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f0ac806e460 0x7f0ac8070920 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f0ad4006010 tx=0x7f0ad401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:46:16.112 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.115+0000 7f0ae0c00700 1 -- 172.21.15.153:0/3460263121 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0acc05ace0 con 0x7f0ae41069c0 2024-03-21T10:46:16.354 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.356+0000 7f0ae8c00700 1 -- 172.21.15.153:0/3460263121 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0ae4196fc0 con 0x7f0ac806e460 2024-03-21T10:46:16.356 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.359+0000 7f0ae0c00700 1 -- 172.21.15.153:0/3460263121 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f0ae4196fc0 con 0x7f0ac806e460 2024-03-21T10:46:16.359 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.362+0000 7f0ae8c00700 1 -- 172.21.15.153:0/3460263121 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f0ac806e460 msgr2=0x7f0ac8070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:46:16.360 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.362+0000 7f0ae8c00700 1 --2- 172.21.15.153:0/3460263121 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f0ac806e460 0x7f0ac8070920 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f0ad4006010 tx=0x7f0ad401a040 comp rx=0 tx=0).stop 2024-03-21T10:46:16.360 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.362+0000 7f0ae8c00700 1 -- 172.21.15.153:0/3460263121 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0ae41069c0 msgr2=0x7f0ae419c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:46:16.360 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.362+0000 7f0ae8c00700 1 --2- 172.21.15.153:0/3460263121 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0ae41069c0 0x7f0ae419c970 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f0acc00b4b0 tx=0x7f0acc00b870 comp rx=0 tx=0).stop 2024-03-21T10:46:16.360 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.362+0000 7f0ae8c00700 1 -- 172.21.15.153:0/3460263121 shutdown_connections 2024-03-21T10:46:16.360 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.362+0000 7f0ae8c00700 1 --2- 172.21.15.153:0/3460263121 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0ae41009c0 0x7f0ae419c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:46:16.360 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.362+0000 7f0ae8c00700 1 --2- 172.21.15.153:0/3460263121 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f0ac806e460 0x7f0ac8070920 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:46:16.360 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.362+0000 7f0ae8c00700 1 --2- 172.21.15.153:0/3460263121 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0ae41069c0 0x7f0ae419c970 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:46:16.360 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.362+0000 7f0ae8c00700 1 -- 172.21.15.153:0/3460263121 >> 172.21.15.153:0/3460263121 conn(0x7f0ae40fc4b0 msgr2=0x7f0ae40fcfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:46:16.360 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.362+0000 7f0ae8c00700 1 -- 172.21.15.153:0/3460263121 shutdown_connections 2024-03-21T10:46:16.360 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.362+0000 7f0ae8c00700 1 -- 172.21.15.153:0/3460263121 wait complete. 2024-03-21T10:46:16.512 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.511+0000 7fa5e1000700 1 -- 172.21.15.153:0/1484878059 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa5dc100da0 msgr2=0x7fa5dc101180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:46:16.512 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.511+0000 7fa5e1000700 1 --2- 172.21.15.153:0/1484878059 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa5dc100da0 0x7fa5dc101180 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc009910 tx=0x7fa5cc009c20 comp rx=0 tx=0).stop 2024-03-21T10:46:16.512 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.512+0000 7fa5e1000700 1 -- 172.21.15.153:0/1484878059 shutdown_connections 2024-03-21T10:46:16.512 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.512+0000 7fa5e1000700 1 --2- 172.21.15.153:0/1484878059 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa5dc101750 0x7fa5dc1057a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:46:16.512 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.512+0000 7fa5e1000700 1 --2- 172.21.15.153:0/1484878059 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa5dc100da0 0x7fa5dc101180 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:46:16.512 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.512+0000 7fa5e1000700 1 -- 172.21.15.153:0/1484878059 >> 172.21.15.153:0/1484878059 conn(0x7fa5dc0fc650 msgr2=0x7fa5dc0fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:46:16.513 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.512+0000 7fa5e1000700 1 -- 172.21.15.153:0/1484878059 shutdown_connections 2024-03-21T10:46:16.513 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.512+0000 7fa5e1000700 1 -- 172.21.15.153:0/1484878059 wait complete. 2024-03-21T10:46:16.513 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.513+0000 7fa5e1000700 1 Processor -- start 2024-03-21T10:46:16.513 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.513+0000 7fa5e1000700 1 -- start start 2024-03-21T10:46:16.513 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.513+0000 7fa5e1000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa5dc100da0 0x7fa5dc19c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:46:16.513 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.513+0000 7fa5e1000700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa5dc101750 0x7fa5dc19ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:46:16.513 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.513+0000 7fa5e1000700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5dc19d0f0 con 0x7fa5dc100da0 2024-03-21T10:46:16.513 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.513+0000 7fa5e1000700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5dc1965a0 con 0x7fa5dc101750 2024-03-21T10:46:16.513 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.513+0000 7fa5d9e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa5dc100da0 0x7fa5dc19c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:46:16.513 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.514+0000 7fa5d9e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa5dc100da0 0x7fa5dc19c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:57030/0 (socket says 172.21.15.153:57030) 2024-03-21T10:46:16.513 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.514+0000 7fa5d9e00700 1 -- 172.21.15.153:0/1936586384 learned_addr learned my addr 172.21.15.153:0/1936586384 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:46:16.513 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.514+0000 7fa5d9e00700 1 -- 172.21.15.153:0/1936586384 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa5dc101750 msgr2=0x7fa5dc19ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:46:16.513 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.514+0000 7fa5d9e00700 1 --2- 172.21.15.153:0/1936586384 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa5dc101750 0x7fa5dc19ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:46:16.513 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.514+0000 7fa5d9e00700 1 -- 172.21.15.153:0/1936586384 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5cc0095f0 con 0x7fa5dc100da0 2024-03-21T10:46:16.514 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.514+0000 7fa5d9e00700 1 --2- 172.21.15.153:0/1936586384 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa5dc100da0 0x7fa5dc19c520 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc006010 tx=0x7fa5cc00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:46:16.514 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.514+0000 7fa5d2a00700 1 -- 172.21.15.153:0/1936586384 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa5cc01d070 con 0x7fa5dc100da0 2024-03-21T10:46:16.514 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.515+0000 7fa5d2a00700 1 -- 172.21.15.153:0/1936586384 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa5cc004fb0 con 0x7fa5dc100da0 2024-03-21T10:46:16.514 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.515+0000 7fa5e1000700 1 -- 172.21.15.153:0/1936586384 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5dc196820 con 0x7fa5dc100da0 2024-03-21T10:46:16.514 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.515+0000 7fa5d2a00700 1 -- 172.21.15.153:0/1936586384 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa5cc022ea0 con 0x7fa5dc100da0 2024-03-21T10:46:16.514 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.515+0000 7fa5e1000700 1 -- 172.21.15.153:0/1936586384 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5dc196d10 con 0x7fa5dc100da0 2024-03-21T10:46:16.515 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.516+0000 7fa5e1000700 1 -- 172.21.15.153:0/1936586384 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5dc109140 con 0x7fa5dc100da0 2024-03-21T10:46:16.515 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.517+0000 7fa5d2a00700 1 -- 172.21.15.153:0/1936586384 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fa5cc022470 con 0x7fa5dc100da0 2024-03-21T10:46:16.515 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.517+0000 7fa5d2a00700 1 --2- 172.21.15.153:0/1936586384 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa5c006e460 0x7fa5c0070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:46:16.515 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.518+0000 7fa5d2a00700 1 -- 172.21.15.153:0/1936586384 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fa5cc08fa80 con 0x7fa5dc100da0 2024-03-21T10:46:16.515 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.518+0000 7fa5d9400700 1 --2- 172.21.15.153:0/1936586384 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa5c006e460 0x7fa5c0070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:46:16.515 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.518+0000 7fa5d9400700 1 --2- 172.21.15.153:0/1936586384 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa5c006e460 0x7fa5c0070920 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc0fce20 tx=0x7fa5c8005c80 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:46:16.519 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.522+0000 7fa5d2a00700 1 -- 172.21.15.153:0/1936586384 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa5cc05c720 con 0x7fa5dc100da0 2024-03-21T10:46:16.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:46:16 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:46:16.748 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.751+0000 7fa5e1000700 1 -- 172.21.15.153:0/1936586384 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fa5dc196ff0 con 0x7fa5c006e460 2024-03-21T10:46:16.749 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:46:16 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:46:16.766 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.769+0000 7fa5d2a00700 1 -- 172.21.15.153:0/1936586384 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fa5dc196ff0 con 0x7fa5c006e460 2024-03-21T10:46:16.767 INFO:teuthology.orchestra.run.smithi153.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-21T10:46:16.767 INFO:teuthology.orchestra.run.smithi153.stdout:alertmanager.smithi153 smithi153 *:9093,9094 running (35m) 10m ago 48m 15.6M - 0.25.0 c8568f914cd2 9ed3f2200b5f 2024-03-21T10:46:16.767 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi153 smithi153 running (49m) 10m ago 49m 10.4M - 18.2.1 5be31c24972a 5b61994396a6 2024-03-21T10:46:16.767 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi174 smithi174 running (47m) 10m ago 47m 10.1M - 18.2.1 5be31c24972a 4f7a5fc3ac28 2024-03-21T10:46:16.767 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi153 smithi153 running (31m) 10m ago 49m 6656k - 19.0.0-2231-g2edfee68 237e43c7a65f 32c21104d5b0 2024-03-21T10:46:16.767 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi174 smithi174 running (31m) 10m ago 47m 6660k - 19.0.0-2231-g2edfee68 237e43c7a65f a04e48967ab6 2024-03-21T10:46:16.767 INFO:teuthology.orchestra.run.smithi153.stdout:grafana.smithi153 smithi153 *:3000 running (34m) 10m ago 48m 75.2M - 9.4.12 2bacad6d85d8 ee24026ab786 2024-03-21T10:46:16.767 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.iuyluj smithi153 running (26m) 10m ago 41m 18.7M - 19.0.0-2231-g2edfee68 237e43c7a65f 87aff8a5de70 2024-03-21T10:46:16.767 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.zipckr smithi153 running (27m) 10m ago 41m 18.9M - 19.0.0-2231-g2edfee68 237e43c7a65f ccc262bd5dd3 2024-03-21T10:46:16.768 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.nhozag smithi174 running (10m) 10m ago 41m 10.9M - 19.0.0-2231-g2edfee68 237e43c7a65f 9e02aea72c96 2024-03-21T10:46:16.768 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.yyimst smithi174 running (26m) 10m ago 41m 18.5M - 19.0.0-2231-g2edfee68 237e43c7a65f fc1962f9351e 2024-03-21T10:46:16.768 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi153.tluxni smithi153 *:8443,9283,8765 running (38m) 10m ago 50m 512M - 19.0.0-2231-g2edfee68 237e43c7a65f 7ffee0e55382 2024-03-21T10:46:16.768 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi174.qpcwat smithi174 *:8443,9283,8765 running (36m) 10m ago 47m 459M - 19.0.0-2231-g2edfee68 237e43c7a65f 748968f216ab 2024-03-21T10:46:16.768 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi153 smithi153 running (32m) 10m ago 50m 70.8M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f dd8038303dc3 2024-03-21T10:46:16.768 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi174 smithi174 running (32m) 10m ago 47m 51.9M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f 29d78a788f2c 2024-03-21T10:46:16.768 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi153 smithi153 *:9100 running (36m) 10m ago 48m 8140k - 1.5.0 0da6a335fe13 bb835edc0e0b 2024-03-21T10:46:16.768 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi174 smithi174 *:9100 running (36m) 10m ago 47m 8610k - 1.5.0 0da6a335fe13 7c4ecad790fa 2024-03-21T10:46:16.768 INFO:teuthology.orchestra.run.smithi153.stdout:osd.0 smithi153 running (30m) 10m ago 46m 118M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 507900fd4791 2024-03-21T10:46:16.768 INFO:teuthology.orchestra.run.smithi153.stdout:osd.1 smithi153 running (29m) 10m ago 45m 93.0M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 0d646452e9c1 2024-03-21T10:46:16.768 INFO:teuthology.orchestra.run.smithi153.stdout:osd.2 smithi153 running (29m) 10m ago 45m 91.2M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 1326f2afdfcd 2024-03-21T10:46:16.768 INFO:teuthology.orchestra.run.smithi153.stdout:osd.3 smithi174 running (28m) 10m ago 44m 87.5M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 70aafac71ad5 2024-03-21T10:46:16.768 INFO:teuthology.orchestra.run.smithi153.stdout:osd.4 smithi174 running (28m) 10m ago 44m 95.8M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 0d746ce6e79d 2024-03-21T10:46:16.768 INFO:teuthology.orchestra.run.smithi153.stdout:osd.5 smithi174 running (27m) 10m ago 43m 90.3M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 53e5f35b8cac 2024-03-21T10:46:16.768 INFO:teuthology.orchestra.run.smithi153.stdout:prometheus.smithi153 smithi153 *:9095 running (35m) 10m ago 48m 88.1M - 2.43.0 a07b618ecd1d 7d95561e50c2 2024-03-21T10:46:16.770 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.773+0000 7fa5e1000700 1 -- 172.21.15.153:0/1936586384 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa5c006e460 msgr2=0x7fa5c0070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:46:16.770 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.773+0000 7fa5e1000700 1 --2- 172.21.15.153:0/1936586384 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa5c006e460 0x7fa5c0070920 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc0fce20 tx=0x7fa5c8005c80 comp rx=0 tx=0).stop 2024-03-21T10:46:16.770 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.773+0000 7fa5e1000700 1 -- 172.21.15.153:0/1936586384 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa5dc100da0 msgr2=0x7fa5dc19c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:46:16.770 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.773+0000 7fa5e1000700 1 --2- 172.21.15.153:0/1936586384 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa5dc100da0 0x7fa5dc19c520 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc006010 tx=0x7fa5cc00be20 comp rx=0 tx=0).stop 2024-03-21T10:46:16.771 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.773+0000 7fa5e1000700 1 -- 172.21.15.153:0/1936586384 shutdown_connections 2024-03-21T10:46:16.771 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.773+0000 7fa5e1000700 1 --2- 172.21.15.153:0/1936586384 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa5c006e460 0x7fa5c0070920 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:46:16.771 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.773+0000 7fa5e1000700 1 --2- 172.21.15.153:0/1936586384 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa5dc100da0 0x7fa5dc19c520 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:46:16.771 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.773+0000 7fa5e1000700 1 --2- 172.21.15.153:0/1936586384 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa5dc101750 0x7fa5dc19ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:46:16.771 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.773+0000 7fa5e1000700 1 -- 172.21.15.153:0/1936586384 >> 172.21.15.153:0/1936586384 conn(0x7fa5dc0fc650 msgr2=0x7fa5dc0fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:46:16.771 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.773+0000 7fa5e1000700 1 -- 172.21.15.153:0/1936586384 shutdown_connections 2024-03-21T10:46:16.771 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.773+0000 7fa5e1000700 1 -- 172.21.15.153:0/1936586384 wait complete. 2024-03-21T10:46:16.921 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.924+0000 7fe377800700 1 -- 172.21.15.153:0/2998860290 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe370102fb0 msgr2=0x7fe370103390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:46:16.922 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.924+0000 7fe377800700 1 --2- 172.21.15.153:0/2998860290 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe370102fb0 0x7fe370103390 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fe364009910 tx=0x7fe364009c20 comp rx=0 tx=0).stop 2024-03-21T10:46:16.922 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.924+0000 7fe377800700 1 -- 172.21.15.153:0/2998860290 shutdown_connections 2024-03-21T10:46:16.922 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.924+0000 7fe377800700 1 --2- 172.21.15.153:0/2998860290 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe370103960 0x7fe3701079b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:46:16.922 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.924+0000 7fe377800700 1 --2- 172.21.15.153:0/2998860290 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe370102fb0 0x7fe370103390 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:46:16.922 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.924+0000 7fe377800700 1 -- 172.21.15.153:0/2998860290 >> 172.21.15.153:0/2998860290 conn(0x7fe3700fe840 msgr2=0x7fe370100c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:46:16.922 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.924+0000 7fe377800700 1 -- 172.21.15.153:0/2998860290 shutdown_connections 2024-03-21T10:46:16.922 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.924+0000 7fe377800700 1 -- 172.21.15.153:0/2998860290 wait complete. 2024-03-21T10:46:16.923 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.925+0000 7fe377800700 1 Processor -- start 2024-03-21T10:46:16.923 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.926+0000 7fe377800700 1 -- start start 2024-03-21T10:46:16.923 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.926+0000 7fe377800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe370102fb0 0x7fe37019e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:46:16.923 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.926+0000 7fe377800700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe370103960 0x7fe37019ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:46:16.923 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.926+0000 7fe377800700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe37019f3e0 con 0x7fe370102fb0 2024-03-21T10:46:16.924 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.926+0000 7fe377800700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe370198890 con 0x7fe370103960 2024-03-21T10:46:16.924 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.926+0000 7fe374e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe370102fb0 0x7fe37019e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:46:16.924 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.926+0000 7fe374e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe370102fb0 0x7fe37019e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:57050/0 (socket says 172.21.15.153:57050) 2024-03-21T10:46:16.924 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.926+0000 7fe374e00700 1 -- 172.21.15.153:0/207979664 learned_addr learned my addr 172.21.15.153:0/207979664 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:46:16.925 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.927+0000 7fe36fe00700 1 --2- 172.21.15.153:0/207979664 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe370103960 0x7fe37019ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:46:16.925 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.927+0000 7fe374e00700 1 -- 172.21.15.153:0/207979664 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe370103960 msgr2=0x7fe37019ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:46:16.925 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.927+0000 7fe374e00700 1 --2- 172.21.15.153:0/207979664 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe370103960 0x7fe37019ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:46:16.925 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.927+0000 7fe374e00700 1 -- 172.21.15.153:0/207979664 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3640095f0 con 0x7fe370102fb0 2024-03-21T10:46:16.925 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.927+0000 7fe374e00700 1 --2- 172.21.15.153:0/207979664 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe370102fb0 0x7fe37019e810 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fe364006010 tx=0x7fe36400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:46:16.925 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.927+0000 7fe36d600700 1 -- 172.21.15.153:0/207979664 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe36401d070 con 0x7fe370102fb0 2024-03-21T10:46:16.925 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.927+0000 7fe36d600700 1 -- 172.21.15.153:0/207979664 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe364004fb0 con 0x7fe370102fb0 2024-03-21T10:46:16.925 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.927+0000 7fe36d600700 1 -- 172.21.15.153:0/207979664 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe364022ea0 con 0x7fe370102fb0 2024-03-21T10:46:16.925 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.927+0000 7fe377800700 1 -- 172.21.15.153:0/207979664 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe370198b10 con 0x7fe370102fb0 2024-03-21T10:46:16.926 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.928+0000 7fe377800700 1 -- 172.21.15.153:0/207979664 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe370199000 con 0x7fe370102fb0 2024-03-21T10:46:16.926 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.929+0000 7fe377800700 1 -- 172.21.15.153:0/207979664 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe37010b350 con 0x7fe370102fb0 2024-03-21T10:46:16.926 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.929+0000 7fe36d600700 1 -- 172.21.15.153:0/207979664 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fe364022470 con 0x7fe370102fb0 2024-03-21T10:46:16.927 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.930+0000 7fe36d600700 1 --2- 172.21.15.153:0/207979664 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe35806e460 0x7fe358070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:46:16.927 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.930+0000 7fe36d600700 1 -- 172.21.15.153:0/207979664 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fe36408fa50 con 0x7fe370102fb0 2024-03-21T10:46:16.928 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.931+0000 7fe36fe00700 1 --2- 172.21.15.153:0/207979664 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe35806e460 0x7fe358070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:46:16.928 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.931+0000 7fe36fe00700 1 --2- 172.21.15.153:0/207979664 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe35806e460 0x7fe358070920 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fe35c006fd0 tx=0x7fe35c008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:46:16.932 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:16.935+0000 7fe36d600700 1 -- 172.21.15.153:0/207979664 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe36405c6d0 con 0x7fe370102fb0 2024-03-21T10:46:17.244 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:17.246+0000 7fe377800700 1 -- 172.21.15.153:0/207979664 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fe37010ad30 con 0x7fe370102fb0 2024-03-21T10:46:17.245 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:17.248+0000 7fe36d600700 1 -- 172.21.15.153:0/207979664 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fe364027280 con 0x7fe370102fb0 2024-03-21T10:46:17.245 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:46:17.245 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": { 2024-03-21T10:46:17.245 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:46:17.245 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:46:17.245 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": { 2024-03-21T10:46:17.245 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:46:17.245 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:46:17.246 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": { 2024-03-21T10:46:17.246 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 6 2024-03-21T10:46:17.246 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:46:17.246 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": { 2024-03-21T10:46:17.246 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 4 2024-03-21T10:46:17.246 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:46:17.246 INFO:teuthology.orchestra.run.smithi153.stdout: "overall": { 2024-03-21T10:46:17.246 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 14 2024-03-21T10:46:17.246 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-03-21T10:46:17.246 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:46:17.248 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:17.250+0000 7fe377800700 1 -- 172.21.15.153:0/207979664 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe35806e460 msgr2=0x7fe358070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:46:17.248 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:17.250+0000 7fe377800700 1 --2- 172.21.15.153:0/207979664 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe35806e460 0x7fe358070920 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fe35c006fd0 tx=0x7fe35c008040 comp rx=0 tx=0).stop 2024-03-21T10:46:17.248 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:17.250+0000 7fe377800700 1 -- 172.21.15.153:0/207979664 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe370102fb0 msgr2=0x7fe37019e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:46:17.248 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:17.250+0000 7fe377800700 1 --2- 172.21.15.153:0/207979664 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe370102fb0 0x7fe37019e810 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fe364006010 tx=0x7fe36400be20 comp rx=0 tx=0).stop 2024-03-21T10:46:17.248 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:17.251+0000 7fe377800700 1 -- 172.21.15.153:0/207979664 shutdown_connections 2024-03-21T10:46:17.248 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:17.251+0000 7fe377800700 1 --2- 172.21.15.153:0/207979664 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe35806e460 0x7fe358070920 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:46:17.248 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:17.251+0000 7fe377800700 1 --2- 172.21.15.153:0/207979664 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe370102fb0 0x7fe37019e810 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:46:17.248 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:17.251+0000 7fe377800700 1 --2- 172.21.15.153:0/207979664 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe370103960 0x7fe37019ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:46:17.249 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:17.251+0000 7fe377800700 1 -- 172.21.15.153:0/207979664 >> 172.21.15.153:0/207979664 conn(0x7fe3700fe840 msgr2=0x7fe3700fff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:46:17.249 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:17.251+0000 7fe377800700 1 -- 172.21.15.153:0/207979664 shutdown_connections 2024-03-21T10:46:17.249 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:17.251+0000 7fe377800700 1 -- 172.21.15.153:0/207979664 wait complete. 2024-03-21T10:46:17.399 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:17.401+0000 7f61e0e00700 1 -- 172.21.15.153:0/1576376423 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f61dc105a40 msgr2=0x7f61dc105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:46:17.399 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:17.401+0000 7f61e0e00700 1 --2- 172.21.15.153:0/1576376423 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f61dc105a40 0x7f61dc105e20 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f61cc009910 tx=0x7f61cc009c20 comp rx=0 tx=0).stop 2024-03-21T10:46:17.399 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:17.401+0000 7f61e0e00700 1 -- 172.21.15.153:0/1576376423 shutdown_connections 2024-03-21T10:46:17.399 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:17.401+0000 7f61e0e00700 1 --2- 172.21.15.153:0/1576376423 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f61dc0ffbd0 0x7f61dc100050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:46:17.399 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:17.401+0000 7f61e0e00700 1 --2- 172.21.15.153:0/1576376423 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f61dc105a40 0x7f61dc105e20 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:46:17.399 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:17.401+0000 7f61e0e00700 1 -- 172.21.15.153:0/1576376423 >> 172.21.15.153:0/1576376423 conn(0x7f61dc0755a0 msgr2=0x7f61dc0759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:46:17.399 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:17.401+0000 7f61e0e00700 1 -- 172.21.15.153:0/1576376423 shutdown_connections 2024-03-21T10:46:17.399 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:17.401+0000 7f61e0e00700 1 -- 172.21.15.153:0/1576376423 wait complete. 2024-03-21T10:46:17.401 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:17.402+0000 7f61e0e00700 1 Processor -- start 2024-03-21T10:46:17.401 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:17.402+0000 7f61e0e00700 1 -- start start 2024-03-21T10:46:17.401 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:17.403+0000 7f61e0e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f61dc0ffbd0 0x7f61dc101610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:46:17.401 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:17.403+0000 7f61e0e00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f61dc105a40 0x7f61dc101b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:46:17.401 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:17.403+0000 7f61e0e00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61dc102090 con 0x7f61dc0ffbd0 2024-03-21T10:46:17.401 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:17.403+0000 7f61e0e00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61dc1021d0 con 0x7f61dc105a40 2024-03-21T10:46:17.401 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:17.403+0000 7f61d9e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f61dc0ffbd0 0x7f61dc101610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:46:17.401 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:17.403+0000 7f61d9e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f61dc0ffbd0 0x7f61dc101610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:57074/0 (socket says 172.21.15.153:57074) 2024-03-21T10:46:17.401 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:17.403+0000 7f61d9e00700 1 -- 172.21.15.153:0/1481571405 learned_addr learned my addr 172.21.15.153:0/1481571405 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:46:17.401 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:17.404+0000 7f61d9e00700 1 -- 172.21.15.153:0/1481571405 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f61dc105a40 msgr2=0x7f61dc101b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:46:17.401 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:17.404+0000 7f61d9400700 1 --2- 172.21.15.153:0/1481571405 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f61dc105a40 0x7f61dc101b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:46:17.401 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:17.404+0000 7f61d9e00700 1 --2- 172.21.15.153:0/1481571405 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f61dc105a40 0x7f61dc101b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:46:17.401 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:17.404+0000 7f61d9e00700 1 -- 172.21.15.153:0/1481571405 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61cc0095f0 con 0x7f61dc0ffbd0 2024-03-21T10:46:17.401 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:17.404+0000 7f61d9e00700 1 --2- 172.21.15.153:0/1481571405 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f61dc0ffbd0 0x7f61dc101610 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f61cc0092e0 tx=0x7f61cc00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:46:17.402 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:17.404+0000 7f61d2a00700 1 -- 172.21.15.153:0/1481571405 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f61cc01d070 con 0x7f61dc0ffbd0 2024-03-21T10:46:17.402 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:17.404+0000 7f61e0e00700 1 -- 172.21.15.153:0/1481571405 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61dc102450 con 0x7f61dc0ffbd0 2024-03-21T10:46:17.402 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:17.404+0000 7f61d2a00700 1 -- 172.21.15.153:0/1481571405 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f61cc005e00 con 0x7f61dc0ffbd0 2024-03-21T10:46:17.402 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:17.404+0000 7f61d2a00700 1 -- 172.21.15.153:0/1481571405 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f61cc022420 con 0x7f61dc0ffbd0 2024-03-21T10:46:17.403 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:17.404+0000 7f61e0e00700 1 -- 172.21.15.153:0/1481571405 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61dc1a2c70 con 0x7f61dc0ffbd0 2024-03-21T10:46:17.403 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:17.405+0000 7f61e0e00700 1 -- 172.21.15.153:0/1481571405 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61dc109050 con 0x7f61dc0ffbd0 2024-03-21T10:46:17.403 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:17.406+0000 7f61d2a00700 1 -- 172.21.15.153:0/1481571405 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f61cc00f460 con 0x7f61dc0ffbd0 2024-03-21T10:46:17.404 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:17.407+0000 7f61d2a00700 1 --2- 172.21.15.153:0/1481571405 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f61c006e460 0x7f61c0070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:46:17.404 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:17.407+0000 7f61d2a00700 1 -- 172.21.15.153:0/1481571405 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f61cc08fbf0 con 0x7f61dc0ffbd0 2024-03-21T10:46:17.404 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:17.407+0000 7f61d9400700 1 --2- 172.21.15.153:0/1481571405 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f61c006e460 0x7f61c0070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:46:17.405 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:17.408+0000 7f61d9400700 1 --2- 172.21.15.153:0/1481571405 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f61c006e460 0x7f61c0070920 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f61c8005950 tx=0x7f61c80058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:46:17.409 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:17.412+0000 7f61d2a00700 1 -- 172.21.15.153:0/1481571405 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f61cc058c10 con 0x7f61dc0ffbd0 2024-03-21T10:46:17.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:46:17 smithi153 ceph-mon[113733]: from='client.35826 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:46:17.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:46:17 smithi153 ceph-mon[113733]: pgmap v1034: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:46:17.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:46:17 smithi153 ceph-mon[113733]: from='client.35830 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:46:17.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:46:17 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/207979664' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:46:17.676 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:17.678+0000 7f61e0e00700 1 -- 172.21.15.153:0/1481571405 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f61dc108a30 con 0x7f61dc0ffbd0 2024-03-21T10:46:17.676 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:17.679+0000 7f61d2a00700 1 -- 172.21.15.153:0/1481571405 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1876 (secure 0 0 0) 0x7f61cc05c2d0 con 0x7f61dc0ffbd0 2024-03-21T10:46:17.677 INFO:teuthology.orchestra.run.smithi153.stdout:e32 2024-03-21T10:46:17.677 INFO:teuthology.orchestra.run.smithi153.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-21T10:46:17.677 INFO:teuthology.orchestra.run.smithi153.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:46:17.677 INFO:teuthology.orchestra.run.smithi153.stdout:legacy client fscid: 1 2024-03-21T10:46:17.677 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:46:17.678 INFO:teuthology.orchestra.run.smithi153.stdout:Filesystem 'cephfs' (1) 2024-03-21T10:46:17.678 INFO:teuthology.orchestra.run.smithi153.stdout:fs_name cephfs 2024-03-21T10:46:17.678 INFO:teuthology.orchestra.run.smithi153.stdout:epoch 23 2024-03-21T10:46:17.678 INFO:teuthology.orchestra.run.smithi153.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-21T10:46:17.678 INFO:teuthology.orchestra.run.smithi153.stdout:created 2024-03-21T10:04:27.086576+0000 2024-03-21T10:46:17.678 INFO:teuthology.orchestra.run.smithi153.stdout:modified 2024-03-21T10:19:05.022092+0000 2024-03-21T10:46:17.678 INFO:teuthology.orchestra.run.smithi153.stdout:tableserver 0 2024-03-21T10:46:17.678 INFO:teuthology.orchestra.run.smithi153.stdout:root 0 2024-03-21T10:46:17.678 INFO:teuthology.orchestra.run.smithi153.stdout:session_timeout 60 2024-03-21T10:46:17.678 INFO:teuthology.orchestra.run.smithi153.stdout:session_autoclose 300 2024-03-21T10:46:17.678 INFO:teuthology.orchestra.run.smithi153.stdout:max_file_size 1099511627776 2024-03-21T10:46:17.678 INFO:teuthology.orchestra.run.smithi153.stdout:max_xattr_size 65536 2024-03-21T10:46:17.678 INFO:teuthology.orchestra.run.smithi153.stdout:required_client_features {} 2024-03-21T10:46:17.678 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure 0 2024-03-21T10:46:17.679 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure_osd_epoch 94 2024-03-21T10:46:17.679 INFO:teuthology.orchestra.run.smithi153.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:46:17.679 INFO:teuthology.orchestra.run.smithi153.stdout:max_mds 2 2024-03-21T10:46:17.679 INFO:teuthology.orchestra.run.smithi153.stdout:in 0,1 2024-03-21T10:46:17.679 INFO:teuthology.orchestra.run.smithi153.stdout:up {} 2024-03-21T10:46:17.679 INFO:teuthology.orchestra.run.smithi153.stdout:failed 0,1 2024-03-21T10:46:17.679 INFO:teuthology.orchestra.run.smithi153.stdout:damaged 2024-03-21T10:46:17.679 INFO:teuthology.orchestra.run.smithi153.stdout:stopped 2024-03-21T10:46:17.679 INFO:teuthology.orchestra.run.smithi153.stdout:data_pools [3] 2024-03-21T10:46:17.679 INFO:teuthology.orchestra.run.smithi153.stdout:metadata_pool 2 2024-03-21T10:46:17.679 INFO:teuthology.orchestra.run.smithi153.stdout:inline_data enabled 2024-03-21T10:46:17.679 INFO:teuthology.orchestra.run.smithi153.stdout:balancer 2024-03-21T10:46:17.679 INFO:teuthology.orchestra.run.smithi153.stdout:bal_rank_mask -1 2024-03-21T10:46:17.679 INFO:teuthology.orchestra.run.smithi153.stdout:standby_count_wanted 1 2024-03-21T10:46:17.679 INFO:teuthology.orchestra.run.smithi153.stdout:qdb_cluster leader: 0 members: 2024-03-21T10:46:17.679 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:46:17.680 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:46:17.680 INFO:teuthology.orchestra.run.smithi153.stdout:Standby daemons: 2024-03-21T10:46:17.680 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:46:17.680 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.zipckr{-1:34442} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6826/1103846295,v1:172.21.15.153:6827/1103846295] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:46:17.680 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.iuyluj{-1:34474} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6828/1738630832,v1:172.21.15.153:6829/1738630832] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:46:17.680 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.yyimst{-1:44313} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.174:6824/4127786797,v1:172.21.15.174:6825/4127786797] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:46:17.680 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.nhozag{-1:44687} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.174:6826/2767517845,v1:172.21.15.174:6827/2767517845] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:46:17.681 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:17.682+0000 7f61e0e00700 1 -- 172.21.15.153:0/1481571405 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f61c006e460 msgr2=0x7f61c0070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:46:17.681 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:17.682+0000 7f61e0e00700 1 --2- 172.21.15.153:0/1481571405 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f61c006e460 0x7f61c0070920 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f61c8005950 tx=0x7f61c80058e0 comp rx=0 tx=0).stop 2024-03-21T10:46:17.681 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:17.682+0000 7f61e0e00700 1 -- 172.21.15.153:0/1481571405 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f61dc0ffbd0 msgr2=0x7f61dc101610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:46:17.681 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:17.682+0000 7f61e0e00700 1 --2- 172.21.15.153:0/1481571405 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f61dc0ffbd0 0x7f61dc101610 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f61cc0092e0 tx=0x7f61cc00be20 comp rx=0 tx=0).stop 2024-03-21T10:46:17.681 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:17.682+0000 7f61e0e00700 1 -- 172.21.15.153:0/1481571405 shutdown_connections 2024-03-21T10:46:17.681 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:17.682+0000 7f61e0e00700 1 --2- 172.21.15.153:0/1481571405 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f61c006e460 0x7f61c0070920 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:46:17.681 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:17.682+0000 7f61e0e00700 1 --2- 172.21.15.153:0/1481571405 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f61dc0ffbd0 0x7f61dc101610 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:46:17.681 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:17.682+0000 7f61e0e00700 1 --2- 172.21.15.153:0/1481571405 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f61dc105a40 0x7f61dc101b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:46:17.681 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:17.682+0000 7f61e0e00700 1 -- 172.21.15.153:0/1481571405 >> 172.21.15.153:0/1481571405 conn(0x7f61dc0755a0 msgr2=0x7f61dc0685f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:46:17.681 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:17.683+0000 7f61e0e00700 1 -- 172.21.15.153:0/1481571405 shutdown_connections 2024-03-21T10:46:17.682 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:17.683+0000 7f61e0e00700 1 -- 172.21.15.153:0/1481571405 wait complete. 2024-03-21T10:46:17.682 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 32 2024-03-21T10:46:17.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:46:17 smithi174 ceph-mon[88004]: from='client.35826 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:46:17.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:46:17 smithi174 ceph-mon[88004]: pgmap v1034: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:46:17.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:46:17 smithi174 ceph-mon[88004]: from='client.35830 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:46:17.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:46:17 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/207979664' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:46:17.836 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:17.838+0000 7f5fd3600700 1 -- 172.21.15.153:0/2541090299 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5fcc100da0 msgr2=0x7f5fcc101180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:46:17.836 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:17.838+0000 7f5fd3600700 1 --2- 172.21.15.153:0/2541090299 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5fcc100da0 0x7f5fcc101180 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f5fc0009910 tx=0x7f5fc0009c20 comp rx=0 tx=0).stop 2024-03-21T10:46:17.836 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:17.838+0000 7f5fd3600700 1 -- 172.21.15.153:0/2541090299 shutdown_connections 2024-03-21T10:46:17.836 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:17.838+0000 7f5fd3600700 1 --2- 172.21.15.153:0/2541090299 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5fcc101750 0x7f5fcc1057a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:46:17.836 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:17.838+0000 7f5fd3600700 1 --2- 172.21.15.153:0/2541090299 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5fcc100da0 0x7f5fcc101180 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:46:17.836 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:17.838+0000 7f5fd3600700 1 -- 172.21.15.153:0/2541090299 >> 172.21.15.153:0/2541090299 conn(0x7f5fcc0fc630 msgr2=0x7f5fcc0fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:46:17.836 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:17.838+0000 7f5fd3600700 1 -- 172.21.15.153:0/2541090299 shutdown_connections 2024-03-21T10:46:17.836 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:17.839+0000 7f5fd3600700 1 -- 172.21.15.153:0/2541090299 wait complete. 2024-03-21T10:46:17.837 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:17.839+0000 7f5fd3600700 1 Processor -- start 2024-03-21T10:46:17.837 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:17.840+0000 7f5fd3600700 1 -- start start 2024-03-21T10:46:17.838 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:17.840+0000 7f5fd3600700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5fcc100da0 0x7f5fcc198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:46:17.838 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:17.840+0000 7f5fd3600700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5fcc101750 0x7f5fcc198ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:46:17.838 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:17.840+0000 7f5fd3600700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fcc199560 con 0x7f5fcc101750 2024-03-21T10:46:17.838 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:17.840+0000 7f5fd3600700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fcc19d370 con 0x7f5fcc100da0 2024-03-21T10:46:17.838 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:17.840+0000 7f5fcbe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5fcc101750 0x7f5fcc198ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:46:17.838 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:17.840+0000 7f5fcbe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5fcc101750 0x7f5fcc198ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:57086/0 (socket says 172.21.15.153:57086) 2024-03-21T10:46:17.838 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:17.840+0000 7f5fcbe00700 1 -- 172.21.15.153:0/123965101 learned_addr learned my addr 172.21.15.153:0/123965101 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:46:17.838 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:17.841+0000 7f5fd0c00700 1 --2- 172.21.15.153:0/123965101 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5fcc100da0 0x7f5fcc198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:46:17.839 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:17.841+0000 7f5fcbe00700 1 -- 172.21.15.153:0/123965101 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5fcc100da0 msgr2=0x7f5fcc198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:46:17.839 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:17.841+0000 7f5fcbe00700 1 --2- 172.21.15.153:0/123965101 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5fcc100da0 0x7f5fcc198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:46:17.839 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:17.841+0000 7f5fcbe00700 1 -- 172.21.15.153:0/123965101 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fc00095f0 con 0x7f5fcc101750 2024-03-21T10:46:17.839 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:17.841+0000 7f5fcbe00700 1 --2- 172.21.15.153:0/123965101 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5fcc101750 0x7f5fcc198ed0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f5fb800c8f0 tx=0x7f5fb800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:46:17.839 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:17.841+0000 7f5fc9600700 1 -- 172.21.15.153:0/123965101 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5fb8007a50 con 0x7f5fcc101750 2024-03-21T10:46:17.839 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:17.841+0000 7f5fc9600700 1 -- 172.21.15.153:0/123965101 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5fb8007bb0 con 0x7f5fcc101750 2024-03-21T10:46:17.839 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:17.841+0000 7f5fc9600700 1 -- 172.21.15.153:0/123965101 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5fb8018630 con 0x7f5fcc101750 2024-03-21T10:46:17.839 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:17.841+0000 7f5fd3600700 1 -- 172.21.15.153:0/123965101 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fcc19d650 con 0x7f5fcc101750 2024-03-21T10:46:17.839 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:17.842+0000 7f5fd3600700 1 -- 172.21.15.153:0/123965101 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fcc19dba0 con 0x7f5fcc101750 2024-03-21T10:46:17.840 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:17.843+0000 7f5fd3600700 1 -- 172.21.15.153:0/123965101 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fcc104f20 con 0x7f5fcc101750 2024-03-21T10:46:17.840 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:17.844+0000 7f5fc9600700 1 -- 172.21.15.153:0/123965101 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f5fb8007d20 con 0x7f5fcc101750 2024-03-21T10:46:17.842 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:17.845+0000 7f5fc9600700 1 --2- 172.21.15.153:0/123965101 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f5fb406e460 0x7f5fb4070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:46:17.842 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:17.845+0000 7f5fc9600700 1 -- 172.21.15.153:0/123965101 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f5fb808e550 con 0x7f5fcc101750 2024-03-21T10:46:17.842 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:17.845+0000 7f5fd0c00700 1 --2- 172.21.15.153:0/123965101 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f5fb406e460 0x7f5fb4070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:46:17.842 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:17.846+0000 7f5fd0c00700 1 --2- 172.21.15.153:0/123965101 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f5fb406e460 0x7f5fb4070920 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f5fc0006010 tx=0x7f5fc001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:46:17.846 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:17.849+0000 7f5fc9600700 1 -- 172.21.15.153:0/123965101 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f5fb805b0a0 con 0x7f5fcc101750 2024-03-21T10:46:18.077 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:18.079+0000 7f5fd3600700 1 -- 172.21.15.153:0/123965101 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5fcc19de90 con 0x7f5fb406e460 2024-03-21T10:46:18.079 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:18.081+0000 7f5fc9600700 1 -- 172.21.15.153:0/123965101 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f5fcc19de90 con 0x7f5fb406e460 2024-03-21T10:46:18.079 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:46:18.079 INFO:teuthology.orchestra.run.smithi153.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232", 2024-03-21T10:46:18.079 INFO:teuthology.orchestra.run.smithi153.stdout: "in_progress": true, 2024-03-21T10:46:18.079 INFO:teuthology.orchestra.run.smithi153.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-21T10:46:18.079 INFO:teuthology.orchestra.run.smithi153.stdout: "services_complete": [ 2024-03-21T10:46:18.079 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr", 2024-03-21T10:46:18.079 INFO:teuthology.orchestra.run.smithi153.stdout: "mon", 2024-03-21T10:46:18.080 INFO:teuthology.orchestra.run.smithi153.stdout: "mds", 2024-03-21T10:46:18.080 INFO:teuthology.orchestra.run.smithi153.stdout: "crash", 2024-03-21T10:46:18.080 INFO:teuthology.orchestra.run.smithi153.stdout: "osd" 2024-03-21T10:46:18.080 INFO:teuthology.orchestra.run.smithi153.stdout: ], 2024-03-21T10:46:18.080 INFO:teuthology.orchestra.run.smithi153.stdout: "progress": "16/23 daemons upgraded", 2024-03-21T10:46:18.080 INFO:teuthology.orchestra.run.smithi153.stdout: "message": "Currently upgrading mds daemons", 2024-03-21T10:46:18.080 INFO:teuthology.orchestra.run.smithi153.stdout: "is_paused": false 2024-03-21T10:46:18.080 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:46:18.081 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:18.084+0000 7f5fd3600700 1 -- 172.21.15.153:0/123965101 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f5fb406e460 msgr2=0x7f5fb4070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:46:18.081 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:18.084+0000 7f5fd3600700 1 --2- 172.21.15.153:0/123965101 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f5fb406e460 0x7f5fb4070920 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f5fc0006010 tx=0x7f5fc001a040 comp rx=0 tx=0).stop 2024-03-21T10:46:18.082 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:18.084+0000 7f5fd3600700 1 -- 172.21.15.153:0/123965101 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5fcc101750 msgr2=0x7f5fcc198ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:46:18.082 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:18.084+0000 7f5fd3600700 1 --2- 172.21.15.153:0/123965101 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5fcc101750 0x7f5fcc198ed0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f5fb800c8f0 tx=0x7f5fb800ccb0 comp rx=0 tx=0).stop 2024-03-21T10:46:18.082 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:18.084+0000 7f5fd3600700 1 -- 172.21.15.153:0/123965101 shutdown_connections 2024-03-21T10:46:18.082 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:18.084+0000 7f5fd3600700 1 --2- 172.21.15.153:0/123965101 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5fcc100da0 0x7f5fcc198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:46:18.082 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:18.084+0000 7f5fd3600700 1 --2- 172.21.15.153:0/123965101 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f5fb406e460 0x7f5fb4070920 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:46:18.082 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:18.084+0000 7f5fd3600700 1 --2- 172.21.15.153:0/123965101 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5fcc101750 0x7f5fcc198ed0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:46:18.082 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:18.084+0000 7f5fd3600700 1 -- 172.21.15.153:0/123965101 >> 172.21.15.153:0/123965101 conn(0x7f5fcc0fc630 msgr2=0x7f5fcc0fea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:46:18.083 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:18.085+0000 7f5fd3600700 1 -- 172.21.15.153:0/123965101 shutdown_connections 2024-03-21T10:46:18.083 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:18.085+0000 7f5fd3600700 1 -- 172.21.15.153:0/123965101 wait complete. 2024-03-21T10:46:18.233 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:18.236+0000 7f7dbae00700 1 -- 172.21.15.153:0/4087469474 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7db41039b0 msgr2=0x7f7db4107a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:46:18.233 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:18.236+0000 7f7dbae00700 1 --2- 172.21.15.153:0/4087469474 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7db41039b0 0x7f7db4107a00 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f7da0009910 tx=0x7f7da0009c20 comp rx=0 tx=0).stop 2024-03-21T10:46:18.234 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:18.236+0000 7f7dbae00700 1 -- 172.21.15.153:0/4087469474 shutdown_connections 2024-03-21T10:46:18.234 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:18.236+0000 7f7dbae00700 1 --2- 172.21.15.153:0/4087469474 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7db41039b0 0x7f7db4107a00 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:46:18.234 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:18.236+0000 7f7dbae00700 1 --2- 172.21.15.153:0/4087469474 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7db4103000 0x7f7db41033e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:46:18.234 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:18.236+0000 7f7dbae00700 1 -- 172.21.15.153:0/4087469474 >> 172.21.15.153:0/4087469474 conn(0x7f7db40fe8d0 msgr2=0x7f7db4100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:46:18.234 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:18.236+0000 7f7dbae00700 1 -- 172.21.15.153:0/4087469474 shutdown_connections 2024-03-21T10:46:18.234 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:18.236+0000 7f7dbae00700 1 -- 172.21.15.153:0/4087469474 wait complete. 2024-03-21T10:46:18.235 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:18.237+0000 7f7dbae00700 1 Processor -- start 2024-03-21T10:46:18.235 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:18.237+0000 7f7dbae00700 1 -- start start 2024-03-21T10:46:18.235 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:18.238+0000 7f7dbae00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7db4103000 0x7f7db419e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:46:18.235 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:18.238+0000 7f7dbae00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7db41039b0 0x7f7db419ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:46:18.235 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:18.238+0000 7f7dbae00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7db419f390 con 0x7f7db4103000 2024-03-21T10:46:18.235 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:18.238+0000 7f7dbae00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7db4198840 con 0x7f7db41039b0 2024-03-21T10:46:18.236 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:18.238+0000 7f7db3e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7db4103000 0x7f7db419e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:46:18.236 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:18.238+0000 7f7db3e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7db4103000 0x7f7db419e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:57090/0 (socket says 172.21.15.153:57090) 2024-03-21T10:46:18.236 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:18.238+0000 7f7db3e00700 1 -- 172.21.15.153:0/3382062852 learned_addr learned my addr 172.21.15.153:0/3382062852 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:46:18.236 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:18.238+0000 7f7db3400700 1 --2- 172.21.15.153:0/3382062852 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7db41039b0 0x7f7db419ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:46:18.236 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:18.238+0000 7f7db3e00700 1 -- 172.21.15.153:0/3382062852 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7db41039b0 msgr2=0x7f7db419ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:46:18.236 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:18.238+0000 7f7db3e00700 1 --2- 172.21.15.153:0/3382062852 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7db41039b0 0x7f7db419ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:46:18.236 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:18.238+0000 7f7db3e00700 1 -- 172.21.15.153:0/3382062852 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7da00095f0 con 0x7f7db4103000 2024-03-21T10:46:18.237 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:18.239+0000 7f7db3e00700 1 --2- 172.21.15.153:0/3382062852 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7db4103000 0x7f7db419e7c0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f7da400c8a0 tx=0x7f7da400cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:46:18.237 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:18.239+0000 7f7db0c00700 1 -- 172.21.15.153:0/3382062852 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7da40078e0 con 0x7f7db4103000 2024-03-21T10:46:18.237 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:18.239+0000 7f7db0c00700 1 -- 172.21.15.153:0/3382062852 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7da400f450 con 0x7f7db4103000 2024-03-21T10:46:18.237 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:18.239+0000 7f7db0c00700 1 -- 172.21.15.153:0/3382062852 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7da400e5e0 con 0x7f7db4103000 2024-03-21T10:46:18.237 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:18.239+0000 7f7dbae00700 1 -- 172.21.15.153:0/3382062852 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7db4198b20 con 0x7f7db4103000 2024-03-21T10:46:18.237 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:18.239+0000 7f7dbae00700 1 -- 172.21.15.153:0/3382062852 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7db4199070 con 0x7f7db4103000 2024-03-21T10:46:18.237 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:18.240+0000 7f7dbae00700 1 -- 172.21.15.153:0/3382062852 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7db410b400 con 0x7f7db4103000 2024-03-21T10:46:18.238 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:18.241+0000 7f7db0c00700 1 -- 172.21.15.153:0/3382062852 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f7da400f5c0 con 0x7f7db4103000 2024-03-21T10:46:18.240 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:18.242+0000 7f7db0c00700 1 --2- 172.21.15.153:0/3382062852 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f7d9806e460 0x7f7d98070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:46:18.240 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:18.242+0000 7f7db0c00700 1 -- 172.21.15.153:0/3382062852 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f7da408e430 con 0x7f7db4103000 2024-03-21T10:46:18.240 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:18.243+0000 7f7db3400700 1 --2- 172.21.15.153:0/3382062852 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f7d9806e460 0x7f7d98070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:46:18.240 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:18.243+0000 7f7db3400700 1 --2- 172.21.15.153:0/3382062852 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f7d9806e460 0x7f7d98070920 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f7db40ff0a0 tx=0x7f7da001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:46:18.244 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:18.247+0000 7f7db0c00700 1 -- 172.21.15.153:0/3382062852 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7da405b000 con 0x7f7db4103000 2024-03-21T10:46:18.550 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:18.552+0000 7f7dbae00700 1 -- 172.21.15.153:0/3382062852 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f7db4199340 con 0x7f7db4103000 2024-03-21T10:46:18.551 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:18.553+0000 7f7db0c00700 1 -- 172.21.15.153:0/3382062852 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4493 (secure 0 0 0) 0x7f7da405ab90 con 0x7f7db4103000 2024-03-21T10:46:18.551 INFO:teuthology.orchestra.run.smithi153.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-03-21T10:46:18.551 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-03-21T10:46:18.551 INFO:teuthology.orchestra.run.smithi153.stdout: host smithi174 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi174/config 2024-03-21T10:46:18.551 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/podman: timeout after 895 seconds 2024-03-21T10:46:18.551 INFO:teuthology.orchestra.run.smithi153.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 -e NODE_NAME=smithi174 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/run/ceph:z -v /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpd910r3_i:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 inventory --format=json-pretty --filter-for-batch 2024-03-21T10:46:18.552 INFO:teuthology.orchestra.run.smithi153.stdout:Traceback (most recent call last): 2024-03-21T10:46:18.552 INFO:teuthology.orchestra.run.smithi153.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-03-21T10:46:18.552 INFO:teuthology.orchestra.run.smithi153.stdout: return _run_code(code, main_globals, None, 2024-03-21T10:46:18.552 INFO:teuthology.orchestra.run.smithi153.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-03-21T10:46:18.552 INFO:teuthology.orchestra.run.smithi153.stdout: exec(code, run_globals) 2024-03-21T10:46:18.552 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 5606, in 2024-03-21T10:46:18.552 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 5594, in main 2024-03-21T10:46:18.552 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 406, in _infer_config 2024-03-21T10:46:18.552 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 321, in _infer_fsid 2024-03-21T10:46:18.552 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 434, in _infer_image 2024-03-21T10:46:18.552 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 308, in _validate_fsid 2024-03-21T10:46:18.552 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 3357, in command_ceph_volume 2024-03-21T10:46:18.552 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-03-21T10:46:18.552 INFO:teuthology.orchestra.run.smithi153.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 -e NODE_NAME=smithi174 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/run/ceph:z -v /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpd910r3_i:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 inventory --format=json-pretty --filter-for-batch 2024-03-21T10:46:18.552 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-21T10:46:18.553 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is degraded 2024-03-21T10:46:18.553 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-21T10:46:18.553 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-21T10:46:18.553 INFO:teuthology.orchestra.run.smithi153.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-21T10:46:18.553 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-21T10:46:18.553 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:18.556+0000 7f7dbae00700 1 -- 172.21.15.153:0/3382062852 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f7d9806e460 msgr2=0x7f7d98070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:46:18.553 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:18.556+0000 7f7dbae00700 1 --2- 172.21.15.153:0/3382062852 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f7d9806e460 0x7f7d98070920 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f7db40ff0a0 tx=0x7f7da001a040 comp rx=0 tx=0).stop 2024-03-21T10:46:18.554 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:18.556+0000 7f7dbae00700 1 -- 172.21.15.153:0/3382062852 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7db4103000 msgr2=0x7f7db419e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:46:18.554 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:18.556+0000 7f7dbae00700 1 --2- 172.21.15.153:0/3382062852 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7db4103000 0x7f7db419e7c0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f7da400c8a0 tx=0x7f7da400cbb0 comp rx=0 tx=0).stop 2024-03-21T10:46:18.554 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:18.557+0000 7f7dbae00700 1 -- 172.21.15.153:0/3382062852 shutdown_connections 2024-03-21T10:46:18.554 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:18.557+0000 7f7dbae00700 1 --2- 172.21.15.153:0/3382062852 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f7d9806e460 0x7f7d98070920 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:46:18.555 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:18.557+0000 7f7dbae00700 1 --2- 172.21.15.153:0/3382062852 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7db4103000 0x7f7db419e7c0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:46:18.555 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:18.557+0000 7f7dbae00700 1 --2- 172.21.15.153:0/3382062852 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7db41039b0 0x7f7db419ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:46:18.555 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:18.557+0000 7f7dbae00700 1 -- 172.21.15.153:0/3382062852 >> 172.21.15.153:0/3382062852 conn(0x7f7db40fe8d0 msgr2=0x7f7db4100cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:46:18.555 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:18.557+0000 7f7dbae00700 1 -- 172.21.15.153:0/3382062852 shutdown_connections 2024-03-21T10:46:18.555 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:18.557+0000 7f7dbae00700 1 -- 172.21.15.153:0/3382062852 wait complete. 2024-03-21T10:46:18.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:46:18 smithi174 ceph-mon[88004]: from='client.35834 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:46:18.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:46:18 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/1481571405' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:46:18.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:46:18 smithi153 ceph-mon[113733]: from='client.35834 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:46:18.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:46:18 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/1481571405' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:46:19.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:46:19 smithi174 ceph-mon[88004]: pgmap v1035: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:46:19.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:46:19 smithi174 ceph-mon[88004]: from='client.35846 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:46:19.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:46:19 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/3382062852' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:46:19.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:46:19 smithi153 ceph-mon[113733]: pgmap v1035: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:46:19.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:46:19 smithi153 ceph-mon[113733]: from='client.35846 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:46:19.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:46:19 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/3382062852' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:46:21.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:46:21 smithi174 ceph-mon[88004]: pgmap v1036: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:46:21.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:46:21 smithi153 ceph-mon[113733]: pgmap v1036: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:46:23.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:46:23 smithi174 ceph-mon[88004]: pgmap v1037: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:46:23.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:46:23 smithi153 ceph-mon[113733]: pgmap v1037: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:46:25.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:46:25 smithi174 ceph-mon[88004]: pgmap v1038: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:46:25.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:46:25 smithi153 ceph-mon[113733]: pgmap v1038: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:46:27.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:46:27 smithi174 ceph-mon[88004]: pgmap v1039: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:46:27.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:46:27 smithi153 ceph-mon[113733]: pgmap v1039: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:46:29.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:46:29 smithi174 ceph-mon[88004]: pgmap v1040: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:46:29.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:46:29 smithi153 ceph-mon[113733]: pgmap v1040: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:46:31.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:46:31 smithi174 ceph-mon[88004]: pgmap v1041: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:46:31.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:46:31 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:46:31.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:46:31 smithi153 ceph-mon[113733]: pgmap v1041: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:46:31.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:46:31 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:46:33.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:46:33 smithi153 ceph-mon[113733]: pgmap v1042: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:46:33.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:46:33 smithi174 ceph-mon[88004]: pgmap v1042: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:46:35.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:46:35 smithi153 ceph-mon[113733]: pgmap v1043: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:46:35.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:46:35 smithi174 ceph-mon[88004]: pgmap v1043: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:46:37.801 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:46:37 smithi153 ceph-mon[113733]: pgmap v1044: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:46:37.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:46:37 smithi174 ceph-mon[88004]: pgmap v1044: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:46:39.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:46:39 smithi153 ceph-mon[113733]: pgmap v1045: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:46:39.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:46:39 smithi174 ceph-mon[88004]: pgmap v1045: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:46:41.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:46:41 smithi153 ceph-mon[113733]: pgmap v1046: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:46:41.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:46:41 smithi174 ceph-mon[88004]: pgmap v1046: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:46:43.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:46:43 smithi174 ceph-mon[88004]: pgmap v1047: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:46:43.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:46:43 smithi153 ceph-mon[113733]: pgmap v1047: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:46:45.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:46:45 smithi153 ceph-mon[113733]: pgmap v1048: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:46:45.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:46:45 smithi174 ceph-mon[88004]: pgmap v1048: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:46:46.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:46:46 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:46:46.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:46:46 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:46:47.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:46:47 smithi153 ceph-mon[113733]: pgmap v1049: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:46:47.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:46:47 smithi174 ceph-mon[88004]: pgmap v1049: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:46:48.723 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:48.726+0000 7f5da9000700 1 -- 172.21.15.153:0/3932564004 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5da4100da0 msgr2=0x7f5da4101180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:46:48.723 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:48.726+0000 7f5da9000700 1 --2- 172.21.15.153:0/3932564004 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5da4100da0 0x7f5da4101180 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f5d94009910 tx=0x7f5d94009c20 comp rx=0 tx=0).stop 2024-03-21T10:46:48.723 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:48.726+0000 7f5da9000700 1 -- 172.21.15.153:0/3932564004 shutdown_connections 2024-03-21T10:46:48.723 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:48.726+0000 7f5da9000700 1 --2- 172.21.15.153:0/3932564004 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5da4101750 0x7f5da41057a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:46:48.723 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:48.726+0000 7f5da9000700 1 --2- 172.21.15.153:0/3932564004 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5da4100da0 0x7f5da4101180 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:46:48.723 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:48.726+0000 7f5da9000700 1 -- 172.21.15.153:0/3932564004 >> 172.21.15.153:0/3932564004 conn(0x7f5da40fc630 msgr2=0x7f5da40fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:46:48.723 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:48.727+0000 7f5da9000700 1 -- 172.21.15.153:0/3932564004 shutdown_connections 2024-03-21T10:46:48.723 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:48.727+0000 7f5da9000700 1 -- 172.21.15.153:0/3932564004 wait complete. 2024-03-21T10:46:48.724 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:48.728+0000 7f5da9000700 1 Processor -- start 2024-03-21T10:46:48.724 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:48.728+0000 7f5da9000700 1 -- start start 2024-03-21T10:46:48.725 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:48.728+0000 7f5da9000700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5da4100da0 0x7f5da419e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:46:48.725 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:48.728+0000 7f5da9000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5da4101750 0x7f5da419ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:46:48.725 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:48.728+0000 7f5da9000700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5da419f410 con 0x7f5da4101750 2024-03-21T10:46:48.725 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:48.728+0000 7f5da9000700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5da41988c0 con 0x7f5da4100da0 2024-03-21T10:46:48.725 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:48.729+0000 7f5d99400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5da4101750 0x7f5da419ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:46:48.725 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:48.729+0000 7f5d99400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5da4101750 0x7f5da419ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:43642/0 (socket says 172.21.15.153:43642) 2024-03-21T10:46:48.725 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:48.729+0000 7f5d99400700 1 -- 172.21.15.153:0/60546104 learned_addr learned my addr 172.21.15.153:0/60546104 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:46:48.726 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:48.729+0000 7f5da1e00700 1 --2- 172.21.15.153:0/60546104 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5da4100da0 0x7f5da419e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:46:48.726 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:48.729+0000 7f5d99400700 1 -- 172.21.15.153:0/60546104 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5da4100da0 msgr2=0x7f5da419e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:46:48.726 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:48.729+0000 7f5d99400700 1 --2- 172.21.15.153:0/60546104 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5da4100da0 0x7f5da419e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:46:48.726 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:48.729+0000 7f5d99400700 1 -- 172.21.15.153:0/60546104 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d940095f0 con 0x7f5da4101750 2024-03-21T10:46:48.726 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:48.729+0000 7f5d99400700 1 --2- 172.21.15.153:0/60546104 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5da4101750 0x7f5da419ed80 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c00b480 tx=0x7f5d8c00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:46:48.726 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:48.730+0000 7f5d9b400700 1 -- 172.21.15.153:0/60546104 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5d8c0118c0 con 0x7f5da4101750 2024-03-21T10:46:48.726 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:48.730+0000 7f5d9b400700 1 -- 172.21.15.153:0/60546104 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5d8c010380 con 0x7f5da4101750 2024-03-21T10:46:48.727 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:48.730+0000 7f5d9b400700 1 -- 172.21.15.153:0/60546104 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5d8c00f4f0 con 0x7f5da4101750 2024-03-21T10:46:48.727 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:48.730+0000 7f5da9000700 1 -- 172.21.15.153:0/60546104 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5da4198ba0 con 0x7f5da4101750 2024-03-21T10:46:48.727 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:48.730+0000 7f5da9000700 1 -- 172.21.15.153:0/60546104 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5da41990f0 con 0x7f5da4101750 2024-03-21T10:46:48.727 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:48.731+0000 7f5da9000700 1 -- 172.21.15.153:0/60546104 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5da4104f20 con 0x7f5da4101750 2024-03-21T10:46:48.728 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:48.732+0000 7f5d9b400700 1 -- 172.21.15.153:0/60546104 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f5d8c0104f0 con 0x7f5da4101750 2024-03-21T10:46:48.729 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:48.733+0000 7f5d9b400700 1 --2- 172.21.15.153:0/60546104 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f5d8806e4b0 0x7f5d88070970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:46:48.729 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:48.733+0000 7f5d9b400700 1 -- 172.21.15.153:0/60546104 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f5d8c08dfe0 con 0x7f5da4101750 2024-03-21T10:46:48.729 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:48.733+0000 7f5da1e00700 1 --2- 172.21.15.153:0/60546104 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f5d8806e4b0 0x7f5d88070970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:46:48.730 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:48.734+0000 7f5da1e00700 1 --2- 172.21.15.153:0/60546104 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f5d8806e4b0 0x7f5d88070970 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f5d94000c00 tx=0x7f5d94005fd0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:46:48.734 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:48.739+0000 7f5d9b400700 1 -- 172.21.15.153:0/60546104 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f5d8c05ac70 con 0x7f5da4101750 2024-03-21T10:46:48.979 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:48.982+0000 7f5da9000700 1 -- 172.21.15.153:0/60546104 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5da4199430 con 0x7f5d8806e4b0 2024-03-21T10:46:48.981 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:48.985+0000 7f5d9b400700 1 -- 172.21.15.153:0/60546104 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f5da4199430 con 0x7f5d8806e4b0 2024-03-21T10:46:48.985 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:48.988+0000 7f5da9000700 1 -- 172.21.15.153:0/60546104 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f5d8806e4b0 msgr2=0x7f5d88070970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:46:48.985 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:48.988+0000 7f5da9000700 1 --2- 172.21.15.153:0/60546104 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f5d8806e4b0 0x7f5d88070970 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f5d94000c00 tx=0x7f5d94005fd0 comp rx=0 tx=0).stop 2024-03-21T10:46:48.985 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:48.988+0000 7f5da9000700 1 -- 172.21.15.153:0/60546104 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5da4101750 msgr2=0x7f5da419ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:46:48.985 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:48.988+0000 7f5da9000700 1 --2- 172.21.15.153:0/60546104 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5da4101750 0x7f5da419ed80 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c00b480 tx=0x7f5d8c00b840 comp rx=0 tx=0).stop 2024-03-21T10:46:48.985 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:48.989+0000 7f5da9000700 1 -- 172.21.15.153:0/60546104 shutdown_connections 2024-03-21T10:46:48.985 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:48.989+0000 7f5da9000700 1 --2- 172.21.15.153:0/60546104 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5da4100da0 0x7f5da419e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:46:48.985 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:48.989+0000 7f5da9000700 1 --2- 172.21.15.153:0/60546104 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f5d8806e4b0 0x7f5d88070970 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:46:48.985 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:48.989+0000 7f5da9000700 1 --2- 172.21.15.153:0/60546104 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5da4101750 0x7f5da419ed80 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:46:48.985 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:48.989+0000 7f5da9000700 1 -- 172.21.15.153:0/60546104 >> 172.21.15.153:0/60546104 conn(0x7f5da40fc630 msgr2=0x7f5da40fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:46:48.985 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:48.989+0000 7f5da9000700 1 -- 172.21.15.153:0/60546104 shutdown_connections 2024-03-21T10:46:48.985 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:48.989+0000 7f5da9000700 1 -- 172.21.15.153:0/60546104 wait complete. 2024-03-21T10:46:49.002 INFO:teuthology.orchestra.run.smithi153.stdout:true 2024-03-21T10:46:49.141 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.145+0000 7fe1f1e00700 1 -- 172.21.15.153:0/2784398402 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe1ec103000 msgr2=0x7fe1ec1033e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:46:49.141 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.145+0000 7fe1f1e00700 1 --2- 172.21.15.153:0/2784398402 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe1ec103000 0x7fe1ec1033e0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc009910 tx=0x7fe1dc009c20 comp rx=0 tx=0).stop 2024-03-21T10:46:49.141 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.145+0000 7fe1f1e00700 1 -- 172.21.15.153:0/2784398402 shutdown_connections 2024-03-21T10:46:49.141 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.145+0000 7fe1f1e00700 1 --2- 172.21.15.153:0/2784398402 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe1ec1039b0 0x7fe1ec107a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:46:49.142 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.145+0000 7fe1f1e00700 1 --2- 172.21.15.153:0/2784398402 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe1ec103000 0x7fe1ec1033e0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:46:49.142 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.145+0000 7fe1f1e00700 1 -- 172.21.15.153:0/2784398402 >> 172.21.15.153:0/2784398402 conn(0x7fe1ec0fe8d0 msgr2=0x7fe1ec100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:46:49.142 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.145+0000 7fe1f1e00700 1 -- 172.21.15.153:0/2784398402 shutdown_connections 2024-03-21T10:46:49.142 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.145+0000 7fe1f1e00700 1 -- 172.21.15.153:0/2784398402 wait complete. 2024-03-21T10:46:49.142 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.146+0000 7fe1f1e00700 1 Processor -- start 2024-03-21T10:46:49.142 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.147+0000 7fe1f1e00700 1 -- start start 2024-03-21T10:46:49.143 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.147+0000 7fe1f1e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe1ec103000 0x7fe1ec19e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:46:49.144 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.147+0000 7fe1f1e00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe1ec1039b0 0x7fe1ec19ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:46:49.144 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.147+0000 7fe1f1e00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1ec19f4d0 con 0x7fe1ec103000 2024-03-21T10:46:49.144 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.147+0000 7fe1f1e00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1ec198980 con 0x7fe1ec1039b0 2024-03-21T10:46:49.144 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.148+0000 7fe1eaa00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe1ec103000 0x7fe1ec19e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:46:49.144 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.148+0000 7fe1eaa00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe1ec103000 0x7fe1ec19e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:43652/0 (socket says 172.21.15.153:43652) 2024-03-21T10:46:49.144 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.148+0000 7fe1eaa00700 1 -- 172.21.15.153:0/2328025050 learned_addr learned my addr 172.21.15.153:0/2328025050 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:46:49.145 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.148+0000 7fe1ea000700 1 --2- 172.21.15.153:0/2328025050 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe1ec1039b0 0x7fe1ec19ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:46:49.145 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.148+0000 7fe1eaa00700 1 -- 172.21.15.153:0/2328025050 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe1ec1039b0 msgr2=0x7fe1ec19ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:46:49.145 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.148+0000 7fe1eaa00700 1 --2- 172.21.15.153:0/2328025050 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe1ec1039b0 0x7fe1ec19ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:46:49.145 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.148+0000 7fe1eaa00700 1 -- 172.21.15.153:0/2328025050 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1dc0095f0 con 0x7fe1ec103000 2024-03-21T10:46:49.145 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.149+0000 7fe1eaa00700 1 --2- 172.21.15.153:0/2328025050 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe1ec103000 0x7fe1ec19e900 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc00b5c0 tx=0x7fe1dc00be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:46:49.145 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.149+0000 7fe1f0a00700 1 -- 172.21.15.153:0/2328025050 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe1dc01d070 con 0x7fe1ec103000 2024-03-21T10:46:49.145 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.149+0000 7fe1f0a00700 1 -- 172.21.15.153:0/2328025050 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe1dc00f460 con 0x7fe1ec103000 2024-03-21T10:46:49.146 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.149+0000 7fe1f1e00700 1 -- 172.21.15.153:0/2328025050 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1ec198c00 con 0x7fe1ec103000 2024-03-21T10:46:49.146 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.149+0000 7fe1f0a00700 1 -- 172.21.15.153:0/2328025050 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe1dc017610 con 0x7fe1ec103000 2024-03-21T10:46:49.146 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.149+0000 7fe1f1e00700 1 -- 172.21.15.153:0/2328025050 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1ec1990f0 con 0x7fe1ec103000 2024-03-21T10:46:49.146 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.150+0000 7fe1f1e00700 1 -- 172.21.15.153:0/2328025050 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1ec10b3a0 con 0x7fe1ec103000 2024-03-21T10:46:49.147 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.151+0000 7fe1f0a00700 1 -- 172.21.15.153:0/2328025050 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fe1dc00f5d0 con 0x7fe1ec103000 2024-03-21T10:46:49.148 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.152+0000 7fe1f0a00700 1 --2- 172.21.15.153:0/2328025050 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe1d006e460 0x7fe1d0070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:46:49.149 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.152+0000 7fe1f0a00700 1 -- 172.21.15.153:0/2328025050 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fe1dc08fad0 con 0x7fe1ec103000 2024-03-21T10:46:49.149 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.153+0000 7fe1ea000700 1 --2- 172.21.15.153:0/2328025050 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe1d006e460 0x7fe1d0070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:46:49.149 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.153+0000 7fe1ea000700 1 --2- 172.21.15.153:0/2328025050 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe1d006e460 0x7fe1d0070920 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fe1d8006fd0 tx=0x7fe1d8008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:46:49.153 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.157+0000 7fe1f0a00700 1 -- 172.21.15.153:0/2328025050 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe1dc05c6a0 con 0x7fe1ec103000 2024-03-21T10:46:49.390 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.394+0000 7fe1f1e00700 1 -- 172.21.15.153:0/2328025050 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe1ec199360 con 0x7fe1d006e460 2024-03-21T10:46:49.392 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.396+0000 7fe1f0a00700 1 -- 172.21.15.153:0/2328025050 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fe1ec199360 con 0x7fe1d006e460 2024-03-21T10:46:49.395 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.399+0000 7fe1f1e00700 1 -- 172.21.15.153:0/2328025050 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe1d006e460 msgr2=0x7fe1d0070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:46:49.396 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.399+0000 7fe1f1e00700 1 --2- 172.21.15.153:0/2328025050 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe1d006e460 0x7fe1d0070920 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fe1d8006fd0 tx=0x7fe1d8008040 comp rx=0 tx=0).stop 2024-03-21T10:46:49.396 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.399+0000 7fe1f1e00700 1 -- 172.21.15.153:0/2328025050 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe1ec103000 msgr2=0x7fe1ec19e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:46:49.396 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.399+0000 7fe1f1e00700 1 --2- 172.21.15.153:0/2328025050 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe1ec103000 0x7fe1ec19e900 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc00b5c0 tx=0x7fe1dc00be00 comp rx=0 tx=0).stop 2024-03-21T10:46:49.396 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.399+0000 7fe1f1e00700 1 -- 172.21.15.153:0/2328025050 shutdown_connections 2024-03-21T10:46:49.396 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.399+0000 7fe1f1e00700 1 --2- 172.21.15.153:0/2328025050 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe1d006e460 0x7fe1d0070920 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:46:49.396 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.399+0000 7fe1f1e00700 1 --2- 172.21.15.153:0/2328025050 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe1ec103000 0x7fe1ec19e900 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:46:49.396 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.399+0000 7fe1f1e00700 1 --2- 172.21.15.153:0/2328025050 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe1ec1039b0 0x7fe1ec19ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:46:49.396 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.399+0000 7fe1f1e00700 1 -- 172.21.15.153:0/2328025050 >> 172.21.15.153:0/2328025050 conn(0x7fe1ec0fe8d0 msgr2=0x7fe1ec0fff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:46:49.396 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.399+0000 7fe1f1e00700 1 -- 172.21.15.153:0/2328025050 shutdown_connections 2024-03-21T10:46:49.396 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.400+0000 7fe1f1e00700 1 -- 172.21.15.153:0/2328025050 wait complete. 2024-03-21T10:46:49.551 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.554+0000 7f9f67c00700 1 -- 172.21.15.153:0/906280555 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9f601039b0 msgr2=0x7f9f60107a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:46:49.551 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.554+0000 7f9f67c00700 1 --2- 172.21.15.153:0/906280555 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9f601039b0 0x7f9f60107a00 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c009910 tx=0x7f9f4c009c20 comp rx=0 tx=0).stop 2024-03-21T10:46:49.551 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.554+0000 7f9f67c00700 1 -- 172.21.15.153:0/906280555 shutdown_connections 2024-03-21T10:46:49.551 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.554+0000 7f9f67c00700 1 --2- 172.21.15.153:0/906280555 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9f601039b0 0x7f9f60107a00 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:46:49.551 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.554+0000 7f9f67c00700 1 --2- 172.21.15.153:0/906280555 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f60103000 0x7f9f601033e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:46:49.551 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.555+0000 7f9f67c00700 1 -- 172.21.15.153:0/906280555 >> 172.21.15.153:0/906280555 conn(0x7f9f600fe8d0 msgr2=0x7f9f60100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:46:49.552 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.555+0000 7f9f67c00700 1 -- 172.21.15.153:0/906280555 shutdown_connections 2024-03-21T10:46:49.552 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.555+0000 7f9f67c00700 1 -- 172.21.15.153:0/906280555 wait complete. 2024-03-21T10:46:49.552 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.556+0000 7f9f67c00700 1 Processor -- start 2024-03-21T10:46:49.553 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.556+0000 7f9f67c00700 1 -- start start 2024-03-21T10:46:49.553 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.556+0000 7f9f67c00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f60103000 0x7f9f6019e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:46:49.553 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.556+0000 7f9f67c00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9f601039b0 0x7f9f6019ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:46:49.553 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.556+0000 7f9f67c00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f6019f390 con 0x7f9f601039b0 2024-03-21T10:46:49.553 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.556+0000 7f9f67c00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f60198840 con 0x7f9f60103000 2024-03-21T10:46:49.553 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.557+0000 7f9f5fe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9f601039b0 0x7f9f6019ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:46:49.553 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.557+0000 7f9f5fe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9f601039b0 0x7f9f6019ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:43672/0 (socket says 172.21.15.153:43672) 2024-03-21T10:46:49.553 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.557+0000 7f9f5fe00700 1 -- 172.21.15.153:0/4207823772 learned_addr learned my addr 172.21.15.153:0/4207823772 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:46:49.554 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.557+0000 7f9f5fe00700 1 -- 172.21.15.153:0/4207823772 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f60103000 msgr2=0x7f9f6019e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:46:49.554 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.557+0000 7f9f5fe00700 1 --2- 172.21.15.153:0/4207823772 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f60103000 0x7f9f6019e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:46:49.554 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.557+0000 7f9f5fe00700 1 -- 172.21.15.153:0/4207823772 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f4c0095f0 con 0x7f9f601039b0 2024-03-21T10:46:49.554 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.557+0000 7f9f5fe00700 1 --2- 172.21.15.153:0/4207823772 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9f601039b0 0x7f9f6019ed00 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c00bda0 tx=0x7f9f4c00be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:46:49.555 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.558+0000 7f9f5d600700 1 -- 172.21.15.153:0/4207823772 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f4c01d070 con 0x7f9f601039b0 2024-03-21T10:46:49.555 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.558+0000 7f9f5d600700 1 -- 172.21.15.153:0/4207823772 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9f4c004fb0 con 0x7f9f601039b0 2024-03-21T10:46:49.555 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.558+0000 7f9f67c00700 1 -- 172.21.15.153:0/4207823772 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f60198ac0 con 0x7f9f601039b0 2024-03-21T10:46:49.555 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.558+0000 7f9f5d600700 1 -- 172.21.15.153:0/4207823772 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f4c00f460 con 0x7f9f601039b0 2024-03-21T10:46:49.555 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.558+0000 7f9f67c00700 1 -- 172.21.15.153:0/4207823772 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f60198fb0 con 0x7f9f601039b0 2024-03-21T10:46:49.555 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.559+0000 7f9f67c00700 1 -- 172.21.15.153:0/4207823772 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f6010b400 con 0x7f9f601039b0 2024-03-21T10:46:49.556 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.560+0000 7f9f5d600700 1 -- 172.21.15.153:0/4207823772 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f9f4c022800 con 0x7f9f601039b0 2024-03-21T10:46:49.557 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.561+0000 7f9f5d600700 1 --2- 172.21.15.153:0/4207823772 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f9f4806e460 0x7f9f48070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:46:49.557 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.561+0000 7f9f5d600700 1 -- 172.21.15.153:0/4207823772 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f9f4c08fa20 con 0x7f9f601039b0 2024-03-21T10:46:49.557 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.561+0000 7f9f65200700 1 --2- 172.21.15.153:0/4207823772 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f9f4806e460 0x7f9f48070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:46:49.558 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.562+0000 7f9f65200700 1 --2- 172.21.15.153:0/4207823772 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f9f4806e460 0x7f9f48070920 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f9f5400a990 tx=0x7f9f54005c80 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:46:49.562 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.566+0000 7f9f5d600700 1 -- 172.21.15.153:0/4207823772 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f9f4c05c7c0 con 0x7f9f601039b0 2024-03-21T10:46:49.786 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.790+0000 7f9f67c00700 1 -- 172.21.15.153:0/4207823772 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f9f60199220 con 0x7f9f4806e460 2024-03-21T10:46:49.787 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:46:49 smithi153 ceph-mon[113733]: pgmap v1050: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:46:49.804 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.808+0000 7f9f5d600700 1 -- 172.21.15.153:0/4207823772 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f9f60199220 con 0x7f9f4806e460 2024-03-21T10:46:49.804 INFO:teuthology.orchestra.run.smithi153.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-21T10:46:49.804 INFO:teuthology.orchestra.run.smithi153.stdout:alertmanager.smithi153 smithi153 *:9093,9094 running (35m) 11m ago 49m 15.6M - 0.25.0 c8568f914cd2 9ed3f2200b5f 2024-03-21T10:46:49.804 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi153 smithi153 running (49m) 11m ago 49m 10.4M - 18.2.1 5be31c24972a 5b61994396a6 2024-03-21T10:46:49.805 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi174 smithi174 running (48m) 11m ago 48m 10.1M - 18.2.1 5be31c24972a 4f7a5fc3ac28 2024-03-21T10:46:49.805 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi153 smithi153 running (31m) 11m ago 49m 6656k - 19.0.0-2231-g2edfee68 237e43c7a65f 32c21104d5b0 2024-03-21T10:46:49.805 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi174 smithi174 running (31m) 11m ago 48m 6660k - 19.0.0-2231-g2edfee68 237e43c7a65f a04e48967ab6 2024-03-21T10:46:49.805 INFO:teuthology.orchestra.run.smithi153.stdout:grafana.smithi153 smithi153 *:3000 running (35m) 11m ago 48m 75.2M - 9.4.12 2bacad6d85d8 ee24026ab786 2024-03-21T10:46:49.805 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.iuyluj smithi153 running (27m) 11m ago 42m 18.7M - 19.0.0-2231-g2edfee68 237e43c7a65f 87aff8a5de70 2024-03-21T10:46:49.805 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.zipckr smithi153 running (27m) 11m ago 42m 18.9M - 19.0.0-2231-g2edfee68 237e43c7a65f ccc262bd5dd3 2024-03-21T10:46:49.805 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.nhozag smithi174 running (11m) 11m ago 42m 10.9M - 19.0.0-2231-g2edfee68 237e43c7a65f 9e02aea72c96 2024-03-21T10:46:49.805 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.yyimst smithi174 running (26m) 11m ago 42m 18.5M - 19.0.0-2231-g2edfee68 237e43c7a65f fc1962f9351e 2024-03-21T10:46:49.805 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi153.tluxni smithi153 *:8443,9283,8765 running (38m) 11m ago 51m 512M - 19.0.0-2231-g2edfee68 237e43c7a65f 7ffee0e55382 2024-03-21T10:46:49.805 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi174.qpcwat smithi174 *:8443,9283,8765 running (37m) 11m ago 47m 459M - 19.0.0-2231-g2edfee68 237e43c7a65f 748968f216ab 2024-03-21T10:46:49.805 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi153 smithi153 running (33m) 11m ago 51m 70.8M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f dd8038303dc3 2024-03-21T10:46:49.805 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi174 smithi174 running (32m) 11m ago 47m 51.9M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f 29d78a788f2c 2024-03-21T10:46:49.805 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi153 smithi153 *:9100 running (36m) 11m ago 49m 8140k - 1.5.0 0da6a335fe13 bb835edc0e0b 2024-03-21T10:46:49.806 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi174 smithi174 *:9100 running (36m) 11m ago 47m 8610k - 1.5.0 0da6a335fe13 7c4ecad790fa 2024-03-21T10:46:49.806 INFO:teuthology.orchestra.run.smithi153.stdout:osd.0 smithi153 running (31m) 11m ago 46m 118M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 507900fd4791 2024-03-21T10:46:49.806 INFO:teuthology.orchestra.run.smithi153.stdout:osd.1 smithi153 running (30m) 11m ago 46m 93.0M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 0d646452e9c1 2024-03-21T10:46:49.806 INFO:teuthology.orchestra.run.smithi153.stdout:osd.2 smithi153 running (29m) 11m ago 45m 91.2M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 1326f2afdfcd 2024-03-21T10:46:49.806 INFO:teuthology.orchestra.run.smithi153.stdout:osd.3 smithi174 running (29m) 11m ago 45m 87.5M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 70aafac71ad5 2024-03-21T10:46:49.806 INFO:teuthology.orchestra.run.smithi153.stdout:osd.4 smithi174 running (28m) 11m ago 44m 95.8M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 0d746ce6e79d 2024-03-21T10:46:49.806 INFO:teuthology.orchestra.run.smithi153.stdout:osd.5 smithi174 running (28m) 11m ago 44m 90.3M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 53e5f35b8cac 2024-03-21T10:46:49.806 INFO:teuthology.orchestra.run.smithi153.stdout:prometheus.smithi153 smithi153 *:9095 running (36m) 11m ago 48m 88.1M - 2.43.0 a07b618ecd1d 7d95561e50c2 2024-03-21T10:46:49.806 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.811+0000 7f9f67c00700 1 -- 172.21.15.153:0/4207823772 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f9f4806e460 msgr2=0x7f9f48070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:46:49.806 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.811+0000 7f9f67c00700 1 --2- 172.21.15.153:0/4207823772 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f9f4806e460 0x7f9f48070920 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f9f5400a990 tx=0x7f9f54005c80 comp rx=0 tx=0).stop 2024-03-21T10:46:49.807 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.811+0000 7f9f67c00700 1 -- 172.21.15.153:0/4207823772 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9f601039b0 msgr2=0x7f9f6019ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:46:49.807 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.811+0000 7f9f67c00700 1 --2- 172.21.15.153:0/4207823772 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9f601039b0 0x7f9f6019ed00 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c00bda0 tx=0x7f9f4c00be80 comp rx=0 tx=0).stop 2024-03-21T10:46:49.807 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.811+0000 7f9f67c00700 1 -- 172.21.15.153:0/4207823772 shutdown_connections 2024-03-21T10:46:49.807 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.811+0000 7f9f67c00700 1 --2- 172.21.15.153:0/4207823772 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f60103000 0x7f9f6019e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:46:49.807 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.811+0000 7f9f67c00700 1 --2- 172.21.15.153:0/4207823772 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f9f4806e460 0x7f9f48070920 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:46:49.807 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.811+0000 7f9f67c00700 1 --2- 172.21.15.153:0/4207823772 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9f601039b0 0x7f9f6019ed00 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:46:49.808 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.811+0000 7f9f67c00700 1 -- 172.21.15.153:0/4207823772 >> 172.21.15.153:0/4207823772 conn(0x7f9f600fe8d0 msgr2=0x7f9f60100cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:46:49.808 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.811+0000 7f9f67c00700 1 -- 172.21.15.153:0/4207823772 shutdown_connections 2024-03-21T10:46:49.808 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.811+0000 7f9f67c00700 1 -- 172.21.15.153:0/4207823772 wait complete. 2024-03-21T10:46:49.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:46:49 smithi174 ceph-mon[88004]: pgmap v1050: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:46:49.959 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.962+0000 7f8d1dc00700 1 -- 172.21.15.153:0/3406500434 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d181039b0 msgr2=0x7f8d18107a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:46:49.959 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.962+0000 7f8d1dc00700 1 --2- 172.21.15.153:0/3406500434 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d181039b0 0x7f8d18107a00 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f8d00009910 tx=0x7f8d00009c20 comp rx=0 tx=0).stop 2024-03-21T10:46:49.959 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.962+0000 7f8d1dc00700 1 -- 172.21.15.153:0/3406500434 shutdown_connections 2024-03-21T10:46:49.959 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.962+0000 7f8d1dc00700 1 --2- 172.21.15.153:0/3406500434 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d181039b0 0x7f8d18107a00 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:46:49.959 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.962+0000 7f8d1dc00700 1 --2- 172.21.15.153:0/3406500434 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8d18103000 0x7f8d181033e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:46:49.959 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.963+0000 7f8d1dc00700 1 -- 172.21.15.153:0/3406500434 >> 172.21.15.153:0/3406500434 conn(0x7f8d180fe8d0 msgr2=0x7f8d18100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:46:49.959 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.963+0000 7f8d1dc00700 1 -- 172.21.15.153:0/3406500434 shutdown_connections 2024-03-21T10:46:49.959 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.963+0000 7f8d1dc00700 1 -- 172.21.15.153:0/3406500434 wait complete. 2024-03-21T10:46:49.960 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.964+0000 7f8d1dc00700 1 Processor -- start 2024-03-21T10:46:49.961 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.964+0000 7f8d1dc00700 1 -- start start 2024-03-21T10:46:49.961 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.964+0000 7f8d1dc00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d18103000 0x7f8d1819e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:46:49.961 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.964+0000 7f8d1dc00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8d181039b0 0x7f8d1819ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:46:49.961 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.964+0000 7f8d1dc00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d1819f390 con 0x7f8d18103000 2024-03-21T10:46:49.961 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.964+0000 7f8d1dc00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d18198840 con 0x7f8d181039b0 2024-03-21T10:46:49.961 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.965+0000 7f8d16a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d18103000 0x7f8d1819e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:46:49.961 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.965+0000 7f8d16a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d18103000 0x7f8d1819e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:43684/0 (socket says 172.21.15.153:43684) 2024-03-21T10:46:49.961 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.965+0000 7f8d16a00700 1 -- 172.21.15.153:0/3604405238 learned_addr learned my addr 172.21.15.153:0/3604405238 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:46:49.962 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.965+0000 7f8d16a00700 1 -- 172.21.15.153:0/3604405238 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8d181039b0 msgr2=0x7f8d1819ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:46:49.962 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.965+0000 7f8d16a00700 1 --2- 172.21.15.153:0/3604405238 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8d181039b0 0x7f8d1819ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:46:49.962 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.965+0000 7f8d16a00700 1 -- 172.21.15.153:0/3604405238 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d000095f0 con 0x7f8d18103000 2024-03-21T10:46:49.962 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.965+0000 7f8d16a00700 1 --2- 172.21.15.153:0/3604405238 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d18103000 0x7f8d1819e7c0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f8d0800d8d0 tx=0x7f8d0800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:46:49.962 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.966+0000 7f8d0f400700 1 -- 172.21.15.153:0/3604405238 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d08009940 con 0x7f8d18103000 2024-03-21T10:46:49.963 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.966+0000 7f8d1dc00700 1 -- 172.21.15.153:0/3604405238 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d18198b20 con 0x7f8d18103000 2024-03-21T10:46:49.963 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.966+0000 7f8d0f400700 1 -- 172.21.15.153:0/3604405238 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8d08010460 con 0x7f8d18103000 2024-03-21T10:46:49.963 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.966+0000 7f8d0f400700 1 -- 172.21.15.153:0/3604405238 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d0800f5f0 con 0x7f8d18103000 2024-03-21T10:46:49.963 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.966+0000 7f8d1dc00700 1 -- 172.21.15.153:0/3604405238 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d18199070 con 0x7f8d18103000 2024-03-21T10:46:49.963 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.967+0000 7f8d1dc00700 1 -- 172.21.15.153:0/3604405238 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d1810b400 con 0x7f8d18103000 2024-03-21T10:46:49.964 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.968+0000 7f8d0f400700 1 -- 172.21.15.153:0/3604405238 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f8d080105d0 con 0x7f8d18103000 2024-03-21T10:46:49.965 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.968+0000 7f8d0f400700 1 --2- 172.21.15.153:0/3604405238 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f8cfc06e460 0x7f8cfc070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:46:49.965 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.969+0000 7f8d0f400700 1 -- 172.21.15.153:0/3604405238 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f8d0808e5b0 con 0x7f8d18103000 2024-03-21T10:46:49.965 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.969+0000 7f8d0fe00700 1 --2- 172.21.15.153:0/3604405238 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f8cfc06e460 0x7f8cfc070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:46:49.965 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.969+0000 7f8d0fe00700 1 --2- 172.21.15.153:0/3604405238 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f8cfc06e460 0x7f8cfc070920 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f8d00006010 tx=0x7f8d0000b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:46:49.970 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:49.974+0000 7f8d0f400700 1 -- 172.21.15.153:0/3604405238 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f8d0805b100 con 0x7f8d18103000 2024-03-21T10:46:50.283 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:50.287+0000 7f8d1dc00700 1 -- 172.21.15.153:0/3604405238 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f8d18199290 con 0x7f8d18103000 2024-03-21T10:46:50.284 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:50.288+0000 7f8d0f400700 1 -- 172.21.15.153:0/3604405238 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f8d0805ac90 con 0x7f8d18103000 2024-03-21T10:46:50.284 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:46:50.284 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": { 2024-03-21T10:46:50.284 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:46:50.284 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:46:50.284 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": { 2024-03-21T10:46:50.284 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:46:50.284 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:46:50.285 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": { 2024-03-21T10:46:50.285 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 6 2024-03-21T10:46:50.285 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:46:50.285 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": { 2024-03-21T10:46:50.285 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 4 2024-03-21T10:46:50.285 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:46:50.285 INFO:teuthology.orchestra.run.smithi153.stdout: "overall": { 2024-03-21T10:46:50.285 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 14 2024-03-21T10:46:50.285 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-03-21T10:46:50.285 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:46:50.287 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:50.291+0000 7f8d1dc00700 1 -- 172.21.15.153:0/3604405238 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f8cfc06e460 msgr2=0x7f8cfc070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:46:50.287 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:50.291+0000 7f8d1dc00700 1 --2- 172.21.15.153:0/3604405238 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f8cfc06e460 0x7f8cfc070920 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f8d00006010 tx=0x7f8d0000b540 comp rx=0 tx=0).stop 2024-03-21T10:46:50.287 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:50.291+0000 7f8d1dc00700 1 -- 172.21.15.153:0/3604405238 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d18103000 msgr2=0x7f8d1819e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:46:50.287 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:50.291+0000 7f8d1dc00700 1 --2- 172.21.15.153:0/3604405238 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d18103000 0x7f8d1819e7c0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f8d0800d8d0 tx=0x7f8d0800dc90 comp rx=0 tx=0).stop 2024-03-21T10:46:50.287 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:50.291+0000 7f8d1dc00700 1 -- 172.21.15.153:0/3604405238 shutdown_connections 2024-03-21T10:46:50.287 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:50.291+0000 7f8d1dc00700 1 --2- 172.21.15.153:0/3604405238 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f8cfc06e460 0x7f8cfc070920 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:46:50.287 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:50.291+0000 7f8d1dc00700 1 --2- 172.21.15.153:0/3604405238 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8d18103000 0x7f8d1819e7c0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:46:50.288 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:50.291+0000 7f8d1dc00700 1 --2- 172.21.15.153:0/3604405238 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8d181039b0 0x7f8d1819ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:46:50.288 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:50.291+0000 7f8d1dc00700 1 -- 172.21.15.153:0/3604405238 >> 172.21.15.153:0/3604405238 conn(0x7f8d180fe8d0 msgr2=0x7f8d18100cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:46:50.288 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:50.291+0000 7f8d1dc00700 1 -- 172.21.15.153:0/3604405238 shutdown_connections 2024-03-21T10:46:50.288 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:50.291+0000 7f8d1dc00700 1 -- 172.21.15.153:0/3604405238 wait complete. 2024-03-21T10:46:50.441 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:50.445+0000 7ff61e800700 1 -- 172.21.15.153:0/2687925971 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff618073160 msgr2=0x7ff618073540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:46:50.442 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:50.445+0000 7ff61e800700 1 --2- 172.21.15.153:0/2687925971 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff618073160 0x7ff618073540 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7ff608009910 tx=0x7ff608009c20 comp rx=0 tx=0).stop 2024-03-21T10:46:50.442 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:50.445+0000 7ff61e800700 1 -- 172.21.15.153:0/2687925971 shutdown_connections 2024-03-21T10:46:50.442 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:50.445+0000 7ff61e800700 1 --2- 172.21.15.153:0/2687925971 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff618073a80 0x7ff61810cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:46:50.442 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:50.445+0000 7ff61e800700 1 --2- 172.21.15.153:0/2687925971 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff618073160 0x7ff618073540 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:46:50.442 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:50.445+0000 7ff61e800700 1 -- 172.21.15.153:0/2687925971 >> 172.21.15.153:0/2687925971 conn(0x7ff6180fc630 msgr2=0x7ff6180fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:46:50.442 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:50.445+0000 7ff61e800700 1 -- 172.21.15.153:0/2687925971 shutdown_connections 2024-03-21T10:46:50.442 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:50.445+0000 7ff61e800700 1 -- 172.21.15.153:0/2687925971 wait complete. 2024-03-21T10:46:50.442 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:50.446+0000 7ff61e800700 1 Processor -- start 2024-03-21T10:46:50.443 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:50.447+0000 7ff61e800700 1 -- start start 2024-03-21T10:46:50.443 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:50.447+0000 7ff61e800700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff618073160 0x7ff61819e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:46:50.443 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:50.447+0000 7ff61e800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff618073a80 0x7ff61819ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:46:50.443 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:50.447+0000 7ff61e800700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff61819f400 con 0x7ff618073a80 2024-03-21T10:46:50.443 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:50.447+0000 7ff61e800700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6181988b0 con 0x7ff618073160 2024-03-21T10:46:50.444 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:50.447+0000 7ff60fe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff618073a80 0x7ff61819ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:46:50.444 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:50.447+0000 7ff60fe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff618073a80 0x7ff61819ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:43704/0 (socket says 172.21.15.153:43704) 2024-03-21T10:46:50.444 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:50.447+0000 7ff60fe00700 1 -- 172.21.15.153:0/3957043347 learned_addr learned my addr 172.21.15.153:0/3957043347 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:46:50.444 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:50.448+0000 7ff617400700 1 --2- 172.21.15.153:0/3957043347 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff618073160 0x7ff61819e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:46:50.445 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:50.448+0000 7ff60fe00700 1 -- 172.21.15.153:0/3957043347 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff618073160 msgr2=0x7ff61819e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:46:50.445 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:50.448+0000 7ff60fe00700 1 --2- 172.21.15.153:0/3957043347 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff618073160 0x7ff61819e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:46:50.445 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:50.448+0000 7ff60fe00700 1 -- 172.21.15.153:0/3957043347 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6080095f0 con 0x7ff618073a80 2024-03-21T10:46:50.445 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:50.448+0000 7ff60fe00700 1 --2- 172.21.15.153:0/3957043347 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff618073a80 0x7ff61819ed70 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7ff60000b4b0 tx=0x7ff60000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:46:50.445 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:50.448+0000 7ff614c00700 1 -- 172.21.15.153:0/3957043347 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff600011800 con 0x7ff618073a80 2024-03-21T10:46:50.445 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:50.449+0000 7ff614c00700 1 -- 172.21.15.153:0/3957043347 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff600011e20 con 0x7ff618073a80 2024-03-21T10:46:50.445 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:50.449+0000 7ff61e800700 1 -- 172.21.15.153:0/3957043347 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff618198b90 con 0x7ff618073a80 2024-03-21T10:46:50.445 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:50.449+0000 7ff614c00700 1 -- 172.21.15.153:0/3957043347 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff60000f2e0 con 0x7ff618073a80 2024-03-21T10:46:50.445 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:50.449+0000 7ff61e800700 1 -- 172.21.15.153:0/3957043347 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6181990e0 con 0x7ff618073a80 2024-03-21T10:46:50.446 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:50.450+0000 7ff61e800700 1 -- 172.21.15.153:0/3957043347 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff61810a5a0 con 0x7ff618073a80 2024-03-21T10:46:50.447 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:50.451+0000 7ff614c00700 1 -- 172.21.15.153:0/3957043347 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7ff600010380 con 0x7ff618073a80 2024-03-21T10:46:50.448 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:50.452+0000 7ff614c00700 1 --2- 172.21.15.153:0/3957043347 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff5fc06e460 0x7ff5fc070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:46:50.448 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:50.452+0000 7ff614c00700 1 -- 172.21.15.153:0/3957043347 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7ff60008e020 con 0x7ff618073a80 2024-03-21T10:46:50.448 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:50.452+0000 7ff617400700 1 --2- 172.21.15.153:0/3957043347 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff5fc06e460 0x7ff5fc070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:46:50.449 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:50.453+0000 7ff617400700 1 --2- 172.21.15.153:0/3957043347 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff5fc06e460 0x7ff5fc070920 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7ff60800b5c0 tx=0x7ff60801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:46:50.452 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:50.456+0000 7ff614c00700 1 -- 172.21.15.153:0/3957043347 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff60005ad40 con 0x7ff618073a80 2024-03-21T10:46:50.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:46:50 smithi153 ceph-mon[113733]: from='client.35854 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:46:50.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:46:50 smithi153 ceph-mon[113733]: from='client.35858 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:46:50.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:46:50 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/3604405238' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:46:50.724 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:50.728+0000 7ff61e800700 1 -- 172.21.15.153:0/3957043347 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7ff61804ea90 con 0x7ff618073a80 2024-03-21T10:46:50.725 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:50.729+0000 7ff614c00700 1 -- 172.21.15.153:0/3957043347 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1876 (secure 0 0 0) 0x7ff60005a8d0 con 0x7ff618073a80 2024-03-21T10:46:50.725 INFO:teuthology.orchestra.run.smithi153.stdout:e32 2024-03-21T10:46:50.725 INFO:teuthology.orchestra.run.smithi153.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-21T10:46:50.725 INFO:teuthology.orchestra.run.smithi153.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:46:50.725 INFO:teuthology.orchestra.run.smithi153.stdout:legacy client fscid: 1 2024-03-21T10:46:50.726 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:46:50.726 INFO:teuthology.orchestra.run.smithi153.stdout:Filesystem 'cephfs' (1) 2024-03-21T10:46:50.726 INFO:teuthology.orchestra.run.smithi153.stdout:fs_name cephfs 2024-03-21T10:46:50.726 INFO:teuthology.orchestra.run.smithi153.stdout:epoch 23 2024-03-21T10:46:50.726 INFO:teuthology.orchestra.run.smithi153.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-21T10:46:50.726 INFO:teuthology.orchestra.run.smithi153.stdout:created 2024-03-21T10:04:27.086576+0000 2024-03-21T10:46:50.726 INFO:teuthology.orchestra.run.smithi153.stdout:modified 2024-03-21T10:19:05.022092+0000 2024-03-21T10:46:50.726 INFO:teuthology.orchestra.run.smithi153.stdout:tableserver 0 2024-03-21T10:46:50.726 INFO:teuthology.orchestra.run.smithi153.stdout:root 0 2024-03-21T10:46:50.726 INFO:teuthology.orchestra.run.smithi153.stdout:session_timeout 60 2024-03-21T10:46:50.726 INFO:teuthology.orchestra.run.smithi153.stdout:session_autoclose 300 2024-03-21T10:46:50.726 INFO:teuthology.orchestra.run.smithi153.stdout:max_file_size 1099511627776 2024-03-21T10:46:50.726 INFO:teuthology.orchestra.run.smithi153.stdout:max_xattr_size 65536 2024-03-21T10:46:50.726 INFO:teuthology.orchestra.run.smithi153.stdout:required_client_features {} 2024-03-21T10:46:50.727 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure 0 2024-03-21T10:46:50.727 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure_osd_epoch 94 2024-03-21T10:46:50.727 INFO:teuthology.orchestra.run.smithi153.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:46:50.727 INFO:teuthology.orchestra.run.smithi153.stdout:max_mds 2 2024-03-21T10:46:50.727 INFO:teuthology.orchestra.run.smithi153.stdout:in 0,1 2024-03-21T10:46:50.727 INFO:teuthology.orchestra.run.smithi153.stdout:up {} 2024-03-21T10:46:50.727 INFO:teuthology.orchestra.run.smithi153.stdout:failed 0,1 2024-03-21T10:46:50.727 INFO:teuthology.orchestra.run.smithi153.stdout:damaged 2024-03-21T10:46:50.727 INFO:teuthology.orchestra.run.smithi153.stdout:stopped 2024-03-21T10:46:50.727 INFO:teuthology.orchestra.run.smithi153.stdout:data_pools [3] 2024-03-21T10:46:50.727 INFO:teuthology.orchestra.run.smithi153.stdout:metadata_pool 2 2024-03-21T10:46:50.727 INFO:teuthology.orchestra.run.smithi153.stdout:inline_data enabled 2024-03-21T10:46:50.727 INFO:teuthology.orchestra.run.smithi153.stdout:balancer 2024-03-21T10:46:50.727 INFO:teuthology.orchestra.run.smithi153.stdout:bal_rank_mask -1 2024-03-21T10:46:50.727 INFO:teuthology.orchestra.run.smithi153.stdout:standby_count_wanted 1 2024-03-21T10:46:50.727 INFO:teuthology.orchestra.run.smithi153.stdout:qdb_cluster leader: 0 members: 2024-03-21T10:46:50.728 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:46:50.728 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:46:50.728 INFO:teuthology.orchestra.run.smithi153.stdout:Standby daemons: 2024-03-21T10:46:50.728 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:46:50.728 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.zipckr{-1:34442} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6826/1103846295,v1:172.21.15.153:6827/1103846295] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:46:50.728 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.iuyluj{-1:34474} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6828/1738630832,v1:172.21.15.153:6829/1738630832] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:46:50.728 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.yyimst{-1:44313} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.174:6824/4127786797,v1:172.21.15.174:6825/4127786797] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:46:50.728 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.nhozag{-1:44687} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.174:6826/2767517845,v1:172.21.15.174:6827/2767517845] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:46:50.729 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:50.732+0000 7ff61e800700 1 -- 172.21.15.153:0/3957043347 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff5fc06e460 msgr2=0x7ff5fc070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:46:50.729 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:50.732+0000 7ff61e800700 1 --2- 172.21.15.153:0/3957043347 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff5fc06e460 0x7ff5fc070920 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7ff60800b5c0 tx=0x7ff60801a040 comp rx=0 tx=0).stop 2024-03-21T10:46:50.729 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:50.732+0000 7ff61e800700 1 -- 172.21.15.153:0/3957043347 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff618073a80 msgr2=0x7ff61819ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:46:50.729 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:50.732+0000 7ff61e800700 1 --2- 172.21.15.153:0/3957043347 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff618073a80 0x7ff61819ed70 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7ff60000b4b0 tx=0x7ff60000b870 comp rx=0 tx=0).stop 2024-03-21T10:46:50.729 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:50.732+0000 7ff61e800700 1 -- 172.21.15.153:0/3957043347 shutdown_connections 2024-03-21T10:46:50.729 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:50.732+0000 7ff61e800700 1 --2- 172.21.15.153:0/3957043347 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff618073160 0x7ff61819e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:46:50.729 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:50.732+0000 7ff61e800700 1 --2- 172.21.15.153:0/3957043347 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff5fc06e460 0x7ff5fc070920 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:46:50.729 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:50.732+0000 7ff61e800700 1 --2- 172.21.15.153:0/3957043347 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff618073a80 0x7ff61819ed70 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:46:50.729 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:50.732+0000 7ff61e800700 1 -- 172.21.15.153:0/3957043347 >> 172.21.15.153:0/3957043347 conn(0x7ff6180fc630 msgr2=0x7ff6181076e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:46:50.729 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:50.732+0000 7ff61e800700 1 -- 172.21.15.153:0/3957043347 shutdown_connections 2024-03-21T10:46:50.729 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:50.733+0000 7ff61e800700 1 -- 172.21.15.153:0/3957043347 wait complete. 2024-03-21T10:46:50.730 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 32 2024-03-21T10:46:50.884 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:50.888+0000 7f8758a00700 1 -- 172.21.15.153:0/1667453933 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f87541039b0 msgr2=0x7f8754107a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:46:50.885 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:50.888+0000 7f8758a00700 1 --2- 172.21.15.153:0/1667453933 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f87541039b0 0x7f8754107a00 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f8740009910 tx=0x7f8740009c20 comp rx=0 tx=0).stop 2024-03-21T10:46:50.885 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:50.888+0000 7f8758a00700 1 -- 172.21.15.153:0/1667453933 shutdown_connections 2024-03-21T10:46:50.885 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:50.888+0000 7f8758a00700 1 --2- 172.21.15.153:0/1667453933 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f87541039b0 0x7f8754107a00 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:46:50.885 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:50.888+0000 7f8758a00700 1 --2- 172.21.15.153:0/1667453933 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8754103000 0x7f87541033e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:46:50.885 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:50.888+0000 7f8758a00700 1 -- 172.21.15.153:0/1667453933 >> 172.21.15.153:0/1667453933 conn(0x7f87540fe8d0 msgr2=0x7f8754100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:46:50.885 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:50.888+0000 7f8758a00700 1 -- 172.21.15.153:0/1667453933 shutdown_connections 2024-03-21T10:46:50.885 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:50.888+0000 7f8758a00700 1 -- 172.21.15.153:0/1667453933 wait complete. 2024-03-21T10:46:50.885 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:50.890+0000 7f8758a00700 1 Processor -- start 2024-03-21T10:46:50.886 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:50.890+0000 7f8758a00700 1 -- start start 2024-03-21T10:46:50.886 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:50.890+0000 7f8758a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8754103000 0x7f875419e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:46:50.886 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:50.890+0000 7f8758a00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f87541039b0 0x7f875419ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:46:50.886 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:50.890+0000 7f8758a00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f875419f390 con 0x7f8754103000 2024-03-21T10:46:50.886 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:50.890+0000 7f8758a00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8754198840 con 0x7f87541039b0 2024-03-21T10:46:50.887 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:50.890+0000 7f8751e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8754103000 0x7f875419e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:46:50.887 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:50.891+0000 7f8751e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8754103000 0x7f875419e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:43712/0 (socket says 172.21.15.153:43712) 2024-03-21T10:46:50.887 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:50.891+0000 7f8751e00700 1 -- 172.21.15.153:0/1575795862 learned_addr learned my addr 172.21.15.153:0/1575795862 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:46:50.887 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:50.891+0000 7f8751400700 1 --2- 172.21.15.153:0/1575795862 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f87541039b0 0x7f875419ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:46:50.888 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:50.891+0000 7f8751e00700 1 -- 172.21.15.153:0/1575795862 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f87541039b0 msgr2=0x7f875419ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:46:50.888 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:50.891+0000 7f8751e00700 1 --2- 172.21.15.153:0/1575795862 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f87541039b0 0x7f875419ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:46:50.888 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:50.891+0000 7f8751e00700 1 -- 172.21.15.153:0/1575795862 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87400095f0 con 0x7f8754103000 2024-03-21T10:46:50.888 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:50.891+0000 7f8751400700 1 --2- 172.21.15.153:0/1575795862 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f87541039b0 0x7f875419ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:46:50.888 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:50.891+0000 7f8751e00700 1 --2- 172.21.15.153:0/1575795862 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8754103000 0x7f875419e7c0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f874400c8a0 tx=0x7f874400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:46:50.888 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:50.892+0000 7f874aa00700 1 -- 172.21.15.153:0/1575795862 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f874400cea0 con 0x7f8754103000 2024-03-21T10:46:50.888 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:50.892+0000 7f874aa00700 1 -- 172.21.15.153:0/1575795862 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8744004830 con 0x7f8754103000 2024-03-21T10:46:50.889 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:50.892+0000 7f8758a00700 1 -- 172.21.15.153:0/1575795862 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8754198b20 con 0x7f8754103000 2024-03-21T10:46:50.889 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:50.892+0000 7f8758a00700 1 -- 172.21.15.153:0/1575795862 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8754199070 con 0x7f8754103000 2024-03-21T10:46:50.889 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:50.892+0000 7f874aa00700 1 -- 172.21.15.153:0/1575795862 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f87440055b0 con 0x7f8754103000 2024-03-21T10:46:50.889 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:50.893+0000 7f8758a00700 1 -- 172.21.15.153:0/1575795862 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f875410b400 con 0x7f8754103000 2024-03-21T10:46:50.890 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:50.894+0000 7f874aa00700 1 -- 172.21.15.153:0/1575795862 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f87440049a0 con 0x7f8754103000 2024-03-21T10:46:50.892 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:50.896+0000 7f874aa00700 1 --2- 172.21.15.153:0/1575795862 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f873806e460 0x7f8738070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:46:50.892 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:50.896+0000 7f874aa00700 1 -- 172.21.15.153:0/1575795862 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f874408e270 con 0x7f8754103000 2024-03-21T10:46:50.892 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:50.896+0000 7f8751400700 1 --2- 172.21.15.153:0/1575795862 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f873806e460 0x7f8738070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:46:50.892 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:50.897+0000 7f8751400700 1 --2- 172.21.15.153:0/1575795862 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f873806e460 0x7f8738070920 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f8740006010 tx=0x7f874000b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:46:50.896 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:50.900+0000 7f874aa00700 1 -- 172.21.15.153:0/1575795862 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f874405adc0 con 0x7f8754103000 2024-03-21T10:46:50.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:46:50 smithi174 ceph-mon[88004]: from='client.35854 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:46:50.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:46:50 smithi174 ceph-mon[88004]: from='client.35858 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:46:50.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:46:50 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/3604405238' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:46:51.143 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:51.147+0000 7f8758a00700 1 -- 172.21.15.153:0/1575795862 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8754199340 con 0x7f873806e460 2024-03-21T10:46:51.145 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:51.149+0000 7f874aa00700 1 -- 172.21.15.153:0/1575795862 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f8754199340 con 0x7f873806e460 2024-03-21T10:46:51.145 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:46:51.145 INFO:teuthology.orchestra.run.smithi153.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232", 2024-03-21T10:46:51.146 INFO:teuthology.orchestra.run.smithi153.stdout: "in_progress": true, 2024-03-21T10:46:51.146 INFO:teuthology.orchestra.run.smithi153.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-21T10:46:51.146 INFO:teuthology.orchestra.run.smithi153.stdout: "services_complete": [ 2024-03-21T10:46:51.146 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr", 2024-03-21T10:46:51.146 INFO:teuthology.orchestra.run.smithi153.stdout: "mon", 2024-03-21T10:46:51.146 INFO:teuthology.orchestra.run.smithi153.stdout: "mds", 2024-03-21T10:46:51.146 INFO:teuthology.orchestra.run.smithi153.stdout: "crash", 2024-03-21T10:46:51.146 INFO:teuthology.orchestra.run.smithi153.stdout: "osd" 2024-03-21T10:46:51.146 INFO:teuthology.orchestra.run.smithi153.stdout: ], 2024-03-21T10:46:51.146 INFO:teuthology.orchestra.run.smithi153.stdout: "progress": "16/23 daemons upgraded", 2024-03-21T10:46:51.146 INFO:teuthology.orchestra.run.smithi153.stdout: "message": "Currently upgrading mds daemons", 2024-03-21T10:46:51.146 INFO:teuthology.orchestra.run.smithi153.stdout: "is_paused": false 2024-03-21T10:46:51.146 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:46:51.148 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:51.151+0000 7f8758a00700 1 -- 172.21.15.153:0/1575795862 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f873806e460 msgr2=0x7f8738070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:46:51.148 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:51.151+0000 7f8758a00700 1 --2- 172.21.15.153:0/1575795862 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f873806e460 0x7f8738070920 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f8740006010 tx=0x7f874000b540 comp rx=0 tx=0).stop 2024-03-21T10:46:51.148 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:51.151+0000 7f8758a00700 1 -- 172.21.15.153:0/1575795862 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8754103000 msgr2=0x7f875419e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:46:51.148 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:51.151+0000 7f8758a00700 1 --2- 172.21.15.153:0/1575795862 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8754103000 0x7f875419e7c0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f874400c8a0 tx=0x7f874400cc60 comp rx=0 tx=0).stop 2024-03-21T10:46:51.148 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:51.152+0000 7f8758a00700 1 -- 172.21.15.153:0/1575795862 shutdown_connections 2024-03-21T10:46:51.148 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:51.152+0000 7f8758a00700 1 --2- 172.21.15.153:0/1575795862 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f873806e460 0x7f8738070920 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:46:51.148 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:51.152+0000 7f8758a00700 1 --2- 172.21.15.153:0/1575795862 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8754103000 0x7f875419e7c0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:46:51.149 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:51.152+0000 7f8758a00700 1 --2- 172.21.15.153:0/1575795862 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f87541039b0 0x7f875419ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:46:51.149 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:51.152+0000 7f8758a00700 1 -- 172.21.15.153:0/1575795862 >> 172.21.15.153:0/1575795862 conn(0x7f87540fe8d0 msgr2=0x7f8754100cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:46:51.149 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:51.152+0000 7f8758a00700 1 -- 172.21.15.153:0/1575795862 shutdown_connections 2024-03-21T10:46:51.149 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:51.152+0000 7f8758a00700 1 -- 172.21.15.153:0/1575795862 wait complete. 2024-03-21T10:46:51.298 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:51.301+0000 7fde0dc00700 1 -- 172.21.15.153:0/1646933582 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde0810d280 msgr2=0x7fde0810d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:46:51.298 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:51.301+0000 7fde0dc00700 1 --2- 172.21.15.153:0/1646933582 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde0810d280 0x7fde0810d660 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fddf8009910 tx=0x7fddf8009c20 comp rx=0 tx=0).stop 2024-03-21T10:46:51.298 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:51.301+0000 7fde0dc00700 1 -- 172.21.15.153:0/1646933582 shutdown_connections 2024-03-21T10:46:51.298 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:51.301+0000 7fde0dc00700 1 --2- 172.21.15.153:0/1646933582 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fde081071d0 0x7fde08107650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:46:51.298 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:51.301+0000 7fde0dc00700 1 --2- 172.21.15.153:0/1646933582 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde0810d280 0x7fde0810d660 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:46:51.298 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:51.301+0000 7fde0dc00700 1 -- 172.21.15.153:0/1646933582 >> 172.21.15.153:0/1646933582 conn(0x7fde080769e0 msgr2=0x7fde08078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:46:51.298 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:51.302+0000 7fde0dc00700 1 -- 172.21.15.153:0/1646933582 shutdown_connections 2024-03-21T10:46:51.299 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:51.302+0000 7fde0dc00700 1 -- 172.21.15.153:0/1646933582 wait complete. 2024-03-21T10:46:51.300 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:51.302+0000 7fde0dc00700 1 Processor -- start 2024-03-21T10:46:51.300 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:51.303+0000 7fde0dc00700 1 -- start start 2024-03-21T10:46:51.300 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:51.303+0000 7fde0dc00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde081071d0 0x7fde0819ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:46:51.300 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:51.303+0000 7fde0dc00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fde0810d280 0x7fde0819d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:46:51.300 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:51.303+0000 7fde0dc00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde0819d8a0 con 0x7fde081071d0 2024-03-21T10:46:51.301 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:51.303+0000 7fde0dc00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde081a05d0 con 0x7fde0810d280 2024-03-21T10:46:51.301 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:51.303+0000 7fde06a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde081071d0 0x7fde0819ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:46:51.301 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:51.303+0000 7fde06a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde081071d0 0x7fde0819ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:43732/0 (socket says 172.21.15.153:43732) 2024-03-21T10:46:51.301 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:51.303+0000 7fde06a00700 1 -- 172.21.15.153:0/1179551848 learned_addr learned my addr 172.21.15.153:0/1179551848 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:46:51.301 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:51.304+0000 7fde06a00700 1 -- 172.21.15.153:0/1179551848 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fde0810d280 msgr2=0x7fde0819d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:46:51.301 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:51.304+0000 7fde06a00700 1 --2- 172.21.15.153:0/1179551848 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fde0810d280 0x7fde0819d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:46:51.301 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:51.304+0000 7fde06a00700 1 -- 172.21.15.153:0/1179551848 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddf80095f0 con 0x7fde081071d0 2024-03-21T10:46:51.301 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:51.304+0000 7fde06a00700 1 --2- 172.21.15.153:0/1179551848 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde081071d0 0x7fde0819ccd0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fddf8006010 tx=0x7fddf800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:46:51.301 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:51.304+0000 7fddff400700 1 -- 172.21.15.153:0/1179551848 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fddf801d070 con 0x7fde081071d0 2024-03-21T10:46:51.301 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:51.304+0000 7fddff400700 1 -- 172.21.15.153:0/1179551848 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fddf8004fb0 con 0x7fde081071d0 2024-03-21T10:46:51.301 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:51.304+0000 7fde0dc00700 1 -- 172.21.15.153:0/1179551848 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde081a0850 con 0x7fde081071d0 2024-03-21T10:46:51.301 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:51.304+0000 7fddff400700 1 -- 172.21.15.153:0/1179551848 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fddf8022ea0 con 0x7fde081071d0 2024-03-21T10:46:51.301 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:51.304+0000 7fde0dc00700 1 -- 172.21.15.153:0/1179551848 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde081a0dc0 con 0x7fde081071d0 2024-03-21T10:46:51.302 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:51.306+0000 7fde0dc00700 1 -- 172.21.15.153:0/1179551848 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde0804ea90 con 0x7fde081071d0 2024-03-21T10:46:51.302 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:51.306+0000 7fddff400700 1 -- 172.21.15.153:0/1179551848 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fddf8022470 con 0x7fde081071d0 2024-03-21T10:46:51.303 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:51.307+0000 7fddff400700 1 --2- 172.21.15.153:0/1179551848 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fddec06e460 0x7fddec070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:46:51.303 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:51.307+0000 7fddff400700 1 -- 172.21.15.153:0/1179551848 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fddf808f990 con 0x7fde081071d0 2024-03-21T10:46:51.303 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:51.308+0000 7fddffe00700 1 --2- 172.21.15.153:0/1179551848 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fddec06e460 0x7fddec070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:46:51.304 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:51.308+0000 7fddffe00700 1 --2- 172.21.15.153:0/1179551848 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fddec06e460 0x7fddec070920 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fde0819e2b0 tx=0x7fddf0005c80 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:46:51.308 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:51.312+0000 7fddff400700 1 -- 172.21.15.153:0/1179551848 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fddf8058b80 con 0x7fde081071d0 2024-03-21T10:46:51.602 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:51.605+0000 7fde0dc00700 1 -- 172.21.15.153:0/1179551848 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fde081a1170 con 0x7fde081071d0 2024-03-21T10:46:51.602 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:51.606+0000 7fddff400700 1 -- 172.21.15.153:0/1179551848 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4493 (secure 0 0 0) 0x7fddf8022720 con 0x7fde081071d0 2024-03-21T10:46:51.603 INFO:teuthology.orchestra.run.smithi153.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-03-21T10:46:51.603 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-03-21T10:46:51.603 INFO:teuthology.orchestra.run.smithi153.stdout: host smithi174 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi174/config 2024-03-21T10:46:51.603 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/podman: timeout after 895 seconds 2024-03-21T10:46:51.603 INFO:teuthology.orchestra.run.smithi153.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 -e NODE_NAME=smithi174 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/run/ceph:z -v /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpd910r3_i:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 inventory --format=json-pretty --filter-for-batch 2024-03-21T10:46:51.603 INFO:teuthology.orchestra.run.smithi153.stdout:Traceback (most recent call last): 2024-03-21T10:46:51.603 INFO:teuthology.orchestra.run.smithi153.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-03-21T10:46:51.603 INFO:teuthology.orchestra.run.smithi153.stdout: return _run_code(code, main_globals, None, 2024-03-21T10:46:51.603 INFO:teuthology.orchestra.run.smithi153.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-03-21T10:46:51.603 INFO:teuthology.orchestra.run.smithi153.stdout: exec(code, run_globals) 2024-03-21T10:46:51.603 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 5606, in 2024-03-21T10:46:51.603 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 5594, in main 2024-03-21T10:46:51.604 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 406, in _infer_config 2024-03-21T10:46:51.604 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 321, in _infer_fsid 2024-03-21T10:46:51.604 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 434, in _infer_image 2024-03-21T10:46:51.604 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 308, in _validate_fsid 2024-03-21T10:46:51.604 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 3357, in command_ceph_volume 2024-03-21T10:46:51.604 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-03-21T10:46:51.604 INFO:teuthology.orchestra.run.smithi153.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 -e NODE_NAME=smithi174 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/run/ceph:z -v /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpd910r3_i:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 inventory --format=json-pretty --filter-for-batch 2024-03-21T10:46:51.604 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-21T10:46:51.604 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is degraded 2024-03-21T10:46:51.604 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-21T10:46:51.604 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-21T10:46:51.604 INFO:teuthology.orchestra.run.smithi153.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-21T10:46:51.604 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-21T10:46:51.605 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:51.609+0000 7fde0dc00700 1 -- 172.21.15.153:0/1179551848 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fddec06e460 msgr2=0x7fddec070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:46:51.606 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:51.609+0000 7fde0dc00700 1 --2- 172.21.15.153:0/1179551848 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fddec06e460 0x7fddec070920 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fde0819e2b0 tx=0x7fddf0005c80 comp rx=0 tx=0).stop 2024-03-21T10:46:51.606 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:51.609+0000 7fde0dc00700 1 -- 172.21.15.153:0/1179551848 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde081071d0 msgr2=0x7fde0819ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:46:51.606 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:51.609+0000 7fde0dc00700 1 --2- 172.21.15.153:0/1179551848 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde081071d0 0x7fde0819ccd0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fddf8006010 tx=0x7fddf800be20 comp rx=0 tx=0).stop 2024-03-21T10:46:51.606 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:51.609+0000 7fde0dc00700 1 -- 172.21.15.153:0/1179551848 shutdown_connections 2024-03-21T10:46:51.606 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:51.609+0000 7fde0dc00700 1 --2- 172.21.15.153:0/1179551848 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fddec06e460 0x7fddec070920 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:46:51.606 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:51.609+0000 7fde0dc00700 1 --2- 172.21.15.153:0/1179551848 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fde081071d0 0x7fde0819ccd0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:46:51.606 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:51.609+0000 7fde0dc00700 1 --2- 172.21.15.153:0/1179551848 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fde0810d280 0x7fde0819d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:46:51.606 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:51.609+0000 7fde0dc00700 1 -- 172.21.15.153:0/1179551848 >> 172.21.15.153:0/1179551848 conn(0x7fde080769e0 msgr2=0x7fde08078320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:46:51.606 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:51.610+0000 7fde0dc00700 1 -- 172.21.15.153:0/1179551848 shutdown_connections 2024-03-21T10:46:51.606 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:46:51.610+0000 7fde0dc00700 1 -- 172.21.15.153:0/1179551848 wait complete. 2024-03-21T10:46:51.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:46:51 smithi153 ceph-mon[113733]: from='client.35862 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:46:51.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:46:51 smithi153 ceph-mon[113733]: pgmap v1051: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:46:51.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:46:51 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/3957043347' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:46:51.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:46:51 smithi174 ceph-mon[88004]: from='client.35862 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:46:51.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:46:51 smithi174 ceph-mon[88004]: pgmap v1051: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:46:51.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:46:51 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/3957043347' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:46:52.645 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:46:52 smithi153 ceph-mon[113733]: from='client.35874 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:46:52.645 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:46:52 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/1179551848' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:46:52.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:46:52 smithi174 ceph-mon[88004]: from='client.35874 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:46:52.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:46:52 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/1179551848' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:46:53.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:46:53 smithi153 ceph-mon[113733]: pgmap v1052: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:46:53.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:46:53 smithi174 ceph-mon[88004]: pgmap v1052: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:46:55.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:46:55 smithi153 ceph-mon[113733]: pgmap v1053: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:46:55.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:46:55 smithi174 ceph-mon[88004]: pgmap v1053: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:46:57.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:46:57 smithi153 ceph-mon[113733]: pgmap v1054: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:46:57.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:46:57 smithi174 ceph-mon[88004]: pgmap v1054: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:46:59.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:46:59 smithi174 ceph-mon[88004]: pgmap v1055: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:47:00.052 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:46:59 smithi153 ceph-mon[113733]: pgmap v1055: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:47:01.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:47:01 smithi174 ceph-mon[88004]: pgmap v1056: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:47:01.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:47:01 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:47:02.052 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:47:01 smithi153 ceph-mon[113733]: pgmap v1056: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:47:02.052 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:47:01 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:47:03.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:47:03 smithi174 ceph-mon[88004]: pgmap v1057: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:47:04.052 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:47:03 smithi153 ceph-mon[113733]: pgmap v1057: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:47:05.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:47:05 smithi174 ceph-mon[88004]: pgmap v1058: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:47:06.052 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:47:05 smithi153 ceph-mon[113733]: pgmap v1058: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:47:07.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:47:07 smithi174 ceph-mon[88004]: pgmap v1059: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:47:08.052 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:47:07 smithi153 ceph-mon[113733]: pgmap v1059: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:47:09.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:47:09 smithi174 ceph-mon[88004]: pgmap v1060: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:47:10.052 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:47:09 smithi153 ceph-mon[113733]: pgmap v1060: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:47:11.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:47:11 smithi174 ceph-mon[88004]: pgmap v1061: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:47:12.052 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:47:11 smithi153 ceph-mon[113733]: pgmap v1061: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:47:13.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:47:13 smithi174 ceph-mon[88004]: pgmap v1062: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:47:14.052 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:47:13 smithi153 ceph-mon[113733]: pgmap v1062: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:47:15.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:47:15 smithi174 ceph-mon[88004]: pgmap v1063: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:47:16.052 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:47:15 smithi153 ceph-mon[113733]: pgmap v1063: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:47:16.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:47:16 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:47:16.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:47:16 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:47:17.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:47:17 smithi174 ceph-mon[88004]: pgmap v1064: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:47:18.052 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:47:17 smithi153 ceph-mon[113733]: pgmap v1064: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:47:18.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:47:18 smithi174 ceph-mon[88004]: pgmap v1065: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:47:19.052 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:47:18 smithi153 ceph-mon[113733]: pgmap v1065: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:47:21.447 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:47:21 smithi174 ceph-mon[88004]: pgmap v1066: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:47:21.541 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:47:21 smithi153 ceph-mon[113733]: pgmap v1066: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:47:21.776 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:21.779+0000 7f263e200700 1 -- 172.21.15.153:0/423766328 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26381071d0 msgr2=0x7f2638107650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:47:21.776 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:21.779+0000 7f263e200700 1 --2- 172.21.15.153:0/423766328 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26381071d0 0x7f2638107650 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f2624009910 tx=0x7f2624009c20 comp rx=0 tx=0).stop 2024-03-21T10:47:21.777 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:21.779+0000 7f263e200700 1 -- 172.21.15.153:0/423766328 shutdown_connections 2024-03-21T10:47:21.777 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:21.779+0000 7f263e200700 1 --2- 172.21.15.153:0/423766328 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26381071d0 0x7f2638107650 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:47:21.777 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:21.779+0000 7f263e200700 1 --2- 172.21.15.153:0/423766328 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f263810d280 0x7f263810d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:47:21.777 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:21.779+0000 7f263e200700 1 -- 172.21.15.153:0/423766328 >> 172.21.15.153:0/423766328 conn(0x7f26380769e0 msgr2=0x7f2638078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:47:21.777 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:21.779+0000 7f263e200700 1 -- 172.21.15.153:0/423766328 shutdown_connections 2024-03-21T10:47:21.777 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:21.779+0000 7f263e200700 1 -- 172.21.15.153:0/423766328 wait complete. 2024-03-21T10:47:21.778 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:21.780+0000 7f263e200700 1 Processor -- start 2024-03-21T10:47:21.779 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:21.780+0000 7f263e200700 1 -- start start 2024-03-21T10:47:21.779 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:21.781+0000 7f263e200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26381071d0 0x7f263819ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:47:21.779 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:21.781+0000 7f263e200700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f263810d280 0x7f263819d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:47:21.779 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:21.781+0000 7f263e200700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f263819da30 con 0x7f26381071d0 2024-03-21T10:47:21.779 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:21.781+0000 7f263e200700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26381a17c0 con 0x7f263810d280 2024-03-21T10:47:21.779 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:21.781+0000 7f2637400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26381071d0 0x7f263819ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:47:21.779 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:21.781+0000 7f2637400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26381071d0 0x7f263819ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:55090/0 (socket says 172.21.15.153:55090) 2024-03-21T10:47:21.779 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:21.781+0000 7f2637400700 1 -- 172.21.15.153:0/2727764221 learned_addr learned my addr 172.21.15.153:0/2727764221 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:47:21.779 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:21.781+0000 7f2637400700 1 -- 172.21.15.153:0/2727764221 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f263810d280 msgr2=0x7f263819d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:47:21.779 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:21.781+0000 7f2637400700 1 --2- 172.21.15.153:0/2727764221 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f263810d280 0x7f263819d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:47:21.779 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:21.781+0000 7f2637400700 1 -- 172.21.15.153:0/2727764221 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26240095f0 con 0x7f26381071d0 2024-03-21T10:47:21.779 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:21.782+0000 7f2637400700 1 --2- 172.21.15.153:0/2727764221 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26381071d0 0x7f263819ce10 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f262800b4b0 tx=0x7f262800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:47:21.780 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:21.782+0000 7f262fe00700 1 -- 172.21.15.153:0/2727764221 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2628011800 con 0x7f26381071d0 2024-03-21T10:47:21.780 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:21.782+0000 7f262fe00700 1 -- 172.21.15.153:0/2727764221 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2628011e20 con 0x7f26381071d0 2024-03-21T10:47:21.780 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:21.782+0000 7f263e200700 1 -- 172.21.15.153:0/2727764221 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26381a1aa0 con 0x7f26381071d0 2024-03-21T10:47:21.780 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:21.782+0000 7f262fe00700 1 -- 172.21.15.153:0/2727764221 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f262800f2e0 con 0x7f26381071d0 2024-03-21T10:47:21.780 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:21.782+0000 7f263e200700 1 -- 172.21.15.153:0/2727764221 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26381a2070 con 0x7f26381071d0 2024-03-21T10:47:21.781 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:21.783+0000 7f263e200700 1 -- 172.21.15.153:0/2727764221 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f263810f6f0 con 0x7f26381071d0 2024-03-21T10:47:21.781 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:21.784+0000 7f262fe00700 1 -- 172.21.15.153:0/2727764221 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f2628010380 con 0x7f26381071d0 2024-03-21T10:47:21.782 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:21.785+0000 7f262fe00700 1 --2- 172.21.15.153:0/2727764221 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f261c06e460 0x7f261c070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:47:21.782 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:21.785+0000 7f262fe00700 1 -- 172.21.15.153:0/2727764221 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f262808e190 con 0x7f26381071d0 2024-03-21T10:47:21.782 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:21.785+0000 7f2636a00700 1 --2- 172.21.15.153:0/2727764221 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f261c06e460 0x7f261c070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:47:21.783 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:21.786+0000 7f2636a00700 1 --2- 172.21.15.153:0/2727764221 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f261c06e460 0x7f261c070920 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f262400b5c0 tx=0x7f2624005fd0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:47:21.787 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:21.790+0000 7f262fe00700 1 -- 172.21.15.153:0/2727764221 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f26280571b0 con 0x7f26381071d0 2024-03-21T10:47:22.029 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:22.032+0000 7f263e200700 1 -- 172.21.15.153:0/2727764221 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f26381a23e0 con 0x7f261c06e460 2024-03-21T10:47:22.031 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:22.034+0000 7f262fe00700 1 -- 172.21.15.153:0/2727764221 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f26381a23e0 con 0x7f261c06e460 2024-03-21T10:47:22.034 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:22.037+0000 7f263e200700 1 -- 172.21.15.153:0/2727764221 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f261c06e460 msgr2=0x7f261c070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:47:22.034 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:22.037+0000 7f263e200700 1 --2- 172.21.15.153:0/2727764221 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f261c06e460 0x7f261c070920 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f262400b5c0 tx=0x7f2624005fd0 comp rx=0 tx=0).stop 2024-03-21T10:47:22.034 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:22.037+0000 7f263e200700 1 -- 172.21.15.153:0/2727764221 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26381071d0 msgr2=0x7f263819ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:47:22.034 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:22.037+0000 7f263e200700 1 --2- 172.21.15.153:0/2727764221 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26381071d0 0x7f263819ce10 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f262800b4b0 tx=0x7f262800b870 comp rx=0 tx=0).stop 2024-03-21T10:47:22.034 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:22.037+0000 7f263e200700 1 -- 172.21.15.153:0/2727764221 shutdown_connections 2024-03-21T10:47:22.034 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:22.037+0000 7f263e200700 1 --2- 172.21.15.153:0/2727764221 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f261c06e460 0x7f261c070920 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:47:22.034 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:22.037+0000 7f263e200700 1 --2- 172.21.15.153:0/2727764221 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f26381071d0 0x7f263819ce10 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:47:22.034 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:22.037+0000 7f263e200700 1 --2- 172.21.15.153:0/2727764221 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f263810d280 0x7f263819d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:47:22.035 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:22.037+0000 7f263e200700 1 -- 172.21.15.153:0/2727764221 >> 172.21.15.153:0/2727764221 conn(0x7f26380769e0 msgr2=0x7f26380784d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:47:22.035 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:22.037+0000 7f263e200700 1 -- 172.21.15.153:0/2727764221 shutdown_connections 2024-03-21T10:47:22.035 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:22.037+0000 7f263e200700 1 -- 172.21.15.153:0/2727764221 wait complete. 2024-03-21T10:47:22.051 INFO:teuthology.orchestra.run.smithi153.stdout:true 2024-03-21T10:47:22.197 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:22.200+0000 7fdd06800700 1 -- 172.21.15.153:0/1841716295 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdd00101770 msgr2=0x7fdd001057c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:47:22.198 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:22.200+0000 7fdd06800700 1 --2- 172.21.15.153:0/1841716295 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdd00101770 0x7fdd001057c0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fdcec009910 tx=0x7fdcec009c20 comp rx=0 tx=0).stop 2024-03-21T10:47:22.198 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:22.200+0000 7fdd06800700 1 -- 172.21.15.153:0/1841716295 shutdown_connections 2024-03-21T10:47:22.198 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:22.200+0000 7fdd06800700 1 --2- 172.21.15.153:0/1841716295 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdd00101770 0x7fdd001057c0 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:47:22.198 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:22.200+0000 7fdd06800700 1 --2- 172.21.15.153:0/1841716295 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdd00100dc0 0x7fdd001011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:47:22.198 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:22.200+0000 7fdd06800700 1 -- 172.21.15.153:0/1841716295 >> 172.21.15.153:0/1841716295 conn(0x7fdd000fc650 msgr2=0x7fdd000fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:47:22.198 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:22.200+0000 7fdd06800700 1 -- 172.21.15.153:0/1841716295 shutdown_connections 2024-03-21T10:47:22.198 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:22.200+0000 7fdd06800700 1 -- 172.21.15.153:0/1841716295 wait complete. 2024-03-21T10:47:22.200 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:22.201+0000 7fdd06800700 1 Processor -- start 2024-03-21T10:47:22.200 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:22.201+0000 7fdd06800700 1 -- start start 2024-03-21T10:47:22.200 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:22.202+0000 7fdd06800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdd00100dc0 0x7fdd00073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:47:22.200 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:22.202+0000 7fdd06800700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdd00101770 0x7fdd000735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:47:22.200 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:22.202+0000 7fdd06800700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd00073b10 con 0x7fdd00100dc0 2024-03-21T10:47:22.200 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:22.202+0000 7fdd06800700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd00073c50 con 0x7fdd00101770 2024-03-21T10:47:22.200 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:22.202+0000 7fdcff400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdd00100dc0 0x7fdd00073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:47:22.200 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:22.202+0000 7fdcff400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdd00100dc0 0x7fdd00073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:55116/0 (socket says 172.21.15.153:55116) 2024-03-21T10:47:22.200 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:22.202+0000 7fdcff400700 1 -- 172.21.15.153:0/4169738789 learned_addr learned my addr 172.21.15.153:0/4169738789 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:47:22.200 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:22.202+0000 7fdcfea00700 1 --2- 172.21.15.153:0/4169738789 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdd00101770 0x7fdd000735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:47:22.200 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:22.202+0000 7fdcff400700 1 -- 172.21.15.153:0/4169738789 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdd00101770 msgr2=0x7fdd000735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:47:22.201 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:22.202+0000 7fdcff400700 1 --2- 172.21.15.153:0/4169738789 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdd00101770 0x7fdd000735d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:47:22.201 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:22.203+0000 7fdcff400700 1 -- 172.21.15.153:0/4169738789 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcec0095f0 con 0x7fdd00100dc0 2024-03-21T10:47:22.201 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:22.203+0000 7fdcfea00700 1 --2- 172.21.15.153:0/4169738789 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdd00101770 0x7fdd000735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:47:22.201 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:22.203+0000 7fdcff400700 1 --2- 172.21.15.153:0/4169738789 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdd00100dc0 0x7fdd00073090 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fdcf000c8a0 tx=0x7fdcf000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:47:22.201 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:22.203+0000 7fdcf7e00700 1 -- 172.21.15.153:0/4169738789 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdcf000cea0 con 0x7fdd00100dc0 2024-03-21T10:47:22.202 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:22.203+0000 7fdcf7e00700 1 -- 172.21.15.153:0/4169738789 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdcf0004830 con 0x7fdd00100dc0 2024-03-21T10:47:22.202 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:22.203+0000 7fdcf7e00700 1 -- 172.21.15.153:0/4169738789 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdcf00055b0 con 0x7fdd00100dc0 2024-03-21T10:47:22.202 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:22.203+0000 7fdd06800700 1 -- 172.21.15.153:0/4169738789 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd001a2940 con 0x7fdd00100dc0 2024-03-21T10:47:22.202 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:22.203+0000 7fdd06800700 1 -- 172.21.15.153:0/4169738789 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd001a2d50 con 0x7fdd00100dc0 2024-03-21T10:47:22.202 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:22.205+0000 7fdd06800700 1 -- 172.21.15.153:0/4169738789 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd001091c0 con 0x7fdd00100dc0 2024-03-21T10:47:22.202 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:22.206+0000 7fdcf7e00700 1 -- 172.21.15.153:0/4169738789 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fdcf00049a0 con 0x7fdd00100dc0 2024-03-21T10:47:22.204 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:22.206+0000 7fdcf7e00700 1 --2- 172.21.15.153:0/4169738789 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fdce406e460 0x7fdce4070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:47:22.204 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:22.207+0000 7fdcf7e00700 1 -- 172.21.15.153:0/4169738789 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fdcf008e1d0 con 0x7fdd00100dc0 2024-03-21T10:47:22.204 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:22.207+0000 7fdcfea00700 1 --2- 172.21.15.153:0/4169738789 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fdce406e460 0x7fdce4070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:47:22.205 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:22.207+0000 7fdcfea00700 1 --2- 172.21.15.153:0/4169738789 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fdce406e460 0x7fdce4070920 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fdcec006010 tx=0x7fdcec00b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:47:22.208 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:22.212+0000 7fdcf7e00700 1 -- 172.21.15.153:0/4169738789 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fdcf0057290 con 0x7fdd00100dc0 2024-03-21T10:47:22.448 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:22.450+0000 7fdd06800700 1 -- 172.21.15.153:0/4169738789 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdd0002cdd0 con 0x7fdce406e460 2024-03-21T10:47:22.450 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:22.454+0000 7fdcf7e00700 1 -- 172.21.15.153:0/4169738789 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fdd0002cdd0 con 0x7fdce406e460 2024-03-21T10:47:22.453 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:22.456+0000 7fdd06800700 1 -- 172.21.15.153:0/4169738789 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fdce406e460 msgr2=0x7fdce4070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:47:22.453 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:22.456+0000 7fdd06800700 1 --2- 172.21.15.153:0/4169738789 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fdce406e460 0x7fdce4070920 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fdcec006010 tx=0x7fdcec00b540 comp rx=0 tx=0).stop 2024-03-21T10:47:22.454 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:22.456+0000 7fdd06800700 1 -- 172.21.15.153:0/4169738789 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdd00100dc0 msgr2=0x7fdd00073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:47:22.454 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:22.456+0000 7fdd06800700 1 --2- 172.21.15.153:0/4169738789 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdd00100dc0 0x7fdd00073090 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fdcf000c8a0 tx=0x7fdcf000cc60 comp rx=0 tx=0).stop 2024-03-21T10:47:22.454 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:22.456+0000 7fdd06800700 1 -- 172.21.15.153:0/4169738789 shutdown_connections 2024-03-21T10:47:22.454 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:22.456+0000 7fdd06800700 1 --2- 172.21.15.153:0/4169738789 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fdce406e460 0x7fdce4070920 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:47:22.454 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:22.456+0000 7fdd06800700 1 --2- 172.21.15.153:0/4169738789 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdd00100dc0 0x7fdd00073090 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:47:22.454 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:22.456+0000 7fdd06800700 1 --2- 172.21.15.153:0/4169738789 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdd00101770 0x7fdd000735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:47:22.454 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:22.456+0000 7fdd06800700 1 -- 172.21.15.153:0/4169738789 >> 172.21.15.153:0/4169738789 conn(0x7fdd000fc650 msgr2=0x7fdd000fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:47:22.454 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:22.456+0000 7fdd06800700 1 -- 172.21.15.153:0/4169738789 shutdown_connections 2024-03-21T10:47:22.454 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:22.457+0000 7fdd06800700 1 -- 172.21.15.153:0/4169738789 wait complete. 2024-03-21T10:47:22.614 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:22.616+0000 7f10ba200700 1 -- 172.21.15.153:0/1071965063 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f10b4068d30 msgr2=0x7f10b410d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:47:22.614 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:22.616+0000 7f10ba200700 1 --2- 172.21.15.153:0/1071965063 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f10b4068d30 0x7f10b410d240 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f10a0009910 tx=0x7f10a0009c20 comp rx=0 tx=0).stop 2024-03-21T10:47:22.614 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:22.617+0000 7f10ba200700 1 -- 172.21.15.153:0/1071965063 shutdown_connections 2024-03-21T10:47:22.614 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:22.617+0000 7f10ba200700 1 --2- 172.21.15.153:0/1071965063 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f10b4068d30 0x7f10b410d240 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:47:22.614 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:22.617+0000 7f10ba200700 1 --2- 172.21.15.153:0/1071965063 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f10b4068410 0x7f10b40687f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:47:22.614 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:22.617+0000 7f10ba200700 1 -- 172.21.15.153:0/1071965063 >> 172.21.15.153:0/1071965063 conn(0x7f10b40755a0 msgr2=0x7f10b40759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:47:22.615 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:22.617+0000 7f10ba200700 1 -- 172.21.15.153:0/1071965063 shutdown_connections 2024-03-21T10:47:22.615 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:22.617+0000 7f10ba200700 1 -- 172.21.15.153:0/1071965063 wait complete. 2024-03-21T10:47:22.615 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:22.618+0000 7f10ba200700 1 Processor -- start 2024-03-21T10:47:22.615 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:22.618+0000 7f10ba200700 1 -- start start 2024-03-21T10:47:22.616 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:22.619+0000 7f10ba200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f10b4068410 0x7f10b419e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:47:22.616 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:22.619+0000 7f10ba200700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f10b4068d30 0x7f10b419ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:47:22.616 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:22.619+0000 7f10ba200700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10b419f2b0 con 0x7f10b4068410 2024-03-21T10:47:22.616 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:22.619+0000 7f10ba200700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10b4198760 con 0x7f10b4068d30 2024-03-21T10:47:22.616 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:22.619+0000 7f10b3400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f10b4068410 0x7f10b419e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:47:22.617 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:22.619+0000 7f10b3400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f10b4068410 0x7f10b419e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:55144/0 (socket says 172.21.15.153:55144) 2024-03-21T10:47:22.617 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:22.619+0000 7f10b3400700 1 -- 172.21.15.153:0/545205027 learned_addr learned my addr 172.21.15.153:0/545205027 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:47:22.617 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:22.620+0000 7f10b2a00700 1 --2- 172.21.15.153:0/545205027 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f10b4068d30 0x7f10b419ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:47:22.617 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:22.620+0000 7f10b3400700 1 -- 172.21.15.153:0/545205027 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f10b4068d30 msgr2=0x7f10b419ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:47:22.617 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:22.620+0000 7f10b3400700 1 --2- 172.21.15.153:0/545205027 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f10b4068d30 0x7f10b419ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:47:22.617 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:22.620+0000 7f10b3400700 1 -- 172.21.15.153:0/545205027 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10a00095f0 con 0x7f10b4068410 2024-03-21T10:47:22.618 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:22.620+0000 7f10b3400700 1 --2- 172.21.15.153:0/545205027 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f10b4068410 0x7f10b419e6e0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f10a400d900 tx=0x7f10a400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:47:22.618 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:22.621+0000 7f10abe00700 1 -- 172.21.15.153:0/545205027 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f10a40041f0 con 0x7f10b4068410 2024-03-21T10:47:22.618 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:22.621+0000 7f10abe00700 1 -- 172.21.15.153:0/545205027 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f10a4004350 con 0x7f10b4068410 2024-03-21T10:47:22.618 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:22.621+0000 7f10abe00700 1 -- 172.21.15.153:0/545205027 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f10a4003dc0 con 0x7f10b4068410 2024-03-21T10:47:22.618 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:22.621+0000 7f10ba200700 1 -- 172.21.15.153:0/545205027 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10b4198a40 con 0x7f10b4068410 2024-03-21T10:47:22.618 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:22.621+0000 7f10ba200700 1 -- 172.21.15.153:0/545205027 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10b4198f40 con 0x7f10b4068410 2024-03-21T10:47:22.619 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:22.622+0000 7f10ba200700 1 -- 172.21.15.153:0/545205027 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10b410a9b0 con 0x7f10b4068410 2024-03-21T10:47:22.620 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:22.623+0000 7f10abe00700 1 -- 172.21.15.153:0/545205027 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f10a4009730 con 0x7f10b4068410 2024-03-21T10:47:22.621 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:22.624+0000 7f10abe00700 1 --2- 172.21.15.153:0/545205027 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f109806e460 0x7f1098070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:47:22.621 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:22.624+0000 7f10abe00700 1 -- 172.21.15.153:0/545205027 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f10a408e080 con 0x7f10b4068410 2024-03-21T10:47:22.621 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:22.625+0000 7f10b2a00700 1 --2- 172.21.15.153:0/545205027 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f109806e460 0x7f1098070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:47:22.622 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:22.625+0000 7f10b2a00700 1 --2- 172.21.15.153:0/545205027 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f109806e460 0x7f1098070920 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f10b4199bd0 tx=0x7f10a001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:47:22.626 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:22.629+0000 7f10abe00700 1 -- 172.21.15.153:0/545205027 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f10a405ad90 con 0x7f10b4068410 2024-03-21T10:47:22.839 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:22.842+0000 7f10ba200700 1 -- 172.21.15.153:0/545205027 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f10b4199280 con 0x7f109806e460 2024-03-21T10:47:22.870 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:22.873+0000 7f10abe00700 1 -- 172.21.15.153:0/545205027 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f10b4199280 con 0x7f109806e460 2024-03-21T10:47:22.870 INFO:teuthology.orchestra.run.smithi153.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-21T10:47:22.871 INFO:teuthology.orchestra.run.smithi153.stdout:alertmanager.smithi153 smithi153 *:9093,9094 running (36m) 11m ago 49m 15.6M - 0.25.0 c8568f914cd2 9ed3f2200b5f 2024-03-21T10:47:22.871 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi153 smithi153 running (50m) 11m ago 50m 10.4M - 18.2.1 5be31c24972a 5b61994396a6 2024-03-21T10:47:22.871 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi174 smithi174 running (48m) 11m ago 48m 10.1M - 18.2.1 5be31c24972a 4f7a5fc3ac28 2024-03-21T10:47:22.871 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi153 smithi153 running (32m) 11m ago 50m 6656k - 19.0.0-2231-g2edfee68 237e43c7a65f 32c21104d5b0 2024-03-21T10:47:22.871 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi174 smithi174 running (32m) 11m ago 48m 6660k - 19.0.0-2231-g2edfee68 237e43c7a65f a04e48967ab6 2024-03-21T10:47:22.871 INFO:teuthology.orchestra.run.smithi153.stdout:grafana.smithi153 smithi153 *:3000 running (35m) 11m ago 49m 75.2M - 9.4.12 2bacad6d85d8 ee24026ab786 2024-03-21T10:47:22.871 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.iuyluj smithi153 running (27m) 11m ago 42m 18.7M - 19.0.0-2231-g2edfee68 237e43c7a65f 87aff8a5de70 2024-03-21T10:47:22.871 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.zipckr smithi153 running (28m) 11m ago 42m 18.9M - 19.0.0-2231-g2edfee68 237e43c7a65f ccc262bd5dd3 2024-03-21T10:47:22.871 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.nhozag smithi174 running (12m) 11m ago 42m 10.9M - 19.0.0-2231-g2edfee68 237e43c7a65f 9e02aea72c96 2024-03-21T10:47:22.872 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.yyimst smithi174 running (27m) 11m ago 42m 18.5M - 19.0.0-2231-g2edfee68 237e43c7a65f fc1962f9351e 2024-03-21T10:47:22.872 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi153.tluxni smithi153 *:8443,9283,8765 running (39m) 11m ago 51m 512M - 19.0.0-2231-g2edfee68 237e43c7a65f 7ffee0e55382 2024-03-21T10:47:22.872 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi174.qpcwat smithi174 *:8443,9283,8765 running (37m) 11m ago 48m 459M - 19.0.0-2231-g2edfee68 237e43c7a65f 748968f216ab 2024-03-21T10:47:22.872 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi153 smithi153 running (34m) 11m ago 52m 70.8M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f dd8038303dc3 2024-03-21T10:47:22.872 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi174 smithi174 running (33m) 11m ago 48m 51.9M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f 29d78a788f2c 2024-03-21T10:47:22.872 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi153 smithi153 *:9100 running (37m) 11m ago 50m 8140k - 1.5.0 0da6a335fe13 bb835edc0e0b 2024-03-21T10:47:22.873 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi174 smithi174 *:9100 running (37m) 11m ago 48m 8610k - 1.5.0 0da6a335fe13 7c4ecad790fa 2024-03-21T10:47:22.873 INFO:teuthology.orchestra.run.smithi153.stdout:osd.0 smithi153 running (31m) 11m ago 47m 118M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 507900fd4791 2024-03-21T10:47:22.873 INFO:teuthology.orchestra.run.smithi153.stdout:osd.1 smithi153 running (30m) 11m ago 46m 93.0M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 0d646452e9c1 2024-03-21T10:47:22.873 INFO:teuthology.orchestra.run.smithi153.stdout:osd.2 smithi153 running (30m) 11m ago 46m 91.2M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 1326f2afdfcd 2024-03-21T10:47:22.873 INFO:teuthology.orchestra.run.smithi153.stdout:osd.3 smithi174 running (29m) 11m ago 45m 87.5M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 70aafac71ad5 2024-03-21T10:47:22.873 INFO:teuthology.orchestra.run.smithi153.stdout:osd.4 smithi174 running (29m) 11m ago 45m 95.8M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 0d746ce6e79d 2024-03-21T10:47:22.874 INFO:teuthology.orchestra.run.smithi153.stdout:osd.5 smithi174 running (28m) 11m ago 44m 90.3M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 53e5f35b8cac 2024-03-21T10:47:22.874 INFO:teuthology.orchestra.run.smithi153.stdout:prometheus.smithi153 smithi153 *:9095 running (36m) 11m ago 49m 88.1M - 2.43.0 a07b618ecd1d 7d95561e50c2 2024-03-21T10:47:22.874 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:22.876+0000 7f10ba200700 1 -- 172.21.15.153:0/545205027 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f109806e460 msgr2=0x7f1098070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:47:22.875 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:22.876+0000 7f10ba200700 1 --2- 172.21.15.153:0/545205027 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f109806e460 0x7f1098070920 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f10b4199bd0 tx=0x7f10a001a040 comp rx=0 tx=0).stop 2024-03-21T10:47:22.875 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:22.876+0000 7f10ba200700 1 -- 172.21.15.153:0/545205027 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f10b4068410 msgr2=0x7f10b419e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:47:22.875 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:22.876+0000 7f10ba200700 1 --2- 172.21.15.153:0/545205027 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f10b4068410 0x7f10b419e6e0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f10a400d900 tx=0x7f10a400dcc0 comp rx=0 tx=0).stop 2024-03-21T10:47:22.875 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:22.877+0000 7f10ba200700 1 -- 172.21.15.153:0/545205027 shutdown_connections 2024-03-21T10:47:22.875 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:22.877+0000 7f10ba200700 1 --2- 172.21.15.153:0/545205027 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f109806e460 0x7f1098070920 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:47:22.875 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:22.877+0000 7f10ba200700 1 --2- 172.21.15.153:0/545205027 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f10b4068410 0x7f10b419e6e0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:47:22.875 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:22.877+0000 7f10ba200700 1 --2- 172.21.15.153:0/545205027 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f10b4068d30 0x7f10b419ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:47:22.875 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:22.877+0000 7f10ba200700 1 -- 172.21.15.153:0/545205027 >> 172.21.15.153:0/545205027 conn(0x7f10b40755a0 msgr2=0x7f10b40fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:47:22.875 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:22.877+0000 7f10ba200700 1 -- 172.21.15.153:0/545205027 shutdown_connections 2024-03-21T10:47:22.875 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:22.877+0000 7f10ba200700 1 -- 172.21.15.153:0/545205027 wait complete. 2024-03-21T10:47:23.032 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.035+0000 7f6ad1600700 1 -- 172.21.15.153:0/2273338661 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6acc101750 msgr2=0x7f6acc1057a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:47:23.032 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.035+0000 7f6ad1600700 1 --2- 172.21.15.153:0/2273338661 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6acc101750 0x7f6acc1057a0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f6ab8009910 tx=0x7f6ab8009c20 comp rx=0 tx=0).stop 2024-03-21T10:47:23.033 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.035+0000 7f6ad1600700 1 -- 172.21.15.153:0/2273338661 shutdown_connections 2024-03-21T10:47:23.033 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.035+0000 7f6ad1600700 1 --2- 172.21.15.153:0/2273338661 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6acc101750 0x7f6acc1057a0 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:47:23.033 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.035+0000 7f6ad1600700 1 --2- 172.21.15.153:0/2273338661 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6acc100da0 0x7f6acc101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:47:23.033 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.035+0000 7f6ad1600700 1 -- 172.21.15.153:0/2273338661 >> 172.21.15.153:0/2273338661 conn(0x7f6acc0fc630 msgr2=0x7f6acc0fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:47:23.033 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.035+0000 7f6ad1600700 1 -- 172.21.15.153:0/2273338661 shutdown_connections 2024-03-21T10:47:23.033 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.035+0000 7f6ad1600700 1 -- 172.21.15.153:0/2273338661 wait complete. 2024-03-21T10:47:23.034 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.036+0000 7f6ad1600700 1 Processor -- start 2024-03-21T10:47:23.034 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.037+0000 7f6ad1600700 1 -- start start 2024-03-21T10:47:23.034 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.037+0000 7f6ad1600700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6acc100da0 0x7f6acc196730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:47:23.035 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.037+0000 7f6ad1600700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6acc101750 0x7f6acc196c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:47:23.035 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.037+0000 7f6ad1600700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6acc197300 con 0x7f6acc101750 2024-03-21T10:47:23.035 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.037+0000 7f6ad1600700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6acc199fc0 con 0x7f6acc100da0 2024-03-21T10:47:23.035 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.037+0000 7f6aca000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6acc101750 0x7f6acc196c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:47:23.035 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.037+0000 7f6aca000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6acc101750 0x7f6acc196c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39790/0 (socket says 172.21.15.153:39790) 2024-03-21T10:47:23.035 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.037+0000 7f6aca000700 1 -- 172.21.15.153:0/196863852 learned_addr learned my addr 172.21.15.153:0/196863852 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:47:23.036 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.038+0000 7f6acaa00700 1 --2- 172.21.15.153:0/196863852 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6acc100da0 0x7f6acc196730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:47:23.036 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.038+0000 7f6aca000700 1 -- 172.21.15.153:0/196863852 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6acc100da0 msgr2=0x7f6acc196730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:47:23.036 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.038+0000 7f6aca000700 1 --2- 172.21.15.153:0/196863852 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6acc100da0 0x7f6acc196730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:47:23.037 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.038+0000 7f6aca000700 1 -- 172.21.15.153:0/196863852 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ab80095f0 con 0x7f6acc101750 2024-03-21T10:47:23.037 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.038+0000 7f6aca000700 1 --2- 172.21.15.153:0/196863852 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6acc101750 0x7f6acc196c70 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f6ab800bda0 tx=0x7f6ab800be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:47:23.037 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.038+0000 7f6ac3400700 1 -- 172.21.15.153:0/196863852 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6ab801d070 con 0x7f6acc101750 2024-03-21T10:47:23.037 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.039+0000 7f6ad1600700 1 -- 172.21.15.153:0/196863852 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6acc19a240 con 0x7f6acc101750 2024-03-21T10:47:23.037 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.039+0000 7f6ac3400700 1 -- 172.21.15.153:0/196863852 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6ab8004fb0 con 0x7f6acc101750 2024-03-21T10:47:23.037 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.039+0000 7f6ac3400700 1 -- 172.21.15.153:0/196863852 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6ab800f460 con 0x7f6acc101750 2024-03-21T10:47:23.037 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.039+0000 7f6ad1600700 1 -- 172.21.15.153:0/196863852 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6acc19a730 con 0x7f6acc101750 2024-03-21T10:47:23.038 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.040+0000 7f6ad1600700 1 -- 172.21.15.153:0/196863852 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6acc1091a0 con 0x7f6acc101750 2024-03-21T10:47:23.038 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.041+0000 7f6ac3400700 1 -- 172.21.15.153:0/196863852 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f6ab8022800 con 0x7f6acc101750 2024-03-21T10:47:23.039 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.042+0000 7f6ac3400700 1 --2- 172.21.15.153:0/196863852 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f6ab006e460 0x7f6ab0070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:47:23.040 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.043+0000 7f6ac3400700 1 -- 172.21.15.153:0/196863852 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f6ab808fa20 con 0x7f6acc101750 2024-03-21T10:47:23.040 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.043+0000 7f6acaa00700 1 --2- 172.21.15.153:0/196863852 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f6ab006e460 0x7f6ab0070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:47:23.040 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.044+0000 7f6acaa00700 1 --2- 172.21.15.153:0/196863852 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f6ab006e460 0x7f6ab0070920 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f6abc005950 tx=0x7f6abc005cf0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:47:23.044 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.047+0000 7f6ac3400700 1 -- 172.21.15.153:0/196863852 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f6ab8058c90 con 0x7f6acc101750 2024-03-21T10:47:23.302 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:47:23 smithi153 ceph-mon[113733]: pgmap v1067: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:47:23.302 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:47:23 smithi153 ceph-mon[113733]: from='client.35882 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:47:23.303 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:47:23 smithi153 ceph-mon[113733]: from='client.35886 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:47:23.354 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.357+0000 7f6ad1600700 1 -- 172.21.15.153:0/196863852 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f6acc04ea90 con 0x7f6acc101750 2024-03-21T10:47:23.355 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.358+0000 7f6ac3400700 1 -- 172.21.15.153:0/196863852 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f6ab805c350 con 0x7f6acc101750 2024-03-21T10:47:23.355 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:47:23.356 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": { 2024-03-21T10:47:23.356 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:47:23.356 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:47:23.356 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": { 2024-03-21T10:47:23.356 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:47:23.356 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:47:23.356 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": { 2024-03-21T10:47:23.356 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 6 2024-03-21T10:47:23.356 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:47:23.356 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": { 2024-03-21T10:47:23.356 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 4 2024-03-21T10:47:23.356 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:47:23.356 INFO:teuthology.orchestra.run.smithi153.stdout: "overall": { 2024-03-21T10:47:23.356 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 14 2024-03-21T10:47:23.357 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-03-21T10:47:23.357 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:47:23.358 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.361+0000 7f6ad1600700 1 -- 172.21.15.153:0/196863852 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f6ab006e460 msgr2=0x7f6ab0070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:47:23.358 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.361+0000 7f6ad1600700 1 --2- 172.21.15.153:0/196863852 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f6ab006e460 0x7f6ab0070920 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f6abc005950 tx=0x7f6abc005cf0 comp rx=0 tx=0).stop 2024-03-21T10:47:23.358 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.361+0000 7f6ad1600700 1 -- 172.21.15.153:0/196863852 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6acc101750 msgr2=0x7f6acc196c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:47:23.358 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.361+0000 7f6ad1600700 1 --2- 172.21.15.153:0/196863852 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6acc101750 0x7f6acc196c70 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f6ab800bda0 tx=0x7f6ab800be80 comp rx=0 tx=0).stop 2024-03-21T10:47:23.358 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.361+0000 7f6ad1600700 1 -- 172.21.15.153:0/196863852 shutdown_connections 2024-03-21T10:47:23.359 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.361+0000 7f6ad1600700 1 --2- 172.21.15.153:0/196863852 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6acc100da0 0x7f6acc196730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:47:23.359 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.361+0000 7f6ad1600700 1 --2- 172.21.15.153:0/196863852 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f6ab006e460 0x7f6ab0070920 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:47:23.359 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.361+0000 7f6ad1600700 1 --2- 172.21.15.153:0/196863852 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6acc101750 0x7f6acc196c70 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:47:23.359 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.361+0000 7f6ad1600700 1 -- 172.21.15.153:0/196863852 >> 172.21.15.153:0/196863852 conn(0x7f6acc0fc630 msgr2=0x7f6acc0fea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:47:23.359 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.361+0000 7f6ad1600700 1 -- 172.21.15.153:0/196863852 shutdown_connections 2024-03-21T10:47:23.359 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.361+0000 7f6ad1600700 1 -- 172.21.15.153:0/196863852 wait complete. 2024-03-21T10:47:23.447 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:47:23 smithi174 ceph-mon[88004]: pgmap v1067: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:47:23.447 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:47:23 smithi174 ceph-mon[88004]: from='client.35882 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:47:23.447 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:47:23 smithi174 ceph-mon[88004]: from='client.35886 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:47:23.515 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.517+0000 7ff513c00700 1 -- 172.21.15.153:0/4293562529 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff50c103960 msgr2=0x7ff50c1079b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:47:23.515 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.517+0000 7ff513c00700 1 --2- 172.21.15.153:0/4293562529 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff50c103960 0x7ff50c1079b0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7ff4f8009910 tx=0x7ff4f8009c20 comp rx=0 tx=0).stop 2024-03-21T10:47:23.515 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.517+0000 7ff513c00700 1 -- 172.21.15.153:0/4293562529 shutdown_connections 2024-03-21T10:47:23.515 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.517+0000 7ff513c00700 1 --2- 172.21.15.153:0/4293562529 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff50c103960 0x7ff50c1079b0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:47:23.515 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.517+0000 7ff513c00700 1 --2- 172.21.15.153:0/4293562529 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff50c102fb0 0x7ff50c103390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:47:23.515 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.517+0000 7ff513c00700 1 -- 172.21.15.153:0/4293562529 >> 172.21.15.153:0/4293562529 conn(0x7ff50c0fe840 msgr2=0x7ff50c100c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:47:23.515 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.517+0000 7ff513c00700 1 -- 172.21.15.153:0/4293562529 shutdown_connections 2024-03-21T10:47:23.515 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.517+0000 7ff513c00700 1 -- 172.21.15.153:0/4293562529 wait complete. 2024-03-21T10:47:23.518 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.518+0000 7ff513c00700 1 Processor -- start 2024-03-21T10:47:23.518 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.518+0000 7ff513c00700 1 -- start start 2024-03-21T10:47:23.518 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.519+0000 7ff513c00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff50c102fb0 0x7ff50c19e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:47:23.518 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.519+0000 7ff513c00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff50c103960 0x7ff50c19edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:47:23.519 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.519+0000 7ff513c00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff50c19f440 con 0x7ff50c102fb0 2024-03-21T10:47:23.519 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.519+0000 7ff513c00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff50c1988f0 con 0x7ff50c103960 2024-03-21T10:47:23.519 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.519+0000 7ff511200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff50c102fb0 0x7ff50c19e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:47:23.519 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.519+0000 7ff511200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff50c102fb0 0x7ff50c19e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39810/0 (socket says 172.21.15.153:39810) 2024-03-21T10:47:23.519 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.519+0000 7ff511200700 1 -- 172.21.15.153:0/47894971 learned_addr learned my addr 172.21.15.153:0/47894971 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:47:23.519 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.520+0000 7ff50be00700 1 --2- 172.21.15.153:0/47894971 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff50c103960 0x7ff50c19edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:47:23.519 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.520+0000 7ff511200700 1 -- 172.21.15.153:0/47894971 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff50c103960 msgr2=0x7ff50c19edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:47:23.519 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.520+0000 7ff511200700 1 --2- 172.21.15.153:0/47894971 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff50c103960 0x7ff50c19edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:47:23.519 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.520+0000 7ff511200700 1 -- 172.21.15.153:0/47894971 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4f80095f0 con 0x7ff50c102fb0 2024-03-21T10:47:23.519 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.520+0000 7ff50be00700 1 --2- 172.21.15.153:0/47894971 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff50c103960 0x7ff50c19edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:47:23.520 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.520+0000 7ff511200700 1 --2- 172.21.15.153:0/47894971 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff50c102fb0 0x7ff50c19e870 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7ff50000b4b0 tx=0x7ff50000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:47:23.520 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.520+0000 7ff509600700 1 -- 172.21.15.153:0/47894971 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff500011800 con 0x7ff50c102fb0 2024-03-21T10:47:23.520 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.520+0000 7ff509600700 1 -- 172.21.15.153:0/47894971 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff500011e20 con 0x7ff50c102fb0 2024-03-21T10:47:23.520 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.520+0000 7ff513c00700 1 -- 172.21.15.153:0/47894971 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff50c198bd0 con 0x7ff50c102fb0 2024-03-21T10:47:23.520 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.520+0000 7ff509600700 1 -- 172.21.15.153:0/47894971 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff50000f2e0 con 0x7ff50c102fb0 2024-03-21T10:47:23.520 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.521+0000 7ff513c00700 1 -- 172.21.15.153:0/47894971 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff50c199120 con 0x7ff50c102fb0 2024-03-21T10:47:23.521 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.522+0000 7ff513c00700 1 -- 172.21.15.153:0/47894971 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff50c10b2e0 con 0x7ff50c102fb0 2024-03-21T10:47:23.521 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.523+0000 7ff509600700 1 -- 172.21.15.153:0/47894971 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7ff500010380 con 0x7ff50c102fb0 2024-03-21T10:47:23.521 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.524+0000 7ff509600700 1 --2- 172.21.15.153:0/47894971 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff4f406e460 0x7ff4f4070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:47:23.521 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.524+0000 7ff509600700 1 -- 172.21.15.153:0/47894971 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7ff50008e150 con 0x7ff50c102fb0 2024-03-21T10:47:23.521 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.524+0000 7ff50be00700 1 --2- 172.21.15.153:0/47894971 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff4f406e460 0x7ff4f4070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:47:23.522 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.525+0000 7ff50be00700 1 --2- 172.21.15.153:0/47894971 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff4f406e460 0x7ff4f4070920 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7ff4f8006010 tx=0x7ff4f800b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:47:23.526 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.529+0000 7ff509600700 1 -- 172.21.15.153:0/47894971 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff50005ace0 con 0x7ff50c102fb0 2024-03-21T10:47:23.794 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.797+0000 7ff513c00700 1 -- 172.21.15.153:0/47894971 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7ff50c1993a0 con 0x7ff50c102fb0 2024-03-21T10:47:23.795 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.798+0000 7ff509600700 1 -- 172.21.15.153:0/47894971 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1876 (secure 0 0 0) 0x7ff50005a870 con 0x7ff50c102fb0 2024-03-21T10:47:23.796 INFO:teuthology.orchestra.run.smithi153.stdout:e32 2024-03-21T10:47:23.796 INFO:teuthology.orchestra.run.smithi153.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-21T10:47:23.796 INFO:teuthology.orchestra.run.smithi153.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:47:23.796 INFO:teuthology.orchestra.run.smithi153.stdout:legacy client fscid: 1 2024-03-21T10:47:23.796 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:47:23.796 INFO:teuthology.orchestra.run.smithi153.stdout:Filesystem 'cephfs' (1) 2024-03-21T10:47:23.796 INFO:teuthology.orchestra.run.smithi153.stdout:fs_name cephfs 2024-03-21T10:47:23.796 INFO:teuthology.orchestra.run.smithi153.stdout:epoch 23 2024-03-21T10:47:23.796 INFO:teuthology.orchestra.run.smithi153.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-21T10:47:23.796 INFO:teuthology.orchestra.run.smithi153.stdout:created 2024-03-21T10:04:27.086576+0000 2024-03-21T10:47:23.796 INFO:teuthology.orchestra.run.smithi153.stdout:modified 2024-03-21T10:19:05.022092+0000 2024-03-21T10:47:23.797 INFO:teuthology.orchestra.run.smithi153.stdout:tableserver 0 2024-03-21T10:47:23.797 INFO:teuthology.orchestra.run.smithi153.stdout:root 0 2024-03-21T10:47:23.797 INFO:teuthology.orchestra.run.smithi153.stdout:session_timeout 60 2024-03-21T10:47:23.797 INFO:teuthology.orchestra.run.smithi153.stdout:session_autoclose 300 2024-03-21T10:47:23.797 INFO:teuthology.orchestra.run.smithi153.stdout:max_file_size 1099511627776 2024-03-21T10:47:23.797 INFO:teuthology.orchestra.run.smithi153.stdout:max_xattr_size 65536 2024-03-21T10:47:23.797 INFO:teuthology.orchestra.run.smithi153.stdout:required_client_features {} 2024-03-21T10:47:23.797 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure 0 2024-03-21T10:47:23.797 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure_osd_epoch 94 2024-03-21T10:47:23.797 INFO:teuthology.orchestra.run.smithi153.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:47:23.797 INFO:teuthology.orchestra.run.smithi153.stdout:max_mds 2 2024-03-21T10:47:23.797 INFO:teuthology.orchestra.run.smithi153.stdout:in 0,1 2024-03-21T10:47:23.797 INFO:teuthology.orchestra.run.smithi153.stdout:up {} 2024-03-21T10:47:23.798 INFO:teuthology.orchestra.run.smithi153.stdout:failed 0,1 2024-03-21T10:47:23.798 INFO:teuthology.orchestra.run.smithi153.stdout:damaged 2024-03-21T10:47:23.798 INFO:teuthology.orchestra.run.smithi153.stdout:stopped 2024-03-21T10:47:23.798 INFO:teuthology.orchestra.run.smithi153.stdout:data_pools [3] 2024-03-21T10:47:23.798 INFO:teuthology.orchestra.run.smithi153.stdout:metadata_pool 2 2024-03-21T10:47:23.798 INFO:teuthology.orchestra.run.smithi153.stdout:inline_data enabled 2024-03-21T10:47:23.798 INFO:teuthology.orchestra.run.smithi153.stdout:balancer 2024-03-21T10:47:23.798 INFO:teuthology.orchestra.run.smithi153.stdout:bal_rank_mask -1 2024-03-21T10:47:23.798 INFO:teuthology.orchestra.run.smithi153.stdout:standby_count_wanted 1 2024-03-21T10:47:23.798 INFO:teuthology.orchestra.run.smithi153.stdout:qdb_cluster leader: 0 members: 2024-03-21T10:47:23.798 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:47:23.798 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:47:23.798 INFO:teuthology.orchestra.run.smithi153.stdout:Standby daemons: 2024-03-21T10:47:23.799 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:47:23.799 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.zipckr{-1:34442} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6826/1103846295,v1:172.21.15.153:6827/1103846295] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:47:23.799 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.iuyluj{-1:34474} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6828/1738630832,v1:172.21.15.153:6829/1738630832] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:47:23.799 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.yyimst{-1:44313} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.174:6824/4127786797,v1:172.21.15.174:6825/4127786797] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:47:23.799 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.nhozag{-1:44687} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.174:6826/2767517845,v1:172.21.15.174:6827/2767517845] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:47:23.800 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.801+0000 7ff513c00700 1 -- 172.21.15.153:0/47894971 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff4f406e460 msgr2=0x7ff4f4070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:47:23.800 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.801+0000 7ff513c00700 1 --2- 172.21.15.153:0/47894971 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff4f406e460 0x7ff4f4070920 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7ff4f8006010 tx=0x7ff4f800b540 comp rx=0 tx=0).stop 2024-03-21T10:47:23.800 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.801+0000 7ff513c00700 1 -- 172.21.15.153:0/47894971 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff50c102fb0 msgr2=0x7ff50c19e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:47:23.800 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.801+0000 7ff513c00700 1 --2- 172.21.15.153:0/47894971 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff50c102fb0 0x7ff50c19e870 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7ff50000b4b0 tx=0x7ff50000b870 comp rx=0 tx=0).stop 2024-03-21T10:47:23.800 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.801+0000 7ff513c00700 1 -- 172.21.15.153:0/47894971 shutdown_connections 2024-03-21T10:47:23.800 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.801+0000 7ff513c00700 1 --2- 172.21.15.153:0/47894971 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff4f406e460 0x7ff4f4070920 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:47:23.800 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.801+0000 7ff513c00700 1 --2- 172.21.15.153:0/47894971 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff50c102fb0 0x7ff50c19e870 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:47:23.800 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.801+0000 7ff513c00700 1 --2- 172.21.15.153:0/47894971 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff50c103960 0x7ff50c19edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:47:23.800 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.801+0000 7ff513c00700 1 -- 172.21.15.153:0/47894971 >> 172.21.15.153:0/47894971 conn(0x7ff50c0fe840 msgr2=0x7ff50c0fff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:47:23.800 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.801+0000 7ff513c00700 1 -- 172.21.15.153:0/47894971 shutdown_connections 2024-03-21T10:47:23.800 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.802+0000 7ff513c00700 1 -- 172.21.15.153:0/47894971 wait complete. 2024-03-21T10:47:23.800 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 32 2024-03-21T10:47:23.945 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.948+0000 7fca2a600700 1 -- 172.21.15.153:0/749049849 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca24101770 msgr2=0x7fca241057c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:47:23.945 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.948+0000 7fca2a600700 1 --2- 172.21.15.153:0/749049849 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca24101770 0x7fca241057c0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fca0c009910 tx=0x7fca0c009c20 comp rx=0 tx=0).stop 2024-03-21T10:47:23.945 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.948+0000 7fca2a600700 1 -- 172.21.15.153:0/749049849 shutdown_connections 2024-03-21T10:47:23.945 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.948+0000 7fca2a600700 1 --2- 172.21.15.153:0/749049849 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca24101770 0x7fca241057c0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:47:23.945 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.948+0000 7fca2a600700 1 --2- 172.21.15.153:0/749049849 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fca24100dc0 0x7fca241011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:47:23.945 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.948+0000 7fca2a600700 1 -- 172.21.15.153:0/749049849 >> 172.21.15.153:0/749049849 conn(0x7fca240fc670 msgr2=0x7fca240fea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:47:23.946 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.948+0000 7fca2a600700 1 -- 172.21.15.153:0/749049849 shutdown_connections 2024-03-21T10:47:23.946 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.948+0000 7fca2a600700 1 -- 172.21.15.153:0/749049849 wait complete. 2024-03-21T10:47:23.947 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.949+0000 7fca2a600700 1 Processor -- start 2024-03-21T10:47:23.947 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.949+0000 7fca2a600700 1 -- start start 2024-03-21T10:47:23.947 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.949+0000 7fca2a600700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca24100dc0 0x7fca2419e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:47:23.947 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.949+0000 7fca2a600700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fca24101770 0x7fca2419ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:47:23.947 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.949+0000 7fca2a600700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca2419f350 con 0x7fca24100dc0 2024-03-21T10:47:23.947 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.949+0000 7fca2a600700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca24198800 con 0x7fca24101770 2024-03-21T10:47:23.947 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.950+0000 7fca23400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca24100dc0 0x7fca2419e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:47:23.947 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.950+0000 7fca23400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca24100dc0 0x7fca2419e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39830/0 (socket says 172.21.15.153:39830) 2024-03-21T10:47:23.947 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.950+0000 7fca23400700 1 -- 172.21.15.153:0/3588266508 learned_addr learned my addr 172.21.15.153:0/3588266508 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:47:23.948 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.950+0000 7fca1aa00700 1 --2- 172.21.15.153:0/3588266508 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fca24101770 0x7fca2419ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:47:23.948 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.950+0000 7fca23400700 1 -- 172.21.15.153:0/3588266508 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fca24101770 msgr2=0x7fca2419ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:47:23.948 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.950+0000 7fca23400700 1 --2- 172.21.15.153:0/3588266508 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fca24101770 0x7fca2419ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:47:23.948 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.950+0000 7fca23400700 1 -- 172.21.15.153:0/3588266508 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca0c0095f0 con 0x7fca24100dc0 2024-03-21T10:47:23.948 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.951+0000 7fca23400700 1 --2- 172.21.15.153:0/3588266508 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca24100dc0 0x7fca2419e780 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fca1400d8d0 tx=0x7fca1400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:47:23.948 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.951+0000 7fca20c00700 1 -- 172.21.15.153:0/3588266508 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fca14009940 con 0x7fca24100dc0 2024-03-21T10:47:23.948 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.951+0000 7fca20c00700 1 -- 172.21.15.153:0/3588266508 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fca14010460 con 0x7fca24100dc0 2024-03-21T10:47:23.949 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.951+0000 7fca2a600700 1 -- 172.21.15.153:0/3588266508 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca24198ae0 con 0x7fca24100dc0 2024-03-21T10:47:23.949 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.951+0000 7fca20c00700 1 -- 172.21.15.153:0/3588266508 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fca1400f5f0 con 0x7fca24100dc0 2024-03-21T10:47:23.949 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.951+0000 7fca2a600700 1 -- 172.21.15.153:0/3588266508 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca24199030 con 0x7fca24100dc0 2024-03-21T10:47:23.949 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.952+0000 7fca2a600700 1 -- 172.21.15.153:0/3588266508 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca24104f40 con 0x7fca24100dc0 2024-03-21T10:47:23.950 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.953+0000 7fca20c00700 1 -- 172.21.15.153:0/3588266508 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fca140105d0 con 0x7fca24100dc0 2024-03-21T10:47:23.952 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.955+0000 7fca20c00700 1 --2- 172.21.15.153:0/3588266508 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fca0806e460 0x7fca08070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:47:23.952 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.955+0000 7fca20c00700 1 -- 172.21.15.153:0/3588266508 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fca1408e480 con 0x7fca24100dc0 2024-03-21T10:47:23.952 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.955+0000 7fca1aa00700 1 --2- 172.21.15.153:0/3588266508 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fca0806e460 0x7fca08070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:47:23.952 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.956+0000 7fca1aa00700 1 --2- 172.21.15.153:0/3588266508 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fca0806e460 0x7fca08070920 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fca240fce40 tx=0x7fca0c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:47:23.956 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:23.959+0000 7fca20c00700 1 -- 172.21.15.153:0/3588266508 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fca1405b050 con 0x7fca24100dc0 2024-03-21T10:47:24.187 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:24.190+0000 7fca2a600700 1 -- 172.21.15.153:0/3588266508 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fca24199310 con 0x7fca0806e460 2024-03-21T10:47:24.189 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:24.192+0000 7fca20c00700 1 -- 172.21.15.153:0/3588266508 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fca24199310 con 0x7fca0806e460 2024-03-21T10:47:24.190 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:47:24.190 INFO:teuthology.orchestra.run.smithi153.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232", 2024-03-21T10:47:24.190 INFO:teuthology.orchestra.run.smithi153.stdout: "in_progress": true, 2024-03-21T10:47:24.190 INFO:teuthology.orchestra.run.smithi153.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-21T10:47:24.190 INFO:teuthology.orchestra.run.smithi153.stdout: "services_complete": [ 2024-03-21T10:47:24.190 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr", 2024-03-21T10:47:24.190 INFO:teuthology.orchestra.run.smithi153.stdout: "mon", 2024-03-21T10:47:24.190 INFO:teuthology.orchestra.run.smithi153.stdout: "mds", 2024-03-21T10:47:24.190 INFO:teuthology.orchestra.run.smithi153.stdout: "crash", 2024-03-21T10:47:24.190 INFO:teuthology.orchestra.run.smithi153.stdout: "osd" 2024-03-21T10:47:24.190 INFO:teuthology.orchestra.run.smithi153.stdout: ], 2024-03-21T10:47:24.191 INFO:teuthology.orchestra.run.smithi153.stdout: "progress": "16/23 daemons upgraded", 2024-03-21T10:47:24.191 INFO:teuthology.orchestra.run.smithi153.stdout: "message": "Currently upgrading mds daemons", 2024-03-21T10:47:24.191 INFO:teuthology.orchestra.run.smithi153.stdout: "is_paused": false 2024-03-21T10:47:24.191 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:47:24.192 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:24.195+0000 7fca2a600700 1 -- 172.21.15.153:0/3588266508 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fca0806e460 msgr2=0x7fca08070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:47:24.192 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:24.195+0000 7fca2a600700 1 --2- 172.21.15.153:0/3588266508 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fca0806e460 0x7fca08070920 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fca240fce40 tx=0x7fca0c01a040 comp rx=0 tx=0).stop 2024-03-21T10:47:24.192 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:24.195+0000 7fca2a600700 1 -- 172.21.15.153:0/3588266508 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca24100dc0 msgr2=0x7fca2419e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:47:24.192 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:24.195+0000 7fca2a600700 1 --2- 172.21.15.153:0/3588266508 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca24100dc0 0x7fca2419e780 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fca1400d8d0 tx=0x7fca1400dc90 comp rx=0 tx=0).stop 2024-03-21T10:47:24.192 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:24.195+0000 7fca2a600700 1 -- 172.21.15.153:0/3588266508 shutdown_connections 2024-03-21T10:47:24.192 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:24.195+0000 7fca2a600700 1 --2- 172.21.15.153:0/3588266508 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fca0806e460 0x7fca08070920 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:47:24.193 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:24.195+0000 7fca2a600700 1 --2- 172.21.15.153:0/3588266508 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fca24100dc0 0x7fca2419e780 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:47:24.193 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:24.195+0000 7fca2a600700 1 --2- 172.21.15.153:0/3588266508 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fca24101770 0x7fca2419ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:47:24.193 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:24.195+0000 7fca2a600700 1 -- 172.21.15.153:0/3588266508 >> 172.21.15.153:0/3588266508 conn(0x7fca240fc670 msgr2=0x7fca240fe9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:47:24.193 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:24.195+0000 7fca2a600700 1 -- 172.21.15.153:0/3588266508 shutdown_connections 2024-03-21T10:47:24.193 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:24.195+0000 7fca2a600700 1 -- 172.21.15.153:0/3588266508 wait complete. 2024-03-21T10:47:24.342 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:24.344+0000 7f34e3a00700 1 -- 172.21.15.153:0/3571102545 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f34dc10d280 msgr2=0x7f34dc10d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:47:24.342 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:24.344+0000 7f34e3a00700 1 --2- 172.21.15.153:0/3571102545 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f34dc10d280 0x7f34dc10d660 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f34d0009910 tx=0x7f34d0009c20 comp rx=0 tx=0).stop 2024-03-21T10:47:24.342 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:24.345+0000 7f34e3a00700 1 -- 172.21.15.153:0/3571102545 shutdown_connections 2024-03-21T10:47:24.342 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:24.345+0000 7f34e3a00700 1 --2- 172.21.15.153:0/3571102545 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f34dc1071d0 0x7f34dc107650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:47:24.342 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:24.345+0000 7f34e3a00700 1 --2- 172.21.15.153:0/3571102545 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f34dc10d280 0x7f34dc10d660 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:47:24.342 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:24.345+0000 7f34e3a00700 1 -- 172.21.15.153:0/3571102545 >> 172.21.15.153:0/3571102545 conn(0x7f34dc0769e0 msgr2=0x7f34dc078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:47:24.343 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:47:24 smithi153 ceph-mon[113733]: from='client.35890 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:47:24.343 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:47:24 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/196863852' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:47:24.343 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:47:24 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/47894971' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:47:24.344 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:24.345+0000 7f34e3a00700 1 -- 172.21.15.153:0/3571102545 shutdown_connections 2024-03-21T10:47:24.344 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:24.345+0000 7f34e3a00700 1 -- 172.21.15.153:0/3571102545 wait complete. 2024-03-21T10:47:24.345 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:24.346+0000 7f34e3a00700 1 Processor -- start 2024-03-21T10:47:24.345 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:24.346+0000 7f34e3a00700 1 -- start start 2024-03-21T10:47:24.345 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:24.346+0000 7f34e3a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f34dc1071d0 0x7f34dc19ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:47:24.345 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:24.346+0000 7f34e3a00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f34dc10d280 0x7f34dc19d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:47:24.345 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:24.347+0000 7f34e3a00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34dc19d8a0 con 0x7f34dc1071d0 2024-03-21T10:47:24.345 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:24.347+0000 7f34e3a00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34dc1a05d0 con 0x7f34dc10d280 2024-03-21T10:47:24.345 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:24.347+0000 7f34e1000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f34dc1071d0 0x7f34dc19ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:47:24.345 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:24.347+0000 7f34e1000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f34dc1071d0 0x7f34dc19ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39852/0 (socket says 172.21.15.153:39852) 2024-03-21T10:47:24.345 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:24.347+0000 7f34e1000700 1 -- 172.21.15.153:0/1706325237 learned_addr learned my addr 172.21.15.153:0/1706325237 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:47:24.345 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:24.347+0000 7f34e1000700 1 -- 172.21.15.153:0/1706325237 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f34dc10d280 msgr2=0x7f34dc19d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:47:24.345 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:24.347+0000 7f34dbe00700 1 --2- 172.21.15.153:0/1706325237 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f34dc10d280 0x7f34dc19d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:47:24.345 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:24.347+0000 7f34e1000700 1 --2- 172.21.15.153:0/1706325237 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f34dc10d280 0x7f34dc19d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:47:24.345 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:24.347+0000 7f34e1000700 1 -- 172.21.15.153:0/1706325237 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34d00095f0 con 0x7f34dc1071d0 2024-03-21T10:47:24.345 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:24.347+0000 7f34dbe00700 1 --2- 172.21.15.153:0/1706325237 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f34dc10d280 0x7f34dc19d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:47:24.345 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:24.348+0000 7f34e1000700 1 --2- 172.21.15.153:0/1706325237 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f34dc1071d0 0x7f34dc19ccd0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f34d00092e0 tx=0x7f34d000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:47:24.346 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:24.348+0000 7f34d9600700 1 -- 172.21.15.153:0/1706325237 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f34d001d070 con 0x7f34dc1071d0 2024-03-21T10:47:24.346 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:24.348+0000 7f34d9600700 1 -- 172.21.15.153:0/1706325237 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f34d0005e00 con 0x7f34dc1071d0 2024-03-21T10:47:24.346 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:24.348+0000 7f34e3a00700 1 -- 172.21.15.153:0/1706325237 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34dc1a0850 con 0x7f34dc1071d0 2024-03-21T10:47:24.346 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:24.348+0000 7f34d9600700 1 -- 172.21.15.153:0/1706325237 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f34d0017400 con 0x7f34dc1071d0 2024-03-21T10:47:24.347 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:24.348+0000 7f34e3a00700 1 -- 172.21.15.153:0/1706325237 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34dc1a0dc0 con 0x7f34dc1071d0 2024-03-21T10:47:24.347 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:24.349+0000 7f34e3a00700 1 -- 172.21.15.153:0/1706325237 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34dc04ea90 con 0x7f34dc1071d0 2024-03-21T10:47:24.347 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:24.350+0000 7f34d9600700 1 -- 172.21.15.153:0/1706325237 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f34d000f460 con 0x7f34dc1071d0 2024-03-21T10:47:24.348 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:24.351+0000 7f34d9600700 1 --2- 172.21.15.153:0/1706325237 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f34c406e460 0x7f34c4070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:47:24.348 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:24.351+0000 7f34d9600700 1 -- 172.21.15.153:0/1706325237 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f34d008f9b0 con 0x7f34dc1071d0 2024-03-21T10:47:24.348 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:24.351+0000 7f34dbe00700 1 --2- 172.21.15.153:0/1706325237 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f34c406e460 0x7f34c4070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:47:24.349 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:24.352+0000 7f34dbe00700 1 --2- 172.21.15.153:0/1706325237 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f34c406e460 0x7f34c4070920 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f34c8005fd0 tx=0x7f34c8005ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:47:24.352 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:24.356+0000 7f34d9600700 1 -- 172.21.15.153:0/1706325237 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f34d0058a50 con 0x7f34dc1071d0 2024-03-21T10:47:24.447 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:47:24 smithi174 ceph-mon[88004]: from='client.35890 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:47:24.447 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:47:24 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/196863852' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:47:24.447 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:47:24 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/47894971' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:47:24.644 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:24.647+0000 7f34e3a00700 1 -- 172.21.15.153:0/1706325237 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f34dc1a1140 con 0x7f34dc1071d0 2024-03-21T10:47:24.645 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:24.648+0000 7f34d9600700 1 -- 172.21.15.153:0/1706325237 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4493 (secure 0 0 0) 0x7f34d005c110 con 0x7f34dc1071d0 2024-03-21T10:47:24.646 INFO:teuthology.orchestra.run.smithi153.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-03-21T10:47:24.646 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-03-21T10:47:24.646 INFO:teuthology.orchestra.run.smithi153.stdout: host smithi174 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi174/config 2024-03-21T10:47:24.647 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/podman: timeout after 895 seconds 2024-03-21T10:47:24.647 INFO:teuthology.orchestra.run.smithi153.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 -e NODE_NAME=smithi174 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/run/ceph:z -v /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpd910r3_i:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 inventory --format=json-pretty --filter-for-batch 2024-03-21T10:47:24.647 INFO:teuthology.orchestra.run.smithi153.stdout:Traceback (most recent call last): 2024-03-21T10:47:24.647 INFO:teuthology.orchestra.run.smithi153.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-03-21T10:47:24.647 INFO:teuthology.orchestra.run.smithi153.stdout: return _run_code(code, main_globals, None, 2024-03-21T10:47:24.647 INFO:teuthology.orchestra.run.smithi153.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-03-21T10:47:24.647 INFO:teuthology.orchestra.run.smithi153.stdout: exec(code, run_globals) 2024-03-21T10:47:24.647 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 5606, in 2024-03-21T10:47:24.647 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 5594, in main 2024-03-21T10:47:24.647 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 406, in _infer_config 2024-03-21T10:47:24.647 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 321, in _infer_fsid 2024-03-21T10:47:24.647 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 434, in _infer_image 2024-03-21T10:47:24.647 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 308, in _validate_fsid 2024-03-21T10:47:24.647 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 3357, in command_ceph_volume 2024-03-21T10:47:24.648 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-03-21T10:47:24.648 INFO:teuthology.orchestra.run.smithi153.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 -e NODE_NAME=smithi174 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/run/ceph:z -v /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpd910r3_i:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 inventory --format=json-pretty --filter-for-batch 2024-03-21T10:47:24.648 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-21T10:47:24.648 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is degraded 2024-03-21T10:47:24.648 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-21T10:47:24.648 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-21T10:47:24.648 INFO:teuthology.orchestra.run.smithi153.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-21T10:47:24.648 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-21T10:47:24.649 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:24.651+0000 7f34e3a00700 1 -- 172.21.15.153:0/1706325237 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f34c406e460 msgr2=0x7f34c4070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:47:24.649 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:24.651+0000 7f34e3a00700 1 --2- 172.21.15.153:0/1706325237 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f34c406e460 0x7f34c4070920 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f34c8005fd0 tx=0x7f34c8005ee0 comp rx=0 tx=0).stop 2024-03-21T10:47:24.649 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:24.651+0000 7f34e3a00700 1 -- 172.21.15.153:0/1706325237 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f34dc1071d0 msgr2=0x7f34dc19ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:47:24.649 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:24.651+0000 7f34e3a00700 1 --2- 172.21.15.153:0/1706325237 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f34dc1071d0 0x7f34dc19ccd0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f34d00092e0 tx=0x7f34d000be20 comp rx=0 tx=0).stop 2024-03-21T10:47:24.649 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:24.651+0000 7f34e3a00700 1 -- 172.21.15.153:0/1706325237 shutdown_connections 2024-03-21T10:47:24.649 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:24.651+0000 7f34e3a00700 1 --2- 172.21.15.153:0/1706325237 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f34c406e460 0x7f34c4070920 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:47:24.649 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:24.651+0000 7f34e3a00700 1 --2- 172.21.15.153:0/1706325237 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f34dc1071d0 0x7f34dc19ccd0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:47:24.649 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:24.651+0000 7f34e3a00700 1 --2- 172.21.15.153:0/1706325237 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f34dc10d280 0x7f34dc19d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:47:24.650 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:24.651+0000 7f34e3a00700 1 -- 172.21.15.153:0/1706325237 >> 172.21.15.153:0/1706325237 conn(0x7f34dc0769e0 msgr2=0x7f34dc078320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:47:24.650 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:24.652+0000 7f34e3a00700 1 -- 172.21.15.153:0/1706325237 shutdown_connections 2024-03-21T10:47:24.650 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:24.652+0000 7f34e3a00700 1 -- 172.21.15.153:0/1706325237 wait complete. 2024-03-21T10:47:25.447 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:47:25 smithi174 ceph-mon[88004]: pgmap v1068: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:47:25.447 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:47:25 smithi174 ceph-mon[88004]: from='client.35902 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:47:25.447 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:47:25 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/1706325237' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:47:25.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:47:25 smithi153 ceph-mon[113733]: pgmap v1068: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:47:25.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:47:25 smithi153 ceph-mon[113733]: from='client.35902 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:47:25.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:47:25 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/1706325237' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:47:27.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:47:27 smithi153 ceph-mon[113733]: pgmap v1069: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:47:27.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:47:27 smithi174 ceph-mon[88004]: pgmap v1069: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:47:29.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:47:29 smithi153 ceph-mon[113733]: pgmap v1070: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:47:29.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:47:29 smithi174 ceph-mon[88004]: pgmap v1070: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:47:31.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:47:31 smithi153 ceph-mon[113733]: pgmap v1071: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:47:31.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:47:31 smithi174 ceph-mon[88004]: pgmap v1071: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:47:32.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:47:32 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:47:32.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:47:32 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:47:33.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:47:33 smithi153 ceph-mon[113733]: pgmap v1072: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:47:33.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:47:33 smithi174 ceph-mon[88004]: pgmap v1072: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:47:35.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:47:35 smithi153 ceph-mon[113733]: pgmap v1073: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:47:35.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:47:35 smithi174 ceph-mon[88004]: pgmap v1073: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:47:37.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:47:37 smithi153 ceph-mon[113733]: pgmap v1074: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:47:37.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:47:37 smithi174 ceph-mon[88004]: pgmap v1074: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:47:39.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:47:39 smithi153 ceph-mon[113733]: pgmap v1075: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:47:39.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:47:39 smithi174 ceph-mon[88004]: pgmap v1075: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:47:41.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:47:41 smithi174 ceph-mon[88004]: pgmap v1076: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:47:41.801 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:47:41 smithi153 ceph-mon[113733]: pgmap v1076: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:47:43.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:47:43 smithi174 ceph-mon[88004]: pgmap v1077: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:47:43.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:47:43 smithi153 ceph-mon[113733]: pgmap v1077: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:47:45.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:47:45 smithi174 ceph-mon[88004]: pgmap v1078: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:47:45.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:47:45 smithi153 ceph-mon[113733]: pgmap v1078: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:47:46.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:47:46 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:47:46.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:47:46 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:47:47.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:47:47 smithi174 ceph-mon[88004]: pgmap v1079: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:47:47.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:47:47 smithi153 ceph-mon[113733]: pgmap v1079: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:47:49.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:47:49 smithi174 ceph-mon[88004]: pgmap v1080: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:47:49.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:47:49 smithi153 ceph-mon[113733]: pgmap v1080: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:47:51.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:47:51 smithi174 ceph-mon[88004]: pgmap v1081: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:47:51.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:47:51 smithi153 ceph-mon[113733]: pgmap v1081: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:47:53.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:47:53 smithi174 ceph-mon[88004]: pgmap v1082: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:47:53.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:47:53 smithi153 ceph-mon[113733]: pgmap v1082: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:47:54.818 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:54.820+0000 7f35ee600700 1 -- 172.21.15.153:0/4115998000 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f35e80731c0 msgr2=0x7f35e80735a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:47:54.818 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:54.820+0000 7f35ee600700 1 --2- 172.21.15.153:0/4115998000 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f35e80731c0 0x7f35e80735a0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f35dc009940 tx=0x7f35dc009c50 comp rx=0 tx=0).stop 2024-03-21T10:47:54.818 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:54.820+0000 7f35ee600700 1 -- 172.21.15.153:0/4115998000 shutdown_connections 2024-03-21T10:47:54.818 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:54.820+0000 7f35ee600700 1 --2- 172.21.15.153:0/4115998000 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f35e8073b70 0x7f35e810ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:47:54.818 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:54.820+0000 7f35ee600700 1 --2- 172.21.15.153:0/4115998000 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f35e80731c0 0x7f35e80735a0 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:47:54.818 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:54.820+0000 7f35ee600700 1 -- 172.21.15.153:0/4115998000 >> 172.21.15.153:0/4115998000 conn(0x7f35e80fc4c0 msgr2=0x7f35e80fe8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:47:54.818 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:54.821+0000 7f35ee600700 1 -- 172.21.15.153:0/4115998000 shutdown_connections 2024-03-21T10:47:54.818 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:54.821+0000 7f35ee600700 1 -- 172.21.15.153:0/4115998000 wait complete. 2024-03-21T10:47:54.819 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:54.822+0000 7f35ee600700 1 Processor -- start 2024-03-21T10:47:54.819 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:54.822+0000 7f35ee600700 1 -- start start 2024-03-21T10:47:54.819 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:54.822+0000 7f35ee600700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f35e80731c0 0x7f35e819e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:47:54.819 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:54.822+0000 7f35ee600700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f35e8073b70 0x7f35e819ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:47:54.819 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:54.822+0000 7f35ee600700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35e819f0e0 con 0x7f35e80731c0 2024-03-21T10:47:54.819 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:54.822+0000 7f35ee600700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35e8198590 con 0x7f35e8073b70 2024-03-21T10:47:54.821 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:54.823+0000 7f35ed200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f35e80731c0 0x7f35e819e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:47:54.821 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:54.823+0000 7f35ed200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f35e80731c0 0x7f35e819e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:57778/0 (socket says 172.21.15.153:57778) 2024-03-21T10:47:54.821 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:54.823+0000 7f35ed200700 1 -- 172.21.15.153:0/1123015279 learned_addr learned my addr 172.21.15.153:0/1123015279 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:47:54.821 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:54.823+0000 7f35e7e00700 1 --2- 172.21.15.153:0/1123015279 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f35e8073b70 0x7f35e819ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:47:54.821 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:54.823+0000 7f35ed200700 1 -- 172.21.15.153:0/1123015279 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f35e8073b70 msgr2=0x7f35e819ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:47:54.821 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:54.823+0000 7f35ed200700 1 --2- 172.21.15.153:0/1123015279 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f35e8073b70 0x7f35e819ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:47:54.821 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:54.823+0000 7f35ed200700 1 -- 172.21.15.153:0/1123015279 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35dc0095f0 con 0x7f35e80731c0 2024-03-21T10:47:54.821 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:54.824+0000 7f35ed200700 1 --2- 172.21.15.153:0/1123015279 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f35e80731c0 0x7f35e819e510 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f35dc0055f0 tx=0x7f35dc0056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:47:54.821 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:54.824+0000 7f35e5600700 1 -- 172.21.15.153:0/1123015279 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f35dc01d070 con 0x7f35e80731c0 2024-03-21T10:47:54.822 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:54.824+0000 7f35e5600700 1 -- 172.21.15.153:0/1123015279 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f35dc00f460 con 0x7f35e80731c0 2024-03-21T10:47:54.822 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:54.824+0000 7f35e5600700 1 -- 172.21.15.153:0/1123015279 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f35dc021620 con 0x7f35e80731c0 2024-03-21T10:47:54.822 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:54.824+0000 7f35ee600700 1 -- 172.21.15.153:0/1123015279 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35e8198870 con 0x7f35e80731c0 2024-03-21T10:47:54.822 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:54.824+0000 7f35ee600700 1 -- 172.21.15.153:0/1123015279 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35e8198dc0 con 0x7f35e80731c0 2024-03-21T10:47:54.822 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:54.825+0000 7f35ee600700 1 -- 172.21.15.153:0/1123015279 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35e810a3b0 con 0x7f35e80731c0 2024-03-21T10:47:54.823 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:54.826+0000 7f35e5600700 1 -- 172.21.15.153:0/1123015279 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f35dc00fa60 con 0x7f35e80731c0 2024-03-21T10:47:54.824 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:54.827+0000 7f35e5600700 1 --2- 172.21.15.153:0/1123015279 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f35d006e460 0x7f35d0070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:47:54.824 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:54.827+0000 7f35e5600700 1 -- 172.21.15.153:0/1123015279 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f35dc08f7e0 con 0x7f35e80731c0 2024-03-21T10:47:54.824 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:54.827+0000 7f35e7e00700 1 --2- 172.21.15.153:0/1123015279 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f35d006e460 0x7f35d0070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:47:54.825 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:54.828+0000 7f35e7e00700 1 --2- 172.21.15.153:0/1123015279 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f35d006e460 0x7f35d0070920 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f35e80fdba0 tx=0x7f35d4005eb0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:47:54.829 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:54.832+0000 7f35e5600700 1 -- 172.21.15.153:0/1123015279 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f35dc05c3b0 con 0x7f35e80731c0 2024-03-21T10:47:55.072 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:55.075+0000 7f35ee600700 1 -- 172.21.15.153:0/1123015279 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f35e81990f0 con 0x7f35d006e460 2024-03-21T10:47:55.074 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:55.077+0000 7f35e5600700 1 -- 172.21.15.153:0/1123015279 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f35e81990f0 con 0x7f35d006e460 2024-03-21T10:47:55.077 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:55.080+0000 7f35ee600700 1 -- 172.21.15.153:0/1123015279 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f35d006e460 msgr2=0x7f35d0070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:47:55.077 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:55.080+0000 7f35ee600700 1 --2- 172.21.15.153:0/1123015279 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f35d006e460 0x7f35d0070920 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f35e80fdba0 tx=0x7f35d4005eb0 comp rx=0 tx=0).stop 2024-03-21T10:47:55.077 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:55.080+0000 7f35ee600700 1 -- 172.21.15.153:0/1123015279 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f35e80731c0 msgr2=0x7f35e819e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:47:55.077 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:55.080+0000 7f35ee600700 1 --2- 172.21.15.153:0/1123015279 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f35e80731c0 0x7f35e819e510 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f35dc0055f0 tx=0x7f35dc0056d0 comp rx=0 tx=0).stop 2024-03-21T10:47:55.078 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:55.080+0000 7f35ee600700 1 -- 172.21.15.153:0/1123015279 shutdown_connections 2024-03-21T10:47:55.078 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:55.080+0000 7f35ee600700 1 --2- 172.21.15.153:0/1123015279 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f35d006e460 0x7f35d0070920 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:47:55.078 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:55.080+0000 7f35ee600700 1 --2- 172.21.15.153:0/1123015279 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f35e80731c0 0x7f35e819e510 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:47:55.078 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:55.080+0000 7f35ee600700 1 --2- 172.21.15.153:0/1123015279 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f35e8073b70 0x7f35e819ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:47:55.078 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:55.080+0000 7f35ee600700 1 -- 172.21.15.153:0/1123015279 >> 172.21.15.153:0/1123015279 conn(0x7f35e80fc4c0 msgr2=0x7f35e81074f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:47:55.078 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:55.081+0000 7f35ee600700 1 -- 172.21.15.153:0/1123015279 shutdown_connections 2024-03-21T10:47:55.078 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:55.081+0000 7f35ee600700 1 -- 172.21.15.153:0/1123015279 wait complete. 2024-03-21T10:47:55.093 INFO:teuthology.orchestra.run.smithi153.stdout:true 2024-03-21T10:47:55.233 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:55.235+0000 7f485ca00700 1 -- 172.21.15.153:0/3617141670 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4858073160 msgr2=0x7f4858073540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:47:55.233 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:55.235+0000 7f485ca00700 1 --2- 172.21.15.153:0/3617141670 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4858073160 0x7f4858073540 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f4848009910 tx=0x7f4848009c20 comp rx=0 tx=0).stop 2024-03-21T10:47:55.233 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:55.235+0000 7f485ca00700 1 -- 172.21.15.153:0/3617141670 shutdown_connections 2024-03-21T10:47:55.233 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:55.235+0000 7f485ca00700 1 --2- 172.21.15.153:0/3617141670 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4858073a80 0x7f485810cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:47:55.233 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:55.235+0000 7f485ca00700 1 --2- 172.21.15.153:0/3617141670 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4858073160 0x7f4858073540 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:47:55.233 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:55.235+0000 7f485ca00700 1 -- 172.21.15.153:0/3617141670 >> 172.21.15.153:0/3617141670 conn(0x7f48580fc650 msgr2=0x7f48580fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:47:55.233 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:55.236+0000 7f485ca00700 1 -- 172.21.15.153:0/3617141670 shutdown_connections 2024-03-21T10:47:55.233 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:55.236+0000 7f485ca00700 1 -- 172.21.15.153:0/3617141670 wait complete. 2024-03-21T10:47:55.235 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:55.236+0000 7f485ca00700 1 Processor -- start 2024-03-21T10:47:55.235 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:55.237+0000 7f485ca00700 1 -- start start 2024-03-21T10:47:55.235 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:55.237+0000 7f485ca00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4858073160 0x7f4858198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:47:55.235 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:55.237+0000 7f485ca00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4858073a80 0x7f4858198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:47:55.235 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:55.237+0000 7f485ca00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4858199530 con 0x7f4858073a80 2024-03-21T10:47:55.235 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:55.237+0000 7f485ca00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f485819d340 con 0x7f4858073160 2024-03-21T10:47:55.235 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:55.237+0000 7f4855400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4858073a80 0x7f4858198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:47:55.235 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:55.237+0000 7f4855400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4858073a80 0x7f4858198ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:57782/0 (socket says 172.21.15.153:57782) 2024-03-21T10:47:55.235 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:55.237+0000 7f4855400700 1 -- 172.21.15.153:0/1971747952 learned_addr learned my addr 172.21.15.153:0/1971747952 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:47:55.235 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:55.238+0000 7f4855e00700 1 --2- 172.21.15.153:0/1971747952 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4858073160 0x7f4858198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:47:55.236 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:55.238+0000 7f4855400700 1 -- 172.21.15.153:0/1971747952 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4858073160 msgr2=0x7f4858198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:47:55.236 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:55.238+0000 7f4855400700 1 --2- 172.21.15.153:0/1971747952 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4858073160 0x7f4858198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:47:55.236 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:55.238+0000 7f4855400700 1 -- 172.21.15.153:0/1971747952 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48480095f0 con 0x7f4858073a80 2024-03-21T10:47:55.236 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:55.238+0000 7f4855400700 1 --2- 172.21.15.153:0/1971747952 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4858073a80 0x7f4858198ea0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f484400c8a0 tx=0x7f484400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:47:55.236 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:55.238+0000 7f484ea00700 1 -- 172.21.15.153:0/1971747952 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f484400cea0 con 0x7f4858073a80 2024-03-21T10:47:55.237 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:55.238+0000 7f485ca00700 1 -- 172.21.15.153:0/1971747952 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f485819d620 con 0x7f4858073a80 2024-03-21T10:47:55.237 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:55.238+0000 7f484ea00700 1 -- 172.21.15.153:0/1971747952 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4844004830 con 0x7f4858073a80 2024-03-21T10:47:55.237 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:55.238+0000 7f484ea00700 1 -- 172.21.15.153:0/1971747952 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f48440055b0 con 0x7f4858073a80 2024-03-21T10:47:55.237 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:55.239+0000 7f485ca00700 1 -- 172.21.15.153:0/1971747952 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f485819db70 con 0x7f4858073a80 2024-03-21T10:47:55.237 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:55.240+0000 7f485ca00700 1 -- 172.21.15.153:0/1971747952 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f485810a5c0 con 0x7f4858073a80 2024-03-21T10:47:55.237 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:55.241+0000 7f484ea00700 1 -- 172.21.15.153:0/1971747952 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f48440049a0 con 0x7f4858073a80 2024-03-21T10:47:55.238 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:55.242+0000 7f484ea00700 1 --2- 172.21.15.153:0/1971747952 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f483c06e460 0x7f483c070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:47:55.239 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:55.242+0000 7f484ea00700 1 -- 172.21.15.153:0/1971747952 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f484408e0d0 con 0x7f4858073a80 2024-03-21T10:47:55.239 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:55.242+0000 7f4855e00700 1 --2- 172.21.15.153:0/1971747952 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f483c06e460 0x7f483c070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:47:55.239 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:55.243+0000 7f4855e00700 1 --2- 172.21.15.153:0/1971747952 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f483c06e460 0x7f483c070920 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f484800b5c0 tx=0x7f484801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:47:55.243 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:55.246+0000 7f484ea00700 1 -- 172.21.15.153:0/1971747952 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f484405ad40 con 0x7f4858073a80 2024-03-21T10:47:55.485 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:55.488+0000 7f485ca00700 1 -- 172.21.15.153:0/1971747952 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f485819de40 con 0x7f483c06e460 2024-03-21T10:47:55.486 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:55.489+0000 7f484ea00700 1 -- 172.21.15.153:0/1971747952 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f485819de40 con 0x7f483c06e460 2024-03-21T10:47:55.489 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:55.492+0000 7f485ca00700 1 -- 172.21.15.153:0/1971747952 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f483c06e460 msgr2=0x7f483c070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:47:55.489 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:55.492+0000 7f485ca00700 1 --2- 172.21.15.153:0/1971747952 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f483c06e460 0x7f483c070920 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f484800b5c0 tx=0x7f484801a040 comp rx=0 tx=0).stop 2024-03-21T10:47:55.490 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:55.492+0000 7f485ca00700 1 -- 172.21.15.153:0/1971747952 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4858073a80 msgr2=0x7f4858198ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:47:55.490 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:55.492+0000 7f485ca00700 1 --2- 172.21.15.153:0/1971747952 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4858073a80 0x7f4858198ea0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f484400c8a0 tx=0x7f484400cc60 comp rx=0 tx=0).stop 2024-03-21T10:47:55.490 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:55.492+0000 7f485ca00700 1 -- 172.21.15.153:0/1971747952 shutdown_connections 2024-03-21T10:47:55.490 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:55.492+0000 7f485ca00700 1 --2- 172.21.15.153:0/1971747952 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4858073160 0x7f4858198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:47:55.490 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:55.492+0000 7f485ca00700 1 --2- 172.21.15.153:0/1971747952 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f483c06e460 0x7f483c070920 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:47:55.490 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:55.492+0000 7f485ca00700 1 --2- 172.21.15.153:0/1971747952 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4858073a80 0x7f4858198ea0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:47:55.490 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:55.492+0000 7f485ca00700 1 -- 172.21.15.153:0/1971747952 >> 172.21.15.153:0/1971747952 conn(0x7f48580fc650 msgr2=0x7f4858107700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:47:55.490 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:55.492+0000 7f485ca00700 1 -- 172.21.15.153:0/1971747952 shutdown_connections 2024-03-21T10:47:55.490 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:55.492+0000 7f485ca00700 1 -- 172.21.15.153:0/1971747952 wait complete. 2024-03-21T10:47:55.629 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:47:55 smithi174 ceph-mon[88004]: pgmap v1083: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:47:55.647 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:55.650+0000 7fe6bf800700 1 -- 172.21.15.153:0/711863418 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6b80ffa40 msgr2=0x7fe6b80ffec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:47:55.647 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:55.650+0000 7fe6bf800700 1 --2- 172.21.15.153:0/711863418 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6b80ffa40 0x7fe6b80ffec0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fe6a4009910 tx=0x7fe6a4009c20 comp rx=0 tx=0).stop 2024-03-21T10:47:55.647 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:55.650+0000 7fe6bf800700 1 -- 172.21.15.153:0/711863418 shutdown_connections 2024-03-21T10:47:55.647 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:55.650+0000 7fe6bf800700 1 --2- 172.21.15.153:0/711863418 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6b80ffa40 0x7fe6b80ffec0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:47:55.647 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:55.650+0000 7fe6bf800700 1 --2- 172.21.15.153:0/711863418 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe6b8105a40 0x7fe6b8105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:47:55.647 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:55.650+0000 7fe6bf800700 1 -- 172.21.15.153:0/711863418 >> 172.21.15.153:0/711863418 conn(0x7fe6b80fb570 msgr2=0x7fe6b80fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:47:55.648 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:47:55 smithi153 ceph-mon[113733]: pgmap v1083: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:47:55.648 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:55.650+0000 7fe6bf800700 1 -- 172.21.15.153:0/711863418 shutdown_connections 2024-03-21T10:47:55.648 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:55.650+0000 7fe6bf800700 1 -- 172.21.15.153:0/711863418 wait complete. 2024-03-21T10:47:55.648 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:55.652+0000 7fe6bf800700 1 Processor -- start 2024-03-21T10:47:55.649 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:55.652+0000 7fe6bf800700 1 -- start start 2024-03-21T10:47:55.650 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:55.653+0000 7fe6bf800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6b80ffa40 0x7fe6b80fbe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:47:55.650 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:55.653+0000 7fe6bf800700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe6b8105a40 0x7fe6b80fc3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:47:55.650 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:55.653+0000 7fe6bf800700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b80fc910 con 0x7fe6b80ffa40 2024-03-21T10:47:55.650 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:55.653+0000 7fe6bf800700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b80fca80 con 0x7fe6b8105a40 2024-03-21T10:47:55.650 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:55.653+0000 7fe6bce00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6b80ffa40 0x7fe6b80fbe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:47:55.650 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:55.653+0000 7fe6bce00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6b80ffa40 0x7fe6b80fbe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:57796/0 (socket says 172.21.15.153:57796) 2024-03-21T10:47:55.650 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:55.653+0000 7fe6bce00700 1 -- 172.21.15.153:0/3877361136 learned_addr learned my addr 172.21.15.153:0/3877361136 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:47:55.651 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:55.653+0000 7fe6b7e00700 1 --2- 172.21.15.153:0/3877361136 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe6b8105a40 0x7fe6b80fc3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:47:55.651 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:55.653+0000 7fe6bce00700 1 -- 172.21.15.153:0/3877361136 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe6b8105a40 msgr2=0x7fe6b80fc3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:47:55.651 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:55.653+0000 7fe6bce00700 1 --2- 172.21.15.153:0/3877361136 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe6b8105a40 0x7fe6b80fc3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:47:55.651 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:55.653+0000 7fe6bce00700 1 -- 172.21.15.153:0/3877361136 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6a40095f0 con 0x7fe6b80ffa40 2024-03-21T10:47:55.651 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:55.653+0000 7fe6b7e00700 1 --2- 172.21.15.153:0/3877361136 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe6b8105a40 0x7fe6b80fc3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:47:55.652 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:55.654+0000 7fe6bce00700 1 --2- 172.21.15.153:0/3877361136 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6b80ffa40 0x7fe6b80fbe90 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac00c930 tx=0x7fe6ac00ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:47:55.652 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:55.654+0000 7fe6b5600700 1 -- 172.21.15.153:0/3877361136 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe6ac007ad0 con 0x7fe6b80ffa40 2024-03-21T10:47:55.652 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:55.654+0000 7fe6b5600700 1 -- 172.21.15.153:0/3877361136 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe6ac007c30 con 0x7fe6b80ffa40 2024-03-21T10:47:55.652 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:55.654+0000 7fe6b5600700 1 -- 172.21.15.153:0/3877361136 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe6ac018610 con 0x7fe6b80ffa40 2024-03-21T10:47:55.652 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:55.654+0000 7fe6bf800700 1 -- 172.21.15.153:0/3877361136 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6b80fcd00 con 0x7fe6b80ffa40 2024-03-21T10:47:55.652 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:55.654+0000 7fe6bf800700 1 -- 172.21.15.153:0/3877361136 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6b81a2c90 con 0x7fe6b80ffa40 2024-03-21T10:47:55.652 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:55.656+0000 7fe6bf800700 1 -- 172.21.15.153:0/3877361136 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6b8108fe0 con 0x7fe6b80ffa40 2024-03-21T10:47:55.653 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:55.657+0000 7fe6b5600700 1 -- 172.21.15.153:0/3877361136 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fe6ac00f450 con 0x7fe6b80ffa40 2024-03-21T10:47:55.654 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:55.657+0000 7fe6b5600700 1 --2- 172.21.15.153:0/3877361136 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe6a006e460 0x7fe6a0070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:47:55.654 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:55.658+0000 7fe6b5600700 1 -- 172.21.15.153:0/3877361136 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fe6ac08e5f0 con 0x7fe6b80ffa40 2024-03-21T10:47:55.655 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:55.658+0000 7fe6b7e00700 1 --2- 172.21.15.153:0/3877361136 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe6a006e460 0x7fe6a0070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:47:55.655 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:55.659+0000 7fe6b7e00700 1 --2- 172.21.15.153:0/3877361136 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe6a006e460 0x7fe6a0070920 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fe6a4006010 tx=0x7fe6a400b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:47:55.659 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:55.662+0000 7fe6b5600700 1 -- 172.21.15.153:0/3877361136 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe6ac057610 con 0x7fe6b80ffa40 2024-03-21T10:47:55.891 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:55.894+0000 7fe6bf800700 1 -- 172.21.15.153:0/3877361136 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fe6b80fcd00 con 0x7fe6a006e460 2024-03-21T10:47:55.908 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:55.911+0000 7fe6b5600700 1 -- 172.21.15.153:0/3877361136 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fe6b80fcd00 con 0x7fe6a006e460 2024-03-21T10:47:55.909 INFO:teuthology.orchestra.run.smithi153.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-21T10:47:55.909 INFO:teuthology.orchestra.run.smithi153.stdout:alertmanager.smithi153 smithi153 *:9093,9094 running (36m) 12m ago 50m 15.6M - 0.25.0 c8568f914cd2 9ed3f2200b5f 2024-03-21T10:47:55.909 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi153 smithi153 running (50m) 12m ago 50m 10.4M - 18.2.1 5be31c24972a 5b61994396a6 2024-03-21T10:47:55.909 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi174 smithi174 running (49m) 12m ago 49m 10.1M - 18.2.1 5be31c24972a 4f7a5fc3ac28 2024-03-21T10:47:55.909 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi153 smithi153 running (32m) 12m ago 50m 6656k - 19.0.0-2231-g2edfee68 237e43c7a65f 32c21104d5b0 2024-03-21T10:47:55.909 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi174 smithi174 running (32m) 12m ago 49m 6660k - 19.0.0-2231-g2edfee68 237e43c7a65f a04e48967ab6 2024-03-21T10:47:55.909 INFO:teuthology.orchestra.run.smithi153.stdout:grafana.smithi153 smithi153 *:3000 running (36m) 12m ago 49m 75.2M - 9.4.12 2bacad6d85d8 ee24026ab786 2024-03-21T10:47:55.910 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.iuyluj smithi153 running (28m) 12m ago 43m 18.7M - 19.0.0-2231-g2edfee68 237e43c7a65f 87aff8a5de70 2024-03-21T10:47:55.910 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.zipckr smithi153 running (28m) 12m ago 43m 18.9M - 19.0.0-2231-g2edfee68 237e43c7a65f ccc262bd5dd3 2024-03-21T10:47:55.910 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.nhozag smithi174 running (12m) 12m ago 43m 10.9M - 19.0.0-2231-g2edfee68 237e43c7a65f 9e02aea72c96 2024-03-21T10:47:55.910 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.yyimst smithi174 running (27m) 12m ago 43m 18.5M - 19.0.0-2231-g2edfee68 237e43c7a65f fc1962f9351e 2024-03-21T10:47:55.910 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi153.tluxni smithi153 *:8443,9283,8765 running (39m) 12m ago 52m 512M - 19.0.0-2231-g2edfee68 237e43c7a65f 7ffee0e55382 2024-03-21T10:47:55.910 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi174.qpcwat smithi174 *:8443,9283,8765 running (38m) 12m ago 48m 459M - 19.0.0-2231-g2edfee68 237e43c7a65f 748968f216ab 2024-03-21T10:47:55.910 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi153 smithi153 running (34m) 12m ago 52m 70.8M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f dd8038303dc3 2024-03-21T10:47:55.910 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi174 smithi174 running (33m) 12m ago 48m 51.9M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f 29d78a788f2c 2024-03-21T10:47:55.910 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi153 smithi153 *:9100 running (37m) 12m ago 50m 8140k - 1.5.0 0da6a335fe13 bb835edc0e0b 2024-03-21T10:47:55.910 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi174 smithi174 *:9100 running (37m) 12m ago 49m 8610k - 1.5.0 0da6a335fe13 7c4ecad790fa 2024-03-21T10:47:55.910 INFO:teuthology.orchestra.run.smithi153.stdout:osd.0 smithi153 running (32m) 12m ago 47m 118M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 507900fd4791 2024-03-21T10:47:55.910 INFO:teuthology.orchestra.run.smithi153.stdout:osd.1 smithi153 running (31m) 12m ago 47m 93.0M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 0d646452e9c1 2024-03-21T10:47:55.910 INFO:teuthology.orchestra.run.smithi153.stdout:osd.2 smithi153 running (31m) 12m ago 46m 91.2M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 1326f2afdfcd 2024-03-21T10:47:55.910 INFO:teuthology.orchestra.run.smithi153.stdout:osd.3 smithi174 running (30m) 12m ago 46m 87.5M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 70aafac71ad5 2024-03-21T10:47:55.911 INFO:teuthology.orchestra.run.smithi153.stdout:osd.4 smithi174 running (29m) 12m ago 45m 95.8M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 0d746ce6e79d 2024-03-21T10:47:55.911 INFO:teuthology.orchestra.run.smithi153.stdout:osd.5 smithi174 running (29m) 12m ago 45m 90.3M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 53e5f35b8cac 2024-03-21T10:47:55.911 INFO:teuthology.orchestra.run.smithi153.stdout:prometheus.smithi153 smithi153 *:9095 running (37m) 12m ago 49m 88.1M - 2.43.0 a07b618ecd1d 7d95561e50c2 2024-03-21T10:47:55.911 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:55.914+0000 7fe6bf800700 1 -- 172.21.15.153:0/3877361136 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe6a006e460 msgr2=0x7fe6a0070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:47:55.911 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:55.914+0000 7fe6bf800700 1 --2- 172.21.15.153:0/3877361136 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe6a006e460 0x7fe6a0070920 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fe6a4006010 tx=0x7fe6a400b540 comp rx=0 tx=0).stop 2024-03-21T10:47:55.911 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:55.914+0000 7fe6bf800700 1 -- 172.21.15.153:0/3877361136 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6b80ffa40 msgr2=0x7fe6b80fbe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:47:55.912 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:55.914+0000 7fe6bf800700 1 --2- 172.21.15.153:0/3877361136 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6b80ffa40 0x7fe6b80fbe90 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac00c930 tx=0x7fe6ac00ccf0 comp rx=0 tx=0).stop 2024-03-21T10:47:55.912 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:55.914+0000 7fe6bf800700 1 -- 172.21.15.153:0/3877361136 shutdown_connections 2024-03-21T10:47:55.912 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:55.914+0000 7fe6bf800700 1 --2- 172.21.15.153:0/3877361136 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe6a006e460 0x7fe6a0070920 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:47:55.912 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:55.914+0000 7fe6bf800700 1 --2- 172.21.15.153:0/3877361136 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe6b80ffa40 0x7fe6b80fbe90 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:47:55.912 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:55.914+0000 7fe6bf800700 1 --2- 172.21.15.153:0/3877361136 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe6b8105a40 0x7fe6b80fc3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:47:55.912 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:55.914+0000 7fe6bf800700 1 -- 172.21.15.153:0/3877361136 >> 172.21.15.153:0/3877361136 conn(0x7fe6b80fb570 msgr2=0x7fe6b8068f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:47:55.912 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:55.914+0000 7fe6bf800700 1 -- 172.21.15.153:0/3877361136 shutdown_connections 2024-03-21T10:47:55.912 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:55.914+0000 7fe6bf800700 1 -- 172.21.15.153:0/3877361136 wait complete. 2024-03-21T10:47:56.057 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.060+0000 7fa0e0800700 1 -- 172.21.15.153:0/2795718814 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa0d8068670 msgr2=0x7fa0d8068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:47:56.057 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.060+0000 7fa0e0800700 1 --2- 172.21.15.153:0/2795718814 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa0d8068670 0x7fa0d8068a50 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc009910 tx=0x7fa0cc009c20 comp rx=0 tx=0).stop 2024-03-21T10:47:56.058 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.060+0000 7fa0e0800700 1 -- 172.21.15.153:0/2795718814 shutdown_connections 2024-03-21T10:47:56.058 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.060+0000 7fa0e0800700 1 --2- 172.21.15.153:0/2795718814 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa0d8069020 0x7fa0d8105790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:47:56.058 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.060+0000 7fa0e0800700 1 --2- 172.21.15.153:0/2795718814 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa0d8068670 0x7fa0d8068a50 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:47:56.058 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.060+0000 7fa0e0800700 1 -- 172.21.15.153:0/2795718814 >> 172.21.15.153:0/2795718814 conn(0x7fa0d80755a0 msgr2=0x7fa0d80759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:47:56.058 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.060+0000 7fa0e0800700 1 -- 172.21.15.153:0/2795718814 shutdown_connections 2024-03-21T10:47:56.058 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.060+0000 7fa0e0800700 1 -- 172.21.15.153:0/2795718814 wait complete. 2024-03-21T10:47:56.059 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.061+0000 7fa0e0800700 1 Processor -- start 2024-03-21T10:47:56.059 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.061+0000 7fa0e0800700 1 -- start start 2024-03-21T10:47:56.059 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.062+0000 7fa0e0800700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa0d8068670 0x7fa0d819c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:47:56.059 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.062+0000 7fa0e0800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa0d8069020 0x7fa0d819cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:47:56.059 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.062+0000 7fa0e0800700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0d819d290 con 0x7fa0d8069020 2024-03-21T10:47:56.059 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.062+0000 7fa0e0800700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0d8196740 con 0x7fa0d8068670 2024-03-21T10:47:56.059 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.062+0000 7fa0dd400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa0d8069020 0x7fa0d819cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:47:56.059 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.062+0000 7fa0dd400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa0d8069020 0x7fa0d819cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:57806/0 (socket says 172.21.15.153:57806) 2024-03-21T10:47:56.060 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.062+0000 7fa0dd400700 1 -- 172.21.15.153:0/3619700169 learned_addr learned my addr 172.21.15.153:0/3619700169 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:47:56.060 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.062+0000 7fa0dde00700 1 --2- 172.21.15.153:0/3619700169 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa0d8068670 0x7fa0d819c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:47:56.060 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.063+0000 7fa0dd400700 1 -- 172.21.15.153:0/3619700169 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa0d8068670 msgr2=0x7fa0d819c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:47:56.061 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.063+0000 7fa0dd400700 1 --2- 172.21.15.153:0/3619700169 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa0d8068670 0x7fa0d819c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:47:56.061 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.063+0000 7fa0dd400700 1 -- 172.21.15.153:0/3619700169 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0cc0095f0 con 0x7fa0d8069020 2024-03-21T10:47:56.061 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.063+0000 7fa0dd400700 1 --2- 172.21.15.153:0/3619700169 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa0d8069020 0x7fa0d819cc00 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fa0c800b480 tx=0x7fa0c800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:47:56.061 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.063+0000 7fa0d2a00700 1 -- 172.21.15.153:0/3619700169 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa0c80118c0 con 0x7fa0d8069020 2024-03-21T10:47:56.061 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.063+0000 7fa0d2a00700 1 -- 172.21.15.153:0/3619700169 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa0c8010380 con 0x7fa0d8069020 2024-03-21T10:47:56.061 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.063+0000 7fa0e0800700 1 -- 172.21.15.153:0/3619700169 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0d8196a20 con 0x7fa0d8069020 2024-03-21T10:47:56.061 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.063+0000 7fa0d2a00700 1 -- 172.21.15.153:0/3619700169 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa0c800f4f0 con 0x7fa0d8069020 2024-03-21T10:47:56.061 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.063+0000 7fa0e0800700 1 -- 172.21.15.153:0/3619700169 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0d8196f70 con 0x7fa0d8069020 2024-03-21T10:47:56.062 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.064+0000 7fa0e0800700 1 -- 172.21.15.153:0/3619700169 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0d8109130 con 0x7fa0d8069020 2024-03-21T10:47:56.062 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.066+0000 7fa0d2a00700 1 -- 172.21.15.153:0/3619700169 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fa0c80104f0 con 0x7fa0d8069020 2024-03-21T10:47:56.064 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.067+0000 7fa0d2a00700 1 --2- 172.21.15.153:0/3619700169 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa0c006e460 0x7fa0c0070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:47:56.064 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.067+0000 7fa0d2a00700 1 -- 172.21.15.153:0/3619700169 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fa0c808e1e0 con 0x7fa0d8069020 2024-03-21T10:47:56.064 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.067+0000 7fa0dde00700 1 --2- 172.21.15.153:0/3619700169 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa0c006e460 0x7fa0c0070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:47:56.065 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.068+0000 7fa0dde00700 1 --2- 172.21.15.153:0/3619700169 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa0c006e460 0x7fa0c0070920 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc006010 tx=0x7fa0cc01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:47:56.068 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.072+0000 7fa0d2a00700 1 -- 172.21.15.153:0/3619700169 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa0c805ad30 con 0x7fa0d8069020 2024-03-21T10:47:56.359 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.362+0000 7fa0e0800700 1 -- 172.21.15.153:0/3619700169 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fa0d804ea90 con 0x7fa0d8069020 2024-03-21T10:47:56.361 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.364+0000 7fa0d2a00700 1 -- 172.21.15.153:0/3619700169 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fa0c805a8c0 con 0x7fa0d8069020 2024-03-21T10:47:56.362 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:47:56.362 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": { 2024-03-21T10:47:56.362 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:47:56.362 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:47:56.362 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": { 2024-03-21T10:47:56.362 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:47:56.362 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:47:56.362 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": { 2024-03-21T10:47:56.362 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 6 2024-03-21T10:47:56.362 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:47:56.363 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": { 2024-03-21T10:47:56.363 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 4 2024-03-21T10:47:56.363 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:47:56.363 INFO:teuthology.orchestra.run.smithi153.stdout: "overall": { 2024-03-21T10:47:56.363 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 14 2024-03-21T10:47:56.363 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-03-21T10:47:56.363 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:47:56.365 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.367+0000 7fa0e0800700 1 -- 172.21.15.153:0/3619700169 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa0c006e460 msgr2=0x7fa0c0070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:47:56.365 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.367+0000 7fa0e0800700 1 --2- 172.21.15.153:0/3619700169 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa0c006e460 0x7fa0c0070920 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc006010 tx=0x7fa0cc01a040 comp rx=0 tx=0).stop 2024-03-21T10:47:56.365 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.367+0000 7fa0e0800700 1 -- 172.21.15.153:0/3619700169 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa0d8069020 msgr2=0x7fa0d819cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:47:56.365 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.367+0000 7fa0e0800700 1 --2- 172.21.15.153:0/3619700169 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa0d8069020 0x7fa0d819cc00 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fa0c800b480 tx=0x7fa0c800b840 comp rx=0 tx=0).stop 2024-03-21T10:47:56.365 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.368+0000 7fa0e0800700 1 -- 172.21.15.153:0/3619700169 shutdown_connections 2024-03-21T10:47:56.365 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.368+0000 7fa0e0800700 1 --2- 172.21.15.153:0/3619700169 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa0d8068670 0x7fa0d819c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:47:56.365 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.368+0000 7fa0e0800700 1 --2- 172.21.15.153:0/3619700169 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa0c006e460 0x7fa0c0070920 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:47:56.365 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.368+0000 7fa0e0800700 1 --2- 172.21.15.153:0/3619700169 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa0d8069020 0x7fa0d819cc00 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:47:56.366 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.368+0000 7fa0e0800700 1 -- 172.21.15.153:0/3619700169 >> 172.21.15.153:0/3619700169 conn(0x7fa0d80755a0 msgr2=0x7fa0d80fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:47:56.366 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.368+0000 7fa0e0800700 1 -- 172.21.15.153:0/3619700169 shutdown_connections 2024-03-21T10:47:56.366 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.368+0000 7fa0e0800700 1 -- 172.21.15.153:0/3619700169 wait complete. 2024-03-21T10:47:56.513 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.515+0000 7fe4ef200700 1 -- 172.21.15.153:0/1139241655 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe4f0068410 msgr2=0x7fe4f0068890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:47:56.513 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.515+0000 7fe4ef200700 1 --2- 172.21.15.153:0/1139241655 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe4f0068410 0x7fe4f0068890 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fe4d8009910 tx=0x7fe4d8009c20 comp rx=0 tx=0).stop 2024-03-21T10:47:56.513 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.516+0000 7fe4ef200700 1 -- 172.21.15.153:0/1139241655 shutdown_connections 2024-03-21T10:47:56.513 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.516+0000 7fe4ef200700 1 --2- 172.21.15.153:0/1139241655 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe4f0068410 0x7fe4f0068890 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:47:56.513 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.516+0000 7fe4ef200700 1 --2- 172.21.15.153:0/1139241655 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe4f0106a40 0x7fe4f0106e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:47:56.513 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.516+0000 7fe4ef200700 1 -- 172.21.15.153:0/1139241655 >> 172.21.15.153:0/1139241655 conn(0x7fe4f0075330 msgr2=0x7fe4f0075740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:47:56.513 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.516+0000 7fe4ef200700 1 -- 172.21.15.153:0/1139241655 shutdown_connections 2024-03-21T10:47:56.513 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.516+0000 7fe4ef200700 1 -- 172.21.15.153:0/1139241655 wait complete. 2024-03-21T10:47:56.515 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.517+0000 7fe4ef200700 1 Processor -- start 2024-03-21T10:47:56.515 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.517+0000 7fe4ef200700 1 -- start start 2024-03-21T10:47:56.515 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.517+0000 7fe4ef200700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe4f0068410 0x7fe4f0196540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:47:56.515 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.517+0000 7fe4ef200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe4f0106a40 0x7fe4f0196a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:47:56.515 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.517+0000 7fe4ef200700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f0197160 con 0x7fe4f0106a40 2024-03-21T10:47:56.515 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.517+0000 7fe4ef200700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f019aef0 con 0x7fe4f0068410 2024-03-21T10:47:56.515 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.518+0000 7fe4e7e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe4f0106a40 0x7fe4f0196a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:47:56.515 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.518+0000 7fe4e7e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe4f0106a40 0x7fe4f0196a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:57826/0 (socket says 172.21.15.153:57826) 2024-03-21T10:47:56.515 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.518+0000 7fe4e7e00700 1 -- 172.21.15.153:0/4212093120 learned_addr learned my addr 172.21.15.153:0/4212093120 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:47:56.515 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.518+0000 7fe4ede00700 1 --2- 172.21.15.153:0/4212093120 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe4f0068410 0x7fe4f0196540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:47:56.515 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.518+0000 7fe4e7e00700 1 -- 172.21.15.153:0/4212093120 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe4f0068410 msgr2=0x7fe4f0196540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:47:56.515 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.518+0000 7fe4e7e00700 1 --2- 172.21.15.153:0/4212093120 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe4f0068410 0x7fe4f0196540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:47:56.516 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.518+0000 7fe4e7e00700 1 -- 172.21.15.153:0/4212093120 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4d80095f0 con 0x7fe4f0106a40 2024-03-21T10:47:56.516 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.518+0000 7fe4e7e00700 1 --2- 172.21.15.153:0/4212093120 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe4f0106a40 0x7fe4f0196a80 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fe4d8006010 tx=0x7fe4d800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:47:56.516 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.519+0000 7fe4e6a00700 1 -- 172.21.15.153:0/4212093120 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe4d801d070 con 0x7fe4f0106a40 2024-03-21T10:47:56.516 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.519+0000 7fe4e6a00700 1 -- 172.21.15.153:0/4212093120 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe4d800f460 con 0x7fe4f0106a40 2024-03-21T10:47:56.516 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.519+0000 7fe4e6a00700 1 -- 172.21.15.153:0/4212093120 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe4d8017630 con 0x7fe4f0106a40 2024-03-21T10:47:56.517 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.519+0000 7fe4ef200700 1 -- 172.21.15.153:0/4212093120 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4f019b170 con 0x7fe4f0106a40 2024-03-21T10:47:56.517 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.519+0000 7fe4ef200700 1 -- 172.21.15.153:0/4212093120 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4f019b660 con 0x7fe4f0106a40 2024-03-21T10:47:56.517 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.520+0000 7fe4ef200700 1 -- 172.21.15.153:0/4212093120 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4f0103ce0 con 0x7fe4f0106a40 2024-03-21T10:47:56.518 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.521+0000 7fe4e6a00700 1 -- 172.21.15.153:0/4212093120 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fe4d800f5d0 con 0x7fe4f0106a40 2024-03-21T10:47:56.519 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.522+0000 7fe4e6a00700 1 --2- 172.21.15.153:0/4212093120 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe4d406e460 0x7fe4d4070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:47:56.519 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.522+0000 7fe4e6a00700 1 -- 172.21.15.153:0/4212093120 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fe4d808fa10 con 0x7fe4f0106a40 2024-03-21T10:47:56.519 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.522+0000 7fe4ede00700 1 --2- 172.21.15.153:0/4212093120 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe4d406e460 0x7fe4d4070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:47:56.519 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.523+0000 7fe4ede00700 1 --2- 172.21.15.153:0/4212093120 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe4d406e460 0x7fe4d4070920 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fe4e0005950 tx=0x7fe4e00058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:47:56.524 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.527+0000 7fe4e6a00700 1 -- 172.21.15.153:0/4212093120 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe4d8058af0 con 0x7fe4f0106a40 2024-03-21T10:47:56.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:47:56 smithi174 ceph-mon[88004]: from='client.35910 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:47:56.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:47:56 smithi174 ceph-mon[88004]: from='client.35914 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:47:56.792 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.795+0000 7fe4ef200700 1 -- 172.21.15.153:0/4212093120 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fe4f004f2e0 con 0x7fe4f0106a40 2024-03-21T10:47:56.793 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:47:56 smithi153 ceph-mon[113733]: from='client.35910 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:47:56.793 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:47:56 smithi153 ceph-mon[113733]: from='client.35914 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:47:56.793 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.796+0000 7fe4e6a00700 1 -- 172.21.15.153:0/4212093120 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1876 (secure 0 0 0) 0x7fe4d808de60 con 0x7fe4f0106a40 2024-03-21T10:47:56.793 INFO:teuthology.orchestra.run.smithi153.stdout:e32 2024-03-21T10:47:56.794 INFO:teuthology.orchestra.run.smithi153.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-21T10:47:56.794 INFO:teuthology.orchestra.run.smithi153.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:47:56.794 INFO:teuthology.orchestra.run.smithi153.stdout:legacy client fscid: 1 2024-03-21T10:47:56.794 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:47:56.794 INFO:teuthology.orchestra.run.smithi153.stdout:Filesystem 'cephfs' (1) 2024-03-21T10:47:56.794 INFO:teuthology.orchestra.run.smithi153.stdout:fs_name cephfs 2024-03-21T10:47:56.794 INFO:teuthology.orchestra.run.smithi153.stdout:epoch 23 2024-03-21T10:47:56.794 INFO:teuthology.orchestra.run.smithi153.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-21T10:47:56.794 INFO:teuthology.orchestra.run.smithi153.stdout:created 2024-03-21T10:04:27.086576+0000 2024-03-21T10:47:56.794 INFO:teuthology.orchestra.run.smithi153.stdout:modified 2024-03-21T10:19:05.022092+0000 2024-03-21T10:47:56.794 INFO:teuthology.orchestra.run.smithi153.stdout:tableserver 0 2024-03-21T10:47:56.794 INFO:teuthology.orchestra.run.smithi153.stdout:root 0 2024-03-21T10:47:56.795 INFO:teuthology.orchestra.run.smithi153.stdout:session_timeout 60 2024-03-21T10:47:56.795 INFO:teuthology.orchestra.run.smithi153.stdout:session_autoclose 300 2024-03-21T10:47:56.795 INFO:teuthology.orchestra.run.smithi153.stdout:max_file_size 1099511627776 2024-03-21T10:47:56.795 INFO:teuthology.orchestra.run.smithi153.stdout:max_xattr_size 65536 2024-03-21T10:47:56.795 INFO:teuthology.orchestra.run.smithi153.stdout:required_client_features {} 2024-03-21T10:47:56.795 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure 0 2024-03-21T10:47:56.795 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure_osd_epoch 94 2024-03-21T10:47:56.795 INFO:teuthology.orchestra.run.smithi153.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:47:56.795 INFO:teuthology.orchestra.run.smithi153.stdout:max_mds 2 2024-03-21T10:47:56.795 INFO:teuthology.orchestra.run.smithi153.stdout:in 0,1 2024-03-21T10:47:56.795 INFO:teuthology.orchestra.run.smithi153.stdout:up {} 2024-03-21T10:47:56.795 INFO:teuthology.orchestra.run.smithi153.stdout:failed 0,1 2024-03-21T10:47:56.795 INFO:teuthology.orchestra.run.smithi153.stdout:damaged 2024-03-21T10:47:56.795 INFO:teuthology.orchestra.run.smithi153.stdout:stopped 2024-03-21T10:47:56.796 INFO:teuthology.orchestra.run.smithi153.stdout:data_pools [3] 2024-03-21T10:47:56.796 INFO:teuthology.orchestra.run.smithi153.stdout:metadata_pool 2 2024-03-21T10:47:56.796 INFO:teuthology.orchestra.run.smithi153.stdout:inline_data enabled 2024-03-21T10:47:56.796 INFO:teuthology.orchestra.run.smithi153.stdout:balancer 2024-03-21T10:47:56.796 INFO:teuthology.orchestra.run.smithi153.stdout:bal_rank_mask -1 2024-03-21T10:47:56.796 INFO:teuthology.orchestra.run.smithi153.stdout:standby_count_wanted 1 2024-03-21T10:47:56.796 INFO:teuthology.orchestra.run.smithi153.stdout:qdb_cluster leader: 0 members: 2024-03-21T10:47:56.796 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:47:56.796 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:47:56.796 INFO:teuthology.orchestra.run.smithi153.stdout:Standby daemons: 2024-03-21T10:47:56.796 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:47:56.796 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.zipckr{-1:34442} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6826/1103846295,v1:172.21.15.153:6827/1103846295] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:47:56.796 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.iuyluj{-1:34474} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6828/1738630832,v1:172.21.15.153:6829/1738630832] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:47:56.796 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.yyimst{-1:44313} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.174:6824/4127786797,v1:172.21.15.174:6825/4127786797] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:47:56.796 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.nhozag{-1:44687} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.174:6826/2767517845,v1:172.21.15.174:6827/2767517845] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:47:56.797 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.799+0000 7fe4ef200700 1 -- 172.21.15.153:0/4212093120 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe4d406e460 msgr2=0x7fe4d4070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:47:56.797 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.799+0000 7fe4ef200700 1 --2- 172.21.15.153:0/4212093120 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe4d406e460 0x7fe4d4070920 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fe4e0005950 tx=0x7fe4e00058e0 comp rx=0 tx=0).stop 2024-03-21T10:47:56.797 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.799+0000 7fe4ef200700 1 -- 172.21.15.153:0/4212093120 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe4f0106a40 msgr2=0x7fe4f0196a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:47:56.797 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.799+0000 7fe4ef200700 1 --2- 172.21.15.153:0/4212093120 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe4f0106a40 0x7fe4f0196a80 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fe4d8006010 tx=0x7fe4d800be20 comp rx=0 tx=0).stop 2024-03-21T10:47:56.798 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.799+0000 7fe4ef200700 1 -- 172.21.15.153:0/4212093120 shutdown_connections 2024-03-21T10:47:56.798 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.799+0000 7fe4ef200700 1 --2- 172.21.15.153:0/4212093120 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe4f0068410 0x7fe4f0196540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:47:56.798 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.799+0000 7fe4ef200700 1 --2- 172.21.15.153:0/4212093120 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe4d406e460 0x7fe4d4070920 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:47:56.798 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.799+0000 7fe4ef200700 1 --2- 172.21.15.153:0/4212093120 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe4f0106a40 0x7fe4f0196a80 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:47:56.798 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.799+0000 7fe4ef200700 1 -- 172.21.15.153:0/4212093120 >> 172.21.15.153:0/4212093120 conn(0x7fe4f0075330 msgr2=0x7fe4f00fea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:47:56.798 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.799+0000 7fe4ef200700 1 -- 172.21.15.153:0/4212093120 shutdown_connections 2024-03-21T10:47:56.798 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.799+0000 7fe4ef200700 1 -- 172.21.15.153:0/4212093120 wait complete. 2024-03-21T10:47:56.798 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 32 2024-03-21T10:47:56.942 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.945+0000 7fee32600700 1 -- 172.21.15.153:0/4276281024 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee2c102fe0 msgr2=0x7fee2c1033c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:47:56.942 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.945+0000 7fee32600700 1 --2- 172.21.15.153:0/4276281024 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee2c102fe0 0x7fee2c1033c0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fee1c009910 tx=0x7fee1c009c20 comp rx=0 tx=0).stop 2024-03-21T10:47:56.942 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.945+0000 7fee32600700 1 -- 172.21.15.153:0/4276281024 shutdown_connections 2024-03-21T10:47:56.942 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.945+0000 7fee32600700 1 --2- 172.21.15.153:0/4276281024 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fee2c103990 0x7fee2c1079e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:47:56.942 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.945+0000 7fee32600700 1 --2- 172.21.15.153:0/4276281024 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee2c102fe0 0x7fee2c1033c0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:47:56.942 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.945+0000 7fee32600700 1 -- 172.21.15.153:0/4276281024 >> 172.21.15.153:0/4276281024 conn(0x7fee2c0fe8d0 msgr2=0x7fee2c100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:47:56.942 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.945+0000 7fee32600700 1 -- 172.21.15.153:0/4276281024 shutdown_connections 2024-03-21T10:47:56.943 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.945+0000 7fee32600700 1 -- 172.21.15.153:0/4276281024 wait complete. 2024-03-21T10:47:56.943 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.946+0000 7fee32600700 1 Processor -- start 2024-03-21T10:47:56.943 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.946+0000 7fee32600700 1 -- start start 2024-03-21T10:47:56.943 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.947+0000 7fee32600700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fee2c102fe0 0x7fee2c1989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:47:56.944 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.947+0000 7fee32600700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee2c103990 0x7fee2c198ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:47:56.944 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.947+0000 7fee32600700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee2c1994e0 con 0x7fee2c103990 2024-03-21T10:47:56.944 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.947+0000 7fee32600700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee2c199650 con 0x7fee2c102fe0 2024-03-21T10:47:56.946 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.947+0000 7fee23e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee2c103990 0x7fee2c198ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:47:56.946 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.947+0000 7fee23e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee2c103990 0x7fee2c198ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:57832/0 (socket says 172.21.15.153:57832) 2024-03-21T10:47:56.946 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.947+0000 7fee23e00700 1 -- 172.21.15.153:0/74806176 learned_addr learned my addr 172.21.15.153:0/74806176 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:47:56.946 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.947+0000 7fee2b400700 1 --2- 172.21.15.153:0/74806176 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fee2c102fe0 0x7fee2c1989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:47:56.946 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.948+0000 7fee23e00700 1 -- 172.21.15.153:0/74806176 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fee2c102fe0 msgr2=0x7fee2c1989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:47:56.946 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.948+0000 7fee23e00700 1 --2- 172.21.15.153:0/74806176 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fee2c102fe0 0x7fee2c1989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:47:56.946 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.948+0000 7fee23e00700 1 -- 172.21.15.153:0/74806176 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee1c0095f0 con 0x7fee2c103990 2024-03-21T10:47:56.946 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.948+0000 7fee23e00700 1 --2- 172.21.15.153:0/74806176 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee2c103990 0x7fee2c198ee0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fee1400c960 tx=0x7fee1400cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:47:56.946 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.948+0000 7fee28c00700 1 -- 172.21.15.153:0/74806176 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fee14007a70 con 0x7fee2c103990 2024-03-21T10:47:56.946 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.948+0000 7fee28c00700 1 -- 172.21.15.153:0/74806176 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fee14007bd0 con 0x7fee2c103990 2024-03-21T10:47:56.947 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.948+0000 7fee28c00700 1 -- 172.21.15.153:0/74806176 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fee140186c0 con 0x7fee2c103990 2024-03-21T10:47:56.947 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.948+0000 7fee32600700 1 -- 172.21.15.153:0/74806176 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee2c19d4d0 con 0x7fee2c103990 2024-03-21T10:47:56.947 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.948+0000 7fee32600700 1 -- 172.21.15.153:0/74806176 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee2c19da20 con 0x7fee2c103990 2024-03-21T10:47:56.947 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.950+0000 7fee32600700 1 -- 172.21.15.153:0/74806176 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee2c10b380 con 0x7fee2c103990 2024-03-21T10:47:56.947 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.951+0000 7fee28c00700 1 -- 172.21.15.153:0/74806176 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fee1400faa0 con 0x7fee2c103990 2024-03-21T10:47:56.949 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.952+0000 7fee28c00700 1 --2- 172.21.15.153:0/74806176 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fee1006e460 0x7fee10070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:47:56.949 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.952+0000 7fee28c00700 1 -- 172.21.15.153:0/74806176 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fee1408e520 con 0x7fee2c103990 2024-03-21T10:47:56.949 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.952+0000 7fee2b400700 1 --2- 172.21.15.153:0/74806176 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fee1006e460 0x7fee10070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:47:56.949 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.953+0000 7fee2b400700 1 --2- 172.21.15.153:0/74806176 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fee1006e460 0x7fee10070920 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fee1c00b5c0 tx=0x7fee1c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:47:56.953 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:56.956+0000 7fee28c00700 1 -- 172.21.15.153:0/74806176 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fee1405ae60 con 0x7fee2c103990 2024-03-21T10:47:57.185 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:57.188+0000 7fee32600700 1 -- 172.21.15.153:0/74806176 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fee2c02cce0 con 0x7fee1006e460 2024-03-21T10:47:57.187 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:57.190+0000 7fee28c00700 1 -- 172.21.15.153:0/74806176 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fee2c02cce0 con 0x7fee1006e460 2024-03-21T10:47:57.187 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:47:57.187 INFO:teuthology.orchestra.run.smithi153.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232", 2024-03-21T10:47:57.187 INFO:teuthology.orchestra.run.smithi153.stdout: "in_progress": true, 2024-03-21T10:47:57.187 INFO:teuthology.orchestra.run.smithi153.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-21T10:47:57.187 INFO:teuthology.orchestra.run.smithi153.stdout: "services_complete": [ 2024-03-21T10:47:57.188 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr", 2024-03-21T10:47:57.188 INFO:teuthology.orchestra.run.smithi153.stdout: "mon", 2024-03-21T10:47:57.188 INFO:teuthology.orchestra.run.smithi153.stdout: "mds", 2024-03-21T10:47:57.188 INFO:teuthology.orchestra.run.smithi153.stdout: "crash", 2024-03-21T10:47:57.188 INFO:teuthology.orchestra.run.smithi153.stdout: "osd" 2024-03-21T10:47:57.188 INFO:teuthology.orchestra.run.smithi153.stdout: ], 2024-03-21T10:47:57.188 INFO:teuthology.orchestra.run.smithi153.stdout: "progress": "16/23 daemons upgraded", 2024-03-21T10:47:57.188 INFO:teuthology.orchestra.run.smithi153.stdout: "message": "Currently upgrading mds daemons", 2024-03-21T10:47:57.188 INFO:teuthology.orchestra.run.smithi153.stdout: "is_paused": false 2024-03-21T10:47:57.188 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:47:57.190 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:57.193+0000 7fee32600700 1 -- 172.21.15.153:0/74806176 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fee1006e460 msgr2=0x7fee10070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:47:57.190 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:57.193+0000 7fee32600700 1 --2- 172.21.15.153:0/74806176 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fee1006e460 0x7fee10070920 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fee1c00b5c0 tx=0x7fee1c01a040 comp rx=0 tx=0).stop 2024-03-21T10:47:57.190 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:57.193+0000 7fee32600700 1 -- 172.21.15.153:0/74806176 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee2c103990 msgr2=0x7fee2c198ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:47:57.191 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:57.193+0000 7fee32600700 1 --2- 172.21.15.153:0/74806176 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee2c103990 0x7fee2c198ee0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fee1400c960 tx=0x7fee1400cd20 comp rx=0 tx=0).stop 2024-03-21T10:47:57.191 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:57.193+0000 7fee32600700 1 -- 172.21.15.153:0/74806176 shutdown_connections 2024-03-21T10:47:57.191 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:57.193+0000 7fee32600700 1 --2- 172.21.15.153:0/74806176 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fee2c102fe0 0x7fee2c1989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:47:57.191 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:57.193+0000 7fee32600700 1 --2- 172.21.15.153:0/74806176 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fee1006e460 0x7fee10070920 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:47:57.191 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:57.193+0000 7fee32600700 1 --2- 172.21.15.153:0/74806176 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fee2c103990 0x7fee2c198ee0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:47:57.191 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:57.193+0000 7fee32600700 1 -- 172.21.15.153:0/74806176 >> 172.21.15.153:0/74806176 conn(0x7fee2c0fe8d0 msgr2=0x7fee2c0fff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:47:57.191 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:57.193+0000 7fee32600700 1 -- 172.21.15.153:0/74806176 shutdown_connections 2024-03-21T10:47:57.191 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:57.194+0000 7fee32600700 1 -- 172.21.15.153:0/74806176 wait complete. 2024-03-21T10:47:57.345 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:57.348+0000 7f1291000700 1 -- 172.21.15.153:0/4056233962 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f128c102fe0 msgr2=0x7f128c1033c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:47:57.346 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:57.348+0000 7f1291000700 1 --2- 172.21.15.153:0/4056233962 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f128c102fe0 0x7f128c1033c0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f127c009910 tx=0x7f127c009c20 comp rx=0 tx=0).stop 2024-03-21T10:47:57.346 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:57.348+0000 7f1291000700 1 -- 172.21.15.153:0/4056233962 shutdown_connections 2024-03-21T10:47:57.346 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:57.348+0000 7f1291000700 1 --2- 172.21.15.153:0/4056233962 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f128c103990 0x7f128c1079e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:47:57.346 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:57.348+0000 7f1291000700 1 --2- 172.21.15.153:0/4056233962 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f128c102fe0 0x7f128c1033c0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:47:57.346 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:57.348+0000 7f1291000700 1 -- 172.21.15.153:0/4056233962 >> 172.21.15.153:0/4056233962 conn(0x7f128c0fe8d0 msgr2=0x7f128c100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:47:57.346 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:57.348+0000 7f1291000700 1 -- 172.21.15.153:0/4056233962 shutdown_connections 2024-03-21T10:47:57.346 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:57.348+0000 7f1291000700 1 -- 172.21.15.153:0/4056233962 wait complete. 2024-03-21T10:47:57.348 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:57.349+0000 7f1291000700 1 Processor -- start 2024-03-21T10:47:57.348 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:57.349+0000 7f1291000700 1 -- start start 2024-03-21T10:47:57.348 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:57.350+0000 7f1291000700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f128c102fe0 0x7f128c1989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:47:57.349 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:57.350+0000 7f1291000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f128c103990 0x7f128c198ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:47:57.349 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:57.350+0000 7f1291000700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f128c199570 con 0x7f128c103990 2024-03-21T10:47:57.349 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:57.350+0000 7f1291000700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f128c19d380 con 0x7f128c102fe0 2024-03-21T10:47:57.349 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:57.350+0000 7f1289400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f128c103990 0x7f128c198ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:47:57.349 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:57.350+0000 7f1289400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f128c103990 0x7f128c198ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:57850/0 (socket says 172.21.15.153:57850) 2024-03-21T10:47:57.349 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:57.350+0000 7f1289400700 1 -- 172.21.15.153:0/3673493232 learned_addr learned my addr 172.21.15.153:0/3673493232 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:47:57.349 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:57.350+0000 7f1289e00700 1 --2- 172.21.15.153:0/3673493232 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f128c102fe0 0x7f128c1989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:47:57.349 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:57.350+0000 7f1289400700 1 -- 172.21.15.153:0/3673493232 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f128c102fe0 msgr2=0x7f128c1989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:47:57.349 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:57.350+0000 7f1289400700 1 --2- 172.21.15.153:0/3673493232 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f128c102fe0 0x7f128c1989a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:47:57.350 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:57.350+0000 7f1289400700 1 -- 172.21.15.153:0/3673493232 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f127c0095f0 con 0x7f128c103990 2024-03-21T10:47:57.350 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:57.351+0000 7f1289400700 1 --2- 172.21.15.153:0/3673493232 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f128c103990 0x7f128c198ee0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f127800d900 tx=0x7f127800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:47:57.350 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:57.351+0000 7f1282a00700 1 -- 172.21.15.153:0/3673493232 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1278009900 con 0x7f128c103990 2024-03-21T10:47:57.350 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:57.351+0000 7f1282a00700 1 -- 172.21.15.153:0/3673493232 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1278010460 con 0x7f128c103990 2024-03-21T10:47:57.350 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:57.351+0000 7f1291000700 1 -- 172.21.15.153:0/3673493232 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f128c19d660 con 0x7f128c103990 2024-03-21T10:47:57.350 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:57.351+0000 7f1282a00700 1 -- 172.21.15.153:0/3673493232 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f127800f5f0 con 0x7f128c103990 2024-03-21T10:47:57.350 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:57.351+0000 7f1291000700 1 -- 172.21.15.153:0/3673493232 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f128c19dbb0 con 0x7f128c103990 2024-03-21T10:47:57.351 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:57.352+0000 7f1291000700 1 -- 172.21.15.153:0/3673493232 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f128c10b380 con 0x7f128c103990 2024-03-21T10:47:57.351 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:57.353+0000 7f1282a00700 1 -- 172.21.15.153:0/3673493232 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f12780105d0 con 0x7f128c103990 2024-03-21T10:47:57.351 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:57.354+0000 7f1282a00700 1 --2- 172.21.15.153:0/3673493232 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f127006e460 0x7f1270070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:47:57.351 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:57.354+0000 7f1282a00700 1 -- 172.21.15.153:0/3673493232 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f127808e370 con 0x7f128c103990 2024-03-21T10:47:57.352 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:57.354+0000 7f1289e00700 1 --2- 172.21.15.153:0/3673493232 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f127006e460 0x7f1270070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:47:57.352 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:57.355+0000 7f1289e00700 1 --2- 172.21.15.153:0/3673493232 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f127006e460 0x7f1270070920 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f127c006010 tx=0x7f127c00b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:47:57.355 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:57.359+0000 7f1282a00700 1 -- 172.21.15.153:0/3673493232 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f127805af60 con 0x7f128c103990 2024-03-21T10:47:57.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:47:57 smithi153 ceph-mon[113733]: from='client.35918 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:47:57.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:47:57 smithi153 ceph-mon[113733]: pgmap v1084: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:47:57.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:47:57 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/3619700169' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:47:57.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:47:57 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/4212093120' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:47:57.670 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:57.673+0000 7f1291000700 1 -- 172.21.15.153:0/3673493232 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f128c04ea90 con 0x7f128c103990 2024-03-21T10:47:57.670 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:57.673+0000 7f1282a00700 1 -- 172.21.15.153:0/3673493232 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4493 (secure 0 0 0) 0x7f127805aaf0 con 0x7f128c103990 2024-03-21T10:47:57.671 INFO:teuthology.orchestra.run.smithi153.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-03-21T10:47:57.671 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-03-21T10:47:57.671 INFO:teuthology.orchestra.run.smithi153.stdout: host smithi174 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi174/config 2024-03-21T10:47:57.671 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/podman: timeout after 895 seconds 2024-03-21T10:47:57.671 INFO:teuthology.orchestra.run.smithi153.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 -e NODE_NAME=smithi174 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/run/ceph:z -v /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpd910r3_i:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 inventory --format=json-pretty --filter-for-batch 2024-03-21T10:47:57.671 INFO:teuthology.orchestra.run.smithi153.stdout:Traceback (most recent call last): 2024-03-21T10:47:57.671 INFO:teuthology.orchestra.run.smithi153.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-03-21T10:47:57.672 INFO:teuthology.orchestra.run.smithi153.stdout: return _run_code(code, main_globals, None, 2024-03-21T10:47:57.672 INFO:teuthology.orchestra.run.smithi153.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-03-21T10:47:57.672 INFO:teuthology.orchestra.run.smithi153.stdout: exec(code, run_globals) 2024-03-21T10:47:57.672 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 5606, in 2024-03-21T10:47:57.672 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 5594, in main 2024-03-21T10:47:57.672 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 406, in _infer_config 2024-03-21T10:47:57.672 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 321, in _infer_fsid 2024-03-21T10:47:57.672 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 434, in _infer_image 2024-03-21T10:47:57.672 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 308, in _validate_fsid 2024-03-21T10:47:57.672 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 3357, in command_ceph_volume 2024-03-21T10:47:57.672 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-03-21T10:47:57.672 INFO:teuthology.orchestra.run.smithi153.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 -e NODE_NAME=smithi174 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/run/ceph:z -v /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpd910r3_i:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 inventory --format=json-pretty --filter-for-batch 2024-03-21T10:47:57.672 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-21T10:47:57.672 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is degraded 2024-03-21T10:47:57.673 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-21T10:47:57.673 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-21T10:47:57.673 INFO:teuthology.orchestra.run.smithi153.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-21T10:47:57.673 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-21T10:47:57.673 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:57.676+0000 7f1291000700 1 -- 172.21.15.153:0/3673493232 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f127006e460 msgr2=0x7f1270070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:47:57.674 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:57.676+0000 7f1291000700 1 --2- 172.21.15.153:0/3673493232 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f127006e460 0x7f1270070920 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f127c006010 tx=0x7f127c00b540 comp rx=0 tx=0).stop 2024-03-21T10:47:57.674 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:57.676+0000 7f1291000700 1 -- 172.21.15.153:0/3673493232 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f128c103990 msgr2=0x7f128c198ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:47:57.674 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:57.676+0000 7f1291000700 1 --2- 172.21.15.153:0/3673493232 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f128c103990 0x7f128c198ee0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f127800d900 tx=0x7f127800dcc0 comp rx=0 tx=0).stop 2024-03-21T10:47:57.674 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:57.676+0000 7f1291000700 1 -- 172.21.15.153:0/3673493232 shutdown_connections 2024-03-21T10:47:57.674 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:57.676+0000 7f1291000700 1 --2- 172.21.15.153:0/3673493232 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f128c102fe0 0x7f128c1989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:47:57.674 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:57.676+0000 7f1291000700 1 --2- 172.21.15.153:0/3673493232 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f127006e460 0x7f1270070920 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:47:57.674 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:57.676+0000 7f1291000700 1 --2- 172.21.15.153:0/3673493232 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f128c103990 0x7f128c198ee0 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:47:57.674 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:57.676+0000 7f1291000700 1 -- 172.21.15.153:0/3673493232 >> 172.21.15.153:0/3673493232 conn(0x7f128c0fe8d0 msgr2=0x7f128c0fff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:47:57.674 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:57.677+0000 7f1291000700 1 -- 172.21.15.153:0/3673493232 shutdown_connections 2024-03-21T10:47:57.674 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:47:57.677+0000 7f1291000700 1 -- 172.21.15.153:0/3673493232 wait complete. 2024-03-21T10:47:57.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:47:57 smithi174 ceph-mon[88004]: from='client.35918 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:47:57.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:47:57 smithi174 ceph-mon[88004]: pgmap v1084: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:47:57.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:47:57 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/3619700169' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:47:57.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:47:57 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/4212093120' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:47:58.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:47:58 smithi174 ceph-mon[88004]: from='client.35930 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:47:58.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:47:58 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/3673493232' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:47:58.801 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:47:58 smithi153 ceph-mon[113733]: from='client.35930 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:47:58.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:47:58 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/3673493232' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:47:59.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:47:59 smithi174 ceph-mon[88004]: pgmap v1085: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:47:59.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:47:59 smithi153 ceph-mon[113733]: pgmap v1085: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:48:01.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:48:01 smithi174 ceph-mon[88004]: pgmap v1086: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:48:01.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:48:01 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:48:01.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:48:01 smithi153 ceph-mon[113733]: pgmap v1086: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:48:01.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:48:01 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:48:03.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:48:03 smithi174 ceph-mon[88004]: pgmap v1087: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:48:03.801 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:48:03 smithi153 ceph-mon[113733]: pgmap v1087: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:48:05.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:48:05 smithi174 ceph-mon[88004]: pgmap v1088: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:48:05.801 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:48:05 smithi153 ceph-mon[113733]: pgmap v1088: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:48:07.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:48:07 smithi174 ceph-mon[88004]: pgmap v1089: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:48:07.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:48:07 smithi153 ceph-mon[113733]: pgmap v1089: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:48:09.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:48:09 smithi174 ceph-mon[88004]: pgmap v1090: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:48:09.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:48:09 smithi153 ceph-mon[113733]: pgmap v1090: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:48:11.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:48:11 smithi174 ceph-mon[88004]: pgmap v1091: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:48:11.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:48:11 smithi153 ceph-mon[113733]: pgmap v1091: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:48:13.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:48:13 smithi174 ceph-mon[88004]: pgmap v1092: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:48:13.801 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:48:13 smithi153 ceph-mon[113733]: pgmap v1092: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:48:15.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:48:15 smithi174 ceph-mon[88004]: pgmap v1093: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:48:15.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:48:15 smithi153 ceph-mon[113733]: pgmap v1093: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:48:16.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:48:16 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:48:16.801 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:48:16 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:48:17.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:48:17 smithi153 ceph-mon[113733]: pgmap v1094: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:48:17.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:48:17 smithi174 ceph-mon[88004]: pgmap v1094: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:48:19.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:48:19 smithi153 ceph-mon[113733]: pgmap v1095: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:48:19.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:48:19 smithi174 ceph-mon[88004]: pgmap v1095: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:48:21.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:48:21 smithi174 ceph-mon[88004]: pgmap v1096: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:48:21.801 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:48:21 smithi153 ceph-mon[113733]: pgmap v1096: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:48:23.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:48:23 smithi153 ceph-mon[113733]: pgmap v1097: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:48:23.946 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:48:23 smithi174 ceph-mon[88004]: pgmap v1097: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:48:25.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:48:25 smithi153 ceph-mon[113733]: pgmap v1098: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:48:25.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:48:25 smithi174 ceph-mon[88004]: pgmap v1098: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:48:27.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:48:27 smithi153 ceph-mon[113733]: pgmap v1099: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:48:27.845 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:27.847+0000 7f1681800700 1 -- 172.21.15.153:0/3086606547 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f167c1039b0 msgr2=0x7f167c107a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:48:27.845 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:27.847+0000 7f1681800700 1 --2- 172.21.15.153:0/3086606547 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f167c1039b0 0x7f167c107a00 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f1668009910 tx=0x7f1668009c20 comp rx=0 tx=0).stop 2024-03-21T10:48:27.845 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:27.848+0000 7f1681800700 1 -- 172.21.15.153:0/3086606547 shutdown_connections 2024-03-21T10:48:27.845 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:27.848+0000 7f1681800700 1 --2- 172.21.15.153:0/3086606547 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f167c1039b0 0x7f167c107a00 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:48:27.845 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:27.848+0000 7f1681800700 1 --2- 172.21.15.153:0/3086606547 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f167c103000 0x7f167c1033e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:48:27.845 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:27.848+0000 7f1681800700 1 -- 172.21.15.153:0/3086606547 >> 172.21.15.153:0/3086606547 conn(0x7f167c0fe8d0 msgr2=0x7f167c100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:48:27.845 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:27.848+0000 7f1681800700 1 -- 172.21.15.153:0/3086606547 shutdown_connections 2024-03-21T10:48:27.846 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:27.848+0000 7f1681800700 1 -- 172.21.15.153:0/3086606547 wait complete. 2024-03-21T10:48:27.846 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:27.849+0000 7f1681800700 1 Processor -- start 2024-03-21T10:48:27.846 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:27.849+0000 7f1681800700 1 -- start start 2024-03-21T10:48:27.846 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:27.850+0000 7f1681800700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f167c103000 0x7f167c19e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:48:27.847 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:27.850+0000 7f1681800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f167c1039b0 0x7f167c19ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:48:27.847 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:27.850+0000 7f1681800700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f167c19f490 con 0x7f167c1039b0 2024-03-21T10:48:27.847 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:27.850+0000 7f1681800700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f167c198940 con 0x7f167c103000 2024-03-21T10:48:27.847 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:27.850+0000 7f167a000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f167c1039b0 0x7f167c19ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:48:27.848 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:27.850+0000 7f167a000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f167c1039b0 0x7f167c19ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:53978/0 (socket says 172.21.15.153:53978) 2024-03-21T10:48:27.848 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:27.850+0000 7f167a000700 1 -- 172.21.15.153:0/1984324995 learned_addr learned my addr 172.21.15.153:0/1984324995 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:48:27.848 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:27.850+0000 7f167aa00700 1 --2- 172.21.15.153:0/1984324995 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f167c103000 0x7f167c19e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:48:27.848 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:27.850+0000 7f167a000700 1 -- 172.21.15.153:0/1984324995 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f167c103000 msgr2=0x7f167c19e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:48:27.848 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:27.850+0000 7f167a000700 1 --2- 172.21.15.153:0/1984324995 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f167c103000 0x7f167c19e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:48:27.848 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:27.850+0000 7f167a000700 1 -- 172.21.15.153:0/1984324995 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16680095f0 con 0x7f167c1039b0 2024-03-21T10:48:27.848 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:27.851+0000 7f167a000700 1 --2- 172.21.15.153:0/1984324995 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f167c1039b0 0x7f167c19ee00 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f166800bda0 tx=0x7f166800be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:48:27.849 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:27.851+0000 7f1673400700 1 -- 172.21.15.153:0/1984324995 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f166801d070 con 0x7f167c1039b0 2024-03-21T10:48:27.849 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:27.851+0000 7f1673400700 1 -- 172.21.15.153:0/1984324995 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1668004fb0 con 0x7f167c1039b0 2024-03-21T10:48:27.849 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:27.851+0000 7f1681800700 1 -- 172.21.15.153:0/1984324995 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f167c198bc0 con 0x7f167c1039b0 2024-03-21T10:48:27.849 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:27.851+0000 7f1681800700 1 -- 172.21.15.153:0/1984324995 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f167c1990b0 con 0x7f167c1039b0 2024-03-21T10:48:27.849 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:27.851+0000 7f1673400700 1 -- 172.21.15.153:0/1984324995 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f166800f460 con 0x7f167c1039b0 2024-03-21T10:48:27.849 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:27.853+0000 7f1681800700 1 -- 172.21.15.153:0/1984324995 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f167c10b350 con 0x7f167c1039b0 2024-03-21T10:48:27.850 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:27.854+0000 7f1673400700 1 -- 172.21.15.153:0/1984324995 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f1668022800 con 0x7f167c1039b0 2024-03-21T10:48:27.851 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:27.854+0000 7f1673400700 1 --2- 172.21.15.153:0/1984324995 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f166006e460 0x7f1660070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:48:27.851 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:27.855+0000 7f1673400700 1 -- 172.21.15.153:0/1984324995 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f166808faa0 con 0x7f167c1039b0 2024-03-21T10:48:27.851 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:27.855+0000 7f167aa00700 1 --2- 172.21.15.153:0/1984324995 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f166006e460 0x7f1660070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:48:27.852 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:27.855+0000 7f167aa00700 1 --2- 172.21.15.153:0/1984324995 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f166006e460 0x7f1660070920 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f166c006fd0 tx=0x7f166c008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:48:27.856 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:27.860+0000 7f1673400700 1 -- 172.21.15.153:0/1984324995 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f166805c6a0 con 0x7f167c1039b0 2024-03-21T10:48:27.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:48:27 smithi174 ceph-mon[88004]: pgmap v1099: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:48:28.098 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:28.101+0000 7f1681800700 1 -- 172.21.15.153:0/1984324995 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f167c199320 con 0x7f166006e460 2024-03-21T10:48:28.100 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:28.103+0000 7f1673400700 1 -- 172.21.15.153:0/1984324995 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f167c199320 con 0x7f166006e460 2024-03-21T10:48:28.103 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:28.106+0000 7f1681800700 1 -- 172.21.15.153:0/1984324995 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f166006e460 msgr2=0x7f1660070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:48:28.103 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:28.106+0000 7f1681800700 1 --2- 172.21.15.153:0/1984324995 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f166006e460 0x7f1660070920 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f166c006fd0 tx=0x7f166c008040 comp rx=0 tx=0).stop 2024-03-21T10:48:28.103 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:28.106+0000 7f1681800700 1 -- 172.21.15.153:0/1984324995 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f167c1039b0 msgr2=0x7f167c19ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:48:28.103 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:28.106+0000 7f1681800700 1 --2- 172.21.15.153:0/1984324995 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f167c1039b0 0x7f167c19ee00 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f166800bda0 tx=0x7f166800be80 comp rx=0 tx=0).stop 2024-03-21T10:48:28.103 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:28.106+0000 7f1681800700 1 -- 172.21.15.153:0/1984324995 shutdown_connections 2024-03-21T10:48:28.103 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:28.106+0000 7f1681800700 1 --2- 172.21.15.153:0/1984324995 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f167c103000 0x7f167c19e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:48:28.104 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:28.106+0000 7f1681800700 1 --2- 172.21.15.153:0/1984324995 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f166006e460 0x7f1660070920 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:48:28.104 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:28.106+0000 7f1681800700 1 --2- 172.21.15.153:0/1984324995 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f167c1039b0 0x7f167c19ee00 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:48:28.104 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:28.106+0000 7f1681800700 1 -- 172.21.15.153:0/1984324995 >> 172.21.15.153:0/1984324995 conn(0x7f167c0fe8d0 msgr2=0x7f167c0fffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:48:28.104 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:28.106+0000 7f1681800700 1 -- 172.21.15.153:0/1984324995 shutdown_connections 2024-03-21T10:48:28.104 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:28.107+0000 7f1681800700 1 -- 172.21.15.153:0/1984324995 wait complete. 2024-03-21T10:48:28.119 INFO:teuthology.orchestra.run.smithi153.stdout:true 2024-03-21T10:48:28.262 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:28.265+0000 7fa21b000700 1 -- 172.21.15.153:0/3980519396 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa21410d280 msgr2=0x7fa21410d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:48:28.262 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:28.265+0000 7fa21b000700 1 --2- 172.21.15.153:0/3980519396 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa21410d280 0x7fa21410d660 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fa204009910 tx=0x7fa204009c20 comp rx=0 tx=0).stop 2024-03-21T10:48:28.262 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:28.265+0000 7fa21b000700 1 -- 172.21.15.153:0/3980519396 shutdown_connections 2024-03-21T10:48:28.262 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:28.265+0000 7fa21b000700 1 --2- 172.21.15.153:0/3980519396 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2141071d0 0x7fa214107650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:48:28.262 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:28.265+0000 7fa21b000700 1 --2- 172.21.15.153:0/3980519396 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa21410d280 0x7fa21410d660 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:48:28.262 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:28.265+0000 7fa21b000700 1 -- 172.21.15.153:0/3980519396 >> 172.21.15.153:0/3980519396 conn(0x7fa2140769e0 msgr2=0x7fa214078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:48:28.262 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:28.265+0000 7fa21b000700 1 -- 172.21.15.153:0/3980519396 shutdown_connections 2024-03-21T10:48:28.262 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:28.265+0000 7fa21b000700 1 -- 172.21.15.153:0/3980519396 wait complete. 2024-03-21T10:48:28.263 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:28.266+0000 7fa21b000700 1 Processor -- start 2024-03-21T10:48:28.263 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:28.266+0000 7fa21b000700 1 -- start start 2024-03-21T10:48:28.264 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:28.266+0000 7fa21b000700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2141071d0 0x7fa21419ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:48:28.264 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:28.266+0000 7fa21b000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa21410d280 0x7fa21419d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:48:28.264 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:28.267+0000 7fa21b000700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa21419d810 con 0x7fa21410d280 2024-03-21T10:48:28.264 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:28.267+0000 7fa21b000700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa21419d980 con 0x7fa2141071d0 2024-03-21T10:48:28.264 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:28.267+0000 7fa213400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa21410d280 0x7fa21419d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:48:28.264 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:28.267+0000 7fa213400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa21410d280 0x7fa21419d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:53986/0 (socket says 172.21.15.153:53986) 2024-03-21T10:48:28.264 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:28.267+0000 7fa213400700 1 -- 172.21.15.153:0/2321360826 learned_addr learned my addr 172.21.15.153:0/2321360826 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:48:28.264 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:28.267+0000 7fa213e00700 1 --2- 172.21.15.153:0/2321360826 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2141071d0 0x7fa21419ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:48:28.265 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:28.267+0000 7fa213400700 1 -- 172.21.15.153:0/2321360826 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2141071d0 msgr2=0x7fa21419ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:48:28.265 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:28.267+0000 7fa213400700 1 --2- 172.21.15.153:0/2321360826 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2141071d0 0x7fa21419ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:48:28.265 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:28.267+0000 7fa213400700 1 -- 172.21.15.153:0/2321360826 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2040095f0 con 0x7fa21410d280 2024-03-21T10:48:28.265 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:28.268+0000 7fa213400700 1 --2- 172.21.15.153:0/2321360826 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa21410d280 0x7fa21419d210 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fa20000c930 tx=0x7fa20000ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:48:28.265 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:28.268+0000 7fa210c00700 1 -- 172.21.15.153:0/2321360826 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa200007ad0 con 0x7fa21410d280 2024-03-21T10:48:28.265 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:28.268+0000 7fa210c00700 1 -- 172.21.15.153:0/2321360826 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa200007c30 con 0x7fa21410d280 2024-03-21T10:48:28.266 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:28.268+0000 7fa210c00700 1 -- 172.21.15.153:0/2321360826 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa200018610 con 0x7fa21410d280 2024-03-21T10:48:28.266 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:28.268+0000 7fa21b000700 1 -- 172.21.15.153:0/2321360826 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2141a0720 con 0x7fa21410d280 2024-03-21T10:48:28.266 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:28.268+0000 7fa21b000700 1 -- 172.21.15.153:0/2321360826 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2141a0c50 con 0x7fa21410d280 2024-03-21T10:48:28.266 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:28.269+0000 7fa21b000700 1 -- 172.21.15.153:0/2321360826 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa21404ea90 con 0x7fa21410d280 2024-03-21T10:48:28.267 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:28.270+0000 7fa210c00700 1 -- 172.21.15.153:0/2321360826 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fa20000f450 con 0x7fa21410d280 2024-03-21T10:48:28.268 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:28.272+0000 7fa210c00700 1 --2- 172.21.15.153:0/2321360826 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa1f806e460 0x7fa1f8070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:48:28.269 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:28.272+0000 7fa210c00700 1 -- 172.21.15.153:0/2321360826 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fa20008e7b0 con 0x7fa21410d280 2024-03-21T10:48:28.269 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:28.272+0000 7fa213e00700 1 --2- 172.21.15.153:0/2321360826 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa1f806e460 0x7fa1f8070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:48:28.269 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:28.273+0000 7fa213e00700 1 --2- 172.21.15.153:0/2321360826 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa1f806e460 0x7fa1f8070920 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fa204006010 tx=0x7fa20401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:48:28.273 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:28.276+0000 7fa210c00700 1 -- 172.21.15.153:0/2321360826 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa2000577d0 con 0x7fa21410d280 2024-03-21T10:48:28.512 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:28.515+0000 7fa21b000700 1 -- 172.21.15.153:0/2321360826 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa2141a1210 con 0x7fa1f806e460 2024-03-21T10:48:28.515 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:28.518+0000 7fa210c00700 1 -- 172.21.15.153:0/2321360826 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fa2141a1210 con 0x7fa1f806e460 2024-03-21T10:48:28.517 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:28.520+0000 7fa21b000700 1 -- 172.21.15.153:0/2321360826 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa1f806e460 msgr2=0x7fa1f8070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:48:28.517 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:28.520+0000 7fa21b000700 1 --2- 172.21.15.153:0/2321360826 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa1f806e460 0x7fa1f8070920 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fa204006010 tx=0x7fa20401a040 comp rx=0 tx=0).stop 2024-03-21T10:48:28.517 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:28.520+0000 7fa21b000700 1 -- 172.21.15.153:0/2321360826 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa21410d280 msgr2=0x7fa21419d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:48:28.517 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:28.520+0000 7fa21b000700 1 --2- 172.21.15.153:0/2321360826 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa21410d280 0x7fa21419d210 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fa20000c930 tx=0x7fa20000ccf0 comp rx=0 tx=0).stop 2024-03-21T10:48:28.518 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:28.521+0000 7fa21b000700 1 -- 172.21.15.153:0/2321360826 shutdown_connections 2024-03-21T10:48:28.518 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:28.521+0000 7fa21b000700 1 --2- 172.21.15.153:0/2321360826 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2141071d0 0x7fa21419ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:48:28.518 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:28.521+0000 7fa21b000700 1 --2- 172.21.15.153:0/2321360826 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa1f806e460 0x7fa1f8070920 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:48:28.518 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:28.521+0000 7fa21b000700 1 --2- 172.21.15.153:0/2321360826 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa21410d280 0x7fa21419d210 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:48:28.518 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:28.521+0000 7fa21b000700 1 -- 172.21.15.153:0/2321360826 >> 172.21.15.153:0/2321360826 conn(0x7fa2140769e0 msgr2=0x7fa214078320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:48:28.518 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:28.521+0000 7fa21b000700 1 -- 172.21.15.153:0/2321360826 shutdown_connections 2024-03-21T10:48:28.518 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:28.521+0000 7fa21b000700 1 -- 172.21.15.153:0/2321360826 wait complete. 2024-03-21T10:48:28.676 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:28.679+0000 7f7426000700 1 -- 172.21.15.153:0/3398778990 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7420068670 msgr2=0x7f7420068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:48:28.677 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:28.679+0000 7f7426000700 1 --2- 172.21.15.153:0/3398778990 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7420068670 0x7f7420068a50 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f7410009910 tx=0x7f7410009c20 comp rx=0 tx=0).stop 2024-03-21T10:48:28.677 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:28.679+0000 7f7426000700 1 -- 172.21.15.153:0/3398778990 shutdown_connections 2024-03-21T10:48:28.677 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:28.679+0000 7f7426000700 1 --2- 172.21.15.153:0/3398778990 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7420069020 0x7f7420105790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:48:28.677 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:28.679+0000 7f7426000700 1 --2- 172.21.15.153:0/3398778990 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7420068670 0x7f7420068a50 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:48:28.677 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:28.679+0000 7f7426000700 1 -- 172.21.15.153:0/3398778990 >> 172.21.15.153:0/3398778990 conn(0x7f74200755a0 msgr2=0x7f74200759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:48:28.677 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:28.679+0000 7f7426000700 1 -- 172.21.15.153:0/3398778990 shutdown_connections 2024-03-21T10:48:28.677 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:28.679+0000 7f7426000700 1 -- 172.21.15.153:0/3398778990 wait complete. 2024-03-21T10:48:28.678 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:28.680+0000 7f7426000700 1 Processor -- start 2024-03-21T10:48:28.678 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:28.680+0000 7f7426000700 1 -- start start 2024-03-21T10:48:28.678 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:28.681+0000 7f7426000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7420068670 0x7f742019c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:48:28.678 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:28.681+0000 7f7426000700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7420069020 0x7f742019ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:48:28.678 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:28.681+0000 7f7426000700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f742019d050 con 0x7f7420068670 2024-03-21T10:48:28.679 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:28.681+0000 7f7426000700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7420196590 con 0x7f7420069020 2024-03-21T10:48:28.679 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:28.681+0000 7f741f400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7420068670 0x7f742019c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:48:28.679 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:28.681+0000 7f741f400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7420068670 0x7f742019c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:54000/0 (socket says 172.21.15.153:54000) 2024-03-21T10:48:28.679 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:28.681+0000 7f741f400700 1 -- 172.21.15.153:0/3311084037 learned_addr learned my addr 172.21.15.153:0/3311084037 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:48:28.679 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:28.681+0000 7f741ea00700 1 --2- 172.21.15.153:0/3311084037 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7420069020 0x7f742019ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:48:28.679 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:28.682+0000 7f741f400700 1 -- 172.21.15.153:0/3311084037 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7420069020 msgr2=0x7f742019ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:48:28.679 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:28.682+0000 7f741f400700 1 --2- 172.21.15.153:0/3311084037 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7420069020 0x7f742019ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:48:28.679 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:28.682+0000 7f741f400700 1 -- 172.21.15.153:0/3311084037 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74100095f0 con 0x7f7420068670 2024-03-21T10:48:28.680 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:28.682+0000 7f741f400700 1 --2- 172.21.15.153:0/3311084037 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7420068670 0x7f742019c510 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f741000b5c0 tx=0x7f741000be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:48:28.680 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:28.682+0000 7f7417e00700 1 -- 172.21.15.153:0/3311084037 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f741001d070 con 0x7f7420068670 2024-03-21T10:48:28.680 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:28.682+0000 7f7417e00700 1 -- 172.21.15.153:0/3311084037 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f741000f460 con 0x7f7420068670 2024-03-21T10:48:28.680 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:28.682+0000 7f7426000700 1 -- 172.21.15.153:0/3311084037 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7420196810 con 0x7f7420068670 2024-03-21T10:48:28.680 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:28.682+0000 7f7417e00700 1 -- 172.21.15.153:0/3311084037 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7410017610 con 0x7f7420068670 2024-03-21T10:48:28.680 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:28.683+0000 7f7426000700 1 -- 172.21.15.153:0/3311084037 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7420196d00 con 0x7f7420068670 2024-03-21T10:48:28.681 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:28.684+0000 7f7426000700 1 -- 172.21.15.153:0/3311084037 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7420109130 con 0x7f7420068670 2024-03-21T10:48:28.681 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:28.685+0000 7f7417e00700 1 -- 172.21.15.153:0/3311084037 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f741000f5d0 con 0x7f7420068670 2024-03-21T10:48:28.682 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:28.685+0000 7f7417e00700 1 --2- 172.21.15.153:0/3311084037 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f740406e460 0x7f7404070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:48:28.682 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:28.686+0000 7f7417e00700 1 -- 172.21.15.153:0/3311084037 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f741008faf0 con 0x7f7420068670 2024-03-21T10:48:28.683 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:28.686+0000 7f741ea00700 1 --2- 172.21.15.153:0/3311084037 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f740406e460 0x7f7404070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:48:28.683 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:28.686+0000 7f741ea00700 1 --2- 172.21.15.153:0/3311084037 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f740406e460 0x7f7404070920 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f740c005ea0 tx=0x7f740c005e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:48:28.687 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:28.690+0000 7f7417e00700 1 -- 172.21.15.153:0/3311084037 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f741005c6e0 con 0x7f7420068670 2024-03-21T10:48:28.912 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:28.915+0000 7f7426000700 1 -- 172.21.15.153:0/3311084037 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f7420196f20 con 0x7f740406e460 2024-03-21T10:48:28.943 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:28.946+0000 7f7417e00700 1 -- 172.21.15.153:0/3311084037 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f7420196f20 con 0x7f740406e460 2024-03-21T10:48:28.944 INFO:teuthology.orchestra.run.smithi153.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-21T10:48:28.944 INFO:teuthology.orchestra.run.smithi153.stdout:alertmanager.smithi153 smithi153 *:9093,9094 running (37m) 13m ago 51m 15.6M - 0.25.0 c8568f914cd2 9ed3f2200b5f 2024-03-21T10:48:28.944 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi153 smithi153 running (51m) 13m ago 51m 10.4M - 18.2.1 5be31c24972a 5b61994396a6 2024-03-21T10:48:28.944 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi174 smithi174 running (49m) 13m ago 49m 10.1M - 18.2.1 5be31c24972a 4f7a5fc3ac28 2024-03-21T10:48:28.944 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi153 smithi153 running (33m) 13m ago 51m 6656k - 19.0.0-2231-g2edfee68 237e43c7a65f 32c21104d5b0 2024-03-21T10:48:28.944 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi174 smithi174 running (33m) 13m ago 49m 6660k - 19.0.0-2231-g2edfee68 237e43c7a65f a04e48967ab6 2024-03-21T10:48:28.944 INFO:teuthology.orchestra.run.smithi153.stdout:grafana.smithi153 smithi153 *:3000 running (36m) 13m ago 50m 75.2M - 9.4.12 2bacad6d85d8 ee24026ab786 2024-03-21T10:48:28.944 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.iuyluj smithi153 running (28m) 13m ago 43m 18.7M - 19.0.0-2231-g2edfee68 237e43c7a65f 87aff8a5de70 2024-03-21T10:48:28.944 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.zipckr smithi153 running (29m) 13m ago 43m 18.9M - 19.0.0-2231-g2edfee68 237e43c7a65f ccc262bd5dd3 2024-03-21T10:48:28.945 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.nhozag smithi174 running (13m) 13m ago 43m 10.9M - 19.0.0-2231-g2edfee68 237e43c7a65f 9e02aea72c96 2024-03-21T10:48:28.945 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.yyimst smithi174 running (28m) 13m ago 43m 18.5M - 19.0.0-2231-g2edfee68 237e43c7a65f fc1962f9351e 2024-03-21T10:48:28.945 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi153.tluxni smithi153 *:8443,9283,8765 running (40m) 13m ago 52m 512M - 19.0.0-2231-g2edfee68 237e43c7a65f 7ffee0e55382 2024-03-21T10:48:28.945 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi174.qpcwat smithi174 *:8443,9283,8765 running (38m) 13m ago 49m 459M - 19.0.0-2231-g2edfee68 237e43c7a65f 748968f216ab 2024-03-21T10:48:28.945 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi153 smithi153 running (35m) 13m ago 53m 70.8M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f dd8038303dc3 2024-03-21T10:48:28.945 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi174 smithi174 running (34m) 13m ago 49m 51.9M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f 29d78a788f2c 2024-03-21T10:48:28.945 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi153 smithi153 *:9100 running (38m) 13m ago 51m 8140k - 1.5.0 0da6a335fe13 bb835edc0e0b 2024-03-21T10:48:28.945 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi174 smithi174 *:9100 running (38m) 13m ago 49m 8610k - 1.5.0 0da6a335fe13 7c4ecad790fa 2024-03-21T10:48:28.945 INFO:teuthology.orchestra.run.smithi153.stdout:osd.0 smithi153 running (32m) 13m ago 48m 118M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 507900fd4791 2024-03-21T10:48:28.945 INFO:teuthology.orchestra.run.smithi153.stdout:osd.1 smithi153 running (32m) 13m ago 47m 93.0M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 0d646452e9c1 2024-03-21T10:48:28.945 INFO:teuthology.orchestra.run.smithi153.stdout:osd.2 smithi153 running (31m) 13m ago 47m 91.2M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 1326f2afdfcd 2024-03-21T10:48:28.945 INFO:teuthology.orchestra.run.smithi153.stdout:osd.3 smithi174 running (31m) 13m ago 46m 87.5M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 70aafac71ad5 2024-03-21T10:48:28.945 INFO:teuthology.orchestra.run.smithi153.stdout:osd.4 smithi174 running (30m) 13m ago 46m 95.8M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 0d746ce6e79d 2024-03-21T10:48:28.945 INFO:teuthology.orchestra.run.smithi153.stdout:osd.5 smithi174 running (29m) 13m ago 45m 90.3M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 53e5f35b8cac 2024-03-21T10:48:28.945 INFO:teuthology.orchestra.run.smithi153.stdout:prometheus.smithi153 smithi153 *:9095 running (37m) 13m ago 50m 88.1M - 2.43.0 a07b618ecd1d 7d95561e50c2 2024-03-21T10:48:28.946 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:28.949+0000 7f7426000700 1 -- 172.21.15.153:0/3311084037 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f740406e460 msgr2=0x7f7404070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:48:28.947 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:28.949+0000 7f7426000700 1 --2- 172.21.15.153:0/3311084037 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f740406e460 0x7f7404070920 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f740c005ea0 tx=0x7f740c005e30 comp rx=0 tx=0).stop 2024-03-21T10:48:28.947 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:28.949+0000 7f7426000700 1 -- 172.21.15.153:0/3311084037 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7420068670 msgr2=0x7f742019c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:48:28.947 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:28.949+0000 7f7426000700 1 --2- 172.21.15.153:0/3311084037 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7420068670 0x7f742019c510 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f741000b5c0 tx=0x7f741000be00 comp rx=0 tx=0).stop 2024-03-21T10:48:28.947 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:28.949+0000 7f7426000700 1 -- 172.21.15.153:0/3311084037 shutdown_connections 2024-03-21T10:48:28.947 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:28.949+0000 7f7426000700 1 --2- 172.21.15.153:0/3311084037 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f740406e460 0x7f7404070920 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:48:28.947 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:28.949+0000 7f7426000700 1 --2- 172.21.15.153:0/3311084037 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7420068670 0x7f742019c510 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:48:28.947 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:28.949+0000 7f7426000700 1 --2- 172.21.15.153:0/3311084037 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7420069020 0x7f742019ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:48:28.947 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:28.949+0000 7f7426000700 1 -- 172.21.15.153:0/3311084037 >> 172.21.15.153:0/3311084037 conn(0x7f74200755a0 msgr2=0x7f74200fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:48:28.947 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:28.950+0000 7f7426000700 1 -- 172.21.15.153:0/3311084037 shutdown_connections 2024-03-21T10:48:28.947 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:28.950+0000 7f7426000700 1 -- 172.21.15.153:0/3311084037 wait complete. 2024-03-21T10:48:29.087 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.090+0000 7f1a38000700 1 -- 172.21.15.153:0/3301657008 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1a30102ff0 msgr2=0x7f1a301033d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:48:29.088 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.090+0000 7f1a38000700 1 --2- 172.21.15.153:0/3301657008 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1a30102ff0 0x7f1a301033d0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f1a24009910 tx=0x7f1a24009c20 comp rx=0 tx=0).stop 2024-03-21T10:48:29.088 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.090+0000 7f1a38000700 1 -- 172.21.15.153:0/3301657008 shutdown_connections 2024-03-21T10:48:29.088 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.090+0000 7f1a38000700 1 --2- 172.21.15.153:0/3301657008 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1a301039a0 0x7f1a301079f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:48:29.088 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.090+0000 7f1a38000700 1 --2- 172.21.15.153:0/3301657008 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1a30102ff0 0x7f1a301033d0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:48:29.088 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.090+0000 7f1a38000700 1 -- 172.21.15.153:0/3301657008 >> 172.21.15.153:0/3301657008 conn(0x7f1a300fe880 msgr2=0x7f1a30100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:48:29.088 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.090+0000 7f1a38000700 1 -- 172.21.15.153:0/3301657008 shutdown_connections 2024-03-21T10:48:29.088 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.090+0000 7f1a38000700 1 -- 172.21.15.153:0/3301657008 wait complete. 2024-03-21T10:48:29.089 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.091+0000 7f1a38000700 1 Processor -- start 2024-03-21T10:48:29.089 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.091+0000 7f1a38000700 1 -- start start 2024-03-21T10:48:29.089 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.092+0000 7f1a38000700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1a30102ff0 0x7f1a3019e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:48:29.089 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.092+0000 7f1a38000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1a301039a0 0x7f1a3019ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:48:29.090 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.092+0000 7f1a38000700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a3019f3f0 con 0x7f1a301039a0 2024-03-21T10:48:29.090 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.092+0000 7f1a38000700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a301988a0 con 0x7f1a30102ff0 2024-03-21T10:48:29.090 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.092+0000 7f1a34c00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1a301039a0 0x7f1a3019ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:48:29.090 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.092+0000 7f1a34c00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1a301039a0 0x7f1a3019ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:54020/0 (socket says 172.21.15.153:54020) 2024-03-21T10:48:29.090 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.092+0000 7f1a34c00700 1 -- 172.21.15.153:0/3712687605 learned_addr learned my addr 172.21.15.153:0/3712687605 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:48:29.090 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.092+0000 7f1a35600700 1 --2- 172.21.15.153:0/3712687605 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1a30102ff0 0x7f1a3019e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:48:29.090 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.092+0000 7f1a34c00700 1 -- 172.21.15.153:0/3712687605 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1a30102ff0 msgr2=0x7f1a3019e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:48:29.090 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.093+0000 7f1a34c00700 1 --2- 172.21.15.153:0/3712687605 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1a30102ff0 0x7f1a3019e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:48:29.090 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.093+0000 7f1a34c00700 1 -- 172.21.15.153:0/3712687605 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a240095f0 con 0x7f1a301039a0 2024-03-21T10:48:29.091 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.093+0000 7f1a34c00700 1 --2- 172.21.15.153:0/3712687605 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1a301039a0 0x7f1a3019ed60 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c00b480 tx=0x7f1a1c00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:48:29.091 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.093+0000 7f1a2e000700 1 -- 172.21.15.153:0/3712687605 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1a1c0118c0 con 0x7f1a301039a0 2024-03-21T10:48:29.091 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.093+0000 7f1a38000700 1 -- 172.21.15.153:0/3712687605 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a30198b80 con 0x7f1a301039a0 2024-03-21T10:48:29.091 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.093+0000 7f1a2e000700 1 -- 172.21.15.153:0/3712687605 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1a1c010380 con 0x7f1a301039a0 2024-03-21T10:48:29.091 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.093+0000 7f1a2e000700 1 -- 172.21.15.153:0/3712687605 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1a1c00f4f0 con 0x7f1a301039a0 2024-03-21T10:48:29.091 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.093+0000 7f1a38000700 1 -- 172.21.15.153:0/3712687605 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a301990d0 con 0x7f1a301039a0 2024-03-21T10:48:29.092 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.094+0000 7f1a38000700 1 -- 172.21.15.153:0/3712687605 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a3010b390 con 0x7f1a301039a0 2024-03-21T10:48:29.092 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.095+0000 7f1a2e000700 1 -- 172.21.15.153:0/3712687605 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f1a1c0104f0 con 0x7f1a301039a0 2024-03-21T10:48:29.093 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.096+0000 7f1a2e000700 1 --2- 172.21.15.153:0/3712687605 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f1a1806e460 0x7f1a18070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:48:29.093 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.096+0000 7f1a2e000700 1 -- 172.21.15.153:0/3712687605 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f1a1c08e120 con 0x7f1a301039a0 2024-03-21T10:48:29.093 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.097+0000 7f1a35600700 1 --2- 172.21.15.153:0/3712687605 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f1a1806e460 0x7f1a18070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:48:29.094 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.097+0000 7f1a35600700 1 --2- 172.21.15.153:0/3712687605 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f1a1806e460 0x7f1a18070920 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f1a2400b5c0 tx=0x7f1a2401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:48:29.097 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.101+0000 7f1a2e000700 1 -- 172.21.15.153:0/3712687605 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f1a1c05acf0 con 0x7f1a301039a0 2024-03-21T10:48:29.407 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.409+0000 7f1a38000700 1 -- 172.21.15.153:0/3712687605 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f1a30199db0 con 0x7f1a301039a0 2024-03-21T10:48:29.407 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.411+0000 7f1a2e000700 1 -- 172.21.15.153:0/3712687605 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f1a1c05a880 con 0x7f1a301039a0 2024-03-21T10:48:29.408 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:48:29.408 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": { 2024-03-21T10:48:29.408 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:48:29.408 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:48:29.408 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": { 2024-03-21T10:48:29.408 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:48:29.408 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:48:29.408 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": { 2024-03-21T10:48:29.408 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 6 2024-03-21T10:48:29.408 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:48:29.409 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": { 2024-03-21T10:48:29.409 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 4 2024-03-21T10:48:29.409 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:48:29.409 INFO:teuthology.orchestra.run.smithi153.stdout: "overall": { 2024-03-21T10:48:29.409 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 14 2024-03-21T10:48:29.409 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-03-21T10:48:29.409 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:48:29.411 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.414+0000 7f1a38000700 1 -- 172.21.15.153:0/3712687605 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f1a1806e460 msgr2=0x7f1a18070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:48:29.411 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.414+0000 7f1a38000700 1 --2- 172.21.15.153:0/3712687605 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f1a1806e460 0x7f1a18070920 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f1a2400b5c0 tx=0x7f1a2401a040 comp rx=0 tx=0).stop 2024-03-21T10:48:29.411 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.414+0000 7f1a38000700 1 -- 172.21.15.153:0/3712687605 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1a301039a0 msgr2=0x7f1a3019ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:48:29.411 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.414+0000 7f1a38000700 1 --2- 172.21.15.153:0/3712687605 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1a301039a0 0x7f1a3019ed60 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c00b480 tx=0x7f1a1c00b840 comp rx=0 tx=0).stop 2024-03-21T10:48:29.411 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.414+0000 7f1a38000700 1 -- 172.21.15.153:0/3712687605 shutdown_connections 2024-03-21T10:48:29.411 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.414+0000 7f1a38000700 1 --2- 172.21.15.153:0/3712687605 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1a30102ff0 0x7f1a3019e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:48:29.411 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.414+0000 7f1a38000700 1 --2- 172.21.15.153:0/3712687605 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f1a1806e460 0x7f1a18070920 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:48:29.411 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.414+0000 7f1a38000700 1 --2- 172.21.15.153:0/3712687605 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1a301039a0 0x7f1a3019ed60 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:48:29.411 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.414+0000 7f1a38000700 1 -- 172.21.15.153:0/3712687605 >> 172.21.15.153:0/3712687605 conn(0x7f1a300fe880 msgr2=0x7f1a30100000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:48:29.411 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.414+0000 7f1a38000700 1 -- 172.21.15.153:0/3712687605 shutdown_connections 2024-03-21T10:48:29.412 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.414+0000 7f1a38000700 1 -- 172.21.15.153:0/3712687605 wait complete. 2024-03-21T10:48:29.564 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.567+0000 7fa579c00700 1 -- 172.21.15.153:0/135744384 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa574073a50 msgr2=0x7fa574111490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:48:29.564 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.567+0000 7fa579c00700 1 --2- 172.21.15.153:0/135744384 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa574073a50 0x7fa574111490 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fa560009910 tx=0x7fa560009c20 comp rx=0 tx=0).stop 2024-03-21T10:48:29.564 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.567+0000 7fa579c00700 1 -- 172.21.15.153:0/135744384 shutdown_connections 2024-03-21T10:48:29.564 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.567+0000 7fa579c00700 1 --2- 172.21.15.153:0/135744384 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa574073a50 0x7fa574111490 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:48:29.565 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.567+0000 7fa579c00700 1 --2- 172.21.15.153:0/135744384 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa574073130 0x7fa574073510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:48:29.565 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.567+0000 7fa579c00700 1 -- 172.21.15.153:0/135744384 >> 172.21.15.153:0/135744384 conn(0x7fa5740fc4f0 msgr2=0x7fa5740fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:48:29.566 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.567+0000 7fa579c00700 1 -- 172.21.15.153:0/135744384 shutdown_connections 2024-03-21T10:48:29.566 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.567+0000 7fa579c00700 1 -- 172.21.15.153:0/135744384 wait complete. 2024-03-21T10:48:29.566 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.568+0000 7fa579c00700 1 Processor -- start 2024-03-21T10:48:29.566 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.568+0000 7fa579c00700 1 -- start start 2024-03-21T10:48:29.566 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.569+0000 7fa579c00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa574073130 0x7fa5741a29c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:48:29.566 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.569+0000 7fa579c00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa574073a50 0x7fa5741a2f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:48:29.567 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.569+0000 7fa579c00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5741a3500 con 0x7fa574073a50 2024-03-21T10:48:29.567 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.569+0000 7fa579c00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa57419ca40 con 0x7fa574073130 2024-03-21T10:48:29.567 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.569+0000 7fa573400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa574073a50 0x7fa5741a2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:48:29.567 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.569+0000 7fa573400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa574073a50 0x7fa5741a2f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:54028/0 (socket says 172.21.15.153:54028) 2024-03-21T10:48:29.567 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.569+0000 7fa573400700 1 -- 172.21.15.153:0/2427138192 learned_addr learned my addr 172.21.15.153:0/2427138192 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:48:29.567 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.569+0000 7fa573e00700 1 --2- 172.21.15.153:0/2427138192 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa574073130 0x7fa5741a29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:48:29.567 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.569+0000 7fa573400700 1 -- 172.21.15.153:0/2427138192 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa574073130 msgr2=0x7fa5741a29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:48:29.567 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.569+0000 7fa573400700 1 --2- 172.21.15.153:0/2427138192 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa574073130 0x7fa5741a29c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:48:29.567 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.569+0000 7fa573400700 1 -- 172.21.15.153:0/2427138192 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5600095f0 con 0x7fa574073a50 2024-03-21T10:48:29.567 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.569+0000 7fa573e00700 1 --2- 172.21.15.153:0/2427138192 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa574073130 0x7fa5741a29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:48:29.568 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.570+0000 7fa573400700 1 --2- 172.21.15.153:0/2427138192 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa574073a50 0x7fa5741a2f00 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fa560006010 tx=0x7fa56000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:48:29.568 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.570+0000 7fa570c00700 1 -- 172.21.15.153:0/2427138192 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa56001d070 con 0x7fa574073a50 2024-03-21T10:48:29.568 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.570+0000 7fa570c00700 1 -- 172.21.15.153:0/2427138192 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa560004fb0 con 0x7fa574073a50 2024-03-21T10:48:29.568 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.570+0000 7fa570c00700 1 -- 172.21.15.153:0/2427138192 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa560022ea0 con 0x7fa574073a50 2024-03-21T10:48:29.568 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.570+0000 7fa579c00700 1 -- 172.21.15.153:0/2427138192 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa57419cd20 con 0x7fa574073a50 2024-03-21T10:48:29.568 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.570+0000 7fa579c00700 1 -- 172.21.15.153:0/2427138192 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa57419d270 con 0x7fa574073a50 2024-03-21T10:48:29.568 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.571+0000 7fa579c00700 1 -- 172.21.15.153:0/2427138192 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa57410ec10 con 0x7fa574073a50 2024-03-21T10:48:29.569 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.572+0000 7fa570c00700 1 -- 172.21.15.153:0/2427138192 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fa560022470 con 0x7fa574073a50 2024-03-21T10:48:29.570 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.573+0000 7fa570c00700 1 --2- 172.21.15.153:0/2427138192 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa55806e460 0x7fa558070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:48:29.570 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.573+0000 7fa570c00700 1 -- 172.21.15.153:0/2427138192 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fa56008f960 con 0x7fa574073a50 2024-03-21T10:48:29.570 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.573+0000 7fa573e00700 1 --2- 172.21.15.153:0/2427138192 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa55806e460 0x7fa558070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:48:29.571 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.574+0000 7fa573e00700 1 --2- 172.21.15.153:0/2427138192 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa55806e460 0x7fa558070920 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fa564005fd0 tx=0x7fa564005e20 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:48:29.575 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.578+0000 7fa570c00700 1 -- 172.21.15.153:0/2427138192 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa56005c680 con 0x7fa574073a50 2024-03-21T10:48:29.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:48:29 smithi153 ceph-mon[113733]: pgmap v1100: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:48:29.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:48:29 smithi153 ceph-mon[113733]: from='client.35938 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:48:29.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:48:29 smithi153 ceph-mon[113733]: from='client.35942 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:48:29.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:48:29 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/3712687605' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:48:29.828 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.831+0000 7fa579c00700 1 -- 172.21.15.153:0/2427138192 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fa57419ceb0 con 0x7fa574073a50 2024-03-21T10:48:29.829 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.832+0000 7fa570c00700 1 -- 172.21.15.153:0/2427138192 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1876 (secure 0 0 0) 0x7fa560027770 con 0x7fa574073a50 2024-03-21T10:48:29.830 INFO:teuthology.orchestra.run.smithi153.stdout:e32 2024-03-21T10:48:29.830 INFO:teuthology.orchestra.run.smithi153.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-21T10:48:29.830 INFO:teuthology.orchestra.run.smithi153.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:48:29.830 INFO:teuthology.orchestra.run.smithi153.stdout:legacy client fscid: 1 2024-03-21T10:48:29.830 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:48:29.830 INFO:teuthology.orchestra.run.smithi153.stdout:Filesystem 'cephfs' (1) 2024-03-21T10:48:29.830 INFO:teuthology.orchestra.run.smithi153.stdout:fs_name cephfs 2024-03-21T10:48:29.830 INFO:teuthology.orchestra.run.smithi153.stdout:epoch 23 2024-03-21T10:48:29.830 INFO:teuthology.orchestra.run.smithi153.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-21T10:48:29.830 INFO:teuthology.orchestra.run.smithi153.stdout:created 2024-03-21T10:04:27.086576+0000 2024-03-21T10:48:29.830 INFO:teuthology.orchestra.run.smithi153.stdout:modified 2024-03-21T10:19:05.022092+0000 2024-03-21T10:48:29.831 INFO:teuthology.orchestra.run.smithi153.stdout:tableserver 0 2024-03-21T10:48:29.831 INFO:teuthology.orchestra.run.smithi153.stdout:root 0 2024-03-21T10:48:29.831 INFO:teuthology.orchestra.run.smithi153.stdout:session_timeout 60 2024-03-21T10:48:29.831 INFO:teuthology.orchestra.run.smithi153.stdout:session_autoclose 300 2024-03-21T10:48:29.831 INFO:teuthology.orchestra.run.smithi153.stdout:max_file_size 1099511627776 2024-03-21T10:48:29.831 INFO:teuthology.orchestra.run.smithi153.stdout:max_xattr_size 65536 2024-03-21T10:48:29.831 INFO:teuthology.orchestra.run.smithi153.stdout:required_client_features {} 2024-03-21T10:48:29.831 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure 0 2024-03-21T10:48:29.831 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure_osd_epoch 94 2024-03-21T10:48:29.831 INFO:teuthology.orchestra.run.smithi153.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:48:29.831 INFO:teuthology.orchestra.run.smithi153.stdout:max_mds 2 2024-03-21T10:48:29.831 INFO:teuthology.orchestra.run.smithi153.stdout:in 0,1 2024-03-21T10:48:29.831 INFO:teuthology.orchestra.run.smithi153.stdout:up {} 2024-03-21T10:48:29.831 INFO:teuthology.orchestra.run.smithi153.stdout:failed 0,1 2024-03-21T10:48:29.832 INFO:teuthology.orchestra.run.smithi153.stdout:damaged 2024-03-21T10:48:29.832 INFO:teuthology.orchestra.run.smithi153.stdout:stopped 2024-03-21T10:48:29.832 INFO:teuthology.orchestra.run.smithi153.stdout:data_pools [3] 2024-03-21T10:48:29.832 INFO:teuthology.orchestra.run.smithi153.stdout:metadata_pool 2 2024-03-21T10:48:29.832 INFO:teuthology.orchestra.run.smithi153.stdout:inline_data enabled 2024-03-21T10:48:29.832 INFO:teuthology.orchestra.run.smithi153.stdout:balancer 2024-03-21T10:48:29.832 INFO:teuthology.orchestra.run.smithi153.stdout:bal_rank_mask -1 2024-03-21T10:48:29.832 INFO:teuthology.orchestra.run.smithi153.stdout:standby_count_wanted 1 2024-03-21T10:48:29.832 INFO:teuthology.orchestra.run.smithi153.stdout:qdb_cluster leader: 0 members: 2024-03-21T10:48:29.832 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:48:29.832 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:48:29.832 INFO:teuthology.orchestra.run.smithi153.stdout:Standby daemons: 2024-03-21T10:48:29.832 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:48:29.832 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.zipckr{-1:34442} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6826/1103846295,v1:172.21.15.153:6827/1103846295] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:48:29.833 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.iuyluj{-1:34474} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6828/1738630832,v1:172.21.15.153:6829/1738630832] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:48:29.833 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.yyimst{-1:44313} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.174:6824/4127786797,v1:172.21.15.174:6825/4127786797] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:48:29.833 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.nhozag{-1:44687} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.174:6826/2767517845,v1:172.21.15.174:6827/2767517845] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:48:29.833 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.835+0000 7fa579c00700 1 -- 172.21.15.153:0/2427138192 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa55806e460 msgr2=0x7fa558070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:48:29.834 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.835+0000 7fa579c00700 1 --2- 172.21.15.153:0/2427138192 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa55806e460 0x7fa558070920 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fa564005fd0 tx=0x7fa564005e20 comp rx=0 tx=0).stop 2024-03-21T10:48:29.834 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.835+0000 7fa579c00700 1 -- 172.21.15.153:0/2427138192 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa574073a50 msgr2=0x7fa5741a2f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:48:29.834 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.835+0000 7fa579c00700 1 --2- 172.21.15.153:0/2427138192 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa574073a50 0x7fa5741a2f00 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fa560006010 tx=0x7fa56000be20 comp rx=0 tx=0).stop 2024-03-21T10:48:29.834 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.835+0000 7fa579c00700 1 -- 172.21.15.153:0/2427138192 shutdown_connections 2024-03-21T10:48:29.834 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.835+0000 7fa579c00700 1 --2- 172.21.15.153:0/2427138192 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa574073130 0x7fa5741a29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:48:29.834 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.835+0000 7fa579c00700 1 --2- 172.21.15.153:0/2427138192 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa55806e460 0x7fa558070920 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:48:29.834 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.835+0000 7fa579c00700 1 --2- 172.21.15.153:0/2427138192 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa574073a50 0x7fa5741a2f00 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:48:29.834 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.835+0000 7fa579c00700 1 -- 172.21.15.153:0/2427138192 >> 172.21.15.153:0/2427138192 conn(0x7fa5740fc4f0 msgr2=0x7fa574102fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:48:29.834 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.835+0000 7fa579c00700 1 -- 172.21.15.153:0/2427138192 shutdown_connections 2024-03-21T10:48:29.834 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.835+0000 7fa579c00700 1 -- 172.21.15.153:0/2427138192 wait complete. 2024-03-21T10:48:29.834 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 32 2024-03-21T10:48:29.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:48:29 smithi174 ceph-mon[88004]: pgmap v1100: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:48:29.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:48:29 smithi174 ceph-mon[88004]: from='client.35938 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:48:29.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:48:29 smithi174 ceph-mon[88004]: from='client.35942 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:48:29.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:48:29 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/3712687605' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:48:29.984 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.987+0000 7f2672600700 1 -- 172.21.15.153:0/2121242362 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f266c106980 msgr2=0x7f266c106d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:48:29.985 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.987+0000 7f2672600700 1 --2- 172.21.15.153:0/2121242362 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f266c106980 0x7f266c106d60 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f265c009910 tx=0x7f265c009c20 comp rx=0 tx=0).stop 2024-03-21T10:48:29.985 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.987+0000 7f2672600700 1 -- 172.21.15.153:0/2121242362 shutdown_connections 2024-03-21T10:48:29.985 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.987+0000 7f2672600700 1 --2- 172.21.15.153:0/2121242362 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f266c100980 0x7f266c100e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:48:29.985 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.987+0000 7f2672600700 1 --2- 172.21.15.153:0/2121242362 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f266c106980 0x7f266c106d60 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:48:29.985 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.987+0000 7f2672600700 1 -- 172.21.15.153:0/2121242362 >> 172.21.15.153:0/2121242362 conn(0x7f266c0fc470 msgr2=0x7f266c0fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:48:29.985 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.987+0000 7f2672600700 1 -- 172.21.15.153:0/2121242362 shutdown_connections 2024-03-21T10:48:29.985 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.987+0000 7f2672600700 1 -- 172.21.15.153:0/2121242362 wait complete. 2024-03-21T10:48:29.987 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.988+0000 7f2672600700 1 Processor -- start 2024-03-21T10:48:29.987 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.988+0000 7f2672600700 1 -- start start 2024-03-21T10:48:29.987 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.989+0000 7f2672600700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f266c100980 0x7f266c19e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:48:29.987 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.989+0000 7f2672600700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f266c106980 0x7f266c19ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:48:29.988 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.989+0000 7f2672600700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f266c19f0f0 con 0x7f266c106980 2024-03-21T10:48:29.988 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.989+0000 7f2672600700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f266c1985a0 con 0x7f266c100980 2024-03-21T10:48:29.988 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.989+0000 7f265be00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f266c106980 0x7f266c19ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:48:29.988 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.989+0000 7f265be00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f266c106980 0x7f266c19ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:54048/0 (socket says 172.21.15.153:54048) 2024-03-21T10:48:29.988 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.989+0000 7f265be00700 1 -- 172.21.15.153:0/174171159 learned_addr learned my addr 172.21.15.153:0/174171159 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:48:29.988 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.990+0000 7f2671200700 1 --2- 172.21.15.153:0/174171159 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f266c100980 0x7f266c19e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:48:29.988 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.990+0000 7f265be00700 1 -- 172.21.15.153:0/174171159 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f266c100980 msgr2=0x7f266c19e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:48:29.988 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.990+0000 7f265be00700 1 --2- 172.21.15.153:0/174171159 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f266c100980 0x7f266c19e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:48:29.988 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.990+0000 7f265be00700 1 -- 172.21.15.153:0/174171159 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f265c0095f0 con 0x7f266c106980 2024-03-21T10:48:29.988 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.990+0000 7f265be00700 1 --2- 172.21.15.153:0/174171159 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f266c106980 0x7f266c19ea60 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f265400c8a0 tx=0x7f265400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:48:29.988 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.990+0000 7f266a000700 1 -- 172.21.15.153:0/174171159 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f265400cea0 con 0x7f266c106980 2024-03-21T10:48:29.988 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.990+0000 7f266a000700 1 -- 172.21.15.153:0/174171159 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2654004830 con 0x7f266c106980 2024-03-21T10:48:29.988 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.990+0000 7f2672600700 1 -- 172.21.15.153:0/174171159 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f266c198880 con 0x7f266c106980 2024-03-21T10:48:29.988 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.990+0000 7f266a000700 1 -- 172.21.15.153:0/174171159 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f26540055b0 con 0x7f266c106980 2024-03-21T10:48:29.989 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.990+0000 7f2672600700 1 -- 172.21.15.153:0/174171159 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f266c198dd0 con 0x7f266c106980 2024-03-21T10:48:29.989 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.991+0000 7f2672600700 1 -- 172.21.15.153:0/174171159 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f266c04f2e0 con 0x7f266c106980 2024-03-21T10:48:29.989 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.993+0000 7f266a000700 1 -- 172.21.15.153:0/174171159 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f26540049a0 con 0x7f266c106980 2024-03-21T10:48:29.990 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.993+0000 7f266a000700 1 --2- 172.21.15.153:0/174171159 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f264c06e460 0x7f264c070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:48:29.991 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.994+0000 7f266a000700 1 -- 172.21.15.153:0/174171159 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f265408e170 con 0x7f266c106980 2024-03-21T10:48:29.991 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.994+0000 7f2671200700 1 --2- 172.21.15.153:0/174171159 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f264c06e460 0x7f264c070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:48:29.992 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.994+0000 7f2671200700 1 --2- 172.21.15.153:0/174171159 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f264c06e460 0x7f264c070920 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f265c006010 tx=0x7f265c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:48:29.995 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:29.999+0000 7f266a000700 1 -- 172.21.15.153:0/174171159 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f265405ad40 con 0x7f266c106980 2024-03-21T10:48:30.232 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:30.235+0000 7f2672600700 1 -- 172.21.15.153:0/174171159 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f266c199250 con 0x7f264c06e460 2024-03-21T10:48:30.236 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:30.239+0000 7f266a000700 1 -- 172.21.15.153:0/174171159 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f266c199250 con 0x7f264c06e460 2024-03-21T10:48:30.236 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:48:30.236 INFO:teuthology.orchestra.run.smithi153.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232", 2024-03-21T10:48:30.236 INFO:teuthology.orchestra.run.smithi153.stdout: "in_progress": true, 2024-03-21T10:48:30.236 INFO:teuthology.orchestra.run.smithi153.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-21T10:48:30.236 INFO:teuthology.orchestra.run.smithi153.stdout: "services_complete": [ 2024-03-21T10:48:30.236 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr", 2024-03-21T10:48:30.236 INFO:teuthology.orchestra.run.smithi153.stdout: "mon", 2024-03-21T10:48:30.236 INFO:teuthology.orchestra.run.smithi153.stdout: "mds", 2024-03-21T10:48:30.237 INFO:teuthology.orchestra.run.smithi153.stdout: "crash", 2024-03-21T10:48:30.237 INFO:teuthology.orchestra.run.smithi153.stdout: "osd" 2024-03-21T10:48:30.237 INFO:teuthology.orchestra.run.smithi153.stdout: ], 2024-03-21T10:48:30.237 INFO:teuthology.orchestra.run.smithi153.stdout: "progress": "16/23 daemons upgraded", 2024-03-21T10:48:30.237 INFO:teuthology.orchestra.run.smithi153.stdout: "message": "Currently upgrading mds daemons", 2024-03-21T10:48:30.237 INFO:teuthology.orchestra.run.smithi153.stdout: "is_paused": false 2024-03-21T10:48:30.237 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:48:30.239 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:30.242+0000 7f2672600700 1 -- 172.21.15.153:0/174171159 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f264c06e460 msgr2=0x7f264c070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:48:30.239 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:30.242+0000 7f2672600700 1 --2- 172.21.15.153:0/174171159 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f264c06e460 0x7f264c070920 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f265c006010 tx=0x7f265c01a040 comp rx=0 tx=0).stop 2024-03-21T10:48:30.239 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:30.242+0000 7f2672600700 1 -- 172.21.15.153:0/174171159 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f266c106980 msgr2=0x7f266c19ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:48:30.239 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:30.242+0000 7f2672600700 1 --2- 172.21.15.153:0/174171159 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f266c106980 0x7f266c19ea60 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f265400c8a0 tx=0x7f265400cc60 comp rx=0 tx=0).stop 2024-03-21T10:48:30.239 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:30.242+0000 7f2672600700 1 -- 172.21.15.153:0/174171159 shutdown_connections 2024-03-21T10:48:30.240 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:30.242+0000 7f2672600700 1 --2- 172.21.15.153:0/174171159 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f266c100980 0x7f266c19e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:48:30.240 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:30.242+0000 7f2672600700 1 --2- 172.21.15.153:0/174171159 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f264c06e460 0x7f264c070920 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:48:30.240 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:30.242+0000 7f2672600700 1 --2- 172.21.15.153:0/174171159 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f266c106980 0x7f266c19ea60 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:48:30.240 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:30.242+0000 7f2672600700 1 -- 172.21.15.153:0/174171159 >> 172.21.15.153:0/174171159 conn(0x7f266c0fc470 msgr2=0x7f266c073020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:48:30.240 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:30.243+0000 7f2672600700 1 -- 172.21.15.153:0/174171159 shutdown_connections 2024-03-21T10:48:30.240 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:30.243+0000 7f2672600700 1 -- 172.21.15.153:0/174171159 wait complete. 2024-03-21T10:48:30.389 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:30.391+0000 7f0ed6c00700 1 -- 172.21.15.153:0/2685150731 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0ed0068670 msgr2=0x7f0ed0068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:48:30.389 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:30.391+0000 7f0ed6c00700 1 --2- 172.21.15.153:0/2685150731 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0ed0068670 0x7f0ed0068a50 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f0ec0009910 tx=0x7f0ec0009c20 comp rx=0 tx=0).stop 2024-03-21T10:48:30.389 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:30.391+0000 7f0ed6c00700 1 -- 172.21.15.153:0/2685150731 shutdown_connections 2024-03-21T10:48:30.389 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:30.391+0000 7f0ed6c00700 1 --2- 172.21.15.153:0/2685150731 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0ed0069020 0x7f0ed0105790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:48:30.389 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:30.391+0000 7f0ed6c00700 1 --2- 172.21.15.153:0/2685150731 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0ed0068670 0x7f0ed0068a50 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:48:30.389 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:30.391+0000 7f0ed6c00700 1 -- 172.21.15.153:0/2685150731 >> 172.21.15.153:0/2685150731 conn(0x7f0ed00755a0 msgr2=0x7f0ed00759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:48:30.389 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:30.391+0000 7f0ed6c00700 1 -- 172.21.15.153:0/2685150731 shutdown_connections 2024-03-21T10:48:30.389 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:30.392+0000 7f0ed6c00700 1 -- 172.21.15.153:0/2685150731 wait complete. 2024-03-21T10:48:30.391 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:30.392+0000 7f0ed6c00700 1 Processor -- start 2024-03-21T10:48:30.391 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:30.393+0000 7f0ed6c00700 1 -- start start 2024-03-21T10:48:30.391 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:30.393+0000 7f0ed6c00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0ed0068670 0x7f0ed019c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:48:30.392 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:30.393+0000 7f0ed6c00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0ed0069020 0x7f0ed019ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:48:30.392 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:30.393+0000 7f0ed6c00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed019d0e0 con 0x7f0ed0069020 2024-03-21T10:48:30.392 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:30.393+0000 7f0ed6c00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed0196590 con 0x7f0ed0068670 2024-03-21T10:48:30.392 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:30.393+0000 7f0ec7400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0ed0069020 0x7f0ed019ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:48:30.392 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:30.393+0000 7f0ec7400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0ed0069020 0x7f0ed019ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:54080/0 (socket says 172.21.15.153:54080) 2024-03-21T10:48:30.392 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:30.393+0000 7f0ec7400700 1 -- 172.21.15.153:0/2040639347 learned_addr learned my addr 172.21.15.153:0/2040639347 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:48:30.392 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:30.394+0000 7f0ecfe00700 1 --2- 172.21.15.153:0/2040639347 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0ed0068670 0x7f0ed019c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:48:30.392 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:30.394+0000 7f0ec7400700 1 -- 172.21.15.153:0/2040639347 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0ed0068670 msgr2=0x7f0ed019c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:48:30.392 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:30.394+0000 7f0ec7400700 1 --2- 172.21.15.153:0/2040639347 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0ed0068670 0x7f0ed019c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:48:30.392 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:30.394+0000 7f0ec7400700 1 -- 172.21.15.153:0/2040639347 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ec00095f0 con 0x7f0ed0069020 2024-03-21T10:48:30.392 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:30.394+0000 7f0ec7400700 1 --2- 172.21.15.153:0/2040639347 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0ed0069020 0x7f0ed019ca50 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f0eb800c8f0 tx=0x7f0eb800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:48:30.392 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:30.394+0000 7f0ecd600700 1 -- 172.21.15.153:0/2040639347 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0eb8007a50 con 0x7f0ed0069020 2024-03-21T10:48:30.392 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:30.394+0000 7f0ecd600700 1 -- 172.21.15.153:0/2040639347 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0eb8007bb0 con 0x7f0ed0069020 2024-03-21T10:48:30.393 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:30.394+0000 7f0ed6c00700 1 -- 172.21.15.153:0/2040639347 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ed0196870 con 0x7f0ed0069020 2024-03-21T10:48:30.393 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:30.394+0000 7f0ecd600700 1 -- 172.21.15.153:0/2040639347 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0eb8018630 con 0x7f0ed0069020 2024-03-21T10:48:30.393 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:30.395+0000 7f0ed6c00700 1 -- 172.21.15.153:0/2040639347 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ed0196dc0 con 0x7f0ed0069020 2024-03-21T10:48:30.393 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:30.396+0000 7f0ed6c00700 1 -- 172.21.15.153:0/2040639347 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ed0109130 con 0x7f0ed0069020 2024-03-21T10:48:30.393 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:30.397+0000 7f0ecd600700 1 -- 172.21.15.153:0/2040639347 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f0eb8007d20 con 0x7f0ed0069020 2024-03-21T10:48:30.395 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:30.398+0000 7f0ecd600700 1 --2- 172.21.15.153:0/2040639347 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f0eb406e460 0x7f0eb4070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:48:30.395 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:30.398+0000 7f0ecd600700 1 -- 172.21.15.153:0/2040639347 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f0eb808e3b0 con 0x7f0ed0069020 2024-03-21T10:48:30.395 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:30.398+0000 7f0ecfe00700 1 --2- 172.21.15.153:0/2040639347 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f0eb406e460 0x7f0eb4070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:48:30.396 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:30.399+0000 7f0ecfe00700 1 --2- 172.21.15.153:0/2040639347 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f0eb406e460 0x7f0eb4070920 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f0ec0006010 tx=0x7f0ec001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:48:30.400 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:30.403+0000 7f0ecd600700 1 -- 172.21.15.153:0/2040639347 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0eb805b020 con 0x7f0ed0069020 2024-03-21T10:48:30.694 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:30.697+0000 7f0ed6c00700 1 -- 172.21.15.153:0/2040639347 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f0ed01970c0 con 0x7f0ed0069020 2024-03-21T10:48:30.694 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:30.698+0000 7f0ecd600700 1 -- 172.21.15.153:0/2040639347 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4493 (secure 0 0 0) 0x7f0eb805abb0 con 0x7f0ed0069020 2024-03-21T10:48:30.695 INFO:teuthology.orchestra.run.smithi153.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-03-21T10:48:30.695 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-03-21T10:48:30.695 INFO:teuthology.orchestra.run.smithi153.stdout: host smithi174 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi174/config 2024-03-21T10:48:30.695 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/podman: timeout after 895 seconds 2024-03-21T10:48:30.695 INFO:teuthology.orchestra.run.smithi153.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 -e NODE_NAME=smithi174 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/run/ceph:z -v /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpd910r3_i:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 inventory --format=json-pretty --filter-for-batch 2024-03-21T10:48:30.695 INFO:teuthology.orchestra.run.smithi153.stdout:Traceback (most recent call last): 2024-03-21T10:48:30.695 INFO:teuthology.orchestra.run.smithi153.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-03-21T10:48:30.695 INFO:teuthology.orchestra.run.smithi153.stdout: return _run_code(code, main_globals, None, 2024-03-21T10:48:30.695 INFO:teuthology.orchestra.run.smithi153.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-03-21T10:48:30.695 INFO:teuthology.orchestra.run.smithi153.stdout: exec(code, run_globals) 2024-03-21T10:48:30.695 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 5606, in 2024-03-21T10:48:30.696 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 5594, in main 2024-03-21T10:48:30.696 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 406, in _infer_config 2024-03-21T10:48:30.696 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 321, in _infer_fsid 2024-03-21T10:48:30.696 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 434, in _infer_image 2024-03-21T10:48:30.696 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 308, in _validate_fsid 2024-03-21T10:48:30.696 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 3357, in command_ceph_volume 2024-03-21T10:48:30.696 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-03-21T10:48:30.696 INFO:teuthology.orchestra.run.smithi153.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 -e NODE_NAME=smithi174 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/run/ceph:z -v /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpd910r3_i:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 inventory --format=json-pretty --filter-for-batch 2024-03-21T10:48:30.696 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-21T10:48:30.696 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is degraded 2024-03-21T10:48:30.696 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-21T10:48:30.696 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-21T10:48:30.696 INFO:teuthology.orchestra.run.smithi153.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-21T10:48:30.696 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-21T10:48:30.697 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:30.701+0000 7f0ed6c00700 1 -- 172.21.15.153:0/2040639347 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f0eb406e460 msgr2=0x7f0eb4070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:48:30.697 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:30.701+0000 7f0ed6c00700 1 --2- 172.21.15.153:0/2040639347 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f0eb406e460 0x7f0eb4070920 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f0ec0006010 tx=0x7f0ec001a040 comp rx=0 tx=0).stop 2024-03-21T10:48:30.698 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:30.701+0000 7f0ed6c00700 1 -- 172.21.15.153:0/2040639347 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0ed0069020 msgr2=0x7f0ed019ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:48:30.698 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:30.701+0000 7f0ed6c00700 1 --2- 172.21.15.153:0/2040639347 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0ed0069020 0x7f0ed019ca50 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f0eb800c8f0 tx=0x7f0eb800ccb0 comp rx=0 tx=0).stop 2024-03-21T10:48:30.698 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:30.701+0000 7f0ed6c00700 1 -- 172.21.15.153:0/2040639347 shutdown_connections 2024-03-21T10:48:30.698 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:30.701+0000 7f0ed6c00700 1 --2- 172.21.15.153:0/2040639347 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0ed0068670 0x7f0ed019c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:48:30.698 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:30.701+0000 7f0ed6c00700 1 --2- 172.21.15.153:0/2040639347 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f0eb406e460 0x7f0eb4070920 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:48:30.698 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:30.702+0000 7f0ed6c00700 1 --2- 172.21.15.153:0/2040639347 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0ed0069020 0x7f0ed019ca50 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:48:30.698 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:30.702+0000 7f0ed6c00700 1 -- 172.21.15.153:0/2040639347 >> 172.21.15.153:0/2040639347 conn(0x7f0ed00755a0 msgr2=0x7f0ed00fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:48:30.698 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:30.702+0000 7f0ed6c00700 1 -- 172.21.15.153:0/2040639347 shutdown_connections 2024-03-21T10:48:30.698 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:48:30.702+0000 7f0ed6c00700 1 -- 172.21.15.153:0/2040639347 wait complete. 2024-03-21T10:48:30.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:48:30 smithi153 ceph-mon[113733]: from='client.35946 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:48:30.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:48:30 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/2427138192' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:48:30.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:48:30 smithi174 ceph-mon[88004]: from='client.35946 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:48:30.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:48:30 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/2427138192' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:48:31.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:48:31 smithi153 ceph-mon[113733]: pgmap v1101: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:48:31.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:48:31 smithi153 ceph-mon[113733]: from='client.35958 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:48:31.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:48:31 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/2040639347' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:48:31.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:48:31 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:48:31.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:48:31 smithi174 ceph-mon[88004]: pgmap v1101: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:48:31.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:48:31 smithi174 ceph-mon[88004]: from='client.35958 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:48:31.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:48:31 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/2040639347' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:48:31.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:48:31 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:48:33.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:48:33 smithi153 ceph-mon[113733]: pgmap v1102: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:48:33.946 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:48:33 smithi174 ceph-mon[88004]: pgmap v1102: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:48:35.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:48:35 smithi153 ceph-mon[113733]: pgmap v1103: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:48:35.946 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:48:35 smithi174 ceph-mon[88004]: pgmap v1103: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:48:37.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:48:37 smithi153 ceph-mon[113733]: pgmap v1104: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:48:37.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:48:37 smithi174 ceph-mon[88004]: pgmap v1104: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:48:39.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:48:39 smithi153 ceph-mon[113733]: pgmap v1105: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:48:39.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:48:39 smithi174 ceph-mon[88004]: pgmap v1105: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:48:41.801 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:48:41 smithi153 ceph-mon[113733]: pgmap v1106: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:48:41.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:48:41 smithi174 ceph-mon[88004]: pgmap v1106: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:48:43.801 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:48:43 smithi153 ceph-mon[113733]: pgmap v1107: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:48:43.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:48:43 smithi174 ceph-mon[88004]: pgmap v1107: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:48:45.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:48:45 smithi174 ceph-mon[88004]: pgmap v1108: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:48:46.052 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:48:45 smithi153 ceph-mon[113733]: pgmap v1108: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:48:46.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:48:46 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:48:47.052 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:48:46 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:48:47.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:48:47 smithi174 ceph-mon[88004]: pgmap v1109: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:48:48.051 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:48:47 smithi153 ceph-mon[113733]: pgmap v1109: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:48:49.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:48:49 smithi174 ceph-mon[88004]: pgmap v1110: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:48:50.051 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:48:49 smithi153 ceph-mon[113733]: pgmap v1110: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:48:51.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:48:51 smithi174 ceph-mon[88004]: pgmap v1111: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:48:52.052 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:48:51 smithi153 ceph-mon[113733]: pgmap v1111: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:48:53.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:48:53 smithi174 ceph-mon[88004]: pgmap v1112: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:48:54.052 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:48:53 smithi153 ceph-mon[113733]: pgmap v1112: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:48:55.946 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:48:55 smithi174 ceph-mon[88004]: pgmap v1113: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:48:56.052 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:48:55 smithi153 ceph-mon[113733]: pgmap v1113: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:48:57.946 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:48:57 smithi174 ceph-mon[88004]: pgmap v1114: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:48:58.051 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:48:57 smithi153 ceph-mon[113733]: pgmap v1114: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:48:59.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:48:59 smithi153 ceph-mon[113733]: pgmap v1115: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:48:59.946 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:48:59 smithi174 ceph-mon[88004]: pgmap v1115: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:49:00.868 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:00.871+0000 7f1543800700 1 -- 172.21.15.153:0/152592612 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f153c1039b0 msgr2=0x7f153c107a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:49:00.868 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:00.871+0000 7f1543800700 1 --2- 172.21.15.153:0/152592612 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f153c1039b0 0x7f153c107a00 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f1524009910 tx=0x7f1524009c20 comp rx=0 tx=0).stop 2024-03-21T10:49:00.868 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:00.871+0000 7f1543800700 1 -- 172.21.15.153:0/152592612 shutdown_connections 2024-03-21T10:49:00.868 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:00.871+0000 7f1543800700 1 --2- 172.21.15.153:0/152592612 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f153c1039b0 0x7f153c107a00 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:49:00.868 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:00.871+0000 7f1543800700 1 --2- 172.21.15.153:0/152592612 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f153c103000 0x7f153c1033e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:49:00.868 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:00.871+0000 7f1543800700 1 -- 172.21.15.153:0/152592612 >> 172.21.15.153:0/152592612 conn(0x7f153c0fe8d0 msgr2=0x7f153c100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:49:00.868 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:00.871+0000 7f1543800700 1 -- 172.21.15.153:0/152592612 shutdown_connections 2024-03-21T10:49:00.868 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:00.871+0000 7f1543800700 1 -- 172.21.15.153:0/152592612 wait complete. 2024-03-21T10:49:00.869 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:00.872+0000 7f1543800700 1 Processor -- start 2024-03-21T10:49:00.869 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:00.872+0000 7f1543800700 1 -- start start 2024-03-21T10:49:00.870 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:00.873+0000 7f1543800700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f153c103000 0x7f153c19e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:49:00.870 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:00.873+0000 7f1543800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f153c1039b0 0x7f153c19ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:49:00.870 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:00.873+0000 7f1543800700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f153c19f390 con 0x7f153c1039b0 2024-03-21T10:49:00.870 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:00.873+0000 7f1543800700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f153c198840 con 0x7f153c103000 2024-03-21T10:49:00.870 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:00.873+0000 7f1533e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f153c1039b0 0x7f153c19ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:49:00.870 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:00.873+0000 7f1533e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f153c1039b0 0x7f153c19ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:35762/0 (socket says 172.21.15.153:35762) 2024-03-21T10:49:00.870 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:00.873+0000 7f1533e00700 1 -- 172.21.15.153:0/2097694427 learned_addr learned my addr 172.21.15.153:0/2097694427 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:49:00.870 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:00.873+0000 7f1540e00700 1 --2- 172.21.15.153:0/2097694427 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f153c103000 0x7f153c19e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:49:00.871 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:00.873+0000 7f1533e00700 1 -- 172.21.15.153:0/2097694427 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f153c103000 msgr2=0x7f153c19e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:49:00.871 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:00.873+0000 7f1533e00700 1 --2- 172.21.15.153:0/2097694427 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f153c103000 0x7f153c19e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:49:00.871 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:00.874+0000 7f1533e00700 1 -- 172.21.15.153:0/2097694427 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15240095f0 con 0x7f153c1039b0 2024-03-21T10:49:00.871 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:00.874+0000 7f1533e00700 1 --2- 172.21.15.153:0/2097694427 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f153c1039b0 0x7f153c19ed00 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f152400bd70 tx=0x7f152400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:49:00.871 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:00.874+0000 7f153a000700 1 -- 172.21.15.153:0/2097694427 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f152401d070 con 0x7f153c1039b0 2024-03-21T10:49:00.871 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:00.874+0000 7f153a000700 1 -- 172.21.15.153:0/2097694427 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1524005dc0 con 0x7f153c1039b0 2024-03-21T10:49:00.871 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:00.874+0000 7f1543800700 1 -- 172.21.15.153:0/2097694427 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f153c198ac0 con 0x7f153c1039b0 2024-03-21T10:49:00.872 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:00.874+0000 7f153a000700 1 -- 172.21.15.153:0/2097694427 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1524017400 con 0x7f153c1039b0 2024-03-21T10:49:00.872 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:00.874+0000 7f1543800700 1 -- 172.21.15.153:0/2097694427 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f153c198fb0 con 0x7f153c1039b0 2024-03-21T10:49:00.872 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:00.876+0000 7f1543800700 1 -- 172.21.15.153:0/2097694427 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f153c10b400 con 0x7f153c1039b0 2024-03-21T10:49:00.873 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:00.876+0000 7f153a000700 1 -- 172.21.15.153:0/2097694427 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f152400f7f0 con 0x7f153c1039b0 2024-03-21T10:49:00.874 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:00.877+0000 7f153a000700 1 --2- 172.21.15.153:0/2097694427 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f152006e460 0x7f1520070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:49:00.874 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:00.877+0000 7f153a000700 1 -- 172.21.15.153:0/2097694427 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f152408fa60 con 0x7f153c1039b0 2024-03-21T10:49:00.874 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:00.878+0000 7f1540e00700 1 --2- 172.21.15.153:0/2097694427 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f152006e460 0x7f1520070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:49:00.875 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:00.878+0000 7f1540e00700 1 --2- 172.21.15.153:0/2097694427 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f152006e460 0x7f1520070920 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f152c005950 tx=0x7f152c0058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:49:00.879 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:00.883+0000 7f153a000700 1 -- 172.21.15.153:0/2097694427 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f152405c5b0 con 0x7f153c1039b0 2024-03-21T10:49:01.117 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:01.120+0000 7f1543800700 1 -- 172.21.15.153:0/2097694427 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f153c02cd30 con 0x7f152006e460 2024-03-21T10:49:01.119 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:01.122+0000 7f153a000700 1 -- 172.21.15.153:0/2097694427 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f153c02cd30 con 0x7f152006e460 2024-03-21T10:49:01.121 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:01.125+0000 7f1543800700 1 -- 172.21.15.153:0/2097694427 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f152006e460 msgr2=0x7f1520070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:49:01.122 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:01.125+0000 7f1543800700 1 --2- 172.21.15.153:0/2097694427 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f152006e460 0x7f1520070920 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f152c005950 tx=0x7f152c0058e0 comp rx=0 tx=0).stop 2024-03-21T10:49:01.122 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:01.125+0000 7f1543800700 1 -- 172.21.15.153:0/2097694427 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f153c1039b0 msgr2=0x7f153c19ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:49:01.122 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:01.125+0000 7f1543800700 1 --2- 172.21.15.153:0/2097694427 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f153c1039b0 0x7f153c19ed00 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f152400bd70 tx=0x7f152400be50 comp rx=0 tx=0).stop 2024-03-21T10:49:01.122 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:01.125+0000 7f1543800700 1 -- 172.21.15.153:0/2097694427 shutdown_connections 2024-03-21T10:49:01.123 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:01.125+0000 7f1543800700 1 --2- 172.21.15.153:0/2097694427 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f153c103000 0x7f153c19e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:49:01.123 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:01.125+0000 7f1543800700 1 --2- 172.21.15.153:0/2097694427 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f152006e460 0x7f1520070920 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:49:01.123 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:01.125+0000 7f1543800700 1 --2- 172.21.15.153:0/2097694427 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f153c1039b0 0x7f153c19ed00 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:49:01.123 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:01.125+0000 7f1543800700 1 -- 172.21.15.153:0/2097694427 >> 172.21.15.153:0/2097694427 conn(0x7f153c0fe8d0 msgr2=0x7f153c100cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:49:01.123 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:01.125+0000 7f1543800700 1 -- 172.21.15.153:0/2097694427 shutdown_connections 2024-03-21T10:49:01.123 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:01.125+0000 7f1543800700 1 -- 172.21.15.153:0/2097694427 wait complete. 2024-03-21T10:49:01.138 INFO:teuthology.orchestra.run.smithi153.stdout:true 2024-03-21T10:49:01.275 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:01.278+0000 7fcbe4c00700 1 -- 172.21.15.153:0/3622320252 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcbe0068410 msgr2=0x7fcbe00687f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:49:01.275 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:01.278+0000 7fcbe4c00700 1 --2- 172.21.15.153:0/3622320252 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcbe0068410 0x7fcbe00687f0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fcbd0009910 tx=0x7fcbd0009c20 comp rx=0 tx=0).stop 2024-03-21T10:49:01.275 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:01.278+0000 7fcbe4c00700 1 -- 172.21.15.153:0/3622320252 shutdown_connections 2024-03-21T10:49:01.275 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:01.278+0000 7fcbe4c00700 1 --2- 172.21.15.153:0/3622320252 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcbe0068d30 0x7fcbe010d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:49:01.275 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:01.278+0000 7fcbe4c00700 1 --2- 172.21.15.153:0/3622320252 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcbe0068410 0x7fcbe00687f0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:49:01.275 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:01.278+0000 7fcbe4c00700 1 -- 172.21.15.153:0/3622320252 >> 172.21.15.153:0/3622320252 conn(0x7fcbe0075330 msgr2=0x7fcbe0075740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:49:01.276 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:01.278+0000 7fcbe4c00700 1 -- 172.21.15.153:0/3622320252 shutdown_connections 2024-03-21T10:49:01.276 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:01.278+0000 7fcbe4c00700 1 -- 172.21.15.153:0/3622320252 wait complete. 2024-03-21T10:49:01.276 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:01.279+0000 7fcbe4c00700 1 Processor -- start 2024-03-21T10:49:01.276 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:01.279+0000 7fcbe4c00700 1 -- start start 2024-03-21T10:49:01.277 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:01.280+0000 7fcbe4c00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcbe0068410 0x7fcbe019e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:49:01.277 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:01.280+0000 7fcbe4c00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcbe0068d30 0x7fcbe019eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:49:01.277 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:01.280+0000 7fcbe4c00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbe019f1f0 con 0x7fcbe0068d30 2024-03-21T10:49:01.277 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:01.280+0000 7fcbe4c00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbe01986a0 con 0x7fcbe0068410 2024-03-21T10:49:01.278 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:01.280+0000 7fcbdea00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcbe0068d30 0x7fcbe019eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:49:01.278 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:01.280+0000 7fcbdea00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcbe0068d30 0x7fcbe019eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:35776/0 (socket says 172.21.15.153:35776) 2024-03-21T10:49:01.278 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:01.280+0000 7fcbdea00700 1 -- 172.21.15.153:0/1705951159 learned_addr learned my addr 172.21.15.153:0/1705951159 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:49:01.278 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:01.280+0000 7fcbdf400700 1 --2- 172.21.15.153:0/1705951159 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcbe0068410 0x7fcbe019e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:49:01.278 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:01.281+0000 7fcbdea00700 1 -- 172.21.15.153:0/1705951159 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcbe0068410 msgr2=0x7fcbe019e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:49:01.278 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:01.281+0000 7fcbdea00700 1 --2- 172.21.15.153:0/1705951159 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcbe0068410 0x7fcbe019e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:49:01.278 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:01.281+0000 7fcbdea00700 1 -- 172.21.15.153:0/1705951159 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbd00095f0 con 0x7fcbe0068d30 2024-03-21T10:49:01.279 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:01.281+0000 7fcbdea00700 1 --2- 172.21.15.153:0/1705951159 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcbe0068d30 0x7fcbe019eb60 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc00b4b0 tx=0x7fcbcc00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:49:01.279 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:01.281+0000 7fcbd7e00700 1 -- 172.21.15.153:0/1705951159 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcbcc011800 con 0x7fcbe0068d30 2024-03-21T10:49:01.279 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:01.281+0000 7fcbd7e00700 1 -- 172.21.15.153:0/1705951159 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcbcc011e20 con 0x7fcbe0068d30 2024-03-21T10:49:01.279 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:01.281+0000 7fcbe4c00700 1 -- 172.21.15.153:0/1705951159 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbe0198980 con 0x7fcbe0068d30 2024-03-21T10:49:01.279 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:01.281+0000 7fcbd7e00700 1 -- 172.21.15.153:0/1705951159 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcbcc00f2e0 con 0x7fcbe0068d30 2024-03-21T10:49:01.279 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:01.281+0000 7fcbe4c00700 1 -- 172.21.15.153:0/1705951159 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbe0198ed0 con 0x7fcbe0068d30 2024-03-21T10:49:01.279 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:01.283+0000 7fcbe4c00700 1 -- 172.21.15.153:0/1705951159 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbe010a7f0 con 0x7fcbe0068d30 2024-03-21T10:49:01.280 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:01.284+0000 7fcbd7e00700 1 -- 172.21.15.153:0/1705951159 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fcbcc011960 con 0x7fcbe0068d30 2024-03-21T10:49:01.282 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:01.285+0000 7fcbd7e00700 1 --2- 172.21.15.153:0/1705951159 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fcbc406e460 0x7fcbc4070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:49:01.282 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:01.285+0000 7fcbd7e00700 1 -- 172.21.15.153:0/1705951159 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fcbcc08e060 con 0x7fcbe0068d30 2024-03-21T10:49:01.282 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:01.285+0000 7fcbdf400700 1 --2- 172.21.15.153:0/1705951159 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fcbc406e460 0x7fcbc4070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:49:01.283 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:01.286+0000 7fcbdf400700 1 --2- 172.21.15.153:0/1705951159 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fcbc406e460 0x7fcbc4070920 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fcbd0006010 tx=0x7fcbd001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:49:01.287 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:01.291+0000 7fcbd7e00700 1 -- 172.21.15.153:0/1705951159 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fcbcc05ac30 con 0x7fcbe0068d30 2024-03-21T10:49:01.522 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:01.525+0000 7fcbe4c00700 1 -- 172.21.15.153:0/1705951159 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcbe0199210 con 0x7fcbc406e460 2024-03-21T10:49:01.525 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:01.528+0000 7fcbd7e00700 1 -- 172.21.15.153:0/1705951159 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fcbe0199210 con 0x7fcbc406e460 2024-03-21T10:49:01.528 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:01.531+0000 7fcbe4c00700 1 -- 172.21.15.153:0/1705951159 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fcbc406e460 msgr2=0x7fcbc4070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:49:01.528 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:01.531+0000 7fcbe4c00700 1 --2- 172.21.15.153:0/1705951159 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fcbc406e460 0x7fcbc4070920 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fcbd0006010 tx=0x7fcbd001a040 comp rx=0 tx=0).stop 2024-03-21T10:49:01.528 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:01.531+0000 7fcbe4c00700 1 -- 172.21.15.153:0/1705951159 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcbe0068d30 msgr2=0x7fcbe019eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:49:01.528 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:01.531+0000 7fcbe4c00700 1 --2- 172.21.15.153:0/1705951159 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcbe0068d30 0x7fcbe019eb60 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc00b4b0 tx=0x7fcbcc00b870 comp rx=0 tx=0).stop 2024-03-21T10:49:01.528 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:01.531+0000 7fcbe4c00700 1 -- 172.21.15.153:0/1705951159 shutdown_connections 2024-03-21T10:49:01.528 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:01.531+0000 7fcbe4c00700 1 --2- 172.21.15.153:0/1705951159 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcbe0068410 0x7fcbe019e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:49:01.528 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:01.531+0000 7fcbe4c00700 1 --2- 172.21.15.153:0/1705951159 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fcbc406e460 0x7fcbc4070920 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:49:01.529 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:01.531+0000 7fcbe4c00700 1 --2- 172.21.15.153:0/1705951159 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fcbe0068d30 0x7fcbe019eb60 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:49:01.529 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:01.531+0000 7fcbe4c00700 1 -- 172.21.15.153:0/1705951159 >> 172.21.15.153:0/1705951159 conn(0x7fcbe0075330 msgr2=0x7fcbe00fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:49:01.529 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:01.531+0000 7fcbe4c00700 1 -- 172.21.15.153:0/1705951159 shutdown_connections 2024-03-21T10:49:01.529 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:01.531+0000 7fcbe4c00700 1 -- 172.21.15.153:0/1705951159 wait complete. 2024-03-21T10:49:01.677 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:01.680+0000 7f283c600700 1 -- 172.21.15.153:0/2241916955 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f283410d280 msgr2=0x7f283410d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:49:01.677 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:01.680+0000 7f283c600700 1 --2- 172.21.15.153:0/2241916955 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f283410d280 0x7f283410d660 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f2828009910 tx=0x7f2828009c20 comp rx=0 tx=0).stop 2024-03-21T10:49:01.677 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:01.680+0000 7f283c600700 1 -- 172.21.15.153:0/2241916955 shutdown_connections 2024-03-21T10:49:01.678 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:01.680+0000 7f283c600700 1 --2- 172.21.15.153:0/2241916955 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f28341071d0 0x7f2834107650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:49:01.678 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:01.680+0000 7f283c600700 1 --2- 172.21.15.153:0/2241916955 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f283410d280 0x7f283410d660 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:49:01.678 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:01.680+0000 7f283c600700 1 -- 172.21.15.153:0/2241916955 >> 172.21.15.153:0/2241916955 conn(0x7f28340769e0 msgr2=0x7f2834078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:49:01.678 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:01.680+0000 7f283c600700 1 -- 172.21.15.153:0/2241916955 shutdown_connections 2024-03-21T10:49:01.678 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:01.680+0000 7f283c600700 1 -- 172.21.15.153:0/2241916955 wait complete. 2024-03-21T10:49:01.680 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:01.681+0000 7f283c600700 1 Processor -- start 2024-03-21T10:49:01.680 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:01.681+0000 7f283c600700 1 -- start start 2024-03-21T10:49:01.680 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:01.682+0000 7f283c600700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f28341071d0 0x7f28341a2a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:49:01.680 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:01.682+0000 7f283c600700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f283410d280 0x7f28341a2f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:49:01.680 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:01.682+0000 7f283c600700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28341a3640 con 0x7f283410d280 2024-03-21T10:49:01.680 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:01.682+0000 7f283c600700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f283419cb00 con 0x7f28341071d0 2024-03-21T10:49:01.680 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:01.682+0000 7f2839200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f283410d280 0x7f28341a2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:49:01.680 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:01.682+0000 7f2839200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f283410d280 0x7f28341a2f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:35792/0 (socket says 172.21.15.153:35792) 2024-03-21T10:49:01.680 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:01.682+0000 7f2839200700 1 -- 172.21.15.153:0/58100427 learned_addr learned my addr 172.21.15.153:0/58100427 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:49:01.680 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:01.682+0000 7f2839c00700 1 --2- 172.21.15.153:0/58100427 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f28341071d0 0x7f28341a2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:49:01.680 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:01.682+0000 7f2839200700 1 -- 172.21.15.153:0/58100427 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f28341071d0 msgr2=0x7f28341a2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:49:01.681 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:01.682+0000 7f2839200700 1 --2- 172.21.15.153:0/58100427 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f28341071d0 0x7f28341a2a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:49:01.681 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:01.682+0000 7f2839200700 1 -- 172.21.15.153:0/58100427 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28280095f0 con 0x7f283410d280 2024-03-21T10:49:01.681 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:01.683+0000 7f2839c00700 1 --2- 172.21.15.153:0/58100427 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f28341071d0 0x7f28341a2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:49:01.681 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:01.683+0000 7f2839200700 1 --2- 172.21.15.153:0/58100427 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f283410d280 0x7f28341a2f60 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f282000c8a0 tx=0x7f282000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:49:01.681 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:01.683+0000 7f2832000700 1 -- 172.21.15.153:0/58100427 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f282000cea0 con 0x7f283410d280 2024-03-21T10:49:01.681 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:01.683+0000 7f2832000700 1 -- 172.21.15.153:0/58100427 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2820004830 con 0x7f283410d280 2024-03-21T10:49:01.681 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:01.683+0000 7f283c600700 1 -- 172.21.15.153:0/58100427 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f283419cde0 con 0x7f283410d280 2024-03-21T10:49:01.681 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:01.683+0000 7f2832000700 1 -- 172.21.15.153:0/58100427 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f28200055b0 con 0x7f283410d280 2024-03-21T10:49:01.682 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:01.683+0000 7f283c600700 1 -- 172.21.15.153:0/58100427 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f283419d3b0 con 0x7f283410d280 2024-03-21T10:49:01.682 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:01.684+0000 7f283c600700 1 -- 172.21.15.153:0/58100427 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f283404ea90 con 0x7f283410d280 2024-03-21T10:49:01.682 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:01.685+0000 7f2832000700 1 -- 172.21.15.153:0/58100427 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f28200049a0 con 0x7f283410d280 2024-03-21T10:49:01.683 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:01.686+0000 7f2832000700 1 --2- 172.21.15.153:0/58100427 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f281c06e460 0x7f281c070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:49:01.683 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:01.686+0000 7f2832000700 1 -- 172.21.15.153:0/58100427 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f282008e270 con 0x7f283410d280 2024-03-21T10:49:01.683 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:01.686+0000 7f2839c00700 1 --2- 172.21.15.153:0/58100427 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f281c06e460 0x7f281c070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:49:01.684 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:01.687+0000 7f2839c00700 1 --2- 172.21.15.153:0/58100427 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f281c06e460 0x7f281c070920 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f2828006010 tx=0x7f282800b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:49:01.687 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:01.691+0000 7f2832000700 1 -- 172.21.15.153:0/58100427 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f282005adc0 con 0x7f283410d280 2024-03-21T10:49:01.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:49:01 smithi153 ceph-mon[113733]: pgmap v1116: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:49:01.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:49:01 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:49:01.904 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:01.907+0000 7f283c600700 1 -- 172.21.15.153:0/58100427 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f283419d910 con 0x7f281c06e460 2024-03-21T10:49:01.920 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:01.923+0000 7f2832000700 1 -- 172.21.15.153:0/58100427 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f283419d910 con 0x7f281c06e460 2024-03-21T10:49:01.921 INFO:teuthology.orchestra.run.smithi153.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-21T10:49:01.921 INFO:teuthology.orchestra.run.smithi153.stdout:alertmanager.smithi153 smithi153 *:9093,9094 running (37m) 13m ago 51m 15.6M - 0.25.0 c8568f914cd2 9ed3f2200b5f 2024-03-21T10:49:01.921 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi153 smithi153 running (51m) 13m ago 51m 10.4M - 18.2.1 5be31c24972a 5b61994396a6 2024-03-21T10:49:01.921 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi174 smithi174 running (50m) 13m ago 50m 10.1M - 18.2.1 5be31c24972a 4f7a5fc3ac28 2024-03-21T10:49:01.921 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi153 smithi153 running (34m) 13m ago 51m 6656k - 19.0.0-2231-g2edfee68 237e43c7a65f 32c21104d5b0 2024-03-21T10:49:01.921 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi174 smithi174 running (33m) 13m ago 50m 6660k - 19.0.0-2231-g2edfee68 237e43c7a65f a04e48967ab6 2024-03-21T10:49:01.921 INFO:teuthology.orchestra.run.smithi153.stdout:grafana.smithi153 smithi153 *:3000 running (37m) 13m ago 51m 75.2M - 9.4.12 2bacad6d85d8 ee24026ab786 2024-03-21T10:49:01.921 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.iuyluj smithi153 running (29m) 13m ago 44m 18.7M - 19.0.0-2231-g2edfee68 237e43c7a65f 87aff8a5de70 2024-03-21T10:49:01.921 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.zipckr smithi153 running (29m) 13m ago 44m 18.9M - 19.0.0-2231-g2edfee68 237e43c7a65f ccc262bd5dd3 2024-03-21T10:49:01.921 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.nhozag smithi174 running (13m) 13m ago 44m 10.9M - 19.0.0-2231-g2edfee68 237e43c7a65f 9e02aea72c96 2024-03-21T10:49:01.921 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.yyimst smithi174 running (28m) 13m ago 44m 18.5M - 19.0.0-2231-g2edfee68 237e43c7a65f fc1962f9351e 2024-03-21T10:49:01.922 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi153.tluxni smithi153 *:8443,9283,8765 running (40m) 13m ago 53m 512M - 19.0.0-2231-g2edfee68 237e43c7a65f 7ffee0e55382 2024-03-21T10:49:01.922 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi174.qpcwat smithi174 *:8443,9283,8765 running (39m) 13m ago 50m 459M - 19.0.0-2231-g2edfee68 237e43c7a65f 748968f216ab 2024-03-21T10:49:01.922 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi153 smithi153 running (35m) 13m ago 53m 70.8M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f dd8038303dc3 2024-03-21T10:49:01.922 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi174 smithi174 running (35m) 13m ago 49m 51.9M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f 29d78a788f2c 2024-03-21T10:49:01.922 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi153 smithi153 *:9100 running (39m) 13m ago 51m 8140k - 1.5.0 0da6a335fe13 bb835edc0e0b 2024-03-21T10:49:01.922 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi174 smithi174 *:9100 running (38m) 13m ago 50m 8610k - 1.5.0 0da6a335fe13 7c4ecad790fa 2024-03-21T10:49:01.922 INFO:teuthology.orchestra.run.smithi153.stdout:osd.0 smithi153 running (33m) 13m ago 49m 118M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 507900fd4791 2024-03-21T10:49:01.922 INFO:teuthology.orchestra.run.smithi153.stdout:osd.1 smithi153 running (32m) 13m ago 48m 93.0M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 0d646452e9c1 2024-03-21T10:49:01.922 INFO:teuthology.orchestra.run.smithi153.stdout:osd.2 smithi153 running (32m) 13m ago 47m 91.2M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 1326f2afdfcd 2024-03-21T10:49:01.922 INFO:teuthology.orchestra.run.smithi153.stdout:osd.3 smithi174 running (31m) 13m ago 47m 87.5M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 70aafac71ad5 2024-03-21T10:49:01.922 INFO:teuthology.orchestra.run.smithi153.stdout:osd.4 smithi174 running (30m) 13m ago 46m 95.8M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 0d746ce6e79d 2024-03-21T10:49:01.922 INFO:teuthology.orchestra.run.smithi153.stdout:osd.5 smithi174 running (30m) 13m ago 46m 90.3M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 53e5f35b8cac 2024-03-21T10:49:01.922 INFO:teuthology.orchestra.run.smithi153.stdout:prometheus.smithi153 smithi153 *:9095 running (38m) 13m ago 50m 88.1M - 2.43.0 a07b618ecd1d 7d95561e50c2 2024-03-21T10:49:01.924 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:01.926+0000 7f283c600700 1 -- 172.21.15.153:0/58100427 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f281c06e460 msgr2=0x7f281c070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:49:01.924 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:01.926+0000 7f283c600700 1 --2- 172.21.15.153:0/58100427 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f281c06e460 0x7f281c070920 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f2828006010 tx=0x7f282800b540 comp rx=0 tx=0).stop 2024-03-21T10:49:01.924 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:01.926+0000 7f283c600700 1 -- 172.21.15.153:0/58100427 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f283410d280 msgr2=0x7f28341a2f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:49:01.924 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:01.926+0000 7f283c600700 1 --2- 172.21.15.153:0/58100427 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f283410d280 0x7f28341a2f60 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f282000c8a0 tx=0x7f282000cc60 comp rx=0 tx=0).stop 2024-03-21T10:49:01.924 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:01.926+0000 7f283c600700 1 -- 172.21.15.153:0/58100427 shutdown_connections 2024-03-21T10:49:01.924 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:01.926+0000 7f283c600700 1 --2- 172.21.15.153:0/58100427 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f28341071d0 0x7f28341a2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:49:01.924 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:01.926+0000 7f283c600700 1 --2- 172.21.15.153:0/58100427 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f281c06e460 0x7f281c070920 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:49:01.924 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:01.926+0000 7f283c600700 1 --2- 172.21.15.153:0/58100427 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f283410d280 0x7f28341a2f60 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:49:01.924 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:01.926+0000 7f283c600700 1 -- 172.21.15.153:0/58100427 >> 172.21.15.153:0/58100427 conn(0x7f28340769e0 msgr2=0x7f2834078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:49:01.924 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:01.927+0000 7f283c600700 1 -- 172.21.15.153:0/58100427 shutdown_connections 2024-03-21T10:49:01.924 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:01.927+0000 7f283c600700 1 -- 172.21.15.153:0/58100427 wait complete. 2024-03-21T10:49:01.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:49:01 smithi174 ceph-mon[88004]: pgmap v1116: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:49:01.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:49:01 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:49:02.079 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.082+0000 7f9cf8200700 1 -- 172.21.15.153:0/5787445 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9cf0100da0 msgr2=0x7f9cf0101180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:49:02.079 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.082+0000 7f9cf8200700 1 --2- 172.21.15.153:0/5787445 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9cf0100da0 0x7f9cf0101180 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f9ce400b0a0 tx=0x7f9ce400b3b0 comp rx=0 tx=0).stop 2024-03-21T10:49:02.079 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.082+0000 7f9cf8200700 1 -- 172.21.15.153:0/5787445 shutdown_connections 2024-03-21T10:49:02.079 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.082+0000 7f9cf8200700 1 --2- 172.21.15.153:0/5787445 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9cf0101750 0x7f9cf01057a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:49:02.079 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.082+0000 7f9cf8200700 1 --2- 172.21.15.153:0/5787445 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9cf0100da0 0x7f9cf0101180 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:49:02.079 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.082+0000 7f9cf8200700 1 -- 172.21.15.153:0/5787445 >> 172.21.15.153:0/5787445 conn(0x7f9cf00fc630 msgr2=0x7f9cf00fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:49:02.080 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.082+0000 7f9cf8200700 1 -- 172.21.15.153:0/5787445 shutdown_connections 2024-03-21T10:49:02.080 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.082+0000 7f9cf8200700 1 -- 172.21.15.153:0/5787445 wait complete. 2024-03-21T10:49:02.081 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.083+0000 7f9cf8200700 1 Processor -- start 2024-03-21T10:49:02.081 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.083+0000 7f9cf8200700 1 -- start start 2024-03-21T10:49:02.081 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.084+0000 7f9cf8200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9cf0100da0 0x7f9cf0198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:49:02.081 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.084+0000 7f9cf8200700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9cf0101750 0x7f9cf0198ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:49:02.081 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.084+0000 7f9cf8200700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf0199560 con 0x7f9cf0100da0 2024-03-21T10:49:02.082 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.084+0000 7f9cf8200700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf019d370 con 0x7f9cf0101750 2024-03-21T10:49:02.082 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.084+0000 7f9cf5800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9cf0100da0 0x7f9cf0198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:49:02.082 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.084+0000 7f9cf5800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9cf0100da0 0x7f9cf0198990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:35802/0 (socket says 172.21.15.153:35802) 2024-03-21T10:49:02.082 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.084+0000 7f9cf5800700 1 -- 172.21.15.153:0/3147962834 learned_addr learned my addr 172.21.15.153:0/3147962834 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:49:02.082 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.084+0000 7f9cf4e00700 1 --2- 172.21.15.153:0/3147962834 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9cf0101750 0x7f9cf0198ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:49:02.082 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.084+0000 7f9cf5800700 1 -- 172.21.15.153:0/3147962834 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9cf0101750 msgr2=0x7f9cf0198ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:49:02.082 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.084+0000 7f9cf5800700 1 --2- 172.21.15.153:0/3147962834 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9cf0101750 0x7f9cf0198ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:49:02.082 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.084+0000 7f9cf5800700 1 -- 172.21.15.153:0/3147962834 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ce4009d00 con 0x7f9cf0100da0 2024-03-21T10:49:02.082 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.085+0000 7f9cf5800700 1 --2- 172.21.15.153:0/3147962834 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9cf0100da0 0x7f9cf0198990 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f9ce400b070 tx=0x7f9ce400cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:49:02.083 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.085+0000 7f9cee000700 1 -- 172.21.15.153:0/3147962834 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9ce400e040 con 0x7f9cf0100da0 2024-03-21T10:49:02.083 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.085+0000 7f9cee000700 1 -- 172.21.15.153:0/3147962834 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9ce4003e60 con 0x7f9cf0100da0 2024-03-21T10:49:02.083 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.085+0000 7f9cf8200700 1 -- 172.21.15.153:0/3147962834 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cf019d5f0 con 0x7f9cf0100da0 2024-03-21T10:49:02.083 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.085+0000 7f9cee000700 1 -- 172.21.15.153:0/3147962834 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9ce401d910 con 0x7f9cf0100da0 2024-03-21T10:49:02.083 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.085+0000 7f9cf8200700 1 -- 172.21.15.153:0/3147962834 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cf019dae0 con 0x7f9cf0100da0 2024-03-21T10:49:02.083 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.086+0000 7f9cf8200700 1 -- 172.21.15.153:0/3147962834 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cf0104f20 con 0x7f9cf0100da0 2024-03-21T10:49:02.084 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.087+0000 7f9cee000700 1 -- 172.21.15.153:0/3147962834 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f9ce4004820 con 0x7f9cf0100da0 2024-03-21T10:49:02.085 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.089+0000 7f9cee000700 1 --2- 172.21.15.153:0/3147962834 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f9cd806e460 0x7f9cd8070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:49:02.085 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.089+0000 7f9cee000700 1 -- 172.21.15.153:0/3147962834 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f9ce408fe90 con 0x7f9cf0100da0 2024-03-21T10:49:02.086 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.089+0000 7f9cf4e00700 1 --2- 172.21.15.153:0/3147962834 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f9cd806e460 0x7f9cd8070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:49:02.086 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.089+0000 7f9cf4e00700 1 --2- 172.21.15.153:0/3147962834 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f9cd806e460 0x7f9cd8070920 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f9cf0199fb0 tx=0x7f9cdc005cf0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:49:02.090 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.094+0000 7f9cee000700 1 -- 172.21.15.153:0/3147962834 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f9ce405c9e0 con 0x7f9cf0100da0 2024-03-21T10:49:02.401 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.404+0000 7f9cf8200700 1 -- 172.21.15.153:0/3147962834 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f9cf004ea90 con 0x7f9cf0100da0 2024-03-21T10:49:02.401 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.405+0000 7f9cee000700 1 -- 172.21.15.153:0/3147962834 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f9ce405c570 con 0x7f9cf0100da0 2024-03-21T10:49:02.402 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:49:02.402 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": { 2024-03-21T10:49:02.402 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:49:02.402 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:49:02.402 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": { 2024-03-21T10:49:02.402 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:49:02.402 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:49:02.402 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": { 2024-03-21T10:49:02.402 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 6 2024-03-21T10:49:02.402 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:49:02.402 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": { 2024-03-21T10:49:02.402 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 4 2024-03-21T10:49:02.403 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:49:02.403 INFO:teuthology.orchestra.run.smithi153.stdout: "overall": { 2024-03-21T10:49:02.403 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 14 2024-03-21T10:49:02.403 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-03-21T10:49:02.403 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:49:02.404 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.407+0000 7f9cf8200700 1 -- 172.21.15.153:0/3147962834 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f9cd806e460 msgr2=0x7f9cd8070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:49:02.405 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.407+0000 7f9cf8200700 1 --2- 172.21.15.153:0/3147962834 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f9cd806e460 0x7f9cd8070920 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f9cf0199fb0 tx=0x7f9cdc005cf0 comp rx=0 tx=0).stop 2024-03-21T10:49:02.405 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.407+0000 7f9cf8200700 1 -- 172.21.15.153:0/3147962834 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9cf0100da0 msgr2=0x7f9cf0198990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:49:02.405 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.407+0000 7f9cf8200700 1 --2- 172.21.15.153:0/3147962834 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9cf0100da0 0x7f9cf0198990 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f9ce400b070 tx=0x7f9ce400cdd0 comp rx=0 tx=0).stop 2024-03-21T10:49:02.405 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.408+0000 7f9cf8200700 1 -- 172.21.15.153:0/3147962834 shutdown_connections 2024-03-21T10:49:02.405 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.408+0000 7f9cf8200700 1 --2- 172.21.15.153:0/3147962834 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f9cd806e460 0x7f9cd8070920 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:49:02.405 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.408+0000 7f9cf8200700 1 --2- 172.21.15.153:0/3147962834 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9cf0100da0 0x7f9cf0198990 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:49:02.405 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.408+0000 7f9cf8200700 1 --2- 172.21.15.153:0/3147962834 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9cf0101750 0x7f9cf0198ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:49:02.405 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.408+0000 7f9cf8200700 1 -- 172.21.15.153:0/3147962834 >> 172.21.15.153:0/3147962834 conn(0x7f9cf00fc630 msgr2=0x7f9cf00fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:49:02.405 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.408+0000 7f9cf8200700 1 -- 172.21.15.153:0/3147962834 shutdown_connections 2024-03-21T10:49:02.405 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.408+0000 7f9cf8200700 1 -- 172.21.15.153:0/3147962834 wait complete. 2024-03-21T10:49:02.557 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.560+0000 7feb53400700 1 -- 172.21.15.153:0/1490670730 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feb4c0ff290 msgr2=0x7feb4c0ff670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:49:02.557 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.560+0000 7feb53400700 1 --2- 172.21.15.153:0/1490670730 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feb4c0ff290 0x7feb4c0ff670 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7feb40009910 tx=0x7feb40009c20 comp rx=0 tx=0).stop 2024-03-21T10:49:02.557 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.560+0000 7feb53400700 1 -- 172.21.15.153:0/1490670730 shutdown_connections 2024-03-21T10:49:02.557 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.560+0000 7feb53400700 1 --2- 172.21.15.153:0/1490670730 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb4c0ffc40 0x7feb4c105780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:49:02.557 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.560+0000 7feb53400700 1 --2- 172.21.15.153:0/1490670730 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feb4c0ff290 0x7feb4c0ff670 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:49:02.557 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.560+0000 7feb53400700 1 -- 172.21.15.153:0/1490670730 >> 172.21.15.153:0/1490670730 conn(0x7feb4c075e30 msgr2=0x7feb4c0fd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:49:02.557 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.560+0000 7feb53400700 1 -- 172.21.15.153:0/1490670730 shutdown_connections 2024-03-21T10:49:02.558 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.560+0000 7feb53400700 1 -- 172.21.15.153:0/1490670730 wait complete. 2024-03-21T10:49:02.558 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.561+0000 7feb53400700 1 Processor -- start 2024-03-21T10:49:02.558 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.561+0000 7feb53400700 1 -- start start 2024-03-21T10:49:02.558 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.562+0000 7feb53400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feb4c0ff290 0x7feb4c100210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:49:02.559 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.562+0000 7feb53400700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb4c0ffc40 0x7feb4c100750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:49:02.559 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.562+0000 7feb53400700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb4c100c90 con 0x7feb4c0ff290 2024-03-21T10:49:02.559 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.562+0000 7feb53400700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb4c100dd0 con 0x7feb4c0ffc40 2024-03-21T10:49:02.560 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.562+0000 7feb50a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feb4c0ff290 0x7feb4c100210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:49:02.560 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.562+0000 7feb50a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feb4c0ff290 0x7feb4c100210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:35818/0 (socket says 172.21.15.153:35818) 2024-03-21T10:49:02.560 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.562+0000 7feb50a00700 1 -- 172.21.15.153:0/2016740836 learned_addr learned my addr 172.21.15.153:0/2016740836 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:49:02.560 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.562+0000 7feb4be00700 1 --2- 172.21.15.153:0/2016740836 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb4c0ffc40 0x7feb4c100750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:49:02.560 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.563+0000 7feb50a00700 1 -- 172.21.15.153:0/2016740836 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb4c0ffc40 msgr2=0x7feb4c100750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:49:02.560 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.563+0000 7feb50a00700 1 --2- 172.21.15.153:0/2016740836 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb4c0ffc40 0x7feb4c100750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:49:02.560 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.563+0000 7feb50a00700 1 -- 172.21.15.153:0/2016740836 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb400095f0 con 0x7feb4c0ff290 2024-03-21T10:49:02.561 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.563+0000 7feb50a00700 1 --2- 172.21.15.153:0/2016740836 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feb4c0ff290 0x7feb4c100210 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7feb400092e0 tx=0x7feb4000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:49:02.561 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.563+0000 7feb49600700 1 -- 172.21.15.153:0/2016740836 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feb4001d070 con 0x7feb4c0ff290 2024-03-21T10:49:02.561 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.563+0000 7feb49600700 1 -- 172.21.15.153:0/2016740836 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7feb40005e00 con 0x7feb4c0ff290 2024-03-21T10:49:02.561 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.563+0000 7feb53400700 1 -- 172.21.15.153:0/2016740836 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb4c101050 con 0x7feb4c0ff290 2024-03-21T10:49:02.561 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.563+0000 7feb49600700 1 -- 172.21.15.153:0/2016740836 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feb40017400 con 0x7feb4c0ff290 2024-03-21T10:49:02.561 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.563+0000 7feb53400700 1 -- 172.21.15.153:0/2016740836 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb4c1a2c20 con 0x7feb4c0ff290 2024-03-21T10:49:02.561 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.564+0000 7feb53400700 1 -- 172.21.15.153:0/2016740836 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb4c109120 con 0x7feb4c0ff290 2024-03-21T10:49:02.562 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.565+0000 7feb49600700 1 -- 172.21.15.153:0/2016740836 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7feb4000f460 con 0x7feb4c0ff290 2024-03-21T10:49:02.563 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.566+0000 7feb49600700 1 --2- 172.21.15.153:0/2016740836 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7feb3406e460 0x7feb34070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:49:02.563 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.567+0000 7feb49600700 1 -- 172.21.15.153:0/2016740836 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7feb4008f9f0 con 0x7feb4c0ff290 2024-03-21T10:49:02.564 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.567+0000 7feb4be00700 1 --2- 172.21.15.153:0/2016740836 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7feb3406e460 0x7feb34070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:49:02.564 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.568+0000 7feb4be00700 1 --2- 172.21.15.153:0/2016740836 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7feb3406e460 0x7feb34070920 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7feb38005950 tx=0x7feb380058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:49:02.569 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.572+0000 7feb49600700 1 -- 172.21.15.153:0/2016740836 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7feb40058a90 con 0x7feb4c0ff290 2024-03-21T10:49:02.832 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.835+0000 7feb53400700 1 -- 172.21.15.153:0/2016740836 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7feb4c04ea90 con 0x7feb4c0ff290 2024-03-21T10:49:02.832 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:49:02 smithi153 ceph-mon[113733]: from='client.35966 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:49:02.832 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:49:02 smithi153 ceph-mon[113733]: from='client.35970 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:49:02.832 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:49:02 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/3147962834' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:49:02.833 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.836+0000 7feb49600700 1 -- 172.21.15.153:0/2016740836 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1876 (secure 0 0 0) 0x7feb4005c150 con 0x7feb4c0ff290 2024-03-21T10:49:02.833 INFO:teuthology.orchestra.run.smithi153.stdout:e32 2024-03-21T10:49:02.833 INFO:teuthology.orchestra.run.smithi153.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-21T10:49:02.833 INFO:teuthology.orchestra.run.smithi153.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:49:02.833 INFO:teuthology.orchestra.run.smithi153.stdout:legacy client fscid: 1 2024-03-21T10:49:02.834 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:49:02.834 INFO:teuthology.orchestra.run.smithi153.stdout:Filesystem 'cephfs' (1) 2024-03-21T10:49:02.834 INFO:teuthology.orchestra.run.smithi153.stdout:fs_name cephfs 2024-03-21T10:49:02.834 INFO:teuthology.orchestra.run.smithi153.stdout:epoch 23 2024-03-21T10:49:02.834 INFO:teuthology.orchestra.run.smithi153.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-21T10:49:02.834 INFO:teuthology.orchestra.run.smithi153.stdout:created 2024-03-21T10:04:27.086576+0000 2024-03-21T10:49:02.834 INFO:teuthology.orchestra.run.smithi153.stdout:modified 2024-03-21T10:19:05.022092+0000 2024-03-21T10:49:02.834 INFO:teuthology.orchestra.run.smithi153.stdout:tableserver 0 2024-03-21T10:49:02.834 INFO:teuthology.orchestra.run.smithi153.stdout:root 0 2024-03-21T10:49:02.834 INFO:teuthology.orchestra.run.smithi153.stdout:session_timeout 60 2024-03-21T10:49:02.834 INFO:teuthology.orchestra.run.smithi153.stdout:session_autoclose 300 2024-03-21T10:49:02.834 INFO:teuthology.orchestra.run.smithi153.stdout:max_file_size 1099511627776 2024-03-21T10:49:02.834 INFO:teuthology.orchestra.run.smithi153.stdout:max_xattr_size 65536 2024-03-21T10:49:02.834 INFO:teuthology.orchestra.run.smithi153.stdout:required_client_features {} 2024-03-21T10:49:02.835 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure 0 2024-03-21T10:49:02.835 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure_osd_epoch 94 2024-03-21T10:49:02.835 INFO:teuthology.orchestra.run.smithi153.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:49:02.835 INFO:teuthology.orchestra.run.smithi153.stdout:max_mds 2 2024-03-21T10:49:02.835 INFO:teuthology.orchestra.run.smithi153.stdout:in 0,1 2024-03-21T10:49:02.835 INFO:teuthology.orchestra.run.smithi153.stdout:up {} 2024-03-21T10:49:02.835 INFO:teuthology.orchestra.run.smithi153.stdout:failed 0,1 2024-03-21T10:49:02.835 INFO:teuthology.orchestra.run.smithi153.stdout:damaged 2024-03-21T10:49:02.835 INFO:teuthology.orchestra.run.smithi153.stdout:stopped 2024-03-21T10:49:02.835 INFO:teuthology.orchestra.run.smithi153.stdout:data_pools [3] 2024-03-21T10:49:02.835 INFO:teuthology.orchestra.run.smithi153.stdout:metadata_pool 2 2024-03-21T10:49:02.835 INFO:teuthology.orchestra.run.smithi153.stdout:inline_data enabled 2024-03-21T10:49:02.835 INFO:teuthology.orchestra.run.smithi153.stdout:balancer 2024-03-21T10:49:02.835 INFO:teuthology.orchestra.run.smithi153.stdout:bal_rank_mask -1 2024-03-21T10:49:02.836 INFO:teuthology.orchestra.run.smithi153.stdout:standby_count_wanted 1 2024-03-21T10:49:02.836 INFO:teuthology.orchestra.run.smithi153.stdout:qdb_cluster leader: 0 members: 2024-03-21T10:49:02.836 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:49:02.836 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:49:02.836 INFO:teuthology.orchestra.run.smithi153.stdout:Standby daemons: 2024-03-21T10:49:02.836 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:49:02.836 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.zipckr{-1:34442} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6826/1103846295,v1:172.21.15.153:6827/1103846295] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:49:02.836 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.iuyluj{-1:34474} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6828/1738630832,v1:172.21.15.153:6829/1738630832] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:49:02.836 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.yyimst{-1:44313} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.174:6824/4127786797,v1:172.21.15.174:6825/4127786797] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:49:02.836 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.nhozag{-1:44687} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.174:6826/2767517845,v1:172.21.15.174:6827/2767517845] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:49:02.837 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.839+0000 7feb53400700 1 -- 172.21.15.153:0/2016740836 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7feb3406e460 msgr2=0x7feb34070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:49:02.837 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.839+0000 7feb53400700 1 --2- 172.21.15.153:0/2016740836 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7feb3406e460 0x7feb34070920 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7feb38005950 tx=0x7feb380058e0 comp rx=0 tx=0).stop 2024-03-21T10:49:02.837 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.839+0000 7feb53400700 1 -- 172.21.15.153:0/2016740836 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feb4c0ff290 msgr2=0x7feb4c100210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:49:02.837 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.839+0000 7feb53400700 1 --2- 172.21.15.153:0/2016740836 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feb4c0ff290 0x7feb4c100210 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7feb400092e0 tx=0x7feb4000be20 comp rx=0 tx=0).stop 2024-03-21T10:49:02.837 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.839+0000 7feb53400700 1 -- 172.21.15.153:0/2016740836 shutdown_connections 2024-03-21T10:49:02.837 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.839+0000 7feb53400700 1 --2- 172.21.15.153:0/2016740836 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7feb3406e460 0x7feb34070920 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:49:02.837 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.839+0000 7feb53400700 1 --2- 172.21.15.153:0/2016740836 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7feb4c0ff290 0x7feb4c100210 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:49:02.838 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.839+0000 7feb53400700 1 --2- 172.21.15.153:0/2016740836 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb4c0ffc40 0x7feb4c100750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:49:02.838 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.839+0000 7feb53400700 1 -- 172.21.15.153:0/2016740836 >> 172.21.15.153:0/2016740836 conn(0x7feb4c075e30 msgr2=0x7feb4c1049d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:49:02.838 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.839+0000 7feb53400700 1 -- 172.21.15.153:0/2016740836 shutdown_connections 2024-03-21T10:49:02.838 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.839+0000 7feb53400700 1 -- 172.21.15.153:0/2016740836 wait complete. 2024-03-21T10:49:02.838 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 32 2024-03-21T10:49:02.946 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:49:02 smithi174 ceph-mon[88004]: from='client.35966 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:49:02.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:49:02 smithi174 ceph-mon[88004]: from='client.35970 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:49:02.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:49:02 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/3147962834' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:49:02.984 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.987+0000 7fd8ae000700 1 -- 172.21.15.153:0/3120587515 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd8a810d280 msgr2=0x7fd8a810d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:49:02.984 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.987+0000 7fd8ae000700 1 --2- 172.21.15.153:0/3120587515 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd8a810d280 0x7fd8a810d660 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fd898009910 tx=0x7fd898009c20 comp rx=0 tx=0).stop 2024-03-21T10:49:02.984 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.987+0000 7fd8ae000700 1 -- 172.21.15.153:0/3120587515 shutdown_connections 2024-03-21T10:49:02.984 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.987+0000 7fd8ae000700 1 --2- 172.21.15.153:0/3120587515 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8a81071d0 0x7fd8a8107650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:49:02.985 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.987+0000 7fd8ae000700 1 --2- 172.21.15.153:0/3120587515 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd8a810d280 0x7fd8a810d660 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:49:02.985 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.987+0000 7fd8ae000700 1 -- 172.21.15.153:0/3120587515 >> 172.21.15.153:0/3120587515 conn(0x7fd8a80769e0 msgr2=0x7fd8a8078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:49:02.985 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.987+0000 7fd8ae000700 1 -- 172.21.15.153:0/3120587515 shutdown_connections 2024-03-21T10:49:02.985 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.987+0000 7fd8ae000700 1 -- 172.21.15.153:0/3120587515 wait complete. 2024-03-21T10:49:02.986 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.988+0000 7fd8ae000700 1 Processor -- start 2024-03-21T10:49:02.986 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.988+0000 7fd8ae000700 1 -- start start 2024-03-21T10:49:02.986 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.989+0000 7fd8ae000700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8a81071d0 0x7fd8a819ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:49:02.986 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.989+0000 7fd8ae000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd8a810d280 0x7fd8a819d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:49:02.986 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.989+0000 7fd8ae000700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a819da30 con 0x7fd8a810d280 2024-03-21T10:49:02.986 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.989+0000 7fd8ae000700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a81a1700 con 0x7fd8a81071d0 2024-03-21T10:49:02.986 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.989+0000 7fd89fe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd8a810d280 0x7fd8a819d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:49:02.987 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.989+0000 7fd89fe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd8a810d280 0x7fd8a819d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:56456/0 (socket says 172.21.15.153:56456) 2024-03-21T10:49:02.987 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.989+0000 7fd89fe00700 1 -- 172.21.15.153:0/1475217749 learned_addr learned my addr 172.21.15.153:0/1475217749 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:49:02.987 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.989+0000 7fd8a7400700 1 --2- 172.21.15.153:0/1475217749 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8a81071d0 0x7fd8a819ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:49:02.987 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.989+0000 7fd89fe00700 1 -- 172.21.15.153:0/1475217749 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8a81071d0 msgr2=0x7fd8a819ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:49:02.987 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.989+0000 7fd89fe00700 1 --2- 172.21.15.153:0/1475217749 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8a81071d0 0x7fd8a819ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:49:02.987 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.989+0000 7fd89fe00700 1 -- 172.21.15.153:0/1475217749 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8980095f0 con 0x7fd8a810d280 2024-03-21T10:49:02.987 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.989+0000 7fd8a7400700 1 --2- 172.21.15.153:0/1475217749 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8a81071d0 0x7fd8a819ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:49:02.988 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.990+0000 7fd89fe00700 1 --2- 172.21.15.153:0/1475217749 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd8a810d280 0x7fd8a819d350 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fd89000b4b0 tx=0x7fd89000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:49:02.988 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.990+0000 7fd8a4c00700 1 -- 172.21.15.153:0/1475217749 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd890011800 con 0x7fd8a810d280 2024-03-21T10:49:02.988 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.990+0000 7fd8ae000700 1 -- 172.21.15.153:0/1475217749 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8a81a19e0 con 0x7fd8a810d280 2024-03-21T10:49:02.988 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.990+0000 7fd8a4c00700 1 -- 172.21.15.153:0/1475217749 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd890011e20 con 0x7fd8a810d280 2024-03-21T10:49:02.988 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.990+0000 7fd8a4c00700 1 -- 172.21.15.153:0/1475217749 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd89000f2e0 con 0x7fd8a810d280 2024-03-21T10:49:02.988 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.990+0000 7fd8ae000700 1 -- 172.21.15.153:0/1475217749 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8a81a1fb0 con 0x7fd8a810d280 2024-03-21T10:49:02.988 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.991+0000 7fd8ae000700 1 -- 172.21.15.153:0/1475217749 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8a810f6f0 con 0x7fd8a810d280 2024-03-21T10:49:02.989 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.992+0000 7fd8a4c00700 1 -- 172.21.15.153:0/1475217749 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fd890010380 con 0x7fd8a810d280 2024-03-21T10:49:02.990 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.993+0000 7fd8a4c00700 1 --2- 172.21.15.153:0/1475217749 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd88c06e460 0x7fd88c070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:49:02.990 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.993+0000 7fd8a4c00700 1 -- 172.21.15.153:0/1475217749 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fd89008dff0 con 0x7fd8a810d280 2024-03-21T10:49:02.990 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.993+0000 7fd8a7400700 1 --2- 172.21.15.153:0/1475217749 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd88c06e460 0x7fd88c070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:49:02.990 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.994+0000 7fd8a7400700 1 --2- 172.21.15.153:0/1475217749 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd88c06e460 0x7fd88c070920 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fd898006010 tx=0x7fd89800b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:49:02.994 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:02.998+0000 7fd8a4c00700 1 -- 172.21.15.153:0/1475217749 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd8900570f0 con 0x7fd8a810d280 2024-03-21T10:49:03.228 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:03.231+0000 7fd8ae000700 1 -- 172.21.15.153:0/1475217749 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd8a81a2280 con 0x7fd88c06e460 2024-03-21T10:49:03.230 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:03.233+0000 7fd8a4c00700 1 -- 172.21.15.153:0/1475217749 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fd8a81a2280 con 0x7fd88c06e460 2024-03-21T10:49:03.231 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:49:03.231 INFO:teuthology.orchestra.run.smithi153.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232", 2024-03-21T10:49:03.231 INFO:teuthology.orchestra.run.smithi153.stdout: "in_progress": true, 2024-03-21T10:49:03.231 INFO:teuthology.orchestra.run.smithi153.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-21T10:49:03.231 INFO:teuthology.orchestra.run.smithi153.stdout: "services_complete": [ 2024-03-21T10:49:03.231 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr", 2024-03-21T10:49:03.231 INFO:teuthology.orchestra.run.smithi153.stdout: "mon", 2024-03-21T10:49:03.231 INFO:teuthology.orchestra.run.smithi153.stdout: "mds", 2024-03-21T10:49:03.231 INFO:teuthology.orchestra.run.smithi153.stdout: "crash", 2024-03-21T10:49:03.231 INFO:teuthology.orchestra.run.smithi153.stdout: "osd" 2024-03-21T10:49:03.231 INFO:teuthology.orchestra.run.smithi153.stdout: ], 2024-03-21T10:49:03.231 INFO:teuthology.orchestra.run.smithi153.stdout: "progress": "16/23 daemons upgraded", 2024-03-21T10:49:03.232 INFO:teuthology.orchestra.run.smithi153.stdout: "message": "Currently upgrading mds daemons", 2024-03-21T10:49:03.232 INFO:teuthology.orchestra.run.smithi153.stdout: "is_paused": false 2024-03-21T10:49:03.232 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:49:03.234 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:03.237+0000 7fd8ae000700 1 -- 172.21.15.153:0/1475217749 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd88c06e460 msgr2=0x7fd88c070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:49:03.234 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:03.237+0000 7fd8ae000700 1 --2- 172.21.15.153:0/1475217749 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd88c06e460 0x7fd88c070920 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fd898006010 tx=0x7fd89800b540 comp rx=0 tx=0).stop 2024-03-21T10:49:03.235 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:03.237+0000 7fd8ae000700 1 -- 172.21.15.153:0/1475217749 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd8a810d280 msgr2=0x7fd8a819d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:49:03.235 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:03.237+0000 7fd8ae000700 1 --2- 172.21.15.153:0/1475217749 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd8a810d280 0x7fd8a819d350 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fd89000b4b0 tx=0x7fd89000b870 comp rx=0 tx=0).stop 2024-03-21T10:49:03.235 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:03.237+0000 7fd8ae000700 1 -- 172.21.15.153:0/1475217749 shutdown_connections 2024-03-21T10:49:03.235 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:03.237+0000 7fd8ae000700 1 --2- 172.21.15.153:0/1475217749 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8a81071d0 0x7fd8a819ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:49:03.235 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:03.237+0000 7fd8ae000700 1 --2- 172.21.15.153:0/1475217749 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd88c06e460 0x7fd88c070920 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:49:03.235 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:03.237+0000 7fd8ae000700 1 --2- 172.21.15.153:0/1475217749 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd8a810d280 0x7fd8a819d350 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:49:03.235 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:03.237+0000 7fd8ae000700 1 -- 172.21.15.153:0/1475217749 >> 172.21.15.153:0/1475217749 conn(0x7fd8a80769e0 msgr2=0x7fd8a80785c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:49:03.235 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:03.238+0000 7fd8ae000700 1 -- 172.21.15.153:0/1475217749 shutdown_connections 2024-03-21T10:49:03.235 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:03.238+0000 7fd8ae000700 1 -- 172.21.15.153:0/1475217749 wait complete. 2024-03-21T10:49:03.392 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:03.395+0000 7fe7d2a00700 1 -- 172.21.15.153:0/1720253816 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe7cc1017c0 msgr2=0x7fe7cc105810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:49:03.392 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:03.395+0000 7fe7d2a00700 1 --2- 172.21.15.153:0/1720253816 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe7cc1017c0 0x7fe7cc105810 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fe7b8009910 tx=0x7fe7b8009c20 comp rx=0 tx=0).stop 2024-03-21T10:49:03.393 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:03.395+0000 7fe7d2a00700 1 -- 172.21.15.153:0/1720253816 shutdown_connections 2024-03-21T10:49:03.393 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:03.395+0000 7fe7d2a00700 1 --2- 172.21.15.153:0/1720253816 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe7cc1017c0 0x7fe7cc105810 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:49:03.393 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:03.395+0000 7fe7d2a00700 1 --2- 172.21.15.153:0/1720253816 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe7cc100e10 0x7fe7cc1011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:49:03.393 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:03.395+0000 7fe7d2a00700 1 -- 172.21.15.153:0/1720253816 >> 172.21.15.153:0/1720253816 conn(0x7fe7cc0fc6c0 msgr2=0x7fe7cc0feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:49:03.393 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:03.395+0000 7fe7d2a00700 1 -- 172.21.15.153:0/1720253816 shutdown_connections 2024-03-21T10:49:03.393 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:03.395+0000 7fe7d2a00700 1 -- 172.21.15.153:0/1720253816 wait complete. 2024-03-21T10:49:03.393 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:03.397+0000 7fe7d2a00700 1 Processor -- start 2024-03-21T10:49:03.393 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:03.397+0000 7fe7d2a00700 1 -- start start 2024-03-21T10:49:03.394 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:03.397+0000 7fe7d2a00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe7cc100e10 0x7fe7cc19c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:49:03.394 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:03.397+0000 7fe7d2a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe7cc1017c0 0x7fe7cc19cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:49:03.395 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:03.397+0000 7fe7d2a00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7cc19d1a0 con 0x7fe7cc1017c0 2024-03-21T10:49:03.395 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:03.397+0000 7fe7d2a00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7cc196650 con 0x7fe7cc100e10 2024-03-21T10:49:03.395 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:03.398+0000 7fe7cb400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe7cc1017c0 0x7fe7cc19cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:49:03.395 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:03.398+0000 7fe7cb400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe7cc1017c0 0x7fe7cc19cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:56478/0 (socket says 172.21.15.153:56478) 2024-03-21T10:49:03.395 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:03.398+0000 7fe7cb400700 1 -- 172.21.15.153:0/301313610 learned_addr learned my addr 172.21.15.153:0/301313610 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:49:03.396 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:03.398+0000 7fe7cbe00700 1 --2- 172.21.15.153:0/301313610 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe7cc100e10 0x7fe7cc19c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:49:03.396 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:03.398+0000 7fe7cb400700 1 -- 172.21.15.153:0/301313610 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe7cc100e10 msgr2=0x7fe7cc19c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:49:03.396 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:03.398+0000 7fe7cb400700 1 --2- 172.21.15.153:0/301313610 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe7cc100e10 0x7fe7cc19c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:49:03.396 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:03.398+0000 7fe7cb400700 1 -- 172.21.15.153:0/301313610 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7b80095f0 con 0x7fe7cc1017c0 2024-03-21T10:49:03.396 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:03.399+0000 7fe7cb400700 1 --2- 172.21.15.153:0/301313610 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe7cc1017c0 0x7fe7cc19cb10 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fe7b8005f50 tx=0x7fe7b800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:49:03.396 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:03.399+0000 7fe7c8c00700 1 -- 172.21.15.153:0/301313610 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe7b801d070 con 0x7fe7cc1017c0 2024-03-21T10:49:03.396 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:03.399+0000 7fe7c8c00700 1 -- 172.21.15.153:0/301313610 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe7b800f460 con 0x7fe7cc1017c0 2024-03-21T10:49:03.396 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:03.399+0000 7fe7c8c00700 1 -- 172.21.15.153:0/301313610 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe7b8017630 con 0x7fe7cc1017c0 2024-03-21T10:49:03.396 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:03.399+0000 7fe7d2a00700 1 -- 172.21.15.153:0/301313610 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7cc1968d0 con 0x7fe7cc1017c0 2024-03-21T10:49:03.396 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:03.399+0000 7fe7d2a00700 1 -- 172.21.15.153:0/301313610 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7cc196dc0 con 0x7fe7cc1017c0 2024-03-21T10:49:03.397 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:03.400+0000 7fe7d2a00700 1 -- 172.21.15.153:0/301313610 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7cc109210 con 0x7fe7cc1017c0 2024-03-21T10:49:03.398 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:03.401+0000 7fe7c8c00700 1 -- 172.21.15.153:0/301313610 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fe7b800f5d0 con 0x7fe7cc1017c0 2024-03-21T10:49:03.399 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:03.402+0000 7fe7c8c00700 1 --2- 172.21.15.153:0/301313610 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe7b006e460 0x7fe7b0070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:49:03.399 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:03.403+0000 7fe7c8c00700 1 -- 172.21.15.153:0/301313610 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fe7b808faa0 con 0x7fe7cc1017c0 2024-03-21T10:49:03.400 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:03.403+0000 7fe7cbe00700 1 --2- 172.21.15.153:0/301313610 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe7b006e460 0x7fe7b0070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:49:03.400 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:03.404+0000 7fe7cbe00700 1 --2- 172.21.15.153:0/301313610 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe7b006e460 0x7fe7b0070920 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fe7bc005fd0 tx=0x7fe7bc00a560 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:49:03.403 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:03.407+0000 7fe7c8c00700 1 -- 172.21.15.153:0/301313610 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe7b805c670 con 0x7fe7cc1017c0 2024-03-21T10:49:03.701 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:03.704+0000 7fe7d2a00700 1 -- 172.21.15.153:0/301313610 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fe7cc04ea90 con 0x7fe7cc1017c0 2024-03-21T10:49:03.701 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:03.704+0000 7fe7c8c00700 1 -- 172.21.15.153:0/301313610 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4493 (secure 0 0 0) 0x7fe7b805c200 con 0x7fe7cc1017c0 2024-03-21T10:49:03.702 INFO:teuthology.orchestra.run.smithi153.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-03-21T10:49:03.702 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-03-21T10:49:03.702 INFO:teuthology.orchestra.run.smithi153.stdout: host smithi174 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi174/config 2024-03-21T10:49:03.702 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/podman: timeout after 895 seconds 2024-03-21T10:49:03.703 INFO:teuthology.orchestra.run.smithi153.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 -e NODE_NAME=smithi174 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/run/ceph:z -v /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpd910r3_i:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 inventory --format=json-pretty --filter-for-batch 2024-03-21T10:49:03.703 INFO:teuthology.orchestra.run.smithi153.stdout:Traceback (most recent call last): 2024-03-21T10:49:03.703 INFO:teuthology.orchestra.run.smithi153.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-03-21T10:49:03.703 INFO:teuthology.orchestra.run.smithi153.stdout: return _run_code(code, main_globals, None, 2024-03-21T10:49:03.703 INFO:teuthology.orchestra.run.smithi153.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-03-21T10:49:03.703 INFO:teuthology.orchestra.run.smithi153.stdout: exec(code, run_globals) 2024-03-21T10:49:03.703 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 5606, in 2024-03-21T10:49:03.703 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 5594, in main 2024-03-21T10:49:03.703 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 406, in _infer_config 2024-03-21T10:49:03.703 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 321, in _infer_fsid 2024-03-21T10:49:03.703 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 434, in _infer_image 2024-03-21T10:49:03.703 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 308, in _validate_fsid 2024-03-21T10:49:03.703 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 3357, in command_ceph_volume 2024-03-21T10:49:03.703 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-03-21T10:49:03.704 INFO:teuthology.orchestra.run.smithi153.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 -e NODE_NAME=smithi174 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/run/ceph:z -v /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpd910r3_i:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 inventory --format=json-pretty --filter-for-batch 2024-03-21T10:49:03.704 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-21T10:49:03.704 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is degraded 2024-03-21T10:49:03.704 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-21T10:49:03.704 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-21T10:49:03.704 INFO:teuthology.orchestra.run.smithi153.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-21T10:49:03.704 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-21T10:49:03.705 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:03.707+0000 7fe7d2a00700 1 -- 172.21.15.153:0/301313610 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe7b006e460 msgr2=0x7fe7b0070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:49:03.705 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:03.707+0000 7fe7d2a00700 1 --2- 172.21.15.153:0/301313610 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe7b006e460 0x7fe7b0070920 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fe7bc005fd0 tx=0x7fe7bc00a560 comp rx=0 tx=0).stop 2024-03-21T10:49:03.705 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:03.707+0000 7fe7d2a00700 1 -- 172.21.15.153:0/301313610 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe7cc1017c0 msgr2=0x7fe7cc19cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:49:03.705 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:03.707+0000 7fe7d2a00700 1 --2- 172.21.15.153:0/301313610 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe7cc1017c0 0x7fe7cc19cb10 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fe7b8005f50 tx=0x7fe7b800be20 comp rx=0 tx=0).stop 2024-03-21T10:49:03.705 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:03.707+0000 7fe7d2a00700 1 -- 172.21.15.153:0/301313610 shutdown_connections 2024-03-21T10:49:03.705 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:03.707+0000 7fe7d2a00700 1 --2- 172.21.15.153:0/301313610 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe7cc100e10 0x7fe7cc19c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:49:03.705 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:03.707+0000 7fe7d2a00700 1 --2- 172.21.15.153:0/301313610 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe7b006e460 0x7fe7b0070920 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:49:03.705 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:03.707+0000 7fe7d2a00700 1 --2- 172.21.15.153:0/301313610 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe7cc1017c0 0x7fe7cc19cb10 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:49:03.705 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:03.707+0000 7fe7d2a00700 1 -- 172.21.15.153:0/301313610 >> 172.21.15.153:0/301313610 conn(0x7fe7cc0fc6c0 msgr2=0x7fe7cc0fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:49:03.705 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:03.708+0000 7fe7d2a00700 1 -- 172.21.15.153:0/301313610 shutdown_connections 2024-03-21T10:49:03.706 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:03.708+0000 7fe7d2a00700 1 -- 172.21.15.153:0/301313610 wait complete. 2024-03-21T10:49:03.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:49:03 smithi174 ceph-mon[88004]: from='client.35974 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:49:03.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:49:03 smithi174 ceph-mon[88004]: pgmap v1117: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:49:03.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:49:03 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/2016740836' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:49:04.052 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:49:03 smithi153 ceph-mon[113733]: from='client.35974 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:49:04.052 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:49:03 smithi153 ceph-mon[113733]: pgmap v1117: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:49:04.052 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:49:03 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/2016740836' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:49:04.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:49:04 smithi174 ceph-mon[88004]: from='client.35986 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:49:04.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:49:04 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/301313610' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:49:05.052 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:49:04 smithi153 ceph-mon[113733]: from='client.35986 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:49:05.052 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:49:04 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/301313610' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:49:05.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:49:05 smithi174 ceph-mon[88004]: pgmap v1118: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:49:06.051 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:49:05 smithi153 ceph-mon[113733]: pgmap v1118: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:49:06.946 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:49:06 smithi174 ceph-mon[88004]: pgmap v1119: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:49:07.051 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:49:06 smithi153 ceph-mon[113733]: pgmap v1119: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:49:09.447 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:49:09 smithi174 ceph-mon[88004]: pgmap v1120: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:49:09.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:49:09 smithi153 ceph-mon[113733]: pgmap v1120: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:49:11.447 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:49:11 smithi174 ceph-mon[88004]: pgmap v1121: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:49:11.551 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:49:11 smithi153 ceph-mon[113733]: pgmap v1121: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:49:13.446 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:49:13 smithi174 ceph-mon[88004]: pgmap v1122: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:49:13.551 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:49:13 smithi153 ceph-mon[113733]: pgmap v1122: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:49:15.301 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:49:15 smithi153 ceph-mon[113733]: pgmap v1123: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:49:15.446 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:49:15 smithi174 ceph-mon[88004]: pgmap v1123: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:49:17.446 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:49:17 smithi174 ceph-mon[88004]: pgmap v1124: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:49:17.447 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:49:17 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:49:17.551 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:49:17 smithi153 ceph-mon[113733]: pgmap v1124: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:49:17.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:49:17 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:49:19.447 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:49:19 smithi174 ceph-mon[88004]: pgmap v1125: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:49:19.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:49:19 smithi153 ceph-mon[113733]: pgmap v1125: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:49:21.447 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:49:21 smithi174 ceph-mon[88004]: pgmap v1126: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:49:21.551 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:49:21 smithi153 ceph-mon[113733]: pgmap v1126: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:49:23.447 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:49:23 smithi174 ceph-mon[88004]: pgmap v1127: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:49:23.551 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:49:23 smithi153 ceph-mon[113733]: pgmap v1127: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:49:25.446 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:49:25 smithi174 ceph-mon[88004]: pgmap v1128: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:49:25.551 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:49:25 smithi153 ceph-mon[113733]: pgmap v1128: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:49:27.446 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:49:27 smithi174 ceph-mon[88004]: pgmap v1129: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:49:27.551 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:49:27 smithi153 ceph-mon[113733]: pgmap v1129: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:49:29.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:49:29 smithi153 ceph-mon[113733]: pgmap v1130: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:49:29.696 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:49:29 smithi174 ceph-mon[88004]: pgmap v1130: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:49:31.551 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:49:31 smithi153 ceph-mon[113733]: pgmap v1131: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:49:31.696 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:49:31 smithi174 ceph-mon[88004]: pgmap v1131: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:49:32.551 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:49:32 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:49:32.696 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:49:32 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:49:33.551 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:49:33 smithi153 ceph-mon[113733]: pgmap v1132: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:49:33.662 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:49:33 smithi174 ceph-mon[88004]: pgmap v1132: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:49:33.876 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:33.879+0000 7fce2d000700 1 -- 172.21.15.153:0/2438544582 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fce28107d80 msgr2=0x7fce2810bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:49:33.876 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:33.879+0000 7fce2d000700 1 --2- 172.21.15.153:0/2438544582 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fce28107d80 0x7fce2810bd80 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fce14009910 tx=0x7fce14009c20 comp rx=0 tx=0).stop 2024-03-21T10:49:33.877 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:33.879+0000 7fce2d000700 1 -- 172.21.15.153:0/2438544582 shutdown_connections 2024-03-21T10:49:33.877 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:33.879+0000 7fce2d000700 1 --2- 172.21.15.153:0/2438544582 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fce28107d80 0x7fce2810bd80 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:49:33.877 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:33.879+0000 7fce2d000700 1 --2- 172.21.15.153:0/2438544582 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fce281073d0 0x7fce281077b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:49:33.877 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:33.879+0000 7fce2d000700 1 -- 172.21.15.153:0/2438544582 >> 172.21.15.153:0/2438544582 conn(0x7fce28076970 msgr2=0x7fce28078d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:49:33.877 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:33.879+0000 7fce2d000700 1 -- 172.21.15.153:0/2438544582 shutdown_connections 2024-03-21T10:49:33.877 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:33.880+0000 7fce2d000700 1 -- 172.21.15.153:0/2438544582 wait complete. 2024-03-21T10:49:33.877 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:33.881+0000 7fce2d000700 1 Processor -- start 2024-03-21T10:49:33.877 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:33.881+0000 7fce2d000700 1 -- start start 2024-03-21T10:49:33.878 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:33.881+0000 7fce2d000700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fce281073d0 0x7fce2819ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:49:33.879 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:33.881+0000 7fce2d000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fce28107d80 0x7fce2819d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:49:33.879 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:33.881+0000 7fce2d000700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce2819d880 con 0x7fce28107d80 2024-03-21T10:49:33.879 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:33.881+0000 7fce2d000700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce281a05b0 con 0x7fce281073d0 2024-03-21T10:49:33.879 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:33.881+0000 7fce25400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fce28107d80 0x7fce2819d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:49:33.879 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:33.881+0000 7fce25400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fce28107d80 0x7fce2819d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:49910/0 (socket says 172.21.15.153:49910) 2024-03-21T10:49:33.879 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:33.881+0000 7fce25400700 1 -- 172.21.15.153:0/2127271787 learned_addr learned my addr 172.21.15.153:0/2127271787 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:49:33.879 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:33.882+0000 7fce25e00700 1 --2- 172.21.15.153:0/2127271787 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fce281073d0 0x7fce2819ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:49:33.879 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:33.882+0000 7fce25400700 1 -- 172.21.15.153:0/2127271787 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fce281073d0 msgr2=0x7fce2819ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:49:33.879 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:33.882+0000 7fce25400700 1 --2- 172.21.15.153:0/2127271787 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fce281073d0 0x7fce2819ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:49:33.879 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:33.882+0000 7fce25400700 1 -- 172.21.15.153:0/2127271787 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce140095f0 con 0x7fce28107d80 2024-03-21T10:49:33.879 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:33.882+0000 7fce25400700 1 --2- 172.21.15.153:0/2127271787 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fce28107d80 0x7fce2819d1f0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fce14005f50 tx=0x7fce1400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:49:33.879 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:33.882+0000 7fce1ea00700 1 -- 172.21.15.153:0/2127271787 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fce1401d070 con 0x7fce28107d80 2024-03-21T10:49:33.879 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:33.883+0000 7fce1ea00700 1 -- 172.21.15.153:0/2127271787 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fce1400f460 con 0x7fce28107d80 2024-03-21T10:49:33.880 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:33.883+0000 7fce2d000700 1 -- 172.21.15.153:0/2127271787 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce281a0830 con 0x7fce28107d80 2024-03-21T10:49:33.880 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:33.883+0000 7fce1ea00700 1 -- 172.21.15.153:0/2127271787 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fce14017630 con 0x7fce28107d80 2024-03-21T10:49:33.880 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:33.883+0000 7fce2d000700 1 -- 172.21.15.153:0/2127271787 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce281a0d20 con 0x7fce28107d80 2024-03-21T10:49:33.880 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:33.884+0000 7fce2d000700 1 -- 172.21.15.153:0/2127271787 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce2810f660 con 0x7fce28107d80 2024-03-21T10:49:33.880 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:33.885+0000 7fce1ea00700 1 -- 172.21.15.153:0/2127271787 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fce1400f5d0 con 0x7fce28107d80 2024-03-21T10:49:33.882 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:33.886+0000 7fce1ea00700 1 --2- 172.21.15.153:0/2127271787 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fce0c06e460 0x7fce0c070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:49:33.882 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:33.886+0000 7fce1ea00700 1 -- 172.21.15.153:0/2127271787 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fce1408fb90 con 0x7fce28107d80 2024-03-21T10:49:33.882 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:33.886+0000 7fce25e00700 1 --2- 172.21.15.153:0/2127271787 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fce0c06e460 0x7fce0c070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:49:33.882 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:33.887+0000 7fce25e00700 1 --2- 172.21.15.153:0/2127271787 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fce0c06e460 0x7fce0c070920 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fce18005fd0 tx=0x7fce1800a560 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:49:33.887 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:33.891+0000 7fce1ea00700 1 -- 172.21.15.153:0/2127271787 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fce14058bb0 con 0x7fce28107d80 2024-03-21T10:49:34.128 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:34.132+0000 7fce2d000700 1 -- 172.21.15.153:0/2127271787 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fce281a1000 con 0x7fce0c06e460 2024-03-21T10:49:34.130 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:34.134+0000 7fce1ea00700 1 -- 172.21.15.153:0/2127271787 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fce281a1000 con 0x7fce0c06e460 2024-03-21T10:49:34.133 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:34.137+0000 7fce2d000700 1 -- 172.21.15.153:0/2127271787 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fce0c06e460 msgr2=0x7fce0c070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:49:34.133 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:34.137+0000 7fce2d000700 1 --2- 172.21.15.153:0/2127271787 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fce0c06e460 0x7fce0c070920 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fce18005fd0 tx=0x7fce1800a560 comp rx=0 tx=0).stop 2024-03-21T10:49:34.134 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:34.137+0000 7fce2d000700 1 -- 172.21.15.153:0/2127271787 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fce28107d80 msgr2=0x7fce2819d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:49:34.134 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:34.137+0000 7fce2d000700 1 --2- 172.21.15.153:0/2127271787 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fce28107d80 0x7fce2819d1f0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fce14005f50 tx=0x7fce1400be20 comp rx=0 tx=0).stop 2024-03-21T10:49:34.134 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:34.138+0000 7fce2d000700 1 -- 172.21.15.153:0/2127271787 shutdown_connections 2024-03-21T10:49:34.134 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:34.138+0000 7fce2d000700 1 --2- 172.21.15.153:0/2127271787 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fce281073d0 0x7fce2819ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:49:34.134 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:34.138+0000 7fce2d000700 1 --2- 172.21.15.153:0/2127271787 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fce0c06e460 0x7fce0c070920 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:49:34.134 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:34.138+0000 7fce2d000700 1 --2- 172.21.15.153:0/2127271787 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fce28107d80 0x7fce2819d1f0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:49:34.134 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:34.138+0000 7fce2d000700 1 -- 172.21.15.153:0/2127271787 >> 172.21.15.153:0/2127271787 conn(0x7fce28076970 msgr2=0x7fce280780e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:49:34.135 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:34.138+0000 7fce2d000700 1 -- 172.21.15.153:0/2127271787 shutdown_connections 2024-03-21T10:49:34.135 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:34.138+0000 7fce2d000700 1 -- 172.21.15.153:0/2127271787 wait complete. 2024-03-21T10:49:34.149 INFO:teuthology.orchestra.run.smithi153.stdout:true 2024-03-21T10:49:34.287 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:34.290+0000 7f6427e00700 1 -- 172.21.15.153:0/446223724 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6428107bd0 msgr2=0x7f642810bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:49:34.287 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:34.290+0000 7f6427e00700 1 --2- 172.21.15.153:0/446223724 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6428107bd0 0x7f642810bc10 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f6414009910 tx=0x7f6414009c20 comp rx=0 tx=0).stop 2024-03-21T10:49:34.287 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:34.291+0000 7f6427e00700 1 -- 172.21.15.153:0/446223724 shutdown_connections 2024-03-21T10:49:34.287 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:34.291+0000 7f6427e00700 1 --2- 172.21.15.153:0/446223724 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6428107bd0 0x7f642810bc10 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:49:34.287 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:34.291+0000 7f6427e00700 1 --2- 172.21.15.153:0/446223724 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6428107220 0x7f6428107600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:49:34.287 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:34.291+0000 7f6427e00700 1 -- 172.21.15.153:0/446223724 >> 172.21.15.153:0/446223724 conn(0x7f64280767e0 msgr2=0x7f6428078c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:49:34.287 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:34.291+0000 7f6427e00700 1 -- 172.21.15.153:0/446223724 shutdown_connections 2024-03-21T10:49:34.287 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:34.291+0000 7f6427e00700 1 -- 172.21.15.153:0/446223724 wait complete. 2024-03-21T10:49:34.289 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:34.292+0000 7f6427e00700 1 Processor -- start 2024-03-21T10:49:34.289 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:34.292+0000 7f6427e00700 1 -- start start 2024-03-21T10:49:34.289 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:34.292+0000 7f6427e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6428107220 0x7f642819cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:49:34.289 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:34.292+0000 7f6427e00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6428107bd0 0x7f642819d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:49:34.289 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:34.292+0000 7f6427e00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f642819d9c0 con 0x7f6428107220 2024-03-21T10:49:34.289 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:34.292+0000 7f6427e00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64281a1750 con 0x7f6428107bd0 2024-03-21T10:49:34.289 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:34.293+0000 7f6426a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6428107220 0x7f642819cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:49:34.289 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:34.293+0000 7f6426a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6428107220 0x7f642819cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:49922/0 (socket says 172.21.15.153:49922) 2024-03-21T10:49:34.289 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:34.293+0000 7f6426a00700 1 -- 172.21.15.153:0/355651373 learned_addr learned my addr 172.21.15.153:0/355651373 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:49:34.289 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:34.293+0000 7f6426a00700 1 -- 172.21.15.153:0/355651373 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6428107bd0 msgr2=0x7f642819d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:49:34.289 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:34.293+0000 7f6426a00700 1 --2- 172.21.15.153:0/355651373 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6428107bd0 0x7f642819d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:49:34.289 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:34.293+0000 7f6426a00700 1 -- 172.21.15.153:0/355651373 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64140095f0 con 0x7f6428107220 2024-03-21T10:49:34.290 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:34.293+0000 7f6426a00700 1 --2- 172.21.15.153:0/355651373 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6428107220 0x7f642819cda0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f641800b4b0 tx=0x7f641800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:49:34.290 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:34.293+0000 7f642ca00700 1 -- 172.21.15.153:0/355651373 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6418011800 con 0x7f6428107220 2024-03-21T10:49:34.290 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:34.294+0000 7f6427e00700 1 -- 172.21.15.153:0/355651373 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64281a1a30 con 0x7f6428107220 2024-03-21T10:49:34.290 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:34.294+0000 7f642ca00700 1 -- 172.21.15.153:0/355651373 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6418011e20 con 0x7f6428107220 2024-03-21T10:49:34.290 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:34.294+0000 7f642ca00700 1 -- 172.21.15.153:0/355651373 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f641800f2e0 con 0x7f6428107220 2024-03-21T10:49:34.291 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:34.294+0000 7f6427e00700 1 -- 172.21.15.153:0/355651373 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64281a1f80 con 0x7f6428107220 2024-03-21T10:49:34.291 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:34.295+0000 7f6427e00700 1 -- 172.21.15.153:0/355651373 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f642810f540 con 0x7f6428107220 2024-03-21T10:49:34.291 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:34.296+0000 7f642ca00700 1 -- 172.21.15.153:0/355651373 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f6418010380 con 0x7f6428107220 2024-03-21T10:49:34.292 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:34.296+0000 7f642ca00700 1 --2- 172.21.15.153:0/355651373 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f640c06e460 0x7f640c070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:49:34.292 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:34.297+0000 7f642ca00700 1 -- 172.21.15.153:0/355651373 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f641808e190 con 0x7f6428107220 2024-03-21T10:49:34.292 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:34.297+0000 7f6426000700 1 --2- 172.21.15.153:0/355651373 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f640c06e460 0x7f640c070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:49:34.293 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:34.297+0000 7f6426000700 1 --2- 172.21.15.153:0/355651373 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f640c06e460 0x7f640c070920 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f641400b5c0 tx=0x7f6414005fd0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:49:34.297 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:34.301+0000 7f642ca00700 1 -- 172.21.15.153:0/355651373 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f64180571b0 con 0x7f6428107220 2024-03-21T10:49:34.535 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:34.538+0000 7f6427e00700 1 -- 172.21.15.153:0/355651373 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f64281a22c0 con 0x7f640c06e460 2024-03-21T10:49:34.536 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:34.540+0000 7f642ca00700 1 -- 172.21.15.153:0/355651373 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f64281a22c0 con 0x7f640c06e460 2024-03-21T10:49:34.539 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:34.543+0000 7f6427e00700 1 -- 172.21.15.153:0/355651373 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f640c06e460 msgr2=0x7f640c070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:49:34.539 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:34.543+0000 7f6427e00700 1 --2- 172.21.15.153:0/355651373 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f640c06e460 0x7f640c070920 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f641400b5c0 tx=0x7f6414005fd0 comp rx=0 tx=0).stop 2024-03-21T10:49:34.539 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:34.543+0000 7f6427e00700 1 -- 172.21.15.153:0/355651373 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6428107220 msgr2=0x7f642819cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:49:34.539 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:34.543+0000 7f6427e00700 1 --2- 172.21.15.153:0/355651373 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6428107220 0x7f642819cda0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f641800b4b0 tx=0x7f641800b870 comp rx=0 tx=0).stop 2024-03-21T10:49:34.539 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:34.543+0000 7f6427e00700 1 -- 172.21.15.153:0/355651373 shutdown_connections 2024-03-21T10:49:34.540 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:34.543+0000 7f6427e00700 1 --2- 172.21.15.153:0/355651373 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f640c06e460 0x7f640c070920 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:49:34.540 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:34.543+0000 7f6427e00700 1 --2- 172.21.15.153:0/355651373 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6428107220 0x7f642819cda0 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:49:34.540 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:34.543+0000 7f6427e00700 1 --2- 172.21.15.153:0/355651373 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6428107bd0 0x7f642819d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:49:34.540 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:34.543+0000 7f6427e00700 1 -- 172.21.15.153:0/355651373 >> 172.21.15.153:0/355651373 conn(0x7f64280767e0 msgr2=0x7f6428077f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:49:34.540 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:34.544+0000 7f6427e00700 1 -- 172.21.15.153:0/355651373 shutdown_connections 2024-03-21T10:49:34.540 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:34.544+0000 7f6427e00700 1 -- 172.21.15.153:0/355651373 wait complete. 2024-03-21T10:49:34.695 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:34.699+0000 7f61c3800700 1 -- 172.21.15.153:0/1837783470 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f61bc1071d0 msgr2=0x7f61bc107650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:49:34.696 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:34.699+0000 7f61c3800700 1 --2- 172.21.15.153:0/1837783470 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f61bc1071d0 0x7f61bc107650 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f61a8009910 tx=0x7f61a8009c20 comp rx=0 tx=0).stop 2024-03-21T10:49:34.696 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:34.699+0000 7f61c3800700 1 -- 172.21.15.153:0/1837783470 shutdown_connections 2024-03-21T10:49:34.696 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:34.699+0000 7f61c3800700 1 --2- 172.21.15.153:0/1837783470 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f61bc1071d0 0x7f61bc107650 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:49:34.696 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:34.699+0000 7f61c3800700 1 --2- 172.21.15.153:0/1837783470 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f61bc10d280 0x7f61bc10d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:49:34.696 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:34.699+0000 7f61c3800700 1 -- 172.21.15.153:0/1837783470 >> 172.21.15.153:0/1837783470 conn(0x7f61bc0769e0 msgr2=0x7f61bc078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:49:34.696 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:34.699+0000 7f61c3800700 1 -- 172.21.15.153:0/1837783470 shutdown_connections 2024-03-21T10:49:34.696 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:34.699+0000 7f61c3800700 1 -- 172.21.15.153:0/1837783470 wait complete. 2024-03-21T10:49:34.697 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:34.700+0000 7f61c3800700 1 Processor -- start 2024-03-21T10:49:34.698 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:34.701+0000 7f61c3800700 1 -- start start 2024-03-21T10:49:34.698 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:34.701+0000 7f61c3800700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f61bc1071d0 0x7f61bc19ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:49:34.698 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:34.701+0000 7f61c3800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f61bc10d280 0x7f61bc19d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:49:34.698 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:34.701+0000 7f61c3800700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61bc19d880 con 0x7f61bc10d280 2024-03-21T10:49:34.698 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:34.701+0000 7f61c3800700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61bc1a05b0 con 0x7f61bc1071d0 2024-03-21T10:49:34.698 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:34.701+0000 7f61bbe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f61bc10d280 0x7f61bc19d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:49:34.698 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:34.701+0000 7f61bbe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f61bc10d280 0x7f61bc19d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:49936/0 (socket says 172.21.15.153:49936) 2024-03-21T10:49:34.698 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:34.701+0000 7f61bbe00700 1 -- 172.21.15.153:0/2996988825 learned_addr learned my addr 172.21.15.153:0/2996988825 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:49:34.698 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:34.702+0000 7f61c0e00700 1 --2- 172.21.15.153:0/2996988825 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f61bc1071d0 0x7f61bc19ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:49:34.698 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:34.702+0000 7f61bbe00700 1 -- 172.21.15.153:0/2996988825 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f61bc1071d0 msgr2=0x7f61bc19ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:49:34.698 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:34.702+0000 7f61bbe00700 1 --2- 172.21.15.153:0/2996988825 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f61bc1071d0 0x7f61bc19ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:49:34.698 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:34.702+0000 7f61bbe00700 1 -- 172.21.15.153:0/2996988825 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61a80095f0 con 0x7f61bc10d280 2024-03-21T10:49:34.699 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:34.702+0000 7f61bbe00700 1 --2- 172.21.15.153:0/2996988825 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f61bc10d280 0x7f61bc19d1f0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f61a800b5c0 tx=0x7f61a800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:49:34.699 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:34.702+0000 7f61b9600700 1 -- 172.21.15.153:0/2996988825 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f61a801d070 con 0x7f61bc10d280 2024-03-21T10:49:34.699 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:34.702+0000 7f61b9600700 1 -- 172.21.15.153:0/2996988825 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f61a8005e00 con 0x7f61bc10d280 2024-03-21T10:49:34.699 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:34.702+0000 7f61c3800700 1 -- 172.21.15.153:0/2996988825 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61bc1a0830 con 0x7f61bc10d280 2024-03-21T10:49:34.699 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:34.702+0000 7f61b9600700 1 -- 172.21.15.153:0/2996988825 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f61a8017400 con 0x7f61bc10d280 2024-03-21T10:49:34.699 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:34.703+0000 7f61c3800700 1 -- 172.21.15.153:0/2996988825 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61bc1a0da0 con 0x7f61bc10d280 2024-03-21T10:49:34.700 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:34.704+0000 7f61c3800700 1 -- 172.21.15.153:0/2996988825 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61bc10f6c0 con 0x7f61bc10d280 2024-03-21T10:49:34.700 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:34.705+0000 7f61b9600700 1 -- 172.21.15.153:0/2996988825 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f61a800f460 con 0x7f61bc10d280 2024-03-21T10:49:34.702 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:34.706+0000 7f61b9600700 1 --2- 172.21.15.153:0/2996988825 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f61a406e460 0x7f61a4070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:49:34.702 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:34.706+0000 7f61b9600700 1 -- 172.21.15.153:0/2996988825 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f61a808fa90 con 0x7f61bc10d280 2024-03-21T10:49:34.702 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:34.706+0000 7f61c0e00700 1 --2- 172.21.15.153:0/2996988825 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f61a406e460 0x7f61a4070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:49:34.702 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:34.707+0000 7f61c0e00700 1 --2- 172.21.15.153:0/2996988825 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f61a406e460 0x7f61a4070920 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f61b0005950 tx=0x7f61b00058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:49:34.706 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:34.711+0000 7f61b9600700 1 -- 172.21.15.153:0/2996988825 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f61a8058ab0 con 0x7f61bc10d280 2024-03-21T10:49:34.923 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:34.927+0000 7f61c3800700 1 -- 172.21.15.153:0/2996988825 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f61bc1a1080 con 0x7f61a406e460 2024-03-21T10:49:34.938 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:34.943+0000 7f61b9600700 1 -- 172.21.15.153:0/2996988825 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f61bc1a1080 con 0x7f61a406e460 2024-03-21T10:49:34.939 INFO:teuthology.orchestra.run.smithi153.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-21T10:49:34.939 INFO:teuthology.orchestra.run.smithi153.stdout:alertmanager.smithi153 smithi153 *:9093,9094 running (38m) 14m ago 52m 15.6M - 0.25.0 c8568f914cd2 9ed3f2200b5f 2024-03-21T10:49:34.939 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi153 smithi153 running (52m) 14m ago 52m 10.4M - 18.2.1 5be31c24972a 5b61994396a6 2024-03-21T10:49:34.939 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi174 smithi174 running (50m) 14m ago 50m 10.1M - 18.2.1 5be31c24972a 4f7a5fc3ac28 2024-03-21T10:49:34.939 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi153 smithi153 running (34m) 14m ago 52m 6656k - 19.0.0-2231-g2edfee68 237e43c7a65f 32c21104d5b0 2024-03-21T10:49:34.939 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi174 smithi174 running (34m) 14m ago 50m 6660k - 19.0.0-2231-g2edfee68 237e43c7a65f a04e48967ab6 2024-03-21T10:49:34.940 INFO:teuthology.orchestra.run.smithi153.stdout:grafana.smithi153 smithi153 *:3000 running (37m) 14m ago 51m 75.2M - 9.4.12 2bacad6d85d8 ee24026ab786 2024-03-21T10:49:34.940 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.iuyluj smithi153 running (29m) 14m ago 44m 18.7M - 19.0.0-2231-g2edfee68 237e43c7a65f 87aff8a5de70 2024-03-21T10:49:34.940 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.zipckr smithi153 running (30m) 14m ago 45m 18.9M - 19.0.0-2231-g2edfee68 237e43c7a65f ccc262bd5dd3 2024-03-21T10:49:34.940 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.nhozag smithi174 running (14m) 14m ago 44m 10.9M - 19.0.0-2231-g2edfee68 237e43c7a65f 9e02aea72c96 2024-03-21T10:49:34.940 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.yyimst smithi174 running (29m) 14m ago 44m 18.5M - 19.0.0-2231-g2edfee68 237e43c7a65f fc1962f9351e 2024-03-21T10:49:34.940 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi153.tluxni smithi153 *:8443,9283,8765 running (41m) 14m ago 53m 512M - 19.0.0-2231-g2edfee68 237e43c7a65f 7ffee0e55382 2024-03-21T10:49:34.940 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi174.qpcwat smithi174 *:8443,9283,8765 running (40m) 14m ago 50m 459M - 19.0.0-2231-g2edfee68 237e43c7a65f 748968f216ab 2024-03-21T10:49:34.940 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi153 smithi153 running (36m) 14m ago 54m 70.8M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f dd8038303dc3 2024-03-21T10:49:34.940 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi174 smithi174 running (35m) 14m ago 50m 51.9M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f 29d78a788f2c 2024-03-21T10:49:34.940 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi153 smithi153 *:9100 running (39m) 14m ago 52m 8140k - 1.5.0 0da6a335fe13 bb835edc0e0b 2024-03-21T10:49:34.940 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi174 smithi174 *:9100 running (39m) 14m ago 50m 8610k - 1.5.0 0da6a335fe13 7c4ecad790fa 2024-03-21T10:49:34.940 INFO:teuthology.orchestra.run.smithi153.stdout:osd.0 smithi153 running (33m) 14m ago 49m 118M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 507900fd4791 2024-03-21T10:49:34.940 INFO:teuthology.orchestra.run.smithi153.stdout:osd.1 smithi153 running (33m) 14m ago 48m 93.0M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 0d646452e9c1 2024-03-21T10:49:34.940 INFO:teuthology.orchestra.run.smithi153.stdout:osd.2 smithi153 running (32m) 14m ago 48m 91.2M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 1326f2afdfcd 2024-03-21T10:49:34.941 INFO:teuthology.orchestra.run.smithi153.stdout:osd.3 smithi174 running (32m) 14m ago 47m 87.5M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 70aafac71ad5 2024-03-21T10:49:34.941 INFO:teuthology.orchestra.run.smithi153.stdout:osd.4 smithi174 running (31m) 14m ago 47m 95.8M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 0d746ce6e79d 2024-03-21T10:49:34.941 INFO:teuthology.orchestra.run.smithi153.stdout:osd.5 smithi174 running (30m) 14m ago 46m 90.3M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 53e5f35b8cac 2024-03-21T10:49:34.941 INFO:teuthology.orchestra.run.smithi153.stdout:prometheus.smithi153 smithi153 *:9095 running (38m) 14m ago 51m 88.1M - 2.43.0 a07b618ecd1d 7d95561e50c2 2024-03-21T10:49:34.942 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:34.946+0000 7f61c3800700 1 -- 172.21.15.153:0/2996988825 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f61a406e460 msgr2=0x7f61a4070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:49:34.942 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:34.946+0000 7f61c3800700 1 --2- 172.21.15.153:0/2996988825 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f61a406e460 0x7f61a4070920 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f61b0005950 tx=0x7f61b00058e0 comp rx=0 tx=0).stop 2024-03-21T10:49:34.942 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:34.946+0000 7f61c3800700 1 -- 172.21.15.153:0/2996988825 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f61bc10d280 msgr2=0x7f61bc19d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:49:34.942 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:34.946+0000 7f61c3800700 1 --2- 172.21.15.153:0/2996988825 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f61bc10d280 0x7f61bc19d1f0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f61a800b5c0 tx=0x7f61a800be20 comp rx=0 tx=0).stop 2024-03-21T10:49:34.942 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:34.946+0000 7f61c3800700 1 -- 172.21.15.153:0/2996988825 shutdown_connections 2024-03-21T10:49:34.942 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:34.946+0000 7f61c3800700 1 --2- 172.21.15.153:0/2996988825 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f61bc1071d0 0x7f61bc19ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:49:34.942 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:34.946+0000 7f61c3800700 1 --2- 172.21.15.153:0/2996988825 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f61a406e460 0x7f61a4070920 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:49:34.942 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:34.946+0000 7f61c3800700 1 --2- 172.21.15.153:0/2996988825 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f61bc10d280 0x7f61bc19d1f0 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:49:34.942 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:34.946+0000 7f61c3800700 1 -- 172.21.15.153:0/2996988825 >> 172.21.15.153:0/2996988825 conn(0x7f61bc0769e0 msgr2=0x7f61bc078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:49:34.943 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:34.946+0000 7f61c3800700 1 -- 172.21.15.153:0/2996988825 shutdown_connections 2024-03-21T10:49:34.943 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:34.946+0000 7f61c3800700 1 -- 172.21.15.153:0/2996988825 wait complete. 2024-03-21T10:49:35.100 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.104+0000 7f6d82a00700 1 -- 172.21.15.153:0/2448263600 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6d7c1039b0 msgr2=0x7f6d7c107a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:49:35.100 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.104+0000 7f6d82a00700 1 --2- 172.21.15.153:0/2448263600 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6d7c1039b0 0x7f6d7c107a00 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f6d68009910 tx=0x7f6d68009c20 comp rx=0 tx=0).stop 2024-03-21T10:49:35.100 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.104+0000 7f6d82a00700 1 -- 172.21.15.153:0/2448263600 shutdown_connections 2024-03-21T10:49:35.100 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.104+0000 7f6d82a00700 1 --2- 172.21.15.153:0/2448263600 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6d7c1039b0 0x7f6d7c107a00 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:49:35.101 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.104+0000 7f6d82a00700 1 --2- 172.21.15.153:0/2448263600 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6d7c103000 0x7f6d7c1033e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:49:35.101 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.104+0000 7f6d82a00700 1 -- 172.21.15.153:0/2448263600 >> 172.21.15.153:0/2448263600 conn(0x7f6d7c0fe8d0 msgr2=0x7f6d7c100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:49:35.101 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.104+0000 7f6d82a00700 1 -- 172.21.15.153:0/2448263600 shutdown_connections 2024-03-21T10:49:35.101 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.104+0000 7f6d82a00700 1 -- 172.21.15.153:0/2448263600 wait complete. 2024-03-21T10:49:35.102 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.105+0000 7f6d82a00700 1 Processor -- start 2024-03-21T10:49:35.102 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.105+0000 7f6d82a00700 1 -- start start 2024-03-21T10:49:35.102 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.106+0000 7f6d82a00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6d7c103000 0x7f6d7c19e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:49:35.103 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.106+0000 7f6d82a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6d7c1039b0 0x7f6d7c19ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:49:35.103 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.106+0000 7f6d82a00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d7c19f390 con 0x7f6d7c1039b0 2024-03-21T10:49:35.103 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.106+0000 7f6d82a00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d7c198840 con 0x7f6d7c103000 2024-03-21T10:49:35.103 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.106+0000 7f6d7b400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6d7c1039b0 0x7f6d7c19ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:49:35.103 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.106+0000 7f6d7b400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6d7c1039b0 0x7f6d7c19ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:49966/0 (socket says 172.21.15.153:49966) 2024-03-21T10:49:35.103 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.106+0000 7f6d7b400700 1 -- 172.21.15.153:0/1686121077 learned_addr learned my addr 172.21.15.153:0/1686121077 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:49:35.103 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.106+0000 7f6d7b400700 1 -- 172.21.15.153:0/1686121077 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6d7c103000 msgr2=0x7f6d7c19e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:49:35.103 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.106+0000 7f6d7b400700 1 --2- 172.21.15.153:0/1686121077 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6d7c103000 0x7f6d7c19e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:49:35.103 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.106+0000 7f6d7b400700 1 -- 172.21.15.153:0/1686121077 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d680095f0 con 0x7f6d7c1039b0 2024-03-21T10:49:35.103 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.107+0000 7f6d7b400700 1 --2- 172.21.15.153:0/1686121077 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6d7c1039b0 0x7f6d7c19ed00 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f6d68005f50 tx=0x7f6d6800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:49:35.104 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.107+0000 7f6d78c00700 1 -- 172.21.15.153:0/1686121077 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6d6801d070 con 0x7f6d7c1039b0 2024-03-21T10:49:35.104 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.107+0000 7f6d78c00700 1 -- 172.21.15.153:0/1686121077 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6d6800f460 con 0x7f6d7c1039b0 2024-03-21T10:49:35.104 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.107+0000 7f6d78c00700 1 -- 172.21.15.153:0/1686121077 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6d68017630 con 0x7f6d7c1039b0 2024-03-21T10:49:35.104 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.107+0000 7f6d82a00700 1 -- 172.21.15.153:0/1686121077 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d7c198ac0 con 0x7f6d7c1039b0 2024-03-21T10:49:35.104 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.107+0000 7f6d82a00700 1 -- 172.21.15.153:0/1686121077 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d7c198fb0 con 0x7f6d7c1039b0 2024-03-21T10:49:35.104 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.108+0000 7f6d82a00700 1 -- 172.21.15.153:0/1686121077 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d7c10b400 con 0x7f6d7c1039b0 2024-03-21T10:49:35.105 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.109+0000 7f6d78c00700 1 -- 172.21.15.153:0/1686121077 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f6d6800f5d0 con 0x7f6d7c1039b0 2024-03-21T10:49:35.106 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.110+0000 7f6d78c00700 1 --2- 172.21.15.153:0/1686121077 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f6d6006e460 0x7f6d60070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:49:35.106 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.110+0000 7f6d78c00700 1 -- 172.21.15.153:0/1686121077 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f6d6808faa0 con 0x7f6d7c1039b0 2024-03-21T10:49:35.106 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.110+0000 7f6d7be00700 1 --2- 172.21.15.153:0/1686121077 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f6d6006e460 0x7f6d60070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:49:35.106 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.111+0000 7f6d7be00700 1 --2- 172.21.15.153:0/1686121077 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f6d6006e460 0x7f6d60070920 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c00b420 tx=0x7f6d6c008540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:49:35.110 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.115+0000 7f6d78c00700 1 -- 172.21.15.153:0/1686121077 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f6d6805c670 con 0x7f6d7c1039b0 2024-03-21T10:49:35.414 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.418+0000 7f6d82a00700 1 -- 172.21.15.153:0/1686121077 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f6d7c10ade0 con 0x7f6d7c1039b0 2024-03-21T10:49:35.414 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.419+0000 7f6d78c00700 1 -- 172.21.15.153:0/1686121077 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f6d6805c200 con 0x7f6d7c1039b0 2024-03-21T10:49:35.415 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:49:35.415 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": { 2024-03-21T10:49:35.415 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:49:35.415 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:49:35.415 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": { 2024-03-21T10:49:35.415 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:49:35.415 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:49:35.415 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": { 2024-03-21T10:49:35.415 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 6 2024-03-21T10:49:35.415 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:49:35.415 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": { 2024-03-21T10:49:35.415 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 4 2024-03-21T10:49:35.415 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:49:35.415 INFO:teuthology.orchestra.run.smithi153.stdout: "overall": { 2024-03-21T10:49:35.416 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 14 2024-03-21T10:49:35.416 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-03-21T10:49:35.416 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:49:35.417 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.421+0000 7f6d82a00700 1 -- 172.21.15.153:0/1686121077 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f6d6006e460 msgr2=0x7f6d60070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:49:35.417 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.421+0000 7f6d82a00700 1 --2- 172.21.15.153:0/1686121077 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f6d6006e460 0x7f6d60070920 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c00b420 tx=0x7f6d6c008540 comp rx=0 tx=0).stop 2024-03-21T10:49:35.417 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.421+0000 7f6d82a00700 1 -- 172.21.15.153:0/1686121077 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6d7c1039b0 msgr2=0x7f6d7c19ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:49:35.417 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.421+0000 7f6d82a00700 1 --2- 172.21.15.153:0/1686121077 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6d7c1039b0 0x7f6d7c19ed00 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f6d68005f50 tx=0x7f6d6800be20 comp rx=0 tx=0).stop 2024-03-21T10:49:35.417 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.421+0000 7f6d82a00700 1 -- 172.21.15.153:0/1686121077 shutdown_connections 2024-03-21T10:49:35.417 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.422+0000 7f6d82a00700 1 --2- 172.21.15.153:0/1686121077 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6d7c103000 0x7f6d7c19e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:49:35.417 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.422+0000 7f6d82a00700 1 --2- 172.21.15.153:0/1686121077 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f6d6006e460 0x7f6d60070920 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:49:35.418 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.422+0000 7f6d82a00700 1 --2- 172.21.15.153:0/1686121077 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6d7c1039b0 0x7f6d7c19ed00 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:49:35.418 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.422+0000 7f6d82a00700 1 -- 172.21.15.153:0/1686121077 >> 172.21.15.153:0/1686121077 conn(0x7f6d7c0fe8d0 msgr2=0x7f6d7c100cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:49:35.418 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.422+0000 7f6d82a00700 1 -- 172.21.15.153:0/1686121077 shutdown_connections 2024-03-21T10:49:35.418 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.422+0000 7f6d82a00700 1 -- 172.21.15.153:0/1686121077 wait complete. 2024-03-21T10:49:35.551 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:49:35 smithi153 ceph-mon[113733]: pgmap v1133: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:49:35.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:49:35 smithi153 ceph-mon[113733]: from='client.35994 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:49:35.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:49:35 smithi153 ceph-mon[113733]: from='client.35998 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:49:35.565 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.569+0000 7fe346a00700 1 -- 172.21.15.153:0/164552075 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe340102ff0 msgr2=0x7fe3401033d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:49:35.566 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.569+0000 7fe346a00700 1 --2- 172.21.15.153:0/164552075 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe340102ff0 0x7fe3401033d0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fe330009910 tx=0x7fe330009c20 comp rx=0 tx=0).stop 2024-03-21T10:49:35.566 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.570+0000 7fe346a00700 1 -- 172.21.15.153:0/164552075 shutdown_connections 2024-03-21T10:49:35.566 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.570+0000 7fe346a00700 1 --2- 172.21.15.153:0/164552075 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe3401039a0 0x7fe3401079f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:49:35.566 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.570+0000 7fe346a00700 1 --2- 172.21.15.153:0/164552075 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe340102ff0 0x7fe3401033d0 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:49:35.566 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.570+0000 7fe346a00700 1 -- 172.21.15.153:0/164552075 >> 172.21.15.153:0/164552075 conn(0x7fe3400fe880 msgr2=0x7fe340100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:49:35.566 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.570+0000 7fe346a00700 1 -- 172.21.15.153:0/164552075 shutdown_connections 2024-03-21T10:49:35.566 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.570+0000 7fe346a00700 1 -- 172.21.15.153:0/164552075 wait complete. 2024-03-21T10:49:35.568 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.571+0000 7fe346a00700 1 Processor -- start 2024-03-21T10:49:35.568 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.571+0000 7fe346a00700 1 -- start start 2024-03-21T10:49:35.568 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.571+0000 7fe346a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe340102ff0 0x7fe34019e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:49:35.568 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.571+0000 7fe346a00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe3401039a0 0x7fe34019ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:49:35.568 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.571+0000 7fe346a00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe34019f3f0 con 0x7fe340102ff0 2024-03-21T10:49:35.568 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.571+0000 7fe346a00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3401988a0 con 0x7fe3401039a0 2024-03-21T10:49:35.568 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.572+0000 7fe33fe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe340102ff0 0x7fe34019e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:49:35.568 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.572+0000 7fe33fe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe340102ff0 0x7fe34019e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:49986/0 (socket says 172.21.15.153:49986) 2024-03-21T10:49:35.568 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.572+0000 7fe33fe00700 1 -- 172.21.15.153:0/4064394030 learned_addr learned my addr 172.21.15.153:0/4064394030 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:49:35.568 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.572+0000 7fe33f400700 1 --2- 172.21.15.153:0/4064394030 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe3401039a0 0x7fe34019ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:49:35.568 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.572+0000 7fe33fe00700 1 -- 172.21.15.153:0/4064394030 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe3401039a0 msgr2=0x7fe34019ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:49:35.568 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.572+0000 7fe33fe00700 1 --2- 172.21.15.153:0/4064394030 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe3401039a0 0x7fe34019ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:49:35.569 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.572+0000 7fe33fe00700 1 -- 172.21.15.153:0/4064394030 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3300095f0 con 0x7fe340102ff0 2024-03-21T10:49:35.569 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.572+0000 7fe33fe00700 1 --2- 172.21.15.153:0/4064394030 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe340102ff0 0x7fe34019e820 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fe33000b5c0 tx=0x7fe33000be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:49:35.569 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.573+0000 7fe33cc00700 1 -- 172.21.15.153:0/4064394030 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe33001d070 con 0x7fe340102ff0 2024-03-21T10:49:35.569 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.573+0000 7fe33cc00700 1 -- 172.21.15.153:0/4064394030 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe33000f460 con 0x7fe340102ff0 2024-03-21T10:49:35.569 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.573+0000 7fe346a00700 1 -- 172.21.15.153:0/4064394030 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe340198b20 con 0x7fe340102ff0 2024-03-21T10:49:35.570 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.573+0000 7fe33cc00700 1 -- 172.21.15.153:0/4064394030 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe330017610 con 0x7fe340102ff0 2024-03-21T10:49:35.570 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.573+0000 7fe346a00700 1 -- 172.21.15.153:0/4064394030 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe340199010 con 0x7fe340102ff0 2024-03-21T10:49:35.570 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.574+0000 7fe346a00700 1 -- 172.21.15.153:0/4064394030 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe34010b390 con 0x7fe340102ff0 2024-03-21T10:49:35.571 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.575+0000 7fe33cc00700 1 -- 172.21.15.153:0/4064394030 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fe33000f5d0 con 0x7fe340102ff0 2024-03-21T10:49:35.572 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.576+0000 7fe33cc00700 1 --2- 172.21.15.153:0/4064394030 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe32406e460 0x7fe324070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:49:35.572 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.576+0000 7fe33cc00700 1 -- 172.21.15.153:0/4064394030 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fe33008fcc0 con 0x7fe340102ff0 2024-03-21T10:49:35.572 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.577+0000 7fe33f400700 1 --2- 172.21.15.153:0/4064394030 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe32406e460 0x7fe324070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:49:35.573 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.577+0000 7fe33f400700 1 --2- 172.21.15.153:0/4064394030 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe32406e460 0x7fe324070920 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fe32c006fd0 tx=0x7fe32c008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:49:35.576 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.580+0000 7fe33cc00700 1 -- 172.21.15.153:0/4064394030 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe33005c910 con 0x7fe340102ff0 2024-03-21T10:49:35.696 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:49:35 smithi174 ceph-mon[88004]: pgmap v1133: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:49:35.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:49:35 smithi174 ceph-mon[88004]: from='client.35994 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:49:35.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:49:35 smithi174 ceph-mon[88004]: from='client.35998 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:49:35.844 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.848+0000 7fe346a00700 1 -- 172.21.15.153:0/4064394030 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fe34004ea90 con 0x7fe340102ff0 2024-03-21T10:49:35.844 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.849+0000 7fe33cc00700 1 -- 172.21.15.153:0/4064394030 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1876 (secure 0 0 0) 0x7fe33005c4a0 con 0x7fe340102ff0 2024-03-21T10:49:35.846 INFO:teuthology.orchestra.run.smithi153.stdout:e32 2024-03-21T10:49:35.846 INFO:teuthology.orchestra.run.smithi153.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-21T10:49:35.846 INFO:teuthology.orchestra.run.smithi153.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:49:35.846 INFO:teuthology.orchestra.run.smithi153.stdout:legacy client fscid: 1 2024-03-21T10:49:35.846 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:49:35.846 INFO:teuthology.orchestra.run.smithi153.stdout:Filesystem 'cephfs' (1) 2024-03-21T10:49:35.847 INFO:teuthology.orchestra.run.smithi153.stdout:fs_name cephfs 2024-03-21T10:49:35.847 INFO:teuthology.orchestra.run.smithi153.stdout:epoch 23 2024-03-21T10:49:35.847 INFO:teuthology.orchestra.run.smithi153.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-21T10:49:35.847 INFO:teuthology.orchestra.run.smithi153.stdout:created 2024-03-21T10:04:27.086576+0000 2024-03-21T10:49:35.847 INFO:teuthology.orchestra.run.smithi153.stdout:modified 2024-03-21T10:19:05.022092+0000 2024-03-21T10:49:35.847 INFO:teuthology.orchestra.run.smithi153.stdout:tableserver 0 2024-03-21T10:49:35.847 INFO:teuthology.orchestra.run.smithi153.stdout:root 0 2024-03-21T10:49:35.847 INFO:teuthology.orchestra.run.smithi153.stdout:session_timeout 60 2024-03-21T10:49:35.847 INFO:teuthology.orchestra.run.smithi153.stdout:session_autoclose 300 2024-03-21T10:49:35.847 INFO:teuthology.orchestra.run.smithi153.stdout:max_file_size 1099511627776 2024-03-21T10:49:35.847 INFO:teuthology.orchestra.run.smithi153.stdout:max_xattr_size 65536 2024-03-21T10:49:35.848 INFO:teuthology.orchestra.run.smithi153.stdout:required_client_features {} 2024-03-21T10:49:35.848 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure 0 2024-03-21T10:49:35.848 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure_osd_epoch 94 2024-03-21T10:49:35.848 INFO:teuthology.orchestra.run.smithi153.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:49:35.848 INFO:teuthology.orchestra.run.smithi153.stdout:max_mds 2 2024-03-21T10:49:35.848 INFO:teuthology.orchestra.run.smithi153.stdout:in 0,1 2024-03-21T10:49:35.848 INFO:teuthology.orchestra.run.smithi153.stdout:up {} 2024-03-21T10:49:35.848 INFO:teuthology.orchestra.run.smithi153.stdout:failed 0,1 2024-03-21T10:49:35.848 INFO:teuthology.orchestra.run.smithi153.stdout:damaged 2024-03-21T10:49:35.848 INFO:teuthology.orchestra.run.smithi153.stdout:stopped 2024-03-21T10:49:35.848 INFO:teuthology.orchestra.run.smithi153.stdout:data_pools [3] 2024-03-21T10:49:35.848 INFO:teuthology.orchestra.run.smithi153.stdout:metadata_pool 2 2024-03-21T10:49:35.848 INFO:teuthology.orchestra.run.smithi153.stdout:inline_data enabled 2024-03-21T10:49:35.848 INFO:teuthology.orchestra.run.smithi153.stdout:balancer 2024-03-21T10:49:35.849 INFO:teuthology.orchestra.run.smithi153.stdout:bal_rank_mask -1 2024-03-21T10:49:35.849 INFO:teuthology.orchestra.run.smithi153.stdout:standby_count_wanted 1 2024-03-21T10:49:35.849 INFO:teuthology.orchestra.run.smithi153.stdout:qdb_cluster leader: 0 members: 2024-03-21T10:49:35.849 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:49:35.849 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:49:35.849 INFO:teuthology.orchestra.run.smithi153.stdout:Standby daemons: 2024-03-21T10:49:35.849 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:49:35.849 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.zipckr{-1:34442} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6826/1103846295,v1:172.21.15.153:6827/1103846295] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:49:35.849 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.iuyluj{-1:34474} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6828/1738630832,v1:172.21.15.153:6829/1738630832] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:49:35.849 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.yyimst{-1:44313} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.174:6824/4127786797,v1:172.21.15.174:6825/4127786797] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:49:35.849 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.nhozag{-1:44687} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.174:6826/2767517845,v1:172.21.15.174:6827/2767517845] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:49:35.850 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.851+0000 7fe346a00700 1 -- 172.21.15.153:0/4064394030 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe32406e460 msgr2=0x7fe324070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:49:35.850 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.851+0000 7fe346a00700 1 --2- 172.21.15.153:0/4064394030 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe32406e460 0x7fe324070920 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fe32c006fd0 tx=0x7fe32c008040 comp rx=0 tx=0).stop 2024-03-21T10:49:35.850 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.851+0000 7fe346a00700 1 -- 172.21.15.153:0/4064394030 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe340102ff0 msgr2=0x7fe34019e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:49:35.850 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.851+0000 7fe346a00700 1 --2- 172.21.15.153:0/4064394030 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe340102ff0 0x7fe34019e820 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fe33000b5c0 tx=0x7fe33000be00 comp rx=0 tx=0).stop 2024-03-21T10:49:35.850 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.852+0000 7fe346a00700 1 -- 172.21.15.153:0/4064394030 shutdown_connections 2024-03-21T10:49:35.850 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.852+0000 7fe346a00700 1 --2- 172.21.15.153:0/4064394030 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe32406e460 0x7fe324070920 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:49:35.851 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.852+0000 7fe346a00700 1 --2- 172.21.15.153:0/4064394030 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe340102ff0 0x7fe34019e820 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:49:35.851 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.852+0000 7fe346a00700 1 --2- 172.21.15.153:0/4064394030 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe3401039a0 0x7fe34019ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:49:35.851 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.852+0000 7fe346a00700 1 -- 172.21.15.153:0/4064394030 >> 172.21.15.153:0/4064394030 conn(0x7fe3400fe880 msgr2=0x7fe340100000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:49:35.851 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.852+0000 7fe346a00700 1 -- 172.21.15.153:0/4064394030 shutdown_connections 2024-03-21T10:49:35.851 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.852+0000 7fe346a00700 1 -- 172.21.15.153:0/4064394030 wait complete. 2024-03-21T10:49:35.851 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 32 2024-03-21T10:49:35.992 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.996+0000 7f8f4e800700 1 -- 172.21.15.153:0/4184380745 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f481071d0 msgr2=0x7f8f48107650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:49:35.992 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.996+0000 7f8f4e800700 1 --2- 172.21.15.153:0/4184380745 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f481071d0 0x7f8f48107650 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f8f34009910 tx=0x7f8f34009c20 comp rx=0 tx=0).stop 2024-03-21T10:49:35.992 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.996+0000 7f8f4e800700 1 -- 172.21.15.153:0/4184380745 shutdown_connections 2024-03-21T10:49:35.993 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.996+0000 7f8f4e800700 1 --2- 172.21.15.153:0/4184380745 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f481071d0 0x7f8f48107650 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:49:35.993 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.996+0000 7f8f4e800700 1 --2- 172.21.15.153:0/4184380745 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8f4810d280 0x7f8f4810d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:49:35.993 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.996+0000 7f8f4e800700 1 -- 172.21.15.153:0/4184380745 >> 172.21.15.153:0/4184380745 conn(0x7f8f480769e0 msgr2=0x7f8f48078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:49:35.993 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.996+0000 7f8f4e800700 1 -- 172.21.15.153:0/4184380745 shutdown_connections 2024-03-21T10:49:35.993 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.996+0000 7f8f4e800700 1 -- 172.21.15.153:0/4184380745 wait complete. 2024-03-21T10:49:35.994 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.997+0000 7f8f4e800700 1 Processor -- start 2024-03-21T10:49:35.994 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.997+0000 7f8f4e800700 1 -- start start 2024-03-21T10:49:35.994 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.998+0000 7f8f4e800700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8f481071d0 0x7f8f4819ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:49:35.994 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.998+0000 7f8f4e800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f4810d280 0x7f8f4819d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:49:35.995 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.998+0000 7f8f4e800700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4819d880 con 0x7f8f4810d280 2024-03-21T10:49:35.995 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.998+0000 7f8f4e800700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f481a05b0 con 0x7f8f481071d0 2024-03-21T10:49:35.995 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.998+0000 7f8f46a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f4810d280 0x7f8f4819d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:49:35.995 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.998+0000 7f8f46a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f4810d280 0x7f8f4819d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:50014/0 (socket says 172.21.15.153:50014) 2024-03-21T10:49:35.995 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.998+0000 7f8f46a00700 1 -- 172.21.15.153:0/2029703422 learned_addr learned my addr 172.21.15.153:0/2029703422 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:49:35.995 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.998+0000 7f8f47400700 1 --2- 172.21.15.153:0/2029703422 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8f481071d0 0x7f8f4819ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:49:35.995 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.998+0000 7f8f46a00700 1 -- 172.21.15.153:0/2029703422 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8f481071d0 msgr2=0x7f8f4819ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:49:35.995 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.998+0000 7f8f46a00700 1 --2- 172.21.15.153:0/2029703422 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8f481071d0 0x7f8f4819ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:49:35.995 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.998+0000 7f8f46a00700 1 -- 172.21.15.153:0/2029703422 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f340095f0 con 0x7f8f4810d280 2024-03-21T10:49:35.995 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.999+0000 7f8f47400700 1 --2- 172.21.15.153:0/2029703422 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8f481071d0 0x7f8f4819ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:49:35.995 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.999+0000 7f8f46a00700 1 --2- 172.21.15.153:0/2029703422 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f4810d280 0x7f8f4819d1f0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f8f340092e0 tx=0x7f8f3400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:49:35.996 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.999+0000 7f8f3fe00700 1 -- 172.21.15.153:0/2029703422 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8f3401d070 con 0x7f8f4810d280 2024-03-21T10:49:35.996 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.999+0000 7f8f3fe00700 1 -- 172.21.15.153:0/2029703422 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8f34005e00 con 0x7f8f4810d280 2024-03-21T10:49:35.996 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.999+0000 7f8f3fe00700 1 -- 172.21.15.153:0/2029703422 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8f34017400 con 0x7f8f4810d280 2024-03-21T10:49:35.996 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.999+0000 7f8f4e800700 1 -- 172.21.15.153:0/2029703422 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f481a0830 con 0x7f8f4810d280 2024-03-21T10:49:35.996 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:35.999+0000 7f8f4e800700 1 -- 172.21.15.153:0/2029703422 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f481a0d50 con 0x7f8f4810d280 2024-03-21T10:49:35.996 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:36.000+0000 7f8f4e800700 1 -- 172.21.15.153:0/2029703422 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f4810f6c0 con 0x7f8f4810d280 2024-03-21T10:49:35.997 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:36.001+0000 7f8f3fe00700 1 -- 172.21.15.153:0/2029703422 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f8f3400f460 con 0x7f8f4810d280 2024-03-21T10:49:35.998 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:36.002+0000 7f8f3fe00700 1 --2- 172.21.15.153:0/2029703422 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f8f2c06e460 0x7f8f2c070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:49:35.998 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:36.002+0000 7f8f3fe00700 1 -- 172.21.15.153:0/2029703422 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f8f3408f9e0 con 0x7f8f4810d280 2024-03-21T10:49:35.998 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:36.003+0000 7f8f47400700 1 --2- 172.21.15.153:0/2029703422 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f8f2c06e460 0x7f8f2c070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:49:35.999 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:36.003+0000 7f8f47400700 1 --2- 172.21.15.153:0/2029703422 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f8f2c06e460 0x7f8f2c070920 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f8f38005fd0 tx=0x7f8f38005ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:49:36.002 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:36.007+0000 7f8f3fe00700 1 -- 172.21.15.153:0/2029703422 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f8f34058aa0 con 0x7f8f4810d280 2024-03-21T10:49:36.225 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:36.229+0000 7f8f4e800700 1 -- 172.21.15.153:0/2029703422 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8f4802cce0 con 0x7f8f2c06e460 2024-03-21T10:49:36.229 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:36.233+0000 7f8f3fe00700 1 -- 172.21.15.153:0/2029703422 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f8f4802cce0 con 0x7f8f2c06e460 2024-03-21T10:49:36.229 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:49:36.229 INFO:teuthology.orchestra.run.smithi153.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232", 2024-03-21T10:49:36.229 INFO:teuthology.orchestra.run.smithi153.stdout: "in_progress": true, 2024-03-21T10:49:36.229 INFO:teuthology.orchestra.run.smithi153.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-21T10:49:36.229 INFO:teuthology.orchestra.run.smithi153.stdout: "services_complete": [ 2024-03-21T10:49:36.229 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr", 2024-03-21T10:49:36.229 INFO:teuthology.orchestra.run.smithi153.stdout: "mon", 2024-03-21T10:49:36.230 INFO:teuthology.orchestra.run.smithi153.stdout: "mds", 2024-03-21T10:49:36.230 INFO:teuthology.orchestra.run.smithi153.stdout: "crash", 2024-03-21T10:49:36.230 INFO:teuthology.orchestra.run.smithi153.stdout: "osd" 2024-03-21T10:49:36.230 INFO:teuthology.orchestra.run.smithi153.stdout: ], 2024-03-21T10:49:36.230 INFO:teuthology.orchestra.run.smithi153.stdout: "progress": "16/23 daemons upgraded", 2024-03-21T10:49:36.230 INFO:teuthology.orchestra.run.smithi153.stdout: "message": "Currently upgrading mds daemons", 2024-03-21T10:49:36.230 INFO:teuthology.orchestra.run.smithi153.stdout: "is_paused": false 2024-03-21T10:49:36.230 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:49:36.232 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:36.236+0000 7f8f4e800700 1 -- 172.21.15.153:0/2029703422 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f8f2c06e460 msgr2=0x7f8f2c070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:49:36.232 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:36.236+0000 7f8f4e800700 1 --2- 172.21.15.153:0/2029703422 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f8f2c06e460 0x7f8f2c070920 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f8f38005fd0 tx=0x7f8f38005ee0 comp rx=0 tx=0).stop 2024-03-21T10:49:36.232 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:36.236+0000 7f8f4e800700 1 -- 172.21.15.153:0/2029703422 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f4810d280 msgr2=0x7f8f4819d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:49:36.233 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:36.236+0000 7f8f4e800700 1 --2- 172.21.15.153:0/2029703422 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f4810d280 0x7f8f4819d1f0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f8f340092e0 tx=0x7f8f3400be20 comp rx=0 tx=0).stop 2024-03-21T10:49:36.233 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:36.236+0000 7f8f4e800700 1 -- 172.21.15.153:0/2029703422 shutdown_connections 2024-03-21T10:49:36.233 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:36.236+0000 7f8f4e800700 1 --2- 172.21.15.153:0/2029703422 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8f481071d0 0x7f8f4819ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:49:36.233 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:36.236+0000 7f8f4e800700 1 --2- 172.21.15.153:0/2029703422 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f8f2c06e460 0x7f8f2c070920 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:49:36.233 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:36.236+0000 7f8f4e800700 1 --2- 172.21.15.153:0/2029703422 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8f4810d280 0x7f8f4819d1f0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:49:36.233 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:36.236+0000 7f8f4e800700 1 -- 172.21.15.153:0/2029703422 >> 172.21.15.153:0/2029703422 conn(0x7f8f480769e0 msgr2=0x7f8f48078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:49:36.233 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:36.236+0000 7f8f4e800700 1 -- 172.21.15.153:0/2029703422 shutdown_connections 2024-03-21T10:49:36.233 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:36.236+0000 7f8f4e800700 1 -- 172.21.15.153:0/2029703422 wait complete. 2024-03-21T10:49:36.384 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:36.387+0000 7f60f9c00700 1 -- 172.21.15.153:0/1034484501 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f60f4073160 msgr2=0x7f60f4073540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:49:36.384 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:36.387+0000 7f60f9c00700 1 --2- 172.21.15.153:0/1034484501 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f60f4073160 0x7f60f4073540 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f60e4009910 tx=0x7f60e4009c20 comp rx=0 tx=0).stop 2024-03-21T10:49:36.384 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:36.388+0000 7f60f9c00700 1 -- 172.21.15.153:0/1034484501 shutdown_connections 2024-03-21T10:49:36.384 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:36.388+0000 7f60f9c00700 1 --2- 172.21.15.153:0/1034484501 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f60f4073a80 0x7f60f410cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:49:36.384 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:36.388+0000 7f60f9c00700 1 --2- 172.21.15.153:0/1034484501 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f60f4073160 0x7f60f4073540 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:49:36.384 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:36.388+0000 7f60f9c00700 1 -- 172.21.15.153:0/1034484501 >> 172.21.15.153:0/1034484501 conn(0x7f60f40fc650 msgr2=0x7f60f40fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:49:36.384 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:36.388+0000 7f60f9c00700 1 -- 172.21.15.153:0/1034484501 shutdown_connections 2024-03-21T10:49:36.384 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:36.388+0000 7f60f9c00700 1 -- 172.21.15.153:0/1034484501 wait complete. 2024-03-21T10:49:36.385 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:36.389+0000 7f60f9c00700 1 Processor -- start 2024-03-21T10:49:36.385 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:36.389+0000 7f60f9c00700 1 -- start start 2024-03-21T10:49:36.386 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:36.389+0000 7f60f9c00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f60f4073160 0x7f60f4198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:49:36.386 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:36.389+0000 7f60f9c00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f60f4073a80 0x7f60f4198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:49:36.386 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:36.389+0000 7f60f9c00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60f4199530 con 0x7f60f4073a80 2024-03-21T10:49:36.386 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:36.389+0000 7f60f9c00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60f419d340 con 0x7f60f4073160 2024-03-21T10:49:36.386 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:36.390+0000 7f60f2000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f60f4073a80 0x7f60f4198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:49:36.386 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:36.390+0000 7f60f2000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f60f4073a80 0x7f60f4198ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:50024/0 (socket says 172.21.15.153:50024) 2024-03-21T10:49:36.386 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:36.390+0000 7f60f2000700 1 -- 172.21.15.153:0/226341908 learned_addr learned my addr 172.21.15.153:0/226341908 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:49:36.386 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:36.390+0000 7f60f2a00700 1 --2- 172.21.15.153:0/226341908 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f60f4073160 0x7f60f4198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:49:36.387 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:36.390+0000 7f60f2000700 1 -- 172.21.15.153:0/226341908 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f60f4073160 msgr2=0x7f60f4198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:49:36.387 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:36.390+0000 7f60f2000700 1 --2- 172.21.15.153:0/226341908 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f60f4073160 0x7f60f4198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:49:36.387 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:36.390+0000 7f60f2000700 1 -- 172.21.15.153:0/226341908 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60e40095f0 con 0x7f60f4073a80 2024-03-21T10:49:36.387 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:36.390+0000 7f60f2000700 1 --2- 172.21.15.153:0/226341908 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f60f4073a80 0x7f60f4198ea0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f60e000d8d0 tx=0x7f60e000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:49:36.387 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:36.391+0000 7f60eb400700 1 -- 172.21.15.153:0/226341908 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f60e0009940 con 0x7f60f4073a80 2024-03-21T10:49:36.387 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:36.391+0000 7f60f9c00700 1 -- 172.21.15.153:0/226341908 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60f419d620 con 0x7f60f4073a80 2024-03-21T10:49:36.387 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:36.391+0000 7f60eb400700 1 -- 172.21.15.153:0/226341908 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f60e0010460 con 0x7f60f4073a80 2024-03-21T10:49:36.387 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:36.391+0000 7f60eb400700 1 -- 172.21.15.153:0/226341908 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f60e000f5f0 con 0x7f60f4073a80 2024-03-21T10:49:36.387 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:36.391+0000 7f60f9c00700 1 -- 172.21.15.153:0/226341908 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60f419db70 con 0x7f60f4073a80 2024-03-21T10:49:36.388 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:36.392+0000 7f60f9c00700 1 -- 172.21.15.153:0/226341908 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60f410a5c0 con 0x7f60f4073a80 2024-03-21T10:49:36.389 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:36.393+0000 7f60eb400700 1 -- 172.21.15.153:0/226341908 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f60e00105d0 con 0x7f60f4073a80 2024-03-21T10:49:36.390 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:36.394+0000 7f60eb400700 1 --2- 172.21.15.153:0/226341908 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f60d806e460 0x7f60d8070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:49:36.390 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:36.394+0000 7f60eb400700 1 -- 172.21.15.153:0/226341908 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f60e008e5a0 con 0x7f60f4073a80 2024-03-21T10:49:36.390 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:36.394+0000 7f60f2a00700 1 --2- 172.21.15.153:0/226341908 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f60d806e460 0x7f60d8070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:49:36.390 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:36.395+0000 7f60f2a00700 1 --2- 172.21.15.153:0/226341908 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f60d806e460 0x7f60d8070920 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f60e400b5c0 tx=0x7f60e401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:49:36.395 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:36.399+0000 7f60eb400700 1 -- 172.21.15.153:0/226341908 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f60e005b0f0 con 0x7f60f4073a80 2024-03-21T10:49:36.551 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:49:36 smithi153 ceph-mon[113733]: from='client.36002 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:49:36.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:49:36 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/1686121077' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:49:36.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:49:36 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/4064394030' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:49:36.694 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:36.698+0000 7f60f9c00700 1 -- 172.21.15.153:0/226341908 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f60f419dd90 con 0x7f60f4073a80 2024-03-21T10:49:36.694 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:36.699+0000 7f60eb400700 1 -- 172.21.15.153:0/226341908 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4493 (secure 0 0 0) 0x7f60e005ac80 con 0x7f60f4073a80 2024-03-21T10:49:36.695 INFO:teuthology.orchestra.run.smithi153.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-03-21T10:49:36.695 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-03-21T10:49:36.695 INFO:teuthology.orchestra.run.smithi153.stdout: host smithi174 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi174/config 2024-03-21T10:49:36.695 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/podman: timeout after 895 seconds 2024-03-21T10:49:36.695 INFO:teuthology.orchestra.run.smithi153.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 -e NODE_NAME=smithi174 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/run/ceph:z -v /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpd910r3_i:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 inventory --format=json-pretty --filter-for-batch 2024-03-21T10:49:36.695 INFO:teuthology.orchestra.run.smithi153.stdout:Traceback (most recent call last): 2024-03-21T10:49:36.695 INFO:teuthology.orchestra.run.smithi153.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-03-21T10:49:36.695 INFO:teuthology.orchestra.run.smithi153.stdout: return _run_code(code, main_globals, None, 2024-03-21T10:49:36.695 INFO:teuthology.orchestra.run.smithi153.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-03-21T10:49:36.695 INFO:teuthology.orchestra.run.smithi153.stdout: exec(code, run_globals) 2024-03-21T10:49:36.696 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 5606, in 2024-03-21T10:49:36.696 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 5594, in main 2024-03-21T10:49:36.696 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 406, in _infer_config 2024-03-21T10:49:36.696 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 321, in _infer_fsid 2024-03-21T10:49:36.696 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 434, in _infer_image 2024-03-21T10:49:36.696 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 308, in _validate_fsid 2024-03-21T10:49:36.696 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 3357, in command_ceph_volume 2024-03-21T10:49:36.696 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-03-21T10:49:36.696 INFO:teuthology.orchestra.run.smithi153.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 -e NODE_NAME=smithi174 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/run/ceph:z -v /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpd910r3_i:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 inventory --format=json-pretty --filter-for-batch 2024-03-21T10:49:36.696 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-21T10:49:36.696 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is degraded 2024-03-21T10:49:36.696 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-21T10:49:36.696 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-21T10:49:36.696 INFO:teuthology.orchestra.run.smithi153.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-21T10:49:36.696 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-21T10:49:36.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:49:36 smithi174 ceph-mon[88004]: from='client.36002 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:49:36.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:49:36 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/1686121077' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:49:36.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:49:36 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/4064394030' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:49:36.698 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:36.701+0000 7f60f9c00700 1 -- 172.21.15.153:0/226341908 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f60d806e460 msgr2=0x7f60d8070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:49:36.698 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:36.701+0000 7f60f9c00700 1 --2- 172.21.15.153:0/226341908 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f60d806e460 0x7f60d8070920 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f60e400b5c0 tx=0x7f60e401a040 comp rx=0 tx=0).stop 2024-03-21T10:49:36.698 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:36.701+0000 7f60f9c00700 1 -- 172.21.15.153:0/226341908 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f60f4073a80 msgr2=0x7f60f4198ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:49:36.698 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:36.701+0000 7f60f9c00700 1 --2- 172.21.15.153:0/226341908 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f60f4073a80 0x7f60f4198ea0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f60e000d8d0 tx=0x7f60e000dc90 comp rx=0 tx=0).stop 2024-03-21T10:49:36.698 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:36.702+0000 7f60f9c00700 1 -- 172.21.15.153:0/226341908 shutdown_connections 2024-03-21T10:49:36.698 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:36.702+0000 7f60f9c00700 1 --2- 172.21.15.153:0/226341908 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f60f4073160 0x7f60f4198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:49:36.698 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:36.702+0000 7f60f9c00700 1 --2- 172.21.15.153:0/226341908 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f60d806e460 0x7f60d8070920 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:49:36.698 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:36.702+0000 7f60f9c00700 1 --2- 172.21.15.153:0/226341908 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f60f4073a80 0x7f60f4198ea0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:49:36.698 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:36.702+0000 7f60f9c00700 1 -- 172.21.15.153:0/226341908 >> 172.21.15.153:0/226341908 conn(0x7f60f40fc650 msgr2=0x7f60f4107700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:49:36.698 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:36.702+0000 7f60f9c00700 1 -- 172.21.15.153:0/226341908 shutdown_connections 2024-03-21T10:49:36.698 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:49:36.702+0000 7f60f9c00700 1 -- 172.21.15.153:0/226341908 wait complete. 2024-03-21T10:49:37.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:49:37 smithi153 ceph-mon[113733]: pgmap v1134: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:49:37.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:49:37 smithi153 ceph-mon[113733]: from='client.36014 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:49:37.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:49:37 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/226341908' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:49:37.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:49:37 smithi174 ceph-mon[88004]: pgmap v1134: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:49:37.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:49:37 smithi174 ceph-mon[88004]: from='client.36014 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:49:37.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:49:37 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/226341908' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:49:39.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:49:39 smithi153 ceph-mon[113733]: pgmap v1135: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:49:39.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:49:39 smithi174 ceph-mon[88004]: pgmap v1135: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:49:41.551 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:49:41 smithi153 ceph-mon[113733]: pgmap v1136: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:49:41.696 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:49:41 smithi174 ceph-mon[88004]: pgmap v1136: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:49:43.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:49:43 smithi153 ceph-mon[113733]: pgmap v1137: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:49:43.696 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:49:43 smithi174 ceph-mon[88004]: pgmap v1137: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:49:45.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:49:45 smithi153 ceph-mon[113733]: pgmap v1138: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:49:45.696 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:49:45 smithi174 ceph-mon[88004]: pgmap v1138: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:49:46.696 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:49:46 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:49:46.801 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:49:46 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:49:47.696 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:49:47 smithi174 ceph-mon[88004]: pgmap v1139: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:49:47.801 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:49:47 smithi153 ceph-mon[113733]: pgmap v1139: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:49:49.446 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:49:49 smithi174 ceph-mon[88004]: pgmap v1140: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:49:49.801 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:49:49 smithi153 ceph-mon[113733]: pgmap v1140: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:49:51.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:49:51 smithi153 ceph-mon[113733]: pgmap v1141: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:49:51.696 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:49:51 smithi174 ceph-mon[88004]: pgmap v1141: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:49:53.696 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:49:53 smithi174 ceph-mon[88004]: pgmap v1142: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:49:53.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:49:53 smithi153 ceph-mon[113733]: pgmap v1142: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:49:55.696 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:49:55 smithi174 ceph-mon[88004]: pgmap v1143: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:49:55.801 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:49:55 smithi153 ceph-mon[113733]: pgmap v1143: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:49:57.696 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:49:57 smithi174 ceph-mon[88004]: pgmap v1144: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:49:57.801 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:49:57 smithi153 ceph-mon[113733]: pgmap v1144: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:49:59.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:49:59 smithi174 ceph-mon[88004]: pgmap v1145: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:49:59.801 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:49:59 smithi153 ceph-mon[113733]: pgmap v1145: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:50:00.302 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:00 smithi153 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-mon-smithi153[113729]: 2024-03-21T10:49:59.998+0000 7f2169e00700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-03-21T10:50:00.696 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:00 smithi174 ceph-mon[88004]: overall HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-03-21T10:50:00.801 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:00 smithi153 ceph-mon[113733]: overall HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-03-21T10:50:01.696 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:01 smithi174 ceph-mon[88004]: pgmap v1146: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:50:01.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:01 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:50:01.801 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:01 smithi153 ceph-mon[113733]: pgmap v1146: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:50:01.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:01 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:50:03.696 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:03 smithi174 ceph-mon[88004]: pgmap v1147: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:50:03.801 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:03 smithi153 ceph-mon[113733]: pgmap v1147: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:50:05.696 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:05 smithi174 ceph-mon[88004]: pgmap v1148: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:50:05.801 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:05 smithi153 ceph-mon[113733]: pgmap v1148: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:50:06.868 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:06.871+0000 7f496f400700 1 -- 172.21.15.153:0/3609524754 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49681071d0 msgr2=0x7f4968107650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:50:06.869 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:06.871+0000 7f496f400700 1 --2- 172.21.15.153:0/3609524754 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49681071d0 0x7f4968107650 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f4954009910 tx=0x7f4954009c20 comp rx=0 tx=0).stop 2024-03-21T10:50:06.869 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:06.871+0000 7f496f400700 1 -- 172.21.15.153:0/3609524754 shutdown_connections 2024-03-21T10:50:06.869 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:06.871+0000 7f496f400700 1 --2- 172.21.15.153:0/3609524754 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49681071d0 0x7f4968107650 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:50:06.869 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:06.871+0000 7f496f400700 1 --2- 172.21.15.153:0/3609524754 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f496810d280 0x7f496810d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:50:06.869 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:06.871+0000 7f496f400700 1 -- 172.21.15.153:0/3609524754 >> 172.21.15.153:0/3609524754 conn(0x7f49680769e0 msgr2=0x7f4968078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:50:06.869 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:06.871+0000 7f496f400700 1 -- 172.21.15.153:0/3609524754 shutdown_connections 2024-03-21T10:50:06.869 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:06.871+0000 7f496f400700 1 -- 172.21.15.153:0/3609524754 wait complete. 2024-03-21T10:50:06.870 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:06.872+0000 7f496f400700 1 Processor -- start 2024-03-21T10:50:06.871 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:06.872+0000 7f496f400700 1 -- start start 2024-03-21T10:50:06.871 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:06.873+0000 7f496f400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49681071d0 0x7f496819ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:50:06.871 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:06.873+0000 7f496f400700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f496810d280 0x7f496819d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:50:06.871 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:06.873+0000 7f496f400700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f496819d880 con 0x7f49681071d0 2024-03-21T10:50:06.871 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:06.873+0000 7f496f400700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49681a05b0 con 0x7f496810d280 2024-03-21T10:50:06.871 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:06.873+0000 7f496ca00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49681071d0 0x7f496819ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:50:06.871 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:06.873+0000 7f496ca00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49681071d0 0x7f496819ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:49172/0 (socket says 172.21.15.153:49172) 2024-03-21T10:50:06.871 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:06.873+0000 7f496ca00700 1 -- 172.21.15.153:0/1347401882 learned_addr learned my addr 172.21.15.153:0/1347401882 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:50:06.871 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:06.874+0000 7f4967e00700 1 --2- 172.21.15.153:0/1347401882 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f496810d280 0x7f496819d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:50:06.871 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:06.874+0000 7f496ca00700 1 -- 172.21.15.153:0/1347401882 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f496810d280 msgr2=0x7f496819d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:50:06.871 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:06.874+0000 7f496ca00700 1 --2- 172.21.15.153:0/1347401882 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f496810d280 0x7f496819d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:50:06.871 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:06.874+0000 7f496ca00700 1 -- 172.21.15.153:0/1347401882 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49540095f0 con 0x7f49681071d0 2024-03-21T10:50:06.871 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:06.874+0000 7f4967e00700 1 --2- 172.21.15.153:0/1347401882 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f496810d280 0x7f496819d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:50:06.872 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:06.874+0000 7f496ca00700 1 --2- 172.21.15.153:0/1347401882 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49681071d0 0x7f496819ccb0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f495c00ada0 tx=0x7f495c00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:50:06.872 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:06.874+0000 7f4965600700 1 -- 172.21.15.153:0/1347401882 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f495c00f070 con 0x7f49681071d0 2024-03-21T10:50:06.872 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:06.874+0000 7f4965600700 1 -- 172.21.15.153:0/1347401882 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f495c007bf0 con 0x7f49681071d0 2024-03-21T10:50:06.873 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:06.874+0000 7f4965600700 1 -- 172.21.15.153:0/1347401882 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f495c005690 con 0x7f49681071d0 2024-03-21T10:50:06.873 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:06.874+0000 7f496f400700 1 -- 172.21.15.153:0/1347401882 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49681a0890 con 0x7f49681071d0 2024-03-21T10:50:06.873 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:06.875+0000 7f496f400700 1 -- 172.21.15.153:0/1347401882 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49681a0e60 con 0x7f49681071d0 2024-03-21T10:50:06.873 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:06.876+0000 7f496f400700 1 -- 172.21.15.153:0/1347401882 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f496810f6c0 con 0x7f49681071d0 2024-03-21T10:50:06.873 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:06.877+0000 7f4965600700 1 -- 172.21.15.153:0/1347401882 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f495c005850 con 0x7f49681071d0 2024-03-21T10:50:06.874 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:06.878+0000 7f4965600700 1 --2- 172.21.15.153:0/1347401882 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f495006e460 0x7f4950070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:50:06.874 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:06.878+0000 7f4965600700 1 -- 172.21.15.153:0/1347401882 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f495c08df10 con 0x7f49681071d0 2024-03-21T10:50:06.875 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:06.878+0000 7f4967e00700 1 --2- 172.21.15.153:0/1347401882 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f495006e460 0x7f4950070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:50:06.875 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:06.879+0000 7f4967e00700 1 --2- 172.21.15.153:0/1347401882 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f495006e460 0x7f4950070920 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f4954006010 tx=0x7f495400b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:50:06.880 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:06.883+0000 7f4965600700 1 -- 172.21.15.153:0/1347401882 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f495c056fb0 con 0x7f49681071d0 2024-03-21T10:50:07.123 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:07.126+0000 7f496f400700 1 -- 172.21.15.153:0/1347401882 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f49681a1140 con 0x7f495006e460 2024-03-21T10:50:07.125 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:07.129+0000 7f4965600700 1 -- 172.21.15.153:0/1347401882 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f49681a1140 con 0x7f495006e460 2024-03-21T10:50:07.129 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:07.132+0000 7f496f400700 1 -- 172.21.15.153:0/1347401882 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f495006e460 msgr2=0x7f4950070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:50:07.129 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:07.132+0000 7f496f400700 1 --2- 172.21.15.153:0/1347401882 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f495006e460 0x7f4950070920 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f4954006010 tx=0x7f495400b540 comp rx=0 tx=0).stop 2024-03-21T10:50:07.129 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:07.132+0000 7f496f400700 1 -- 172.21.15.153:0/1347401882 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49681071d0 msgr2=0x7f496819ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:50:07.129 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:07.132+0000 7f496f400700 1 --2- 172.21.15.153:0/1347401882 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49681071d0 0x7f496819ccb0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f495c00ada0 tx=0x7f495c00c330 comp rx=0 tx=0).stop 2024-03-21T10:50:07.129 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:07.132+0000 7f496f400700 1 -- 172.21.15.153:0/1347401882 shutdown_connections 2024-03-21T10:50:07.129 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:07.132+0000 7f496f400700 1 --2- 172.21.15.153:0/1347401882 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f495006e460 0x7f4950070920 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:50:07.129 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:07.132+0000 7f496f400700 1 --2- 172.21.15.153:0/1347401882 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f49681071d0 0x7f496819ccb0 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:50:07.129 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:07.132+0000 7f496f400700 1 --2- 172.21.15.153:0/1347401882 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f496810d280 0x7f496819d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:50:07.130 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:07.132+0000 7f496f400700 1 -- 172.21.15.153:0/1347401882 >> 172.21.15.153:0/1347401882 conn(0x7f49680769e0 msgr2=0x7f4968078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:50:07.130 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:07.132+0000 7f496f400700 1 -- 172.21.15.153:0/1347401882 shutdown_connections 2024-03-21T10:50:07.130 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:07.132+0000 7f496f400700 1 -- 172.21.15.153:0/1347401882 wait complete. 2024-03-21T10:50:07.145 INFO:teuthology.orchestra.run.smithi153.stdout:true 2024-03-21T10:50:07.289 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:07.292+0000 7f02c3200700 1 -- 172.21.15.153:0/916064654 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f02c410d010 msgr2=0x7f02c410d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:50:07.289 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:07.292+0000 7f02c3200700 1 --2- 172.21.15.153:0/916064654 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f02c410d010 0x7f02c410d3f0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f02b4009910 tx=0x7f02b4009c20 comp rx=0 tx=0).stop 2024-03-21T10:50:07.289 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:07.292+0000 7f02c3200700 1 -- 172.21.15.153:0/916064654 shutdown_connections 2024-03-21T10:50:07.290 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:07.292+0000 7f02c3200700 1 --2- 172.21.15.153:0/916064654 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f02c4106f60 0x7f02c41073e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:50:07.290 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:07.292+0000 7f02c3200700 1 --2- 172.21.15.153:0/916064654 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f02c410d010 0x7f02c410d3f0 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:50:07.290 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:07.292+0000 7f02c3200700 1 -- 172.21.15.153:0/916064654 >> 172.21.15.153:0/916064654 conn(0x7f02c4076760 msgr2=0x7f02c4078b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:50:07.290 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:07.292+0000 7f02c3200700 1 -- 172.21.15.153:0/916064654 shutdown_connections 2024-03-21T10:50:07.290 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:07.292+0000 7f02c3200700 1 -- 172.21.15.153:0/916064654 wait complete. 2024-03-21T10:50:07.291 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:07.293+0000 7f02c3200700 1 Processor -- start 2024-03-21T10:50:07.291 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:07.294+0000 7f02c3200700 1 -- start start 2024-03-21T10:50:07.291 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:07.294+0000 7f02c3200700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f02c4106f60 0x7f02c41a29b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:50:07.291 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:07.294+0000 7f02c3200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f02c410d010 0x7f02c41a2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:50:07.292 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:07.294+0000 7f02c3200700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02c419c9a0 con 0x7f02c410d010 2024-03-21T10:50:07.292 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:07.294+0000 7f02c3200700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02c419cb10 con 0x7f02c4106f60 2024-03-21T10:50:07.292 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:07.294+0000 7f02c1400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f02c410d010 0x7f02c41a2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:50:07.292 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:07.294+0000 7f02c1400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f02c410d010 0x7f02c41a2ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:49182/0 (socket says 172.21.15.153:49182) 2024-03-21T10:50:07.292 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:07.294+0000 7f02c1400700 1 -- 172.21.15.153:0/1654283259 learned_addr learned my addr 172.21.15.153:0/1654283259 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:50:07.292 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:07.295+0000 7f02c1e00700 1 --2- 172.21.15.153:0/1654283259 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f02c4106f60 0x7f02c41a29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:50:07.292 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:07.295+0000 7f02c1400700 1 -- 172.21.15.153:0/1654283259 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f02c4106f60 msgr2=0x7f02c41a29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:50:07.292 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:07.295+0000 7f02c1400700 1 --2- 172.21.15.153:0/1654283259 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f02c4106f60 0x7f02c41a29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:50:07.292 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:07.295+0000 7f02c1400700 1 -- 172.21.15.153:0/1654283259 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02b40095f0 con 0x7f02c410d010 2024-03-21T10:50:07.293 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:07.295+0000 7f02c1400700 1 --2- 172.21.15.153:0/1654283259 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f02c410d010 0x7f02c41a2ef0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f02b000d8d0 tx=0x7f02b000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:50:07.293 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:07.295+0000 7f02baa00700 1 -- 172.21.15.153:0/1654283259 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f02b0009940 con 0x7f02c410d010 2024-03-21T10:50:07.293 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:07.295+0000 7f02baa00700 1 -- 172.21.15.153:0/1654283259 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f02b0010460 con 0x7f02c410d010 2024-03-21T10:50:07.293 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:07.295+0000 7f02c3200700 1 -- 172.21.15.153:0/1654283259 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02c419cdf0 con 0x7f02c410d010 2024-03-21T10:50:07.294 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:07.295+0000 7f02baa00700 1 -- 172.21.15.153:0/1654283259 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f02b000f5f0 con 0x7f02c410d010 2024-03-21T10:50:07.294 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:07.296+0000 7f02c3200700 1 -- 172.21.15.153:0/1654283259 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02c419d340 con 0x7f02c410d010 2024-03-21T10:50:07.294 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:07.297+0000 7f02c3200700 1 -- 172.21.15.153:0/1654283259 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02c410f480 con 0x7f02c410d010 2024-03-21T10:50:07.294 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:07.298+0000 7f02baa00700 1 -- 172.21.15.153:0/1654283259 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f02b00105d0 con 0x7f02c410d010 2024-03-21T10:50:07.295 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:07.299+0000 7f02baa00700 1 --2- 172.21.15.153:0/1654283259 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f02a806e460 0x7f02a8070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:50:07.296 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:07.299+0000 7f02baa00700 1 -- 172.21.15.153:0/1654283259 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f02b008e480 con 0x7f02c410d010 2024-03-21T10:50:07.296 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:07.299+0000 7f02c1e00700 1 --2- 172.21.15.153:0/1654283259 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f02a806e460 0x7f02a8070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:50:07.296 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:07.300+0000 7f02c1e00700 1 --2- 172.21.15.153:0/1654283259 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f02a806e460 0x7f02a8070920 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f02b4006010 tx=0x7f02b401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:50:07.300 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:07.303+0000 7f02baa00700 1 -- 172.21.15.153:0/1654283259 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f02b0057520 con 0x7f02c410d010 2024-03-21T10:50:07.536 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:07.539+0000 7f02c3200700 1 -- 172.21.15.153:0/1654283259 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f02c419d620 con 0x7f02a806e460 2024-03-21T10:50:07.538 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:07.542+0000 7f02baa00700 1 -- 172.21.15.153:0/1654283259 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f02c419d620 con 0x7f02a806e460 2024-03-21T10:50:07.541 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:07.544+0000 7f02c3200700 1 -- 172.21.15.153:0/1654283259 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f02a806e460 msgr2=0x7f02a8070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:50:07.541 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:07.544+0000 7f02c3200700 1 --2- 172.21.15.153:0/1654283259 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f02a806e460 0x7f02a8070920 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f02b4006010 tx=0x7f02b401a040 comp rx=0 tx=0).stop 2024-03-21T10:50:07.541 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:07.544+0000 7f02c3200700 1 -- 172.21.15.153:0/1654283259 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f02c410d010 msgr2=0x7f02c41a2ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:50:07.541 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:07.544+0000 7f02c3200700 1 --2- 172.21.15.153:0/1654283259 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f02c410d010 0x7f02c41a2ef0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f02b000d8d0 tx=0x7f02b000dc90 comp rx=0 tx=0).stop 2024-03-21T10:50:07.542 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:07.545+0000 7f02c3200700 1 -- 172.21.15.153:0/1654283259 shutdown_connections 2024-03-21T10:50:07.542 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:07.545+0000 7f02c3200700 1 --2- 172.21.15.153:0/1654283259 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f02c4106f60 0x7f02c41a29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:50:07.542 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:07.545+0000 7f02c3200700 1 --2- 172.21.15.153:0/1654283259 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f02a806e460 0x7f02a8070920 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:50:07.542 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:07.545+0000 7f02c3200700 1 --2- 172.21.15.153:0/1654283259 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f02c410d010 0x7f02c41a2ef0 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:50:07.542 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:07.545+0000 7f02c3200700 1 -- 172.21.15.153:0/1654283259 >> 172.21.15.153:0/1654283259 conn(0x7f02c4076760 msgr2=0x7f02c4078120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:50:07.542 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:07.545+0000 7f02c3200700 1 -- 172.21.15.153:0/1654283259 shutdown_connections 2024-03-21T10:50:07.543 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:07.545+0000 7f02c3200700 1 -- 172.21.15.153:0/1654283259 wait complete. 2024-03-21T10:50:07.687 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:07.690+0000 7ff566000700 1 -- 172.21.15.153:0/440631674 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff560103000 msgr2=0x7ff5601033e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:50:07.687 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:07.690+0000 7ff566000700 1 --2- 172.21.15.153:0/440631674 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff560103000 0x7ff5601033e0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7ff550009910 tx=0x7ff550009c20 comp rx=0 tx=0).stop 2024-03-21T10:50:07.687 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:07.690+0000 7ff566000700 1 -- 172.21.15.153:0/440631674 shutdown_connections 2024-03-21T10:50:07.687 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:07.690+0000 7ff566000700 1 --2- 172.21.15.153:0/440631674 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff5601039b0 0x7ff560107a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:50:07.687 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:07.690+0000 7ff566000700 1 --2- 172.21.15.153:0/440631674 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff560103000 0x7ff5601033e0 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:50:07.687 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:07.690+0000 7ff566000700 1 -- 172.21.15.153:0/440631674 >> 172.21.15.153:0/440631674 conn(0x7ff5600fe8d0 msgr2=0x7ff560100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:50:07.687 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:07.690+0000 7ff566000700 1 -- 172.21.15.153:0/440631674 shutdown_connections 2024-03-21T10:50:07.688 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:07 smithi153 ceph-mon[113733]: pgmap v1149: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:50:07.689 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:07.690+0000 7ff566000700 1 -- 172.21.15.153:0/440631674 wait complete. 2024-03-21T10:50:07.689 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:07.691+0000 7ff566000700 1 Processor -- start 2024-03-21T10:50:07.689 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:07.691+0000 7ff566000700 1 -- start start 2024-03-21T10:50:07.689 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:07.691+0000 7ff566000700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff560103000 0x7ff56019e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:50:07.689 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:07.692+0000 7ff566000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff5601039b0 0x7ff56019ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:50:07.689 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:07.692+0000 7ff566000700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff56019f3c0 con 0x7ff5601039b0 2024-03-21T10:50:07.689 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:07.692+0000 7ff566000700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff560198870 con 0x7ff560103000 2024-03-21T10:50:07.689 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:07.692+0000 7ff55ea00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff5601039b0 0x7ff56019ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:50:07.689 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:07.692+0000 7ff55ea00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff5601039b0 0x7ff56019ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:49210/0 (socket says 172.21.15.153:49210) 2024-03-21T10:50:07.689 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:07.692+0000 7ff55ea00700 1 -- 172.21.15.153:0/1023654190 learned_addr learned my addr 172.21.15.153:0/1023654190 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:50:07.689 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:07.692+0000 7ff55f400700 1 --2- 172.21.15.153:0/1023654190 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff560103000 0x7ff56019e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:50:07.690 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:07.692+0000 7ff55ea00700 1 -- 172.21.15.153:0/1023654190 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff560103000 msgr2=0x7ff56019e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:50:07.690 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:07.692+0000 7ff55ea00700 1 --2- 172.21.15.153:0/1023654190 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff560103000 0x7ff56019e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:50:07.690 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:07.692+0000 7ff55ea00700 1 -- 172.21.15.153:0/1023654190 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5500095f0 con 0x7ff5601039b0 2024-03-21T10:50:07.690 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:07.693+0000 7ff55ea00700 1 --2- 172.21.15.153:0/1023654190 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff5601039b0 0x7ff56019ed30 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7ff54c00d900 tx=0x7ff54c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:50:07.690 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:07.693+0000 7ff557e00700 1 -- 172.21.15.153:0/1023654190 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff54c009900 con 0x7ff5601039b0 2024-03-21T10:50:07.691 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:07.693+0000 7ff557e00700 1 -- 172.21.15.153:0/1023654190 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff54c010460 con 0x7ff5601039b0 2024-03-21T10:50:07.691 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:07.693+0000 7ff566000700 1 -- 172.21.15.153:0/1023654190 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff560198b50 con 0x7ff5601039b0 2024-03-21T10:50:07.691 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:07.693+0000 7ff557e00700 1 -- 172.21.15.153:0/1023654190 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff54c00f5f0 con 0x7ff5601039b0 2024-03-21T10:50:07.691 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:07.693+0000 7ff566000700 1 -- 172.21.15.153:0/1023654190 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5601990a0 con 0x7ff5601039b0 2024-03-21T10:50:07.691 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:07.694+0000 7ff566000700 1 -- 172.21.15.153:0/1023654190 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff56010b3a0 con 0x7ff5601039b0 2024-03-21T10:50:07.692 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:07.695+0000 7ff557e00700 1 -- 172.21.15.153:0/1023654190 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7ff54c0105d0 con 0x7ff5601039b0 2024-03-21T10:50:07.693 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:07.696+0000 7ff557e00700 1 --2- 172.21.15.153:0/1023654190 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff54406e460 0x7ff544070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:50:07.693 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:07.696+0000 7ff557e00700 1 -- 172.21.15.153:0/1023654190 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7ff54c08e410 con 0x7ff5601039b0 2024-03-21T10:50:07.693 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:07.697+0000 7ff55f400700 1 --2- 172.21.15.153:0/1023654190 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff54406e460 0x7ff544070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:50:07.694 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:07.697+0000 7ff55f400700 1 --2- 172.21.15.153:0/1023654190 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff54406e460 0x7ff544070920 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7ff55000b5c0 tx=0x7ff55001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:50:07.696 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:07 smithi174 ceph-mon[88004]: pgmap v1149: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:50:07.697 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:07.701+0000 7ff557e00700 1 -- 172.21.15.153:0/1023654190 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff54c05af60 con 0x7ff5601039b0 2024-03-21T10:50:07.927 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:07.930+0000 7ff566000700 1 -- 172.21.15.153:0/1023654190 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7ff5601992c0 con 0x7ff54406e460 2024-03-21T10:50:07.958 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:07.961+0000 7ff557e00700 1 -- 172.21.15.153:0/1023654190 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7ff5601992c0 con 0x7ff54406e460 2024-03-21T10:50:07.959 INFO:teuthology.orchestra.run.smithi153.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-21T10:50:07.959 INFO:teuthology.orchestra.run.smithi153.stdout:alertmanager.smithi153 smithi153 *:9093,9094 running (38m) 14m ago 52m 15.6M - 0.25.0 c8568f914cd2 9ed3f2200b5f 2024-03-21T10:50:07.959 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi153 smithi153 running (53m) 14m ago 53m 10.4M - 18.2.1 5be31c24972a 5b61994396a6 2024-03-21T10:50:07.959 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi174 smithi174 running (51m) 14m ago 51m 10.1M - 18.2.1 5be31c24972a 4f7a5fc3ac28 2024-03-21T10:50:07.959 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi153 smithi153 running (35m) 14m ago 52m 6656k - 19.0.0-2231-g2edfee68 237e43c7a65f 32c21104d5b0 2024-03-21T10:50:07.959 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi174 smithi174 running (35m) 14m ago 51m 6660k - 19.0.0-2231-g2edfee68 237e43c7a65f a04e48967ab6 2024-03-21T10:50:07.959 INFO:teuthology.orchestra.run.smithi153.stdout:grafana.smithi153 smithi153 *:3000 running (38m) 14m ago 52m 75.2M - 9.4.12 2bacad6d85d8 ee24026ab786 2024-03-21T10:50:07.959 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.iuyluj smithi153 running (30m) 14m ago 45m 18.7M - 19.0.0-2231-g2edfee68 237e43c7a65f 87aff8a5de70 2024-03-21T10:50:07.959 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.zipckr smithi153 running (30m) 14m ago 45m 18.9M - 19.0.0-2231-g2edfee68 237e43c7a65f ccc262bd5dd3 2024-03-21T10:50:07.959 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.nhozag smithi174 running (14m) 14m ago 45m 10.9M - 19.0.0-2231-g2edfee68 237e43c7a65f 9e02aea72c96 2024-03-21T10:50:07.960 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.yyimst smithi174 running (30m) 14m ago 45m 18.5M - 19.0.0-2231-g2edfee68 237e43c7a65f fc1962f9351e 2024-03-21T10:50:07.960 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi153.tluxni smithi153 *:8443,9283,8765 running (41m) 14m ago 54m 512M - 19.0.0-2231-g2edfee68 237e43c7a65f 7ffee0e55382 2024-03-21T10:50:07.960 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi174.qpcwat smithi174 *:8443,9283,8765 running (40m) 14m ago 51m 459M - 19.0.0-2231-g2edfee68 237e43c7a65f 748968f216ab 2024-03-21T10:50:07.960 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi153 smithi153 running (36m) 14m ago 54m 70.8M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f dd8038303dc3 2024-03-21T10:50:07.960 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi174 smithi174 running (36m) 14m ago 51m 51.9M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f 29d78a788f2c 2024-03-21T10:50:07.960 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi153 smithi153 *:9100 running (40m) 14m ago 52m 8140k - 1.5.0 0da6a335fe13 bb835edc0e0b 2024-03-21T10:50:07.960 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi174 smithi174 *:9100 running (39m) 14m ago 51m 8610k - 1.5.0 0da6a335fe13 7c4ecad790fa 2024-03-21T10:50:07.960 INFO:teuthology.orchestra.run.smithi153.stdout:osd.0 smithi153 running (34m) 14m ago 50m 118M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 507900fd4791 2024-03-21T10:50:07.960 INFO:teuthology.orchestra.run.smithi153.stdout:osd.1 smithi153 running (33m) 14m ago 49m 93.0M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 0d646452e9c1 2024-03-21T10:50:07.960 INFO:teuthology.orchestra.run.smithi153.stdout:osd.2 smithi153 running (33m) 14m ago 48m 91.2M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 1326f2afdfcd 2024-03-21T10:50:07.960 INFO:teuthology.orchestra.run.smithi153.stdout:osd.3 smithi174 running (32m) 14m ago 48m 87.5M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 70aafac71ad5 2024-03-21T10:50:07.960 INFO:teuthology.orchestra.run.smithi153.stdout:osd.4 smithi174 running (32m) 14m ago 47m 95.8M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 0d746ce6e79d 2024-03-21T10:50:07.960 INFO:teuthology.orchestra.run.smithi153.stdout:osd.5 smithi174 running (31m) 14m ago 47m 90.3M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 53e5f35b8cac 2024-03-21T10:50:07.961 INFO:teuthology.orchestra.run.smithi153.stdout:prometheus.smithi153 smithi153 *:9095 running (39m) 14m ago 51m 88.1M - 2.43.0 a07b618ecd1d 7d95561e50c2 2024-03-21T10:50:07.962 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:07.965+0000 7ff566000700 1 -- 172.21.15.153:0/1023654190 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff54406e460 msgr2=0x7ff544070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:50:07.962 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:07.965+0000 7ff566000700 1 --2- 172.21.15.153:0/1023654190 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff54406e460 0x7ff544070920 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7ff55000b5c0 tx=0x7ff55001a040 comp rx=0 tx=0).stop 2024-03-21T10:50:07.962 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:07.965+0000 7ff566000700 1 -- 172.21.15.153:0/1023654190 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff5601039b0 msgr2=0x7ff56019ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:50:07.962 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:07.965+0000 7ff566000700 1 --2- 172.21.15.153:0/1023654190 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff5601039b0 0x7ff56019ed30 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7ff54c00d900 tx=0x7ff54c00dcc0 comp rx=0 tx=0).stop 2024-03-21T10:50:07.963 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:07.965+0000 7ff566000700 1 -- 172.21.15.153:0/1023654190 shutdown_connections 2024-03-21T10:50:07.963 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:07.965+0000 7ff566000700 1 --2- 172.21.15.153:0/1023654190 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff560103000 0x7ff56019e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:50:07.963 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:07.965+0000 7ff566000700 1 --2- 172.21.15.153:0/1023654190 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff54406e460 0x7ff544070920 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:50:07.963 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:07.965+0000 7ff566000700 1 --2- 172.21.15.153:0/1023654190 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff5601039b0 0x7ff56019ed30 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:50:07.963 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:07.965+0000 7ff566000700 1 -- 172.21.15.153:0/1023654190 >> 172.21.15.153:0/1023654190 conn(0x7ff5600fe8d0 msgr2=0x7ff5600ffee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:50:07.963 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:07.966+0000 7ff566000700 1 -- 172.21.15.153:0/1023654190 shutdown_connections 2024-03-21T10:50:07.963 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:07.966+0000 7ff566000700 1 -- 172.21.15.153:0/1023654190 wait complete. 2024-03-21T10:50:08.109 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:08.111+0000 7f557a000700 1 -- 172.21.15.153:0/1752375061 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5574106970 msgr2=0x7f5574106d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:50:08.109 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:08.111+0000 7f557a000700 1 --2- 172.21.15.153:0/1752375061 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5574106970 0x7f5574106d50 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f5564009910 tx=0x7f5564009c20 comp rx=0 tx=0).stop 2024-03-21T10:50:08.109 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:08.112+0000 7f557a000700 1 -- 172.21.15.153:0/1752375061 shutdown_connections 2024-03-21T10:50:08.109 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:08.112+0000 7f557a000700 1 --2- 172.21.15.153:0/1752375061 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5574100970 0x7f5574100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:50:08.109 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:08.112+0000 7f557a000700 1 --2- 172.21.15.153:0/1752375061 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5574106970 0x7f5574106d50 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:50:08.109 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:08.112+0000 7f557a000700 1 -- 172.21.15.153:0/1752375061 >> 172.21.15.153:0/1752375061 conn(0x7f55740fc440 msgr2=0x7f55740fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:50:08.109 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:08.112+0000 7f557a000700 1 -- 172.21.15.153:0/1752375061 shutdown_connections 2024-03-21T10:50:08.109 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:08.112+0000 7f557a000700 1 -- 172.21.15.153:0/1752375061 wait complete. 2024-03-21T10:50:08.111 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:08.113+0000 7f557a000700 1 Processor -- start 2024-03-21T10:50:08.111 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:08.113+0000 7f557a000700 1 -- start start 2024-03-21T10:50:08.111 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:08.113+0000 7f557a000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5574100970 0x7f557419e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:50:08.111 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:08.113+0000 7f557a000700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5574106970 0x7f557419eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:50:08.111 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:08.113+0000 7f557a000700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f557419f210 con 0x7f5574100970 2024-03-21T10:50:08.111 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:08.113+0000 7f557a000700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55741986c0 con 0x7f5574106970 2024-03-21T10:50:08.111 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:08.113+0000 7f5578c00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5574100970 0x7f557419e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:50:08.111 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:08.114+0000 7f5578c00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5574100970 0x7f557419e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:49238/0 (socket says 172.21.15.153:49238) 2024-03-21T10:50:08.111 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:08.114+0000 7f5578c00700 1 -- 172.21.15.153:0/2757817294 learned_addr learned my addr 172.21.15.153:0/2757817294 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:50:08.111 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:08.114+0000 7f5578c00700 1 -- 172.21.15.153:0/2757817294 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5574106970 msgr2=0x7f557419eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:50:08.111 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:08.114+0000 7f556be00700 1 --2- 172.21.15.153:0/2757817294 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5574106970 0x7f557419eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:50:08.111 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:08.114+0000 7f5578c00700 1 --2- 172.21.15.153:0/2757817294 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5574106970 0x7f557419eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:50:08.111 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:08.114+0000 7f5578c00700 1 -- 172.21.15.153:0/2757817294 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55640095f0 con 0x7f5574100970 2024-03-21T10:50:08.112 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:08.114+0000 7f5578c00700 1 --2- 172.21.15.153:0/2757817294 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5574100970 0x7f557419e640 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f5564006010 tx=0x7f556400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:50:08.112 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:08.114+0000 7f5572000700 1 -- 172.21.15.153:0/2757817294 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f556401d070 con 0x7f5574100970 2024-03-21T10:50:08.112 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:08.115+0000 7f5572000700 1 -- 172.21.15.153:0/2757817294 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5564004fb0 con 0x7f5574100970 2024-03-21T10:50:08.112 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:08.115+0000 7f5572000700 1 -- 172.21.15.153:0/2757817294 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5564022ea0 con 0x7f5574100970 2024-03-21T10:50:08.112 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:08.115+0000 7f557a000700 1 -- 172.21.15.153:0/2757817294 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5574198940 con 0x7f5574100970 2024-03-21T10:50:08.113 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:08.115+0000 7f557a000700 1 -- 172.21.15.153:0/2757817294 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5574198e30 con 0x7f5574100970 2024-03-21T10:50:08.113 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:08.116+0000 7f557a000700 1 -- 172.21.15.153:0/2757817294 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f557404f2e0 con 0x7f5574100970 2024-03-21T10:50:08.113 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:08.117+0000 7f5572000700 1 -- 172.21.15.153:0/2757817294 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f5564022470 con 0x7f5574100970 2024-03-21T10:50:08.114 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:08.117+0000 7f5572000700 1 --2- 172.21.15.153:0/2757817294 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f555806e460 0x7f5558070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:50:08.114 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:08.118+0000 7f5572000700 1 -- 172.21.15.153:0/2757817294 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f556408fa70 con 0x7f5574100970 2024-03-21T10:50:08.114 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:08.118+0000 7f556be00700 1 --2- 172.21.15.153:0/2757817294 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f555806e460 0x7f5558070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:50:08.115 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:08.118+0000 7f556be00700 1 --2- 172.21.15.153:0/2757817294 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f555806e460 0x7f5558070920 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f555c006fd0 tx=0x7f555c008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:50:08.119 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:08.122+0000 7f5572000700 1 -- 172.21.15.153:0/2757817294 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f556405c810 con 0x7f5574100970 2024-03-21T10:50:08.426 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:08.429+0000 7f557a000700 1 -- 172.21.15.153:0/2757817294 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f5574199bd0 con 0x7f5574100970 2024-03-21T10:50:08.427 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:08.430+0000 7f5572000700 1 -- 172.21.15.153:0/2757817294 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f5564022720 con 0x7f5574100970 2024-03-21T10:50:08.427 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:50:08.428 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": { 2024-03-21T10:50:08.428 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:50:08.428 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:50:08.428 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": { 2024-03-21T10:50:08.428 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:50:08.428 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:50:08.428 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": { 2024-03-21T10:50:08.428 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 6 2024-03-21T10:50:08.428 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:50:08.428 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": { 2024-03-21T10:50:08.428 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 4 2024-03-21T10:50:08.428 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:50:08.428 INFO:teuthology.orchestra.run.smithi153.stdout: "overall": { 2024-03-21T10:50:08.428 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 14 2024-03-21T10:50:08.428 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-03-21T10:50:08.429 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:50:08.430 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:08.433+0000 7f557a000700 1 -- 172.21.15.153:0/2757817294 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f555806e460 msgr2=0x7f5558070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:50:08.430 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:08.433+0000 7f557a000700 1 --2- 172.21.15.153:0/2757817294 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f555806e460 0x7f5558070920 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f555c006fd0 tx=0x7f555c008040 comp rx=0 tx=0).stop 2024-03-21T10:50:08.430 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:08.433+0000 7f557a000700 1 -- 172.21.15.153:0/2757817294 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5574100970 msgr2=0x7f557419e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:50:08.430 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:08.433+0000 7f557a000700 1 --2- 172.21.15.153:0/2757817294 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5574100970 0x7f557419e640 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f5564006010 tx=0x7f556400be20 comp rx=0 tx=0).stop 2024-03-21T10:50:08.430 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:08.433+0000 7f557a000700 1 -- 172.21.15.153:0/2757817294 shutdown_connections 2024-03-21T10:50:08.430 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:08.433+0000 7f557a000700 1 --2- 172.21.15.153:0/2757817294 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f555806e460 0x7f5558070920 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:50:08.430 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:08.433+0000 7f557a000700 1 --2- 172.21.15.153:0/2757817294 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5574100970 0x7f557419e640 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:50:08.430 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:08.433+0000 7f557a000700 1 --2- 172.21.15.153:0/2757817294 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5574106970 0x7f557419eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:50:08.430 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:08.433+0000 7f557a000700 1 -- 172.21.15.153:0/2757817294 >> 172.21.15.153:0/2757817294 conn(0x7f55740fc440 msgr2=0x7f5574073000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:50:08.431 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:08.433+0000 7f557a000700 1 -- 172.21.15.153:0/2757817294 shutdown_connections 2024-03-21T10:50:08.431 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:08.433+0000 7f557a000700 1 -- 172.21.15.153:0/2757817294 wait complete. 2024-03-21T10:50:08.582 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:08.585+0000 7ff47f800700 1 -- 172.21.15.153:0/1286841586 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff4781071d0 msgr2=0x7ff478107650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:50:08.582 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:08.585+0000 7ff47f800700 1 --2- 172.21.15.153:0/1286841586 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff4781071d0 0x7ff478107650 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7ff464009910 tx=0x7ff464009c20 comp rx=0 tx=0).stop 2024-03-21T10:50:08.582 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:08.585+0000 7ff47f800700 1 -- 172.21.15.153:0/1286841586 shutdown_connections 2024-03-21T10:50:08.582 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:08.585+0000 7ff47f800700 1 --2- 172.21.15.153:0/1286841586 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff4781071d0 0x7ff478107650 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:50:08.582 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:08.585+0000 7ff47f800700 1 --2- 172.21.15.153:0/1286841586 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff47810d280 0x7ff47810d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:50:08.582 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:08.585+0000 7ff47f800700 1 -- 172.21.15.153:0/1286841586 >> 172.21.15.153:0/1286841586 conn(0x7ff4780769e0 msgr2=0x7ff478078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:50:08.582 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:08.585+0000 7ff47f800700 1 -- 172.21.15.153:0/1286841586 shutdown_connections 2024-03-21T10:50:08.583 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:08.585+0000 7ff47f800700 1 -- 172.21.15.153:0/1286841586 wait complete. 2024-03-21T10:50:08.585 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:08.586+0000 7ff47f800700 1 Processor -- start 2024-03-21T10:50:08.585 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:08.586+0000 7ff47f800700 1 -- start start 2024-03-21T10:50:08.585 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:08.586+0000 7ff47f800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff4781071d0 0x7ff47819ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:50:08.585 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:08.586+0000 7ff47f800700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff47810d280 0x7ff47819d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:50:08.585 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:08.586+0000 7ff47f800700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff47819d880 con 0x7ff4781071d0 2024-03-21T10:50:08.586 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:08.586+0000 7ff47f800700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4781a05b0 con 0x7ff47810d280 2024-03-21T10:50:08.586 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:08.587+0000 7ff47ce00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff4781071d0 0x7ff47819ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:50:08.586 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:08.587+0000 7ff47ce00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff4781071d0 0x7ff47819ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:49252/0 (socket says 172.21.15.153:49252) 2024-03-21T10:50:08.586 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:08.587+0000 7ff47ce00700 1 -- 172.21.15.153:0/4110842711 learned_addr learned my addr 172.21.15.153:0/4110842711 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:50:08.586 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:08.587+0000 7ff47ce00700 1 -- 172.21.15.153:0/4110842711 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff47810d280 msgr2=0x7ff47819d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:50:08.586 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:08.587+0000 7ff47ce00700 1 --2- 172.21.15.153:0/4110842711 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff47810d280 0x7ff47819d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:50:08.586 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:08.587+0000 7ff47ce00700 1 -- 172.21.15.153:0/4110842711 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4640095f0 con 0x7ff4781071d0 2024-03-21T10:50:08.586 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:08.587+0000 7ff47ce00700 1 --2- 172.21.15.153:0/4110842711 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff4781071d0 0x7ff47819ccb0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7ff46c00d8d0 tx=0x7ff46c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:50:08.586 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:08.588+0000 7ff475600700 1 -- 172.21.15.153:0/4110842711 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff46c009940 con 0x7ff4781071d0 2024-03-21T10:50:08.586 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:08.588+0000 7ff475600700 1 -- 172.21.15.153:0/4110842711 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff46c010460 con 0x7ff4781071d0 2024-03-21T10:50:08.586 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:08.588+0000 7ff475600700 1 -- 172.21.15.153:0/4110842711 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff46c00f5f0 con 0x7ff4781071d0 2024-03-21T10:50:08.586 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:08.588+0000 7ff47f800700 1 -- 172.21.15.153:0/4110842711 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4781a0890 con 0x7ff4781071d0 2024-03-21T10:50:08.586 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:08.588+0000 7ff47f800700 1 -- 172.21.15.153:0/4110842711 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4781a0e60 con 0x7ff4781071d0 2024-03-21T10:50:08.587 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:08.589+0000 7ff47f800700 1 -- 172.21.15.153:0/4110842711 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff47810f6c0 con 0x7ff4781071d0 2024-03-21T10:50:08.587 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:08.590+0000 7ff475600700 1 -- 172.21.15.153:0/4110842711 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7ff46c0105d0 con 0x7ff4781071d0 2024-03-21T10:50:08.587 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:08.591+0000 7ff475600700 1 --2- 172.21.15.153:0/4110842711 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff46006e460 0x7ff460070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:50:08.588 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:08.591+0000 7ff475600700 1 -- 172.21.15.153:0/4110842711 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7ff46c08e480 con 0x7ff4781071d0 2024-03-21T10:50:08.588 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:08.591+0000 7ff477e00700 1 --2- 172.21.15.153:0/4110842711 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff46006e460 0x7ff460070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:50:08.588 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:08.592+0000 7ff477e00700 1 --2- 172.21.15.153:0/4110842711 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff46006e460 0x7ff460070920 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7ff464006010 tx=0x7ff46400b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:50:08.592 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:08.595+0000 7ff475600700 1 -- 172.21.15.153:0/4110842711 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff46c057520 con 0x7ff4781071d0 2024-03-21T10:50:08.696 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:08 smithi174 ceph-mon[88004]: from='client.36022 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:50:08.696 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:08 smithi174 ceph-mon[88004]: from='client.36026 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:50:08.801 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:08 smithi153 ceph-mon[113733]: from='client.36022 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:50:08.801 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:08 smithi153 ceph-mon[113733]: from='client.36026 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:50:08.859 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:08.862+0000 7ff47f800700 1 -- 172.21.15.153:0/4110842711 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7ff47804ea90 con 0x7ff4781071d0 2024-03-21T10:50:08.860 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:08.863+0000 7ff475600700 1 -- 172.21.15.153:0/4110842711 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1876 (secure 0 0 0) 0x7ff46c016770 con 0x7ff4781071d0 2024-03-21T10:50:08.860 INFO:teuthology.orchestra.run.smithi153.stdout:e32 2024-03-21T10:50:08.860 INFO:teuthology.orchestra.run.smithi153.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-21T10:50:08.861 INFO:teuthology.orchestra.run.smithi153.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:50:08.861 INFO:teuthology.orchestra.run.smithi153.stdout:legacy client fscid: 1 2024-03-21T10:50:08.861 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:50:08.861 INFO:teuthology.orchestra.run.smithi153.stdout:Filesystem 'cephfs' (1) 2024-03-21T10:50:08.861 INFO:teuthology.orchestra.run.smithi153.stdout:fs_name cephfs 2024-03-21T10:50:08.861 INFO:teuthology.orchestra.run.smithi153.stdout:epoch 23 2024-03-21T10:50:08.861 INFO:teuthology.orchestra.run.smithi153.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-21T10:50:08.861 INFO:teuthology.orchestra.run.smithi153.stdout:created 2024-03-21T10:04:27.086576+0000 2024-03-21T10:50:08.861 INFO:teuthology.orchestra.run.smithi153.stdout:modified 2024-03-21T10:19:05.022092+0000 2024-03-21T10:50:08.861 INFO:teuthology.orchestra.run.smithi153.stdout:tableserver 0 2024-03-21T10:50:08.861 INFO:teuthology.orchestra.run.smithi153.stdout:root 0 2024-03-21T10:50:08.861 INFO:teuthology.orchestra.run.smithi153.stdout:session_timeout 60 2024-03-21T10:50:08.861 INFO:teuthology.orchestra.run.smithi153.stdout:session_autoclose 300 2024-03-21T10:50:08.862 INFO:teuthology.orchestra.run.smithi153.stdout:max_file_size 1099511627776 2024-03-21T10:50:08.862 INFO:teuthology.orchestra.run.smithi153.stdout:max_xattr_size 65536 2024-03-21T10:50:08.862 INFO:teuthology.orchestra.run.smithi153.stdout:required_client_features {} 2024-03-21T10:50:08.862 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure 0 2024-03-21T10:50:08.862 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure_osd_epoch 94 2024-03-21T10:50:08.862 INFO:teuthology.orchestra.run.smithi153.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:50:08.862 INFO:teuthology.orchestra.run.smithi153.stdout:max_mds 2 2024-03-21T10:50:08.862 INFO:teuthology.orchestra.run.smithi153.stdout:in 0,1 2024-03-21T10:50:08.862 INFO:teuthology.orchestra.run.smithi153.stdout:up {} 2024-03-21T10:50:08.862 INFO:teuthology.orchestra.run.smithi153.stdout:failed 0,1 2024-03-21T10:50:08.862 INFO:teuthology.orchestra.run.smithi153.stdout:damaged 2024-03-21T10:50:08.862 INFO:teuthology.orchestra.run.smithi153.stdout:stopped 2024-03-21T10:50:08.862 INFO:teuthology.orchestra.run.smithi153.stdout:data_pools [3] 2024-03-21T10:50:08.862 INFO:teuthology.orchestra.run.smithi153.stdout:metadata_pool 2 2024-03-21T10:50:08.863 INFO:teuthology.orchestra.run.smithi153.stdout:inline_data enabled 2024-03-21T10:50:08.863 INFO:teuthology.orchestra.run.smithi153.stdout:balancer 2024-03-21T10:50:08.863 INFO:teuthology.orchestra.run.smithi153.stdout:bal_rank_mask -1 2024-03-21T10:50:08.863 INFO:teuthology.orchestra.run.smithi153.stdout:standby_count_wanted 1 2024-03-21T10:50:08.863 INFO:teuthology.orchestra.run.smithi153.stdout:qdb_cluster leader: 0 members: 2024-03-21T10:50:08.863 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:50:08.863 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:50:08.863 INFO:teuthology.orchestra.run.smithi153.stdout:Standby daemons: 2024-03-21T10:50:08.863 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:50:08.863 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.zipckr{-1:34442} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6826/1103846295,v1:172.21.15.153:6827/1103846295] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:50:08.863 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.iuyluj{-1:34474} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.153:6828/1738630832,v1:172.21.15.153:6829/1738630832] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:50:08.863 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.yyimst{-1:44313} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.174:6824/4127786797,v1:172.21.15.174:6825/4127786797] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:50:08.863 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.nhozag{-1:44687} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.174:6826/2767517845,v1:172.21.15.174:6827/2767517845] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:50:08.864 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:08.867+0000 7ff47f800700 1 -- 172.21.15.153:0/4110842711 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff46006e460 msgr2=0x7ff460070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:50:08.864 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:08.867+0000 7ff47f800700 1 --2- 172.21.15.153:0/4110842711 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff46006e460 0x7ff460070920 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7ff464006010 tx=0x7ff46400b540 comp rx=0 tx=0).stop 2024-03-21T10:50:08.864 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:08.867+0000 7ff47f800700 1 -- 172.21.15.153:0/4110842711 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff4781071d0 msgr2=0x7ff47819ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:50:08.864 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:08.867+0000 7ff47f800700 1 --2- 172.21.15.153:0/4110842711 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff4781071d0 0x7ff47819ccb0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7ff46c00d8d0 tx=0x7ff46c00dc90 comp rx=0 tx=0).stop 2024-03-21T10:50:08.865 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:08.867+0000 7ff47f800700 1 -- 172.21.15.153:0/4110842711 shutdown_connections 2024-03-21T10:50:08.865 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:08.867+0000 7ff47f800700 1 --2- 172.21.15.153:0/4110842711 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff46006e460 0x7ff460070920 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:50:08.865 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:08.867+0000 7ff47f800700 1 --2- 172.21.15.153:0/4110842711 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff4781071d0 0x7ff47819ccb0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:50:08.865 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:08.867+0000 7ff47f800700 1 --2- 172.21.15.153:0/4110842711 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff47810d280 0x7ff47819d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:50:08.865 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:08.867+0000 7ff47f800700 1 -- 172.21.15.153:0/4110842711 >> 172.21.15.153:0/4110842711 conn(0x7ff4780769e0 msgr2=0x7ff478078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:50:08.865 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:08.867+0000 7ff47f800700 1 -- 172.21.15.153:0/4110842711 shutdown_connections 2024-03-21T10:50:08.865 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:08.867+0000 7ff47f800700 1 -- 172.21.15.153:0/4110842711 wait complete. 2024-03-21T10:50:08.865 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 32 2024-03-21T10:50:09.011 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:09.014+0000 7fda0d000700 1 -- 172.21.15.153:0/3984149503 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fda08068670 msgr2=0x7fda08068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:50:09.011 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:09.014+0000 7fda0d000700 1 --2- 172.21.15.153:0/3984149503 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fda08068670 0x7fda08068a50 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fd9f800b0a0 tx=0x7fd9f800b3b0 comp rx=0 tx=0).stop 2024-03-21T10:50:09.011 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:09.014+0000 7fda0d000700 1 -- 172.21.15.153:0/3984149503 shutdown_connections 2024-03-21T10:50:09.011 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:09.014+0000 7fda0d000700 1 --2- 172.21.15.153:0/3984149503 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fda08069020 0x7fda08105790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:50:09.011 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:09.014+0000 7fda0d000700 1 --2- 172.21.15.153:0/3984149503 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fda08068670 0x7fda08068a50 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:50:09.011 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:09.014+0000 7fda0d000700 1 -- 172.21.15.153:0/3984149503 >> 172.21.15.153:0/3984149503 conn(0x7fda080755a0 msgr2=0x7fda080759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:50:09.012 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:09.014+0000 7fda0d000700 1 -- 172.21.15.153:0/3984149503 shutdown_connections 2024-03-21T10:50:09.012 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:09.014+0000 7fda0d000700 1 -- 172.21.15.153:0/3984149503 wait complete. 2024-03-21T10:50:09.012 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:09.015+0000 7fda0d000700 1 Processor -- start 2024-03-21T10:50:09.013 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:09.015+0000 7fda0d000700 1 -- start start 2024-03-21T10:50:09.013 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:09.016+0000 7fda0d000700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fda08068670 0x7fda0819c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:50:09.013 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:09.016+0000 7fda0d000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fda08069020 0x7fda0819ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:50:09.013 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:09.016+0000 7fda0d000700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda0819d0e0 con 0x7fda08069020 2024-03-21T10:50:09.013 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:09.016+0000 7fda0d000700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda08196590 con 0x7fda08068670 2024-03-21T10:50:09.013 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:09.016+0000 7fd9fd400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fda08069020 0x7fda0819ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:50:09.013 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:09.016+0000 7fd9fd400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fda08069020 0x7fda0819ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:49276/0 (socket says 172.21.15.153:49276) 2024-03-21T10:50:09.013 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:09.016+0000 7fd9fd400700 1 -- 172.21.15.153:0/1857734878 learned_addr learned my addr 172.21.15.153:0/1857734878 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:50:09.014 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:09.016+0000 7fd9fd400700 1 -- 172.21.15.153:0/1857734878 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fda08068670 msgr2=0x7fda0819c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:50:09.014 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:09.016+0000 7fd9fd400700 1 --2- 172.21.15.153:0/1857734878 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fda08068670 0x7fda0819c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:50:09.014 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:09.016+0000 7fd9fd400700 1 -- 172.21.15.153:0/1857734878 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9f8009d00 con 0x7fda08069020 2024-03-21T10:50:09.014 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:09.017+0000 7fd9fd400700 1 --2- 172.21.15.153:0/1857734878 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fda08069020 0x7fda0819ca50 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fd9f000d8d0 tx=0x7fd9f000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:50:09.014 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:09.017+0000 7fd9ff400700 1 -- 172.21.15.153:0/1857734878 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd9f00041f0 con 0x7fda08069020 2024-03-21T10:50:09.014 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:09.017+0000 7fd9ff400700 1 -- 172.21.15.153:0/1857734878 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd9f0007620 con 0x7fda08069020 2024-03-21T10:50:09.015 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:09.017+0000 7fd9ff400700 1 -- 172.21.15.153:0/1857734878 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd9f0003c50 con 0x7fda08069020 2024-03-21T10:50:09.015 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:09.017+0000 7fda0d000700 1 -- 172.21.15.153:0/1857734878 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda08196870 con 0x7fda08069020 2024-03-21T10:50:09.015 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:09.017+0000 7fda0d000700 1 -- 172.21.15.153:0/1857734878 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda08196dc0 con 0x7fda08069020 2024-03-21T10:50:09.015 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:09.018+0000 7fda0d000700 1 -- 172.21.15.153:0/1857734878 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda08109130 con 0x7fda08069020 2024-03-21T10:50:09.016 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:09.019+0000 7fd9ff400700 1 -- 172.21.15.153:0/1857734878 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fd9f0004350 con 0x7fda08069020 2024-03-21T10:50:09.017 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:09.020+0000 7fd9ff400700 1 --2- 172.21.15.153:0/1857734878 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd9ec06e460 0x7fd9ec070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:50:09.017 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:09.020+0000 7fd9ff400700 1 -- 172.21.15.153:0/1857734878 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fd9f008dff0 con 0x7fda08069020 2024-03-21T10:50:09.017 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:09.021+0000 7fda05e00700 1 --2- 172.21.15.153:0/1857734878 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd9ec06e460 0x7fd9ec070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:50:09.018 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:09.021+0000 7fda05e00700 1 --2- 172.21.15.153:0/1857734878 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd9ec06e460 0x7fd9ec070920 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fd9f8009480 tx=0x7fd9f8006000 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:50:09.022 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:09.025+0000 7fd9ff400700 1 -- 172.21.15.153:0/1857734878 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd9f005ac60 con 0x7fda08069020 2024-03-21T10:50:09.253 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:09.256+0000 7fda0d000700 1 -- 172.21.15.153:0/1857734878 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fda08197110 con 0x7fd9ec06e460 2024-03-21T10:50:09.255 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:09.258+0000 7fd9ff400700 1 -- 172.21.15.153:0/1857734878 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fda08197110 con 0x7fd9ec06e460 2024-03-21T10:50:09.255 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:50:09.255 INFO:teuthology.orchestra.run.smithi153.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232", 2024-03-21T10:50:09.256 INFO:teuthology.orchestra.run.smithi153.stdout: "in_progress": true, 2024-03-21T10:50:09.256 INFO:teuthology.orchestra.run.smithi153.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-21T10:50:09.256 INFO:teuthology.orchestra.run.smithi153.stdout: "services_complete": [ 2024-03-21T10:50:09.256 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr", 2024-03-21T10:50:09.256 INFO:teuthology.orchestra.run.smithi153.stdout: "mon", 2024-03-21T10:50:09.256 INFO:teuthology.orchestra.run.smithi153.stdout: "mds", 2024-03-21T10:50:09.256 INFO:teuthology.orchestra.run.smithi153.stdout: "crash", 2024-03-21T10:50:09.256 INFO:teuthology.orchestra.run.smithi153.stdout: "osd" 2024-03-21T10:50:09.256 INFO:teuthology.orchestra.run.smithi153.stdout: ], 2024-03-21T10:50:09.256 INFO:teuthology.orchestra.run.smithi153.stdout: "progress": "16/23 daemons upgraded", 2024-03-21T10:50:09.256 INFO:teuthology.orchestra.run.smithi153.stdout: "message": "Currently upgrading mds daemons", 2024-03-21T10:50:09.256 INFO:teuthology.orchestra.run.smithi153.stdout: "is_paused": false 2024-03-21T10:50:09.256 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:50:09.258 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:09.261+0000 7fda0d000700 1 -- 172.21.15.153:0/1857734878 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd9ec06e460 msgr2=0x7fd9ec070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:50:09.258 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:09.261+0000 7fda0d000700 1 --2- 172.21.15.153:0/1857734878 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd9ec06e460 0x7fd9ec070920 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fd9f8009480 tx=0x7fd9f8006000 comp rx=0 tx=0).stop 2024-03-21T10:50:09.258 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:09.261+0000 7fda0d000700 1 -- 172.21.15.153:0/1857734878 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fda08069020 msgr2=0x7fda0819ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:50:09.258 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:09.261+0000 7fda0d000700 1 --2- 172.21.15.153:0/1857734878 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fda08069020 0x7fda0819ca50 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fd9f000d8d0 tx=0x7fd9f000dc90 comp rx=0 tx=0).stop 2024-03-21T10:50:09.261 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:09.261+0000 7fda0d000700 1 -- 172.21.15.153:0/1857734878 shutdown_connections 2024-03-21T10:50:09.261 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:09.261+0000 7fda0d000700 1 --2- 172.21.15.153:0/1857734878 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fda08068670 0x7fda0819c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:50:09.262 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:09.261+0000 7fda0d000700 1 --2- 172.21.15.153:0/1857734878 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd9ec06e460 0x7fd9ec070920 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:50:09.262 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:09.261+0000 7fda0d000700 1 --2- 172.21.15.153:0/1857734878 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fda08069020 0x7fda0819ca50 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:50:09.262 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:09.261+0000 7fda0d000700 1 -- 172.21.15.153:0/1857734878 >> 172.21.15.153:0/1857734878 conn(0x7fda080755a0 msgr2=0x7fda080fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:50:09.262 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:09.262+0000 7fda0d000700 1 -- 172.21.15.153:0/1857734878 shutdown_connections 2024-03-21T10:50:09.262 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:09.262+0000 7fda0d000700 1 -- 172.21.15.153:0/1857734878 wait complete. 2024-03-21T10:50:09.418 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:09.420+0000 7fb106800700 1 -- 172.21.15.153:0/4175068126 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb100073ae0 msgr2=0x7fb10010ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:50:09.418 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:09.420+0000 7fb106800700 1 --2- 172.21.15.153:0/4175068126 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb100073ae0 0x7fb10010ce40 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec009940 tx=0x7fb0ec009c50 comp rx=0 tx=0).stop 2024-03-21T10:50:09.418 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:09.421+0000 7fb106800700 1 -- 172.21.15.153:0/4175068126 shutdown_connections 2024-03-21T10:50:09.418 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:09.421+0000 7fb106800700 1 --2- 172.21.15.153:0/4175068126 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb100073ae0 0x7fb10010ce40 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:50:09.418 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:09.421+0000 7fb106800700 1 --2- 172.21.15.153:0/4175068126 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb1000731c0 0x7fb1000735a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:50:09.418 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:09.421+0000 7fb106800700 1 -- 172.21.15.153:0/4175068126 >> 172.21.15.153:0/4175068126 conn(0x7fb1000fc630 msgr2=0x7fb1000fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:50:09.419 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:09.421+0000 7fb106800700 1 -- 172.21.15.153:0/4175068126 shutdown_connections 2024-03-21T10:50:09.419 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:09.421+0000 7fb106800700 1 -- 172.21.15.153:0/4175068126 wait complete. 2024-03-21T10:50:09.420 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:09.422+0000 7fb106800700 1 Processor -- start 2024-03-21T10:50:09.420 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:09.422+0000 7fb106800700 1 -- start start 2024-03-21T10:50:09.420 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:09.422+0000 7fb106800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb1000731c0 0x7fb10019e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:50:09.420 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:09.422+0000 7fb106800700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb100073ae0 0x7fb10019ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:50:09.420 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:09.422+0000 7fb106800700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb10019f2c0 con 0x7fb1000731c0 2024-03-21T10:50:09.420 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:09.423+0000 7fb106800700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb100198770 con 0x7fb100073ae0 2024-03-21T10:50:09.420 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:09.423+0000 7fb0ff400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb1000731c0 0x7fb10019e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:50:09.421 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:09.423+0000 7fb0ff400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb1000731c0 0x7fb10019e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:49288/0 (socket says 172.21.15.153:49288) 2024-03-21T10:50:09.421 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:09.423+0000 7fb0ff400700 1 -- 172.21.15.153:0/998965847 learned_addr learned my addr 172.21.15.153:0/998965847 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:50:09.421 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:09.423+0000 7fb0fea00700 1 --2- 172.21.15.153:0/998965847 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb100073ae0 0x7fb10019ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:50:09.421 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:09.423+0000 7fb0ff400700 1 -- 172.21.15.153:0/998965847 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb100073ae0 msgr2=0x7fb10019ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:50:09.421 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:09.423+0000 7fb0ff400700 1 --2- 172.21.15.153:0/998965847 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb100073ae0 0x7fb10019ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:50:09.421 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:09.423+0000 7fb0ff400700 1 -- 172.21.15.153:0/998965847 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0ec0095f0 con 0x7fb1000731c0 2024-03-21T10:50:09.421 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:09.424+0000 7fb0ff400700 1 --2- 172.21.15.153:0/998965847 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb1000731c0 0x7fb10019e6f0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fb0f000e920 tx=0x7fb0f000ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:50:09.422 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:09.424+0000 7fb0f7e00700 1 -- 172.21.15.153:0/998965847 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb0f0009790 con 0x7fb1000731c0 2024-03-21T10:50:09.422 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:09.424+0000 7fb106800700 1 -- 172.21.15.153:0/998965847 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb100198a50 con 0x7fb1000731c0 2024-03-21T10:50:09.422 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:09.424+0000 7fb0f7e00700 1 -- 172.21.15.153:0/998965847 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb0f000cba0 con 0x7fb1000731c0 2024-03-21T10:50:09.422 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:09.424+0000 7fb0f7e00700 1 -- 172.21.15.153:0/998965847 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb0f0018650 con 0x7fb1000731c0 2024-03-21T10:50:09.422 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:09.424+0000 7fb106800700 1 -- 172.21.15.153:0/998965847 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb100198fa0 con 0x7fb1000731c0 2024-03-21T10:50:09.422 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:09.425+0000 7fb106800700 1 -- 172.21.15.153:0/998965847 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb10010a540 con 0x7fb1000731c0 2024-03-21T10:50:09.423 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:09.426+0000 7fb0f7e00700 1 -- 172.21.15.153:0/998965847 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fb0f000cd10 con 0x7fb1000731c0 2024-03-21T10:50:09.424 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:09.428+0000 7fb0f7e00700 1 --2- 172.21.15.153:0/998965847 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb0e406e460 0x7fb0e4070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:50:09.424 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:09.428+0000 7fb0f7e00700 1 -- 172.21.15.153:0/998965847 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fb0f0014070 con 0x7fb1000731c0 2024-03-21T10:50:09.425 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:09.428+0000 7fb0fea00700 1 --2- 172.21.15.153:0/998965847 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb0e406e460 0x7fb0e4070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:50:09.425 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:09.429+0000 7fb0fea00700 1 --2- 172.21.15.153:0/998965847 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb0e406e460 0x7fb0e4070920 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec006010 tx=0x7fb0ec0058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:50:09.429 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:09.433+0000 7fb0f7e00700 1 -- 172.21.15.153:0/998965847 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb0f0053740 con 0x7fb1000731c0 2024-03-21T10:50:09.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:09 smithi153 ceph-mon[113733]: from='client.36030 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:50:09.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:09 smithi153 ceph-mon[113733]: pgmap v1150: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:50:09.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:09 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/2757817294' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:50:09.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:09 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/4110842711' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:50:09.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:09 smithi174 ceph-mon[88004]: from='client.36030 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:50:09.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:09 smithi174 ceph-mon[88004]: pgmap v1150: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:50:09.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:09 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/2757817294' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:50:09.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:09 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/4110842711' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:50:09.720 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:09.723+0000 7fb106800700 1 -- 172.21.15.153:0/998965847 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fb10004ea90 con 0x7fb1000731c0 2024-03-21T10:50:09.720 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:09.723+0000 7fb0f7e00700 1 -- 172.21.15.153:0/998965847 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4493 (secure 0 0 0) 0x7fb0f005b230 con 0x7fb1000731c0 2024-03-21T10:50:09.721 INFO:teuthology.orchestra.run.smithi153.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-03-21T10:50:09.721 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-03-21T10:50:09.721 INFO:teuthology.orchestra.run.smithi153.stdout: host smithi174 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi174/config 2024-03-21T10:50:09.721 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/podman: timeout after 895 seconds 2024-03-21T10:50:09.721 INFO:teuthology.orchestra.run.smithi153.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 -e NODE_NAME=smithi174 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/run/ceph:z -v /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpd910r3_i:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 inventory --format=json-pretty --filter-for-batch 2024-03-21T10:50:09.721 INFO:teuthology.orchestra.run.smithi153.stdout:Traceback (most recent call last): 2024-03-21T10:50:09.721 INFO:teuthology.orchestra.run.smithi153.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-03-21T10:50:09.721 INFO:teuthology.orchestra.run.smithi153.stdout: return _run_code(code, main_globals, None, 2024-03-21T10:50:09.721 INFO:teuthology.orchestra.run.smithi153.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-03-21T10:50:09.721 INFO:teuthology.orchestra.run.smithi153.stdout: exec(code, run_globals) 2024-03-21T10:50:09.722 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 5606, in 2024-03-21T10:50:09.722 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 5594, in main 2024-03-21T10:50:09.722 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 406, in _infer_config 2024-03-21T10:50:09.722 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 321, in _infer_fsid 2024-03-21T10:50:09.722 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 434, in _infer_image 2024-03-21T10:50:09.722 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 308, in _validate_fsid 2024-03-21T10:50:09.722 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 3357, in command_ceph_volume 2024-03-21T10:50:09.722 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-03-21T10:50:09.722 INFO:teuthology.orchestra.run.smithi153.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 -e NODE_NAME=smithi174 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/run/ceph:z -v /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpd910r3_i:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 inventory --format=json-pretty --filter-for-batch 2024-03-21T10:50:09.722 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-21T10:50:09.722 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is degraded 2024-03-21T10:50:09.722 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-21T10:50:09.723 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-21T10:50:09.723 INFO:teuthology.orchestra.run.smithi153.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-21T10:50:09.723 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-21T10:50:09.723 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:09.726+0000 7fb106800700 1 -- 172.21.15.153:0/998965847 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb0e406e460 msgr2=0x7fb0e4070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:50:09.724 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:09.726+0000 7fb106800700 1 --2- 172.21.15.153:0/998965847 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb0e406e460 0x7fb0e4070920 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec006010 tx=0x7fb0ec0058e0 comp rx=0 tx=0).stop 2024-03-21T10:50:09.724 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:09.726+0000 7fb106800700 1 -- 172.21.15.153:0/998965847 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb1000731c0 msgr2=0x7fb10019e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:50:09.724 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:09.726+0000 7fb106800700 1 --2- 172.21.15.153:0/998965847 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb1000731c0 0x7fb10019e6f0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fb0f000e920 tx=0x7fb0f000ece0 comp rx=0 tx=0).stop 2024-03-21T10:50:09.724 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:09.726+0000 7fb106800700 1 -- 172.21.15.153:0/998965847 shutdown_connections 2024-03-21T10:50:09.724 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:09.726+0000 7fb106800700 1 --2- 172.21.15.153:0/998965847 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb0e406e460 0x7fb0e4070920 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:50:09.724 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:09.726+0000 7fb106800700 1 --2- 172.21.15.153:0/998965847 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb1000731c0 0x7fb10019e6f0 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:50:09.724 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:09.726+0000 7fb106800700 1 --2- 172.21.15.153:0/998965847 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb100073ae0 0x7fb10019ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:50:09.724 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:09.726+0000 7fb106800700 1 -- 172.21.15.153:0/998965847 >> 172.21.15.153:0/998965847 conn(0x7fb1000fc630 msgr2=0x7fb100107680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:50:09.724 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:09.727+0000 7fb106800700 1 -- 172.21.15.153:0/998965847 shutdown_connections 2024-03-21T10:50:09.724 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:09.727+0000 7fb106800700 1 -- 172.21.15.153:0/998965847 wait complete. 2024-03-21T10:50:10.696 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:10 smithi174 ceph-mon[88004]: from='client.36042 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:50:10.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:10 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/998965847' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:50:10.801 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:10 smithi153 ceph-mon[113733]: from='client.36042 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:50:10.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:10 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/998965847' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:50:11.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:11 smithi174 ceph-mon[88004]: pgmap v1151: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:50:11.767 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:11 smithi153 ceph-mon[113733]: pgmap v1151: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:50:13.696 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:13 smithi174 ceph-mon[88004]: pgmap v1152: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:50:13.801 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:13 smithi153 ceph-mon[113733]: pgmap v1152: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:50:15.696 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:15 smithi174 ceph-mon[88004]: pgmap v1153: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:50:15.801 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:15 smithi153 ceph-mon[113733]: pgmap v1153: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:50:16.801 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:16 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:50:16.946 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:16 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:50:17.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:17 smithi153 ceph-mon[113733]: pgmap v1154: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:50:17.946 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:17 smithi174 ceph-mon[88004]: pgmap v1154: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:50:19.801 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:19 smithi153 ceph-mon[113733]: pgmap v1155: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:50:19.946 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:19 smithi174 ceph-mon[88004]: pgmap v1155: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:50:21.801 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:21 smithi153 ceph-mon[113733]: pgmap v1156: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:50:21.946 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:21 smithi174 ceph-mon[88004]: pgmap v1156: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:50:23.696 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:23 smithi174 ceph-mon[88004]: pgmap v1157: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:50:23.801 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:23 smithi153 ceph-mon[113733]: pgmap v1157: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:50:24.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:24 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:50:24.803 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:24 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:50:24.803 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:24 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:50:24.803 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:24 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:50:24.803 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:24 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:50:24.803 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:24 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:50:24.803 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:24 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:50:24.803 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:24 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:50:24.803 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:24 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:50:24.803 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:24 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:50:24.803 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:24 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi153.iuyluj"}]: dispatch 2024-03-21T10:50:24.803 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:24 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi153.iuyluj"}]': finished 2024-03-21T10:50:24.804 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:24 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi153.zipckr"}]: dispatch 2024-03-21T10:50:24.804 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:24 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi153.zipckr"}]': finished 2024-03-21T10:50:24.804 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:24 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi174.nhozag"}]: dispatch 2024-03-21T10:50:24.804 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:24 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi174.nhozag"}]': finished 2024-03-21T10:50:24.804 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:24 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi174.yyimst"}]: dispatch 2024-03-21T10:50:24.804 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:24 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi174.yyimst"}]': finished 2024-03-21T10:50:24.804 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:24 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-03-21T10:50:24.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:24 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:50:24.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:24 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:50:24.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:24 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:50:24.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:24 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:50:24.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:24 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:50:24.948 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:24 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:50:24.948 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:24 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:50:24.948 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:24 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:50:24.948 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:24 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:50:24.948 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:24 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:50:24.948 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:24 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi153.iuyluj"}]: dispatch 2024-03-21T10:50:24.948 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:24 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi153.iuyluj"}]': finished 2024-03-21T10:50:24.948 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:24 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi153.zipckr"}]: dispatch 2024-03-21T10:50:24.948 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:24 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi153.zipckr"}]': finished 2024-03-21T10:50:24.948 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:24 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi174.nhozag"}]: dispatch 2024-03-21T10:50:24.948 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:24 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi174.nhozag"}]': finished 2024-03-21T10:50:24.948 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:24 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi174.yyimst"}]: dispatch 2024-03-21T10:50:24.948 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:24 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi174.yyimst"}]': finished 2024-03-21T10:50:24.948 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:24 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-03-21T10:50:25.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:25 smithi174 ceph-mon[88004]: pgmap v1158: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:50:25.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:25 smithi174 ceph-mon[88004]: pgmap v1159: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:50:25.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:25 smithi174 ceph-mon[88004]: Upgrade: Setting container_image for all mds 2024-03-21T10:50:25.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:25 smithi174 ceph-mon[88004]: Upgrade: Setting filesystem cephfs Joinable 2024-03-21T10:50:25.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:25 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-03-21T10:50:25.698 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:25 smithi174 ceph-mon[88004]: Standby daemon mds.cephfs.smithi153.zipckr assigned to filesystem cephfs as rank 0 2024-03-21T10:50:25.698 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:25 smithi174 ceph-mon[88004]: Standby daemon mds.cephfs.smithi153.iuyluj assigned to filesystem cephfs as rank 1 2024-03-21T10:50:25.698 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:25 smithi174 ceph-mon[88004]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-21T10:50:25.698 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:25 smithi174 ceph-mon[88004]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-03-21T10:50:25.698 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:25 smithi174 ceph-mon[88004]: fsmap cephfs:2/2 {0=cephfs.smithi153.zipckr=up:replay,1=cephfs.smithi153.iuyluj=up:replay} 2 up:standby 2024-03-21T10:50:25.698 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:25 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]: dispatch 2024-03-21T10:50:25.801 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:25 smithi153 ceph-mon[113733]: pgmap v1158: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:50:25.801 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:25 smithi153 ceph-mon[113733]: pgmap v1159: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-03-21T10:50:25.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:25 smithi153 ceph-mon[113733]: Upgrade: Setting container_image for all mds 2024-03-21T10:50:25.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:25 smithi153 ceph-mon[113733]: Upgrade: Setting filesystem cephfs Joinable 2024-03-21T10:50:25.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:25 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-03-21T10:50:25.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:25 smithi153 ceph-mon[113733]: Standby daemon mds.cephfs.smithi153.zipckr assigned to filesystem cephfs as rank 0 2024-03-21T10:50:25.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:25 smithi153 ceph-mon[113733]: Standby daemon mds.cephfs.smithi153.iuyluj assigned to filesystem cephfs as rank 1 2024-03-21T10:50:25.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:25 smithi153 ceph-mon[113733]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-21T10:50:25.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:25 smithi153 ceph-mon[113733]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-03-21T10:50:25.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:25 smithi153 ceph-mon[113733]: fsmap cephfs:2/2 {0=cephfs.smithi153.zipckr=up:replay,1=cephfs.smithi153.iuyluj=up:replay} 2 up:standby 2024-03-21T10:50:25.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:25 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]: dispatch 2024-03-21T10:50:26.801 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:26 smithi153 ceph-mon[113733]: Upgrade: Enabling allow_standby_replay on filesystem cephfs 2024-03-21T10:50:26.801 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:26 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]': finished 2024-03-21T10:50:26.801 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:26 smithi153 ceph-mon[113733]: fsmap cephfs:2/2 {0=cephfs.smithi153.zipckr=up:replay,1=cephfs.smithi153.iuyluj=up:replay} 2 up:standby 2024-03-21T10:50:26.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:26 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:50:26.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:26 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:50:26.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:26 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:50:26.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:26 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:50:26.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:26 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:50:26.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:26 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:50:26.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:26 smithi174 ceph-mon[88004]: Upgrade: Enabling allow_standby_replay on filesystem cephfs 2024-03-21T10:50:26.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:26 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]': finished 2024-03-21T10:50:26.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:26 smithi174 ceph-mon[88004]: fsmap cephfs:2/2 {0=cephfs.smithi153.zipckr=up:replay,1=cephfs.smithi153.iuyluj=up:replay} 2 up:standby 2024-03-21T10:50:26.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:26 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:50:26.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:26 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:50:26.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:26 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:50:26.948 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:26 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:50:26.948 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:26 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:50:26.948 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:26 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:50:27.496 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:27 smithi153 ceph-mon[113733]: pgmap v1160: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 4.5 MiB/s rd, 2 op/s 2024-03-21T10:50:27.496 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:27 smithi153 ceph-mon[113733]: Upgrade: Setting container_image for all rgw 2024-03-21T10:50:27.496 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:27 smithi153 ceph-mon[113733]: Upgrade: Setting container_image for all rbd-mirror 2024-03-21T10:50:27.946 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:27 smithi174 ceph-mon[88004]: pgmap v1160: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 4.5 MiB/s rd, 2 op/s 2024-03-21T10:50:27.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:27 smithi174 ceph-mon[88004]: Upgrade: Setting container_image for all rgw 2024-03-21T10:50:27.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:27 smithi174 ceph-mon[88004]: Upgrade: Setting container_image for all rbd-mirror 2024-03-21T10:50:29.801 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:29 smithi153 ceph-mon[113733]: pgmap v1161: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 9.8 MiB/s rd, 4 op/s 2024-03-21T10:50:29.801 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:29 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:50:29.801 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:29 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi153", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-21T10:50:29.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:29 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:50:29.946 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:29 smithi174 ceph-mon[88004]: pgmap v1161: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 9.8 MiB/s rd, 4 op/s 2024-03-21T10:50:29.946 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:29 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:50:29.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:29 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi153", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-21T10:50:29.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:29 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:50:30.696 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:30 smithi153 ceph-mon[113733]: Upgrade: Updating ceph-exporter.smithi153 (1/2) 2024-03-21T10:50:30.696 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:30 smithi153 ceph-mon[113733]: Deploying daemon ceph-exporter.smithi153 on smithi153 2024-03-21T10:50:30.696 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:30 smithi174 ceph-mon[88004]: Upgrade: Updating ceph-exporter.smithi153 (1/2) 2024-03-21T10:50:30.696 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:30 smithi174 ceph-mon[88004]: Deploying daemon ceph-exporter.smithi153 on smithi153 2024-03-21T10:50:31.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:31 smithi153 ceph-mon[113733]: pgmap v1162: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 16 MiB/s rd, 6 op/s 2024-03-21T10:50:31.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:31 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:50:31.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:31 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:50:31.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:31 smithi174 ceph-mon[88004]: pgmap v1162: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 16 MiB/s rd, 6 op/s 2024-03-21T10:50:31.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:31 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:50:31.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:31 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:50:33.801 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:33 smithi153 ceph-mon[113733]: pgmap v1163: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 28 MiB/s rd, 9 op/s 2024-03-21T10:50:33.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:33 smithi153 ceph-mon[113733]: mds.? [v2:172.21.15.153:6828/1738630832,v1:172.21.15.153:6829/1738630832] up:resolve 2024-03-21T10:50:33.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:33 smithi153 ceph-mon[113733]: fsmap cephfs:2/2 {0=cephfs.smithi153.zipckr=up:replay,1=cephfs.smithi153.iuyluj=up:resolve} 2 up:standby 2024-03-21T10:50:33.946 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:33 smithi174 ceph-mon[88004]: pgmap v1163: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 28 MiB/s rd, 9 op/s 2024-03-21T10:50:33.946 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:33 smithi174 ceph-mon[88004]: mds.? [v2:172.21.15.153:6828/1738630832,v1:172.21.15.153:6829/1738630832] up:resolve 2024-03-21T10:50:33.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:33 smithi174 ceph-mon[88004]: fsmap cephfs:2/2 {0=cephfs.smithi153.zipckr=up:replay,1=cephfs.smithi153.iuyluj=up:resolve} 2 up:standby 2024-03-21T10:50:34.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:34 smithi153 ceph-mon[113733]: mds.? [v2:172.21.15.153:6826/1103846295,v1:172.21.15.153:6827/1103846295] up:resolve 2024-03-21T10:50:34.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:34 smithi153 ceph-mon[113733]: fsmap cephfs:2/2 {0=cephfs.smithi153.zipckr=up:resolve,1=cephfs.smithi153.iuyluj=up:resolve} 2 up:standby 2024-03-21T10:50:34.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:34 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:50:34.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:34 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:50:34.946 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:34 smithi174 ceph-mon[88004]: mds.? [v2:172.21.15.153:6826/1103846295,v1:172.21.15.153:6827/1103846295] up:resolve 2024-03-21T10:50:34.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:34 smithi174 ceph-mon[88004]: fsmap cephfs:2/2 {0=cephfs.smithi153.zipckr=up:resolve,1=cephfs.smithi153.iuyluj=up:resolve} 2 up:standby 2024-03-21T10:50:34.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:34 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:50:34.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:34 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:50:35.580 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:35 smithi174 ceph-mon[88004]: pgmap v1164: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 30 MiB/s rd, 9 op/s 2024-03-21T10:50:35.580 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:35 smithi174 ceph-mon[88004]: reconnect by client.24319 192.168.0.1:0/482977964 after 0.00399993 2024-03-21T10:50:35.580 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:35 smithi174 ceph-mon[88004]: reconnect by client.24319 192.168.0.1:0/482977964 after 0.00299994 2024-03-21T10:50:35.580 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:35 smithi174 ceph-mon[88004]: reconnect by client.14538 192.168.0.1:0/2069638377 after 0.00499991 2024-03-21T10:50:35.581 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:35 smithi174 ceph-mon[88004]: reconnect by client.14538 192.168.0.1:0/2069638377 after 0.00499991 2024-03-21T10:50:35.581 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:35 smithi174 ceph-mon[88004]: mds.? [v2:172.21.15.153:6826/1103846295,v1:172.21.15.153:6827/1103846295] up:reconnect 2024-03-21T10:50:35.581 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:35 smithi174 ceph-mon[88004]: mds.? [v2:172.21.15.153:6828/1738630832,v1:172.21.15.153:6829/1738630832] up:reconnect 2024-03-21T10:50:35.581 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:35 smithi174 ceph-mon[88004]: fsmap cephfs:2/2 {0=cephfs.smithi153.zipckr=up:reconnect,1=cephfs.smithi153.iuyluj=up:reconnect} 2 up:standby 2024-03-21T10:50:35.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:35 smithi153 ceph-mon[113733]: pgmap v1164: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 30 MiB/s rd, 9 op/s 2024-03-21T10:50:35.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:35 smithi153 ceph-mon[113733]: reconnect by client.24319 192.168.0.1:0/482977964 after 0.00399993 2024-03-21T10:50:35.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:35 smithi153 ceph-mon[113733]: reconnect by client.24319 192.168.0.1:0/482977964 after 0.00299994 2024-03-21T10:50:35.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:35 smithi153 ceph-mon[113733]: reconnect by client.14538 192.168.0.1:0/2069638377 after 0.00499991 2024-03-21T10:50:35.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:35 smithi153 ceph-mon[113733]: reconnect by client.14538 192.168.0.1:0/2069638377 after 0.00499991 2024-03-21T10:50:35.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:35 smithi153 ceph-mon[113733]: mds.? [v2:172.21.15.153:6826/1103846295,v1:172.21.15.153:6827/1103846295] up:reconnect 2024-03-21T10:50:35.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:35 smithi153 ceph-mon[113733]: mds.? [v2:172.21.15.153:6828/1738630832,v1:172.21.15.153:6829/1738630832] up:reconnect 2024-03-21T10:50:35.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:35 smithi153 ceph-mon[113733]: fsmap cephfs:2/2 {0=cephfs.smithi153.zipckr=up:reconnect,1=cephfs.smithi153.iuyluj=up:reconnect} 2 up:standby 2024-03-21T10:50:36.801 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:36 smithi153 ceph-mon[113733]: mds.? [v2:172.21.15.153:6826/1103846295,v1:172.21.15.153:6827/1103846295] up:rejoin 2024-03-21T10:50:36.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:36 smithi153 ceph-mon[113733]: mds.? [v2:172.21.15.153:6828/1738630832,v1:172.21.15.153:6829/1738630832] up:rejoin 2024-03-21T10:50:36.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:36 smithi153 ceph-mon[113733]: fsmap cephfs:2/2 {0=cephfs.smithi153.zipckr=up:rejoin,1=cephfs.smithi153.iuyluj=up:rejoin} 2 up:standby 2024-03-21T10:50:36.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:36 smithi153 ceph-mon[113733]: daemon mds.cephfs.smithi153.zipckr is now active in filesystem cephfs as rank 0 2024-03-21T10:50:36.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:36 smithi153 ceph-mon[113733]: daemon mds.cephfs.smithi153.iuyluj is now active in filesystem cephfs as rank 1 2024-03-21T10:50:36.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:36 smithi153 ceph-mon[113733]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-21T10:50:36.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:36 smithi153 ceph-mon[113733]: mds.? [v2:172.21.15.153:6826/1103846295,v1:172.21.15.153:6827/1103846295] up:active 2024-03-21T10:50:36.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:36 smithi153 ceph-mon[113733]: mds.? [v2:172.21.15.153:6828/1738630832,v1:172.21.15.153:6829/1738630832] up:active 2024-03-21T10:50:36.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:36 smithi153 ceph-mon[113733]: fsmap cephfs:2 {0=cephfs.smithi153.zipckr=up:active,1=cephfs.smithi153.iuyluj=up:active} 2 up:standby-replay 2024-03-21T10:50:36.806 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:36 smithi174 ceph-mon[88004]: mds.? [v2:172.21.15.153:6826/1103846295,v1:172.21.15.153:6827/1103846295] up:rejoin 2024-03-21T10:50:36.806 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:36 smithi174 ceph-mon[88004]: mds.? [v2:172.21.15.153:6828/1738630832,v1:172.21.15.153:6829/1738630832] up:rejoin 2024-03-21T10:50:36.806 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:36 smithi174 ceph-mon[88004]: fsmap cephfs:2/2 {0=cephfs.smithi153.zipckr=up:rejoin,1=cephfs.smithi153.iuyluj=up:rejoin} 2 up:standby 2024-03-21T10:50:36.806 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:36 smithi174 ceph-mon[88004]: daemon mds.cephfs.smithi153.zipckr is now active in filesystem cephfs as rank 0 2024-03-21T10:50:36.806 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:36 smithi174 ceph-mon[88004]: daemon mds.cephfs.smithi153.iuyluj is now active in filesystem cephfs as rank 1 2024-03-21T10:50:36.807 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:36 smithi174 ceph-mon[88004]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-21T10:50:36.807 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:36 smithi174 ceph-mon[88004]: mds.? [v2:172.21.15.153:6826/1103846295,v1:172.21.15.153:6827/1103846295] up:active 2024-03-21T10:50:36.807 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:36 smithi174 ceph-mon[88004]: mds.? [v2:172.21.15.153:6828/1738630832,v1:172.21.15.153:6829/1738630832] up:active 2024-03-21T10:50:36.807 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:36 smithi174 ceph-mon[88004]: fsmap cephfs:2 {0=cephfs.smithi153.zipckr=up:active,1=cephfs.smithi153.iuyluj=up:active} 2 up:standby-replay 2024-03-21T10:50:37.918 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:37 smithi174 ceph-mon[88004]: pgmap v1165: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 26 MiB/s rd, 85 B/s wr, 8 op/s 2024-03-21T10:50:37.918 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:37 smithi174 ceph-mon[88004]: Upgrade: Updating ceph-exporter.smithi174 (2/2) 2024-03-21T10:50:37.918 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:37 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:50:37.918 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:37 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi174", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-21T10:50:37.918 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:37 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:50:38.051 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:37 smithi153 ceph-mon[113733]: pgmap v1165: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 26 MiB/s rd, 85 B/s wr, 8 op/s 2024-03-21T10:50:38.052 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:37 smithi153 ceph-mon[113733]: Upgrade: Updating ceph-exporter.smithi174 (2/2) 2024-03-21T10:50:38.052 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:37 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:50:38.052 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:37 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi174", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-21T10:50:38.052 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:37 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:50:38.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:38 smithi174 ceph-mon[88004]: Deploying daemon ceph-exporter.smithi174 on smithi174 2024-03-21T10:50:38.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:38 smithi174 ceph-mon[88004]: pgmap v1166: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 26 MiB/s rd, 85 B/s wr, 9 op/s 2024-03-21T10:50:39.051 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:38 smithi153 ceph-mon[113733]: Deploying daemon ceph-exporter.smithi174 on smithi174 2024-03-21T10:50:39.051 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:38 smithi153 ceph-mon[113733]: pgmap v1166: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 26 MiB/s rd, 85 B/s wr, 9 op/s 2024-03-21T10:50:39.901 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:39.902+0000 7fd1ab200700 1 -- 172.21.15.153:0/4197491169 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd1a4102ff0 msgr2=0x7fd1a41033d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:50:39.901 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:39.902+0000 7fd1ab200700 1 --2- 172.21.15.153:0/4197491169 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd1a4102ff0 0x7fd1a41033d0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fd194009910 tx=0x7fd194009c20 comp rx=0 tx=0).stop 2024-03-21T10:50:39.901 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:39.902+0000 7fd1ab200700 1 -- 172.21.15.153:0/4197491169 shutdown_connections 2024-03-21T10:50:39.901 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:39.902+0000 7fd1ab200700 1 --2- 172.21.15.153:0/4197491169 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd1a41039a0 0x7fd1a41079f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:50:39.901 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:39.902+0000 7fd1ab200700 1 --2- 172.21.15.153:0/4197491169 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd1a4102ff0 0x7fd1a41033d0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:50:39.901 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:39.902+0000 7fd1ab200700 1 -- 172.21.15.153:0/4197491169 >> 172.21.15.153:0/4197491169 conn(0x7fd1a40fe880 msgr2=0x7fd1a4100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:50:39.901 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:39.902+0000 7fd1ab200700 1 -- 172.21.15.153:0/4197491169 shutdown_connections 2024-03-21T10:50:39.901 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:39.902+0000 7fd1ab200700 1 -- 172.21.15.153:0/4197491169 wait complete. 2024-03-21T10:50:39.901 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:39.903+0000 7fd1ab200700 1 Processor -- start 2024-03-21T10:50:39.901 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:39.903+0000 7fd1ab200700 1 -- start start 2024-03-21T10:50:39.901 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:39.904+0000 7fd1ab200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd1a4102ff0 0x7fd1a4198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:50:39.902 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:39.904+0000 7fd1ab200700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd1a41039a0 0x7fd1a4198eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:50:39.902 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:39.904+0000 7fd1ab200700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a4199540 con 0x7fd1a4102ff0 2024-03-21T10:50:39.902 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:39.904+0000 7fd1ab200700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a419d350 con 0x7fd1a41039a0 2024-03-21T10:50:39.902 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:39.904+0000 7fd1a3e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd1a4102ff0 0x7fd1a4198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:50:39.902 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:39.904+0000 7fd1a3e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd1a4102ff0 0x7fd1a4198970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39782/0 (socket says 172.21.15.153:39782) 2024-03-21T10:50:39.902 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:39.904+0000 7fd1a3e00700 1 -- 172.21.15.153:0/253852099 learned_addr learned my addr 172.21.15.153:0/253852099 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:50:39.902 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:39.904+0000 7fd1a3400700 1 --2- 172.21.15.153:0/253852099 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd1a41039a0 0x7fd1a4198eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:50:39.903 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:39.905+0000 7fd1a3e00700 1 -- 172.21.15.153:0/253852099 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd1a41039a0 msgr2=0x7fd1a4198eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:50:39.903 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:39.905+0000 7fd1a3e00700 1 --2- 172.21.15.153:0/253852099 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd1a41039a0 0x7fd1a4198eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:50:39.903 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:39.905+0000 7fd1a3e00700 1 -- 172.21.15.153:0/253852099 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1940095f0 con 0x7fd1a4102ff0 2024-03-21T10:50:39.903 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:39.905+0000 7fd1a3e00700 1 --2- 172.21.15.153:0/253852099 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd1a4102ff0 0x7fd1a4198970 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fd1940092e0 tx=0x7fd19400b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:50:39.903 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:39.905+0000 7fd1a0c00700 1 -- 172.21.15.153:0/253852099 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd19401d070 con 0x7fd1a4102ff0 2024-03-21T10:50:39.903 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:39.905+0000 7fd1a0c00700 1 -- 172.21.15.153:0/253852099 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd1940052a0 con 0x7fd1a4102ff0 2024-03-21T10:50:39.903 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:39.905+0000 7fd1ab200700 1 -- 172.21.15.153:0/253852099 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1a419d5d0 con 0x7fd1a4102ff0 2024-03-21T10:50:39.903 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:39.905+0000 7fd1a0c00700 1 -- 172.21.15.153:0/253852099 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd194017400 con 0x7fd1a4102ff0 2024-03-21T10:50:39.904 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:39.905+0000 7fd1ab200700 1 -- 172.21.15.153:0/253852099 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1a419dac0 con 0x7fd1a4102ff0 2024-03-21T10:50:39.904 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:39.907+0000 7fd1ab200700 1 -- 172.21.15.153:0/253852099 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1a410b390 con 0x7fd1a4102ff0 2024-03-21T10:50:39.906 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:39.909+0000 7fd1a0c00700 1 -- 172.21.15.153:0/253852099 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fd19400f820 con 0x7fd1a4102ff0 2024-03-21T10:50:39.907 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:39.910+0000 7fd1a0c00700 1 --2- 172.21.15.153:0/253852099 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd18806e460 0x7fd188070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:50:39.907 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:39.911+0000 7fd1a0c00700 1 -- 172.21.15.153:0/253852099 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fd19408f900 con 0x7fd1a4102ff0 2024-03-21T10:50:39.908 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:39.911+0000 7fd1a3400700 1 --2- 172.21.15.153:0/253852099 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd18806e460 0x7fd188070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:50:39.909 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:39.912+0000 7fd1a3400700 1 --2- 172.21.15.153:0/253852099 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd18806e460 0x7fd188070920 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fd1a4199f90 tx=0x7fd190009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:50:39.912 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:39.916+0000 7fd1a0c00700 1 -- 172.21.15.153:0/253852099 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd19405c450 con 0x7fd1a4102ff0 2024-03-21T10:50:40.160 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:40.163+0000 7fd1ab200700 1 -- 172.21.15.153:0/253852099 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd1a402cd30 con 0x7fd18806e460 2024-03-21T10:50:40.161 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:40.165+0000 7fd1a0c00700 1 -- 172.21.15.153:0/253852099 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+441 (secure 0 0 0) 0x7fd1a402cd30 con 0x7fd18806e460 2024-03-21T10:50:40.165 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:40.168+0000 7fd1ab200700 1 -- 172.21.15.153:0/253852099 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd18806e460 msgr2=0x7fd188070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:50:40.165 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:40.168+0000 7fd1ab200700 1 --2- 172.21.15.153:0/253852099 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd18806e460 0x7fd188070920 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fd1a4199f90 tx=0x7fd190009260 comp rx=0 tx=0).stop 2024-03-21T10:50:40.165 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:40.168+0000 7fd1ab200700 1 -- 172.21.15.153:0/253852099 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd1a4102ff0 msgr2=0x7fd1a4198970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:50:40.165 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:40.168+0000 7fd1ab200700 1 --2- 172.21.15.153:0/253852099 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd1a4102ff0 0x7fd1a4198970 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fd1940092e0 tx=0x7fd19400b6c0 comp rx=0 tx=0).stop 2024-03-21T10:50:40.165 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:40.168+0000 7fd1ab200700 1 -- 172.21.15.153:0/253852099 shutdown_connections 2024-03-21T10:50:40.165 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:40.168+0000 7fd1ab200700 1 --2- 172.21.15.153:0/253852099 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fd18806e460 0x7fd188070920 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:50:40.166 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:40.168+0000 7fd1ab200700 1 --2- 172.21.15.153:0/253852099 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fd1a4102ff0 0x7fd1a4198970 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:50:40.166 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:40.168+0000 7fd1ab200700 1 --2- 172.21.15.153:0/253852099 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd1a41039a0 0x7fd1a4198eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:50:40.166 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:40.168+0000 7fd1ab200700 1 -- 172.21.15.153:0/253852099 >> 172.21.15.153:0/253852099 conn(0x7fd1a40fe880 msgr2=0x7fd1a40fff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:50:40.166 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:40.169+0000 7fd1ab200700 1 -- 172.21.15.153:0/253852099 shutdown_connections 2024-03-21T10:50:40.166 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:40.169+0000 7fd1ab200700 1 -- 172.21.15.153:0/253852099 wait complete. 2024-03-21T10:50:40.186 INFO:teuthology.orchestra.run.smithi153.stdout:true 2024-03-21T10:50:40.365 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:40.368+0000 7f729f400700 1 -- 172.21.15.153:0/2594559254 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7298101d00 msgr2=0x7f72981020e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:50:40.366 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:40.368+0000 7f729f400700 1 --2- 172.21.15.153:0/2594559254 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7298101d00 0x7f72981020e0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f728c009910 tx=0x7f728c009c20 comp rx=0 tx=0).stop 2024-03-21T10:50:40.366 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:40.368+0000 7f729f400700 1 -- 172.21.15.153:0/2594559254 shutdown_connections 2024-03-21T10:50:40.366 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:40.368+0000 7f729f400700 1 --2- 172.21.15.153:0/2594559254 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7298102620 0x7f729810ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:50:40.366 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:40.368+0000 7f729f400700 1 --2- 172.21.15.153:0/2594559254 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7298101d00 0x7f72981020e0 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:50:40.366 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:40.368+0000 7f729f400700 1 -- 172.21.15.153:0/2594559254 >> 172.21.15.153:0/2594559254 conn(0x7f72980fb570 msgr2=0x7f72980fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:50:40.366 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:40.368+0000 7f729f400700 1 -- 172.21.15.153:0/2594559254 shutdown_connections 2024-03-21T10:50:40.366 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:40.368+0000 7f729f400700 1 -- 172.21.15.153:0/2594559254 wait complete. 2024-03-21T10:50:40.367 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:40.370+0000 7f729f400700 1 Processor -- start 2024-03-21T10:50:40.367 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:40.370+0000 7f729f400700 1 -- start start 2024-03-21T10:50:40.367 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:40.370+0000 7f729f400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7298101d00 0x7f729819e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:50:40.367 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:40.370+0000 7f729f400700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7298102620 0x7f729819ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:50:40.368 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:40.370+0000 7f729f400700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f729819f290 con 0x7f7298101d00 2024-03-21T10:50:40.368 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:40.370+0000 7f729f400700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7298198740 con 0x7f7298102620 2024-03-21T10:50:40.368 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:40.371+0000 7f729ca00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7298101d00 0x7f729819e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:50:40.368 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:40.371+0000 7f729ca00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7298101d00 0x7f729819e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39806/0 (socket says 172.21.15.153:39806) 2024-03-21T10:50:40.368 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:40.371+0000 7f729ca00700 1 -- 172.21.15.153:0/137976045 learned_addr learned my addr 172.21.15.153:0/137976045 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:50:40.368 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:40.371+0000 7f7297e00700 1 --2- 172.21.15.153:0/137976045 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7298102620 0x7f729819ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:50:40.369 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:40.371+0000 7f729ca00700 1 -- 172.21.15.153:0/137976045 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7298102620 msgr2=0x7f729819ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:50:40.369 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:40.371+0000 7f729ca00700 1 --2- 172.21.15.153:0/137976045 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7298102620 0x7f729819ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:50:40.369 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:40.371+0000 7f729ca00700 1 -- 172.21.15.153:0/137976045 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f728c0095f0 con 0x7f7298101d00 2024-03-21T10:50:40.369 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:40.372+0000 7f729ca00700 1 --2- 172.21.15.153:0/137976045 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7298101d00 0x7f729819e6c0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f728c006010 tx=0x7f728c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:50:40.369 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:40.372+0000 7f7295600700 1 -- 172.21.15.153:0/137976045 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f728c01d070 con 0x7f7298101d00 2024-03-21T10:50:40.369 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:40.372+0000 7f7295600700 1 -- 172.21.15.153:0/137976045 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f728c005dc0 con 0x7f7298101d00 2024-03-21T10:50:40.369 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:40.372+0000 7f729f400700 1 -- 172.21.15.153:0/137976045 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72981989c0 con 0x7f7298101d00 2024-03-21T10:50:40.369 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:40.372+0000 7f7295600700 1 -- 172.21.15.153:0/137976045 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f728c017400 con 0x7f7298101d00 2024-03-21T10:50:40.370 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:40.372+0000 7f729f400700 1 -- 172.21.15.153:0/137976045 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7298198eb0 con 0x7f7298101d00 2024-03-21T10:50:40.370 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:40.373+0000 7f729f400700 1 -- 172.21.15.153:0/137976045 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f729810a670 con 0x7f7298101d00 2024-03-21T10:50:40.371 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:40.375+0000 7f7295600700 1 -- 172.21.15.153:0/137976045 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f728c003a60 con 0x7f7298101d00 2024-03-21T10:50:40.373 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:40.376+0000 7f7295600700 1 --2- 172.21.15.153:0/137976045 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f728006e460 0x7f7280070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:50:40.373 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:40.376+0000 7f7295600700 1 -- 172.21.15.153:0/137976045 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f728c08f940 con 0x7f7298101d00 2024-03-21T10:50:40.373 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:40.376+0000 7f7297e00700 1 --2- 172.21.15.153:0/137976045 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f728006e460 0x7f7280070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:50:40.374 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:40.377+0000 7f7297e00700 1 --2- 172.21.15.153:0/137976045 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f728006e460 0x7f7280070920 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f7298199bb0 tx=0x7f728400b360 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:50:40.379 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:40.383+0000 7f7295600700 1 -- 172.21.15.153:0/137976045 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f728c05c530 con 0x7f7298101d00 2024-03-21T10:50:40.691 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:40.694+0000 7f729f400700 1 -- 172.21.15.153:0/137976045 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f729802cd60 con 0x7f728006e460 2024-03-21T10:50:40.693 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:40.696+0000 7f7295600700 1 -- 172.21.15.153:0/137976045 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+441 (secure 0 0 0) 0x7f729802cd60 con 0x7f728006e460 2024-03-21T10:50:40.697 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:40.700+0000 7f729f400700 1 -- 172.21.15.153:0/137976045 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f728006e460 msgr2=0x7f7280070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:50:40.697 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:40.700+0000 7f729f400700 1 --2- 172.21.15.153:0/137976045 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f728006e460 0x7f7280070920 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f7298199bb0 tx=0x7f728400b360 comp rx=0 tx=0).stop 2024-03-21T10:50:40.697 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:40.700+0000 7f729f400700 1 -- 172.21.15.153:0/137976045 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7298101d00 msgr2=0x7f729819e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:50:40.697 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:40.700+0000 7f729f400700 1 --2- 172.21.15.153:0/137976045 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7298101d00 0x7f729819e6c0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f728c006010 tx=0x7f728c00bd70 comp rx=0 tx=0).stop 2024-03-21T10:50:40.697 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:40.700+0000 7f729f400700 1 -- 172.21.15.153:0/137976045 shutdown_connections 2024-03-21T10:50:40.697 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:40.700+0000 7f729f400700 1 --2- 172.21.15.153:0/137976045 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f728006e460 0x7f7280070920 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:50:40.697 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:40.700+0000 7f729f400700 1 --2- 172.21.15.153:0/137976045 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7298101d00 0x7f729819e6c0 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:50:40.698 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:40.700+0000 7f729f400700 1 --2- 172.21.15.153:0/137976045 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7298102620 0x7f729819ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:50:40.698 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:40.700+0000 7f729f400700 1 -- 172.21.15.153:0/137976045 >> 172.21.15.153:0/137976045 conn(0x7f72980fb570 msgr2=0x7f7298100120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:50:40.698 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:40.700+0000 7f729f400700 1 -- 172.21.15.153:0/137976045 shutdown_connections 2024-03-21T10:50:40.698 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:40.701+0000 7f729f400700 1 -- 172.21.15.153:0/137976045 wait complete. 2024-03-21T10:50:40.857 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:40.860+0000 7ff10c400700 1 -- 172.21.15.153:0/4016529119 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff104068670 msgr2=0x7ff104068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:50:40.857 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:40.860+0000 7ff10c400700 1 --2- 172.21.15.153:0/4016529119 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff104068670 0x7ff104068a50 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7ff0f8009910 tx=0x7ff0f8009c20 comp rx=0 tx=0).stop 2024-03-21T10:50:40.857 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:40.860+0000 7ff10c400700 1 -- 172.21.15.153:0/4016529119 shutdown_connections 2024-03-21T10:50:40.857 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:40.860+0000 7ff10c400700 1 --2- 172.21.15.153:0/4016529119 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff104069020 0x7ff104105790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:50:40.857 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:40.860+0000 7ff10c400700 1 --2- 172.21.15.153:0/4016529119 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff104068670 0x7ff104068a50 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:50:40.857 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:40.860+0000 7ff10c400700 1 -- 172.21.15.153:0/4016529119 >> 172.21.15.153:0/4016529119 conn(0x7ff1040755a0 msgr2=0x7ff1040759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:50:40.857 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:40.860+0000 7ff10c400700 1 -- 172.21.15.153:0/4016529119 shutdown_connections 2024-03-21T10:50:40.857 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:40.860+0000 7ff10c400700 1 -- 172.21.15.153:0/4016529119 wait complete. 2024-03-21T10:50:40.858 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:40.862+0000 7ff10c400700 1 Processor -- start 2024-03-21T10:50:40.858 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:40.862+0000 7ff10c400700 1 -- start start 2024-03-21T10:50:40.859 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:40.862+0000 7ff10c400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff104068670 0x7ff1040ffca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:50:40.859 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:40.862+0000 7ff10c400700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff104069020 0x7ff1041001e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:50:40.859 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:40.862+0000 7ff10c400700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff104100720 con 0x7ff104068670 2024-03-21T10:50:40.859 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:40.862+0000 7ff10c400700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff104100860 con 0x7ff104069020 2024-03-21T10:50:40.860 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:40.862+0000 7ff109a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff104068670 0x7ff1040ffca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:50:40.860 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:40.862+0000 7ff109a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff104068670 0x7ff1040ffca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39824/0 (socket says 172.21.15.153:39824) 2024-03-21T10:50:40.860 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:40.862+0000 7ff109a00700 1 -- 172.21.15.153:0/1575497358 learned_addr learned my addr 172.21.15.153:0/1575497358 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:50:40.860 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:40.863+0000 7ff109000700 1 --2- 172.21.15.153:0/1575497358 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff104069020 0x7ff1041001e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:50:40.860 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:40.863+0000 7ff109a00700 1 -- 172.21.15.153:0/1575497358 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff104069020 msgr2=0x7ff1041001e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:50:40.860 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:40.863+0000 7ff109a00700 1 --2- 172.21.15.153:0/1575497358 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff104069020 0x7ff1041001e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:50:40.860 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:40.863+0000 7ff109a00700 1 -- 172.21.15.153:0/1575497358 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0f80095f0 con 0x7ff104068670 2024-03-21T10:50:40.860 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:40.863+0000 7ff109a00700 1 --2- 172.21.15.153:0/1575497358 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff104068670 0x7ff1040ffca0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7ff0f80092e0 tx=0x7ff0f800b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:50:40.861 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:40.864+0000 7ff102000700 1 -- 172.21.15.153:0/1575497358 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff0f801d070 con 0x7ff104068670 2024-03-21T10:50:40.861 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:40.864+0000 7ff102000700 1 -- 172.21.15.153:0/1575497358 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff0f80052a0 con 0x7ff104068670 2024-03-21T10:50:40.861 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:40.864+0000 7ff10c400700 1 -- 172.21.15.153:0/1575497358 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff104100ae0 con 0x7ff104068670 2024-03-21T10:50:40.861 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:40.864+0000 7ff102000700 1 -- 172.21.15.153:0/1575497358 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff0f8017400 con 0x7ff104068670 2024-03-21T10:50:40.861 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:40.864+0000 7ff10c400700 1 -- 172.21.15.153:0/1575497358 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1041a2c40 con 0x7ff104068670 2024-03-21T10:50:40.861 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:40.865+0000 7ff10c400700 1 -- 172.21.15.153:0/1575497358 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff104109130 con 0x7ff104068670 2024-03-21T10:50:40.862 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:40.866+0000 7ff102000700 1 -- 172.21.15.153:0/1575497358 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7ff0f800f820 con 0x7ff104068670 2024-03-21T10:50:40.864 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:40.868+0000 7ff102000700 1 --2- 172.21.15.153:0/1575497358 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff0ec06e460 0x7ff0ec070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:50:40.864 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:40.868+0000 7ff102000700 1 -- 172.21.15.153:0/1575497358 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7ff0f808f7d0 con 0x7ff104068670 2024-03-21T10:50:40.865 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:40.868+0000 7ff109000700 1 --2- 172.21.15.153:0/1575497358 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff0ec06e460 0x7ff0ec070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:50:40.865 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:40.869+0000 7ff109000700 1 --2- 172.21.15.153:0/1575497358 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff0ec06e460 0x7ff0ec070920 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7ff104101490 tx=0x7ff0f0009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:50:40.869 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:40.872+0000 7ff102000700 1 -- 172.21.15.153:0/1575497358 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff0f8058870 con 0x7ff104068670 2024-03-21T10:50:40.932 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:40 smithi174 ceph-mon[88004]: pgmap v1167: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 25 MiB/s rd, 597 B/s wr, 10 op/s 2024-03-21T10:50:40.932 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:40 smithi174 ceph-mon[88004]: from='client.36052 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:50:41.051 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:40 smithi153 ceph-mon[113733]: pgmap v1167: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 25 MiB/s rd, 597 B/s wr, 10 op/s 2024-03-21T10:50:41.052 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:40 smithi153 ceph-mon[113733]: from='client.36052 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:50:41.229 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:41.232+0000 7ff10c400700 1 -- 172.21.15.153:0/1575497358 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7ff1040611d0 con 0x7ff0ec06e460 2024-03-21T10:50:41.250 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:41.254+0000 7ff102000700 1 -- 172.21.15.153:0/1575497358 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7ff1040611d0 con 0x7ff0ec06e460 2024-03-21T10:50:41.251 INFO:teuthology.orchestra.run.smithi153.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-21T10:50:41.251 INFO:teuthology.orchestra.run.smithi153.stdout:alertmanager.smithi153 smithi153 *:9093,9094 running (39m) 15m ago 53m 15.6M - 0.25.0 c8568f914cd2 9ed3f2200b5f 2024-03-21T10:50:41.251 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi153 smithi153 starting - - - - 2024-03-21T10:50:41.251 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi174 smithi174 running (52m) 15m ago 51m 10.1M - 18.2.1 5be31c24972a 4f7a5fc3ac28 2024-03-21T10:50:41.251 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi153 smithi153 running (35m) 15m ago 53m 6656k - 19.0.0-2231-g2edfee68 237e43c7a65f 32c21104d5b0 2024-03-21T10:50:41.251 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi174 smithi174 running (35m) 15m ago 51m 6660k - 19.0.0-2231-g2edfee68 237e43c7a65f a04e48967ab6 2024-03-21T10:50:41.251 INFO:teuthology.orchestra.run.smithi153.stdout:grafana.smithi153 smithi153 *:3000 running (38m) 15m ago 52m 75.2M - 9.4.12 2bacad6d85d8 ee24026ab786 2024-03-21T10:50:41.251 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.iuyluj smithi153 running (31m) 15m ago 46m 18.7M - 19.0.0-2231-g2edfee68 237e43c7a65f 87aff8a5de70 2024-03-21T10:50:41.251 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.zipckr smithi153 running (31m) 15m ago 46m 18.9M - 19.0.0-2231-g2edfee68 237e43c7a65f ccc262bd5dd3 2024-03-21T10:50:41.251 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.nhozag smithi174 running (15m) 15m ago 45m 10.9M - 19.0.0-2231-g2edfee68 237e43c7a65f 9e02aea72c96 2024-03-21T10:50:41.251 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.yyimst smithi174 running (30m) 15m ago 46m 18.5M - 19.0.0-2231-g2edfee68 237e43c7a65f fc1962f9351e 2024-03-21T10:50:41.251 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi153.tluxni smithi153 *:8443,9283,8765 running (42m) 15m ago 55m 512M - 19.0.0-2231-g2edfee68 237e43c7a65f 7ffee0e55382 2024-03-21T10:50:41.251 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi174.qpcwat smithi174 *:8443,9283,8765 running (41m) 15m ago 51m 459M - 19.0.0-2231-g2edfee68 237e43c7a65f 748968f216ab 2024-03-21T10:50:41.251 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi153 smithi153 running (37m) 15m ago 55m 70.8M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f dd8038303dc3 2024-03-21T10:50:41.252 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi174 smithi174 running (36m) 15m ago 51m 51.9M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f 29d78a788f2c 2024-03-21T10:50:41.252 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi153 smithi153 *:9100 running (40m) 15m ago 53m 8140k - 1.5.0 0da6a335fe13 bb835edc0e0b 2024-03-21T10:50:41.252 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi174 smithi174 *:9100 running (40m) 15m ago 51m 8610k - 1.5.0 0da6a335fe13 7c4ecad790fa 2024-03-21T10:50:41.252 INFO:teuthology.orchestra.run.smithi153.stdout:osd.0 smithi153 running (35m) 15m ago 50m 118M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 507900fd4791 2024-03-21T10:50:41.252 INFO:teuthology.orchestra.run.smithi153.stdout:osd.1 smithi153 running (34m) 15m ago 50m 93.0M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 0d646452e9c1 2024-03-21T10:50:41.252 INFO:teuthology.orchestra.run.smithi153.stdout:osd.2 smithi153 running (33m) 15m ago 49m 91.2M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 1326f2afdfcd 2024-03-21T10:50:41.252 INFO:teuthology.orchestra.run.smithi153.stdout:osd.3 smithi174 running (33m) 15m ago 48m 87.5M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 70aafac71ad5 2024-03-21T10:50:41.252 INFO:teuthology.orchestra.run.smithi153.stdout:osd.4 smithi174 running (32m) 15m ago 48m 95.8M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 0d746ce6e79d 2024-03-21T10:50:41.252 INFO:teuthology.orchestra.run.smithi153.stdout:osd.5 smithi174 running (31m) 15m ago 47m 90.3M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 53e5f35b8cac 2024-03-21T10:50:41.252 INFO:teuthology.orchestra.run.smithi153.stdout:prometheus.smithi153 smithi153 *:9095 running (39m) 15m ago 52m 88.1M - 2.43.0 a07b618ecd1d 7d95561e50c2 2024-03-21T10:50:41.254 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:41.257+0000 7ff10c400700 1 -- 172.21.15.153:0/1575497358 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff0ec06e460 msgr2=0x7ff0ec070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:50:41.254 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:41.257+0000 7ff10c400700 1 --2- 172.21.15.153:0/1575497358 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff0ec06e460 0x7ff0ec070920 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7ff104101490 tx=0x7ff0f0009260 comp rx=0 tx=0).stop 2024-03-21T10:50:41.254 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:41.257+0000 7ff10c400700 1 -- 172.21.15.153:0/1575497358 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff104068670 msgr2=0x7ff1040ffca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:50:41.254 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:41.257+0000 7ff10c400700 1 --2- 172.21.15.153:0/1575497358 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff104068670 0x7ff1040ffca0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7ff0f80092e0 tx=0x7ff0f800b6c0 comp rx=0 tx=0).stop 2024-03-21T10:50:41.255 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:41.257+0000 7ff10c400700 1 -- 172.21.15.153:0/1575497358 shutdown_connections 2024-03-21T10:50:41.255 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:41.257+0000 7ff10c400700 1 --2- 172.21.15.153:0/1575497358 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ff0ec06e460 0x7ff0ec070920 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:50:41.255 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:41.257+0000 7ff10c400700 1 --2- 172.21.15.153:0/1575497358 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ff104068670 0x7ff1040ffca0 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:50:41.255 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:41.257+0000 7ff10c400700 1 --2- 172.21.15.153:0/1575497358 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff104069020 0x7ff1041001e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:50:41.255 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:41.257+0000 7ff10c400700 1 -- 172.21.15.153:0/1575497358 >> 172.21.15.153:0/1575497358 conn(0x7ff1040755a0 msgr2=0x7ff1040fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:50:41.255 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:41.258+0000 7ff10c400700 1 -- 172.21.15.153:0/1575497358 shutdown_connections 2024-03-21T10:50:41.255 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:41.258+0000 7ff10c400700 1 -- 172.21.15.153:0/1575497358 wait complete. 2024-03-21T10:50:41.425 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:41.428+0000 7fb43be00700 1 -- 172.21.15.153:0/3730282211 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb434068950 msgr2=0x7fb434068dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:50:41.425 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:41.428+0000 7fb43be00700 1 --2- 172.21.15.153:0/3730282211 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb434068950 0x7fb434068dd0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fb420009910 tx=0x7fb420009c20 comp rx=0 tx=0).stop 2024-03-21T10:50:41.425 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:41.428+0000 7fb43be00700 1 -- 172.21.15.153:0/3730282211 shutdown_connections 2024-03-21T10:50:41.425 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:41.428+0000 7fb43be00700 1 --2- 172.21.15.153:0/3730282211 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb434068950 0x7fb434068dd0 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:50:41.425 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:41.428+0000 7fb43be00700 1 --2- 172.21.15.153:0/3730282211 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb434105b50 0x7fb434068410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:50:41.426 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:41.428+0000 7fb43be00700 1 -- 172.21.15.153:0/3730282211 >> 172.21.15.153:0/3730282211 conn(0x7fb4340755a0 msgr2=0x7fb4340759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:50:41.426 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:41.428+0000 7fb43be00700 1 -- 172.21.15.153:0/3730282211 shutdown_connections 2024-03-21T10:50:41.426 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:41.428+0000 7fb43be00700 1 -- 172.21.15.153:0/3730282211 wait complete. 2024-03-21T10:50:41.426 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:41.429+0000 7fb43be00700 1 Processor -- start 2024-03-21T10:50:41.426 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:41.430+0000 7fb43be00700 1 -- start start 2024-03-21T10:50:41.427 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:41.430+0000 7fb43be00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb434068950 0x7fb4341a2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:50:41.427 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:41.430+0000 7fb43be00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb434105b50 0x7fb4341a3010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:50:41.427 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:41.430+0000 7fb43be00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4341a36a0 con 0x7fb434068950 2024-03-21T10:50:41.427 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:41.430+0000 7fb43be00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb43419cb50 con 0x7fb434105b50 2024-03-21T10:50:41.427 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:41.430+0000 7fb439400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb434068950 0x7fb4341a2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:50:41.427 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:41.430+0000 7fb439400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb434068950 0x7fb4341a2ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39848/0 (socket says 172.21.15.153:39848) 2024-03-21T10:50:41.428 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:41.430+0000 7fb439400700 1 -- 172.21.15.153:0/3408005205 learned_addr learned my addr 172.21.15.153:0/3408005205 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:50:41.428 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:41.430+0000 7fb438a00700 1 --2- 172.21.15.153:0/3408005205 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb434105b50 0x7fb4341a3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:50:41.428 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:41.431+0000 7fb439400700 1 -- 172.21.15.153:0/3408005205 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb434105b50 msgr2=0x7fb4341a3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:50:41.428 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:41.431+0000 7fb439400700 1 --2- 172.21.15.153:0/3408005205 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb434105b50 0x7fb4341a3010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:50:41.428 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:41.431+0000 7fb439400700 1 -- 172.21.15.153:0/3408005205 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4200095f0 con 0x7fb434068950 2024-03-21T10:50:41.429 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:41.431+0000 7fb439400700 1 --2- 172.21.15.153:0/3408005205 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb434068950 0x7fb4341a2ad0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fb42800e8f0 tx=0x7fb42800ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:50:41.429 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:41.432+0000 7fb432000700 1 -- 172.21.15.153:0/3408005205 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb428009790 con 0x7fb434068950 2024-03-21T10:50:41.429 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:41.432+0000 7fb432000700 1 -- 172.21.15.153:0/3408005205 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb42800c810 con 0x7fb434068950 2024-03-21T10:50:41.429 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:41.432+0000 7fb432000700 1 -- 172.21.15.153:0/3408005205 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb428003d20 con 0x7fb434068950 2024-03-21T10:50:41.429 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:41.432+0000 7fb43be00700 1 -- 172.21.15.153:0/3408005205 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb43419ce30 con 0x7fb434068950 2024-03-21T10:50:41.429 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:41.432+0000 7fb43be00700 1 -- 172.21.15.153:0/3408005205 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb43419d330 con 0x7fb434068950 2024-03-21T10:50:41.430 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:41.433+0000 7fb43be00700 1 -- 172.21.15.153:0/3408005205 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb43404ea90 con 0x7fb434068950 2024-03-21T10:50:41.432 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:41.435+0000 7fb432000700 1 -- 172.21.15.153:0/3408005205 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fb4280041f0 con 0x7fb434068950 2024-03-21T10:50:41.433 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:41.437+0000 7fb432000700 1 --2- 172.21.15.153:0/3408005205 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb41c06e460 0x7fb41c070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:50:41.434 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:41.437+0000 7fb432000700 1 -- 172.21.15.153:0/3408005205 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fb428014070 con 0x7fb434068950 2024-03-21T10:50:41.434 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:41.437+0000 7fb438a00700 1 --2- 172.21.15.153:0/3408005205 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb41c06e460 0x7fb41c070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:50:41.434 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:41.438+0000 7fb438a00700 1 --2- 172.21.15.153:0/3408005205 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb41c06e460 0x7fb41c070920 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fb434103430 tx=0x7fb420005a90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:50:41.438 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:41.441+0000 7fb432000700 1 -- 172.21.15.153:0/3408005205 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb42805b3c0 con 0x7fb434068950 2024-03-21T10:50:41.767 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:41.769+0000 7fb43be00700 1 -- 172.21.15.153:0/3408005205 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fb43419d7f0 con 0x7fb434068950 2024-03-21T10:50:41.767 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:41.770+0000 7fb432000700 1 -- 172.21.15.153:0/3408005205 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fb42805af50 con 0x7fb434068950 2024-03-21T10:50:41.767 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:50:41.767 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": { 2024-03-21T10:50:41.767 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:50:41.768 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:50:41.768 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": { 2024-03-21T10:50:41.768 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:50:41.768 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:50:41.768 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": { 2024-03-21T10:50:41.768 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 6 2024-03-21T10:50:41.768 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:50:41.768 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": { 2024-03-21T10:50:41.768 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 4 2024-03-21T10:50:41.768 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:50:41.768 INFO:teuthology.orchestra.run.smithi153.stdout: "overall": { 2024-03-21T10:50:41.768 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 14 2024-03-21T10:50:41.768 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-03-21T10:50:41.769 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:50:41.771 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:41.774+0000 7fb43be00700 1 -- 172.21.15.153:0/3408005205 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb41c06e460 msgr2=0x7fb41c070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:50:41.771 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:41.774+0000 7fb43be00700 1 --2- 172.21.15.153:0/3408005205 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb41c06e460 0x7fb41c070920 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fb434103430 tx=0x7fb420005a90 comp rx=0 tx=0).stop 2024-03-21T10:50:41.771 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:41.774+0000 7fb43be00700 1 -- 172.21.15.153:0/3408005205 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb434068950 msgr2=0x7fb4341a2ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:50:41.771 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:41.774+0000 7fb43be00700 1 --2- 172.21.15.153:0/3408005205 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb434068950 0x7fb4341a2ad0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fb42800e8f0 tx=0x7fb42800ec00 comp rx=0 tx=0).stop 2024-03-21T10:50:41.771 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:41.774+0000 7fb43be00700 1 -- 172.21.15.153:0/3408005205 shutdown_connections 2024-03-21T10:50:41.772 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:41.774+0000 7fb43be00700 1 --2- 172.21.15.153:0/3408005205 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb41c06e460 0x7fb41c070920 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:50:41.772 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:41.774+0000 7fb43be00700 1 --2- 172.21.15.153:0/3408005205 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb434068950 0x7fb4341a2ad0 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:50:41.772 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:41.774+0000 7fb43be00700 1 --2- 172.21.15.153:0/3408005205 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb434105b50 0x7fb4341a3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:50:41.772 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:41.774+0000 7fb43be00700 1 -- 172.21.15.153:0/3408005205 >> 172.21.15.153:0/3408005205 conn(0x7fb4340755a0 msgr2=0x7fb434101510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:50:41.772 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:41.775+0000 7fb43be00700 1 -- 172.21.15.153:0/3408005205 shutdown_connections 2024-03-21T10:50:41.772 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:41.775+0000 7fb43be00700 1 -- 172.21.15.153:0/3408005205 wait complete. 2024-03-21T10:50:41.967 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:41.967+0000 7f3dea400700 1 -- 172.21.15.153:0/2317097294 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3de4106a40 msgr2=0x7f3de4106e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:50:41.967 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:41.967+0000 7f3dea400700 1 --2- 172.21.15.153:0/2317097294 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3de4106a40 0x7f3de4106e20 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f3dd8009910 tx=0x7f3dd8009c20 comp rx=0 tx=0).stop 2024-03-21T10:50:41.967 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:41.967+0000 7f3dea400700 1 -- 172.21.15.153:0/2317097294 shutdown_connections 2024-03-21T10:50:41.967 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:41.967+0000 7f3dea400700 1 --2- 172.21.15.153:0/2317097294 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3de4068410 0x7f3de4068890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:50:41.967 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:41.967+0000 7f3dea400700 1 --2- 172.21.15.153:0/2317097294 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3de4106a40 0x7f3de4106e20 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:50:41.967 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:41.967+0000 7f3dea400700 1 -- 172.21.15.153:0/2317097294 >> 172.21.15.153:0/2317097294 conn(0x7f3de4075330 msgr2=0x7f3de4075740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:50:41.967 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:41.967+0000 7f3dea400700 1 -- 172.21.15.153:0/2317097294 shutdown_connections 2024-03-21T10:50:41.967 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:41.967+0000 7f3dea400700 1 -- 172.21.15.153:0/2317097294 wait complete. 2024-03-21T10:50:41.967 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:41.968+0000 7f3dea400700 1 Processor -- start 2024-03-21T10:50:41.967 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:41.968+0000 7f3dea400700 1 -- start start 2024-03-21T10:50:41.967 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:41.969+0000 7f3dea400700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3de4068410 0x7f3de419e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:50:41.967 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:41.969+0000 7f3dea400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3de4106a40 0x7f3de419ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:50:41.968 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:41.969+0000 7f3dea400700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3de419f110 con 0x7f3de4106a40 2024-03-21T10:50:41.968 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:41.969+0000 7f3dea400700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3de41985c0 con 0x7f3de4068410 2024-03-21T10:50:41.968 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:41.969+0000 7f3de3e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3de4106a40 0x7f3de419ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:50:41.968 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:41.969+0000 7f3de9000700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3de4068410 0x7f3de419e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:50:41.968 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:41.969+0000 7f3de9000700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3de4068410 0x7f3de419e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.153:56698/0 (socket says 172.21.15.153:56698) 2024-03-21T10:50:41.968 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:41.969+0000 7f3de9000700 1 -- 172.21.15.153:0/1743585988 learned_addr learned my addr 172.21.15.153:0/1743585988 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:50:41.968 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:41.970+0000 7f3de9000700 1 -- 172.21.15.153:0/1743585988 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3de4106a40 msgr2=0x7f3de419ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:50:41.968 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:41.970+0000 7f3de9000700 1 --2- 172.21.15.153:0/1743585988 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3de4106a40 0x7f3de419ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:50:41.968 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:41.970+0000 7f3de9000700 1 -- 172.21.15.153:0/1743585988 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dd80095f0 con 0x7f3de4068410 2024-03-21T10:50:41.968 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:41.970+0000 7f3de3e00700 1 --2- 172.21.15.153:0/1743585988 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3de4106a40 0x7f3de419ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T10:50:41.968 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:41.970+0000 7f3de9000700 1 --2- 172.21.15.153:0/1743585988 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3de4068410 0x7f3de419e540 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3dd80092e0 tx=0x7f3dd800bd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:50:41.969 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:41.970+0000 7f3de1600700 1 -- 172.21.15.153:0/1743585988 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3dd801d070 con 0x7f3de4068410 2024-03-21T10:50:41.969 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:41.971+0000 7f3dea400700 1 -- 172.21.15.153:0/1743585988 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3de4198840 con 0x7f3de4068410 2024-03-21T10:50:41.969 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:41.971+0000 7f3de1600700 1 -- 172.21.15.153:0/1743585988 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3dd800f460 con 0x7f3de4068410 2024-03-21T10:50:41.969 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:41.971+0000 7f3de1600700 1 -- 172.21.15.153:0/1743585988 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3dd8005e20 con 0x7f3de4068410 2024-03-21T10:50:41.969 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:41.971+0000 7f3dea400700 1 -- 172.21.15.153:0/1743585988 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3de4198d30 con 0x7f3de4068410 2024-03-21T10:50:41.969 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:41.972+0000 7f3dea400700 1 -- 172.21.15.153:0/1743585988 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3de404f2e0 con 0x7f3de4068410 2024-03-21T10:50:41.970 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:41.973+0000 7f3de1600700 1 -- 172.21.15.153:0/1743585988 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f3dd800f5d0 con 0x7f3de4068410 2024-03-21T10:50:41.971 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:41.974+0000 7f3de1600700 1 --2- 172.21.15.153:0/1743585988 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f3dcc06e460 0x7f3dcc070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:50:41.971 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:41.974+0000 7f3de1600700 1 -- 172.21.15.153:0/1743585988 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f3dd808fbe0 con 0x7f3de4068410 2024-03-21T10:50:41.971 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:41.975+0000 7f3de3e00700 1 --2- 172.21.15.153:0/1743585988 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f3dcc06e460 0x7f3dcc070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:50:41.972 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:41.975+0000 7f3de3e00700 1 --2- 172.21.15.153:0/1743585988 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f3dcc06e460 0x7f3dcc070920 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f3de40fe280 tx=0x7f3dd000b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:50:41.980 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:41.984+0000 7f3de1600700 1 -- 172.21.15.153:0/1743585988 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f3dd805c870 con 0x7f3de4068410 2024-03-21T10:50:42.051 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:41 smithi153 ceph-mon[113733]: from='client.36056 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:50:42.051 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:41 smithi153 ceph-mon[113733]: from='client.36060 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:50:42.051 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:41 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/3408005205' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:50:42.196 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:41 smithi174 ceph-mon[88004]: from='client.36056 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:50:42.197 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:41 smithi174 ceph-mon[88004]: from='client.36060 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:50:42.197 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:41 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/3408005205' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:50:42.286 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:42.289+0000 7f3dea400700 1 -- 172.21.15.153:0/1743585988 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f3de4199ad0 con 0x7f3de4068410 2024-03-21T10:50:42.286 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:42.290+0000 7f3de1600700 1 -- 172.21.15.153:0/1743585988 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 40 v40) v1 ==== 76+0+1948 (secure 0 0 0) 0x7f3dd805c400 con 0x7f3de4068410 2024-03-21T10:50:42.288 INFO:teuthology.orchestra.run.smithi153.stdout:e40 2024-03-21T10:50:42.288 INFO:teuthology.orchestra.run.smithi153.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-21T10:50:42.288 INFO:teuthology.orchestra.run.smithi153.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:50:42.288 INFO:teuthology.orchestra.run.smithi153.stdout:legacy client fscid: 1 2024-03-21T10:50:42.288 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:50:42.288 INFO:teuthology.orchestra.run.smithi153.stdout:Filesystem 'cephfs' (1) 2024-03-21T10:50:42.288 INFO:teuthology.orchestra.run.smithi153.stdout:fs_name cephfs 2024-03-21T10:50:42.288 INFO:teuthology.orchestra.run.smithi153.stdout:epoch 40 2024-03-21T10:50:42.288 INFO:teuthology.orchestra.run.smithi153.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-21T10:50:42.288 INFO:teuthology.orchestra.run.smithi153.stdout:created 2024-03-21T10:04:27.086576+0000 2024-03-21T10:50:42.288 INFO:teuthology.orchestra.run.smithi153.stdout:modified 2024-03-21T10:50:36.318024+0000 2024-03-21T10:50:42.288 INFO:teuthology.orchestra.run.smithi153.stdout:tableserver 0 2024-03-21T10:50:42.289 INFO:teuthology.orchestra.run.smithi153.stdout:root 0 2024-03-21T10:50:42.289 INFO:teuthology.orchestra.run.smithi153.stdout:session_timeout 60 2024-03-21T10:50:42.289 INFO:teuthology.orchestra.run.smithi153.stdout:session_autoclose 300 2024-03-21T10:50:42.289 INFO:teuthology.orchestra.run.smithi153.stdout:max_file_size 1099511627776 2024-03-21T10:50:42.289 INFO:teuthology.orchestra.run.smithi153.stdout:max_xattr_size 65536 2024-03-21T10:50:42.289 INFO:teuthology.orchestra.run.smithi153.stdout:required_client_features {} 2024-03-21T10:50:42.289 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure 0 2024-03-21T10:50:42.289 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure_osd_epoch 94 2024-03-21T10:50:42.289 INFO:teuthology.orchestra.run.smithi153.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-21T10:50:42.289 INFO:teuthology.orchestra.run.smithi153.stdout:max_mds 2 2024-03-21T10:50:42.289 INFO:teuthology.orchestra.run.smithi153.stdout:in 0,1 2024-03-21T10:50:42.289 INFO:teuthology.orchestra.run.smithi153.stdout:up {0=34442,1=34474} 2024-03-21T10:50:42.289 INFO:teuthology.orchestra.run.smithi153.stdout:failed 2024-03-21T10:50:42.289 INFO:teuthology.orchestra.run.smithi153.stdout:damaged 2024-03-21T10:50:42.289 INFO:teuthology.orchestra.run.smithi153.stdout:stopped 2024-03-21T10:50:42.290 INFO:teuthology.orchestra.run.smithi153.stdout:data_pools [3] 2024-03-21T10:50:42.290 INFO:teuthology.orchestra.run.smithi153.stdout:metadata_pool 2 2024-03-21T10:50:42.290 INFO:teuthology.orchestra.run.smithi153.stdout:inline_data enabled 2024-03-21T10:50:42.290 INFO:teuthology.orchestra.run.smithi153.stdout:balancer 2024-03-21T10:50:42.290 INFO:teuthology.orchestra.run.smithi153.stdout:bal_rank_mask -1 2024-03-21T10:50:42.290 INFO:teuthology.orchestra.run.smithi153.stdout:standby_count_wanted 1 2024-03-21T10:50:42.290 INFO:teuthology.orchestra.run.smithi153.stdout:qdb_cluster leader: 34442 members: 34442,34474 2024-03-21T10:50:42.290 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.zipckr{0:34442} state up:active seq 474 join_fscid=1 addr [v2:172.21.15.153:6826/1103846295,v1:172.21.15.153:6827/1103846295] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:50:42.290 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.yyimst{0:44313} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.174:6824/4127786797,v1:172.21.15.174:6825/4127786797] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:50:42.290 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.iuyluj{1:34474} state up:active seq 468 join_fscid=1 addr [v2:172.21.15.153:6828/1738630832,v1:172.21.15.153:6829/1738630832] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:50:42.290 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.nhozag{1:44687} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.174:6826/2767517845,v1:172.21.15.174:6827/2767517845] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:50:42.290 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:50:42.290 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:50:42.291 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:42.293+0000 7f3dea400700 1 -- 172.21.15.153:0/1743585988 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f3dcc06e460 msgr2=0x7f3dcc070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:50:42.291 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:42.293+0000 7f3dea400700 1 --2- 172.21.15.153:0/1743585988 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f3dcc06e460 0x7f3dcc070920 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f3de40fe280 tx=0x7f3dd000b410 comp rx=0 tx=0).stop 2024-03-21T10:50:42.292 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:42.293+0000 7f3dea400700 1 -- 172.21.15.153:0/1743585988 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3de4068410 msgr2=0x7f3de419e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:50:42.292 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:42.293+0000 7f3dea400700 1 --2- 172.21.15.153:0/1743585988 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3de4068410 0x7f3de419e540 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3dd80092e0 tx=0x7f3dd800bd70 comp rx=0 tx=0).stop 2024-03-21T10:50:42.292 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:42.293+0000 7f3dea400700 1 -- 172.21.15.153:0/1743585988 shutdown_connections 2024-03-21T10:50:42.292 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:42.293+0000 7f3dea400700 1 --2- 172.21.15.153:0/1743585988 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3de4068410 0x7f3de419e540 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:50:42.292 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:42.293+0000 7f3dea400700 1 --2- 172.21.15.153:0/1743585988 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f3dcc06e460 0x7f3dcc070920 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:50:42.292 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:42.293+0000 7f3dea400700 1 --2- 172.21.15.153:0/1743585988 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f3de4106a40 0x7f3de419ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:50:42.292 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:42.293+0000 7f3dea400700 1 -- 172.21.15.153:0/1743585988 >> 172.21.15.153:0/1743585988 conn(0x7f3de4075330 msgr2=0x7f3de40ffe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:50:42.292 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:42.293+0000 7f3dea400700 1 -- 172.21.15.153:0/1743585988 shutdown_connections 2024-03-21T10:50:42.292 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:42.293+0000 7f3dea400700 1 -- 172.21.15.153:0/1743585988 wait complete. 2024-03-21T10:50:42.292 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 40 2024-03-21T10:50:42.483 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:42.486+0000 7f7620000700 1 -- 172.21.15.153:0/1496561801 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f76180ffbd0 msgr2=0x7f7618100050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:50:42.483 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:42.486+0000 7f7620000700 1 --2- 172.21.15.153:0/1496561801 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f76180ffbd0 0x7f7618100050 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f7604009910 tx=0x7f7604009c20 comp rx=0 tx=0).stop 2024-03-21T10:50:42.483 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:42.486+0000 7f7620000700 1 -- 172.21.15.153:0/1496561801 shutdown_connections 2024-03-21T10:50:42.483 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:42.486+0000 7f7620000700 1 --2- 172.21.15.153:0/1496561801 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f76180ffbd0 0x7f7618100050 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:50:42.483 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:42.486+0000 7f7620000700 1 --2- 172.21.15.153:0/1496561801 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7618105a40 0x7f7618105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:50:42.483 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:42.486+0000 7f7620000700 1 -- 172.21.15.153:0/1496561801 >> 172.21.15.153:0/1496561801 conn(0x7f76180755a0 msgr2=0x7f76180759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:50:42.484 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:42.486+0000 7f7620000700 1 -- 172.21.15.153:0/1496561801 shutdown_connections 2024-03-21T10:50:42.484 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:42.486+0000 7f7620000700 1 -- 172.21.15.153:0/1496561801 wait complete. 2024-03-21T10:50:42.484 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:42.487+0000 7f7620000700 1 Processor -- start 2024-03-21T10:50:42.484 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:42.488+0000 7f7620000700 1 -- start start 2024-03-21T10:50:42.485 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:42.488+0000 7f7620000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f76180ffbd0 0x7f761819e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:50:42.485 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:42.488+0000 7f7620000700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7618105a40 0x7f761819ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:50:42.485 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:42.488+0000 7f7620000700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f761819f2c0 con 0x7f76180ffbd0 2024-03-21T10:50:42.485 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:42.488+0000 7f7620000700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7618198770 con 0x7f7618105a40 2024-03-21T10:50:42.485 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:42.488+0000 7f761d600700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f76180ffbd0 0x7f761819e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:50:42.485 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:42.488+0000 7f761d600700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f76180ffbd0 0x7f761819e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39874/0 (socket says 172.21.15.153:39874) 2024-03-21T10:50:42.485 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:42.488+0000 7f761d600700 1 -- 172.21.15.153:0/738395046 learned_addr learned my addr 172.21.15.153:0/738395046 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:50:42.485 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:42.488+0000 7f761cc00700 1 --2- 172.21.15.153:0/738395046 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7618105a40 0x7f761819ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:50:42.486 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:42.489+0000 7f761d600700 1 -- 172.21.15.153:0/738395046 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7618105a40 msgr2=0x7f761819ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:50:42.486 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:42.489+0000 7f761d600700 1 --2- 172.21.15.153:0/738395046 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7618105a40 0x7f761819ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:50:42.487 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:42.489+0000 7f761d600700 1 -- 172.21.15.153:0/738395046 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76040095f0 con 0x7f76180ffbd0 2024-03-21T10:50:42.487 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:42.489+0000 7f761d600700 1 --2- 172.21.15.153:0/738395046 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f76180ffbd0 0x7f761819e6f0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f760c00e920 tx=0x7f760c00ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:50:42.487 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:42.489+0000 7f7616000700 1 -- 172.21.15.153:0/738395046 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f760c00c9d0 con 0x7f76180ffbd0 2024-03-21T10:50:42.487 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:42.490+0000 7f7616000700 1 -- 172.21.15.153:0/738395046 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f760c010430 con 0x7f76180ffbd0 2024-03-21T10:50:42.487 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:42.490+0000 7f7620000700 1 -- 172.21.15.153:0/738395046 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7618198a50 con 0x7f76180ffbd0 2024-03-21T10:50:42.487 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:42.490+0000 7f7616000700 1 -- 172.21.15.153:0/738395046 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f760c018630 con 0x7f76180ffbd0 2024-03-21T10:50:42.487 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:42.490+0000 7f7620000700 1 -- 172.21.15.153:0/738395046 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7618198fa0 con 0x7f76180ffbd0 2024-03-21T10:50:42.488 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:42.491+0000 7f7620000700 1 -- 172.21.15.153:0/738395046 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f761804ea90 con 0x7f76180ffbd0 2024-03-21T10:50:42.489 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:42.492+0000 7f7616000700 1 -- 172.21.15.153:0/738395046 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f760c0105a0 con 0x7f76180ffbd0 2024-03-21T10:50:42.490 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:42.494+0000 7f7616000700 1 --2- 172.21.15.153:0/738395046 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f760006e460 0x7f7600070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:50:42.491 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:42.494+0000 7f7616000700 1 -- 172.21.15.153:0/738395046 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f760c014070 con 0x7f76180ffbd0 2024-03-21T10:50:42.491 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:42.494+0000 7f761cc00700 1 --2- 172.21.15.153:0/738395046 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f760006e460 0x7f7600070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:50:42.491 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:42.495+0000 7f761cc00700 1 --2- 172.21.15.153:0/738395046 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f760006e460 0x7f7600070920 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f7618199be0 tx=0x7f7604005a90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:50:42.498 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:42.502+0000 7f7616000700 1 -- 172.21.15.153:0/738395046 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f760c053740 con 0x7f76180ffbd0 2024-03-21T10:50:42.803 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:42.806+0000 7f7620000700 1 -- 172.21.15.153:0/738395046 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f76181993c0 con 0x7f760006e460 2024-03-21T10:50:42.804 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:42.808+0000 7f7616000700 1 -- 172.21.15.153:0/738395046 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+441 (secure 0 0 0) 0x7f76181993c0 con 0x7f760006e460 2024-03-21T10:50:42.805 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:50:42.805 INFO:teuthology.orchestra.run.smithi153.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232", 2024-03-21T10:50:42.805 INFO:teuthology.orchestra.run.smithi153.stdout: "in_progress": true, 2024-03-21T10:50:42.805 INFO:teuthology.orchestra.run.smithi153.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-21T10:50:42.805 INFO:teuthology.orchestra.run.smithi153.stdout: "services_complete": [ 2024-03-21T10:50:42.805 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr", 2024-03-21T10:50:42.805 INFO:teuthology.orchestra.run.smithi153.stdout: "mon", 2024-03-21T10:50:42.805 INFO:teuthology.orchestra.run.smithi153.stdout: "mds", 2024-03-21T10:50:42.805 INFO:teuthology.orchestra.run.smithi153.stdout: "crash", 2024-03-21T10:50:42.805 INFO:teuthology.orchestra.run.smithi153.stdout: "osd" 2024-03-21T10:50:42.805 INFO:teuthology.orchestra.run.smithi153.stdout: ], 2024-03-21T10:50:42.805 INFO:teuthology.orchestra.run.smithi153.stdout: "progress": "16/23 daemons upgraded", 2024-03-21T10:50:42.806 INFO:teuthology.orchestra.run.smithi153.stdout: "message": "Currently upgrading ceph-exporter daemons", 2024-03-21T10:50:42.806 INFO:teuthology.orchestra.run.smithi153.stdout: "is_paused": false 2024-03-21T10:50:42.806 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:50:42.807 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:42.811+0000 7f7620000700 1 -- 172.21.15.153:0/738395046 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f760006e460 msgr2=0x7f7600070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:50:42.808 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:42.811+0000 7f7620000700 1 --2- 172.21.15.153:0/738395046 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f760006e460 0x7f7600070920 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f7618199be0 tx=0x7f7604005a90 comp rx=0 tx=0).stop 2024-03-21T10:50:42.808 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:42.811+0000 7f7620000700 1 -- 172.21.15.153:0/738395046 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f76180ffbd0 msgr2=0x7f761819e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:50:42.808 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:42.811+0000 7f7620000700 1 --2- 172.21.15.153:0/738395046 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f76180ffbd0 0x7f761819e6f0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f760c00e920 tx=0x7f760c00ec30 comp rx=0 tx=0).stop 2024-03-21T10:50:42.808 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:42.811+0000 7f7620000700 1 -- 172.21.15.153:0/738395046 shutdown_connections 2024-03-21T10:50:42.808 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:42.811+0000 7f7620000700 1 --2- 172.21.15.153:0/738395046 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f760006e460 0x7f7600070920 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:50:42.808 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:42.811+0000 7f7620000700 1 --2- 172.21.15.153:0/738395046 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f76180ffbd0 0x7f761819e6f0 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:50:42.808 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:42.811+0000 7f7620000700 1 --2- 172.21.15.153:0/738395046 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7618105a40 0x7f761819ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:50:42.808 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:42.811+0000 7f7620000700 1 -- 172.21.15.153:0/738395046 >> 172.21.15.153:0/738395046 conn(0x7f76180755a0 msgr2=0x7f7618068410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:50:42.808 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:42.812+0000 7f7620000700 1 -- 172.21.15.153:0/738395046 shutdown_connections 2024-03-21T10:50:42.809 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:42.812+0000 7f7620000700 1 -- 172.21.15.153:0/738395046 wait complete. 2024-03-21T10:50:42.986 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:42.988+0000 7fada7400700 1 -- 172.21.15.153:0/734682961 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fada0069020 msgr2=0x7fada0105790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:50:42.986 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:42.988+0000 7fada7400700 1 --2- 172.21.15.153:0/734682961 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fada0069020 0x7fada0105790 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fad8c009910 tx=0x7fad8c009c20 comp rx=0 tx=0).stop 2024-03-21T10:50:42.987 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:42.988+0000 7fada7400700 1 -- 172.21.15.153:0/734682961 shutdown_connections 2024-03-21T10:50:42.987 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:42.988+0000 7fada7400700 1 --2- 172.21.15.153:0/734682961 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fada0069020 0x7fada0105790 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:50:42.987 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:42.988+0000 7fada7400700 1 --2- 172.21.15.153:0/734682961 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fada0068670 0x7fada0068a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:50:42.987 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:42.988+0000 7fada7400700 1 -- 172.21.15.153:0/734682961 >> 172.21.15.153:0/734682961 conn(0x7fada00755a0 msgr2=0x7fada00759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:50:42.987 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:42.988+0000 7fada7400700 1 -- 172.21.15.153:0/734682961 shutdown_connections 2024-03-21T10:50:42.987 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:42.988+0000 7fada7400700 1 -- 172.21.15.153:0/734682961 wait complete. 2024-03-21T10:50:42.987 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:42.989+0000 7fada7400700 1 Processor -- start 2024-03-21T10:50:42.987 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:42.990+0000 7fada7400700 1 -- start start 2024-03-21T10:50:42.989 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:42.990+0000 7fada7400700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fada0068670 0x7fada00ffca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:50:42.989 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:42.990+0000 7fada7400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fada0069020 0x7fada01001e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:50:42.989 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:42.990+0000 7fada7400700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fada0100720 con 0x7fada0069020 2024-03-21T10:50:42.989 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:42.990+0000 7fada7400700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fada0100860 con 0x7fada0068670 2024-03-21T10:50:42.989 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:42.990+0000 7fada4a00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fada0068670 0x7fada00ffca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:50:42.989 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:42.991+0000 7fada4a00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fada0068670 0x7fada00ffca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.153:37974/0 (socket says 172.21.15.153:37974) 2024-03-21T10:50:42.989 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:42.991+0000 7fada4a00700 1 -- 172.21.15.153:0/2032368631 learned_addr learned my addr 172.21.15.153:0/2032368631 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:50:42.989 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:42.991+0000 7fad9fe00700 1 --2- 172.21.15.153:0/2032368631 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fada0069020 0x7fada01001e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:50:42.989 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:42.991+0000 7fada4a00700 1 -- 172.21.15.153:0/2032368631 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fada0069020 msgr2=0x7fada01001e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:50:42.989 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:42.991+0000 7fada4a00700 1 --2- 172.21.15.153:0/2032368631 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fada0069020 0x7fada01001e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:50:42.989 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:42.991+0000 7fada4a00700 1 -- 172.21.15.153:0/2032368631 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad8c0095f0 con 0x7fada0068670 2024-03-21T10:50:42.990 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:42.991+0000 7fada4a00700 1 --2- 172.21.15.153:0/2032368631 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fada0068670 0x7fada00ffca0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fad9400e920 tx=0x7fad9400ece0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:50:42.990 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:42.992+0000 7fad9d600700 1 -- 172.21.15.153:0/2032368631 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fad94009790 con 0x7fada0068670 2024-03-21T10:50:42.990 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:42.992+0000 7fada7400700 1 -- 172.21.15.153:0/2032368631 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fada01a28d0 con 0x7fada0068670 2024-03-21T10:50:42.990 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:42.992+0000 7fad9d600700 1 -- 172.21.15.153:0/2032368631 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fad9400cba0 con 0x7fada0068670 2024-03-21T10:50:42.990 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:42.992+0000 7fad9d600700 1 -- 172.21.15.153:0/2032368631 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fad94018650 con 0x7fada0068670 2024-03-21T10:50:42.990 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:42.992+0000 7fada7400700 1 -- 172.21.15.153:0/2032368631 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fada01a2ce0 con 0x7fada0068670 2024-03-21T10:50:42.990 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:42.994+0000 7fada7400700 1 -- 172.21.15.153:0/2032368631 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad80005320 con 0x7fada0068670 2024-03-21T10:50:42.991 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:42.995+0000 7fad9d600700 1 -- 172.21.15.153:0/2032368631 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fad9400cd10 con 0x7fada0068670 2024-03-21T10:50:42.993 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:42.996+0000 7fad9d600700 1 --2- 172.21.15.153:0/2032368631 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fad8806e460 0x7fad88070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:50:42.993 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:42.996+0000 7fad9d600700 1 -- 172.21.15.153:0/2032368631 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fad94014070 con 0x7fada0068670 2024-03-21T10:50:42.993 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:42.996+0000 7fad9fe00700 1 --2- 172.21.15.153:0/2032368631 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fad8806e460 0x7fad88070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:50:42.993 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:42.997+0000 7fad9fe00700 1 --2- 172.21.15.153:0/2032368631 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fad8806e460 0x7fad88070920 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fad8c005b40 tx=0x7fad8c005a90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:50:43.003 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:43.006+0000 7fad9d600700 1 -- 172.21.15.153:0/2032368631 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fad94053740 con 0x7fada0068670 2024-03-21T10:50:43.196 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:42 smithi174 ceph-mon[88004]: pgmap v1168: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 32 MiB/s rd, 7.6 KiB/s wr, 18 op/s 2024-03-21T10:50:43.196 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:42 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/1743585988' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:50:43.301 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:42 smithi153 ceph-mon[113733]: pgmap v1168: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 32 MiB/s rd, 7.6 KiB/s wr, 18 op/s 2024-03-21T10:50:43.301 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:42 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/1743585988' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:50:43.324 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:43.327+0000 7fada7400700 1 -- 172.21.15.153:0/2032368631 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fad80005190 con 0x7fada0068670 2024-03-21T10:50:43.324 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:43.328+0000 7fad9d600700 1 -- 172.21.15.153:0/2032368631 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4271 (secure 0 0 0) 0x7fad9405b3e0 con 0x7fada0068670 2024-03-21T10:50:43.325 INFO:teuthology.orchestra.run.smithi153.stdout:HEALTH_WARN failed to probe daemons or devices; 1 filesystem with deprecated feature inline_data 2024-03-21T10:50:43.325 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-03-21T10:50:43.325 INFO:teuthology.orchestra.run.smithi153.stdout: host smithi174 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi174/config 2024-03-21T10:50:43.325 INFO:teuthology.orchestra.run.smithi153.stdout:/usr/bin/podman: timeout after 895 seconds 2024-03-21T10:50:43.325 INFO:teuthology.orchestra.run.smithi153.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 -e NODE_NAME=smithi174 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/run/ceph:z -v /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp24y57pwk:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 inventory --format=json-pretty --filter-for-batch 2024-03-21T10:50:43.325 INFO:teuthology.orchestra.run.smithi153.stdout:Traceback (most recent call last): 2024-03-21T10:50:43.326 INFO:teuthology.orchestra.run.smithi153.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-03-21T10:50:43.326 INFO:teuthology.orchestra.run.smithi153.stdout: return _run_code(code, main_globals, None, 2024-03-21T10:50:43.326 INFO:teuthology.orchestra.run.smithi153.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-03-21T10:50:43.326 INFO:teuthology.orchestra.run.smithi153.stdout: exec(code, run_globals) 2024-03-21T10:50:43.326 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 5606, in 2024-03-21T10:50:43.326 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 5594, in main 2024-03-21T10:50:43.326 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 406, in _infer_config 2024-03-21T10:50:43.326 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 321, in _infer_fsid 2024-03-21T10:50:43.326 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 434, in _infer_image 2024-03-21T10:50:43.326 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 308, in _validate_fsid 2024-03-21T10:50:43.326 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/__main__.py", line 3357, in command_ceph_volume 2024-03-21T10:50:43.326 INFO:teuthology.orchestra.run.smithi153.stdout: File "/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/cephadm.d744b1a6a4366ae1e46ebaa230f96e064f94a2137981404df94e6fd1bb95d942/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-03-21T10:50:43.326 INFO:teuthology.orchestra.run.smithi153.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 -e NODE_NAME=smithi174 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/run/ceph:z -v /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp24y57pwk:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 inventory --format=json-pretty --filter-for-batch 2024-03-21T10:50:43.326 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-21T10:50:43.327 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-21T10:50:43.328 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:43.331+0000 7fada7400700 1 -- 172.21.15.153:0/2032368631 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fad8806e460 msgr2=0x7fad88070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:50:43.328 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:43.331+0000 7fada7400700 1 --2- 172.21.15.153:0/2032368631 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fad8806e460 0x7fad88070920 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fad8c005b40 tx=0x7fad8c005a90 comp rx=0 tx=0).stop 2024-03-21T10:50:43.328 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:43.331+0000 7fada7400700 1 -- 172.21.15.153:0/2032368631 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fada0068670 msgr2=0x7fada00ffca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:50:43.328 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:43.331+0000 7fada7400700 1 --2- 172.21.15.153:0/2032368631 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fada0068670 0x7fada00ffca0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fad9400e920 tx=0x7fad9400ece0 comp rx=0 tx=0).stop 2024-03-21T10:50:43.329 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:43.331+0000 7fada7400700 1 -- 172.21.15.153:0/2032368631 shutdown_connections 2024-03-21T10:50:43.329 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:43.331+0000 7fada7400700 1 --2- 172.21.15.153:0/2032368631 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fada0068670 0x7fada00ffca0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:50:43.329 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:43.331+0000 7fada7400700 1 --2- 172.21.15.153:0/2032368631 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fad8806e460 0x7fad88070920 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:50:43.329 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:43.331+0000 7fada7400700 1 --2- 172.21.15.153:0/2032368631 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fada0069020 0x7fada01001e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:50:43.329 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:43.331+0000 7fada7400700 1 -- 172.21.15.153:0/2032368631 >> 172.21.15.153:0/2032368631 conn(0x7fada00755a0 msgr2=0x7fada0105000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:50:43.329 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:43.332+0000 7fada7400700 1 -- 172.21.15.153:0/2032368631 shutdown_connections 2024-03-21T10:50:43.329 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:50:43.332+0000 7fada7400700 1 -- 172.21.15.153:0/2032368631 wait complete. 2024-03-21T10:50:44.177 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:43 smithi174 ceph-mon[88004]: from='client.36072 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:50:44.177 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:43 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/2032368631' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:50:44.301 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:43 smithi153 ceph-mon[113733]: from='client.36072 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:50:44.301 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:43 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/2032368631' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:50:45.301 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:44 smithi153 ceph-mon[113733]: pgmap v1169: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 26 MiB/s rd, 7.6 KiB/s wr, 16 op/s 2024-03-21T10:50:45.446 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:44 smithi174 ceph-mon[88004]: pgmap v1169: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 26 MiB/s rd, 7.6 KiB/s wr, 16 op/s 2024-03-21T10:50:47.551 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:47 smithi153 ceph-mon[113733]: pgmap v1170: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 27 MiB/s rd, 12 KiB/s wr, 19 op/s 2024-03-21T10:50:47.551 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:47 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:50:47.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:47 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:50:47.696 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:47 smithi174 ceph-mon[88004]: pgmap v1170: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 27 MiB/s rd, 12 KiB/s wr, 19 op/s 2024-03-21T10:50:47.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:47 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:50:47.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:47 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:50:48.696 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:48 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:50:48.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:48 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:50:48.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:48 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:50:48.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:48 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:50:48.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:48 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:50:48.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:48 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:50:48.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:48 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:50:48.801 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:48 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:50:48.801 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:48 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:50:48.801 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:48 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:50:48.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:48 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:50:48.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:48 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:50:48.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:48 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:50:48.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:48 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:50:49.801 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:49 smithi153 ceph-mon[113733]: pgmap v1171: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 26 MiB/s rd, 12 KiB/s wr, 19 op/s 2024-03-21T10:50:49.837 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:49 smithi174 ceph-mon[88004]: pgmap v1171: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 26 MiB/s rd, 12 KiB/s wr, 19 op/s 2024-03-21T10:50:51.709 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:51 smithi153 ceph-mon[113733]: pgmap v1172: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 23 MiB/s rd, 12 KiB/s wr, 17 op/s 2024-03-21T10:50:51.946 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:51 smithi174 ceph-mon[88004]: pgmap v1172: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 23 MiB/s rd, 12 KiB/s wr, 17 op/s 2024-03-21T10:50:53.196 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:52 smithi174 ceph-mon[88004]: pgmap v1173: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 19 MiB/s rd, 11 KiB/s wr, 15 op/s 2024-03-21T10:50:53.197 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:52 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:50:53.197 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:52 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:50:53.301 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:52 smithi153 ceph-mon[113733]: pgmap v1173: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 19 MiB/s rd, 11 KiB/s wr, 15 op/s 2024-03-21T10:50:53.302 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:52 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:50:53.302 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:52 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:50:55.051 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:54 smithi153 ceph-mon[113733]: pgmap v1174: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 6.8 MiB/s rd, 4.5 KiB/s wr, 5 op/s 2024-03-21T10:50:55.052 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:54 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:50:55.052 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:54 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:50:55.052 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:54 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:50:55.052 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:54 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:50:55.078 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:54 smithi174 ceph-mon[88004]: pgmap v1174: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 6.8 MiB/s rd, 4.5 KiB/s wr, 5 op/s 2024-03-21T10:50:55.078 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:54 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:50:55.079 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:54 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:50:55.079 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:54 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:50:55.079 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:54 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:50:57.604 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:57 smithi153 ceph-mon[113733]: pgmap v1175: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 3.2 MiB/s rd, 4.5 KiB/s wr, 6 op/s 2024-03-21T10:50:57.604 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:57 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:50:57.604 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:57 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:50:57.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:57 smithi174 ceph-mon[88004]: pgmap v1175: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 3.2 MiB/s rd, 4.5 KiB/s wr, 6 op/s 2024-03-21T10:50:57.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:57 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:50:57.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:57 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:50:59.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:59 smithi174 ceph-mon[88004]: pgmap v1176: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:50:59.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:59 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:50:59.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:50:59 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:50:59.801 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:59 smithi153 ceph-mon[113733]: pgmap v1176: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:50:59.801 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:59 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:50:59.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:50:59 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:51:01.551 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:01 smithi153 ceph-mon[113733]: pgmap v1177: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:51:01.551 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:01 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:51:01.946 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:01 smithi174 ceph-mon[88004]: pgmap v1177: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:51:01.946 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:01 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:51:03.052 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:02 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:51:03.052 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:02 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:51:03.052 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:02 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:51:03.052 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:02 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:51:03.052 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:02 smithi153 ceph-mon[113733]: pgmap v1178: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:51:03.052 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:02 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:51:03.052 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:02 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:51:03.052 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:02 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:51:03.052 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:02 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:51:03.052 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:02 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:51:03.053 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:02 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:51:03.053 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:02 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:51:03.053 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:02 smithi153 ceph-mon[113733]: Upgrade: Setting filesystem cephfs Joinable 2024-03-21T10:51:03.053 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:02 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-03-21T10:51:03.196 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:02 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:51:03.197 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:02 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:51:03.197 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:02 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:51:03.197 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:02 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:51:03.197 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:02 smithi174 ceph-mon[88004]: pgmap v1178: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:51:03.197 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:02 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:51:03.197 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:02 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:51:03.197 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:02 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:51:03.197 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:02 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:51:03.197 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:02 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:51:03.197 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:02 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:51:03.197 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:02 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:51:03.197 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:02 smithi174 ceph-mon[88004]: Upgrade: Setting filesystem cephfs Joinable 2024-03-21T10:51:03.197 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:02 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-03-21T10:51:04.053 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:03 smithi153 ceph-mon[113733]: Health check cleared: CEPHADM_REFRESH_FAILED (was: failed to probe daemons or devices) 2024-03-21T10:51:04.053 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:03 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-03-21T10:51:04.053 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:03 smithi153 ceph-mon[113733]: fsmap cephfs:2 {0=cephfs.smithi153.zipckr=up:active,1=cephfs.smithi153.iuyluj=up:active} 2 up:standby-replay 2024-03-21T10:51:04.053 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:03 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:51:04.053 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:03 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:51:04.054 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:03 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:51:04.054 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:03 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:51:04.054 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:03 smithi153 ceph-mon[113733]: Upgrade: Setting container_image for all ceph-exporter 2024-03-21T10:51:04.054 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:03 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:51:04.054 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:03 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi153"}]: dispatch 2024-03-21T10:51:04.054 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:03 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi153"}]': finished 2024-03-21T10:51:04.054 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:03 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi174"}]: dispatch 2024-03-21T10:51:04.054 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:03 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi174"}]': finished 2024-03-21T10:51:04.054 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:03 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:51:04.054 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:03 smithi153 ceph-mon[113733]: Upgrade: Setting container_image for all iscsi 2024-03-21T10:51:04.054 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:03 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:51:04.054 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:03 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:51:04.054 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:03 smithi153 ceph-mon[113733]: Upgrade: Setting container_image for all nfs 2024-03-21T10:51:04.055 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:03 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:51:04.055 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:03 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:51:04.055 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:03 smithi153 ceph-mon[113733]: Upgrade: Setting container_image for all nvmeof 2024-03-21T10:51:04.055 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:03 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:51:04.055 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:03 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:51:04.055 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:03 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:51:04.055 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:03 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:51:04.055 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:03 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:51:04.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:03 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:51:04.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:03 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:51:04.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:03 smithi153 ceph-mon[113733]: Upgrade: Finalizing container_image settings 2024-03-21T10:51:04.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:03 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-03-21T10:51:04.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:03 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-03-21T10:51:04.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:03 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-21T10:51:04.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:03 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-03-21T10:51:04.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:03 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-03-21T10:51:04.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:03 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-03-21T10:51:04.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:03 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-03-21T10:51:04.056 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:03 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-03-21T10:51:04.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:03 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-03-21T10:51:04.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:03 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-03-21T10:51:04.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:03 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-03-21T10:51:04.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:03 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-03-21T10:51:04.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:03 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-03-21T10:51:04.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:03 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-03-21T10:51:04.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:03 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-21T10:51:04.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:03 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-03-21T10:51:04.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:03 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-03-21T10:51:04.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:03 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-03-21T10:51:04.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:03 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-03-21T10:51:04.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:03 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-03-21T10:51:04.057 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:03 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-03-21T10:51:04.058 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:03 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-03-21T10:51:04.058 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:03 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-03-21T10:51:04.058 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:03 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-21T10:51:04.058 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:03 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-21T10:51:04.058 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:03 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-21T10:51:04.058 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:03 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-21T10:51:04.058 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:03 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-21T10:51:04.058 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:03 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-21T10:51:04.058 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:03 smithi153 ceph-mon[113733]: Upgrade: Complete! 2024-03-21T10:51:04.058 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:03 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-03-21T10:51:04.058 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:03 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-03-21T10:51:04.058 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:03 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:51:04.058 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:03 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:51:04.058 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:03 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:51:04.059 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:03 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:51:04.059 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:03 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:51:04.059 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:03 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:51:04.059 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:03 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:51:04.059 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:03 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:51:04.197 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:03 smithi174 ceph-mon[88004]: Health check cleared: CEPHADM_REFRESH_FAILED (was: failed to probe daemons or devices) 2024-03-21T10:51:04.197 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:03 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-03-21T10:51:04.198 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:03 smithi174 ceph-mon[88004]: fsmap cephfs:2 {0=cephfs.smithi153.zipckr=up:active,1=cephfs.smithi153.iuyluj=up:active} 2 up:standby-replay 2024-03-21T10:51:04.198 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:03 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:51:04.198 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:03 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:51:04.198 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:03 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:51:04.198 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:03 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:51:04.198 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:03 smithi174 ceph-mon[88004]: Upgrade: Setting container_image for all ceph-exporter 2024-03-21T10:51:04.198 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:03 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:51:04.198 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:03 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi153"}]: dispatch 2024-03-21T10:51:04.198 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:03 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi153"}]': finished 2024-03-21T10:51:04.198 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:03 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi174"}]: dispatch 2024-03-21T10:51:04.198 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:03 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi174"}]': finished 2024-03-21T10:51:04.198 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:03 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:51:04.199 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:03 smithi174 ceph-mon[88004]: Upgrade: Setting container_image for all iscsi 2024-03-21T10:51:04.199 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:03 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:51:04.199 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:03 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:51:04.199 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:03 smithi174 ceph-mon[88004]: Upgrade: Setting container_image for all nfs 2024-03-21T10:51:04.199 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:03 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:51:04.199 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:03 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:51:04.199 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:03 smithi174 ceph-mon[88004]: Upgrade: Setting container_image for all nvmeof 2024-03-21T10:51:04.199 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:03 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:51:04.199 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:03 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:51:04.199 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:03 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:51:04.199 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:03 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:51:04.199 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:03 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:51:04.200 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:03 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:51:04.200 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:03 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:51:04.200 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:03 smithi174 ceph-mon[88004]: Upgrade: Finalizing container_image settings 2024-03-21T10:51:04.200 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:03 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-03-21T10:51:04.200 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:03 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-03-21T10:51:04.200 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:03 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-21T10:51:04.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:03 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-03-21T10:51:04.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:03 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-03-21T10:51:04.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:03 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-03-21T10:51:04.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:03 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-03-21T10:51:04.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:03 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-03-21T10:51:04.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:03 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-03-21T10:51:04.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:03 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-03-21T10:51:04.201 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:03 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-03-21T10:51:04.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:03 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-03-21T10:51:04.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:03 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-03-21T10:51:04.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:03 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-03-21T10:51:04.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:03 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-21T10:51:04.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:03 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-03-21T10:51:04.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:03 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-03-21T10:51:04.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:03 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-03-21T10:51:04.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:03 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-03-21T10:51:04.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:03 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-03-21T10:51:04.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:03 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-03-21T10:51:04.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:03 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-03-21T10:51:04.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:03 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-03-21T10:51:04.202 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:03 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-21T10:51:04.203 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:03 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-21T10:51:04.203 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:03 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-21T10:51:04.203 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:03 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-21T10:51:04.203 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:03 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-21T10:51:04.203 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:03 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-21T10:51:04.203 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:03 smithi174 ceph-mon[88004]: Upgrade: Complete! 2024-03-21T10:51:04.203 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:03 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-03-21T10:51:04.203 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:03 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-03-21T10:51:04.203 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:03 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:51:04.203 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:03 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:51:04.204 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:03 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:51:04.204 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:03 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:51:04.204 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:03 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:51:04.204 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:03 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:51:04.204 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:03 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:51:04.204 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:03 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:51:05.051 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:04 smithi153 ceph-mon[113733]: pgmap v1179: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:51:05.196 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:04 smithi174 ceph-mon[88004]: pgmap v1179: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:51:07.196 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:06 smithi174 ceph-mon[88004]: pgmap v1180: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:51:07.301 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:06 smithi153 ceph-mon[113733]: pgmap v1180: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:51:08.196 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:07 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:51:08.359 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:07 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:51:09.118 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:08 smithi153 ceph-mon[113733]: pgmap v1181: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:51:09.127 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:08 smithi174 ceph-mon[88004]: pgmap v1181: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:51:10.946 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:10 smithi174 ceph-mon[88004]: pgmap v1182: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:51:11.301 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:10 smithi153 ceph-mon[113733]: pgmap v1182: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:51:13.052 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:12 smithi153 ceph-mon[113733]: pgmap v1183: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:51:13.197 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:12 smithi174 ceph-mon[88004]: pgmap v1183: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:51:13.503 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:13.506+0000 7f74f0800700 1 -- 172.21.15.153:0/1183171439 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f74e8069020 msgr2=0x7f74e8105790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:51:13.504 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:13.506+0000 7f74f0800700 1 --2- 172.21.15.153:0/1183171439 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f74e8069020 0x7f74e8105790 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f74d8009910 tx=0x7f74d8009c20 comp rx=0 tx=0).stop 2024-03-21T10:51:13.504 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:13.506+0000 7f74f0800700 1 -- 172.21.15.153:0/1183171439 shutdown_connections 2024-03-21T10:51:13.504 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:13.506+0000 7f74f0800700 1 --2- 172.21.15.153:0/1183171439 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f74e8069020 0x7f74e8105790 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:51:13.504 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:13.506+0000 7f74f0800700 1 --2- 172.21.15.153:0/1183171439 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f74e8068670 0x7f74e8068a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:51:13.504 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:13.506+0000 7f74f0800700 1 -- 172.21.15.153:0/1183171439 >> 172.21.15.153:0/1183171439 conn(0x7f74e80755a0 msgr2=0x7f74e80759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:51:13.504 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:13.506+0000 7f74f0800700 1 -- 172.21.15.153:0/1183171439 shutdown_connections 2024-03-21T10:51:13.504 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:13.506+0000 7f74f0800700 1 -- 172.21.15.153:0/1183171439 wait complete. 2024-03-21T10:51:13.505 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:13.508+0000 7f74f0800700 1 Processor -- start 2024-03-21T10:51:13.505 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:13.508+0000 7f74f0800700 1 -- start start 2024-03-21T10:51:13.505 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:13.508+0000 7f74f0800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f74e8068670 0x7f74e819e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:51:13.505 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:13.508+0000 7f74f0800700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f74e8069020 0x7f74e819ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:51:13.505 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:13.508+0000 7f74f0800700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74e819f2e0 con 0x7f74e8068670 2024-03-21T10:51:13.505 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:13.508+0000 7f74f0800700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74e8198790 con 0x7f74e8069020 2024-03-21T10:51:13.506 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:13.509+0000 7f74ede00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f74e8068670 0x7f74e819e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:51:13.506 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:13.509+0000 7f74ede00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f74e8068670 0x7f74e819e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39278/0 (socket says 172.21.15.153:39278) 2024-03-21T10:51:13.506 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:13.509+0000 7f74ede00700 1 -- 172.21.15.153:0/189663280 learned_addr learned my addr 172.21.15.153:0/189663280 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:51:13.507 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:13.509+0000 7f74ed400700 1 --2- 172.21.15.153:0/189663280 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f74e8069020 0x7f74e819ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:51:13.507 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:13.510+0000 7f74ede00700 1 -- 172.21.15.153:0/189663280 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f74e8069020 msgr2=0x7f74e819ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:51:13.507 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:13.510+0000 7f74ede00700 1 --2- 172.21.15.153:0/189663280 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f74e8069020 0x7f74e819ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:51:13.507 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:13.510+0000 7f74ede00700 1 -- 172.21.15.153:0/189663280 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74d80095f0 con 0x7f74e8068670 2024-03-21T10:51:13.507 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:13.510+0000 7f74ede00700 1 --2- 172.21.15.153:0/189663280 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f74e8068670 0x7f74e819e710 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f74dc00c8a0 tx=0x7f74dc00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:51:13.507 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:13.510+0000 7f74e2a00700 1 -- 172.21.15.153:0/189663280 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f74dc00cea0 con 0x7f74e8068670 2024-03-21T10:51:13.507 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:13.510+0000 7f74e2a00700 1 -- 172.21.15.153:0/189663280 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f74dc004830 con 0x7f74e8068670 2024-03-21T10:51:13.508 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:13.510+0000 7f74f0800700 1 -- 172.21.15.153:0/189663280 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74e8198a70 con 0x7f74e8068670 2024-03-21T10:51:13.508 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:13.510+0000 7f74e2a00700 1 -- 172.21.15.153:0/189663280 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f74dc0055b0 con 0x7f74e8068670 2024-03-21T10:51:13.508 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:13.511+0000 7f74f0800700 1 -- 172.21.15.153:0/189663280 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74e8198fc0 con 0x7f74e8068670 2024-03-21T10:51:13.511 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:13.513+0000 7f74f0800700 1 -- 172.21.15.153:0/189663280 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74e804ea90 con 0x7f74e8068670 2024-03-21T10:51:13.511 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:13.513+0000 7f74e2a00700 1 -- 172.21.15.153:0/189663280 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f74dc0049a0 con 0x7f74e8068670 2024-03-21T10:51:13.512 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:13.515+0000 7f74e2a00700 1 --2- 172.21.15.153:0/189663280 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f74d006e460 0x7f74d0070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:51:13.512 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:13.515+0000 7f74e2a00700 1 -- 172.21.15.153:0/189663280 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f74dc08de60 con 0x7f74e8068670 2024-03-21T10:51:13.513 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:13.515+0000 7f74ed400700 1 --2- 172.21.15.153:0/189663280 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f74d006e460 0x7f74d0070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:51:13.513 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:13.516+0000 7f74ed400700 1 --2- 172.21.15.153:0/189663280 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f74d006e460 0x7f74d0070920 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f74e8199c00 tx=0x7f74d801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:51:13.518 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:13.521+0000 7f74e2a00700 1 -- 172.21.15.153:0/189663280 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f74dc05aad0 con 0x7f74e8068670 2024-03-21T10:51:13.761 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:13.764+0000 7f74f0800700 1 -- 172.21.15.153:0/189663280 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f74e8199480 con 0x7f74d006e460 2024-03-21T10:51:13.763 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:13.766+0000 7f74e2a00700 1 -- 172.21.15.153:0/189663280 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7f74e8199480 con 0x7f74d006e460 2024-03-21T10:51:13.767 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:13.769+0000 7f74f0800700 1 -- 172.21.15.153:0/189663280 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f74d006e460 msgr2=0x7f74d0070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:51:13.767 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:13.769+0000 7f74f0800700 1 --2- 172.21.15.153:0/189663280 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f74d006e460 0x7f74d0070920 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f74e8199c00 tx=0x7f74d801a040 comp rx=0 tx=0).stop 2024-03-21T10:51:13.767 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:13.769+0000 7f74f0800700 1 -- 172.21.15.153:0/189663280 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f74e8068670 msgr2=0x7f74e819e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:51:13.767 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:13.769+0000 7f74f0800700 1 --2- 172.21.15.153:0/189663280 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f74e8068670 0x7f74e819e710 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f74dc00c8a0 tx=0x7f74dc00cc60 comp rx=0 tx=0).stop 2024-03-21T10:51:13.767 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:13.770+0000 7f74f0800700 1 -- 172.21.15.153:0/189663280 shutdown_connections 2024-03-21T10:51:13.768 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:13.770+0000 7f74f0800700 1 --2- 172.21.15.153:0/189663280 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f74d006e460 0x7f74d0070920 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:51:13.768 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:13.770+0000 7f74f0800700 1 --2- 172.21.15.153:0/189663280 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f74e8068670 0x7f74e819e710 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:51:13.768 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:13.770+0000 7f74f0800700 1 --2- 172.21.15.153:0/189663280 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f74e8069020 0x7f74e819ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:51:13.768 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:13.770+0000 7f74f0800700 1 -- 172.21.15.153:0/189663280 >> 172.21.15.153:0/189663280 conn(0x7f74e80755a0 msgr2=0x7f74e8104770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:51:13.768 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:13.770+0000 7f74f0800700 1 -- 172.21.15.153:0/189663280 shutdown_connections 2024-03-21T10:51:13.768 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:13.770+0000 7f74f0800700 1 -- 172.21.15.153:0/189663280 wait complete. 2024-03-21T10:51:14.577 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 070ed3e4-e769-11ee-95cd-87774f69a715 -e sha1=2edfee68ad7243bed422a049b6f5f58619300f11 -- bash -c 'ceph orch ps' 2024-03-21T10:51:15.044 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi153/config 2024-03-21T10:51:15.193 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:14 smithi153 ceph-mon[113733]: pgmap v1184: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:51:15.193 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:14 smithi153 ceph-mon[113733]: from='client.36082 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:51:15.196 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:14 smithi174 ceph-mon[88004]: pgmap v1184: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:51:15.196 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:14 smithi174 ceph-mon[88004]: from='client.36082 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:51:17.230 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:17.234+0000 7f9757000700 1 -- 172.21.15.153:0/404295980 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9750100dc0 msgr2=0x7f97501011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:51:17.230 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:17.234+0000 7f9757000700 1 --2- 172.21.15.153:0/404295980 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9750100dc0 0x7f97501011a0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f9740009910 tx=0x7f9740009c20 comp rx=0 tx=0).stop 2024-03-21T10:51:17.230 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:17.234+0000 7f9757000700 1 -- 172.21.15.153:0/404295980 shutdown_connections 2024-03-21T10:51:17.230 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:17.234+0000 7f9757000700 1 --2- 172.21.15.153:0/404295980 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9750101770 0x7f97501057c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:51:17.230 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:17.234+0000 7f9757000700 1 --2- 172.21.15.153:0/404295980 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9750100dc0 0x7f97501011a0 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:51:17.231 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:17.234+0000 7f9757000700 1 -- 172.21.15.153:0/404295980 >> 172.21.15.153:0/404295980 conn(0x7f97500fc650 msgr2=0x7f97500fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:51:17.231 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:17.234+0000 7f9757000700 1 -- 172.21.15.153:0/404295980 shutdown_connections 2024-03-21T10:51:17.231 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:17.234+0000 7f9757000700 1 -- 172.21.15.153:0/404295980 wait complete. 2024-03-21T10:51:17.231 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:17.235+0000 7f9757000700 1 Processor -- start 2024-03-21T10:51:17.231 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:17.236+0000 7f9757000700 1 -- start start 2024-03-21T10:51:17.232 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:17.236+0000 7f9757000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9750100dc0 0x7f9750198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:51:17.232 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:17.236+0000 7f9757000700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9750101770 0x7f9750198eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:51:17.232 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:17.236+0000 7f9757000700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9750199540 con 0x7f9750100dc0 2024-03-21T10:51:17.232 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:17.236+0000 7f9757000700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f975019d350 con 0x7f9750101770 2024-03-21T10:51:17.233 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:17.237+0000 7f974fe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9750100dc0 0x7f9750198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:51:17.233 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:17.237+0000 7f974fe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9750100dc0 0x7f9750198970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39294/0 (socket says 172.21.15.153:39294) 2024-03-21T10:51:17.233 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:17.237+0000 7f974fe00700 1 -- 172.21.15.153:0/1543710185 learned_addr learned my addr 172.21.15.153:0/1543710185 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:51:17.233 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:17.237+0000 7f974f400700 1 --2- 172.21.15.153:0/1543710185 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9750101770 0x7f9750198eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:51:17.233 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:17.237+0000 7f974fe00700 1 -- 172.21.15.153:0/1543710185 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9750101770 msgr2=0x7f9750198eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:51:17.233 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:17.237+0000 7f974fe00700 1 --2- 172.21.15.153:0/1543710185 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9750101770 0x7f9750198eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:51:17.234 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:17.237+0000 7f974fe00700 1 -- 172.21.15.153:0/1543710185 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97400095f0 con 0x7f9750100dc0 2024-03-21T10:51:17.234 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:17.238+0000 7f974fe00700 1 --2- 172.21.15.153:0/1543710185 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9750100dc0 0x7f9750198970 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f974000b5c0 tx=0x7f974000be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:51:17.234 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:17.238+0000 7f974cc00700 1 -- 172.21.15.153:0/1543710185 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f974001d070 con 0x7f9750100dc0 2024-03-21T10:51:17.234 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:17.238+0000 7f974cc00700 1 -- 172.21.15.153:0/1543710185 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f974000f460 con 0x7f9750100dc0 2024-03-21T10:51:17.234 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:17.238+0000 7f9757000700 1 -- 172.21.15.153:0/1543710185 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f975019d5d0 con 0x7f9750100dc0 2024-03-21T10:51:17.234 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:17.238+0000 7f974cc00700 1 -- 172.21.15.153:0/1543710185 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9740017610 con 0x7f9750100dc0 2024-03-21T10:51:17.235 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:17.238+0000 7f9757000700 1 -- 172.21.15.153:0/1543710185 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f975019dac0 con 0x7f9750100dc0 2024-03-21T10:51:17.235 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:17.239+0000 7f9757000700 1 -- 172.21.15.153:0/1543710185 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9750104f40 con 0x7f9750100dc0 2024-03-21T10:51:17.237 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:17.241+0000 7f974cc00700 1 -- 172.21.15.153:0/1543710185 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f974000f5d0 con 0x7f9750100dc0 2024-03-21T10:51:17.238 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:17.242+0000 7f974cc00700 1 --2- 172.21.15.153:0/1543710185 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f973406e460 0x7f9734070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:51:17.238 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:17.242+0000 7f974cc00700 1 -- 172.21.15.153:0/1543710185 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f974008fad0 con 0x7f9750100dc0 2024-03-21T10:51:17.239 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:17.243+0000 7f974f400700 1 --2- 172.21.15.153:0/1543710185 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f973406e460 0x7f9734070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:51:17.239 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:17.243+0000 7f974f400700 1 --2- 172.21.15.153:0/1543710185 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f973406e460 0x7f9734070920 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f973c005fd0 tx=0x7f973c00a560 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:51:17.247 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:17.252+0000 7f974cc00700 1 -- 172.21.15.153:0/1543710185 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f974005c6a0 con 0x7f9750100dc0 2024-03-21T10:51:17.489 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:17.493+0000 7f9757000700 1 -- 172.21.15.153:0/1543710185 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f975002cce0 con 0x7f973406e460 2024-03-21T10:51:17.489 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:17 smithi153 ceph-mon[113733]: pgmap v1185: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:51:17.489 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:17 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:51:17.489 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:17 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:51:17.505 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:17.509+0000 7f974cc00700 1 -- 172.21.15.153:0/1543710185 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f975002cce0 con 0x7f973406e460 2024-03-21T10:51:17.505 INFO:teuthology.orchestra.run.smithi153.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-21T10:51:17.506 INFO:teuthology.orchestra.run.smithi153.stdout:alertmanager.smithi153 smithi153 *:9093,9094 running (40m) 23s ago 53m 14.6M - 0.25.0 c8568f914cd2 9ed3f2200b5f 2024-03-21T10:51:17.506 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi153 smithi153 running (43s) 23s ago 54m 8002k - 19.0.0-2231-g2edfee68 237e43c7a65f 45ef129785aa 2024-03-21T10:51:17.506 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi174 smithi174 running (33s) 25s ago 52m 7117k - 19.0.0-2231-g2edfee68 237e43c7a65f 7212fd21b283 2024-03-21T10:51:17.506 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi153 smithi153 running (36m) 23s ago 54m 6656k - 19.0.0-2231-g2edfee68 237e43c7a65f 32c21104d5b0 2024-03-21T10:51:17.506 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi174 smithi174 running (36m) 25s ago 52m 6660k - 19.0.0-2231-g2edfee68 237e43c7a65f a04e48967ab6 2024-03-21T10:51:17.506 INFO:teuthology.orchestra.run.smithi153.stdout:grafana.smithi153 smithi153 *:3000 running (39m) 23s ago 53m 76.3M - 9.4.12 2bacad6d85d8 ee24026ab786 2024-03-21T10:51:17.506 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.iuyluj smithi153 running (31m) 23s ago 46m 61.3M - 19.0.0-2231-g2edfee68 237e43c7a65f 87aff8a5de70 2024-03-21T10:51:17.506 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.zipckr smithi153 running (32m) 23s ago 46m 62.7M - 19.0.0-2231-g2edfee68 237e43c7a65f ccc262bd5dd3 2024-03-21T10:51:17.506 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.nhozag smithi174 running (15m) 25s ago 46m 59.9M - 19.0.0-2231-g2edfee68 237e43c7a65f 9e02aea72c96 2024-03-21T10:51:17.506 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.yyimst smithi174 running (31m) 25s ago 46m 61.3M - 19.0.0-2231-g2edfee68 237e43c7a65f fc1962f9351e 2024-03-21T10:51:17.507 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi153.tluxni smithi153 *:8443,9283,8765 running (43m) 23s ago 55m 516M - 19.0.0-2231-g2edfee68 237e43c7a65f 7ffee0e55382 2024-03-21T10:51:17.507 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi174.qpcwat smithi174 *:8443,9283,8765 running (41m) 25s ago 52m 459M - 19.0.0-2231-g2edfee68 237e43c7a65f 748968f216ab 2024-03-21T10:51:17.507 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi153 smithi153 running (37m) 23s ago 55m 86.3M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f dd8038303dc3 2024-03-21T10:51:17.507 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi174 smithi174 running (37m) 25s ago 52m 59.6M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f 29d78a788f2c 2024-03-21T10:51:17.507 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi153 smithi153 *:9100 running (41m) 23s ago 54m 8338k - 1.5.0 0da6a335fe13 bb835edc0e0b 2024-03-21T10:51:17.507 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi174 smithi174 *:9100 running (41m) 25s ago 52m 8250k - 1.5.0 0da6a335fe13 7c4ecad790fa 2024-03-21T10:51:17.507 INFO:teuthology.orchestra.run.smithi153.stdout:osd.0 smithi153 running (35m) 23s ago 51m 113M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 507900fd4791 2024-03-21T10:51:17.507 INFO:teuthology.orchestra.run.smithi153.stdout:osd.1 smithi153 running (34m) 23s ago 50m 104M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 0d646452e9c1 2024-03-21T10:51:17.507 INFO:teuthology.orchestra.run.smithi153.stdout:osd.2 smithi153 running (34m) 23s ago 50m 99.5M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 1326f2afdfcd 2024-03-21T10:51:17.507 INFO:teuthology.orchestra.run.smithi153.stdout:osd.3 smithi174 running (33m) 25s ago 49m 98.8M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 70aafac71ad5 2024-03-21T10:51:17.507 INFO:teuthology.orchestra.run.smithi153.stdout:osd.4 smithi174 running (33m) 25s ago 49m 102M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 0d746ce6e79d 2024-03-21T10:51:17.508 INFO:teuthology.orchestra.run.smithi153.stdout:osd.5 smithi174 running (32m) 25s ago 48m 98.7M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 53e5f35b8cac 2024-03-21T10:51:17.508 INFO:teuthology.orchestra.run.smithi153.stdout:prometheus.smithi153 smithi153 *:9095 running (40m) 23s ago 53m 82.5M - 2.43.0 a07b618ecd1d 7d95561e50c2 2024-03-21T10:51:17.509 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:17.512+0000 7f9757000700 1 -- 172.21.15.153:0/1543710185 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f973406e460 msgr2=0x7f9734070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:51:17.509 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:17.512+0000 7f9757000700 1 --2- 172.21.15.153:0/1543710185 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f973406e460 0x7f9734070920 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f973c005fd0 tx=0x7f973c00a560 comp rx=0 tx=0).stop 2024-03-21T10:51:17.509 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:17.512+0000 7f9757000700 1 -- 172.21.15.153:0/1543710185 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9750100dc0 msgr2=0x7f9750198970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:51:17.509 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:17.512+0000 7f9757000700 1 --2- 172.21.15.153:0/1543710185 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9750100dc0 0x7f9750198970 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f974000b5c0 tx=0x7f974000be00 comp rx=0 tx=0).stop 2024-03-21T10:51:17.509 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:17.512+0000 7f9757000700 1 -- 172.21.15.153:0/1543710185 shutdown_connections 2024-03-21T10:51:17.509 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:17.512+0000 7f9757000700 1 --2- 172.21.15.153:0/1543710185 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f973406e460 0x7f9734070920 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:51:17.509 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:17.512+0000 7f9757000700 1 --2- 172.21.15.153:0/1543710185 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9750100dc0 0x7f9750198970 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:51:17.509 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:17.512+0000 7f9757000700 1 --2- 172.21.15.153:0/1543710185 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9750101770 0x7f9750198eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:51:17.509 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:17.512+0000 7f9757000700 1 -- 172.21.15.153:0/1543710185 >> 172.21.15.153:0/1543710185 conn(0x7f97500fc650 msgr2=0x7f97500fddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:51:17.509 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:17.512+0000 7f9757000700 1 -- 172.21.15.153:0/1543710185 shutdown_connections 2024-03-21T10:51:17.509 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:17.512+0000 7f9757000700 1 -- 172.21.15.153:0/1543710185 wait complete. 2024-03-21T10:51:17.696 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:17 smithi174 ceph-mon[88004]: pgmap v1185: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:51:17.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:17 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:51:17.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:17 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:51:18.296 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 070ed3e4-e769-11ee-95cd-87774f69a715 -e sha1=2edfee68ad7243bed422a049b6f5f58619300f11 -- bash -c 'ceph orch upgrade status' 2024-03-21T10:51:18.551 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:18 smithi153 ceph-mon[113733]: from='client.36086 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:51:18.696 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:18 smithi174 ceph-mon[88004]: from='client.36086 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:51:18.705 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi153/config 2024-03-21T10:51:19.551 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:19 smithi153 ceph-mon[113733]: pgmap v1186: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:51:19.696 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:19 smithi174 ceph-mon[88004]: pgmap v1186: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:51:20.799 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:20.802+0000 7fe8efc00700 1 -- 172.21.15.153:0/760117699 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe8e8073ae0 msgr2=0x7fe8e810ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:51:20.799 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:20.802+0000 7fe8efc00700 1 --2- 172.21.15.153:0/760117699 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe8e8073ae0 0x7fe8e810ce40 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fe8d4009910 tx=0x7fe8d4009c20 comp rx=0 tx=0).stop 2024-03-21T10:51:20.799 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:20.802+0000 7fe8efc00700 1 -- 172.21.15.153:0/760117699 shutdown_connections 2024-03-21T10:51:20.799 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:20.802+0000 7fe8efc00700 1 --2- 172.21.15.153:0/760117699 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe8e8073ae0 0x7fe8e810ce40 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:51:20.799 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:20.802+0000 7fe8efc00700 1 --2- 172.21.15.153:0/760117699 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe8e80731c0 0x7fe8e80735a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:51:20.799 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:20.802+0000 7fe8efc00700 1 -- 172.21.15.153:0/760117699 >> 172.21.15.153:0/760117699 conn(0x7fe8e80fc630 msgr2=0x7fe8e80fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:51:20.799 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:20.802+0000 7fe8efc00700 1 -- 172.21.15.153:0/760117699 shutdown_connections 2024-03-21T10:51:20.799 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:20.802+0000 7fe8efc00700 1 -- 172.21.15.153:0/760117699 wait complete. 2024-03-21T10:51:20.800 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:20.804+0000 7fe8efc00700 1 Processor -- start 2024-03-21T10:51:20.800 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:20.804+0000 7fe8efc00700 1 -- start start 2024-03-21T10:51:20.800 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:20.804+0000 7fe8efc00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe8e80731c0 0x7fe8e8198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:51:20.800 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:20.804+0000 7fe8efc00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe8e8073ae0 0x7fe8e8198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:51:20.800 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:20.804+0000 7fe8efc00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e8199530 con 0x7fe8e80731c0 2024-03-21T10:51:20.800 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:20.804+0000 7fe8efc00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e819d340 con 0x7fe8e8073ae0 2024-03-21T10:51:20.801 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:20.805+0000 7fe8ed200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe8e80731c0 0x7fe8e8198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:51:20.801 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:20.805+0000 7fe8ed200700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe8e80731c0 0x7fe8e8198960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39318/0 (socket says 172.21.15.153:39318) 2024-03-21T10:51:20.801 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:20.805+0000 7fe8ed200700 1 -- 172.21.15.153:0/2798147715 learned_addr learned my addr 172.21.15.153:0/2798147715 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:51:20.802 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:20.805+0000 7fe8e7e00700 1 --2- 172.21.15.153:0/2798147715 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe8e8073ae0 0x7fe8e8198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:51:20.802 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:20.805+0000 7fe8ed200700 1 -- 172.21.15.153:0/2798147715 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe8e8073ae0 msgr2=0x7fe8e8198ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:51:20.802 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:20.805+0000 7fe8ed200700 1 --2- 172.21.15.153:0/2798147715 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe8e8073ae0 0x7fe8e8198ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:51:20.802 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:20.805+0000 7fe8ed200700 1 -- 172.21.15.153:0/2798147715 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8d40095f0 con 0x7fe8e80731c0 2024-03-21T10:51:20.802 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:20.806+0000 7fe8ed200700 1 --2- 172.21.15.153:0/2798147715 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe8e80731c0 0x7fe8e8198960 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc00c8a0 tx=0x7fe8dc00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:51:20.802 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:20.806+0000 7fe8e5600700 1 -- 172.21.15.153:0/2798147715 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe8dc00cea0 con 0x7fe8e80731c0 2024-03-21T10:51:20.802 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:20.806+0000 7fe8e5600700 1 -- 172.21.15.153:0/2798147715 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe8dc004830 con 0x7fe8e80731c0 2024-03-21T10:51:20.802 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:20.806+0000 7fe8e5600700 1 -- 172.21.15.153:0/2798147715 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe8dc0055b0 con 0x7fe8e80731c0 2024-03-21T10:51:20.802 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:20.806+0000 7fe8efc00700 1 -- 172.21.15.153:0/2798147715 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8e819d620 con 0x7fe8e80731c0 2024-03-21T10:51:20.803 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:20.806+0000 7fe8efc00700 1 -- 172.21.15.153:0/2798147715 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8e819db70 con 0x7fe8e80731c0 2024-03-21T10:51:20.803 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:20.808+0000 7fe8efc00700 1 -- 172.21.15.153:0/2798147715 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8e810a540 con 0x7fe8e80731c0 2024-03-21T10:51:20.804 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:20.809+0000 7fe8e5600700 1 -- 172.21.15.153:0/2798147715 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fe8dc0049a0 con 0x7fe8e80731c0 2024-03-21T10:51:20.805 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:20.810+0000 7fe8e5600700 1 --2- 172.21.15.153:0/2798147715 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe8d006e460 0x7fe8d0070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:51:20.806 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:20.810+0000 7fe8e5600700 1 -- 172.21.15.153:0/2798147715 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fe8dc08e150 con 0x7fe8e80731c0 2024-03-21T10:51:20.806 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:20.810+0000 7fe8e7e00700 1 --2- 172.21.15.153:0/2798147715 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe8d006e460 0x7fe8d0070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:51:20.806 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:20.811+0000 7fe8e7e00700 1 --2- 172.21.15.153:0/2798147715 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe8d006e460 0x7fe8d0070920 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fe8d4006010 tx=0x7fe8d400b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:51:20.811 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:20.815+0000 7fe8e5600700 1 -- 172.21.15.153:0/2798147715 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe8dc05ad20 con 0x7fe8e80731c0 2024-03-21T10:51:21.100 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:21.105+0000 7fe8efc00700 1 -- 172.21.15.153:0/2798147715 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe8e819de40 con 0x7fe8d006e460 2024-03-21T10:51:21.102 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:21.106+0000 7fe8e5600700 1 -- 172.21.15.153:0/2798147715 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7fe8e819de40 con 0x7fe8d006e460 2024-03-21T10:51:21.102 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:51:21.102 INFO:teuthology.orchestra.run.smithi153.stdout: "target_image": null, 2024-03-21T10:51:21.102 INFO:teuthology.orchestra.run.smithi153.stdout: "in_progress": false, 2024-03-21T10:51:21.102 INFO:teuthology.orchestra.run.smithi153.stdout: "which": "", 2024-03-21T10:51:21.102 INFO:teuthology.orchestra.run.smithi153.stdout: "services_complete": [], 2024-03-21T10:51:21.102 INFO:teuthology.orchestra.run.smithi153.stdout: "progress": null, 2024-03-21T10:51:21.102 INFO:teuthology.orchestra.run.smithi153.stdout: "message": "", 2024-03-21T10:51:21.102 INFO:teuthology.orchestra.run.smithi153.stdout: "is_paused": false 2024-03-21T10:51:21.102 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:51:21.105 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:21.109+0000 7fe8efc00700 1 -- 172.21.15.153:0/2798147715 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe8d006e460 msgr2=0x7fe8d0070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:51:21.105 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:21.109+0000 7fe8efc00700 1 --2- 172.21.15.153:0/2798147715 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe8d006e460 0x7fe8d0070920 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fe8d4006010 tx=0x7fe8d400b540 comp rx=0 tx=0).stop 2024-03-21T10:51:21.105 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:21.109+0000 7fe8efc00700 1 -- 172.21.15.153:0/2798147715 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe8e80731c0 msgr2=0x7fe8e8198960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:51:21.105 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:21.109+0000 7fe8efc00700 1 --2- 172.21.15.153:0/2798147715 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe8e80731c0 0x7fe8e8198960 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc00c8a0 tx=0x7fe8dc00cc60 comp rx=0 tx=0).stop 2024-03-21T10:51:21.105 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:21.109+0000 7fe8efc00700 1 -- 172.21.15.153:0/2798147715 shutdown_connections 2024-03-21T10:51:21.105 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:21.109+0000 7fe8efc00700 1 --2- 172.21.15.153:0/2798147715 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe8d006e460 0x7fe8d0070920 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:51:21.106 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:21.109+0000 7fe8efc00700 1 --2- 172.21.15.153:0/2798147715 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe8e80731c0 0x7fe8e8198960 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:51:21.106 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:21.109+0000 7fe8efc00700 1 --2- 172.21.15.153:0/2798147715 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe8e8073ae0 0x7fe8e8198ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:51:21.106 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:21.109+0000 7fe8efc00700 1 -- 172.21.15.153:0/2798147715 >> 172.21.15.153:0/2798147715 conn(0x7fe8e80fc630 msgr2=0x7fe8e8107680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:51:21.106 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:21.109+0000 7fe8efc00700 1 -- 172.21.15.153:0/2798147715 shutdown_connections 2024-03-21T10:51:21.106 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:21.110+0000 7fe8efc00700 1 -- 172.21.15.153:0/2798147715 wait complete. 2024-03-21T10:51:21.508 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:21 smithi153 ceph-mon[113733]: pgmap v1187: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:51:21.696 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:21 smithi174 ceph-mon[88004]: pgmap v1187: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:51:21.897 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 070ed3e4-e769-11ee-95cd-87774f69a715 -e sha1=2edfee68ad7243bed422a049b6f5f58619300f11 -- bash -c 'ceph health detail' 2024-03-21T10:51:22.321 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi153/config 2024-03-21T10:51:22.551 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:22 smithi153 ceph-mon[113733]: from='client.36090 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:51:22.696 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:22 smithi174 ceph-mon[88004]: from='client.36090 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:51:23.551 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:23 smithi153 ceph-mon[113733]: pgmap v1188: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:51:23.696 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:23 smithi174 ceph-mon[88004]: pgmap v1188: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:51:24.478 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:24.481+0000 7f64f6e00700 1 -- 172.21.15.153:0/1299807070 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f64f0069020 msgr2=0x7f64f0105790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:51:24.478 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:24.481+0000 7f64f6e00700 1 --2- 172.21.15.153:0/1299807070 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f64f0069020 0x7f64f0105790 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f64dc009910 tx=0x7f64dc009c20 comp rx=0 tx=0).stop 2024-03-21T10:51:24.478 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:24.481+0000 7f64f6e00700 1 -- 172.21.15.153:0/1299807070 shutdown_connections 2024-03-21T10:51:24.478 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:24.481+0000 7f64f6e00700 1 --2- 172.21.15.153:0/1299807070 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f64f0069020 0x7f64f0105790 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:51:24.478 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:24.481+0000 7f64f6e00700 1 --2- 172.21.15.153:0/1299807070 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f64f0068670 0x7f64f0068a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:51:24.478 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:24.481+0000 7f64f6e00700 1 -- 172.21.15.153:0/1299807070 >> 172.21.15.153:0/1299807070 conn(0x7f64f00755a0 msgr2=0x7f64f00759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:51:24.478 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:24.481+0000 7f64f6e00700 1 -- 172.21.15.153:0/1299807070 shutdown_connections 2024-03-21T10:51:24.478 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:24.481+0000 7f64f6e00700 1 -- 172.21.15.153:0/1299807070 wait complete. 2024-03-21T10:51:24.479 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:24.483+0000 7f64f6e00700 1 Processor -- start 2024-03-21T10:51:24.479 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:24.484+0000 7f64f6e00700 1 -- start start 2024-03-21T10:51:24.480 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:24.484+0000 7f64f6e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f64f0068670 0x7f64f0071d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:51:24.480 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:24.484+0000 7f64f6e00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f64f0069020 0x7f64f0072260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:51:24.480 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:24.484+0000 7f64f6e00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64f0101ac0 con 0x7f64f0068670 2024-03-21T10:51:24.480 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:24.484+0000 7f64f6e00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64f0101c00 con 0x7f64f0069020 2024-03-21T10:51:24.480 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:24.485+0000 7f64efe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f64f0068670 0x7f64f0071d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:51:24.481 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:24.485+0000 7f64efe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f64f0068670 0x7f64f0071d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:60198/0 (socket says 172.21.15.153:60198) 2024-03-21T10:51:24.481 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:24.485+0000 7f64efe00700 1 -- 172.21.15.153:0/875145679 learned_addr learned my addr 172.21.15.153:0/875145679 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:51:24.481 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:24.485+0000 7f64ef400700 1 --2- 172.21.15.153:0/875145679 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f64f0069020 0x7f64f0072260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:51:24.481 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:24.486+0000 7f64efe00700 1 -- 172.21.15.153:0/875145679 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f64f0069020 msgr2=0x7f64f0072260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:51:24.481 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:24.486+0000 7f64efe00700 1 --2- 172.21.15.153:0/875145679 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f64f0069020 0x7f64f0072260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:51:24.482 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:24.486+0000 7f64efe00700 1 -- 172.21.15.153:0/875145679 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64dc0095f0 con 0x7f64f0068670 2024-03-21T10:51:24.482 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:24.486+0000 7f64efe00700 1 --2- 172.21.15.153:0/875145679 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f64f0068670 0x7f64f0071d20 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f64e000e920 tx=0x7f64e000ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:51:24.482 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:24.487+0000 7f64ecc00700 1 -- 172.21.15.153:0/875145679 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f64e000c9d0 con 0x7f64f0068670 2024-03-21T10:51:24.482 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:24.487+0000 7f64ecc00700 1 -- 172.21.15.153:0/875145679 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f64e0010430 con 0x7f64f0068670 2024-03-21T10:51:24.482 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:24.487+0000 7f64f6e00700 1 -- 172.21.15.153:0/875145679 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64f0072860 con 0x7f64f0068670 2024-03-21T10:51:24.482 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:24.487+0000 7f64ecc00700 1 -- 172.21.15.153:0/875145679 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f64e0018650 con 0x7f64f0068670 2024-03-21T10:51:24.483 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:24.487+0000 7f64f6e00700 1 -- 172.21.15.153:0/875145679 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64f01ab600 con 0x7f64f0068670 2024-03-21T10:51:24.484 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:24.488+0000 7f64f6e00700 1 -- 172.21.15.153:0/875145679 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64f01090a0 con 0x7f64f0068670 2024-03-21T10:51:24.485 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:24.490+0000 7f64ecc00700 1 -- 172.21.15.153:0/875145679 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f64e00105a0 con 0x7f64f0068670 2024-03-21T10:51:24.487 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:24.491+0000 7f64ecc00700 1 --2- 172.21.15.153:0/875145679 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f64d406e460 0x7f64d4070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:51:24.487 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:24.491+0000 7f64ecc00700 1 -- 172.21.15.153:0/875145679 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f64e0014070 con 0x7f64f0068670 2024-03-21T10:51:24.487 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:24.491+0000 7f64ef400700 1 --2- 172.21.15.153:0/875145679 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f64d406e460 0x7f64d4070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:51:24.489 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:24.492+0000 7f64ef400700 1 --2- 172.21.15.153:0/875145679 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f64d406e460 0x7f64d4070920 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f64dc009fd0 tx=0x7f64dc005a90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:51:24.492 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:24.496+0000 7f64ecc00700 1 -- 172.21.15.153:0/875145679 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f64e0053740 con 0x7f64f0068670 2024-03-21T10:51:24.819 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:24.823+0000 7f64f6e00700 1 -- 172.21.15.153:0/875145679 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f64f002cc70 con 0x7f64f0068670 2024-03-21T10:51:24.820 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:24.824+0000 7f64ecc00700 1 -- 172.21.15.153:0/875145679 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f64e005b3b0 con 0x7f64f0068670 2024-03-21T10:51:24.820 INFO:teuthology.orchestra.run.smithi153.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-03-21T10:51:24.820 INFO:teuthology.orchestra.run.smithi153.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-03-21T10:51:24.820 INFO:teuthology.orchestra.run.smithi153.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-03-21T10:51:24.823 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:24.828+0000 7f64f6e00700 1 -- 172.21.15.153:0/875145679 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f64d406e460 msgr2=0x7f64d4070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:51:24.824 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:24.828+0000 7f64f6e00700 1 --2- 172.21.15.153:0/875145679 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f64d406e460 0x7f64d4070920 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f64dc009fd0 tx=0x7f64dc005a90 comp rx=0 tx=0).stop 2024-03-21T10:51:24.824 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:24.828+0000 7f64f6e00700 1 -- 172.21.15.153:0/875145679 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f64f0068670 msgr2=0x7f64f0071d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:51:24.824 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:24.828+0000 7f64f6e00700 1 --2- 172.21.15.153:0/875145679 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f64f0068670 0x7f64f0071d20 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f64e000e920 tx=0x7f64e000ec30 comp rx=0 tx=0).stop 2024-03-21T10:51:24.824 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:24.828+0000 7f64f6e00700 1 -- 172.21.15.153:0/875145679 shutdown_connections 2024-03-21T10:51:24.824 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:24.828+0000 7f64f6e00700 1 --2- 172.21.15.153:0/875145679 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f64d406e460 0x7f64d4070920 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:51:24.824 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:24.828+0000 7f64f6e00700 1 --2- 172.21.15.153:0/875145679 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f64f0068670 0x7f64f0071d20 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:51:24.824 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:24.828+0000 7f64f6e00700 1 --2- 172.21.15.153:0/875145679 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f64f0069020 0x7f64f0072260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:51:24.825 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:24.828+0000 7f64f6e00700 1 -- 172.21.15.153:0/875145679 >> 172.21.15.153:0/875145679 conn(0x7f64f00755a0 msgr2=0x7f64f0105000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:51:24.825 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:24.829+0000 7f64f6e00700 1 -- 172.21.15.153:0/875145679 shutdown_connections 2024-03-21T10:51:24.825 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:24.829+0000 7f64f6e00700 1 -- 172.21.15.153:0/875145679 wait complete. 2024-03-21T10:51:25.511 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:25 smithi153 ceph-mon[113733]: pgmap v1189: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:51:25.511 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:25 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/875145679' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:51:25.607 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 070ed3e4-e769-11ee-95cd-87774f69a715 -e sha1=2edfee68ad7243bed422a049b6f5f58619300f11 -- bash -c 'ceph versions' 2024-03-21T10:51:25.644 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:25 smithi174 ceph-mon[88004]: pgmap v1189: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:51:25.644 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:25 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/875145679' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-21T10:51:26.002 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi153/config 2024-03-21T10:51:27.614 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:27 smithi153 ceph-mon[113733]: pgmap v1190: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:51:27.696 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:27 smithi174 ceph-mon[88004]: pgmap v1190: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:51:28.135 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:28.139+0000 7f05ff800700 1 -- 172.21.15.153:0/372382022 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f05f8068d30 msgr2=0x7f05f810d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:51:28.135 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:28.139+0000 7f05ff800700 1 --2- 172.21.15.153:0/372382022 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f05f8068d30 0x7f05f810d240 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f05e4009910 tx=0x7f05e4009c20 comp rx=0 tx=0).stop 2024-03-21T10:51:28.136 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:28.139+0000 7f05ff800700 1 -- 172.21.15.153:0/372382022 shutdown_connections 2024-03-21T10:51:28.136 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:28.139+0000 7f05ff800700 1 --2- 172.21.15.153:0/372382022 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f05f8068d30 0x7f05f810d240 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:51:28.136 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:28.139+0000 7f05ff800700 1 --2- 172.21.15.153:0/372382022 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f05f8068410 0x7f05f80687f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:51:28.136 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:28.139+0000 7f05ff800700 1 -- 172.21.15.153:0/372382022 >> 172.21.15.153:0/372382022 conn(0x7f05f80755a0 msgr2=0x7f05f80759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:51:28.136 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:28.139+0000 7f05ff800700 1 -- 172.21.15.153:0/372382022 shutdown_connections 2024-03-21T10:51:28.136 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:28.139+0000 7f05ff800700 1 -- 172.21.15.153:0/372382022 wait complete. 2024-03-21T10:51:28.136 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:28.141+0000 7f05ff800700 1 Processor -- start 2024-03-21T10:51:28.136 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:28.141+0000 7f05ff800700 1 -- start start 2024-03-21T10:51:28.137 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:28.142+0000 7f05ff800700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f05f8068410 0x7f05f819e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:51:28.137 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:28.142+0000 7f05ff800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f05f8068d30 0x7f05f819ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:51:28.138 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:28.142+0000 7f05ff800700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05f819f2b0 con 0x7f05f8068d30 2024-03-21T10:51:28.138 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:28.142+0000 7f05ff800700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05f8198760 con 0x7f05f8068410 2024-03-21T10:51:28.138 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:28.142+0000 7f05f7e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f05f8068d30 0x7f05f819ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:51:28.138 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:28.142+0000 7f05f7e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f05f8068d30 0x7f05f819ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:60218/0 (socket says 172.21.15.153:60218) 2024-03-21T10:51:28.138 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:28.142+0000 7f05f7e00700 1 -- 172.21.15.153:0/4136685292 learned_addr learned my addr 172.21.15.153:0/4136685292 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:51:28.138 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:28.142+0000 7f05fce00700 1 --2- 172.21.15.153:0/4136685292 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f05f8068410 0x7f05f819e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:51:28.138 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:28.143+0000 7f05f7e00700 1 -- 172.21.15.153:0/4136685292 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f05f8068410 msgr2=0x7f05f819e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:51:28.138 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:28.143+0000 7f05f7e00700 1 --2- 172.21.15.153:0/4136685292 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f05f8068410 0x7f05f819e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:51:28.138 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:28.143+0000 7f05f7e00700 1 -- 172.21.15.153:0/4136685292 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05e40095f0 con 0x7f05f8068d30 2024-03-21T10:51:28.139 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:28.143+0000 7f05f7e00700 1 --2- 172.21.15.153:0/4136685292 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f05f8068d30 0x7f05f819ec20 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f05e400b5c0 tx=0x7f05e400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:51:28.139 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:28.143+0000 7f05f5600700 1 -- 172.21.15.153:0/4136685292 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f05e401d070 con 0x7f05f8068d30 2024-03-21T10:51:28.139 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:28.143+0000 7f05f5600700 1 -- 172.21.15.153:0/4136685292 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f05e4005e00 con 0x7f05f8068d30 2024-03-21T10:51:28.139 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:28.143+0000 7f05ff800700 1 -- 172.21.15.153:0/4136685292 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05f8198a40 con 0x7f05f8068d30 2024-03-21T10:51:28.139 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:28.143+0000 7f05f5600700 1 -- 172.21.15.153:0/4136685292 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f05e4017400 con 0x7f05f8068d30 2024-03-21T10:51:28.140 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:28.144+0000 7f05ff800700 1 -- 172.21.15.153:0/4136685292 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05f8198f90 con 0x7f05f8068d30 2024-03-21T10:51:28.140 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:28.145+0000 7f05ff800700 1 -- 172.21.15.153:0/4136685292 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05f810a9b0 con 0x7f05f8068d30 2024-03-21T10:51:28.142 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:28.147+0000 7f05f5600700 1 -- 172.21.15.153:0/4136685292 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f05e400f460 con 0x7f05f8068d30 2024-03-21T10:51:28.143 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:28.148+0000 7f05f5600700 1 --2- 172.21.15.153:0/4136685292 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f05e006e460 0x7f05e0070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:51:28.143 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:28.148+0000 7f05f5600700 1 -- 172.21.15.153:0/4136685292 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f05e408f980 con 0x7f05f8068d30 2024-03-21T10:51:28.144 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:28.148+0000 7f05fce00700 1 --2- 172.21.15.153:0/4136685292 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f05e006e460 0x7f05e0070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:51:28.144 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:28.149+0000 7f05fce00700 1 --2- 172.21.15.153:0/4136685292 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f05e006e460 0x7f05e0070920 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f05f8076b00 tx=0x7f05ec009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:51:28.149 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:28.154+0000 7f05f5600700 1 -- 172.21.15.153:0/4136685292 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f05e405c550 con 0x7f05f8068d30 2024-03-21T10:51:28.499 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:28.503+0000 7f05ff800700 1 -- 172.21.15.153:0/4136685292 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f05f804ea90 con 0x7f05f8068d30 2024-03-21T10:51:28.500 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:28.504+0000 7f05f5600700 1 -- 172.21.15.153:0/4136685292 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f05e405c0e0 con 0x7f05f8068d30 2024-03-21T10:51:28.500 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:51:28.500 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": { 2024-03-21T10:51:28.500 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:51:28.500 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:51:28.501 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": { 2024-03-21T10:51:28.501 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:51:28.501 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:51:28.501 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": { 2024-03-21T10:51:28.501 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 6 2024-03-21T10:51:28.501 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:51:28.501 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": { 2024-03-21T10:51:28.501 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 4 2024-03-21T10:51:28.501 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:51:28.501 INFO:teuthology.orchestra.run.smithi153.stdout: "overall": { 2024-03-21T10:51:28.501 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 14 2024-03-21T10:51:28.501 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-03-21T10:51:28.501 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:51:28.503 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:28.507+0000 7f05ff800700 1 -- 172.21.15.153:0/4136685292 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f05e006e460 msgr2=0x7f05e0070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:51:28.503 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:28.507+0000 7f05ff800700 1 --2- 172.21.15.153:0/4136685292 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f05e006e460 0x7f05e0070920 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f05f8076b00 tx=0x7f05ec009040 comp rx=0 tx=0).stop 2024-03-21T10:51:28.503 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:28.507+0000 7f05ff800700 1 -- 172.21.15.153:0/4136685292 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f05f8068d30 msgr2=0x7f05f819ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:51:28.503 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:28.507+0000 7f05ff800700 1 --2- 172.21.15.153:0/4136685292 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f05f8068d30 0x7f05f819ec20 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f05e400b5c0 tx=0x7f05e400be20 comp rx=0 tx=0).stop 2024-03-21T10:51:28.503 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:28.507+0000 7f05ff800700 1 -- 172.21.15.153:0/4136685292 shutdown_connections 2024-03-21T10:51:28.503 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:28.507+0000 7f05ff800700 1 --2- 172.21.15.153:0/4136685292 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f05f8068410 0x7f05f819e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:51:28.503 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:28.507+0000 7f05ff800700 1 --2- 172.21.15.153:0/4136685292 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f05e006e460 0x7f05e0070920 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:51:28.503 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:28.507+0000 7f05ff800700 1 --2- 172.21.15.153:0/4136685292 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f05f8068d30 0x7f05f819ec20 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:51:28.504 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:28.508+0000 7f05ff800700 1 -- 172.21.15.153:0/4136685292 >> 172.21.15.153:0/4136685292 conn(0x7f05f80755a0 msgr2=0x7f05f80fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:51:28.504 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:28.508+0000 7f05ff800700 1 -- 172.21.15.153:0/4136685292 shutdown_connections 2024-03-21T10:51:28.504 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:51:28.508+0000 7f05ff800700 1 -- 172.21.15.153:0/4136685292 wait complete. 2024-03-21T10:51:29.209 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 070ed3e4-e769-11ee-95cd-87774f69a715 -e sha1=2edfee68ad7243bed422a049b6f5f58619300f11 -- bash -c 'echo "wait for servicemap items w/ changing names to refresh"' 2024-03-21T10:51:29.551 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:29 smithi153 ceph-mon[113733]: pgmap v1191: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:51:29.551 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:29 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/4136685292' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:51:29.629 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi153/config 2024-03-21T10:51:29.696 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:29 smithi174 ceph-mon[88004]: pgmap v1191: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:51:29.696 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:29 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/4136685292' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:51:31.597 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:31 smithi153 ceph-mon[113733]: pgmap v1192: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:51:31.597 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:31 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:51:31.696 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:31 smithi174 ceph-mon[88004]: pgmap v1192: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:51:31.696 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:31 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:51:31.826 INFO:teuthology.orchestra.run.smithi153.stdout:wait for servicemap items w/ changing names to refresh 2024-03-21T10:51:32.562 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 070ed3e4-e769-11ee-95cd-87774f69a715 -e sha1=2edfee68ad7243bed422a049b6f5f58619300f11 -- bash -c 'sleep 60' 2024-03-21T10:51:32.976 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi153/config 2024-03-21T10:51:33.375 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:33 smithi153 ceph-mon[113733]: pgmap v1193: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:51:33.696 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:33 smithi174 ceph-mon[88004]: pgmap v1193: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:51:35.696 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:35 smithi174 ceph-mon[88004]: pgmap v1194: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:51:35.751 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:35 smithi153 ceph-mon[113733]: pgmap v1194: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:51:37.696 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:37 smithi174 ceph-mon[88004]: pgmap v1195: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:51:37.801 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:37 smithi153 ceph-mon[113733]: pgmap v1195: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:51:39.696 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:39 smithi174 ceph-mon[88004]: pgmap v1196: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:51:39.801 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:39 smithi153 ceph-mon[113733]: pgmap v1196: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:51:41.696 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:41 smithi174 ceph-mon[88004]: pgmap v1197: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:51:41.801 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:41 smithi153 ceph-mon[113733]: pgmap v1197: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:51:43.696 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:43 smithi174 ceph-mon[88004]: pgmap v1198: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:51:43.801 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:43 smithi153 ceph-mon[113733]: pgmap v1198: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:51:45.696 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:45 smithi174 ceph-mon[88004]: pgmap v1199: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:51:45.801 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:45 smithi153 ceph-mon[113733]: pgmap v1199: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:51:46.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:46 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:51:46.801 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:46 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:51:47.696 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:47 smithi174 ceph-mon[88004]: pgmap v1200: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:51:47.801 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:47 smithi153 ceph-mon[113733]: pgmap v1200: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:51:49.696 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:49 smithi174 ceph-mon[88004]: pgmap v1201: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:51:49.801 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:49 smithi153 ceph-mon[113733]: pgmap v1201: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:51:51.696 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:51 smithi174 ceph-mon[88004]: pgmap v1202: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:51:51.801 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:51 smithi153 ceph-mon[113733]: pgmap v1202: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:51:53.675 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:53 smithi153 ceph-mon[113733]: pgmap v1203: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:51:53.696 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:53 smithi174 ceph-mon[88004]: pgmap v1203: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:51:55.696 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:55 smithi174 ceph-mon[88004]: pgmap v1204: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:51:55.801 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:55 smithi153 ceph-mon[113733]: pgmap v1204: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:51:57.696 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:57 smithi174 ceph-mon[88004]: pgmap v1205: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:51:57.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:57 smithi153 ceph-mon[113733]: pgmap v1205: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:51:59.696 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:51:59 smithi174 ceph-mon[88004]: pgmap v1206: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:51:59.801 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:51:59 smithi153 ceph-mon[113733]: pgmap v1206: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:52:01.696 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:52:01 smithi174 ceph-mon[88004]: pgmap v1207: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:52:01.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:52:01 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:52:01.798 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:52:01 smithi153 ceph-mon[113733]: pgmap v1207: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:52:01.798 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:52:01 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:52:03.801 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:52:03 smithi153 ceph-mon[113733]: pgmap v1208: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:52:03.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:52:03 smithi174 ceph-mon[88004]: pgmap v1208: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:52:04.732 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:52:04 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:52:04.946 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:52:04 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:52:05.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:52:05 smithi153 ceph-mon[113733]: pgmap v1209: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:52:05.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:52:05 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:52:05.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:52:05 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:52:05.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:52:05 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:52:05.946 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:52:05 smithi174 ceph-mon[88004]: pgmap v1209: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:52:05.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:52:05 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:52:05.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:52:05 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:52:05.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:52:05 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:52:07.801 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:52:07 smithi153 ceph-mon[113733]: pgmap v1210: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:52:07.946 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:52:07 smithi174 ceph-mon[88004]: pgmap v1210: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:52:09.749 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:52:09 smithi174 ceph-mon[88004]: pgmap v1211: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:52:09.801 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:52:09 smithi153 ceph-mon[113733]: pgmap v1211: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:52:11.801 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:52:11 smithi153 ceph-mon[113733]: pgmap v1212: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:52:11.946 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:52:11 smithi174 ceph-mon[88004]: pgmap v1212: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:52:13.801 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:52:13 smithi153 ceph-mon[113733]: pgmap v1213: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:52:13.946 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:52:13 smithi174 ceph-mon[88004]: pgmap v1213: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:52:15.801 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:52:15 smithi153 ceph-mon[113733]: pgmap v1214: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:52:15.946 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:52:15 smithi174 ceph-mon[88004]: pgmap v1214: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:52:16.801 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:52:16 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:52:16.946 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:52:16 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:52:17.696 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:52:17 smithi174 ceph-mon[88004]: pgmap v1215: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:52:17.801 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:52:17 smithi153 ceph-mon[113733]: pgmap v1215: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:52:19.801 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:52:19 smithi153 ceph-mon[113733]: pgmap v1216: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:52:19.946 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:52:19 smithi174 ceph-mon[88004]: pgmap v1216: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:52:21.801 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:52:21 smithi153 ceph-mon[113733]: pgmap v1217: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:52:21.946 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:52:21 smithi174 ceph-mon[88004]: pgmap v1217: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:52:23.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:52:23 smithi153 ceph-mon[113733]: pgmap v1218: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:52:23.946 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:52:23 smithi174 ceph-mon[88004]: pgmap v1218: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:52:25.801 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:52:25 smithi153 ceph-mon[113733]: pgmap v1219: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:52:25.946 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:52:25 smithi174 ceph-mon[88004]: pgmap v1219: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:52:27.801 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:52:27 smithi153 ceph-mon[113733]: pgmap v1220: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:52:27.946 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:52:27 smithi174 ceph-mon[88004]: pgmap v1220: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:52:29.801 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:52:29 smithi153 ceph-mon[113733]: pgmap v1221: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:52:29.946 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:52:29 smithi174 ceph-mon[88004]: pgmap v1221: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:52:31.801 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:52:31 smithi153 ceph-mon[113733]: pgmap v1222: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:52:31.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:52:31 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:52:31.946 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:52:31 smithi174 ceph-mon[88004]: pgmap v1222: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:52:31.946 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:52:31 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:52:33.946 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:52:33 smithi174 ceph-mon[88004]: pgmap v1223: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:52:34.051 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:52:33 smithi153 ceph-mon[113733]: pgmap v1223: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:52:35.727 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:52:35 smithi153 ceph-mon[113733]: pgmap v1224: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:52:35.770 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 070ed3e4-e769-11ee-95cd-87774f69a715 -e sha1=2edfee68ad7243bed422a049b6f5f58619300f11 -- bash -c 'ceph orch ps' 2024-03-21T10:52:35.946 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:52:35 smithi174 ceph-mon[88004]: pgmap v1224: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:52:36.176 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi153/config 2024-03-21T10:52:37.801 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:52:37 smithi153 ceph-mon[113733]: pgmap v1225: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:52:37.946 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:52:37 smithi174 ceph-mon[88004]: pgmap v1225: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:52:38.341 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:38.343+0000 7f870fe00700 1 -- 172.21.15.153:0/28516020 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8710100970 msgr2=0x7f8710100df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:52:38.341 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:38.343+0000 7f870fe00700 1 --2- 172.21.15.153:0/28516020 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8710100970 0x7f8710100df0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f86fc009910 tx=0x7f86fc009c20 comp rx=0 tx=0).stop 2024-03-21T10:52:38.341 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:38.343+0000 7f870fe00700 1 -- 172.21.15.153:0/28516020 shutdown_connections 2024-03-21T10:52:38.341 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:38.343+0000 7f870fe00700 1 --2- 172.21.15.153:0/28516020 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8710100970 0x7f8710100df0 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:52:38.341 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:38.343+0000 7f870fe00700 1 --2- 172.21.15.153:0/28516020 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8710106970 0x7f8710106d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:52:38.341 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:38.343+0000 7f870fe00700 1 -- 172.21.15.153:0/28516020 >> 172.21.15.153:0/28516020 conn(0x7f87100fc440 msgr2=0x7f87100fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:52:38.341 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:38.344+0000 7f870fe00700 1 -- 172.21.15.153:0/28516020 shutdown_connections 2024-03-21T10:52:38.341 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:38.344+0000 7f870fe00700 1 -- 172.21.15.153:0/28516020 wait complete. 2024-03-21T10:52:38.343 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:38.345+0000 7f870fe00700 1 Processor -- start 2024-03-21T10:52:38.343 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:38.346+0000 7f870fe00700 1 -- start start 2024-03-21T10:52:38.343 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:38.346+0000 7f870fe00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8710100970 0x7f871019e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:52:38.343 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:38.346+0000 7f870fe00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8710106970 0x7f871019eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:52:38.343 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:38.346+0000 7f870fe00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f871019f210 con 0x7f8710106970 2024-03-21T10:52:38.343 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:38.346+0000 7f870fe00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87101986c0 con 0x7f8710100970 2024-03-21T10:52:38.344 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:38.347+0000 7f870e000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8710106970 0x7f871019eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:52:38.345 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:38.347+0000 7f870e000700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8710106970 0x7f871019eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:37400/0 (socket says 172.21.15.153:37400) 2024-03-21T10:52:38.345 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:38.347+0000 7f870e000700 1 -- 172.21.15.153:0/4083108378 learned_addr learned my addr 172.21.15.153:0/4083108378 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:52:38.345 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:38.347+0000 7f870ea00700 1 --2- 172.21.15.153:0/4083108378 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8710100970 0x7f871019e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:52:38.345 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:38.347+0000 7f870e000700 1 -- 172.21.15.153:0/4083108378 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8710100970 msgr2=0x7f871019e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:52:38.345 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:38.347+0000 7f870e000700 1 --2- 172.21.15.153:0/4083108378 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8710100970 0x7f871019e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:52:38.345 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:38.347+0000 7f870e000700 1 -- 172.21.15.153:0/4083108378 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86fc0095f0 con 0x7f8710106970 2024-03-21T10:52:38.345 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:38.348+0000 7f870e000700 1 --2- 172.21.15.153:0/4083108378 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8710106970 0x7f871019eb80 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f86fc00b5c0 tx=0x7f86fc00be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:52:38.345 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:38.348+0000 7f8707400700 1 -- 172.21.15.153:0/4083108378 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f86fc01d070 con 0x7f8710106970 2024-03-21T10:52:38.345 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:38.348+0000 7f8707400700 1 -- 172.21.15.153:0/4083108378 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f86fc00f460 con 0x7f8710106970 2024-03-21T10:52:38.345 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:38.348+0000 7f8707400700 1 -- 172.21.15.153:0/4083108378 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f86fc017610 con 0x7f8710106970 2024-03-21T10:52:38.345 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:38.348+0000 7f870fe00700 1 -- 172.21.15.153:0/4083108378 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8710198940 con 0x7f8710106970 2024-03-21T10:52:38.346 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:38.348+0000 7f870fe00700 1 -- 172.21.15.153:0/4083108378 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8710198e30 con 0x7f8710106970 2024-03-21T10:52:38.346 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:38.350+0000 7f870fe00700 1 -- 172.21.15.153:0/4083108378 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f871004f2e0 con 0x7f8710106970 2024-03-21T10:52:38.347 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:38.351+0000 7f8707400700 1 -- 172.21.15.153:0/4083108378 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f86fc00f5d0 con 0x7f8710106970 2024-03-21T10:52:38.349 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:38.353+0000 7f8707400700 1 --2- 172.21.15.153:0/4083108378 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f86f406e460 0x7f86f4070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:52:38.349 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:38.353+0000 7f8707400700 1 -- 172.21.15.153:0/4083108378 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f86fc08fa30 con 0x7f8710106970 2024-03-21T10:52:38.350 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:38.353+0000 7f870ea00700 1 --2- 172.21.15.153:0/4083108378 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f86f406e460 0x7f86f4070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:52:38.351 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:38.354+0000 7f870ea00700 1 --2- 172.21.15.153:0/4083108378 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f86f406e460 0x7f86f4070920 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f8700006fd0 tx=0x7f8700008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:52:38.356 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:38.360+0000 7f8707400700 1 -- 172.21.15.153:0/4083108378 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f86fc05c6a0 con 0x7f8710106970 2024-03-21T10:52:38.590 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:38.593+0000 7f870fe00700 1 -- 172.21.15.153:0/4083108378 --> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f8710199250 con 0x7f86f406e460 2024-03-21T10:52:38.618 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:38.621+0000 7f8707400700 1 -- 172.21.15.153:0/4083108378 <== mgr.34104 v2:172.21.15.153:6800/3112920964 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f8710199250 con 0x7f86f406e460 2024-03-21T10:52:38.618 INFO:teuthology.orchestra.run.smithi153.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-21T10:52:38.618 INFO:teuthology.orchestra.run.smithi153.stdout:alertmanager.smithi153 smithi153 *:9093,9094 running (41m) 104s ago 55m 14.6M - 0.25.0 c8568f914cd2 9ed3f2200b5f 2024-03-21T10:52:38.618 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi153 smithi153 running (2m) 104s ago 55m 8002k - 19.0.0-2231-g2edfee68 237e43c7a65f 45ef129785aa 2024-03-21T10:52:38.618 INFO:teuthology.orchestra.run.smithi153.stdout:ceph-exporter.smithi174 smithi174 running (114s) 106s ago 53m 7117k - 19.0.0-2231-g2edfee68 237e43c7a65f 7212fd21b283 2024-03-21T10:52:38.618 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi153 smithi153 running (37m) 104s ago 55m 6656k - 19.0.0-2231-g2edfee68 237e43c7a65f 32c21104d5b0 2024-03-21T10:52:38.619 INFO:teuthology.orchestra.run.smithi153.stdout:crash.smithi174 smithi174 running (37m) 106s ago 53m 6660k - 19.0.0-2231-g2edfee68 237e43c7a65f a04e48967ab6 2024-03-21T10:52:38.619 INFO:teuthology.orchestra.run.smithi153.stdout:grafana.smithi153 smithi153 *:3000 running (40m) 104s ago 54m 76.3M - 9.4.12 2bacad6d85d8 ee24026ab786 2024-03-21T10:52:38.619 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.iuyluj smithi153 running (32m) 104s ago 47m 61.3M - 19.0.0-2231-g2edfee68 237e43c7a65f 87aff8a5de70 2024-03-21T10:52:38.619 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi153.zipckr smithi153 running (33m) 104s ago 48m 62.7M - 19.0.0-2231-g2edfee68 237e43c7a65f ccc262bd5dd3 2024-03-21T10:52:38.619 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.nhozag smithi174 running (17m) 106s ago 47m 59.9M - 19.0.0-2231-g2edfee68 237e43c7a65f 9e02aea72c96 2024-03-21T10:52:38.619 INFO:teuthology.orchestra.run.smithi153.stdout:mds.cephfs.smithi174.yyimst smithi174 running (32m) 106s ago 48m 61.3M - 19.0.0-2231-g2edfee68 237e43c7a65f fc1962f9351e 2024-03-21T10:52:38.619 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi153.tluxni smithi153 *:8443,9283,8765 running (44m) 104s ago 57m 516M - 19.0.0-2231-g2edfee68 237e43c7a65f 7ffee0e55382 2024-03-21T10:52:38.619 INFO:teuthology.orchestra.run.smithi153.stdout:mgr.smithi174.qpcwat smithi174 *:8443,9283,8765 running (43m) 106s ago 53m 459M - 19.0.0-2231-g2edfee68 237e43c7a65f 748968f216ab 2024-03-21T10:52:38.619 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi153 smithi153 running (39m) 104s ago 57m 86.3M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f dd8038303dc3 2024-03-21T10:52:38.619 INFO:teuthology.orchestra.run.smithi153.stdout:mon.smithi174 smithi174 running (38m) 106s ago 53m 59.6M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f 29d78a788f2c 2024-03-21T10:52:38.619 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi153 smithi153 *:9100 running (42m) 104s ago 55m 8338k - 1.5.0 0da6a335fe13 bb835edc0e0b 2024-03-21T10:52:38.619 INFO:teuthology.orchestra.run.smithi153.stdout:node-exporter.smithi174 smithi174 *:9100 running (42m) 106s ago 53m 8250k - 1.5.0 0da6a335fe13 7c4ecad790fa 2024-03-21T10:52:38.619 INFO:teuthology.orchestra.run.smithi153.stdout:osd.0 smithi153 running (37m) 104s ago 52m 113M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 507900fd4791 2024-03-21T10:52:38.619 INFO:teuthology.orchestra.run.smithi153.stdout:osd.1 smithi153 running (36m) 104s ago 52m 104M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 0d646452e9c1 2024-03-21T10:52:38.619 INFO:teuthology.orchestra.run.smithi153.stdout:osd.2 smithi153 running (35m) 104s ago 51m 99.5M 1217M 19.0.0-2231-g2edfee68 237e43c7a65f 1326f2afdfcd 2024-03-21T10:52:38.620 INFO:teuthology.orchestra.run.smithi153.stdout:osd.3 smithi174 running (35m) 106s ago 50m 98.8M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 70aafac71ad5 2024-03-21T10:52:38.620 INFO:teuthology.orchestra.run.smithi153.stdout:osd.4 smithi174 running (34m) 106s ago 50m 102M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 0d746ce6e79d 2024-03-21T10:52:38.620 INFO:teuthology.orchestra.run.smithi153.stdout:osd.5 smithi174 running (33m) 106s ago 49m 98.7M 2241M 19.0.0-2231-g2edfee68 237e43c7a65f 53e5f35b8cac 2024-03-21T10:52:38.620 INFO:teuthology.orchestra.run.smithi153.stdout:prometheus.smithi153 smithi153 *:9095 running (41m) 104s ago 54m 82.5M - 2.43.0 a07b618ecd1d 7d95561e50c2 2024-03-21T10:52:38.621 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:38.625+0000 7f870fe00700 1 -- 172.21.15.153:0/4083108378 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f86f406e460 msgr2=0x7f86f4070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:52:38.621 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:38.625+0000 7f870fe00700 1 --2- 172.21.15.153:0/4083108378 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f86f406e460 0x7f86f4070920 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f8700006fd0 tx=0x7f8700008040 comp rx=0 tx=0).stop 2024-03-21T10:52:38.621 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:38.625+0000 7f870fe00700 1 -- 172.21.15.153:0/4083108378 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8710106970 msgr2=0x7f871019eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:52:38.621 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:38.625+0000 7f870fe00700 1 --2- 172.21.15.153:0/4083108378 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8710106970 0x7f871019eb80 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f86fc00b5c0 tx=0x7f86fc00be00 comp rx=0 tx=0).stop 2024-03-21T10:52:38.622 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:38.625+0000 7f870fe00700 1 -- 172.21.15.153:0/4083108378 shutdown_connections 2024-03-21T10:52:38.622 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:38.625+0000 7f870fe00700 1 --2- 172.21.15.153:0/4083108378 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8710100970 0x7f871019e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:52:38.622 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:38.625+0000 7f870fe00700 1 --2- 172.21.15.153:0/4083108378 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f86f406e460 0x7f86f4070920 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:52:38.622 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:38.625+0000 7f870fe00700 1 --2- 172.21.15.153:0/4083108378 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8710106970 0x7f871019eb80 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:52:38.622 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:38.625+0000 7f870fe00700 1 -- 172.21.15.153:0/4083108378 >> 172.21.15.153:0/4083108378 conn(0x7f87100fc440 msgr2=0x7f8710072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:52:38.622 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:38.625+0000 7f870fe00700 1 -- 172.21.15.153:0/4083108378 shutdown_connections 2024-03-21T10:52:38.623 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:38.625+0000 7f870fe00700 1 -- 172.21.15.153:0/4083108378 wait complete. 2024-03-21T10:52:39.368 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 070ed3e4-e769-11ee-95cd-87774f69a715 -e sha1=2edfee68ad7243bed422a049b6f5f58619300f11 -- bash -c 'ceph versions' 2024-03-21T10:52:39.785 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi153/config 2024-03-21T10:52:39.801 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:52:39 smithi153 ceph-mon[113733]: pgmap v1226: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:52:39.801 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:52:39 smithi153 ceph-mon[113733]: from='client.36102 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:52:39.946 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:52:39 smithi174 ceph-mon[88004]: pgmap v1226: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:52:39.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:52:39 smithi174 ceph-mon[88004]: from='client.36102 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T10:52:41.867 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:41.870+0000 7fe341e00700 1 -- 172.21.15.153:0/2634901774 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe33c068410 msgr2=0x7fe33c0687f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:52:41.868 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:41.870+0000 7fe341e00700 1 --2- 172.21.15.153:0/2634901774 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe33c068410 0x7fe33c0687f0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fe330009910 tx=0x7fe330009c20 comp rx=0 tx=0).stop 2024-03-21T10:52:41.868 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:52:41 smithi153 ceph-mon[113733]: pgmap v1227: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:52:41.868 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:41.870+0000 7fe341e00700 1 -- 172.21.15.153:0/2634901774 shutdown_connections 2024-03-21T10:52:41.868 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:41.870+0000 7fe341e00700 1 --2- 172.21.15.153:0/2634901774 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe33c068d30 0x7fe33c10d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:52:41.869 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:41.870+0000 7fe341e00700 1 --2- 172.21.15.153:0/2634901774 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe33c068410 0x7fe33c0687f0 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:52:41.869 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:41.870+0000 7fe341e00700 1 -- 172.21.15.153:0/2634901774 >> 172.21.15.153:0/2634901774 conn(0x7fe33c075330 msgr2=0x7fe33c075740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:52:41.869 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:41.871+0000 7fe341e00700 1 -- 172.21.15.153:0/2634901774 shutdown_connections 2024-03-21T10:52:41.869 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:41.871+0000 7fe341e00700 1 -- 172.21.15.153:0/2634901774 wait complete. 2024-03-21T10:52:41.869 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:41.872+0000 7fe341e00700 1 Processor -- start 2024-03-21T10:52:41.870 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:41.872+0000 7fe341e00700 1 -- start start 2024-03-21T10:52:41.870 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:41.873+0000 7fe341e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe33c068410 0x7fe33c19e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:52:41.870 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:41.873+0000 7fe341e00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe33c068d30 0x7fe33c19eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:52:41.870 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:41.873+0000 7fe341e00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe33c19f0f0 con 0x7fe33c068410 2024-03-21T10:52:41.870 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:41.873+0000 7fe341e00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe33c198630 con 0x7fe33c068d30 2024-03-21T10:52:41.870 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:41.873+0000 7fe340a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe33c068410 0x7fe33c19e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:52:41.870 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:41.873+0000 7fe340a00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe33c068410 0x7fe33c19e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:37428/0 (socket says 172.21.15.153:37428) 2024-03-21T10:52:41.870 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:41.873+0000 7fe340a00700 1 -- 172.21.15.153:0/2924663762 learned_addr learned my addr 172.21.15.153:0/2924663762 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:52:41.872 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:41.874+0000 7fe33be00700 1 --2- 172.21.15.153:0/2924663762 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe33c068d30 0x7fe33c19eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:52:41.872 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:41.874+0000 7fe340a00700 1 -- 172.21.15.153:0/2924663762 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe33c068d30 msgr2=0x7fe33c19eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:52:41.872 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:41.874+0000 7fe340a00700 1 --2- 172.21.15.153:0/2924663762 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe33c068d30 0x7fe33c19eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:52:41.872 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:41.874+0000 7fe340a00700 1 -- 172.21.15.153:0/2924663762 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3300095f0 con 0x7fe33c068410 2024-03-21T10:52:41.872 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:41.874+0000 7fe340a00700 1 --2- 172.21.15.153:0/2924663762 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe33c068410 0x7fe33c19e5b0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fe33000bda0 tx=0x7fe33000be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:52:41.872 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:41.875+0000 7fe339600700 1 -- 172.21.15.153:0/2924663762 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe33001d070 con 0x7fe33c068410 2024-03-21T10:52:41.872 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:41.875+0000 7fe339600700 1 -- 172.21.15.153:0/2924663762 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe330004fb0 con 0x7fe33c068410 2024-03-21T10:52:41.872 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:41.875+0000 7fe341e00700 1 -- 172.21.15.153:0/2924663762 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe33c198910 con 0x7fe33c068410 2024-03-21T10:52:41.872 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:41.875+0000 7fe339600700 1 -- 172.21.15.153:0/2924663762 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe33000f460 con 0x7fe33c068410 2024-03-21T10:52:41.872 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:41.875+0000 7fe341e00700 1 -- 172.21.15.153:0/2924663762 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe33c198e60 con 0x7fe33c068410 2024-03-21T10:52:41.873 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:41.876+0000 7fe341e00700 1 -- 172.21.15.153:0/2924663762 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe33c10a7f0 con 0x7fe33c068410 2024-03-21T10:52:41.874 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:41.877+0000 7fe339600700 1 -- 172.21.15.153:0/2924663762 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fe330022800 con 0x7fe33c068410 2024-03-21T10:52:41.875 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:41.878+0000 7fe339600700 1 --2- 172.21.15.153:0/2924663762 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe32406e460 0x7fe324070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:52:41.875 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:41.878+0000 7fe339600700 1 -- 172.21.15.153:0/2924663762 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fe33008f900 con 0x7fe33c068410 2024-03-21T10:52:41.875 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:41.879+0000 7fe33be00700 1 --2- 172.21.15.153:0/2924663762 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe32406e460 0x7fe324070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:52:41.876 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:41.879+0000 7fe33be00700 1 --2- 172.21.15.153:0/2924663762 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe32406e460 0x7fe324070920 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fe328005ea0 tx=0x7fe328005e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:52:41.880 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:41.884+0000 7fe339600700 1 -- 172.21.15.153:0/2924663762 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe33005c620 con 0x7fe33c068410 2024-03-21T10:52:41.946 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:52:41 smithi174 ceph-mon[88004]: pgmap v1227: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:52:42.191 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:42.195+0000 7fe341e00700 1 -- 172.21.15.153:0/2924663762 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fe33c04f2e0 con 0x7fe33c068410 2024-03-21T10:52:42.192 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:42.195+0000 7fe339600700 1 -- 172.21.15.153:0/2924663762 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fe33005c1b0 con 0x7fe33c068410 2024-03-21T10:52:42.193 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2024-03-21T10:52:42.193 INFO:teuthology.orchestra.run.smithi153.stdout: "mon": { 2024-03-21T10:52:42.193 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:52:42.193 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:52:42.193 INFO:teuthology.orchestra.run.smithi153.stdout: "mgr": { 2024-03-21T10:52:42.193 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 2 2024-03-21T10:52:42.194 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:52:42.194 INFO:teuthology.orchestra.run.smithi153.stdout: "osd": { 2024-03-21T10:52:42.194 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 6 2024-03-21T10:52:42.194 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:52:42.194 INFO:teuthology.orchestra.run.smithi153.stdout: "mds": { 2024-03-21T10:52:42.194 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 4 2024-03-21T10:52:42.195 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2024-03-21T10:52:42.195 INFO:teuthology.orchestra.run.smithi153.stdout: "overall": { 2024-03-21T10:52:42.195 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)": 14 2024-03-21T10:52:42.195 INFO:teuthology.orchestra.run.smithi153.stdout: } 2024-03-21T10:52:42.195 INFO:teuthology.orchestra.run.smithi153.stdout:} 2024-03-21T10:52:42.197 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:42.199+0000 7fe341e00700 1 -- 172.21.15.153:0/2924663762 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe32406e460 msgr2=0x7fe324070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:52:42.197 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:42.199+0000 7fe341e00700 1 --2- 172.21.15.153:0/2924663762 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe32406e460 0x7fe324070920 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fe328005ea0 tx=0x7fe328005e30 comp rx=0 tx=0).stop 2024-03-21T10:52:42.197 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:42.199+0000 7fe341e00700 1 -- 172.21.15.153:0/2924663762 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe33c068410 msgr2=0x7fe33c19e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:52:42.197 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:42.199+0000 7fe341e00700 1 --2- 172.21.15.153:0/2924663762 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe33c068410 0x7fe33c19e5b0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fe33000bda0 tx=0x7fe33000be80 comp rx=0 tx=0).stop 2024-03-21T10:52:42.198 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:42.199+0000 7fe341e00700 1 -- 172.21.15.153:0/2924663762 shutdown_connections 2024-03-21T10:52:42.198 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:42.199+0000 7fe341e00700 1 --2- 172.21.15.153:0/2924663762 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe32406e460 0x7fe324070920 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:52:42.198 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:42.199+0000 7fe341e00700 1 --2- 172.21.15.153:0/2924663762 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe33c068410 0x7fe33c19e5b0 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:52:42.198 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:42.199+0000 7fe341e00700 1 --2- 172.21.15.153:0/2924663762 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe33c068d30 0x7fe33c19eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:52:42.198 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:42.199+0000 7fe341e00700 1 -- 172.21.15.153:0/2924663762 >> 172.21.15.153:0/2924663762 conn(0x7fe33c075330 msgr2=0x7fe33c0fe540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:52:42.198 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:42.199+0000 7fe341e00700 1 -- 172.21.15.153:0/2924663762 shutdown_connections 2024-03-21T10:52:42.198 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:42.199+0000 7fe341e00700 1 -- 172.21.15.153:0/2924663762 wait complete. 2024-03-21T10:52:42.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:52:42 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/2924663762' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:52:42.970 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:52:42 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/2924663762' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:52:43.010 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 070ed3e4-e769-11ee-95cd-87774f69a715 -e sha1=2edfee68ad7243bed422a049b6f5f58619300f11 -- bash -c 'ceph versions | jq -e '"'"'.overall | length == 1'"'"'' 2024-03-21T10:52:43.460 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi153/config 2024-03-21T10:52:43.940 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:52:43 smithi153 ceph-mon[113733]: pgmap v1228: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:52:43.946 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:52:43 smithi174 ceph-mon[88004]: pgmap v1228: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:52:45.789 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:52:45 smithi153 ceph-mon[113733]: pgmap v1229: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:52:45.855 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:45.858+0000 7f1b4ec00700 1 -- 172.21.15.153:0/4218348740 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1b48106a40 msgr2=0x7f1b48106e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:52:45.855 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:45.858+0000 7f1b4ec00700 1 --2- 172.21.15.153:0/4218348740 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1b48106a40 0x7f1b48106e20 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c00b0a0 tx=0x7f1b3c00b3b0 comp rx=0 tx=0).stop 2024-03-21T10:52:45.855 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:45.858+0000 7f1b4ec00700 1 -- 172.21.15.153:0/4218348740 shutdown_connections 2024-03-21T10:52:45.855 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:45.858+0000 7f1b4ec00700 1 --2- 172.21.15.153:0/4218348740 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1b48068410 0x7f1b48068890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:52:45.855 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:45.858+0000 7f1b4ec00700 1 --2- 172.21.15.153:0/4218348740 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1b48106a40 0x7f1b48106e20 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:52:45.855 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:45.858+0000 7f1b4ec00700 1 -- 172.21.15.153:0/4218348740 >> 172.21.15.153:0/4218348740 conn(0x7f1b48075330 msgr2=0x7f1b48075740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:52:45.856 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:45.858+0000 7f1b4ec00700 1 -- 172.21.15.153:0/4218348740 shutdown_connections 2024-03-21T10:52:45.856 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:45.858+0000 7f1b4ec00700 1 -- 172.21.15.153:0/4218348740 wait complete. 2024-03-21T10:52:45.856 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:45.859+0000 7f1b4ec00700 1 Processor -- start 2024-03-21T10:52:45.857 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:45.859+0000 7f1b4ec00700 1 -- start start 2024-03-21T10:52:45.857 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:45.860+0000 7f1b4ec00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1b48068410 0x7f1b48198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:52:45.857 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:45.860+0000 7f1b4ec00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1b48106a40 0x7f1b48198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:52:45.857 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:45.860+0000 7f1b4ec00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b48199530 con 0x7f1b48068410 2024-03-21T10:52:45.857 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:45.860+0000 7f1b4ec00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b4819d340 con 0x7f1b48106a40 2024-03-21T10:52:45.857 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:45.860+0000 7f1b4d800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1b48068410 0x7f1b48198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:52:45.857 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:45.860+0000 7f1b4d800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1b48068410 0x7f1b48198960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:36174/0 (socket says 172.21.15.153:36174) 2024-03-21T10:52:45.857 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:45.860+0000 7f1b4d800700 1 -- 172.21.15.153:0/3922361446 learned_addr learned my addr 172.21.15.153:0/3922361446 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:52:45.858 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:45.860+0000 7f1b4ce00700 1 --2- 172.21.15.153:0/3922361446 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1b48106a40 0x7f1b48198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:52:45.858 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:45.860+0000 7f1b4d800700 1 -- 172.21.15.153:0/3922361446 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1b48106a40 msgr2=0x7f1b48198ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:52:45.858 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:45.860+0000 7f1b4d800700 1 --2- 172.21.15.153:0/3922361446 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1b48106a40 0x7f1b48198ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:52:45.858 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:45.860+0000 7f1b4d800700 1 -- 172.21.15.153:0/3922361446 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b3c009d00 con 0x7f1b48068410 2024-03-21T10:52:45.858 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:45.861+0000 7f1b4ce00700 1 --2- 172.21.15.153:0/3922361446 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1b48106a40 0x7f1b48198ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:52:45.858 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:45.861+0000 7f1b4d800700 1 --2- 172.21.15.153:0/3922361446 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1b48068410 0x7f1b48198960 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c00b070 tx=0x7f1b3c00cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:52:45.859 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:45.861+0000 7f1b46000700 1 -- 172.21.15.153:0/3922361446 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1b3c00e040 con 0x7f1b48068410 2024-03-21T10:52:45.859 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:45.861+0000 7f1b46000700 1 -- 172.21.15.153:0/3922361446 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1b3c003e60 con 0x7f1b48068410 2024-03-21T10:52:45.859 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:45.861+0000 7f1b46000700 1 -- 172.21.15.153:0/3922361446 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1b3c01d910 con 0x7f1b48068410 2024-03-21T10:52:45.859 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:45.861+0000 7f1b4ec00700 1 -- 172.21.15.153:0/3922361446 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b4819d5c0 con 0x7f1b48068410 2024-03-21T10:52:45.859 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:45.862+0000 7f1b4ec00700 1 -- 172.21.15.153:0/3922361446 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b4819da60 con 0x7f1b48068410 2024-03-21T10:52:45.859 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:45.863+0000 7f1b4ec00700 1 -- 172.21.15.153:0/3922361446 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b4810a3c0 con 0x7f1b48068410 2024-03-21T10:52:45.860 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:45.864+0000 7f1b46000700 1 -- 172.21.15.153:0/3922361446 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f1b3c004820 con 0x7f1b48068410 2024-03-21T10:52:45.861 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:45.865+0000 7f1b46000700 1 --2- 172.21.15.153:0/3922361446 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f1b3006e460 0x7f1b30070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:52:45.862 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:45.865+0000 7f1b46000700 1 -- 172.21.15.153:0/3922361446 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f1b3c08fdf0 con 0x7f1b48068410 2024-03-21T10:52:45.866 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:45.866+0000 7f1b4ce00700 1 --2- 172.21.15.153:0/3922361446 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f1b3006e460 0x7f1b30070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:52:45.867 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:45.866+0000 7f1b4ce00700 1 --2- 172.21.15.153:0/3922361446 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f1b3006e460 0x7f1b30070920 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f1b48199f80 tx=0x7f1b34008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:52:45.867 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:45.870+0000 7f1b46000700 1 -- 172.21.15.153:0/3922361446 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f1b3c05c9e0 con 0x7f1b48068410 2024-03-21T10:52:45.946 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:52:45 smithi174 ceph-mon[88004]: pgmap v1229: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:52:46.228 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:46.231+0000 7f1b4ec00700 1 -- 172.21.15.153:0/3922361446 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f1b48199d90 con 0x7f1b48068410 2024-03-21T10:52:46.228 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:46.232+0000 7f1b46000700 1 -- 172.21.15.153:0/3922361446 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f1b3c05c570 con 0x7f1b48068410 2024-03-21T10:52:46.232 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:46.235+0000 7f1b4ec00700 1 -- 172.21.15.153:0/3922361446 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f1b3006e460 msgr2=0x7f1b30070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:52:46.232 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:46.235+0000 7f1b4ec00700 1 --2- 172.21.15.153:0/3922361446 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f1b3006e460 0x7f1b30070920 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f1b48199f80 tx=0x7f1b34008040 comp rx=0 tx=0).stop 2024-03-21T10:52:46.232 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:46.235+0000 7f1b4ec00700 1 -- 172.21.15.153:0/3922361446 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1b48068410 msgr2=0x7f1b48198960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:52:46.232 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:46.235+0000 7f1b4ec00700 1 --2- 172.21.15.153:0/3922361446 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1b48068410 0x7f1b48198960 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c00b070 tx=0x7f1b3c00cdd0 comp rx=0 tx=0).stop 2024-03-21T10:52:46.232 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:46.235+0000 7f1b4ec00700 1 -- 172.21.15.153:0/3922361446 shutdown_connections 2024-03-21T10:52:46.232 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:46.235+0000 7f1b4ec00700 1 --2- 172.21.15.153:0/3922361446 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f1b3006e460 0x7f1b30070920 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:52:46.232 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:46.235+0000 7f1b4ec00700 1 --2- 172.21.15.153:0/3922361446 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f1b48068410 0x7f1b48198960 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:52:46.233 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:46.235+0000 7f1b4ec00700 1 --2- 172.21.15.153:0/3922361446 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1b48106a40 0x7f1b48198ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:52:46.233 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:46.235+0000 7f1b4ec00700 1 -- 172.21.15.153:0/3922361446 >> 172.21.15.153:0/3922361446 conn(0x7f1b48075330 msgr2=0x7f1b480fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:52:46.233 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:46.236+0000 7f1b4ec00700 1 -- 172.21.15.153:0/3922361446 shutdown_connections 2024-03-21T10:52:46.233 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:46.236+0000 7f1b4ec00700 1 -- 172.21.15.153:0/3922361446 wait complete. 2024-03-21T10:52:46.249 INFO:teuthology.orchestra.run.smithi153.stdout:true 2024-03-21T10:52:46.926 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:52:46 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/3922361446' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:52:46.926 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:52:46 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:52:46.946 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:52:46 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/3922361446' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:52:46.946 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:52:46 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:52:46.967 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 070ed3e4-e769-11ee-95cd-87774f69a715 -e sha1=2edfee68ad7243bed422a049b6f5f58619300f11 -- bash -c 'ceph versions | jq -e '"'"'.overall | keys'"'"' | grep $sha1' 2024-03-21T10:52:47.383 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi153/config 2024-03-21T10:52:47.801 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:52:47 smithi153 ceph-mon[113733]: pgmap v1230: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:52:47.946 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:52:47 smithi174 ceph-mon[88004]: pgmap v1230: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:52:49.587 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:49.590+0000 7f91e4e00700 1 -- 172.21.15.153:0/677387595 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91e0100980 msgr2=0x7f91e0100e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:52:49.588 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:49.590+0000 7f91e4e00700 1 --2- 172.21.15.153:0/677387595 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91e0100980 0x7f91e0100e00 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f91cc009910 tx=0x7f91cc009c20 comp rx=0 tx=0).stop 2024-03-21T10:52:49.588 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:49.591+0000 7f91e4e00700 1 -- 172.21.15.153:0/677387595 shutdown_connections 2024-03-21T10:52:49.588 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:49.591+0000 7f91e4e00700 1 --2- 172.21.15.153:0/677387595 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91e0100980 0x7f91e0100e00 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:52:49.588 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:49.591+0000 7f91e4e00700 1 --2- 172.21.15.153:0/677387595 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f91e0106980 0x7f91e0106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:52:49.588 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:49.591+0000 7f91e4e00700 1 -- 172.21.15.153:0/677387595 >> 172.21.15.153:0/677387595 conn(0x7f91e00fc470 msgr2=0x7f91e00fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:52:49.588 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:49.591+0000 7f91e4e00700 1 -- 172.21.15.153:0/677387595 shutdown_connections 2024-03-21T10:52:49.588 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:49.591+0000 7f91e4e00700 1 -- 172.21.15.153:0/677387595 wait complete. 2024-03-21T10:52:49.589 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:49.592+0000 7f91e4e00700 1 Processor -- start 2024-03-21T10:52:49.589 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:49.592+0000 7f91e4e00700 1 -- start start 2024-03-21T10:52:49.589 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:49.593+0000 7f91e4e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91e0100980 0x7f91e0073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:52:49.589 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:49.593+0000 7f91e4e00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f91e0106980 0x7f91e00735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:52:49.589 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:49.593+0000 7f91e4e00700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91e0073b10 con 0x7f91e0100980 2024-03-21T10:52:49.589 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:49.593+0000 7f91e4e00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91e0073c50 con 0x7f91e0106980 2024-03-21T10:52:49.590 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:49.593+0000 7f91df400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91e0100980 0x7f91e0073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:52:49.590 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:49.593+0000 7f91df400700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91e0100980 0x7f91e0073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:36188/0 (socket says 172.21.15.153:36188) 2024-03-21T10:52:49.590 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:49.593+0000 7f91df400700 1 -- 172.21.15.153:0/2071854869 learned_addr learned my addr 172.21.15.153:0/2071854869 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:52:49.590 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:49.593+0000 7f91dea00700 1 --2- 172.21.15.153:0/2071854869 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f91e0106980 0x7f91e00735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:52:49.591 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:49.594+0000 7f91df400700 1 -- 172.21.15.153:0/2071854869 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f91e0106980 msgr2=0x7f91e00735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:52:49.591 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:49.594+0000 7f91df400700 1 --2- 172.21.15.153:0/2071854869 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f91e0106980 0x7f91e00735d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:52:49.591 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:49.594+0000 7f91df400700 1 -- 172.21.15.153:0/2071854869 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91cc0095f0 con 0x7f91e0100980 2024-03-21T10:52:49.591 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:49.594+0000 7f91df400700 1 --2- 172.21.15.153:0/2071854869 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91e0100980 0x7f91e0073090 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f91d000c8a0 tx=0x7f91d000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:52:49.591 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:49.594+0000 7f91d7e00700 1 -- 172.21.15.153:0/2071854869 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f91d000cea0 con 0x7f91e0100980 2024-03-21T10:52:49.591 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:49.594+0000 7f91d7e00700 1 -- 172.21.15.153:0/2071854869 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f91d0004cf0 con 0x7f91e0100980 2024-03-21T10:52:49.591 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:49.594+0000 7f91e4e00700 1 -- 172.21.15.153:0/2071854869 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91e01962d0 con 0x7f91e0100980 2024-03-21T10:52:49.592 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:49.595+0000 7f91e4e00700 1 -- 172.21.15.153:0/2071854869 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91e0196820 con 0x7f91e0100980 2024-03-21T10:52:49.592 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:49.595+0000 7f91d7e00700 1 -- 172.21.15.153:0/2071854869 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f91d0007920 con 0x7f91e0100980 2024-03-21T10:52:49.592 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:49.596+0000 7f91e4e00700 1 -- 172.21.15.153:0/2071854869 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91c0005320 con 0x7f91e0100980 2024-03-21T10:52:49.593 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:49.597+0000 7f91d7e00700 1 -- 172.21.15.153:0/2071854869 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f91d00052b0 con 0x7f91e0100980 2024-03-21T10:52:49.595 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:49.598+0000 7f91d7e00700 1 --2- 172.21.15.153:0/2071854869 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f91c406e4b0 0x7f91c4070970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:52:49.595 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:49.599+0000 7f91d7e00700 1 -- 172.21.15.153:0/2071854869 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f91d0096420 con 0x7f91e0100980 2024-03-21T10:52:49.595 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:49.599+0000 7f91dea00700 1 --2- 172.21.15.153:0/2071854869 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f91c406e4b0 0x7f91c4070970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:52:49.596 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:49.600+0000 7f91dea00700 1 --2- 172.21.15.153:0/2071854869 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f91c406e4b0 0x7f91c4070970 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f91e0074880 tx=0x7f91cc01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:52:49.600 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:49.604+0000 7f91d7e00700 1 -- 172.21.15.153:0/2071854869 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f91d005f600 con 0x7f91e0100980 2024-03-21T10:52:49.801 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:52:49 smithi153 ceph-mon[113733]: pgmap v1231: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:52:49.937 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:49.940+0000 7f91e4e00700 1 -- 172.21.15.153:0/2071854869 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f91c0006200 con 0x7f91e0100980 2024-03-21T10:52:49.937 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:49.941+0000 7f91d7e00700 1 -- 172.21.15.153:0/2071854869 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f91d0062cc0 con 0x7f91e0100980 2024-03-21T10:52:49.940 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:49.943+0000 7f91e4e00700 1 -- 172.21.15.153:0/2071854869 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f91c406e4b0 msgr2=0x7f91c4070970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:52:49.940 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:49.943+0000 7f91e4e00700 1 --2- 172.21.15.153:0/2071854869 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f91c406e4b0 0x7f91c4070970 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f91e0074880 tx=0x7f91cc01a040 comp rx=0 tx=0).stop 2024-03-21T10:52:49.940 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:49.943+0000 7f91e4e00700 1 -- 172.21.15.153:0/2071854869 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91e0100980 msgr2=0x7f91e0073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:52:49.941 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:49.943+0000 7f91e4e00700 1 --2- 172.21.15.153:0/2071854869 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91e0100980 0x7f91e0073090 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f91d000c8a0 tx=0x7f91d000cc60 comp rx=0 tx=0).stop 2024-03-21T10:52:49.941 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:49.944+0000 7f91e4e00700 1 -- 172.21.15.153:0/2071854869 shutdown_connections 2024-03-21T10:52:49.941 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:49.944+0000 7f91e4e00700 1 --2- 172.21.15.153:0/2071854869 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f91c406e4b0 0x7f91c4070970 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:52:49.941 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:49.944+0000 7f91e4e00700 1 --2- 172.21.15.153:0/2071854869 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f91e0100980 0x7f91e0073090 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:52:49.941 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:49.944+0000 7f91e4e00700 1 --2- 172.21.15.153:0/2071854869 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f91e0106980 0x7f91e00735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:52:49.941 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:49.944+0000 7f91e4e00700 1 -- 172.21.15.153:0/2071854869 >> 172.21.15.153:0/2071854869 conn(0x7f91e00fc470 msgr2=0x7f91e00fcec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:52:49.941 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:49.944+0000 7f91e4e00700 1 -- 172.21.15.153:0/2071854869 shutdown_connections 2024-03-21T10:52:49.941 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:49.944+0000 7f91e4e00700 1 -- 172.21.15.153:0/2071854869 wait complete. 2024-03-21T10:52:49.946 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:52:49 smithi174 ceph-mon[88004]: pgmap v1231: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:52:49.958 INFO:teuthology.orchestra.run.smithi153.stdout: "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)" 2024-03-21T10:52:50.824 DEBUG:teuthology.parallel:result is None 2024-03-21T10:52:50.825 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-21T10:52:50.837 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi153.front.sepia.ceph.com 2024-03-21T10:52:50.837 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 070ed3e4-e769-11ee-95cd-87774f69a715 -- bash -c 'ceph fs dump' 2024-03-21T10:52:50.912 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:52:50 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/2071854869' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:52:50.946 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:52:50 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/2071854869' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-21T10:52:51.285 INFO:teuthology.orchestra.run.smithi153.stderr:Inferring config /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/mon.smithi153/config 2024-03-21T10:52:51.753 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:52:51 smithi153 ceph-mon[113733]: pgmap v1232: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:52:51.946 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:52:51 smithi174 ceph-mon[88004]: pgmap v1232: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:52:53.452 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:53.452+0000 7fe127800700 1 -- 172.21.15.153:0/48018500 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe120100da0 msgr2=0x7fe120101180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:52:53.452 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:53.452+0000 7fe127800700 1 --2- 172.21.15.153:0/48018500 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe120100da0 0x7fe120101180 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fe114009910 tx=0x7fe114009c20 comp rx=0 tx=0).stop 2024-03-21T10:52:53.452 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:53.452+0000 7fe127800700 1 -- 172.21.15.153:0/48018500 shutdown_connections 2024-03-21T10:52:53.452 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:53.452+0000 7fe127800700 1 --2- 172.21.15.153:0/48018500 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe120101750 0x7fe1201057a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:52:53.452 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:53.452+0000 7fe127800700 1 --2- 172.21.15.153:0/48018500 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe120100da0 0x7fe120101180 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:52:53.452 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:53.452+0000 7fe127800700 1 -- 172.21.15.153:0/48018500 >> 172.21.15.153:0/48018500 conn(0x7fe1200fc630 msgr2=0x7fe1200fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:52:53.452 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:53.452+0000 7fe127800700 1 -- 172.21.15.153:0/48018500 shutdown_connections 2024-03-21T10:52:53.452 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:53.452+0000 7fe127800700 1 -- 172.21.15.153:0/48018500 wait complete. 2024-03-21T10:52:53.452 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:53.453+0000 7fe127800700 1 Processor -- start 2024-03-21T10:52:53.452 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:53.453+0000 7fe127800700 1 -- start start 2024-03-21T10:52:53.453 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:53.454+0000 7fe127800700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe120100da0 0x7fe12019dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:52:53.453 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:53.454+0000 7fe127800700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe120101750 0x7fe12019e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:52:53.453 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:53.454+0000 7fe127800700 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe12019e7f0 con 0x7fe120100da0 2024-03-21T10:52:53.453 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:53.454+0000 7fe127800700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe12019e930 con 0x7fe120101750 2024-03-21T10:52:53.453 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:53.454+0000 7fe124e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe120100da0 0x7fe12019dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:52:53.453 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:53.454+0000 7fe124e00700 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe120100da0 0x7fe12019dce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:34054/0 (socket says 172.21.15.153:34054) 2024-03-21T10:52:53.453 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:53.454+0000 7fe124e00700 1 -- 172.21.15.153:0/3024122738 learned_addr learned my addr 172.21.15.153:0/3024122738 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:52:53.453 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:53.454+0000 7fe11fe00700 1 --2- 172.21.15.153:0/3024122738 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe120101750 0x7fe12019e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:52:53.453 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:53.455+0000 7fe124e00700 1 -- 172.21.15.153:0/3024122738 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe120101750 msgr2=0x7fe12019e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:52:53.453 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:53.455+0000 7fe124e00700 1 --2- 172.21.15.153:0/3024122738 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe120101750 0x7fe12019e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:52:53.453 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:53.455+0000 7fe124e00700 1 -- 172.21.15.153:0/3024122738 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1140095f0 con 0x7fe120100da0 2024-03-21T10:52:53.453 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:53.455+0000 7fe124e00700 1 --2- 172.21.15.153:0/3024122738 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe120100da0 0x7fe12019dce0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fe11400b5c0 tx=0x7fe11400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:52:53.454 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:53.456+0000 7fe11d600700 1 -- 172.21.15.153:0/3024122738 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe11401d070 con 0x7fe120100da0 2024-03-21T10:52:53.454 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:53.456+0000 7fe11d600700 1 -- 172.21.15.153:0/3024122738 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe114005e00 con 0x7fe120100da0 2024-03-21T10:52:53.454 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:53.456+0000 7fe11d600700 1 -- 172.21.15.153:0/3024122738 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe114017400 con 0x7fe120100da0 2024-03-21T10:52:53.454 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:53.456+0000 7fe127800700 1 -- 172.21.15.153:0/3024122738 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1201a6be0 con 0x7fe120100da0 2024-03-21T10:52:53.454 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:53.456+0000 7fe127800700 1 -- 172.21.15.153:0/3024122738 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1201a70d0 con 0x7fe120100da0 2024-03-21T10:52:53.454 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:53.457+0000 7fe127800700 1 -- 172.21.15.153:0/3024122738 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe120109140 con 0x7fe120100da0 2024-03-21T10:52:53.455 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:53.459+0000 7fe11d600700 1 -- 172.21.15.153:0/3024122738 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fe11400f460 con 0x7fe120100da0 2024-03-21T10:52:53.456 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:53.460+0000 7fe11d600700 1 --2- 172.21.15.153:0/3024122738 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe10806e460 0x7fe108070920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:52:53.456 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:53.460+0000 7fe11d600700 1 -- 172.21.15.153:0/3024122738 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fe11408f9c0 con 0x7fe120100da0 2024-03-21T10:52:53.456 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:53.460+0000 7fe11fe00700 1 --2- 172.21.15.153:0/3024122738 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe10806e460 0x7fe108070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:52:53.457 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:53.461+0000 7fe11fe00700 1 --2- 172.21.15.153:0/3024122738 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe10806e460 0x7fe108070920 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fe10c005950 tx=0x7fe10c0058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:52:53.462 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:53.465+0000 7fe11d600700 1 -- 172.21.15.153:0/3024122738 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe114058a60 con 0x7fe120100da0 2024-03-21T10:52:53.925 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:53.928+0000 7fe127800700 1 -- 172.21.15.153:0/3024122738 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fe12004ea90 con 0x7fe120100da0 2024-03-21T10:52:53.926 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:52:53 smithi153 ceph-mon[113733]: pgmap v1233: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:52:53.926 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:53.929+0000 7fe11d600700 1 -- 172.21.15.153:0/3024122738 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 41 v41) v1 ==== 76+0+1948 (secure 0 0 0) 0x7fe11405c120 con 0x7fe120100da0 2024-03-21T10:52:53.926 INFO:teuthology.orchestra.run.smithi153.stdout:e41 2024-03-21T10:52:53.927 INFO:teuthology.orchestra.run.smithi153.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-21T10:52:53.927 INFO:teuthology.orchestra.run.smithi153.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-21T10:52:53.927 INFO:teuthology.orchestra.run.smithi153.stdout:legacy client fscid: 1 2024-03-21T10:52:53.927 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:52:53.927 INFO:teuthology.orchestra.run.smithi153.stdout:Filesystem 'cephfs' (1) 2024-03-21T10:52:53.927 INFO:teuthology.orchestra.run.smithi153.stdout:fs_name cephfs 2024-03-21T10:52:53.927 INFO:teuthology.orchestra.run.smithi153.stdout:epoch 41 2024-03-21T10:52:53.927 INFO:teuthology.orchestra.run.smithi153.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-21T10:52:53.927 INFO:teuthology.orchestra.run.smithi153.stdout:created 2024-03-21T10:04:27.086576+0000 2024-03-21T10:52:53.927 INFO:teuthology.orchestra.run.smithi153.stdout:modified 2024-03-21T10:51:02.096522+0000 2024-03-21T10:52:53.927 INFO:teuthology.orchestra.run.smithi153.stdout:tableserver 0 2024-03-21T10:52:53.927 INFO:teuthology.orchestra.run.smithi153.stdout:root 0 2024-03-21T10:52:53.928 INFO:teuthology.orchestra.run.smithi153.stdout:session_timeout 60 2024-03-21T10:52:53.928 INFO:teuthology.orchestra.run.smithi153.stdout:session_autoclose 300 2024-03-21T10:52:53.928 INFO:teuthology.orchestra.run.smithi153.stdout:max_file_size 1099511627776 2024-03-21T10:52:53.928 INFO:teuthology.orchestra.run.smithi153.stdout:max_xattr_size 65536 2024-03-21T10:52:53.928 INFO:teuthology.orchestra.run.smithi153.stdout:required_client_features {} 2024-03-21T10:52:53.928 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure 0 2024-03-21T10:52:53.928 INFO:teuthology.orchestra.run.smithi153.stdout:last_failure_osd_epoch 94 2024-03-21T10:52:53.928 INFO:teuthology.orchestra.run.smithi153.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-21T10:52:53.928 INFO:teuthology.orchestra.run.smithi153.stdout:max_mds 2 2024-03-21T10:52:53.928 INFO:teuthology.orchestra.run.smithi153.stdout:in 0,1 2024-03-21T10:52:53.928 INFO:teuthology.orchestra.run.smithi153.stdout:up {0=34442,1=34474} 2024-03-21T10:52:53.928 INFO:teuthology.orchestra.run.smithi153.stdout:failed 2024-03-21T10:52:53.928 INFO:teuthology.orchestra.run.smithi153.stdout:damaged 2024-03-21T10:52:53.929 INFO:teuthology.orchestra.run.smithi153.stdout:stopped 2024-03-21T10:52:53.929 INFO:teuthology.orchestra.run.smithi153.stdout:data_pools [3] 2024-03-21T10:52:53.929 INFO:teuthology.orchestra.run.smithi153.stdout:metadata_pool 2 2024-03-21T10:52:53.929 INFO:teuthology.orchestra.run.smithi153.stdout:inline_data enabled 2024-03-21T10:52:53.929 INFO:teuthology.orchestra.run.smithi153.stdout:balancer 2024-03-21T10:52:53.929 INFO:teuthology.orchestra.run.smithi153.stdout:bal_rank_mask -1 2024-03-21T10:52:53.929 INFO:teuthology.orchestra.run.smithi153.stdout:standby_count_wanted 1 2024-03-21T10:52:53.929 INFO:teuthology.orchestra.run.smithi153.stdout:qdb_cluster leader: 34442 members: 34474,34442 2024-03-21T10:52:53.929 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.zipckr{0:34442} state up:active seq 474 join_fscid=1 addr [v2:172.21.15.153:6826/1103846295,v1:172.21.15.153:6827/1103846295] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:52:53.929 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.yyimst{0:44313} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.174:6824/4127786797,v1:172.21.15.174:6825/4127786797] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:52:53.929 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi153.iuyluj{1:34474} state up:active seq 468 join_fscid=1 addr [v2:172.21.15.153:6828/1738630832,v1:172.21.15.153:6829/1738630832] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:52:53.929 INFO:teuthology.orchestra.run.smithi153.stdout:[mds.cephfs.smithi174.nhozag{1:44687} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.174:6826/2767517845,v1:172.21.15.174:6827/2767517845] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T10:52:53.930 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:52:53.930 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:52:53.930 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:53.932+0000 7fe127800700 1 -- 172.21.15.153:0/3024122738 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe10806e460 msgr2=0x7fe108070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:52:53.930 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:53.932+0000 7fe127800700 1 --2- 172.21.15.153:0/3024122738 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe10806e460 0x7fe108070920 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fe10c005950 tx=0x7fe10c0058e0 comp rx=0 tx=0).stop 2024-03-21T10:52:53.931 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:53.932+0000 7fe127800700 1 -- 172.21.15.153:0/3024122738 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe120100da0 msgr2=0x7fe12019dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:52:53.931 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:53.932+0000 7fe127800700 1 --2- 172.21.15.153:0/3024122738 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe120100da0 0x7fe12019dce0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fe11400b5c0 tx=0x7fe11400be20 comp rx=0 tx=0).stop 2024-03-21T10:52:53.931 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:53.933+0000 7fe127800700 1 -- 172.21.15.153:0/3024122738 shutdown_connections 2024-03-21T10:52:53.931 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:53.933+0000 7fe127800700 1 --2- 172.21.15.153:0/3024122738 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe10806e460 0x7fe108070920 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:52:53.931 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:53.933+0000 7fe127800700 1 --2- 172.21.15.153:0/3024122738 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe120100da0 0x7fe12019dce0 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:52:53.931 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:53.933+0000 7fe127800700 1 --2- 172.21.15.153:0/3024122738 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe120101750 0x7fe12019e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:52:53.931 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:53.933+0000 7fe127800700 1 -- 172.21.15.153:0/3024122738 >> 172.21.15.153:0/3024122738 conn(0x7fe1200fc630 msgr2=0x7fe1200fdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:52:53.931 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:53.933+0000 7fe127800700 1 -- 172.21.15.153:0/3024122738 shutdown_connections 2024-03-21T10:52:53.931 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:53.933+0000 7fe127800700 1 -- 172.21.15.153:0/3024122738 wait complete. 2024-03-21T10:52:53.931 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 41 2024-03-21T10:52:53.946 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:52:53 smithi174 ceph-mon[88004]: pgmap v1233: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:52:54.669 INFO:teuthology.run_tasks:Running task fs.post_upgrade_checks... 2024-03-21T10:52:54.684 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T10:52:54.885 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:54.882+0000 7f9133e00640 1 -- 172.21.15.153:0/273885209 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9134100120 msgr2=0x7f9134100500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:52:54.885 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:54.882+0000 7f9133e00640 1 --2- 172.21.15.153:0/273885209 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9134100120 0x7f9134100500 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f9124009880 tx=0x7f912402bcf0 comp rx=0 tx=0).stop 2024-03-21T10:52:54.885 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:54.882+0000 7f9133e00640 1 -- 172.21.15.153:0/273885209 shutdown_connections 2024-03-21T10:52:54.885 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:54.882+0000 7f9133e00640 1 --2- 172.21.15.153:0/273885209 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9134100ad0 0x7f9134104b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:52:54.885 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:54.882+0000 7f9133e00640 1 --2- 172.21.15.153:0/273885209 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9134100120 0x7f9134100500 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:52:54.885 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:54.882+0000 7f9133e00640 1 -- 172.21.15.153:0/273885209 >> 172.21.15.153:0/273885209 conn(0x7f91340fb9c0 msgr2=0x7f91340fdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:52:54.885 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:54.882+0000 7f9133e00640 1 -- 172.21.15.153:0/273885209 shutdown_connections 2024-03-21T10:52:54.885 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:54.882+0000 7f9133e00640 1 -- 172.21.15.153:0/273885209 wait complete. 2024-03-21T10:52:54.885 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:54.883+0000 7f9133e00640 1 Processor -- start 2024-03-21T10:52:54.885 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:54.883+0000 7f9133e00640 1 -- start start 2024-03-21T10:52:54.885 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:54.884+0000 7f9133e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9134100120 0x7f913419b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:52:54.886 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:54.884+0000 7f9133e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f913419bde0 0x7f9134195910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:52:54.886 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:54.884+0000 7f9133e00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f913419c320 con 0x7f9134100120 2024-03-21T10:52:54.886 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:54.884+0000 7f9133e00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f913419c460 con 0x7f913419bde0 2024-03-21T10:52:54.886 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:54.884+0000 7f9132a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9134100120 0x7f913419b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:52:54.886 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:54.884+0000 7f9132a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9134100120 0x7f913419b8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:34080/0 (socket says 172.21.15.153:34080) 2024-03-21T10:52:54.886 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:54.884+0000 7f9132a00640 1 -- 172.21.15.153:0/2318613128 learned_addr learned my addr 172.21.15.153:0/2318613128 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:52:54.886 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:54.884+0000 7f9132000640 1 --2- 172.21.15.153:0/2318613128 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f913419bde0 0x7f9134195910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:52:54.886 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:54.884+0000 7f9132a00640 1 -- 172.21.15.153:0/2318613128 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f913419bde0 msgr2=0x7f9134195910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:52:54.886 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:54.884+0000 7f9132a00640 1 --2- 172.21.15.153:0/2318613128 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f913419bde0 0x7f9134195910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:52:54.886 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:54.884+0000 7f9132a00640 1 -- 172.21.15.153:0/2318613128 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9124009560 con 0x7f9134100120 2024-03-21T10:52:54.886 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:54.885+0000 7f9132a00640 1 --2- 172.21.15.153:0/2318613128 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9134100120 0x7f913419b8a0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f9124036040 tx=0x7f9124004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:52:54.886 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:54.885+0000 7f912b400640 1 -- 172.21.15.153:0/2318613128 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f91240035f0 con 0x7f9134100120 2024-03-21T10:52:54.886 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:54.885+0000 7f912b400640 1 -- 172.21.15.153:0/2318613128 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9124003c10 con 0x7f9134100120 2024-03-21T10:52:54.887 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:54.885+0000 7f9133e00640 1 -- 172.21.15.153:0/2318613128 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9134195f10 con 0x7f9134100120 2024-03-21T10:52:54.887 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:54.885+0000 7f912b400640 1 -- 172.21.15.153:0/2318613128 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9124034440 con 0x7f9134100120 2024-03-21T10:52:54.887 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:54.885+0000 7f9133e00640 1 -- 172.21.15.153:0/2318613128 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9134196470 con 0x7f9134100120 2024-03-21T10:52:54.887 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:54.887+0000 7f9133e00640 1 -- 172.21.15.153:0/2318613128 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f913404e890 con 0x7f9134100120 2024-03-21T10:52:54.887 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:54.888+0000 7f912b400640 1 -- 172.21.15.153:0/2318613128 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f9124005060 con 0x7f9134100120 2024-03-21T10:52:54.888 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:54.889+0000 7f912b400640 1 --2- 172.21.15.153:0/2318613128 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f911806e490 0x7f9118070950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:52:54.888 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:54.889+0000 7f912b400640 1 -- 172.21.15.153:0/2318613128 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f91240b08a0 con 0x7f9134100120 2024-03-21T10:52:54.888 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:54.889+0000 7f9132000640 1 --2- 172.21.15.153:0/2318613128 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f911806e490 0x7f9118070950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:52:54.888 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:54.890+0000 7f9132000640 1 --2- 172.21.15.153:0/2318613128 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f911806e490 0x7f9118070950 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f9134069300 tx=0x7f9120005cc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:52:54.891 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:54.894+0000 7f912b400640 1 -- 172.21.15.153:0/2318613128 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f91240799e0 con 0x7f9134100120 2024-03-21T10:52:54.946 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:52:54 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/3024122738' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:52:55.051 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:52:54 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/3024122738' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T10:52:55.090 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:55.094+0000 7f9133e00640 1 -- 172.21.15.153:0/2318613128 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9134196e20 con 0x7f9134100120 2024-03-21T10:52:55.092 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:55.095+0000 7f912b400640 1 -- 172.21.15.153:0/2318613128 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) v1 ==== 74+0+17231 (secure 0 0 0) 0x7f912407d0a0 con 0x7f9134100120 2024-03-21T10:52:55.092 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:52:55.093 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":98,"fsid":"070ed3e4-e769-11ee-95cd-87774f69a715","created":"2024-03-21T09:55:19.360240+0000","modified":"2024-03-21T10:35:13.153133+0000","last_up_change":"2024-03-21T10:18:57.856952+0000","last_in_change":"2024-03-21T10:02:32.306573+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T10:01:18.943639+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-03-21T10:04:25.621358+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.309999942779541,"score_stable":1.309999942779541,"optimal_score":1,"raw_score_acting":1.309999942779541,"raw_score_stable":1.309999942779541,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-03-21T10:04:26.593674+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0f2e1224-6f45-4f23-b97a-5b89319a90f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":50,"up_from":61,"up_thru":90,"down_at":51,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":2525147935},{"type":"v1","addr":"172.21.15.153:6803","nonce":2525147935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":2525147935},{"type":"v1","addr":"172.21.15.153:6805","nonce":2525147935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":2525147935},{"type":"v1","addr":"172.21.15.153:6809","nonce":2525147935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":2525147935},{"type":"v1","addr":"172.21.15.153:6807","nonce":2525147935}]},"public_addr":"172.21.15.153:6803/2525147935","cluster_addr":"172.21.15.153:6805/2525147935","heartbeat_back_addr":"172.21.15.153:6809/2525147935","heartbeat_front_addr":"172.21.15.153:6807/2525147935","state":["exists","up"]},{"osd":1,"uuid":"76195edb-aa7a-4627-a211-09a578de65c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":66,"up_from":70,"up_thru":90,"down_at":67,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":1529835318},{"type":"v1","addr":"172.21.15.153:6811","nonce":1529835318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":1529835318},{"type":"v1","addr":"172.21.15.153:6813","nonce":1529835318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":1529835318},{"type":"v1","addr":"172.21.15.153:6817","nonce":1529835318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":1529835318},{"type":"v1","addr":"172.21.15.153:6815","nonce":1529835318}]},"public_addr":"172.21.15.153:6811/1529835318","cluster_addr":"172.21.15.153:6813/1529835318","heartbeat_back_addr":"172.21.15.153:6817/1529835318","heartbeat_front_addr":"172.21.15.153:6815/1529835318","state":["exists","up"]},{"osd":2,"uuid":"3ec17cf8-f0ea-45c4-b084-b304024d6e92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":71,"up_from":75,"up_thru":90,"down_at":72,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":292763575},{"type":"v1","addr":"172.21.15.153:6819","nonce":292763575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":292763575},{"type":"v1","addr":"172.21.15.153:6821","nonce":292763575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":292763575},{"type":"v1","addr":"172.21.15.153:6825","nonce":292763575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":292763575},{"type":"v1","addr":"172.21.15.153:6823","nonce":292763575}]},"public_addr":"172.21.15.153:6819/292763575","cluster_addr":"172.21.15.153:6821/292763575","heartbeat_back_addr":"172.21.15.153:6825/292763575","heartbeat_front_addr":"172.21.15.153:6823/292763575","state":["exists","up"]},{"osd":3,"uuid":"285146f6-6209-47ee-9e5b-2e075c949231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":27,"last_clean_end":76,"up_from":80,"up_thru":90,"down_at":77,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1830370519},{"type":"v1","addr":"172.21.15.174:6801","nonce":1830370519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1830370519},{"type":"v1","addr":"172.21.15.174:6803","nonce":1830370519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1830370519},{"type":"v1","addr":"172.21.15.174:6807","nonce":1830370519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":1830370519},{"type":"v1","addr":"172.21.15.174:6805","nonce":1830370519}]},"public_addr":"172.21.15.174:6801/1830370519","cluster_addr":"172.21.15.174:6803/1830370519","heartbeat_back_addr":"172.21.15.174:6807/1830370519","heartbeat_front_addr":"172.21.15.174:6805/1830370519","state":["exists","up"]},{"osd":4,"uuid":"f05cec2d-c649-489d-98a3-f24a75fe1f95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":35,"last_clean_end":81,"up_from":85,"up_thru":90,"down_at":82,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1143789116},{"type":"v1","addr":"172.21.15.174:6809","nonce":1143789116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1143789116},{"type":"v1","addr":"172.21.15.174:6811","nonce":1143789116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1143789116},{"type":"v1","addr":"172.21.15.174:6815","nonce":1143789116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1143789116},{"type":"v1","addr":"172.21.15.174:6813","nonce":1143789116}]},"public_addr":"172.21.15.174:6809/1143789116","cluster_addr":"172.21.15.174:6811/1143789116","heartbeat_back_addr":"172.21.15.174:6815/1143789116","heartbeat_front_addr":"172.21.15.174:6813/1143789116","state":["exists","up"]},{"osd":5,"uuid":"f430a3b2-e1b5-4698-9089-812fcf3a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":40,"last_clean_end":86,"up_from":90,"up_thru":90,"down_at":87,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":385015973},{"type":"v1","addr":"172.21.15.174:6817","nonce":385015973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":385015973},{"type":"v1","addr":"172.21.15.174:6819","nonce":385015973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":385015973},{"type":"v1","addr":"172.21.15.174:6823","nonce":385015973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":385015973},{"type":"v1","addr":"172.21.15.174:6821","nonce":385015973}]},"public_addr":"172.21.15.174:6817/385015973","cluster_addr":"172.21.15.174:6819/385015973","heartbeat_back_addr":"172.21.15.174:6823/385015973","heartbeat_front_addr":"172.21.15.174:6821/385015973","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:00:04.356688+0000","dead_epoch":50},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:00:39.988638+0000","dead_epoch":66},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:01:14.882381+0000","dead_epoch":71},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:01:48.072468+0000","dead_epoch":76},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:02:21.211701+0000","dead_epoch":81},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:02:54.161138+0000","dead_epoch":86}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":3,"to":2}]},{"pgid":"2.14","mappings":[{"from":3,"to":0}]},{"pgid":"2.16","mappings":[{"from":3,"to":0}]},{"pgid":"3.7","mappings":[{"from":1,"to":0}]},{"pgid":"3.b","mappings":[{"from":1,"to":2},{"from":4,"to":5}]},{"pgid":"3.15","mappings":[{"from":3,"to":2}]},{"pgid":"3.19","mappings":[{"from":3,"to":5}]},{"pgid":"3.1c","mappings":[{"from":4,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.174:6825/1776947618":"2024-03-22T10:19:05.021561+0000","172.21.15.174:6824/1776947618":"2024-03-22T10:19:05.021561+0000","172.21.15.153:6829/866138999":"2024-03-22T10:19:05.021528+0000","172.21.15.153:0/3553873361":"2024-03-22T10:13:31.087809+0000","172.21.15.174:0/1056981865":"2024-03-22T10:08:33.113521+0000","172.21.15.153:0/2407536999":"2024-03-22T09:56:00.849533+0000","172.21.15.153:0/2969012027":"2024-03-22T10:07:30.531722+0000","172.21.15.153:6826/1353876997":"2024-03-22T10:04:45.049156+0000","172.21.15.153:0/623594592":"2024-03-22T09:56:00.849533+0000","172.21.15.153:0/1163205431":"2024-03-22T09:56:43.480866+0000","172.21.15.153:0/3278686288":"2024-03-22T09:58:24.250138+0000","172.21.15.153:0/652104290":"2024-03-22T09:58:24.250138+0000","172.21.15.153:6800/2524145044":"2024-03-22T10:13:31.087809+0000","172.21.15.174:0/3456690749":"2024-03-22T10:08:33.113521+0000","172.21.15.153:0/4237751158":"2024-03-22T10:07:30.531722+0000","172.21.15.153:0/3456116624":"2024-03-22T09:56:43.480866+0000","172.21.15.153:0/3342434133":"2024-03-22T10:13:31.087809+0000","172.21.15.153:0/2801644471":"2024-03-22T10:07:30.531722+0000","172.21.15.153:6801/1639929315":"2024-03-22T10:07:30.531722+0000","172.21.15.174:0/950700094":"2024-03-22T10:08:33.113521+0000","172.21.15.153:6827/1353876997":"2024-03-22T10:04:45.049156+0000","172.21.15.174:0/2227377986":"2024-03-22T10:08:33.113521+0000","172.21.15.153:6800/1143522828":"2024-03-22T09:58:24.250138+0000","172.21.15.153:0/3435678189":"2024-03-22T09:56:00.849533+0000","172.21.15.153:0/2431412876":"2024-03-22T10:07:30.531722+0000","172.21.15.153:6801/1143522828":"2024-03-22T09:58:24.250138+0000","172.21.15.174:6829/1155541738":"2024-03-22T10:08:33.113521+0000","172.21.15.153:0/517714023":"2024-03-22T10:13:31.087809+0000","172.21.15.174:0/828964242":"2024-03-22T10:08:33.113521+0000","172.21.15.153:0/405164318":"2024-03-22T09:58:24.250138+0000","172.21.15.153:0/816489762":"2024-03-22T09:56:43.480866+0000","172.21.15.153:0/2445011785":"2024-03-22T10:13:31.087809+0000","172.21.15.153:6800/3178125243":"2024-03-22T09:56:43.480866+0000","172.21.15.174:6828/1155541738":"2024-03-22T10:08:33.113521+0000","172.21.15.153:6801/3178125243":"2024-03-22T09:56:43.480866+0000","172.21.15.153:6828/866138999":"2024-03-22T10:19:05.021528+0000","172.21.15.153:6800/270837948":"2024-03-22T09:56:00.849533+0000","172.21.15.153:6801/270837948":"2024-03-22T09:56:00.849533+0000","172.21.15.174:0/2577665242":"2024-03-22T10:08:33.113521+0000","172.21.15.153:6800/1639929315":"2024-03-22T10:07:30.531722+0000","172.21.15.153:6801/2524145044":"2024-03-22T10:13:31.087809+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T10:52:55.095 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:55.098+0000 7f9133e00640 1 -- 172.21.15.153:0/2318613128 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f911806e490 msgr2=0x7f9118070950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:52:55.095 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:55.098+0000 7f9133e00640 1 --2- 172.21.15.153:0/2318613128 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f911806e490 0x7f9118070950 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f9134069300 tx=0x7f9120005cc0 comp rx=0 tx=0).stop 2024-03-21T10:52:55.095 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:55.098+0000 7f9133e00640 1 -- 172.21.15.153:0/2318613128 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9134100120 msgr2=0x7f913419b8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:52:55.095 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:55.098+0000 7f9133e00640 1 --2- 172.21.15.153:0/2318613128 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9134100120 0x7f913419b8a0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f9124036040 tx=0x7f9124004270 comp rx=0 tx=0).stop 2024-03-21T10:52:55.095 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:55.098+0000 7f9133e00640 1 -- 172.21.15.153:0/2318613128 shutdown_connections 2024-03-21T10:52:55.095 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:55.098+0000 7f9133e00640 1 --2- 172.21.15.153:0/2318613128 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f911806e490 0x7f9118070950 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:52:55.095 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:55.098+0000 7f9133e00640 1 --2- 172.21.15.153:0/2318613128 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f913419bde0 0x7f9134195910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:52:55.095 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:55.098+0000 7f9133e00640 1 --2- 172.21.15.153:0/2318613128 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9134100120 0x7f913419b8a0 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:52:55.095 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:55.098+0000 7f9133e00640 1 -- 172.21.15.153:0/2318613128 >> 172.21.15.153:0/2318613128 conn(0x7f91340fb9c0 msgr2=0x7f91340fd080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:52:55.095 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:55.098+0000 7f9133e00640 1 -- 172.21.15.153:0/2318613128 shutdown_connections 2024-03-21T10:52:55.096 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:55.098+0000 7f9133e00640 1 -- 172.21.15.153:0/2318613128 wait complete. 2024-03-21T10:52:55.112 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-21T10:01:18.943639+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-03-21T10:04:25.621358+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.309999942779541, 'score_stable': 1.309999942779541, 'optimal_score': 1, 'raw_score_acting': 1.309999942779541, 'raw_score_stable': 1.309999942779541, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-03-21T10:04:26.593674+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-21T10:52:55.112 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-21T10:52:55.330 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:55.333+0000 7fc164600640 1 -- 172.21.15.153:0/4189617927 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc15c1024a0 msgr2=0x7fc15c102880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:52:55.330 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:55.333+0000 7fc164600640 1 --2- 172.21.15.153:0/4189617927 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc15c1024a0 0x7fc15c102880 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fc150009880 tx=0x7fc15002bcf0 comp rx=0 tx=0).stop 2024-03-21T10:52:55.330 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:55.333+0000 7fc164600640 1 -- 172.21.15.153:0/4189617927 shutdown_connections 2024-03-21T10:52:55.330 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:55.333+0000 7fc164600640 1 --2- 172.21.15.153:0/4189617927 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc15c102e50 0x7fc15c106e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:52:55.330 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:55.333+0000 7fc164600640 1 --2- 172.21.15.153:0/4189617927 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc15c1024a0 0x7fc15c102880 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:52:55.330 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:55.333+0000 7fc164600640 1 -- 172.21.15.153:0/4189617927 >> 172.21.15.153:0/4189617927 conn(0x7fc15c0fdd00 msgr2=0x7fc15c100120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:52:55.330 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:55.333+0000 7fc164600640 1 -- 172.21.15.153:0/4189617927 shutdown_connections 2024-03-21T10:52:55.331 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:55.333+0000 7fc164600640 1 -- 172.21.15.153:0/4189617927 wait complete. 2024-03-21T10:52:55.332 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:55.334+0000 7fc164600640 1 Processor -- start 2024-03-21T10:52:55.333 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:55.334+0000 7fc164600640 1 -- start start 2024-03-21T10:52:55.333 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:55.335+0000 7fc164600640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc15c1024a0 0x7fc15c198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:52:55.333 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:55.335+0000 7fc164600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc15c198580 0x7fc15c19c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:52:55.333 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:55.335+0000 7fc164600640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc15c198b80 con 0x7fc15c198580 2024-03-21T10:52:55.333 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:55.335+0000 7fc164600640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc15c198cf0 con 0x7fc15c1024a0 2024-03-21T10:52:55.333 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:55.335+0000 7fc161200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc15c198580 0x7fc15c19c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:52:55.333 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:55.335+0000 7fc161200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc15c198580 0x7fc15c19c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:34090/0 (socket says 172.21.15.153:34090) 2024-03-21T10:52:55.333 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:55.335+0000 7fc161200640 1 -- 172.21.15.153:0/3253946788 learned_addr learned my addr 172.21.15.153:0/3253946788 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:52:55.333 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:55.335+0000 7fc161c00640 1 --2- 172.21.15.153:0/3253946788 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc15c1024a0 0x7fc15c198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:52:55.333 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:55.335+0000 7fc161200640 1 -- 172.21.15.153:0/3253946788 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc15c1024a0 msgr2=0x7fc15c198040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:52:55.333 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:55.335+0000 7fc161200640 1 --2- 172.21.15.153:0/3253946788 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc15c1024a0 0x7fc15c198040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:52:55.333 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:55.335+0000 7fc161200640 1 -- 172.21.15.153:0/3253946788 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc150009560 con 0x7fc15c198580 2024-03-21T10:52:55.333 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:55.336+0000 7fc161200640 1 --2- 172.21.15.153:0/3253946788 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc15c198580 0x7fc15c19c9e0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fc14800d6f0 tx=0x7fc14800dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:52:55.334 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:55.336+0000 7fc15a000640 1 -- 172.21.15.153:0/3253946788 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc1480051f0 con 0x7fc15c198580 2024-03-21T10:52:55.334 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:55.336+0000 7fc15a000640 1 -- 172.21.15.153:0/3253946788 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc148009ce0 con 0x7fc15c198580 2024-03-21T10:52:55.334 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:55.336+0000 7fc164600640 1 -- 172.21.15.153:0/3253946788 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc15c19cfe0 con 0x7fc15c198580 2024-03-21T10:52:55.334 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:55.336+0000 7fc15a000640 1 -- 172.21.15.153:0/3253946788 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc148004e80 con 0x7fc15c198580 2024-03-21T10:52:55.334 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:55.336+0000 7fc164600640 1 -- 172.21.15.153:0/3253946788 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc15c19d540 con 0x7fc15c198580 2024-03-21T10:52:55.334 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:55.337+0000 7fc164600640 1 -- 172.21.15.153:0/3253946788 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc15c19d810 con 0x7fc15c198580 2024-03-21T10:52:55.335 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:55.339+0000 7fc15a000640 1 -- 172.21.15.153:0/3253946788 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fc1480046c0 con 0x7fc15c198580 2024-03-21T10:52:55.339 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:55.343+0000 7fc15a000640 1 --2- 172.21.15.153:0/3253946788 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fc14406e3d0 0x7fc144070890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:52:55.339 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:55.343+0000 7fc15a000640 1 -- 172.21.15.153:0/3253946788 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fc14808e060 con 0x7fc15c198580 2024-03-21T10:52:55.340 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:55.343+0000 7fc161c00640 1 --2- 172.21.15.153:0/3253946788 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fc14406e3d0 0x7fc144070890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:52:55.340 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:55.344+0000 7fc161c00640 1 --2- 172.21.15.153:0/3253946788 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fc14406e3d0 0x7fc144070890 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fc15002c8b0 tx=0x7fc150000bc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:52:55.342 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:55.346+0000 7fc15a000640 1 -- 172.21.15.153:0/3253946788 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc1480c2740 con 0x7fc15c198580 2024-03-21T10:52:55.611 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:55.615+0000 7fc164600640 1 -- 172.21.15.153:0/3253946788 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc15c1993f0 con 0x7fc15c198580 2024-03-21T10:52:55.612 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:55.615+0000 7fc15a000640 1 -- 172.21.15.153:0/3253946788 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v98) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc148057080 con 0x7fc15c198580 2024-03-21T10:52:55.612 INFO:teuthology.orchestra.run.smithi153.stdout:pg_num: 1 2024-03-21T10:52:55.615 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:55.619+0000 7fc164600640 1 -- 172.21.15.153:0/3253946788 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fc14406e3d0 msgr2=0x7fc144070890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:52:55.615 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:55.619+0000 7fc164600640 1 --2- 172.21.15.153:0/3253946788 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fc14406e3d0 0x7fc144070890 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fc15002c8b0 tx=0x7fc150000bc0 comp rx=0 tx=0).stop 2024-03-21T10:52:55.615 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:55.619+0000 7fc164600640 1 -- 172.21.15.153:0/3253946788 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc15c198580 msgr2=0x7fc15c19c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:52:55.616 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:55.619+0000 7fc164600640 1 --2- 172.21.15.153:0/3253946788 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc15c198580 0x7fc15c19c9e0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fc14800d6f0 tx=0x7fc14800dbb0 comp rx=0 tx=0).stop 2024-03-21T10:52:55.616 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:55.619+0000 7fc164600640 1 -- 172.21.15.153:0/3253946788 shutdown_connections 2024-03-21T10:52:55.616 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:55.619+0000 7fc164600640 1 --2- 172.21.15.153:0/3253946788 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fc14406e3d0 0x7fc144070890 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:52:55.616 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:55.619+0000 7fc164600640 1 --2- 172.21.15.153:0/3253946788 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc15c198580 0x7fc15c19c9e0 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:52:55.616 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:55.619+0000 7fc164600640 1 --2- 172.21.15.153:0/3253946788 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc15c1024a0 0x7fc15c198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:52:55.616 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:55.619+0000 7fc164600640 1 -- 172.21.15.153:0/3253946788 >> 172.21.15.153:0/3253946788 conn(0x7fc15c0fdd00 msgr2=0x7fc15c0ff3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:52:55.616 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:55.619+0000 7fc164600640 1 -- 172.21.15.153:0/3253946788 shutdown_connections 2024-03-21T10:52:55.616 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:55.619+0000 7fc164600640 1 -- 172.21.15.153:0/3253946788 wait complete. 2024-03-21T10:52:55.633 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-03-21T10:52:55.841 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:55.844+0000 7f94f6400640 1 -- 172.21.15.153:0/722654151 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f94f01022e0 msgr2=0x7f94f01026c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:52:55.841 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:55.844+0000 7f94f6400640 1 --2- 172.21.15.153:0/722654151 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f94f01022e0 0x7f94f01026c0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f94e4009880 tx=0x7f94e402bce0 comp rx=0 tx=0).stop 2024-03-21T10:52:55.841 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:55.844+0000 7f94f6400640 1 -- 172.21.15.153:0/722654151 shutdown_connections 2024-03-21T10:52:55.841 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:55.844+0000 7f94f6400640 1 --2- 172.21.15.153:0/722654151 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f94f0102c90 0x7f94f0106cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:52:55.841 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:55.844+0000 7f94f6400640 1 --2- 172.21.15.153:0/722654151 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f94f01022e0 0x7f94f01026c0 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:52:55.841 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:55.844+0000 7f94f6400640 1 -- 172.21.15.153:0/722654151 >> 172.21.15.153:0/722654151 conn(0x7f94f00fdb60 msgr2=0x7f94f00fff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:52:55.841 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:55.844+0000 7f94f6400640 1 -- 172.21.15.153:0/722654151 shutdown_connections 2024-03-21T10:52:55.841 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:55.844+0000 7f94f6400640 1 -- 172.21.15.153:0/722654151 wait complete. 2024-03-21T10:52:55.843 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:55.845+0000 7f94f6400640 1 Processor -- start 2024-03-21T10:52:55.843 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:55.845+0000 7f94f6400640 1 -- start start 2024-03-21T10:52:55.843 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:55.845+0000 7f94f6400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f94f01022e0 0x7f94f0197dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:52:55.843 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:55.845+0000 7f94f6400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f94f0198310 0x7f94f019c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:52:55.843 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:55.845+0000 7f94f6400640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94f0198910 con 0x7f94f01022e0 2024-03-21T10:52:55.843 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:55.845+0000 7f94f6400640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94f0198a80 con 0x7f94f0198310 2024-03-21T10:52:55.843 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:55.846+0000 7f94f5000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f94f01022e0 0x7f94f0197dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:52:55.843 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:55.846+0000 7f94f5000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f94f01022e0 0x7f94f0197dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:34110/0 (socket says 172.21.15.153:34110) 2024-03-21T10:52:55.844 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:55.846+0000 7f94f5000640 1 -- 172.21.15.153:0/1376962282 learned_addr learned my addr 172.21.15.153:0/1376962282 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:52:55.844 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:55.846+0000 7f94efe00640 1 --2- 172.21.15.153:0/1376962282 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f94f0198310 0x7f94f019c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:52:55.844 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:55.846+0000 7f94f5000640 1 -- 172.21.15.153:0/1376962282 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f94f0198310 msgr2=0x7f94f019c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:52:55.844 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:55.846+0000 7f94f5000640 1 --2- 172.21.15.153:0/1376962282 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f94f0198310 0x7f94f019c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:52:55.844 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:55.846+0000 7f94f5000640 1 -- 172.21.15.153:0/1376962282 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94e4009560 con 0x7f94f01022e0 2024-03-21T10:52:55.844 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:55.847+0000 7f94f5000640 1 --2- 172.21.15.153:0/1376962282 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f94f01022e0 0x7f94f0197dd0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f94e4009430 tx=0x7f94e4004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:52:55.845 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:55.847+0000 7f94ed600640 1 -- 172.21.15.153:0/1376962282 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f94e40035f0 con 0x7f94f01022e0 2024-03-21T10:52:55.845 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:55.847+0000 7f94ed600640 1 -- 172.21.15.153:0/1376962282 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f94e4003c10 con 0x7f94f01022e0 2024-03-21T10:52:55.845 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:55.847+0000 7f94ed600640 1 -- 172.21.15.153:0/1376962282 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f94e4034440 con 0x7f94f01022e0 2024-03-21T10:52:55.845 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:55.847+0000 7f94f6400640 1 -- 172.21.15.153:0/1376962282 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94f019cd10 con 0x7f94f01022e0 2024-03-21T10:52:55.845 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:55.847+0000 7f94f6400640 1 -- 172.21.15.153:0/1376962282 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94f019d270 con 0x7f94f01022e0 2024-03-21T10:52:55.845 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:55.849+0000 7f94f6400640 1 -- 172.21.15.153:0/1376962282 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94f004e040 con 0x7f94f01022e0 2024-03-21T10:52:55.846 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:55.850+0000 7f94ed600640 1 -- 172.21.15.153:0/1376962282 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f94e4004fd0 con 0x7f94f01022e0 2024-03-21T10:52:55.847 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:55.851+0000 7f94ed600640 1 --2- 172.21.15.153:0/1376962282 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f94d806e3d0 0x7f94d8070890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:52:55.848 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:55.851+0000 7f94ed600640 1 -- 172.21.15.153:0/1376962282 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f94e40b0a30 con 0x7f94f01022e0 2024-03-21T10:52:55.848 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:55.851+0000 7f94efe00640 1 --2- 172.21.15.153:0/1376962282 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f94d806e3d0 0x7f94d8070890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:52:55.848 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:55.852+0000 7f94efe00640 1 --2- 172.21.15.153:0/1376962282 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f94d806e3d0 0x7f94d8070890 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f94dc0058c0 tx=0x7f94dc005c60 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:52:55.852 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:55.856+0000 7f94ed600640 1 -- 172.21.15.153:0/1376962282 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f94e4079a50 con 0x7f94f01022e0 2024-03-21T10:52:55.946 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:52:55 smithi174 ceph-mon[88004]: pgmap v1234: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:52:55.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:52:55 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/2318613128' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T10:52:55.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:52:55 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/3253946788' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T10:52:56.051 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:52:55 smithi153 ceph-mon[113733]: pgmap v1234: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:52:56.051 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:52:55 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/2318613128' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T10:52:56.052 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:52:55 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/3253946788' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T10:52:56.053 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:56.057+0000 7f94f6400640 1 -- 172.21.15.153:0/1376962282 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f94f010a5e0 con 0x7f94f01022e0 2024-03-21T10:52:56.054 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:56.057+0000 7f94ed600640 1 -- 172.21.15.153:0/1376962282 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v98) v1 ==== 107+0+11 (secure 0 0 0) 0x7f94e407d110 con 0x7f94f01022e0 2024-03-21T10:52:56.054 INFO:teuthology.orchestra.run.smithi153.stdout:pg_num: 32 2024-03-21T10:52:56.057 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:56.060+0000 7f94f6400640 1 -- 172.21.15.153:0/1376962282 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f94d806e3d0 msgr2=0x7f94d8070890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:52:56.057 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:56.060+0000 7f94f6400640 1 --2- 172.21.15.153:0/1376962282 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f94d806e3d0 0x7f94d8070890 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f94dc0058c0 tx=0x7f94dc005c60 comp rx=0 tx=0).stop 2024-03-21T10:52:56.057 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:56.060+0000 7f94f6400640 1 -- 172.21.15.153:0/1376962282 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f94f01022e0 msgr2=0x7f94f0197dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:52:56.057 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:56.060+0000 7f94f6400640 1 --2- 172.21.15.153:0/1376962282 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f94f01022e0 0x7f94f0197dd0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f94e4009430 tx=0x7f94e4004040 comp rx=0 tx=0).stop 2024-03-21T10:52:56.057 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:56.060+0000 7f94f6400640 1 -- 172.21.15.153:0/1376962282 shutdown_connections 2024-03-21T10:52:56.057 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:56.061+0000 7f94f6400640 1 --2- 172.21.15.153:0/1376962282 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f94d806e3d0 0x7f94d8070890 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:52:56.058 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:56.061+0000 7f94f6400640 1 --2- 172.21.15.153:0/1376962282 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f94f0198310 0x7f94f019c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:52:56.058 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:56.061+0000 7f94f6400640 1 --2- 172.21.15.153:0/1376962282 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f94f01022e0 0x7f94f0197dd0 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:52:56.058 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:56.061+0000 7f94f6400640 1 -- 172.21.15.153:0/1376962282 >> 172.21.15.153:0/1376962282 conn(0x7f94f00fdb60 msgr2=0x7f94f00ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:52:56.058 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:56.061+0000 7f94f6400640 1 -- 172.21.15.153:0/1376962282 shutdown_connections 2024-03-21T10:52:56.058 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:56.061+0000 7f94f6400640 1 -- 172.21.15.153:0/1376962282 wait complete. 2024-03-21T10:52:56.075 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-03-21T10:52:56.271 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:56.274+0000 7fe729800640 1 -- 172.21.15.153:0/804468167 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe724067f70 msgr2=0x7fe724104c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:52:56.271 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:56.274+0000 7fe729800640 1 --2- 172.21.15.153:0/804468167 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe724067f70 0x7fe724104c70 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fe70c009880 tx=0x7fe70c02bce0 comp rx=0 tx=0).stop 2024-03-21T10:52:56.271 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:56.274+0000 7fe729800640 1 -- 172.21.15.153:0/804468167 shutdown_connections 2024-03-21T10:52:56.271 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:56.274+0000 7fe729800640 1 --2- 172.21.15.153:0/804468167 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe724067f70 0x7fe724104c70 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:52:56.271 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:56.274+0000 7fe729800640 1 --2- 172.21.15.153:0/804468167 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe7240675c0 0x7fe7240679a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:52:56.271 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:56.274+0000 7fe729800640 1 -- 172.21.15.153:0/804468167 >> 172.21.15.153:0/804468167 conn(0x7fe724074a50 msgr2=0x7fe724074e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:52:56.271 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:56.274+0000 7fe729800640 1 -- 172.21.15.153:0/804468167 shutdown_connections 2024-03-21T10:52:56.271 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:56.274+0000 7fe729800640 1 -- 172.21.15.153:0/804468167 wait complete. 2024-03-21T10:52:56.272 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:56.276+0000 7fe729800640 1 Processor -- start 2024-03-21T10:52:56.272 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:56.276+0000 7fe729800640 1 -- start start 2024-03-21T10:52:56.273 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:56.276+0000 7fe729800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe7240675c0 0x7fe724195d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:52:56.273 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:56.277+0000 7fe729800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe7241962a0 0x7fe72419a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:52:56.273 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:56.277+0000 7fe729800640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7241968a0 con 0x7fe7240675c0 2024-03-21T10:52:56.273 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:56.277+0000 7fe729800640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe724196a10 con 0x7fe7241962a0 2024-03-21T10:52:56.273 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:56.277+0000 7fe722a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe7240675c0 0x7fe724195d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:52:56.273 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:56.277+0000 7fe722a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe7240675c0 0x7fe724195d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:34118/0 (socket says 172.21.15.153:34118) 2024-03-21T10:52:56.274 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:56.277+0000 7fe722a00640 1 -- 172.21.15.153:0/1550522226 learned_addr learned my addr 172.21.15.153:0/1550522226 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:52:56.275 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:56.277+0000 7fe71a000640 1 --2- 172.21.15.153:0/1550522226 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe7241962a0 0x7fe72419a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:52:56.275 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:56.277+0000 7fe722a00640 1 -- 172.21.15.153:0/1550522226 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe7241962a0 msgr2=0x7fe72419a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:52:56.275 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:56.277+0000 7fe722a00640 1 --2- 172.21.15.153:0/1550522226 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe7241962a0 0x7fe72419a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:52:56.275 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:56.277+0000 7fe722a00640 1 -- 172.21.15.153:0/1550522226 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe70c009560 con 0x7fe7240675c0 2024-03-21T10:52:56.275 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:56.278+0000 7fe722a00640 1 --2- 172.21.15.153:0/1550522226 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe7240675c0 0x7fe724195d60 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fe71400d900 tx=0x7fe71400ddc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:52:56.275 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:56.278+0000 7fe71be00640 1 -- 172.21.15.153:0/1550522226 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe714009c00 con 0x7fe7240675c0 2024-03-21T10:52:56.275 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:56.278+0000 7fe71be00640 1 -- 172.21.15.153:0/1550522226 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe714009d60 con 0x7fe7240675c0 2024-03-21T10:52:56.275 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:56.278+0000 7fe729800640 1 -- 172.21.15.153:0/1550522226 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe72419ad00 con 0x7fe7240675c0 2024-03-21T10:52:56.275 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:56.278+0000 7fe71be00640 1 -- 172.21.15.153:0/1550522226 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe7140049d0 con 0x7fe7240675c0 2024-03-21T10:52:56.275 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:56.278+0000 7fe729800640 1 -- 172.21.15.153:0/1550522226 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe72419b260 con 0x7fe7240675c0 2024-03-21T10:52:56.276 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:56.280+0000 7fe729800640 1 -- 172.21.15.153:0/1550522226 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe72419b540 con 0x7fe7240675c0 2024-03-21T10:52:56.277 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:56.280+0000 7fe71be00640 1 -- 172.21.15.153:0/1550522226 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fe714004020 con 0x7fe7240675c0 2024-03-21T10:52:56.278 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:56.281+0000 7fe71be00640 1 --2- 172.21.15.153:0/1550522226 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe70806e3d0 0x7fe708070890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:52:56.278 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:56.281+0000 7fe71be00640 1 -- 172.21.15.153:0/1550522226 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fe71408e340 con 0x7fe7240675c0 2024-03-21T10:52:56.278 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:56.282+0000 7fe71a000640 1 --2- 172.21.15.153:0/1550522226 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe70806e3d0 0x7fe708070890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:52:56.278 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:56.282+0000 7fe71a000640 1 --2- 172.21.15.153:0/1550522226 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe70806e3d0 0x7fe708070890 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fe70c036040 tx=0x7fe70c02cf90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:52:56.286 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:56.290+0000 7fe71be00640 1 -- 172.21.15.153:0/1550522226 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe714057360 con 0x7fe7240675c0 2024-03-21T10:52:56.486 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:56.489+0000 7fe729800640 1 -- 172.21.15.153:0/1550522226 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7fe724108580 con 0x7fe7240675c0 2024-03-21T10:52:56.486 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:56.490+0000 7fe71be00640 1 -- 172.21.15.153:0/1550522226 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v98) v1 ==== 107+0+11 (secure 0 0 0) 0x7fe71405aa20 con 0x7fe7240675c0 2024-03-21T10:52:56.487 INFO:teuthology.orchestra.run.smithi153.stdout:pg_num: 32 2024-03-21T10:52:56.489 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:56.492+0000 7fe729800640 1 -- 172.21.15.153:0/1550522226 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe70806e3d0 msgr2=0x7fe708070890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:52:56.489 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:56.492+0000 7fe729800640 1 --2- 172.21.15.153:0/1550522226 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe70806e3d0 0x7fe708070890 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fe70c036040 tx=0x7fe70c02cf90 comp rx=0 tx=0).stop 2024-03-21T10:52:56.489 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:56.492+0000 7fe729800640 1 -- 172.21.15.153:0/1550522226 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe7240675c0 msgr2=0x7fe724195d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:52:56.489 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:56.492+0000 7fe729800640 1 --2- 172.21.15.153:0/1550522226 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe7240675c0 0x7fe724195d60 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fe71400d900 tx=0x7fe71400ddc0 comp rx=0 tx=0).stop 2024-03-21T10:52:56.490 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:56.493+0000 7fe729800640 1 -- 172.21.15.153:0/1550522226 shutdown_connections 2024-03-21T10:52:56.490 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:56.493+0000 7fe729800640 1 --2- 172.21.15.153:0/1550522226 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe70806e3d0 0x7fe708070890 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:52:56.490 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:56.493+0000 7fe729800640 1 --2- 172.21.15.153:0/1550522226 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe7241962a0 0x7fe72419a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:52:56.490 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:56.493+0000 7fe729800640 1 --2- 172.21.15.153:0/1550522226 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe7240675c0 0x7fe724195d60 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:52:56.490 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:56.493+0000 7fe729800640 1 -- 172.21.15.153:0/1550522226 >> 172.21.15.153:0/1550522226 conn(0x7fe724074a50 msgr2=0x7fe7240fdbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:52:56.490 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:56.493+0000 7fe729800640 1 -- 172.21.15.153:0/1550522226 shutdown_connections 2024-03-21T10:52:56.490 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:56.493+0000 7fe729800640 1 -- 172.21.15.153:0/1550522226 wait complete. 2024-03-21T10:52:56.510 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T10:52:56.716 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:56.719+0000 7fed14200640 1 -- 172.21.15.153:0/3601973673 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fed0c100280 msgr2=0x7fed0c100660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:52:56.716 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:56.719+0000 7fed14200640 1 --2- 172.21.15.153:0/3601973673 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fed0c100280 0x7fed0c100660 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fed00009880 tx=0x7fed0002bce0 comp rx=0 tx=0).stop 2024-03-21T10:52:56.716 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:56.719+0000 7fed14200640 1 -- 172.21.15.153:0/3601973673 shutdown_connections 2024-03-21T10:52:56.716 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:56.719+0000 7fed14200640 1 --2- 172.21.15.153:0/3601973673 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fed0c100c30 0x7fed0c104c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:52:56.716 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:56.719+0000 7fed14200640 1 --2- 172.21.15.153:0/3601973673 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fed0c100280 0x7fed0c100660 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:52:56.716 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:56.719+0000 7fed14200640 1 -- 172.21.15.153:0/3601973673 >> 172.21.15.153:0/3601973673 conn(0x7fed0c0fbb20 msgr2=0x7fed0c0fdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:52:56.716 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:56.719+0000 7fed14200640 1 -- 172.21.15.153:0/3601973673 shutdown_connections 2024-03-21T10:52:56.716 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:56.719+0000 7fed14200640 1 -- 172.21.15.153:0/3601973673 wait complete. 2024-03-21T10:52:56.718 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:56.720+0000 7fed14200640 1 Processor -- start 2024-03-21T10:52:56.718 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:56.720+0000 7fed14200640 1 -- start start 2024-03-21T10:52:56.718 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:56.720+0000 7fed14200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fed0c100280 0x7fed0c195cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:52:56.718 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:56.720+0000 7fed14200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fed0c1961f0 0x7fed0c1995a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:52:56.718 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:56.720+0000 7fed14200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed0c1967f0 con 0x7fed0c1961f0 2024-03-21T10:52:56.718 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:56.720+0000 7fed14200640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed0c196930 con 0x7fed0c100280 2024-03-21T10:52:56.718 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:56.721+0000 7fed10e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fed0c1961f0 0x7fed0c1995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:52:56.718 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:56.721+0000 7fed10e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fed0c1961f0 0x7fed0c1995a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:34124/0 (socket says 172.21.15.153:34124) 2024-03-21T10:52:56.718 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:56.721+0000 7fed10e00640 1 -- 172.21.15.153:0/459522013 learned_addr learned my addr 172.21.15.153:0/459522013 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:52:56.718 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:56.721+0000 7fed10e00640 1 -- 172.21.15.153:0/459522013 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fed0c100280 msgr2=0x7fed0c195cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:52:56.718 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:56.721+0000 7fed10e00640 1 --2- 172.21.15.153:0/459522013 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fed0c100280 0x7fed0c195cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:52:56.718 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:56.721+0000 7fed10e00640 1 -- 172.21.15.153:0/459522013 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed00009560 con 0x7fed0c1961f0 2024-03-21T10:52:56.718 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:56.721+0000 7fed10e00640 1 --2- 172.21.15.153:0/459522013 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fed0c1961f0 0x7fed0c1995a0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fecf8005fd0 tx=0x7fecf800dd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:52:56.719 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:56.722+0000 7fed0a000640 1 -- 172.21.15.153:0/459522013 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fecf8009c00 con 0x7fed0c1961f0 2024-03-21T10:52:56.719 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:56.722+0000 7fed0a000640 1 -- 172.21.15.153:0/459522013 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fecf800de90 con 0x7fed0c1961f0 2024-03-21T10:52:56.719 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:56.722+0000 7fed14200640 1 -- 172.21.15.153:0/459522013 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed0c199ba0 con 0x7fed0c1961f0 2024-03-21T10:52:56.720 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:56.722+0000 7fed0a000640 1 -- 172.21.15.153:0/459522013 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fecf8003ea0 con 0x7fed0c1961f0 2024-03-21T10:52:56.720 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:56.722+0000 7fed14200640 1 -- 172.21.15.153:0/459522013 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed0c19a100 con 0x7fed0c1961f0 2024-03-21T10:52:56.720 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:56.723+0000 7fed14200640 1 -- 172.21.15.153:0/459522013 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed0c04e040 con 0x7fed0c1961f0 2024-03-21T10:52:56.720 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:56.724+0000 7fed0a000640 1 -- 172.21.15.153:0/459522013 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fecf8004000 con 0x7fed0c1961f0 2024-03-21T10:52:56.722 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:56.725+0000 7fed0a000640 1 --2- 172.21.15.153:0/459522013 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fecf406e490 0x7fecf4070950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:52:56.722 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:56.725+0000 7fed0a000640 1 -- 172.21.15.153:0/459522013 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fecf808e450 con 0x7fed0c1961f0 2024-03-21T10:52:56.722 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:56.726+0000 7fed11800640 1 --2- 172.21.15.153:0/459522013 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fecf406e490 0x7fecf4070950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:52:56.723 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:56.727+0000 7fed11800640 1 --2- 172.21.15.153:0/459522013 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fecf406e490 0x7fecf4070950 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fed0002efd0 tx=0x7fed00000bc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:52:56.729 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:56.731+0000 7fed0a000640 1 -- 172.21.15.153:0/459522013 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fecf8057470 con 0x7fed0c1961f0 2024-03-21T10:52:56.801 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:52:56 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/1376962282' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-21T10:52:56.801 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:52:56 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/1550522226' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-21T10:52:56.946 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:52:56 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/1376962282' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-21T10:52:56.946 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:52:56 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/1550522226' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-21T10:52:57.003 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:57.006+0000 7fed14200640 1 -- 172.21.15.153:0/459522013 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fed0c108580 con 0x7fed0c1961f0 2024-03-21T10:52:57.004 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:57.007+0000 7fed0a000640 1 -- 172.21.15.153:0/459522013 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+5090 (secure 0 0 0) 0x7fecf805ab30 con 0x7fed0c1961f0 2024-03-21T10:52:57.004 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:52:57.004 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":41,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":41,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T10:04:27.086576+0000","modified":"2024-03-21T10:51:02.096522+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":94,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34442,"mds_1":34474},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34442":{"gid":34442,"name":"cephfs.smithi153.zipckr","rank":0,"incarnation":34,"state":"up:active","state_seq":474,"addr":"172.21.15.153:6827/1103846295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":1103846295},{"type":"v1","addr":"172.21.15.153:6827","nonce":1103846295}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34474":{"gid":34474,"name":"cephfs.smithi153.iuyluj","rank":1,"incarnation":34,"state":"up:active","state_seq":468,"addr":"172.21.15.153:6829/1738630832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":1738630832},{"type":"v1","addr":"172.21.15.153:6829","nonce":1738630832}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_44313":{"gid":44313,"name":"cephfs.smithi174.yyimst","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6825/4127786797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":4127786797},{"type":"v1","addr":"172.21.15.174:6825","nonce":4127786797}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_44687":{"gid":44687,"name":"cephfs.smithi174.nhozag","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6827/2767517845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":2767517845},{"type":"v1","addr":"172.21.15.174:6827","nonce":2767517845}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34442,"qdb_cluster":[34474,34442]},"id":1}]} 2024-03-21T10:52:57.004 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 41 2024-03-21T10:52:57.006 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:57.009+0000 7fed14200640 1 -- 172.21.15.153:0/459522013 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fecf406e490 msgr2=0x7fecf4070950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:52:57.007 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:57.009+0000 7fed14200640 1 --2- 172.21.15.153:0/459522013 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fecf406e490 0x7fecf4070950 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fed0002efd0 tx=0x7fed00000bc0 comp rx=0 tx=0).stop 2024-03-21T10:52:57.007 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:57.010+0000 7fed14200640 1 -- 172.21.15.153:0/459522013 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fed0c1961f0 msgr2=0x7fed0c1995a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:52:57.007 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:57.010+0000 7fed14200640 1 --2- 172.21.15.153:0/459522013 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fed0c1961f0 0x7fed0c1995a0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fecf8005fd0 tx=0x7fecf800dd60 comp rx=0 tx=0).stop 2024-03-21T10:52:57.007 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:57.010+0000 7fed14200640 1 -- 172.21.15.153:0/459522013 shutdown_connections 2024-03-21T10:52:57.007 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:57.010+0000 7fed14200640 1 --2- 172.21.15.153:0/459522013 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fecf406e490 0x7fecf4070950 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:52:57.007 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:57.010+0000 7fed14200640 1 --2- 172.21.15.153:0/459522013 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fed0c1961f0 0x7fed0c1995a0 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:52:57.007 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:57.010+0000 7fed14200640 1 --2- 172.21.15.153:0/459522013 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fed0c100280 0x7fed0c195cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:52:57.007 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:57.010+0000 7fed14200640 1 -- 172.21.15.153:0/459522013 >> 172.21.15.153:0/459522013 conn(0x7fed0c0fbb20 msgr2=0x7fed0c0fd200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:52:57.007 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:57.010+0000 7fed14200640 1 -- 172.21.15.153:0/459522013 shutdown_connections 2024-03-21T10:52:57.007 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:57.010+0000 7fed14200640 1 -- 172.21.15.153:0/459522013 wait complete. 2024-03-21T10:52:57.024 DEBUG:tasks.fs:checking fs fscid=1,name=cephfs state = {'epoch': 17, 'max_mds': 2, 'flags': 50} 2024-03-21T10:52:57.025 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 18 2024-03-21T10:52:57.220 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:57.223+0000 7f0e17200640 1 -- 172.21.15.153:0/880333274 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e10101200 msgr2=0x7f0e101015e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:52:57.221 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:57.223+0000 7f0e17200640 1 --2- 172.21.15.153:0/880333274 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e10101200 0x7f0e101015e0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f0e00009880 tx=0x7f0e0002bd70 comp rx=0 tx=0).stop 2024-03-21T10:52:57.221 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:57.224+0000 7f0e17200640 1 -- 172.21.15.153:0/880333274 shutdown_connections 2024-03-21T10:52:57.221 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:57.224+0000 7f0e17200640 1 --2- 172.21.15.153:0/880333274 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0e10101b20 0x7f0e1010a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:52:57.221 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:57.224+0000 7f0e17200640 1 --2- 172.21.15.153:0/880333274 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e10101200 0x7f0e101015e0 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:52:57.221 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:57.224+0000 7f0e17200640 1 -- 172.21.15.153:0/880333274 >> 172.21.15.153:0/880333274 conn(0x7f0e100faa20 msgr2=0x7f0e100fce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:52:57.221 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:57.224+0000 7f0e17200640 1 -- 172.21.15.153:0/880333274 shutdown_connections 2024-03-21T10:52:57.221 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:57.224+0000 7f0e17200640 1 -- 172.21.15.153:0/880333274 wait complete. 2024-03-21T10:52:57.223 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:57.225+0000 7f0e17200640 1 Processor -- start 2024-03-21T10:52:57.223 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:57.225+0000 7f0e17200640 1 -- start start 2024-03-21T10:52:57.223 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:57.225+0000 7f0e17200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e10101200 0x7f0e10195d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:52:57.224 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:57.225+0000 7f0e17200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0e10101b20 0x7f0e101962b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:52:57.224 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:57.225+0000 7f0e17200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e10196940 con 0x7f0e10101200 2024-03-21T10:52:57.224 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:57.225+0000 7f0e17200640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e1019a710 con 0x7f0e10101b20 2024-03-21T10:52:57.224 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:57.225+0000 7f0e0fe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e10101200 0x7f0e10195d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:52:57.224 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:57.225+0000 7f0e0fe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e10101200 0x7f0e10195d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:34128/0 (socket says 172.21.15.153:34128) 2024-03-21T10:52:57.224 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:57.225+0000 7f0e0fe00640 1 -- 172.21.15.153:0/2846394548 learned_addr learned my addr 172.21.15.153:0/2846394548 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:52:57.224 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:57.226+0000 7f0e0f400640 1 --2- 172.21.15.153:0/2846394548 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0e10101b20 0x7f0e101962b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:52:57.224 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:57.226+0000 7f0e0fe00640 1 -- 172.21.15.153:0/2846394548 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0e10101b20 msgr2=0x7f0e101962b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:52:57.224 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:57.226+0000 7f0e0fe00640 1 --2- 172.21.15.153:0/2846394548 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0e10101b20 0x7f0e101962b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:52:57.224 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:57.226+0000 7f0e0fe00640 1 -- 172.21.15.153:0/2846394548 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e00009560 con 0x7f0e10101200 2024-03-21T10:52:57.224 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:57.226+0000 7f0e0f400640 1 --2- 172.21.15.153:0/2846394548 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0e10101b20 0x7f0e101962b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:52:57.225 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:57.226+0000 7f0e0fe00640 1 --2- 172.21.15.153:0/2846394548 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e10101200 0x7f0e10195d70 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f0e00009230 tx=0x7f0e00004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:52:57.225 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:57.227+0000 7f0e0cc00640 1 -- 172.21.15.153:0/2846394548 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0e000035f0 con 0x7f0e10101200 2024-03-21T10:52:57.225 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:57.227+0000 7f0e17200640 1 -- 172.21.15.153:0/2846394548 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e1019a990 con 0x7f0e10101200 2024-03-21T10:52:57.225 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:57.227+0000 7f0e0cc00640 1 -- 172.21.15.153:0/2846394548 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0e00003c10 con 0x7f0e10101200 2024-03-21T10:52:57.225 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:57.227+0000 7f0e0cc00640 1 -- 172.21.15.153:0/2846394548 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0e0003e450 con 0x7f0e10101200 2024-03-21T10:52:57.225 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:57.227+0000 7f0e17200640 1 -- 172.21.15.153:0/2846394548 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e1019aef0 con 0x7f0e10101200 2024-03-21T10:52:57.225 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:57.228+0000 7f0e17200640 1 -- 172.21.15.153:0/2846394548 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e1004e040 con 0x7f0e10101200 2024-03-21T10:52:57.226 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:57.229+0000 7f0e0cc00640 1 -- 172.21.15.153:0/2846394548 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f0e00004f60 con 0x7f0e10101200 2024-03-21T10:52:57.227 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:57.230+0000 7f0e0cc00640 1 --2- 172.21.15.153:0/2846394548 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f0df406e3d0 0x7f0df4070890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:52:57.227 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:57.230+0000 7f0e0cc00640 1 -- 172.21.15.153:0/2846394548 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f0e000b1170 con 0x7f0e10101200 2024-03-21T10:52:57.227 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:57.230+0000 7f0e0f400640 1 --2- 172.21.15.153:0/2846394548 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f0df406e3d0 0x7f0df4070890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:52:57.227 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:57.231+0000 7f0e0f400640 1 --2- 172.21.15.153:0/2846394548 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f0df406e3d0 0x7f0df4070890 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f0e10197380 tx=0x7f0dfc009670 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:52:57.232 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:57.235+0000 7f0e0cc00640 1 -- 172.21.15.153:0/2846394548 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0e0007a190 con 0x7f0e10101200 2024-03-21T10:52:57.499 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:57.502+0000 7f0e17200640 1 -- 172.21.15.153:0/2846394548 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 18, "format": "json"} v 0) v1 -- 0x7f0e10107670 con 0x7f0e10101200 2024-03-21T10:52:57.499 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:57.503+0000 7f0e0cc00640 1 -- 172.21.15.153:0/2846394548 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 18, "format": "json"}]=0 dumped fsmap epoch 18 v41) v1 ==== 107+0+4895 (secure 0 0 0) 0x7f0e0007d850 con 0x7f0e10101200 2024-03-21T10:52:57.500 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:52:57.500 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":18,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T10:04:27.086576+0000","modified":"2024-03-21T10:07:06.496071+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14484,"mds_1":24273},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14484":{"gid":14484,"name":"cephfs.smithi153.iuyluj","rank":0,"incarnation":13,"state":"up:active","state_seq":6,"addr":"172.21.15.153:6829/866138999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":866138999},{"type":"v1","addr":"172.21.15.153:6829","nonce":866138999}]},"join_fscid":1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14494":{"gid":14494,"name":"cephfs.smithi153.zipckr","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6827/4033622615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":4033622615},{"type":"v1","addr":"172.21.15.153:6827","nonce":4033622615}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24273":{"gid":24273,"name":"cephfs.smithi174.yyimst","rank":1,"incarnation":7,"state":"up:active","state_seq":5,"addr":"172.21.15.174:6825/1776947618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":1776947618},{"type":"v1","addr":"172.21.15.174:6825","nonce":1776947618}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24285":{"gid":24285,"name":"cephfs.smithi174.nhozag","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":2,"addr":"172.21.15.174:6827/806095873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":806095873},{"type":"v1","addr":"172.21.15.174:6827","nonce":806095873}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-21T10:52:57.500 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 18 2024-03-21T10:52:57.503 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:57.506+0000 7f0e17200640 1 -- 172.21.15.153:0/2846394548 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f0df406e3d0 msgr2=0x7f0df4070890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:52:57.503 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:57.506+0000 7f0e17200640 1 --2- 172.21.15.153:0/2846394548 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f0df406e3d0 0x7f0df4070890 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f0e10197380 tx=0x7f0dfc009670 comp rx=0 tx=0).stop 2024-03-21T10:52:57.503 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:57.506+0000 7f0e17200640 1 -- 172.21.15.153:0/2846394548 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e10101200 msgr2=0x7f0e10195d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:52:57.503 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:57.506+0000 7f0e17200640 1 --2- 172.21.15.153:0/2846394548 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e10101200 0x7f0e10195d70 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f0e00009230 tx=0x7f0e00004040 comp rx=0 tx=0).stop 2024-03-21T10:52:57.503 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:57.506+0000 7f0e17200640 1 -- 172.21.15.153:0/2846394548 shutdown_connections 2024-03-21T10:52:57.503 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:57.506+0000 7f0e17200640 1 --2- 172.21.15.153:0/2846394548 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f0df406e3d0 0x7f0df4070890 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:52:57.503 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:57.506+0000 7f0e17200640 1 --2- 172.21.15.153:0/2846394548 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0e10101b20 0x7f0e101962b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:52:57.503 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:57.506+0000 7f0e17200640 1 --2- 172.21.15.153:0/2846394548 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0e10101200 0x7f0e10195d70 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:52:57.503 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:57.506+0000 7f0e17200640 1 -- 172.21.15.153:0/2846394548 >> 172.21.15.153:0/2846394548 conn(0x7f0e100faa20 msgr2=0x7f0e10104840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:52:57.503 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:57.506+0000 7f0e17200640 1 -- 172.21.15.153:0/2846394548 shutdown_connections 2024-03-21T10:52:57.503 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:57.506+0000 7f0e17200640 1 -- 172.21.15.153:0/2846394548 wait complete. 2024-03-21T10:52:57.523 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 19 2024-03-21T10:52:57.726 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:57.729+0000 7fb886200640 1 -- 172.21.15.153:0/528476831 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb8801000f0 msgr2=0x7fb8801004d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:52:57.726 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:57.729+0000 7fb886200640 1 --2- 172.21.15.153:0/528476831 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb8801000f0 0x7fb8801004d0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fb874009880 tx=0x7fb87402bce0 comp rx=0 tx=0).stop 2024-03-21T10:52:57.726 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:57.729+0000 7fb886200640 1 -- 172.21.15.153:0/528476831 shutdown_connections 2024-03-21T10:52:57.726 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:57.729+0000 7fb886200640 1 --2- 172.21.15.153:0/528476831 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb880100aa0 0x7fb880104ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:52:57.726 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:57.729+0000 7fb886200640 1 --2- 172.21.15.153:0/528476831 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb8801000f0 0x7fb8801004d0 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:52:57.726 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:57.729+0000 7fb886200640 1 -- 172.21.15.153:0/528476831 >> 172.21.15.153:0/528476831 conn(0x7fb8800fb970 msgr2=0x7fb8800fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:52:57.726 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:57.729+0000 7fb886200640 1 -- 172.21.15.153:0/528476831 shutdown_connections 2024-03-21T10:52:57.726 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:57.729+0000 7fb886200640 1 -- 172.21.15.153:0/528476831 wait complete. 2024-03-21T10:52:57.727 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:57.730+0000 7fb886200640 1 Processor -- start 2024-03-21T10:52:57.727 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:57.730+0000 7fb886200640 1 -- start start 2024-03-21T10:52:57.728 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:57.731+0000 7fb886200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb8801000f0 0x7fb880195cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:52:57.728 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:57.731+0000 7fb886200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb8801961f0 0x7fb8801995a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:52:57.728 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:57.731+0000 7fb886200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8801967f0 con 0x7fb8801000f0 2024-03-21T10:52:57.728 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:57.731+0000 7fb886200640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb880196930 con 0x7fb8801961f0 2024-03-21T10:52:57.729 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:57.731+0000 7fb884e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb8801000f0 0x7fb880195cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:52:57.729 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:57.731+0000 7fb884e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb8801000f0 0x7fb880195cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:34148/0 (socket says 172.21.15.153:34148) 2024-03-21T10:52:57.729 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:57.731+0000 7fb884e00640 1 -- 172.21.15.153:0/3397084237 learned_addr learned my addr 172.21.15.153:0/3397084237 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:52:57.729 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:57.732+0000 7fb87fe00640 1 --2- 172.21.15.153:0/3397084237 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb8801961f0 0x7fb8801995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:52:57.729 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:57.732+0000 7fb884e00640 1 -- 172.21.15.153:0/3397084237 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb8801961f0 msgr2=0x7fb8801995a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:52:57.729 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:57.732+0000 7fb884e00640 1 --2- 172.21.15.153:0/3397084237 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb8801961f0 0x7fb8801995a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:52:57.729 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:57.732+0000 7fb884e00640 1 -- 172.21.15.153:0/3397084237 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb874009560 con 0x7fb8801000f0 2024-03-21T10:52:57.729 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:57.732+0000 7fb87fe00640 1 --2- 172.21.15.153:0/3397084237 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb8801961f0 0x7fb8801995a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:52:57.729 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:57.732+0000 7fb884e00640 1 --2- 172.21.15.153:0/3397084237 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb8801000f0 0x7fb880195cb0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fb874000c00 tx=0x7fb874004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:52:57.730 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:57.732+0000 7fb87d600640 1 -- 172.21.15.153:0/3397084237 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb8740037c0 con 0x7fb8801000f0 2024-03-21T10:52:57.730 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:57.732+0000 7fb87d600640 1 -- 172.21.15.153:0/3397084237 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb874003de0 con 0x7fb8801000f0 2024-03-21T10:52:57.730 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:57.732+0000 7fb87d600640 1 -- 172.21.15.153:0/3397084237 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb874034460 con 0x7fb8801000f0 2024-03-21T10:52:57.730 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:57.732+0000 7fb886200640 1 -- 172.21.15.153:0/3397084237 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb880199b40 con 0x7fb8801000f0 2024-03-21T10:52:57.730 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:57.733+0000 7fb886200640 1 -- 172.21.15.153:0/3397084237 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb88019a0a0 con 0x7fb8801000f0 2024-03-21T10:52:57.730 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:57.734+0000 7fb886200640 1 -- 172.21.15.153:0/3397084237 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb88004e040 con 0x7fb8801000f0 2024-03-21T10:52:57.731 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:57.735+0000 7fb87d600640 1 -- 172.21.15.153:0/3397084237 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fb874003920 con 0x7fb8801000f0 2024-03-21T10:52:57.732 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:57.736+0000 7fb87d600640 1 --2- 172.21.15.153:0/3397084237 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb86806e3d0 0x7fb868070890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:52:57.732 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:57.736+0000 7fb87d600640 1 -- 172.21.15.153:0/3397084237 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fb8740b0ab0 con 0x7fb8801000f0 2024-03-21T10:52:57.732 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:57.736+0000 7fb87fe00640 1 --2- 172.21.15.153:0/3397084237 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb86806e3d0 0x7fb868070890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:52:57.733 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:57.737+0000 7fb87fe00640 1 --2- 172.21.15.153:0/3397084237 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb86806e3d0 0x7fb868070890 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fb86c006fd0 tx=0x7fb86c008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:52:57.737 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:57.741+0000 7fb87d600640 1 -- 172.21.15.153:0/3397084237 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb8740b2e60 con 0x7fb8801000f0 2024-03-21T10:52:57.801 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:52:57 smithi153 ceph-mon[113733]: pgmap v1235: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:52:57.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:52:57 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/459522013' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:52:57.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:52:57 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/2846394548' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 18, "format": "json"}]: dispatch 2024-03-21T10:52:57.946 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:52:57 smithi174 ceph-mon[88004]: pgmap v1235: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:52:57.946 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:52:57 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/459522013' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T10:52:57.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:52:57 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/2846394548' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 18, "format": "json"}]: dispatch 2024-03-21T10:52:57.986 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:57.989+0000 7fb886200640 1 -- 172.21.15.153:0/3397084237 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 19, "format": "json"} v 0) v1 -- 0x7fb8801083f0 con 0x7fb8801000f0 2024-03-21T10:52:57.987 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:57.991+0000 7fb87d600640 1 -- 172.21.15.153:0/3397084237 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 19, "format": "json"}]=0 dumped fsmap epoch 19 v41) v1 ==== 107+0+4896 (secure 0 0 0) 0x7fb874079b90 con 0x7fb8801000f0 2024-03-21T10:52:57.987 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:52:57.987 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T10:04:27.086576+0000","modified":"2024-03-21T10:07:56.307654+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14484,"mds_1":24273},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14484":{"gid":14484,"name":"cephfs.smithi153.iuyluj","rank":0,"incarnation":13,"state":"up:active","state_seq":6,"addr":"172.21.15.153:6829/866138999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":866138999},{"type":"v1","addr":"172.21.15.153:6829","nonce":866138999}]},"join_fscid":1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14494":{"gid":14494,"name":"cephfs.smithi153.zipckr","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6827/4033622615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":4033622615},{"type":"v1","addr":"172.21.15.153:6827","nonce":4033622615}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24273":{"gid":24273,"name":"cephfs.smithi174.yyimst","rank":1,"incarnation":7,"state":"up:active","state_seq":5,"addr":"172.21.15.174:6825/1776947618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":1776947618},{"type":"v1","addr":"172.21.15.174:6825","nonce":1776947618}]},"join_fscid":1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24285":{"gid":24285,"name":"cephfs.smithi174.nhozag","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":2,"addr":"172.21.15.174:6827/806095873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":806095873},{"type":"v1","addr":"172.21.15.174:6827","nonce":806095873}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-21T10:52:57.988 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 19 2024-03-21T10:52:57.992 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:57.994+0000 7fb886200640 1 -- 172.21.15.153:0/3397084237 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb86806e3d0 msgr2=0x7fb868070890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:52:57.992 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:57.994+0000 7fb886200640 1 --2- 172.21.15.153:0/3397084237 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb86806e3d0 0x7fb868070890 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fb86c006fd0 tx=0x7fb86c008040 comp rx=0 tx=0).stop 2024-03-21T10:52:57.992 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:57.994+0000 7fb886200640 1 -- 172.21.15.153:0/3397084237 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb8801000f0 msgr2=0x7fb880195cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:52:57.992 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:57.994+0000 7fb886200640 1 --2- 172.21.15.153:0/3397084237 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb8801000f0 0x7fb880195cb0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fb874000c00 tx=0x7fb874004270 comp rx=0 tx=0).stop 2024-03-21T10:52:57.992 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:57.995+0000 7fb886200640 1 -- 172.21.15.153:0/3397084237 shutdown_connections 2024-03-21T10:52:57.992 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:57.995+0000 7fb886200640 1 --2- 172.21.15.153:0/3397084237 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fb86806e3d0 0x7fb868070890 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:52:57.992 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:57.995+0000 7fb886200640 1 --2- 172.21.15.153:0/3397084237 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb8801961f0 0x7fb8801995a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:52:57.992 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:57.995+0000 7fb886200640 1 --2- 172.21.15.153:0/3397084237 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fb8801000f0 0x7fb880195cb0 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:52:57.992 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:57.995+0000 7fb886200640 1 -- 172.21.15.153:0/3397084237 >> 172.21.15.153:0/3397084237 conn(0x7fb8800fb970 msgr2=0x7fb8800fd110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:52:57.992 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:57.995+0000 7fb886200640 1 -- 172.21.15.153:0/3397084237 shutdown_connections 2024-03-21T10:52:57.992 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:57.995+0000 7fb886200640 1 -- 172.21.15.153:0/3397084237 wait complete. 2024-03-21T10:52:58.010 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 20 2024-03-21T10:52:58.207 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:58.209+0000 7f735f200640 1 -- 172.21.15.153:0/1654636350 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7358102e50 msgr2=0x7f7358106e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:52:58.207 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:58.209+0000 7f735f200640 1 --2- 172.21.15.153:0/1654636350 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7358102e50 0x7f7358106e90 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f7344009880 tx=0x7f734402bce0 comp rx=0 tx=0).stop 2024-03-21T10:52:58.207 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:58.210+0000 7f735f200640 1 -- 172.21.15.153:0/1654636350 shutdown_connections 2024-03-21T10:52:58.207 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:58.210+0000 7f735f200640 1 --2- 172.21.15.153:0/1654636350 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7358102e50 0x7f7358106e90 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:52:58.207 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:58.210+0000 7f735f200640 1 --2- 172.21.15.153:0/1654636350 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f73581024a0 0x7f7358102880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:52:58.207 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:58.210+0000 7f735f200640 1 -- 172.21.15.153:0/1654636350 >> 172.21.15.153:0/1654636350 conn(0x7f73580fdd00 msgr2=0x7f7358100120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:52:58.207 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:58.210+0000 7f735f200640 1 -- 172.21.15.153:0/1654636350 shutdown_connections 2024-03-21T10:52:58.207 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:58.210+0000 7f735f200640 1 -- 172.21.15.153:0/1654636350 wait complete. 2024-03-21T10:52:58.208 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:58.211+0000 7f735f200640 1 Processor -- start 2024-03-21T10:52:58.208 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:58.211+0000 7f735f200640 1 -- start start 2024-03-21T10:52:58.209 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:58.212+0000 7f735f200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f73581024a0 0x7f7358197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:52:58.209 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:58.212+0000 7f735f200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7358198520 0x7f735819c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:52:58.209 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:58.212+0000 7f735f200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7358198b20 con 0x7f73581024a0 2024-03-21T10:52:58.209 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:58.212+0000 7f735f200640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7358198c90 con 0x7f7358198520 2024-03-21T10:52:58.209 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:58.212+0000 7f7357e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f73581024a0 0x7f7358197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:52:58.210 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:58.212+0000 7f7357e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f73581024a0 0x7f7358197fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:34158/0 (socket says 172.21.15.153:34158) 2024-03-21T10:52:58.210 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:58.212+0000 7f7357e00640 1 -- 172.21.15.153:0/3964583657 learned_addr learned my addr 172.21.15.153:0/3964583657 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:52:58.210 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:58.213+0000 7f7357400640 1 --2- 172.21.15.153:0/3964583657 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7358198520 0x7f735819c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:52:58.210 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:58.213+0000 7f7357e00640 1 -- 172.21.15.153:0/3964583657 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7358198520 msgr2=0x7f735819c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:52:58.210 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:58.213+0000 7f7357e00640 1 --2- 172.21.15.153:0/3964583657 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7358198520 0x7f735819c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:52:58.210 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:58.213+0000 7f7357e00640 1 -- 172.21.15.153:0/3964583657 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7344009560 con 0x7f73581024a0 2024-03-21T10:52:58.210 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:58.213+0000 7f7357e00640 1 --2- 172.21.15.153:0/3964583657 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f73581024a0 0x7f7358197fe0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f734800d720 tx=0x7f734800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:52:58.211 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:58.213+0000 7f7354c00640 1 -- 172.21.15.153:0/3964583657 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f73480051f0 con 0x7f73581024a0 2024-03-21T10:52:58.211 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:58.213+0000 7f7354c00640 1 -- 172.21.15.153:0/3964583657 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7348009ce0 con 0x7f73581024a0 2024-03-21T10:52:58.211 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:58.213+0000 7f735f200640 1 -- 172.21.15.153:0/3964583657 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f735819cf80 con 0x7f73581024a0 2024-03-21T10:52:58.211 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:58.213+0000 7f7354c00640 1 -- 172.21.15.153:0/3964583657 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7348010460 con 0x7f73581024a0 2024-03-21T10:52:58.211 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:58.214+0000 7f735f200640 1 -- 172.21.15.153:0/3964583657 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f735819d4e0 con 0x7f73581024a0 2024-03-21T10:52:58.211 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:58.215+0000 7f735f200640 1 -- 172.21.15.153:0/3964583657 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f735804e040 con 0x7f73581024a0 2024-03-21T10:52:58.212 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:58.216+0000 7f7354c00640 1 -- 172.21.15.153:0/3964583657 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f7348005350 con 0x7f73581024a0 2024-03-21T10:52:58.213 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:58.217+0000 7f7354c00640 1 --2- 172.21.15.153:0/3964583657 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f733c06e3d0 0x7f733c070890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:52:58.214 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:58.217+0000 7f7354c00640 1 -- 172.21.15.153:0/3964583657 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f734808e060 con 0x7f73581024a0 2024-03-21T10:52:58.214 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:58.217+0000 7f7357400640 1 --2- 172.21.15.153:0/3964583657 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f733c06e3d0 0x7f733c070890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:52:58.214 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:58.218+0000 7f7357400640 1 --2- 172.21.15.153:0/3964583657 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f733c06e3d0 0x7f733c070890 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f7344006010 tx=0x7f734402cf90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:52:58.218 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:58.221+0000 7f7354c00640 1 -- 172.21.15.153:0/3964583657 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7348057120 con 0x7f73581024a0 2024-03-21T10:52:58.484 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:58.488+0000 7f735f200640 1 -- 172.21.15.153:0/3964583657 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 20, "format": "json"} v 0) v1 -- 0x7f735819d760 con 0x7f73581024a0 2024-03-21T10:52:58.485 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:58.489+0000 7f7354c00640 1 -- 172.21.15.153:0/3964583657 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 20, "format": "json"}]=0 dumped fsmap epoch 20 v41) v1 ==== 107+0+4895 (secure 0 0 0) 0x7f734805a7e0 con 0x7f73581024a0 2024-03-21T10:52:58.485 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:52:58.485 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T10:04:27.086576+0000","modified":"2024-03-21T10:11:36.506217+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14484,"mds_1":24273},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14484":{"gid":14484,"name":"cephfs.smithi153.iuyluj","rank":0,"incarnation":13,"state":"up:active","state_seq":6,"addr":"172.21.15.153:6829/866138999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":866138999},{"type":"v1","addr":"172.21.15.153:6829","nonce":866138999}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14494":{"gid":14494,"name":"cephfs.smithi153.zipckr","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6827/4033622615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":4033622615},{"type":"v1","addr":"172.21.15.153:6827","nonce":4033622615}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24273":{"gid":24273,"name":"cephfs.smithi174.yyimst","rank":1,"incarnation":7,"state":"up:active","state_seq":5,"addr":"172.21.15.174:6825/1776947618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":1776947618},{"type":"v1","addr":"172.21.15.174:6825","nonce":1776947618}]},"join_fscid":1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24285":{"gid":24285,"name":"cephfs.smithi174.nhozag","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":2,"addr":"172.21.15.174:6827/806095873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":806095873},{"type":"v1","addr":"172.21.15.174:6827","nonce":806095873}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-21T10:52:58.486 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 20 2024-03-21T10:52:58.488 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:58.491+0000 7f735f200640 1 -- 172.21.15.153:0/3964583657 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f733c06e3d0 msgr2=0x7f733c070890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:52:58.488 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:58.491+0000 7f735f200640 1 --2- 172.21.15.153:0/3964583657 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f733c06e3d0 0x7f733c070890 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f7344006010 tx=0x7f734402cf90 comp rx=0 tx=0).stop 2024-03-21T10:52:58.489 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:58.491+0000 7f735f200640 1 -- 172.21.15.153:0/3964583657 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f73581024a0 msgr2=0x7f7358197fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:52:58.489 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:58.491+0000 7f735f200640 1 --2- 172.21.15.153:0/3964583657 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f73581024a0 0x7f7358197fe0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f734800d720 tx=0x7f734800dbe0 comp rx=0 tx=0).stop 2024-03-21T10:52:58.489 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:58.492+0000 7f735f200640 1 -- 172.21.15.153:0/3964583657 shutdown_connections 2024-03-21T10:52:58.489 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:58.492+0000 7f735f200640 1 --2- 172.21.15.153:0/3964583657 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f733c06e3d0 0x7f733c070890 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:52:58.489 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:58.492+0000 7f735f200640 1 --2- 172.21.15.153:0/3964583657 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7358198520 0x7f735819c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:52:58.489 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:58.492+0000 7f735f200640 1 --2- 172.21.15.153:0/3964583657 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f73581024a0 0x7f7358197fe0 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:52:58.489 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:58.492+0000 7f735f200640 1 -- 172.21.15.153:0/3964583657 >> 172.21.15.153:0/3964583657 conn(0x7f73580fdd00 msgr2=0x7f73580ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:52:58.489 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:58.492+0000 7f735f200640 1 -- 172.21.15.153:0/3964583657 shutdown_connections 2024-03-21T10:52:58.489 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:58.492+0000 7f735f200640 1 -- 172.21.15.153:0/3964583657 wait complete. 2024-03-21T10:52:58.506 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 21 2024-03-21T10:52:58.700 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:58.703+0000 7f715f200640 1 -- 172.21.15.153:0/736944476 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f71601019a0 msgr2=0x7f7160109e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:52:58.701 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:58.703+0000 7f715f200640 1 --2- 172.21.15.153:0/736944476 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f71601019a0 0x7f7160109e80 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f714c009880 tx=0x7f714c02bd70 comp rx=0 tx=0).stop 2024-03-21T10:52:58.701 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:58.703+0000 7f715f200640 1 -- 172.21.15.153:0/736944476 shutdown_connections 2024-03-21T10:52:58.701 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:58.703+0000 7f715f200640 1 --2- 172.21.15.153:0/736944476 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f71601019a0 0x7f7160109e80 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:52:58.701 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:58.703+0000 7f715f200640 1 --2- 172.21.15.153:0/736944476 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7160101080 0x7f7160101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:52:58.701 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:58.703+0000 7f715f200640 1 -- 172.21.15.153:0/736944476 >> 172.21.15.153:0/736944476 conn(0x7f71600fa820 msgr2=0x7f71600fcc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:52:58.701 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:58.704+0000 7f715f200640 1 -- 172.21.15.153:0/736944476 shutdown_connections 2024-03-21T10:52:58.701 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:58.704+0000 7f715f200640 1 -- 172.21.15.153:0/736944476 wait complete. 2024-03-21T10:52:58.702 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:58.705+0000 7f715f200640 1 Processor -- start 2024-03-21T10:52:58.703 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:58.705+0000 7f715f200640 1 -- start start 2024-03-21T10:52:58.703 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:58.705+0000 7f715f200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7160101080 0x7f7160197d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:52:58.703 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:58.705+0000 7f715f200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f71601019a0 0x7f71601982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:52:58.703 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:58.705+0000 7f715f200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7160198940 con 0x7f7160101080 2024-03-21T10:52:58.703 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:58.705+0000 7f715f200640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f716019c710 con 0x7f71601019a0 2024-03-21T10:52:58.703 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:58.705+0000 7f715de00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7160101080 0x7f7160197d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:52:58.703 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:58.705+0000 7f715de00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7160101080 0x7f7160197d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:34168/0 (socket says 172.21.15.153:34168) 2024-03-21T10:52:58.703 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:58.705+0000 7f715de00640 1 -- 172.21.15.153:0/1967463078 learned_addr learned my addr 172.21.15.153:0/1967463078 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:52:58.703 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:58.706+0000 7f715d400640 1 --2- 172.21.15.153:0/1967463078 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f71601019a0 0x7f71601982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:52:58.703 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:58.706+0000 7f715de00640 1 -- 172.21.15.153:0/1967463078 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f71601019a0 msgr2=0x7f71601982b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:52:58.703 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:58.706+0000 7f715de00640 1 --2- 172.21.15.153:0/1967463078 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f71601019a0 0x7f71601982b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:52:58.703 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:58.706+0000 7f715de00640 1 -- 172.21.15.153:0/1967463078 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f714c009560 con 0x7f7160101080 2024-03-21T10:52:58.703 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:58.706+0000 7f715de00640 1 --2- 172.21.15.153:0/1967463078 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7160101080 0x7f7160197d70 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f715000d6f0 tx=0x7f715000dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:52:58.704 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:58.706+0000 7f7156a00640 1 -- 172.21.15.153:0/1967463078 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7150009a30 con 0x7f7160101080 2024-03-21T10:52:58.704 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:58.707+0000 7f7156a00640 1 -- 172.21.15.153:0/1967463078 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f71500054e0 con 0x7f7160101080 2024-03-21T10:52:58.704 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:58.707+0000 7f7156a00640 1 -- 172.21.15.153:0/1967463078 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7150010460 con 0x7f7160101080 2024-03-21T10:52:58.705 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:58.707+0000 7f715f200640 1 -- 172.21.15.153:0/1967463078 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f716019c9f0 con 0x7f7160101080 2024-03-21T10:52:58.705 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:58.707+0000 7f715f200640 1 -- 172.21.15.153:0/1967463078 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f716019cf50 con 0x7f7160101080 2024-03-21T10:52:58.705 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:58.708+0000 7f715f200640 1 -- 172.21.15.153:0/1967463078 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f716004e040 con 0x7f7160101080 2024-03-21T10:52:58.705 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:58.709+0000 7f7156a00640 1 -- 172.21.15.153:0/1967463078 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f7150009b90 con 0x7f7160101080 2024-03-21T10:52:58.707 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:58.711+0000 7f7156a00640 1 --2- 172.21.15.153:0/1967463078 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f714406e3d0 0x7f7144070890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:52:58.707 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:58.711+0000 7f7156a00640 1 -- 172.21.15.153:0/1967463078 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f715008dff0 con 0x7f7160101080 2024-03-21T10:52:58.707 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:58.711+0000 7f715d400640 1 --2- 172.21.15.153:0/1967463078 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f714406e3d0 0x7f7144070890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:52:58.708 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:58.712+0000 7f715d400640 1 --2- 172.21.15.153:0/1967463078 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f714406e3d0 0x7f7144070890 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f7160199380 tx=0x7f714c000bc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:52:58.712 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:58.715+0000 7f7156a00640 1 -- 172.21.15.153:0/1967463078 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f71500570f0 con 0x7f7160101080 2024-03-21T10:52:58.801 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:52:58 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/3397084237' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 19, "format": "json"}]: dispatch 2024-03-21T10:52:58.801 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:52:58 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/3964583657' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 20, "format": "json"}]: dispatch 2024-03-21T10:52:58.946 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:52:58 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/3397084237' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 19, "format": "json"}]: dispatch 2024-03-21T10:52:58.946 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:52:58 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/3964583657' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 20, "format": "json"}]: dispatch 2024-03-21T10:52:59.058 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:59.061+0000 7f715f200640 1 -- 172.21.15.153:0/1967463078 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 21, "format": "json"} v 0) v1 -- 0x7f7160109950 con 0x7f7160101080 2024-03-21T10:52:59.059 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:59.062+0000 7f7156a00640 1 -- 172.21.15.153:0/1967463078 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 21, "format": "json"}]=0 dumped fsmap epoch 21 v41) v1 ==== 107+0+4894 (secure 0 0 0) 0x7f715005a7b0 con 0x7f7160101080 2024-03-21T10:52:59.059 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:52:59.059 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T10:04:27.086576+0000","modified":"2024-03-21T10:13:01.455594+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14484,"mds_1":24273},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14484":{"gid":14484,"name":"cephfs.smithi153.iuyluj","rank":0,"incarnation":13,"state":"up:active","state_seq":6,"addr":"172.21.15.153:6829/866138999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":866138999},{"type":"v1","addr":"172.21.15.153:6829","nonce":866138999}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14494":{"gid":14494,"name":"cephfs.smithi153.zipckr","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.153:6827/4033622615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":4033622615},{"type":"v1","addr":"172.21.15.153:6827","nonce":4033622615}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24273":{"gid":24273,"name":"cephfs.smithi174.yyimst","rank":1,"incarnation":7,"state":"up:active","state_seq":5,"addr":"172.21.15.174:6825/1776947618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":1776947618},{"type":"v1","addr":"172.21.15.174:6825","nonce":1776947618}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24285":{"gid":24285,"name":"cephfs.smithi174.nhozag","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":2,"addr":"172.21.15.174:6827/806095873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":806095873},{"type":"v1","addr":"172.21.15.174:6827","nonce":806095873}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-21T10:52:59.059 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 21 2024-03-21T10:52:59.062 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:59.065+0000 7f715f200640 1 -- 172.21.15.153:0/1967463078 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f714406e3d0 msgr2=0x7f7144070890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:52:59.062 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:59.065+0000 7f715f200640 1 --2- 172.21.15.153:0/1967463078 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f714406e3d0 0x7f7144070890 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f7160199380 tx=0x7f714c000bc0 comp rx=0 tx=0).stop 2024-03-21T10:52:59.062 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:59.065+0000 7f715f200640 1 -- 172.21.15.153:0/1967463078 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7160101080 msgr2=0x7f7160197d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:52:59.062 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:59.065+0000 7f715f200640 1 --2- 172.21.15.153:0/1967463078 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7160101080 0x7f7160197d70 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f715000d6f0 tx=0x7f715000dbb0 comp rx=0 tx=0).stop 2024-03-21T10:52:59.062 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:59.065+0000 7f715f200640 1 -- 172.21.15.153:0/1967463078 shutdown_connections 2024-03-21T10:52:59.062 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:59.065+0000 7f715f200640 1 --2- 172.21.15.153:0/1967463078 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f714406e3d0 0x7f7144070890 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:52:59.062 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:59.065+0000 7f715f200640 1 --2- 172.21.15.153:0/1967463078 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f71601019a0 0x7f71601982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:52:59.062 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:59.065+0000 7f715f200640 1 --2- 172.21.15.153:0/1967463078 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7160101080 0x7f7160197d70 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:52:59.062 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:59.065+0000 7f715f200640 1 -- 172.21.15.153:0/1967463078 >> 172.21.15.153:0/1967463078 conn(0x7f71600fa820 msgr2=0x7f71600ff490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:52:59.062 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:59.065+0000 7f715f200640 1 -- 172.21.15.153:0/1967463078 shutdown_connections 2024-03-21T10:52:59.062 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:59.065+0000 7f715f200640 1 -- 172.21.15.153:0/1967463078 wait complete. 2024-03-21T10:52:59.080 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 22 2024-03-21T10:52:59.278 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:59.281+0000 7ffa3b200640 1 -- 172.21.15.153:0/926045752 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffa3c102c80 msgr2=0x7ffa3c106cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:52:59.278 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:59.281+0000 7ffa3b200640 1 --2- 172.21.15.153:0/926045752 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffa3c102c80 0x7ffa3c106cc0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7ffa28009880 tx=0x7ffa2802bce0 comp rx=0 tx=0).stop 2024-03-21T10:52:59.278 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:59.281+0000 7ffa3b200640 1 -- 172.21.15.153:0/926045752 shutdown_connections 2024-03-21T10:52:59.278 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:59.281+0000 7ffa3b200640 1 --2- 172.21.15.153:0/926045752 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffa3c102c80 0x7ffa3c106cc0 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:52:59.279 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:59.281+0000 7ffa3b200640 1 --2- 172.21.15.153:0/926045752 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffa3c1022d0 0x7ffa3c1026b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:52:59.279 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:59.281+0000 7ffa3b200640 1 -- 172.21.15.153:0/926045752 >> 172.21.15.153:0/926045752 conn(0x7ffa3c0fdb50 msgr2=0x7ffa3c0fff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:52:59.279 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:59.281+0000 7ffa3b200640 1 -- 172.21.15.153:0/926045752 shutdown_connections 2024-03-21T10:52:59.279 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:59.282+0000 7ffa3b200640 1 -- 172.21.15.153:0/926045752 wait complete. 2024-03-21T10:52:59.280 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:59.282+0000 7ffa3b200640 1 Processor -- start 2024-03-21T10:52:59.280 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:59.283+0000 7ffa3b200640 1 -- start start 2024-03-21T10:52:59.280 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:59.283+0000 7ffa3b200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffa3c1022d0 0x7ffa3c197d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:52:59.280 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:59.283+0000 7ffa3b200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffa3c1982c0 0x7ffa3c19c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:52:59.280 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:59.283+0000 7ffa3b200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa3c1988c0 con 0x7ffa3c1022d0 2024-03-21T10:52:59.280 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:59.283+0000 7ffa3b200640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa3c198a30 con 0x7ffa3c1982c0 2024-03-21T10:52:59.281 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:59.283+0000 7ffa39e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffa3c1022d0 0x7ffa3c197d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:52:59.281 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:59.283+0000 7ffa39e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffa3c1022d0 0x7ffa3c197d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:34188/0 (socket says 172.21.15.153:34188) 2024-03-21T10:52:59.281 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:59.283+0000 7ffa39e00640 1 -- 172.21.15.153:0/2735185297 learned_addr learned my addr 172.21.15.153:0/2735185297 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:52:59.281 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:59.284+0000 7ffa39400640 1 --2- 172.21.15.153:0/2735185297 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffa3c1982c0 0x7ffa3c19c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:52:59.281 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:59.284+0000 7ffa39e00640 1 -- 172.21.15.153:0/2735185297 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffa3c1982c0 msgr2=0x7ffa3c19c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:52:59.281 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:59.284+0000 7ffa39e00640 1 --2- 172.21.15.153:0/2735185297 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffa3c1982c0 0x7ffa3c19c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:52:59.281 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:59.284+0000 7ffa39e00640 1 -- 172.21.15.153:0/2735185297 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa28009560 con 0x7ffa3c1022d0 2024-03-21T10:52:59.281 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:59.284+0000 7ffa39e00640 1 --2- 172.21.15.153:0/2735185297 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffa3c1022d0 0x7ffa3c197d80 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c00d720 tx=0x7ffa2c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:52:59.282 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:59.284+0000 7ffa32a00640 1 -- 172.21.15.153:0/2735185297 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffa2c0051f0 con 0x7ffa3c1022d0 2024-03-21T10:52:59.282 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:59.284+0000 7ffa32a00640 1 -- 172.21.15.153:0/2735185297 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ffa2c009ce0 con 0x7ffa3c1022d0 2024-03-21T10:52:59.282 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:59.284+0000 7ffa3b200640 1 -- 172.21.15.153:0/2735185297 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa3c19cd20 con 0x7ffa3c1022d0 2024-03-21T10:52:59.282 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:59.284+0000 7ffa32a00640 1 -- 172.21.15.153:0/2735185297 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffa2c010460 con 0x7ffa3c1022d0 2024-03-21T10:52:59.282 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:59.284+0000 7ffa3b200640 1 -- 172.21.15.153:0/2735185297 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa3c19d280 con 0x7ffa3c1022d0 2024-03-21T10:52:59.282 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:59.286+0000 7ffa3b200640 1 -- 172.21.15.153:0/2735185297 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa3c19d550 con 0x7ffa3c1022d0 2024-03-21T10:52:59.284 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:59.288+0000 7ffa32a00640 1 -- 172.21.15.153:0/2735185297 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7ffa2c005350 con 0x7ffa3c1022d0 2024-03-21T10:52:59.286 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:59.289+0000 7ffa32a00640 1 --2- 172.21.15.153:0/2735185297 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ffa2006e420 0x7ffa200708e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:52:59.286 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:59.289+0000 7ffa32a00640 1 -- 172.21.15.153:0/2735185297 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7ffa2c08e070 con 0x7ffa3c1022d0 2024-03-21T10:52:59.286 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:59.290+0000 7ffa39400640 1 --2- 172.21.15.153:0/2735185297 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ffa2006e420 0x7ffa200708e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:52:59.287 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:59.291+0000 7ffa39400640 1 --2- 172.21.15.153:0/2735185297 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ffa2006e420 0x7ffa200708e0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7ffa28036040 tx=0x7ffa28005fd0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:52:59.290 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:59.294+0000 7ffa32a00640 1 -- 172.21.15.153:0/2735185297 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ffa2c057110 con 0x7ffa3c1022d0 2024-03-21T10:52:59.576 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:59.580+0000 7ffa3b200640 1 -- 172.21.15.153:0/2735185297 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 22, "format": "json"} v 0) v1 -- 0x7ffa3c1990d0 con 0x7ffa3c1022d0 2024-03-21T10:52:59.577 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:59.581+0000 7ffa32a00640 1 -- 172.21.15.153:0/2735185297 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 22, "format": "json"}]=0 dumped fsmap epoch 22 v41) v1 ==== 107+0+3334 (secure 0 0 0) 0x7ffa2c020d20 con 0x7ffa3c1022d0 2024-03-21T10:52:59.578 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:52:59.578 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T10:04:27.086576+0000","modified":"2024-03-21T10:19:04.696423+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14484,"mds_1":24273},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14484":{"gid":14484,"name":"cephfs.smithi153.iuyluj","rank":0,"incarnation":13,"state":"up:active","state_seq":6,"addr":"172.21.15.153:6829/866138999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":866138999},{"type":"v1","addr":"172.21.15.153:6829","nonce":866138999}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24273":{"gid":24273,"name":"cephfs.smithi174.yyimst","rank":1,"incarnation":7,"state":"up:active","state_seq":5,"addr":"172.21.15.174:6825/1776947618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":1776947618},{"type":"v1","addr":"172.21.15.174:6825","nonce":1776947618}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":14484,"qdb_cluster":[14484,24273]},"id":1}]} 2024-03-21T10:52:59.578 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 22 2024-03-21T10:52:59.581 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:59.584+0000 7ffa3b200640 1 -- 172.21.15.153:0/2735185297 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ffa2006e420 msgr2=0x7ffa200708e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:52:59.581 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:59.584+0000 7ffa3b200640 1 --2- 172.21.15.153:0/2735185297 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ffa2006e420 0x7ffa200708e0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7ffa28036040 tx=0x7ffa28005fd0 comp rx=0 tx=0).stop 2024-03-21T10:52:59.581 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:59.584+0000 7ffa3b200640 1 -- 172.21.15.153:0/2735185297 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffa3c1022d0 msgr2=0x7ffa3c197d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:52:59.581 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:59.584+0000 7ffa3b200640 1 --2- 172.21.15.153:0/2735185297 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffa3c1022d0 0x7ffa3c197d80 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c00d720 tx=0x7ffa2c00dbe0 comp rx=0 tx=0).stop 2024-03-21T10:52:59.581 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:59.584+0000 7ffa3b200640 1 -- 172.21.15.153:0/2735185297 shutdown_connections 2024-03-21T10:52:59.581 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:59.584+0000 7ffa3b200640 1 --2- 172.21.15.153:0/2735185297 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7ffa2006e420 0x7ffa200708e0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:52:59.581 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:59.584+0000 7ffa3b200640 1 --2- 172.21.15.153:0/2735185297 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffa3c1982c0 0x7ffa3c19c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:52:59.581 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:59.584+0000 7ffa3b200640 1 --2- 172.21.15.153:0/2735185297 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7ffa3c1022d0 0x7ffa3c197d80 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:52:59.581 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:59.584+0000 7ffa3b200640 1 -- 172.21.15.153:0/2735185297 >> 172.21.15.153:0/2735185297 conn(0x7ffa3c0fdb50 msgr2=0x7ffa3c0ff2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:52:59.581 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:59.584+0000 7ffa3b200640 1 -- 172.21.15.153:0/2735185297 shutdown_connections 2024-03-21T10:52:59.582 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:59.584+0000 7ffa3b200640 1 -- 172.21.15.153:0/2735185297 wait complete. 2024-03-21T10:52:59.600 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 22 2024-03-21T10:52:59.601 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 23 2024-03-21T10:52:59.799 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:59.800+0000 7f8ce9c00640 1 -- 172.21.15.153:0/2547010201 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8ce41022d0 msgr2=0x7f8ce41026b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:52:59.799 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:59.800+0000 7f8ce9c00640 1 --2- 172.21.15.153:0/2547010201 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8ce41022d0 0x7f8ce41026b0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f8cd4009880 tx=0x7f8cd402bcf0 comp rx=0 tx=0).stop 2024-03-21T10:52:59.799 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:59.800+0000 7f8ce9c00640 1 -- 172.21.15.153:0/2547010201 shutdown_connections 2024-03-21T10:52:59.799 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:59.800+0000 7f8ce9c00640 1 --2- 172.21.15.153:0/2547010201 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8ce4102c80 0x7f8ce4106cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:52:59.799 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:59.800+0000 7f8ce9c00640 1 --2- 172.21.15.153:0/2547010201 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8ce41022d0 0x7f8ce41026b0 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:52:59.799 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:59.800+0000 7f8ce9c00640 1 -- 172.21.15.153:0/2547010201 >> 172.21.15.153:0/2547010201 conn(0x7f8ce40fdb50 msgr2=0x7f8ce40fff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:52:59.799 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:59.801+0000 7f8ce9c00640 1 -- 172.21.15.153:0/2547010201 shutdown_connections 2024-03-21T10:52:59.799 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:59.801+0000 7f8ce9c00640 1 -- 172.21.15.153:0/2547010201 wait complete. 2024-03-21T10:52:59.799 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:59.802+0000 7f8ce9c00640 1 Processor -- start 2024-03-21T10:52:59.799 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:59.802+0000 7f8ce9c00640 1 -- start start 2024-03-21T10:52:59.799 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:59.802+0000 7f8ce9c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8ce41022d0 0x7f8ce4197e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:52:59.799 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:59.802+0000 7f8ce9c00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8ce4198360 0x7f8ce419c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:52:59.799 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:59.802+0000 7f8ce9c00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ce4198960 con 0x7f8ce41022d0 2024-03-21T10:52:59.800 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:59.802+0000 7f8ce9c00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ce4198ad0 con 0x7f8ce4198360 2024-03-21T10:52:59.800 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:59.802+0000 7f8ce3e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8ce41022d0 0x7f8ce4197e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:52:59.800 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:59.802+0000 7f8ce3e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8ce41022d0 0x7f8ce4197e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:34208/0 (socket says 172.21.15.153:34208) 2024-03-21T10:52:59.800 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:59.802+0000 7f8ce3e00640 1 -- 172.21.15.153:0/106151614 learned_addr learned my addr 172.21.15.153:0/106151614 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:52:59.801 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:59.803+0000 7f8ce3e00640 1 -- 172.21.15.153:0/106151614 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8ce4198360 msgr2=0x7f8ce419c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:52:59.801 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:59.803+0000 7f8ce3e00640 1 --2- 172.21.15.153:0/106151614 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8ce4198360 0x7f8ce419c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:52:59.801 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:59.803+0000 7f8ce3e00640 1 -- 172.21.15.153:0/106151614 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cd4009560 con 0x7f8ce41022d0 2024-03-21T10:52:59.801 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:59.803+0000 7f8ce3e00640 1 --2- 172.21.15.153:0/106151614 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8ce41022d0 0x7f8ce4197e20 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f8cd4009430 tx=0x7f8cd4004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:52:59.801 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:59.804+0000 7f8ce0c00640 1 -- 172.21.15.153:0/106151614 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8cd40035f0 con 0x7f8ce41022d0 2024-03-21T10:52:59.801 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:59.804+0000 7f8ce0c00640 1 -- 172.21.15.153:0/106151614 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8cd4003c10 con 0x7f8ce41022d0 2024-03-21T10:52:59.801 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:59.804+0000 7f8ce9c00640 1 -- 172.21.15.153:0/106151614 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ce419cdc0 con 0x7f8ce41022d0 2024-03-21T10:52:59.801 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:59.804+0000 7f8ce0c00640 1 -- 172.21.15.153:0/106151614 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8cd4034440 con 0x7f8ce41022d0 2024-03-21T10:52:59.801 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:59.804+0000 7f8ce9c00640 1 -- 172.21.15.153:0/106151614 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ce419d320 con 0x7f8ce41022d0 2024-03-21T10:52:59.802 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:59.806+0000 7f8ce9c00640 1 -- 172.21.15.153:0/106151614 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ce404e890 con 0x7f8ce41022d0 2024-03-21T10:52:59.803 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:59.807+0000 7f8ce0c00640 1 -- 172.21.15.153:0/106151614 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f8cd4005060 con 0x7f8ce41022d0 2024-03-21T10:52:59.805 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:59.809+0000 7f8ce0c00640 1 --2- 172.21.15.153:0/106151614 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f8cc806e470 0x7f8cc8070930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:52:59.806 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:59.809+0000 7f8ce0c00640 1 -- 172.21.15.153:0/106151614 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f8cd40b08e0 con 0x7f8ce41022d0 2024-03-21T10:52:59.806 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:59.809+0000 7f8ce3400640 1 --2- 172.21.15.153:0/106151614 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f8cc806e470 0x7f8cc8070930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:52:59.806 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:59.810+0000 7f8ce3400640 1 --2- 172.21.15.153:0/106151614 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f8cc806e470 0x7f8cc8070930 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f8ce4073fc0 tx=0x7f8cd0006c90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:52:59.809 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:52:59.813+0000 7f8ce0c00640 1 -- 172.21.15.153:0/106151614 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f8cd4079980 con 0x7f8ce41022d0 2024-03-21T10:52:59.946 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:52:59 smithi174 ceph-mon[88004]: pgmap v1236: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:52:59.946 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:52:59 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/1967463078' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 21, "format": "json"}]: dispatch 2024-03-21T10:52:59.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:52:59 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/2735185297' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 22, "format": "json"}]: dispatch 2024-03-21T10:53:00.051 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:52:59 smithi153 ceph-mon[113733]: pgmap v1236: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:53:00.051 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:52:59 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/1967463078' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 21, "format": "json"}]: dispatch 2024-03-21T10:53:00.051 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:52:59 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/2735185297' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 22, "format": "json"}]: dispatch 2024-03-21T10:53:00.097 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:00.101+0000 7f8ce9c00640 1 -- 172.21.15.153:0/106151614 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 23, "format": "json"} v 0) v1 -- 0x7f8ce4102c80 con 0x7f8ce41022d0 2024-03-21T10:53:00.098 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:00.102+0000 7f8ce0c00640 1 -- 172.21.15.153:0/106151614 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 23, "format": "json"}]=0 dumped fsmap epoch 23 v41) v1 ==== 107+0+3301 (secure 0 0 0) 0x7f8cd4041d00 con 0x7f8ce41022d0 2024-03-21T10:53:00.098 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:53:00.098 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34430,"name":"cephfs.smithi153.zipckr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6827/2010306761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":2010306761},{"type":"v1","addr":"172.21.15.153:6827","nonce":2010306761}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23},{"gid":44269,"name":"cephfs.smithi174.nhozag","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6827/3967223227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":3967223227},{"type":"v1","addr":"172.21.15.174:6827","nonce":3967223227}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":23,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T10:04:27.086576+0000","modified":"2024-03-21T10:19:05.022092+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":94,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-21T10:53:00.098 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 23 2024-03-21T10:53:00.101 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:00.105+0000 7f8ce9c00640 1 -- 172.21.15.153:0/106151614 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f8cc806e470 msgr2=0x7f8cc8070930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:53:00.102 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:00.105+0000 7f8ce9c00640 1 --2- 172.21.15.153:0/106151614 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f8cc806e470 0x7f8cc8070930 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f8ce4073fc0 tx=0x7f8cd0006c90 comp rx=0 tx=0).stop 2024-03-21T10:53:00.102 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:00.105+0000 7f8ce9c00640 1 -- 172.21.15.153:0/106151614 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8ce41022d0 msgr2=0x7f8ce4197e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:53:00.102 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:00.105+0000 7f8ce9c00640 1 --2- 172.21.15.153:0/106151614 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8ce41022d0 0x7f8ce4197e20 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f8cd4009430 tx=0x7f8cd4004040 comp rx=0 tx=0).stop 2024-03-21T10:53:00.102 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:00.105+0000 7f8ce9c00640 1 -- 172.21.15.153:0/106151614 shutdown_connections 2024-03-21T10:53:00.102 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:00.105+0000 7f8ce9c00640 1 --2- 172.21.15.153:0/106151614 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f8cc806e470 0x7f8cc8070930 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:00.102 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:00.105+0000 7f8ce9c00640 1 --2- 172.21.15.153:0/106151614 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8ce4198360 0x7f8ce419c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:00.102 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:00.105+0000 7f8ce9c00640 1 --2- 172.21.15.153:0/106151614 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f8ce41022d0 0x7f8ce4197e20 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:00.102 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:00.105+0000 7f8ce9c00640 1 -- 172.21.15.153:0/106151614 >> 172.21.15.153:0/106151614 conn(0x7f8ce40fdb50 msgr2=0x7f8ce40ff310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:53:00.102 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:00.105+0000 7f8ce9c00640 1 -- 172.21.15.153:0/106151614 shutdown_connections 2024-03-21T10:53:00.102 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:00.105+0000 7f8ce9c00640 1 -- 172.21.15.153:0/106151614 wait complete. 2024-03-21T10:53:00.119 DEBUG:tasks.fs:max_mds not reduced in epoch 23 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-03-21T10:53:00.120 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 23 2024-03-21T10:53:00.120 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 24 2024-03-21T10:53:00.326 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:00.330+0000 7f775b000640 1 -- 172.21.15.153:0/2534158574 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f77540fea00 msgr2=0x7f77540fede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:53:00.327 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:00.330+0000 7f775b000640 1 --2- 172.21.15.153:0/2534158574 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f77540fea00 0x7f77540fede0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f7744009880 tx=0x7f774402bcf0 comp rx=0 tx=0).stop 2024-03-21T10:53:00.327 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:00.330+0000 7f775b000640 1 -- 172.21.15.153:0/2534158574 shutdown_connections 2024-03-21T10:53:00.327 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:00.330+0000 7f775b000640 1 --2- 172.21.15.153:0/2534158574 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f77540ff3b0 0x7f7754104cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:00.327 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:00.330+0000 7f775b000640 1 --2- 172.21.15.153:0/2534158574 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f77540fea00 0x7f77540fede0 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:00.327 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:00.330+0000 7f775b000640 1 -- 172.21.15.153:0/2534158574 >> 172.21.15.153:0/2534158574 conn(0x7f77540752c0 msgr2=0x7f77540756d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:53:00.327 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:00.330+0000 7f775b000640 1 -- 172.21.15.153:0/2534158574 shutdown_connections 2024-03-21T10:53:00.327 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:00.330+0000 7f775b000640 1 -- 172.21.15.153:0/2534158574 wait complete. 2024-03-21T10:53:00.328 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:00.331+0000 7f775b000640 1 Processor -- start 2024-03-21T10:53:00.328 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:00.331+0000 7f775b000640 1 -- start start 2024-03-21T10:53:00.328 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:00.332+0000 7f775b000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f77540fea00 0x7f77540ff970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:53:00.328 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:00.332+0000 7f775b000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f77540ffeb0 0x7f7754100310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:53:00.329 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:00.332+0000 7f775b000640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7754101900 con 0x7f77540ffeb0 2024-03-21T10:53:00.329 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:00.332+0000 7f775b000640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77541a2020 con 0x7f77540fea00 2024-03-21T10:53:00.329 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:00.332+0000 7f7753400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f77540ffeb0 0x7f7754100310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:53:00.329 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:00.332+0000 7f7753400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f77540ffeb0 0x7f7754100310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:34216/0 (socket says 172.21.15.153:34216) 2024-03-21T10:53:00.329 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:00.332+0000 7f7753400640 1 -- 172.21.15.153:0/621407275 learned_addr learned my addr 172.21.15.153:0/621407275 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:53:00.330 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:00.332+0000 7f7753400640 1 -- 172.21.15.153:0/621407275 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f77540fea00 msgr2=0x7f77540ff970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:53:00.330 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:00.332+0000 7f7753e00640 1 --2- 172.21.15.153:0/621407275 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f77540fea00 0x7f77540ff970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:53:00.330 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:00.332+0000 7f7753400640 1 --2- 172.21.15.153:0/621407275 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f77540fea00 0x7f77540ff970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:00.330 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:00.332+0000 7f7753400640 1 -- 172.21.15.153:0/621407275 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7744009560 con 0x7f77540ffeb0 2024-03-21T10:53:00.330 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:00.333+0000 7f7753400640 1 --2- 172.21.15.153:0/621407275 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f77540ffeb0 0x7f7754100310 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f774000c920 tx=0x7f774000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:53:00.330 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:00.333+0000 7f7750c00640 1 -- 172.21.15.153:0/621407275 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7740004410 con 0x7f77540ffeb0 2024-03-21T10:53:00.330 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:00.333+0000 7f7750c00640 1 -- 172.21.15.153:0/621407275 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7740007c60 con 0x7f77540ffeb0 2024-03-21T10:53:00.330 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:00.333+0000 7f7750c00640 1 -- 172.21.15.153:0/621407275 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f774000f450 con 0x7f77540ffeb0 2024-03-21T10:53:00.330 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:00.333+0000 7f775b000640 1 -- 172.21.15.153:0/621407275 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77541a2220 con 0x7f77540ffeb0 2024-03-21T10:53:00.330 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:00.333+0000 7f775b000640 1 -- 172.21.15.153:0/621407275 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77541a2780 con 0x7f77540ffeb0 2024-03-21T10:53:00.331 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:00.334+0000 7f775b000640 1 -- 172.21.15.153:0/621407275 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f775404e040 con 0x7f77540ffeb0 2024-03-21T10:53:00.333 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:00.337+0000 7f7750c00640 1 -- 172.21.15.153:0/621407275 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f77400077a0 con 0x7f77540ffeb0 2024-03-21T10:53:00.334 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:00.338+0000 7f7750c00640 1 --2- 172.21.15.153:0/621407275 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f773806e3d0 0x7f7738070890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:53:00.335 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:00.338+0000 7f7750c00640 1 -- 172.21.15.153:0/621407275 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f774008e4a0 con 0x7f77540ffeb0 2024-03-21T10:53:00.335 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:00.339+0000 7f7753e00640 1 --2- 172.21.15.153:0/621407275 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f773806e3d0 0x7f7738070890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:53:00.336 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:00.340+0000 7f7753e00640 1 --2- 172.21.15.153:0/621407275 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f773806e3d0 0x7f7738070890 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f774402c8b0 tx=0x7f7744005fd0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:53:00.339 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:00.343+0000 7f7750c00640 1 -- 172.21.15.153:0/621407275 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7740057540 con 0x7f77540ffeb0 2024-03-21T10:53:00.582 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:00.585+0000 7f775b000640 1 -- 172.21.15.153:0/621407275 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 24, "format": "json"} v 0) v1 -- 0x7f77541085d0 con 0x7f77540ffeb0 2024-03-21T10:53:00.583 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:00.586+0000 7f7750c00640 1 -- 172.21.15.153:0/621407275 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 24, "format": "json"}]=0 dumped fsmap epoch 24 v41) v1 ==== 107+0+4859 (secure 0 0 0) 0x7f774005ac00 con 0x7f77540ffeb0 2024-03-21T10:53:00.583 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:53:00.583 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34430,"name":"cephfs.smithi153.zipckr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6827/2010306761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":2010306761},{"type":"v1","addr":"172.21.15.153:6827","nonce":2010306761}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23},{"gid":34438,"name":"cephfs.smithi153.iuyluj","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6829/147840850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":147840850},{"type":"v1","addr":"172.21.15.153:6829","nonce":147840850}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24},{"gid":44269,"name":"cephfs.smithi174.nhozag","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6827/3967223227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":3967223227},{"type":"v1","addr":"172.21.15.174:6827","nonce":3967223227}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23},{"gid":44277,"name":"cephfs.smithi174.yyimst","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6825/787559196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":787559196},{"type":"v1","addr":"172.21.15.174:6825","nonce":787559196}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":23,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T10:04:27.086576+0000","modified":"2024-03-21T10:19:05.022092+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":94,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-21T10:53:00.583 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 24 2024-03-21T10:53:00.585 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:00.589+0000 7f775b000640 1 -- 172.21.15.153:0/621407275 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f773806e3d0 msgr2=0x7f7738070890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:53:00.586 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:00.589+0000 7f775b000640 1 --2- 172.21.15.153:0/621407275 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f773806e3d0 0x7f7738070890 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f774402c8b0 tx=0x7f7744005fd0 comp rx=0 tx=0).stop 2024-03-21T10:53:00.586 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:00.589+0000 7f775b000640 1 -- 172.21.15.153:0/621407275 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f77540ffeb0 msgr2=0x7f7754100310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:53:00.586 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:00.589+0000 7f775b000640 1 --2- 172.21.15.153:0/621407275 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f77540ffeb0 0x7f7754100310 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f774000c920 tx=0x7f774000cde0 comp rx=0 tx=0).stop 2024-03-21T10:53:00.586 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:00.589+0000 7f775b000640 1 -- 172.21.15.153:0/621407275 shutdown_connections 2024-03-21T10:53:00.586 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:00.589+0000 7f775b000640 1 --2- 172.21.15.153:0/621407275 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f773806e3d0 0x7f7738070890 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:00.586 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:00.589+0000 7f775b000640 1 --2- 172.21.15.153:0/621407275 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f77540ffeb0 0x7f7754100310 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:00.586 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:00.589+0000 7f775b000640 1 --2- 172.21.15.153:0/621407275 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f77540fea00 0x7f77540ff970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:00.586 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:00.589+0000 7f775b000640 1 -- 172.21.15.153:0/621407275 >> 172.21.15.153:0/621407275 conn(0x7f77540752c0 msgr2=0x7f77540fd320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:53:00.586 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:00.589+0000 7f775b000640 1 -- 172.21.15.153:0/621407275 shutdown_connections 2024-03-21T10:53:00.586 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:00.589+0000 7f775b000640 1 -- 172.21.15.153:0/621407275 wait complete. 2024-03-21T10:53:00.603 DEBUG:tasks.fs:max_mds not reduced in epoch 24 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-03-21T10:53:00.603 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 24 2024-03-21T10:53:00.603 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 25 2024-03-21T10:53:00.808 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:00.811+0000 7f022fe00640 1 -- 172.21.15.153:0/1244381178 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0228101af0 msgr2=0x7f0228109fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:53:00.808 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:00.811+0000 7f022fe00640 1 --2- 172.21.15.153:0/1244381178 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0228101af0 0x7f0228109fd0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f0214009880 tx=0x7f021402bd70 comp rx=0 tx=0).stop 2024-03-21T10:53:00.808 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:00.811+0000 7f022fe00640 1 -- 172.21.15.153:0/1244381178 shutdown_connections 2024-03-21T10:53:00.808 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:00.811+0000 7f022fe00640 1 --2- 172.21.15.153:0/1244381178 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0228101af0 0x7f0228109fd0 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:00.808 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:00.811+0000 7f022fe00640 1 --2- 172.21.15.153:0/1244381178 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f02281011d0 0x7f02281015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:00.808 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:00.811+0000 7f022fe00640 1 -- 172.21.15.153:0/1244381178 >> 172.21.15.153:0/1244381178 conn(0x7f02280fa9d0 msgr2=0x7f02280fcdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:53:00.808 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:00.811+0000 7f022fe00640 1 -- 172.21.15.153:0/1244381178 shutdown_connections 2024-03-21T10:53:00.809 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:00.811+0000 7f022fe00640 1 -- 172.21.15.153:0/1244381178 wait complete. 2024-03-21T10:53:00.810 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:00.812+0000 7f022fe00640 1 Processor -- start 2024-03-21T10:53:00.810 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:00.812+0000 7f022fe00640 1 -- start start 2024-03-21T10:53:00.810 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:00.813+0000 7f022fe00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f02281011d0 0x7f0228197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:53:00.810 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:00.813+0000 7f022fe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0228101af0 0x7f02281983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:53:00.810 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:00.813+0000 7f022fe00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0228198a50 con 0x7f0228101af0 2024-03-21T10:53:00.810 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:00.813+0000 7f022fe00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f022819c820 con 0x7f02281011d0 2024-03-21T10:53:00.810 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:00.813+0000 7f022ca00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0228101af0 0x7f02281983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:53:00.810 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:00.813+0000 7f022ca00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0228101af0 0x7f02281983c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:34220/0 (socket says 172.21.15.153:34220) 2024-03-21T10:53:00.811 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:00.813+0000 7f022ca00640 1 -- 172.21.15.153:0/2535146699 learned_addr learned my addr 172.21.15.153:0/2535146699 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:53:00.811 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:00.814+0000 7f022d400640 1 --2- 172.21.15.153:0/2535146699 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f02281011d0 0x7f0228197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:53:00.811 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:00.814+0000 7f022ca00640 1 -- 172.21.15.153:0/2535146699 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f02281011d0 msgr2=0x7f0228197e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:53:00.811 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:00.814+0000 7f022ca00640 1 --2- 172.21.15.153:0/2535146699 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f02281011d0 0x7f0228197e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:00.811 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:00.814+0000 7f022ca00640 1 -- 172.21.15.153:0/2535146699 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0214009560 con 0x7f0228101af0 2024-03-21T10:53:00.812 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:00.814+0000 7f022ca00640 1 --2- 172.21.15.153:0/2535146699 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0228101af0 0x7f02281983c0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f0214036040 tx=0x7f021402c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:53:00.812 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:00.814+0000 7f0226000640 1 -- 172.21.15.153:0/2535146699 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f02140035f0 con 0x7f0228101af0 2024-03-21T10:53:00.812 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:00.814+0000 7f0226000640 1 -- 172.21.15.153:0/2535146699 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0214003c10 con 0x7f0228101af0 2024-03-21T10:53:00.812 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:00.814+0000 7f022fe00640 1 -- 172.21.15.153:0/2535146699 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f022819caa0 con 0x7f0228101af0 2024-03-21T10:53:00.812 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:00.814+0000 7f0226000640 1 -- 172.21.15.153:0/2535146699 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0214005600 con 0x7f0228101af0 2024-03-21T10:53:00.812 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:00.815+0000 7f022fe00640 1 -- 172.21.15.153:0/2535146699 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f022819d000 con 0x7f0228101af0 2024-03-21T10:53:00.813 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:00.816+0000 7f022fe00640 1 -- 172.21.15.153:0/2535146699 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f022804e040 con 0x7f0228101af0 2024-03-21T10:53:00.814 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:00.818+0000 7f0226000640 1 -- 172.21.15.153:0/2535146699 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f0214004dd0 con 0x7f0228101af0 2024-03-21T10:53:00.816 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:00.820+0000 7f0226000640 1 --2- 172.21.15.153:0/2535146699 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f021006e490 0x7f0210070950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:53:00.816 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:00.820+0000 7f0226000640 1 -- 172.21.15.153:0/2535146699 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f02140b1020 con 0x7f0228101af0 2024-03-21T10:53:00.816 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:00.820+0000 7f022d400640 1 --2- 172.21.15.153:0/2535146699 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f021006e490 0x7f0210070950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:53:00.817 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:00.821+0000 7f022d400640 1 --2- 172.21.15.153:0/2535146699 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f021006e490 0x7f0210070950 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f021c005fd0 tx=0x7f021c005850 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:53:00.821 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:00.824+0000 7f0226000640 1 -- 172.21.15.153:0/2535146699 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f02140e5740 con 0x7f0228101af0 2024-03-21T10:53:00.946 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:53:00 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/106151614' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 23, "format": "json"}]: dispatch 2024-03-21T10:53:00.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:53:00 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/621407275' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 24, "format": "json"}]: dispatch 2024-03-21T10:53:01.051 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:53:00 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/106151614' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 23, "format": "json"}]: dispatch 2024-03-21T10:53:01.051 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:53:00 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/621407275' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 24, "format": "json"}]: dispatch 2024-03-21T10:53:01.082 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:01.086+0000 7f022fe00640 1 -- 172.21.15.153:0/2535146699 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 25, "format": "json"} v 0) v1 -- 0x7f0228109a40 con 0x7f0228101af0 2024-03-21T10:53:01.084 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:01.087+0000 7f0226000640 1 -- 172.21.15.153:0/2535146699 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 25, "format": "json"}]=0 dumped fsmap epoch 25 v41) v1 ==== 107+0+4077 (secure 0 0 0) 0x7f0214043090 con 0x7f0228101af0 2024-03-21T10:53:01.084 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:53:01.084 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34438,"name":"cephfs.smithi153.iuyluj","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6829/147840850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":147840850},{"type":"v1","addr":"172.21.15.153:6829","nonce":147840850}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24},{"gid":44269,"name":"cephfs.smithi174.nhozag","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6827/3967223227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":3967223227},{"type":"v1","addr":"172.21.15.174:6827","nonce":3967223227}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23},{"gid":44277,"name":"cephfs.smithi174.yyimst","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6825/787559196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":787559196},{"type":"v1","addr":"172.21.15.174:6825","nonce":787559196}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":23,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T10:04:27.086576+0000","modified":"2024-03-21T10:19:05.022092+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":94,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-21T10:53:01.084 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 25 2024-03-21T10:53:01.087 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:01.091+0000 7f022fe00640 1 -- 172.21.15.153:0/2535146699 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f021006e490 msgr2=0x7f0210070950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:53:01.088 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:01.091+0000 7f022fe00640 1 --2- 172.21.15.153:0/2535146699 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f021006e490 0x7f0210070950 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f021c005fd0 tx=0x7f021c005850 comp rx=0 tx=0).stop 2024-03-21T10:53:01.088 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:01.091+0000 7f022fe00640 1 -- 172.21.15.153:0/2535146699 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0228101af0 msgr2=0x7f02281983c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:53:01.088 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:01.091+0000 7f022fe00640 1 --2- 172.21.15.153:0/2535146699 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0228101af0 0x7f02281983c0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f0214036040 tx=0x7f021402c7e0 comp rx=0 tx=0).stop 2024-03-21T10:53:01.088 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:01.091+0000 7f022fe00640 1 -- 172.21.15.153:0/2535146699 shutdown_connections 2024-03-21T10:53:01.088 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:01.091+0000 7f022fe00640 1 --2- 172.21.15.153:0/2535146699 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f021006e490 0x7f0210070950 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:01.088 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:01.091+0000 7f022fe00640 1 --2- 172.21.15.153:0/2535146699 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0228101af0 0x7f02281983c0 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:01.088 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:01.091+0000 7f022fe00640 1 --2- 172.21.15.153:0/2535146699 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f02281011d0 0x7f0228197e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:01.088 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:01.091+0000 7f022fe00640 1 -- 172.21.15.153:0/2535146699 >> 172.21.15.153:0/2535146699 conn(0x7f02280fa9d0 msgr2=0x7f0228106c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:53:01.088 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:01.091+0000 7f022fe00640 1 -- 172.21.15.153:0/2535146699 shutdown_connections 2024-03-21T10:53:01.088 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:01.091+0000 7f022fe00640 1 -- 172.21.15.153:0/2535146699 wait complete. 2024-03-21T10:53:01.105 DEBUG:tasks.fs:max_mds not reduced in epoch 25 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-03-21T10:53:01.106 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 25 2024-03-21T10:53:01.106 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 26 2024-03-21T10:53:01.326 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:01.329+0000 7f7413e00640 1 -- 172.21.15.153:0/3197800329 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f74141000f0 msgr2=0x7f74141004d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:53:01.326 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:01.329+0000 7f7413e00640 1 --2- 172.21.15.153:0/3197800329 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f74141000f0 0x7f74141004d0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f7404009880 tx=0x7f740402bce0 comp rx=0 tx=0).stop 2024-03-21T10:53:01.326 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:01.329+0000 7f7413e00640 1 -- 172.21.15.153:0/3197800329 shutdown_connections 2024-03-21T10:53:01.326 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:01.329+0000 7f7413e00640 1 --2- 172.21.15.153:0/3197800329 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7414100aa0 0x7f7414104ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:01.326 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:01.329+0000 7f7413e00640 1 --2- 172.21.15.153:0/3197800329 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f74141000f0 0x7f74141004d0 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:01.326 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:01.329+0000 7f7413e00640 1 -- 172.21.15.153:0/3197800329 >> 172.21.15.153:0/3197800329 conn(0x7f74140fb970 msgr2=0x7f74140fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:53:01.326 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:01.329+0000 7f7413e00640 1 -- 172.21.15.153:0/3197800329 shutdown_connections 2024-03-21T10:53:01.326 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:01.329+0000 7f7413e00640 1 -- 172.21.15.153:0/3197800329 wait complete. 2024-03-21T10:53:01.328 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:01.330+0000 7f7413e00640 1 Processor -- start 2024-03-21T10:53:01.328 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:01.330+0000 7f7413e00640 1 -- start start 2024-03-21T10:53:01.328 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:01.330+0000 7f7413e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f74141000f0 0x7f7414197e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:53:01.328 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:01.330+0000 7f7413e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7414198380 0x7f741419c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:53:01.328 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:01.330+0000 7f7413e00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7414198980 con 0x7f7414198380 2024-03-21T10:53:01.328 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:01.330+0000 7f7413e00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7414198af0 con 0x7f74141000f0 2024-03-21T10:53:01.328 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:01.331+0000 7f7412000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7414198380 0x7f741419c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:53:01.329 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:01.331+0000 7f7412000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7414198380 0x7f741419c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:34250/0 (socket says 172.21.15.153:34250) 2024-03-21T10:53:01.329 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:01.331+0000 7f7412000640 1 -- 172.21.15.153:0/710717766 learned_addr learned my addr 172.21.15.153:0/710717766 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:53:01.329 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:01.331+0000 7f7412a00640 1 --2- 172.21.15.153:0/710717766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f74141000f0 0x7f7414197e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:53:01.329 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:01.331+0000 7f7412000640 1 -- 172.21.15.153:0/710717766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f74141000f0 msgr2=0x7f7414197e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:53:01.329 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:01.331+0000 7f7412000640 1 --2- 172.21.15.153:0/710717766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f74141000f0 0x7f7414197e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:01.329 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:01.331+0000 7f7412000640 1 -- 172.21.15.153:0/710717766 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7404009560 con 0x7f7414198380 2024-03-21T10:53:01.329 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:01.331+0000 7f7412000640 1 --2- 172.21.15.153:0/710717766 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7414198380 0x7f741419c7e0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f740000d6f0 tx=0x7f740000dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:53:01.329 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:01.332+0000 7f740b400640 1 -- 172.21.15.153:0/710717766 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f74000051f0 con 0x7f7414198380 2024-03-21T10:53:01.329 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:01.332+0000 7f740b400640 1 -- 172.21.15.153:0/710717766 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7400009ce0 con 0x7f7414198380 2024-03-21T10:53:01.329 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:01.332+0000 7f7413e00640 1 -- 172.21.15.153:0/710717766 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f741419cde0 con 0x7f7414198380 2024-03-21T10:53:01.329 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:01.332+0000 7f740b400640 1 -- 172.21.15.153:0/710717766 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7400004e80 con 0x7f7414198380 2024-03-21T10:53:01.329 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:01.332+0000 7f7413e00640 1 -- 172.21.15.153:0/710717766 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f741419d340 con 0x7f7414198380 2024-03-21T10:53:01.330 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:01.333+0000 7f7413e00640 1 -- 172.21.15.153:0/710717766 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f741419d610 con 0x7f7414198380 2024-03-21T10:53:01.330 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:01.334+0000 7f740b400640 1 -- 172.21.15.153:0/710717766 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f74000046c0 con 0x7f7414198380 2024-03-21T10:53:01.331 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:01.335+0000 7f740b400640 1 --2- 172.21.15.153:0/710717766 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f73f806e380 0x7f73f8070840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:53:01.331 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:01.335+0000 7f740b400640 1 -- 172.21.15.153:0/710717766 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f740008dfa0 con 0x7f7414198380 2024-03-21T10:53:01.332 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:01.335+0000 7f7412a00640 1 --2- 172.21.15.153:0/710717766 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f73f806e380 0x7f73f8070840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:53:01.332 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:01.336+0000 7f7412a00640 1 --2- 172.21.15.153:0/710717766 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f73f806e380 0x7f73f8070840 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f7404036040 tx=0x7f7404005fd0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:53:01.337 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:01.340+0000 7f740b400640 1 -- 172.21.15.153:0/710717766 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7400057060 con 0x7f7414198380 2024-03-21T10:53:01.578 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:01.582+0000 7f7413e00640 1 -- 172.21.15.153:0/710717766 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 26, "format": "json"} v 0) v1 -- 0x7f741404e090 con 0x7f7414198380 2024-03-21T10:53:01.579 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:01.583+0000 7f740b400640 1 -- 172.21.15.153:0/710717766 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 26, "format": "json"}]=0 dumped fsmap epoch 26 v41) v1 ==== 107+0+4893 (secure 0 0 0) 0x7f740005a720 con 0x7f7414198380 2024-03-21T10:53:01.579 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:53:01.579 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34438,"name":"cephfs.smithi153.iuyluj","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6829/147840850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":147840850},{"type":"v1","addr":"172.21.15.153:6829","nonce":147840850}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24},{"gid":34442,"name":"cephfs.smithi153.zipckr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6827/1103846295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":1103846295},{"type":"v1","addr":"172.21.15.153:6827","nonce":1103846295}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26},{"gid":44269,"name":"cephfs.smithi174.nhozag","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6827/3967223227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":3967223227},{"type":"v1","addr":"172.21.15.174:6827","nonce":3967223227}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23},{"gid":44277,"name":"cephfs.smithi174.yyimst","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6825/787559196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":787559196},{"type":"v1","addr":"172.21.15.174:6825","nonce":787559196}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":23,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T10:04:27.086576+0000","modified":"2024-03-21T10:19:05.022092+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":94,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-21T10:53:01.580 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 26 2024-03-21T10:53:01.583 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:01.586+0000 7f7413e00640 1 -- 172.21.15.153:0/710717766 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f73f806e380 msgr2=0x7f73f8070840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:53:01.583 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:01.586+0000 7f7413e00640 1 --2- 172.21.15.153:0/710717766 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f73f806e380 0x7f73f8070840 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f7404036040 tx=0x7f7404005fd0 comp rx=0 tx=0).stop 2024-03-21T10:53:01.583 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:01.586+0000 7f7413e00640 1 -- 172.21.15.153:0/710717766 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7414198380 msgr2=0x7f741419c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:53:01.583 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:01.586+0000 7f7413e00640 1 --2- 172.21.15.153:0/710717766 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7414198380 0x7f741419c7e0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f740000d6f0 tx=0x7f740000dbb0 comp rx=0 tx=0).stop 2024-03-21T10:53:01.584 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:01.587+0000 7f7413e00640 1 -- 172.21.15.153:0/710717766 shutdown_connections 2024-03-21T10:53:01.584 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:01.587+0000 7f7413e00640 1 --2- 172.21.15.153:0/710717766 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f73f806e380 0x7f73f8070840 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:01.584 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:01.587+0000 7f7413e00640 1 --2- 172.21.15.153:0/710717766 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f7414198380 0x7f741419c7e0 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:01.584 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:01.587+0000 7f7413e00640 1 --2- 172.21.15.153:0/710717766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f74141000f0 0x7f7414197e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:01.584 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:01.587+0000 7f7413e00640 1 -- 172.21.15.153:0/710717766 >> 172.21.15.153:0/710717766 conn(0x7f74140fb970 msgr2=0x7f74140fd030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:53:01.584 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:01.587+0000 7f7413e00640 1 -- 172.21.15.153:0/710717766 shutdown_connections 2024-03-21T10:53:01.584 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:01.587+0000 7f7413e00640 1 -- 172.21.15.153:0/710717766 wait complete. 2024-03-21T10:53:01.604 DEBUG:tasks.fs:max_mds not reduced in epoch 26 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-03-21T10:53:01.605 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 26 2024-03-21T10:53:01.605 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 27 2024-03-21T10:53:01.821 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:01.824+0000 7f6838800640 1 -- 172.21.15.153:0/3808325040 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6830101b20 msgr2=0x7f683010a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:53:01.821 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:01.824+0000 7f6838800640 1 --2- 172.21.15.153:0/3808325040 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6830101b20 0x7f683010a000 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f6820009880 tx=0x7f682002bd70 comp rx=0 tx=0).stop 2024-03-21T10:53:01.821 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:01.824+0000 7f6838800640 1 -- 172.21.15.153:0/3808325040 shutdown_connections 2024-03-21T10:53:01.821 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:01.824+0000 7f6838800640 1 --2- 172.21.15.153:0/3808325040 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6830101b20 0x7f683010a000 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:01.822 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:01.824+0000 7f6838800640 1 --2- 172.21.15.153:0/3808325040 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6830101200 0x7f68301015e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:01.822 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:01.824+0000 7f6838800640 1 -- 172.21.15.153:0/3808325040 >> 172.21.15.153:0/3808325040 conn(0x7f68300faa20 msgr2=0x7f68300fce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:53:01.822 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:01.824+0000 7f6838800640 1 -- 172.21.15.153:0/3808325040 shutdown_connections 2024-03-21T10:53:01.822 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:01.824+0000 7f6838800640 1 -- 172.21.15.153:0/3808325040 wait complete. 2024-03-21T10:53:01.824 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:01.825+0000 7f6838800640 1 Processor -- start 2024-03-21T10:53:01.824 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:01.826+0000 7f6838800640 1 -- start start 2024-03-21T10:53:01.824 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:01.826+0000 7f6838800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6830101200 0x7f6830198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:53:01.824 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:01.826+0000 7f6838800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6830101b20 0x7f6830198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:53:01.824 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:01.826+0000 7f6838800640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6830198bd0 con 0x7f6830101200 2024-03-21T10:53:01.824 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:01.826+0000 7f6838800640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f683019c9a0 con 0x7f6830101b20 2024-03-21T10:53:01.824 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:01.826+0000 7f6835e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6830101200 0x7f6830198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:53:01.824 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:01.826+0000 7f6835e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6830101200 0x7f6830198000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:34266/0 (socket says 172.21.15.153:34266) 2024-03-21T10:53:01.824 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:01.826+0000 7f6835e00640 1 -- 172.21.15.153:0/3925106685 learned_addr learned my addr 172.21.15.153:0/3925106685 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:53:01.824 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:01.827+0000 7f6835e00640 1 -- 172.21.15.153:0/3925106685 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6830101b20 msgr2=0x7f6830198540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:53:01.824 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:01.827+0000 7f6835e00640 1 --2- 172.21.15.153:0/3925106685 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6830101b20 0x7f6830198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:01.824 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:01.827+0000 7f6835e00640 1 -- 172.21.15.153:0/3925106685 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6820009560 con 0x7f6830101200 2024-03-21T10:53:01.824 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:01.827+0000 7f6835e00640 1 --2- 172.21.15.153:0/3925106685 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6830101200 0x7f6830198000 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f682400d6f0 tx=0x7f682400dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:53:01.825 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:01.827+0000 7f682aa00640 1 -- 172.21.15.153:0/3925106685 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6824009a30 con 0x7f6830101200 2024-03-21T10:53:01.825 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:01.827+0000 7f682aa00640 1 -- 172.21.15.153:0/3925106685 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f68240054e0 con 0x7f6830101200 2024-03-21T10:53:01.825 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:01.827+0000 7f6838800640 1 -- 172.21.15.153:0/3925106685 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f683019cc80 con 0x7f6830101200 2024-03-21T10:53:01.825 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:01.827+0000 7f682aa00640 1 -- 172.21.15.153:0/3925106685 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6824010460 con 0x7f6830101200 2024-03-21T10:53:01.825 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:01.827+0000 7f6838800640 1 -- 172.21.15.153:0/3925106685 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f683019d1e0 con 0x7f6830101200 2024-03-21T10:53:01.825 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:01.829+0000 7f6838800640 1 -- 172.21.15.153:0/3925106685 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f683004e040 con 0x7f6830101200 2024-03-21T10:53:01.826 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:01.830+0000 7f682aa00640 1 -- 172.21.15.153:0/3925106685 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f6824009b90 con 0x7f6830101200 2024-03-21T10:53:01.827 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:01.831+0000 7f682aa00640 1 --2- 172.21.15.153:0/3925106685 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f681806e3d0 0x7f6818070890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:53:01.827 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:01.831+0000 7f682aa00640 1 -- 172.21.15.153:0/3925106685 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f682408dff0 con 0x7f6830101200 2024-03-21T10:53:01.828 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:01.831+0000 7f6835400640 1 --2- 172.21.15.153:0/3925106685 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f681806e3d0 0x7f6818070890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:53:01.828 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:01.832+0000 7f6835400640 1 --2- 172.21.15.153:0/3925106685 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f681806e3d0 0x7f6818070890 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f6820036040 tx=0x7f6820000bc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:53:01.834 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:01.838+0000 7f682aa00640 1 -- 172.21.15.153:0/3925106685 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f68240570f0 con 0x7f6830101200 2024-03-21T10:53:01.946 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:53:01 smithi174 ceph-mon[88004]: pgmap v1237: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:53:01.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:53:01 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/2535146699' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 25, "format": "json"}]: dispatch 2024-03-21T10:53:01.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:53:01 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:53:01.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:53:01 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/710717766' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 26, "format": "json"}]: dispatch 2024-03-21T10:53:02.051 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:53:01 smithi153 ceph-mon[113733]: pgmap v1237: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:53:02.051 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:53:01 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/2535146699' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 25, "format": "json"}]: dispatch 2024-03-21T10:53:02.051 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:53:01 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:53:02.051 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:53:01 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/710717766' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 26, "format": "json"}]: dispatch 2024-03-21T10:53:02.078 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:02.082+0000 7f6838800640 1 -- 172.21.15.153:0/3925106685 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 27, "format": "json"} v 0) v1 -- 0x7f6830109ad0 con 0x7f6830101200 2024-03-21T10:53:02.079 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:02.083+0000 7f682aa00640 1 -- 172.21.15.153:0/3925106685 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 27, "format": "json"}]=0 dumped fsmap epoch 27 v41) v1 ==== 107+0+4114 (secure 0 0 0) 0x7f682405a7b0 con 0x7f6830101200 2024-03-21T10:53:02.079 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:53:02.080 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34442,"name":"cephfs.smithi153.zipckr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6827/1103846295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":1103846295},{"type":"v1","addr":"172.21.15.153:6827","nonce":1103846295}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26},{"gid":44269,"name":"cephfs.smithi174.nhozag","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6827/3967223227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":3967223227},{"type":"v1","addr":"172.21.15.174:6827","nonce":3967223227}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23},{"gid":44277,"name":"cephfs.smithi174.yyimst","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6825/787559196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":787559196},{"type":"v1","addr":"172.21.15.174:6825","nonce":787559196}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":23,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T10:04:27.086576+0000","modified":"2024-03-21T10:19:05.022092+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":94,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-21T10:53:02.080 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 27 2024-03-21T10:53:02.082 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:02.086+0000 7f6838800640 1 -- 172.21.15.153:0/3925106685 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f681806e3d0 msgr2=0x7f6818070890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:53:02.083 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:02.086+0000 7f6838800640 1 --2- 172.21.15.153:0/3925106685 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f681806e3d0 0x7f6818070890 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f6820036040 tx=0x7f6820000bc0 comp rx=0 tx=0).stop 2024-03-21T10:53:02.083 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:02.086+0000 7f6838800640 1 -- 172.21.15.153:0/3925106685 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6830101200 msgr2=0x7f6830198000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:53:02.083 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:02.086+0000 7f6838800640 1 --2- 172.21.15.153:0/3925106685 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6830101200 0x7f6830198000 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f682400d6f0 tx=0x7f682400dbb0 comp rx=0 tx=0).stop 2024-03-21T10:53:02.083 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:02.086+0000 7f6838800640 1 -- 172.21.15.153:0/3925106685 shutdown_connections 2024-03-21T10:53:02.083 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:02.086+0000 7f6838800640 1 --2- 172.21.15.153:0/3925106685 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f681806e3d0 0x7f6818070890 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:02.083 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:02.086+0000 7f6838800640 1 --2- 172.21.15.153:0/3925106685 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6830101b20 0x7f6830198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:02.083 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:02.086+0000 7f6838800640 1 --2- 172.21.15.153:0/3925106685 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6830101200 0x7f6830198000 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:02.083 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:02.086+0000 7f6838800640 1 -- 172.21.15.153:0/3925106685 >> 172.21.15.153:0/3925106685 conn(0x7f68300faa20 msgr2=0x7f68300ff5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:53:02.083 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:02.086+0000 7f6838800640 1 -- 172.21.15.153:0/3925106685 shutdown_connections 2024-03-21T10:53:02.083 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:02.086+0000 7f6838800640 1 -- 172.21.15.153:0/3925106685 wait complete. 2024-03-21T10:53:02.101 DEBUG:tasks.fs:max_mds not reduced in epoch 27 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-03-21T10:53:02.101 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 27 2024-03-21T10:53:02.101 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 28 2024-03-21T10:53:02.297 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:02.300+0000 7f18cb200640 1 -- 172.21.15.153:0/3554331850 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f18cc1000c0 msgr2=0x7f18cc1004a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:53:02.297 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:02.300+0000 7f18cb200640 1 --2- 172.21.15.153:0/3554331850 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f18cc1000c0 0x7f18cc1004a0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f18bc009880 tx=0x7f18bc02bcf0 comp rx=0 tx=0).stop 2024-03-21T10:53:02.297 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:02.300+0000 7f18cb200640 1 -- 172.21.15.153:0/3554331850 shutdown_connections 2024-03-21T10:53:02.297 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:02.300+0000 7f18cb200640 1 --2- 172.21.15.153:0/3554331850 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f18cc100a70 0x7f18cc104ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:02.297 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:02.300+0000 7f18cb200640 1 --2- 172.21.15.153:0/3554331850 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f18cc1000c0 0x7f18cc1004a0 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:02.297 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:02.300+0000 7f18cb200640 1 -- 172.21.15.153:0/3554331850 >> 172.21.15.153:0/3554331850 conn(0x7f18cc0fb920 msgr2=0x7f18cc0fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:53:02.297 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:02.300+0000 7f18cb200640 1 -- 172.21.15.153:0/3554331850 shutdown_connections 2024-03-21T10:53:02.297 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:02.300+0000 7f18cb200640 1 -- 172.21.15.153:0/3554331850 wait complete. 2024-03-21T10:53:02.298 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:02.301+0000 7f18cb200640 1 Processor -- start 2024-03-21T10:53:02.298 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:02.301+0000 7f18cb200640 1 -- start start 2024-03-21T10:53:02.299 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:02.302+0000 7f18cb200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f18cc1000c0 0x7f18cc197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:53:02.299 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:02.302+0000 7f18cb200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f18cc198460 0x7f18cc19c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:53:02.299 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:02.302+0000 7f18cb200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18cc198a60 con 0x7f18cc1000c0 2024-03-21T10:53:02.299 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:02.302+0000 7f18cb200640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18cc198bd0 con 0x7f18cc198460 2024-03-21T10:53:02.299 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:02.302+0000 7f18c9e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f18cc1000c0 0x7f18cc197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:53:02.300 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:02.302+0000 7f18c9e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f18cc1000c0 0x7f18cc197f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:34290/0 (socket says 172.21.15.153:34290) 2024-03-21T10:53:02.300 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:02.302+0000 7f18c9e00640 1 -- 172.21.15.153:0/1326357545 learned_addr learned my addr 172.21.15.153:0/1326357545 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:53:02.300 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:02.303+0000 7f18c9e00640 1 -- 172.21.15.153:0/1326357545 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f18cc198460 msgr2=0x7f18cc19c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:53:02.300 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:02.303+0000 7f18c9400640 1 --2- 172.21.15.153:0/1326357545 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f18cc198460 0x7f18cc19c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:53:02.300 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:02.303+0000 7f18c9e00640 1 --2- 172.21.15.153:0/1326357545 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f18cc198460 0x7f18cc19c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:02.300 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:02.303+0000 7f18c9e00640 1 -- 172.21.15.153:0/1326357545 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18bc009560 con 0x7f18cc1000c0 2024-03-21T10:53:02.300 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:02.303+0000 7f18c9400640 1 --2- 172.21.15.153:0/1326357545 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f18cc198460 0x7f18cc19c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:53:02.300 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:02.303+0000 7f18c9e00640 1 --2- 172.21.15.153:0/1326357545 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f18cc1000c0 0x7f18cc197f20 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f18bc009430 tx=0x7f18bc004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:53:02.301 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:02.303+0000 7f18c2a00640 1 -- 172.21.15.153:0/1326357545 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f18bc0035f0 con 0x7f18cc1000c0 2024-03-21T10:53:02.301 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:02.303+0000 7f18c2a00640 1 -- 172.21.15.153:0/1326357545 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f18bc003c10 con 0x7f18cc1000c0 2024-03-21T10:53:02.301 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:02.303+0000 7f18cb200640 1 -- 172.21.15.153:0/1326357545 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18cc19cec0 con 0x7f18cc1000c0 2024-03-21T10:53:02.301 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:02.303+0000 7f18c2a00640 1 -- 172.21.15.153:0/1326357545 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f18bc03e460 con 0x7f18cc1000c0 2024-03-21T10:53:02.301 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:02.304+0000 7f18cb200640 1 -- 172.21.15.153:0/1326357545 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18cc19d420 con 0x7f18cc1000c0 2024-03-21T10:53:02.301 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:02.305+0000 7f18cb200640 1 -- 172.21.15.153:0/1326357545 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18cc04e040 con 0x7f18cc1000c0 2024-03-21T10:53:02.302 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:02.305+0000 7f18c2a00640 1 -- 172.21.15.153:0/1326357545 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f18bc004fd0 con 0x7f18cc1000c0 2024-03-21T10:53:02.303 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:02.306+0000 7f18c2a00640 1 --2- 172.21.15.153:0/1326357545 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f18b006e3d0 0x7f18b0070890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:53:02.303 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:02.306+0000 7f18c2a00640 1 -- 172.21.15.153:0/1326357545 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f18bc0b0e20 con 0x7f18cc1000c0 2024-03-21T10:53:02.303 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:02.307+0000 7f18c9400640 1 --2- 172.21.15.153:0/1326357545 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f18b006e3d0 0x7f18b0070890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:53:02.304 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:02.307+0000 7f18c9400640 1 --2- 172.21.15.153:0/1326357545 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f18b006e3d0 0x7f18b0070890 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f18b8006fd0 tx=0x7f18b8008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:53:02.308 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:02.311+0000 7f18c2a00640 1 -- 172.21.15.153:0/1326357545 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f18bc079ee0 con 0x7f18cc1000c0 2024-03-21T10:53:02.717 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:02.721+0000 7f18cb200640 1 -- 172.21.15.153:0/1326357545 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 28, "format": "json"} v 0) v1 -- 0x7f18cc19d5b0 con 0x7f18cc1000c0 2024-03-21T10:53:02.718 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:02.722+0000 7f18c2a00640 1 -- 172.21.15.153:0/1326357545 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 28, "format": "json"}]=0 dumped fsmap epoch 28 v41) v1 ==== 107+0+4930 (secure 0 0 0) 0x7f18bc07d5a0 con 0x7f18cc1000c0 2024-03-21T10:53:02.718 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:53:02.719 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34442,"name":"cephfs.smithi153.zipckr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6827/1103846295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":1103846295},{"type":"v1","addr":"172.21.15.153:6827","nonce":1103846295}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26},{"gid":34474,"name":"cephfs.smithi153.iuyluj","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6829/1738630832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":1738630832},{"type":"v1","addr":"172.21.15.153:6829","nonce":1738630832}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":44269,"name":"cephfs.smithi174.nhozag","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6827/3967223227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":3967223227},{"type":"v1","addr":"172.21.15.174:6827","nonce":3967223227}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23},{"gid":44277,"name":"cephfs.smithi174.yyimst","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6825/787559196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":787559196},{"type":"v1","addr":"172.21.15.174:6825","nonce":787559196}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":23,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T10:04:27.086576+0000","modified":"2024-03-21T10:19:05.022092+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":94,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-21T10:53:02.719 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 28 2024-03-21T10:53:02.721 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:02.724+0000 7f18cb200640 1 -- 172.21.15.153:0/1326357545 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f18b006e3d0 msgr2=0x7f18b0070890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:53:02.721 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:02.724+0000 7f18cb200640 1 --2- 172.21.15.153:0/1326357545 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f18b006e3d0 0x7f18b0070890 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f18b8006fd0 tx=0x7f18b8008040 comp rx=0 tx=0).stop 2024-03-21T10:53:02.721 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:02.724+0000 7f18cb200640 1 -- 172.21.15.153:0/1326357545 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f18cc1000c0 msgr2=0x7f18cc197f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:53:02.721 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:02.724+0000 7f18cb200640 1 --2- 172.21.15.153:0/1326357545 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f18cc1000c0 0x7f18cc197f20 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f18bc009430 tx=0x7f18bc004040 comp rx=0 tx=0).stop 2024-03-21T10:53:02.722 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:02.725+0000 7f18cb200640 1 -- 172.21.15.153:0/1326357545 shutdown_connections 2024-03-21T10:53:02.722 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:02.725+0000 7f18cb200640 1 --2- 172.21.15.153:0/1326357545 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f18b006e3d0 0x7f18b0070890 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:02.722 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:02.725+0000 7f18cb200640 1 --2- 172.21.15.153:0/1326357545 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f18cc198460 0x7f18cc19c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:02.722 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:02.725+0000 7f18cb200640 1 --2- 172.21.15.153:0/1326357545 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f18cc1000c0 0x7f18cc197f20 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:02.722 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:02.725+0000 7f18cb200640 1 -- 172.21.15.153:0/1326357545 >> 172.21.15.153:0/1326357545 conn(0x7f18cc0fb920 msgr2=0x7f18cc0fd150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:53:02.722 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:02.725+0000 7f18cb200640 1 -- 172.21.15.153:0/1326357545 shutdown_connections 2024-03-21T10:53:02.722 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:02.725+0000 7f18cb200640 1 -- 172.21.15.153:0/1326357545 wait complete. 2024-03-21T10:53:02.740 DEBUG:tasks.fs:max_mds not reduced in epoch 28 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-03-21T10:53:02.741 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 28 2024-03-21T10:53:02.741 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 29 2024-03-21T10:53:02.939 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:02.942+0000 7fbd5f200640 1 -- 172.21.15.153:0/1867354618 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbd580ffb50 msgr2=0x7fbd58103b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:53:02.939 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:02.942+0000 7fbd5f200640 1 --2- 172.21.15.153:0/1867354618 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbd580ffb50 0x7fbd58103b90 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fbd40009880 tx=0x7fbd4002bce0 comp rx=0 tx=0).stop 2024-03-21T10:53:02.939 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:02.942+0000 7fbd5f200640 1 -- 172.21.15.153:0/1867354618 shutdown_connections 2024-03-21T10:53:02.939 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:02.942+0000 7fbd5f200640 1 --2- 172.21.15.153:0/1867354618 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbd580ffb50 0x7fbd58103b90 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:02.940 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:02.942+0000 7fbd5f200640 1 --2- 172.21.15.153:0/1867354618 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbd580ff1a0 0x7fbd580ff580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:02.940 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:02.942+0000 7fbd5f200640 1 -- 172.21.15.153:0/1867354618 >> 172.21.15.153:0/1867354618 conn(0x7fbd580faa20 msgr2=0x7fbd580fce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:53:02.940 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:02.942+0000 7fbd5f200640 1 -- 172.21.15.153:0/1867354618 shutdown_connections 2024-03-21T10:53:02.940 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:02.942+0000 7fbd5f200640 1 -- 172.21.15.153:0/1867354618 wait complete. 2024-03-21T10:53:02.942 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:02.943+0000 7fbd5f200640 1 Processor -- start 2024-03-21T10:53:02.942 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:02.943+0000 7fbd5f200640 1 -- start start 2024-03-21T10:53:02.942 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:02.944+0000 7fbd5f200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbd580ff1a0 0x7fbd58100c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:53:02.942 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:02.944+0000 7fbd5f200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbd58101170 0x7fbd581015d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:53:02.942 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:02.944+0000 7fbd5f200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd581a1f40 con 0x7fbd58101170 2024-03-21T10:53:02.942 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:02.944+0000 7fbd5f200640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd581a2080 con 0x7fbd580ff1a0 2024-03-21T10:53:02.942 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:02.944+0000 7fbd4f400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbd58101170 0x7fbd581015d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:53:02.942 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:02.944+0000 7fbd4f400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbd58101170 0x7fbd581015d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42672/0 (socket says 172.21.15.153:42672) 2024-03-21T10:53:02.942 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:02.944+0000 7fbd4f400640 1 -- 172.21.15.153:0/2641877285 learned_addr learned my addr 172.21.15.153:0/2641877285 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:53:02.943 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:02.944+0000 7fbd4f400640 1 -- 172.21.15.153:0/2641877285 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbd580ff1a0 msgr2=0x7fbd58100c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:53:02.943 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:02.944+0000 7fbd57e00640 1 --2- 172.21.15.153:0/2641877285 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbd580ff1a0 0x7fbd58100c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:53:02.943 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:02.944+0000 7fbd4f400640 1 --2- 172.21.15.153:0/2641877285 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbd580ff1a0 0x7fbd58100c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:02.943 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:02.944+0000 7fbd4f400640 1 -- 172.21.15.153:0/2641877285 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd40009560 con 0x7fbd58101170 2024-03-21T10:53:02.943 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:02.945+0000 7fbd4f400640 1 --2- 172.21.15.153:0/2641877285 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbd58101170 0x7fbd581015d0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fbd40036040 tx=0x7fbd40004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:53:02.943 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:02.945+0000 7fbd55600640 1 -- 172.21.15.153:0/2641877285 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbd400035f0 con 0x7fbd58101170 2024-03-21T10:53:02.943 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:02.945+0000 7fbd55600640 1 -- 172.21.15.153:0/2641877285 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbd40003c10 con 0x7fbd58101170 2024-03-21T10:53:02.943 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:02.945+0000 7fbd55600640 1 -- 172.21.15.153:0/2641877285 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbd40034440 con 0x7fbd58101170 2024-03-21T10:53:02.943 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:02.945+0000 7fbd5f200640 1 -- 172.21.15.153:0/2641877285 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd581a21c0 con 0x7fbd58101170 2024-03-21T10:53:02.943 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:02.945+0000 7fbd5f200640 1 -- 172.21.15.153:0/2641877285 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd581a2670 con 0x7fbd58101170 2024-03-21T10:53:02.944 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:02.947+0000 7fbd5f200640 1 -- 172.21.15.153:0/2641877285 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd5804e040 con 0x7fbd58101170 2024-03-21T10:53:02.944 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:02.947+0000 7fbd55600640 1 -- 172.21.15.153:0/2641877285 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fbd40005060 con 0x7fbd58101170 2024-03-21T10:53:02.945 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:02.949+0000 7fbd55600640 1 --2- 172.21.15.153:0/2641877285 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fbd3c06e3d0 0x7fbd3c070890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:53:02.945 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:02.949+0000 7fbd55600640 1 -- 172.21.15.153:0/2641877285 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fbd400b0a10 con 0x7fbd58101170 2024-03-21T10:53:02.945 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:02.949+0000 7fbd57e00640 1 --2- 172.21.15.153:0/2641877285 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fbd3c06e3d0 0x7fbd3c070890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:53:02.946 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:02.950+0000 7fbd57e00640 1 --2- 172.21.15.153:0/2641877285 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fbd3c06e3d0 0x7fbd3c070890 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fbd48005c40 tx=0x7fbd48008230 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:53:02.946 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:53:02 smithi174 ceph-mon[88004]: pgmap v1238: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:53:02.946 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:53:02 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/3925106685' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 27, "format": "json"}]: dispatch 2024-03-21T10:53:02.950 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:02.954+0000 7fbd55600640 1 -- 172.21.15.153:0/2641877285 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fbd40079a30 con 0x7fbd58101170 2024-03-21T10:53:03.051 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:53:02 smithi153 ceph-mon[113733]: pgmap v1238: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:53:03.051 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:53:02 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/3925106685' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 27, "format": "json"}]: dispatch 2024-03-21T10:53:03.360 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:03.363+0000 7fbd5f200640 1 -- 172.21.15.153:0/2641877285 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 29, "format": "json"} v 0) v1 -- 0x7fbd58102160 con 0x7fbd58101170 2024-03-21T10:53:03.361 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:03.364+0000 7fbd55600640 1 -- 172.21.15.153:0/2641877285 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 29, "format": "json"}]=0 dumped fsmap epoch 29 v41) v1 ==== 107+0+4151 (secure 0 0 0) 0x7fbd400aee60 con 0x7fbd58101170 2024-03-21T10:53:03.361 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:53:03.361 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34442,"name":"cephfs.smithi153.zipckr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6827/1103846295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":1103846295},{"type":"v1","addr":"172.21.15.153:6827","nonce":1103846295}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26},{"gid":34474,"name":"cephfs.smithi153.iuyluj","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6829/1738630832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":1738630832},{"type":"v1","addr":"172.21.15.153:6829","nonce":1738630832}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":44269,"name":"cephfs.smithi174.nhozag","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6827/3967223227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":3967223227},{"type":"v1","addr":"172.21.15.174:6827","nonce":3967223227}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":23,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T10:04:27.086576+0000","modified":"2024-03-21T10:19:05.022092+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":94,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-21T10:53:03.362 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 29 2024-03-21T10:53:03.363 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:03.366+0000 7fbd5f200640 1 -- 172.21.15.153:0/2641877285 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fbd3c06e3d0 msgr2=0x7fbd3c070890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:53:03.364 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:03.366+0000 7fbd5f200640 1 --2- 172.21.15.153:0/2641877285 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fbd3c06e3d0 0x7fbd3c070890 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fbd48005c40 tx=0x7fbd48008230 comp rx=0 tx=0).stop 2024-03-21T10:53:03.364 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:03.366+0000 7fbd5f200640 1 -- 172.21.15.153:0/2641877285 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbd58101170 msgr2=0x7fbd581015d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:53:03.364 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:03.366+0000 7fbd5f200640 1 --2- 172.21.15.153:0/2641877285 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbd58101170 0x7fbd581015d0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fbd40036040 tx=0x7fbd40004040 comp rx=0 tx=0).stop 2024-03-21T10:53:03.364 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:03.367+0000 7fbd5f200640 1 -- 172.21.15.153:0/2641877285 shutdown_connections 2024-03-21T10:53:03.364 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:03.367+0000 7fbd5f200640 1 --2- 172.21.15.153:0/2641877285 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fbd3c06e3d0 0x7fbd3c070890 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:03.364 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:03.367+0000 7fbd5f200640 1 --2- 172.21.15.153:0/2641877285 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbd58101170 0x7fbd581015d0 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:03.364 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:03.367+0000 7fbd5f200640 1 --2- 172.21.15.153:0/2641877285 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbd580ff1a0 0x7fbd58100c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:03.364 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:03.367+0000 7fbd5f200640 1 -- 172.21.15.153:0/2641877285 >> 172.21.15.153:0/2641877285 conn(0x7fbd580faa20 msgr2=0x7fbd58103340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:53:03.364 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:03.367+0000 7fbd5f200640 1 -- 172.21.15.153:0/2641877285 shutdown_connections 2024-03-21T10:53:03.364 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:03.367+0000 7fbd5f200640 1 -- 172.21.15.153:0/2641877285 wait complete. 2024-03-21T10:53:03.380 DEBUG:tasks.fs:max_mds not reduced in epoch 29 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-03-21T10:53:03.380 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 29 2024-03-21T10:53:03.380 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 30 2024-03-21T10:53:03.596 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:03.599+0000 7fce07000640 1 -- 172.21.15.153:0/2308998151 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fce00102c80 msgr2=0x7fce00106cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:53:03.596 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:03.599+0000 7fce07000640 1 --2- 172.21.15.153:0/2308998151 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fce00102c80 0x7fce00106cc0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fcdec009880 tx=0x7fcdec02bce0 comp rx=0 tx=0).stop 2024-03-21T10:53:03.596 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:03.600+0000 7fce07000640 1 -- 172.21.15.153:0/2308998151 shutdown_connections 2024-03-21T10:53:03.597 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:03.600+0000 7fce07000640 1 --2- 172.21.15.153:0/2308998151 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fce00102c80 0x7fce00106cc0 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:03.597 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:03.600+0000 7fce07000640 1 --2- 172.21.15.153:0/2308998151 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fce001022d0 0x7fce001026b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:03.597 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:03.600+0000 7fce07000640 1 -- 172.21.15.153:0/2308998151 >> 172.21.15.153:0/2308998151 conn(0x7fce000fdb30 msgr2=0x7fce000fff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:53:03.597 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:03.600+0000 7fce07000640 1 -- 172.21.15.153:0/2308998151 shutdown_connections 2024-03-21T10:53:03.597 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:03.600+0000 7fce07000640 1 -- 172.21.15.153:0/2308998151 wait complete. 2024-03-21T10:53:03.597 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:03.601+0000 7fce07000640 1 Processor -- start 2024-03-21T10:53:03.598 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:03.601+0000 7fce07000640 1 -- start start 2024-03-21T10:53:03.598 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:03.602+0000 7fce07000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fce001022d0 0x7fce00197d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:53:03.598 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:03.602+0000 7fce07000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fce00198270 0x7fce0019c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:53:03.598 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:03.602+0000 7fce07000640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce00198870 con 0x7fce00198270 2024-03-21T10:53:03.598 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:03.602+0000 7fce07000640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce001989e0 con 0x7fce001022d0 2024-03-21T10:53:03.599 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:03.602+0000 7fce05200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fce00198270 0x7fce0019c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:53:03.599 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:03.602+0000 7fce05200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fce00198270 0x7fce0019c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42680/0 (socket says 172.21.15.153:42680) 2024-03-21T10:53:03.599 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:03.602+0000 7fce05200640 1 -- 172.21.15.153:0/3801914469 learned_addr learned my addr 172.21.15.153:0/3801914469 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:53:03.599 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:03.603+0000 7fce05c00640 1 --2- 172.21.15.153:0/3801914469 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fce001022d0 0x7fce00197d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:53:03.600 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:03.603+0000 7fce05200640 1 -- 172.21.15.153:0/3801914469 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fce001022d0 msgr2=0x7fce00197d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:53:03.600 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:03.603+0000 7fce05200640 1 --2- 172.21.15.153:0/3801914469 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fce001022d0 0x7fce00197d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:03.600 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:03.603+0000 7fce05200640 1 -- 172.21.15.153:0/3801914469 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdec009560 con 0x7fce00198270 2024-03-21T10:53:03.600 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:03.603+0000 7fce05200640 1 --2- 172.21.15.153:0/3801914469 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fce00198270 0x7fce0019c6d0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fcdec006010 tx=0x7fcdec02c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:53:03.601 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:03.604+0000 7fcdfe000640 1 -- 172.21.15.153:0/3801914469 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcdec0035f0 con 0x7fce00198270 2024-03-21T10:53:03.601 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:03.604+0000 7fcdfe000640 1 -- 172.21.15.153:0/3801914469 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcdec003c10 con 0x7fce00198270 2024-03-21T10:53:03.601 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:03.604+0000 7fce07000640 1 -- 172.21.15.153:0/3801914469 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce0019cc70 con 0x7fce00198270 2024-03-21T10:53:03.601 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:03.604+0000 7fcdfe000640 1 -- 172.21.15.153:0/3801914469 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcdec034440 con 0x7fce00198270 2024-03-21T10:53:03.601 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:03.604+0000 7fce07000640 1 -- 172.21.15.153:0/3801914469 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce0019d1d0 con 0x7fce00198270 2024-03-21T10:53:03.602 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:03.605+0000 7fce07000640 1 -- 172.21.15.153:0/3801914469 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce0019d500 con 0x7fce00198270 2024-03-21T10:53:03.602 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:03.606+0000 7fcdfe000640 1 -- 172.21.15.153:0/3801914469 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fcdec004ed0 con 0x7fce00198270 2024-03-21T10:53:03.604 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:03.607+0000 7fcdfe000640 1 --2- 172.21.15.153:0/3801914469 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fcde406e420 0x7fcde40708e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:53:03.604 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:03.608+0000 7fcdfe000640 1 -- 172.21.15.153:0/3801914469 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fcdec0b0970 con 0x7fce00198270 2024-03-21T10:53:03.604 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:03.608+0000 7fce05c00640 1 --2- 172.21.15.153:0/3801914469 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fcde406e420 0x7fcde40708e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:53:03.605 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:03.609+0000 7fce05c00640 1 --2- 172.21.15.153:0/3801914469 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fcde406e420 0x7fcde40708e0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fcdf0005fd0 tx=0x7fcdf0009870 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:53:03.608 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:03.612+0000 7fcdfe000640 1 -- 172.21.15.153:0/3801914469 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fcdec079990 con 0x7fce00198270 2024-03-21T10:53:03.865 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:03.868+0000 7fce07000640 1 -- 172.21.15.153:0/3801914469 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 30, "format": "json"} v 0) v1 -- 0x7fce001990d0 con 0x7fce00198270 2024-03-21T10:53:03.866 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:03.869+0000 7fcdfe000640 1 -- 172.21.15.153:0/3801914469 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 30, "format": "json"}]=0 dumped fsmap epoch 30 v41) v1 ==== 107+0+4967 (secure 0 0 0) 0x7fcdec07d050 con 0x7fce00198270 2024-03-21T10:53:03.866 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:53:03.866 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34442,"name":"cephfs.smithi153.zipckr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6827/1103846295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":1103846295},{"type":"v1","addr":"172.21.15.153:6827","nonce":1103846295}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26},{"gid":34474,"name":"cephfs.smithi153.iuyluj","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6829/1738630832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":1738630832},{"type":"v1","addr":"172.21.15.153:6829","nonce":1738630832}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":44269,"name":"cephfs.smithi174.nhozag","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6827/3967223227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":3967223227},{"type":"v1","addr":"172.21.15.174:6827","nonce":3967223227}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23},{"gid":44313,"name":"cephfs.smithi174.yyimst","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6825/4127786797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":4127786797},{"type":"v1","addr":"172.21.15.174:6825","nonce":4127786797}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":23,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T10:04:27.086576+0000","modified":"2024-03-21T10:19:05.022092+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":94,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-21T10:53:03.867 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 30 2024-03-21T10:53:03.868 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:03.872+0000 7fce07000640 1 -- 172.21.15.153:0/3801914469 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fcde406e420 msgr2=0x7fcde40708e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:53:03.868 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:03.872+0000 7fce07000640 1 --2- 172.21.15.153:0/3801914469 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fcde406e420 0x7fcde40708e0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fcdf0005fd0 tx=0x7fcdf0009870 comp rx=0 tx=0).stop 2024-03-21T10:53:03.869 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:03.872+0000 7fce07000640 1 -- 172.21.15.153:0/3801914469 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fce00198270 msgr2=0x7fce0019c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:53:03.869 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:03.872+0000 7fce07000640 1 --2- 172.21.15.153:0/3801914469 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fce00198270 0x7fce0019c6d0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fcdec006010 tx=0x7fcdec02c7c0 comp rx=0 tx=0).stop 2024-03-21T10:53:03.869 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:03.872+0000 7fce07000640 1 -- 172.21.15.153:0/3801914469 shutdown_connections 2024-03-21T10:53:03.869 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:03.872+0000 7fce07000640 1 --2- 172.21.15.153:0/3801914469 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fcde406e420 0x7fcde40708e0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:03.869 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:03.872+0000 7fce07000640 1 --2- 172.21.15.153:0/3801914469 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fce00198270 0x7fce0019c6d0 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:03.869 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:03.872+0000 7fce07000640 1 --2- 172.21.15.153:0/3801914469 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fce001022d0 0x7fce00197d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:03.869 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:03.872+0000 7fce07000640 1 -- 172.21.15.153:0/3801914469 >> 172.21.15.153:0/3801914469 conn(0x7fce000fdb30 msgr2=0x7fce000ff1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:53:03.869 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:03.872+0000 7fce07000640 1 -- 172.21.15.153:0/3801914469 shutdown_connections 2024-03-21T10:53:03.869 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:03.872+0000 7fce07000640 1 -- 172.21.15.153:0/3801914469 wait complete. 2024-03-21T10:53:03.887 DEBUG:tasks.fs:max_mds not reduced in epoch 30 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-03-21T10:53:03.887 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 30 2024-03-21T10:53:03.887 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 31 2024-03-21T10:53:03.946 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:53:03 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/1326357545' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 28, "format": "json"}]: dispatch 2024-03-21T10:53:03.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:53:03 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/2641877285' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 29, "format": "json"}]: dispatch 2024-03-21T10:53:03.959 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:53:03 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/1326357545' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 28, "format": "json"}]: dispatch 2024-03-21T10:53:03.960 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:53:03 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/2641877285' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 29, "format": "json"}]: dispatch 2024-03-21T10:53:04.110 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:04.112+0000 7f5c61200640 1 -- 172.21.15.153:0/3263770276 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5c5c100aa0 msgr2=0x7f5c5c104ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:53:04.110 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:04.112+0000 7f5c61200640 1 --2- 172.21.15.153:0/3263770276 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5c5c100aa0 0x7f5c5c104ae0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f5c48009880 tx=0x7f5c4802bcf0 comp rx=0 tx=0).stop 2024-03-21T10:53:04.111 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:04.112+0000 7f5c61200640 1 -- 172.21.15.153:0/3263770276 shutdown_connections 2024-03-21T10:53:04.111 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:04.112+0000 7f5c61200640 1 --2- 172.21.15.153:0/3263770276 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5c5c100aa0 0x7f5c5c104ae0 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:04.111 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:04.112+0000 7f5c61200640 1 --2- 172.21.15.153:0/3263770276 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5c5c1000f0 0x7f5c5c1004d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:04.111 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:04.112+0000 7f5c61200640 1 -- 172.21.15.153:0/3263770276 >> 172.21.15.153:0/3263770276 conn(0x7f5c5c0fb970 msgr2=0x7f5c5c0fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:53:04.111 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:04.112+0000 7f5c61200640 1 -- 172.21.15.153:0/3263770276 shutdown_connections 2024-03-21T10:53:04.111 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:04.113+0000 7f5c61200640 1 -- 172.21.15.153:0/3263770276 wait complete. 2024-03-21T10:53:04.111 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:04.113+0000 7f5c61200640 1 Processor -- start 2024-03-21T10:53:04.111 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:04.114+0000 7f5c61200640 1 -- start start 2024-03-21T10:53:04.111 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:04.114+0000 7f5c61200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5c5c1000f0 0x7f5c5c19d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:53:04.111 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:04.114+0000 7f5c61200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5c5c19d6a0 0x7f5c5c19db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:53:04.111 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:04.114+0000 7f5c61200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c5c1a61d0 con 0x7f5c5c19d6a0 2024-03-21T10:53:04.111 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:04.114+0000 7f5c61200640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c5c1a6310 con 0x7f5c5c1000f0 2024-03-21T10:53:04.113 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:04.114+0000 7f5c5aa00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5c5c19d6a0 0x7f5c5c19db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:53:04.113 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:04.114+0000 7f5c5aa00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5c5c19d6a0 0x7f5c5c19db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42708/0 (socket says 172.21.15.153:42708) 2024-03-21T10:53:04.113 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:04.114+0000 7f5c5aa00640 1 -- 172.21.15.153:0/916843650 learned_addr learned my addr 172.21.15.153:0/916843650 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:53:04.113 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:04.115+0000 7f5c5b400640 1 --2- 172.21.15.153:0/916843650 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5c5c1000f0 0x7f5c5c19d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:53:04.113 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:04.115+0000 7f5c5aa00640 1 -- 172.21.15.153:0/916843650 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5c5c1000f0 msgr2=0x7f5c5c19d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:53:04.113 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:04.115+0000 7f5c5aa00640 1 --2- 172.21.15.153:0/916843650 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5c5c1000f0 0x7f5c5c19d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:04.113 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:04.115+0000 7f5c5aa00640 1 -- 172.21.15.153:0/916843650 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c48009560 con 0x7f5c5c19d6a0 2024-03-21T10:53:04.113 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:04.115+0000 7f5c5aa00640 1 --2- 172.21.15.153:0/916843650 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5c5c19d6a0 0x7f5c5c19db00 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f5c48036040 tx=0x7f5c48004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:53:04.113 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:04.115+0000 7f5c53e00640 1 -- 172.21.15.153:0/916843650 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5c480037c0 con 0x7f5c5c19d6a0 2024-03-21T10:53:04.113 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:04.115+0000 7f5c53e00640 1 -- 172.21.15.153:0/916843650 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5c48003de0 con 0x7f5c5c19d6a0 2024-03-21T10:53:04.114 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:04.115+0000 7f5c61200640 1 -- 172.21.15.153:0/916843650 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c5c1a6510 con 0x7f5c5c19d6a0 2024-03-21T10:53:04.114 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:04.115+0000 7f5c53e00640 1 -- 172.21.15.153:0/916843650 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5c4803e460 con 0x7f5c5c19d6a0 2024-03-21T10:53:04.114 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:04.116+0000 7f5c61200640 1 -- 172.21.15.153:0/916843650 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c5c1a6a20 con 0x7f5c5c19d6a0 2024-03-21T10:53:04.114 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:04.117+0000 7f5c61200640 1 -- 172.21.15.153:0/916843650 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c5c04e040 con 0x7f5c5c19d6a0 2024-03-21T10:53:04.115 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:04.118+0000 7f5c53e00640 1 -- 172.21.15.153:0/916843650 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f5c48004fd0 con 0x7f5c5c19d6a0 2024-03-21T10:53:04.116 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:04.119+0000 7f5c53e00640 1 --2- 172.21.15.153:0/916843650 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f5c4006e3d0 0x7f5c40070890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:53:04.116 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:04.119+0000 7f5c53e00640 1 -- 172.21.15.153:0/916843650 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f5c480b0e20 con 0x7f5c5c19d6a0 2024-03-21T10:53:04.116 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:04.120+0000 7f5c5b400640 1 --2- 172.21.15.153:0/916843650 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f5c4006e3d0 0x7f5c40070890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:53:04.117 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:04.120+0000 7f5c5b400640 1 --2- 172.21.15.153:0/916843650 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f5c4006e3d0 0x7f5c40070890 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c009890 tx=0x7f5c4c008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:53:04.125 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:04.129+0000 7f5c53e00640 1 -- 172.21.15.153:0/916843650 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f5c48079ee0 con 0x7f5c5c19d6a0 2024-03-21T10:53:04.367 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:04.370+0000 7f5c61200640 1 -- 172.21.15.153:0/916843650 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 31, "format": "json"} v 0) v1 -- 0x7f5c5c1083f0 con 0x7f5c5c19d6a0 2024-03-21T10:53:04.367 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:04.371+0000 7f5c53e00640 1 -- 172.21.15.153:0/916843650 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 31, "format": "json"}]=0 dumped fsmap epoch 31 v41) v1 ==== 107+0+4185 (secure 0 0 0) 0x7f5c4807d5a0 con 0x7f5c5c19d6a0 2024-03-21T10:53:04.368 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:53:04.368 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34442,"name":"cephfs.smithi153.zipckr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6827/1103846295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":1103846295},{"type":"v1","addr":"172.21.15.153:6827","nonce":1103846295}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26},{"gid":34474,"name":"cephfs.smithi153.iuyluj","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6829/1738630832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":1738630832},{"type":"v1","addr":"172.21.15.153:6829","nonce":1738630832}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":44313,"name":"cephfs.smithi174.yyimst","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6825/4127786797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":4127786797},{"type":"v1","addr":"172.21.15.174:6825","nonce":4127786797}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":23,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T10:04:27.086576+0000","modified":"2024-03-21T10:19:05.022092+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":94,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-21T10:53:04.368 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 31 2024-03-21T10:53:04.371 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:04.375+0000 7f5c61200640 1 -- 172.21.15.153:0/916843650 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f5c4006e3d0 msgr2=0x7f5c40070890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:53:04.371 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:04.375+0000 7f5c61200640 1 --2- 172.21.15.153:0/916843650 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f5c4006e3d0 0x7f5c40070890 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c009890 tx=0x7f5c4c008040 comp rx=0 tx=0).stop 2024-03-21T10:53:04.371 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:04.375+0000 7f5c61200640 1 -- 172.21.15.153:0/916843650 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5c5c19d6a0 msgr2=0x7f5c5c19db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:53:04.372 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:04.375+0000 7f5c61200640 1 --2- 172.21.15.153:0/916843650 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5c5c19d6a0 0x7f5c5c19db00 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f5c48036040 tx=0x7f5c48004270 comp rx=0 tx=0).stop 2024-03-21T10:53:04.372 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:04.375+0000 7f5c61200640 1 -- 172.21.15.153:0/916843650 shutdown_connections 2024-03-21T10:53:04.372 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:04.375+0000 7f5c61200640 1 --2- 172.21.15.153:0/916843650 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f5c4006e3d0 0x7f5c40070890 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:04.372 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:04.375+0000 7f5c61200640 1 --2- 172.21.15.153:0/916843650 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5c5c19d6a0 0x7f5c5c19db00 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:04.372 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:04.375+0000 7f5c61200640 1 --2- 172.21.15.153:0/916843650 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5c5c1000f0 0x7f5c5c19d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:04.372 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:04.375+0000 7f5c61200640 1 -- 172.21.15.153:0/916843650 >> 172.21.15.153:0/916843650 conn(0x7f5c5c0fb970 msgr2=0x7f5c5c0fd070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:53:04.372 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:04.375+0000 7f5c61200640 1 -- 172.21.15.153:0/916843650 shutdown_connections 2024-03-21T10:53:04.372 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:04.375+0000 7f5c61200640 1 -- 172.21.15.153:0/916843650 wait complete. 2024-03-21T10:53:04.390 DEBUG:tasks.fs:max_mds not reduced in epoch 31 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-03-21T10:53:04.390 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 31 2024-03-21T10:53:04.390 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 32 2024-03-21T10:53:04.624 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:04.628+0000 7fe3a4e00640 1 -- 172.21.15.153:0/3651561344 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe3a0100f20 msgr2=0x7fe3a0101300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:53:04.625 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:04.628+0000 7fe3a4e00640 1 --2- 172.21.15.153:0/3651561344 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe3a0100f20 0x7fe3a0101300 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fe390009880 tx=0x7fe39002bce0 comp rx=0 tx=0).stop 2024-03-21T10:53:04.625 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:04.628+0000 7fe3a4e00640 1 -- 172.21.15.153:0/3651561344 shutdown_connections 2024-03-21T10:53:04.625 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:04.628+0000 7fe3a4e00640 1 --2- 172.21.15.153:0/3651561344 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe3a00fe630 0x7fe3a00fea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:04.625 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:04.628+0000 7fe3a4e00640 1 --2- 172.21.15.153:0/3651561344 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe3a0100f20 0x7fe3a0101300 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:04.625 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:04.628+0000 7fe3a4e00640 1 -- 172.21.15.153:0/3651561344 >> 172.21.15.153:0/3651561344 conn(0x7fe3a00fa6d0 msgr2=0x7fe3a00fcac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:53:04.625 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:04.628+0000 7fe3a4e00640 1 -- 172.21.15.153:0/3651561344 shutdown_connections 2024-03-21T10:53:04.625 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:04.628+0000 7fe3a4e00640 1 -- 172.21.15.153:0/3651561344 wait complete. 2024-03-21T10:53:04.628 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:04.630+0000 7fe3a4e00640 1 Processor -- start 2024-03-21T10:53:04.628 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:04.630+0000 7fe3a4e00640 1 -- start start 2024-03-21T10:53:04.628 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:04.630+0000 7fe3a4e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe3a00fe630 0x7fe3a0197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:53:04.628 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:04.631+0000 7fe3a4e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe3a0100f20 0x7fe3a0198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:53:04.629 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:04.631+0000 7fe3a4e00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a0198a20 con 0x7fe3a00fe630 2024-03-21T10:53:04.629 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:04.631+0000 7fe3a4e00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a019c7f0 con 0x7fe3a0100f20 2024-03-21T10:53:04.629 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:04.631+0000 7fe39de00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe3a00fe630 0x7fe3a0197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:53:04.629 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:04.631+0000 7fe39de00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe3a00fe630 0x7fe3a0197e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42728/0 (socket says 172.21.15.153:42728) 2024-03-21T10:53:04.629 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:04.631+0000 7fe39de00640 1 -- 172.21.15.153:0/3486457119 learned_addr learned my addr 172.21.15.153:0/3486457119 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:53:04.629 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:04.631+0000 7fe39de00640 1 -- 172.21.15.153:0/3486457119 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe3a0100f20 msgr2=0x7fe3a0198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:53:04.629 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:04.631+0000 7fe39de00640 1 --2- 172.21.15.153:0/3486457119 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe3a0100f20 0x7fe3a0198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:04.630 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:04.631+0000 7fe39de00640 1 -- 172.21.15.153:0/3486457119 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe390009560 con 0x7fe3a00fe630 2024-03-21T10:53:04.630 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:04.632+0000 7fe39de00640 1 --2- 172.21.15.153:0/3486457119 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe3a00fe630 0x7fe3a0197e50 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fe390009230 tx=0x7fe390004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:53:04.630 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:04.632+0000 7fe396a00640 1 -- 172.21.15.153:0/3486457119 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe3900037c0 con 0x7fe3a00fe630 2024-03-21T10:53:04.630 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:04.632+0000 7fe396a00640 1 -- 172.21.15.153:0/3486457119 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe390005d50 con 0x7fe3a00fe630 2024-03-21T10:53:04.630 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:04.632+0000 7fe3a4e00640 1 -- 172.21.15.153:0/3486457119 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3a019ca70 con 0x7fe3a00fe630 2024-03-21T10:53:04.630 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:04.632+0000 7fe396a00640 1 -- 172.21.15.153:0/3486457119 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe390005660 con 0x7fe3a00fe630 2024-03-21T10:53:04.630 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:04.632+0000 7fe3a4e00640 1 -- 172.21.15.153:0/3486457119 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3a019cfd0 con 0x7fe3a00fe630 2024-03-21T10:53:04.630 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:04.633+0000 7fe3a4e00640 1 -- 172.21.15.153:0/3486457119 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3a004e040 con 0x7fe3a00fe630 2024-03-21T10:53:04.631 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:04.635+0000 7fe396a00640 1 -- 172.21.15.153:0/3486457119 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fe390004ed0 con 0x7fe3a00fe630 2024-03-21T10:53:04.632 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:04.636+0000 7fe396a00640 1 --2- 172.21.15.153:0/3486457119 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe38406e490 0x7fe384070950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:53:04.632 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:04.636+0000 7fe396a00640 1 -- 172.21.15.153:0/3486457119 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fe3900b0a10 con 0x7fe3a00fe630 2024-03-21T10:53:04.632 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:04.636+0000 7fe39d400640 1 --2- 172.21.15.153:0/3486457119 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe38406e490 0x7fe384070950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:53:04.633 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:04.637+0000 7fe39d400640 1 --2- 172.21.15.153:0/3486457119 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe38406e490 0x7fe384070950 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fe38c005fd0 tx=0x7fe38c005850 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:53:04.638 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:04.643+0000 7fe396a00640 1 -- 172.21.15.153:0/3486457119 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe390079a30 con 0x7fe3a00fe630 2024-03-21T10:53:04.874 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:04.878+0000 7fe3a4e00640 1 -- 172.21.15.153:0/3486457119 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 32, "format": "json"} v 0) v1 -- 0x7fe3a01991a0 con 0x7fe3a00fe630 2024-03-21T10:53:04.875 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:04.879+0000 7fe396a00640 1 -- 172.21.15.153:0/3486457119 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 32, "format": "json"}]=0 dumped fsmap epoch 32 v41) v1 ==== 107+0+5001 (secure 0 0 0) 0x7fe3900aee60 con 0x7fe3a00fe630 2024-03-21T10:53:04.875 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:53:04.875 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34442,"name":"cephfs.smithi153.zipckr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6827/1103846295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":1103846295},{"type":"v1","addr":"172.21.15.153:6827","nonce":1103846295}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26},{"gid":34474,"name":"cephfs.smithi153.iuyluj","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6829/1738630832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":1738630832},{"type":"v1","addr":"172.21.15.153:6829","nonce":1738630832}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":44313,"name":"cephfs.smithi174.yyimst","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6825/4127786797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":4127786797},{"type":"v1","addr":"172.21.15.174:6825","nonce":4127786797}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":44687,"name":"cephfs.smithi174.nhozag","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6827/2767517845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":2767517845},{"type":"v1","addr":"172.21.15.174:6827","nonce":2767517845}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":23,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T10:04:27.086576+0000","modified":"2024-03-21T10:19:05.022092+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":94,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-21T10:53:04.875 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 32 2024-03-21T10:53:04.879 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:04.883+0000 7fe3a4e00640 1 -- 172.21.15.153:0/3486457119 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe38406e490 msgr2=0x7fe384070950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:53:04.879 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:04.883+0000 7fe3a4e00640 1 --2- 172.21.15.153:0/3486457119 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe38406e490 0x7fe384070950 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fe38c005fd0 tx=0x7fe38c005850 comp rx=0 tx=0).stop 2024-03-21T10:53:04.879 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:04.883+0000 7fe3a4e00640 1 -- 172.21.15.153:0/3486457119 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe3a00fe630 msgr2=0x7fe3a0197e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:53:04.879 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:04.883+0000 7fe3a4e00640 1 --2- 172.21.15.153:0/3486457119 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe3a00fe630 0x7fe3a0197e50 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fe390009230 tx=0x7fe390004270 comp rx=0 tx=0).stop 2024-03-21T10:53:04.879 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:04.883+0000 7fe3a4e00640 1 -- 172.21.15.153:0/3486457119 shutdown_connections 2024-03-21T10:53:04.879 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:04.883+0000 7fe3a4e00640 1 --2- 172.21.15.153:0/3486457119 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe38406e490 0x7fe384070950 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:04.879 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:04.883+0000 7fe3a4e00640 1 --2- 172.21.15.153:0/3486457119 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe3a0100f20 0x7fe3a0198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:04.879 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:04.883+0000 7fe3a4e00640 1 --2- 172.21.15.153:0/3486457119 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe3a00fe630 0x7fe3a0197e50 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:04.879 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:04.883+0000 7fe3a4e00640 1 -- 172.21.15.153:0/3486457119 >> 172.21.15.153:0/3486457119 conn(0x7fe3a00fa6d0 msgr2=0x7fe3a00fbe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:53:04.879 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:04.883+0000 7fe3a4e00640 1 -- 172.21.15.153:0/3486457119 shutdown_connections 2024-03-21T10:53:04.879 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:04.883+0000 7fe3a4e00640 1 -- 172.21.15.153:0/3486457119 wait complete. 2024-03-21T10:53:04.905 DEBUG:tasks.fs:max_mds not reduced in epoch 32 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-03-21T10:53:04.906 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 32 2024-03-21T10:53:04.906 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 33 2024-03-21T10:53:04.946 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:53:04 smithi174 ceph-mon[88004]: pgmap v1239: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:53:04.946 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:53:04 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/3801914469' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 30, "format": "json"}]: dispatch 2024-03-21T10:53:04.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:53:04 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/916843650' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 31, "format": "json"}]: dispatch 2024-03-21T10:53:04.970 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:53:04 smithi153 ceph-mon[113733]: pgmap v1239: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:53:04.970 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:53:04 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/3801914469' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 30, "format": "json"}]: dispatch 2024-03-21T10:53:04.970 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:53:04 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/916843650' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 31, "format": "json"}]: dispatch 2024-03-21T10:53:05.097 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:05.101+0000 7fdab9c00640 1 -- 172.21.15.153:0/1998070811 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdab4072f90 msgr2=0x7fdab410c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:53:05.097 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:05.101+0000 7fdab9c00640 1 --2- 172.21.15.153:0/1998070811 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdab4072f90 0x7fdab410c390 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fda9c009880 tx=0x7fda9c02bdc0 comp rx=0 tx=0).stop 2024-03-21T10:53:05.097 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:05.101+0000 7fdab9c00640 1 -- 172.21.15.153:0/1998070811 shutdown_connections 2024-03-21T10:53:05.097 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:05.101+0000 7fdab9c00640 1 --2- 172.21.15.153:0/1998070811 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdab4072f90 0x7fdab410c390 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:05.097 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:05.101+0000 7fdab9c00640 1 --2- 172.21.15.153:0/1998070811 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdab4072670 0x7fdab4072a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:05.097 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:05.101+0000 7fdab9c00640 1 -- 172.21.15.153:0/1998070811 >> 172.21.15.153:0/1998070811 conn(0x7fdab40fbb20 msgr2=0x7fdab40fdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:53:05.097 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:05.101+0000 7fdab9c00640 1 -- 172.21.15.153:0/1998070811 shutdown_connections 2024-03-21T10:53:05.097 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:05.101+0000 7fdab9c00640 1 -- 172.21.15.153:0/1998070811 wait complete. 2024-03-21T10:53:05.098 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:05.102+0000 7fdab9c00640 1 Processor -- start 2024-03-21T10:53:05.098 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:05.102+0000 7fdab9c00640 1 -- start start 2024-03-21T10:53:05.098 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:05.103+0000 7fdab9c00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdab4072670 0x7fdab4197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:53:05.098 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:05.103+0000 7fdab9c00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdab4072f90 0x7fdab4198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:53:05.098 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:05.103+0000 7fdab9c00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdab4198ac0 con 0x7fdab4072f90 2024-03-21T10:53:05.098 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:05.103+0000 7fdab9c00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdab419c890 con 0x7fdab4072670 2024-03-21T10:53:05.100 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:05.103+0000 7fdaabe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdab4072f90 0x7fdab4198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:53:05.100 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:05.103+0000 7fdaabe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdab4072f90 0x7fdab4198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42756/0 (socket says 172.21.15.153:42756) 2024-03-21T10:53:05.100 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:05.103+0000 7fdaabe00640 1 -- 172.21.15.153:0/720849771 learned_addr learned my addr 172.21.15.153:0/720849771 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:53:05.100 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:05.103+0000 7fdab2a00640 1 --2- 172.21.15.153:0/720849771 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdab4072670 0x7fdab4197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:53:05.100 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:05.104+0000 7fdaabe00640 1 -- 172.21.15.153:0/720849771 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdab4072670 msgr2=0x7fdab4197ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:53:05.100 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:05.104+0000 7fdaabe00640 1 --2- 172.21.15.153:0/720849771 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdab4072670 0x7fdab4197ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:05.100 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:05.104+0000 7fdaabe00640 1 -- 172.21.15.153:0/720849771 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda9c009560 con 0x7fdab4072f90 2024-03-21T10:53:05.100 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:05.104+0000 7fdaabe00640 1 --2- 172.21.15.153:0/720849771 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdab4072f90 0x7fdab4198430 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fda9c036040 tx=0x7fda9c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:53:05.100 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:05.104+0000 7fdaab400640 1 -- 172.21.15.153:0/720849771 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fda9c0035f0 con 0x7fdab4072f90 2024-03-21T10:53:05.100 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:05.104+0000 7fdaab400640 1 -- 172.21.15.153:0/720849771 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fda9c003c10 con 0x7fdab4072f90 2024-03-21T10:53:05.100 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:05.104+0000 7fdab9c00640 1 -- 172.21.15.153:0/720849771 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdab419cb10 con 0x7fdab4072f90 2024-03-21T10:53:05.100 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:05.104+0000 7fdaab400640 1 -- 172.21.15.153:0/720849771 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fda9c03e450 con 0x7fdab4072f90 2024-03-21T10:53:05.101 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:05.105+0000 7fdab9c00640 1 -- 172.21.15.153:0/720849771 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdab419d070 con 0x7fdab4072f90 2024-03-21T10:53:05.101 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:05.106+0000 7fdab9c00640 1 -- 172.21.15.153:0/720849771 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdab404e040 con 0x7fdab4072f90 2024-03-21T10:53:05.103 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:05.107+0000 7fdaab400640 1 -- 172.21.15.153:0/720849771 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fda9c005060 con 0x7fdab4072f90 2024-03-21T10:53:05.104 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:05.109+0000 7fdaab400640 1 --2- 172.21.15.153:0/720849771 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fda9806e3d0 0x7fda98070890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:53:05.104 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:05.109+0000 7fdaab400640 1 -- 172.21.15.153:0/720849771 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fda9c0b1170 con 0x7fdab4072f90 2024-03-21T10:53:05.104 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:05.109+0000 7fdab2a00640 1 --2- 172.21.15.153:0/720849771 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fda9806e3d0 0x7fda98070890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:53:05.105 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:05.110+0000 7fdab2a00640 1 --2- 172.21.15.153:0/720849771 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fda9806e3d0 0x7fda98070890 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fdaa4005fd0 tx=0x7fdaa4005e20 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:53:05.109 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:05.114+0000 7fdaab400640 1 -- 172.21.15.153:0/720849771 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fda9c07a230 con 0x7fdab4072f90 2024-03-21T10:53:05.422 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:05.426+0000 7fdab9c00640 1 -- 172.21.15.153:0/720849771 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 33, "format": "json"} v 0) v1 -- 0x7fdab410f480 con 0x7fdab4072f90 2024-03-21T10:53:05.423 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:05.427+0000 7fdaab400640 1 -- 172.21.15.153:0/720849771 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 33, "format": "json"}]=0 dumped fsmap epoch 33 v41) v1 ==== 107+0+5000 (secure 0 0 0) 0x7fda9c003d80 con 0x7fdab4072f90 2024-03-21T10:53:05.423 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:53:05.423 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34442,"name":"cephfs.smithi153.zipckr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6827/1103846295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":1103846295},{"type":"v1","addr":"172.21.15.153:6827","nonce":1103846295}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26},{"gid":34474,"name":"cephfs.smithi153.iuyluj","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.153:6829/1738630832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":1738630832},{"type":"v1","addr":"172.21.15.153:6829","nonce":1738630832}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":44313,"name":"cephfs.smithi174.yyimst","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6825/4127786797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":4127786797},{"type":"v1","addr":"172.21.15.174:6825","nonce":4127786797}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":44687,"name":"cephfs.smithi174.nhozag","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6827/2767517845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":2767517845},{"type":"v1","addr":"172.21.15.174:6827","nonce":2767517845}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T10:04:27.086576+0000","modified":"2024-03-21T10:50:24.271385+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":94,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-21T10:53:05.423 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 33 2024-03-21T10:53:05.425 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:05.430+0000 7fdab9c00640 1 -- 172.21.15.153:0/720849771 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fda9806e3d0 msgr2=0x7fda98070890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:53:05.425 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:05.430+0000 7fdab9c00640 1 --2- 172.21.15.153:0/720849771 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fda9806e3d0 0x7fda98070890 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fdaa4005fd0 tx=0x7fdaa4005e20 comp rx=0 tx=0).stop 2024-03-21T10:53:05.425 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:05.430+0000 7fdab9c00640 1 -- 172.21.15.153:0/720849771 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdab4072f90 msgr2=0x7fdab4198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:53:05.426 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:05.430+0000 7fdab9c00640 1 --2- 172.21.15.153:0/720849771 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdab4072f90 0x7fdab4198430 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fda9c036040 tx=0x7fda9c004040 comp rx=0 tx=0).stop 2024-03-21T10:53:05.426 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:05.430+0000 7fdab9c00640 1 -- 172.21.15.153:0/720849771 shutdown_connections 2024-03-21T10:53:05.426 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:05.430+0000 7fdab9c00640 1 --2- 172.21.15.153:0/720849771 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fda9806e3d0 0x7fda98070890 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:05.426 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:05.430+0000 7fdab9c00640 1 --2- 172.21.15.153:0/720849771 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fdab4072f90 0x7fdab4198430 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:05.426 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:05.430+0000 7fdab9c00640 1 --2- 172.21.15.153:0/720849771 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdab4072670 0x7fdab4197ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:05.426 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:05.430+0000 7fdab9c00640 1 -- 172.21.15.153:0/720849771 >> 172.21.15.153:0/720849771 conn(0x7fdab40fbb20 msgr2=0x7fdab4106bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:53:05.426 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:05.430+0000 7fdab9c00640 1 -- 172.21.15.153:0/720849771 shutdown_connections 2024-03-21T10:53:05.426 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:05.430+0000 7fdab9c00640 1 -- 172.21.15.153:0/720849771 wait complete. 2024-03-21T10:53:05.445 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 33 2024-03-21T10:53:05.445 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 34 2024-03-21T10:53:05.647 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:05.651+0000 7f965e000640 1 -- 172.21.15.153:0/503749689 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f96580fee00 msgr2=0x7f96580ff240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:53:05.647 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:05.651+0000 7f965e000640 1 --2- 172.21.15.153:0/503749689 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f96580fee00 0x7f96580ff240 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f9640009c20 tx=0x7f964002c1b0 comp rx=0 tx=0).stop 2024-03-21T10:53:05.647 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:05.651+0000 7f965e000640 1 -- 172.21.15.153:0/503749689 shutdown_connections 2024-03-21T10:53:05.647 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:05.651+0000 7f965e000640 1 --2- 172.21.15.153:0/503749689 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f96580fee00 0x7f96580ff240 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:05.648 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:05.651+0000 7f965e000640 1 --2- 172.21.15.153:0/503749689 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9658104e20 0x7f9658105200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:05.648 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:05.651+0000 7f965e000640 1 -- 172.21.15.153:0/503749689 >> 172.21.15.153:0/503749689 conn(0x7f9658074930 msgr2=0x7f9658074d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:53:05.648 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:05.652+0000 7f965e000640 1 -- 172.21.15.153:0/503749689 shutdown_connections 2024-03-21T10:53:05.648 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:05.652+0000 7f965e000640 1 -- 172.21.15.153:0/503749689 wait complete. 2024-03-21T10:53:05.648 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:05.653+0000 7f965e000640 1 Processor -- start 2024-03-21T10:53:05.648 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:05.653+0000 7f965e000640 1 -- start start 2024-03-21T10:53:05.649 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:05.654+0000 7f965e000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f96580fee00 0x7f965810ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:53:05.650 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:05.654+0000 7f965e000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9658104e20 0x7f965810b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:53:05.650 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:05.654+0000 7f965e000640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f965810b6b0 con 0x7f9658104e20 2024-03-21T10:53:05.650 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:05.654+0000 7f965e000640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f965810b820 con 0x7f96580fee00 2024-03-21T10:53:05.650 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:05.654+0000 7f9657e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9658104e20 0x7f965810b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:53:05.650 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:05.654+0000 7f9657e00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9658104e20 0x7f965810b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42776/0 (socket says 172.21.15.153:42776) 2024-03-21T10:53:05.650 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:05.654+0000 7f9657e00640 1 -- 172.21.15.153:0/659035819 learned_addr learned my addr 172.21.15.153:0/659035819 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:53:05.650 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:05.654+0000 7f965cc00640 1 --2- 172.21.15.153:0/659035819 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f96580fee00 0x7f965810ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:53:05.650 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:05.654+0000 7f9657e00640 1 -- 172.21.15.153:0/659035819 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f96580fee00 msgr2=0x7f965810ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:53:05.650 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:05.654+0000 7f9657e00640 1 --2- 172.21.15.153:0/659035819 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f96580fee00 0x7f965810ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:05.650 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:05.654+0000 7f9657e00640 1 -- 172.21.15.153:0/659035819 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9640009900 con 0x7f9658104e20 2024-03-21T10:53:05.651 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:05.655+0000 7f9657e00640 1 --2- 172.21.15.153:0/659035819 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9658104e20 0x7f965810b170 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f9640000f80 tx=0x7f9640032b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:53:05.651 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:05.655+0000 7f9655600640 1 -- 172.21.15.153:0/659035819 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9640034820 con 0x7f9658104e20 2024-03-21T10:53:05.651 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:05.655+0000 7f9655600640 1 -- 172.21.15.153:0/659035819 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9640003e30 con 0x7f9658104e20 2024-03-21T10:53:05.651 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:05.655+0000 7f9655600640 1 -- 172.21.15.153:0/659035819 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f964003f440 con 0x7f9658104e20 2024-03-21T10:53:05.652 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:05.655+0000 7f965e000640 1 -- 172.21.15.153:0/659035819 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f965810ba40 con 0x7f9658104e20 2024-03-21T10:53:05.652 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:05.655+0000 7f965e000640 1 -- 172.21.15.153:0/659035819 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96581a2070 con 0x7f9658104e20 2024-03-21T10:53:05.652 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:05.656+0000 7f965e000640 1 -- 172.21.15.153:0/659035819 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f965804e040 con 0x7f9658104e20 2024-03-21T10:53:05.653 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:05.658+0000 7f9655600640 1 -- 172.21.15.153:0/659035819 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f964003b040 con 0x7f9658104e20 2024-03-21T10:53:05.654 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:05.659+0000 7f9655600640 1 --2- 172.21.15.153:0/659035819 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f963c06e3d0 0x7f963c070890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:53:05.654 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:05.659+0000 7f9655600640 1 -- 172.21.15.153:0/659035819 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f96400b1160 con 0x7f9658104e20 2024-03-21T10:53:05.655 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:05.659+0000 7f965cc00640 1 --2- 172.21.15.153:0/659035819 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f963c06e3d0 0x7f963c070890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:53:05.655 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:05.660+0000 7f965cc00640 1 --2- 172.21.15.153:0/659035819 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f963c06e3d0 0x7f963c070890 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f9648005fd0 tx=0x7f9648005850 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:53:05.659 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:05.664+0000 7f9655600640 1 -- 172.21.15.153:0/659035819 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f964007a220 con 0x7f9658104e20 2024-03-21T10:53:05.965 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:05.970+0000 7f965e000640 1 -- 172.21.15.153:0/659035819 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 34, "format": "json"} v 0) v1 -- 0x7f965810d1f0 con 0x7f9658104e20 2024-03-21T10:53:05.966 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:05.971+0000 7f9655600640 1 -- 172.21.15.153:0/659035819 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 34, "format": "json"}]=0 dumped fsmap epoch 34 v41) v1 ==== 107+0+5057 (secure 0 0 0) 0x7f964007d8e0 con 0x7f9658104e20 2024-03-21T10:53:05.966 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:53:05.966 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44313,"name":"cephfs.smithi174.yyimst","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6825/4127786797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":4127786797},{"type":"v1","addr":"172.21.15.174:6825","nonce":4127786797}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":44687,"name":"cephfs.smithi174.nhozag","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6827/2767517845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":2767517845},{"type":"v1","addr":"172.21.15.174:6827","nonce":2767517845}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T10:04:27.086576+0000","modified":"2024-03-21T10:50:25.271421+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":94,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34442,"mds_1":34474},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34442":{"gid":34442,"name":"cephfs.smithi153.zipckr","rank":0,"incarnation":34,"state":"up:replay","state_seq":1,"addr":"172.21.15.153:6827/1103846295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":1103846295},{"type":"v1","addr":"172.21.15.153:6827","nonce":1103846295}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34474":{"gid":34474,"name":"cephfs.smithi153.iuyluj","rank":1,"incarnation":34,"state":"up:replay","state_seq":1,"addr":"172.21.15.153:6829/1738630832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":1738630832},{"type":"v1","addr":"172.21.15.153:6829","nonce":1738630832}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-21T10:53:05.967 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 34 2024-03-21T10:53:05.969 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:05.973+0000 7f965e000640 1 -- 172.21.15.153:0/659035819 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f963c06e3d0 msgr2=0x7f963c070890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:53:05.969 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:05.973+0000 7f965e000640 1 --2- 172.21.15.153:0/659035819 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f963c06e3d0 0x7f963c070890 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f9648005fd0 tx=0x7f9648005850 comp rx=0 tx=0).stop 2024-03-21T10:53:05.969 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:05.973+0000 7f965e000640 1 -- 172.21.15.153:0/659035819 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9658104e20 msgr2=0x7f965810b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:53:05.969 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:05.973+0000 7f965e000640 1 --2- 172.21.15.153:0/659035819 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9658104e20 0x7f965810b170 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f9640000f80 tx=0x7f9640032b60 comp rx=0 tx=0).stop 2024-03-21T10:53:05.969 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:05.973+0000 7f965e000640 1 -- 172.21.15.153:0/659035819 shutdown_connections 2024-03-21T10:53:05.969 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:05.973+0000 7f965e000640 1 --2- 172.21.15.153:0/659035819 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f963c06e3d0 0x7f963c070890 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:05.969 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:05.973+0000 7f965e000640 1 --2- 172.21.15.153:0/659035819 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f9658104e20 0x7f965810b170 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:05.969 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:05.973+0000 7f965e000640 1 --2- 172.21.15.153:0/659035819 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f96580fee00 0x7f965810ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:05.969 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:05.973+0000 7f965e000640 1 -- 172.21.15.153:0/659035819 >> 172.21.15.153:0/659035819 conn(0x7f9658074930 msgr2=0x7f96580fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:53:05.969 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:05.973+0000 7f965e000640 1 -- 172.21.15.153:0/659035819 shutdown_connections 2024-03-21T10:53:05.969 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:05.973+0000 7f965e000640 1 -- 172.21.15.153:0/659035819 wait complete. 2024-03-21T10:53:05.986 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:53:05 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/3486457119' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 32, "format": "json"}]: dispatch 2024-03-21T10:53:05.986 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:53:05 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:53:05.986 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:53:05 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/720849771' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 33, "format": "json"}]: dispatch 2024-03-21T10:53:05.990 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 34 2024-03-21T10:53:05.990 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 35 2024-03-21T10:53:06.193 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:06.197+0000 7f6b99200640 1 -- 172.21.15.153:0/3495562706 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6b941000e0 msgr2=0x7f6b941004c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:53:06.193 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:06.197+0000 7f6b99200640 1 --2- 172.21.15.153:0/3495562706 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6b941000e0 0x7f6b941004c0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f6b84009880 tx=0x7f6b8402bce0 comp rx=0 tx=0).stop 2024-03-21T10:53:06.193 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:06.197+0000 7f6b99200640 1 -- 172.21.15.153:0/3495562706 shutdown_connections 2024-03-21T10:53:06.193 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:06.197+0000 7f6b99200640 1 --2- 172.21.15.153:0/3495562706 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6b94100a90 0x7f6b94104ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:06.193 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:06.197+0000 7f6b99200640 1 --2- 172.21.15.153:0/3495562706 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6b941000e0 0x7f6b941004c0 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:06.193 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:06.197+0000 7f6b99200640 1 -- 172.21.15.153:0/3495562706 >> 172.21.15.153:0/3495562706 conn(0x7f6b940fb960 msgr2=0x7f6b940fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:53:06.193 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:06.197+0000 7f6b99200640 1 -- 172.21.15.153:0/3495562706 shutdown_connections 2024-03-21T10:53:06.194 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:06.197+0000 7f6b99200640 1 -- 172.21.15.153:0/3495562706 wait complete. 2024-03-21T10:53:06.194 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:06.199+0000 7f6b99200640 1 Processor -- start 2024-03-21T10:53:06.194 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:06.199+0000 7f6b99200640 1 -- start start 2024-03-21T10:53:06.195 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:06.199+0000 7f6b99200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6b941000e0 0x7f6b940725a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:53:06.195 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:06.199+0000 7f6b99200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6b94072ae0 0x7f6b94072f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:53:06.195 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:06.199+0000 7f6b99200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b941a1e10 con 0x7f6b941000e0 2024-03-21T10:53:06.195 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:06.199+0000 7f6b99200640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b941a1f50 con 0x7f6b94072ae0 2024-03-21T10:53:06.195 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:06.200+0000 7f6b93400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6b941000e0 0x7f6b940725a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:53:06.195 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:06.200+0000 7f6b93400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6b941000e0 0x7f6b940725a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42800/0 (socket says 172.21.15.153:42800) 2024-03-21T10:53:06.195 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:06.200+0000 7f6b93400640 1 -- 172.21.15.153:0/3024545325 learned_addr learned my addr 172.21.15.153:0/3024545325 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:53:06.195 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:06.200+0000 7f6b92a00640 1 --2- 172.21.15.153:0/3024545325 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6b94072ae0 0x7f6b94072f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:53:06.196 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:06.200+0000 7f6b93400640 1 -- 172.21.15.153:0/3024545325 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6b94072ae0 msgr2=0x7f6b94072f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:53:06.196 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:06.200+0000 7f6b93400640 1 --2- 172.21.15.153:0/3024545325 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6b94072ae0 0x7f6b94072f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:06.197 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:06.200+0000 7f6b93400640 1 -- 172.21.15.153:0/3024545325 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b84009560 con 0x7f6b941000e0 2024-03-21T10:53:06.197 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:06.201+0000 7f6b93400640 1 --2- 172.21.15.153:0/3024545325 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6b941000e0 0x7f6b940725a0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f6b84006010 tx=0x7f6b8402c6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:53:06.197 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:06.201+0000 7f6b8be00640 1 -- 172.21.15.153:0/3024545325 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6b840035f0 con 0x7f6b941000e0 2024-03-21T10:53:06.197 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:06.201+0000 7f6b8be00640 1 -- 172.21.15.153:0/3024545325 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6b84003c10 con 0x7f6b941000e0 2024-03-21T10:53:06.197 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:06.201+0000 7f6b8be00640 1 -- 172.21.15.153:0/3024545325 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6b840055e0 con 0x7f6b941000e0 2024-03-21T10:53:06.197 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:06.201+0000 7f6b99200640 1 -- 172.21.15.153:0/3024545325 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b941a2090 con 0x7f6b941000e0 2024-03-21T10:53:06.197 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:06.201+0000 7f6b99200640 1 -- 172.21.15.153:0/3024545325 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b941a2510 con 0x7f6b941000e0 2024-03-21T10:53:06.197 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:53:05 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/3486457119' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 32, "format": "json"}]: dispatch 2024-03-21T10:53:06.197 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:53:05 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:53:06.197 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:53:05 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/720849771' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 33, "format": "json"}]: dispatch 2024-03-21T10:53:06.198 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:06.203+0000 7f6b99200640 1 -- 172.21.15.153:0/3024545325 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b9404e890 con 0x7f6b941000e0 2024-03-21T10:53:06.199 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:06.203+0000 7f6b8be00640 1 -- 172.21.15.153:0/3024545325 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f6b84003750 con 0x7f6b941000e0 2024-03-21T10:53:06.200 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:06.205+0000 7f6b8be00640 1 --2- 172.21.15.153:0/3024545325 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f6b7806e420 0x7f6b780708e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:53:06.201 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:06.205+0000 7f6b8be00640 1 -- 172.21.15.153:0/3024545325 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f6b840b06b0 con 0x7f6b941000e0 2024-03-21T10:53:06.201 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:06.206+0000 7f6b92a00640 1 --2- 172.21.15.153:0/3024545325 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f6b7806e420 0x7f6b780708e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:53:06.202 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:06.207+0000 7f6b92a00640 1 --2- 172.21.15.153:0/3024545325 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f6b7806e420 0x7f6b780708e0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f6b94067c60 tx=0x7f6b80005f90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:53:06.207 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:06.211+0000 7f6b8be00640 1 -- 172.21.15.153:0/3024545325 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f6b840798a0 con 0x7f6b941000e0 2024-03-21T10:53:06.530 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:06.535+0000 7f6b99200640 1 -- 172.21.15.153:0/3024545325 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 35, "format": "json"} v 0) v1 -- 0x7f6b941083e0 con 0x7f6b941000e0 2024-03-21T10:53:06.531 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:06.536+0000 7f6b8be00640 1 -- 172.21.15.153:0/3024545325 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 35, "format": "json"}]=0 dumped fsmap epoch 35 v41) v1 ==== 107+0+5056 (secure 0 0 0) 0x7f6b84079030 con 0x7f6b941000e0 2024-03-21T10:53:06.531 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:53:06.532 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44313,"name":"cephfs.smithi174.yyimst","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6825/4127786797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":4127786797},{"type":"v1","addr":"172.21.15.174:6825","nonce":4127786797}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":44687,"name":"cephfs.smithi174.nhozag","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6827/2767517845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":2767517845},{"type":"v1","addr":"172.21.15.174:6827","nonce":2767517845}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":35,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T10:04:27.086576+0000","modified":"2024-03-21T10:50:25.280111+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":94,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34442,"mds_1":34474},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34442":{"gid":34442,"name":"cephfs.smithi153.zipckr","rank":0,"incarnation":34,"state":"up:replay","state_seq":1,"addr":"172.21.15.153:6827/1103846295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":1103846295},{"type":"v1","addr":"172.21.15.153:6827","nonce":1103846295}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34474":{"gid":34474,"name":"cephfs.smithi153.iuyluj","rank":1,"incarnation":34,"state":"up:replay","state_seq":1,"addr":"172.21.15.153:6829/1738630832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":1738630832},{"type":"v1","addr":"172.21.15.153:6829","nonce":1738630832}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-21T10:53:06.532 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 35 2024-03-21T10:53:06.534 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:06.538+0000 7f6b99200640 1 -- 172.21.15.153:0/3024545325 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f6b7806e420 msgr2=0x7f6b780708e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:53:06.534 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:06.538+0000 7f6b99200640 1 --2- 172.21.15.153:0/3024545325 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f6b7806e420 0x7f6b780708e0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f6b94067c60 tx=0x7f6b80005f90 comp rx=0 tx=0).stop 2024-03-21T10:53:06.535 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:06.538+0000 7f6b99200640 1 -- 172.21.15.153:0/3024545325 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6b941000e0 msgr2=0x7f6b940725a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:53:06.535 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:06.538+0000 7f6b99200640 1 --2- 172.21.15.153:0/3024545325 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6b941000e0 0x7f6b940725a0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f6b84006010 tx=0x7f6b8402c6f0 comp rx=0 tx=0).stop 2024-03-21T10:53:06.535 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:06.538+0000 7f6b99200640 1 -- 172.21.15.153:0/3024545325 shutdown_connections 2024-03-21T10:53:06.535 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:06.538+0000 7f6b99200640 1 --2- 172.21.15.153:0/3024545325 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f6b7806e420 0x7f6b780708e0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:06.535 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:06.538+0000 7f6b99200640 1 --2- 172.21.15.153:0/3024545325 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6b94072ae0 0x7f6b94072f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:06.535 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:06.538+0000 7f6b99200640 1 --2- 172.21.15.153:0/3024545325 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6b941000e0 0x7f6b940725a0 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:06.535 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:06.538+0000 7f6b99200640 1 -- 172.21.15.153:0/3024545325 >> 172.21.15.153:0/3024545325 conn(0x7f6b940fb960 msgr2=0x7f6b940fcff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:53:06.535 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:06.538+0000 7f6b99200640 1 -- 172.21.15.153:0/3024545325 shutdown_connections 2024-03-21T10:53:06.535 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:06.538+0000 7f6b99200640 1 -- 172.21.15.153:0/3024545325 wait complete. 2024-03-21T10:53:06.559 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 36 2024-03-21T10:53:06.764 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:06.768+0000 7fa2dce00640 1 -- 172.21.15.153:0/1806694883 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa2d80feff0 msgr2=0x7fa2d80ff3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:53:06.764 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:06.768+0000 7fa2dce00640 1 --2- 172.21.15.153:0/1806694883 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa2d80feff0 0x7fa2d80ff3d0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fa2c8009880 tx=0x7fa2c802bd90 comp rx=0 tx=0).stop 2024-03-21T10:53:06.764 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:06.768+0000 7fa2dce00640 1 -- 172.21.15.153:0/1806694883 shutdown_connections 2024-03-21T10:53:06.764 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:06.768+0000 7fa2dce00640 1 --2- 172.21.15.153:0/1806694883 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2d80ff910 0x7fa2d8103dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:06.764 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:06.768+0000 7fa2dce00640 1 --2- 172.21.15.153:0/1806694883 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa2d80feff0 0x7fa2d80ff3d0 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:06.764 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:06.768+0000 7fa2dce00640 1 -- 172.21.15.153:0/1806694883 >> 172.21.15.153:0/1806694883 conn(0x7fa2d80faa20 msgr2=0x7fa2d80fce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:53:06.765 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:06.768+0000 7fa2dce00640 1 -- 172.21.15.153:0/1806694883 shutdown_connections 2024-03-21T10:53:06.765 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:06.768+0000 7fa2dce00640 1 -- 172.21.15.153:0/1806694883 wait complete. 2024-03-21T10:53:06.765 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:06.770+0000 7fa2dce00640 1 Processor -- start 2024-03-21T10:53:06.765 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:06.770+0000 7fa2dce00640 1 -- start start 2024-03-21T10:53:06.766 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:06.770+0000 7fa2dce00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2d80feff0 0x7fa2d819c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:53:06.766 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:06.770+0000 7fa2dce00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa2d80ff910 0x7fa2d819c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:53:06.766 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:06.770+0000 7fa2dce00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d819cf40 con 0x7fa2d80ff910 2024-03-21T10:53:06.766 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:06.770+0000 7fa2dce00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d81a0d10 con 0x7fa2d80feff0 2024-03-21T10:53:06.767 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:06.771+0000 7fa2cfe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa2d80ff910 0x7fa2d819c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:53:06.767 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:06.771+0000 7fa2cfe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa2d80ff910 0x7fa2d819c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42818/0 (socket says 172.21.15.153:42818) 2024-03-21T10:53:06.767 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:06.771+0000 7fa2cfe00640 1 -- 172.21.15.153:0/1810872099 learned_addr learned my addr 172.21.15.153:0/1810872099 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:53:06.767 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:06.771+0000 7fa2d5e00640 1 --2- 172.21.15.153:0/1810872099 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2d80feff0 0x7fa2d819c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:53:06.767 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:06.771+0000 7fa2cfe00640 1 -- 172.21.15.153:0/1810872099 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2d80feff0 msgr2=0x7fa2d819c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:53:06.767 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:06.771+0000 7fa2cfe00640 1 --2- 172.21.15.153:0/1810872099 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2d80feff0 0x7fa2d819c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:06.769 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:06.771+0000 7fa2cfe00640 1 -- 172.21.15.153:0/1810872099 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2c8009560 con 0x7fa2d80ff910 2024-03-21T10:53:06.769 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:06.772+0000 7fa2cfe00640 1 --2- 172.21.15.153:0/1810872099 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa2d80ff910 0x7fa2d819c8b0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fa2c000d6d0 tx=0x7fa2c000db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:53:06.770 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:06.772+0000 7fa2cea00640 1 -- 172.21.15.153:0/1810872099 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa2c00051f0 con 0x7fa2d80ff910 2024-03-21T10:53:06.770 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:06.772+0000 7fa2cea00640 1 -- 172.21.15.153:0/1810872099 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa2c0009ce0 con 0x7fa2d80ff910 2024-03-21T10:53:06.770 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:06.772+0000 7fa2dce00640 1 -- 172.21.15.153:0/1810872099 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2d81a0ff0 con 0x7fa2d80ff910 2024-03-21T10:53:06.770 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:06.772+0000 7fa2cea00640 1 -- 172.21.15.153:0/1810872099 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa2c0004db0 con 0x7fa2d80ff910 2024-03-21T10:53:06.770 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:06.772+0000 7fa2dce00640 1 -- 172.21.15.153:0/1810872099 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2d81a1550 con 0x7fa2d80ff910 2024-03-21T10:53:06.770 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:06.773+0000 7fa2dce00640 1 -- 172.21.15.153:0/1810872099 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2d804e040 con 0x7fa2d80ff910 2024-03-21T10:53:06.771 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:06.775+0000 7fa2cea00640 1 -- 172.21.15.153:0/1810872099 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fa2c0009820 con 0x7fa2d80ff910 2024-03-21T10:53:06.772 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:06.776+0000 7fa2cea00640 1 --2- 172.21.15.153:0/1810872099 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa2bc06e380 0x7fa2bc070840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:53:06.772 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:06.777+0000 7fa2cea00640 1 -- 172.21.15.153:0/1810872099 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fa2c008df20 con 0x7fa2d80ff910 2024-03-21T10:53:06.772 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:06.777+0000 7fa2d5e00640 1 --2- 172.21.15.153:0/1810872099 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa2bc06e380 0x7fa2bc070840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:53:06.773 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:06.778+0000 7fa2d5e00640 1 --2- 172.21.15.153:0/1810872099 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa2bc06e380 0x7fa2bc070840 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fa2c8005ab0 tx=0x7fa2c8005a40 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:53:06.780 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:06.784+0000 7fa2cea00640 1 -- 172.21.15.153:0/1810872099 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa2c00c2740 con 0x7fa2d80ff910 2024-03-21T10:53:07.036 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:07.041+0000 7fa2dce00640 1 -- 172.21.15.153:0/1810872099 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 36, "format": "json"} v 0) v1 -- 0x7fa2d8101480 con 0x7fa2d80ff910 2024-03-21T10:53:07.038 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:07.042+0000 7fa2cea00640 1 -- 172.21.15.153:0/1810872099 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 36, "format": "json"}]=0 dumped fsmap epoch 36 v41) v1 ==== 107+0+5059 (secure 0 0 0) 0x7fa2c0017090 con 0x7fa2d80ff910 2024-03-21T10:53:07.038 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:53:07.038 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44313,"name":"cephfs.smithi174.yyimst","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6825/4127786797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":4127786797},{"type":"v1","addr":"172.21.15.174:6825","nonce":4127786797}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":44687,"name":"cephfs.smithi174.nhozag","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6827/2767517845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":2767517845},{"type":"v1","addr":"172.21.15.174:6827","nonce":2767517845}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T10:04:27.086576+0000","modified":"2024-03-21T10:50:32.154857+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":94,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34442,"mds_1":34474},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34442":{"gid":34442,"name":"cephfs.smithi153.zipckr","rank":0,"incarnation":34,"state":"up:replay","state_seq":1,"addr":"172.21.15.153:6827/1103846295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":1103846295},{"type":"v1","addr":"172.21.15.153:6827","nonce":1103846295}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34474":{"gid":34474,"name":"cephfs.smithi153.iuyluj","rank":1,"incarnation":34,"state":"up:resolve","state_seq":465,"addr":"172.21.15.153:6829/1738630832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":1738630832},{"type":"v1","addr":"172.21.15.153:6829","nonce":1738630832}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-21T10:53:07.038 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 36 2024-03-21T10:53:07.042 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:07.046+0000 7fa2dce00640 1 -- 172.21.15.153:0/1810872099 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa2bc06e380 msgr2=0x7fa2bc070840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:53:07.042 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:07.046+0000 7fa2dce00640 1 --2- 172.21.15.153:0/1810872099 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa2bc06e380 0x7fa2bc070840 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fa2c8005ab0 tx=0x7fa2c8005a40 comp rx=0 tx=0).stop 2024-03-21T10:53:07.042 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:07.046+0000 7fa2dce00640 1 -- 172.21.15.153:0/1810872099 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa2d80ff910 msgr2=0x7fa2d819c8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:53:07.042 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:07.046+0000 7fa2dce00640 1 --2- 172.21.15.153:0/1810872099 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa2d80ff910 0x7fa2d819c8b0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fa2c000d6d0 tx=0x7fa2c000db90 comp rx=0 tx=0).stop 2024-03-21T10:53:07.042 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:07.046+0000 7fa2dce00640 1 -- 172.21.15.153:0/1810872099 shutdown_connections 2024-03-21T10:53:07.042 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:07.046+0000 7fa2dce00640 1 --2- 172.21.15.153:0/1810872099 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fa2bc06e380 0x7fa2bc070840 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:07.042 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:07.046+0000 7fa2dce00640 1 --2- 172.21.15.153:0/1810872099 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fa2d80ff910 0x7fa2d819c8b0 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:07.043 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:07.046+0000 7fa2dce00640 1 --2- 172.21.15.153:0/1810872099 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2d80feff0 0x7fa2d819c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:07.043 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:07.046+0000 7fa2dce00640 1 -- 172.21.15.153:0/1810872099 >> 172.21.15.153:0/1810872099 conn(0x7fa2d80faa20 msgr2=0x7fa2d8110450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:53:07.043 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:07.046+0000 7fa2dce00640 1 -- 172.21.15.153:0/1810872099 shutdown_connections 2024-03-21T10:53:07.043 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:07.046+0000 7fa2dce00640 1 -- 172.21.15.153:0/1810872099 wait complete. 2024-03-21T10:53:07.051 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:53:06 smithi153 ceph-mon[113733]: pgmap v1240: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:53:07.051 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:53:06 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/659035819' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 34, "format": "json"}]: dispatch 2024-03-21T10:53:07.051 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:53:06 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:53:07.051 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:53:06 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:53:07.051 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:53:06 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:53:07.051 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:53:06 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/3024545325' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 35, "format": "json"}]: dispatch 2024-03-21T10:53:07.062 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 37 2024-03-21T10:53:07.196 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:53:06 smithi174 ceph-mon[88004]: pgmap v1240: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:53:07.197 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:53:06 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/659035819' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 34, "format": "json"}]: dispatch 2024-03-21T10:53:07.197 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:53:06 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:53:07.197 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:53:06 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:53:07.197 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:53:06 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:53:07.197 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:53:06 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/3024545325' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 35, "format": "json"}]: dispatch 2024-03-21T10:53:07.272 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:07.277+0000 7f6c36200640 1 -- 172.21.15.153:0/1977249731 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6c30100c20 msgr2=0x7f6c30104c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:53:07.273 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:07.277+0000 7f6c36200640 1 --2- 172.21.15.153:0/1977249731 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6c30100c20 0x7f6c30104c60 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f6c18009880 tx=0x7f6c1802bce0 comp rx=0 tx=0).stop 2024-03-21T10:53:07.273 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:07.277+0000 7f6c36200640 1 -- 172.21.15.153:0/1977249731 shutdown_connections 2024-03-21T10:53:07.273 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:07.277+0000 7f6c36200640 1 --2- 172.21.15.153:0/1977249731 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6c30100c20 0x7f6c30104c60 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:07.273 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:07.277+0000 7f6c36200640 1 --2- 172.21.15.153:0/1977249731 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6c30100270 0x7f6c30100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:07.273 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:07.277+0000 7f6c36200640 1 -- 172.21.15.153:0/1977249731 >> 172.21.15.153:0/1977249731 conn(0x7f6c300fbaf0 msgr2=0x7f6c300fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:53:07.273 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:07.277+0000 7f6c36200640 1 -- 172.21.15.153:0/1977249731 shutdown_connections 2024-03-21T10:53:07.273 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:07.277+0000 7f6c36200640 1 -- 172.21.15.153:0/1977249731 wait complete. 2024-03-21T10:53:07.275 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:07.278+0000 7f6c36200640 1 Processor -- start 2024-03-21T10:53:07.275 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:07.278+0000 7f6c36200640 1 -- start start 2024-03-21T10:53:07.275 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:07.278+0000 7f6c36200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6c30100270 0x7f6c30197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:53:07.275 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:07.278+0000 7f6c36200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6c301983a0 0x7f6c3019c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:53:07.275 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:07.278+0000 7f6c36200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c301989a0 con 0x7f6c30100270 2024-03-21T10:53:07.275 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:07.278+0000 7f6c36200640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c30198b10 con 0x7f6c301983a0 2024-03-21T10:53:07.275 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:07.279+0000 7f6c2f400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6c30100270 0x7f6c30197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:53:07.275 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:07.279+0000 7f6c2f400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6c30100270 0x7f6c30197e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42848/0 (socket says 172.21.15.153:42848) 2024-03-21T10:53:07.275 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:07.279+0000 7f6c2f400640 1 -- 172.21.15.153:0/1113826422 learned_addr learned my addr 172.21.15.153:0/1113826422 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:53:07.275 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:07.279+0000 7f6c2f400640 1 -- 172.21.15.153:0/1113826422 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6c301983a0 msgr2=0x7f6c3019c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:53:07.275 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:07.279+0000 7f6c2f400640 1 --2- 172.21.15.153:0/1113826422 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6c301983a0 0x7f6c3019c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:07.276 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:07.279+0000 7f6c2f400640 1 -- 172.21.15.153:0/1113826422 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c18009560 con 0x7f6c30100270 2024-03-21T10:53:07.276 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:07.279+0000 7f6c2f400640 1 --2- 172.21.15.153:0/1113826422 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6c30100270 0x7f6c30197e60 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f6c2000ea30 tx=0x7f6c2000eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:53:07.276 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:07.280+0000 7f6c2cc00640 1 -- 172.21.15.153:0/1113826422 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c2000cdb0 con 0x7f6c30100270 2024-03-21T10:53:07.276 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:07.280+0000 7f6c2cc00640 1 -- 172.21.15.153:0/1113826422 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6c20005020 con 0x7f6c30100270 2024-03-21T10:53:07.276 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:07.280+0000 7f6c36200640 1 -- 172.21.15.153:0/1113826422 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c3019ce00 con 0x7f6c30100270 2024-03-21T10:53:07.277 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:07.280+0000 7f6c2cc00640 1 -- 172.21.15.153:0/1113826422 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c200116a0 con 0x7f6c30100270 2024-03-21T10:53:07.277 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:07.280+0000 7f6c36200640 1 -- 172.21.15.153:0/1113826422 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c3019d360 con 0x7f6c30100270 2024-03-21T10:53:07.277 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:07.281+0000 7f6c36200640 1 -- 172.21.15.153:0/1113826422 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c3019d640 con 0x7f6c30100270 2024-03-21T10:53:07.277 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:07.282+0000 7f6c2cc00640 1 -- 172.21.15.153:0/1113826422 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f6c20005190 con 0x7f6c30100270 2024-03-21T10:53:07.278 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:07.283+0000 7f6c2cc00640 1 --2- 172.21.15.153:0/1113826422 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f6c1406e490 0x7f6c14070950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:53:07.278 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:07.283+0000 7f6c2cc00640 1 -- 172.21.15.153:0/1113826422 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f6c2008e970 con 0x7f6c30100270 2024-03-21T10:53:07.279 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:07.283+0000 7f6c26a00640 1 --2- 172.21.15.153:0/1113826422 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f6c1406e490 0x7f6c14070950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:53:07.279 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:07.284+0000 7f6c26a00640 1 --2- 172.21.15.153:0/1113826422 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f6c1406e490 0x7f6c14070950 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f6c18036040 tx=0x7f6c18000bc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:53:07.283 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:07.288+0000 7f6c2cc00640 1 -- 172.21.15.153:0/1113826422 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f6c20057a30 con 0x7f6c30100270 2024-03-21T10:53:07.524 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:07.528+0000 7f6c36200640 1 -- 172.21.15.153:0/1113826422 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 37, "format": "json"} v 0) v1 -- 0x7f6c301991e0 con 0x7f6c30100270 2024-03-21T10:53:07.524 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:07.529+0000 7f6c2cc00640 1 -- 172.21.15.153:0/1113826422 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 37, "format": "json"}]=0 dumped fsmap epoch 37 v41) v1 ==== 107+0+5062 (secure 0 0 0) 0x7f6c2008ce60 con 0x7f6c30100270 2024-03-21T10:53:07.525 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:53:07.525 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44313,"name":"cephfs.smithi174.yyimst","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6825/4127786797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":4127786797},{"type":"v1","addr":"172.21.15.174:6825","nonce":4127786797}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":44687,"name":"cephfs.smithi174.nhozag","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6827/2767517845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":2767517845},{"type":"v1","addr":"172.21.15.174:6827","nonce":2767517845}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":37,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T10:04:27.086576+0000","modified":"2024-03-21T10:50:32.694055+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":94,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34442,"mds_1":34474},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34442":{"gid":34442,"name":"cephfs.smithi153.zipckr","rank":0,"incarnation":34,"state":"up:resolve","state_seq":471,"addr":"172.21.15.153:6827/1103846295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":1103846295},{"type":"v1","addr":"172.21.15.153:6827","nonce":1103846295}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34474":{"gid":34474,"name":"cephfs.smithi153.iuyluj","rank":1,"incarnation":34,"state":"up:resolve","state_seq":465,"addr":"172.21.15.153:6829/1738630832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":1738630832},{"type":"v1","addr":"172.21.15.153:6829","nonce":1738630832}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-21T10:53:07.525 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 37 2024-03-21T10:53:07.527 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:07.532+0000 7f6c36200640 1 -- 172.21.15.153:0/1113826422 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f6c1406e490 msgr2=0x7f6c14070950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:53:07.527 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:07.532+0000 7f6c36200640 1 --2- 172.21.15.153:0/1113826422 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f6c1406e490 0x7f6c14070950 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f6c18036040 tx=0x7f6c18000bc0 comp rx=0 tx=0).stop 2024-03-21T10:53:07.528 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:07.532+0000 7f6c36200640 1 -- 172.21.15.153:0/1113826422 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6c30100270 msgr2=0x7f6c30197e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:53:07.528 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:07.532+0000 7f6c36200640 1 --2- 172.21.15.153:0/1113826422 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6c30100270 0x7f6c30197e60 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f6c2000ea30 tx=0x7f6c2000eef0 comp rx=0 tx=0).stop 2024-03-21T10:53:07.528 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:07.532+0000 7f6c36200640 1 -- 172.21.15.153:0/1113826422 shutdown_connections 2024-03-21T10:53:07.528 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:07.532+0000 7f6c36200640 1 --2- 172.21.15.153:0/1113826422 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f6c1406e490 0x7f6c14070950 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:07.528 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:07.532+0000 7f6c36200640 1 --2- 172.21.15.153:0/1113826422 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6c301983a0 0x7f6c3019c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:07.528 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:07.532+0000 7f6c36200640 1 --2- 172.21.15.153:0/1113826422 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f6c30100270 0x7f6c30197e60 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:07.528 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:07.532+0000 7f6c36200640 1 -- 172.21.15.153:0/1113826422 >> 172.21.15.153:0/1113826422 conn(0x7f6c300fbaf0 msgr2=0x7f6c300fd170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:53:07.528 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:07.532+0000 7f6c36200640 1 -- 172.21.15.153:0/1113826422 shutdown_connections 2024-03-21T10:53:07.528 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:07.532+0000 7f6c36200640 1 -- 172.21.15.153:0/1113826422 wait complete. 2024-03-21T10:53:07.549 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 38 2024-03-21T10:53:07.771 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:07.775+0000 7fbef7200640 1 -- 172.21.15.153:0/4081835747 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbef80fe290 msgr2=0x7fbef80fe710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:53:07.771 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:07.775+0000 7fbef7200640 1 --2- 172.21.15.153:0/4081835747 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbef80fe290 0x7fbef80fe710 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fbee4009880 tx=0x7fbee402bcf0 comp rx=0 tx=0).stop 2024-03-21T10:53:07.771 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:07.775+0000 7fbef7200640 1 -- 172.21.15.153:0/4081835747 shutdown_connections 2024-03-21T10:53:07.771 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:07.775+0000 7fbef7200640 1 --2- 172.21.15.153:0/4081835747 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbef80fe290 0x7fbef80fe710 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:07.771 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:07.775+0000 7fbef7200640 1 --2- 172.21.15.153:0/4081835747 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbef8104e80 0x7fbef8105260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:07.771 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:07.775+0000 7fbef7200640 1 -- 172.21.15.153:0/4081835747 >> 172.21.15.153:0/4081835747 conn(0x7fbef8075140 msgr2=0x7fbef80fc4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:53:07.771 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:07.775+0000 7fbef7200640 1 -- 172.21.15.153:0/4081835747 shutdown_connections 2024-03-21T10:53:07.771 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:07.775+0000 7fbef7200640 1 -- 172.21.15.153:0/4081835747 wait complete. 2024-03-21T10:53:07.773 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:07.776+0000 7fbef7200640 1 Processor -- start 2024-03-21T10:53:07.773 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:07.776+0000 7fbef7200640 1 -- start start 2024-03-21T10:53:07.773 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:07.777+0000 7fbef7200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbef80fe290 0x7fbef80ff470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:53:07.773 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:07.777+0000 7fbef7200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbef8104e80 0x7fbef80ff9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:53:07.773 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:07.777+0000 7fbef7200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbef80ffef0 con 0x7fbef8104e80 2024-03-21T10:53:07.773 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:07.777+0000 7fbef7200640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbef8100060 con 0x7fbef80fe290 2024-03-21T10:53:07.773 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:07.777+0000 7fbef5400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbef8104e80 0x7fbef80ff9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:53:07.773 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:07.777+0000 7fbef5400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbef8104e80 0x7fbef80ff9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42858/0 (socket says 172.21.15.153:42858) 2024-03-21T10:53:07.773 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:07.777+0000 7fbef5400640 1 -- 172.21.15.153:0/1533940177 learned_addr learned my addr 172.21.15.153:0/1533940177 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:53:07.773 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:07.777+0000 7fbef5e00640 1 --2- 172.21.15.153:0/1533940177 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbef80fe290 0x7fbef80ff470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:53:07.774 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:07.777+0000 7fbef5400640 1 -- 172.21.15.153:0/1533940177 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbef80fe290 msgr2=0x7fbef80ff470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:53:07.774 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:07.777+0000 7fbef5400640 1 --2- 172.21.15.153:0/1533940177 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbef80fe290 0x7fbef80ff470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:07.774 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:07.777+0000 7fbef5400640 1 -- 172.21.15.153:0/1533940177 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbee4009560 con 0x7fbef8104e80 2024-03-21T10:53:07.774 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:07.778+0000 7fbef5400640 1 --2- 172.21.15.153:0/1533940177 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbef8104e80 0x7fbef80ff9b0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fbee4036040 tx=0x7fbee402c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:53:07.775 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:07.778+0000 7fbeeea00640 1 -- 172.21.15.153:0/1533940177 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbee4004520 con 0x7fbef8104e80 2024-03-21T10:53:07.775 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:07.778+0000 7fbeeea00640 1 -- 172.21.15.153:0/1533940177 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbee4003ab0 con 0x7fbef8104e80 2024-03-21T10:53:07.775 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:07.778+0000 7fbef7200640 1 -- 172.21.15.153:0/1533940177 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbef8100280 con 0x7fbef8104e80 2024-03-21T10:53:07.775 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:07.778+0000 7fbeeea00640 1 -- 172.21.15.153:0/1533940177 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbee4034440 con 0x7fbef8104e80 2024-03-21T10:53:07.775 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:07.778+0000 7fbef7200640 1 -- 172.21.15.153:0/1533940177 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbef81a2110 con 0x7fbef8104e80 2024-03-21T10:53:07.775 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:07.779+0000 7fbef7200640 1 -- 172.21.15.153:0/1533940177 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbef804e040 con 0x7fbef8104e80 2024-03-21T10:53:07.776 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:07.781+0000 7fbeeea00640 1 -- 172.21.15.153:0/1533940177 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fbee4003c20 con 0x7fbef8104e80 2024-03-21T10:53:07.777 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:07.782+0000 7fbeeea00640 1 --2- 172.21.15.153:0/1533940177 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fbedc06e3d0 0x7fbedc070890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:53:07.778 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:07.782+0000 7fbeeea00640 1 -- 172.21.15.153:0/1533940177 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fbee40b0880 con 0x7fbef8104e80 2024-03-21T10:53:07.778 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:07.782+0000 7fbef5e00640 1 --2- 172.21.15.153:0/1533940177 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fbedc06e3d0 0x7fbedc070890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:53:07.778 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:07.783+0000 7fbef5e00640 1 --2- 172.21.15.153:0/1533940177 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fbedc06e3d0 0x7fbedc070890 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fbee8005fd0 tx=0x7fbee8005850 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:53:07.784 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:07.789+0000 7fbeeea00640 1 -- 172.21.15.153:0/1533940177 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fbee40798a0 con 0x7fbef8104e80 2024-03-21T10:53:07.801 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:53:07 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/1810872099' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 36, "format": "json"}]: dispatch 2024-03-21T10:53:07.802 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:53:07 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/1113826422' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 37, "format": "json"}]: dispatch 2024-03-21T10:53:08.026 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:08.031+0000 7fbef7200640 1 -- 172.21.15.153:0/1533940177 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 38, "format": "json"} v 0) v1 -- 0x7fbef8108360 con 0x7fbef8104e80 2024-03-21T10:53:08.027 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:08.032+0000 7fbeeea00640 1 -- 172.21.15.153:0/1533940177 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 38, "format": "json"}]=0 dumped fsmap epoch 38 v41) v1 ==== 107+0+5066 (secure 0 0 0) 0x7fbee4079020 con 0x7fbef8104e80 2024-03-21T10:53:08.028 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:53:08.028 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44313,"name":"cephfs.smithi174.yyimst","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6825/4127786797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":4127786797},{"type":"v1","addr":"172.21.15.174:6825","nonce":4127786797}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":44687,"name":"cephfs.smithi174.nhozag","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6827/2767517845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":2767517845},{"type":"v1","addr":"172.21.15.174:6827","nonce":2767517845}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":38,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T10:04:27.086576+0000","modified":"2024-03-21T10:50:33.608447+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":94,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34442,"mds_1":34474},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34442":{"gid":34442,"name":"cephfs.smithi153.zipckr","rank":0,"incarnation":34,"state":"up:reconnect","state_seq":472,"addr":"172.21.15.153:6827/1103846295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":1103846295},{"type":"v1","addr":"172.21.15.153:6827","nonce":1103846295}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34474":{"gid":34474,"name":"cephfs.smithi153.iuyluj","rank":1,"incarnation":34,"state":"up:reconnect","state_seq":466,"addr":"172.21.15.153:6829/1738630832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":1738630832},{"type":"v1","addr":"172.21.15.153:6829","nonce":1738630832}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-21T10:53:08.028 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 38 2024-03-21T10:53:08.030 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:08.035+0000 7fbef7200640 1 -- 172.21.15.153:0/1533940177 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fbedc06e3d0 msgr2=0x7fbedc070890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:53:08.031 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:08.035+0000 7fbef7200640 1 --2- 172.21.15.153:0/1533940177 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fbedc06e3d0 0x7fbedc070890 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fbee8005fd0 tx=0x7fbee8005850 comp rx=0 tx=0).stop 2024-03-21T10:53:08.031 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:08.035+0000 7fbef7200640 1 -- 172.21.15.153:0/1533940177 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbef8104e80 msgr2=0x7fbef80ff9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:53:08.031 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:08.035+0000 7fbef7200640 1 --2- 172.21.15.153:0/1533940177 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbef8104e80 0x7fbef80ff9b0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fbee4036040 tx=0x7fbee402c840 comp rx=0 tx=0).stop 2024-03-21T10:53:08.031 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:08.035+0000 7fbef7200640 1 -- 172.21.15.153:0/1533940177 shutdown_connections 2024-03-21T10:53:08.031 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:08.035+0000 7fbef7200640 1 --2- 172.21.15.153:0/1533940177 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fbedc06e3d0 0x7fbedc070890 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:08.031 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:08.035+0000 7fbef7200640 1 --2- 172.21.15.153:0/1533940177 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbef8104e80 0x7fbef80ff9b0 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:08.031 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:08.035+0000 7fbef7200640 1 --2- 172.21.15.153:0/1533940177 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbef80fe290 0x7fbef80ff470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:08.031 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:08.035+0000 7fbef7200640 1 -- 172.21.15.153:0/1533940177 >> 172.21.15.153:0/1533940177 conn(0x7fbef8075140 msgr2=0x7fbef8103970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:53:08.031 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:08.035+0000 7fbef7200640 1 -- 172.21.15.153:0/1533940177 shutdown_connections 2024-03-21T10:53:08.031 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:08.035+0000 7fbef7200640 1 -- 172.21.15.153:0/1533940177 wait complete. 2024-03-21T10:53:08.053 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 39 2024-03-21T10:53:08.196 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:53:07 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/1810872099' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 36, "format": "json"}]: dispatch 2024-03-21T10:53:08.196 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:53:07 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/1113826422' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 37, "format": "json"}]: dispatch 2024-03-21T10:53:08.246 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:08.250+0000 7f32f2200640 1 -- 172.21.15.153:0/1532984124 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32ec0ffb90 msgr2=0x7f32ec103bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:53:08.246 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:08.250+0000 7f32f2200640 1 --2- 172.21.15.153:0/1532984124 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32ec0ffb90 0x7f32ec103bd0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f32d8009880 tx=0x7f32d802bce0 comp rx=0 tx=0).stop 2024-03-21T10:53:08.246 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:08.250+0000 7f32f2200640 1 -- 172.21.15.153:0/1532984124 shutdown_connections 2024-03-21T10:53:08.246 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:08.250+0000 7f32f2200640 1 --2- 172.21.15.153:0/1532984124 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32ec0ffb90 0x7f32ec103bd0 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:08.246 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:08.250+0000 7f32f2200640 1 --2- 172.21.15.153:0/1532984124 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f32ec0ff1e0 0x7f32ec0ff5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:08.247 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:08.250+0000 7f32f2200640 1 -- 172.21.15.153:0/1532984124 >> 172.21.15.153:0/1532984124 conn(0x7f32ec0faa60 msgr2=0x7f32ec0fce80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:53:08.247 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:08.250+0000 7f32f2200640 1 -- 172.21.15.153:0/1532984124 shutdown_connections 2024-03-21T10:53:08.247 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:08.251+0000 7f32f2200640 1 -- 172.21.15.153:0/1532984124 wait complete. 2024-03-21T10:53:08.247 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:08.252+0000 7f32f2200640 1 Processor -- start 2024-03-21T10:53:08.247 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:08.252+0000 7f32f2200640 1 -- start start 2024-03-21T10:53:08.249 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:08.253+0000 7f32f2200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32ec0ff1e0 0x7f32ec1960b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:53:08.249 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:08.253+0000 7f32f2200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f32ec1965f0 0x7f32ec19aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:53:08.249 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:08.253+0000 7f32f2200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32ec196bf0 con 0x7f32ec0ff1e0 2024-03-21T10:53:08.249 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:08.253+0000 7f32f2200640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32ec196d60 con 0x7f32ec1965f0 2024-03-21T10:53:08.249 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:08.253+0000 7f32eb400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32ec0ff1e0 0x7f32ec1960b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:53:08.249 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:08.253+0000 7f32eb400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32ec0ff1e0 0x7f32ec1960b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42884/0 (socket says 172.21.15.153:42884) 2024-03-21T10:53:08.249 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:08.253+0000 7f32eb400640 1 -- 172.21.15.153:0/604415582 learned_addr learned my addr 172.21.15.153:0/604415582 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:53:08.249 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:08.253+0000 7f32eaa00640 1 --2- 172.21.15.153:0/604415582 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f32ec1965f0 0x7f32ec19aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:53:08.249 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:08.253+0000 7f32eb400640 1 -- 172.21.15.153:0/604415582 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f32ec1965f0 msgr2=0x7f32ec19aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:53:08.249 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:08.253+0000 7f32eb400640 1 --2- 172.21.15.153:0/604415582 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f32ec1965f0 0x7f32ec19aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:08.249 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:08.253+0000 7f32eb400640 1 -- 172.21.15.153:0/604415582 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32d8009560 con 0x7f32ec0ff1e0 2024-03-21T10:53:08.250 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:08.254+0000 7f32eb400640 1 --2- 172.21.15.153:0/604415582 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32ec0ff1e0 0x7f32ec1960b0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f32dc00d720 tx=0x7f32dc00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:53:08.250 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:08.254+0000 7f32e3e00640 1 -- 172.21.15.153:0/604415582 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f32dc0051f0 con 0x7f32ec0ff1e0 2024-03-21T10:53:08.250 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:08.254+0000 7f32f2200640 1 -- 172.21.15.153:0/604415582 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32ec19b050 con 0x7f32ec0ff1e0 2024-03-21T10:53:08.250 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:08.254+0000 7f32e3e00640 1 -- 172.21.15.153:0/604415582 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f32dc009ce0 con 0x7f32ec0ff1e0 2024-03-21T10:53:08.250 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:08.254+0000 7f32e3e00640 1 -- 172.21.15.153:0/604415582 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f32dc010460 con 0x7f32ec0ff1e0 2024-03-21T10:53:08.251 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:08.254+0000 7f32f2200640 1 -- 172.21.15.153:0/604415582 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32ec19b5b0 con 0x7f32ec0ff1e0 2024-03-21T10:53:08.251 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:08.255+0000 7f32f2200640 1 -- 172.21.15.153:0/604415582 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32ec04e040 con 0x7f32ec0ff1e0 2024-03-21T10:53:08.252 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:08.257+0000 7f32e3e00640 1 -- 172.21.15.153:0/604415582 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f32dc005350 con 0x7f32ec0ff1e0 2024-03-21T10:53:08.253 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:08.257+0000 7f32e3e00640 1 --2- 172.21.15.153:0/604415582 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f32d006e3d0 0x7f32d0070890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:53:08.253 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:08.258+0000 7f32e3e00640 1 -- 172.21.15.153:0/604415582 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f32dc08e060 con 0x7f32ec0ff1e0 2024-03-21T10:53:08.253 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:08.258+0000 7f32eaa00640 1 --2- 172.21.15.153:0/604415582 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f32d006e3d0 0x7f32d0070890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:53:08.254 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:08.259+0000 7f32eaa00640 1 --2- 172.21.15.153:0/604415582 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f32d006e3d0 0x7f32d0070890 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f32d8036040 tx=0x7f32d802cf90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:53:08.262 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:08.267+0000 7f32e3e00640 1 -- 172.21.15.153:0/604415582 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f32dc057120 con 0x7f32ec0ff1e0 2024-03-21T10:53:08.550 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:08.555+0000 7f32f2200640 1 -- 172.21.15.153:0/604415582 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 39, "format": "json"} v 0) v1 -- 0x7f32ec197490 con 0x7f32ec0ff1e0 2024-03-21T10:53:08.551 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:08.555+0000 7f32e3e00640 1 -- 172.21.15.153:0/604415582 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 39, "format": "json"}]=0 dumped fsmap epoch 39 v41) v1 ==== 107+0+5060 (secure 0 0 0) 0x7f32dc05a7e0 con 0x7f32ec0ff1e0 2024-03-21T10:53:08.551 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:53:08.551 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44313,"name":"cephfs.smithi174.yyimst","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6825/4127786797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":4127786797},{"type":"v1","addr":"172.21.15.174:6825","nonce":4127786797}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":44687,"name":"cephfs.smithi174.nhozag","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6827/2767517845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":2767517845},{"type":"v1","addr":"172.21.15.174:6827","nonce":2767517845}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T10:04:27.086576+0000","modified":"2024-03-21T10:50:34.559312+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":94,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34442,"mds_1":34474},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34442":{"gid":34442,"name":"cephfs.smithi153.zipckr","rank":0,"incarnation":34,"state":"up:rejoin","state_seq":473,"addr":"172.21.15.153:6827/1103846295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":1103846295},{"type":"v1","addr":"172.21.15.153:6827","nonce":1103846295}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34474":{"gid":34474,"name":"cephfs.smithi153.iuyluj","rank":1,"incarnation":34,"state":"up:rejoin","state_seq":467,"addr":"172.21.15.153:6829/1738630832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":1738630832},{"type":"v1","addr":"172.21.15.153:6829","nonce":1738630832}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-21T10:53:08.551 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 39 2024-03-21T10:53:08.554 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:08.558+0000 7f32f2200640 1 -- 172.21.15.153:0/604415582 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f32d006e3d0 msgr2=0x7f32d0070890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:53:08.554 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:08.558+0000 7f32f2200640 1 --2- 172.21.15.153:0/604415582 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f32d006e3d0 0x7f32d0070890 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f32d8036040 tx=0x7f32d802cf90 comp rx=0 tx=0).stop 2024-03-21T10:53:08.554 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:08.558+0000 7f32f2200640 1 -- 172.21.15.153:0/604415582 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32ec0ff1e0 msgr2=0x7f32ec1960b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:53:08.554 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:08.558+0000 7f32f2200640 1 --2- 172.21.15.153:0/604415582 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32ec0ff1e0 0x7f32ec1960b0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f32dc00d720 tx=0x7f32dc00dbe0 comp rx=0 tx=0).stop 2024-03-21T10:53:08.554 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:08.558+0000 7f32f2200640 1 -- 172.21.15.153:0/604415582 shutdown_connections 2024-03-21T10:53:08.554 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:08.558+0000 7f32f2200640 1 --2- 172.21.15.153:0/604415582 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f32d006e3d0 0x7f32d0070890 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:08.554 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:08.558+0000 7f32f2200640 1 --2- 172.21.15.153:0/604415582 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f32ec1965f0 0x7f32ec19aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:08.554 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:08.558+0000 7f32f2200640 1 --2- 172.21.15.153:0/604415582 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f32ec0ff1e0 0x7f32ec1960b0 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:08.554 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:08.558+0000 7f32f2200640 1 -- 172.21.15.153:0/604415582 >> 172.21.15.153:0/604415582 conn(0x7f32ec0faa60 msgr2=0x7f32ec0fc110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:53:08.554 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:08.558+0000 7f32f2200640 1 -- 172.21.15.153:0/604415582 shutdown_connections 2024-03-21T10:53:08.554 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:08.558+0000 7f32f2200640 1 -- 172.21.15.153:0/604415582 wait complete. 2024-03-21T10:53:08.575 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 40 2024-03-21T10:53:08.795 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:08.799+0000 7f0408000640 1 -- 172.21.15.153:0/4258995355 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0400101b20 msgr2=0x7f040010a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:53:08.795 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:08.799+0000 7f0408000640 1 --2- 172.21.15.153:0/4258995355 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0400101b20 0x7f040010a000 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f03f4009880 tx=0x7f03f402bd70 comp rx=0 tx=0).stop 2024-03-21T10:53:08.795 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:08.799+0000 7f0408000640 1 -- 172.21.15.153:0/4258995355 shutdown_connections 2024-03-21T10:53:08.795 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:08.799+0000 7f0408000640 1 --2- 172.21.15.153:0/4258995355 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0400101b20 0x7f040010a000 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:08.795 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:08.799+0000 7f0408000640 1 --2- 172.21.15.153:0/4258995355 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0400101200 0x7f04001015e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:08.795 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:08.799+0000 7f0408000640 1 -- 172.21.15.153:0/4258995355 >> 172.21.15.153:0/4258995355 conn(0x7f04000faa20 msgr2=0x7f04000fce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:53:08.795 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:08.799+0000 7f0408000640 1 -- 172.21.15.153:0/4258995355 shutdown_connections 2024-03-21T10:53:08.795 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:08.799+0000 7f0408000640 1 -- 172.21.15.153:0/4258995355 wait complete. 2024-03-21T10:53:08.797 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:08.800+0000 7f0408000640 1 Processor -- start 2024-03-21T10:53:08.797 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:08.800+0000 7f0408000640 1 -- start start 2024-03-21T10:53:08.797 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:08.800+0000 7f0408000640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0400101200 0x7f0400195d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:53:08.797 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:08.801+0000 7f0408000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0400101b20 0x7f0400196290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:53:08.797 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:08.801+0000 7f0408000640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0400196920 con 0x7f0400101200 2024-03-21T10:53:08.797 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:08.801+0000 7f0408000640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f040019a6f0 con 0x7f0400101b20 2024-03-21T10:53:08.798 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:08.801+0000 7f0405600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0400101200 0x7f0400195d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:53:08.798 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:08.801+0000 7f0405600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0400101200 0x7f0400195d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42902/0 (socket says 172.21.15.153:42902) 2024-03-21T10:53:08.798 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:08.801+0000 7f0405600640 1 -- 172.21.15.153:0/279744923 learned_addr learned my addr 172.21.15.153:0/279744923 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:53:08.798 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:08.801+0000 7f0405600640 1 -- 172.21.15.153:0/279744923 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0400101b20 msgr2=0x7f0400196290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:53:08.798 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:08.801+0000 7f0405600640 1 --2- 172.21.15.153:0/279744923 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0400101b20 0x7f0400196290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:08.798 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:08.801+0000 7f0405600640 1 -- 172.21.15.153:0/279744923 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03f4009560 con 0x7f0400101200 2024-03-21T10:53:08.798 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:08.801+0000 7f0405600640 1 --2- 172.21.15.153:0/279744923 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0400101200 0x7f0400195d50 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f03ec00d6f0 tx=0x7f03ec00dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:53:08.798 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:08.802+0000 7f03fe000640 1 -- 172.21.15.153:0/279744923 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f03ec009a30 con 0x7f0400101200 2024-03-21T10:53:08.798 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:08.802+0000 7f03fe000640 1 -- 172.21.15.153:0/279744923 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f03ec0054e0 con 0x7f0400101200 2024-03-21T10:53:08.798 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:08.802+0000 7f0408000640 1 -- 172.21.15.153:0/279744923 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f040019a9d0 con 0x7f0400101200 2024-03-21T10:53:08.798 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:08.802+0000 7f03fe000640 1 -- 172.21.15.153:0/279744923 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f03ec010460 con 0x7f0400101200 2024-03-21T10:53:08.798 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:08.802+0000 7f0408000640 1 -- 172.21.15.153:0/279744923 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f040019af30 con 0x7f0400101200 2024-03-21T10:53:08.799 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:08.803+0000 7f0408000640 1 -- 172.21.15.153:0/279744923 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f040004e040 con 0x7f0400101200 2024-03-21T10:53:08.799 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:08.804+0000 7f03fe000640 1 -- 172.21.15.153:0/279744923 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f03ec009b90 con 0x7f0400101200 2024-03-21T10:53:08.801 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:08.805+0000 7f03fe000640 1 --2- 172.21.15.153:0/279744923 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f03e806e3d0 0x7f03e8070890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:53:08.801 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:08.805+0000 7f03fe000640 1 -- 172.21.15.153:0/279744923 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f03ec08e140 con 0x7f0400101200 2024-03-21T10:53:08.801 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:08.806+0000 7f0404c00640 1 --2- 172.21.15.153:0/279744923 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f03e806e3d0 0x7f03e8070890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:53:08.802 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:08.806+0000 7f0404c00640 1 --2- 172.21.15.153:0/279744923 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f03e806e3d0 0x7f03e8070890 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f03f4036040 tx=0x7f03f4000bc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:53:08.808 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:08.813+0000 7f03fe000640 1 -- 172.21.15.153:0/279744923 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f03ec057160 con 0x7f0400101200 2024-03-21T10:53:09.028 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:53:08 smithi174 ceph-mon[88004]: pgmap v1241: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:53:09.028 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:53:08 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/1533940177' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 38, "format": "json"}]: dispatch 2024-03-21T10:53:09.028 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:53:08 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/604415582' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 39, "format": "json"}]: dispatch 2024-03-21T10:53:09.049 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:09.053+0000 7f0408000640 1 -- 172.21.15.153:0/279744923 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 40, "format": "json"} v 0) v1 -- 0x7f0400107670 con 0x7f0400101200 2024-03-21T10:53:09.050 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:09.054+0000 7f03fe000640 1 -- 172.21.15.153:0/279744923 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 40, "format": "json"}]=0 dumped fsmap epoch 40 v41) v1 ==== 107+0+5090 (secure 0 0 0) 0x7f03ec05a820 con 0x7f0400101200 2024-03-21T10:53:09.050 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:53:09.050 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":40,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T10:04:27.086576+0000","modified":"2024-03-21T10:50:36.318024+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":94,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34442,"mds_1":34474},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34442":{"gid":34442,"name":"cephfs.smithi153.zipckr","rank":0,"incarnation":34,"state":"up:active","state_seq":474,"addr":"172.21.15.153:6827/1103846295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":1103846295},{"type":"v1","addr":"172.21.15.153:6827","nonce":1103846295}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34474":{"gid":34474,"name":"cephfs.smithi153.iuyluj","rank":1,"incarnation":34,"state":"up:active","state_seq":468,"addr":"172.21.15.153:6829/1738630832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":1738630832},{"type":"v1","addr":"172.21.15.153:6829","nonce":1738630832}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_44313":{"gid":44313,"name":"cephfs.smithi174.yyimst","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6825/4127786797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":4127786797},{"type":"v1","addr":"172.21.15.174:6825","nonce":4127786797}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_44687":{"gid":44687,"name":"cephfs.smithi174.nhozag","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6827/2767517845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":2767517845},{"type":"v1","addr":"172.21.15.174:6827","nonce":2767517845}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34442,"qdb_cluster":[34442,34474]},"id":1}]} 2024-03-21T10:53:09.050 INFO:teuthology.orchestra.run.smithi153.stderr:dumped fsmap epoch 40 2024-03-21T10:53:09.051 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:53:08 smithi153 ceph-mon[113733]: pgmap v1241: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:53:09.051 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:53:08 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/1533940177' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 38, "format": "json"}]: dispatch 2024-03-21T10:53:09.051 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:53:08 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/604415582' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 39, "format": "json"}]: dispatch 2024-03-21T10:53:09.052 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:09.057+0000 7f0408000640 1 -- 172.21.15.153:0/279744923 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f03e806e3d0 msgr2=0x7f03e8070890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:53:09.052 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:09.057+0000 7f0408000640 1 --2- 172.21.15.153:0/279744923 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f03e806e3d0 0x7f03e8070890 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f03f4036040 tx=0x7f03f4000bc0 comp rx=0 tx=0).stop 2024-03-21T10:53:09.052 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:09.057+0000 7f0408000640 1 -- 172.21.15.153:0/279744923 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0400101200 msgr2=0x7f0400195d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:53:09.052 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:09.057+0000 7f0408000640 1 --2- 172.21.15.153:0/279744923 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0400101200 0x7f0400195d50 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f03ec00d6f0 tx=0x7f03ec00dbb0 comp rx=0 tx=0).stop 2024-03-21T10:53:09.053 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:09.057+0000 7f0408000640 1 -- 172.21.15.153:0/279744923 shutdown_connections 2024-03-21T10:53:09.053 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:09.057+0000 7f0408000640 1 --2- 172.21.15.153:0/279744923 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f03e806e3d0 0x7f03e8070890 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:09.053 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:09.057+0000 7f0408000640 1 --2- 172.21.15.153:0/279744923 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0400101b20 0x7f0400196290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:09.053 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:09.057+0000 7f0408000640 1 --2- 172.21.15.153:0/279744923 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f0400101200 0x7f0400195d50 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:09.053 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:09.057+0000 7f0408000640 1 -- 172.21.15.153:0/279744923 >> 172.21.15.153:0/279744923 conn(0x7f04000faa20 msgr2=0x7f0400104840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:53:09.053 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:09.057+0000 7f0408000640 1 -- 172.21.15.153:0/279744923 shutdown_connections 2024-03-21T10:53:09.053 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:09.057+0000 7f0408000640 1 -- 172.21.15.153:0/279744923 wait complete. 2024-03-21T10:53:09.070 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-03-21T10:53:09.081 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-03-21T10:53:09.082 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-03-21T10:53:09.082 DEBUG:teuthology.orchestra.run.smithi153:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-21T10:53:09.125 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-03-21T10:53:09.125 DEBUG:teuthology.orchestra.run.smithi153:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-21T10:53:09.208 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T10:53:09.453 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:09.457+0000 7fbd65800640 1 -- 172.21.15.153:0/1985799767 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbd60101100 msgr2=0x7fbd601014e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:53:09.453 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:09.457+0000 7fbd65800640 1 --2- 172.21.15.153:0/1985799767 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbd60101100 0x7fbd601014e0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fbd500098b0 tx=0x7fbd5002bdf0 comp rx=0 tx=0).stop 2024-03-21T10:53:09.453 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:09.457+0000 7fbd65800640 1 -- 172.21.15.153:0/1985799767 shutdown_connections 2024-03-21T10:53:09.453 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:09.457+0000 7fbd65800640 1 --2- 172.21.15.153:0/1985799767 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbd60101a20 0x7fbd60109f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:09.453 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:09.457+0000 7fbd65800640 1 --2- 172.21.15.153:0/1985799767 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbd60101100 0x7fbd601014e0 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:09.453 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:09.457+0000 7fbd65800640 1 -- 172.21.15.153:0/1985799767 >> 172.21.15.153:0/1985799767 conn(0x7fbd600fadb0 msgr2=0x7fbd600fd1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:53:09.453 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:09.457+0000 7fbd65800640 1 -- 172.21.15.153:0/1985799767 shutdown_connections 2024-03-21T10:53:09.453 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:09.457+0000 7fbd65800640 1 -- 172.21.15.153:0/1985799767 wait complete. 2024-03-21T10:53:09.455 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:09.458+0000 7fbd65800640 1 Processor -- start 2024-03-21T10:53:09.455 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:09.458+0000 7fbd65800640 1 -- start start 2024-03-21T10:53:09.455 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:09.459+0000 7fbd65800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbd60101100 0x7fbd600fed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:53:09.455 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:09.459+0000 7fbd65800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbd60101a20 0x7fbd600ff290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:53:09.455 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:09.459+0000 7fbd65800640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd600ff7d0 con 0x7fbd60101100 2024-03-21T10:53:09.455 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:09.459+0000 7fbd65800640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd600ff940 con 0x7fbd60101a20 2024-03-21T10:53:09.455 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:09.459+0000 7fbd5fe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbd60101100 0x7fbd600fed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:53:09.456 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:09.459+0000 7fbd5fe00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbd60101100 0x7fbd600fed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42916/0 (socket says 172.21.15.153:42916) 2024-03-21T10:53:09.456 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:09.459+0000 7fbd5fe00640 1 -- 172.21.15.153:0/3059685483 learned_addr learned my addr 172.21.15.153:0/3059685483 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:53:09.456 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:09.459+0000 7fbd5f400640 1 --2- 172.21.15.153:0/3059685483 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbd60101a20 0x7fbd600ff290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:53:09.456 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:09.459+0000 7fbd5fe00640 1 -- 172.21.15.153:0/3059685483 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbd60101a20 msgr2=0x7fbd600ff290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:53:09.456 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:09.459+0000 7fbd5fe00640 1 --2- 172.21.15.153:0/3059685483 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbd60101a20 0x7fbd600ff290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:09.456 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:09.459+0000 7fbd5fe00640 1 -- 172.21.15.153:0/3059685483 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd50009560 con 0x7fbd60101100 2024-03-21T10:53:09.456 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:09.459+0000 7fbd5f400640 1 --2- 172.21.15.153:0/3059685483 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbd60101a20 0x7fbd600ff290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:53:09.456 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:09.460+0000 7fbd5fe00640 1 --2- 172.21.15.153:0/3059685483 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbd60101100 0x7fbd600fed50 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fbd50009880 tx=0x7fbd5002c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:53:09.457 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:09.460+0000 7fbd5cc00640 1 -- 172.21.15.153:0/3059685483 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbd500035f0 con 0x7fbd60101100 2024-03-21T10:53:09.457 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:09.460+0000 7fbd65800640 1 -- 172.21.15.153:0/3059685483 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd600ffbc0 con 0x7fbd60101100 2024-03-21T10:53:09.457 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:09.460+0000 7fbd5cc00640 1 -- 172.21.15.153:0/3059685483 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbd50003c10 con 0x7fbd60101100 2024-03-21T10:53:09.457 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:09.460+0000 7fbd5cc00640 1 -- 172.21.15.153:0/3059685483 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbd50005600 con 0x7fbd60101100 2024-03-21T10:53:09.457 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:09.460+0000 7fbd65800640 1 -- 172.21.15.153:0/3059685483 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd601a1ff0 con 0x7fbd60101100 2024-03-21T10:53:09.457 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:09.462+0000 7fbd65800640 1 -- 172.21.15.153:0/3059685483 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd6004e890 con 0x7fbd60101100 2024-03-21T10:53:09.458 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:09.463+0000 7fbd5cc00640 1 -- 172.21.15.153:0/3059685483 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fbd50004dd0 con 0x7fbd60101100 2024-03-21T10:53:09.459 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:09.463+0000 7fbd5cc00640 1 --2- 172.21.15.153:0/3059685483 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fbd4406e490 0x7fbd44070950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:53:09.459 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:09.464+0000 7fbd5cc00640 1 -- 172.21.15.153:0/3059685483 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fbd500b0f80 con 0x7fbd60101100 2024-03-21T10:53:09.459 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:09.464+0000 7fbd5f400640 1 --2- 172.21.15.153:0/3059685483 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fbd4406e490 0x7fbd44070950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:53:09.460 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:09.465+0000 7fbd5f400640 1 --2- 172.21.15.153:0/3059685483 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fbd4406e490 0x7fbd44070950 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fbd601004f0 tx=0x7fbd4c005f90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:53:09.463 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:09.468+0000 7fbd5cc00640 1 -- 172.21.15.153:0/3059685483 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fbd5007a160 con 0x7fbd60101100 2024-03-21T10:53:09.724 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:09.729+0000 7fbd65800640 1 -- 172.21.15.153:0/3059685483 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbd60107570 con 0x7fbd60101100 2024-03-21T10:53:09.726 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:09.731+0000 7fbd5cc00640 1 -- 172.21.15.153:0/3059685483 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) v1 ==== 74+0+17231 (secure 0 0 0) 0x7fbd5007d820 con 0x7fbd60101100 2024-03-21T10:53:09.727 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:53:09.727 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":98,"fsid":"070ed3e4-e769-11ee-95cd-87774f69a715","created":"2024-03-21T09:55:19.360240+0000","modified":"2024-03-21T10:35:13.153133+0000","last_up_change":"2024-03-21T10:18:57.856952+0000","last_in_change":"2024-03-21T10:02:32.306573+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T10:01:18.943639+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-03-21T10:04:25.621358+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.309999942779541,"score_stable":1.309999942779541,"optimal_score":1,"raw_score_acting":1.309999942779541,"raw_score_stable":1.309999942779541,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-03-21T10:04:26.593674+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0f2e1224-6f45-4f23-b97a-5b89319a90f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":50,"up_from":61,"up_thru":90,"down_at":51,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":2525147935},{"type":"v1","addr":"172.21.15.153:6803","nonce":2525147935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":2525147935},{"type":"v1","addr":"172.21.15.153:6805","nonce":2525147935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":2525147935},{"type":"v1","addr":"172.21.15.153:6809","nonce":2525147935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":2525147935},{"type":"v1","addr":"172.21.15.153:6807","nonce":2525147935}]},"public_addr":"172.21.15.153:6803/2525147935","cluster_addr":"172.21.15.153:6805/2525147935","heartbeat_back_addr":"172.21.15.153:6809/2525147935","heartbeat_front_addr":"172.21.15.153:6807/2525147935","state":["exists","up"]},{"osd":1,"uuid":"76195edb-aa7a-4627-a211-09a578de65c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":66,"up_from":70,"up_thru":90,"down_at":67,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":1529835318},{"type":"v1","addr":"172.21.15.153:6811","nonce":1529835318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":1529835318},{"type":"v1","addr":"172.21.15.153:6813","nonce":1529835318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":1529835318},{"type":"v1","addr":"172.21.15.153:6817","nonce":1529835318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":1529835318},{"type":"v1","addr":"172.21.15.153:6815","nonce":1529835318}]},"public_addr":"172.21.15.153:6811/1529835318","cluster_addr":"172.21.15.153:6813/1529835318","heartbeat_back_addr":"172.21.15.153:6817/1529835318","heartbeat_front_addr":"172.21.15.153:6815/1529835318","state":["exists","up"]},{"osd":2,"uuid":"3ec17cf8-f0ea-45c4-b084-b304024d6e92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":71,"up_from":75,"up_thru":90,"down_at":72,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":292763575},{"type":"v1","addr":"172.21.15.153:6819","nonce":292763575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":292763575},{"type":"v1","addr":"172.21.15.153:6821","nonce":292763575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":292763575},{"type":"v1","addr":"172.21.15.153:6825","nonce":292763575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":292763575},{"type":"v1","addr":"172.21.15.153:6823","nonce":292763575}]},"public_addr":"172.21.15.153:6819/292763575","cluster_addr":"172.21.15.153:6821/292763575","heartbeat_back_addr":"172.21.15.153:6825/292763575","heartbeat_front_addr":"172.21.15.153:6823/292763575","state":["exists","up"]},{"osd":3,"uuid":"285146f6-6209-47ee-9e5b-2e075c949231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":27,"last_clean_end":76,"up_from":80,"up_thru":90,"down_at":77,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1830370519},{"type":"v1","addr":"172.21.15.174:6801","nonce":1830370519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1830370519},{"type":"v1","addr":"172.21.15.174:6803","nonce":1830370519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1830370519},{"type":"v1","addr":"172.21.15.174:6807","nonce":1830370519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":1830370519},{"type":"v1","addr":"172.21.15.174:6805","nonce":1830370519}]},"public_addr":"172.21.15.174:6801/1830370519","cluster_addr":"172.21.15.174:6803/1830370519","heartbeat_back_addr":"172.21.15.174:6807/1830370519","heartbeat_front_addr":"172.21.15.174:6805/1830370519","state":["exists","up"]},{"osd":4,"uuid":"f05cec2d-c649-489d-98a3-f24a75fe1f95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":35,"last_clean_end":81,"up_from":85,"up_thru":90,"down_at":82,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1143789116},{"type":"v1","addr":"172.21.15.174:6809","nonce":1143789116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1143789116},{"type":"v1","addr":"172.21.15.174:6811","nonce":1143789116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1143789116},{"type":"v1","addr":"172.21.15.174:6815","nonce":1143789116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1143789116},{"type":"v1","addr":"172.21.15.174:6813","nonce":1143789116}]},"public_addr":"172.21.15.174:6809/1143789116","cluster_addr":"172.21.15.174:6811/1143789116","heartbeat_back_addr":"172.21.15.174:6815/1143789116","heartbeat_front_addr":"172.21.15.174:6813/1143789116","state":["exists","up"]},{"osd":5,"uuid":"f430a3b2-e1b5-4698-9089-812fcf3a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":40,"last_clean_end":86,"up_from":90,"up_thru":90,"down_at":87,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":385015973},{"type":"v1","addr":"172.21.15.174:6817","nonce":385015973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":385015973},{"type":"v1","addr":"172.21.15.174:6819","nonce":385015973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":385015973},{"type":"v1","addr":"172.21.15.174:6823","nonce":385015973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":385015973},{"type":"v1","addr":"172.21.15.174:6821","nonce":385015973}]},"public_addr":"172.21.15.174:6817/385015973","cluster_addr":"172.21.15.174:6819/385015973","heartbeat_back_addr":"172.21.15.174:6823/385015973","heartbeat_front_addr":"172.21.15.174:6821/385015973","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:00:04.356688+0000","dead_epoch":50},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:00:39.988638+0000","dead_epoch":66},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:01:14.882381+0000","dead_epoch":71},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:01:48.072468+0000","dead_epoch":76},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:02:21.211701+0000","dead_epoch":81},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:02:54.161138+0000","dead_epoch":86}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":3,"to":2}]},{"pgid":"2.14","mappings":[{"from":3,"to":0}]},{"pgid":"2.16","mappings":[{"from":3,"to":0}]},{"pgid":"3.7","mappings":[{"from":1,"to":0}]},{"pgid":"3.b","mappings":[{"from":1,"to":2},{"from":4,"to":5}]},{"pgid":"3.15","mappings":[{"from":3,"to":2}]},{"pgid":"3.19","mappings":[{"from":3,"to":5}]},{"pgid":"3.1c","mappings":[{"from":4,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.174:6825/1776947618":"2024-03-22T10:19:05.021561+0000","172.21.15.174:6824/1776947618":"2024-03-22T10:19:05.021561+0000","172.21.15.153:6829/866138999":"2024-03-22T10:19:05.021528+0000","172.21.15.153:0/3553873361":"2024-03-22T10:13:31.087809+0000","172.21.15.174:0/1056981865":"2024-03-22T10:08:33.113521+0000","172.21.15.153:0/2407536999":"2024-03-22T09:56:00.849533+0000","172.21.15.153:0/2969012027":"2024-03-22T10:07:30.531722+0000","172.21.15.153:6826/1353876997":"2024-03-22T10:04:45.049156+0000","172.21.15.153:0/623594592":"2024-03-22T09:56:00.849533+0000","172.21.15.153:0/1163205431":"2024-03-22T09:56:43.480866+0000","172.21.15.153:0/3278686288":"2024-03-22T09:58:24.250138+0000","172.21.15.153:0/652104290":"2024-03-22T09:58:24.250138+0000","172.21.15.153:6800/2524145044":"2024-03-22T10:13:31.087809+0000","172.21.15.174:0/3456690749":"2024-03-22T10:08:33.113521+0000","172.21.15.153:0/4237751158":"2024-03-22T10:07:30.531722+0000","172.21.15.153:0/3456116624":"2024-03-22T09:56:43.480866+0000","172.21.15.153:0/3342434133":"2024-03-22T10:13:31.087809+0000","172.21.15.153:0/2801644471":"2024-03-22T10:07:30.531722+0000","172.21.15.153:6801/1639929315":"2024-03-22T10:07:30.531722+0000","172.21.15.174:0/950700094":"2024-03-22T10:08:33.113521+0000","172.21.15.153:6827/1353876997":"2024-03-22T10:04:45.049156+0000","172.21.15.174:0/2227377986":"2024-03-22T10:08:33.113521+0000","172.21.15.153:6800/1143522828":"2024-03-22T09:58:24.250138+0000","172.21.15.153:0/3435678189":"2024-03-22T09:56:00.849533+0000","172.21.15.153:0/2431412876":"2024-03-22T10:07:30.531722+0000","172.21.15.153:6801/1143522828":"2024-03-22T09:58:24.250138+0000","172.21.15.174:6829/1155541738":"2024-03-22T10:08:33.113521+0000","172.21.15.153:0/517714023":"2024-03-22T10:13:31.087809+0000","172.21.15.174:0/828964242":"2024-03-22T10:08:33.113521+0000","172.21.15.153:0/405164318":"2024-03-22T09:58:24.250138+0000","172.21.15.153:0/816489762":"2024-03-22T09:56:43.480866+0000","172.21.15.153:0/2445011785":"2024-03-22T10:13:31.087809+0000","172.21.15.153:6800/3178125243":"2024-03-22T09:56:43.480866+0000","172.21.15.174:6828/1155541738":"2024-03-22T10:08:33.113521+0000","172.21.15.153:6801/3178125243":"2024-03-22T09:56:43.480866+0000","172.21.15.153:6828/866138999":"2024-03-22T10:19:05.021528+0000","172.21.15.153:6800/270837948":"2024-03-22T09:56:00.849533+0000","172.21.15.153:6801/270837948":"2024-03-22T09:56:00.849533+0000","172.21.15.174:0/2577665242":"2024-03-22T10:08:33.113521+0000","172.21.15.153:6800/1639929315":"2024-03-22T10:07:30.531722+0000","172.21.15.153:6801/2524145044":"2024-03-22T10:13:31.087809+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T10:53:09.729 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:09.734+0000 7fbd65800640 1 -- 172.21.15.153:0/3059685483 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fbd4406e490 msgr2=0x7fbd44070950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:53:09.729 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:09.734+0000 7fbd65800640 1 --2- 172.21.15.153:0/3059685483 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fbd4406e490 0x7fbd44070950 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fbd601004f0 tx=0x7fbd4c005f90 comp rx=0 tx=0).stop 2024-03-21T10:53:09.729 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:09.734+0000 7fbd65800640 1 -- 172.21.15.153:0/3059685483 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbd60101100 msgr2=0x7fbd600fed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:53:09.729 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:09.734+0000 7fbd65800640 1 --2- 172.21.15.153:0/3059685483 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbd60101100 0x7fbd600fed50 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fbd50009880 tx=0x7fbd5002c8d0 comp rx=0 tx=0).stop 2024-03-21T10:53:09.730 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:09.734+0000 7fbd65800640 1 -- 172.21.15.153:0/3059685483 shutdown_connections 2024-03-21T10:53:09.730 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:09.734+0000 7fbd65800640 1 --2- 172.21.15.153:0/3059685483 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fbd4406e490 0x7fbd44070950 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:09.730 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:09.734+0000 7fbd65800640 1 --2- 172.21.15.153:0/3059685483 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbd60101a20 0x7fbd600ff290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:09.730 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:09.734+0000 7fbd65800640 1 --2- 172.21.15.153:0/3059685483 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fbd60101100 0x7fbd600fed50 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:09.731 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:09.734+0000 7fbd65800640 1 -- 172.21.15.153:0/3059685483 >> 172.21.15.153:0/3059685483 conn(0x7fbd600fadb0 msgr2=0x7fbd60104740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:53:09.731 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:09.734+0000 7fbd65800640 1 -- 172.21.15.153:0/3059685483 shutdown_connections 2024-03-21T10:53:09.731 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:09.734+0000 7fbd65800640 1 -- 172.21.15.153:0/3059685483 wait complete. 2024-03-21T10:53:09.750 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-21T10:01:18.943639+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-03-21T10:04:25.621358+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.309999942779541, 'score_stable': 1.309999942779541, 'optimal_score': 1, 'raw_score_acting': 1.309999942779541, 'raw_score_stable': 1.309999942779541, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-03-21T10:04:26.593674+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-21T10:53:09.750 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-21T10:53:09.962 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:09.967+0000 7f4eccc00640 1 -- 172.21.15.153:0/1234094355 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4ec8102e50 msgr2=0x7f4ec8106e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:53:09.963 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:09.967+0000 7f4eccc00640 1 --2- 172.21.15.153:0/1234094355 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4ec8102e50 0x7f4ec8106e90 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f4eb4009880 tx=0x7f4eb402bce0 comp rx=0 tx=0).stop 2024-03-21T10:53:09.963 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:09.967+0000 7f4eccc00640 1 -- 172.21.15.153:0/1234094355 shutdown_connections 2024-03-21T10:53:09.963 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:09.967+0000 7f4eccc00640 1 --2- 172.21.15.153:0/1234094355 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4ec8102e50 0x7f4ec8106e90 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:09.963 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:09.967+0000 7f4eccc00640 1 --2- 172.21.15.153:0/1234094355 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4ec81024a0 0x7f4ec8102880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:09.963 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:09.967+0000 7f4eccc00640 1 -- 172.21.15.153:0/1234094355 >> 172.21.15.153:0/1234094355 conn(0x7f4ec80fdd00 msgr2=0x7f4ec8100120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:53:09.963 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:09.967+0000 7f4eccc00640 1 -- 172.21.15.153:0/1234094355 shutdown_connections 2024-03-21T10:53:09.963 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:09.967+0000 7f4eccc00640 1 -- 172.21.15.153:0/1234094355 wait complete. 2024-03-21T10:53:09.964 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:09.968+0000 7f4eccc00640 1 Processor -- start 2024-03-21T10:53:09.964 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:09.968+0000 7f4eccc00640 1 -- start start 2024-03-21T10:53:09.965 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:09.968+0000 7f4eccc00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4ec81024a0 0x7f4ec8197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:53:09.965 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:09.968+0000 7f4eccc00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4ec8198520 0x7f4ec819c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:53:09.965 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:09.968+0000 7f4eccc00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec8198b20 con 0x7f4ec8198520 2024-03-21T10:53:09.965 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:09.968+0000 7f4eccc00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec8198c90 con 0x7f4ec81024a0 2024-03-21T10:53:09.965 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:09.969+0000 7f4ec5400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4ec8198520 0x7f4ec819c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:53:09.965 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:09.969+0000 7f4ec5400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4ec8198520 0x7f4ec819c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42932/0 (socket says 172.21.15.153:42932) 2024-03-21T10:53:09.965 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:09.969+0000 7f4ec5400640 1 -- 172.21.15.153:0/2518554853 learned_addr learned my addr 172.21.15.153:0/2518554853 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:53:09.965 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:09.969+0000 7f4ec5e00640 1 --2- 172.21.15.153:0/2518554853 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4ec81024a0 0x7f4ec8197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:53:09.965 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:09.969+0000 7f4ec5400640 1 -- 172.21.15.153:0/2518554853 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4ec81024a0 msgr2=0x7f4ec8197fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:53:09.965 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:09.969+0000 7f4ec5400640 1 --2- 172.21.15.153:0/2518554853 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4ec81024a0 0x7f4ec8197fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:09.965 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:09.969+0000 7f4ec5400640 1 -- 172.21.15.153:0/2518554853 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4eb4009560 con 0x7f4ec8198520 2024-03-21T10:53:09.966 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:09.970+0000 7f4ec5400640 1 --2- 172.21.15.153:0/2518554853 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4ec8198520 0x7f4ec819c980 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f4eb4036040 tx=0x7f4eb402c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:53:09.966 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:09.970+0000 7f4ebea00640 1 -- 172.21.15.153:0/2518554853 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4eb4004520 con 0x7f4ec8198520 2024-03-21T10:53:09.966 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:09.970+0000 7f4ebea00640 1 -- 172.21.15.153:0/2518554853 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4eb4003ab0 con 0x7f4ec8198520 2024-03-21T10:53:09.966 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:09.970+0000 7f4ebea00640 1 -- 172.21.15.153:0/2518554853 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4eb4034440 con 0x7f4ec8198520 2024-03-21T10:53:09.966 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:09.970+0000 7f4eccc00640 1 -- 172.21.15.153:0/2518554853 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ec819cf20 con 0x7f4ec8198520 2024-03-21T10:53:09.967 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:09.970+0000 7f4eccc00640 1 -- 172.21.15.153:0/2518554853 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ec819d430 con 0x7f4ec8198520 2024-03-21T10:53:09.967 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:09.972+0000 7f4eccc00640 1 -- 172.21.15.153:0/2518554853 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ec804e040 con 0x7f4ec8198520 2024-03-21T10:53:09.967 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:09.972+0000 7f4ebea00640 1 -- 172.21.15.153:0/2518554853 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f4eb4003c20 con 0x7f4ec8198520 2024-03-21T10:53:09.969 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:09.974+0000 7f4ebea00640 1 --2- 172.21.15.153:0/2518554853 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f4eac06e3d0 0x7f4eac070890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:53:09.969 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:09.974+0000 7f4ebea00640 1 -- 172.21.15.153:0/2518554853 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f4eb40b08f0 con 0x7f4ec8198520 2024-03-21T10:53:09.969 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:09.974+0000 7f4ec5e00640 1 --2- 172.21.15.153:0/2518554853 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f4eac06e3d0 0x7f4eac070890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:53:09.970 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:09.975+0000 7f4ec5e00640 1 --2- 172.21.15.153:0/2518554853 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f4eac06e3d0 0x7f4eac070890 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f4eb8005fd0 tx=0x7f4eb8005850 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:53:09.973 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:09.978+0000 7f4ebea00640 1 -- 172.21.15.153:0/2518554853 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4eb40799b0 con 0x7f4ec8198520 2024-03-21T10:53:10.051 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:53:09 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/279744923' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 40, "format": "json"}]: dispatch 2024-03-21T10:53:10.131 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:53:09 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/3059685483' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T10:53:10.196 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:53:09 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/279744923' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 40, "format": "json"}]: dispatch 2024-03-21T10:53:10.196 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:53:09 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/3059685483' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T10:53:10.348 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:10.352+0000 7f4eccc00640 1 -- 172.21.15.153:0/2518554853 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4ec819d6a0 con 0x7f4ec8198520 2024-03-21T10:53:10.348 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:10.353+0000 7f4ebea00640 1 -- 172.21.15.153:0/2518554853 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v98) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4eb407d070 con 0x7f4ec8198520 2024-03-21T10:53:10.349 INFO:teuthology.orchestra.run.smithi153.stdout:pg_num: 1 2024-03-21T10:53:10.351 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:10.355+0000 7f4eccc00640 1 -- 172.21.15.153:0/2518554853 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f4eac06e3d0 msgr2=0x7f4eac070890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:53:10.351 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:10.355+0000 7f4eccc00640 1 --2- 172.21.15.153:0/2518554853 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f4eac06e3d0 0x7f4eac070890 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f4eb8005fd0 tx=0x7f4eb8005850 comp rx=0 tx=0).stop 2024-03-21T10:53:10.351 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:10.355+0000 7f4eccc00640 1 -- 172.21.15.153:0/2518554853 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4ec8198520 msgr2=0x7f4ec819c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:53:10.351 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:10.355+0000 7f4eccc00640 1 --2- 172.21.15.153:0/2518554853 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4ec8198520 0x7f4ec819c980 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f4eb4036040 tx=0x7f4eb402c7a0 comp rx=0 tx=0).stop 2024-03-21T10:53:10.351 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:10.356+0000 7f4eccc00640 1 -- 172.21.15.153:0/2518554853 shutdown_connections 2024-03-21T10:53:10.351 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:10.356+0000 7f4eccc00640 1 --2- 172.21.15.153:0/2518554853 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f4eac06e3d0 0x7f4eac070890 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:10.351 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:10.356+0000 7f4eccc00640 1 --2- 172.21.15.153:0/2518554853 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4ec8198520 0x7f4ec819c980 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:10.352 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:10.356+0000 7f4eccc00640 1 --2- 172.21.15.153:0/2518554853 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4ec81024a0 0x7f4ec8197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:10.352 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:10.356+0000 7f4eccc00640 1 -- 172.21.15.153:0/2518554853 >> 172.21.15.153:0/2518554853 conn(0x7f4ec80fdd00 msgr2=0x7f4ec80ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:53:10.352 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:10.356+0000 7f4eccc00640 1 -- 172.21.15.153:0/2518554853 shutdown_connections 2024-03-21T10:53:10.352 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:10.356+0000 7f4eccc00640 1 -- 172.21.15.153:0/2518554853 wait complete. 2024-03-21T10:53:10.371 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-03-21T10:53:10.562 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:10.566+0000 7f5d95400640 1 -- 172.21.15.153:0/1066472592 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5d90101200 msgr2=0x7f5d901015e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:53:10.562 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:10.566+0000 7f5d95400640 1 --2- 172.21.15.153:0/1066472592 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5d90101200 0x7f5d901015e0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f5d80009880 tx=0x7f5d8002bd90 comp rx=0 tx=0).stop 2024-03-21T10:53:10.562 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:10.566+0000 7f5d95400640 1 -- 172.21.15.153:0/1066472592 shutdown_connections 2024-03-21T10:53:10.562 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:10.566+0000 7f5d95400640 1 --2- 172.21.15.153:0/1066472592 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5d90101b20 0x7f5d9010a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:10.562 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:10.566+0000 7f5d95400640 1 --2- 172.21.15.153:0/1066472592 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5d90101200 0x7f5d901015e0 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:10.562 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:10.566+0000 7f5d95400640 1 -- 172.21.15.153:0/1066472592 >> 172.21.15.153:0/1066472592 conn(0x7f5d900faa20 msgr2=0x7f5d900fce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:53:10.562 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:10.566+0000 7f5d95400640 1 -- 172.21.15.153:0/1066472592 shutdown_connections 2024-03-21T10:53:10.562 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:10.566+0000 7f5d95400640 1 -- 172.21.15.153:0/1066472592 wait complete. 2024-03-21T10:53:10.564 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:10.567+0000 7f5d95400640 1 Processor -- start 2024-03-21T10:53:10.564 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:10.567+0000 7f5d95400640 1 -- start start 2024-03-21T10:53:10.564 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:10.567+0000 7f5d95400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5d90101200 0x7f5d90198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:53:10.564 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:10.568+0000 7f5d95400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5d90101b20 0x7f5d90198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:53:10.564 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:10.568+0000 7f5d95400640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d90198c20 con 0x7f5d90101200 2024-03-21T10:53:10.564 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:10.568+0000 7f5d95400640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d9019c950 con 0x7f5d90101b20 2024-03-21T10:53:10.564 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:10.568+0000 7f5d8e800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5d90101200 0x7f5d90198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:53:10.564 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:10.568+0000 7f5d8e800640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5d90101200 0x7f5d90198050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42958/0 (socket says 172.21.15.153:42958) 2024-03-21T10:53:10.564 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:10.568+0000 7f5d8e800640 1 -- 172.21.15.153:0/2054498222 learned_addr learned my addr 172.21.15.153:0/2054498222 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:53:10.564 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:10.568+0000 7f5d8e800640 1 -- 172.21.15.153:0/2054498222 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5d90101b20 msgr2=0x7f5d90198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:53:10.564 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:10.568+0000 7f5d8e800640 1 --2- 172.21.15.153:0/2054498222 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5d90101b20 0x7f5d90198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:10.565 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:10.568+0000 7f5d8e800640 1 -- 172.21.15.153:0/2054498222 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d80009560 con 0x7f5d90101200 2024-03-21T10:53:10.565 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:10.569+0000 7f5d8e800640 1 --2- 172.21.15.153:0/2054498222 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5d90101200 0x7f5d90198050 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f5d80036040 tx=0x7f5d8002c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:53:10.565 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:10.569+0000 7f5d87400640 1 -- 172.21.15.153:0/2054498222 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5d800035f0 con 0x7f5d90101200 2024-03-21T10:53:10.565 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:10.569+0000 7f5d87400640 1 -- 172.21.15.153:0/2054498222 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5d80003c10 con 0x7f5d90101200 2024-03-21T10:53:10.565 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:10.569+0000 7f5d95400640 1 -- 172.21.15.153:0/2054498222 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d9019cbd0 con 0x7f5d90101200 2024-03-21T10:53:10.566 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:10.569+0000 7f5d87400640 1 -- 172.21.15.153:0/2054498222 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5d80005690 con 0x7f5d90101200 2024-03-21T10:53:10.566 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:10.569+0000 7f5d95400640 1 -- 172.21.15.153:0/2054498222 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d9019d130 con 0x7f5d90101200 2024-03-21T10:53:10.566 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:10.570+0000 7f5d95400640 1 -- 172.21.15.153:0/2054498222 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d9004e040 con 0x7f5d90101200 2024-03-21T10:53:10.567 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:10.571+0000 7f5d87400640 1 -- 172.21.15.153:0/2054498222 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f5d80004ed0 con 0x7f5d90101200 2024-03-21T10:53:10.568 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:10.572+0000 7f5d87400640 1 --2- 172.21.15.153:0/2054498222 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f5d7406e370 0x7f5d74070830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:53:10.568 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:10.572+0000 7f5d87400640 1 -- 172.21.15.153:0/2054498222 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f5d800b1140 con 0x7f5d90101200 2024-03-21T10:53:10.568 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:10.573+0000 7f5d8de00640 1 --2- 172.21.15.153:0/2054498222 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f5d7406e370 0x7f5d74070830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:53:10.568 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:10.573+0000 7f5d8de00640 1 --2- 172.21.15.153:0/2054498222 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f5d7406e370 0x7f5d74070830 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c005fd0 tx=0x7f5d7c005ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:53:10.572 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:10.577+0000 7f5d87400640 1 -- 172.21.15.153:0/2054498222 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f5d8007a200 con 0x7f5d90101200 2024-03-21T10:53:10.804 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:10.809+0000 7f5d95400640 1 -- 172.21.15.153:0/2054498222 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f5d90109a70 con 0x7f5d90101200 2024-03-21T10:53:10.805 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:10.809+0000 7f5d87400640 1 -- 172.21.15.153:0/2054498222 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v98) v1 ==== 107+0+11 (secure 0 0 0) 0x7f5d8007d8c0 con 0x7f5d90101200 2024-03-21T10:53:10.805 INFO:teuthology.orchestra.run.smithi153.stdout:pg_num: 32 2024-03-21T10:53:10.808 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:10.812+0000 7f5d95400640 1 -- 172.21.15.153:0/2054498222 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f5d7406e370 msgr2=0x7f5d74070830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:53:10.808 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:10.812+0000 7f5d95400640 1 --2- 172.21.15.153:0/2054498222 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f5d7406e370 0x7f5d74070830 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c005fd0 tx=0x7f5d7c005ee0 comp rx=0 tx=0).stop 2024-03-21T10:53:10.808 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:10.812+0000 7f5d95400640 1 -- 172.21.15.153:0/2054498222 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5d90101200 msgr2=0x7f5d90198050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:53:10.808 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:10.812+0000 7f5d95400640 1 --2- 172.21.15.153:0/2054498222 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5d90101200 0x7f5d90198050 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f5d80036040 tx=0x7f5d8002c900 comp rx=0 tx=0).stop 2024-03-21T10:53:10.808 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:10.812+0000 7f5d95400640 1 -- 172.21.15.153:0/2054498222 shutdown_connections 2024-03-21T10:53:10.808 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:10.812+0000 7f5d95400640 1 --2- 172.21.15.153:0/2054498222 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f5d7406e370 0x7f5d74070830 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:10.808 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:10.812+0000 7f5d95400640 1 --2- 172.21.15.153:0/2054498222 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5d90101b20 0x7f5d90198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:10.809 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:10.812+0000 7f5d95400640 1 --2- 172.21.15.153:0/2054498222 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f5d90101200 0x7f5d90198050 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:10.809 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:10.812+0000 7f5d95400640 1 -- 172.21.15.153:0/2054498222 >> 172.21.15.153:0/2054498222 conn(0x7f5d900faa20 msgr2=0x7f5d90106c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:53:10.809 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:10.813+0000 7f5d95400640 1 -- 172.21.15.153:0/2054498222 shutdown_connections 2024-03-21T10:53:10.809 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:10.813+0000 7f5d95400640 1 -- 172.21.15.153:0/2054498222 wait complete. 2024-03-21T10:53:10.827 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-03-21T10:53:11.037 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:11.042+0000 7f47b8a00640 1 -- 172.21.15.153:0/410454013 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f47b4107510 msgr2=0x7f47b41078f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:53:11.038 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:11.042+0000 7f47b8a00640 1 --2- 172.21.15.153:0/410454013 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f47b4107510 0x7f47b41078f0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f47ac009c20 tx=0x7f47ac02c1b0 comp rx=0 tx=0).stop 2024-03-21T10:53:11.038 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:11.042+0000 7f47b8a00640 1 -- 172.21.15.153:0/410454013 shutdown_connections 2024-03-21T10:53:11.038 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:11.042+0000 7f47b8a00640 1 --2- 172.21.15.153:0/410454013 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f47b4107ec0 0x7f47b410c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:11.038 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:11.042+0000 7f47b8a00640 1 --2- 172.21.15.153:0/410454013 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f47b4107510 0x7f47b41078f0 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:11.038 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:11.042+0000 7f47b8a00640 1 -- 172.21.15.153:0/410454013 >> 172.21.15.153:0/410454013 conn(0x7f47b406af20 msgr2=0x7f47b406b330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:53:11.038 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:11.042+0000 7f47b8a00640 1 -- 172.21.15.153:0/410454013 shutdown_connections 2024-03-21T10:53:11.038 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:11.042+0000 7f47b8a00640 1 -- 172.21.15.153:0/410454013 wait complete. 2024-03-21T10:53:11.039 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:11.043+0000 7f47b8a00640 1 Processor -- start 2024-03-21T10:53:11.039 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:11.044+0000 7f47b8a00640 1 -- start start 2024-03-21T10:53:11.040 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:11.044+0000 7f47b8a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f47b4107510 0x7f47b41a4960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:53:11.040 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:11.044+0000 7f47b8a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f47b41a4ea0 0x7f47b41a9330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:53:11.040 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:11.044+0000 7f47b8a00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47b41a54c0 con 0x7f47b41a4ea0 2024-03-21T10:53:11.040 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:11.044+0000 7f47b8a00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47b41a5600 con 0x7f47b4107510 2024-03-21T10:53:11.040 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:11.044+0000 7f47b2a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f47b41a4ea0 0x7f47b41a9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:53:11.040 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:11.044+0000 7f47b2a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f47b41a4ea0 0x7f47b41a9330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42964/0 (socket says 172.21.15.153:42964) 2024-03-21T10:53:11.041 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:11.044+0000 7f47b2a00640 1 -- 172.21.15.153:0/1967610788 learned_addr learned my addr 172.21.15.153:0/1967610788 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:53:11.041 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:11.045+0000 7f47b3400640 1 --2- 172.21.15.153:0/1967610788 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f47b4107510 0x7f47b41a4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:53:11.041 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:11.045+0000 7f47b2a00640 1 -- 172.21.15.153:0/1967610788 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f47b4107510 msgr2=0x7f47b41a4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:53:11.041 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:11.045+0000 7f47b2a00640 1 --2- 172.21.15.153:0/1967610788 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f47b4107510 0x7f47b41a4960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:11.041 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:11.045+0000 7f47b2a00640 1 -- 172.21.15.153:0/1967610788 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47ac009900 con 0x7f47b41a4ea0 2024-03-21T10:53:11.042 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:11.045+0000 7f47b2a00640 1 --2- 172.21.15.153:0/1967610788 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f47b41a4ea0 0x7f47b41a9330 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f47a400d6c0 tx=0x7f47a400db80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:53:11.042 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:11.045+0000 7f47abe00640 1 -- 172.21.15.153:0/1967610788 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f47a40051f0 con 0x7f47b41a4ea0 2024-03-21T10:53:11.042 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:11.046+0000 7f47abe00640 1 -- 172.21.15.153:0/1967610788 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f47a4009ce0 con 0x7f47b41a4ea0 2024-03-21T10:53:11.042 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:11.046+0000 7f47b8a00640 1 -- 172.21.15.153:0/1967610788 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47b41a98d0 con 0x7f47b41a4ea0 2024-03-21T10:53:11.042 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:11.046+0000 7f47abe00640 1 -- 172.21.15.153:0/1967610788 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f47a4010460 con 0x7f47b41a4ea0 2024-03-21T10:53:11.042 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:11.046+0000 7f47b8a00640 1 -- 172.21.15.153:0/1967610788 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47b41a9eb0 con 0x7f47b41a4ea0 2024-03-21T10:53:11.042 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:11.047+0000 7f47b8a00640 1 -- 172.21.15.153:0/1967610788 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47b41aa190 con 0x7f47b41a4ea0 2024-03-21T10:53:11.044 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:11.049+0000 7f47abe00640 1 -- 172.21.15.153:0/1967610788 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f47a4005350 con 0x7f47b41a4ea0 2024-03-21T10:53:11.046 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:11.051+0000 7f47abe00640 1 --2- 172.21.15.153:0/1967610788 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f479806e3d0 0x7f4798070890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:53:11.046 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:11.051+0000 7f47abe00640 1 -- 172.21.15.153:0/1967610788 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f47a408e0a0 con 0x7f47b41a4ea0 2024-03-21T10:53:11.046 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:11.051+0000 7f47b3400640 1 --2- 172.21.15.153:0/1967610788 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f479806e3d0 0x7f4798070890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:53:11.047 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:11.052+0000 7f47b3400640 1 --2- 172.21.15.153:0/1967610788 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f479806e3d0 0x7f4798070890 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f47ac02c7c0 tx=0x7f47ac009380 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:53:11.050 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:11.055+0000 7f47abe00640 1 -- 172.21.15.153:0/1967610788 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f47a40570c0 con 0x7f47b41a4ea0 2024-03-21T10:53:11.051 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:53:10 smithi153 ceph-mon[113733]: pgmap v1242: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:53:11.051 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:53:10 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/2518554853' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T10:53:11.196 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:53:10 smithi174 ceph-mon[88004]: pgmap v1242: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:53:11.196 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:53:10 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/2518554853' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T10:53:11.265 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:11.269+0000 7f47b8a00640 1 -- 172.21.15.153:0/1967610788 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7f47b41a9a60 con 0x7f47b41a4ea0 2024-03-21T10:53:11.266 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:11.270+0000 7f47abe00640 1 -- 172.21.15.153:0/1967610788 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v98) v1 ==== 107+0+11 (secure 0 0 0) 0x7f47b41a9a60 con 0x7f47b41a4ea0 2024-03-21T10:53:11.266 INFO:teuthology.orchestra.run.smithi153.stdout:pg_num: 32 2024-03-21T10:53:11.270 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:11.274+0000 7f47b8a00640 1 -- 172.21.15.153:0/1967610788 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f479806e3d0 msgr2=0x7f4798070890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:53:11.270 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:11.274+0000 7f47b8a00640 1 --2- 172.21.15.153:0/1967610788 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f479806e3d0 0x7f4798070890 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f47ac02c7c0 tx=0x7f47ac009380 comp rx=0 tx=0).stop 2024-03-21T10:53:11.270 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:11.274+0000 7f47b8a00640 1 -- 172.21.15.153:0/1967610788 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f47b41a4ea0 msgr2=0x7f47b41a9330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:53:11.270 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:11.274+0000 7f47b8a00640 1 --2- 172.21.15.153:0/1967610788 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f47b41a4ea0 0x7f47b41a9330 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f47a400d6c0 tx=0x7f47a400db80 comp rx=0 tx=0).stop 2024-03-21T10:53:11.270 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:11.274+0000 7f47b8a00640 1 -- 172.21.15.153:0/1967610788 shutdown_connections 2024-03-21T10:53:11.270 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:11.274+0000 7f47b8a00640 1 --2- 172.21.15.153:0/1967610788 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f479806e3d0 0x7f4798070890 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:11.270 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:11.274+0000 7f47b8a00640 1 --2- 172.21.15.153:0/1967610788 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f47b41a4ea0 0x7f47b41a9330 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:11.270 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:11.274+0000 7f47b8a00640 1 --2- 172.21.15.153:0/1967610788 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f47b4107510 0x7f47b41a4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:11.270 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:11.274+0000 7f47b8a00640 1 -- 172.21.15.153:0/1967610788 >> 172.21.15.153:0/1967610788 conn(0x7f47b406af20 msgr2=0x7f47b4119a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:53:11.270 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:11.274+0000 7f47b8a00640 1 -- 172.21.15.153:0/1967610788 shutdown_connections 2024-03-21T10:53:11.270 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:11.274+0000 7f47b8a00640 1 -- 172.21.15.153:0/1967610788 wait complete. 2024-03-21T10:53:11.288 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-21T10:53:11.498 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:11.502+0000 7f4ce6a00640 1 -- 172.21.15.153:0/676425491 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4ce0072f60 msgr2=0x7f4ce0110940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:53:11.498 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:11.502+0000 7f4ce6a00640 1 --2- 172.21.15.153:0/676425491 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4ce0072f60 0x7f4ce0110940 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc009880 tx=0x7f4ccc02bd70 comp rx=0 tx=0).stop 2024-03-21T10:53:11.498 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:11.502+0000 7f4ce6a00640 1 -- 172.21.15.153:0/676425491 shutdown_connections 2024-03-21T10:53:11.498 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:11.502+0000 7f4ce6a00640 1 --2- 172.21.15.153:0/676425491 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4ce0072f60 0x7f4ce0110940 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:11.498 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:11.502+0000 7f4ce6a00640 1 --2- 172.21.15.153:0/676425491 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4ce0072640 0x7f4ce0072a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:11.498 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:11.502+0000 7f4ce6a00640 1 -- 172.21.15.153:0/676425491 >> 172.21.15.153:0/676425491 conn(0x7f4ce00fb960 msgr2=0x7f4ce00fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:53:11.498 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:11.502+0000 7f4ce6a00640 1 -- 172.21.15.153:0/676425491 shutdown_connections 2024-03-21T10:53:11.498 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:11.502+0000 7f4ce6a00640 1 -- 172.21.15.153:0/676425491 wait complete. 2024-03-21T10:53:11.499 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:11.504+0000 7f4ce6a00640 1 Processor -- start 2024-03-21T10:53:11.499 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:11.504+0000 7f4ce6a00640 1 -- start start 2024-03-21T10:53:11.500 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:11.504+0000 7f4ce6a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4ce0072640 0x7f4ce019c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:53:11.500 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:11.504+0000 7f4ce6a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4ce0072f60 0x7f4ce019c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:53:11.500 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:11.504+0000 7f4ce6a00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce019cc80 con 0x7f4ce0072640 2024-03-21T10:53:11.500 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:11.504+0000 7f4ce6a00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce01a0a50 con 0x7f4ce0072f60 2024-03-21T10:53:11.501 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:11.505+0000 7f4ce5600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4ce0072640 0x7f4ce019c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:53:11.501 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:11.505+0000 7f4ce5600640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4ce0072640 0x7f4ce019c0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42974/0 (socket says 172.21.15.153:42974) 2024-03-21T10:53:11.501 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:11.505+0000 7f4ce5600640 1 -- 172.21.15.153:0/3246856400 learned_addr learned my addr 172.21.15.153:0/3246856400 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:53:11.501 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:11.505+0000 7f4ce4c00640 1 --2- 172.21.15.153:0/3246856400 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4ce0072f60 0x7f4ce019c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:53:11.501 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:11.505+0000 7f4ce5600640 1 -- 172.21.15.153:0/3246856400 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4ce0072f60 msgr2=0x7f4ce019c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:53:11.501 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:11.505+0000 7f4ce5600640 1 --2- 172.21.15.153:0/3246856400 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4ce0072f60 0x7f4ce019c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:11.501 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:11.505+0000 7f4ce5600640 1 -- 172.21.15.153:0/3246856400 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ccc009560 con 0x7f4ce0072640 2024-03-21T10:53:11.501 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:11.505+0000 7f4ce4c00640 1 --2- 172.21.15.153:0/3246856400 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4ce0072f60 0x7f4ce019c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:53:11.502 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:11.505+0000 7f4ce5600640 1 --2- 172.21.15.153:0/3246856400 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4ce0072640 0x7f4ce019c0b0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f4cd400d6f0 tx=0x7f4cd400dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:53:11.502 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:11.506+0000 7f4cde000640 1 -- 172.21.15.153:0/3246856400 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4cd4009a30 con 0x7f4ce0072640 2024-03-21T10:53:11.502 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:11.506+0000 7f4cde000640 1 -- 172.21.15.153:0/3246856400 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4cd40054e0 con 0x7f4ce0072640 2024-03-21T10:53:11.502 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:11.506+0000 7f4ce6a00640 1 -- 172.21.15.153:0/3246856400 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ce01a0d30 con 0x7f4ce0072640 2024-03-21T10:53:11.502 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:11.506+0000 7f4cde000640 1 -- 172.21.15.153:0/3246856400 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4cd4010460 con 0x7f4ce0072640 2024-03-21T10:53:11.502 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:11.506+0000 7f4ce6a00640 1 -- 172.21.15.153:0/3246856400 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ce01a1290 con 0x7f4ce0072640 2024-03-21T10:53:11.503 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:11.507+0000 7f4ce6a00640 1 -- 172.21.15.153:0/3246856400 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ce004e890 con 0x7f4ce0072640 2024-03-21T10:53:11.503 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:11.508+0000 7f4cde000640 1 -- 172.21.15.153:0/3246856400 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f4cd4009b90 con 0x7f4ce0072640 2024-03-21T10:53:11.504 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:11.509+0000 7f4cde000640 1 --2- 172.21.15.153:0/3246856400 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f4cc806e420 0x7f4cc80708e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:53:11.504 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:11.509+0000 7f4cde000640 1 -- 172.21.15.153:0/3246856400 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f4cd408dfb0 con 0x7f4ce0072640 2024-03-21T10:53:11.505 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:11.509+0000 7f4ce4c00640 1 --2- 172.21.15.153:0/3246856400 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f4cc806e420 0x7f4cc80708e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:53:11.505 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:11.510+0000 7f4ce4c00640 1 --2- 172.21.15.153:0/3246856400 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f4cc806e420 0x7f4cc80708e0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc02c930 tx=0x7f4ccc005a20 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:53:11.509 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:11.514+0000 7f4cde000640 1 -- 172.21.15.153:0/3246856400 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4cd40570f0 con 0x7f4ce0072640 2024-03-21T10:53:11.845 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:11.849+0000 7f4ce6a00640 1 -- 172.21.15.153:0/3246856400 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f4ce010e030 con 0x7f4ce0072640 2024-03-21T10:53:11.845 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:11.850+0000 7f4cde000640 1 -- 172.21.15.153:0/3246856400 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 41 entries v98) v1 ==== 81+0+2462 (secure 0 0 0) 0x7f4cd40179b0 con 0x7f4ce0072640 2024-03-21T10:53:11.845 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.15.174:6825/1776947618 2024-03-22T10:19:05.021561+0000 2024-03-21T10:53:11.846 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.15.174:6824/1776947618 2024-03-22T10:19:05.021561+0000 2024-03-21T10:53:11.846 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.15.153:6829/866138999 2024-03-22T10:19:05.021528+0000 2024-03-21T10:53:11.846 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.15.153:0/3553873361 2024-03-22T10:13:31.087809+0000 2024-03-21T10:53:11.846 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.15.174:0/1056981865 2024-03-22T10:08:33.113521+0000 2024-03-21T10:53:11.846 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.15.153:0/2407536999 2024-03-22T09:56:00.849533+0000 2024-03-21T10:53:11.846 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.15.153:0/2969012027 2024-03-22T10:07:30.531722+0000 2024-03-21T10:53:11.846 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.15.153:6826/1353876997 2024-03-22T10:04:45.049156+0000 2024-03-21T10:53:11.846 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.15.153:0/623594592 2024-03-22T09:56:00.849533+0000 2024-03-21T10:53:11.846 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.15.153:0/1163205431 2024-03-22T09:56:43.480866+0000 2024-03-21T10:53:11.846 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.15.153:0/3278686288 2024-03-22T09:58:24.250138+0000 2024-03-21T10:53:11.846 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.15.153:0/652104290 2024-03-22T09:58:24.250138+0000 2024-03-21T10:53:11.847 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.15.153:6800/2524145044 2024-03-22T10:13:31.087809+0000 2024-03-21T10:53:11.847 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.15.174:0/3456690749 2024-03-22T10:08:33.113521+0000 2024-03-21T10:53:11.847 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.15.153:0/4237751158 2024-03-22T10:07:30.531722+0000 2024-03-21T10:53:11.847 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.15.153:0/3456116624 2024-03-22T09:56:43.480866+0000 2024-03-21T10:53:11.847 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.15.153:0/3342434133 2024-03-22T10:13:31.087809+0000 2024-03-21T10:53:11.847 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.15.153:0/2801644471 2024-03-22T10:07:30.531722+0000 2024-03-21T10:53:11.847 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.15.153:6801/1639929315 2024-03-22T10:07:30.531722+0000 2024-03-21T10:53:11.847 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.15.174:0/950700094 2024-03-22T10:08:33.113521+0000 2024-03-21T10:53:11.847 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.15.153:6827/1353876997 2024-03-22T10:04:45.049156+0000 2024-03-21T10:53:11.847 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.15.174:0/2227377986 2024-03-22T10:08:33.113521+0000 2024-03-21T10:53:11.847 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.15.153:6800/1143522828 2024-03-22T09:58:24.250138+0000 2024-03-21T10:53:11.847 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.15.153:0/3435678189 2024-03-22T09:56:00.849533+0000 2024-03-21T10:53:11.847 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.15.153:0/2431412876 2024-03-22T10:07:30.531722+0000 2024-03-21T10:53:11.847 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.15.153:6801/1143522828 2024-03-22T09:58:24.250138+0000 2024-03-21T10:53:11.847 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.15.174:6829/1155541738 2024-03-22T10:08:33.113521+0000 2024-03-21T10:53:11.848 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.15.153:0/517714023 2024-03-22T10:13:31.087809+0000 2024-03-21T10:53:11.848 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.15.174:0/828964242 2024-03-22T10:08:33.113521+0000 2024-03-21T10:53:11.848 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.15.153:0/405164318 2024-03-22T09:58:24.250138+0000 2024-03-21T10:53:11.848 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.15.153:0/816489762 2024-03-22T09:56:43.480866+0000 2024-03-21T10:53:11.848 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.15.153:0/2445011785 2024-03-22T10:13:31.087809+0000 2024-03-21T10:53:11.848 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.15.153:6800/3178125243 2024-03-22T09:56:43.480866+0000 2024-03-21T10:53:11.848 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.15.174:6828/1155541738 2024-03-22T10:08:33.113521+0000 2024-03-21T10:53:11.848 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.15.153:6801/3178125243 2024-03-22T09:56:43.480866+0000 2024-03-21T10:53:11.848 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.15.153:6828/866138999 2024-03-22T10:19:05.021528+0000 2024-03-21T10:53:11.848 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.15.153:6800/270837948 2024-03-22T09:56:00.849533+0000 2024-03-21T10:53:11.848 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.15.153:6801/270837948 2024-03-22T09:56:00.849533+0000 2024-03-21T10:53:11.848 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.15.174:0/2577665242 2024-03-22T10:08:33.113521+0000 2024-03-21T10:53:11.848 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.15.153:6800/1639929315 2024-03-22T10:07:30.531722+0000 2024-03-21T10:53:11.848 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.15.153:6801/2524145044 2024-03-22T10:13:31.087809+0000 2024-03-21T10:53:11.849 INFO:teuthology.orchestra.run.smithi153.stderr:listed 41 entries 2024-03-21T10:53:11.849 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:11.853+0000 7f4ce6a00640 1 -- 172.21.15.153:0/3246856400 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f4cc806e420 msgr2=0x7f4cc80708e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:53:11.849 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:11.853+0000 7f4ce6a00640 1 --2- 172.21.15.153:0/3246856400 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f4cc806e420 0x7f4cc80708e0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc02c930 tx=0x7f4ccc005a20 comp rx=0 tx=0).stop 2024-03-21T10:53:11.850 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:11.853+0000 7f4ce6a00640 1 -- 172.21.15.153:0/3246856400 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4ce0072640 msgr2=0x7f4ce019c0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:53:11.850 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:11.853+0000 7f4ce6a00640 1 --2- 172.21.15.153:0/3246856400 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4ce0072640 0x7f4ce019c0b0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f4cd400d6f0 tx=0x7f4cd400dbb0 comp rx=0 tx=0).stop 2024-03-21T10:53:11.850 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:11.853+0000 7f4ce6a00640 1 -- 172.21.15.153:0/3246856400 shutdown_connections 2024-03-21T10:53:11.850 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:11.853+0000 7f4ce6a00640 1 --2- 172.21.15.153:0/3246856400 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f4cc806e420 0x7f4cc80708e0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:11.850 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:11.853+0000 7f4ce6a00640 1 --2- 172.21.15.153:0/3246856400 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4ce0072f60 0x7f4ce019c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:11.850 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:11.853+0000 7f4ce6a00640 1 --2- 172.21.15.153:0/3246856400 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f4ce0072640 0x7f4ce019c0b0 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:11.850 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:11.853+0000 7f4ce6a00640 1 -- 172.21.15.153:0/3246856400 >> 172.21.15.153:0/3246856400 conn(0x7f4ce00fb960 msgr2=0x7f4ce0102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:53:11.850 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:11.854+0000 7f4ce6a00640 1 -- 172.21.15.153:0/3246856400 shutdown_connections 2024-03-21T10:53:11.850 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:11.854+0000 7f4ce6a00640 1 -- 172.21.15.153:0/3246856400 wait complete. 2024-03-21T10:53:11.867 DEBUG:teuthology.orchestra.run.smithi153:> set -ex 2024-03-21T10:53:11.867 DEBUG:teuthology.orchestra.run.smithi153:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-21T10:53:11.910 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T10:53:12.025 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:53:11 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/2054498222' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-21T10:53:12.025 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:53:11 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/1967610788' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-21T10:53:12.156 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:12.159+0000 7fc041200640 1 -- 172.21.15.153:0/3794446446 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc03c1000b0 msgr2=0x7fc03c100490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:53:12.156 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:12.159+0000 7fc041200640 1 --2- 172.21.15.153:0/3794446446 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc03c1000b0 0x7fc03c100490 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fc02c009880 tx=0x7fc02c02bce0 comp rx=0 tx=0).stop 2024-03-21T10:53:12.156 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:12.160+0000 7fc041200640 1 -- 172.21.15.153:0/3794446446 shutdown_connections 2024-03-21T10:53:12.156 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:12.160+0000 7fc041200640 1 --2- 172.21.15.153:0/3794446446 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc03c100a60 0x7fc03c104aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:12.156 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:12.160+0000 7fc041200640 1 --2- 172.21.15.153:0/3794446446 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc03c1000b0 0x7fc03c100490 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:12.156 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:12.160+0000 7fc041200640 1 -- 172.21.15.153:0/3794446446 >> 172.21.15.153:0/3794446446 conn(0x7fc03c0fb930 msgr2=0x7fc03c0fdd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:53:12.157 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:12.160+0000 7fc041200640 1 -- 172.21.15.153:0/3794446446 shutdown_connections 2024-03-21T10:53:12.157 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:12.160+0000 7fc041200640 1 -- 172.21.15.153:0/3794446446 wait complete. 2024-03-21T10:53:12.157 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:12.161+0000 7fc041200640 1 Processor -- start 2024-03-21T10:53:12.157 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:12.161+0000 7fc041200640 1 -- start start 2024-03-21T10:53:12.158 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:12.161+0000 7fc041200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc03c1000b0 0x7fc03c19dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:53:12.158 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:12.161+0000 7fc041200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc03c19e010 0x7fc03c197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:53:12.158 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:12.161+0000 7fc041200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc03c19e5e0 con 0x7fc03c1000b0 2024-03-21T10:53:12.159 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:12.161+0000 7fc041200640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc03c198080 con 0x7fc03c19e010 2024-03-21T10:53:12.159 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:12.161+0000 7fc03b400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc03c1000b0 0x7fc03c19dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:53:12.159 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:12.161+0000 7fc03b400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc03c1000b0 0x7fc03c19dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:42994/0 (socket says 172.21.15.153:42994) 2024-03-21T10:53:12.159 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:12.162+0000 7fc03b400640 1 -- 172.21.15.153:0/98305349 learned_addr learned my addr 172.21.15.153:0/98305349 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:53:12.159 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:12.162+0000 7fc03aa00640 1 --2- 172.21.15.153:0/98305349 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc03c19e010 0x7fc03c197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:53:12.159 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:12.162+0000 7fc03b400640 1 -- 172.21.15.153:0/98305349 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc03c19e010 msgr2=0x7fc03c197b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:53:12.159 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:12.162+0000 7fc03b400640 1 --2- 172.21.15.153:0/98305349 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc03c19e010 0x7fc03c197b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:12.159 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:12.162+0000 7fc03b400640 1 -- 172.21.15.153:0/98305349 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc02c009560 con 0x7fc03c1000b0 2024-03-21T10:53:12.159 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:12.162+0000 7fc03b400640 1 --2- 172.21.15.153:0/98305349 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc03c1000b0 0x7fc03c19dad0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fc02c009230 tx=0x7fc02c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:53:12.159 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:12.162+0000 7fc033e00640 1 -- 172.21.15.153:0/98305349 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc02c0035f0 con 0x7fc03c1000b0 2024-03-21T10:53:12.159 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:12.163+0000 7fc033e00640 1 -- 172.21.15.153:0/98305349 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc02c003c10 con 0x7fc03c1000b0 2024-03-21T10:53:12.159 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:12.163+0000 7fc033e00640 1 -- 172.21.15.153:0/98305349 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc02c034440 con 0x7fc03c1000b0 2024-03-21T10:53:12.159 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:12.163+0000 7fc041200640 1 -- 172.21.15.153:0/98305349 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc03c198300 con 0x7fc03c1000b0 2024-03-21T10:53:12.159 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:12.163+0000 7fc041200640 1 -- 172.21.15.153:0/98305349 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc03c198800 con 0x7fc03c1000b0 2024-03-21T10:53:12.160 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:12.164+0000 7fc041200640 1 -- 172.21.15.153:0/98305349 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc03c04e890 con 0x7fc03c1000b0 2024-03-21T10:53:12.161 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:12.166+0000 7fc033e00640 1 -- 172.21.15.153:0/98305349 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fc02c004fd0 con 0x7fc03c1000b0 2024-03-21T10:53:12.163 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:12.167+0000 7fc033e00640 1 --2- 172.21.15.153:0/98305349 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fc02006e490 0x7fc020070950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:53:12.163 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:12.168+0000 7fc033e00640 1 -- 172.21.15.153:0/98305349 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fc02c0b08a0 con 0x7fc03c1000b0 2024-03-21T10:53:12.163 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:12.168+0000 7fc03aa00640 1 --2- 172.21.15.153:0/98305349 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fc02006e490 0x7fc020070950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:53:12.164 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:12.168+0000 7fc03aa00640 1 --2- 172.21.15.153:0/98305349 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fc02006e490 0x7fc020070950 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fc0280058c0 tx=0x7fc028005c60 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:53:12.168 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:12.173+0000 7fc033e00640 1 -- 172.21.15.153:0/98305349 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc02c0799e0 con 0x7fc03c1000b0 2024-03-21T10:53:12.196 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:53:11 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/2054498222' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-21T10:53:12.196 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:53:11 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/1967610788' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-21T10:53:12.371 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:12.376+0000 7fc041200640 1 -- 172.21.15.153:0/98305349 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc03c199130 con 0x7fc03c1000b0 2024-03-21T10:53:12.374 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:12.378+0000 7fc033e00640 1 -- 172.21.15.153:0/98305349 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) v1 ==== 74+0+17231 (secure 0 0 0) 0x7fc03c199130 con 0x7fc03c1000b0 2024-03-21T10:53:12.374 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-03-21T10:53:12.375 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":98,"fsid":"070ed3e4-e769-11ee-95cd-87774f69a715","created":"2024-03-21T09:55:19.360240+0000","modified":"2024-03-21T10:35:13.153133+0000","last_up_change":"2024-03-21T10:18:57.856952+0000","last_in_change":"2024-03-21T10:02:32.306573+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T10:01:18.943639+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-03-21T10:04:25.621358+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.309999942779541,"score_stable":1.309999942779541,"optimal_score":1,"raw_score_acting":1.309999942779541,"raw_score_stable":1.309999942779541,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-03-21T10:04:26.593674+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0f2e1224-6f45-4f23-b97a-5b89319a90f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":50,"up_from":61,"up_thru":90,"down_at":51,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":2525147935},{"type":"v1","addr":"172.21.15.153:6803","nonce":2525147935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":2525147935},{"type":"v1","addr":"172.21.15.153:6805","nonce":2525147935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":2525147935},{"type":"v1","addr":"172.21.15.153:6809","nonce":2525147935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":2525147935},{"type":"v1","addr":"172.21.15.153:6807","nonce":2525147935}]},"public_addr":"172.21.15.153:6803/2525147935","cluster_addr":"172.21.15.153:6805/2525147935","heartbeat_back_addr":"172.21.15.153:6809/2525147935","heartbeat_front_addr":"172.21.15.153:6807/2525147935","state":["exists","up"]},{"osd":1,"uuid":"76195edb-aa7a-4627-a211-09a578de65c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":66,"up_from":70,"up_thru":90,"down_at":67,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6810","nonce":1529835318},{"type":"v1","addr":"172.21.15.153:6811","nonce":1529835318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":1529835318},{"type":"v1","addr":"172.21.15.153:6813","nonce":1529835318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6816","nonce":1529835318},{"type":"v1","addr":"172.21.15.153:6817","nonce":1529835318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":1529835318},{"type":"v1","addr":"172.21.15.153:6815","nonce":1529835318}]},"public_addr":"172.21.15.153:6811/1529835318","cluster_addr":"172.21.15.153:6813/1529835318","heartbeat_back_addr":"172.21.15.153:6817/1529835318","heartbeat_front_addr":"172.21.15.153:6815/1529835318","state":["exists","up"]},{"osd":2,"uuid":"3ec17cf8-f0ea-45c4-b084-b304024d6e92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":71,"up_from":75,"up_thru":90,"down_at":72,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":292763575},{"type":"v1","addr":"172.21.15.153:6819","nonce":292763575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":292763575},{"type":"v1","addr":"172.21.15.153:6821","nonce":292763575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":292763575},{"type":"v1","addr":"172.21.15.153:6825","nonce":292763575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":292763575},{"type":"v1","addr":"172.21.15.153:6823","nonce":292763575}]},"public_addr":"172.21.15.153:6819/292763575","cluster_addr":"172.21.15.153:6821/292763575","heartbeat_back_addr":"172.21.15.153:6825/292763575","heartbeat_front_addr":"172.21.15.153:6823/292763575","state":["exists","up"]},{"osd":3,"uuid":"285146f6-6209-47ee-9e5b-2e075c949231","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":27,"last_clean_end":76,"up_from":80,"up_thru":90,"down_at":77,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1830370519},{"type":"v1","addr":"172.21.15.174:6801","nonce":1830370519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1830370519},{"type":"v1","addr":"172.21.15.174:6803","nonce":1830370519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1830370519},{"type":"v1","addr":"172.21.15.174:6807","nonce":1830370519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":1830370519},{"type":"v1","addr":"172.21.15.174:6805","nonce":1830370519}]},"public_addr":"172.21.15.174:6801/1830370519","cluster_addr":"172.21.15.174:6803/1830370519","heartbeat_back_addr":"172.21.15.174:6807/1830370519","heartbeat_front_addr":"172.21.15.174:6805/1830370519","state":["exists","up"]},{"osd":4,"uuid":"f05cec2d-c649-489d-98a3-f24a75fe1f95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":35,"last_clean_end":81,"up_from":85,"up_thru":90,"down_at":82,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1143789116},{"type":"v1","addr":"172.21.15.174:6809","nonce":1143789116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1143789116},{"type":"v1","addr":"172.21.15.174:6811","nonce":1143789116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1143789116},{"type":"v1","addr":"172.21.15.174:6815","nonce":1143789116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1143789116},{"type":"v1","addr":"172.21.15.174:6813","nonce":1143789116}]},"public_addr":"172.21.15.174:6809/1143789116","cluster_addr":"172.21.15.174:6811/1143789116","heartbeat_back_addr":"172.21.15.174:6815/1143789116","heartbeat_front_addr":"172.21.15.174:6813/1143789116","state":["exists","up"]},{"osd":5,"uuid":"f430a3b2-e1b5-4698-9089-812fcf3a4959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":40,"last_clean_end":86,"up_from":90,"up_thru":90,"down_at":87,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":385015973},{"type":"v1","addr":"172.21.15.174:6817","nonce":385015973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":385015973},{"type":"v1","addr":"172.21.15.174:6819","nonce":385015973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":385015973},{"type":"v1","addr":"172.21.15.174:6823","nonce":385015973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":385015973},{"type":"v1","addr":"172.21.15.174:6821","nonce":385015973}]},"public_addr":"172.21.15.174:6817/385015973","cluster_addr":"172.21.15.174:6819/385015973","heartbeat_back_addr":"172.21.15.174:6823/385015973","heartbeat_front_addr":"172.21.15.174:6821/385015973","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:00:04.356688+0000","dead_epoch":50},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:00:39.988638+0000","dead_epoch":66},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:01:14.882381+0000","dead_epoch":71},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:01:48.072468+0000","dead_epoch":76},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:02:21.211701+0000","dead_epoch":81},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T10:02:54.161138+0000","dead_epoch":86}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":3,"to":2}]},{"pgid":"2.14","mappings":[{"from":3,"to":0}]},{"pgid":"2.16","mappings":[{"from":3,"to":0}]},{"pgid":"3.7","mappings":[{"from":1,"to":0}]},{"pgid":"3.b","mappings":[{"from":1,"to":2},{"from":4,"to":5}]},{"pgid":"3.15","mappings":[{"from":3,"to":2}]},{"pgid":"3.19","mappings":[{"from":3,"to":5}]},{"pgid":"3.1c","mappings":[{"from":4,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.174:6825/1776947618":"2024-03-22T10:19:05.021561+0000","172.21.15.174:6824/1776947618":"2024-03-22T10:19:05.021561+0000","172.21.15.153:6829/866138999":"2024-03-22T10:19:05.021528+0000","172.21.15.153:0/3553873361":"2024-03-22T10:13:31.087809+0000","172.21.15.174:0/1056981865":"2024-03-22T10:08:33.113521+0000","172.21.15.153:0/2407536999":"2024-03-22T09:56:00.849533+0000","172.21.15.153:0/2969012027":"2024-03-22T10:07:30.531722+0000","172.21.15.153:6826/1353876997":"2024-03-22T10:04:45.049156+0000","172.21.15.153:0/623594592":"2024-03-22T09:56:00.849533+0000","172.21.15.153:0/1163205431":"2024-03-22T09:56:43.480866+0000","172.21.15.153:0/3278686288":"2024-03-22T09:58:24.250138+0000","172.21.15.153:0/652104290":"2024-03-22T09:58:24.250138+0000","172.21.15.153:6800/2524145044":"2024-03-22T10:13:31.087809+0000","172.21.15.174:0/3456690749":"2024-03-22T10:08:33.113521+0000","172.21.15.153:0/4237751158":"2024-03-22T10:07:30.531722+0000","172.21.15.153:0/3456116624":"2024-03-22T09:56:43.480866+0000","172.21.15.153:0/3342434133":"2024-03-22T10:13:31.087809+0000","172.21.15.153:0/2801644471":"2024-03-22T10:07:30.531722+0000","172.21.15.153:6801/1639929315":"2024-03-22T10:07:30.531722+0000","172.21.15.174:0/950700094":"2024-03-22T10:08:33.113521+0000","172.21.15.153:6827/1353876997":"2024-03-22T10:04:45.049156+0000","172.21.15.174:0/2227377986":"2024-03-22T10:08:33.113521+0000","172.21.15.153:6800/1143522828":"2024-03-22T09:58:24.250138+0000","172.21.15.153:0/3435678189":"2024-03-22T09:56:00.849533+0000","172.21.15.153:0/2431412876":"2024-03-22T10:07:30.531722+0000","172.21.15.153:6801/1143522828":"2024-03-22T09:58:24.250138+0000","172.21.15.174:6829/1155541738":"2024-03-22T10:08:33.113521+0000","172.21.15.153:0/517714023":"2024-03-22T10:13:31.087809+0000","172.21.15.174:0/828964242":"2024-03-22T10:08:33.113521+0000","172.21.15.153:0/405164318":"2024-03-22T09:58:24.250138+0000","172.21.15.153:0/816489762":"2024-03-22T09:56:43.480866+0000","172.21.15.153:0/2445011785":"2024-03-22T10:13:31.087809+0000","172.21.15.153:6800/3178125243":"2024-03-22T09:56:43.480866+0000","172.21.15.174:6828/1155541738":"2024-03-22T10:08:33.113521+0000","172.21.15.153:6801/3178125243":"2024-03-22T09:56:43.480866+0000","172.21.15.153:6828/866138999":"2024-03-22T10:19:05.021528+0000","172.21.15.153:6800/270837948":"2024-03-22T09:56:00.849533+0000","172.21.15.153:6801/270837948":"2024-03-22T09:56:00.849533+0000","172.21.15.174:0/2577665242":"2024-03-22T10:08:33.113521+0000","172.21.15.153:6800/1639929315":"2024-03-22T10:07:30.531722+0000","172.21.15.153:6801/2524145044":"2024-03-22T10:13:31.087809+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T10:53:12.376 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:12.381+0000 7fc041200640 1 -- 172.21.15.153:0/98305349 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fc02006e490 msgr2=0x7fc020070950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:53:12.377 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:12.381+0000 7fc041200640 1 --2- 172.21.15.153:0/98305349 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fc02006e490 0x7fc020070950 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fc0280058c0 tx=0x7fc028005c60 comp rx=0 tx=0).stop 2024-03-21T10:53:12.377 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:12.381+0000 7fc041200640 1 -- 172.21.15.153:0/98305349 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc03c1000b0 msgr2=0x7fc03c19dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:53:12.377 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:12.381+0000 7fc041200640 1 --2- 172.21.15.153:0/98305349 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc03c1000b0 0x7fc03c19dad0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fc02c009230 tx=0x7fc02c004040 comp rx=0 tx=0).stop 2024-03-21T10:53:12.377 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:12.381+0000 7fc041200640 1 -- 172.21.15.153:0/98305349 shutdown_connections 2024-03-21T10:53:12.377 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:12.381+0000 7fc041200640 1 --2- 172.21.15.153:0/98305349 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fc02006e490 0x7fc020070950 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:12.377 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:12.381+0000 7fc041200640 1 --2- 172.21.15.153:0/98305349 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc03c19e010 0x7fc03c197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:12.377 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:12.381+0000 7fc041200640 1 --2- 172.21.15.153:0/98305349 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fc03c1000b0 0x7fc03c19dad0 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:12.377 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:12.381+0000 7fc041200640 1 -- 172.21.15.153:0/98305349 >> 172.21.15.153:0/98305349 conn(0x7fc03c0fb930 msgr2=0x7fc03c0fd020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:53:12.377 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:12.381+0000 7fc041200640 1 -- 172.21.15.153:0/98305349 shutdown_connections 2024-03-21T10:53:12.377 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:12.381+0000 7fc041200640 1 -- 172.21.15.153:0/98305349 wait complete. 2024-03-21T10:53:12.394 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-21T10:01:18.943639+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-03-21T10:04:25.621358+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.309999942779541, 'score_stable': 1.309999942779541, 'optimal_score': 1, 'raw_score_acting': 1.309999942779541, 'raw_score_stable': 1.309999942779541, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-03-21T10:04:26.593674+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-21T10:53:12.395 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-21T10:53:12.604 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:12.608+0000 7fe273a00640 1 -- 172.21.15.153:0/3327861609 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe26c0fee10 msgr2=0x7fe26c0ff1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:53:12.604 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:12.608+0000 7fe273a00640 1 --2- 172.21.15.153:0/3327861609 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe26c0fee10 0x7fe26c0ff1f0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fe260009880 tx=0x7fe26002bd30 comp rx=0 tx=0).stop 2024-03-21T10:53:12.604 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:12.608+0000 7fe273a00640 1 -- 172.21.15.153:0/3327861609 shutdown_connections 2024-03-21T10:53:12.604 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:12.608+0000 7fe273a00640 1 --2- 172.21.15.153:0/3327861609 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe26c0ff7c0 0x7fe26c10c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:12.605 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:12.608+0000 7fe273a00640 1 --2- 172.21.15.153:0/3327861609 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe26c0fee10 0x7fe26c0ff1f0 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:12.605 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:12.608+0000 7fe273a00640 1 -- 172.21.15.153:0/3327861609 >> 172.21.15.153:0/3327861609 conn(0x7fe26c0faa20 msgr2=0x7fe26c0fce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:53:12.605 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:12.608+0000 7fe273a00640 1 -- 172.21.15.153:0/3327861609 shutdown_connections 2024-03-21T10:53:12.605 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:12.608+0000 7fe273a00640 1 -- 172.21.15.153:0/3327861609 wait complete. 2024-03-21T10:53:12.606 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:12.609+0000 7fe273a00640 1 Processor -- start 2024-03-21T10:53:12.606 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:12.610+0000 7fe273a00640 1 -- start start 2024-03-21T10:53:12.606 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:12.610+0000 7fe273a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe26c0fee10 0x7fe26c197e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:53:12.606 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:12.610+0000 7fe273a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe26c0ff7c0 0x7fe26c1983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:53:12.606 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:12.610+0000 7fe273a00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe26c198a60 con 0x7fe26c0ff7c0 2024-03-21T10:53:12.607 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:12.610+0000 7fe273a00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe26c19c830 con 0x7fe26c0fee10 2024-03-21T10:53:12.607 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:12.610+0000 7fe26be00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe26c0ff7c0 0x7fe26c1983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:53:12.607 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:12.610+0000 7fe26be00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe26c0ff7c0 0x7fe26c1983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:43002/0 (socket says 172.21.15.153:43002) 2024-03-21T10:53:12.607 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:12.610+0000 7fe26be00640 1 -- 172.21.15.153:0/2098667358 learned_addr learned my addr 172.21.15.153:0/2098667358 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:53:12.607 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:12.611+0000 7fe271000640 1 --2- 172.21.15.153:0/2098667358 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe26c0fee10 0x7fe26c197e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:53:12.607 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:12.611+0000 7fe26be00640 1 -- 172.21.15.153:0/2098667358 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe26c0fee10 msgr2=0x7fe26c197e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:53:12.607 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:12.611+0000 7fe26be00640 1 --2- 172.21.15.153:0/2098667358 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe26c0fee10 0x7fe26c197e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:12.607 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:12.611+0000 7fe26be00640 1 -- 172.21.15.153:0/2098667358 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe260009560 con 0x7fe26c0ff7c0 2024-03-21T10:53:12.608 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:12.611+0000 7fe26be00640 1 --2- 172.21.15.153:0/2098667358 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe26c0ff7c0 0x7fe26c1983d0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fe25800d6f0 tx=0x7fe25800dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:53:12.608 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:12.612+0000 7fe269600640 1 -- 172.21.15.153:0/2098667358 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe258009a30 con 0x7fe26c0ff7c0 2024-03-21T10:53:12.608 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:12.612+0000 7fe269600640 1 -- 172.21.15.153:0/2098667358 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe2580054e0 con 0x7fe26c0ff7c0 2024-03-21T10:53:12.608 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:12.612+0000 7fe273a00640 1 -- 172.21.15.153:0/2098667358 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe26c19cb10 con 0x7fe26c0ff7c0 2024-03-21T10:53:12.608 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:12.612+0000 7fe269600640 1 -- 172.21.15.153:0/2098667358 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe258010460 con 0x7fe26c0ff7c0 2024-03-21T10:53:12.608 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:12.612+0000 7fe273a00640 1 -- 172.21.15.153:0/2098667358 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe26c19d070 con 0x7fe26c0ff7c0 2024-03-21T10:53:12.609 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:12.613+0000 7fe273a00640 1 -- 172.21.15.153:0/2098667358 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe26c04e040 con 0x7fe26c0ff7c0 2024-03-21T10:53:12.610 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:12.615+0000 7fe269600640 1 -- 172.21.15.153:0/2098667358 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7fe258009b90 con 0x7fe26c0ff7c0 2024-03-21T10:53:12.611 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:12.616+0000 7fe269600640 1 --2- 172.21.15.153:0/2098667358 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe25406e3d0 0x7fe254070890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:53:12.611 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:12.616+0000 7fe269600640 1 -- 172.21.15.153:0/2098667358 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fe25808dff0 con 0x7fe26c0ff7c0 2024-03-21T10:53:12.611 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:12.616+0000 7fe271000640 1 --2- 172.21.15.153:0/2098667358 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe25406e3d0 0x7fe254070890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:53:12.612 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:12.617+0000 7fe271000640 1 --2- 172.21.15.153:0/2098667358 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe25406e3d0 0x7fe254070890 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fe260006010 tx=0x7fe2600058d0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:53:12.616 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:12.621+0000 7fe269600640 1 -- 172.21.15.153:0/2098667358 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe258057160 con 0x7fe26c0ff7c0 2024-03-21T10:53:12.919 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:12.924+0000 7fe273a00640 1 -- 172.21.15.153:0/2098667358 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe26c10f420 con 0x7fe26c0ff7c0 2024-03-21T10:53:12.920 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:12.924+0000 7fe269600640 1 -- 172.21.15.153:0/2098667358 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v98) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe25805a820 con 0x7fe26c0ff7c0 2024-03-21T10:53:12.920 INFO:teuthology.orchestra.run.smithi153.stdout:pg_num: 1 2024-03-21T10:53:12.923 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:12.927+0000 7fe273a00640 1 -- 172.21.15.153:0/2098667358 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe25406e3d0 msgr2=0x7fe254070890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:53:12.923 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:12.928+0000 7fe273a00640 1 --2- 172.21.15.153:0/2098667358 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe25406e3d0 0x7fe254070890 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fe260006010 tx=0x7fe2600058d0 comp rx=0 tx=0).stop 2024-03-21T10:53:12.923 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:12.928+0000 7fe273a00640 1 -- 172.21.15.153:0/2098667358 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe26c0ff7c0 msgr2=0x7fe26c1983d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:53:12.923 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:12.928+0000 7fe273a00640 1 --2- 172.21.15.153:0/2098667358 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe26c0ff7c0 0x7fe26c1983d0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fe25800d6f0 tx=0x7fe25800dbb0 comp rx=0 tx=0).stop 2024-03-21T10:53:12.924 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:12.928+0000 7fe273a00640 1 -- 172.21.15.153:0/2098667358 shutdown_connections 2024-03-21T10:53:12.924 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:12.928+0000 7fe273a00640 1 --2- 172.21.15.153:0/2098667358 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7fe25406e3d0 0x7fe254070890 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:12.924 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:12.928+0000 7fe273a00640 1 --2- 172.21.15.153:0/2098667358 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7fe26c0ff7c0 0x7fe26c1983d0 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:12.924 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:12.928+0000 7fe273a00640 1 --2- 172.21.15.153:0/2098667358 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe26c0fee10 0x7fe26c197e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:12.924 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:12.928+0000 7fe273a00640 1 -- 172.21.15.153:0/2098667358 >> 172.21.15.153:0/2098667358 conn(0x7fe26c0faa20 msgr2=0x7fe26c0fc080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:53:12.924 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:12.928+0000 7fe273a00640 1 -- 172.21.15.153:0/2098667358 shutdown_connections 2024-03-21T10:53:12.925 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:12.928+0000 7fe273a00640 1 -- 172.21.15.153:0/2098667358 wait complete. 2024-03-21T10:53:12.939 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:53:12 smithi153 ceph-mon[113733]: pgmap v1243: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:53:12.939 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:53:12 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/3246856400' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-21T10:53:12.939 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:53:12 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/98305349' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T10:53:12.942 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-03-21T10:53:13.143 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:13.147+0000 7f52d5200640 1 -- 172.21.15.153:0/836226316 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f52d0105ef0 msgr2=0x7f52d01062d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:53:13.143 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:13.147+0000 7f52d5200640 1 --2- 172.21.15.153:0/836226316 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f52d0105ef0 0x7f52d01062d0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f52c0009880 tx=0x7f52c002bce0 comp rx=0 tx=0).stop 2024-03-21T10:53:13.143 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:13.147+0000 7f52d5200640 1 -- 172.21.15.153:0/836226316 shutdown_connections 2024-03-21T10:53:13.143 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:13.147+0000 7f52d5200640 1 --2- 172.21.15.153:0/836226316 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f52d0067350 0x7f52d0067790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:13.143 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:13.147+0000 7f52d5200640 1 --2- 172.21.15.153:0/836226316 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f52d0105ef0 0x7f52d01062d0 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:13.143 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:13.147+0000 7f52d5200640 1 -- 172.21.15.153:0/836226316 >> 172.21.15.153:0/836226316 conn(0x7f52d0074930 msgr2=0x7f52d0074d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:53:13.143 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:13.147+0000 7f52d5200640 1 -- 172.21.15.153:0/836226316 shutdown_connections 2024-03-21T10:53:13.144 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:13.147+0000 7f52d5200640 1 -- 172.21.15.153:0/836226316 wait complete. 2024-03-21T10:53:13.145 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:13.148+0000 7f52d5200640 1 Processor -- start 2024-03-21T10:53:13.146 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:13.148+0000 7f52d5200640 1 -- start start 2024-03-21T10:53:13.146 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:13.149+0000 7f52d5200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f52d0067350 0x7f52d0197d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:53:13.146 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:13.149+0000 7f52d5200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f52d0105ef0 0x7f52d01982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:53:13.146 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:13.149+0000 7f52d5200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52d0198930 con 0x7f52d0067350 2024-03-21T10:53:13.146 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:13.149+0000 7f52d5200640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52d019c700 con 0x7f52d0105ef0 2024-03-21T10:53:13.146 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:13.149+0000 7f52cf400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f52d0067350 0x7f52d0197d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:53:13.146 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:13.149+0000 7f52cf400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f52d0067350 0x7f52d0197d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39758/0 (socket says 172.21.15.153:39758) 2024-03-21T10:53:13.146 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:13.149+0000 7f52cf400640 1 -- 172.21.15.153:0/3473527156 learned_addr learned my addr 172.21.15.153:0/3473527156 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:53:13.146 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:13.149+0000 7f52cea00640 1 --2- 172.21.15.153:0/3473527156 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f52d0105ef0 0x7f52d01982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:53:13.146 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:13.149+0000 7f52cf400640 1 -- 172.21.15.153:0/3473527156 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f52d0105ef0 msgr2=0x7f52d01982a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:53:13.146 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:13.149+0000 7f52cf400640 1 --2- 172.21.15.153:0/3473527156 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f52d0105ef0 0x7f52d01982a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:13.146 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:13.149+0000 7f52cf400640 1 -- 172.21.15.153:0/3473527156 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52c0009560 con 0x7f52d0067350 2024-03-21T10:53:13.147 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:13.149+0000 7f52cea00640 1 --2- 172.21.15.153:0/3473527156 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f52d0105ef0 0x7f52d01982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:53:13.147 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:13.150+0000 7f52cf400640 1 --2- 172.21.15.153:0/3473527156 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f52d0067350 0x7f52d0197d60 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f52c0000c00 tx=0x7f52c002c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:53:13.147 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:13.150+0000 7f52c7e00640 1 -- 172.21.15.153:0/3473527156 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f52c0004520 con 0x7f52d0067350 2024-03-21T10:53:13.147 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:13.150+0000 7f52d5200640 1 -- 172.21.15.153:0/3473527156 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52d019c980 con 0x7f52d0067350 2024-03-21T10:53:13.147 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:13.150+0000 7f52c7e00640 1 -- 172.21.15.153:0/3473527156 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f52c0003ab0 con 0x7f52d0067350 2024-03-21T10:53:13.147 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:13.150+0000 7f52c7e00640 1 -- 172.21.15.153:0/3473527156 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f52c0034440 con 0x7f52d0067350 2024-03-21T10:53:13.147 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:13.150+0000 7f52d5200640 1 -- 172.21.15.153:0/3473527156 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52d019ce80 con 0x7f52d0067350 2024-03-21T10:53:13.147 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:13.152+0000 7f52d5200640 1 -- 172.21.15.153:0/3473527156 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52d004e040 con 0x7f52d0067350 2024-03-21T10:53:13.148 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:13.153+0000 7f52c7e00640 1 -- 172.21.15.153:0/3473527156 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f52c0003c20 con 0x7f52d0067350 2024-03-21T10:53:13.149 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:13.153+0000 7f52c7e00640 1 --2- 172.21.15.153:0/3473527156 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f52b406e3d0 0x7f52b4070890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:53:13.149 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:13.154+0000 7f52c7e00640 1 -- 172.21.15.153:0/3473527156 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f52c00b0890 con 0x7f52d0067350 2024-03-21T10:53:13.149 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:13.154+0000 7f52cea00640 1 --2- 172.21.15.153:0/3473527156 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f52b406e3d0 0x7f52b4070890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:53:13.150 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:13.155+0000 7f52cea00640 1 --2- 172.21.15.153:0/3473527156 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f52b406e3d0 0x7f52b4070890 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f52d0199370 tx=0x7f52bc005e50 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:53:13.157 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:13.162+0000 7f52c7e00640 1 -- 172.21.15.153:0/3473527156 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f52c0079930 con 0x7f52d0067350 2024-03-21T10:53:13.196 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:53:12 smithi174 ceph-mon[88004]: pgmap v1243: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:53:13.196 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:53:12 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/3246856400' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-21T10:53:13.196 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:53:12 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/98305349' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T10:53:13.357 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:13.360+0000 7f52d5200640 1 -- 172.21.15.153:0/3473527156 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f52d01990b0 con 0x7f52d0067350 2024-03-21T10:53:13.357 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:13.361+0000 7f52c7e00640 1 -- 172.21.15.153:0/3473527156 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v98) v1 ==== 107+0+11 (secure 0 0 0) 0x7f52c004b020 con 0x7f52d0067350 2024-03-21T10:53:13.357 INFO:teuthology.orchestra.run.smithi153.stdout:pg_num: 32 2024-03-21T10:53:13.359 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:13.363+0000 7f52d5200640 1 -- 172.21.15.153:0/3473527156 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f52b406e3d0 msgr2=0x7f52b4070890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:53:13.359 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:13.363+0000 7f52d5200640 1 --2- 172.21.15.153:0/3473527156 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f52b406e3d0 0x7f52b4070890 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f52d0199370 tx=0x7f52bc005e50 comp rx=0 tx=0).stop 2024-03-21T10:53:13.359 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:13.363+0000 7f52d5200640 1 -- 172.21.15.153:0/3473527156 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f52d0067350 msgr2=0x7f52d0197d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:53:13.359 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:13.363+0000 7f52d5200640 1 --2- 172.21.15.153:0/3473527156 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f52d0067350 0x7f52d0197d60 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f52c0000c00 tx=0x7f52c002c850 comp rx=0 tx=0).stop 2024-03-21T10:53:13.359 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:13.364+0000 7f52d5200640 1 -- 172.21.15.153:0/3473527156 shutdown_connections 2024-03-21T10:53:13.359 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:13.364+0000 7f52d5200640 1 --2- 172.21.15.153:0/3473527156 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f52b406e3d0 0x7f52b4070890 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:13.360 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:13.364+0000 7f52d5200640 1 --2- 172.21.15.153:0/3473527156 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f52d0105ef0 0x7f52d01982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:13.360 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:13.364+0000 7f52d5200640 1 --2- 172.21.15.153:0/3473527156 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f52d0067350 0x7f52d0197d60 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:13.360 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:13.364+0000 7f52d5200640 1 -- 172.21.15.153:0/3473527156 >> 172.21.15.153:0/3473527156 conn(0x7f52d0074930 msgr2=0x7f52d00fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:53:13.360 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:13.364+0000 7f52d5200640 1 -- 172.21.15.153:0/3473527156 shutdown_connections 2024-03-21T10:53:13.360 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:13.364+0000 7f52d5200640 1 -- 172.21.15.153:0/3473527156 wait complete. 2024-03-21T10:53:13.377 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-03-21T10:53:13.578 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:13.581+0000 7f20b8a00640 1 -- 172.21.15.153:0/3118106911 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f20b41000e0 msgr2=0x7f20b41004c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:53:13.578 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:13.581+0000 7f20b8a00640 1 --2- 172.21.15.153:0/3118106911 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f20b41000e0 0x7f20b41004c0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f20a4009880 tx=0x7f20a402bce0 comp rx=0 tx=0).stop 2024-03-21T10:53:13.578 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:13.582+0000 7f20b8a00640 1 -- 172.21.15.153:0/3118106911 shutdown_connections 2024-03-21T10:53:13.578 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:13.582+0000 7f20b8a00640 1 --2- 172.21.15.153:0/3118106911 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f20b4100a90 0x7f20b4104ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:13.578 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:13.582+0000 7f20b8a00640 1 --2- 172.21.15.153:0/3118106911 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f20b41000e0 0x7f20b41004c0 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:13.578 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:13.582+0000 7f20b8a00640 1 -- 172.21.15.153:0/3118106911 >> 172.21.15.153:0/3118106911 conn(0x7f20b40fb960 msgr2=0x7f20b40fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:53:13.578 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:13.582+0000 7f20b8a00640 1 -- 172.21.15.153:0/3118106911 shutdown_connections 2024-03-21T10:53:13.578 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:13.582+0000 7f20b8a00640 1 -- 172.21.15.153:0/3118106911 wait complete. 2024-03-21T10:53:13.579 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:13.583+0000 7f20b8a00640 1 Processor -- start 2024-03-21T10:53:13.580 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:13.583+0000 7f20b8a00640 1 -- start start 2024-03-21T10:53:13.580 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:13.584+0000 7f20b8a00640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f20b41000e0 0x7f20b4197dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:53:13.580 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:13.584+0000 7f20b8a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f20b4198300 0x7f20b419c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:53:13.580 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:13.584+0000 7f20b8a00640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20b4198900 con 0x7f20b41000e0 2024-03-21T10:53:13.580 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:13.584+0000 7f20b8a00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20b4198a70 con 0x7f20b4198300 2024-03-21T10:53:13.580 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:13.584+0000 7f20b3400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f20b41000e0 0x7f20b4197dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:53:13.580 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:13.584+0000 7f20b3400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f20b41000e0 0x7f20b4197dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39762/0 (socket says 172.21.15.153:39762) 2024-03-21T10:53:13.580 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:13.584+0000 7f20b3400640 1 -- 172.21.15.153:0/1746025018 learned_addr learned my addr 172.21.15.153:0/1746025018 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:53:13.582 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:13.585+0000 7f20b3400640 1 -- 172.21.15.153:0/1746025018 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f20b4198300 msgr2=0x7f20b419c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:53:13.582 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:13.585+0000 7f20b2a00640 1 --2- 172.21.15.153:0/1746025018 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f20b4198300 0x7f20b419c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:53:13.582 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:13.585+0000 7f20b3400640 1 --2- 172.21.15.153:0/1746025018 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f20b4198300 0x7f20b419c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:13.582 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:13.585+0000 7f20b3400640 1 -- 172.21.15.153:0/1746025018 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20a4009560 con 0x7f20b41000e0 2024-03-21T10:53:13.582 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:13.585+0000 7f20b2a00640 1 --2- 172.21.15.153:0/1746025018 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f20b4198300 0x7f20b419c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T10:53:13.582 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:13.585+0000 7f20b3400640 1 --2- 172.21.15.153:0/1746025018 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f20b41000e0 0x7f20b4197dc0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f20a4036040 tx=0x7f20a402c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:53:13.582 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:13.585+0000 7f20abe00640 1 -- 172.21.15.153:0/1746025018 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f20a4004520 con 0x7f20b41000e0 2024-03-21T10:53:13.582 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:13.585+0000 7f20abe00640 1 -- 172.21.15.153:0/1746025018 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f20a4003ab0 con 0x7f20b41000e0 2024-03-21T10:53:13.582 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:13.585+0000 7f20abe00640 1 -- 172.21.15.153:0/1746025018 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f20a4034440 con 0x7f20b41000e0 2024-03-21T10:53:13.583 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:13.585+0000 7f20b8a00640 1 -- 172.21.15.153:0/1746025018 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20b419cd00 con 0x7f20b41000e0 2024-03-21T10:53:13.583 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:13.586+0000 7f20b8a00640 1 -- 172.21.15.153:0/1746025018 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20b419d260 con 0x7f20b41000e0 2024-03-21T10:53:13.583 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:13.587+0000 7f20b8a00640 1 -- 172.21.15.153:0/1746025018 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20b404e040 con 0x7f20b41000e0 2024-03-21T10:53:13.583 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:13.588+0000 7f20abe00640 1 -- 172.21.15.153:0/1746025018 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f20a4003c20 con 0x7f20b41000e0 2024-03-21T10:53:13.585 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:13.589+0000 7f20abe00640 1 --2- 172.21.15.153:0/1746025018 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f209806e3d0 0x7f2098070890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:53:13.585 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:13.589+0000 7f20abe00640 1 -- 172.21.15.153:0/1746025018 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f20a40b0870 con 0x7f20b41000e0 2024-03-21T10:53:13.585 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:13.590+0000 7f20b2a00640 1 --2- 172.21.15.153:0/1746025018 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f209806e3d0 0x7f2098070890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:53:13.586 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:13.590+0000 7f20b2a00640 1 --2- 172.21.15.153:0/1746025018 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f209806e3d0 0x7f2098070890 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f20b4067ad0 tx=0x7f20a0005e50 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:53:13.589 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:13.594+0000 7f20abe00640 1 -- 172.21.15.153:0/1746025018 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f20a4079910 con 0x7f20b41000e0 2024-03-21T10:53:13.812 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:13.817+0000 7f20b8a00640 1 -- 172.21.15.153:0/1746025018 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7f20b41041c0 con 0x7f20b41000e0 2024-03-21T10:53:13.813 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:13.817+0000 7f20abe00640 1 -- 172.21.15.153:0/1746025018 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v98) v1 ==== 107+0+11 (secure 0 0 0) 0x7f20a404b020 con 0x7f20b41000e0 2024-03-21T10:53:13.813 INFO:teuthology.orchestra.run.smithi153.stdout:pg_num: 32 2024-03-21T10:53:13.816 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:13.820+0000 7f20b8a00640 1 -- 172.21.15.153:0/1746025018 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f209806e3d0 msgr2=0x7f2098070890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:53:13.816 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:13.820+0000 7f20b8a00640 1 --2- 172.21.15.153:0/1746025018 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f209806e3d0 0x7f2098070890 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f20b4067ad0 tx=0x7f20a0005e50 comp rx=0 tx=0).stop 2024-03-21T10:53:13.816 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:13.820+0000 7f20b8a00640 1 -- 172.21.15.153:0/1746025018 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f20b41000e0 msgr2=0x7f20b4197dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:53:13.816 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:13.820+0000 7f20b8a00640 1 --2- 172.21.15.153:0/1746025018 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f20b41000e0 0x7f20b4197dc0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f20a4036040 tx=0x7f20a402c850 comp rx=0 tx=0).stop 2024-03-21T10:53:13.816 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:13.820+0000 7f20b8a00640 1 -- 172.21.15.153:0/1746025018 shutdown_connections 2024-03-21T10:53:13.816 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:13.820+0000 7f20b8a00640 1 --2- 172.21.15.153:0/1746025018 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f209806e3d0 0x7f2098070890 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:13.816 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:13.820+0000 7f20b8a00640 1 --2- 172.21.15.153:0/1746025018 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f20b4198300 0x7f20b419c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:13.816 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:13.820+0000 7f20b8a00640 1 --2- 172.21.15.153:0/1746025018 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f20b41000e0 0x7f20b4197dc0 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:13.816 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:13.820+0000 7f20b8a00640 1 -- 172.21.15.153:0/1746025018 >> 172.21.15.153:0/1746025018 conn(0x7f20b40fb960 msgr2=0x7f20b40fd0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:53:13.816 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:13.820+0000 7f20b8a00640 1 -- 172.21.15.153:0/1746025018 shutdown_connections 2024-03-21T10:53:13.816 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:13.821+0000 7f20b8a00640 1 -- 172.21.15.153:0/1746025018 wait complete. 2024-03-21T10:53:13.838 DEBUG:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-21T10:53:14.051 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:53:13 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/2098667358' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T10:53:14.051 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:53:13 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/3473527156' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-21T10:53:14.062 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:14.066+0000 7f224b200640 1 -- 172.21.15.153:0/3201792160 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f224c072ff0 msgr2=0x7f224c10c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:53:14.062 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:14.066+0000 7f224b200640 1 --2- 172.21.15.153:0/3201792160 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f224c072ff0 0x7f224c10c170 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f2238009880 tx=0x7f223802bd70 comp rx=0 tx=0).stop 2024-03-21T10:53:14.062 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:14.066+0000 7f224b200640 1 -- 172.21.15.153:0/3201792160 shutdown_connections 2024-03-21T10:53:14.062 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:14.066+0000 7f224b200640 1 --2- 172.21.15.153:0/3201792160 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f224c072ff0 0x7f224c10c170 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:14.062 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:14.066+0000 7f224b200640 1 --2- 172.21.15.153:0/3201792160 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f224c0726d0 0x7f224c072ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:14.062 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:14.066+0000 7f224b200640 1 -- 172.21.15.153:0/3201792160 >> 172.21.15.153:0/3201792160 conn(0x7f224c0fb960 msgr2=0x7f224c0fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:53:14.062 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:14.066+0000 7f224b200640 1 -- 172.21.15.153:0/3201792160 shutdown_connections 2024-03-21T10:53:14.062 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:14.066+0000 7f224b200640 1 -- 172.21.15.153:0/3201792160 wait complete. 2024-03-21T10:53:14.063 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:14.067+0000 7f224b200640 1 Processor -- start 2024-03-21T10:53:14.063 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:14.068+0000 7f224b200640 1 -- start start 2024-03-21T10:53:14.063 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:14.068+0000 7f224b200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f224c0726d0 0x7f224c197cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:53:14.063 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:14.068+0000 7f224b200640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f224c072ff0 0x7f224c198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:53:14.063 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:14.068+0000 7f224b200640 1 -- --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f224c1988f0 con 0x7f224c072ff0 2024-03-21T10:53:14.064 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:14.068+0000 7f224b200640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f224c19b5c0 con 0x7f224c0726d0 2024-03-21T10:53:14.065 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:14.068+0000 7f2249400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f224c072ff0 0x7f224c198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:53:14.065 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:14.068+0000 7f2249400640 1 --2- >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f224c072ff0 0x7f224c198210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.153:3300/0 says I am v2:172.21.15.153:39784/0 (socket says 172.21.15.153:39784) 2024-03-21T10:53:14.065 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:14.068+0000 7f2249400640 1 -- 172.21.15.153:0/2940107250 learned_addr learned my addr 172.21.15.153:0/2940107250 (peer_addr_for_me v2:172.21.15.153:0/0) 2024-03-21T10:53:14.065 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:14.069+0000 7f2249e00640 1 --2- 172.21.15.153:0/2940107250 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f224c0726d0 0x7f224c197cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:53:14.065 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:14.069+0000 7f2249400640 1 -- 172.21.15.153:0/2940107250 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f224c0726d0 msgr2=0x7f224c197cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:53:14.065 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:14.069+0000 7f2249400640 1 --2- 172.21.15.153:0/2940107250 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f224c0726d0 0x7f224c197cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:14.065 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:14.069+0000 7f2249400640 1 -- 172.21.15.153:0/2940107250 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2238009560 con 0x7f224c072ff0 2024-03-21T10:53:14.065 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:14.069+0000 7f2249400640 1 --2- 172.21.15.153:0/2940107250 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f224c072ff0 0x7f224c198210 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f2238006010 tx=0x7f223802c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:53:14.066 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:14.069+0000 7f2242a00640 1 -- 172.21.15.153:0/2940107250 <== mon.0 v2:172.21.15.153:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f22380035f0 con 0x7f224c072ff0 2024-03-21T10:53:14.066 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:14.069+0000 7f2242a00640 1 -- 172.21.15.153:0/2940107250 <== mon.0 v2:172.21.15.153:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2238003c10 con 0x7f224c072ff0 2024-03-21T10:53:14.066 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:14.070+0000 7f224b200640 1 -- 172.21.15.153:0/2940107250 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f224c19b840 con 0x7f224c072ff0 2024-03-21T10:53:14.066 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:14.070+0000 7f2242a00640 1 -- 172.21.15.153:0/2940107250 <== mon.0 v2:172.21.15.153:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2238005590 con 0x7f224c072ff0 2024-03-21T10:53:14.066 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:14.070+0000 7f224b200640 1 -- 172.21.15.153:0/2940107250 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f224c19bda0 con 0x7f224c072ff0 2024-03-21T10:53:14.066 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:14.071+0000 7f224b200640 1 -- 172.21.15.153:0/2940107250 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f224c04e890 con 0x7f224c072ff0 2024-03-21T10:53:14.067 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:14.072+0000 7f2242a00640 1 -- 172.21.15.153:0/2940107250 <== mon.0 v2:172.21.15.153:3300/0 4 ==== mgrmap(e 38) v1 ==== 92219+0+0 (secure 0 0 0) 0x7f2238004dd0 con 0x7f224c072ff0 2024-03-21T10:53:14.068 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:14.073+0000 7f2242a00640 1 --2- 172.21.15.153:0/2940107250 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f223006e420 0x7f22300708e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T10:53:14.069 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:14.073+0000 7f2242a00640 1 -- 172.21.15.153:0/2940107250 <== mon.0 v2:172.21.15.153:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f22380b0fb0 con 0x7f224c072ff0 2024-03-21T10:53:14.070 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:14.074+0000 7f2249e00640 1 --2- 172.21.15.153:0/2940107250 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f223006e420 0x7f22300708e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T10:53:14.070 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:14.074+0000 7f2249e00640 1 --2- 172.21.15.153:0/2940107250 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f223006e420 0x7f22300708e0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f223c005fd0 tx=0x7f223c005850 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T10:53:14.074 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:14.078+0000 7f2242a00640 1 -- 172.21.15.153:0/2940107250 <== mon.0 v2:172.21.15.153:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f223807a0f0 con 0x7f224c072ff0 2024-03-21T10:53:14.195 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:53:13 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/2098667358' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T10:53:14.196 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:53:13 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/3473527156' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-21T10:53:14.287 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:14.291+0000 7f224b200640 1 -- 172.21.15.153:0/2940107250 --> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f224c1097e0 con 0x7f224c072ff0 2024-03-21T10:53:14.287 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:14.292+0000 7f2242a00640 1 -- 172.21.15.153:0/2940107250 <== mon.0 v2:172.21.15.153:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 41 entries v98) v1 ==== 81+0+2462 (secure 0 0 0) 0x7f22380439b0 con 0x7f224c072ff0 2024-03-21T10:53:14.288 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.15.174:6825/1776947618 2024-03-22T10:19:05.021561+0000 2024-03-21T10:53:14.288 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.15.174:6824/1776947618 2024-03-22T10:19:05.021561+0000 2024-03-21T10:53:14.288 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.15.153:6829/866138999 2024-03-22T10:19:05.021528+0000 2024-03-21T10:53:14.288 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.15.153:0/3553873361 2024-03-22T10:13:31.087809+0000 2024-03-21T10:53:14.288 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.15.174:0/1056981865 2024-03-22T10:08:33.113521+0000 2024-03-21T10:53:14.288 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.15.153:0/2407536999 2024-03-22T09:56:00.849533+0000 2024-03-21T10:53:14.288 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.15.153:0/2969012027 2024-03-22T10:07:30.531722+0000 2024-03-21T10:53:14.288 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.15.153:6826/1353876997 2024-03-22T10:04:45.049156+0000 2024-03-21T10:53:14.288 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.15.153:0/623594592 2024-03-22T09:56:00.849533+0000 2024-03-21T10:53:14.288 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.15.153:0/1163205431 2024-03-22T09:56:43.480866+0000 2024-03-21T10:53:14.288 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.15.153:0/3278686288 2024-03-22T09:58:24.250138+0000 2024-03-21T10:53:14.288 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.15.153:0/652104290 2024-03-22T09:58:24.250138+0000 2024-03-21T10:53:14.288 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.15.153:6800/2524145044 2024-03-22T10:13:31.087809+0000 2024-03-21T10:53:14.289 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.15.174:0/3456690749 2024-03-22T10:08:33.113521+0000 2024-03-21T10:53:14.289 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.15.153:0/4237751158 2024-03-22T10:07:30.531722+0000 2024-03-21T10:53:14.289 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.15.153:0/3456116624 2024-03-22T09:56:43.480866+0000 2024-03-21T10:53:14.289 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.15.153:0/3342434133 2024-03-22T10:13:31.087809+0000 2024-03-21T10:53:14.289 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.15.153:0/2801644471 2024-03-22T10:07:30.531722+0000 2024-03-21T10:53:14.289 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.15.153:6801/1639929315 2024-03-22T10:07:30.531722+0000 2024-03-21T10:53:14.289 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.15.174:0/950700094 2024-03-22T10:08:33.113521+0000 2024-03-21T10:53:14.289 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.15.153:6827/1353876997 2024-03-22T10:04:45.049156+0000 2024-03-21T10:53:14.289 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.15.174:0/2227377986 2024-03-22T10:08:33.113521+0000 2024-03-21T10:53:14.289 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.15.153:6800/1143522828 2024-03-22T09:58:24.250138+0000 2024-03-21T10:53:14.289 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.15.153:0/3435678189 2024-03-22T09:56:00.849533+0000 2024-03-21T10:53:14.289 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.15.153:0/2431412876 2024-03-22T10:07:30.531722+0000 2024-03-21T10:53:14.289 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.15.153:6801/1143522828 2024-03-22T09:58:24.250138+0000 2024-03-21T10:53:14.289 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.15.174:6829/1155541738 2024-03-22T10:08:33.113521+0000 2024-03-21T10:53:14.290 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.15.153:0/517714023 2024-03-22T10:13:31.087809+0000 2024-03-21T10:53:14.290 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.15.174:0/828964242 2024-03-22T10:08:33.113521+0000 2024-03-21T10:53:14.290 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.15.153:0/405164318 2024-03-22T09:58:24.250138+0000 2024-03-21T10:53:14.290 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.15.153:0/816489762 2024-03-22T09:56:43.480866+0000 2024-03-21T10:53:14.290 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.15.153:0/2445011785 2024-03-22T10:13:31.087809+0000 2024-03-21T10:53:14.290 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.15.153:6800/3178125243 2024-03-22T09:56:43.480866+0000 2024-03-21T10:53:14.290 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.15.174:6828/1155541738 2024-03-22T10:08:33.113521+0000 2024-03-21T10:53:14.290 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.15.153:6801/3178125243 2024-03-22T09:56:43.480866+0000 2024-03-21T10:53:14.290 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.15.153:6828/866138999 2024-03-22T10:19:05.021528+0000 2024-03-21T10:53:14.290 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.15.153:6800/270837948 2024-03-22T09:56:00.849533+0000 2024-03-21T10:53:14.290 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.15.153:6801/270837948 2024-03-22T09:56:00.849533+0000 2024-03-21T10:53:14.290 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.15.174:0/2577665242 2024-03-22T10:08:33.113521+0000 2024-03-21T10:53:14.290 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.15.153:6800/1639929315 2024-03-22T10:07:30.531722+0000 2024-03-21T10:53:14.291 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.15.153:6801/2524145044 2024-03-22T10:13:31.087809+0000 2024-03-21T10:53:14.292 INFO:teuthology.orchestra.run.smithi153.stderr:listed 41 entries 2024-03-21T10:53:14.292 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:14.295+0000 7f224b200640 1 -- 172.21.15.153:0/2940107250 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f223006e420 msgr2=0x7f22300708e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:53:14.292 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:14.295+0000 7f224b200640 1 --2- 172.21.15.153:0/2940107250 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f223006e420 0x7f22300708e0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f223c005fd0 tx=0x7f223c005850 comp rx=0 tx=0).stop 2024-03-21T10:53:14.292 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:14.295+0000 7f224b200640 1 -- 172.21.15.153:0/2940107250 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f224c072ff0 msgr2=0x7f224c198210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T10:53:14.292 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:14.295+0000 7f224b200640 1 --2- 172.21.15.153:0/2940107250 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f224c072ff0 0x7f224c198210 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f2238006010 tx=0x7f223802c8e0 comp rx=0 tx=0).stop 2024-03-21T10:53:14.292 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:14.295+0000 7f224b200640 1 -- 172.21.15.153:0/2940107250 shutdown_connections 2024-03-21T10:53:14.292 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:14.295+0000 7f224b200640 1 --2- 172.21.15.153:0/2940107250 >> [v2:172.21.15.153:6800/3112920964,v1:172.21.15.153:6801/3112920964] conn(0x7f223006e420 0x7f22300708e0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:14.292 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:14.295+0000 7f224b200640 1 --2- 172.21.15.153:0/2940107250 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x7f224c072ff0 0x7f224c198210 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:14.292 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:14.295+0000 7f224b200640 1 --2- 172.21.15.153:0/2940107250 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f224c0726d0 0x7f224c197cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T10:53:14.292 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:14.295+0000 7f224b200640 1 -- 172.21.15.153:0/2940107250 >> 172.21.15.153:0/2940107250 conn(0x7f224c0fb960 msgr2=0x7f224c1069b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T10:53:14.292 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:14.296+0000 7f224b200640 1 -- 172.21.15.153:0/2940107250 shutdown_connections 2024-03-21T10:53:14.292 INFO:teuthology.orchestra.run.smithi153.stderr:2024-03-21T10:53:14.296+0000 7f224b200640 1 -- 172.21.15.153:0/2940107250 wait complete. 2024-03-21T10:53:14.316 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi153.front.sepia.ceph.com... 2024-03-21T10:53:14.316 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T10:53:14.317 DEBUG:teuthology.orchestra.run.smithi153:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-03-21T10:53:14.403 INFO:teuthology.orchestra.run:waiting for 300 2024-03-21T10:53:15.051 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:53:14 smithi153 ceph-mon[113733]: pgmap v1244: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:53:15.051 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:53:14 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/1746025018' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-21T10:53:15.052 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:53:14 smithi153 ceph-mon[113733]: from='client.? 172.21.15.153:0/2940107250' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-21T10:53:15.196 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:53:14 smithi174 ceph-mon[88004]: pgmap v1244: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:53:15.196 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:53:14 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/1746025018' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-21T10:53:15.197 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:53:14 smithi174 ceph-mon[88004]: from='client.? 172.21.15.153:0/2940107250' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-21T10:53:17.196 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:53:16 smithi174 ceph-mon[88004]: pgmap v1245: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:53:17.196 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:53:16 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:53:17.301 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:53:16 smithi153 ceph-mon[113733]: pgmap v1245: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:53:17.302 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:53:16 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:53:19.196 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:53:18 smithi174 ceph-mon[88004]: pgmap v1246: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:53:19.301 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:53:18 smithi153 ceph-mon[113733]: pgmap v1246: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:53:21.196 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:53:20 smithi174 ceph-mon[88004]: pgmap v1247: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:53:21.301 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:53:20 smithi153 ceph-mon[113733]: pgmap v1247: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:53:23.051 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:53:22 smithi153 ceph-mon[113733]: pgmap v1248: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:53:23.196 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:53:22 smithi174 ceph-mon[88004]: pgmap v1248: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:53:25.196 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:53:24 smithi174 ceph-mon[88004]: pgmap v1249: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:53:25.288 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:53:24 smithi153 ceph-mon[113733]: pgmap v1249: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:53:27.196 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:53:26 smithi174 ceph-mon[88004]: pgmap v1250: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:53:27.301 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:53:26 smithi153 ceph-mon[113733]: pgmap v1250: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:53:29.161 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:53:28 smithi153 ceph-mon[113733]: pgmap v1251: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:53:29.196 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:53:28 smithi174 ceph-mon[88004]: pgmap v1251: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:53:31.196 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:53:30 smithi174 ceph-mon[88004]: pgmap v1252: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:53:31.302 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:53:30 smithi153 ceph-mon[113733]: pgmap v1252: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:53:32.196 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:53:31 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:53:32.301 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:53:31 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:53:33.196 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:53:32 smithi174 ceph-mon[88004]: pgmap v1253: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:53:33.301 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:53:32 smithi153 ceph-mon[113733]: pgmap v1253: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:53:35.051 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:53:34 smithi153 ceph-mon[113733]: pgmap v1254: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:53:35.196 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:53:34 smithi174 ceph-mon[88004]: pgmap v1254: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:53:37.196 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:53:36 smithi174 ceph-mon[88004]: pgmap v1255: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:53:37.302 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:53:36 smithi153 ceph-mon[113733]: pgmap v1255: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:53:39.197 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:53:38 smithi174 ceph-mon[88004]: pgmap v1256: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:53:39.301 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:53:38 smithi153 ceph-mon[113733]: pgmap v1256: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:53:41.196 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:53:40 smithi174 ceph-mon[88004]: pgmap v1257: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:53:41.301 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:53:40 smithi153 ceph-mon[113733]: pgmap v1257: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:53:43.301 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:53:42 smithi153 ceph-mon[113733]: pgmap v1258: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:53:43.446 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:53:42 smithi174 ceph-mon[88004]: pgmap v1258: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:53:45.051 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:53:44 smithi153 ceph-mon[113733]: pgmap v1259: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:53:45.212 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:53:44 smithi174 ceph-mon[88004]: pgmap v1259: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:53:47.301 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:53:46 smithi153 ceph-mon[113733]: pgmap v1260: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:53:47.302 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:53:46 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:53:47.446 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:53:46 smithi174 ceph-mon[88004]: pgmap v1260: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:53:47.446 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:53:46 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:53:49.301 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:53:48 smithi153 ceph-mon[113733]: pgmap v1261: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:53:49.446 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:53:48 smithi174 ceph-mon[88004]: pgmap v1261: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:53:51.301 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:53:50 smithi153 ceph-mon[113733]: pgmap v1262: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:53:51.446 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:53:50 smithi174 ceph-mon[88004]: pgmap v1262: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:53:53.301 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:53:52 smithi153 ceph-mon[113733]: pgmap v1263: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:53:53.446 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:53:52 smithi174 ceph-mon[88004]: pgmap v1263: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:53:55.301 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:53:54 smithi153 ceph-mon[113733]: pgmap v1264: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:53:55.446 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:53:54 smithi174 ceph-mon[88004]: pgmap v1264: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:53:57.301 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:53:56 smithi153 ceph-mon[113733]: pgmap v1265: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:53:57.446 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:53:56 smithi174 ceph-mon[88004]: pgmap v1265: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:53:59.301 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:53:59 smithi153 ceph-mon[113733]: pgmap v1266: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:53:59.446 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:53:59 smithi174 ceph-mon[88004]: pgmap v1266: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:54:01.301 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:54:01 smithi153 ceph-mon[113733]: pgmap v1267: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:54:01.446 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:54:01 smithi174 ceph-mon[88004]: pgmap v1267: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:54:02.301 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:54:02 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:54:02.446 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:54:02 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:54:03.301 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:54:03 smithi153 ceph-mon[113733]: pgmap v1268: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:54:03.446 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:54:03 smithi174 ceph-mon[88004]: pgmap v1268: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:54:05.301 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:54:05 smithi153 ceph-mon[113733]: pgmap v1269: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:54:05.446 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:54:05 smithi174 ceph-mon[88004]: pgmap v1269: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:54:07.301 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:54:07 smithi153 ceph-mon[113733]: pgmap v1270: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:54:07.302 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:54:07 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:54:07.446 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:54:07 smithi174 ceph-mon[88004]: pgmap v1270: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:54:07.446 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:54:07 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:54:09.301 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:54:09 smithi153 ceph-mon[113733]: pgmap v1271: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:54:09.302 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:54:09 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:54:09.302 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:54:09 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:54:09.302 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:54:09 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:54:09.446 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:54:09 smithi174 ceph-mon[88004]: pgmap v1271: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:54:09.447 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:54:09 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:54:09.447 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:54:09 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:54:09.447 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:54:09 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:54:11.446 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:54:11 smithi174 ceph-mon[88004]: pgmap v1272: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:54:11.551 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:54:11 smithi153 ceph-mon[113733]: pgmap v1272: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:54:13.446 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:54:13 smithi174 ceph-mon[88004]: pgmap v1273: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:54:13.551 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:54:13 smithi153 ceph-mon[113733]: pgmap v1273: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:54:15.446 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:54:15 smithi174 ceph-mon[88004]: pgmap v1274: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:54:15.551 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:54:15 smithi153 ceph-mon[113733]: pgmap v1274: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:54:17.446 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:54:17 smithi174 ceph-mon[88004]: pgmap v1275: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:54:17.447 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:54:17 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:54:17.551 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:54:17 smithi153 ceph-mon[113733]: pgmap v1275: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:54:17.551 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:54:17 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:54:19.446 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:54:19 smithi174 ceph-mon[88004]: pgmap v1276: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:54:19.551 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:54:19 smithi153 ceph-mon[113733]: pgmap v1276: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:54:21.301 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:54:21 smithi153 ceph-mon[113733]: pgmap v1277: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:54:21.390 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:54:21 smithi174 ceph-mon[88004]: pgmap v1277: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:54:23.446 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:54:23 smithi174 ceph-mon[88004]: pgmap v1278: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:54:23.551 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:54:23 smithi153 ceph-mon[113733]: pgmap v1278: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:54:25.446 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:54:25 smithi174 ceph-mon[88004]: pgmap v1279: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:54:25.551 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:54:25 smithi153 ceph-mon[113733]: pgmap v1279: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:54:27.446 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:54:27 smithi174 ceph-mon[88004]: pgmap v1280: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:54:27.551 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:54:27 smithi153 ceph-mon[113733]: pgmap v1280: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:54:29.446 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:54:29 smithi174 ceph-mon[88004]: pgmap v1281: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:54:29.551 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:54:29 smithi153 ceph-mon[113733]: pgmap v1281: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:54:31.446 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:54:31 smithi174 ceph-mon[88004]: pgmap v1282: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:54:31.551 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:54:31 smithi153 ceph-mon[113733]: pgmap v1282: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:54:32.446 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:54:32 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:54:32.551 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:54:32 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:54:33.446 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:54:33 smithi174 ceph-mon[88004]: pgmap v1283: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:54:33.551 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:54:33 smithi153 ceph-mon[113733]: pgmap v1283: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:54:35.446 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:54:35 smithi174 ceph-mon[88004]: pgmap v1284: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:54:35.551 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:54:35 smithi153 ceph-mon[113733]: pgmap v1284: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:54:37.196 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:54:37 smithi174 ceph-mon[88004]: pgmap v1285: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:54:37.551 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:54:37 smithi153 ceph-mon[113733]: pgmap v1285: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:54:39.446 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:54:39 smithi174 ceph-mon[88004]: pgmap v1286: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:54:39.551 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:54:39 smithi153 ceph-mon[113733]: pgmap v1286: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:54:41.446 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:54:41 smithi174 ceph-mon[88004]: pgmap v1287: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:54:41.551 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:54:41 smithi153 ceph-mon[113733]: pgmap v1287: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:54:43.446 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:54:43 smithi174 ceph-mon[88004]: pgmap v1288: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:54:43.551 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:54:43 smithi153 ceph-mon[113733]: pgmap v1288: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:54:45.446 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:54:45 smithi174 ceph-mon[88004]: pgmap v1289: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:54:45.551 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:54:45 smithi153 ceph-mon[113733]: pgmap v1289: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:54:47.446 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:54:47 smithi174 ceph-mon[88004]: pgmap v1290: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:54:47.447 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:54:47 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:54:47.551 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:54:47 smithi153 ceph-mon[113733]: pgmap v1290: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:54:47.551 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:54:47 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:54:49.446 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:54:49 smithi174 ceph-mon[88004]: pgmap v1291: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:54:49.551 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:54:49 smithi153 ceph-mon[113733]: pgmap v1291: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:54:51.446 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:54:51 smithi174 ceph-mon[88004]: pgmap v1292: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:54:51.551 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:54:51 smithi153 ceph-mon[113733]: pgmap v1292: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:54:53.446 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:54:53 smithi174 ceph-mon[88004]: pgmap v1293: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:54:53.551 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:54:53 smithi153 ceph-mon[113733]: pgmap v1293: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:54:55.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:54:55 smithi153 ceph-mon[113733]: pgmap v1294: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:54:55.696 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:54:55 smithi174 ceph-mon[88004]: pgmap v1294: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:54:57.551 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:54:57 smithi153 ceph-mon[113733]: pgmap v1295: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-21T10:54:57.575 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:54:57 smithi174 ceph-mon[88004]: pgmap v1295: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-21T10:54:59.551 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:54:59 smithi153 ceph-mon[113733]: pgmap v1296: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-21T10:54:59.696 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:54:59 smithi174 ceph-mon[88004]: pgmap v1296: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-21T10:55:01.551 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:55:01 smithi153 ceph-mon[113733]: pgmap v1297: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-03-21T10:55:01.696 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:55:01 smithi174 ceph-mon[88004]: pgmap v1297: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-03-21T10:55:02.551 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:55:02 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:55:02.696 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:55:02 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:55:03.551 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:55:03 smithi153 ceph-mon[113733]: pgmap v1298: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 4 op/s 2024-03-21T10:55:03.696 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:55:03 smithi174 ceph-mon[88004]: pgmap v1298: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 4 op/s 2024-03-21T10:55:05.551 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:55:05 smithi153 ceph-mon[113733]: pgmap v1299: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-21T10:55:05.696 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:55:05 smithi174 ceph-mon[88004]: pgmap v1299: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-21T10:55:07.551 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:55:07 smithi153 ceph-mon[113733]: pgmap v1300: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-21T10:55:07.696 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:55:07 smithi174 ceph-mon[88004]: pgmap v1300: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-21T10:55:09.551 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:55:09 smithi153 ceph-mon[113733]: pgmap v1301: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:55:09.551 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:55:09 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:55:09.696 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:55:09 smithi174 ceph-mon[88004]: pgmap v1301: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:55:09.696 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:55:09 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:55:10.551 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:55:10 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:55:10.551 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:55:10 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:55:10.551 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:55:10 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:55:10.696 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:55:10 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:55:10.696 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:55:10 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:55:10.697 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:55:10 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:55:11.946 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:55:11 smithi174 ceph-mon[88004]: pgmap v1302: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:55:12.051 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:55:11 smithi153 ceph-mon[113733]: pgmap v1302: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:55:13.949 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:55:13 smithi174 ceph-mon[88004]: pgmap v1303: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:55:14.051 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:55:13 smithi153 ceph-mon[113733]: pgmap v1303: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:55:15.946 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:55:15 smithi174 ceph-mon[88004]: pgmap v1304: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:55:16.051 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:55:15 smithi153 ceph-mon[113733]: pgmap v1304: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:55:16.946 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:55:16 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:55:17.051 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:55:16 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:55:17.946 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:55:17 smithi174 ceph-mon[88004]: pgmap v1305: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:55:18.006 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:55:17 smithi153 ceph-mon[113733]: pgmap v1305: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:55:20.051 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:55:19 smithi153 ceph-mon[113733]: pgmap v1306: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:55:20.196 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:55:19 smithi174 ceph-mon[88004]: pgmap v1306: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:55:21.051 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:55:20 smithi153 ceph-mon[113733]: pgmap v1307: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:55:21.196 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:55:20 smithi174 ceph-mon[88004]: pgmap v1307: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:55:23.184 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:55:22 smithi153 ceph-mon[113733]: pgmap v1308: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:55:23.265 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:55:22 smithi174 ceph-mon[88004]: pgmap v1308: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:55:25.196 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:55:24 smithi174 ceph-mon[88004]: pgmap v1309: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:55:25.301 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:55:24 smithi153 ceph-mon[113733]: pgmap v1309: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:55:27.696 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:55:27 smithi174 ceph-mon[88004]: pgmap v1310: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:55:27.801 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:55:27 smithi153 ceph-mon[113733]: pgmap v1310: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:55:29.696 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:55:29 smithi174 ceph-mon[88004]: pgmap v1311: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:55:29.801 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:55:29 smithi153 ceph-mon[113733]: pgmap v1311: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:55:31.696 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:55:31 smithi174 ceph-mon[88004]: pgmap v1312: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:55:31.696 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:55:31 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:55:31.801 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:55:31 smithi153 ceph-mon[113733]: pgmap v1312: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:55:31.801 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:55:31 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:55:33.696 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:55:33 smithi174 ceph-mon[88004]: pgmap v1313: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:55:33.801 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:55:33 smithi153 ceph-mon[113733]: pgmap v1313: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:55:35.696 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:55:35 smithi174 ceph-mon[88004]: pgmap v1314: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:55:35.801 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:55:35 smithi153 ceph-mon[113733]: pgmap v1314: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:55:37.696 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:55:37 smithi174 ceph-mon[88004]: pgmap v1315: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:55:37.801 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:55:37 smithi153 ceph-mon[113733]: pgmap v1315: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:55:39.696 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:55:39 smithi174 ceph-mon[88004]: pgmap v1316: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:55:39.801 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:55:39 smithi153 ceph-mon[113733]: pgmap v1316: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:55:41.801 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:55:41 smithi153 ceph-mon[113733]: pgmap v1317: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:55:41.946 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:55:41 smithi174 ceph-mon[88004]: pgmap v1317: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:55:43.801 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:55:43 smithi153 ceph-mon[113733]: pgmap v1318: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:55:43.946 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:55:43 smithi174 ceph-mon[88004]: pgmap v1318: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:55:45.801 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:55:45 smithi153 ceph-mon[113733]: pgmap v1319: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:55:45.946 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:55:45 smithi174 ceph-mon[88004]: pgmap v1319: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:55:46.801 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:55:46 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:55:46.946 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:55:46 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:55:47.801 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:55:47 smithi153 ceph-mon[113733]: pgmap v1320: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:55:47.946 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:55:47 smithi174 ceph-mon[88004]: pgmap v1320: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:55:49.696 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:55:49 smithi174 ceph-mon[88004]: pgmap v1321: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:55:49.801 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:55:49 smithi153 ceph-mon[113733]: pgmap v1321: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:55:51.801 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:55:51 smithi153 ceph-mon[113733]: pgmap v1322: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:55:51.946 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:55:51 smithi174 ceph-mon[88004]: pgmap v1322: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:55:53.801 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:55:53 smithi153 ceph-mon[113733]: pgmap v1323: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:55:53.946 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:55:53 smithi174 ceph-mon[88004]: pgmap v1323: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:55:55.801 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:55:55 smithi153 ceph-mon[113733]: pgmap v1324: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:55:55.946 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:55:55 smithi174 ceph-mon[88004]: pgmap v1324: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:55:57.801 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:55:57 smithi153 ceph-mon[113733]: pgmap v1325: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:55:57.946 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:55:57 smithi174 ceph-mon[88004]: pgmap v1325: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:55:59.801 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:55:59 smithi153 ceph-mon[113733]: pgmap v1326: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:55:59.946 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:55:59 smithi174 ceph-mon[88004]: pgmap v1326: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:56:01.801 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:56:01 smithi153 ceph-mon[113733]: pgmap v1327: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:56:01.801 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:56:01 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:56:01.946 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:56:01 smithi174 ceph-mon[88004]: pgmap v1327: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:56:01.946 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:56:01 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:56:03.801 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:56:03 smithi153 ceph-mon[113733]: pgmap v1328: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:56:03.946 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:56:03 smithi174 ceph-mon[88004]: pgmap v1328: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:56:05.801 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:56:05 smithi153 ceph-mon[113733]: pgmap v1329: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:56:05.946 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:56:05 smithi174 ceph-mon[88004]: pgmap v1329: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:56:07.801 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:56:07 smithi153 ceph-mon[113733]: pgmap v1330: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-21T10:56:07.946 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:56:07 smithi174 ceph-mon[88004]: pgmap v1330: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-21T10:56:09.801 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:56:09 smithi153 ceph-mon[113733]: pgmap v1331: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-21T10:56:09.938 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:56:09 smithi174 ceph-mon[88004]: pgmap v1331: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-21T10:56:10.801 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:56:10 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:56:10.946 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:56:10 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:56:11.946 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:56:11 smithi174 ceph-mon[88004]: pgmap v1332: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-03-21T10:56:11.946 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:56:11 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:56:11.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:56:11 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:56:11.947 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:56:11 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:56:12.051 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:56:11 smithi153 ceph-mon[113733]: pgmap v1332: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-03-21T10:56:12.051 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:56:11 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:56:12.051 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:56:11 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:56:12.051 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:56:11 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:56:13.946 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:56:13 smithi174 ceph-mon[88004]: pgmap v1333: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 4 op/s 2024-03-21T10:56:14.051 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:56:13 smithi153 ceph-mon[113733]: pgmap v1333: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 4 op/s 2024-03-21T10:56:15.946 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:56:15 smithi174 ceph-mon[88004]: pgmap v1334: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-21T10:56:16.051 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:56:15 smithi153 ceph-mon[113733]: pgmap v1334: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-21T10:56:16.946 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:56:16 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:56:17.051 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:56:16 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:56:17.946 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:56:17 smithi174 ceph-mon[88004]: pgmap v1335: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-21T10:56:18.051 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:56:17 smithi153 ceph-mon[113733]: pgmap v1335: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-21T10:56:19.946 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:56:19 smithi174 ceph-mon[88004]: pgmap v1336: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:56:20.051 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:56:19 smithi153 ceph-mon[113733]: pgmap v1336: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:56:21.946 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:56:21 smithi174 ceph-mon[88004]: pgmap v1337: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:56:22.051 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:56:21 smithi153 ceph-mon[113733]: pgmap v1337: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:56:23.946 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:56:23 smithi174 ceph-mon[88004]: pgmap v1338: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:56:24.051 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:56:23 smithi153 ceph-mon[113733]: pgmap v1338: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:56:25.696 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:56:25 smithi174 ceph-mon[88004]: pgmap v1339: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:56:25.801 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:56:25 smithi153 ceph-mon[113733]: pgmap v1339: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:56:27.946 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:56:27 smithi174 ceph-mon[88004]: pgmap v1340: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:56:28.051 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:56:27 smithi153 ceph-mon[113733]: pgmap v1340: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:56:29.946 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:56:29 smithi174 ceph-mon[88004]: pgmap v1341: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:56:30.051 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:56:29 smithi153 ceph-mon[113733]: pgmap v1341: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:56:31.946 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:56:31 smithi174 ceph-mon[88004]: pgmap v1342: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:56:31.946 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:56:31 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:56:32.051 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:56:31 smithi153 ceph-mon[113733]: pgmap v1342: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:56:32.051 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:56:31 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:56:33.946 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:56:33 smithi174 ceph-mon[88004]: pgmap v1343: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:56:34.051 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:56:33 smithi153 ceph-mon[113733]: pgmap v1343: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:56:35.946 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:56:35 smithi174 ceph-mon[88004]: pgmap v1344: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:56:36.051 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:56:35 smithi153 ceph-mon[113733]: pgmap v1344: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:56:37.946 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:56:37 smithi174 ceph-mon[88004]: pgmap v1345: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:56:38.052 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:56:37 smithi153 ceph-mon[113733]: pgmap v1345: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:56:39.946 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:56:39 smithi174 ceph-mon[88004]: pgmap v1346: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:56:40.051 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:56:39 smithi153 ceph-mon[113733]: pgmap v1346: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:56:41.946 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:56:41 smithi174 ceph-mon[88004]: pgmap v1347: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:56:42.051 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:56:41 smithi153 ceph-mon[113733]: pgmap v1347: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:56:43.946 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:56:43 smithi174 ceph-mon[88004]: pgmap v1348: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:56:44.051 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:56:43 smithi153 ceph-mon[113733]: pgmap v1348: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:56:45.946 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:56:45 smithi174 ceph-mon[88004]: pgmap v1349: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:56:46.051 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:56:45 smithi153 ceph-mon[113733]: pgmap v1349: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:56:46.946 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:56:46 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:56:47.051 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:56:46 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:56:47.946 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:56:47 smithi174 ceph-mon[88004]: pgmap v1350: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:56:48.051 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:56:47 smithi153 ceph-mon[113733]: pgmap v1350: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:56:49.946 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:56:49 smithi174 ceph-mon[88004]: pgmap v1351: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:56:50.051 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:56:49 smithi153 ceph-mon[113733]: pgmap v1351: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:56:51.946 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:56:51 smithi174 ceph-mon[88004]: pgmap v1352: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:56:52.051 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:56:51 smithi153 ceph-mon[113733]: pgmap v1352: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:56:54.051 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:56:53 smithi153 ceph-mon[113733]: pgmap v1353: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:56:54.196 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:56:53 smithi174 ceph-mon[88004]: pgmap v1353: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:56:56.051 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:56:55 smithi153 ceph-mon[113733]: pgmap v1354: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:56:56.196 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:56:55 smithi174 ceph-mon[88004]: pgmap v1354: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:56:58.051 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:56:57 smithi153 ceph-mon[113733]: pgmap v1355: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:56:58.196 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:56:57 smithi174 ceph-mon[88004]: pgmap v1355: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:56:59.051 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:56:58 smithi153 ceph-mon[113733]: pgmap v1356: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:56:59.196 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:56:58 smithi174 ceph-mon[88004]: pgmap v1356: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:57:01.301 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:57:00 smithi153 ceph-mon[113733]: pgmap v1357: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:57:01.446 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:57:00 smithi174 ceph-mon[88004]: pgmap v1357: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:57:02.301 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:57:01 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:57:02.446 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:57:01 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:57:03.301 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:57:02 smithi153 ceph-mon[113733]: pgmap v1358: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:57:03.446 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:57:02 smithi174 ceph-mon[88004]: pgmap v1358: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:57:05.301 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:57:04 smithi153 ceph-mon[113733]: pgmap v1359: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:57:05.446 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:57:04 smithi174 ceph-mon[88004]: pgmap v1359: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:57:07.196 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:57:06 smithi174 ceph-mon[88004]: pgmap v1360: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:57:07.303 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:57:06 smithi153 ceph-mon[113733]: pgmap v1360: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:57:09.301 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:57:08 smithi153 ceph-mon[113733]: pgmap v1361: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:57:09.446 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:57:08 smithi174 ceph-mon[88004]: pgmap v1361: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:57:11.301 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:57:10 smithi153 ceph-mon[113733]: pgmap v1362: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:57:11.446 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:57:10 smithi174 ceph-mon[88004]: pgmap v1362: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:57:12.196 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:57:12 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:57:12.301 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:57:12 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:57:13.301 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:57:13 smithi153 ceph-mon[113733]: pgmap v1363: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:57:13.302 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:57:13 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:57:13.302 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:57:13 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:57:13.302 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:57:13 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:57:13.446 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:57:13 smithi174 ceph-mon[88004]: pgmap v1363: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:57:13.446 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:57:13 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T10:57:13.446 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:57:13 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T10:57:13.446 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:57:13 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' 2024-03-21T10:57:15.301 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:57:15 smithi153 ceph-mon[113733]: pgmap v1364: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:57:15.446 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:57:15 smithi174 ceph-mon[88004]: pgmap v1364: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:57:17.301 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:57:17 smithi153 ceph-mon[113733]: pgmap v1365: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:57:17.301 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:57:17 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:57:17.446 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:57:17 smithi174 ceph-mon[88004]: pgmap v1365: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:57:17.446 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:57:17 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:57:19.301 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:57:19 smithi153 ceph-mon[113733]: pgmap v1366: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:57:19.446 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:57:19 smithi174 ceph-mon[88004]: pgmap v1366: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:57:21.301 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:57:21 smithi153 ceph-mon[113733]: pgmap v1367: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:57:21.446 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:57:21 smithi174 ceph-mon[88004]: pgmap v1367: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:57:23.301 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:57:23 smithi153 ceph-mon[113733]: pgmap v1368: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:57:23.446 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:57:23 smithi174 ceph-mon[88004]: pgmap v1368: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:57:25.301 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:57:25 smithi153 ceph-mon[113733]: pgmap v1369: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:57:25.446 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:57:25 smithi174 ceph-mon[88004]: pgmap v1369: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:57:27.301 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:57:27 smithi153 ceph-mon[113733]: pgmap v1370: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:57:27.443 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:57:27 smithi174 ceph-mon[88004]: pgmap v1370: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:57:29.446 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:57:29 smithi174 ceph-mon[88004]: pgmap v1371: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:57:29.551 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:57:29 smithi153 ceph-mon[113733]: pgmap v1371: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:57:31.446 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:57:31 smithi174 ceph-mon[88004]: pgmap v1372: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:57:31.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:57:31 smithi153 ceph-mon[113733]: pgmap v1372: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:57:32.447 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:57:32 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:57:32.552 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:57:32 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:57:33.446 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:57:33 smithi174 ceph-mon[88004]: pgmap v1373: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:57:33.551 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:57:33 smithi153 ceph-mon[113733]: pgmap v1373: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:57:35.446 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:57:35 smithi174 ceph-mon[88004]: pgmap v1374: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:57:35.551 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:57:35 smithi153 ceph-mon[113733]: pgmap v1374: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:57:37.446 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:57:37 smithi174 ceph-mon[88004]: pgmap v1375: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:57:37.551 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:57:37 smithi153 ceph-mon[113733]: pgmap v1375: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:57:39.446 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:57:39 smithi174 ceph-mon[88004]: pgmap v1376: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:57:39.551 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:57:39 smithi153 ceph-mon[113733]: pgmap v1376: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:57:41.446 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:57:41 smithi174 ceph-mon[88004]: pgmap v1377: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:57:41.551 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:57:41 smithi153 ceph-mon[113733]: pgmap v1377: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:57:43.446 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:57:43 smithi174 ceph-mon[88004]: pgmap v1378: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:57:43.551 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:57:43 smithi153 ceph-mon[113733]: pgmap v1378: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:57:45.446 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:57:45 smithi174 ceph-mon[88004]: pgmap v1379: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:57:45.551 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:57:45 smithi153 ceph-mon[113733]: pgmap v1379: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:57:47.446 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:57:47 smithi174 ceph-mon[88004]: pgmap v1380: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:57:47.446 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:57:47 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:57:47.551 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:57:47 smithi153 ceph-mon[113733]: pgmap v1380: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:57:47.551 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:57:47 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:57:49.446 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:57:49 smithi174 ceph-mon[88004]: pgmap v1381: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:57:49.551 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:57:49 smithi153 ceph-mon[113733]: pgmap v1381: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:57:51.446 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:57:51 smithi174 ceph-mon[88004]: pgmap v1382: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:57:51.551 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:57:51 smithi153 ceph-mon[113733]: pgmap v1382: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:57:53.446 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:57:53 smithi174 ceph-mon[88004]: pgmap v1383: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:57:53.551 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:57:53 smithi153 ceph-mon[113733]: pgmap v1383: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:57:55.446 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:57:55 smithi174 ceph-mon[88004]: pgmap v1384: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:57:55.551 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:57:55 smithi153 ceph-mon[113733]: pgmap v1384: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:57:57.446 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:57:57 smithi174 ceph-mon[88004]: pgmap v1385: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:57:57.551 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:57:57 smithi153 ceph-mon[113733]: pgmap v1385: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:57:59.446 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:57:59 smithi174 ceph-mon[88004]: pgmap v1386: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:57:59.551 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:57:59 smithi153 ceph-mon[113733]: pgmap v1386: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:58:01.446 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:58:01 smithi174 ceph-mon[88004]: pgmap v1387: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:58:01.551 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:58:01 smithi153 ceph-mon[113733]: pgmap v1387: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:58:02.446 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:58:02 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:58:02.551 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:58:02 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T10:58:03.446 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:58:03 smithi174 ceph-mon[88004]: pgmap v1388: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:58:03.551 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:58:03 smithi153 ceph-mon[113733]: pgmap v1388: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:58:05.446 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:58:05 smithi174 ceph-mon[88004]: pgmap v1389: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:58:05.551 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:58:05 smithi153 ceph-mon[113733]: pgmap v1389: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:58:07.446 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:58:07 smithi174 ceph-mon[88004]: pgmap v1390: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:58:07.551 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:58:07 smithi153 ceph-mon[113733]: pgmap v1390: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-21T10:58:09.551 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:58:09 smithi153 ceph-mon[113733]: pgmap v1391: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:58:09.696 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:58:09 smithi174 ceph-mon[88004]: pgmap v1391: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-21T10:58:11.551 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:58:11 smithi153 ceph-mon[113733]: pgmap v1392: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:58:11.696 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:58:11 smithi174 ceph-mon[88004]: pgmap v1392: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-21T10:58:13.474 ERROR:tasks.cephfs.fuse_mount:process failed to terminate after unmount. This probably indicates a bug within ceph-fuse. 2024-03-21T10:58:13.475 ERROR:teuthology.run_tasks:Manager failed: ceph-fuse Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_2edfee68ad7243bed422a049b6f5f58619300f11/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph-c_2edfee68ad7243bed422a049b6f5f58619300f11/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-03-21T10:58:13.477 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-03-21T10:58:13.488 INFO:tasks.cephadm:Teardown begin 2024-03-21T10:58:13.488 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_2edfee68ad7243bed422a049b6f5f58619300f11/qa/tasks/cephadm.py", line 1867, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_2edfee68ad7243bed422a049b6f5f58619300f11/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph-c_2edfee68ad7243bed422a049b6f5f58619300f11/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-03-21T10:58:13.489 DEBUG:teuthology.orchestra.run.smithi153:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-21T10:58:13.551 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:58:13 smithi153 ceph-mon[113733]: pgmap v1393: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:58:13.551 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:58:13 smithi153 ceph-mon[113733]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:58:13.583 DEBUG:teuthology.orchestra.run.smithi174:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-21T10:58:13.659 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:58:13 smithi174 ceph-mon[88004]: pgmap v1393: 65 pgs: 65 active+clean; 317 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-21T10:58:13.659 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:58:13 smithi174 ceph-mon[88004]: from='mgr.34104 172.21.15.153:0/3700424112' entity='mgr.smithi153.tluxni' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T10:58:13.671 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-03-21T10:58:13.671 DEBUG:teuthology.orchestra.run.smithi153:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-21T10:58:13.717 DEBUG:teuthology.orchestra.run.smithi174:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-21T10:58:13.766 INFO:tasks.cephadm:Stopping all daemons... 2024-03-21T10:58:13.767 INFO:tasks.cephadm.mon.smithi153:Stopping mon.smithi153... 2024-03-21T10:58:13.767 DEBUG:teuthology.orchestra.run.smithi153:> sudo systemctl stop ceph-070ed3e4-e769-11ee-95cd-87774f69a715@mon.smithi153 2024-03-21T10:58:14.301 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:58:13 smithi153 systemd[1]: Stopping Ceph mon.smithi153 for 070ed3e4-e769-11ee-95cd-87774f69a715... 2024-03-21T10:58:14.748 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:58:14 smithi153 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-mon-smithi153[113729]: 2024-03-21T10:58:14.341+0000 7f216da00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi153 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T10:58:14.748 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:58:14 smithi153 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-mon-smithi153[113729]: 2024-03-21T10:58:14.341+0000 7f216da00700 -1 mon.smithi153@0(leader) e3 *** Got Signal Terminated *** 2024-03-21T10:58:14.748 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:58:14 smithi153 podman[170513]: 2024-03-21 10:58:14.393820795 +0000 UTC m=+0.213108892 container died dd8038303dc317e8543d4a848b533df3ed09720587219ba01a349d97247df207 (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-mon-smithi153, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-03-21T10:58:15.051 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:58:14 smithi153 podman[170513]: 2024-03-21 10:58:14.935604962 +0000 UTC m=+0.754893083 container cleanup dd8038303dc317e8543d4a848b533df3ed09720587219ba01a349d97247df207 (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-mon-smithi153, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T10:58:15.051 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:58:14 smithi153 bash[170513]: ceph-070ed3e4-e769-11ee-95cd-87774f69a715-mon-smithi153 2024-03-21T10:58:15.509 INFO:journalctl@ceph.mon.smithi153.smithi153.stdout:Mar 21 10:58:15 smithi153 podman[170538]: 2024-03-21 10:58:15.170520668 +0000 UTC m=+0.767288592 container remove dd8038303dc317e8543d4a848b533df3ed09720587219ba01a349d97247df207 (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-mon-smithi153, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, io.buildah.version=1.33.5) 2024-03-21T10:58:15.531 DEBUG:teuthology.orchestra.run.smithi153:> sudo pkill -f 'journalctl -f -n 0 -u ceph-070ed3e4-e769-11ee-95cd-87774f69a715@mon.smithi153.service' 2024-03-21T10:58:15.646 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T10:58:15.647 INFO:tasks.cephadm.mon.smithi153:Stopped mon.smithi153 2024-03-21T10:58:15.647 INFO:tasks.cephadm.mon.smithi174:Stopping mon.smithi174... 2024-03-21T10:58:15.647 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ceph-070ed3e4-e769-11ee-95cd-87774f69a715@mon.smithi174 2024-03-21T10:58:16.196 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:58:15 smithi174 systemd[1]: Stopping Ceph mon.smithi174 for 070ed3e4-e769-11ee-95cd-87774f69a715... 2024-03-21T10:58:16.568 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:58:16 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-mon-smithi174[88000]: 2024-03-21T10:58:16.294+0000 7fdde0600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi174 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T10:58:16.568 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:58:16 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-mon-smithi174[88000]: 2024-03-21T10:58:16.294+0000 7fdde0600700 -1 mon.smithi174@1(peon) e3 *** Got Signal Terminated *** 2024-03-21T10:58:16.863 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:58:16 smithi174 podman[124855]: 2024-03-21 10:58:16.572351425 +0000 UTC m=+0.427282742 container died 29d78a788f2cf394f70c4b9c11f4bf74d6f46c36e05cb7f0a0a8a17bc9718a44 (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-mon-smithi174, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_CLEAN=True) 2024-03-21T10:58:17.115 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:58:17 smithi174 podman[124855]: 2024-03-21 10:58:17.097933295 +0000 UTC m=+0.952864602 container cleanup 29d78a788f2cf394f70c4b9c11f4bf74d6f46c36e05cb7f0a0a8a17bc9718a44 (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-mon-smithi174, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS) 2024-03-21T10:58:17.115 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:58:17 smithi174 bash[124855]: ceph-070ed3e4-e769-11ee-95cd-87774f69a715-mon-smithi174 2024-03-21T10:58:17.446 INFO:journalctl@ceph.mon.smithi174.smithi174.stdout:Mar 21 10:58:17 smithi174 podman[124869]: 2024-03-21 10:58:17.36391792 +0000 UTC m=+0.775208526 container remove 29d78a788f2cf394f70c4b9c11f4bf74d6f46c36e05cb7f0a0a8a17bc9718a44 (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-mon-smithi174, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-21T10:58:17.854 DEBUG:teuthology.orchestra.run.smithi174:> sudo pkill -f 'journalctl -f -n 0 -u ceph-070ed3e4-e769-11ee-95cd-87774f69a715@mon.smithi174.service' 2024-03-21T10:58:17.990 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T10:58:17.991 INFO:tasks.cephadm.mon.smithi174:Stopped mon.smithi174 2024-03-21T10:58:17.991 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-03-21T10:58:17.991 DEBUG:teuthology.orchestra.run.smithi153:> sudo systemctl stop ceph-070ed3e4-e769-11ee-95cd-87774f69a715@osd.0 2024-03-21T10:58:18.367 INFO:journalctl@ceph.osd.0.smithi153.stdout:Mar 21 10:58:18 smithi153 systemd[1]: Stopping Ceph osd.0 for 070ed3e4-e769-11ee-95cd-87774f69a715... 2024-03-21T10:58:18.801 INFO:journalctl@ceph.osd.0.smithi153.stdout:Mar 21 10:58:18 smithi153 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-0[125335]: 2024-03-21T10:58:18.580+0000 7f1507c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T10:58:18.801 INFO:journalctl@ceph.osd.0.smithi153.stdout:Mar 21 10:58:18 smithi153 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-0[125335]: 2024-03-21T10:58:18.580+0000 7f1507c00700 -1 osd.0 98 *** Got signal Terminated *** 2024-03-21T10:58:18.802 INFO:journalctl@ceph.osd.0.smithi153.stdout:Mar 21 10:58:18 smithi153 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-0[125335]: 2024-03-21T10:58:18.580+0000 7f1507c00700 -1 osd.0 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-21T10:58:24.458 INFO:journalctl@ceph.osd.0.smithi153.stdout:Mar 21 10:58:24 smithi153 podman[170665]: 2024-03-21 10:58:24.09974558 +0000 UTC m=+5.695906497 container died 507900fd4791712a0c96135fc03a723571112a9198e19c14d19724a63134ee19 (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-03-21T10:58:24.795 INFO:journalctl@ceph.osd.0.smithi153.stdout:Mar 21 10:58:24 smithi153 podman[170665]: 2024-03-21 10:58:24.572905729 +0000 UTC m=+6.169066640 container cleanup 507900fd4791712a0c96135fc03a723571112a9198e19c14d19724a63134ee19 (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_CLEAN=True) 2024-03-21T10:58:24.795 INFO:journalctl@ceph.osd.0.smithi153.stdout:Mar 21 10:58:24 smithi153 bash[170665]: ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-0 2024-03-21T10:58:25.051 INFO:journalctl@ceph.osd.0.smithi153.stdout:Mar 21 10:58:24 smithi153 podman[170683]: 2024-03-21 10:58:24.799338521 +0000 UTC m=+0.685385521 container remove 507900fd4791712a0c96135fc03a723571112a9198e19c14d19724a63134ee19 (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, io.buildah.version=1.33.5) 2024-03-21T10:58:25.801 INFO:journalctl@ceph.osd.0.smithi153.stdout:Mar 21 10:58:25 smithi153 podman[170766]: 2024-03-21 10:58:25.41671854 +0000 UTC m=+0.041609837 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T10:58:25.801 INFO:journalctl@ceph.osd.0.smithi153.stdout:Mar 21 10:58:25 smithi153 podman[170766]: 2024-03-21 10:58:25.558811826 +0000 UTC m=+0.183703160 container create 862a86e3613248d7ee1e9360d188c9d577735c8e36a97a85d80ee70cc2629137 (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-0-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.build-date=20240319, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-03-21T10:58:26.297 INFO:journalctl@ceph.osd.0.smithi153.stdout:Mar 21 10:58:25 smithi153 podman[170766]: 2024-03-21 10:58:25.82984761 +0000 UTC m=+0.454738907 container init 862a86e3613248d7ee1e9360d188c9d577735c8e36a97a85d80ee70cc2629137 (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-0-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6) 2024-03-21T10:58:26.297 INFO:journalctl@ceph.osd.0.smithi153.stdout:Mar 21 10:58:25 smithi153 podman[170766]: 2024-03-21 10:58:25.84228618 +0000 UTC m=+0.467177482 container start 862a86e3613248d7ee1e9360d188c9d577735c8e36a97a85d80ee70cc2629137 (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-0-deactivate, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-03-21T10:58:26.297 INFO:journalctl@ceph.osd.0.smithi153.stdout:Mar 21 10:58:25 smithi153 podman[170766]: 2024-03-21 10:58:25.944659657 +0000 UTC m=+0.569551014 container attach 862a86e3613248d7ee1e9360d188c9d577735c8e36a97a85d80ee70cc2629137 (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-0-deactivate, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-21T10:58:26.551 INFO:journalctl@ceph.osd.0.smithi153.stdout:Mar 21 10:58:26 smithi153 podman[170766]: 2024-03-21 10:58:26.30250728 +0000 UTC m=+0.927398590 container died 862a86e3613248d7ee1e9360d188c9d577735c8e36a97a85d80ee70cc2629137 (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-0-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-03-21T10:58:27.008 DEBUG:teuthology.orchestra.run.smithi153:> sudo pkill -f 'journalctl -f -n 0 -u ceph-070ed3e4-e769-11ee-95cd-87774f69a715@osd.0.service' 2024-03-21T10:58:27.110 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T10:58:27.111 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-03-21T10:58:27.111 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-03-21T10:58:27.111 DEBUG:teuthology.orchestra.run.smithi153:> sudo systemctl stop ceph-070ed3e4-e769-11ee-95cd-87774f69a715@osd.1 2024-03-21T10:58:27.551 INFO:journalctl@ceph.osd.1.smithi153.stdout:Mar 21 10:58:27 smithi153 systemd[1]: Stopping Ceph osd.1 for 070ed3e4-e769-11ee-95cd-87774f69a715... 2024-03-21T10:58:28.052 INFO:journalctl@ceph.osd.1.smithi153.stdout:Mar 21 10:58:27 smithi153 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-1[131211]: 2024-03-21T10:58:27.674+0000 7f9eb0a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T10:58:28.052 INFO:journalctl@ceph.osd.1.smithi153.stdout:Mar 21 10:58:27 smithi153 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-1[131211]: 2024-03-21T10:58:27.674+0000 7f9eb0a00700 -1 osd.1 98 *** Got signal Terminated *** 2024-03-21T10:58:28.052 INFO:journalctl@ceph.osd.1.smithi153.stdout:Mar 21 10:58:27 smithi153 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-1[131211]: 2024-03-21T10:58:27.674+0000 7f9eb0a00700 -1 osd.1 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-21T10:58:33.344 INFO:journalctl@ceph.osd.1.smithi153.stdout:Mar 21 10:58:33 smithi153 podman[170872]: 2024-03-21 10:58:33.093339291 +0000 UTC m=+5.559189889 container died 0d646452e9c1c131b93daee5f73a85f0a3086b9c3c5f0d9b4725770fd3516727 (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-1, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6) 2024-03-21T10:58:33.653 INFO:journalctl@ceph.osd.1.smithi153.stdout:Mar 21 10:58:33 smithi153 podman[170872]: 2024-03-21 10:58:33.492951777 +0000 UTC m=+5.958802402 container cleanup 0d646452e9c1c131b93daee5f73a85f0a3086b9c3c5f0d9b4725770fd3516727 (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, ceph=True) 2024-03-21T10:58:33.653 INFO:journalctl@ceph.osd.1.smithi153.stdout:Mar 21 10:58:33 smithi153 bash[170872]: ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-1 2024-03-21T10:58:34.051 INFO:journalctl@ceph.osd.1.smithi153.stdout:Mar 21 10:58:33 smithi153 podman[170889]: 2024-03-21 10:58:33.711246016 +0000 UTC m=+0.603133939 container remove 0d646452e9c1c131b93daee5f73a85f0a3086b9c3c5f0d9b4725770fd3516727 (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-1, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240319, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-03-21T10:58:34.801 INFO:journalctl@ceph.osd.1.smithi153.stdout:Mar 21 10:58:34 smithi153 podman[170974]: 2024-03-21 10:58:34.45289943 +0000 UTC m=+0.032433772 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T10:58:34.801 INFO:journalctl@ceph.osd.1.smithi153.stdout:Mar 21 10:58:34 smithi153 podman[170974]: 2024-03-21 10:58:34.570107819 +0000 UTC m=+0.149642197 container create a0d317298ae91a452ef4262a83ac78cbe1b2b336bb246507a4c24a44d7ff94df (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-1-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, ceph=True, org.label-schema.schema-version=1.0) 2024-03-21T10:58:35.265 INFO:journalctl@ceph.osd.1.smithi153.stdout:Mar 21 10:58:34 smithi153 podman[170974]: 2024-03-21 10:58:34.831934029 +0000 UTC m=+0.411468400 container init a0d317298ae91a452ef4262a83ac78cbe1b2b336bb246507a4c24a44d7ff94df (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-1-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, GIT_CLEAN=True) 2024-03-21T10:58:35.266 INFO:journalctl@ceph.osd.1.smithi153.stdout:Mar 21 10:58:34 smithi153 podman[170974]: 2024-03-21 10:58:34.843051164 +0000 UTC m=+0.422585516 container start a0d317298ae91a452ef4262a83ac78cbe1b2b336bb246507a4c24a44d7ff94df (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-1-deactivate, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319) 2024-03-21T10:58:35.266 INFO:journalctl@ceph.osd.1.smithi153.stdout:Mar 21 10:58:34 smithi153 podman[170974]: 2024-03-21 10:58:34.939464493 +0000 UTC m=+0.518998901 container attach a0d317298ae91a452ef4262a83ac78cbe1b2b336bb246507a4c24a44d7ff94df (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-1-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True) 2024-03-21T10:58:35.554 INFO:journalctl@ceph.osd.1.smithi153.stdout:Mar 21 10:58:35 smithi153 podman[170974]: 2024-03-21 10:58:35.270811453 +0000 UTC m=+0.850345824 container died a0d317298ae91a452ef4262a83ac78cbe1b2b336bb246507a4c24a44d7ff94df (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-1-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319) 2024-03-21T10:58:36.057 DEBUG:teuthology.orchestra.run.smithi153:> sudo pkill -f 'journalctl -f -n 0 -u ceph-070ed3e4-e769-11ee-95cd-87774f69a715@osd.1.service' 2024-03-21T10:58:36.170 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T10:58:36.170 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-03-21T10:58:36.170 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-03-21T10:58:36.170 DEBUG:teuthology.orchestra.run.smithi153:> sudo systemctl stop ceph-070ed3e4-e769-11ee-95cd-87774f69a715@osd.2 2024-03-21T10:58:36.551 INFO:journalctl@ceph.osd.2.smithi153.stdout:Mar 21 10:58:36 smithi153 systemd[1]: Stopping Ceph osd.2 for 070ed3e4-e769-11ee-95cd-87774f69a715... 2024-03-21T10:58:37.051 INFO:journalctl@ceph.osd.2.smithi153.stdout:Mar 21 10:58:36 smithi153 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-2[136939]: 2024-03-21T10:58:36.735+0000 7fa578200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T10:58:37.051 INFO:journalctl@ceph.osd.2.smithi153.stdout:Mar 21 10:58:36 smithi153 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-2[136939]: 2024-03-21T10:58:36.735+0000 7fa578200700 -1 osd.2 98 *** Got signal Terminated *** 2024-03-21T10:58:37.051 INFO:journalctl@ceph.osd.2.smithi153.stdout:Mar 21 10:58:36 smithi153 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-2[136939]: 2024-03-21T10:58:36.735+0000 7fa578200700 -1 osd.2 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-21T10:58:42.429 INFO:journalctl@ceph.osd.2.smithi153.stdout:Mar 21 10:58:42 smithi153 podman[171080]: 2024-03-21 10:58:42.106738172 +0000 UTC m=+5.496497205 container died 1326f2afdfcd42fc768f7dc44b6f9626fe9db845493e08c37364fb131f7054c1 (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T10:58:42.750 INFO:journalctl@ceph.osd.2.smithi153.stdout:Mar 21 10:58:42 smithi153 podman[171080]: 2024-03-21 10:58:42.545519365 +0000 UTC m=+5.935278409 container cleanup 1326f2afdfcd42fc768f7dc44b6f9626fe9db845493e08c37364fb131f7054c1 (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-2, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True) 2024-03-21T10:58:42.750 INFO:journalctl@ceph.osd.2.smithi153.stdout:Mar 21 10:58:42 smithi153 bash[171080]: ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-2 2024-03-21T10:58:43.051 INFO:journalctl@ceph.osd.2.smithi153.stdout:Mar 21 10:58:42 smithi153 podman[171098]: 2024-03-21 10:58:42.754801277 +0000 UTC m=+0.638882786 container remove 1326f2afdfcd42fc768f7dc44b6f9626fe9db845493e08c37364fb131f7054c1 (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-03-21T10:58:43.801 INFO:journalctl@ceph.osd.2.smithi153.stdout:Mar 21 10:58:43 smithi153 podman[171185]: 2024-03-21 10:58:43.447217238 +0000 UTC m=+0.039767031 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T10:58:43.801 INFO:journalctl@ceph.osd.2.smithi153.stdout:Mar 21 10:58:43 smithi153 podman[171185]: 2024-03-21 10:58:43.555598838 +0000 UTC m=+0.148148630 container create 6d6db1b07a3a7bca2fcbeb7a3cf7e7e2708d0b0455e37cd53831bdc51eefd4b0 (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-2-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True) 2024-03-21T10:58:44.267 INFO:journalctl@ceph.osd.2.smithi153.stdout:Mar 21 10:58:43 smithi153 podman[171185]: 2024-03-21 10:58:43.817920736 +0000 UTC m=+0.410470565 container init 6d6db1b07a3a7bca2fcbeb7a3cf7e7e2708d0b0455e37cd53831bdc51eefd4b0 (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-2-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-03-21T10:58:44.267 INFO:journalctl@ceph.osd.2.smithi153.stdout:Mar 21 10:58:43 smithi153 podman[171185]: 2024-03-21 10:58:43.831823545 +0000 UTC m=+0.424373359 container start 6d6db1b07a3a7bca2fcbeb7a3cf7e7e2708d0b0455e37cd53831bdc51eefd4b0 (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-2-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_CLEAN=True) 2024-03-21T10:58:44.267 INFO:journalctl@ceph.osd.2.smithi153.stdout:Mar 21 10:58:43 smithi153 podman[171185]: 2024-03-21 10:58:43.916680294 +0000 UTC m=+0.509230106 container attach 6d6db1b07a3a7bca2fcbeb7a3cf7e7e2708d0b0455e37cd53831bdc51eefd4b0 (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-2-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-03-21T10:58:44.551 INFO:journalctl@ceph.osd.2.smithi153.stdout:Mar 21 10:58:44 smithi153 podman[171185]: 2024-03-21 10:58:44.27167714 +0000 UTC m=+0.864226968 container died 6d6db1b07a3a7bca2fcbeb7a3cf7e7e2708d0b0455e37cd53831bdc51eefd4b0 (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-2-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-03-21T10:58:44.978 DEBUG:teuthology.orchestra.run.smithi153:> sudo pkill -f 'journalctl -f -n 0 -u ceph-070ed3e4-e769-11ee-95cd-87774f69a715@osd.2.service' 2024-03-21T10:58:45.079 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T10:58:45.079 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-03-21T10:58:45.079 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-03-21T10:58:45.079 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ceph-070ed3e4-e769-11ee-95cd-87774f69a715@osd.3 2024-03-21T10:58:45.446 INFO:journalctl@ceph.osd.3.smithi174.stdout:Mar 21 10:58:45 smithi174 systemd[1]: Stopping Ceph osd.3 for 070ed3e4-e769-11ee-95cd-87774f69a715... 2024-03-21T10:58:45.946 INFO:journalctl@ceph.osd.4.smithi174.stdout:Mar 21 10:58:45 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-4[102038]: 2024-03-21T10:58:45.867+0000 7f19b3c00700 -1 osd.4 98 heartbeat_check: no reply from 172.21.15.153:6806 osd.0 since back 2024-03-21T10:58:21.375484+0000 front 2024-03-21T10:58:21.375970+0000 (oldest deadline 2024-03-21T10:58:44.875510+0000) 2024-03-21T10:58:45.947 INFO:journalctl@ceph.osd.3.smithi174.stdout:Mar 21 10:58:45 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-3[97141]: 2024-03-21T10:58:45.663+0000 7f274ec00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T10:58:45.947 INFO:journalctl@ceph.osd.3.smithi174.stdout:Mar 21 10:58:45 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-3[97141]: 2024-03-21T10:58:45.663+0000 7f274ec00700 -1 osd.3 98 *** Got signal Terminated *** 2024-03-21T10:58:45.947 INFO:journalctl@ceph.osd.3.smithi174.stdout:Mar 21 10:58:45 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-3[97141]: 2024-03-21T10:58:45.663+0000 7f274ec00700 -1 osd.3 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-21T10:58:46.614 INFO:journalctl@ceph.osd.3.smithi174.stdout:Mar 21 10:58:46 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-3[97141]: 2024-03-21T10:58:46.231+0000 7f274a400700 -1 osd.3 98 heartbeat_check: no reply from 172.21.15.153:6806 osd.0 since back 2024-03-21T10:58:22.231892+0000 front 2024-03-21T10:58:22.231714+0000 (oldest deadline 2024-03-21T10:58:45.731169+0000) 2024-03-21T10:58:46.946 INFO:journalctl@ceph.osd.4.smithi174.stdout:Mar 21 10:58:46 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-4[102038]: 2024-03-21T10:58:46.863+0000 7f19b3c00700 -1 osd.4 98 heartbeat_check: no reply from 172.21.15.153:6806 osd.0 since back 2024-03-21T10:58:21.375484+0000 front 2024-03-21T10:58:21.375970+0000 (oldest deadline 2024-03-21T10:58:44.875510+0000) 2024-03-21T10:58:46.947 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 21 10:58:46 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-5[107363]: 2024-03-21T10:58:46.614+0000 7f6deb000700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.153:6806 osd.0 since back 2024-03-21T10:58:19.910879+0000 front 2024-03-21T10:58:19.910752+0000 (oldest deadline 2024-03-21T10:58:45.810752+0000) 2024-03-21T10:58:47.637 INFO:journalctl@ceph.osd.3.smithi174.stdout:Mar 21 10:58:47 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-3[97141]: 2024-03-21T10:58:47.276+0000 7f274a400700 -1 osd.3 98 heartbeat_check: no reply from 172.21.15.153:6806 osd.0 since back 2024-03-21T10:58:22.231892+0000 front 2024-03-21T10:58:22.231714+0000 (oldest deadline 2024-03-21T10:58:45.731169+0000) 2024-03-21T10:58:47.946 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 21 10:58:47 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-5[107363]: 2024-03-21T10:58:47.640+0000 7f6deb000700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.153:6806 osd.0 since back 2024-03-21T10:58:19.910879+0000 front 2024-03-21T10:58:19.910752+0000 (oldest deadline 2024-03-21T10:58:45.810752+0000) 2024-03-21T10:58:47.946 INFO:journalctl@ceph.osd.4.smithi174.stdout:Mar 21 10:58:47 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-4[102038]: 2024-03-21T10:58:47.845+0000 7f19b3c00700 -1 osd.4 98 heartbeat_check: no reply from 172.21.15.153:6806 osd.0 since back 2024-03-21T10:58:21.375484+0000 front 2024-03-21T10:58:21.375970+0000 (oldest deadline 2024-03-21T10:58:44.875510+0000) 2024-03-21T10:58:48.596 INFO:journalctl@ceph.osd.3.smithi174.stdout:Mar 21 10:58:48 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-3[97141]: 2024-03-21T10:58:48.265+0000 7f274a400700 -1 osd.3 98 heartbeat_check: no reply from 172.21.15.153:6806 osd.0 since back 2024-03-21T10:58:22.231892+0000 front 2024-03-21T10:58:22.231714+0000 (oldest deadline 2024-03-21T10:58:45.731169+0000) 2024-03-21T10:58:48.877 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 21 10:58:48 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-5[107363]: 2024-03-21T10:58:48.599+0000 7f6deb000700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.153:6806 osd.0 since back 2024-03-21T10:58:19.910879+0000 front 2024-03-21T10:58:19.910752+0000 (oldest deadline 2024-03-21T10:58:45.810752+0000) 2024-03-21T10:58:49.196 INFO:journalctl@ceph.osd.4.smithi174.stdout:Mar 21 10:58:48 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-4[102038]: 2024-03-21T10:58:48.880+0000 7f19b3c00700 -1 osd.4 98 heartbeat_check: no reply from 172.21.15.153:6806 osd.0 since back 2024-03-21T10:58:21.375484+0000 front 2024-03-21T10:58:21.375970+0000 (oldest deadline 2024-03-21T10:58:44.875510+0000) 2024-03-21T10:58:49.581 INFO:journalctl@ceph.osd.3.smithi174.stdout:Mar 21 10:58:49 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-3[97141]: 2024-03-21T10:58:49.290+0000 7f274a400700 -1 osd.3 98 heartbeat_check: no reply from 172.21.15.153:6806 osd.0 since back 2024-03-21T10:58:22.231892+0000 front 2024-03-21T10:58:22.231714+0000 (oldest deadline 2024-03-21T10:58:45.731169+0000) 2024-03-21T10:58:49.855 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 21 10:58:49 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-5[107363]: 2024-03-21T10:58:49.583+0000 7f6deb000700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.153:6806 osd.0 since back 2024-03-21T10:58:19.910879+0000 front 2024-03-21T10:58:19.910752+0000 (oldest deadline 2024-03-21T10:58:45.810752+0000) 2024-03-21T10:58:50.196 INFO:journalctl@ceph.osd.4.smithi174.stdout:Mar 21 10:58:49 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-4[102038]: 2024-03-21T10:58:49.859+0000 7f19b3c00700 -1 osd.4 98 heartbeat_check: no reply from 172.21.15.153:6806 osd.0 since back 2024-03-21T10:58:21.375484+0000 front 2024-03-21T10:58:21.375970+0000 (oldest deadline 2024-03-21T10:58:44.875510+0000) 2024-03-21T10:58:50.538 INFO:journalctl@ceph.osd.3.smithi174.stdout:Mar 21 10:58:50 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-3[97141]: 2024-03-21T10:58:50.251+0000 7f274a400700 -1 osd.3 98 heartbeat_check: no reply from 172.21.15.153:6806 osd.0 since back 2024-03-21T10:58:22.231892+0000 front 2024-03-21T10:58:22.231714+0000 (oldest deadline 2024-03-21T10:58:45.731169+0000) 2024-03-21T10:58:50.830 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 21 10:58:50 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-5[107363]: 2024-03-21T10:58:50.541+0000 7f6deb000700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.153:6806 osd.0 since back 2024-03-21T10:58:19.910879+0000 front 2024-03-21T10:58:19.910752+0000 (oldest deadline 2024-03-21T10:58:45.810752+0000) 2024-03-21T10:58:51.115 INFO:journalctl@ceph.osd.4.smithi174.stdout:Mar 21 10:58:50 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-4[102038]: 2024-03-21T10:58:50.832+0000 7f19b3c00700 -1 osd.4 98 heartbeat_check: no reply from 172.21.15.153:6806 osd.0 since back 2024-03-21T10:58:21.375484+0000 front 2024-03-21T10:58:21.375970+0000 (oldest deadline 2024-03-21T10:58:44.875510+0000) 2024-03-21T10:58:51.410 INFO:journalctl@ceph.osd.3.smithi174.stdout:Mar 21 10:58:51 smithi174 podman[124976]: 2024-03-21 10:58:51.11932828 +0000 UTC m=+5.618294913 container died 70aafac71ad5172e74975729b2b146172e4465077c25e200138804e94bb98d64 (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-3, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-03-21T10:58:51.696 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 21 10:58:51 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-5[107363]: 2024-03-21T10:58:51.496+0000 7f6deb000700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.153:6806 osd.0 since back 2024-03-21T10:58:19.910879+0000 front 2024-03-21T10:58:19.910752+0000 (oldest deadline 2024-03-21T10:58:45.810752+0000) 2024-03-21T10:58:51.696 INFO:journalctl@ceph.osd.3.smithi174.stdout:Mar 21 10:58:51 smithi174 podman[124976]: 2024-03-21 10:58:51.523280166 +0000 UTC m=+6.022246835 container cleanup 70aafac71ad5172e74975729b2b146172e4465077c25e200138804e94bb98d64 (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-3, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-03-21T10:58:51.697 INFO:journalctl@ceph.osd.3.smithi174.stdout:Mar 21 10:58:51 smithi174 bash[124976]: ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-3 2024-03-21T10:58:52.196 INFO:journalctl@ceph.osd.3.smithi174.stdout:Mar 21 10:58:51 smithi174 podman[124996]: 2024-03-21 10:58:51.857902144 +0000 UTC m=+0.721593978 container remove 70aafac71ad5172e74975729b2b146172e4465077c25e200138804e94bb98d64 (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-3, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True) 2024-03-21T10:58:52.197 INFO:journalctl@ceph.osd.4.smithi174.stdout:Mar 21 10:58:51 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-4[102038]: 2024-03-21T10:58:51.789+0000 7f19b3c00700 -1 osd.4 98 heartbeat_check: no reply from 172.21.15.153:6806 osd.0 since back 2024-03-21T10:58:21.375484+0000 front 2024-03-21T10:58:21.375970+0000 (oldest deadline 2024-03-21T10:58:44.875510+0000) 2024-03-21T10:58:52.818 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 21 10:58:52 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-5[107363]: 2024-03-21T10:58:52.525+0000 7f6deb000700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.153:6806 osd.0 since back 2024-03-21T10:58:19.910879+0000 front 2024-03-21T10:58:19.910752+0000 (oldest deadline 2024-03-21T10:58:45.810752+0000) 2024-03-21T10:58:52.818 INFO:journalctl@ceph.osd.3.smithi174.stdout:Mar 21 10:58:52 smithi174 podman[125088]: 2024-03-21 10:58:52.526027307 +0000 UTC m=+0.033794037 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T10:58:52.818 INFO:journalctl@ceph.osd.3.smithi174.stdout:Mar 21 10:58:52 smithi174 podman[125088]: 2024-03-21 10:58:52.641936034 +0000 UTC m=+0.149702801 container create 55d917fca627e6163fa9ff2c739a28a16b4d9108525ba02b22caddf824624d3f (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-3-deactivate, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_CLEAN=True, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2) 2024-03-21T10:58:53.197 INFO:journalctl@ceph.osd.4.smithi174.stdout:Mar 21 10:58:52 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-4[102038]: 2024-03-21T10:58:52.832+0000 7f19b3c00700 -1 osd.4 98 heartbeat_check: no reply from 172.21.15.153:6806 osd.0 since back 2024-03-21T10:58:21.375484+0000 front 2024-03-21T10:58:21.375970+0000 (oldest deadline 2024-03-21T10:58:44.875510+0000) 2024-03-21T10:58:53.197 INFO:journalctl@ceph.osd.3.smithi174.stdout:Mar 21 10:58:52 smithi174 podman[125088]: 2024-03-21 10:58:52.928704987 +0000 UTC m=+0.436471761 container init 55d917fca627e6163fa9ff2c739a28a16b4d9108525ba02b22caddf824624d3f (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-3-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-03-21T10:58:53.197 INFO:journalctl@ceph.osd.3.smithi174.stdout:Mar 21 10:58:52 smithi174 podman[125088]: 2024-03-21 10:58:52.940782879 +0000 UTC m=+0.448549652 container start 55d917fca627e6163fa9ff2c739a28a16b4d9108525ba02b22caddf824624d3f (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-3-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=) 2024-03-21T10:58:53.197 INFO:journalctl@ceph.osd.3.smithi174.stdout:Mar 21 10:58:53 smithi174 podman[125088]: 2024-03-21 10:58:53.019186305 +0000 UTC m=+0.526953126 container attach 55d917fca627e6163fa9ff2c739a28a16b4d9108525ba02b22caddf824624d3f (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-3-deactivate, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-03-21T10:58:53.696 INFO:journalctl@ceph.osd.3.smithi174.stdout:Mar 21 10:58:53 smithi174 podman[125088]: 2024-03-21 10:58:53.374031839 +0000 UTC m=+0.881798598 container died 55d917fca627e6163fa9ff2c739a28a16b4d9108525ba02b22caddf824624d3f (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-3-deactivate, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-03-21T10:58:53.697 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 21 10:58:53 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-5[107363]: 2024-03-21T10:58:53.569+0000 7f6deb000700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.153:6806 osd.0 since back 2024-03-21T10:58:19.910879+0000 front 2024-03-21T10:58:19.910752+0000 (oldest deadline 2024-03-21T10:58:45.810752+0000) 2024-03-21T10:58:54.035 INFO:journalctl@ceph.osd.4.smithi174.stdout:Mar 21 10:58:53 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-4[102038]: 2024-03-21T10:58:53.808+0000 7f19b3c00700 -1 osd.4 98 heartbeat_check: no reply from 172.21.15.153:6806 osd.0 since back 2024-03-21T10:58:21.375484+0000 front 2024-03-21T10:58:21.375970+0000 (oldest deadline 2024-03-21T10:58:44.875510+0000) 2024-03-21T10:58:54.083 DEBUG:teuthology.orchestra.run.smithi174:> sudo pkill -f 'journalctl -f -n 0 -u ceph-070ed3e4-e769-11ee-95cd-87774f69a715@osd.3.service' 2024-03-21T10:58:54.205 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T10:58:54.205 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-03-21T10:58:54.206 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-03-21T10:58:54.206 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ceph-070ed3e4-e769-11ee-95cd-87774f69a715@osd.4 2024-03-21T10:58:54.600 INFO:journalctl@ceph.osd.4.smithi174.stdout:Mar 21 10:58:54 smithi174 systemd[1]: Stopping Ceph osd.4 for 070ed3e4-e769-11ee-95cd-87774f69a715... 2024-03-21T10:58:54.947 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 21 10:58:54 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-5[107363]: 2024-03-21T10:58:54.603+0000 7f6deb000700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.153:6806 osd.0 since back 2024-03-21T10:58:19.910879+0000 front 2024-03-21T10:58:19.910752+0000 (oldest deadline 2024-03-21T10:58:45.810752+0000) 2024-03-21T10:58:54.947 INFO:journalctl@ceph.osd.4.smithi174.stdout:Mar 21 10:58:54 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-4[102038]: 2024-03-21T10:58:54.813+0000 7f19b3c00700 -1 osd.4 98 heartbeat_check: no reply from 172.21.15.153:6806 osd.0 since back 2024-03-21T10:58:21.375484+0000 front 2024-03-21T10:58:21.375970+0000 (oldest deadline 2024-03-21T10:58:44.875510+0000) 2024-03-21T10:58:54.947 INFO:journalctl@ceph.osd.4.smithi174.stdout:Mar 21 10:58:54 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-4[102038]: 2024-03-21T10:58:54.822+0000 7f19bd400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T10:58:54.947 INFO:journalctl@ceph.osd.4.smithi174.stdout:Mar 21 10:58:54 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-4[102038]: 2024-03-21T10:58:54.822+0000 7f19bd400700 -1 osd.4 98 *** Got signal Terminated *** 2024-03-21T10:58:54.947 INFO:journalctl@ceph.osd.4.smithi174.stdout:Mar 21 10:58:54 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-4[102038]: 2024-03-21T10:58:54.822+0000 7f19bd400700 -1 osd.4 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-21T10:58:55.843 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 21 10:58:55 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-5[107363]: 2024-03-21T10:58:55.586+0000 7f6deb000700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.153:6806 osd.0 since back 2024-03-21T10:58:19.910879+0000 front 2024-03-21T10:58:19.910752+0000 (oldest deadline 2024-03-21T10:58:45.810752+0000) 2024-03-21T10:58:56.196 INFO:journalctl@ceph.osd.4.smithi174.stdout:Mar 21 10:58:55 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-4[102038]: 2024-03-21T10:58:55.846+0000 7f19b3c00700 -1 osd.4 98 heartbeat_check: no reply from 172.21.15.153:6806 osd.0 since back 2024-03-21T10:58:21.375484+0000 front 2024-03-21T10:58:21.375970+0000 (oldest deadline 2024-03-21T10:58:44.875510+0000) 2024-03-21T10:58:56.887 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 21 10:58:56 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-5[107363]: 2024-03-21T10:58:56.558+0000 7f6deb000700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.153:6806 osd.0 since back 2024-03-21T10:58:19.910879+0000 front 2024-03-21T10:58:19.910752+0000 (oldest deadline 2024-03-21T10:58:45.810752+0000) 2024-03-21T10:58:56.887 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 21 10:58:56 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-5[107363]: 2024-03-21T10:58:56.558+0000 7f6deb000700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.153:6814 osd.1 since back 2024-03-21T10:58:32.213299+0000 front 2024-03-21T10:58:32.212968+0000 (oldest deadline 2024-03-21T10:58:55.712803+0000) 2024-03-21T10:58:57.196 INFO:journalctl@ceph.osd.4.smithi174.stdout:Mar 21 10:58:56 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-4[102038]: 2024-03-21T10:58:56.890+0000 7f19b3c00700 -1 osd.4 98 heartbeat_check: no reply from 172.21.15.153:6806 osd.0 since back 2024-03-21T10:58:21.375484+0000 front 2024-03-21T10:58:21.375970+0000 (oldest deadline 2024-03-21T10:58:44.875510+0000) 2024-03-21T10:58:57.936 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 21 10:58:57 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-5[107363]: 2024-03-21T10:58:57.577+0000 7f6deb000700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.153:6806 osd.0 since back 2024-03-21T10:58:19.910879+0000 front 2024-03-21T10:58:19.910752+0000 (oldest deadline 2024-03-21T10:58:45.810752+0000) 2024-03-21T10:58:57.937 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 21 10:58:57 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-5[107363]: 2024-03-21T10:58:57.577+0000 7f6deb000700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.153:6814 osd.1 since back 2024-03-21T10:58:32.213299+0000 front 2024-03-21T10:58:32.212968+0000 (oldest deadline 2024-03-21T10:58:55.712803+0000) 2024-03-21T10:58:58.196 INFO:journalctl@ceph.osd.4.smithi174.stdout:Mar 21 10:58:57 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-4[102038]: 2024-03-21T10:58:57.940+0000 7f19b3c00700 -1 osd.4 98 heartbeat_check: no reply from 172.21.15.153:6806 osd.0 since back 2024-03-21T10:58:21.375484+0000 front 2024-03-21T10:58:21.375970+0000 (oldest deadline 2024-03-21T10:58:44.875510+0000) 2024-03-21T10:58:58.196 INFO:journalctl@ceph.osd.4.smithi174.stdout:Mar 21 10:58:57 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-4[102038]: 2024-03-21T10:58:57.940+0000 7f19b3c00700 -1 osd.4 98 heartbeat_check: no reply from 172.21.15.153:6814 osd.1 since back 2024-03-21T10:58:31.277970+0000 front 2024-03-21T10:58:31.278011+0000 (oldest deadline 2024-03-21T10:58:57.678621+0000) 2024-03-21T10:58:58.946 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 21 10:58:58 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-5[107363]: 2024-03-21T10:58:58.600+0000 7f6deb000700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.153:6806 osd.0 since back 2024-03-21T10:58:19.910879+0000 front 2024-03-21T10:58:19.910752+0000 (oldest deadline 2024-03-21T10:58:45.810752+0000) 2024-03-21T10:58:58.946 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 21 10:58:58 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-5[107363]: 2024-03-21T10:58:58.600+0000 7f6deb000700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.153:6814 osd.1 since back 2024-03-21T10:58:32.213299+0000 front 2024-03-21T10:58:32.212968+0000 (oldest deadline 2024-03-21T10:58:55.712803+0000) 2024-03-21T10:58:59.446 INFO:journalctl@ceph.osd.4.smithi174.stdout:Mar 21 10:58:58 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-4[102038]: 2024-03-21T10:58:58.955+0000 7f19b3c00700 -1 osd.4 98 heartbeat_check: no reply from 172.21.15.153:6806 osd.0 since back 2024-03-21T10:58:21.375484+0000 front 2024-03-21T10:58:21.375970+0000 (oldest deadline 2024-03-21T10:58:44.875510+0000) 2024-03-21T10:58:59.447 INFO:journalctl@ceph.osd.4.smithi174.stdout:Mar 21 10:58:58 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-4[102038]: 2024-03-21T10:58:58.955+0000 7f19b3c00700 -1 osd.4 98 heartbeat_check: no reply from 172.21.15.153:6814 osd.1 since back 2024-03-21T10:58:31.277970+0000 front 2024-03-21T10:58:31.278011+0000 (oldest deadline 2024-03-21T10:58:57.678621+0000) 2024-03-21T10:58:59.946 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 21 10:58:59 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-5[107363]: 2024-03-21T10:58:59.560+0000 7f6deb000700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.153:6806 osd.0 since back 2024-03-21T10:58:19.910879+0000 front 2024-03-21T10:58:19.910752+0000 (oldest deadline 2024-03-21T10:58:45.810752+0000) 2024-03-21T10:58:59.946 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 21 10:58:59 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-5[107363]: 2024-03-21T10:58:59.560+0000 7f6deb000700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.153:6814 osd.1 since back 2024-03-21T10:58:32.213299+0000 front 2024-03-21T10:58:32.212968+0000 (oldest deadline 2024-03-21T10:58:55.712803+0000) 2024-03-21T10:59:00.684 INFO:journalctl@ceph.osd.4.smithi174.stdout:Mar 21 10:59:00 smithi174 podman[125192]: 2024-03-21 10:59:00.384565457 +0000 UTC m=+5.725360346 container died 0d746ce6e79d8411fd4fa57061dc9fed9bbd3ac66fd24a7d190a00b47059c9e9 (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-03-21T10:59:00.685 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 21 10:59:00 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-5[107363]: 2024-03-21T10:59:00.526+0000 7f6deb000700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.153:6806 osd.0 since back 2024-03-21T10:58:19.910879+0000 front 2024-03-21T10:58:19.910752+0000 (oldest deadline 2024-03-21T10:58:45.810752+0000) 2024-03-21T10:59:00.685 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 21 10:59:00 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-5[107363]: 2024-03-21T10:59:00.526+0000 7f6deb000700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.153:6814 osd.1 since back 2024-03-21T10:58:32.213299+0000 front 2024-03-21T10:58:32.212968+0000 (oldest deadline 2024-03-21T10:58:55.712803+0000) 2024-03-21T10:59:00.946 INFO:journalctl@ceph.osd.4.smithi174.stdout:Mar 21 10:59:00 smithi174 podman[125192]: 2024-03-21 10:59:00.792074426 +0000 UTC m=+6.132869343 container cleanup 0d746ce6e79d8411fd4fa57061dc9fed9bbd3ac66fd24a7d190a00b47059c9e9 (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-4, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=) 2024-03-21T10:59:00.946 INFO:journalctl@ceph.osd.4.smithi174.stdout:Mar 21 10:59:00 smithi174 bash[125192]: ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-4 2024-03-21T10:59:01.446 INFO:journalctl@ceph.osd.4.smithi174.stdout:Mar 21 10:59:01 smithi174 podman[125211]: 2024-03-21 10:59:01.084849479 +0000 UTC m=+0.686462944 container remove 0d746ce6e79d8411fd4fa57061dc9fed9bbd3ac66fd24a7d190a00b47059c9e9 (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-4, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True) 2024-03-21T10:59:01.863 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 21 10:59:01 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-5[107363]: 2024-03-21T10:59:01.535+0000 7f6deb000700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.153:6806 osd.0 since back 2024-03-21T10:58:19.910879+0000 front 2024-03-21T10:58:19.910752+0000 (oldest deadline 2024-03-21T10:58:45.810752+0000) 2024-03-21T10:59:01.863 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 21 10:59:01 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-5[107363]: 2024-03-21T10:59:01.535+0000 7f6deb000700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.153:6814 osd.1 since back 2024-03-21T10:58:32.213299+0000 front 2024-03-21T10:58:32.212968+0000 (oldest deadline 2024-03-21T10:58:55.712803+0000) 2024-03-21T10:59:02.196 INFO:journalctl@ceph.osd.4.smithi174.stdout:Mar 21 10:59:01 smithi174 podman[125301]: 2024-03-21 10:59:01.77409777 +0000 UTC m=+0.033288852 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T10:59:02.196 INFO:journalctl@ceph.osd.4.smithi174.stdout:Mar 21 10:59:01 smithi174 podman[125301]: 2024-03-21 10:59:01.885844696 +0000 UTC m=+0.145035791 container create 7092b41feac1bbacba87f128da562a5b9c1ec750ccb13a67fabfd70437ac5cbc (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-4-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-03-21T10:59:02.504 INFO:journalctl@ceph.osd.4.smithi174.stdout:Mar 21 10:59:02 smithi174 podman[125301]: 2024-03-21 10:59:02.206238727 +0000 UTC m=+0.465429895 container init 7092b41feac1bbacba87f128da562a5b9c1ec750ccb13a67fabfd70437ac5cbc (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-4-deactivate, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-03-21T10:59:02.504 INFO:journalctl@ceph.osd.4.smithi174.stdout:Mar 21 10:59:02 smithi174 podman[125301]: 2024-03-21 10:59:02.217154536 +0000 UTC m=+0.476345638 container start 7092b41feac1bbacba87f128da562a5b9c1ec750ccb13a67fabfd70437ac5cbc (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-4-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0) 2024-03-21T10:59:02.504 INFO:journalctl@ceph.osd.4.smithi174.stdout:Mar 21 10:59:02 smithi174 podman[125301]: 2024-03-21 10:59:02.296668963 +0000 UTC m=+0.555860088 container attach 7092b41feac1bbacba87f128da562a5b9c1ec750ccb13a67fabfd70437ac5cbc (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-4-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-03-21T10:59:02.946 INFO:journalctl@ceph.osd.4.smithi174.stdout:Mar 21 10:59:02 smithi174 podman[125301]: 2024-03-21 10:59:02.643145115 +0000 UTC m=+0.902336240 container died 7092b41feac1bbacba87f128da562a5b9c1ec750ccb13a67fabfd70437ac5cbc (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-4-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6) 2024-03-21T10:59:02.947 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 21 10:59:02 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-5[107363]: 2024-03-21T10:59:02.506+0000 7f6deb000700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.153:6806 osd.0 since back 2024-03-21T10:58:19.910879+0000 front 2024-03-21T10:58:19.910752+0000 (oldest deadline 2024-03-21T10:58:45.810752+0000) 2024-03-21T10:59:02.947 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 21 10:59:02 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-5[107363]: 2024-03-21T10:59:02.506+0000 7f6deb000700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.153:6814 osd.1 since back 2024-03-21T10:58:32.213299+0000 front 2024-03-21T10:58:32.212968+0000 (oldest deadline 2024-03-21T10:58:55.712803+0000) 2024-03-21T10:59:03.395 DEBUG:teuthology.orchestra.run.smithi174:> sudo pkill -f 'journalctl -f -n 0 -u ceph-070ed3e4-e769-11ee-95cd-87774f69a715@osd.4.service' 2024-03-21T10:59:03.515 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T10:59:03.516 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-03-21T10:59:03.516 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-03-21T10:59:03.516 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ceph-070ed3e4-e769-11ee-95cd-87774f69a715@osd.5 2024-03-21T10:59:03.598 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 21 10:59:03 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-5[107363]: 2024-03-21T10:59:03.462+0000 7f6deb000700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.153:6806 osd.0 since back 2024-03-21T10:58:19.910879+0000 front 2024-03-21T10:58:19.910752+0000 (oldest deadline 2024-03-21T10:58:45.810752+0000) 2024-03-21T10:59:03.599 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 21 10:59:03 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-5[107363]: 2024-03-21T10:59:03.462+0000 7f6deb000700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.153:6814 osd.1 since back 2024-03-21T10:58:32.213299+0000 front 2024-03-21T10:58:32.212968+0000 (oldest deadline 2024-03-21T10:58:55.712803+0000) 2024-03-21T10:59:03.599 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 21 10:59:03 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-5[107363]: 2024-03-21T10:59:03.462+0000 7f6deb000700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.153:6822 osd.2 since back 2024-03-21T10:58:40.915026+0000 front 2024-03-21T10:58:40.914913+0000 (oldest deadline 2024-03-21T10:59:03.214638+0000) 2024-03-21T10:59:03.946 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 21 10:59:03 smithi174 systemd[1]: Stopping Ceph osd.5 for 070ed3e4-e769-11ee-95cd-87774f69a715... 2024-03-21T10:59:04.446 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 21 10:59:04 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-5[107363]: 2024-03-21T10:59:04.117+0000 7f6df4800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T10:59:04.447 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 21 10:59:04 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-5[107363]: 2024-03-21T10:59:04.117+0000 7f6df4800700 -1 osd.5 98 *** Got signal Terminated *** 2024-03-21T10:59:04.447 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 21 10:59:04 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-5[107363]: 2024-03-21T10:59:04.117+0000 7f6df4800700 -1 osd.5 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-21T10:59:04.946 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 21 10:59:04 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-5[107363]: 2024-03-21T10:59:04.506+0000 7f6deb000700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.153:6806 osd.0 since back 2024-03-21T10:58:19.910879+0000 front 2024-03-21T10:58:19.910752+0000 (oldest deadline 2024-03-21T10:58:45.810752+0000) 2024-03-21T10:59:04.947 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 21 10:59:04 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-5[107363]: 2024-03-21T10:59:04.507+0000 7f6deb000700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.153:6814 osd.1 since back 2024-03-21T10:58:32.213299+0000 front 2024-03-21T10:58:32.212968+0000 (oldest deadline 2024-03-21T10:58:55.712803+0000) 2024-03-21T10:59:04.947 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 21 10:59:04 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-5[107363]: 2024-03-21T10:59:04.507+0000 7f6deb000700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.153:6822 osd.2 since back 2024-03-21T10:58:40.915026+0000 front 2024-03-21T10:58:40.914913+0000 (oldest deadline 2024-03-21T10:59:03.214638+0000) 2024-03-21T10:59:05.697 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 21 10:59:05 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-5[107363]: 2024-03-21T10:59:05.535+0000 7f6deb000700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.153:6806 osd.0 since back 2024-03-21T10:58:19.910879+0000 front 2024-03-21T10:58:19.910752+0000 (oldest deadline 2024-03-21T10:58:45.810752+0000) 2024-03-21T10:59:05.697 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 21 10:59:05 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-5[107363]: 2024-03-21T10:59:05.535+0000 7f6deb000700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.153:6814 osd.1 since back 2024-03-21T10:58:32.213299+0000 front 2024-03-21T10:58:32.212968+0000 (oldest deadline 2024-03-21T10:58:55.712803+0000) 2024-03-21T10:59:05.697 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 21 10:59:05 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-5[107363]: 2024-03-21T10:59:05.535+0000 7f6deb000700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.153:6822 osd.2 since back 2024-03-21T10:58:40.915026+0000 front 2024-03-21T10:58:40.914913+0000 (oldest deadline 2024-03-21T10:59:03.214638+0000) 2024-03-21T10:59:06.946 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 21 10:59:06 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-5[107363]: 2024-03-21T10:59:06.493+0000 7f6deb000700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.153:6806 osd.0 since back 2024-03-21T10:58:19.910879+0000 front 2024-03-21T10:58:19.910752+0000 (oldest deadline 2024-03-21T10:58:45.810752+0000) 2024-03-21T10:59:06.947 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 21 10:59:06 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-5[107363]: 2024-03-21T10:59:06.493+0000 7f6deb000700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.153:6814 osd.1 since back 2024-03-21T10:58:32.213299+0000 front 2024-03-21T10:58:32.212968+0000 (oldest deadline 2024-03-21T10:58:55.712803+0000) 2024-03-21T10:59:06.947 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 21 10:59:06 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-5[107363]: 2024-03-21T10:59:06.493+0000 7f6deb000700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.153:6822 osd.2 since back 2024-03-21T10:58:40.915026+0000 front 2024-03-21T10:58:40.914913+0000 (oldest deadline 2024-03-21T10:59:03.214638+0000) 2024-03-21T10:59:07.946 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 21 10:59:07 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-5[107363]: 2024-03-21T10:59:07.486+0000 7f6deb000700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.153:6806 osd.0 since back 2024-03-21T10:58:19.910879+0000 front 2024-03-21T10:58:19.910752+0000 (oldest deadline 2024-03-21T10:58:45.810752+0000) 2024-03-21T10:59:07.947 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 21 10:59:07 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-5[107363]: 2024-03-21T10:59:07.486+0000 7f6deb000700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.153:6814 osd.1 since back 2024-03-21T10:58:32.213299+0000 front 2024-03-21T10:58:32.212968+0000 (oldest deadline 2024-03-21T10:58:55.712803+0000) 2024-03-21T10:59:07.947 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 21 10:59:07 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-5[107363]: 2024-03-21T10:59:07.486+0000 7f6deb000700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.153:6822 osd.2 since back 2024-03-21T10:58:40.915026+0000 front 2024-03-21T10:58:40.914913+0000 (oldest deadline 2024-03-21T10:59:03.214638+0000) 2024-03-21T10:59:08.947 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 21 10:59:08 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-5[107363]: 2024-03-21T10:59:08.452+0000 7f6deb000700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.153:6806 osd.0 since back 2024-03-21T10:58:19.910879+0000 front 2024-03-21T10:58:19.910752+0000 (oldest deadline 2024-03-21T10:58:45.810752+0000) 2024-03-21T10:59:08.947 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 21 10:59:08 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-5[107363]: 2024-03-21T10:59:08.452+0000 7f6deb000700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.153:6814 osd.1 since back 2024-03-21T10:58:32.213299+0000 front 2024-03-21T10:58:32.212968+0000 (oldest deadline 2024-03-21T10:58:55.712803+0000) 2024-03-21T10:59:08.947 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 21 10:59:08 smithi174 ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-5[107363]: 2024-03-21T10:59:08.452+0000 7f6deb000700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.153:6822 osd.2 since back 2024-03-21T10:58:40.915026+0000 front 2024-03-21T10:58:40.914913+0000 (oldest deadline 2024-03-21T10:59:03.214638+0000) 2024-03-21T10:59:09.798 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 21 10:59:09 smithi174 podman[125407]: 2024-03-21 10:59:09.542134948 +0000 UTC m=+5.589212007 container died 53e5f35b8cac50c4e87ebf5975a1afe9781dec032450aaa403128bc38f3c0a4d (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-03-21T10:59:10.093 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 21 10:59:09 smithi174 podman[125407]: 2024-03-21 10:59:09.912498372 +0000 UTC m=+5.959575471 container cleanup 53e5f35b8cac50c4e87ebf5975a1afe9781dec032450aaa403128bc38f3c0a4d (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2) 2024-03-21T10:59:10.093 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 21 10:59:09 smithi174 bash[125407]: ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-5 2024-03-21T10:59:10.446 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 21 10:59:10 smithi174 podman[125429]: 2024-03-21 10:59:10.096800156 +0000 UTC m=+0.541935510 container remove 53e5f35b8cac50c4e87ebf5975a1afe9781dec032450aaa403128bc38f3c0a4d (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-5, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2) 2024-03-21T10:59:11.196 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 21 10:59:10 smithi174 podman[125516]: 2024-03-21 10:59:10.847165889 +0000 UTC m=+0.032870404 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T10:59:11.197 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 21 10:59:10 smithi174 podman[125516]: 2024-03-21 10:59:10.947611693 +0000 UTC m=+0.133316261 container create 3639e5434e2bbc28a2100f7457373c40cee5dcde4eff7435853bf6e9021c94fd (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-5-deactivate, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-03-21T10:59:11.694 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 21 10:59:11 smithi174 podman[125516]: 2024-03-21 10:59:11.262002322 +0000 UTC m=+0.447706852 container init 3639e5434e2bbc28a2100f7457373c40cee5dcde4eff7435853bf6e9021c94fd (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-5-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, io.buildah.version=1.33.5) 2024-03-21T10:59:11.695 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 21 10:59:11 smithi174 podman[125516]: 2024-03-21 10:59:11.271670582 +0000 UTC m=+0.457375110 container start 3639e5434e2bbc28a2100f7457373c40cee5dcde4eff7435853bf6e9021c94fd (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-5-deactivate, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319) 2024-03-21T10:59:11.695 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 21 10:59:11 smithi174 podman[125516]: 2024-03-21 10:59:11.374962186 +0000 UTC m=+0.560666740 container attach 3639e5434e2bbc28a2100f7457373c40cee5dcde4eff7435853bf6e9021c94fd (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-5-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0) 2024-03-21T10:59:11.946 INFO:journalctl@ceph.osd.5.smithi174.stdout:Mar 21 10:59:11 smithi174 podman[125516]: 2024-03-21 10:59:11.699676834 +0000 UTC m=+0.885381355 container died 3639e5434e2bbc28a2100f7457373c40cee5dcde4eff7435853bf6e9021c94fd (image=quay.ceph.io/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-070ed3e4-e769-11ee-95cd-87774f69a715-osd-5-deactivate, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-03-21T10:59:12.446 DEBUG:teuthology.orchestra.run.smithi174:> sudo pkill -f 'journalctl -f -n 0 -u ceph-070ed3e4-e769-11ee-95cd-87774f69a715@osd.5.service' 2024-03-21T10:59:12.546 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T10:59:12.546 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-03-21T10:59:12.546 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 070ed3e4-e769-11ee-95cd-87774f69a715 --force --keep-logs 2024-03-21T10:59:12.830 INFO:teuthology.orchestra.run.smithi153.stdout:Deleting cluster with fsid: 070ed3e4-e769-11ee-95cd-87774f69a715 2024-03-21T10:59:19.341 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi153.stderr:ceph-fuse[77502]: fuse finished with error 0 and tester_r 0 2024-03-21T10:59:42.801 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 070ed3e4-e769-11ee-95cd-87774f69a715 --force --keep-logs 2024-03-21T10:59:43.080 INFO:teuthology.orchestra.run.smithi174.stdout:Deleting cluster with fsid: 070ed3e4-e769-11ee-95cd-87774f69a715 2024-03-21T11:00:01.757 DEBUG:teuthology.orchestra.run.smithi153:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-21T11:00:01.860 DEBUG:teuthology.orchestra.run.smithi174:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-21T11:00:01.962 INFO:tasks.cephadm:Archiving crash dumps... 2024-03-21T11:00:01.964 DEBUG:teuthology.misc:Transferring archived files from smithi153:/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/crash to /home/teuthworker/archive/vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi/7612201/remote/smithi153/crash 2024-03-21T11:00:01.965 DEBUG:teuthology.orchestra.run.smithi153:> sudo tar c -f - -C /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/crash -- . 2024-03-21T11:00:02.063 INFO:teuthology.orchestra.run.smithi153.stderr:tar: /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/crash: Cannot open: No such file or directory 2024-03-21T11:00:02.063 INFO:teuthology.orchestra.run.smithi153.stderr:tar: Error is not recoverable: exiting now 2024-03-21T11:00:02.068 DEBUG:teuthology.misc:Transferring archived files from smithi174:/var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/crash to /home/teuthworker/archive/vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi/7612201/remote/smithi174/crash 2024-03-21T11:00:02.069 DEBUG:teuthology.orchestra.run.smithi174:> sudo tar c -f - -C /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/crash -- . 2024-03-21T11:00:02.161 INFO:teuthology.orchestra.run.smithi174.stderr:tar: /var/lib/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/crash: Cannot open: No such file or directory 2024-03-21T11:00:02.162 INFO:teuthology.orchestra.run.smithi174.stderr:tar: Error is not recoverable: exiting now 2024-03-21T11:00:02.165 INFO:tasks.cephadm:Checking cluster log for badness... 2024-03-21T11:00:02.165 DEBUG:teuthology.orchestra.run.smithi153:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v OSD_DOWN | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | head -n 1 2024-03-21T11:00:02.272 INFO:teuthology.orchestra.run.smithi153.stdout:2024-03-21T10:15:33.273842+0000 mon.smithi153 (mon.0) 274 : cluster [WRN] Health check failed: Degraded data redundancy: 66/348 objects degraded (18.966%), 18 pgs degraded (PG_DEGRADED) 2024-03-21T11:00:02.273 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-03-21T11:00:02.273 INFO:tasks.cephadm:Compressing logs... 2024-03-21T11:00:02.273 DEBUG:teuthology.orchestra.run.smithi153:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-21T11:00:02.277 DEBUG:teuthology.orchestra.run.smithi174:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-21T11:00:02.342 INFO:teuthology.orchestra.run.smithi153.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-21T11:00:02.342 INFO:teuthology.orchestra.run.smithi153.stderr:: No such file or directory 2024-03-21T11:00:02.343 INFO:teuthology.orchestra.run.smithi153.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.log 2024-03-21T11:00:02.344 INFO:teuthology.orchestra.run.smithi153.stderr:gzip -5 --verbose -- /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/ceph-mds.cephfs.smithi153.zipckr.log 2024-03-21T11:00:02.346 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/ceph-client.0.log: gzip -5 --verbose -- /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/ceph-mon.smithi153.log 2024-03-21T11:00:02.347 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/ceph-mds.cephfs.smithi153.zipckr.log: gzip -5 --verbose -- /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/ceph-mgr.smithi153.tluxni.log 2024-03-21T11:00:02.348 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/ceph-mon.smithi153.log: gzip -5 --verbose -- /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/ceph-client.ceph-exporter.smithi153.log 2024-03-21T11:00:02.350 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/ceph-mgr.smithi153.tluxni.log: gzip -5 --verbose -- /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/ceph-mds.cephfs.smithi153.iuyluj.log 2024-03-21T11:00:02.352 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/ceph-client.ceph-exporter.smithi153.log: /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/ceph-mds.cephfs.smithi153.iuyluj.log: gzip -5 --verbose -- /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/ceph.audit.log 2024-03-21T11:00:02.357 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/ceph-osd.1.log 2024-03-21T11:00:02.358 INFO:teuthology.orchestra.run.smithi153.stderr: 94.5% -- replaced with /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/ceph-client.ceph-exporter.smithi153.log.gz 2024-03-21T11:00:02.359 INFO:teuthology.orchestra.run.smithi153.stderr:gzip -5 --verbose -- /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/ceph-osd.0.log 2024-03-21T11:00:02.362 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/ceph.cephadm.log 2024-03-21T11:00:02.368 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/ceph-osd.0.log: /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/ceph.log 2024-03-21T11:00:02.368 INFO:teuthology.orchestra.run.smithi153.stderr: 85.7% -- replaced with /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/ceph.cephadm.log.gz 2024-03-21T11:00:02.370 INFO:teuthology.orchestra.run.smithi153.stderr:gzip -5 --verbose -- /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/ceph-osd.2.log 2024-03-21T11:00:02.370 INFO:teuthology.orchestra.run.smithi174.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-03-21T11:00:02.370 INFO:teuthology.orchestra.run.smithi174.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-21T11:00:02.370 INFO:teuthology.orchestra.run.smithi174.stderr:gzip -5 --verbose -- /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/ceph-client.ceph-exporter.smithi174.log 2024-03-21T11:00:02.371 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/ceph.audit.log 2024-03-21T11:00:02.372 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/ceph-client.ceph-exporter.smithi174.log: gzip -5 --verbose -- /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/ceph-mgr.smithi174.qpcwat.log 2024-03-21T11:00:02.372 INFO:teuthology.orchestra.run.smithi174.stderr: 92.3% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-21T11:00:02.373 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/ceph-osd.4.log 2024-03-21T11:00:02.374 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/ceph-mgr.smithi174.qpcwat.log: gzip -5 --verbose -- /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/ceph-mds.cephfs.smithi174.yyimst.log 2024-03-21T11:00:02.375 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/ceph.log: /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/ceph-volume.log 2024-03-21T11:00:02.376 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/ceph-mon.smithi174.log 2024-03-21T11:00:02.377 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/ceph-mds.cephfs.smithi174.yyimst.log: gzip -5 --verbose -- /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/ceph-osd.5.log 2024-03-21T11:00:02.378 INFO:teuthology.orchestra.run.smithi174.stderr: 94.5% -- replaced with /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/ceph-client.ceph-exporter.smithi174.log.gz 2024-03-21T11:00:02.379 INFO:teuthology.orchestra.run.smithi153.stderr: 91.4% -- replaced with /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/ceph.audit.log.gz 2024-03-21T11:00:02.379 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/ceph-mon.smithi174.log: 91.5% -- replaced with /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/ceph.audit.log.gz 2024-03-21T11:00:02.379 INFO:teuthology.orchestra.run.smithi174.stderr:gzip -5 --verbose -- /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/ceph-osd.3.log 2024-03-21T11:00:02.380 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/ceph.cephadm.log 2024-03-21T11:00:02.382 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/ceph.log 2024-03-21T11:00:02.383 INFO:teuthology.orchestra.run.smithi153.stderr:/var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/ceph-volume.log: 90.7% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-21T11:00:02.384 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/ceph.cephadm.log: 85.7% -- replaced with /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/ceph.cephadm.log.gz 2024-03-21T11:00:02.384 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/ceph.log: gzip -5 --verbose -- /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/ceph-volume.log 2024-03-21T11:00:02.385 INFO:teuthology.orchestra.run.smithi153.stderr: 89.5% -- replaced with /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/ceph.log.gz 2024-03-21T11:00:02.387 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/ceph-mds.cephfs.smithi174.nhozag.log 2024-03-21T11:00:02.391 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/ceph-mds.cephfs.smithi174.nhozag.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.log 2024-03-21T11:00:02.391 INFO:teuthology.orchestra.run.smithi174.stderr: 89.6% -- replaced with /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/ceph.log.gz 2024-03-21T11:00:02.427 INFO:teuthology.orchestra.run.smithi153.stderr: 90.8% -- replaced with /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/ceph-volume.log.gz 2024-03-21T11:00:02.432 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/ceph-client.1.log: 90.8% -- replaced with /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/ceph-volume.log.gz 2024-03-21T11:00:02.442 INFO:teuthology.orchestra.run.smithi174.stderr: 89.8% -- replaced with /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/ceph-mgr.smithi174.qpcwat.log.gz 2024-03-21T11:00:03.061 INFO:teuthology.orchestra.run.smithi174.stderr: 92.5% -- replaced with /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/ceph-mon.smithi174.log.gz 2024-03-21T11:00:03.211 INFO:teuthology.orchestra.run.smithi153.stderr: 89.3% -- replaced with /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/ceph-mgr.smithi153.tluxni.log.gz 2024-03-21T11:00:05.114 INFO:teuthology.orchestra.run.smithi153.stderr: 89.9% -- replaced with /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/ceph-mon.smithi153.log.gz 2024-03-21T11:00:07.073 INFO:teuthology.orchestra.run.smithi174.stderr: 93.6% -- replaced with /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/ceph-osd.4.log.gz 2024-03-21T11:00:09.045 INFO:teuthology.orchestra.run.smithi153.stderr: 93.5% -- replaced with /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/ceph-osd.2.log.gz 2024-03-21T11:00:09.565 INFO:teuthology.orchestra.run.smithi174.stderr: 93.7% -- replaced with /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/ceph-osd.5.log.gz 2024-03-21T11:00:10.691 INFO:teuthology.orchestra.run.smithi153.stderr: 93.5% -- replaced with /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/ceph-osd.0.log.gz 2024-03-21T11:00:10.697 INFO:teuthology.orchestra.run.smithi153.stderr: 95.1% -- replaced with /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/ceph-mds.cephfs.smithi153.zipckr.log.gz 2024-03-21T11:00:11.122 INFO:teuthology.orchestra.run.smithi174.stderr: 93.6% -- replaced with /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/ceph-osd.3.log.gz 2024-03-21T11:00:11.197 INFO:teuthology.orchestra.run.smithi153.stderr: 93.6% -- replaced with /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/ceph-osd.1.log.gz 2024-03-21T11:00:11.284 INFO:teuthology.orchestra.run.smithi174.stderr: 95.1% -- replaced with /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/ceph-mds.cephfs.smithi174.nhozag.log.gz 2024-03-21T11:00:18.233 INFO:teuthology.orchestra.run.smithi153.stderr: 93.0% -- replaced with /var/log/ceph/ceph-client.0.log.gz 2024-03-21T11:00:19.577 INFO:teuthology.orchestra.run.smithi174.stderr:gzip: /var/log/ceph/ceph-client.1.log: file size changed while zipping 2024-03-21T11:00:19.578 INFO:teuthology.orchestra.run.smithi174.stderr: 93.0% -- replaced with /var/log/ceph/ceph-client.1.log.gz 2024-03-21T11:00:49.367 INFO:teuthology.orchestra.run.smithi174.stderr: 93.4% -- replaced with /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/ceph-mds.cephfs.smithi174.yyimst.log.gz 2024-03-21T11:00:49.371 INFO:teuthology.orchestra.run.smithi174.stderr: 2024-03-21T11:00:49.371 INFO:teuthology.orchestra.run.smithi174.stderr:real 0m47.043s 2024-03-21T11:00:49.371 INFO:teuthology.orchestra.run.smithi174.stderr:user 1m28.288s 2024-03-21T11:00:49.372 INFO:teuthology.orchestra.run.smithi174.stderr:sys 0m6.286s 2024-03-21T11:00:52.569 INFO:teuthology.orchestra.run.smithi153.stderr: 93.4% -- replaced with /var/log/ceph/070ed3e4-e769-11ee-95cd-87774f69a715/ceph-mds.cephfs.smithi153.iuyluj.log.gz 2024-03-21T11:00:52.572 INFO:teuthology.orchestra.run.smithi153.stderr: 2024-03-21T11:00:52.572 INFO:teuthology.orchestra.run.smithi153.stderr:real 0m50.263s 2024-03-21T11:00:52.572 INFO:teuthology.orchestra.run.smithi153.stderr:user 1m33.635s 2024-03-21T11:00:52.572 INFO:teuthology.orchestra.run.smithi153.stderr:sys 0m7.519s 2024-03-21T11:00:52.574 INFO:tasks.cephadm:Archiving logs... 2024-03-21T11:00:52.574 DEBUG:teuthology.misc:Transferring archived files from smithi153:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi/7612201/remote/smithi153/log 2024-03-21T11:00:52.575 DEBUG:teuthology.orchestra.run.smithi153:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-21T11:01:05.566 DEBUG:teuthology.misc:Transferring archived files from smithi174:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi/7612201/remote/smithi174/log 2024-03-21T11:01:05.567 DEBUG:teuthology.orchestra.run.smithi174:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-21T11:01:18.083 INFO:tasks.cephadm:Removing cluster... 2024-03-21T11:01:18.084 DEBUG:teuthology.orchestra.run.smithi153:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 070ed3e4-e769-11ee-95cd-87774f69a715 --force 2024-03-21T11:01:18.378 INFO:teuthology.orchestra.run.smithi153.stdout:Deleting cluster with fsid: 070ed3e4-e769-11ee-95cd-87774f69a715 2024-03-21T11:01:19.299 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 070ed3e4-e769-11ee-95cd-87774f69a715 --force 2024-03-21T11:01:19.586 INFO:teuthology.orchestra.run.smithi174.stdout:Deleting cluster with fsid: 070ed3e4-e769-11ee-95cd-87774f69a715 2024-03-21T11:01:20.464 INFO:tasks.cephadm:Removing cephadm ... 2024-03-21T11:01:20.464 DEBUG:teuthology.orchestra.run.smithi153:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-21T11:01:20.520 DEBUG:teuthology.orchestra.run.smithi174:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-21T11:01:20.572 INFO:tasks.cephadm:Teardown complete 2024-03-21T11:01:20.572 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-03-21T11:01:20.584 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/task/install/__init__.py", line 626, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_2edfee68ad7243bed422a049b6f5f58619300f11/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph-c_2edfee68ad7243bed422a049b6f5f58619300f11/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-03-21T11:01:20.584 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-03-21T11:01:20.585 DEBUG:teuthology.orchestra.run.smithi153:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-21T11:01:20.592 DEBUG:teuthology.orchestra.run.smithi174:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-21T11:01:20.728 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-03-21T11:01:20.740 INFO:teuthology.task.clock:Checking final clock skew... 2024-03-21T11:01:20.740 DEBUG:teuthology.orchestra.run.smithi153:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-21T11:01:20.748 DEBUG:teuthology.orchestra.run.smithi174:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-21T11:01:20.791 INFO:teuthology.orchestra.run.smithi174.stderr:bash: line 1: ntpq: command not found 2024-03-21T11:01:20.794 INFO:teuthology.orchestra.run.smithi153.stderr:bash: line 1: ntpq: command not found 2024-03-21T11:01:20.798 INFO:teuthology.orchestra.run.smithi174.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-21T11:01:20.798 INFO:teuthology.orchestra.run.smithi174.stdout:=============================================================================== 2024-03-21T11:01:20.798 INFO:teuthology.orchestra.run.smithi174.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 12 +690us[ +690us] +/- 91ms 2024-03-21T11:01:20.798 INFO:teuthology.orchestra.run.smithi174.stdout:^* hv02.front.sepia.ceph.com 3 7 377 79 -1009us[-1013us] +/- 65ms 2024-03-21T11:01:20.798 INFO:teuthology.orchestra.run.smithi174.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 10 +551us[ +551us] +/- 81ms 2024-03-21T11:01:20.799 INFO:teuthology.orchestra.run.smithi174.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T11:01:20.804 INFO:teuthology.orchestra.run.smithi153.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-21T11:01:20.804 INFO:teuthology.orchestra.run.smithi153.stdout:=============================================================================== 2024-03-21T11:01:20.804 INFO:teuthology.orchestra.run.smithi153.stdout:^+ hv01.front.sepia.ceph.com 2 7 377 13 +752us[ +752us] +/- 91ms 2024-03-21T11:01:20.804 INFO:teuthology.orchestra.run.smithi153.stdout:^* hv02.front.sepia.ceph.com 3 7 377 75 -992us[-1062us] +/- 65ms 2024-03-21T11:01:20.804 INFO:teuthology.orchestra.run.smithi153.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 13 +533us[ +533us] +/- 80ms 2024-03-21T11:01:20.804 INFO:teuthology.orchestra.run.smithi153.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T11:01:20.805 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-03-21T11:01:20.816 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-03-21T11:01:20.817 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-03-21T11:01:20.844 DEBUG:teuthology.orchestra.run.smithi153:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-21T11:01:21.088 DEBUG:teuthology.orchestra.run.smithi174:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-21T11:01:21.309 DEBUG:teuthology.orchestra.run.smithi153:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-21T11:01:21.415 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-21T11:01:21.415 DEBUG:teuthology.orchestra.run.smithi174:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-21T11:01:21.510 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-21T11:01:21.511 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-03-21T11:01:21.523 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-03-21T11:01:21.561 INFO:teuthology.task.internal:Duration was 4527.269278 seconds 2024-03-21T11:01:21.562 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-03-21T11:01:21.595 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-03-21T11:01:21.595 DEBUG:teuthology.orchestra.run.smithi153:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-21T11:01:21.603 DEBUG:teuthology.orchestra.run.smithi174:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-21T11:01:21.762 INFO:teuthology.orchestra.run.smithi174.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-21T11:01:21.763 INFO:teuthology.orchestra.run.smithi153.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-21T11:01:22.143 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-03-21T11:01:22.143 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi153.front.sepia.ceph.com 2024-03-21T11:01:22.143 DEBUG:teuthology.orchestra.run.smithi153:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-21T11:01:22.204 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi174.front.sepia.ceph.com 2024-03-21T11:01:22.205 DEBUG:teuthology.orchestra.run.smithi174:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-21T11:01:22.280 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-03-21T11:01:22.281 DEBUG:teuthology.orchestra.run.smithi153:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-21T11:01:22.285 DEBUG:teuthology.orchestra.run.smithi174:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-21T11:01:22.364 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-03-21T11:01:22.365 DEBUG:teuthology.orchestra.run.smithi153:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-21T11:01:22.369 DEBUG:teuthology.orchestra.run.smithi174:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-21T11:01:23.707 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-03-21T11:01:23.719 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-03-21T11:01:23.719 DEBUG:teuthology.orchestra.run.smithi153:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-21T11:01:23.807 DEBUG:teuthology.orchestra.run.smithi174:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-21T11:01:23.909 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-03-21T11:01:23.931 DEBUG:teuthology.orchestra.run.smithi153:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-21T11:01:23.939 DEBUG:teuthology.orchestra.run.smithi174:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-21T11:01:24.033 INFO:teuthology.orchestra.run.smithi153.stdout:kernel.core_pattern = core 2024-03-21T11:01:24.044 INFO:teuthology.orchestra.run.smithi174.stdout:kernel.core_pattern = core 2024-03-21T11:01:24.100 DEBUG:teuthology.orchestra.run.smithi153:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-21T11:01:24.153 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-21T11:01:24.153 DEBUG:teuthology.orchestra.run.smithi174:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-21T11:01:24.206 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-21T11:01:24.207 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-03-21T11:01:24.219 INFO:teuthology.task.internal:Transferring archived files... 2024-03-21T11:01:24.220 DEBUG:teuthology.misc:Transferring archived files from smithi153:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi/7612201/remote/smithi153 2024-03-21T11:01:24.220 DEBUG:teuthology.orchestra.run.smithi153:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-21T11:01:24.356 DEBUG:teuthology.misc:Transferring archived files from smithi174:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi/7612201/remote/smithi174 2024-03-21T11:01:24.357 DEBUG:teuthology.orchestra.run.smithi174:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-21T11:01:24.469 INFO:teuthology.task.internal:Removing archive directory... 2024-03-21T11:01:24.470 DEBUG:teuthology.orchestra.run.smithi153:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-21T11:01:24.477 DEBUG:teuthology.orchestra.run.smithi174:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-21T11:01:24.533 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-03-21T11:01:24.546 INFO:teuthology.task.internal:Not uploading archives. 2024-03-21T11:01:24.546 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-03-21T11:01:24.578 INFO:teuthology.task.internal:Tidying up after the test... 2024-03-21T11:01:24.579 DEBUG:teuthology.orchestra.run.smithi153:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-21T11:01:24.586 DEBUG:teuthology.orchestra.run.smithi174:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-21T11:01:24.636 INFO:teuthology.orchestra.run.smithi153.stdout: 395193 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Mar 21 11:01 /home/ubuntu/cephtest 2024-03-21T11:01:24.636 INFO:teuthology.orchestra.run.smithi153.stdout: 812381 4 d--------- 2 ubuntu ubuntu 4096 Mar 21 10:05 /home/ubuntu/cephtest/mnt.0 2024-03-21T11:01:24.636 INFO:teuthology.orchestra.run.smithi153.stderr:find: ‘/home/ubuntu/cephtest/mnt.0’: Permission denied 2024-03-21T11:01:24.639 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-21T11:01:24.639 INFO:teuthology.orchestra.run.smithi153.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty 2024-03-21T11:01:24.640 ERROR:teuthology.run_tasks:Manager failed: internal.base Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/task/internal/__init__.py", line 48, in base yield File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_2edfee68ad7243bed422a049b6f5f58619300f11/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph-c_2edfee68ad7243bed422a049b6f5f58619300f11/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/task/internal/__init__.py", line 53, in base run.wait( File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi153 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2024-03-21T11:01:24.640 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-03-21T11:01:24.653 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-03-21T11:01:24.680 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-03-21T11:01:24.680 INFO:teuthology.nuke:Checking targets against current locks 2024-03-21T11:01:24.706 DEBUG:teuthology.nuke:shortname: smithi153 2024-03-21T11:01:24.707 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-21T11:01:24.729 DEBUG:teuthology.nuke:shortname: smithi174 2024-03-21T11:01:24.729 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-21T11:01:24.732 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi153.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi/7612201', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-21 09:35:50.188678', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEK+pZRh8C5g97Z5EKqiwVZN1TgOhSP+anuzhpn57soIsbFFu2yJZ61lZNaSxEA7pqBweuQJEQNVej1INKNuN0M='} 2024-03-21T11:01:24.755 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi174.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi/7612201', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-21 09:35:50.190512', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBK9SuABKKZw4eM/oCdrw27HqA0dR2KfwELS4U22iliOPzYjSRZHyVzTcbup4BjtRM4QJfydoBH6WOH7s5E9o2xo='} 2024-03-21T11:01:24.782 INFO:teuthology.orchestra.console.smithi153:Power off 2024-03-21T11:01:24.782 DEBUG:teuthology.orchestra.console.smithi153:pexpect command: ipmitool -H smithi153.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-21T11:01:24.805 DEBUG:teuthology.orchestra.console.smithi153:power off output: Chassis Power Control: Down/Off 2024-03-21T11:01:24.806 DEBUG:teuthology.orchestra.console.smithi153:pexpect command: ipmitool -H smithi153.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-21T11:01:24.825 INFO:teuthology.orchestra.console.smithi174:Power off 2024-03-21T11:01:24.825 DEBUG:teuthology.orchestra.console.smithi174:pexpect command: ipmitool -H smithi174.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-21T11:01:24.835 DEBUG:teuthology.orchestra.console.smithi153:check power output: Chassis Power is on 2024-03-21T11:01:24.847 DEBUG:teuthology.orchestra.console.smithi174:power off output: Chassis Power Control: Down/Off 2024-03-21T11:01:24.847 DEBUG:teuthology.orchestra.console.smithi174:pexpect command: ipmitool -H smithi174.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-21T11:01:24.870 DEBUG:teuthology.orchestra.console.smithi174:check power output: Chassis Power is on 2024-03-21T11:01:28.836 DEBUG:teuthology.orchestra.console.smithi153:pexpect command: ipmitool -H smithi153.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-21T11:01:28.871 DEBUG:teuthology.orchestra.console.smithi174:pexpect command: ipmitool -H smithi174.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-21T11:01:28.947 DEBUG:teuthology.orchestra.console.smithi153:check power output: Chassis Power is on 2024-03-21T11:01:28.982 DEBUG:teuthology.orchestra.console.smithi174:check power output: Chassis Power is on 2024-03-21T11:01:32.952 DEBUG:teuthology.orchestra.console.smithi153:pexpect command: ipmitool -H smithi153.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-21T11:01:32.983 DEBUG:teuthology.orchestra.console.smithi174:pexpect command: ipmitool -H smithi174.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-21T11:01:33.064 DEBUG:teuthology.orchestra.console.smithi153:check power output: Chassis Power is on 2024-03-21T11:01:33.093 DEBUG:teuthology.orchestra.console.smithi174:check power output: Chassis Power is off 2024-03-21T11:01:33.194 INFO:teuthology.orchestra.console.smithi174:Power off completed 2024-03-21T11:01:37.073 DEBUG:teuthology.orchestra.console.smithi153:pexpect command: ipmitool -H smithi153.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-21T11:01:37.182 DEBUG:teuthology.orchestra.console.smithi153:check power output: Chassis Power is off 2024-03-21T11:01:37.283 INFO:teuthology.orchestra.console.smithi153:Power off completed 2024-03-21T11:01:37.388 INFO:teuthology.run:Summary data: description: fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/reef/{v18.2.1} 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} duration: 4527.269278287888 failure_reason: reached maximum tries (51) after waiting for 300 seconds flavor: default host.a-kernel-sha1: 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 host.b-kernel-sha1: 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 owner: scheduled_vshankar@teuthology status: fail success: false 2024-03-21T11:01:37.388 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-21T11:01:37.467 INFO:teuthology.run:FAIL